[
  {
    "path": ".gitattributes",
    "content": "*.pdf -crlf -diff\n"
  },
  {
    "path": ".gitignore",
    "content": "/Makefile\n/Emakefile\n/.Emakefile\n/config.status\n/ct_run.*ct*@*\n/ct_run.nonode*\n/all_runs.html\n/index.html\n/.externalToolBuilders\n/.Makefile.swp\n/.make-doc\n/variables-ct*@*\n/variables-*_ct*@*\n/variables-nonode@nohost\n/erl_crash.dump\n/autom4te.cache\n/aclocal.m4\n*.beam\n*~\n/ct_log_cache\n/result_increment_10_1000.txt\n/result_increment_1_10000.txt\n/result_read_10_10000.txt\n/result_read_1_100000.txt\n/Thumbs.db\n/python3-api\n/TAGS*\n/*.log\nebin/\n*.swp\n*.swo\n.DS_Store\nSession.vim\n.vimrc\n/src/user_default.erl\n/ct_default.css\n/jquery-latest.js\n/jquery.tablesorter.min.js\n/contrib/toke\n/contrib/erlang_js\ncontrib/slurm/slurm-*.out\n/java-api/target\n/cpp-api/scalaris\n/cpp-api/ssl-tests\n/.maven\n/priv/*.so\n/priv/*.so.dSYM/\n/contrib/log4erl/src/log4erl_lex.erl\n/contrib/log4erl/src/log4erl_parser.erl\n/cpp-api/tests.dSYM/\n/cpp-api/compile_commands.json\n/bin/jsonclient\n"
  },
  {
    "path": ".project",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<projectDescription>\n\t<name>Scalaris</name>\n\t<comment></comment>\n\t<projects>\n\t</projects>\n\t<buildSpec>\n\t\t<buildCommand>\n\t\t\t<name>org.python.pydev.PyDevBuilder</name>\n\t\t\t<arguments>\n\t\t\t</arguments>\n\t\t</buildCommand>\n\t\t<buildCommand>\n\t\t\t<name>net.sourceforge.texlipse.builder.TexlipseBuilder</name>\n\t\t\t<arguments>\n\t\t\t</arguments>\n\t\t</buildCommand>\n\t\t<buildCommand>\n\t\t\t<name>org.erlide.core.erlbuilder</name>\n\t\t\t<arguments>\n\t\t\t</arguments>\n\t\t</buildCommand>\n\t</buildSpec>\n\t<natures>\n\t\t<nature>net.sourceforge.texlipse.builder.TexlipseNature</nature>\n\t\t<nature>org.erlide.core.erlnature</nature>\n\t\t<nature>org.python.pydev.pythonNature</nature>\n\t</natures>\n\t<filteredResources>\n\t\t<filter>\n\t\t\t<id>1489147896928</id>\n\t\t\t<name></name>\n\t\t\t<type>10</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-false-false-java-api</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1489147896931</id>\n\t\t\t<name></name>\n\t\t\t<type>14</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-*ct*@*</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1489147896934</id>\n\t\t\t<name></name>\n\t\t\t<type>6</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-ct_*.log</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1489147896937</id>\n\t\t\t<name></name>\n\t\t\t<type>6</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-TAGS*</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1489147896939</id>\n\t\t\t<name></name>\n\t\t\t<type>6</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-all_runs.html</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1489147896942</id>\n\t\t\t<name></name>\n\t\t\t<type>6</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-ct_default.css</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1489147896944</id>\n\t\t\t<name></name>\n\t\t\t<type>6</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-jquery-latest.js</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1489147896946</id>\n\t\t\t<name></name>\n\t\t\t<type>6</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-jquery.tablesorter.min.js</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1489147896948</id>\n\t\t\t<name></name>\n\t\t\t<type>6</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-ct_log_cache</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1489147896950</id>\n\t\t\t<name></name>\n\t\t\t<type>6</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-config.log</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1489147896952</id>\n\t\t\t<name></name>\n\t\t\t<type>6</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-config.status</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1489147896954</id>\n\t\t\t<name></name>\n\t\t\t<type>6</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-index.html</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1318159459801</id>\n\t\t\t<name>contrib</name>\n\t\t\t<type>10</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-false-false-wikipedia</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1378048114063</id>\n\t\t\t<name>test</name>\n\t\t\t<type>6</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-*.beam</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t</filteredResources>\n</projectDescription>\n"
  },
  {
    "path": ".pydevproject",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<?eclipse-pydev version=\"1.0\"?>\n\n<pydev_project>\n<pydev_property name=\"org.python.pydev.PYTHON_PROJECT_INTERPRETER\">Default</pydev_property>\n<pydev_property name=\"org.python.pydev.PYTHON_PROJECT_VERSION\">python 2.7</pydev_property>\n</pydev_project>\n"
  },
  {
    "path": ".settings/org.eclipse.core.resources.prefs",
    "content": "#Wed Jul 28 11:29:12 CEST 2010\neclipse.preferences.version=1\nencoding//user-dev-guide/main.tex=ISO-8859-1\n"
  },
  {
    "path": ".settings/org.erlide.core.prefs",
    "content": "backend_version=R16\ncompiler/raw={d, enable_debug}, {d, with_crypto_hash}, {d, with_rand}, {d, have_callback_support}, {d, with_maps}\\n  , {d, 'HAVE_ERLANG_SENDFILE'}, {d, 'HAVE_CRYPTO_HASH'}\neclipse.preferences.version=1\nexternal_includes=\nexternal_modules=\ninclude_dirs=contrib/yaws/include;contrib/log4erl/include;include;\noutput_dir=ebin/eclipse\nsource_dirs=test;src;src/comm_layer;src/json;src/rbr;src/tx;src/transactions;src/rrepair;src/cp;src/paxos;\nuse_pathz=false\n"
  },
  {
    "path": ".settings/org.erlide.model.prefs",
    "content": "builderData=INTERNAL|compile|clean|test|\nconfigType=INTERNAL\neclipse.preferences.version=1\n"
  },
  {
    "path": ".travis.yml",
    "content": "language: generic\n\nmatrix:\n  include:\n    - os: osx\n      osx_image: xcode11.4\n      addons:\n        homebrew:\n          packages:\n            - boost\n            - erlang\n            - python\n            - ant\n            - openssl\n    - os: linux\n      sudo: false\n      addons:\n        apt:\n          sources:\n            - boost-latest\n          packages:\n            - automake\n            - clang\n            - libboost-all-dev\n            - python3\n            - erlang\n            - erlang-dev\n            - ant\n\nscript:\n  - env | grep TRAVIS\n  - clang++ -v\n  - 'if [[ \"$TRAVIS_OS_NAME\" == \"osx\" ]]; then CXX=clang++ ./configure --with-openssl=/usr/local/opt/openssl; fi'\n  - 'if [[ \"$TRAVIS_OS_NAME\" == \"linux\" ]]; then CXX=clang++ ./configure; fi'\n  - touch TAGS.root TAGS.contrib\n  - make\n  - make cpp\n  - make python\n  - 'if [[ \"$TRAVIS_OS_NAME\" == \"linux\" ]]; then make python3; fi'\n  - make java\n"
  },
  {
    "path": "AUTHORS",
    "content": "Ufuk Celebi\nJens Fischer\nChristian Hennig\nMikael Högqvist\nMarie Hoffmann\nStefan Keidel\nNico Kruber\nMaik Lange\nMaximilian Michels\nMonika Moser\nMagnus Müller\nStefan Plantikow\nTanguy Racinet\nFlorian Schintke\nThorsten Schütt\nJan Skrzypczak\nJeroen Vlek\n"
  },
  {
    "path": "ChangeLog",
    "content": "Scalaris 0.9.0 (codename \"Vriesea scalaris\") - February 29, 2016\n============================================================\n(partly supported by the EU project IES Cities http://iescities.eu/)\n\nPackaging:\n\n- add a libscalaris-dev[el] package with the C++ API (rpm, deb)\n- Maven: let Maven also create a source jar file for the Java-API\n- Maven: allow proper snapshot versions for the maven repository at\n         https://scalaris-team.github.io/scalaris/maven\n\nAPI:\n\n- CPP-API: provide a new C++ client library\n- DataNucleus: provide a new storage back-end for Scalaris\n- add a Scalaris adapter for YCSB\n- new functions in all APIs for generic routing table operations\n\nBusiness Logic:\n\n- rrepair: considerably improve the set reconciliation algorithms' performance\n           and run more processes in parallel\n- rrepair: further reduce the reconciliation costs\n- rrepair: re-use left-over failure probability from phase 1 for phase 2\n- rrepair: increase accuracy in several formulaes of the set reconciliation\n           algorithms\n- rrepair: support creating multiple Merkle trees for fewer message rounds\n           ('rr_merkle_num_trees' configuration variable)\n- rrepair: remove support for aligning bit sizes (no improvements there)\n- transactions: speed up write ops when the transaction log already contains\n                a read op on the same key\n- RM: optimise the traffic of state exchanges during ring maintenance\n- RT: improve routing parallelism by handling more messages in the routing\n      process itself instead of inside the dht_node process\n- RT: improve performance along the routing path\n- support for dynamic replication factors given by the 'replication_factor'\n  configuration variable\n- pid_groups: shuffle pids in find_a/1 for more better distributed (parallel)\n              requests\n\nInfrastructure:\n\n- pid_groups: group names are now atoms instead of strings\n              (allows faster access)\n- scalarisctl: do not set any default Erlang scheduler flags (allow changes\n               through the ERL_SCHED_FLAGS environment variable)\n- extend the example scripts for running Scalaris on a SLURM cluster\n- add SLURM scripts for running Basho Bench\n- use configure to select the routing table type (--with-rt=rt_chord)\n- change hanoidb detection to \"configure --enable-hanoidb=<dir>\"\n- change erlang_js detection to \"configure --enable-erlang-js=<dir>\"\n- dht_node_monitor: switch messages on/off via configuration variable\n                    'dht_node_monitor'\n- msg_delay: remove extraneous error messages for trigger messages of killed\n             processes\n- prbr: allow write_filter to return value to qwrite calling process\n- update yaws to version 2.0.2\n- support for Erlang R14B04 up to 18.2.3 and current otp master\n\nTests:\n\n- allow running single test groups via \"make <test suite>:<group name>_GROUP\"\n\nTools:\n\n- gen_component: allow processes to provide their own start functions\n- trace_mpath: write whole message of log_info to the tex output\n\nBugs:\n\n- fix cleanup of some special cases with transactions\n- fix several math issues and increase the math functions' performance\n- fix some errors which occurred in large scale deployments\n- fix several warnings reported by Coverity Scan\n- fix numerous more bugs\n\nScalaris 0.8.2 - July 24, 2015\n============================================================\n(partly supported by the EU project IES Cities http://iescities.eu/)\n\nBusiness Logic:\n- rrepair: minor performance improvements\n\nBugs:\n\n- rrepair: fix a rare crash due to hash collisions\n- rrepair: fix calculation of the next MaxItemCount in the merkle tree\n           repair protocol (was one level behind)\n- tests: fix rare crashes in some unit tests\n- fix a few more minor bugs\n\nScalaris 0.8.1 - July 12, 2015\n============================================================\n(partly supported by the EU project IES Cities http://iescities.eu/)\n\nBusiness Logic:\n\n- rrepair: allow specifying the minimum number of hash bits to use\n           to create non-approximate algorithms\n\nBugs:\n\n- rrepair: fix a badarith if rr_recon_version_bits is set to 'variable'\n- rrepair: fix a bug in the accuracy of the evaluation\n- fix two crashes in gossip\n\nScalaris 0.8.0 (codename \"Picoides scalaris\") - July 10, 2015\n============================================================\n(partly supported by the EU project IES Cities http://iescities.eu/\nand the EIT ICT Labs project MCData)\n\nPackaging:\n\n- also install docs on Arch Linux\n- fix lintian errors and warnings for Debian-based packages\n- fix permissions on the log directory (owned by the 'scalaris' user)\n  and the config files (not owned by the 'scalaris' user anymore!)\n- adapt packages for newer distributions\n\nAPI:\n\n- Java-API: integrate new OtpErlang library (1.5.12 from Erlang 17.4)\n- Python-API: do not include a shebang with a pre-defined path to python\n              anymore\n- Python-API: fix unclosed sockets in various tests\n- Python-API: increase python{2,3} test speed\n- Ruby-API: try to fallback to the normal json module if the gem is missing\n- Ruby-API: considerably increase the ruby test speed\n- remove the Publish/Subscribe API\n\nBusiness Logic:\n\n- rrepair: faster and more accurate bloom filter repair protocol\n- rrepair: faster, more accurate and less bandwidth-consuming merkle tree\n           repair protocol\n- rrepair: reduce memory footprint of merkle_tree\n- rrepair: more accurate and less bandwidth-consuming trivial repair\n           protocol\n- rrepair: add a new simple hash ('shash') algorithm\n- rrepair: switch to (the more efficient) push-only resolve phase for\n           synchronisation\n- rrepair: add replica repair algorithm evaluation scripts\n- RM: faster cyclon cache integration, especially during node jumps\n- comm_layer: envelops can be nested\n- comm_layer: allow send options for comm:send_local/3 and msg_delay\n              (currently only ?quiet supported)\n\nInfrastructure:\n\n- FD: simplify the API, support envelopes and remove own cookie handling\n- FD: extend the failure detector with generic notifications (not just\n      crashes) to subscribed nodes\n- DB: add an (experimental) mnesia back-end\n- DB: add an (experimental) hanoidb back-end\n- DB: make the back-end configurable via the scalaris.cfg option\n      'db_backend'\n- improve the speed of several utility functions\n- scalarisctl: add '-l' parameter for specifying the logdir\n- scalarisctl: do not use the '-s' parameter anymore, instead specify the\n               start type with the new '-t' parameter\n- scalarisctl: allow starting several dht_nodes per vm with given keys via\n               the new parameter '-j'\n- update yaws to version 1.99\n- make \"$HOME/.scalaris/log\" the default log dir\n- support for Erlang R14B04 up to 18.0.2 and current otp master\n- add example scripts for running Scalaris on a SLURM cluster\n- move sources to github: https://github.com/scalaris-team/scalaris\n- new project homepage: http://scalaris.zib.de\n\nTests:\n\n- reduce log spam in proto_sched tests and only print proto_sched results\n  when the tests fail\n- extended and improved various test suites\n\nTools:\n\n- proto_sched: support callbacks\n- proto_sched: add support for user msg shepherds\n- proto_sched: better error handling when sending messages to dead\n               (local) processes\n- tester: add value creator and type checker for orddict:orddict()\n- tester: full support for Erlang 18.0\n- let 'make dialyzer' run over test as well and ignore some false-positives\n  for Erlang >= 18.0 using the '-dialyzer()' attribute\n\nBugs:\n\n- slide: fix some race conditions with 'jump' operations\n- JSON-API: fix test_and_set not working correctly inside req_list\n- fix numerous more bugs\n\nScalaris 0.7.2 - October 23, 2014\n============================================================\n(partly supported by the EU project IES Cities http://iescities.eu/\nand the EIT ICT Labs project MCData)\n\nPackaging:\n\n- fix ArchLinux packages with newest Java versions\n\nDemonstrator \"Wiki on Scalaris\":\n\n- fix the separate count key optimisation not using Scalaris' increment\n  operation\n\nBusiness Logic:\n\n- rrepair: let the trivial algorithm assume the worst case in order to always\n           meet the configured \"recon probability of one error\" (p1e)\n- rrepair: fix the trivial algorithm having an effectively doubled p1e\n- rrepair: fix the bloom algorithm having an effectively tripled p1e\n- rrepair: allow disabling byte-alignment\n\nBugs:\n\n- fix a few minor bugs\n\nScalaris 0.7.1 - September 30, 2014\n============================================================\n(partly supported by the EU project IES Cities http://iescities.eu/\nand the EIT ICT Labs project MCData)\n\nPackaging:\n\n- add support for new distribution versions\n- support systemd and SELinux\n- include daemon for monitoring Scalaris through JMX\n\nAPI:\n\n- Java-API: integrate new OtpErlang library (1.5.10 from Erlang 17.3)\n\nDemonstrator \"Wiki on Scalaris\":\n\n- fix storing template back-links for magic words\n- fix ARTICLE_COUNT partitioning with hashes\n\nBusiness Logic:\n\n- more robust (still experimental) support for active load balancing with\n  Karger and Ruhl's algorithm including more flexible \"load\" definitions\n- rm_tman: less overhead by only sending unknown nodes to neighbours\n- slide: better support for slide aborts\n- node join: less overhead during joins, especially in setups with huge\n             lists of known nodes\n- cyclon: move to new gossip framework\n- vivaldi: move to new gossip framework\n- monitor: move performance monitor to the basic_services group\n           (once per VM)\n\nInfrastructure:\n\n- several smaller performance optimisations\n- support for Erlang R13B01 up to 17.3 and current otp master\n\nTests:\n\n- some new unit tests and higher test coverage\n\nDocumentation:\n\n- add rrepair sequence diagrams\n\nBugs:\n\n- fix numerous bugs\n\nScalaris 0.7.0 (codename \"Stauroderus scalaris\") - April 28, 2014\n============================================================\n\nAPI:\n\n- Java-API: integrate new OtpErlang library (1.5.9 from Erlang 17.0)\n\nDemonstrator \"Wiki on Scalaris\" (supported by 4CaaSt http://www.4caast.eu/\n                                 and Contrail http://contrail-project.eu):\n\n- add support for reading 7z dumps\n\nBusiness Logic (partly supported by the EU project IES Cities http://iescities.eu/\n                and the EIT ICT Labs project MCData):\n\n- add an experimental Map-Reduce framework on top of Scalaris\n- add experimental support for active load balancing with Karger and\n  Ruhl's algorithm including more flexible \"load\" definitions\n- rrepair: completely new merkle tree sync protocol with an order of\n           magnitude lower traffic costs (use dynamic signature sizes based\n           on a \"recon probability of one error\" (p1e) in the leaf nodes)\n- rrepair: add a trivial probabilistic reconciliation protocol using the\n           \"recon probability of one error\" (p1e)\n- rrepair: replace bloom_fpr parameter with a generic \"recon probability of\n           one error\" (p1e)\n- rrepair: add a trivial reconciliation phase for the differences identified\n           by bloom\n- rrepair: more efficient resolve also including a list of keys to request\n- gossip: completely new (more flexible) gossip framework\n- slide: add support for 'jump' operations\n- slide: do not create a timer for each received message (use a periodic\n         cleanup)\n- comm_layer: tune TCP connection parameters to increase throughput and\n              latency (no delay_send)\n- comm_layer: close idle TCP connections, support for no_keep_alive\n              connections (used by cyclon, for example)\n- comm_layer: try to bundle more messages\n- node join: less overhead during joins, especially in small rings\n- rm_tman: less overhead by not adding nodes in the cache to the dn_cache\n- rt_chord: skip trying to contact nodes in the own range\n- rt_chord: stop stabilize when the own node is reached\n\nInfrastructure:\n\n- intervals: more compact (and transfer-friendly) representation\n- trigger: replace modules with msg_delay:send_trigger/2\n- config: replace implementation with a public ets table\n- tune the garbage collection of some core processes and periodically\n  garbage collect all processes\n- reduce the number of generated atoms and the number of ets tables used\n- several smaller performance optimisations\n- support for Erlang R13B01 up to 17.0 and current otp master\n\nTests:\n\n- improve test coverage (manual tests and random tests)\n- change test definitions and allow groups of tests, e.g.\n  test (default), test-skipped,\n  all_TESTS, all_with_cover_TESTS, performance_TESTS, proto_sched_TESTS,\n  type_check_TESTS\n\nDocumentation:\n\n- add rrepair documentation\n\nTools:\n\n- proto_sched: new API, better determinism, more strict self-control\n- proto_sched: support for short-lived processes\n- proto_sched: more information in get_infos\n- trace_mpath: more flexible LaTeX exports with more detailed message info\n- trace_mpath: support meaningful PID names when tracing remote messages\n- tester: full support for Erlang 17.0\n- make: re-compile after Emakefile changes\n\nBugs:\n\n- rrepair: fix unnecessary feedback for KVV items of the same version\n- bulkowner: some fixes for (gracefully) leaving nodes\n- fix numerous more (less severe) bugs\n\nScalaris 0.6.1 - October 11, 2013\n============================================================\n\nPackaging:\n\n- add ArchLinux ruby API packages\n\nAPI:\n\n- Java-API: add Maven build support\n- Java-API: add CircularByteArrayOutputStream#clear()\n- Java-API: fix ConnectionPool#getConnection(timeout) throwing\n            IllegalMonitorStateException if a single ConnectionPool is used\n            by multiple threads and no more connections are available\n- Java-API: add a \"-monitor\" command line parameter\n- api_monitor: return the latency and stddev values of the micro-benchmark\n               executed by monitor_perf for node and service performance\n\nDemonstrator \"Wiki on Scalaris\" (supported by 4CaaSt http://www.4caast.eu/\n                                 and Contrail http://contrail-project.eu):\n\n- separate list counters from their list partitions for a better data layout\n- fix high memory use of the Scalaris import if the import is slow\n- use tomcat 7.0.42\n\nBusiness Logic:\n\n- rrepair: reduce overhead of ART reconciliation\n- rrepair: allow resolving of multiple merkle node leaves with a single\n           resolve request\n- rrepair: don't create resolve requests for empty intervals\n- rrepair: reduce overhead of update_key_entry requests (use a single request\n           with all the data instead of one request for each item)\n- rrepair: allow arbitrary intervals in interval_upd and interval_upd_send\n           resolve requests again\n- rrepair: when hashing merkle_tree/ART children, also include the represented\n           interval (fixes indistinguishable empty leaf nodes in ART)\n- tx_tm_rtm: re-enable takeover by rtms on tm crash\n- rt_chord: only re-build the RT if the pred or succ processes change or the\n            own new node ID is not between the new pred and succ any more\n\nInfrastructure:\n\n- DB: improve performance of fold[lr] implementations by a factor of 2\n- FD: get rid of annoying, wrongly raised log warnings\n- RM: provide a more generic Reason for RM subscriptions\n- RM: remove trigger infections\n- RT: don't trigger an update when a slide finishes\n- RT, FRT: reduce the number of messages sent on lookups\n- log: don't exit with a badmatch if our error_logger is not the only one\n\nTests:\n\n- add protocol scheduler tests for slide, join and leave\n\nDocumentation:\n\n- user-dev-guide: add section on scalarisctl checkinstallation\n- user-dev-guide: clarify the section about how to set up Scalaris\n\nTools:\n\n- proto scheduler: continue in case of send errors\n- top: improve process messages output\n\nBugs:\n\n- node join: fix not being able to join a system with passive load balancer\n             if the number of items in the DB is too high\n- tx_tm_rtm: fix wrong asserts\n- fix node resposibility check not always including message forward and\n  db_range intervals\n- tx_tp: add missing snapshot number in tp_do_commit_abort message\n- bulkowner: respect the nodes' DB ranges and forward requests for\n             non-responsible ranges\n- dn_cache: fix reporting wrong PIDs back as zombies after node reboots\n- fix some more (less severe) bugs\n\nScalaris 0.6.0 (codename \"Conus scalaris\") - August 19, 2013\n============================================================\n\nPackaging:\n\n- add ArchLinux packages\n- add support for new distribution versions\n\nAPI:\n\n- no more timeouts in client APIs\n- Java-API: re-worked the request and result list handling\n  -> move result processing to the operation classes\n- Java-API: better support for custom operations\n- Java-API: support the new partial reads:\n            ReadRandomFromListOp and ReadSublistOp\n- Java-API: compile with \"vars\" debug info\n- Java-API: integrate new OtpErlang library (1.5.8 from R16B) with fixed\n            support for compressed binaries\n- Java-API: add back-ports from the Wiki on Scalaris demonstrator:\n  * list-change operations: ScalarisChangeListOp and ScalarisListAppendRemoveOp\n  * MultiMap classes are now in de.zib.tools\n  * CircularByteArrayOutputStream\n- Java-API: fix hostname issues with Erlang and Java\n- Java-API: slightly changed the delete API\n- JSON-API: add API for auto-scale requests\n- Python-API: add API for auto-scale requests\n- Python-API: use default socket timeout\n- Ruby-API: use default socket timeout\n- all APIs: support lists of composite types\n\nDemonstrator \"Wiki on Scalaris\" (supported by 4CaaSt http://www.4caast.eu/\n                                 and Contrail http://contrail-project.eu):\n\n- allow monitoring via JMX in the FourCaastMonitoringPlugin\n- support for getting random articles via the new partial read op\n- new optimisation scheme \"Buckets with Write Cache\" - uses a single big list\n  to read from and the rest of the buckets to write to\n- improve import and dump-processing (faster, more memory-efficient)\n- add on-the-fly conversion to the different optimisation schemes during import\n  (only one prepared DB dump needed now)\n- several UI enhancements and rendering fixes\n- update bliki lib (includes code ported to upstream)\n- add auto-import ability\n- use tomcat 7.0.33\n\nBusiness Logic:\n\n- replace common message tags with integers to reduce bandwidth\n- more flexible read operations (easier to extend)\n- add support for the following partial reads: random_from_list and sublist\n- save bandwidth by not returning the full value for write operations\n  (only the version is required)\n- new DB back-end implementation with a smaller and cleaner interface\n- faster DB get_chunk processing\n- tx: allow overwriting old/outdated DB entries\n- tx: allow overwriting old/outdated write-locked entries\n- tx: allow setting write lock on old/outdated read-locked entries\n- tx: always reply when the majority replied during read\n- tx: make sure that if not_found is reported to the user (while reading),\n      a write cannot go through if it is not also based on not_found\n- tx: committing a test_and_set op on a non-existing entry now fails as well\n      (the op itself already returned the failure)\n- tx: add a 2s delay to wait for slow learner_decide answers before cleaning up\n      (results in a faster state cleanup after the fourth response)\n- tx: small performance improvements in several modules\n- rm: only add alive, non-leaving nodes\n- rm: if a predecessor crashes, start repairing the range (rrepair)\n- rrepair: stabilised rrepair (not considered experimental any more)\n- rrepair: also update entries with existing but outdated WriteLocks\n- rrepair: several performance improvements\n           (bloom, merkle_tree, art and rrepair processes in general)\n- rrepair: re-design of rr_recon\n- rrepair: don't offload heavy work onto the dht_node (increases responsiveness\n           of the dht_node process during replica repair)\n- rrepair: improve db_generator tool and random_bias binomial distribution used\n           for tests\n- rrepair: support differently configured nodes (use the same reconciliation\n           structure parameters)\n- rrepair: de-activate self-repair (a node with multiple copies of the same\n           items does not need a reconciliation structure to repair some of\n           them)\n- rrepair: activate rrepair periodically every 10 minutes with a probability\n           of 33%\n- slide v2.0: fewer message to initiate a slide\n- slide v2.0: generic (asynchronous) call-backs for different ring maintenance\n              algorithms\n- slide v2.0: re-work handling of planned next operations, e.g. used by\n              incremental slides\n- slide v2.0: don't directly work on the DB any more (there may be more data\n              needed to slide) - let dht_node_state decide\n- slide v2.0: activate incremental join and leave operations\n- slide v2.0: actively report graceful node shutdown to the local FD of the\n              leaving node to inform subscribers\n- slide v2.0: code clean-up\n- slide v2.0: some fixes for incremental slides\n- slide v2.0: more robust in general\n- more smooth node joins by also reporting when a join is not possible due to\n  a running slide at the existing node\n- passive load balancing: random selection of (equally qualified) nodes\n- add new routing algorithms FRT-Chord (flexible routing tables) and\n  GFRT-Chord (supports proximity routing and data centers) as alternatives to\n  Chord (see rt_frtchord and rt_gfrtchord modules)\n- add auto-scale framework, e.g. for cloud environments (supported by\n  Contrail http://contrail-project.eu/) which is able to scale the deployment\n  to maintain a given target latency of executed transactions\n- cache config reads in the process dictionary for better performance\n- cyclon: if the cache is empty, try one of the nodes in known_hosts\n- add support for consistent snapshots (experimental)\n\nInfrastructure:\n\n- add a daemon to monitor Scalaris via JMX\n- disable message compression (only client values are compressed - the rest is\n  too expensive, at least on GbE)\n- support for distributions with python3 available as \"python\" and\n  python2 as \"python2\"\n- support for Ruby 1.9\n- yaws 1.96 (with patch to compile on otp master and a patch to fix a\n  performance regression)\n- support for Erlang R13B01 up to R16B01 and current otp master\n\nTests:\n\n- add test suite to find memory leaks\n- let \"make test\" run the major test suites and \"make test-skipped\" for some\n  more (time-consuming) tests\n- clean-up ring after timetrap timeout failures via common test hook\n- new ?compare macro for custom comparison functions\n- higher test coverage with more random-testing via the \"tester\"\n\nDocumentation:\n\n- user-dev-guide: add user tutorial on using scalaris\n- user-dev-guide: add a section about the slide protocol\n- user-dev-guide: extended description of scientific background\n- add replica repair sequence diagrams\n- better code descriptions\n\nTools:\n\n- gen_component: synchronous breakpoint set and delete for more deterministic\n                 usage\n- trace_mpath: allow selective tracing via filter fun\n- trace_mpath: fix several triggers becoming infected by trace_mpath resulting\n               in infinite tracing\n- trace_mpath: improve latex output of traces\n- tester: copy dictionary to worker threads\n- tester: add support for more types, e.g. neg_integer(), gb_rees\n- tester: better type check error reporting\n- tester: print tester last calls when aborting unit tests\n          (timeout or exception)\n- tester: add support for constraints in type specs (\"when is_subtype(A,B)\")\n- web debug interface: add cluster graph visualisation\n- web debug interface: display vivaldi distance\n- web debug interface: add IP addresses and ports to the ring charts and tables\n- web debug interface: allow navigating to the web interfaces of shown nodes\n- top: support for showing messages in message queue of an inspected PID\n- top: support for showing larger dictionary values\n- allow recursive reply_as envelopes\n- experimental protocol scheduler to check protocols with random message\n  interleavings (see proto_sched module)\n\nBugs:\n\n- fix RM handling of (out-dated) nodes with the same ID as newly added nodes\n- fix ganglia integration not working any more\n- restore the ability to start nodes at a specific key via \"scalarisctl -k <key> ...\"\n- fix some memory leaks in the tx system\n- fix statistics of comm_connection (not send in some cases, not overflow-aware)\n- use /bin/bash instead of /bin/sh which may not result in a bash session\n- fix init.d scripts not checking for existing processes correctly\n- fix dc_clustering\n- fix numerous other bugs\n\nScalaris 0.5.0 (codename \"Saperda scalaris\") - October 11, 2012\n===============================================================\n\nPackaging:\n\n- new init.d script to start Scalaris\n- added chef scripts to deploy Scalaris nodes\n- improved Windows start scripts (support for R15B01 and R15B02, don't close\n  command prompt window immediately after shutdown if double-clicked)\n- more flexible scalarisctl (arbitrary parameter order, allow setting cookie,\n  ports and number of nodes in VM via parameters, allow using screen for\n  daemonised sessions, allow graceful leave via \"gstop\" command, new \"status\"\n  command)\n- support for new linux distributions (Fedora 17, Ubuntu 12.04, openSUSE 12.2)\n- let scalarisctl checkinstallation also perform runtime tests for the APIs\n\nAPI:\n\n- allow Scalaris monitoring via JMX through the Java API\n- added an executor-service to the Java-API (de.zib.scalaris.executor.*)\n- added a node discovery daemon to the Java-API\n- allow compressed communication between the Java-API and Erlang for increased\n  performance, especially if the two are on separate nodes\n- added VM management support to the JSON- and Python-API\n- added transaction log filtering to the Java-API, i.e. only sent the needed\n  parts of the tlog back to Erlang and re-combine the result\n- fixed api_tx:req_list_commit_each/1 not running requests in parallel\n  -> do not assure any order of requests, even if on same key!\n\nDemonstrator \"Wiki on Scalaris\" (supported by 4CaaSt http://www.4caast.eu/\n\t                         and Contrail http://contrail-project.eu):\n\n- allow different partitioned data models for better performance and\n  scalability\n- allow logging of user requests\n- added support for checking whether another article exists (approximate)\n  -> show link colours based on this check\n- added check for bad page titles\n- allow SERVERNAME and SERVERPATH in config for setups with load balancers\n- reduced memory footprint of Wiki data in Scalaris\n- support for newer wiki xml dumps\n- added support for using a MediaWiki-like SQLite-DB backend for e.g.\n  filtering\n- improved overall performance\n- several rendering fixes\n\nBusiness Logic:\n\n- added (experimental) support for replica repair (disabled by default)\n  (thanks to Maik Lange)\n- added monitoring of memory statistics (also available via web interface)\n- better error reporting in the failure detector\n- reduced message overhead by UIDs and message/tuple tags\n- reduced overall message size of transactions:\n  * do not include the (uncompressed) value in messages of the read phase of\n    write operations\n  * do not include the value in init_TP messages\n- allow VM-flag \"first\" to be set via config file\n- gather overall connection statistics in comm_stats (also available via web\n  interface)\n- reduced erroneous failure messages on node shutdown\n- integrated comm_layer into comm_server\n- better scalability in pid_groups (find processes round-robin in find_a/1)\n- several changes to improve overall performance and/or CPU time at the nodes\n\nTests:\n\n- support for more types in the runtime type-checker\n- verify several API functions via runtime type-checker (also test private\n  functions if possible!)\n\nTools:\n\n- distributed protocol visualisation via trace_mpath (text-based or latex-file\n  for graphical presentation)\n- better profiling via top for Erlang processes\n- better debugging, e.g. via ASCII supervisor-tree rendering in verbose mode\n\nBugs:\n\n- fixed memory leaks in read and write operations\n- fixed memory leaks in tx_tm_rtm\n- prevent potential endless loops in tx_tm_rtm\n- fixed inform RTMs sometimes informing the wrong RTMs\n- fixed numerous other bugs\n\nScalaris 0.4.1 - March 22, 2012\n===============================\n\nPackaging:\n\n- new official ConPaaS packages (http://www.conpaas.eu/)\n- install rubygem dependencies in Debian postinstall scripts for Ruby API\n- improved Windows start scripts (if set, uses the ERLANG_HOME environment\n  variable to find Erlang, otherwise searches for Erlang in common paths)\n\nBugs:\n\n- better tx cleanup (should fix rare occurance of duplicate client inform)\n- forward additional parameters of the start scripts to new syntax of scalarisctl\n\nScalaris 0.4.0 (codename \"Pomacea scalaris\") - January 25, 2012\n===============================================================\n\nAPI:\n\n- new functions for incremental data change:\n  test_and_set: check for a provided old value before setting a new one\n  add_on_nr: increment a numeric value\n  add_del_on_list: append or delete entries from a list value\n- added VM API to manage Scalaris nodes inside an Erlang virtual machine\n- added monitoring API to retrieve some live metrics\n- added a connection pool convenience class (Java, Python)\n\nDemonstrator \"Wiki on Scalaris\" (supported by 4CaaSt http://www.4caast.eu/):\n\n- improved performance of page edits\n- improved performance of Wikipedia dump loading\n- several rendering fixes\n\nBusiness Logic:\n\n- improved handling of large values by reducing overhead of transaction\n  log handling (empty TLog after commit), no copy of value in TLog\n  returned to user after read requests)\n- eliminated timeouts in data hand-over protocol (relies on fd now)\n- added a DB subscribe mechanism, e.g. to become informed when locks\n  are freed\n- fixed a strong consistency issue in the tx protocol\n- gather some run-time statistics and expose them via the APIs and the\n  web debug interface\n\nInfrastructure:\n\n- support for Erlang 15B\n- fd now also uses feedback from TCP layer\n- made message sending more flexible (gets an option list)\n- added and corrected several Erlang type specifications\n- added scripts to create Scalaris images for OpenNebula\n- added tools for using Scalaris as the Database as a Service\n  component in ConPaaS (http://www.conpaas.eu/) which is part of the\n  EU project Contrail (http://contrail-project.eu/)\n- added a separate communication channel for priority messages, e.g. fd\n  (reduces falsely reported node crashes under heavy load)\n\nTests:\n\n- added runtime type-checker for random testing extended unittests\n\nDocumentation:\n\n- updated documentation to extended APIs\n\nBugs:\n\n- fixed numerous bugs\n\n\nScalaris 0.3.0 (codename \"Rhinechis Scalaris\") - July 15, 2011\n==============================================================\n\nAPI:\n\n- new API with interoperable bindings to Java, Python, Ruby, and JSON\n- support for several data types, including strings, integers, JSON\n  objects, binary objects.\n- new transaction interface with support for bundled requests for\n  better latency.\n- separate APIs to access the raw DHT, a DHT with replication, and the\n  transactional DHT\n\nDemonstrator:\n\n- added Wikipedia-hosting using Scalaris as demonstrator application\n\nBusiness Logic:\n\n- fault-tolerant startup: start Scalaris when a quorum of the\n  known_hosts becomes available (option -q in bin/scalarisctl)\n- perform data hand-over when nodes join/gracefully leave\n  (also works when transactions are executed concurrently)\n- added passive load balancing (when a node joins a ring, it samples\n  several other nodes and joins at the node that balances the number\n  of stored items the most)\n- completely rewritten transaction layer (more modular, more\n  extendible, less latency)\n- modularized / reimplemented Paxos algorithm, so the algorithm can\n  also be used outside transactions (e.g. used for quorum-startup)\n- switched almost all components to our component framework 'gen_component'\n- added gossiping for estimating e.g. the number of nodes or the\n  average load in a ring\n- more reliable unreliable look-up\n- better ring start-up on slow networks\n\nInfrastructure:\n\n- Vivaldi and topology inference\n- support for Erlang 13B01 and newer\n- faster TCP/IP communication between Scalaris nodes\n- completely rewritten failure detector framework for more accurate\n  detection of node failures\n- added numerous Erlang type specifications\n- extended unittests\n\nTests:\n\n- added own random testing framework that reads type specifications\n  and scans the source code for constants to generate proper random\n  test-data\n- extended gen_component with breakpoint-support for debugging and\n  testing (perform deterministic pseudo-random message interleaving\n  tests)\n- added numerous unittests\n- added language-binding interoperability tests\n\nDocumentation:\n\n- extended, but - as always - by far not enough...\n\nBugs:\n\n- fixed countless bugs\n\nScalaris 0.2.0 - 0.2.3\n======================\n\n- changes not explicitly recorded (see the corresponding\n  commit messages)\n\n"
  },
  {
    "path": "Doxyfile",
    "content": "# Doxyfile 1.8.9.1\n\n# This file describes the settings to be used by the documentation system\n# doxygen (www.doxygen.org) for a project.\n#\n# All text after a double hash (##) is considered a comment and is placed in\n# front of the TAG it is preceding.\n#\n# All text after a single hash (#) is considered a comment and will be ignored.\n# The format is:\n# TAG = value [value, ...]\n# For lists, items can also be appended using:\n# TAG += value [value, ...]\n# Values that contain spaces should be placed between quotes (\\\" \\\").\n\n#---------------------------------------------------------------------------\n# Project related configuration options\n#---------------------------------------------------------------------------\n\n# This tag specifies the encoding used for all characters in the config file\n# that follow. The default is UTF-8 which is also the encoding used for all text\n# before the first occurrence of this tag. Doxygen uses libiconv (or the iconv\n# built into libc) for the transcoding. See http://www.gnu.org/software/libiconv\n# for the list of possible encodings.\n# The default value is: UTF-8.\n\nDOXYFILE_ENCODING      = UTF-8\n\n# The PROJECT_NAME tag is a single word (or a sequence of words surrounded by\n# double-quotes, unless you are using Doxywizard) that should identify the\n# project for which the documentation is generated. This name is used in the\n# title of most generated pages and in a few other places.\n# The default value is: My Project.\n\nPROJECT_NAME           = \"Scalaris Project\"\n\n# The PROJECT_NUMBER tag can be used to enter a project or revision number. This\n# could be handy for archiving the generated documentation or if some version\n# control system is used.\n\nPROJECT_NUMBER         =\n\n# Using the PROJECT_BRIEF tag one can provide an optional one line description\n# for a project that appears at the top of each page and should give viewer a\n# quick idea about the purpose of the project. Keep the description short.\n\nPROJECT_BRIEF          = \"Scalaris, a distributed, transactional key-value store\"\n\n# With the PROJECT_LOGO tag one can specify a logo or an icon that is included\n# in the documentation. The maximum height of the logo should not exceed 55\n# pixels and the maximum width should not exceed 200 pixels. Doxygen will copy\n# the logo to the output directory.\n\nPROJECT_LOGO           =\n\n# The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute) path\n# into which the generated documentation will be written. If a relative path is\n# entered, it will be relative to the location where doxygen was started. If\n# left blank the current directory will be used.\n\nOUTPUT_DIRECTORY       = docs\n\n# If the CREATE_SUBDIRS tag is set to YES then doxygen will create 4096 sub-\n# directories (in 2 levels) under the output directory of each output format and\n# will distribute the generated files over these directories. Enabling this\n# option can be useful when feeding doxygen a huge amount of source files, where\n# putting all generated files in the same directory would otherwise causes\n# performance problems for the file system.\n# The default value is: NO.\n\nCREATE_SUBDIRS         = NO\n\n# If the ALLOW_UNICODE_NAMES tag is set to YES, doxygen will allow non-ASCII\n# characters to appear in the names of generated files. If set to NO, non-ASCII\n# characters will be escaped, for example _xE3_x81_x84 will be used for Unicode\n# U+3044.\n# The default value is: NO.\n\nALLOW_UNICODE_NAMES    = NO\n\n# The OUTPUT_LANGUAGE tag is used to specify the language in which all\n# documentation generated by doxygen is written. Doxygen will use this\n# information to generate all constant output in the proper language.\n# Possible values are: Afrikaans, Arabic, Armenian, Brazilian, Catalan, Chinese,\n# Chinese-Traditional, Croatian, Czech, Danish, Dutch, English (United States),\n# Esperanto, Farsi (Persian), Finnish, French, German, Greek, Hungarian,\n# Indonesian, Italian, Japanese, Japanese-en (Japanese with English messages),\n# Korean, Korean-en (Korean with English messages), Latvian, Lithuanian,\n# Macedonian, Norwegian, Persian (Farsi), Polish, Portuguese, Romanian, Russian,\n# Serbian, Serbian-Cyrillic, Slovak, Slovene, Spanish, Swedish, Turkish,\n# Ukrainian and Vietnamese.\n# The default value is: English.\n\nOUTPUT_LANGUAGE        = English\n\n# If the BRIEF_MEMBER_DESC tag is set to YES, doxygen will include brief member\n# descriptions after the members that are listed in the file and class\n# documentation (similar to Javadoc). Set to NO to disable this.\n# The default value is: YES.\n\nBRIEF_MEMBER_DESC      = YES\n\n# If the REPEAT_BRIEF tag is set to YES, doxygen will prepend the brief\n# description of a member or function before the detailed description\n#\n# Note: If both HIDE_UNDOC_MEMBERS and BRIEF_MEMBER_DESC are set to NO, the\n# brief descriptions will be completely suppressed.\n# The default value is: YES.\n\nREPEAT_BRIEF           = YES\n\n# This tag implements a quasi-intelligent brief description abbreviator that is\n# used to form the text in various listings. Each string in this list, if found\n# as the leading text of the brief description, will be stripped from the text\n# and the result, after processing the whole list, is used as the annotated\n# text. Otherwise, the brief description is used as-is. If left blank, the\n# following values are used ($name is automatically replaced with the name of\n# the entity):The $name class, The $name widget, The $name file, is, provides,\n# specifies, contains, represents, a, an and the.\n\nABBREVIATE_BRIEF       =\n\n# If the ALWAYS_DETAILED_SEC and REPEAT_BRIEF tags are both set to YES then\n# doxygen will generate a detailed section even if there is only a brief\n# description.\n# The default value is: NO.\n\nALWAYS_DETAILED_SEC    = NO\n\n# If the INLINE_INHERITED_MEMB tag is set to YES, doxygen will show all\n# inherited members of a class in the documentation of that class as if those\n# members were ordinary class members. Constructors, destructors and assignment\n# operators of the base classes will not be shown.\n# The default value is: NO.\n\nINLINE_INHERITED_MEMB  = NO\n\n# If the FULL_PATH_NAMES tag is set to YES, doxygen will prepend the full path\n# before files name in the file list and in the header files. If set to NO the\n# shortest path that makes the file name unique will be used\n# The default value is: YES.\n\nFULL_PATH_NAMES        = YES\n\n# The STRIP_FROM_PATH tag can be used to strip a user-defined part of the path.\n# Stripping is only done if one of the specified strings matches the left-hand\n# part of the path. The tag can be used to show relative paths in the file list.\n# If left blank the directory from which doxygen is run is used as the path to\n# strip.\n#\n# Note that you can specify absolute paths here, but also relative paths, which\n# will be relative from the directory where doxygen is started.\n# This tag requires that the tag FULL_PATH_NAMES is set to YES.\n\nSTRIP_FROM_PATH        =\n\n# The STRIP_FROM_INC_PATH tag can be used to strip a user-defined part of the\n# path mentioned in the documentation of a class, which tells the reader which\n# header file to include in order to use a class. If left blank only the name of\n# the header file containing the class definition is used. Otherwise one should\n# specify the list of include paths that are normally passed to the compiler\n# using the -I flag.\n\nSTRIP_FROM_INC_PATH    =\n\n# If the SHORT_NAMES tag is set to YES, doxygen will generate much shorter (but\n# less readable) file names. This can be useful is your file systems doesn't\n# support long names like on DOS, Mac, or CD-ROM.\n# The default value is: NO.\n\nSHORT_NAMES            = NO\n\n# If the JAVADOC_AUTOBRIEF tag is set to YES then doxygen will interpret the\n# first line (until the first dot) of a Javadoc-style comment as the brief\n# description. If set to NO, the Javadoc-style will behave just like regular Qt-\n# style comments (thus requiring an explicit @brief command for a brief\n# description.)\n# The default value is: NO.\n\nJAVADOC_AUTOBRIEF      = NO\n\n# If the QT_AUTOBRIEF tag is set to YES then doxygen will interpret the first\n# line (until the first dot) of a Qt-style comment as the brief description. If\n# set to NO, the Qt-style will behave just like regular Qt-style comments (thus\n# requiring an explicit \\brief command for a brief description.)\n# The default value is: NO.\n\nQT_AUTOBRIEF           = NO\n\n# The MULTILINE_CPP_IS_BRIEF tag can be set to YES to make doxygen treat a\n# multi-line C++ special comment block (i.e. a block of //! or /// comments) as\n# a brief description. This used to be the default behavior. The new default is\n# to treat a multi-line C++ comment block as a detailed description. Set this\n# tag to YES if you prefer the old behavior instead.\n#\n# Note that setting this tag to YES also means that rational rose comments are\n# not recognized any more.\n# The default value is: NO.\n\nMULTILINE_CPP_IS_BRIEF = NO\n\n# If the INHERIT_DOCS tag is set to YES then an undocumented member inherits the\n# documentation from any documented member that it re-implements.\n# The default value is: YES.\n\nINHERIT_DOCS           = YES\n\n# If the SEPARATE_MEMBER_PAGES tag is set to YES then doxygen will produce a new\n# page for each member. If set to NO, the documentation of a member will be part\n# of the file/class/namespace that contains it.\n# The default value is: NO.\n\nSEPARATE_MEMBER_PAGES  = NO\n\n# The TAB_SIZE tag can be used to set the number of spaces in a tab. Doxygen\n# uses this value to replace tabs by spaces in code fragments.\n# Minimum value: 1, maximum value: 16, default value: 4.\n\nTAB_SIZE               = 4\n\n# This tag can be used to specify a number of aliases that act as commands in\n# the documentation. An alias has the form:\n# name=value\n# For example adding\n# \"sideeffect=@par Side Effects:\\n\"\n# will allow you to put the command \\sideeffect (or @sideeffect) in the\n# documentation, which will result in a user-defined paragraph with heading\n# \"Side Effects:\". You can put \\n's in the value part of an alias to insert\n# newlines.\n\nALIASES                =\n\n# This tag can be used to specify a number of word-keyword mappings (TCL only).\n# A mapping has the form \"name=value\". For example adding \"class=itcl::class\"\n# will allow you to use the command class in the itcl::class meaning.\n\nTCL_SUBST              =\n\n# Set the OPTIMIZE_OUTPUT_FOR_C tag to YES if your project consists of C sources\n# only. Doxygen will then generate output that is more tailored for C. For\n# instance, some of the names that are used will be different. The list of all\n# members will be omitted, etc.\n# The default value is: NO.\n\nOPTIMIZE_OUTPUT_FOR_C  = YES\n\n# Set the OPTIMIZE_OUTPUT_JAVA tag to YES if your project consists of Java or\n# Python sources only. Doxygen will then generate output that is more tailored\n# for that language. For instance, namespaces will be presented as packages,\n# qualified scopes will look different, etc.\n# The default value is: NO.\n\nOPTIMIZE_OUTPUT_JAVA   = NO\n\n# Set the OPTIMIZE_FOR_FORTRAN tag to YES if your project consists of Fortran\n# sources. Doxygen will then generate output that is tailored for Fortran.\n# The default value is: NO.\n\nOPTIMIZE_FOR_FORTRAN   = NO\n\n# Set the OPTIMIZE_OUTPUT_VHDL tag to YES if your project consists of VHDL\n# sources. Doxygen will then generate output that is tailored for VHDL.\n# The default value is: NO.\n\nOPTIMIZE_OUTPUT_VHDL   = NO\n\n# Doxygen selects the parser to use depending on the extension of the files it\n# parses. With this tag you can assign which parser to use for a given\n# extension. Doxygen has a built-in mapping, but you can override or extend it\n# using this tag. The format is ext=language, where ext is a file extension, and\n# language is one of the parsers supported by doxygen: IDL, Java, Javascript,\n# C#, C, C++, D, PHP, Objective-C, Python, Fortran (fixed format Fortran:\n# FortranFixed, free formatted Fortran: FortranFree, unknown formatted Fortran:\n# Fortran. In the later case the parser tries to guess whether the code is fixed\n# or free formatted code, this is the default for Fortran type files), VHDL. For\n# instance to make doxygen treat .inc files as Fortran files (default is PHP),\n# and .f files as C (default is Fortran), use: inc=Fortran f=C.\n#\n# Note: For files without extension you can use no_extension as a placeholder.\n#\n# Note that for custom extensions you also need to set FILE_PATTERNS otherwise\n# the files are not read by doxygen.\n\nEXTENSION_MAPPING      =\n\n# If the MARKDOWN_SUPPORT tag is enabled then doxygen pre-processes all comments\n# according to the Markdown format, which allows for more readable\n# documentation. See http://daringfireball.net/projects/markdown/ for details.\n# The output of markdown processing is further processed by doxygen, so you can\n# mix doxygen, HTML, and XML commands with Markdown formatting. Disable only in\n# case of backward compatibilities issues.\n# The default value is: YES.\n\nMARKDOWN_SUPPORT       = YES\n\n# When enabled doxygen tries to link words that correspond to documented\n# classes, or namespaces to their corresponding documentation. Such a link can\n# be prevented in individual cases by putting a % sign in front of the word or\n# globally by setting AUTOLINK_SUPPORT to NO.\n# The default value is: YES.\n\nAUTOLINK_SUPPORT       = YES\n\n# If you use STL classes (i.e. std::string, std::vector, etc.) but do not want\n# to include (a tag file for) the STL sources as input, then you should set this\n# tag to YES in order to let doxygen match functions declarations and\n# definitions whose arguments contain STL classes (e.g. func(std::string);\n# versus func(std::string) {}). This also make the inheritance and collaboration\n# diagrams that involve STL classes more complete and accurate.\n# The default value is: NO.\n\nBUILTIN_STL_SUPPORT    = YES\n\n# If you use Microsoft's C++/CLI language, you should set this option to YES to\n# enable parsing support.\n# The default value is: NO.\n\nCPP_CLI_SUPPORT        = NO\n\n# Set the SIP_SUPPORT tag to YES if your project consists of sip (see:\n# http://www.riverbankcomputing.co.uk/software/sip/intro) sources only. Doxygen\n# will parse them like normal C++ but will assume all classes use public instead\n# of private inheritance when no explicit protection keyword is present.\n# The default value is: NO.\n\nSIP_SUPPORT            = NO\n\n# For Microsoft's IDL there are propget and propput attributes to indicate\n# getter and setter methods for a property. Setting this option to YES will make\n# doxygen to replace the get and set methods by a property in the documentation.\n# This will only work if the methods are indeed getting or setting a simple\n# type. If this is not the case, or you want to show the methods anyway, you\n# should set this option to NO.\n# The default value is: YES.\n\nIDL_PROPERTY_SUPPORT   = YES\n\n# If member grouping is used in the documentation and the DISTRIBUTE_GROUP_DOC\n# tag is set to YES then doxygen will reuse the documentation of the first\n# member in the group (if any) for the other members of the group. By default\n# all members of a group must be documented explicitly.\n# The default value is: NO.\n\nDISTRIBUTE_GROUP_DOC   = NO\n\n# Set the SUBGROUPING tag to YES to allow class member groups of the same type\n# (for instance a group of public functions) to be put as a subgroup of that\n# type (e.g. under the Public Functions section). Set it to NO to prevent\n# subgrouping. Alternatively, this can be done per class using the\n# \\nosubgrouping command.\n# The default value is: YES.\n\nSUBGROUPING            = YES\n\n# When the INLINE_GROUPED_CLASSES tag is set to YES, classes, structs and unions\n# are shown inside the group in which they are included (e.g. using \\ingroup)\n# instead of on a separate page (for HTML and Man pages) or section (for LaTeX\n# and RTF).\n#\n# Note that this feature does not work in combination with\n# SEPARATE_MEMBER_PAGES.\n# The default value is: NO.\n\nINLINE_GROUPED_CLASSES = NO\n\n# When the INLINE_SIMPLE_STRUCTS tag is set to YES, structs, classes, and unions\n# with only public data fields or simple typedef fields will be shown inline in\n# the documentation of the scope in which they are defined (i.e. file,\n# namespace, or group documentation), provided this scope is documented. If set\n# to NO, structs, classes, and unions are shown on a separate page (for HTML and\n# Man pages) or section (for LaTeX and RTF).\n# The default value is: NO.\n\nINLINE_SIMPLE_STRUCTS  = NO\n\n# When TYPEDEF_HIDES_STRUCT tag is enabled, a typedef of a struct, union, or\n# enum is documented as struct, union, or enum with the name of the typedef. So\n# typedef struct TypeS {} TypeT, will appear in the documentation as a struct\n# with name TypeT. When disabled the typedef will appear as a member of a file,\n# namespace, or class. And the struct will be named TypeS. This can typically be\n# useful for C code in case the coding convention dictates that all compound\n# types are typedef'ed and only the typedef is referenced, never the tag name.\n# The default value is: NO.\n\nTYPEDEF_HIDES_STRUCT   = NO\n\n# The size of the symbol lookup cache can be set using LOOKUP_CACHE_SIZE. This\n# cache is used to resolve symbols given their name and scope. Since this can be\n# an expensive process and often the same symbol appears multiple times in the\n# code, doxygen keeps a cache of pre-resolved symbols. If the cache is too small\n# doxygen will become slower. If the cache is too large, memory is wasted. The\n# cache size is given by this formula: 2^(16+LOOKUP_CACHE_SIZE). The valid range\n# is 0..9, the default is 0, corresponding to a cache size of 2^16=65536\n# symbols. At the end of a run doxygen will report the cache usage and suggest\n# the optimal cache size from a speed point of view.\n# Minimum value: 0, maximum value: 9, default value: 0.\n\nLOOKUP_CACHE_SIZE      = 0\n\n#---------------------------------------------------------------------------\n# Build related configuration options\n#---------------------------------------------------------------------------\n\n# If the EXTRACT_ALL tag is set to YES, doxygen will assume all entities in\n# documentation are documented, even if no documentation was available. Private\n# class members and static file members will be hidden unless the\n# EXTRACT_PRIVATE respectively EXTRACT_STATIC tags are set to YES.\n# Note: This will also disable the warnings about undocumented members that are\n# normally produced when WARNINGS is set to YES.\n# The default value is: NO.\n\nEXTRACT_ALL            = NO\n\n# If the EXTRACT_PRIVATE tag is set to YES, all private members of a class will\n# be included in the documentation.\n# The default value is: NO.\n\nEXTRACT_PRIVATE        = NO\n\n# If the EXTRACT_PACKAGE tag is set to YES, all members with package or internal\n# scope will be included in the documentation.\n# The default value is: NO.\n\nEXTRACT_PACKAGE        = NO\n\n# If the EXTRACT_STATIC tag is set to YES, all static members of a file will be\n# included in the documentation.\n# The default value is: NO.\n\nEXTRACT_STATIC         = NO\n\n# If the EXTRACT_LOCAL_CLASSES tag is set to YES, classes (and structs) defined\n# locally in source files will be included in the documentation. If set to NO,\n# only classes defined in header files are included. Does not have any effect\n# for Java sources.\n# The default value is: YES.\n\nEXTRACT_LOCAL_CLASSES  = YES\n\n# This flag is only useful for Objective-C code. If set to YES, local methods,\n# which are defined in the implementation section but not in the interface are\n# included in the documentation. If set to NO, only methods in the interface are\n# included.\n# The default value is: NO.\n\nEXTRACT_LOCAL_METHODS  = NO\n\n# If this flag is set to YES, the members of anonymous namespaces will be\n# extracted and appear in the documentation as a namespace called\n# 'anonymous_namespace{file}', where file will be replaced with the base name of\n# the file that contains the anonymous namespace. By default anonymous namespace\n# are hidden.\n# The default value is: NO.\n\nEXTRACT_ANON_NSPACES   = NO\n\n# If the HIDE_UNDOC_MEMBERS tag is set to YES, doxygen will hide all\n# undocumented members inside documented classes or files. If set to NO these\n# members will be included in the various overviews, but no documentation\n# section is generated. This option has no effect if EXTRACT_ALL is enabled.\n# The default value is: NO.\n\nHIDE_UNDOC_MEMBERS     = NO\n\n# If the HIDE_UNDOC_CLASSES tag is set to YES, doxygen will hide all\n# undocumented classes that are normally visible in the class hierarchy. If set\n# to NO, these classes will be included in the various overviews. This option\n# has no effect if EXTRACT_ALL is enabled.\n# The default value is: NO.\n\nHIDE_UNDOC_CLASSES     = NO\n\n# If the HIDE_FRIEND_COMPOUNDS tag is set to YES, doxygen will hide all friend\n# (class|struct|union) declarations. If set to NO, these declarations will be\n# included in the documentation.\n# The default value is: NO.\n\nHIDE_FRIEND_COMPOUNDS  = NO\n\n# If the HIDE_IN_BODY_DOCS tag is set to YES, doxygen will hide any\n# documentation blocks found inside the body of a function. If set to NO, these\n# blocks will be appended to the function's detailed documentation block.\n# The default value is: NO.\n\nHIDE_IN_BODY_DOCS      = NO\n\n# The INTERNAL_DOCS tag determines if documentation that is typed after a\n# \\internal command is included. If the tag is set to NO then the documentation\n# will be excluded. Set it to YES to include the internal documentation.\n# The default value is: NO.\n\nINTERNAL_DOCS          = NO\n\n# If the CASE_SENSE_NAMES tag is set to NO then doxygen will only generate file\n# names in lower-case letters. If set to YES, upper-case letters are also\n# allowed. This is useful if you have classes or files whose names only differ\n# in case and if your file system supports case sensitive file names. Windows\n# and Mac users are advised to set this option to NO.\n# The default value is: system dependent.\n\nCASE_SENSE_NAMES       = NO\n\n# If the HIDE_SCOPE_NAMES tag is set to NO then doxygen will show members with\n# their full class and namespace scopes in the documentation. If set to YES, the\n# scope will be hidden.\n# The default value is: NO.\n\nHIDE_SCOPE_NAMES       = NO\n\n# If the HIDE_COMPOUND_REFERENCE tag is set to NO (default) then doxygen will\n# append additional text to a page's title, such as Class Reference. If set to\n# YES the compound reference will be hidden.\n# The default value is: NO.\n\nHIDE_COMPOUND_REFERENCE= NO\n\n# If the SHOW_INCLUDE_FILES tag is set to YES then doxygen will put a list of\n# the files that are included by a file in the documentation of that file.\n# The default value is: YES.\n\nSHOW_INCLUDE_FILES     = YES\n\n# If the SHOW_GROUPED_MEMB_INC tag is set to YES then Doxygen will add for each\n# grouped member an include statement to the documentation, telling the reader\n# which file to include in order to use the member.\n# The default value is: NO.\n\nSHOW_GROUPED_MEMB_INC  = NO\n\n# If the FORCE_LOCAL_INCLUDES tag is set to YES then doxygen will list include\n# files with double quotes in the documentation rather than with sharp brackets.\n# The default value is: NO.\n\nFORCE_LOCAL_INCLUDES   = NO\n\n# If the INLINE_INFO tag is set to YES then a tag [inline] is inserted in the\n# documentation for inline members.\n# The default value is: YES.\n\nINLINE_INFO            = YES\n\n# If the SORT_MEMBER_DOCS tag is set to YES then doxygen will sort the\n# (detailed) documentation of file and class members alphabetically by member\n# name. If set to NO, the members will appear in declaration order.\n# The default value is: YES.\n\nSORT_MEMBER_DOCS       = YES\n\n# If the SORT_BRIEF_DOCS tag is set to YES then doxygen will sort the brief\n# descriptions of file, namespace and class members alphabetically by member\n# name. If set to NO, the members will appear in declaration order. Note that\n# this will also influence the order of the classes in the class list.\n# The default value is: NO.\n\nSORT_BRIEF_DOCS        = NO\n\n# If the SORT_MEMBERS_CTORS_1ST tag is set to YES then doxygen will sort the\n# (brief and detailed) documentation of class members so that constructors and\n# destructors are listed first. If set to NO the constructors will appear in the\n# respective orders defined by SORT_BRIEF_DOCS and SORT_MEMBER_DOCS.\n# Note: If SORT_BRIEF_DOCS is set to NO this option is ignored for sorting brief\n# member documentation.\n# Note: If SORT_MEMBER_DOCS is set to NO this option is ignored for sorting\n# detailed member documentation.\n# The default value is: NO.\n\nSORT_MEMBERS_CTORS_1ST = NO\n\n# If the SORT_GROUP_NAMES tag is set to YES then doxygen will sort the hierarchy\n# of group names into alphabetical order. If set to NO the group names will\n# appear in their defined order.\n# The default value is: NO.\n\nSORT_GROUP_NAMES       = NO\n\n# If the SORT_BY_SCOPE_NAME tag is set to YES, the class list will be sorted by\n# fully-qualified names, including namespaces. If set to NO, the class list will\n# be sorted only by class name, not including the namespace part.\n# Note: This option is not very useful if HIDE_SCOPE_NAMES is set to YES.\n# Note: This option applies only to the class list, not to the alphabetical\n# list.\n# The default value is: NO.\n\nSORT_BY_SCOPE_NAME     = NO\n\n# If the STRICT_PROTO_MATCHING option is enabled and doxygen fails to do proper\n# type resolution of all parameters of a function it will reject a match between\n# the prototype and the implementation of a member function even if there is\n# only one candidate or it is obvious which candidate to choose by doing a\n# simple string match. By disabling STRICT_PROTO_MATCHING doxygen will still\n# accept a match between prototype and implementation in such cases.\n# The default value is: NO.\n\nSTRICT_PROTO_MATCHING  = NO\n\n# The GENERATE_TODOLIST tag can be used to enable (YES) or disable (NO) the todo\n# list. This list is created by putting \\todo commands in the documentation.\n# The default value is: YES.\n\nGENERATE_TODOLIST      = YES\n\n# The GENERATE_TESTLIST tag can be used to enable (YES) or disable (NO) the test\n# list. This list is created by putting \\test commands in the documentation.\n# The default value is: YES.\n\nGENERATE_TESTLIST      = YES\n\n# The GENERATE_BUGLIST tag can be used to enable (YES) or disable (NO) the bug\n# list. This list is created by putting \\bug commands in the documentation.\n# The default value is: YES.\n\nGENERATE_BUGLIST       = YES\n\n# The GENERATE_DEPRECATEDLIST tag can be used to enable (YES) or disable (NO)\n# the deprecated list. This list is created by putting \\deprecated commands in\n# the documentation.\n# The default value is: YES.\n\nGENERATE_DEPRECATEDLIST= YES\n\n# The ENABLED_SECTIONS tag can be used to enable conditional documentation\n# sections, marked by \\if <section_label> ... \\endif and \\cond <section_label>\n# ... \\endcond blocks.\n\nENABLED_SECTIONS       =\n\n# The MAX_INITIALIZER_LINES tag determines the maximum number of lines that the\n# initial value of a variable or macro / define can have for it to appear in the\n# documentation. If the initializer consists of more lines than specified here\n# it will be hidden. Use a value of 0 to hide initializers completely. The\n# appearance of the value of individual variables and macros / defines can be\n# controlled using \\showinitializer or \\hideinitializer command in the\n# documentation regardless of this setting.\n# Minimum value: 0, maximum value: 10000, default value: 30.\n\nMAX_INITIALIZER_LINES  = 30\n\n# Set the SHOW_USED_FILES tag to NO to disable the list of files generated at\n# the bottom of the documentation of classes and structs. If set to YES, the\n# list will mention the files that were used to generate the documentation.\n# The default value is: YES.\n\nSHOW_USED_FILES        = YES\n\n# Set the SHOW_FILES tag to NO to disable the generation of the Files page. This\n# will remove the Files entry from the Quick Index and from the Folder Tree View\n# (if specified).\n# The default value is: YES.\n\nSHOW_FILES             = YES\n\n# Set the SHOW_NAMESPACES tag to NO to disable the generation of the Namespaces\n# page. This will remove the Namespaces entry from the Quick Index and from the\n# Folder Tree View (if specified).\n# The default value is: YES.\n\nSHOW_NAMESPACES        = YES\n\n# The FILE_VERSION_FILTER tag can be used to specify a program or script that\n# doxygen should invoke to get the current version for each file (typically from\n# the version control system). Doxygen will invoke the program by executing (via\n# popen()) the command command input-file, where command is the value of the\n# FILE_VERSION_FILTER tag, and input-file is the name of an input file provided\n# by doxygen. Whatever the program writes to standard output is used as the file\n# version. For an example see the documentation.\n\nFILE_VERSION_FILTER    =\n\n# The LAYOUT_FILE tag can be used to specify a layout file which will be parsed\n# by doxygen. The layout file controls the global structure of the generated\n# output files in an output format independent way. To create the layout file\n# that represents doxygen's defaults, run doxygen with the -l option. You can\n# optionally specify a file name after the option, if omitted DoxygenLayout.xml\n# will be used as the name of the layout file.\n#\n# Note that if you run doxygen from a directory containing a file called\n# DoxygenLayout.xml, doxygen will parse it automatically even if the LAYOUT_FILE\n# tag is left empty.\n\nLAYOUT_FILE            =\n\n# The CITE_BIB_FILES tag can be used to specify one or more bib files containing\n# the reference definitions. This must be a list of .bib files. The .bib\n# extension is automatically appended if omitted. This requires the bibtex tool\n# to be installed. See also http://en.wikipedia.org/wiki/BibTeX for more info.\n# For LaTeX the style of the bibliography can be controlled using\n# LATEX_BIB_STYLE. To use this feature you need bibtex and perl available in the\n# search path. See also \\cite for info how to create references.\n\nCITE_BIB_FILES         =\n\n#---------------------------------------------------------------------------\n# Configuration options related to warning and progress messages\n#---------------------------------------------------------------------------\n\n# The QUIET tag can be used to turn on/off the messages that are generated to\n# standard output by doxygen. If QUIET is set to YES this implies that the\n# messages are off.\n# The default value is: NO.\n\nQUIET                  = NO\n\n# The WARNINGS tag can be used to turn on/off the warning messages that are\n# generated to standard error (stderr) by doxygen. If WARNINGS is set to YES\n# this implies that the warnings are on.\n#\n# Tip: Turn warnings on while writing the documentation.\n# The default value is: YES.\n\nWARNINGS               = YES\n\n# If the WARN_IF_UNDOCUMENTED tag is set to YES then doxygen will generate\n# warnings for undocumented members. If EXTRACT_ALL is set to YES then this flag\n# will automatically be disabled.\n# The default value is: YES.\n\nWARN_IF_UNDOCUMENTED   = YES\n\n# If the WARN_IF_DOC_ERROR tag is set to YES, doxygen will generate warnings for\n# potential errors in the documentation, such as not documenting some parameters\n# in a documented function, or documenting parameters that don't exist or using\n# markup commands wrongly.\n# The default value is: YES.\n\nWARN_IF_DOC_ERROR      = YES\n\n# This WARN_NO_PARAMDOC option can be enabled to get warnings for functions that\n# are documented, but have no documentation for their parameters or return\n# value. If set to NO, doxygen will only warn about wrong or incomplete\n# parameter documentation, but not about the absence of documentation.\n# The default value is: NO.\n\nWARN_NO_PARAMDOC       = NO\n\n# The WARN_FORMAT tag determines the format of the warning messages that doxygen\n# can produce. The string should contain the $file, $line, and $text tags, which\n# will be replaced by the file and line number from which the warning originated\n# and the warning text. Optionally the format may contain $version, which will\n# be replaced by the version of the file (if it could be obtained via\n# FILE_VERSION_FILTER)\n# The default value is: $file:$line: $text.\n\nWARN_FORMAT            = \"$file:$line: $text\"\n\n# The WARN_LOGFILE tag can be used to specify a file to which warning and error\n# messages should be written. If left blank the output is written to standard\n# error (stderr).\n\nWARN_LOGFILE           =\n\n#---------------------------------------------------------------------------\n# Configuration options related to the input files\n#---------------------------------------------------------------------------\n\n# The INPUT tag is used to specify the files and/or directories that contain\n# documented source files. You may enter file names like myfile.cpp or\n# directories like /usr/src/myproject. Separate the files or directories with\n# spaces.\n# Note: If this tag is empty the current directory is searched.\n\nINPUT                  = cpp-api\n\n# This tag can be used to specify the character encoding of the source files\n# that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses\n# libiconv (or the iconv built into libc) for the transcoding. See the libiconv\n# documentation (see: http://www.gnu.org/software/libiconv) for the list of\n# possible encodings.\n# The default value is: UTF-8.\n\nINPUT_ENCODING         = UTF-8\n\n# If the value of the INPUT tag contains directories, you can use the\n# FILE_PATTERNS tag to specify one or more wildcard patterns (like *.cpp and\n# *.h) to filter out the source-files in the directories. If left blank the\n# following patterns are tested:*.c, *.cc, *.cxx, *.cpp, *.c++, *.java, *.ii,\n# *.ixx, *.ipp, *.i++, *.inl, *.idl, *.ddl, *.odl, *.h, *.hh, *.hxx, *.hpp,\n# *.h++, *.cs, *.d, *.php, *.php4, *.php5, *.phtml, *.inc, *.m, *.markdown,\n# *.md, *.mm, *.dox, *.py, *.f90, *.f, *.for, *.tcl, *.vhd, *.vhdl, *.ucf,\n# *.qsf, *.as and *.js.\n\nFILE_PATTERNS          = *.hpp *.h *.cpp\n\n# The RECURSIVE tag can be used to specify whether or not subdirectories should\n# be searched for input files as well.\n# The default value is: NO.\n\nRECURSIVE              = YES\n\n# The EXCLUDE tag can be used to specify files and/or directories that should be\n# excluded from the INPUT source files. This way you can easily exclude a\n# subdirectory from a directory tree whose root is specified with the INPUT tag.\n#\n# Note that relative paths are relative to the directory from which doxygen is\n# run.\n\nEXCLUDE                =\n\n# The EXCLUDE_SYMLINKS tag can be used to select whether or not files or\n# directories that are symbolic links (a Unix file system feature) are excluded\n# from the input.\n# The default value is: NO.\n\nEXCLUDE_SYMLINKS       = NO\n\n# If the value of the INPUT tag contains directories, you can use the\n# EXCLUDE_PATTERNS tag to specify one or more wildcard patterns to exclude\n# certain files from those directories.\n#\n# Note that the wildcards are matched against the file with absolute path, so to\n# exclude all test directories for example use the pattern */test/*\n\nEXCLUDE_PATTERNS       =\n\n# The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names\n# (namespaces, classes, functions, etc.) that should be excluded from the\n# output. The symbol name can be a fully qualified name, a word, or if the\n# wildcard * is used, a substring. Examples: ANamespace, AClass,\n# AClass::ANamespace, ANamespace::*Test\n#\n# Note that the wildcards are matched against the file with absolute path, so to\n# exclude all test directories use the pattern */test/*\n\nEXCLUDE_SYMBOLS        =\n\n# The EXAMPLE_PATH tag can be used to specify one or more files or directories\n# that contain example code fragments that are included (see the \\include\n# command).\n\nEXAMPLE_PATH           =\n\n# If the value of the EXAMPLE_PATH tag contains directories, you can use the\n# EXAMPLE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp and\n# *.h) to filter out the source-files in the directories. If left blank all\n# files are included.\n\nEXAMPLE_PATTERNS       =\n\n# If the EXAMPLE_RECURSIVE tag is set to YES then subdirectories will be\n# searched for input files to be used with the \\include or \\dontinclude commands\n# irrespective of the value of the RECURSIVE tag.\n# The default value is: NO.\n\nEXAMPLE_RECURSIVE      = NO\n\n# The IMAGE_PATH tag can be used to specify one or more files or directories\n# that contain images that are to be included in the documentation (see the\n# \\image command).\n\nIMAGE_PATH             =\n\n# The INPUT_FILTER tag can be used to specify a program that doxygen should\n# invoke to filter for each input file. Doxygen will invoke the filter program\n# by executing (via popen()) the command:\n#\n# <filter> <input-file>\n#\n# where <filter> is the value of the INPUT_FILTER tag, and <input-file> is the\n# name of an input file. Doxygen will then use the output that the filter\n# program writes to standard output. If FILTER_PATTERNS is specified, this tag\n# will be ignored.\n#\n# Note that the filter must not add or remove lines; it is applied before the\n# code is scanned, but not when the output code is generated. If lines are added\n# or removed, the anchors will not be placed correctly.\n\nINPUT_FILTER           =\n\n# The FILTER_PATTERNS tag can be used to specify filters on a per file pattern\n# basis. Doxygen will compare the file name with each pattern and apply the\n# filter if there is a match. The filters are a list of the form: pattern=filter\n# (like *.cpp=my_cpp_filter). See INPUT_FILTER for further information on how\n# filters are used. If the FILTER_PATTERNS tag is empty or if none of the\n# patterns match the file name, INPUT_FILTER is applied.\n\nFILTER_PATTERNS        =\n\n# If the FILTER_SOURCE_FILES tag is set to YES, the input filter (if set using\n# INPUT_FILTER) will also be used to filter the input files that are used for\n# producing the source files to browse (i.e. when SOURCE_BROWSER is set to YES).\n# The default value is: NO.\n\nFILTER_SOURCE_FILES    = NO\n\n# The FILTER_SOURCE_PATTERNS tag can be used to specify source filters per file\n# pattern. A pattern will override the setting for FILTER_PATTERN (if any) and\n# it is also possible to disable source filtering for a specific pattern using\n# *.ext= (so without naming a filter).\n# This tag requires that the tag FILTER_SOURCE_FILES is set to YES.\n\nFILTER_SOURCE_PATTERNS =\n\n# If the USE_MDFILE_AS_MAINPAGE tag refers to the name of a markdown file that\n# is part of the input, its contents will be placed on the main page\n# (index.html). This can be useful if you have a project on for instance GitHub\n# and want to reuse the introduction page also for the doxygen output.\n\nUSE_MDFILE_AS_MAINPAGE =\n\n#---------------------------------------------------------------------------\n# Configuration options related to source browsing\n#---------------------------------------------------------------------------\n\n# If the SOURCE_BROWSER tag is set to YES then a list of source files will be\n# generated. Documented entities will be cross-referenced with these sources.\n#\n# Note: To get rid of all source code in the generated output, make sure that\n# also VERBATIM_HEADERS is set to NO.\n# The default value is: NO.\n\nSOURCE_BROWSER         = NO\n\n# Setting the INLINE_SOURCES tag to YES will include the body of functions,\n# classes and enums directly into the documentation.\n# The default value is: NO.\n\nINLINE_SOURCES         = NO\n\n# Setting the STRIP_CODE_COMMENTS tag to YES will instruct doxygen to hide any\n# special comment blocks from generated source code fragments. Normal C, C++ and\n# Fortran comments will always remain visible.\n# The default value is: YES.\n\nSTRIP_CODE_COMMENTS    = YES\n\n# If the REFERENCED_BY_RELATION tag is set to YES then for each documented\n# function all documented functions referencing it will be listed.\n# The default value is: NO.\n\nREFERENCED_BY_RELATION = NO\n\n# If the REFERENCES_RELATION tag is set to YES then for each documented function\n# all documented entities called/used by that function will be listed.\n# The default value is: NO.\n\nREFERENCES_RELATION    = NO\n\n# If the REFERENCES_LINK_SOURCE tag is set to YES and SOURCE_BROWSER tag is set\n# to YES then the hyperlinks from functions in REFERENCES_RELATION and\n# REFERENCED_BY_RELATION lists will link to the source code. Otherwise they will\n# link to the documentation.\n# The default value is: YES.\n\nREFERENCES_LINK_SOURCE = YES\n\n# If SOURCE_TOOLTIPS is enabled (the default) then hovering a hyperlink in the\n# source code will show a tooltip with additional information such as prototype,\n# brief description and links to the definition and documentation. Since this\n# will make the HTML file larger and loading of large files a bit slower, you\n# can opt to disable this feature.\n# The default value is: YES.\n# This tag requires that the tag SOURCE_BROWSER is set to YES.\n\nSOURCE_TOOLTIPS        = YES\n\n# If the USE_HTAGS tag is set to YES then the references to source code will\n# point to the HTML generated by the htags(1) tool instead of doxygen built-in\n# source browser. The htags tool is part of GNU's global source tagging system\n# (see http://www.gnu.org/software/global/global.html). You will need version\n# 4.8.6 or higher.\n#\n# To use it do the following:\n# - Install the latest version of global\n# - Enable SOURCE_BROWSER and USE_HTAGS in the config file\n# - Make sure the INPUT points to the root of the source tree\n# - Run doxygen as normal\n#\n# Doxygen will invoke htags (and that will in turn invoke gtags), so these\n# tools must be available from the command line (i.e. in the search path).\n#\n# The result: instead of the source browser generated by doxygen, the links to\n# source code will now point to the output of htags.\n# The default value is: NO.\n# This tag requires that the tag SOURCE_BROWSER is set to YES.\n\nUSE_HTAGS              = NO\n\n# If the VERBATIM_HEADERS tag is set the YES then doxygen will generate a\n# verbatim copy of the header file for each class for which an include is\n# specified. Set to NO to disable this.\n# See also: Section \\class.\n# The default value is: YES.\n\nVERBATIM_HEADERS       = YES\n\n#---------------------------------------------------------------------------\n# Configuration options related to the alphabetical class index\n#---------------------------------------------------------------------------\n\n# If the ALPHABETICAL_INDEX tag is set to YES, an alphabetical index of all\n# compounds will be generated. Enable this if the project contains a lot of\n# classes, structs, unions or interfaces.\n# The default value is: YES.\n\nALPHABETICAL_INDEX     = YES\n\n# The COLS_IN_ALPHA_INDEX tag can be used to specify the number of columns in\n# which the alphabetical index list will be split.\n# Minimum value: 1, maximum value: 20, default value: 5.\n# This tag requires that the tag ALPHABETICAL_INDEX is set to YES.\n\nCOLS_IN_ALPHA_INDEX    = 5\n\n# In case all classes in a project start with a common prefix, all classes will\n# be put under the same header in the alphabetical index. The IGNORE_PREFIX tag\n# can be used to specify a prefix (or a list of prefixes) that should be ignored\n# while generating the index headers.\n# This tag requires that the tag ALPHABETICAL_INDEX is set to YES.\n\nIGNORE_PREFIX          =\n\n#---------------------------------------------------------------------------\n# Configuration options related to the HTML output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_HTML tag is set to YES, doxygen will generate HTML output\n# The default value is: YES.\n\nGENERATE_HTML          = YES\n\n# The HTML_OUTPUT tag is used to specify where the HTML docs will be put. If a\n# relative path is entered the value of OUTPUT_DIRECTORY will be put in front of\n# it.\n# The default directory is: html.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_OUTPUT            = html\n\n# The HTML_FILE_EXTENSION tag can be used to specify the file extension for each\n# generated HTML page (for example: .htm, .php, .asp).\n# The default value is: .html.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_FILE_EXTENSION    = .html\n\n# The HTML_HEADER tag can be used to specify a user-defined HTML header file for\n# each generated HTML page. If the tag is left blank doxygen will generate a\n# standard header.\n#\n# To get valid HTML the header file that includes any scripts and style sheets\n# that doxygen needs, which is dependent on the configuration options used (e.g.\n# the setting GENERATE_TREEVIEW). It is highly recommended to start with a\n# default header using\n# doxygen -w html new_header.html new_footer.html new_stylesheet.css\n# YourConfigFile\n# and then modify the file new_header.html. See also section \"Doxygen usage\"\n# for information on how to generate the default header that doxygen normally\n# uses.\n# Note: The header is subject to change so you typically have to regenerate the\n# default header when upgrading to a newer version of doxygen. For a description\n# of the possible markers and block names see the documentation.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_HEADER            =\n\n# The HTML_FOOTER tag can be used to specify a user-defined HTML footer for each\n# generated HTML page. If the tag is left blank doxygen will generate a standard\n# footer. See HTML_HEADER for more information on how to generate a default\n# footer and what special commands can be used inside the footer. See also\n# section \"Doxygen usage\" for information on how to generate the default footer\n# that doxygen normally uses.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_FOOTER            =\n\n# The HTML_STYLESHEET tag can be used to specify a user-defined cascading style\n# sheet that is used by each HTML page. It can be used to fine-tune the look of\n# the HTML output. If left blank doxygen will generate a default style sheet.\n# See also section \"Doxygen usage\" for information on how to generate the style\n# sheet that doxygen normally uses.\n# Note: It is recommended to use HTML_EXTRA_STYLESHEET instead of this tag, as\n# it is more robust and this tag (HTML_STYLESHEET) will in the future become\n# obsolete.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_STYLESHEET        =\n\n# The HTML_EXTRA_STYLESHEET tag can be used to specify additional user-defined\n# cascading style sheets that are included after the standard style sheets\n# created by doxygen. Using this option one can overrule certain style aspects.\n# This is preferred over using HTML_STYLESHEET since it does not replace the\n# standard style sheet and is therefore more robust against future updates.\n# Doxygen will copy the style sheet files to the output directory.\n# Note: The order of the extra style sheet files is of importance (e.g. the last\n# style sheet in the list overrules the setting of the previous ones in the\n# list). For an example see the documentation.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_EXTRA_STYLESHEET  =\n\n# The HTML_EXTRA_FILES tag can be used to specify one or more extra images or\n# other source files which should be copied to the HTML output directory. Note\n# that these files will be copied to the base HTML output directory. Use the\n# $relpath^ marker in the HTML_HEADER and/or HTML_FOOTER files to load these\n# files. In the HTML_STYLESHEET file, use the file name only. Also note that the\n# files will be copied as-is; there are no commands or markers available.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_EXTRA_FILES       =\n\n# The HTML_COLORSTYLE_HUE tag controls the color of the HTML output. Doxygen\n# will adjust the colors in the style sheet and background images according to\n# this color. Hue is specified as an angle on a colorwheel, see\n# http://en.wikipedia.org/wiki/Hue for more information. For instance the value\n# 0 represents red, 60 is yellow, 120 is green, 180 is cyan, 240 is blue, 300\n# purple, and 360 is red again.\n# Minimum value: 0, maximum value: 359, default value: 220.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_COLORSTYLE_HUE    = 220\n\n# The HTML_COLORSTYLE_SAT tag controls the purity (or saturation) of the colors\n# in the HTML output. For a value of 0 the output will use grayscales only. A\n# value of 255 will produce the most vivid colors.\n# Minimum value: 0, maximum value: 255, default value: 100.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_COLORSTYLE_SAT    = 100\n\n# The HTML_COLORSTYLE_GAMMA tag controls the gamma correction applied to the\n# luminance component of the colors in the HTML output. Values below 100\n# gradually make the output lighter, whereas values above 100 make the output\n# darker. The value divided by 100 is the actual gamma applied, so 80 represents\n# a gamma of 0.8, The value 220 represents a gamma of 2.2, and 100 does not\n# change the gamma.\n# Minimum value: 40, maximum value: 240, default value: 80.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_COLORSTYLE_GAMMA  = 80\n\n# If the HTML_TIMESTAMP tag is set to YES then the footer of each generated HTML\n# page will contain the date and time when the page was generated. Setting this\n# to NO can help when comparing the output of multiple runs.\n# The default value is: YES.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_TIMESTAMP         = YES\n\n# If the HTML_DYNAMIC_SECTIONS tag is set to YES then the generated HTML\n# documentation will contain sections that can be hidden and shown after the\n# page has loaded.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_DYNAMIC_SECTIONS  = NO\n\n# With HTML_INDEX_NUM_ENTRIES one can control the preferred number of entries\n# shown in the various tree structured indices initially; the user can expand\n# and collapse entries dynamically later on. Doxygen will expand the tree to\n# such a level that at most the specified number of entries are visible (unless\n# a fully collapsed tree already exceeds this amount). So setting the number of\n# entries 1 will produce a full collapsed tree by default. 0 is a special value\n# representing an infinite number of entries and will result in a full expanded\n# tree by default.\n# Minimum value: 0, maximum value: 9999, default value: 100.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_INDEX_NUM_ENTRIES = 100\n\n# If the GENERATE_DOCSET tag is set to YES, additional index files will be\n# generated that can be used as input for Apple's Xcode 3 integrated development\n# environment (see: http://developer.apple.com/tools/xcode/), introduced with\n# OSX 10.5 (Leopard). To create a documentation set, doxygen will generate a\n# Makefile in the HTML output directory. Running make will produce the docset in\n# that directory and running make install will install the docset in\n# ~/Library/Developer/Shared/Documentation/DocSets so that Xcode will find it at\n# startup. See http://developer.apple.com/tools/creatingdocsetswithdoxygen.html\n# for more information.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nGENERATE_DOCSET        = NO\n\n# This tag determines the name of the docset feed. A documentation feed provides\n# an umbrella under which multiple documentation sets from a single provider\n# (such as a company or product suite) can be grouped.\n# The default value is: Doxygen generated docs.\n# This tag requires that the tag GENERATE_DOCSET is set to YES.\n\nDOCSET_FEEDNAME        = \"Doxygen generated docs\"\n\n# This tag specifies a string that should uniquely identify the documentation\n# set bundle. This should be a reverse domain-name style string, e.g.\n# com.mycompany.MyDocSet. Doxygen will append .docset to the name.\n# The default value is: org.doxygen.Project.\n# This tag requires that the tag GENERATE_DOCSET is set to YES.\n\nDOCSET_BUNDLE_ID       = org.doxygen.Project\n\n# The DOCSET_PUBLISHER_ID tag specifies a string that should uniquely identify\n# the documentation publisher. This should be a reverse domain-name style\n# string, e.g. com.mycompany.MyDocSet.documentation.\n# The default value is: org.doxygen.Publisher.\n# This tag requires that the tag GENERATE_DOCSET is set to YES.\n\nDOCSET_PUBLISHER_ID    = org.doxygen.Publisher\n\n# The DOCSET_PUBLISHER_NAME tag identifies the documentation publisher.\n# The default value is: Publisher.\n# This tag requires that the tag GENERATE_DOCSET is set to YES.\n\nDOCSET_PUBLISHER_NAME  = Publisher\n\n# If the GENERATE_HTMLHELP tag is set to YES then doxygen generates three\n# additional HTML index files: index.hhp, index.hhc, and index.hhk. The\n# index.hhp is a project file that can be read by Microsoft's HTML Help Workshop\n# (see: http://www.microsoft.com/en-us/download/details.aspx?id=21138) on\n# Windows.\n#\n# The HTML Help Workshop contains a compiler that can convert all HTML output\n# generated by doxygen into a single compiled HTML file (.chm). Compiled HTML\n# files are now used as the Windows 98 help format, and will replace the old\n# Windows help format (.hlp) on all Windows platforms in the future. Compressed\n# HTML files also contain an index, a table of contents, and you can search for\n# words in the documentation. The HTML workshop also contains a viewer for\n# compressed HTML files.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nGENERATE_HTMLHELP      = NO\n\n# The CHM_FILE tag can be used to specify the file name of the resulting .chm\n# file. You can add a path in front of the file if the result should not be\n# written to the html output directory.\n# This tag requires that the tag GENERATE_HTMLHELP is set to YES.\n\nCHM_FILE               =\n\n# The HHC_LOCATION tag can be used to specify the location (absolute path\n# including file name) of the HTML help compiler (hhc.exe). If non-empty,\n# doxygen will try to run the HTML help compiler on the generated index.hhp.\n# The file has to be specified with full path.\n# This tag requires that the tag GENERATE_HTMLHELP is set to YES.\n\nHHC_LOCATION           =\n\n# The GENERATE_CHI flag controls if a separate .chi index file is generated\n# (YES) or that it should be included in the master .chm file (NO).\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTMLHELP is set to YES.\n\nGENERATE_CHI           = NO\n\n# The CHM_INDEX_ENCODING is used to encode HtmlHelp index (hhk), content (hhc)\n# and project file content.\n# This tag requires that the tag GENERATE_HTMLHELP is set to YES.\n\nCHM_INDEX_ENCODING     =\n\n# The BINARY_TOC flag controls whether a binary table of contents is generated\n# (YES) or a normal table of contents (NO) in the .chm file. Furthermore it\n# enables the Previous and Next buttons.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTMLHELP is set to YES.\n\nBINARY_TOC             = NO\n\n# The TOC_EXPAND flag can be set to YES to add extra items for group members to\n# the table of contents of the HTML help documentation and to the tree view.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTMLHELP is set to YES.\n\nTOC_EXPAND             = NO\n\n# If the GENERATE_QHP tag is set to YES and both QHP_NAMESPACE and\n# QHP_VIRTUAL_FOLDER are set, an additional index file will be generated that\n# can be used as input for Qt's qhelpgenerator to generate a Qt Compressed Help\n# (.qch) of the generated HTML documentation.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nGENERATE_QHP           = NO\n\n# If the QHG_LOCATION tag is specified, the QCH_FILE tag can be used to specify\n# the file name of the resulting .qch file. The path specified is relative to\n# the HTML output folder.\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQCH_FILE               =\n\n# The QHP_NAMESPACE tag specifies the namespace to use when generating Qt Help\n# Project output. For more information please see Qt Help Project / Namespace\n# (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#namespace).\n# The default value is: org.doxygen.Project.\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQHP_NAMESPACE          = org.doxygen.Project\n\n# The QHP_VIRTUAL_FOLDER tag specifies the namespace to use when generating Qt\n# Help Project output. For more information please see Qt Help Project / Virtual\n# Folders (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#virtual-\n# folders).\n# The default value is: doc.\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQHP_VIRTUAL_FOLDER     = doc\n\n# If the QHP_CUST_FILTER_NAME tag is set, it specifies the name of a custom\n# filter to add. For more information please see Qt Help Project / Custom\n# Filters (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#custom-\n# filters).\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQHP_CUST_FILTER_NAME   =\n\n# The QHP_CUST_FILTER_ATTRS tag specifies the list of the attributes of the\n# custom filter to add. For more information please see Qt Help Project / Custom\n# Filters (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#custom-\n# filters).\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQHP_CUST_FILTER_ATTRS  =\n\n# The QHP_SECT_FILTER_ATTRS tag specifies the list of the attributes this\n# project's filter section matches. Qt Help Project / Filter Attributes (see:\n# http://qt-project.org/doc/qt-4.8/qthelpproject.html#filter-attributes).\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQHP_SECT_FILTER_ATTRS  =\n\n# The QHG_LOCATION tag can be used to specify the location of Qt's\n# qhelpgenerator. If non-empty doxygen will try to run qhelpgenerator on the\n# generated .qhp file.\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQHG_LOCATION           =\n\n# If the GENERATE_ECLIPSEHELP tag is set to YES, additional index files will be\n# generated, together with the HTML files, they form an Eclipse help plugin. To\n# install this plugin and make it available under the help contents menu in\n# Eclipse, the contents of the directory containing the HTML and XML files needs\n# to be copied into the plugins directory of eclipse. The name of the directory\n# within the plugins directory should be the same as the ECLIPSE_DOC_ID value.\n# After copying Eclipse needs to be restarted before the help appears.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nGENERATE_ECLIPSEHELP   = NO\n\n# A unique identifier for the Eclipse help plugin. When installing the plugin\n# the directory name containing the HTML and XML files should also have this\n# name. Each documentation set should have its own identifier.\n# The default value is: org.doxygen.Project.\n# This tag requires that the tag GENERATE_ECLIPSEHELP is set to YES.\n\nECLIPSE_DOC_ID         = org.doxygen.Project\n\n# If you want full control over the layout of the generated HTML pages it might\n# be necessary to disable the index and replace it with your own. The\n# DISABLE_INDEX tag can be used to turn on/off the condensed index (tabs) at top\n# of each HTML page. A value of NO enables the index and the value YES disables\n# it. Since the tabs in the index contain the same information as the navigation\n# tree, you can set this option to YES if you also set GENERATE_TREEVIEW to YES.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nDISABLE_INDEX          = NO\n\n# The GENERATE_TREEVIEW tag is used to specify whether a tree-like index\n# structure should be generated to display hierarchical information. If the tag\n# value is set to YES, a side panel will be generated containing a tree-like\n# index structure (just like the one that is generated for HTML Help). For this\n# to work a browser that supports JavaScript, DHTML, CSS and frames is required\n# (i.e. any modern browser). Windows users are probably better off using the\n# HTML help feature. Via custom style sheets (see HTML_EXTRA_STYLESHEET) one can\n# further fine-tune the look of the index. As an example, the default style\n# sheet generated by doxygen has an example that shows how to put an image at\n# the root of the tree instead of the PROJECT_NAME. Since the tree basically has\n# the same information as the tab index, you could consider setting\n# DISABLE_INDEX to YES when enabling this option.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nGENERATE_TREEVIEW      = NO\n\n# The ENUM_VALUES_PER_LINE tag can be used to set the number of enum values that\n# doxygen will group on one line in the generated HTML documentation.\n#\n# Note that a value of 0 will completely suppress the enum values from appearing\n# in the overview section.\n# Minimum value: 0, maximum value: 20, default value: 4.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nENUM_VALUES_PER_LINE   = 4\n\n# If the treeview is enabled (see GENERATE_TREEVIEW) then this tag can be used\n# to set the initial width (in pixels) of the frame in which the tree is shown.\n# Minimum value: 0, maximum value: 1500, default value: 250.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nTREEVIEW_WIDTH         = 250\n\n# If the EXT_LINKS_IN_WINDOW option is set to YES, doxygen will open links to\n# external symbols imported via tag files in a separate window.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nEXT_LINKS_IN_WINDOW    = NO\n\n# Use this tag to change the font size of LaTeX formulas included as images in\n# the HTML documentation. When you change the font size after a successful\n# doxygen run you need to manually remove any form_*.png images from the HTML\n# output directory to force them to be regenerated.\n# Minimum value: 8, maximum value: 50, default value: 10.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nFORMULA_FONTSIZE       = 10\n\n# Use the FORMULA_TRANPARENT tag to determine whether or not the images\n# generated for formulas are transparent PNGs. Transparent PNGs are not\n# supported properly for IE 6.0, but are supported on all modern browsers.\n#\n# Note that when changing this option you need to delete any form_*.png files in\n# the HTML output directory before the changes have effect.\n# The default value is: YES.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nFORMULA_TRANSPARENT    = YES\n\n# Enable the USE_MATHJAX option to render LaTeX formulas using MathJax (see\n# http://www.mathjax.org) which uses client side Javascript for the rendering\n# instead of using pre-rendered bitmaps. Use this if you do not have LaTeX\n# installed or if you want to formulas look prettier in the HTML output. When\n# enabled you may also need to install MathJax separately and configure the path\n# to it using the MATHJAX_RELPATH option.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nUSE_MATHJAX            = NO\n\n# When MathJax is enabled you can set the default output format to be used for\n# the MathJax output. See the MathJax site (see:\n# http://docs.mathjax.org/en/latest/output.html) for more details.\n# Possible values are: HTML-CSS (which is slower, but has the best\n# compatibility), NativeMML (i.e. MathML) and SVG.\n# The default value is: HTML-CSS.\n# This tag requires that the tag USE_MATHJAX is set to YES.\n\nMATHJAX_FORMAT         = HTML-CSS\n\n# When MathJax is enabled you need to specify the location relative to the HTML\n# output directory using the MATHJAX_RELPATH option. The destination directory\n# should contain the MathJax.js script. For instance, if the mathjax directory\n# is located at the same level as the HTML output directory, then\n# MATHJAX_RELPATH should be ../mathjax. The default value points to the MathJax\n# Content Delivery Network so you can quickly see the result without installing\n# MathJax. However, it is strongly recommended to install a local copy of\n# MathJax from http://www.mathjax.org before deployment.\n# The default value is: http://cdn.mathjax.org/mathjax/latest.\n# This tag requires that the tag USE_MATHJAX is set to YES.\n\nMATHJAX_RELPATH        = http://cdn.mathjax.org/mathjax/latest\n\n# The MATHJAX_EXTENSIONS tag can be used to specify one or more MathJax\n# extension names that should be enabled during MathJax rendering. For example\n# MATHJAX_EXTENSIONS = TeX/AMSmath TeX/AMSsymbols\n# This tag requires that the tag USE_MATHJAX is set to YES.\n\nMATHJAX_EXTENSIONS     =\n\n# The MATHJAX_CODEFILE tag can be used to specify a file with javascript pieces\n# of code that will be used on startup of the MathJax code. See the MathJax site\n# (see: http://docs.mathjax.org/en/latest/output.html) for more details. For an\n# example see the documentation.\n# This tag requires that the tag USE_MATHJAX is set to YES.\n\nMATHJAX_CODEFILE       =\n\n# When the SEARCHENGINE tag is enabled doxygen will generate a search box for\n# the HTML output. The underlying search engine uses javascript and DHTML and\n# should work on any modern browser. Note that when using HTML help\n# (GENERATE_HTMLHELP), Qt help (GENERATE_QHP), or docsets (GENERATE_DOCSET)\n# there is already a search function so this one should typically be disabled.\n# For large projects the javascript based search engine can be slow, then\n# enabling SERVER_BASED_SEARCH may provide a better solution. It is possible to\n# search using the keyboard; to jump to the search box use <access key> + S\n# (what the <access key> is depends on the OS and browser, but it is typically\n# <CTRL>, <ALT>/<option>, or both). Inside the search box use the <cursor down\n# key> to jump into the search results window, the results can be navigated\n# using the <cursor keys>. Press <Enter> to select an item or <escape> to cancel\n# the search. The filter options can be selected when the cursor is inside the\n# search box by pressing <Shift>+<cursor down>. Also here use the <cursor keys>\n# to select a filter and <Enter> or <escape> to activate or cancel the filter\n# option.\n# The default value is: YES.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nSEARCHENGINE           = YES\n\n# When the SERVER_BASED_SEARCH tag is enabled the search engine will be\n# implemented using a web server instead of a web client using Javascript. There\n# are two flavors of web server based searching depending on the EXTERNAL_SEARCH\n# setting. When disabled, doxygen will generate a PHP script for searching and\n# an index file used by the script. When EXTERNAL_SEARCH is enabled the indexing\n# and searching needs to be provided by external tools. See the section\n# \"External Indexing and Searching\" for details.\n# The default value is: NO.\n# This tag requires that the tag SEARCHENGINE is set to YES.\n\nSERVER_BASED_SEARCH    = NO\n\n# When EXTERNAL_SEARCH tag is enabled doxygen will no longer generate the PHP\n# script for searching. Instead the search results are written to an XML file\n# which needs to be processed by an external indexer. Doxygen will invoke an\n# external search engine pointed to by the SEARCHENGINE_URL option to obtain the\n# search results.\n#\n# Doxygen ships with an example indexer (doxyindexer) and search engine\n# (doxysearch.cgi) which are based on the open source search engine library\n# Xapian (see: http://xapian.org/).\n#\n# See the section \"External Indexing and Searching\" for details.\n# The default value is: NO.\n# This tag requires that the tag SEARCHENGINE is set to YES.\n\nEXTERNAL_SEARCH        = NO\n\n# The SEARCHENGINE_URL should point to a search engine hosted by a web server\n# which will return the search results when EXTERNAL_SEARCH is enabled.\n#\n# Doxygen ships with an example indexer (doxyindexer) and search engine\n# (doxysearch.cgi) which are based on the open source search engine library\n# Xapian (see: http://xapian.org/). See the section \"External Indexing and\n# Searching\" for details.\n# This tag requires that the tag SEARCHENGINE is set to YES.\n\nSEARCHENGINE_URL       =\n\n# When SERVER_BASED_SEARCH and EXTERNAL_SEARCH are both enabled the unindexed\n# search data is written to a file for indexing by an external tool. With the\n# SEARCHDATA_FILE tag the name of this file can be specified.\n# The default file is: searchdata.xml.\n# This tag requires that the tag SEARCHENGINE is set to YES.\n\nSEARCHDATA_FILE        = searchdata.xml\n\n# When SERVER_BASED_SEARCH and EXTERNAL_SEARCH are both enabled the\n# EXTERNAL_SEARCH_ID tag can be used as an identifier for the project. This is\n# useful in combination with EXTRA_SEARCH_MAPPINGS to search through multiple\n# projects and redirect the results back to the right project.\n# This tag requires that the tag SEARCHENGINE is set to YES.\n\nEXTERNAL_SEARCH_ID     =\n\n# The EXTRA_SEARCH_MAPPINGS tag can be used to enable searching through doxygen\n# projects other than the one defined by this configuration file, but that are\n# all added to the same external search index. Each project needs to have a\n# unique id set via EXTERNAL_SEARCH_ID. The search mapping then maps the id of\n# to a relative location where the documentation can be found. The format is:\n# EXTRA_SEARCH_MAPPINGS = tagname1=loc1 tagname2=loc2 ...\n# This tag requires that the tag SEARCHENGINE is set to YES.\n\nEXTRA_SEARCH_MAPPINGS  =\n\n#---------------------------------------------------------------------------\n# Configuration options related to the LaTeX output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_LATEX tag is set to YES, doxygen will generate LaTeX output.\n# The default value is: YES.\n\nGENERATE_LATEX         = NO\n\n# The LATEX_OUTPUT tag is used to specify where the LaTeX docs will be put. If a\n# relative path is entered the value of OUTPUT_DIRECTORY will be put in front of\n# it.\n# The default directory is: latex.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nLATEX_OUTPUT           = latex\n\n# The LATEX_CMD_NAME tag can be used to specify the LaTeX command name to be\n# invoked.\n#\n# Note that when enabling USE_PDFLATEX this option is only used for generating\n# bitmaps for formulas in the HTML output, but not in the Makefile that is\n# written to the output directory.\n# The default file is: latex.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nLATEX_CMD_NAME         = latex\n\n# The MAKEINDEX_CMD_NAME tag can be used to specify the command name to generate\n# index for LaTeX.\n# The default file is: makeindex.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nMAKEINDEX_CMD_NAME     = makeindex\n\n# If the COMPACT_LATEX tag is set to YES, doxygen generates more compact LaTeX\n# documents. This may be useful for small projects and may help to save some\n# trees in general.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nCOMPACT_LATEX          = NO\n\n# The PAPER_TYPE tag can be used to set the paper type that is used by the\n# printer.\n# Possible values are: a4 (210 x 297 mm), letter (8.5 x 11 inches), legal (8.5 x\n# 14 inches) and executive (7.25 x 10.5 inches).\n# The default value is: a4.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nPAPER_TYPE             = a4\n\n# The EXTRA_PACKAGES tag can be used to specify one or more LaTeX package names\n# that should be included in the LaTeX output. To get the times font for\n# instance you can specify\n# EXTRA_PACKAGES=times\n# If left blank no extra packages will be included.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nEXTRA_PACKAGES         =\n\n# The LATEX_HEADER tag can be used to specify a personal LaTeX header for the\n# generated LaTeX document. The header should contain everything until the first\n# chapter. If it is left blank doxygen will generate a standard header. See\n# section \"Doxygen usage\" for information on how to let doxygen write the\n# default header to a separate file.\n#\n# Note: Only use a user-defined header if you know what you are doing! The\n# following commands have a special meaning inside the header: $title,\n# $datetime, $date, $doxygenversion, $projectname, $projectnumber,\n# $projectbrief, $projectlogo. Doxygen will replace $title with the empty\n# string, for the replacement values of the other commands the user is referred\n# to HTML_HEADER.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nLATEX_HEADER           =\n\n# The LATEX_FOOTER tag can be used to specify a personal LaTeX footer for the\n# generated LaTeX document. The footer should contain everything after the last\n# chapter. If it is left blank doxygen will generate a standard footer. See\n# LATEX_HEADER for more information on how to generate a default footer and what\n# special commands can be used inside the footer.\n#\n# Note: Only use a user-defined footer if you know what you are doing!\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nLATEX_FOOTER           =\n\n# The LATEX_EXTRA_STYLESHEET tag can be used to specify additional user-defined\n# LaTeX style sheets that are included after the standard style sheets created\n# by doxygen. Using this option one can overrule certain style aspects. Doxygen\n# will copy the style sheet files to the output directory.\n# Note: The order of the extra style sheet files is of importance (e.g. the last\n# style sheet in the list overrules the setting of the previous ones in the\n# list).\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nLATEX_EXTRA_STYLESHEET =\n\n# The LATEX_EXTRA_FILES tag can be used to specify one or more extra images or\n# other source files which should be copied to the LATEX_OUTPUT output\n# directory. Note that the files will be copied as-is; there are no commands or\n# markers available.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nLATEX_EXTRA_FILES      =\n\n# If the PDF_HYPERLINKS tag is set to YES, the LaTeX that is generated is\n# prepared for conversion to PDF (using ps2pdf or pdflatex). The PDF file will\n# contain links (just like the HTML output) instead of page references. This\n# makes the output suitable for online browsing using a PDF viewer.\n# The default value is: YES.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nPDF_HYPERLINKS         = YES\n\n# If the USE_PDFLATEX tag is set to YES, doxygen will use pdflatex to generate\n# the PDF file directly from the LaTeX files. Set this option to YES, to get a\n# higher quality PDF documentation.\n# The default value is: YES.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nUSE_PDFLATEX           = YES\n\n# If the LATEX_BATCHMODE tag is set to YES, doxygen will add the \\batchmode\n# command to the generated LaTeX files. This will instruct LaTeX to keep running\n# if errors occur, instead of asking the user for help. This option is also used\n# when generating formulas in HTML.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nLATEX_BATCHMODE        = NO\n\n# If the LATEX_HIDE_INDICES tag is set to YES then doxygen will not include the\n# index chapters (such as File Index, Compound Index, etc.) in the output.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nLATEX_HIDE_INDICES     = NO\n\n# If the LATEX_SOURCE_CODE tag is set to YES then doxygen will include source\n# code with syntax highlighting in the LaTeX output.\n#\n# Note that which sources are shown also depends on other settings such as\n# SOURCE_BROWSER.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nLATEX_SOURCE_CODE      = NO\n\n# The LATEX_BIB_STYLE tag can be used to specify the style to use for the\n# bibliography, e.g. plainnat, or ieeetr. See\n# http://en.wikipedia.org/wiki/BibTeX and \\cite for more info.\n# The default value is: plain.\n# This tag requires that the tag GENERATE_LATEX is set to YES.\n\nLATEX_BIB_STYLE        = plain\n\n#---------------------------------------------------------------------------\n# Configuration options related to the RTF output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_RTF tag is set to YES, doxygen will generate RTF output. The\n# RTF output is optimized for Word 97 and may not look too pretty with other RTF\n# readers/editors.\n# The default value is: NO.\n\nGENERATE_RTF           = NO\n\n# The RTF_OUTPUT tag is used to specify where the RTF docs will be put. If a\n# relative path is entered the value of OUTPUT_DIRECTORY will be put in front of\n# it.\n# The default directory is: rtf.\n# This tag requires that the tag GENERATE_RTF is set to YES.\n\nRTF_OUTPUT             = rtf\n\n# If the COMPACT_RTF tag is set to YES, doxygen generates more compact RTF\n# documents. This may be useful for small projects and may help to save some\n# trees in general.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_RTF is set to YES.\n\nCOMPACT_RTF            = NO\n\n# If the RTF_HYPERLINKS tag is set to YES, the RTF that is generated will\n# contain hyperlink fields. The RTF file will contain links (just like the HTML\n# output) instead of page references. This makes the output suitable for online\n# browsing using Word or some other Word compatible readers that support those\n# fields.\n#\n# Note: WordPad (write) and others do not support links.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_RTF is set to YES.\n\nRTF_HYPERLINKS         = NO\n\n# Load stylesheet definitions from file. Syntax is similar to doxygen's config\n# file, i.e. a series of assignments. You only have to provide replacements,\n# missing definitions are set to their default value.\n#\n# See also section \"Doxygen usage\" for information on how to generate the\n# default style sheet that doxygen normally uses.\n# This tag requires that the tag GENERATE_RTF is set to YES.\n\nRTF_STYLESHEET_FILE    =\n\n# Set optional variables used in the generation of an RTF document. Syntax is\n# similar to doxygen's config file. A template extensions file can be generated\n# using doxygen -e rtf extensionFile.\n# This tag requires that the tag GENERATE_RTF is set to YES.\n\nRTF_EXTENSIONS_FILE    =\n\n# If the RTF_SOURCE_CODE tag is set to YES then doxygen will include source code\n# with syntax highlighting in the RTF output.\n#\n# Note that which sources are shown also depends on other settings such as\n# SOURCE_BROWSER.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_RTF is set to YES.\n\nRTF_SOURCE_CODE        = NO\n\n#---------------------------------------------------------------------------\n# Configuration options related to the man page output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_MAN tag is set to YES, doxygen will generate man pages for\n# classes and files.\n# The default value is: NO.\n\nGENERATE_MAN           = NO\n\n# The MAN_OUTPUT tag is used to specify where the man pages will be put. If a\n# relative path is entered the value of OUTPUT_DIRECTORY will be put in front of\n# it. A directory man3 will be created inside the directory specified by\n# MAN_OUTPUT.\n# The default directory is: man.\n# This tag requires that the tag GENERATE_MAN is set to YES.\n\nMAN_OUTPUT             = man\n\n# The MAN_EXTENSION tag determines the extension that is added to the generated\n# man pages. In case the manual section does not start with a number, the number\n# 3 is prepended. The dot (.) at the beginning of the MAN_EXTENSION tag is\n# optional.\n# The default value is: .3.\n# This tag requires that the tag GENERATE_MAN is set to YES.\n\nMAN_EXTENSION          = .3\n\n# The MAN_SUBDIR tag determines the name of the directory created within\n# MAN_OUTPUT in which the man pages are placed. If defaults to man followed by\n# MAN_EXTENSION with the initial . removed.\n# This tag requires that the tag GENERATE_MAN is set to YES.\n\nMAN_SUBDIR             =\n\n# If the MAN_LINKS tag is set to YES and doxygen generates man output, then it\n# will generate one additional man file for each entity documented in the real\n# man page(s). These additional files only source the real man page, but without\n# them the man command would be unable to find the correct page.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_MAN is set to YES.\n\nMAN_LINKS              = NO\n\n#---------------------------------------------------------------------------\n# Configuration options related to the XML output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_XML tag is set to YES, doxygen will generate an XML file that\n# captures the structure of the code including all documentation.\n# The default value is: NO.\n\nGENERATE_XML           = NO\n\n# The XML_OUTPUT tag is used to specify where the XML pages will be put. If a\n# relative path is entered the value of OUTPUT_DIRECTORY will be put in front of\n# it.\n# The default directory is: xml.\n# This tag requires that the tag GENERATE_XML is set to YES.\n\nXML_OUTPUT             = xml\n\n# If the XML_PROGRAMLISTING tag is set to YES, doxygen will dump the program\n# listings (including syntax highlighting and cross-referencing information) to\n# the XML output. Note that enabling this will significantly increase the size\n# of the XML output.\n# The default value is: YES.\n# This tag requires that the tag GENERATE_XML is set to YES.\n\nXML_PROGRAMLISTING     = YES\n\n#---------------------------------------------------------------------------\n# Configuration options related to the DOCBOOK output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_DOCBOOK tag is set to YES, doxygen will generate Docbook files\n# that can be used to generate PDF.\n# The default value is: NO.\n\nGENERATE_DOCBOOK       = NO\n\n# The DOCBOOK_OUTPUT tag is used to specify where the Docbook pages will be put.\n# If a relative path is entered the value of OUTPUT_DIRECTORY will be put in\n# front of it.\n# The default directory is: docbook.\n# This tag requires that the tag GENERATE_DOCBOOK is set to YES.\n\nDOCBOOK_OUTPUT         = docbook\n\n# If the DOCBOOK_PROGRAMLISTING tag is set to YES, doxygen will include the\n# program listings (including syntax highlighting and cross-referencing\n# information) to the DOCBOOK output. Note that enabling this will significantly\n# increase the size of the DOCBOOK output.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_DOCBOOK is set to YES.\n\nDOCBOOK_PROGRAMLISTING = NO\n\n#---------------------------------------------------------------------------\n# Configuration options for the AutoGen Definitions output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_AUTOGEN_DEF tag is set to YES, doxygen will generate an\n# AutoGen Definitions (see http://autogen.sf.net) file that captures the\n# structure of the code including all documentation. Note that this feature is\n# still experimental and incomplete at the moment.\n# The default value is: NO.\n\nGENERATE_AUTOGEN_DEF   = NO\n\n#---------------------------------------------------------------------------\n# Configuration options related to the Perl module output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_PERLMOD tag is set to YES, doxygen will generate a Perl module\n# file that captures the structure of the code including all documentation.\n#\n# Note that this feature is still experimental and incomplete at the moment.\n# The default value is: NO.\n\nGENERATE_PERLMOD       = NO\n\n# If the PERLMOD_LATEX tag is set to YES, doxygen will generate the necessary\n# Makefile rules, Perl scripts and LaTeX code to be able to generate PDF and DVI\n# output from the Perl module output.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_PERLMOD is set to YES.\n\nPERLMOD_LATEX          = NO\n\n# If the PERLMOD_PRETTY tag is set to YES, the Perl module output will be nicely\n# formatted so it can be parsed by a human reader. This is useful if you want to\n# understand what is going on. On the other hand, if this tag is set to NO, the\n# size of the Perl module output will be much smaller and Perl will parse it\n# just the same.\n# The default value is: YES.\n# This tag requires that the tag GENERATE_PERLMOD is set to YES.\n\nPERLMOD_PRETTY         = YES\n\n# The names of the make variables in the generated doxyrules.make file are\n# prefixed with the string contained in PERLMOD_MAKEVAR_PREFIX. This is useful\n# so different doxyrules.make files included by the same Makefile don't\n# overwrite each other's variables.\n# This tag requires that the tag GENERATE_PERLMOD is set to YES.\n\nPERLMOD_MAKEVAR_PREFIX =\n\n#---------------------------------------------------------------------------\n# Configuration options related to the preprocessor\n#---------------------------------------------------------------------------\n\n# If the ENABLE_PREPROCESSING tag is set to YES, doxygen will evaluate all\n# C-preprocessor directives found in the sources and include files.\n# The default value is: YES.\n\nENABLE_PREPROCESSING   = YES\n\n# If the MACRO_EXPANSION tag is set to YES, doxygen will expand all macro names\n# in the source code. If set to NO, only conditional compilation will be\n# performed. Macro expansion can be done in a controlled way by setting\n# EXPAND_ONLY_PREDEF to YES.\n# The default value is: NO.\n# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.\n\nMACRO_EXPANSION        = NO\n\n# If the EXPAND_ONLY_PREDEF and MACRO_EXPANSION tags are both set to YES then\n# the macro expansion is limited to the macros specified with the PREDEFINED and\n# EXPAND_AS_DEFINED tags.\n# The default value is: NO.\n# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.\n\nEXPAND_ONLY_PREDEF     = NO\n\n# If the SEARCH_INCLUDES tag is set to YES, the include files in the\n# INCLUDE_PATH will be searched if a #include is found.\n# The default value is: YES.\n# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.\n\nSEARCH_INCLUDES        = YES\n\n# The INCLUDE_PATH tag can be used to specify one or more directories that\n# contain include files that are not input files but should be processed by the\n# preprocessor.\n# This tag requires that the tag SEARCH_INCLUDES is set to YES.\n\nINCLUDE_PATH           =\n\n# You can use the INCLUDE_FILE_PATTERNS tag to specify one or more wildcard\n# patterns (like *.h and *.hpp) to filter out the header-files in the\n# directories. If left blank, the patterns specified with FILE_PATTERNS will be\n# used.\n# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.\n\nINCLUDE_FILE_PATTERNS  =\n\n# The PREDEFINED tag can be used to specify one or more macro names that are\n# defined before the preprocessor is started (similar to the -D option of e.g.\n# gcc). The argument of the tag is a list of macros of the form: name or\n# name=definition (no spaces). If the definition and the \"=\" are omitted, \"=1\"\n# is assumed. To prevent a macro definition from being undefined via #undef or\n# recursively expanded use the := operator instead of the = operator.\n# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.\n\nPREDEFINED             =\n\n# If the MACRO_EXPANSION and EXPAND_ONLY_PREDEF tags are set to YES then this\n# tag can be used to specify a list of macro names that should be expanded. The\n# macro definition that is found in the sources will be used. Use the PREDEFINED\n# tag if you want to use a different macro definition that overrules the\n# definition found in the source code.\n# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.\n\nEXPAND_AS_DEFINED      =\n\n# If the SKIP_FUNCTION_MACROS tag is set to YES then doxygen's preprocessor will\n# remove all references to function-like macros that are alone on a line, have\n# an all uppercase name, and do not end with a semicolon. Such function macros\n# are typically used for boiler-plate code, and will confuse the parser if not\n# removed.\n# The default value is: YES.\n# This tag requires that the tag ENABLE_PREPROCESSING is set to YES.\n\nSKIP_FUNCTION_MACROS   = YES\n\n#---------------------------------------------------------------------------\n# Configuration options related to external references\n#---------------------------------------------------------------------------\n\n# The TAGFILES tag can be used to specify one or more tag files. For each tag\n# file the location of the external documentation should be added. The format of\n# a tag file without this location is as follows:\n# TAGFILES = file1 file2 ...\n# Adding location for the tag files is done as follows:\n# TAGFILES = file1=loc1 \"file2 = loc2\" ...\n# where loc1 and loc2 can be relative or absolute paths or URLs. See the\n# section \"Linking to external documentation\" for more information about the use\n# of tag files.\n# Note: Each tag file must have a unique name (where the name does NOT include\n# the path). If a tag file is not located in the directory in which doxygen is\n# run, you must also specify the path to the tagfile here.\n\nTAGFILES               =\n\n# When a file name is specified after GENERATE_TAGFILE, doxygen will create a\n# tag file that is based on the input files it reads. See section \"Linking to\n# external documentation\" for more information about the usage of tag files.\n\nGENERATE_TAGFILE       =\n\n# If the ALLEXTERNALS tag is set to YES, all external class will be listed in\n# the class index. If set to NO, only the inherited external classes will be\n# listed.\n# The default value is: NO.\n\nALLEXTERNALS           = NO\n\n# If the EXTERNAL_GROUPS tag is set to YES, all external groups will be listed\n# in the modules index. If set to NO, only the current project's groups will be\n# listed.\n# The default value is: YES.\n\nEXTERNAL_GROUPS        = YES\n\n# If the EXTERNAL_PAGES tag is set to YES, all external pages will be listed in\n# the related pages index. If set to NO, only the current project's pages will\n# be listed.\n# The default value is: YES.\n\nEXTERNAL_PAGES         = YES\n\n# The PERL_PATH should be the absolute path and name of the perl script\n# interpreter (i.e. the result of 'which perl').\n# The default file (with absolute path) is: /usr/bin/perl.\n\nPERL_PATH              = /usr/bin/perl\n\n#---------------------------------------------------------------------------\n# Configuration options related to the dot tool\n#---------------------------------------------------------------------------\n\n# If the CLASS_DIAGRAMS tag is set to YES, doxygen will generate a class diagram\n# (in HTML and LaTeX) for classes with base or super classes. Setting the tag to\n# NO turns the diagrams off. Note that this option also works with HAVE_DOT\n# disabled, but it is recommended to install and use dot, since it yields more\n# powerful graphs.\n# The default value is: YES.\n\nCLASS_DIAGRAMS         = YES\n\n# You can define message sequence charts within doxygen comments using the \\msc\n# command. Doxygen will then run the mscgen tool (see:\n# http://www.mcternan.me.uk/mscgen/)) to produce the chart and insert it in the\n# documentation. The MSCGEN_PATH tag allows you to specify the directory where\n# the mscgen tool resides. If left empty the tool is assumed to be found in the\n# default search path.\n\nMSCGEN_PATH            =\n\n# You can include diagrams made with dia in doxygen documentation. Doxygen will\n# then run dia to produce the diagram and insert it in the documentation. The\n# DIA_PATH tag allows you to specify the directory where the dia binary resides.\n# If left empty dia is assumed to be found in the default search path.\n\nDIA_PATH               =\n\n# If set to YES the inheritance and collaboration graphs will hide inheritance\n# and usage relations if the target is undocumented or is not a class.\n# The default value is: YES.\n\nHIDE_UNDOC_RELATIONS   = YES\n\n# If you set the HAVE_DOT tag to YES then doxygen will assume the dot tool is\n# available from the path. This tool is part of Graphviz (see:\n# http://www.graphviz.org/), a graph visualization toolkit from AT&T and Lucent\n# Bell Labs. The other options in this section have no effect if this option is\n# set to NO\n# The default value is: NO.\n\nHAVE_DOT               = NO\n\n# The DOT_NUM_THREADS specifies the number of dot invocations doxygen is allowed\n# to run in parallel. When set to 0 doxygen will base this on the number of\n# processors available in the system. You can set it explicitly to a value\n# larger than 0 to get control over the balance between CPU load and processing\n# speed.\n# Minimum value: 0, maximum value: 32, default value: 0.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nDOT_NUM_THREADS        = 0\n\n# When you want a differently looking font in the dot files that doxygen\n# generates you can specify the font name using DOT_FONTNAME. You need to make\n# sure dot is able to find the font, which can be done by putting it in a\n# standard location or by setting the DOTFONTPATH environment variable or by\n# setting DOT_FONTPATH to the directory containing the font.\n# The default value is: Helvetica.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nDOT_FONTNAME           = Helvetica\n\n# The DOT_FONTSIZE tag can be used to set the size (in points) of the font of\n# dot graphs.\n# Minimum value: 4, maximum value: 24, default value: 10.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nDOT_FONTSIZE           = 10\n\n# By default doxygen will tell dot to use the default font as specified with\n# DOT_FONTNAME. If you specify a different font using DOT_FONTNAME you can set\n# the path where dot can find it using this tag.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nDOT_FONTPATH           =\n\n# If the CLASS_GRAPH tag is set to YES then doxygen will generate a graph for\n# each documented class showing the direct and indirect inheritance relations.\n# Setting this tag to YES will force the CLASS_DIAGRAMS tag to NO.\n# The default value is: YES.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nCLASS_GRAPH            = YES\n\n# If the COLLABORATION_GRAPH tag is set to YES then doxygen will generate a\n# graph for each documented class showing the direct and indirect implementation\n# dependencies (inheritance, containment, and class references variables) of the\n# class with other documented classes.\n# The default value is: YES.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nCOLLABORATION_GRAPH    = YES\n\n# If the GROUP_GRAPHS tag is set to YES then doxygen will generate a graph for\n# groups, showing the direct groups dependencies.\n# The default value is: YES.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nGROUP_GRAPHS           = YES\n\n# If the UML_LOOK tag is set to YES, doxygen will generate inheritance and\n# collaboration diagrams in a style similar to the OMG's Unified Modeling\n# Language.\n# The default value is: NO.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nUML_LOOK               = NO\n\n# If the UML_LOOK tag is enabled, the fields and methods are shown inside the\n# class node. If there are many fields or methods and many nodes the graph may\n# become too big to be useful. The UML_LIMIT_NUM_FIELDS threshold limits the\n# number of items for each type to make the size more manageable. Set this to 0\n# for no limit. Note that the threshold may be exceeded by 50% before the limit\n# is enforced. So when you set the threshold to 10, up to 15 fields may appear,\n# but if the number exceeds 15, the total amount of fields shown is limited to\n# 10.\n# Minimum value: 0, maximum value: 100, default value: 10.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nUML_LIMIT_NUM_FIELDS   = 10\n\n# If the TEMPLATE_RELATIONS tag is set to YES then the inheritance and\n# collaboration graphs will show the relations between templates and their\n# instances.\n# The default value is: NO.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nTEMPLATE_RELATIONS     = NO\n\n# If the INCLUDE_GRAPH, ENABLE_PREPROCESSING and SEARCH_INCLUDES tags are set to\n# YES then doxygen will generate a graph for each documented file showing the\n# direct and indirect include dependencies of the file with other documented\n# files.\n# The default value is: YES.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nINCLUDE_GRAPH          = YES\n\n# If the INCLUDED_BY_GRAPH, ENABLE_PREPROCESSING and SEARCH_INCLUDES tags are\n# set to YES then doxygen will generate a graph for each documented file showing\n# the direct and indirect include dependencies of the file with other documented\n# files.\n# The default value is: YES.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nINCLUDED_BY_GRAPH      = YES\n\n# If the CALL_GRAPH tag is set to YES then doxygen will generate a call\n# dependency graph for every global function or class method.\n#\n# Note that enabling this option will significantly increase the time of a run.\n# So in most cases it will be better to enable call graphs for selected\n# functions only using the \\callgraph command.\n# The default value is: NO.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nCALL_GRAPH             = NO\n\n# If the CALLER_GRAPH tag is set to YES then doxygen will generate a caller\n# dependency graph for every global function or class method.\n#\n# Note that enabling this option will significantly increase the time of a run.\n# So in most cases it will be better to enable caller graphs for selected\n# functions only using the \\callergraph command.\n# The default value is: NO.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nCALLER_GRAPH           = NO\n\n# If the GRAPHICAL_HIERARCHY tag is set to YES then doxygen will graphical\n# hierarchy of all classes instead of a textual one.\n# The default value is: YES.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nGRAPHICAL_HIERARCHY    = YES\n\n# If the DIRECTORY_GRAPH tag is set to YES then doxygen will show the\n# dependencies a directory has on other directories in a graphical way. The\n# dependency relations are determined by the #include relations between the\n# files in the directories.\n# The default value is: YES.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nDIRECTORY_GRAPH        = YES\n\n# The DOT_IMAGE_FORMAT tag can be used to set the image format of the images\n# generated by dot.\n# Note: If you choose svg you need to set HTML_FILE_EXTENSION to xhtml in order\n# to make the SVG files visible in IE 9+ (other browsers do not have this\n# requirement).\n# Possible values are: png, jpg, gif and svg.\n# The default value is: png.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nDOT_IMAGE_FORMAT       = png\n\n# If DOT_IMAGE_FORMAT is set to svg, then this option can be set to YES to\n# enable generation of interactive SVG images that allow zooming and panning.\n#\n# Note that this requires a modern browser other than Internet Explorer. Tested\n# and working are Firefox, Chrome, Safari, and Opera.\n# Note: For IE 9+ you need to set HTML_FILE_EXTENSION to xhtml in order to make\n# the SVG files visible. Older versions of IE do not have SVG support.\n# The default value is: NO.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nINTERACTIVE_SVG        = NO\n\n# The DOT_PATH tag can be used to specify the path where the dot tool can be\n# found. If left blank, it is assumed the dot tool can be found in the path.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nDOT_PATH               =\n\n# The DOTFILE_DIRS tag can be used to specify one or more directories that\n# contain dot files that are included in the documentation (see the \\dotfile\n# command).\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nDOTFILE_DIRS           =\n\n# The MSCFILE_DIRS tag can be used to specify one or more directories that\n# contain msc files that are included in the documentation (see the \\mscfile\n# command).\n\nMSCFILE_DIRS           =\n\n# The DIAFILE_DIRS tag can be used to specify one or more directories that\n# contain dia files that are included in the documentation (see the \\diafile\n# command).\n\nDIAFILE_DIRS           =\n\n# When using plantuml, the PLANTUML_JAR_PATH tag should be used to specify the\n# path where java can find the plantuml.jar file. If left blank, it is assumed\n# PlantUML is not used or called during a preprocessing step. Doxygen will\n# generate a warning when it encounters a \\startuml command in this case and\n# will not generate output for the diagram.\n\nPLANTUML_JAR_PATH      =\n\n# When using plantuml, the specified paths are searched for files specified by\n# the !include statement in a plantuml block.\n\nPLANTUML_INCLUDE_PATH  =\n\n# The DOT_GRAPH_MAX_NODES tag can be used to set the maximum number of nodes\n# that will be shown in the graph. If the number of nodes in a graph becomes\n# larger than this value, doxygen will truncate the graph, which is visualized\n# by representing a node as a red box. Note that doxygen if the number of direct\n# children of the root node in a graph is already larger than\n# DOT_GRAPH_MAX_NODES then the graph will not be shown at all. Also note that\n# the size of a graph can be further restricted by MAX_DOT_GRAPH_DEPTH.\n# Minimum value: 0, maximum value: 10000, default value: 50.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nDOT_GRAPH_MAX_NODES    = 50\n\n# The MAX_DOT_GRAPH_DEPTH tag can be used to set the maximum depth of the graphs\n# generated by dot. A depth value of 3 means that only nodes reachable from the\n# root by following a path via at most 3 edges will be shown. Nodes that lay\n# further from the root node will be omitted. Note that setting this option to 1\n# or 2 may greatly reduce the computation time needed for large code bases. Also\n# note that the size of a graph can be further restricted by\n# DOT_GRAPH_MAX_NODES. Using a depth of 0 means no depth restriction.\n# Minimum value: 0, maximum value: 1000, default value: 0.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nMAX_DOT_GRAPH_DEPTH    = 0\n\n# Set the DOT_TRANSPARENT tag to YES to generate images with a transparent\n# background. This is disabled by default, because dot on Windows does not seem\n# to support this out of the box.\n#\n# Warning: Depending on the platform used, enabling this option may lead to\n# badly anti-aliased labels on the edges of a graph (i.e. they become hard to\n# read).\n# The default value is: NO.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nDOT_TRANSPARENT        = NO\n\n# Set the DOT_MULTI_TARGETS tag to YES to allow dot to generate multiple output\n# files in one run (i.e. multiple -o and -T options on the command line). This\n# makes dot run faster, but since only newer versions of dot (>1.8.10) support\n# this, this feature is disabled by default.\n# The default value is: NO.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nDOT_MULTI_TARGETS      = NO\n\n# If the GENERATE_LEGEND tag is set to YES doxygen will generate a legend page\n# explaining the meaning of the various boxes and arrows in the dot generated\n# graphs.\n# The default value is: YES.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nGENERATE_LEGEND        = YES\n\n# If the DOT_CLEANUP tag is set to YES, doxygen will remove the intermediate dot\n# files that are used to generate the various graphs.\n# The default value is: YES.\n# This tag requires that the tag HAVE_DOT is set to YES.\n\nDOT_CLEANUP            = YES\n"
  },
  {
    "path": "Emakefile.in",
    "content": "% behaviors first - otherwise there are compile warnings\n{[\"src/cloud_beh.erl\",\n  \"src/crdt/types/crdt_beh.erl\",\n  \"src/db_backend_beh.erl\",\n  \"src/gen_component.erl\",\n  \"src/gossip_beh.erl\",\n  \"src/gossip_load_beh.erl\",\n  \"src/lb_active_beh.erl\",\n  \"src/lb_psv_beh.erl\",\n  \"src/pdb_beh.erl\",\n  \"src/rm_beh.erl\",\n  \"src/rrepair/hfs_beh.erl\",\n  \"src/rt_beh.erl\",\n  \"src/slide_beh.erl\",\n  \"src/transactions/tx_op_beh.erl\"],\n [debug_info,\n  inline_list_funcs,\n  warn_exported_vars,\n  warn_missing_spec,\n  warn_untyped_record,\n  warn_unused_import,\n  {outdir, \"ebin\"},\n  {i, \"include\"},\n  {i, \"contrib/yaws/include\"},\n  {i, \"contrib/log4erl/include\"}\n  @EMAKEFILEDEFINES@\n  @YAWS_OPTIONS@\n ]}.\n% now the source and all of its sub-directories:\n{[\"src/*\",\n  \"src/comm_layer/*\",\n  \"src/cp/*\",\n  \"src/crdt/*\",\n  \"src/crdt/types/*\",\n  \"src/json/*\",\n  \"src/paxos/*\",\n  \"src/rbr/*\",\n  \"src/rrepair/*\",\n  \"src/simulation/*\",\n  \"src/time/*\",\n  \"src/transactions/*\",\n  \"src/tx/*\"],\n [debug_info,\n  inline_list_funcs,\n  warn_exported_vars,\n  warn_missing_spec,\n  warn_untyped_record,\n  warn_unused_import,\n  {outdir, \"ebin\"},\n  {i, \"ebin\"},\n  {i, \"include\"},\n  {i, \"contrib/yaws/include\"},\n  {i, \"contrib/log4erl/include\"}\n  @EMAKEFILEDEFINES@\n  @YAWS_OPTIONS@\n ]}.\n% unit tests:\n{\"test/*\",\n [debug_info,\n  inline_list_funcs,\n  nowarn_export_all,\n  warn_exported_vars,\n  warn_missing_spec,\n  warn_untyped_record,\n  warn_unused_import,\n  {outdir, \"test\"},\n  {i, \"include\"}\n  @EMAKEFILEDEFINES@\n ]}.\n% contributing libraries:\n{\"contrib/log4erl/src/*\",\n [debug_info,\n  inline_list_funcs,\n  nowarn_obsolete_guard,\n  nowarn_unused_function,\n  nowarn_unused_vars,\n  {outdir, \"contrib/log4erl/ebin/\"},\n  {i, \"contrib/log4erl/include\"}\n ]}.\n{\"contrib/dotto/src/*\",\n [debug_info,\n  inline_list_funcs,\n  nowarn_obsolete_guard,\n  nowarn_unused_function,\n  nowarn_unused_vars,\n  {outdir, \"contrib/dotto/ebin/\"}\n ]}.\n{\"contrib/yaws/src/*\",\n [debug_info,\n  inline_list_funcs,\n  nowarn_obsolete_guard,\n  nowarn_unused_function,\n  nowarn_unused_vars,\n  {outdir, \"contrib/yaws/ebin/\"},\n  {i, \"contrib/yaws/include\"}\n  @YAWS_OPTIONS@\n ]}.\n"
  },
  {
    "path": "LICENSE",
    "content": "\n                                 Apache License\n                           Version 2.0, January 2004\n                        http://www.apache.org/licenses/\n\n   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n   1. Definitions.\n\n      \"License\" shall mean the terms and conditions for use, reproduction,\n      and distribution as defined by Sections 1 through 9 of this document.\n\n      \"Licensor\" shall mean the copyright owner or entity authorized by\n      the copyright owner that is granting the License.\n\n      \"Legal Entity\" shall mean the union of the acting entity and all\n      other entities that control, are controlled by, or are under common\n      control with that entity. For the purposes of this definition,\n      \"control\" means (i) the power, direct or indirect, to cause the\n      direction or management of such entity, whether by contract or\n      otherwise, or (ii) ownership of fifty percent (50%) or more of the\n      outstanding shares, or (iii) beneficial ownership of such entity.\n\n      \"You\" (or \"Your\") shall mean an individual or Legal Entity\n      exercising permissions granted by this License.\n\n      \"Source\" form shall mean the preferred form for making modifications,\n      including but not limited to software source code, documentation\n      source, and configuration files.\n\n      \"Object\" form shall mean any form resulting from mechanical\n      transformation or translation of a Source form, including but\n      not limited to compiled object code, generated documentation,\n      and conversions to other media types.\n\n      \"Work\" shall mean the work of authorship, whether in Source or\n      Object form, made available under the License, as indicated by a\n      copyright notice that is included in or attached to the work\n      (an example is provided in the Appendix below).\n\n      \"Derivative Works\" shall mean any work, whether in Source or Object\n      form, that is based on (or derived from) the Work and for which the\n      editorial revisions, annotations, elaborations, or other modifications\n      represent, as a whole, an original work of authorship. For the purposes\n      of this License, Derivative Works shall not include works that remain\n      separable from, or merely link (or bind by name) to the interfaces of,\n      the Work and Derivative Works thereof.\n\n      \"Contribution\" shall mean any work of authorship, including\n      the original version of the Work and any modifications or additions\n      to that Work or Derivative Works thereof, that is intentionally\n      submitted to Licensor for inclusion in the Work by the copyright owner\n      or by an individual or Legal Entity authorized to submit on behalf of\n      the copyright owner. For the purposes of this definition, \"submitted\"\n      means any form of electronic, verbal, or written communication sent\n      to the Licensor or its representatives, including but not limited to\n      communication on electronic mailing lists, source code control systems,\n      and issue tracking systems that are managed by, or on behalf of, the\n      Licensor for the purpose of discussing and improving the Work, but\n      excluding communication that is conspicuously marked or otherwise\n      designated in writing by the copyright owner as \"Not a Contribution.\"\n\n      \"Contributor\" shall mean Licensor and any individual or Legal Entity\n      on behalf of whom a Contribution has been received by Licensor and\n      subsequently incorporated within the Work.\n\n   2. Grant of Copyright License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      copyright license to reproduce, prepare Derivative Works of,\n      publicly display, publicly perform, sublicense, and distribute the\n      Work and such Derivative Works in Source or Object form.\n\n   3. Grant of Patent License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      (except as stated in this section) patent license to make, have made,\n      use, offer to sell, sell, import, and otherwise transfer the Work,\n      where such license applies only to those patent claims licensable\n      by such Contributor that are necessarily infringed by their\n      Contribution(s) alone or by combination of their Contribution(s)\n      with the Work to which such Contribution(s) was submitted. If You\n      institute patent litigation against any entity (including a\n      cross-claim or counterclaim in a lawsuit) alleging that the Work\n      or a Contribution incorporated within the Work constitutes direct\n      or contributory patent infringement, then any patent licenses\n      granted to You under this License for that Work shall terminate\n      as of the date such litigation is filed.\n\n   4. Redistribution. You may reproduce and distribute copies of the\n      Work or Derivative Works thereof in any medium, with or without\n      modifications, and in Source or Object form, provided that You\n      meet the following conditions:\n\n      (a) You must give any other recipients of the Work or\n          Derivative Works a copy of this License; and\n\n      (b) You must cause any modified files to carry prominent notices\n          stating that You changed the files; and\n\n      (c) You must retain, in the Source form of any Derivative Works\n          that You distribute, all copyright, patent, trademark, and\n          attribution notices from the Source form of the Work,\n          excluding those notices that do not pertain to any part of\n          the Derivative Works; and\n\n      (d) If the Work includes a \"NOTICE\" text file as part of its\n          distribution, then any Derivative Works that You distribute must\n          include a readable copy of the attribution notices contained\n          within such NOTICE file, excluding those notices that do not\n          pertain to any part of the Derivative Works, in at least one\n          of the following places: within a NOTICE text file distributed\n          as part of the Derivative Works; within the Source form or\n          documentation, if provided along with the Derivative Works; or,\n          within a display generated by the Derivative Works, if and\n          wherever such third-party notices normally appear. The contents\n          of the NOTICE file are for informational purposes only and\n          do not modify the License. You may add Your own attribution\n          notices within Derivative Works that You distribute, alongside\n          or as an addendum to the NOTICE text from the Work, provided\n          that such additional attribution notices cannot be construed\n          as modifying the License.\n\n      You may add Your own copyright statement to Your modifications and\n      may provide additional or different license terms and conditions\n      for use, reproduction, or distribution of Your modifications, or\n      for any such Derivative Works as a whole, provided Your use,\n      reproduction, and distribution of the Work otherwise complies with\n      the conditions stated in this License.\n\n   5. Submission of Contributions. Unless You explicitly state otherwise,\n      any Contribution intentionally submitted for inclusion in the Work\n      by You to the Licensor shall be under the terms and conditions of\n      this License, without any additional terms or conditions.\n      Notwithstanding the above, nothing herein shall supersede or modify\n      the terms of any separate license agreement you may have executed\n      with Licensor regarding such Contributions.\n\n   6. Trademarks. This License does not grant permission to use the trade\n      names, trademarks, service marks, or product names of the Licensor,\n      except as required for reasonable and customary use in describing the\n      origin of the Work and reproducing the content of the NOTICE file.\n\n   7. Disclaimer of Warranty. Unless required by applicable law or\n      agreed to in writing, Licensor provides the Work (and each\n      Contributor provides its Contributions) on an \"AS IS\" BASIS,\n      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n      implied, including, without limitation, any warranties or conditions\n      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n      PARTICULAR PURPOSE. You are solely responsible for determining the\n      appropriateness of using or redistributing the Work and assume any\n      risks associated with Your exercise of permissions under this License.\n\n   8. Limitation of Liability. In no event and under no legal theory,\n      whether in tort (including negligence), contract, or otherwise,\n      unless required by applicable law (such as deliberate and grossly\n      negligent acts) or agreed to in writing, shall any Contributor be\n      liable to You for damages, including any direct, indirect, special,\n      incidental, or consequential damages of any character arising as a\n      result of this License or out of the use or inability to use the\n      Work (including but not limited to damages for loss of goodwill,\n      work stoppage, computer failure or malfunction, or any and all\n      other commercial damages or losses), even if such Contributor\n      has been advised of the possibility of such damages.\n\n   9. Accepting Warranty or Additional Liability. While redistributing\n      the Work or Derivative Works thereof, You may choose to offer,\n      and charge a fee for, acceptance of support, warranty, indemnity,\n      or other liability obligations and/or rights consistent with this\n      License. However, in accepting such obligations, You may act only\n      on Your own behalf and on Your sole responsibility, not on behalf\n      of any other Contributor, and only if You agree to indemnify,\n      defend, and hold each Contributor harmless for any liability\n      incurred by, or claims asserted against, such Contributor by reason\n      of your accepting any such warranty or additional liability.\n\n   END OF TERMS AND CONDITIONS\n\n   APPENDIX: How to apply the Apache License to your work.\n\n      To apply the Apache License to your work, attach the following\n      boilerplate notice, with the fields enclosed by brackets \"[]\"\n      replaced with your own identifying information. (Don't include\n      the brackets!)  The text should be enclosed in the appropriate\n      comment syntax for the file format. We also recommend that a\n      file or class name and description of purpose be included on the\n      same \"printed page\" as the copyright notice for easier\n      identification within third-party archives.\n\n   Copyright [yyyy] [name of copyright owner]\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\n"
  },
  {
    "path": "Makefile.in",
    "content": "\n# Copyright 2007-2018 Zuse Institute Berlin\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nERLC = @ERLC@\nERL = @ERL@\nDIALYZER = @ERLANG_ROOT_DIR@/bin/dialyzer\nERL_LIB_DIR = @ERLANG_LIB_DIR@\nCD = cd\nMAKE = make\nPYTHON2=@PYTHON2@\nPYTHON3=@PYTHON3@\nPYTHON3_2TO3=@PYTHON3_2TO3@\nVERSION=@PACKAGE_VERSION@\nOTP_ERLANG_VERSION=1.6.1\nCOMMONS_CLI_VERSION=1.2\n\nSESSIONKEY:=$(shell cat /dev/urandom | env LC_CTYPE=C tr -dc 'a-z' | fold -w 32 | head -n 1)\n\n.PHONY: all bench bench-ssl bench-ssl-strict bench-ssl-strict-bad compile test test-vts cpp cpp-clean cpp-test java java-test java-bench java-* datanucleus datanucleus-test ycsb coverity python python-compile python-clean python-test python-bench python-doc-html python-doc-pdf python3 python3-convert python3-compile python3-clean python3-test python3-bench ruby-test interop-test dist-clean clean erlang-clean tags tags-clean docs dialyzer dialyzer-col rrd-init install install-all install-erlang install-java install-docs install-java-doc install-python-doc-html install-python-doc-pdf install-ruby install-python install-python3 test-clean nifs %_SUITE\n\nAPP_NAME = @PACKAGE_TARNAME@\nPACKAGE_TARNAME = @PACKAGE_TARNAME@\n\nprefix = @prefix@\nexec_prefix = @exec_prefix@\ndatarootdir = @datarootdir@\nbindir = @bindir@\nsysconfdir = @sysconfdir@\nlocalstatedir = @localstatedir@\n\n# /bin\nBINDIR = $(DESTDIR)@bindir@\n# /sbin\nSBINDIR = $(DESTDIR)@sbindir@\n# /usr/include\nINCLUDEDIR = $(DESTDIR)@includedir@\n# /usr/lib[64]\nLIBDIR = $(DESTDIR)@libdir@\n# /etc/scalaris/\nETCDIR = $(DESTDIR)@sysconfdir@/scalaris\n# /etc/init.d/\nINITDIR = $(DESTDIR)@sysconfdir@/init.d\n# /etc/conf.d\nSYSTEMD_CONFDIR = $(DESTDIR)@sysconfdir@/conf.d\n# /usr/lib/systemd/system\nSYSTEMD_UNITDIR = $(DESTDIR)@SYSTEMD_UNITDIR@\n# /lib/scalaris/\nSCALARISDIR = $(DESTDIR)${prefix}/lib/scalaris\n# /lib/scalaris/ebin\nBEAMDIR = $(SCALARISDIR)/ebin\n# /lib/scalaris/docroot\nDOCROOTDIR=$(SCALARISDIR)/docroot\n# /share/doc/scalaris\nDOCDIR=$(DESTDIR)@docdir@\n# /share/java\nJAVADIR=$(DESTDIR)@datarootdir@/java\n# /var/log/scalaris\nLOGDIR=$(DESTDIR)@localstatedir@/log/scalaris\n# /var/run\nRUNDIR=$(DESTDIR)@localstatedir@/run\n# /usr/lib/ruby/site_ruby/1.8\nRUBYSITELIBDIR=$(DESTDIR)@RUBYSITELIBDIR@\n# /usr/lib/python2.7/site-packages\nPYTHON2SITELIBDIR=$(DESTDIR)@PYTHON2SITELIBDIR@\n# /usr/lib/python3.1/site-packages\nPYTHON3SITELIBDIR=$(DESTDIR)@PYTHON3SITELIBDIR@\n\n# paths of source files for make doc\nSOURCEPATHS_ERL=src/* src/*/* include/* test/* contrib/log4erl/src/* contrib/yaws/src/*\n\n# paths to contrib ebins that will be added to erlangs code path when\n# executing a test related make target\nCONTRIB_EBIN_PATHS = `pwd`/contrib/yaws/ebin `pwd`/contrib/log4erl/ebin `pwd`/contrib/dotto/ebin\n\n# libraries needed to build a plt suitable for the dialyzer target\n# (only add them if they are available (via wildcard call which\n# results in nothing when no match exists))\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/common_test-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/compiler-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/crypto-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/debugger-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/edoc-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/erlang_js-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/erts-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/et-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/gs-*/ebin)\n# currently not compatible:\n# DIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/hanoidb-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/hipe-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/inets-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/kernel-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/mnesia-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/observer-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/os_mon-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/public_key-*/ebin/)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/runtime_tools-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/sasl-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/snmp-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/ssh-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/ssl-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/stdlib-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/syntax_tools-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/test_server-*/ebin/)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/toke-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/tools-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/webtool-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/wx-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard $(ERL_LIB_DIR)/xmerl-*/ebin)\nDIALYZER_PLT_LIBS += $(wildcard contrib/log4erl/ebin)\nDIALYZER_PLT_LIBS += $(wildcard contrib/yaws/ebin)\nDIALYZER_PLT_LIBS += $(wildcard contrib/dotto/ebin)\nDIALYZER_PLT_LIBS += $(wildcard @BITCASK_LIBS@/ebin)\n\n#DIALYZER_PLT_LIBS=$(DIALYZER_PLT_LIBS) $(ERL_LIB_DIR)/gs-*/contribs/ebin $(ERL_LIB_DIR)/wx-*/examples/*\nDIALYZER_FLAGS ?=\n\nSCALARIS_CT_PREFIX ?= ct\nSCALARIS_CT_NAME = $(SCALARIS_CT_PREFIX)@`hostname -s`\n# for java-test, java-bench and datanucleus-test:\nSCALARIS_UNITTEST_PORT ?= 14194\nSCALARIS_UNITTEST_YAWS_PORT ?= 8000\nSCALARIS_JTEST_NAME ?= jtest_boot\n\nNIFS_SRC_CPP=$(wildcard src/nifs/*.cpp)\nNIFS_SO=$(subst src/nifs,priv,$(NIFS_SRC_CPP:.cpp=.so))\n\nall: Makefile nifs compile tags\n#all: compile dialyzer java test docs\n\n#cp priv/xbin/*beam bin/\ncompile: .Emakefile contrib/log4erl/src/log4erl_parser.erl contrib/log4erl/src/log4erl_lex.erl\n\t@$(ERL) -pa contrib/yaws -pa ebin -noinput +B -eval 'case make:all() of up_to_date -> halt(0); error -> halt(1) end.'\n\n.Emakefile: Emakefile\n\t$(MAKE) erlang-clean\n\ttouch $@\n\ncontrib/log4erl/src/log4erl_parser.erl: contrib/log4erl/src/log4erl_parser.yrl\n\tcd contrib/log4erl/src && $(ERLC) log4erl_parser.yrl\n\ncontrib/log4erl/src/log4erl_lex.erl: contrib/log4erl/src/log4erl_lex.xrl\n\tcd contrib/log4erl/src && $(ERL) -noinput -noshell -s leex file log4erl_lex.xrl -s init stop\n\ntest-all: all_TESTS java-test python-test ruby-test interop-test datanucleus-test\n\ntest: default_TESTS\n\ntest-skipped: skipped_TESTS\n\ntest-repeated: repeated_TESTS\n\ntest-ssl: ssl_TESTS\n\nSUITES := $(patsubst test/%.erl,%,$(wildcard test/*_SUITE.erl))\n\n${SUITES}: compile\n# use non standard ports to run a single SUITE so it can be executed\n# in parallel to a full test suite run or a locally running default\n# scalaris session.\n\tSCALARIS_UNITTEST_PORT=14193 SCALARIS_UNITTEST_YAWS_PORT=7999 @ERL@ -noshell -noinput @ERLANG_ERLANGJS_FLAGS@ @ERLANG_HANOIDB_FLAGS@ @ERLANG_TOKE_FLAGS@ @ERLANG_BITCASK_FLAGS@ -pa `pwd`/ebin `pwd`/test $(CONTRIB_EBIN_PATHS) -eval \"runner:run_suite(\\\"test/$@\\\"), halt(0)\" | tee ct_$@.log ; \\\n\t! grep -E \"TEST COMPLETE, [0-9]+ ok, [1-9]+ failed\" ct_$@.log\n\n# define targets for single test cases of the form <test-suite>:<test-case>:\ndefine TEST_CASE_TARGET\n# test group\n$(1)\\:%_GROUP: compile\n# use non standard ports to run a single SUITE's test case so it can be executed\n# in parallel to a full test suite run or a locally running default\n# scalaris session.\n\tSCALARIS_UNITTEST_PORT=14193 SCALARIS_UNITTEST_YAWS_PORT=7999 @ERL@ -noshell -noinput -pa `pwd`/ebin `pwd`/test $(CONTRIB_EBIN_PATHS) -eval \"runner:run_suite_group(\\\"$(1)\\\", \\\"$$*\\\"), halt(0)\" | tee ct_$(1).$$*_GROUP.log ; \\\n\t! grep -E \"TEST COMPLETE, [0-9]+ ok, [1-9]+ failed\" ct_$(1).$$*_GROUP.log\n\n# single test case\n$(1)\\:%: compile\n\tSCALARIS_UNITTEST_PORT=14193 SCALARIS_UNITTEST_YAWS_PORT=7999 @ERL@ -noshell -noinput -pa `pwd`/ebin `pwd`/test $(CONTRIB_EBIN_PATHS)  -eval \"runner:run_suite_case(\\\"$(1)\\\", \\\"$$*\\\"), halt(0)\" | tee ct_$(1).$$*.log ; \\\n\t! grep -E \"TEST COMPLETE, [0-9]+ ok, [1-9]+ failed\" ct_$(1).$$*.log\nendef\n$(foreach SUITE,$(SUITES),$(eval $(call TEST_CASE_TARGET,$(SUITE))))\n\n%_SUITE-with-cover: compile\n\t@RUN_TEST@ $(UNITTESTARGS) -name $(SCALARIS_CT_NAME)_$@ @ERLANG_ERLANGJS_FLAGS@ @ERLANG_HANOIDB_FLAGS@ @ERLANG_TOKE_FLAGS@ @ERLANG_BITCASK_FLAGS@ -pa `pwd`/ebin `pwd`/test `pwd`-$(CONTRIB_EBIN_PATHS) -suite $*_SUITE -cover test/scalaris.coverspec -ct_hooks scalaris_cth -noshell | tee ct_$@.log ; \\\n\t! grep -E \"TEST COMPLETE, [0-9]+ ok, [1-9]+ failed\" ct_$@.log\n\n%_TESTS: compile\n\t@ERL@ -noshell -noinput @ERLANG_ERLANGJS_FLAGS@ @ERLANG_HANOIDB_FLAGS@ @ERLANG_TOKE_FLAGS@ @ERLANG_BITCASK_FLAGS@ -pa `pwd`/ebin `pwd`/test $(CONTRIB_EBIN_PATHS) -eval \"runner:run_spec(\\\"test/$@.cfg\\\"), halt(0)\" | tee ct_$@.log ; \\\n\t! grep -E \"TEST COMPLETE, [0-9]+ ok, [1-9]+ failed\" ct_$@.log\n\ntest-vts: compile\n\t@RUN_TEST@ @ERLANG_ERLANGJS_FLAGS@ @ERLANG_HANOIDB_FLAGS@ @ERLANG_TOKE_FLAGS@ @ERLANG_BITCASK_FLAGS@ -pa `pwd`/ebin `pwd`/test $(CONTRIB_EBIN_PATHS) -dir . -cover test/scalaris.coverspec -vts -browser @BROWSER@ -ct_hooks scalaris_cth\n\ntest-clean:\n\t-rm -rf ct_run.*@* ct.log ct-skipped.log ct_*_TESTS.log ct_*_SUITE.log ct_*_SUITE.*.log index.html all_runs.html variables-*@* ct_log_cache ct_default.css \\\n            result_increment_10_1000.txt result_increment_1_10000.txt \\\n            result_read_10_10000.txt result_read_1_100000.txt \\\n            data/ct*@*\n\nbench: compile\n\t./bin/setup-ring.sh --ring-size 4 --session-key $(SESSIONKEY) start\n\t./bin/jsonclient -p $(SCALARIS_UNITTEST_YAWS_PORT) benchmark\n\t./bin/setup-ring.sh --ring-size 4 --session-key $(SESSIONKEY) stop\n\nbench-ssl: compile\n\t./bin/setup-ring.sh --ring-size 4 --session-key $(SESSIONKEY) --ssl start\n\t./bin/jsonclient -s -p $(SCALARIS_UNITTEST_YAWS_PORT) benchmark\n\t./bin/setup-ring.sh --ring-size 4 --session-key $(SESSIONKEY) --ssl stop\n\nbench-ssl-strict: compile\n\t./bin/setup-ring.sh --ring-size 4 --session-key $(SESSIONKEY) --ssl-strict start\n\t./bin/jsonclient -s -p $(SCALARIS_UNITTEST_YAWS_PORT) benchmark\n\t./bin/setup-ring.sh --ring-size 4 --session-key $(SESSIONKEY) --ssl-strict stop\n\nbench-ssl-strict-bad: compile\n\t( ./bin/setup-ring-for-benchmarks.sh --ssl-strict --bad-nodes 2 --ring-size 4 --node-prefix ebench_node start && \\\n\techo \"running bench:increment(10, 500)...\" && \\\n\t$(ERL) -setcookie \"chocolate chip cookie\" -name bench_ctl -pa ebin/ -noinput -eval \"rpc:call('ebench_node1@`hostname -f`', bench, increment, [10, 500]), halt(0).\" && \\\n\techo \"running bench:quorum_read(10, 5000)...\" && \\\n\t$(ERL) -setcookie \"chocolate chip cookie\" -name bench_ctl -pa ebin/ -noinput -eval \"rpc:call('ebench_node1@`hostname -f`', bench, quorum_read, [10, 5000]), halt(0).\" ) ; \\\n\tFAILED=$$? ; \\\n\t./bin/setup-ring-for-benchmarks.sh --ssl-strict --bad-nodes 2 --ring-size 4 --node-prefix ebench_node stop ; \\\n\tif [ $$FAILED -ne 0 ]; then \\\n\t  echo \"####################\" ; \\\n\t  echo \"Erlang messages (last 50 per log file):\" ; \\\n\t  echo \"####################\" ; \\\n\t  tail -n 50 log/ebench_*@\"`hostname -f`\"/* ; \\\n\t  echo \"####################\" ; \\\n\t  false ; \\\n\telse \\\n\t  true ; \\\n\tfi\n\ncpp:\n\t(cd cpp-api && make)\n\ncpp-clean:\n\t(cd cpp-api && make clean)\n\ncpp-test: compile cpp\n\tSCALARIS_ADDITIONAL_PARAMETERS=\"-scalaris mgmt_server {{127,0,0,1},$(SCALARIS_UNITTEST_PORT),mgmt_server} -scalaris known_hosts [{{127,0,0,1},$(SCALARIS_UNITTEST_PORT),service_per_vm}]\" \\\n\t./bin/scalarisctl -d \\\n\t-y $(SCALARIS_UNITTEST_YAWS_PORT) \\\n\t-p $(SCALARIS_UNITTEST_PORT) \\\n\t-n \"$(SCALARIS_JTEST_NAME)\" -m -t first start ; \\\n\t( ./bin/scalarisctl -n \"$(SCALARIS_JTEST_NAME)\" dbg-check-ring 1 30 && \\\n\t( cd cpp-api && make cpp-test ) ) ; \\\n\tFAILED=$$? ; \\\n\t./bin/scalarisctl -n \"$(SCALARIS_JTEST_NAME)\" stop ; \\\n\tif [ $$FAILED -ne 0 ]; then \\\n\t  echo \"####################\" ; \\\n\t  echo \"Erlang messages (last 50 per log file):\" ; \\\n\t  echo \"####################\" ; \\\n\t  tail -n 50 \"log/$(SCALARIS_JTEST_NAME)@`hostname -f`/\"* ; \\\n\t  echo \"####################\" ; \\\n\t  false ; \\\n\telse \\\n\t  true ; \\\n\tfi\n\njava:\n\tant -buildfile java-api/build.xml jar tools.test.compile scalaris.test.compile\n\njava-test: compile java\n\tSCALARIS_ADDITIONAL_PARAMETERS=\"-scalaris mgmt_server {{127,0,0,1},$(SCALARIS_UNITTEST_PORT),mgmt_server} -scalaris known_hosts [{{127,0,0,1},$(SCALARIS_UNITTEST_PORT),service_per_vm}]\" \\\n\t./bin/scalarisctl -d \\\n\t-y $(SCALARIS_UNITTEST_YAWS_PORT) \\\n\t-p $(SCALARIS_UNITTEST_PORT) \\\n\t-c \"chocolate chip cookie\" \\\n\t-n \"$(SCALARIS_JTEST_NAME)\" -m -t first start ; \\\n\t( ./bin/scalarisctl -c \"chocolate chip cookie\" -n \"$(SCALARIS_JTEST_NAME)\" dbg-check-ring 1 30 && \\\n\tant -Dscalaris.node=\"$(SCALARIS_JTEST_NAME)\" -buildfile java-api/build.xml test ) ; \\\n\tFAILED=$$? ; \\\n\t./bin/scalarisctl -c \"chocolate chip cookie\" -n \"$(SCALARIS_JTEST_NAME)\" stop ; \\\n\tif [ $$FAILED -ne 0 ]; then \\\n\t  echo \"####################\" ; \\\n\t  echo \"Erlang messages (last 50 per log file):\" ; \\\n\t  echo \"####################\" ; \\\n\t  tail -n 50 \"log/$(SCALARIS_JTEST_NAME)@`hostname -f`/\"* ; \\\n\t  echo \"####################\" ; \\\n\t  false ; \\\n\telse \\\n\t  true ; \\\n\tfi\n\njava-bench: compile java\n\t( ./bin/setup-ring-for-benchmarks.sh --ring-size 4 --node-prefix jbench_node start && \\\n\tSCALARIS_JAPI_NODE=\"jbench_node1 jbench_node2 jbench_node3 jbench_node4\" ./java-api/scalaris --noconfig -b $(SCALARIS_BENCHMARKS) ) ; \\\n\tFAILED=$$? ; \\\n\t./bin/setup-ring-for-benchmarks.sh --ring-size 4 --node-prefix jbench_node stop ; \\\n\tif [ $$FAILED -ne 0 ]; then \\\n\t  echo \"####################\" ; \\\n\t  echo \"Erlang messages (last 50 per log file):\" ; \\\n\t  echo \"####################\" ; \\\n\t  tail -n 50 log/jbench_*@\"`hostname -f`\"/* ; \\\n\t  echo \"####################\" ; \\\n\t  false ; \\\n\telse \\\n\t  true ; \\\n\tfi\n\njava-%:\n\tant -buildfile java-api/build.xml $*\n\ndatanucleus:\n\tmvn -DskipGit -f java-api/pom.xml clean install\n\tmvn -DskipGit -f contrib/datanucleus/scalaris-datanucleus-store/pom.xml clean install\n\tmvn -f contrib/datanucleus/scalaris-datanucleus-store-test/pom.xml test -DskipTests\n\ndatanucleus-test: compile datanucleus\n\tSCALARIS_ADDITIONAL_PARAMETERS=\"-scalaris mgmt_server {{127,0,0,1},$(SCALARIS_UNITTEST_PORT),mgmt_server} -scalaris known_hosts [{{127,0,0,1},$(SCALARIS_UNITTEST_PORT),service_per_vm}]\" \\\n\t./bin/scalarisctl -d \\\n\t-y $(SCALARIS_UNITTEST_YAWS_PORT) \\\n\t-p $(SCALARIS_UNITTEST_PORT) \\\n\t-c \"chocolate chip cookie\" \\\n\t-n \"$(SCALARIS_JTEST_NAME)\" -m -t first start ; \\\n\t( ./bin/scalarisctl -c \"chocolate chip cookie\" -n \"$(SCALARIS_JTEST_NAME)\" dbg-check-ring 1 30 && \\\n\tmvn -Dscalaris.node=\"$(SCALARIS_JTEST_NAME)\" -f contrib/datanucleus/scalaris-datanucleus-store-test/pom.xml test ) ; \\\n\tFAILED=$$? ; \\\n\t./bin/scalarisctl -c \"chocolate chip cookie\" -n \"$(SCALARIS_JTEST_NAME)\" stop ; \\\n\tif [ $$FAILED -ne 0 ]; then \\\n\t  echo \"####################\" ; \\\n\t  echo \"Erlang messages (last 50 per log file):\" ; \\\n\t  echo \"####################\" ; \\\n\t  tail -n 50 \"log/$(SCALARIS_JTEST_NAME)@`hostname -f`/\"* ; \\\n\t  echo \"####################\" ; \\\n\t  echo \"Java messages (last 150 per log file):\" ; \\\n\t  echo \"####################\" ; \\\n\t  tail -n 150 \"contrib/datanucleus/scalaris-datanucleus-store-test/target/surefire-reports/\"*.txt ; \\\n\t  echo \"####################\" ; \\\n\t  false ; \\\n\telse \\\n\t  true ; \\\n\tfi\n\nycsb:\n\tmvn -f contrib/ycsb/pom.xml clean install\n\ncoverity: java datanucleus cpp\n\tmvn -f contrib/ycsb/scalaris/pom.xml clean install\n\npython: python-compile\n\npython-compile:\n\t$(PYTHON2) -mcompileall python-api\n\t$(PYTHON2) -O -mcompileall python-api\n\npython-clean:\n\trm -f python-api/*.pyc\n\trm -f python-api/*.pyo\n\npython-test: compile python\n\tSCALARIS_ADDITIONAL_PARAMETERS=\"-scalaris mgmt_server {{127,0,0,1},$(SCALARIS_UNITTEST_PORT),mgmt_server} -scalaris known_hosts [{{127,0,0,1},$(SCALARIS_UNITTEST_PORT),service_per_vm}]\" \\\n\t./bin/scalarisctl -d \\\n\t-y $(SCALARIS_UNITTEST_YAWS_PORT) \\\n\t-p $(SCALARIS_UNITTEST_PORT) \\\n\t-n \"$(SCALARIS_JTEST_NAME)\" -m -t first start ; \\\n\t( ./bin/scalarisctl -n \"$(SCALARIS_JTEST_NAME)\" dbg-check-ring 1 30 && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" $(PYTHON2) python-api/scalaris_test.py -v ) ; \\\n\tFAILED=$$? ; \\\n\t./bin/scalarisctl -n \"$(SCALARIS_JTEST_NAME)\" stop ; \\\n\tif [ $$FAILED -ne 0 ]; then \\\n\t  echo \"####################\" ; \\\n\t  echo \"Erlang messages (last 50 per log file):\" ; \\\n\t  echo \"####################\" ; \\\n\t  tail -n 50 \"log/$(SCALARIS_JTEST_NAME)@`hostname -f`/\"* ; \\\n\t  echo \"####################\" ; \\\n\t  false ; \\\n\telse \\\n\t  true ; \\\n\tfi\n\npython-bench: compile python\n\t( ./bin/setup-ring-for-benchmarks.sh --ring-size 4 start --node-prefix pbench_node && \\\n\tSCALARIS_JSON_URLS=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT) http://localhost:$$(($(SCALARIS_UNITTEST_YAWS_PORT)+1)) http://localhost:$$(($(SCALARIS_UNITTEST_YAWS_PORT)+2)) http://localhost:$$(($(SCALARIS_UNITTEST_YAWS_PORT)+3))\" $(PYTHON2) python-api/scalaris_bench.py $(SCALARIS_BENCHMARKS) ) ; \\\n\tFAILED=$$? ; \\\n\t./bin/setup-ring-for-benchmarks.sh --ring-size 4 --node-prefix pbench_node stop ; \\\n\tif [ $$FAILED -ne 0 ]; then \\\n\t  echo \"####################\" ; \\\n\t  echo \"Erlang messages (last 50 per log file):\" ; \\\n\t  echo \"####################\" ; \\\n\t  tail -n 50 log/pbench_*@\"`hostname -f`\"/* ; \\\n\t  echo \"####################\" ; \\\n\t  false ; \\\n\telse \\\n\t  true ; \\\n\tfi\n\npython-doc-html:\n\tcd python-api ; epydoc --config=epydoc.html.cfg ; FAILED=$$? ; cd .. ; \\\n\t  if [ $$FAILED -ne 0 ]; then false ; else  true ; fi\n\npython-doc-pdf:\n\tcd python-api ; epydoc --config=epydoc.pdf.cfg ; FAILED=$$? ; cd .. ; \\\n\t  if [ $$FAILED -ne 0 ]; then false ; else  true ; fi\n\npython3: @ENABLEPYTHON3@\n\npython3-not-found:\n\t@echo \"cannot create the Python3 API - python 3.x or 2to3 not found\"\n\t@false\n\npython3-convert: python3-clean\n\tif [ ! -d python3-api ]; then \\\n\t  mkdir -p python3-api ; \\\n\tfi\n\tcp python-api/*.py python3-api/\n\tcd python3-api && $(PYTHON3_2TO3) -f all -f buffer -w -n . >/dev/null\n\npython3-compile: python3-convert\n\t$(PYTHON3) -mcompileall python3-api\n\t$(PYTHON3) -O -mcompileall python3-api\n\npython3-clean:\n\tif [ -d python3-api ]; then \\\n\t  rm -f python3-api/*.py ; \\\n\t  rm -f python3-api/*.pyc ; \\\n\t  rm -f python3-api/*.pyo ; \\\n\t  if [ -d python3-api/__pycache__ ]; then \\\n\t    rm -f python3-api/__pycache__/*.py ; \\\n\t    rm -f python3-api/__pycache__/*.pyc ; \\\n\t    rm -f python3-api/__pycache__/*.pyo ; \\\n\t    rmdir --ignore-fail-on-non-empty python3-api/__pycache__ ; \\\n\t  fi ; \\\n\t  rmdir --ignore-fail-on-non-empty python3-api/ ; \\\n\tfi\n\npython3-test: compile python3\n\tSCALARIS_ADDITIONAL_PARAMETERS=\"-scalaris mgmt_server {{127,0,0,1},$(SCALARIS_UNITTEST_PORT),mgmt_server} -scalaris known_hosts [{{127,0,0,1},$(SCALARIS_UNITTEST_PORT),service_per_vm}]\" \\\n\t./bin/scalarisctl -d \\\n\t-y $(SCALARIS_UNITTEST_YAWS_PORT) \\\n\t-p $(SCALARIS_UNITTEST_PORT) \\\n\t-n \"$(SCALARIS_JTEST_NAME)\" -m -t first start ; \\\n\t( ./bin/scalarisctl -n \"$(SCALARIS_JTEST_NAME)\" dbg-check-ring 1 30 && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" $(PYTHON3) python3-api/scalaris_test.py -v ) ; \\\n\tFAILED=$$? ; \\\n\t./bin/scalarisctl -n \"$(SCALARIS_JTEST_NAME)\" stop ; \\\n\tif [ $$FAILED -ne 0 ]; then \\\n\t  echo \"####################\" ; \\\n\t  echo \"Erlang messages (last 50 per log file):\" ; \\\n\t  echo \"####################\" ; \\\n\t  tail -n 50 \"log/$(SCALARIS_JTEST_NAME)@`hostname -f`/\"* ; \\\n\t  echo \"####################\" ; \\\n\t  false ; \\\n\telse \\\n\t  true ; \\\n\tfi\n\npython3-bench: compile python3\n\t( ./bin/setup-ring-for-benchmarks.sh --ring-size 4 --node-prefix p3bench_node start && \\\n\tSCALARIS_JSON_URLS=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT) http://localhost:$$(($(SCALARIS_UNITTEST_YAWS_PORT)+1)) http://localhost:$$(($(SCALARIS_UNITTEST_YAWS_PORT)+2)) http://localhost:$$(($(SCALARIS_UNITTEST_YAWS_PORT)+3))\" $(PYTHON3) python3-api/scalaris_bench.py $(SCALARIS_BENCHMARKS) ) ; \\\n\tFAILED=$$? ; \\\n\t./bin/setup-ring-for-benchmarks.sh --ring-size 4 --node-prefix p3bench_node stop ; \\\n\tif [ $$FAILED -ne 0 ]; then \\\n\t  echo \"####################\" ; \\\n\t  echo \"Erlang messages (last 50 per log file):\" ; \\\n\t  echo \"####################\" ; \\\n\t  tail -n 50 log/p3bench_*@\"`hostname -f`\"/* ; \\\n\t  echo \"####################\" ; \\\n\t  false ; \\\n\telse \\\n\t  true ; \\\n\tfi\n\nruby-test: compile\n\tSCALARIS_ADDITIONAL_PARAMETERS=\"-scalaris mgmt_server {{127,0,0,1},$(SCALARIS_UNITTEST_PORT),mgmt_server} -scalaris known_hosts [{{127,0,0,1},$(SCALARIS_UNITTEST_PORT),service_per_vm}]\" \\\n\t./bin/scalarisctl -d \\\n\t-y $(SCALARIS_UNITTEST_YAWS_PORT) \\\n\t-p $(SCALARIS_UNITTEST_PORT) \\\n\t-n \"$(SCALARIS_JTEST_NAME)\" -m -t first start ; \\\n\t( ./bin/scalarisctl -n \"$(SCALARIS_JTEST_NAME)\" dbg-check-ring 1 30 && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" ruby-api/scalaris_test.rb -v ) ; \\\n\tFAILED=$$? ; \\\n\t./bin/scalarisctl -n \"$(SCALARIS_JTEST_NAME)\" stop ; \\\n\tif [ $$FAILED -ne 0 ]; then \\\n\t  echo \"####################\" ; \\\n\t  echo \"Erlang messages (last 50 per log file):\" ; \\\n\t  echo \"####################\" ; \\\n\t  tail -n 50 \"log/$(SCALARIS_JTEST_NAME)@`hostname -f`/\"* ; \\\n\t  echo \"####################\" ; \\\n\t  false ; \\\n\telse \\\n\t  true ; \\\n\tfi\n\ninterop-test: compile java java-scalaris.test.compile python\n\tSCALARIS_ADDITIONAL_PARAMETERS=\"-scalaris mgmt_server {{127,0,0,1},$(SCALARIS_UNITTEST_PORT),mgmt_server} -scalaris known_hosts [{{127,0,0,1},$(SCALARIS_UNITTEST_PORT),service_per_vm}]\" \\\n\t./bin/scalarisctl -d \\\n\t-y $(SCALARIS_UNITTEST_YAWS_PORT) \\\n\t-p $(SCALARIS_UNITTEST_PORT) \\\n\t-c \"chocolate chip cookie\" \\\n\t-n \"$(SCALARIS_JTEST_NAME)\" -m -t first start ; \\\n\t( ./bin/scalarisctl -c \"chocolate chip cookie\" -n \"$(SCALARIS_JTEST_NAME)\" dbg-check-ring 1 30 && \\\n\tjava -cp \"java-api/scalaris.jar:java-api/lib/jakarta-commons-cli-1.2.jar:java-api/lib/OtpErlang-$(OTP_ERLANG_VERSION).jar:java-api/classes\" -Dscalaris.node=\"$(SCALARIS_JTEST_NAME)\" de.zib.scalaris.InterOpTest -write interop_test > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.java_write.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.java_write.stderr.log && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" PYTHONIOENCODING=\"utf_8\" $(PYTHON2) python-api/scalaris_interop_test.py write interop_test > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_write.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_write.stderr.log && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" ruby-api/scalaris_interop_test.rb write interop_test > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.ruby_write.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.ruby_write.stderr.log && \\\n\tjava -cp \"java-api/scalaris.jar:java-api/lib/jakarta-commons-cli-1.2.jar:java-api/lib/OtpErlang-$(OTP_ERLANG_VERSION).jar:java-api/classes\" -Dscalaris.node=\"$(SCALARIS_JTEST_NAME)\" de.zib.scalaris.InterOpTest -read interop_test java > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.java_read_java.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.java_read_java.stderr.log && \\\n\tjava -cp \"java-api/scalaris.jar:java-api/lib/jakarta-commons-cli-1.2.jar:java-api/lib/OtpErlang-$(OTP_ERLANG_VERSION).jar:java-api/classes\" -Dscalaris.node=\"$(SCALARIS_JTEST_NAME)\" de.zib.scalaris.InterOpTest -read interop_test json_python > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.java_read_python.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.java_read_python.stderr.log && \\\n\tjava -cp \"java-api/scalaris.jar:java-api/lib/jakarta-commons-cli-1.2.jar:java-api/lib/OtpErlang-$(OTP_ERLANG_VERSION).jar:java-api/classes\" -Dscalaris.node=\"$(SCALARIS_JTEST_NAME)\" de.zib.scalaris.InterOpTest -read interop_test json_ruby > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.java_read_ruby.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.java_read_ruby.stderr.log && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" PYTHONIOENCODING=\"utf_8\" $(PYTHON2) python-api/scalaris_interop_test.py read interop_test java > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_read_java.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_read_java.stderr.log && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" PYTHONIOENCODING=\"utf_8\" $(PYTHON2) python-api/scalaris_interop_test.py read interop_test json_python > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_read_python.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_read_python.stderr.log && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" PYTHONIOENCODING=\"utf_8\" $(PYTHON2) python-api/scalaris_interop_test.py read interop_test json_ruby > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_read_ruby.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_read_ruby.stderr.log && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" ruby-api/scalaris_interop_test.rb read interop_test java > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.ruby_read_java.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.ruby_read_java.stderr.log && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" ruby-api/scalaris_interop_test.rb read interop_test json_python > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.ruby_read_python.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.ruby_read_python.stderr.log && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" ruby-api/scalaris_interop_test.rb read interop_test json_ruby > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.ruby_read_ruby.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.ruby_read_ruby.stderr.log ) ; \\\n\tFAILED=$$? ; \\\n\t./bin/scalarisctl -c \"chocolate chip cookie\" -n \"$(SCALARIS_JTEST_NAME)\" stop ; \\\n\tif [ $$FAILED -ne 0 ]; then \\\n\t  echo \"####################\" ; \\\n\t  cat \"log/$(SCALARIS_JTEST_NAME)@`hostname -f`/\"interop.*write*.stderr.log ; \\\n\t  cat \"log/$(SCALARIS_JTEST_NAME)@`hostname -f`/\"interop.*read*.stderr.log ; \\\n\t  echo \"####################\" ; \\\n\t  echo \"Erlang messages (last 50 per log file):\" ; \\\n\t  echo \"####################\" ; \\\n\t  tail -n 50 \"log/$(SCALARIS_JTEST_NAME)@`hostname -f`/\"{*.auth,report.log,scalaris_error_logger,scalaris_log4erl.txt} ; \\\n\t  echo \"####################\" ; \\\n\t  false ; \\\n\telse \\\n\t  echo \"all tests successful\" ; \\\n\t  true ; \\\n\tfi\n\n# only test whether python2 and python3 read/write the same values - the rest is checked by interop-test\ninterop3-test: interop-test python3\n\tSCALARIS_ADDITIONAL_PARAMETERS=\"-scalaris mgmt_server {{127,0,0,1},$(SCALARIS_UNITTEST_PORT),mgmt_server} -scalaris known_hosts [{{127,0,0,1},$(SCALARIS_UNITTEST_PORT),service_per_vm}]\" \\\n\t./bin/scalarisctl -d \\\n\t-y $(SCALARIS_UNITTEST_YAWS_PORT) \\\n\t-p $(SCALARIS_UNITTEST_PORT) \\\n\t-n \"$(SCALARIS_JTEST_NAME)\" -m -t first start ; \\\n\t( ./bin/scalarisctl -n \"$(SCALARIS_JTEST_NAME)\" dbg-check-ring 1 30 && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" PYTHONIOENCODING=\"utf_8\" $(PYTHON2) python-api/scalaris_interop_test.py write interop_test > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_write.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_write.stderr.log && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" PYTHONIOENCODING=\"utf_8\" $(PYTHON3) python3-api/scalaris_interop_test.py write interop_test3 > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python3_write.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python3_write.stderr.log && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" PYTHONIOENCODING=\"utf_8\" $(PYTHON2) python-api/scalaris_interop_test.py read interop_test json_python > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_read_python.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_read_python.stderr.log && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" PYTHONIOENCODING=\"utf_8\" $(PYTHON2) python-api/scalaris_interop_test.py read interop_test3 json_python > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_read_python3.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python_read_python3.stderr.log && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" PYTHONIOENCODING=\"utf_8\" $(PYTHON3) python3-api/scalaris_interop_test.py read interop_test json_python > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python3_read_python.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python3_read_python.stderr.log && \\\n\tSCALARIS_JSON_URL=\"http://localhost:$(SCALARIS_UNITTEST_YAWS_PORT)\" PYTHONIOENCODING=\"utf_8\" $(PYTHON3) python3-api/scalaris_interop_test.py read interop_test3 json_python > log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python3_read_python3.log 2> log/$(SCALARIS_JTEST_NAME)@`hostname -f`/interop.python3_read_python3.stderr.log ) ; \\\n\tFAILED=$$? ; \\\n\t./bin/scalarisctl -n \"$(SCALARIS_JTEST_NAME)\" stop ; \\\n\tif [ $$FAILED -ne 0 ]; then \\\n\t  echo \"####################\" ; \\\n\t  cat \"log/$(SCALARIS_JTEST_NAME)@`hostname -f`/\"interop.*write*.stderr.log ; \\\n\t  cat \"log/$(SCALARIS_JTEST_NAME)@`hostname -f`/\"interop.*read*.stderr.log ; \\\n\t  echo \"####################\" ; \\\n\t  echo \"Erlang messages (last 50 per log file):\" ; \\\n\t  echo \"####################\" ; \\\n\t  tail -n 50 \"log/$(SCALARIS_JTEST_NAME)@`hostname -f`/\"{*.auth,report.log,scalaris_error_logger,scalaris_log4erl.txt} ; \\\n\t  echo \"####################\" ; \\\n\t  false ; \\\n\telse \\\n\t  echo \"all tests successful\" ; \\\n\t  true ; \\\n\tfi\n\n# delete all generated files (including test results and logs)\ndist-clean: clean test-clean log-clean\n\nlog-clean:\n\t-rm -rf log/*\n\n# delete all generated files except test results and logs\nclean: erlang-clean nifs-clean tags-clean cpp-clean java-clean python-clean python3-clean\n\n# delete all files related to our erlang sources\nerlang-clean:\n\t-rm -f .make-doc\n\t-rm -f bin/*.beam\n\t-rm -f ebin/*.beam\n\t-rm -f src/*.beam\n\t-rm -f src/comm_layer/*.beam\n\t-rm -f contrib/yaws/ebin/*.beam\n\t-rm -f contrib/log4erl/ebin/*.beam\n\t-rm -f contrib/log4erl/src/log4erl_lex.erl\n\t-rm -f contrib/log4erl/src/log4erl_parser.erl\n\t-rm -f contrib/dotto/ebin/*\n\t-rm -f test/*.beam\n\t-rm -f doc/*.html\n\t-rm -f doc/*.css\n\t-rm -rf doc/edoc-info\n\t-rm -f docroot/graphs/*.png\n\t-rm -f TAGS TAGS.contrib\n\ntags-clean:\n\t-rm -f TAGS TAGS.root TAGS.contrib\n\ntags: TAGS.root TAGS.contrib\n\t@$(ERL) -noinput -noshell -eval 'tags:subdirs([\"src\", \"include\", \"test\"]), halt(0).' ; true\n\t@cat TAGS.contrib TAGS.root >> TAGS ; true\n\nTAGS.contrib:\n\t@echo \"making TAGS.contrib\"\n\t@$(ERL) -noinput -noshell -eval 'tags:subdir(\"contrib\", [{outfile, \"TAGS.contrib\"}]), halt(0).' ; true\n\nTAGS.root:\n\t@echo \"making TAGS.root\"\n\t@$(ERL) -noinput -noshell -eval 'tags:root([{outfile, \"TAGS.root\"}]), halt(0).' ; true\n\ndoc: .make-doc\n\n.make-doc: $(SOURCEPATHS_ERL)\n\t$(ERL) -noshell -run edoc_run application \"'$(APP_NAME)'\" \\\n\t'\".\"' '[{def, {vsn, \"$(VERSION)\"}}, no_packages, {preprocess, true}, {includes, [\"include\", \"contrib/yaws/include\", \"contrib/log4erl/include\"]}, {sort_functions, false}, {pretty_printer, erl_pp}, {macros, [@EDOCMACROS@]}]'\n\t@touch .make-doc\n\ndocs: doc java-doc\n\ndialyzer: compile\n\t$(DIALYZER) @DIALYZER_FLAGS@ $(DIALYZER_FLAGS) -Wunmatched_returns -Wunknown -I include/ -I contrib/yaws/include/ -I contrib/log4erl/include/ --src -c src src/*/ src/*/*/ test --no_native --statistics\n\ndialyzer-col:\n\t$(MAKE) dialyzer | sed -f contrib/dialyzer-col.sed\n\nplt: compile\n\t$(DIALYZER) --plt_info > /dev/null ; \\\n\tEXISTSFAILED=$$? ; \\\n\t$(DIALYZER) --check_plt > /dev/null ; \\\n\tCHECKFAILED=$$? ; \\\n\tif [ $$EXISTSFAILED -ne 0 ]; then \\\n\t  echo \"building plt\" ; \\\n\t  $(DIALYZER) --build_plt  $(DIALYZER_PLT_LIBS) ; \\\n\telif [ $$CHECKFAILED -ne 0 ]; then \\\n\t  echo \"rebuilding outdated plt\" ; \\\n\t  $(DIALYZER) --build_plt  $(DIALYZER_PLT_LIBS) ; \\\n\telse \\\n\t  echo \"adding Scalaris modules to plt\" ; \\\n\t  $(DIALYZER) --add_to_plt  $(DIALYZER_PLT_LIBS) ; \\\n\tfi\n\nrrd-init:\n\t@echo This target is deprecated\n\nnifs: $(NIFS_SO)\n\nnifs-clean:\n\t-rm -f $(NIFS_SO)\n\npriv/%.so: src/nifs/%.cpp Makefile\n\t@CXX@ @NIFFLAGS@ -I@ERLANG_ROOT_DIR@/erts-@ERLANG_LIB_VER_erts@/include @CXXFLAGS@ -fPIC -o $@ $<\n\ninstall-doc: doc\n\tinstall -d $(DOCDIR)\n\tinstall -m 644 -t $(DOCDIR) \\\n\t           AUTHORS README.md LICENSE ChangeLog\n\tinstall -m 644 user-dev-guide/main.pdf $(DOCDIR)/user-dev-guide.pdf\n\tinstall -d $(DOCDIR)/erlang\n\tinstall -m 644 -t $(DOCDIR)/erlang \\\n\t           doc/*.html \\\n\t           doc/*.png\n\tinstall -d $(DOCROOTDIR)\n\tln -sf @docdir@/erlang $(DOCROOTDIR)/doc\n\ninstall-java-doc: java-doc\n\tfor dir in `find java-api/doc -type d`; do \\\n\t  install -d \"$(DOCDIR)/java-api$${dir#java-api/doc}\" ; \\\n\t  for file in `find \"$$dir\" -maxdepth 1 -type f`; do \\\n\t    install -m 644 -t \"$(DOCDIR)/java-api$${dir#java-api/doc}\" \"$${file}\" ; \\\n\t  done ; \\\n\tdone\n\ninstall-python-doc-html: python-doc-html\n\tinstall -d $(DOCDIR)/python-api\n\tinstall -d $(DOCDIR)/python-api/html\n\tinstall -m 644 -t $(DOCDIR)/python-api/html \\\n\t           python-api/doc/html/*\n\ninstall-python-doc-pdf: python-doc-pdf\n\tinstall -d $(DOCDIR)/python-api\n\tinstall -d $(DOCDIR)/python-api/pdf\n\tinstall -m 644 -t $(DOCDIR)/python-api/pdf \\\n\t           python-api/doc/pdf/*.pdf\n\ninstall-docs: install-doc install-java-doc\n\ncontrib/init.d/scalaris-first-monitor: contrib/init.d/scalaris-monitor\n\tsed -e 's|initd.conf|initd-first.conf|g' \\\n\t    -e 's|Provides:          scalaris-monitor|Provides:          scalaris-first-monitor|g' \\\n\t    $${srcdir}$< > $@\n\ninstall-java: java contrib/init.d/scalaris-first-monitor\n\tinstall -d $(JAVADIR)/scalaris\n\tinstall -d $(JAVADIR)/scalaris/lib\n\tinstall -m 644 -t $(JAVADIR)/scalaris \\\n\t           java-api/scalaris.jar\n\tinstall -m 644 -t $(JAVADIR)/scalaris/lib \\\n\t           java-api/lib/OtpErlang-$(OTP_ERLANG_VERSION).jar\n\tinstall -m 644 -t $(JAVADIR)/scalaris/lib \\\n\t           java-api/lib/jakarta-commons-cli-$(COMMONS_CLI_VERSION).jar\n\tinstall -d $(ETCDIR)\n\tinstall -m 644 java-api/scalaris-java.conf \\\n\t           $(ETCDIR)/scalaris-java.conf\n\tinstall -m 644 -t $(ETCDIR) \\\n\t           java-api/scalaris-java.conf.sample\n\tinstall -m 644 -t $(ETCDIR) \\\n\t           java-api/scalaris.properties\n\tinstall -d $(BINDIR)\n\tinstall -m 755 -t $(BINDIR) \\\n\t           java-api/scalaris\n\tinstall -d $(INITDIR)\n\tinstall -m 744 -t $(INITDIR) \\\n\t           contrib/init.d/scalaris-monitor \\\n\t           contrib/init.d/scalaris-first-monitor\n\ncontrib/init.d/scalaris-first: contrib/init.d/scalaris\n\tsed -e 's|initd.conf|initd-first.conf|g' \\\n\t    -e 's|Provides:          scalaris|Provides:          scalaris-first|g' \\\n\t    $${srcdir}$< > $@\n\ninstall-initd: contrib/init.d/scalaris-first\n\tinstall -d $(ETCDIR)\n\tinstall -m 644 -t $(ETCDIR) \\\n\t           contrib/init.d/initd.conf \\\n\t           contrib/init.d/initd-first.conf\n\tinstall -d $(INITDIR)\n\tinstall -m 744 -t $(INITDIR) \\\n\t           contrib/init.d/scalaris \\\n\t           contrib/init.d/scalaris-first\n\tinstall -d $(SBINDIR)\n\tln -sf @sysconfdir@/init.d/scalaris $(SBINDIR)/rcscalaris\n\ncontrib/systemd/scalaris.service: contrib/systemd/scalaris.service.in Makefile\n\tsed -e 's|@bindir[@]|$(bindir)|g' \\\n\t    -e 's|@sysconfdir[@]|$(sysconfdir)|g' \\\n\t    -e 's|@localstatedir[@]|$(localstatedir)|g' \\\n\t    -e 's|Description=Scalaris node|Description=Scalaris management server and first node|g' \\\n\t    $${srcdir}$@.in > $@\n\ncontrib/systemd/scalaris-first.service: contrib/systemd/scalaris.service\n\tsed -e 's|conf.d/scalaris|conf.d/scalaris-first|g' \\\n\t    $${srcdir}$< > $@\n\ninstall-systemd: contrib/systemd/scalaris.service contrib/systemd/scalaris-first.service\n\tinstall -d $(SYSTEMD_CONFDIR)\n\tinstall -m 644 contrib/systemd/scalaris.conf \\\n\t           $(SYSTEMD_CONFDIR)/scalaris\n\tinstall -m 644 contrib/systemd/scalaris-first.conf \\\n\t           $(SYSTEMD_CONFDIR)/scalaris-first\n\tinstall -d $(SYSTEMD_UNITDIR)\n\tinstall -m 644 -t $(SYSTEMD_UNITDIR) \\\n\t           contrib/systemd/scalaris.service \\\n\t           contrib/systemd/scalaris-first.service\n\ninstall-erlang: compile @INSTALL_INIT@\n\tinstall -d $(BEAMDIR)\n\tinstall -m 644 -t $(BEAMDIR) \\\n\t           ebin/*.beam\n\tinstall -d $(ETCDIR)\n\tinstall -m 644 -t $(ETCDIR) \\\n\t           bin/scalarisctl.conf \\\n\t           bin/scalaris.cfg \\\n\t           bin/scalaris.local.cfg.example\n\tif [ -f bin/scalaris.local.cfg ]; then \\\n\t  install -m 644 -t $(ETCDIR) \\\n\t           bin/scalaris.local.cfg; \\\n\telse \\\n\t  touch $(ETCDIR)/scalaris.local.cfg ; \\\n\tfi\n\tinstall -d $(DOCROOTDIR) \\\n\t           $(DOCROOTDIR)/api \\\n\t           $(DOCROOTDIR)/flot \\\n\t           $(DOCROOTDIR)/icons \\\n\t           $(DOCROOTDIR)/images/default \\\n\t           $(DOCROOTDIR)/images/default/dd \\\n\t           $(DOCROOTDIR)/images/default/panel \\\n\t           $(DOCROOTDIR)/images/default/tree \\\n\t           $(DOCROOTDIR)/images/default/grid \\\n\t           $(DOCROOTDIR)/tablesort/blue\n\tinstall -m 644 -t $(DOCROOTDIR) \\\n\t           docroot/*.yaws \\\n\t           docroot/*.css \\\n\t           docroot/*.js \\\n\t           docroot/*.gif \\\n\t           docroot/*.ico\n\tinstall -m 644 -t $(DOCROOTDIR)/api \\\n\t           docroot/api/*\n\tinstall -m 644 -t $(DOCROOTDIR)/flot \\\n\t           docroot/flot/*\n\tinstall -m 644 -t $(DOCROOTDIR)/icons \\\n\t           docroot/icons/*.gif\n\tinstall -m 644 -t $(DOCROOTDIR)/images/default/dd \\\n\t           docroot/images/default/dd/*.gif\n\tinstall -m 644 -t $(DOCROOTDIR)/images/default/panel \\\n\t           docroot/images/default/panel/*.gif\n\tinstall -m 644 -t $(DOCROOTDIR)/images/default/panel \\\n\t           docroot/images/default/panel/*.png\n\tinstall -m 644 -t $(DOCROOTDIR)/images/default/tree \\\n\t           docroot/images/default/tree/*.gif\n\tinstall -m 644 -t $(DOCROOTDIR)/images/default/grid \\\n\t           docroot/images/default/grid/*.gif\n\tinstall -m 644 -t $(DOCROOTDIR)/images/default/grid \\\n\t           docroot/images/default/grid/*.png\n\tinstall -m 644 -t $(DOCROOTDIR)/tablesort \\\n\t           docroot/tablesort/*.js\n\tinstall -m 644 -t $(DOCROOTDIR)/tablesort/blue \\\n\t           docroot/tablesort/blue/*.gif\n\tinstall -m 644 -t $(DOCROOTDIR)/tablesort/blue \\\n\t           docroot/tablesort/blue/*.css\n\tinstall -d $(SCALARISDIR)/contrib/yaws/ebin\n\tinstall -d $(SCALARISDIR)/contrib/yaws/include\n\tinstall -m 644 -t $(SCALARISDIR)/contrib/yaws/ebin \\\n\t           contrib/yaws/ebin/*.beam\n\tinstall -m 644 -t $(SCALARISDIR)/contrib/yaws/include \\\n\t           contrib/yaws/include/yaws_api.hrl\n\tinstall -d $(SCALARISDIR)/contrib/log4erl/ebin\n\tinstall -m 644 -t $(SCALARISDIR)/contrib/log4erl/ebin \\\n\t           contrib/log4erl/ebin/*.beam\n\tinstall -d $(LOGDIR)\n\tinstall -d $(RUNDIR)\n\tinstall -d $(BINDIR)\n\tinstall -m 755 -t $(BINDIR) \\\n\t           bin/scalarisctl\n\ninstall-cpp: cpp\n\tinstall -d $(INCLUDEDIR)/scalaris\n\tfor dir in `find cpp-api/include -type d`; do \\\n\t  install -d \"$(INCLUDEDIR)/scalaris$${dir#cpp-api/include}\" ; \\\n\t  for file in `find \"$$dir\" -maxdepth 1 -type f`; do \\\n\t    install -m 644 -t \"$(INCLUDEDIR)/scalaris$${dir#cpp-api/include}\" \"$${file}\" ; \\\n\t  done ; \\\n\tdone\n\tinstall -d $(LIBDIR)\n\tinstall -m 644 -t $(LIBDIR) cpp-api/libscalaris.a\n\ninstall-ruby:\n\tinstall -d $(RUBYSITELIBDIR)\n\tinstall -m 644 -t $(RUBYSITELIBDIR) ruby-api/scalaris.rb\n\tinstall -m 755 -t $(RUBYSITELIBDIR) ruby-api/scalaris_client.rb\n\tinstall -d $(BINDIR)\n\tinstall -m 755 ruby-api/scalaris $(BINDIR)/scalaris-ruby\n\ninstall-python: python\n\tinstall -d $(PYTHON2SITELIBDIR)\n\tinstall -m 644 -p -t $(PYTHON2SITELIBDIR) python-api/scalaris.py*\n\tinstall -m 644 -p -t $(PYTHON2SITELIBDIR) python-api/scalaris_bench.py*\n\tinstall -m 755 -p -t $(PYTHON2SITELIBDIR) python-api/scalaris_client.py*\n\tinstall -d $(BINDIR)\n\tinstall -m 755 -p python-api/scalaris $(BINDIR)/scalaris-python\n\ninstall-python3: python3\n\tinstall -d $(PYTHON3SITELIBDIR)\n\tinstall -m 644 -p -t $(PYTHON3SITELIBDIR) python3-api/scalaris.py*\n\tinstall -m 644 -p -t $(PYTHON3SITELIBDIR) python3-api/scalaris_bench.py*\n\tinstall -m 755 -p -t $(PYTHON3SITELIBDIR) python3-api/scalaris_client.py*\n\tif [ -d python3-api/__pycache__ ]; then \\\n\t  install -d $(PYTHON3SITELIBDIR)/__pycache__ ; \\\n\t  install -m 644 -p -t $(PYTHON3SITELIBDIR)/__pycache__ \\\n\t           python3-api/__pycache__/scalaris.*.py*; \\\n\t  install -m 644 -p -t $(PYTHON3SITELIBDIR)/__pycache__ \\\n\t           python3-api/__pycache__/scalaris_bench.*.py*; \\\n\tfi\n\tinstall -d $(BINDIR)\n\tinstall -m 755 -p python3-api/scalaris $(BINDIR)/scalaris-python3\n\ninstall: install-erlang\n\ninstall-all: install-erlang install-java install-docs @ENABLERUBYINSTALL@ @ENABLEPYTHON2INSTALL@ @ENABLEPYTHON3INSTALL@\n\nMakefile: Makefile.in\n\t@echo\n\t@echo \"Makefile is outdated. Please run './configure'.\"\n\t@if [ -f ./config.status ]; then \\\n\t   if [ \"\" != \"`./config.status --config`\" ]; then \\\n\t     echo \"For the parameters you used last time you may call\"; \\\n\t     echo \"'./configure `./config.status --config|sed -e \"s/'//g\"`'.\"; \\\n\tfi fi\n\t@echo\n\t@exit 1\n\n"
  },
  {
    "path": "README.md",
    "content": "[![Build Status](https://travis-ci.org/scalaris-team/scalaris.svg?branch=master)](https://travis-ci.org/scalaris-team/scalaris)\n[![Project Stats](https://www.openhub.net/p/scalaris/widgets/project_thin_badge.gif)](https://www.openhub.net/p/scalaris)\n\nScalaris is a scalable, transactional, distributed and fault-tolerant\nkey-value-store with strong data consistency for online databases and\nWeb 2.0 services.\n\nInstructions can be found in the [Users and Developers Guide](https://github.com/scalaris-team/scalaris/blob/master/user-dev-guide/main.pdf)\n\nAdditional information, including download links for rpm-packages, can\nbe found on the project homepage:\nhttp://scalaris.zib.de/\n"
  },
  {
    "path": "VERSION",
    "content": "0.9.0+git\n"
  },
  {
    "path": "bin/.gitignore",
    "content": "/scalaris.local.cfg\n/scalarisclient\n/scalarisctl\n/cs_local.sh\n/cs_local2.sh\n/cs_local3.sh\n/boot.sh\n/.scalaris.cfg.swp\n"
  },
  {
    "path": "bin/find_erlang.bat",
    "content": "@echo off\r\n:: set path to erlang installation\r\n\r\nSETLOCAL ENABLEDELAYEDEXPANSION\r\n\r\nset ERL_R18_0=7.0.2 7.0.1 7.0\r\nset ERL_R17_5=6.4.1 6.4\r\nset ERL_R17_4=6.3.1 6.3\r\nset ERL_R17_3=6.2.1 6.2\r\nset ERL_R17_1=6.1.2 6.1.1 6.1\r\nset ERL_R17=6.0.1 6.0\r\nset ERL_R16B03=5.10.4.1 5.10.4\r\nset ERL_R16B02=5.10.3.1 5.10.3\r\nset ERL_R16B01=5.10.2\r\nset ERL_R16B=5.10.1.2 5.10.1.1 5.10.1\r\nset ERL_R15B03=5.9.3.1 5.9.3\r\nset ERL_R15B02=5.9.2\r\nset ERL_R15B01=5.9.1.2 5.9.1.1 5.9.1\r\nset ERL_R15B=5.9.0.1 5.9\r\nset ERL_R14B04=5.8.5\r\nset ERL_ALL=%ERL_R18_0% %ERL_R17_5% %ERL_R17_4% %ERL_R17_3% %ERL_R17_1% %ERL_R17% %ERL_R16B03% %ERL_R16B02% %ERL_R16B01% %ERL_R16B% %ERL_R15B03% %ERL_R15B02% %ERL_R15B01% %ERL_R15B% %ERL_R14B04%\r\nif not defined ERLANG_HOME (\r\n    FOR %%c in (%ERL_ALL%) DO (\r\n        FOR %%p in (\"%ProgramFiles%\\erl%%c\" \"%ProgramFiles(x86)%\\erl%%c\") DO (\r\n            set FULL_PATH=%%p\r\n            if exist !FULL_PATH! (\r\n                set ERLANG_HOME=%%p\r\n                goto found\r\n            )\r\n        )\r\n    )\r\n)\r\n\r\n:found\r\n\r\necho ERLANG_HOME=%ERLANG_HOME%\r\n\r\nENDLOCAL & set ERLANG_HOME=%ERLANG_HOME%\r\n"
  },
  {
    "path": "bin/firstnode.bat",
    "content": "@echo off\r\n:: Copyright 2010-2015 Zuse Institute Berlin\r\n::\r\n::    Licensed under the Apache License, Version 2.0 (the \"License\");\r\n::    you may not use this file except in compliance with the License.\r\n::    You may obtain a copy of the License at\r\n::\r\n::        http://www.apache.org/licenses/LICENSE-2.0\r\n::\r\n::    Unless required by applicable law or agreed to in writing, software\r\n::    distributed under the License is distributed on an \"AS IS\" BASIS,\r\n::    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\n::    See the License for the specific language governing permissions and\r\n::    limitations under the License.\r\nSETLOCAL\r\n\r\nfor %%x in (%cmdcmdline%) do if %%~x==/c set DOUBLECLICKED=1\r\n\r\n:: Script to start a new Scalaris system with an initial node.\r\nset SCRIPTDIR=%~dp0\r\nset ID=0\r\nset params=%*\r\n\r\nset NODE_NAME=firstnode\r\nset /a CSPORT=14195+%ID%\r\nset /a YAWSPORT=8000+%ID%\r\nset SCALARIS_ADDITIONAL_PARAMETERS=-scalaris port %CSPORT% -scalaris yaws_port %YAWSPORT%\r\n\r\n:::::::::::::::::::::::::::::::::::::::::::::::::::::::::\r\n:: scalaris configuration parameters\r\nset SCALARIS_COOKIE=chocolate chip cookie\r\nset SCALARISDIR=%SCRIPTDIR%..\r\nset BEAMDIR=%SCALARISDIR%\\ebin\r\nset BACKGROUND=\r\n::set BACKGROUND=-detached\r\nset TOKEFLAGS=\r\n:: note: paths passed as strings to erlang applications need to be escaped!\r\nset LOGDIR=%SCALARISDIR:\\=\\\\%\\\\log\r\nset DOCROOTDIR=%SCALARISDIR:\\=\\\\%\\\\docroot\r\nset ETCDIR=%SCALARISDIR:\\=\\\\%\\\\bin\r\n:::::::::::::::::::::::::::::::::::::::::::::::::::::::::\r\n\r\n:: set path to erlang installation\r\ncall \"%SCRIPTDIR%\"\\find_erlang.bat\r\n\r\n@echo on\r\npushd %BEAMDIR%\r\n%ERLANG_HOME%\\bin\\erl.exe -setcookie \"%SCALARIS_COOKIE%\" ^\r\n  -pa \"%SCALARISDIR%\\contrib\\yaws\\ebin\" ^\r\n  -pa \"%SCALARISDIR%\\contrib\\log4erl\\ebin\" ^\r\n  -pa \"%BEAMDIR%\" %TOKEFLAGS% %BACKGROUND% ^\r\n  -yaws embedded true ^\r\n  -scalaris log_path \"\\\"%LOGDIR%\\\"\" ^\r\n  -scalaris docroot \"\\\"%DOCROOTDIR%\\\"\" ^\r\n  -scalaris config \"\\\"%ETCDIR%\\\\scalaris.cfg\\\"\" ^\r\n  -scalaris local_config \"\\\"%ETCDIR%\\\\scalaris.local.cfg\\\"\" ^\r\n  -scalaris start_type first ^\r\n  -scalaris start_mgmt_server true ^\r\n  -connect_all false -hidden -name %NODE_NAME% ^\r\n  %SCALARIS_ADDITIONAL_PARAMETERS% ^\r\n  -s scalaris %ERL_SCHED_FLAGS% %params%\r\npopd\r\n@echo off\r\n\r\nif defined DOUBLECLICKED PAUSE\r\n"
  },
  {
    "path": "bin/firstnode.sh",
    "content": "#!/bin/bash\n# Copyright 2007-2015 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\n# Script to start a new Scalaris system with an initial node.\nID=0\nNAME=\"firstnode\"\nPORT=$((14195+$ID))\nYAWSPORT=$((8000+$ID))\n\nABSPATH=\"$(cd \"${0%/*}\" 2>/dev/null; echo \"$PWD\"/\"${0##*/}\")\"\nDIRNAME=`dirname $ABSPATH`\n\nif [ 1 -le $# ]; then\nERLFLAGS=\"$@\"\nelse\nERLFLAGS=\" \"\nfi\n\n# start a mgmt_server (-m)\n# start the first node (-t first)\nNODE_NAME_AND_PORTS=\"-n $NAME -p $PORT -y $YAWSPORT\"\n$DIRNAME/scalarisctl -e \"$ERLFLAGS\" -m $NODE_NAME_AND_PORTS -t first start\n"
  },
  {
    "path": "bin/joining_node.bat",
    "content": "@echo off\r\n:: Copyright 2010-2015 Zuse Institute Berlin\r\n::\r\n::    Licensed under the Apache License, Version 2.0 (the \"License\");\r\n::    you may not use this file except in compliance with the License.\r\n::    You may obtain a copy of the License at\r\n::\r\n::        http://www.apache.org/licenses/LICENSE-2.0\r\n::\r\n::    Unless required by applicable law or agreed to in writing, software\r\n::    distributed under the License is distributed on an \"AS IS\" BASIS,\r\n::    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\n::    See the License for the specific language governing permissions and\r\n::    limitations under the License.\r\nSETLOCAL\r\n\r\nfor %%x in (%cmdcmdline%) do if %%~x==/c set DOUBLECLICKED=1\r\n\r\n:: Script to start a node, that joins a running Scalaris system.\r\nset SCRIPTDIR=%~dp0\r\nset ID=1\r\nset params=%*\r\n\r\n:: extract node ID from first command line parameter\r\nIF [%1]==[] goto start\r\nset ID=%1\r\nset params=\r\n:: there is no easy way to get the rest of the parameters\r\n:: -> loop through them and remove the first\r\n:loop\r\nshift\r\nif [%1]==[] goto start\r\nset params=%params% %1\r\ngoto loop\r\n)\r\n:start\r\n\r\nset NODE_NAME=node%ID%\r\nset /a CSPORT=14195+%ID%\r\nset /a YAWSPORT=8000+%ID%\r\nset SCALARIS_ADDITIONAL_PARAMETERS=-scalaris port %CSPORT% -scalaris yaws_port %YAWSPORT%\r\n\r\n:::::::::::::::::::::::::::::::::::::::::::::::::::::::::\r\n:: scalaris configuration parameters\r\nset SCALARIS_COOKIE=chocolate chip cookie\r\nset SCALARISDIR=%SCRIPTDIR%..\r\nset BEAMDIR=%SCALARISDIR%\\ebin\r\nset BACKGROUND=\r\n::set BACKGROUND=-detached\r\nset TOKEFLAGS=\r\n:: note: paths passed as strings to erlang applications need to be escaped!\r\nset LOGDIR=%SCALARISDIR:\\=\\\\%\\\\log\r\nset DOCROOTDIR=%SCALARISDIR:\\=\\\\%\\\\docroot\r\nset ETCDIR=%SCALARISDIR:\\=\\\\%\\\\bin\r\n:::::::::::::::::::::::::::::::::::::::::::::::::::::::::\r\n\r\n:: set path to erlang installation\r\ncall \"%SCRIPTDIR%\"\\find_erlang.bat\r\n\r\n@echo on\r\npushd %BEAMDIR%\r\n%ERLANG_HOME%\\bin\\erl.exe -setcookie \"%SCALARIS_COOKIE%\" ^\r\n  -pa \"%SCALARISDIR%\\contrib\\yaws\\ebin\" ^\r\n  -pa \"%SCALARISDIR%\\contrib\\log4erl\\ebin\" ^\r\n  -pa \"%BEAMDIR%\" %TOKEFLAGS% %BACKGROUND% ^\r\n  -yaws embedded true ^\r\n  -scalaris log_path \"\\\"%LOGDIR%\\\"\" ^\r\n  -scalaris docroot \"\\\"%DOCROOTDIR%\\\"\" ^\r\n  -scalaris config \"\\\"%ETCDIR%\\\\scalaris.cfg\\\"\" ^\r\n  -scalaris local_config \"\\\"%ETCDIR%\\\\scalaris.local.cfg\\\"\" ^\r\n  -scalaris start_type joining ^\r\n  -connect_all false -hidden -name %NODE_NAME% ^\r\n  %SCALARIS_ADDITIONAL_PARAMETERS% ^\r\n  -s scalaris %ERL_SCHED_FLAGS% %params%\r\npopd\r\n@echo off\r\n\r\nif defined DOUBLECLICKED PAUSE\r\n"
  },
  {
    "path": "bin/joining_node.sh",
    "content": "#!/bin/bash\n# Copyright 2007-2017 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\n# Script to start a node, that joins a running Scalaris system.\nID=1\n\nif [ 1 -le $# ]; then\n    if echo $1 | grep \"^[0-9]*$\"> /dev/null\n    then\n        ID=$1\n        shift\n    fi;\nfi\n\nNAME=\"node$ID\"\nCSPORT=$((14195+$ID))\nYAWSPORT=$((8000+$ID))\n\nABSPATH=\"$(cd \"${0%/*}\" 2>/dev/null; echo \"$PWD\"/\"${0##*/}\")\"\nDIRNAME=`dirname $ABSPATH`\n\nif [ 1 -le $# ]; then\nERLFLAGS=\"$@\"\nelse\nERLFLAGS=\" \"\nfi\n\n$DIRNAME/scalarisctl -e \"$ERLFLAGS -scalaris config_kvs \\\"[{availibility_zone_id, $ID}]\\\"\" -n $NAME -p $CSPORT -y $YAWSPORT -t joining start\n"
  },
  {
    "path": "bin/jsonclient.in",
    "content": "#!/bin/bash\n# Copyright 2007-2018 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nprefix=@prefix@\nexec_prefix=@exec_prefix@\nBINDIR=@bindir@\n\nIP=\"{127,0,0,1}\"\nPORT=8000\nRINGSIZE=4\nSSL=false\n\nusage(){\n    echo \"usage: jsonclient [options] <cmd>\"\n    echo \" options:\"\n    echo \"    -h              - print this help message\"\n    echo \"    -i <ip>         - TCP IP for YAWS as Erlang tuple (default {127,0,0,1})\"\n    echo \"    -p <port>       - TCP port for YAWS (default 8000)\"\n    echo \"    -r <size>       - ring size (default 4)\"\n    echo \"    -s              - enable ssl (default false)\"\n    echo \" <cmd>:\"\n    echo \"    benchmark       - run benchmarks\"\n    echo \"    ringsize        - get ring size\"\n    echo \"    waitforringsize - wait for ring size\"\n    echo \"\"\n    exit $1\n}\n\nfix_paths() {\n    ABSPATH=\"$(cd \"${0%/*}\" 2>/dev/null; echo \"$PWD\"/\"${0##*/}\")\"\n    DIRNAME=`dirname $ABSPATH`\n    # is this a source checkout or an (rpm/deb/manual) installation?\n    if [ \"$DIRNAME\" != \"$BINDIR\" -a \"$DIRNAME\" != \"/bin\" ]; then\n        # scalaris\n        SCALARISDIR=`dirname $DIRNAME`\n        # scalaris/bin\n        BINDIR=$SCALARISDIR/bin\n    fi\n}\n\njsonbenchmark() {\n    $BINDIR/scalarisctl -t client -e \"-noinput -eval \\\"jsonclient:run_benchmark($IP, $PORT, $SSL), halt(0).\\\"\" start\n}\n\nget_ring_size() {\n    $BINDIR/scalarisctl -t client -e \"-noinput -eval \\\"jsonclient:get_ring_size(10000, $IP, $PORT, $SSL), halt(0).\\\"\" start\n}\n\nwait_for_ring_size() {\n    $BINDIR/scalarisctl -t client -e \"-noinput -eval \\\"jsonclient:wait_for_ring_size($RINGSIZE, 10000, $IP, $PORT, $SSL), halt(0).\\\"\" start\n}\n\nfix_paths\n\nuntil [ -z \"$1\" ]; do\n  OPTIND=1\n  case $1 in\n    \"--help\")\n      shift\n      usage 0;;\n    benchmark | ringsize | waitforringsize)\n      cmd=\"$1\"\n      shift;;\n    *)\n      if getopts \"hsx:i:p:r:\" optionName; then\n        case \"$optionName\" in\n            i) IP=$OPTARG;;\n            p) PORT=$OPTARG;;\n            r) RINGSIZE=$OPTARG;;\n            s) SSL=true;;\n            h) usage 0;;\n          [?]) echo \"Wrong parameter $1.\"\n               usage 1;;\n        esac\n        shift $(($OPTIND-1))\n      else\n        shift\n        usage 1\n      fi;;\n  esac\ndone\n\ncase $cmd in\n    benchmark)\n        jsonbenchmark;;\n    ringsize)\n        get_ring_size;;\n    waitforringsize)\n        wait_for_ring_size;;\n    *)\n        echo \"Unknown command: $cmd.\"\n        usage 1;;\nesac\n"
  },
  {
    "path": "bin/mgmt-server.sh",
    "content": "#!/bin/bash\n# Copyright 2007-2015 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\n# Script to start an empty Scalaris management server.\nID=0\nNAME=\"mgmt_server\"\nPORT=$((14195+$ID))\nYAWSPORT=$((8000+$ID))\n\nABSPATH=\"$(cd \"${0%/*}\" 2>/dev/null; echo \"$PWD\"/\"${0##*/}\")\"\nDIRNAME=`dirname $ABSPATH`\n\n# start a mgmt_server\nMGMT_NAME_AND_PORTS=\"-n mgmt_server -p 14194 -y 7999 \"\n$DIRNAME/scalarisctl $MGMT_NAME_AND_PORTS -t nostart -m \"$@\" start\n"
  },
  {
    "path": "bin/quorum_node.sh",
    "content": "#!/bin/bash\n# Copyright 2007-2012 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\n# Script to start a node, that joins a running Scalaris system.\nID=0\n\nif [ 1 == $# ]; then\n    ID=$1\n    shift;\nfi\nNAME=\"node$ID\"\nCSPORT=$((14195+$ID))\nYAWSPORT=$((8000+$ID))\n\nABSPATH=\"$(cd \"${0%/*}\" 2>/dev/null; echo \"$PWD\"/\"${0##*/}\")\"\nDIRNAME=`dirname $ABSPATH`\n\nif [ 1 -le $# ]; then\nERLFLAGS=\"$@\"\nelse\nERLFLAGS=\" \"\nfi\n\n$DIRNAME/scalarisctl -e \"$ERLFLAGS\" -n $NAME -p $CSPORT -y $YAWSPORT -t qourum -m -s start\n"
  },
  {
    "path": "bin/scalaris.cfg",
    "content": "% @copyright 2007-2017 Zuse Institute Berlin\n\n%    Licensed under the Apache License, Version 2.0 (the \"License\");\n%    you may not use this file except in compliance with the License.\n%    You may obtain a copy of the License at\n%\n%        http://www.apache.org/licenses/LICENSE-2.0\n%\n%    Unless required by applicable law or agreed to in writing, software\n%    distributed under the License is distributed on an \"AS IS\" BASIS,\n%    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%    See the License for the specific language governing permissions and\n%    limitations under the License.\n\n% $Id$\n\n%% @doc how many scalaris nodes per vm\n{nodes_per_vm,1}.\n\n%% DeadNode Cache Parameters\n%% @doc zombieDetectorInterval\n{zombieDetectorInterval, 10000}.\n%% @doc how many dead nodes have to be observed\n{zombieDetectorSize, 10}.\n\n%% log4erl log parameters (before log4erl is started, error_logger with\n%% parameters from preconfig will be used)\n{log_path, \"../log\"}.\n%% @doc Loglevel: debug < info < warn < error < fatal < none\n{log_level, warn}.\n{log_level_file, warn}.\n%% @doc Specify name without extension (\".txt\" will be used).\n{log_file_name_log4erl, \"scalaris_log4erl\"}.\n{log_file_size, 1048576}. % 1024*1024\n{log_file_rotations, 4}. % results in to up to 5 files\n\n%% @doc LogFormat, see Readme for log4erl\n{log_format,\"[%L] %l%n\"}.\n{log_format_file,\"%j %T [%L] %l%n\"}.\n\n\n%% @doc the initial number of milliseconds between two failure\n%% detector checks\n{failure_detector_interval, 3000}.\n\n%% @doc number of milliseconds between two re-registrations at the mgmt_server\n{reregister_interval, 600000}.\n\n%% @doc the replication degree of the system\n{replication_factor, 3}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Overlay/ring maintenance\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc the interval between two stabilization runs\n{stabilization_interval_base, 10000}.\n\n%% @doc the length of the successor list\n{succ_list_length, 9}.\n\n%% @doc the length of the predecessor list\n{pred_list_length, 9}.\n\n%% @doc the interval between two requests for new cyclon nodes to integrate\n%%      (in Seconds)\n{tman_cyclon_interval, 5}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% routing\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% Use a cache which circumvents routing protocols once the destination\n%% is known. However, this should only be used for benchmarking purposes etc.\n%% in cases where the routing overhead is not desired, as this mechanism is\n%% cannot handle node failures or ring reconfigurations.\n%% See dht_node_cache.erl for details.\n{cache_dht_nodes, false}.\n\n%% @doc the interval between two finger/pointer stabilization runs\n{pointer_base_stabilization_interval, 30000}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% redundancy\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% data redundancy strategy (default is replication)\n{redundancy_module, replication}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% replication\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% key_creation algorithm\n{key_creator, random}.\n%% for modr\n{availability_zone_id, 0}.\n\n%{key_creator, random_with_bit_mask}.\n% (randomkey band mask2) bor mask1\n%{key_creator_bitmask, {16#00000000000000000000000000000000, 16#3FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF}}.\n%{key_creator_bitmask, {16#40000000000000000000000000000000, 16#3FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF}}.\n%{key_creator_bitmask, {16#80000000000000000000000000000000, 16#3FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF}}.\n%{key_creator_bitmask, {16#C0000000000000000000000000000000, 16#3FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF}}.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% chord\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{chord_base, 64}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% (g)frtchord\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc enable active learning lookups\n{rt_frt_al, true}.\n\n%% @doc the interval between active learning lookups for rt_frt\n{rt_frt_al_interval, 5}. % in s\n\n%% @doc configure the maximum size for routing tables.\n{rt_frt_max_entries, 32}.\n\n%% @doc the interval between gossip is queried for a ring size estimate\n{rt_frt_gossip_interval, 10}. % in s\n\n%% @doc choose the strategy for rt refinement.\n%% - best_rt_reduction_ratio: send message if worse than reduction ratio for a best rt\n%% - convergent_rt_reduction_ratio: send message if worse than reduction ratio for a convergent rt\n{rt_frt_reduction_ratio_strategy, best_rt_reduction_ratio}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Gossip\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% Gossip based load aggregation\n\n{gossip_load_interval, 1000}.\n{gossip_load_min_cycles_per_round, 10}.\n{gossip_load_max_cycles_per_round, 1000}.\n{gossip_load_fanout, 1}.\n{gossip_load_convergence_count_best_values, 10}.\n{gossip_load_convergence_count_new_round, 20}.\n{gossip_load_convergence_epsilon, 5.0}.\n{gossip_load_number_of_buckets, 10}.\n{gossip_load_additional_modules, [lb_active_gossip_load_metric, lb_active_gossip_request_metric]}.\n\n%% Cyclon (gossip based peer sampling)\n\n{gossip_cyclon_interval, 4900}.\n{gossip_cyclon_cache_size, 20}.\n{gossip_cyclon_shuffle_length, 8}.\n\n\n%% Vivaldi (network coordinates)\n\n{gossip_vivaldi_interval, 10000}. % in milliseconds\n{gossip_vivaldi_dimensions, 2}.\n{gossip_vivaldi_count_measurements, 10}.\n{gossip_vivaldi_measurements_delay, 1}. % in seconds\n{gossip_vivaldi_latency_timeout, 60}. % in seconds\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Ganglia\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{ganglia_enable, false}.\n{ganglia_interval, 30000}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% DC clustering\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{dc_clustering_enable, false}.\n{dc_clustering_interval, 20000}.\n{dc_clustering_reset_interval, 60000}.\n{dc_clustering_radius, 10000.0}. % 10 ms radius\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% transactions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{range_read_timeout, 5000}.\n{tx_rtm_update_interval, 60000}.\n{tx_timeout, 5000}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% paxos\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{learner_noinit_timeout, 5500}.\n{acceptor_noinit_timeout, 5500}.\n\n%% When set to true, replica will try to repair itself\n%% if its sequence of write operations has gaps due to node failure.\n%% Repair process is triggered upon receiving a write.\n%% Only relevant if partial write operations are used (see write filter).\n{prbr_repair_on_write, true}.\n\n%% Maximum number of times a reading proposer retries its request without round\n%% number increments consecutively observing an inconsistent state without progress\n%% (i.e. the round numbers remain unchanged). If progress is made by some other proposer,\n%% the number of retries is resetted. Using this option prevents write-through\n%% cascades caused by concurrent readers.\n{read_attempts_without_progress, 10}.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% CRDT\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% The implementaiton of linearizable RSM for CRDTs.\n% wf_crdt_paxos -> wait free wrapper around crdt_paxos\n% crdt_paxos -> modified paxos optimized for CRDTs\n% lattice -> implementation of Generalized Lattice Agreement (not practical msg sizes increase over time)\n% zheng -> implementation of GLA_delta (Zheng et al: Linearizable Replicated State Machines With Lattice Agreement)\n{crdt_rsm, crdt_paxos}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% node move\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{move_use_incremental_slides, true}.\n{move_max_transport_entries, 1000}.\n{move_wait_for_reply_timeout, 30000}. % logs a warning if no reply is received within this many ms (will be checked every (<value> div 4) ms)\n{move_send_msg_retry_delay, 1000}. % after receiving a send_error, wait this many ms before re-trying\n{move_send_msg_retries, 2}. % logs a warning if a message could not be delivered after this many retries\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% node join\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{join_request_timeout, 1000}.\n{join_request_timeouts, 3}.\n{join_lookup_timeout, 3000}.\n{join_known_hosts_timeout, 1000}.\n{join_timeout, 30000}.\n{join_get_number_of_samples_timeout, 1000}.\n{join_lb_psv, lb_psv_gossip}.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% passive load balancing\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{lb_psv_samples, 2}. % rely on lb_psv_gossip to give O(log n) samples\n{lb_psv_split_fallback, split_address}. % or: keep_key\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% active load balancing\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{lb_active, false}.\n%% see lb_active.erl for available modules\n%% set to none to disable balancing\n{lb_active_module, lb_active_karger}.\n\n%% see lb_stats.erl for available metrics\n{lb_active_load_metric, reductions}.\n%% set to db_reads, db_writes, or db_all\n{lb_active_request_metric, db_all}.\n%% set to items or requests\n{lb_active_balance, requests}.\n%% fall back to items if no requests available\n{lb_active_fall_back_to_items, true}.\n%% use global information about metrics\n{lb_active_use_gossip, true}.\n%% threshold to calculate the minimum change in standard deviation to\n%% determine whether to initiate a load balancing operation\n{lb_active_gossip_stddev_threshold, 1}.\n%% between 0 and 1 to set the minimum ratio between the requests\n%% received on items and the number of total items at a node\n{lb_active_request_confidence, 0.1}.\n\n%% size of the request histogram\n%% (when higher more accuracy but slower processing)\n{lb_active_histogram_size, 100}.\n%% interval size for metric data\n{lb_active_monitor_resolution, 5000}.\n%% how often to collect metric data\n{lb_active_monitor_interval, 3000}.\n%% minimum number of old metric values to consider\n{lb_active_monitor_history_min, 3}.\n%% maximum number of old metric values to consider\n{lb_active_monitor_history_max, 12}.\n\n%% maximum of time to wait for pending ops\n{lb_active_wait_for_pending_ops, 10000}.\n\n%% Karger and Ruhl module (lb_active_karger)\n{lb_active_karger_interval, 10000}.\n%% 0 < epsilon < 0.25 or set to self_tuning\n{lb_active_karger_epsilon, 0.24}.\n%% random nodes to contact each round\n{lb_active_karger_rnd_nodes, 2}.\n%% make a decision after this timeout\n{lb_active_karger_simulation_timeout, 3000}.\n\n%% Directory module (lb_active_directories)\n%% how often to publish node information in directories\n{lb_active_directories_publish_interval, 10000}.\n%% how often to check a directory for load balancing\n{lb_active_directories_directory_interval, 10000}.\n%% maximum load balancing operations for a directory to initiate at once\n{lb_active_directories_max_transfer, 2}.\n\n% in case of jumps:\n% supports active load balancing with passive load balancing\n% when the node rejoins the ring\n{lb_active_and_psv, false}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% monitoring\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% for reported monitoring data, keep this many time slots (the slot length depends on the data)\n{monitor_timeslots_to_keep, 360}. % 10s time slot -> keep data for 1 hour\n{monitor_perf_interval, 30}. % start micro-benchmark every 30s (0 to disable)\n{rrd_timing_hist_size, 20}. % for every timing value, create a histogram of size 20\n\n% explicitly start OS monitor without active load balancing\n{start_os_mon, false}.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% network\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% port(s) for incoming communications, try one in this range\n% (overwritten in startup scripts)\n{port, {14195, 14198}}.\n\n% http web server port for debug interface, JSON interface\n% (overwritten in startup scripts)\n{yaws_port, 8000}.\n\n% the maximum size of a post request for the JSON-RPC\n% (either a number of bytes, or nolimit)\n{yaws_max_post_data, 1048576}. % 1MB\n\n% http web server docroot\n{docroot, \"../docroot\"}.\n\n% Allows to restrict access to the web debug interface (including the JSON-RPC!).\n% Expects a list of {\"User\", \"Password\"} tuples.\n% Note: This will effectively disable the JSON-RPC including the Python and Ruby\n%       APIs as they are not prepared to handle authentication yet.\n{yaws_auth, []}.\n%{yaws_auth, [{\"User\", \"Password\"}]}.\n\n%% the following two parameters should both be smaller than failure_detector_interval\n{tcp_send_timeout, 1000}.\n{tcp_connect_timeout, 1000}.\n{tcp_idle_timeout, 30000}.\n\n% default: listen on all interfaces:\n{listen_ip, {0,0,0,0}}.\n\n% default assume local testing\n{mgmt_server, {{127,0,0,1},14195,mgmt_server}}.\n{known_hosts, [{{127,0,0,1},14195, service_per_vm}, {{127,0,0,1},14196, service_per_vm},\n               {{127,0,0,1},14197, service_per_vm}, {{127,0,0,1},14198, service_per_vm}]}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% dht_node process modules (used for mockup_dht_node in unittests)\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{dht_node, dht_node}.\n\n%% collect lookup statistics?\n{dht_node_monitor, true}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% rrepair processes\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{rrepair_enabled, true}.\n%{rr_recon_method, trivial}.\n%{rr_recon_method, shash}.\n%{rr_recon_method, bloom}.\n{rr_recon_method, merkle_tree}.\n%{rr_recon_method, art}.\n\n% defines how many bits to use for encoding the version number in the 'trivial'\n% phases of all above algorithms\n{rr_recon_version_bits, 32}.\n% minimum number of bits to use for hash signatures other than the version number (> 0)\n% note that additionally, this value is limited by the used algorithm's max value\n% -> you should not change this!\n{rr_recon_min_sig_size, 1}.\n\n% when a non-graceful leave of a pred is found, resolve the missing items\n{rrepair_after_crash, true}.\n\n% probability of starting a synchronisation with a random node if trigger\n% has fired. ]0,100]\n{rr_trigger_probability, 33}.\n\n% integer duration until next triggering (milliseconds)\n%{rr_trigger_interval, 0}. % = periodic trigger disabled\n{rr_trigger_interval, 600000}. % =10*60*1000 (every 10 minutes)\n\n% garbage collector execution interval (milliseconds)\n{rr_gc_interval, 3600000}. % = 1*60*60*1000 (every hour)\n\n% time to live for sessions until they are garbage collected (milliseconds)\n{rr_session_ttl, 600000}. % =10*60*1000 (10 minutes)\n\n% set reconciliaton accuracy in terms of the expected number of errors (>0)\n{rr_recon_failure_rate, 0.1}.\n\n% defines the expected amount of differences between two nodes in percent (>= 0, =< 100)\n% this is used to tune the failure probabilities for the worst case with this delta\n{rr_recon_expected_delta, 5}.\n\n% specifies how many items to retrieve from the DB at once.\n% (tries to reduce the load of a single request in the dht_node process)\n{rr_max_items, 50000}. % pos_integer() | all.\n\n% set reconciliation algorithm name, parameters (merkle)\n{rr_merkle_branch_factor, 4}. % merkle tree branching factor thus number of childs per node (> 1)\n{rr_merkle_bucket_size, 3}. % size of merkle tree leaf buckets (> 0)\n% number of merkle trees a sync range is split into to reduce the number of\n% message rounds when differences exist (it increases message costs when no\n% differences exist though!)\n{rr_merkle_num_trees, 32}. % (> 0)\n\n% set reconciliation algorithm name, parameters (art)\n% rr_merkle_branch_factor and rr_merkle_bucket_size from above are also used\n{rr_art_inner_fpr, 0.001}.\n{rr_art_leaf_fpr, 0.01}.\n{rr_art_correction_factor, 2}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% leases\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%{leases, true}.\n{leases_delta, 10}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% autoscale and cloud modules\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{autoscale, false}.\n{autoscale_cloud_module, cloud_local}.\n{autoscale_alarms, []}.\n\n{autoscale_server, false}.\n{autoscale_server_plot_path, \"\"}.\n\n{cloud_local_min_vms, 0}.\n{cloud_local_max_vms, 10}.\n\n{cloud_ssh_hosts, []}.\n{cloud_ssh_path, \"\"}.\n{cloud_ssh_services, []}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% worker pool\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{wpool_maxw, 2}.\n{wpool_js, false}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% db backend\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{db_backend, db_ets}. % Can be db_ets, db_mnesia, db_toke, db_hanoidb\n\n{db_directory, \"../data\"}.\n\n{ensure_recover, false}. % if true this will check on startup whether the current configuration allows recovery\n\n{db_prbr_chunker, db_prbr}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% bitcask\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% interval in which a bitcask merge is triggered in seconds\n{bitcask_merge_interval, 300}.\n\n% initial time in seconds to wait before first merge. Note: Should be set to a value\n% that ensures that all nodes are available at this time.\n{bitcask_merge_offset, 60}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% extensions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n{extensions, []}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% ssl support\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n{ssl_mode, normal}. %% host keys or ca with passwords\n\n{comm_backend, gen_tcp}. % change to ssl\n\n{cacertfile, \"../ca.crt\"}.\n{certfile, \"../host.cert\"}.\n{keyfile, \"../host.key\"}.\n\n{ssl_password, \"PaSSw0rd\"}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% yaws ssl support\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n{yaws_ssl, false}. %% set to true\n\n{yaws_cacertfile, \"../ca.crt\"}.\n{yaws_certfile, \"../host.cert\"}.\n{yaws_keyfile, \"../host.key\"}.\n\n{yaws_sslpassword, \"PaSSw0rd\"}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% protocol scheduler\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% define if proto_sched shoud preserve intra-link message delivery\n% order by default. Note that this can be overridden on a per-execution\n% basis by using proto_sched:thread_num/3.\n{ordered_links, true}.\n\n"
  },
  {
    "path": "bin/scalaris.local.cfg.example",
    "content": "%% userdevguide-begin local_cfg:distributed\n% Insert the appropriate IP-addresses for your setup\n% as comma separated integers:\n% IP Address, Port, and label of the boot server\n{mgmt_server, {{127,0,0,1}, 14195, mgmt_server}}.\n\n% IP Address, Port, and label of a node which is already in the system\n{known_hosts, [{{127,0,0,1}, 14195, service_per_vm}]}.\n%% userdevguide-end local_cfg:distributed\n\n\n\n"
  },
  {
    "path": "bin/scalaris.local.ssl.bad.cfg",
    "content": "% default: listen on all interfaces:\n{listen_ip, {127,0,0,1}}.\n\n% default assume local testing\n{mgmt_server, {{127,0,0,1},14195,mgmt_server}}.\n{known_hosts, [{{127,0,0,1},14195, service_per_vm}]}.\n\n{ssl, true}.\n\n{ssl_mode, strict}.\n\n{comm_backend, ssl}.\n\n\n{ssl_password, \"password\"}.\n\n{cacertfile, \"../badca.crt\"}.\n{certfile, \"../badclient.crt\"}.\n{keyfile, \"../badclient.key\"}.\n\n"
  },
  {
    "path": "bin/scalaris.local.ssl.cfg",
    "content": "% default: listen on all interfaces:\n{listen_ip, {127,0,0,1}}.\n\n% default assume local testing\n{mgmt_server, {{127,0,0,1},14195,mgmt_server}}.\n{known_hosts, [{{127,0,0,1},14195, service_per_vm}]}.\n\n{ssl, true}.\n\n{ssl_mode, normal}.\n\n{comm_backend, ssl}.\n\n{certfile, \"../host.cert\"}.\n{keyfile, \"../host.key\"}.\n\n"
  },
  {
    "path": "bin/scalaris.local.ssl.good.cfg",
    "content": "% default: listen on all interfaces:\n{listen_ip, {127,0,0,1}}.\n\n% default assume local testing\n{mgmt_server, {{127,0,0,1},14195,mgmt_server}}.\n{known_hosts, [{{127,0,0,1},14195, service_per_vm}]}.\n\n{ssl, true}.\n\n{ssl_mode, strict}.\n\n{comm_backend, ssl}.\n\n\n{ssl_password, \"password\"}.\n\n{cacertfile, \"../goodca.crt\"}.\n{certfile, \"../goodclient.crt\"}.\n{keyfile, \"../goodclient.key\"}.\n\n"
  },
  {
    "path": "bin/scalarisctl.conf",
    "content": "# Copyright 2011-2018 Zuse Institute Berlin\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\"); you\n# may not use this file except in compliance with the License.  You\n# may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n# implied.  See the License for the specific language governing\n# permissions and limitations under the License.\n\n# Scalaris settings\n\n"
  },
  {
    "path": "bin/scalarisctl.in",
    "content": "#!/bin/bash\n# Copyright 2007-2021 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nprefix=@prefix@\nexec_prefix=@exec_prefix@\ndatarootdir=@datarootdir@\nERL=@ERL@\nEPMD=@EPMD@\nSCREEN=@SCREEN@\n\n# /bin\nBINDIR=@bindir@\n# /etc/scalaris/\nETCDIR=@sysconfdir@/scalaris\n# /lib/scalaris/\nSCALARISDIR=${prefix}/lib/scalaris/\n# /lib/scalaris/ebin\nBEAMDIR=${SCALARISDIR}/ebin\n# /lib/scalaris/docroot\nDOCROOTDIR=${SCALARISDIR}/docroot\n# /share/doc/scalaris\nDOCDIR=@docdir@\n# /share/java\nJAVADIR=@datarootdir@/java\n# may be overridden with the -l parameter\nLOGDIR=\"$HOME/.scalaris/log\"\n# /usr/bin/scalaris\nSCALARISCLIENT_JAVA=${exec_prefix}/bin/scalaris\nSCALARISCLIENT_PYTHON=${exec_prefix}/bin/scalaris-python\nSCALARISCLIENT_PYTHON3=${exec_prefix}/bin/scalaris-python3\nSCALARISCLIENT_RUBY=${exec_prefix}/bin/scalaris-ruby\n\nERLANG_ADDITIONAL_FLAGS=\"@ERLANG_ERLANGJS_FLAGS@ @ERLANG_BITCASK_FLAGS@ @ERLANG_HANOIDB_FLAGS@ @ERLANG_TOKE_FLAGS@\"\n\n# By default, no emulator flags are used.\n# This is still useful for passing flags to erl though.\nERL_SCHED_FLAGS=${ERL_SCHED_FLAGS-\"\"}\n\n# default values (commandline options change these parameters)\nNODE_NAME=\"\"\nERLANG_HOSTNAME=${ERLANG_HOSTNAME-\"\"}\ndaemonize=0\nfirst=0\nverbose=0\nport=\"\"\nyaws_port=\"\"\njoin_at=\"\"\njoin_at_list=\"\"\nstart_mgmt_server=0\nDIST_ERL_PORT=\"\"\nDAEMONIZE_SCREEN=0\ncmd=\"\"\nNODES_PER_VM=\"\"\nSCALARIS_LOCAL_CFG=\"scalaris.local.cfg\"\nVERBATIM=\"\"\n\nusage(){\n    echo \"usage: scalarisctl [options] <cmd>\"\n    echo \" options:\"\n    echo \"    -h          - print this help message\"\n    echo \"    -b          - pass verbatim string to Scalaris\"\n    echo \"    -d          - daemonize\"\n    echo \"    --screen    - if daemonized, put an interactive session into screen\"\n    echo \"    -e <params> - pass additional parameters to erl\"\n    echo \"    -n <name>   - Erlang process name (starts distributed Erlang required for Java API)\"\n    echo \"    -c <cookie> - Erlang cookie to use (for distributed Erlang)\"\n    echo \"    -p <port>   - TCP port for the Scalaris node\"\n    echo \"    -y <port>   - TCP port for the built-in webserver (YAWS)\"\n    echo \"    -k <key>    - join at the given key\"\n    echo \"    -j <list>   - join at the given list of keys\"\n    echo \"    -v          - verbose\"\n    echo \"    -l <dir>    - use this logdir base directory (will create a sub-folder\"\n    echo \"                  per node)\"\n    echo \"    --dist-erl-port <port>\"\n    echo \"                - (single) port distributed erlang listens on\"\n    echo \"    --nodes-per-vm <number>\"\n    echo \"                - number of Scalaris nodes to start inside the VM\"\n    echo \"    -t <stype>  - select start type: first|joining|quorum|recover|nostart|first_nostart|client\"\n    echo \"    -m          - start global Scalaris management server\"\n    echo \"    -s <file>   - use the given file name as the scalaris.local.cfg\"\n    echo \" <cmd>:\"\n    echo \"    checkinstallation\"\n    echo \"                - test installation\"\n    echo \"    start       - start services (see -m and -t)\"\n    echo \"    stop        - stop a scalaris process defined by its name (see -n)\"\n    echo \"    restart     - restart a scalaris process by its name (see -n)\"\n    echo \"\"\n    echo \"    list        - list locally running Erlang VMs\"\n    echo \"    debug       - connect to a running node via an Erlang shell\"\n    echo \"    dbg-check-ring <ring-size> <attempts>\"\n    echo \"                - checks (up to) <attempts> times whether Scalaris has\"\n    echo \"                  <ring-size> nodes and the ring maintenance has settled\"\n    echo \"                  (requires a mgmt_server)\"\n    echo \"\"\n    exit $1\n}\n\nfix_paths() {\n    ABSPATH=\"$(cd \"${0%/*}\" 2>/dev/null; echo \"$PWD\"/\"${0##*/}\")\"\n    DIRNAME=`dirname $ABSPATH`\n    # is this a source checkout or an (rpm/deb/manual) installation?\n    if [ \"$DIRNAME\" != \"$BINDIR\" -a \"$DIRNAME\" != \"/bin\" ]; then\n        # scalaris\n        SCALARISDIR=`dirname $DIRNAME`\n        # /etc/scalaris/\n        ETCDIR=$SCALARISDIR/bin\n        # scalaris/ebin\n        BEAMDIR=$SCALARISDIR/ebin\n        # scalaris/docroot\n        DOCROOTDIR=$SCALARISDIR/docroot\n        # scalaris/log\n        LOGDIR=$SCALARISDIR/log\n        # /usr/bin/scalaris\n        SCALARISCLIENT_JAVA=\"$SCALARISDIR/java-api/scalaris --noconfig\"\n        SCALARISCLIENT_PYTHON=\"$SCALARISDIR/python-api/scalaris\"\n        SCALARISCLIENT_PYTHON3=\"$SCALARISDIR/python3-api/scalaris\"\n        SCALARISCLIENT_RUBY=\"$SCALARISDIR/ruby-api/scalaris\"\n        SCALARISCLIENT_CPP=\"$SCALARISDIR/cpp-api/scalaris\"\n    fi\n}\n\nget_hostname() {\n    # Uses the OS hostname from \"hostname -f\" for the Erlang long node name.\n    if [ -z \"$ERLANG_HOSTNAME\" ] ; then\n        ERLANG_HOSTNAME=\"`get_sys_hostname`\"\n    fi\n    # Alternative using the hostname that erlang chooses if set up to start with a long node\n    # name and falls back to \"hostname -f\" if this is not possible:\n    # NOTE: In this case, the Java-API (if used as a library) is not always able\n    #       to guess the host name that Erlang expects and connections may fail.\n    #       Therefore, the convenience node spec \"<node>@localhost\" may not be used.\n#     if [ -z \"$ERLANG_HOSTNAME\" ] ; then\n#         ERLANG_HOSTNAME=\"`get_erlang_hostname`\"\n#         if [ -z \"$ERLANG_HOSTNAME\" ] ; then\n#             # could not start erlang with long node name (no FQDN?)\n#             ERLANG_HOSTNAME=\"`get_sys_hostname`\"\n#         fi\n#     fi\n    echo \"$ERLANG_HOSTNAME\"\n}\n\nget_erlang_hostname() {\n    MY_HOSTNAME_ERL=\"`$ERL -name foo$$ -noinput -eval 'N=erlang:atom_to_list(node()),io:format(\"~s\", [string:sub_string(N, string:chr(N, $@)+1)]), halt(0).' 2>/dev/null`\"\n    if [ $? -ne 0 ] ; then\n        MY_HOSTNAME_ERL=\n    fi\n    echo \"$MY_HOSTNAME_ERL\"\n}\n\nget_sys_hostname() {\n    hostname -f\n}\n\nfix_node_name() {\n    # if the given node name includes an '@', leave untouched\n    if [ \"\" != \"$NODE_NAME\" ]; then\n        if ! echo $NODE_NAME | grep \"@\" >/dev/null ; then\n            NODE_NAME=\"$NODE_NAME@`get_hostname`\"\n        fi\n    fi\n}\n\n# only get cookie from scalarisctl.conf if installed!\nget_cookie() {\n    ABSPATH=\"$(cd \"${0%/*}\" 2>/dev/null; echo \"$PWD\"/\"${0##*/}\")\"\n    DIRNAME=`dirname $ABSPATH`\n    if [ \"$DIRNAME\" = \"$BINDIR\" ]; then\n      # system config\n      . ${ETCDIR}/scalarisctl.conf\n\n      # load user scalaris configuration (overrides system config)\n      if [ -f \"$HOME/.scalaris/scalarisctl.conf\" ] ; then\n        . \"$HOME/.scalaris/scalarisctl.conf\"\n      fi\n    fi\n    echo -n ${cookie:-\"\"}\n}\n\ncleanup_before_checkinstallation() {\n  if [ \"\" != \"$NODE_NAME\" ]; then\n    for pid in `ps aux | grep beam.smp | grep $NODE_NAME | awk '{print $2}'`;\n    do\n      #echo \"Killing $pid...\"\n      kill $pid\n    done\n  fi\n}\n\ncheckinstallation() {\n    if [ \"\" = \"$NODE_NAME\" ]; then\n        NODE_NAME=`cat /dev/urandom | env LC_CTYPE=C tr -dc 'a-z' | fold -w 8 | head -n 1`\n        fix_node_name\n    fi\n    echo \"Running cleanup...\"\n    cleanup_before_checkinstallation\n    echo -n \"Running basic tests...\"\n    MSG_FAILED=\"\\x1b[1;31mFAILED\\x1b[0m\" #red\n    MSG_ATTENTION=\"\\x1b[1;34mATTENTION\\x1b[0m\" #blue\n    MSG_SUCCESS=\"\\x1b[1;32mSUCCESS\\x1b[0m\" #green\n    MSG_NOTINSTALLED=\"\\x1b[1;34mNOT INSTALLED\\x1b[0m\" #blue\n    MSG_DONE=\"DONE\"\n    # checking for $HOME\n    if [ \"x$HOME\" = \"x\" ]; then\n        echo -e \"$MSG_FAILED\"\n        echo \"ERROR: \\$HOME is not set\"\n        echo \"We were trying to read: \\$HOME\"\n        exit 1\n    fi\n    if [ ! -d \"$HOME\" ]; then\n        echo -e \"$MSG_FAILED\"\n        echo \"ERROR: \\$HOME does not point to a valid directory\"\n        echo \"We were trying to read: \\$HOME\"\n        exit 1\n    fi\n    $ERL -noinput -eval 'halt(0).'\n    FAILED=$?\n    if [ $FAILED -ne 0 ]; then\n        echo -e \"$MSG_FAILED\"\n        echo \"ERROR: erl could not be run\"\n        echo \"We were trying to run: erl -noinput -eval 'halt(0).'\"\n        exit 1\n    fi\n\n    if [ ! -s `which erl` ]; then\n        echo -e \"$MSG_FAILED\"\n        echo \"ERROR: your erl executable is of size zero bytes: `which erl`\"\n        echo \"ls -l `which erl`\"\n        exit 1\n    fi\n\n    $ERL -noinput -name \"checkinstallation\" -eval 'halt(0).' > /dev/null 2>&1\n    FAILED=$?\n    if [ $FAILED -ne 0 ]; then\n        echo -e \"$MSG_ATTENTION\"\n        echo \"ERROR: erl could not create a Erlang VM named 'checkinstallation'\"\n        echo \"Please check the configuration of your hostnames\"\n        echo \"Are you already running another checkinstallation?\"\n        echo \"'scalarisctl list' gives you a list of currently registered Erlang VMs\"\n        echo \"We were trying to run: erl -noinput -name \\\"checkinstallation\\\" -eval 'halt(0).'\"\n        echo -n \"Continuing basic tests...\"\n    fi\n\n    $ERL -noinput -name \"checkinstallation@`get_hostname`\" -eval 'halt(0).' > /dev/null 2>&1\n    FAILED=$?\n    if [ $FAILED -ne 0 ]; then\n        echo -e \"$MSG_ATTENTION\"\n        echo \"ERROR: erl could not create a Erlang VM named 'checkinstallation@`get_hostname`'\"\n        echo \"Are you already running another checkinstallation?\"\n        echo \"'scalarisctl list' gives you a list of currently registered Erlang VMs\"\n        echo \"We were trying to run: erl -noinput -name \\\"checkinstallation$NODE_NAME@`get_hostname`\\\" -eval 'halt(0).'\"\n        echo -n \"Continuing basic tests...\"\n    fi\n\n    $ERL -noinput -eval 'case catch crypto:start() of ok -> halt(0); _ -> halt(1) end.' > /dev/null 2>&1\n    FAILED=$?\n    if [ $FAILED -ne 0 ]; then\n        echo -e \"$MSG_FAILED\"\n        echo \"ERROR: erlang could not start the crypto module\"\n        echo \"We were trying to run: $ERL -noinput -eval 'case catch crypto:start() of ok -> halt(0); _ -> halt(1) end.'\"\n        exit 1\n    fi\n\n    if [ -e $ETCDIR/scalaris.cfg ]; then\n        $ERL -noinput -eval \"case file:consult(\\\"$ETCDIR/scalaris.cfg\\\") of {error, _} -> halt(1); _ -> halt(0) end\" > /dev/null 2>&1\n        FAILED=$?\n        if [ $FAILED -ne 0 ]; then\n            echo -e \"$MSG_FAILED\"\n            echo \"ERROR: there is a syntax error in: $ETCDIR/scalaris.cfg\"\n            echo \"We were trying to run: $ERL -noinput -eval \\\"A = file:consult(\\\"$ETCDIR/scalaris.cfg\\\"), io:format(\\\"~p~n\\\", [A]), halt(0) end\\\"\"\n            exit 1\n        fi\n    else\n        echo -e \"$MSG_FAILED\"\n        echo \"ERROR: there is no config file: $ETCDIR/scalaris.cfg\"\n        exit 1\n    fi\n\n    if [ -e $ETCDIR/$SCALARIS_LOCAL_CFG ]; then\n        $ERL -noinput -eval \"case file:consult(\\\"$ETCDIR/$SCALARIS_LOCAL_CFG\\\") of {error, _} -> halt(1); _ -> halt(0) end\" > /dev/null 2>&1\n        FAILED=$?\n        if [ $FAILED -ne 0 ]; then\n            echo -e \"$MSG_FAILED\"\n            echo \"ERROR: there is a syntax error in: $ETCDIR/$SCALARIS_LOCAL_CFG\"\n            echo \"We were trying to run: $ERL -noinput -eval \\\"A = file:consult(\\\"$ETCDIR/$SCALARIS_LOCAL_CFG\\\"), io:format(\\\"~p~n\\\", [A]), halt(0) end\\\"\"\n            exit 1\n        fi\n    fi\n\n    $ERL -noinput -pa $BEAMDIR -eval 'case {code:ensure_loaded(mgmt_server), code:ensure_loaded(dht_node)} of {{module,mgmt_server},{module,dht_node}} -> halt(0); X -> halt(1) end.' > /dev/null 2>&1\n    FAILED=$?\n    if [ $FAILED -ne 0 ]; then\n        echo -e \"$MSG_FAILED\"\n        echo \"ERROR: could not find Scalaris' beam files\"\n        echo \"We were trying to run: $ERL -noinput -pa $BEAMDIR -eval 'case {code:ensure_loaded(mgmt_server), code:ensure_loaded(dht_node)} of {{module,mgmt_server},{module,dht_node}} -> halt(0); X -> halt(1) end.'\"\n        exit 1\n    fi\n\n    HOSTNAME_ERL=`get_erlang_hostname`\n    HOSTNAME_SYS=`get_sys_hostname`\n    if [ \"$HOSTNAME_ERL\" != \"$HOSTNAME_SYS\" ]; then\n        echo -e \"$MSG_ATTENTION\"\n        echo \"ERROR: the hostnames reported by Erlang and the OS do not match.\"\n        echo \"erlang (get_erlang_hostname): \"\n        echo \"     -> $HOSTNAME_ERL\"\n        echo \"system (get_sys_hostname):\"\n        echo \"     -> $HOSTNAME_SYS\"\n        echo \" If debugging does not work, try one of the following solutions:\"\n        echo \" - during start, specify a full node name using IPs with '-n node@<IP>'\"\n        echo \" - set environment variable ERLANG_HOSTNAME to your IP\"\n        if [ -n \"$HOSTNAME_ERL\" ] ; then\n            echo \" - during start, specify a full node name using the erlang hostname with '-n node@$HOSTNAME_ERL'\"\n            echo \" - set environment variable ERLANG_HOSTNAME to the erlang hostname\"\n        fi\n        echo \" - fix your hostname setup system-wide\"\n        echo \" Please refer to your system's manual to set your hostname, e.g. change the\"\n        echo \" values in /etc/hosts or /etc/hostname, and check that it is consistent with\"\n        echo \" the DNS.\"\n        exit 1\n    fi\n\n    $SCALARISCLIENT_JAVA -h > /dev/null 2>&1\n    FAILED=$?\n    if [ $FAILED -ne 0 ]; then\n        HAS_JAVACLIENT=0\n        echo -e \"$MSG_ATTENTION\"\n        echo \"INFO: could not find Scalaris' Java-API files.\"\n        echo \"You won't be able to use the 'scalaris' command line script to access Scalaris.\"\n        echo \"  'make java' will build the Java-API\"\n        echo \"We were trying to run: $SCALARISCLIENT_JAVA -h\"\n        echo -n \"Continuing basic tests...\"\n    else\n        HAS_JAVACLIENT=1\n        HOSTNAME_CTL=`get_hostname`\n        HOSTNAME_JAVA=`$SCALARISCLIENT_JAVA -lh --noerl`\n        if [ \"$HOSTNAME_CTL\" != \"$HOSTNAME_JAVA\" ]; then\n            echo -e \"$MSG_ATTENTION\"\n            echo \"WARNING: the hostnames reported by Erlang, OS and Java do not match.\"\n            echo \" You may run into trouble with the automatic '<node>@localhost' to\"\n            echo \" '<node>@<hostname>' conversion done by the Java-API, e.g. if used through\"\n            echo \" scalaris.jar. Also, the java unit tests will probably not work.\"\n            echo \" The java-api/scalaris script will work around this by using the same mechanism\"\n            echo \" that scalarisctl uses to start nodes.\"\n            echo \" If you implement your own application either do not use \\\"@localhost\\\" in the\"\n            echo \" scalaris.properties file or set the scalaris.erlang.nodename system property\"\n            echo \" with the correct hostname (only the domain name part after the \\\"@\\\") or fix\"\n            echo \" your hostname setup system-wide.\"\n            echo \" Please refer to your system's manual to set your hostname, e.g. change the\"\n            echo \" values in /etc/hosts or /etc/hostname, and check that it is consistent with\"\n            echo \" the DNS.\"\n            echo \"scalarisctl: \"\n            echo \"     -> $HOSTNAME_CTL\"\n            echo \"java:   $SCALARISCLIENT_JAVA -lh\"\n            echo \"     -> $HOSTNAME_JAVA\"\n            echo -n \"Continuing basic tests...\"\n        fi\n    fi\n\n    CHECKCONFIGMSG=`$ERL -noinput -pa $BEAMDIR \\\n         -scalaris config \\\"$ETCDIR/scalaris.cfg\\\" \\\n         -scalaris local_config \\\"$ETCDIR/$SCALARIS_LOCAL_CFG\\\" \\\n         -scalaris start_type nostart \\\n         -s scalaris load -eval \"config:init([]), halt(0).\" 2>&1`\n    FAILED=$?\n    if [ $FAILED -ne 0 ]; then\n        echo -e \"$MSG_FAILED\"\n        echo \"$CHECKCONFIGMSG\"\n        echo \"ERROR: some parameters in the config files are wrong or missing\"\n        echo \"We were trying to run: \\\n          $ERL -noinput -pa $BEAMDIR \\\n-scalaris config \\\"$ETCDIR/scalaris.cfg\\\" \\\n-scalaris local_config \\\"$ETCDIR/$SCALARIS_LOCAL_CFG\\\" \\\n-scalaris start_type nostart \\\n-s scalaris load -eval \\\"config:init([]), halt(0).\\\"\"\n        exit 1\n    fi\n\n\n    CHECKCONFIGMSG=`$ERL $ERL_SCHED_FLAGS -noinput -eval 'halt(0).'`\n    FAILED=$?\n    if [ $FAILED -ne 0 ]; then\n        echo -e \"$MSG_ATTENTION\"\n        echo \"$CHECKCONFIGMSG\"\n        echo \"WARNING: could not start erlang with the given $ERL_SCHED_FLAGS.\"\n        echo \" Defaults will be used when running Scalaris.\"\n        echo \" Use the ERL_SCHED_FLAGS environment variable to override the scheduler flags.\"\n        echo \" You may need to include the +sct flag to set the CPU topology manually.\"\n        echo \" Be default, not scheduler flags are used.\"\n        echo \"We were trying to run: $ERL $ERL_SCHED_FLAGS -noinput -eval 'halt(0).'\"\n        echo -n \"Continuing basic tests...\"\n    fi\n\n    CHECKCONFIGMSG=`$ERL -noinput -sasl sasl_error_logger false -boot start_sasl \\\n        -eval \"ok = application:start(erlang_js), {ok, JS} = js_driver:new(), ok = js:define(JS, <<\\\"var addOne = function(a){ return a + 1; }\\\">>), {ok,4} = js:call(JS, <<\\\"addOne\\\">>, [3]), halt(0).\"  > /dev/null 2>&1`\n    FAILED=$?\n    if [ $FAILED -ne 0 ]; then\n        echo -e \"$MSG_ATTENTION\"\n        echo \"$CHECKCONFIGMSG\"\n        echo \"WARNING: unable to start the erlang_js application\"\n        echo \" You won't be able to use JavaScript Map-Reduce jobs\"\n        echo \"We were trying to run: $ERL -noinput -sasl sasl_error_logger false -boot start_sasl -eval \\\"ok = application:start(erlang_js), {ok, JS} = js_driver:new(), ok = js:define(JS, <<\\\"var addOne = function(a){ return a + 1; }\\\">>), {ok,4} = js:call(JS, <<\\\"addOne\\\">>, [3]), halt(0).\\\"\"\n        echo -n \"Continuing basic tests...\"\n    fi\n\n    echo -e \"$MSG_DONE\"\n\n    echo \"Running Scalaris run-time tests...\"\n    # test runtime\n    if [ -z \"$yaws_port\" ]; then\n        echo \"  NOTE: yaws port not specified (you can specify it using -y <port> for $0).\"\n        echo \"        Python, Python3, Ruby, C++ API tests may fail if the port is different in\"\n        echo \"        one of the config files.\"\n    else\n        export SCALARIS_JSON_URL=\"http://localhost:$yaws_port\" # for single operations\n        export SCALARIS_JSON_URLS=\"http://localhost:$yaws_port\" # for benchmarks\n    fi\n    if [ -n \"$port\" ]; then\n        export SCALARIS_ADDITIONAL_PARAMETERS=\"-scalaris mgmt_server {{127,0,0,1},$port,mgmt_server} -scalaris known_hosts [{{127,0,0,1},$port,service_per_vm}]\"\n    fi\n\n    export SCALARIS_JAPI_NODE=\"$NODE_NAME\"\n    $0 -e \"$erl_flags\" -c \"chocolate chip cookie\" -d -t first -m -n \"$NODE_NAME\" -p \"$port\" -y \"$yaws_port\" start > /dev/null 2>&1\n    FAILED=$?\n    if [ $FAILED -ne 0 ]; then\n        echo -e \"  Starting Scalaris \\x1b[1;31mFAILED\\x1b[0m\"\n        echo \"  we were trying to run: $0 -d -t first -m start\"\n        # try to stop anyway (it could have been a false alarm):\n        $0 -e \"$erl_flags\" -c \"chocolate chip cookie\" -d -t first -m -n \"$NODE_NAME\" -p \"$port\" -y \"$yaws_port\" stop > /dev/null 2>&1\n        exit 1\n    fi\n\n    # Check if Scalaris node has been started.\n    sleep 2s # let Scalaris settle\n    max_tries=58\n    try=0\n    while [ $try -lt $max_tries ] ; do\n      scalaris_nodes_count_actual=\"$(scalariscall \"Res = rpc:call('$NODE_NAME', admin, number_of_nodes, []), io:format(\\\"~p~n\\\", [Res]), case Res of {badrpc, _Reason} -> halt(1); _ -> halt(0) end.\")\"\n\n      if [ $? -eq 0 ] ; then\n        if [ \"$scalaris_nodes_count_actual\" -ge 1 ] ; then\n          break\n        fi\n      fi\n\n      try=$((10#$try + 1))\n      sleep 1s\n\n      if [ $try -eq $max_tries ] ; then\n        echo -e \"  Starting Scalaris \\x1b[1;31mFAILED\\x1b[0m\"\n        echo \"  (result: ${scalaris_nodes_count_actual}).\"\n        echo \"  we were trying to run: $0 -t first -m -d start\"\n        # try to stop anyway (it could have been a false alarm):\n        $0 -e \"$erl_flags\" -c \"chocolate chip cookie\" -d -t first -m -n \"$NODE_NAME\" -p \"$port\" -y \"$yaws_port\" stop > /dev/null 2>&1\n        exit 1\n      fi\n    done\n\n    # testing Java:\n    echo -n \"  Java-API ... \"\n    if [ \"$HAS_JAVACLIENT\" -eq 1 ]; then\n        # simple read/write\n        KEY=\"checkinstallation-java-$RANDOM\"\n        JAVA_OUTPUT=`echo -e \"> $SCALARISCLIENT_JAVA -r ${KEY}\"`\n        JAVAREAD1_OUTPUT=`$SCALARISCLIENT_JAVA -r ${KEY} 2>&1`\n        echo \"$JAVAREAD1_OUTPUT\" | grep \"failed with not found\" > /dev/null 2>&1\n        JAVAREAD1_FAILED=$?\n        JAVA_OUTPUT=`echo -e \"$JAVA_OUTPUT\\n$JAVAREAD1_OUTPUT\"`\n\n        JAVA_OUTPUT=`echo -e \"$JAVA_OUTPUT\\n> $SCALARISCLIENT_JAVA -w ${KEY} 1\"`\n        JAVAWRITE1_OUTPUT=`$SCALARISCLIENT_JAVA -w ${KEY} 1 2>&1`\n        JAVAWRITE1_FAILED=$?\n        JAVA_OUTPUT=`echo -e \"$JAVA_OUTPUT\\n$JAVAWRITE1_OUTPUT\"`\n\n        JAVA_OUTPUT=`echo -e \"$JAVA_OUTPUT\\n> $SCALARISCLIENT_JAVA -r ${KEY}\"`\n        JAVAREAD2_OUTPUT=`$SCALARISCLIENT_JAVA -r ${KEY} 2>&1`\n        JAVAREAD2_FAILED=$?\n        JAVA_OUTPUT=`echo -e \"$JAVA_OUTPUT\\n$JAVAREAD2_OUTPUT\"`\n\n        if [ $JAVAREAD1_FAILED -ne 0 -o $JAVAWRITE1_FAILED -ne 0 -o $JAVAREAD2_FAILED -ne 0 ]; then\n            JAVA_FAILED=1\n            JAVA_RESULT=\"$MSG_FAILED\"\n        else\n            JAVA_FAILED=0\n            JAVA_RESULT=\"$MSG_SUCCESS\"\n        fi\n    else\n        JAVA_FAILED=0\n        JAVA_RESULT=\"$MSG_NOTINSTALLED\"\n    fi\n    echo -e \"$JAVA_RESULT\"\n    if [ \"$verbose\" -eq 1 -o \"$JAVA_FAILED\" -ne 0 ]; then\n        echo \"$JAVA_OUTPUT\"\n    fi\n\n    # testing Python:\n    echo -n \"  Python-API ... \"\n    if [ -f \"$SCALARISCLIENT_PYTHON\" ]; then\n        # simple read/write\n        KEY=\"checkinstallation-python-$RANDOM\"\n        PYTHON_OUTPUT=`echo -e \"> $SCALARISCLIENT_PYTHON -r ${KEY}\"`\n        PYTHONREAD1_OUTPUT=`$SCALARISCLIENT_PYTHON -r ${KEY} 2>&1`\n        echo \"$PYTHONREAD1_OUTPUT\" | grep \"failed with not_found\" > /dev/null 2>&1\n        PYTHONREAD1_FAILED=$?\n        PYTHON_OUTPUT=`echo -e \"$PYTHON_OUTPUT\\n$PYTHONREAD1_OUTPUT\"`\n\n        PYTHON_OUTPUT=`echo -e \"$PYTHON_OUTPUT\\n> $SCALARISCLIENT_PYTHON -w ${KEY} 1\"`\n        PYTHONWRITE1_OUTPUT=`$SCALARISCLIENT_PYTHON -w ${KEY} 1 2>&1`\n        PYTHONWRITE1_FAILED=$?\n        PYTHON_OUTPUT=`echo -e \"$PYTHON_OUTPUT\\n$PYTHONWRITE1_OUTPUT\"`\n\n        PYTHON_OUTPUT=`echo -e \"$PYTHON_OUTPUT\\n> $SCALARISCLIENT_PYTHON -r ${KEY}\"`\n        PYTHONREAD2_OUTPUT=`$SCALARISCLIENT_PYTHON -r ${KEY} 2>&1`\n        PYTHONREAD2_FAILED=$?\n        PYTHON_OUTPUT=`echo -e \"$PYTHON_OUTPUT\\n$PYTHONREAD2_OUTPUT\"`\n\n        if [ $PYTHONREAD1_FAILED -ne 0 -o $PYTHONWRITE1_FAILED -ne 0 -o $PYTHONREAD2_FAILED -ne 0 ]; then\n            PYTHON_FAILED=1\n            PYTHON_RESULT=\"$MSG_FAILED\"\n        else\n            PYTHON_FAILED=0\n            PYTHON_RESULT=\"$MSG_SUCCESS\"\n        fi\n    else\n        PYTHON_FAILED=0\n        PYTHON_RESULT=\"$MSG_NOTINSTALLED\"\n    fi\n    echo -e \"$PYTHON_RESULT\"\n    if [ \"$verbose\" -eq 1 -o \"$PYTHON_FAILED\" -ne 0 ]; then\n        echo \"$PYTHON_OUTPUT\"\n    fi\n\n    # testing Python3:\n    echo -n \"  Python3-API ... \"\n    if [ -f \"$SCALARISCLIENT_PYTHON3\" ]; then\n        # simple read/write\n        KEY=\"checkinstallation-python3-$RANDOM\"\n        PYTHON3_OUTPUT=`echo -e \"> $SCALARISCLIENT_PYTHON3 -r ${KEY}\"`\n        PYTHON3READ1_OUTPUT=`$SCALARISCLIENT_PYTHON3 -r ${KEY} 2>&1`\n        echo \"$PYTHON3READ1_OUTPUT\" | grep \"failed with not_found\" > /dev/null 2>&1\n        PYTHON3READ1_FAILED=$?\n        PYTHON3_OUTPUT=`echo -e \"$PYTHON3_OUTPUT\\n$PYTHON3READ1_OUTPUT\"`\n\n        PYTHON3_OUTPUT=`echo -e \"$PYTHON3_OUTPUT\\n> $SCALARISCLIENT_PYTHON3 -w ${KEY} 1\"`\n        PYTHON3WRITE1_OUTPUT=`$SCALARISCLIENT_PYTHON3 -w ${KEY} 1 2>&1`\n        PYTHON3WRITE1_FAILED=$?\n        PYTHON3_OUTPUT=`echo -e \"$PYTHON3_OUTPUT\\n$PYTHON3WRITE1_OUTPUT\"`\n\n        PYTHON3_OUTPUT=`echo -e \"$PYTHON3_OUTPUT\\n> $SCALARISCLIENT_PYTHON3 -r ${KEY}\"`\n        PYTHON3READ2_OUTPUT=`$SCALARISCLIENT_PYTHON3 -r ${KEY} 2>&1`\n        PYTHON3READ2_FAILED=$?\n        PYTHON3_OUTPUT=`echo -e \"$PYTHON3_OUTPUT\\n$PYTHON3READ2_OUTPUT\"`\n\n        if [ $PYTHON3READ1_FAILED -ne 0 -o $PYTHON3WRITE1_FAILED -ne 0 -o $PYTHON3READ2_FAILED -ne 0 ]; then\n            PYTHON3_FAILED=1\n            PYTHON3_RESULT=\"$MSG_FAILED\"\n        else\n            PYTHON3_FAILED=0\n            PYTHON3_RESULT=\"$MSG_SUCCESS\"\n        fi\n    else\n        PYTHON3_FAILED=0\n        PYTHON3_RESULT=\"$MSG_NOTINSTALLED\"\n    fi\n    echo -e \"$PYTHON3_RESULT\"\n    if [ \"$verbose\" -eq 1 -o \"$PYTHON3_FAILED\" -ne 0 ]; then\n        echo \"$PYTHON3_OUTPUT\"\n    fi\n\n    # testing Ruby:\n    echo -n \"  Ruby-API ... \"\n    if [ -f \"$SCALARISCLIENT_RUBY\" ]; then\n        RUBY_OUTPUT=`echo -e \"> $SCALARISCLIENT_RUBY -h\"`\n        RUBYHELP_OUTPUT=`$SCALARISCLIENT_RUBY -h 2>&1`\n        RUBYHELP_FAILED=$?\n        RUBY_OUTPUT=`echo -e \"$RUBY_OUTPUT\\n$RUBYHELP_OUTPUT\"`\n        if [ $RUBYHELP_FAILED -eq 0 ]; then\n            # simple read/write\n            KEY=\"checkinstallation-ruby-$RANDOM\"\n            RUBY_OUTPUT=`echo -e \"> $SCALARISCLIENT_RUBY -r ${KEY}\"`\n            RUBYREAD1_OUTPUT=`$SCALARISCLIENT_RUBY -r ${KEY} 2>&1`\n            echo \"$RUBYREAD1_OUTPUT\" | grep \"Scalaris::NotFoundError\" > /dev/null 2>&1\n            RUBYREAD1_FAILED=$?\n            RUBY_OUTPUT=`echo -e \"$RUBY_OUTPUT\\n$RUBYREAD1_OUTPUT\"`\n\n            RUBY_OUTPUT=`echo -e \"$RUBY_OUTPUT\\n> $SCALARISCLIENT_RUBY -w ${KEY},1\"`\n            RUBYWRITE1_OUTPUT=`$SCALARISCLIENT_RUBY -w ${KEY},1 2>&1`\n            RUBYWRITE1_FAILED=$?\n            RUBY_OUTPUT=`echo -e \"$RUBY_OUTPUT\\n$RUBYWRITE1_OUTPUT\"`\n\n            RUBY_OUTPUT=`echo -e \"$RUBY_OUTPUT\\n> $SCALARISCLIENT_RUBY -r ${KEY}\"`\n            RUBYREAD2_OUTPUT=`$SCALARISCLIENT_RUBY -r ${KEY} 2>&1`\n            RUBYREAD2_FAILED=$?\n            RUBY_OUTPUT=`echo -e \"$RUBY_OUTPUT\\n$RUBYREAD2_OUTPUT\"`\n\n            # note: we don't have a minibench for ruby...\n\n            if [ $RUBYREAD1_FAILED -ne 0 -o $RUBYWRITE1_FAILED -ne 0 -o $RUBYREAD2_FAILED -ne 0 ]; then\n                RUBY_FAILED=1\n                RUBY_RESULT=\"$MSG_FAILED\"\n            else\n                RUBY_FAILED=0\n                RUBY_RESULT=\"$MSG_SUCCESS\"\n            fi\n        else\n            RUBY_FAILED=1\n            RUBY_RESULT=\"$MSG_FAILED\"\n        fi\n    else\n        RUBY_FAILED=0\n        RUBY_RESULT=\"$MSG_NOTINSTALLED\"\n    fi\n    echo -e \"$RUBY_RESULT\"\n    if [ \"$verbose\" -eq 1 -o \"$RUBY_FAILED\" -ne 0 ]; then\n        echo \"$RUBY_OUTPUT\"\n    fi\n\n    # testing C++:\n    echo -n \"  C++ API ... \"\n    if [ -f \"$SCALARISCLIENT_CPP\" ]; then\n        CPP_OUTPUT=`echo -e \"> $SCALARISCLIENT_CPP --help\"`\n        CPPHELP_OUTPUT=`$SCALARISCLIENT_CPP --help 2>&1`\n        CPPHELP_FAILED=$?\n        if [ $CPPHELP_FAILED -eq 0 ]; then\n            # simple read/write\n            KEY=\"checkinstallation-cpp-$RANDOM\"\n            CPP_OUTPUT=`echo -e \"> $SCALARISCLIENT_CPP --read ${KEY}\"`\n            CPPREAD1_OUTPUT=`$SCALARISCLIENT_CPP --read ${KEY} 2>&1`\n            echo \"$CPPREAD1_OUTPUT\" | grep \"ReadFailedError\" > /dev/null 2>&1\n            CPPREAD1_FAILED=$?\n            CPP_OUTPUT=`echo -e \"$CPP_OUTPUT\\n$CPPREAD1_OUTPUT\"`\n\n            CPP_OUTPUT=`echo -e \"$CPP_OUTPUT\\n> $SCALARISCLIENT_CPP --write ${KEY},1\"`\n            CPPWRITE1_OUTPUT=`$SCALARISCLIENT_CPP --write ${KEY},1 2>&1`\n            CPPWRITE1_FAILED=$?\n            CPP_OUTPUT=`echo -e \"$CPP_OUTPUT\\n$CPPWRITE1_OUTPUT\"`\n\n            CPP_OUTPUT=`echo -e \"$CPP_OUTPUT\\n> $SCALARISCLIENT_CPP --read ${KEY}\"`\n            CPPREAD2_OUTPUT=`$SCALARISCLIENT_CPP --read ${KEY} 2>&1`\n            CPPREAD2_FAILED=$?\n            CPP_OUTPUT=`echo -e \"$CPP_OUTPUT\\n$CPPREAD2_OUTPUT\"`\n\n            if [ $CPPREAD1_FAILED -ne 0 -o $CPPWRITE1_FAILED -ne 0 -o $CPPREAD2_FAILED -ne 0 ]; then\n                CPP_FAILED=1\n                CPP_RESULT=\"$MSG_FAILED\"\n            else\n                CPP_FAILED=0\n                CPP_RESULT=\"$MSG_SUCCESS\"\n            fi\n        else\n            CPP_FAILED=1\n            CPP_RESULT=\"$MSG_FAILED\"\n        fi\n    else\n        CPP_FAILED=0\n        CPP_RESULT=\"$MSG_NOTINSTALLED\"\n    fi\n    echo -e \"$CPP_RESULT\"\n    if [ \"$verbose\" -eq 1 -o \"$CPP_FAILED\" -ne 0 ]; then\n        echo \"$CPP_OUTPUT\"\n    fi\n\n    $0 -e \"$erl_flags\" -c \"chocolate chip cookie\" -d -f -m -n \"$NODE_NAME\" -p \"$port\" -y \"$yaws_port\" stop > /dev/null 2>&1\n\n    if [ \"$JAVA_FAILED\" -ne 0 -o \"$PYTHON_FAILED\" -ne 0 \\\n         -o \"$PYTHON3_FAILED\" -ne 0 -o \"$RUBY_FAILED\" -ne 0 -o \"$CPP_FAILED\" -ne 0 ]; then\n        exit 1\n    fi\n\n    exit 0\n}\n\nscalarisstart(){\n    echo $NODE_NAME\n    if [ \"\" = \"$NODE_NAME\" ]; then\n        DISTRERL=\"\"\n        echo \"Starting without distributed Erlang. Java-API will not work.\"\n        echo\n    else\n        DISTRERL=\"-name $NODE_NAME\"\n    fi\n    $ERL -noinput $DISTRERL -eval 'halt(0).' 2>/dev/null > /dev/null\n    FAILED=$?\n    if [ $FAILED -ne 0 ]; then\n        echo \"\"\n        echo \"ERROR: erl could not create a Erlang VM named '$NODE_NAME'\"\n        echo \"Are you already running a Scalaris server with this name?\"\n        echo \"You can change the name by the -n option (see -h).\"\n        echo \"'scalarisctl list' gives you a list of currently registered Erlang VMs\"\n        echo \"We were trying to run: erl -noinput $DISTRERL -eval 'halt(0).'\"\n        list_erlang_processes\n        exit 1\n    fi\n    erl_flags=\"$ERL_SCHED_FLAGS $erl_flags\"\n    if [ $daemonize -ne 0 -a $DAEMONIZE_SCREEN -eq 0 ]; then\n        BACKGROUND=\"-detached\"\n    fi\n    local tmp\n    local COOKIE\n    tmp=\"\"\n    COOKIE=\"\"\n    if [ \"$start_type\" != \"\" ]; then\n        tmp=\"$tmp -scalaris start_type $start_type\"\n    else\n      echo \"the start-type is missing\"\n      exit 1\n    fi\n    if [ \"$port\" != \"\" ]; then\n        tmp=\"$tmp -scalaris port $port\"\n    fi\n    if [ \"$VERBATIM\" != \"\" ]; then\n        tmp=\"$tmp -scalaris verbatim $VERBATIM\"\n    fi\n    if [ \"$yaws_port\" != \"\" ]; then\n        tmp=\"$tmp -scalaris yaws_port $yaws_port\"\n    fi\n    if [ \"$join_at\" != \"\" ]; then\n        tmp=\"$tmp -scalaris join_at $join_at\"\n    fi\n    if [ \"$join_at_list\" != \"\" ]; then\n        tmp=\"$tmp -scalaris join_at_list '$join_at_list'\"\n    fi\n    if [ $start_mgmt_server -eq 1 ]; then\n        tmp=\"$tmp -scalaris start_mgmt_server true\"\n    fi\n    if [ \"$verbose\" -eq 1 ]; then\n        tmp=\"$tmp -scalaris verbose true\"\n    fi\n    if [ -n \"$DIST_ERL_PORT\" ]; then\n        tmp=\"$tmp -kernel inet_dist_listen_min $DIST_ERL_PORT inet_dist_listen_max $DIST_ERL_PORT\"\n    fi\n    if [ -n \"$NODES_PER_VM\" ]; then\n        tmp=\"$tmp -scalaris nodes_per_vm \\\"$NODES_PER_VM\\\"\"\n    fi\n    if [ \"$SCALARIS_COOKIE\" != \"\" ]; then\n        COOKIE=\"-setcookie '$SCALARIS_COOKIE'\"\n    fi\n    SCALARIS_ADDITIONAL_PARAMETERS=\"$SCALARIS_ADDITIONAL_PARAMETERS $tmp\"\n    # if YAWSHOME is not the logdir, make sure to create the directory!\n    export YAWSHOME=\"$LOGDIR/$NODE_NAME\"\n    mkdir -p \"$LOGDIR/$NODE_NAME\"\n    pushd $BEAMDIR > /dev/null\n    START_CMD=\"\\\"$ERL\\\" -pa $SCALARISDIR/contrib/yaws/ebin\\\n -pa $SCALARISDIR/contrib/log4erl/ebin\\\n -pa $SCALARISDIR/contrib/dotto/ebin\\\n -pa $BEAMDIR $ERLANG_ADDITIONAL_FLAGS $BACKGROUND\\\n -sasl sasl_error_logger false\\\n -yaws embedded true\\\n -scalaris log_path \\\"\\\\\\\"$LOGDIR/$NODE_NAME\\\\\\\"\\\"\\\n -scalaris docroot \\\"\\\\\\\"$DOCROOTDIR\\\\\\\"\\\"\\\n -scalaris config \\\"\\\\\\\"$ETCDIR/scalaris.cfg\\\\\\\"\\\"\\\n -scalaris local_config \\\"\\\\\\\"$ETCDIR/$SCALARIS_LOCAL_CFG\\\\\\\"\\\"\\\n -connect_all false -hidden $DISTRERL\\\n $SCALARIS_ADDITIONAL_PARAMETERS\\\n $COOKIE\\\n -s scalaris `echo $erl_flags`\"\n    if [ $daemonize -ne 0 -a $DAEMONIZE_SCREEN -ne 0 ]; then\n        if [ -n \"$SLURM_JOBID\" ]; then\n          session_name=\"scalaris_$NODE_NAME\"\n          $SCREEN -S \"scalaris_${NODE_NAME}_SLURM_JOBID_${SLURM_JOBID}\" -d -m bash -x -f +B -c \"$START_CMD; sleep 365d\"\n        else\n          $SCREEN -S \"scalaris_$NODE_NAME\" -d -m bash -x -f +B -c \"$START_CMD; sleep 365d\"\n        fi\n    else\n        bash -x -f +B -c \"$START_CMD\"\n    fi\n    FAILED=$?\n    if [ $FAILED -ne 0 ]; then\n        echo \"ERROR: could not start scalaris\"\n        echo \"  consider running scalarisctl checkinstallation\"\n        exit 1\n    fi\n    popd > /dev/null 2>&1 || true\n}\n\nscalarisstop(){\n    FAILED=\n    pushd $BEAMDIR > /dev/null\n    if [ \"$SCALARIS_COOKIE\" != \"\" ]; then\n      $ERL -setcookie \"$SCALARIS_COOKIE\"\\\n        -name \"ctl_$RANDOM@`get_hostname`\" \\\n        -pa $BEAMDIR \\\n        -noinput \\\n        -s scalaris cli -extra $NODE_NAME stop\n      FAILED=$?\n    else\n      $ERL \\\n        -name \"ctl_$RANDOM@`get_hostname`\" \\\n        -pa $BEAMDIR \\\n        -noinput \\\n        -s scalaris cli -extra $NODE_NAME stop\n      FAILED=$?\n    fi\n    popd > /dev/null 2>&1 || true\n    return $FAILED\n}\n\nscalariscall(){\n    FAILED=\n    pushd $BEAMDIR > /dev/null\n    if [ \"$SCALARIS_COOKIE\" != \"\" ]; then\n      $ERL -setcookie \"$SCALARIS_COOKIE\" \\\n        -name \"ctl_$RANDOM@`get_hostname`\" \\\n        -pa $BEAMDIR \\\n        -noinput \\\n        -eval \"$1\"\n      FAILED=$?\n    else\n      $ERL \\\n        -name \"ctl_$RANDOM@`get_hostname`\" \\\n        -pa $BEAMDIR \\\n        -noinput \\\n        -eval \"$1\"\n      FAILED=$?\n    fi\n    popd > /dev/null 2>&1 || true\n    return $FAILED\n}\n\nscalarisgstop(){\n    scalariscall \"Res = rpc:call('$NODE_NAME', api_vm, shutdown_vm, []), case Res of ok -> halt(0); _ -> io:format(\\\"~p~n\\\", [Res]), halt(1) end.\"\n    return $?\n}\n\nscalarisstatus(){\n    scalariscall \"Res = rpc:call('$NODE_NAME', api_vm, number_of_nodes, []), case Res of N when is_integer(N) -> io:format(\\\"Number of nodes: ~p~n\\\", [Res]), halt(0); _ -> io:format(\\\"~p~n\\\", [Res]), halt(1) end.\"\n    return $?\n}\n\nlist_erlang_processes(){\n    $EPMD -names || echo \"could not run epmd\"\n}\n\ndebug(){\n    # look into epmd -list: boot or node\n    $ERL -pa $BEAMDIR -setcookie \"$SCALARIS_COOKIE\" -name \"rem@`get_hostname`\" -remsh $NODE_NAME `echo $erl_flags`\n}\n\n# checks whether Scalaris has <ring-size> nodes and ring maintenance has settled\n# (requires a mgmt_server)\ndbg_check_ring(){\n  exp_nodes=$1\n  max_tries=$2\n  try=1\n  while [ $try -le $max_tries ] ; do\n    scalaris_nodes_count_actual=\"$(scalariscall \"Res = rpc:call('$NODE_NAME', admin, number_of_nodes, []), case Res of ${exp_nodes} -> halt(0); _ -> io:format(\\\"~p~n\\\", [Res]), halt(1) end.\")\"\n    if [ $? -eq 0 ] ; then\n      scalaris_check_ring=\"$(scalariscall \"Res = rpc:call('$NODE_NAME', admin, check_ring_deep, []), case Res of ok -> halt(0); _ -> io:format(\\\"~p~n\\\", [Res]), halt(1) end.\")\"\n      if [ $? -eq 0 ] ; then\n        return 0\n      fi\n    fi\n\n    try=$((10#$try + 1))\n    sleep 1\n\n    if [ $try -gt $max_tries ] ; then\n      echo \"ERROR: Failed to verify a total of ${exp_nodes} scalaris node(s) at node ${NODE_NAME}\"\n      echo \"       (result: ${scalaris_nodes_count_actual}).\"\n      if [ -n \"$scalaris_check_ring\" ] ; then\n        echo \"ERROR: check_ring_deep: ${scalaris_check_ring}.\"\n      fi\n      exit 1\n    fi\n  done\n}\n\nfix_paths\nSCALARIS_COOKIE=\"`get_cookie`\"\n\nuntil [ -z \"$1\" ]; do\n  OPTIND=1\n  case $1 in\n    \"--help\")\n      shift\n      usage 0;;\n    \"--dist-erl-port\")\n      shift\n      DIST_ERL_PORT=$1\n      shift;;\n    \"--screen\")\n      shift\n      DAEMONIZE_SCREEN=1;;\n    \"--nodes-per-vm\")\n      shift\n      NODES_PER_VM=$1\n      shift;;\n    checkinstallation | start | stop | gstop | restart | list | debug | status)\n      cmd=\"$1\"\n      shift;;\n    \"dbg-check-ring\")\n      cmd=\"$1\"\n      shift\n      DBG_CHECK_RING_SIZE=$1\n      shift\n      DBG_CHECK_RING_TIMEOUT=$1\n      shift;;\n    *)\n      if getopts \"dhifqmvn:p:y:k:j:e:c:l:s:t:b:\" optionName; then\n        case \"$optionName\" in\n            b) VERBATIM=$OPTARG;;\n            c) SCALARIS_COOKIE=$OPTARG;;\n            e) erl_flags=$OPTARG;;\n            d) daemonize=1;;\n            h) usage 0;;\n            i) echo \"WARN: interactive is now default\";;\n            k) join_at=$OPTARG;;\n            j) join_at_list=\"$OPTARG\";;\n            l) LOGDIR=$(readlink -m \"$OPTARG\");;\n            n) NODE_NAME=$OPTARG;;\n            p) port=$OPTARG;;\n            y) yaws_port=$OPTARG;;\n            m) start_mgmt_server=1;;\n            s) SCALARIS_LOCAL_CFG=$OPTARG;;\n            t) start_type=$OPTARG;;\n            v) echo setting verbose from cli\n               verbose=1;;\n          [?]) echo \"Wrong parameter $1.\"\n               usage 1;;\n        esac\n        shift $(($OPTIND-1))\n      else\n        shift\n        usage 1\n      fi;;\n  esac\ndone\n\nfix_node_name\n\ncase $cmd in\n    checkinstallation)\n        SCALARIS_COOKIE=\"chocolate chip cookie\" # java hack\n        checkinstallation;;\n    start)\n        scalarisstart;;\n    status)\n        scalarisstatus;;\n    stop)\n        scalarisstop;;\n    gstop)\n        scalarisgstop;;\n    restart)\n        scalarisstop\n        scalarisstart;;\n    list)\n        list_erlang_processes;;\n    debug)\n        debug;;\n    dbg-check-ring)\n        if [ -z \"$DBG_CHECK_RING_SIZE\" -o -z \"$DBG_CHECK_RING_TIMEOUT\" ] ; then\n          usage 1\n        else\n          dbg_check_ring \"$DBG_CHECK_RING_SIZE\" \"$DBG_CHECK_RING_TIMEOUT\"\n        fi;;\n    *)\n        echo \"Unknown command: $cmd.\"\n        usage 1;;\nesac\n"
  },
  {
    "path": "bin/setup-ring-for-benchmarks.sh",
    "content": "#!/bin/bash\n# Copyright 2007-2017 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nRINGSIZE=4\nNODEPREFIX=ebench_node\nBADNODES=0\nWITHSSL=\"\"\nSCALARIS_UNITTEST_PORT=${SCALARIS_UNITTEST_PORT-\"14195\"}\nSCALARIS_UNITTEST_YAWS_PORT=${SCALARIS_UNITTEST_YAWS_PORT-\"8000\"}\nusage(){\n    echo \"usage: setup-ring-for-benchmarks [options] <cmd>\"\n    echo \" options:\"\n    echo \"    --ring-size <number>   - the number of nodes\"\n    echo \"    --ssl                  - enable ssl\"\n    echo \"    --ssl-strict           - enable ssl with CA\"\n    echo \"    --bad-nodes <number>   - the number of nodes with different CA\"\n    echo \" <cmd>:\"\n    echo \"    start       - start a ring\"\n    echo \"    stop        - stop a ring\"\n    exit $1\n}\n\nstart_ring(){\n    KEYS=`./bin/scalarisctl -t nostart -n \"${NODEPREFIX}\" -e \"-noinput -eval \\\"io:format('\\n%~p', [api_rt:escaped_list_of_keys(api_rt:get_evenly_spaced_keys($RINGSIZE))]), halt(0)\\\"\" start | grep % | cut -c 3- | rev | cut -c 2- | rev`\n\n    export SCALARIS_ADDITIONAL_PARAMETERS=\"-scalaris mgmt_server {{127,0,0,1},$SCALARIS_UNITTEST_PORT,mgmt_server} -scalaris known_hosts [{{127,0,0,1},$SCALARIS_UNITTEST_PORT,service_per_vm}]\"\n    idx=0\n    for key in $KEYS; do\n        let idx+=1\n\n        STARTTYPE=null\n        if [ \"x$idx\" == x1 ]\n        then\n            STARTTYPE=\"first -m\"\n            TESTPORT=$SCALARIS_UNITTEST_PORT\n            YAWSPORT=$SCALARIS_UNITTEST_YAWS_PORT\n        else\n            STARTTYPE=\"joining\"\n            let TESTPORT=$SCALARIS_UNITTEST_PORT+$idx-1\n            let YAWSPORT=$SCALARIS_UNITTEST_YAWS_PORT+$idx-1\n        fi\n\n        ./bin/scalarisctl -d -k $key -n \"${NODEPREFIX}$idx\" -p $TESTPORT -y $YAWSPORT -t $STARTTYPE start $WITHSSL\n\n    done\n\n    for nodes in `seq 1 $BADNODES`; do\n        let idx+=1\n        let TESTPORT=$SCALARIS_UNITTEST_PORT+$idx-1\n        let YAWSPORT=$SCALARIS_UNITTEST_YAWS_PORT+$idx-1\n        ./bin/scalarisctl -d -n \"${NODEPREFIX}$idx\" -p $TESTPORT -y $YAWSPORT -t joining -s scalaris.local.ssl.bad.cfg start\n    done\n\n    if [ $BADNODES -ne 0 ]; then\n        sleep 5 ## give bad nodes a chance to join\n    fi\n\n    ./bin/scalarisctl -n \"${NODEPREFIX}1\" dbg-check-ring $RINGSIZE 30\n}\n\nstop_ring(){\n    let NODES=$RINGSIZE+$BADNODES\n    for idx in `seq 1 $NODES`; do\n        ./bin/scalarisctl -n \"${NODEPREFIX}$idx\" stop\n    done\n}\n\nuntil [ -z \"$1\" ]; do\n    OPTIND=1\n    case $1 in\n        \"--help\")\n            shift\n            usage 0;;\n        \"--ssl\")\n            WITHSSL=\"-s scalaris.local.ssl.cfg\"\n            shift;;\n        \"--ssl-strict\")\n            WITHSSL=\"-s scalaris.local.ssl.good.cfg\"\n            shift;;\n        \"--bad-nodes\")\n            shift\n            BADNODES=$1\n            shift;;\n        \"--ring-size\")\n            shift\n            RINGSIZE=$1\n            shift;;\n        \"--node-prefix\")\n            shift\n            NODEPREFIX=$1\n            shift;;\n        start | stop)\n            cmd=\"$1\"\n            shift;;\n        *)\n            shift\n            usage 1\n    esac\ndone\n\ncase $cmd in\n    start)\n        start_ring;;\n    stop)\n        stop_ring;;\n    *)\n        echo \"Unknown command: $cmd.\"\n        usage 1;;\nesac\n"
  },
  {
    "path": "bin/setup-ring.sh",
    "content": "#!/bin/bash\n# Copyright 2018, 2019 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nRINGSIZE=4\nWITHSSL=\"\"\nSSL=\"\"\nVMPREFIX=\"\"\n\nSCALARIS_UNITTEST_PORT=${SCALARIS_UNITTEST_PORT-\"14195\"}\nSCALARIS_UNITTEST_YAWS_PORT=${SCALARIS_UNITTEST_YAWS_PORT-\"8000\"}\n\n#SESSIONKEY=`cat /dev/urandom | env LC_CTYPE=C tr -dc 'a-z' | fold -w 32 | head -n 1`\n\nusage(){\n    echo \"usage: setup-ring [options] <cmd>\"\n    echo \" options:\"\n    echo \"    --help                  - show this text\"\n    echo \"    --ring-size <number>    - the number of nodes (default: 4)\"\n    echo \"    --session-key <number>  - the session's key (mandatory)\"\n    echo \"    --vm-prefix <string>    - the prefix for the vm names\"\n    echo \" <cmd>:\"\n    echo \"    start       - start a ring\"\n    echo \"    stop        - stop a ring\"\n    echo \"    kill        - kill a node\"\n    echo \"    recover     - recover a ring\"\n    exit $1\n}\n\nstart_ring(){\n    KEYS=`./bin/scalarisctl -t client -e \"-noinput -eval \\\"io:format('\\n%~p', [api_rt:escaped_list_of_keys(api_rt:get_evenly_spaced_keys($RINGSIZE))]), halt(0)\\\"\" start | grep % | cut -c 3- | rev | cut -c 2- | rev`\n\n    ADDITIONAL_PARAMETERS=\"-scalaris mgmt_server {{127,0,0,1},$SCALARIS_UNITTEST_PORT,mgmt_server} -scalaris known_hosts [{{127,0,0,1},$SCALARIS_UNITTEST_PORT,service_per_vm}] -scalaris verbatim $SESSIONKEY\"\n    ADDITIONAL_PARAMETERS_BAK=\"$ADDITIONAL_PARAMETERS\"\n    idx=0\n    for key in $KEYS; do\n        let idx+=1\n\n        STARTTYPE=null\n        if [ \"x$idx\" == x1 ]\n        then\n            STARTTYPE=\"first -m\"\n            TESTPORT=$SCALARIS_UNITTEST_PORT\n            YAWSPORT=$SCALARIS_UNITTEST_YAWS_PORT\n        else\n            STARTTYPE=\"joining\"\n            let TESTPORT=$SCALARIS_UNITTEST_PORT+$idx-1\n            let YAWSPORT=$SCALARIS_UNITTEST_YAWS_PORT+$idx-1\n        fi\n\n        if [ -n $VMPREFIX ]\n        then\n            export SCALARIS_ADDITIONAL_PARAMETERS=\"$ADDITIONAL_PARAMETERS -scalaris vmname $VMPREFIX$idx\"\n        else\n            export SCALARIS_ADDITIONAL_PARAMETERS=\"$ADDITIONAL_PARAMETERS\"\n        fi\n\n        ./bin/scalarisctl -d -k $key -p $TESTPORT -y $YAWSPORT -t $STARTTYPE start $WITHSSL\n\n    done\n    export SCALARIS_ADDITIONAL_PARAMETERS=\"$ADDITIONAL_PARAMETERS_BAK\"\n\n    let TESTPORT=$SCALARIS_UNITTEST_PORT+$RINGSIZE\n    let YAWSPORT=$SCALARIS_UNITTEST_YAWS_PORT+$RINGSIZE\n\n    ./bin/jsonclient -p $SCALARIS_UNITTEST_YAWS_PORT -r $RINGSIZE waitforringsize\n\n    # $SSL\n}\n\nrecover_ring(){\n    KEYS=`./bin/scalarisctl -t client -e \"-noinput -eval \\\"io:format('\\n%~p', [api_rt:escaped_list_of_keys(api_rt:get_evenly_spaced_keys($RINGSIZE))]), halt(0)\\\"\" start | grep % | cut -c 3- | rev | cut -c 2- | rev`\n\n    ADDITIONAL_PARAMETERS=\"-scalaris mgmt_server {{127,0,0,1},$SCALARIS_UNITTEST_PORT,mgmt_server} -scalaris known_hosts [{{127,0,0,1},$SCALARIS_UNITTEST_PORT,service_per_vm}] -scalaris verbatim $SESSIONKEY\"\n    ADDITIONAL_PARAMETERS_BAK=\"$ADDITIONAL_PARAMETERS\"\n    idx=0\n    for key in $KEYS; do\n        let idx+=1\n\n        STARTTYPE=\"recover\"\n        if [ \"x$idx\" == x1 ]\n        then\n            TESTPORT=$SCALARIS_UNITTEST_PORT\n            YAWSPORT=$SCALARIS_UNITTEST_YAWS_PORT\n        else\n            let TESTPORT=$SCALARIS_UNITTEST_PORT+$idx-1\n            let YAWSPORT=$SCALARIS_UNITTEST_YAWS_PORT+$idx-1\n        fi\n\n        if [ -n $VMPREFIX ]\n        then\n            export SCALARIS_ADDITIONAL_PARAMETERS=\"$ADDITIONAL_PARAMETERS -scalaris vmname $VMPREFIX$idx\"\n        else\n            export SCALARIS_ADDITIONAL_PARAMETERS=\"$ADDITIONAL_PARAMETERS\"\n        fi\n\n        # to recover, we have to be located in the bin directory\n        (cd bin ; ./scalarisctl -d -k $key -p $TESTPORT -y $YAWSPORT -t $STARTTYPE start $WITHSSL)\n\n    done\n    export SCALARIS_ADDITIONAL_PARAMETERS=\"$ADDITIONAL_PARAMETERS_BAK\"\n\n    let TESTPORT=$SCALARIS_UNITTEST_PORT+$RINGSIZE\n    let YAWSPORT=$SCALARIS_UNITTEST_YAWS_PORT+$RINGSIZE\n\n    ## TODO: also recover mgmt_server, so this check can be done\n    #./bin/jsonclient -p $SCALARIS_UNITTEST_YAWS_PORT -r $RINGSIZE waitforringsize\n\n    # $SSL\n}\n\nstop_ring(){\n    ps aux | grep beam.smp | grep \"verbatim $SESSIONKEY\" | awk '{print $2}'  | xargs -n1 kill -9\n}\n\nkill_node(){\n    ps aux | grep beam.smp | grep \"verbatim $SESSIONKEY\" | grep -v \"join_at 0\" | awk '{print $2}' | head -n1 | xargs -n1 kill -9\n}\n\nMANDATORY=false\n\nuntil [ -z \"$1\" ]; do\n    OPTIND=1\n    case $1 in\n        \"--help\")\n            shift\n            usage 0;;\n        \"--ring-size\")\n            shift\n            RINGSIZE=$1\n            shift;;\n        \"--vm-prefix\")\n            shift\n            VMPREFIX=$1\n            shift;;\n        \"--session-key\")\n            shift\n            SESSIONKEY=$1\n            MANDATORY=true\n            shift;;\n        \"--ssl\")\n            WITHSSL=\"-s scalaris.local.ssl.cfg\"\n            SSL=\"-s\"\n            shift;;\n        \"--ssl-strict\")\n            WITHSSL=\"-s scalaris.local.ssl.good.cfg\"\n            shift;;\n        start | stop)\n            cmd=\"$1\"\n            shift;;\n        kill)\n            cmd=\"$1\"\n            shift;;\n        recover)\n            cmd=\"$1\"\n            shift;;\n        *)\n            echo \"Unknown parameter: $1.\"\n            shift\n            usage 1\n    esac\ndone\n\nif [ \"x$MANDATORY\" == xfalse ]\nthen\n   echo \"Mandatory --session-key parameter is missing\"\n   usage 1\nfi\n\ncase $cmd in\n    start)\n        start_ring;;\n    stop)\n        stop_ring;;\n    kill)\n        kill_node;;\n    recover)\n        recover_ring;;\n    *)\n        echo \"Unknown command: $cmd.\"\n        usage 1;;\nesac\n"
  },
  {
    "path": "bin/slurmctl",
    "content": "#!/bin/bash\n\nFIRSTNODE=`scontrol show hostnames | head -n1`\nLASTNODES=`scontrol show hostnames | tail -n +2`\nJOINNODES=`echo $LASTNODES | sed s/\\ /\\,/g`\n\nETCDIR=\"\"\n\nfix_paths() {\n    ABSPATH=\"$(cd \"${0%/*}\" 2>/dev/null; echo \"$PWD\"/\"${0##*/}\")\"\n    DIRNAME=`dirname $ABSPATH`\n    ERLPATH=`which erl`\n    BINDIR=\"dirname $ERLPATH\"\n    # is this a svn checkout or an (rpm/deb/manual) installation?\n    if [ \"$DIRNAME\" != \"$BINDIR\" -a \"$DIRNAME\" != \"/bin\" ]; then\n        # scalaris\n        SCALARISDIR=`dirname $DIRNAME`\n        # /etc/scalaris/\n        ETCDIR=$SCALARISDIR/bin\n    else\n        echo \"error: we do not support rpm/deb/manual installations\"\n        exit\n    fi\n}\n\nfix_known_hosts() {\n    if [ ! -f \"$ETCDIR/scalaris.local.cfg\" ]; then\n        touch \"$ETCDIR/scalaris.local.cfg\"\n    fi\n    # save scalaris.local.cfg\n    cp $ETCDIR/scalaris.local.cfg $ETCDIR/scalaris.local.cfg.slurm-backup\n    echo \"{known_hosts, [\" >> $ETCDIR/scalaris.local.cfg\n    LASTNODE=`scontrol show hostnames | tail -n1`\n    for host in `scontrol show hostnames`; do\n        IP=$host\n        ## TODO\n        ## IP=`nslookup $host | grep Address | grep -v \"#\" | cut -d ' ' -f 2 | sed s/\\\\./\\,/g`\n        IP1=`srun --nodelist=$host -N1 ifconfig eth0 | grep 'inet addr:' | cut -d ':' -f 2 | cut -d ' ' -f 1`\n        echo -n \"{{\" >> $ETCDIR/scalaris.local.cfg\n        echo -n $IP1 | sed s/\\\\./\\,/g >> $ETCDIR/scalaris.local.cfg\n        if [ $LASTNODE != $host ]; then\n            echo \"}, 14195, service_per_vm},\" >> $ETCDIR/scalaris.local.cfg\n        else\n            echo \"}, 14195, service_per_vm}\" >> $ETCDIR/scalaris.local.cfg\n        fi\n    done\n    echo \"]}.\" >> $ETCDIR/scalaris.local.cfg\n\n    ## fix mgmt_server\n    echo -n \"{mgmt_server, {{\" >> $ETCDIR/scalaris.local.cfg\n    IP1=`srun --nodelist=$FIRSTNODE -N1 ifconfig eth0 | grep 'inet addr:' | cut -d ':' -f 2 | cut -d ' ' -f 1`\n    echo -n $IP1 | sed s/\\\\./\\,/g >> $ETCDIR/scalaris.local.cfg\n    echo \"}, 14195, mgmt_server}}.\" >> $ETCDIR/scalaris.local.cfg\n}\n\nusage(){\n    echo \"usage slurm [options] <cmd>\"\n    echo \" options:\"\n    echo \"    --help      - print this message\"\n    echo \" <cmd>:\"\n    echo \"    start\"\n    echo \"                - start joining nodes on all but the first node\"\n    echo \"                  start an interactive first node on the first node\"\n    echo \"    stop\"\n    echo \"                - stop Erlang VM on all but the first node\"\n    echo \"    kill\"\n    echo \"                - kill all beam.smp processes\"\n    echo \"    killscreens\"\n    echo \"                - kill all scalaris screen sessions\"\n    echo \"\"\n    exit $1\n}\n\nscalarisstart() {\n    fix_known_hosts\n\n    srun --nodelist=$JOINNODES -N$(expr $SLURM_NNODES - 1) ./bin/scalarisctl --screen -d -t joining start\n    srun --nodelist=$FIRSTNODE --pty -N1  ./bin/scalarisctl -m -t first start\n}\n\nscalarisstop() {\n    srun --nodelist=$JOINNODES -N$(expr $SLURM_NNODES - 1) ./bin/scalarisctl -t joining stop\n    scalariskillscreens\n    ## restore scalaris.local.cfg\n    mv $ETCDIR/scalaris.local.cfg.slurm-backup $ETCDIR/scalaris.local.cfg\n}\n\nscalariskill() {\n    srun -N4 killall beam.smp\n}\n\nscalariskillscreens () {\n    srun -N4 bash -c \"screen -ls | grep Detached | grep scalaris_node | cut -d. -f1 | awk '{print $1}' | xargs -r kill\"\n}\n\nfix_paths\n\n\ncmd=\"\"\n\nuntil [ -z \"$1\" ]; do\n  OPTIND=1\n  case $1 in\n    \"--help\")\n      shift\n      usage 0;;\n    start | stop | kill | killscreens)\n      cmd=\"$1\"\n      shift;;\n    *)\n  esac\ndone\n\ncase $cmd in\n    start)\n        scalarisstart;;\n    stop)\n        scalarisstop;;\n    kill)\n        scalariskill;;\n    killscreens)\n        scalariskillscreens;;\n    *)\n        echo \"Unknown command: $cmd.\"\n        usage 1;;\nesac\n"
  },
  {
    "path": "bootstrap.sh",
    "content": "#!/bin/bash\n# Copyright 2007-2018 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\n## update version ##\n\nVERSION=`cat VERSION`\nVERSION_NOPLUS=`echo \"${VERSION}\" | tr + _`\nif [[ \"$VERSION\" == *git* ]]; then\n  ## maven snapshot versions must have revision incremented by 1 so that\n  ## maven sees them as a newer version than the release\n  VERSION_NOGIT=${VERSION%+git*}\n  MAJOR_MINOR=${VERSION_NOGIT%.*}\n  REVISION=${VERSION_NOGIT##*.}\n  ((REVISION++))\n  VERSION_MAVEN=\"$MAJOR_MINOR.$REVISION-SNAPSHOT\"\nelse\n  VERSION_MAVEN=$VERSION\nfi\n\n\n## echo \"Setting Scalaris version to ${VERSION}...\"\n## sed -e \"s/-define(SCALARIS_VERSION, \\\".*\\\")\\\\./-define(SCALARIS_VERSION, \\\"${VERSION}\\\")./g\" \\\n##     -i include/scalaris.hrl\n## sed -e \"s/AC_INIT(scalaris, .*, scalaris@googlegroups.com)/AC_INIT(scalaris, ${VERSION}, scalaris@googlegroups.com)/g\" \\\n##     -i configure.ac\n## sed -e \"s/public static final String version = \\\".*\\\";/public static final String version = \\\"${VERSION}\\\";/g\" \\\n##     -i contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/WikiServlet.java\n## sed -e \"s/version='.*',/version='${VERSION}',/g\" \\\n##     -i python-api/setup.py\n## sed -e \"s/SCALARIS_VERSION=\\\".*\\\"/SCALARIS_VERSION=\\\"${VERSION}\\\"/g\" \\\n##     -i contrib/packages/*/checkout.sh\n## sed -e \"s/%define pkg_version .*/%define pkg_version ${VERSION}/g\" \\\n##     -i contrib/packages/*/*.spec\n## sed -e \"s/Version: .*-.*/Version: ${VERSION}-1/g\" \\\n##     -i contrib/packages/*/*.dsc\n## sed -e \"0,/(.*-.*)/s//(${VERSION}-1)/\" \\\n##     -i contrib/packages/*/debian.changelog\n## sed -e \"0,/<version>.*<\\/version>/s//<version>${VERSION_MAVEN}<\\/version>/\" \\\n##     -i java-api/pom.xml contrib/datanucleus/scalaris-datanucleus-store/pom.xml\n## sed -e \"s/module scalaris .*;/module scalaris ${VERSION_NOPLUS};/g\" \\\n##     -i contrib/systemd/scalaris.te\n## if [[ \"$VERSION\" == *git* ]]; then\n##   RELEASE=\"unstable\"\n## else\n##   RELEASE=\"stable\"\n## fi\n## sed -e \"0,/u*n*stable;/s//${RELEASE};/\" \\\n##     -i contrib/packages/*/debian.changelog\n## echo \"done\"\n## sed -e \"s/pkgver=.*/pkgver=${VERSION}/g\" \\\n##     -i contrib/packages/*/PKGBUILD\n\n##########\n\nif [ -z `which automake` ]\nthen\n    echo \"automake is missing.\"\n    echo \"Please install automake.\"\n    exit\nfi\n\nif [ -z `which aclocal` ]\nthen\n    echo \"aclocal is missing.\"\n    echo \"Please install automake.\"\n    exit\nfi\n\n# pretend to use automake\ntouch Makefile.am NEWS READ COPYING README INSTALL\n\nif [ ! -d python3-api ]\nthen\n    mkdir python3-api\n    touch python3-api/scalaris.in\nfi\n\ncp Makefile.in Makefile.in.bak\n\necho \"Creating configure script\"\nautoreconf --verbose --force --install # -Wall\necho \"\"\necho \"A ./configure file should be created.\"\necho \"Please proceed with calling './configure'\"\n\n# undo automake\nrm Makefile.am NEWS READ COPYING README INSTALL\nmv Makefile.in.bak Makefile.in\n\nACLOCAL=`which aclocal`\nACLOCAL_BIN_DIRECTORY=`dirname $ACLOCAL`\nACLOCAL_BASE_DIRECTORY=`dirname $ACLOCAL_BIN_DIRECTORY`\nACLOCAL_SHARE_DIRECTORY=\"$ACLOCAL_BASE_DIRECTORY/share/aclocal\"\n#echo $ACLOCAL_BIN_DIRECTORY\n#echo $ACLOCAL_BASE_DIRECTORY\n#echo $ACLOCAL_SHARE_DIRECTORY\n\nif [ -e \"$ACLOCAL_SHARE_DIRECTORY/intlmacosx.m4\" ]; then\n    #echo \"found aclocal directory\"\n    if [ ! -e \"$ACLOCAL_SHARE_DIRECTORY/ax_boost_system.m4\" ]; then\n        echo \"\"\n        echo \"for the save creation of the configure file, it is helpful to have the autoconf-archive package installed\"\n        echo \"autoconf-archive seems to be missing on your system\"\n        echo \"we checked for $ACLOCAL_SHARE_DIRECTORY/ax_boost_system.m4, but it does not exist\"\n    fi\nfi\n"
  },
  {
    "path": "build-aux/compile",
    "content": "#! /bin/sh\n# Wrapper for compilers which do not understand '-c -o'.\n\nscriptversion=2016-01-11.22; # UTC\n\n# Copyright (C) 1999-2017 Free Software Foundation, Inc.\n# Written by Tom Tromey <tromey@cygnus.com>.\n#\n# This program is free software; you can redistribute it and/or modify\n# it under the terms of the GNU General Public License as published by\n# the Free Software Foundation; either version 2, or (at your option)\n# any later version.\n#\n# This program is distributed in the hope that it will be useful,\n# but WITHOUT ANY WARRANTY; without even the implied warranty of\n# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\n# GNU General Public License for more details.\n#\n# You should have received a copy of the GNU General Public License\n# along with this program.  If not, see <http://www.gnu.org/licenses/>.\n\n# As a special exception to the GNU General Public License, if you\n# distribute this file as part of a program that contains a\n# configuration script generated by Autoconf, you may include it under\n# the same distribution terms that you use for the rest of that program.\n\n# This file is maintained in Automake, please report\n# bugs to <bug-automake@gnu.org> or send patches to\n# <automake-patches@gnu.org>.\n\nnl='\n'\n\n# We need space, tab and new line, in precisely that order.  Quoting is\n# there to prevent tools from complaining about whitespace usage.\nIFS=\" \"\"\t$nl\"\n\nfile_conv=\n\n# func_file_conv build_file lazy\n# Convert a $build file to $host form and store it in $file\n# Currently only supports Windows hosts. If the determined conversion\n# type is listed in (the comma separated) LAZY, no conversion will\n# take place.\nfunc_file_conv ()\n{\n  file=$1\n  case $file in\n    / | /[!/]*) # absolute file, and not a UNC file\n      if test -z \"$file_conv\"; then\n\t# lazily determine how to convert abs files\n\tcase `uname -s` in\n\t  MINGW*)\n\t    file_conv=mingw\n\t    ;;\n\t  CYGWIN*)\n\t    file_conv=cygwin\n\t    ;;\n\t  *)\n\t    file_conv=wine\n\t    ;;\n\tesac\n      fi\n      case $file_conv/,$2, in\n\t*,$file_conv,*)\n\t  ;;\n\tmingw/*)\n\t  file=`cmd //C echo \"$file \" | sed -e 's/\"\\(.*\\) \" *$/\\1/'`\n\t  ;;\n\tcygwin/*)\n\t  file=`cygpath -m \"$file\" || echo \"$file\"`\n\t  ;;\n\twine/*)\n\t  file=`winepath -w \"$file\" || echo \"$file\"`\n\t  ;;\n      esac\n      ;;\n  esac\n}\n\n# func_cl_dashL linkdir\n# Make cl look for libraries in LINKDIR\nfunc_cl_dashL ()\n{\n  func_file_conv \"$1\"\n  if test -z \"$lib_path\"; then\n    lib_path=$file\n  else\n    lib_path=\"$lib_path;$file\"\n  fi\n  linker_opts=\"$linker_opts -LIBPATH:$file\"\n}\n\n# func_cl_dashl library\n# Do a library search-path lookup for cl\nfunc_cl_dashl ()\n{\n  lib=$1\n  found=no\n  save_IFS=$IFS\n  IFS=';'\n  for dir in $lib_path $LIB\n  do\n    IFS=$save_IFS\n    if $shared && test -f \"$dir/$lib.dll.lib\"; then\n      found=yes\n      lib=$dir/$lib.dll.lib\n      break\n    fi\n    if test -f \"$dir/$lib.lib\"; then\n      found=yes\n      lib=$dir/$lib.lib\n      break\n    fi\n    if test -f \"$dir/lib$lib.a\"; then\n      found=yes\n      lib=$dir/lib$lib.a\n      break\n    fi\n  done\n  IFS=$save_IFS\n\n  if test \"$found\" != yes; then\n    lib=$lib.lib\n  fi\n}\n\n# func_cl_wrapper cl arg...\n# Adjust compile command to suit cl\nfunc_cl_wrapper ()\n{\n  # Assume a capable shell\n  lib_path=\n  shared=:\n  linker_opts=\n  for arg\n  do\n    if test -n \"$eat\"; then\n      eat=\n    else\n      case $1 in\n\t-o)\n\t  # configure might choose to run compile as 'compile cc -o foo foo.c'.\n\t  eat=1\n\t  case $2 in\n\t    *.o | *.[oO][bB][jJ])\n\t      func_file_conv \"$2\"\n\t      set x \"$@\" -Fo\"$file\"\n\t      shift\n\t      ;;\n\t    *)\n\t      func_file_conv \"$2\"\n\t      set x \"$@\" -Fe\"$file\"\n\t      shift\n\t      ;;\n\t  esac\n\t  ;;\n\t-I)\n\t  eat=1\n\t  func_file_conv \"$2\" mingw\n\t  set x \"$@\" -I\"$file\"\n\t  shift\n\t  ;;\n\t-I*)\n\t  func_file_conv \"${1#-I}\" mingw\n\t  set x \"$@\" -I\"$file\"\n\t  shift\n\t  ;;\n\t-l)\n\t  eat=1\n\t  func_cl_dashl \"$2\"\n\t  set x \"$@\" \"$lib\"\n\t  shift\n\t  ;;\n\t-l*)\n\t  func_cl_dashl \"${1#-l}\"\n\t  set x \"$@\" \"$lib\"\n\t  shift\n\t  ;;\n\t-L)\n\t  eat=1\n\t  func_cl_dashL \"$2\"\n\t  ;;\n\t-L*)\n\t  func_cl_dashL \"${1#-L}\"\n\t  ;;\n\t-static)\n\t  shared=false\n\t  ;;\n\t-Wl,*)\n\t  arg=${1#-Wl,}\n\t  save_ifs=\"$IFS\"; IFS=','\n\t  for flag in $arg; do\n\t    IFS=\"$save_ifs\"\n\t    linker_opts=\"$linker_opts $flag\"\n\t  done\n\t  IFS=\"$save_ifs\"\n\t  ;;\n\t-Xlinker)\n\t  eat=1\n\t  linker_opts=\"$linker_opts $2\"\n\t  ;;\n\t-*)\n\t  set x \"$@\" \"$1\"\n\t  shift\n\t  ;;\n\t*.cc | *.CC | *.cxx | *.CXX | *.[cC]++)\n\t  func_file_conv \"$1\"\n\t  set x \"$@\" -Tp\"$file\"\n\t  shift\n\t  ;;\n\t*.c | *.cpp | *.CPP | *.lib | *.LIB | *.Lib | *.OBJ | *.obj | *.[oO])\n\t  func_file_conv \"$1\" mingw\n\t  set x \"$@\" \"$file\"\n\t  shift\n\t  ;;\n\t*)\n\t  set x \"$@\" \"$1\"\n\t  shift\n\t  ;;\n      esac\n    fi\n    shift\n  done\n  if test -n \"$linker_opts\"; then\n    linker_opts=\"-link$linker_opts\"\n  fi\n  exec \"$@\" $linker_opts\n  exit 1\n}\n\neat=\n\ncase $1 in\n  '')\n     echo \"$0: No command.  Try '$0 --help' for more information.\" 1>&2\n     exit 1;\n     ;;\n  -h | --h*)\n    cat <<\\EOF\nUsage: compile [--help] [--version] PROGRAM [ARGS]\n\nWrapper for compilers which do not understand '-c -o'.\nRemove '-o dest.o' from ARGS, run PROGRAM with the remaining\narguments, and rename the output as expected.\n\nIf you are trying to build a whole package this is not the\nright script to run: please start by reading the file 'INSTALL'.\n\nReport bugs to <bug-automake@gnu.org>.\nEOF\n    exit $?\n    ;;\n  -v | --v*)\n    echo \"compile $scriptversion\"\n    exit $?\n    ;;\n  cl | *[/\\\\]cl | cl.exe | *[/\\\\]cl.exe | \\\n  icl | *[/\\\\]icl | icl.exe | *[/\\\\]icl.exe )\n    func_cl_wrapper \"$@\"      # Doesn't return...\n    ;;\nesac\n\nofile=\ncfile=\n\nfor arg\ndo\n  if test -n \"$eat\"; then\n    eat=\n  else\n    case $1 in\n      -o)\n\t# configure might choose to run compile as 'compile cc -o foo foo.c'.\n\t# So we strip '-o arg' only if arg is an object.\n\teat=1\n\tcase $2 in\n\t  *.o | *.obj)\n\t    ofile=$2\n\t    ;;\n\t  *)\n\t    set x \"$@\" -o \"$2\"\n\t    shift\n\t    ;;\n\tesac\n\t;;\n      *.c)\n\tcfile=$1\n\tset x \"$@\" \"$1\"\n\tshift\n\t;;\n      *)\n\tset x \"$@\" \"$1\"\n\tshift\n\t;;\n    esac\n  fi\n  shift\ndone\n\nif test -z \"$ofile\" || test -z \"$cfile\"; then\n  # If no '-o' option was seen then we might have been invoked from a\n  # pattern rule where we don't need one.  That is ok -- this is a\n  # normal compilation that the losing compiler can handle.  If no\n  # '.c' file was seen then we are probably linking.  That is also\n  # ok.\n  exec \"$@\"\nfi\n\n# Name of file we expect compiler to create.\ncofile=`echo \"$cfile\" | sed 's|^.*[\\\\/]||; s|^[a-zA-Z]:||; s/\\.c$/.o/'`\n\n# Create the lock directory.\n# Note: use '[/\\\\:.-]' here to ensure that we don't use the same name\n# that we are using for the .o file.  Also, base the name on the expected\n# object file name, since that is what matters with a parallel build.\nlockdir=`echo \"$cofile\" | sed -e 's|[/\\\\:.-]|_|g'`.d\nwhile true; do\n  if mkdir \"$lockdir\" >/dev/null 2>&1; then\n    break\n  fi\n  sleep 1\ndone\n# FIXME: race condition here if user kills between mkdir and trap.\ntrap \"rmdir '$lockdir'; exit 1\" 1 2 15\n\n# Run the compile.\n\"$@\"\nret=$?\n\nif test -f \"$cofile\"; then\n  test \"$cofile\" = \"$ofile\" || mv \"$cofile\" \"$ofile\"\nelif test -f \"${cofile}bj\"; then\n  test \"${cofile}bj\" = \"$ofile\" || mv \"${cofile}bj\" \"$ofile\"\nfi\n\nrmdir \"$lockdir\"\nexit $ret\n\n# Local Variables:\n# mode: shell-script\n# sh-indentation: 2\n# eval: (add-hook 'write-file-hooks 'time-stamp)\n# time-stamp-start: \"scriptversion=\"\n# time-stamp-format: \"%:y-%02m-%02d.%02H\"\n# time-stamp-time-zone: \"UTC0\"\n# time-stamp-end: \"; # UTC\"\n# End:\n"
  },
  {
    "path": "build-aux/config.guess",
    "content": "#! /bin/sh\n# Attempt to guess a canonical system name.\n#   Copyright 1992-2017 Free Software Foundation, Inc.\n\ntimestamp='2017-05-27'\n\n# This file is free software; you can redistribute it and/or modify it\n# under the terms of the GNU General Public License as published by\n# the Free Software Foundation; either version 3 of the License, or\n# (at your option) any later version.\n#\n# This program is distributed in the hope that it will be useful, but\n# WITHOUT ANY WARRANTY; without even the implied warranty of\n# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU\n# General Public License for more details.\n#\n# You should have received a copy of the GNU General Public License\n# along with this program; if not, see <http://www.gnu.org/licenses/>.\n#\n# As a special exception to the GNU General Public License, if you\n# distribute this file as part of a program that contains a\n# configuration script generated by Autoconf, you may include it under\n# the same distribution terms that you use for the rest of that\n# program.  This Exception is an additional permission under section 7\n# of the GNU General Public License, version 3 (\"GPLv3\").\n#\n# Originally written by Per Bothner; maintained since 2000 by Ben Elliston.\n#\n# You can get the latest version of this script from:\n# http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess\n#\n# Please send patches to <config-patches@gnu.org>.\n\n\nme=`echo \"$0\" | sed -e 's,.*/,,'`\n\nusage=\"\\\nUsage: $0 [OPTION]\n\nOutput the configuration name of the system \\`$me' is run on.\n\nOperation modes:\n  -h, --help         print this help, then exit\n  -t, --time-stamp   print date of last modification, then exit\n  -v, --version      print version number, then exit\n\nReport bugs and patches to <config-patches@gnu.org>.\"\n\nversion=\"\\\nGNU config.guess ($timestamp)\n\nOriginally written by Per Bothner.\nCopyright 1992-2017 Free Software Foundation, Inc.\n\nThis is free software; see the source for copying conditions.  There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\"\n\nhelp=\"\nTry \\`$me --help' for more information.\"\n\n# Parse command line\nwhile test $# -gt 0 ; do\n  case $1 in\n    --time-stamp | --time* | -t )\n       echo \"$timestamp\" ; exit ;;\n    --version | -v )\n       echo \"$version\" ; exit ;;\n    --help | --h* | -h )\n       echo \"$usage\"; exit ;;\n    -- )     # Stop option processing\n       shift; break ;;\n    - )\t# Use stdin as input.\n       break ;;\n    -* )\n       echo \"$me: invalid option $1$help\" >&2\n       exit 1 ;;\n    * )\n       break ;;\n  esac\ndone\n\nif test $# != 0; then\n  echo \"$me: too many arguments$help\" >&2\n  exit 1\nfi\n\ntrap 'exit 1' 1 2 15\n\n# CC_FOR_BUILD -- compiler used by this script. Note that the use of a\n# compiler to aid in system detection is discouraged as it requires\n# temporary files to be created and, as you can see below, it is a\n# headache to deal with in a portable fashion.\n\n# Historically, `CC_FOR_BUILD' used to be named `HOST_CC'. We still\n# use `HOST_CC' if defined, but it is deprecated.\n\n# Portable tmp directory creation inspired by the Autoconf team.\n\nset_cc_for_build='\ntrap \"exitcode=\\$?; (rm -f \\$tmpfiles 2>/dev/null; rmdir \\$tmp 2>/dev/null) && exit \\$exitcode\" 0 ;\ntrap \"rm -f \\$tmpfiles 2>/dev/null; rmdir \\$tmp 2>/dev/null; exit 1\" 1 2 13 15 ;\n: ${TMPDIR=/tmp} ;\n { tmp=`(umask 077 && mktemp -d \"$TMPDIR/cgXXXXXX\") 2>/dev/null` && test -n \"$tmp\" && test -d \"$tmp\" ; } ||\n { test -n \"$RANDOM\" && tmp=$TMPDIR/cg$$-$RANDOM && (umask 077 && mkdir $tmp) ; } ||\n { tmp=$TMPDIR/cg-$$ && (umask 077 && mkdir $tmp) && echo \"Warning: creating insecure temp directory\" >&2 ; } ||\n { echo \"$me: cannot create a temporary directory in $TMPDIR\" >&2 ; exit 1 ; } ;\ndummy=$tmp/dummy ;\ntmpfiles=\"$dummy.c $dummy.o $dummy.rel $dummy\" ;\ncase $CC_FOR_BUILD,$HOST_CC,$CC in\n ,,)    echo \"int x;\" > $dummy.c ;\n\tfor c in cc gcc c89 c99 ; do\n\t  if ($c -c -o $dummy.o $dummy.c) >/dev/null 2>&1 ; then\n\t     CC_FOR_BUILD=\"$c\"; break ;\n\t  fi ;\n\tdone ;\n\tif test x\"$CC_FOR_BUILD\" = x ; then\n\t  CC_FOR_BUILD=no_compiler_found ;\n\tfi\n\t;;\n ,,*)   CC_FOR_BUILD=$CC ;;\n ,*,*)  CC_FOR_BUILD=$HOST_CC ;;\nesac ; set_cc_for_build= ;'\n\n# This is needed to find uname on a Pyramid OSx when run in the BSD universe.\n# (ghazi@noc.rutgers.edu 1994-08-24)\nif (test -f /.attbin/uname) >/dev/null 2>&1 ; then\n\tPATH=$PATH:/.attbin ; export PATH\nfi\n\nUNAME_MACHINE=`(uname -m) 2>/dev/null` || UNAME_MACHINE=unknown\nUNAME_RELEASE=`(uname -r) 2>/dev/null` || UNAME_RELEASE=unknown\nUNAME_SYSTEM=`(uname -s) 2>/dev/null`  || UNAME_SYSTEM=unknown\nUNAME_VERSION=`(uname -v) 2>/dev/null` || UNAME_VERSION=unknown\n\ncase \"${UNAME_SYSTEM}\" in\nLinux|GNU|GNU/*)\n\t# If the system lacks a compiler, then just pick glibc.\n\t# We could probably try harder.\n\tLIBC=gnu\n\n\teval $set_cc_for_build\n\tcat <<-EOF > $dummy.c\n\t#include <features.h>\n\t#if defined(__UCLIBC__)\n\tLIBC=uclibc\n\t#elif defined(__dietlibc__)\n\tLIBC=dietlibc\n\t#else\n\tLIBC=gnu\n\t#endif\n\tEOF\n\teval `$CC_FOR_BUILD -E $dummy.c 2>/dev/null | grep '^LIBC' | sed 's, ,,g'`\n\t;;\nesac\n\n# Note: order is significant - the case branches are not exclusive.\n\ncase \"${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}\" in\n    *:NetBSD:*:*)\n\t# NetBSD (nbsd) targets should (where applicable) match one or\n\t# more of the tuples: *-*-netbsdelf*, *-*-netbsdaout*,\n\t# *-*-netbsdecoff* and *-*-netbsd*.  For targets that recently\n\t# switched to ELF, *-*-netbsd* would select the old\n\t# object file format.  This provides both forward\n\t# compatibility and a consistent mechanism for selecting the\n\t# object file format.\n\t#\n\t# Note: NetBSD doesn't particularly care about the vendor\n\t# portion of the name.  We always set it to \"unknown\".\n\tsysctl=\"sysctl -n hw.machine_arch\"\n\tUNAME_MACHINE_ARCH=`(uname -p 2>/dev/null || \\\n\t    /sbin/$sysctl 2>/dev/null || \\\n\t    /usr/sbin/$sysctl 2>/dev/null || \\\n\t    echo unknown)`\n\tcase \"${UNAME_MACHINE_ARCH}\" in\n\t    armeb) machine=armeb-unknown ;;\n\t    arm*) machine=arm-unknown ;;\n\t    sh3el) machine=shl-unknown ;;\n\t    sh3eb) machine=sh-unknown ;;\n\t    sh5el) machine=sh5le-unknown ;;\n\t    earmv*)\n\t\tarch=`echo ${UNAME_MACHINE_ARCH} | sed -e 's,^e\\(armv[0-9]\\).*$,\\1,'`\n\t\tendian=`echo ${UNAME_MACHINE_ARCH} | sed -ne 's,^.*\\(eb\\)$,\\1,p'`\n\t\tmachine=${arch}${endian}-unknown\n\t\t;;\n\t    *) machine=${UNAME_MACHINE_ARCH}-unknown ;;\n\tesac\n\t# The Operating System including object format, if it has switched\n\t# to ELF recently (or will in the future) and ABI.\n\tcase \"${UNAME_MACHINE_ARCH}\" in\n\t    earm*)\n\t\tos=netbsdelf\n\t\t;;\n\t    arm*|i386|m68k|ns32k|sh3*|sparc|vax)\n\t\teval $set_cc_for_build\n\t\tif echo __ELF__ | $CC_FOR_BUILD -E - 2>/dev/null \\\n\t\t\t| grep -q __ELF__\n\t\tthen\n\t\t    # Once all utilities can be ECOFF (netbsdecoff) or a.out (netbsdaout).\n\t\t    # Return netbsd for either.  FIX?\n\t\t    os=netbsd\n\t\telse\n\t\t    os=netbsdelf\n\t\tfi\n\t\t;;\n\t    *)\n\t\tos=netbsd\n\t\t;;\n\tesac\n\t# Determine ABI tags.\n\tcase \"${UNAME_MACHINE_ARCH}\" in\n\t    earm*)\n\t\texpr='s/^earmv[0-9]/-eabi/;s/eb$//'\n\t\tabi=`echo ${UNAME_MACHINE_ARCH} | sed -e \"$expr\"`\n\t\t;;\n\tesac\n\t# The OS release\n\t# Debian GNU/NetBSD machines have a different userland, and\n\t# thus, need a distinct triplet. However, they do not need\n\t# kernel version information, so it can be replaced with a\n\t# suitable tag, in the style of linux-gnu.\n\tcase \"${UNAME_VERSION}\" in\n\t    Debian*)\n\t\trelease='-gnu'\n\t\t;;\n\t    *)\n\t\trelease=`echo ${UNAME_RELEASE} | sed -e 's/[-_].*//' | cut -d. -f1,2`\n\t\t;;\n\tesac\n\t# Since CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM:\n\t# contains redundant information, the shorter form:\n\t# CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM is used.\n\techo \"${machine}-${os}${release}${abi}\"\n\texit ;;\n    *:Bitrig:*:*)\n\tUNAME_MACHINE_ARCH=`arch | sed 's/Bitrig.//'`\n\techo ${UNAME_MACHINE_ARCH}-unknown-bitrig${UNAME_RELEASE}\n\texit ;;\n    *:OpenBSD:*:*)\n\tUNAME_MACHINE_ARCH=`arch | sed 's/OpenBSD.//'`\n\techo ${UNAME_MACHINE_ARCH}-unknown-openbsd${UNAME_RELEASE}\n\texit ;;\n    *:LibertyBSD:*:*)\n\tUNAME_MACHINE_ARCH=`arch | sed 's/^.*BSD\\.//'`\n\techo ${UNAME_MACHINE_ARCH}-unknown-libertybsd${UNAME_RELEASE}\n\texit ;;\n    *:ekkoBSD:*:*)\n\techo ${UNAME_MACHINE}-unknown-ekkobsd${UNAME_RELEASE}\n\texit ;;\n    *:SolidBSD:*:*)\n\techo ${UNAME_MACHINE}-unknown-solidbsd${UNAME_RELEASE}\n\texit ;;\n    macppc:MirBSD:*:*)\n\techo powerpc-unknown-mirbsd${UNAME_RELEASE}\n\texit ;;\n    *:MirBSD:*:*)\n\techo ${UNAME_MACHINE}-unknown-mirbsd${UNAME_RELEASE}\n\texit ;;\n    *:Sortix:*:*)\n\techo ${UNAME_MACHINE}-unknown-sortix\n\texit ;;\n    alpha:OSF1:*:*)\n\tcase $UNAME_RELEASE in\n\t*4.0)\n\t\tUNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $3}'`\n\t\t;;\n\t*5.*)\n\t\tUNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $4}'`\n\t\t;;\n\tesac\n\t# According to Compaq, /usr/sbin/psrinfo has been available on\n\t# OSF/1 and Tru64 systems produced since 1995.  I hope that\n\t# covers most systems running today.  This code pipes the CPU\n\t# types through head -n 1, so we only detect the type of CPU 0.\n\tALPHA_CPU_TYPE=`/usr/sbin/psrinfo -v | sed -n -e 's/^  The alpha \\(.*\\) processor.*$/\\1/p' | head -n 1`\n\tcase \"$ALPHA_CPU_TYPE\" in\n\t    \"EV4 (21064)\")\n\t\tUNAME_MACHINE=alpha ;;\n\t    \"EV4.5 (21064)\")\n\t\tUNAME_MACHINE=alpha ;;\n\t    \"LCA4 (21066/21068)\")\n\t\tUNAME_MACHINE=alpha ;;\n\t    \"EV5 (21164)\")\n\t\tUNAME_MACHINE=alphaev5 ;;\n\t    \"EV5.6 (21164A)\")\n\t\tUNAME_MACHINE=alphaev56 ;;\n\t    \"EV5.6 (21164PC)\")\n\t\tUNAME_MACHINE=alphapca56 ;;\n\t    \"EV5.7 (21164PC)\")\n\t\tUNAME_MACHINE=alphapca57 ;;\n\t    \"EV6 (21264)\")\n\t\tUNAME_MACHINE=alphaev6 ;;\n\t    \"EV6.7 (21264A)\")\n\t\tUNAME_MACHINE=alphaev67 ;;\n\t    \"EV6.8CB (21264C)\")\n\t\tUNAME_MACHINE=alphaev68 ;;\n\t    \"EV6.8AL (21264B)\")\n\t\tUNAME_MACHINE=alphaev68 ;;\n\t    \"EV6.8CX (21264D)\")\n\t\tUNAME_MACHINE=alphaev68 ;;\n\t    \"EV6.9A (21264/EV69A)\")\n\t\tUNAME_MACHINE=alphaev69 ;;\n\t    \"EV7 (21364)\")\n\t\tUNAME_MACHINE=alphaev7 ;;\n\t    \"EV7.9 (21364A)\")\n\t\tUNAME_MACHINE=alphaev79 ;;\n\tesac\n\t# A Pn.n version is a patched version.\n\t# A Vn.n version is a released version.\n\t# A Tn.n version is a released field test version.\n\t# A Xn.n version is an unreleased experimental baselevel.\n\t# 1.2 uses \"1.2\" for uname -r.\n\techo ${UNAME_MACHINE}-dec-osf`echo ${UNAME_RELEASE} | sed -e 's/^[PVTX]//' | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz`\n\t# Reset EXIT trap before exiting to avoid spurious non-zero exit code.\n\texitcode=$?\n\ttrap '' 0\n\texit $exitcode ;;\n    Alpha\\ *:Windows_NT*:*)\n\t# How do we know it's Interix rather than the generic POSIX subsystem?\n\t# Should we change UNAME_MACHINE based on the output of uname instead\n\t# of the specific Alpha model?\n\techo alpha-pc-interix\n\texit ;;\n    21064:Windows_NT:50:3)\n\techo alpha-dec-winnt3.5\n\texit ;;\n    Amiga*:UNIX_System_V:4.0:*)\n\techo m68k-unknown-sysv4\n\texit ;;\n    *:[Aa]miga[Oo][Ss]:*:*)\n\techo ${UNAME_MACHINE}-unknown-amigaos\n\texit ;;\n    *:[Mm]orph[Oo][Ss]:*:*)\n\techo ${UNAME_MACHINE}-unknown-morphos\n\texit ;;\n    *:OS/390:*:*)\n\techo i370-ibm-openedition\n\texit ;;\n    *:z/VM:*:*)\n\techo s390-ibm-zvmoe\n\texit ;;\n    *:OS400:*:*)\n\techo powerpc-ibm-os400\n\texit ;;\n    arm:RISC*:1.[012]*:*|arm:riscix:1.[012]*:*)\n\techo arm-acorn-riscix${UNAME_RELEASE}\n\texit ;;\n    arm*:riscos:*:*|arm*:RISCOS:*:*)\n\techo arm-unknown-riscos\n\texit ;;\n    SR2?01:HI-UX/MPP:*:* | SR8000:HI-UX/MPP:*:*)\n\techo hppa1.1-hitachi-hiuxmpp\n\texit ;;\n    Pyramid*:OSx*:*:* | MIS*:OSx*:*:* | MIS*:SMP_DC-OSx*:*:*)\n\t# akee@wpdis03.wpafb.af.mil (Earle F. Ake) contributed MIS and NILE.\n\tif test \"`(/bin/universe) 2>/dev/null`\" = att ; then\n\t\techo pyramid-pyramid-sysv3\n\telse\n\t\techo pyramid-pyramid-bsd\n\tfi\n\texit ;;\n    NILE*:*:*:dcosx)\n\techo pyramid-pyramid-svr4\n\texit ;;\n    DRS?6000:unix:4.0:6*)\n\techo sparc-icl-nx6\n\texit ;;\n    DRS?6000:UNIX_SV:4.2*:7* | DRS?6000:isis:4.2*:7*)\n\tcase `/usr/bin/uname -p` in\n\t    sparc) echo sparc-icl-nx7; exit ;;\n\tesac ;;\n    s390x:SunOS:*:*)\n\techo ${UNAME_MACHINE}-ibm-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'`\n\texit ;;\n    sun4H:SunOS:5.*:*)\n\techo sparc-hal-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'`\n\texit ;;\n    sun4*:SunOS:5.*:* | tadpole*:SunOS:5.*:*)\n\techo sparc-sun-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'`\n\texit ;;\n    i86pc:AuroraUX:5.*:* | i86xen:AuroraUX:5.*:*)\n\techo i386-pc-auroraux${UNAME_RELEASE}\n\texit ;;\n    i86pc:SunOS:5.*:* | i86xen:SunOS:5.*:*)\n\teval $set_cc_for_build\n\tSUN_ARCH=i386\n\t# If there is a compiler, see if it is configured for 64-bit objects.\n\t# Note that the Sun cc does not turn __LP64__ into 1 like gcc does.\n\t# This test works for both compilers.\n\tif [ \"$CC_FOR_BUILD\" != no_compiler_found ]; then\n\t    if (echo '#ifdef __amd64'; echo IS_64BIT_ARCH; echo '#endif') | \\\n\t\t(CCOPTS=\"\" $CC_FOR_BUILD -E - 2>/dev/null) | \\\n\t\tgrep IS_64BIT_ARCH >/dev/null\n\t    then\n\t\tSUN_ARCH=x86_64\n\t    fi\n\tfi\n\techo ${SUN_ARCH}-pc-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'`\n\texit ;;\n    sun4*:SunOS:6*:*)\n\t# According to config.sub, this is the proper way to canonicalize\n\t# SunOS6.  Hard to guess exactly what SunOS6 will be like, but\n\t# it's likely to be more like Solaris than SunOS4.\n\techo sparc-sun-solaris3`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'`\n\texit ;;\n    sun4*:SunOS:*:*)\n\tcase \"`/usr/bin/arch -k`\" in\n\t    Series*|S4*)\n\t\tUNAME_RELEASE=`uname -v`\n\t\t;;\n\tesac\n\t# Japanese Language versions have a version number like `4.1.3-JL'.\n\techo sparc-sun-sunos`echo ${UNAME_RELEASE}|sed -e 's/-/_/'`\n\texit ;;\n    sun3*:SunOS:*:*)\n\techo m68k-sun-sunos${UNAME_RELEASE}\n\texit ;;\n    sun*:*:4.2BSD:*)\n\tUNAME_RELEASE=`(sed 1q /etc/motd | awk '{print substr($5,1,3)}') 2>/dev/null`\n\ttest \"x${UNAME_RELEASE}\" = x && UNAME_RELEASE=3\n\tcase \"`/bin/arch`\" in\n\t    sun3)\n\t\techo m68k-sun-sunos${UNAME_RELEASE}\n\t\t;;\n\t    sun4)\n\t\techo sparc-sun-sunos${UNAME_RELEASE}\n\t\t;;\n\tesac\n\texit ;;\n    aushp:SunOS:*:*)\n\techo sparc-auspex-sunos${UNAME_RELEASE}\n\texit ;;\n    # The situation for MiNT is a little confusing.  The machine name\n    # can be virtually everything (everything which is not\n    # \"atarist\" or \"atariste\" at least should have a processor\n    # > m68000).  The system name ranges from \"MiNT\" over \"FreeMiNT\"\n    # to the lowercase version \"mint\" (or \"freemint\").  Finally\n    # the system name \"TOS\" denotes a system which is actually not\n    # MiNT.  But MiNT is downward compatible to TOS, so this should\n    # be no problem.\n    atarist[e]:*MiNT:*:* | atarist[e]:*mint:*:* | atarist[e]:*TOS:*:*)\n\techo m68k-atari-mint${UNAME_RELEASE}\n\texit ;;\n    atari*:*MiNT:*:* | atari*:*mint:*:* | atarist[e]:*TOS:*:*)\n\techo m68k-atari-mint${UNAME_RELEASE}\n\texit ;;\n    *falcon*:*MiNT:*:* | *falcon*:*mint:*:* | *falcon*:*TOS:*:*)\n\techo m68k-atari-mint${UNAME_RELEASE}\n\texit ;;\n    milan*:*MiNT:*:* | milan*:*mint:*:* | *milan*:*TOS:*:*)\n\techo m68k-milan-mint${UNAME_RELEASE}\n\texit ;;\n    hades*:*MiNT:*:* | hades*:*mint:*:* | *hades*:*TOS:*:*)\n\techo m68k-hades-mint${UNAME_RELEASE}\n\texit ;;\n    *:*MiNT:*:* | *:*mint:*:* | *:*TOS:*:*)\n\techo m68k-unknown-mint${UNAME_RELEASE}\n\texit ;;\n    m68k:machten:*:*)\n\techo m68k-apple-machten${UNAME_RELEASE}\n\texit ;;\n    powerpc:machten:*:*)\n\techo powerpc-apple-machten${UNAME_RELEASE}\n\texit ;;\n    RISC*:Mach:*:*)\n\techo mips-dec-mach_bsd4.3\n\texit ;;\n    RISC*:ULTRIX:*:*)\n\techo mips-dec-ultrix${UNAME_RELEASE}\n\texit ;;\n    VAX*:ULTRIX*:*:*)\n\techo vax-dec-ultrix${UNAME_RELEASE}\n\texit ;;\n    2020:CLIX:*:* | 2430:CLIX:*:*)\n\techo clipper-intergraph-clix${UNAME_RELEASE}\n\texit ;;\n    mips:*:*:UMIPS | mips:*:*:RISCos)\n\teval $set_cc_for_build\n\tsed 's/^\t//' << EOF >$dummy.c\n#ifdef __cplusplus\n#include <stdio.h>  /* for printf() prototype */\n\tint main (int argc, char *argv[]) {\n#else\n\tint main (argc, argv) int argc; char *argv[]; {\n#endif\n\t#if defined (host_mips) && defined (MIPSEB)\n\t#if defined (SYSTYPE_SYSV)\n\t  printf (\"mips-mips-riscos%ssysv\\n\", argv[1]); exit (0);\n\t#endif\n\t#if defined (SYSTYPE_SVR4)\n\t  printf (\"mips-mips-riscos%ssvr4\\n\", argv[1]); exit (0);\n\t#endif\n\t#if defined (SYSTYPE_BSD43) || defined(SYSTYPE_BSD)\n\t  printf (\"mips-mips-riscos%sbsd\\n\", argv[1]); exit (0);\n\t#endif\n\t#endif\n\t  exit (-1);\n\t}\nEOF\n\t$CC_FOR_BUILD -o $dummy $dummy.c &&\n\t  dummyarg=`echo \"${UNAME_RELEASE}\" | sed -n 's/\\([0-9]*\\).*/\\1/p'` &&\n\t  SYSTEM_NAME=`$dummy $dummyarg` &&\n\t    { echo \"$SYSTEM_NAME\"; exit; }\n\techo mips-mips-riscos${UNAME_RELEASE}\n\texit ;;\n    Motorola:PowerMAX_OS:*:*)\n\techo powerpc-motorola-powermax\n\texit ;;\n    Motorola:*:4.3:PL8-*)\n\techo powerpc-harris-powermax\n\texit ;;\n    Night_Hawk:*:*:PowerMAX_OS | Synergy:PowerMAX_OS:*:*)\n\techo powerpc-harris-powermax\n\texit ;;\n    Night_Hawk:Power_UNIX:*:*)\n\techo powerpc-harris-powerunix\n\texit ;;\n    m88k:CX/UX:7*:*)\n\techo m88k-harris-cxux7\n\texit ;;\n    m88k:*:4*:R4*)\n\techo m88k-motorola-sysv4\n\texit ;;\n    m88k:*:3*:R3*)\n\techo m88k-motorola-sysv3\n\texit ;;\n    AViiON:dgux:*:*)\n\t# DG/UX returns AViiON for all architectures\n\tUNAME_PROCESSOR=`/usr/bin/uname -p`\n\tif [ $UNAME_PROCESSOR = mc88100 ] || [ $UNAME_PROCESSOR = mc88110 ]\n\tthen\n\t    if [ ${TARGET_BINARY_INTERFACE}x = m88kdguxelfx ] || \\\n\t       [ ${TARGET_BINARY_INTERFACE}x = x ]\n\t    then\n\t\techo m88k-dg-dgux${UNAME_RELEASE}\n\t    else\n\t\techo m88k-dg-dguxbcs${UNAME_RELEASE}\n\t    fi\n\telse\n\t    echo i586-dg-dgux${UNAME_RELEASE}\n\tfi\n\texit ;;\n    M88*:DolphinOS:*:*)\t# DolphinOS (SVR3)\n\techo m88k-dolphin-sysv3\n\texit ;;\n    M88*:*:R3*:*)\n\t# Delta 88k system running SVR3\n\techo m88k-motorola-sysv3\n\texit ;;\n    XD88*:*:*:*) # Tektronix XD88 system running UTekV (SVR3)\n\techo m88k-tektronix-sysv3\n\texit ;;\n    Tek43[0-9][0-9]:UTek:*:*) # Tektronix 4300 system running UTek (BSD)\n\techo m68k-tektronix-bsd\n\texit ;;\n    *:IRIX*:*:*)\n\techo mips-sgi-irix`echo ${UNAME_RELEASE}|sed -e 's/-/_/g'`\n\texit ;;\n    ????????:AIX?:[12].1:2)   # AIX 2.2.1 or AIX 2.1.1 is RT/PC AIX.\n\techo romp-ibm-aix     # uname -m gives an 8 hex-code CPU id\n\texit ;;               # Note that: echo \"'`uname -s`'\" gives 'AIX '\n    i*86:AIX:*:*)\n\techo i386-ibm-aix\n\texit ;;\n    ia64:AIX:*:*)\n\tif [ -x /usr/bin/oslevel ] ; then\n\t\tIBM_REV=`/usr/bin/oslevel`\n\telse\n\t\tIBM_REV=${UNAME_VERSION}.${UNAME_RELEASE}\n\tfi\n\techo ${UNAME_MACHINE}-ibm-aix${IBM_REV}\n\texit ;;\n    *:AIX:2:3)\n\tif grep bos325 /usr/include/stdio.h >/dev/null 2>&1; then\n\t\teval $set_cc_for_build\n\t\tsed 's/^\t\t//' << EOF >$dummy.c\n\t\t#include <sys/systemcfg.h>\n\n\t\tmain()\n\t\t\t{\n\t\t\tif (!__power_pc())\n\t\t\t\texit(1);\n\t\t\tputs(\"powerpc-ibm-aix3.2.5\");\n\t\t\texit(0);\n\t\t\t}\nEOF\n\t\tif $CC_FOR_BUILD -o $dummy $dummy.c && SYSTEM_NAME=`$dummy`\n\t\tthen\n\t\t\techo \"$SYSTEM_NAME\"\n\t\telse\n\t\t\techo rs6000-ibm-aix3.2.5\n\t\tfi\n\telif grep bos324 /usr/include/stdio.h >/dev/null 2>&1; then\n\t\techo rs6000-ibm-aix3.2.4\n\telse\n\t\techo rs6000-ibm-aix3.2\n\tfi\n\texit ;;\n    *:AIX:*:[4567])\n\tIBM_CPU_ID=`/usr/sbin/lsdev -C -c processor -S available | sed 1q | awk '{ print $1 }'`\n\tif /usr/sbin/lsattr -El ${IBM_CPU_ID} | grep ' POWER' >/dev/null 2>&1; then\n\t\tIBM_ARCH=rs6000\n\telse\n\t\tIBM_ARCH=powerpc\n\tfi\n\tif [ -x /usr/bin/lslpp ] ; then\n\t\tIBM_REV=`/usr/bin/lslpp -Lqc bos.rte.libc |\n\t\t\t   awk -F: '{ print $3 }' | sed s/[0-9]*$/0/`\n\telse\n\t\tIBM_REV=${UNAME_VERSION}.${UNAME_RELEASE}\n\tfi\n\techo ${IBM_ARCH}-ibm-aix${IBM_REV}\n\texit ;;\n    *:AIX:*:*)\n\techo rs6000-ibm-aix\n\texit ;;\n    ibmrt:4.4BSD:*|romp-ibm:BSD:*)\n\techo romp-ibm-bsd4.4\n\texit ;;\n    ibmrt:*BSD:*|romp-ibm:BSD:*)            # covers RT/PC BSD and\n\techo romp-ibm-bsd${UNAME_RELEASE}   # 4.3 with uname added to\n\texit ;;                             # report: romp-ibm BSD 4.3\n    *:BOSX:*:*)\n\techo rs6000-bull-bosx\n\texit ;;\n    DPX/2?00:B.O.S.:*:*)\n\techo m68k-bull-sysv3\n\texit ;;\n    9000/[34]??:4.3bsd:1.*:*)\n\techo m68k-hp-bsd\n\texit ;;\n    hp300:4.4BSD:*:* | 9000/[34]??:4.3bsd:2.*:*)\n\techo m68k-hp-bsd4.4\n\texit ;;\n    9000/[34678]??:HP-UX:*:*)\n\tHPUX_REV=`echo ${UNAME_RELEASE}|sed -e 's/[^.]*.[0B]*//'`\n\tcase \"${UNAME_MACHINE}\" in\n\t    9000/31? )            HP_ARCH=m68000 ;;\n\t    9000/[34]?? )         HP_ARCH=m68k ;;\n\t    9000/[678][0-9][0-9])\n\t\tif [ -x /usr/bin/getconf ]; then\n\t\t    sc_cpu_version=`/usr/bin/getconf SC_CPU_VERSION 2>/dev/null`\n\t\t    sc_kernel_bits=`/usr/bin/getconf SC_KERNEL_BITS 2>/dev/null`\n\t\t    case \"${sc_cpu_version}\" in\n\t\t      523) HP_ARCH=hppa1.0 ;; # CPU_PA_RISC1_0\n\t\t      528) HP_ARCH=hppa1.1 ;; # CPU_PA_RISC1_1\n\t\t      532)                      # CPU_PA_RISC2_0\n\t\t\tcase \"${sc_kernel_bits}\" in\n\t\t\t  32) HP_ARCH=hppa2.0n ;;\n\t\t\t  64) HP_ARCH=hppa2.0w ;;\n\t\t\t  '') HP_ARCH=hppa2.0 ;;   # HP-UX 10.20\n\t\t\tesac ;;\n\t\t    esac\n\t\tfi\n\t\tif [ \"${HP_ARCH}\" = \"\" ]; then\n\t\t    eval $set_cc_for_build\n\t\t    sed 's/^\t\t//' << EOF >$dummy.c\n\n\t\t#define _HPUX_SOURCE\n\t\t#include <stdlib.h>\n\t\t#include <unistd.h>\n\n\t\tint main ()\n\t\t{\n\t\t#if defined(_SC_KERNEL_BITS)\n\t\t    long bits = sysconf(_SC_KERNEL_BITS);\n\t\t#endif\n\t\t    long cpu  = sysconf (_SC_CPU_VERSION);\n\n\t\t    switch (cpu)\n\t\t\t{\n\t\t\tcase CPU_PA_RISC1_0: puts (\"hppa1.0\"); break;\n\t\t\tcase CPU_PA_RISC1_1: puts (\"hppa1.1\"); break;\n\t\t\tcase CPU_PA_RISC2_0:\n\t\t#if defined(_SC_KERNEL_BITS)\n\t\t\t    switch (bits)\n\t\t\t\t{\n\t\t\t\tcase 64: puts (\"hppa2.0w\"); break;\n\t\t\t\tcase 32: puts (\"hppa2.0n\"); break;\n\t\t\t\tdefault: puts (\"hppa2.0\"); break;\n\t\t\t\t} break;\n\t\t#else  /* !defined(_SC_KERNEL_BITS) */\n\t\t\t    puts (\"hppa2.0\"); break;\n\t\t#endif\n\t\t\tdefault: puts (\"hppa1.0\"); break;\n\t\t\t}\n\t\t    exit (0);\n\t\t}\nEOF\n\t\t    (CCOPTS=\"\" $CC_FOR_BUILD -o $dummy $dummy.c 2>/dev/null) && HP_ARCH=`$dummy`\n\t\t    test -z \"$HP_ARCH\" && HP_ARCH=hppa\n\t\tfi ;;\n\tesac\n\tif [ ${HP_ARCH} = hppa2.0w ]\n\tthen\n\t    eval $set_cc_for_build\n\n\t    # hppa2.0w-hp-hpux* has a 64-bit kernel and a compiler generating\n\t    # 32-bit code.  hppa64-hp-hpux* has the same kernel and a compiler\n\t    # generating 64-bit code.  GNU and HP use different nomenclature:\n\t    #\n\t    # $ CC_FOR_BUILD=cc ./config.guess\n\t    # => hppa2.0w-hp-hpux11.23\n\t    # $ CC_FOR_BUILD=\"cc +DA2.0w\" ./config.guess\n\t    # => hppa64-hp-hpux11.23\n\n\t    if echo __LP64__ | (CCOPTS=\"\" $CC_FOR_BUILD -E - 2>/dev/null) |\n\t\tgrep -q __LP64__\n\t    then\n\t\tHP_ARCH=hppa2.0w\n\t    else\n\t\tHP_ARCH=hppa64\n\t    fi\n\tfi\n\techo ${HP_ARCH}-hp-hpux${HPUX_REV}\n\texit ;;\n    ia64:HP-UX:*:*)\n\tHPUX_REV=`echo ${UNAME_RELEASE}|sed -e 's/[^.]*.[0B]*//'`\n\techo ia64-hp-hpux${HPUX_REV}\n\texit ;;\n    3050*:HI-UX:*:*)\n\teval $set_cc_for_build\n\tsed 's/^\t//' << EOF >$dummy.c\n\t#include <unistd.h>\n\tint\n\tmain ()\n\t{\n\t  long cpu = sysconf (_SC_CPU_VERSION);\n\t  /* The order matters, because CPU_IS_HP_MC68K erroneously returns\n\t     true for CPU_PA_RISC1_0.  CPU_IS_PA_RISC returns correct\n\t     results, however.  */\n\t  if (CPU_IS_PA_RISC (cpu))\n\t    {\n\t      switch (cpu)\n\t\t{\n\t\t  case CPU_PA_RISC1_0: puts (\"hppa1.0-hitachi-hiuxwe2\"); break;\n\t\t  case CPU_PA_RISC1_1: puts (\"hppa1.1-hitachi-hiuxwe2\"); break;\n\t\t  case CPU_PA_RISC2_0: puts (\"hppa2.0-hitachi-hiuxwe2\"); break;\n\t\t  default: puts (\"hppa-hitachi-hiuxwe2\"); break;\n\t\t}\n\t    }\n\t  else if (CPU_IS_HP_MC68K (cpu))\n\t    puts (\"m68k-hitachi-hiuxwe2\");\n\t  else puts (\"unknown-hitachi-hiuxwe2\");\n\t  exit (0);\n\t}\nEOF\n\t$CC_FOR_BUILD -o $dummy $dummy.c && SYSTEM_NAME=`$dummy` &&\n\t\t{ echo \"$SYSTEM_NAME\"; exit; }\n\techo unknown-hitachi-hiuxwe2\n\texit ;;\n    9000/7??:4.3bsd:*:* | 9000/8?[79]:4.3bsd:*:* )\n\techo hppa1.1-hp-bsd\n\texit ;;\n    9000/8??:4.3bsd:*:*)\n\techo hppa1.0-hp-bsd\n\texit ;;\n    *9??*:MPE/iX:*:* | *3000*:MPE/iX:*:*)\n\techo hppa1.0-hp-mpeix\n\texit ;;\n    hp7??:OSF1:*:* | hp8?[79]:OSF1:*:* )\n\techo hppa1.1-hp-osf\n\texit ;;\n    hp8??:OSF1:*:*)\n\techo hppa1.0-hp-osf\n\texit ;;\n    i*86:OSF1:*:*)\n\tif [ -x /usr/sbin/sysversion ] ; then\n\t    echo ${UNAME_MACHINE}-unknown-osf1mk\n\telse\n\t    echo ${UNAME_MACHINE}-unknown-osf1\n\tfi\n\texit ;;\n    parisc*:Lites*:*:*)\n\techo hppa1.1-hp-lites\n\texit ;;\n    C1*:ConvexOS:*:* | convex:ConvexOS:C1*:*)\n\techo c1-convex-bsd\n\texit ;;\n    C2*:ConvexOS:*:* | convex:ConvexOS:C2*:*)\n\tif getsysinfo -f scalar_acc\n\tthen echo c32-convex-bsd\n\telse echo c2-convex-bsd\n\tfi\n\texit ;;\n    C34*:ConvexOS:*:* | convex:ConvexOS:C34*:*)\n\techo c34-convex-bsd\n\texit ;;\n    C38*:ConvexOS:*:* | convex:ConvexOS:C38*:*)\n\techo c38-convex-bsd\n\texit ;;\n    C4*:ConvexOS:*:* | convex:ConvexOS:C4*:*)\n\techo c4-convex-bsd\n\texit ;;\n    CRAY*Y-MP:*:*:*)\n\techo ymp-cray-unicos${UNAME_RELEASE} | sed -e 's/\\.[^.]*$/.X/'\n\texit ;;\n    CRAY*[A-Z]90:*:*:*)\n\techo ${UNAME_MACHINE}-cray-unicos${UNAME_RELEASE} \\\n\t| sed -e 's/CRAY.*\\([A-Z]90\\)/\\1/' \\\n\t      -e y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/ \\\n\t      -e 's/\\.[^.]*$/.X/'\n\texit ;;\n    CRAY*TS:*:*:*)\n\techo t90-cray-unicos${UNAME_RELEASE} | sed -e 's/\\.[^.]*$/.X/'\n\texit ;;\n    CRAY*T3E:*:*:*)\n\techo alphaev5-cray-unicosmk${UNAME_RELEASE} | sed -e 's/\\.[^.]*$/.X/'\n\texit ;;\n    CRAY*SV1:*:*:*)\n\techo sv1-cray-unicos${UNAME_RELEASE} | sed -e 's/\\.[^.]*$/.X/'\n\texit ;;\n    *:UNICOS/mp:*:*)\n\techo craynv-cray-unicosmp${UNAME_RELEASE} | sed -e 's/\\.[^.]*$/.X/'\n\texit ;;\n    F30[01]:UNIX_System_V:*:* | F700:UNIX_System_V:*:*)\n\tFUJITSU_PROC=`uname -m | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz`\n\tFUJITSU_SYS=`uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\\///'`\n\tFUJITSU_REL=`echo ${UNAME_RELEASE} | sed -e 's/ /_/'`\n\techo \"${FUJITSU_PROC}-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}\"\n\texit ;;\n    5000:UNIX_System_V:4.*:*)\n\tFUJITSU_SYS=`uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\\///'`\n\tFUJITSU_REL=`echo ${UNAME_RELEASE} | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/ /_/'`\n\techo \"sparc-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}\"\n\texit ;;\n    i*86:BSD/386:*:* | i*86:BSD/OS:*:* | *:Ascend\\ Embedded/OS:*:*)\n\techo ${UNAME_MACHINE}-pc-bsdi${UNAME_RELEASE}\n\texit ;;\n    sparc*:BSD/OS:*:*)\n\techo sparc-unknown-bsdi${UNAME_RELEASE}\n\texit ;;\n    *:BSD/OS:*:*)\n\techo ${UNAME_MACHINE}-unknown-bsdi${UNAME_RELEASE}\n\texit ;;\n    *:FreeBSD:*:*)\n\tUNAME_PROCESSOR=`/usr/bin/uname -p`\n\tcase ${UNAME_PROCESSOR} in\n\t    amd64)\n\t\tUNAME_PROCESSOR=x86_64 ;;\n\t    i386)\n\t\tUNAME_PROCESSOR=i586 ;;\n\tesac\n\techo ${UNAME_PROCESSOR}-unknown-freebsd`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`\n\texit ;;\n    i*:CYGWIN*:*)\n\techo ${UNAME_MACHINE}-pc-cygwin\n\texit ;;\n    *:MINGW64*:*)\n\techo ${UNAME_MACHINE}-pc-mingw64\n\texit ;;\n    *:MINGW*:*)\n\techo ${UNAME_MACHINE}-pc-mingw32\n\texit ;;\n    *:MSYS*:*)\n\techo ${UNAME_MACHINE}-pc-msys\n\texit ;;\n    i*:windows32*:*)\n\t# uname -m includes \"-pc\" on this system.\n\techo ${UNAME_MACHINE}-mingw32\n\texit ;;\n    i*:PW*:*)\n\techo ${UNAME_MACHINE}-pc-pw32\n\texit ;;\n    *:Interix*:*)\n\tcase ${UNAME_MACHINE} in\n\t    x86)\n\t\techo i586-pc-interix${UNAME_RELEASE}\n\t\texit ;;\n\t    authenticamd | genuineintel | EM64T)\n\t\techo x86_64-unknown-interix${UNAME_RELEASE}\n\t\texit ;;\n\t    IA64)\n\t\techo ia64-unknown-interix${UNAME_RELEASE}\n\t\texit ;;\n\tesac ;;\n    [345]86:Windows_95:* | [345]86:Windows_98:* | [345]86:Windows_NT:*)\n\techo i${UNAME_MACHINE}-pc-mks\n\texit ;;\n    8664:Windows_NT:*)\n\techo x86_64-pc-mks\n\texit ;;\n    i*:Windows_NT*:* | Pentium*:Windows_NT*:*)\n\t# How do we know it's Interix rather than the generic POSIX subsystem?\n\t# It also conflicts with pre-2.0 versions of AT&T UWIN. Should we\n\t# UNAME_MACHINE based on the output of uname instead of i386?\n\techo i586-pc-interix\n\texit ;;\n    i*:UWIN*:*)\n\techo ${UNAME_MACHINE}-pc-uwin\n\texit ;;\n    amd64:CYGWIN*:*:* | x86_64:CYGWIN*:*:*)\n\techo x86_64-unknown-cygwin\n\texit ;;\n    p*:CYGWIN*:*)\n\techo powerpcle-unknown-cygwin\n\texit ;;\n    prep*:SunOS:5.*:*)\n\techo powerpcle-unknown-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'`\n\texit ;;\n    *:GNU:*:*)\n\t# the GNU system\n\techo `echo ${UNAME_MACHINE}|sed -e 's,[-/].*$,,'`-unknown-${LIBC}`echo ${UNAME_RELEASE}|sed -e 's,/.*$,,'`\n\texit ;;\n    *:GNU/*:*:*)\n\t# other systems with GNU libc and userland\n\techo ${UNAME_MACHINE}-unknown-`echo ${UNAME_SYSTEM} | sed 's,^[^/]*/,,' | tr \"[:upper:]\" \"[:lower:]\"``echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`-${LIBC}\n\texit ;;\n    i*86:Minix:*:*)\n\techo ${UNAME_MACHINE}-pc-minix\n\texit ;;\n    aarch64:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    aarch64_be:Linux:*:*)\n\tUNAME_MACHINE=aarch64_be\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    alpha:Linux:*:*)\n\tcase `sed -n '/^cpu model/s/^.*: \\(.*\\)/\\1/p' < /proc/cpuinfo` in\n\t  EV5)   UNAME_MACHINE=alphaev5 ;;\n\t  EV56)  UNAME_MACHINE=alphaev56 ;;\n\t  PCA56) UNAME_MACHINE=alphapca56 ;;\n\t  PCA57) UNAME_MACHINE=alphapca56 ;;\n\t  EV6)   UNAME_MACHINE=alphaev6 ;;\n\t  EV67)  UNAME_MACHINE=alphaev67 ;;\n\t  EV68*) UNAME_MACHINE=alphaev68 ;;\n\tesac\n\tobjdump --private-headers /bin/sh | grep -q ld.so.1\n\tif test \"$?\" = 0 ; then LIBC=gnulibc1 ; fi\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    arc:Linux:*:* | arceb:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    arm*:Linux:*:*)\n\teval $set_cc_for_build\n\tif echo __ARM_EABI__ | $CC_FOR_BUILD -E - 2>/dev/null \\\n\t    | grep -q __ARM_EABI__\n\tthen\n\t    echo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\telse\n\t    if echo __ARM_PCS_VFP | $CC_FOR_BUILD -E - 2>/dev/null \\\n\t\t| grep -q __ARM_PCS_VFP\n\t    then\n\t\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}eabi\n\t    else\n\t\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}eabihf\n\t    fi\n\tfi\n\texit ;;\n    avr32*:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    cris:Linux:*:*)\n\techo ${UNAME_MACHINE}-axis-linux-${LIBC}\n\texit ;;\n    crisv32:Linux:*:*)\n\techo ${UNAME_MACHINE}-axis-linux-${LIBC}\n\texit ;;\n    e2k:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    frv:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    hexagon:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    i*86:Linux:*:*)\n\techo ${UNAME_MACHINE}-pc-linux-${LIBC}\n\texit ;;\n    ia64:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    k1om:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    m32r*:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    m68*:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    mips:Linux:*:* | mips64:Linux:*:*)\n\teval $set_cc_for_build\n\tsed 's/^\t//' << EOF >$dummy.c\n\t#undef CPU\n\t#undef ${UNAME_MACHINE}\n\t#undef ${UNAME_MACHINE}el\n\t#if defined(__MIPSEL__) || defined(__MIPSEL) || defined(_MIPSEL) || defined(MIPSEL)\n\tCPU=${UNAME_MACHINE}el\n\t#else\n\t#if defined(__MIPSEB__) || defined(__MIPSEB) || defined(_MIPSEB) || defined(MIPSEB)\n\tCPU=${UNAME_MACHINE}\n\t#else\n\tCPU=\n\t#endif\n\t#endif\nEOF\n\teval `$CC_FOR_BUILD -E $dummy.c 2>/dev/null | grep '^CPU'`\n\ttest x\"${CPU}\" != x && { echo \"${CPU}-unknown-linux-${LIBC}\"; exit; }\n\t;;\n    mips64el:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    openrisc*:Linux:*:*)\n\techo or1k-unknown-linux-${LIBC}\n\texit ;;\n    or32:Linux:*:* | or1k*:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    padre:Linux:*:*)\n\techo sparc-unknown-linux-${LIBC}\n\texit ;;\n    parisc64:Linux:*:* | hppa64:Linux:*:*)\n\techo hppa64-unknown-linux-${LIBC}\n\texit ;;\n    parisc:Linux:*:* | hppa:Linux:*:*)\n\t# Look for CPU level\n\tcase `grep '^cpu[^a-z]*:' /proc/cpuinfo 2>/dev/null | cut -d' ' -f2` in\n\t  PA7*) echo hppa1.1-unknown-linux-${LIBC} ;;\n\t  PA8*) echo hppa2.0-unknown-linux-${LIBC} ;;\n\t  *)    echo hppa-unknown-linux-${LIBC} ;;\n\tesac\n\texit ;;\n    ppc64:Linux:*:*)\n\techo powerpc64-unknown-linux-${LIBC}\n\texit ;;\n    ppc:Linux:*:*)\n\techo powerpc-unknown-linux-${LIBC}\n\texit ;;\n    ppc64le:Linux:*:*)\n\techo powerpc64le-unknown-linux-${LIBC}\n\texit ;;\n    ppcle:Linux:*:*)\n\techo powerpcle-unknown-linux-${LIBC}\n\texit ;;\n    riscv32:Linux:*:* | riscv64:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    s390:Linux:*:* | s390x:Linux:*:*)\n\techo ${UNAME_MACHINE}-ibm-linux-${LIBC}\n\texit ;;\n    sh64*:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    sh*:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    sparc:Linux:*:* | sparc64:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    tile*:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    vax:Linux:*:*)\n\techo ${UNAME_MACHINE}-dec-linux-${LIBC}\n\texit ;;\n    x86_64:Linux:*:*)\n\techo ${UNAME_MACHINE}-pc-linux-${LIBC}\n\texit ;;\n    xtensa*:Linux:*:*)\n\techo ${UNAME_MACHINE}-unknown-linux-${LIBC}\n\texit ;;\n    i*86:DYNIX/ptx:4*:*)\n\t# ptx 4.0 does uname -s correctly, with DYNIX/ptx in there.\n\t# earlier versions are messed up and put the nodename in both\n\t# sysname and nodename.\n\techo i386-sequent-sysv4\n\texit ;;\n    i*86:UNIX_SV:4.2MP:2.*)\n\t# Unixware is an offshoot of SVR4, but it has its own version\n\t# number series starting with 2...\n\t# I am not positive that other SVR4 systems won't match this,\n\t# I just have to hope.  -- rms.\n\t# Use sysv4.2uw... so that sysv4* matches it.\n\techo ${UNAME_MACHINE}-pc-sysv4.2uw${UNAME_VERSION}\n\texit ;;\n    i*86:OS/2:*:*)\n\t# If we were able to find `uname', then EMX Unix compatibility\n\t# is probably installed.\n\techo ${UNAME_MACHINE}-pc-os2-emx\n\texit ;;\n    i*86:XTS-300:*:STOP)\n\techo ${UNAME_MACHINE}-unknown-stop\n\texit ;;\n    i*86:atheos:*:*)\n\techo ${UNAME_MACHINE}-unknown-atheos\n\texit ;;\n    i*86:syllable:*:*)\n\techo ${UNAME_MACHINE}-pc-syllable\n\texit ;;\n    i*86:LynxOS:2.*:* | i*86:LynxOS:3.[01]*:* | i*86:LynxOS:4.[02]*:*)\n\techo i386-unknown-lynxos${UNAME_RELEASE}\n\texit ;;\n    i*86:*DOS:*:*)\n\techo ${UNAME_MACHINE}-pc-msdosdjgpp\n\texit ;;\n    i*86:*:4.*:* | i*86:SYSTEM_V:4.*:*)\n\tUNAME_REL=`echo ${UNAME_RELEASE} | sed 's/\\/MP$//'`\n\tif grep Novell /usr/include/link.h >/dev/null 2>/dev/null; then\n\t\techo ${UNAME_MACHINE}-univel-sysv${UNAME_REL}\n\telse\n\t\techo ${UNAME_MACHINE}-pc-sysv${UNAME_REL}\n\tfi\n\texit ;;\n    i*86:*:5:[678]*)\n\t# UnixWare 7.x, OpenUNIX and OpenServer 6.\n\tcase `/bin/uname -X | grep \"^Machine\"` in\n\t    *486*)\t     UNAME_MACHINE=i486 ;;\n\t    *Pentium)\t     UNAME_MACHINE=i586 ;;\n\t    *Pent*|*Celeron) UNAME_MACHINE=i686 ;;\n\tesac\n\techo ${UNAME_MACHINE}-unknown-sysv${UNAME_RELEASE}${UNAME_SYSTEM}${UNAME_VERSION}\n\texit ;;\n    i*86:*:3.2:*)\n\tif test -f /usr/options/cb.name; then\n\t\tUNAME_REL=`sed -n 's/.*Version //p' </usr/options/cb.name`\n\t\techo ${UNAME_MACHINE}-pc-isc$UNAME_REL\n\telif /bin/uname -X 2>/dev/null >/dev/null ; then\n\t\tUNAME_REL=`(/bin/uname -X|grep Release|sed -e 's/.*= //')`\n\t\t(/bin/uname -X|grep i80486 >/dev/null) && UNAME_MACHINE=i486\n\t\t(/bin/uname -X|grep '^Machine.*Pentium' >/dev/null) \\\n\t\t\t&& UNAME_MACHINE=i586\n\t\t(/bin/uname -X|grep '^Machine.*Pent *II' >/dev/null) \\\n\t\t\t&& UNAME_MACHINE=i686\n\t\t(/bin/uname -X|grep '^Machine.*Pentium Pro' >/dev/null) \\\n\t\t\t&& UNAME_MACHINE=i686\n\t\techo ${UNAME_MACHINE}-pc-sco$UNAME_REL\n\telse\n\t\techo ${UNAME_MACHINE}-pc-sysv32\n\tfi\n\texit ;;\n    pc:*:*:*)\n\t# Left here for compatibility:\n\t# uname -m prints for DJGPP always 'pc', but it prints nothing about\n\t# the processor, so we play safe by assuming i586.\n\t# Note: whatever this is, it MUST be the same as what config.sub\n\t# prints for the \"djgpp\" host, or else GDB configure will decide that\n\t# this is a cross-build.\n\techo i586-pc-msdosdjgpp\n\texit ;;\n    Intel:Mach:3*:*)\n\techo i386-pc-mach3\n\texit ;;\n    paragon:*:*:*)\n\techo i860-intel-osf1\n\texit ;;\n    i860:*:4.*:*) # i860-SVR4\n\tif grep Stardent /usr/include/sys/uadmin.h >/dev/null 2>&1 ; then\n\t  echo i860-stardent-sysv${UNAME_RELEASE} # Stardent Vistra i860-SVR4\n\telse # Add other i860-SVR4 vendors below as they are discovered.\n\t  echo i860-unknown-sysv${UNAME_RELEASE}  # Unknown i860-SVR4\n\tfi\n\texit ;;\n    mini*:CTIX:SYS*5:*)\n\t# \"miniframe\"\n\techo m68010-convergent-sysv\n\texit ;;\n    mc68k:UNIX:SYSTEM5:3.51m)\n\techo m68k-convergent-sysv\n\texit ;;\n    M680?0:D-NIX:5.3:*)\n\techo m68k-diab-dnix\n\texit ;;\n    M68*:*:R3V[5678]*:*)\n\ttest -r /sysV68 && { echo 'm68k-motorola-sysv'; exit; } ;;\n    3[345]??:*:4.0:3.0 | 3[34]??A:*:4.0:3.0 | 3[34]??,*:*:4.0:3.0 | 3[34]??/*:*:4.0:3.0 | 4400:*:4.0:3.0 | 4850:*:4.0:3.0 | SKA40:*:4.0:3.0 | SDS2:*:4.0:3.0 | SHG2:*:4.0:3.0 | S7501*:*:4.0:3.0)\n\tOS_REL=''\n\ttest -r /etc/.relid \\\n\t&& OS_REL=.`sed -n 's/[^ ]* [^ ]* \\([0-9][0-9]\\).*/\\1/p' < /etc/.relid`\n\t/bin/uname -p 2>/dev/null | grep 86 >/dev/null \\\n\t  && { echo i486-ncr-sysv4.3${OS_REL}; exit; }\n\t/bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \\\n\t  && { echo i586-ncr-sysv4.3${OS_REL}; exit; } ;;\n    3[34]??:*:4.0:* | 3[34]??,*:*:4.0:*)\n\t/bin/uname -p 2>/dev/null | grep 86 >/dev/null \\\n\t  && { echo i486-ncr-sysv4; exit; } ;;\n    NCR*:*:4.2:* | MPRAS*:*:4.2:*)\n\tOS_REL='.3'\n\ttest -r /etc/.relid \\\n\t    && OS_REL=.`sed -n 's/[^ ]* [^ ]* \\([0-9][0-9]\\).*/\\1/p' < /etc/.relid`\n\t/bin/uname -p 2>/dev/null | grep 86 >/dev/null \\\n\t    && { echo i486-ncr-sysv4.3${OS_REL}; exit; }\n\t/bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \\\n\t    && { echo i586-ncr-sysv4.3${OS_REL}; exit; }\n\t/bin/uname -p 2>/dev/null | /bin/grep pteron >/dev/null \\\n\t    && { echo i586-ncr-sysv4.3${OS_REL}; exit; } ;;\n    m68*:LynxOS:2.*:* | m68*:LynxOS:3.0*:*)\n\techo m68k-unknown-lynxos${UNAME_RELEASE}\n\texit ;;\n    mc68030:UNIX_System_V:4.*:*)\n\techo m68k-atari-sysv4\n\texit ;;\n    TSUNAMI:LynxOS:2.*:*)\n\techo sparc-unknown-lynxos${UNAME_RELEASE}\n\texit ;;\n    rs6000:LynxOS:2.*:*)\n\techo rs6000-unknown-lynxos${UNAME_RELEASE}\n\texit ;;\n    PowerPC:LynxOS:2.*:* | PowerPC:LynxOS:3.[01]*:* | PowerPC:LynxOS:4.[02]*:*)\n\techo powerpc-unknown-lynxos${UNAME_RELEASE}\n\texit ;;\n    SM[BE]S:UNIX_SV:*:*)\n\techo mips-dde-sysv${UNAME_RELEASE}\n\texit ;;\n    RM*:ReliantUNIX-*:*:*)\n\techo mips-sni-sysv4\n\texit ;;\n    RM*:SINIX-*:*:*)\n\techo mips-sni-sysv4\n\texit ;;\n    *:SINIX-*:*:*)\n\tif uname -p 2>/dev/null >/dev/null ; then\n\t\tUNAME_MACHINE=`(uname -p) 2>/dev/null`\n\t\techo ${UNAME_MACHINE}-sni-sysv4\n\telse\n\t\techo ns32k-sni-sysv\n\tfi\n\texit ;;\n    PENTIUM:*:4.0*:*)\t# Unisys `ClearPath HMP IX 4000' SVR4/MP effort\n\t\t\t# says <Richard.M.Bartel@ccMail.Census.GOV>\n\techo i586-unisys-sysv4\n\texit ;;\n    *:UNIX_System_V:4*:FTX*)\n\t# From Gerald Hewes <hewes@openmarket.com>.\n\t# How about differentiating between stratus architectures? -djm\n\techo hppa1.1-stratus-sysv4\n\texit ;;\n    *:*:*:FTX*)\n\t# From seanf@swdc.stratus.com.\n\techo i860-stratus-sysv4\n\texit ;;\n    i*86:VOS:*:*)\n\t# From Paul.Green@stratus.com.\n\techo ${UNAME_MACHINE}-stratus-vos\n\texit ;;\n    *:VOS:*:*)\n\t# From Paul.Green@stratus.com.\n\techo hppa1.1-stratus-vos\n\texit ;;\n    mc68*:A/UX:*:*)\n\techo m68k-apple-aux${UNAME_RELEASE}\n\texit ;;\n    news*:NEWS-OS:6*:*)\n\techo mips-sony-newsos6\n\texit ;;\n    R[34]000:*System_V*:*:* | R4000:UNIX_SYSV:*:* | R*000:UNIX_SV:*:*)\n\tif [ -d /usr/nec ]; then\n\t\techo mips-nec-sysv${UNAME_RELEASE}\n\telse\n\t\techo mips-unknown-sysv${UNAME_RELEASE}\n\tfi\n\texit ;;\n    BeBox:BeOS:*:*)\t# BeOS running on hardware made by Be, PPC only.\n\techo powerpc-be-beos\n\texit ;;\n    BeMac:BeOS:*:*)\t# BeOS running on Mac or Mac clone, PPC only.\n\techo powerpc-apple-beos\n\texit ;;\n    BePC:BeOS:*:*)\t# BeOS running on Intel PC compatible.\n\techo i586-pc-beos\n\texit ;;\n    BePC:Haiku:*:*)\t# Haiku running on Intel PC compatible.\n\techo i586-pc-haiku\n\texit ;;\n    x86_64:Haiku:*:*)\n\techo x86_64-unknown-haiku\n\texit ;;\n    SX-4:SUPER-UX:*:*)\n\techo sx4-nec-superux${UNAME_RELEASE}\n\texit ;;\n    SX-5:SUPER-UX:*:*)\n\techo sx5-nec-superux${UNAME_RELEASE}\n\texit ;;\n    SX-6:SUPER-UX:*:*)\n\techo sx6-nec-superux${UNAME_RELEASE}\n\texit ;;\n    SX-7:SUPER-UX:*:*)\n\techo sx7-nec-superux${UNAME_RELEASE}\n\texit ;;\n    SX-8:SUPER-UX:*:*)\n\techo sx8-nec-superux${UNAME_RELEASE}\n\texit ;;\n    SX-8R:SUPER-UX:*:*)\n\techo sx8r-nec-superux${UNAME_RELEASE}\n\texit ;;\n    SX-ACE:SUPER-UX:*:*)\n\techo sxace-nec-superux${UNAME_RELEASE}\n\texit ;;\n    Power*:Rhapsody:*:*)\n\techo powerpc-apple-rhapsody${UNAME_RELEASE}\n\texit ;;\n    *:Rhapsody:*:*)\n\techo ${UNAME_MACHINE}-apple-rhapsody${UNAME_RELEASE}\n\texit ;;\n    *:Darwin:*:*)\n\tUNAME_PROCESSOR=`uname -p` || UNAME_PROCESSOR=unknown\n\teval $set_cc_for_build\n\tif test \"$UNAME_PROCESSOR\" = unknown ; then\n\t    UNAME_PROCESSOR=powerpc\n\tfi\n\tif test `echo \"$UNAME_RELEASE\" | sed -e 's/\\..*//'` -le 10 ; then\n\t    if [ \"$CC_FOR_BUILD\" != no_compiler_found ]; then\n\t\tif (echo '#ifdef __LP64__'; echo IS_64BIT_ARCH; echo '#endif') | \\\n\t\t       (CCOPTS=\"\" $CC_FOR_BUILD -E - 2>/dev/null) | \\\n\t\t       grep IS_64BIT_ARCH >/dev/null\n\t\tthen\n\t\t    case $UNAME_PROCESSOR in\n\t\t\ti386) UNAME_PROCESSOR=x86_64 ;;\n\t\t\tpowerpc) UNAME_PROCESSOR=powerpc64 ;;\n\t\t    esac\n\t\tfi\n\t\t# On 10.4-10.6 one might compile for PowerPC via gcc -arch ppc\n\t\tif (echo '#ifdef __POWERPC__'; echo IS_PPC; echo '#endif') | \\\n\t\t       (CCOPTS=\"\" $CC_FOR_BUILD -E - 2>/dev/null) | \\\n\t\t       grep IS_PPC >/dev/null\n\t\tthen\n\t\t    UNAME_PROCESSOR=powerpc\n\t\tfi\n\t    fi\n\telif test \"$UNAME_PROCESSOR\" = i386 ; then\n\t    # Avoid executing cc on OS X 10.9, as it ships with a stub\n\t    # that puts up a graphical alert prompting to install\n\t    # developer tools.  Any system running Mac OS X 10.7 or\n\t    # later (Darwin 11 and later) is required to have a 64-bit\n\t    # processor. This is not true of the ARM version of Darwin\n\t    # that Apple uses in portable devices.\n\t    UNAME_PROCESSOR=x86_64\n\tfi\n\techo ${UNAME_PROCESSOR}-apple-darwin${UNAME_RELEASE}\n\texit ;;\n    *:procnto*:*:* | *:QNX:[0123456789]*:*)\n\tUNAME_PROCESSOR=`uname -p`\n\tif test \"$UNAME_PROCESSOR\" = x86; then\n\t\tUNAME_PROCESSOR=i386\n\t\tUNAME_MACHINE=pc\n\tfi\n\techo ${UNAME_PROCESSOR}-${UNAME_MACHINE}-nto-qnx${UNAME_RELEASE}\n\texit ;;\n    *:QNX:*:4*)\n\techo i386-pc-qnx\n\texit ;;\n    NEO-*:NONSTOP_KERNEL:*:*)\n\techo neo-tandem-nsk${UNAME_RELEASE}\n\texit ;;\n    NSE-*:NONSTOP_KERNEL:*:*)\n\techo nse-tandem-nsk${UNAME_RELEASE}\n\texit ;;\n    NSR-*:NONSTOP_KERNEL:*:*)\n\techo nsr-tandem-nsk${UNAME_RELEASE}\n\texit ;;\n    NSX-*:NONSTOP_KERNEL:*:*)\n\techo nsx-tandem-nsk${UNAME_RELEASE}\n\texit ;;\n    *:NonStop-UX:*:*)\n\techo mips-compaq-nonstopux\n\texit ;;\n    BS2000:POSIX*:*:*)\n\techo bs2000-siemens-sysv\n\texit ;;\n    DS/*:UNIX_System_V:*:*)\n\techo ${UNAME_MACHINE}-${UNAME_SYSTEM}-${UNAME_RELEASE}\n\texit ;;\n    *:Plan9:*:*)\n\t# \"uname -m\" is not consistent, so use $cputype instead. 386\n\t# is converted to i386 for consistency with other x86\n\t# operating systems.\n\tif test \"$cputype\" = 386; then\n\t    UNAME_MACHINE=i386\n\telse\n\t    UNAME_MACHINE=\"$cputype\"\n\tfi\n\techo ${UNAME_MACHINE}-unknown-plan9\n\texit ;;\n    *:TOPS-10:*:*)\n\techo pdp10-unknown-tops10\n\texit ;;\n    *:TENEX:*:*)\n\techo pdp10-unknown-tenex\n\texit ;;\n    KS10:TOPS-20:*:* | KL10:TOPS-20:*:* | TYPE4:TOPS-20:*:*)\n\techo pdp10-dec-tops20\n\texit ;;\n    XKL-1:TOPS-20:*:* | TYPE5:TOPS-20:*:*)\n\techo pdp10-xkl-tops20\n\texit ;;\n    *:TOPS-20:*:*)\n\techo pdp10-unknown-tops20\n\texit ;;\n    *:ITS:*:*)\n\techo pdp10-unknown-its\n\texit ;;\n    SEI:*:*:SEIUX)\n\techo mips-sei-seiux${UNAME_RELEASE}\n\texit ;;\n    *:DragonFly:*:*)\n\techo ${UNAME_MACHINE}-unknown-dragonfly`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`\n\texit ;;\n    *:*VMS:*:*)\n\tUNAME_MACHINE=`(uname -p) 2>/dev/null`\n\tcase \"${UNAME_MACHINE}\" in\n\t    A*) echo alpha-dec-vms ; exit ;;\n\t    I*) echo ia64-dec-vms ; exit ;;\n\t    V*) echo vax-dec-vms ; exit ;;\n\tesac ;;\n    *:XENIX:*:SysV)\n\techo i386-pc-xenix\n\texit ;;\n    i*86:skyos:*:*)\n\techo ${UNAME_MACHINE}-pc-skyos`echo ${UNAME_RELEASE} | sed -e 's/ .*$//'`\n\texit ;;\n    i*86:rdos:*:*)\n\techo ${UNAME_MACHINE}-pc-rdos\n\texit ;;\n    i*86:AROS:*:*)\n\techo ${UNAME_MACHINE}-pc-aros\n\texit ;;\n    x86_64:VMkernel:*:*)\n\techo ${UNAME_MACHINE}-unknown-esx\n\texit ;;\n    amd64:Isilon\\ OneFS:*:*)\n\techo x86_64-unknown-onefs\n\texit ;;\nesac\n\ncat >&2 <<EOF\n$0: unable to guess system type\n\nThis script (version $timestamp), has failed to recognize the\noperating system you are using. If your script is old, overwrite\nconfig.guess and config.sub with the latest versions from:\n\n  http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess\nand\n  http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub\n\nIf $0 has already been updated, send the following data and any\ninformation you think might be pertinent to config-patches@gnu.org to\nprovide the necessary information to handle your system.\n\nconfig.guess timestamp = $timestamp\n\nuname -m = `(uname -m) 2>/dev/null || echo unknown`\nuname -r = `(uname -r) 2>/dev/null || echo unknown`\nuname -s = `(uname -s) 2>/dev/null || echo unknown`\nuname -v = `(uname -v) 2>/dev/null || echo unknown`\n\n/usr/bin/uname -p = `(/usr/bin/uname -p) 2>/dev/null`\n/bin/uname -X     = `(/bin/uname -X) 2>/dev/null`\n\nhostinfo               = `(hostinfo) 2>/dev/null`\n/bin/universe          = `(/bin/universe) 2>/dev/null`\n/usr/bin/arch -k       = `(/usr/bin/arch -k) 2>/dev/null`\n/bin/arch              = `(/bin/arch) 2>/dev/null`\n/usr/bin/oslevel       = `(/usr/bin/oslevel) 2>/dev/null`\n/usr/convex/getsysinfo = `(/usr/convex/getsysinfo) 2>/dev/null`\n\nUNAME_MACHINE = ${UNAME_MACHINE}\nUNAME_RELEASE = ${UNAME_RELEASE}\nUNAME_SYSTEM  = ${UNAME_SYSTEM}\nUNAME_VERSION = ${UNAME_VERSION}\nEOF\n\nexit 1\n\n# Local variables:\n# eval: (add-hook 'write-file-hooks 'time-stamp)\n# time-stamp-start: \"timestamp='\"\n# time-stamp-format: \"%:y-%02m-%02d\"\n# time-stamp-end: \"'\"\n# End:\n"
  },
  {
    "path": "build-aux/config.sub",
    "content": "#! /bin/sh\n# Configuration validation subroutine script.\n#   Copyright 1992-2017 Free Software Foundation, Inc.\n\ntimestamp='2017-04-02'\n\n# This file is free software; you can redistribute it and/or modify it\n# under the terms of the GNU General Public License as published by\n# the Free Software Foundation; either version 3 of the License, or\n# (at your option) any later version.\n#\n# This program is distributed in the hope that it will be useful, but\n# WITHOUT ANY WARRANTY; without even the implied warranty of\n# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU\n# General Public License for more details.\n#\n# You should have received a copy of the GNU General Public License\n# along with this program; if not, see <http://www.gnu.org/licenses/>.\n#\n# As a special exception to the GNU General Public License, if you\n# distribute this file as part of a program that contains a\n# configuration script generated by Autoconf, you may include it under\n# the same distribution terms that you use for the rest of that\n# program.  This Exception is an additional permission under section 7\n# of the GNU General Public License, version 3 (\"GPLv3\").\n\n\n# Please send patches to <config-patches@gnu.org>.\n#\n# Configuration subroutine to validate and canonicalize a configuration type.\n# Supply the specified configuration type as an argument.\n# If it is invalid, we print an error message on stderr and exit with code 1.\n# Otherwise, we print the canonical config type on stdout and succeed.\n\n# You can get the latest version of this script from:\n# http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub\n\n# This file is supposed to be the same for all GNU packages\n# and recognize all the CPU types, system types and aliases\n# that are meaningful with *any* GNU software.\n# Each package is responsible for reporting which valid configurations\n# it does not support.  The user should be able to distinguish\n# a failure to support a valid configuration from a meaningless\n# configuration.\n\n# The goal of this file is to map all the various variations of a given\n# machine specification into a single specification in the form:\n#\tCPU_TYPE-MANUFACTURER-OPERATING_SYSTEM\n# or in some cases, the newer four-part form:\n#\tCPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM\n# It is wrong to echo any other type of specification.\n\nme=`echo \"$0\" | sed -e 's,.*/,,'`\n\nusage=\"\\\nUsage: $0 [OPTION] CPU-MFR-OPSYS or ALIAS\n\nCanonicalize a configuration name.\n\nOperation modes:\n  -h, --help         print this help, then exit\n  -t, --time-stamp   print date of last modification, then exit\n  -v, --version      print version number, then exit\n\nReport bugs and patches to <config-patches@gnu.org>.\"\n\nversion=\"\\\nGNU config.sub ($timestamp)\n\nCopyright 1992-2017 Free Software Foundation, Inc.\n\nThis is free software; see the source for copying conditions.  There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\"\n\nhelp=\"\nTry \\`$me --help' for more information.\"\n\n# Parse command line\nwhile test $# -gt 0 ; do\n  case $1 in\n    --time-stamp | --time* | -t )\n       echo \"$timestamp\" ; exit ;;\n    --version | -v )\n       echo \"$version\" ; exit ;;\n    --help | --h* | -h )\n       echo \"$usage\"; exit ;;\n    -- )     # Stop option processing\n       shift; break ;;\n    - )\t# Use stdin as input.\n       break ;;\n    -* )\n       echo \"$me: invalid option $1$help\"\n       exit 1 ;;\n\n    *local*)\n       # First pass through any local machine types.\n       echo $1\n       exit ;;\n\n    * )\n       break ;;\n  esac\ndone\n\ncase $# in\n 0) echo \"$me: missing argument$help\" >&2\n    exit 1;;\n 1) ;;\n *) echo \"$me: too many arguments$help\" >&2\n    exit 1;;\nesac\n\n# Separate what the user gave into CPU-COMPANY and OS or KERNEL-OS (if any).\n# Here we must recognize all the valid KERNEL-OS combinations.\nmaybe_os=`echo $1 | sed 's/^\\(.*\\)-\\([^-]*-[^-]*\\)$/\\2/'`\ncase $maybe_os in\n  nto-qnx* | linux-gnu* | linux-android* | linux-dietlibc | linux-newlib* | \\\n  linux-musl* | linux-uclibc* | uclinux-uclibc* | uclinux-gnu* | kfreebsd*-gnu* | \\\n  knetbsd*-gnu* | netbsd*-gnu* | netbsd*-eabi* | \\\n  kopensolaris*-gnu* | cloudabi*-eabi* | \\\n  storm-chaos* | os2-emx* | rtmk-nova*)\n    os=-$maybe_os\n    basic_machine=`echo $1 | sed 's/^\\(.*\\)-\\([^-]*-[^-]*\\)$/\\1/'`\n    ;;\n  android-linux)\n    os=-linux-android\n    basic_machine=`echo $1 | sed 's/^\\(.*\\)-\\([^-]*-[^-]*\\)$/\\1/'`-unknown\n    ;;\n  *)\n    basic_machine=`echo $1 | sed 's/-[^-]*$//'`\n    if [ $basic_machine != $1 ]\n    then os=`echo $1 | sed 's/.*-/-/'`\n    else os=; fi\n    ;;\nesac\n\n### Let's recognize common machines as not being operating systems so\n### that things like config.sub decstation-3100 work.  We also\n### recognize some manufacturers as not being operating systems, so we\n### can provide default operating systems below.\ncase $os in\n\t-sun*os*)\n\t\t# Prevent following clause from handling this invalid input.\n\t\t;;\n\t-dec* | -mips* | -sequent* | -encore* | -pc532* | -sgi* | -sony* | \\\n\t-att* | -7300* | -3300* | -delta* | -motorola* | -sun[234]* | \\\n\t-unicom* | -ibm* | -next | -hp | -isi* | -apollo | -altos* | \\\n\t-convergent* | -ncr* | -news | -32* | -3600* | -3100* | -hitachi* |\\\n\t-c[123]* | -convex* | -sun | -crds | -omron* | -dg | -ultra | -tti* | \\\n\t-harris | -dolphin | -highlevel | -gould | -cbm | -ns | -masscomp | \\\n\t-apple | -axis | -knuth | -cray | -microblaze*)\n\t\tos=\n\t\tbasic_machine=$1\n\t\t;;\n\t-bluegene*)\n\t\tos=-cnk\n\t\t;;\n\t-sim | -cisco | -oki | -wec | -winbond)\n\t\tos=\n\t\tbasic_machine=$1\n\t\t;;\n\t-scout)\n\t\t;;\n\t-wrs)\n\t\tos=-vxworks\n\t\tbasic_machine=$1\n\t\t;;\n\t-chorusos*)\n\t\tos=-chorusos\n\t\tbasic_machine=$1\n\t\t;;\n\t-chorusrdb)\n\t\tos=-chorusrdb\n\t\tbasic_machine=$1\n\t\t;;\n\t-hiux*)\n\t\tos=-hiuxwe2\n\t\t;;\n\t-sco6)\n\t\tos=-sco5v6\n\t\tbasic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`\n\t\t;;\n\t-sco5)\n\t\tos=-sco3.2v5\n\t\tbasic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`\n\t\t;;\n\t-sco4)\n\t\tos=-sco3.2v4\n\t\tbasic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`\n\t\t;;\n\t-sco3.2.[4-9]*)\n\t\tos=`echo $os | sed -e 's/sco3.2./sco3.2v/'`\n\t\tbasic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`\n\t\t;;\n\t-sco3.2v[4-9]*)\n\t\t# Don't forget version if it is 3.2v4 or newer.\n\t\tbasic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`\n\t\t;;\n\t-sco5v6*)\n\t\t# Don't forget version if it is 3.2v4 or newer.\n\t\tbasic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`\n\t\t;;\n\t-sco*)\n\t\tos=-sco3.2v2\n\t\tbasic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`\n\t\t;;\n\t-udk*)\n\t\tbasic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`\n\t\t;;\n\t-isc)\n\t\tos=-isc2.2\n\t\tbasic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`\n\t\t;;\n\t-clix*)\n\t\tbasic_machine=clipper-intergraph\n\t\t;;\n\t-isc*)\n\t\tbasic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'`\n\t\t;;\n\t-lynx*178)\n\t\tos=-lynxos178\n\t\t;;\n\t-lynx*5)\n\t\tos=-lynxos5\n\t\t;;\n\t-lynx*)\n\t\tos=-lynxos\n\t\t;;\n\t-ptx*)\n\t\tbasic_machine=`echo $1 | sed -e 's/86-.*/86-sequent/'`\n\t\t;;\n\t-windowsnt*)\n\t\tos=`echo $os | sed -e 's/windowsnt/winnt/'`\n\t\t;;\n\t-psos*)\n\t\tos=-psos\n\t\t;;\n\t-mint | -mint[0-9]*)\n\t\tbasic_machine=m68k-atari\n\t\tos=-mint\n\t\t;;\nesac\n\n# Decode aliases for certain CPU-COMPANY combinations.\ncase $basic_machine in\n\t# Recognize the basic CPU types without company name.\n\t# Some are omitted here because they have special meanings below.\n\t1750a | 580 \\\n\t| a29k \\\n\t| aarch64 | aarch64_be \\\n\t| alpha | alphaev[4-8] | alphaev56 | alphaev6[78] | alphapca5[67] \\\n\t| alpha64 | alpha64ev[4-8] | alpha64ev56 | alpha64ev6[78] | alpha64pca5[67] \\\n\t| am33_2.0 \\\n\t| arc | arceb \\\n\t| arm | arm[bl]e | arme[lb] | armv[2-8] | armv[3-8][lb] | armv7[arm] \\\n\t| avr | avr32 \\\n\t| ba \\\n\t| be32 | be64 \\\n\t| bfin \\\n\t| c4x | c8051 | clipper \\\n\t| d10v | d30v | dlx | dsp16xx \\\n\t| e2k | epiphany \\\n\t| fido | fr30 | frv | ft32 \\\n\t| h8300 | h8500 | hppa | hppa1.[01] | hppa2.0 | hppa2.0[nw] | hppa64 \\\n\t| hexagon \\\n\t| i370 | i860 | i960 | ia16 | ia64 \\\n\t| ip2k | iq2000 \\\n\t| k1om \\\n\t| le32 | le64 \\\n\t| lm32 \\\n\t| m32c | m32r | m32rle | m68000 | m68k | m88k \\\n\t| maxq | mb | microblaze | microblazeel | mcore | mep | metag \\\n\t| mips | mipsbe | mipseb | mipsel | mipsle \\\n\t| mips16 \\\n\t| mips64 | mips64el \\\n\t| mips64octeon | mips64octeonel \\\n\t| mips64orion | mips64orionel \\\n\t| mips64r5900 | mips64r5900el \\\n\t| mips64vr | mips64vrel \\\n\t| mips64vr4100 | mips64vr4100el \\\n\t| mips64vr4300 | mips64vr4300el \\\n\t| mips64vr5000 | mips64vr5000el \\\n\t| mips64vr5900 | mips64vr5900el \\\n\t| mipsisa32 | mipsisa32el \\\n\t| mipsisa32r2 | mipsisa32r2el \\\n\t| mipsisa32r6 | mipsisa32r6el \\\n\t| mipsisa64 | mipsisa64el \\\n\t| mipsisa64r2 | mipsisa64r2el \\\n\t| mipsisa64r6 | mipsisa64r6el \\\n\t| mipsisa64sb1 | mipsisa64sb1el \\\n\t| mipsisa64sr71k | mipsisa64sr71kel \\\n\t| mipsr5900 | mipsr5900el \\\n\t| mipstx39 | mipstx39el \\\n\t| mn10200 | mn10300 \\\n\t| moxie \\\n\t| mt \\\n\t| msp430 \\\n\t| nds32 | nds32le | nds32be \\\n\t| nios | nios2 | nios2eb | nios2el \\\n\t| ns16k | ns32k \\\n\t| open8 | or1k | or1knd | or32 \\\n\t| pdp10 | pdp11 | pj | pjl \\\n\t| powerpc | powerpc64 | powerpc64le | powerpcle \\\n\t| pru \\\n\t| pyramid \\\n\t| riscv32 | riscv64 \\\n\t| rl78 | rx \\\n\t| score \\\n\t| sh | sh[1234] | sh[24]a | sh[24]aeb | sh[23]e | sh[234]eb | sheb | shbe | shle | sh[1234]le | sh3ele \\\n\t| sh64 | sh64le \\\n\t| sparc | sparc64 | sparc64b | sparc64v | sparc86x | sparclet | sparclite \\\n\t| sparcv8 | sparcv9 | sparcv9b | sparcv9v \\\n\t| spu \\\n\t| tahoe | tic4x | tic54x | tic55x | tic6x | tic80 | tron \\\n\t| ubicom32 \\\n\t| v850 | v850e | v850e1 | v850e2 | v850es | v850e2v3 \\\n\t| visium \\\n\t| wasm32 \\\n\t| we32k \\\n\t| x86 | xc16x | xstormy16 | xtensa \\\n\t| z8k | z80)\n\t\tbasic_machine=$basic_machine-unknown\n\t\t;;\n\tc54x)\n\t\tbasic_machine=tic54x-unknown\n\t\t;;\n\tc55x)\n\t\tbasic_machine=tic55x-unknown\n\t\t;;\n\tc6x)\n\t\tbasic_machine=tic6x-unknown\n\t\t;;\n\tleon|leon[3-9])\n\t\tbasic_machine=sparc-$basic_machine\n\t\t;;\n\tm6811 | m68hc11 | m6812 | m68hc12 | m68hcs12x | nvptx | picochip)\n\t\tbasic_machine=$basic_machine-unknown\n\t\tos=-none\n\t\t;;\n\tm88110 | m680[12346]0 | m683?2 | m68360 | m5200 | v70 | w65 | z8k)\n\t\t;;\n\tms1)\n\t\tbasic_machine=mt-unknown\n\t\t;;\n\n\tstrongarm | thumb | xscale)\n\t\tbasic_machine=arm-unknown\n\t\t;;\n\txgate)\n\t\tbasic_machine=$basic_machine-unknown\n\t\tos=-none\n\t\t;;\n\txscaleeb)\n\t\tbasic_machine=armeb-unknown\n\t\t;;\n\n\txscaleel)\n\t\tbasic_machine=armel-unknown\n\t\t;;\n\n\t# We use `pc' rather than `unknown'\n\t# because (1) that's what they normally are, and\n\t# (2) the word \"unknown\" tends to confuse beginning users.\n\ti*86 | x86_64)\n\t  basic_machine=$basic_machine-pc\n\t  ;;\n\t# Object if more than one company name word.\n\t*-*-*)\n\t\techo Invalid configuration \\`$1\\': machine \\`$basic_machine\\' not recognized 1>&2\n\t\texit 1\n\t\t;;\n\t# Recognize the basic CPU types with company name.\n\t580-* \\\n\t| a29k-* \\\n\t| aarch64-* | aarch64_be-* \\\n\t| alpha-* | alphaev[4-8]-* | alphaev56-* | alphaev6[78]-* \\\n\t| alpha64-* | alpha64ev[4-8]-* | alpha64ev56-* | alpha64ev6[78]-* \\\n\t| alphapca5[67]-* | alpha64pca5[67]-* | arc-* | arceb-* \\\n\t| arm-*  | armbe-* | armle-* | armeb-* | armv*-* \\\n\t| avr-* | avr32-* \\\n\t| ba-* \\\n\t| be32-* | be64-* \\\n\t| bfin-* | bs2000-* \\\n\t| c[123]* | c30-* | [cjt]90-* | c4x-* \\\n\t| c8051-* | clipper-* | craynv-* | cydra-* \\\n\t| d10v-* | d30v-* | dlx-* \\\n\t| e2k-* | elxsi-* \\\n\t| f30[01]-* | f700-* | fido-* | fr30-* | frv-* | fx80-* \\\n\t| h8300-* | h8500-* \\\n\t| hppa-* | hppa1.[01]-* | hppa2.0-* | hppa2.0[nw]-* | hppa64-* \\\n\t| hexagon-* \\\n\t| i*86-* | i860-* | i960-* | ia16-* | ia64-* \\\n\t| ip2k-* | iq2000-* \\\n\t| k1om-* \\\n\t| le32-* | le64-* \\\n\t| lm32-* \\\n\t| m32c-* | m32r-* | m32rle-* \\\n\t| m68000-* | m680[012346]0-* | m68360-* | m683?2-* | m68k-* \\\n\t| m88110-* | m88k-* | maxq-* | mcore-* | metag-* \\\n\t| microblaze-* | microblazeel-* \\\n\t| mips-* | mipsbe-* | mipseb-* | mipsel-* | mipsle-* \\\n\t| mips16-* \\\n\t| mips64-* | mips64el-* \\\n\t| mips64octeon-* | mips64octeonel-* \\\n\t| mips64orion-* | mips64orionel-* \\\n\t| mips64r5900-* | mips64r5900el-* \\\n\t| mips64vr-* | mips64vrel-* \\\n\t| mips64vr4100-* | mips64vr4100el-* \\\n\t| mips64vr4300-* | mips64vr4300el-* \\\n\t| mips64vr5000-* | mips64vr5000el-* \\\n\t| mips64vr5900-* | mips64vr5900el-* \\\n\t| mipsisa32-* | mipsisa32el-* \\\n\t| mipsisa32r2-* | mipsisa32r2el-* \\\n\t| mipsisa32r6-* | mipsisa32r6el-* \\\n\t| mipsisa64-* | mipsisa64el-* \\\n\t| mipsisa64r2-* | mipsisa64r2el-* \\\n\t| mipsisa64r6-* | mipsisa64r6el-* \\\n\t| mipsisa64sb1-* | mipsisa64sb1el-* \\\n\t| mipsisa64sr71k-* | mipsisa64sr71kel-* \\\n\t| mipsr5900-* | mipsr5900el-* \\\n\t| mipstx39-* | mipstx39el-* \\\n\t| mmix-* \\\n\t| mt-* \\\n\t| msp430-* \\\n\t| nds32-* | nds32le-* | nds32be-* \\\n\t| nios-* | nios2-* | nios2eb-* | nios2el-* \\\n\t| none-* | np1-* | ns16k-* | ns32k-* \\\n\t| open8-* \\\n\t| or1k*-* \\\n\t| orion-* \\\n\t| pdp10-* | pdp11-* | pj-* | pjl-* | pn-* | power-* \\\n\t| powerpc-* | powerpc64-* | powerpc64le-* | powerpcle-* \\\n\t| pru-* \\\n\t| pyramid-* \\\n\t| riscv32-* | riscv64-* \\\n\t| rl78-* | romp-* | rs6000-* | rx-* \\\n\t| sh-* | sh[1234]-* | sh[24]a-* | sh[24]aeb-* | sh[23]e-* | sh[34]eb-* | sheb-* | shbe-* \\\n\t| shle-* | sh[1234]le-* | sh3ele-* | sh64-* | sh64le-* \\\n\t| sparc-* | sparc64-* | sparc64b-* | sparc64v-* | sparc86x-* | sparclet-* \\\n\t| sparclite-* \\\n\t| sparcv8-* | sparcv9-* | sparcv9b-* | sparcv9v-* | sv1-* | sx*-* \\\n\t| tahoe-* \\\n\t| tic30-* | tic4x-* | tic54x-* | tic55x-* | tic6x-* | tic80-* \\\n\t| tile*-* \\\n\t| tron-* \\\n\t| ubicom32-* \\\n\t| v850-* | v850e-* | v850e1-* | v850es-* | v850e2-* | v850e2v3-* \\\n\t| vax-* \\\n\t| visium-* \\\n\t| wasm32-* \\\n\t| we32k-* \\\n\t| x86-* | x86_64-* | xc16x-* | xps100-* \\\n\t| xstormy16-* | xtensa*-* \\\n\t| ymp-* \\\n\t| z8k-* | z80-*)\n\t\t;;\n\t# Recognize the basic CPU types without company name, with glob match.\n\txtensa*)\n\t\tbasic_machine=$basic_machine-unknown\n\t\t;;\n\t# Recognize the various machine names and aliases which stand\n\t# for a CPU type and a company and sometimes even an OS.\n\t386bsd)\n\t\tbasic_machine=i386-unknown\n\t\tos=-bsd\n\t\t;;\n\t3b1 | 7300 | 7300-att | att-7300 | pc7300 | safari | unixpc)\n\t\tbasic_machine=m68000-att\n\t\t;;\n\t3b*)\n\t\tbasic_machine=we32k-att\n\t\t;;\n\ta29khif)\n\t\tbasic_machine=a29k-amd\n\t\tos=-udi\n\t\t;;\n\tabacus)\n\t\tbasic_machine=abacus-unknown\n\t\t;;\n\tadobe68k)\n\t\tbasic_machine=m68010-adobe\n\t\tos=-scout\n\t\t;;\n\talliant | fx80)\n\t\tbasic_machine=fx80-alliant\n\t\t;;\n\taltos | altos3068)\n\t\tbasic_machine=m68k-altos\n\t\t;;\n\tam29k)\n\t\tbasic_machine=a29k-none\n\t\tos=-bsd\n\t\t;;\n\tamd64)\n\t\tbasic_machine=x86_64-pc\n\t\t;;\n\tamd64-*)\n\t\tbasic_machine=x86_64-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tamdahl)\n\t\tbasic_machine=580-amdahl\n\t\tos=-sysv\n\t\t;;\n\tamiga | amiga-*)\n\t\tbasic_machine=m68k-unknown\n\t\t;;\n\tamigaos | amigados)\n\t\tbasic_machine=m68k-unknown\n\t\tos=-amigaos\n\t\t;;\n\tamigaunix | amix)\n\t\tbasic_machine=m68k-unknown\n\t\tos=-sysv4\n\t\t;;\n\tapollo68)\n\t\tbasic_machine=m68k-apollo\n\t\tos=-sysv\n\t\t;;\n\tapollo68bsd)\n\t\tbasic_machine=m68k-apollo\n\t\tos=-bsd\n\t\t;;\n\taros)\n\t\tbasic_machine=i386-pc\n\t\tos=-aros\n\t\t;;\n\tasmjs)\n\t\tbasic_machine=asmjs-unknown\n\t\t;;\n\taux)\n\t\tbasic_machine=m68k-apple\n\t\tos=-aux\n\t\t;;\n\tbalance)\n\t\tbasic_machine=ns32k-sequent\n\t\tos=-dynix\n\t\t;;\n\tblackfin)\n\t\tbasic_machine=bfin-unknown\n\t\tos=-linux\n\t\t;;\n\tblackfin-*)\n\t\tbasic_machine=bfin-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\tos=-linux\n\t\t;;\n\tbluegene*)\n\t\tbasic_machine=powerpc-ibm\n\t\tos=-cnk\n\t\t;;\n\tc54x-*)\n\t\tbasic_machine=tic54x-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tc55x-*)\n\t\tbasic_machine=tic55x-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tc6x-*)\n\t\tbasic_machine=tic6x-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tc90)\n\t\tbasic_machine=c90-cray\n\t\tos=-unicos\n\t\t;;\n\tcegcc)\n\t\tbasic_machine=arm-unknown\n\t\tos=-cegcc\n\t\t;;\n\tconvex-c1)\n\t\tbasic_machine=c1-convex\n\t\tos=-bsd\n\t\t;;\n\tconvex-c2)\n\t\tbasic_machine=c2-convex\n\t\tos=-bsd\n\t\t;;\n\tconvex-c32)\n\t\tbasic_machine=c32-convex\n\t\tos=-bsd\n\t\t;;\n\tconvex-c34)\n\t\tbasic_machine=c34-convex\n\t\tos=-bsd\n\t\t;;\n\tconvex-c38)\n\t\tbasic_machine=c38-convex\n\t\tos=-bsd\n\t\t;;\n\tcray | j90)\n\t\tbasic_machine=j90-cray\n\t\tos=-unicos\n\t\t;;\n\tcraynv)\n\t\tbasic_machine=craynv-cray\n\t\tos=-unicosmp\n\t\t;;\n\tcr16 | cr16-*)\n\t\tbasic_machine=cr16-unknown\n\t\tos=-elf\n\t\t;;\n\tcrds | unos)\n\t\tbasic_machine=m68k-crds\n\t\t;;\n\tcrisv32 | crisv32-* | etraxfs*)\n\t\tbasic_machine=crisv32-axis\n\t\t;;\n\tcris | cris-* | etrax*)\n\t\tbasic_machine=cris-axis\n\t\t;;\n\tcrx)\n\t\tbasic_machine=crx-unknown\n\t\tos=-elf\n\t\t;;\n\tda30 | da30-*)\n\t\tbasic_machine=m68k-da30\n\t\t;;\n\tdecstation | decstation-3100 | pmax | pmax-* | pmin | dec3100 | decstatn)\n\t\tbasic_machine=mips-dec\n\t\t;;\n\tdecsystem10* | dec10*)\n\t\tbasic_machine=pdp10-dec\n\t\tos=-tops10\n\t\t;;\n\tdecsystem20* | dec20*)\n\t\tbasic_machine=pdp10-dec\n\t\tos=-tops20\n\t\t;;\n\tdelta | 3300 | motorola-3300 | motorola-delta \\\n\t      | 3300-motorola | delta-motorola)\n\t\tbasic_machine=m68k-motorola\n\t\t;;\n\tdelta88)\n\t\tbasic_machine=m88k-motorola\n\t\tos=-sysv3\n\t\t;;\n\tdicos)\n\t\tbasic_machine=i686-pc\n\t\tos=-dicos\n\t\t;;\n\tdjgpp)\n\t\tbasic_machine=i586-pc\n\t\tos=-msdosdjgpp\n\t\t;;\n\tdpx20 | dpx20-*)\n\t\tbasic_machine=rs6000-bull\n\t\tos=-bosx\n\t\t;;\n\tdpx2* | dpx2*-bull)\n\t\tbasic_machine=m68k-bull\n\t\tos=-sysv3\n\t\t;;\n\te500v[12])\n\t\tbasic_machine=powerpc-unknown\n\t\tos=$os\"spe\"\n\t\t;;\n\te500v[12]-*)\n\t\tbasic_machine=powerpc-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\tos=$os\"spe\"\n\t\t;;\n\tebmon29k)\n\t\tbasic_machine=a29k-amd\n\t\tos=-ebmon\n\t\t;;\n\telxsi)\n\t\tbasic_machine=elxsi-elxsi\n\t\tos=-bsd\n\t\t;;\n\tencore | umax | mmax)\n\t\tbasic_machine=ns32k-encore\n\t\t;;\n\tes1800 | OSE68k | ose68k | ose | OSE)\n\t\tbasic_machine=m68k-ericsson\n\t\tos=-ose\n\t\t;;\n\tfx2800)\n\t\tbasic_machine=i860-alliant\n\t\t;;\n\tgenix)\n\t\tbasic_machine=ns32k-ns\n\t\t;;\n\tgmicro)\n\t\tbasic_machine=tron-gmicro\n\t\tos=-sysv\n\t\t;;\n\tgo32)\n\t\tbasic_machine=i386-pc\n\t\tos=-go32\n\t\t;;\n\th3050r* | hiux*)\n\t\tbasic_machine=hppa1.1-hitachi\n\t\tos=-hiuxwe2\n\t\t;;\n\th8300hms)\n\t\tbasic_machine=h8300-hitachi\n\t\tos=-hms\n\t\t;;\n\th8300xray)\n\t\tbasic_machine=h8300-hitachi\n\t\tos=-xray\n\t\t;;\n\th8500hms)\n\t\tbasic_machine=h8500-hitachi\n\t\tos=-hms\n\t\t;;\n\tharris)\n\t\tbasic_machine=m88k-harris\n\t\tos=-sysv3\n\t\t;;\n\thp300-*)\n\t\tbasic_machine=m68k-hp\n\t\t;;\n\thp300bsd)\n\t\tbasic_machine=m68k-hp\n\t\tos=-bsd\n\t\t;;\n\thp300hpux)\n\t\tbasic_machine=m68k-hp\n\t\tos=-hpux\n\t\t;;\n\thp3k9[0-9][0-9] | hp9[0-9][0-9])\n\t\tbasic_machine=hppa1.0-hp\n\t\t;;\n\thp9k2[0-9][0-9] | hp9k31[0-9])\n\t\tbasic_machine=m68000-hp\n\t\t;;\n\thp9k3[2-9][0-9])\n\t\tbasic_machine=m68k-hp\n\t\t;;\n\thp9k6[0-9][0-9] | hp6[0-9][0-9])\n\t\tbasic_machine=hppa1.0-hp\n\t\t;;\n\thp9k7[0-79][0-9] | hp7[0-79][0-9])\n\t\tbasic_machine=hppa1.1-hp\n\t\t;;\n\thp9k78[0-9] | hp78[0-9])\n\t\t# FIXME: really hppa2.0-hp\n\t\tbasic_machine=hppa1.1-hp\n\t\t;;\n\thp9k8[67]1 | hp8[67]1 | hp9k80[24] | hp80[24] | hp9k8[78]9 | hp8[78]9 | hp9k893 | hp893)\n\t\t# FIXME: really hppa2.0-hp\n\t\tbasic_machine=hppa1.1-hp\n\t\t;;\n\thp9k8[0-9][13679] | hp8[0-9][13679])\n\t\tbasic_machine=hppa1.1-hp\n\t\t;;\n\thp9k8[0-9][0-9] | hp8[0-9][0-9])\n\t\tbasic_machine=hppa1.0-hp\n\t\t;;\n\thppa-next)\n\t\tos=-nextstep3\n\t\t;;\n\thppaosf)\n\t\tbasic_machine=hppa1.1-hp\n\t\tos=-osf\n\t\t;;\n\thppro)\n\t\tbasic_machine=hppa1.1-hp\n\t\tos=-proelf\n\t\t;;\n\ti370-ibm* | ibm*)\n\t\tbasic_machine=i370-ibm\n\t\t;;\n\ti*86v32)\n\t\tbasic_machine=`echo $1 | sed -e 's/86.*/86-pc/'`\n\t\tos=-sysv32\n\t\t;;\n\ti*86v4*)\n\t\tbasic_machine=`echo $1 | sed -e 's/86.*/86-pc/'`\n\t\tos=-sysv4\n\t\t;;\n\ti*86v)\n\t\tbasic_machine=`echo $1 | sed -e 's/86.*/86-pc/'`\n\t\tos=-sysv\n\t\t;;\n\ti*86sol2)\n\t\tbasic_machine=`echo $1 | sed -e 's/86.*/86-pc/'`\n\t\tos=-solaris2\n\t\t;;\n\ti386mach)\n\t\tbasic_machine=i386-mach\n\t\tos=-mach\n\t\t;;\n\ti386-vsta | vsta)\n\t\tbasic_machine=i386-unknown\n\t\tos=-vsta\n\t\t;;\n\tiris | iris4d)\n\t\tbasic_machine=mips-sgi\n\t\tcase $os in\n\t\t    -irix*)\n\t\t\t;;\n\t\t    *)\n\t\t\tos=-irix4\n\t\t\t;;\n\t\tesac\n\t\t;;\n\tisi68 | isi)\n\t\tbasic_machine=m68k-isi\n\t\tos=-sysv\n\t\t;;\n\tleon-*|leon[3-9]-*)\n\t\tbasic_machine=sparc-`echo $basic_machine | sed 's/-.*//'`\n\t\t;;\n\tm68knommu)\n\t\tbasic_machine=m68k-unknown\n\t\tos=-linux\n\t\t;;\n\tm68knommu-*)\n\t\tbasic_machine=m68k-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\tos=-linux\n\t\t;;\n\tm88k-omron*)\n\t\tbasic_machine=m88k-omron\n\t\t;;\n\tmagnum | m3230)\n\t\tbasic_machine=mips-mips\n\t\tos=-sysv\n\t\t;;\n\tmerlin)\n\t\tbasic_machine=ns32k-utek\n\t\tos=-sysv\n\t\t;;\n\tmicroblaze*)\n\t\tbasic_machine=microblaze-xilinx\n\t\t;;\n\tmingw64)\n\t\tbasic_machine=x86_64-pc\n\t\tos=-mingw64\n\t\t;;\n\tmingw32)\n\t\tbasic_machine=i686-pc\n\t\tos=-mingw32\n\t\t;;\n\tmingw32ce)\n\t\tbasic_machine=arm-unknown\n\t\tos=-mingw32ce\n\t\t;;\n\tminiframe)\n\t\tbasic_machine=m68000-convergent\n\t\t;;\n\t*mint | -mint[0-9]* | *MiNT | *MiNT[0-9]*)\n\t\tbasic_machine=m68k-atari\n\t\tos=-mint\n\t\t;;\n\tmips3*-*)\n\t\tbasic_machine=`echo $basic_machine | sed -e 's/mips3/mips64/'`\n\t\t;;\n\tmips3*)\n\t\tbasic_machine=`echo $basic_machine | sed -e 's/mips3/mips64/'`-unknown\n\t\t;;\n\tmonitor)\n\t\tbasic_machine=m68k-rom68k\n\t\tos=-coff\n\t\t;;\n\tmorphos)\n\t\tbasic_machine=powerpc-unknown\n\t\tos=-morphos\n\t\t;;\n\tmoxiebox)\n\t\tbasic_machine=moxie-unknown\n\t\tos=-moxiebox\n\t\t;;\n\tmsdos)\n\t\tbasic_machine=i386-pc\n\t\tos=-msdos\n\t\t;;\n\tms1-*)\n\t\tbasic_machine=`echo $basic_machine | sed -e 's/ms1-/mt-/'`\n\t\t;;\n\tmsys)\n\t\tbasic_machine=i686-pc\n\t\tos=-msys\n\t\t;;\n\tmvs)\n\t\tbasic_machine=i370-ibm\n\t\tos=-mvs\n\t\t;;\n\tnacl)\n\t\tbasic_machine=le32-unknown\n\t\tos=-nacl\n\t\t;;\n\tncr3000)\n\t\tbasic_machine=i486-ncr\n\t\tos=-sysv4\n\t\t;;\n\tnetbsd386)\n\t\tbasic_machine=i386-unknown\n\t\tos=-netbsd\n\t\t;;\n\tnetwinder)\n\t\tbasic_machine=armv4l-rebel\n\t\tos=-linux\n\t\t;;\n\tnews | news700 | news800 | news900)\n\t\tbasic_machine=m68k-sony\n\t\tos=-newsos\n\t\t;;\n\tnews1000)\n\t\tbasic_machine=m68030-sony\n\t\tos=-newsos\n\t\t;;\n\tnews-3600 | risc-news)\n\t\tbasic_machine=mips-sony\n\t\tos=-newsos\n\t\t;;\n\tnecv70)\n\t\tbasic_machine=v70-nec\n\t\tos=-sysv\n\t\t;;\n\tnext | m*-next )\n\t\tbasic_machine=m68k-next\n\t\tcase $os in\n\t\t    -nextstep* )\n\t\t\t;;\n\t\t    -ns2*)\n\t\t      os=-nextstep2\n\t\t\t;;\n\t\t    *)\n\t\t      os=-nextstep3\n\t\t\t;;\n\t\tesac\n\t\t;;\n\tnh3000)\n\t\tbasic_machine=m68k-harris\n\t\tos=-cxux\n\t\t;;\n\tnh[45]000)\n\t\tbasic_machine=m88k-harris\n\t\tos=-cxux\n\t\t;;\n\tnindy960)\n\t\tbasic_machine=i960-intel\n\t\tos=-nindy\n\t\t;;\n\tmon960)\n\t\tbasic_machine=i960-intel\n\t\tos=-mon960\n\t\t;;\n\tnonstopux)\n\t\tbasic_machine=mips-compaq\n\t\tos=-nonstopux\n\t\t;;\n\tnp1)\n\t\tbasic_machine=np1-gould\n\t\t;;\n\tneo-tandem)\n\t\tbasic_machine=neo-tandem\n\t\t;;\n\tnse-tandem)\n\t\tbasic_machine=nse-tandem\n\t\t;;\n\tnsr-tandem)\n\t\tbasic_machine=nsr-tandem\n\t\t;;\n\tnsx-tandem)\n\t\tbasic_machine=nsx-tandem\n\t\t;;\n\top50n-* | op60c-*)\n\t\tbasic_machine=hppa1.1-oki\n\t\tos=-proelf\n\t\t;;\n\topenrisc | openrisc-*)\n\t\tbasic_machine=or32-unknown\n\t\t;;\n\tos400)\n\t\tbasic_machine=powerpc-ibm\n\t\tos=-os400\n\t\t;;\n\tOSE68000 | ose68000)\n\t\tbasic_machine=m68000-ericsson\n\t\tos=-ose\n\t\t;;\n\tos68k)\n\t\tbasic_machine=m68k-none\n\t\tos=-os68k\n\t\t;;\n\tpa-hitachi)\n\t\tbasic_machine=hppa1.1-hitachi\n\t\tos=-hiuxwe2\n\t\t;;\n\tparagon)\n\t\tbasic_machine=i860-intel\n\t\tos=-osf\n\t\t;;\n\tparisc)\n\t\tbasic_machine=hppa-unknown\n\t\tos=-linux\n\t\t;;\n\tparisc-*)\n\t\tbasic_machine=hppa-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\tos=-linux\n\t\t;;\n\tpbd)\n\t\tbasic_machine=sparc-tti\n\t\t;;\n\tpbb)\n\t\tbasic_machine=m68k-tti\n\t\t;;\n\tpc532 | pc532-*)\n\t\tbasic_machine=ns32k-pc532\n\t\t;;\n\tpc98)\n\t\tbasic_machine=i386-pc\n\t\t;;\n\tpc98-*)\n\t\tbasic_machine=i386-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tpentium | p5 | k5 | k6 | nexgen | viac3)\n\t\tbasic_machine=i586-pc\n\t\t;;\n\tpentiumpro | p6 | 6x86 | athlon | athlon_*)\n\t\tbasic_machine=i686-pc\n\t\t;;\n\tpentiumii | pentium2 | pentiumiii | pentium3)\n\t\tbasic_machine=i686-pc\n\t\t;;\n\tpentium4)\n\t\tbasic_machine=i786-pc\n\t\t;;\n\tpentium-* | p5-* | k5-* | k6-* | nexgen-* | viac3-*)\n\t\tbasic_machine=i586-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tpentiumpro-* | p6-* | 6x86-* | athlon-*)\n\t\tbasic_machine=i686-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tpentiumii-* | pentium2-* | pentiumiii-* | pentium3-*)\n\t\tbasic_machine=i686-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tpentium4-*)\n\t\tbasic_machine=i786-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tpn)\n\t\tbasic_machine=pn-gould\n\t\t;;\n\tpower)\tbasic_machine=power-ibm\n\t\t;;\n\tppc | ppcbe)\tbasic_machine=powerpc-unknown\n\t\t;;\n\tppc-* | ppcbe-*)\n\t\tbasic_machine=powerpc-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tppcle | powerpclittle)\n\t\tbasic_machine=powerpcle-unknown\n\t\t;;\n\tppcle-* | powerpclittle-*)\n\t\tbasic_machine=powerpcle-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tppc64)\tbasic_machine=powerpc64-unknown\n\t\t;;\n\tppc64-*) basic_machine=powerpc64-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tppc64le | powerpc64little)\n\t\tbasic_machine=powerpc64le-unknown\n\t\t;;\n\tppc64le-* | powerpc64little-*)\n\t\tbasic_machine=powerpc64le-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tps2)\n\t\tbasic_machine=i386-ibm\n\t\t;;\n\tpw32)\n\t\tbasic_machine=i586-unknown\n\t\tos=-pw32\n\t\t;;\n\trdos | rdos64)\n\t\tbasic_machine=x86_64-pc\n\t\tos=-rdos\n\t\t;;\n\trdos32)\n\t\tbasic_machine=i386-pc\n\t\tos=-rdos\n\t\t;;\n\trom68k)\n\t\tbasic_machine=m68k-rom68k\n\t\tos=-coff\n\t\t;;\n\trm[46]00)\n\t\tbasic_machine=mips-siemens\n\t\t;;\n\trtpc | rtpc-*)\n\t\tbasic_machine=romp-ibm\n\t\t;;\n\ts390 | s390-*)\n\t\tbasic_machine=s390-ibm\n\t\t;;\n\ts390x | s390x-*)\n\t\tbasic_machine=s390x-ibm\n\t\t;;\n\tsa29200)\n\t\tbasic_machine=a29k-amd\n\t\tos=-udi\n\t\t;;\n\tsb1)\n\t\tbasic_machine=mipsisa64sb1-unknown\n\t\t;;\n\tsb1el)\n\t\tbasic_machine=mipsisa64sb1el-unknown\n\t\t;;\n\tsde)\n\t\tbasic_machine=mipsisa32-sde\n\t\tos=-elf\n\t\t;;\n\tsei)\n\t\tbasic_machine=mips-sei\n\t\tos=-seiux\n\t\t;;\n\tsequent)\n\t\tbasic_machine=i386-sequent\n\t\t;;\n\tsh)\n\t\tbasic_machine=sh-hitachi\n\t\tos=-hms\n\t\t;;\n\tsh5el)\n\t\tbasic_machine=sh5le-unknown\n\t\t;;\n\tsh64)\n\t\tbasic_machine=sh64-unknown\n\t\t;;\n\tsparclite-wrs | simso-wrs)\n\t\tbasic_machine=sparclite-wrs\n\t\tos=-vxworks\n\t\t;;\n\tsps7)\n\t\tbasic_machine=m68k-bull\n\t\tos=-sysv2\n\t\t;;\n\tspur)\n\t\tbasic_machine=spur-unknown\n\t\t;;\n\tst2000)\n\t\tbasic_machine=m68k-tandem\n\t\t;;\n\tstratus)\n\t\tbasic_machine=i860-stratus\n\t\tos=-sysv4\n\t\t;;\n\tstrongarm-* | thumb-*)\n\t\tbasic_machine=arm-`echo $basic_machine | sed 's/^[^-]*-//'`\n\t\t;;\n\tsun2)\n\t\tbasic_machine=m68000-sun\n\t\t;;\n\tsun2os3)\n\t\tbasic_machine=m68000-sun\n\t\tos=-sunos3\n\t\t;;\n\tsun2os4)\n\t\tbasic_machine=m68000-sun\n\t\tos=-sunos4\n\t\t;;\n\tsun3os3)\n\t\tbasic_machine=m68k-sun\n\t\tos=-sunos3\n\t\t;;\n\tsun3os4)\n\t\tbasic_machine=m68k-sun\n\t\tos=-sunos4\n\t\t;;\n\tsun4os3)\n\t\tbasic_machine=sparc-sun\n\t\tos=-sunos3\n\t\t;;\n\tsun4os4)\n\t\tbasic_machine=sparc-sun\n\t\tos=-sunos4\n\t\t;;\n\tsun4sol2)\n\t\tbasic_machine=sparc-sun\n\t\tos=-solaris2\n\t\t;;\n\tsun3 | sun3-*)\n\t\tbasic_machine=m68k-sun\n\t\t;;\n\tsun4)\n\t\tbasic_machine=sparc-sun\n\t\t;;\n\tsun386 | sun386i | roadrunner)\n\t\tbasic_machine=i386-sun\n\t\t;;\n\tsv1)\n\t\tbasic_machine=sv1-cray\n\t\tos=-unicos\n\t\t;;\n\tsymmetry)\n\t\tbasic_machine=i386-sequent\n\t\tos=-dynix\n\t\t;;\n\tt3e)\n\t\tbasic_machine=alphaev5-cray\n\t\tos=-unicos\n\t\t;;\n\tt90)\n\t\tbasic_machine=t90-cray\n\t\tos=-unicos\n\t\t;;\n\ttile*)\n\t\tbasic_machine=$basic_machine-unknown\n\t\tos=-linux-gnu\n\t\t;;\n\ttx39)\n\t\tbasic_machine=mipstx39-unknown\n\t\t;;\n\ttx39el)\n\t\tbasic_machine=mipstx39el-unknown\n\t\t;;\n\ttoad1)\n\t\tbasic_machine=pdp10-xkl\n\t\tos=-tops20\n\t\t;;\n\ttower | tower-32)\n\t\tbasic_machine=m68k-ncr\n\t\t;;\n\ttpf)\n\t\tbasic_machine=s390x-ibm\n\t\tos=-tpf\n\t\t;;\n\tudi29k)\n\t\tbasic_machine=a29k-amd\n\t\tos=-udi\n\t\t;;\n\tultra3)\n\t\tbasic_machine=a29k-nyu\n\t\tos=-sym1\n\t\t;;\n\tv810 | necv810)\n\t\tbasic_machine=v810-nec\n\t\tos=-none\n\t\t;;\n\tvaxv)\n\t\tbasic_machine=vax-dec\n\t\tos=-sysv\n\t\t;;\n\tvms)\n\t\tbasic_machine=vax-dec\n\t\tos=-vms\n\t\t;;\n\tvpp*|vx|vx-*)\n\t\tbasic_machine=f301-fujitsu\n\t\t;;\n\tvxworks960)\n\t\tbasic_machine=i960-wrs\n\t\tos=-vxworks\n\t\t;;\n\tvxworks68)\n\t\tbasic_machine=m68k-wrs\n\t\tos=-vxworks\n\t\t;;\n\tvxworks29k)\n\t\tbasic_machine=a29k-wrs\n\t\tos=-vxworks\n\t\t;;\n\twasm32)\n\t\tbasic_machine=wasm32-unknown\n\t\t;;\n\tw65*)\n\t\tbasic_machine=w65-wdc\n\t\tos=-none\n\t\t;;\n\tw89k-*)\n\t\tbasic_machine=hppa1.1-winbond\n\t\tos=-proelf\n\t\t;;\n\txbox)\n\t\tbasic_machine=i686-pc\n\t\tos=-mingw32\n\t\t;;\n\txps | xps100)\n\t\tbasic_machine=xps100-honeywell\n\t\t;;\n\txscale-* | xscalee[bl]-*)\n\t\tbasic_machine=`echo $basic_machine | sed 's/^xscale/arm/'`\n\t\t;;\n\tymp)\n\t\tbasic_machine=ymp-cray\n\t\tos=-unicos\n\t\t;;\n\tz8k-*-coff)\n\t\tbasic_machine=z8k-unknown\n\t\tos=-sim\n\t\t;;\n\tz80-*-coff)\n\t\tbasic_machine=z80-unknown\n\t\tos=-sim\n\t\t;;\n\tnone)\n\t\tbasic_machine=none-none\n\t\tos=-none\n\t\t;;\n\n# Here we handle the default manufacturer of certain CPU types.  It is in\n# some cases the only manufacturer, in others, it is the most popular.\n\tw89k)\n\t\tbasic_machine=hppa1.1-winbond\n\t\t;;\n\top50n)\n\t\tbasic_machine=hppa1.1-oki\n\t\t;;\n\top60c)\n\t\tbasic_machine=hppa1.1-oki\n\t\t;;\n\tromp)\n\t\tbasic_machine=romp-ibm\n\t\t;;\n\tmmix)\n\t\tbasic_machine=mmix-knuth\n\t\t;;\n\trs6000)\n\t\tbasic_machine=rs6000-ibm\n\t\t;;\n\tvax)\n\t\tbasic_machine=vax-dec\n\t\t;;\n\tpdp10)\n\t\t# there are many clones, so DEC is not a safe bet\n\t\tbasic_machine=pdp10-unknown\n\t\t;;\n\tpdp11)\n\t\tbasic_machine=pdp11-dec\n\t\t;;\n\twe32k)\n\t\tbasic_machine=we32k-att\n\t\t;;\n\tsh[1234] | sh[24]a | sh[24]aeb | sh[34]eb | sh[1234]le | sh[23]ele)\n\t\tbasic_machine=sh-unknown\n\t\t;;\n\tsparc | sparcv8 | sparcv9 | sparcv9b | sparcv9v)\n\t\tbasic_machine=sparc-sun\n\t\t;;\n\tcydra)\n\t\tbasic_machine=cydra-cydrome\n\t\t;;\n\torion)\n\t\tbasic_machine=orion-highlevel\n\t\t;;\n\torion105)\n\t\tbasic_machine=clipper-highlevel\n\t\t;;\n\tmac | mpw | mac-mpw)\n\t\tbasic_machine=m68k-apple\n\t\t;;\n\tpmac | pmac-mpw)\n\t\tbasic_machine=powerpc-apple\n\t\t;;\n\t*-unknown)\n\t\t# Make sure to match an already-canonicalized machine name.\n\t\t;;\n\t*)\n\t\techo Invalid configuration \\`$1\\': machine \\`$basic_machine\\' not recognized 1>&2\n\t\texit 1\n\t\t;;\nesac\n\n# Here we canonicalize certain aliases for manufacturers.\ncase $basic_machine in\n\t*-digital*)\n\t\tbasic_machine=`echo $basic_machine | sed 's/digital.*/dec/'`\n\t\t;;\n\t*-commodore*)\n\t\tbasic_machine=`echo $basic_machine | sed 's/commodore.*/cbm/'`\n\t\t;;\n\t*)\n\t\t;;\nesac\n\n# Decode manufacturer-specific aliases for certain operating systems.\n\nif [ x\"$os\" != x\"\" ]\nthen\ncase $os in\n\t# First match some system type aliases\n\t# that might get confused with valid system types.\n\t# -solaris* is a basic system type, with this one exception.\n\t-auroraux)\n\t\tos=-auroraux\n\t\t;;\n\t-solaris1 | -solaris1.*)\n\t\tos=`echo $os | sed -e 's|solaris1|sunos4|'`\n\t\t;;\n\t-solaris)\n\t\tos=-solaris2\n\t\t;;\n\t-svr4*)\n\t\tos=-sysv4\n\t\t;;\n\t-unixware*)\n\t\tos=-sysv4.2uw\n\t\t;;\n\t-gnu/linux*)\n\t\tos=`echo $os | sed -e 's|gnu/linux|linux-gnu|'`\n\t\t;;\n\t# First accept the basic system types.\n\t# The portable systems comes first.\n\t# Each alternative MUST END IN A *, to match a version number.\n\t# -sysv* is not here because it comes later, after sysvr4.\n\t-gnu* | -bsd* | -mach* | -minix* | -genix* | -ultrix* | -irix* \\\n\t      | -*vms* | -sco* | -esix* | -isc* | -aix* | -cnk* | -sunos | -sunos[34]*\\\n\t      | -hpux* | -unos* | -osf* | -luna* | -dgux* | -auroraux* | -solaris* \\\n\t      | -sym* | -kopensolaris* | -plan9* \\\n\t      | -amigaos* | -amigados* | -msdos* | -newsos* | -unicos* | -aof* \\\n\t      | -aos* | -aros* | -cloudabi* | -sortix* \\\n\t      | -nindy* | -vxsim* | -vxworks* | -ebmon* | -hms* | -mvs* \\\n\t      | -clix* | -riscos* | -uniplus* | -iris* | -rtu* | -xenix* \\\n\t      | -hiux* | -386bsd* | -knetbsd* | -mirbsd* | -netbsd* \\\n\t      | -bitrig* | -openbsd* | -solidbsd* | -libertybsd* \\\n\t      | -ekkobsd* | -kfreebsd* | -freebsd* | -riscix* | -lynxos* \\\n\t      | -bosx* | -nextstep* | -cxux* | -aout* | -elf* | -oabi* \\\n\t      | -ptx* | -coff* | -ecoff* | -winnt* | -domain* | -vsta* \\\n\t      | -udi* | -eabi* | -lites* | -ieee* | -go32* | -aux* \\\n\t      | -chorusos* | -chorusrdb* | -cegcc* | -glidix* \\\n\t      | -cygwin* | -msys* | -pe* | -psos* | -moss* | -proelf* | -rtems* \\\n\t      | -midipix* | -mingw32* | -mingw64* | -linux-gnu* | -linux-android* \\\n\t      | -linux-newlib* | -linux-musl* | -linux-uclibc* \\\n\t      | -uxpv* | -beos* | -mpeix* | -udk* | -moxiebox* \\\n\t      | -interix* | -uwin* | -mks* | -rhapsody* | -darwin* | -opened* \\\n\t      | -openstep* | -oskit* | -conix* | -pw32* | -nonstopux* \\\n\t      | -storm-chaos* | -tops10* | -tenex* | -tops20* | -its* \\\n\t      | -os2* | -vos* | -palmos* | -uclinux* | -nucleus* \\\n\t      | -morphos* | -superux* | -rtmk* | -rtmk-nova* | -windiss* \\\n\t      | -powermax* | -dnix* | -nx6 | -nx7 | -sei* | -dragonfly* \\\n\t      | -skyos* | -haiku* | -rdos* | -toppers* | -drops* | -es* \\\n\t      | -onefs* | -tirtos* | -phoenix* | -fuchsia* | -redox*)\n\t# Remember, each alternative MUST END IN *, to match a version number.\n\t\t;;\n\t-qnx*)\n\t\tcase $basic_machine in\n\t\t    x86-* | i*86-*)\n\t\t\t;;\n\t\t    *)\n\t\t\tos=-nto$os\n\t\t\t;;\n\t\tesac\n\t\t;;\n\t-nto-qnx*)\n\t\t;;\n\t-nto*)\n\t\tos=`echo $os | sed -e 's|nto|nto-qnx|'`\n\t\t;;\n\t-sim | -es1800* | -hms* | -xray | -os68k* | -none* | -v88r* \\\n\t      | -windows* | -osx | -abug | -netware* | -os9* | -beos* | -haiku* \\\n\t      | -macos* | -mpw* | -magic* | -mmixware* | -mon960* | -lnews*)\n\t\t;;\n\t-mac*)\n\t\tos=`echo $os | sed -e 's|mac|macos|'`\n\t\t;;\n\t-linux-dietlibc)\n\t\tos=-linux-dietlibc\n\t\t;;\n\t-linux*)\n\t\tos=`echo $os | sed -e 's|linux|linux-gnu|'`\n\t\t;;\n\t-sunos5*)\n\t\tos=`echo $os | sed -e 's|sunos5|solaris2|'`\n\t\t;;\n\t-sunos6*)\n\t\tos=`echo $os | sed -e 's|sunos6|solaris3|'`\n\t\t;;\n\t-opened*)\n\t\tos=-openedition\n\t\t;;\n\t-os400*)\n\t\tos=-os400\n\t\t;;\n\t-wince*)\n\t\tos=-wince\n\t\t;;\n\t-osfrose*)\n\t\tos=-osfrose\n\t\t;;\n\t-osf*)\n\t\tos=-osf\n\t\t;;\n\t-utek*)\n\t\tos=-bsd\n\t\t;;\n\t-dynix*)\n\t\tos=-bsd\n\t\t;;\n\t-acis*)\n\t\tos=-aos\n\t\t;;\n\t-atheos*)\n\t\tos=-atheos\n\t\t;;\n\t-syllable*)\n\t\tos=-syllable\n\t\t;;\n\t-386bsd)\n\t\tos=-bsd\n\t\t;;\n\t-ctix* | -uts*)\n\t\tos=-sysv\n\t\t;;\n\t-nova*)\n\t\tos=-rtmk-nova\n\t\t;;\n\t-ns2 )\n\t\tos=-nextstep2\n\t\t;;\n\t-nsk*)\n\t\tos=-nsk\n\t\t;;\n\t# Preserve the version number of sinix5.\n\t-sinix5.*)\n\t\tos=`echo $os | sed -e 's|sinix|sysv|'`\n\t\t;;\n\t-sinix*)\n\t\tos=-sysv4\n\t\t;;\n\t-tpf*)\n\t\tos=-tpf\n\t\t;;\n\t-triton*)\n\t\tos=-sysv3\n\t\t;;\n\t-oss*)\n\t\tos=-sysv3\n\t\t;;\n\t-svr4)\n\t\tos=-sysv4\n\t\t;;\n\t-svr3)\n\t\tos=-sysv3\n\t\t;;\n\t-sysvr4)\n\t\tos=-sysv4\n\t\t;;\n\t# This must come after -sysvr4.\n\t-sysv*)\n\t\t;;\n\t-ose*)\n\t\tos=-ose\n\t\t;;\n\t-es1800*)\n\t\tos=-ose\n\t\t;;\n\t-xenix)\n\t\tos=-xenix\n\t\t;;\n\t-*mint | -mint[0-9]* | -*MiNT | -MiNT[0-9]*)\n\t\tos=-mint\n\t\t;;\n\t-aros*)\n\t\tos=-aros\n\t\t;;\n\t-zvmoe)\n\t\tos=-zvmoe\n\t\t;;\n\t-dicos*)\n\t\tos=-dicos\n\t\t;;\n\t-nacl*)\n\t\t;;\n\t-ios)\n\t\t;;\n\t-none)\n\t\t;;\n\t*)\n\t\t# Get rid of the `-' at the beginning of $os.\n\t\tos=`echo $os | sed 's/[^-]*-//'`\n\t\techo Invalid configuration \\`$1\\': system \\`$os\\' not recognized 1>&2\n\t\texit 1\n\t\t;;\nesac\nelse\n\n# Here we handle the default operating systems that come with various machines.\n# The value should be what the vendor currently ships out the door with their\n# machine or put another way, the most popular os provided with the machine.\n\n# Note that if you're going to try to match \"-MANUFACTURER\" here (say,\n# \"-sun\"), then you have to tell the case statement up towards the top\n# that MANUFACTURER isn't an operating system.  Otherwise, code above\n# will signal an error saying that MANUFACTURER isn't an operating\n# system, and we'll never get to this point.\n\ncase $basic_machine in\n\tscore-*)\n\t\tos=-elf\n\t\t;;\n\tspu-*)\n\t\tos=-elf\n\t\t;;\n\t*-acorn)\n\t\tos=-riscix1.2\n\t\t;;\n\tarm*-rebel)\n\t\tos=-linux\n\t\t;;\n\tarm*-semi)\n\t\tos=-aout\n\t\t;;\n\tc4x-* | tic4x-*)\n\t\tos=-coff\n\t\t;;\n\tc8051-*)\n\t\tos=-elf\n\t\t;;\n\thexagon-*)\n\t\tos=-elf\n\t\t;;\n\ttic54x-*)\n\t\tos=-coff\n\t\t;;\n\ttic55x-*)\n\t\tos=-coff\n\t\t;;\n\ttic6x-*)\n\t\tos=-coff\n\t\t;;\n\t# This must come before the *-dec entry.\n\tpdp10-*)\n\t\tos=-tops20\n\t\t;;\n\tpdp11-*)\n\t\tos=-none\n\t\t;;\n\t*-dec | vax-*)\n\t\tos=-ultrix4.2\n\t\t;;\n\tm68*-apollo)\n\t\tos=-domain\n\t\t;;\n\ti386-sun)\n\t\tos=-sunos4.0.2\n\t\t;;\n\tm68000-sun)\n\t\tos=-sunos3\n\t\t;;\n\tm68*-cisco)\n\t\tos=-aout\n\t\t;;\n\tmep-*)\n\t\tos=-elf\n\t\t;;\n\tmips*-cisco)\n\t\tos=-elf\n\t\t;;\n\tmips*-*)\n\t\tos=-elf\n\t\t;;\n\tor32-*)\n\t\tos=-coff\n\t\t;;\n\t*-tti)\t# must be before sparc entry or we get the wrong os.\n\t\tos=-sysv3\n\t\t;;\n\tsparc-* | *-sun)\n\t\tos=-sunos4.1.1\n\t\t;;\n\tpru-*)\n\t\tos=-elf\n\t\t;;\n\t*-be)\n\t\tos=-beos\n\t\t;;\n\t*-haiku)\n\t\tos=-haiku\n\t\t;;\n\t*-ibm)\n\t\tos=-aix\n\t\t;;\n\t*-knuth)\n\t\tos=-mmixware\n\t\t;;\n\t*-wec)\n\t\tos=-proelf\n\t\t;;\n\t*-winbond)\n\t\tos=-proelf\n\t\t;;\n\t*-oki)\n\t\tos=-proelf\n\t\t;;\n\t*-hp)\n\t\tos=-hpux\n\t\t;;\n\t*-hitachi)\n\t\tos=-hiux\n\t\t;;\n\ti860-* | *-att | *-ncr | *-altos | *-motorola | *-convergent)\n\t\tos=-sysv\n\t\t;;\n\t*-cbm)\n\t\tos=-amigaos\n\t\t;;\n\t*-dg)\n\t\tos=-dgux\n\t\t;;\n\t*-dolphin)\n\t\tos=-sysv3\n\t\t;;\n\tm68k-ccur)\n\t\tos=-rtu\n\t\t;;\n\tm88k-omron*)\n\t\tos=-luna\n\t\t;;\n\t*-next )\n\t\tos=-nextstep\n\t\t;;\n\t*-sequent)\n\t\tos=-ptx\n\t\t;;\n\t*-crds)\n\t\tos=-unos\n\t\t;;\n\t*-ns)\n\t\tos=-genix\n\t\t;;\n\ti370-*)\n\t\tos=-mvs\n\t\t;;\n\t*-next)\n\t\tos=-nextstep3\n\t\t;;\n\t*-gould)\n\t\tos=-sysv\n\t\t;;\n\t*-highlevel)\n\t\tos=-bsd\n\t\t;;\n\t*-encore)\n\t\tos=-bsd\n\t\t;;\n\t*-sgi)\n\t\tos=-irix\n\t\t;;\n\t*-siemens)\n\t\tos=-sysv4\n\t\t;;\n\t*-masscomp)\n\t\tos=-rtu\n\t\t;;\n\tf30[01]-fujitsu | f700-fujitsu)\n\t\tos=-uxpv\n\t\t;;\n\t*-rom68k)\n\t\tos=-coff\n\t\t;;\n\t*-*bug)\n\t\tos=-coff\n\t\t;;\n\t*-apple)\n\t\tos=-macos\n\t\t;;\n\t*-atari*)\n\t\tos=-mint\n\t\t;;\n\t*)\n\t\tos=-none\n\t\t;;\nesac\nfi\n\n# Here we handle the case where we know the os, and the CPU type, but not the\n# manufacturer.  We pick the logical manufacturer.\nvendor=unknown\ncase $basic_machine in\n\t*-unknown)\n\t\tcase $os in\n\t\t\t-riscix*)\n\t\t\t\tvendor=acorn\n\t\t\t\t;;\n\t\t\t-sunos*)\n\t\t\t\tvendor=sun\n\t\t\t\t;;\n\t\t\t-cnk*|-aix*)\n\t\t\t\tvendor=ibm\n\t\t\t\t;;\n\t\t\t-beos*)\n\t\t\t\tvendor=be\n\t\t\t\t;;\n\t\t\t-hpux*)\n\t\t\t\tvendor=hp\n\t\t\t\t;;\n\t\t\t-mpeix*)\n\t\t\t\tvendor=hp\n\t\t\t\t;;\n\t\t\t-hiux*)\n\t\t\t\tvendor=hitachi\n\t\t\t\t;;\n\t\t\t-unos*)\n\t\t\t\tvendor=crds\n\t\t\t\t;;\n\t\t\t-dgux*)\n\t\t\t\tvendor=dg\n\t\t\t\t;;\n\t\t\t-luna*)\n\t\t\t\tvendor=omron\n\t\t\t\t;;\n\t\t\t-genix*)\n\t\t\t\tvendor=ns\n\t\t\t\t;;\n\t\t\t-mvs* | -opened*)\n\t\t\t\tvendor=ibm\n\t\t\t\t;;\n\t\t\t-os400*)\n\t\t\t\tvendor=ibm\n\t\t\t\t;;\n\t\t\t-ptx*)\n\t\t\t\tvendor=sequent\n\t\t\t\t;;\n\t\t\t-tpf*)\n\t\t\t\tvendor=ibm\n\t\t\t\t;;\n\t\t\t-vxsim* | -vxworks* | -windiss*)\n\t\t\t\tvendor=wrs\n\t\t\t\t;;\n\t\t\t-aux*)\n\t\t\t\tvendor=apple\n\t\t\t\t;;\n\t\t\t-hms*)\n\t\t\t\tvendor=hitachi\n\t\t\t\t;;\n\t\t\t-mpw* | -macos*)\n\t\t\t\tvendor=apple\n\t\t\t\t;;\n\t\t\t-*mint | -mint[0-9]* | -*MiNT | -MiNT[0-9]*)\n\t\t\t\tvendor=atari\n\t\t\t\t;;\n\t\t\t-vos*)\n\t\t\t\tvendor=stratus\n\t\t\t\t;;\n\t\tesac\n\t\tbasic_machine=`echo $basic_machine | sed \"s/unknown/$vendor/\"`\n\t\t;;\nesac\n\necho $basic_machine$os\nexit\n\n# Local variables:\n# eval: (add-hook 'write-file-hooks 'time-stamp)\n# time-stamp-start: \"timestamp='\"\n# time-stamp-format: \"%:y-%02m-%02d\"\n# time-stamp-end: \"'\"\n# End:\n"
  },
  {
    "path": "build-aux/install-sh",
    "content": "#!/bin/sh\n# install - install a program, script, or datafile\n\nscriptversion=2016-01-11.22; # UTC\n\n# This originates from X11R5 (mit/util/scripts/install.sh), which was\n# later released in X11R6 (xc/config/util/install.sh) with the\n# following copyright and license.\n#\n# Copyright (C) 1994 X Consortium\n#\n# Permission is hereby granted, free of charge, to any person obtaining a copy\n# of this software and associated documentation files (the \"Software\"), to\n# deal in the Software without restriction, including without limitation the\n# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or\n# sell copies of the Software, and to permit persons to whom the Software is\n# furnished to do so, subject to the following conditions:\n#\n# The above copyright notice and this permission notice shall be included in\n# all copies or substantial portions of the Software.\n#\n# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE\n# X CONSORTIUM BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN\n# AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNEC-\n# TION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n#\n# Except as contained in this notice, the name of the X Consortium shall not\n# be used in advertising or otherwise to promote the sale, use or other deal-\n# ings in this Software without prior written authorization from the X Consor-\n# tium.\n#\n#\n# FSF changes to this file are in the public domain.\n#\n# Calling this script install-sh is preferred over install.sh, to prevent\n# 'make' implicit rules from creating a file called install from it\n# when there is no Makefile.\n#\n# This script is compatible with the BSD install script, but was written\n# from scratch.\n\ntab='\t'\nnl='\n'\nIFS=\" $tab$nl\"\n\n# Set DOITPROG to \"echo\" to test this script.\n\ndoit=${DOITPROG-}\ndoit_exec=${doit:-exec}\n\n# Put in absolute file names if you don't have them in your path;\n# or use environment vars.\n\nchgrpprog=${CHGRPPROG-chgrp}\nchmodprog=${CHMODPROG-chmod}\nchownprog=${CHOWNPROG-chown}\ncmpprog=${CMPPROG-cmp}\ncpprog=${CPPROG-cp}\nmkdirprog=${MKDIRPROG-mkdir}\nmvprog=${MVPROG-mv}\nrmprog=${RMPROG-rm}\nstripprog=${STRIPPROG-strip}\n\nposix_mkdir=\n\n# Desired mode of installed file.\nmode=0755\n\nchgrpcmd=\nchmodcmd=$chmodprog\nchowncmd=\nmvcmd=$mvprog\nrmcmd=\"$rmprog -f\"\nstripcmd=\n\nsrc=\ndst=\ndir_arg=\ndst_arg=\n\ncopy_on_change=false\nis_target_a_directory=possibly\n\nusage=\"\\\nUsage: $0 [OPTION]... [-T] SRCFILE DSTFILE\n   or: $0 [OPTION]... SRCFILES... DIRECTORY\n   or: $0 [OPTION]... -t DIRECTORY SRCFILES...\n   or: $0 [OPTION]... -d DIRECTORIES...\n\nIn the 1st form, copy SRCFILE to DSTFILE.\nIn the 2nd and 3rd, copy all SRCFILES to DIRECTORY.\nIn the 4th, create DIRECTORIES.\n\nOptions:\n     --help     display this help and exit.\n     --version  display version info and exit.\n\n  -c            (ignored)\n  -C            install only if different (preserve the last data modification time)\n  -d            create directories instead of installing files.\n  -g GROUP      $chgrpprog installed files to GROUP.\n  -m MODE       $chmodprog installed files to MODE.\n  -o USER       $chownprog installed files to USER.\n  -s            $stripprog installed files.\n  -t DIRECTORY  install into DIRECTORY.\n  -T            report an error if DSTFILE is a directory.\n\nEnvironment variables override the default commands:\n  CHGRPPROG CHMODPROG CHOWNPROG CMPPROG CPPROG MKDIRPROG MVPROG\n  RMPROG STRIPPROG\n\"\n\nwhile test $# -ne 0; do\n  case $1 in\n    -c) ;;\n\n    -C) copy_on_change=true;;\n\n    -d) dir_arg=true;;\n\n    -g) chgrpcmd=\"$chgrpprog $2\"\n        shift;;\n\n    --help) echo \"$usage\"; exit $?;;\n\n    -m) mode=$2\n        case $mode in\n          *' '* | *\"$tab\"* | *\"$nl\"* | *'*'* | *'?'* | *'['*)\n            echo \"$0: invalid mode: $mode\" >&2\n            exit 1;;\n        esac\n        shift;;\n\n    -o) chowncmd=\"$chownprog $2\"\n        shift;;\n\n    -s) stripcmd=$stripprog;;\n\n    -t)\n        is_target_a_directory=always\n        dst_arg=$2\n        # Protect names problematic for 'test' and other utilities.\n        case $dst_arg in\n          -* | [=\\(\\)!]) dst_arg=./$dst_arg;;\n        esac\n        shift;;\n\n    -T) is_target_a_directory=never;;\n\n    --version) echo \"$0 $scriptversion\"; exit $?;;\n\n    --) shift\n        break;;\n\n    -*) echo \"$0: invalid option: $1\" >&2\n        exit 1;;\n\n    *)  break;;\n  esac\n  shift\ndone\n\n# We allow the use of options -d and -T together, by making -d\n# take the precedence; this is for compatibility with GNU install.\n\nif test -n \"$dir_arg\"; then\n  if test -n \"$dst_arg\"; then\n    echo \"$0: target directory not allowed when installing a directory.\" >&2\n    exit 1\n  fi\nfi\n\nif test $# -ne 0 && test -z \"$dir_arg$dst_arg\"; then\n  # When -d is used, all remaining arguments are directories to create.\n  # When -t is used, the destination is already specified.\n  # Otherwise, the last argument is the destination.  Remove it from $@.\n  for arg\n  do\n    if test -n \"$dst_arg\"; then\n      # $@ is not empty: it contains at least $arg.\n      set fnord \"$@\" \"$dst_arg\"\n      shift # fnord\n    fi\n    shift # arg\n    dst_arg=$arg\n    # Protect names problematic for 'test' and other utilities.\n    case $dst_arg in\n      -* | [=\\(\\)!]) dst_arg=./$dst_arg;;\n    esac\n  done\nfi\n\nif test $# -eq 0; then\n  if test -z \"$dir_arg\"; then\n    echo \"$0: no input file specified.\" >&2\n    exit 1\n  fi\n  # It's OK to call 'install-sh -d' without argument.\n  # This can happen when creating conditional directories.\n  exit 0\nfi\n\nif test -z \"$dir_arg\"; then\n  if test $# -gt 1 || test \"$is_target_a_directory\" = always; then\n    if test ! -d \"$dst_arg\"; then\n      echo \"$0: $dst_arg: Is not a directory.\" >&2\n      exit 1\n    fi\n  fi\nfi\n\nif test -z \"$dir_arg\"; then\n  do_exit='(exit $ret); exit $ret'\n  trap \"ret=129; $do_exit\" 1\n  trap \"ret=130; $do_exit\" 2\n  trap \"ret=141; $do_exit\" 13\n  trap \"ret=143; $do_exit\" 15\n\n  # Set umask so as not to create temps with too-generous modes.\n  # However, 'strip' requires both read and write access to temps.\n  case $mode in\n    # Optimize common cases.\n    *644) cp_umask=133;;\n    *755) cp_umask=22;;\n\n    *[0-7])\n      if test -z \"$stripcmd\"; then\n        u_plus_rw=\n      else\n        u_plus_rw='% 200'\n      fi\n      cp_umask=`expr '(' 777 - $mode % 1000 ')' $u_plus_rw`;;\n    *)\n      if test -z \"$stripcmd\"; then\n        u_plus_rw=\n      else\n        u_plus_rw=,u+rw\n      fi\n      cp_umask=$mode$u_plus_rw;;\n  esac\nfi\n\nfor src\ndo\n  # Protect names problematic for 'test' and other utilities.\n  case $src in\n    -* | [=\\(\\)!]) src=./$src;;\n  esac\n\n  if test -n \"$dir_arg\"; then\n    dst=$src\n    dstdir=$dst\n    test -d \"$dstdir\"\n    dstdir_status=$?\n  else\n\n    # Waiting for this to be detected by the \"$cpprog $src $dsttmp\" command\n    # might cause directories to be created, which would be especially bad\n    # if $src (and thus $dsttmp) contains '*'.\n    if test ! -f \"$src\" && test ! -d \"$src\"; then\n      echo \"$0: $src does not exist.\" >&2\n      exit 1\n    fi\n\n    if test -z \"$dst_arg\"; then\n      echo \"$0: no destination specified.\" >&2\n      exit 1\n    fi\n    dst=$dst_arg\n\n    # If destination is a directory, append the input filename; won't work\n    # if double slashes aren't ignored.\n    if test -d \"$dst\"; then\n      if test \"$is_target_a_directory\" = never; then\n        echo \"$0: $dst_arg: Is a directory\" >&2\n        exit 1\n      fi\n      dstdir=$dst\n      dst=$dstdir/`basename \"$src\"`\n      dstdir_status=0\n    else\n      dstdir=`dirname \"$dst\"`\n      test -d \"$dstdir\"\n      dstdir_status=$?\n    fi\n  fi\n\n  obsolete_mkdir_used=false\n\n  if test $dstdir_status != 0; then\n    case $posix_mkdir in\n      '')\n        # Create intermediate dirs using mode 755 as modified by the umask.\n        # This is like FreeBSD 'install' as of 1997-10-28.\n        umask=`umask`\n        case $stripcmd.$umask in\n          # Optimize common cases.\n          *[2367][2367]) mkdir_umask=$umask;;\n          .*0[02][02] | .[02][02] | .[02]) mkdir_umask=22;;\n\n          *[0-7])\n            mkdir_umask=`expr $umask + 22 \\\n              - $umask % 100 % 40 + $umask % 20 \\\n              - $umask % 10 % 4 + $umask % 2\n            `;;\n          *) mkdir_umask=$umask,go-w;;\n        esac\n\n        # With -d, create the new directory with the user-specified mode.\n        # Otherwise, rely on $mkdir_umask.\n        if test -n \"$dir_arg\"; then\n          mkdir_mode=-m$mode\n        else\n          mkdir_mode=\n        fi\n\n        posix_mkdir=false\n        case $umask in\n          *[123567][0-7][0-7])\n            # POSIX mkdir -p sets u+wx bits regardless of umask, which\n            # is incompatible with FreeBSD 'install' when (umask & 300) != 0.\n            ;;\n          *)\n            tmpdir=${TMPDIR-/tmp}/ins$RANDOM-$$\n            trap 'ret=$?; rmdir \"$tmpdir/d\" \"$tmpdir\" 2>/dev/null; exit $ret' 0\n\n            if (umask $mkdir_umask &&\n                exec $mkdirprog $mkdir_mode -p -- \"$tmpdir/d\") >/dev/null 2>&1\n            then\n              if test -z \"$dir_arg\" || {\n                   # Check for POSIX incompatibilities with -m.\n                   # HP-UX 11.23 and IRIX 6.5 mkdir -m -p sets group- or\n                   # other-writable bit of parent directory when it shouldn't.\n                   # FreeBSD 6.1 mkdir -m -p sets mode of existing directory.\n                   ls_ld_tmpdir=`ls -ld \"$tmpdir\"`\n                   case $ls_ld_tmpdir in\n                     d????-?r-*) different_mode=700;;\n                     d????-?--*) different_mode=755;;\n                     *) false;;\n                   esac &&\n                   $mkdirprog -m$different_mode -p -- \"$tmpdir\" && {\n                     ls_ld_tmpdir_1=`ls -ld \"$tmpdir\"`\n                     test \"$ls_ld_tmpdir\" = \"$ls_ld_tmpdir_1\"\n                   }\n                 }\n              then posix_mkdir=:\n              fi\n              rmdir \"$tmpdir/d\" \"$tmpdir\"\n            else\n              # Remove any dirs left behind by ancient mkdir implementations.\n              rmdir ./$mkdir_mode ./-p ./-- 2>/dev/null\n            fi\n            trap '' 0;;\n        esac;;\n    esac\n\n    if\n      $posix_mkdir && (\n        umask $mkdir_umask &&\n        $doit_exec $mkdirprog $mkdir_mode -p -- \"$dstdir\"\n      )\n    then :\n    else\n\n      # The umask is ridiculous, or mkdir does not conform to POSIX,\n      # or it failed possibly due to a race condition.  Create the\n      # directory the slow way, step by step, checking for races as we go.\n\n      case $dstdir in\n        /*) prefix='/';;\n        [-=\\(\\)!]*) prefix='./';;\n        *)  prefix='';;\n      esac\n\n      oIFS=$IFS\n      IFS=/\n      set -f\n      set fnord $dstdir\n      shift\n      set +f\n      IFS=$oIFS\n\n      prefixes=\n\n      for d\n      do\n        test X\"$d\" = X && continue\n\n        prefix=$prefix$d\n        if test -d \"$prefix\"; then\n          prefixes=\n        else\n          if $posix_mkdir; then\n            (umask=$mkdir_umask &&\n             $doit_exec $mkdirprog $mkdir_mode -p -- \"$dstdir\") && break\n            # Don't fail if two instances are running concurrently.\n            test -d \"$prefix\" || exit 1\n          else\n            case $prefix in\n              *\\'*) qprefix=`echo \"$prefix\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"`;;\n              *) qprefix=$prefix;;\n            esac\n            prefixes=\"$prefixes '$qprefix'\"\n          fi\n        fi\n        prefix=$prefix/\n      done\n\n      if test -n \"$prefixes\"; then\n        # Don't fail if two instances are running concurrently.\n        (umask $mkdir_umask &&\n         eval \"\\$doit_exec \\$mkdirprog $prefixes\") ||\n          test -d \"$dstdir\" || exit 1\n        obsolete_mkdir_used=true\n      fi\n    fi\n  fi\n\n  if test -n \"$dir_arg\"; then\n    { test -z \"$chowncmd\" || $doit $chowncmd \"$dst\"; } &&\n    { test -z \"$chgrpcmd\" || $doit $chgrpcmd \"$dst\"; } &&\n    { test \"$obsolete_mkdir_used$chowncmd$chgrpcmd\" = false ||\n      test -z \"$chmodcmd\" || $doit $chmodcmd $mode \"$dst\"; } || exit 1\n  else\n\n    # Make a couple of temp file names in the proper directory.\n    dsttmp=$dstdir/_inst.$$_\n    rmtmp=$dstdir/_rm.$$_\n\n    # Trap to clean up those temp files at exit.\n    trap 'ret=$?; rm -f \"$dsttmp\" \"$rmtmp\" && exit $ret' 0\n\n    # Copy the file name to the temp name.\n    (umask $cp_umask && $doit_exec $cpprog \"$src\" \"$dsttmp\") &&\n\n    # and set any options; do chmod last to preserve setuid bits.\n    #\n    # If any of these fail, we abort the whole thing.  If we want to\n    # ignore errors from any of these, just make sure not to ignore\n    # errors from the above \"$doit $cpprog $src $dsttmp\" command.\n    #\n    { test -z \"$chowncmd\" || $doit $chowncmd \"$dsttmp\"; } &&\n    { test -z \"$chgrpcmd\" || $doit $chgrpcmd \"$dsttmp\"; } &&\n    { test -z \"$stripcmd\" || $doit $stripcmd \"$dsttmp\"; } &&\n    { test -z \"$chmodcmd\" || $doit $chmodcmd $mode \"$dsttmp\"; } &&\n\n    # If -C, don't bother to copy if it wouldn't change the file.\n    if $copy_on_change &&\n       old=`LC_ALL=C ls -dlL \"$dst\"     2>/dev/null` &&\n       new=`LC_ALL=C ls -dlL \"$dsttmp\"  2>/dev/null` &&\n       set -f &&\n       set X $old && old=:$2:$4:$5:$6 &&\n       set X $new && new=:$2:$4:$5:$6 &&\n       set +f &&\n       test \"$old\" = \"$new\" &&\n       $cmpprog \"$dst\" \"$dsttmp\" >/dev/null 2>&1\n    then\n      rm -f \"$dsttmp\"\n    else\n      # Rename the file to the real destination.\n      $doit $mvcmd -f \"$dsttmp\" \"$dst\" 2>/dev/null ||\n\n      # The rename failed, perhaps because mv can't rename something else\n      # to itself, or perhaps because mv is so ancient that it does not\n      # support -f.\n      {\n        # Now remove or move aside any old file at destination location.\n        # We try this two ways since rm can't unlink itself on some\n        # systems and the destination file might be busy for other\n        # reasons.  In this case, the final cleanup might fail but the new\n        # file should still install successfully.\n        {\n          test ! -f \"$dst\" ||\n          $doit $rmcmd -f \"$dst\" 2>/dev/null ||\n          { $doit $mvcmd -f \"$dst\" \"$rmtmp\" 2>/dev/null &&\n            { $doit $rmcmd -f \"$rmtmp\" 2>/dev/null; :; }\n          } ||\n          { echo \"$0: cannot unlink or rename $dst\" >&2\n            (exit 1); exit 1\n          }\n        } &&\n\n        # Now rename the file to the real destination.\n        $doit $mvcmd \"$dsttmp\" \"$dst\"\n      }\n    fi || exit 1\n\n    trap '' 0\n  fi\ndone\n\n# Local variables:\n# eval: (add-hook 'write-file-hooks 'time-stamp)\n# time-stamp-start: \"scriptversion=\"\n# time-stamp-format: \"%:y-%02m-%02d.%02H\"\n# time-stamp-time-zone: \"UTC0\"\n# time-stamp-end: \"; # UTC\"\n# End:\n"
  },
  {
    "path": "build-aux/missing",
    "content": "#! /bin/sh\n# Common wrapper for a few potentially missing GNU programs.\n\nscriptversion=2016-01-11.22; # UTC\n\n# Copyright (C) 1996-2017 Free Software Foundation, Inc.\n# Originally written by Fran,cois Pinard <pinard@iro.umontreal.ca>, 1996.\n\n# This program is free software; you can redistribute it and/or modify\n# it under the terms of the GNU General Public License as published by\n# the Free Software Foundation; either version 2, or (at your option)\n# any later version.\n\n# This program is distributed in the hope that it will be useful,\n# but WITHOUT ANY WARRANTY; without even the implied warranty of\n# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\n# GNU General Public License for more details.\n\n# You should have received a copy of the GNU General Public License\n# along with this program.  If not, see <http://www.gnu.org/licenses/>.\n\n# As a special exception to the GNU General Public License, if you\n# distribute this file as part of a program that contains a\n# configuration script generated by Autoconf, you may include it under\n# the same distribution terms that you use for the rest of that program.\n\nif test $# -eq 0; then\n  echo 1>&2 \"Try '$0 --help' for more information\"\n  exit 1\nfi\n\ncase $1 in\n\n  --is-lightweight)\n    # Used by our autoconf macros to check whether the available missing\n    # script is modern enough.\n    exit 0\n    ;;\n\n  --run)\n    # Back-compat with the calling convention used by older automake.\n    shift\n    ;;\n\n  -h|--h|--he|--hel|--help)\n    echo \"\\\n$0 [OPTION]... PROGRAM [ARGUMENT]...\n\nRun 'PROGRAM [ARGUMENT]...', returning a proper advice when this fails due\nto PROGRAM being missing or too old.\n\nOptions:\n  -h, --help      display this help and exit\n  -v, --version   output version information and exit\n\nSupported PROGRAM values:\n  aclocal   autoconf  autoheader   autom4te  automake  makeinfo\n  bison     yacc      flex         lex       help2man\n\nVersion suffixes to PROGRAM as well as the prefixes 'gnu-', 'gnu', and\n'g' are ignored when checking the name.\n\nSend bug reports to <bug-automake@gnu.org>.\"\n    exit $?\n    ;;\n\n  -v|--v|--ve|--ver|--vers|--versi|--versio|--version)\n    echo \"missing $scriptversion (GNU Automake)\"\n    exit $?\n    ;;\n\n  -*)\n    echo 1>&2 \"$0: unknown '$1' option\"\n    echo 1>&2 \"Try '$0 --help' for more information\"\n    exit 1\n    ;;\n\nesac\n\n# Run the given program, remember its exit status.\n\"$@\"; st=$?\n\n# If it succeeded, we are done.\ntest $st -eq 0 && exit 0\n\n# Also exit now if we it failed (or wasn't found), and '--version' was\n# passed; such an option is passed most likely to detect whether the\n# program is present and works.\ncase $2 in --version|--help) exit $st;; esac\n\n# Exit code 63 means version mismatch.  This often happens when the user\n# tries to use an ancient version of a tool on a file that requires a\n# minimum version.\nif test $st -eq 63; then\n  msg=\"probably too old\"\nelif test $st -eq 127; then\n  # Program was missing.\n  msg=\"missing on your system\"\nelse\n  # Program was found and executed, but failed.  Give up.\n  exit $st\nfi\n\nperl_URL=http://www.perl.org/\nflex_URL=http://flex.sourceforge.net/\ngnu_software_URL=http://www.gnu.org/software\n\nprogram_details ()\n{\n  case $1 in\n    aclocal|automake)\n      echo \"The '$1' program is part of the GNU Automake package:\"\n      echo \"<$gnu_software_URL/automake>\"\n      echo \"It also requires GNU Autoconf, GNU m4 and Perl in order to run:\"\n      echo \"<$gnu_software_URL/autoconf>\"\n      echo \"<$gnu_software_URL/m4/>\"\n      echo \"<$perl_URL>\"\n      ;;\n    autoconf|autom4te|autoheader)\n      echo \"The '$1' program is part of the GNU Autoconf package:\"\n      echo \"<$gnu_software_URL/autoconf/>\"\n      echo \"It also requires GNU m4 and Perl in order to run:\"\n      echo \"<$gnu_software_URL/m4/>\"\n      echo \"<$perl_URL>\"\n      ;;\n  esac\n}\n\ngive_advice ()\n{\n  # Normalize program name to check for.\n  normalized_program=`echo \"$1\" | sed '\n    s/^gnu-//; t\n    s/^gnu//; t\n    s/^g//; t'`\n\n  printf '%s\\n' \"'$1' is $msg.\"\n\n  configure_deps=\"'configure.ac' or m4 files included by 'configure.ac'\"\n  case $normalized_program in\n    autoconf*)\n      echo \"You should only need it if you modified 'configure.ac',\"\n      echo \"or m4 files included by it.\"\n      program_details 'autoconf'\n      ;;\n    autoheader*)\n      echo \"You should only need it if you modified 'acconfig.h' or\"\n      echo \"$configure_deps.\"\n      program_details 'autoheader'\n      ;;\n    automake*)\n      echo \"You should only need it if you modified 'Makefile.am' or\"\n      echo \"$configure_deps.\"\n      program_details 'automake'\n      ;;\n    aclocal*)\n      echo \"You should only need it if you modified 'acinclude.m4' or\"\n      echo \"$configure_deps.\"\n      program_details 'aclocal'\n      ;;\n   autom4te*)\n      echo \"You might have modified some maintainer files that require\"\n      echo \"the 'autom4te' program to be rebuilt.\"\n      program_details 'autom4te'\n      ;;\n    bison*|yacc*)\n      echo \"You should only need it if you modified a '.y' file.\"\n      echo \"You may want to install the GNU Bison package:\"\n      echo \"<$gnu_software_URL/bison/>\"\n      ;;\n    lex*|flex*)\n      echo \"You should only need it if you modified a '.l' file.\"\n      echo \"You may want to install the Fast Lexical Analyzer package:\"\n      echo \"<$flex_URL>\"\n      ;;\n    help2man*)\n      echo \"You should only need it if you modified a dependency\" \\\n           \"of a man page.\"\n      echo \"You may want to install the GNU Help2man package:\"\n      echo \"<$gnu_software_URL/help2man/>\"\n    ;;\n    makeinfo*)\n      echo \"You should only need it if you modified a '.texi' file, or\"\n      echo \"any other file indirectly affecting the aspect of the manual.\"\n      echo \"You might want to install the Texinfo package:\"\n      echo \"<$gnu_software_URL/texinfo/>\"\n      echo \"The spurious makeinfo call might also be the consequence of\"\n      echo \"using a buggy 'make' (AIX, DU, IRIX), in which case you might\"\n      echo \"want to install GNU make:\"\n      echo \"<$gnu_software_URL/make/>\"\n      ;;\n    *)\n      echo \"You might have modified some files without having the proper\"\n      echo \"tools for further handling them.  Check the 'README' file, it\"\n      echo \"often tells you about the needed prerequisites for installing\"\n      echo \"this package.  You may also peek at any GNU archive site, in\"\n      echo \"case some other package contains this missing '$1' program.\"\n      ;;\n  esac\n}\n\ngive_advice \"$1\" | sed -e '1s/^/WARNING: /' \\\n                       -e '2,$s/^/         /' >&2\n\n# Propagate the correct exit status (expected to be 127 for a program\n# not found, 63 for a program that failed due to version mismatch).\nexit $st\n\n# Local variables:\n# eval: (add-hook 'write-file-hooks 'time-stamp)\n# time-stamp-start: \"scriptversion=\"\n# time-stamp-format: \"%:y-%02m-%02d.%02H\"\n# time-stamp-time-zone: \"UTC0\"\n# time-stamp-end: \"; # UTC\"\n# End:\n"
  },
  {
    "path": "build.bat",
    "content": "@echo off\r\nSETLOCAL\r\n\r\nfor %%x in (%cmdcmdline%) do if %%~x==/c set DOUBLECLICKED=1\r\n\r\n:: set path to erlang installation\r\ncall \"%~dp0\"\\bin\\find_erlang.bat\r\n\r\n::replace @EMAKEFILEDEFINES@ from Emakefile.in and write Emakefile\r\n::(this is what autoconf on *nix would do)\r\n:: depending on your config, you might need to add one or more of the following options:\r\n::  {d, enable_debug}\r\n::  {d, have_ctline_support}\r\n::  {d, have_callback_support}\r\n::  {d, with_crypto_hash}\r\n::  {d, with_rand}\r\n::  {d, with_maps}\r\n:: refer to configure.ac for the appropriate checks for necessity\r\n::if yaws should use the file:sendfile/5 functionality, set @YAWS_OPTIONS@ to\r\n::  {d, 'HAVE_ERLANG_SENDFILE'}\r\n::similarly for yaws, add to @YAWS_OPTIONS@ if required:\r\n::  {d, 'HAVE_CRYPTO_HASH'}\r\n::  {d, 'HAVE_ERLANG_NOW'}\r\n::but note the issues in R15 & R16 mentioned at http://erlang.org/pipermail/erlang-questions/2013-October/075676.html\r\n:: Note: Search & Replace functionality from http://www.dostips.com\r\nif not exist Emakefile (\r\n    echo Creating Emakefile...\r\n    for /f \"tokens=1,* delims=&&&\" %%a in (Emakefile.in) do (\r\n        set \"line=%%a\"\r\n        if defined line (\r\n            call set \"line=echo.%%line:  @EMAKEFILEDEFINES@=%%\"\r\n            call set \"line=%%line:  @YAWS_OPTIONS@=%%\"\r\n            for /f \"delims=\" %%X in ('\"echo.\"%%line%%\"\"') do %%~X >> Emakefile\r\n        ) ELSE echo.\r\n    )\r\n    echo ...done!\r\n)\r\nif not exist include/rt.hrl (\r\n    echo Creating include/rt.hrl using rt_chord...\r\n    echo -include^(\"rt_chord.hrl\"^).>> include/rt.hrl\r\n)\r\n\r\n@echo on\r\npushd \"%~dp0\"\"\\contrib\\log4erl\\src\"\r\n%ERLANG_HOME%\\bin\\erlc.exe log4erl_parser.yrl\r\n%ERLANG_HOME%\\bin\\erl.exe -noinput -noshell -s leex file log4erl_lex.xrl -s init stop\r\npopd\r\n%ERLANG_HOME%\\bin\\erl.exe -pa contrib\\yaws -pa ebin -noinput +B -eval \"case make:all() of up_to_date -> halt(0); error -> halt(1) end.\"\r\n@echo off\r\n\r\nif defined DOUBLECLICKED PAUSE\r\n"
  },
  {
    "path": "configure",
    "content": "#! /bin/sh\n# Guess values for system-dependent variables and create Makefiles.\n# Generated by GNU Autoconf 2.69 for scalaris 0.9.0+git.\n#\n# Report bugs to <scalaris@googlegroups.com>.\n#\n#\n# Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc.\n#\n#\n# This configure script is free software; the Free Software Foundation\n# gives unlimited permission to copy, distribute and modify it.\n## -------------------- ##\n## M4sh Initialization. ##\n## -------------------- ##\n\n# Be more Bourne compatible\nDUALCASE=1; export DUALCASE # for MKS sh\nif test -n \"${ZSH_VERSION+set}\" && (emulate sh) >/dev/null 2>&1; then :\n  emulate sh\n  NULLCMD=:\n  # Pre-4.2 versions of Zsh do word splitting on ${1+\"$@\"}, which\n  # is contrary to our usage.  Disable this feature.\n  alias -g '${1+\"$@\"}'='\"$@\"'\n  setopt NO_GLOB_SUBST\nelse\n  case `(set -o) 2>/dev/null` in #(\n  *posix*) :\n    set -o posix ;; #(\n  *) :\n     ;;\nesac\nfi\n\n\nas_nl='\n'\nexport as_nl\n# Printing a long string crashes Solaris 7 /usr/bin/printf.\nas_echo='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'\nas_echo=$as_echo$as_echo$as_echo$as_echo$as_echo\nas_echo=$as_echo$as_echo$as_echo$as_echo$as_echo$as_echo\n# Prefer a ksh shell builtin over an external printf program on Solaris,\n# but without wasting forks for bash or zsh.\nif test -z \"$BASH_VERSION$ZSH_VERSION\" \\\n    && (test \"X`print -r -- $as_echo`\" = \"X$as_echo\") 2>/dev/null; then\n  as_echo='print -r --'\n  as_echo_n='print -rn --'\nelif (test \"X`printf %s $as_echo`\" = \"X$as_echo\") 2>/dev/null; then\n  as_echo='printf %s\\n'\n  as_echo_n='printf %s'\nelse\n  if test \"X`(/usr/ucb/echo -n -n $as_echo) 2>/dev/null`\" = \"X-n $as_echo\"; then\n    as_echo_body='eval /usr/ucb/echo -n \"$1$as_nl\"'\n    as_echo_n='/usr/ucb/echo -n'\n  else\n    as_echo_body='eval expr \"X$1\" : \"X\\\\(.*\\\\)\"'\n    as_echo_n_body='eval\n      arg=$1;\n      case $arg in #(\n      *\"$as_nl\"*)\n\texpr \"X$arg\" : \"X\\\\(.*\\\\)$as_nl\";\n\targ=`expr \"X$arg\" : \".*$as_nl\\\\(.*\\\\)\"`;;\n      esac;\n      expr \"X$arg\" : \"X\\\\(.*\\\\)\" | tr -d \"$as_nl\"\n    '\n    export as_echo_n_body\n    as_echo_n='sh -c $as_echo_n_body as_echo'\n  fi\n  export as_echo_body\n  as_echo='sh -c $as_echo_body as_echo'\nfi\n\n# The user is always right.\nif test \"${PATH_SEPARATOR+set}\" != set; then\n  PATH_SEPARATOR=:\n  (PATH='/bin;/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 && {\n    (PATH='/bin:/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 ||\n      PATH_SEPARATOR=';'\n  }\nfi\n\n\n# IFS\n# We need space, tab and new line, in precisely that order.  Quoting is\n# there to prevent editors from complaining about space-tab.\n# (If _AS_PATH_WALK were called with IFS unset, it would disable word\n# splitting by setting IFS to empty value.)\nIFS=\" \"\"\t$as_nl\"\n\n# Find who we are.  Look in the path if we contain no directory separator.\nas_myself=\ncase $0 in #((\n  *[\\\\/]* ) as_myself=$0 ;;\n  *) as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    test -r \"$as_dir/$0\" && as_myself=$as_dir/$0 && break\n  done\nIFS=$as_save_IFS\n\n     ;;\nesac\n# We did not find ourselves, most probably we were run as `sh COMMAND'\n# in which case we are not to be found in the path.\nif test \"x$as_myself\" = x; then\n  as_myself=$0\nfi\nif test ! -f \"$as_myself\"; then\n  $as_echo \"$as_myself: error: cannot find myself; rerun with an absolute file name\" >&2\n  exit 1\nfi\n\n# Unset variables that we do not need and which cause bugs (e.g. in\n# pre-3.0 UWIN ksh).  But do not cause bugs in bash 2.01; the \"|| exit 1\"\n# suppresses any \"Segmentation fault\" message there.  '((' could\n# trigger a bug in pdksh 5.2.14.\nfor as_var in BASH_ENV ENV MAIL MAILPATH\ndo eval test x\\${$as_var+set} = xset \\\n  && ( (unset $as_var) || exit 1) >/dev/null 2>&1 && unset $as_var || :\ndone\nPS1='$ '\nPS2='> '\nPS4='+ '\n\n# NLS nuisances.\nLC_ALL=C\nexport LC_ALL\nLANGUAGE=C\nexport LANGUAGE\n\n# CDPATH.\n(unset CDPATH) >/dev/null 2>&1 && unset CDPATH\n\n# Use a proper internal environment variable to ensure we don't fall\n  # into an infinite loop, continuously re-executing ourselves.\n  if test x\"${_as_can_reexec}\" != xno && test \"x$CONFIG_SHELL\" != x; then\n    _as_can_reexec=no; export _as_can_reexec;\n    # We cannot yet assume a decent shell, so we have to provide a\n# neutralization value for shells without unset; and this also\n# works around shells that cannot unset nonexistent variables.\n# Preserve -v and -x to the replacement shell.\nBASH_ENV=/dev/null\nENV=/dev/null\n(unset BASH_ENV) >/dev/null 2>&1 && unset BASH_ENV ENV\ncase $- in # ((((\n  *v*x* | *x*v* ) as_opts=-vx ;;\n  *v* ) as_opts=-v ;;\n  *x* ) as_opts=-x ;;\n  * ) as_opts= ;;\nesac\nexec $CONFIG_SHELL $as_opts \"$as_myself\" ${1+\"$@\"}\n# Admittedly, this is quite paranoid, since all the known shells bail\n# out after a failed `exec'.\n$as_echo \"$0: could not re-execute with $CONFIG_SHELL\" >&2\nas_fn_exit 255\n  fi\n  # We don't want this to propagate to other subprocesses.\n          { _as_can_reexec=; unset _as_can_reexec;}\nif test \"x$CONFIG_SHELL\" = x; then\n  as_bourne_compatible=\"if test -n \\\"\\${ZSH_VERSION+set}\\\" && (emulate sh) >/dev/null 2>&1; then :\n  emulate sh\n  NULLCMD=:\n  # Pre-4.2 versions of Zsh do word splitting on \\${1+\\\"\\$@\\\"}, which\n  # is contrary to our usage.  Disable this feature.\n  alias -g '\\${1+\\\"\\$@\\\"}'='\\\"\\$@\\\"'\n  setopt NO_GLOB_SUBST\nelse\n  case \\`(set -o) 2>/dev/null\\` in #(\n  *posix*) :\n    set -o posix ;; #(\n  *) :\n     ;;\nesac\nfi\n\"\n  as_required=\"as_fn_return () { (exit \\$1); }\nas_fn_success () { as_fn_return 0; }\nas_fn_failure () { as_fn_return 1; }\nas_fn_ret_success () { return 0; }\nas_fn_ret_failure () { return 1; }\n\nexitcode=0\nas_fn_success || { exitcode=1; echo as_fn_success failed.; }\nas_fn_failure && { exitcode=1; echo as_fn_failure succeeded.; }\nas_fn_ret_success || { exitcode=1; echo as_fn_ret_success failed.; }\nas_fn_ret_failure && { exitcode=1; echo as_fn_ret_failure succeeded.; }\nif ( set x; as_fn_ret_success y && test x = \\\"\\$1\\\" ); then :\n\nelse\n  exitcode=1; echo positional parameters were not saved.\nfi\ntest x\\$exitcode = x0 || exit 1\ntest -x / || exit 1\"\n  as_suggested=\"  as_lineno_1=\";as_suggested=$as_suggested$LINENO;as_suggested=$as_suggested\" as_lineno_1a=\\$LINENO\n  as_lineno_2=\";as_suggested=$as_suggested$LINENO;as_suggested=$as_suggested\" as_lineno_2a=\\$LINENO\n  eval 'test \\\"x\\$as_lineno_1'\\$as_run'\\\" != \\\"x\\$as_lineno_2'\\$as_run'\\\" &&\n  test \\\"x\\`expr \\$as_lineno_1'\\$as_run' + 1\\`\\\" = \\\"x\\$as_lineno_2'\\$as_run'\\\"' || exit 1\ntest \\$(( 1 + 1 )) = 2 || exit 1\"\n  if (eval \"$as_required\") 2>/dev/null; then :\n  as_have_required=yes\nelse\n  as_have_required=no\nfi\n  if test x$as_have_required = xyes && (eval \"$as_suggested\") 2>/dev/null; then :\n\nelse\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nas_found=false\nfor as_dir in /bin$PATH_SEPARATOR/usr/bin$PATH_SEPARATOR$PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n  as_found=:\n  case $as_dir in #(\n\t /*)\n\t   for as_base in sh bash ksh sh5; do\n\t     # Try only shells that exist, to save several forks.\n\t     as_shell=$as_dir/$as_base\n\t     if { test -f \"$as_shell\" || test -f \"$as_shell.exe\"; } &&\n\t\t    { $as_echo \"$as_bourne_compatible\"\"$as_required\" | as_run=a \"$as_shell\"; } 2>/dev/null; then :\n  CONFIG_SHELL=$as_shell as_have_required=yes\n\t\t   if { $as_echo \"$as_bourne_compatible\"\"$as_suggested\" | as_run=a \"$as_shell\"; } 2>/dev/null; then :\n  break 2\nfi\nfi\n\t   done;;\n       esac\n  as_found=false\ndone\n$as_found || { if { test -f \"$SHELL\" || test -f \"$SHELL.exe\"; } &&\n\t      { $as_echo \"$as_bourne_compatible\"\"$as_required\" | as_run=a \"$SHELL\"; } 2>/dev/null; then :\n  CONFIG_SHELL=$SHELL as_have_required=yes\nfi; }\nIFS=$as_save_IFS\n\n\n      if test \"x$CONFIG_SHELL\" != x; then :\n  export CONFIG_SHELL\n             # We cannot yet assume a decent shell, so we have to provide a\n# neutralization value for shells without unset; and this also\n# works around shells that cannot unset nonexistent variables.\n# Preserve -v and -x to the replacement shell.\nBASH_ENV=/dev/null\nENV=/dev/null\n(unset BASH_ENV) >/dev/null 2>&1 && unset BASH_ENV ENV\ncase $- in # ((((\n  *v*x* | *x*v* ) as_opts=-vx ;;\n  *v* ) as_opts=-v ;;\n  *x* ) as_opts=-x ;;\n  * ) as_opts= ;;\nesac\nexec $CONFIG_SHELL $as_opts \"$as_myself\" ${1+\"$@\"}\n# Admittedly, this is quite paranoid, since all the known shells bail\n# out after a failed `exec'.\n$as_echo \"$0: could not re-execute with $CONFIG_SHELL\" >&2\nexit 255\nfi\n\n    if test x$as_have_required = xno; then :\n  $as_echo \"$0: This script requires a shell more modern than all\"\n  $as_echo \"$0: the shells that I found on your system.\"\n  if test x${ZSH_VERSION+set} = xset ; then\n    $as_echo \"$0: In particular, zsh $ZSH_VERSION has bugs and should\"\n    $as_echo \"$0: be upgraded to zsh 4.3.4 or later.\"\n  else\n    $as_echo \"$0: Please tell bug-autoconf@gnu.org and\n$0: scalaris@googlegroups.com about your system, including\n$0: any error possibly output before this message. Then\n$0: install a modern shell, or manually run the script\n$0: under such a shell if you do have one.\"\n  fi\n  exit 1\nfi\nfi\nfi\nSHELL=${CONFIG_SHELL-/bin/sh}\nexport SHELL\n# Unset more variables known to interfere with behavior of common tools.\nCLICOLOR_FORCE= GREP_OPTIONS=\nunset CLICOLOR_FORCE GREP_OPTIONS\n\n## --------------------- ##\n## M4sh Shell Functions. ##\n## --------------------- ##\n# as_fn_unset VAR\n# ---------------\n# Portably unset VAR.\nas_fn_unset ()\n{\n  { eval $1=; unset $1;}\n}\nas_unset=as_fn_unset\n\n# as_fn_set_status STATUS\n# -----------------------\n# Set $? to STATUS, without forking.\nas_fn_set_status ()\n{\n  return $1\n} # as_fn_set_status\n\n# as_fn_exit STATUS\n# -----------------\n# Exit the shell with STATUS, even in a \"trap 0\" or \"set -e\" context.\nas_fn_exit ()\n{\n  set +e\n  as_fn_set_status $1\n  exit $1\n} # as_fn_exit\n\n# as_fn_mkdir_p\n# -------------\n# Create \"$as_dir\" as a directory, including parents if necessary.\nas_fn_mkdir_p ()\n{\n\n  case $as_dir in #(\n  -*) as_dir=./$as_dir;;\n  esac\n  test -d \"$as_dir\" || eval $as_mkdir_p || {\n    as_dirs=\n    while :; do\n      case $as_dir in #(\n      *\\'*) as_qdir=`$as_echo \"$as_dir\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"`;; #'(\n      *) as_qdir=$as_dir;;\n      esac\n      as_dirs=\"'$as_qdir' $as_dirs\"\n      as_dir=`$as_dirname -- \"$as_dir\" ||\n$as_expr X\"$as_dir\" : 'X\\(.*[^/]\\)//*[^/][^/]*/*$' \\| \\\n\t X\"$as_dir\" : 'X\\(//\\)[^/]' \\| \\\n\t X\"$as_dir\" : 'X\\(//\\)$' \\| \\\n\t X\"$as_dir\" : 'X\\(/\\)' \\| . 2>/dev/null ||\n$as_echo X\"$as_dir\" |\n    sed '/^X\\(.*[^/]\\)\\/\\/*[^/][^/]*\\/*$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\/\\)[^/].*/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\/\\)$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\).*/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  s/.*/./; q'`\n      test -d \"$as_dir\" && break\n    done\n    test -z \"$as_dirs\" || eval \"mkdir $as_dirs\"\n  } || test -d \"$as_dir\" || as_fn_error $? \"cannot create directory $as_dir\"\n\n\n} # as_fn_mkdir_p\n\n# as_fn_executable_p FILE\n# -----------------------\n# Test if FILE is an executable regular file.\nas_fn_executable_p ()\n{\n  test -f \"$1\" && test -x \"$1\"\n} # as_fn_executable_p\n# as_fn_append VAR VALUE\n# ----------------------\n# Append the text in VALUE to the end of the definition contained in VAR. Take\n# advantage of any shell optimizations that allow amortized linear growth over\n# repeated appends, instead of the typical quadratic growth present in naive\n# implementations.\nif (eval \"as_var=1; as_var+=2; test x\\$as_var = x12\") 2>/dev/null; then :\n  eval 'as_fn_append ()\n  {\n    eval $1+=\\$2\n  }'\nelse\n  as_fn_append ()\n  {\n    eval $1=\\$$1\\$2\n  }\nfi # as_fn_append\n\n# as_fn_arith ARG...\n# ------------------\n# Perform arithmetic evaluation on the ARGs, and store the result in the\n# global $as_val. Take advantage of shells that can avoid forks. The arguments\n# must be portable across $(()) and expr.\nif (eval \"test \\$(( 1 + 1 )) = 2\") 2>/dev/null; then :\n  eval 'as_fn_arith ()\n  {\n    as_val=$(( $* ))\n  }'\nelse\n  as_fn_arith ()\n  {\n    as_val=`expr \"$@\" || test $? -eq 1`\n  }\nfi # as_fn_arith\n\n\n# as_fn_error STATUS ERROR [LINENO LOG_FD]\n# ----------------------------------------\n# Output \"`basename $0`: error: ERROR\" to stderr. If LINENO and LOG_FD are\n# provided, also output the error to LOG_FD, referencing LINENO. Then exit the\n# script with STATUS, using 1 if that was 0.\nas_fn_error ()\n{\n  as_status=$1; test $as_status -eq 0 && as_status=1\n  if test \"$4\"; then\n    as_lineno=${as_lineno-\"$3\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n    $as_echo \"$as_me:${as_lineno-$LINENO}: error: $2\" >&$4\n  fi\n  $as_echo \"$as_me: error: $2\" >&2\n  as_fn_exit $as_status\n} # as_fn_error\n\nif expr a : '\\(a\\)' >/dev/null 2>&1 &&\n   test \"X`expr 00001 : '.*\\(...\\)'`\" = X001; then\n  as_expr=expr\nelse\n  as_expr=false\nfi\n\nif (basename -- /) >/dev/null 2>&1 && test \"X`basename -- / 2>&1`\" = \"X/\"; then\n  as_basename=basename\nelse\n  as_basename=false\nfi\n\nif (as_dir=`dirname -- /` && test \"X$as_dir\" = X/) >/dev/null 2>&1; then\n  as_dirname=dirname\nelse\n  as_dirname=false\nfi\n\nas_me=`$as_basename -- \"$0\" ||\n$as_expr X/\"$0\" : '.*/\\([^/][^/]*\\)/*$' \\| \\\n\t X\"$0\" : 'X\\(//\\)$' \\| \\\n\t X\"$0\" : 'X\\(/\\)' \\| . 2>/dev/null ||\n$as_echo X/\"$0\" |\n    sed '/^.*\\/\\([^/][^/]*\\)\\/*$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\/\\(\\/\\/\\)$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\/\\(\\/\\).*/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  s/.*/./; q'`\n\n# Avoid depending upon Character Ranges.\nas_cr_letters='abcdefghijklmnopqrstuvwxyz'\nas_cr_LETTERS='ABCDEFGHIJKLMNOPQRSTUVWXYZ'\nas_cr_Letters=$as_cr_letters$as_cr_LETTERS\nas_cr_digits='0123456789'\nas_cr_alnum=$as_cr_Letters$as_cr_digits\n\n\n  as_lineno_1=$LINENO as_lineno_1a=$LINENO\n  as_lineno_2=$LINENO as_lineno_2a=$LINENO\n  eval 'test \"x$as_lineno_1'$as_run'\" != \"x$as_lineno_2'$as_run'\" &&\n  test \"x`expr $as_lineno_1'$as_run' + 1`\" = \"x$as_lineno_2'$as_run'\"' || {\n  # Blame Lee E. McMahon (1931-1989) for sed's syntax.  :-)\n  sed -n '\n    p\n    /[$]LINENO/=\n  ' <$as_myself |\n    sed '\n      s/[$]LINENO.*/&-/\n      t lineno\n      b\n      :lineno\n      N\n      :loop\n      s/[$]LINENO\\([^'$as_cr_alnum'_].*\\n\\)\\(.*\\)/\\2\\1\\2/\n      t loop\n      s/-\\n.*//\n    ' >$as_me.lineno &&\n  chmod +x \"$as_me.lineno\" ||\n    { $as_echo \"$as_me: error: cannot create $as_me.lineno; rerun with a POSIX shell\" >&2; as_fn_exit 1; }\n\n  # If we had to re-execute with $CONFIG_SHELL, we're ensured to have\n  # already done that, so ensure we don't try to do so again and fall\n  # in an infinite loop.  This has already happened in practice.\n  _as_can_reexec=no; export _as_can_reexec\n  # Don't try to exec as it changes $[0], causing all sort of problems\n  # (the dirname of $[0] is not the place where we might find the\n  # original and so on.  Autoconf is especially sensitive to this).\n  . \"./$as_me.lineno\"\n  # Exit status is that of the last command.\n  exit\n}\n\nECHO_C= ECHO_N= ECHO_T=\ncase `echo -n x` in #(((((\n-n*)\n  case `echo 'xy\\c'` in\n  *c*) ECHO_T='\t';;\t# ECHO_T is single tab character.\n  xy)  ECHO_C='\\c';;\n  *)   echo `echo ksh88 bug on AIX 6.1` > /dev/null\n       ECHO_T='\t';;\n  esac;;\n*)\n  ECHO_N='-n';;\nesac\n\nrm -f conf$$ conf$$.exe conf$$.file\nif test -d conf$$.dir; then\n  rm -f conf$$.dir/conf$$.file\nelse\n  rm -f conf$$.dir\n  mkdir conf$$.dir 2>/dev/null\nfi\nif (echo >conf$$.file) 2>/dev/null; then\n  if ln -s conf$$.file conf$$ 2>/dev/null; then\n    as_ln_s='ln -s'\n    # ... but there are two gotchas:\n    # 1) On MSYS, both `ln -s file dir' and `ln file dir' fail.\n    # 2) DJGPP < 2.04 has no symlinks; `ln -s' creates a wrapper executable.\n    # In both cases, we have to default to `cp -pR'.\n    ln -s conf$$.file conf$$.dir 2>/dev/null && test ! -f conf$$.exe ||\n      as_ln_s='cp -pR'\n  elif ln conf$$.file conf$$ 2>/dev/null; then\n    as_ln_s=ln\n  else\n    as_ln_s='cp -pR'\n  fi\nelse\n  as_ln_s='cp -pR'\nfi\nrm -f conf$$ conf$$.exe conf$$.dir/conf$$.file conf$$.file\nrmdir conf$$.dir 2>/dev/null\n\nif mkdir -p . 2>/dev/null; then\n  as_mkdir_p='mkdir -p \"$as_dir\"'\nelse\n  test -d ./-p && rmdir ./-p\n  as_mkdir_p=false\nfi\n\nas_test_x='test -x'\nas_executable_p=as_fn_executable_p\n\n# Sed expression to map a string onto a valid CPP name.\nas_tr_cpp=\"eval sed 'y%*$as_cr_letters%P$as_cr_LETTERS%;s%[^_$as_cr_alnum]%_%g'\"\n\n# Sed expression to map a string onto a valid variable name.\nas_tr_sh=\"eval sed 'y%*+%pp%;s%[^_$as_cr_alnum]%_%g'\"\n\n\ntest -n \"$DJDIR\" || exec 7<&0 </dev/null\nexec 6>&1\n\n# Name of the host.\n# hostname on some systems (SVR3.2, old GNU/Linux) returns a bogus exit status,\n# so uname gets run too.\nac_hostname=`(hostname || uname -n) 2>/dev/null | sed 1q`\n\n#\n# Initializations.\n#\nac_default_prefix=/usr/local\nac_clean_files=\nac_config_libobj_dir=.\nLIBOBJS=\ncross_compiling=no\nsubdirs=\nMFLAGS=\nMAKEFLAGS=\n\n# Identity of this package.\nPACKAGE_NAME='scalaris'\nPACKAGE_TARNAME='scalaris'\nPACKAGE_VERSION='0.9.0+git'\nPACKAGE_STRING='scalaris 0.9.0+git'\nPACKAGE_BUGREPORT='scalaris@googlegroups.com'\nPACKAGE_URL=''\n\n# Factoring default headers for most tests.\nac_includes_default=\"\\\n#include <stdio.h>\n#ifdef HAVE_SYS_TYPES_H\n# include <sys/types.h>\n#endif\n#ifdef HAVE_SYS_STAT_H\n# include <sys/stat.h>\n#endif\n#ifdef STDC_HEADERS\n# include <stdlib.h>\n# include <stddef.h>\n#else\n# ifdef HAVE_STDLIB_H\n#  include <stdlib.h>\n# endif\n#endif\n#ifdef HAVE_STRING_H\n# if !defined STDC_HEADERS && defined HAVE_MEMORY_H\n#  include <memory.h>\n# endif\n# include <string.h>\n#endif\n#ifdef HAVE_STRINGS_H\n# include <strings.h>\n#endif\n#ifdef HAVE_INTTYPES_H\n# include <inttypes.h>\n#endif\n#ifdef HAVE_STDINT_H\n# include <stdint.h>\n#endif\n#ifdef HAVE_UNISTD_H\n# include <unistd.h>\n#endif\"\n\nac_subst_vars='am__EXEEXT_FALSE\nam__EXEEXT_TRUE\nLTLIBOBJS\nLIBOBJS\nARCHIVECMD\nNIFFLAGS\nDRIVER_OS\nEGREP\nGREP\nCPP\nENABLEPYTHON3INSTALL\nENABLEPYTHON3\nPYTHON3SITELIBDIR\nPYTHON3\nPYTHON3_2TO3\nENABLEPYTHON2INSTALL\nPYTHON2SITELIBDIR\nPYTHON2\nRUBYSITELIBDIR\nENABLERUBYINSTALL\nBUILDCLASSPATH\nJAVAFUNCTIONS\nRT\nYAWS_OPTIONS\nEMAKEFILECOMPILECOMPAT\nDIALYZER_FLAGS\nEDOCMACROS\nEMAKEFILEDEFINES\nERLANG_ERLANGJS_FLAGS\nERLANG_LIB_VER_erlang_js\nERLANG_LIB_DIR_erlang_js\nBITCASK_LIBS\nERLANG_BITCASK_FLAGS\nERLANG_LIB_VER_bitcask\nERLANG_LIB_DIR_bitcask\nERLANG_HANOIDB_FLAGS\nERLANG_LIB_VER_hanoidb\nERLANG_LIB_DIR_hanoidb\nERLANG_TOKE_FLAGS\nERLANG_LIB_VER_toke\nERLANG_LIB_DIR_toke\nRUN_TEST\nCT_RUN_FILE\nRUN_TEST_FILE\nERLANG_LIB_VER_edoc\nERLANG_LIB_DIR_edoc\nERLANG_LIB_VER_common_test\nERLANG_LIB_DIR_common_test\nERLANG_LIB_VER_xmerl\nERLANG_LIB_DIR_xmerl\nERLANG_LIB_VER_ssl\nERLANG_LIB_DIR_ssl\nERLANG_LIB_VER_inets\nERLANG_LIB_DIR_inets\nERLANG_LIB_VER_tools\nERLANG_LIB_DIR_tools\nERLANG_LIB_VER_os_mon\nERLANG_LIB_DIR_os_mon\nERLANG_LIB_VER_crypto\nERLANG_LIB_DIR_crypto\nERLANG_LIB_VER_compiler\nERLANG_LIB_DIR_compiler\nERLANG_LIB_VER_stdlib\nERLANG_LIB_DIR_stdlib\nERLANG_LIB_VER_kernel\nERLANG_LIB_DIR_kernel\nERLANG_LIB_VER_erts\nERLANG_LIB_DIR_erts\nEPMD\nERLANG_ERTS_VER\nERLANG_LIB_DIR\nERLANG_ROOT_DIR\nERL\nERLCFLAGS\nERLC\nSYSTEMD_UNITDIR\nINSTALL_INIT\nRUNUSER\nSUDO\nSCREEN\nBROWSER\nJSONFLAGS\nCXXSHELL\nCLANGTIDY\nCLANGFORMAT\nOPENSSL_LDFLAGS\nOPENSSL_LIBS\nOPENSSL_INCLUDES\nBOOST_UNIT_TEST_FRAMEWORK_LIB\nBOOST_PROGRAM_OPTIONS_LIB\nBOOST_ASIO_LIB\nBOOST_REGEX_LIB\nBOOST_SYSTEM_LIB\nam__fastdepCC_FALSE\nam__fastdepCC_TRUE\nCCDEPMODE\nac_ct_CC\nCFLAGS\nCC\nBOOST_LDFLAGS\nBOOST_CPPFLAGS\nhost_os\nhost_vendor\nhost_cpu\nhost\nbuild_os\nbuild_vendor\nbuild_cpu\nbuild\nHAVE_CXX14\nam__fastdepCXX_FALSE\nam__fastdepCXX_TRUE\nCXXDEPMODE\nam__nodep\nAMDEPBACKSLASH\nAMDEP_FALSE\nAMDEP_TRUE\nam__include\nDEPDIR\nOBJEXT\nEXEEXT\nac_ct_CXX\nCPPFLAGS\nLDFLAGS\nCXXFLAGS\nCXX\nLIBTOOL\nAR\nDOXYGEN\nSED\nTRUE\nFALSE\nAM_BACKSLASH\nAM_DEFAULT_VERBOSITY\nAM_DEFAULT_V\nAM_V\nam__untar\nam__tar\nAMTAR\nam__leading_dot\nSET_MAKE\nAWK\nmkdir_p\nMKDIR_P\nINSTALL_STRIP_PROGRAM\nSTRIP\ninstall_sh\nMAKEINFO\nAUTOHEADER\nAUTOMAKE\nAUTOCONF\nACLOCAL\nVERSION\nPACKAGE\nCYGPATH_W\nam__isrc\nINSTALL_DATA\nINSTALL_SCRIPT\nINSTALL_PROGRAM\ntarget_alias\nhost_alias\nbuild_alias\nLIBS\nECHO_T\nECHO_N\nECHO_C\nDEFS\nmandir\nlocaledir\nlibdir\npsdir\npdfdir\ndvidir\nhtmldir\ninfodir\ndocdir\noldincludedir\nincludedir\nlocalstatedir\nsharedstatedir\nsysconfdir\ndatadir\ndatarootdir\nlibexecdir\nsbindir\nbindir\nprogram_transform_name\nprefix\nexec_prefix\nPACKAGE_URL\nPACKAGE_BUGREPORT\nPACKAGE_STRING\nPACKAGE_VERSION\nPACKAGE_TARNAME\nPACKAGE_NAME\nPATH_SEPARATOR\nSHELL\nam__quote'\nac_subst_files=''\nac_user_opts='\nenable_option_checking\nenable_silent_rules\nenable_cpp\nenable_dependency_tracking\nwith_boost\nwith_boost_libdir\nwith_boost_system\nwith_boost_regex\nwith_boost_asio\nwith_boost_program_options\nwith_boost_unit_test_framework\nwith_openssl\nenable_toke\nenable_hanoidb\nenable_bitcask\nenable_erlang_js\nwith_browser\nenable_runuser\nwith_systemd\nenable_native\nenable_debug\nenable_txnew\nwith_rt\nwith_java_functions\nwith_build_classpath\nwith_ruby_sitelibdir\n'\n      ac_precious_vars='build_alias\nhost_alias\ntarget_alias\nCXX\nCXXFLAGS\nLDFLAGS\nLIBS\nCPPFLAGS\nCCC\nCC\nCFLAGS\nERLC\nERLCFLAGS\nERL\nCPP'\n\n\n# Initialize some variables set by options.\nac_init_help=\nac_init_version=false\nac_unrecognized_opts=\nac_unrecognized_sep=\n# The variables have the same names as the options, with\n# dashes changed to underlines.\ncache_file=/dev/null\nexec_prefix=NONE\nno_create=\nno_recursion=\nprefix=NONE\nprogram_prefix=NONE\nprogram_suffix=NONE\nprogram_transform_name=s,x,x,\nsilent=\nsite=\nsrcdir=\nverbose=\nx_includes=NONE\nx_libraries=NONE\n\n# Installation directory options.\n# These are left unexpanded so users can \"make install exec_prefix=/foo\"\n# and all the variables that are supposed to be based on exec_prefix\n# by default will actually change.\n# Use braces instead of parens because sh, perl, etc. also accept them.\n# (The list follows the same order as the GNU Coding Standards.)\nbindir='${exec_prefix}/bin'\nsbindir='${exec_prefix}/sbin'\nlibexecdir='${exec_prefix}/libexec'\ndatarootdir='${prefix}/share'\ndatadir='${datarootdir}'\nsysconfdir='${prefix}/etc'\nsharedstatedir='${prefix}/com'\nlocalstatedir='${prefix}/var'\nincludedir='${prefix}/include'\noldincludedir='/usr/include'\ndocdir='${datarootdir}/doc/${PACKAGE_TARNAME}'\ninfodir='${datarootdir}/info'\nhtmldir='${docdir}'\ndvidir='${docdir}'\npdfdir='${docdir}'\npsdir='${docdir}'\nlibdir='${exec_prefix}/lib'\nlocaledir='${datarootdir}/locale'\nmandir='${datarootdir}/man'\n\nac_prev=\nac_dashdash=\nfor ac_option\ndo\n  # If the previous option needs an argument, assign it.\n  if test -n \"$ac_prev\"; then\n    eval $ac_prev=\\$ac_option\n    ac_prev=\n    continue\n  fi\n\n  case $ac_option in\n  *=?*) ac_optarg=`expr \"X$ac_option\" : '[^=]*=\\(.*\\)'` ;;\n  *=)   ac_optarg= ;;\n  *)    ac_optarg=yes ;;\n  esac\n\n  # Accept the important Cygnus configure options, so we can diagnose typos.\n\n  case $ac_dashdash$ac_option in\n  --)\n    ac_dashdash=yes ;;\n\n  -bindir | --bindir | --bindi | --bind | --bin | --bi)\n    ac_prev=bindir ;;\n  -bindir=* | --bindir=* | --bindi=* | --bind=* | --bin=* | --bi=*)\n    bindir=$ac_optarg ;;\n\n  -build | --build | --buil | --bui | --bu)\n    ac_prev=build_alias ;;\n  -build=* | --build=* | --buil=* | --bui=* | --bu=*)\n    build_alias=$ac_optarg ;;\n\n  -cache-file | --cache-file | --cache-fil | --cache-fi \\\n  | --cache-f | --cache- | --cache | --cach | --cac | --ca | --c)\n    ac_prev=cache_file ;;\n  -cache-file=* | --cache-file=* | --cache-fil=* | --cache-fi=* \\\n  | --cache-f=* | --cache-=* | --cache=* | --cach=* | --cac=* | --ca=* | --c=*)\n    cache_file=$ac_optarg ;;\n\n  --config-cache | -C)\n    cache_file=config.cache ;;\n\n  -datadir | --datadir | --datadi | --datad)\n    ac_prev=datadir ;;\n  -datadir=* | --datadir=* | --datadi=* | --datad=*)\n    datadir=$ac_optarg ;;\n\n  -datarootdir | --datarootdir | --datarootdi | --datarootd | --dataroot \\\n  | --dataroo | --dataro | --datar)\n    ac_prev=datarootdir ;;\n  -datarootdir=* | --datarootdir=* | --datarootdi=* | --datarootd=* \\\n  | --dataroot=* | --dataroo=* | --dataro=* | --datar=*)\n    datarootdir=$ac_optarg ;;\n\n  -disable-* | --disable-*)\n    ac_useropt=`expr \"x$ac_option\" : 'x-*disable-\\(.*\\)'`\n    # Reject names that are not valid shell variable names.\n    expr \"x$ac_useropt\" : \".*[^-+._$as_cr_alnum]\" >/dev/null &&\n      as_fn_error $? \"invalid feature name: $ac_useropt\"\n    ac_useropt_orig=$ac_useropt\n    ac_useropt=`$as_echo \"$ac_useropt\" | sed 's/[-+.]/_/g'`\n    case $ac_user_opts in\n      *\"\n\"enable_$ac_useropt\"\n\"*) ;;\n      *) ac_unrecognized_opts=\"$ac_unrecognized_opts$ac_unrecognized_sep--disable-$ac_useropt_orig\"\n\t ac_unrecognized_sep=', ';;\n    esac\n    eval enable_$ac_useropt=no ;;\n\n  -docdir | --docdir | --docdi | --doc | --do)\n    ac_prev=docdir ;;\n  -docdir=* | --docdir=* | --docdi=* | --doc=* | --do=*)\n    docdir=$ac_optarg ;;\n\n  -dvidir | --dvidir | --dvidi | --dvid | --dvi | --dv)\n    ac_prev=dvidir ;;\n  -dvidir=* | --dvidir=* | --dvidi=* | --dvid=* | --dvi=* | --dv=*)\n    dvidir=$ac_optarg ;;\n\n  -enable-* | --enable-*)\n    ac_useropt=`expr \"x$ac_option\" : 'x-*enable-\\([^=]*\\)'`\n    # Reject names that are not valid shell variable names.\n    expr \"x$ac_useropt\" : \".*[^-+._$as_cr_alnum]\" >/dev/null &&\n      as_fn_error $? \"invalid feature name: $ac_useropt\"\n    ac_useropt_orig=$ac_useropt\n    ac_useropt=`$as_echo \"$ac_useropt\" | sed 's/[-+.]/_/g'`\n    case $ac_user_opts in\n      *\"\n\"enable_$ac_useropt\"\n\"*) ;;\n      *) ac_unrecognized_opts=\"$ac_unrecognized_opts$ac_unrecognized_sep--enable-$ac_useropt_orig\"\n\t ac_unrecognized_sep=', ';;\n    esac\n    eval enable_$ac_useropt=\\$ac_optarg ;;\n\n  -exec-prefix | --exec_prefix | --exec-prefix | --exec-prefi \\\n  | --exec-pref | --exec-pre | --exec-pr | --exec-p | --exec- \\\n  | --exec | --exe | --ex)\n    ac_prev=exec_prefix ;;\n  -exec-prefix=* | --exec_prefix=* | --exec-prefix=* | --exec-prefi=* \\\n  | --exec-pref=* | --exec-pre=* | --exec-pr=* | --exec-p=* | --exec-=* \\\n  | --exec=* | --exe=* | --ex=*)\n    exec_prefix=$ac_optarg ;;\n\n  -gas | --gas | --ga | --g)\n    # Obsolete; use --with-gas.\n    with_gas=yes ;;\n\n  -help | --help | --hel | --he | -h)\n    ac_init_help=long ;;\n  -help=r* | --help=r* | --hel=r* | --he=r* | -hr*)\n    ac_init_help=recursive ;;\n  -help=s* | --help=s* | --hel=s* | --he=s* | -hs*)\n    ac_init_help=short ;;\n\n  -host | --host | --hos | --ho)\n    ac_prev=host_alias ;;\n  -host=* | --host=* | --hos=* | --ho=*)\n    host_alias=$ac_optarg ;;\n\n  -htmldir | --htmldir | --htmldi | --htmld | --html | --htm | --ht)\n    ac_prev=htmldir ;;\n  -htmldir=* | --htmldir=* | --htmldi=* | --htmld=* | --html=* | --htm=* \\\n  | --ht=*)\n    htmldir=$ac_optarg ;;\n\n  -includedir | --includedir | --includedi | --included | --include \\\n  | --includ | --inclu | --incl | --inc)\n    ac_prev=includedir ;;\n  -includedir=* | --includedir=* | --includedi=* | --included=* | --include=* \\\n  | --includ=* | --inclu=* | --incl=* | --inc=*)\n    includedir=$ac_optarg ;;\n\n  -infodir | --infodir | --infodi | --infod | --info | --inf)\n    ac_prev=infodir ;;\n  -infodir=* | --infodir=* | --infodi=* | --infod=* | --info=* | --inf=*)\n    infodir=$ac_optarg ;;\n\n  -libdir | --libdir | --libdi | --libd)\n    ac_prev=libdir ;;\n  -libdir=* | --libdir=* | --libdi=* | --libd=*)\n    libdir=$ac_optarg ;;\n\n  -libexecdir | --libexecdir | --libexecdi | --libexecd | --libexec \\\n  | --libexe | --libex | --libe)\n    ac_prev=libexecdir ;;\n  -libexecdir=* | --libexecdir=* | --libexecdi=* | --libexecd=* | --libexec=* \\\n  | --libexe=* | --libex=* | --libe=*)\n    libexecdir=$ac_optarg ;;\n\n  -localedir | --localedir | --localedi | --localed | --locale)\n    ac_prev=localedir ;;\n  -localedir=* | --localedir=* | --localedi=* | --localed=* | --locale=*)\n    localedir=$ac_optarg ;;\n\n  -localstatedir | --localstatedir | --localstatedi | --localstated \\\n  | --localstate | --localstat | --localsta | --localst | --locals)\n    ac_prev=localstatedir ;;\n  -localstatedir=* | --localstatedir=* | --localstatedi=* | --localstated=* \\\n  | --localstate=* | --localstat=* | --localsta=* | --localst=* | --locals=*)\n    localstatedir=$ac_optarg ;;\n\n  -mandir | --mandir | --mandi | --mand | --man | --ma | --m)\n    ac_prev=mandir ;;\n  -mandir=* | --mandir=* | --mandi=* | --mand=* | --man=* | --ma=* | --m=*)\n    mandir=$ac_optarg ;;\n\n  -nfp | --nfp | --nf)\n    # Obsolete; use --without-fp.\n    with_fp=no ;;\n\n  -no-create | --no-create | --no-creat | --no-crea | --no-cre \\\n  | --no-cr | --no-c | -n)\n    no_create=yes ;;\n\n  -no-recursion | --no-recursion | --no-recursio | --no-recursi \\\n  | --no-recurs | --no-recur | --no-recu | --no-rec | --no-re | --no-r)\n    no_recursion=yes ;;\n\n  -oldincludedir | --oldincludedir | --oldincludedi | --oldincluded \\\n  | --oldinclude | --oldinclud | --oldinclu | --oldincl | --oldinc \\\n  | --oldin | --oldi | --old | --ol | --o)\n    ac_prev=oldincludedir ;;\n  -oldincludedir=* | --oldincludedir=* | --oldincludedi=* | --oldincluded=* \\\n  | --oldinclude=* | --oldinclud=* | --oldinclu=* | --oldincl=* | --oldinc=* \\\n  | --oldin=* | --oldi=* | --old=* | --ol=* | --o=*)\n    oldincludedir=$ac_optarg ;;\n\n  -prefix | --prefix | --prefi | --pref | --pre | --pr | --p)\n    ac_prev=prefix ;;\n  -prefix=* | --prefix=* | --prefi=* | --pref=* | --pre=* | --pr=* | --p=*)\n    prefix=$ac_optarg ;;\n\n  -program-prefix | --program-prefix | --program-prefi | --program-pref \\\n  | --program-pre | --program-pr | --program-p)\n    ac_prev=program_prefix ;;\n  -program-prefix=* | --program-prefix=* | --program-prefi=* \\\n  | --program-pref=* | --program-pre=* | --program-pr=* | --program-p=*)\n    program_prefix=$ac_optarg ;;\n\n  -program-suffix | --program-suffix | --program-suffi | --program-suff \\\n  | --program-suf | --program-su | --program-s)\n    ac_prev=program_suffix ;;\n  -program-suffix=* | --program-suffix=* | --program-suffi=* \\\n  | --program-suff=* | --program-suf=* | --program-su=* | --program-s=*)\n    program_suffix=$ac_optarg ;;\n\n  -program-transform-name | --program-transform-name \\\n  | --program-transform-nam | --program-transform-na \\\n  | --program-transform-n | --program-transform- \\\n  | --program-transform | --program-transfor \\\n  | --program-transfo | --program-transf \\\n  | --program-trans | --program-tran \\\n  | --progr-tra | --program-tr | --program-t)\n    ac_prev=program_transform_name ;;\n  -program-transform-name=* | --program-transform-name=* \\\n  | --program-transform-nam=* | --program-transform-na=* \\\n  | --program-transform-n=* | --program-transform-=* \\\n  | --program-transform=* | --program-transfor=* \\\n  | --program-transfo=* | --program-transf=* \\\n  | --program-trans=* | --program-tran=* \\\n  | --progr-tra=* | --program-tr=* | --program-t=*)\n    program_transform_name=$ac_optarg ;;\n\n  -pdfdir | --pdfdir | --pdfdi | --pdfd | --pdf | --pd)\n    ac_prev=pdfdir ;;\n  -pdfdir=* | --pdfdir=* | --pdfdi=* | --pdfd=* | --pdf=* | --pd=*)\n    pdfdir=$ac_optarg ;;\n\n  -psdir | --psdir | --psdi | --psd | --ps)\n    ac_prev=psdir ;;\n  -psdir=* | --psdir=* | --psdi=* | --psd=* | --ps=*)\n    psdir=$ac_optarg ;;\n\n  -q | -quiet | --quiet | --quie | --qui | --qu | --q \\\n  | -silent | --silent | --silen | --sile | --sil)\n    silent=yes ;;\n\n  -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb)\n    ac_prev=sbindir ;;\n  -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \\\n  | --sbi=* | --sb=*)\n    sbindir=$ac_optarg ;;\n\n  -sharedstatedir | --sharedstatedir | --sharedstatedi \\\n  | --sharedstated | --sharedstate | --sharedstat | --sharedsta \\\n  | --sharedst | --shareds | --shared | --share | --shar \\\n  | --sha | --sh)\n    ac_prev=sharedstatedir ;;\n  -sharedstatedir=* | --sharedstatedir=* | --sharedstatedi=* \\\n  | --sharedstated=* | --sharedstate=* | --sharedstat=* | --sharedsta=* \\\n  | --sharedst=* | --shareds=* | --shared=* | --share=* | --shar=* \\\n  | --sha=* | --sh=*)\n    sharedstatedir=$ac_optarg ;;\n\n  -site | --site | --sit)\n    ac_prev=site ;;\n  -site=* | --site=* | --sit=*)\n    site=$ac_optarg ;;\n\n  -srcdir | --srcdir | --srcdi | --srcd | --src | --sr)\n    ac_prev=srcdir ;;\n  -srcdir=* | --srcdir=* | --srcdi=* | --srcd=* | --src=* | --sr=*)\n    srcdir=$ac_optarg ;;\n\n  -sysconfdir | --sysconfdir | --sysconfdi | --sysconfd | --sysconf \\\n  | --syscon | --sysco | --sysc | --sys | --sy)\n    ac_prev=sysconfdir ;;\n  -sysconfdir=* | --sysconfdir=* | --sysconfdi=* | --sysconfd=* | --sysconf=* \\\n  | --syscon=* | --sysco=* | --sysc=* | --sys=* | --sy=*)\n    sysconfdir=$ac_optarg ;;\n\n  -target | --target | --targe | --targ | --tar | --ta | --t)\n    ac_prev=target_alias ;;\n  -target=* | --target=* | --targe=* | --targ=* | --tar=* | --ta=* | --t=*)\n    target_alias=$ac_optarg ;;\n\n  -v | -verbose | --verbose | --verbos | --verbo | --verb)\n    verbose=yes ;;\n\n  -version | --version | --versio | --versi | --vers | -V)\n    ac_init_version=: ;;\n\n  -with-* | --with-*)\n    ac_useropt=`expr \"x$ac_option\" : 'x-*with-\\([^=]*\\)'`\n    # Reject names that are not valid shell variable names.\n    expr \"x$ac_useropt\" : \".*[^-+._$as_cr_alnum]\" >/dev/null &&\n      as_fn_error $? \"invalid package name: $ac_useropt\"\n    ac_useropt_orig=$ac_useropt\n    ac_useropt=`$as_echo \"$ac_useropt\" | sed 's/[-+.]/_/g'`\n    case $ac_user_opts in\n      *\"\n\"with_$ac_useropt\"\n\"*) ;;\n      *) ac_unrecognized_opts=\"$ac_unrecognized_opts$ac_unrecognized_sep--with-$ac_useropt_orig\"\n\t ac_unrecognized_sep=', ';;\n    esac\n    eval with_$ac_useropt=\\$ac_optarg ;;\n\n  -without-* | --without-*)\n    ac_useropt=`expr \"x$ac_option\" : 'x-*without-\\(.*\\)'`\n    # Reject names that are not valid shell variable names.\n    expr \"x$ac_useropt\" : \".*[^-+._$as_cr_alnum]\" >/dev/null &&\n      as_fn_error $? \"invalid package name: $ac_useropt\"\n    ac_useropt_orig=$ac_useropt\n    ac_useropt=`$as_echo \"$ac_useropt\" | sed 's/[-+.]/_/g'`\n    case $ac_user_opts in\n      *\"\n\"with_$ac_useropt\"\n\"*) ;;\n      *) ac_unrecognized_opts=\"$ac_unrecognized_opts$ac_unrecognized_sep--without-$ac_useropt_orig\"\n\t ac_unrecognized_sep=', ';;\n    esac\n    eval with_$ac_useropt=no ;;\n\n  --x)\n    # Obsolete; use --with-x.\n    with_x=yes ;;\n\n  -x-includes | --x-includes | --x-include | --x-includ | --x-inclu \\\n  | --x-incl | --x-inc | --x-in | --x-i)\n    ac_prev=x_includes ;;\n  -x-includes=* | --x-includes=* | --x-include=* | --x-includ=* | --x-inclu=* \\\n  | --x-incl=* | --x-inc=* | --x-in=* | --x-i=*)\n    x_includes=$ac_optarg ;;\n\n  -x-libraries | --x-libraries | --x-librarie | --x-librari \\\n  | --x-librar | --x-libra | --x-libr | --x-lib | --x-li | --x-l)\n    ac_prev=x_libraries ;;\n  -x-libraries=* | --x-libraries=* | --x-librarie=* | --x-librari=* \\\n  | --x-librar=* | --x-libra=* | --x-libr=* | --x-lib=* | --x-li=* | --x-l=*)\n    x_libraries=$ac_optarg ;;\n\n  -*) as_fn_error $? \"unrecognized option: \\`$ac_option'\nTry \\`$0 --help' for more information\"\n    ;;\n\n  *=*)\n    ac_envvar=`expr \"x$ac_option\" : 'x\\([^=]*\\)='`\n    # Reject names that are not valid shell variable names.\n    case $ac_envvar in #(\n      '' | [0-9]* | *[!_$as_cr_alnum]* )\n      as_fn_error $? \"invalid variable name: \\`$ac_envvar'\" ;;\n    esac\n    eval $ac_envvar=\\$ac_optarg\n    export $ac_envvar ;;\n\n  *)\n    # FIXME: should be removed in autoconf 3.0.\n    $as_echo \"$as_me: WARNING: you should use --build, --host, --target\" >&2\n    expr \"x$ac_option\" : \".*[^-._$as_cr_alnum]\" >/dev/null &&\n      $as_echo \"$as_me: WARNING: invalid host type: $ac_option\" >&2\n    : \"${build_alias=$ac_option} ${host_alias=$ac_option} ${target_alias=$ac_option}\"\n    ;;\n\n  esac\ndone\n\nif test -n \"$ac_prev\"; then\n  ac_option=--`echo $ac_prev | sed 's/_/-/g'`\n  as_fn_error $? \"missing argument to $ac_option\"\nfi\n\nif test -n \"$ac_unrecognized_opts\"; then\n  case $enable_option_checking in\n    no) ;;\n    fatal) as_fn_error $? \"unrecognized options: $ac_unrecognized_opts\" ;;\n    *)     $as_echo \"$as_me: WARNING: unrecognized options: $ac_unrecognized_opts\" >&2 ;;\n  esac\nfi\n\n# Check all directory arguments for consistency.\nfor ac_var in\texec_prefix prefix bindir sbindir libexecdir datarootdir \\\n\t\tdatadir sysconfdir sharedstatedir localstatedir includedir \\\n\t\toldincludedir docdir infodir htmldir dvidir pdfdir psdir \\\n\t\tlibdir localedir mandir\ndo\n  eval ac_val=\\$$ac_var\n  # Remove trailing slashes.\n  case $ac_val in\n    */ )\n      ac_val=`expr \"X$ac_val\" : 'X\\(.*[^/]\\)' \\| \"X$ac_val\" : 'X\\(.*\\)'`\n      eval $ac_var=\\$ac_val;;\n  esac\n  # Be sure to have absolute directory names.\n  case $ac_val in\n    [\\\\/$]* | ?:[\\\\/]* )  continue;;\n    NONE | '' ) case $ac_var in *prefix ) continue;; esac;;\n  esac\n  as_fn_error $? \"expected an absolute directory name for --$ac_var: $ac_val\"\ndone\n\n# There might be people who depend on the old broken behavior: `$host'\n# used to hold the argument of --host etc.\n# FIXME: To remove some day.\nbuild=$build_alias\nhost=$host_alias\ntarget=$target_alias\n\n# FIXME: To remove some day.\nif test \"x$host_alias\" != x; then\n  if test \"x$build_alias\" = x; then\n    cross_compiling=maybe\n  elif test \"x$build_alias\" != \"x$host_alias\"; then\n    cross_compiling=yes\n  fi\nfi\n\nac_tool_prefix=\ntest -n \"$host_alias\" && ac_tool_prefix=$host_alias-\n\ntest \"$silent\" = yes && exec 6>/dev/null\n\n\nac_pwd=`pwd` && test -n \"$ac_pwd\" &&\nac_ls_di=`ls -di .` &&\nac_pwd_ls_di=`cd \"$ac_pwd\" && ls -di .` ||\n  as_fn_error $? \"working directory cannot be determined\"\ntest \"X$ac_ls_di\" = \"X$ac_pwd_ls_di\" ||\n  as_fn_error $? \"pwd does not report name of working directory\"\n\n\n# Find the source files, if location was not specified.\nif test -z \"$srcdir\"; then\n  ac_srcdir_defaulted=yes\n  # Try the directory containing this script, then the parent directory.\n  ac_confdir=`$as_dirname -- \"$as_myself\" ||\n$as_expr X\"$as_myself\" : 'X\\(.*[^/]\\)//*[^/][^/]*/*$' \\| \\\n\t X\"$as_myself\" : 'X\\(//\\)[^/]' \\| \\\n\t X\"$as_myself\" : 'X\\(//\\)$' \\| \\\n\t X\"$as_myself\" : 'X\\(/\\)' \\| . 2>/dev/null ||\n$as_echo X\"$as_myself\" |\n    sed '/^X\\(.*[^/]\\)\\/\\/*[^/][^/]*\\/*$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\/\\)[^/].*/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\/\\)$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\).*/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  s/.*/./; q'`\n  srcdir=$ac_confdir\n  if test ! -r \"$srcdir/$ac_unique_file\"; then\n    srcdir=..\n  fi\nelse\n  ac_srcdir_defaulted=no\nfi\nif test ! -r \"$srcdir/$ac_unique_file\"; then\n  test \"$ac_srcdir_defaulted\" = yes && srcdir=\"$ac_confdir or ..\"\n  as_fn_error $? \"cannot find sources ($ac_unique_file) in $srcdir\"\nfi\nac_msg=\"sources are in $srcdir, but \\`cd $srcdir' does not work\"\nac_abs_confdir=`(\n\tcd \"$srcdir\" && test -r \"./$ac_unique_file\" || as_fn_error $? \"$ac_msg\"\n\tpwd)`\n# When building in place, set srcdir=.\nif test \"$ac_abs_confdir\" = \"$ac_pwd\"; then\n  srcdir=.\nfi\n# Remove unnecessary trailing slashes from srcdir.\n# Double slashes in file names in object file debugging info\n# mess up M-x gdb in Emacs.\ncase $srcdir in\n*/) srcdir=`expr \"X$srcdir\" : 'X\\(.*[^/]\\)' \\| \"X$srcdir\" : 'X\\(.*\\)'`;;\nesac\nfor ac_var in $ac_precious_vars; do\n  eval ac_env_${ac_var}_set=\\${${ac_var}+set}\n  eval ac_env_${ac_var}_value=\\$${ac_var}\n  eval ac_cv_env_${ac_var}_set=\\${${ac_var}+set}\n  eval ac_cv_env_${ac_var}_value=\\$${ac_var}\ndone\n\n#\n# Report the --help message.\n#\nif test \"$ac_init_help\" = \"long\"; then\n  # Omit some internal or obsolete options to make the list less imposing.\n  # This message is too long to be a string in the A/UX 3.1 sh.\n  cat <<_ACEOF\n\\`configure' configures scalaris 0.9.0+git to adapt to many kinds of systems.\n\nUsage: $0 [OPTION]... [VAR=VALUE]...\n\nTo assign environment variables (e.g., CC, CFLAGS...), specify them as\nVAR=VALUE.  See below for descriptions of some of the useful variables.\n\nDefaults for the options are specified in brackets.\n\nConfiguration:\n  -h, --help              display this help and exit\n      --help=short        display options specific to this package\n      --help=recursive    display the short help of all the included packages\n  -V, --version           display version information and exit\n  -q, --quiet, --silent   do not print \\`checking ...' messages\n      --cache-file=FILE   cache test results in FILE [disabled]\n  -C, --config-cache      alias for \\`--cache-file=config.cache'\n  -n, --no-create         do not create output files\n      --srcdir=DIR        find the sources in DIR [configure dir or \\`..']\n\nInstallation directories:\n  --prefix=PREFIX         install architecture-independent files in PREFIX\n                          [$ac_default_prefix]\n  --exec-prefix=EPREFIX   install architecture-dependent files in EPREFIX\n                          [PREFIX]\n\nBy default, \\`make install' will install all the files in\n\\`$ac_default_prefix/bin', \\`$ac_default_prefix/lib' etc.  You can specify\nan installation prefix other than \\`$ac_default_prefix' using \\`--prefix',\nfor instance \\`--prefix=\\$HOME'.\n\nFor better control, use the options below.\n\nFine tuning of the installation directories:\n  --bindir=DIR            user executables [EPREFIX/bin]\n  --sbindir=DIR           system admin executables [EPREFIX/sbin]\n  --libexecdir=DIR        program executables [EPREFIX/libexec]\n  --sysconfdir=DIR        read-only single-machine data [PREFIX/etc]\n  --sharedstatedir=DIR    modifiable architecture-independent data [PREFIX/com]\n  --localstatedir=DIR     modifiable single-machine data [PREFIX/var]\n  --libdir=DIR            object code libraries [EPREFIX/lib]\n  --includedir=DIR        C header files [PREFIX/include]\n  --oldincludedir=DIR     C header files for non-gcc [/usr/include]\n  --datarootdir=DIR       read-only arch.-independent data root [PREFIX/share]\n  --datadir=DIR           read-only architecture-independent data [DATAROOTDIR]\n  --infodir=DIR           info documentation [DATAROOTDIR/info]\n  --localedir=DIR         locale-dependent data [DATAROOTDIR/locale]\n  --mandir=DIR            man documentation [DATAROOTDIR/man]\n  --docdir=DIR            documentation root [DATAROOTDIR/doc/scalaris]\n  --htmldir=DIR           html documentation [DOCDIR]\n  --dvidir=DIR            dvi documentation [DOCDIR]\n  --pdfdir=DIR            pdf documentation [DOCDIR]\n  --psdir=DIR             ps documentation [DOCDIR]\n_ACEOF\n\n  cat <<\\_ACEOF\n\nProgram names:\n  --program-prefix=PREFIX            prepend PREFIX to installed program names\n  --program-suffix=SUFFIX            append SUFFIX to installed program names\n  --program-transform-name=PROGRAM   run sed PROGRAM on installed program names\n\nSystem types:\n  --build=BUILD     configure for building on BUILD [guessed]\n  --host=HOST       cross-compile to build programs to run on HOST [BUILD]\n_ACEOF\nfi\n\nif test -n \"$ac_init_help\"; then\n  case $ac_init_help in\n     short | recursive ) echo \"Configuration of scalaris 0.9.0+git:\";;\n   esac\n  cat <<\\_ACEOF\n\nOptional Features:\n  --disable-option-checking  ignore unrecognized --enable/--with options\n  --disable-FEATURE       do not include FEATURE (same as --enable-FEATURE=no)\n  --enable-FEATURE[=ARG]  include FEATURE [ARG=yes]\n  --enable-silent-rules   less verbose build output (undo: \"make V=1\")\n  --disable-silent-rules  verbose build output (undo: \"make V=0\")\n  --disable-cpp           disable support for the C++ bindings\n  --enable-dependency-tracking\n                          do not reject slow dependency extractors\n  --disable-dependency-tracking\n                          speeds up one-time build\n  --enable-toke[=DIR]     enable support for tokyo cabinet through toke\n  --enable-hanoidb[=DIR]  enable support for hanoidb\n  --enable-bitcask[=DIR]  enable support for bitcask\n  --enable-erlang-js[=DIR]\n                          enable support for erlang_js\n  --disable-runuser       disable support for runuser (use sudo instead)\n  --enable-native         enable compilation to native code using HiPE\n  --enable-debug          enable debugging mode\n  --enable-txnew          enable new transaction protocol\n\nOptional Packages:\n  --with-PACKAGE[=ARG]    use PACKAGE [ARG=yes]\n  --without-PACKAGE       do not use PACKAGE (same as --with-PACKAGE=no)\n  --with-boost[=ARG]      use Boost library from a standard location\n                          (ARG=yes), from the specified location (ARG=<path>),\n                          or disable it (ARG=no) [ARG=yes]\n  --with-boost-libdir=LIB_DIR\n                          Force given directory for boost libraries. Note that\n                          this will override library path detection, so use\n                          this parameter only if default library detection\n                          fails and you know exactly where your boost\n                          libraries are located.\n  --with-boost-system[=special-lib]\n                          use the System library from boost - it is possible\n                          to specify a certain library for the linker e.g.\n                          --with-boost-system=boost_system-gcc-mt\n  --with-boost-regex[=special-lib]\n                          use the Regex library from boost - it is possible to\n                          specify a certain library for the linker e.g.\n                          --with-boost-regex=boost_regex-gcc-mt-d-1_33_1\n  --with-boost-asio[=special-lib]\n                          use the ASIO library from boost - it is possible to\n                          specify a certain library for the linker e.g.\n                          --with-boost-asio=boost_system-gcc41-mt-1_34\n  --with-boost-program-options[=special-lib]\n                          use the program options library from boost - it is\n                          possible to specify a certain library for the linker\n                          e.g.\n                          --with-boost-program-options=boost_program_options-gcc-mt-1_33_1\n  --with-boost-unit-test-framework[=special-lib]\n                          use the Unit_Test_Framework library from boost - it\n                          is possible to specify a certain library for the\n                          linker e.g.\n                          --with-boost-unit-test-framework=boost_unit_test_framework-gcc\n  --with-openssl=DIR      root of the OpenSSL directory\n  --with-browser=BROWSER  absolute path of the browser to use for 'make\n                          test-vts'\n  --with-systemd[=UNITDIR]\n                          use systemd, optionally specify the directory for\n                          systemd service files\n  --with-rt[=RTFILE]      optionally specify the routing table type\n                          (default=rt_chord)\n  --with-java-functions=JAVAFUNCTIONS\n                          use the given java-functions script for\n                          java-api/scalaris (default is\n                          /usr/share/java-utils/java-functions)\n  --with-build-classpath=BUILDCLASSPATH\n                          absolute path of the build-classpath script to use\n                          for java-api/scalaris\n  --with-ruby-sitelibdir=SITELIBDIR\n                          where to install ruby libraries\n\nSome influential environment variables:\n  CXX         C++ compiler command\n  CXXFLAGS    C++ compiler flags\n  LDFLAGS     linker flags, e.g. -L<lib dir> if you have libraries in a\n              nonstandard directory <lib dir>\n  LIBS        libraries to pass to the linker, e.g. -l<library>\n  CPPFLAGS    (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if\n              you have headers in a nonstandard directory <include dir>\n  CC          C compiler command\n  CFLAGS      C compiler flags\n  ERLC        Erlang/OTP compiler command [autodetected]\n  ERLCFLAGS   Erlang/OTP compiler flags [none]\n  ERL         Erlang/OTP interpreter command [autodetected]\n  CPP         C preprocessor\n\nUse these variables to override the choices made by `configure' or to help\nit to find libraries and programs with nonstandard names/locations.\n\nReport bugs to <scalaris@googlegroups.com>.\n_ACEOF\nac_status=$?\nfi\n\nif test \"$ac_init_help\" = \"recursive\"; then\n  # If there are subdirs, report their specific --help.\n  for ac_dir in : $ac_subdirs_all; do test \"x$ac_dir\" = x: && continue\n    test -d \"$ac_dir\" ||\n      { cd \"$srcdir\" && ac_pwd=`pwd` && srcdir=. && test -d \"$ac_dir\"; } ||\n      continue\n    ac_builddir=.\n\ncase \"$ac_dir\" in\n.) ac_dir_suffix= ac_top_builddir_sub=. ac_top_build_prefix= ;;\n*)\n  ac_dir_suffix=/`$as_echo \"$ac_dir\" | sed 's|^\\.[\\\\/]||'`\n  # A \"..\" for each directory in $ac_dir_suffix.\n  ac_top_builddir_sub=`$as_echo \"$ac_dir_suffix\" | sed 's|/[^\\\\/]*|/..|g;s|/||'`\n  case $ac_top_builddir_sub in\n  \"\") ac_top_builddir_sub=. ac_top_build_prefix= ;;\n  *)  ac_top_build_prefix=$ac_top_builddir_sub/ ;;\n  esac ;;\nesac\nac_abs_top_builddir=$ac_pwd\nac_abs_builddir=$ac_pwd$ac_dir_suffix\n# for backward compatibility:\nac_top_builddir=$ac_top_build_prefix\n\ncase $srcdir in\n  .)  # We are building in place.\n    ac_srcdir=.\n    ac_top_srcdir=$ac_top_builddir_sub\n    ac_abs_top_srcdir=$ac_pwd ;;\n  [\\\\/]* | ?:[\\\\/]* )  # Absolute name.\n    ac_srcdir=$srcdir$ac_dir_suffix;\n    ac_top_srcdir=$srcdir\n    ac_abs_top_srcdir=$srcdir ;;\n  *) # Relative name.\n    ac_srcdir=$ac_top_build_prefix$srcdir$ac_dir_suffix\n    ac_top_srcdir=$ac_top_build_prefix$srcdir\n    ac_abs_top_srcdir=$ac_pwd/$srcdir ;;\nesac\nac_abs_srcdir=$ac_abs_top_srcdir$ac_dir_suffix\n\n    cd \"$ac_dir\" || { ac_status=$?; continue; }\n    # Check for guested configure.\n    if test -f \"$ac_srcdir/configure.gnu\"; then\n      echo &&\n      $SHELL \"$ac_srcdir/configure.gnu\" --help=recursive\n    elif test -f \"$ac_srcdir/configure\"; then\n      echo &&\n      $SHELL \"$ac_srcdir/configure\" --help=recursive\n    else\n      $as_echo \"$as_me: WARNING: no configuration information is in $ac_dir\" >&2\n    fi || ac_status=$?\n    cd \"$ac_pwd\" || { ac_status=$?; break; }\n  done\nfi\n\ntest -n \"$ac_init_help\" && exit $ac_status\nif $ac_init_version; then\n  cat <<\\_ACEOF\nscalaris configure 0.9.0+git\ngenerated by GNU Autoconf 2.69\n\nCopyright (C) 2012 Free Software Foundation, Inc.\nThis configure script is free software; the Free Software Foundation\ngives unlimited permission to copy, distribute and modify it.\n_ACEOF\n  exit\nfi\n\n## ------------------------ ##\n## Autoconf initialization. ##\n## ------------------------ ##\n\n# ac_fn_cxx_try_compile LINENO\n# ----------------------------\n# Try to compile conftest.$ac_ext, and return whether this succeeded.\nac_fn_cxx_try_compile ()\n{\n  as_lineno=${as_lineno-\"$1\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n  rm -f conftest.$ac_objext\n  if { { ac_try=\"$ac_compile\"\ncase \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_compile\") 2>conftest.err\n  ac_status=$?\n  if test -s conftest.err; then\n    grep -v '^ *+' conftest.err >conftest.er1\n    cat conftest.er1 >&5\n    mv -f conftest.er1 conftest.err\n  fi\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; } && {\n\t test -z \"$ac_cxx_werror_flag\" ||\n\t test ! -s conftest.err\n       } && test -s conftest.$ac_objext; then :\n  ac_retval=0\nelse\n  $as_echo \"$as_me: failed program was:\" >&5\nsed 's/^/| /' conftest.$ac_ext >&5\n\n\tac_retval=1\nfi\n  eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno\n  as_fn_set_status $ac_retval\n\n} # ac_fn_cxx_try_compile\n\n# ac_fn_c_try_compile LINENO\n# --------------------------\n# Try to compile conftest.$ac_ext, and return whether this succeeded.\nac_fn_c_try_compile ()\n{\n  as_lineno=${as_lineno-\"$1\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n  rm -f conftest.$ac_objext\n  if { { ac_try=\"$ac_compile\"\ncase \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_compile\") 2>conftest.err\n  ac_status=$?\n  if test -s conftest.err; then\n    grep -v '^ *+' conftest.err >conftest.er1\n    cat conftest.er1 >&5\n    mv -f conftest.er1 conftest.err\n  fi\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; } && {\n\t test -z \"$ac_c_werror_flag\" ||\n\t test ! -s conftest.err\n       } && test -s conftest.$ac_objext; then :\n  ac_retval=0\nelse\n  $as_echo \"$as_me: failed program was:\" >&5\nsed 's/^/| /' conftest.$ac_ext >&5\n\n\tac_retval=1\nfi\n  eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno\n  as_fn_set_status $ac_retval\n\n} # ac_fn_c_try_compile\n\n# ac_fn_c_try_link LINENO\n# -----------------------\n# Try to link conftest.$ac_ext, and return whether this succeeded.\nac_fn_c_try_link ()\n{\n  as_lineno=${as_lineno-\"$1\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n  rm -f conftest.$ac_objext conftest$ac_exeext\n  if { { ac_try=\"$ac_link\"\ncase \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_link\") 2>conftest.err\n  ac_status=$?\n  if test -s conftest.err; then\n    grep -v '^ *+' conftest.err >conftest.er1\n    cat conftest.er1 >&5\n    mv -f conftest.er1 conftest.err\n  fi\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; } && {\n\t test -z \"$ac_c_werror_flag\" ||\n\t test ! -s conftest.err\n       } && test -s conftest$ac_exeext && {\n\t test \"$cross_compiling\" = yes ||\n\t test -x conftest$ac_exeext\n       }; then :\n  ac_retval=0\nelse\n  $as_echo \"$as_me: failed program was:\" >&5\nsed 's/^/| /' conftest.$ac_ext >&5\n\n\tac_retval=1\nfi\n  # Delete the IPA/IPO (Inter Procedural Analysis/Optimization) information\n  # created by the PGI compiler (conftest_ipa8_conftest.oo), as it would\n  # interfere with the next link command; also delete a directory that is\n  # left behind by Apple's compiler.  We do this before executing the actions.\n  rm -rf conftest.dSYM conftest_ipa8_conftest.oo\n  eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno\n  as_fn_set_status $ac_retval\n\n} # ac_fn_c_try_link\n\n# ac_fn_erl_try_run LINENO\n# ------------------------\n# Try to link conftest.$ac_ext, and return whether this succeeded. Assumes\n# that executables *can* be run.\nac_fn_erl_try_run ()\n{\n  as_lineno=${as_lineno-\"$1\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n  if { { ac_try=\"$ac_link\"\ncase \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_link\") 2>&5\n  ac_status=$?\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; } && { ac_try='./conftest$ac_exeext'\n  { { case \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_try\") 2>&5\n  ac_status=$?\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; }; }; then :\n  ac_retval=0\nelse\n  $as_echo \"$as_me: program exited with status $ac_status\" >&5\n       $as_echo \"$as_me: failed program was:\" >&5\nsed 's/^/| /' conftest.$ac_ext >&5\n\n       ac_retval=$ac_status\nfi\n  rm -rf conftest.dSYM conftest_ipa8_conftest.oo\n  eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno\n  as_fn_set_status $ac_retval\n\n} # ac_fn_erl_try_run\n\n# ac_fn_c_try_cpp LINENO\n# ----------------------\n# Try to preprocess conftest.$ac_ext, and return whether this succeeded.\nac_fn_c_try_cpp ()\n{\n  as_lineno=${as_lineno-\"$1\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n  if { { ac_try=\"$ac_cpp conftest.$ac_ext\"\ncase \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_cpp conftest.$ac_ext\") 2>conftest.err\n  ac_status=$?\n  if test -s conftest.err; then\n    grep -v '^ *+' conftest.err >conftest.er1\n    cat conftest.er1 >&5\n    mv -f conftest.er1 conftest.err\n  fi\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; } > conftest.i && {\n\t test -z \"$ac_c_preproc_warn_flag$ac_c_werror_flag\" ||\n\t test ! -s conftest.err\n       }; then :\n  ac_retval=0\nelse\n  $as_echo \"$as_me: failed program was:\" >&5\nsed 's/^/| /' conftest.$ac_ext >&5\n\n    ac_retval=1\nfi\n  eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno\n  as_fn_set_status $ac_retval\n\n} # ac_fn_c_try_cpp\n\n# ac_fn_c_check_header_mongrel LINENO HEADER VAR INCLUDES\n# -------------------------------------------------------\n# Tests whether HEADER exists, giving a warning if it cannot be compiled using\n# the include files in INCLUDES and setting the cache variable VAR\n# accordingly.\nac_fn_c_check_header_mongrel ()\n{\n  as_lineno=${as_lineno-\"$1\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n  if eval \\${$3+:} false; then :\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $2\" >&5\n$as_echo_n \"checking for $2... \" >&6; }\nif eval \\${$3+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nfi\neval ac_res=\\$$3\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nelse\n  # Is the header compilable?\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking $2 usability\" >&5\n$as_echo_n \"checking $2 usability... \" >&6; }\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n$4\n#include <$2>\n_ACEOF\nif ac_fn_c_try_compile \"$LINENO\"; then :\n  ac_header_compiler=yes\nelse\n  ac_header_compiler=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_header_compiler\" >&5\n$as_echo \"$ac_header_compiler\" >&6; }\n\n# Is the header present?\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking $2 presence\" >&5\n$as_echo_n \"checking $2 presence... \" >&6; }\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <$2>\n_ACEOF\nif ac_fn_c_try_cpp \"$LINENO\"; then :\n  ac_header_preproc=yes\nelse\n  ac_header_preproc=no\nfi\nrm -f conftest.err conftest.i conftest.$ac_ext\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_header_preproc\" >&5\n$as_echo \"$ac_header_preproc\" >&6; }\n\n# So?  What about this header?\ncase $ac_header_compiler:$ac_header_preproc:$ac_c_preproc_warn_flag in #((\n  yes:no: )\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: $2: accepted by the compiler, rejected by the preprocessor!\" >&5\n$as_echo \"$as_me: WARNING: $2: accepted by the compiler, rejected by the preprocessor!\" >&2;}\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: $2: proceeding with the compiler's result\" >&5\n$as_echo \"$as_me: WARNING: $2: proceeding with the compiler's result\" >&2;}\n    ;;\n  no:yes:* )\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: $2: present but cannot be compiled\" >&5\n$as_echo \"$as_me: WARNING: $2: present but cannot be compiled\" >&2;}\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: $2:     check for missing prerequisite headers?\" >&5\n$as_echo \"$as_me: WARNING: $2:     check for missing prerequisite headers?\" >&2;}\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: $2: see the Autoconf documentation\" >&5\n$as_echo \"$as_me: WARNING: $2: see the Autoconf documentation\" >&2;}\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: $2:     section \\\"Present But Cannot Be Compiled\\\"\" >&5\n$as_echo \"$as_me: WARNING: $2:     section \\\"Present But Cannot Be Compiled\\\"\" >&2;}\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: $2: proceeding with the compiler's result\" >&5\n$as_echo \"$as_me: WARNING: $2: proceeding with the compiler's result\" >&2;}\n( $as_echo \"## ---------------------------------------- ##\n## Report this to scalaris@googlegroups.com ##\n## ---------------------------------------- ##\"\n     ) | sed \"s/^/$as_me: WARNING:     /\" >&2\n    ;;\nesac\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $2\" >&5\n$as_echo_n \"checking for $2... \" >&6; }\nif eval \\${$3+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  eval \"$3=\\$ac_header_compiler\"\nfi\neval ac_res=\\$$3\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nfi\n  eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno\n\n} # ac_fn_c_check_header_mongrel\n\n# ac_fn_c_try_run LINENO\n# ----------------------\n# Try to link conftest.$ac_ext, and return whether this succeeded. Assumes\n# that executables *can* be run.\nac_fn_c_try_run ()\n{\n  as_lineno=${as_lineno-\"$1\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n  if { { ac_try=\"$ac_link\"\ncase \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_link\") 2>&5\n  ac_status=$?\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; } && { ac_try='./conftest$ac_exeext'\n  { { case \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_try\") 2>&5\n  ac_status=$?\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; }; }; then :\n  ac_retval=0\nelse\n  $as_echo \"$as_me: program exited with status $ac_status\" >&5\n       $as_echo \"$as_me: failed program was:\" >&5\nsed 's/^/| /' conftest.$ac_ext >&5\n\n       ac_retval=$ac_status\nfi\n  rm -rf conftest.dSYM conftest_ipa8_conftest.oo\n  eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno\n  as_fn_set_status $ac_retval\n\n} # ac_fn_c_try_run\n\n# ac_fn_c_check_header_compile LINENO HEADER VAR INCLUDES\n# -------------------------------------------------------\n# Tests whether HEADER exists and can be compiled using the include files in\n# INCLUDES, setting the cache variable VAR accordingly.\nac_fn_c_check_header_compile ()\n{\n  as_lineno=${as_lineno-\"$1\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $2\" >&5\n$as_echo_n \"checking for $2... \" >&6; }\nif eval \\${$3+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n$4\n#include <$2>\n_ACEOF\nif ac_fn_c_try_compile \"$LINENO\"; then :\n  eval \"$3=yes\"\nelse\n  eval \"$3=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\nfi\neval ac_res=\\$$3\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\n  eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno\n\n} # ac_fn_c_check_header_compile\ncat >config.log <<_ACEOF\nThis file contains any messages produced by compilers while\nrunning configure, to aid debugging if configure makes a mistake.\n\nIt was created by scalaris $as_me 0.9.0+git, which was\ngenerated by GNU Autoconf 2.69.  Invocation command line was\n\n  $ $0 $@\n\n_ACEOF\nexec 5>>config.log\n{\ncat <<_ASUNAME\n## --------- ##\n## Platform. ##\n## --------- ##\n\nhostname = `(hostname || uname -n) 2>/dev/null | sed 1q`\nuname -m = `(uname -m) 2>/dev/null || echo unknown`\nuname -r = `(uname -r) 2>/dev/null || echo unknown`\nuname -s = `(uname -s) 2>/dev/null || echo unknown`\nuname -v = `(uname -v) 2>/dev/null || echo unknown`\n\n/usr/bin/uname -p = `(/usr/bin/uname -p) 2>/dev/null || echo unknown`\n/bin/uname -X     = `(/bin/uname -X) 2>/dev/null     || echo unknown`\n\n/bin/arch              = `(/bin/arch) 2>/dev/null              || echo unknown`\n/usr/bin/arch -k       = `(/usr/bin/arch -k) 2>/dev/null       || echo unknown`\n/usr/convex/getsysinfo = `(/usr/convex/getsysinfo) 2>/dev/null || echo unknown`\n/usr/bin/hostinfo      = `(/usr/bin/hostinfo) 2>/dev/null      || echo unknown`\n/bin/machine           = `(/bin/machine) 2>/dev/null           || echo unknown`\n/usr/bin/oslevel       = `(/usr/bin/oslevel) 2>/dev/null       || echo unknown`\n/bin/universe          = `(/bin/universe) 2>/dev/null          || echo unknown`\n\n_ASUNAME\n\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    $as_echo \"PATH: $as_dir\"\n  done\nIFS=$as_save_IFS\n\n} >&5\n\ncat >&5 <<_ACEOF\n\n\n## ----------- ##\n## Core tests. ##\n## ----------- ##\n\n_ACEOF\n\n\n# Keep a trace of the command line.\n# Strip out --no-create and --no-recursion so they do not pile up.\n# Strip out --silent because we don't want to record it for future runs.\n# Also quote any args containing shell meta-characters.\n# Make two passes to allow for proper duplicate-argument suppression.\nac_configure_args=\nac_configure_args0=\nac_configure_args1=\nac_must_keep_next=false\nfor ac_pass in 1 2\ndo\n  for ac_arg\n  do\n    case $ac_arg in\n    -no-create | --no-c* | -n | -no-recursion | --no-r*) continue ;;\n    -q | -quiet | --quiet | --quie | --qui | --qu | --q \\\n    | -silent | --silent | --silen | --sile | --sil)\n      continue ;;\n    *\\'*)\n      ac_arg=`$as_echo \"$ac_arg\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"` ;;\n    esac\n    case $ac_pass in\n    1) as_fn_append ac_configure_args0 \" '$ac_arg'\" ;;\n    2)\n      as_fn_append ac_configure_args1 \" '$ac_arg'\"\n      if test $ac_must_keep_next = true; then\n\tac_must_keep_next=false # Got value, back to normal.\n      else\n\tcase $ac_arg in\n\t  *=* | --config-cache | -C | -disable-* | --disable-* \\\n\t  | -enable-* | --enable-* | -gas | --g* | -nfp | --nf* \\\n\t  | -q | -quiet | --q* | -silent | --sil* | -v | -verb* \\\n\t  | -with-* | --with-* | -without-* | --without-* | --x)\n\t    case \"$ac_configure_args0 \" in\n\t      \"$ac_configure_args1\"*\" '$ac_arg' \"* ) continue ;;\n\t    esac\n\t    ;;\n\t  -* ) ac_must_keep_next=true ;;\n\tesac\n      fi\n      as_fn_append ac_configure_args \" '$ac_arg'\"\n      ;;\n    esac\n  done\ndone\n{ ac_configure_args0=; unset ac_configure_args0;}\n{ ac_configure_args1=; unset ac_configure_args1;}\n\n# When interrupted or exit'd, cleanup temporary files, and complete\n# config.log.  We remove comments because anyway the quotes in there\n# would cause problems or look ugly.\n# WARNING: Use '\\'' to represent an apostrophe within the trap.\n# WARNING: Do not start the trap code with a newline, due to a FreeBSD 4.0 bug.\ntrap 'exit_status=$?\n  # Save into config.log some information that might help in debugging.\n  {\n    echo\n\n    $as_echo \"## ---------------- ##\n## Cache variables. ##\n## ---------------- ##\"\n    echo\n    # The following way of writing the cache mishandles newlines in values,\n(\n  for ac_var in `(set) 2>&1 | sed -n '\\''s/^\\([a-zA-Z_][a-zA-Z0-9_]*\\)=.*/\\1/p'\\''`; do\n    eval ac_val=\\$$ac_var\n    case $ac_val in #(\n    *${as_nl}*)\n      case $ac_var in #(\n      *_cv_*) { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: cache variable $ac_var contains a newline\" >&5\n$as_echo \"$as_me: WARNING: cache variable $ac_var contains a newline\" >&2;} ;;\n      esac\n      case $ac_var in #(\n      _ | IFS | as_nl) ;; #(\n      BASH_ARGV | BASH_SOURCE) eval $ac_var= ;; #(\n      *) { eval $ac_var=; unset $ac_var;} ;;\n      esac ;;\n    esac\n  done\n  (set) 2>&1 |\n    case $as_nl`(ac_space='\\'' '\\''; set) 2>&1` in #(\n    *${as_nl}ac_space=\\ *)\n      sed -n \\\n\t\"s/'\\''/'\\''\\\\\\\\'\\'''\\''/g;\n\t  s/^\\\\([_$as_cr_alnum]*_cv_[_$as_cr_alnum]*\\\\)=\\\\(.*\\\\)/\\\\1='\\''\\\\2'\\''/p\"\n      ;; #(\n    *)\n      sed -n \"/^[_$as_cr_alnum]*_cv_[_$as_cr_alnum]*=/p\"\n      ;;\n    esac |\n    sort\n)\n    echo\n\n    $as_echo \"## ----------------- ##\n## Output variables. ##\n## ----------------- ##\"\n    echo\n    for ac_var in $ac_subst_vars\n    do\n      eval ac_val=\\$$ac_var\n      case $ac_val in\n      *\\'\\''*) ac_val=`$as_echo \"$ac_val\" | sed \"s/'\\''/'\\''\\\\\\\\\\\\\\\\'\\'''\\''/g\"`;;\n      esac\n      $as_echo \"$ac_var='\\''$ac_val'\\''\"\n    done | sort\n    echo\n\n    if test -n \"$ac_subst_files\"; then\n      $as_echo \"## ------------------- ##\n## File substitutions. ##\n## ------------------- ##\"\n      echo\n      for ac_var in $ac_subst_files\n      do\n\teval ac_val=\\$$ac_var\n\tcase $ac_val in\n\t*\\'\\''*) ac_val=`$as_echo \"$ac_val\" | sed \"s/'\\''/'\\''\\\\\\\\\\\\\\\\'\\'''\\''/g\"`;;\n\tesac\n\t$as_echo \"$ac_var='\\''$ac_val'\\''\"\n      done | sort\n      echo\n    fi\n\n    if test -s confdefs.h; then\n      $as_echo \"## ----------- ##\n## confdefs.h. ##\n## ----------- ##\"\n      echo\n      cat confdefs.h\n      echo\n    fi\n    test \"$ac_signal\" != 0 &&\n      $as_echo \"$as_me: caught signal $ac_signal\"\n    $as_echo \"$as_me: exit $exit_status\"\n  } >&5\n  rm -f core *.core core.conftest.* &&\n    rm -f -r conftest* confdefs* conf$$* $ac_clean_files &&\n    exit $exit_status\n' 0\nfor ac_signal in 1 2 13 15; do\n  trap 'ac_signal='$ac_signal'; as_fn_exit 1' $ac_signal\ndone\nac_signal=0\n\n# confdefs.h avoids OS command line length limits that DEFS can exceed.\nrm -f -r conftest* confdefs.h\n\n$as_echo \"/* confdefs.h */\" > confdefs.h\n\n# Predefined preprocessor variables.\n\ncat >>confdefs.h <<_ACEOF\n#define PACKAGE_NAME \"$PACKAGE_NAME\"\n_ACEOF\n\ncat >>confdefs.h <<_ACEOF\n#define PACKAGE_TARNAME \"$PACKAGE_TARNAME\"\n_ACEOF\n\ncat >>confdefs.h <<_ACEOF\n#define PACKAGE_VERSION \"$PACKAGE_VERSION\"\n_ACEOF\n\ncat >>confdefs.h <<_ACEOF\n#define PACKAGE_STRING \"$PACKAGE_STRING\"\n_ACEOF\n\ncat >>confdefs.h <<_ACEOF\n#define PACKAGE_BUGREPORT \"$PACKAGE_BUGREPORT\"\n_ACEOF\n\ncat >>confdefs.h <<_ACEOF\n#define PACKAGE_URL \"$PACKAGE_URL\"\n_ACEOF\n\n\n# Let the site file select an alternate cache file if it wants to.\n# Prefer an explicitly selected file to automatically selected ones.\nac_site_file1=NONE\nac_site_file2=NONE\nif test -n \"$CONFIG_SITE\"; then\n  # We do not want a PATH search for config.site.\n  case $CONFIG_SITE in #((\n    -*)  ac_site_file1=./$CONFIG_SITE;;\n    */*) ac_site_file1=$CONFIG_SITE;;\n    *)   ac_site_file1=./$CONFIG_SITE;;\n  esac\nelif test \"x$prefix\" != xNONE; then\n  ac_site_file1=$prefix/share/config.site\n  ac_site_file2=$prefix/etc/config.site\nelse\n  ac_site_file1=$ac_default_prefix/share/config.site\n  ac_site_file2=$ac_default_prefix/etc/config.site\nfi\nfor ac_site_file in \"$ac_site_file1\" \"$ac_site_file2\"\ndo\n  test \"x$ac_site_file\" = xNONE && continue\n  if test /dev/null != \"$ac_site_file\" && test -r \"$ac_site_file\"; then\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: loading site script $ac_site_file\" >&5\n$as_echo \"$as_me: loading site script $ac_site_file\" >&6;}\n    sed 's/^/| /' \"$ac_site_file\" >&5\n    . \"$ac_site_file\" \\\n      || { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"failed to load site script $ac_site_file\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n  fi\ndone\n\nif test -r \"$cache_file\"; then\n  # Some versions of bash will fail to source /dev/null (special files\n  # actually), so we avoid doing that.  DJGPP emulates it as a regular file.\n  if test /dev/null != \"$cache_file\" && test -f \"$cache_file\"; then\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: loading cache $cache_file\" >&5\n$as_echo \"$as_me: loading cache $cache_file\" >&6;}\n    case $cache_file in\n      [\\\\/]* | ?:[\\\\/]* ) . \"$cache_file\";;\n      *)                      . \"./$cache_file\";;\n    esac\n  fi\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: creating cache $cache_file\" >&5\n$as_echo \"$as_me: creating cache $cache_file\" >&6;}\n  >$cache_file\nfi\n\n# Check that the precious variables saved in the cache have kept the same\n# value.\nac_cache_corrupted=false\nfor ac_var in $ac_precious_vars; do\n  eval ac_old_set=\\$ac_cv_env_${ac_var}_set\n  eval ac_new_set=\\$ac_env_${ac_var}_set\n  eval ac_old_val=\\$ac_cv_env_${ac_var}_value\n  eval ac_new_val=\\$ac_env_${ac_var}_value\n  case $ac_old_set,$ac_new_set in\n    set,)\n      { $as_echo \"$as_me:${as_lineno-$LINENO}: error: \\`$ac_var' was set to \\`$ac_old_val' in the previous run\" >&5\n$as_echo \"$as_me: error: \\`$ac_var' was set to \\`$ac_old_val' in the previous run\" >&2;}\n      ac_cache_corrupted=: ;;\n    ,set)\n      { $as_echo \"$as_me:${as_lineno-$LINENO}: error: \\`$ac_var' was not set in the previous run\" >&5\n$as_echo \"$as_me: error: \\`$ac_var' was not set in the previous run\" >&2;}\n      ac_cache_corrupted=: ;;\n    ,);;\n    *)\n      if test \"x$ac_old_val\" != \"x$ac_new_val\"; then\n\t# differences in whitespace do not lead to failure.\n\tac_old_val_w=`echo x $ac_old_val`\n\tac_new_val_w=`echo x $ac_new_val`\n\tif test \"$ac_old_val_w\" != \"$ac_new_val_w\"; then\n\t  { $as_echo \"$as_me:${as_lineno-$LINENO}: error: \\`$ac_var' has changed since the previous run:\" >&5\n$as_echo \"$as_me: error: \\`$ac_var' has changed since the previous run:\" >&2;}\n\t  ac_cache_corrupted=:\n\telse\n\t  { $as_echo \"$as_me:${as_lineno-$LINENO}: warning: ignoring whitespace changes in \\`$ac_var' since the previous run:\" >&5\n$as_echo \"$as_me: warning: ignoring whitespace changes in \\`$ac_var' since the previous run:\" >&2;}\n\t  eval $ac_var=\\$ac_old_val\n\tfi\n\t{ $as_echo \"$as_me:${as_lineno-$LINENO}:   former value:  \\`$ac_old_val'\" >&5\n$as_echo \"$as_me:   former value:  \\`$ac_old_val'\" >&2;}\n\t{ $as_echo \"$as_me:${as_lineno-$LINENO}:   current value: \\`$ac_new_val'\" >&5\n$as_echo \"$as_me:   current value: \\`$ac_new_val'\" >&2;}\n      fi;;\n  esac\n  # Pass precious variables to config.status.\n  if test \"$ac_new_set\" = set; then\n    case $ac_new_val in\n    *\\'*) ac_arg=$ac_var=`$as_echo \"$ac_new_val\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"` ;;\n    *) ac_arg=$ac_var=$ac_new_val ;;\n    esac\n    case \" $ac_configure_args \" in\n      *\" '$ac_arg' \"*) ;; # Avoid dups.  Use of quotes ensures accuracy.\n      *) as_fn_append ac_configure_args \" '$ac_arg'\" ;;\n    esac\n  fi\ndone\nif $ac_cache_corrupted; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: error: changes in the environment can compromise the build\" >&5\n$as_echo \"$as_me: error: changes in the environment can compromise the build\" >&2;}\n  as_fn_error $? \"run \\`make distclean' and/or \\`rm $cache_file' and start over\" \"$LINENO\" 5\nfi\n## -------------------- ##\n## Main body of script. ##\n## -------------------- ##\n\nac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\n\n\n# we will never do automake, just pull in config.guess, config.sub, ... for boost\nac_aux_dir=\nfor ac_dir in build-aux \"$srcdir\"/build-aux; do\n  if test -f \"$ac_dir/install-sh\"; then\n    ac_aux_dir=$ac_dir\n    ac_install_sh=\"$ac_aux_dir/install-sh -c\"\n    break\n  elif test -f \"$ac_dir/install.sh\"; then\n    ac_aux_dir=$ac_dir\n    ac_install_sh=\"$ac_aux_dir/install.sh -c\"\n    break\n  elif test -f \"$ac_dir/shtool\"; then\n    ac_aux_dir=$ac_dir\n    ac_install_sh=\"$ac_aux_dir/shtool install -c\"\n    break\n  fi\ndone\nif test -z \"$ac_aux_dir\"; then\n  as_fn_error $? \"cannot find install-sh, install.sh, or shtool in build-aux \\\"$srcdir\\\"/build-aux\" \"$LINENO\" 5\nfi\n\n# These three variables are undocumented and unsupported,\n# and are intended to be withdrawn in a future Autoconf release.\n# They can cause serious problems if a builder's source tree is in a directory\n# whose full name contains unusual characters.\nac_config_guess=\"$SHELL $ac_aux_dir/config.guess\"  # Please don't use this var.\nac_config_sub=\"$SHELL $ac_aux_dir/config.sub\"  # Please don't use this var.\nac_configure=\"$SHELL $ac_aux_dir/configure\"  # Please don't use this var.\n\n\nam__api_version='1.16'\n\n# Find a good install program.  We prefer a C program (faster),\n# so one script is as good as another.  But avoid the broken or\n# incompatible versions:\n# SysV /etc/install, /usr/sbin/install\n# SunOS /usr/etc/install\n# IRIX /sbin/install\n# AIX /bin/install\n# AmigaOS /C/install, which installs bootblocks on floppy discs\n# AIX 4 /usr/bin/installbsd, which doesn't work without a -g flag\n# AFS /usr/afsws/bin/install, which mishandles nonexistent args\n# SVR4 /usr/ucb/install, which tries to use the nonexistent group \"staff\"\n# OS/2's system install, which has a completely different semantic\n# ./install, which can be erroneously created by make from ./install.sh.\n# Reject install programs that cannot install multiple files.\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for a BSD-compatible install\" >&5\n$as_echo_n \"checking for a BSD-compatible install... \" >&6; }\nif test -z \"$INSTALL\"; then\nif ${ac_cv_path_install+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    # Account for people who put trailing slashes in PATH elements.\ncase $as_dir/ in #((\n  ./ | .// | /[cC]/* | \\\n  /etc/* | /usr/sbin/* | /usr/etc/* | /sbin/* | /usr/afsws/bin/* | \\\n  ?:[\\\\/]os2[\\\\/]install[\\\\/]* | ?:[\\\\/]OS2[\\\\/]INSTALL[\\\\/]* | \\\n  /usr/ucb/* ) ;;\n  *)\n    # OSF1 and SCO ODT 3.0 have their own names for install.\n    # Don't use installbsd from OSF since it installs stuff as root\n    # by default.\n    for ac_prog in ginstall scoinst install; do\n      for ac_exec_ext in '' $ac_executable_extensions; do\n\tif as_fn_executable_p \"$as_dir/$ac_prog$ac_exec_ext\"; then\n\t  if test $ac_prog = install &&\n\t    grep dspmsg \"$as_dir/$ac_prog$ac_exec_ext\" >/dev/null 2>&1; then\n\t    # AIX install.  It has an incompatible calling convention.\n\t    :\n\t  elif test $ac_prog = install &&\n\t    grep pwplus \"$as_dir/$ac_prog$ac_exec_ext\" >/dev/null 2>&1; then\n\t    # program-specific install script used by HP pwplus--don't use.\n\t    :\n\t  else\n\t    rm -rf conftest.one conftest.two conftest.dir\n\t    echo one > conftest.one\n\t    echo two > conftest.two\n\t    mkdir conftest.dir\n\t    if \"$as_dir/$ac_prog$ac_exec_ext\" -c conftest.one conftest.two \"`pwd`/conftest.dir\" &&\n\t      test -s conftest.one && test -s conftest.two &&\n\t      test -s conftest.dir/conftest.one &&\n\t      test -s conftest.dir/conftest.two\n\t    then\n\t      ac_cv_path_install=\"$as_dir/$ac_prog$ac_exec_ext -c\"\n\t      break 3\n\t    fi\n\t  fi\n\tfi\n      done\n    done\n    ;;\nesac\n\n  done\nIFS=$as_save_IFS\n\nrm -rf conftest.one conftest.two conftest.dir\n\nfi\n  if test \"${ac_cv_path_install+set}\" = set; then\n    INSTALL=$ac_cv_path_install\n  else\n    # As a last resort, use the slow shell script.  Don't cache a\n    # value for INSTALL within a source directory, because that will\n    # break other packages using the cache if that directory is\n    # removed, or if the value is a relative name.\n    INSTALL=$ac_install_sh\n  fi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $INSTALL\" >&5\n$as_echo \"$INSTALL\" >&6; }\n\n# Use test -z because SunOS4 sh mishandles braces in ${var-val}.\n# It thinks the first close brace ends the variable substitution.\ntest -z \"$INSTALL_PROGRAM\" && INSTALL_PROGRAM='${INSTALL}'\n\ntest -z \"$INSTALL_SCRIPT\" && INSTALL_SCRIPT='${INSTALL}'\n\ntest -z \"$INSTALL_DATA\" && INSTALL_DATA='${INSTALL} -m 644'\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether build environment is sane\" >&5\n$as_echo_n \"checking whether build environment is sane... \" >&6; }\n# Reject unsafe characters in $srcdir or the absolute working directory\n# name.  Accept space and tab only in the latter.\nam_lf='\n'\ncase `pwd` in\n  *[\\\\\\\"\\#\\$\\&\\'\\`$am_lf]*)\n    as_fn_error $? \"unsafe absolute working directory name\" \"$LINENO\" 5;;\nesac\ncase $srcdir in\n  *[\\\\\\\"\\#\\$\\&\\'\\`$am_lf\\ \\\t]*)\n    as_fn_error $? \"unsafe srcdir value: '$srcdir'\" \"$LINENO\" 5;;\nesac\n\n# Do 'set' in a subshell so we don't clobber the current shell's\n# arguments.  Must try -L first in case configure is actually a\n# symlink; some systems play weird games with the mod time of symlinks\n# (eg FreeBSD returns the mod time of the symlink's containing\n# directory).\nif (\n   am_has_slept=no\n   for am_try in 1 2; do\n     echo \"timestamp, slept: $am_has_slept\" > conftest.file\n     set X `ls -Lt \"$srcdir/configure\" conftest.file 2> /dev/null`\n     if test \"$*\" = \"X\"; then\n\t# -L didn't work.\n\tset X `ls -t \"$srcdir/configure\" conftest.file`\n     fi\n     if test \"$*\" != \"X $srcdir/configure conftest.file\" \\\n\t&& test \"$*\" != \"X conftest.file $srcdir/configure\"; then\n\n\t# If neither matched, then we have a broken ls.  This can happen\n\t# if, for instance, CONFIG_SHELL is bash and it inherits a\n\t# broken ls alias from the environment.  This has actually\n\t# happened.  Such a system could not be considered \"sane\".\n\tas_fn_error $? \"ls -t appears to fail.  Make sure there is not a broken\n  alias in your environment\" \"$LINENO\" 5\n     fi\n     if test \"$2\" = conftest.file || test $am_try -eq 2; then\n       break\n     fi\n     # Just in case.\n     sleep 1\n     am_has_slept=yes\n   done\n   test \"$2\" = conftest.file\n   )\nthen\n   # Ok.\n   :\nelse\n   as_fn_error $? \"newly created file is older than distributed files!\nCheck your system clock\" \"$LINENO\" 5\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: yes\" >&5\n$as_echo \"yes\" >&6; }\n# If we didn't sleep, we still need to ensure time stamps of config.status and\n# generated files are strictly newer.\nam_sleep_pid=\nif grep 'slept: no' conftest.file >/dev/null 2>&1; then\n  ( sleep 1 ) &\n  am_sleep_pid=$!\nfi\n\nrm -f conftest.file\n\ntest \"$program_prefix\" != NONE &&\n  program_transform_name=\"s&^&$program_prefix&;$program_transform_name\"\n# Use a double $ so make ignores it.\ntest \"$program_suffix\" != NONE &&\n  program_transform_name=\"s&\\$&$program_suffix&;$program_transform_name\"\n# Double any \\ or $.\n# By default was `s,x,x', remove it if useless.\nac_script='s/[\\\\$]/&&/g;s/;s,x,x,$//'\nprogram_transform_name=`$as_echo \"$program_transform_name\" | sed \"$ac_script\"`\n\n# Expand $ac_aux_dir to an absolute path.\nam_aux_dir=`cd \"$ac_aux_dir\" && pwd`\n\nif test x\"${MISSING+set}\" != xset; then\n  case $am_aux_dir in\n  *\\ * | *\\\t*)\n    MISSING=\"\\${SHELL} \\\"$am_aux_dir/missing\\\"\" ;;\n  *)\n    MISSING=\"\\${SHELL} $am_aux_dir/missing\" ;;\n  esac\nfi\n# Use eval to expand $SHELL\nif eval \"$MISSING --is-lightweight\"; then\n  am_missing_run=\"$MISSING \"\nelse\n  am_missing_run=\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: 'missing' script is too old or missing\" >&5\n$as_echo \"$as_me: WARNING: 'missing' script is too old or missing\" >&2;}\nfi\n\nif test x\"${install_sh+set}\" != xset; then\n  case $am_aux_dir in\n  *\\ * | *\\\t*)\n    install_sh=\"\\${SHELL} '$am_aux_dir/install-sh'\" ;;\n  *)\n    install_sh=\"\\${SHELL} $am_aux_dir/install-sh\"\n  esac\nfi\n\n# Installed binaries are usually stripped using 'strip' when the user\n# run \"make install-strip\".  However 'strip' might not be the right\n# tool to use in cross-compilation environments, therefore Automake\n# will honor the 'STRIP' environment variable to overrule this program.\nif test \"$cross_compiling\" != no; then\n  if test -n \"$ac_tool_prefix\"; then\n  # Extract the first word of \"${ac_tool_prefix}strip\", so it can be a program name with args.\nset dummy ${ac_tool_prefix}strip; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_prog_STRIP+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -n \"$STRIP\"; then\n  ac_cv_prog_STRIP=\"$STRIP\" # Let the user override the test.\nelse\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_prog_STRIP=\"${ac_tool_prefix}strip\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\nfi\nfi\nSTRIP=$ac_cv_prog_STRIP\nif test -n \"$STRIP\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $STRIP\" >&5\n$as_echo \"$STRIP\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\nfi\nif test -z \"$ac_cv_prog_STRIP\"; then\n  ac_ct_STRIP=$STRIP\n  # Extract the first word of \"strip\", so it can be a program name with args.\nset dummy strip; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_prog_ac_ct_STRIP+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -n \"$ac_ct_STRIP\"; then\n  ac_cv_prog_ac_ct_STRIP=\"$ac_ct_STRIP\" # Let the user override the test.\nelse\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_prog_ac_ct_STRIP=\"strip\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\nfi\nfi\nac_ct_STRIP=$ac_cv_prog_ac_ct_STRIP\nif test -n \"$ac_ct_STRIP\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_ct_STRIP\" >&5\n$as_echo \"$ac_ct_STRIP\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n  if test \"x$ac_ct_STRIP\" = x; then\n    STRIP=\":\"\n  else\n    case $cross_compiling:$ac_tool_warned in\nyes:)\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet\" >&5\n$as_echo \"$as_me: WARNING: using cross tools not prefixed with host triplet\" >&2;}\nac_tool_warned=yes ;;\nesac\n    STRIP=$ac_ct_STRIP\n  fi\nelse\n  STRIP=\"$ac_cv_prog_STRIP\"\nfi\n\nfi\nINSTALL_STRIP_PROGRAM=\"\\$(install_sh) -c -s\"\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for a thread-safe mkdir -p\" >&5\n$as_echo_n \"checking for a thread-safe mkdir -p... \" >&6; }\nif test -z \"$MKDIR_P\"; then\n  if ${ac_cv_path_mkdir+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH$PATH_SEPARATOR/opt/sfw/bin\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_prog in mkdir gmkdir; do\n\t for ac_exec_ext in '' $ac_executable_extensions; do\n\t   as_fn_executable_p \"$as_dir/$ac_prog$ac_exec_ext\" || continue\n\t   case `\"$as_dir/$ac_prog$ac_exec_ext\" --version 2>&1` in #(\n\t     'mkdir (GNU coreutils) '* | \\\n\t     'mkdir (coreutils) '* | \\\n\t     'mkdir (fileutils) '4.1*)\n\t       ac_cv_path_mkdir=$as_dir/$ac_prog$ac_exec_ext\n\t       break 3;;\n\t   esac\n\t done\n       done\n  done\nIFS=$as_save_IFS\n\nfi\n\n  test -d ./--version && rmdir ./--version\n  if test \"${ac_cv_path_mkdir+set}\" = set; then\n    MKDIR_P=\"$ac_cv_path_mkdir -p\"\n  else\n    # As a last resort, use the slow shell script.  Don't cache a\n    # value for MKDIR_P within a source directory, because that will\n    # break other packages using the cache if that directory is\n    # removed, or if the value is a relative name.\n    MKDIR_P=\"$ac_install_sh -d\"\n  fi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $MKDIR_P\" >&5\n$as_echo \"$MKDIR_P\" >&6; }\n\nfor ac_prog in gawk mawk nawk awk\ndo\n  # Extract the first word of \"$ac_prog\", so it can be a program name with args.\nset dummy $ac_prog; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_prog_AWK+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -n \"$AWK\"; then\n  ac_cv_prog_AWK=\"$AWK\" # Let the user override the test.\nelse\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_prog_AWK=\"$ac_prog\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\nfi\nfi\nAWK=$ac_cv_prog_AWK\nif test -n \"$AWK\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $AWK\" >&5\n$as_echo \"$AWK\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n  test -n \"$AWK\" && break\ndone\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether ${MAKE-make} sets \\$(MAKE)\" >&5\n$as_echo_n \"checking whether ${MAKE-make} sets \\$(MAKE)... \" >&6; }\nset x ${MAKE-make}\nac_make=`$as_echo \"$2\" | sed 's/+/p/g; s/[^a-zA-Z0-9_]/_/g'`\nif eval \\${ac_cv_prog_make_${ac_make}_set+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  cat >conftest.make <<\\_ACEOF\nSHELL = /bin/sh\nall:\n\t@echo '@@@%%%=$(MAKE)=@@@%%%'\n_ACEOF\n# GNU make sometimes prints \"make[1]: Entering ...\", which would confuse us.\ncase `${MAKE-make} -f conftest.make 2>/dev/null` in\n  *@@@%%%=?*=@@@%%%*)\n    eval ac_cv_prog_make_${ac_make}_set=yes;;\n  *)\n    eval ac_cv_prog_make_${ac_make}_set=no;;\nesac\nrm -f conftest.make\nfi\nif eval test \\$ac_cv_prog_make_${ac_make}_set = yes; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: yes\" >&5\n$as_echo \"yes\" >&6; }\n  SET_MAKE=\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\n  SET_MAKE=\"MAKE=${MAKE-make}\"\nfi\n\nrm -rf .tst 2>/dev/null\nmkdir .tst 2>/dev/null\nif test -d .tst; then\n  am__leading_dot=.\nelse\n  am__leading_dot=_\nfi\nrmdir .tst 2>/dev/null\n\n# Check whether --enable-silent-rules was given.\nif test \"${enable_silent_rules+set}\" = set; then :\n  enableval=$enable_silent_rules;\nfi\n\ncase $enable_silent_rules in # (((\n  yes) AM_DEFAULT_VERBOSITY=0;;\n   no) AM_DEFAULT_VERBOSITY=1;;\n    *) AM_DEFAULT_VERBOSITY=1;;\nesac\nam_make=${MAKE-make}\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether $am_make supports nested variables\" >&5\n$as_echo_n \"checking whether $am_make supports nested variables... \" >&6; }\nif ${am_cv_make_support_nested_variables+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if $as_echo 'TRUE=$(BAR$(V))\nBAR0=false\nBAR1=true\nV=1\nam__doit:\n\t@$(TRUE)\n.PHONY: am__doit' | $am_make -f - >/dev/null 2>&1; then\n  am_cv_make_support_nested_variables=yes\nelse\n  am_cv_make_support_nested_variables=no\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $am_cv_make_support_nested_variables\" >&5\n$as_echo \"$am_cv_make_support_nested_variables\" >&6; }\nif test $am_cv_make_support_nested_variables = yes; then\n    AM_V='$(V)'\n  AM_DEFAULT_V='$(AM_DEFAULT_VERBOSITY)'\nelse\n  AM_V=$AM_DEFAULT_VERBOSITY\n  AM_DEFAULT_V=$AM_DEFAULT_VERBOSITY\nfi\nAM_BACKSLASH='\\'\n\nif test \"`cd $srcdir && pwd`\" != \"`pwd`\"; then\n  # Use -I$(srcdir) only when $(srcdir) != ., so that make's output\n  # is not polluted with repeated \"-I.\"\n  am__isrc=' -I$(srcdir)'\n  # test to see if srcdir already configured\n  if test -f $srcdir/config.status; then\n    as_fn_error $? \"source directory already configured; run \\\"make distclean\\\" there first\" \"$LINENO\" 5\n  fi\nfi\n\n# test whether we have cygpath\nif test -z \"$CYGPATH_W\"; then\n  if (cygpath --version) >/dev/null 2>/dev/null; then\n    CYGPATH_W='cygpath -w'\n  else\n    CYGPATH_W=echo\n  fi\nfi\n\n\n# Define the identity of the package.\n PACKAGE='scalaris'\n VERSION='0.9.0+git'\n\n\ncat >>confdefs.h <<_ACEOF\n#define PACKAGE \"$PACKAGE\"\n_ACEOF\n\n\ncat >>confdefs.h <<_ACEOF\n#define VERSION \"$VERSION\"\n_ACEOF\n\n# Some tools Automake needs.\n\nACLOCAL=${ACLOCAL-\"${am_missing_run}aclocal-${am__api_version}\"}\n\n\nAUTOCONF=${AUTOCONF-\"${am_missing_run}autoconf\"}\n\n\nAUTOMAKE=${AUTOMAKE-\"${am_missing_run}automake-${am__api_version}\"}\n\n\nAUTOHEADER=${AUTOHEADER-\"${am_missing_run}autoheader\"}\n\n\nMAKEINFO=${MAKEINFO-\"${am_missing_run}makeinfo\"}\n\n# For better backward compatibility.  To be removed once Automake 1.9.x\n# dies out for good.  For more background, see:\n# <https://lists.gnu.org/archive/html/automake/2012-07/msg00001.html>\n# <https://lists.gnu.org/archive/html/automake/2012-07/msg00014.html>\nmkdir_p='$(MKDIR_P)'\n\n# We need awk for the \"check\" target (and possibly the TAP driver).  The\n# system \"awk\" is bad on some platforms.\n# Always define AMTAR for backward compatibility.  Yes, it's still used\n# in the wild :-(  We should find a proper way to deprecate it ...\nAMTAR='$${TAR-tar}'\n\n\n# We'll loop over all known methods to create a tar archive until one works.\n_am_tools='gnutar  pax cpio none'\n\nam__tar='$${TAR-tar} chof - \"$$tardir\"' am__untar='$${TAR-tar} xf -'\n\n\n\n\n\n\n# POSIX will say in a future version that running \"rm -f\" with no argument\n# is OK; and we want to be able to make that assumption in our Makefile\n# recipes.  So use an aggressive probe to check that the usage we want is\n# actually supported \"in the wild\" to an acceptable degree.\n# See automake bug#10828.\n# To make any issue more visible, cause the running configure to be aborted\n# by default if the 'rm' program in use doesn't match our expectations; the\n# user can still override this though.\nif rm -f && rm -fr && rm -rf; then : OK; else\n  cat >&2 <<'END'\nOops!\n\nYour 'rm' program seems unable to run without file operands specified\non the command line, even when the '-f' option is present.  This is contrary\nto the behaviour of most rm programs out there, and not conforming with\nthe upcoming POSIX standard: <http://austingroupbugs.net/view.php?id=542>\n\nPlease tell bug-automake@gnu.org about your system, including the value\nof your $PATH and any error possibly output before this message.  This\ncan help us improve future automake versions.\n\nEND\n  if test x\"$ACCEPT_INFERIOR_RM_PROGRAM\" = x\"yes\"; then\n    echo 'Configuration will proceed anyway, since you have set the' >&2\n    echo 'ACCEPT_INFERIOR_RM_PROGRAM variable to \"yes\"' >&2\n    echo >&2\n  else\n    cat >&2 <<'END'\nAborting the configuration process, to ensure you take notice of the issue.\n\nYou can download and install GNU coreutils to get an 'rm' implementation\nthat behaves properly: <https://www.gnu.org/software/coreutils/>.\n\nIf you want to complete the configuration process using your problematic\n'rm' anyway, export the environment variable ACCEPT_INFERIOR_RM_PROGRAM\nto \"yes\", and re-run configure.\n\nEND\n    as_fn_error $? \"Your 'rm' program is bad, sorry.\" \"$LINENO\" 5\n  fi\nfi\n\n\nEMAKEFILEDEFINES=\nEDOCMACROS=\nDIALYZER_FLAGS=\n\n# Extract the first word of \"false\", so it can be a program name with args.\nset dummy false; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_FALSE+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $FALSE in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_FALSE=\"$FALSE\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_FALSE=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  ;;\nesac\nfi\nFALSE=$ac_cv_path_FALSE\nif test -n \"$FALSE\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $FALSE\" >&5\n$as_echo \"$FALSE\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n\n# Extract the first word of \"true\", so it can be a program name with args.\nset dummy true; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_TRUE+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $TRUE in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_TRUE=\"$TRUE\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_TRUE=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  ;;\nesac\nfi\nTRUE=$ac_cv_path_TRUE\nif test -n \"$TRUE\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $TRUE\" >&5\n$as_echo \"$TRUE\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for a sed that does not truncate output\" >&5\n$as_echo_n \"checking for a sed that does not truncate output... \" >&6; }\nif ${ac_cv_path_SED+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n            ac_script=s/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb/\n     for ac_i in 1 2 3 4 5 6 7; do\n       ac_script=\"$ac_script$as_nl$ac_script\"\n     done\n     echo \"$ac_script\" 2>/dev/null | sed 99q >conftest.sed\n     { ac_script=; unset ac_script;}\n     if test -z \"$SED\"; then\n  ac_path_SED_found=false\n  # Loop through the user's path and test for each of PROGNAME-LIST\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_prog in sed gsed; do\n    for ac_exec_ext in '' $ac_executable_extensions; do\n      ac_path_SED=\"$as_dir/$ac_prog$ac_exec_ext\"\n      as_fn_executable_p \"$ac_path_SED\" || continue\n# Check for GNU ac_path_SED and select it if it is found.\n  # Check for GNU $ac_path_SED\ncase `\"$ac_path_SED\" --version 2>&1` in\n*GNU*)\n  ac_cv_path_SED=\"$ac_path_SED\" ac_path_SED_found=:;;\n*)\n  ac_count=0\n  $as_echo_n 0123456789 >\"conftest.in\"\n  while :\n  do\n    cat \"conftest.in\" \"conftest.in\" >\"conftest.tmp\"\n    mv \"conftest.tmp\" \"conftest.in\"\n    cp \"conftest.in\" \"conftest.nl\"\n    $as_echo '' >> \"conftest.nl\"\n    \"$ac_path_SED\" -f conftest.sed < \"conftest.nl\" >\"conftest.out\" 2>/dev/null || break\n    diff \"conftest.out\" \"conftest.nl\" >/dev/null 2>&1 || break\n    as_fn_arith $ac_count + 1 && ac_count=$as_val\n    if test $ac_count -gt ${ac_path_SED_max-0}; then\n      # Best one so far, save it but keep looking for a better one\n      ac_cv_path_SED=\"$ac_path_SED\"\n      ac_path_SED_max=$ac_count\n    fi\n    # 10*(2^10) chars as input seems more than enough\n    test $ac_count -gt 10 && break\n  done\n  rm -f conftest.in conftest.tmp conftest.nl conftest.out;;\nesac\n\n      $ac_path_SED_found && break 3\n    done\n  done\n  done\nIFS=$as_save_IFS\n  if test -z \"$ac_cv_path_SED\"; then\n    as_fn_error $? \"no acceptable sed could be found in \\$PATH\" \"$LINENO\" 5\n  fi\nelse\n  ac_cv_path_SED=$SED\nfi\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_SED\" >&5\n$as_echo \"$ac_cv_path_SED\" >&6; }\n SED=\"$ac_cv_path_SED\"\n  rm -f conftest.sed\n\n# Extract the first word of \"doxygen\", so it can be a program name with args.\nset dummy doxygen; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_DOXYGEN+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $DOXYGEN in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_DOXYGEN=\"$DOXYGEN\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_DOXYGEN=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  ;;\nesac\nfi\nDOXYGEN=$ac_cv_path_DOXYGEN\nif test -n \"$DOXYGEN\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $DOXYGEN\" >&5\n$as_echo \"$DOXYGEN\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n\n\n# Extract the first word of \"ar\", so it can be a program name with args.\nset dummy ar; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_AR+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $AR in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_AR=\"$AR\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_AR=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  ;;\nesac\nfi\nAR=$ac_cv_path_AR\nif test -n \"$AR\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $AR\" >&5\n$as_echo \"$AR\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n\n# Extract the first word of \"libtool\", so it can be a program name with args.\nset dummy libtool; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_LIBTOOL+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $LIBTOOL in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_LIBTOOL=\"$LIBTOOL\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_LIBTOOL=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  ;;\nesac\nfi\nLIBTOOL=$ac_cv_path_LIBTOOL\nif test -n \"$LIBTOOL\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $LIBTOOL\" >&5\n$as_echo \"$LIBTOOL\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n\n\n###########################################################\n#\n# check for C++ and boost\n#\n###########################################################\n\nENABLE_CPP=\n# Check whether --enable-cpp was given.\nif test \"${enable_cpp+set}\" = set; then :\n  enableval=$enable_cpp; ENABLE_CPP=$enableval\nelse\n  ENABLE_CPP=$enableval\nfi\n\n\nenabled_cxx=0\nif test \"x$ENABLE_CPP\" != xno ; then\n  # C++ and boost checks\n  DEPDIR=\"${am__leading_dot}deps\"\n\nac_config_commands=\"$ac_config_commands depfiles\"\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether ${MAKE-make} supports the include directive\" >&5\n$as_echo_n \"checking whether ${MAKE-make} supports the include directive... \" >&6; }\ncat > confinc.mk << 'END'\nam__doit:\n\t@echo this is the am__doit target >confinc.out\n.PHONY: am__doit\nEND\nam__include=\"#\"\nam__quote=\n# BSD make does it like this.\necho '.include \"confinc.mk\" # ignored' > confmf.BSD\n# Other make implementations (GNU, Solaris 10, AIX) do it like this.\necho 'include confinc.mk # ignored' > confmf.GNU\n_am_result=no\nfor s in GNU BSD; do\n  { echo \"$as_me:$LINENO: ${MAKE-make} -f confmf.$s && cat confinc.out\" >&5\n   (${MAKE-make} -f confmf.$s && cat confinc.out) >&5 2>&5\n   ac_status=$?\n   echo \"$as_me:$LINENO: \\$? = $ac_status\" >&5\n   (exit $ac_status); }\n  case $?:`cat confinc.out 2>/dev/null` in #(\n  '0:this is the am__doit target') :\n    case $s in #(\n  BSD) :\n    am__include='.include' am__quote='\"' ;; #(\n  *) :\n    am__include='include' am__quote='' ;;\nesac ;; #(\n  *) :\n     ;;\nesac\n  if test \"$am__include\" != \"#\"; then\n    _am_result=\"yes ($s style)\"\n    break\n  fi\ndone\nrm -f confinc.* confmf.*\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: ${_am_result}\" >&5\n$as_echo \"${_am_result}\" >&6; }\n\n# Check whether --enable-dependency-tracking was given.\nif test \"${enable_dependency_tracking+set}\" = set; then :\n  enableval=$enable_dependency_tracking;\nfi\n\nif test \"x$enable_dependency_tracking\" != xno; then\n  am_depcomp=\"$ac_aux_dir/depcomp\"\n  AMDEPBACKSLASH='\\'\n  am__nodep='_no'\nfi\n if test \"x$enable_dependency_tracking\" != xno; then\n  AMDEP_TRUE=\n  AMDEP_FALSE='#'\nelse\n  AMDEP_TRUE='#'\n  AMDEP_FALSE=\nfi\n\n\nac_ext=cpp\nac_cpp='$CXXCPP $CPPFLAGS'\nac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\nif test -z \"$CXX\"; then\n  if test -n \"$CCC\"; then\n    CXX=$CCC\n  else\n    if test -n \"$ac_tool_prefix\"; then\n  for ac_prog in g++ c++ gpp aCC CC cxx cc++ cl.exe FCC KCC RCC xlC_r xlC\n  do\n    # Extract the first word of \"$ac_tool_prefix$ac_prog\", so it can be a program name with args.\nset dummy $ac_tool_prefix$ac_prog; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_prog_CXX+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -n \"$CXX\"; then\n  ac_cv_prog_CXX=\"$CXX\" # Let the user override the test.\nelse\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_prog_CXX=\"$ac_tool_prefix$ac_prog\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\nfi\nfi\nCXX=$ac_cv_prog_CXX\nif test -n \"$CXX\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $CXX\" >&5\n$as_echo \"$CXX\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n    test -n \"$CXX\" && break\n  done\nfi\nif test -z \"$CXX\"; then\n  ac_ct_CXX=$CXX\n  for ac_prog in g++ c++ gpp aCC CC cxx cc++ cl.exe FCC KCC RCC xlC_r xlC\ndo\n  # Extract the first word of \"$ac_prog\", so it can be a program name with args.\nset dummy $ac_prog; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_prog_ac_ct_CXX+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -n \"$ac_ct_CXX\"; then\n  ac_cv_prog_ac_ct_CXX=\"$ac_ct_CXX\" # Let the user override the test.\nelse\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_prog_ac_ct_CXX=\"$ac_prog\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\nfi\nfi\nac_ct_CXX=$ac_cv_prog_ac_ct_CXX\nif test -n \"$ac_ct_CXX\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_ct_CXX\" >&5\n$as_echo \"$ac_ct_CXX\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n  test -n \"$ac_ct_CXX\" && break\ndone\n\n  if test \"x$ac_ct_CXX\" = x; then\n    CXX=\"g++\"\n  else\n    case $cross_compiling:$ac_tool_warned in\nyes:)\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet\" >&5\n$as_echo \"$as_me: WARNING: using cross tools not prefixed with host triplet\" >&2;}\nac_tool_warned=yes ;;\nesac\n    CXX=$ac_ct_CXX\n  fi\nfi\n\n  fi\nfi\n# Provide some information about the compiler.\n$as_echo \"$as_me:${as_lineno-$LINENO}: checking for C++ compiler version\" >&5\nset X $ac_compile\nac_compiler=$2\nfor ac_option in --version -v -V -qversion; do\n  { { ac_try=\"$ac_compiler $ac_option >&5\"\ncase \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_compiler $ac_option >&5\") 2>conftest.err\n  ac_status=$?\n  if test -s conftest.err; then\n    sed '10a\\\n... rest of stderr output deleted ...\n         10q' conftest.err >conftest.er1\n    cat conftest.er1 >&5\n  fi\n  rm -f conftest.er1 conftest.err\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; }\ndone\n\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\nint\nmain ()\n{\n\n  ;\n  return 0;\n}\n_ACEOF\nac_clean_files_save=$ac_clean_files\nac_clean_files=\"$ac_clean_files a.out a.out.dSYM a.exe b.out\"\n# Try to create an executable without -o first, disregard a.out.\n# It will help us diagnose broken compilers, and finding out an intuition\n# of exeext.\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether the C++ compiler works\" >&5\n$as_echo_n \"checking whether the C++ compiler works... \" >&6; }\nac_link_default=`$as_echo \"$ac_link\" | sed 's/ -o *conftest[^ ]*//'`\n\n# The possible output files:\nac_files=\"a.out conftest.exe conftest a.exe a_out.exe b.out conftest.*\"\n\nac_rmfiles=\nfor ac_file in $ac_files\ndo\n  case $ac_file in\n    *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj ) ;;\n    * ) ac_rmfiles=\"$ac_rmfiles $ac_file\";;\n  esac\ndone\nrm -f $ac_rmfiles\n\nif { { ac_try=\"$ac_link_default\"\ncase \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_link_default\") 2>&5\n  ac_status=$?\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; }; then :\n  # Autoconf-2.13 could set the ac_cv_exeext variable to `no'.\n# So ignore a value of `no', otherwise this would lead to `EXEEXT = no'\n# in a Makefile.  We should not override ac_cv_exeext if it was cached,\n# so that the user can short-circuit this test for compilers unknown to\n# Autoconf.\nfor ac_file in $ac_files ''\ndo\n  test -f \"$ac_file\" || continue\n  case $ac_file in\n    *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj )\n\t;;\n    [ab].out )\n\t# We found the default executable, but exeext='' is most\n\t# certainly right.\n\tbreak;;\n    *.* )\n\tif test \"${ac_cv_exeext+set}\" = set && test \"$ac_cv_exeext\" != no;\n\tthen :; else\n\t   ac_cv_exeext=`expr \"$ac_file\" : '[^.]*\\(\\..*\\)'`\n\tfi\n\t# We set ac_cv_exeext here because the later test for it is not\n\t# safe: cross compilers may not add the suffix if given an `-o'\n\t# argument, so we may need to know it at that point already.\n\t# Even if this section looks crufty: it has the advantage of\n\t# actually working.\n\tbreak;;\n    * )\n\tbreak;;\n  esac\ndone\ntest \"$ac_cv_exeext\" = no && ac_cv_exeext=\n\nelse\n  ac_file=''\nfi\nif test -z \"$ac_file\"; then :\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\n$as_echo \"$as_me: failed program was:\" >&5\nsed 's/^/| /' conftest.$ac_ext >&5\n\n{ { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error 77 \"C++ compiler cannot create executables\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: yes\" >&5\n$as_echo \"yes\" >&6; }\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for C++ compiler default output file name\" >&5\n$as_echo_n \"checking for C++ compiler default output file name... \" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_file\" >&5\n$as_echo \"$ac_file\" >&6; }\nac_exeext=$ac_cv_exeext\n\nrm -f -r a.out a.out.dSYM a.exe conftest$ac_cv_exeext b.out\nac_clean_files=$ac_clean_files_save\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for suffix of executables\" >&5\n$as_echo_n \"checking for suffix of executables... \" >&6; }\nif { { ac_try=\"$ac_link\"\ncase \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_link\") 2>&5\n  ac_status=$?\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; }; then :\n  # If both `conftest.exe' and `conftest' are `present' (well, observable)\n# catch `conftest.exe'.  For instance with Cygwin, `ls conftest' will\n# work properly (i.e., refer to `conftest.exe'), while it won't with\n# `rm'.\nfor ac_file in conftest.exe conftest conftest.*; do\n  test -f \"$ac_file\" || continue\n  case $ac_file in\n    *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj ) ;;\n    *.* ) ac_cv_exeext=`expr \"$ac_file\" : '[^.]*\\(\\..*\\)'`\n\t  break;;\n    * ) break;;\n  esac\ndone\nelse\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot compute suffix of executables: cannot compile and link\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\nrm -f conftest conftest$ac_cv_exeext\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_exeext\" >&5\n$as_echo \"$ac_cv_exeext\" >&6; }\n\nrm -f conftest.$ac_ext\nEXEEXT=$ac_cv_exeext\nac_exeext=$EXEEXT\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <stdio.h>\nint\nmain ()\n{\nFILE *f = fopen (\"conftest.out\", \"w\");\n return ferror (f) || fclose (f) != 0;\n\n  ;\n  return 0;\n}\n_ACEOF\nac_clean_files=\"$ac_clean_files conftest.out\"\n# Check that the compiler produces executables we can run.  If not, either\n# the compiler is broken, or we cross compile.\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether we are cross compiling\" >&5\n$as_echo_n \"checking whether we are cross compiling... \" >&6; }\nif test \"$cross_compiling\" != yes; then\n  { { ac_try=\"$ac_link\"\ncase \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_link\") 2>&5\n  ac_status=$?\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; }\n  if { ac_try='./conftest$ac_cv_exeext'\n  { { case \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_try\") 2>&5\n  ac_status=$?\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; }; }; then\n    cross_compiling=no\n  else\n    if test \"$cross_compiling\" = maybe; then\n\tcross_compiling=yes\n    else\n\t{ { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run C++ compiled programs.\nIf you meant to cross compile, use \\`--host'.\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n    fi\n  fi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $cross_compiling\" >&5\n$as_echo \"$cross_compiling\" >&6; }\n\nrm -f conftest.$ac_ext conftest$ac_cv_exeext conftest.out\nac_clean_files=$ac_clean_files_save\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for suffix of object files\" >&5\n$as_echo_n \"checking for suffix of object files... \" >&6; }\nif ${ac_cv_objext+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\nint\nmain ()\n{\n\n  ;\n  return 0;\n}\n_ACEOF\nrm -f conftest.o conftest.obj\nif { { ac_try=\"$ac_compile\"\ncase \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_compile\") 2>&5\n  ac_status=$?\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; }; then :\n  for ac_file in conftest.o conftest.obj conftest.*; do\n  test -f \"$ac_file\" || continue;\n  case $ac_file in\n    *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM ) ;;\n    *) ac_cv_objext=`expr \"$ac_file\" : '.*\\.\\(.*\\)'`\n       break;;\n  esac\ndone\nelse\n  $as_echo \"$as_me: failed program was:\" >&5\nsed 's/^/| /' conftest.$ac_ext >&5\n\n{ { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot compute suffix of object files: cannot compile\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\nrm -f conftest.$ac_cv_objext conftest.$ac_ext\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_objext\" >&5\n$as_echo \"$ac_cv_objext\" >&6; }\nOBJEXT=$ac_cv_objext\nac_objext=$OBJEXT\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether we are using the GNU C++ compiler\" >&5\n$as_echo_n \"checking whether we are using the GNU C++ compiler... \" >&6; }\nif ${ac_cv_cxx_compiler_gnu+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\nint\nmain ()\n{\n#ifndef __GNUC__\n       choke me\n#endif\n\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"; then :\n  ac_compiler_gnu=yes\nelse\n  ac_compiler_gnu=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\nac_cv_cxx_compiler_gnu=$ac_compiler_gnu\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_cxx_compiler_gnu\" >&5\n$as_echo \"$ac_cv_cxx_compiler_gnu\" >&6; }\nif test $ac_compiler_gnu = yes; then\n  GXX=yes\nelse\n  GXX=\nfi\nac_test_CXXFLAGS=${CXXFLAGS+set}\nac_save_CXXFLAGS=$CXXFLAGS\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether $CXX accepts -g\" >&5\n$as_echo_n \"checking whether $CXX accepts -g... \" >&6; }\nif ${ac_cv_prog_cxx_g+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_save_cxx_werror_flag=$ac_cxx_werror_flag\n   ac_cxx_werror_flag=yes\n   ac_cv_prog_cxx_g=no\n   CXXFLAGS=\"-g\"\n   cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\nint\nmain ()\n{\n\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"; then :\n  ac_cv_prog_cxx_g=yes\nelse\n  CXXFLAGS=\"\"\n      cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\nint\nmain ()\n{\n\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"; then :\n\nelse\n  ac_cxx_werror_flag=$ac_save_cxx_werror_flag\n\t CXXFLAGS=\"-g\"\n\t cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\nint\nmain ()\n{\n\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"; then :\n  ac_cv_prog_cxx_g=yes\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\n   ac_cxx_werror_flag=$ac_save_cxx_werror_flag\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cxx_g\" >&5\n$as_echo \"$ac_cv_prog_cxx_g\" >&6; }\nif test \"$ac_test_CXXFLAGS\" = set; then\n  CXXFLAGS=$ac_save_CXXFLAGS\nelif test $ac_cv_prog_cxx_g = yes; then\n  if test \"$GXX\" = yes; then\n    CXXFLAGS=\"-g -O2\"\n  else\n    CXXFLAGS=\"-g\"\n  fi\nelse\n  if test \"$GXX\" = yes; then\n    CXXFLAGS=\"-O2\"\n  else\n    CXXFLAGS=\n  fi\nfi\nac_ext=cpp\nac_cpp='$CXXCPP $CPPFLAGS'\nac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\n\ndepcc=\"$CXX\"  am_compiler_list=\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking dependency style of $depcc\" >&5\n$as_echo_n \"checking dependency style of $depcc... \" >&6; }\nif ${am_cv_CXX_dependencies_compiler_type+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -z \"$AMDEP_TRUE\" && test -f \"$am_depcomp\"; then\n  # We make a subdir and do the tests there.  Otherwise we can end up\n  # making bogus files that we don't know about and never remove.  For\n  # instance it was reported that on HP-UX the gcc test will end up\n  # making a dummy file named 'D' -- because '-MD' means \"put the output\n  # in D\".\n  rm -rf conftest.dir\n  mkdir conftest.dir\n  # Copy depcomp to subdir because otherwise we won't find it if we're\n  # using a relative directory.\n  cp \"$am_depcomp\" conftest.dir\n  cd conftest.dir\n  # We will build objects and dependencies in a subdirectory because\n  # it helps to detect inapplicable dependency modes.  For instance\n  # both Tru64's cc and ICC support -MD to output dependencies as a\n  # side effect of compilation, but ICC will put the dependencies in\n  # the current directory while Tru64 will put them in the object\n  # directory.\n  mkdir sub\n\n  am_cv_CXX_dependencies_compiler_type=none\n  if test \"$am_compiler_list\" = \"\"; then\n     am_compiler_list=`sed -n 's/^#*\\([a-zA-Z0-9]*\\))$/\\1/p' < ./depcomp`\n  fi\n  am__universal=false\n  case \" $depcc \" in #(\n     *\\ -arch\\ *\\ -arch\\ *) am__universal=true ;;\n     esac\n\n  for depmode in $am_compiler_list; do\n    # Setup a source with many dependencies, because some compilers\n    # like to wrap large dependency lists on column 80 (with \\), and\n    # we should not choose a depcomp mode which is confused by this.\n    #\n    # We need to recreate these files for each test, as the compiler may\n    # overwrite some of them when testing with obscure command lines.\n    # This happens at least with the AIX C compiler.\n    : > sub/conftest.c\n    for i in 1 2 3 4 5 6; do\n      echo '#include \"conftst'$i'.h\"' >> sub/conftest.c\n      # Using \": > sub/conftst$i.h\" creates only sub/conftst1.h with\n      # Solaris 10 /bin/sh.\n      echo '/* dummy */' > sub/conftst$i.h\n    done\n    echo \"${am__include} ${am__quote}sub/conftest.Po${am__quote}\" > confmf\n\n    # We check with '-c' and '-o' for the sake of the \"dashmstdout\"\n    # mode.  It turns out that the SunPro C++ compiler does not properly\n    # handle '-M -o', and we need to detect this.  Also, some Intel\n    # versions had trouble with output in subdirs.\n    am__obj=sub/conftest.${OBJEXT-o}\n    am__minus_obj=\"-o $am__obj\"\n    case $depmode in\n    gcc)\n      # This depmode causes a compiler race in universal mode.\n      test \"$am__universal\" = false || continue\n      ;;\n    nosideeffect)\n      # After this tag, mechanisms are not by side-effect, so they'll\n      # only be used when explicitly requested.\n      if test \"x$enable_dependency_tracking\" = xyes; then\n\tcontinue\n      else\n\tbreak\n      fi\n      ;;\n    msvc7 | msvc7msys | msvisualcpp | msvcmsys)\n      # This compiler won't grok '-c -o', but also, the minuso test has\n      # not run yet.  These depmodes are late enough in the game, and\n      # so weak that their functioning should not be impacted.\n      am__obj=conftest.${OBJEXT-o}\n      am__minus_obj=\n      ;;\n    none) break ;;\n    esac\n    if depmode=$depmode \\\n       source=sub/conftest.c object=$am__obj \\\n       depfile=sub/conftest.Po tmpdepfile=sub/conftest.TPo \\\n       $SHELL ./depcomp $depcc -c $am__minus_obj sub/conftest.c \\\n         >/dev/null 2>conftest.err &&\n       grep sub/conftst1.h sub/conftest.Po > /dev/null 2>&1 &&\n       grep sub/conftst6.h sub/conftest.Po > /dev/null 2>&1 &&\n       grep $am__obj sub/conftest.Po > /dev/null 2>&1 &&\n       ${MAKE-make} -s -f confmf > /dev/null 2>&1; then\n      # icc doesn't choke on unknown options, it will just issue warnings\n      # or remarks (even with -Werror).  So we grep stderr for any message\n      # that says an option was ignored or not supported.\n      # When given -MP, icc 7.0 and 7.1 complain thusly:\n      #   icc: Command line warning: ignoring option '-M'; no argument required\n      # The diagnosis changed in icc 8.0:\n      #   icc: Command line remark: option '-MP' not supported\n      if (grep 'ignoring option' conftest.err ||\n          grep 'not supported' conftest.err) >/dev/null 2>&1; then :; else\n        am_cv_CXX_dependencies_compiler_type=$depmode\n        break\n      fi\n    fi\n  done\n\n  cd ..\n  rm -rf conftest.dir\nelse\n  am_cv_CXX_dependencies_compiler_type=none\nfi\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $am_cv_CXX_dependencies_compiler_type\" >&5\n$as_echo \"$am_cv_CXX_dependencies_compiler_type\" >&6; }\nCXXDEPMODE=depmode=$am_cv_CXX_dependencies_compiler_type\n\n if\n  test \"x$enable_dependency_tracking\" != xno \\\n  && test \"$am_cv_CXX_dependencies_compiler_type\" = gcc3; then\n  am__fastdepCXX_TRUE=\n  am__fastdepCXX_FALSE='#'\nelse\n  am__fastdepCXX_TRUE='#'\n  am__fastdepCXX_FALSE=\nfi\n\n\n\n  ax_cxx_compile_alternatives=\"14 1y\"    ax_cxx_compile_cxx14_required=false\n  ac_ext=cpp\nac_cpp='$CXXCPP $CPPFLAGS'\nac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\n  ac_success=no\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether $CXX supports C++14 features by default\" >&5\n$as_echo_n \"checking whether $CXX supports C++14 features by default... \" >&6; }\nif ${ax_cv_cxx_compile_cxx14+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n\n// If the compiler admits that it is not ready for C++11, why torture it?\n// Hopefully, this will speed up the test.\n\n#ifndef __cplusplus\n\n#error \"This is not a C++ compiler\"\n\n#elif __cplusplus < 201103L\n\n#error \"This is not a C++11 compiler\"\n\n#else\n\nnamespace cxx11\n{\n\n  namespace test_static_assert\n  {\n\n    template <typename T>\n    struct check\n    {\n      static_assert(sizeof(int) <= sizeof(T), \"not big enough\");\n    };\n\n  }\n\n  namespace test_final_override\n  {\n\n    struct Base\n    {\n      virtual void f() {}\n    };\n\n    struct Derived : public Base\n    {\n      virtual void f() override {}\n    };\n\n  }\n\n  namespace test_double_right_angle_brackets\n  {\n\n    template < typename T >\n    struct check {};\n\n    typedef check<void> single_type;\n    typedef check<check<void>> double_type;\n    typedef check<check<check<void>>> triple_type;\n    typedef check<check<check<check<void>>>> quadruple_type;\n\n  }\n\n  namespace test_decltype\n  {\n\n    int\n    f()\n    {\n      int a = 1;\n      decltype(a) b = 2;\n      return a + b;\n    }\n\n  }\n\n  namespace test_type_deduction\n  {\n\n    template < typename T1, typename T2 >\n    struct is_same\n    {\n      static const bool value = false;\n    };\n\n    template < typename T >\n    struct is_same<T, T>\n    {\n      static const bool value = true;\n    };\n\n    template < typename T1, typename T2 >\n    auto\n    add(T1 a1, T2 a2) -> decltype(a1 + a2)\n    {\n      return a1 + a2;\n    }\n\n    int\n    test(const int c, volatile int v)\n    {\n      static_assert(is_same<int, decltype(0)>::value == true, \"\");\n      static_assert(is_same<int, decltype(c)>::value == false, \"\");\n      static_assert(is_same<int, decltype(v)>::value == false, \"\");\n      auto ac = c;\n      auto av = v;\n      auto sumi = ac + av + 'x';\n      auto sumf = ac + av + 1.0;\n      static_assert(is_same<int, decltype(ac)>::value == true, \"\");\n      static_assert(is_same<int, decltype(av)>::value == true, \"\");\n      static_assert(is_same<int, decltype(sumi)>::value == true, \"\");\n      static_assert(is_same<int, decltype(sumf)>::value == false, \"\");\n      static_assert(is_same<int, decltype(add(c, v))>::value == true, \"\");\n      return (sumf > 0.0) ? sumi : add(c, v);\n    }\n\n  }\n\n  namespace test_noexcept\n  {\n\n    int f() { return 0; }\n    int g() noexcept { return 0; }\n\n    static_assert(noexcept(f()) == false, \"\");\n    static_assert(noexcept(g()) == true, \"\");\n\n  }\n\n  namespace test_constexpr\n  {\n\n    template < typename CharT >\n    unsigned long constexpr\n    strlen_c_r(const CharT *const s, const unsigned long acc) noexcept\n    {\n      return *s ? strlen_c_r(s + 1, acc + 1) : acc;\n    }\n\n    template < typename CharT >\n    unsigned long constexpr\n    strlen_c(const CharT *const s) noexcept\n    {\n      return strlen_c_r(s, 0UL);\n    }\n\n    static_assert(strlen_c(\"\") == 0UL, \"\");\n    static_assert(strlen_c(\"1\") == 1UL, \"\");\n    static_assert(strlen_c(\"example\") == 7UL, \"\");\n    static_assert(strlen_c(\"another\\0example\") == 7UL, \"\");\n\n  }\n\n  namespace test_rvalue_references\n  {\n\n    template < int N >\n    struct answer\n    {\n      static constexpr int value = N;\n    };\n\n    answer<1> f(int&)       { return answer<1>(); }\n    answer<2> f(const int&) { return answer<2>(); }\n    answer<3> f(int&&)      { return answer<3>(); }\n\n    void\n    test()\n    {\n      int i = 0;\n      const int c = 0;\n      static_assert(decltype(f(i))::value == 1, \"\");\n      static_assert(decltype(f(c))::value == 2, \"\");\n      static_assert(decltype(f(0))::value == 3, \"\");\n    }\n\n  }\n\n  namespace test_uniform_initialization\n  {\n\n    struct test\n    {\n      static const int zero {};\n      static const int one {1};\n    };\n\n    static_assert(test::zero == 0, \"\");\n    static_assert(test::one == 1, \"\");\n\n  }\n\n  namespace test_lambdas\n  {\n\n    void\n    test1()\n    {\n      auto lambda1 = [](){};\n      auto lambda2 = lambda1;\n      lambda1();\n      lambda2();\n    }\n\n    int\n    test2()\n    {\n      auto a = [](int i, int j){ return i + j; }(1, 2);\n      auto b = []() -> int { return '0'; }();\n      auto c = [=](){ return a + b; }();\n      auto d = [&](){ return c; }();\n      auto e = [a, &b](int x) mutable {\n        const auto identity = [](int y){ return y; };\n        for (auto i = 0; i < a; ++i)\n          a += b--;\n        return x + identity(a + b);\n      }(0);\n      return a + b + c + d + e;\n    }\n\n    int\n    test3()\n    {\n      const auto nullary = [](){ return 0; };\n      const auto unary = [](int x){ return x; };\n      using nullary_t = decltype(nullary);\n      using unary_t = decltype(unary);\n      const auto higher1st = [](nullary_t f){ return f(); };\n      const auto higher2nd = [unary](nullary_t f1){\n        return [unary, f1](unary_t f2){ return f2(unary(f1())); };\n      };\n      return higher1st(nullary) + higher2nd(nullary)(unary);\n    }\n\n  }\n\n  namespace test_variadic_templates\n  {\n\n    template <int...>\n    struct sum;\n\n    template <int N0, int... N1toN>\n    struct sum<N0, N1toN...>\n    {\n      static constexpr auto value = N0 + sum<N1toN...>::value;\n    };\n\n    template <>\n    struct sum<>\n    {\n      static constexpr auto value = 0;\n    };\n\n    static_assert(sum<>::value == 0, \"\");\n    static_assert(sum<1>::value == 1, \"\");\n    static_assert(sum<23>::value == 23, \"\");\n    static_assert(sum<1, 2>::value == 3, \"\");\n    static_assert(sum<5, 5, 11>::value == 21, \"\");\n    static_assert(sum<2, 3, 5, 7, 11, 13>::value == 41, \"\");\n\n  }\n\n  // http://stackoverflow.com/questions/13728184/template-aliases-and-sfinae\n  // Clang 3.1 fails with headers of libstd++ 4.8.3 when using std::function\n  // because of this.\n  namespace test_template_alias_sfinae\n  {\n\n    struct foo {};\n\n    template<typename T>\n    using member = typename T::member_type;\n\n    template<typename T>\n    void func(...) {}\n\n    template<typename T>\n    void func(member<T>*) {}\n\n    void test();\n\n    void test() { func<foo>(0); }\n\n  }\n\n}  // namespace cxx11\n\n#endif  // __cplusplus >= 201103L\n\n\n\n\n// If the compiler admits that it is not ready for C++14, why torture it?\n// Hopefully, this will speed up the test.\n\n#ifndef __cplusplus\n\n#error \"This is not a C++ compiler\"\n\n#elif __cplusplus < 201402L\n\n#error \"This is not a C++14 compiler\"\n\n#else\n\nnamespace cxx14\n{\n\n  namespace test_polymorphic_lambdas\n  {\n\n    int\n    test()\n    {\n      const auto lambda = [](auto&&... args){\n        const auto istiny = [](auto x){\n          return (sizeof(x) == 1UL) ? 1 : 0;\n        };\n        const int aretiny[] = { istiny(args)... };\n        return aretiny[0];\n      };\n      return lambda(1, 1L, 1.0f, '1');\n    }\n\n  }\n\n  namespace test_binary_literals\n  {\n\n    constexpr auto ivii = 0b0000000000101010;\n    static_assert(ivii == 42, \"wrong value\");\n\n  }\n\n  namespace test_generalized_constexpr\n  {\n\n    template < typename CharT >\n    constexpr unsigned long\n    strlen_c(const CharT *const s) noexcept\n    {\n      auto length = 0UL;\n      for (auto p = s; *p; ++p)\n        ++length;\n      return length;\n    }\n\n    static_assert(strlen_c(\"\") == 0UL, \"\");\n    static_assert(strlen_c(\"x\") == 1UL, \"\");\n    static_assert(strlen_c(\"test\") == 4UL, \"\");\n    static_assert(strlen_c(\"another\\0test\") == 7UL, \"\");\n\n  }\n\n  namespace test_lambda_init_capture\n  {\n\n    int\n    test()\n    {\n      auto x = 0;\n      const auto lambda1 = [a = x](int b){ return a + b; };\n      const auto lambda2 = [a = lambda1(x)](){ return a; };\n      return lambda2();\n    }\n\n  }\n\n  namespace test_digit_separators\n  {\n\n    constexpr auto ten_million = 100'000'000;\n    static_assert(ten_million == 100000000, \"\");\n\n  }\n\n  namespace test_return_type_deduction\n  {\n\n    auto f(int& x) { return x; }\n    decltype(auto) g(int& x) { return x; }\n\n    template < typename T1, typename T2 >\n    struct is_same\n    {\n      static constexpr auto value = false;\n    };\n\n    template < typename T >\n    struct is_same<T, T>\n    {\n      static constexpr auto value = true;\n    };\n\n    int\n    test()\n    {\n      auto x = 0;\n      static_assert(is_same<int, decltype(f(x))>::value, \"\");\n      static_assert(is_same<int&, decltype(g(x))>::value, \"\");\n      return x;\n    }\n\n  }\n\n}  // namespace cxx14\n\n#endif  // __cplusplus >= 201402L\n\n\n\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"; then :\n  ax_cv_cxx_compile_cxx14=yes\nelse\n  ax_cv_cxx_compile_cxx14=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ax_cv_cxx_compile_cxx14\" >&5\n$as_echo \"$ax_cv_cxx_compile_cxx14\" >&6; }\n  if test x$ax_cv_cxx_compile_cxx14 = xyes; then\n    ac_success=yes\n  fi\n\n\n\n    if test x$ac_success = xno; then\n                for alternative in ${ax_cxx_compile_alternatives}; do\n      for switch in -std=c++${alternative} +std=c++${alternative} \"-h std=c++${alternative}\"; do\n        cachevar=`$as_echo \"ax_cv_cxx_compile_cxx14_$switch\" | $as_tr_sh`\n        { $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether $CXX supports C++14 features with $switch\" >&5\n$as_echo_n \"checking whether $CXX supports C++14 features with $switch... \" >&6; }\nif eval \\${$cachevar+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_save_CXX=\"$CXX\"\n           CXX=\"$CXX $switch\"\n           cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n\n// If the compiler admits that it is not ready for C++11, why torture it?\n// Hopefully, this will speed up the test.\n\n#ifndef __cplusplus\n\n#error \"This is not a C++ compiler\"\n\n#elif __cplusplus < 201103L\n\n#error \"This is not a C++11 compiler\"\n\n#else\n\nnamespace cxx11\n{\n\n  namespace test_static_assert\n  {\n\n    template <typename T>\n    struct check\n    {\n      static_assert(sizeof(int) <= sizeof(T), \"not big enough\");\n    };\n\n  }\n\n  namespace test_final_override\n  {\n\n    struct Base\n    {\n      virtual void f() {}\n    };\n\n    struct Derived : public Base\n    {\n      virtual void f() override {}\n    };\n\n  }\n\n  namespace test_double_right_angle_brackets\n  {\n\n    template < typename T >\n    struct check {};\n\n    typedef check<void> single_type;\n    typedef check<check<void>> double_type;\n    typedef check<check<check<void>>> triple_type;\n    typedef check<check<check<check<void>>>> quadruple_type;\n\n  }\n\n  namespace test_decltype\n  {\n\n    int\n    f()\n    {\n      int a = 1;\n      decltype(a) b = 2;\n      return a + b;\n    }\n\n  }\n\n  namespace test_type_deduction\n  {\n\n    template < typename T1, typename T2 >\n    struct is_same\n    {\n      static const bool value = false;\n    };\n\n    template < typename T >\n    struct is_same<T, T>\n    {\n      static const bool value = true;\n    };\n\n    template < typename T1, typename T2 >\n    auto\n    add(T1 a1, T2 a2) -> decltype(a1 + a2)\n    {\n      return a1 + a2;\n    }\n\n    int\n    test(const int c, volatile int v)\n    {\n      static_assert(is_same<int, decltype(0)>::value == true, \"\");\n      static_assert(is_same<int, decltype(c)>::value == false, \"\");\n      static_assert(is_same<int, decltype(v)>::value == false, \"\");\n      auto ac = c;\n      auto av = v;\n      auto sumi = ac + av + 'x';\n      auto sumf = ac + av + 1.0;\n      static_assert(is_same<int, decltype(ac)>::value == true, \"\");\n      static_assert(is_same<int, decltype(av)>::value == true, \"\");\n      static_assert(is_same<int, decltype(sumi)>::value == true, \"\");\n      static_assert(is_same<int, decltype(sumf)>::value == false, \"\");\n      static_assert(is_same<int, decltype(add(c, v))>::value == true, \"\");\n      return (sumf > 0.0) ? sumi : add(c, v);\n    }\n\n  }\n\n  namespace test_noexcept\n  {\n\n    int f() { return 0; }\n    int g() noexcept { return 0; }\n\n    static_assert(noexcept(f()) == false, \"\");\n    static_assert(noexcept(g()) == true, \"\");\n\n  }\n\n  namespace test_constexpr\n  {\n\n    template < typename CharT >\n    unsigned long constexpr\n    strlen_c_r(const CharT *const s, const unsigned long acc) noexcept\n    {\n      return *s ? strlen_c_r(s + 1, acc + 1) : acc;\n    }\n\n    template < typename CharT >\n    unsigned long constexpr\n    strlen_c(const CharT *const s) noexcept\n    {\n      return strlen_c_r(s, 0UL);\n    }\n\n    static_assert(strlen_c(\"\") == 0UL, \"\");\n    static_assert(strlen_c(\"1\") == 1UL, \"\");\n    static_assert(strlen_c(\"example\") == 7UL, \"\");\n    static_assert(strlen_c(\"another\\0example\") == 7UL, \"\");\n\n  }\n\n  namespace test_rvalue_references\n  {\n\n    template < int N >\n    struct answer\n    {\n      static constexpr int value = N;\n    };\n\n    answer<1> f(int&)       { return answer<1>(); }\n    answer<2> f(const int&) { return answer<2>(); }\n    answer<3> f(int&&)      { return answer<3>(); }\n\n    void\n    test()\n    {\n      int i = 0;\n      const int c = 0;\n      static_assert(decltype(f(i))::value == 1, \"\");\n      static_assert(decltype(f(c))::value == 2, \"\");\n      static_assert(decltype(f(0))::value == 3, \"\");\n    }\n\n  }\n\n  namespace test_uniform_initialization\n  {\n\n    struct test\n    {\n      static const int zero {};\n      static const int one {1};\n    };\n\n    static_assert(test::zero == 0, \"\");\n    static_assert(test::one == 1, \"\");\n\n  }\n\n  namespace test_lambdas\n  {\n\n    void\n    test1()\n    {\n      auto lambda1 = [](){};\n      auto lambda2 = lambda1;\n      lambda1();\n      lambda2();\n    }\n\n    int\n    test2()\n    {\n      auto a = [](int i, int j){ return i + j; }(1, 2);\n      auto b = []() -> int { return '0'; }();\n      auto c = [=](){ return a + b; }();\n      auto d = [&](){ return c; }();\n      auto e = [a, &b](int x) mutable {\n        const auto identity = [](int y){ return y; };\n        for (auto i = 0; i < a; ++i)\n          a += b--;\n        return x + identity(a + b);\n      }(0);\n      return a + b + c + d + e;\n    }\n\n    int\n    test3()\n    {\n      const auto nullary = [](){ return 0; };\n      const auto unary = [](int x){ return x; };\n      using nullary_t = decltype(nullary);\n      using unary_t = decltype(unary);\n      const auto higher1st = [](nullary_t f){ return f(); };\n      const auto higher2nd = [unary](nullary_t f1){\n        return [unary, f1](unary_t f2){ return f2(unary(f1())); };\n      };\n      return higher1st(nullary) + higher2nd(nullary)(unary);\n    }\n\n  }\n\n  namespace test_variadic_templates\n  {\n\n    template <int...>\n    struct sum;\n\n    template <int N0, int... N1toN>\n    struct sum<N0, N1toN...>\n    {\n      static constexpr auto value = N0 + sum<N1toN...>::value;\n    };\n\n    template <>\n    struct sum<>\n    {\n      static constexpr auto value = 0;\n    };\n\n    static_assert(sum<>::value == 0, \"\");\n    static_assert(sum<1>::value == 1, \"\");\n    static_assert(sum<23>::value == 23, \"\");\n    static_assert(sum<1, 2>::value == 3, \"\");\n    static_assert(sum<5, 5, 11>::value == 21, \"\");\n    static_assert(sum<2, 3, 5, 7, 11, 13>::value == 41, \"\");\n\n  }\n\n  // http://stackoverflow.com/questions/13728184/template-aliases-and-sfinae\n  // Clang 3.1 fails with headers of libstd++ 4.8.3 when using std::function\n  // because of this.\n  namespace test_template_alias_sfinae\n  {\n\n    struct foo {};\n\n    template<typename T>\n    using member = typename T::member_type;\n\n    template<typename T>\n    void func(...) {}\n\n    template<typename T>\n    void func(member<T>*) {}\n\n    void test();\n\n    void test() { func<foo>(0); }\n\n  }\n\n}  // namespace cxx11\n\n#endif  // __cplusplus >= 201103L\n\n\n\n\n// If the compiler admits that it is not ready for C++14, why torture it?\n// Hopefully, this will speed up the test.\n\n#ifndef __cplusplus\n\n#error \"This is not a C++ compiler\"\n\n#elif __cplusplus < 201402L\n\n#error \"This is not a C++14 compiler\"\n\n#else\n\nnamespace cxx14\n{\n\n  namespace test_polymorphic_lambdas\n  {\n\n    int\n    test()\n    {\n      const auto lambda = [](auto&&... args){\n        const auto istiny = [](auto x){\n          return (sizeof(x) == 1UL) ? 1 : 0;\n        };\n        const int aretiny[] = { istiny(args)... };\n        return aretiny[0];\n      };\n      return lambda(1, 1L, 1.0f, '1');\n    }\n\n  }\n\n  namespace test_binary_literals\n  {\n\n    constexpr auto ivii = 0b0000000000101010;\n    static_assert(ivii == 42, \"wrong value\");\n\n  }\n\n  namespace test_generalized_constexpr\n  {\n\n    template < typename CharT >\n    constexpr unsigned long\n    strlen_c(const CharT *const s) noexcept\n    {\n      auto length = 0UL;\n      for (auto p = s; *p; ++p)\n        ++length;\n      return length;\n    }\n\n    static_assert(strlen_c(\"\") == 0UL, \"\");\n    static_assert(strlen_c(\"x\") == 1UL, \"\");\n    static_assert(strlen_c(\"test\") == 4UL, \"\");\n    static_assert(strlen_c(\"another\\0test\") == 7UL, \"\");\n\n  }\n\n  namespace test_lambda_init_capture\n  {\n\n    int\n    test()\n    {\n      auto x = 0;\n      const auto lambda1 = [a = x](int b){ return a + b; };\n      const auto lambda2 = [a = lambda1(x)](){ return a; };\n      return lambda2();\n    }\n\n  }\n\n  namespace test_digit_separators\n  {\n\n    constexpr auto ten_million = 100'000'000;\n    static_assert(ten_million == 100000000, \"\");\n\n  }\n\n  namespace test_return_type_deduction\n  {\n\n    auto f(int& x) { return x; }\n    decltype(auto) g(int& x) { return x; }\n\n    template < typename T1, typename T2 >\n    struct is_same\n    {\n      static constexpr auto value = false;\n    };\n\n    template < typename T >\n    struct is_same<T, T>\n    {\n      static constexpr auto value = true;\n    };\n\n    int\n    test()\n    {\n      auto x = 0;\n      static_assert(is_same<int, decltype(f(x))>::value, \"\");\n      static_assert(is_same<int&, decltype(g(x))>::value, \"\");\n      return x;\n    }\n\n  }\n\n}  // namespace cxx14\n\n#endif  // __cplusplus >= 201402L\n\n\n\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"; then :\n  eval $cachevar=yes\nelse\n  eval $cachevar=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\n           CXX=\"$ac_save_CXX\"\nfi\neval ac_res=\\$$cachevar\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\n        if eval test x\\$$cachevar = xyes; then\n          CXX=\"$CXX $switch\"\n          if test -n \"$CXXCPP\" ; then\n            CXXCPP=\"$CXXCPP $switch\"\n          fi\n          ac_success=yes\n          break\n        fi\n      done\n      if test x$ac_success = xyes; then\n        break\n      fi\n    done\n  fi\n  ac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\n  if test x$ax_cxx_compile_cxx14_required = xtrue; then\n    if test x$ac_success = xno; then\n      as_fn_error $? \"*** A compiler with support for C++14 language features is required.\" \"$LINENO\" 5\n    fi\n  fi\n  if test x$ac_success = xno; then\n    HAVE_CXX14=0\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: No compiler with C++14 support was found\" >&5\n$as_echo \"$as_me: No compiler with C++14 support was found\" >&6;}\n  else\n    HAVE_CXX14=1\n\n$as_echo \"#define HAVE_CXX14 1\" >>confdefs.h\n\n  fi\n\n\n\n  if test \"x$HAVE_CXX14\" = \"x1\" ; then\n    have_boost_147=\n    # Make sure we can run config.sub.\n$SHELL \"$ac_aux_dir/config.sub\" sun4 >/dev/null 2>&1 ||\n  as_fn_error $? \"cannot run $SHELL $ac_aux_dir/config.sub\" \"$LINENO\" 5\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking build system type\" >&5\n$as_echo_n \"checking build system type... \" >&6; }\nif ${ac_cv_build+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_build_alias=$build_alias\ntest \"x$ac_build_alias\" = x &&\n  ac_build_alias=`$SHELL \"$ac_aux_dir/config.guess\"`\ntest \"x$ac_build_alias\" = x &&\n  as_fn_error $? \"cannot guess build type; you must specify one\" \"$LINENO\" 5\nac_cv_build=`$SHELL \"$ac_aux_dir/config.sub\" $ac_build_alias` ||\n  as_fn_error $? \"$SHELL $ac_aux_dir/config.sub $ac_build_alias failed\" \"$LINENO\" 5\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_build\" >&5\n$as_echo \"$ac_cv_build\" >&6; }\ncase $ac_cv_build in\n*-*-*) ;;\n*) as_fn_error $? \"invalid value of canonical build\" \"$LINENO\" 5;;\nesac\nbuild=$ac_cv_build\nac_save_IFS=$IFS; IFS='-'\nset x $ac_cv_build\nshift\nbuild_cpu=$1\nbuild_vendor=$2\nshift; shift\n# Remember, the first character of IFS is used to create $*,\n# except with old shells:\nbuild_os=$*\nIFS=$ac_save_IFS\ncase $build_os in *\\ *) build_os=`echo \"$build_os\" | sed 's/ /-/g'`;; esac\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking host system type\" >&5\n$as_echo_n \"checking host system type... \" >&6; }\nif ${ac_cv_host+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"x$host_alias\" = x; then\n  ac_cv_host=$ac_cv_build\nelse\n  ac_cv_host=`$SHELL \"$ac_aux_dir/config.sub\" $host_alias` ||\n    as_fn_error $? \"$SHELL $ac_aux_dir/config.sub $host_alias failed\" \"$LINENO\" 5\nfi\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_host\" >&5\n$as_echo \"$ac_cv_host\" >&6; }\ncase $ac_cv_host in\n*-*-*) ;;\n*) as_fn_error $? \"invalid value of canonical host\" \"$LINENO\" 5;;\nesac\nhost=$ac_cv_host\nac_save_IFS=$IFS; IFS='-'\nset x $ac_cv_host\nshift\nhost_cpu=$1\nhost_vendor=$2\nshift; shift\n# Remember, the first character of IFS is used to create $*,\n# except with old shells:\nhost_os=$*\nIFS=$ac_save_IFS\ncase $host_os in *\\ *) host_os=`echo \"$host_os\" | sed 's/ /-/g'`;; esac\n\n\n\n\n# Check whether --with-boost was given.\nif test \"${with_boost+set}\" = set; then :\n  withval=$with_boost;\n    if test \"$withval\" = \"no\"; then\n        want_boost=\"no\"\n    elif test \"$withval\" = \"yes\"; then\n        want_boost=\"yes\"\n        ac_boost_path=\"\"\n    else\n        want_boost=\"yes\"\n        ac_boost_path=\"$withval\"\n    fi\n\nelse\n  want_boost=\"yes\"\nfi\n\n\n\n\n# Check whether --with-boost-libdir was given.\nif test \"${with_boost_libdir+set}\" = set; then :\n  withval=$with_boost_libdir;\n        if test -d \"$withval\"\n        then\n                ac_boost_lib_path=\"$withval\"\n        else\n                as_fn_error $? \"--with-boost-libdir expected directory name\" \"$LINENO\" 5\n        fi\n\nelse\n  ac_boost_lib_path=\"\"\n\nfi\n\n\nif test \"x$want_boost\" = \"xyes\"; then\n    boost_lib_version_req=1.47\n    boost_lib_version_req_shorten=`expr $boost_lib_version_req : '\\([0-9]*\\.[0-9]*\\)'`\n    boost_lib_version_req_major=`expr $boost_lib_version_req : '\\([0-9]*\\)'`\n    boost_lib_version_req_minor=`expr $boost_lib_version_req : '[0-9]*\\.\\([0-9]*\\)'`\n    boost_lib_version_req_sub_minor=`expr $boost_lib_version_req : '[0-9]*\\.[0-9]*\\.\\([0-9]*\\)'`\n    if test \"x$boost_lib_version_req_sub_minor\" = \"x\" ; then\n        boost_lib_version_req_sub_minor=\"0\"\n        fi\n    WANT_BOOST_VERSION=`expr $boost_lib_version_req_major \\* 100000 \\+  $boost_lib_version_req_minor \\* 100 \\+ $boost_lib_version_req_sub_minor`\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: checking for boostlib >= $boost_lib_version_req\" >&5\n$as_echo_n \"checking for boostlib >= $boost_lib_version_req... \" >&6; }\n    succeeded=no\n\n                        libsubdirs=\"lib\"\n    ax_arch=`uname -m`\n    case $ax_arch in\n      x86_64)\n        libsubdirs=\"lib64 libx32 lib lib64\"\n        ;;\n      ppc64|s390x|sparc64|aarch64|ppc64le)\n        libsubdirs=\"lib64 lib lib64 ppc64le\"\n        ;;\n    esac\n\n\n    libsubdirs=\"lib/${host_cpu}-${host_os} $libsubdirs\"\n\n    case ${host_cpu} in\n      i?86)\n        libsubdirs=\"lib/i386-${host_os} $libsubdirs\"\n        ;;\n    esac\n\n                if test \"$ac_boost_path\" != \"\"; then\n        BOOST_CPPFLAGS=\"-I$ac_boost_path/include\"\n        for ac_boost_path_tmp in $libsubdirs; do\n                if test -d \"$ac_boost_path\"/\"$ac_boost_path_tmp\" ; then\n                        BOOST_LDFLAGS=\"-L$ac_boost_path/$ac_boost_path_tmp\"\n                        break\n                fi\n        done\n    elif test \"$cross_compiling\" != yes; then\n        for ac_boost_path_tmp in /usr /usr/local /opt /opt/local ; do\n            if test -d \"$ac_boost_path_tmp/include/boost\" && test -r \"$ac_boost_path_tmp/include/boost\"; then\n                for libsubdir in $libsubdirs ; do\n                    if ls \"$ac_boost_path_tmp/$libsubdir/libboost_\"* >/dev/null 2>&1 ; then break; fi\n                done\n                BOOST_LDFLAGS=\"-L$ac_boost_path_tmp/$libsubdir\"\n                BOOST_CPPFLAGS=\"-I$ac_boost_path_tmp/include\"\n                break;\n            fi\n        done\n    fi\n\n            if test \"$ac_boost_lib_path\" != \"\"; then\n       BOOST_LDFLAGS=\"-L$ac_boost_lib_path\"\n    fi\n\n    CPPFLAGS_SAVED=\"$CPPFLAGS\"\n    CPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n    export CPPFLAGS\n\n    LDFLAGS_SAVED=\"$LDFLAGS\"\n    LDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n    export LDFLAGS\n\n\n    ac_ext=cpp\nac_cpp='$CXXCPP $CPPFLAGS'\nac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\n\n        cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n    #include <boost/version.hpp>\n\nint\nmain ()\n{\n\n    #if BOOST_VERSION >= $WANT_BOOST_VERSION\n    // Everything is okay\n    #else\n    #  error Boost version is too old\n    #endif\n\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"; then :\n\n        { $as_echo \"$as_me:${as_lineno-$LINENO}: result: yes\" >&5\n$as_echo \"yes\" >&6; }\n    succeeded=yes\n    found_system=yes\n\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\n    ac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\n\n\n\n            if test \"x$succeeded\" != \"xyes\"; then\n        _version=0\n        if test \"$ac_boost_path\" != \"\"; then\n            if test -d \"$ac_boost_path\" && test -r \"$ac_boost_path\"; then\n                for i in `ls -d $ac_boost_path/include/boost-* 2>/dev/null`; do\n                    _version_tmp=`echo $i | sed \"s#$ac_boost_path##\" | sed 's/\\/include\\/boost-//' | sed 's/_/./'`\n                    V_CHECK=`expr $_version_tmp \\> $_version`\n                    if test \"$V_CHECK\" = \"1\" ; then\n                        _version=$_version_tmp\n                    fi\n                    VERSION_UNDERSCORE=`echo $_version | sed 's/\\./_/'`\n                    BOOST_CPPFLAGS=\"-I$ac_boost_path/include/boost-$VERSION_UNDERSCORE\"\n                done\n            fi\n        else\n            if test \"$cross_compiling\" != yes; then\n                for ac_boost_path in /usr /usr/local /opt /opt/local ; do\n                    if test -d \"$ac_boost_path\" && test -r \"$ac_boost_path\"; then\n                        for i in `ls -d $ac_boost_path/include/boost-* 2>/dev/null`; do\n                            _version_tmp=`echo $i | sed \"s#$ac_boost_path##\" | sed 's/\\/include\\/boost-//' | sed 's/_/./'`\n                            V_CHECK=`expr $_version_tmp \\> $_version`\n                            if test \"$V_CHECK\" = \"1\" ; then\n                                _version=$_version_tmp\n                                best_path=$ac_boost_path\n                            fi\n                        done\n                    fi\n                done\n\n                VERSION_UNDERSCORE=`echo $_version | sed 's/\\./_/'`\n                BOOST_CPPFLAGS=\"-I$best_path/include/boost-$VERSION_UNDERSCORE\"\n                if test \"$ac_boost_lib_path\" = \"\"; then\n                    for libsubdir in $libsubdirs ; do\n                        if ls \"$best_path/$libsubdir/libboost_\"* >/dev/null 2>&1 ; then break; fi\n                    done\n                    BOOST_LDFLAGS=\"-L$best_path/$libsubdir\"\n                fi\n            fi\n\n            if test \"x$BOOST_ROOT\" != \"x\"; then\n                for libsubdir in $libsubdirs ; do\n                    if ls \"$BOOST_ROOT/stage/$libsubdir/libboost_\"* >/dev/null 2>&1 ; then break; fi\n                done\n                if test -d \"$BOOST_ROOT\" && test -r \"$BOOST_ROOT\" && test -d \"$BOOST_ROOT/stage/$libsubdir\" && test -r \"$BOOST_ROOT/stage/$libsubdir\"; then\n                    version_dir=`expr //$BOOST_ROOT : '.*/\\(.*\\)'`\n                    stage_version=`echo $version_dir | sed 's/boost_//' | sed 's/_/./g'`\n                        stage_version_shorten=`expr $stage_version : '\\([0-9]*\\.[0-9]*\\)'`\n                    V_CHECK=`expr $stage_version_shorten \\>\\= $_version`\n                    if test \"$V_CHECK\" = \"1\" -a \"$ac_boost_lib_path\" = \"\" ; then\n                        { $as_echo \"$as_me:${as_lineno-$LINENO}: We will use a staged boost library from $BOOST_ROOT\" >&5\n$as_echo \"$as_me: We will use a staged boost library from $BOOST_ROOT\" >&6;}\n                        BOOST_CPPFLAGS=\"-I$BOOST_ROOT\"\n                        BOOST_LDFLAGS=\"-L$BOOST_ROOT/stage/$libsubdir\"\n                    fi\n                fi\n            fi\n        fi\n\n        CPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n        export CPPFLAGS\n        LDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n        export LDFLAGS\n\n        ac_ext=cpp\nac_cpp='$CXXCPP $CPPFLAGS'\nac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\n\n            cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n        #include <boost/version.hpp>\n\nint\nmain ()\n{\n\n        #if BOOST_VERSION >= $WANT_BOOST_VERSION\n        // Everything is okay\n        #else\n        #  error Boost version is too old\n        #endif\n\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"; then :\n\n            { $as_echo \"$as_me:${as_lineno-$LINENO}: result: yes\" >&5\n$as_echo \"yes\" >&6; }\n        succeeded=yes\n        found_system=yes\n\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\n        ac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\n    fi\n\n    if test \"$succeeded\" != \"yes\" ; then\n        if test \"$_version\" = \"0\" ; then\n            { $as_echo \"$as_me:${as_lineno-$LINENO}: We could not detect the boost libraries (version $boost_lib_version_req_shorten or higher). If you have a staged boost library (still not installed) please specify \\$BOOST_ROOT in your environment and do not give a PATH to --with-boost option.  If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.\" >&5\n$as_echo \"$as_me: We could not detect the boost libraries (version $boost_lib_version_req_shorten or higher). If you have a staged boost library (still not installed) please specify \\$BOOST_ROOT in your environment and do not give a PATH to --with-boost option.  If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.\" >&6;}\n        else\n            { $as_echo \"$as_me:${as_lineno-$LINENO}: Your boost libraries seems to old (version $_version).\" >&5\n$as_echo \"$as_me: Your boost libraries seems to old (version $_version).\" >&6;}\n        fi\n        # execute ACTION-IF-NOT-FOUND (if present):\n        have_boost_147=no\n    else\n\n\n\n$as_echo \"#define HAVE_BOOST /**/\" >>confdefs.h\n\n        # execute ACTION-IF-FOUND (if present):\n        have_boost_147=yes\n    fi\n\n    CPPFLAGS=\"$CPPFLAGS_SAVED\"\n    LDFLAGS=\"$LDFLAGS_SAVED\"\nfi\n\n\n    ac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\nif test -n \"$ac_tool_prefix\"; then\n  # Extract the first word of \"${ac_tool_prefix}gcc\", so it can be a program name with args.\nset dummy ${ac_tool_prefix}gcc; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_prog_CC+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -n \"$CC\"; then\n  ac_cv_prog_CC=\"$CC\" # Let the user override the test.\nelse\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_prog_CC=\"${ac_tool_prefix}gcc\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\nfi\nfi\nCC=$ac_cv_prog_CC\nif test -n \"$CC\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $CC\" >&5\n$as_echo \"$CC\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\nfi\nif test -z \"$ac_cv_prog_CC\"; then\n  ac_ct_CC=$CC\n  # Extract the first word of \"gcc\", so it can be a program name with args.\nset dummy gcc; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_prog_ac_ct_CC+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -n \"$ac_ct_CC\"; then\n  ac_cv_prog_ac_ct_CC=\"$ac_ct_CC\" # Let the user override the test.\nelse\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_prog_ac_ct_CC=\"gcc\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\nfi\nfi\nac_ct_CC=$ac_cv_prog_ac_ct_CC\nif test -n \"$ac_ct_CC\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_ct_CC\" >&5\n$as_echo \"$ac_ct_CC\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n  if test \"x$ac_ct_CC\" = x; then\n    CC=\"\"\n  else\n    case $cross_compiling:$ac_tool_warned in\nyes:)\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet\" >&5\n$as_echo \"$as_me: WARNING: using cross tools not prefixed with host triplet\" >&2;}\nac_tool_warned=yes ;;\nesac\n    CC=$ac_ct_CC\n  fi\nelse\n  CC=\"$ac_cv_prog_CC\"\nfi\n\nif test -z \"$CC\"; then\n          if test -n \"$ac_tool_prefix\"; then\n    # Extract the first word of \"${ac_tool_prefix}cc\", so it can be a program name with args.\nset dummy ${ac_tool_prefix}cc; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_prog_CC+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -n \"$CC\"; then\n  ac_cv_prog_CC=\"$CC\" # Let the user override the test.\nelse\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_prog_CC=\"${ac_tool_prefix}cc\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\nfi\nfi\nCC=$ac_cv_prog_CC\nif test -n \"$CC\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $CC\" >&5\n$as_echo \"$CC\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n  fi\nfi\nif test -z \"$CC\"; then\n  # Extract the first word of \"cc\", so it can be a program name with args.\nset dummy cc; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_prog_CC+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -n \"$CC\"; then\n  ac_cv_prog_CC=\"$CC\" # Let the user override the test.\nelse\n  ac_prog_rejected=no\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    if test \"$as_dir/$ac_word$ac_exec_ext\" = \"/usr/ucb/cc\"; then\n       ac_prog_rejected=yes\n       continue\n     fi\n    ac_cv_prog_CC=\"cc\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\nif test $ac_prog_rejected = yes; then\n  # We found a bogon in the path, so make sure we never use it.\n  set dummy $ac_cv_prog_CC\n  shift\n  if test $# != 0; then\n    # We chose a different compiler from the bogus one.\n    # However, it has the same basename, so the bogon will be chosen\n    # first if we set CC to just the basename; use the full file name.\n    shift\n    ac_cv_prog_CC=\"$as_dir/$ac_word${1+' '}$@\"\n  fi\nfi\nfi\nfi\nCC=$ac_cv_prog_CC\nif test -n \"$CC\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $CC\" >&5\n$as_echo \"$CC\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\nfi\nif test -z \"$CC\"; then\n  if test -n \"$ac_tool_prefix\"; then\n  for ac_prog in cl.exe\n  do\n    # Extract the first word of \"$ac_tool_prefix$ac_prog\", so it can be a program name with args.\nset dummy $ac_tool_prefix$ac_prog; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_prog_CC+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -n \"$CC\"; then\n  ac_cv_prog_CC=\"$CC\" # Let the user override the test.\nelse\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_prog_CC=\"$ac_tool_prefix$ac_prog\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\nfi\nfi\nCC=$ac_cv_prog_CC\nif test -n \"$CC\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $CC\" >&5\n$as_echo \"$CC\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n    test -n \"$CC\" && break\n  done\nfi\nif test -z \"$CC\"; then\n  ac_ct_CC=$CC\n  for ac_prog in cl.exe\ndo\n  # Extract the first word of \"$ac_prog\", so it can be a program name with args.\nset dummy $ac_prog; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_prog_ac_ct_CC+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -n \"$ac_ct_CC\"; then\n  ac_cv_prog_ac_ct_CC=\"$ac_ct_CC\" # Let the user override the test.\nelse\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_prog_ac_ct_CC=\"$ac_prog\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\nfi\nfi\nac_ct_CC=$ac_cv_prog_ac_ct_CC\nif test -n \"$ac_ct_CC\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_ct_CC\" >&5\n$as_echo \"$ac_ct_CC\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n  test -n \"$ac_ct_CC\" && break\ndone\n\n  if test \"x$ac_ct_CC\" = x; then\n    CC=\"\"\n  else\n    case $cross_compiling:$ac_tool_warned in\nyes:)\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet\" >&5\n$as_echo \"$as_me: WARNING: using cross tools not prefixed with host triplet\" >&2;}\nac_tool_warned=yes ;;\nesac\n    CC=$ac_ct_CC\n  fi\nfi\n\nfi\n\n\ntest -z \"$CC\" && { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"no acceptable C compiler found in \\$PATH\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\n# Provide some information about the compiler.\n$as_echo \"$as_me:${as_lineno-$LINENO}: checking for C compiler version\" >&5\nset X $ac_compile\nac_compiler=$2\nfor ac_option in --version -v -V -qversion; do\n  { { ac_try=\"$ac_compiler $ac_option >&5\"\ncase \"(($ac_try\" in\n  *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n  *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\n$as_echo \"$ac_try_echo\"; } >&5\n  (eval \"$ac_compiler $ac_option >&5\") 2>conftest.err\n  ac_status=$?\n  if test -s conftest.err; then\n    sed '10a\\\n... rest of stderr output deleted ...\n         10q' conftest.err >conftest.er1\n    cat conftest.er1 >&5\n  fi\n  rm -f conftest.er1 conftest.err\n  $as_echo \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n  test $ac_status = 0; }\ndone\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether we are using the GNU C compiler\" >&5\n$as_echo_n \"checking whether we are using the GNU C compiler... \" >&6; }\nif ${ac_cv_c_compiler_gnu+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\nint\nmain ()\n{\n#ifndef __GNUC__\n       choke me\n#endif\n\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_compile \"$LINENO\"; then :\n  ac_compiler_gnu=yes\nelse\n  ac_compiler_gnu=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\nac_cv_c_compiler_gnu=$ac_compiler_gnu\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_c_compiler_gnu\" >&5\n$as_echo \"$ac_cv_c_compiler_gnu\" >&6; }\nif test $ac_compiler_gnu = yes; then\n  GCC=yes\nelse\n  GCC=\nfi\nac_test_CFLAGS=${CFLAGS+set}\nac_save_CFLAGS=$CFLAGS\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether $CC accepts -g\" >&5\n$as_echo_n \"checking whether $CC accepts -g... \" >&6; }\nif ${ac_cv_prog_cc_g+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_save_c_werror_flag=$ac_c_werror_flag\n   ac_c_werror_flag=yes\n   ac_cv_prog_cc_g=no\n   CFLAGS=\"-g\"\n   cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\nint\nmain ()\n{\n\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_compile \"$LINENO\"; then :\n  ac_cv_prog_cc_g=yes\nelse\n  CFLAGS=\"\"\n      cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\nint\nmain ()\n{\n\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_compile \"$LINENO\"; then :\n\nelse\n  ac_c_werror_flag=$ac_save_c_werror_flag\n\t CFLAGS=\"-g\"\n\t cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\nint\nmain ()\n{\n\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_compile \"$LINENO\"; then :\n  ac_cv_prog_cc_g=yes\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\n   ac_c_werror_flag=$ac_save_c_werror_flag\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cc_g\" >&5\n$as_echo \"$ac_cv_prog_cc_g\" >&6; }\nif test \"$ac_test_CFLAGS\" = set; then\n  CFLAGS=$ac_save_CFLAGS\nelif test $ac_cv_prog_cc_g = yes; then\n  if test \"$GCC\" = yes; then\n    CFLAGS=\"-g -O2\"\n  else\n    CFLAGS=\"-g\"\n  fi\nelse\n  if test \"$GCC\" = yes; then\n    CFLAGS=\"-O2\"\n  else\n    CFLAGS=\n  fi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $CC option to accept ISO C89\" >&5\n$as_echo_n \"checking for $CC option to accept ISO C89... \" >&6; }\nif ${ac_cv_prog_cc_c89+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_cv_prog_cc_c89=no\nac_save_CC=$CC\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <stdarg.h>\n#include <stdio.h>\nstruct stat;\n/* Most of the following tests are stolen from RCS 5.7's src/conf.sh.  */\nstruct buf { int x; };\nFILE * (*rcsopen) (struct buf *, struct stat *, int);\nstatic char *e (p, i)\n     char **p;\n     int i;\n{\n  return p[i];\n}\nstatic char *f (char * (*g) (char **, int), char **p, ...)\n{\n  char *s;\n  va_list v;\n  va_start (v,p);\n  s = g (p, va_arg (v,int));\n  va_end (v);\n  return s;\n}\n\n/* OSF 4.0 Compaq cc is some sort of almost-ANSI by default.  It has\n   function prototypes and stuff, but not '\\xHH' hex character constants.\n   These don't provoke an error unfortunately, instead are silently treated\n   as 'x'.  The following induces an error, until -std is added to get\n   proper ANSI mode.  Curiously '\\x00'!='x' always comes out true, for an\n   array size at least.  It's necessary to write '\\x00'==0 to get something\n   that's true only with -std.  */\nint osf4_cc_array ['\\x00' == 0 ? 1 : -1];\n\n/* IBM C 6 for AIX is almost-ANSI by default, but it replaces macro parameters\n   inside strings and character constants.  */\n#define FOO(x) 'x'\nint xlc6_cc_array[FOO(a) == 'x' ? 1 : -1];\n\nint test (int i, double x);\nstruct s1 {int (*f) (int a);};\nstruct s2 {int (*f) (double a);};\nint pairnames (int, char **, FILE *(*)(struct buf *, struct stat *, int), int, int);\nint argc;\nchar **argv;\nint\nmain ()\n{\nreturn f (e, argv, 0) != argv[0]  ||  f (e, argv, 1) != argv[1];\n  ;\n  return 0;\n}\n_ACEOF\nfor ac_arg in '' -qlanglvl=extc89 -qlanglvl=ansi -std \\\n\t-Ae \"-Aa -D_HPUX_SOURCE\" \"-Xc -D__EXTENSIONS__\"\ndo\n  CC=\"$ac_save_CC $ac_arg\"\n  if ac_fn_c_try_compile \"$LINENO\"; then :\n  ac_cv_prog_cc_c89=$ac_arg\nfi\nrm -f core conftest.err conftest.$ac_objext\n  test \"x$ac_cv_prog_cc_c89\" != \"xno\" && break\ndone\nrm -f conftest.$ac_ext\nCC=$ac_save_CC\n\nfi\n# AC_CACHE_VAL\ncase \"x$ac_cv_prog_cc_c89\" in\n  x)\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: result: none needed\" >&5\n$as_echo \"none needed\" >&6; } ;;\n  xno)\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: result: unsupported\" >&5\n$as_echo \"unsupported\" >&6; } ;;\n  *)\n    CC=\"$CC $ac_cv_prog_cc_c89\"\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cc_c89\" >&5\n$as_echo \"$ac_cv_prog_cc_c89\" >&6; } ;;\nesac\nif test \"x$ac_cv_prog_cc_c89\" != xno; then :\n\nfi\n\nac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\nac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether $CC understands -c and -o together\" >&5\n$as_echo_n \"checking whether $CC understands -c and -o together... \" >&6; }\nif ${am_cv_prog_cc_c_o+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\nint\nmain ()\n{\n\n  ;\n  return 0;\n}\n_ACEOF\n  # Make sure it works both with $CC and with simple cc.\n  # Following AC_PROG_CC_C_O, we do the test twice because some\n  # compilers refuse to overwrite an existing .o file with -o,\n  # though they will create one.\n  am_cv_prog_cc_c_o=yes\n  for am_i in 1 2; do\n    if { echo \"$as_me:$LINENO: $CC -c conftest.$ac_ext -o conftest2.$ac_objext\" >&5\n   ($CC -c conftest.$ac_ext -o conftest2.$ac_objext) >&5 2>&5\n   ac_status=$?\n   echo \"$as_me:$LINENO: \\$? = $ac_status\" >&5\n   (exit $ac_status); } \\\n         && test -f conftest2.$ac_objext; then\n      : OK\n    else\n      am_cv_prog_cc_c_o=no\n      break\n    fi\n  done\n  rm -f core conftest*\n  unset am_i\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $am_cv_prog_cc_c_o\" >&5\n$as_echo \"$am_cv_prog_cc_c_o\" >&6; }\nif test \"$am_cv_prog_cc_c_o\" != yes; then\n   # Losing compiler, so override with the script.\n   # FIXME: It is wrong to rewrite CC.\n   # But if we don't then we get into trouble of one sort or another.\n   # A longer-term fix would be to have automake use am__CC in this case,\n   # and then we could set am__CC=\"\\$(top_srcdir)/compile \\$(CC)\"\n   CC=\"$am_aux_dir/compile $CC\"\nfi\nac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\n\ndepcc=\"$CC\"   am_compiler_list=\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking dependency style of $depcc\" >&5\n$as_echo_n \"checking dependency style of $depcc... \" >&6; }\nif ${am_cv_CC_dependencies_compiler_type+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -z \"$AMDEP_TRUE\" && test -f \"$am_depcomp\"; then\n  # We make a subdir and do the tests there.  Otherwise we can end up\n  # making bogus files that we don't know about and never remove.  For\n  # instance it was reported that on HP-UX the gcc test will end up\n  # making a dummy file named 'D' -- because '-MD' means \"put the output\n  # in D\".\n  rm -rf conftest.dir\n  mkdir conftest.dir\n  # Copy depcomp to subdir because otherwise we won't find it if we're\n  # using a relative directory.\n  cp \"$am_depcomp\" conftest.dir\n  cd conftest.dir\n  # We will build objects and dependencies in a subdirectory because\n  # it helps to detect inapplicable dependency modes.  For instance\n  # both Tru64's cc and ICC support -MD to output dependencies as a\n  # side effect of compilation, but ICC will put the dependencies in\n  # the current directory while Tru64 will put them in the object\n  # directory.\n  mkdir sub\n\n  am_cv_CC_dependencies_compiler_type=none\n  if test \"$am_compiler_list\" = \"\"; then\n     am_compiler_list=`sed -n 's/^#*\\([a-zA-Z0-9]*\\))$/\\1/p' < ./depcomp`\n  fi\n  am__universal=false\n  case \" $depcc \" in #(\n     *\\ -arch\\ *\\ -arch\\ *) am__universal=true ;;\n     esac\n\n  for depmode in $am_compiler_list; do\n    # Setup a source with many dependencies, because some compilers\n    # like to wrap large dependency lists on column 80 (with \\), and\n    # we should not choose a depcomp mode which is confused by this.\n    #\n    # We need to recreate these files for each test, as the compiler may\n    # overwrite some of them when testing with obscure command lines.\n    # This happens at least with the AIX C compiler.\n    : > sub/conftest.c\n    for i in 1 2 3 4 5 6; do\n      echo '#include \"conftst'$i'.h\"' >> sub/conftest.c\n      # Using \": > sub/conftst$i.h\" creates only sub/conftst1.h with\n      # Solaris 10 /bin/sh.\n      echo '/* dummy */' > sub/conftst$i.h\n    done\n    echo \"${am__include} ${am__quote}sub/conftest.Po${am__quote}\" > confmf\n\n    # We check with '-c' and '-o' for the sake of the \"dashmstdout\"\n    # mode.  It turns out that the SunPro C++ compiler does not properly\n    # handle '-M -o', and we need to detect this.  Also, some Intel\n    # versions had trouble with output in subdirs.\n    am__obj=sub/conftest.${OBJEXT-o}\n    am__minus_obj=\"-o $am__obj\"\n    case $depmode in\n    gcc)\n      # This depmode causes a compiler race in universal mode.\n      test \"$am__universal\" = false || continue\n      ;;\n    nosideeffect)\n      # After this tag, mechanisms are not by side-effect, so they'll\n      # only be used when explicitly requested.\n      if test \"x$enable_dependency_tracking\" = xyes; then\n\tcontinue\n      else\n\tbreak\n      fi\n      ;;\n    msvc7 | msvc7msys | msvisualcpp | msvcmsys)\n      # This compiler won't grok '-c -o', but also, the minuso test has\n      # not run yet.  These depmodes are late enough in the game, and\n      # so weak that their functioning should not be impacted.\n      am__obj=conftest.${OBJEXT-o}\n      am__minus_obj=\n      ;;\n    none) break ;;\n    esac\n    if depmode=$depmode \\\n       source=sub/conftest.c object=$am__obj \\\n       depfile=sub/conftest.Po tmpdepfile=sub/conftest.TPo \\\n       $SHELL ./depcomp $depcc -c $am__minus_obj sub/conftest.c \\\n         >/dev/null 2>conftest.err &&\n       grep sub/conftst1.h sub/conftest.Po > /dev/null 2>&1 &&\n       grep sub/conftst6.h sub/conftest.Po > /dev/null 2>&1 &&\n       grep $am__obj sub/conftest.Po > /dev/null 2>&1 &&\n       ${MAKE-make} -s -f confmf > /dev/null 2>&1; then\n      # icc doesn't choke on unknown options, it will just issue warnings\n      # or remarks (even with -Werror).  So we grep stderr for any message\n      # that says an option was ignored or not supported.\n      # When given -MP, icc 7.0 and 7.1 complain thusly:\n      #   icc: Command line warning: ignoring option '-M'; no argument required\n      # The diagnosis changed in icc 8.0:\n      #   icc: Command line remark: option '-MP' not supported\n      if (grep 'ignoring option' conftest.err ||\n          grep 'not supported' conftest.err) >/dev/null 2>&1; then :; else\n        am_cv_CC_dependencies_compiler_type=$depmode\n        break\n      fi\n    fi\n  done\n\n  cd ..\n  rm -rf conftest.dir\nelse\n  am_cv_CC_dependencies_compiler_type=none\nfi\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $am_cv_CC_dependencies_compiler_type\" >&5\n$as_echo \"$am_cv_CC_dependencies_compiler_type\" >&6; }\nCCDEPMODE=depmode=$am_cv_CC_dependencies_compiler_type\n\n if\n  test \"x$enable_dependency_tracking\" != xno \\\n  && test \"$am_cv_CC_dependencies_compiler_type\" = gcc3; then\n  am__fastdepCC_TRUE=\n  am__fastdepCC_FALSE='#'\nelse\n  am__fastdepCC_TRUE='#'\n  am__fastdepCC_FALSE=\nfi\n\n\n\n\n\n# Check whether --with-boost-system was given.\nif test \"${with_boost_system+set}\" = set; then :\n  withval=$with_boost_system;\n        if test \"$withval\" = \"no\"; then\n\t\t\twant_boost=\"no\"\n        elif test \"$withval\" = \"yes\"; then\n            want_boost=\"yes\"\n            ax_boost_user_system_lib=\"\"\n        else\n\t\t    want_boost=\"yes\"\n\t\tax_boost_user_system_lib=\"$withval\"\n\t\tfi\n\nelse\n  want_boost=\"yes\"\n\nfi\n\n\n\tif test \"x$want_boost\" = \"xyes\"; then\n\n\n\t\tCPPFLAGS_SAVED=\"$CPPFLAGS\"\n\t\tCPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n\t\texport CPPFLAGS\n\n\t\tLDFLAGS_SAVED=\"$LDFLAGS\"\n\t\tLDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n\t\texport LDFLAGS\n\n        { $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether the Boost::System library is available\" >&5\n$as_echo_n \"checking whether the Boost::System library is available... \" >&6; }\nif ${ax_cv_boost_system+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=cpp\nac_cpp='$CXXCPP $CPPFLAGS'\nac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\n\n\t\t\t CXXFLAGS_SAVE=$CXXFLAGS\n\n\t\t\t cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <boost/system/error_code.hpp>\nint\nmain ()\n{\nboost::system::system_category\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"; then :\n  ax_cv_boost_system=yes\nelse\n  ax_cv_boost_system=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\n\t\t\t CXXFLAGS=$CXXFLAGS_SAVE\n             ac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ax_cv_boost_system\" >&5\n$as_echo \"$ax_cv_boost_system\" >&6; }\n\t\tif test \"x$ax_cv_boost_system\" = \"xyes\"; then\n\n\n\n$as_echo \"#define HAVE_BOOST_SYSTEM /**/\" >>confdefs.h\n\n            BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/[^\\/]*//'`\n\n\t\t\tLDFLAGS_SAVE=$LDFLAGS\n            if test \"x$ax_boost_user_system_lib\" = \"x\"; then\n                for libextension in `ls -r $BOOSTLIBDIR/libboost_system* 2>/dev/null | sed 's,.*/lib,,' | sed 's,\\..*,,'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    as_ac_Lib=`$as_echo \"ac_cv_lib_$ax_lib''_exit\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for exit in -l$ax_lib\" >&5\n$as_echo_n \"checking for exit in -l$ax_lib... \" >&6; }\nif eval \\${$as_ac_Lib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_check_lib_save_LIBS=$LIBS\nLIBS=\"-l$ax_lib  $LIBS\"\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n/* Override any GCC internal prototype to avoid an error.\n   Use char because int might match the return type of a GCC\n   builtin and then its argument prototype would still apply.  */\n#ifdef __cplusplus\nextern \"C\"\n#endif\nchar exit ();\nint\nmain ()\n{\nreturn exit ();\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_link \"$LINENO\"; then :\n  eval \"$as_ac_Lib=yes\"\nelse\n  eval \"$as_ac_Lib=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext \\\n    conftest$ac_exeext conftest.$ac_ext\nLIBS=$ac_check_lib_save_LIBS\nfi\neval ac_res=\\$$as_ac_Lib\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_Lib\"\\\" = x\"yes\"; then :\n  BOOST_SYSTEM_LIB=\"-l$ax_lib\";  link_system=\"yes\"; break\nelse\n  link_system=\"no\"\nfi\n\n\t\t\t\tdone\n                if test \"x$link_system\" != \"xyes\"; then\n                for libextension in `ls -r $BOOSTLIBDIR/boost_system* 2>/dev/null | sed 's,.*/,,' | sed -e 's,\\..*,,'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    as_ac_Lib=`$as_echo \"ac_cv_lib_$ax_lib''_exit\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for exit in -l$ax_lib\" >&5\n$as_echo_n \"checking for exit in -l$ax_lib... \" >&6; }\nif eval \\${$as_ac_Lib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_check_lib_save_LIBS=$LIBS\nLIBS=\"-l$ax_lib  $LIBS\"\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n/* Override any GCC internal prototype to avoid an error.\n   Use char because int might match the return type of a GCC\n   builtin and then its argument prototype would still apply.  */\n#ifdef __cplusplus\nextern \"C\"\n#endif\nchar exit ();\nint\nmain ()\n{\nreturn exit ();\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_link \"$LINENO\"; then :\n  eval \"$as_ac_Lib=yes\"\nelse\n  eval \"$as_ac_Lib=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext \\\n    conftest$ac_exeext conftest.$ac_ext\nLIBS=$ac_check_lib_save_LIBS\nfi\neval ac_res=\\$$as_ac_Lib\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_Lib\"\\\" = x\"yes\"; then :\n  BOOST_SYSTEM_LIB=\"-l$ax_lib\";  link_system=\"yes\"; break\nelse\n  link_system=\"no\"\nfi\n\n\t\t\t\tdone\n                fi\n\n            else\n               for ax_lib in $ax_boost_user_system_lib boost_system-$ax_boost_user_system_lib; do\n\t\t\t\t      as_ac_Lib=`$as_echo \"ac_cv_lib_$ax_lib''_exit\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for exit in -l$ax_lib\" >&5\n$as_echo_n \"checking for exit in -l$ax_lib... \" >&6; }\nif eval \\${$as_ac_Lib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_check_lib_save_LIBS=$LIBS\nLIBS=\"-l$ax_lib  $LIBS\"\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n/* Override any GCC internal prototype to avoid an error.\n   Use char because int might match the return type of a GCC\n   builtin and then its argument prototype would still apply.  */\n#ifdef __cplusplus\nextern \"C\"\n#endif\nchar exit ();\nint\nmain ()\n{\nreturn exit ();\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_link \"$LINENO\"; then :\n  eval \"$as_ac_Lib=yes\"\nelse\n  eval \"$as_ac_Lib=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext \\\n    conftest$ac_exeext conftest.$ac_ext\nLIBS=$ac_check_lib_save_LIBS\nfi\neval ac_res=\\$$as_ac_Lib\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_Lib\"\\\" = x\"yes\"; then :\n  BOOST_SYSTEM_LIB=\"-l$ax_lib\";  link_system=\"yes\"; break\nelse\n  link_system=\"no\"\nfi\n\n                  done\n\n            fi\n            if test \"x$ax_lib\" = \"x\"; then\n                as_fn_error $? \"Could not find a version of the library!\" \"$LINENO\" 5\n            fi\n\t\t\tif test \"x$link_system\" = \"xno\"; then\n\t\t\t\tas_fn_error $? \"Could not link against $ax_lib !\" \"$LINENO\" 5\n\t\t\tfi\n\t\tfi\n\n\t\tCPPFLAGS=\"$CPPFLAGS_SAVED\"\n\tLDFLAGS=\"$LDFLAGS_SAVED\"\n\tfi\n\n\n\n# Check whether --with-boost-regex was given.\nif test \"${with_boost_regex+set}\" = set; then :\n  withval=$with_boost_regex;\n        if test \"$withval\" = \"no\"; then\n\t\t\twant_boost=\"no\"\n        elif test \"$withval\" = \"yes\"; then\n            want_boost=\"yes\"\n            ax_boost_user_regex_lib=\"\"\n        else\n\t\t    want_boost=\"yes\"\n\t\tax_boost_user_regex_lib=\"$withval\"\n\t\tfi\n\nelse\n  want_boost=\"yes\"\n\nfi\n\n\n\tif test \"x$want_boost\" = \"xyes\"; then\n\n\t\tCPPFLAGS_SAVED=\"$CPPFLAGS\"\n\t\tCPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n\t\texport CPPFLAGS\n\n\t\tLDFLAGS_SAVED=\"$LDFLAGS\"\n\t\tLDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n\t\texport LDFLAGS\n\n        { $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether the Boost::Regex library is available\" >&5\n$as_echo_n \"checking whether the Boost::Regex library is available... \" >&6; }\nif ${ax_cv_boost_regex+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=cpp\nac_cpp='$CXXCPP $CPPFLAGS'\nac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\n\n\t\t\t cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <boost/regex.hpp>\n\nint\nmain ()\n{\nboost::regex r(); return 0;\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"; then :\n  ax_cv_boost_regex=yes\nelse\n  ax_cv_boost_regex=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\n         ac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ax_cv_boost_regex\" >&5\n$as_echo \"$ax_cv_boost_regex\" >&6; }\n\t\tif test \"x$ax_cv_boost_regex\" = \"xyes\"; then\n\n$as_echo \"#define HAVE_BOOST_REGEX /**/\" >>confdefs.h\n\n            BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/[^\\/]*//'`\n            if test \"x$ax_boost_user_regex_lib\" = \"x\"; then\n                for libextension in `ls $BOOSTLIBDIR/libboost_regex*.so* $BOOSTLIBDIR/libboost_regex*.dylib* $BOOSTLIBDIR/libboost_regex*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\\(boost_regex.*\\)\\.so.*$;\\1;' -e 's;^lib\\(boost_regex.*\\)\\.dylib.*;\\1;' -e 's;^lib\\(boost_regex.*\\)\\.a.*$;\\1;'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    as_ac_Lib=`$as_echo \"ac_cv_lib_$ax_lib''_exit\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for exit in -l$ax_lib\" >&5\n$as_echo_n \"checking for exit in -l$ax_lib... \" >&6; }\nif eval \\${$as_ac_Lib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_check_lib_save_LIBS=$LIBS\nLIBS=\"-l$ax_lib  $LIBS\"\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n/* Override any GCC internal prototype to avoid an error.\n   Use char because int might match the return type of a GCC\n   builtin and then its argument prototype would still apply.  */\n#ifdef __cplusplus\nextern \"C\"\n#endif\nchar exit ();\nint\nmain ()\n{\nreturn exit ();\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_link \"$LINENO\"; then :\n  eval \"$as_ac_Lib=yes\"\nelse\n  eval \"$as_ac_Lib=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext \\\n    conftest$ac_exeext conftest.$ac_ext\nLIBS=$ac_check_lib_save_LIBS\nfi\neval ac_res=\\$$as_ac_Lib\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_Lib\"\\\" = x\"yes\"; then :\n  BOOST_REGEX_LIB=\"-l$ax_lib\";  link_regex=\"yes\"; break\nelse\n  link_regex=\"no\"\nfi\n\n\t\t\t\tdone\n                if test \"x$link_regex\" != \"xyes\"; then\n                for libextension in `ls $BOOSTLIBDIR/boost_regex*.dll* $BOOSTLIBDIR/boost_regex*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\\(boost_regex.*\\)\\.dll.*$;\\1;' -e 's;^\\(boost_regex.*\\)\\.a.*$;\\1;'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    as_ac_Lib=`$as_echo \"ac_cv_lib_$ax_lib''_exit\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for exit in -l$ax_lib\" >&5\n$as_echo_n \"checking for exit in -l$ax_lib... \" >&6; }\nif eval \\${$as_ac_Lib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_check_lib_save_LIBS=$LIBS\nLIBS=\"-l$ax_lib  $LIBS\"\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n/* Override any GCC internal prototype to avoid an error.\n   Use char because int might match the return type of a GCC\n   builtin and then its argument prototype would still apply.  */\n#ifdef __cplusplus\nextern \"C\"\n#endif\nchar exit ();\nint\nmain ()\n{\nreturn exit ();\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_link \"$LINENO\"; then :\n  eval \"$as_ac_Lib=yes\"\nelse\n  eval \"$as_ac_Lib=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext \\\n    conftest$ac_exeext conftest.$ac_ext\nLIBS=$ac_check_lib_save_LIBS\nfi\neval ac_res=\\$$as_ac_Lib\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_Lib\"\\\" = x\"yes\"; then :\n  BOOST_REGEX_LIB=\"-l$ax_lib\";  link_regex=\"yes\"; break\nelse\n  link_regex=\"no\"\nfi\n\n\t\t\t\tdone\n                fi\n\n            else\n               for ax_lib in $ax_boost_user_regex_lib boost_regex-$ax_boost_user_regex_lib; do\n\t\t\t\t      as_ac_Lib=`$as_echo \"ac_cv_lib_$ax_lib''_main\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for main in -l$ax_lib\" >&5\n$as_echo_n \"checking for main in -l$ax_lib... \" >&6; }\nif eval \\${$as_ac_Lib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_check_lib_save_LIBS=$LIBS\nLIBS=\"-l$ax_lib  $LIBS\"\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n\nint\nmain ()\n{\nreturn main ();\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_link \"$LINENO\"; then :\n  eval \"$as_ac_Lib=yes\"\nelse\n  eval \"$as_ac_Lib=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext \\\n    conftest$ac_exeext conftest.$ac_ext\nLIBS=$ac_check_lib_save_LIBS\nfi\neval ac_res=\\$$as_ac_Lib\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_Lib\"\\\" = x\"yes\"; then :\n  BOOST_REGEX_LIB=\"-l$ax_lib\";  link_regex=\"yes\"; break\nelse\n  link_regex=\"no\"\nfi\n\n               done\n            fi\n            if test \"x$ax_lib\" = \"x\"; then\n                as_fn_error $? \"Could not find a version of the Boost::Regex library!\" \"$LINENO\" 5\n            fi\n\t\t\tif test \"x$link_regex\" != \"xyes\"; then\n\t\t\t\tas_fn_error $? \"Could not link against $ax_lib !\" \"$LINENO\" 5\n\t\t\tfi\n\t\tfi\n\n\t\tCPPFLAGS=\"$CPPFLAGS_SAVED\"\n\tLDFLAGS=\"$LDFLAGS_SAVED\"\n\tfi\n\n\n\n# Check whether --with-boost-asio was given.\nif test \"${with_boost_asio+set}\" = set; then :\n  withval=$with_boost_asio;\n        if test \"$withval\" = \"no\"; then\n\t\t\twant_boost=\"no\"\n        elif test \"$withval\" = \"yes\"; then\n            want_boost=\"yes\"\n            ax_boost_user_asio_lib=\"\"\n        else\n\t\t    want_boost=\"yes\"\n\t\tax_boost_user_asio_lib=\"$withval\"\n\t\tfi\n\nelse\n  want_boost=\"yes\"\n\nfi\n\n\n\tif test \"x$want_boost\" = \"xyes\"; then\n\n\t\tCPPFLAGS_SAVED=\"$CPPFLAGS\"\n\t\tCPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n\t\texport CPPFLAGS\n\n\t\tLDFLAGS_SAVED=\"$LDFLAGS\"\n\t\tLDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n\t\texport LDFLAGS\n\n        { $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether the Boost::ASIO library is available\" >&5\n$as_echo_n \"checking whether the Boost::ASIO library is available... \" >&6; }\nif ${ax_cv_boost_asio+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=cpp\nac_cpp='$CXXCPP $CPPFLAGS'\nac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\n\n\t\t cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n #include <boost/asio.hpp>\n\nint\nmain ()\n{\n\n\n                                    boost::asio::io_service io;\n                                    boost::system::error_code timer_result;\n                                    boost::asio::deadline_timer t(io);\n                                    t.cancel();\n                                    io.run_one();\n\t\t\t\t\t\t\t\t\treturn 0;\n\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"; then :\n  ax_cv_boost_asio=yes\nelse\n  ax_cv_boost_asio=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\n         ac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ax_cv_boost_asio\" >&5\n$as_echo \"$ax_cv_boost_asio\" >&6; }\n\t\tif test \"x$ax_cv_boost_asio\" = \"xyes\"; then\n\n$as_echo \"#define HAVE_BOOST_ASIO /**/\" >>confdefs.h\n\n\t\t\tBN=boost_system\n\t\t\tBOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/[^\\/]*//'`\n            if test \"x$ax_boost_user_asio_lib\" = \"x\"; then\n\t\t\t\tfor ax_lib in `ls $BOOSTLIBDIR/libboost_system*.so* $BOOSTLIBDIR/libboost_system*.dylib* $BOOSTLIBDIR/libboost_system*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\\(boost_system.*\\)\\.so.*$;\\1;' -e 's;^lib\\(boost_system.*\\)\\.dylib.*$;\\1;' -e 's;^lib\\(boost_system.*\\)\\.a.*$;\\1;' ` ; do\n\t\t\t\t    as_ac_Lib=`$as_echo \"ac_cv_lib_$ax_lib''_main\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for main in -l$ax_lib\" >&5\n$as_echo_n \"checking for main in -l$ax_lib... \" >&6; }\nif eval \\${$as_ac_Lib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_check_lib_save_LIBS=$LIBS\nLIBS=\"-l$ax_lib  $LIBS\"\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n\nint\nmain ()\n{\nreturn main ();\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_link \"$LINENO\"; then :\n  eval \"$as_ac_Lib=yes\"\nelse\n  eval \"$as_ac_Lib=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext \\\n    conftest$ac_exeext conftest.$ac_ext\nLIBS=$ac_check_lib_save_LIBS\nfi\neval ac_res=\\$$as_ac_Lib\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_Lib\"\\\" = x\"yes\"; then :\n  BOOST_ASIO_LIB=\"-l$ax_lib\"  link_thread=\"yes\" break\nelse\n  link_thread=\"no\"\nfi\n\n\t\t\t\tdone\n            else\n               for ax_lib in $ax_boost_user_asio_lib $BN-$ax_boost_user_asio_lib; do\n\t\t\t\t      as_ac_Lib=`$as_echo \"ac_cv_lib_$ax_lib''_main\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for main in -l$ax_lib\" >&5\n$as_echo_n \"checking for main in -l$ax_lib... \" >&6; }\nif eval \\${$as_ac_Lib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_check_lib_save_LIBS=$LIBS\nLIBS=\"-l$ax_lib  $LIBS\"\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n\nint\nmain ()\n{\nreturn main ();\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_link \"$LINENO\"; then :\n  eval \"$as_ac_Lib=yes\"\nelse\n  eval \"$as_ac_Lib=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext \\\n    conftest$ac_exeext conftest.$ac_ext\nLIBS=$ac_check_lib_save_LIBS\nfi\neval ac_res=\\$$as_ac_Lib\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_Lib\"\\\" = x\"yes\"; then :\n  BOOST_ASIO_LIB=\"-l$ax_lib\"  link_asio=\"yes\" break\nelse\n  link_asio=\"no\"\nfi\n\n                  done\n\n            fi\n            if test \"x$ax_lib\" = \"x\"; then\n                as_fn_error $? \"Could not find a version of the library!\" \"$LINENO\" 5\n            fi\n\t\t\tif test \"x$link_asio\" = \"xno\"; then\n\t\t\t\tas_fn_error $? \"Could not link against $ax_lib !\" \"$LINENO\" 5\n\t\t\tfi\n\t\tfi\n\n\t\tCPPFLAGS=\"$CPPFLAGS_SAVED\"\n\tLDFLAGS=\"$LDFLAGS_SAVED\"\n\tfi\n\n\n\n# Check whether --with-boost-program-options was given.\nif test \"${with_boost_program_options+set}\" = set; then :\n  withval=$with_boost_program_options;\n        if test \"$withval\" = \"no\"; then\n\t\t\twant_boost=\"no\"\n        elif test \"$withval\" = \"yes\"; then\n            want_boost=\"yes\"\n            ax_boost_user_program_options_lib=\"\"\n        else\n\t\t    want_boost=\"yes\"\n\t\tax_boost_user_program_options_lib=\"$withval\"\n\t\tfi\n\nelse\n  want_boost=\"yes\"\n\nfi\n\n\n\tif test \"x$want_boost\" = \"xyes\"; then\n\n\t    export want_boost\n\t\tCPPFLAGS_SAVED=\"$CPPFLAGS\"\n\t\tCPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n\t\texport CPPFLAGS\n\t\tLDFLAGS_SAVED=\"$LDFLAGS\"\n\t\tLDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n\t\texport LDFLAGS\n\t\t{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether the Boost::Program_Options library is available\" >&5\n$as_echo_n \"checking whether the Boost::Program_Options library is available... \" >&6; }\nif ${ax_cv_boost_program_options+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=cpp\nac_cpp='$CXXCPP $CPPFLAGS'\nac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\n\n\t\t\t\tcat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <boost/program_options.hpp>\n\nint\nmain ()\n{\nboost::program_options::options_description generic(\"Generic options\");\n                                   return 0;\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"; then :\n  ax_cv_boost_program_options=yes\nelse\n  ax_cv_boost_program_options=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\n\t\t\t\t\tac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ax_cv_boost_program_options\" >&5\n$as_echo \"$ax_cv_boost_program_options\" >&6; }\n\t\tif test \"$ax_cv_boost_program_options\" = yes; then\n\n$as_echo \"#define HAVE_BOOST_PROGRAM_OPTIONS /**/\" >>confdefs.h\n\n                  BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/[^\\/]*//'`\n                if test \"x$ax_boost_user_program_options_lib\" = \"x\"; then\n                for libextension in `ls $BOOSTLIBDIR/libboost_program_options*.so* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\\(boost_program_options.*\\)\\.so.*$;\\1;'` `ls $BOOSTLIBDIR/libboost_program_options*.dylib* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\\(boost_program_options.*\\)\\.dylib.*$;\\1;'` `ls $BOOSTLIBDIR/libboost_program_options*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\\(boost_program_options.*\\)\\.a.*$;\\1;'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    as_ac_Lib=`$as_echo \"ac_cv_lib_$ax_lib''_exit\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for exit in -l$ax_lib\" >&5\n$as_echo_n \"checking for exit in -l$ax_lib... \" >&6; }\nif eval \\${$as_ac_Lib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_check_lib_save_LIBS=$LIBS\nLIBS=\"-l$ax_lib  $LIBS\"\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n/* Override any GCC internal prototype to avoid an error.\n   Use char because int might match the return type of a GCC\n   builtin and then its argument prototype would still apply.  */\n#ifdef __cplusplus\nextern \"C\"\n#endif\nchar exit ();\nint\nmain ()\n{\nreturn exit ();\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_link \"$LINENO\"; then :\n  eval \"$as_ac_Lib=yes\"\nelse\n  eval \"$as_ac_Lib=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext \\\n    conftest$ac_exeext conftest.$ac_ext\nLIBS=$ac_check_lib_save_LIBS\nfi\neval ac_res=\\$$as_ac_Lib\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_Lib\"\\\" = x\"yes\"; then :\n  BOOST_PROGRAM_OPTIONS_LIB=\"-l$ax_lib\";  link_program_options=\"yes\"; break\nelse\n  link_program_options=\"no\"\nfi\n\n\t\t\t\tdone\n                if test \"x$link_program_options\" != \"xyes\"; then\n                for libextension in `ls $BOOSTLIBDIR/boost_program_options*.dll* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\\(boost_program_options.*\\)\\.dll.*$;\\1;'` `ls $BOOSTLIBDIR/boost_program_options*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\\(boost_program_options.*\\)\\.a.*$;\\1;'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    as_ac_Lib=`$as_echo \"ac_cv_lib_$ax_lib''_exit\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for exit in -l$ax_lib\" >&5\n$as_echo_n \"checking for exit in -l$ax_lib... \" >&6; }\nif eval \\${$as_ac_Lib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_check_lib_save_LIBS=$LIBS\nLIBS=\"-l$ax_lib  $LIBS\"\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n/* Override any GCC internal prototype to avoid an error.\n   Use char because int might match the return type of a GCC\n   builtin and then its argument prototype would still apply.  */\n#ifdef __cplusplus\nextern \"C\"\n#endif\nchar exit ();\nint\nmain ()\n{\nreturn exit ();\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_link \"$LINENO\"; then :\n  eval \"$as_ac_Lib=yes\"\nelse\n  eval \"$as_ac_Lib=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext \\\n    conftest$ac_exeext conftest.$ac_ext\nLIBS=$ac_check_lib_save_LIBS\nfi\neval ac_res=\\$$as_ac_Lib\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_Lib\"\\\" = x\"yes\"; then :\n  BOOST_PROGRAM_OPTIONS_LIB=\"-l$ax_lib\";  link_program_options=\"yes\"; break\nelse\n  link_program_options=\"no\"\nfi\n\n\t\t\t\tdone\n                fi\n                else\n                  for ax_lib in $ax_boost_user_program_options_lib boost_program_options-$ax_boost_user_program_options_lib; do\n\t\t\t\t      as_ac_Lib=`$as_echo \"ac_cv_lib_$ax_lib''_main\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for main in -l$ax_lib\" >&5\n$as_echo_n \"checking for main in -l$ax_lib... \" >&6; }\nif eval \\${$as_ac_Lib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_check_lib_save_LIBS=$LIBS\nLIBS=\"-l$ax_lib  $LIBS\"\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n\nint\nmain ()\n{\nreturn main ();\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_link \"$LINENO\"; then :\n  eval \"$as_ac_Lib=yes\"\nelse\n  eval \"$as_ac_Lib=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext \\\n    conftest$ac_exeext conftest.$ac_ext\nLIBS=$ac_check_lib_save_LIBS\nfi\neval ac_res=\\$$as_ac_Lib\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_Lib\"\\\" = x\"yes\"; then :\n  BOOST_PROGRAM_OPTIONS_LIB=\"-l$ax_lib\";  link_program_options=\"yes\"; break\nelse\n  link_program_options=\"no\"\nfi\n\n                  done\n                fi\n            if test \"x$ax_lib\" = \"x\"; then\n                as_fn_error $? \"Could not find a version of the library!\" \"$LINENO\" 5\n            fi\n\t\t\t\tif test \"x$link_program_options\" != \"xyes\"; then\n\t\t\t\t\tas_fn_error $? \"Could not link against $ax_lib !\" \"$LINENO\" 5\n\t\t\t\tfi\n\t\tfi\n\t\tCPPFLAGS=\"$CPPFLAGS_SAVED\"\n\tLDFLAGS=\"$LDFLAGS_SAVED\"\n\tfi\n\n\n\n# Check whether --with-boost-unit-test-framework was given.\nif test \"${with_boost_unit_test_framework+set}\" = set; then :\n  withval=$with_boost_unit_test_framework;\n        if test \"$withval\" = \"no\"; then\n\t\t\twant_boost=\"no\"\n        elif test \"$withval\" = \"yes\"; then\n            want_boost=\"yes\"\n            ax_boost_user_unit_test_framework_lib=\"\"\n        else\n\t\t    want_boost=\"yes\"\n\t\tax_boost_user_unit_test_framework_lib=\"$withval\"\n\t\tfi\n\nelse\n  want_boost=\"yes\"\n\nfi\n\n\n\tif test \"x$want_boost\" = \"xyes\"; then\n\n\t\tCPPFLAGS_SAVED=\"$CPPFLAGS\"\n\t\tCPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n\t\texport CPPFLAGS\n\n\t\tLDFLAGS_SAVED=\"$LDFLAGS\"\n\t\tLDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n\t\texport LDFLAGS\n\n        { $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether the Boost::Unit_Test_Framework library is available\" >&5\n$as_echo_n \"checking whether the Boost::Unit_Test_Framework library is available... \" >&6; }\nif ${ax_cv_boost_unit_test_framework+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=cpp\nac_cpp='$CXXCPP $CPPFLAGS'\nac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\n\n\t\t\t cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <boost/test/unit_test.hpp>\nint\nmain ()\n{\nusing boost::unit_test::test_suite;\n\t\t\t\t\t\t\t test_suite* test= BOOST_TEST_SUITE( \"Unit test example 1\" ); return 0;\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"; then :\n  ax_cv_boost_unit_test_framework=yes\nelse\n  ax_cv_boost_unit_test_framework=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\n         ac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ax_cv_boost_unit_test_framework\" >&5\n$as_echo \"$ax_cv_boost_unit_test_framework\" >&6; }\n\t\tif test \"x$ax_cv_boost_unit_test_framework\" = \"xyes\"; then\n\n$as_echo \"#define HAVE_BOOST_UNIT_TEST_FRAMEWORK /**/\" >>confdefs.h\n\n            BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/[^\\/]*//'`\n\n            if test \"x$ax_boost_user_unit_test_framework_lib\" = \"x\"; then\n\t\t\tsaved_ldflags=\"${LDFLAGS}\"\n                for monitor_library in `ls $BOOSTLIBDIR/libboost_unit_test_framework*.so* $BOOSTLIBDIR/libboost_unit_test_framework*.dylib* $BOOSTLIBDIR/libboost_unit_test_framework*.a* 2>/dev/null` ; do\n                    if test -r $monitor_library ; then\n                       libextension=`echo $monitor_library | sed 's,.*/,,' | sed -e 's;^lib\\(boost_unit_test_framework.*\\)\\.so.*$;\\1;' -e 's;^lib\\(boost_unit_test_framework.*\\)\\.dylib.*$;\\1;' -e 's;^lib\\(boost_unit_test_framework.*\\)\\.a.*$;\\1;'`\n                       ax_lib=${libextension}\n                       link_unit_test_framework=\"yes\"\n                    else\n                       link_unit_test_framework=\"no\"\n                    fi\n\n\t\t\t    if test \"x$link_unit_test_framework\" = \"xyes\"; then\n                      BOOST_UNIT_TEST_FRAMEWORK_LIB=\"-l$ax_lib\"\n\n\t\t\t\t\t  break\n\t\t\t\t    fi\n                done\n                if test \"x$link_unit_test_framework\" != \"xyes\"; then\n                for libextension in `ls $BOOSTLIBDIR/boost_unit_test_framework*.dll* $BOOSTLIBDIR/boost_unit_test_framework*.a* 2>/dev/null  | sed 's,.*/,,' | sed -e 's;^\\(boost_unit_test_framework.*\\)\\.dll.*$;\\1;' -e 's;^\\(boost_unit_test_framework.*\\)\\.a.*$;\\1;'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    as_ac_Lib=`$as_echo \"ac_cv_lib_$ax_lib''_exit\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for exit in -l$ax_lib\" >&5\n$as_echo_n \"checking for exit in -l$ax_lib... \" >&6; }\nif eval \\${$as_ac_Lib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_check_lib_save_LIBS=$LIBS\nLIBS=\"-l$ax_lib  $LIBS\"\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\n/* Override any GCC internal prototype to avoid an error.\n   Use char because int might match the return type of a GCC\n   builtin and then its argument prototype would still apply.  */\n#ifdef __cplusplus\nextern \"C\"\n#endif\nchar exit ();\nint\nmain ()\n{\nreturn exit ();\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_link \"$LINENO\"; then :\n  eval \"$as_ac_Lib=yes\"\nelse\n  eval \"$as_ac_Lib=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext \\\n    conftest$ac_exeext conftest.$ac_ext\nLIBS=$ac_check_lib_save_LIBS\nfi\neval ac_res=\\$$as_ac_Lib\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_Lib\"\\\" = x\"yes\"; then :\n  BOOST_UNIT_TEST_FRAMEWORK_LIB=\"-l$ax_lib\";  link_unit_test_framework=\"yes\"; break\nelse\n  link_unit_test_framework=\"no\"\nfi\n\n\t\t\t\tdone\n                fi\n            else\n                link_unit_test_framework=\"no\"\n\t\t\tsaved_ldflags=\"${LDFLAGS}\"\n                for ax_lib in boost_unit_test_framework-$ax_boost_user_unit_test_framework_lib $ax_boost_user_unit_test_framework_lib ; do\n                   if test \"x$link_unit_test_framework\" = \"xyes\"; then\n                      break;\n                   fi\n                   for unittest_library in `ls $BOOSTLIBDIR/lib${ax_lib}.so* $BOOSTLIBDIR/lib${ax_lib}.a* 2>/dev/null` ; do\n                   if test -r $unittest_library ; then\n                       libextension=`echo $unittest_library | sed 's,.*/,,' | sed -e 's;^lib\\(boost_unit_test_framework.*\\)\\.so.*$;\\1;' -e 's;^lib\\(boost_unit_test_framework.*\\)\\.a*$;\\1;'`\n                       ax_lib=${libextension}\n                       link_unit_test_framework=\"yes\"\n                    else\n                       link_unit_test_framework=\"no\"\n                    fi\n\n\t\t\t\tif test \"x$link_unit_test_framework\" = \"xyes\"; then\n                        BOOST_UNIT_TEST_FRAMEWORK_LIB=\"-l$ax_lib\"\n\n\t\t\t\t\t    break\n\t\t\t\t    fi\n                  done\n               done\n            fi\n            if test \"x$ax_lib\" = \"x\"; then\n                as_fn_error $? \"Could not find a version of the library!\" \"$LINENO\" 5\n            fi\n\t\t\tif test \"x$link_unit_test_framework\" != \"xyes\"; then\n\t\t\t\tas_fn_error $? \"Could not link against $ax_lib !\" \"$LINENO\" 5\n\t\t\tfi\n\t\tfi\n\n\t\tCPPFLAGS=\"$CPPFLAGS_SAVED\"\n\tLDFLAGS=\"$LDFLAGS_SAVED\"\n\tfi\n\n\n\n    found=false\n\n# Check whether --with-openssl was given.\nif test \"${with_openssl+set}\" = set; then :\n  withval=$with_openssl;\n            case \"$withval\" in\n            \"\" | y | ye | yes | n | no)\n            as_fn_error $? \"Invalid --with-openssl value\" \"$LINENO\" 5\n              ;;\n            *) ssldirs=\"$withval\"\n              ;;\n            esac\n\nelse\n\n            ## # if pkg-config is installed and openssl has installed a .pc file,\n            ## # then use that information and don't search ssldirs\n            ## AC_CHECK_TOOL([PKG_CONFIG], [pkg-config])\n            ## if test x\"$PKG_CONFIG\" != x\"\"; then\n            ##     OPENSSL_LDFLAGS=`$PKG_CONFIG openssl --libs-only-L 2>/dev/null`\n            ##     if test $? = 0; then\n            ##         OPENSSL_LIBS=`$PKG_CONFIG openssl --libs-only-l 2>/dev/null`\n            ##         OPENSSL_INCLUDES=`$PKG_CONFIG openssl --cflags-only-I 2>/dev/null`\n            ##         found=true\n            ##     fi\n            ## fi\n\n            # no such luck; use some default ssldirs\n            if ! $found; then\n                ssldirs=\"/usr/local/ssl /usr/lib/ssl /usr/ssl /usr/pkg /usr/local /usr /usr/local/Cellar/openssl/*\"\n            fi\n\n\nfi\n\n\n\n    # note that we #include <openssl/foo.h>, so the OpenSSL headers have to be in\n    # an 'openssl' subdirectory\n\n    if ! $found; then\n        OPENSSL_INCLUDES=\n        for ssldir in $ssldirs; do\n            { $as_echo \"$as_me:${as_lineno-$LINENO}: checking for openssl/ssl.h in $ssldir\" >&5\n$as_echo_n \"checking for openssl/ssl.h in $ssldir... \" >&6; }\n            if test -f \"$ssldir/include/openssl/ssl.h\"; then\n                OPENSSL_INCLUDES=\"-I$ssldir/include\"\n                OPENSSL_LDFLAGS=\"-L$ssldir/lib\"\n                OPENSSL_LIBS=\"-lssl -lcrypto\"\n                found=true\n                { $as_echo \"$as_me:${as_lineno-$LINENO}: result: yes\" >&5\n$as_echo \"yes\" >&6; }\n                break\n            else\n                { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\n            fi\n        done\n\n        # if the file wasn't found, well, go ahead and try the link anyway -- maybe\n        # it will just work!\n    fi\n\n    # try the preprocessor and linker with our new flags,\n    # being careful not to pollute the global LIBS, LDFLAGS, and CPPFLAGS\n\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether compiling and linking against OpenSSL works\" >&5\n$as_echo_n \"checking whether compiling and linking against OpenSSL works... \" >&6; }\n    echo \"Trying link with OPENSSL_LDFLAGS=$OPENSSL_LDFLAGS;\" \\\n        \"OPENSSL_LIBS=$OPENSSL_LIBS; OPENSSL_INCLUDES=$OPENSSL_INCLUDES\" >&5\n\n    save_LIBS=\"$LIBS\"\n    save_LDFLAGS=\"$LDFLAGS\"\n    save_CPPFLAGS=\"$CPPFLAGS\"\n    LDFLAGS=\"$LDFLAGS $OPENSSL_LDFLAGS\"\n    LIBS=\"$OPENSSL_LIBS $LIBS\"\n    CPPFLAGS=\"$OPENSSL_INCLUDES $CPPFLAGS\"\n    cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <openssl/ssl.h>\nint\nmain ()\n{\nSSL_new(NULL)\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_link \"$LINENO\"; then :\n\n            { $as_echo \"$as_me:${as_lineno-$LINENO}: result: yes\" >&5\n$as_echo \"yes\" >&6; }\n            ENABLE_SSL=yes\n\nelse\n\n            { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\n\n\nfi\nrm -f core conftest.err conftest.$ac_objext \\\n    conftest$ac_exeext conftest.$ac_ext\n    CPPFLAGS=\"$save_CPPFLAGS\"\n    LDFLAGS=\"$save_LDFLAGS\"\n    LIBS=\"$save_LIBS\"\n\n\n\n\n\n\n    if test \"x$have_boost_147\" = \"xyes\" -a \"x$ax_cv_boost_system\" = \"xyes\" -a \"x$ax_cv_boost_regex\" = \"xyes\"  -a \"x$ax_cv_boost_program_options\" = \"xyes\"  -a \"x$ax_cv_boost_unit_test_framework\" = \"xyes\" -a \"x$ENABLE_SSL\" = \"xyes\" ; then\n      enabled_cxx=1\n    fi\n  fi\nfi\n\nif test \"x$enabled_cxx\" = \"x0\" ; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: c++ bindings disabled ('make cpp' will fail)\" >&5\n$as_echo \"$as_me: c++ bindings disabled ('make cpp' will fail)\" >&6;}\nfi\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking whether C compiler accepts -MJ foo\" >&5\n$as_echo_n \"checking whether C compiler accepts -MJ foo... \" >&6; }\nif ${ax_cv_check_cflags___MJ_foo+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n\n  ax_check_save_flags=$CFLAGS\n  CFLAGS=\"$CFLAGS  -MJ foo\"\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n\nint\nmain ()\n{\n\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_compile \"$LINENO\"; then :\n  ax_cv_check_cflags___MJ_foo=yes\nelse\n  ax_cv_check_cflags___MJ_foo=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\n  CFLAGS=$ax_check_save_flags\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ax_cv_check_cflags___MJ_foo\" >&5\n$as_echo \"$ax_cv_check_cflags___MJ_foo\" >&6; }\nif test \"x$ax_cv_check_cflags___MJ_foo\" = xyes; then :\n  :\nelse\n  :\nfi\n\n\nJSONFLAGS=\"\"\nif test \"x$ax_cv_check_cflags___MJ_foo\" = \"xyes\"; then\nJSONFLAGS='-MJ $@.json'\nfi\n\nCXXSHELL=$SHELL\nif test -f /usr/local/bin/bash; then\n  CXXSHELL=/usr/local/bin/bash\nelif test -f /bin/bash; then\n  CXXSHELL=/bin/bash\nelif test -f /bin/zsh; then\n  CXXSHELL=/bin/zsh\nfi\n\n# Extract the first word of \"clang-format\", so it can be a program name with args.\nset dummy clang-format; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_CLANGFORMAT+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $CLANGFORMAT in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_CLANGFORMAT=\"$CLANGFORMAT\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_CLANGFORMAT=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  test -z \"$ac_cv_path_CLANGFORMAT\" && ac_cv_path_CLANGFORMAT=\"$ac_cv_path_TRUE\"\n  ;;\nesac\nfi\nCLANGFORMAT=$ac_cv_path_CLANGFORMAT\nif test -n \"$CLANGFORMAT\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $CLANGFORMAT\" >&5\n$as_echo \"$CLANGFORMAT\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n# Extract the first word of \"clang-tidy\", so it can be a program name with args.\nset dummy clang-tidy; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_CLANGTIDY+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $CLANGTIDY in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_CLANGTIDY=\"$CLANGTIDY\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_CLANGTIDY=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  test -z \"$ac_cv_path_CLANGTIDY\" && ac_cv_path_CLANGTIDY=\"$ac_cv_path_TRUE\"\n  ;;\nesac\nfi\nCLANGTIDY=$ac_cv_path_CLANGTIDY\nif test -n \"$CLANGTIDY\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $CLANGTIDY\" >&5\n$as_echo \"$CLANGTIDY\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n###########################################################\n#\n# check tokyo cabinet and toke\n#\n###########################################################\n\nENABLE_TOKE=\nTOKEPREFIX=\n# Check whether --enable-toke was given.\nif test \"${enable_toke+set}\" = set; then :\n  enableval=$enable_toke; ENABLE_TOKE=yes\n               TOKEPREFIX=$enableval\nfi\n\n\n###########################################################\n#\n# check hanoidb\n#\n###########################################################\n\nENABLE_HANOIDB=\nHANOIDBPREFIX=\n# Check whether --enable-hanoidb was given.\nif test \"${enable_hanoidb+set}\" = set; then :\n  enableval=$enable_hanoidb; ENABLE_HANOIDB=yes\n               HANOIDBPREFIX=$enableval\nfi\n\n\n###########################################################\n#\n# check bitcask\n#\n###########################################################\n\nENABLE_BITCASK=\nBITCASKPREFIX=\n# Check whether --enable-bitcask was given.\nif test \"${enable_bitcask+set}\" = set; then :\n  enableval=$enable_bitcask; ENABLE_BITCASK=yes\n               BITCASKPREFIX=$enableval\nfi\n\n\n###########################################################\n#\n# check erlang_js\n#\n###########################################################\n\nENABLE_ERLANGJS=\nERLANGJSPREFIX=\n# Check whether --enable-erlang-js was given.\nif test \"${enable_erlang_js+set}\" = set; then :\n  enableval=$enable_erlang_js; ENABLE_ERLANGJS=yes\n               ERLANGJSPREFIX=$enableval\nfi\n\n\n###########################################################\n#\n# check browser for 'make test-vts'\n#\n###########################################################\n\nBROWSER=\n\n# Check whether --with-browser was given.\nif test \"${with_browser+set}\" = set; then :\n  withval=$with_browser; if test -n \"$withval\"; then\n               BROWSER=\"$withval\"\n               if test ! -x \"$BROWSER\"; then :\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: \\\"$BROWSER\\\" was not found or is not executable ('make test-vts' will likely fail)\" >&5\n$as_echo \"$as_me: WARNING: \\\"$BROWSER\\\" was not found or is not executable ('make test-vts' will likely fail)\" >&2;}\nfi\n             fi\nelse\n  # Extract the first word of \"xdg-open\", so it can be a program name with args.\nset dummy xdg-open; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_BROWSER+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $BROWSER in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_BROWSER=\"$BROWSER\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_BROWSER=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  ;;\nesac\nfi\nBROWSER=$ac_cv_path_BROWSER\nif test -n \"$BROWSER\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $BROWSER\" >&5\n$as_echo \"$BROWSER\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\nfi\n\n\n\nif test \"$BROWSER\" = \"\"; then :\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: xdg-open was not found ('make test-vts' will likely fail)\" >&5\n$as_echo \"$as_me: WARNING: xdg-open was not found ('make test-vts' will likely fail)\" >&2;}\nfi\n\n###########################################################\n#\n# check screen availability for 'scalarisctl -d --screen'\n#\n###########################################################\n# Extract the first word of \"screen\", so it can be a program name with args.\nset dummy screen; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_SCREEN+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $SCREEN in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_SCREEN=\"$SCREEN\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_SCREEN=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  test -z \"$ac_cv_path_SCREEN\" && ac_cv_path_SCREEN=\"$FALSE\"\n  ;;\nesac\nfi\nSCREEN=$ac_cv_path_SCREEN\nif test -n \"$SCREEN\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $SCREEN\" >&5\n$as_echo \"$SCREEN\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\nif test \"$ac_cv_path_SCREEN\" = \"$FALSE\" ; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: screen not found - you won't be able to run scalaris deamonized with screen, i.e. 'scalarisctl -d --screen'\" >&5\n$as_echo \"$as_me: screen not found - you won't be able to run scalaris deamonized with screen, i.e. 'scalarisctl -d --screen'\" >&6;}\nfi\n\n###########################################################\n#\n# check sudo, runuser availability for the init.d script\n#\n###########################################################\n\n# Extract the first word of \"sudo\", so it can be a program name with args.\nset dummy sudo; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_SUDO+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $SUDO in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_SUDO=\"$SUDO\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_SUDO=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  test -z \"$ac_cv_path_SUDO\" && ac_cv_path_SUDO=\"$FALSE\"\n  ;;\nesac\nfi\nSUDO=$ac_cv_path_SUDO\nif test -n \"$SUDO\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $SUDO\" >&5\n$as_echo \"$SUDO\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n\nRUNUSER=\"\"\n# Check whether --enable-runuser was given.\nif test \"${enable_runuser+set}\" = set; then :\n  enableval=$enable_runuser;\nfi\n\nif test \"$enable_runuser\" != \"no\"; then :\n  # Extract the first word of \"runuser\", so it can be a program name with args.\nset dummy runuser; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_RUNUSER+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $RUNUSER in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_RUNUSER=\"$RUNUSER\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH$PATH_SEPARATOR/usr/sbin\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_RUNUSER=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  test -z \"$ac_cv_path_RUNUSER\" && ac_cv_path_RUNUSER=\"\"\"\"\n  ;;\nesac\nfi\nRUNUSER=$ac_cv_path_RUNUSER\nif test -n \"$RUNUSER\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $RUNUSER\" >&5\n$as_echo \"$RUNUSER\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\nfi\n\nif test \"$ac_cv_path_SUDO\" = \"$FALSE\" -a \"$ac_cv_path_RUNUSER\" = \"\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: sudo or runuser not found - you won't be able to use our init.d script\" >&5\n$as_echo \"$as_me: sudo or runuser not found - you won't be able to use our init.d script\" >&6;}\nfi\n\n###########################################################\n#\n# check systemd support\n#\n###########################################################\n\nINSTALL_INIT=install-initd\nSYSTEMD_UNITDIR=\n\n# Check whether --with-systemd was given.\nif test \"${with_systemd+set}\" = set; then :\n  withval=$with_systemd; if test -d \"$withval\"; then\n               SYSTEMD_UNITDIR=\"$withval\"\n               INSTALL_INIT=\"install-systemd\"\n             else\n               if test \"$withval\" = \"yes\"; then\n                 SYSTEMD_UNITDIR=\"\\${prefix}/lib/systemd/system/\"\n                 INSTALL_INIT=\"install-systemd\"\n               fi\n             fi\nfi\n\n\n\n\n###########################################################\n#\n# check whether to compile to native code using HiPE\n#\n###########################################################\n\nCOMPILE_NATIVE=no\n# Check whether --enable-native was given.\nif test \"${enable_native+set}\" = set; then :\n  enableval=$enable_native; COMPILE_NATIVE=yes\nfi\n\n\n###########################################################\n#\n# check wether to enable debugging, e.g. enable the ASSERT macro\n#\n###########################################################\n\n# Check whether --enable-debug was given.\nif test \"${enable_debug+set}\" = set; then :\n  enableval=$enable_debug; if test \"no\" != \"$enableval\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, enable_debug}\"\n                      EDOCMACROS=\"${EDOCMACROS}, {enable_debug, true}\"\n                      DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Denable_debug\"\nfi\nfi\n\n\n###########################################################\n#\n# enable new transaction protocol\n#\n###########################################################\n\n# Check whether --enable-txnew was given.\nif test \"${enable_txnew+set}\" = set; then :\n  enableval=$enable_txnew; if test \"no\" != \"$enableval\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, enable_txnew}\"\n                      EDOCMACROS=\"${EDOCMACROS}, {enable_txnew, true}\"\n                      DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Denable_txnew\"\nfi\nfi\n\n\n###########################################################\n#\n# check erlang\n#\n###########################################################\nac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\n\nif test -n \"$ERLC\"; then\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: checking for erlc\" >&5\n$as_echo_n \"checking for erlc... \" >&6; }\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ERLC\" >&5\n$as_echo \"$ERLC\" >&6; }\nelse\n    if test -n \"$ac_tool_prefix\"; then\n  # Extract the first word of \"${ac_tool_prefix}erlc\", so it can be a program name with args.\nset dummy ${ac_tool_prefix}erlc; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_ERLC+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $ERLC in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_ERLC=\"$ERLC\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_ERLC=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  ;;\nesac\nfi\nERLC=$ac_cv_path_ERLC\nif test -n \"$ERLC\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ERLC\" >&5\n$as_echo \"$ERLC\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\nfi\nif test -z \"$ac_cv_path_ERLC\"; then\n  ac_pt_ERLC=$ERLC\n  # Extract the first word of \"erlc\", so it can be a program name with args.\nset dummy erlc; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_ac_pt_ERLC+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $ac_pt_ERLC in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_ac_pt_ERLC=\"$ac_pt_ERLC\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_ac_pt_ERLC=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  ;;\nesac\nfi\nac_pt_ERLC=$ac_cv_path_ac_pt_ERLC\nif test -n \"$ac_pt_ERLC\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_pt_ERLC\" >&5\n$as_echo \"$ac_pt_ERLC\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n  if test \"x$ac_pt_ERLC\" = x; then\n    ERLC=\"not found\"\n  else\n    case $cross_compiling:$ac_tool_warned in\nyes:)\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet\" >&5\n$as_echo \"$as_me: WARNING: using cross tools not prefixed with host triplet\" >&2;}\nac_tool_warned=yes ;;\nesac\n    ERLC=$ac_pt_ERLC\n  fi\nelse\n  ERLC=\"$ac_cv_path_ERLC\"\nfi\n\nfi\n\nif test \"$ERLC\" = \"not found\"; then\n    as_fn_error $? \"Erlang/OTP compiler (erlc) not found but required\" \"$LINENO\" 5\nfi\n\nif test -n \"$ERL\"; then\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: checking for erl\" >&5\n$as_echo_n \"checking for erl... \" >&6; }\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ERL\" >&5\n$as_echo \"$ERL\" >&6; }\nelse\n    if test -n \"$ac_tool_prefix\"; then\n  # Extract the first word of \"${ac_tool_prefix}erl\", so it can be a program name with args.\nset dummy ${ac_tool_prefix}erl; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_ERL+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $ERL in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_ERL=\"$ERL\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_ERL=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  ;;\nesac\nfi\nERL=$ac_cv_path_ERL\nif test -n \"$ERL\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ERL\" >&5\n$as_echo \"$ERL\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\nfi\nif test -z \"$ac_cv_path_ERL\"; then\n  ac_pt_ERL=$ERL\n  # Extract the first word of \"erl\", so it can be a program name with args.\nset dummy erl; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_ac_pt_ERL+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $ac_pt_ERL in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_ac_pt_ERL=\"$ac_pt_ERL\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_ac_pt_ERL=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  ;;\nesac\nfi\nac_pt_ERL=$ac_cv_path_ac_pt_ERL\nif test -n \"$ac_pt_ERL\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_pt_ERL\" >&5\n$as_echo \"$ac_pt_ERL\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n  if test \"x$ac_pt_ERL\" = x; then\n    ERL=\"not found\"\n  else\n    case $cross_compiling:$ac_tool_warned in\nyes:)\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet\" >&5\n$as_echo \"$as_me: WARNING: using cross tools not prefixed with host triplet\" >&2;}\nac_tool_warned=yes ;;\nesac\n    ERL=$ac_pt_ERL\n  fi\nelse\n  ERL=\"$ac_cv_path_ERL\"\nfi\nfi\n\nif test \"$ERL\" = \"not found\"; then\n    as_fn_error $? \"Erlang/OTP interpreter (erl) not found but required\" \"$LINENO\" 5\nfi\n\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP root directory\" >&5\n$as_echo_n \"checking for Erlang/OTP root directory... \" >&6; }\nif ${ac_cv_erlang_root_dir+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    RootDir = code:root_dir(),\n\t    file:write_file(\"conftest.out\", RootDir),\n\t    ReturnValue = 0,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_root_dir=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  rm -f conftest.out\n\t { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_root_dir\" >&5\n$as_echo \"$ac_cv_erlang_root_dir\" >&6; }\nERLANG_ROOT_DIR=$ac_cv_erlang_root_dir\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP library base directory\" >&5\n$as_echo_n \"checking for Erlang/OTP library base directory... \" >&6; }\nif ${ac_cv_erlang_lib_dir+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    LibDir = code:lib_dir(),\n\t    file:write_file(\"conftest.out\", LibDir),\n\t    ReturnValue = 0,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  rm -f conftest.out\n\t { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir\" >&6; }\nERLANG_LIB_DIR=$ac_cv_erlang_lib_dir\n\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP ERTS version\" >&5\n$as_echo_n \"checking for Erlang/OTP ERTS version... \" >&6; }\nif ${erlang_cv_erts_ver+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n            Version = erlang:system_info(version),\n            file:write_file(\"conftest.out\", Version),\n            ReturnValue = 0,\n            halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  erlang_cv_erts_ver=`cat conftest.out`\nelse\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $erlang_cv_erts_ver\" >&5\n$as_echo \"$erlang_cv_erts_ver\" >&6; }\nERLANG_ERTS_VER=$erlang_cv_erts_ver\n\n\n# split erlang erts version:\nset `echo $erlang_cv_erts_ver | $SED 's/\\./ /g'`\nERTS_MAJOR=$1\nERTS_MINOR=$2\nERTS_MAINT=${3:-0}\n\n# require Erlang >= R14B04, i.e. ERTS >= 5.8.5\nif test $ERTS_MAJOR -gt 5 -o \\\n       '(' $ERTS_MAJOR -eq 5 -a $ERTS_MINOR -gt 8 ')' -o \\\n       '(' $ERTS_MAJOR -eq 5 -a $ERTS_MINOR -eq 8 -a $ERTS_MAINT -ge 5 ')'; then :\n\nelse\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"Erlang >= R14B04 required\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\n\n# Extract the first word of \"epmd\", so it can be a program name with args.\nset dummy epmd; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_EPMD+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $EPMD in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_EPMD=\"$EPMD\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nas_dummy=\"\"$PATH:$ERLANG_ROOT_DIR/bin:$ERLANG_ROOT_DIR/erts-$ERLANG_ERTS_VER/bin\"\"\nfor as_dir in $as_dummy\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_EPMD=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  test -z \"$ac_cv_path_EPMD\" && ac_cv_path_EPMD=\"$FALSE\"\n  ;;\nesac\nfi\nEPMD=$ac_cv_path_EPMD\nif test -n \"$EPMD\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $EPMD\" >&5\n$as_echo \"$EPMD\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n\nif test \"x$COMPILE_NATIVE\" != xno; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, native\"\nfi\n\n# required libs:\nERLANG_UNAVAILABLE_LIBS=\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'erts' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'erts' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_erts+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"erts\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_erts=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_erts=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_erts\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_erts\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'erts' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'erts' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_erts+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_erts\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_erts=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_erts=`$as_echo \"$ac_cv_erlang_lib_dir_erts\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_erts\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_erts\" >&6; }\nERLANG_LIB_DIR_erts=$ac_cv_erlang_lib_dir_erts\n\nERLANG_LIB_VER_erts=$ac_cv_erlang_lib_ver_erts\n\nif test \"$ac_cv_erlang_lib_dir_erts\" = \"not found\"; then :\n  ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS erts\"\nfi\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'kernel' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'kernel' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_kernel+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"kernel\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_kernel=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_kernel=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_kernel\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_kernel\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'kernel' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'kernel' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_kernel+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_kernel\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_kernel=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_kernel=`$as_echo \"$ac_cv_erlang_lib_dir_kernel\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_kernel\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_kernel\" >&6; }\nERLANG_LIB_DIR_kernel=$ac_cv_erlang_lib_dir_kernel\n\nERLANG_LIB_VER_kernel=$ac_cv_erlang_lib_ver_kernel\n\nif test \"$ac_cv_erlang_lib_dir_kernel\" = \"not found\"; then :\n  ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS kernel\"\nfi\n\nas_ac_File=`$as_echo \"ac_cv_file_$ERLANG_LIB_DIR_kernel/include/inet.hrl\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ERLANG_LIB_DIR_kernel/include/inet.hrl\" >&5\n$as_echo_n \"checking for $ERLANG_LIB_DIR_kernel/include/inet.hrl... \" >&6; }\nif eval \\${$as_ac_File+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  test \"$cross_compiling\" = yes &&\n  as_fn_error $? \"cannot check for file existence when cross compiling\" \"$LINENO\" 5\nif test -r \"$ERLANG_LIB_DIR_kernel/include/inet.hrl\"; then\n  eval \"$as_ac_File=yes\"\nelse\n  eval \"$as_ac_File=no\"\nfi\nfi\neval ac_res=\\$$as_ac_File\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_File\"\\\" = x\"yes\"; then :\n\nelse\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"kernel/include/inet.hrl not found - you won't be able to compile the erlang sources\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'stdlib' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'stdlib' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_stdlib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"stdlib\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_stdlib=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_stdlib=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_stdlib\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_stdlib\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'stdlib' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'stdlib' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_stdlib+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_stdlib\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_stdlib=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_stdlib=`$as_echo \"$ac_cv_erlang_lib_dir_stdlib\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_stdlib\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_stdlib\" >&6; }\nERLANG_LIB_DIR_stdlib=$ac_cv_erlang_lib_dir_stdlib\n\nERLANG_LIB_VER_stdlib=$ac_cv_erlang_lib_ver_stdlib\n\nif test \"$ac_cv_erlang_lib_dir_stdlib\" = \"not found\"; then :\n  ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS stdlib\"\nfi\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'compiler' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'compiler' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_compiler+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"compiler\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_compiler=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_compiler=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_compiler\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_compiler\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'compiler' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'compiler' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_compiler+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_compiler\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_compiler=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_compiler=`$as_echo \"$ac_cv_erlang_lib_dir_compiler\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_compiler\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_compiler\" >&6; }\nERLANG_LIB_DIR_compiler=$ac_cv_erlang_lib_dir_compiler\n\nERLANG_LIB_VER_compiler=$ac_cv_erlang_lib_ver_compiler\n\nif test \"$ac_cv_erlang_lib_dir_compiler\" = \"not found\"; then :\n  ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS compiler\"\nfi\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'crypto' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'crypto' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_crypto+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"crypto\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_crypto=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_crypto=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_crypto\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_crypto\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'crypto' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'crypto' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_crypto+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_crypto\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_crypto=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_crypto=`$as_echo \"$ac_cv_erlang_lib_dir_crypto\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_crypto\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_crypto\" >&6; }\nERLANG_LIB_DIR_crypto=$ac_cv_erlang_lib_dir_crypto\n\nERLANG_LIB_VER_crypto=$ac_cv_erlang_lib_ver_crypto\n\nif test \"$ac_cv_erlang_lib_dir_crypto\" = \"not found\"; then :\n  ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS crypto\"\nfi\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'os_mon' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'os_mon' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_os_mon+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"os_mon\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_os_mon=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_os_mon=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_os_mon\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_os_mon\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'os_mon' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'os_mon' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_os_mon+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_os_mon\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_os_mon=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_os_mon=`$as_echo \"$ac_cv_erlang_lib_dir_os_mon\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_os_mon\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_os_mon\" >&6; }\nERLANG_LIB_DIR_os_mon=$ac_cv_erlang_lib_dir_os_mon\n\nERLANG_LIB_VER_os_mon=$ac_cv_erlang_lib_ver_os_mon\n\nif test \"$ac_cv_erlang_lib_dir_os_mon\" = \"not found\"; then :\n  ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS os_mon\"\nfi\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'tools' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'tools' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_tools+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"tools\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_tools=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_tools=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_tools\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_tools\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'tools' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'tools' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_tools+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_tools\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_tools=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_tools=`$as_echo \"$ac_cv_erlang_lib_dir_tools\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_tools\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_tools\" >&6; }\nERLANG_LIB_DIR_tools=$ac_cv_erlang_lib_dir_tools\n\nERLANG_LIB_VER_tools=$ac_cv_erlang_lib_ver_tools\n\nif test \"$ac_cv_erlang_lib_dir_tools\" = \"not found\"; then :\n  ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS tools\"\nfi\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'inets' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'inets' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_inets+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"inets\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_inets=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_inets=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_inets\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_inets\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'inets' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'inets' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_inets+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_inets\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_inets=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_inets=`$as_echo \"$ac_cv_erlang_lib_dir_inets\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_inets\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_inets\" >&6; }\nERLANG_LIB_DIR_inets=$ac_cv_erlang_lib_dir_inets\n\nERLANG_LIB_VER_inets=$ac_cv_erlang_lib_ver_inets\n\nif test \"$ac_cv_erlang_lib_dir_inets\" = \"not found\"; then :\n  ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS inets\"\nfi\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'ssl' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'ssl' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_ssl+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"ssl\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_ssl=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_ssl=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_ssl\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_ssl\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'ssl' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'ssl' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_ssl+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_ssl\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_ssl=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_ssl=`$as_echo \"$ac_cv_erlang_lib_dir_ssl\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_ssl\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_ssl\" >&6; }\nERLANG_LIB_DIR_ssl=$ac_cv_erlang_lib_dir_ssl\n\nERLANG_LIB_VER_ssl=$ac_cv_erlang_lib_ver_ssl\n\nif test \"$ac_cv_erlang_lib_dir_ssl\" = \"not found\"; then :\n  ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS ssl\"\nfi\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'xmerl' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'xmerl' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_xmerl+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"xmerl\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_xmerl=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_xmerl=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_xmerl\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_xmerl\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'xmerl' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'xmerl' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_xmerl+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_xmerl\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_xmerl=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_xmerl=`$as_echo \"$ac_cv_erlang_lib_dir_xmerl\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_xmerl\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_xmerl\" >&6; }\nERLANG_LIB_DIR_xmerl=$ac_cv_erlang_lib_dir_xmerl\n\nERLANG_LIB_VER_xmerl=$ac_cv_erlang_lib_ver_xmerl\n\nif test \"$ac_cv_erlang_lib_dir_xmerl\" = \"not found\"; then :\n  ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS xmerl\"\nfi\n\n\nif test \"x$ERLANG_UNAVAILABLE_LIBS\" != x; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"$ERLANG_UNAVAILABLE_LIBS not found - you won't be able to compile or run the erlang sources\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\n\n# libs for optional build tasks:\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'common_test' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'common_test' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_common_test+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"common_test\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_common_test=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_common_test=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_common_test\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_common_test\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'common_test' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'common_test' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_common_test+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_common_test\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_common_test=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_common_test=`$as_echo \"$ac_cv_erlang_lib_dir_common_test\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_common_test\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_common_test\" >&6; }\nERLANG_LIB_DIR_common_test=$ac_cv_erlang_lib_dir_common_test\n\nERLANG_LIB_VER_common_test=$ac_cv_erlang_lib_ver_common_test\n\nif test \"$ac_cv_erlang_lib_dir_common_test\" = \"not found\"; then :\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: erlang-common_test not found - you won't be able to run the unit tests without common_test\" >&5\n$as_echo \"$as_me: erlang-common_test not found - you won't be able to run the unit tests without common_test\" >&6;}\nfi\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'edoc' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'edoc' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_edoc+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"edoc\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_edoc=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_edoc=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_edoc\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_edoc\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'edoc' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'edoc' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_edoc+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_edoc\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_edoc=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_edoc=`$as_echo \"$ac_cv_erlang_lib_dir_edoc\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_edoc\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_edoc\" >&6; }\nERLANG_LIB_DIR_edoc=$ac_cv_erlang_lib_dir_edoc\n\nERLANG_LIB_VER_edoc=$ac_cv_erlang_lib_ver_edoc\n\nif test \"$ac_cv_erlang_lib_dir_edoc\" = \"not found\"; then :\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: erlang-edoc not found - you won't be able to create the documentation or run 'make install'\" >&5\n$as_echo \"$as_me: erlang-edoc not found - you won't be able to create the documentation or run 'make install'\" >&6;}\nfi\n\n\nas_ac_File=`$as_echo \"ac_cv_file_$ERLANG_LIB_DIR_common_test/priv/bin/run_test\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ERLANG_LIB_DIR_common_test/priv/bin/run_test\" >&5\n$as_echo_n \"checking for $ERLANG_LIB_DIR_common_test/priv/bin/run_test... \" >&6; }\nif eval \\${$as_ac_File+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  test \"$cross_compiling\" = yes &&\n  as_fn_error $? \"cannot check for file existence when cross compiling\" \"$LINENO\" 5\nif test -r \"$ERLANG_LIB_DIR_common_test/priv/bin/run_test\"; then\n  eval \"$as_ac_File=yes\"\nelse\n  eval \"$as_ac_File=no\"\nfi\nfi\neval ac_res=\\$$as_ac_File\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_File\"\\\" = x\"yes\"; then :\n  RUN_TEST_FILE=$ERLANG_LIB_DIR_common_test/priv/bin/run_test\nelse\n  as_ac_File=`$as_echo \"ac_cv_file_$ac_cv_erlang_root_dir/bin/run_test\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_cv_erlang_root_dir/bin/run_test\" >&5\n$as_echo_n \"checking for $ac_cv_erlang_root_dir/bin/run_test... \" >&6; }\nif eval \\${$as_ac_File+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  test \"$cross_compiling\" = yes &&\n  as_fn_error $? \"cannot check for file existence when cross compiling\" \"$LINENO\" 5\nif test -r \"$ac_cv_erlang_root_dir/bin/run_test\"; then\n  eval \"$as_ac_File=yes\"\nelse\n  eval \"$as_ac_File=no\"\nfi\nfi\neval ac_res=\\$$as_ac_File\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_File\"\\\" = x\"yes\"; then :\n  RUN_TEST_FILE=$ac_cv_erlang_root_dir/bin/run_test\nelse\n  # Extract the first word of \"run_test\", so it can be a program name with args.\nset dummy run_test; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_RUN_TEST_FILE+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $RUN_TEST_FILE in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_RUN_TEST_FILE=\"$RUN_TEST_FILE\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_RUN_TEST_FILE=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  test -z \"$ac_cv_path_RUN_TEST_FILE\" && ac_cv_path_RUN_TEST_FILE=\"\"\"\"\n  ;;\nesac\nfi\nRUN_TEST_FILE=$ac_cv_path_RUN_TEST_FILE\nif test -n \"$RUN_TEST_FILE\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $RUN_TEST_FILE\" >&5\n$as_echo \"$RUN_TEST_FILE\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\nfi\n\nfi\n\nas_ac_File=`$as_echo \"ac_cv_file_$ERLANG_LIB_DIR_common_test/priv/bin/ct_run\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ERLANG_LIB_DIR_common_test/priv/bin/ct_run\" >&5\n$as_echo_n \"checking for $ERLANG_LIB_DIR_common_test/priv/bin/ct_run... \" >&6; }\nif eval \\${$as_ac_File+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  test \"$cross_compiling\" = yes &&\n  as_fn_error $? \"cannot check for file existence when cross compiling\" \"$LINENO\" 5\nif test -r \"$ERLANG_LIB_DIR_common_test/priv/bin/ct_run\"; then\n  eval \"$as_ac_File=yes\"\nelse\n  eval \"$as_ac_File=no\"\nfi\nfi\neval ac_res=\\$$as_ac_File\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_File\"\\\" = x\"yes\"; then :\n  CT_RUN_FILE=$ERLANG_LIB_DIR_common_test/priv/bin/ct_run\nelse\n  as_ac_File=`$as_echo \"ac_cv_file_$ac_cv_erlang_root_dir/bin/ct_run\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_cv_erlang_root_dir/bin/ct_run\" >&5\n$as_echo_n \"checking for $ac_cv_erlang_root_dir/bin/ct_run... \" >&6; }\nif eval \\${$as_ac_File+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  test \"$cross_compiling\" = yes &&\n  as_fn_error $? \"cannot check for file existence when cross compiling\" \"$LINENO\" 5\nif test -r \"$ac_cv_erlang_root_dir/bin/ct_run\"; then\n  eval \"$as_ac_File=yes\"\nelse\n  eval \"$as_ac_File=no\"\nfi\nfi\neval ac_res=\\$$as_ac_File\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_File\"\\\" = x\"yes\"; then :\n  CT_RUN_FILE=$ac_cv_erlang_root_dir/bin/ct_run\nelse\n  # Extract the first word of \"ct_run\", so it can be a program name with args.\nset dummy ct_run; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_CT_RUN_FILE+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $CT_RUN_FILE in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_CT_RUN_FILE=\"$CT_RUN_FILE\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_CT_RUN_FILE=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  test -z \"$ac_cv_path_CT_RUN_FILE\" && ac_cv_path_CT_RUN_FILE=\"\"\"\"\n  ;;\nesac\nfi\nCT_RUN_FILE=$ac_cv_path_CT_RUN_FILE\nif test -n \"$CT_RUN_FILE\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $CT_RUN_FILE\" >&5\n$as_echo \"$CT_RUN_FILE\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\nfi\n\nfi\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for unit-test runner\" >&5\n$as_echo_n \"checking for unit-test runner... \" >&6; }\nif test -n \"$RUN_TEST_FILE\"; then\n  RUN_TEST=\"$RUN_TEST_FILE\"\n\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $RUN_TEST_FILE\" >&5\n$as_echo \"$RUN_TEST_FILE\" >&6; }\nelif test -n \"$CT_RUN_FILE\"; then\n  RUN_TEST=\"$CT_RUN_FILE\"\n\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $CT_RUN_FILE\" >&5\n$as_echo \"$CT_RUN_FILE\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: neither run_test nor ct_run found - on erlang < R14 consider running install.sh in the common_test directory otherwise you won't be able to run the unit tests\" >&5\n$as_echo \"neither run_test nor ct_run found - on erlang < R14 consider running install.sh in the common_test directory otherwise you won't be able to run the unit tests\" >&6; }\nfi\n\n# toke\nERLANG_TOKE_FLAGS=\nif test \"x$TOKEPREFIX\" != xno; then :\n\n            ERLANG_TOKE_MESSAGE=\n            if test \"x$TOKEPREFIX\" != x; then :\n  export ERL_LIBS=\"$TOKEPREFIX\"\nfi\n            { $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'toke' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'toke' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_toke+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"toke\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_toke=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_toke=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_toke\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_toke\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'toke' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'toke' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_toke+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_toke\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_toke=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_toke=`$as_echo \"$ac_cv_erlang_lib_dir_toke\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_toke\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_toke\" >&6; }\nERLANG_LIB_DIR_toke=$ac_cv_erlang_lib_dir_toke\n\nERLANG_LIB_VER_toke=$ac_cv_erlang_lib_ver_toke\n\nif test \"$ac_cv_erlang_lib_dir_toke\" = \"not found\"; then :\n  ERLANG_TOKE_MESSAGE=\"toke erlang library not found\"\nelse\n  as_ac_File=`$as_echo \"ac_cv_file_$ERLANG_LIB_DIR_toke/priv/libtoke.so\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ERLANG_LIB_DIR_toke/priv/libtoke.so\" >&5\n$as_echo_n \"checking for $ERLANG_LIB_DIR_toke/priv/libtoke.so... \" >&6; }\nif eval \\${$as_ac_File+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  test \"$cross_compiling\" = yes &&\n  as_fn_error $? \"cannot check for file existence when cross compiling\" \"$LINENO\" 5\nif test -r \"$ERLANG_LIB_DIR_toke/priv/libtoke.so\"; then\n  eval \"$as_ac_File=yes\"\nelse\n  eval \"$as_ac_File=no\"\nfi\nfi\neval ac_res=\\$$as_ac_File\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_File\"\\\" = x\"yes\"; then :\n  ERLANG_TOKE_FLAGS=\"-pa $ERLANG_LIB_DIR_toke/ebin\"\nelse\n  ERLANG_TOKE_MESSAGE=\"toke library libtoke.so not found\"\nfi\n\n\nfi\n\n            if test \"x$TOKEPREFIX\" != x; then :\n  export ERL_LIBS=\"\"\nfi\n            if test \"x$ERLANG_TOKE_MESSAGE\" != x; then :\n  if test \"x$ENABLE_TOKE\" != xyes; then :\n  ERLANG_TOKE_MESSAGE=\"$ERLANG_TOKE_MESSAGE, disabling toke support...\"\n                 { $as_echo \"$as_me:${as_lineno-$LINENO}: $ERLANG_TOKE_MESSAGE\" >&5\n$as_echo \"$as_me: $ERLANG_TOKE_MESSAGE\" >&6;}\nelse\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"$ERLANG_TOKE_MESSAGE\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\nfi\n\nfi\n\n\n# hanoidb\nERLANG_HANOIDB_FLAGS=\nif test \"x$HANOIDBPREFIX\" != xno; then :\n\n            ERLANG_HANOIDB_MESSAGE=\n            if test \"x$HANOIDBPREFIX\" != x; then :\n  export ERL_LIBS=\"$HANOIDBPREFIX\"\nfi\n            { $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'hanoidb' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'hanoidb' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_hanoidb+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"hanoidb\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_hanoidb=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_hanoidb=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_hanoidb\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_hanoidb\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'hanoidb' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'hanoidb' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_hanoidb+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_hanoidb\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_hanoidb=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_hanoidb=`$as_echo \"$ac_cv_erlang_lib_dir_hanoidb\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_hanoidb\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_hanoidb\" >&6; }\nERLANG_LIB_DIR_hanoidb=$ac_cv_erlang_lib_dir_hanoidb\n\nERLANG_LIB_VER_hanoidb=$ac_cv_erlang_lib_ver_hanoidb\n\nif test \"$ac_cv_erlang_lib_dir_hanoidb\" = \"not found\"; then :\n  ERLANG_HANOIDB_MESSAGE=\"hanoidb erlang not found\"\nelse\n  ERLANG_HANOIDB_FLAGS=\"-pa $ERLANG_LIB_DIR_hanoidb/ebin\"\nfi\n\n            if test \"x$HANOIDBPREFIX\" != x; then :\n  export ERL_LIBS=\"\"\nfi\n            if test \"x$ERLANG_HANOIDB_MESSAGE\" != x; then :\n  if test \"x$ENABLE_HANOIDB\" != xyes; then :\n  ERLANG_HANOIDB_MESSAGE=\"$ERLANG_HANOIDB_MESSAGE, disabling hanoidb support...\"\n                 { $as_echo \"$as_me:${as_lineno-$LINENO}: $ERLANG_HANOIDB_MESSAGE\" >&5\n$as_echo \"$as_me: $ERLANG_HANOIDB_MESSAGE\" >&6;}\nelse\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"$ERLANG_HANOIDB_MESSAGE\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\nfi\n\nfi\n\n\n# bitcask\nERLANG_BITCASK_FLAGS=\nBITCASK_LIBS=\nif test \"x$BITCASKPREFIX\" != xno; then :\n\n            ERLANG_BITCASK_MESSAGE=\n            if test \"x$BITCASKPREFIX\" != x; then :\n  export ERL_LIBS=\"$BITCASKPREFIX\"\nfi\n            { $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'bitcask' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'bitcask' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_bitcask+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"bitcask\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_bitcask=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_bitcask=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_bitcask\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_bitcask\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'bitcask' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'bitcask' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_bitcask+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_bitcask\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_bitcask=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_bitcask=`$as_echo \"$ac_cv_erlang_lib_dir_bitcask\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_bitcask\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_bitcask\" >&6; }\nERLANG_LIB_DIR_bitcask=$ac_cv_erlang_lib_dir_bitcask\n\nERLANG_LIB_VER_bitcask=$ac_cv_erlang_lib_ver_bitcask\n\nif test \"$ac_cv_erlang_lib_dir_bitcask\" = \"not found\"; then :\n  ERLANG_BITCASK_MESSAGE=\"bitcask erlang not found\"\nelse\n\n                EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_bitcask}\"\n                EDOCMACROS=\"${EDOCMACROS}, {have_bitcask, true}\"\n                DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_bitcask\"\n                ERLANG_BITCASK_FLAGS=\"-pa $ERLANG_LIB_DIR_bitcask/ebin\"\n                BITCASK_LIBS=\"$ERLANG_LIB_DIR_bitcask\"\n\nfi\n\n            if test \"x$BITCASKPREFIX\" != x; then :\n  export ERL_LIBS=\"\"\nfi\n            if test \"x$ERLANG_BITCASK_MESSAGE\" != x; then :\n  if test \"x$ENABLE_BITCASK\" != xyes; then :\n  ERLANG_BITCASK_MESSAGE=\"$ERLANG_BITCASK_MESSAGE, disabling bitcask support...\"\n                 { $as_echo \"$as_me:${as_lineno-$LINENO}: $ERLANG_BITCASK_MESSAGE\" >&5\n$as_echo \"$as_me: $ERLANG_BITCASK_MESSAGE\" >&6;}\nelse\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"$ERLANG_BITCASK_MESSAGE\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\nfi\n\nfi\n\n\n\n# erlang_js\nERLANG_ERLANGJS_FLAGS=\nif test \"x$ERLANGJSPREFIX\" != xno; then :\n\n            ERLANG_ERLANGJS_MESSAGE=\n            if test \"x$ERLANGJSPREFIX\" != x; then :\n  export ERL_LIBS=\"$ERLANGJSPREFIX\"\nfi\n            { $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'erlang_js' library subdirectory\" >&5\n$as_echo_n \"checking for Erlang/OTP 'erlang_js' library subdirectory... \" >&6; }\nif ${ac_cv_erlang_lib_dir_erlang_js+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n     if test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\n\t    ReturnValue = case code:lib_dir(\"erlang_js\") of\n\t    {error, bad_name} ->\n\t\tfile:write_file(\"conftest.out\", \"not found\\n\"),\n\t\t1;\n\t    LibDir ->\n\t\tfile:write_file(\"conftest.out\", LibDir),\n\t\t0\n\t    end,\n\t    halt(ReturnValue)\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  ac_cv_erlang_lib_dir_erlang_js=`cat conftest.out`\n\t rm -f conftest.out\nelse\n  if test ! -f conftest.out; then\n\t     { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"test Erlang program execution failed\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n\t else\n\t     ac_cv_erlang_lib_dir_erlang_js=\"not found\"\n\t     rm -f conftest.out\n\t fi\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n     ac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_dir_erlang_js\" >&5\n$as_echo \"$ac_cv_erlang_lib_dir_erlang_js\" >&6; }\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for Erlang/OTP 'erlang_js' library version\" >&5\n$as_echo_n \"checking for Erlang/OTP 'erlang_js' library version... \" >&6; }\nif ${ac_cv_erlang_lib_ver_erlang_js+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test \"$ac_cv_erlang_lib_dir_erlang_js\" = \"not found\"; then :\n  ac_cv_erlang_lib_ver_erlang_js=\"not found\"\nelse\n  ac_cv_erlang_lib_ver_erlang_js=`$as_echo \"$ac_cv_erlang_lib_dir_erlang_js\" | sed -n -e 's,^.*-\\([^/-]*\\)$,\\1,p'`\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_erlang_lib_ver_erlang_js\" >&5\n$as_echo \"$ac_cv_erlang_lib_ver_erlang_js\" >&6; }\nERLANG_LIB_DIR_erlang_js=$ac_cv_erlang_lib_dir_erlang_js\n\nERLANG_LIB_VER_erlang_js=$ac_cv_erlang_lib_ver_erlang_js\n\nif test \"$ac_cv_erlang_lib_dir_erlang_js\" = \"not found\"; then :\n  ERLANG_ERLANGJS_MESSAGE=\"erlang_js erlang library not found\"\nelse\n  as_ac_File=`$as_echo \"ac_cv_file_$ERLANG_LIB_DIR_erlang_js/priv/erlang_js_drv.so\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ERLANG_LIB_DIR_erlang_js/priv/erlang_js_drv.so\" >&5\n$as_echo_n \"checking for $ERLANG_LIB_DIR_erlang_js/priv/erlang_js_drv.so... \" >&6; }\nif eval \\${$as_ac_File+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  test \"$cross_compiling\" = yes &&\n  as_fn_error $? \"cannot check for file existence when cross compiling\" \"$LINENO\" 5\nif test -r \"$ERLANG_LIB_DIR_erlang_js/priv/erlang_js_drv.so\"; then\n  eval \"$as_ac_File=yes\"\nelse\n  eval \"$as_ac_File=no\"\nfi\nfi\neval ac_res=\\$$as_ac_File\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_File\"\\\" = x\"yes\"; then :\n  ERLANG_ERLANGJS_FLAGS=\"-pa $ERLANG_LIB_DIR_erlang_js/ebin\"\nelse\n  ERLANG_ERLANGJS_MESSAGE=\"erlang_js library erlang_js_drv.so not found\"\nfi\n\n\nfi\n\n            if test \"x$ERLANGJSPREFIX\" != x; then :\n  export ERL_LIBS=\"\"\nfi\n            if test \"x$ERLANG_ERLANGJS_MESSAGE\" != x; then :\n  if test \"x$ENABLE_ERLANGJS\" != xyes; then :\n  ERLANG_ERLANGJS_MESSAGE=\"$ERLANG_ERLANGJS_MESSAGE, disabling erlang_js support...\"\n                 { $as_echo \"$as_me:${as_lineno-$LINENO}: $ERLANG_ERLANGJS_MESSAGE\" >&5\n$as_echo \"$as_me: $ERLANG_ERLANGJS_MESSAGE\" >&6;}\nelse\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"$ERLANG_ERLANGJS_MESSAGE\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\nfi\n\nfi\n\n\ncrypto_start=`\"${ERL}\" -noshell -eval 'io:format(\"~p~n\", [crypto:start()]), erlang:halt().' | tail -1`\nif test \"${crypto_start}\" = \"ok\"; then :\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: crypto:start() is available\" >&5\n$as_echo \"$as_me: crypto:start() is available\" >&6;}\nelse\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot start the crypto subsystem\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\n\ncrypto_rand_uniform=`\"${ERL}\" -noshell -eval 'error_logger:tty(false), code:load_file(crypto), io:format(\"~p~n\", [erlang:function_exported(crypto,rand_uniform,2)]), erlang:halt().' | tail -1`\nif test \"${crypto_rand_uniform}\" = \"true\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_crypto_randuniform_support}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_crypto_randuniform_support, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_crypto_randuniform_support\"\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: crypto:rand_uniform/2 is available\" >&5\n$as_echo \"$as_me: crypto:rand_uniform/2 is available\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: crypto:rand_uniform/2 is not available\" >&5\n$as_echo \"$as_me: crypto:rand_uniform/2 is not available\" >&6;}\nfi\n\ncrypto_hash=`\"${ERL}\" -noshell -eval 'error_logger:tty(false), code:load_file(crypto), io:format(\"~p~n\", [erlang:function_exported(crypto,hash,2)]), erlang:halt().' | tail -1`\nif test \"${crypto_hash}\" = \"true\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, with_crypto_hash}\"\n       EDOCMACROS=\"${EDOCMACROS}, {with_crypto_hash, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dwith_crypto_hash\"\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: crypto:hash/2 is available and usable\" >&5\n$as_echo \"$as_me: crypto:hash/2 is available and usable\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: crypto:hash/2 is not available or not usable, falling back to crypto:md5/1 and crypto:sha1/1\" >&5\n$as_echo \"$as_me: crypto:hash/2 is not available or not usable, falling back to crypto:md5/1 and crypto:sha1/1\" >&6;}\nfi\n\ncrypto_bytes_to_integer=`\"${ERL}\" -noshell -eval 'error_logger:tty(false), code:load_file(crypto), io:format(\"~p~n\", [erlang:function_exported(crypto,bytes_to_integer,1)]), erlang:halt().' | tail -1`\nif test \"${crypto_bytes_to_integer}\" = \"true\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, with_crypto_bytes_to_integer}\"\n       EDOCMACROS=\"${EDOCMACROS}, {with_crypto_bytes_to_integer, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dwith_crypto_bytes_to_integer\"\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: crypto:bytes_to_integer/1 is available and usable\" >&5\n$as_echo \"$as_me: crypto:bytes_to_integer/1 is available and usable\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: crypto:bytes_to_integer/1 is not available or not usable, falling back to rand:uniform/2\" >&5\n$as_echo \"$as_me: crypto:bytes_to_integer/1 is not available or not usable, falling back to rand:uniform/2\" >&6;}\nfi\n\nct_line=`\"${ERL}\" -noshell -eval 'error_logger:tty(false), code:load_file(ct_line), io:format(\"~p~n\", [erlang:function_exported(ct_line,parse_transform,2)]), erlang:halt().' | tail -1`\nif test \"${ct_line}\" = \"true\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_ctline_support}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_ctline_support, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_ctline_support\"\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: ct_line is available\" >&5\n$as_echo \"$as_me: ct_line is available\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: ct_line is not available\" >&5\n$as_echo \"$as_me: ct_line is not available\" >&6;}\nfi\n\nrand_module=`\"${ERL}\" -noshell -eval 'error_logger:tty(false), code:load_file(rand), io:format(\"~p~n\", [erlang:function_exported(rand,uniform,0)]), erlang:halt().' | tail -1`\nif test \"${rand_module}\" = \"true\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, with_rand}\"\n       EDOCMACROS=\"${EDOCMACROS}, {with_rand, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dwith_rand\"\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: rand:uniform/0 is available and usable\" >&5\n$as_echo \"$as_me: rand:uniform/0 is available and usable\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: rand:uniform/0 is not available or not usable, falling back to random:uniform/0\" >&5\n$as_echo \"$as_me: rand:uniform/0 is not available or not usable, falling back to random:uniform/0\" >&6;}\nfi\n\nif test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n-callback init() -> ok.\nstart() ->\nok\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_callback_support}\"\n        EDOCMACROS=\"${EDOCMACROS}, {have_callback_support, true}\"\n        DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_callback_support\"\n        { $as_echo \"$as_me:${as_lineno-$LINENO}: -callback is available\" >&5\n$as_echo \"$as_me: -callback is available\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: -callback is not available\" >&5\n$as_echo \"$as_me: -callback is not available\" >&6;}\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n\nmaps_module=`\"${ERL}\" -noshell -eval 'error_logger:tty(false), code:load_file(maps), io:format(\"~p~n\", [erlang:function_exported(maps,new,0)]), erlang:halt().' | tail -1`\nif test \"${maps_module}\" = \"true\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, with_maps}\"\n       EDOCMACROS=\"${EDOCMACROS}, {with_maps, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dwith_maps\"\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: maps are available and seem to be usable\" >&5\n$as_echo \"$as_me: maps are available and seem to be usable\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: maps are not available or not usable, falling back to gb_trees\" >&5\n$as_echo \"$as_me: maps are not available or not usable, falling back to gb_trees\" >&6;}\nfi\n\n# creates forms specifying a minimal erlang module which spezifies a\n# type based on dict:dict() and checks if compilation of this module fails\ndict_type=`\"${ERL}\" -noshell -eval 'X = fun(S) -> element(2,erl_parse:parse_form(element(2,erl_scan:string(S)))) end, io:format(\"~p~n\", [element(1, compile:forms([X(\"-module(test).\"), X(\"-type test() :: dict:dict().\")]))]), erlang:halt().' 2> /dev/null | tail -1`\nif test \"${dict_type}\" = \"ok\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, namespaced_dict}\"\n       EDOCMACROS=\"${EDOCMACROS}, {namespaced_dict, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dnamespaced_dict\"\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: erlang dict type is defined as dict:dict()\" >&5\n$as_echo \"$as_me: erlang dict type is defined as dict:dict()\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: erlang dict type is defined as dict()\" >&5\n$as_echo \"$as_me: erlang dict type is defined as dict()\" >&6;}\nfi\n\n###########################################################\n#\n# check for file:send_file/5 for yaws\n#\n###########################################################\nYAWS_OPTIONS=\nHAVE_ERLANG_SENDFILE=false\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for file:sendfile/5\" >&5\n$as_echo_n \"checking for file:sendfile/5... \" >&6; }\nfile_sendfile=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(file), io:format(\"~p~n\",[erlang:function_exported(file,sendfile,5)]), erlang:halt().' | tail -1`\nif test \"$file_sendfile\" = true; then\n  if test $ERTS_MAJOR -gt 5 -o \\\n         '(' $ERTS_MAJOR -eq 5 -a $ERTS_MINOR -gt 9 ')' -o \\\n         '(' $ERTS_MAJOR -eq 5 -a $ERTS_MINOR -eq 9 -a $ERTS_MAINT -ge 1 ')'; then :\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: yes\" >&5\n$as_echo \"yes\" >&6; }\n         HAVE_ERLANG_SENDFILE=true\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\nelse\n        { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\nif test \"$HAVE_ERLANG_SENDFILE\" = true; then :\n  YAWS_OPTIONS=\", {d, 'HAVE_ERLANG_SENDFILE'}\"\n       YAWS_OPTIONS_DIALYZER=\" -DHAVE_ERLANG_SENDFILE\"\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: using file:sendfile/5 for yaws\" >&5\n$as_echo \"$as_me: using file:sendfile/5 for yaws\" >&6;}\nelse\n  YAWS_OPTIONS=\"\"\n       YAWS_OPTIONS_DIALYZER=\"\"\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: using fallback gen_tcp:send/2 for yaws\" >&5\n$as_echo \"$as_me: using fallback gen_tcp:send/2 for yaws\" >&6;}\nfi\n\nif test \"${crypto_hash}\" = \"true\"; then :\n  YAWS_OPTIONS=\"${YAWS_OPTIONS}, {d, 'HAVE_CRYPTO_HASH'}\"\n       YAWS_OPTIONS_DIALYZER=\"${YAWS_OPTIONS_DIALYZER} -DHAVE_CRYPTO_HASH\"\nfi\n\n###########################################################\n#\n# check for erlang:timestamp/0 for yaws\n#\n###########################################################\nerlang_timestamp=`\"${ERL}\" -noshell -eval 'io:format(\"~p~n\", [erlang:function_exported(erlang,timestamp,0)]), erlang:halt().' | tail -1`\nif test \"${erlang_timestamp}\" = \"true\"; then :\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: erlang:timestamp() is available\" >&5\n$as_echo \"$as_me: erlang:timestamp() is available\" >&6;}\nelse\n  YAWS_OPTIONS=\"${YAWS_OPTIONS}, {d, 'HAVE_ERLANG_NOW'}\"\n       YAWS_OPTIONS_DIALYZER=\"${YAWS_OPTIONS_DIALYZER} -DHAVE_ERLANG_NOW\"\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: erlang:timestamp() is not available\" >&5\n$as_echo \"$as_me: erlang:timestamp() is not available\" >&6;}\nfi\n\n###########################################################\n#\n# check for ssl:getstat/2 for ssl\n#\n###########################################################\n## https://github.com/erlang/otp/commit/84fd2c325c9e38b5ea2307b6133c3d15b33a3241\nSSL_GETSTAT=false\nssl_getstat=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(ssl),io:format(\"~p~n\", [erlang:function_exported(ssl,getstat,2)]), erlang:halt().' | tail -1`\nif test \"${ssl_getstat}\" = \"true\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_ssl_getstat}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_ssl_getstat, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_ssl_getstat\"\n       SSL_GETSTAT=true\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: ssl:getstat() is available\" >&5\n$as_echo \"$as_me: ssl:getstat() is available\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: ssl:getstat() is not available\" >&5\n$as_echo \"$as_me: ssl:getstat() is not available\" >&6;}\nfi\n\n###########################################################\n#\n# check for ssl:handshake/1 for ssl\n#\n###########################################################\n## https://github.com/erlang/otp/commit/8dcfffd4fb8520239b189357e81122d4fdacb42d\nSSL_HANDSHAKE=false\nssl_handshake=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(ssl),io:format(\"~p~n\", [erlang:function_exported(ssl,handshake,1)]), erlang:halt().' | tail -1`\nif test \"${ssl_handshake}\" = \"true\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_ssl_handshake}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_ssl_handshake, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_ssl_handshake\"\n       SSL_HANDSHAKE=true\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: ssl:handshake() is available\" >&5\n$as_echo \"$as_me: ssl:handshake() is available\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: ssl:handshake() is not available\" >&5\n$as_echo \"$as_me: ssl:handshake() is not available\" >&6;}\nfi\n\n###########################################################\n#\n# check for new stacktrace syntax\n#\n###########################################################\n# http://erlang.org/eeps/eep-0047.md\n\nerlang_cv_new_stacktrace=0\nac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\nif test \"$cross_compiling\" = yes; then :\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot run test program while cross compiling\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse\n  cat > conftest.$ac_ext <<_ACEOF\n-module(conftest).\n-export([start/0]).\n\nstart() ->\ntry\n  5\ncatch\n  C:E:Stk ->\n          Stk\nend\n.\n\n_ACEOF\nif ac_fn_erl_try_run \"$LINENO\"; then :\n  erlang_cv_new_stacktrace=true\nelse\n  erlang_cv_new_stacktrace=false\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\nac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nif test \"${erlang_cv_new_stacktrace}\" = \"true\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_new_stacktrace}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_new_stacktrace, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_new_stacktrace\"\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: new stacktrace syntax is available\" >&5\n$as_echo \"$as_me: new stacktrace syntax is available\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: new stacktrace syntax is not available\" >&5\n$as_echo \"$as_me: new stacktrace syntax is not available\" >&6;}\nfi\n\n###########################################################\n#\n# check for socket:open/2 since 22.0\n#\n###########################################################\n# https://github.com/erlang/otp/commit/3ca71520bfb664f0ea809ffdf41505936e4d5e90\nSOCKET_OPEN=false\nsocket_open=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(socket),io:format(\"~p~n\", [erlang:function_exported(socket,open,2)]), erlang:halt().' | tail -1`\nif test \"${socket_open}\" = \"true\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_socket_open}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_socket_open, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_socket_open\"\n       SOCKET_OPEN=true\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: socket:open/2 is available\" >&5\n$as_echo \"$as_me: socket:open/2 is available\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: socket:open/2 is not available\" >&5\n$as_echo \"$as_me: socket:open/2 is not available\" >&6;}\nfi\n\n###########################################################\n#\n# check for persistent_term:get/2 since 21.2.\n#\n###########################################################\n# https://github.com/erlang/otp/commit/805748eb668d5562fe17f3172cdae07a86166c3f\nPERSISTENT_TERM_GET=false\npersistent_term_get=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(persistent_term),io:format(\"~p~n\", [erlang:function_exported(persistent_term,get,1)]), erlang:halt().' | tail -1`\nif test \"${persistent_term_get}\" = \"true\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_persistent_term_get}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_persistent_term_get, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_persistent_term_get\"\n       PERSISTENT_TERM_GET=true\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: persistent_term:get/2 is available\" >&5\n$as_echo \"$as_me: persistent_term:get/2 is available\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: persistent_term:get/2 is not available\" >&5\n$as_echo \"$as_me: persistent_term:get/2 is not available\" >&6;}\nfi\n\n###########################################################\n#\n# check for counters:get/2 since 21.2.\n#\n###########################################################\n# https://github.com/erlang/otp/commit/fefb5d039e87ff7137e78b3d5f2eaf01e498ec4d\nCOUNTERS_GET=false\ncounters_get=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(counters),io:format(\"~p~n\", [erlang:function_exported(counters,get,2)]), erlang:halt().' | tail -1`\nif test \"${counters_get}\" = \"true\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_counters_get}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_counters_get, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_counters_get\"\n       COUNTERS_GET=true\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: counters:get/2 is available\" >&5\n$as_echo \"$as_me: counters:get/2 is available\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: counters:get/2 is not available\" >&5\n$as_echo \"$as_me: counters:get/2 is not available\" >&6;}\nfi\n\n###########################################################\n#\n# check for atomics:new/2 since 21.2.\n#\n###########################################################\n# https://github.com/erlang/otp/commit/1315c6457e49595fdd3f91693c0506964416c9f0\nATOMICS_NEW=false\natomics_new=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(atomics),io:format(\"~p~n\", [erlang:function_exported(atomics,new,2)]), erlang:halt().' | tail -1`\nif test \"${atomics_new}\" = \"true\"; then :\n  EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_atomics_new}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_atomics_new, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_atomics_new\"\n       ATOMICS_NEW=true\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: atomics:new/2 is available\" >&5\n$as_echo \"$as_me: atomics:new/2 is available\" >&6;}\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: atomics:new/2 is not available\" >&5\n$as_echo \"$as_me: atomics:new/2 is not available\" >&6;}\nfi\n\n###########################################################\n#\n# all defines together...\n#\n###########################################################\n\nEMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}\"\nEDOCMACROS=\"${EDOCMACROS#, }\"\nDIALYZER_FLAGS=\"${DIALYZER_FLAGS## }${YAWS_OPTIONS_DIALYZER}\"\n\n\n\n\n\n\n\n\n###########################################################\n#\n# check for routing table type\n#\n###########################################################\n\nRT=rt_chord\n\n# Check whether --with-rt was given.\nif test \"${with_rt+set}\" = set; then :\n  withval=$with_rt; if test \"$withval\"; then\n               RT=\"$withval\"\n             fi\nfi\n\n\n\n###########################################################\n#\n# check java-functions, build-classpath availability for java-api/scalaris\n#\n###########################################################\n\nJAVAFUNCTIONS=\n\n# Check whether --with-java-functions was given.\nif test \"${with_java_functions+set}\" = set; then :\n  withval=$with_java_functions; with_java_functions=$withval\nelse\n  with_java_functions=/usr/share/java-utils/java-functions\nfi\n\n\nas_ac_File=`$as_echo \"ac_cv_file_$with_java_functions\" | $as_tr_sh`\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $with_java_functions\" >&5\n$as_echo_n \"checking for $with_java_functions... \" >&6; }\nif eval \\${$as_ac_File+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  test \"$cross_compiling\" = yes &&\n  as_fn_error $? \"cannot check for file existence when cross compiling\" \"$LINENO\" 5\nif test -r \"$with_java_functions\"; then\n  eval \"$as_ac_File=yes\"\nelse\n  eval \"$as_ac_File=no\"\nfi\nfi\neval ac_res=\\$$as_ac_File\n\t       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\n$as_echo \"$ac_res\" >&6; }\nif eval test \\\"x\\$\"$as_ac_File\"\\\" = x\"yes\"; then :\n  JAVAFUNCTIONS=$with_java_functions\nelse\n  JAVAFUNCTIONS=$with_java_functions\n               { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: java-functions was not found in \\\"$with_java_functions\\\" (java-api/scalaris may fail)\" >&5\n$as_echo \"$as_me: WARNING: java-functions was not found in \\\"$with_java_functions\\\" (java-api/scalaris may fail)\" >&2;}\nfi\n\n\n\nBUILDCLASSPATH=\n\n# Check whether --with-build-classpath was given.\nif test \"${with_build_classpath+set}\" = set; then :\n  withval=$with_build_classpath; if test -n \"$withval\"; then\n               BUILDCLASSPATH=\"$withval\"\n             fi\nelse\n  # Extract the first word of \"build-classpath\", so it can be a program name with args.\nset dummy build-classpath; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_BUILDCLASSPATH+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $BUILDCLASSPATH in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_BUILDCLASSPATH=\"$BUILDCLASSPATH\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_BUILDCLASSPATH=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  ;;\nesac\nfi\nBUILDCLASSPATH=$ac_cv_path_BUILDCLASSPATH\nif test -n \"$BUILDCLASSPATH\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $BUILDCLASSPATH\" >&5\n$as_echo \"$BUILDCLASSPATH\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\nfi\n\n\n\nif test ! -x \"$BUILDCLASSPATH\"; then :\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: build-classpath was not found in \\\"$BUILDCLASSPATH\\\" or is not executable (java-api/scalaris may fail)\" >&5\n$as_echo \"$as_me: WARNING: build-classpath was not found in \\\"$BUILDCLASSPATH\\\" or is not executable (java-api/scalaris may fail)\" >&2;}\nfi\n\n###########################################################\n#\n# check ruby\n#\n###########################################################\n\nENABLERUBYINSTALL=\nRUBYSITELIBDIR=\n\n# Check whether --with-ruby-sitelibdir was given.\nif test \"${with_ruby_sitelibdir+set}\" = set; then :\n  withval=$with_ruby_sitelibdir; if test -n \"$withval\"; then\n               RUBYSITELIBDIR=\"$withval\"\n               ENABLERUBYINSTALL=\"install-ruby\"\n             fi\nfi\n\n\n\n\n###########################################################\n#\n# check python, python3\n#\n###########################################################\nPYTHON2SITELIBDIR=\nENABLEPYTHON2INSTALL=\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for python 2.x >= 2.6\" >&5\n$as_echo_n \"checking for python 2.x >= 2.6... \" >&6; }\nif ${ac_cv_path_PYTHON2+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -z \"$PYTHON2\"; then\n  ac_path_PYTHON2_found=false\n  # Loop through the user's path and test for each of PROGNAME-LIST\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_prog in python python2; do\n    for ac_exec_ext in '' $ac_executable_extensions; do\n      ac_path_PYTHON2=\"$as_dir/$ac_prog$ac_exec_ext\"\n      as_fn_executable_p \"$ac_path_PYTHON2\" || continue\n$ac_path_PYTHON2 -V 2>&1 | grep \"^Python 2.[6789]\" > /dev/null && \\\n      PYTHON2SITELIBDIR=`$ac_path_PYTHON2 -c 'from distutils.sysconfig import get_python_lib; print(get_python_lib(0,0,prefix=\"${prefix}\"))'  2>/dev/null` && \\\n      ac_cv_path_PYTHON2=$ac_path_PYTHON2 ac_path_PYTHON2_found=:\n      $ac_path_PYTHON2_found && break 3\n    done\n  done\n  done\nIFS=$as_save_IFS\n  if test -z \"$ac_cv_path_PYTHON2\"; then\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: result: not found\" >&5\n$as_echo \"not found\" >&6; }\n  fi\nelse\n  ac_cv_path_PYTHON2=$PYTHON2\nfi\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_PYTHON2\" >&5\n$as_echo \"$ac_cv_path_PYTHON2\" >&6; }\n\nif test \"$ac_cv_path_PYTHON2\" != \"\"; then :\n  PYTHON2=$ac_cv_path_PYTHON2\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: checking for python 2.x sitelibdir\" >&5\n$as_echo_n \"checking for python 2.x sitelibdir... \" >&6; }\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $PYTHON2SITELIBDIR\" >&5\n$as_echo \"$PYTHON2SITELIBDIR\" >&6; }\n       ENABLEPYTHON2INSTALL=\"install-python\"\nelse\n  PYTHON2=$FALSE\nfi\n\n\n\n\nPYTHON3SITELIBDIR=\nENABLEPYTHON3INSTALL=\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for python 3.x\" >&5\n$as_echo_n \"checking for python 3.x... \" >&6; }\nif ${ac_cv_path_PYTHON3+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -z \"$PYTHON3\"; then\n  ac_path_PYTHON3_found=false\n  # Loop through the user's path and test for each of PROGNAME-LIST\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_prog in python3 python; do\n    for ac_exec_ext in '' $ac_executable_extensions; do\n      ac_path_PYTHON3=\"$as_dir/$ac_prog$ac_exec_ext\"\n      as_fn_executable_p \"$ac_path_PYTHON3\" || continue\n$ac_path_PYTHON3 -V 2>&1 | grep \"^Python 3.\" > /dev/null && \\\n      PYTHON3SITELIBDIR=`$ac_path_PYTHON3 -c 'from distutils.sysconfig import get_python_lib; print(get_python_lib(0,0,prefix=\"${prefix}\"))'  2>/dev/null` && \\\n      ac_cv_path_PYTHON3=$ac_path_PYTHON3 ac_path_PYTHON3_found=:\n      $ac_path_PYTHON3_found && break 3\n    done\n  done\n  done\nIFS=$as_save_IFS\n  if test -z \"$ac_cv_path_PYTHON3\"; then\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: result: not found\" >&5\n$as_echo \"not found\" >&6; }\n  fi\nelse\n  ac_cv_path_PYTHON3=$PYTHON3\nfi\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_PYTHON3\" >&5\n$as_echo \"$ac_cv_path_PYTHON3\" >&6; }\n\nPYTHON3=\nPYTHON3_2TO3=\nENABLEPYTHON3=python3-not-found\nENABLEPYTHON3INSTALL=\nif test \"$ac_cv_path_PYTHON3\" != \"\"; then :\n  PYTHON3=$ac_cv_path_PYTHON3\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: checking for python 3.x sitelibdir\" >&5\n$as_echo_n \"checking for python 3.x sitelibdir... \" >&6; }\n       { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $PYTHON3SITELIBDIR\" >&5\n$as_echo \"$PYTHON3SITELIBDIR\" >&6; }\n       # Extract the first word of \"2to3\", so it can be a program name with args.\nset dummy 2to3; ac_word=$2\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\n$as_echo_n \"checking for $ac_word... \" >&6; }\nif ${ac_cv_path_PYTHON3_2TO3+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  case $PYTHON3_2TO3 in\n  [\\\\/]* | ?:[\\\\/]*)\n  ac_cv_path_PYTHON3_2TO3=\"$PYTHON3_2TO3\" # Let the user override the test with a path.\n  ;;\n  *)\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_exec_ext in '' $ac_executable_extensions; do\n  if as_fn_executable_p \"$as_dir/$ac_word$ac_exec_ext\"; then\n    ac_cv_path_PYTHON3_2TO3=\"$as_dir/$ac_word$ac_exec_ext\"\n    $as_echo \"$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext\" >&5\n    break 2\n  fi\ndone\n  done\nIFS=$as_save_IFS\n\n  test -z \"$ac_cv_path_PYTHON3_2TO3\" && ac_cv_path_PYTHON3_2TO3=\"\"\"\"\n  ;;\nesac\nfi\nPYTHON3_2TO3=$ac_cv_path_PYTHON3_2TO3\nif test -n \"$PYTHON3_2TO3\"; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: $PYTHON3_2TO3\" >&5\n$as_echo \"$PYTHON3_2TO3\" >&6; }\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\n$as_echo \"no\" >&6; }\nfi\n\n\n       if test -x \"$PYTHON3_2TO3\"; then :\n  ENABLEPYTHON3=\"python3-compile\"\n              ENABLEPYTHON3INSTALL=\"install-python3\"\nelse\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: 2to3 not found - you won't be able to build the Python3 API\" >&5\n$as_echo \"$as_me: WARNING: 2to3 not found - you won't be able to build the Python3 API\" >&2;}\nfi\nelse\n  PYTHON3=$FALSE\nfi\n\n\n\n\n\n\n###########################################################\n#\n# check MACOSX vs. Linux for flexbisonparse driver\n#\n###########################################################\n\nSAVECFLAGS=$CFLAGS\nCFLAGS=\"-I$ERLANG_ROOT_DIR/erts-$ERLANG_LIB_VER_erts/include $CFLAGS\"\nac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\nac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking how to run the C preprocessor\" >&5\n$as_echo_n \"checking how to run the C preprocessor... \" >&6; }\n# On Suns, sometimes $CPP names a directory.\nif test -n \"$CPP\" && test -d \"$CPP\"; then\n  CPP=\nfi\nif test -z \"$CPP\"; then\n  if ${ac_cv_prog_CPP+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n      # Double quotes because CPP needs to be expanded\n    for CPP in \"$CC -E\" \"$CC -E -traditional-cpp\" \"/lib/cpp\"\n    do\n      ac_preproc_ok=false\nfor ac_c_preproc_warn_flag in '' yes\ndo\n  # Use a header file that comes with gcc, so configuring glibc\n  # with a fresh cross-compiler works.\n  # Prefer <limits.h> to <assert.h> if __STDC__ is defined, since\n  # <limits.h> exists even on freestanding compilers.\n  # On the NeXT, cc -E runs the code through the compiler's parser,\n  # not just through cpp. \"Syntax error\" is here to catch this case.\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#ifdef __STDC__\n# include <limits.h>\n#else\n# include <assert.h>\n#endif\n\t\t     Syntax error\n_ACEOF\nif ac_fn_c_try_cpp \"$LINENO\"; then :\n\nelse\n  # Broken: fails on valid input.\ncontinue\nfi\nrm -f conftest.err conftest.i conftest.$ac_ext\n\n  # OK, works on sane cases.  Now check whether nonexistent headers\n  # can be detected and how.\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <ac_nonexistent.h>\n_ACEOF\nif ac_fn_c_try_cpp \"$LINENO\"; then :\n  # Broken: success on invalid input.\ncontinue\nelse\n  # Passes both tests.\nac_preproc_ok=:\nbreak\nfi\nrm -f conftest.err conftest.i conftest.$ac_ext\n\ndone\n# Because of `break', _AC_PREPROC_IFELSE's cleaning code was skipped.\nrm -f conftest.i conftest.err conftest.$ac_ext\nif $ac_preproc_ok; then :\n  break\nfi\n\n    done\n    ac_cv_prog_CPP=$CPP\n\nfi\n  CPP=$ac_cv_prog_CPP\nelse\n  ac_cv_prog_CPP=$CPP\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $CPP\" >&5\n$as_echo \"$CPP\" >&6; }\nac_preproc_ok=false\nfor ac_c_preproc_warn_flag in '' yes\ndo\n  # Use a header file that comes with gcc, so configuring glibc\n  # with a fresh cross-compiler works.\n  # Prefer <limits.h> to <assert.h> if __STDC__ is defined, since\n  # <limits.h> exists even on freestanding compilers.\n  # On the NeXT, cc -E runs the code through the compiler's parser,\n  # not just through cpp. \"Syntax error\" is here to catch this case.\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#ifdef __STDC__\n# include <limits.h>\n#else\n# include <assert.h>\n#endif\n\t\t     Syntax error\n_ACEOF\nif ac_fn_c_try_cpp \"$LINENO\"; then :\n\nelse\n  # Broken: fails on valid input.\ncontinue\nfi\nrm -f conftest.err conftest.i conftest.$ac_ext\n\n  # OK, works on sane cases.  Now check whether nonexistent headers\n  # can be detected and how.\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <ac_nonexistent.h>\n_ACEOF\nif ac_fn_c_try_cpp \"$LINENO\"; then :\n  # Broken: success on invalid input.\ncontinue\nelse\n  # Passes both tests.\nac_preproc_ok=:\nbreak\nfi\nrm -f conftest.err conftest.i conftest.$ac_ext\n\ndone\n# Because of `break', _AC_PREPROC_IFELSE's cleaning code was skipped.\nrm -f conftest.i conftest.err conftest.$ac_ext\nif $ac_preproc_ok; then :\n\nelse\n  { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"C preprocessor \\\"$CPP\\\" fails sanity check\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\n\nac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for grep that handles long lines and -e\" >&5\n$as_echo_n \"checking for grep that handles long lines and -e... \" >&6; }\nif ${ac_cv_path_GREP+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if test -z \"$GREP\"; then\n  ac_path_GREP_found=false\n  # Loop through the user's path and test for each of PROGNAME-LIST\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH$PATH_SEPARATOR/usr/xpg4/bin\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_prog in grep ggrep; do\n    for ac_exec_ext in '' $ac_executable_extensions; do\n      ac_path_GREP=\"$as_dir/$ac_prog$ac_exec_ext\"\n      as_fn_executable_p \"$ac_path_GREP\" || continue\n# Check for GNU ac_path_GREP and select it if it is found.\n  # Check for GNU $ac_path_GREP\ncase `\"$ac_path_GREP\" --version 2>&1` in\n*GNU*)\n  ac_cv_path_GREP=\"$ac_path_GREP\" ac_path_GREP_found=:;;\n*)\n  ac_count=0\n  $as_echo_n 0123456789 >\"conftest.in\"\n  while :\n  do\n    cat \"conftest.in\" \"conftest.in\" >\"conftest.tmp\"\n    mv \"conftest.tmp\" \"conftest.in\"\n    cp \"conftest.in\" \"conftest.nl\"\n    $as_echo 'GREP' >> \"conftest.nl\"\n    \"$ac_path_GREP\" -e 'GREP$' -e '-(cannot match)-' < \"conftest.nl\" >\"conftest.out\" 2>/dev/null || break\n    diff \"conftest.out\" \"conftest.nl\" >/dev/null 2>&1 || break\n    as_fn_arith $ac_count + 1 && ac_count=$as_val\n    if test $ac_count -gt ${ac_path_GREP_max-0}; then\n      # Best one so far, save it but keep looking for a better one\n      ac_cv_path_GREP=\"$ac_path_GREP\"\n      ac_path_GREP_max=$ac_count\n    fi\n    # 10*(2^10) chars as input seems more than enough\n    test $ac_count -gt 10 && break\n  done\n  rm -f conftest.in conftest.tmp conftest.nl conftest.out;;\nesac\n\n      $ac_path_GREP_found && break 3\n    done\n  done\n  done\nIFS=$as_save_IFS\n  if test -z \"$ac_cv_path_GREP\"; then\n    as_fn_error $? \"no acceptable grep could be found in $PATH$PATH_SEPARATOR/usr/xpg4/bin\" \"$LINENO\" 5\n  fi\nelse\n  ac_cv_path_GREP=$GREP\nfi\n\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_GREP\" >&5\n$as_echo \"$ac_cv_path_GREP\" >&6; }\n GREP=\"$ac_cv_path_GREP\"\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for egrep\" >&5\n$as_echo_n \"checking for egrep... \" >&6; }\nif ${ac_cv_path_EGREP+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  if echo a | $GREP -E '(a|b)' >/dev/null 2>&1\n   then ac_cv_path_EGREP=\"$GREP -E\"\n   else\n     if test -z \"$EGREP\"; then\n  ac_path_EGREP_found=false\n  # Loop through the user's path and test for each of PROGNAME-LIST\n  as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH$PATH_SEPARATOR/usr/xpg4/bin\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    for ac_prog in egrep; do\n    for ac_exec_ext in '' $ac_executable_extensions; do\n      ac_path_EGREP=\"$as_dir/$ac_prog$ac_exec_ext\"\n      as_fn_executable_p \"$ac_path_EGREP\" || continue\n# Check for GNU ac_path_EGREP and select it if it is found.\n  # Check for GNU $ac_path_EGREP\ncase `\"$ac_path_EGREP\" --version 2>&1` in\n*GNU*)\n  ac_cv_path_EGREP=\"$ac_path_EGREP\" ac_path_EGREP_found=:;;\n*)\n  ac_count=0\n  $as_echo_n 0123456789 >\"conftest.in\"\n  while :\n  do\n    cat \"conftest.in\" \"conftest.in\" >\"conftest.tmp\"\n    mv \"conftest.tmp\" \"conftest.in\"\n    cp \"conftest.in\" \"conftest.nl\"\n    $as_echo 'EGREP' >> \"conftest.nl\"\n    \"$ac_path_EGREP\" 'EGREP$' < \"conftest.nl\" >\"conftest.out\" 2>/dev/null || break\n    diff \"conftest.out\" \"conftest.nl\" >/dev/null 2>&1 || break\n    as_fn_arith $ac_count + 1 && ac_count=$as_val\n    if test $ac_count -gt ${ac_path_EGREP_max-0}; then\n      # Best one so far, save it but keep looking for a better one\n      ac_cv_path_EGREP=\"$ac_path_EGREP\"\n      ac_path_EGREP_max=$ac_count\n    fi\n    # 10*(2^10) chars as input seems more than enough\n    test $ac_count -gt 10 && break\n  done\n  rm -f conftest.in conftest.tmp conftest.nl conftest.out;;\nesac\n\n      $ac_path_EGREP_found && break 3\n    done\n  done\n  done\nIFS=$as_save_IFS\n  if test -z \"$ac_cv_path_EGREP\"; then\n    as_fn_error $? \"no acceptable egrep could be found in $PATH$PATH_SEPARATOR/usr/xpg4/bin\" \"$LINENO\" 5\n  fi\nelse\n  ac_cv_path_EGREP=$EGREP\nfi\n\n   fi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_EGREP\" >&5\n$as_echo \"$ac_cv_path_EGREP\" >&6; }\n EGREP=\"$ac_cv_path_EGREP\"\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking for ANSI C header files\" >&5\n$as_echo_n \"checking for ANSI C header files... \" >&6; }\nif ${ac_cv_header_stdc+:} false; then :\n  $as_echo_n \"(cached) \" >&6\nelse\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <stdlib.h>\n#include <stdarg.h>\n#include <string.h>\n#include <float.h>\n\nint\nmain ()\n{\n\n  ;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_compile \"$LINENO\"; then :\n  ac_cv_header_stdc=yes\nelse\n  ac_cv_header_stdc=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.$ac_ext\n\nif test $ac_cv_header_stdc = yes; then\n  # SunOS 4.x string.h does not declare mem*, contrary to ANSI.\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <string.h>\n\n_ACEOF\nif (eval \"$ac_cpp conftest.$ac_ext\") 2>&5 |\n  $EGREP \"memchr\" >/dev/null 2>&1; then :\n\nelse\n  ac_cv_header_stdc=no\nfi\nrm -f conftest*\n\nfi\n\nif test $ac_cv_header_stdc = yes; then\n  # ISC 2.0.2 stdlib.h does not declare free, contrary to ANSI.\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <stdlib.h>\n\n_ACEOF\nif (eval \"$ac_cpp conftest.$ac_ext\") 2>&5 |\n  $EGREP \"free\" >/dev/null 2>&1; then :\n\nelse\n  ac_cv_header_stdc=no\nfi\nrm -f conftest*\n\nfi\n\nif test $ac_cv_header_stdc = yes; then\n  # /bin/cc in Irix-4.0.5 gets non-ANSI ctype macros unless using -ansi.\n  if test \"$cross_compiling\" = yes; then :\n  :\nelse\n  cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h.  */\n#include <ctype.h>\n#include <stdlib.h>\n#if ((' ' & 0x0FF) == 0x020)\n# define ISLOWER(c) ('a' <= (c) && (c) <= 'z')\n# define TOUPPER(c) (ISLOWER(c) ? 'A' + ((c) - 'a') : (c))\n#else\n# define ISLOWER(c) \\\n\t\t   (('a' <= (c) && (c) <= 'i') \\\n\t\t     || ('j' <= (c) && (c) <= 'r') \\\n\t\t     || ('s' <= (c) && (c) <= 'z'))\n# define TOUPPER(c) (ISLOWER(c) ? ((c) | 0x40) : (c))\n#endif\n\n#define XOR(e, f) (((e) && !(f)) || (!(e) && (f)))\nint\nmain ()\n{\n  int i;\n  for (i = 0; i < 256; i++)\n    if (XOR (islower (i), ISLOWER (i))\n\t|| toupper (i) != TOUPPER (i))\n      return 2;\n  return 0;\n}\n_ACEOF\nif ac_fn_c_try_run \"$LINENO\"; then :\n\nelse\n  ac_cv_header_stdc=no\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n  conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\nfi\nfi\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_header_stdc\" >&5\n$as_echo \"$ac_cv_header_stdc\" >&6; }\nif test $ac_cv_header_stdc = yes; then\n\n$as_echo \"#define STDC_HEADERS 1\" >>confdefs.h\n\nfi\n\n# On IRIX 5.3, sys/types and inttypes.h are conflicting.\nfor ac_header in sys/types.h sys/stat.h stdlib.h string.h memory.h strings.h \\\n\t\t  inttypes.h stdint.h unistd.h\ndo :\n  as_ac_Header=`$as_echo \"ac_cv_header_$ac_header\" | $as_tr_sh`\nac_fn_c_check_header_compile \"$LINENO\" \"$ac_header\" \"$as_ac_Header\" \"$ac_includes_default\n\"\nif eval test \\\"x\\$\"$as_ac_Header\"\\\" = x\"yes\"; then :\n  cat >>confdefs.h <<_ACEOF\n#define `$as_echo \"HAVE_$ac_header\" | $as_tr_cpp` 1\n_ACEOF\n\nfi\n\ndone\n\n\nac_fn_c_check_header_mongrel \"$LINENO\" \"erl_nif.h\" \"ac_cv_header_erl_nif_h\" \"$ac_includes_default\"\nif test \"x$ac_cv_header_erl_nif_h\" = xyes; then :\n\nfi\n\n\nac_ext=erl\nac_compile='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5'\nac_link='$ERLC $ERLCFLAGS -b beam conftest.$ac_ext >&5 && echo \"#!/bin/sh\" > conftest$ac_exeext && $as_echo \"\\\"$ERL\\\" -run conftest start -run init stop -noshell\" >> conftest$ac_exeext && chmod +x conftest$ac_exeext'\n\nCFLAGS=$SAVECFLAGS\n\n#http://unix.stackexchange.com/questions/35183/how-do-i-identify-which-linux-distro-is-running\n\nLINUXVERSION=\"unknown linux\"\nif test -f /etc/SuSE-release ; then\n  LINUXVERSION=`cat /etc/SuSE-release`\nelif test -f /etc/redhat-release ; then\n  LINUXVERSION=`cat /etc/redhat-release`\nelif test -f /etc/fedora-release ; then\n  LINUXVERSION=`cat /etc/fedora-release`\nelif test -f /etc/debian_release ; then\n  LINUXVERSION=`cat /etc/debian_release`\nelif test -f /etc/debian_version ; then\n  LINUXVERSION=`cat /etc/debian_version`\nelif test -f /etc/gentoo-release ; then\n  LINUXVERSION=`cat /etc/gentoo-release`\nelif test -f /etc/arch-release ; then\n  LINUXVERSION=`uname -a`\nfi\n\nOS=\"\"\ncase `uname -s` in\n linux*)\n     NIFFLAGS=\"-shared\"\n     DRIVER_OS=LINUX\n     OS=$LINUXVERSION\n     ARCHIVECMD=\"$ac_cv_path_AR rv\"\n     ;;\n Linux*)\n     NIFFLAGS=\"-shared\"\n     DRIVER_OS=LINUX\n     OS=$LINUXVERSION\n     ARCHIVECMD=\"$ac_cv_path_AR rv\"\n     ;;\n Darwin*)\n     NIFFLAGS=\"-flat_namespace -undefined dynamic_lookup\"\n     DRIVER_OS=MACOSX\n     OS=\"OSX `sw_vers -productVersion`\"\n     ARCHIVECMD=\"$ac_cv_path_LIBTOOL -static -o\"\n     ;;\n darwin*)\n     NIFFLAGS=\"-flat_namespace -undefined dynamic_lookup\"\n     DRIVER_OS=MACOSX\n     OS=\"OSX `sw_vers -productVersion`\"\n     ARCHIVECMD=\"$ac_cv_path_LIBTOOL -static -o\"\n     ;;\nesac\n\nGIT=`command -v git`\nSTAT=`command -v stat`\nSHA1SUM=`command -v sha1sum`\nSRC=\"download\"\nCONFIGURESTATUS=\"\"\n\nif test \"x$GIT\" != \"x\" -a -d .git; then\n  SRC=`git log --pretty=format:'%h' -n 1 .`\n  CONFIGURESTATUS=`git status -s configure | cut -d ' ' -f 2`\nelif test \"x$GIT\" != \"x\"; then\n  CONFIGURESTATUS=`git hash-object configure`\nelif test \"x$DRIVER_OS\" = \"xMACOSX\" ; then\n  # BSD\n  SIZE=`/usr/bin/stat -f \"%z\" configure`\n  CONFIGURESTATUS=`(/usr/bin/printf \"blob $SIZE\\0\" ; /bin/cat configure) | /usr/bin/openssl sha1`\nelif test \"x$STAT\" != \"x\" -a \"x$SHA1SUM\" != \"x\" ; then\n  CONFIGURESTATUS=`(stat --printf=\"blob %s\\0\" configure; cat configure) | sha1sum -b | cut -d \" \" -f1`\nelse\n  CONFIGURESTATUS=\"\"\nfi\n\necho \"detected OS is: '$OS' $SRC $CONFIGURESTATUS\"\n\n\n\n\nmkdir -p ~/.yaws 2> /dev/null\n\nmkdir -p python3-api\n$SED -e \"s|python-api|python3-api|g\" \\\n     -e \"s|@PYTHONSITELIBDIR@|@PYTHON3SITELIBDIR@|g\" \\\n     -e \"s|PYTHON2|PYTHON3|g\" python-api/scalaris.in > python3-api/scalaris.in\n\nac_config_files=\"$ac_config_files Emakefile Makefile bin/scalarisctl bin/jsonclient include/rt.hrl cpp-api/Makefile java-api/scalaris java-api/scalaris-java.conf contrib/init.d/scalaris contrib/init.d/scalaris-monitor python-api/scalaris python3-api/scalaris ruby-api/scalaris\"\n\ncat >confcache <<\\_ACEOF\n# This file is a shell script that caches the results of configure\n# tests run on this system so they can be shared between configure\n# scripts and configure runs, see configure's option --config-cache.\n# It is not useful on other systems.  If it contains results you don't\n# want to keep, you may remove or edit it.\n#\n# config.status only pays attention to the cache file if you give it\n# the --recheck option to rerun configure.\n#\n# `ac_cv_env_foo' variables (set or unset) will be overridden when\n# loading this file, other *unset* `ac_cv_foo' will be assigned the\n# following values.\n\n_ACEOF\n\n# The following way of writing the cache mishandles newlines in values,\n# but we know of no workaround that is simple, portable, and efficient.\n# So, we kill variables containing newlines.\n# Ultrix sh set writes to stderr and can't be redirected directly,\n# and sets the high bit in the cache file unless we assign to the vars.\n(\n  for ac_var in `(set) 2>&1 | sed -n 's/^\\([a-zA-Z_][a-zA-Z0-9_]*\\)=.*/\\1/p'`; do\n    eval ac_val=\\$$ac_var\n    case $ac_val in #(\n    *${as_nl}*)\n      case $ac_var in #(\n      *_cv_*) { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: cache variable $ac_var contains a newline\" >&5\n$as_echo \"$as_me: WARNING: cache variable $ac_var contains a newline\" >&2;} ;;\n      esac\n      case $ac_var in #(\n      _ | IFS | as_nl) ;; #(\n      BASH_ARGV | BASH_SOURCE) eval $ac_var= ;; #(\n      *) { eval $ac_var=; unset $ac_var;} ;;\n      esac ;;\n    esac\n  done\n\n  (set) 2>&1 |\n    case $as_nl`(ac_space=' '; set) 2>&1` in #(\n    *${as_nl}ac_space=\\ *)\n      # `set' does not quote correctly, so add quotes: double-quote\n      # substitution turns \\\\\\\\ into \\\\, and sed turns \\\\ into \\.\n      sed -n \\\n\t\"s/'/'\\\\\\\\''/g;\n\t  s/^\\\\([_$as_cr_alnum]*_cv_[_$as_cr_alnum]*\\\\)=\\\\(.*\\\\)/\\\\1='\\\\2'/p\"\n      ;; #(\n    *)\n      # `set' quotes correctly as required by POSIX, so do not add quotes.\n      sed -n \"/^[_$as_cr_alnum]*_cv_[_$as_cr_alnum]*=/p\"\n      ;;\n    esac |\n    sort\n) |\n  sed '\n     /^ac_cv_env_/b end\n     t clear\n     :clear\n     s/^\\([^=]*\\)=\\(.*[{}].*\\)$/test \"${\\1+set}\" = set || &/\n     t end\n     s/^\\([^=]*\\)=\\(.*\\)$/\\1=${\\1=\\2}/\n     :end' >>confcache\nif diff \"$cache_file\" confcache >/dev/null 2>&1; then :; else\n  if test -w \"$cache_file\"; then\n    if test \"x$cache_file\" != \"x/dev/null\"; then\n      { $as_echo \"$as_me:${as_lineno-$LINENO}: updating cache $cache_file\" >&5\n$as_echo \"$as_me: updating cache $cache_file\" >&6;}\n      if test ! -f \"$cache_file\" || test -h \"$cache_file\"; then\n\tcat confcache >\"$cache_file\"\n      else\n        case $cache_file in #(\n        */* | ?:*)\n\t  mv -f confcache \"$cache_file\"$$ &&\n\t  mv -f \"$cache_file\"$$ \"$cache_file\" ;; #(\n        *)\n\t  mv -f confcache \"$cache_file\" ;;\n\tesac\n      fi\n    fi\n  else\n    { $as_echo \"$as_me:${as_lineno-$LINENO}: not updating unwritable cache $cache_file\" >&5\n$as_echo \"$as_me: not updating unwritable cache $cache_file\" >&6;}\n  fi\nfi\nrm -f confcache\n\ntest \"x$prefix\" = xNONE && prefix=$ac_default_prefix\n# Let make expand exec_prefix.\ntest \"x$exec_prefix\" = xNONE && exec_prefix='${prefix}'\n\n# Transform confdefs.h into DEFS.\n# Protect against shell expansion while executing Makefile rules.\n# Protect against Makefile macro expansion.\n#\n# If the first sed substitution is executed (which looks for macros that\n# take arguments), then branch to the quote section.  Otherwise,\n# look for a macro that doesn't take arguments.\nac_script='\n:mline\n/\\\\$/{\n N\n s,\\\\\\n,,\n b mline\n}\nt clear\n:clear\ns/^[\t ]*#[\t ]*define[\t ][\t ]*\\([^\t (][^\t (]*([^)]*)\\)[\t ]*\\(.*\\)/-D\\1=\\2/g\nt quote\ns/^[\t ]*#[\t ]*define[\t ][\t ]*\\([^\t ][^\t ]*\\)[\t ]*\\(.*\\)/-D\\1=\\2/g\nt quote\nb any\n:quote\ns/[\t `~#$^&*(){}\\\\|;'\\''\"<>?]/\\\\&/g\ns/\\[/\\\\&/g\ns/\\]/\\\\&/g\ns/\\$/$$/g\nH\n:any\n${\n\tg\n\ts/^\\n//\n\ts/\\n/ /g\n\tp\n}\n'\nDEFS=`sed -n \"$ac_script\" confdefs.h`\n\n\nac_libobjs=\nac_ltlibobjs=\nU=\nfor ac_i in : $LIBOBJS; do test \"x$ac_i\" = x: && continue\n  # 1. Remove the extension, and $U if already installed.\n  ac_script='s/\\$U\\././;s/\\.o$//;s/\\.obj$//'\n  ac_i=`$as_echo \"$ac_i\" | sed \"$ac_script\"`\n  # 2. Prepend LIBOBJDIR.  When used with automake>=1.10 LIBOBJDIR\n  #    will be set to the directory where LIBOBJS objects are built.\n  as_fn_append ac_libobjs \" \\${LIBOBJDIR}$ac_i\\$U.$ac_objext\"\n  as_fn_append ac_ltlibobjs \" \\${LIBOBJDIR}$ac_i\"'$U.lo'\ndone\nLIBOBJS=$ac_libobjs\n\nLTLIBOBJS=$ac_ltlibobjs\n\n\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: checking that generated files are newer than configure\" >&5\n$as_echo_n \"checking that generated files are newer than configure... \" >&6; }\n   if test -n \"$am_sleep_pid\"; then\n     # Hide warnings about reused PIDs.\n     wait $am_sleep_pid 2>/dev/null\n   fi\n   { $as_echo \"$as_me:${as_lineno-$LINENO}: result: done\" >&5\n$as_echo \"done\" >&6; }\n if test -n \"$EXEEXT\"; then\n  am__EXEEXT_TRUE=\n  am__EXEEXT_FALSE='#'\nelse\n  am__EXEEXT_TRUE='#'\n  am__EXEEXT_FALSE=\nfi\n\nif test -z \"${AMDEP_TRUE}\" && test -z \"${AMDEP_FALSE}\"; then\n  as_fn_error $? \"conditional \\\"AMDEP\\\" was never defined.\nUsually this means the macro was only invoked conditionally.\" \"$LINENO\" 5\nfi\nif test -z \"${am__fastdepCXX_TRUE}\" && test -z \"${am__fastdepCXX_FALSE}\"; then\n  as_fn_error $? \"conditional \\\"am__fastdepCXX\\\" was never defined.\nUsually this means the macro was only invoked conditionally.\" \"$LINENO\" 5\nfi\nif test -z \"${am__fastdepCC_TRUE}\" && test -z \"${am__fastdepCC_FALSE}\"; then\n  as_fn_error $? \"conditional \\\"am__fastdepCC\\\" was never defined.\nUsually this means the macro was only invoked conditionally.\" \"$LINENO\" 5\nfi\n\n: \"${CONFIG_STATUS=./config.status}\"\nac_write_fail=0\nac_clean_files_save=$ac_clean_files\nac_clean_files=\"$ac_clean_files $CONFIG_STATUS\"\n{ $as_echo \"$as_me:${as_lineno-$LINENO}: creating $CONFIG_STATUS\" >&5\n$as_echo \"$as_me: creating $CONFIG_STATUS\" >&6;}\nas_write_fail=0\ncat >$CONFIG_STATUS <<_ASEOF || as_write_fail=1\n#! $SHELL\n# Generated by $as_me.\n# Run this file to recreate the current configuration.\n# Compiler output produced by configure, useful for debugging\n# configure, is in config.log if it exists.\n\ndebug=false\nac_cs_recheck=false\nac_cs_silent=false\n\nSHELL=\\${CONFIG_SHELL-$SHELL}\nexport SHELL\n_ASEOF\ncat >>$CONFIG_STATUS <<\\_ASEOF || as_write_fail=1\n## -------------------- ##\n## M4sh Initialization. ##\n## -------------------- ##\n\n# Be more Bourne compatible\nDUALCASE=1; export DUALCASE # for MKS sh\nif test -n \"${ZSH_VERSION+set}\" && (emulate sh) >/dev/null 2>&1; then :\n  emulate sh\n  NULLCMD=:\n  # Pre-4.2 versions of Zsh do word splitting on ${1+\"$@\"}, which\n  # is contrary to our usage.  Disable this feature.\n  alias -g '${1+\"$@\"}'='\"$@\"'\n  setopt NO_GLOB_SUBST\nelse\n  case `(set -o) 2>/dev/null` in #(\n  *posix*) :\n    set -o posix ;; #(\n  *) :\n     ;;\nesac\nfi\n\n\nas_nl='\n'\nexport as_nl\n# Printing a long string crashes Solaris 7 /usr/bin/printf.\nas_echo='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'\nas_echo=$as_echo$as_echo$as_echo$as_echo$as_echo\nas_echo=$as_echo$as_echo$as_echo$as_echo$as_echo$as_echo\n# Prefer a ksh shell builtin over an external printf program on Solaris,\n# but without wasting forks for bash or zsh.\nif test -z \"$BASH_VERSION$ZSH_VERSION\" \\\n    && (test \"X`print -r -- $as_echo`\" = \"X$as_echo\") 2>/dev/null; then\n  as_echo='print -r --'\n  as_echo_n='print -rn --'\nelif (test \"X`printf %s $as_echo`\" = \"X$as_echo\") 2>/dev/null; then\n  as_echo='printf %s\\n'\n  as_echo_n='printf %s'\nelse\n  if test \"X`(/usr/ucb/echo -n -n $as_echo) 2>/dev/null`\" = \"X-n $as_echo\"; then\n    as_echo_body='eval /usr/ucb/echo -n \"$1$as_nl\"'\n    as_echo_n='/usr/ucb/echo -n'\n  else\n    as_echo_body='eval expr \"X$1\" : \"X\\\\(.*\\\\)\"'\n    as_echo_n_body='eval\n      arg=$1;\n      case $arg in #(\n      *\"$as_nl\"*)\n\texpr \"X$arg\" : \"X\\\\(.*\\\\)$as_nl\";\n\targ=`expr \"X$arg\" : \".*$as_nl\\\\(.*\\\\)\"`;;\n      esac;\n      expr \"X$arg\" : \"X\\\\(.*\\\\)\" | tr -d \"$as_nl\"\n    '\n    export as_echo_n_body\n    as_echo_n='sh -c $as_echo_n_body as_echo'\n  fi\n  export as_echo_body\n  as_echo='sh -c $as_echo_body as_echo'\nfi\n\n# The user is always right.\nif test \"${PATH_SEPARATOR+set}\" != set; then\n  PATH_SEPARATOR=:\n  (PATH='/bin;/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 && {\n    (PATH='/bin:/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 ||\n      PATH_SEPARATOR=';'\n  }\nfi\n\n\n# IFS\n# We need space, tab and new line, in precisely that order.  Quoting is\n# there to prevent editors from complaining about space-tab.\n# (If _AS_PATH_WALK were called with IFS unset, it would disable word\n# splitting by setting IFS to empty value.)\nIFS=\" \"\"\t$as_nl\"\n\n# Find who we are.  Look in the path if we contain no directory separator.\nas_myself=\ncase $0 in #((\n  *[\\\\/]* ) as_myself=$0 ;;\n  *) as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n  IFS=$as_save_IFS\n  test -z \"$as_dir\" && as_dir=.\n    test -r \"$as_dir/$0\" && as_myself=$as_dir/$0 && break\n  done\nIFS=$as_save_IFS\n\n     ;;\nesac\n# We did not find ourselves, most probably we were run as `sh COMMAND'\n# in which case we are not to be found in the path.\nif test \"x$as_myself\" = x; then\n  as_myself=$0\nfi\nif test ! -f \"$as_myself\"; then\n  $as_echo \"$as_myself: error: cannot find myself; rerun with an absolute file name\" >&2\n  exit 1\nfi\n\n# Unset variables that we do not need and which cause bugs (e.g. in\n# pre-3.0 UWIN ksh).  But do not cause bugs in bash 2.01; the \"|| exit 1\"\n# suppresses any \"Segmentation fault\" message there.  '((' could\n# trigger a bug in pdksh 5.2.14.\nfor as_var in BASH_ENV ENV MAIL MAILPATH\ndo eval test x\\${$as_var+set} = xset \\\n  && ( (unset $as_var) || exit 1) >/dev/null 2>&1 && unset $as_var || :\ndone\nPS1='$ '\nPS2='> '\nPS4='+ '\n\n# NLS nuisances.\nLC_ALL=C\nexport LC_ALL\nLANGUAGE=C\nexport LANGUAGE\n\n# CDPATH.\n(unset CDPATH) >/dev/null 2>&1 && unset CDPATH\n\n\n# as_fn_error STATUS ERROR [LINENO LOG_FD]\n# ----------------------------------------\n# Output \"`basename $0`: error: ERROR\" to stderr. If LINENO and LOG_FD are\n# provided, also output the error to LOG_FD, referencing LINENO. Then exit the\n# script with STATUS, using 1 if that was 0.\nas_fn_error ()\n{\n  as_status=$1; test $as_status -eq 0 && as_status=1\n  if test \"$4\"; then\n    as_lineno=${as_lineno-\"$3\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n    $as_echo \"$as_me:${as_lineno-$LINENO}: error: $2\" >&$4\n  fi\n  $as_echo \"$as_me: error: $2\" >&2\n  as_fn_exit $as_status\n} # as_fn_error\n\n\n# as_fn_set_status STATUS\n# -----------------------\n# Set $? to STATUS, without forking.\nas_fn_set_status ()\n{\n  return $1\n} # as_fn_set_status\n\n# as_fn_exit STATUS\n# -----------------\n# Exit the shell with STATUS, even in a \"trap 0\" or \"set -e\" context.\nas_fn_exit ()\n{\n  set +e\n  as_fn_set_status $1\n  exit $1\n} # as_fn_exit\n\n# as_fn_unset VAR\n# ---------------\n# Portably unset VAR.\nas_fn_unset ()\n{\n  { eval $1=; unset $1;}\n}\nas_unset=as_fn_unset\n# as_fn_append VAR VALUE\n# ----------------------\n# Append the text in VALUE to the end of the definition contained in VAR. Take\n# advantage of any shell optimizations that allow amortized linear growth over\n# repeated appends, instead of the typical quadratic growth present in naive\n# implementations.\nif (eval \"as_var=1; as_var+=2; test x\\$as_var = x12\") 2>/dev/null; then :\n  eval 'as_fn_append ()\n  {\n    eval $1+=\\$2\n  }'\nelse\n  as_fn_append ()\n  {\n    eval $1=\\$$1\\$2\n  }\nfi # as_fn_append\n\n# as_fn_arith ARG...\n# ------------------\n# Perform arithmetic evaluation on the ARGs, and store the result in the\n# global $as_val. Take advantage of shells that can avoid forks. The arguments\n# must be portable across $(()) and expr.\nif (eval \"test \\$(( 1 + 1 )) = 2\") 2>/dev/null; then :\n  eval 'as_fn_arith ()\n  {\n    as_val=$(( $* ))\n  }'\nelse\n  as_fn_arith ()\n  {\n    as_val=`expr \"$@\" || test $? -eq 1`\n  }\nfi # as_fn_arith\n\n\nif expr a : '\\(a\\)' >/dev/null 2>&1 &&\n   test \"X`expr 00001 : '.*\\(...\\)'`\" = X001; then\n  as_expr=expr\nelse\n  as_expr=false\nfi\n\nif (basename -- /) >/dev/null 2>&1 && test \"X`basename -- / 2>&1`\" = \"X/\"; then\n  as_basename=basename\nelse\n  as_basename=false\nfi\n\nif (as_dir=`dirname -- /` && test \"X$as_dir\" = X/) >/dev/null 2>&1; then\n  as_dirname=dirname\nelse\n  as_dirname=false\nfi\n\nas_me=`$as_basename -- \"$0\" ||\n$as_expr X/\"$0\" : '.*/\\([^/][^/]*\\)/*$' \\| \\\n\t X\"$0\" : 'X\\(//\\)$' \\| \\\n\t X\"$0\" : 'X\\(/\\)' \\| . 2>/dev/null ||\n$as_echo X/\"$0\" |\n    sed '/^.*\\/\\([^/][^/]*\\)\\/*$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\/\\(\\/\\/\\)$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\/\\(\\/\\).*/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  s/.*/./; q'`\n\n# Avoid depending upon Character Ranges.\nas_cr_letters='abcdefghijklmnopqrstuvwxyz'\nas_cr_LETTERS='ABCDEFGHIJKLMNOPQRSTUVWXYZ'\nas_cr_Letters=$as_cr_letters$as_cr_LETTERS\nas_cr_digits='0123456789'\nas_cr_alnum=$as_cr_Letters$as_cr_digits\n\nECHO_C= ECHO_N= ECHO_T=\ncase `echo -n x` in #(((((\n-n*)\n  case `echo 'xy\\c'` in\n  *c*) ECHO_T='\t';;\t# ECHO_T is single tab character.\n  xy)  ECHO_C='\\c';;\n  *)   echo `echo ksh88 bug on AIX 6.1` > /dev/null\n       ECHO_T='\t';;\n  esac;;\n*)\n  ECHO_N='-n';;\nesac\n\nrm -f conf$$ conf$$.exe conf$$.file\nif test -d conf$$.dir; then\n  rm -f conf$$.dir/conf$$.file\nelse\n  rm -f conf$$.dir\n  mkdir conf$$.dir 2>/dev/null\nfi\nif (echo >conf$$.file) 2>/dev/null; then\n  if ln -s conf$$.file conf$$ 2>/dev/null; then\n    as_ln_s='ln -s'\n    # ... but there are two gotchas:\n    # 1) On MSYS, both `ln -s file dir' and `ln file dir' fail.\n    # 2) DJGPP < 2.04 has no symlinks; `ln -s' creates a wrapper executable.\n    # In both cases, we have to default to `cp -pR'.\n    ln -s conf$$.file conf$$.dir 2>/dev/null && test ! -f conf$$.exe ||\n      as_ln_s='cp -pR'\n  elif ln conf$$.file conf$$ 2>/dev/null; then\n    as_ln_s=ln\n  else\n    as_ln_s='cp -pR'\n  fi\nelse\n  as_ln_s='cp -pR'\nfi\nrm -f conf$$ conf$$.exe conf$$.dir/conf$$.file conf$$.file\nrmdir conf$$.dir 2>/dev/null\n\n\n# as_fn_mkdir_p\n# -------------\n# Create \"$as_dir\" as a directory, including parents if necessary.\nas_fn_mkdir_p ()\n{\n\n  case $as_dir in #(\n  -*) as_dir=./$as_dir;;\n  esac\n  test -d \"$as_dir\" || eval $as_mkdir_p || {\n    as_dirs=\n    while :; do\n      case $as_dir in #(\n      *\\'*) as_qdir=`$as_echo \"$as_dir\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"`;; #'(\n      *) as_qdir=$as_dir;;\n      esac\n      as_dirs=\"'$as_qdir' $as_dirs\"\n      as_dir=`$as_dirname -- \"$as_dir\" ||\n$as_expr X\"$as_dir\" : 'X\\(.*[^/]\\)//*[^/][^/]*/*$' \\| \\\n\t X\"$as_dir\" : 'X\\(//\\)[^/]' \\| \\\n\t X\"$as_dir\" : 'X\\(//\\)$' \\| \\\n\t X\"$as_dir\" : 'X\\(/\\)' \\| . 2>/dev/null ||\n$as_echo X\"$as_dir\" |\n    sed '/^X\\(.*[^/]\\)\\/\\/*[^/][^/]*\\/*$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\/\\)[^/].*/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\/\\)$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\).*/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  s/.*/./; q'`\n      test -d \"$as_dir\" && break\n    done\n    test -z \"$as_dirs\" || eval \"mkdir $as_dirs\"\n  } || test -d \"$as_dir\" || as_fn_error $? \"cannot create directory $as_dir\"\n\n\n} # as_fn_mkdir_p\nif mkdir -p . 2>/dev/null; then\n  as_mkdir_p='mkdir -p \"$as_dir\"'\nelse\n  test -d ./-p && rmdir ./-p\n  as_mkdir_p=false\nfi\n\n\n# as_fn_executable_p FILE\n# -----------------------\n# Test if FILE is an executable regular file.\nas_fn_executable_p ()\n{\n  test -f \"$1\" && test -x \"$1\"\n} # as_fn_executable_p\nas_test_x='test -x'\nas_executable_p=as_fn_executable_p\n\n# Sed expression to map a string onto a valid CPP name.\nas_tr_cpp=\"eval sed 'y%*$as_cr_letters%P$as_cr_LETTERS%;s%[^_$as_cr_alnum]%_%g'\"\n\n# Sed expression to map a string onto a valid variable name.\nas_tr_sh=\"eval sed 'y%*+%pp%;s%[^_$as_cr_alnum]%_%g'\"\n\n\nexec 6>&1\n## ----------------------------------- ##\n## Main body of $CONFIG_STATUS script. ##\n## ----------------------------------- ##\n_ASEOF\ntest $as_write_fail = 0 && chmod +x $CONFIG_STATUS || ac_write_fail=1\n\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\n# Save the log message, to keep $0 and so on meaningful, and to\n# report actual input values of CONFIG_FILES etc. instead of their\n# values after options handling.\nac_log=\"\nThis file was extended by scalaris $as_me 0.9.0+git, which was\ngenerated by GNU Autoconf 2.69.  Invocation command line was\n\n  CONFIG_FILES    = $CONFIG_FILES\n  CONFIG_HEADERS  = $CONFIG_HEADERS\n  CONFIG_LINKS    = $CONFIG_LINKS\n  CONFIG_COMMANDS = $CONFIG_COMMANDS\n  $ $0 $@\n\non `(hostname || uname -n) 2>/dev/null | sed 1q`\n\"\n\n_ACEOF\n\ncase $ac_config_files in *\"\n\"*) set x $ac_config_files; shift; ac_config_files=$*;;\nesac\n\n\n\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\n# Files that config.status was made for.\nconfig_files=\"$ac_config_files\"\nconfig_commands=\"$ac_config_commands\"\n\n_ACEOF\n\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\nac_cs_usage=\"\\\n\\`$as_me' instantiates files and other configuration actions\nfrom templates according to the current configuration.  Unless the files\nand actions are specified as TAGs, all are instantiated by default.\n\nUsage: $0 [OPTION]... [TAG]...\n\n  -h, --help       print this help, then exit\n  -V, --version    print version number and configuration settings, then exit\n      --config     print configuration, then exit\n  -q, --quiet, --silent\n                   do not print progress messages\n  -d, --debug      don't remove temporary files\n      --recheck    update $as_me by reconfiguring in the same conditions\n      --file=FILE[:TEMPLATE]\n                   instantiate the configuration file FILE\n\nConfiguration files:\n$config_files\n\nConfiguration commands:\n$config_commands\n\nReport bugs to <scalaris@googlegroups.com>.\"\n\n_ACEOF\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\nac_cs_config=\"`$as_echo \"$ac_configure_args\" | sed 's/^ //; s/[\\\\\"\"\\`\\$]/\\\\\\\\&/g'`\"\nac_cs_version=\"\\\\\nscalaris config.status 0.9.0+git\nconfigured by $0, generated by GNU Autoconf 2.69,\n  with options \\\\\"\\$ac_cs_config\\\\\"\n\nCopyright (C) 2012 Free Software Foundation, Inc.\nThis config.status script is free software; the Free Software Foundation\ngives unlimited permission to copy, distribute and modify it.\"\n\nac_pwd='$ac_pwd'\nsrcdir='$srcdir'\nINSTALL='$INSTALL'\nMKDIR_P='$MKDIR_P'\nAWK='$AWK'\ntest -n \"\\$AWK\" || AWK=awk\n_ACEOF\n\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\n# The default lists apply if the user does not specify any file.\nac_need_defaults=:\nwhile test $# != 0\ndo\n  case $1 in\n  --*=?*)\n    ac_option=`expr \"X$1\" : 'X\\([^=]*\\)='`\n    ac_optarg=`expr \"X$1\" : 'X[^=]*=\\(.*\\)'`\n    ac_shift=:\n    ;;\n  --*=)\n    ac_option=`expr \"X$1\" : 'X\\([^=]*\\)='`\n    ac_optarg=\n    ac_shift=:\n    ;;\n  *)\n    ac_option=$1\n    ac_optarg=$2\n    ac_shift=shift\n    ;;\n  esac\n\n  case $ac_option in\n  # Handling of the options.\n  -recheck | --recheck | --rechec | --reche | --rech | --rec | --re | --r)\n    ac_cs_recheck=: ;;\n  --version | --versio | --versi | --vers | --ver | --ve | --v | -V )\n    $as_echo \"$ac_cs_version\"; exit ;;\n  --config | --confi | --conf | --con | --co | --c )\n    $as_echo \"$ac_cs_config\"; exit ;;\n  --debug | --debu | --deb | --de | --d | -d )\n    debug=: ;;\n  --file | --fil | --fi | --f )\n    $ac_shift\n    case $ac_optarg in\n    *\\'*) ac_optarg=`$as_echo \"$ac_optarg\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"` ;;\n    '') as_fn_error $? \"missing file argument\" ;;\n    esac\n    as_fn_append CONFIG_FILES \" '$ac_optarg'\"\n    ac_need_defaults=false;;\n  --he | --h |  --help | --hel | -h )\n    $as_echo \"$ac_cs_usage\"; exit ;;\n  -q | -quiet | --quiet | --quie | --qui | --qu | --q \\\n  | -silent | --silent | --silen | --sile | --sil | --si | --s)\n    ac_cs_silent=: ;;\n\n  # This is an error.\n  -*) as_fn_error $? \"unrecognized option: \\`$1'\nTry \\`$0 --help' for more information.\" ;;\n\n  *) as_fn_append ac_config_targets \" $1\"\n     ac_need_defaults=false ;;\n\n  esac\n  shift\ndone\n\nac_configure_extra_args=\n\nif $ac_cs_silent; then\n  exec 6>/dev/null\n  ac_configure_extra_args=\"$ac_configure_extra_args --silent\"\nfi\n\n_ACEOF\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\nif \\$ac_cs_recheck; then\n  set X $SHELL '$0' $ac_configure_args \\$ac_configure_extra_args --no-create --no-recursion\n  shift\n  \\$as_echo \"running CONFIG_SHELL=$SHELL \\$*\" >&6\n  CONFIG_SHELL='$SHELL'\n  export CONFIG_SHELL\n  exec \"\\$@\"\nfi\n\n_ACEOF\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\nexec 5>>config.log\n{\n  echo\n  sed 'h;s/./-/g;s/^.../## /;s/...$/ ##/;p;x;p;x' <<_ASBOX\n## Running $as_me. ##\n_ASBOX\n  $as_echo \"$ac_log\"\n} >&5\n\n_ACEOF\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\n#\n# INIT-COMMANDS\n#\nAMDEP_TRUE=\"$AMDEP_TRUE\" MAKE=\"${MAKE-make}\"\n\n_ACEOF\n\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\n\n# Handling of arguments.\nfor ac_config_target in $ac_config_targets\ndo\n  case $ac_config_target in\n    \"depfiles\") CONFIG_COMMANDS=\"$CONFIG_COMMANDS depfiles\" ;;\n    \"Emakefile\") CONFIG_FILES=\"$CONFIG_FILES Emakefile\" ;;\n    \"Makefile\") CONFIG_FILES=\"$CONFIG_FILES Makefile\" ;;\n    \"bin/scalarisctl\") CONFIG_FILES=\"$CONFIG_FILES bin/scalarisctl\" ;;\n    \"bin/jsonclient\") CONFIG_FILES=\"$CONFIG_FILES bin/jsonclient\" ;;\n    \"include/rt.hrl\") CONFIG_FILES=\"$CONFIG_FILES include/rt.hrl\" ;;\n    \"cpp-api/Makefile\") CONFIG_FILES=\"$CONFIG_FILES cpp-api/Makefile\" ;;\n    \"java-api/scalaris\") CONFIG_FILES=\"$CONFIG_FILES java-api/scalaris\" ;;\n    \"java-api/scalaris-java.conf\") CONFIG_FILES=\"$CONFIG_FILES java-api/scalaris-java.conf\" ;;\n    \"contrib/init.d/scalaris\") CONFIG_FILES=\"$CONFIG_FILES contrib/init.d/scalaris\" ;;\n    \"contrib/init.d/scalaris-monitor\") CONFIG_FILES=\"$CONFIG_FILES contrib/init.d/scalaris-monitor\" ;;\n    \"python-api/scalaris\") CONFIG_FILES=\"$CONFIG_FILES python-api/scalaris\" ;;\n    \"python3-api/scalaris\") CONFIG_FILES=\"$CONFIG_FILES python3-api/scalaris\" ;;\n    \"ruby-api/scalaris\") CONFIG_FILES=\"$CONFIG_FILES ruby-api/scalaris\" ;;\n\n  *) as_fn_error $? \"invalid argument: \\`$ac_config_target'\" \"$LINENO\" 5;;\n  esac\ndone\n\n\n# If the user did not use the arguments to specify the items to instantiate,\n# then the envvar interface is used.  Set only those that are not.\n# We use the long form for the default assignment because of an extremely\n# bizarre bug on SunOS 4.1.3.\nif $ac_need_defaults; then\n  test \"${CONFIG_FILES+set}\" = set || CONFIG_FILES=$config_files\n  test \"${CONFIG_COMMANDS+set}\" = set || CONFIG_COMMANDS=$config_commands\nfi\n\n# Have a temporary directory for convenience.  Make it in the build tree\n# simply because there is no reason against having it here, and in addition,\n# creating and moving files from /tmp can sometimes cause problems.\n# Hook for its removal unless debugging.\n# Note that there is a small window in which the directory will not be cleaned:\n# after its creation but before its name has been assigned to `$tmp'.\n$debug ||\n{\n  tmp= ac_tmp=\n  trap 'exit_status=$?\n  : \"${ac_tmp:=$tmp}\"\n  { test ! -d \"$ac_tmp\" || rm -fr \"$ac_tmp\"; } && exit $exit_status\n' 0\n  trap 'as_fn_exit 1' 1 2 13 15\n}\n# Create a (secure) tmp directory for tmp files.\n\n{\n  tmp=`(umask 077 && mktemp -d \"./confXXXXXX\") 2>/dev/null` &&\n  test -d \"$tmp\"\n}  ||\n{\n  tmp=./conf$$-$RANDOM\n  (umask 077 && mkdir \"$tmp\")\n} || as_fn_error $? \"cannot create a temporary directory in .\" \"$LINENO\" 5\nac_tmp=$tmp\n\n# Set up the scripts for CONFIG_FILES section.\n# No need to generate them if there are no CONFIG_FILES.\n# This happens for instance with `./config.status config.h'.\nif test -n \"$CONFIG_FILES\"; then\n\n\nac_cr=`echo X | tr X '\\015'`\n# On cygwin, bash can eat \\r inside `` if the user requested igncr.\n# But we know of no other shell where ac_cr would be empty at this\n# point, so we can use a bashism as a fallback.\nif test \"x$ac_cr\" = x; then\n  eval ac_cr=\\$\\'\\\\r\\'\nfi\nac_cs_awk_cr=`$AWK 'BEGIN { print \"a\\rb\" }' </dev/null 2>/dev/null`\nif test \"$ac_cs_awk_cr\" = \"a${ac_cr}b\"; then\n  ac_cs_awk_cr='\\\\r'\nelse\n  ac_cs_awk_cr=$ac_cr\nfi\n\necho 'BEGIN {' >\"$ac_tmp/subs1.awk\" &&\n_ACEOF\n\n\n{\n  echo \"cat >conf$$subs.awk <<_ACEOF\" &&\n  echo \"$ac_subst_vars\" | sed 's/.*/&!$&$ac_delim/' &&\n  echo \"_ACEOF\"\n} >conf$$subs.sh ||\n  as_fn_error $? \"could not make $CONFIG_STATUS\" \"$LINENO\" 5\nac_delim_num=`echo \"$ac_subst_vars\" | grep -c '^'`\nac_delim='%!_!# '\nfor ac_last_try in false false false false false :; do\n  . ./conf$$subs.sh ||\n    as_fn_error $? \"could not make $CONFIG_STATUS\" \"$LINENO\" 5\n\n  ac_delim_n=`sed -n \"s/.*$ac_delim\\$/X/p\" conf$$subs.awk | grep -c X`\n  if test $ac_delim_n = $ac_delim_num; then\n    break\n  elif $ac_last_try; then\n    as_fn_error $? \"could not make $CONFIG_STATUS\" \"$LINENO\" 5\n  else\n    ac_delim=\"$ac_delim!$ac_delim _$ac_delim!! \"\n  fi\ndone\nrm -f conf$$subs.sh\n\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\ncat >>\"\\$ac_tmp/subs1.awk\" <<\\\\_ACAWK &&\n_ACEOF\nsed -n '\nh\ns/^/S[\"/; s/!.*/\"]=/\np\ng\ns/^[^!]*!//\n:repl\nt repl\ns/'\"$ac_delim\"'$//\nt delim\n:nl\nh\ns/\\(.\\{148\\}\\)..*/\\1/\nt more1\ns/[\"\\\\]/\\\\&/g; s/^/\"/; s/$/\\\\n\"\\\\/\np\nn\nb repl\n:more1\ns/[\"\\\\]/\\\\&/g; s/^/\"/; s/$/\"\\\\/\np\ng\ns/.\\{148\\}//\nt nl\n:delim\nh\ns/\\(.\\{148\\}\\)..*/\\1/\nt more2\ns/[\"\\\\]/\\\\&/g; s/^/\"/; s/$/\"/\np\nb\n:more2\ns/[\"\\\\]/\\\\&/g; s/^/\"/; s/$/\"\\\\/\np\ng\ns/.\\{148\\}//\nt delim\n' <conf$$subs.awk | sed '\n/^[^\"\"]/{\n  N\n  s/\\n//\n}\n' >>$CONFIG_STATUS || ac_write_fail=1\nrm -f conf$$subs.awk\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\n_ACAWK\ncat >>\"\\$ac_tmp/subs1.awk\" <<_ACAWK &&\n  for (key in S) S_is_set[key] = 1\n  FS = \"\u0007\"\n\n}\n{\n  line = $ 0\n  nfields = split(line, field, \"@\")\n  substed = 0\n  len = length(field[1])\n  for (i = 2; i < nfields; i++) {\n    key = field[i]\n    keylen = length(key)\n    if (S_is_set[key]) {\n      value = S[key]\n      line = substr(line, 1, len) \"\" value \"\" substr(line, len + keylen + 3)\n      len += length(value) + length(field[++i])\n      substed = 1\n    } else\n      len += 1 + keylen\n  }\n\n  print line\n}\n\n_ACAWK\n_ACEOF\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\nif sed \"s/$ac_cr//\" < /dev/null > /dev/null 2>&1; then\n  sed \"s/$ac_cr\\$//; s/$ac_cr/$ac_cs_awk_cr/g\"\nelse\n  cat\nfi < \"$ac_tmp/subs1.awk\" > \"$ac_tmp/subs.awk\" \\\n  || as_fn_error $? \"could not setup config files machinery\" \"$LINENO\" 5\n_ACEOF\n\n# VPATH may cause trouble with some makes, so we remove sole $(srcdir),\n# ${srcdir} and @srcdir@ entries from VPATH if srcdir is \".\", strip leading and\n# trailing colons and then remove the whole line if VPATH becomes empty\n# (actually we leave an empty line to preserve line numbers).\nif test \"x$srcdir\" = x.; then\n  ac_vpsub='/^[\t ]*VPATH[\t ]*=[\t ]*/{\nh\ns///\ns/^/:/\ns/[\t ]*$/:/\ns/:\\$(srcdir):/:/g\ns/:\\${srcdir}:/:/g\ns/:@srcdir@:/:/g\ns/^:*//\ns/:*$//\nx\ns/\\(=[\t ]*\\).*/\\1/\nG\ns/\\n//\ns/^[^=]*=[\t ]*$//\n}'\nfi\n\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\nfi # test -n \"$CONFIG_FILES\"\n\n\neval set X \"  :F $CONFIG_FILES      :C $CONFIG_COMMANDS\"\nshift\nfor ac_tag\ndo\n  case $ac_tag in\n  :[FHLC]) ac_mode=$ac_tag; continue;;\n  esac\n  case $ac_mode$ac_tag in\n  :[FHL]*:*);;\n  :L* | :C*:*) as_fn_error $? \"invalid tag \\`$ac_tag'\" \"$LINENO\" 5;;\n  :[FH]-) ac_tag=-:-;;\n  :[FH]*) ac_tag=$ac_tag:$ac_tag.in;;\n  esac\n  ac_save_IFS=$IFS\n  IFS=:\n  set x $ac_tag\n  IFS=$ac_save_IFS\n  shift\n  ac_file=$1\n  shift\n\n  case $ac_mode in\n  :L) ac_source=$1;;\n  :[FH])\n    ac_file_inputs=\n    for ac_f\n    do\n      case $ac_f in\n      -) ac_f=\"$ac_tmp/stdin\";;\n      *) # Look for the file first in the build tree, then in the source tree\n\t # (if the path is not absolute).  The absolute path cannot be DOS-style,\n\t # because $ac_f cannot contain `:'.\n\t test -f \"$ac_f\" ||\n\t   case $ac_f in\n\t   [\\\\/$]*) false;;\n\t   *) test -f \"$srcdir/$ac_f\" && ac_f=\"$srcdir/$ac_f\";;\n\t   esac ||\n\t   as_fn_error 1 \"cannot find input file: \\`$ac_f'\" \"$LINENO\" 5;;\n      esac\n      case $ac_f in *\\'*) ac_f=`$as_echo \"$ac_f\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"`;; esac\n      as_fn_append ac_file_inputs \" '$ac_f'\"\n    done\n\n    # Let's still pretend it is `configure' which instantiates (i.e., don't\n    # use $as_me), people would be surprised to read:\n    #    /* config.h.  Generated by config.status.  */\n    configure_input='Generated from '`\n\t  $as_echo \"$*\" | sed 's|^[^:]*/||;s|:[^:]*/|, |g'\n\t`' by configure.'\n    if test x\"$ac_file\" != x-; then\n      configure_input=\"$ac_file.  $configure_input\"\n      { $as_echo \"$as_me:${as_lineno-$LINENO}: creating $ac_file\" >&5\n$as_echo \"$as_me: creating $ac_file\" >&6;}\n    fi\n    # Neutralize special characters interpreted by sed in replacement strings.\n    case $configure_input in #(\n    *\\&* | *\\|* | *\\\\* )\n       ac_sed_conf_input=`$as_echo \"$configure_input\" |\n       sed 's/[\\\\\\\\&|]/\\\\\\\\&/g'`;; #(\n    *) ac_sed_conf_input=$configure_input;;\n    esac\n\n    case $ac_tag in\n    *:-:* | *:-) cat >\"$ac_tmp/stdin\" \\\n      || as_fn_error $? \"could not create $ac_file\" \"$LINENO\" 5 ;;\n    esac\n    ;;\n  esac\n\n  ac_dir=`$as_dirname -- \"$ac_file\" ||\n$as_expr X\"$ac_file\" : 'X\\(.*[^/]\\)//*[^/][^/]*/*$' \\| \\\n\t X\"$ac_file\" : 'X\\(//\\)[^/]' \\| \\\n\t X\"$ac_file\" : 'X\\(//\\)$' \\| \\\n\t X\"$ac_file\" : 'X\\(/\\)' \\| . 2>/dev/null ||\n$as_echo X\"$ac_file\" |\n    sed '/^X\\(.*[^/]\\)\\/\\/*[^/][^/]*\\/*$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\/\\)[^/].*/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\/\\)$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\).*/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  s/.*/./; q'`\n  as_dir=\"$ac_dir\"; as_fn_mkdir_p\n  ac_builddir=.\n\ncase \"$ac_dir\" in\n.) ac_dir_suffix= ac_top_builddir_sub=. ac_top_build_prefix= ;;\n*)\n  ac_dir_suffix=/`$as_echo \"$ac_dir\" | sed 's|^\\.[\\\\/]||'`\n  # A \"..\" for each directory in $ac_dir_suffix.\n  ac_top_builddir_sub=`$as_echo \"$ac_dir_suffix\" | sed 's|/[^\\\\/]*|/..|g;s|/||'`\n  case $ac_top_builddir_sub in\n  \"\") ac_top_builddir_sub=. ac_top_build_prefix= ;;\n  *)  ac_top_build_prefix=$ac_top_builddir_sub/ ;;\n  esac ;;\nesac\nac_abs_top_builddir=$ac_pwd\nac_abs_builddir=$ac_pwd$ac_dir_suffix\n# for backward compatibility:\nac_top_builddir=$ac_top_build_prefix\n\ncase $srcdir in\n  .)  # We are building in place.\n    ac_srcdir=.\n    ac_top_srcdir=$ac_top_builddir_sub\n    ac_abs_top_srcdir=$ac_pwd ;;\n  [\\\\/]* | ?:[\\\\/]* )  # Absolute name.\n    ac_srcdir=$srcdir$ac_dir_suffix;\n    ac_top_srcdir=$srcdir\n    ac_abs_top_srcdir=$srcdir ;;\n  *) # Relative name.\n    ac_srcdir=$ac_top_build_prefix$srcdir$ac_dir_suffix\n    ac_top_srcdir=$ac_top_build_prefix$srcdir\n    ac_abs_top_srcdir=$ac_pwd/$srcdir ;;\nesac\nac_abs_srcdir=$ac_abs_top_srcdir$ac_dir_suffix\n\n\n  case $ac_mode in\n  :F)\n  #\n  # CONFIG_FILE\n  #\n\n  case $INSTALL in\n  [\\\\/$]* | ?:[\\\\/]* ) ac_INSTALL=$INSTALL ;;\n  *) ac_INSTALL=$ac_top_build_prefix$INSTALL ;;\n  esac\n  ac_MKDIR_P=$MKDIR_P\n  case $MKDIR_P in\n  [\\\\/$]* | ?:[\\\\/]* ) ;;\n  */*) ac_MKDIR_P=$ac_top_build_prefix$MKDIR_P ;;\n  esac\n_ACEOF\n\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\n# If the template does not know about datarootdir, expand it.\n# FIXME: This hack should be removed a few years after 2.60.\nac_datarootdir_hack=; ac_datarootdir_seen=\nac_sed_dataroot='\n/datarootdir/ {\n  p\n  q\n}\n/@datadir@/p\n/@docdir@/p\n/@infodir@/p\n/@localedir@/p\n/@mandir@/p'\ncase `eval \"sed -n \\\"\\$ac_sed_dataroot\\\" $ac_file_inputs\"` in\n*datarootdir*) ac_datarootdir_seen=yes;;\n*@datadir@*|*@docdir@*|*@infodir@*|*@localedir@*|*@mandir@*)\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: $ac_file_inputs seems to ignore the --datarootdir setting\" >&5\n$as_echo \"$as_me: WARNING: $ac_file_inputs seems to ignore the --datarootdir setting\" >&2;}\n_ACEOF\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\n  ac_datarootdir_hack='\n  s&@datadir@&$datadir&g\n  s&@docdir@&$docdir&g\n  s&@infodir@&$infodir&g\n  s&@localedir@&$localedir&g\n  s&@mandir@&$mandir&g\n  s&\\\\\\${datarootdir}&$datarootdir&g' ;;\nesac\n_ACEOF\n\n# Neutralize VPATH when `$srcdir' = `.'.\n# Shell code in configure.ac might set extrasub.\n# FIXME: do we really want to maintain this feature?\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\nac_sed_extra=\"$ac_vpsub\n$extrasub\n_ACEOF\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\n:t\n/@[a-zA-Z_][a-zA-Z_0-9]*@/!b\ns|@configure_input@|$ac_sed_conf_input|;t t\ns&@top_builddir@&$ac_top_builddir_sub&;t t\ns&@top_build_prefix@&$ac_top_build_prefix&;t t\ns&@srcdir@&$ac_srcdir&;t t\ns&@abs_srcdir@&$ac_abs_srcdir&;t t\ns&@top_srcdir@&$ac_top_srcdir&;t t\ns&@abs_top_srcdir@&$ac_abs_top_srcdir&;t t\ns&@builddir@&$ac_builddir&;t t\ns&@abs_builddir@&$ac_abs_builddir&;t t\ns&@abs_top_builddir@&$ac_abs_top_builddir&;t t\ns&@INSTALL@&$ac_INSTALL&;t t\ns&@MKDIR_P@&$ac_MKDIR_P&;t t\n$ac_datarootdir_hack\n\"\neval sed \\\"\\$ac_sed_extra\\\" \"$ac_file_inputs\" | $AWK -f \"$ac_tmp/subs.awk\" \\\n  >$ac_tmp/out || as_fn_error $? \"could not create $ac_file\" \"$LINENO\" 5\n\ntest -z \"$ac_datarootdir_hack$ac_datarootdir_seen\" &&\n  { ac_out=`sed -n '/\\${datarootdir}/p' \"$ac_tmp/out\"`; test -n \"$ac_out\"; } &&\n  { ac_out=`sed -n '/^[\t ]*datarootdir[\t ]*:*=/p' \\\n      \"$ac_tmp/out\"`; test -z \"$ac_out\"; } &&\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: $ac_file contains a reference to the variable \\`datarootdir'\nwhich seems to be undefined.  Please make sure it is defined\" >&5\n$as_echo \"$as_me: WARNING: $ac_file contains a reference to the variable \\`datarootdir'\nwhich seems to be undefined.  Please make sure it is defined\" >&2;}\n\n  rm -f \"$ac_tmp/stdin\"\n  case $ac_file in\n  -) cat \"$ac_tmp/out\" && rm -f \"$ac_tmp/out\";;\n  *) rm -f \"$ac_file\" && mv \"$ac_tmp/out\" \"$ac_file\";;\n  esac \\\n  || as_fn_error $? \"could not create $ac_file\" \"$LINENO\" 5\n ;;\n\n\n  :C)  { $as_echo \"$as_me:${as_lineno-$LINENO}: executing $ac_file commands\" >&5\n$as_echo \"$as_me: executing $ac_file commands\" >&6;}\n ;;\n  esac\n\n\n  case $ac_file$ac_mode in\n    \"depfiles\":C) test x\"$AMDEP_TRUE\" != x\"\" || {\n  # Older Autoconf quotes --file arguments for eval, but not when files\n  # are listed without --file.  Let's play safe and only enable the eval\n  # if we detect the quoting.\n  # TODO: see whether this extra hack can be removed once we start\n  # requiring Autoconf 2.70 or later.\n  case $CONFIG_FILES in #(\n  *\\'*) :\n    eval set x \"$CONFIG_FILES\" ;; #(\n  *) :\n    set x $CONFIG_FILES ;; #(\n  *) :\n     ;;\nesac\n  shift\n  # Used to flag and report bootstrapping failures.\n  am_rc=0\n  for am_mf\n  do\n    # Strip MF so we end up with the name of the file.\n    am_mf=`$as_echo \"$am_mf\" | sed -e 's/:.*$//'`\n    # Check whether this is an Automake generated Makefile which includes\n    # dependency-tracking related rules and includes.\n    # Grep'ing the whole file directly is not great: AIX grep has a line\n    # limit of 2048, but all sed's we know have understand at least 4000.\n    sed -n 's,^am--depfiles:.*,X,p' \"$am_mf\" | grep X >/dev/null 2>&1 \\\n      || continue\n    am_dirpart=`$as_dirname -- \"$am_mf\" ||\n$as_expr X\"$am_mf\" : 'X\\(.*[^/]\\)//*[^/][^/]*/*$' \\| \\\n\t X\"$am_mf\" : 'X\\(//\\)[^/]' \\| \\\n\t X\"$am_mf\" : 'X\\(//\\)$' \\| \\\n\t X\"$am_mf\" : 'X\\(/\\)' \\| . 2>/dev/null ||\n$as_echo X\"$am_mf\" |\n    sed '/^X\\(.*[^/]\\)\\/\\/*[^/][^/]*\\/*$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\/\\)[^/].*/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\/\\)$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\(\\/\\).*/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  s/.*/./; q'`\n    am_filepart=`$as_basename -- \"$am_mf\" ||\n$as_expr X/\"$am_mf\" : '.*/\\([^/][^/]*\\)/*$' \\| \\\n\t X\"$am_mf\" : 'X\\(//\\)$' \\| \\\n\t X\"$am_mf\" : 'X\\(/\\)' \\| . 2>/dev/null ||\n$as_echo X/\"$am_mf\" |\n    sed '/^.*\\/\\([^/][^/]*\\)\\/*$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\/\\(\\/\\/\\)$/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  /^X\\/\\(\\/\\).*/{\n\t    s//\\1/\n\t    q\n\t  }\n\t  s/.*/./; q'`\n    { echo \"$as_me:$LINENO: cd \"$am_dirpart\" \\\n      && sed -e '/# am--include-marker/d' \"$am_filepart\" \\\n        | $MAKE -f - am--depfiles\" >&5\n   (cd \"$am_dirpart\" \\\n      && sed -e '/# am--include-marker/d' \"$am_filepart\" \\\n        | $MAKE -f - am--depfiles) >&5 2>&5\n   ac_status=$?\n   echo \"$as_me:$LINENO: \\$? = $ac_status\" >&5\n   (exit $ac_status); } || am_rc=$?\n  done\n  if test $am_rc -ne 0; then\n    { { $as_echo \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\n$as_echo \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"Something went wrong bootstrapping makefile fragments\n    for automatic dependency tracking.  Try re-running configure with the\n    '--disable-dependency-tracking' option to at least be able to build\n    the package (albeit without support for automatic dependency tracking).\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n  fi\n  { am_dirpart=; unset am_dirpart;}\n  { am_filepart=; unset am_filepart;}\n  { am_mf=; unset am_mf;}\n  { am_rc=; unset am_rc;}\n  rm -f conftest-deps.mk\n}\n ;;\n\n  esac\ndone # for ac_tag\n\n\nas_fn_exit 0\n_ACEOF\nac_clean_files=$ac_clean_files_save\n\ntest $ac_write_fail = 0 ||\n  as_fn_error $? \"write failure creating $CONFIG_STATUS\" \"$LINENO\" 5\n\n\n# configure is writing to config.log, and then calls config.status.\n# config.status does its own redirection, appending to config.log.\n# Unfortunately, on DOS this fails, as config.log is still kept open\n# by configure, so config.status won't be able to write to it; its\n# output is simply discarded.  So we exec the FD to /dev/null,\n# effectively closing config.log, so it can be properly (re)opened and\n# appended to by config.status.  When coming back to configure, we\n# need to make the FD available again.\nif test \"$no_create\" != yes; then\n  ac_cs_success=:\n  ac_config_status_args=\n  test \"$silent\" = yes &&\n    ac_config_status_args=\"$ac_config_status_args --quiet\"\n  exec 5>/dev/null\n  $SHELL $CONFIG_STATUS $ac_config_status_args || ac_cs_success=false\n  exec 5>>config.log\n  # Use ||, not &&, to avoid exiting from the if with $? = 1, which\n  # would make configure fail if this is the last instruction.\n  $ac_cs_success || as_fn_exit 1\nfi\nif test -n \"$ac_unrecognized_opts\" && test \"$enable_option_checking\" != no; then\n  { $as_echo \"$as_me:${as_lineno-$LINENO}: WARNING: unrecognized options: $ac_unrecognized_opts\" >&5\n$as_echo \"$as_me: WARNING: unrecognized options: $ac_unrecognized_opts\" >&2;}\nfi\n\n\nchmod u+x bin/scalarisctl\nchmod u+x bin/jsonclient\nchmod u+x java-api/scalaris\nchmod u+x python-api/scalaris\nchmod u+x python3-api/scalaris\nchmod u+x ruby-api/scalaris\nchmod u+x contrib/init.d/scalaris\nchmod u+x contrib/init.d/scalaris-monitor\n\necho \"Erlang\"\nif test \"x$BITCASK_LIBS\" = \"x\" ; then\n  echo \"  Bitcask              : no\"\nelse\n  echo \"  Bitcask              : yes\"\nfi\n\nif test \"x$ERLANG_HANOIDB_FLAGS\" = \"x\" ; then\n  echo \"  hanoidb              : no\"\nelse\n  echo \"  hanoidb              : yes\"\nfi\n\nif test \"x$ERLANG_TOKE_FLAGS\" = \"x\" ; then\n  echo \"  toke                 : no\"\nelse\n  echo \"  toke                 : yes\"\nfi\n\nif test \"x$SSL_GETSTAT\" = \"xfalse\" ; then\n  echo \"  ssl                  : no\"\nelse\n  echo \"  ssl                  : yes\"\nfi\n\nif test \"x$enabled_cxx\" = \"x0\" ; then\n  echo \"C++ API                : no\"\nelse\n  echo \"C++ API                : yes\"\nfi\n\nif test \"x$ENABLEPYTHON2INSTALL\" = \"xinstall-python\" ; then\n  echo \"Python API             : yes\"\nelse\n  echo \"Python API             : no\"\nfi\n\nif test \"x$ENABLEPYTHON3\" = \"xpython3-compile\" ; then\n  echo \"Python3 API            : yes\"\nelse\n  echo \"Python3 API            : no\"\nfi\n"
  },
  {
    "path": "configure.ac",
    "content": "#   Copyright 2007-2019 Konrad-Zuse-Zentrum für Informationstechnik Berlin\n#\n#   Licensed under the Apache License, Version 2.0 (the \"License\");\n#   you may not use this file except in compliance with the License.\n#   You may obtain a copy of the License at\n#\n#       http://www.apache.org/licenses/LICENSE-2.0\n#\n#   Unless required by applicable law or agreed to in writing, software\n#   distributed under the License is distributed on an \"AS IS\" BASIS,\n#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#   See the License for the specific language governing permissions and\n#   limitations under the License.\n\nAC_PREREQ(2.65)\nAC_INIT(scalaris, 0.9.0+git, scalaris@googlegroups.com)\nAC_CONFIG_MACRO_DIR([m4])\n# we will never do automake, just pull in config.guess, config.sub, ... for boost\nAC_CONFIG_AUX_DIR([build-aux])\nAM_INIT_AUTOMAKE()\n\nEMAKEFILEDEFINES=\nEDOCMACROS=\nDIALYZER_FLAGS=\n\nAC_PATH_PROG(FALSE, false)\nAC_SUBST(FALSE)\nAC_PATH_PROG(TRUE, true)\nAC_SUBST(TRUE)\nAC_PROG_SED\nAC_PATH_PROG(DOXYGEN, doxygen)\nAC_SUBST(DOXYGEN)\n\nAC_PATH_PROG(AR, ar)\nAC_SUBST(AR)\nAC_PATH_PROG(LIBTOOL, libtool)\nAC_SUBST(LIBTOOL)\n\n###########################################################\n#\n# check for C++ and boost\n#\n###########################################################\n\nENABLE_CPP=\nAC_ARG_ENABLE([cpp],\n              AS_HELP_STRING([--disable-cpp],\n                 [disable support for the C++ bindings]),\n              [ENABLE_CPP=$enableval],\n              [ENABLE_CPP=$enableval])\n\nenabled_cxx=0\nif test \"x$ENABLE_CPP\" != xno ; then\n  # C++ and boost checks\n  AX_CXX_COMPILE_STDCXX_14([noext],[optional])\n  if test \"x$HAVE_CXX14\" = \"x1\" ; then\n    have_boost_147=\n    AX_BOOST_BASE([1.47], [have_boost_147=yes], [have_boost_147=no])\n    AX_BOOST_SYSTEM\n    AX_BOOST_REGEX\n    AX_BOOST_ASIO\n    AX_BOOST_PROGRAM_OPTIONS\n    AX_BOOST_UNIT_TEST_FRAMEWORK\n\n    AX_CHECK_OPENSSL([ENABLE_SSL=yes])\n\n    if test \"x$have_boost_147\" = \"xyes\" -a \"x$ax_cv_boost_system\" = \"xyes\" -a \"x$ax_cv_boost_regex\" = \"xyes\"  -a \"x$ax_cv_boost_program_options\" = \"xyes\"  -a \"x$ax_cv_boost_unit_test_framework\" = \"xyes\" -a \"x$ENABLE_SSL\" = \"xyes\" ; then\n      enabled_cxx=1\n    fi\n  fi\nfi\n\nif test \"x$enabled_cxx\" = \"x0\" ; then\n  AC_MSG_NOTICE([c++ bindings disabled ('make cpp' will fail)])\nfi\n\nAX_CHECK_COMPILE_FLAG(-MJ foo)\n\nJSONFLAGS=\"\"\nif test \"x$ax_cv_check_cflags___MJ_foo\" = \"xyes\"; then\nJSONFLAGS='-MJ $@.json'\nfi\n\nCXXSHELL=$SHELL\nif test -f /usr/local/bin/bash; then\n  CXXSHELL=/usr/local/bin/bash\nelif test -f /bin/bash; then\n  CXXSHELL=/bin/bash\nelif test -f /bin/zsh; then\n  CXXSHELL=/bin/zsh\nfi\n\nAC_PATH_PROG(CLANGFORMAT, clang-format, $ac_cv_path_TRUE)\nAC_PATH_PROG(CLANGTIDY, clang-tidy, $ac_cv_path_TRUE)\n\nAC_SUBST(OPENSSL_INCLUDES)\nAC_SUBST(OPENSSL_LDFLAGS)\nAC_SUBST(OPENSSL_LIBS)\n\nAC_SUBST(CXX)\nAC_SUBST(CXXFLAGS)\nAC_SUBST(CXXSHELL)\nAC_SUBST(JSONFLAGS)\nAC_SUBST(CLANGFORMAT)\nAC_SUBST(CLANGTIDY)\n\n###########################################################\n#\n# check tokyo cabinet and toke\n#\n###########################################################\n\nENABLE_TOKE=\nTOKEPREFIX=\nAC_ARG_ENABLE([toke],\n              AS_HELP_STRING([--enable-toke@<:@=DIR@:>@],\n                 [enable support for tokyo cabinet through toke]),\n              [ENABLE_TOKE=yes\n               TOKEPREFIX=$enableval])\n\n###########################################################\n#\n# check hanoidb\n#\n###########################################################\n\nENABLE_HANOIDB=\nHANOIDBPREFIX=\nAC_ARG_ENABLE([hanoidb],\n              AS_HELP_STRING([--enable-hanoidb@<:@=DIR@:>@],\n                 [enable support for hanoidb]),\n              [ENABLE_HANOIDB=yes\n               HANOIDBPREFIX=$enableval])\n\n###########################################################\n#\n# check bitcask\n#\n###########################################################\n\nENABLE_BITCASK=\nBITCASKPREFIX=\nAC_ARG_ENABLE([bitcask],\n              AS_HELP_STRING([--enable-bitcask@<:@=DIR@:>@],\n                 [enable support for bitcask]),\n              [ENABLE_BITCASK=yes\n               BITCASKPREFIX=$enableval])\n\n###########################################################\n#\n# check erlang_js\n#\n###########################################################\n\nENABLE_ERLANGJS=\nERLANGJSPREFIX=\nAC_ARG_ENABLE([erlang-js],\n              AS_HELP_STRING([--enable-erlang-js@<:@=DIR@:>@],\n                 [enable support for erlang_js]),\n              [ENABLE_ERLANGJS=yes\n               ERLANGJSPREFIX=$enableval])\n\n###########################################################\n#\n# check browser for 'make test-vts'\n#\n###########################################################\n\nBROWSER=\nAC_ARG_WITH([browser],\n            [AS_HELP_STRING([--with-browser=BROWSER],\n               [absolute path of the browser to use for 'make test-vts'])],\n            [if test -n \"$withval\"; then\n               BROWSER=\"$withval\"\n               AS_IF([test ! -x \"$BROWSER\"],\n                 [AC_MSG_WARN([\"$BROWSER\" was not found or is not executable ('make test-vts' will likely fail)])])\n             fi],\n            [AC_PATH_PROG([BROWSER], [xdg-open])])\nAC_SUBST(BROWSER)\n\nAS_IF([test \"$BROWSER\" = \"\"],\n      [AC_MSG_WARN([xdg-open was not found ('make test-vts' will likely fail)])])\n\n###########################################################\n#\n# check screen availability for 'scalarisctl -d --screen'\n#\n###########################################################\nAC_PATH_PROG(SCREEN, screen, [$FALSE])\nif test \"$ac_cv_path_SCREEN\" = \"$FALSE\" ; then\n  AC_MSG_NOTICE([screen not found - you won't be able to run scalaris deamonized with screen, i.e. 'scalarisctl -d --screen'])\nfi\n\n###########################################################\n#\n# check sudo, runuser availability for the init.d script\n#\n###########################################################\n\nAC_PATH_PROG(SUDO, sudo, [$FALSE])\n\nRUNUSER=\"\"\nAC_ARG_ENABLE([runuser],\n              AS_HELP_STRING([--disable-runuser],\n                 [disable support for runuser (use sudo instead)]))\nAS_IF([test \"$enable_runuser\" != \"no\"],\n      [AC_PATH_PROG(RUNUSER, runuser, \"\", [$PATH$PATH_SEPARATOR/usr/sbin])])\n\nif test \"$ac_cv_path_SUDO\" = \"$FALSE\" -a \"$ac_cv_path_RUNUSER\" = \"\"; then\n  AC_MSG_NOTICE([sudo or runuser not found - you won't be able to use our init.d script])\nfi\n\n###########################################################\n#\n# check systemd support\n#\n###########################################################\n\nINSTALL_INIT=install-initd\nSYSTEMD_UNITDIR=\nAC_ARG_WITH([systemd],\n            [AS_HELP_STRING([--with-systemd[[=UNITDIR]]],\n                            [use systemd, optionally specify the directory for systemd service files])],\n            [if test -d \"$withval\"; then\n               SYSTEMD_UNITDIR=\"$withval\"\n               INSTALL_INIT=\"install-systemd\"\n             else\n               if test \"$withval\" = \"yes\"; then\n                 SYSTEMD_UNITDIR=\"\\${prefix}/lib/systemd/system/\"\n                 INSTALL_INIT=\"install-systemd\"\n               fi\n             fi],\n            [])\nAC_SUBST(INSTALL_INIT)\nAC_SUBST(SYSTEMD_UNITDIR)\n\n###########################################################\n#\n# check whether to compile to native code using HiPE\n#\n###########################################################\n\nCOMPILE_NATIVE=no\nAC_ARG_ENABLE([native],\n              AS_HELP_STRING([--enable-native],\n                [enable compilation to native code using HiPE]),\n              [COMPILE_NATIVE=yes])\n\n###########################################################\n#\n# check wether to enable debugging, e.g. enable the ASSERT macro\n#\n###########################################################\n\nAC_ARG_ENABLE([debug],\n              AS_HELP_STRING([--enable-debug], [enable debugging mode]),\n              [AS_IF([test \"no\" != \"$enableval\"],\n                     [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, enable_debug}\"\n                      EDOCMACROS=\"${EDOCMACROS}, {enable_debug, true}\"\n                      DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Denable_debug\"])])\n\n###########################################################\n#\n# enable new transaction protocol\n#\n###########################################################\n\nAC_ARG_ENABLE([txnew],\n              AS_HELP_STRING([--enable-txnew], [enable new transaction protocol]),\n              [AS_IF([test \"no\" != \"$enableval\"],\n                     [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, enable_txnew}\"\n                      EDOCMACROS=\"${EDOCMACROS}, {enable_txnew, true}\"\n                      DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Denable_txnew\"])])\n\n###########################################################\n#\n# check erlang\n#\n###########################################################\nAC_LANG([Erlang])\n\nAC_ERLANG_NEED_ERLC\nAC_ERLANG_NEED_ERL\n\nAC_ERLANG_SUBST_ROOT_DIR\nAC_ERLANG_SUBST_LIB_DIR\n\nAC_CACHE_CHECK([for Erlang/OTP ERTS version],\n    [erlang_cv_erts_ver],\n    [AC_LANG_PUSH([Erlang])[]dnl\n     AC_RUN_IFELSE(\n        [AC_LANG_PROGRAM([], [dnl\n            Version = erlang:system_info(version),\n            file:write_file(\"conftest.out\", Version),\n            ReturnValue = 0,\n            halt(ReturnValue)])],\n        [erlang_cv_erts_ver=`cat conftest.out`],\n        [AC_MSG_FAILURE([test Erlang program execution failed])])\n     AC_LANG_POP([Erlang])[]dnl\n    ])\nAC_SUBST([ERLANG_ERTS_VER], [$erlang_cv_erts_ver])\n\n# split erlang erts version:\nset `echo $erlang_cv_erts_ver | $SED 's/\\./ /g'`\nERTS_MAJOR=$1\nERTS_MINOR=$2\nERTS_MAINT=${3:-0}\n\n# require Erlang >= R14B04, i.e. ERTS >= 5.8.5\nAS_IF([test $ERTS_MAJOR -gt 5 -o \\\n       '(' $ERTS_MAJOR -eq 5 -a $ERTS_MINOR -gt 8 ')' -o \\\n       '(' $ERTS_MAJOR -eq 5 -a $ERTS_MINOR -eq 8 -a $ERTS_MAINT -ge 5 ')'],\n      [],\n      [AC_MSG_FAILURE([Erlang >= R14B04 required])])\n\nAC_PATH_PROG(EPMD, epmd, [$FALSE], \"$PATH:$ERLANG_ROOT_DIR/bin:$ERLANG_ROOT_DIR/erts-$ERLANG_ERTS_VER/bin\")\n\nAS_IF([test \"x$COMPILE_NATIVE\" != xno], [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, native\"])\n\n# required libs:\nERLANG_UNAVAILABLE_LIBS=\nAC_ERLANG_CHECK_LIB(erts, , ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS erts\")\n\nAC_ERLANG_CHECK_LIB(kernel, , ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS kernel\")\nAC_CHECK_FILE($ERLANG_LIB_DIR_kernel/include/inet.hrl, ,\n              AC_MSG_FAILURE([kernel/include/inet.hrl not found - you won't be able to compile the erlang sources]))\n\nAC_ERLANG_CHECK_LIB(stdlib, , ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS stdlib\")\n\nAC_ERLANG_CHECK_LIB(compiler, , ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS compiler\")\n\nAC_ERLANG_CHECK_LIB(crypto, , ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS crypto\")\n\nAC_ERLANG_CHECK_LIB(os_mon, , ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS os_mon\")\n\nAC_ERLANG_CHECK_LIB(tools, , ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS tools\")\n\nAC_ERLANG_CHECK_LIB(inets, , ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS inets\")\n\nAC_ERLANG_CHECK_LIB(ssl, , ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS ssl\")\n\nAC_ERLANG_CHECK_LIB(xmerl, , ERLANG_UNAVAILABLE_LIBS=\"$ERLANG_UNAVAILABLE_LIBS xmerl\")\n\nAS_IF([test \"x$ERLANG_UNAVAILABLE_LIBS\" != x], AC_MSG_FAILURE([$ERLANG_UNAVAILABLE_LIBS not found - you won't be able to compile or run the erlang sources]))\n\n# libs for optional build tasks:\nAC_ERLANG_CHECK_LIB(common_test, , AC_MSG_NOTICE([erlang-common_test not found - you won't be able to run the unit tests without common_test]))\n\nAC_ERLANG_CHECK_LIB(edoc, , AC_MSG_NOTICE([erlang-edoc not found - you won't be able to create the documentation or run 'make install']))\n\nAC_CHECK_FILE($ERLANG_LIB_DIR_common_test/priv/bin/run_test,\n              [RUN_TEST_FILE=$ERLANG_LIB_DIR_common_test/priv/bin/run_test],\n              [AC_CHECK_FILE($ac_cv_erlang_root_dir/bin/run_test,\n                       [RUN_TEST_FILE=$ac_cv_erlang_root_dir/bin/run_test],\n                       [AC_PATH_PROG([RUN_TEST_FILE],[run_test],\"\")])])\nAC_CHECK_FILE($ERLANG_LIB_DIR_common_test/priv/bin/ct_run,\n              [CT_RUN_FILE=$ERLANG_LIB_DIR_common_test/priv/bin/ct_run],\n              [AC_CHECK_FILE($ac_cv_erlang_root_dir/bin/ct_run,\n                       [CT_RUN_FILE=$ac_cv_erlang_root_dir/bin/ct_run],\n                       [AC_PATH_PROG([CT_RUN_FILE],[ct_run],\"\")])])\n\nAC_MSG_CHECKING([for unit-test runner])\nif test -n \"$RUN_TEST_FILE\"; then\n  AC_SUBST(RUN_TEST,\"$RUN_TEST_FILE\")\n  AC_MSG_RESULT([$RUN_TEST_FILE])\nelif test -n \"$CT_RUN_FILE\"; then\n  AC_SUBST(RUN_TEST,\"$CT_RUN_FILE\")\n  AC_MSG_RESULT([$CT_RUN_FILE])\nelse\n  AC_MSG_RESULT([neither run_test nor ct_run found - on erlang < R14 consider running install.sh in the common_test directory otherwise you won't be able to run the unit tests])\nfi\n\n# toke\nERLANG_TOKE_FLAGS=\nAS_IF([test \"x$TOKEPREFIX\" != xno],\n            [\n            ERLANG_TOKE_MESSAGE=\n            AS_IF([test \"x$TOKEPREFIX\" != x], [export ERL_LIBS=\"$TOKEPREFIX\"])\n            AC_ERLANG_CHECK_LIB(toke,\n              [AC_CHECK_FILE($ERLANG_LIB_DIR_toke/priv/libtoke.so,\n                [ERLANG_TOKE_FLAGS=\"-pa $ERLANG_LIB_DIR_toke/ebin\"],\n                [ERLANG_TOKE_MESSAGE=\"toke library libtoke.so not found\"])\n              ],\n              [ERLANG_TOKE_MESSAGE=\"toke erlang library not found\"])\n            AS_IF([test \"x$TOKEPREFIX\" != x], [export ERL_LIBS=\"\"])\n            AS_IF([test \"x$ERLANG_TOKE_MESSAGE\" != x],\n              [AS_IF([test \"x$ENABLE_TOKE\" != xyes],\n                [ERLANG_TOKE_MESSAGE=\"$ERLANG_TOKE_MESSAGE, disabling toke support...\"\n                 AC_MSG_NOTICE($ERLANG_TOKE_MESSAGE)],\n                AC_MSG_FAILURE($ERLANG_TOKE_MESSAGE))])\n            ])\nAC_SUBST([ERLANG_TOKE_FLAGS])\n\n# hanoidb\nERLANG_HANOIDB_FLAGS=\nAS_IF([test \"x$HANOIDBPREFIX\" != xno],\n            [\n            ERLANG_HANOIDB_MESSAGE=\n            AS_IF([test \"x$HANOIDBPREFIX\" != x], [export ERL_LIBS=\"$HANOIDBPREFIX\"])\n            AC_ERLANG_CHECK_LIB(hanoidb,\n              [ERLANG_HANOIDB_FLAGS=\"-pa $ERLANG_LIB_DIR_hanoidb/ebin\"],\n              [ERLANG_HANOIDB_MESSAGE=\"hanoidb erlang not found\"])\n            AS_IF([test \"x$HANOIDBPREFIX\" != x], [export ERL_LIBS=\"\"])\n            AS_IF([test \"x$ERLANG_HANOIDB_MESSAGE\" != x],\n              [AS_IF([test \"x$ENABLE_HANOIDB\" != xyes],\n                [ERLANG_HANOIDB_MESSAGE=\"$ERLANG_HANOIDB_MESSAGE, disabling hanoidb support...\"\n                 AC_MSG_NOTICE($ERLANG_HANOIDB_MESSAGE)],\n                AC_MSG_FAILURE($ERLANG_HANOIDB_MESSAGE))])\n            ])\nAC_SUBST([ERLANG_HANOIDB_FLAGS])\n\n# bitcask\nERLANG_BITCASK_FLAGS=\nBITCASK_LIBS=\nAS_IF([test \"x$BITCASKPREFIX\" != xno],\n            [\n            ERLANG_BITCASK_MESSAGE=\n            AS_IF([test \"x$BITCASKPREFIX\" != x], [export ERL_LIBS=\"$BITCASKPREFIX\"])\n            AC_ERLANG_CHECK_LIB(bitcask,\n              [\n                EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_bitcask}\"\n                EDOCMACROS=\"${EDOCMACROS}, {have_bitcask, true}\"\n                DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_bitcask\"\n                ERLANG_BITCASK_FLAGS=\"-pa $ERLANG_LIB_DIR_bitcask/ebin\"\n                BITCASK_LIBS=\"$ERLANG_LIB_DIR_bitcask\"\n              ],\n              [ERLANG_BITCASK_MESSAGE=\"bitcask erlang not found\"])\n            AS_IF([test \"x$BITCASKPREFIX\" != x], [export ERL_LIBS=\"\"])\n            AS_IF([test \"x$ERLANG_BITCASK_MESSAGE\" != x],\n              [AS_IF([test \"x$ENABLE_BITCASK\" != xyes],\n                [ERLANG_BITCASK_MESSAGE=\"$ERLANG_BITCASK_MESSAGE, disabling bitcask support...\"\n                 AC_MSG_NOTICE($ERLANG_BITCASK_MESSAGE)],\n                AC_MSG_FAILURE($ERLANG_BITCASK_MESSAGE))])\n            ])\nAC_SUBST([ERLANG_BITCASK_FLAGS])\nAC_SUBST([BITCASK_LIBS])\n\n# erlang_js\nERLANG_ERLANGJS_FLAGS=\nAS_IF([test \"x$ERLANGJSPREFIX\" != xno],\n            [\n            ERLANG_ERLANGJS_MESSAGE=\n            AS_IF([test \"x$ERLANGJSPREFIX\" != x], [export ERL_LIBS=\"$ERLANGJSPREFIX\"])\n            AC_ERLANG_CHECK_LIB(erlang_js,\n              [AC_CHECK_FILE($ERLANG_LIB_DIR_erlang_js/priv/erlang_js_drv.so,\n                [ERLANG_ERLANGJS_FLAGS=\"-pa $ERLANG_LIB_DIR_erlang_js/ebin\"],\n                [ERLANG_ERLANGJS_MESSAGE=\"erlang_js library erlang_js_drv.so not found\"])\n              ],\n              [ERLANG_ERLANGJS_MESSAGE=\"erlang_js erlang library not found\"])\n            AS_IF([test \"x$ERLANGJSPREFIX\" != x], [export ERL_LIBS=\"\"])\n            AS_IF([test \"x$ERLANG_ERLANGJS_MESSAGE\" != x],\n              [AS_IF([test \"x$ENABLE_ERLANGJS\" != xyes],\n                [ERLANG_ERLANGJS_MESSAGE=\"$ERLANG_ERLANGJS_MESSAGE, disabling erlang_js support...\"\n                 AC_MSG_NOTICE($ERLANG_ERLANGJS_MESSAGE)],\n                AC_MSG_FAILURE($ERLANG_ERLANGJS_MESSAGE))])\n            ])\nAC_SUBST([ERLANG_ERLANGJS_FLAGS])\n\ncrypto_start=`\"${ERL}\" -noshell -eval 'io:format(\"~p~n\", [[crypto:start()]]), erlang:halt().' | tail -1`\nAS_IF([test \"${crypto_start}\" = \"ok\"],\n      [AC_MSG_NOTICE([crypto:start() is available])],\n      [AC_MSG_FAILURE([cannot start the crypto subsystem])])\n\ncrypto_rand_uniform=`\"${ERL}\" -noshell -eval 'error_logger:tty(false), code:load_file(crypto), io:format(\"~p~n\", [[erlang:function_exported(crypto,rand_uniform,2)]]), erlang:halt().' | tail -1`\nAS_IF([test \"${crypto_rand_uniform}\" = \"true\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_crypto_randuniform_support}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_crypto_randuniform_support, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_crypto_randuniform_support\"\n       AC_MSG_NOTICE([crypto:rand_uniform/2 is available])],\n      [AC_MSG_NOTICE([crypto:rand_uniform/2 is not available])])\n\ncrypto_hash=`\"${ERL}\" -noshell -eval 'error_logger:tty(false), code:load_file(crypto), io:format(\"~p~n\", [[erlang:function_exported(crypto,hash,2)]]), erlang:halt().' | tail -1`\nAS_IF([test \"${crypto_hash}\" = \"true\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, with_crypto_hash}\"\n       EDOCMACROS=\"${EDOCMACROS}, {with_crypto_hash, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dwith_crypto_hash\"\n       AC_MSG_NOTICE([crypto:hash/2 is available and usable])],\n      [AC_MSG_NOTICE([crypto:hash/2 is not available or not usable, falling back to crypto:md5/1 and crypto:sha1/1])])\n\ncrypto_bytes_to_integer=`\"${ERL}\" -noshell -eval 'error_logger:tty(false), code:load_file(crypto), io:format(\"~p~n\", [[erlang:function_exported(crypto,bytes_to_integer,1)]]), erlang:halt().' | tail -1`\nAS_IF([test \"${crypto_bytes_to_integer}\" = \"true\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, with_crypto_bytes_to_integer}\"\n       EDOCMACROS=\"${EDOCMACROS}, {with_crypto_bytes_to_integer, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dwith_crypto_bytes_to_integer\"\n       AC_MSG_NOTICE([crypto:bytes_to_integer/1 is available and usable])],\n      [AC_MSG_NOTICE([crypto:bytes_to_integer/1 is not available or not usable, falling back to rand:uniform/2])])\n\nct_line=`\"${ERL}\" -noshell -eval 'error_logger:tty(false), code:load_file(ct_line), io:format(\"~p~n\", [[erlang:function_exported(ct_line,parse_transform,2)]]), erlang:halt().' | tail -1`\nAS_IF([test \"${ct_line}\" = \"true\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_ctline_support}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_ctline_support, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_ctline_support\"\n       AC_MSG_NOTICE([ct_line is available])],\n      [AC_MSG_NOTICE([ct_line is not available])])\n\nrand_module=`\"${ERL}\" -noshell -eval 'error_logger:tty(false), code:load_file(rand), io:format(\"~p~n\", [[erlang:function_exported(rand,uniform,0)]]), erlang:halt().' | tail -1`\nAS_IF([test \"${rand_module}\" = \"true\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, with_rand}\"\n       EDOCMACROS=\"${EDOCMACROS}, {with_rand, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dwith_rand\"\n       AC_MSG_NOTICE([rand:uniform/0 is available and usable])],\n      [AC_MSG_NOTICE([rand:uniform/0 is not available or not usable, falling back to random:uniform/0])])\n\nAC_RUN_IFELSE(\n       [AC_LANG_PROGRAM([-callback init() -> ok.], [ok])],\n       [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_callback_support}\"\n        EDOCMACROS=\"${EDOCMACROS}, {have_callback_support, true}\"\n        DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_callback_support\"\n        AC_MSG_NOTICE([-callback is available])],\n       [AC_MSG_NOTICE([-callback is not available])])\n\nmaps_module=`\"${ERL}\" -noshell -eval 'error_logger:tty(false), code:load_file(maps), io:format(\"~p~n\", [[erlang:function_exported(maps,new,0)]]), erlang:halt().' | tail -1`\nAS_IF([test \"${maps_module}\" = \"true\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, with_maps}\"\n       EDOCMACROS=\"${EDOCMACROS}, {with_maps, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dwith_maps\"\n       AC_MSG_NOTICE([maps are available and seem to be usable])],\n      [AC_MSG_NOTICE([maps are not available or not usable, falling back to gb_trees])])\n\n# creates forms specifying a minimal erlang module which spezifies a\n# type based on dict:dict() and checks if compilation of this module fails\ndict_type=`\"${ERL}\" -noshell -eval 'X = fun(S) -> element(2,erl_parse:parse_form(element(2,erl_scan:string(S)))) end, io:format(\"~p~n\", [[element(1, compile:forms([X(\"-module(test).\"), X(\"-type test() :: dict:dict().\")]))]]), erlang:halt().' 2> /dev/null | tail -1`\nAS_IF([test \"${dict_type}\" = \"ok\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, namespaced_dict}\"\n       EDOCMACROS=\"${EDOCMACROS}, {namespaced_dict, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dnamespaced_dict\"\n       AC_MSG_NOTICE([erlang dict type is defined as dict:dict()])],\n      [AC_MSG_NOTICE([erlang dict type is defined as dict()])])\n\n###########################################################\n#\n# check for file:send_file/5 for yaws\n#\n###########################################################\nYAWS_OPTIONS=\nHAVE_ERLANG_SENDFILE=false\n\nAC_MSG_CHECKING([for file:sendfile/5])\nfile_sendfile=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(file), io:format(\"~p~n\",[[erlang:function_exported(file,sendfile,5)]]), erlang:halt().' | tail -1`\nif test \"$file_sendfile\" = true; then\n  AS_IF([test $ERTS_MAJOR -gt 5 -o \\\n         '(' $ERTS_MAJOR -eq 5 -a $ERTS_MINOR -gt 9 ')' -o \\\n         '(' $ERTS_MAJOR -eq 5 -a $ERTS_MINOR -eq 9 -a $ERTS_MAINT -ge 1 ')'],\n        [AC_MSG_RESULT([yes])\n         HAVE_ERLANG_SENDFILE=true],\n        [AC_MSG_RESULT([no])])\nelse\n        AC_MSG_RESULT([no])\nfi\n\nAS_IF([test \"$HAVE_ERLANG_SENDFILE\" = true],\n      [YAWS_OPTIONS=\", {d, 'HAVE_ERLANG_SENDFILE'}\"\n       YAWS_OPTIONS_DIALYZER=\" -DHAVE_ERLANG_SENDFILE\"\n       AC_MSG_NOTICE([using file:sendfile/5 for yaws])],\n      [YAWS_OPTIONS=\"\"\n       YAWS_OPTIONS_DIALYZER=\"\"\n       AC_MSG_NOTICE([using fallback gen_tcp:send/2 for yaws])])\n\nAS_IF([test \"${crypto_hash}\" = \"true\"],\n      [YAWS_OPTIONS=\"${YAWS_OPTIONS}, {d, 'HAVE_CRYPTO_HASH'}\"\n       YAWS_OPTIONS_DIALYZER=\"${YAWS_OPTIONS_DIALYZER} -DHAVE_CRYPTO_HASH\"])\n\n###########################################################\n#\n# check for erlang:timestamp/0 for yaws\n#\n###########################################################\nerlang_timestamp=`\"${ERL}\" -noshell -eval 'io:format(\"~p~n\", [[erlang:function_exported(erlang,timestamp,0)]]), erlang:halt().' | tail -1`\nAS_IF([test \"${erlang_timestamp}\" = \"true\"],\n      [AC_MSG_NOTICE([erlang:timestamp() is available])],\n      [YAWS_OPTIONS=\"${YAWS_OPTIONS}, {d, 'HAVE_ERLANG_NOW'}\"\n       YAWS_OPTIONS_DIALYZER=\"${YAWS_OPTIONS_DIALYZER} -DHAVE_ERLANG_NOW\"\n       AC_MSG_NOTICE([erlang:timestamp() is not available])])\n\n###########################################################\n#\n# check for ssl:getstat/2 for ssl\n#\n###########################################################\n## https://github.com/erlang/otp/commit/84fd2c325c9e38b5ea2307b6133c3d15b33a3241\nSSL_GETSTAT=false\nssl_getstat=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(ssl),io:format(\"~p~n\", [[erlang:function_exported(ssl,getstat,2)]]), erlang:halt().' | tail -1`\nAS_IF([test \"${ssl_getstat}\" = \"true\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_ssl_getstat}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_ssl_getstat, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_ssl_getstat\"\n       SSL_GETSTAT=true\n       AC_MSG_NOTICE([ssl:getstat() is available])],\n      [AC_MSG_NOTICE([ssl:getstat() is not available])])\n\n###########################################################\n#\n# check for ssl:handshake/1 for ssl\n#\n###########################################################\n## https://github.com/erlang/otp/commit/8dcfffd4fb8520239b189357e81122d4fdacb42d\nSSL_HANDSHAKE=false\nssl_handshake=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(ssl),io:format(\"~p~n\", [[erlang:function_exported(ssl,handshake,1)]]), erlang:halt().' | tail -1`\nAS_IF([test \"${ssl_handshake}\" = \"true\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_ssl_handshake}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_ssl_handshake, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_ssl_handshake\"\n       SSL_HANDSHAKE=true\n       AC_MSG_NOTICE([ssl:handshake() is available])],\n      [AC_MSG_NOTICE([ssl:handshake() is not available])])\n\n###########################################################\n#\n# check for new stacktrace syntax\n#\n###########################################################\n# http://erlang.org/eeps/eep-0047.md\n\nerlang_cv_new_stacktrace=0\nAC_LANG_PUSH([Erlang])[]dnl\nAC_RUN_IFELSE(\n[AC_LANG_PROGRAM([], [dnl\ntry\n  5\ncatch\n  C:E:Stk ->\n          Stk\nend])],\n[erlang_cv_new_stacktrace=true],\n[erlang_cv_new_stacktrace=false])\nAC_LANG_POP([Erlang])[]dnl\n\nAS_IF([test \"${erlang_cv_new_stacktrace}\" = \"true\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_new_stacktrace}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_new_stacktrace, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_new_stacktrace\"\n       AC_MSG_NOTICE([new stacktrace syntax is available])],\n      [AC_MSG_NOTICE([new stacktrace syntax is not available])])\n\n###########################################################\n#\n# check for socket:open/2 since 22.0\n#\n###########################################################\n# https://github.com/erlang/otp/commit/3ca71520bfb664f0ea809ffdf41505936e4d5e90\nSOCKET_OPEN=false\nsocket_open=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(socket),io:format(\"~p~n\", [[erlang:function_exported(socket,open,2)]]), erlang:halt().' | tail -1`\nAS_IF([test \"${socket_open}\" = \"true\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_socket_open}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_socket_open, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_socket_open\"\n       SOCKET_OPEN=true\n       AC_MSG_NOTICE([socket:open/2 is available])],\n      [AC_MSG_NOTICE([socket:open/2 is not available])])\n\n###########################################################\n#\n# check for persistent_term:get/2 since 21.2.\n#\n###########################################################\n# https://github.com/erlang/otp/commit/805748eb668d5562fe17f3172cdae07a86166c3f\nPERSISTENT_TERM_GET=false\npersistent_term_get=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(persistent_term),io:format(\"~p~n\", [[erlang:function_exported(persistent_term,get,1)]]), erlang:halt().' | tail -1`\nAS_IF([test \"${persistent_term_get}\" = \"true\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_persistent_term_get}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_persistent_term_get, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_persistent_term_get\"\n       PERSISTENT_TERM_GET=true\n       AC_MSG_NOTICE([persistent_term:get/2 is available])],\n      [AC_MSG_NOTICE([persistent_term:get/2 is not available])])\n\n###########################################################\n#\n# check for counters:get/2 since 21.2.\n#\n###########################################################\n# https://github.com/erlang/otp/commit/fefb5d039e87ff7137e78b3d5f2eaf01e498ec4d\nCOUNTERS_GET=false\ncounters_get=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(counters),io:format(\"~p~n\", [[erlang:function_exported(counters,get,2)]]), erlang:halt().' | tail -1`\nAS_IF([test \"${counters_get}\" = \"true\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_counters_get}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_counters_get, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_counters_get\"\n       COUNTERS_GET=true\n       AC_MSG_NOTICE([counters:get/2 is available])],\n      [AC_MSG_NOTICE([counters:get/2 is not available])])\n\n###########################################################\n#\n# check for atomics:new/2 since 21.2.\n#\n###########################################################\n# https://github.com/erlang/otp/commit/1315c6457e49595fdd3f91693c0506964416c9f0\nATOMICS_NEW=false\natomics_new=`\"${ERL}\" -noshell -eval 'code:ensure_loaded(atomics),io:format(\"~p~n\", [[erlang:function_exported(atomics,new,2)]]), erlang:halt().' | tail -1`\nAS_IF([test \"${atomics_new}\" = \"true\"],\n      [EMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}, {d, have_atomics_new}\"\n       EDOCMACROS=\"${EDOCMACROS}, {have_atomics_new, true}\"\n       DIALYZER_FLAGS=\"${DIALYZER_FLAGS} -Dhave_atomics_new\"\n       ATOMICS_NEW=true\n       AC_MSG_NOTICE([atomics:new/2 is available])],\n      [AC_MSG_NOTICE([atomics:new/2 is not available])])\n\n###########################################################\n#\n# all defines together...\n#\n###########################################################\n\nEMAKEFILEDEFINES=\"${EMAKEFILEDEFINES}\"\nEDOCMACROS=\"${EDOCMACROS#, }\"\nDIALYZER_FLAGS=\"${DIALYZER_FLAGS## }${YAWS_OPTIONS_DIALYZER}\"\n\nAC_SUBST(EMAKEFILEDEFINES)\nAC_SUBST(EDOCMACROS)\nAC_SUBST(DIALYZER_FLAGS)\n\nAC_SUBST(EMAKEFILECOMPILECOMPAT)\nAC_SUBST(YAWS_OPTIONS)\n\n###########################################################\n#\n# check for routing table type\n#\n###########################################################\n\nRT=rt_chord\nAC_ARG_WITH([rt],\n            [AS_HELP_STRING([--with-rt[[=RTFILE]]],\n                            [optionally specify the routing table type (default=rt_chord)])],\n            [if test \"$withval\"; then\n               RT=\"$withval\"\n             fi],\n            [])\nAC_SUBST(RT)\n\n###########################################################\n#\n# check java-functions, build-classpath availability for java-api/scalaris\n#\n###########################################################\n\nJAVAFUNCTIONS=\nAC_ARG_WITH([java-functions],\n            [AS_HELP_STRING([--with-java-functions=JAVAFUNCTIONS],\n               [use the given java-functions script for java-api/scalaris (default is /usr/share/java-utils/java-functions)])],\n            [with_java_functions=$withval],\n            [with_java_functions=/usr/share/java-utils/java-functions])\n\nAC_CHECK_FILE($with_java_functions, [JAVAFUNCTIONS=$with_java_functions],\n              [JAVAFUNCTIONS=$with_java_functions\n               AC_MSG_WARN([java-functions was not found in \"$with_java_functions\" (java-api/scalaris may fail)])])\nAC_SUBST(JAVAFUNCTIONS)\n\nBUILDCLASSPATH=\nAC_ARG_WITH([build-classpath],\n            [AS_HELP_STRING([--with-build-classpath=BUILDCLASSPATH],\n               [absolute path of the build-classpath script to use for java-api/scalaris])],\n            [if test -n \"$withval\"; then\n               BUILDCLASSPATH=\"$withval\"\n             fi],\n            [AC_PATH_PROG([BUILDCLASSPATH], [build-classpath])])\nAC_SUBST(BUILDCLASSPATH)\n\nAS_IF([test ! -x \"$BUILDCLASSPATH\"],\n      [AC_MSG_WARN([build-classpath was not found in \"$BUILDCLASSPATH\" or is not executable (java-api/scalaris may fail)])])\n\n###########################################################\n#\n# check ruby\n#\n###########################################################\n\nENABLERUBYINSTALL=\nRUBYSITELIBDIR=\nAC_ARG_WITH([ruby-sitelibdir],\n            [AS_HELP_STRING([--with-ruby-sitelibdir=SITELIBDIR],\n            [where to install ruby libraries])],\n            [if test -n \"$withval\"; then\n               RUBYSITELIBDIR=\"$withval\"\n               ENABLERUBYINSTALL=\"install-ruby\"\n             fi],\n            [])\nAC_SUBST(ENABLERUBYINSTALL)\nAC_SUBST(RUBYSITELIBDIR)\n\n###########################################################\n#\n# check python, python3\n#\n###########################################################\nPYTHON2SITELIBDIR=\nENABLEPYTHON2INSTALL=\nAC_CACHE_CHECK([for python 2.x >= 2.6], [ac_cv_path_PYTHON2],\n  [AC_PATH_PROGS_FEATURE_CHECK([PYTHON2], [python python2],\n    [[$ac_path_PYTHON2 -V 2>&1 | grep \"^Python 2.[6789]\" > /dev/null && \\\n      PYTHON2SITELIBDIR=`$ac_path_PYTHON2 -c 'from distutils.sysconfig import get_python_lib; print(get_python_lib(0,0,prefix=\"${prefix}\"))'  2>/dev/null` && \\\n      ac_cv_path_PYTHON2=$ac_path_PYTHON2 ac_path_PYTHON2_found=:]],\n    [AC_MSG_RESULT([not found])])])\n\nAS_IF([test \"$ac_cv_path_PYTHON2\" != \"\"],\n      [PYTHON2=$ac_cv_path_PYTHON2\n       AC_MSG_CHECKING([for python 2.x sitelibdir])\n       AC_MSG_RESULT([$PYTHON2SITELIBDIR])\n       ENABLEPYTHON2INSTALL=\"install-python\"],\n      [PYTHON2=$FALSE])\nAC_SUBST(PYTHON2)\nAC_SUBST(PYTHON2SITELIBDIR)\nAC_SUBST(ENABLEPYTHON2INSTALL)\n\nPYTHON3SITELIBDIR=\nENABLEPYTHON3INSTALL=\nAC_CACHE_CHECK([for python 3.x], [ac_cv_path_PYTHON3],\n  [AC_PATH_PROGS_FEATURE_CHECK([PYTHON3], [python3 python],\n    [[$ac_path_PYTHON3 -V 2>&1 | grep \"^Python 3.\" > /dev/null && \\\n      PYTHON3SITELIBDIR=`$ac_path_PYTHON3 -c 'from distutils.sysconfig import get_python_lib; print(get_python_lib(0,0,prefix=\"${prefix}\"))'  2>/dev/null` && \\\n      ac_cv_path_PYTHON3=$ac_path_PYTHON3 ac_path_PYTHON3_found=:]],\n    [AC_MSG_RESULT([not found])])])\n\nPYTHON3=\nPYTHON3_2TO3=\nENABLEPYTHON3=python3-not-found\nENABLEPYTHON3INSTALL=\nAS_IF([test \"$ac_cv_path_PYTHON3\" != \"\"],\n      [PYTHON3=$ac_cv_path_PYTHON3\n       AC_MSG_CHECKING([for python 3.x sitelibdir])\n       AC_MSG_RESULT([$PYTHON3SITELIBDIR])\n       AC_PATH_PROG([PYTHON3_2TO3], [2to3], \"\")\n       AS_IF([test -x \"$PYTHON3_2TO3\"],\n             [ENABLEPYTHON3=\"python3-compile\"\n              ENABLEPYTHON3INSTALL=\"install-python3\"],\n             [AC_MSG_WARN([2to3 not found - you won't be able to build the Python3 API])])],\n      [PYTHON3=$FALSE])\nAC_SUBST(PYTHON3)\nAC_SUBST(PYTHON3_2TO3)\nAC_SUBST(PYTHON3SITELIBDIR)\nAC_SUBST(ENABLEPYTHON3)\nAC_SUBST(ENABLEPYTHON3INSTALL)\n\n###########################################################\n#\n# check MACOSX vs. Linux for flexbisonparse driver\n#\n###########################################################\n\nSAVECFLAGS=$CFLAGS\nCFLAGS=\"-I$ERLANG_ROOT_DIR/erts-$ERLANG_LIB_VER_erts/include $CFLAGS\"\nAC_LANG_PUSH([C])\nAC_CHECK_HEADER([erl_nif.h], [], [])\nAC_LANG_POP([C])\nCFLAGS=$SAVECFLAGS\n\n#http://unix.stackexchange.com/questions/35183/how-do-i-identify-which-linux-distro-is-running\n\nLINUXVERSION=\"unknown linux\"\nif test -f /etc/SuSE-release ; then\n  LINUXVERSION=`cat /etc/SuSE-release`\nelif test -f /etc/redhat-release ; then\n  LINUXVERSION=`cat /etc/redhat-release`\nelif test -f /etc/fedora-release ; then\n  LINUXVERSION=`cat /etc/fedora-release`\nelif test -f /etc/debian_release ; then\n  LINUXVERSION=`cat /etc/debian_release`\nelif test -f /etc/debian_version ; then\n  LINUXVERSION=`cat /etc/debian_version`\nelif test -f /etc/gentoo-release ; then\n  LINUXVERSION=`cat /etc/gentoo-release`\nelif test -f /etc/arch-release ; then\n  LINUXVERSION=`uname -a`\nfi\n\nOS=\"\"\ncase `uname -s` in\n linux*)\n     NIFFLAGS=\"-shared\"\n     DRIVER_OS=LINUX\n     OS=$LINUXVERSION\n     ARCHIVECMD=\"$ac_cv_path_AR rv\"\n     ;;\n Linux*)\n     NIFFLAGS=\"-shared\"\n     DRIVER_OS=LINUX\n     OS=$LINUXVERSION\n     ARCHIVECMD=\"$ac_cv_path_AR rv\"\n     ;;\n Darwin*)\n     NIFFLAGS=\"-flat_namespace -undefined dynamic_lookup\"\n     DRIVER_OS=MACOSX\n     OS=\"OSX `sw_vers -productVersion`\"\n     ARCHIVECMD=\"$ac_cv_path_LIBTOOL -static -o\"\n     ;;\n darwin*)\n     NIFFLAGS=\"-flat_namespace -undefined dynamic_lookup\"\n     DRIVER_OS=MACOSX\n     OS=\"OSX `sw_vers -productVersion`\"\n     ARCHIVECMD=\"$ac_cv_path_LIBTOOL -static -o\"\n     ;;\nesac\n\nGIT=`command -v git`\nSTAT=`command -v stat`\nSHA1SUM=`command -v sha1sum`\nSRC=\"download\"\nCONFIGURESTATUS=\"\"\n\nif test \"x$GIT\" != \"x\" -a -d .git; then\n  SRC=`git log --pretty=format:'%h' -n 1 .`\n  CONFIGURESTATUS=`git status -s configure | cut -d ' ' -f 2`\nelif test \"x$GIT\" != \"x\"; then\n  CONFIGURESTATUS=`git hash-object configure`\nelif test \"x$DRIVER_OS\" = \"xMACOSX\" ; then\n  # BSD\n  SIZE=`/usr/bin/stat -f \"%z\" configure`\n  CONFIGURESTATUS=`(/usr/bin/printf \"blob $SIZE\\0\" ; /bin/cat configure) | /usr/bin/openssl sha1`\nelif test \"x$STAT\" != \"x\" -a \"x$SHA1SUM\" != \"x\" ; then\n  CONFIGURESTATUS=`(stat --printf=\"blob %s\\0\" configure; cat configure) | sha1sum -b | cut -d \" \" -f1`\nelse\n  CONFIGURESTATUS=\"\"\nfi\n\necho \"detected OS is: '$OS' $SRC $CONFIGURESTATUS\"\nAC_SUBST(DRIVER_OS)\nAC_SUBST(NIFFLAGS)\nAC_SUBST(ARCHIVECMD)\n\nmkdir -p ~/.yaws 2> /dev/null\n\nmkdir -p python3-api\n$SED -e \"s|python-api|python3-api|g\" \\\n     -e \"s|@PYTHONSITELIBDIR@|@PYTHON3SITELIBDIR@|g\" \\\n     -e \"s|PYTHON2|PYTHON3|g\" python-api/scalaris.in > python3-api/scalaris.in\n\nAC_CONFIG_FILES([Emakefile Makefile bin/scalarisctl bin/jsonclient include/rt.hrl cpp-api/Makefile java-api/scalaris java-api/scalaris-java.conf contrib/init.d/scalaris contrib/init.d/scalaris-monitor python-api/scalaris python3-api/scalaris ruby-api/scalaris])\nAC_OUTPUT\n\nchmod u+x bin/scalarisctl\nchmod u+x bin/jsonclient\nchmod u+x java-api/scalaris\nchmod u+x python-api/scalaris\nchmod u+x python3-api/scalaris\nchmod u+x ruby-api/scalaris\nchmod u+x contrib/init.d/scalaris\nchmod u+x contrib/init.d/scalaris-monitor\n\necho \"Erlang\"\nif test \"x$BITCASK_LIBS\" = \"x\" ; then\n  echo \"  Bitcask              : no\"\nelse\n  echo \"  Bitcask              : yes\"\nfi\n\nif test \"x$ERLANG_HANOIDB_FLAGS\" = \"x\" ; then\n  echo \"  hanoidb              : no\"\nelse\n  echo \"  hanoidb              : yes\"\nfi\n\nif test \"x$ERLANG_TOKE_FLAGS\" = \"x\" ; then\n  echo \"  toke                 : no\"\nelse\n  echo \"  toke                 : yes\"\nfi\n\nif test \"x$SSL_GETSTAT\" = \"xfalse\" ; then\n  echo \"  ssl                  : no\"\nelse\n  echo \"  ssl                  : yes\"\nfi\n\nif test \"x$enabled_cxx\" = \"x0\" ; then\n  echo \"C++ API                : no\"\nelse\n  echo \"C++ API                : yes\"\nfi\n\nif test \"x$ENABLEPYTHON2INSTALL\" = \"xinstall-python\" ; then\n  echo \"Python API             : yes\"\nelse\n  echo \"Python API             : no\"\nfi\n\nif test \"x$ENABLEPYTHON3\" = \"xpython3-compile\" ; then\n  echo \"Python3 API            : yes\"\nelse\n  echo \"Python3 API            : no\"\nfi\n"
  },
  {
    "path": "contrib/.gitignore",
    "content": "/*.pyc\n"
  },
  {
    "path": "contrib/4caast/blueprints/Scalaris-v01.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<bp:blueprint xmlns:bp=\"http://www.4caast.eu/blueprint\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n  xsi:schemaLocation=\"http://www.4caast.eu/blueprint blueprint_template_v01.xsd \">\n  <bp:basic_properties_section>\n    <bp:blueprint_id>Scalaris-01-Blueprint</bp:blueprint_id>\n    <bp:blueprint_name>Scalaris-01-Blueprint</bp:blueprint_name>\n    <bp:description>\n      Blueprint of Scalaris, a transactional distributed key-value store.\n    </bp:description>\n    <bp:ownership>\n      <bp:name>Zuse Institute Berlin</bp:name>\n      <bp:uri>http://www.zib.de</bp:uri>\n    </bp:ownership>\r\n    <bp:version>1.1</bp:version>\r\n    <bp:release_date>2012-07-25</bp:release_date>\r\n    <bp:isProduct>true</bp:isProduct>\n    <bp:status>unresolved</bp:status>\n  </bp:basic_properties_section>\n\n  <bp:offering_section>\n    <bp:offering>\n      <bp:offering_id>Scalaris-01</bp:offering_id>\n      <bp:resource_name>Scalaris key-value store</bp:resource_name>\n      <bp:resource_type>Database</bp:resource_type>\n      <bp:structural_interface>https://code.google.com/p/scalaris/</bp:structural_interface>\n      <bp:range_of_instance>\n        <bp:minimum>1</bp:minimum>\n        <bp:maximum>9999</bp:maximum>\n      </bp:range_of_instance>\n    </bp:offering>\n  </bp:offering_section>\n\n  <bp:deployment_artefact_section>\n    <bp:deployment_artefact>\n      <bp:artefact_id>scalaris-01-node-packages</bp:artefact_id>\n      <bp:artefact_name>scalaris node binary packages</bp:artefact_name>\n      <bp:artefact_type>binaries</bp:artefact_type>\n      <bp:artefact_location>http://download.opensuse.org/repositories/home:/scalaris/</bp:artefact_location>\n    </bp:deployment_artefact>\n  </bp:deployment_artefact_section>\n</bp:blueprint>\r\n"
  },
  {
    "path": "contrib/4caast/blueprints/Scalaris-v02.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<bp:blueprint xmlns:bp=\"http://www.4caast.eu/blueprint\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\r\n  xmlns:metering=\"http://www.4caast.eu/metering\"\r\n  xsi:schemaLocation=\"http://www.4caast.eu/blueprint blueprint_template_v01.xsd http://www.4caast.eu/metering MeteringSchema_v03.xsd\r\n  \">\n  <bp:basic_properties_section>\n    <bp:blueprint_id>Scalaris-Blueprint</bp:blueprint_id>\n    <bp:blueprint_name>Scalaris-Blueprint</bp:blueprint_name>\n    <bp:description>\n      Blueprint of Scalaris, a transactional distributed key-value store.\n    </bp:description>\n    <bp:ownership>\n      <bp:name>Zuse Institute Berlin</bp:name>\n      <bp:uri>http://www.zib.de</bp:uri>\n    </bp:ownership>\r\n    <bp:version>2</bp:version>\r\n    <bp:release_date>2012-10-22</bp:release_date>\r\n    <bp:isProduct>true</bp:isProduct>\n    <bp:status>unresolved</bp:status>\r\n    <bp:ext_property>\r\n      <bp:p_name>metering</bp:p_name>\r\n      <metering:metering_section xmlns:metering=\"http://www.4caast.eu/MeteringSchema\">\r\n        <metering:metering_instructions_time>\r\n          <metering:id>de.zib.scalaris.hours_used</metering:id>\r\n          <metering:unit>Hour</metering:unit>\r\n          <metering:unit_label>Hours the service is used</metering:unit_label>\r\n        </metering:metering_instructions_time>\r\n        <metering:metering_instructions_quantity>\r\n          <metering:id>de.zib.scalaris.memory_used</metering:id>\r\n          <metering:unit>MB</metering:unit>\r\n          <metering:unit_label>MB used</metering:unit_label>\r\n        </metering:metering_instructions_quantity>\r\n      </metering:metering_section>\r\n    </bp:ext_property>\n  </bp:basic_properties_section>\n\n  <bp:offering_section>\n    <bp:offering>\n      <bp:offering_id>Scalaris</bp:offering_id>\n      <bp:resource_name>Scalaris key-value store</bp:resource_name>\n      <bp:resource_type>Database</bp:resource_type>\n      <bp:structural_interface>https://code.google.com/p/scalaris/</bp:structural_interface>\n      <bp:range_of_instance>\n        <bp:minimum>1</bp:minimum>\n        <bp:maximum>9999</bp:maximum>\n      </bp:range_of_instance>\n    </bp:offering>\n  </bp:offering_section>\n\n  <bp:deployment_artefact_section>\n    <bp:deployment_artefact>\n      <bp:artefact_id>scalaris-node-packages</bp:artefact_id>\n      <bp:artefact_name>scalaris node binary packages</bp:artefact_name>\n      <bp:artefact_type>binaries</bp:artefact_type>\n      <bp:artefact_location>http://download.opensuse.org/repositories/home:/scalaris/</bp:artefact_location>\n    </bp:deployment_artefact>\n  </bp:deployment_artefact_section>\n</bp:blueprint>\r\n"
  },
  {
    "path": "contrib/4caast/blueprints/Scalaris-v03.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<bp:blueprint xmlns:bp=\"http://www.4caast.eu/blueprint\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\r\n  xmlns:metering=\"http://www.4caast.eu/metering\"\r\n  xmlns:monitoring=\"http://www.4caast.eu/MonitoringSchema\"\r\n  xsi:schemaLocation=\"http://www.4caast.eu/blueprint blueprint_template_v01.xsd\r\n  http://www.4caast.eu/metering MeteringSchema_v03.xsd\r\n  http://www.4caast.eu/MonitoringSchema MonitoringSchema_v01.xsd\r\n  \">\n  <bp:basic_properties_section>\n    <bp:blueprint_id>Scalaris-Blueprint</bp:blueprint_id>\n    <bp:blueprint_name>Scalaris-Blueprint</bp:blueprint_name>\n    <bp:description>\n      Blueprint of Scalaris, a transactional distributed key-value store.\n    </bp:description>\n    <bp:ownership>\n      <bp:name>Zuse Institute Berlin</bp:name>\n      <bp:uri>http://www.zib.de</bp:uri>\n    </bp:ownership>\r\n    <bp:version>3</bp:version>\r\n    <bp:release_date>2013-07-04</bp:release_date>\r\n    <bp:isProduct>true</bp:isProduct>\n    <bp:status>unresolved</bp:status>\r\n    <bp:ext_property>\r\n      <bp:p_name>metering</bp:p_name>\r\n      <metering:metering_section xmlns:metering=\"http://www.4caast.eu/MeteringSchema\">\r\n        <metering:metering_instructions_time>\r\n          <metering:id>de.zib.scalaris.hours_used</metering:id>\r\n          <metering:unit>Hour</metering:unit>\r\n          <metering:unit_label>Hours the service is used</metering:unit_label>\r\n        </metering:metering_instructions_time>\r\n        <metering:metering_instructions_quantity>\r\n          <metering:id>de.zib.scalaris.memory_used</metering:id>\r\n          <metering:unit>MB</metering:unit>\r\n          <metering:unit_label>MB used</metering:unit_label>\r\n        </metering:metering_instructions_quantity>\r\n      </metering:metering_section>\r\n    </bp:ext_property>\r\n    <bp:ext_property>\r\n      <bp:p_name>monitoring</bp:p_name>\r\n      <monitoring:monitoring_section>\r\n        <monitoring:monitoring_instruction>\r\n          <monitoring:id>ScalarisNode_CurLatencyAvg</monitoring:id>\r\n          <monitoring:metric>Load average for the last minute<!-- well, not really but there's not much of a choice! --></monitoring:metric>\r\n          <monitoring:unit>Millisecond</monitoring:unit>\r\n          <monitoring:unit_label>Average latency of transactions at the node</monitoring:unit_label>\r\n        </monitoring:monitoring_instruction>\r\n        <monitoring:monitoring_instruction>\r\n          <monitoring:id>ScalarisService_CurLatencyAvg</monitoring:id>\r\n          <monitoring:metric>Load average for the last minute<!-- well, not really but there's not much of a choice! --></monitoring:metric>\r\n          <monitoring:unit>Millisecond</monitoring:unit>\r\n          <monitoring:unit_label>Average latency of transactions aggregated over the whole system</monitoring:unit_label>\r\n        </monitoring:monitoring_instruction>\r\n        <monitoring:monitoring_instruction>\r\n          <monitoring:id>ScalarisService_LoadEstimate</monitoring:id>\r\n          <monitoring:metric>Physical memory</monitoring:metric>\r\n          <monitoring:unit>Count</monitoring:unit>\r\n          <monitoring:unit_label>Number of overall stored items (estimate)</monitoring:unit_label>\r\n        </monitoring:monitoring_instruction>\r\n      </monitoring:monitoring_section>\r\n    </bp:ext_property>\n  </bp:basic_properties_section>\n\n  <bp:offering_section>\n    <bp:offering>\n      <bp:offering_id>Scalaris</bp:offering_id>\n      <bp:resource_name>Scalaris key-value store</bp:resource_name>\n      <bp:resource_type>Database</bp:resource_type>\n      <bp:structural_interface>https://code.google.com/p/scalaris/</bp:structural_interface>\n      <bp:range_of_instance>\n        <bp:minimum>1</bp:minimum>\n        <bp:maximum>9999</bp:maximum>\n      </bp:range_of_instance>\n    </bp:offering>\n  </bp:offering_section>\n\n  <bp:deployment_artefact_section>\n    <bp:deployment_artefact>\n      <bp:artefact_id>scalaris-node-packages</bp:artefact_id>\n      <bp:artefact_name>scalaris node binary packages</bp:artefact_name>\n      <bp:artefact_type>binaries</bp:artefact_type>\n      <bp:artefact_location>http://download.opensuse.org/repositories/home:/scalaris/</bp:artefact_location>\n    </bp:deployment_artefact>\n  </bp:deployment_artefact_section>\n</bp:blueprint>\r\n"
  },
  {
    "path": "contrib/4caast/blueprints/WikiOnScalaris-v01.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<bp:blueprint xmlns:bp=\"http://www.4caast.eu/blueprint\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\r\n  xsi:schemaLocation=\"http://www.4caast.eu/blueprint blueprint_template_v03.xsd \">\r\n  <bp:basic_properties_section>\r\n    <bp:blueprint_id>WikiOnScalaris-01-Blueprint</bp:blueprint_id>\r\n    <bp:blueprint_name>WikiOnScalaris-01-Blueprint</bp:blueprint_name>\r\n    <bp:description>Blueprint for the \"Wiki on Scalaris\" application.</bp:description>\r\n    <bp:ownership>\r\n      <bp:name>Zuse Institute Berlin</bp:name>\r\n      <bp:uri>http://www.zib.de</bp:uri>\r\n    </bp:ownership>\r\n    <bp:version>1.1</bp:version>\r\n    <bp:release_date>2012-07-25</bp:release_date>\r\n    <bp:isProduct>true</bp:isProduct>\r\n    <bp:status>unresolved</bp:status>\r\n  </bp:basic_properties_section>\r\n\r\n  <bp:offering_section>\r\n    <bp:offering>\r\n      <bp:offering_id>WikiOnScalaris-01</bp:offering_id>\r\n      <bp:resource_name>Wiki on Scalaris</bp:resource_name>\r\n      <bp:resource_type>PaaS</bp:resource_type>\r\n      <bp:endpoint_location>http://localhost:8080/scalaris-wiki/</bp:endpoint_location>\r\n      <bp:range_of_instance>\r\n        <bp:minimum>1</bp:minimum>\r\n        <bp:maximum>9999</bp:maximum>\r\n      </bp:range_of_instance>\r\n    </bp:offering>\r\n  </bp:offering_section>\r\n\r\n  <bp:deployment_artefact_section>\r\n    <bp:deployment_artefact>\r\n      <bp:artefact_id>WikiOnScalaris-01-war</bp:artefact_id>\r\n      <bp:artefact_name>scalaris-wiki.war</bp:artefact_name>\r\n      <bp:artefact_type>war-file</bp:artefact_type>\r\n      <bp:artefact_location>\r\n        https://svn.forge.morfeo-project.org/4caast/trunk/WP6/datastore/demo/scalaris-wiki.war\r\n      </bp:artefact_location>\r\n    </bp:deployment_artefact>\r\n  </bp:deployment_artefact_section>\r\n\r\n  <bp:resource_requirement_section>\r\n    <bp:resource_requirement>\r\n      <bp:requirement_id>WikiOnScalaris-01-ServletContainer</bp:requirement_id>\r\n      <bp:resource_name>servlet container v2.5</bp:resource_name>\r\n      <bp:resource_type>Servlet Container v2.5</bp:resource_type>\r\n      <bp:range_of_instances>\r\n        <bp:minimum>1</bp:minimum>\r\n        <bp:maximum>9999</bp:maximum>\r\n      </bp:range_of_instances>\r\n      <bp:policy_constraint>ServletContainer.constraints</bp:policy_constraint>\r\n    </bp:resource_requirement>\r\n    <bp:resource_requirement>\r\n      <bp:requirement_id>WikiOnScalaris-01-Scalaris</bp:requirement_id>\r\n      <bp:resource_name>Running Scalaris node</bp:resource_name>\r\n      <bp:resource_type>Scalaris node</bp:resource_type>\r\n      <bp:range_of_instances>\r\n        <bp:minimum>1</bp:minimum>\r\n        <bp:maximum>9999</bp:maximum>\r\n      </bp:range_of_instances>\r\n    </bp:resource_requirement>\r\n  </bp:resource_requirement_section>\r\n\r\n  <bp:topology_section>\r\n    <bp:vertical_dependency>\r\n      <bp:source>WikiOnScalaris-01-war</bp:source>\r\n      <bp:target>WikiOnScalaris-01-ServletContainer</bp:target>\r\n    </bp:vertical_dependency>\r\n    <bp:horizontal_dependency>\r\n      <bp:source>WikiOnScalaris-01-war</bp:source>\r\n      <bp:target>WikiOnScalaris-01-Scalaris</bp:target>\r\n    </bp:horizontal_dependency>\r\n  </bp:topology_section>\r\n</bp:blueprint>\r\n"
  },
  {
    "path": "contrib/4caast/blueprints/WikiOnScalaris-v02.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<bp:blueprint xmlns:bp=\"http://www.4caast.eu/blueprint\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n  xmlns:metering=\"http://www.4caast.eu/metering\"\r\n  xsi:schemaLocation=\"http://www.4caast.eu/blueprint blueprint_template_v01.xsd http://www.4caast.eu/metering MeteringSchema_v03.xsd\n  \">\n  <bp:basic_properties_section>\n    <bp:blueprint_id>WikiOnScalaris-Blueprint</bp:blueprint_id>\n    <bp:blueprint_name>WikiOnScalaris-Blueprint</bp:blueprint_name>\n    <bp:description>Blueprint for the \"Wiki on Scalaris\" application.</bp:description>\n    <bp:ownership>\n      <bp:name>Zuse Institute Berlin</bp:name>\n      <bp:uri>http://www.zib.de</bp:uri>\n    </bp:ownership>\n    <bp:version>2</bp:version>\n    <bp:release_date>2012-10-22</bp:release_date>\n    <bp:isProduct>true</bp:isProduct>\n    <bp:status>unresolved</bp:status>\r\n    <bp:ext_property>\r\n      <bp:p_name>metering</bp:p_name>\r\n      <metering:metering_section xmlns:metering=\"http://www.4caast.eu/MeteringSchema\">\r\n        <metering:metering_instructions_event>\r\n          <metering:id>de.zib.scalaris.examples.wikipedia.pages_viewed</metering:id>\r\n          <metering:unit>Invocation</metering:unit>\r\n          <metering:unit_label>Pages viewed</metering:unit_label>\r\n        </metering:metering_instructions_event>\r\n      </metering:metering_section>\r\n    </bp:ext_property>\n  </bp:basic_properties_section>\n\n  <bp:offering_section>\n    <bp:offering>\n      <bp:offering_id>WikiOnScalaris</bp:offering_id>\n      <bp:resource_name>Wiki on Scalaris</bp:resource_name>\n      <bp:resource_type>PaaS</bp:resource_type>\n      <bp:endpoint_location>http://localhost:8080/scalaris-wiki/</bp:endpoint_location>\n      <bp:range_of_instance>\n        <bp:minimum>1</bp:minimum>\n        <bp:maximum>9999</bp:maximum>\n      </bp:range_of_instance>\n    </bp:offering>\n  </bp:offering_section>\n\n  <bp:deployment_artefact_section>\n    <bp:deployment_artefact>\n      <bp:artefact_id>WikiOnScalaris-war</bp:artefact_id>\n      <bp:artefact_name>scalaris-wiki.war</bp:artefact_name>\n      <bp:artefact_type>war-file</bp:artefact_type>\n      <bp:artefact_location>\n        https://svn.forge.morfeo-project.org/4caast/trunk/WP6/datastore/demo/scalaris-wiki.war\n      </bp:artefact_location>\n    </bp:deployment_artefact>\n  </bp:deployment_artefact_section>\n\n  <bp:resource_requirement_section>\n    <bp:resource_requirement>\n      <bp:requirement_id>WikiOnScalaris-ServletContainer</bp:requirement_id>\n      <bp:resource_name>servlet container v2.5</bp:resource_name>\n      <bp:resource_type>Servlet Container v2.5</bp:resource_type>\n      <bp:range_of_instances>\n        <bp:minimum>1</bp:minimum>\n        <bp:maximum>9999</bp:maximum>\n      </bp:range_of_instances>\n      <bp:policy_constraint>ServletContainer.constraints</bp:policy_constraint>\n    </bp:resource_requirement>\n    <bp:resource_requirement>\n      <bp:requirement_id>WikiOnScalaris-Scalaris</bp:requirement_id>\n      <bp:resource_name>Running Scalaris node</bp:resource_name>\n      <bp:resource_type>Scalaris node</bp:resource_type>\n      <bp:range_of_instances>\n        <bp:minimum>1</bp:minimum>\n        <bp:maximum>9999</bp:maximum>\n      </bp:range_of_instances>\n    </bp:resource_requirement>\n  </bp:resource_requirement_section>\n\n  <bp:topology_section>\n    <bp:vertical_dependency>\n      <bp:source>WikiOnScalaris-war</bp:source>\n      <bp:target>WikiOnScalaris-ServletContainer</bp:target>\n    </bp:vertical_dependency>\n    <bp:horizontal_dependency>\n      <bp:source>WikiOnScalaris-war</bp:source>\n      <bp:target>WikiOnScalaris-Scalaris</bp:target>\n    </bp:horizontal_dependency>\n  </bp:topology_section>\n</bp:blueprint>\r\n"
  },
  {
    "path": "contrib/4caast/blueprints/WikiOnScalaris-v03.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<bp:blueprint xmlns:bp=\"http://www.4caast.eu/blueprint\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n  xmlns:metering=\"http://www.4caast.eu/metering\"\r\n  xmlns:monitoring=\"http://www.4caast.eu/MonitoringSchema\"\r\n  xsi:schemaLocation=\"http://www.4caast.eu/blueprint blueprint_template_v01.xsd\r\n  http://www.4caast.eu/metering MeteringSchema_v03.xsd\r\n  http://www.4caast.eu/MonitoringSchema MonitoringSchema_v01.xsd\n  \">\n  <bp:basic_properties_section>\n    <bp:blueprint_id>WikiOnScalaris-Blueprint</bp:blueprint_id>\n    <bp:blueprint_name>WikiOnScalaris-Blueprint</bp:blueprint_name>\n    <bp:description>Blueprint for the \"Wiki on Scalaris\" application.</bp:description>\n    <bp:ownership>\n      <bp:name>Zuse Institute Berlin</bp:name>\n      <bp:uri>http://www.zib.de</bp:uri>\n    </bp:ownership>\n    <bp:version>3</bp:version>\n    <bp:release_date>2013-07-04</bp:release_date>\n    <bp:isProduct>true</bp:isProduct>\n    <bp:status>unresolved</bp:status>\r\n    <bp:ext_property>\r\n      <bp:p_name>metering</bp:p_name>\r\n      <metering:metering_section xmlns:metering=\"http://www.4caast.eu/MeteringSchema\">\r\n        <metering:metering_instructions_event>\r\n          <metering:id>de.zib.scalaris.examples.wikipedia.pages_viewed</metering:id>\r\n          <metering:unit>Invocation</metering:unit>\r\n          <metering:unit_label>Pages viewed</metering:unit_label>\r\n        </metering:metering_instructions_event>\r\n      </metering:metering_section>\r\n    </bp:ext_property>\r\n    <bp:ext_property>\r\n      <bp:p_name>monitoring</bp:p_name>\r\n      <monitoring:monitoring_section>\r\n        <monitoring:monitoring_instruction>\r\n          <monitoring:id>WikiOnScalaris_CurDbTime</monitoring:id>\r\n          <monitoring:metric>Load average for the last minute<!-- well, not really but there's not much of a choice! --></monitoring:metric>\r\n          <monitoring:unit>Millisecond</monitoring:unit>\r\n          <monitoring:unit_label>Time spent in DB while serving the last request</monitoring:unit_label>\r\n        </monitoring:monitoring_instruction>\r\n        <monitoring:monitoring_instruction>\r\n          <monitoring:id>WikiOnScalaris_CurRenderTime</monitoring:id>\r\n          <monitoring:metric>Load average for the last minute<!-- well, not really but there's not much of a choice! --></monitoring:metric>\r\n          <monitoring:unit>Millisecond</monitoring:unit>\r\n          <monitoring:unit_label>Time spent for rendering the page to HTML while serving the last request (without DB time)</monitoring:unit_label>\r\n        </monitoring:monitoring_instruction>\r\n        <monitoring:monitoring_instruction>\r\n          <monitoring:id>WikiOnScalaris_CurServerTime</monitoring:id>\r\n          <monitoring:metric>Load average for the last minute<!-- well, not really but there's not much of a choice! --></monitoring:metric>\r\n          <monitoring:unit>Millisecond</monitoring:unit>\r\n          <monitoring:unit_label>Time spent in server code while serving the last request (total time for the request on server side)</monitoring:unit_label>\r\n        </monitoring:monitoring_instruction>\r\n      </monitoring:monitoring_section>\r\n    </bp:ext_property>\n  </bp:basic_properties_section>\n\n  <bp:offering_section>\n    <bp:offering>\n      <bp:offering_id>WikiOnScalaris</bp:offering_id>\n      <bp:resource_name>Wiki on Scalaris</bp:resource_name>\n      <bp:resource_type>PaaS</bp:resource_type>\n      <bp:endpoint_location>http://localhost:8080/scalaris-wiki/</bp:endpoint_location>\n      <bp:range_of_instance>\n        <bp:minimum>1</bp:minimum>\n        <bp:maximum>9999</bp:maximum>\n      </bp:range_of_instance>\n    </bp:offering>\n  </bp:offering_section>\n\n  <bp:deployment_artefact_section>\n    <bp:deployment_artefact>\n      <bp:artefact_id>WikiOnScalaris-war</bp:artefact_id>\n      <bp:artefact_name>scalaris-wiki.war</bp:artefact_name>\n      <bp:artefact_type>war-file</bp:artefact_type>\n      <bp:artefact_location>\n        https://svn.forge.morfeo-project.org/4caast/trunk/WP6/datastore/demo/scalaris-wiki.war\n      </bp:artefact_location>\n    </bp:deployment_artefact>\n  </bp:deployment_artefact_section>\n\n  <bp:resource_requirement_section>\n    <bp:resource_requirement>\n      <bp:requirement_id>WikiOnScalaris-ServletContainer</bp:requirement_id>\n      <bp:resource_name>servlet container v2.5</bp:resource_name>\n      <bp:resource_type>Servlet Container v2.5</bp:resource_type>\n      <bp:range_of_instances>\n        <bp:minimum>1</bp:minimum>\n        <bp:maximum>9999</bp:maximum>\n      </bp:range_of_instances>\n      <bp:policy_constraint>ServletContainer.constraints</bp:policy_constraint>\n    </bp:resource_requirement>\n    <bp:resource_requirement>\n      <bp:requirement_id>WikiOnScalaris-Scalaris</bp:requirement_id>\n      <bp:resource_name>Running Scalaris node</bp:resource_name>\n      <bp:resource_type>Scalaris node</bp:resource_type>\n      <bp:range_of_instances>\n        <bp:minimum>1</bp:minimum>\n        <bp:maximum>9999</bp:maximum>\n      </bp:range_of_instances>\n    </bp:resource_requirement>\n  </bp:resource_requirement_section>\n\n  <bp:topology_section>\n    <bp:vertical_dependency>\n      <bp:source>WikiOnScalaris-war</bp:source>\n      <bp:target>WikiOnScalaris-ServletContainer</bp:target>\n    </bp:vertical_dependency>\n    <bp:horizontal_dependency>\n      <bp:source>WikiOnScalaris-war</bp:source>\n      <bp:target>WikiOnScalaris-Scalaris</bp:target>\n    </bp:horizontal_dependency>\n  </bp:topology_section>\n</bp:blueprint>\r\n"
  },
  {
    "path": "contrib/4caast/monitoring/README",
    "content": "=== How to use JASMINe monitoring probes with Scalaris === \n\n1) configure, build, run Scalaris\n\ncd <scalaris-dir>\n./configure && make && ./bin/firstnode.sh\n\n2) start the JMX wrapper (reads monitoring values from Scalaris and exposes these via JMX):\n\ncd java-api\n./scalaris --jvmopts \"-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=14193 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false\" -jmx firstnode@localhost\n\n=== How to use JASMINe monitoring probes with \"Wiki on Scalaris\" === \n\n1) build Wiki on Scalaris\n\ncd <scalaris-dir>/contrib/wikipedia/\nant\n\n2) start the Tomcat server with specific JMX port:\n\nant run -Dscalaris.node=\"node1@localhost\" -Dscalaris.cookie=\"chocolate chip cookie\" -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=14192 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false\n\n=== Setup JASMINe monitoring probes === \n\n3) download, extract, configure JASMINe probe standalone\n\nwget http://download.forge.objectweb.org/jasmine/jasmine-probe-standalone-1.1.1.zip\nunzip jasmine-probe-standalone-1.1.1.zip\n\n3a)\ncp <scalaris-dir>/contrib/4caast/probe-config.xml jasmine-probe-standalone-1.1.1/conf/probe-config.xml\n\n3b)\ncp <scalaris-dir>/contrib/4caast/probe-config.wiki.xml jasmine-probe-standalone-1.1.1/conf/probe-config.xml\n\n4) run JASMINe probe standalone\n\ncd jasmine-probe-standalone-1.1.1\nexport JPROBE_ROOT=$PWD\n./jasmine-probe.sh start\n\n5) stop JASMINe probe standalone when finished\n\n./jasmine-probe.sh stop\n"
  },
  {
    "path": "contrib/4caast/monitoring/probe-config.wiki.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<!--  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n - Copyright 2012 Zuse Institute Berlin\n - Contact: kruber@zib.de\n -\n - Licensed under the Apache License, Version 2.0 (the \"License\");\n - you may not use this file except in compliance with the License.\n - You may obtain a copy of the License at\n -\n -     http://www.apache.org/licenses/LICENSE-2.0\n -\n - Unless required by applicable law or agreed to in writing, software\n - distributed under the License is distributed on an \"AS IS\" BASIS,\n - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n - See the License for the specific language governing permissions and\n - limitations under the License.\n - - - - - - -w - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n-->\n<probe-config xmlns=\"org.ow2.jasmine.probe:probe-config\"\n              xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n              xsi:schemaLocation=\"org.ow2.jasmine.probe:probe-config\n                                  http://jasmine.ow2.org/xsds/probe-config-1.4.xsd\">\n\n  <!-- outputs -->\n  <output name=\"stdio\" type=\"console\" default=\"true\" />\n  <output name=\"log1\" type=\"file\">\n    <property key=\"path\" value=\"wiki-on-scalaris.csv\"/>\n  </output>\n\n  <!-- Wiki on Scalaris Java JMX target -->\n  <target name=\"wiki-on-scalaris\" type=\"jmx\">\n    <property key=\"url\" value=\"service:jmx:rmi:///jndi/rmi://localhost:14192/jmxrmi\"/>\n  </target>\n\n  <!-- Wiki on Scalaris Java JMX node indicators -->\n  <indicator name=\"WikiOnScalaris_CurDbTime\" type=\"jmx\">\n    <property key=\"target\" value=\"wiki-on-scalaris\"/>\n    <property key=\"mbean\" value=\"de.zib.scalaris:type=FourCaastMonitoringMBean\"/>\n    <property key=\"attr\" value=\"LastDbTime\"/>\n  </indicator>\n  <indicator name=\"WikiOnScalaris_CurRenderTime\" type=\"jmx\">\n    <property key=\"target\" value=\"wiki-on-scalaris\"/>\n    <property key=\"mbean\" value=\"de.zib.scalaris:type=FourCaastMonitoringMBean\"/>\n    <property key=\"attr\" value=\"LastRenderTime\"/>\n  </indicator>\n  <indicator name=\"WikiOnScalaris_CurServerTime\" type=\"jmx\">\n    <property key=\"target\" value=\"wiki-on-scalaris\"/>\n    <property key=\"mbean\" value=\"de.zib.scalaris:type=FourCaastMonitoringMBean\"/>\n    <property key=\"attr\" value=\"LastServerTime\"/>\n  </indicator>\n\n  <!-- Wiki on Scalaris Java JMX probe -->\n  <probe id=\"WikiOnScalarisProbe\" status=\"started\">\n    <output>stdio</output>\n    <output>log1</output>\n    <indicator>WikiOnScalaris_CurDbTime</indicator>\n    <indicator>WikiOnScalaris_CurRenderTime</indicator>\n    <indicator>WikiOnScalaris_CurServerTime</indicator>\n  </probe>\n</probe-config>\n"
  },
  {
    "path": "contrib/4caast/monitoring/probe-config.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<!--  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n - Copyright 2012 Zuse Institute Berlin\n - Contact: kruber@zib.de\n -\n - Licensed under the Apache License, Version 2.0 (the \"License\");\n - you may not use this file except in compliance with the License.\n - You may obtain a copy of the License at\n -\n -     http://www.apache.org/licenses/LICENSE-2.0\n -\n - Unless required by applicable law or agreed to in writing, software\n - distributed under the License is distributed on an \"AS IS\" BASIS,\n - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n - See the License for the specific language governing permissions and\n - limitations under the License.\n - - - - - - -w - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -\n-->\n<probe-config xmlns=\"org.ow2.jasmine.probe:probe-config\"\n              xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n              xsi:schemaLocation=\"org.ow2.jasmine.probe:probe-config\n                                  http://jasmine.ow2.org/xsds/probe-config-1.4.xsd\">\n\n  <!-- outputs -->\n  <output name=\"stdio\" type=\"console\" default=\"true\" />\n  <output name=\"log1\" type=\"file\">\n    <property key=\"path\" value=\"scalaris.csv\"/>\n  </output>\n\n  <!-- Scalaris Java JMX target -->\n  <target name=\"scalaris\" type=\"jmx\">\n    <property key=\"url\" value=\"service:jmx:rmi:///jndi/rmi://localhost:14193/jmxrmi\"/>\n  </target>\n\n  <!-- Scalaris Java JMX node indicators -->\n  <indicator name=\"ScalarisNode_CurLatencyAvg\" type=\"jmx\">\n    <property key=\"target\" value=\"scalaris\"/>\n    <property key=\"mbean\" value=\"de.zib.scalaris:type=MonitorNode\"/>\n    <property key=\"attr\" value=\"CurLatencyAvg\"/>\n  </indicator>\n  <indicator name=\"ScalarisNode_CurLatencyStddev\" type=\"jmx\">\n    <property key=\"target\" value=\"scalaris\"/>\n    <property key=\"mbean\" value=\"de.zib.scalaris:type=MonitorNode\"/>\n    <property key=\"attr\" value=\"CurLatencyStddev\"/>\n  </indicator>\n  \n  <!-- Scalaris Java JMX service indicators -->\n  <indicator name=\"ScalarisService_CurLatencyAvg\" type=\"jmx\">\n    <property key=\"target\" value=\"scalaris\"/>\n    <property key=\"mbean\" value=\"de.zib.scalaris:type=MonitorService\"/>\n    <property key=\"attr\" value=\"CurLatencyAvg\"/>\n  </indicator>\n  <indicator name=\"ScalarisService_CurLatencyStddev\" type=\"jmx\">\n    <property key=\"target\" value=\"scalaris\"/>\n    <property key=\"mbean\" value=\"de.zib.scalaris:type=MonitorService\"/>\n    <property key=\"attr\" value=\"CurLatencyStddev\"/>\n  </indicator>\n  <indicator name=\"ScalarisService_LoadEstimate\" type=\"jmx\">\n    <property key=\"target\" value=\"scalaris\"/>\n    <property key=\"mbean\" value=\"de.zib.scalaris:type=MonitorService\"/>\n    <property key=\"attr\" value=\"TotalLoad\"/>\n  </indicator>\n\n  <!-- Scalaris Java JMX probe -->\n  <probe id=\"ScalarisProbe\" status=\"started\">\n    <output>stdio</output>\n    <output>log1</output>\n    <indicator>ScalarisNode_CurLatencyAvg</indicator>\n    <indicator>ScalarisNode_CurLatencyStddev</indicator>\n    <indicator>ScalarisService_CurLatencyAvg</indicator>\n    <indicator>ScalarisService_CurLatencyStddev</indicator>\n    <indicator>ScalarisService_LoadEstimate</indicator>\n  </probe>\n</probe-config>\n"
  },
  {
    "path": "contrib/benchmark/bench_tt",
    "content": "#!/bin/bash\n#######################################\n#PBS -N [% name %]_[% server %]_4_de\n#PBS -l nodes=[% server %]:ppn=4\n#PBS -l walltime=8:00:00\n#PBS -q gbe \n#PBS -M hennig@zib.de \n\n#init\nsource /etc/profile\nsource $HOME/.bashrc\n\n#echo \"running on $HOSTNAME\"\nUUID=`uuidgen`\nSERVER=[% server %]\nRESULT_DIR=[% resdir %]\n\nITERATIONS_PER_SERVER=20000\n\nSTART_TIME=`date +%m%d%y%H%M%S`\ncp -r  [% scalaris %] ~/$START_TIME-$SERVER-$UUID\nWORKING_DIR=~/$START_TIME-$SERVER-$UUID/bin\n\ncd $WORKING_DIR\n\nLOG_FILE=\"bench_log_de-$SERVER-$START_TIME\"\nLOG_FILE_CLEAN=\"bench_sum_de-$SERVER-$START_TIME\"\n#HOSTS=\"localhost\"\nHOSTS=`cat $PBS_NODEFILE |  sort -u`\nfor host in $HOSTS\ndo\n        ssh $host \"killall -9 bench_master.sh\"\n        ssh $host \"killall -9 bench_slave.sh\"\n        ssh $host \"killall -9 beam.smp \"\ndone \nVMS_PER_SERVER_LIST=\"1 2 4\"\nCLIENTS_PER_SERVER_LIST=\"1 2 5 10 50 100 200\"\nCSNODES_PER_SERVER_LIST=\"4 16 32\"\n\ndate\nfor CLIENTS_PER_SERVER in $CLIENTS_PER_SERVER_LIST\ndo\nfor VMS_PER_SERVER in $VMS_PER_SERVER_LIST\ndo\n  for CSNODES_PER_SERVER in $CSNODES_PER_SERVER_LIST\n  do\n        \n\tNODES_VM=$((CSNODES_PER_SERVER/VMS_PER_SERVER))\n\tCLIENTS_PER_VM=$((CLIENTS_PER_SERVER/VMS_PER_SERVER))\n        ITERATIONS_PER_CLIENT=$((ITERATIONS_PER_SERVER/(CLIENTS_PER_SERVER)))\n\t#ITERATION=$((ITERATIONS/(VMS_PER_SERVER*SERVER)))\n\tif [ $NODES_VM -gt 0 ]; then\n\tif [ $CLIENTS_PER_VM -gt 0 ]; then \n\t        i=0\n\t        RING_SIZE=$((SERVER*VMS_PER_SERVER*NODES_VM))\n\t        echo \"######################\"\n\t        echo \"RS $RING_SIZE VPS $VMS_PER_SERVER NPS $CSNODES_PER_SERVER NPV $NODES_VM C: $CLIENTS_PER_SERVER IT: $ITERATIONS_PER_CLIENT\"\n\t\t    for host in $HOSTS\n\t        do\n\t                for vm in `seq 1 $VMS_PER_SERVER`\n\t                do\n\t                        i=$((i+1))\n\t                        \n\t                        case \"$i\" in\n\t                                1 )     BOOTIP=`/sbin/ifconfig  eth1 | grep inet\\ | awk '{ print $2 }' | cut -c 6- | sed 's/\\./,/g'`\n                              \t\t\t\t\t\tcd $WORKING_DIR\n[% IF cl == 'tcp' %]  \n                                          echo \"{boot_host,{{$BOOTIP},14195,boot}}.\" > scalaris.local.cfg\n                                          echo \"{log_host,{{$BOOTIP},14195,boot_logger}}.\" >> scalaris.local.cfg\n[% ELSE %]\n                                          echo \"{boot_host, {boot,'boot@$host.1001.zib.de'}}.\" > scalaris.local.cfg\n\t                                        echo \"{log_host,{boot_logger,'boot@$host.1001.zib.de'}}.\" >> scalaris.local.cfg\n[% END %]\n\t\t\t\t\t                                echo \"####################################################################################\" >> $LOG_FILE\n\t                                        echo \"SV: $SERVER RS $RING_SIZE VPS $VMS_PER_SERVER NPS $CSNODES_PER_SERVER NPV $NODES_VM C: $CLIENTS_PER_SERVER IT: $ITERATIONS_PER_CLIENT\" >> $LOG_FILE\n                              \t\t\t\t\t\tssh $host \" cd ~/$START_TIME-$SERVER-$UUID/bin ; ./bench_master.sh $NODES_VM $CLIENTS_PER_VM $ITERATIONS_PER_CLIENT $RING_SIZE >> $LOG_FILE\" &\n                              \t\t\t\t\t\tBOOTPID=$! \n\t\t\t\t\t                                ;;\n\t                                * )     ssh $host \"cd ~/$START_TIME-$SERVER-$UUID/bin ; ./bench_slave.sh  $NODES_VM $i >> log_$host-$i\"  & ;;\n\t                        esac\n\t\n\t                done\n\t        done\n\t\t\n\t\teval \"sleep 1000 ; kill  $BOOTPID \" &\t\t\n\t    \twait $BOOTPID\n\t\tif [ $? -eq 143 ]; then\n\t\t\techo \"WARNING bench_master.sh reached  timeout of 1000 secs \"\n\t\tfi\n                killall sleep\n\t\tsleep 1\n\t\tfor host in $HOSTS \n\t\tdo\n\t\t    ssh $host \"killall -9 bench_master.sh\"\n\t\t    ssh $host \"killall -9 bench_slave.sh \"\n\t\t    ssh $host \"killall beam.smp \"\n                    ssh $host \"killall epmd\"\n\t\tdone \n\t\tsleep 5\n\tfi\n\tfi\n  done\ndone\ndone\ncat $LOG_FILE | egrep 1\\/s\\|NPV | awk '{ if($1 == \"SV:\") {  b = $0;  x=1} if($1 == \"1/s:\") {  b = b \" \" $0;  x++}if($1 == \"1/s:\") {  b = b \" \" $0;  x++}if(x==3) print b}' | sort -r -n -k 16 > $LOG_FILE_CLEAN\necho \"Best config for your System:\"\nhead -n1 $LOG_FILE_CLEAN\ncp  $LOG_FILE  $RESULT_DIR\ncp   $LOG_FILE_CLEAN   $RESULT_DIR\nrm -rf  ~/$START_TIME-$SERVER-$UUID\ndate\n\n#XPBS -o testjob.out\n\n"
  },
  {
    "path": "contrib/benchmark/clean.sh",
    "content": "LOG_FILE=$1\nLOG_FILE_CLEAN=`echo $1 | sed 's/log/sum/g'`\ncat $LOG_FILE | egrep 1\\/s\\|NPV | xargs -n 18 | sort -r -n -k 16 > $LOG_FILE_CLEAN\n"
  },
  {
    "path": "contrib/benchmark/cluster_run.pl",
    "content": "#!/usr/bin/perl\nif(@ARGV < 3) {\n  print \"usage: $0 [revision|HEAD] [tcp] Projectname \\n\";\n  exit;\n}\n$rev = $ARGV[0];\n$cl = $ARGV[1];\n$name = $ARGV[2].\"-\".$cl.\"-\".$rev.\"-\".`date +%m%d%y%H%M%S`; \nchomp($name);\nuse Template;\nmy @Servers = (1,2,5,10,20);\nmy $resdir = `pwd`;\nchomp($resdir);\n$resdir.=\"/$name\";\nsystem \"mkdir $resdir\";\nsystem \"git clone https://github.com/scalaris-team/scalaris.git $resdir/scalaris-read-only && cd $resdir/scalaris-read-only/ && git checkout $rev\";\n#build scalairs\nsystem \"cd  $resdir/scalaris-read-only/ ; ./configure\";\nsystem \"cd  $resdir/scalaris-read-only/ ; make\";\nsystem \"cd  $resdir/scalaris-read-only/bin ; chmod u+x bench_master.sh ; chmod u+x bench_slave.sh \" ;\nmy $runfile= $resdir.\"/qsub.sh\";\nopen(RUNFILE,\">$runfile\");\n$resdir =~ s/NFS3/NFS4/g ;\nforeach  my $s (@Servers) {\n    my $tt = Template->new;\n    $tt->process('bench_tt',  { server => $s , resdir => $resdir, cl => $cl , scalaris => $resdir.\"/scalaris-read-only/\" , name => $ARGV[2].\"-\".$cl.\"-\".$rev },  $name.\"/bench_\".$s.\"_run\")\n    \t\t|| die $tt->error;\n   print(RUNFILE \"qsub bench_\".$s.\"_run\\n\");\n}\n\n"
  },
  {
    "path": "contrib/benchmark/latex.rb",
    "content": "#!/usr/bin/ruby\n\nrequire 'erb'\nrequire 'set'\n\nclass Run\n  attr_reader :servers\n  attr_reader :vms_per_server\n  attr_reader :nodes_per_server\n  attr_reader :clients_per_server\n  attr_reader :iterations_per_server\n  attr_reader :reads\n  attr_reader :increments\n\n  def initialize(servers, vms_per_server, nodes_per_server, clients_per_server,\n                 iterations_per_server, reads, increments)\n    @servers = servers\n    @vms_per_server = vms_per_server\n    @nodes_per_server = nodes_per_server\n    @clients_per_server = clients_per_server\n    @iterations_per_server = iterations_per_server\n    @reads = reads\n    @increments = increments\n  end\n\n  def to_gnuplot\n    \"#{servers} #{vms_per_server} #{nodes_per_server} #{clients_per_server} #{iterations_per_server} #{reads} #{increments}\"\n  end\nend\n\nclass ServerGroup\n  attr_reader :servers\n  attr_reader :runs\n\n  def initialize(servers)\n    @servers = servers.to_i\n    @runs = Set.new\n  end\n\n  def add(run)\n    @runs.add(run)\n  end\n\n  def top_read(count)\n    sorted_runs = @runs.sort {|x,y| y.reads <=> x.reads}\n    sorted_runs[0, count]\n  end\n\n  def top_increment(count)\n    sorted_runs = @runs.sort {|x,y| y.increments <=> x.increments}\n    sorted_runs[0, count]\n  end\n\n  def <=>(other)\n    @servers <=> other.servers\n  end\nend\n\nif ARGV.length != 1\n  puts \"latex.rb <logfile>\"\n  exit\nend\n\nfn = ARGV[0]\n\nservers = Set.new\nruns = Set.new\nIO.foreach(fn) {|line|\n  elements = line.split(' ')\n  server_count = elements[1].to_i \n  clients_per_server = elements[11].to_i\n  iterations_per_server = elements[13].to_f * clients_per_server\n  reads = elements[17].to_f\n  increments = elements[15].to_f\n  if server_count > 100\n    puts \"#{elements[1]} #{elements[11]}\"\n    puts line\n  end\n  servers.add(server_count)\n  runs.add(Run.new(server_count, elements[5].to_i, elements[7].to_i, clients_per_server,\n                   iterations_per_server, reads, increments))\n}\n\nservergroups = Hash.new\nservers.each {|server_count|\n  servergroups[server_count] = ServerGroup.new(server_count)\n}\n\nmin_server = servers.min\nmax_server = servers.max\n\n# create groups per server_count\nf = File.new(\"all.data\", \"w+\")\nruns.each {|run|\n  servergroups[run.servers].add(run)\n  f.puts(run.to_gnuplot)\n}\nf.close\n\n#top 1\nr = File.new(\"top1read.data\", \"w+\")\nw = File.new(\"top1write.data\", \"w+\")\nservers.to_a.sort.each {|server_count|\n  servergroups[server_count].top_read(1).each {|run|\n    r.puts(run.to_gnuplot)\n  }\n  servergroups[server_count].top_increment(1).each {|run|\n    w.puts(run.to_gnuplot)\n  }\n}\nr.close\nw.close\n\ntemplate = File.read('plotall.gnuplot.erb')\neruby = ERB.new(template)\nf = File.new(\"plotall.gnuplot\", \"w+\")\nf.puts eruby.result(binding())\nf.close\n\nsystem \"gnuplot plotall.gnuplot\"\n\n\n# create gnuplot files\nservers.each {|server_count|\n  f = File.new(\"#{server_count}.data\", \"w+\")\n  servergroups[server_count].runs.each {|run|\n    f.puts(run.to_gnuplot)\n  }\n  f.close\n\n  template = File.read('plot.gnuplot.erb')\n  eruby = ERB.new(template)\n  f = File.new(\"plot.gnuplot\", \"w+\")\n  f.puts eruby.result(binding())\n  f.close\n\n  system \"gnuplot plot.gnuplot\"\n}\n\nfn =~ %r{sum-test.run-(\\d+)-\\d+}\n#gitinfo = `git log --pretty=format:'%h' -n 1`\ngitinfo = $1\ntemplate = File.read('main.tex.erb')\neruby = ERB.new(template)\nf = File.new(\"main.tex\", \"w+\")\nf.puts eruby.result(binding())\nf.close\n\nsystem \"rm main.toc main.aux main.log\"\nsystem \"pdflatex main.tex > texlog.txt\"\nsystem \"pdflatex main.tex >> texlog.txt\"\nsystem \"mv main.pdf #{fn + '.pdf'}\"\nputs \"wrote results to #{fn + '.pdf'}\"\n"
  },
  {
    "path": "contrib/benchmark/main.tex.erb",
    "content": "\\documentclass[11pt]{article}\n\\usepackage{color,graphicx}\n\\usepackage{keyval}\n\\usepackage{float}\n\\usepackage{tikz}\n\\usetikzlibrary{patterns}\n\\usepackage{listings}\n\\usepackage[pdftex,\n        colorlinks=true,\n        %urlcolor=rltblue,       % \\href{...}{...} external (URL)\n        citecolor=green,\n        filecolor=rltgreen,     % \\href{...} local file\n        linkcolor=black,      % \\ref{...} and \\pageref{...}\n%        pagebackref=true,\n        bookmarksopen=true]{hyperref}\n\n\\title{Scalaris Benchmark Report for <%= gitinfo %>}\n\n\\begin{document}\n\\maketitle\n\n\\setcounter{tocdepth}{1}\n\\tableofcontents\n\n\\section{All}\n\n\\begin{tabular}{cc}\n\\includegraphics[scale=0.5]{gnuplot/all_reads.pdf} &\n\\includegraphics[scale=0.5]{gnuplot/all_increments.pdf} \\\\\n\\end{tabular}\n\n\\begin{tabular}{cc}\n\\includegraphics[scale=0.5]{gnuplot/top1_reads.pdf} &\n\\includegraphics[scale=0.5]{gnuplot/top1_increments.pdf} \\\\\n\\end{tabular}\n\n\n<% servergroups.values.sort.each do |servergroup| %>\n\n\\section{Servers: <%= servergroup.servers %>}\n\n\\subsection{Top 5}\n\nRead\n\n\\begin{tabular}{|l|l|l|l|l|l|}\n\\hline\nVMs/   & Nodes/ & Clients/ & Iterations/ & Read/s & Increments/s\\\\\nServer & Server & Server   & Server      &        &              \\\\\n\\hline\n<% servergroup.top_read(5).each {|run| %>\n<%= run.vms_per_server %> & <%= run.nodes_per_server %> & <%= run.clients_per_server %> & <%= run.iterations_per_server %> & <%= run.reads %> & <%= run.increments %>\\\\\n\\hline\n<% } %>\n\\end{tabular}\n\nIncrement\n\n\\begin{tabular}{|l|l|l|l|l|l|}\n\\hline\nVMs/   & Nodes/ & Clients/ & Iterations/ & Reads/s & Increments/s\\\\\nServer & Server & Server   & Server      &        &              \\\\\n\\hline\n<% servergroup.top_increment(5).each {|run| %>\n<%= run.vms_per_server %> & <%= run.nodes_per_server %> & <%= run.clients_per_server %> & <%= run.iterations_per_server %> & <%= run.reads %> & <%= run.increments %>\\\\\n\\hline\n<% } %>\n\\end{tabular}\n\n\\subsection{Graphs: Read}\n\n\\begin{tabular}{cc}\n\\includegraphics[scale=0.5]{gnuplot/<%= servergroup.servers %>_over_clients_reads.pdf} &\n\\includegraphics[scale=0.5]{gnuplot/<%= servergroup.servers %>_over_iterations_reads.pdf} \\\\\n\\includegraphics[scale=0.5]{gnuplot/<%= servergroup.servers %>_over_nodes_reads.pdf} &\n\\includegraphics[scale=0.5]{gnuplot/<%= servergroup.servers %>_over_vms_reads.pdf} \\\\\n\\end{tabular}\n\n\\subsection{Graphs: Increments}\n\n\\begin{tabular}{cc}\n\\includegraphics[scale=0.5]{gnuplot/<%= servergroup.servers %>_over_clients_increments.pdf} &\n\\includegraphics[scale=0.5]{gnuplot/<%= servergroup.servers %>_over_iterations_increments.pdf} \\\\\n\\includegraphics[scale=0.5]{gnuplot/<%= servergroup.servers %>_over_nodes_increments.pdf} &\n\\includegraphics[scale=0.5]{gnuplot/<%= servergroup.servers %>_over_vms_increments.pdf} \\\\\n\\end{tabular}\n\n<% end%>\n\n\\end{document}\n"
  },
  {
    "path": "contrib/benchmark/plot.gnuplot.erb",
    "content": "set terminal pdf\n\nset output \"gnuplot/<%= server_count %>_over_vms_reads.pdf\"\nset xlabel \"VMs/server\"\nset ylabel \"reads/s\"\nplot \"<%= server_count %>.data\" using 2:6\n\nset output \"gnuplot/<%= server_count %>_over_nodes_reads.pdf\"\nset xlabel \"Nodes/server\"\nset ylabel \"reads/s\"\nplot \"<%= server_count %>.data\" using 3:6\n\nset output \"gnuplot/<%= server_count %>_over_clients_reads.pdf\"\nset xlabel \"Clients/server\"\nset ylabel \"reads/s\"\nplot \"<%= server_count %>.data\" using 4:6\n\nset output \"gnuplot/<%= server_count %>_over_iterations_reads.pdf\"\nset xlabel \"iterations/server\"\nset ylabel \"reads/s\"\nplot \"<%= server_count %>.data\" using 5:6\n\nset output \"gnuplot/<%= server_count %>_over_vms_increments.pdf\"\nset xlabel \"VMs/server\"\nset ylabel \"increments/s\"\nplot \"<%= server_count %>.data\" using 2:7\n\nset output \"gnuplot/<%= server_count %>_over_nodes_increments.pdf\"\nset xlabel \"Nodes/server\"\nset ylabel \"increments/s\"\nplot \"<%= server_count %>.data\" using 3:7\n\nset output \"gnuplot/<%= server_count %>_over_clients_increments.pdf\"\nset xlabel \"Clients/server\"\nset ylabel \"increments/s\"\nplot \"<%= server_count %>.data\" using 4:7\n\nset output \"gnuplot/<%= server_count %>_over_iterations_increments.pdf\"\nset xlabel \"iterations/server\"\nset ylabel \"increments/s\"\nplot \"<%= server_count %>.data\" using 5:7\n\n"
  },
  {
    "path": "contrib/benchmark/plotall.gnuplot.erb",
    "content": "set terminal pdf\n\nset xrange [<%= min_server - 1%>:<%= max_server + 1%>]\nset yrange [0:]\n\nset output \"gnuplot/all_reads.pdf\"\nset xlabel \"server\"\nset ylabel \"reads/s\"\nplot \"all.data\" using 1:6 title \"Reads/s\"\n\nset output \"gnuplot/all_increments.pdf\"\nset xlabel \"server\"\nset ylabel \"increments/s\"\nplot \"all.data\" using 1:7 title \"Increments/s\"\n\n\n\n\n\n\nset output \"gnuplot/top1_reads.pdf\"\nset xlabel \"server\"\nset ylabel \"reads/s\"\nplot \"top1read.data\" using 1:6 title \"Reads/s\" with lines\n\nset output \"gnuplot/top1_increments.pdf\"\nset xlabel \"server\"\nset ylabel \"increments/s\"\nplot \"top1write.data\" using 1:7 title \"Increments/s\" with lines\n\n\n\n\n\n"
  },
  {
    "path": "contrib/benchmark/run_local_bench.sh",
    "content": "#!/bin/bash\n\nRUN=$1\nsource $RUN\n\n\ncd ../../bin/\n\nSERVER=1\n\nSTART_TIME=`date +%m%d%y%H%m%S`\nREV=`git log --pretty=format:'%h' -n 1`\nLOG_FILE=\"../contrib/benchmark/log-$RUN-$REV-$START_TIME\"\nLOG_FILE_CLEAN=\"../contrib/benchmark/sum-$RUN-$REV-$START_TIME\"\nHOSTS=\"localhost\"\n\n\ndate\nfor CLIENTS_PER_SERVER in $CLIENTS_PER_SERVER_LIST\ndo\nfor VMS_PER_SERVER in $VMS_PER_SERVER_LIST\ndo\n  for CSNODES_PER_SERVER in $CSNODES_PER_SERVER_LIST\n  do\n        \n\tNODES_VM=$((CSNODES_PER_SERVER/VMS_PER_SERVER))\n\tCLIENTS_PER_VM=$((CLIENTS_PER_SERVER/VMS_PER_SERVER))\n        ITERATIONS_PER_CLIENT=$((ITERATIONS_PER_SERVER/(CLIENTS_PER_SERVER)))\n\t#ITERATION=$((ITERATIONS/(VMS_PER_SERVER*SERVER)))\n\tif [ $NODES_VM -gt 0 ]; then\n        if [ $CLIENTS_PER_VM -gt 0 ]; then\n\t        i=0\n\t        RING_SIZE=$((SERVER*VMS_PER_SERVER*NODES_VM))\n\t        echo \"######################\"\n\t        echo \"RS $RING_SIZE VPS $VMS_PER_SERVER NPS $CSNODES_PER_SERVER NPV $NODES_VM C: $CLIENTS_PER_SERVER IT: $ITERATIONS_PER_CLIENT\"\n\t\t    for host in $HOSTS\n\t        do\n\t                for vm in `seq 1 $VMS_PER_SERVER`\n\t                do\n\t                        i=$((i+1))\n\t                        \n\t                        case \"$i\" in\n\t                                1 )     BOOTIP=`/sbin/ifconfig  eth0 | grep inet\\ | awk '{ print $2 }' | cut -c 6- | sed 's/\\./,/g'`\n\t                                        echo \"{boot_host, {{$BOOTIP},14195,boot}}.\" > scalaris.local.cfg\n\t                                        echo \"{log_host,{{$BOOTIP},14195,boot_logger}}.\" >> scalaris.local.cfg\n\t\t\t\t\t\t                              echo \"####################################################################################\" >> $LOG_FILE\n\t                                        echo \"SV: $SERVER RS $RING_SIZE VPS $VMS_PER_SERVER NPS $CSNODES_PER_SERVER NPV $NODES_VM C: $CLIENTS_PER_SERVER IT: $ITERATIONS_PER_CLIENT\" >> $LOG_FILE\n\t\t\t\t\t\t                              ./bench_master.sh $NODES_VM $CLIENTS_PER_VM $ITERATIONS_PER_CLIENT $RING_SIZE >> $LOG_FILE &\n\t\t\t\t\t\t                              BOOTPID=$! \n\t\t\t\t\t\t                              ;;\n\t                                * )     ./bench_slave.sh  $NODES_VM $i   >> log_$host-$i & ;;\n\t                        esac\n\t\n\t                done\n\t        done\n\t    wait $BOOTPID\n\t\tkillall -9 bench_slave.sh\n\t\tkillall -9 beam.smp\n\t\tsleep 1 \n\tfi\n        fi\n  done\ndone\ndone\ncat $LOG_FILE | egrep 1\\/s\\|NPV | awk '{ if($1 == \"SV:\") {  b = $0;  x=1} if($1 == \"1/s:\") {  b = b \" \" $0;  x++}if($1 == \"1/s:\") {  b = b \" \" $0;  x++}if(x==3) print b}' | sort -r -n -k 16 > $LOG_FILE_CLEAN\necho \"Best config for your System:\"\nhead -n1 $LOG_FILE_CLEAN\ndate\n"
  },
  {
    "path": "contrib/benchmark/simple.run",
    "content": "\n\nITERATIONS_PER_SERVER=5000\nVMS_PER_SERVER_LIST=\"1 2\"\nCLIENTS_PER_SERVER_LIST=\"1 4 8 16 32 64\"\nCSNODES_PER_SERVER_LIST=\"1 2 3 5 8 10 32\"\n\n"
  },
  {
    "path": "contrib/chef/apt/README.md",
    "content": "Description\r\n===========\r\n\r\nConfigures various APT components on Debian-like systems.  Also includes a LWRP.\r\n\r\nRecipes\r\n=======\r\n\r\ndefault\r\n-------\r\nThe default recipe runs apt-get update during the Compile Phase of the Chef run to ensure that the system's package cache is updated with the latest. It is recommended that this recipe appear first in a node's run list (directly or through a role) to ensure that when installing packages, Chef will be able to download the latest version available on the remote APT repository.\r\n\r\nThis recipe also sets up a local cache directory for preseeding packages.\r\n\r\ncacher\r\n------\r\nInstalls the apt-cacher package and service so the system can provide APT caching. You can check the usage report at http://{hostname}:3142/report. The cacher recipe includes the `cacher-client` recipe, so it helps seed itself.\r\n\r\ncacher-client\r\n-------------\r\nConfigures the node to use the apt-cacher server as a client.\r\n\r\nResources/Providers\r\n===================\r\n\r\nThis LWRP provides an easy way to manage additional APT repositories.\r\n\r\n# Actions\r\n\r\n- :add: creates a repository file and builds the repository listing\r\n- :remove: removes the repository file\r\n\r\n# Attribute Parameters\r\n\r\n- repo_name: name attribute. The name of the channel to discover\r\n- uri: the base of the Debian distribution\r\n- distribution: this is usually your release's codename...ie something like `karmic`, `lucid` or `maverick`\r\n- components: package groupings..when it doubt use `main`\r\n- deb_src: whether or not to add the repository as a source repo as well\r\n- key_server: the GPG keyserver where the key for the repo should be retrieved\r\n- key: if a `key_server` is provided, this is assumed to be the fingerprint, otherwise it is the URI to the GPG key for the repo\r\n\r\n# Example\r\n\r\n    # add the Zenoss repo\r\n    apt_repository \"zenoss\" do\r\n      uri \"http://dev.zenoss.org/deb\"\r\n      components [\"main\",\"stable\"]\r\n      action :add\r\n    end\r\n    \r\n    # add the Nginx PPA; grab key from keyserver\r\n    apt_repository \"nginx-php\" do\r\n      uri \"http://ppa.launchpad.net/nginx/php5/ubuntu\"\r\n      distribution node['lsb']['codename']\r\n      components [\"main\"]\r\n      keyserver \"keyserver.ubuntu.com\"\r\n      key \"C300EE8C\"\r\n      action :add\r\n    end\r\n    \r\n    # add the Cloudkick Repo\r\n    apt_repository \"cloudkick\" do\r\n      uri \"http://packages.cloudkick.com/ubuntu\"\r\n      distribution node['lsb']['codename']\r\n      components [\"main\"]\r\n      key \"http://packages.cloudkick.com/cloudkick.packages.key\"\r\n      action :add\r\n    end\r\n    \r\n    # remove Zenoss repo\r\n    apt_repository \"zenoss\" do\r\n      action :remove\r\n    end\r\n    \r\nUsage\r\n=====\r\n\r\nPut `recipe[apt]` first in the run list. If you have other recipes that you want to use to configure how apt behaves, like new sources, notify the execute resource to run, e.g.:\r\n\r\n    template \"/etc/apt/sources.list.d/my_apt_sources.list\" do\r\n      notifies :run, resources(:execute => \"apt-get update\"), :immediately\r\n    end\r\n\r\nThe above will run during execution phase since it is a normal template resource, and should appear before other package resources that need the sources in the template.\r\n\r\nPut `recipe[apt::cacher]` in the run_list for a server to provide APT caching and add `recipe[apt::cacher-client]` on the rest of the Debian-based nodes to take advantage of the caching server.\r\n\r\nLicense and Author\r\n==================\r\n\r\nAuthor:: Joshua Timberman (<joshua@opscode.com>)\r\nAuthor:: Matt Ray (<matt@opscode.com>)\r\nAuthor:: Seth Chisamore (<schisamo@opscode.com>)\r\n\r\nCopyright 2009-2011 Opscode, Inc.\r\n\r\nLicensed under the Apache License, Version 2.0 (the \"License\");\r\nyou may not use this file except in compliance with the License.\r\nYou may obtain a copy of the License at\r\n\r\n    http://www.apache.org/licenses/LICENSE-2.0\r\n\r\nUnless required by applicable law or agreed to in writing, software\r\ndistributed under the License is distributed on an \"AS IS\" BASIS,\r\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nSee the License for the specific language governing permissions and\r\nlimitations under the License.\r\n\r\n"
  },
  {
    "path": "contrib/chef/apt/TODO.org",
    "content": "* remove proxy from /etc/apt/apt.conf if it's listed (from preseed install)\r\n* check contents of /etc/apt/apt.conf.d/01proxy\r\n* investigate apt-cacher-ng\r\n"
  },
  {
    "path": "contrib/chef/apt/files/default/apt-cacher",
    "content": "# apt-cacher startup configuration file\r\n\r\n# IMPORTANT: check the apt-cacher.conf file before using apt-cacher as daemon.\r\n\r\n# set to 1 to start the daemon at boot time\r\nAUTOSTART=1\r\n\r\n# extra settings to override the ones in apt-cacher.conf\r\n# EXTRAOPT=\" daemon_port=3142 limit=30 \"\r\n"
  },
  {
    "path": "contrib/chef/apt/files/default/apt-cacher.conf",
    "content": "# This file has been modified by ./apt-proxy-to-apt-cacher\r\n# Some lines may have been appended at the bottom of this file\r\n# This file has been modified by /usr/share/apt-cacher/apt-proxy-to-apt-cacher\r\n# Some lines may have been appended at the bottom of this file\r\n#################################################################\r\n# This is the config file for apt-cacher. On most Debian systems\r\n# you can safely leave the defaults alone.\r\n#################################################################\r\n\r\n# cache_dir is used to set the location of the local cache. This can\r\n# become quite large, so make sure it is somewhere with plenty of space.\r\ncache_dir=/var/cache/apt-cacher\r\n\r\n# The email address of the administrator is displayed in the info page\r\n# and traffic reports.\r\nadmin_email=root@localhost\r\n\r\n# For the daemon startup settings please edit the file /etc/default/apt-cacher.\r\n\r\n# Daemon port setting, only useful in stand-alone mode. You need to run the\r\n# daemon as root to use privileged ports (<1024).\r\ndaemon_port = 3142\r\n\r\n# optional settings, user and group to run the daemon as. Make sure they have\r\n# sufficient permissions on the cache and log directories. Comment the settings\r\n# to run apt-cacher as the native user.\r\ngroup=www-data\r\nuser=www-data\r\n\r\n# optional setting, binds the listening daemon to one specified IP. Use IP\r\n# ranges for more advanced configuration, see below.\r\n# daemon_addr=localhost\r\n\r\n# If your apt-cacher machine is directly exposed to the Internet and you are\r\n# worried about unauthorised machines fetching packages through it, you can\r\n# specify a list of IPv4 addresses which are allowed to use it and another\r\n# list of IPv4 addresses which aren't.\r\n# Localhost (127.0.0.1) is always allowed. Other addresses must be matched\r\n# by allowed_hosts and not by denied_hosts to be permitted to use the cache.\r\n# Setting allowed_hosts to \"*\" means \"allow all\".\r\n# Otherwise the format is a comma-separated list containing addresses,\r\n# optionally with masks (like 10.0.0.0/22), or ranges of addresses (two\r\n# addresses separated by a hyphen, no masks, like '192.168.0.3-192.168.0.56').\r\nallowed_hosts=*\r\ndenied_hosts=\r\n\r\n# And similiarly for IPv6 with allowed_hosts_6 and denied_hosts_6.\r\n# Note that IPv4-mapped IPv6 addresses (::ffff:w.x.y.z) are truncated to\r\n# w.x.y.z and are handled as IPv4.\r\nallowed_hosts_6=fec0::/16\r\ndenied_hosts_6=\r\n\r\n# This thing can be done by Apache but is much simplier here - limit access to\r\n# Debian mirrors based on server names in the URLs\r\n#allowed_locations=ftp.uni-kl.de,ftp.nerim.net,debian.tu-bs.de\r\n\r\n# Apt-cacher can generate usage reports every 24 hours if you set this\r\n# directive to 1. You can view the reports in a web browser by pointing\r\n# to your cache machine with '/apt-cacher/report' on the end, like this:\r\n#      http://yourcache.example.com/apt-cacher/report\r\n# Generating reports is very fast even with many thousands of logfile\r\n# lines, so you can safely turn this on without creating much \r\n# additional system load.\r\ngenerate_reports=1\r\n\r\n# Apt-cacher can clean up its cache directory every 24 hours if you set\r\n# this directive to 1. Cleaning the cache can take some time to run\r\n# (generally in the order of a few minutes) and removes all package\r\n# files that are not mentioned in any existing 'Packages' lists. This\r\n# has the effect of deleting packages that have been superseded by an\r\n# updated 'Packages' list.\r\nclean_cache=1\r\n\r\n# The directory to use for apt-cacher access and error logs.\r\n# The access log records every request in the format:\r\n# date-time|client ip address|HIT/MISS/EXPIRED|object size|object name\r\n# The error log is slightly more free-form, and is also used for debug\r\n# messages if debug mode is turned on.\r\n# Note that the old 'logfile' and 'errorfile' directives are\r\n# deprecated: if you set them explicitly they will be honoured, but it's\r\n# better to just get rid of them from old config files.\r\nlogdir=/var/log/apt-cacher\r\n\r\n# apt-cacher can use different methods to decide whether package lists need to\r\n# be updated,\r\n# A) looking at the age of the cached files\r\n# B) getting HTTP header from server and comparing that with cached data. This\r\n# method is more reliable and avoids desynchronisation of data and index files\r\n# but needs to transfer few bytes from the server every time somebody requests\r\n# the files (\"apt-get update\")\r\n# Set the following value to the maximum age (in hours) for method A or to 0\r\n# for method B\r\nexpire_hours=0\r\n\r\n# Apt-cacher can pass all its requests to an external http proxy like\r\n# Squid, which could be very useful if you are using an ISP that blocks\r\n# port 80 and requires all web traffic to go through its proxy. The\r\n# format is 'hostname:port', eg: 'proxy.example.com:8080'.\r\nhttp_proxy=proxy.example.com:8080\r\n\r\n# Use of an external proxy can be turned on or off with this flag.\r\n# Value should be either 0 (off) or 1 (on).\r\nuse_proxy=0\r\n\r\n# External http proxy sometimes need authentication to get full access. The\r\n# format is 'username:password'.\r\nhttp_proxy_auth=proxyuser:proxypass\r\n\r\n# Use of external proxy authentication can be turned on or off with this flag.\r\n# Value should be either 0 (off) or 1 (on).\r\nuse_proxy_auth=0\r\n\r\n# Rate limiting sets the maximum bandwidth in bytes per second to use\r\n# for fetching packages. Syntax is fully defined in 'man wget'.\r\n# Use 'k' or 'm' to use kilobits or megabits / second: eg, 'limit=25k'.\r\n# Use 0 or a negative value for no rate limiting.\r\nlimit=0\r\n\r\n# Debug mode makes apt-cacher spew a lot of extra debug junk to the\r\n# error log (whose location is defined with the 'logdir' directive).\r\n# Leave this off unless you need it, or your error log will get very\r\n# big. Acceptable values are 0 or 1.\r\ndebug=0\r\n\r\n# Adapt the line in the usage info web page to match your server configuration\r\n# example_sources_line=deb&nbsp;http://<b>my.cacher.server:3142/</b>ftp.au.debian.org/debian&nbsp;unstable&nbsp;main&nbsp;contrib&nbsp;non-free\r\n\r\n# Print a 410 (Gone) HTTP message with the specified text when accessed via\r\n# CGI. Useful to tell users to adapt their sources.list files when the\r\n# apt-cacher server is beeing relocated (via apt-get's error messages while\r\n# running \"update\")\r\n#cgi_advise_to_use = Please use http://cacheserver:3142/ as apt-cacher access URL\r\n#cgi_advise_to_use = Server relocated. To change sources.list, run perl -pe \"s,/apt-cacher\\??,:3142,\" -i /etc/apt/sources.list\r\n\r\n# Server mapping - this allows to hide real server names behind virtual paths\r\n# that appear in the access URL. This method is known from apt-proxy. This is\r\n# also the only method to use FTP access to the target hosts. The syntax is simple, the part of the beginning to replace, followed by a list of mirror urls, all space separated. Multiple profile are separated by semicolons\r\n# path_map = debian ftp.uni-kl.de/pub/linux/debian ftp2.de.debian.org/debian ; ubuntu archive.ubuntu.com/ubuntu ; security security.debian.org/debian-security ftp2.de.debian.org/debian-security\r\n# Note that you need to specify all target servers in the allowed_locations\r\n# options if you make use of it. Also note that the paths should not overlap\r\n# each other. FTP access method not supported yet, maybe in the future.\r\n\r\n# extra setting from apt-proxy configuration\r\npath_map =  ubuntu us.archive.ubuntu.com/ubuntu ; ubuntu-security security.ubuntu.com/ubuntu ; debian debian.osuosl.org/debian/ ; security security.debian.org/debian-security\r\n"
  },
  {
    "path": "contrib/chef/apt/files/default/apt-proxy-v2.conf",
    "content": "[DEFAULT]\r\n;; All times are in seconds, but you can add a suffix\r\n;; for minutes(m), hours(h) or days(d)\r\n\r\n;; commented out address so apt-proxy will listen on all IPs\r\n;; address = 127.0.0.1\r\nport = 9999\r\ncache_dir = /var/cache/apt-proxy\r\n\r\n;; Control files (Packages/Sources/Contents) refresh rate\r\nmin_refresh_delay = 1s\r\ncomplete_clientless_downloads = 1\r\n\r\n;; Debugging settings.\r\ndebug = all:4 db:0\r\n\r\ntime = 30\r\npassive_ftp = on\r\n\r\n;;--------------------------------------------------------------\r\n;; Cache housekeeping\r\n\r\ncleanup_freq = 1d\r\nmax_age = 120d\r\nmax_versions = 3\r\n\r\n;;---------------------------------------------------------------\r\n;; Backend servers\r\n;;\r\n;; Place each server in its own [section]\r\n\r\n[ubuntu]\r\n; Ubuntu archive\r\nbackends =\r\n        http://us.archive.ubuntu.com/ubuntu\r\n\r\n[ubuntu-security]\r\n; Ubuntu security updates\r\nbackends = http://security.ubuntu.com/ubuntu\r\n\r\n[debian]\r\n;; Backend servers, in order of preference\r\nbackends = \r\n        http://debian.osuosl.org/debian/\r\n\r\n[security]\r\n;; Debian security archive\r\nbackends = \r\n        http://security.debian.org/debian-security\r\n        http://ftp2.de.debian.org/debian-security\r\n"
  },
  {
    "path": "contrib/chef/apt/metadata.json",
    "content": "{\r\n  \"dependencies\": {\r\n  },\r\n  \"name\": \"apt\",\r\n  \"maintainer_email\": \"cookbooks@opscode.com\",\r\n  \"attributes\": {\r\n  },\r\n  \"license\": \"Apache 2.0\",\r\n  \"suggestions\": {\r\n  },\r\n  \"platforms\": {\r\n    \"debian\": \">= 0.0.0\",\r\n    \"ubuntu\": \">= 0.0.0\"\r\n  },\r\n  \"maintainer\": \"Opscode, Inc.\",\r\n  \"long_description\": \"Description\\n===========\\n\\nConfigures various APT components on Debian-like systems.  Also includes a LWRP.\\n\\nRecipes\\n=======\\n\\ndefault\\n-------\\nThe default recipe runs apt-get update during the Compile Phase of the Chef run to ensure that the system's package cache is updated with the latest. It is recommended that this recipe appear first in a node's run list (directly or through a role) to ensure that when installing packages, Chef will be able to download the latest version available on the remote APT repository.\\n\\nThis recipe also sets up a local cache directory for preseeding packages.\\n\\ncacher\\n------\\nInstalls the apt-cacher package and service so the system can provide APT caching. You can check the usage report at http://{hostname}:3142/report. The cacher recipe includes the `cacher-client` recipe, so it helps seed itself.\\n\\ncacher-client\\n-------------\\nConfigures the node to use the apt-cacher server as a client.\\n\\nResources/Providers\\n===================\\n\\nThis LWRP provides an easy way to manage additional APT repositories.\\n\\n# Actions\\n\\n- :add: creates a repository file and builds the repository listing\\n- :remove: removes the repository file\\n\\n# Attribute Parameters\\n\\n- repo_name: name attribute. The name of the channel to discover\\n- uri: the base of the Debian distribution\\n- distribution: this is usually your release's codename...ie something like `karmic`, `lucid` or `maverick`\\n- components: package groupings..when it doubt use `main`\\n- deb_src: whether or not to add the repository as a source repo as well\\n- key_server: the GPG keyserver where the key for the repo should be retrieved\\n- key: if a `key_server` is provided, this is assumed to be the fingerprint, otherwise it is the URI to the GPG key for the repo\\n\\n# Example\\n\\n    # add the Zenoss repo\\n    apt_repository \\\"zenoss\\\" do\\n      uri \\\"http://dev.zenoss.org/deb\\\"\\n      components [\\\"main\\\",\\\"stable\\\"]\\n      action :add\\n    end\\n    \\n    # add the Nginx PPA; grab key from keyserver\\n    apt_repository \\\"nginx-php\\\" do\\n      uri \\\"http://ppa.launchpad.net/nginx/php5/ubuntu\\\"\\n      distribution node['lsb']['codename']\\n      components [\\\"main\\\"]\\n      keyserver \\\"keyserver.ubuntu.com\\\"\\n      key \\\"C300EE8C\\\"\\n      action :add\\n    end\\n    \\n    # add the Cloudkick Repo\\n    apt_repository \\\"cloudkick\\\" do\\n      uri \\\"http://packages.cloudkick.com/ubuntu\\\"\\n      distribution node['lsb']['codename']\\n      components [\\\"main\\\"]\\n      key \\\"http://packages.cloudkick.com/cloudkick.packages.key\\\"\\n      action :add\\n    end\\n    \\n    # remove Zenoss repo\\n    apt_repository \\\"zenoss\\\" do\\n      action :remove\\n    end\\n    \\nUsage\\n=====\\n\\nPut `recipe[apt]` first in the run list. If you have other recipes that you want to use to configure how apt behaves, like new sources, notify the execute resource to run, e.g.:\\n\\n    template \\\"/etc/apt/sources.list.d/my_apt_sources.list\\\" do\\n      notifies :run, resources(:execute => \\\"apt-get update\\\"), :immediately\\n    end\\n\\nThe above will run during execution phase since it is a normal template resource, and should appear before other package resources that need the sources in the template.\\n\\nPut `recipe[apt::cacher]` in the run_list for a server to provide APT caching and add `recipe[apt::cacher-client]` on the rest of the Debian-based nodes to take advantage of the caching server.\\n\\nLicense and Author\\n==================\\n\\nAuthor:: Joshua Timberman (<joshua@opscode.com>)\\nAuthor:: Matt Ray (<matt@opscode.com>)\\nAuthor:: Seth Chisamore (<schisamo@opscode.com>)\\n\\nCopyright 2009-2011 Opscode, Inc.\\n\\nLicensed under the Apache License, Version 2.0 (the \\\"License\\\");\\nyou may not use this file except in compliance with the License.\\nYou may obtain a copy of the License at\\n\\n    http://www.apache.org/licenses/LICENSE-2.0\\n\\nUnless required by applicable law or agreed to in writing, software\\ndistributed under the License is distributed on an \\\"AS IS\\\" BASIS,\\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\\nSee the License for the specific language governing permissions and\\nlimitations under the License.\\n\\n\",\r\n  \"version\": \"1.1.2\",\r\n  \"recommendations\": {\r\n  },\r\n  \"recipes\": {\r\n    \"apt\": \"Runs apt-get update during compile phase and sets up preseed directories\",\r\n    \"apt::cacher-client\": \"Client for the apt::cacher server\",\r\n    \"apt::cacher\": \"Set up an APT cache\"\r\n  },\r\n  \"groupings\": {\r\n  },\r\n  \"conflicting\": {\r\n  },\r\n  \"replacing\": {\r\n  },\r\n  \"description\": \"Configures apt and apt services and an LWRP for managing apt repositories\",\r\n  \"providing\": {\r\n  }\r\n}"
  },
  {
    "path": "contrib/chef/apt/metadata.rb",
    "content": "maintainer        \"Opscode, Inc.\"\r\nmaintainer_email  \"cookbooks@opscode.com\"\r\nlicense           \"Apache 2.0\"\r\ndescription       \"Configures apt and apt services and an LWRP for managing apt repositories\"\r\nlong_description  IO.read(File.join(File.dirname(__FILE__), 'README.md'))\r\nversion           \"1.1.2\"\r\nrecipe            \"apt\", \"Runs apt-get update during compile phase and sets up preseed directories\"\r\nrecipe            \"apt::cacher\", \"Set up an APT cache\"\r\nrecipe            \"apt::cacher-client\", \"Client for the apt::cacher server\"\r\n\r\n%w{ ubuntu debian }.each do |os|\r\n  supports os\r\nend\r\n"
  },
  {
    "path": "contrib/chef/apt/providers/repository.rb",
    "content": "#\r\n# Cookbook Name:: apt\r\n# Provider:: repository\r\n#\r\n# Copyright 2010-2011, Opscode, Inc.\r\n#\r\n# Licensed under the Apache License, Version 2.0 (the \"License\");\r\n# you may not use this file except in compliance with the License.\r\n# You may obtain a copy of the License at\r\n#\r\n#     http://www.apache.org/licenses/LICENSE-2.0\r\n#\r\n# Unless required by applicable law or agreed to in writing, software\r\n# distributed under the License is distributed on an \"AS IS\" BASIS,\r\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\n# See the License for the specific language governing permissions and\r\n# limitations under the License.\r\n#\r\n\r\naction :add do\r\n  unless ::File.exists?(\"/etc/apt/sources.list.d/#{new_resource.repo_name}-source.list\")\r\n    Chef::Log.info \"Adding #{new_resource.repo_name} repository to /etc/apt/sources.list.d/#{new_resource.repo_name}-source.list\"\r\n    # add key\r\n    if new_resource.keyserver && new_resource.key\r\n      execute \"install-key #{new_resource.key}\" do\r\n        command \"apt-key adv --keyserver #{new_resource.keyserver} --recv #{new_resource.key}\"\r\n        action :nothing\r\n      end.run_action(:run)\r\n    elsif new_resource.key && (new_resource.key =~ /http/)\r\n      key_name = new_resource.key.split(/\\//).last\r\n      remote_file \"#{Chef::Config[:file_cache_path]}/#{key_name}\" do\r\n        source new_resource.key\r\n        mode \"0644\"\r\n        action :nothing\r\n      end.run_action(:create_if_missing)\r\n      execute \"install-key #{key_name}\" do\r\n        command \"apt-key add #{Chef::Config[:file_cache_path]}/#{key_name}\"\r\n        action :nothing\r\n      end.run_action(:run)\r\n    end\r\n    # build our listing\r\n    repository = \"deb\"\r\n    repository = \"deb-src\" if new_resource.deb_src\r\n    repository = \"# Created by the Chef apt_repository LWRP\\n\" + repository\r\n    repository += \" #{new_resource.uri}\"\r\n    repository += \" #{new_resource.distribution}\"\r\n    new_resource.components.each {|component| repository += \" #{component}\"}\r\n    # write out the file, replace it if it already exists\r\n    file \"/etc/apt/sources.list.d/#{new_resource.repo_name}-source.list\" do\r\n      owner \"root\"\r\n      group \"root\"\r\n      mode 0644\r\n      content repository + \"\\n\"\r\n      action :nothing\r\n    end.run_action(:create)\r\n    execute \"update package index\" do\r\n      command \"apt-get update\"\r\n      action :nothing\r\n    end.run_action(:run)\r\n    new_resource.updated_by_last_action(true)\r\n  end\r\nend\r\n\r\naction :remove do\r\n  if ::File.exists?(\"/etc/apt/sources.list.d/#{new_resource.repo_name}-source.list\")\r\n    Chef::Log.info \"Removing #{new_resource.repo_name} repository from /etc/apt/sources.list.d/\"\r\n    file \"/etc/apt/sources.list.d/#{new_resource.repo_name}-source.list\" do\r\n      action :delete\r\n    end\r\n    new_resource.updated_by_last_action(true)\r\n  end\r\nend\r\n"
  },
  {
    "path": "contrib/chef/apt/recipes/cacher-client.rb",
    "content": "#\r\n# Cookbook Name:: apt\r\n# Recipe:: cacher-client\r\n#\r\n# Copyright 2011, Opscode, Inc.\r\n#\r\n# Licensed under the Apache License, Version 2.0 (the \"License\");\r\n# you may not use this file except in compliance with the License.\r\n# You may obtain a copy of the License at\r\n#\r\n#     http://www.apache.org/licenses/LICENSE-2.0\r\n#\r\n# Unless required by applicable law or agreed to in writing, software\r\n# distributed under the License is distributed on an \"AS IS\" BASIS,\r\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\n# See the License for the specific language governing permissions and\r\n# limitations under the License.\r\n#\r\n\r\n#remove Acquire::http::Proxy lines from /etc/apt/apt.conf since we use 01proxy\r\n#these are leftover from preseed installs\r\nexecute \"Remove proxy from /etc/apt/apt.conf\" do\r\n  command \"sed --in-place '/^Acquire::http::Proxy/d' /etc/apt/apt.conf\"\r\n  only_if \"grep Acquire::http::Proxy /etc/apt/apt.conf\"\r\nend\r\n\r\nservers = search(:node, 'recipes:apt\\:\\:cacher') || []\r\nif servers.length > 0\r\n  Chef::Log.info(\"apt-cacher server found on #{servers[0]}.\")\r\n  proxy = \"Acquire::http::Proxy \\\"http://#{servers[0].ipaddress}:3142\\\";\\n\"\r\n  file \"/etc/apt/apt.conf.d/01proxy\" do\r\n    owner \"root\"\r\n    group \"root\"\r\n    mode \"0644\"\r\n    content proxy\r\n    action :create\r\n  end\r\nelse\r\n  Chef::Log.info(\"No apt-cacher server found.\")\r\n  file \"/etc/apt/apt.conf.d/01proxy\" do\r\n    action :delete\r\n    only_if {File.exists?(\"/etc/apt/apt.conf.d/01proxy\")}\r\n  end\r\nend\r\n"
  },
  {
    "path": "contrib/chef/apt/recipes/cacher.rb",
    "content": "#\r\n# Cookbook Name:: apt\r\n# Recipe:: cacher\r\n#\r\n# Copyright 2008-2011, Opscode, Inc.\r\n#\r\n# Licensed under the Apache License, Version 2.0 (the \"License\");\r\n# you may not use this file except in compliance with the License.\r\n# You may obtain a copy of the License at\r\n#\r\n#     http://www.apache.org/licenses/LICENSE-2.0\r\n#\r\n# Unless required by applicable law or agreed to in writing, software\r\n# distributed under the License is distributed on an \"AS IS\" BASIS,\r\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\n# See the License for the specific language governing permissions and\r\n# limitations under the License.\r\n#\r\npackage \"apt-cacher\" do\r\n  action :install\r\nend\r\n\r\nservice \"apt-cacher\" do\r\n  supports :restart => true, :status => false\r\n  action [ :enable, :start ]\r\nend\r\n\r\ncookbook_file \"/etc/apt-cacher/apt-cacher.conf\" do\r\n  source \"apt-cacher.conf\"\r\n  owner \"root\"\r\n  group \"root\"\r\n  mode 0644\r\n  notifies :restart, resources(:service => \"apt-cacher\")\r\nend\r\n\r\ncookbook_file \"/etc/default/apt-cacher\" do\r\n  source \"apt-cacher\"\r\n  owner \"root\"\r\n  group \"root\"\r\n  mode 0644\r\n  notifies :restart, resources(:service => \"apt-cacher\")\r\nend\r\n\r\n#this will help seed the proxy\r\ninclude_recipe \"apt::cacher-client\"\r\n"
  },
  {
    "path": "contrib/chef/apt/recipes/default.rb",
    "content": "#\r\n# Cookbook Name:: apt\r\n# Recipe:: default\r\n#\r\n# Copyright 2008-2009, Opscode, Inc.\r\n#\r\n# Licensed under the Apache License, Version 2.0 (the \"License\");\r\n# you may not use this file except in compliance with the License.\r\n# You may obtain a copy of the License at\r\n#\r\n#     http://www.apache.org/licenses/LICENSE-2.0\r\n#\r\n# Unless required by applicable law or agreed to in writing, software\r\n# distributed under the License is distributed on an \"AS IS\" BASIS,\r\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\n# See the License for the specific language governing permissions and\r\n# limitations under the License.\r\n#\r\n\r\ne = execute \"apt-get update\" do\r\n  action :nothing\r\nend\r\n\r\ne.run_action(:run)\r\n\r\n%w{/var/cache/local /var/cache/local/preseeding}.each do |dirname|\r\n  directory dirname do\r\n    owner \"root\"\r\n    group \"root\"\r\n    mode  0755\r\n    action :create\r\n  end\r\nend\r\n"
  },
  {
    "path": "contrib/chef/apt/resources/repository.rb",
    "content": "#\r\n# Cookbook Name:: apt\r\n# Resource:: repository\r\n#\r\n# Copyright 2010-2011, Opscode, Inc.\r\n#\r\n# Licensed under the Apache License, Version 2.0 (the \"License\");\r\n# you may not use this file except in compliance with the License.\r\n# You may obtain a copy of the License at\r\n#\r\n#     http://www.apache.org/licenses/LICENSE-2.0\r\n#\r\n# Unless required by applicable law or agreed to in writing, software\r\n# distributed under the License is distributed on an \"AS IS\" BASIS,\r\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\n# See the License for the specific language governing permissions and\r\n# limitations under the License.\r\n#\r\n\r\nactions :add, :remove\r\n\r\n#name of the repo, used for source.list filename\r\nattribute :repo_name, :kind_of => String, :name_attribute => true\r\nattribute :uri, :kind_of => String\r\nattribute :distribution, :kind_of => String\r\nattribute :components, :kind_of => Array, :default => []\r\n#whether or not to add the repository as a source repo as well\r\nattribute :deb_src, :default => false\r\nattribute :keyserver, :kind_of => String, :default => nil\r\nattribute :key, :kind_of => String, :default => nil\r\n"
  },
  {
    "path": "contrib/chef/scalaris_PIC/attributes/default.rb",
    "content": "#\n# Cookbook Name:: scalaris_PIC\n# Attributes:: default\n#\n# Copyright 2012-2013, Zuse Institute Berlin\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\ndefault[:REC][:PICs][:scalaris_PIC] = [\n  {:attributes => {\n                    :scalaris_port => 14195,\n                    :scalaris_port_web => 8000,\n                    :scalaris_node => \"node@#{node['ipaddress']}\",\n                    :scalaris_start_first => true,\n                    :scalaris_start_mgmt_server => true,\n                    :scalaris_mgmt_server => {\"ip4\" => \"#{node['ipaddress']}\", \"port\" => 14195},\n                    :scalaris_known_hosts => [{\"ip4\" => \"#{node['ipaddress']}\", \"port\" => 14195}], # update on deployment to known IP of another node\n                    :scalaris_nodes_per_vm => 1,\n                    :scalaris_max_json_req_size => 1024*1024,\n                    :scalaris_users => [] # [{\"user\" => \"User\", \"password\" => \"Password\"}], if empty => no restrictions\n                   }\n  }]\n\nnormal[:scalaris_PIC][:kpis] = {\n  :jmx_url => \"service:jmx:rmi:///jndi/rmi://localhost:14193/jmxrmi\",\n  :kpis => {\"ScalarisNode_CurLatencyAvg\" =>\n             {:on => \"de.zib.scalaris:type=MonitorNode\",\n              :att => \"CurLatencyAvg\",\n              :period => 1},\n            \"ScalarisNode_CurLatencyStddev\" =>\n             {:on => \"de.zib.scalaris:type=MonitorNode\",\n              :att => \"CurLatencyStddev\",\n              :period => 1},\n            \"ScalarisService_CurLatencyAvg\" =>\n             {:on => \"de.zib.scalaris:type=MonitorService\",\n              :att => \"CurLatencyAvg\",\n              :period => 1},\n            \"ScalarisService_CurLatencyStddev\" =>\n             {:on => \"de.zib.scalaris:type=MonitorService\",\n              :att => \"CurLatencyStddev\",\n              :period => 1},\n            \"ScalarisService_LoadEstimate\" =>\n             {:on => \"de.zib.scalaris:type=MonitorService\",\n              :att => \"TotalLoad\",\n              :period => 1}\n           }\n  }\n\ndefault[:REC][:PICs][:JASMINe_Probe][:ACs] = [\n  {:kpis_name => [\n                  \"ScalarisNode_CurLatencyAvg\",\n                  \"ScalarisNode_CurLatencyStddev\",\n                  \"ScalarisService_CurLatencyAvg\",\n                  \"ScalarisService_CurLatencyStddev\",\n                  \"ScalarisService_LoadEstimate\"\n                 ],\n   :pic_cookbook_name => \"scalaris_PIC\"\n  }]\n\n#puts \"Printing scalaris attributes from attributes file: #{node[:REC][:PICs][:scalaris_PIC][0][:attributes]} \"\n"
  },
  {
    "path": "contrib/chef/scalaris_PIC/metadata.rb",
    "content": "maintainer       \"Zuse Institute Berlin\"\nmaintainer_email \"kruber@zib.de\"\nlicense          \"Apache 2.0\"\ndescription      \"Installs/Configures Scalaris for the 4CaaSt platform\"\n#long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))\nversion          \"0.0.1\"\n\ndepends \"apt\", \"= 1.1.2\"\n\n%w{ debian ubuntu centos redhat fedora suse }.each do |os|\n  supports os\nend\n\nrecipe \"scalaris_PIC::Deploy_PIC\", \"Installs and configures Scalaris\"\nrecipe \"scalaris_PIC::Start_PIC\", \"Starts Scalaris\"\nrecipe \"scalaris_PIC::Stop_PIC\", \"Stops Scalaris\"\nrecipe \"scalaris_PIC::Undeploy_PIC\", \"Undeploys Scalaris\"\n"
  },
  {
    "path": "contrib/chef/scalaris_PIC/recipes/Deploy_PIC.rb",
    "content": "#\n# Cookbook Name:: scalaris_PIC\n# Recipe:: Deploy_PIC\n#\n# Copyright 2012, Zuse Institute Berlin\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n\ninclude_recipe \"apt\"\n\nscalaris_pkgs = [\"scalaris\", \"scalaris-java\"]\n\nscalaris_repo_base = \"http://download.opensuse.org/repositories/home:/scalaris:/svn/\"\n\n# debian ubuntu centos redhat fedora suse\nscalaris_repo = value_for_platform(\n  \"debian\" => {:default => scalaris_repo_base + \"Debian_\" + node[:platform_version]},\n  \"ubuntu\" => {:default => scalaris_repo_base + \"xUbuntu_\" + node[:platform_version]},\n  \"centos\" => {:default => scalaris_repo_base + \"CentOS_\" + node[:platform_version]},\n  \"redhat\" => {:default => scalaris_repo_base + \"RHEL_\" + node[:platform_version]},\n  \"fedora\" => {:default => scalaris_repo_base + \"Fedora_\" + node[:platform_version]},\n  \"suse\"   => {:default => scalaris_repo_base + \"openSUSE_\" + node[:platform_version]}\n  )\n\ncase node[:platform]\nwhen \"debian\", \"ubuntu\"\n  apt_repository \"scalaris-svn\" do\n    action :add\n    uri scalaris_repo + \" ./\"\n    key \"http://download.opensuse.org/repositories/home:/scalaris:/svn/openSUSE_Factory/repodata/repomd.xml.key\"\n  end\nwhen \"centos\", \"redhat\", \"fedora\"\n  execute \"create-yum-cache\" do\n    command \"yum -q makecache\"\n    action :nothing\n  end\n \n  ruby_block \"reload-internal-yum-cache\" do\n    block do\n      Chef::Provider::Package::Yum::YumCache.instance.reload\n    end\n    action :nothing\n  end\n \n  remote_file \"/etc/yum.repos.d/home:scalaris:svn.repo\" do\n    source scalaris_repo + \"/home:scalaris:svn.repo\"\n    mode \"0644\"\n    notifies :run, resources(:execute => \"create-yum-cache\"), :immediately\n    notifies :create, resources(:ruby_block => \"reload-internal-yum-cache\"), :immediately\n  end\nwhen \"suse\"\n  execute \"create-zypper-repo\" do\n    command \"zypper --gpg-auto-import-keys addrepo --refresh \\\"\" + scalaris_repo + \"\\\" scalaris-svn\"\n    action :run\n  end\nend\n\nscalaris_pkgs.each do |pkg|\n  package pkg do\n    action :install\n  end\nend\n\ntemplate \"/etc/scalaris/initd.conf\" do\n  source \"initd.conf.erb\"\n  owner \"scalaris\"\n  group \"scalaris\"\n  mode \"0644\"\n  variables(\n    :scalaris_node => node[:REC][:PICs][:scalaris_PIC][0][:attributes][:scalaris_node]\n  )\n#  notifies :restart, resources(:service => \"scalaris\")\nend\n\ntemplate \"/etc/scalaris/scalaris.local.cfg\" do\n  source \"scalaris.local.cfg.erb\"\n  owner \"scalaris\"\n  group \"scalaris\"\n  mode \"0644\"\n  variables(\n    :scalaris_port => node[:REC][:PICs][:scalaris_PIC][0][:attributes][:scalaris_port],\n    :scalaris_port_web => node[:REC][:PICs][:scalaris_PIC][0][:attributes][:scalaris_port_web],\n    :scalaris_start_first => node[:REC][:PICs][:scalaris_PIC][0][:attributes][:scalaris_start_first],\n    :scalaris_start_mgmt_server => node[:REC][:PICs][:scalaris_PIC][0][:attributes][:scalaris_start_mgmt_server],\n    :scalaris_mgmt_server => node[:REC][:PICs][:scalaris_PIC][0][:attributes][:scalaris_mgmt_server],\n    :scalaris_known_hosts => node[:REC][:PICs][:scalaris_PIC][0][:attributes][:scalaris_known_hosts],\n    :scalaris_nodes_per_vm => node[:REC][:PICs][:scalaris_PIC][0][:attributes][:scalaris_nodes_per_vm],\n    :scalaris_max_json_req_size => node[:REC][:PICs][:scalaris_PIC][0][:attributes][:scalaris_max_json_req_size],\n    :scalaris_users => node[:REC][:PICs][:scalaris_PIC][0][:attributes][:scalaris_users]\n  )\n#  notifies :restart, resources(:service => \"scalaris\")\nend\n"
  },
  {
    "path": "contrib/chef/scalaris_PIC/recipes/Start_PIC.rb",
    "content": "#\n# Cookbook Name:: scalaris_PIC\n# Recipe:: Start_PIC\n# \t\tNote: Start_PIC assumes that Deploy_PIC ahas been executed already...\n#\n# Copyright 2012, Zuse Institute Berlin\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n\nservice \"scalaris\" do\n  service_name \"scalaris\"\n  supports :status => true, :start => true, :stop => true, :restart => true\n  # TODO: if last node, we have to kill (not stop) the node! - the following check is not really checking that\n  if node[:REC][:PICs][:scalaris_PIC][0][:attributes][:scalaris_start_first] then\n    stop_command \"/etc/init.d/scalaris kill\"\n  end\n  action [ :enable, :start ]\n  case node[:platform]\n  when \"ubuntu\", \"debian\"\n    provider Chef::Provider::Service::Init::Debian\n  else\n    provider Chef::Provider::Service::Init\n  end\nend\n\nservice \"scalaris-monitor\" do\n  service_name \"scalaris-monitor\"\n  supports :status => true, :start => true, :stop => true, :restart => true\n  action [ :enable, :start ]\n  case node[:platform]\n  when \"ubuntu\", \"debian\"\n    provider Chef::Provider::Service::Init::Debian\n  else\n    provider Chef::Provider::Service::Init\n  end\nend\n"
  },
  {
    "path": "contrib/chef/scalaris_PIC/recipes/Stop_PIC.rb",
    "content": "#\n# Cookbook Name:: scalaris_PIC\n# Recipe:: Stop_PIC\n#\n# Copyright 2012, Zuse Institute Berlin\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n\nservice \"scalaris\" do\n  service_name \"scalaris\"\n  supports :status => true, :start => true, :stop => true, :restart => true\n  # TODO: if last node, we have to kill (not stop) the node! - the following check is not really checking that\n  if node[:REC][:PICs][:scalaris_PIC][0][:attributes][:scalaris_start_first] then\n    stop_command \"/etc/init.d/scalaris kill\"\n  end\n  action [ :disable, :stop ]\n  case node[:platform]\n  when \"ubuntu\", \"debian\"\n    provider Chef::Provider::Service::Init::Debian\n  else\n    provider Chef::Provider::Service::Init\n  end\nend\n"
  },
  {
    "path": "contrib/chef/scalaris_PIC/recipes/Undeploy_PIC.rb",
    "content": "#\n# Cookbook Name:: scalaris_PIC\n# Recipe:: Undeploy_PIC\n# \t\tNote: Undeploy_PIC assumes that Stop_PIC has been executed already...\n#\n# Copyright 2012, Zuse Institute Berlin\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n\ninclude_recipe \"apt\"\n\n[ \"/etc/scalaris/initd.conf\", \"/etc/scalaris/scalaris.local.cfg\"].each do |conffile|\n  file conffile do\n    action :delete\n  end\nend\n\nscalaris_pkgs = [\"scalaris\"]\n\nscalaris_pkgs.each do |pkg|\n  package pkg do\n    action [ :purge, :remove ]\n  end\nend\n\ncase node[:platform]\nwhen \"debian\", \"ubuntu\"\n  apt_repository \"scalaris-svn\" do\n    action :remove\n  end\nwhen \"centos\", \"redhat\", \"fedora\"\n  execute \"clean-yum-cache\" do\n    command \"yum clean all\"\n    action :nothing\n  end\n \n  file \"/etc/yum.repos.d/home:scalaris:svn.repo\" do\n    action :delete\n    notifies :run, resources(:execute => \"clean-yum-cache\"), :immediately\n    notifies :create, resources(:ruby_block => \"reload-internal-yum-cache\"), :immediately\n  end\nwhen \"suse\"\n  execute \"remove-zypper-repo\" do\n    command \"zypper removerepo scalaris-svn\"\n    action :run\n  end\nend\n"
  },
  {
    "path": "contrib/chef/scalaris_PIC/templates/default/initd.conf.erb",
    "content": "#!/bin/bash\n\nSCALARIS_NODE=\"<%=@scalaris_node%>\"\nSCALARIS_ADDITIONAL_PARAMETERS=\"\"\n"
  },
  {
    "path": "contrib/chef/scalaris_PIC/templates/default/scalaris.local.cfg.erb",
    "content": "% port(s) for incoming communications, try one in this range\n{port, <%=@scalaris_port%>}.\n\n% http web server port for debug interface, JSON interface\n{yaws_port, <%=@scalaris_port_web%>}.\n\n{first, <%=@scalaris_start_first%>}.\n{start_mgmt_server, <%=@scalaris_start_mgmt_server%>}.\n\n% Insert the appropriate IP-addresses for your setup\n% as comma separated integers:\n% IP Address, Port, and label of the boot server\n{mgmt_server, {{<%=@scalaris_mgmt_server[\"ip4\"].gsub('.', ',')%>},<%=@scalaris_mgmt_server[\"port\"]%>,mgmt_server}}.\n\n% IP Address, Port, and label of a node which is already in the system\n{known_hosts, [<%=@scalaris_known_hosts.map {|host| \"{{\" + host[\"ip4\"].gsub('.', ',') + \"},\" + host[\"port\"].to_s + \",service_per_vm}\"}.join(\", \")%>]}.\n\n% how many scalaris nodes per vm\n{nodes_per_vm, <%=@scalaris_nodes_per_vm%>}.\n\n% the maximum size of a post request for the JSON-RPC\n% (either a number of bytes, or nolimit)\n{yaws_max_post_data, <%=@scalaris_max_json_req_size%>}.\n\n% Allows to restrict access to the web debug interface (including the JSON-RPC!).\n% Expects a list of {\"User\", \"Password\"} tuples.\n% Note: This will effectively disable the JSON-RPC including the Python and Ruby\n%       APIs as they are not prepared to handle authentication yet.\n{yaws_auth, [ <% @scalaris_users.each do |usr_pwd| puts \"{ \\\"\" + usr_pwd[\"user\"] + \"\\\", \\\"\" + usr_pwd[\"password\"] + \"\\\" }\" end -%> ]}.\n"
  },
  {
    "path": "contrib/datanucleus/.gitignore",
    "content": "*.class\ndatanucleus.log\n\n*.jar\n/bin\n\n# maven\ntarget/\nrelease.properties\n\n# eclipse\n*.project\n*.metadata\n*.tmp\nlocal.properties\n.loadpath\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/.classpath",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<classpath>\n\t<classpathentry including=\"**/*.java\" kind=\"src\" output=\"target/classes\" path=\"src/java\">\n\t\t<attributes>\n\t\t\t<attribute name=\"optional\" value=\"true\"/>\n\t\t\t<attribute name=\"maven.pomderived\" value=\"true\"/>\n\t\t</attributes>\n\t</classpathentry>\n\t<classpathentry excluding=\"**\" kind=\"src\" output=\"target/classes\" path=\"META-INF\">\n\t\t<attributes>\n\t\t\t<attribute name=\"maven.pomderived\" value=\"true\"/>\n\t\t</attributes>\n\t</classpathentry>\n\t<classpathentry kind=\"src\" output=\"target/test-classes\" path=\"src/test/java\">\n\t\t<attributes>\n\t\t\t<attribute name=\"optional\" value=\"true\"/>\n\t\t\t<attribute name=\"maven.pomderived\" value=\"true\"/>\n\t\t</attributes>\n\t</classpathentry>\n\t<classpathentry kind=\"con\" path=\"org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.6\">\n\t\t<attributes>\n\t\t\t<attribute name=\"maven.pomderived\" value=\"true\"/>\n\t\t</attributes>\n\t</classpathentry>\n\t<classpathentry kind=\"con\" path=\"org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER\">\n\t\t<attributes>\n\t\t\t<attribute name=\"maven.pomderived\" value=\"true\"/>\n\t\t</attributes>\n\t</classpathentry>\n\t<classpathentry kind=\"output\" path=\"target/classes\"/>\n</classpath>\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/.settings/org.eclipse.core.resources.prefs",
    "content": "eclipse.preferences.version=1\nencoding//src/java=UTF-8\nencoding/<project>=UTF-8\nencoding/META-INF=UTF-8\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/.settings/org.eclipse.jdt.core.prefs",
    "content": "eclipse.preferences.version=1\norg.eclipse.jdt.core.compiler.codegen.targetPlatform=1.6\norg.eclipse.jdt.core.compiler.compliance=1.6\norg.eclipse.jdt.core.compiler.problem.forbiddenReference=warning\norg.eclipse.jdt.core.compiler.source=1.6\norg.eclipse.jdt.core.formatter.align_type_members_on_columns=false\norg.eclipse.jdt.core.formatter.alignment_for_arguments_in_allocation_expression=16\norg.eclipse.jdt.core.formatter.alignment_for_arguments_in_annotation=0\norg.eclipse.jdt.core.formatter.alignment_for_arguments_in_enum_constant=16\norg.eclipse.jdt.core.formatter.alignment_for_arguments_in_explicit_constructor_call=16\norg.eclipse.jdt.core.formatter.alignment_for_arguments_in_method_invocation=16\norg.eclipse.jdt.core.formatter.alignment_for_arguments_in_qualified_allocation_expression=16\norg.eclipse.jdt.core.formatter.alignment_for_assignment=0\norg.eclipse.jdt.core.formatter.alignment_for_binary_expression=16\norg.eclipse.jdt.core.formatter.alignment_for_compact_if=16\norg.eclipse.jdt.core.formatter.alignment_for_conditional_expression=80\norg.eclipse.jdt.core.formatter.alignment_for_enum_constants=0\norg.eclipse.jdt.core.formatter.alignment_for_expressions_in_array_initializer=16\norg.eclipse.jdt.core.formatter.alignment_for_method_declaration=0\norg.eclipse.jdt.core.formatter.alignment_for_multiple_fields=16\norg.eclipse.jdt.core.formatter.alignment_for_parameters_in_constructor_declaration=16\norg.eclipse.jdt.core.formatter.alignment_for_parameters_in_method_declaration=16\norg.eclipse.jdt.core.formatter.alignment_for_resources_in_try=80\norg.eclipse.jdt.core.formatter.alignment_for_selector_in_method_invocation=16\norg.eclipse.jdt.core.formatter.alignment_for_superclass_in_type_declaration=16\norg.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_enum_declaration=16\norg.eclipse.jdt.core.formatter.alignment_for_superinterfaces_in_type_declaration=16\norg.eclipse.jdt.core.formatter.alignment_for_throws_clause_in_constructor_declaration=16\norg.eclipse.jdt.core.formatter.alignment_for_throws_clause_in_method_declaration=16\norg.eclipse.jdt.core.formatter.alignment_for_union_type_in_multicatch=16\norg.eclipse.jdt.core.formatter.blank_lines_after_imports=1\norg.eclipse.jdt.core.formatter.blank_lines_after_package=1\norg.eclipse.jdt.core.formatter.blank_lines_before_field=0\norg.eclipse.jdt.core.formatter.blank_lines_before_first_class_body_declaration=0\norg.eclipse.jdt.core.formatter.blank_lines_before_imports=1\norg.eclipse.jdt.core.formatter.blank_lines_before_member_type=1\norg.eclipse.jdt.core.formatter.blank_lines_before_method=1\norg.eclipse.jdt.core.formatter.blank_lines_before_new_chunk=1\norg.eclipse.jdt.core.formatter.blank_lines_before_package=0\norg.eclipse.jdt.core.formatter.blank_lines_between_import_groups=1\norg.eclipse.jdt.core.formatter.blank_lines_between_type_declarations=1\norg.eclipse.jdt.core.formatter.brace_position_for_annotation_type_declaration=end_of_line\norg.eclipse.jdt.core.formatter.brace_position_for_anonymous_type_declaration=end_of_line\norg.eclipse.jdt.core.formatter.brace_position_for_array_initializer=end_of_line\norg.eclipse.jdt.core.formatter.brace_position_for_block=end_of_line\norg.eclipse.jdt.core.formatter.brace_position_for_block_in_case=end_of_line\norg.eclipse.jdt.core.formatter.brace_position_for_constructor_declaration=end_of_line\norg.eclipse.jdt.core.formatter.brace_position_for_enum_constant=end_of_line\norg.eclipse.jdt.core.formatter.brace_position_for_enum_declaration=end_of_line\norg.eclipse.jdt.core.formatter.brace_position_for_method_declaration=end_of_line\norg.eclipse.jdt.core.formatter.brace_position_for_switch=end_of_line\norg.eclipse.jdt.core.formatter.brace_position_for_type_declaration=end_of_line\norg.eclipse.jdt.core.formatter.comment.clear_blank_lines_in_block_comment=false\norg.eclipse.jdt.core.formatter.comment.clear_blank_lines_in_javadoc_comment=false\norg.eclipse.jdt.core.formatter.comment.format_block_comments=true\norg.eclipse.jdt.core.formatter.comment.format_header=false\norg.eclipse.jdt.core.formatter.comment.format_html=true\norg.eclipse.jdt.core.formatter.comment.format_javadoc_comments=true\norg.eclipse.jdt.core.formatter.comment.format_line_comments=true\norg.eclipse.jdt.core.formatter.comment.format_source_code=true\norg.eclipse.jdt.core.formatter.comment.indent_parameter_description=true\norg.eclipse.jdt.core.formatter.comment.indent_root_tags=true\norg.eclipse.jdt.core.formatter.comment.insert_new_line_before_root_tags=insert\norg.eclipse.jdt.core.formatter.comment.insert_new_line_for_parameter=insert\norg.eclipse.jdt.core.formatter.comment.line_length=80\norg.eclipse.jdt.core.formatter.comment.new_lines_at_block_boundaries=true\norg.eclipse.jdt.core.formatter.comment.new_lines_at_javadoc_boundaries=true\norg.eclipse.jdt.core.formatter.comment.preserve_white_space_between_code_and_line_comments=false\norg.eclipse.jdt.core.formatter.compact_else_if=true\norg.eclipse.jdt.core.formatter.continuation_indentation=2\norg.eclipse.jdt.core.formatter.continuation_indentation_for_array_initializer=2\norg.eclipse.jdt.core.formatter.disabling_tag=@formatter\\:off\norg.eclipse.jdt.core.formatter.enabling_tag=@formatter\\:on\norg.eclipse.jdt.core.formatter.format_guardian_clause_on_one_line=false\norg.eclipse.jdt.core.formatter.format_line_comment_starting_on_first_column=true\norg.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_annotation_declaration_header=true\norg.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_enum_constant_header=true\norg.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_enum_declaration_header=true\norg.eclipse.jdt.core.formatter.indent_body_declarations_compare_to_type_header=true\norg.eclipse.jdt.core.formatter.indent_breaks_compare_to_cases=true\norg.eclipse.jdt.core.formatter.indent_empty_lines=false\norg.eclipse.jdt.core.formatter.indent_statements_compare_to_block=true\norg.eclipse.jdt.core.formatter.indent_statements_compare_to_body=true\norg.eclipse.jdt.core.formatter.indent_switchstatements_compare_to_cases=true\norg.eclipse.jdt.core.formatter.indent_switchstatements_compare_to_switch=false\norg.eclipse.jdt.core.formatter.indentation.size=4\norg.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_field=insert\norg.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_local_variable=insert\norg.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_method=insert\norg.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_package=insert\norg.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_parameter=do not insert\norg.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_type=insert\norg.eclipse.jdt.core.formatter.insert_new_line_after_label=do not insert\norg.eclipse.jdt.core.formatter.insert_new_line_after_opening_brace_in_array_initializer=do not insert\norg.eclipse.jdt.core.formatter.insert_new_line_at_end_of_file_if_missing=do not insert\norg.eclipse.jdt.core.formatter.insert_new_line_before_catch_in_try_statement=do not insert\norg.eclipse.jdt.core.formatter.insert_new_line_before_closing_brace_in_array_initializer=do not insert\norg.eclipse.jdt.core.formatter.insert_new_line_before_else_in_if_statement=do not insert\norg.eclipse.jdt.core.formatter.insert_new_line_before_finally_in_try_statement=do not insert\norg.eclipse.jdt.core.formatter.insert_new_line_before_while_in_do_statement=do not insert\norg.eclipse.jdt.core.formatter.insert_new_line_in_empty_annotation_declaration=insert\norg.eclipse.jdt.core.formatter.insert_new_line_in_empty_anonymous_type_declaration=insert\norg.eclipse.jdt.core.formatter.insert_new_line_in_empty_block=insert\norg.eclipse.jdt.core.formatter.insert_new_line_in_empty_enum_constant=insert\norg.eclipse.jdt.core.formatter.insert_new_line_in_empty_enum_declaration=insert\norg.eclipse.jdt.core.formatter.insert_new_line_in_empty_method_body=insert\norg.eclipse.jdt.core.formatter.insert_new_line_in_empty_type_declaration=insert\norg.eclipse.jdt.core.formatter.insert_space_after_and_in_type_parameter=insert\norg.eclipse.jdt.core.formatter.insert_space_after_assignment_operator=insert\norg.eclipse.jdt.core.formatter.insert_space_after_at_in_annotation=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_at_in_annotation_type_declaration=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_binary_operator=insert\norg.eclipse.jdt.core.formatter.insert_space_after_closing_angle_bracket_in_type_arguments=insert\norg.eclipse.jdt.core.formatter.insert_space_after_closing_angle_bracket_in_type_parameters=insert\norg.eclipse.jdt.core.formatter.insert_space_after_closing_brace_in_block=insert\norg.eclipse.jdt.core.formatter.insert_space_after_closing_paren_in_cast=insert\norg.eclipse.jdt.core.formatter.insert_space_after_colon_in_assert=insert\norg.eclipse.jdt.core.formatter.insert_space_after_colon_in_case=insert\norg.eclipse.jdt.core.formatter.insert_space_after_colon_in_conditional=insert\norg.eclipse.jdt.core.formatter.insert_space_after_colon_in_for=insert\norg.eclipse.jdt.core.formatter.insert_space_after_colon_in_labeled_statement=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_allocation_expression=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_annotation=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_array_initializer=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_constructor_declaration_parameters=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_constructor_declaration_throws=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_enum_constant_arguments=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_enum_declarations=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_explicitconstructorcall_arguments=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_for_increments=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_for_inits=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_declaration_parameters=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_declaration_throws=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_method_invocation_arguments=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_multiple_field_declarations=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_multiple_local_declarations=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_parameterized_type_reference=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_superinterfaces=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_type_arguments=insert\norg.eclipse.jdt.core.formatter.insert_space_after_comma_in_type_parameters=insert\norg.eclipse.jdt.core.formatter.insert_space_after_ellipsis=insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_parameterized_type_reference=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_type_arguments=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_angle_bracket_in_type_parameters=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_brace_in_array_initializer=insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_bracket_in_array_allocation_expression=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_bracket_in_array_reference=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_annotation=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_cast=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_catch=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_constructor_declaration=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_enum_constant=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_for=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_if=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_method_declaration=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_method_invocation=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_parenthesized_expression=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_switch=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_synchronized=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_try=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_opening_paren_in_while=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_postfix_operator=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_prefix_operator=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_question_in_conditional=insert\norg.eclipse.jdt.core.formatter.insert_space_after_question_in_wildcard=do not insert\norg.eclipse.jdt.core.formatter.insert_space_after_semicolon_in_for=insert\norg.eclipse.jdt.core.formatter.insert_space_after_semicolon_in_try_resources=insert\norg.eclipse.jdt.core.formatter.insert_space_after_unary_operator=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_and_in_type_parameter=insert\norg.eclipse.jdt.core.formatter.insert_space_before_assignment_operator=insert\norg.eclipse.jdt.core.formatter.insert_space_before_at_in_annotation_type_declaration=insert\norg.eclipse.jdt.core.formatter.insert_space_before_binary_operator=insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_parameterized_type_reference=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_type_arguments=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_angle_bracket_in_type_parameters=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_brace_in_array_initializer=insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_bracket_in_array_allocation_expression=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_bracket_in_array_reference=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_annotation=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_cast=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_catch=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_constructor_declaration=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_enum_constant=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_for=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_if=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_method_declaration=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_method_invocation=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_parenthesized_expression=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_switch=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_synchronized=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_try=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_closing_paren_in_while=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_colon_in_assert=insert\norg.eclipse.jdt.core.formatter.insert_space_before_colon_in_case=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_colon_in_conditional=insert\norg.eclipse.jdt.core.formatter.insert_space_before_colon_in_default=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_colon_in_for=insert\norg.eclipse.jdt.core.formatter.insert_space_before_colon_in_labeled_statement=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_allocation_expression=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_annotation=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_array_initializer=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_constructor_declaration_parameters=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_constructor_declaration_throws=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_enum_constant_arguments=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_enum_declarations=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_explicitconstructorcall_arguments=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_for_increments=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_for_inits=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_declaration_parameters=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_declaration_throws=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_method_invocation_arguments=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_multiple_field_declarations=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_multiple_local_declarations=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_parameterized_type_reference=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_superinterfaces=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_type_arguments=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_comma_in_type_parameters=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_ellipsis=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_parameterized_type_reference=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_type_arguments=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_angle_bracket_in_type_parameters=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_annotation_type_declaration=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_anonymous_type_declaration=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_array_initializer=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_block=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_constructor_declaration=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_enum_constant=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_enum_declaration=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_method_declaration=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_switch=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_brace_in_type_declaration=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_allocation_expression=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_reference=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_bracket_in_array_type_reference=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_annotation=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_annotation_type_member_declaration=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_catch=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_constructor_declaration=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_enum_constant=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_for=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_if=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_method_declaration=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_method_invocation=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_parenthesized_expression=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_switch=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_synchronized=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_try=insert\norg.eclipse.jdt.core.formatter.insert_space_before_opening_paren_in_while=insert\norg.eclipse.jdt.core.formatter.insert_space_before_parenthesized_expression_in_return=insert\norg.eclipse.jdt.core.formatter.insert_space_before_parenthesized_expression_in_throw=insert\norg.eclipse.jdt.core.formatter.insert_space_before_postfix_operator=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_prefix_operator=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_question_in_conditional=insert\norg.eclipse.jdt.core.formatter.insert_space_before_question_in_wildcard=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_semicolon=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_semicolon_in_for=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_semicolon_in_try_resources=do not insert\norg.eclipse.jdt.core.formatter.insert_space_before_unary_operator=do not insert\norg.eclipse.jdt.core.formatter.insert_space_between_brackets_in_array_type_reference=do not insert\norg.eclipse.jdt.core.formatter.insert_space_between_empty_braces_in_array_initializer=do not insert\norg.eclipse.jdt.core.formatter.insert_space_between_empty_brackets_in_array_allocation_expression=do not insert\norg.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_annotation_type_member_declaration=do not insert\norg.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_constructor_declaration=do not insert\norg.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_enum_constant=do not insert\norg.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_method_declaration=do not insert\norg.eclipse.jdt.core.formatter.insert_space_between_empty_parens_in_method_invocation=do not insert\norg.eclipse.jdt.core.formatter.join_lines_in_comments=true\norg.eclipse.jdt.core.formatter.join_wrapped_lines=true\norg.eclipse.jdt.core.formatter.keep_else_statement_on_same_line=false\norg.eclipse.jdt.core.formatter.keep_empty_array_initializer_on_one_line=false\norg.eclipse.jdt.core.formatter.keep_imple_if_on_one_line=false\norg.eclipse.jdt.core.formatter.keep_then_statement_on_same_line=false\norg.eclipse.jdt.core.formatter.lineSplit=80\norg.eclipse.jdt.core.formatter.never_indent_block_comments_on_first_column=false\norg.eclipse.jdt.core.formatter.never_indent_line_comments_on_first_column=false\norg.eclipse.jdt.core.formatter.number_of_blank_lines_at_beginning_of_method_body=0\norg.eclipse.jdt.core.formatter.number_of_empty_lines_to_preserve=1\norg.eclipse.jdt.core.formatter.put_empty_statement_on_new_line=true\norg.eclipse.jdt.core.formatter.tabulation.char=space\norg.eclipse.jdt.core.formatter.tabulation.size=4\norg.eclipse.jdt.core.formatter.use_on_off_tags=false\norg.eclipse.jdt.core.formatter.use_tabs_only_for_leading_indentations=false\norg.eclipse.jdt.core.formatter.wrap_before_binary_operator=true\norg.eclipse.jdt.core.formatter.wrap_before_or_operator_multicatch=true\norg.eclipse.jdt.core.formatter.wrap_outer_expressions_when_nested=true\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/.settings/org.eclipse.jdt.ui.prefs",
    "content": "eclipse.preferences.version=1\nformatter_profile=_Eclipse [built-in] (Whitspace only)\nformatter_settings_version=12\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/.settings/org.eclipse.m2e.core.prefs",
    "content": "activeProfiles=\neclipse.preferences.version=1\nresolveWorkspaceProjects=true\nversion=1\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/License.txt",
    "content": "/**********************************************************************\nCopyright (c) 2013 Orange. All rights reserved.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\nContributors:\n    ...\n **********************************************************************/\n\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/META-INF/LICENSE.txt",
    "content": "\r\n                                 Apache License\r\n                           Version 2.0, January 2004\r\n                        http://www.apache.org/licenses/\r\n\r\n   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\r\n\r\n   1. Definitions.\r\n\r\n      \"License\" shall mean the terms and conditions for use, reproduction,\r\n      and distribution as defined by Sections 1 through 9 of this document.\r\n\r\n      \"Licensor\" shall mean the copyright owner or entity authorized by\r\n      the copyright owner that is granting the License.\r\n\r\n      \"Legal Entity\" shall mean the union of the acting entity and all\r\n      other entities that control, are controlled by, or are under common\r\n      control with that entity. For the purposes of this definition,\r\n      \"control\" means (i) the power, direct or indirect, to cause the\r\n      direction or management of such entity, whether by contract or\r\n      otherwise, or (ii) ownership of fifty percent (50%) or more of the\r\n      outstanding shares, or (iii) beneficial ownership of such entity.\r\n\r\n      \"You\" (or \"Your\") shall mean an individual or Legal Entity\r\n      exercising permissions granted by this License.\r\n\r\n      \"Source\" form shall mean the preferred form for making modifications,\r\n      including but not limited to software source code, documentation\r\n      source, and configuration files.\r\n\r\n      \"Object\" form shall mean any form resulting from mechanical\r\n      transformation or translation of a Source form, including but\r\n      not limited to compiled object code, generated documentation,\r\n      and conversions to other media types.\r\n\r\n      \"Work\" shall mean the work of authorship, whether in Source or\r\n      Object form, made available under the License, as indicated by a\r\n      copyright notice that is included in or attached to the work\r\n      (an example is provided in the Appendix below).\r\n\r\n      \"Derivative Works\" shall mean any work, whether in Source or Object\r\n      form, that is based on (or derived from) the Work and for which the\r\n      editorial revisions, annotations, elaborations, or other modifications\r\n      represent, as a whole, an original work of authorship. For the purposes\r\n      of this License, Derivative Works shall not include works that remain\r\n      separable from, or merely link (or bind by name) to the interfaces of,\r\n      the Work and Derivative Works thereof.\r\n\r\n      \"Contribution\" shall mean any work of authorship, including\r\n      the original version of the Work and any modifications or additions\r\n      to that Work or Derivative Works thereof, that is intentionally\r\n      submitted to Licensor for inclusion in the Work by the copyright owner\r\n      or by an individual or Legal Entity authorized to submit on behalf of\r\n      the copyright owner. For the purposes of this definition, \"submitted\"\r\n      means any form of electronic, verbal, or written communication sent\r\n      to the Licensor or its representatives, including but not limited to\r\n      communication on electronic mailing lists, source code control systems,\r\n      and issue tracking systems that are managed by, or on behalf of, the\r\n      Licensor for the purpose of discussing and improving the Work, but\r\n      excluding communication that is conspicuously marked or otherwise\r\n      designated in writing by the copyright owner as \"Not a Contribution.\"\r\n\r\n      \"Contributor\" shall mean Licensor and any individual or Legal Entity\r\n      on behalf of whom a Contribution has been received by Licensor and\r\n      subsequently incorporated within the Work.\r\n\r\n   2. Grant of Copyright License. Subject to the terms and conditions of\r\n      this License, each Contributor hereby grants to You a perpetual,\r\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\r\n      copyright license to reproduce, prepare Derivative Works of,\r\n      publicly display, publicly perform, sublicense, and distribute the\r\n      Work and such Derivative Works in Source or Object form.\r\n\r\n   3. Grant of Patent License. Subject to the terms and conditions of\r\n      this License, each Contributor hereby grants to You a perpetual,\r\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\r\n      (except as stated in this section) patent license to make, have made,\r\n      use, offer to sell, sell, import, and otherwise transfer the Work,\r\n      where such license applies only to those patent claims licensable\r\n      by such Contributor that are necessarily infringed by their\r\n      Contribution(s) alone or by combination of their Contribution(s)\r\n      with the Work to which such Contribution(s) was submitted. If You\r\n      institute patent litigation against any entity (including a\r\n      cross-claim or counterclaim in a lawsuit) alleging that the Work\r\n      or a Contribution incorporated within the Work constitutes direct\r\n      or contributory patent infringement, then any patent licenses\r\n      granted to You under this License for that Work shall terminate\r\n      as of the date such litigation is filed.\r\n\r\n   4. Redistribution. You may reproduce and distribute copies of the\r\n      Work or Derivative Works thereof in any medium, with or without\r\n      modifications, and in Source or Object form, provided that You\r\n      meet the following conditions:\r\n\r\n      (a) You must give any other recipients of the Work or\r\n          Derivative Works a copy of this License; and\r\n\r\n      (b) You must cause any modified files to carry prominent notices\r\n          stating that You changed the files; and\r\n\r\n      (c) You must retain, in the Source form of any Derivative Works\r\n          that You distribute, all copyright, patent, trademark, and\r\n          attribution notices from the Source form of the Work,\r\n          excluding those notices that do not pertain to any part of\r\n          the Derivative Works; and\r\n\r\n      (d) If the Work includes a \"NOTICE\" text file as part of its\r\n          distribution, then any Derivative Works that You distribute must\r\n          include a readable copy of the attribution notices contained\r\n          within such NOTICE file, excluding those notices that do not\r\n          pertain to any part of the Derivative Works, in at least one\r\n          of the following places: within a NOTICE text file distributed\r\n          as part of the Derivative Works; within the Source form or\r\n          documentation, if provided along with the Derivative Works; or,\r\n          within a display generated by the Derivative Works, if and\r\n          wherever such third-party notices normally appear. The contents\r\n          of the NOTICE file are for informational purposes only and\r\n          do not modify the License. You may add Your own attribution\r\n          notices within Derivative Works that You distribute, alongside\r\n          or as an addendum to the NOTICE text from the Work, provided\r\n          that such additional attribution notices cannot be construed\r\n          as modifying the License.\r\n\r\n      You may add Your own copyright statement to Your modifications and\r\n      may provide additional or different license terms and conditions\r\n      for use, reproduction, or distribution of Your modifications, or\r\n      for any such Derivative Works as a whole, provided Your use,\r\n      reproduction, and distribution of the Work otherwise complies with\r\n      the conditions stated in this License.\r\n\r\n   5. Submission of Contributions. Unless You explicitly state otherwise,\r\n      any Contribution intentionally submitted for inclusion in the Work\r\n      by You to the Licensor shall be under the terms and conditions of\r\n      this License, without any additional terms or conditions.\r\n      Notwithstanding the above, nothing herein shall supersede or modify\r\n      the terms of any separate license agreement you may have executed\r\n      with Licensor regarding such Contributions.\r\n\r\n   6. Trademarks. This License does not grant permission to use the trade\r\n      names, trademarks, service marks, or product names of the Licensor,\r\n      except as required for reasonable and customary use in describing the\r\n      origin of the Work and reproducing the content of the NOTICE file.\r\n\r\n   7. Disclaimer of Warranty. Unless required by applicable law or\r\n      agreed to in writing, Licensor provides the Work (and each\r\n      Contributor provides its Contributions) on an \"AS IS\" BASIS,\r\n      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\r\n      implied, including, without limitation, any warranties or conditions\r\n      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\r\n      PARTICULAR PURPOSE. You are solely responsible for determining the\r\n      appropriateness of using or redistributing the Work and assume any\r\n      risks associated with Your exercise of permissions under this License.\r\n\r\n   8. Limitation of Liability. In no event and under no legal theory,\r\n      whether in tort (including negligence), contract, or otherwise,\r\n      unless required by applicable law (such as deliberate and grossly\r\n      negligent acts) or agreed to in writing, shall any Contributor be\r\n      liable to You for damages, including any direct, indirect, special,\r\n      incidental, or consequential damages of any character arising as a\r\n      result of this License or out of the use or inability to use the\r\n      Work (including but not limited to damages for loss of goodwill,\r\n      work stoppage, computer failure or malfunction, or any and all\r\n      other commercial damages or losses), even if such Contributor\r\n      has been advised of the possibility of such damages.\r\n\r\n   9. Accepting Warranty or Additional Liability. While redistributing\r\n      the Work or Derivative Works thereof, You may choose to offer,\r\n      and charge a fee for, acceptance of support, warranty, indemnity,\r\n      or other liability obligations and/or rights consistent with this\r\n      License. However, in accepting such obligations, You may act only\r\n      on Your own behalf and on Your sole responsibility, not on behalf\r\n      of any other Contributor, and only if You agree to indemnify,\r\n      defend, and hold each Contributor harmless for any liability\r\n      incurred by, or claims asserted against, such Contributor by reason\r\n      of your accepting any such warranty or additional liability.\r\n\r\n   END OF TERMS AND CONDITIONS\r\n\r\n   Copyright 2008-2008 DataNucleus\r\n\r\n   Licensed under the Apache License, Version 2.0 (the \"License\");\r\n   you may not use this file except in compliance with the License.\r\n   You may obtain a copy of the License at\r\n\r\n       http://www.apache.org/licenses/LICENSE-2.0\r\n\r\n   Unless required by applicable law or agreed to in writing, software\r\n   distributed under the License is distributed on an \"AS IS\" BASIS,\r\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\n   See the License for the specific language governing permissions and\r\n   limitations under the License.\r\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/META-INF/MANIFEST.MF",
    "content": "Manifest-Version: 1.0\nBundle-ManifestVersion: 2\nBundle-Name: JSON Plug-in\nBundle-SymbolicName: org.datanucleus.store.scalaris;singleton:=true\nBundle-Version: 3.2.0.release\nBundle-Vendor: com.orange.DataNucleus\nRequire-Bundle: org.datanucleus;bundle-version=\"3.2.0.m3\"\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/META-INF/NOTICE.txt",
    "content": "=========================================================================\r\n==  NOTICE file corresponding to section 4(d) of the Apache License,   ==\r\n==  Version 2.0, in this case for the DataNucleus distribution.        ==\r\n=========================================================================\r\n\r\n===================================================================\r\nThis product includes software developed by many individuals,\r\nincluding the following:\r\n===================================================================\r\nErik Bengtson\r\nAndy Jefferson\r\n\r\n\r\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/maven_git_hook.sh",
    "content": "#!/bin/bash\n# Deploy hook functions to glue maven and svn together\n# This should be called by the command \"maven deploy\".\n\n# scalaris maven repo\nurl=\"git@github.com:scalaris-team/scalaris.git\"\n# maven repo checkout folder\nfolder=\"../../../.maven\"\n\ncheckout () {\n    # check out if maven folder doesn't exist\n    # otherwise update\n    if [ ! -d ${folder} ]; then\n        echo \"checkout ${url} -> ${folder} ...\"\n        git clone --branch gh-pages --single-branch \"${url}\" \"${folder}\"\n        result=$?\n    else\n        echo \"update ${url} -> ${folder} ...\"\n        cd \"${folder}\"\n        git pull\n        result=$?\n        cd - >/dev/null\n    fi\n\n    if [ ${result} -eq 0 ]; then\n        echo \"Maven repository has been updated locally.\"\n    else\n        echo \"Maven repository couldn't be updated.\"\n        exit 1\n    fi\n}\n\ncommit () {\n    # update the remote maven repository\n    echo -n \"Do you want to update the remote maven repository? [y/N] \"\n    read -e answer\n    if [[ ${answer} == \"y\" ]]; then\n        cd \"${folder}\"\n        git add maven\n        git commit\n        git push\n        cd - >/dev/null\n    fi\n}\n\n\nif [[ $1 == \"checkout\" ]]; then\n    checkout\nelif [[ $1 == \"commit\" ]]; then\n    commit\nelse\n    echo \"Missing an argument.\"\n    exit 1\nfi\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/mkThirdPartyLicenseFile.sh",
    "content": "#!/bin/bash -e\n\n\nmvn license:aggregate-add-third-party -PlicenseReport\ncp target/generated-sources/license/THIRD-PARTY.txt Licences-Third-Party.txt \n\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/plugin.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<?eclipse version=\"3.2\"?>\r\n<plugin>\r\n    <extension point=\"org.datanucleus.store_manager\">\r\n        <store-manager\r\n            class-name=\"org.datanucleus.store.scalaris.ScalarisStoreManager\"\r\n            key=\"scalaris\" url-key=\"scalaris\">\r\n        </store-manager>\r\n    </extension>\r\n\r\n    <extension point=\"org.datanucleus.store_connectionfactory\">\r\n        <connectionfactory\r\n            class-name=\"org.datanucleus.store.scalaris.ConnectionFactoryImpl\"\r\n            datastore=\"scalaris\" name=\"scalaris/tx\" transactional=\"true\">\r\n            <persistence-property name=\"scalaris.cookie\" />\r\n            <persistence-property name=\"scalaris.node\" />\r\n            <persistence-property name=\"scalaris.client.name\" />\r\n            <persistence-property name=\"scalaris.client.appendUUID\" />\r\n            <persistence-property name=\"scalaris.debug\"/>\r\n        </connectionfactory>\r\n        <connectionfactory\r\n            class-name=\"org.datanucleus.store.scalaris.ConnectionFactoryImpl\"\r\n            datastore=\"scalaris\" name=\"scalaris/nontx\" transactional=\"false\" />\r\n    </extension>\r\n\r\n    <extension point=\"org.datanucleus.store_query_query\">\r\n        <query class-name=\"org.datanucleus.store.scalaris.query.JPQLQuery\"\r\n            datastore=\"scalaris\" name=\"JPQL\">\r\n        </query>\r\n        <query class-name=\"org.datanucleus.store.scalaris.query.JDOQLQuery\"\r\n            datastore=\"scalaris\" name=\"JDOQL\">\r\n        </query>\r\n    </extension>\r\n\r\n    <extension point=\"org.datanucleus.persistence_properties\">\r\n        <persistence-property name=\"datanucleus.cloud.storage.bucket\" />\r\n    </extension>\r\n\r\n</plugin>\r\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/pom.xml",
    "content": "<project xmlns=\"http://maven.apache.org/POM/4.0.0\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n    xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd\">\n    <modelVersion>4.0.0</modelVersion>\n\n    <groupId>de.zib.scalaris</groupId>\n    <version>0.9.1-SNAPSHOT</version>\n    <artifactId>datanucleus-store</artifactId>\n    <name>datanucleus.store.scalaris</name>\n\n    <description>&quot;Scalaris&quot; data store plugin for Datanucleus deriving from  Datanucleus-json plugin.\n\t\n\tThe research leading to these results has received funding from the European Union’s Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 258862.\n\n</description>\n    <url>http://www.datanucleus.org</url>\n\n    <properties>\n        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>\n    </properties>\n\n    <licenses>\n        <license>\n            <name>The Apache Software License, Version 2.0</name>\n            <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>\n        </license>\n    </licenses>\n    <inceptionYear>2012</inceptionYear>\n    <organization>\n        <name>Orange</name>\n    </organization>\n\n    <repositories>\n        <repository>\n            <id>scalaris-repo</id>\n            <url>https://scalaris-team.github.io/scalaris/maven</url>\n        </repository>\n    </repositories>\n\n    <distributionManagement>\n        <repository>\n            <id>scalaris</id>\n            <url>file:../../../.maven/maven</url>\n        </repository>\n    </distributionManagement>\n\n    <dependencies>\n        <dependency>\n            <groupId>org.datanucleus</groupId>\n            <artifactId>datanucleus-core</artifactId>\n            <version>4.1.0-release</version>\n            <scope>provided</scope>\n        </dependency>\n        <dependency>\n            <groupId>de.zib.scalaris</groupId>\n            <artifactId>java-api</artifactId>\n            <version>[0.7.2,)</version>\n        </dependency>\n        <dependency>\n        <groupId>org.json</groupId>\n            <artifactId>json</artifactId>\n            <version>20231013</version>\n        </dependency>\n        <dependency>\n            <!-- Scalaris does not set its dependencies correctly, add here: -->\n            <groupId>org.erlang.otp</groupId>\n            <artifactId>jinterface</artifactId>\n            <version>[1.5.6-custom,)</version>\n        </dependency>\n    </dependencies>\n\n    <!-- Build process -->\n    <build>\n        <sourceDirectory>src/java</sourceDirectory>\n        <resources>\n            <resource>\n                <filtering>true</filtering>\n                <directory>src/java</directory>\n                <includes>\n                    <include>**/plugin.xml</include>\n                    <include>**/*.properties</include>\n                </includes>\n            </resource>\n            <resource>\n                <directory>${basedir}</directory>\n                <includes>\n                    <include>plugin.xml</include>\n                </includes>\n            </resource>\n            <resource>\n                <!-- Need to have MANIFEST-MF in place before packaging, \n                    so unit-tests will work -->\n                <directory>${basedir}/META-INF</directory>\n                <targetPath>META-INF</targetPath>\n                <includes>\n                    <include>MANIFEST.MF</include>\n                    <include>LICENSE.txt</include>\n                    <include>NOTICE.txt</include>\n                </includes>\n            </resource>\n        </resources>\n\n        <plugins>\n            <plugin>\n                <artifactId>maven-compiler-plugin</artifactId>\n                <version>3.1</version>\n                <configuration>\n                    <source>1.6</source>\n                    <target>1.6</target>\n                </configuration>\n            </plugin>\n            <plugin>\n                <artifactId>maven-jar-plugin</artifactId>\n                <version>2.3.2</version>\n                <configuration>\n                    <archive>\n                        <manifestFile>META-INF/MANIFEST.MF</manifestFile>\n                    </archive>\n                </configuration>\n            </plugin>\n            <plugin>\n                <groupId>org.apache.maven.plugins</groupId>\n                <artifactId>maven-source-plugin</artifactId>\n                <version>2.2.1</version>\n                <executions>\n                    <execution>\n                        <id>attach-sources</id>\n                        <goals>\n                            <goal>jar</goal>\n                        </goals>\n                    </execution>\n                </executions>\n            </plugin>\n            <plugin>\n                <artifactId>exec-maven-plugin</artifactId>\n                <groupId>org.codehaus.mojo</groupId>\n                <version>1.4.0</version>\n                <executions>\n                    <execution>\n                        <id>Check out maven repository from git</id>\n                        <!-- Execute before deploy phase -->\n                        <!-- http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html -->\n                        <phase>install</phase>\n                        <goals>\n                            <goal>exec</goal>\n                        </goals>\n                        <configuration>\n                            <skip>${skipGit}</skip>\n                            <executable>maven_git_hook.sh</executable>\n                            <arguments>\n                                <argument>checkout</argument>\n                            </arguments>\n                        </configuration>\n                    </execution>\n                    <execution>\n                        <id>Update the scalaris maven repository</id>\n                        <phase>deploy</phase>\n                        <goals>\n                            <goal>exec</goal>\n                        </goals>\n                        <configuration>\n                            <skip>${skipGit}</skip>\n                            <executable>maven_git_hook.sh</executable>\n                            <arguments>\n                                <argument>commit</argument>\n                            </arguments>\n                        </configuration>\n                    </execution>\n                </executions>\n            </plugin>\n        </plugins>\n    </build>\n</project>\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/src/java/org/datanucleus/store/scalaris/ConnectionFactoryImpl.java",
    "content": "/**********************************************************************\r\nCopyright (c) 2008 Erik Bengtson and others. All rights reserved.\r\nLicensed under the Apache License, Version 2.0 (the \"License\");\r\nyou may not use this file except in compliance with the License.\r\nYou may obtain a copy of the License at\r\n\r\n    http://www.apache.org/licenses/LICENSE-2.0\r\n\r\nUnless required by applicable law or agreed to in writing, software\r\ndistributed under the License is distributed on an \"AS IS\" BASIS,\r\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nSee the License for the specific language governing permissions and\r\nlimitations under the License.\r\n\r\nContributors:\r\n2013 Orange - port to Scalaris key/value store\r\n    ...\r\n **********************************************************************/\r\npackage org.datanucleus.store.scalaris;\r\n\r\nimport java.util.Map;\r\nimport java.util.Properties;\r\n\r\nimport javax.transaction.xa.XAResource;\r\n\r\nimport org.datanucleus.ExecutionContext;\r\nimport org.datanucleus.exceptions.NucleusDataStoreException;\r\nimport org.datanucleus.store.StoreManager;\r\nimport org.datanucleus.store.connection.AbstractConnectionFactory;\r\nimport org.datanucleus.store.connection.AbstractManagedConnection;\r\nimport org.datanucleus.store.connection.ManagedConnection;\r\n\r\nimport de.zib.scalaris.ConnectionException;\r\nimport de.zib.scalaris.ConnectionFactory;\r\nimport de.zib.scalaris.ConnectionPool;\r\n\r\n/**\r\n * Implementation of a ConnectionFactory for Scalaris.\r\n */\r\n@SuppressWarnings(\"rawtypes\")\r\npublic class ConnectionFactoryImpl extends AbstractConnectionFactory {\r\n\r\n    /**\r\n     * Maximum number of open connections at the same time.\r\n     */\r\n    private static final int DEFAULT_MAX_CONNECTIONS = 50;\r\n    public static final String PROPERTY_MAX_CONNECTIONS = \"scalaris.connection.max\";\r\n\r\n    /**\r\n     * Period of time (in ms) which will be waited for a connection\r\n     * to get released if maximum number of connections is reached.\r\n     */\r\n    private static final int DEFAULT_CONNECTION_TIMEOUT = 200;\r\n    public static final String PROPERTY_CONNECTION_TIMEOUT = \"scalaris.connection.timeout\";\r\n    private static int newConnectionTimeout;\r\n\r\n\r\n    /**\r\n     * Symbolic Name of property used in persistence-unit configuration file.\r\n     * Property value Cookie used for connecting to Scalaris node. <property\r\n     * name=\"...\" value=\"...\" />\r\n     */\r\n    public final static String PROPERTY_SCALARIS_COOKIE = \"scalaris.cookie\";\r\n    /**\r\n     * Symbolic Name of property used in persistence-unit configuration file.\r\n     * Should be the same as defined in Scalaris server <property name=\"...\"\r\n     * value=\"...\" />\r\n     */\r\n    public final static String PROPERTY_SCALARIS_DEBUG = \"scalaris.debug\";\r\n    /**\r\n     * Symbolic Name of property used in persistence-unit configuration file\r\n     * <property name=\"...\" value=\"...\" />\r\n     */\r\n    public final static String PROPERTY_SCALARIS_NODE = \"scalaris.node\";\r\n    /**\r\n     * Symbolic Name of property used in persistence-unit configuration file\r\n     * <property name=\"...\" value=\"...\" />\r\n     */\r\n    public final static String PROPERTY_SCALARIS_CLIENT_NAME = \"scalaris.client.name\";\r\n    /**\r\n     * Symbolic Name of property used in persistence-unit configuration file\r\n     * <property name=\"...\" value=\"...\" />\r\n     */\r\n    public final static String PROPERTY_SCALARIS_CLIENT_APPENDUUID = \"scalaris.client.appendUUID\";\r\n\r\n    private static volatile ConnectionPool connPool = null;\r\n\r\n    /**\r\n     * Constructor.\r\n     * \r\n     * @param storeMgr\r\n     *            Store Manager\r\n     * @param resourceType\r\n     *            Type of resource (tx, nontx)\r\n     */\r\n    public ConnectionFactoryImpl(final StoreManager storeMgr,\r\n            final String resourceType) {\r\n        super(storeMgr, resourceType);\r\n\r\n        initConnectionPool();\r\n    }\r\n\r\n    private void initConnectionPool() {\r\n        synchronized(ConnectionFactoryImpl.class) {\r\n            if (connPool == null) {\r\n                Properties properties = new Properties();\r\n                copyProperty(PROPERTY_SCALARIS_NODE, properties, \"scalaris.node\",\r\n                        (String) null);\r\n                copyProperty(PROPERTY_SCALARIS_COOKIE, properties,\r\n                        \"scalaris.cookie\", (String) null);\r\n                copyProperty(PROPERTY_SCALARIS_CLIENT_APPENDUUID, properties,\r\n                        \"scalaris.client.appendUUID\", true);\r\n                ConnectionFactory connectionFactory = new ConnectionFactory(properties);\r\n\r\n                int maxConnectionNum = getIntProperty(PROPERTY_MAX_CONNECTIONS, DEFAULT_MAX_CONNECTIONS);\r\n                newConnectionTimeout = getIntProperty(PROPERTY_CONNECTION_TIMEOUT, DEFAULT_CONNECTION_TIMEOUT);\r\n\r\n                connPool = new ConnectionPool(connectionFactory, maxConnectionNum);\r\n            }\r\n        }\r\n    }\r\n\r\n    /**\r\n     * Helper for reading a integer property with default value\r\n     * @return\r\n     */\r\n    int getIntProperty(String propertyName, int defaultValue) {\r\n        String stringVal = storeMgr.getStringProperty(propertyName);\r\n        return stringVal == null ? defaultValue : Integer.parseInt(stringVal);\r\n    }\r\n\r\n    /**\r\n     * Helper for copying setting from Datanucleus to Scalaris\r\n     * \r\n     * @return\r\n     * */\r\n    String copyProperty(final String propertyNameFrom,\r\n            final Properties propertiesTo, final String propertyNameTo,\r\n            final String defaultIfNotSet) {\r\n        final String v = storeMgr.getStringProperty(propertyNameFrom);\r\n        if (defaultIfNotSet == null && v == null) {\r\n            throw new NucleusDataStoreException(\"Property \"\r\n                    + propertyNameFrom + \" is mandatory\");\r\n        }\r\n        final String ret = v == null ? defaultIfNotSet : v;\r\n        propertiesTo.put(propertyNameTo, ret);\r\n        return ret;\r\n    }\r\n\r\n    /**\r\n     * Helper for copying setting from Datanucleus to Scalaris\r\n     * \r\n     * @return\r\n     * */\r\n    boolean copyProperty(final String propertyNameFrom,\r\n            final Properties propertiesTo, final String propertyNameTo,\r\n            final boolean defaultIfNotSet) {\r\n        final boolean ret = storeMgr.getBooleanProperty(propertyNameFrom,\r\n                defaultIfNotSet);\r\n        if (propertyNameTo != null)\r\n            propertiesTo.put(propertyNameTo, ret);\r\n        return ret;\r\n    }\r\n    \r\n    /**\r\n     * Obtain a connection from the Factory. The connection will be enlisted\r\n     * within the transaction associated to the ExecutionContext\r\n     * \r\n     * @param ec\r\n     *            the pool that is bound the connection during its lifecycle (or\r\n     *            null)\r\n     * @param txnOptions\r\n     *            Any options for creating the connection\r\n     * @return the {@link org.datanucleus.store.connection.ManagedConnection}\r\n     */\r\n    public ManagedConnection createManagedConnection(final ExecutionContext ec,\r\n            final Map txnOptions) {\r\n        return new ManagedConnectionImpl(txnOptions);\r\n    }\r\n\r\n    /**\r\n     * Implementation of a ManagedConnection for Scalaris database.\r\n     */\r\n    public static class ManagedConnectionImpl extends AbstractManagedConnection {\r\n\r\n        public ManagedConnectionImpl(final Map optionsl) {\r\n            // TODO: handle options?\r\n        }\r\n\r\n        public synchronized Object getConnection() {\r\n            try {\r\n                if (conn == null) {\r\n                    conn = connPool.getConnection(newConnectionTimeout);\r\n                    if (conn == null) {\r\n                        throw new ConnectionException(\"Timeout when waiting for a new connection \");\r\n                    }\r\n                }\r\n            } catch (ConnectionException e) {\r\n                throw new NucleusDataStoreException(\"Could not create a connection\", e);\r\n            }\r\n            return conn;\r\n        }\r\n\r\n        public XAResource getXAResource() {\r\n            return null;\r\n        }\r\n\r\n        public synchronized void close() {\r\n            if (conn != null) {\r\n                connPool.releaseConnection((de.zib.scalaris.Connection) conn);\r\n                conn = null;\r\n            }\r\n        }\r\n\r\n        @Override\r\n        public boolean closeAfterTransactionEnd() {\r\n            // Do NOT close immediately after datanucleus transaction ends,\r\n            // since the corresponding scalaris transaction can be still\r\n            // active for a short period of time, causing the connection pool\r\n            // to release the connection a little bit to early - early enough\r\n            // so that this connection could be reused in rare cases in a\r\n            // different thread. This causes potentially UnknownExceptions\r\n            // since Connection objects are not thread safe.\r\n            return false;\r\n        }\r\n\r\n        @Override\r\n        public void setCloseOnRelease(boolean closeOnRelease) {\r\n            // Because of the override of closeAfterTransactionEnd\r\n            // Datanucleus tries to set this to false. But this value\r\n            // needs to be true, otherwise close() won't be called and\r\n            // the connection pool will never release any connections.\r\n            this.closeOnRelease = true;\r\n        }\r\n    }\r\n}\r\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/src/java/org/datanucleus/store/scalaris/Localisation.properties",
    "content": "################################################################################\r\n# Copyright (c) 2009 Erik Bengtson and others. All rights reserved. \r\n# Licensed under the Apache License, Version 2.0 (the \"License\");\r\n# you may not use this file except in compliance with the License.\r\n# You may obtain a copy of the License at\r\n#\r\n#     http://www.apache.org/licenses/LICENSE-2.0\r\n#\r\n# Unless required by applicable law or agreed to in writing, software\r\n# distributed under the License is distributed on an \"AS IS\" BASIS,\r\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\n# See the License for the specific language governing permissions and\r\n# limitations under the License.\r\n#\r\n# Contributions :\r\n# 2011 Andy Jefferson - all but the first two lines\r\n#     ...\r\n################################################################################\r\n\r\nScalaris.DatastoreID=Datastore ID not supported for this datastore, but class \"{0}\" uses it.\r\n\r\nScalaris.Insert.Start=Object \"{0}\" being inserted into Scalaris with all reachable objects\r\nScalaris.Insert.ObjectWithIdAlreadyExists=Object \"{0}\" being inserted has id \"{1}\" yet an object with this id already exists in the datastore!\r\nScalaris.Insert.ObjectPersistedWithVersion=Object \"{0}\" (id=\"{1}) persisted to Scalaris with version \"{2}\"\r\nScalaris.Update.Start=Object \"{0}\" (id=\"{1}\") being updated in Scalaris (for fields \"{2}\") with all reachable objects\r\nScalaris.Delete.Start=Object \"{0}\" (id=\"{1}\") being deleted from Scalaris with all dependent objects\r\nScalaris.Delete.ObjectDeleted=Object \"{0}\" (id=\"{1}\") has been deleted\r\nScalaris.Fetch.Start=Object \"{0}\" (id=\"{1}\") being retrieved from Scalaris\r\nScalaris.ExecutionTime=Execution Time = {0} ms\r\n\r\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/src/java/org/datanucleus/store/scalaris/ScalarisPersistenceHandler.java",
    "content": "/**********************************************************************\r\nCopyright (c) 2008 Erik Bengtson and others. All rights reserved.\r\nLicensed under the Apache License, Version 2.0 (the \"License\");\r\nyou may not use this file except in compliance with the License.\r\nYou may obtain a copy of the License at\r\n\r\n    http://www.apache.org/licenses/LICENSE-2.0\r\n\r\nUnless required by applicable law or agreed to in writing, software\r\ndistributed under the License is distributed on an \"AS IS\" BASIS,\r\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nSee the License for the specific language governing permissions and\r\nlimitations under the License.\r\n\r\nContributors:\r\n2013 Orange - port to Scalaris key/value store\r\n    ...\r\n **********************************************************************/\r\npackage org.datanucleus.store.scalaris;\r\n\r\nimport java.util.ArrayList;\r\nimport java.util.List;\r\n\r\nimport org.datanucleus.ExecutionContext;\r\nimport org.datanucleus.exceptions.NucleusDataStoreException;\r\nimport org.datanucleus.exceptions.NucleusException;\r\nimport org.datanucleus.exceptions.NucleusObjectNotFoundException;\r\nimport org.datanucleus.identity.IdentityUtils;\r\nimport org.datanucleus.metadata.AbstractClassMetaData;\r\nimport org.datanucleus.state.ObjectProvider;\r\nimport org.datanucleus.store.AbstractPersistenceHandler;\r\nimport org.datanucleus.store.StoreManager;\r\nimport org.datanucleus.store.connection.ManagedConnection;\r\nimport org.datanucleus.store.scalaris.fieldmanager.FetchFieldManager;\r\nimport org.datanucleus.store.scalaris.fieldmanager.StoreFieldManager;\r\nimport org.datanucleus.util.Localiser;\r\nimport org.datanucleus.util.NucleusLogger;\r\n\r\nimport org.json.JSONObject;\r\n\r\nimport de.zib.scalaris.AbortException;\r\nimport de.zib.scalaris.ConnectionException;\r\nimport de.zib.scalaris.NotFoundException;\r\nimport de.zib.scalaris.Transaction;\r\nimport de.zib.scalaris.TransactionSingleOp;\r\nimport de.zib.scalaris.UnknownException;\r\n\r\n@SuppressWarnings(\"rawtypes\")\r\npublic class ScalarisPersistenceHandler extends AbstractPersistenceHandler {\r\n\r\n    /** Setup localizer for messages. */\r\n    static {\r\n        Localiser.registerBundle(\"org.datanucleus.store.scalaris.Localisation\",\r\n                ScalarisStoreManager.class.getClassLoader());\r\n    }\r\n\r\n    ScalarisPersistenceHandler(StoreManager storeMgr) {\r\n        super(storeMgr);\r\n    }\r\n\r\n    public void close() {\r\n        // nothing to do\r\n    }\r\n\r\n    /**\r\n     * Populates JSONObject with information of the object provider\r\n     * \r\n     * @param jsonobj\r\n     *            updated with data from op\r\n     * @param op\r\n     *            data source\r\n     * @return primary key as string\r\n     */\r\n    private void populateJsonObj(JSONObject jsonobj, ObjectProvider op) {\r\n        AbstractClassMetaData acmd = op.getClassMetaData();\r\n        int[] fieldNumbers = acmd.getAllMemberPositions();\r\n        op.provideFields(fieldNumbers, new StoreFieldManager(op, jsonobj, true));\r\n    }\r\n\r\n\r\n    public void insertObject(ObjectProvider op) {\r\n        // Check if read-only so update not permitted\r\n        assertReadOnlyForUpdateOfObject(op);\r\n        ExecutionContext ec = op.getExecutionContext();\r\n        Transaction scalarisTransaction = ((ScalarisStoreManager) storeMgr)\r\n                .getScalarisTransaction(ec);\r\n        boolean dnTransactionStarted = scalarisTransaction != null;\r\n\r\n        ManagedConnection mConn = null;\r\n        try {\r\n            if (!dnTransactionStarted) {\r\n                mConn = storeMgr.getConnection(ec);\r\n                de.zib.scalaris.Connection scalarisConnection =\r\n                        (de.zib.scalaris.Connection) mConn.getConnection();\r\n                scalarisTransaction = new Transaction(scalarisConnection);\r\n            }\r\n\r\n            long startTime = System.currentTimeMillis();\r\n            if (NucleusLogger.DATASTORE_PERSIST.isDebugEnabled()) {\r\n                NucleusLogger.DATASTORE_PERSIST.debug(Localiser.msg(\r\n                        \"Scalaris.Insert.Start\", op.getObjectAsPrintable(),\r\n                        op.getInternalObjectId()));\r\n            }\r\n\r\n            // prepare object\r\n            ScalarisUtils.generatePersistableIdentity(op);\r\n            JSONObject jsonobj = new JSONObject();\r\n            populateJsonObj(jsonobj, op);\r\n\r\n            if (NucleusLogger.DATASTORE_NATIVE.isDebugEnabled()) {\r\n                NucleusLogger.DATASTORE_NATIVE.debug(\"POST \"\r\n                        + jsonobj.toString());\r\n            }\r\n\r\n            // insert object\r\n            ScalarisUtils.performScalarisObjectInsert(op, jsonobj, scalarisTransaction);\r\n            if (!dnTransactionStarted) {\r\n                scalarisTransaction.commit();\r\n            }\r\n            if (ec.getStatistics() != null) {\r\n                // Add to statistics\r\n                ec.getStatistics().incrementNumWrites();\r\n                ec.getStatistics().incrementInsertCount();\r\n            }\r\n\r\n            if (NucleusLogger.DATASTORE_PERSIST.isDebugEnabled()) {\r\n                NucleusLogger.DATASTORE_PERSIST.debug(Localiser.msg(\r\n                        \"Scalaris.ExecutionTime\",\r\n                        (System.currentTimeMillis() - startTime)));\r\n            }\r\n        } catch (UnknownException e) {\r\n            throw new NucleusDataStoreException(e.getMessage(), e);\r\n        } catch (ConnectionException e) {\r\n            throw new NucleusDataStoreException(e.getMessage(), e);\r\n        } catch (AbortException e) {\r\n            throw new NucleusDataStoreException(e.getMessage(), e);\r\n        } finally {\r\n            if (mConn != null) {\r\n                mConn.release();\r\n            }\r\n        }\r\n    }\r\n\r\n    public void updateObject(ObjectProvider op, int[] updatedFieldNumbers) {\r\n        // Check if read-only so update not permitted\r\n        assertReadOnlyForUpdateOfObject(op);\r\n\r\n        ExecutionContext ec = op.getExecutionContext();\r\n        Transaction scalarisTransaction = ((ScalarisStoreManager) storeMgr)\r\n                .getScalarisTransaction(ec);\r\n        boolean dnTransactionStarted = scalarisTransaction != null;\r\n\r\n        ManagedConnection mConn = null;\r\n        try {\r\n            if (!dnTransactionStarted) {\r\n                mConn = storeMgr.getConnection(ec);\r\n                de.zib.scalaris.Connection scalarisConnection =\r\n                        (de.zib.scalaris.Connection) mConn.getConnection();\r\n                scalarisTransaction = new Transaction(scalarisConnection);\r\n            }\r\n            \r\n            AbstractClassMetaData cmd = op.getClassMetaData();\r\n\r\n            long startTime = System.currentTimeMillis();\r\n            if (NucleusLogger.DATASTORE_PERSIST.isDebugEnabled()) {\r\n                StringBuffer fieldStr = new StringBuffer();\r\n                for (int i = 0; i < updatedFieldNumbers.length; i++) {\r\n                    if (i > 0) {\r\n                        fieldStr.append(\",\");\r\n                    }\r\n                    fieldStr.append(cmd\r\n                            .getMetaDataForManagedMemberAtAbsolutePosition(\r\n                                    updatedFieldNumbers[i]).getName());\r\n                }\r\n                NucleusLogger.DATASTORE_PERSIST.debug(Localiser.msg(\r\n                        \"Scalaris.Update.Start\", op.getObjectAsPrintable(),\r\n                        op.getInternalObjectId(), fieldStr.toString()));\r\n            }\r\n\r\n            ScalarisUtils.performScalarisObjectUpdate(op, updatedFieldNumbers, scalarisTransaction);\r\n            if (!dnTransactionStarted) {\r\n                scalarisTransaction.commit();\r\n            }\r\n            if (ec.getStatistics() != null) {\r\n                // Add to statistics\r\n                ec.getStatistics().incrementNumWrites();\r\n                ec.getStatistics().incrementUpdateCount();\r\n            }\r\n            if (NucleusLogger.DATASTORE_PERSIST.isDebugEnabled()) {\r\n                NucleusLogger.DATASTORE_PERSIST.debug(Localiser.msg(\r\n                        \"Scalaris.ExecutionTime\",\r\n                        (System.currentTimeMillis() - startTime)));\r\n            }\r\n        } catch (ConnectionException e) {\r\n            throw new NucleusDataStoreException(e.getMessage(), e);\r\n        }catch (NotFoundException e) {\r\n            // if we have an update we should already have this object stored\r\n            throw new NucleusObjectNotFoundException(\"Could not update object since its original value was not found\", e);\r\n        } catch (AbortException e) {\r\n            throw new NucleusDataStoreException(e.getMessage(), e);\r\n        } finally {\r\n            if (mConn != null) {\r\n                mConn.release();\r\n            }\r\n        }\r\n    }\r\n\r\n    /**\r\n     * Deletes a persistent object from the database. The delete can take place\r\n     * in several steps, one delete per table that it is stored in. e.g When\r\n     * deleting an object that uses \"new-table\" inheritance for each level of\r\n     * the inheritance tree then will get an DELETE for each table. When\r\n     * deleting an object that uses \"complete-table\" inheritance then will get a\r\n     * single DELETE for its table.\r\n     * \r\n     * @param op\r\n     *            The ObjectProvider of the object to be deleted.\r\n     * \r\n     * @throws NucleusDataStoreException\r\n     *             when an error occurs in the datastore communication\r\n     */\r\n    public void deleteObject(ObjectProvider op) {\r\n        // Check if read-only so update not permitted\r\n        assertReadOnlyForUpdateOfObject(op);\r\n\r\n        ExecutionContext ec = op.getExecutionContext();\r\n        Transaction scalarisTransaction = ((ScalarisStoreManager) storeMgr)\r\n                .getScalarisTransaction(ec);\r\n        boolean dnTransactionStarted = scalarisTransaction != null;\r\n\r\n        ManagedConnection mConn = null;\r\n        try {\r\n            if (!dnTransactionStarted) {\r\n                mConn = storeMgr.getConnection(ec);\r\n                de.zib.scalaris.Connection scalarisConnection =\r\n                        (de.zib.scalaris.Connection) mConn.getConnection();\r\n                scalarisTransaction = new Transaction(scalarisConnection);\r\n            }\r\n\r\n            long startTime = System.currentTimeMillis();\r\n            if (NucleusLogger.DATASTORE_PERSIST.isDebugEnabled()) {\r\n                NucleusLogger.DATASTORE_PERSIST.debug(Localiser.msg(\r\n                        \"Scalaris.Delete.Start\", op.getObjectAsPrintable(),\r\n                        op.getInternalObjectId()));\r\n            }\r\n\r\n            ScalarisUtils.performScalarisObjectDelete(op, scalarisTransaction);\r\n            if (!dnTransactionStarted) {\r\n                scalarisTransaction.commit();\r\n            }\r\n            if (ec.getStatistics() != null) {\r\n                ec.getStatistics().incrementNumWrites();\r\n                ec.getStatistics().incrementDeleteCount();\r\n            }\r\n\r\n            if (NucleusLogger.DATASTORE_PERSIST.isDebugEnabled()) {\r\n                NucleusLogger.DATASTORE_PERSIST.debug(Localiser.msg(\r\n                        \"Scalaris.ExecutionTime\",\r\n                        (System.currentTimeMillis() - startTime)));\r\n            }\r\n        } catch (ConnectionException e) {\r\n            throw new NucleusDataStoreException(e.getMessage(), e);\r\n        } catch (NotFoundException e) {\r\n            throw new NucleusObjectNotFoundException(e.getMessage(), e);\r\n        } catch (AbortException e) {\r\n            throw new NucleusObjectNotFoundException(e.getMessage(), e);\r\n        } finally {\r\n            if (mConn != null) {\r\n                mConn.release();\r\n            }\r\n        }\r\n    }\r\n\r\n    /**\r\n     * Fetches (fields of) a persistent object from the database. This does a\r\n     * single SELECT on the candidate of the class in question. Will join to\r\n     * inherited tables as appropriate to get values persisted into other\r\n     * tables. Can also join to the tables of related objects (1-1, N-1) as\r\n     * neccessary to retrieve those objects.\r\n     * \r\n     * @param op\r\n     *            Object Provider of the object to be fetched.\r\n     * @param memberNumbers\r\n     *            The numbers of the members to be fetched.\r\n     * @throws NucleusObjectNotFoundException\r\n     *             if the object doesn't exist\r\n     * @throws NucleusDataStoreException\r\n     *             when an error occurs in the datastore communication\r\n     */\r\n    public void fetchObject(ObjectProvider op, int[] fieldNumbers) {\r\n        ExecutionContext ec = op.getExecutionContext();\r\n        Transaction scalarisTransaction = ((ScalarisStoreManager) storeMgr)\r\n                .getScalarisTransaction(ec);\r\n\r\n        final long startTime = System.currentTimeMillis();\r\n\r\n        try {\r\n            JSONObject result;\r\n            if (scalarisTransaction != null) {\r\n                result = ScalarisUtils.performScalarisObjectFetch(op, scalarisTransaction);\r\n            } else {\r\n                // non transactional read\r\n                ManagedConnection mConn = storeMgr.getConnection(ec);\r\n                try {\r\n                    de.zib.scalaris.Connection scalarisConnection =\r\n                            (de.zib.scalaris.Connection) mConn.getConnection();\r\n                    result = ScalarisUtils.performScalarisObjectFetch(op, scalarisConnection);\r\n                } finally {\r\n                    mConn.release();\r\n                }\r\n            }\r\n\r\n            if (ScalarisUtils.isDeletedRecord(result)) {\r\n                throw new NucleusObjectNotFoundException(\r\n                        \"Record has been deleted\");\r\n            }\r\n\r\n            final String declaredClassQName = result.getString(\"class\");\r\n            final Class declaredClass = op.getExecutionContext()\r\n                    .getClassLoaderResolver()\r\n                    .classForName(declaredClassQName);\r\n            final Class<?> objectClass = op.getObject().getClass();\r\n\r\n            if (!objectClass.isAssignableFrom(declaredClass)) {\r\n                    System.out.println(\"Type found in db not compatible with requested type\");\r\n                throw new NucleusObjectNotFoundException(\r\n                        \"Type found in db not compatible with requested type\");\r\n            }\r\n\r\n            op.replaceFields(fieldNumbers, new FetchFieldManager(op, result));\r\n\r\n            if (NucleusLogger.DATASTORE_NATIVE.isDebugEnabled()) {\r\n                NucleusLogger.DATASTORE_NATIVE\r\n                        .debug(\"GET \" + result.toString());\r\n            }\r\n        } catch (NotFoundException e) {\r\n            // throwing the NeucleusExpection directly would cause the StateManagerImpl\r\n            // to print a warning. This is annoying because all that happened was that\r\n            // we couldn't find an object in the store\r\n            NucleusException ne = new NucleusObjectNotFoundException(e.getMessage(), e);\r\n            throw storeMgr.getApiAdapter().getApiExceptionForNucleusException(ne);\r\n        } catch (ConnectionException e) {\r\n            throw new NucleusDataStoreException(e.getMessage(), e);\r\n        }\r\n\r\n        if (ec.getStatistics() != null) {\r\n            // Add to statistics\r\n            ec.getStatistics().incrementNumReads();\r\n            ec.getStatistics().incrementFetchCount();\r\n        }\r\n\r\n        if (NucleusLogger.DATASTORE_RETRIEVE.isDebugEnabled()) {\r\n            NucleusLogger.DATASTORE_RETRIEVE.debug(Localiser.msg(\r\n                    \"Scalaris.ExecutionTime\",\r\n                    (System.currentTimeMillis() - startTime)));\r\n        }\r\n    }\r\n\r\n    /**\r\n     * Method to return a persistable object with the specified id. Optional\r\n     * operation for StoreManagers. Should return a (at least) hollow\r\n     * PersistenceCapable object if the store manager supports the operation. If\r\n     * the StoreManager is managing the in-memory object instantiation (as part\r\n     * of co-managing the object lifecycle in general), then the StoreManager\r\n     * has to create the object during this call (if it is not already created).\r\n     * Most relational databases leave the in-memory object instantion to Core,\r\n     * but some object databases may manage the in-memory object instantion,\r\n     * effectively preventing Core of doing this.\r\n     * <p>\r\n     * StoreManager implementations may simply return null, indicating that they\r\n     * leave the object instantiate to us. Other implementations may instantiate\r\n     * the object in question (whether the implementation may trust that the\r\n     * object is not already instantiated has still to be determined). If an\r\n     * implementation believes that an object with the given ID should exist,\r\n     * but in fact does not exist, then the implementation should throw a\r\n     * RuntimeException. It should not silently return null in this case.\r\n     * </p>\r\n     * \r\n     * @param ec\r\n     *            execution context\r\n     * @param id\r\n     *            the id of the object in question.\r\n     * @return a persistable object with a valid object state (for example:\r\n     *         hollow) or null, indicating that the implementation leaves the\r\n     *         instantiation work to us.\r\n     */\r\n    public Object findObject(ExecutionContext ec, Object id) {\r\n        return null;\r\n    }\r\n\r\n    /**\r\n     * Locates this object in the datastore.\r\n     * \r\n     * @param op\r\n     *            ObjectProvider for the object to be found\r\n     * @throws NucleusObjectNotFoundException\r\n     *             if the object doesnt exist\r\n     * @throws NucleusDataStoreException\r\n     *             when an error occurs in the datastore communication\r\n     */\r\n    public void locateObject(ObjectProvider op) {\r\n        // The implementation of this method did not server an apparent purpose,\r\n        // since removing all code here did not result in a test failure.\r\n        // But removing it greatly improved (read: doubled in some cases) the speed \r\n        // of read operations. \r\n        // TODO: What does this method do?\r\n    }\r\n\r\n    /**\r\n     * Convenience method to get all objects of the candidate type from the\r\n     * specified connection. Objects of subclasses are ignored.\r\n     * \r\n     * @param ec\r\n     * @param mconn\r\n     * @param candidateClass\r\n     */\r\n    public List<Object> getObjectsOfCandidateType(ExecutionContext ec,\r\n            Class<?> candidateClass,AbstractClassMetaData cmd) {\r\n        List<Object> results = new ArrayList<Object>();\r\n        String idIndexKey = ScalarisSchemaHandler.getIDIndexKeyName(candidateClass);\r\n\r\n        ManagedConnection mconn = ec.getStoreManager().getConnection(ec);\r\n        de.zib.scalaris.Connection conn = (de.zib.scalaris.Connection) mconn\r\n                .getConnection();\r\n        Transaction scalarisTransaction = ((ScalarisStoreManager) storeMgr)\r\n                .getScalarisTransaction(ec);\r\n        boolean dnTransactionStarted = scalarisTransaction != null;\r\n\r\n        try {\r\n            // read the management key\r\n            List<String> idIndex;\r\n            if (dnTransactionStarted) {\r\n                idIndex = scalarisTransaction.read(idIndexKey).stringListValue();\r\n            } else {\r\n                TransactionSingleOp t = new TransactionSingleOp(conn);\r\n                idIndex = t.read(idIndexKey).stringListValue();\r\n            }\r\n\r\n            // retrieve all values from the management key\r\n            for (String id : idIndex) {\r\n                results.add(IdentityUtils.getObjectFromPersistableIdentity(id, cmd, ec));\r\n            }\r\n\r\n        } catch (NotFoundException e) {\r\n            // the management key does not exist which means there\r\n            // are no instances of this class stored.\r\n        } catch (ConnectionException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        } catch (UnknownException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        } finally {\r\n            if (mconn != null) {\r\n                mconn.release();\r\n            }\r\n        }\r\n\r\n        return results;\r\n    }\r\n\r\n    /**\r\n     * Returns an object persisted in the store by an unique member value.\r\n     * If there is no such object, null is returned\r\n     * @param ec\r\n     * @param mconn Connection used to connect to the store.\r\n     * @param objectClass Class of the object\r\n     * @param memberName The (simple) name of the unique member which is used to retrieve\r\n     *      the object. The object is only correctly returned if the member has an \r\n     *      '@Unique' annotation.\r\n     * @param memberValue Value of the unique member\r\n     * @return The persisted object, or null if there is no such object.\r\n     */\r\n    public Object getObjectByUniqueMember(ExecutionContext ec,\r\n                Class<?> objectClass, String memberName, String memberValue) {\r\n        AbstractClassMetaData cmd = ec.getMetaDataManager()\r\n                .getMetaDataForClass(objectClass,\r\n                        ec.getClassLoaderResolver());\r\n        ManagedConnection mconn = ec.getStoreManager().getConnection(ec);\r\n        de.zib.scalaris.Connection conn = (de.zib.scalaris.Connection) mconn\r\n                .getConnection();\r\n        Transaction scalarisTransaction = ((ScalarisStoreManager) storeMgr)\r\n                .getScalarisTransaction(ec);\r\n        boolean dnTransactionStarted = scalarisTransaction != null;\r\n\r\n        String uniqueMemberValueKey = ScalarisSchemaHandler.getUniqueMemberKey(\r\n                objectClass.getCanonicalName(), memberName, memberValue);\r\n        try {\r\n            String uniqueObjectId;\r\n            if (dnTransactionStarted) {\r\n                uniqueObjectId = scalarisTransaction.read(uniqueMemberValueKey).stringValue();\r\n            } else {\r\n                TransactionSingleOp t = new TransactionSingleOp(conn);\r\n                uniqueObjectId = t.read(uniqueMemberValueKey).stringValue();\r\n            }\r\n\r\n            if (!ScalarisUtils.isDeletedRecord(uniqueObjectId)) {\r\n               return IdentityUtils.getObjectFromPersistableIdentity(uniqueObjectId, cmd, ec);\r\n            }\r\n        } catch (ConnectionException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        } catch (NotFoundException e) {\r\n            // there does not exist an object with the unique member value\r\n        } catch (UnknownException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        } finally {\r\n            if (mconn != null) {\r\n                mconn.release();\r\n            }\r\n        }\r\n        return null;\r\n    }\r\n}\r\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/src/java/org/datanucleus/store/scalaris/ScalarisSchemaHandler.java",
    "content": "package org.datanucleus.store.scalaris;\n\nimport org.datanucleus.store.StoreManager;\nimport org.datanucleus.store.schema.AbstractStoreSchemaHandler;\n\npublic class ScalarisSchemaHandler extends AbstractStoreSchemaHandler {\n\n    /**\n     * Used to signal a key at which all foreign key actions for an object are stored.\n     */\n    private static final String FKA_KEY_SUFFIX = \"FKA\";\n\n    /**\n     * Used to signal a key at which the IDs of all persisted objects of the same\n     * class are stored.\n     */\n    private static final String ALL_ID_SUFFIX = \"ALL_IDS\";\n\n    /**\n     * Key prefix used to signal a key which is used for identity generation.\n     */\n    private static final String ID_GEN_SUFFIX = \"ID_GEN\";\n\n    /**\n     * Used to signal a key at which the persisted object ID is stored which\n     * belongs to a given unique member value. \n     */\n    private static final String UNIQUE_MEMBER_SUFFIX = \"UNIQUE\";\n\n\n    public ScalarisSchemaHandler(StoreManager storeMgr) {\n        super(storeMgr);\n    }\n\n    /**\n     * Returns the Scalaris key used to store object with full class name \n     * objClassName and ID objectId.\n     * @param objClassName \n     *      The full class name of the object\n     * @param objectId \n     *      The ID of the object\n     * @return The Scalaris key\n     */\n    public static String getObjectStorageKey(String objClassName, String objectId) {\n        if (objectId.startsWith(objClassName)) {\n            return objectId;\n        }\n        return String.format(\"%s:%s\", objClassName, objectId);\n    }\n\n    /**\n     * Returns the Scalaris key at which the IDs of all persisted \n     * objects of a given class are stored.\n     * @param clazz \n     *      The objects class\n     * @return The Scalaris key\n     */\n    static String getIDIndexKeyName(Class<?> clazz) {\n        return getIDIndexKeyName(clazz.getCanonicalName());\n    }\n\n    /**\n     * Returns the Scalaris key at which the IDs of all persisted \n     * objects of a given class are stored.\n     * @param clazz \n     *      The objects class\n     * @return The Scalaris key\n     */\n    static String getIDIndexKeyName(String className) {\n        return String.format(\"%s_%s\", className, ALL_ID_SUFFIX);\n    }\n \n    /**\n     * Returns the Scalaris key used for generating IDs for new object instances\n     * persisted in the data store.\n     * @param className \n     *      The full class name of the object the ID will be generated for.\n     * @return The Scalaris key\n     */\n    static String getIDGeneratorKeyName(String className) {\n        return String.format(\"%s_%s\", className, ID_GEN_SUFFIX);\n    }\n\n    /**\n     * Returns the Scalaris key used to store which persisted object\n     * has stored the given unique value.\n     * @param className\n     *      Class of the object with the unique member annotation\n     * @param memberName\n     *      Simple member name which has an 'unique' annotation\n     * @param memberValue\n     *      The unique value of the member\n     * @return The Scalaris key\n     */\n    static String getUniqueMemberKey(String className, String memberName, String memberValue) {\n        return String.format(\"%s_%s_%s_%s\", className, memberName, memberValue, UNIQUE_MEMBER_SUFFIX);\n    }\n\n    /**\n     * Returns the Scalaris key used to store all foreign key actions\n     * of the given object.\n     * @param foreignObjectClass\n     *      Class of the object\n     * @param foreignObjectId\n     *      ID of the object\n     * @return The Scalaris key\n     */\n    static String getForeignKeyActionKey(String foreignObjectClass, String foreignObjectId) {\n        String storageKey = getObjectStorageKey(foreignObjectClass, foreignObjectId);\n        return String.format(\"%s_%s\", storageKey, FKA_KEY_SUFFIX);\n    }\n}\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/src/java/org/datanucleus/store/scalaris/ScalarisStoreManager.java",
    "content": "/**********************************************************************\r\nCopyright (c) 2008 Erik Bengtson and others. All rights reserved.\r\nLicensed under the Apache License, Version 2.0 (the \"License\");\r\nyou may not use this file except in compliance with the License.\r\nYou may obtain a copy of the License at\r\n\r\n    http://www.apache.org/licenses/LICENSE-2.0\r\n\r\nUnless required by applicable law or agreed to in writing, software\r\ndistributed under the License is distributed on an \"AS IS\" BASIS,\r\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nSee the License for the specific language governing permissions and\r\nlimitations under the License.\r\n\r\nContributors:\r\n2008 Andy Jefferson - abstracted methods up to AbstractStoreManager\r\n2013 Orange - port to Scalaris key/value store\r\n    ...\r\n **********************************************************************/\r\npackage org.datanucleus.store.scalaris;\r\n\r\nimport java.util.Collections;\r\nimport java.util.HashMap;\r\nimport java.util.Map;\r\n\r\nimport org.datanucleus.ClassLoaderResolver;\r\nimport org.datanucleus.ExecutionContext;\r\nimport org.datanucleus.PersistenceNucleusContext;\r\nimport org.datanucleus.TransactionEventListener;\r\nimport org.datanucleus.exceptions.NucleusDataStoreException;\r\nimport org.datanucleus.metadata.AbstractClassMetaData;\r\nimport org.datanucleus.store.AbstractStoreManager;\r\nimport org.datanucleus.store.NucleusConnection;\r\nimport org.datanucleus.store.connection.ManagedConnection;\r\n\r\nimport de.zib.scalaris.AbortException;\r\nimport de.zib.scalaris.ConnectionException;\r\nimport de.zib.scalaris.Transaction;\r\n\r\npublic class ScalarisStoreManager extends AbstractStoreManager {\r\n\r\n    private Map<org.datanucleus.Transaction, de.zib.scalaris.Transaction> transactionMap;\r\n\r\n    public ScalarisStoreManager(ClassLoaderResolver clr,\r\n            PersistenceNucleusContext ctx, Map<String, Object> props) {\r\n        super(\"scalaris\", clr, ctx, props);\r\n\r\n        transactionMap = Collections.synchronizedMap(\r\n                new HashMap<org.datanucleus.Transaction, de.zib.scalaris.Transaction>());\r\n\r\n        // Handler for persistence process\r\n        persistenceHandler = new ScalarisPersistenceHandler(this);\r\n        schemaHandler = new ScalarisSchemaHandler(this);\r\n\r\n        logConfiguration();\r\n    }\r\n\r\n    public NucleusConnection getNucleusConnection(ExecutionContext om) {\r\n        throw new UnsupportedOperationException();\r\n    }\r\n\r\n    @Override\r\n    public void transactionStarted(final ExecutionContext ec) {\r\n        final org.datanucleus.Transaction dnTransaction = ec.getTransaction();\r\n        final ManagedConnection mConn = getConnection(ec);\r\n        de.zib.scalaris.Connection conn = (de.zib.scalaris.Connection) mConn\r\n                .getConnection();\r\n        final Transaction scalarisTransaction = new Transaction(conn);\r\n\r\n        if (transactionMap.containsKey(dnTransaction)) {\r\n            throw new NucleusDataStoreException(\"Cannot start the same transaction multiple times\");\r\n        }\r\n        transactionMap.put(dnTransaction, scalarisTransaction);\r\n        dnTransaction.addTransactionEventListener(new TransactionEventListener() {\r\n            /**\r\n             * Signal if managed connection was released in pre-commit, preventing\r\n             * multiple releases which can lead to undefined behavior.\r\n             */\r\n            private boolean connectionIsReleased = false;\r\n\r\n            public void transactionPreCommit() {\r\n                try {\r\n                    // ensure that all updates operations are executed before commit\r\n                    ec.flushInternal(true);\r\n                    scalarisTransaction.commit();\r\n                } catch (ConnectionException e) {\r\n                    throw new NucleusDataStoreException(e.getMessage(), e);\r\n                } catch (AbortException e) {\r\n                    throw new NucleusDataStoreException(e.getMessage(), e);\r\n                } finally {\r\n                    transactionMap.remove(dnTransaction);\r\n                    if (mConn != null && !connectionIsReleased) {\r\n                        connectionIsReleased = true;\r\n                        mConn.release();\r\n                    }\r\n                }\r\n            }\r\n\r\n            public void transactionPreRollBack() {\r\n                scalarisTransaction.abort();\r\n                transactionMap.remove(dnTransaction);\r\n                if (mConn != null && !connectionIsReleased) {\r\n                    connectionIsReleased = true;\r\n                    mConn.release();\r\n                }\r\n            }\r\n\r\n            // Not implemented because not needed\r\n            public void transactionStarted() {}\r\n            public void transactionEnded() {}\r\n            public void transactionPreFlush() {}\r\n            public void transactionFlushed() {}\r\n            public void transactionCommitted() {}\r\n            public void transactionRolledBack() {}\r\n            public void transactionSetSavepoint(String name) {}\r\n            public void transactionReleaseSavepoint(String name) {}\r\n            public void transactionRollbackToSavepoint(String name) {}\r\n        });\r\n    }\r\n\r\n    /**\r\n     * Returns the Scalaris transaction which belongs to the\r\n     * transaction of the passed ExecutionContext. If its transaction\r\n     * is not active, no Scalaris transaction exists, thus returning null\r\n     * \r\n     * @param ec\r\n     *      ExecutionContext\r\n     * @return The Scalaris transaction or null if ec has no active\r\n     *      transaction.\r\n     */\r\n    public Transaction getScalarisTransaction(ExecutionContext ec) {\r\n        return transactionMap.get(ec.getTransaction());\r\n    }\r\n\r\n    /**\r\n     * Method defining which value strategy to use when the user specified native strategy \r\n     * or no strategy.\r\n     */\r\n    @Override\r\n    public String getStrategyForNative(AbstractClassMetaData cmd, int absFiledNumber) {\r\n        return \"uuid-hex\";\r\n    }\r\n}\r\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/src/java/org/datanucleus/store/scalaris/ScalarisUtils.java",
    "content": "package org.datanucleus.store.scalaris;\n\nimport java.util.ArrayList;\nimport java.util.Collection;\nimport java.util.HashMap;\nimport java.util.Iterator;\nimport java.util.List;\nimport java.util.Map.Entry;\n\nimport org.datanucleus.ExecutionContext;\nimport org.datanucleus.exceptions.NucleusDataStoreException;\nimport org.datanucleus.exceptions.NucleusException;\nimport org.datanucleus.exceptions.NucleusObjectNotFoundException;\nimport org.datanucleus.exceptions.NucleusUserException;\nimport org.datanucleus.identity.IdentityUtils;\nimport org.datanucleus.metadata.AbstractClassMetaData;\nimport org.datanucleus.metadata.AbstractMemberMetaData;\nimport org.datanucleus.metadata.ForeignKeyAction;\nimport org.datanucleus.metadata.ForeignKeyMetaData;\nimport org.datanucleus.metadata.UniqueMetaData;\nimport org.datanucleus.state.ObjectProvider;\nimport org.datanucleus.store.StoreManager;\nimport org.datanucleus.store.connection.ManagedConnection;\nimport org.datanucleus.store.scalaris.fieldmanager.StoreFieldManager;\nimport org.datanucleus.transaction.NucleusTransactionException;\nimport org.datanucleus.util.NucleusLogger;\n\nimport com.ericsson.otp.erlang.OtpErlangLong;\n\nimport org.json.JSONArray;\nimport org.json.JSONException;\nimport org.json.JSONObject;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.Connection;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.NotAListException;\nimport de.zib.scalaris.NotANumberException;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.Transaction;\nimport de.zib.scalaris.TransactionSingleOp;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * This class contains convenience methods to provide functionalities that are not natively supported by \n * Scalaris. For example generating identities, managing all primary keys of a class to \"iterate\" over all stored \n * instances, or ensuring uniqueness. \n */\npublic class ScalarisUtils {\n\n    /**\n     * Placeholder to signal a foreign key action deleting the whole object,\n     * instead of an element of a collection.\n     */\n    private static final String FKA_DELETE_OBJ = \"_DEL_OBJECT\";\n\n    /**\n     * Value which will be used to signal a deleted key.\n     */\n    public static final String DELETED_RECORD_VALUE = new JSONObject().toString();\n\n    /* **********************************************************************\n     *                  ID GENERATION\n     * **********************************************************************/\n\n    /**\n     * Generate a new ID which can be used to store a value at an unique key.\n     * Every time this function is called the value stored at key ID_GEN_KEY is\n     * incremented by one. The value stored there is the value which is returned by\n     * this function. Every object class has its own ID generator key\n     * \n     * @param op\n     *            ObjectProvider of the object this ID is generated for.\n     * @return A new ID.\n     */\n    private synchronized static long generateNextIdentity(ObjectProvider<?> op) {\n        StoreManager storeMgr = op.getExecutionContext().getStoreManager();\n        \n        ExecutionContext ec = op.getExecutionContext();\n        String keyName = ScalarisSchemaHandler.getIDGeneratorKeyName(op.getClassMetaData().getFullClassName());\n        ManagedConnection mConn = storeMgr.getConnection(ec);\n        de.zib.scalaris.Connection conn = (de.zib.scalaris.Connection) mConn\n                .getConnection();\n\n        long newID = 0l;\n        Transaction t = new Transaction(conn);\n        try {\n            try {\n                ErlangValue storedVal = t.read(keyName);\n                newID = storedVal.longValue() + 1l;\n                t.addOnNr(keyName, new OtpErlangLong(1l));\n            } catch (NotFoundException e) {\n                // No ID was generated yet\n                newID = 1l;\n                t.write(keyName, newID);\n            }\n\n            t.commit();\n        } catch (ConnectionException e) {\n            throw new NucleusTransactionException(\n                    \"Could not generate a new ID because of transaction failure\",\n                    e);\n        } catch (AbortException e) {\n            throw new NucleusTransactionException(\n                    \"Could not generate a new ID becasue of transaction failure\",\n                    e);\n        } catch (NotANumberException e) {\n            throw new NucleusTransactionException(\n                    \"The value of the ID generator key was altered to an invalid value\",\n                    e);\n        } finally {\n            mConn.release();\n        }\n\n        return newID;\n    }\n\n    /**\n     * Generates the ID of an object which will be persisted, but is not persisted yet, \n     * in the data store.  \n     * ATTENTION: If the data store is (partially) responsible to generate an ID (e.g. \n     * because of IdGeneratorStrategy.IDENTITY). This method may change primary key\n     * attribute values. This method should be used only when inserting a new object\n     * in the data store, otherwise consider using getPersitableIdentity.\n     * \n     * @param op\n     *            data source.\n     * @return Identity of object provided by op or null if at least one primary\n     *         key field is not loaded.\n     */\n    static String generatePersistableIdentity(ObjectProvider<?> op) {\n        StoreManager storeMgr = op.getExecutionContext().getStoreManager();\n        AbstractClassMetaData cmd = op.getClassMetaData();\n\n        if (cmd.pkIsDatastoreAttributed(storeMgr)) {\n            // The primary key must be (partially) calculated by the data store.\n            // There is no distinction between APPLICATION and DATASTORE\n            // IdentityType (yet)\n\n            int[] pkFieldNumbers = cmd.getPKMemberPositions();\n            long idKey = 0;\n            for (int i = 0; i < pkFieldNumbers.length; i++) {\n                AbstractMemberMetaData mmd = cmd\n                        .getMetaDataForManagedMemberAtAbsolutePosition(pkFieldNumbers[i]);\n\n                if (storeMgr.isStrategyDatastoreAttributed(cmd,\n                        pkFieldNumbers[i])) {\n                    Class<?> mType = mmd.getType();\n                    if (!(mType.equals(Long.class) || mType.equals(long.class)\n                            || mType.equals(Integer.class) || mType\n                                .equals(int.class))) {\n                        // Field type must be long/Long or int/Integer since\n                        // this is the only IDENTITY value that is currently\n                        // supported\n                        throw new NucleusUserException(\n                                \"Any field using IDENTITY value generation with Scalaris should be of type long/Long or int/Integer\");\n                    }\n                    idKey = generateNextIdentity(op);\n                    if (mType.equals(Integer.class) || mType.equals(int.class)) {\n                        if (idKey > Integer.MAX_VALUE) {\n                            throw new NucleusException(\"We ran out of integer IDs!\");\n                        }\n                        op.replaceField(mmd.getAbsoluteFieldNumber(), (int) idKey);\n                    } else {\n                        op.replaceField(mmd.getAbsoluteFieldNumber(), idKey);\n                    }\n                    if (NucleusLogger.DATASTORE_PERSIST.isDebugEnabled()) {\n                        NucleusLogger.DATASTORE_PERSIST.debug(\"Generated ID \" + idKey + \" for object \" +\n                        \t\t\"of class \" + cmd.getFullClassName());\n                    }\n                }\n            }\n        }\n        return getPersistableIdentity(op);\n    }\n\n    /**\n     * Returns the object identity as string. If this ObjectProvider has not yet\n     * set its external object id, it is set to the returned ID.\n     * \n     * @param op\n     * @return Identity of object provided by op or null if at least one primary\n     *         key field is not loaded.\n     */\n    public static String getPersistableIdentity(ObjectProvider<?> op) {\n        AbstractClassMetaData cmd = op.getClassMetaData();\n        String keySeparator = \":\";\n\n        if (op.getExternalObjectId() == null) {\n            // The primary key must be (partially) calculated by the data store.\n            // There is no distinction between APPLICATION and DATASTORE\n            // IdentityType (yet)\n\n            StringBuilder keyBuilder = new StringBuilder();\n\n            int[] pkFieldNumbers = cmd.getPKMemberPositions();\n            for (int i = 0; i < pkFieldNumbers.length; i++) {\n                AbstractMemberMetaData mmd = cmd\n                        .getMetaDataForManagedMemberAtAbsolutePosition(pkFieldNumbers[i]);\n\n                Object keyVal = op.provideField(mmd.getAbsoluteFieldNumber());\n                if (i > 0) {\n                    keyBuilder.append(keySeparator);\n                }\n                keyBuilder.append(keyVal);\n            }\n\n            op.setPostStoreNewObjectId(keyBuilder.toString());\n        }\n\n        String id = IdentityUtils.getPersistableIdentityForId(op.getExternalObjectId());\n        return id;\n    }\n\n    /* **********************************************************************\n     *                  HOOKS FOR ScalarisStoreManager\n     * **********************************************************************/\n \n    /**\n     * Retrieves an object from Scalaris based on the ObjectProviders ID and class.\n     * @param op \n     *      ObjectProvider representing the stored object in DataNucleus\n     * @param conn \n     *      Connection used for the operation\n     * @return The stored object\n     */\n    static JSONObject performScalarisObjectFetch(ObjectProvider<?> op, Transaction scalarisTransaction)\n            throws ConnectionException, NotFoundException {\n\n        final String objId = ScalarisUtils.getPersistableIdentity(op);\n        final String objClassName = op.getClassMetaData().getFullClassName();\n        final String objKey = ScalarisSchemaHandler.getObjectStorageKey(objClassName, objId);\n        \n        return new JSONObject(scalarisTransaction.read(objKey).stringValue());\n    }\n    \n    static JSONObject performScalarisObjectFetch(ObjectProvider<?> op, Connection scalarisConnection)\n            throws ConnectionException, NotFoundException {\n\n        final String objId = ScalarisUtils.getPersistableIdentity(op);\n        final String objClassName = op.getClassMetaData().getFullClassName();\n        final String objKey = ScalarisSchemaHandler.getObjectStorageKey(objClassName, objId);\n\n        TransactionSingleOp scalarisTransaction = new TransactionSingleOp(scalarisConnection);\n        return new JSONObject(scalarisTransaction.read(objKey).stringValue());\n    }\n    /**\n     * Inserts an object into Scalaris and performs all necessary management operations (updating indices etc.)\n     * ScalarisUtils.WRITE_LOCK is used to ensure thread safety.\n     * @param op\n     *      ObjectPrevider representing the object to be inserted \n     * @param json\n     *      The JSON representation of the object\n     * @param conn\n     *      Connection used for the operation\n     */\n    static void performScalarisObjectInsert(ObjectProvider<?> op, JSONObject json, Transaction scalarisTransaction)\n            throws ConnectionException {\n\n        try {\n            String objectId = ScalarisUtils.getPersistableIdentity(op);\n\n            insertObjectToIDIndex(op, scalarisTransaction);\n            updateUniqueMemberKey(op, json, null, scalarisTransaction);\n            insertToForeignKeyAction(op, json, scalarisTransaction);\n\n            String className = op.getClassMetaData().getFullClassName();\n            String storageKey = ScalarisSchemaHandler.getObjectStorageKey(className, objectId);\n\n            scalarisTransaction.write(storageKey, json.toString());\n        } catch (NotAListException e) {\n            throw new NucleusDataStoreException(\"Keys used internally have values of unexpected structure\", e);\n        }\n    }\n\n    /**\n     * Updates a stored object and performs all necessary management operations (updating indices etc.)\n     * ScalarisUtils.WRITE_LOCK is used to ensure thread safety.\n     * @param op\n     *      ObjectProvider representing the updated state of the object \n     * @param updatedFieldNumbers\n     *      The ObjectProviders fields which shall be updated \n     * @param conn\n     *      Connection used for the operation\n     */\n    static void performScalarisObjectUpdate(ObjectProvider<?> op, int[] updatedFieldNumbers, Transaction scalarisTransaction)\n            throws ConnectionException, NotFoundException {\n\n        String objectId = ScalarisUtils.getPersistableIdentity(op);\n\n        // get old value\n        String className = op.getClassMetaData().getFullClassName();\n        String objectKey = ScalarisSchemaHandler.getObjectStorageKey(className, objectId);\n\n        JSONObject stored = new JSONObject(scalarisTransaction.read(objectKey).stringValue());\n\n        JSONObject changedVals = new JSONObject();\n        op.provideFields(updatedFieldNumbers, new StoreFieldManager(op,\n                changedVals, true));\n\n        // update stored object values\n        JSONObject changedValsOld = new JSONObject();\n        Iterator<?> keyIter = changedVals.keys();\n        while (keyIter.hasNext()) {\n            String key = (String) keyIter.next();\n            if (stored.has(key)) {\n                changedValsOld.put(key, stored.get(key));\n            }\n            stored.put(key, changedVals.get(key));\n        }\n        try {\n            updateUniqueMemberKey(op, changedVals, changedValsOld, scalarisTransaction);\n            updateForeignKeyAction(op, changedVals, changedValsOld, scalarisTransaction);\n\n            scalarisTransaction.write(objectKey, stored.toString());\n        } catch (NotAListException e) {\n            throw new NucleusDataStoreException(\"Keys used internally have values of unexpected structure\", e);\n        }\n    }\n\n    /**\n     * Deletes a stored object and performs all necessary management operations (updating indices etc.)\n     * ScalarisUtils.WRITE_LOCK is used to ensure thread safety.\n     * @param op\n     *      ObjectProvider representing the object to be deleted \n     * @param conn\n     *      Connection used for the operation\n     */\n    static void performScalarisObjectDelete(ObjectProvider<?> op, Transaction scalarisTransaction)\n            throws ConnectionException, NotFoundException {\n\n        String objectId = ScalarisUtils.getPersistableIdentity(op);\n        String className = op.getClassMetaData().getFullClassName();\n        String objectKey = ScalarisSchemaHandler.getObjectStorageKey(className, objectId);\n\n        JSONObject oldJson = new JSONObject(scalarisTransaction.read(objectKey).stringValue());\n\n        try {\n            removeObjectFromIDIndex(op, scalarisTransaction);\n            removeObjectFromUniqueMemberKey(op, oldJson, scalarisTransaction);\n            performForeignKeyActionDelete(op, scalarisTransaction);\n\n            scalarisTransaction.write(objectKey,  DELETED_RECORD_VALUE);\n        } catch (NotAListException e) {\n            throw new NucleusDataStoreException(\"Keys used internally have values of unexpected structure\", e);\n        }\n    }\n\n    /* **********************************************************************\n     *          INDICIES OF ALL OBJECTS OF ONE TYPE (used for queries)\n     * **********************************************************************/\n\n    /**\n     * To support queries it is necessary to have the possibility to iterate\n     * over all stored objects of a specific type. Since Scalaris stores only\n     * key-value pairs without structured tables, this is not \"natively\"\n     * supported. Therefore an extra key is added to the store containing all\n     * keys of available objects of a type.\n     * \n     * This methods adds another entry to such a key based on the passed\n     * ObjectProvider. If no such key-value pair exists, it is created.\n     * \n     * @param op\n     *            The data source\n     * @throws ConnectionException\n     * @throws NotAListExceptionn\n     */\n    private static void insertObjectToIDIndex(ObjectProvider<?> op, Transaction t)\n            throws ConnectionException, NotAListException {\n        AbstractClassMetaData cmd = op.getClassMetaData();\n        String key = ScalarisSchemaHandler.getIDIndexKeyName(cmd.getFullClassName());\n        String objectStringIdentity = getPersistableIdentity(op);\n        List<ErlangValue> toAdd = new ArrayList<ErlangValue>();\n        toAdd.add(new ErlangValue(objectStringIdentity));\n        t.addDelOnList(key, toAdd, new ArrayList<ErlangValue>());\n    }\n\n    /**\n     * To support queries it is necessary to have the possibility to iterate\n     * over all stored objects of a specific type. Since Scalaris stores only\n     * key-value pairs without structured tables, this is not \"natively\"\n     * supported. Therefore an extra key is added to the store containing all\n     * keys of available objects of a type.\n     * \n     * This methods removes an entry of such a key based on the passed\n     * ObjectProvider. If no such key-value pair exists, nothing happens.\n     * \n     * @param op\n     *            The data source\n     * @throws NotAListException \n     * @throws ConnectionException \n     */\n    private static void removeObjectFromIDIndex(ObjectProvider<?> op, Transaction t)\n            throws ConnectionException, NotAListException {\n\n        AbstractClassMetaData cmd = op.getClassMetaData();\n        String key = ScalarisSchemaHandler.getIDIndexKeyName(cmd.getFullClassName());\n        String objectStringIdentity = getPersistableIdentity(op);\n\n        List<ErlangValue> toRemove = new ArrayList<ErlangValue>();\n        toRemove.add(new ErlangValue(objectStringIdentity));\n        t.addDelOnList(key, new ArrayList<ErlangValue>(), toRemove);\n    }\n\n    /* **********************************************************************\n     *                  ACTIONS TO GUARANTEE UNIQUENESS\n     * **********************************************************************/\n\n    /**\n     * To support unique member values (@Unique annotation) an extra key is\n     * inserted to signal the object ID whose object has stored the given value.\n     * Removed the keys belonging to the old state of the object and inserts new\n     * keys according to the new state.\n     * If this method is called and it turns out that the updated member value is\n     * stored by a different object a NucleusDataStoreException is thrown.\n     * @param op\n     *      The data source representing the object\n     * @param newJson\n     *      The new state of the object in JSON form\n     * @param oldJson\n     *      The old state of the object in JSON form. If this is null it is assumed\n     *      that this object is inserted for the first time and thus no old value will\n     *      be deleted\n     * @param t\n     *      The transaction used for the performed operations. \n     * @throws ConnectionException\n     * @throws UnknownException\n     */\n    private static void updateUniqueMemberKey(ObjectProvider<?> op, JSONObject newJson, JSONObject oldJson, Transaction t) \n            throws ConnectionException, UnknownException {\n        AbstractClassMetaData cmd = op.getClassMetaData();\n        String objectStringIdentity = getPersistableIdentity(op);\n        String className = cmd.getFullClassName();\n       \n        for (int field : cmd.getAllMemberPositions()) {\n            AbstractMemberMetaData mmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition(field);\n            UniqueMetaData umd = mmd.getUniqueMetaData();\n            if (umd != null) {\n                // this member has @Unique annotation -> lookup all stored values for this member\n                String fieldName = mmd.getName();\n                String oldFieldValue = null, newFieldValue = null;\n                try {\n                    newFieldValue = (newJson != null && newJson.has(fieldName)) ? newJson.getString(fieldName) : null;\n                    oldFieldValue = (oldJson != null && oldJson.has(fieldName)) ? oldJson.getString(fieldName) : null;\n                } catch (JSONException e) {\n                    // unique members can be null which means they are not found in the JSON\n                }\n                if (newFieldValue != null && newFieldValue.equals(oldFieldValue)) {\n                    // this field has not changed -> skip update\n                    continue;\n                }\n                \n                if (oldFieldValue != null) {\n                    // mark the old key as removed\n                    String oldValueKey = ScalarisSchemaHandler.getUniqueMemberKey(className, fieldName, oldFieldValue);\n                    t.write(oldValueKey, DELETED_RECORD_VALUE);\n                }\n                \n                if (newFieldValue != null) {\n                    // check if this value already exists\n                    // if it does -> exception; if not -> store\n                    String newValueKey = ScalarisSchemaHandler.getUniqueMemberKey(className, fieldName, newFieldValue);\n                    String idStoringThisValue = null;\n                    try {\n                        idStoringThisValue = t.read(newValueKey).stringValue();\n                    } catch (NotFoundException e) {} // this value does not exist yet, therefore there is no conflict\n                    \n                    if (idStoringThisValue != null && !isDeletedRecord(idStoringThisValue) \n                            && !idStoringThisValue.equals(objectStringIdentity)) {\n                        // another object has stored this value -> violation of uniqueness\n                        throw new NucleusDataStoreException(\"The value '\" + newFieldValue + \"' of unique member '\" + \n                                fieldName + \"' of class '\" + className + \"' already exists\");\n                    } else {\n                        t.write(newValueKey, objectStringIdentity);\n                    }\n                }\n            }\n        }\n    }\n\n    /**\n     * To support unique member values (@Unique annotation) an extra key is\n     * inserted to signal the object ID whose object has stored the given value.\n     * Deletes the keys belonging to the object represented by the ObjectProvider\n     * @param op\n     *      The data source representing the object\n     * @param oldJson\n     *      The current state of the object in JSON form.\n     * @param t\n     *      The transaction used for the performed operations. \n     * @throws ConnectionException\n     * @throws UnknownException\n     */\n    private static void removeObjectFromUniqueMemberKey(ObjectProvider<?> op, JSONObject oldJson, Transaction t) \n            throws ConnectionException, UnknownException {\n        AbstractClassMetaData cmd = op.getClassMetaData();\n        String className = cmd.getFullClassName();\n        \n        for (int field : cmd.getAllMemberPositions()) {\n            AbstractMemberMetaData mmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition(field);\n            UniqueMetaData umd = mmd.getUniqueMetaData();\n            if (umd != null) {\n                // this member has @Unique annotation -> lookup all stored values for this member\n                String fieldName = mmd.getName();\n\n                String oldFieldValue = null;\n                try {\n                    oldFieldValue = oldJson.has(fieldName) ? oldJson.getString(fieldName) : null;\n                } catch (JSONException e) {} // can not happen since it is checked before \n                \n                if (oldFieldValue != null) {\n                    String oldValueKey = ScalarisSchemaHandler.getUniqueMemberKey(className, fieldName, oldFieldValue);\n                    t.write(oldValueKey, DELETED_RECORD_VALUE);\n                }\n            }\n        }\n    }\n\n    /* **********************************************************************\n     *                  FOREIGN KEY ACTIONS\n     * **********************************************************************/\n\n    /**\n     * All foreign key action of an object are stored at a separate key.\n     * If (persisted) object A has a ForeignKeyAction.CASCADE annotation at a member\n     * whose value is (persisted) object B, An entry is added to B's FKA-index.\n     * If an persisted object is deleted, all entries of its FKA-index will be processed.\n     *\n     * Checks a newly inserted object for foreign key actions.\n     *\n     * @param op\n     *      The data source of the object\n     * @param objToInsert\n     *      The JSON representation of the object.\n     * @param t\n     *      The transaction used to perform the operations\n     * @throws ConnectionException\n     * @throws NotAListException\n     */\n    private static void insertToForeignKeyAction(ObjectProvider<?> op, JSONObject objToInsert, Transaction t)\n            throws ConnectionException, NotAListException {\n        updateForeignKeyAction(op, objToInsert, null, t);\n    }\n\n    /**\n     * All foreign key action of an object are stored at a separate key.\n     * If (persisted) object A has a ForeignKeyAction.CASCADE annotation at a member\n     * whose value is (persisted) object B, An entry is added to B's FKA-index.\n     * If an persisted object is deleted, all entries of its FKA-index will be processed.\n     *\n     * Deletes the FKAs of the old objects state and adds new FKAs according to its new state.\n     *\n     * @param op\n     *      The data source of the object\n     * @param changedFiledsNewVal\n     *      The JSON representation of the objects new state. It is sufficient if the JSON\n     *      only contains the changed fields value.\n     * @param changedFiledsOldVal\n     *      The JSON representation of the objects old state. It is sufficient if the JSON\n     *      only contains the changed fields value. If this is null this operation is treated\n     *      as an insert of new object, thus not deleting any old FKAs\n     * @param t\n     *      The transaction used to perform the operations\n     * @throws ConnectionException\n     * @throws NotAListException\n     */\n    @SuppressWarnings(\"unchecked\")\n    private static void updateForeignKeyAction(ObjectProvider<?> op, JSONObject changedFieldsNewVal, \n            JSONObject changedFieldsOldVal, Transaction t)\n            throws ConnectionException, NotAListException {\n        AbstractClassMetaData cmd = op.getClassMetaData();\n        String objectStringIdentity = getPersistableIdentity(op);\n\n        // the map will store all elements which must be added/removed from which key\n        HashMap<String, List<ErlangValue>> toAddToKey = new HashMap<String, List<ErlangValue>>();\n        HashMap<String, List<ErlangValue>> toRemoveFromKey = new HashMap<String, List<ErlangValue>>();\n\n        for (int field : cmd.getAllMemberPositions()) {\n            AbstractMemberMetaData mmd = cmd.getMetaDataForManagedMemberAtAbsolutePosition(field);\n            // do nothing if this field has not changed\n            if (mmd == null || !changedFieldsNewVal.has(mmd.getName())) continue;\n\n            ForeignKeyMetaData fmd = mmd.getForeignKeyMetaData();\n            boolean isJoin = false;\n            if (mmd.getJoinMetaData() != null) {\n                // The member is a collection with an ForeignKeyAction attached\n                fmd = mmd.getJoinMetaData().getForeignKeyMetaData();\n                isJoin = true;\n            }\n            // add to actions keys if it is a cascading delete\n            if (fmd != null && fmd.getDeleteAction() == ForeignKeyAction.CASCADE) {\n                String fieldName = mmd.getName();\n                // parse JSON entries \n                ArrayList<String> foreignObjectIdsNew = new ArrayList<String>();\n                ArrayList<String> foreignObjectIdsOld = new ArrayList<String>();\n                try {\n                    if (isJoin) {\n                        String elementClassName = mmd.getCollection().getElementType();\n                        JSONArray arrNew = changedFieldsNewVal.getJSONArray(fieldName);\n                        JSONArray arrOld = new JSONArray();\n                        if (changedFieldsOldVal != null) {\n                            arrOld = changedFieldsOldVal.getJSONArray(fieldName);\n                        }\n                        for (int i = 0; i < arrNew.length(); i++) {\n                            foreignObjectIdsNew.add(\n                                    ScalarisSchemaHandler.getForeignKeyActionKey(\n                                            elementClassName, arrNew.getString(i)));\n                        }\n                        for (int i = 0; i < arrOld.length(); i++) {\n                            foreignObjectIdsOld.add(\n                                    ScalarisSchemaHandler.getForeignKeyActionKey(\n                                            elementClassName, arrOld.getString(i)));\n                        }\n                    } else {\n                        String className = mmd.getType().getCanonicalName();\n                        if (className == null) {\n                             className = mmd.getType().getName();\n                        }\n                        foreignObjectIdsNew.add(\n                                ScalarisSchemaHandler.getForeignKeyActionKey(\n                                        className, changedFieldsNewVal.getString(fieldName)));\n                    }\n                } catch (JSONException e) {\n                    // not found -> this action will be skipped\n                }\n                // ignore the objects in both the new and old list\n                for (int i = foreignObjectIdsNew.size() - 1; i >= 0; i--) {\n                    String s = foreignObjectIdsNew.get(i);\n                    if (foreignObjectIdsOld.remove(s)) {\n                        foreignObjectIdsNew.remove(s);\n                    }\n                }\n\n                // construct the FKA key for every foreign object id\n                ArrayList<String>[] bothNewAndOld = new ArrayList[]{foreignObjectIdsOld, foreignObjectIdsNew};\n                for (ArrayList<String> changeList : bothNewAndOld) {\n                    for (String fkaKey : changeList) {\n                        if (fkaKey == null) continue;\n\n                        List<String> newFka = new ArrayList<String>(2);\n                        newFka.add(objectStringIdentity);\n                        newFka.add(cmd.getFullClassName());\n                        if (isJoin) {\n                            newFka.add(fieldName);\n                        } else {\n                            newFka.add(FKA_DELETE_OBJ);\n                        }\n\n                        // check in which list we are currently in to choose the\n                        // HashMap to which the action must be added to\n                        HashMap<String, List<ErlangValue>> toChangeMap = (foreignObjectIdsNew == changeList) ?\n                                toAddToKey : toRemoveFromKey;\n\n                        List<ErlangValue> toChange = toChangeMap.get(fkaKey);\n                        if (toChange == null) {\n                            toChange = new ArrayList<ErlangValue>();\n                        }\n                        toChange.add(new ErlangValue(newFka));\n                        toChangeMap.put(fkaKey, toChange);\n                    }\n                }\n            }\n        }\n\n        // update all keys where new entries are added\n        for (Entry<String, List<ErlangValue>> entry : toAddToKey.entrySet()) {\n            List<ErlangValue> toAdd = entry.getValue();\n            List<ErlangValue> toRemove = new ArrayList<ErlangValue>(0);\n            if (toRemoveFromKey.containsKey(entry.getKey())) {\n                toRemove = toRemoveFromKey.get(entry.getKey());\n            }\n            t.addDelOnList(entry.getKey(), toAdd, toRemove);\n        }\n        // update the remaining keys (only deletions)\n        for (Entry<String, List<ErlangValue>> entry : toRemoveFromKey.entrySet()) {\n            if (toAddToKey.containsKey(entry.getKey())) {\n                List<ErlangValue> toRemove = entry.getValue();\n                t.addDelOnList(entry.getKey(), new ArrayList<ErlangValue>(0), toRemove);\n            }\n        }\n    }\n\n    /**\n     * All foreign key action of an object are stored at a separate key.\n     * If (persisted) object A has a ForeignKeyAction.CASCADE annotation at a member\n     * whose value is (persisted) object B, An entry is added to B's FKA-index.\n     * If an persisted object is deleted, all entries of its FKA-index will be processed.\n     *\n     * This method will perform all foreign key actions of the object represented by the\n     * passed ObjectProvider.\n     *\n     * @param op\n     *      The data source of the object\n     * @param t\n     *      The transaction used to perform the operations\n     * @throws ConnectionException\n     * @throws NotAListException\n     */\n    private static void performForeignKeyActionDelete(ObjectProvider<?> op, Transaction t)\n            throws ConnectionException, NotAListException {\n        AbstractClassMetaData cmd = op.getClassMetaData();\n        String objClassName = cmd.getFullClassName();\n        String objectStringIdentity = getPersistableIdentity(op);\n        \n        ExecutionContext ec = op.getExecutionContext();\n        StoreManager storeMgr = ec.getStoreManager();\n        \n        String fkaKey = ScalarisSchemaHandler.getForeignKeyActionKey(objClassName, objectStringIdentity);\n        List<ErlangValue> attachedActions;\n        try {\n            attachedActions = t.read(fkaKey).listValue();\n        } catch (NotFoundException e) {\n            // there is no FKA key for this object --> there are no actions to perform\n            return;\n        }\n        // now search in every found action entries with op's id and start a delete as sub transaction\n        for (ErlangValue action : attachedActions) {\n            try {\n                List<String> actionAsList = action.stringListValue();\n                \n                String objId = actionAsList.get(0);\n                String toDeleteClassName = actionAsList.get(1);\n                String memberToDelete = actionAsList.get(2);\n                \n                AbstractClassMetaData obCmd = storeMgr.getMetaDataManager()\n                        .getMetaDataForClass(toDeleteClassName, ec.getClassLoaderResolver());\n                Object obj = IdentityUtils.getObjectFromPersistableIdentity(objId, obCmd, ec);\n                \n                if (FKA_DELETE_OBJ.equals(memberToDelete)) {\n                    // delete the complete object\n                    ec.deleteObject(obj);\n                } else {\n                    // remove the object reference of the deleted object from the collection\n    \n                    ObjectProvider<?> toDelOp = ec.findObjectProvider(obj);\n    \n                    int memberId = toDelOp.getClassMetaData().getAbsolutePositionOfMember(memberToDelete);\n    \n                    if (toDelOp.isFieldLoaded(memberId)) {\n                        // the field is loaded which means that the object could be used by the overlying \n                        // application\n                        Object objColl = toDelOp.provideField(memberId);\n    \n                        if (objColl instanceof Collection) {\n                            Collection<?> collection = (Collection<?>) objColl;\n    \n                            ArrayList<Object> toRemove = new ArrayList<Object>();\n                            Iterator<?> iter = collection.iterator();\n                            while (iter.hasNext()) {\n                                Object item = iter.next();\n                                ObjectProvider<?> itemOp = ec.findObjectProvider(item);\n                                String itemId = getPersistableIdentity(itemOp);\n    \n                                if (itemId.equals(objectStringIdentity)) {\n                                    toRemove.add(item);\n                                }\n                            }\n                            for (Object o : toRemove) {\n                                collection.remove(o);\n                            }\n                            storeMgr.getPersistenceHandler().updateObject(toDelOp, new int[]{memberId});\n                            // updated field must be marked as dirty to ensure that instances of this object used\n                            // somewhere else will fetch the updated value\n                            toDelOp.makeDirty(memberId);\n                        }\n                    } else {\n                        // if the member is not loaded it is way faster to directly alter the\n                        // stored JSON object\n                        String objKey = ScalarisSchemaHandler.getObjectStorageKey(toDeleteClassName, objId);\n                        JSONObject objAsJson = new JSONObject(t.read(objKey).stringValue());\n                        if (isDeletedRecord(objAsJson)) {\n                            continue;\n                        }\n                        JSONArray memberArr = objAsJson.getJSONArray(memberToDelete);\n                        JSONArray newMemberArr = new JSONArray();\n                        for (int j = 0; j < memberArr.length(); j++) {\n                            if (!memberArr.get(j).equals(objectStringIdentity)) {\n                                newMemberArr.put(memberArr.get(j));\n                            }\n                        }\n                        // only write new value if something changed\n                        if (newMemberArr.length() != memberArr.length()) {\n                            objAsJson.put(memberToDelete, newMemberArr);\n                            t.write(objId, objAsJson.toString());\n                            // if the object is not removed from cache after this update\n                            // it is possible that the cached value returned is out dated\n                            ec.removeObjectFromLevel1Cache(toDelOp.getInternalObjectId());\n                            ec.removeObjectFromLevel2Cache(toDelOp.getInternalObjectId());\n                        }\n                    }\n                }\n            } catch (NucleusObjectNotFoundException e) {\n                // the object we want to delete is already deleted \n                // nothing must be done\n            } catch (NotFoundException e) {\n                throw new NucleusObjectNotFoundException(e.getMessage(), e);\n            } catch (JSONException e) {\n                throw new NucleusDataStoreException(e.getMessage(), e);\n                // the member containing the current object does not exist any more\n                // that means we don't have to remove it any more\n            }\n        }\n        // all objects are gone\n        t.write(fkaKey, DELETED_RECORD_VALUE);\n    }\n\n    /* **********************************************************************\n     *                     MISC\n     * **********************************************************************/\n\n    /**\n     * Scalaris does not support deletion (in a usable way). Therefore, deletion\n     * is simulated by overwriting an object with a \"deleted\" value.\n     * \n     * This method returns true if record is a JSON of a deleted record.\n     * \n     * @param record\n     *      The JSON to check.\n     * @return true if record is JSON of a deleted record, false otherwise.\n     */\n    public static boolean isDeletedRecord(final JSONObject record) {\n        return record == null || isDeletedRecord(record.toString());\n    }\n    \n    public static boolean isDeletedRecord(final String record) {\n        return record == null || record.isEmpty() || record.equals(DELETED_RECORD_VALUE);\n    }\n}\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/src/java/org/datanucleus/store/scalaris/fieldmanager/FetchFieldManager.java",
    "content": "/**********************************************************************\r\nCopyright (c) 2008 Erik Bengtson and others. All rights reserved.\r\nLicensed under the Apache License, Version 2.0 (the \"License\");\r\nyou may not use this file except in compliance with the License.\r\nYou may obtain a copy of the License at\r\n\r\n    http://www.apache.org/licenses/LICENSE-2.0\r\n\r\nUnless required by applicable law or agreed to in writing, software\r\ndistributed under the License is distributed on an \"AS IS\" BASIS,\r\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nSee the License for the specific language governing permissions and\r\nlimitations under the License.\r\n\r\nContributors:\r\n2013 Orange - port to Scalaris key/value store\r\n    ...\r\n **********************************************************************/\r\npackage org.datanucleus.store.scalaris.fieldmanager;\r\n\r\nimport java.lang.reflect.Array;\r\nimport java.math.BigDecimal;\r\nimport java.math.BigInteger;\r\nimport java.util.Collection;\r\nimport java.util.Date;\r\nimport java.util.Iterator;\r\nimport java.util.Map;\r\n\r\nimport org.datanucleus.ClassLoaderResolver;\r\nimport org.datanucleus.ExecutionContext;\r\nimport org.datanucleus.exceptions.NucleusDataStoreException;\r\nimport org.datanucleus.exceptions.NucleusException;\r\nimport org.datanucleus.exceptions.NucleusObjectNotFoundException;\r\nimport org.datanucleus.identity.IdentityUtils;\r\nimport org.datanucleus.metadata.AbstractClassMetaData;\r\nimport org.datanucleus.metadata.AbstractMemberMetaData;\r\nimport org.datanucleus.metadata.ColumnMetaData;\r\nimport org.datanucleus.metadata.JdbcType;\r\nimport org.datanucleus.metadata.RelationType;\r\nimport org.datanucleus.state.ObjectProvider;\r\nimport org.datanucleus.store.StoreManager;\r\nimport org.datanucleus.store.fieldmanager.AbstractFieldManager;\r\nimport org.datanucleus.store.schema.naming.ColumnType;\r\nimport org.datanucleus.store.types.SCOUtils;\r\nimport org.datanucleus.store.types.converters.TypeConverter;\r\nimport org.datanucleus.store.types.converters.TypeConverterHelper;\r\nimport org.datanucleus.util.TypeConversionHelper;\r\n\r\nimport org.json.JSONArray;\r\nimport org.json.JSONException;\r\nimport org.json.JSONObject;\r\n\r\n/**\r\n * FieldManager for fetching from JSON.\r\n */\r\npublic class FetchFieldManager extends AbstractFieldManager {\r\n    protected final ObjectProvider<?> op;\r\n    protected final AbstractClassMetaData acmd;\r\n    protected final ExecutionContext ec;\r\n    protected final JSONObject result;\r\n    protected StoreManager storeMgr;\r\n\r\n    public FetchFieldManager(ExecutionContext ec, AbstractClassMetaData acmd,\r\n            JSONObject result) {\r\n        this.acmd = acmd;\r\n        this.ec = ec;\r\n        this.result = result;\r\n        this.op = null;\r\n        this.storeMgr = ec.getStoreManager();\r\n    }\r\n\r\n    public FetchFieldManager(ObjectProvider<?> op, JSONObject result) {\r\n        this.acmd = op.getClassMetaData();\r\n        this.ec = op.getExecutionContext();\r\n        this.result = result;\r\n        this.op = op;\r\n        this.storeMgr = ec.getStoreManager();\r\n    }\r\n\r\n    public boolean fetchBooleanField(int fieldNumber) {\r\n        String memberName = storeMgr\r\n                .getNamingFactory()\r\n                .getColumnName(\r\n                        acmd.getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber),\r\n                        ColumnType.COLUMN);\r\n        if (result.isNull(memberName)) {\r\n            return false;\r\n        }\r\n        try {\r\n            return result.getBoolean(memberName);\r\n        } catch (JSONException e) {\r\n            // ignore\r\n            return false;\r\n        }\r\n    }\r\n\r\n    public byte fetchByteField(int fieldNumber) {\r\n        String memberName = storeMgr\r\n                .getNamingFactory()\r\n                .getColumnName(\r\n                        acmd.getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber),\r\n                        ColumnType.COLUMN);\r\n        if (result.isNull(memberName)) {\r\n            return 0;\r\n        }\r\n        try {\r\n            String str = result.getString(memberName);\r\n            return Byte.valueOf(str).byteValue();\r\n        } catch (JSONException e) {\r\n            // ignore\r\n            return 0;\r\n        }\r\n    }\r\n\r\n    public char fetchCharField(int fieldNumber) {\r\n        String memberName = storeMgr\r\n                .getNamingFactory()\r\n                .getColumnName(\r\n                        acmd.getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber),\r\n                        ColumnType.COLUMN);\r\n        if (result.isNull(memberName)) {\r\n            return 0;\r\n        }\r\n        try {\r\n            return result.getString(memberName).charAt(0);\r\n        } catch (JSONException e) {\r\n            // ignore\r\n            return 0;\r\n        }\r\n    }\r\n\r\n    public double fetchDoubleField(int fieldNumber) {\r\n        String memberName = storeMgr\r\n                .getNamingFactory()\r\n                .getColumnName(\r\n                        acmd.getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber),\r\n                        ColumnType.COLUMN);\r\n        if (result.isNull(memberName)) {\r\n            return 0;\r\n        }\r\n        try {\r\n            return result.getDouble(memberName);\r\n        } catch (JSONException e) {\r\n            // ignore\r\n            return 0;\r\n        }\r\n    }\r\n\r\n    public float fetchFloatField(int fieldNumber) {\r\n        String memberName = storeMgr\r\n                .getNamingFactory()\r\n                .getColumnName(\r\n                        acmd.getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber),\r\n                        ColumnType.COLUMN);\r\n        if (result.isNull(memberName)) {\r\n            return 0;\r\n        }\r\n        try {\r\n            return (float) result.getDouble(memberName);\r\n        } catch (JSONException e) {\r\n            // ignore\r\n            return 0;\r\n        }\r\n    }\r\n\r\n    public int fetchIntField(int fieldNumber) {\r\n        String memberName = storeMgr\r\n                .getNamingFactory()\r\n                .getColumnName(\r\n                        acmd.getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber),\r\n                        ColumnType.COLUMN);\r\n        if (result.isNull(memberName)) {\r\n            return 0;\r\n        }\r\n        try {\r\n            return result.getInt(memberName);\r\n        } catch (JSONException e) {\r\n            // ignore\r\n            return 0;\r\n        }\r\n    }\r\n\r\n    public long fetchLongField(int fieldNumber) {\r\n        String memberName = storeMgr\r\n                .getNamingFactory()\r\n                .getColumnName(\r\n                        acmd.getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber),\r\n                        ColumnType.COLUMN);\r\n        if (result.isNull(memberName)) {\r\n            return 0;\r\n        }\r\n        try {\r\n            return result.getLong(memberName);\r\n        } catch (JSONException e) {\r\n            // ignore\r\n            return 0;\r\n        }\r\n    }\r\n\r\n    public short fetchShortField(int fieldNumber) {\r\n        String memberName = storeMgr\r\n                .getNamingFactory()\r\n                .getColumnName(\r\n                        acmd.getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber),\r\n                        ColumnType.COLUMN);\r\n        if (result.isNull(memberName)) {\r\n            return 0;\r\n        }\r\n        try {\r\n            return (short) result.getInt(memberName);\r\n        } catch (JSONException e) {\r\n            // ignore\r\n            return 0;\r\n        }\r\n    }\r\n\r\n    public String fetchStringField(int fieldNumber) {\r\n        String memberName = storeMgr\r\n                .getNamingFactory()\r\n                .getColumnName(\r\n                        acmd.getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber),\r\n                        ColumnType.COLUMN);\r\n        if (result.isNull(memberName)) {\r\n            return null;\r\n        }\r\n        try {\r\n            return result.getString(memberName);\r\n        } catch (JSONException e) {\r\n            // ignore\r\n            return null;\r\n        }\r\n    }\r\n\r\n    public Object fetchObjectField(int fieldNumber) {\r\n        AbstractMemberMetaData mmd = acmd\r\n                .getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber);\r\n        String memberName = storeMgr.getNamingFactory().getColumnName(mmd,\r\n                ColumnType.COLUMN);\r\n\r\n        if (result.isNull(memberName)) {\r\n            return null;\r\n        }\r\n\r\n        // Special cases\r\n        ClassLoaderResolver clr = ec.getClassLoaderResolver();\r\n        RelationType relationType = mmd.getRelationType(clr);\r\n        if (RelationType.isRelationSingleValued(relationType)\r\n                && mmd.isEmbedded()) {\r\n            throw new NucleusException(\r\n                    \"Don't currently support embedded fields\");\r\n        }\r\n\r\n        try {\r\n            return fetchObjectFieldInternal(mmd, memberName, clr);\r\n        } catch (JSONException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        }\r\n    }\r\n\r\n    @SuppressWarnings({ \"unchecked\", \"rawtypes\" })\r\n    protected Object fetchObjectFieldInternal_RelationTypeNone(\r\n            AbstractMemberMetaData mmd, String memberName,\r\n            ClassLoaderResolver clr) throws JSONException {\r\n        final Object returnValue;\r\n        if (mmd.getTypeConverterName() != null) {\r\n            // User-defined converter\r\n            TypeConverter<Object,Object> conv = ec.getNucleusContext().getTypeManager()\r\n                    .getTypeConverterForName(mmd.getTypeConverterName());\r\n            Class<?> datastoreType = TypeConverterHelper\r\n                    .getDatastoreTypeForTypeConverter(conv, mmd.getType());\r\n            if (datastoreType == String.class) {\r\n                returnValue = (Object)conv.toMemberType(result.getString(memberName));\r\n            } else if (datastoreType == Boolean.class) {\r\n                returnValue = conv.toMemberType(result.getBoolean(memberName));\r\n            } else if (datastoreType == Double.class) {\r\n                returnValue = conv.toMemberType(result.getDouble(memberName));\r\n            } else if (datastoreType == Float.class) {\r\n                returnValue = conv.toMemberType(result.getDouble(memberName));\r\n            } else if (datastoreType == Integer.class) {\r\n                returnValue = conv.toMemberType(result.getInt(memberName));\r\n            } else if (datastoreType == Long.class) {\r\n                returnValue = conv.toMemberType(result.getLong(memberName));\r\n            } else {\r\n                returnValue = null;\r\n            }\r\n            if (op != null) {\r\n                return SCOUtils.wrapSCOField(op, mmd.getAbsoluteFieldNumber(), returnValue, true);\r\n            }\r\n        } else if (Boolean.class.isAssignableFrom(mmd.getType())) {\r\n            return result.getBoolean(memberName);\r\n        } else if (Integer.class.isAssignableFrom(mmd.getType())) {\r\n            return result.getInt(memberName);\r\n        } else if (Long.class.isAssignableFrom(mmd.getType())) {\r\n            return result.getLong(memberName);\r\n        } else if (Double.class.isAssignableFrom(mmd.getType())) {\r\n            return result.getDouble(memberName);\r\n        } else if (Date.class.isAssignableFrom(mmd.getType())) {\r\n            Long dateValue = result.getLong(memberName);\r\n            return new Date(dateValue);\r\n        } else if (Enum.class.isAssignableFrom(mmd.getType())) {\r\n            if (mmd.getType().getEnumConstants() != null) {\r\n                return mmd.getType().getEnumConstants()[result\r\n                        .getInt(memberName)];\r\n            } else {\r\n                return Enum.valueOf(mmd.getType(),\r\n                        (String) result.get(memberName));\r\n            }\r\n        } else if (BigDecimal.class.isAssignableFrom(mmd.getType())\r\n                || BigInteger.class.isAssignableFrom(mmd.getType())) {\r\n            return TypeConversionHelper.convertTo(result.get(memberName),\r\n                    mmd.getType());\r\n        } else if (Collection.class.isAssignableFrom(mmd.getType())) {\r\n            // Collection<Non-PC>\r\n            Collection<Object> coll;\r\n            try {\r\n                Class<?> instanceType = SCOUtils.getContainerInstanceType(\r\n                        mmd.getType(), mmd.getOrderMetaData() != null);\r\n                coll = (Collection<Object>) instanceType.newInstance();\r\n            } catch (Exception e) {\r\n                throw new NucleusDataStoreException(e.getMessage(), e);\r\n            }\r\n\r\n            JSONArray array = result.getJSONArray(memberName);\r\n            Class<?> elementCls = null;\r\n            if (mmd.getCollection() != null\r\n                    && mmd.getCollection().getElementType() != null) {\r\n                elementCls = clr.classForName(mmd.getCollection()\r\n                        .getElementType());\r\n            }\r\n            for (int i = 0; i < array.length(); i++) {\r\n                if (array.isNull(i)) {\r\n                    coll.add(null);\r\n                } else {\r\n                    Object value = array.get(i);\r\n                    if (elementCls != null) {\r\n                        coll.add(TypeConversionHelper.convertTo(value,\r\n                                elementCls));\r\n                    } else {\r\n                        coll.add(value);\r\n                    }\r\n                }\r\n            }\r\n\r\n            if (op != null) {\r\n                return SCOUtils.wrapSCOField(op, mmd.getAbsoluteFieldNumber(), coll, true);\r\n            }\r\n            return coll;\r\n        } else if (Map.class.isAssignableFrom(mmd.getType())) {\r\n            // Map<Non-PC, Non-PC>\r\n            Map map;\r\n            try {\r\n                Class<?> instanceType = SCOUtils.getContainerInstanceType(\r\n                        mmd.getType(), false);\r\n                map = (Map) instanceType.newInstance();\r\n            } catch (Exception e) {\r\n                throw new NucleusDataStoreException(e.getMessage(), e);\r\n            }\r\n\r\n            JSONObject mapValue = result.getJSONObject(memberName);\r\n            Iterator<?> keyIter = mapValue.keys();\r\n            Class<?> keyCls = null;\r\n            if (mmd.getMap() != null && mmd.getMap().getKeyType() != null) {\r\n                keyCls = clr.classForName(mmd.getMap().getKeyType());\r\n            }\r\n            Class<?> valCls = null;\r\n            if (mmd.getMap() != null && mmd.getMap().getValueType() != null) {\r\n                valCls = clr.classForName(mmd.getMap().getValueType());\r\n            }\r\n\r\n            while (keyIter.hasNext()) {\r\n                Object jsonKey = keyIter.next();\r\n\r\n                Object key = jsonKey;\r\n                if (keyCls != null) {\r\n                    key = TypeConversionHelper.convertTo(jsonKey, keyCls);\r\n                }\r\n\r\n                Object jsonVal = mapValue.get((String) key);\r\n                Object val = jsonVal;\r\n                if (valCls != null) {\r\n                    val = TypeConversionHelper.convertTo(jsonVal, valCls);\r\n                }\r\n                map.put(key, val);\r\n            }\r\n\r\n            if (op != null) {\r\n                return SCOUtils.wrapSCOField(op, mmd.getAbsoluteFieldNumber(), map, true);\r\n            }\r\n            return map;\r\n        } else if (mmd.getType().isArray()) {\r\n            // Non-PC[]\r\n            JSONArray arrayJson = result.getJSONArray(memberName);\r\n            Object array = Array.newInstance(mmd.getType().getComponentType(),\r\n                    arrayJson.length());\r\n            for (int i = 0; i < arrayJson.length(); i++) {\r\n                if (arrayJson.isNull(i)) {\r\n                    Array.set(array, i, null);\r\n                } else {\r\n                    Object value = arrayJson.get(i);\r\n                    Array.set(array, i, TypeConversionHelper.convertTo(\r\n                            value, mmd.getType().getComponentType()));\r\n                }\r\n            }\r\n            return array;\r\n        } else {\r\n            // Fallback to built-in type converters\r\n            boolean useLong = false;\r\n            ColumnMetaData[] colmds = mmd.getColumnMetaData();\r\n            if (colmds != null && colmds.length == 1) {\r\n                JdbcType jdbcType = colmds[0].getJdbcType();\r\n                if (jdbcType != null) {\r\n                    String jdbc = jdbcType.name();\r\n                    if (jdbc != null\r\n                            && (jdbc.equalsIgnoreCase(\"INTEGER\") || jdbc\r\n                                    .equalsIgnoreCase(\"NUMERIC\"))) {\r\n                        useLong = true;\r\n                    }\r\n                }\r\n            }\r\n            TypeConverter strConv = ec.getNucleusContext().getTypeManager()\r\n                    .getTypeConverterForType(mmd.getType(), String.class);\r\n            TypeConverter longConv = ec.getNucleusContext().getTypeManager()\r\n                    .getTypeConverterForType(mmd.getType(), Long.class);\r\n            \r\n            if (useLong && longConv != null) {\r\n                returnValue = longConv.toMemberType(result.getLong(memberName));\r\n            } else if (!useLong && strConv != null) {\r\n                returnValue = strConv.toMemberType((String) result\r\n                        .get(memberName));\r\n            } else if (!useLong && longConv != null) {\r\n                returnValue = longConv.toMemberType(result.getLong(memberName));\r\n            } else {\r\n                returnValue = TypeConversionHelper.convertTo(\r\n                        result.get(memberName), mmd.getType());\r\n            }\r\n\r\n            if (op != null) {\r\n                return SCOUtils.wrapSCOField(op, mmd.getAbsoluteFieldNumber(), returnValue, true);\r\n            }\r\n        }\r\n        return returnValue;\r\n    }\r\n\r\n    @SuppressWarnings({ \"unchecked\", \"rawtypes\" })\r\n    protected Object fetchObjectFieldInternal(AbstractMemberMetaData mmd,\r\n            String memberName, ClassLoaderResolver clr) throws JSONException {\r\n\r\n        RelationType relationType = mmd.getRelationType(clr);\r\n        if (relationType == RelationType.NONE) {\r\n            return fetchObjectFieldInternal_RelationTypeNone(mmd, memberName,\r\n                    clr);\r\n        } else if (RelationType.isRelationSingleValued(relationType)) {\r\n            // Persistable object - retrieve the string form of the identity,\r\n            // and find the object\r\n            String idStr = (String) result.get(memberName);\r\n            if (idStr == null) {\r\n                return null;\r\n            }\r\n            AbstractClassMetaData acmd = ec.getMetaDataManager().getMetaDataForClass(mmd.getType(), clr);\r\n            return getNestedObjectById(idStr,acmd, ec);\r\n            \r\n        } else if (RelationType.isRelationMultiValued(relationType)) {\r\n            if (mmd.hasCollection()) {\r\n                // Collection<PC>\r\n                JSONArray array = (JSONArray) result.get(memberName);\r\n                Collection<Object> coll;\r\n                try {\r\n                    Class<?> instanceType = SCOUtils.getContainerInstanceType(\r\n                            mmd.getType(), mmd.getOrderMetaData() != null);\r\n                    coll = (Collection<Object>) instanceType.newInstance();\r\n                } catch (Exception e) {\r\n                    throw new NucleusDataStoreException(e.getMessage(), e);\r\n                }\r\n\r\n                AbstractClassMetaData elementCmd = mmd.getCollection()\r\n                        .getElementClassMetaData(ec.getClassLoaderResolver(),\r\n                                ec.getMetaDataManager());\r\n                for (int i = 0; i < array.length(); i++) {\r\n                    String idStr = (String) array.get(i);\r\n                    if (idStr == null) {\r\n                        coll.add(idStr);\r\n                    } else {\r\n                        Object element = getNestedObjectById(idStr, elementCmd, ec);\r\n                        if (element != null) {\r\n                            coll.add(element);\r\n                        }\r\n                    }\r\n                }\r\n\r\n                if (op != null) {\r\n                    return SCOUtils.wrapSCOField(op, mmd.getAbsoluteFieldNumber(), coll, true);\r\n                }\r\n                return coll;\r\n            } else if (mmd.hasArray()) {\r\n                // PC[]\r\n                JSONArray array = (JSONArray) result.get(memberName);\r\n                Object arrayField = Array.newInstance(mmd.getType()\r\n                        .getComponentType(), array.length());\r\n\r\n                AbstractClassMetaData elementCmd = mmd.getCollection()\r\n                        .getElementClassMetaData(ec.getClassLoaderResolver(),\r\n                                ec.getMetaDataManager());\r\n                for (int i = 0; i < array.length(); i++) {\r\n                    String idStr = (String) array.get(i);\r\n                    if (idStr == null) {\r\n                        Array.set(arrayField, i, idStr);\r\n                    } else {\r\n                        Object element = getNestedObjectById(idStr, elementCmd, ec);\r\n                        Array.set(arrayField, i, element);\r\n                    }\r\n                }\r\n\r\n                if (op != null) {\r\n                    return SCOUtils.wrapSCOField(op, mmd.getAbsoluteFieldNumber(), arrayField, true);\r\n                }\r\n                return arrayField;\r\n            } else if (mmd.hasMap()) {\r\n                // Map<Non-PC, PC>, Map<PC, PC>, Map<PC, Non-PC>\r\n                JSONObject mapVal = (JSONObject) result.get(memberName);\r\n                Map map;\r\n                try {\r\n                    Class<?> instanceType = SCOUtils.getContainerInstanceType(\r\n                            mmd.getType(), false);\r\n                    map = (Map) instanceType.newInstance();\r\n                } catch (Exception e) {\r\n                    throw new NucleusDataStoreException(e.getMessage(), e);\r\n                }\r\n\r\n                AbstractClassMetaData keyCmd = mmd.getMap()\r\n                        .getKeyClassMetaData(clr, ec.getMetaDataManager());\r\n                AbstractClassMetaData valCmd = mmd.getMap()\r\n                        .getValueClassMetaData(clr, ec.getMetaDataManager());\r\n\r\n                Iterator<?> keyIter = mapVal.keys();\r\n                while (keyIter.hasNext()) {\r\n                    Object jsonKey = keyIter.next();\r\n                    Object key = null;\r\n                    if (keyCmd != null) {\r\n                        // The jsonKey is the string form of the identity\r\n                        String idStr = (String) jsonKey;\r\n                        key = getNestedObjectById(idStr, keyCmd, ec);\r\n                    } else {\r\n                        Class<?> keyCls = ec.getClassLoaderResolver()\r\n                                .classForName(mmd.getMap().getKeyType());\r\n                        key = TypeConversionHelper.convertTo(jsonKey, keyCls);\r\n                    }\r\n\r\n                    Object jsonVal = mapVal.get((String) key);\r\n                    Object val = null;\r\n                    if (valCmd != null) {\r\n                        // The jsonVal is the string form of the identity\r\n                        String idStr = (String) jsonVal;\r\n                        val = getNestedObjectById(idStr, valCmd, ec);\r\n                    } else {\r\n                        Class valCls = ec.getClassLoaderResolver()\r\n                                .classForName(mmd.getMap().getValueType());\r\n                        val = TypeConversionHelper.convertTo(jsonVal, valCls);\r\n                    }\r\n\r\n                    map.put(key, val);\r\n                }\r\n\r\n                if (op != null) {\r\n                    return SCOUtils.wrapSCOField(op, mmd.getAbsoluteFieldNumber(), map, true);\r\n                }\r\n                return map;\r\n            }\r\n        }\r\n\r\n        throw new NucleusException(\"Dont currently support field \"\r\n                + mmd.getFullFieldName() + \" of type \" + mmd.getTypeName());\r\n    }\r\n\r\n    private Object getNestedObjectById(String persistableId, AbstractClassMetaData acmd, ExecutionContext ec) {\r\n        try {\r\n            return IdentityUtils.getObjectFromPersistableIdentity(persistableId, acmd, ec);\r\n        } catch (NucleusObjectNotFoundException e) {\r\n            // TODO This can happen when an object currently in use in the external\r\n            // application has references to other objects, which in its lifetime are deleted but the\r\n            // reference stored is not updated yet\r\n            // This happening should be prevented.\r\n            return null;\r\n        }\r\n    }\r\n}\r\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/src/java/org/datanucleus/store/scalaris/fieldmanager/StoreFieldManager.java",
    "content": "/**********************************************************************\r\nCopyright (c) 2008 Erik Bengtson and others. All rights reserved.\r\nLicensed under the Apache License, Version 2.0 (the \"License\");\r\nyou may not use this file except in compliance with the License.\r\nYou may obtain a copy of the License at\r\n\r\n    http://www.apache.org/licenses/LICENSE-2.0\r\n\r\nUnless required by applicable law or agreed to in writing, software\r\ndistributed under the License is distributed on an \"AS IS\" BASIS,\r\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nSee the License for the specific language governing permissions and\r\nlimitations under the License.\r\n\r\nContributors:\r\n2013 Orange - port to Scalaris key/value store\r\n    ...\r\n **********************************************************************/\r\npackage org.datanucleus.store.scalaris.fieldmanager;\r\n\r\nimport java.lang.reflect.Array;\r\nimport java.math.BigDecimal;\r\nimport java.math.BigInteger;\r\nimport java.util.ArrayList;\r\nimport java.util.Collection;\r\nimport java.util.Date;\r\nimport java.util.HashMap;\r\nimport java.util.Iterator;\r\nimport java.util.Map;\r\n\r\nimport org.datanucleus.ClassLoaderResolver;\r\nimport org.datanucleus.ExecutionContext;\r\nimport org.datanucleus.exceptions.NucleusException;\r\nimport org.datanucleus.identity.IdentityUtils;\r\nimport org.datanucleus.metadata.AbstractClassMetaData;\r\nimport org.datanucleus.metadata.AbstractMemberMetaData;\r\nimport org.datanucleus.metadata.ColumnMetaData;\r\nimport org.datanucleus.metadata.JdbcType;\r\nimport org.datanucleus.metadata.RelationType;\r\nimport org.datanucleus.state.LifeCycleState;\r\nimport org.datanucleus.state.ObjectProvider;\r\nimport org.datanucleus.store.StoreManager;\r\nimport org.datanucleus.store.fieldmanager.AbstractStoreFieldManager;\r\nimport org.datanucleus.store.schema.naming.ColumnType;\r\nimport org.datanucleus.store.types.converters.TypeConverter;\r\n\r\nimport org.json.JSONException;\r\nimport org.json.JSONObject;\r\n\r\n/**\r\n * FieldManager for inserting data into the provided JSONObject from the\r\n * ObjectProvider.\r\n */\r\npublic class StoreFieldManager extends AbstractStoreFieldManager {\r\n\r\n    final JSONObject jsonobj;\r\n    final StoreManager storeMgr;\r\n\r\n    public StoreFieldManager(ObjectProvider<?> op, JSONObject jsonobj,\r\n            boolean insert) {\r\n        super(op, insert);\r\n\r\n        this.storeMgr = op.getExecutionContext().getStoreManager();\r\n        this.jsonobj = jsonobj;\r\n\r\n        try {\r\n            jsonobj.put(\"class\", cmd.getFullClassName());\r\n        } catch (JSONException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        }\r\n    }\r\n\r\n    public void storeBooleanField(int fieldNumber, boolean value) {\r\n        if (!isStorable(fieldNumber)) {\r\n            return;\r\n        }\r\n\r\n        AbstractMemberMetaData mmd = cmd\r\n                .getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber);\r\n        String name = storeMgr.getNamingFactory().getColumnName(mmd,\r\n                ColumnType.COLUMN);\r\n        try {\r\n            jsonobj.put(name, (boolean) value);\r\n        } catch (JSONException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        }\r\n    }\r\n\r\n    public void storeByteField(int fieldNumber, byte value) {\r\n        if (!isStorable(fieldNumber)) {\r\n            return;\r\n        }\r\n\r\n        AbstractMemberMetaData mmd = cmd\r\n                .getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber);\r\n        String name = storeMgr.getNamingFactory().getColumnName(mmd,\r\n                ColumnType.COLUMN);\r\n        try {\r\n            jsonobj.put(name, (int) value);\r\n        } catch (JSONException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        }\r\n    }\r\n\r\n    public void storeCharField(int fieldNumber, char value) {\r\n        if (!isStorable(fieldNumber)) {\r\n            return;\r\n        }\r\n\r\n        AbstractMemberMetaData mmd = cmd\r\n                .getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber);\r\n        String name = storeMgr.getNamingFactory().getColumnName(mmd,\r\n                ColumnType.COLUMN);\r\n        try {\r\n            jsonobj.put(name, new Character(value));\r\n        } catch (JSONException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        }\r\n    }\r\n\r\n    public void storeDoubleField(int fieldNumber, double value) {\r\n        if (!isStorable(fieldNumber)) {\r\n            return;\r\n        }\r\n\r\n        AbstractMemberMetaData mmd = cmd\r\n                .getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber);\r\n        String name = storeMgr.getNamingFactory().getColumnName(mmd,\r\n                ColumnType.COLUMN);\r\n        try {\r\n            jsonobj.put(name, (double) value);\r\n        } catch (JSONException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        }\r\n    }\r\n\r\n    public void storeFloatField(int fieldNumber, float value) {\r\n        if (!isStorable(fieldNumber)) {\r\n            return;\r\n        }\r\n\r\n        AbstractMemberMetaData mmd = cmd\r\n                .getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber);\r\n        String name = storeMgr.getNamingFactory().getColumnName(mmd,\r\n                ColumnType.COLUMN);\r\n        try {\r\n            jsonobj.put(name, (double) value);\r\n        } catch (JSONException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        }\r\n    }\r\n\r\n    public void storeIntField(int fieldNumber, int value) {\r\n        if (!isStorable(fieldNumber)) {\r\n            return;\r\n        }\r\n\r\n        AbstractMemberMetaData mmd = cmd\r\n                .getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber);\r\n        String name = storeMgr.getNamingFactory().getColumnName(mmd,\r\n                ColumnType.COLUMN);\r\n        try {\r\n            jsonobj.put(name, (int) value);\r\n        } catch (JSONException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        }\r\n    }\r\n\r\n    public void storeLongField(int fieldNumber, long value) {\r\n        if (!isStorable(fieldNumber)) {\r\n            return;\r\n        }\r\n\r\n        AbstractMemberMetaData mmd = cmd\r\n                .getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber);\r\n        String name = storeMgr.getNamingFactory().getColumnName(mmd,\r\n                ColumnType.COLUMN);\r\n        try {\r\n            jsonobj.put(name, (long) value);\r\n        } catch (JSONException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        }\r\n    }\r\n\r\n    public void storeShortField(int fieldNumber, short value) {\r\n        if (!isStorable(fieldNumber)) {\r\n            return;\r\n        }\r\n\r\n        AbstractMemberMetaData mmd = cmd\r\n                .getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber);\r\n        String name = storeMgr.getNamingFactory().getColumnName(mmd,\r\n                ColumnType.COLUMN);\r\n        try {\r\n            jsonobj.put(name, (int) value);\r\n        } catch (JSONException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        }\r\n    }\r\n\r\n    public void storeStringField(int fieldNumber, String value) {\r\n        if (!isStorable(fieldNumber)) {\r\n            return;\r\n        }\r\n\r\n        AbstractMemberMetaData mmd = cmd\r\n                .getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber);\r\n        String name = storeMgr.getNamingFactory().getColumnName(mmd,\r\n                ColumnType.COLUMN);\r\n\r\n        try {\r\n            if (value != null) {\r\n                jsonobj.put(name, value);\r\n            }\r\n        } catch (JSONException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        }\r\n    }\r\n\r\n    public void storeObjectField(int fieldNumber, Object value) {\r\n        AbstractMemberMetaData mmd = cmd\r\n                .getMetaDataForManagedMemberAtAbsolutePosition(fieldNumber);\r\n        if (!isStorable(mmd)) {\r\n            return;\r\n        }\r\n\r\n        ExecutionContext ec = op.getExecutionContext();\r\n        ClassLoaderResolver clr = ec.getClassLoaderResolver();\r\n        RelationType relationType = mmd.getRelationType(clr);\r\n        if (RelationType.isRelationSingleValued(relationType)\r\n                && mmd.isEmbedded()) {\r\n            // Persistable object embedded into this table TODO Support this\r\n            throw new NucleusException(\"Embedded fields are not supported\");\r\n        }\r\n\r\n        try {\r\n            storeObjectFieldInternal(fieldNumber, value, mmd, clr);\r\n        } catch (JSONException e) {\r\n            throw new NucleusException(e.getMessage(), e);\r\n        }\r\n    }\r\n\r\n    @SuppressWarnings({ \"rawtypes\", \"unchecked\" })\r\n    protected void storeObjectFieldInternal(int fieldNumber, Object value,\r\n            AbstractMemberMetaData mmd, ClassLoaderResolver clr)\r\n            throws JSONException {\r\n        RelationType relationType = mmd.getRelationType(clr);\r\n        String name = storeMgr.getNamingFactory().getColumnName(mmd,\r\n                ColumnType.COLUMN);\r\n\r\n        if (value == null) {\r\n            jsonobj.put(name, JSONObject.NULL);\r\n            return;\r\n        } else if (relationType == RelationType.NONE) {\r\n            if (mmd.getTypeConverterName() != null) {\r\n                // User-defined converter\r\n                TypeConverter conv = op.getExecutionContext().getTypeManager()\r\n                        .getTypeConverterForName(mmd.getTypeConverterName());\r\n                jsonobj.put(name, conv.toDatastoreType(value));\r\n                return;\r\n            } else if (value instanceof Boolean) {\r\n                jsonobj.put(name, ((Boolean) value).booleanValue());\r\n            } else if (value instanceof Integer) {\r\n                jsonobj.put(name, ((Integer) value).intValue());\r\n            } else if (value instanceof Long) {\r\n                jsonobj.put(name, ((Long) value).longValue());\r\n            } else if (value instanceof Double) {\r\n                jsonobj.put(name, ((Double) value).doubleValue());\r\n            } else if (value instanceof Date) {\r\n                Date dateValue = (Date) value;\r\n                jsonobj.put(name, dateValue.getTime());\r\n            } else if (value instanceof Enum) {\r\n                jsonobj.put(name, ((Enum) value).ordinal());\r\n            } else if (value instanceof BigDecimal) {\r\n                jsonobj.put(name, value);\r\n            } else if (value instanceof BigInteger) {\r\n                jsonobj.put(name, value);\r\n            } else if (value instanceof Collection) {\r\n                // Collection<Non-PC> will be returned as JSONArray\r\n                jsonobj.put(name, value);\r\n            } else if (value instanceof Map) {\r\n                jsonobj.put(name, value);\r\n            } else {\r\n                // See if we can persist it as a Long/String\r\n                boolean useLong = false;\r\n                ColumnMetaData[] colmds = mmd.getColumnMetaData();\r\n                if (colmds != null && colmds.length == 1) {\r\n                    JdbcType jdbcType = colmds[0].getJdbcType();\r\n                    if (jdbcType != null) {\r\n                        String jdbc = jdbcType.name();\r\n                        if (jdbc != null\r\n                                && (jdbc.equalsIgnoreCase(\"INTEGER\") || jdbc\r\n                                        .equalsIgnoreCase(\"NUMERIC\"))) {\r\n                            useLong = true;\r\n                        }\r\n                    }\r\n                }\r\n                TypeConverter strConv = op.getExecutionContext()\r\n                        .getNucleusContext().getTypeManager()\r\n                        .getTypeConverterForType(mmd.getType(), String.class);\r\n                TypeConverter longConv = op.getExecutionContext()\r\n                        .getNucleusContext().getTypeManager()\r\n                        .getTypeConverterForType(mmd.getType(), Long.class);\r\n                if (useLong) {\r\n                    if (longConv != null) {\r\n                        jsonobj.put(name, longConv.toDatastoreType(value));\r\n                        return;\r\n                    }\r\n                } else {\r\n                    if (strConv != null) {\r\n                        jsonobj.put(name, strConv.toDatastoreType(value));\r\n                        return;\r\n                    } else if (longConv != null) {\r\n                        jsonobj.put(name, longConv.toDatastoreType(value));\r\n                        return;\r\n                    }\r\n                }\r\n\r\n                // Fallback to persist as a JSONObject and see what happens\r\n                JSONObject jsonobjfield = new JSONObject(value);\r\n                jsonobjfield.put(\"class\", value.getClass().getName());\r\n                jsonobj.put(name, jsonobjfield);\r\n            }\r\n\r\n            return;\r\n        } else if (RelationType.isRelationSingleValued(relationType)) {\r\n            // 1-1, N-1 relation, so store the \"id\"\r\n            final Object valuePC = op.getExecutionContext()\r\n                    .persistObjectInternal(value, op, fieldNumber, -1);\r\n            final Object valueId = op.getExecutionContext().getApiAdapter()\r\n                    .getIdForObject(valuePC);\r\n            if (valueId != null) {\r\n                jsonobj.put(name,\r\n                        IdentityUtils.getPersistableIdentityForId(valueId));\r\n\r\n                ObjectProvider<?> intern = ec.findObjectProvider(valuePC);\r\n                if (intern.getLifecycleState().stateType() == LifeCycleState.P_NEW) {\r\n                    // Loaded flags must be cleared for the internal object in case\r\n                    // it was only persisted indirectly via PersistenceManager.\r\n                    // This prevents registering updates on the objects if it is\r\n                    // not fetched once before.\r\n                    intern.clearLoadedFlags();\r\n                }\r\n            }\r\n            return;\r\n        } else if (RelationType.isRelationMultiValued(relationType)) {\r\n            // Collection/Map/Array\r\n            if (mmd.hasCollection()) {\r\n                final Collection<String> idColl = new ArrayList<String>();\r\n                final Collection<?> coll = (Collection<?>) value;\r\n                final Iterator<?> collIter = coll.iterator();\r\n                while (collIter.hasNext()) {\r\n                    final Object element = collIter.next();\r\n                    final Object elementPC = op\r\n                            .getExecutionContext()\r\n                            .persistObjectInternal(element, op, fieldNumber, -1);\r\n                    final Object elementID = op.getExecutionContext()\r\n                            .getApiAdapter().getIdForObject(elementPC);\r\n                    \r\n                    String objId = IdentityUtils.getPersistableIdentityForId(elementID);\r\n                    if (objId != null) {\r\n                        // object with this ID was already deleted\r\n                        idColl.add(objId);\r\n                    }\r\n                }\r\n\r\n                jsonobj.put(name, idColl);\r\n                return;\r\n            } else if (mmd.hasArray()) {\r\n                Collection ids = new ArrayList(Array.getLength(value));\r\n                for (int i = 0; i < Array.getLength(value); i++) {\r\n                    final Object element = Array.get(value, i);\r\n                    final Object elementPC = op\r\n                            .getExecutionContext()\r\n                            .persistObjectInternal(element, op, fieldNumber, -1);\r\n                    final Object elementID = op.getExecutionContext()\r\n                            .getApiAdapter().getIdForObject(elementPC);\r\n                    \r\n                    String objId = IdentityUtils.getPersistableIdentityForId(elementID);\r\n                    if (objId != null) {\r\n                        // object with this ID was already deleted\r\n                        ids.add(objId);\r\n                    }\r\n                }\r\n                jsonobj.put(name, ids);\r\n                return;\r\n            } else if (mmd.hasMap()) {\r\n                AbstractClassMetaData keyCmd = mmd.getMap()\r\n                        .getKeyClassMetaData(clr,\r\n                                op.getExecutionContext().getMetaDataManager());\r\n                AbstractClassMetaData valCmd = mmd.getMap()\r\n                        .getValueClassMetaData(clr,\r\n                                op.getExecutionContext().getMetaDataManager());\r\n\r\n                Map idMap = new HashMap();\r\n                Map map = (Map) value;\r\n                Iterator<Map.Entry> mapIter = map.entrySet().iterator();\r\n                while (mapIter.hasNext()) {\r\n                    Map.Entry entry = mapIter.next();\r\n                    Object key = null;\r\n                    Object val = null;\r\n                    if (keyCmd != null) {\r\n                        Object keyPC = op.getExecutionContext()\r\n                                .persistObjectInternal(entry.getKey(), op,\r\n                                        fieldNumber, -1);\r\n                        key = op.getExecutionContext().getApiAdapter()\r\n                                .getIdForObject(keyPC);\r\n                    } else {\r\n                        key = entry.getKey();\r\n                    }\r\n                    if (valCmd != null) {\r\n                        Object valPC = op.getExecutionContext()\r\n                                .persistObjectInternal(entry.getValue(), op,\r\n                                        fieldNumber, -1);\r\n                        val = op.getExecutionContext().getApiAdapter()\r\n                                .getIdForObject(valPC);\r\n                    } else {\r\n                        val = entry.getValue();\r\n                    }\r\n                    idMap.put(key, val);\r\n                }\r\n                jsonobj.put(name, idMap);\r\n                return;\r\n            }\r\n        }\r\n\r\n        throw new NucleusException(\"Dont currently support field \"\r\n                + mmd.getFullFieldName() + \" of type \" + mmd.getTypeName());\r\n    }\r\n}\r\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/src/java/org/datanucleus/store/scalaris/query/JDOQLQuery.java",
    "content": "/**********************************************************************\r\nCopyright (c) 2008 Erik Bengtson and others. All rights reserved.\r\nLicensed under the Apache License, Version 2.0 (the \"License\");\r\nyou may not use this file except in compliance with the License.\r\nYou may obtain a copy of the License at\r\n\r\n    http://www.apache.org/licenses/LICENSE-2.0\r\n\r\nUnless required by applicable law or agreed to in writing, software\r\ndistributed under the License is distributed on an \"AS IS\" BASIS,\r\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nSee the License for the specific language governing permissions and\r\nlimitations under the License.\r\n\r\nContributors :\r\n2008 Andy Jefferson - refactored JSON specific code to JSONUtils\r\n2008 Andy Jefferson - compilation process\r\n2013 Orange - port to Scalaris key/value store\r\n    ...\r\n ***********************************************************************/\r\npackage org.datanucleus.store.scalaris.query;\r\n\r\nimport java.util.ArrayList;\r\nimport java.util.Collection;\r\nimport java.util.Map;\r\n\r\nimport org.datanucleus.ExecutionContext;\r\nimport org.datanucleus.metadata.AbstractClassMetaData;\r\nimport org.datanucleus.metadata.AbstractMemberMetaData;\r\nimport org.datanucleus.query.evaluator.JavaQueryEvaluator;\r\nimport org.datanucleus.query.expression.DyadicExpression;\r\nimport org.datanucleus.query.expression.Expression;\r\nimport org.datanucleus.query.expression.Expression.Operator;\r\nimport org.datanucleus.query.expression.ParameterExpression;\r\nimport org.datanucleus.query.expression.PrimaryExpression;\r\nimport org.datanucleus.store.StoreManager;\r\nimport org.datanucleus.store.query.AbstractJDOQLQuery;\r\nimport org.datanucleus.store.scalaris.ScalarisPersistenceHandler;\r\n\r\n/**\r\n * JDOQL query for scalaris datastores.\r\n */\r\npublic class JDOQLQuery extends AbstractJDOQLQuery {\r\n    /**\r\n     * Default serial version....\r\n     */\r\n    private static final long serialVersionUID = 1L;\r\n\r\n    /**\r\n     * Constructs a new query instance that uses the given persistence manager.\r\n     * \r\n     * @param storeMgr\r\n     *            StoreManager for this query\r\n     * @param om\r\n     *            the associated ExecutionContext for this query.\r\n     */\r\n    public JDOQLQuery(StoreManager storeMgr, ExecutionContext om) {\r\n        this(storeMgr, om, (JDOQLQuery) null);\r\n    }\r\n\r\n    /**\r\n     * Constructs a new query instance having the same criteria as the given\r\n     * query.\r\n     * \r\n     * @param storeMgr\r\n     *            StoreManager for this query\r\n     * @param om\r\n     *            The ExecutionContext\r\n     * @param q\r\n     *            The query from which to copy criteria.\r\n     */\r\n    public JDOQLQuery(StoreManager storeMgr, ExecutionContext om, JDOQLQuery q) {\r\n        super(storeMgr, om, q);\r\n    }\r\n\r\n    /**\r\n     * Constructor for a JDOQL query where the query is specified using the\r\n     * \"Single-String\" format.\r\n     * \r\n     * @param storeMgr\r\n     *            StoreManager for this query\r\n     * @param om\r\n     *            The persistence manager\r\n     * @param query\r\n     *            The query string\r\n     */\r\n    public JDOQLQuery(StoreManager storeMgr, ExecutionContext om, String query) {\r\n        super(storeMgr, om, query);\r\n    }\r\n\r\n    @SuppressWarnings({ \"rawtypes\", \"unchecked\" })\r\n    protected Object performExecute(Map parameters) {\r\n        AbstractClassMetaData cmd = ec.getMetaDataManager()\r\n                .getMetaDataForClass(candidateClass,\r\n                        ec.getClassLoaderResolver());\r\n\r\n        // get all stored instances of class candidateClass\r\n        Collection candidates;\r\n        if (candidateCollection == null) {\r\n\r\n            Expression filterExpr = compilation.getExprFilter();\r\n            String[] memberNameValuePair = getUniqueMemberNameValuePairIfExist(cmd, filterExpr, parameters);\r\n            if (memberNameValuePair == null) {\r\n                // it is not possible to pre-select which objects are needed, therefore\r\n                // all objects must be fetched first\r\n                candidates = ((ScalarisPersistenceHandler) ec.getStoreManager().getPersistenceHandler())\r\n                        .getObjectsOfCandidateType(ec, candidateClass, cmd);\r\n            } else {\r\n                // the member is marked by 'unique' annotation, which\r\n                // means that there can only be at most one object fitting the criteria\r\n                candidates = new ArrayList(1);\r\n                Object uniqueObject = ((ScalarisPersistenceHandler) ec.getStoreManager().getPersistenceHandler())\r\n                        .getObjectByUniqueMember(ec, candidateClass, memberNameValuePair[0], memberNameValuePair[1]);\r\n                if (uniqueObject != null) {\r\n                    candidates.add(uniqueObject);\r\n                }\r\n            }\r\n        } else {\r\n            candidates = new ArrayList<Object>(candidateCollection);\r\n        }\r\n\r\n        // execute query\r\n        JavaQueryEvaluator resultMapper = new ScalarisJDOQLEvaluator(this,\r\n                candidateClass, candidates, compilation, parameters,\r\n                ec.getClassLoaderResolver(), ec);\r\n        Collection result = resultMapper.execute(true, true, true, true, true);\r\n\r\n        return result;\r\n    }\r\n\r\n    /**\r\n     * Checks if the filter expression of the query contains a test for equality of a unique\r\n     * member (marked by 'Unique' annotation) of the candidate class. If this is the case\r\n     * returns the member name and tested value as String array. If there is no such test\r\n     * null is returned.\r\n     * @param candidateClassMD ClassMetaData of the class which is checked for the unique member\r\n     * @param filterExpr The filter expression used in the query (WHERE clause)\r\n     * @param parameters The parameters passed to the query\r\n     * @return new String{memberName, testedMemberValue} if there is such a test, null otherwise\r\n     */\r\n    private String[] getUniqueMemberNameValuePairIfExist(AbstractClassMetaData candidateClassMD,\r\n            Expression filterExpr, Map<?,?> parameters) {\r\n        if (filterExpr == null || !(filterExpr instanceof DyadicExpression)) {\r\n            return null;\r\n        }\r\n\r\n        Operator op = filterExpr.getOperator();\r\n        if (op.equals(Expression.OP_AND)) {\r\n            // composite of two filters, check left side first\r\n            String[] leftResult =\r\n                    getUniqueMemberNameValuePairIfExist(candidateClassMD,\r\n                            filterExpr.getLeft(), parameters);\r\n            return leftResult != null? leftResult :\r\n                    getUniqueMemberNameValuePairIfExist(candidateClassMD,\r\n                            filterExpr.getRight(), parameters);\r\n\r\n        } else if (op.equals(Expression.OP_EQ)) {\r\n            PrimaryExpression priExpr;\r\n            ParameterExpression paramExpr;\r\n            if (filterExpr.getLeft() instanceof PrimaryExpression \r\n                    && filterExpr.getRight() instanceof ParameterExpression) {\r\n                priExpr = (PrimaryExpression) filterExpr.getLeft();\r\n                paramExpr = (ParameterExpression) filterExpr.getRight();\r\n\r\n            } else if (filterExpr.getRight() instanceof PrimaryExpression\r\n                    && filterExpr.getLeft() instanceof ParameterExpression) {\r\n                priExpr = (PrimaryExpression) filterExpr.getRight();\r\n                paramExpr = (ParameterExpression) filterExpr.getLeft();\r\n\r\n            } else {\r\n                return null;\r\n            }\r\n\r\n            if (priExpr.getSymbol() != null) {\r\n                String memberFilteredOn = priExpr.getSymbol().getQualifiedName();\r\n                AbstractMemberMetaData mmd = candidateClassMD.getMetaDataForMember(memberFilteredOn);\r\n                if (mmd != null && mmd.getUniqueMetaData() != null) {\r\n\r\n                    Object filterValue = parameters.get(paramExpr.getPosition());\r\n                    if (filterValue != null) {\r\n                        return new String[]{memberFilteredOn, filterValue.toString()};\r\n                    }\r\n                }\r\n            }\r\n        }\r\n\r\n        return null;\r\n    }\r\n}\r\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/src/java/org/datanucleus/store/scalaris/query/JPQLQuery.java",
    "content": "/**********************************************************************\r\nCopyright (c) 2008 Andy Jefferson and others. All rights reserved.\r\nLicensed under the Apache License, Version 2.0 (the \"License\");\r\nyou may not use this file except in compliance with the License.\r\nYou may obtain a copy of the License at\r\n\r\n    http://www.apache.org/licenses/LICENSE-2.0\r\n\r\nUnless required by applicable law or agreed to in writing, software\r\ndistributed under the License is distributed on an \"AS IS\" BASIS,\r\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nSee the License for the specific language governing permissions and\r\nlimitations under the License.\r\n\r\nContributors :\r\n2013 Orange - port to Scalaris key/value store\r\n    ...\r\n ***********************************************************************/\r\npackage org.datanucleus.store.scalaris.query;\r\n\r\nimport java.util.Map;\r\n\r\nimport org.datanucleus.ExecutionContext;\r\nimport org.datanucleus.exceptions.NucleusException;\r\nimport org.datanucleus.store.StoreManager;\r\nimport org.datanucleus.store.query.AbstractJPQLQuery;\r\n\r\n/**\r\n * JPQL query for JSON datastores.\r\n */\r\npublic class JPQLQuery extends AbstractJPQLQuery {\r\n    /**\r\n     * Default serial version....\r\n     */\r\n    private static final long serialVersionUID = 1L;\r\n\r\n    /**\r\n     * Constructs a new query instance that uses the given persistence manager.\r\n     * \r\n     * @param storeMgr\r\n     *            StoreManager for this query\r\n     * @param ec\r\n     *            the associated ExecutionContext for this query.\r\n     */\r\n    public JPQLQuery(StoreManager storeMgr, ExecutionContext ec) {\r\n        this(storeMgr, ec, (JPQLQuery) null);\r\n    }\r\n\r\n    /**\r\n     * Constructs a new query instance having the same criteria as the given\r\n     * query.\r\n     * \r\n     * @param storeMgr\r\n     *            StoreManager for this query\r\n     * @param ec\r\n     *            The ExecutionContext\r\n     * @param q\r\n     *            The query from which to copy criteria.\r\n     */\r\n    public JPQLQuery(StoreManager storeMgr, ExecutionContext ec, JPQLQuery q) {\r\n        super(storeMgr, ec, q);\r\n    }\r\n\r\n    /**\r\n     * Constructor for a JPQL query where the query is specified using the\r\n     * \"Single-String\" format.\r\n     * \r\n     * @param storeMgr\r\n     *            StoreManager for this query\r\n     * @param ec\r\n     *            The persistence manager\r\n     * @param query\r\n     *            The query string\r\n     */\r\n    public JPQLQuery(StoreManager storeMgr, ExecutionContext ec, String query) {\r\n        super(storeMgr, ec, query);\r\n    }\r\n\r\n    protected Object performExecute(@SuppressWarnings(\"rawtypes\") Map parameters) {\r\n\r\n        throw new NucleusException(\"Don't currently support JPQL\");\r\n    }\r\n}\r\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/src/java/org/datanucleus/store/scalaris/query/ScalarisJDOQLEvaluator.java",
    "content": "package org.datanucleus.store.scalaris.query;\n\nimport java.util.ArrayList;\nimport java.util.Collection;\nimport java.util.Iterator;\nimport java.util.List;\nimport java.util.Map;\n\nimport org.datanucleus.ClassLoaderResolver;\nimport org.datanucleus.ExecutionContext;\nimport org.datanucleus.exceptions.NucleusUserException;\nimport org.datanucleus.metadata.AbstractClassMetaData;\nimport org.datanucleus.query.QueryUtils;\nimport org.datanucleus.query.compiler.QueryCompilation;\nimport org.datanucleus.query.evaluator.JDOQLEvaluator;\nimport org.datanucleus.query.expression.CreatorExpression;\nimport org.datanucleus.query.expression.Expression;\nimport org.datanucleus.query.inmemory.InMemoryExpressionEvaluator;\nimport org.datanucleus.query.inmemory.InMemoryFailure;\nimport org.datanucleus.query.inmemory.VariableNotSetException;\nimport org.datanucleus.store.query.Query;\nimport org.datanucleus.store.query.Query.SubqueryDefinition;\nimport org.datanucleus.store.scalaris.ScalarisPersistenceHandler;\nimport org.datanucleus.util.Localiser;\nimport org.datanucleus.util.NucleusLogger;\nimport org.datanucleus.util.StringUtils;\n\n@SuppressWarnings({\"rawtypes\", \"unchecked\"})\npublic class ScalarisJDOQLEvaluator extends JDOQLEvaluator {\n\n    private Class candidateClass;\n    private ExecutionContext ec;\n\n    public ScalarisJDOQLEvaluator(Query query, Class candidateClass, Collection<?> candidates,\n            QueryCompilation compilation, Map parameterValues,\n            ClassLoaderResolver clr, ExecutionContext ec) {\n        super(query, candidates, compilation, parameterValues, clr);\n        this.candidateClass = candidateClass;\n        this.ec = ec;\n    }\n\n    @Override\n    public Collection execute(boolean applyFilter, boolean applyOrdering, boolean applyResult, boolean applyResultClass, boolean applyRange) {\n        // execute subqueries\n        String[] subqueryAliases = compilation.getSubqueryAliases();\n        if (subqueryAliases != null) {\n            for (int i=0;i<subqueryAliases.length;i++) {\n                // Evaluate subquery first\n                Query<?> subquery = query.getSubqueryForVariable(subqueryAliases[i]).getQuery();\n                QueryCompilation subqueryCompilation =\n                    compilation.getCompilationForSubquery(subqueryAliases[i]);\n                if (subqueryCompilation.getExprFrom() != null) {\n                    // TODO Evaluate \"from\"\n                    NucleusLogger.QUERY.warn(\"In-memory evaluation of subquery with 'from'=\" + \n                        StringUtils.objectArrayToString(subqueryCompilation.getExprFrom()) +\n                        \" but from clause evaluation not currently supported!\");\n                }\n                Collection<?> subqueryResult = evaluateSubquery(subquery, subqueryCompilation, candidates, null);\n\n                if (QueryUtils.queryReturnsSingleRow(subquery)) {\n                    // Subquery is expected to return single value\n                    state.put(subqueryAliases[i], subqueryResult.iterator().next());\n                } else {\n                    state.put(subqueryAliases[i], subqueryResult);\n                }\n            }\n        }\n\n        // apply filter\n        List resultList = new ArrayList(candidates);\n        Expression filter = compilation.getExprFilter();\n        if (applyFilter && filter != null) {\n            // super.handleFilter throws an VariableNotSetException when working with sub-queries\n            candidates = handleFilter(resultList);\n        }\n        Collection queryResult = super.execute(false, applyOrdering, applyResult, false, applyRange);\n\n        if (applyResultClass) {\n            // apply a custom ResultClassMapper because the class-mapper used by \n            // DataNucleus does not support alias' when mapping \n            Expression[] expResult = compilation.getExprResult();\n            if (expResult != null && query.getResultClass() != null && !(expResult[0] instanceof CreatorExpression)){\n                return mapResultClass(queryResult, expResult);\n            }\n        }\n        return queryResult;\n    }\n\n    @Override\n    protected Collection evaluateSubquery(Query subquery, QueryCompilation compilation, Collection candidates,\n            Object outerCandidate){\n        // check if this sub-query was already executed\n        String[] subqueryAliases = compilation.getSubqueryAliases();\n        if (subqueryAliases != null) {\n            for (String subqueryAlias : subqueryAliases) {\n                SubqueryDefinition sqd = query.getSubqueryForVariable(subqueryAlias);\n                Query<?> tmpSubquery = (sqd != null) ? sqd.getQuery() : subquery;\n                if (tmpSubquery.equals(subquery)) {\n                    if (state.containsKey(subqueryAlias)) {\n                        return (Collection) state.get(subqueryAlias);\n                    }\n                }\n            }\n        }\n\n        if (!subquery.getCandidateClass().equals(candidateClass)) {\n            // if the sub-query queries over a different candidate class, all objects of this\n            // class must be fetched beforehand\n            AbstractClassMetaData cmd = ec.getMetaDataManager()\n                    .getMetaDataForClass(subquery.getCandidateClass(),ec.getClassLoaderResolver());\n            candidates = ((ScalarisPersistenceHandler) ec.getStoreManager().getPersistenceHandler())\n                    .getObjectsOfCandidateType(ec, subquery.getCandidateClass(), cmd);\n        }\n        return super.evaluateSubquery(subquery, compilation, candidates, outerCandidate);\n    }\n\n    /*\n     * @see org.datanucleus.query.evaluator.JavaQueryEvaluator#handleFilter\n     * This is virtually the same method as JavaQueryEvaluator#handleFilter, but it\n     * sets the variables of the used InMemoryExpressionEvaluator as needed. This is necessary to\n     * prevent an NullPointerException when executing a query with sub-queries\n     */\n    private List handleFilter(List set) {\n        Expression filter = compilation.getExprFilter();\n        if (filter == null) {\n            return set;\n        }\n\n        // Store current results in case we have an aggregate in the filter\n        state.put(RESULTS_SET, set);\n\n        List result = new ArrayList();\n        Iterator it = set.iterator();\n        if (NucleusLogger.QUERY.isDebugEnabled()) {\n            NucleusLogger.QUERY.debug(\"Evaluating filter for \" + set.size() + \" candidates\");\n        }\n\n        while (it.hasNext()) {\n            // Set the value of the candidate being tested, and evaluate it\n            Object obj = it.next();\n            if (!state.containsKey(candidateAlias)) {\n                throw new NucleusUserException(\"Alias \\\"\" + candidateAlias + \"\\\" doesn't exist in the query or the candidate alias wasn't defined\");\n            }\n            state.put(candidateAlias, obj);\n\n            InMemoryExpressionEvaluator eval = new InMemoryExpressionEvaluator(query.getExecutionContext(), \n                    parameterValues, state, query.getParsedImports(), clr, candidateAlias, query.getLanguage());\n\n            for (String stateKey : state.keySet()) {\n                eval.setVariableValue(stateKey, state.get(stateKey));\n            }\n\n            Object evalResult = evaluateBooleanExpression(filter, eval);\n            if (Boolean.TRUE.equals(evalResult)) {\n                if (NucleusLogger.QUERY.isDebugEnabled()) {\n                    NucleusLogger.QUERY.debug(Localiser.msg(\"021023\", StringUtils.toJVMIDString(obj)));\n                }\n                result.add(obj);\n            }\n        }\n        return result;\n    }\n\n    /* \n     * @see org.datanucleus.query.evaluator.JavaQueryEvaluator#evaluateBooleanExpression\n     * This is an exact copy JavaQueryEvaluator#evaluateBooleanExpression, since it is private\n     * but it is needed here because of the changed implementation of handleFilter. (This is ugly)\n     */\n    private Boolean evaluateBooleanExpression(Expression expr, InMemoryExpressionEvaluator eval) {\n        try {\n            Object result = expr.evaluate(eval);\n            return ((result instanceof InMemoryFailure) ? Boolean.FALSE : (Boolean)result);\n        }\n        catch (VariableNotSetException vnse) {\n            if (NucleusLogger.QUERY.isDebugEnabled()) {\n                NucleusLogger.QUERY.debug(Localiser.msg(\"021024\", vnse.getVariableExpression().getId(), \n                        StringUtils.objectArrayToString(vnse.getValues())));\n            }\n\n            if (vnse.getValues() == null || vnse.getValues().length == 0) {\n                // No values available for this variable, so the result is interpreted as false\n                if (NucleusLogger.QUERY.isDebugEnabled()) {\n                    NucleusLogger.QUERY.debug(Localiser.msg(\"021025\", vnse.getVariableExpression().getId(), \"(null)\"));\n                }\n                return Boolean.FALSE;\n            } else {\n                // Set this variable and start iteration over the possible variable values\n                for (int i=0;i<vnse.getValues().length;i++) {\n                    eval.setVariableValue(vnse.getVariableExpression().getId(), vnse.getValues()[i]);\n                    if (NucleusLogger.QUERY.isDebugEnabled()) {\n                        NucleusLogger.QUERY.debug(Localiser.msg(\"021025\", vnse.getVariableExpression().getId(), vnse.getValues()[i]));\n                    }\n                    if (Boolean.TRUE.equals(evaluateBooleanExpression(expr, eval))) {\n                        return Boolean.TRUE;\n                    }\n                }\n            }\n\n            // No variable value was successful so return FALSE\n            if (NucleusLogger.QUERY.isDebugEnabled()) {\n                NucleusLogger.QUERY.debug(Localiser.msg(\"021026\", vnse.getVariableExpression().getId()));\n            }\n            eval.removeVariableValue(vnse.getVariableExpression().getId());\n            return Boolean.FALSE;\n        }\n    }\n\n    /**\n     * Constructs ResultClassMapper and calls its map function\n     * @param resultSet The resultSet containing the instances handled by setResult\n     * @return The resultSet containing instances of the Class defined by setResultClass\n     */\n    Collection<?> mapResultClass(Collection<?> result, Expression[] expResult) {\n        return new ScalarisJDOQLResultClassMapper(query.getResultClass()).map(result, expResult);\n    }\n}"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/src/java/org/datanucleus/store/scalaris/query/ScalarisJDOQLResultClassMapper.java",
    "content": "/**********************************************************************\nCopyright (c) 2007 Marcel Wirth and others. All rights reserved.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\nContributors:\n2008 Andy Jefferson - reworked to make extensive reuse of QueryUtils\n2015 Jan Skrzypczak - support alias\n    ...\n **********************************************************************/\n\npackage org.datanucleus.store.scalaris.query;\n\nimport java.lang.reflect.Field;\nimport java.security.AccessController;\nimport java.security.PrivilegedAction;\nimport java.util.ArrayList;\nimport java.util.Collection;\nimport java.util.Iterator;\nimport java.util.List;\n\nimport org.datanucleus.exceptions.NucleusUserException;\nimport org.datanucleus.query.QueryUtils;\nimport org.datanucleus.query.expression.Expression;\nimport org.datanucleus.query.expression.ParameterExpression;\nimport org.datanucleus.query.expression.PrimaryExpression;\nimport org.datanucleus.util.Localiser;\nimport org.datanucleus.util.NucleusLogger;\nimport org.datanucleus.util.StringUtils;\n\n\n/*\n * This class is heavily based on \n * org.datanucleus.query.evaluator.AbstractResultClassMapper with only\n * some minor changes to support alias\n */\n@SuppressWarnings(\"rawtypes\")\npublic class ScalarisJDOQLResultClassMapper {\n\n    protected Class resultClass;\n    \n    public ScalarisJDOQLResultClassMapper(Class resultClass) {\n        this.resultClass = resultClass;\n    }\n\n    /**\n     * Method to map the input results to the required result class type.\n     * @param inputResults The results to process\n     * @param resultNames Expressions for the result components of the input results (columns)\n     * @return Collection&lt;resultClass&gt;\n     */\n    @SuppressWarnings(\"unchecked\")    \n    public Collection map(final Collection inputResults, final Expression[] resultNames)\n    {\n        // Do as PrivilegedAction since can use reflection\n        return (Collection) AccessController.doPrivileged(new PrivilegedAction()\n        {\n            public Object run()\n            {\n                String[] fieldNames = new String[resultNames.length];\n                Field[] fields = new Field[fieldNames.length];\n                for (int i=0;i<fieldNames.length;i++)\n                {\n                    if (resultNames[i] instanceof PrimaryExpression)\n                    {\n                        fieldNames[i] = ((PrimaryExpression)resultNames[i]).getId();\n                        if (fieldNames[i].indexOf('.') > 0)\n                        {\n                            // Just take last part of name (for when the user specifies something like \"p.firstName\")\n                            int pos = fieldNames[i].lastIndexOf('.');\n                            fieldNames[i] = fieldNames[i].substring(pos+1);\n                        }\n                        String alias = ((PrimaryExpression)resultNames[i]).getAlias();\n                        if (alias != null) {\n                            if (fieldNames[i].indexOf('.') > 0) {\n                                int pos = alias.lastIndexOf('.');\n                                fieldNames[i] = alias.substring(pos+1);\n                            } else {\n                                fieldNames[i] = alias;\n                            }\n                        }\n                      \n                        fields[i] = getFieldForFieldNameInResultClass(resultClass, fieldNames[i]);\n                    }\n                    else if (resultNames[i] instanceof ParameterExpression)\n                    {\n                        // TODO We need to cater for outputting the parameter value. Need SymbolTable\n                        // This code below is wrong.\n                        fieldNames[i] = ((ParameterExpression)resultNames[i]).getId();\n                        String alias = ((ParameterExpression)resultNames[i]).getAlias();\n                        if (alias == null) {\n                            fieldNames[i] = alias;\n                        }\n                        fields[i] = getFieldForFieldNameInResultClass(resultClass, fieldNames[i]);\n                    }\n                    else\n                    {\n                        fieldNames[i] = resultNames[i].getAlias();\n                        fields[i] = null;\n                    }\n                }\n    \n                List outputResults = new ArrayList();\n                Iterator it = inputResults.iterator();\n                while (it.hasNext())\n                {\n                    Object inputResult = it.next();\n                    Object row = getResultForResultSetRow(inputResult, fieldNames, fields);\n                    outputResults.add(row);\n                }\n                return outputResults;\n            }\n        });\n    }\n\n    /**\n     * Method to take the result(s) of a row of the query and convert it into an object of the resultClass\n     * type, using the rules from the JDO spec.\n     * @param inputResult The result from the query\n     * @param fieldNames Names of the fields (in the query, ordered)\n     * @param fields The Field objects for the fields of the result class (ordered)\n     * @return Object of the resultClass type for the input result\n     */\n    @SuppressWarnings(\"unchecked\")\n    Object getResultForResultSetRow(Object inputResult, String[] fieldNames, Field[] fields)\n    {\n        if (resultClass == Object[].class)\n        {\n            return inputResult;\n        }\n        else if (QueryUtils.resultClassIsSimple(resultClass.getName()))\n        {\n            // User wants a single field\n            if (fieldNames.length == 1)\n            {\n                if (inputResult == null || resultClass.isAssignableFrom(inputResult.getClass()))\n                {\n                    return inputResult;\n                }\n\n                String msg = Localiser.msg(\"021202\", resultClass.getName(), inputResult.getClass().getName());\n                NucleusLogger.QUERY.error(msg);\n                throw new NucleusUserException(msg);\n            }\n            else if (fieldNames.length > 1)\n            {\n                String msg = Localiser.msg(\"021201\", resultClass.getName());\n                NucleusLogger.QUERY.error(msg);\n                throw new NucleusUserException(msg);\n            }\n            else\n            {\n                // 0 columns in the query ?\n                return null;\n            }\n        }\n        else if (fieldNames.length == 1 && resultClass.isAssignableFrom(inputResult.getClass()))\n        {\n            // Only 1 column, and the input result is of the type of the result class, so return it\n            return inputResult;\n        }\n        else\n        {\n            Object[] fieldValues = null;\n            if (inputResult instanceof Object[])\n            {\n                fieldValues = (Object[])inputResult;\n            }\n            else\n            {\n                fieldValues = new Object[1];\n                fieldValues[0] = inputResult;\n            }\n            Object obj = QueryUtils.createResultObjectUsingArgumentedConstructor(resultClass, fieldValues, null);\n            if (obj != null)\n            {\n                return obj;\n            }\n            else if (NucleusLogger.QUERY.isDebugEnabled())\n            {\n                // Give debug message that no constructor was found with the right args\n                Class[] ctr_arg_types = new Class[fieldNames.length];\n                for (int i=0;i<fieldNames.length;i++)\n                {\n                    if (fieldValues[i] != null)\n                    {\n                        ctr_arg_types[i] = fieldValues[i].getClass();\n                    }\n                    else\n                    {\n                        ctr_arg_types[i] = null;\n                    }\n                }\n                NucleusLogger.QUERY.debug(Localiser.msg(\"021206\",\n                    resultClass.getName(), StringUtils.objectArrayToString(ctr_arg_types)));\n            }\n    \n            // B. No argumented constructor so create object and update fields using fields/put()/setXXX()\n            return QueryUtils.createResultObjectUsingDefaultConstructorAndSetters(resultClass, fieldNames, \n                fields, fieldValues);\n        }\n    }\n\n    /**\n     * Accessor for the Field for the specified field name of the supplied class.\n     * Caters for the field being in superclasses.\n     * @param cls The class\n     * @param fieldName Name of the field\n     * @return The field\n     */\n    Field getFieldForFieldNameInResultClass(Class cls, String fieldName)\n    {\n        try\n        {\n            return cls.getDeclaredField(fieldName);\n        }\n        catch (NoSuchFieldException nsfe)\n        {\n            if (cls.getSuperclass() != null)\n            {\n                return getFieldForFieldNameInResultClass(cls.getSuperclass(), fieldName);\n            }\n        }\n        return null;\n    }\n}"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store/start-scalaris.sh",
    "content": "#!/bin/bash\n\ncd $(dirname $0)\n\nmkdir -p target\ncd target \n\nif [ ! -f scalaris-0.6.0.tar.gz ]\nthen\n\twget http://scalaris.googlecode.com/files/scalaris-0.6.0.tar.gz\nfi\n\nif [ ! -d  scalaris-0.6.0 ]\nthen\n\ttar xvfz  scalaris-0.6.0.tar.gz\nfi\n\ncd scalaris-0.6.0\n\nif [ ! -f bin/scalarisctl ]\nthen\n\tsudo apt-get install erlang make\n\t./configure\n\tmake\nfi\t\n\n./bin/scalarisctl -m -s -f start\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/.classpath",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<classpath>\n\t<classpathentry kind=\"src\" output=\"target/classes\" path=\"src/main/java\">\n\t\t<attributes>\n\t\t\t<attribute name=\"optional\" value=\"true\"/>\n\t\t\t<attribute name=\"maven.pomderived\" value=\"true\"/>\n\t\t</attributes>\n\t</classpathentry>\n\t<classpathentry excluding=\"**\" kind=\"src\" output=\"target/classes\" path=\"src/main/resources\">\n\t\t<attributes>\n\t\t\t<attribute name=\"maven.pomderived\" value=\"true\"/>\n\t\t</attributes>\n\t</classpathentry>\n\t<classpathentry kind=\"src\" output=\"target/test-classes\" path=\"src/test/java\">\n\t\t<attributes>\n\t\t\t<attribute name=\"optional\" value=\"true\"/>\n\t\t\t<attribute name=\"maven.pomderived\" value=\"true\"/>\n\t\t</attributes>\n\t</classpathentry>\n\t<classpathentry kind=\"con\" path=\"org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.7\">\n\t\t<attributes>\n\t\t\t<attribute name=\"maven.pomderived\" value=\"true\"/>\n\t\t</attributes>\n\t</classpathentry>\n\t<classpathentry kind=\"con\" path=\"org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER\">\n\t\t<attributes>\n\t\t\t<attribute name=\"maven.pomderived\" value=\"true\"/>\n\t\t</attributes>\n\t</classpathentry>\n\t<classpathentry kind=\"output\" path=\"target/classes\"/>\n</classpath>\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/.settings/org.eclipse.core.resources.prefs",
    "content": "eclipse.preferences.version=1\nencoding//src/main/java=UTF-8\nencoding//src/main/resources=UTF-8\nencoding//src/test/java=UTF-8\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/.settings/org.eclipse.jdt.core.prefs",
    "content": "eclipse.preferences.version=1\norg.eclipse.jdt.core.compiler.codegen.targetPlatform=1.7\norg.eclipse.jdt.core.compiler.compliance=1.7\norg.eclipse.jdt.core.compiler.problem.forbiddenReference=warning\norg.eclipse.jdt.core.compiler.source=1.7\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/.settings/org.eclipse.m2e.core.prefs",
    "content": "activeProfiles=\neclipse.preferences.version=1\nresolveWorkspaceProjects=true\nversion=1\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/LICENSE.txt",
    "content": " \n                                 Apache License\n                           Version 2.0, January 2004\n                        http://www.apache.org/licenses/\n\n   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n   1. Definitions.\n\n      \"License\" shall mean the terms and conditions for use, reproduction,\n      and distribution as defined by Sections 1 through 9 of this document.\n\n      \"Licensor\" shall mean the copyright owner or entity authorized by\n      the copyright owner that is granting the License.\n\n      \"Legal Entity\" shall mean the union of the acting entity and all\n      other entities that control, are controlled by, or are under common\n      control with that entity. For the purposes of this definition,\n      \"control\" means (i) the power, direct or indirect, to cause the\n      direction or management of such entity, whether by contract or\n      otherwise, or (ii) ownership of fifty percent (50%) or more of the\n      outstanding shares, or (iii) beneficial ownership of such entity.\n\n      \"You\" (or \"Your\") shall mean an individual or Legal Entity\n      exercising permissions granted by this License.\n\n      \"Source\" form shall mean the preferred form for making modifications,\n      including but not limited to software source code, documentation\n      source, and configuration files.\n\n      \"Object\" form shall mean any form resulting from mechanical\n      transformation or translation of a Source form, including but\n      not limited to compiled object code, generated documentation,\n      and conversions to other media types.\n\n      \"Work\" shall mean the work of authorship, whether in Source or\n      Object form, made available under the License, as indicated by a\n      copyright notice that is included in or attached to the work\n      (an example is provided in the Appendix below).\n\n      \"Derivative Works\" shall mean any work, whether in Source or Object\n      form, that is based on (or derived from) the Work and for which the\n      editorial revisions, annotations, elaborations, or other modifications\n      represent, as a whole, an original work of authorship. For the purposes\n      of this License, Derivative Works shall not include works that remain\n      separable from, or merely link (or bind by name) to the interfaces of,\n      the Work and Derivative Works thereof.\n\n      \"Contribution\" shall mean any work of authorship, including\n      the original version of the Work and any modifications or additions\n      to that Work or Derivative Works thereof, that is intentionally\n      submitted to Licensor for inclusion in the Work by the copyright owner\n      or by an individual or Legal Entity authorized to submit on behalf of\n      the copyright owner. For the purposes of this definition, \"submitted\"\n      means any form of electronic, verbal, or written communication sent\n      to the Licensor or its representatives, including but not limited to\n      communication on electronic mailing lists, source code control systems,\n      and issue tracking systems that are managed by, or on behalf of, the\n      Licensor for the purpose of discussing and improving the Work, but\n      excluding communication that is conspicuously marked or otherwise\n      designated in writing by the copyright owner as \"Not a Contribution.\"\n\n      \"Contributor\" shall mean Licensor and any individual or Legal Entity\n      on behalf of whom a Contribution has been received by Licensor and\n      subsequently incorporated within the Work.\n\n   2. Grant of Copyright License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      copyright license to reproduce, prepare Derivative Works of,\n      publicly display, publicly perform, sublicense, and distribute the\n      Work and such Derivative Works in Source or Object form.\n\n   3. Grant of Patent License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      (except as stated in this section) patent license to make, have made,\n      use, offer to sell, sell, import, and otherwise transfer the Work,\n      where such license applies only to those patent claims licensable\n      by such Contributor that are necessarily infringed by their\n      Contribution(s) alone or by combination of their Contribution(s)\n      with the Work to which such Contribution(s) was submitted. If You\n      institute patent litigation against any entity (including a\n      cross-claim or counterclaim in a lawsuit) alleging that the Work\n      or a Contribution incorporated within the Work constitutes direct\n      or contributory patent infringement, then any patent licenses\n      granted to You under this License for that Work shall terminate\n      as of the date such litigation is filed.\n\n   4. Redistribution. You may reproduce and distribute copies of the\n      Work or Derivative Works thereof in any medium, with or without\n      modifications, and in Source or Object form, provided that You\n      meet the following conditions:\n\n      (a) You must give any other recipients of the Work or\n          Derivative Works a copy of this License; and\n\n      (b) You must cause any modified files to carry prominent notices\n          stating that You changed the files; and\n\n      (c) You must retain, in the Source form of any Derivative Works\n          that You distribute, all copyright, patent, trademark, and\n          attribution notices from the Source form of the Work,\n          excluding those notices that do not pertain to any part of\n          the Derivative Works; and\n\n      (d) If the Work includes a \"NOTICE\" text file as part of its\n          distribution, then any Derivative Works that You distribute must\n          include a readable copy of the attribution notices contained\n          within such NOTICE file, excluding those notices that do not\n          pertain to any part of the Derivative Works, in at least one\n          of the following places: within a NOTICE text file distributed\n          as part of the Derivative Works; within the Source form or\n          documentation, if provided along with the Derivative Works; or,\n          within a display generated by the Derivative Works, if and\n          wherever such third-party notices normally appear. The contents\n          of the NOTICE file are for informational purposes only and\n          do not modify the License. You may add Your own attribution\n          notices within Derivative Works that You distribute, alongside\n          or as an addendum to the NOTICE text from the Work, provided\n          that such additional attribution notices cannot be construed\n          as modifying the License.\n\n      You may add Your own copyright statement to Your modifications and\n      may provide additional or different license terms and conditions\n      for use, reproduction, or distribution of Your modifications, or\n      for any such Derivative Works as a whole, provided Your use,\n      reproduction, and distribution of the Work otherwise complies with\n      the conditions stated in this License.\n\n   5. Submission of Contributions. Unless You explicitly state otherwise,\n      any Contribution intentionally submitted for inclusion in the Work\n      by You to the Licensor shall be under the terms and conditions of\n      this License, without any additional terms or conditions.\n      Notwithstanding the above, nothing herein shall supersede or modify\n      the terms of any separate license agreement you may have executed\n      with Licensor regarding such Contributions.\n\n   6. Trademarks. This License does not grant permission to use the trade\n      names, trademarks, service marks, or product names of the Licensor,\n      except as required for reasonable and customary use in describing the\n      origin of the Work and reproducing the content of the NOTICE file.\n\n   7. Disclaimer of Warranty. Unless required by applicable law or\n      agreed to in writing, Licensor provides the Work (and each\n      Contributor provides its Contributions) on an \"AS IS\" BASIS,\n      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n      implied, including, without limitation, any warranties or conditions\n      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n      PARTICULAR PURPOSE. You are solely responsible for determining the\n      appropriateness of using or redistributing the Work and assume any\n      risks associated with Your exercise of permissions under this License.\n\n   8. Limitation of Liability. In no event and under no legal theory,\n      whether in tort (including negligence), contract, or otherwise,\n      unless required by applicable law (such as deliberate and grossly\n      negligent acts) or agreed to in writing, shall any Contributor be\n      liable to You for damages, including any direct, indirect, special,\n      incidental, or consequential damages of any character arising as a\n      result of this License or out of the use or inability to use the\n      Work (including but not limited to damages for loss of goodwill,\n      work stoppage, computer failure or malfunction, or any and all\n      other commercial damages or losses), even if such Contributor\n      has been advised of the possibility of such damages.\n\n   9. Accepting Warranty or Additional Liability. While redistributing\n      the Work or Derivative Works thereof, You may choose to offer,\n      and charge a fee for, acceptance of support, warranty, indemnity,\n      or other liability obligations and/or rights consistent with this\n      License. However, in accepting such obligations, You may act only\n      on Your own behalf and on Your sole responsibility, not on behalf\n      of any other Contributor, and only if You agree to indemnify,\n      defend, and hold each Contributor harmless for any liability\n      incurred by, or claims asserted against, such Contributor by reason\n      of your accepting any such warranty or additional liability.\n\n   END OF TERMS AND CONDITIONS\n\n   Copyright 2008-2013 DataNucleus\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/pom.xml",
    "content": "<project xmlns=\"http://maven.apache.org/POM/4.0.0\"\n    xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n    xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd\">\n    <modelVersion>4.0.0</modelVersion>\n    <groupId>de.zib.scalaris</groupId>\n    <artifactId>datanucleus-store-test</artifactId>\n    <version>0.0.1-SNAPSHOT</version>\n    <name>datanucleus-store-test</name>\n    <packaging>jar</packaging>\n    <description>\n        Artifact for testing the scalaris datastore for datanucleus using JDO.\n        This is based of the datanucleus 4.0 JDO tutorial samples\n        (http://sourceforge.net/projects/datanucleus/files/datanucleus-samples/).\n    </description>\n\n    <properties>\n        <encoding>UTF-8</encoding>\n        <maven.compiler.source>1.7</maven.compiler.source>\n        <maven.compiler.target>1.7</maven.compiler.target>\n    </properties>\n\n    <repositories>\n        <repository>\n            <id>scalaris</id>\n            <url>https://scalaris-team.github.io/scalaris/maven</url>\n        </repository>\n    </repositories>\n\n    <dependencies>\n        <dependency>\n            <groupId>de.zib.scalaris</groupId>\n            <artifactId>datanucleus-store</artifactId>\n            <version>[0,)</version>\n        </dependency>\n\n        <dependency>\n            <groupId>org.apache.poi</groupId>\n            <artifactId>poi</artifactId>\n            <version>4.1.1</version>\n        </dependency>\n\n        <dependency>\n            <groupId>junit</groupId>\n            <artifactId>junit</artifactId>\n            <version>[4.11,)</version>\n        </dependency>\n        <dependency>\n            <groupId>org.datanucleus</groupId>\n            <artifactId>datanucleus-core</artifactId>\n            <version>[4.0.3,4.99)</version>\n        </dependency>\n        <dependency>\n            <groupId>org.datanucleus</groupId>\n            <artifactId>datanucleus-api-jdo</artifactId>\n            <version>4.1.0-release</version>\n        </dependency>\n        <dependency>\n            <groupId>javax.jdo</groupId>\n            <artifactId>jdo-api</artifactId>\n            <version>3.0.1</version>\n        </dependency>\n        <dependency>\n            <groupId>log4j</groupId>\n            <artifactId>log4j</artifactId>\n            <version>[1.2, 1.3]</version>\n        </dependency>\n    </dependencies>\n\n    <build>\n        <plugins>\n            <plugin>\n                <artifactId>maven-clean-plugin</artifactId>\n                <version>2.5</version>\n                <configuration>\n                    <filesets>\n                        <fileset>\n                            <directory>${basedir}</directory>\n                            <includes>\n                                <include>*.log</include>\n                                <include>tutorial.ods</include>\n                                <include>tutorial.xls</include>\n                            </includes>\n                        </fileset>\n                        <fileset>\n                            <directory>${basedir}/testDB</directory>\n                        </fileset>\n                    </filesets>\n                </configuration>\n            </plugin>\n            <plugin>\n                <artifactId>maven-compiler-plugin</artifactId>\n                <version>2.3.2</version>\n            </plugin>\n            <plugin>\n                <groupId>org.datanucleus</groupId>\n                <artifactId>datanucleus-maven-plugin</artifactId>\n                <version>4.0.0-m2</version>\n                <configuration>\n                    <api>JDO</api>\n                    <persistenceUnitName>Scalaris_Test</persistenceUnitName>\n                    <log4jConfiguration>${basedir}/src/main/resources/log4j.properties</log4jConfiguration>\n                    <verbose>false</verbose>\n                </configuration>\n                <executions>\n                    <execution>\n                        <phase>compile</phase>\n                        <goals>\n                            <goal>enhance</goal>\n                        </goals>\n                    </execution>\n                </executions>\n            </plugin>\n            <plugin>\n                <groupId>org.apache.maven.plugins</groupId>\n                <artifactId>maven-surefire-plugin</artifactId>\n                <version>3.0.0-M1</version>\n                <configuration>\n                    <systemPropertyVariables>\n                        <scalaris.node>${scalaris.node}</scalaris.node>\n                    </systemPropertyVariables>\n                </configuration>\n            </plugin>\n            <plugin>\n                <artifactId>maven-assembly-plugin</artifactId>\n                <version>2.2.1</version>\n                <configuration>\n                    <descriptors>\n                        <descriptor>${basedir}/assembly.xml</descriptor>\n                    </descriptors>\n                </configuration>\n            </plugin>\n        </plugins>\n    </build>\n</project>\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/src/main/java/de/zib/scalaris/datanucleus/store/test/Author.java",
    "content": "/**********************************************************************\nCopyright (c) 2003 Andy Jefferson and others. All rights reserved.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\nContributors:\n    ...\n **********************************************************************/\npackage de.zib.scalaris.datanucleus.store.test;\n\nimport javax.jdo.annotations.PersistenceCapable;\nimport javax.jdo.annotations.PrimaryKey;\n\n@PersistenceCapable\npublic class Author {\n\n    @PrimaryKey\n    protected String name = null;\n\n    public Author(String name) {\n        this.name = name;\n    }\n\n    public void setName(String name) {\n        this.name = name;\n    }\n\n    public String toString() {\n        return \"Author : \" + name;\n    }\n\n    public boolean equals(Object o) {\n        if (o instanceof Author) {\n            Author other = (Author) o;\n            return other.name.equals(name);\n        }\n        return false;\n    }\n\n    @Override\n    public int hashCode() {\n        return name != null? name.hashCode() : 0;\n    }\n}"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/src/main/java/de/zib/scalaris/datanucleus/store/test/Book.java",
    "content": "/**********************************************************************\nCopyright (c) 2003 Andy Jefferson and others. All rights reserved.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\nContributors:\n    ...\n **********************************************************************/\npackage de.zib.scalaris.datanucleus.store.test;\n\nimport javax.jdo.annotations.PersistenceCapable;\nimport javax.jdo.annotations.Persistent;\n\n/**\n * Definition of a Book. Extends basic Product class.\n */\n@PersistenceCapable\npublic class Book extends Product {\n\n    @Persistent(defaultFetchGroup = \"true\")\n    protected Author author = null;\n\n    protected String isbn = null;\n\n    protected String publisher = null;\n\n    public Book(String name, String description, double price, Author author,\n            String isbn, String publisher) {\n        super(name, description, price);\n        this.author = author;\n        this.isbn = isbn;\n        this.publisher = publisher;\n    }\n\n    public Author getAuthor() {\n        return author;\n    }\n\n    public String getIsbn() {\n        return isbn;\n    }\n\n    public String getPublisher() {\n        return publisher;\n    }\n\n    public void setAuthor(Author author) {\n        this.author = author;\n    }\n\n    public void setIsbn(String isbn) {\n        this.isbn = isbn;\n    }\n\n    public void setPublisher(String publisher) {\n        this.publisher = publisher;\n    }\n\n    public String toString() {\n        return \"Book : \" + author + \" - \" + name;\n    }\n\n    public boolean equals(Object o) {\n        if (o instanceof Book && super.equals(o)) {\n            Book other = (Book) o;\n\n            if (other.isbn.equals(isbn) && other.publisher.equals(publisher)) {\n\n                if (author == null && other.author == null)\n                    return true;\n                if (author == null || other.author == null)\n                    return false;\n\n                return other.author.equals(author);\n            }\n        }\n        return false;\n    }\n\n    @Override\n    public int hashCode() {\n        int authorHash = author != null? author.hashCode() : 0;\n        int isbnHash = isbn != null ? isbn.hashCode() : 0;\n        int publisherHash = publisher != null? publisher.hashCode() : 0;\n\n        return super.hashCode() + authorHash + isbnHash + publisherHash;\n    }\n}"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/src/main/java/de/zib/scalaris/datanucleus/store/test/Inventory.java",
    "content": "/**********************************************************************\nCopyright (c) 2011 Andy Jefferson and others. All rights reserved.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\nContributors:\n    ...\n **********************************************************************/\npackage de.zib.scalaris.datanucleus.store.test;\n\nimport java.util.Arrays;\nimport java.util.HashSet;\nimport java.util.Set;\n\nimport javax.jdo.annotations.PersistenceCapable;\nimport javax.jdo.annotations.Persistent;\nimport javax.jdo.annotations.PrimaryKey;\n\n/**\n * Definition of an Inventory of products.\n */\n@PersistenceCapable\npublic class Inventory {\n\n    @PrimaryKey\n    protected String name = null;\n\n    @Persistent(defaultFetchGroup = \"true\")\n    protected Set<Product> products = new HashSet<Product>();\n\n    public Inventory(String name) {\n        this.name = name;\n    }\n\n    public String getName() {\n        return name;\n    }\n\n    public Set<Product> getProducts() {\n        return products;\n    }\n\n    public void addAll(Product... p) {\n        for (Product prod : p) {\n            add(prod);\n        }\n    }\n\n    public void add(Product p) {\n        products.add(p);\n    }\n\n    public String toString() {\n        StringBuilder prodString = new StringBuilder(\"null\");\n        if (products != null) {\n            Product[] prod = products.toArray(new Product[0]);\n            prodString = new StringBuilder(\"[\");\n            for (Product p : prod) {\n                prodString.append(\"{\").append(p.toString()).append(\"},\");\n            }\n            prodString.append(\"]\");\n        }\n        return \"Inventory: \" + name + \"; Products: \" + prodString;\n    }\n\n    public boolean equals(Object o) {\n        if (o instanceof Inventory) {\n            Inventory other = (Inventory) o;\n            if (!name.equals(other.name))\n                return false;\n\n            // compare their products\n            if (products == null && other.products == null)\n                return true;\n            if (products == null || other.products == null)\n                return false;\n\n            Product[] prods1 = products.toArray(new Product[0]);\n            Product[] prods2 = other.products.toArray(new Product[0]);\n\n            return Arrays.asList(prods1).containsAll(other.products)\n                    && Arrays.asList(prods2).containsAll(products);\n        }\n        return false;\n    }\n\n    @Override\n    public int hashCode() {\n        int hash = name.hashCode();\n        if (products == null) return hash;\n\n        for (Product p : products) {\n            hash += p.hashCode();\n        }\n        return hash;\n    }\n}"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/src/main/java/de/zib/scalaris/datanucleus/store/test/Product.java",
    "content": "/**********************************************************************\nCopyright (c) 2003 Andy Jefferson and others. All rights reserved.\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\nContributors:\n    ...\n **********************************************************************/\npackage de.zib.scalaris.datanucleus.store.test;\n\nimport javax.jdo.annotations.IdGeneratorStrategy;\nimport javax.jdo.annotations.PersistenceCapable;\nimport javax.jdo.annotations.Persistent;\nimport javax.jdo.annotations.PrimaryKey;\n\n/**\n * Definition of a Product Represents a product, and contains the key aspects of\n * the item.\n */\n@PersistenceCapable\npublic class Product {\n\n    @PrimaryKey\n    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)\n    private long id;\n\n    protected String name = null;\n\n    protected String description = null;\n\n    protected double price = 0.0;\n\n    public Product(String name, String description, double price) {\n        this.name = name;\n        this.description = description;\n        this.price = price;\n    }\n\n    public long getId() {\n        return id;\n    }\n\n    public String getName() {\n        return name;\n    }\n\n    public String getDescription() {\n        return description;\n    }\n\n    public double getPrice() {\n        return price;\n    }\n\n    public void setName(String name) {\n        this.name = name;\n    }\n\n    public void setDescription(String description) {\n        this.description = description;\n    }\n\n    public void setPrice(double price) {\n        this.price = price;\n    }\n\n    public String toString() {\n        return \"Product : \" + id + \" name=\" + name + \" [\" + description + \"]\";\n    }\n\n    @Override\n    public boolean equals(Object o) {\n        if (o instanceof Product) {\n            Product other = (Product) o;\n            return other.price == price\n                    && other.description.equals(description)\n                    && name.equals(other.name);\n        }\n        return false;\n    }\n\n    @Override\n    public int hashCode() {\n        int descHash = description != null? description.hashCode() : 0;\n        int nameHash = name != null? name.hashCode() : 0;\n\n        return descHash + nameHash + ((int) price);\n    }\n}"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/src/main/resources/META-INF/persistence.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\" ?>\n<persistence xmlns=\"http://java.sun.com/xml/ns/persistence\"\n    xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n    xsi:schemaLocation=\"http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd\" version=\"1.0\">\n\n    <!-- Scalaris Test \"unit\" -->\n    <persistence-unit name=\"Scalaris_Test\">\n        <class>de.zib.scalaris.datanucleus.store.test.Inventory</class>\n        <class>de.zib.scalaris.datanucleus.store.test.Product</class>\n        <class>de.zib.scalaris.datanucleus.store.test.Book</class>\n        <class>de.zib.scalaris.datanucleus.store.test.Author</class>\n        <exclude-unlisted-classes/>\n        <properties>\n            <property name=\"javax.jdo.option.ConnectionURL\" value=\"scalaris:node\" />\n\n            <property name=\"scalaris.cookie\" value=\"chocolate chip cookie\" />\n            <property name=\"scalaris.node\" value=\"firstnode@localhost\" />\n            <property name=\"scalaris.debug\" value=\"true\" />\n            <property name=\"scalaris.client.appendUUID\" value=\"true\" />\n            <property name=\"scalaris.client.name\" value=\"java_client_datatnucleus\" />\n            <property name=\"scalaris.connection.max\" value=\"100\" />\n            <property name=\"scalaris.connection.timeout\" value=\"250\" />\n\n            <!-- Without this it is neccessary to detach an persistet object manually to be able\n               to use it outside of an transaction -->\n            <property name=\"datanucleus.DetachAllOnCommit\" value=\"true\" />\n        </properties>\n    </persistence-unit>\n</persistence>\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/src/main/resources/log4j.properties",
    "content": "# LOG4J Configuration\r\n# ===================\r\n\r\n# Basic logging goes to \"datanucleus.log\"\r\nlog4j.appender.A1=org.apache.log4j.FileAppender\r\nlog4j.appender.A1.File=datanucleus.log\r\nlog4j.appender.A1.layout=org.apache.log4j.PatternLayout\r\nlog4j.appender.A1.layout.ConversionPattern=%d{HH:mm:ss,SSS} (%t) %-5p [%c] - %m%n\r\n#log4j.appender.A1.Threshold=INFO\r\n\r\n# Categories\r\n# Each category can be set to a \"level\", and to direct to an appender\r\n\r\n# Default to DEBUG level for all DataNucleus categories\r\nlog4j.logger.DataNucleus = OFF\r\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/src/main/resources/package-excel.orm",
    "content": "<?xml version=\"1.0\"?>\n<!DOCTYPE orm SYSTEM \"file:/javax/jdo/orm.dtd\">\n<orm>\n    <package name=\"org.datanucleus.samples.jdo.tutorial\">\n        <class name=\"Inventory\" table=\"Inventories\">\n            <field name=\"name\">\n                <column name=\"Name\" length=\"100\"/>\n            </field>\n            <field name=\"products\"/>\n        </class>\n\n        <class name=\"Product\" table=\"Products\">\n            <inheritance strategy=\"complete-table\"/>\n            <field name=\"id\">\n                <column name=\"Id\" position=\"0\"/>\n            </field>\n            <field name=\"name\">\n                <column name=\"Name\" position=\"1\"/>\n            </field>\n            <field name=\"description\">\n                <column name=\"Description\" position=\"2\"/>\n            </field>\n            <field name=\"price\">\n                <column name=\"Price\" position=\"3\"/>\n            </field>\n        </class>\n\n        <class name=\"Book\" table=\"Books\">\n            <inheritance strategy=\"complete-table\"/>\n            <field name=\"Product.id\">\n                <column name=\"Id\" position=\"0\"/>\n            </field>\n            <field name=\"author\">\n                <column name=\"Author\" position=\"4\"/>\n            </field>\n            <field name=\"isbn\">\n                <column name=\"ISBN\" position=\"5\"/>\n            </field>\n            <field name=\"publisher\">\n                <column name=\"Publisher\" position=\"6\"/>\n            </field>\n        </class>\n    </package>\n</orm>\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/src/test/java/de/zib/scalaris/datanucleus/store/test/ScalarisStoreTests.java",
    "content": "package de.zib.scalaris.datanucleus.store.test;\n\nimport org.junit.runner.RunWith;\nimport org.junit.runners.Suite;\nimport org.junit.runners.Suite.SuiteClasses;\n\n@RunWith(Suite.class)\n@SuiteClasses({ TestScalarisStorage.class, TestScalarisQuery.class })\npublic class ScalarisStoreTests {\n\n}\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/src/test/java/de/zib/scalaris/datanucleus/store/test/StoreUtils.java",
    "content": "package de.zib.scalaris.datanucleus.store.test;\n\nimport java.util.HashMap;\nimport java.util.Map;\nimport java.util.Properties;\n\nimport javax.jdo.JDOHelper;\nimport javax.jdo.PersistenceManager;\nimport javax.jdo.PersistenceManagerFactory;\nimport javax.jdo.Query;\nimport javax.jdo.Transaction;\n\npublic class StoreUtils {\n\n    public static final String PERSISTENCE_UNIT_NAME = \"Scalaris_Test\";\n\n    // For some reason Datanucleus complains sometimes that this property is\n    // not set and fails.\n    static {\n        Properties props = System.getProperties();\n        props.setProperty(\"javax.jdo.option.PersistenecUnitName\", PERSISTENCE_UNIT_NAME);\n    }\n\n    /**\n     * Returns a new PersistenceManager of persistence unit\n     * PERSISTANCE_UNIT_NAME\n     * \n     * @return a new PersistenceManager\n     */\n    public static PersistenceManager getNewPersistenceManager() {\n        Map<String, String> propOverrides = new HashMap<>();\n        String scalarisNodeOverride = System.getProperty(\"scalaris.node\");\n        if (scalarisNodeOverride != null) {\n            propOverrides.put(\"scalaris.node\", scalarisNodeOverride);\n        }\n\n        PersistenceManagerFactory pmf = JDOHelper\n                .getPersistenceManagerFactory(propOverrides, PERSISTENCE_UNIT_NAME);\n        return pmf.getPersistenceManager();\n    }\n\n    /**\n     * Delete all instances stored in the data store of the passed candidate\n     * classes\n     * \n     * @param clazzes\n     *            Candidate classes used for deletion.\n     * @return Number of deleted elements.\n     */\n    public static long deleteAllInstances(Class<?>... clazzes) {\n        long deleted = 0;\n\n        for (Class<?> clazz : clazzes) {\n            PersistenceManager pm = getNewPersistenceManager();\n            Query q = pm.newQuery(clazz);\n            deleted += q.deletePersistentAll();\n        }\n        return deleted;\n    }\n\n    /*\n     * Utility to store multiple objects. The order in which objects are stored\n     * is not guaranteed.\n     */\n    public static void storeObjects(Object... o) {\n        for (Object obj : o) {\n            storeObject(obj);\n        }\n    }\n\n    /**\n     * Store an object in the data store. Returns the identity of the stored\n     * object.\n     */\n    public static Object storeObject(Object o) {\n        PersistenceManager pm = getNewPersistenceManager();\n        Transaction tx = pm.currentTransaction();\n\n        Object objectID;\n        try {\n            tx.begin();\n            pm.makePersistent(o);\n            objectID = pm.getObjectId(o);\n            tx.commit();\n        } finally {\n            if (tx.isActive()) {\n                tx.rollback();\n            }\n            pm.close();\n        }\n        return objectID;\n    }\n\n    /**\n     * Retrieve an object stored in the data store by its ID. Can throw\n     * exceptions if unsuccessful (e.g. JDOObjectNotFoundException).\n     */\n    public static Object retrieveObjectById(Object objectId) {\n        PersistenceManager pm = getNewPersistenceManager();\n        Transaction tx = pm.currentTransaction();\n\n        Object retrieved;\n        try {\n            tx.begin();\n            retrieved = pm.getObjectById(objectId);\n            tx.commit();\n        } finally {\n            if (tx.isActive()) {\n                tx.rollback();\n            }\n            pm.close();\n        }\n        return retrieved;\n    }\n\n    /**\n     * This method can be used to retrieve an object by its primary key if it\n     * consists of only one attribute. Can throw exceptions if unsuccessful\n     * (e.g. JDOObjectNotFoundException).\n     */\n    public static Object retrieveObjectBySingleKey(Class<?> objectClass,\n            Object keyValue) {\n        PersistenceManager pm = getNewPersistenceManager();\n        Transaction tx = pm.currentTransaction();\n\n        Object retrieved;\n        try {\n            tx.begin();\n            retrieved = pm.getObjectById(objectClass, keyValue);\n            tx.commit();\n        } finally {\n            if (tx.isActive()) {\n                tx.rollback();\n            }\n            pm.close();\n        }\n        return retrieved;\n    }\n\n    /**\n     * Delete an object stored in the data store by its ID. Can throw exceptions\n     * if unsuccessful (e.g. JDOObjectNotFoundException).\n     */\n    public static void deleteObjectById(Object objectId) {\n        PersistenceManager pm = getNewPersistenceManager();\n        Transaction tx = pm.currentTransaction();\n        try {\n            tx.begin();\n            Object retrieved = pm.getObjectById(objectId);\n            pm.deletePersistent(retrieved);\n            tx.commit();\n        } finally {\n            if (tx.isActive()) {\n                tx.rollback();\n            }\n            pm.close();\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/src/test/java/de/zib/scalaris/datanucleus/store/test/TestScalarisQuery.java",
    "content": "package de.zib.scalaris.datanucleus.store.test;\n\nimport static org.junit.Assert.*;\nimport static de.zib.scalaris.datanucleus.store.test.StoreUtils.*;\n\nimport java.util.List;\n\nimport javax.jdo.PersistenceManager;\nimport javax.jdo.Query;\n\nimport org.junit.AfterClass;\nimport org.junit.BeforeClass;\nimport org.junit.Test;\n\n@SuppressWarnings(\"rawtypes\")\npublic class TestScalarisQuery {\n\n    private static Product discman = new Product(\"Sony Discman\",\n            \"A standard discman from Sony\", 1.99);\n\n    @BeforeClass\n    public static void setup() {\n        Product[] products = {\n                discman,\n                new Product(\"Sony Xperia Z1\", \"A fancy smartphone\", 200.3),\n                new Product(\"Sony Xperia Z2\", \"Another smartphone\", 300.1),\n                new Product(\"Sony Xperia Z3\", \"Yet another smartphone\", 400.2),\n                new Product(\"Sony Xperia Z1 Compact\", \"A compact smartphone\",\n                        250.7), };\n        Inventory invSony = new Inventory(\"Sony\");\n        invSony.addAll(products);\n        storeObject(invSony);\n\n        Author tolkien = new Author(\"JRR Tolkien\");\n        Author lovecraft = new Author(\"H. P. LoveCraft\");\n        Object[] books = {\n                new Book(\"Lord of the rings 1\", \"Stuff happens\", 56.00,\n                        tolkien, \"111111\", \"PublisherA\"),\n                new Book(\"Lord of the rings 2\", \"More Stuff happens\", 23.00,\n                        tolkien, \"22222\", \"PublisherB\"),\n                new Book(\"Cthulhu\", \"horror stuff\", 19.99, lovecraft, \"43433\",\n                        \"PublisherC\") };\n        storeObjects(books);\n    }\n\n    @AfterClass\n    public static void tearDown() {\n        deleteAllInstances(Author.class, Product.class, Book.class,\n                Inventory.class);\n    }\n\n    @Test\n    public void testQueryProductByName() {\n        PersistenceManager pm = getNewPersistenceManager();\n        Query q = pm.newQuery(Product.class, \"name == 'Sony Discman'\");\n        List result = (List) q.execute();\n        assertEquals(1, result.size());\n\n        Product p = (Product) result.get(0);\n        assertEquals(\"Sony Discman\", p.name);\n\n        assertEquals(discman, p);\n    }\n\n    @Test\n    public void testQueryFilter() {\n        PersistenceManager pm = getNewPersistenceManager();\n        Query q = pm.newQuery(Product.class);\n        q.setFilter(\"price < 300.1\");\n        List result = (List) q.execute();\n        assertEquals(3, result.size());\n    }\n\n    @Test\n    public void testQueryOrderByPrice() {\n        PersistenceManager pm = getNewPersistenceManager();\n        Query q = pm.newQuery(Product.class);\n        q.setOrdering(\"price ascending\");\n        List result = (List) q.execute();\n        assertEquals(5, result.size());\n\n        // check if order is correct\n        double lastSeenPrice = -1;\n        for (int i = 0; i < result.size(); i++) {\n            Product p = (Product) result.get(i);\n            assertTrue(lastSeenPrice <= p.getPrice());\n            lastSeenPrice = p.getPrice();\n        }\n    }\n}"
  },
  {
    "path": "contrib/datanucleus/scalaris-datanucleus-store-test/src/test/java/de/zib/scalaris/datanucleus/store/test/TestScalarisStorage.java",
    "content": "package de.zib.scalaris.datanucleus.store.test;\n\nimport static org.junit.Assert.assertEquals;\nimport static org.junit.Assert.fail;\n\nimport javax.jdo.JDOObjectNotFoundException;\n\nimport org.junit.After;\nimport org.junit.Test;\n\nimport static de.zib.scalaris.datanucleus.store.test.StoreUtils.*;\n\npublic class TestScalarisStorage {\n\n    @After\n    public void after() {\n        deleteAllInstances(Author.class, Product.class, Book.class,\n                Inventory.class);\n    }\n\n    /**\n     * Store a simple object (without relationships).\n     */\n    @Test\n    public void test01Store() {\n        Product product = new Product(\"Sony Discman\",\n                \"A standard discman from Sony\", 1.99);\n        storeObject(product);\n    }\n\n    /**\n     * Store a object with a relationship to another persistable object.\n     */\n    @Test\n    public void test02Store() {\n        Author author = new Author(\"Jrr\");\n        Book book = new Book(\"Lord of the Rings by Tolkien\",\n                \"The classic story\", 49.99, author, \"12345678\",\n                \"MyBooks Factory\");\n        storeObject(book);\n    }\n\n    /**\n     * Store an object with 1-N relationships.\n     */\n    @Test\n    public void test03Store() {\n        Author author = new Author(\"Tolkien\");\n        Inventory inv = new Inventory(\"My Inventory\");\n        Product product = new Product(\"Sony Discman\",\n                \"A standard discman from Sony\", 200.00);\n        Book book = new Book(\"Lord of the Rings by Tolkien\",\n                \"The classic story\", 49.99, author, \"12345678\",\n                \"MyBooks Factory\");\n        inv.add(product);\n        inv.add(book);\n\n        storeObject(inv);\n    }\n\n    /**\n     * Retrieve a simple object (without relationships) by its ID.\n     */\n    @Test\n    public void test01RetrieveById() {\n        Product product = new Product(\"Sony Discman\",\n                \"A standard discman from Sony\", 1.99);\n        Object productId = storeObject(product);\n\n        Product retrieved = (Product) retrieveObjectById(productId);\n        assertEquals(product, retrieved);\n    }\n\n    /**\n     * Retrieve a simple object which inherits from another persistence class\n     * and has relationships by its ID.\n     */\n    @Test\n    public void test02RetrieveById() {\n        Author author = new Author(\"JRRR Tolkien\");\n        Book book = new Book(\"Lord of the Rings by Tolkien\",\n                \"The classic story\", 49.99, author, \"12345678\",\n                \"MyBooks Factory\");\n        Object bookId = storeObject(book);\n\n        Book retrieved = (Book) retrieveObjectById(bookId);\n        assertEquals(book, retrieved);\n    }\n\n    /**\n     * Retrieve an object with 1-N relationships by its ID.\n     */\n    @Test\n    public void test03RetrieveById() {\n        Inventory inv = new Inventory(\"Retrieval_Inventory\");\n        Product product = new Product(\"Sony Discman\",\n                \"A standard discman from Sony\", 200.00);\n        Author author = new Author(\"JRR Tolkien\");\n        Book book = new Book(\"Lord of the Rings by Tolkien\",\n                \"The classic story\", 49.99, author, \"12345678\",\n                \"MyBooks Factory\");\n        inv.add(product);\n        inv.add(book);\n        Object inventoryId = storeObject(inv);\n\n        // check child objects\n        Product retrievedProduct = (Product) retrieveObjectBySingleKey(\n                Product.class, product.getId());\n        assertEquals(product, retrievedProduct);\n        Book retrievedBook = (Book) retrieveObjectBySingleKey(Book.class,\n                book.getId());\n        assertEquals(book, retrievedBook);\n\n        // check parent\n        Inventory retrieved = (Inventory) retrieveObjectById(inventoryId);\n        assertEquals(inv, retrieved);\n    }\n\n    /**\n     * Retrieve an object by its primary key which consists of only one\n     * attribute.\n     */\n    @Test\n    public void test01RetrieveBySingleKey() {\n        Product product = new Product(\"Sony Discman\",\n                \"A standard discman from Sony\", 1.99);\n        storeObject(product);\n\n        Product retrieved = (Product) retrieveObjectBySingleKey(\n                product.getClass(), product.getId());\n        assertEquals(product, retrieved);\n    }\n\n    /**\n     * Try to retrieve an object by its primary key which does not exist.\n     */\n    @Test\n    public void test02RetrieveBySingleKey() {\n        Product product = new Product(\"Sony Discman\",\n                \"A standard discman from Sony\", 1.99);\n\n        try {\n            retrieveObjectBySingleKey(product.getClass(), product.getId());\n            fail(\"Expected an expcetion to be thrown because the stored object does not exist\");\n        } catch (JDOObjectNotFoundException e) {\n            // good\n        }\n    }\n\n    /**\n     * Delete an object by its ID\n     */\n    @Test\n    public void test01DeleteById() {\n        Author author = new Author(\"JRR\");\n        Book book = new Book(\"Lord\", \"The \", 49.99, author, \"1234\", \"MyBooks\");\n        Object bookId = storeObject(book);\n\n        deleteObjectById(bookId);\n\n        try {\n            retrieveObjectById(bookId);\n            fail(\"Excepted JDOObjectNotFoundException\");\n        } catch (JDOObjectNotFoundException e) {\n            // if we are here everything worked fine\n        }\n    }\n\n    /**\n     * Update one field of a stored object\n     */\n    @Test\n    public void test01SingleFieldUpdate() {\n        Author author = new Author(\"JRR2\");\n        Book book = new Book(\"Lord\", \"The \", 49.99, author, \"1234\", \"MyBooks\");\n        Object bookId = storeObject(book);\n        Book retrieved = (Book) retrieveObjectById(bookId);\n        assertEquals(book, retrieved);\n\n        retrieved.setPrice(100.99);\n        Object updatedId = storeObject(retrieved);\n\n        Book updated = (Book) retrieveObjectById(updatedId);\n        assertEquals(retrieved, updated);\n    }\n}"
  },
  {
    "path": "contrib/dialyzer-col.sed",
    "content": "# for color codes see http://en.wikipedia.org/wiki/ANSI_escape_code\n# blue bold\ns/^.*erl:[0123456789]*/\\x1b[1;34m\\0\\x1b[0m/\n# blue normal\ns/is a supertype of the success typing/\\x1b[34m\\0\\x1b[0m/\ns/that the function might also return/\\x1b[34m\\0\\x1b[0m/\ns/but the inferred return is/\\x1b[34m\\0\\x1b[0m/\ns/but this value is unmatched/\\x1b[34m\\0\\x1b[0m/\n# green bold\ns/will never be called/\\x1b[1;32m\\0\\x1b[0m/\ns/has no local return/\\x1b[1;32m\\0\\x1b[0m/\n# red bold\ns/can never succeed/\\x1b[1;31m\\0\\x1b[0m/\ns/Call to missing or unexported function/\\x1b[1;31m\\0\\x1b[0m/\ns/contains an opaque term/\\x1b[1;31m\\0\\x1b[0m/\ns/contains opaque terms/\\x1b[1;31m\\0\\x1b[0m/\ns/might fail due to a possible race condition/\\x1b[1;31m\\0\\x1b[0m/\ns/breaks the contract/\\x1b[1;31m\\0\\x1b[0m/\ns/breaks the opaqueness of the term/\\x1b[1;31m\\0\\x1b[0m/\ns/can never match/\\x1b[1;31m\\0\\x1b[0m/\ns/Invalid type specification/\\x1b[1;31m\\0\\x1b[0m/\ns/Unknown types:/\\x1b[1;31m\\0\\x1b[0m/\ns/Unknown functions:/\\x1b[1;31m\\0\\x1b[0m/\n\n"
  },
  {
    "path": "contrib/dotgraph_cyclon.sh",
    "content": "#!/bin/bash\n\n# Script to create a graph (using graphviz / dot) from all cyclon caches from one cycle.\n# Use the ?PRINT_CACHE_FOR_DOT macro in gossip_cyclon to print the caches to the logfile\n# Only works within one VM. You may need to delete old logfiles first.\n\n# To compare the graphs from different cycles run something like\n#   for i in {0..10}; do ./dotgraph.sh $i cyclon_graph$i; done\n# to get graphs for the first 10 cycles.\n\n# set -x\n\nCYCLE=$1\nLOGFILE=\"scalaris_log4erl.txt\"\nFILENAME=\"cyclon\"\nKEEP_GV_FILE=true\nRADIUS=3 # value depens on the number of nodes. Default: 3, Increase for > 10 Nodes.\n\nif [ -n \"$2\" ]; then\n    FILENAME=$2\nelse\n    FILENAME=\"cyclon\"\nfi\n\nrm -f $FILENAME.gv\n\ncat <<EOF > $FILENAME.gv\ndigraph $FILENAME {\n    mindist=1\n    layout=neato\n    edge [arrowsize=.5, arrowhead=nonenoneonormal]\n    node [pin=true]\n    //splines=polyline\n\nEOF\n\n# Get all the pids as array\nNodes=$(grep \"\\[Cycle: $CYCLE\\]\" scalaris_log4erl.txt | grep -oh \"<[0-9]*\\.[0-9]*\\.[0-9]*> \" | sort | uniq | tr -d '\\n')\nif [ -z Nodes ]; then\n    echo \"no compatible entries in logfile\"\n    exit 1\nfi\ndeclare -a nodes=( $Nodes )\nlength=${#nodes[@]}\n\n# Calculate node positions (as a circle)\n# Using the pos attribute is the only way to fix node positions between multiple\n# graphs (layout=circo produces similar looking results, but the node positions\n# change depending on the edges)\npi=$(echo \"scale=10; 4*a(1)\" | bc -l)\nfor i in $(seq 1 $length); do\n    x=$(echo \"c((($pi*2)/$length)*$i)*$RADIUS\" | bc -l)\n    y=$(echo \"s((($pi*2)/$length)*$i)*$RADIUS\" | bc -l)\n    index=$(echo \"$i-1\"|bc)\n    echo \"    $(echo \"${nodes[$index]}[pos=\\\"$x,$y!\\\"]\");\" >> $FILENAME.gv\ndone\necho -e \"\\n\" >> $FILENAME.gv\n\n# Insert the edges from the print_cache_log() output in the logfile\ngrep \"\\[Cycle: $CYCLE\\]\" $LOGFILE | sed \"s/.*Cycle: $CYCLE] /    /\" >> $FILENAME.gv\n\n# close the graph\necho \"}\" >> $FILENAME.gv\n\n# create pdf\ndot -Tpdf $FILENAME.gv -o $FILENAME.pdf\n\n# delete gv files\nif [ $KEEP_GV_FILE = false ]; then\n    rm $FILENAME.gv\nfi\n"
  },
  {
    "path": "contrib/dotto/.gitignore",
    "content": "ebin\nlogs\n"
  },
  {
    "path": "contrib/dotto/LICENSE",
    "content": "Mozilla Public License Version 2.0\n==================================\n\n1. Definitions\n--------------\n\n1.1. \"Contributor\"\n    means each individual or legal entity that creates, contributes to\n    the creation of, or owns Covered Software.\n\n1.2. \"Contributor Version\"\n    means the combination of the Contributions of others (if any) used\n    by a Contributor and that particular Contributor's Contribution.\n\n1.3. \"Contribution\"\n    means Covered Software of a particular Contributor.\n\n1.4. \"Covered Software\"\n    means Source Code Form to which the initial Contributor has attached\n    the notice in Exhibit A, the Executable Form of such Source Code\n    Form, and Modifications of such Source Code Form, in each case\n    including portions thereof.\n\n1.5. \"Incompatible With Secondary Licenses\"\n    means\n\n    (a) that the initial Contributor has attached the notice described\n        in Exhibit B to the Covered Software; or\n\n    (b) that the Covered Software was made available under the terms of\n        version 1.1 or earlier of the License, but not also under the\n        terms of a Secondary License.\n\n1.6. \"Executable Form\"\n    means any form of the work other than Source Code Form.\n\n1.7. \"Larger Work\"\n    means a work that combines Covered Software with other material, in \n    a separate file or files, that is not Covered Software.\n\n1.8. \"License\"\n    means this document.\n\n1.9. \"Licensable\"\n    means having the right to grant, to the maximum extent possible,\n    whether at the time of the initial grant or subsequently, any and\n    all of the rights conveyed by this License.\n\n1.10. \"Modifications\"\n    means any of the following:\n\n    (a) any file in Source Code Form that results from an addition to,\n        deletion from, or modification of the contents of Covered\n        Software; or\n\n    (b) any new file in Source Code Form that contains any Covered\n        Software.\n\n1.11. \"Patent Claims\" of a Contributor\n    means any patent claim(s), including without limitation, method,\n    process, and apparatus claims, in any patent Licensable by such\n    Contributor that would be infringed, but for the grant of the\n    License, by the making, using, selling, offering for sale, having\n    made, import, or transfer of either its Contributions or its\n    Contributor Version.\n\n1.12. \"Secondary License\"\n    means either the GNU General Public License, Version 2.0, the GNU\n    Lesser General Public License, Version 2.1, the GNU Affero General\n    Public License, Version 3.0, or any later versions of those\n    licenses.\n\n1.13. \"Source Code Form\"\n    means the form of the work preferred for making modifications.\n\n1.14. \"You\" (or \"Your\")\n    means an individual or a legal entity exercising rights under this\n    License. For legal entities, \"You\" includes any entity that\n    controls, is controlled by, or is under common control with You. For\n    purposes of this definition, \"control\" means (a) the power, direct\n    or indirect, to cause the direction or management of such entity,\n    whether by contract or otherwise, or (b) ownership of more than\n    fifty percent (50%) of the outstanding shares or beneficial\n    ownership of such entity.\n\n2. License Grants and Conditions\n--------------------------------\n\n2.1. Grants\n\nEach Contributor hereby grants You a world-wide, royalty-free,\nnon-exclusive license:\n\n(a) under intellectual property rights (other than patent or trademark)\n    Licensable by such Contributor to use, reproduce, make available,\n    modify, display, perform, distribute, and otherwise exploit its\n    Contributions, either on an unmodified basis, with Modifications, or\n    as part of a Larger Work; and\n\n(b) under Patent Claims of such Contributor to make, use, sell, offer\n    for sale, have made, import, and otherwise transfer either its\n    Contributions or its Contributor Version.\n\n2.2. Effective Date\n\nThe licenses granted in Section 2.1 with respect to any Contribution\nbecome effective for each Contribution on the date the Contributor first\ndistributes such Contribution.\n\n2.3. Limitations on Grant Scope\n\nThe licenses granted in this Section 2 are the only rights granted under\nthis License. No additional rights or licenses will be implied from the\ndistribution or licensing of Covered Software under this License.\nNotwithstanding Section 2.1(b) above, no patent license is granted by a\nContributor:\n\n(a) for any code that a Contributor has removed from Covered Software;\n    or\n\n(b) for infringements caused by: (i) Your and any other third party's\n    modifications of Covered Software, or (ii) the combination of its\n    Contributions with other software (except as part of its Contributor\n    Version); or\n\n(c) under Patent Claims infringed by Covered Software in the absence of\n    its Contributions.\n\nThis License does not grant any rights in the trademarks, service marks,\nor logos of any Contributor (except as may be necessary to comply with\nthe notice requirements in Section 3.4).\n\n2.4. Subsequent Licenses\n\nNo Contributor makes additional grants as a result of Your choice to\ndistribute the Covered Software under a subsequent version of this\nLicense (see Section 10.2) or under the terms of a Secondary License (if\npermitted under the terms of Section 3.3).\n\n2.5. Representation\n\nEach Contributor represents that the Contributor believes its\nContributions are its original creation(s) or it has sufficient rights\nto grant the rights to its Contributions conveyed by this License.\n\n2.6. Fair Use\n\nThis License is not intended to limit any rights You have under\napplicable copyright doctrines of fair use, fair dealing, or other\nequivalents.\n\n2.7. Conditions\n\nSections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted\nin Section 2.1.\n\n3. Responsibilities\n-------------------\n\n3.1. Distribution of Source Form\n\nAll distribution of Covered Software in Source Code Form, including any\nModifications that You create or to which You contribute, must be under\nthe terms of this License. You must inform recipients that the Source\nCode Form of the Covered Software is governed by the terms of this\nLicense, and how they can obtain a copy of this License. You may not\nattempt to alter or restrict the recipients' rights in the Source Code\nForm.\n\n3.2. Distribution of Executable Form\n\nIf You distribute Covered Software in Executable Form then:\n\n(a) such Covered Software must also be made available in Source Code\n    Form, as described in Section 3.1, and You must inform recipients of\n    the Executable Form how they can obtain a copy of such Source Code\n    Form by reasonable means in a timely manner, at a charge no more\n    than the cost of distribution to the recipient; and\n\n(b) You may distribute such Executable Form under the terms of this\n    License, or sublicense it under different terms, provided that the\n    license for the Executable Form does not attempt to limit or alter\n    the recipients' rights in the Source Code Form under this License.\n\n3.3. Distribution of a Larger Work\n\nYou may create and distribute a Larger Work under terms of Your choice,\nprovided that You also comply with the requirements of this License for\nthe Covered Software. If the Larger Work is a combination of Covered\nSoftware with a work governed by one or more Secondary Licenses, and the\nCovered Software is not Incompatible With Secondary Licenses, this\nLicense permits You to additionally distribute such Covered Software\nunder the terms of such Secondary License(s), so that the recipient of\nthe Larger Work may, at their option, further distribute the Covered\nSoftware under the terms of either this License or such Secondary\nLicense(s).\n\n3.4. Notices\n\nYou may not remove or alter the substance of any license notices\n(including copyright notices, patent notices, disclaimers of warranty,\nor limitations of liability) contained within the Source Code Form of\nthe Covered Software, except that You may alter any license notices to\nthe extent required to remedy known factual inaccuracies.\n\n3.5. Application of Additional Terms\n\nYou may choose to offer, and to charge a fee for, warranty, support,\nindemnity or liability obligations to one or more recipients of Covered\nSoftware. However, You may do so only on Your own behalf, and not on\nbehalf of any Contributor. You must make it absolutely clear that any\nsuch warranty, support, indemnity, or liability obligation is offered by\nYou alone, and You hereby agree to indemnify every Contributor for any\nliability incurred by such Contributor as a result of warranty, support,\nindemnity or liability terms You offer. You may include additional\ndisclaimers of warranty and limitations of liability specific to any\njurisdiction.\n\n4. Inability to Comply Due to Statute or Regulation\n---------------------------------------------------\n\nIf it is impossible for You to comply with any of the terms of this\nLicense with respect to some or all of the Covered Software due to\nstatute, judicial order, or regulation then You must: (a) comply with\nthe terms of this License to the maximum extent possible; and (b)\ndescribe the limitations and the code they affect. Such description must\nbe placed in a text file included with all distributions of the Covered\nSoftware under this License. Except to the extent prohibited by statute\nor regulation, such description must be sufficiently detailed for a\nrecipient of ordinary skill to be able to understand it.\n\n5. Termination\n--------------\n\n5.1. The rights granted under this License will terminate automatically\nif You fail to comply with any of its terms. However, if You become\ncompliant, then the rights granted under this License from a particular\nContributor are reinstated (a) provisionally, unless and until such\nContributor explicitly and finally terminates Your grants, and (b) on an\nongoing basis, if such Contributor fails to notify You of the\nnon-compliance by some reasonable means prior to 60 days after You have\ncome back into compliance. Moreover, Your grants from a particular\nContributor are reinstated on an ongoing basis if such Contributor\nnotifies You of the non-compliance by some reasonable means, this is the\nfirst time You have received notice of non-compliance with this License\nfrom such Contributor, and You become compliant prior to 30 days after\nYour receipt of the notice.\n\n5.2. If You initiate litigation against any entity by asserting a patent\ninfringement claim (excluding declaratory judgment actions,\ncounter-claims, and cross-claims) alleging that a Contributor Version\ndirectly or indirectly infringes any patent, then the rights granted to\nYou by any and all Contributors for the Covered Software under Section\n2.1 of this License shall terminate.\n\n5.3. In the event of termination under Sections 5.1 or 5.2 above, all\nend user license agreements (excluding distributors and resellers) which\nhave been validly granted by You or Your distributors under this License\nprior to termination shall survive termination.\n\n************************************************************************\n*                                                                      *\n*  6. Disclaimer of Warranty                                           *\n*  -------------------------                                           *\n*                                                                      *\n*  Covered Software is provided under this License on an \"as is\"       *\n*  basis, without warranty of any kind, either expressed, implied, or  *\n*  statutory, including, without limitation, warranties that the       *\n*  Covered Software is free of defects, merchantable, fit for a        *\n*  particular purpose or non-infringing. The entire risk as to the     *\n*  quality and performance of the Covered Software is with You.        *\n*  Should any Covered Software prove defective in any respect, You     *\n*  (not any Contributor) assume the cost of any necessary servicing,   *\n*  repair, or correction. This disclaimer of warranty constitutes an   *\n*  essential part of this License. No use of any Covered Software is   *\n*  authorized under this License except under this disclaimer.         *\n*                                                                      *\n************************************************************************\n\n************************************************************************\n*                                                                      *\n*  7. Limitation of Liability                                          *\n*  --------------------------                                          *\n*                                                                      *\n*  Under no circumstances and under no legal theory, whether tort      *\n*  (including negligence), contract, or otherwise, shall any           *\n*  Contributor, or anyone who distributes Covered Software as          *\n*  permitted above, be liable to You for any direct, indirect,         *\n*  special, incidental, or consequential damages of any character      *\n*  including, without limitation, damages for lost profits, loss of    *\n*  goodwill, work stoppage, computer failure or malfunction, or any    *\n*  and all other commercial damages or losses, even if such party      *\n*  shall have been informed of the possibility of such damages. This   *\n*  limitation of liability shall not apply to liability for death or   *\n*  personal injury resulting from such party's negligence to the       *\n*  extent applicable law prohibits such limitation. Some               *\n*  jurisdictions do not allow the exclusion or limitation of           *\n*  incidental or consequential damages, so this exclusion and          *\n*  limitation may not apply to You.                                    *\n*                                                                      *\n************************************************************************\n\n8. Litigation\n-------------\n\nAny litigation relating to this License may be brought only in the\ncourts of a jurisdiction where the defendant maintains its principal\nplace of business and such litigation shall be governed by laws of that\njurisdiction, without reference to its conflict-of-law provisions.\nNothing in this Section shall prevent a party's ability to bring\ncross-claims or counter-claims.\n\n9. Miscellaneous\n----------------\n\nThis License represents the complete agreement concerning the subject\nmatter hereof. If any provision of this License is held to be\nunenforceable, such provision shall be reformed only to the extent\nnecessary to make it enforceable. Any law or regulation which provides\nthat the language of a contract shall be construed against the drafter\nshall not be used to construe this License against a Contributor.\n\n10. Versions of the License\n---------------------------\n\n10.1. New Versions\n\nMozilla Foundation is the license steward. Except as provided in Section\n10.3, no one other than the license steward has the right to modify or\npublish new versions of this License. Each version will be given a\ndistinguishing version number.\n\n10.2. Effect of New Versions\n\nYou may distribute the Covered Software under the terms of the version\nof the License under which You originally received the Covered Software,\nor under the terms of any subsequent version published by the license\nsteward.\n\n10.3. Modified Versions\n\nIf you create software not governed by this License, and you want to\ncreate a new license for such software, you may create and use a\nmodified version of this License if you rename the license and remove\nany references to the name of the license steward (except to note that\nsuch modified license differs from this License).\n\n10.4. Distributing Source Code Form that is Incompatible With Secondary\nLicenses\n\nIf You choose to distribute Source Code Form that is Incompatible With\nSecondary Licenses under the terms of this version of the License, the\nnotice described in Exhibit B of this License must be attached.\n\nExhibit A - Source Code Form License Notice\n-------------------------------------------\n\n  This Source Code Form is subject to the terms of the Mozilla Public\n  License, v. 2.0. If a copy of the MPL was not distributed with this\n  file, You can obtain one at http://mozilla.org/MPL/2.0/.\n\nIf it is not possible or desirable to put the notice in a particular\nfile, then You may include the notice in a location (such as a LICENSE\nfile in a relevant directory) where a recipient would be likely to look\nfor such a notice.\n\nYou may add additional accurate notices of copyright ownership.\n\nExhibit B - \"Incompatible With Secondary Licenses\" Notice\n---------------------------------------------------------\n\n  This Source Code Form is \"Incompatible With Secondary Licenses\", as\n  defined by the Mozilla Public License, v. 2.0.\n"
  },
  {
    "path": "contrib/dotto/Makefile",
    "content": "PROJECT = dotto\ninclude erlang.mk\n"
  },
  {
    "path": "contrib/dotto/README.rst",
    "content": "dotto\n-----\n\npatcho dotto...\n\na data driven data structure manipulation library for erlang, a superset of\n`json patch <http://tools.ietf.org/html/rfc6902>`_\n\nThe primary change of this fork is the replacement of maps with\ndicts to support older erlang versions.\n\ntests\n-----\n\n::\n\n    make tests\n\nthis are just basic, hardcoded tests\n\nusage\n-----\n\nTODO: Due to compatibility with older erlang versions, maps where replaced\nwith dicts.\n\n::\n\n    Data = #{name => \"bob\", age => 29, friends => [\"sandy\", \"patrick\"], data => #{numbers => [10,11,12]}}.\n\n    dotto:remove(Data, [friends, 1]).\n\n    % {ok,#{age => 29, data => #{numbers => \"\\n\\v\\f\"}, friends => [\"sandy\"], name => \"bob\"}}\n\n    dotto:replace(Data, [data, numbers, 2], 42).\n\n    % {ok,#{age => 29, data => #{numbers => \"\\n\\v*\"}, friends => [\"sandy\",\"patrick\"], name => \"bob\"}}\n\n    % <<\"-\">> means append at the end, see json patch rfc\n    dotto:add(Data, [friends, <<\"-\">>], \"plankton\").\n\n    % {ok,#{age => 29, data => #{numbers => \"\\n\\v\\f\"}, friends => [\"sandy\",\"patrick\",\"plankton\"], name => \"bob\"}}\n\nAPI\n---\n\nthis are the direct implementations of the RFC methods::\n\n    add(Obj, Path, Val): add value at Path to Val\n    remove(Obj, Path): remove value in Path\n    replace(Obj, Path, Val): set value in Path to Val if set\n    move(Obj, FromPath, ToPath): move value from FromPath to ToPath (remove + add)\n    copy(Obj, FromPath, ToPath): copy value from FromPath to ToPath (fetch + add)\n    test(Obj, Path, Val): tests that Path contains Val, {ok, true} if equal, {ok, false} if not equal, {error, Reason} if error.\n\nextra methods::\n\n    fetch(Obj, Path): {ok, Value} if found, {error, Reason} otherwise\n    fetch(Obj, Path, Default): {ok, Value} if found, {ok, Default} otherwise\n\nLicense\n-------\n\nMPL 2.0\n"
  },
  {
    "path": "contrib/dotto/erlang.mk",
    "content": "# Copyright (c) 2013-2014, Loïc Hoguin <essen@ninenines.eu>\n#\n# Permission to use, copy, modify, and/or distribute this software for any\n# purpose with or without fee is hereby granted, provided that the above\n# copyright notice and this permission notice appear in all copies.\n#\n# THE SOFTWARE IS PROVIDED \"AS IS\" AND THE AUTHOR DISCLAIMS ALL WARRANTIES\n# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF\n# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR\n# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES\n# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN\n# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF\n# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\n\n.PHONY: all deps app rel docs tests clean distclean help\n\nERLANG_MK_VERSION = 1\n\n# Core configuration.\n\nPROJECT ?= $(notdir $(CURDIR))\nPROJECT := $(strip $(PROJECT))\n\n# Verbosity.\n\nV ?= 0\n\ngen_verbose_0 = @echo \" GEN   \" $@;\ngen_verbose = $(gen_verbose_$(V))\n\n# Core targets.\n\nall:: deps app rel\n\nclean::\n\t$(gen_verbose) rm -f erl_crash.dump\n\ndistclean:: clean\n\nhelp::\n\t@printf \"%s\\n\" \\\n\t\t\"erlang.mk (version $(ERLANG_MK_VERSION)) is distributed under the terms of the ISC License.\" \\\n\t\t\"Copyright (c) 2013-2014 Loïc Hoguin <essen@ninenines.eu>\" \\\n\t\t\"\" \\\n\t\t\"Usage: [V=1] make [target]\" \\\n\t\t\"\" \\\n\t\t\"Core targets:\" \\\n\t\t\"  all         Run deps, app and rel targets in that order\" \\\n\t\t\"  deps        Fetch dependencies (if needed) and compile them\" \\\n\t\t\"  app         Compile the project\" \\\n\t\t\"  rel         Build a release for this project, if applicable\" \\\n\t\t\"  docs        Build the documentation for this project\" \\\n\t\t\"  tests       Run the tests for this project\" \\\n\t\t\"  clean       Delete temporary and output files from most targets\" \\\n\t\t\"  distclean   Delete all temporary and output files\" \\\n\t\t\"  help        Display this help and exit\" \\\n\t\t\"\" \\\n\t\t\"The target clean only removes files that are commonly removed.\" \\\n\t\t\"Dependencies and releases are left untouched.\" \\\n\t\t\"\" \\\n\t\t\"Setting V=1 when calling make enables verbose mode.\"\n\n# Core functions.\n\nifeq ($(shell which wget 2>/dev/null | wc -l), 1)\ndefine core_http_get\n\twget --no-check-certificate -O $(1) $(2)|| rm $(1)\nendef\nelse\ndefine core_http_get\n\terl -noshell -eval 'ssl:start(), inets:start(), case httpc:request(get, {\"$(2)\", []}, [{autoredirect, true}], []) of {ok, {{_, 200, _}, _, Body}} -> case file:write_file(\"$(1)\", Body) of ok -> ok; {error, R1} -> halt(R1) end; {error, R2} -> halt(R2) end, halt(0).'\nendef\nendif\n\n# Copyright (c) 2013-2014, Loïc Hoguin <essen@ninenines.eu>\n# This file is part of erlang.mk and subject to the terms of the ISC License.\n\n.PHONY: distclean-deps distclean-pkg pkg-list pkg-search\n\n# Configuration.\n\nDEPS_DIR ?= $(CURDIR)/deps\nexport DEPS_DIR\n\nREBAR_DEPS_DIR = $(DEPS_DIR)\nexport REBAR_DEPS_DIR\n\nALL_DEPS_DIRS = $(addprefix $(DEPS_DIR)/,$(DEPS))\n\nifeq ($(filter $(DEPS_DIR),$(subst :, ,$(ERL_LIBS))),)\nifeq ($(ERL_LIBS),)\n\tERL_LIBS = $(DEPS_DIR)\nelse\n\tERL_LIBS := $(ERL_LIBS):$(DEPS_DIR)\nendif\nendif\nexport ERL_LIBS\n\nPKG_FILE2 ?= $(CURDIR)/.erlang.mk.packages.v2\nexport PKG_FILE2\n\nPKG_FILE_URL ?= https://raw.githubusercontent.com/ninenines/erlang.mk/master/packages.v2.tsv\n\n# Core targets.\n\ndeps:: $(ALL_DEPS_DIRS)\n\t@for dep in $(ALL_DEPS_DIRS) ; do \\\n\t\tif [ -f $$dep/GNUmakefile ] || [ -f $$dep/makefile ] || [ -f $$dep/Makefile ] ; then \\\n\t\t\t$(MAKE) -C $$dep ; \\\n\t\telse \\\n\t\t\techo \"include $(CURDIR)/erlang.mk\" | ERLC_OPTS=+debug_info $(MAKE) -f - -C $$dep ; \\\n\t\tfi ; \\\n\tdone\n\ndistclean:: distclean-deps distclean-pkg\n\n# Deps related targets.\n\ndefine dep_fetch\n\tif [ \"$$$$VS\" = \"git\" ]; then \\\n\t\tgit clone -n -- $$$$REPO $(DEPS_DIR)/$(1); \\\n\t\tcd $(DEPS_DIR)/$(1) && git checkout -q $$$$COMMIT; \\\n\telif [ \"$$$$VS\" = \"hg\" ]; then \\\n\t\thg clone -U $$$$REPO $(DEPS_DIR)/$(1); \\\n\t\tcd $(DEPS_DIR)/$(1) && hg update -q $$$$COMMIT; \\\n\telse \\\n\t\techo \"Unknown or invalid dependency: $(1). Please consult the erlang.mk README for instructions.\" >&2; \\\n\t\texit 78; \\\n\tfi\nendef\n\ndefine dep_target\n$(DEPS_DIR)/$(1):\n\t@mkdir -p $(DEPS_DIR)\nifeq (,$(dep_$(1)))\n\t@if [ ! -f $(PKG_FILE2) ]; then $(call core_http_get,$(PKG_FILE2),$(PKG_FILE_URL)); fi\n\t@DEPPKG=$$$$(awk 'BEGIN { FS = \"\\t\" }; $$$$1 == \"$(1)\" { print $$$$2 \" \" $$$$3 \" \" $$$$4 }' $(PKG_FILE2);); \\\n\tVS=$$$$(echo $$$$DEPPKG | cut -d \" \" -f1); \\\n\tREPO=$$$$(echo $$$$DEPPKG | cut -d \" \" -f2); \\\n\tCOMMIT=$$$$(echo $$$$DEPPKG | cut -d \" \" -f3); \\\n\t$(call dep_fetch,$(1))\nelse\n\t@VS=$(word 1,$(dep_$(1))); \\\n\tREPO=$(word 2,$(dep_$(1))); \\\n\tCOMMIT=$(word 3,$(dep_$(1))); \\\n\t$(call dep_fetch,$(1))\nendif\nendef\n\n$(foreach dep,$(DEPS),$(eval $(call dep_target,$(dep))))\n\ndistclean-deps:\n\t$(gen_verbose) rm -rf $(DEPS_DIR)\n\n# Packages related targets.\n\n$(PKG_FILE2):\n\t@$(call core_http_get,$(PKG_FILE2),$(PKG_FILE_URL))\n\npkg-list: $(PKG_FILE2)\n\t@cat $(PKG_FILE2) | awk 'BEGIN { FS = \"\\t\" }; { print \\\n\t\t\"Name:\\t\\t\" $$1 \"\\n\" \\\n\t\t\"Repository:\\t\" $$3 \"\\n\" \\\n\t\t\"Website:\\t\" $$5 \"\\n\" \\\n\t\t\"Description:\\t\" $$6 \"\\n\" }'\n\nifdef q\npkg-search: $(PKG_FILE2)\n\t@cat $(PKG_FILE2) | grep -i ${q} | awk 'BEGIN { FS = \"\\t\" }; { print \\\n\t\t\"Name:\\t\\t\" $$1 \"\\n\" \\\n\t\t\"Repository:\\t\" $$3 \"\\n\" \\\n\t\t\"Website:\\t\" $$5 \"\\n\" \\\n\t\t\"Description:\\t\" $$6 \"\\n\" }'\nelse\npkg-search:\n\t$(error Usage: make pkg-search q=STRING)\nendif\n\nifeq ($(PKG_FILE2),$(CURDIR)/.erlang.mk.packages.v2)\ndistclean-pkg:\n\t$(gen_verbose) rm -f $(PKG_FILE2)\nendif\n\nhelp::\n\t@printf \"%s\\n\" \"\" \\\n\t\t\"Package-related targets:\" \\\n\t\t\"  pkg-list              List all known packages\" \\\n\t\t\"  pkg-search q=STRING   Search for STRING in the package index\"\n\n# Copyright (c) 2013-2014, Loïc Hoguin <essen@ninenines.eu>\n# This file is part of erlang.mk and subject to the terms of the ISC License.\n\n.PHONY: clean-app\n\n# Configuration.\n\nERLC_OPTS ?= -Werror +debug_info +warn_export_all +warn_export_vars \\\n\t+warn_shadow_vars +warn_obsolete_guard # +bin_opt_info +warn_missing_spec\nCOMPILE_FIRST ?=\nCOMPILE_FIRST_PATHS = $(addprefix src/,$(addsuffix .erl,$(COMPILE_FIRST)))\n\n# Verbosity.\n\nappsrc_verbose_0 = @echo \" APP   \" $(PROJECT).app.src;\nappsrc_verbose = $(appsrc_verbose_$(V))\n\nerlc_verbose_0 = @echo \" ERLC  \" $(filter %.erl %.core,$(?F));\nerlc_verbose = $(erlc_verbose_$(V))\n\nxyrl_verbose_0 = @echo \" XYRL  \" $(filter %.xrl %.yrl,$(?F));\nxyrl_verbose = $(xyrl_verbose_$(V))\n\n# Core targets.\n\napp:: erlc-include ebin/$(PROJECT).app\n\t$(eval MODULES := $(shell find ebin -type f -name \\*.beam \\\n\t\t| sed \"s/ebin\\//'/;s/\\.beam/',/\" | sed '$$s/.$$//'))\n\t@if [ -z \"$$(grep -E '^[^%]*{modules,' src/$(PROJECT).app.src)\" ]; then \\\n\t\techo \"Empty modules entry not found in $(PROJECT).app.src. Please consult the erlang.mk README for instructions.\" >&2; \\\n\t\texit 1; \\\n\tfi\n\t$(eval GITDESCRIBE := $(shell git describe --dirty --abbrev=7 --tags --always --first-parent 2>/dev/null || true))\n\t$(appsrc_verbose) cat src/$(PROJECT).app.src \\\n\t\t| sed \"s/{modules,[[:space:]]*\\[\\]}/{modules, \\[$(MODULES)\\]}/\" \\\n\t\t| sed \"s/{id,[[:space:]]*\\\"git\\\"}/{id, \\\"$(GITDESCRIBE)\\\"}/\" \\\n\t\t> ebin/$(PROJECT).app\n\ndefine compile_erl\n\t$(erlc_verbose) erlc -v $(ERLC_OPTS) -o ebin/ \\\n\t\t-pa ebin/ -I include/ $(COMPILE_FIRST_PATHS) $(1)\nendef\n\ndefine compile_xyrl\n\t$(xyrl_verbose) erlc -v -o ebin/ $(1)\n\t$(xyrl_verbose) erlc $(ERLC_OPTS) -o ebin/ ebin/*.erl\n\t@rm ebin/*.erl\nendef\n\nifneq ($(wildcard src/),)\nebin/$(PROJECT).app::\n\t@mkdir -p ebin/\n\nebin/$(PROJECT).app:: $(shell find src -type f -name \\*.erl) \\\n\t\t$(shell find src -type f -name \\*.core)\n\t$(if $(strip $?),$(call compile_erl,$?))\n\nebin/$(PROJECT).app:: $(shell find src -type f -name \\*.xrl) \\\n\t\t$(shell find src -type f -name \\*.yrl)\n\t$(if $(strip $?),$(call compile_xyrl,$?))\nendif\n\nclean:: clean-app\n\n# Extra targets.\n\nerlc-include:\n\t-@if [ -d ebin/ ]; then \\\n\t\tfind include/ src/ -type f -name \\*.hrl -newer ebin -exec touch $(shell find src/ -type f -name \"*.erl\") \\; 2>/dev/null || printf ''; \\\n\tfi\n\nclean-app:\n\t$(gen_verbose) rm -rf ebin/\n\n# Copyright (c) 2014, Loïc Hoguin <essen@ninenines.eu>\n# This file is part of erlang.mk and subject to the terms of the ISC License.\n\n.PHONY: bootstrap bootstrap-lib bootstrap-rel new list-templates\n\n# Core targets.\n\nhelp::\n\t@printf \"%s\\n\" \"\" \\\n\t\t\"Bootstrap targets:\" \\\n\t\t\"  bootstrap          Generate a skeleton of an OTP application\" \\\n\t\t\"  bootstrap-lib      Generate a skeleton of an OTP library\" \\\n\t\t\"  bootstrap-rel      Generate the files needed to build a release\" \\\n\t\t\"  new t=TPL n=NAME   Generate a module NAME based on the template TPL\" \\\n\t\t\"  list-templates     List available templates\"\n\n# Bootstrap templates.\n\nbs_appsrc = \"{application, $(PROJECT), [\" \\\n\t\"\t{description, \\\"\\\"},\" \\\n\t\"\t{vsn, \\\"0.1.0\\\"},\" \\\n\t\"\t{id, \\\"git\\\"},\" \\\n\t\"\t{modules, []},\" \\\n\t\"\t{registered, []},\" \\\n\t\"\t{applications, [\" \\\n\t\"\t\tkernel,\" \\\n\t\"\t\tstdlib\" \\\n\t\"\t]},\" \\\n\t\"\t{mod, {$(PROJECT)_app, []}},\" \\\n\t\"\t{env, []}\" \\\n\t\"]}.\"\nbs_appsrc_lib = \"{application, $(PROJECT), [\" \\\n\t\"\t{description, \\\"\\\"},\" \\\n\t\"\t{vsn, \\\"0.1.0\\\"},\" \\\n\t\"\t{id, \\\"git\\\"},\" \\\n\t\"\t{modules, []},\" \\\n\t\"\t{registered, []},\" \\\n\t\"\t{applications, [\" \\\n\t\"\t\tkernel,\" \\\n\t\"\t\tstdlib\" \\\n\t\"\t]}\" \\\n\t\"]}.\"\nbs_Makefile = \"PROJECT = $(PROJECT)\" \\\n\t\"include erlang.mk\"\nbs_app = \"-module($(PROJECT)_app).\" \\\n\t\"-behaviour(application).\" \\\n\t\"\" \\\n\t\"-export([start/2]).\" \\\n\t\"-export([stop/1]).\" \\\n\t\"\" \\\n\t\"start(_Type, _Args) ->\" \\\n\t\"\t$(PROJECT)_sup:start_link().\" \\\n\t\"\" \\\n\t\"stop(_State) ->\" \\\n\t\"\tok.\"\nbs_relx_config = \"{release, {$(PROJECT)_release, \\\"1\\\"}, [$(PROJECT)]}.\" \\\n\t\"{extended_start_script, true}.\" \\\n\t\"{sys_config, \\\"rel/sys.config\\\"}.\" \\\n\t\"{vm_args, \\\"rel/vm.args\\\"}.\"\nbs_sys_config = \"[\" \\\n\t\"].\"\nbs_vm_args = \"-name $(PROJECT)@127.0.0.1\" \\\n\t\"-setcookie $(PROJECT)\" \\\n\t\"-heart\"\n# Normal templates.\ntpl_supervisor = \"-module($(n)).\" \\\n\t\"-behaviour(supervisor).\" \\\n\t\"\" \\\n\t\"-export([start_link/0]).\" \\\n\t\"-export([init/1]).\" \\\n\t\"\" \\\n\t\"start_link() ->\" \\\n\t\"\tsupervisor:start_link({local, ?MODULE}, ?MODULE, []).\" \\\n\t\"\" \\\n\t\"init([]) ->\" \\\n\t\"\tProcs = [],\" \\\n\t\"\t{ok, {{one_for_one, 1, 5}, Procs}}.\"\ntpl_gen_server = \"-module($(n)).\" \\\n\t\"-behaviour(gen_server).\" \\\n\t\"\" \\\n\t\"%% API.\" \\\n\t\"-export([start_link/0]).\" \\\n\t\"\" \\\n\t\"%% gen_server.\" \\\n\t\"-export([init/1]).\" \\\n\t\"-export([handle_call/3]).\" \\\n\t\"-export([handle_cast/2]).\" \\\n\t\"-export([handle_info/2]).\" \\\n\t\"-export([terminate/2]).\" \\\n\t\"-export([code_change/3]).\" \\\n\t\"\" \\\n\t\"-record(state, {\" \\\n\t\"}).\" \\\n\t\"\" \\\n\t\"%% API.\" \\\n\t\"\" \\\n\t\"-spec start_link() -> {ok, pid()}.\" \\\n\t\"start_link() ->\" \\\n\t\"\tgen_server:start_link(?MODULE, [], []).\" \\\n\t\"\" \\\n\t\"%% gen_server.\" \\\n\t\"\" \\\n\t\"init([]) ->\" \\\n\t\"\t{ok, \\#state{}}.\" \\\n\t\"\" \\\n\t\"handle_call(_Request, _From, State) ->\" \\\n\t\"\t{reply, ignored, State}.\" \\\n\t\"\" \\\n\t\"handle_cast(_Msg, State) ->\" \\\n\t\"\t{noreply, State}.\" \\\n\t\"\" \\\n\t\"handle_info(_Info, State) ->\" \\\n\t\"\t{noreply, State}.\" \\\n\t\"\" \\\n\t\"terminate(_Reason, _State) ->\" \\\n\t\"\tok.\" \\\n\t\"\" \\\n\t\"code_change(_OldVsn, State, _Extra) ->\" \\\n\t\"\t{ok, State}.\"\ntpl_cowboy_http = \"-module($(n)).\" \\\n\t\"-behaviour(cowboy_http_handler).\" \\\n\t\"\" \\\n\t\"-export([init/3]).\" \\\n\t\"-export([handle/2]).\" \\\n\t\"-export([terminate/3]).\" \\\n\t\"\" \\\n\t\"-record(state, {\" \\\n\t\"}).\" \\\n\t\"\" \\\n\t\"init(_, Req, _Opts) ->\" \\\n\t\"\t{ok, Req, \\#state{}}.\" \\\n\t\"\" \\\n\t\"handle(Req, State=\\#state{}) ->\" \\\n\t\"\t{ok, Req2} = cowboy_req:reply(200, Req),\" \\\n\t\"\t{ok, Req2, State}.\" \\\n\t\"\" \\\n\t\"terminate(_Reason, _Req, _State) ->\" \\\n\t\"\tok.\"\ntpl_cowboy_loop = \"-module($(n)).\" \\\n\t\"-behaviour(cowboy_loop_handler).\" \\\n\t\"\" \\\n\t\"-export([init/3]).\" \\\n\t\"-export([info/3]).\" \\\n\t\"-export([terminate/3]).\" \\\n\t\"\" \\\n\t\"-record(state, {\" \\\n\t\"}).\" \\\n\t\"\" \\\n\t\"init(_, Req, _Opts) ->\" \\\n\t\"\t{loop, Req, \\#state{}, 5000, hibernate}.\" \\\n\t\"\" \\\n\t\"info(_Info, Req, State) ->\" \\\n\t\"\t{loop, Req, State, hibernate}.\" \\\n\t\"\" \\\n\t\"terminate(_Reason, _Req, _State) ->\" \\\n\t\"\tok.\"\ntpl_cowboy_rest = \"-module($(n)).\" \\\n\t\"\" \\\n\t\"-export([init/3]).\" \\\n\t\"-export([content_types_provided/2]).\" \\\n\t\"-export([get_html/2]).\" \\\n\t\"\" \\\n\t\"init(_, _Req, _Opts) ->\" \\\n\t\"\t{upgrade, protocol, cowboy_rest}.\" \\\n\t\"\" \\\n\t\"content_types_provided(Req, State) ->\" \\\n\t\"\t{[{{<<\\\"text\\\">>, <<\\\"html\\\">>, '*'}, get_html}], Req, State}.\" \\\n\t\"\" \\\n\t\"get_html(Req, State) ->\" \\\n\t\"\t{<<\\\"<html><body>This is REST!</body></html>\\\">>, Req, State}.\"\ntpl_cowboy_ws = \"-module($(n)).\" \\\n\t\"-behaviour(cowboy_websocket_handler).\" \\\n\t\"\" \\\n\t\"-export([init/3]).\" \\\n\t\"-export([websocket_init/3]).\" \\\n\t\"-export([websocket_handle/3]).\" \\\n\t\"-export([websocket_info/3]).\" \\\n\t\"-export([websocket_terminate/3]).\" \\\n\t\"\" \\\n\t\"-record(state, {\" \\\n\t\"}).\" \\\n\t\"\" \\\n\t\"init(_, _, _) ->\" \\\n\t\"\t{upgrade, protocol, cowboy_websocket}.\" \\\n\t\"\" \\\n\t\"websocket_init(_, Req, _Opts) ->\" \\\n\t\"\tReq2 = cowboy_req:compact(Req),\" \\\n\t\"\t{ok, Req2, \\#state{}}.\" \\\n\t\"\" \\\n\t\"websocket_handle({text, Data}, Req, State) ->\" \\\n\t\"\t{reply, {text, Data}, Req, State};\" \\\n\t\"websocket_handle({binary, Data}, Req, State) ->\" \\\n\t\"\t{reply, {binary, Data}, Req, State};\" \\\n\t\"websocket_handle(_Frame, Req, State) ->\" \\\n\t\"\t{ok, Req, State}.\" \\\n\t\"\" \\\n\t\"websocket_info(_Info, Req, State) ->\" \\\n\t\"\t{ok, Req, State}.\" \\\n\t\"\" \\\n\t\"websocket_terminate(_Reason, _Req, _State) ->\" \\\n\t\"\tok.\"\ntpl_ranch_protocol = \"-module($(n)).\" \\\n\t\"-behaviour(ranch_protocol).\" \\\n\t\"\" \\\n\t\"-export([start_link/4]).\" \\\n\t\"-export([init/4]).\" \\\n\t\"\" \\\n\t\"-type opts() :: [].\" \\\n\t\"-export_type([opts/0]).\" \\\n\t\"\" \\\n\t\"-record(state, {\" \\\n\t\"\tsocket :: inet:socket(),\" \\\n\t\"\ttransport :: module()\" \\\n\t\"}).\" \\\n\t\"\" \\\n\t\"start_link(Ref, Socket, Transport, Opts) ->\" \\\n\t\"\tPid = spawn_link(?MODULE, init, [Ref, Socket, Transport, Opts]),\" \\\n\t\"\t{ok, Pid}.\" \\\n\t\"\" \\\n\t\"-spec init(ranch:ref(), inet:socket(), module(), opts()) -> ok.\" \\\n\t\"init(Ref, Socket, Transport, _Opts) ->\" \\\n\t\"\tok = ranch:accept_ack(Ref),\" \\\n\t\"\tloop(\\#state{socket=Socket, transport=Transport}).\" \\\n\t\"\" \\\n\t\"loop(State) ->\" \\\n\t\"\tloop(State).\"\n\n# Plugin-specific targets.\n\nbootstrap:\nifneq ($(wildcard src/),)\n\t$(error Error: src/ directory already exists)\nendif\n\t@printf \"%s\\n\" $(bs_Makefile) > Makefile\n\t@mkdir src/\n\t@printf \"%s\\n\" $(bs_appsrc) > src/$(PROJECT).app.src\n\t@printf \"%s\\n\" $(bs_app) > src/$(PROJECT)_app.erl\n\t$(eval n := $(PROJECT)_sup)\n\t@printf \"%s\\n\" $(tpl_supervisor) > src/$(PROJECT)_sup.erl\n\nbootstrap-lib:\nifneq ($(wildcard src/),)\n\t$(error Error: src/ directory already exists)\nendif\n\t@printf \"%s\\n\" $(bs_Makefile) > Makefile\n\t@mkdir src/\n\t@printf \"%s\\n\" $(bs_appsrc_lib) > src/$(PROJECT).app.src\n\nbootstrap-rel:\nifneq ($(wildcard relx.config),)\n\t$(error Error: relx.config already exists)\nendif\nifneq ($(wildcard rel/),)\n\t$(error Error: rel/ directory already exists)\nendif\n\t@printf \"%s\\n\" $(bs_relx_config) > relx.config\n\t@mkdir rel/\n\t@printf \"%s\\n\" $(bs_sys_config) > rel/sys.config\n\t@printf \"%s\\n\" $(bs_vm_args) > rel/vm.args\n\nnew:\nifeq ($(wildcard src/),)\n\t$(error Error: src/ directory does not exist)\nendif\nifndef t\n\t$(error Usage: make new t=TEMPLATE n=NAME)\nendif\nifndef tpl_$(t)\n\t$(error Unknown template)\nendif\nifndef n\n\t$(error Usage: make new t=TEMPLATE n=NAME)\nendif\n\t@printf \"%s\\n\" $(tpl_$(t)) > src/$(n).erl\n\nlist-templates:\n\t@echo Available templates: $(sort $(patsubst tpl_%,%,$(filter tpl_%,$(.VARIABLES))))\n\n# Copyright (c) 2013-2014, Loïc Hoguin <essen@ninenines.eu>\n# This file is part of erlang.mk and subject to the terms of the ISC License.\n\n.PHONY: build-ct-deps build-ct-suites tests-ct clean-ct distclean-ct\n\n# Configuration.\n\nCT_OPTS ?=\nifneq ($(wildcard test/),)\n\tCT_SUITES ?= $(sort $(subst _SUITE.erl,,$(shell find test -type f -name \\*_SUITE.erl -exec basename {} \\;)))\nelse\n\tCT_SUITES ?=\nendif\n\nTEST_ERLC_OPTS ?= +debug_info +warn_export_vars +warn_shadow_vars +warn_obsolete_guard\nTEST_ERLC_OPTS += -DTEST=1 -DEXTRA=1 +'{parse_transform, eunit_autoexport}'\n\n# Core targets.\n\ntests:: tests-ct\n\nclean:: clean-ct\n\ndistclean:: distclean-ct\n\nhelp::\n\t@printf \"%s\\n\" \"\" \\\n\t\t\"All your common_test suites have their associated targets.\" \\\n\t\t\"A suite named http_SUITE can be ran using the ct-http target.\"\n\n# Plugin-specific targets.\n\nALL_TEST_DEPS_DIRS = $(addprefix $(DEPS_DIR)/,$(TEST_DEPS))\n\nCT_RUN = ct_run \\\n\t-no_auto_compile \\\n\t-noshell \\\n\t-pa $(realpath ebin) $(DEPS_DIR)/*/ebin \\\n\t-dir test \\\n\t-logdir logs\n\n$(foreach dep,$(TEST_DEPS),$(eval $(call dep_target,$(dep))))\n\nbuild-ct-deps: $(ALL_TEST_DEPS_DIRS)\n\t@for dep in $(ALL_TEST_DEPS_DIRS) ; do $(MAKE) -C $$dep; done\n\nbuild-ct-suites: build-ct-deps\n\t$(gen_verbose) erlc -v $(TEST_ERLC_OPTS) -I include/ -o test/ \\\n\t\t$(wildcard test/*.erl test/*/*.erl) -pa ebin/\n\ntests-ct: ERLC_OPTS = $(TEST_ERLC_OPTS)\ntests-ct: clean deps app build-ct-suites\n\t@if [ -d \"test\" ] ; \\\n\tthen \\\n\t\tmkdir -p logs/ ; \\\n\t\t$(CT_RUN) -suite $(addsuffix _SUITE,$(CT_SUITES)) $(CT_OPTS) ; \\\n\tfi\n\t$(gen_verbose) rm -f test/*.beam\n\ndefine ct_suite_target\nct-$(1): ERLC_OPTS = $(TEST_ERLC_OPTS)\nct-$(1): clean deps app build-ct-suites\n\t@if [ -d \"test\" ] ; \\\n\tthen \\\n\t\tmkdir -p logs/ ; \\\n\t\t$(CT_RUN) -suite $(addsuffix _SUITE,$(1)) $(CT_OPTS) ; \\\n\tfi\n\t$(gen_verbose) rm -f test/*.beam\nendef\n\n$(foreach test,$(CT_SUITES),$(eval $(call ct_suite_target,$(test))))\n\nclean-ct:\n\t$(gen_verbose) rm -rf test/*.beam\n\ndistclean-ct:\n\t$(gen_verbose) rm -rf logs/\n\n# Copyright (c) 2013-2014, Loïc Hoguin <essen@ninenines.eu>\n# This file is part of erlang.mk and subject to the terms of the ISC License.\n\n.PHONY: plt distclean-plt dialyze\n\n# Configuration.\n\nDIALYZER_PLT ?= $(CURDIR)/.$(PROJECT).plt\nexport DIALYZER_PLT\n\nPLT_APPS ?=\nDIALYZER_DIRS ?= --src -r src\nDIALYZER_OPTS ?= -Werror_handling -Wrace_conditions \\\n\t-Wunmatched_returns # -Wunderspecs\n\n# Core targets.\n\ndistclean:: distclean-plt\n\nhelp::\n\t@printf \"%s\\n\" \"\" \\\n\t\t\"Dialyzer targets:\" \\\n\t\t\"  plt         Build a PLT file for this project\" \\\n\t\t\"  dialyze     Analyze the project using Dialyzer\"\n\n# Plugin-specific targets.\n\n$(DIALYZER_PLT): deps app\n\t@dialyzer --build_plt --apps erts kernel stdlib $(PLT_APPS) $(ALL_DEPS_DIRS)\n\nplt: $(DIALYZER_PLT)\n\ndistclean-plt:\n\t$(gen_verbose) rm -f $(DIALYZER_PLT)\n\nifneq ($(wildcard $(DIALYZER_PLT)),)\ndialyze:\nelse\ndialyze: $(DIALYZER_PLT)\nendif\n\t@dialyzer --no_native $(DIALYZER_DIRS) $(DIALYZER_OPTS)\n\n# Copyright (c) 2013-2014, Loïc Hoguin <essen@ninenines.eu>\n# This file is part of erlang.mk and subject to the terms of the ISC License.\n\n.PHONY: distclean-edoc\n\n# Configuration.\n\nEDOC_OPTS ?=\n\n# Core targets.\n\ndocs:: distclean-edoc\n\t$(gen_verbose) erl -noshell \\\n\t\t-eval 'edoc:application($(PROJECT), \".\", [$(EDOC_OPTS)]), init:stop().'\n\ndistclean:: distclean-edoc\n\n# Plugin-specific targets.\n\ndistclean-edoc:\n\t$(gen_verbose) rm -f doc/*.css doc/*.html doc/*.png doc/edoc-info\n\n# Copyright (c) 2014, Juan Facorro <juan@inaka.net>\n# This file is part of erlang.mk and subject to the terms of the ISC License.\n\n.PHONY: elvis distclean-elvis\n\n# Configuration.\n\nELVIS_CONFIG ?= $(CURDIR)/elvis.config\n\nELVIS ?= $(CURDIR)/elvis\nexport ELVIS\n\nELVIS_URL ?= https://github.com/inaka/elvis/releases/download/0.2.3/elvis\nELVIS_CONFIG_URL ?= https://github.com/inaka/elvis/releases/download/0.2.3/elvis.config\nELVIS_OPTS ?=\n\n# Core targets.\n\nhelp::\n\t@printf \"%s\\n\" \"\" \\\n\t\t\"Elvis targets:\" \\\n\t\t\"  elvis       Run Elvis using the local elvis.config or download the default otherwise\"\n\nifneq ($(wildcard $(ELVIS_CONFIG)),)\nrel:: distclean-elvis\nendif\n\ndistclean:: distclean-elvis\n\n# Plugin-specific targets.\n\n$(ELVIS):\n\t@$(call core_http_get,$(ELVIS_CONFIG),$(ELVIS_CONFIG_URL))\n\t@$(call core_http_get,$(ELVIS),$(ELVIS_URL))\n\t@chmod +x $(ELVIS)\n\nelvis: $(ELVIS)\n\t@$(ELVIS) rock -c $(ELVIS_CONFIG) $(ELVIS_OPTS)\n\ndistclean-elvis:\n\t$(gen_verbose) rm -rf $(ELVIS)\n\n# Copyright (c) 2013-2014, Loïc Hoguin <essen@ninenines.eu>\n# This file is part of erlang.mk and subject to the terms of the ISC License.\n\n# Verbosity.\n\ndtl_verbose_0 = @echo \" DTL   \" $(filter %.dtl,$(?F));\ndtl_verbose = $(dtl_verbose_$(V))\n\n# Core targets.\n\ndefine compile_erlydtl\n\t$(dtl_verbose) erl -noshell -pa ebin/ $(DEPS_DIR)/erlydtl/ebin/ -eval ' \\\n\t\tCompile = fun(F) -> \\\n\t\t\tModule = list_to_atom( \\\n\t\t\t\tstring:to_lower(filename:basename(F, \".dtl\")) ++ \"_dtl\"), \\\n\t\t\terlydtl:compile(F, Module, [{out_dir, \"ebin/\"}]) \\\n\t\tend, \\\n\t\t_ = [Compile(F) || F <- string:tokens(\"$(1)\", \" \")], \\\n\t\tinit:stop()'\nendef\n\nifneq ($(wildcard src/),)\nebin/$(PROJECT).app:: $(shell find templates -type f -name \\*.dtl 2>/dev/null)\n\t$(if $(strip $?),$(call compile_erlydtl,$?))\nendif\n\n# Copyright (c) 2013-2014, Loïc Hoguin <essen@ninenines.eu>\n# This file is part of erlang.mk and subject to the terms of the ISC License.\n\n.PHONY: relx-rel distclean-relx-rel distclean-relx\n\n# Configuration.\n\nRELX_CONFIG ?= $(CURDIR)/relx.config\n\nRELX ?= $(CURDIR)/relx\nexport RELX\n\nRELX_URL ?= https://github.com/erlware/relx/releases/download/v1.0.2/relx\nRELX_OPTS ?=\nRELX_OUTPUT_DIR ?= _rel\n\nifeq ($(firstword $(RELX_OPTS)),-o)\n\tRELX_OUTPUT_DIR = $(word 2,$(RELX_OPTS))\nelse\n\tRELX_OPTS += -o $(RELX_OUTPUT_DIR)\nendif\n\n# Core targets.\n\nifneq ($(wildcard $(RELX_CONFIG)),)\nrel:: distclean-relx-rel relx-rel\nendif\n\ndistclean:: distclean-relx-rel distclean-relx\n\n# Plugin-specific targets.\n\ndefine relx_fetch\n\t$(call core_http_get,$(RELX),$(RELX_URL))\n\tchmod +x $(RELX)\nendef\n\n$(RELX):\n\t@$(call relx_fetch)\n\nrelx-rel: $(RELX)\n\t@$(RELX) -c $(RELX_CONFIG) $(RELX_OPTS)\n\ndistclean-relx-rel:\n\t$(gen_verbose) rm -rf $(RELX_OUTPUT_DIR)\n\ndistclean-relx:\n\t$(gen_verbose) rm -rf $(RELX)\n\n# Copyright (c) 2014, M Robert Martin <rob@version2beta.com>\n# This file is contributed to erlang.mk and subject to the terms of the ISC License.\n\n.PHONY: shell\n\n# Configuration.\n\nSHELL_PATH ?= -pa ../$(PROJECT)/ebin $(DEPS_DIR)/*/ebin\nSHELL_OPTS ?=\n\nALL_SHELL_DEPS_DIRS = $(addprefix $(DEPS_DIR)/,$(SHELL_DEPS))\n\n# Core targets\n\nhelp::\n\t@printf \"%s\\n\" \"\" \\\n\t\t\"Shell targets:\" \\\n\t\t\"  shell              Run an erlang shell with SHELL_OPTS or reasonable default\"\n\n# Plugin-specific targets.\n\n$(foreach dep,$(SHELL_DEPS),$(eval $(call dep_target,$(dep))))\n\nbuild-shell-deps: $(ALL_SHELL_DEPS_DIRS)\n\t@for dep in $(ALL_SHELL_DEPS_DIRS) ; do $(MAKE) -C $$dep ; done\n\nshell: build-shell-deps\n\t$(gen_verbose) erl $(SHELL_PATH) $(SHELL_OPTS)\n"
  },
  {
    "path": "contrib/dotto/src/dotto.app.src",
    "content": "{application, dotto, [\n\t{description, \"\"},\n\t{vsn, \"0.1.0\"},\n\t{id, \"git\"},\n\t{modules, []},\n\t{registered, []},\n\t{applications, [\n\t\tkernel,\n\t\tstdlib\n\t]}\n]}.\n"
  },
  {
    "path": "contrib/dotto/src/dotto.erl",
    "content": "-module(dotto).\n\n%% Hacky guard check for dicts since it relies on dicts internal\n%% structure which might change in the future...\n-define(IS_DICT(X), is_tuple(X) andalso element(1, X) =:= dict).\n\n-export([apply/2, add/3, remove/2, replace/3, move/3, copy/3,\n         test/3,\n         fetch/2, fetch/3]).\n\napply(Ops, Obj) when is_list(Ops) ->\n    do_apply(Ops, Obj, []);\n\napply({add, Path, Val}, Obj) -> add(Obj, Path, Val);\napply({remove, Path}, Obj) -> remove(Obj, Path);\napply({replace, Path, Val}, Obj) -> replace(Obj, Path, Val);\napply({move, FromPath, ToPath}, Obj) -> move(Obj, FromPath, ToPath);\napply({copy, FromPath, ToPath}, Obj) -> copy(Obj, FromPath, ToPath);\napply({test, Path, Val}, Obj) -> test(Obj, Path, Val).\n\ndo_apply([], Obj, []) ->\n    {ok, Obj};\ndo_apply([], Obj, Errors) ->\n    {error, Obj, Errors};\n\ndo_apply([{test, _Path, _Val}=Op|Ops], Obj, Errors) ->\n    case dotto:apply(Op, Obj) of\n        {ok, true} -> do_apply(Ops, Obj, Errors);\n        {ok, false} -> do_apply(Ops, Obj, [{error, {testfail, Op}}|Errors]);\n        Other -> do_apply(Ops, Obj, [Other|Errors])\n    end;\n\ndo_apply([Op|Ops], Obj, Errors) ->\n    case dotto:apply(Op, Obj) of\n        {ok, NewObj} -> do_apply(Ops, NewObj, Errors);\n        {error, Error} -> do_apply(Ops, Obj, [Error|Errors])\n    end.\n\n% This will only match if you try to replace the whole document with an empty path\n% When strictly following the RFC, Val is allowed to be anything. However,\n% if it is not a valid JSON than the result is not a JSON as well which we want\n% to prevent in our use case.\nadd(_Obj, [], Val) when ?IS_DICT(Val)->\n    {ok, Val};\nadd(Obj, [], Val) ->\n    {error, {cantset, Obj, [], Val}};\nadd(Obj, [Field], Val) ->\n    add_(Obj, Field, Val);\n\nadd(Obj, [Field|Fields], Val) ->\n    case get_(Obj, Field) of\n        {ok, FieldObj} ->\n            case add(FieldObj, Fields, Val) of\n                % XXX what happens if a \"-\" is in the middle of the path?\n                {ok, NewVal} -> set_(Obj, Field, NewVal);\n                Other -> Other\n            end;\n        notfound -> {error, {notfound, Obj, Field}};\n        Other -> Other\n    end.\n\n% calling remove with an empty path (i.e the documents root) removes all\n% elements. This is equivalent to a new, empty object\nremove(Obj, []) ->\n    {ok, dict:new()};\n\nremove(Obj, [Field]) ->\n    case get_(Obj, Field) of\n        {ok, _FieldObj} ->\n            del_(Obj, Field);\n        notfound -> {error, {notfound, Obj, Field}};\n        Other -> Other\n    end;\n\nremove(Obj, [Field|Fields]) ->\n    case get_(Obj, Field) of\n        {ok, FieldObj} ->\n            case remove(FieldObj, Fields) of\n                {ok, NewVal} -> set_(Obj, Field, NewVal);\n                Other -> Other\n            end;\n        notfound -> {error, {notfound, Obj, Field}};\n        Other -> Other\n    end.\n\n% this will only match if you try to replace the whole document with an empty path\n% However, this is only allowed if the resulting document is a valid JSON as well\nreplace(_Obj, [], Val) when ?IS_DICT(Val)->\n    {ok, Val};\nreplace(Obj, [], Val) ->\n    {error, {cantset, Obj, [], Val}};\nreplace(Obj, [Field], Val) ->\n    case get_(Obj, Field) of\n        {ok, _FieldObj} ->\n            set_(Obj, Field, Val);\n        notfound -> {error, {notfound, Obj, Field}};\n        Other -> Other\n    end;\n\nreplace(Obj, [Field|Fields], Val) ->\n    case get_(Obj, Field) of\n        {ok, FieldObj} ->\n            case replace(FieldObj, Fields, Val) of\n                {ok, NewVal} -> set_(Obj, Field, NewVal);\n                Other -> Other\n            end;\n        notfound -> {error, {notfound, Obj, Field}};\n        Other -> Other\n    end.\n\n% XXX The \"from\" location MUST NOT be a proper prefix of the \"path\"\n% location; i.e., a location cannot be moved into one of its children.\nmove(Obj, FromPath, ToPath) ->\n    case fetch(Obj, FromPath) of\n        {ok, Value} ->\n            case remove(Obj, FromPath) of\n                {ok, Obj1} ->\n                    add(Obj1, ToPath, Value);\n                Error -> Error\n            end;\n        Error -> Error\n    end.\n\ncopy(Obj, FromPath, ToPath) ->\n    case fetch(Obj, FromPath) of\n        {ok, Value} ->\n            add(Obj, ToPath, Value);\n        Error -> Error\n    end.\n\ntest(Obj, Path, Val) ->\n    case fetch(Obj, Path) of\n        {ok, Value} ->\n            {ok, Value =:= Val};\n        Other -> Other\n    end.\n\n% non RFC 6902 functions\n\nfetch(Obj, []) ->\n    {ok, Obj};\n\nfetch(Obj, [Field|Fields]) ->\n    case get_(Obj, Field) of\n        {ok, FieldObj} ->\n            fetch(FieldObj, Fields);\n        notfound -> {error, {notfound, Obj, Field}};\n        Other -> Other\n    end.\n\nfetch(Obj, Path, Default) ->\n    case fetch(Obj, Path) of\n        {ok, _} = Result -> Result;\n        {error, _} -> {ok, Default}\n    end.\n\n% private api\n\n\nadd_(Obj, Field, Value) when ?IS_DICT(Obj) ->\n    {ok, set_element(Obj, Field, Value)};\n\nadd_(Obj, <<\"-\">>, Value) when is_list(Obj) ->\n    {ok, Obj ++ [Value]};\n\nadd_(Obj, Field, Value) when is_list(Obj) andalso is_integer(Field) andalso\n                             length(Obj) >= Field andalso Field >= 0->\n    {L1, L2} = lists:split(Field, Obj),\n    {ok, L1 ++ [Value|L2]};\n\nadd_(Obj, Field, Value) ->\n    {error, {cantset, Obj, Field, Value}}.\n\nset_(Obj, Field, Value) when ?IS_DICT(Obj) ->\n    {ok, set_element(Obj, Field, Value)};\n\nset_(Obj, Field, Value) when is_list(Obj) andalso is_integer(Field)\n                             andalso length(Obj) >= Field andalso Field >= 0->\n    Result = case lists:split(Field, Obj) of\n                 {[], [_|T]} -> [Value] ++ T;\n                 {H1, [_|T]} -> H1 ++ [Value] ++ T;\n                 % TODO: should droplast(H1)? I think not since it's 0 based\n                 {H1, []} -> H1 ++ [Value]\n             end,\n    {ok, Result};\n\n\nset_(Obj, Field, Value) ->\n    {error, {cantset, Obj, Field, Value}}.\n\ndel_(Obj, Field) when ?IS_DICT(Obj) ->\n    {ok, dict:erase(Field, Obj)};\n\n% XXX not sure if this case is in RFC 6902\ndel_(Obj, <<\"-\">>) when is_list(Obj) ->\n    {ok, lists:droplast(Obj)};\n\ndel_(Obj, Field) when is_list(Obj) andalso is_integer(Field)\n                      andalso length(Obj) >= Field andalso Field >= 0->\n    Result = case lists:split(Field, Obj) of\n                 {[], [_|T]} -> T;\n                 {H1, [_|T]} -> H1 ++ T;\n                 {H1, []} -> lists:droplast(H1)\n             end,\n    {ok, Result};\n\ndel_(Obj, Field) ->\n    {error, {cantremove, Obj, Field}}.\n\nget_(Obj, Field) when ?IS_DICT(Obj) ->\n    case dict:find(Field, Obj) of\n        {ok, Value} -> {ok, Value};\n        error -> notfound\n    end;\n\nget_(Obj, <<\"-\">>=Index) when is_list(Obj) ->\n    {error, {invalidindex, Obj, Index}};\n\nget_(Obj, Field) when is_list(Obj) andalso is_integer(Field) ->\n    InsideList = Field >= 0 andalso Field  < length(Obj),\n    if InsideList -> {ok, lists:nth(Field + 1, Obj)};\n       true -> notfound\n    end;\n\nget_(_Obj, _Field) ->\n    notfound.\n\nset_element(Obj, Field, NewValue) when ?IS_DICT(Obj) ->\n    dict:update(Field, fun (_Old) -> NewValue end, NewValue, Obj).\n"
  },
  {
    "path": "contrib/dotto/test/dotto_SUITE.erl",
    "content": "-module(dotto_SUITE).\n-compile(export_all).\n\n-define(EMPTY, dict:new()).\n-define(DICT(X), dict:from_list(X)).\n-define(DICT(X,Y), ?DICT([{X,Y}])).\n\nall() ->\n    [add_to_empty_list, add_to_empty_dict, add_start_list, add_middle_list,\n     override_dict_value, add_level_1_dict, add_level_1_existing_dict,\n     add_level_2_list, add_level_2_not_list, add_level_2_not_dict,\n\n     replace_to_empty_list, replace_to_empty_dict, replace_start_list,\n     replace_middle_list, replace_end_list, replace_dict_value,\n     replace_level_1_dict, replace_level_1_inexistent_dict, replace_level_2_list,\n     replace_level_2_not_list, replace_level_2_not_dict,\n    \n     fetch_default,\n\n     remove_empty_list, remove_item_list, remove_item_dict].\n\nadd_to_empty_list(_) ->\n    {ok, [42]} = dotto:add([], [<<\"-\">>], 42).\n\nadd_to_empty_dict(_) ->\n    Expected = ?DICT(foo, 42),\n    {ok, Expected} = dotto:add(?EMPTY, [foo], 42).\n\nadd_start_list(_) ->\n    {ok, [42, 1]} = dotto:add([1], [0], 42).\n\nadd_middle_list(_) ->\n    {ok, [1, 42, 2]} = dotto:add([1, 2], [1], 42).\n\noverride_dict_value(_) ->\n    Start = ?DICT(foo, 1),\n    Expected = ?DICT(foo, 42),\n    {ok, Expected} = dotto:add(Start, [foo], 42).\n\nadd_level_1_dict(_) ->\n    Start = ?DICT(foo, ?EMPTY),\n    Expected = ?DICT(foo, ?DICT(bar, 42)),\n    {ok, Expected} = dotto:add(Start, [foo, bar], 42).\n\nadd_level_1_existing_dict(_) ->\n    Start = ?DICT(foo, ?DICT(bar, 12)),\n    Expected = ?DICT(foo, ?DICT(bar, 42)),\n    {ok, Expected} = dotto:add(Start, [foo, bar], 42).\n\nadd_level_2_list(_) ->\n    Start = ?DICT(foo, ?DICT(bar, [1,2])),\n    Expected = ?DICT(foo, ?DICT(bar, [1,42,2])),\n    {ok, Expected} = dotto:add(Start, [foo, bar, 1], 42).\n\nadd_level_2_not_list(_) ->\n    Start = ?DICT(foo, ?DICT(bar, notalist)),\n    {error, {cantset, notalist, 1, 42}} = dotto:add(Start, [foo, bar, 1], 42).\n\nadd_level_2_not_dict(_) ->\n    Start = ?DICT(foo, ?DICT(bar, notadict)),\n    {error, {cantset, notadict, baz, 42}}=dotto:add(Start, [foo, bar, baz], 42).\n\n%% replace\n\n\nreplace_to_empty_list(_) ->\n    {error, {invalidindex, [], <<\"-\">>}} = dotto:replace([], [<<\"-\">>], 42).\n\nreplace_to_empty_dict(_) ->\n    Empty = ?EMPTY,\n    {error, {notfound, Empty, foo}} = dotto:replace(Empty, [foo], 42).\n\nreplace_start_list(_) ->\n    {ok, [42]} = dotto:replace([1], [0], 42),\n    {ok, [42, 2]} = dotto:replace([1, 2], [0], 42).\n\nreplace_middle_list(_) ->\n    {ok, [1, 42, 3]} = dotto:replace([1, 2, 3], [1], 42).\n\nreplace_end_list(_) ->\n    {ok, [42]} = dotto:replace([1], [0], 42),\n    {ok, [1, 42]} = dotto:replace([1, 2], [1], 42),\n    {ok, [1, 2, 42]} = dotto:replace([1, 2, 3], [2], 42).\n\nreplace_dict_value(_) ->\n    Start = ?DICT(foo, 1),\n    Expected = ?DICT(foo, 42),\n    {ok, Expected} = dotto:replace(Start, [foo], 42).\n\nreplace_level_1_dict(_) ->\n    Start = ?DICT(foo, ?DICT(bar, 1)),\n    Expected = ?DICT(foo, ?DICT(bar, 42)),\n    {ok, Expected} = dotto:replace(Start, [foo, bar], 42).\n\nreplace_level_1_inexistent_dict(_) ->\n    Start = ?DICT(foo, ?EMPTY),\n    Nested = ?EMPTY,\n    {error, {notfound, Nested, bar}} = dotto:replace(Start, [foo, bar], 42).\n\nreplace_level_2_list(_) ->\n    Start = ?DICT(foo, ?DICT(bar, [1,2,3])),\n    Expected = ?DICT(foo, ?DICT(bar, [1,42,3])),\n    {ok, Expected} = dotto:replace(Start, [foo, bar, 1], 42).\n\nreplace_level_2_not_list(_) ->\n    Start = ?DICT(foo, ?DICT(bar, notalist)),\n    {error, {notfound, notalist, 1}} = dotto:replace(Start, [foo, bar, 1], 42).\n\nreplace_level_2_not_dict(_) ->\n    Start = ?DICT(foo, ?DICT(bar, notadict)),\n    {error, {notfound, notadict, baz}} = dotto:replace(Start, [foo, bar, baz], 42).\n\n%% remove\n\nremove_empty_list(_) ->\n    {error, {notfound, [], 1}} = dotto:remove([], [1]),\n    {error, {notfound, [], 0}} = dotto:remove([], [0]).\n\nremove_item_list(_) ->\n    {ok, []} = dotto:remove([42], [0]),\n    {ok, [1]} = dotto:remove([1, 42], [1]),\n    {ok, [1, 2]} = dotto:remove([1, 42, 2], [1]),\n    {ok, [42, 2]} = dotto:remove([1, 42, 2], [0]),\n    {ok, [1, [2, 3]]} = dotto:remove([1, [2, 42, 3]], [1, 1]),\n    {error, {notfound, [], 0}} = dotto:remove([], [0]),\n    {error, {notfound, [10, 11], 2}} = dotto:remove([10, 11], [2]).\n\nremove_item_dict(_) ->\n    A = ?DICT([{a,1},{b,2}]),\n    B = ?DICT(a, 1),\n    C = ?EMPTY,\n    {ok, B} = dotto:remove(A, [b]),\n    {ok, C} = dotto:remove(B, [a]),\n\n    Sub = ?DICT(c, 2), \n    Start = ?DICT(a, ?DICT([{b, 1}, {c, 2}])),\n    Expected = ?DICT(a, Sub),\n    {ok, Expected} = dotto:remove(Start, [a, b]),\n    {error, {notfound, C, a}} = dotto:remove(C, [a]),\n    {error, {notfound, C, a}} = dotto:remove(C, [a, b]),\n    {error, {notfound, Sub, b}} = dotto:remove(Expected, [a, b]).\n\n% fetch\n\nfetch_default(_) ->\n    Obj1 = ?DICT(a, 1),\n    SubObj1 = ?DICT(c, 2),\n    Obj2 = ?DICT([{a, 1}, {b, SubObj1}]),\n\n    {ok, 1} = dotto:fetch(Obj1, [a]),\n    {error, {notfound, Obj1, b}} = dotto:fetch(Obj1, [b]),\n    {ok, 2} = dotto:fetch(Obj1, [b], 2),\n\n    {ok, 2} = dotto:fetch(Obj2, [b, c]),\n    {error, {notfound, SubObj1, d}} = dotto:fetch(Obj2, [b, d]),\n    {ok, 3} = dotto:fetch(Obj2, [b, d], 3).\n\n"
  },
  {
    "path": "contrib/init.d/.gitignore",
    "content": "/scalaris\n/scalaris-monitor\n/scalaris-first\n/scalaris-first-monitor\n"
  },
  {
    "path": "contrib/init.d/initd-first.conf",
    "content": "SCALARIS_NODE=\"node\"\nSCALARIS_ADDITIONAL_PARAMETERS=\"-p 14195 -y 8000 -m -t first\"\n"
  },
  {
    "path": "contrib/init.d/initd.conf",
    "content": "SCALARIS_NODE=\"node1\"\nSCALARIS_ADDITIONAL_PARAMETERS=\"-p 14196 -y 8001 -t joining\"\n"
  },
  {
    "path": "contrib/init.d/scalaris-monitor.in",
    "content": "#!/bin/bash\n\n### BEGIN INIT INFO\n# Provides:          scalaris-monitor\n# Required-Start:    $network $remote_fs scalaris\n# Required-Stop:     $network $remote_fs scalaris\n# Default-Start:     3 5\n# Default-Stop:\n# Short-Description: JMX monitor for a Scalaris node\n# Description:       Provides a small Java service exposing monitoring stats via JMX for a Scalaris node. http://code.google.com/p/scalaris/\n### END INIT INFO\n\n# Source function library.\nif [ -e /lib/lsb/init-functions ]; then\n    . /lib/lsb/init-functions\nelse\n    . /etc/init.d/functions\nfi\n\nprefix=@prefix@\nexec_prefix=@exec_prefix@\nSUDO=@SUDO@\nRUNUSER=@RUNUSER@\n\n# Source config.\nif [ -e @sysconfdir@/scalaris/initd.conf ]; then\n    . @sysconfdir@/scalaris/initd.conf\nfi\n\nSCALARIS_NODE=${SCALARIS_NODE-\"node\"}\n\nSCALARIS_USER=scalaris\n\nPID_DS=\"@localstatedir@/run/scalaris-monitor_${SCALARIS_NODE}.pid\"\n\nLOG_DS=\"@localstatedir@/log/scalaris/initd-monitor_${SCALARIS_NODE}.log\"\n\nSCALARISMON=\"@bindir@/scalaris --jvmopts \\\"-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=14193 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dscalaris.client.name=scalaris_monitor\\\" -jmx ${SCALARIS_NODE}\"\n\n# For SELinux we need to use 'runuser' not 'sudo'\nif [ -n \"$RUNUSER\" -a -x \"$RUNUSER\" ]; then\n    SU=\"$RUNUSER\"\n    SU_CMD=\"$RUNUSER $SCALARIS_USER -s /bin/bash -c\"\nelse\n    SU=\"$SUDO\"\n    SU_CMD=\"$SUDO -u $SCALARIS_USER /bin/bash -c\"\nfi\n\nstart() {\n    if [ -f \"$PID_DS\" ]; then\n        PROCPID=`cat $PID_DS`\n        if [ -e \"/proc/$PROCPID\" ];then\n            echo \"Scalaris monitor for \\\"${SCALARIS_NODE}\\\" already started\"\n            return 0\n        fi\n    fi\n\n    echo >> \"$LOG_DS\"\n    date >> \"$LOG_DS\"\n    echo -e \"Starting Scalaris monitor for \\\"${SCALARIS_NODE}\\\"...\\n\\n\" >> \"$LOG_DS\"\n    echo -n \"Starting Scalaris monitor for \\\"${SCALARIS_NODE}\\\"... \"\n    $SU_CMD \"$SCALARISMON\" >> \"$LOG_DS\" 2>&1 &\n    PROCPID=$!\n    echo $PROCPID > \"$PID_DS\"\n    sleep 1s\n\n    if [ -e \"/proc/$PROCPID\" ]; then\n        echo \"success\"\n    else\n        echo \"failed\"\n        return 1\n    fi\n\n    return 0\n}\n\nstop() {\n    result=0\n    if [ -f \"$PID_DS\" ]; then\n        echo -n \"Stopping Scalaris monitor for \\\"${SCALARIS_NODE}\\\"... \"\n        killproc -p \"$PID_DS\" \"$SU\"\n        result=$?\n        if [ $result -eq 0 ]; then\n            rm -f \"$PID_DS\"\n            echo \"success\"\n        else\n            echo \"failed\"\n        fi\n    else\n        echo \"Scalaris monitor for \\\"${SCALARIS_NODE}\\\" has not been running\"\n    fi\n\n    return $result\n}\n\nstatus() {\n    if [ -f \"$PID_DS\" ]; then\n        PROCPID=`cat $PID_DS`\n        if [ ! -e \"/proc/$PROCPID\" ];then\n            echo \"Scalaris monitor for \\\"${SCALARIS_NODE}\\\" has crashed\"\n            return 1 # program is dead and /var/run pid file exists\n        else\n            echo \"Scalaris monitor for \\\"${SCALARIS_NODE}\\\" is running\"\n            return 0\n        fi\n    else\n        echo \"Scalaris monitor for \\\"${SCALARIS_NODE}\\\" is not running\"\n        return 3 # program is not running\n    fi\n}\n\nrestart() {\n    stop && sleep 1 && start\n    return $?\n}\n\n# See how we were called.\ncase \"$1\" in\n    start)\n        start\n        result=$?\n        ;;\n    stop)\n        stop\n        result=$?\n        ;;\n    restart)\n        restart\n        result=$?\n        ;;\n    try-restart)\n        ## Stop the service and if this succeeds (i.e. the\n        ## service was running before), start it again.\n        if [ `$0 status >/dev/null` ]; then\n          $0 restart\n          result=$?\n        else\n          result=0\n        fi\n        ;;\n    reload)\n        result=3\n        ;;\n    force-reload)\n        restart\n        result=$?\n        ;;\n    status)\n        status\n        result=$?\n        ;;\n    *)\n        echo -e \"Usage: $0 {start|stop|restart|try-restart|reload|force-reload|status}\\n\"\n        result=1\n        ;;\nesac\n\nexit $result\n"
  },
  {
    "path": "contrib/init.d/scalaris.in",
    "content": "#!/bin/bash\n\n### BEGIN INIT INFO\n# Provides:          scalaris\n# Required-Start:    $network $remote_fs\n# Required-Stop:     $network $remote_fs\n# Default-Start:     3 5\n# Default-Stop:\n# Short-Description: Scalaris node\n# Description:       Default Scalaris node. http://code.google.com/p/scalaris/\n### END INIT INFO\n\n# Source function library.\nif [ -e /lib/lsb/init-functions ]; then\n    . /lib/lsb/init-functions\nelse\n    . /etc/init.d/functions\nfi\n\nprefix=@prefix@\nexec_prefix=@exec_prefix@\ndatarootdir=@datarootdir@\nSCREEN=@SCREEN@\nSUDO=@SUDO@\nRUNUSER=@RUNUSER@\n\n# Source config.\nif [ -e @sysconfdir@/scalaris/initd.conf ]; then\n    . @sysconfdir@/scalaris/initd.conf\nfi\n\nSCALARIS_NODE=${SCALARIS_NODE-\"node\"}\n\nSCALARIS_USER=scalaris\n\nPID_DS=\"@localstatedir@/run/scalaris_${SCALARIS_NODE}.pid\"\n\nLOG_DS=\"@localstatedir@/log/scalaris/initd_${SCALARIS_NODE}.log\"\n\nSCALARISCTL=\"@bindir@/scalarisctl -n ${SCALARIS_NODE} -d --screen -l @localstatedir@/log/scalaris ${SCALARIS_ADDITIONAL_PARAMETERS}\"\n\n# For SELinux we need to use 'runuser' not 'sudo'\nif [ -n \"$RUNUSER\" -a -x \"$RUNUSER\" ]; then\n    SU=\"$RUNUSER\"\n    SU_CMD=\"$RUNUSER $SCALARIS_USER -s /bin/bash -c\"\nelse\n    SU=\"$SUDO\"\n    SU_CMD=\"$SUDO -u $SCALARIS_USER -H /bin/bash -c\"\nfi\n\nstart() {\n    if [ -f \"$PID_DS\" ]; then\n        PROCPID=`cat \"$PID_DS\"`\n        if [ -e \"/proc/$PROCPID\" ];then\n            echo \"Scalaris node \\\"${SCALARIS_NODE}\\\" already started\"\n            return 0\n        fi\n    fi\n\n    echo >> \"$LOG_DS\"\n    date >> \"$LOG_DS\"\n    echo -e \"Starting Scalaris node \\\"${SCALARIS_NODE}\\\"...\\n\\n\" >> \"$LOG_DS\"\n    echo -n \"Starting Scalaris node \\\"${SCALARIS_NODE}\\\"... \"\n    $SU_CMD \"$SCALARISCTL start\" >> \"$LOG_DS\" 2>&1\n    if [ $? -ne 0 ]; then\n        echo \"failed\"\n        return 1 # generic or unspecified error\n    fi\n    sleep 1s\n    $SU_CMD \"$SCREEN -ls | grep \\\"scalaris_${SCALARIS_NODE}\\\" | cut -d'.' -f1 | head -n1 | xargs echo\" > \"$PID_DS\"\n\n    if [ -e \"/proc/$PROCPID\" ]; then\n        echo \"success\"\n    else\n        echo \"failed\"\n        return 1\n    fi\n\n    return 0\n}\n\ngstop() {\n    result=0\n    if [ -f \"$PID_DS\" ]; then\n        echo -n \"Shutting down Scalaris node \\\"${SCALARIS_NODE}\\\"... \"\n        PROCPID=`cat \"$PID_DS\"`\n        if [ -e \"/proc/$PROCPID\" ];then\n            $SU_CMD \"$SCALARISCTL gstop\"\n            result=$?\n        else\n            # not running anymore\n            result=0\n        fi\n        if [ $result -eq 0 ]; then\n            rm -f \"$PID_DS\"\n            echo \"success\"\n        else\n            echo \"failed\"\n        fi\n    else\n        echo \"Scalaris node \\\"${SCALARIS_NODE}\\\" has not been running\"\n    fi\n\n    return $result\n}\n\nstop() {\n    result=0\n    if [ -f \"$PID_DS\" ]; then\n        echo -n \"Killing Scalaris node \\\"${SCALARIS_NODE}\\\"... \"\n        killproc -p \"$PID_DS\" \"$SCREEN\"\n        result=$?\n        if [ $result -eq 0 ]; then\n            rm -f \"$PID_DS\"\n            echo \"success\"\n        else\n            echo \"failed\"\n        fi\n    else\n        echo \"Scalaris node \\\"${SCALARIS_NODE}\\\" has not been running\"\n    fi\n\n    return $result\n}\n\nstatus() {\n    if [ -f \"$PID_DS\" ]; then\n        PROCPID=`cat \"$PID_DS\"`\n        if [ ! -e \"/proc/$PROCPID\" ];then\n            echo \"Scalaris node \\\"${SCALARIS_NODE}\\\" has crashed\"\n            return 1 # program is dead and /var/run pid file exists\n        else\n            echo -n \"Checking Scalaris node \\\"${SCALARIS_NODE}\\\"... \"\n            $SU_CMD \"$SCALARISCTL status\"\n            if [ $? -ne 0 ]; then\n                echo \"probably crashed\"\n                return 4 # program or service status is unknown\n            fi\n            echo \"running\"\n            return 0\n        fi\n    else\n        echo \"Scalaris node \\\"${SCALARIS_NODE}\\\" is not running\"\n        return 3 # program is not running\n    fi\n}\n\nrestart() {\n    gstop && sleep 1 && start\n    return $?\n}\n\n# See how we were called.\ncase \"$1\" in\n    start)\n        start\n        result=$?\n        ;;\n    stop)\n        gstop\n        result=$?\n        ;;\n    kill)\n        stop\n        result=$?\n        ;;\n    restart)\n        restart\n        result=$?\n        ;;\n    try-restart)\n        ## Stop the service and if this succeeds (i.e. the\n        ## service was running before), start it again.\n        if [ `$0 status >/dev/null` ]; then\n          $0 restart\n          result=$?\n        else\n          result=0\n        fi\n        ;;\n    reload)\n        result=3\n        ;;\n    force-reload)\n        restart\n        result=$?\n        ;;\n    status)\n        status\n        result=$?\n        ;;\n    *)\n        echo -e \"Usage: $0 {start|stop|kill|restart|try-restart|reload|force-reload|status}\\n\"\n        result=1\n        ;;\nesac\n\nexit $result\n"
  },
  {
    "path": "contrib/log4erl/API.txt",
    "content": "NOTE:\n=====\nPlease be informed that the API below as of now is not stable and any of the functions/parameters below\nmay be changed without prior notice.\n\nFurthermore, you can find appenders and their configurations in 'Appenders_API.txt' file included in release.\n\n**> log4erl:conf(File) -> ok | {error, E}\n  File :: string()\n\n  File is the name of the configuration file. Please check CONFIGURATION.txt for the format of this file.\n\n**> log4erl:add_logger(Name) -> ok | {error, E}\n  Name :: atom() name of the logger \n\n  Example:\n  log4erl:add_logger(chat_logger)\n  This will create a new logger with name 'chat_logger\"\n\n**> log4erl:change_log_level(Level) -> ok\n    log4erl:change_log_level(Logger, Level) -> ok\n  Level::atom() = {all, debug, info, error, warn, fatal, none}\n  Logger::atom()\n\n  This will change log level for default logger or named logger to the level\n  specified.\n\n  Example:\n  log4erl:change_log_level(info). %% This will change level of default logger to info\n  log4erl:change_log_level(test_log, warn) %% This will change level of test_log logger to warn\n\n**> log4erl:change_level(Appender, Level) -> ok\n    log4erl:change_level(Logger, Appender, Level) -> ok\n    Logger :: atom()\n    Appender :: atom()\n    Level :: atom()\n\n  @since version 0.8.4\n\n  This will change log level for a specific appender, as opposed to change_log_level/2,3 which changes\n  level for all appenders in a logger.\n\n  Example:\n  log4erl:change_level(appender1, all).\n\n**> log4erl:change_format(Appender, Format) -> ok\n    log4erl:change_format(Logger, Appender, Format) -> ok\n  Appender :: atom()\n  Logger :: atom()\n  Format :: string()\n\n  @since version 0.8.3\n\n  This will change the output format to the specified Format. Format is a pattern string similar to\n  PatternLayout in Log4j. patterns is an arbitrary string with specifiers (proceeded by '%').\n  Possible specifiers are below:\n\t   d - output date (2008-3-7)\n\t   j - output date (2008-03-07)\n\t   t - time (2:13:9)\n\t   T - time (02:28:01.811637)\n\t   y - year in YY format (08)\n\t   Y - year in YYYY format (2008)\n\t   M - month (2)\n\t   b - short name of month (Feb)\n\t   B - long name of month (February)\n\t   D - day\n\t   h - hour\n\t   m - minute\n\t   s - second\n\t   i - milli-seconds\n\t   l - the actual log message\n\t   I - ISO format with universal GMT time (equivilant to \"%jT%TZ\").\n\t   S - ISO format with local time and time zone offset\n\t   Z - timezone (+04:00)\n\t   L - log level\n\t   n - new line\n\t   % - the percentage sign (%)\n\n  Example: \n  log4erl:change_format(file1, \"%j %T [%L] %l%n\").\n  Will result in the following output (on log4erl:warn(\"hello\"))\n  \n  27-10-2008 15:28:59,98621 [warn] hello\n\n**> log4erl:log(Level, Log) -> ok\n    Level :: atom()\n    Log :: string()\n\n  This will log the text Log to the default logger with level Level.\n\n  Example:\n  log4erl:log(warn, \"Hello there\").\n  log4erl:log(test_level, \"Hello there\").\n\n**> log4erl:log(Level, Log, Data) -> ok\n    Level :: atom()\n    Log :: string()\n    Data :: list()\n\n  This will log the text Log to the default logger with level Level and\n  will use Data to format the log text.\n\n  Example:\n  log4erl:log(info, \"received message ~p\", [Msg]).\n\n**> log4erl:log(Logger, Level, Log, Data) -> ok\n    Logger :: atom()\n    Level :: atom()\n    Log :: string()\n    Data :: list()\n\n  This will log the (Log, Data) to Logger with level Level\n\n  Example:\n  log4erl:log(chat_log, debug, \"user entered chat text: ~p\", [Chat]).\n\n**> log4erl:Level(Log) -> ok\n    log4erl:Level(Log, Data) -> ok\n    log4erl:Level(Logger, Data) -> ok\n    log4erl:Level(Logger, Log, Data) -> ok\n\n  Level :: warn | info | error | fatal | debug\n  Logger :: atom()\n  Log :: string()\n  Data :: list()\n\n  Exmaple:\n  log4erl:info(\"This is an info msg\").\n  log4erl:warn(\"Received error ~p\",[Msg]).\n  log4erl:fatal(chat_log, \"exception occured\").\n  log4erl:debug(chat_log, \"message received is ~p\", [Msg]).\n  log4erl:error(\"Error\").\n\n**> log4erl:error_logger_handler() -> ok\n    log4erl:error_logger_handler(Mappings) -> ok\n   where\n   Mappings :: [Mapping]\n   Mapping :: {Elevels, level}\n   ELevels :: error | info_msg | warning_msg | error_report | info_report | warning_report\n\n   Mappings basically is a list of mapping between error_logger log levels and log4erl log levels.\n   Currently, report messages are not handled by this error_logger handler. The default Mappings are:\n\t      {error=error, info_msg=info, warning_msg=warn,\n\t      error_report=error, info_report=info,\n\t      warning_report=warn}\n\n   Example:\n   log4erl:error_logger_handler([{error, fatal}, {info_msg, info}]).\n\n   which treats all error messages coming from error_logger as fatal and all info_msg\n   messages coming from error_logger as info.\n   \n"
  },
  {
    "path": "contrib/log4erl/Appenders_API.txt",
    "content": "**> log4erl:add_file_appender(Name, Spec) -> ok | {error, E}\n  Name :: atom() name of the appender. This value will be used\n  to uniquely references this file appender and can be used to \n  change log level and format of the appended file. This appender\n  will be added to the default_logger\n\n  Spec :: tuple() of the form\n       {LogDir, LogFileName, {size, Size}, NumOfRotations, Suffix, LogLevel}\n  or\n       {LogDir, LogFileName, {size, Size}, NumOfRotations, Suffix, LogLevel, Format}\n  This tuple defines the appender's attributes.\n  If Format is not specified, the default format is \"[%L] %l%n\".\n\n  Example:\n  log4erl:add_file_appender(chat_room1, {\"../logs\", \"room1\", {size, 100000}, 4, \"txt\", warn}).\n  This will directs log4erl to create a file appender and add it to the default\n  logger. all log messages towards default logger will be written to file\n  \"../logs/room1.txt\". The file will have a size limit of 100000, after which\n  the file will be rotated for 4 times. The log level will be warn, which means\n  info & debug messages will not be written. Format is a list of specifiers. The meanin of these \n  specifiers can be found in description of log4erl:change_format/2 below.\n\n  After suffeciantly long time, the directory \"../logs\" will look like this:\n  $> ls -l\n  -rw-rw-r--  1 ahmed ahmed 103845 Jun 25 11:41 logger1_2.elog\n  -rw-rw-r--  1 ahmed ahmed 102095 Jun 25 11:41 logger1_3.elog\n  -rw-rw-r--  1 ahmed ahmed 106435 Jun 25 11:41 logger1_4.elog\n  -rw-rw-r--  1 ahmed ahmed 103390 Jun 25 11:41 logger1_1.elog\n  -rw-rw-r--  1 ahmed ahmed   7385 Jun 25 11:41 logger1.elog  \n\n**> log4erl:add_file_appender(Logger, Appender, SpecFile) -> ok\n  Logger :: atom()\n  Appender :: atom()\n\n  This will create a new appender and associate spec in Spec or in SpecFile to it. Spec and\n  content of SpecFile are the same as in add_file_appender/2\n\n  Example:\n  log4erl:add_file_appender(chat_log, file_logger, \"../priv/chat_logs.conf\").\n  This will add another logger with the name \"chat_log\" and use configuration in\n  the file \"../priv/chat_logs.conf\" for it. To write to this log, you need to specify\n  the name of the logger.\n\n**> log4erl:add_console_appender(Appender, Spec) -> ok\n    log4erl:add_console_appender(Logger, Appender, Spec) -> ok\n  Logger :: atom()\n  Appender :: atom()\n  Spec :: tuple()\n\n  This will create a new console appender to either Logger or default logger and associate spec in Spec to it.\n  Spec is in the form {Level, Format}, which means exactly as it is in file_appender Spec\n\n  Example:\n  log4erl:add_console_appender(cmd_logs, {info, \"%j %T [%L] %l%n\"}).\n\n**> log4erl:add_smtp_appender(Appender, Spec) -> ok\n    log4erl:add_smtp_appender(Logger, Appender, Spec) -> ok\n  Logger :: atom()\n  Appender :: atom()\n  Spec :: tuple() | list()\n\n  @since version 0.8.4\n\n  This will create a new smtp appender to either Logger of default logger and initialize it to spec in Spec or\n  file named Spec if Spec is a list. Spec is in the below form. MsgFormat is the format of the body of the email\n  has the same meaning as the format of Format in add_file_logger/2,3.\n       {Level, Server, Auth, Msg}\n\t       where:\n\t       Level = atom()\n\t       Server = {Ip, Port} | {Ip} | Ip\n\t       Auth = {Username, Passwd} | no_auth\n\t       Msg = {From, To, Title, MsgFormat} | {To, Title, MsgFormat} | {To, MsgFormat}\n\n  Example:\n  log4erl:add_smpt_appender(appender1, {warn, {\"smtp.domain.com\"}, {\"name\",\"secret\"},\n\t\t\t\t       {\"My_app\", \"alarm@domain.com\", \"Alarm log\", \"%L - %j %T %n%l%n\"}}).\n\n  This will create an smtp_appender called appender1 with level 'warn'. The appender will connect to\n  \"smtp.domain.com\" email server with default SMTP port (25) and authenticate using username \"name\" and\n  password \"secret\". The message will have 'From' field of \"My_app\", will be sent to 'alarm@domain.com',\n  have a title of \"Alarm log\" and the body of the email will be formatted according to the MsgFormat\n  \"%L - %j %T %n %l%n\". An email sent using this appender will be as following:\n  \n  From: <My_app>\n  To: <alarm@domain.com>\n  Title: <Alarm log>\n  Body:\n  <warn - 01-03-2009 12:21:09,818534\n  test smtp appender>\n\n**> log4erl:add_syslog_appender(Appender, Spec) -> ok\n    log4erl:add_syslog_appender(Logger, Appender, Spec) -> ok\n  Logger :: atom()\n  Appender :: atom()\n  Spec :: tuple() | list()\n\n  @since version 0.8.5\n\n  This will create a new syslog appender to either Logger of default logger and initialize it to spec in Spec or\n  file named Spec if Spec is a list. Spec is in the below form. MsgFormat is the format of the message and it\n  has the same meaning as the format of Format in add_file_logger/2,3.\n\n  Note: In for syslog_appender to work, you need to enable syslogd to accept network messages. In linux, you\n  need to use '-r' switch when running syslogd.\n\n  Spec =\n       {Level, Fac} |\n       {Level, Fac, Host} |\n       {Level, Fac, Host, Format} |\n       {Level, Fac, Host, Port, Format}\n\t       where:\n\t       Level = atom()\n\t       Fac = atom()\n\t       Host = string() | tuple()\n\t       Port = integer()\n\t       Format = string()\n\n  Example:\n  log4erl:add_syslog_appender(appender1, {warn, user, \"syslog_srvr\", \"(%L) - %S %l%n\"}}).\n\n  This will create a syslog_appender called appender1 with level 'warn' and uses 'user' facility.\n  The appender will connect to host \"syslog_srvr\" with default port (514). The messages will be\n  formatted according to the format \"%L - %j %T %n %l%n\".\n\n  Default host is \"localhost\".\n  Default port is 514.\n  Default Format is the same as default format of file_appender\n\n**> log4erl:add_file_appender(Logger, Appender, SpecFile) -> ok\n  Logger :: atom()\n  Appender :: atom()\n\n  This will create a new appender and associate spec in Spec or in SpecFile to it. Spec and\n  content of SpecFile are the same as in add_file_appender/2\n\n  Example:\n  log4erl:add_file_appender(chat_log, file_logger, \"../priv/chat_logs.conf\").\n  This will add another logger with the name \"chat_log\" and use configuration in\n  the file \"../priv/chat_logs.conf\" for it. To write to this log, you need to specify\n  the name of the logger.\n\n**> log4erl:add_xml_appender(Appender, Spec) -> ok\n    log4erl:add_xml_appender(Logger, Appender, Spec) -> ok\n  Logger :: atom()\n  Appender :: atom()\n  Spec :: tuple()\n\n  @since version 0.8.6\n\n  This will create a new XML Appender with configuration specified in Spec where Spec is\n  {Dir, Fname, {Type, Max}, Rot, Suf, Level} | \n  {Dir, Fname, {Type, Max}, Rot, Suf, Level, XmlSpecs} \n\n  All tuple elements are the same as in file_appender except for XmlSpecs. XmlSpecs is as follows\n\n  XmlSpecs :: [XmlSpec]\n  XmlSpec :: {Name, Format, Type}\n  Name :: atom() | string()\n  Format :: string()\n  Type :: att | elem\n\n  Each XmlSpec item represent an attribute or element in xml record for a log. For example:\n  {level, \"%L\", att} means that this record is going to have an attribute named 'level' with\n  value \"%L\", which is the level of the message. Therefore, the XmlSpec mentioned in the example\n  will result in a record like\n       <log ... level=\"warn\"..>..</log>\n  However, XmlSpec {level, \"%L\", elem} will result in the the following:\n       <log ...>...<level>warn</level>...</log>\n\n  All attributes and elements are pre-proccessed to escape XML (i.e. '<' replaced by '&lt;')\n\n  You cannot provide XmlSpecs in file-based configuration yet and default XmlSpecs will be used.\n  If you needed to change XmlSpecs, you'll have to add_xml_appender/2,3 manually.\n\n  The default XmlSpecs if not provided is: \n      [{level, \"%L\", att},\n       {date, \"%j\", att},\n       {time, \"%T\", att},\n       {message, \"%l\", elem}]\n\n"
  },
  {
    "path": "contrib/log4erl/CHANGELOG.txt",
    "content": "log4erl 0.9.0:\n==============\n*\n\nlog4erl 0.8.6:\n==============\n* Added xml_appender for outputing logs to xml files\n* Changed default behaviour to use sync logging (helps to eleminate\n  crash due to many processes due to high load)\n* file_appender renames files when rotating instead of copying\n* syslog_appender add missing facilities\n* Updated leex to fix a critical bug\n* Added error_logger_log4erl_h.erl which is a handler for error_logger\n  Makes it possible to forward error_logger logs to log4erl.\n\nlog4erl 0.8.5:\n==============\n* Added new file configuration (log4erl:conf(File)).\n* Added syslog_appender.\n* Added support for ISO standard format to appenders (%I, %S & %Z specifiers).\n* Changed %d & %j to output the date in YYYY-MM-DD format according to ISO standard format\n\nlog4erl 0.8.4:\n==============\n* Added smtp_appender (new functions add_smtp_appender/2,3)\n* Added change_level/2,3 for changing level of specific Appender\n* Immproved feedback in case of errors\n* Seperated API documentation into 2 files: API.txt & Appenders_API.txt.\n  The first includes general functions and the other includes appenders\n  specific API, such as file appenders and SMTP appenders API.\n* Created a github.com repository (http://github.com/ahmednawras/log4erl/).\n\nlog4erl 0.8.3:\n==============\n* Added console_appender\n* Added more modifiers to the possible pattern of formats (milliseconds, short/long name of month)\n* Added an extra atom in the tuple argument for add_file_appender/2,3 for specifying format\n* Fixed a bug when rotation. (_02)\n\nlog4erl 0.8.2:\n==============\n* Fixed issues with log4erl process not part of the supervisor\n* Added support for simple log formatters (similar to Layouts in Log4J)\n* Added change_format/2,3\n* Added change_log_level/1,2\n* Added get_appenders/0,1\n\nKnown issues:\n- When file_appender crashes, the format is lost and log4erl will use the\n  default format\n\nlog4erl 0.8.1:\n==============\n* Fixed unnecessary formatting of date/time for file_appender\n* Added add_file_appender/2, add_file_appender/3\n* Added add_dummy_appender/2, add_dummy_appender/3\n\nlog4erl 0.8:\n============\n* Added the notion of appenders in order to allow different appenders\nfor loggers and changed API to reflect this\n* Fixed restarting issues when appenders crash\n* Added dummy_appender for testing multiple appenders\n* Added get_appenders/0,1 function\n\nlog4erl 0.7.1:\n==============\n* Changed default behaviour of new file_logger to append in order to preserve previous logs.\n* Changed file_logger_guard.erl to logger_guard.erl because this module will be useful once more\n  logger types are added (e.g. SNMP loggers, Syslog loggers...).\n* Added a function to change log level of loggers during run-time.\n* moved utility functions in file_logger.erl to log4erl_utils.\n* Added log_manager.erl in order to easily allow for more logger types\n"
  },
  {
    "path": "contrib/log4erl/CONFIGURATION.txt",
    "content": "\t\t\t\t\tConfiguration Guide:\n\t\t\t\t\t====================\n\nTOC:\n====\n* Introduction\n* Loggers\n  - Example\n* Appenders\n  - common properties\n  - file_appender\n  - console_appender\n  - smtp_appender\n  - syslog_appender\n\nIntroduction:\n=============\nThis document explains the format of log4erl configuarion file. The configuration file is simple\nand easy to write. At any place in the file, any thing you type after '%' is a comment.\nYou can find a sample configuration in the file 'log4erl.conf' under 'priv' directory.\n\n* Loggers:\n==========\n It contains a list of logs with the format below, where\n<name> is any name you'd like to call it. <name> can later on be used with any of the log messages\nin log4erl API (e.g. log4erl:info/2,3,4). If no value for <name> exists or value of \"default\" suplied,\nthen the elements inside '{' and '}' will be applied to the default logger.\n\nlogger [<name>] {\n       ...\n}\n\nExample:\n--------\nlogger application_logger{\n       ...\n}\n\n%% default logger\nlogger {\n       ...\n}\n\n* Appenders:\n============\nInside each logger, there can be 1 or more appenders. These appenders can be \nadded inside loggers according to the format below.\n\n<appender_type> <name> {\n\t...\n}\n\nExample:\n--------\nfile_appender file1{\n\t...\n}\n\ninsider appenders' block, you can add properties for that appender in the format 'property=value'. \nMultiple properties are seperated by a comman ','Each appender type has different sets of properties,\nwhich are detailed below.\n\ncommon properties:\n------------------\nlevel = <Level>           => level of log (e.g. warn)\nformat = <F>\t\t  => format of the output (look at 'Appenders.txt')\n\nfile_appender:\n--------------\ndir = <Dir>\t\t  => directory of output (e.g. /var/log/my_app)\nfile = <File>\t\t  => name of the log file (e.g. my_app_log)\ntype = <Type>             => either size or time. Only size is implemented currently\nmax = <Max>\t\t  => Maximum size of each rotation\nsuffix = <Suf>\t\t  => Suffix of the log file (e.g. log)\nrotation = <R>\t\t  => number of rotations before over-writing log files\n\nconsole_appender:\n-----------------\nNothing more than common properties.\n\nsmtp_appender:\n--------------\nip = <IP>\t\t => ip of the SMTP server\nport = <Port>\t\t => SMTP prot [Optional]\nno_auth = true|false\t => if specified, no authentication is performed even if \n\t\t\t    username or password below is provided\nusername = <U>\t\t => SMTP username\npassword = <P>\t\t => SMTP password\nfrom = <From>\t\t => value of the From field [Optional]\nto = <To>\t\t => email to send to\ntitle = <T>\t\t => title of email [Optional]\nmsg = <Format>\t\t => format of the email message [Optional]\n\nsyslog_appender:\n----------------\nfacility = <F>\t\t => Facility to be used (e.g. ftp)\nhost = <H>\t\t => Host to send syslog messages to [Optional]\nport = <P>\t\t => syslog port [Optioanl]\n\n"
  },
  {
    "path": "contrib/log4erl/LICENSE.txt",
    "content": "                          MOZILLA PUBLIC LICENSE\n                                Version 1.1\n\n                              ---------------\n\n1. Definitions.\n\n     1.0.1. \"Commercial Use\" means distribution or otherwise making the\n     Covered Code available to a third party.\n\n     1.1. \"Contributor\" means each entity that creates or contributes to\n     the creation of Modifications.\n\n     1.2. \"Contributor Version\" means the combination of the Original\n     Code, prior Modifications used by a Contributor, and the Modifications\n     made by that particular Contributor.\n\n     1.3. \"Covered Code\" means the Original Code or Modifications or the\n     combination of the Original Code and Modifications, in each case\n     including portions thereof.\n\n     1.4. \"Electronic Distribution Mechanism\" means a mechanism generally\n     accepted in the software development community for the electronic\n     transfer of data.\n\n     1.5. \"Executable\" means Covered Code in any form other than Source\n     Code.\n\n     1.6. \"Initial Developer\" means the individual or entity identified\n     as the Initial Developer in the Source Code notice required by Exhibit\n     A.\n\n     1.7. \"Larger Work\" means a work which combines Covered Code or\n     portions thereof with code not governed by the terms of this License.\n\n     1.8. \"License\" means this document.\n\n     1.8.1. \"Licensable\" means having the right to grant, to the maximum\n     extent possible, whether at the time of the initial grant or\n     subsequently acquired, any and all of the rights conveyed herein.\n\n     1.9. \"Modifications\" means any addition to or deletion from the\n     substance or structure of either the Original Code or any previous\n     Modifications. When Covered Code is released as a series of files, a\n     Modification is:\n          A. Any addition to or deletion from the contents of a file\n          containing Original Code or previous Modifications.\n\n          B. Any new file that contains any part of the Original Code or\n          previous Modifications.\n\n     1.10. \"Original Code\" means Source Code of computer software code\n     which is described in the Source Code notice required by Exhibit A as\n     Original Code, and which, at the time of its release under this\n     License is not already Covered Code governed by this License.\n\n     1.10.1. \"Patent Claims\" means any patent claim(s), now owned or\n     hereafter acquired, including without limitation,  method, process,\n     and apparatus claims, in any patent Licensable by grantor.\n\n     1.11. \"Source Code\" means the preferred form of the Covered Code for\n     making modifications to it, including all modules it contains, plus\n     any associated interface definition files, scripts used to control\n     compilation and installation of an Executable, or source code\n     differential comparisons against either the Original Code or another\n     well known, available Covered Code of the Contributor's choice. The\n     Source Code can be in a compressed or archival form, provided the\n     appropriate decompression or de-archiving software is widely available\n     for no charge.\n\n     1.12. \"You\" (or \"Your\")  means an individual or a legal entity\n     exercising rights under, and complying with all of the terms of, this\n     License or a future version of this License issued under Section 6.1.\n     For legal entities, \"You\" includes any entity which controls, is\n     controlled by, or is under common control with You. For purposes of\n     this definition, \"control\" means (a) the power, direct or indirect,\n     to cause the direction or management of such entity, whether by\n     contract or otherwise, or (b) ownership of more than fifty percent\n     (50%) of the outstanding shares or beneficial ownership of such\n     entity.\n\n2. Source Code License.\n\n     2.1. The Initial Developer Grant.\n     The Initial Developer hereby grants You a world-wide, royalty-free,\n     non-exclusive license, subject to third party intellectual property\n     claims:\n          (a)  under intellectual property rights (other than patent or\n          trademark) Licensable by Initial Developer to use, reproduce,\n          modify, display, perform, sublicense and distribute the Original\n          Code (or portions thereof) with or without Modifications, and/or\n          as part of a Larger Work; and\n\n          (b) under Patents Claims infringed by the making, using or\n          selling of Original Code, to make, have made, use, practice,\n          sell, and offer for sale, and/or otherwise dispose of the\n          Original Code (or portions thereof).\n\n          (c) the licenses granted in this Section 2.1(a) and (b) are\n          effective on the date Initial Developer first distributes\n          Original Code under the terms of this License.\n\n          (d) Notwithstanding Section 2.1(b) above, no patent license is\n          granted: 1) for code that You delete from the Original Code; 2)\n          separate from the Original Code;  or 3) for infringements caused\n          by: i) the modification of the Original Code or ii) the\n          combination of the Original Code with other software or devices.\n\n     2.2. Contributor Grant.\n     Subject to third party intellectual property claims, each Contributor\n     hereby grants You a world-wide, royalty-free, non-exclusive license\n\n          (a)  under intellectual property rights (other than patent or\n          trademark) Licensable by Contributor, to use, reproduce, modify,\n          display, perform, sublicense and distribute the Modifications\n          created by such Contributor (or portions thereof) either on an\n          unmodified basis, with other Modifications, as Covered Code\n          and/or as part of a Larger Work; and\n\n          (b) under Patent Claims infringed by the making, using, or\n          selling of  Modifications made by that Contributor either alone\n          and/or in combination with its Contributor Version (or portions\n          of such combination), to make, use, sell, offer for sale, have\n          made, and/or otherwise dispose of: 1) Modifications made by that\n          Contributor (or portions thereof); and 2) the combination of\n          Modifications made by that Contributor with its Contributor\n          Version (or portions of such combination).\n\n          (c) the licenses granted in Sections 2.2(a) and 2.2(b) are\n          effective on the date Contributor first makes Commercial Use of\n          the Covered Code.\n\n          (d)    Notwithstanding Section 2.2(b) above, no patent license is\n          granted: 1) for any code that Contributor has deleted from the\n          Contributor Version; 2)  separate from the Contributor Version;\n          3)  for infringements caused by: i) third party modifications of\n          Contributor Version or ii)  the combination of Modifications made\n          by that Contributor with other software  (except as part of the\n          Contributor Version) or other devices; or 4) under Patent Claims\n          infringed by Covered Code in the absence of Modifications made by\n          that Contributor.\n\n3. Distribution Obligations.\n\n     3.1. Application of License.\n     The Modifications which You create or to which You contribute are\n     governed by the terms of this License, including without limitation\n     Section 2.2. The Source Code version of Covered Code may be\n     distributed only under the terms of this License or a future version\n     of this License released under Section 6.1, and You must include a\n     copy of this License with every copy of the Source Code You\n     distribute. You may not offer or impose any terms on any Source Code\n     version that alters or restricts the applicable version of this\n     License or the recipients' rights hereunder. However, You may include\n     an additional document offering the additional rights described in\n     Section 3.5.\n\n     3.2. Availability of Source Code.\n     Any Modification which You create or to which You contribute must be\n     made available in Source Code form under the terms of this License\n     either on the same media as an Executable version or via an accepted\n     Electronic Distribution Mechanism to anyone to whom you made an\n     Executable version available; and if made available via Electronic\n     Distribution Mechanism, must remain available for at least twelve (12)\n     months after the date it initially became available, or at least six\n     (6) months after a subsequent version of that particular Modification\n     has been made available to such recipients. You are responsible for\n     ensuring that the Source Code version remains available even if the\n     Electronic Distribution Mechanism is maintained by a third party.\n\n     3.3. Description of Modifications.\n     You must cause all Covered Code to which You contribute to contain a\n     file documenting the changes You made to create that Covered Code and\n     the date of any change. You must include a prominent statement that\n     the Modification is derived, directly or indirectly, from Original\n     Code provided by the Initial Developer and including the name of the\n     Initial Developer in (a) the Source Code, and (b) in any notice in an\n     Executable version or related documentation in which You describe the\n     origin or ownership of the Covered Code.\n\n     3.4. Intellectual Property Matters\n          (a) Third Party Claims.\n          If Contributor has knowledge that a license under a third party's\n          intellectual property rights is required to exercise the rights\n          granted by such Contributor under Sections 2.1 or 2.2,\n          Contributor must include a text file with the Source Code\n          distribution titled \"LEGAL\" which describes the claim and the\n          party making the claim in sufficient detail that a recipient will\n          know whom to contact. If Contributor obtains such knowledge after\n          the Modification is made available as described in Section 3.2,\n          Contributor shall promptly modify the LEGAL file in all copies\n          Contributor makes available thereafter and shall take other steps\n          (such as notifying appropriate mailing lists or newsgroups)\n          reasonably calculated to inform those who received the Covered\n          Code that new knowledge has been obtained.\n\n          (b) Contributor APIs.\n          If Contributor's Modifications include an application programming\n          interface and Contributor has knowledge of patent licenses which\n          are reasonably necessary to implement that API, Contributor must\n          also include this information in the LEGAL file.\n\n               (c)    Representations.\n          Contributor represents that, except as disclosed pursuant to\n          Section 3.4(a) above, Contributor believes that Contributor's\n          Modifications are Contributor's original creation(s) and/or\n          Contributor has sufficient rights to grant the rights conveyed by\n          this License.\n\n     3.5. Required Notices.\n     You must duplicate the notice in Exhibit A in each file of the Source\n     Code.  If it is not possible to put such notice in a particular Source\n     Code file due to its structure, then You must include such notice in a\n     location (such as a relevant directory) where a user would be likely\n     to look for such a notice.  If You created one or more Modification(s)\n     You may add your name as a Contributor to the notice described in\n     Exhibit A.  You must also duplicate this License in any documentation\n     for the Source Code where You describe recipients' rights or ownership\n     rights relating to Covered Code.  You may choose to offer, and to\n     charge a fee for, warranty, support, indemnity or liability\n     obligations to one or more recipients of Covered Code. However, You\n     may do so only on Your own behalf, and not on behalf of the Initial\n     Developer or any Contributor. You must make it absolutely clear than\n     any such warranty, support, indemnity or liability obligation is\n     offered by You alone, and You hereby agree to indemnify the Initial\n     Developer and every Contributor for any liability incurred by the\n     Initial Developer or such Contributor as a result of warranty,\n     support, indemnity or liability terms You offer.\n\n     3.6. Distribution of Executable Versions.\n     You may distribute Covered Code in Executable form only if the\n     requirements of Section 3.1-3.5 have been met for that Covered Code,\n     and if You include a notice stating that the Source Code version of\n     the Covered Code is available under the terms of this License,\n     including a description of how and where You have fulfilled the\n     obligations of Section 3.2. The notice must be conspicuously included\n     in any notice in an Executable version, related documentation or\n     collateral in which You describe recipients' rights relating to the\n     Covered Code. You may distribute the Executable version of Covered\n     Code or ownership rights under a license of Your choice, which may\n     contain terms different from this License, provided that You are in\n     compliance with the terms of this License and that the license for the\n     Executable version does not attempt to limit or alter the recipient's\n     rights in the Source Code version from the rights set forth in this\n     License. If You distribute the Executable version under a different\n     license You must make it absolutely clear that any terms which differ\n     from this License are offered by You alone, not by the Initial\n     Developer or any Contributor. You hereby agree to indemnify the\n     Initial Developer and every Contributor for any liability incurred by\n     the Initial Developer or such Contributor as a result of any such\n     terms You offer.\n\n     3.7. Larger Works.\n     You may create a Larger Work by combining Covered Code with other code\n     not governed by the terms of this License and distribute the Larger\n     Work as a single product. In such a case, You must make sure the\n     requirements of this License are fulfilled for the Covered Code.\n\n4. Inability to Comply Due to Statute or Regulation.\n\n     If it is impossible for You to comply with any of the terms of this\n     License with respect to some or all of the Covered Code due to\n     statute, judicial order, or regulation then You must: (a) comply with\n     the terms of this License to the maximum extent possible; and (b)\n     describe the limitations and the code they affect. Such description\n     must be included in the LEGAL file described in Section 3.4 and must\n     be included with all distributions of the Source Code. Except to the\n     extent prohibited by statute or regulation, such description must be\n     sufficiently detailed for a recipient of ordinary skill to be able to\n     understand it.\n\n5. Application of this License.\n\n     This License applies to code to which the Initial Developer has\n     attached the notice in Exhibit A and to related Covered Code.\n\n6. Versions of the License.\n\n     6.1. New Versions.\n     Netscape Communications Corporation (\"Netscape\") may publish revised\n     and/or new versions of the License from time to time. Each version\n     will be given a distinguishing version number.\n\n     6.2. Effect of New Versions.\n     Once Covered Code has been published under a particular version of the\n     License, You may always continue to use it under the terms of that\n     version. You may also choose to use such Covered Code under the terms\n     of any subsequent version of the License published by Netscape. No one\n     other than Netscape has the right to modify the terms applicable to\n     Covered Code created under this License.\n\n     6.3. Derivative Works.\n     If You create or use a modified version of this License (which you may\n     only do in order to apply it to code which is not already Covered Code\n     governed by this License), You must (a) rename Your license so that\n     the phrases \"Mozilla\", \"MOZILLAPL\", \"MOZPL\", \"Netscape\",\n     \"MPL\", \"NPL\" or any confusingly similar phrase do not appear in your\n     license (except to note that your license differs from this License)\n     and (b) otherwise make it clear that Your version of the license\n     contains terms which differ from the Mozilla Public License and\n     Netscape Public License. (Filling in the name of the Initial\n     Developer, Original Code or Contributor in the notice described in\n     Exhibit A shall not of themselves be deemed to be modifications of\n     this License.)\n\n7. DISCLAIMER OF WARRANTY.\n\n     COVERED CODE IS PROVIDED UNDER THIS LICENSE ON AN \"AS IS\" BASIS,\n     WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING,\n     WITHOUT LIMITATION, WARRANTIES THAT THE COVERED CODE IS FREE OF\n     DEFECTS, MERCHANTABLE, FIT FOR A PARTICULAR PURPOSE OR NON-INFRINGING.\n     THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE COVERED CODE\n     IS WITH YOU. SHOULD ANY COVERED CODE PROVE DEFECTIVE IN ANY RESPECT,\n     YOU (NOT THE INITIAL DEVELOPER OR ANY OTHER CONTRIBUTOR) ASSUME THE\n     COST OF ANY NECESSARY SERVICING, REPAIR OR CORRECTION. THIS DISCLAIMER\n     OF WARRANTY CONSTITUTES AN ESSENTIAL PART OF THIS LICENSE. NO USE OF\n     ANY COVERED CODE IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS DISCLAIMER.\n\n8. TERMINATION.\n\n     8.1.  This License and the rights granted hereunder will terminate\n     automatically if You fail to comply with terms herein and fail to cure\n     such breach within 30 days of becoming aware of the breach. All\n     sublicenses to the Covered Code which are properly granted shall\n     survive any termination of this License. Provisions which, by their\n     nature, must remain in effect beyond the termination of this License\n     shall survive.\n\n     8.2.  If You initiate litigation by asserting a patent infringement\n     claim (excluding declatory judgment actions) against Initial Developer\n     or a Contributor (the Initial Developer or Contributor against whom\n     You file such action is referred to as \"Participant\")  alleging that:\n\n     (a)  such Participant's Contributor Version directly or indirectly\n     infringes any patent, then any and all rights granted by such\n     Participant to You under Sections 2.1 and/or 2.2 of this License\n     shall, upon 60 days notice from Participant terminate prospectively,\n     unless if within 60 days after receipt of notice You either: (i)\n     agree in writing to pay Participant a mutually agreeable reasonable\n     royalty for Your past and future use of Modifications made by such\n     Participant, or (ii) withdraw Your litigation claim with respect to\n     the Contributor Version against such Participant.  If within 60 days\n     of notice, a reasonable royalty and payment arrangement are not\n     mutually agreed upon in writing by the parties or the litigation claim\n     is not withdrawn, the rights granted by Participant to You under\n     Sections 2.1 and/or 2.2 automatically terminate at the expiration of\n     the 60 day notice period specified above.\n\n     (b)  any software, hardware, or device, other than such Participant's\n     Contributor Version, directly or indirectly infringes any patent, then\n     any rights granted to You by such Participant under Sections 2.1(b)\n     and 2.2(b) are revoked effective as of the date You first made, used,\n     sold, distributed, or had made, Modifications made by that\n     Participant.\n\n     8.3.  If You assert a patent infringement claim against Participant\n     alleging that such Participant's Contributor Version directly or\n     indirectly infringes any patent where such claim is resolved (such as\n     by license or settlement) prior to the initiation of patent\n     infringement litigation, then the reasonable value of the licenses\n     granted by such Participant under Sections 2.1 or 2.2 shall be taken\n     into account in determining the amount or value of any payment or\n     license.\n\n     8.4.  In the event of termination under Sections 8.1 or 8.2 above,\n     all end user license agreements (excluding distributors and resellers)\n     which have been validly granted by You or any distributor hereunder\n     prior to termination shall survive termination.\n\n9. LIMITATION OF LIABILITY.\n\n     UNDER NO CIRCUMSTANCES AND UNDER NO LEGAL THEORY, WHETHER TORT\n     (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE, SHALL YOU, THE INITIAL\n     DEVELOPER, ANY OTHER CONTRIBUTOR, OR ANY DISTRIBUTOR OF COVERED CODE,\n     OR ANY SUPPLIER OF ANY OF SUCH PARTIES, BE LIABLE TO ANY PERSON FOR\n     ANY INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES OF ANY\n     CHARACTER INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF GOODWILL,\n     WORK STOPPAGE, COMPUTER FAILURE OR MALFUNCTION, OR ANY AND ALL OTHER\n     COMMERCIAL DAMAGES OR LOSSES, EVEN IF SUCH PARTY SHALL HAVE BEEN\n     INFORMED OF THE POSSIBILITY OF SUCH DAMAGES. THIS LIMITATION OF\n     LIABILITY SHALL NOT APPLY TO LIABILITY FOR DEATH OR PERSONAL INJURY\n     RESULTING FROM SUCH PARTY'S NEGLIGENCE TO THE EXTENT APPLICABLE LAW\n     PROHIBITS SUCH LIMITATION. SOME JURISDICTIONS DO NOT ALLOW THE\n     EXCLUSION OR LIMITATION OF INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO\n     THIS EXCLUSION AND LIMITATION MAY NOT APPLY TO YOU.\n\n10. U.S. GOVERNMENT END USERS.\n\n     The Covered Code is a \"commercial item,\" as that term is defined in\n     48 C.F.R. 2.101 (Oct. 1995), consisting of \"commercial computer\n     software\" and \"commercial computer software documentation,\" as such\n     terms are used in 48 C.F.R. 12.212 (Sept. 1995). Consistent with 48\n     C.F.R. 12.212 and 48 C.F.R. 227.7202-1 through 227.7202-4 (June 1995),\n     all U.S. Government End Users acquire Covered Code with only those\n     rights set forth herein.\n\n11. MISCELLANEOUS.\n\n     This License represents the complete agreement concerning subject\n     matter hereof. If any provision of this License is held to be\n     unenforceable, such provision shall be reformed only to the extent\n     necessary to make it enforceable. This License shall be governed by\n     California law provisions (except to the extent applicable law, if\n     any, provides otherwise), excluding its conflict-of-law provisions.\n     With respect to disputes in which at least one party is a citizen of,\n     or an entity chartered or registered to do business in the United\n     States of America, any litigation relating to this License shall be\n     subject to the jurisdiction of the Federal Courts of the Northern\n     District of California, with venue lying in Santa Clara County,\n     California, with the losing party responsible for costs, including\n     without limitation, court costs and reasonable attorneys' fees and\n     expenses. The application of the United Nations Convention on\n     Contracts for the International Sale of Goods is expressly excluded.\n     Any law or regulation which provides that the language of a contract\n     shall be construed against the drafter shall not apply to this\n     License.\n\n12. RESPONSIBILITY FOR CLAIMS.\n\n     As between Initial Developer and the Contributors, each party is\n     responsible for claims and damages arising, directly or indirectly,\n     out of its utilization of rights under this License and You agree to\n     work with Initial Developer and Contributors to distribute such\n     responsibility on an equitable basis. Nothing herein is intended or\n     shall be deemed to constitute any admission of liability.\n\n13. MULTIPLE-LICENSED CODE.\n\n     Initial Developer may designate portions of the Covered Code as\n     \"Multiple-Licensed\".  \"Multiple-Licensed\" means that the Initial\n     Developer permits you to utilize portions of the Covered Code under\n     Your choice of the NPL or the alternative licenses, if any, specified\n     by the Initial Developer in the file described in Exhibit A.\n\nEXHIBIT A -Mozilla Public License.\n\n     ``The contents of this file are subject to the Mozilla Public License\n     Version 1.1 (the \"License\"); you may not use this file except in\n     compliance with the License. You may obtain a copy of the License at\n     http://www.mozilla.org/MPL/\n\n     Software distributed under the License is distributed on an \"AS IS\"\n     basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the\n     License for the specific language governing rights and limitations\n     under the License.\n\n     The Original Code is ______________________________________.\n\n     The Initial Developer of the Original Code is ________________________.\n     Portions created by ______________________ are Copyright (C) ______\n     _______________________. All Rights Reserved.\n\n     Contributor(s): ______________________________________.\n\n     Alternatively, the contents of this file may be used under the terms\n     of the _____ license (the  \"[___] License\"), in which case the\n     provisions of [______] License are applicable instead of those\n     above.  If you wish to allow use of your version of this file only\n     under the terms of the [____] License and not to allow others to use\n     your version of this file under the MPL, indicate your decision by\n     deleting  the provisions above and replace  them with the notice and\n     other provisions required by the [___] License.  If you do not delete\n     the provisions above, a recipient may use your version of this file\n     under either the MPL or the [___] License.\"\n\n     [NOTE: The text of this Exhibit A may differ slightly from the text of\n     the notices in the Source Code files of the Original Code. You should\n     use the text of this Exhibit A rather than the text found in the\n     Original Code Source Code for Your Modifications.]\n\n"
  },
  {
    "path": "contrib/log4erl/Makefile",
    "content": "all: compile\n\ndepends:\n\t./rebar get-deps\n\t./rebar update-deps\n\ncompile:\n\t./rebar compile\n\nclean:\n\t./rebar clean\n\n.PHONY: all depends compile\n"
  },
  {
    "path": "contrib/log4erl/README.txt",
    "content": "log4erl Manual:\n===============\n\nTOC:\n====\n1. Features\n2. Installation\n3. Usage\n4. API\n5. Configuration\n6. Known issues\n7. Future development\n8. License\n\n1. FEATURES:\n============\n- Multiple logs\n- Currently, only size-based log rotation of files for file appender\n- Support default logger if no logger specified\n- 5 predifined log levels (debug, info, warn, error, fatal)\n- A log handler for error_logger\n- Support for user-specified log levels\n- Support for a log formatter (similar to Layouts in Log4J)\n- Support for console log\n- Support for smtp formatter\n- Support for XML logs\n- Support for syslog \n- Support for changing format and level of appender during run-time\n\n2. INSTALLATION:\n================\nTo compile & install log4erl, download source from google code's website\n(http://code.google.com/p/log4erl/) or  from svn:\n$> svn checkout http://log4erl.googlecode.com/svn/trunk/ log4erl\n\nor from github public repository (http://github.com/ahmednawras/log4erl/).\n\n$> cd log4erl\n$> make\n\nor you can run the below from erlang shell:\n\n$> cd src\n$> erl\n1> make:all([{outdir, \"../ebin\"}]).\n\n3. USAGE:\n=========\n1- In order to use log4erl, you need to first include it in the path. There\nare 2 ways to do this:\na) include the \"log4erl\" directory in erlang's \"lib\" directory in the target\nmachine (cp -Rf log4erl /where/erlang/is/lib).\n$> cp -Rf log4erl /usr/local/lib/erlang/lib/\n\nb) include the \"log4erl\" ebin directory in the path when running you program\n$> erl -pz /path/to/log4erl ...\n\n2- Once the log4erl directory is included, you can use its API as described in section \"API\". but before, \nyou need to run:\n> application:start(log4erl).\n\n3- Create a configuration file and load it using log4erl:conf(File)\n> log4erl:conf(\"priv/log4erl.conf\").\n\n4- Alternatevly, you can create loggers & add appenders to them programmatically as appropriate. \nYou can do this as per the API below.\n> log4erl:add_logger(messages_log).\n> log4erl:add_console_appender(messages_log, cmd_logs, {warn, \"[%L] %l%n\"}).\n  where Conf is the erlang term describing how the log is to be handled\n  You can also add more types of appenders, which is explained in API.txt and Appneders_API.txt.\n\n5- Now, you can log whatever messages you like as per logging functions described in API.\n> log4erl:info(\"Information message\").\n\nPrecedance of log levels are:\nall = debug < info < warn < error < fatal < none\nUser defined levels are always written except when none level is specified in the logger specification\n(See below).\n\n4. API:\n=======\nPlease look at API.txt file for more information about the API.\n\n5. CONFIGURATION:\n=================\nPlease look at CONFIGURATION.txt for more information about how to configure log4erl.\n\n6. KNOWN ISSUES:\n================\n- Name of both loggers & appenders should be unique and not registered since log4erl will try and register\n  their names. If the name is already registered, nothing will happen. This will be fixed soon.\n- If you run change_log_format/1,2 and appender crashed, a restart from the supervisor will not record the latest\n  format used. It will only use either the default format or the format used in the argument is supplied.  \n\n7. FUTURE DEVELOPMENT:\n======================\n- Add support for different log persistance methods (e.g files, XML, console, DB, SNMP, syslog...etc)\n- Add support for time-based log rotation\n- Multiple configuration format (Erlang terms, XML?, properties files?)\n- Add support for NDC & MDC ???\n\nPlease send your suggestion to ahmed.nawras <at @ at> gmail <dot . dot> com\n\n8. LICENSE:\n===========\nThis software is subject to \"Mozilla Public License 1.1\". You can find the license terms\nin the file 'LICENSE.txt' shipping along with the source code. You may also get a copy\nof the license term from the URL: \"http://www.mozilla.org/MPL/MPL-1.1.html\".\n"
  },
  {
    "path": "contrib/log4erl/TODO.txt",
    "content": "TODO:\n=====\n\n* Add formatters functions that can be added to appenders to change\ndefault formatting patters [DONE]\n* Add support for pattern appender [DONE]\n* Update manual [DONE (latest for log4erl 0.8.2)]\n* Add performance benchmarks\n* Add support for file-based configuration to log4erl instead of only\nprogrammable-only configuration that is working now [DONE]\n* Add support for SNMP appender\n* Add support for DB appender (MySQL, Postgres?, Mnesia,...etc)\n* Add support for SMTP appender [DONE]\n* Add support for NT event appender\n* Add support for syslog appender [DONE]\n* Add support for NDC/MDC\n* Add support for time-based log rotation (file_appender)\n\n"
  },
  {
    "path": "contrib/log4erl/include/log4erl.hrl",
    "content": "-define(ROTATION_CHECK, 10).\r\n\r\n-define(DEFAULT_CONF,\"log4erl.conf\").\r\n\r\n-define(DEFAULT_FORMAT, \"[%L] %l%n\").\r\n\r\n-define(DEFAULT_LEVEL, warn).\r\n\r\n-define(DEFAULT_LOGGER, default_logger).\r\n-define(DEFAULT_LOGGER_GUARD, default_logger_guard).\r\n\r\n-define(FILE_OPTIONS,[write, raw, binary, delayed_write]).\r\n-define(FILE_OPTIONS_ROTATE,[write, raw, binary]).\r\n\r\n%-define(DEBUG, true).\r\n\r\n-ifdef(DEBUG).\r\n-define(LOG(X), io:format(\"[~p:~p] \" ++ X ++ \"~n\",[?MODULE, ?LINE])).\r\n-define(LOG2(X,D), io:format(\"[~p:~p] \" ++ X ++ \"~n\",[?MODULE, ?LINE | D])).\r\n-else.\r\n-define(LOG(_X), ok).\r\n-define(LOG2(_X,_D), ok).\r\n-endif.\r\n\r\n\r\n%% type = time | size\r\n%% max = seconds (for time) | or kiloBytes (for size)\r\n-record(log_type,{type, max}).\r\n\r\n%% file_name = the name of the file without counter \r\n%% fd = the descriptior for the file\r\n%% counter = current counter, used for appending to file_name in case of rotation\r\n%% log_type is a log_type record\r\n%% rotation = number of rotation before the logger wraps around the coutner (>1)\r\n%% suffix = suffix of file name (e.g. txt)\r\n%% The filename of a log is file_name ++ \"_\" ++ counter ++ \".\" ++ suffix\r\n%% e.g. log_1.txt\r\n%% tokens = format tokens generated from log_formatter:parse/1\r\n-record(file_appender, {dir, file_name, fd, counter, log_type, rotation, suffix, level, format=?DEFAULT_FORMAT}).\r\n\r\n-record(console_appender, {level=?DEFAULT_LEVEL, format=?DEFAULT_FORMAT}).\r\n\r\n-record(rotation_state, {state, timer}).\r\n\r\n-record(smtp_appender, {level=?DEFAULT_LEVEL, srvr_opts, auth_opts, msg_opts}).\r\n-record(srvr_opts, {ip, port}).\r\n-record(auth_opts, {username, password}).\r\n-record(msg_opts, {from, to, title, msg=?DEFAULT_FORMAT}).\r\n\r\n-record(syslog_appender, {level=?DEFAULT_LEVEL, facility=user,host,port=415, socket, format=?DEFAULT_FORMAT}).\r\n\r\n%% log record\r\n-record(log, {level, msg, data, time, millis}).\r\n"
  },
  {
    "path": "contrib/log4erl/priv/l4e.conf",
    "content": "%% Default logger\n%% it includes a file appender and a console appender\nlogger{\n\tfile_appender app2{\n\t\tdir = \".\",\n\t\tlevel = info,\n\t\tfile = my_app,\n\t\ttype = time,\n\t\tmax = 60,\n\t\tsuffix = log,\n\t\trotation = 10,\n\t\tformat = '[%L]: %I, %l%n'\n\t}\n\n\tconsole_appender app1{\n\t\tlevel = warn,\n\t\tformat = '%T %j [%L] %l%n'\n\t}\n}\n"
  },
  {
    "path": "contrib/log4erl/priv/log4erl.conf",
    "content": "cutoff_level = warn\n\n%% Default logger\n%% it includes a file appender and a console appender\nlogger{\n\tfile_appender app2{\n\t\tdir = \".\",\n\t\tlevel = info,\n\t\tfile = my_app,\n\t\ttype = time,\n\t\t%max = 100000,\n\t\tmax = 5,\n\t\tsuffix = log,\n\t\trotation = 5,\n\t\tformat = '[%L] %I %l%n'\n\t}\n\n\tconsole_appender app1{\n\t\tlevel = warn,\n\t\tformat = '%T %j [%L] %l%n'\n\t}\n}\n\n%% XML appender\nlogger xmllogs{\n    xml_appender xml{\n\tlevel = all,\n\tdir = \"logs\",\n\tfile = \"xml_app\",\n\ttype = size,\n\tmax = 10000,\n\tsuffix = xml,\n\trotation = 5\n    }\n}\n\n%% email logger\nlogger email_logger{\n\tsmtp_appender app3{\n\t\tlevel=all,\n\t\tip = \"10.1.102.6\",\n\t\t%port = 25,\n\t\tno_auth = true,\n\t\t%username = user,\n\t\t%password = pass,\n\t\tfrom = \"admin@log4erl.org\",\n\t\tto = \"notification@log4erl.org\",\n\t\ttitle = \"System info\",\n\t\tmsg = \"[%T %j] %L:%n%l%n\"\n\t}\n}\n\n%% A logger with 1 syslog appender\nlogger syslog{\n\tsyslog_appender app4{\n\t\tlevel = all,\n\t\tfacility = ftp,\n\t\thost = \"localhost\",\n\t\tport = 514,\n\t\tformat = \"%l%n\"\n\t}\n}\n\n"
  },
  {
    "path": "contrib/log4erl/src/Makefile",
    "content": "EBIN_DIR=../ebin\nINCLUDE_DIR=../include\nERLS=$(wildcard *.erl)\nERLC=erlc -o $(EBIN_DIR) -I ${INCLUDE_DIR}\nBEAMS=$(ERLS:%.erl=$(EBIN_DIR)/%.beam)\n\n# leave these lines alone\n.SUFFIXES: .erl .beam .yrl\n\n.erl.beam:\n\t$(ERLC) -W $<\n\n#.yrl.erl:\n#\t$(ERLC)-W $<\n\nall: compile\n\n$(EBIN_DIR)/%.beam: %.erl\n\t@echo \">>\" compiling: $<\n\t@$(ERLC) $<\n\ngenerate_parser:\n\t@echo \">>\" generating log4erl_parser.erl from log4erl_parser.yrl $<\n\terlc log4erl_parser.yrl\n\n# export LEEX_PATH env var for this to work\ngenerate_lex:\n\t@echo \">>\" generating log4erl_lex.erl from log4erl_lex.xrl $<\n\terl -noinput -noshell -pa $$LEEX_PATH/ebin -s leex file log4erl_lex.xrl -s init stop\n\ncompile: ${BEAMS}\n\nclean:\n\trm -rf ../ebin/*.beam erl_crash.dump\n"
  },
  {
    "path": "contrib/log4erl/src/console_appender.erl",
    "content": "-module(console_appender).\r\n\r\n-include(\"../include/log4erl.hrl\").\r\n\r\n-behaviour(gen_event).\r\n%% gen_event callbacks\r\n-export([init/1, handle_event/2, handle_call/2, \r\n\t handle_info/2, terminate/2, code_change/3]).\r\n\r\ninit({conf, Conf}) when is_list(Conf) ->\r\n    CL = lists:foldl(fun(X, List) ->\r\n\t\t\t     [proplists:get_value(X,Conf)|List]\r\n\t\t     end,\r\n\t\t     [],\r\n\t\t     [level, format]),\r\n    \r\n    %% in case format doesn't exist\r\n    Res = case hd(CL) of\r\n\t      undefined ->\r\n\t\t  [_|CL2] = CL,\r\n\t\t  lists:reverse(CL2);\r\n\t      _ ->\r\n\t\t  lists:reverse(CL)\r\n\t  end,\r\n    init(list_to_tuple(Res));\r\ninit({Level}) ->\r\n    init({Level, ?DEFAULT_FORMAT});\r\ninit({Level, Format} = _Args) ->\r\n    ?LOG2(\"Initializing console_appender with args =  ~p~n\",[_Args]),\r\n    {ok, Toks} = log_formatter:parse(Format),\r\n    ?LOG2(\"Tokens received is ~p~n\",[Toks]),\r\n    State = #console_appender{level = Level, format = Toks},\r\n    ?LOG2(\"State is ~p~n\",[State]),\r\n    {ok, State}.\r\n\r\nhandle_event({change_level, Level}, State) ->\r\n    State2 = State#console_appender{level = Level},\r\n    ?LOG2(\"Changed level to ~p~n\",[Level]),\r\n    {ok, State2};\r\nhandle_event({log,LLog}, State) ->\r\n    ?LOG2(\"handl_event:log = ~p~n\",[LLog]),\r\n    do_log(LLog, State),\r\n    {ok, State}.\r\n\r\nhandle_call({change_format, Format}, State) ->\r\n    ?LOG2(\"Old State in console_appender is ~p~n\",[State]),\r\n    {ok, Tokens} = log_formatter:parse(Format),\r\n    ?LOG2(\"Adding format of ~p~n\",[Tokens]),\r\n    State1 = State#console_appender{format=Tokens},\r\n    {ok, ok, State1};\r\nhandle_call({change_level, Level}, State) ->\r\n    State2 = State#console_appender{level = Level},\r\n    ?LOG2(\"Changed level to ~p~n\",[Level]),\r\n    {ok, ok, State2};\r\nhandle_call(_Request, State) ->\r\n    Reply = ok,\r\n    ?LOG2(\"Received unknown request ~p~n\", [_Request]),\r\n    {ok, Reply, State}.\r\n\r\nhandle_info(_Info, State) ->\r\n    {ok, State}.\r\n\r\nterminate(_Reason, _State) ->\r\n    ok.\r\n\r\ncode_change(_OldVsn, State, _Extra) ->\r\n    {ok, State}.\r\n\r\ndo_log(#log{level = L} = Log,#console_appender{level=Level, format=Format}) ->\r\n    ToLog = log4erl_utils:to_log(L, Level),\r\n    case ToLog of\r\n\ttrue ->\r\n\t    M = log_formatter:format(Log, Format),\r\n\t    ?LOG2(\"console_appender result message is ~s~n\",[M]),\r\n\t    io:format(M);\r\n\tfalse ->\r\n\t    ok\r\n    end.\r\n"
  },
  {
    "path": "contrib/log4erl/src/dummy_appender.erl",
    "content": "-module(dummy_appender).\r\n\r\n-include(\"../include/log4erl.hrl\").\r\n\r\n-behaviour(gen_event).\r\n%% gen_event callbacks\r\n-export([init/1, handle_event/2, handle_call/2, \r\n\t handle_info/2, terminate/2, code_change/3]).\r\n\r\n-record(state, {}).\r\n\r\ninit(_Args) ->\r\n    io:format(\"Initializing dummy_appender with args =  ~p~n\",[_Args]),\r\n    {ok, #state{}}.\r\n\r\nhandle_event(_Event, State) ->\r\n    io:format(\"dummy_appender received event ~p~n\",[_Event]),\r\n    {ok, State}.\r\n\r\nhandle_call(_Request, State) ->\r\n    io:format(\"dummy_appender called with message ~p~n\",[_Request]),\r\n    Reply = ok,\r\n    {ok, Reply, State}.\r\n\r\nhandle_info(_Info, State) ->\r\n    io:format(\"dummy_appender got info message ~p~n\",[_Info]),\r\n    {ok, State}.\r\n\r\nterminate(_Reason, _State) ->\r\n    ok.\r\n\r\ncode_change(_OldVsn, State, _Extra) ->\r\n    {ok, State}.\r\n\r\n"
  },
  {
    "path": "contrib/log4erl/src/email_msg.erl",
    "content": "%%\n%% file: email_msg.erl\n%% author: Michael Bradford <michael.bradford@t-mobile.uk.net>\n%% description: a very simple module that creates a non-multipart\n%% email message.\n%% Doesn't support MIME or non-ascii characters\n%% \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Usage Example\n%%\n%% 2> c(email_msg).\n%% {ok,email_msg}\n%% 3> From = \"michael.bradford@t-mobile.uk.net\".\n%% \"michael.bradford@t-mobile.uk.net\"\n%% 4> To = \"test@test.co.uk\".\n%% \"test@test.co.uk\"\n%% 5> Subject = \"Testing !!!!\".\n%% \"Testing !!!!\"\n%% 6> Content = \"Hi Mike, this is a test, bye\".\n%% \"Hi Mike, this is a test, bye\"\n%% 7> Msg = email_msg:simp_msg(From,To,Subject,Content).\n%% \"from: michael.bradford@t-mobile.uk.net\\r\\nto: test@test.co.uk\\r\\n\n%% subject: Testi ng !!!!\\r\\n\\r\\nHi Mike, this is a test, bye\\r\\n\"\n%% 8> \n%%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%\n-module(email_msg).\n-vsn('v1.0').\n\n-export([simp_msg/4]).\n\nsimp_msg(From, To, Subject, Message) ->\n\tFromStr = [\"from: \", From, \"\\r\\n\"],\n\tToStr = [\"to: \", To, \"\\r\\n\"],\n\tSubjStr = [\"subject: \", Subject, \"\\r\\n\"],\n\tMsgStr = [\"\\r\\n\", Message],\n\tlists:concat(lists:concat([FromStr, ToStr, SubjStr, MsgStr, [\"\\r\\n\"]])).\n"
  },
  {
    "path": "contrib/log4erl/src/error_logger_log4erl_h.erl",
    "content": "%% Initially written by Jakub Odias and later modified by Ahmed Al-Issaei\r\n%% Main addition, add mapping between error_logger msgs and log4erl log levels\r\n\r\n%% Code is released as MPL with Jakub's permission\r\n\r\n-module(error_logger_log4erl_h).\r\n-behaviour(gen_event).\r\n\r\n%% @author Jakub Odias\r\n%% @copyright Proximetry\r\n%% @version 1.0.0\r\n%% @doc Handles events reported by error_logger and forwards it to log4erl\r\n%% @end\r\n\r\n%% gen_event callbacks\r\n-export([init/1, handle_event/2, handle_call/2, \r\n\t handle_info/2, terminate/2, code_change/3]).\r\n\r\n-export([add_handler/0, add_handler/1]).\r\n\r\n-record(elogger_l4e_mappings, {error=error, info_msg=info, warning_msg=warn,\r\n\t\t\t       error_report=error, info_report=info,\r\n\t\t\t       warning_report=warn}).\r\n\r\n%%======================================\r\n%% gen_event callback functions\r\n%%======================================\r\ninit([])->\r\n    {ok, #elogger_l4e_mappings{}};\r\ninit(Conf) ->\r\n    io:format(\"Conf ~p~n\",[Conf]),\r\n    {ok, mapping(Conf, #elogger_l4e_mappings{})}.\r\n\r\nadd_handler(Args) ->\r\n    error_logger:add_report_handler(?MODULE, Args).\r\n\r\nadd_handler() ->\r\n    error_logger:add_report_handler(?MODULE,[]).\r\n\r\nmapping([], M) ->\r\n    M;\r\nmapping([H | C], M) ->\r\n    mapping(C, m(H, M)).\r\n\r\nm({error, L}, M) ->\r\n    M#elogger_l4e_mappings{error=L};\r\nm({info_msg, L}, M) ->\r\n    M#elogger_l4e_mappings{info_msg=L};\r\nm({warning_msg, L}, M) ->\r\n    M#elogger_l4e_mappings{warning_msg=L};\r\nm({error_report, L}, M) ->\r\n    M#elogger_l4e_mappings{error_report=L};\r\nm({info_report, L}, M) ->\r\n    M#elogger_l4e_mappings{info_report=L};\r\nm({warning_report, L}, M) ->\r\n    M#elogger_l4e_mappings{warning_report=L};\r\nm(_, M) ->\r\n    M.\r\n\r\nhandle_event({error, _GLeader, {_PID, Msg, Data}}, #elogger_l4e_mappings{error=L} = State) ->\r\n    R = log4erl:log(L, Msg, Data),\r\n    {R, State};\r\nhandle_event({info_msg, _GLeader, {_PID, Msg, Data}}, #elogger_l4e_mappings{info_msg=L} = State) ->\r\n    R = log4erl:log(L, Msg, Data),\r\n    {R, State};\r\nhandle_event({warning_msg, _GLeader, {_PID, Msg, Data}}, #elogger_l4e_mappings{warning_msg=L} = State) ->\r\n    R = log4erl:log(L, Msg, Data),\r\n    {R, State};\r\nhandle_event({error_report, _GLeader, _}, State) ->\r\n    {ok, State};\r\nhandle_event({info_report, _GLeader, _}, State) ->\r\n    {ok, State};\r\nhandle_event({warning_report, _GLeader, _}, State) ->\r\n    {ok, State}.\r\n\r\nhandle_call(_Request, State) ->\r\n    Reply = ok,\r\n    {ok, Reply, State}.\r\n\r\nhandle_info(_Info, State) ->\r\n    {ok, State}.\r\n\r\nterminate(_Reason, _State) ->\r\n    ok.\r\n\r\ncode_change(_OldVsn, State, _Extra) ->\r\n    {ok, State}.\r\n"
  },
  {
    "path": "contrib/log4erl/src/file_appender.erl",
    "content": "-module(file_appender).\r\n\r\n-author(\"Ahmed Al-Issaei\").\r\n-license(\"MPL-1.1\").\r\n\r\n-behaviour(gen_event).\r\n\r\n-include(\"../include/log4erl.hrl\").\r\n-include_lib(\"kernel/include/file.hrl\").\r\n\r\n-import(log4erl_utils, [to_list/1, to_atom/1, to_int/1]).\r\n\r\n%% gen_event callbacks\r\n-export([init/1, handle_event/2, handle_call/2, \r\n\t handle_info/2, terminate/2, code_change/3]).\r\n\r\n%%======================================\r\n%% gen_event callback functions\r\n%%======================================\r\ninit({conf, Conf}) when is_list(Conf) ->\r\n    CL = lists:foldl(fun(X, List) ->\r\n\t\t\t     [proplists:get_value(X,Conf)|List]\r\n\t\t     end,\r\n\t\t     [],\r\n\t\t     [dir, file, type, max, rotation, suffix, level, format]),\r\n    \r\n    %% in case format doesn't exist\r\n    Res = case hd(CL) of\r\n\t      undefined ->\r\n\t\t  [_|CL2] = CL,\r\n\t\t  lists:reverse(CL2);\r\n\t      _ ->\r\n\t\t  lists:reverse(CL)\r\n\t  end,\r\n    init(list_to_tuple(Res));\r\ninit({Dir, Fname, {Type, Max}, Rot, Suf, Level})->\r\n    ?LOG(\"file_appender:init() - with default format~n\"),\r\n    init({Dir, Fname, {Type, Max}, Rot, Suf, Level, ?DEFAULT_FORMAT});\r\n%% This one with custom format\r\ninit({Dir, Fname, {Type, Max}, Rot, Suf, Level, Pattern} = _Conf) ->\r\n    ?LOG2(\"file_appender:init() - 1 ~p~n\",[_Conf]),\r\n    File = Dir ++ \"/\" ++ Fname ++ \".\" ++ Suf,\r\n    {ok, Fd} = file:open(File, ?FILE_OPTIONS),\r\n    Ltype = #log_type{type = Type, max = Max},\r\n\r\n    case Type of\r\n\ttime ->\r\n\t    start_file_timer(Max);\r\n\t_ ->\r\n\t    ok\r\n    end,\r\n    \r\n    % Check Rot >= 0\r\n    Rot1 = case Rot < 0 of\r\n\t       true ->\r\n\t\t   0;\r\n\t       false ->\r\n\t\t   Rot\r\n\t   end,\r\n    \r\n    ?LOG2(\"To parse format with customer format ~p~n\",[Pattern]),\r\n    {ok, Format} = log_formatter:parse(Pattern),\r\n    ?LOG2(\"Adding format of ~p~n\",[Format]),\r\n    State = #file_appender{dir = Dir, file_name = Fname, fd = Fd, counter=0,\r\n\t\t\t   log_type = Ltype, rotation = Rot1, suffix=Suf,\r\n\t\t\t   level=Level, format=Format},\r\n    ?LOG2(\"file_appender:init() with conf ~p~n\",[State]),\r\n    {ok, State};\r\n% These 2 are for result of reading conf file\r\ninit({Dir, Fname, Type, Max, Rot, Suf, Level}) ->\r\n    init({to_list(Dir), to_list(Fname), {to_atom(Type), to_int(Max)}, to_int(Rot), to_list(Suf), to_atom(Level)});\r\ninit({Dir, Fname, Type, Max, Rot, Suf, Level, Format}) ->\r\n    init({to_list(Dir), to_list(Fname), {to_atom(Type), to_int(Max)}, to_int(Rot), to_list(Suf), to_atom(Level), to_list(Format)});\r\n% This is for reading from a config file only for one appender\r\ninit(Conf) when is_list(Conf) ->\r\n    ?LOG2(\"file_appender:init() ~p~n\",[Conf]),\r\n    case file:consult(Conf) of\r\n\t{error, Reason} ->\r\n\t    error_logger:error_msg(\"file_appender: couldn't consult Conf file~n\"),\r\n\t    {error, file:format_error(Reason)};\r\n\t{ok, [Terms]} ->\r\n\t    init(Terms)\r\n    end.\r\n\r\nhandle_event({change_level, Level}, State) ->\r\n    State2 = State#file_appender{level = Level},\r\n    ?LOG2(\"Changed level to ~p~n\",[Level]),\r\n    {ok, State2};\r\nhandle_event({log,LLog}, State) ->\r\n    ?LOG2(\"handl_event:log = ~p~n\",[LLog]),\r\n    do_log(LLog, State),\r\n    Res = check_rotation(State),\r\n    {ok, Res}.\r\n\r\nhandle_call({change_format, Format}, State) ->\r\n    ?LOG2(\"Old State in file_appender is ~p~n\",[State]),\r\n    {ok, Tokens} = log_formatter:parse(Format),\r\n    ?LOG2(\"Adding format of ~p~n\",[Tokens]),\r\n    State1 = State#file_appender{format=Tokens},\r\n    {ok, ok, State1};\r\nhandle_call({change_level, Level}, State) ->\r\n    State2 = State#file_appender{level = Level},\r\n    ?LOG2(\"Changed level to ~p~n\",[Level]),\r\n    {ok, ok, State2};\r\nhandle_call({change_filename, Fname}, #file_appender{dir=Dir, suffix=Suf} = State) ->\r\n    ok = file:close(State#file_appender.fd),\r\n    File = Dir ++ \"/\" ++ Fname ++ \".\" ++ Suf,\r\n    {ok, Fd} = file:open(File, ?FILE_OPTIONS),\r\n    State2 = State#file_appender{file_name = Fname, fd = Fd},\r\n    ?LOG2(\"Changed filename to ~p~n\",[File]),\r\n    {ok, ok, State2};\r\nhandle_call(_Request, State) ->\r\n    Reply = ok,\r\n    {ok, Reply, State}.\r\n\r\nhandle_info(rotate_timer, #file_appender{log_type=#log_type{max=Max}} = State) ->\r\n    ?LOG2(\"rotate_timer msg is received\", []),\r\n    {ok, State2} = rotate(State),\r\n    start_file_timer(Max),\r\n    {ok, State2};\r\nhandle_info(_Info, State) ->\r\n    ?LOG2(\"~w received unknown message: ~p~n\", [?MODULE, _Info]),\r\n    {ok, State}.\r\n\r\nterminate(_Reason, _State) ->\r\n    ok.\r\n\r\ncode_change(_OldVsn, State, _Extra) ->\r\n    {ok, State}.\r\n\r\n%%======================================\r\n%% internal callback functions\r\n%%======================================\r\ndo_log(#log{level = L} = Log,#file_appender{fd = Fd, level=Level, format=Format} = _State) when is_atom(L) ->\r\n    ?LOG2(\"Formatting ~p~n\",[Format]),\r\n    ToLog = log4erl_utils:to_log(L, Level),\r\n    case ToLog of\r\n\ttrue ->\r\n\t    M = log_formatter:format(Log, Format),\r\n\t    file:write(Fd, M);\r\n\tfalse ->\r\n\t    ok\r\n    end;\r\ndo_log(_Other, _State) ->\r\n    ?LOG2(\"unknown level ~p~n\",[_Other]),\r\n    ok.\r\n\r\nrotate(#file_appender{fd = Fd, dir=Dir,  file_name=Fn, counter=Cntr, rotation=Rot, suffix=Suf, log_type=Ltype, level=Level, format=Format} = _S) ->\r\n    file:close(Fd),\r\n    ?LOG(\"Starting rotation~n\"),\r\n    rotate_file(Dir ++ \"/\" ++ Fn, Rot - 1, Suf),\r\n    Src = Dir ++ \"/\" ++ Fn ++ \".\" ++ Suf,\r\n    {ok ,Fd2} = file:open(Src, ?FILE_OPTIONS_ROTATE),\r\n    State2 = #file_appender{dir = Dir, file_name = Fn, fd = Fd2, counter=Cntr, log_type = Ltype, rotation = Rot, suffix=Suf, level=Level, format=Format},\r\n    {ok, State2}.\r\n\r\nrotate_file(FileBase, Index, Suffix) when Index > 0 ->\r\n    file:rename(FileBase ++ \"_\" ++ integer_to_list(Index) ++ \".\" ++ Suffix, \r\n\t\tFileBase ++ \"_\" ++ integer_to_list(Index + 1) ++ \".\" ++ Suffix),\r\n    rotate_file(FileBase, Index - 1, Suffix);\r\nrotate_file(FileBase, _Index, Suffix) ->\r\n    file:rename(FileBase ++ \".\" ++ Suffix, FileBase ++ \"_1.\" ++ Suffix).\r\n\r\n\r\n% Check if the file needs to be rotated\r\n% ignore in case of if log type is set to time instead of size\t    \r\ncheck_rotation(State) ->\r\n    #file_appender{dir=Dir, file_name=Fname, log_type = #log_type{type=T, max=Max}, suffix=Suf} = State,\r\n    case T of\r\n\tsize ->\r\n\t    File = Dir ++ \"/\" ++ Fname ++  \".\" ++ Suf,\r\n\t    {ok, Finfo} = file:read_file_info(File),\r\n\t    Size = Finfo#file_info.size,\r\n\t    if\r\n\t\tSize > Max ->\r\n\t\t    {ok, State2} = rotate(State),\r\n\t\t    State2;\r\n\t\ttrue ->\r\n\t\t    State\r\n\t    end;\r\n\t%% time-based rotation is implemented in a seperate process\r\n\t_ ->\r\n\t    State\r\n    end.\r\n\r\nstart_file_timer(Max) ->\r\n    Self = self(),\r\n    ?LOG(\"starting file timer\"),\r\n    spawn_link(fun() ->\r\n\t\t       %% time is in seconds\r\n\t\t       timer:sleep(Max*1000),\r\n\t\t       Self ! rotate_timer\r\n\t       end),\r\n    ok.\r\n\t\t       \r\n"
  },
  {
    "path": "contrib/log4erl/src/log4erl.app.src",
    "content": "%% -*- mode: erlang -*-\n\n{application, log4erl,\n[{description, \"Logger for erlang in the spirit of Log4J\"},\n {vsn, \"0.9.0\"},\n {registered,[log4erl]},\n {applications, [kernel,stdlib]},\n {mod, {log4erl,[]}},\n {start_phases, []}\n]}.\n"
  },
  {
    "path": "contrib/log4erl/src/log4erl.erl",
    "content": "-module(log4erl).\r\n\r\n-author(\"Ahmed Al-Issaei\").\r\n-license(\"MPL-1.1\").\r\n\r\n-behaviour(application).\r\n\r\n-include(\"../include/log4erl.hrl\").\r\n\r\n%% API\r\n-export([change_log_level/1, change_log_level/2]).\r\n-export([change_level/2, change_level/3]).\r\n-export([change_filename/2, change_filename/3]).\r\n-export([add_logger/1, conf/1]).\r\n-export([add_appender/2, add_appender/3]).\r\n-export([add_file_appender/2, add_file_appender/3]).\r\n-export([add_console_appender/2, add_console_appender/3]).\r\n-export([add_smtp_appender/2, add_smtp_appender/3]).\r\n-export([add_syslog_appender/2, add_syslog_appender/3]).\r\n-export([add_xml_appender/2, add_xml_appender/3]).\r\n-export([add_dummy_appender/2, add_dummy_appender/3]).\r\n-export([get_appenders/0, get_appenders/1]).\r\n-export([change_format/2, change_format/3]).\r\n-export([error_logger_handler/0, error_logger_handler/1]).\r\n\r\n-export([log/2, log/3, log/4]).\r\n\r\n-export([warn/1, warn/2, warn/3]).\r\n-export([info/1, info/2, info/3]).\r\n-export([error/1, error/2, error/3]).\r\n-export([fatal/1, fatal/2, fatal/3]).\r\n-export([debug/1, debug/2, debug/3]).\r\n\r\n%% Application callbacks\r\n-export([start/2, stop/1]).\r\n\r\n%%======================================\r\n%% application callback functions\r\n%%======================================\r\nstart(_Type, []) ->\r\n    ?LOG(\"Starting log4erl app~n\"),\r\n    log4erl_sup:start_link(?DEFAULT_LOGGER).\r\n\r\nstop(_State) ->\r\n    log_filter_codegen:reset(),\r\n    ok.\r\n\r\nadd_logger(Logger) ->\r\n    try_msg({add_logger, Logger}).\r\n\r\n%% Appender = {Appender, Name}\r\nadd_appender(Appender, Conf) ->\r\n    try_msg({add_appender, ?DEFAULT_LOGGER ,Appender, Conf}).\r\n\r\n%% Appender = {Appender, Name}\r\nadd_appender(Logger, Appender, Conf) ->\r\n    try_msg({add_appender, Logger, Appender, Conf}).\r\n    \r\nadd_console_appender(AName, Conf) ->\r\n    add_appender(?DEFAULT_LOGGER, {console_appender, AName}, Conf).\r\n\r\nadd_console_appender(Logger, AName, Conf) ->\r\n    add_appender(Logger, {console_appender, AName}, Conf).\r\n\r\nadd_file_appender(AName, Conf) ->\r\n    add_appender(?DEFAULT_LOGGER, {file_appender, AName}, Conf).\r\n\r\nadd_file_appender(Logger, AName, Conf) ->\r\n    add_appender(Logger, {file_appender, AName}, Conf).\r\n\r\nadd_smtp_appender(Name, Conf) ->\r\n    add_appender(?DEFAULT_LOGGER, {smtp_appender, Name}, Conf).\r\n\r\nadd_smtp_appender(Logger, Name, Conf) ->\r\n    add_appender(Logger, {smtp_appender, Name}, Conf).\r\n\r\nadd_syslog_appender(Name, Conf) ->\r\n    add_appender(?DEFAULT_LOGGER, {syslog_appender, Name}, Conf).\r\n\r\nadd_syslog_appender(Logger, Name, Conf) ->\r\n    add_appender(Logger, {syslog_appender, Name}, Conf).\r\n\r\nadd_xml_appender(Name, Conf) ->\r\n    add_appender(?DEFAULT_LOGGER, {xml_appender, Name}, Conf).\r\n\r\nadd_xml_appender(Logger, Name, Conf) ->\r\n    add_appender(Logger, {xml_appender, Name}, Conf).\r\n\r\nadd_dummy_appender(AName, Conf) ->\r\n    add_appender(?DEFAULT_LOGGER, {dummy_appender, AName}, Conf).\r\n\r\nadd_dummy_appender(Logger, AName, Conf) ->\r\n    add_appender(Logger, {dummy_appender, AName}, Conf).\r\n\r\nget_appenders() ->\r\n    try_msg({get_appenders, ?DEFAULT_LOGGER}).\r\n\r\nget_appenders(Logger) ->\r\n    try_msg({get_appenders, Logger}).\r\n\r\nconf(File) ->\r\n    log4erl_conf:conf(File).\r\n\r\nchange_format(Appender, Format) ->\r\n    try_msg({change_format, ?DEFAULT_LOGGER, Appender, Format}).\r\nchange_format(Logger, Appender, Format) ->\r\n    try_msg({change_format, Logger, Appender, Format}).\r\n\r\nchange_level(Appender, Level) ->\r\n    try_msg({change_level, ?DEFAULT_LOGGER, Appender, Level}).\r\n\r\nchange_level(Logger, Appender, Level) ->\r\n    try_msg({change_level, Logger, Appender, Level}).\r\n\r\nchange_filename(Appender, Filename) ->\r\n    try_msg({change_filename, ?DEFAULT_LOGGER, Appender, Filename}).\r\n\r\nchange_filename(Logger, Appender, Filename) ->\r\n    try_msg({change_filename, Logger, Appender, Filename}).\r\n\r\nerror_logger_handler() ->\r\n    error_logger_log4erl_h:add_handler().\r\n\r\nerror_logger_handler(Args) ->\r\n    error_logger_log4erl_h:add_handler(Args).\r\n\r\n%% For default logger\r\nchange_log_level(Level) ->\r\n    try_msg({change_log_level, ?DEFAULT_LOGGER, Level}).\r\nchange_log_level(Logger, Level) ->\r\n    try_msg({change_log_level, Logger, Level}).\r\n\r\ntry_msg(Msg) ->\r\n     try\r\n\t handle_call(Msg)\r\n     catch\r\n\t exit:{noproc, _M} ->\r\n\t     io:format(\"log4erl has not been initialized yet. To do so, please run~n\"),\r\n\t     io:format(\"> application:start(log4erl).~n\"),\r\n\t     {error, log4erl_not_started};\r\n\t   E:M ->\r\n\t     ?LOG2(\"Error message received by log4erl is ~p:~p~n\",[E, M]),\r\n\t     {E, M}\r\n     end.\r\n\r\nlog(Level, Log) ->\r\n    log(Level, Log, []).\r\nlog(Level, Log, Data) ->\r\n    %filter_log(Level, {log, ?DEFAULT_LOGGER, Level, Log, Data}).\r\n    log(?DEFAULT_LOGGER, Level, Log, Data).\r\nlog(Logger, Level, Log, Data) ->\r\n    filter_log(Level, {log, Logger, Level, Log, Data}).\r\n\r\nfilter_log(Level, Msg) ->\r\n    %case log4erl_utils:to_log(log_filter:cutoff_level(), Level) of\r\n    case log4erl_utils:to_log(Level, log_filter:cutoff_level()) of\r\n        true ->\r\n            try_msg(Msg);\r\n        false ->\r\n\t    ?LOG(\">>--- Message filtered by cutoff logger~n\"),\r\n            ok\r\n    end.\r\n\r\nwarn(Log) ->\r\n    log(warn, Log).\r\n%% If 1st parameter is atom, then it is Logger\r\nwarn(Logger, Log) when is_atom(Logger) ->\r\n    log(Logger, warn, Log, []);\r\nwarn(Log, Data) ->\r\n    log(warn, Log, Data).\r\nwarn(Logger, Log, Data) ->\r\n    log(Logger, warn , Log, Data).\r\n\r\ninfo(Log) ->\r\n    log(info, Log).\r\ninfo(Logger, Log) when is_atom(Logger) ->\r\n    log(Logger, info, Log, []);\r\ninfo(Log, Data) ->\r\n    log(info, Log, Data).\r\ninfo(Logger, Log, Data) ->\r\n    log(Logger, info, Log, Data).\r\n\r\nerror(Log) ->\r\n    log(error, Log).\r\nerror(Logger, Log) when is_atom(Logger) ->\r\n    log(Logger, error, Log, []);\r\nerror(Log, Data) ->\r\n    log(error, Log, Data).\r\nerror(Logger, Log, Data) ->\r\n    log(Logger, error, Log, Data).\r\n\r\nfatal(Log) ->\r\n    log(fatal, Log).\r\nfatal(Logger, Log) when is_atom(Logger) ->\r\n    log(Logger, fatal, Log, []);\r\nfatal(Log, Data) ->\r\n    log(fatal, Log, Data).\r\nfatal(Logger, Log, Data) ->\r\n    log(Logger, fatal, Log, Data).\r\n\r\ndebug(Log) ->\r\n    log(debug, Log).\r\ndebug(Logger, Log) when is_atom(Logger) ->\r\n    log(Logger, debug, Log, []);\r\ndebug(Log, Data) ->\r\n    log(debug, Log, Data).\r\ndebug(Logger, Log, Data) ->\r\n    log(Logger, debug, Log, Data).\r\n\r\nhandle_call({add_logger, Logger}) ->\r\n    log_manager:add_logger(Logger);\r\nhandle_call({add_appender, Logger, Appender, Conf}) ->\r\n    log_manager:add_appender(Logger, Appender, Conf);\r\nhandle_call({get_appenders, Logger}) ->\r\n    gen_event:which_handlers(Logger);\r\nhandle_call({change_level, Logger, Appender, Level}) ->\r\n    log_manager:change_level(Logger, Appender, Level);\r\nhandle_call({change_log_level, Logger, Level}) ->\r\n    log_manager:change_log_level(Logger, Level);\r\nhandle_call({change_format, Logger, Appender, Format}) ->\r\n    log_manager:change_format(Logger, Appender, Format);\r\nhandle_call({change_filename, Logger, Appender, Filename}) ->\r\n    log_manager:change_filename(Logger, Appender, Filename);\r\nhandle_call({log, Logger, Level, Log, Data}) ->\r\n    log_manager:log(Logger, Level, Log, Data).\r\n\r\n"
  },
  {
    "path": "contrib/log4erl/src/log4erl_conf.erl",
    "content": "-module(log4erl_conf).\n\n-export([conf/1]).\n\nleex(File) ->\n    {ok, Bin} = file:read_file(File),\n    F = binary_to_list(Bin),\n    R = log4erl_lex:string(F),\n    case R of\n\t{ok, Tokens, _} ->\n\t    Tokens;\n\tError ->\n\t    io:format(\"Error ~p~n\",[R]),\n\t    erlang:error(Error, [File])\n    end.\n\nparse(Tokens) ->\n    R = log4erl_parser:parse(Tokens),\n    case R of\n\t{ok, T} ->\n\t    T;\n\t{error, {Line, _, EMsg}} ->\n\t    io:format(\"Error in line ~p with message: ~p~n\",[Line, lists:flatten(EMsg)]),\n\t    throw({parser_error, Line, EMsg})\n    end.\n\nconf(File) ->\n    application:start(log4erl),\n    Tree = parse(leex(File)),\n    traverse(Tree).\n\ntraverse([]) ->\n    ok;\ntraverse([H|Tree]) ->\n    element(H),\n    traverse(Tree).\n\nelement({cutoff_level, CutoffLevel}) ->\n    log_filter_codegen:set_cutoff_level(CutoffLevel);\nelement({default_logger, Appenders}) ->\n    appenders(Appenders);\nelement({logger, Logger, Appenders}) ->\n    log4erl:add_logger(Logger),\n    appenders(Logger, Appenders).\n\nappenders([]) ->\n    ok;\nappenders([H|Apps]) ->\n    appender(H),\n    appenders(Apps).\n\nappenders(_, []) ->\n    ok;\nappenders(Logger, [H|Apps]) ->\n    appender(Logger, H),\n    appenders(Logger, Apps).\n\nappender({appender, App, Name, Conf}) ->\n    log4erl:add_appender({App, Name}, {conf, Conf}).\n\nappender(Logger, {appender, App, Name, Conf}) ->\n    log4erl:add_appender(Logger, {App, Name}, {conf, Conf}).\n\n"
  },
  {
    "path": "contrib/log4erl/src/log4erl_lex.xrl",
    "content": "Definitions.\n\nD   = [0-9]\nL   = [A-Z0-9a-z_]\nLS  = [][A-Z0-9a-z_\\s%*$#@().><:/\\-\\+,;:?!]\nWS  = ([\\000-\\s]|%.*)\n\nRules.\n\nlogger   \t: {token,{loger,TokenLine, list_to_atom(TokenChars)}}.\ndefault   \t: {token,{default,TokenLine, list_to_atom(TokenChars)}}.\n{D}+            : {token,{integer,TokenLine,list_to_integer(TokenChars)}}.\n{L}+\t \t: {token,{atom,TokenLine,list_to_atom(TokenChars)}}.\n'{LS}+' \t: {token,{val,TokenLine,strip(TokenChars,TokenLen)}}.\n\"{LS}+\" \t: {token,{val,TokenLine,strip(TokenChars,TokenLen)}}.\n[{},=]     \t: {token,{list_to_atom(TokenChars),TokenLine}}.\n{WS}+\t\t: skip_token.\n\nErlang code.\n\nstrip(TokenChars,TokenLen) -> \n    lists:sublist(TokenChars, 2, TokenLen - 2).\n"
  },
  {
    "path": "contrib/log4erl/src/log4erl_parser.yrl",
    "content": "Nonterminals cutoff_level loggers logger appenders appender props prop value.\n\n% 'loger' is missing 'g' not to be confused with Nonterminals \n% check 'log4erl_conf.xrl'\nTerminals '{' '}' ',' '=' 'loger' 'default' 'integer' 'val' 'atom'.\n\nRootsymbol cutoff_level.\n\ncutoff_level -> loggers : '$1'.\ncutoff_level -> prop loggers : ['$1'] ++ '$2'.\n\nloggers -> logger : ['$1'].\nloggers -> logger loggers : ['$1'] ++ '$2'.\n\nlogger -> loger value '{' appenders '}' : {logger, '$2', '$4'}.\nlogger -> loger '{' appenders '}' : {default_logger, '$3'}.\nlogger -> loger default '{' appenders '}' : {default_logger, '$4'}.\n\nappenders -> appender : ['$1'].\nappenders -> appender appenders : ['$1'] ++ '$2'.\n\nappender -> value value '{' props '}' : {appender, '$1', '$2', '$4'}.\n\nprops -> prop : ['$1'].\nprops -> prop ',' props : ['$1'] ++ '$3'.\n\nprop -> value '=' value : {'$1', '$3'}.\n\nvalue -> val : unwrap('$1').\nvalue -> integer : unwrap('$1').\nvalue -> atom : unwrap('$1').\n\nErlang code.\n\nunwrap({_,_,V}) -> V.\n"
  },
  {
    "path": "contrib/log4erl/src/log4erl_sup.erl",
    "content": "-module(log4erl_sup).\r\n\r\n-author(\"Ahmed Al-Issaei\").\r\n-license(\"MPL-1.1\").\r\n\r\n-behaviour(supervisor).\r\n\r\n-include(\"../include/log4erl.hrl\").\r\n\r\n%% API\r\n-export([start_link/1]).\r\n-export([add_logger/1]).\r\n-export([add_guard/4]).\r\n\r\n%% Supervisor callbacks\r\n-export([init/1]).\r\n\r\nstart_link(Default_logger) ->\r\n    R = supervisor:start_link({local, ?MODULE}, ?MODULE, []),\r\n    %log4erl:start_link(Default_logger),\r\n    add_logger(Default_logger),\r\n    ?LOG2(\"Result in supervisor is ~p~n\",[R]),\r\n    R.\r\n\r\nadd_guard(Logger, Appender, Name, Conf) ->\r\n    C = {Name,\r\n\t {logger_guard, start_link ,[Logger, Appender, Name, Conf]},\r\n\t permanent,\r\n\t 10000,\r\n\t worker,\r\n\t [logger_guard]},\r\n    ?LOG2(\"Adding ~p to ~p~n\",[C, ?MODULE]),\r\n    supervisor:start_child(?MODULE, C).\r\n    \r\nadd_logger(Name) when is_atom(Name) ->\r\n    N = atom_to_list(Name),\r\n    add_logger(N);\r\nadd_logger(Name) when is_list(Name) ->\r\n    C1 = {Name,\r\n\t  {log_manager, start_link ,[Name]},\r\n\t  permanent,\r\n\t  10000,\r\n\t  worker,\r\n\t  [log_manager]},\r\n    \r\n    ?LOG2(\"Adding ~p to ~p~n\",[C1, ?MODULE]),\r\n    supervisor:start_child(?MODULE, C1).\r\n    %add_guard(N2).\r\n\r\n%%======================================\r\n%% supervisor callback functions\r\n%%======================================\r\ninit([]) ->\r\n    ?LOG(\"Starting supervisor~n\"),\r\n    %% No children to be added yet.\r\n    %% The default has to be added from log4erl\r\n    \r\n    % start log4erl gen_server\r\n    %% _Child =  {log4erl_p,\r\n%% \t  {log4erl, start_link ,[Default_logger]},\r\n%% \t  permanent,\r\n%% \t  10000,\r\n%% \t  worker,\r\n%% \t  [log4erl]},\r\n    \r\n    {ok,\r\n     {\r\n       {one_for_one,3,10}, \r\n       []\r\n       %[]\r\n      }\r\n    }.\r\n\r\n\r\n"
  },
  {
    "path": "contrib/log4erl/src/log4erl_utils.erl",
    "content": "-module(log4erl_utils).\r\n\r\n-export([gen_log_txt/1,return_2columns/1, return_Ncolumns/2]).\r\n-export([get_current_time/1, to_log/2]).\r\n-export([get_id/0, get_month_name/1, get_month_long_name/1]).\r\n-export([to_atom/1, to_list/1, to_int/1]).\r\n\r\n% a function to make the log level text look pretty\r\ngen_log_txt(L) when is_list(L) ->\r\n    case string:len(L) of\r\n\t1 ->\r\n\t    \"[\" ++ L ++ \"]    \";\r\n\t2 ->\r\n\t    \"[\" ++ L ++ \"]   \";\r\n\t3 ->\r\n\t    \"[\" ++ L ++ \"]  \";\r\n\t4 ->\r\n\t    \"[\" ++ L ++ \"] \";\r\n\t_ ->\r\n\t    \"[\" ++ L ++ \"]\"\r\n    end.\r\n\r\n% a function to format date/time properly (e.g. 09 instead of 9)\r\nreturn_2columns(X) ->\r\n    return_Ncolumns(X, 2).\r\n\r\nreturn_Ncolumns(X, N) ->\r\n    Len = string:len(X),\r\n    if Len < N ->\r\n\t    string:chars($0, N - Len) ++ X;\r\n\ttrue ->\r\n\t    X\r\n    end.\r\n\r\n% returns date/time as a properly formatted string (e.g. \"01-01-2000 12:12:12\")\r\n%get_current_time() ->\r\nget_current_time({{Y, M, D}, {H, Mi, S}}) ->\r\n    %{{Y, M, D}, {H, Mi, S}} = calendar:local_time(),\r\n    L = lists:map(fun(X) ->\r\n\t\t\t  X2=integer_to_list(X),\r\n\t\t\t  return_2columns(X2)\r\n\t\t  end,\r\n\t\t  [Y, M, D, H, Mi, S]\r\n\t\t ),\r\n    [Y2, M2, D2, H2, Mi2, S2] = L,\r\n    Y2 ++ \"-\" ++ M2 ++ \"-\" ++ D2 ++ \" \" ++ H2 ++ \":\" ++ Mi2 ++ \":\" ++ S2.\r\n\r\n%% DEBUG <- INFO <- WARN <- ERROR <- FATAL\r\n%% user defined levels are always logged\r\nto_log(Cur, Level) ->\r\n    case lists:member(Cur, [debug, info, warn, error, fatal]) of\r\n        true ->\r\n            case Level of\r\n                debug ->\r\n                    true;\r\n                info ->\r\n                    ((Cur == info) or (Cur == warn) or (Cur == error) or (Cur == fatal));\r\n                warn ->\r\n                    ((Cur == warn) or (Cur == error) or (Cur == fatal));\r\n                error ->\r\n                    ((Cur == error) or (Cur == fatal));\r\n                fatal ->\r\n                    (Cur == fatal);\r\n                none ->\r\n                    false;\r\n                _ ->\r\n                    true\r\n            end;\r\n        _ ->\r\n            true\r\n    end.\r\n\r\nget_id() ->\r\n    {_,_,N} = now(),\r\n    Id = \"log4erl_\" ++ integer_to_list(random:uniform(N)),\r\n    list_to_atom(Id).\r\n\r\nget_month_name(Month) ->\r\n    case Month of\r\n\t1 ->\r\n\t    \"Jan\";\r\n\t2 ->\r\n\t    \"Feb\";\r\n\t3 ->\r\n\t    \"Mar\";\r\n\t4 ->\r\n\t    \"Apr\";\r\n\t5 ->\r\n\t    \"May\";\r\n\t6 ->\r\n\t    \"Jun\";\r\n\t7 ->\r\n\t    \"Jul\";\r\n\t8 ->\r\n\t    \"Aug\";\r\n\t9 ->\r\n\t    \"Sep\";\r\n\t10 ->\r\n\t    \"Oct\";\r\n\t11 ->\r\n\t    \"Nov\";\r\n\t12 ->\r\n\t    \"Dec\"\r\n    end.\r\n\r\nget_month_long_name(Month) ->\r\n    case Month of\r\n\t1 ->\r\n\t    \"January\";\r\n\t2 ->\r\n\t    \"February\";\r\n\t3 ->\r\n\t    \"March\";\r\n\t4 ->\r\n\t    \"April\";\r\n\t5 ->\r\n\t    \"May\";\r\n\t6 ->\r\n\t    \"June\";\r\n\t7 ->\r\n\t    \"July\";\r\n\t8 ->\r\n\t    \"August\";\r\n\t9 ->\r\n\t    \"September\";\r\n\t10 ->\r\n\t    \"October\";\r\n\t11 ->\r\n\t    \"November\";\r\n\t12 ->\r\n\t    \"December\"\r\n    end.\r\n\r\nto_list(A) when is_atom(A) ->\r\n    atom_to_list(A);\r\nto_list(A) when is_integer(A) ->\r\n    integer_to_list(A);\r\nto_list(A) when is_binary(A) ->\r\n    binary_to_list(A);\r\nto_list(A) when is_list(A) ->\r\n    A.\r\n\r\nto_atom(A) when is_list(A) ->\r\n    list_to_atom(A);\r\nto_atom(A) when is_binary(A) ->\r\n    list_to_atom(binary_to_list(A));\r\nto_atom(A) when is_atom(A) ->\r\n    A.\r\n\r\nto_int(A) when is_list(A) ->\r\n    list_to_integer(A);\r\nto_int(A) when is_integer(A) ->\r\n    A.\r\n"
  },
  {
    "path": "contrib/log4erl/src/log_filter.erl",
    "content": "%% Author: David Dossot <david@dossot.net>\n%% Created: Oct 25, 2009\n-module(log_filter).\n\n-export([cutoff_level/0]).\n\ncutoff_level() -> all.\n"
  },
  {
    "path": "contrib/log4erl/src/log_filter_codegen.erl",
    "content": "%% Author: David Dossot <david@dossot.net>\n%% Created: Oct 25, 2009\n%% Description: Generates log_filter based on the log4erl cutoff level.\n-module(log_filter_codegen).\n\n-export([reset/0, set_cutoff_level/1]).\n\nreset() ->\n    set_cutoff_level(all).\n\nset_cutoff_level(CutoffLevel) ->\n    LogFilterMod = log_filter_mod(CutoffLevel),\n    compile_and_load_abstract_form(LogFilterMod).\n\n%% Private functions\ncompile_and_load_abstract_form(AbsForm) ->\n    CompRes = compile:forms(AbsForm),\n    {ok, Mod, Code} = CompRes,\n    code:purge(Mod),\n    code:delete(Mod),\n    {module, _} = load_module(Mod, Code),\n    ok.\n\nlog_filter_mod(CutoffLevel) ->\n    [{attribute,1,module,log_filter},\n     {attribute,3,export,[{cutoff_level,0}]},\n     {function,5,cutoff_level,0,[{clause,5,[],[],[{atom,5,CutoffLevel}]}]},\n     {eof,5}].\n"
  },
  {
    "path": "contrib/log4erl/src/log_formatter.erl",
    "content": "-module(log_formatter).\n\n-compile(export_all).\n\n-include(\"../include/log4erl.hrl\").\n\n%-record(log, {level, msg, data, time}).\n\ntest() ->\n    Log = #log{level = warn,\n\t       msg = \"logging message for testing purposes ~p\",\n\t       data = [tt],\n\t       time = calendar:local_time()},\n    Ts = \"[%L] %l%n\",\n    {ok, Tokens} = parse(Ts),\n    T = format(Log, Tokens),\n    io:format(\"~s\",[T]).\n\ntest_dst() ->\n    HourSwitchingFromDST = {{2012,10,28},{2,13,37}},\n    HourSwitchingToDST = {{2013,03,31},{2,13,37}},\n    Log1 = #log{level = warn,\n\t       msg = \"Testing logging during switch FROM DST! ~p\",\n\t       data = [tt],\n\t       time = HourSwitchingFromDST,\n               millis = 111},\n    Log2 = #log{level = warn,\n\t       msg = \"Testing logging during switch TO DST! ~p\",\n\t       data = [tt],\n\t       time = HourSwitchingToDST,\n               millis = 111},\n    FormatTokens = fun(Ts) ->\n                           {ok, Tokens} = parse(Ts),\n                           T1 = format(Log1, Tokens),\n                           T2 = format(Log2, Tokens),\n                           io:format(\"~s\",[T1]),\n                           io:format(\"~s\",[T2])\n                   end,\n    lists:map(FormatTokens, [\"[%I] %l%n\", %% iso_format\n                             \"[%S] %l%n\", %% iso_format2\n                             \"[%Z] %l%n\"  %% time_zone\n                            ]),\n    ok.\n\ntest2(Num) ->\n    Ts = \"%j %T [%L] - %l\",\n    {ok, Tokens} = parse(Ts),\n    Logs = lists:map(fun(X) ->\n\t\t\t     {X, make_log(warn, \"testing a lot\", [])}\n\t\t     end,\n\t\t     lists:seq(1, Num)),\n    lists:map(fun(X) ->\n\t\t      N = element(1,X),\n\t\t      X1 = element(2, X),\n\t\t      X2 = format(X1, Tokens),\n\t\t      io:format(\"~p: ~s~n\",[N, X2])\n\t      end, Logs),\n    ok.\n\nmake_log(Level, Msg, Data) ->\n    #log{level = Level,\n\t msg = Msg,\n\t data = Data,\n\t time = calendar:local_time()}.\n\n\n%%%%%%%%%%%%%%%%%%%\n%%  functions\n%%%%%%%%%%%%%%%%%%%\nformat(Log, Tokens) ->\n    ?LOG2(\"log_formatter formatting log: ~p~n\",[Log]),\n    F = fun(X) ->\n\t\t?LOG2(\"X == ~p and Log = ~p~n\",[X,Log]),\n\t\tM = get_token_value(X,Log),\n\t\tM\n\tend,\n    L = lists:map(F, Tokens),\n    L.\n\nget_token_value(date, Log) ->\n    D = Log#log.time,\n    {{Y, M, Dd},_} = D,\n    [C,B,A] = lists:map(\n\t\tfun(X) ->\n\t\t\tinteger_to_list(X)\n\t\tend,\n\t\t[Y,M,Dd]),\n    Res = A ++ \"-\" ++ B ++ \"-\" ++ C,\n    Res;\nget_token_value(date2, Log) ->\n    D = Log#log.time,\n    {{Y, M, Dd},_} = D,\n    [A,B,C] = lists:map(\n\t\tfun(X) ->\n\t\t\tX2 = integer_to_list(X),\n\t\t\tlog4erl_utils:return_2columns(X2)\n\t\tend,\n\t\t[Y,M,Dd]),\n    Res = A ++ \"-\" ++ B ++ \"-\" ++ C,\n    Res;\nget_token_value(time, Log) ->\n    D = Log#log.time,\n    {_,{H, M, S}} = D,\n    [A,B,C] = lists:map(\n\t\tfun(X) ->\n\t\t\tlog4erl_utils:return_2columns(integer_to_list(X))\n\t\tend,\n\t\t[H,M,S]),    \n    Res = A ++ \":\" ++ B ++ \":\" ++ C,\n    Res;\nget_token_value(time2, Log) ->\n    D = Log#log.time,\n    Ms = Log#log.millis,\n    {_,{H, M, S}} = D,\n    [A,B,C] = lists:map(\n\t\tfun(X) ->\n\t\t\tX2 = integer_to_list(X),\n\t\t\tlog4erl_utils:return_2columns(X2)\n\t\tend,\n\t\t[H,M,S]),\n    E = log4erl_utils:return_Ncolumns(integer_to_list(Ms), 3),\n    Res = A ++ \":\" ++ B ++ \":\" ++ C ++ \".\" ++ E,\n    Res;\nget_token_value(year4, Log) ->\n    D = Log#log.time,\n    {{Y, _,_},_} = D,\n    integer_to_list(Y);\nget_token_value(year2, Log) ->\n    D = Log#log.time,\n    {{Y, _,_},_} = D,\n    L = integer_to_list(Y),\n    string:substr(L,3,2);\nget_token_value(month, Log) ->\n    {{_, M,_},_} = Log#log.time,\n    integer_to_list(M);\nget_token_value(month2, Log) ->\n    {{_,M,_},_} = Log#log.time,\n    log4erl_utils:get_month_name(M);\nget_token_value(month3, Log) ->\n    {{_, M,_},_} = Log#log.time,\n    log4erl_utils:get_month_long_name(M);\nget_token_value(day, Log) ->\n    D = Log#log.time,\n    {{_, _,Dd},_} = D,\n    integer_to_list(Dd);\nget_token_value(hour, Log) ->\n    D = Log#log.time,\n    {_,{H, _,_}} = D,\n    integer_to_list(H);\nget_token_value(minute, Log) ->\n    D = Log#log.time,\n    {_,{_, M,_}} = D,\n    integer_to_list(M);\nget_token_value(second, Log) ->\n    D = Log#log.time,\n    {_,{_, _,S}} = D,\n    integer_to_list(S);\nget_token_value(millis, Log) ->\n    Ms = Log#log.millis,\n    log4erl_utils:return_Ncolumns(integer_to_list(Ms), 3);\nget_token_value(log, Log) ->\n    Msg = Log#log.msg,\n    Data = Log#log.data,\n    io_lib:format(Msg, Data);\nget_token_value(level, Log) ->\n    atom_to_list(Log#log.level);\nget_token_value(new_line, _Log) ->\n    \"\\n\";\n% GMT TZ\nget_token_value(iso_format, Log) ->\n    Date = local_time_to_pseudo_universal_time_dst(Log#log.time),\n    get_token_value(date2, Log) ++ \"T\" ++\n        get_token_value(time2, Log#log{time=Date}) ++ \"Z\";\n% With TZ\nget_token_value(iso_format2, Log) ->\n    Date = Log#log.time,\n    UD = local_time_to_pseudo_universal_time_dst(Date),\n    Ds = calendar:datetime_to_gregorian_seconds(Date),\n    UDs = calendar:datetime_to_gregorian_seconds(UD),\n    TZ = case Ds-UDs > 0 of\n\t     true ->\n\t\t {_,{A,B,_}} = calendar:gregorian_seconds_to_datetime(Ds-UDs),\n\t\t A2 = log4erl_utils:return_2columns(integer_to_list(A)),\n\t\t B2 = log4erl_utils:return_2columns(integer_to_list(B)),\n\t\t \"+\" ++ A2 ++ \":\" ++ B2;\n\t     _ ->\n\t\t {_,{C,D,_}} = calendar:gregorian_seconds_to_datetime(UDs-Ds),\n\t\t C2 = log4erl_utils:return_2columns(integer_to_list(C)),\n\t\t D2 = log4erl_utils:return_2columns(integer_to_list(D)),\n\t\t \"-\" ++ C2 ++ \":\" ++ D2\n\t end,\n    get_token_value(date2, Log) ++ \"T\" ++ get_token_value(time2, Log) ++ TZ;\n% TZ only\nget_token_value(time_zone, Log) ->\n    Date = Log#log.time,\n    UD = local_time_to_pseudo_universal_time_dst(Date),\n    Ds = calendar:datetime_to_gregorian_seconds(Date),\n    UDs = calendar:datetime_to_gregorian_seconds(UD),\n    TZ = case Ds-UDs > 0 of\n\t     true ->\n\t\t {_,{A,B,_}} = calendar:gregorian_seconds_to_datetime(Ds-UDs),\n\t\t A2 = log4erl_utils:return_2columns(integer_to_list(A)),\n\t\t B2 = log4erl_utils:return_2columns(integer_to_list(B)),\n\t\t \"+\" ++ A2 ++ \":\" ++ B2;\n\t     _ ->\n\t\t {_,{C,D,_}} = calendar:gregorian_seconds_to_datetime(UDs-Ds),\n\t\t C2 = log4erl_utils:return_2columns(integer_to_list(C)),\n\t\t D2 = log4erl_utils:return_2columns(integer_to_list(D)),\n\t\t \"-\" ++ C2 ++ \":\" ++ D2\n\t end,\n    TZ;\nget_token_value(A, _Log) ->\n    A.\n\n%%%%%%%%%%%%%%%%%%%\n%% parse functions\n%%%%%%%%%%%%%%%%%%%\nparse(M) ->\n    ?LOG2(\"log_formatter parsing ~p~n\",[M]),\n    try\n\tTokens = parse2(M,[]),\n\t?LOG2(\"Received tokens ~p~n\",[Tokens]),\n\t{ok, Tokens}\n    catch\n\tE:R ->\n\t    {error, {E,R}}\n    end.\n\nparse2([], Acc) ->\n    lists:reverse(Acc);\nparse2([$\\% | R], Acc) ->\n    [S|R2] = R,\n    T = parse_char(S),\n    parse2(R2,[T|Acc]);\nparse2([S | R], Acc) ->\n    parse2(R, [S|Acc]).\n\nparse_char($d) ->\n    date;\nparse_char($j) ->\n    date2;\nparse_char($t) ->\n    time;\nparse_char($T) ->\n    time2;\nparse_char($y) ->\n    year2;\nparse_char($Y) ->\n    year4;\n% 1\nparse_char($M) ->\n    month;\n% Jan\nparse_char($b) ->\n    month2;\n% January\nparse_char($B) ->\n    month3;\nparse_char($D) ->\n    day;\nparse_char($h) ->\n    hour;\nparse_char($m) ->\n    minute;\nparse_char($s) ->\n    second;\nparse_char($l) ->\n    log;\nparse_char($L) ->\n    level;\nparse_char($n) ->\n    new_line;\nparse_char($i) ->\n    millis;\nparse_char($I) ->\n    iso_format;\nparse_char($S) ->\n    iso_format2;\nparse_char($Z) ->\n    time_zone;\nparse_char(C) ->\n    C.\n\n%% Always log a time, even though corresponding UTC time during periods\n%% switching TO or FROM DST may not make sense.\nlocal_time_to_pseudo_universal_time_dst(DateTime1) ->\n    case calendar:local_time_to_universal_time_dst(DateTime1) of\n        [] -> DateTime1;\n        [DstDateTimeUTC, _DateTimeUTC] -> DstDateTimeUTC;\n        [DateTimeUTC] -> DateTimeUTC\n    end.\n"
  },
  {
    "path": "contrib/log4erl/src/log_manager.erl",
    "content": "-module(log_manager).\r\n\r\n-author(\"Ahmed Al-Issaei\").\r\n-license(\"MPL-1.1\").\r\n\r\n-include(\"../include/log4erl.hrl\").\r\n\r\n%% API\r\n-export([start_link/1]).\r\n-export([change_log_level/2]).\r\n-export([change_level/3]).\r\n-export([change_filename/3]).\r\n-export([add_logger/1]).\r\n-export([add_appender/3]).\r\n-export([change_format/3]).\r\n-export([log/4]).\r\n%% Don't want compile-warnings about the use of erlang:now/0 in\r\n%% this module.\r\n-compile(nowarn_deprecated_function).\r\n\r\nstart_link(Logger) when is_atom(Logger) ->\r\n    ?LOG2(\"log_manager adding Logger ~p~n\",[Logger]),\r\n    gen_event:start_link({local, Logger});\r\nstart_link(Logger) when is_list(Logger) ->\r\n    ?LOG2(\"log_manager adding Logger ~p~n\",[Logger]),    \r\n    gen_event:start_link({local, list_to_atom(Logger)}).\r\n\r\nadd_logger(Logger) ->\r\n    log4erl_sup:add_logger(Logger).\r\n\r\nadd_appender(Logger, {Appender, Name} , Conf) ->\r\n    ?LOG2(\"add_appender ~p with name ~p to ~p with Conf ~p ~n\",[Appender, Name, Logger, Conf]),\r\n    log4erl_sup:add_guard(Logger, Appender, Name, Conf).\r\n    \r\nchange_log_level(Logger, Level) ->\r\n    notify_logger(Logger, {change_level, Level}).\r\n\r\nchange_level(Logger, Appender, Level) ->\r\n    call_appender(Logger, Appender, {change_level, Level}).\r\n\r\nchange_format(Logger, Appender, Format) ->\r\n    call_appender(Logger, Appender, {change_format, Format}).\r\n\r\nchange_filename(Logger, Appender, Filename) ->\r\n    call_appender(Logger, Appender, {change_filename, Filename}).\r\n\r\n%%--------------------------------------------------------------------\r\n%% Logger API functions\r\n%%--------------------------------------------------------------------\r\nlog(Logger, Level, Log, Data) ->\r\n    Now = {_, _, Ms} = timestamp(),\r\n    T = calendar:now_to_local_time(Now),\r\n    ?LOG2(\"Logging:~n ~p ~p ~p ~p~n\",[Logger, Level, Log, Data]),\r\n    LL = #log{level=Level, msg=Log, data=Data, time=T, millis = Ms div 1000},\r\n    notify_logger(Logger, {log, LL}).\r\n\r\nnotify_logger(Logger, Msg) ->\r\n    try\r\n\tgen_event:sync_notify(Logger, Msg)\r\n    catch\r\n\texit:noproc ->\r\n\t    {error, no_such_logger}\r\n    end.\r\n\r\ncall_appender(Logger, Appender, Msg) ->\r\n    try\r\n\tApps = gen_event:which_handlers(Logger),\r\n\t?LOG2(\"log_manager:call_appender/3 get apps ~p~n\",[Apps]),\r\n\tcase lists:filter(fun({_,X}) -> X =:= Appender end, Apps) of\r\n\t    [] ->\r\n\t\t{error, no_appender_found};\r\n\t    [Apps1] ->\r\n\t\t?LOG2(\"get apps: ~p~n\",[Apps1]),\r\n\t\tgen_event:call(Logger, Apps1, Msg)\r\n\tend\r\n    catch\r\n\texit:noproc ->\r\n\t    {error, no_such_logger}\r\n    end.\r\n    \r\n    \r\ntimestamp() ->\r\n    erlang:now().\r\n"
  },
  {
    "path": "contrib/log4erl/src/logger_guard.erl",
    "content": "-module(logger_guard).\r\n\r\n-author(\"Ahmed Al-Issaei\").\r\n-license(\"MPL-1.1\").\r\n\r\n-behaviour(gen_server).\r\n\r\n-include(\"../include/log4erl.hrl\").\r\n\r\n%% API\r\n-export([start_link/4, add_sup_handler/3]).\r\n\r\n%% gen_server callbacks\r\n-export([init/1, handle_call/3, handle_cast/2, handle_info/2,\r\n\t terminate/2, code_change/3]).\r\n\r\nstart_link(Logger, Appender, Name, Conf) ->\r\n    %?LOG2(\"starting guard for logger ~p~n\",[Logger]),\r\n    {ok, Pid} = gen_server:start_link(?MODULE, [Appender, Name], []),\r\n    case add_sup_handler(Pid, Logger, Conf) of\r\n\t{error, E} ->\r\n\t    gen_server:call(Pid, stop),\r\n\t    {error, E};\r\n\t_R ->\r\n\t    {ok, Pid}\r\n    end.\r\n\r\nadd_sup_handler(G_pid, Logger, Conf) ->\r\n    ?LOG(\"add_sup()~n\"),\r\n    gen_server:call(G_pid, {add_sup_handler, Logger, Conf}).\r\n\r\n%%=========================================\r\n%% gen_server callback functions\r\n%%=========================================\r\ninit([Appender, Name]) ->\r\n    ?LOG2(\"Starting guard ~n\",[]),\r\n    {ok, [{appender, Appender, Name}]}.\r\n\r\nhandle_call({add_sup_handler, Logger, Conf}, _From, [{appender, Appender, Name}] = State) ->\r\n    ?LOG2(\"Adding handler ~p with name ~p for ~p From ~p~n\",[Appender, Name, Logger, _From]),\r\n    try\r\n\tRes = gen_event:add_sup_handler(Logger, {Appender, Name}, Conf),\r\n\t{reply, Res, State}\r\n    catch\r\n\tE:R ->\r\n\t    {reply, {error, {E,R}}, State}\r\n    end;\r\nhandle_call(stop, _From, State) ->\r\n    {stop, stop, ok, State};\r\nhandle_call(_Msg, _From, State) ->\r\n    {reply, ok, State}.\r\n\r\nhandle_cast(_Msg, State) ->\r\n    {noreply, State}.\r\n\r\nhandle_info({gen_event_EXIT, _Mod, R}, State) ->\r\n    ?LOG2(\"logger_guard received exit message. Terminating ~p~n\",[now]),\r\n    {stop, {appender_died, R}, State};\r\nhandle_info(_Info, [{appender, _Appender, _Name}] = State) ->\r\n    ?LOG2(\"logger_guard received msg ~p for appender ~p with name ~p~n \", [_Info, _Appender, _Name]),\r\n    {noreply, State}.\r\n\r\nterminate(_Reason, _State) ->\r\n    ok.\r\n\r\ncode_change(_OldVsn, State, _Extra) ->\r\n    {ok, State}.\r\n\r\n"
  },
  {
    "path": "contrib/log4erl/src/smtp_appender.erl",
    "content": "-module(smtp_appender).\r\n\r\n-include(\"../include/log4erl.hrl\").\r\n\r\n-behaviour(gen_event).\r\n%% gen_event callbacks\r\n-export([init/1, handle_event/2, handle_call/2, \r\n\t handle_info/2, terminate/2, code_change/3]).\r\n\r\n-define(DEFAULT_TITLE, \"log\").\r\n-define(DEFAULT_PORT, 25).\r\n\r\ninit({conf, Conf}) when is_list(Conf) ->\r\n    Level = proplists:get_value(level, Conf),\r\n    Server = lists:foldl(fun(X, List) ->\r\n\t\t\t     [proplists:get_value(X,Conf)|List]\r\n\t\t     end,\r\n\t\t     [],\r\n\t\t     [ip, port]),\r\n    Auth = lists:foldl(fun(X, List) ->\r\n\t\t\t     [proplists:get_value(X,Conf)|List]\r\n\t\t     end,\r\n\t\t     [],\r\n\t\t     [username, password, no_auth]),\r\n\r\n    SRes = case hd(Server) of\r\n\t       undefined ->\r\n\t\t   % in case port is not supplied\r\n\t\t   [_|S2] = Server,\r\n\t\t   list_to_tuple(lists:reverse(S2));\r\n\t       _ ->\r\n\t\t   list_to_tuple(lists:reverse(Server))\r\n\t   end,\r\n    ARes = case hd(Auth) of\r\n\t       undefined ->\r\n\t\t   % in case no_auth not true\r\n\t\t   [_|A2] = Auth,\r\n\t\t   list_to_tuple(lists:reverse(A2));\r\n\t       _ ->\r\n\t\t   % no_auth has priority\r\n\t\t   no_auth\r\n\t   end,\r\n    MRes = msg_conf(Conf),\r\n    init({Level, SRes, ARes, MRes});\r\n%% Server = {Ip, Port} | {Ip} | Ip\r\n%% Auth = {Uname, Pass} | Uname | {Uname} | no_auth\r\n%% MsgInfo = {From, To, Title, Msg} \r\n%%         | {To, Title, Msg}\r\n%%         | {To, Msg}\r\ninit({Level, Server, Auth, MsgInfo}) ->\r\n    Srvr = check_opts(get_srv_opts(Server)),\r\n    Auth2 = check_opts(get_auth_opts(Auth)),\r\n    MsgI = check_opts(get_msg_opts(MsgInfo)),\r\n    State = #smtp_appender{level=Level, srvr_opts=Srvr, auth_opts=Auth2, msg_opts=MsgI},\r\n    {ok, State};\r\ninit(File) when is_list(File) ->\r\n    case file:consult(File) of\r\n\t{error, Reason} ->\r\n\t    error_logger:error_msg(\"smtp_appender: couldn't consult Conf file~n\"),\r\n\t    {error, file:format_error(Reason)};\r\n\t{ok, [Terms]} ->\r\n\t    init(Terms)\r\n    end.\r\n\r\nhandle_event({change_level, Level}, State) ->\r\n    State2 = State#smtp_appender{level = Level},\r\n    ?LOG2(\"Changed level to ~p~n\",[Level]),\r\n    {ok, State2};\r\nhandle_event({log,LLog}, State) ->\r\n    ?LOG2(\"handl_event:log = ~p~n\",[LLog]),\r\n    do_log(LLog, State),\r\n    {ok, State}.\r\n\r\nhandle_call({change_format, Format}, State) ->\r\n    ?LOG2(\"Old State in console_appender is ~p~n\",[State]),\r\n    {ok, Tokens} = log_formatter:parse(Format),\r\n    ?LOG2(\"Adding format of ~p~n\",[Tokens]),\r\n    S = State#smtp_appender.msg_opts,\r\n    S2 = S#msg_opts{msg=Tokens},\r\n    State2 = State#smtp_appender{msg_opts=S2},\r\n    {ok, ok, State2};\r\nhandle_call({change_level, Level}, State) ->\r\n    State2 = State#smtp_appender{level = Level},\r\n    ?LOG2(\"Changed level to ~p~n\",[Level]),\r\n    {ok, ok, State2};\r\nhandle_call(_Request, State) ->\r\n    {ok, ok, State}.\r\n\r\nhandle_info(_Info, State) ->\r\n    {ok, State}.\r\n\r\nterminate(_Reason, _S) ->\r\n    ok.\r\n\r\ncode_change(_OldVsn, State, _Extra) ->\r\n    {ok, State}.\r\n\r\ndo_log(#log{level = L} = Log, #smtp_appender{level=Level} = State) ->\r\n    ToLog = log4erl_utils:to_log(L, Level),\r\n    case ToLog of\r\n\ttrue ->\r\n\t    MsgOpts = State#smtp_appender.msg_opts,\r\n\t    \r\n\t    From = MsgOpts#msg_opts.from,\r\n\t    To = MsgOpts#msg_opts.to,\r\n\t    Title = MsgOpts#msg_opts.title,\r\n\t    Msg = log_formatter:format(Log, MsgOpts#msg_opts.msg),\r\n\t    M = email_msg:simp_msg(From, To, Title, Msg),\r\n\t    \r\n\t    SrvrOpts = State#smtp_appender.srvr_opts,\r\n\t    AuthOpts = State#smtp_appender.auth_opts,\r\n\t    \r\n\t    Ip = SrvrOpts#srvr_opts.ip,\r\n\t    Port = SrvrOpts#srvr_opts.port,\r\n\t    Uname = AuthOpts#auth_opts.username,\r\n\t    Pass = AuthOpts#auth_opts.password,\r\n\t    Pid = init_smtp(Ip, Port, Uname, Pass),\r\n\t    \r\n\t    smtp_fsm:sendemail(Pid, From, To, M);\r\n\tfalse ->\r\n\t    ok\r\n    end.\r\n\r\ninit_smtp(Ip, Port, Uname, Password) ->\r\n    {ok, Pid} = smtp_fsm:start(Ip, Port),\r\n    smtp_fsm:ehlo(Pid),\r\n    %smtp_fsm:features(Pid),\r\n    case {Uname, Password} of\r\n\t{undefined, undefined} ->\r\n\t    ok;\r\n\t_ ->\r\n\t    smtp_fsm:login(Pid, Uname, Password)\r\n    end,\r\n    Pid.\r\n\t    \r\n\r\nget_srv_opts({Ip, Port}) ->\r\n    #srvr_opts{ip=Ip, port=Port};\r\nget_srv_opts({Ip}) when is_list(Ip) ->\r\n    #srvr_opts{ip=Ip, port=?DEFAULT_PORT};\r\nget_srv_opts(Ip) when is_list(Ip) ->\r\n    #srvr_opts{ip=Ip, port=?DEFAULT_PORT};\r\nget_srv_opts(E) ->\r\n    {error, E}.\r\n\r\nget_auth_opts({Uname, Pass}) ->\r\n    #auth_opts{username=Uname, password=Pass};\r\nget_auth_opts(no_auth) ->\r\n    #auth_opts{};\r\nget_auth_opts({Uname}) ->\r\n    #auth_opts{username=Uname};\r\nget_auth_opts(Uname) when is_list(Uname) ->\r\n    #auth_opts{username=Uname};\r\nget_auth_opts(E) ->\r\n    {error, E}.\r\n\r\nget_msg_opts({From, To, Title, Msg}) ->\r\n    {ok, Mtokens} = log_formatter:parse(Msg),\r\n    #msg_opts{from=From, to=To, title=Title, msg=Mtokens};\r\nget_msg_opts({To, Title, Msg}) ->\r\n    {ok, Mtokens} = log_formatter:parse(Msg),\r\n    #msg_opts{to=To, title=Title, msg=Mtokens};\r\nget_msg_opts({To, Msg}) ->\r\n    {ok, Mtokens} = log_formatter:parse(Msg),\r\n    #msg_opts{to=To, msg=Mtokens};\r\nget_msg_opts(E) ->\r\n    {error, E}.\r\n\r\ncheck_opts(Opts) ->\r\n    case Opts of\r\n\t{error, E} ->\r\n\t    ?LOG2(\"error in getting opts with param ~p~n\",[Opts]),\r\n\t    throw({smtp_appender_opts, E});\r\n\tR ->\r\n\t    R\r\n    end.\r\n\r\nmsg_conf(Conf) ->\r\n    To = proplists:get_value(to, Conf),\r\n    Msg = proplists:get_value(msg, Conf),\r\n\r\n    case {To, Msg} of\r\n\t{undefined, _} ->\r\n\t    throw(smtp_to_null);\r\n\t{_, undefined} ->\r\n\t    throw(smtp_msg_null);\r\n\t{_,_} ->\r\n\t    ok\r\n    end,\r\n\r\n    Title = proplists:get_value(title, Conf),\r\n    From = proplists:get_value(from, Conf),\r\n\r\n    case {Title, From} of\r\n\t{undefined, _} ->\r\n\t    {To, Msg};\r\n\t{_, undefined} ->\r\n\t    {To, Title, Msg};\r\n\t{_, _} ->\r\n\t    {From, To, Title, Msg}\r\n    end.\r\n"
  },
  {
    "path": "contrib/log4erl/src/smtp_fsm.erl",
    "content": "%%\n%% file: smtp_fsm.erl\n%% author: Michael Bradford <michael.bradford@t-mobile.uk.net>\n%% description: simple smtp client using gen_fsm behaviour\n%%\n%% this client has been\n%% fully tested against exim 3.36\n%% partially tested against sendmail (apart from auth)\n%%\n%% 0.2 adds md5 authentication\n%% 0.3 included revisions to style by Francesco\n%% 0.4 fixes md5 authentication\n%% 1.0 is first release, including\n%%  extra fixes for MD5\n%%  better error messages for sendemail/4\n%% 1.1 fixes mail from & rcpt to bugs where they weren't\n%%  strictly standards compliant\n%%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Usage Example\n%%\n%% 6> c(smtp_fsm).\n%% ./smtp_fsm.erl:15: Warning: undefined call-back function\n%%   code_change/4\n%% ./smtp_fsm.erl:15: Warning: undefined call-back function\n%%   handle_info/3\n%% {ok,smtp_fsm}\n%% 7> {ok,Pid} = smtp_fsm:start(\"172.24.75.203\").\n%% connecting to \"172.24.75.203\"\n%% socket open\n%% {ok,<0.45.0>}\n%% 8> smtp_fsm:ehlo(Pid).\n%% {ok,\"250-smtp.test.co.uk Hello test.test.co.uk [172.24.72.71]\n%%   \\r\\n250-SIZE\\r\\n250-PIPELINING\\r\\n250-AUTH PLAIN LOGIN\n%%   CRAM-MD5\\r\\n250-HELP\\r\\n\"}\n%% 9> smtp_fsm:features(Pid).\n%% {ok,[\"SIZE\",\"PIPELINING\",\"AUTH PLAIN LOGIN CRAM-MD5\",\"HELP\"]}\n%% 10> smtp_fsm:login(Pid,\"mike\",\"secret\").\n%% {ok,\"235 Authentication succeeded\\r\\n\"}\n%% 11> smtp_fsm:sendemail(Pid,\"michael.bradford@t-mobile.uk.net\",\n%%   \"test@test.co.uk\", Msg).\n%% ok\n%% 12> smtp_fsm:close(Pid).\n%% ok\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%\n-module(smtp_fsm).\n-behaviour(gen_fsm).\n-vsn('v1.2').\n\n%% Client Functions\n-export([start/0, start/1, start/2, ehlo/1, ehlo/2, helo/1,\n     helo/2,close/1,noop/1,rset/1, mail_from/2,\n     rcpt_to/2,message/2,sendemail/4,features/1,\n     login/3,plain_login/3,login_login/3,md5_login/3]).\n\n%% states\n-export([smtp_start/3, smtp_conn/3, smtp_data/3,\n     smtp_login/3, smtp_md5/3]).\n\n%% call back behaviours\n-export([handle_event/3, handle_sync_event/4, init/1,\n        code_change/4, handle_info/3,\n     terminate/3]).\n\n%% testing\n-export([]).\n\n-record(info, {socket, features}).\n-define(AUTHLIST, [ \"CRAM-MD5\", \"PLAIN\", \"LOGIN\"]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% internal functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% works out fully qualified domain name for host\ngetFQDN()->\n    {ok, Hostname} = inet:gethostname(),\n    {ok, HostEnt} = inet:gethostbyname(Hostname),\n    {hostent,Fqdn,_,inet,_,_IP_addrs} = HostEnt,\n    Fqdn.\n\n%% encodes user & password as required by PLAIN AUTH method\nplain_encode(User, Pwd) ->\n    Plain_str = lists:concat([User,\"\\0\",User,\"\\0\",Pwd]),\n    base64:encode(Plain_str).\n\n%% extracts & decodes server challenge from server response\n%% used in MD5 AUTH\ndecode_challenge(Resp) ->\n    Str = string:substr(Resp,5,string:len(Resp)-6),\n    base64:decode(string:strip(Str)).\n\n%% returns a hex digest of a binary value\nhexdigest(Binary) ->\n    List = binary_to_list(Binary),\n    Hexlist = lists:map(fun hexit/1,List),\n    string:to_lower(lists:concat(Hexlist)).\n\n%% returns hex value of integer (always) as 2 chr string\nhexit(Int) ->\n    Hex = httpd_util:integer_to_hexlist(Int),\n    case string:len(Hex) of\n    1 -> \"0\" ++ Hex;\n    _ -> Hex\n    end.\n\n%% generates MD5 mac and returns as hex string\nmd5_hmac(Challenge,Pwd) ->\n    crypto:start(),\n    Md5_bin = crypto:hmac(md5, Pwd, Challenge),\n    crypto:stop(),\n    hexdigest(Md5_bin).\n\n%% selects the preferred AUTH method supported\npref_auth(Supports)->\n    pref_auth(?AUTHLIST, Supports).\npref_auth([H|T], Supports) ->\n    case lists:member(H, Supports) of\n    true  -> {ok, H};\n    false -> pref_auth(T,Supports)\n    end;\npref_auth([], _Supports) ->\n    {error, none}.\n\n%% checks Features list if AUTH is supported and returns\n%% a list of methods\ndoes_auth([\"AUTH\"++Supp|_]) -> string:tokens(Supp, \" \");\ndoes_auth([_Other|Tail])     -> does_auth(Tail);\ndoes_auth([])                -> no.\n\n%% listens for smtp response & parses to get code\nget_response(Socket) ->\n    receive\n    {tcp, Socket, Resp} ->\n        {lists:sublist(Resp, 3), Resp};\n    {tcp_error, Socket, Reason} ->\n        {conn_error, Reason};\n    {tcp_closed,Socket} ->\n        conn_closed\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% interface\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% use start/ to connect to the SMTP server & port\n%% default port (if undefined) is 25\n%% default host (if undefined) is localhost\nstart()          -> start(\"localhost\", 25).\nstart(Host)      -> start(Host, 25).\nstart(Host,Port) -> gen_fsm:start(?MODULE,[Host,Port],[]).\n\n%% use helo/ when you want to use SMTP\n%% if the client name is undefined then the FQDN is used\nhelo(Fsm)      -> helo(Fsm, getFQDN()).\nhelo(Fsm,Name) -> gen_fsm:sync_send_event(Fsm, {helo,Name}).\n\n%% use ehlo/ when you want to use ESMTP\n%% if the client name is undefined then the FQDN is used\nehlo(Fsm)      -> ehlo(Fsm, getFQDN()).\nehlo(Fsm,Name) -> gen_fsm:sync_send_event(Fsm,{ehlo,Name}).\n\n%% use sendemail/4 if you can't be bothered sending the\n%% individual SMTP commands\nsendemail(Fsm,From,To,Message) ->\n    case mail_from(Fsm,From) of\n    {ok, _Resp}         -> sendemail(Fsm,To,Message);\n    {mfrom_error, _} = Resp ->\n        io:format(\"mfrom_error ~p~n\",[Resp]),\n        rset(Fsm),\n        Resp;\n    Resp                -> Resp\n    end.\n%% sendemail/3 only used by sendemail/4\nsendemail(Fsm,To,Message) ->\n    case rcpt_to(Fsm,To) of\n    {ok, _Resp}        -> sendemail(Fsm,Message);\n    {rcpt_error, _} = Resp ->\n        io:format(\"rcpt_error ~p~n\",[Resp]),\n        rset(Fsm),\n        Resp;\n    Resp               -> Resp\n    end.\n%% sendemail/2 only used by sendemail/3\nsendemail(Fsm,Message) ->\n    case message(Fsm,Message) of\n    {ok, _Resp}        -> ok;\n    {data_error, _} = Resp ->\n        io:format(\"data_error ~p~n\",[Resp]),\n        rset(Fsm),\n        Resp;\n    Resp               -> Resp\n    end.\n\n%% individual functions used for sending emails\n%%\n%% use mail_from/2 to send SMTP mail from:\nmail_from(Fsm,Address) ->\n    gen_fsm:sync_send_event(Fsm,{mfrom,Address}).\n%% use rcpt_to/2 to send SMTP rcpt to:\nrcpt_to(Fsm,Address) ->\n    gen_fsm:sync_send_event(Fsm,{rcpt_to,Address}).\n%% use message/2 to send email content\nmessage(Fsm,Message) ->\n    case gen_fsm:sync_send_event(Fsm,data) of\n    {ok, _Resp} -> msg(Fsm,Message);\n    Resp -> Resp\n    end.\n%% msg/2 only used by message/2\nmsg(Fsm,Message) ->\n    gen_fsm:sync_send_event(Fsm,{msg,Message}).\n\n%% login methods\n%%\n%% use login/3 if you want the smtp client to choose the\n%% AUTH method\nlogin(Fsm,User,Pwd) ->\n    {ok, Features} = features(Fsm),\n    case does_auth(Features) of\n    no       -> {auth_error, \"AUTH not supported\"};\n    Supports -> choose_login(Fsm,User,Pwd,Supports)\n    end.\n%% choose_login/4 only used by login/3\nchoose_login(Fsm,User,Pwd,Supports) ->\n    case pref_auth(Supports) of\n    {ok, \"CRAM-MD5\"}-> md5_login(Fsm,User,Pwd);\n    {ok, \"PLAIN\"}   -> plain_login(Fsm,User,Pwd);\n    {ok, \"LOGIN\"}   -> login_login(Fsm,User,Pwd);\n    {error, Error}  -> {error, Error}\n    end.\n\n%% interfaces for plain, login & md5 authentication\n%% use plain_login/3 for SMTP AUTH PLAIN\nplain_login(Fsm,User,Pwd) ->\n    gen_fsm:sync_send_event(Fsm,{plain_login,User,Pwd}).\n\n%% use login_login/3 for SMTP AUTH LOGIN\nlogin_login(Fsm,User,Pwd) ->\n    %% send the user name\n    case gen_fsm:sync_send_event(Fsm,{login_login,User}) of\n    {ok, _Resp} -> login_login(Fsm,Pwd);\n    Resp        -> Resp\n    end.\n%% login_login/2 only used by login_login/3\nlogin_login(Fsm,Pwd) ->\n    %% then send the password\n    case gen_fsm:sync_send_event(Fsm,{login_pass,Pwd}) of\n    {ok, Resp} -> {ok, Resp};\n    Resp       -> Resp\n    end.\n\n%% use md5_login/3 for SMTP AUTH MD5\nmd5_login(Fsm,User,Pwd) ->\n    case gen_fsm:sync_send_event(Fsm,md5_login) of\n    {ok, Resp} ->\n        md5_login(Fsm,decode_challenge(Resp),User,Pwd);\n    Resp ->\n        Resp\n    end.\n%% md5_login/4 only used by md5_login/3\nmd5_login(Fsm,Challenge,User,Pwd) ->\n    Md5Code = md5_hmac(Challenge,Pwd),\n    gen_fsm:sync_send_event(Fsm, {md5_send,User,Md5Code}).\n\n%% use these for noop & rset as defined in ESMTP rfc\nnoop(Fsm) -> gen_fsm:sync_send_event(Fsm, noop).\nrset(Fsm) -> gen_fsm:sync_send_all_state_event(Fsm, rset).\n\n%% use this to close the SMTP connection\nclose(Fsm) -> gen_fsm:send_all_state_event(Fsm, close).\n\n%% use this to find out what ESMTP features are supported\nfeatures(Fsm) -> gen_fsm:sync_send_event(Fsm, features).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% callbacks\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ninit([Server,Port]) ->\n    io:format(\"connecting to ~p~n\",[Server]),\n    Args = [list,{packet,0}],\n    {ok,Socket} = gen_tcp:connect(Server,Port,Args),\n    io:format(\"socket open~n\"),\n    case get_response(Socket) of\n    {\"220\", _Resp} ->\n        {ok, smtp_start, #info{socket=Socket}};\n    _Error ->\n        {stop, conn_error}\n    end.\n\n%% state=smtp_start after connection but before we've sent\n%% helo or ehlo\nsmtp_start({helo, Name}, _Pid, Info)->\n    Msg = [\"helo \", Name, \"\\r\\n\"],\n    ok = gen_tcp:send(Info#info.socket, Msg),\n    case get_response(Info#info.socket) of\n    {\"250\", Resp} ->\n        {reply,{ok,Resp},smtp_conn,Info};\n    {_Code, Resp} ->\n        {reply,{helo_error,Resp},smtp_start, Info};\n    _Error ->\n        {stop, conn_error, conn_error, []}\n    end;\nsmtp_start({ehlo, Name}, _Pid, Info) ->\n    Msg = [\"ehlo \", Name, \"\\r\\n\"],\n    ok = gen_tcp:send(Info#info.socket, Msg),\n    case get_response(Info#info.socket) of\n    {\"250\", Resp} ->\n        Tokens = string:tokens(Resp, \"\\r\\n\"),\n        Strs = [string:sub_string(X,5) || X <- Tokens],\n        NewInfo = Info#info{features = tl(Strs)},\n        {reply, {ok, Resp}, smtp_conn, NewInfo};\n    {_Code, Resp} ->\n        {reply, {ehlo_error, Resp}, smtp_start, Info};\n    Error ->\n        {stop, conn_error, {conn_error, Error}, []}\n    end.\n\n%% state=smtp_conn - we can now send emails, login etc.\n%%\nsmtp_conn(noop, _Pid, Info) ->\n    ok = gen_tcp:send(Info#info.socket, \"noop\\r\\n\"),\n    case get_response(Info#info.socket) of\n    {\"250\", Resp} ->\n        {reply, {ok, Resp}, smtp_conn, Info};\n    {_Code, Resp} ->\n        {reply, {noop_error, Resp}, smtp_conn, Info};\n    Error ->\n        {stop, conn_error, {conn_error, Error}, []}\n    end;\nsmtp_conn({mfrom, Address}, _Pid, Info) ->\n    Msg = [\"mail from:\", Address, \"\\r\\n\"],\n    ok = gen_tcp:send(Info#info.socket, Msg),\n    case get_response(Info#info.socket) of\n    {\"250\", Resp} ->\n        {reply, {ok, Resp}, smtp_conn, Info};\n    {_Code, Resp} ->\n        {reply, {mfrom_error, Resp}, smtp_conn, Info};\n    Error ->\n        {stop, conn_error, {conn_error, Error}, []}\n    end;\nsmtp_conn({rcpt_to, Address}, _Pid, Info) ->\n    Msg = [\"rcpt to:\", Address, \"\\r\\n\"],\n    ok = gen_tcp:send(Info#info.socket, Msg),\n    case get_response(Info#info.socket) of\n    {\"250\", Resp} ->\n        {reply, {ok, Resp}, smtp_conn, Info};\n    {_Code, Resp} ->\n        {reply, {rcpt_error, Resp}, smtp_conn, Info};\n    Error ->\n        {stop, conn_error, {conn_error, Error}, []}\n    end;\nsmtp_conn(data, _Pid, Info) ->\n    ok = gen_tcp:send(Info#info.socket, \"Data\\r\\n\"),\n    case get_response(Info#info.socket) of\n    {\"354\", Resp} ->\n        {reply, {ok, Resp}, smtp_data, Info};\n    {_Code, Resp} ->\n        {reply, {data_error, Resp}, smtp_conn, Info};\n    Resp ->\n        {stop, conn_error, {conn_error, Resp}, []}\n    end;\nsmtp_conn({plain_login,User,Pwd}, _Pid, Info) ->\n    Msg =  [\"AUTH PLAIN \", plain_encode(User,Pwd), \"\\r\\n\"],\n    ok = gen_tcp:send(Info#info.socket, Msg),\n    case get_response(Info#info.socket) of\n    {\"235\", Resp} ->\n        {reply, {ok, Resp}, smtp_conn, Info};\n    {_Code, Resp} ->\n        {reply, {auth_error, Resp}, smtp_conn, Info};\n    Error ->\n        {stop, conn_error, {conn_error, Error}, []}\n    end;\nsmtp_conn({login_login,User}, _Pid, Info) ->\n    B64Usr =  base64:encode(User),\n    Msg = [\"AUTH LOGIN \", B64Usr, \"\\r\\n\"],\n    ok = gen_tcp:send(Info#info.socket, Msg),\n    case get_response(Info#info.socket) of\n    {\"334\", Resp} ->\n        {reply, {ok, Resp}, smtp_login, Info};\n    {_Code, Resp} ->\n        {reply, {auth_error, Resp}, smtp_conn, Info};\n    Error ->\n        {stop, conn_error, {conn_error, Error}, []}\n    end;\nsmtp_conn(features, _Pid, Info) ->\n    {reply, {ok,Info#info.features}, smtp_conn, Info};\nsmtp_conn(md5_login, _Pid, Info) ->\n    ok = gen_tcp:send(Info#info.socket,\"AUTH CRAM-MD5\\r\\n\"),\n    case get_response(Info#info.socket) of\n    {\"334\", Resp} ->\n        {reply, {ok, Resp}, smtp_md5, Info};\n    {_Code, Resp} ->\n        {reply, {auth_error, Resp}, smtp_conn, Info};\n    Error ->\n        {stop, conn_error, {conn_error, Error}, []}\n    end.\n\n%% state used in md5 authentication\nsmtp_md5({md5_send, User, Md5_hmac}, _Pid, Info) ->\n    Md5Usr = base64:encode(User++\" \"++Md5_hmac),\n    ok = gen_tcp:send(Info#info.socket, [Md5Usr,\"\\r\\n\"]),\n    case get_response(Info#info.socket) of\n    {\"235\", Resp} ->\n        {reply, {ok, Resp}, smtp_conn, Info};\n    {_Code, Resp} ->\n        {reply, {auth_error, Resp}, smtp_conn, Info};\n    Error ->\n        {stop, conn_error, {conn_error, Error}, []}\n    end.\n\n%% state used in sending data, it's after data has been sent,\n%% but before the message has been sent\nsmtp_data({msg,Message}, _Pid, Info) ->\n    Msg = [Message, \"\\r\\n.\\r\\n\"],\n    ok = gen_tcp:send(Info#info.socket, Msg),\n    case get_response(Info#info.socket) of\n    {\"250\", Resp} ->\n        {reply, {ok, Resp}, smtp_conn, Info};\n    {_Code, Resp} ->\n        {reply, {data_error, Resp}, smtp_conn, Info};\n    Error ->\n        {stop, conn_error, {conn_error, Error}, []}\n    end.\n\n%% state used in login, it's after the username has been sent,\n%% but before the password has been sent.\nsmtp_login({login_pass,Pwd}, _Pid, Info) ->\n    Msg = [base64:encode(Pwd), \"\\r\\n\"],\n    ok = gen_tcp:send(Info#info.socket, Msg),\n    case get_response(Info#info.socket) of\n    {\"235\", Resp} ->\n        {reply, {ok, Resp}, smtp_conn, Info};\n    {_Code, Resp} ->\n        {reply, {auth_error, Resp}, smtp_conn, Info};\n    Resp ->\n        {stop, conn_error, {conn_error, Resp}, []}\n    end.\n\n%% non-specific callbacks\n\nhandle_sync_event(rset, _Pid, _State, Info) ->\n    ok = gen_tcp:send(Info#info.socket, \"rset\\r\\n\"),\n    case get_response(Info#info.socket) of\n    {\"250\", Resp} ->\n        {reply, {ok, Resp}, smtp_conn, Info};\n    {_Code, Resp} ->\n        {reply, {rset_error, Resp}, smtp_conn, Info};\n    Error ->\n        {stop, conn_error, {conn_error, Error}, []}\n    end.\n\nhandle_event(close, _State, Info) ->\n    ok = gen_tcp:send(Info#info.socket, \"quit\\r\\n\"),\n    {stop, normal, Info}.\n\nterminate(normal, _StateName, _StateData)->\n    normal;\nterminate(Reason,_StateName,_StateData) ->\n    {terminated, Reason}.\n\nhandle_info(_Info, StateName, StateData) ->\n    {next_state, StateName, StateData}.\n\ncode_change(_OldVsn, StateName, StateData, _Extra) ->\n    {ok, StateName, StateData}.\n"
  },
  {
    "path": "contrib/log4erl/src/syslog_appender.erl",
    "content": "-module(syslog_appender).\r\n\r\n-include(\"../include/log4erl.hrl\").\r\n\r\n-behaviour(gen_event).\r\n%% gen_event callbacks\r\n-export([init/1, handle_event/2, handle_call/2, \r\n\t handle_info/2, terminate/2, code_change/3]).\r\n\r\n-define(DEFAULT_HOST, \"localhost\").\r\n-define(DEFAULT_PORT, 514).\r\n\r\ninit({conf, Conf}) ->\r\n    Level = proplists:get_value(level, Conf,nil),\r\n    Fac = proplists:get_value(facility, Conf,nil),\r\n    Host =  proplists:get_value(host, Conf,nil),\r\n    Port =  proplists:get_value(port, Conf,nil),\r\n    Format =  proplists:get_value(format, Conf,nil),\r\n\r\n    Tup = {Level, Fac, Host, Port, Format},\r\n\r\n    case Tup of\r\n\t{L, F, nil, nil, nil} ->\r\n\t    init({L, F});\r\n\t{L, F, H, nil, nil} ->\r\n\t    init({L, F, H});\r\n\t{L, F, H, nil, Fmt} when not (Fmt == nil) ->\r\n\t    init({L, F, H, Fmt});\r\n\t{L, F, H, P, Fmt} ->\r\n\t    init({L,F,H,P,Fmt})\r\n    end;\r\n%% {Level, Facility, Host, Port, Format}\r\n%% \r\ninit({Level, Fac}) ->\r\n    init({Level, Fac, ?DEFAULT_HOST, ?DEFAULT_PORT, ?DEFAULT_FORMAT});\r\ninit({Level, Fac, Host}) ->\r\n    init({Level, Fac, Host, ?DEFAULT_PORT, ?DEFAULT_FORMAT});\r\ninit({Level, Fac, Host, Format}) ->\r\n    init({Level, Fac, Host, ?DEFAULT_PORT, Format});\r\ninit({Level, Fac, Host, Port, Format}) ->\r\n    {ok, Socket} = gen_udp:open(0),\r\n    F = Fac,\r\n    {ok, Form} = log_formatter:parse(Format),\r\n    facility(Fac),\r\n    State = #syslog_appender{level=Level, facility=F, host=Host, port=Port, socket=Socket, format=Form},\r\n    {ok, State};\r\ninit(File) when is_list(File) ->\r\n    case file:consult(File) of\r\n\t{error, Reason} ->\r\n\t    error_logger:error_msg(\"syslog_appender: couldn't consult Conf file~n\"),\r\n\t    {error, file:format_error(Reason)};\r\n\t{ok, [Terms]} ->\r\n\t    init(Terms)\r\n    end.\r\n\r\nhandle_event({change_level, Level}, State) ->\r\n    State2 = State#syslog_appender{level = Level},\r\n    ?LOG2(\"Changed level to ~p~n\",[Level]),\r\n    {ok, State2};\r\nhandle_event({log,LLog}, State) ->\r\n    ?LOG2(\"handl_event:log = ~p~n\",[LLog]),\r\n    do_log(LLog, State),\r\n    {ok, State}.\r\n\r\nhandle_call({change_format, Format}, State) ->\r\n    ?LOG2(\"Old State in console_appender is ~p~n\",[State]),\r\n    {ok, Tokens} = log_formatter:parse(Format),\r\n    ?LOG2(\"Adding format of ~p~n\",[Tokens]),\r\n    S2 = State#syslog_appender{format=Tokens},\r\n    {ok, ok, S2};\r\nhandle_call({change_level, Level}, State) ->\r\n    State2 = State#syslog_appender{level = Level},\r\n    ?LOG2(\"Changed level to ~p~n\",[Level]),\r\n    {ok, ok, State2};\r\nhandle_call(_Request, State) ->\r\n    {ok, ok, State}.\r\n\r\nhandle_info(_Info, State) ->\r\n    {ok, State}.\r\n\r\nterminate(_Reason, _S) ->\r\n    ok.\r\n\r\ncode_change(_OldVsn, State, _Extra) ->\r\n    {ok, State}.\r\n\r\ndo_log(#log{level = L} = Log, #syslog_appender{level=Level} = State) ->\r\n    %% Syslog levels are a little bit differnt\r\n    case should_log(L, Level) of\r\n\ttrue ->\r\n\t    Pid = State#syslog_appender.socket,\r\n\t    Host = State#syslog_appender.host,\r\n\t    Port = State#syslog_appender.port,\r\n\t    Format = State#syslog_appender.format,\r\n\t    Fac = State#syslog_appender.facility,\r\n\t    Msg = log_formatter:format(Log, Format),\r\n\r\n\t    Who = node(),\r\n\t    do_send(Pid, Host, Port, {Who, Fac, L, Msg});\r\n\tfalse ->\r\n\t    ok\r\n    end.\r\n\r\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\r\n%% The following code has been shamefully copied from\r\n%% syslog.erl written by Torbjorn Tornkvist and downloaded\r\n%% from the user contribution section in erlang.org\r\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\r\n\r\n%% Convenient routine for specifying levels.\r\n%% modified by ahmed al-issaei\r\n\r\nlevel(emergency) -> 0; % system is unusable\r\nlevel(emerg)     -> 0; % shortcut for emergency\r\nlevel(alert)     -> 1; % action must be taken immediately \r\nlevel(critical)  -> 2; % critical conditions\r\nlevel(crit)      -> 2; % shortcut for critical\r\nlevel(fatal)     -> 2; % another shortcut to critical\r\nlevel(error)     -> 3; % error conditions\r\nlevel(warning)   -> 4; % warning conditions\r\nlevel(warn)      -> 4; % shortcut for warning\r\nlevel(notice)    -> 5; % normal but significant condition \r\nlevel(info)      -> 6; % informational\r\nlevel(debug)     -> 7; % debug-level messages\r\nlevel(all)       -> 7; % to allow 'all' in conf parameters for level\r\nlevel(_)         -> 1. % anything else is alert\r\n\r\n%% Convenient routine for specifying facility codes\r\n%% modified by ahmed al-issaei\r\nfacility(kern)     -> (0 bsl 3); % kernel messages \r\nfacility(user)     -> (1 bsl 3); % random user-level messages \r\nfacility(mail)     -> (2 bsl 3); % mail system \r\nfacility(daemon)   -> (3 bsl 3); % system daemons \r\nfacility(auth)     -> (4 bsl 3); % security/authorization messages\r\nfacility(syslog)   -> (5 bsl 3); % messages generated internally by syslogd \r\nfacility(lpr)      -> (6 bsl 3); % line printer subsystem \r\nfacility(news)     -> (7 bsl 3); % network news subsystem \r\nfacility(uucp)     -> (8 bsl 3); % UUCP subsystem \r\nfacility(cron)     -> (9 bsl 3); % clock daemon \r\nfacility(authpriv) -> (10 bsl 3); % security/authorization messages (private) \r\nfacility(ftp)      -> (11 bsl 3); % ftp daemon\r\nfacility(ntp)      -> (12 bsl 3); % ntp daemon\r\nfacility(audit)    -> (13 bsl 3); % log audit\r\nfacility(alert)    -> (14 bsl 3); % log alert\r\nfacility(clock)    -> (15 bsl 3); % clock daemon\r\nfacility(local0)   -> (16 bsl 3); % local use 0\r\nfacility(local1)   -> (17 bsl 3); % local use 1\r\nfacility(local2)   -> (18 bsl 3); % local use 2\r\nfacility(local3)   -> (19 bsl 3); % local use 3\r\nfacility(local4)   -> (20 bsl 3); % local use 4\r\nfacility(local5)   -> (21 bsl 3); % local use 5\r\nfacility(local6)   -> (22 bsl 3); % local use 6\r\nfacility(local7)   -> (23 bsl 3); % local use 7\r\nfacility(_)        -> facility(user). % anything else is user\r\n\r\n%% priorities/facilities are encoded into a single 32-bit \r\n%% quantity, where the bottom 3 bits are the priority (0-7) \r\n%% and the top 28 bits are the facility (0-big number).    \r\ndo_send(S,Host,Port,{Who,Facility,Level,Msg})\r\n  when is_atom(Who), is_atom(Facility), is_atom(Level) ->\r\n    Type = (facility(Facility) bor level(Level)),\r\n    Packet = \"<\" ++ i2l(Type) ++ \"> \" ++ a2l(Who) ++ \": \" ++ Msg ++ \"\\n\",\r\n    gen_udp:send(S,Host,Port,lists:flatten(Packet)).\r\n\r\ni2l(Int) when is_integer(Int) ->\r\n    integer_to_list(Int);\r\ni2l(Int) ->\r\n    Int.\r\n\r\na2l(Atom) when is_atom(Atom) ->\r\n    atom_to_list(Atom);\r\na2l(Atom) ->\r\n    Atom.\r\n\r\n%% Log4erl uses decreasing values for priority\r\nshould_log(Cur, Level) when is_atom(Cur), is_atom(Level) ->\r\n    level(Cur) =< level(Level).\r\n"
  },
  {
    "path": "contrib/log4erl/src/xml_appender.erl",
    "content": "-module(xml_appender).\r\n\r\n-behaviour(gen_event).\r\n\r\n-include(\"../include/log4erl.hrl\").\r\n-include_lib(\"kernel/include/file.hrl\").\r\n\r\n-import(log4erl_utils, [to_list/1, to_atom/1, to_int/1]).\r\n\r\n%% gen_event callbacks\r\n-export([init/1, handle_event/2, handle_call/2, \r\n\t handle_info/2, terminate/2, code_change/3]).\r\n\r\n-define(DEFAULT_XMLSPEC, [{level, \"%L\", att},\r\n\t\t\t  {date, \"%j\", att},\r\n\t\t\t  {time, \"%T\", att},\r\n\t\t\t  {message, \"%l\", elem}]).\r\n\r\n%% xmlspec is a list of xml_spec records\r\n-record(xml_appender, {dir, file_name, fd, counter, log_type, rotation, suffix=\"xml\", level, xmlspec}).\r\n\r\n%% xmlspec is of the form {Name, Format, Type}\r\n%% where Name is the name of the element or attribute\r\n%% Format is the format of the value\r\n%% Type is either 'att' or 'elem'\r\n-record(xml_spec, {name, format, type}).\r\n\r\n%%======================================\r\n%% gen_event callback functions\r\n%%======================================\r\ninit({conf, Conf}) when is_list(Conf) ->\r\n    CL = lists:foldl(fun(X, List) ->\r\n\t\t\t     case X of\r\n\t\t\t\t {X1, D} ->\r\n\t\t\t\t     [proplists:get_value(X1,Conf,D)|List];\r\n\t\t\t\t _ ->\r\n\t\t\t\t     [proplists:get_value(X,Conf)|List]\r\n\t\t\t     end\r\n\t\t     end,\r\n\t\t     [],\r\n\t\t     [dir, file, type, max, rotation, suffix, level, {xmlspec, ?DEFAULT_XMLSPEC}]),\r\n    \r\n    init(list_to_tuple(lists:reverse(CL)));\r\ninit({Dir, Fname, {Type, Max}, Rot, Suf, Level, Spec} = _Conf) ->\r\n    ?LOG2(\"xml_appender:init() - 1 ~p~n\",[_Conf]),\r\n    File = Dir ++ \"/\" ++ Fname ++ \".\" ++ Suf,\r\n    Ltype = #log_type{type = Type, max = Max},\r\n    % Check Rot >= 0\r\n    Rot1 = case Rot < 0 of\r\n\t       true ->\r\n\t\t   0;\r\n\t       false ->\r\n\t\t   Rot\r\n\t   end,\r\n    {ok, Fd} = file:open(File, ?FILE_OPTIONS),\r\n\r\n    %% Translate {Name, Format, Type} to #xml_spec records\r\n    XmlSpec = lists:map(fun({N, F, T}) ->\r\n\t\t\t\t{ok, Tokens} = log_formatter:parse(F),\r\n\t\t\t\t#xml_spec{name=N, format=Tokens, type=T}\r\n\t\t\tend, Spec),\r\n\r\n    %% Start xml \r\n    file:write(Fd, \"<?xml>\\n<log4erl>\"),\r\n\r\n    State = #xml_appender{dir = Dir, file_name = Fname, fd = Fd, counter=0,\r\n\t\t\t   log_type = Ltype, rotation = Rot1, suffix=Suf,\r\n\t\t\t   level=Level, xmlspec=XmlSpec},\r\n    ?LOG2(\"xml_appender:init() with conf ~p~n\",[State]),\r\n    {ok, State};\r\n% These 2 are for result of reading conf file\r\ninit({Dir, Fname, Type, Max, Rot, Suf, Level, Spec}) ->\r\n    init({to_list(Dir), to_list(Fname), {to_atom(Type), to_int(Max)}, to_int(Rot), to_list(Suf), to_atom(Level), Spec}).\r\n\r\nhandle_event({change_level, Level}, State) ->\r\n    State2 = State#xml_appender{level = Level},\r\n    ?LOG2(\"Changed level to ~p~n\",[Level]),\r\n    {ok, State2};\r\nhandle_event({log,LLog}, State) ->\r\n    ?LOG2(\"handl_event:log = ~p~n\",[LLog]),\r\n    do_log(LLog, State),\r\n    Res = check_rotation(State),\r\n    {ok, Res}.\r\n\r\nhandle_call({change_format, _Format}, State) ->\r\n    ?LOG(\"Cannot change format in xml_appender~n\"),\r\n    {ok, ok, State};\r\nhandle_call({change_level, Level}, State) ->\r\n    State2 = State#xml_appender{level = Level},\r\n    ?LOG2(\"Changed level to ~p~n\",[Level]),\r\n    {ok, ok, State2};\r\nhandle_call({change_filename, Fname}, #xml_appender{dir=Dir, suffix=Suf} = State) ->\r\n    File = Dir ++ \"/\" ++ Fname ++ \".\" ++ Suf,\r\n    {ok, Fd} = file:open(File, ?FILE_OPTIONS),\r\n    State2 = State#xml_appender{file_name = Fname, fd = Fd},\r\n    ?LOG2(\"Changed filename to ~p~n\",[File]),\r\n    {ok, ok, State2};\r\nhandle_call(_Request, State) ->\r\n    Reply = ok,\r\n    {ok, Reply, State}.\r\n\r\nhandle_info(_Info, State) ->\r\n    ?LOG2(\"~w received unknown message: ~p~n\", [?MODULE, _Info]),\r\n    {ok, State}.\r\n\r\nterminate(_Reason, #xml_appender{fd=Fd}) ->\r\n    file:write(Fd, \"</log4erl>\\n\"),\r\n    ok.\r\n\r\ncode_change(_OldVsn, State, _Extra) ->\r\n    {ok, State}.\r\n\r\n%%======================================\r\n%% internal callback functions\r\n%%======================================\r\ndo_log(#log{level = L} = Log,#xml_appender{fd = Fd, level=Level, xmlspec=XmlSpec} = _State) when is_atom(L) ->\r\n    ToLog = log4erl_utils:to_log(L, Level),\r\n    case ToLog of\r\n\ttrue ->\r\n\t    M = format_xml(Log, XmlSpec),\r\n\t    file:write(Fd, M);\r\n\tfalse ->\r\n\t    ok\r\n    end;\r\ndo_log(_Other, _State) ->\r\n    ?LOG2(\"unknown level ~p~n\",[_Other]),\r\n    ok.\r\n\r\nrotate(#xml_appender{fd = Fd, dir=Dir,  file_name=Fn, counter=Cntr, rotation=Rot, suffix=Suf} = S) ->\r\n    file:write(Fd, \"</log4erl>\\n\"),\r\n    file:close(Fd),\r\n    ?LOG(\"Starting rotation~n\"),\r\n    C = if\r\n\t    Rot == 0 ->\r\n\t\t0;\r\n\t    Cntr >= Rot ->\r\n\t\t1;\r\n\t    true ->\r\n\t\tCntr+1\r\n\tend,\r\n    Src = Dir ++ \"/\" ++ Fn ++ \".\" ++ Suf,\r\n    Fname = case C of\r\n\t\t0 ->\r\n\t\t    Dir ++ \"/\" ++ Fn ++ \".\" ++ Suf;\r\n\t\t_ ->\r\n\t\t    Dir ++ \"/\" ++ Fn ++ \"_\" ++ integer_to_list(C) ++ \".\" ++ Suf\r\n\t    end,\r\n    ?LOG2(\"Renaming file from ~p to ~p~n\",[Src, Fname]),\r\n    file:rename(Src, Fname),\r\n    {ok ,Fd2} = file:open(Src, ?FILE_OPTIONS_ROTATE),\r\n    file:write(Fd2, \"<?xml>\\n<log4erl>\"),\r\n    State2 = S#xml_appender{dir = Dir, file_name = Fn, fd = Fd2, rotation = Rot, suffix=Suf},\r\n    {ok, State2}.\r\n\r\n% Check if the file needs to be rotated\r\n% ignore in case of if log type is set to time instead of size\t    \r\ncheck_rotation(State) ->\r\n    #xml_appender{dir=Dir, file_name=Fname, log_type = #log_type{type=T, max=Max}, suffix=Suf} = State,\r\n    case T of\r\n\tsize ->\r\n\t    File = Dir ++ \"/\" ++ Fname ++  \".\" ++ Suf,\r\n\t    {ok, Finfo} = file:read_file_info(File),\r\n\t    Size = Finfo#file_info.size,\r\n\t    if\r\n\t\tSize > Max ->\r\n\t\t    {ok, State2} = rotate(State),\r\n\t\t    State2;\r\n\t\ttrue ->\r\n\t\t    State\r\n\t    end;\r\n\t%% time-based rotation is not implemented yet\r\n\t_ ->\r\n\t    State\r\n    end.\r\n\r\nformat_xml(Log, Spec) ->\r\n    Att = lists:filter(fun(#xml_spec{type=T}) ->\r\n\t\t\t       T == att\r\n\t\t       end,\r\n\t\t       Spec),\r\n    Elems = lists:filter(fun(#xml_spec{type=T}) ->\r\n\t\t\t\t T == elem\r\n\t\t\t end,\r\n\t\t\t Spec),\r\n\r\n    A = attributes(Log, Att),\r\n    B = elements(Log, Elems),\r\n\r\n    \"<log\" ++ A ++ \">\\n\" ++ B ++ \"</log>\\n\".\r\n\r\nattributes(_, []) ->\r\n    [];\r\nattributes(Log, Atts) ->\r\n    attributes(Log, Atts, []).\r\n\r\nattributes(_, [], Acc) ->\r\n    lists:reverse(Acc);\r\nattributes(Log, [#xml_spec{name=N,format=F}|Rest], Acc) ->\r\n    S = \" \" ++ escape_attr(log4erl_utils:to_list(N)) ++ \"=\\\"\" ++ escape_attr(log_formatter:format(Log, F)) ++\"\\\"\",\r\n    attributes(Log, Rest, [S|Acc]).\r\n\r\nelements(_, []) ->\r\n    [];\r\nelements(Log, Elems) ->\r\n    elements(Log, Elems, []).\r\n\r\nelements(_, [], Acc) ->\r\n    lists:reverse(Acc);\r\nelements(Log, [#xml_spec{name=N, format=F}|Rest], Acc) ->\r\n    Name = escape_attr(log4erl_utils:to_list(N)),\r\n    L = lists:flatten(log_formatter:format(Log, F)),\r\n    S = \"\\t<\" ++ Name ++ \">\" ++ escape(L) ++\"</\" ++ Name ++ \">\\n\",\r\n    elements(Log, Rest, [S|Acc]).\r\n\r\n%% The following code is copied/pasted from mochiweb's 'mochiweb_html' & 'mochinum' modules\r\n%% See http://code.google.com/p/mochiweb\r\n\r\n%% @spec escape(string() | atom() | binary()) -> binary()\r\n%% @doc Escape a string such that it's safe for HTML (amp; lt; gt;).\r\nescape(B) when is_binary(B) ->\r\n    escape(binary_to_list(B), []);\r\nescape(A) when is_atom(A) ->\r\n    escape(atom_to_list(A), []);\r\nescape(S) when is_list(S) ->\r\n    escape(S, []).\r\n\r\n%% @spec escape_attr(string() | binary() | atom() | integer() | float()) -> binary()\r\n%% @doc Escape a string such that it's safe for HTML attrs\r\n%%      (amp; lt; gt; quot;).\r\nescape_attr(B) when is_binary(B) ->\r\n    escape_attr(binary_to_list(B), []);\r\nescape_attr(A) when is_atom(A) ->\r\n    escape_attr(atom_to_list(A), []);\r\nescape_attr(S) when is_list(S) ->\r\n    escape_attr(S, []);\r\nescape_attr(I) when is_integer(I) ->\r\n    escape_attr(integer_to_list(I), []);\r\nescape_attr(F) when is_float(F) ->\r\n    escape_attr(mochinum:digits(F), []).\r\n\r\nescape([], Acc) ->\r\n    lists:reverse(Acc);\r\nescape(\"<\" ++ Rest, Acc) ->\r\n    escape(Rest, lists:reverse(\"&lt;\", Acc));\r\nescape(\">\" ++ Rest, Acc) ->\r\n    escape(Rest, lists:reverse(\"&gt;\", Acc));\r\nescape(\"&\" ++ Rest, Acc) ->\r\n    escape(Rest, lists:reverse(\"&amp;\", Acc));\r\nescape([C | Rest], Acc) ->\r\n    escape(Rest, [C | Acc]).\r\n\r\n-define(QUOTE, $\\\").\r\n\r\nescape_attr([], Acc) ->\r\n    lists:reverse(Acc);\r\nescape_attr(\"<\" ++ Rest, Acc) ->\r\n    escape_attr(Rest, lists:reverse(\"&lt;\", Acc));\r\nescape_attr(\">\" ++ Rest, Acc) ->\r\n    escape_attr(Rest, lists:reverse(\"&gt;\", Acc));\r\nescape_attr(\"&\" ++ Rest, Acc) ->\r\n    escape_attr(Rest, lists:reverse(\"&amp;\", Acc));\r\nescape_attr([?QUOTE | Rest], Acc) ->\r\n    escape_attr(Rest, lists:reverse(\"&quot;\", Acc));\r\nescape_attr([C | Rest], Acc) ->\r\n    escape_attr(Rest, [C | Acc]).\r\n\r\n"
  },
  {
    "path": "contrib/log4erl-setup.txt",
    "content": "16.11.2016: download from https://github.com/ahmednawras/log4erl\n1) download new version\n2) remove append from FILE_OPTIONS in include/log4erl.hrl\n2) add delayed_write to FILE_OPTIONS in include/log4erl.hrl to increase performance at the cost of losing log entries if the log4erl process dies\n7) that's it, go back to <scalaris>/ and run ./configure && make\n\nYou probably have to patch log4erl to make the erlang/dev dialyzer happy.\n"
  },
  {
    "path": "contrib/packages/bindings/PKGBUILD",
    "content": "# Maintainer: Nico Kruber <kruber at zib dot de>\n\npkgname=scalaris-bindings\npkgver=0.9.0+git\npkgrel=1\npkgdesc=\"Bindings and clients for Scalaris.\"\narch=('i686' 'x86_64')\nurl=\"http://scalaris.zib.de\"\nlicense=('Apache')\ndepends=('java-runtime>=1.6.0' 'python2>=2.6' 'python>=3.0' 'ruby>=1.8')\nmakedepends=('java-environment>=1.6.0' 'apache-ant' 'glib2' 'erlang>=R14B04')\noptdepends=('java-runtime: Java API and client'\n            'python2: Python2 API and client'\n            'python: Python3 API and client'\n            'ruby: Ruby API and client')\nbackup=('etc/scalaris/scalaris-java.conf' 'etc/scalaris/scalaris.properties')\nsource=(scalaris-$pkgver.tar.gz)\nmd5sums=('f188e12f4cf893022a87a1b29971aa81')\n\nbuild() {\n  # set JAVA_HOME for ant:\n  source /etc/profile.d/jre.sh\n  export JAVA_HOME=/usr/lib/jvm/default\n  RUBY_SITELIB=$(ruby -rrbconfig -e 'puts RbConfig::CONFIG[\"sitelibdir\"]')\n  cd \"$srcdir/scalaris-$pkgver\"\n  ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var \\\n    --with-ruby-sitelibdir=$RUBY_SITELIB\n  ANT_OPTS=\"-Dfile.encoding=utf8 -Dant.build.javac.source=1.6 -Dant.build.javac.target=1.6\" \\\n    make java\n  make java-doc\n  make python\n  make python3\n}\n\npackage() {\n  cd \"$srcdir/scalaris-$pkgver\"\n  make install-java DESTDIR=\"$pkgdir\"\n  make install-java-doc DESTDIR=\"$pkgdir\"\n  make install-python DESTDIR=\"$pkgdir\"\n  make install-python3 DESTDIR=\"$pkgdir\"\n  make install-ruby DESTDIR=\"$pkgdir\"\n  \n  # put the LICENSE file to the licenses\n  install -D -m644 LICENSE \"$pkgdir/usr/share/licenses/$pkgname/LICENSE\"\n}\n"
  },
  {
    "path": "contrib/packages/bindings/checkout.sh",
    "content": "#!/bin/bash\n\nSCALARIS_VERSION=\"0.9.0+git\"\ndate=`date +\"%Y%m%d.%H%M\"`\nname=\"scalaris\" # folder base name (without version)\npkg_name=\"scalaris-bindings\" # package name\nurl=\"https://github.com/scalaris-team/scalaris.git\"\ndeletefolder=0 # set to 1 to delete the folder the repository is checked out to\n\n#####\nif [[ \"$SCALARIS_VERSION\" == *git* ]]; then\n  MODE=\"snapshot\"\nelse\n  MODE=\"tag\"\nfi\n\nif [ \"$MODE\" = \"snapshot\" ] ; then\n  folder=\"${name}\"\n  if [ ! -d \"${folder}\" ]; then\n    echo \"checkout ${url} -> ${folder} ...\"\n    git clone --branch master --single-branch --depth=1 \"${url}\" \"${folder}\"\n    result=$?\n  else\n    echo \"update ${url} -> ${folder} ...\"\n    cd \"${folder}\"\n    git pull\n    result=$?\n    cd - >/dev/null\n  fi\n\n  if [ ${result} -eq 0 ]; then\n    echo -n \"get git revision ...\"\n    revision=`cd \"${folder}\" && git log --pretty=format:'%h' -n 1`\n    result=$?\n    echo \" ${revision}\"\n    pkg_version=\"${SCALARIS_VERSION}${date}.${revision}\"\n  fi\nelse\n  pkg_version=\"${SCALARIS_VERSION}\"\n  folder=\"${name}-${pkg_version}\"\n  tarfile=\"${folder}.tar.gz\"\n  echo \"downloading archive ${url%.git}/archive/${SCALARIS_VERSION}.tar.gz ...\"\n  wget \"${url%.git}/archive/${SCALARIS_VERSION}.tar.gz\" -O \"${tarfile}\" && \\\n  tar -xzf \"${tarfile}\"\n  result=$?\n  if [ ! -d \"${folder}\" ]; then\n    echo \"wrong archive contents, expecting folder ${folder}\"\n    exit 1\n  fi\nfi\n\nif ! diff -q checkout.sh \"${folder}/contrib/packages/bindings/checkout.sh\" > /dev/null ; then\n  echo \"checkout-script changed - re-run ./checkout.sh\"\n  cp \"${folder}/contrib/packages/bindings/checkout.sh\" ./\n  exit 1\nfi\n\nif [ \"$MODE\" = \"snapshot\" ] ; then\n  if [ ${result} -eq 0 ]; then\n    tarfile=\"${name}-${pkg_version}.tar.gz\"\n    newfoldername=\"${name}-${pkg_version}\"\n    echo \"making ${tarfile} ...\"\n    mv \"${folder}\" \"${newfoldername}\" && tar -czf \"${tarfile}\" \"${newfoldername}\" --exclude-vcs && mv \"${newfoldername}\" \"${folder}\"\n    result=$?\n  fi\nfi\n\nif [ ${result} -eq 0 ]; then\n  echo \"extracting .spec file ...\"\n  sourcefolder=\"${folder}/contrib/packages/bindings\"\n  sed -e \"s/%define pkg_version .*/%define pkg_version ${pkg_version}/g\" \\\n      < \"${sourcefolder}/${pkg_name}.spec\"           > \"./${pkg_name}.spec\" && \\\n  cp  \"${sourcefolder}/${pkg_name}.changes\"            \"./${pkg_name}.changes\"\n  result=$?\nfi\n\nif [ ${result} -eq 0 ]; then\n  echo \"extracting Debian package files ...\"\n  sourcefolder=\"${folder}/contrib/packages/bindings\"\n  sed -e \"s/Version: .*-.*/Version: ${pkg_version}-1/g\" \\\n      -e \"s/scalaris.*\\\\.orig\\\\.tar\\\\.gz/${pkg_name}-${pkg_version}\\\\.orig\\\\.tar\\\\.gz/g\" \\\n      -e \"s/scalaris.*\\\\.diff\\\\.tar\\\\.gz/${pkg_name}-${pkg_version}\\\\.diff\\\\.tar\\\\.gz/g\" \\\n      < \"${sourcefolder}/${pkg_name}.dsc\"             > \"./${pkg_name}.dsc\" && \\\n  ( ( test \"$MODE\" != \"snapshot\" && \\\n      sed -e \"0,/(.*-.*)/s//(${pkg_version}-1)/\" \\\n      < \"${sourcefolder}/debian.changelog\"            > ./debian.changelog ) || \\\n    sed -e \"0,/(.*-.*)/s//(${pkg_version}-1)/\" \\\n        -e \"0,/ -- Nico Kruber <kruber@zib.de>  .*/s// -- Nico Kruber <kruber@zib.de>  `LANG=C date -R`/\" \\\n      < \"${sourcefolder}/debian.changelog\"            > ./debian.changelog ) && \\\n  cp  \"${sourcefolder}/debian.compat\"                   ./debian.compat && \\\n  cp  \"${sourcefolder}/debian.control\"                  ./debian.control && \\\n  cp  \"${sourcefolder}/debian.rules\"                    ./debian.rules && \\\n  cp  \"${sourcefolder}/debian.source.lintian-overrides\" ./debian.source.lintian-overrides && \\\n  cp  \"${folder}/LICENSE\"                               ./debian.copyright\n  result=$?\nfi\n\nif [ ${result} -eq 0 ]; then\n  echo \"extracting ArchLinux package files ...\"\n  sourcefolder=\"${folder}/contrib/packages/bindings\"\n  tarmd5sum=`md5sum \"${tarfile}\" | cut -d' ' -f 1` && \\\n  sed -e \"s/pkgver=.*/pkgver=${pkg_version}/g\" \\\n      -e \"s/md5sums=('.*')/md5sums=('${tarmd5sum}')/g\" \\\n      < \"${sourcefolder}/PKGBUILD\"                   > ./PKGBUILD\n  result=$?\nfi\n\nif [ ${result} -eq 0 -a ${deletefolder} -eq 1 ]; then\n  echo \"removing ${folder} ...\"\n  rm -rf \"${folder}\"\nfi\n"
  },
  {
    "path": "contrib/packages/bindings/debian.changelog",
    "content": "scalaris-bindings (0.9.0+git-1) unstable; urgency=low\n\n  * git snapshot from master\n\n -- Nico Kruber <kruber@zib.de>  Mon, 29 Feb 2016 18:00:00 +0200\n\nscalaris-bindings (0.9.0-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n  * Packaging:\n    - add a libscalaris-dev[el] package with the C++ API (rpm, deb)\n  * API:\n    - CPP-API: provide a new C++ client library\n    - DataNucleus: provide a new storage back-end for Scalaris\n    - add a Scalaris adapter for YCSB\n    - new functions in all APIs for generic routing table operations\n\n -- Nico Kruber <kruber@zib.de>  Mon, 29 Feb 2016 18:00:00 +0200\n\nscalaris-bindings (0.8.2-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n\n -- Nico Kruber <kruber@zib.de>  Fri, 24 Jul 2015 12:00:00 +0200\n\nscalaris-bindings (0.8.1-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n\n -- Nico Kruber <kruber@zib.de>  Sun, 12 Jul 2015 12:00:00 +0200\n\nscalaris-bindings (0.8.0-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n    and the EIT ICT Labs project MCData\n  * Packaging:\n    - also install docs on Arch Linux\n    - fix lintian errors and warnings for Debian-based packages\n    - fix permissions on the log directory (owned by the 'scalaris' user)\n      and the config files (not owned by the 'scalaris' user anymore!)\n    - adapt packages for newer distributions\n  * API:\n    - Java-API: integrate new OtpErlang library (1.5.12 from Erlang 17.4)\n    - Python-API: do not include a shebang with a pre-defined path to python\n                  anymore\n    - Python-API: fix unclosed sockets in various tests\n    - Python-API: increase python{2,3} test speed\n    - Ruby-API: try to fallback to the normal json module if the gem is missing\n    - Ruby-API: considerably increase the ruby test speed\n    - remove the Publish/Subscribe API\n  * Infrastructure:\n    - move sources to github: https://github.com/scalaris-team/scalaris\n    - new project homepage: http://scalaris.zib.de\n  * Bugs:\n    - JSON-API: fix test_and_set not working correctly inside req_list\n    - fix numerous more bugs\n\n -- Nico Kruber <kruber@zib.de>  Fri, 10 Jul 2015 12:00:00 +0200\n\nscalaris-bindings (0.7.2-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n    and the EIT ICT Labs project MCData\n  * Packaging:\n    - fix ArchLinux packages with newest Java versions\n\n -- Nico Kruber <kruber@zib.de>  Mon, 23 Oct 2014 18:15:00 +0200\n\nscalaris-bindings (0.7.1-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n    and the EIT ICT Labs project MCData\n  * Packaging:\n    - add support for new distribution versions\n    - include daemon for monitoring Scalaris through JMX\n  * API:\n    - Java-API: integrate new OtpErlang library (1.5.10 from Erlang 17.3)\n  * Bugs:\n    - fix numerous bugs\n\n -- Nico Kruber <kruber@zib.de>  Mon, 30 Sep 2014 17:00:00 +0200\n\nscalaris-bindings (0.7.0-1) stable; urgency=low\n\n  * API:\n    - Java-API: integrate new OtpErlang library (1.5.9 from Erlang 17.0)\n  * Bugs:\n    - fix numerous more (less severe) bugs\n\n -- Nico Kruber <kruber@zib.de>  Mon, 28 Apr 2014 12:00:00 +0200\n\nscalaris-bindings (0.6.1-1) stable; urgency=low\n\n  * Packaging:\n    - add ArchLinux ruby API packages\n  * API:\n    - Java-API: add Maven build support\n    - Java-API: add CircularByteArrayOutputStream#clear()\n    - Java-API: fix ConnectionPool#getConnection(timeout) throwing\n                IllegalMonitorStateException if a single ConnectionPool is used\n                by multiple threads and no more connections are available\n    - Java-API: add a \"-monitor\" command line parameter\n    - api_monitor: return the latency and stddev values of the micro-benchmark\n                   executed by monitor_perf for node and service performance\n  * Bugs:\n    - fix some more (less severe) bugs\n\n -- Nico Kruber <kruber@zib.de>  Fri, 11 Oct 2013 15:00:00 +0200\n\nscalaris-bindings (0.6.0-1) stable; urgency=low\n\n  * Packaging:\n    - add ArchLinux packages\n    - add support for new distribution versions\n  * API:\n    - no more timeouts in client APIs\n    - Java-API: re-worked the request and result list handling\n      -> move result processing to the operation classes\n    - Java-API: better support for custom operations\n    - Java-API: support the new partial reads:\n                ReadRandomFromListOp and ReadSublistOp\n    - Java-API: compile with \"vars\" debug info\n    - Java-API: integrate new OtpErlang library (1.5.8 from R16B) with fixed\n                support for compressed binaries\n    - Java-API: add back-ports from the Wiki on Scalaris demonstrator:\n      * list-change operations: ScalarisChangeListOp and\n                                ScalarisListAppendRemoveOp\n      * MultiMap classes are now in de.zib.tools\n      * CircularByteArrayOutputStream\n    - Java-API: fix hostname issues with Erlang and Java\n    - Java-API: slightly changed the delete API\n    - JSON-API: add API for auto-scale requests\n    - Python-API: add API for auto-scale requests\n    - Python-API: use default socket timeout\n    - Ruby-API: use default socket timeout\n    - all APIs: support lists of composite types\n  * Infrastructure:\n    - add a daemon to monitor Scalaris via JMX\n    - support for distributions with python3 available as \"python\" and\n      python2 as \"python2\"\n    - support for Ruby 1.9\n  * Bugs:\n    - use /bin/bash instead of /bin/sh which may not result in a bash session\n    - fix numerous other bugs\n\n -- Nico Kruber <kruber@zib.de>  Fri, 16 Aug 2013 23:20:13 +0200\n\nscalaris-bindings (0.5.0-1) stable; urgency=low\n  * API:\n    - allow Scalaris monitoring via JMX through the Java API\n    - added an executor-service to the Java-API (de.zib.scalaris.executor.*)\n    - added a node discovery daemon to the Java-API\n    - allow compressed communication between the Java-API and Erlang for\n      increased performance, especially if the two are on separate nodes\n    - added VM management support to the JSON- and Python-API\n    - added transaction log filtering to the Java-API, i.e. only sent the\n      needed parts of the tlog back to Erlang and re-combine the result\n    - fixed api_tx:req_list_commit_each/1 not running requests in parallel\n      -> do not assure any order of requests, even if on same key!\n  * Bugs:\n    - fixed numerous bugs\n\n -- Nico Kruber <kruber@zib.de>  Thu, 11 Oct 2012 12:30:00 +0200\n\nscalaris-bindings (0.4.1-1) stable; urgency=low\n\n  * Packaging:\n    - install rubygem dependencies in Debian postinstall scripts for Ruby API\n\n -- Nico Kruber <kruber@zib.de>  Thu, 22 Mar 2012 10:51:00 +0100\n\nscalaris-bindings (0.4.0-1) stable; urgency=low\n\n  * API:\n    - new functions for incremental data change:\n      test_and_set: check for a provided old value before setting a new one\n      add_on_nr: increment a numeric value\n      add_del_on_list: append or delete entries from a list value\n    - added VM API to manage Scalaris nodes inside an Erlang virtual machine\n    - added monitoring API to retrieve some live metrics\n    - added a connection pool convenience class (Java, Python)\n  * Documentation:\n    - updated documentation to extended APIs\n  * Bugs:\n    - fixed numerous bugs\n\n -- Nico Kruber <kruber@zib.de>  Wed, 25 Jan 2012 00:48:00 +0100\n\nscalaris-bindings (0.3.0-1) stable; urgency=low\n\n  * API\n    - new API with interoperable bindings to Java, Python, Ruby, and JSON\n    - support for several data types, including strings, integers, JSON\n      objects, binary objects.\n    - new transaction interface with support for bundled requests for better\n      latency.\n  * Tests\n    - added numerous unittests\n    - added language-binding interoperability tests\n  * Documentation\n    - extended, but - as always - by far not enough...\n  * Bugs\n    - fixed countless bugs\n\n -- Nico Kruber <kruber@zib.de>  Fri, 15 Jul 2011 17:01:00 +0200\n"
  },
  {
    "path": "contrib/packages/bindings/debian.compat",
    "content": "7\n"
  },
  {
    "path": "contrib/packages/bindings/debian.control",
    "content": "Source: scalaris-bindings\nSection: devel\nPriority: optional\nMaintainer: Nico Kruber <kruber@zib.de>\nBuild-Depends: debhelper (>= 4.1.16), erlang-dev (>= 14.b.4), erlang-tools (>= 14.b.4), erlang-edoc (>= 14.b.4), erlang-crypto (>= 14.b.4), ant, openjdk-7-jdk | java6-sdk, pkg-config, ruby, python-dev (>= 2.6.0), python-epydoc, graphviz, python3-dev, automake, libboost-dev (>= 1.47), libboost-regex-dev (>= 1.47), libboost-system-dev (>= 1.47), libboost-test-dev (>= 1.47), libboost-program-options-dev (>= 1.47)\nHomepage: http://scalaris.zib.de\nStandards-Version: 3.9.6\n\nPackage: scalaris-java\nSection: java\nArchitecture: all\nDepends: ${misc:Depends}, debianutils, hostname, java6-runtime\nDescription: Java-API and Java-Client for Scalaris\n Java Bindings and command line client for Scalaris.\n\nPackage: libscalaris-dev\nSection: devel\nArchitecture: any\nDepends: ${misc:Depends}, libboost-dev (>= 1.47), libboost-regex-dev (>= 1.47), libboost-system-dev (>= 1.47), libboost-test-dev (>= 1.47), libboost-program-options-dev (>= 1.47)\nDescription: C++-API for Scalaris\n C++ Bindings for Scalaris.\n\nPackage: scalaris-ruby\nSection: ruby\nArchitecture: all\nDepends: ${misc:Depends}, ruby, ruby-json (>= 1.4.1)\nDescription: Ruby-API and Ruby-client for Scalaris\n Ruby bindings and Ruby command line client for Scalaris.\n\nPackage: python-scalaris\nSection: python\nArchitecture: all\nDepends: ${misc:Depends}, python (>= 2.6.0)\nDescription: Python-API and Python-client for Scalaris\n Python bindings and Python command line client for Scalaris.\n\nPackage: python3-scalaris\nSection: python\nArchitecture: all\nDepends: ${misc:Depends}, python3\nDescription: Python3-API and Python3-client for Scalaris\n Python3 bindings and Python3 command line client for Scalaris.\n"
  },
  {
    "path": "contrib/packages/bindings/debian.rules",
    "content": "#!/usr/bin/make -f\n# Sample debian/rules that uses debhelper.\n# GNU copyright 1997 to 1999 by Joey Hess.\n\n# Uncomment this to turn on verbose mode.\n#export DH_VERBOSE=1\n\nRUBY_SITELIB:=$(shell ruby -rrbconfig -e 'puts Config::CONFIG[\"sitelibdir\"] ')\nPYTHON_SITELIB:=$(shell python -c 'from distutils.sysconfig import get_python_lib; print (get_python_lib())'  2>/dev/null || echo PYTHON-NOT-FOUND)\nPYTHON3_SITELIB:=$(shell python3 -c 'from distutils.sysconfig import get_python_lib; print (get_python_lib())'  2>/dev/null || echo PYTHON3-NOT-FOUND)\n\nWITH_PYTHON:=$(shell python -c \"import sys; a,b=sys.version_info[:2]; print a >= 2 and b >= 6 and '1' or '0'\")\nWITH_PYTHON3:=$(shell python3 -c 'print('1');'  2>/dev/null || echo 0)\n\nbuild-arch: build\nbuild-indep: build\nbuild: build-stamp\nbuild-stamp:\n\tdh_testdir\n\n\t# Add here commands to compile the package.\n\t./configure --prefix=/usr \\\n\t  --sysconfdir=/etc \\\n\t  --localstatedir=/var \\\n\t  --docdir=/usr/share/doc/scalaris \\\n\t  --with-ruby-sitelibdir=$(RUBY_SITELIB)\n\tANT_OPTS=\"-Dfile.encoding=utf8 -Dant.build.javac.source=1.6 -Dant.build.javac.target=1.6\" \\\n\t  make java java-doc cpp\n\tif [ \"$(WITH_PYTHON)\" = 1 ]; then make python ; fi\n\tif [ \"$(WITH_PYTHON3)\" = 1 ]; then make python3 ; fi\n\n\ttouch build-stamp\n\nclean:\n\tdh_testdir\n\tdh_testroot\n\trm -f build-stamp\n\n\t# Add here commands to clean up after the build process.\n\trm -rf build_dir\n\n\tdh_clean\n\nBUILD_DIR_JAVA:=$(CURDIR)/debian/scalaris-java\nBUILD_DIR_CPP:=$(CURDIR)/debian/libscalaris-dev\nBUILD_DIR_RUBY:=$(CURDIR)/debian/scalaris-ruby\nBUILD_DIR_PYTHON:=$(CURDIR)/debian/python-scalaris\nBUILD_DIR_PYTHON3:=$(CURDIR)/debian/python3-scalaris\ninstall: build\n\tdh_testdir\n\tdh_testroot\n\tdh_prep\n\tdh_installdirs\n\n\t# Add here commands to install the package into debian/gentoo.\n\tmake install-java DESTDIR=$(BUILD_DIR_JAVA)\n\tmake install-java-doc DESTDIR=$(BUILD_DIR_JAVA)\n\tmake install-cpp DESTDIR=$(BUILD_DIR_CPP)\n\tmake install-ruby DESTDIR=$(BUILD_DIR_RUBY)\n\tif [ \"$(WITH_PYTHON)\" = 1 ]; then make install-python DESTDIR=$(BUILD_DIR_PYTHON) ; make install-python-doc-html DESTDIR=$(BUILD_DIR_PYTHON) ; fi\n\tif [ \"$(WITH_PYTHON3)\" = 1 ]; then make install-python3 DESTDIR=$(BUILD_DIR_PYTHON3) ; fi\n\t\n\t# handle config files:\n\tcp $(BUILD_DIR_JAVA)/etc/scalaris/scalaris.properties \\\n\t   $(BUILD_DIR_JAVA)/etc/scalaris/scalaris.properties.example\n\n# Build architecture-independent files here.\nbinary-indep: build install\n\tdh_testdir\n\tdh_testroot\n#\tdh_installdebconf\n\tdh_installdocs\n\tdh_installexamples\n\tdh_installmenu\n#\tdh_installlogrotate\n#\tdh_installemacsen\n#\tdh_installpam\n#\tdh_installmime\n#\tdh_installinit\n\tdh_installcron\n\tdh_installman\n\tdh_installinfo\n#\tdh_undocumented\n\tdh_installchangelogs\n\tdh_link\n\tdh_strip\n\tdh_compress\n\tdh_fixperms\n#\tdh_makeshlibs\n\tdh_installdeb\n#\tdh_perl\n\tdh_shlibdeps\n\tdh_gencontrol\n\tdh_md5sums\n\tdh_builddeb\n\n# Build architecture-dependent files here.\nbinary-arch: build install\n\t# We have nothing to do by default.\n\nbinary: binary-indep binary-arch\n.PHONY: build clean binary-indep binary-arch binary install\n"
  },
  {
    "path": "contrib/packages/bindings/debian.source.lintian-overrides",
    "content": "scalaris-bindings source: source-is-missing docroot/ext-base.js\nscalaris-bindings source: source-is-missing docroot/ext-all.js\nscalaris-bindings source: source-is-missing docroot/tablesort/jquery.tablesorter.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.flot.threshold.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.flot.symbol.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.flot.stack.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.flot.selection.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.flot.resize.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.flot.pie.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.flot.navigate.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.flot.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.flot.image.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.flot.fillbetween.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.flot.errorbars.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.flot.crosshair.min.js\nscalaris-bindings source: source-is-missing docroot/flot/jquery.colorhelpers.min.js\nscalaris-bindings source: source-is-missing docroot/flot/excanvas.min.js\nscalaris-bindings source: source-is-missing contrib/wikipedia/contrib/libsqlite4java-linux-i386.so\nscalaris-bindings source: source-is-missing contrib/wikipedia/contrib/libsqlite4java-linux-amd64.so\n"
  },
  {
    "path": "contrib/packages/bindings/scalaris-bindings.changes",
    "content": "-------------------------------------------------------------------\nMon Feb 29 16:00:00 UTC 2016 - kruber@zib.de\n\n- Scalaris 0.9.0 (codename \"Vriesea scalaris\")\n  (partly supported by the EU project IES Cities http://iescities.eu/)\n  * Packaging:\n    - add a libscalaris-dev[el] package with the C++ API (rpm, deb)\n  * API:\n    - CPP-API: provide a new C++ client library\n    - DataNucleus: provide a new storage back-end for Scalaris\n    - add a Scalaris adapter for YCSB\n    - new functions in all APIs for generic routing table operations\n\n-------------------------------------------------------------------\nFri Jul 24 10:00:00 UTC 2015 - kruber@zib.de\n\n- Scalaris 0.8.2\n  (partly supported by the EU project IES Cities http://iescities.eu/)\n\n-------------------------------------------------------------------\nSun Jul 12 10:00:00 UTC 2015 - kruber@zib.de\n\n- Scalaris 0.8.1\n  (partly supported by the EU project IES Cities http://iescities.eu/)\n\n-------------------------------------------------------------------\nFri Jul 10 10:00:00 UTC 2015 - kruber@zib.de\n\n- Scalaris 0.8.0 (codename \"Picoides scalaris\")\n  (partly supported by the EU project IES Cities http://iescities.eu/\n  and the EIT ICT Labs project MCData)\n  * Packaging:\n    - also install docs on Arch Linux\n    - fix lintian errors and warnings for Debian-based packages\n    - fix permissions on the log directory (owned by the 'scalaris' user)\n      and the config files (not owned by the 'scalaris' user anymore!)\n    - adapt packages for newer distributions\n  * API:\n    - Java-API: integrate new OtpErlang library (1.5.12 from Erlang 17.4)\n    - Python-API: do not include a shebang with a pre-defined path to python\n                  anymore\n    - Python-API: fix unclosed sockets in various tests\n    - Python-API: increase python{2,3} test speed\n    - Ruby-API: try to fallback to the normal json module if the gem is missing\n    - Ruby-API: considerably increase the ruby test speed\n    - remove the Publish/Subscribe API\n  * Infrastructure:\n    - move sources to github: https://github.com/scalaris-team/scalaris\n    - new project homepage: http://scalaris.zib.de\n  * Bugs:\n    - JSON-API: fix test_and_set not working correctly inside req_list\n    - fix numerous more bugs\n\n-------------------------------------------------------------------\nMon Oct 23 16:15:00 UTC 2014 - kruber@zib.de\n\n- Scalaris 0.7.2\n  (partly supported by the EU project IES Cities http://iescities.eu/\n  and the EIT ICT Labs project MCData)\n  * Packaging:\n    - fix ArchLinux packages with newest Java versions\n\n-------------------------------------------------------------------\nMon Sep 29 15:00:00 UTC 2014 - kruber@zib.de\n\n- Scalaris 0.7.1\n  (partly supported by the EU project IES Cities http://iescities.eu/\n  and the EIT ICT Labs project MCData)\n  * Packaging:\n    - add support for new distribution versions\n    - include daemon for monitoring Scalaris through JMX\n  * API:\n    - Java-API: integrate new OtpErlang library (1.5.10 from Erlang 17.3)\n  * Bugs:\n    - fix numerous bugs\n\n-------------------------------------------------------------------\nMon Apr 28 10:00:00 UTC 2014 - kruber@zib.de\n\n- Scalaris 0.7.0 (codename \"Stauroderus scalaris\")\n  * API:\n    - Java-API: integrate new OtpErlang library (1.5.9 from Erlang 17.0)\n  * Bugs:\n    - fix numerous more (less severe) bugs\n\n-------------------------------------------------------------------\nFri Oct 11 13:00:00 UTC 2013 - kruber@zib.de\n\n- Scalaris 0.6.1\n  * Packaging:\n    - add ArchLinux ruby API packages\n  * API:\n    - Java-API: add Maven build support\n    - Java-API: add CircularByteArrayOutputStream#clear()\n    - Java-API: fix ConnectionPool#getConnection(timeout) throwing\n                IllegalMonitorStateException if a single ConnectionPool is used\n                by multiple threads and no more connections are available\n    - Java-API: add a \"-monitor\" command line parameter\n    - api_monitor: return the latency and stddev values of the micro-benchmark\n                   executed by monitor_perf for node and service performance\n  * Bugs:\n    - fix some more (less severe) bugs\n\n-------------------------------------------------------------------\nFri Aug 16 21:20:13 UTC 2013 - kruber@zib.de\n\n- Scalaris 0.6.0 (codename \"Conus scalaris\")\n  * Packaging:\n    - add ArchLinux packages\n    - add support for new distribution versions\n  * API:\n    - no more timeouts in client APIs\n    - Java-API: re-worked the request and result list handling\n      -> move result processing to the operation classes\n    - Java-API: better support for custom operations\n    - Java-API: support the new partial reads:\n                ReadRandomFromListOp and ReadSublistOp\n    - Java-API: compile with \"vars\" debug info\n    - Java-API: integrate new OtpErlang library (1.5.8 from R16B) with fixed\n                support for compressed binaries\n    - Java-API: add back-ports from the Wiki on Scalaris demonstrator:\n      * list-change operations: ScalarisChangeListOp and\n                                ScalarisListAppendRemoveOp\n      * MultiMap classes are now in de.zib.tools\n      * CircularByteArrayOutputStream\n    - Java-API: fix hostname issues with Erlang and Java\n    - Java-API: slightly changed the delete API\n    - JSON-API: add API for auto-scale requests\n    - Python-API: add API for auto-scale requests\n    - Python-API: use default socket timeout\n    - Ruby-API: use default socket timeout\n    - all APIs: support lists of composite types\n  * Infrastructure:\n    - add a daemon to monitor Scalaris via JMX\n    - support for distributions with python3 available as \"python\" and\n      python2 as \"python2\"\n    - support for Ruby 1.9\n  * Bugs:\n    - use /bin/bash instead of /bin/sh which may not result in a bash session\n    - fix numerous other bugs\n\n-------------------------------------------------------------------\nThu Oct 11 10:30:00 UTC 2012 - kruber@zib.de\n\n- Scalaris 0.5.0 (codename \"Saperda scalaris\")\n  * API:\n    - allow Scalaris monitoring via JMX through the Java API\n    - added an executor-service to the Java-API (de.zib.scalaris.executor.*)\n    - added a node discovery daemon to the Java-API\n    - allow compressed communication between the Java-API and Erlang for\n      increased performance, especially if the two are on separate nodes\n    - added VM management support to the JSON- and Python-API\n    - added transaction log filtering to the Java-API, i.e. only sent the\n      needed parts of the tlog back to Erlang and re-combine the result\n    - fixed api_tx:req_list_commit_each/1 not running requests in parallel\n      -> do not assure any order of requests, even if on same key!\n  * Bugs:\n    - fixed numerous bugs\n\n-------------------------------------------------------------------\nThu Mar 22 09:51:00 UTC 2012 - kruber@zib.de\n\n- Scalaris 0.4.1\n  * Packaging:\n    - install rubygem dependencies in Debian postinstall scripts for Ruby API\n\n-------------------------------------------------------------------\nWed Jan 24 23:48:00 UTC 2012 - kruber@zib.de\n\n- Scalaris 0.4.0 (codename \"Pomacea scalaris\")\n  * API:\n    - new functions for incremental data change:\n      test_and_set: check for a provided old value before setting a new one\n      add_on_nr: increment a numeric value\n      add_del_on_list: append or delete entries from a list value\n    - added VM API to manage Scalaris nodes inside an Erlang virtual machine\n    - added monitoring API to retrieve some live metrics\n    - added a connection pool convenience class (Java, Python)\n  * Documentation:\n    - updated documentation to extended APIs\n  * Bugs:\n    - fixed numerous bugs\n\n-------------------------------------------------------------------\nFri Jul 15 15:01:00 UTC 2011 - kruber@zib.de\n\n- Scalaris 0.3.0 (codename \"Rhinechis Scalaris\")\n  * API\n    - new API with interoperable bindings to Java, Python, Ruby, and JSON\n    - support for several data types, including strings, integers, JSON\n      objects, binary objects.\n    - new transaction interface with support for bundled requests for better\n      latency.\n  * Tests\n    - added numerous unittests\n    - added language-binding interoperability tests\n  * Documentation\n    - extended, but - as always - by far not enough...\n  * Bugs\n    - fixed countless bugs\n"
  },
  {
    "path": "contrib/packages/bindings/scalaris-bindings.dsc",
    "content": "Format: 1.0\nSource: scalaris-bindings\nVersion: 0.9.0+git-1\nDEBTRANSFORM-RELEASE: 1\nMaintainer: Nico Kruber <kruber@zib.de>\nArchitecture: all\nStandards-Version: 3.9.6\nBuild-Depends: debhelper (>= 4.1.16), erlang-dev (>= 14.b.4), erlang-tools (>= 14.b.4), erlang-edoc (>= 14.b.4), erlang-crypto (>= 14.b.4), ant, openjdk-7-jdk | java6-sdk, pkg-config, ruby, python-dev (>= 2.6.0), python-epydoc, graphviz, python3-dev, automake, libboost-dev (>= 1.47), libboost-regex-dev (>= 1.47), libboost-system-dev (>= 1.47), libboost-test-dev (>= 1.47), libboost-program-options-dev (>= 1.47)\nFiles:\n 6ea6383ef9251ff5ac641df84ddbcba0 2008366 scalaris.orig.tar.gz\n 2fecf324a32123b08cefc0f047bca5ee 0 scalaris.diff.tar.gz\n"
  },
  {
    "path": "contrib/packages/bindings/scalaris-bindings.spec",
    "content": "# norootforbuild\n\n%define pkg_version 0.9.0+git\nName:           scalaris-bindings\nSummary:        Scalable Distributed key-value store\nVersion:        %{pkg_version}\nRelease:        1\nLicense:        Apache-2.0\nGroup:          Productivity/Databases/Servers\nURL:            http://scalaris.zib.de\nSource0:        scalaris-%{version}.tar.gz\nSource100:      checkout.sh\nBuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-build\nBuildRequires:  ant\nBuildRequires:  java-devel >= 1.6.0\nBuildRequires:  ruby >= 1.8\n\n##########################################################################################\n## Fedora, RHEL or CentOS\n##########################################################################################\n%if 0%{?fedora_version} || 0%{?rhel_version} || 0%{?centos_version}\nBuildRequires:  erlang-erts >= R14B04, erlang-kernel, erlang-stdlib, erlang-compiler, erlang-crypto, erlang-edoc, erlang-inets, erlang-ssl, erlang-tools, erlang-xmerl, erlang-os_mon\nBuildRequires:  pkgconfig\n%if 0%{?fedora_version} >= 19 || 0%{?rhel_version} >= 700 || 0%{?centos_version} >= 700\n%define with_cpp 1\n# our cpp-api currently only provides a static library for which\n# find-debuginfo.sh cannot find the debug infos, see\n# https://fedoraproject.org/wiki/Packaging:Debuginfo\n%global debug_package %{nil}\nBuildRequires:  ruby(release) >= 1.8\n%else\n%define with_cpp 0\nBuildRequires:  ruby(abi) >= 1.8\n%endif\n%if 0%{?fedora_version} >= 12 || 0%{?centos_version} >= 600\n%define with_python 1\n%define with_python_doc_html 1\n%define with_python_doc_pdf 0\n%endif\n%if 0%{?rhel_version} >= 600\n%define with_python 1\n%define with_python_doc_html 0\n%define with_python_doc_pdf 0\n%endif\n%if 0%{?fedora_version} >= 13\n%define with_python3 1\nBuildRequires:  python3-setuptools python-tools\n%endif\n%endif\n\n###########################################################################################\n# SuSE, openSUSE\n###########################################################################################\n%if 0%{?suse_version}\n# note: erlang is still needed for configure\nBuildRequires:  erlang >= R14B04\nBuildRequires:  pkg-config\n%if 0%{?suse_version} >= 1110 || 0%{?sles_version} >= 11 \n%define with_python 1\n%if 0%{?suse_version} == 1315\n%define with_python_doc_html 0\n%else\n%define with_python_doc_html 1\n%endif\n%define with_python_doc_pdf 1\n%if 0%{?suse_version} > 1220\n%define with_python_doc_pdf 0\n%endif\n%if 0%{?suse_version} >= 1130 \n%else\n# py_requires is no longer needed since 11.3\n%py_requires\n%endif\n%endif\n%if 0%{?suse_version} >= 1120\n%define with_python3 1\n# these macros are not integrated yet:\n%global python3_ver      %(python3 -c \"import sys; v=sys.version_info[:2]; print('%%d.%%d'%%v)\" 2>/dev/null || echo PYTHON-NOT-FOUND)\n%global python3_prefix   %(python3 -c \"import sys; print(sys.prefix)\" 2>/dev/null || echo PYTHON-NOT-FOUND)\n%global python3_libdir   %{python3_prefix}/%{_lib}/python%{python3_ver}\n%global python3_sitedir  %{python3_libdir}/site-packages\n%endif\n%if 0%{?suse_version} > 1210\nBuildRequires:  python3-2to3\n%endif\n%if %{?suse_version} >= 1130 && %{?suse_version} <= 1310\nBuildRequires:  ruby(abi) >= 1.8\n%endif\n%if 0%{?suse_version} >= 1310 || 0%{?sles_version} >= 12\n%define with_cpp 1\n%else\n%define with_cpp 0\n%endif\n%endif\n\n%{!?rb_sitelib: %global rb_sitelib %(ruby -rrbconfig -e 'puts Config::CONFIG[\"sitelibdir\"] ')}\n\n%if 0%{?with_cpp}\nBuildRequires:  automake\nBuildRequires:  boost-devel >= 1.47\nBuildRequires:  gcc-c++\n%endif\n\n%if 0%{?with_python}\n%if 0%{?with_python_doc_html}\nBuildRequires:  epydoc graphviz graphviz-gd\n%endif\n%if 0%{?with_python_doc_pdf}\nBuildRequires:  epydoc texlive-latex\n%endif\nBuildRequires:  python-devel >= 2.6\n%{!?python_sitelib: %global python_sitelib %(python -c 'from distutils.sysconfig import get_python_lib; print (get_python_lib())')}\n%endif\n\n%if 0%{?with_python3}\nBuildRequires:  python3-devel\n%{!?python3_sitelib: %global python3_sitelib %(python3 -c 'from distutils.sysconfig import get_python_lib; print (get_python_lib())')}\n%endif\n\n%description\nScalaris is a scalable, transactional, distributed key-value store. It\ncan be used for building scalable services. Scalaris uses a structured\noverlay with a non-blocking Paxos commit protocol for transaction\nprocessing with strong consistency over replicas. Scalaris is\nimplemented in Erlang.\n\n%package -n scalaris-java\nSummary:    Java-API and Java-Client for Scalaris\nGroup:      Productivity/Databases/Clients\nRequires:   jre >= 1.6.0\nRequires:   net-tools\n%if 0%{?fedora_version} || 0%{?rhel_version} || 0%{?centos_version} || 0%{?suse_version} >= 1310\nRequires:       which\n%else\nRequires:       util-linux\n%endif\n%if 0%{?sles_version} == 10 || 0%{?sles_version} == 11\n# once noarch, always noarch on SLE <= 11\n%else\nBuildArch:  noarch\n%endif\n\n%description -n scalaris-java\nJava Bindings and command line client for Scalaris\n\n%if 0%{?with_cpp}\n%package -n libscalaris-devel\nSummary:    C++-API for Scalaris\nGroup:      Productivity/Databases/Clients\nProvides:   libscalaris-static = %{version}-%{release}\nRequires:   boost-devel >= 1.47\n\n%description -n libscalaris-devel\nC++ Bindings for Scalaris\n%endif\n\n%package -n ruby-scalaris\nSummary:    Ruby-API and Ruby-client for Scalaris\nGroup:      Productivity/Databases/Clients\n%if 0%{?fedora_version} >= 19 || 0%{?rhel_version} >= 700 || 0%{?centos_version} >= 700\nRequires:   ruby(release) >= 1.8\n%else\nRequires:   ruby(abi) >= 1.8\n%endif\nRequires:   rubygems\nRequires:   rubygem-json >= 1.4.1\n%if 0%{?rhel_version} || 0%{?centos_version}\n# (Recommends tag not supported by RHEL and CentOS 5-7)\n%else\n# Drag in the pure Ruby implementation too, so that jruby has something to\n# fall back to: https://bugzilla.redhat.com/show_bug.cgi?id=1219502\nRecommends: rubygem-json_pure >= 1.4.1\n%endif\n\n%description -n ruby-scalaris\nRuby bindings and Ruby command line client for Scalaris\n\n%if 0%{?with_python}\n%package -n python-scalaris\nSummary:    Python-API and Python-client for Scalaris\nGroup:      Productivity/Databases/Clients\nRequires:   python >= 2.6\n%if 0%{?sles_version} == 10 || 0%{?sles_version} == 11\n# once noarch, always noarch on SLE <= 11\n%else\nBuildArch:  noarch\n%endif\n\n%description -n python-scalaris\nPython bindings and Python command line client for Scalaris\n%endif\n\n%if 0%{?with_python3}\n%package -n python3-scalaris\nSummary:    Python3-API and Python3-client for Scalaris\nGroup:      Productivity/Databases/Clients\nRequires:   python3\n%if 0%{?sles_version} == 10 || 0%{?sles_version} == 11\n# once noarch, always noarch on SLE <= 11\n%else\nBuildArch:  noarch\n%endif\n\n%description -n python3-scalaris\nPython3 bindings and Python3 command line client for Scalaris\n%endif\n\n%prep\n%setup -q -n scalaris-%{version}\n\n%build\nexport ANT_OPTS=\"-Dfile.encoding=utf8 -Dant.build.javac.source=1.6 -Dant.build.javac.target=1.6\"\n\nexport CFLAGS=\"%{optflags}\"\nexport CXXFLAGS=$CFLAGS\n\n%if 0%{?fedora_version} >= 18\nexport PATH=\"%{_bindir}:$PATH\"\n%endif\n./configure --prefix=%{_prefix} \\\n    --exec-prefix=%{_exec_prefix} \\\n    --bindir=%{_bindir} \\\n    --sbindir=%{_sbindir} \\\n    --sysconfdir=%{_sysconfdir} \\\n    --datadir=%{_datadir} \\\n    --includedir=%{_includedir} \\\n    --libdir=%{_libdir} \\\n    --libexecdir=%{_libexecdir} \\\n    --localstatedir=%{_localstatedir} \\\n    --sharedstatedir=%{_sharedstatedir} \\\n    --mandir=%{_mandir} \\\n    --infodir=%{_infodir} \\\n    --docdir=%{_docdir}/scalaris \\\n%if 0%{?with_cpp} == 0\n    --disable-cpp \\\n%endif\n    --with-ruby-sitelibdir=%{rb_sitelib}\nmake java\nmake java-doc\n%if 0%{?with_cpp}\nmake cpp\n%endif\n%if 0%{?with_python}\nmake python\n%endif\n%if 0%{?with_python3}\nmake python3\n%endif\n\n%install\n# see http://en.opensuse.org/openSUSE:Packaging_Java#bytecode_version_error\nexport NO_BRP_CHECK_BYTECODE_VERSION=true\nrm -rf $RPM_BUILD_ROOT\nmake install-java DESTDIR=$RPM_BUILD_ROOT\nmake install-java-doc DESTDIR=$RPM_BUILD_ROOT\nmake install-ruby DESTDIR=$RPM_BUILD_ROOT\n%if 0%{?with_python}\nmake install-python DESTDIR=$RPM_BUILD_ROOT\n%if 0%{?with_python_doc_html}\nmake install-python-doc-html DESTDIR=$RPM_BUILD_ROOT\n%endif\n%if 0%{?with_python_doc_pdf}\nmake install-python-doc-pdf DESTDIR=$RPM_BUILD_ROOT\n%endif\n%endif\n%if 0%{?with_python3}\nmake install-python3 DESTDIR=$RPM_BUILD_ROOT\n%endif\n%if 0%{?with_cpp}\nmake install-cpp DESTDIR=$RPM_BUILD_ROOT\n%endif\n\n%clean\nrm -rf $RPM_BUILD_ROOT\n\n%files -n scalaris-java\n%defattr(-,root,root,-)\n%{_javadir}/scalaris\n%dir %{_sysconfdir}/scalaris\n%config(noreplace) %{_sysconfdir}/scalaris/scalaris-java.conf\n%config %{_sysconfdir}/scalaris/scalaris-java.conf.sample\n%config(noreplace) %{_sysconfdir}/scalaris/scalaris.properties\n%{_bindir}/scalaris\n%dir %{_docdir}/scalaris/\n%doc %{_docdir}/scalaris/java-api\n%{_sysconfdir}/init.d/scalaris-monitor\n%{_sysconfdir}/init.d/scalaris-first-monitor\n\n%if 0%{?with_cpp}\n%files -n libscalaris-devel\n%defattr(-,root,root,-)\n%{_includedir}/scalaris\n%{_libdir}/libscalaris.a\n%endif\n\n%files -n ruby-scalaris\n%defattr(-,root,root,-)\n%{_bindir}/scalaris-ruby\n%{rb_sitelib}/scalaris.rb\n%{rb_sitelib}/scalaris_client.rb\n\n%if 0%{?with_python}\n%files -n python-scalaris\n%defattr(-,root,root,-)\n%{_bindir}/scalaris-python\n%{python_sitelib}/*\n%if 0%{?with_python_doc_html} || 0%{?with_python_doc_pdf}\n%doc %{_docdir}/scalaris/python-api\n%endif\n%endif\n\n%if 0%{?with_python3}\n%files -n python3-scalaris\n%defattr(-,root,root,-)\n%{_bindir}/scalaris-python3\n%if 0%{?suse_version}\n%dir %{python3_sitelib}\n%dir %{python3_sitelib}/..\n%endif\n%{python3_sitelib}/*\n%endif\n\n%changelog\n"
  },
  {
    "path": "contrib/packages/examples-wiki/checkout.sh",
    "content": "#!/bin/bash\n\nSCALARIS_VERSION=\"0.9.0+git\"\ndate=`date +\"%Y%m%d.%H%M\"`\nname=\"scalaris\" # folder base name (without version)\npkg_name=\"scalaris-examples-wiki\" # package name\nurl=\"https://github.com/scalaris-team/scalaris.git\"\ndeletefolder=0 # set to 1 to delete the folder the repository is checked out to\n\n#####\nif [[ \"$SCALARIS_VERSION\" == *git* ]]; then\n  MODE=\"snapshot\"\nelse\n  MODE=\"tag\"\nfi\n\nif [ \"$MODE\" = \"snapshot\" ] ; then\n  folder=\"${name}\"\n  if [ ! -d \"${folder}\" ]; then\n    echo \"checkout ${url} -> ${folder} ...\"\n    git clone --branch master --single-branch --depth=1 \"${url}\" \"${folder}\"\n    result=$?\n  else\n    echo \"update ${url} -> ${folder} ...\"\n    cd \"${folder}\"\n    git pull\n    result=$?\n    cd - >/dev/null\n  fi\n\n  if [ ${result} -eq 0 ]; then\n    echo -n \"get git revision ...\"\n    revision=`cd \"${folder}\" && git log --pretty=format:'%h' -n 1`\n    result=$?\n    echo \" ${revision}\"\n    pkg_version=\"${SCALARIS_VERSION}${date}.${revision}\"\n  fi\nelse\n  pkg_version=\"${SCALARIS_VERSION}\"\n  folder=\"${name}-${pkg_version}\"\n  echo \"downloading archive ${url%.git}/archive/${SCALARIS_VERSION}.tar.gz ...\"\n  wget \"${url%.git}/archive/${SCALARIS_VERSION}.tar.gz\" -O - | tar -xz \n  result=$?\n  if [ ! -d \"${folder}\" ]; then\n    echo \"wrong archive contents, expecting folder ${folder}\"\n    exit 1\n  fi\nfi\n\nif ! diff -q checkout.sh \"${folder}/contrib/packages/examples-wiki/checkout.sh\" > /dev/null ; then\n  echo \"checkout-script changed - re-run ./checkout.sh\"\n  cp \"${folder}/contrib/packages/examples-wiki/checkout.sh\" ./ ; exit 1\nfi\n\nif [ ${result} -eq 0 ]; then\n  tarfile=\"${pkg_name}-${pkg_version}.tar.gz\"\n  newfoldername=\"${pkg_name}-${pkg_version}\"\n  echo \"making ${tarfile} ...\"\n  mv \"${folder}/contrib/wikipedia\" \"${newfoldername}\" && tar -czf \"${tarfile}\" \"${newfoldername}\" --exclude-vcs && mv \"${newfoldername}\" \"${folder}/contrib/wikipedia\"\n  result=$?\nfi\n\nif [ ${result} -eq 0 ]; then\n  echo \"extracting .spec file ...\"\n  sourcefolder=\"${folder}/contrib/packages/examples-wiki\"\n  sed -e \"s/%define pkg_version .*/%define pkg_version ${pkg_version}/g\" \\\n      < \"${sourcefolder}/${pkg_name}.spec\"           > \"./${pkg_name}.spec\" && \\\n  cp  \"${sourcefolder}/${pkg_name}.changes\"            \"./${pkg_name}.changes\" && \\\n  cp  \"${sourcefolder}/${pkg_name}-rpmlintrc\"          \"./${pkg_name}-rpmlintrc\"\n  result=$?\nfi\n\nif [ ${result} -eq 0 ]; then\n  echo \"extracting Debian package files ...\"\n  sourcefolder=\"${folder}/contrib/packages/examples-wiki\"\n  sed -e \"s/Version: .*-.*/Version: ${pkg_version}-1/g\" \\\n      -e \"s/scalaris.*\\\\.orig\\\\.tar\\\\.gz/${pkg_name}-${pkg_version}\\\\.orig\\\\.tar\\\\.gz/g\" \\\n      -e \"s/scalaris.*\\\\.diff\\\\.tar\\\\.gz/${pkg_name}-${pkg_version}\\\\.diff\\\\.tar\\\\.gz/g\" \\\n      < \"${sourcefolder}/${pkg_name}.dsc\"             > \"./${pkg_name}.dsc\" && \\\n  ( ( test \"$MODE\" != \"snapshot\" && \\\n      sed -e \"0,/(.*-.*)/s//(${pkg_version}-1)/\" \\\n      < \"${sourcefolder}/debian.changelog\"            > ./debian.changelog ) || \\\n    sed -e \"0,/(.*-.*)/s//(${pkg_version}-1)/\" \\\n        -e \"0,/ -- Nico Kruber <kruber@zib.de>  .*/s// -- Nico Kruber <kruber@zib.de>  `LANG=C date -R`/\" \\\n      < \"${sourcefolder}/debian.changelog\"            > ./debian.changelog ) && \\\n  cp  \"${sourcefolder}/debian.compat\"                   ./debian.compat && \\\n  cp  \"${sourcefolder}/debian.control\"                  ./debian.control && \\\n  cp  \"${sourcefolder}/debian.rules\"                    ./debian.rules && \\\n  cp  \"${sourcefolder}/debian.scalaris-examples-wiki-tomcat5.conffiles\" \\\n                                                        ./debian.scalaris-examples-wiki-tomcat5.conffiles && \\\n  cp  \"${sourcefolder}/debian.scalaris-examples-wiki-tomcat6.conffiles\" \\\n                                                        ./debian.scalaris-examples-wiki-tomcat6.conffiles && \\\n  cp  \"${sourcefolder}/debian.source.lintian-overrides\" ./debian.source.lintian-overrides && \\\n  cp  \"${folder}/LICENSE\"                               ./debian.copyright\n  result=$?\nfi\n\nif [ ${result} -eq 0 -a ${deletefolder} -eq 1 ]; then\n  echo \"removing ${folder} ...\"\n  rm -rf \"${folder}\"\nfi\n"
  },
  {
    "path": "contrib/packages/examples-wiki/debian.changelog",
    "content": "scalaris-examples-wiki (0.9.0+git-1) unstable; urgency=low\n\n  * git snapshot from master\n\n -- Nico Kruber <kruber@zib.de>  Mon, 29 Feb 2016 18:00:00 +0200\n\nscalaris-examples-wiki (0.9.0-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n\n -- Nico Kruber <kruber@zib.de>  Mon, 29 Feb 2016 18:00:00 +0200\n\nscalaris-examples-wiki (0.8.2-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n\n -- Nico Kruber <kruber@zib.de>  Fri, 24 Jul 2015 12:00:00 +0200\n\nscalaris-examples-wiki (0.8.1-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n\n -- Nico Kruber <kruber@zib.de>  Sun, 12 Jul 2015 12:00:00 +0200\n\nscalaris-examples-wiki (0.8.0-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n    and the EIT ICT Labs project MCData\n  * Infrastructure:\n    - move sources to github: https://github.com/scalaris-team/scalaris\n    - new project homepage: http://scalaris.zib.de\n\n -- Nico Kruber <kruber@zib.de>  Fri, 10 Jul 2015 12:00:00 +0200\n\nscalaris-examples-wiki (0.7.2-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n    and the EIT ICT Labs project MCData\n  * Demonstrator \"Wiki on Scalaris\":\n    - fix the separate count key optimisation not using Scalaris' increment\n      operation\n  * Bugs:\n    - fix a few minor bugs\n\n -- Nico Kruber <kruber@zib.de>  Mon, 23 Oct 2014 18:15:00 +0200\n\nscalaris-examples-wiki (0.7.1-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n    and the EIT ICT Labs project MCData\n  * Demonstrator \"Wiki on Scalaris\":\n    - fix storing template back-links for magic words\n    - fix ARTICLE_COUNT partitioning with hashes\n  * Bugs:\n    - fix numerous bugs\n\n -- Nico Kruber <kruber@zib.de>  Mon, 30 Sep 2014 17:00:00 +0200\n\nscalaris-examples-wiki (0.7.0-1) stable; urgency=low\n\n  * Demonstrator \"Wiki on Scalaris\"\n    (supported by 4CaaSt http://www.4caast.eu/\n     and Contrail http://contrail-project.eu):\n    - add support for reading 7z dumps\n  * Bugs:\n    - fix numerous more (less severe) bugs\n\n -- Nico Kruber <kruber@zib.de>  Mon, 28 Apr 2014 12:00:00 +0200\n\nscalaris-examples-wiki (0.6.0-1) stable; urgency=low\n\n  * Demonstrator \"Wiki on Scalaris\" (supported by 4CaaSt http://www.4caast.eu/\n                                     and Contrail http://contrail-project.eu):\n    - separate list counters from their list partitions for a better data layout\n    - fix high memory use of the Scalaris import if the import is slow\n    - use tomcat 7.0.42\n  * Bugs:\n    - fix some more (less severe) bugs\n\n -- Nico Kruber <kruber@zib.de>  Fri, 11 Oct 2013 15:00:00 +0200\n\nscalaris-examples-wiki (0.6.0-1) stable; urgency=low\n\n  * Demonstrator \"Wiki on Scalaris\" (supported by 4CaaSt http://www.4caast.eu/\n                                     and Contrail http://contrail-project.eu):\n    - allow monitoring via JMX in the FourCaastMonitoringPlugin\n    - support for getting random articles via the new partial read op\n    - new optimisation scheme \"Buckets with Write Cache\" - uses a single big\n      list to read from and the rest of the buckets to write to\n    - improve import and dump-processing (faster, more memory-efficient)\n    - add on-the-fly conversion to the different optimisation schemes during\n      import (only one prepared DB dump needed now)\n    - several UI enhancements and rendering fixes\n    - update bliki lib (includes code ported to upstream)\n    - add auto-import ability\n    - use tomcat 7.0.33\n  * Bugs:\n    - fix numerous bugs\n\n -- Nico Kruber <kruber@zib.de>  Fri, 16 Aug 2013 23:20:13 +0200\n\nscalaris-examples-wiki (0.5.0-1) stable; urgency=low\n  * Demonstrator \"Wiki on Scalaris\"\n    (supported by 4CaaSt http://www.4caast.eu/):\n    - allow different partitioned data models for better performance and\n      scalability\n    - allow logging of user requests\n    - added support for checking whether another article exists (approximate)\n      -> show link colours based on this check\n    - added check for bad page titles\n    - allow SERVERNAME and SERVERPATH in config for setups with load\n      balancers\n    - reduced memory footprint of Wiki data in Scalaris\n    - support for newer wiki xml dumps\n    - added support for using a MediaWiki-like SQLite-DB backend for e.g.\n      filtering\n    - improved overall performance\n    - several rendering fixes\n  * Bugs:\n    - fixed numerous bugs\n\n -- Nico Kruber <kruber@zib.de>  Thu, 11 Oct 2012 12:30:00 +0200\n\nscalaris-examples-wiki (0.4.1-1) stable; urgency=low\n\n  * no changes recorded\n\n -- Nico Kruber <kruber@zib.de>  Thu, 22 Mar 2012 10:51:00 +0100\n\nscalaris-examples-wiki (0.4.0-1) stable; urgency=low\n\n  * Demonstrator \"Wiki on Scalaris\"\n    (supported by 4CaaSt http://www.4caast.eu/):\n    - improved performance of page edits\n    - improved performance of Wikipedia dump loading\n    - several rendering fixes\n  * Bugs:\n    - fixed numerous bugs\n\n -- Nico Kruber <kruber@zib.de>  Wed, 25 Jan 2012 00:48:00 +0100\n\nscalaris-examples-wiki (0.3.0-1) stable; urgency=low\n\n  * Demonstrator\n    - added Wikipedia-hosting using Scalaris as demonstrator application\n\n -- Nico Kruber <kruber@zib.de>  Fri, 15 Jul 2011 17:01:00 +0200\n"
  },
  {
    "path": "contrib/packages/examples-wiki/debian.compat",
    "content": "7\n"
  },
  {
    "path": "contrib/packages/examples-wiki/debian.control",
    "content": "Source: scalaris-examples-wiki\nSection: web\nPriority: optional\nMaintainer: Nico Kruber <kruber@zib.de>\nBuild-Depends: debhelper (>= 4.1.16), ant, openjdk-7-jdk | java6-sdk, scalaris-java (>= 0.8.0)\nHomepage: http://scalaris.zib.de\nStandards-Version: 3.9.6\n\nPackage: scalaris-examples-wiki-tomcat5\nArchitecture: all\nDepends: ${misc:Depends}, tomcat5, scalaris-java (>= 0.8.0)\nDescription: Wikipedia on Scalaris example\n This web application demonstrates the use of Scalaris as a data-store\n back-end for a Wikipedia-like application.\n\nPackage: scalaris-examples-wiki-tomcat6\nArchitecture: all\nDepends: ${misc:Depends}, tomcat6, scalaris-java (>= 0.8.0)\nDescription: Wikipedia on Scalaris example\n This web application demonstrates the use of Scalaris as a data-store\n back-end for a Wikipedia-like application.\n"
  },
  {
    "path": "contrib/packages/examples-wiki/debian.rules",
    "content": "#!/usr/bin/make -f\n# Sample debian/rules that uses debhelper.\n# GNU copyright 1997 to 1999 by Joey Hess.\n\n# Uncomment this to turn on verbose mode.\n#export DH_VERBOSE=1\n\nbuild-arch: build\nbuild-indep: build\nbuild: build-stamp\nbuild-stamp:\n\tdh_testdir\n\n\t# Add here commands to compile the package.\n\tln -s /usr/share/java/scalaris/scalaris.jar ./contrib/\n\tJINTERFACE_VERSION=`ls /usr/share/java/scalaris/lib/ | grep ^OtpErlang- | sed \"s|OtpErlang-||\" | sed \"s|.jar||\"`; \\\n\tln -s /usr/share/java/scalaris/lib/OtpErlang-$$JINTERFACE_VERSION.jar ./contrib/\n\tANT_OPTS=\"-Dfile.encoding=utf8 -Dant.build.javac.source=1.6 -Dant.build.javac.target=1.6\" \\\n\t  ant build\n\n\ttouch build-stamp\n\nclean:\n\tdh_testdir\n\tdh_testroot\n\trm -f build-stamp\n\n\t# Add here commands to clean up after the build process.\n\trm -rf build_dir\n\n\tdh_clean\n\nBUILD_DIR_TOMCAT5:=$(CURDIR)/debian/scalaris-examples-wiki-tomcat5\nBUILD_DIR_TOMCAT6:=$(CURDIR)/debian/scalaris-examples-wiki-tomcat6\ninstall: build\n\tdh_testdir\n\tdh_testroot\n\tdh_prep\n\tdh_installdirs\n\n\t# Tomcat 5:\n\tmkdir -p $(BUILD_DIR_TOMCAT5)/etc/tomcat5/Catalina/localhost/\n\tmkdir -p $(BUILD_DIR_TOMCAT5)/usr/share/tomcat5/webapps\n\tcp -r scalaris-wiki $(BUILD_DIR_TOMCAT5)/usr/share/tomcat5/webapps/scalaris-wiki\n\tmkdir -p $(BUILD_DIR_TOMCAT5)/etc/tomcat5/Catalina/localhost/\n\techo \"<Context path=\\\"/scalaris-wiki\\\" docBase=\\\"/usr/share/tomcat5/webapps/scalaris-wiki\\\" />\" >  $(BUILD_DIR_TOMCAT5)/etc/tomcat5/Catalina/localhost/scalaris-wiki.xml\n\n\t# Tomcat 6:\n\tmkdir -p $(BUILD_DIR_TOMCAT6)/usr/share/tomcat6/webapps\n\tcp -r scalaris-wiki $(BUILD_DIR_TOMCAT6)/usr/share/tomcat6/webapps/scalaris-wiki\n\tmkdir -p $(BUILD_DIR_TOMCAT6)/etc/tomcat6/Catalina/localhost/\n\techo \"<Context path=\\\"/scalaris-wiki\\\" docBase=\\\"/usr/share/tomcat6/webapps/scalaris-wiki\\\" />\" >  $(BUILD_DIR_TOMCAT6)/etc/tomcat6/Catalina/localhost/scalaris-wiki.xml\n\n# Build architecture-independent files here.\nbinary-indep: build install\n\tdh_testdir\n\tdh_testroot\n#\tdh_installdebconf\n\tdh_installdocs\n\tdh_installexamples\n\tdh_installmenu\n#\tdh_installlogrotate\n#\tdh_installemacsen\n#\tdh_installpam\n#\tdh_installmime\n#\tdh_installinit\n\tdh_installcron\n\tdh_installman\n\tdh_installinfo\n#\tdh_undocumented\n\tdh_installchangelogs\n\tdh_link\n\tdh_strip\n\tdh_compress\n\tdh_fixperms\n#\tdh_makeshlibs\n\tdh_installdeb\n#\tdh_perl\n#\tdh_shlibdeps\n\tdh_gencontrol\n\tdh_md5sums\n\tdh_builddeb\n\n# Build architecture-dependent files here.\nbinary-arch: build install\n\t# We have nothing to do by default.\n\nbinary: binary-indep binary-arch\n.PHONY: build clean binary-indep binary-arch binary install\n"
  },
  {
    "path": "contrib/packages/examples-wiki/debian.scalaris-examples-wiki-tomcat5.conffiles",
    "content": "/usr/share/tomcat5/webapps/scalaris-wiki/WEB-INF/web.xml\n"
  },
  {
    "path": "contrib/packages/examples-wiki/debian.scalaris-examples-wiki-tomcat6.conffiles",
    "content": "/usr/share/tomcat6/webapps/scalaris-wiki/WEB-INF/web.xml\n"
  },
  {
    "path": "contrib/packages/examples-wiki/debian.source.lintian-overrides",
    "content": "scalaris-examples-wiki source: source-is-missing contrib/libsqlite4java-linux-i386.so\nscalaris-examples-wiki source: source-is-missing contrib/libsqlite4java-linux-amd64.so\n"
  },
  {
    "path": "contrib/packages/examples-wiki/scalaris-examples-wiki-rpmlintrc",
    "content": "addFilter(\"arch-independent-package-contains-binary-or-object.*/usr/share/tomcat.*/webapps/scalaris-wiki/WEB-INF/lib/libsqlite4java.*\")\n"
  },
  {
    "path": "contrib/packages/examples-wiki/scalaris-examples-wiki.changes",
    "content": "-------------------------------------------------------------------\nMon Feb 29 16:00:00 UTC 2016 - kruber@zib.de\n\n- Scalaris 0.9.0 (codename \"Vriesea scalaris\")\n  (partly supported by the EU project IES Cities http://iescities.eu/)\n\n-------------------------------------------------------------------\nFri Jul 24 10:00:00 UTC 2015 - kruber@zib.de\n\n- Scalaris 0.8.2\n  (partly supported by the EU project IES Cities http://iescities.eu/)\n\n-------------------------------------------------------------------\nSun Jul 12 10:00:00 UTC 2015 - kruber@zib.de\n\n- Scalaris 0.8.1\n  (partly supported by the EU project IES Cities http://iescities.eu/)\n\n-------------------------------------------------------------------\nFri Jul 10 10:00:00 UTC 2015 - kruber@zib.de\n\n- Scalaris 0.8.0 (codename \"Picoides scalaris\")\n  (partly supported by the EU project IES Cities http://iescities.eu/\n  and the EIT ICT Labs project MCData)\n  * Infrastructure:\n    - move sources to github: https://github.com/scalaris-team/scalaris\n    - new project homepage: http://scalaris.zib.de\n\n-------------------------------------------------------------------\nMon Oct 23 16:15:00 UTC 2014 - kruber@zib.de\n\n- Scalaris 0.7.2\n  (partly supported by the EU project IES Cities http://iescities.eu/\n  and the EIT ICT Labs project MCData)\n  * Demonstrator \"Wiki on Scalaris\":\n    - fix the separate count key optimisation not using Scalaris' increment\n      operation\n  * Bugs:\n    - fix a few minor bugs\n\n-------------------------------------------------------------------\nMon Sep 29 15:00:00 UTC 2014 - kruber@zib.de\n\n- Scalaris 0.7.1\n  (partly supported by the EU project IES Cities http://iescities.eu/\n  and the EIT ICT Labs project MCData)\n  * Demonstrator \"Wiki on Scalaris\":\n    - fix storing template back-links for magic words\n    - fix ARTICLE_COUNT partitioning with hashes\n  * Bugs:\n    - fix numerous bugs\n\n-------------------------------------------------------------------\nMon Apr 28 10:00:00 UTC 2014 - kruber@zib.de\n\n- Scalaris 0.7.0 (codename \"Stauroderus scalaris\")\n  * Demonstrator \"Wiki on Scalaris\"\n    (supported by 4CaaSt http://www.4caast.eu/\n     and Contrail http://contrail-project.eu):\n    - add support for reading 7z dumps\n  * Bugs:\n    - fix numerous more (less severe) bugs\n\n-------------------------------------------------------------------\nFri Oct 11 13:00:00 UTC 2013 - kruber@zib.de\n\n- Scalaris 0.6.1\n  * Demonstrator \"Wiki on Scalaris\" (supported by 4CaaSt http://www.4caast.eu/\n                                     and Contrail http://contrail-project.eu):\n    - separate list counters from their list partitions for a better data layout\n    - fix high memory use of the Scalaris import if the import is slow\n    - use tomcat 7.0.42\n  * Bugs:\n    - fix some more (less severe) bugs\n\n-------------------------------------------------------------------\nFri Aug 16 21:20:13 UTC 2013 - kruber@zib.de\n\n- Scalaris 0.6.0 (codename \"Conus scalaris\")\n  * Demonstrator \"Wiki on Scalaris\" (supported by 4CaaSt http://www.4caast.eu/\n                                     and Contrail http://contrail-project.eu):\n    - allow monitoring via JMX in the FourCaastMonitoringPlugin\n    - support for getting random articles via the new partial read op\n    - new optimisation scheme \"Buckets with Write Cache\" - uses a single big\n      list to read from and the rest of the buckets to write to\n    - improve import and dump-processing (faster, more memory-efficient)\n    - add on-the-fly conversion to the different optimisation schemes during\n      import (only one prepared DB dump needed now)\n    - several UI enhancements and rendering fixes\n    - update bliki lib (includes code ported to upstream)\n    - add auto-import ability\n    - use tomcat 7.0.33\n  * Bugs:\n    - fix numerous bugs\n\n-------------------------------------------------------------------\nThu Oct 11 10:30:00 UTC 2012 - kruber@zib.de\n\n- Scalaris 0.5.0 (codename \"Saperda scalaris\")\n  * Demonstrator \"Wiki on Scalaris\"\n    (supported by 4CaaSt http://www.4caast.eu/):\n    - allow different partitioned data models for better performance and\n      scalability\n    - allow logging of user requests\n    - added support for checking whether another article exists (approximate)\n      -> show link colours based on this check\n    - added check for bad page titles\n    - allow SERVERNAME and SERVERPATH in config for setups with load\n      balancers\n    - reduced memory footprint of Wiki data in Scalaris\n    - support for newer wiki xml dumps\n    - added support for using a MediaWiki-like SQLite-DB backend for e.g.\n      filtering\n    - improved overall performance\n    - several rendering fixes\n  * Bugs:\n    - fixed numerous bugs\n\n-------------------------------------------------------------------\nThu Mar 22 09:51:00 UTC 2012 - kruber@zib.de\n\n- Scalaris 0.4.1\n  * no changes recorded\n\n-------------------------------------------------------------------\nWed Jan 24 23:48:00 UTC 2012 - kruber@zib.de\n\n- Scalaris 0.4.0 (codename \"Pomacea scalaris\")\n  * Demonstrator \"Wiki on Scalaris\"\n    (supported by 4CaaSt http://www.4caast.eu/):\n    - improved performance of page edits\n    - improved performance of Wikipedia dump loading\n    - several rendering fixes\n  * Bugs:\n    - fixed numerous bugs\n\n-------------------------------------------------------------------\nFri Jul 15 15:01:00 UTC 2011 - kruber@zib.de\n\n- Scalaris 0.3.0 (codename \"Rhinechis Scalaris\")\n  * Demonstrator\n    - added Wikipedia-hosting using Scalaris as demonstrator application\n"
  },
  {
    "path": "contrib/packages/examples-wiki/scalaris-examples-wiki.dsc",
    "content": "Format: 1.0\nSource: scalaris-examples-wiki\nVersion: 0.9.0+git-1\nDEBTRANSFORM-RELEASE: 1\nMaintainer: Nico Kruber <kruber@zib.de>\nArchitecture: all\nStandards-Version: 3.9.6\nBuild-Depends: debhelper (>= 4.1.16), ant, openjdk-7-jdk | java6-sdk, scalaris-java (>= 0.8.0)\nFiles:\n 6ea6383ef9251ff5ac641df84ddbcba0 2008366 scalaris-examples-wiki.orig.tar.gz\n 2fecf324a32123b08cefc0f047bca5ee 0 scalaris-examples-wiki.diff.tar.gz\n"
  },
  {
    "path": "contrib/packages/examples-wiki/scalaris-examples-wiki.spec",
    "content": "# norootforbuild\n\n%define pkg_version 0.9.0+git\nName:           scalaris-examples-wiki\nSummary:        Wikipedia on Scalaris example\nVersion:        %{pkg_version}\nRelease:        1\nLicense:        Apache-2.0\nGroup:          Productivity/Databases/Servers\nURL:            http://scalaris.zib.de\nSource0:        %{name}-%{version}.tar.gz\nSource100:      checkout.sh\nBuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-build\nBuildRequires:  ant\nBuildRequires:  java-devel >= 1.6.0\nBuildRequires:  scalaris-java >= 0.8.0\n\n##########################################################################################\n## Fedora, RHEL or CentOS\n##########################################################################################\n%if 0%{?fedora_version} || 0%{?rhel_version} || 0%{?centos_version}\n# the wiki includes some libraries which are normally not allowed in noarch packages:\n%global _binaries_in_noarch_packages_terminate_build 0\n# we also do not need a debuginfo sub package\n%global debug_package %{nil}\n%if 0%{?centos_version} >= 600 || 0%{?rhel_version} >= 600 || 0%{?fedora_version} >= 17\n%define with_tomcat5 0\n%else\n%define with_tomcat5 1\n%endif\n%if 0%{?fedora_version} >= 19\n%define with_tomcat6 0\n%else\n%if 0%{?centos_version} >= 600 || 0%{?rhel_version} >= 600 || 0%{?fedora_version}\n%define with_tomcat6 1\n%else\n%define with_tomcat6 0\n%endif\n%endif\n%if 0%{?fedora_version} >= 16\n%define with_tomcat7 1\n%else\n%define with_tomcat7 0\n%endif\n%endif\n\n###########################################################################################\n# SuSE, openSUSE\n###########################################################################################\n%if 0%{?suse_version}\n%if 0%{?suse_version} >= 1110\n%define with_tomcat5 0\n%else\n%define with_tomcat5 1\n%endif\n%if 0%{?sles_version} == 10 || 0%{?suse_version} >= 1220\n%define with_tomcat6 0\n%else\n%define with_tomcat6 1\n%endif\n%if 0%{?suse_version} >= 1220\n%define with_tomcat7 1\n%else\n%define with_tomcat7 0\n%endif\n%endif\n\n%description\nThis web application demonstrates the use of Scalaris as a data-store back-end for a\nWikipedia-like application.\n\n%if 0%{?with_tomcat5}\n%package -n scalaris-examples-wiki-tomcat5\nSummary:    Wikipedia on Scalaris example using tomcat5\nGroup:      Productivity/Networking/Web/Servers\nRequires:   tomcat5\nRequires:   scalaris-java >= 0.8.0\nBuildArch:  noarch\n\n%description -n scalaris-examples-wiki-tomcat5\nThis web application demonstrates the use of Scalaris as a data-store back-end for a\nWikipedia-like application.\n%endif\n\n%if 0%{?with_tomcat6}\n%package -n scalaris-examples-wiki-tomcat6\nSummary:    Wikipedia on Scalaris example using tomcat6\nGroup:      Productivity/Networking/Web/Servers\nRequires:   tomcat6\nRequires:   scalaris-java >= 0.8.0\nBuildArch:  noarch\n\n%description -n scalaris-examples-wiki-tomcat6\nThis web application demonstrates the use of Scalaris as a data-store back-end for a\nWikipedia-like application.\n%endif\n\n%if 0%{?with_tomcat7}\n%package -n scalaris-examples-wiki-tomcat7\nSummary:    Wikipedia on Scalaris example using tomcat7\nGroup:      Productivity/Networking/Web/Servers\nRequires:   tomcat >= 7.0.0\nRequires:   scalaris-java >= 0.8.0\nBuildArch:  noarch\n\n%description -n scalaris-examples-wiki-tomcat7\nThis web application demonstrates the use of Scalaris as a data-store back-end for a\nWikipedia-like application.\n%endif\n\n%prep\n%setup -q -n %{name}-%{version}\n\n%build\nexport ANT_OPTS=\"-Dfile.encoding=utf8 -Dant.build.javac.source=1.6 -Dant.build.javac.target=1.6\"\n\nln -s %{_javadir}/scalaris/scalaris.jar ./contrib/\nexport JINTERFACE_VERSION=`ls %{_javadir}/scalaris/lib/ | grep ^OtpErlang- | sed \"s|OtpErlang-||\" | sed \"s|.jar||\"`\nln -s %{_javadir}/scalaris/lib/OtpErlang-$JINTERFACE_VERSION.jar ./contrib/\nant build\n\n%install\n# see http://en.opensuse.org/Java/Packaging/Cookbook#bytecode_version_error\nexport NO_BRP_CHECK_BYTECODE_VERSION=true\nrm -rf $RPM_BUILD_ROOT\n\n%if 0%{?with_tomcat5}\nmkdir -p %{buildroot}/usr/share/tomcat5/webapps\ncp -r scalaris-wiki %{buildroot}/usr/share/tomcat5/webapps/scalaris-wiki\n%endif\n%if 0%{?with_tomcat6}\nmkdir -p %{buildroot}/usr/share/tomcat6/webapps\ncp -r scalaris-wiki %{buildroot}/usr/share/tomcat6/webapps/scalaris-wiki\n%endif\n%if 0%{?with_tomcat7}\nmkdir -p %{buildroot}/usr/share/tomcat/webapps\ncp -r scalaris-wiki %{buildroot}/usr/share/tomcat/webapps/scalaris-wiki\n%endif\n\n%clean\nrm -rf $RPM_BUILD_ROOT\n\n%if 0%{?with_tomcat5}\n%files -n scalaris-examples-wiki-tomcat5\n%defattr(-,root,root,-)\n%dir /usr/share/tomcat5\n%dir /usr/share/tomcat5/webapps\n/usr/share/tomcat5/webapps/scalaris-wiki\n%endif\n\n%if 0%{?with_tomcat6}\n%files -n scalaris-examples-wiki-tomcat6\n%defattr(-,root,root,-)\n%dir /usr/share/tomcat6\n%dir /usr/share/tomcat6/webapps\n/usr/share/tomcat6/webapps/scalaris-wiki\n%endif\n\n%if 0%{?with_tomcat7}\n%files -n scalaris-examples-wiki-tomcat7\n%defattr(-,root,root,-)\n%dir /usr/share/tomcat\n%dir /usr/share/tomcat/webapps\n/usr/share/tomcat/webapps/scalaris-wiki\n%endif\n\n%changelog\n"
  },
  {
    "path": "contrib/packages/main/PKGBUILD",
    "content": "# Maintainer: Nico Kruber <kruber at zib dot de>\n# Contributor: Roman Parykin <donderom at ymail dot com>\npkgname=scalaris\npkgver=0.9.0+git\npkgrel=1\npkgdesc=\"A scalable, transactional, distributed key-value store.\"\narch=('i686' 'x86_64')\nurl=\"http://scalaris.zib.de\"\nlicense=('Apache')\ndepends=('openssl' 'erlang>=R14B04' 'util-linux>=2.23' 'systemd')\nmakedepends=('screen')\noptdepends=('tokyocabinet: storage on disk'\n            'screen: scalaris daemon with screen')\nbackup=('etc/conf.d/scalaris' 'etc/conf.d/scalaris-first' 'etc/scalaris/scalaris.local.cfg' 'etc/scalaris/scalarisctl.conf')\nsource=($pkgname-$pkgver.tar.gz)\nmd5sums=('f188e12f4cf893022a87a1b29971aa81')\ninstall=install\n\nbuild() {\n  cd \"$srcdir/$pkgname-$pkgver\"\n  ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var \\\n              --with-systemd\n  make all\n  make doc\n}\n\npackage() {\n  cd \"$srcdir/$pkgname-$pkgver\"\n  make DESTDIR=\"$pkgdir/\" install\n  rmdir $pkgdir/var/run\n  make DESTDIR=\"$pkgdir/\" install-doc\n  \n  # put the LICENSE file to the licenses\n  install -D -m644 LICENSE \"$pkgdir/usr/share/licenses/$pkgname/LICENSE\"\n}\n"
  },
  {
    "path": "contrib/packages/main/checkout.sh",
    "content": "#!/bin/bash\n\nSCALARIS_VERSION=\"0.9.0+git\"\ndate=`date +\"%Y%m%d.%H%M\"`\nname=\"scalaris\" # folder base name (without version)\npkg_name=\"scalaris\" # package name\nurl=\"https://github.com/scalaris-team/scalaris.git\"\ndeletefolder=0 # set to 1 to delete the folder the repository is checked out to\n\n#####\nif [[ \"$SCALARIS_VERSION\" == *git* ]]; then\n  MODE=\"snapshot\"\nelse\n  MODE=\"tag\"\nfi\n\nif [ \"$MODE\" = \"snapshot\" ] ; then\n  folder=\"${name}\"\n  if [ ! -d \"${folder}\" ]; then\n    echo \"checkout ${url} -> ${folder} ...\"\n    git clone --branch master --single-branch --depth=1 \"${url}\" \"${folder}\"\n    result=$?\n  else\n    echo \"update ${url} -> ${folder} ...\"\n    cd \"${folder}\"\n    git pull\n    result=$?\n    cd - >/dev/null\n  fi\n\n  if [ ${result} -eq 0 ]; then\n    echo -n \"get git revision ...\"\n    revision=`cd \"${folder}\" && git log --pretty=format:'%h' -n 1`\n    result=$?\n    echo \" ${revision}\"\n    pkg_version=\"${SCALARIS_VERSION}${date}.${revision}\"\n  fi\nelse\n  pkg_version=\"${SCALARIS_VERSION}\"\n  folder=\"${name}-${pkg_version}\"\n  tarfile=\"${folder}.tar.gz\"\n  echo \"downloading archive ${url%.git}/archive/${SCALARIS_VERSION}.tar.gz ...\"\n  wget \"${url%.git}/archive/${SCALARIS_VERSION}.tar.gz\" -O \"${tarfile}\" && \\\n  tar -xzf \"${tarfile}\"\n  result=$?\n  if [ ! -d \"${folder}\" ]; then\n    echo \"wrong archive contents, expecting folder ${folder}\"\n    exit 1\n  fi\nfi\n\nif ! diff -q checkout.sh \"${folder}/contrib/packages/main/checkout.sh\" > /dev/null ; then\n  echo \"checkout-script changed - re-run ./checkout.sh\"\n  cp \"${folder}/contrib/packages/main/checkout.sh\" ./\n  exit 1\nfi\n\nif [ \"$MODE\" = \"snapshot\" ] ; then\n  if [ ${result} -eq 0 ]; then\n    tarfile=\"${name}-${pkg_version}.tar.gz\"\n    newfoldername=\"${name}-${pkg_version}\"\n    echo \"making ${tarfile} ...\"\n    mv \"${folder}\" \"${newfoldername}\" && tar -czf \"${tarfile}\" \"${newfoldername}\" --exclude-vcs && mv \"${newfoldername}\" \"${folder}\"\n    result=$?\n  fi\nfi\n\nif [ ${result} -eq 0 ]; then\n  echo \"extracting .spec file ...\"\n  sourcefolder=\"${folder}/contrib/packages/main\"\n  sed -e \"s/%define pkg_version .*/%define pkg_version ${pkg_version}/g\" \\\n      < \"${sourcefolder}/${pkg_name}.spec\"           > \"./${pkg_name}.spec\" && \\\n  cp  \"${sourcefolder}/${pkg_name}.changes\"            \"./${pkg_name}.changes\" && \\\n  cp  \"${sourcefolder}/${pkg_name}-rpmlintrc\"          \"./${pkg_name}-rpmlintrc\"\n  result=$?\nfi\n\nif [ ${result} -eq 0 ]; then\n  echo \"extracting Debian package files ...\"\n  sourcefolder=\"${folder}/contrib/packages/main\"\n  sed -e \"s/Version: .*-.*/Version: ${pkg_version}-1/g\" \\\n      -e \"s/scalaris.*\\\\.orig\\\\.tar\\\\.gz/${pkg_name}-${pkg_version}\\\\.orig\\\\.tar\\\\.gz/g\" \\\n      -e \"s/scalaris.*\\\\.diff\\\\.tar\\\\.gz/${pkg_name}-${pkg_version}\\\\.diff\\\\.tar\\\\.gz/g\" \\\n      < \"${sourcefolder}/${pkg_name}.dsc\"             > \"./${pkg_name}.dsc\" && \\\n  ( ( test \"$MODE\" != \"snapshot\" && \\\n      sed -e \"0,/(.*-.*)/s//(${pkg_version}-1)/\" \\\n      < \"${sourcefolder}/debian.changelog\"            > ./debian.changelog ) || \\\n    sed -e \"0,/(.*-.*)/s//(${pkg_version}-1)/\" \\\n        -e \"0,/ -- Nico Kruber <kruber@zib.de>  .*/s// -- Nico Kruber <kruber@zib.de>  `LANG=C date -R`/\" \\\n      < \"${sourcefolder}/debian.changelog\"            > ./debian.changelog ) && \\\n  cp  \"${sourcefolder}/debian.compat\"                   ./debian.compat && \\\n  cp  \"${sourcefolder}/debian.control\"                  ./debian.control && \\\n  cp  \"${sourcefolder}/debian.rules\"                    ./debian.rules && \\\n  cp  \"${sourcefolder}/debian.scalaris.prerm\"           ./debian.scalaris.prerm && \\\n  cp  \"${sourcefolder}/debian.scalaris.postrm\"          ./debian.scalaris.postrm && \\\n  cp  \"${sourcefolder}/debian.scalaris.postinst\"        ./debian.scalaris.postinst && \\\n  cp  \"${sourcefolder}/debian.source.lintian-overrides\" ./debian.source.lintian-overrides && \\\n  cp  \"${folder}/LICENSE\"                               ./debian.copyright\n  result=$?\nfi\n\nif [ ${result} -eq 0 ]; then\n  echo \"extracting ArchLinux package files ...\"\n  sourcefolder=\"${folder}/contrib/packages/main\"\n  tarmd5sum=`md5sum \"${tarfile}\" | cut -d' ' -f 1` && \\\n  sed -e \"s/pkgver=.*/pkgver=${pkg_version}/g\" \\\n      -e \"s/md5sums=('.*')/md5sums=('${tarmd5sum}')/g\" \\\n      < \"${sourcefolder}/PKGBUILD\"                   > ./PKGBUILD && \\\n  cp  \"${sourcefolder}/install\"                        ./install\n  result=$?\nfi\n\nif [ ${result} -eq 0 -a ${deletefolder} -eq 1 ]; then\n  echo \"removing ${folder} ...\"\n  rm -rf \"${folder}\"\nfi\n"
  },
  {
    "path": "contrib/packages/main/debian.changelog",
    "content": "scalaris (0.9.0+git-1) unstable; urgency=low\n\n  * git snapshot from master\n\n -- Nico Kruber <kruber@zib.de>  Mon, 29 Feb 2016 18:00:00 +0200\n\nscalaris (0.9.0-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n  * Packaging:\n    - add a libscalaris-dev[el] package with the C++ API (rpm, deb)\n    - Maven: let Maven also create a source jar file for the Java-API\n    - Maven: allow proper snapshot versions for the maven repository at\n             https://scalaris-team.github.io/scalaris/maven\n  * API:\n    - CPP-API: provide a new C++ client library\n    - DataNucleus: provide a new storage back-end for Scalaris\n    - add a Scalaris adapter for YCSB\n    - new functions in all APIs for generic routing table operations\n  * Business Logic:\n    - rrepair: considerably improve the set reconciliation algorithms' performance\n               and run more processes in parallel\n    - rrepair: further reduce the reconciliation costs\n    - rrepair: re-use left-over failure probability from phase 1 for phase 2\n    - rrepair: increase accuracy in several formulaes of the set reconciliation\n               algorithms\n    - rrepair: support creating multiple Merkle trees for fewer message rounds\n               ('rr_merkle_num_trees' configuration variable)\n    - rrepair: remove support for aligning bit sizes (no improvements there)\n    - transactions: speed up write ops when the transaction log already contains\n                    a read op on the same key\n    - RM: optimise the traffic of state exchanges during ring maintenance\n    - RT: improve routing parallelism by handling more messages in the routing\n          process itself instead of inside the dht_node process\n    - RT: improve performance along the routing path\n    - support for dynamic replication factors given by the 'replication_factor'\n      configuration variable\n    - pid_groups: shuffle pids in find_a/1 for more better distributed (parallel)\n                  requests\n  * Infrastructure:\n    - pid_groups: group names are now atoms instead of strings\n                  (allows faster access)\n    - scalarisctl: do not set any default Erlang scheduler flags (allow changes\n                   through the ERL_SCHED_FLAGS environment variable)\n    - extend the example scripts for running Scalaris on a SLURM cluster\n    - add SLURM scripts for running Basho Bench\n    - use configure to select the routing table type (--with-rt=rt_chord)\n    - change hanoidb detection to \"configure --enable-hanoidb=<dir>\"\n    - change erlang_js detection to \"configure --enable-erlang-js=<dir>\"\n    - dht_node_monitor: switch messages on/off via configuration variable\n                        'dht_node_monitor'\n    - msg_delay: remove extraneous error messages for trigger messages of killed\n                 processes\n    - prbr: allow write_filter to return value to qwrite calling process\n    - update yaws to version 2.0.2\n    - support for Erlang R14B04 up to 18.2.3 and current otp master\n  * Tests:\n    - allow running single test groups via \"make <test suite>:<group name>_GROUP\"\n  * Tools:\n    - gen_component: allow processes to provide their own start functions\n    - trace_mpath: write whole message of log_info to the tex output\n  * Bugs:\n    - fix cleanup of some special cases with transactions\n    - fix several math issues and increase the math functions' performance\n    - fix some errors which occurred in large scale deployments\n    - fix several warnings reported by Coverity Scan\n    - fix numerous more bugs\n\n -- Nico Kruber <kruber@zib.de>  Mon, 29 Feb 2016 18:00:00 +0200\n\nscalaris (0.8.2-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n  * Business Logic:\n    - rrepair: minor performance improvements\n  * Bugs:\n    - rrepair: fix a rare crash due to hash collisions\n    - rrepair: fix calculation of the next MaxItemCount in the merkle tree\n               repair protocol (was one level behind)\n    - tests: fix rare crashes in some unit tests\n    - fix a few more minor bugs\n\n -- Nico Kruber <kruber@zib.de>  Fri, 24 Jul 2015 12:00:00 +0200\n\nscalaris (0.8.1-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n  * Business Logic:\n    - rrepair: allow specifying the minimum number of hash bits to use\n               to create non-approximate algorithms\n  * Bugs:\n    - rrepair: fix a badarith if rr_recon_version_bits is set to 'variable'\n    - rrepair: fix a bug in the accuracy of the evaluation\n    - fix two crashes in gossip\n\n -- Nico Kruber <kruber@zib.de>  Sun, 12 Jul 2015 12:00:00 +0200\n\nscalaris (0.8.0-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n    and the EIT ICT Labs project MCData\n  * Packaging:\n    - also install docs on Arch Linux\n    - fix lintian errors and warnings for Debian-based packages\n    - fix permissions on the log directory (owned by the 'scalaris' user)\n      and the config files (not owned by the 'scalaris' user anymore!)\n    - adapt packages for newer distributions\n  * API:\n    - Java-API: integrate new OtpErlang library (1.5.12 from Erlang 17.4)\n    - Python-API: do not include a shebang with a pre-defined path to python\n                  anymore\n    - Python-API: fix unclosed sockets in various tests\n    - Python-API: increase python{2,3} test speed\n    - Ruby-API: try to fallback to the normal json module if the gem is missing\n    - Ruby-API: considerably increase the ruby test speed\n    - remove the Publish/Subscribe API\n  * Business Logic:\n    - rrepair: faster and more accurate bloom filter repair protocol\n    - rrepair: faster, more accurate and less bandwidth-consuming merkle tree\n               repair protocol\n    - rrepair: reduce memory footprint of merkle_tree\n    - rrepair: more accurate and less bandwidth-consuming trivial repair\n               protocol\n    - rrepair: add a new simple hash ('shash') algorithm\n    - rrepair: switch to (the more efficient) push-only resolve phase for\n               synchronisation\n    - rrepair: add replica repair algorithm evaluation scripts\n    - RM: faster cyclon cache integration, especially during node jumps\n    - comm_layer: envelops can be nested\n    - comm_layer: allow send options for comm:send_local/3 and msg_delay\n                  (currently only ?quiet supported)\n  * Infrastructure:\n    - FD: simplify the API, support envelopes and remove own cookie handling\n    - FD: extend the failure detector with generic notifications (not just\n          crashes) to subscribed nodes\n    - DB: add an (experimental) mnesia back-end\n    - DB: add an (experimental) hanoidb back-end\n    - DB: make the back-end configurable via the scalaris.cfg option\n          'db_backend'\n    - improve the speed of several utility functions\n    - scalarisctl: add '-l' parameter for specifying the logdir\n    - scalarisctl: do not use the '-s' parameter anymore, instead specify the\n                   start type with the new '-t' parameter\n    - scalarisctl: allow starting several dht_nodes per vm with given keys via\n                   the new parameter '-j'\n    - update yaws to version 1.99\n    - make \"$HOME/.scalaris/log\" the default log dir\n    - support for Erlang R14B04 up to 18.0.2 and current otp master\n    - add example scripts for running Scalaris on a SLURM cluster\n    - move sources to github: https://github.com/scalaris-team/scalaris\n    - new project homepage: http://scalaris.zib.de\n  * Tests:\n    - reduce log spam in proto_sched tests and only print proto_sched results\n      when the tests fail\n    - extended and improved various test suites\n  * Tools:\n    - proto_sched: support callbacks\n    - proto_sched: add support for user msg shepherds\n    - proto_sched: better error handling when sending messages to dead\n                   (local) processes\n    - tester: add value creator and type checker for orddict:orddict()\n    - tester: full support for Erlang 18.0\n    - let 'make dialyzer' run over test as well and ignore some false-positives\n      for Erlang >= 18.0 using the '-dialyzer()' attribute\n  * Bugs:\n    - slide: fix some race conditions with 'jump' operations\n    - JSON-API: fix test_and_set not working correctly inside req_list\n    - fix numerous more bugs\n\n -- Nico Kruber <kruber@zib.de>  Fri, 10 Jul 2015 12:00:00 +0200\n\nscalaris (0.7.2-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n    and the EIT ICT Labs project MCData\n  * Packaging:\n    - fix ArchLinux packages with newest Java versions\n  * Demonstrator \"Wiki on Scalaris\":\n    - fix the separate count key optimisation not using Scalaris' increment\n      operation\n  * Business Logic:\n    - rrepair: let the trivial algorithm assume the worst case in order to always\n               meet the configured \"recon probability of one error\" (p1e)\n    - rrepair: fix the trivial algorithm having an effectively doubled p1e\n    - rrepair: fix the bloom algorithm having an effectively tripled p1e\n    - rrepair: allow disabling byte-alignment\n  * Bugs:\n    - fix a few minor bugs\n\n -- Nico Kruber <kruber@zib.de>  Mon, 23 Oct 2014 18:15:00 +0200\n\nscalaris (0.7.1-1) stable; urgency=low\n\n  * partly supported by the EU project IES Cities http://iescities.eu/\n    and the EIT ICT Labs project MCData\n  * Packaging:\n    - add support for new distribution versions\n    - support systemd and SELinux\n    - include daemon for monitoring Scalaris through JMX\n  * API:\n    - Java-API: integrate new OtpErlang library (1.5.10 from Erlang 17.3)\n  * Demonstrator \"Wiki on Scalaris\":\n    - fix storing template back-links for magic words\n    - fix ARTICLE_COUNT partitioning with hashes\n  * Business Logic:\n    - more robust (still experimental) support for active load balancing with\n      Karger and Ruhl's algorithm including more flexible \"load\" definitions\n    - rm_tman: less overhead by only sending unknown nodes to neighbours\n    - slide: better support for slide aborts\n    - node join: less overhead during joins, especially in setups with huge\n                 lists of known nodes\n    - cyclon: move to new gossip framework\n    - vivaldi: move to new gossip framework\n    - monitor: move performance monitor to the basic_services group\n               (once per VM)\n  * Infrastructure:\n    - several smaller performance optimisations\n    - support for Erlang R13B01 up to 17.3 and current otp master\n  * Tests:\n    - some new unit tests and higher test coverage\n  * Documentation:\n    - add rrepair sequence diagrams\n  * Bugs:\n    - fix numerous bugs\n\n -- Nico Kruber <kruber@zib.de>  Mon, 30 Sep 2014 17:00:00 +0200\n\nscalaris (0.7.0-1) stable; urgency=low\n\n  * API:\n    - Java-API: integrate new OtpErlang library (1.5.9 from Erlang 17.0)\n  * Demonstrator \"Wiki on Scalaris\"\n    (supported by 4CaaSt http://www.4caast.eu/\n     and Contrail http://contrail-project.eu):\n    - add support for reading 7z dumps\n  * Business Logic\n    (partly supported by the EU project IES Cities http://iescities.eu/\n     and the EIT ICT Labs project MCData):\n    - add an experimental Map-Reduce framework on top of Scalaris\n    - add experimental support for active load balancing with Karger and\n      Ruhl's algorithm including more flexible \"load\" definitions\n    - rrepair: completely new merkle tree sync protocol with an order of\n               magnitude lower traffic costs (use dynamic signature sizes based\n               on a \"recon probability of one error\" (p1e) in the leaf nodes)\n    - rrepair: add a trivial probabilistic reconciliation protocol using the\n               \"recon probability of one error\" (p1e)\n    - rrepair: replace bloom_fpr parameter with a generic \"recon probability of\n               one error\" (p1e)\n    - rrepair: add a trivial reconciliation phase for the differences identified\n               by bloom\n    - rrepair: more efficient resolve also including a list of keys to request\n    - gossip: completely new (more flexible) gossip framework\n    - slide: add support for 'jump' operations\n    - slide: do not create a timer for each received message (use a periodic\n             cleanup)\n    - comm_layer: tune TCP connection parameters to increase throughput and\n                  latency (no delay_send)\n    - comm_layer: close idle TCP connections, support for no_keep_alive\n                  connections (used by cyclon, for example)\n    - comm_layer: try to bundle more messages\n    - node join: less overhead during joins, especially in small rings\n    - rm_tman: less overhead by not adding nodes in the cache to the dn_cache\n    - rt_chord: skip trying to contact nodes in the own range\n    - rt_chord: stop stabilize when the own node is reached\n  * Infrastructure:\n    - intervals: more compact (and transfer-friendly) representation\n    - trigger: replace modules with msg_delay:send_trigger/2\n    - config: replace implementation with a public ets table\n    - tune the garbage collection of some core processes and periodically\n      garbage collect all processes\n    - reduce the number of generated atoms and the number of ets tables used\n    - several smaller performance optimisations\n    - support for Erlang R13B01 up to 17.0 and current otp master\n  * Tests:\n    - improve test coverage (manual tests and random tests)\n    - change test definitions and allow groups of tests, e.g.\n      test (default), test-skipped,\n      all_TESTS, all_with_cover_TESTS, performance_TESTS, proto_sched_TESTS,\n      type_check_TESTS\n  * Documentation:\n    - add rrepair documentation\n  * Tools:\n    - proto_sched: new API, better determinism, more strict self-control\n    - proto_sched: support for short-lived processes\n    - proto_sched: more information in get_infos\n    - trace_mpath: more flexible LaTeX exports with more detailed message info\n    - trace_mpath: support meaningful PID names when tracing remote messages\n    - tester: full support for Erlang 17.0\n    - make: re-compile after Emakefile changes\n  * Bugs:\n    - rrepair: fix unnecessary feedback for KVV items of the same version\n    - bulkowner: some fixes for (gracefully) leaving nodes\n    - fix numerous more (less severe) bugs\n\n -- Nico Kruber <kruber@zib.de>  Mon, 28 Apr 2014 12:00:00 +0200\n\nscalaris (0.6.1-1) stable; urgency=low\n\n  * Packaging:\n    - add ArchLinux ruby API packages\n  * API:\n    - Java-API: add Maven build support\n    - Java-API: add CircularByteArrayOutputStream#clear()\n    - Java-API: fix ConnectionPool#getConnection(timeout) throwing\n                IllegalMonitorStateException if a single ConnectionPool is used\n                by multiple threads and no more connections are available\n    - Java-API: add a \"-monitor\" command line parameter\n    - api_monitor: return the latency and stddev values of the micro-benchmark\n                   executed by monitor_perf for node and service performance\n  * Demonstrator \"Wiki on Scalaris\" (supported by 4CaaSt http://www.4caast.eu/\n                                     and Contrail http://contrail-project.eu):\n    - separate list counters from their list partitions for a better data layout\n    - fix high memory use of the Scalaris import if the import is slow\n    - use tomcat 7.0.42\n  * Business Logic:\n    - rrepair: reduce overhead of ART reconciliation\n    - rrepair: allow resolving of multiple merkle node leaves with a single\n               resolve request\n    - rrepair: don't create resolve requests for empty intervals\n    - rrepair: reduce overhead of update_key_entry requests (use a single\n               request with all the data instead of one request for each item)\n    - rrepair: allow arbitrary intervals in interval_upd and interval_upd_send\n               resolve requests again\n    - rrepair: when hashing merkle_tree/ART children, also include the\n               represented interval (fixes indistinguishable empty leaf nodes\n               in ART)\n    - tx_tm_rtm: re-enable takeover by rtms on tm crash\n    - rt_chord: only re-build the RT if the pred or succ processes change or the\n                own new node ID is not between the new pred and succ any more\n  * Infrastructure:\n    - DB: improve performance of fold[lr] implementations by a factor of 2\n    - FD: get rid of annoying, wrongly raised log warnings\n    - RM: provide a more generic Reason for RM subscriptions\n    - RM: remove trigger infections\n    - RT: don't trigger an update when a slide finishes\n    - RT, FRT: reduce the number of messages sent on lookups\n    - log: don't exit with a badmatch if our error_logger is not the only one\n  * Tests:\n    - add protocol scheduler tests for slide, join and leave\n  * Documentation:\n    - user-dev-guide: add section on scalarisctl checkinstallation\n    - user-dev-guide: clarify the section about how to set up Scalaris\n  * Tools:\n    - proto scheduler: continue in case of send errors\n    - top: improve process messages output\n  * Bugs:\n    - node join: fix not being able to join a system with passive load balancer\n                 if the number of items in the DB is too high\n    - tx_tm_rtm: fix wrong asserts\n    - fix node resposibility check not always including message forward and\n      db_range intervals\n    - tx_tp: add missing snapshot number in tp_do_commit_abort message\n    - bulkowner: respect the nodes' DB ranges and forward requests for\n                 non-responsible ranges\n    - dn_cache: fix reporting wrong PIDs back as zombies after node reboots\n    - fix some more (less severe) bugs\n\n -- Nico Kruber <kruber@zib.de>  Fri, 11 Oct 2013 15:00:00 +0200\n\nscalaris (0.6.0-1) stable; urgency=low\n\n  * Packaging:\n    - add ArchLinux packages\n    - add support for new distribution versions\n  * API:\n    - no more timeouts in client APIs\n    - Java-API: re-worked the request and result list handling\n      -> move result processing to the operation classes\n    - Java-API: better support for custom operations\n    - Java-API: support the new partial reads:\n                ReadRandomFromListOp and ReadSublistOp\n    - Java-API: compile with \"vars\" debug info\n    - Java-API: integrate new OtpErlang library (1.5.8 from R16B) with fixed\n                support for compressed binaries\n    - Java-API: add back-ports from the Wiki on Scalaris demonstrator:\n      * list-change operations: ScalarisChangeListOp and\n                                ScalarisListAppendRemoveOp\n      * MultiMap classes are now in de.zib.tools\n      * CircularByteArrayOutputStream\n    - Java-API: fix hostname issues with Erlang and Java\n    - Java-API: slightly changed the delete API\n    - JSON-API: add API for auto-scale requests\n    - Python-API: add API for auto-scale requests\n    - Python-API: use default socket timeout\n    - Ruby-API: use default socket timeout\n    - all APIs: support lists of composite types\n  * Demonstrator \"Wiki on Scalaris\" (supported by 4CaaSt http://www.4caast.eu/\n                                     and Contrail http://contrail-project.eu):\n    - allow monitoring via JMX in the FourCaastMonitoringPlugin\n    - support for getting random articles via the new partial read op\n    - new optimisation scheme \"Buckets with Write Cache\" - uses a single big\n      list to read from and the rest of the buckets to write to\n    - improve import and dump-processing (faster, more memory-efficient)\n    - add on-the-fly conversion to the different optimisation schemes during\n      import (only one prepared DB dump needed now)\n    - several UI enhancements and rendering fixes\n    - update bliki lib (includes code ported to upstream)\n    - add auto-import ability\n    - use tomcat 7.0.33\n  * Business Logic:\n    - replace common message tags with integers to reduce bandwidth\n    - more flexible read operations (easier to extend)\n    - add support for the following partial reads: random_from_list and sublist\n    - save bandwidth by not returning the full value for write operations\n      (only the version is required)\n    - new DB back-end implementation with a smaller and cleaner interface\n    - faster DB get_chunk processing\n    - tx: allow overwriting old/outdated DB entries\n    - tx: allow overwriting old/outdated write-locked entries\n    - tx: allow setting write lock on old/outdated read-locked entries\n    - tx: always reply when the majority replied during read\n    - tx: make sure that if not_found is reported to the user (while reading),\n          a write cannot go through if it is not also based on not_found\n    - tx: committing a test_and_set op on a non-existing entry now fails as well\n          (the op itself already returned the failure)\n    - tx: add a 2s delay to wait for slow learner_decide answers before cleaning\n          up (results in a faster state cleanup after the fourth response)\n    - tx: small performance improvements in several modules\n    - rm: only add alive, non-leaving nodes\n    - rm: if a predecessor crashes, start repairing the range (rrepair)\n    - rrepair: stabilised rrepair (not considered experimental any more)\n    - rrepair: also update entries with existing but outdated WriteLocks\n    - rrepair: several performance improvements\n               (bloom, merkle_tree, art and rrepair processes in general)\n    - rrepair: re-design of rr_recon\n    - rrepair: don't offload heavy work onto the dht_node (increases\n               responsiveness of the dht_node process during replica repair)\n    - rrepair: improve db_generator tool and random_bias binomial distribution\n               used for tests\n    - rrepair: support differently configured nodes (use the same reconciliation\n               structure parameters)\n    - rrepair: de-activate self-repair (a node with multiple copies of the same\n               items does not need a reconciliation structure to repair some of\n               them)\n    - rrepair: activate rrepair periodically every 10 minutes with a probability\n               of 33%\n    - slide v2.0: fewer message to initiate a slide\n    - slide v2.0: generic (asynchronous) call-backs for different ring\n                  maintenance algorithms\n    - slide v2.0: re-work handling of planned next operations, e.g. used by\n                  incremental slides\n    - slide v2.0: don't directly work on the DB any more (there may be more\n                  data needed to slide) - let dht_node_state decide\n    - slide v2.0: activate incremental join and leave operations\n    - slide v2.0: actively report graceful node shutdown to the local FD of the\n                  leaving node to inform subscribers\n    - slide v2.0: code clean-up\n    - slide v2.0: some fixes for incremental slides\n    - slide v2.0: more robust in general\n    - more smooth node joins by also reporting when a join is not possible due\n      to a running slide at the existing node\n    - passive load balancing: random selection of (equally qualified) nodes\n    - add new routing algorithms FRT-Chord (flexible routing tables) and\n      GFRT-Chord (supports proximity routing and data centers) as alternatives\n      to Chord (see rt_frtchord and rt_gfrtchord modules)\n    - add auto-scale framework, e.g. for cloud environments (supported by\n      Contrail http://contrail-project.eu/) which is able to scale the\n      deployment to maintain a given target latency of executed transactions\n    - cache config reads in the process dictionary for better performance\n    - cyclon: if the cache is empty, try one of the nodes in known_hosts\n    - add support for consistent snapshots (experimental)\n  * Infrastructure:\n    - add a daemon to monitor Scalaris via JMX\n    - disable message compression (only client values are compressed - the rest\n      is too expensive, at least on GbE)\n    - support for distributions with python3 available as \"python\" and\n      python2 as \"python2\"\n    - support for Ruby 1.9\n    - yaws 1.96 (with patch to compile on otp master and a patch to fix a\n      performance regression)\n    - support for Erlang R13B01 up to R16B01 and current otp master\n  * Tests:\n    - add test suite to find memory leaks\n    - let \"make test\" run the major test suites and \"make test-skipped\" for\n      some more (time-consuming) tests\n    - clean-up ring after timetrap timeout failures via common test hook\n    - new ?compare macro for custom comparison functions\n    - higher test coverage with more random-testing via the \"tester\"\n  * Documentation:\n    - user-dev-guide: add user tutorial on using scalaris\n    - user-dev-guide: add a section about the slide protocol\n    - user-dev-guide: extended description of scientific background\n    - add replica repair sequence diagrams\n    - better code descriptions\n  * Tools:\n    - gen_component: synchronous breakpoint set and delete for more\n                     deterministic usage\n    - trace_mpath: allow selective tracing via filter fun\n    - trace_mpath: fix several triggers becoming infected by trace_mpath\n                   resulting in infinite tracing\n    - trace_mpath: improve latex output of traces\n    - tester: copy dictionary to worker threads\n    - tester: add support for more types, e.g. neg_integer(), gb_rees\n    - tester: better type check error reporting\n    - tester: print tester last calls when aborting unit tests\n              (timeout or exception)\n    - tester: add support for constraints in type specs (\"when is_subtype(A,B)\")\n    - web debug interface: add cluster graph visualisation\n    - web debug interface: display vivaldi distance\n    - web debug interface: add IP addresses and ports to the ring charts and\n                           tables\n    - web debug interface: allow navigating to the web interfaces of shown nodes\n    - top: support for showing messages in message queue of an inspected PID\n    - top: support for showing larger dictionary values\n    - allow recursive reply_as envelopes\n    - experimental protocol scheduler to check protocols with random message\n      interleavings (see proto_sched module)\n  * Bugs:\n    - fix RM handling of (out-dated) nodes with the same ID as newly added nodes\n    - fix ganglia integration not working any more\n    - restore the ability to start nodes at a specific key via\n      \"scalarisctl -k <key> ...\"\n    - fix some memory leaks in the tx system\n    - fix statistics of comm_connection (not send in some cases, not\n      overflow-aware)\n    - use /bin/bash instead of /bin/sh which may not result in a bash session\n    - fix init.d scripts not checking for existing processes correctly\n    - fix dc_clustering\n    - fix numerous other bugs\n\n -- Nico Kruber <kruber@zib.de>  Fri, 16 Aug 2013 23:20:13 +0200\n\nscalaris (0.5.0-1) stable; urgency=low\n\n  * Packaging:\n    - new init.d script to start Scalaris\n    - added chef scripts to deploy Scalaris nodes\n    - improved Windows start scripts (support for R15B01 and R15B02, don't\n      close command prompt window immediately after shutdown if\n      double-clicked)\n    - more flexible scalarisctl (arbitrary parameter order, allow setting\n      cookie, ports and number of nodes in VM via parameters, allow using\n      screen for daemonised sessions, allow graceful leave via \"gstop\"\n      command, new \"status\" command)\n    - support for new linux distributions (Fedora 17, Ubuntu 12.04,\n      openSUSE 12.2)\n    - let scalarisctl checkinstallation also perform runtime tests for the\n      APIs\n  * API:\n    - allow Scalaris monitoring via JMX through the Java API\n    - added an executor-service to the Java-API (de.zib.scalaris.executor.*)\n    - added a node discovery daemon to the Java-API\n    - allow compressed communication between the Java-API and Erlang for\n      increased performance, especially if the two are on separate nodes\n    - added VM management support to the JSON- and Python-API\n    - added transaction log filtering to the Java-API, i.e. only sent the\n      needed parts of the tlog back to Erlang and re-combine the result\n    - fixed api_tx:req_list_commit_each/1 not running requests in parallel\n      -> do not assure any order of requests, even if on same key!\n  * Demonstrator \"Wiki on Scalaris\"\n    (supported by 4CaaSt http://www.4caast.eu/):\n    - allow different partitioned data models for better performance and\n      scalability\n    - allow logging of user requests\n    - added support for checking whether another article exists (approximate)\n      -> show link colours based on this check\n    - added check for bad page titles\n    - allow SERVERNAME and SERVERPATH in config for setups with load\n      balancers\n    - reduced memory footprint of Wiki data in Scalaris\n    - support for newer wiki xml dumps\n    - added support for using a MediaWiki-like SQLite-DB backend for e.g.\n      filtering\n    - improved overall performance\n    - several rendering fixes\n  * Business Logic:\n    - added (experimental) support for replica repair (disabled by default)\n      (thanks to Maik Lange)\n    - added monitoring of memory statistics (also available via web\n      interface)\n    - better error reporting in the failure detector\n    - reduced message overhead by UIDs and message/tuple tags\n    - reduced overall message size of transactions:\n      * do not include the (uncompressed) value in messages of the read phase\n        of write operations\n      * do not include the value in init_TP messages\n    - allow VM-flag \"first\" to be set via config file\n    - gather overall connection statistics in comm_stats (also available via\n      web interface)\n    - reduced erroneous failure messages on node shutdown\n    - integrated comm_layer into comm_server\n    - better scalability in pid_groups (find processes round-robin in\n      find_a/1)\n    - several changes to improve overall performance and/or CPU time at the\n      nodes\n  * Tests:\n    - support for more types in the runtime type-checker\n    - verify several API functions via runtime type-checker (also test\n      private functions if possible!)\n  * Tools:\n    - distributed protocol visualisation via trace_mpath (text-based or\n      latex-file for graphical presentation)\n    - better profiling via top for Erlang processes\n    - better debugging, e.g. via ASCII supervisor-tree rendering in verbose\n      mode\n  * Bugs:\n    - fixed memory leaks in read and write operations\n    - fixed memory leaks in tx_tm_rtm\n    - prevent potential endless loops in tx_tm_rtm\n    - fixed inform RTMs sometimes informing the wrong RTMs\n    - fixed numerous other bugs\n\n -- Nico Kruber <kruber@zib.de>  Thu, 11 Oct 2012 12:30:00 +0200\n\nscalaris (0.4.1-1) stable; urgency=low\n\n  * Packaging:\n    - new official ConPaaS packages (http://www.conpaas.eu/)\n    - install rubygem dependencies in Debian postinstall scripts for Ruby API\n    - improved Windows start scripts (if set, uses the ERLANG_HOME\n      environment variable to find Erlang, otherwise searches for Erlang in\n      common paths)\n  * Bugs:\n    - better tx cleanup (should fix rare occurance of duplicate client\n      inform)\n    - forward additional parameters of the start scripts to new syntax of\n      scalarisctl\n\n -- Nico Kruber <kruber@zib.de>  Thu, 22 Mar 2012 10:51:00 +0100\n\nscalaris (0.4.0-1) stable; urgency=low\n\n  * API:\n    - new functions for incremental data change:\n      test_and_set: check for a provided old value before setting a new one\n      add_on_nr: increment a numeric value\n      add_del_on_list: append or delete entries from a list value\n    - added VM API to manage Scalaris nodes inside an Erlang virtual machine\n    - added monitoring API to retrieve some live metrics\n    - added a connection pool convenience class (Java, Python)\n  * Demonstrator \"Wiki on Scalaris\"\n    (supported by 4CaaSt http://www.4caast.eu/):\n    - improved performance of page edits\n    - improved performance of Wikipedia dump loading\n    - several rendering fixes\n  * Business Logic:\n    - improved handling of large values by reducing overhead of transaction\n      log handling (empty TLog after commit), no copy of value in TLog\n      returned to user after read requests)\n    - eliminated timeouts in data hand-over protocol (relies on fd now)\n    - added a DB subscribe mechanism, e.g. to become informed when locks\n      are freed\n    - fixed a strong consistency issue in the tx protocol\n    - gather some run-time statistics and expose them via the APIs and the\n      web debug interface\n  * Infrastructure:\n    - support for Erlang 15B\n    - fd now also uses feedback from TCP layer\n    - made message sending more flexible (gets an option list)\n    - added and corrected several Erlang type specifications\n    - added scripts to create Scalaris images for OpenNebula\n    - added tools for using Scalaris as the Database as a Service\n      component in ConPaaS (http://www.conpaas.eu/) which is part of the\n      EU project Contrail (http://contrail-project.eu/)\n    - added a separate communication channel for priority messages, e.g. fd\n      (reduces falsely reported node crashes under heavy load)\n  * Tests:\n    - added runtime type-checker for random testing extended unittests\n  * Documentation:\n    - updated documentation to extended APIs\n  * Bugs:\n    - fixed numerous bugs\n\n -- Nico Kruber <kruber@zib.de>  Wed, 25 Jan 2012 00:48:00 +0100\n\nscalaris (0.3.0-1) stable; urgency=low\n\n  * API\n    - new API with interoperable bindings to Java, Python, Ruby, and JSON\n    - support for several data types, including strings, integers, JSON\n      objects, binary objects.\n    - new transaction interface with support for bundled requests for better\n      latency.\n    - separate APIs to access the raw DHT, a DHT with replication, and the\n      transactional DHT\n  * Demonstrator\n    - added Wikipedia-hosting using Scalaris as demonstrator application\n  * Business Logic\n    - fault-tolerant startup: start Scalaris when a quorum of the known_hosts\n      becomes available (option -q in bin/scalarisctl)\n    - perform data hand-over when nodes join/gracefully leave (also works when\n      transactions are executed concurrently)\n    - added passive load balancing (when a node joins a ring, it samples\n      several other nodes and joins at the node that balances the number of\n      stored items the most)\n    - completely rewritten transaction layer (more modular, more extendible,\n      less latency)\n    - modularized / reimplemented Paxos algorithm, so the algorithm can also\n      be used outside transactions (e.g. used for quorum-startup)\n    - switched almost all components to our component framework\n     'gen_component'\n    - added gossiping for estimating e.g. the number of nodes or the average\n      load in a ring\n    - more reliable unreliable look-up\n    - better ring start-up on slow networks\n  * Infrastructure\n    - Vivaldi and topology inference\n    - support for Erlang 13B01 and newer\n    - faster TCP/IP communication between Scalaris nodes\n    - completely rewritten failure detector framework for more accurate\n      detection of node failures\n    - added numerous Erlang type specifications\n    - extended unittests\n  * Tests\n    - added own random testing framework that reads type specifications and\n      scans the source code for constants to generate proper random test-data\n    - extended gen_component with breakpoint-support for debugging and testing\n      (perform deterministic pseudo-random message interleaving tests)\n    - added numerous unittests\n    - added language-binding interoperability tests\n  * Documentation\n    - extended, but - as always - by far not enough...\n  * Bugs\n    - fixed countless bugs\n\n -- Nico Kruber <kruber@zib.de>  Fri, 15 Jul 2011 17:01:00 +0200\n"
  },
  {
    "path": "contrib/packages/main/debian.compat",
    "content": "7\n"
  },
  {
    "path": "contrib/packages/main/debian.control",
    "content": "Source: scalaris\nSection: database\nPriority: optional\nMaintainer: Nico Kruber <kruber@zib.de>\nBuild-Depends: debhelper (>= 4.1.16), sudo, screen, erlang-dev (>= 14.b.4), erlang-parsetools (>= 14.b.4), erlang-tools (>= 14.b.4), erlang-edoc (>= 14.b.4), erlang-crypto (>= 14.b.4), pkg-config\nHomepage: http://scalaris.zib.de\nStandards-Version: 3.9.6\n\nPackage: scalaris\nArchitecture: all\nDepends: ${misc:Depends}, adduser, sudo, screen, debianutils, hostname, erlang-base (>= 14.b.4), erlang-crypto (>= 14.b.4), erlang-inets (>= 14.b.4), erlang-ssl (>= 14.b.4), erlang-xmerl (>= 14.b.4)\nSuggests: scalaris-java, scalaris-doc\nDescription: Scalable Distributed key-value store\n Scalaris is a scalable, transactional, distributed key-value store. It\n can be used for building scalable services. Scalaris uses a structured\n overlay with a non-blocking Paxos commit protocol for transaction\n processing with strong consistency over replicas. Scalaris is\n implemented in Erlang.\n\nPackage: scalaris-doc\nSection: doc\nArchitecture: all\nDepends: ${misc:Depends}\nRecommends: scalaris (= ${binary:Version})\nDescription: Documentation for scalaris\n Scalaris is a scalable, transactional, distributed key-value store. It\n can be used for building scalable services. Scalaris uses a structured\n overlay with a non-blocking Paxos commit protocol for transaction\n processing with strong consistency over replicas. Scalaris is\n implemented in Erlang.\n"
  },
  {
    "path": "contrib/packages/main/debian.rules",
    "content": "#!/usr/bin/make -f\n# Sample debian/rules that uses debhelper.\n# GNU copyright 1997 to 1999 by Joey Hess.\n\n# Uncomment this to turn on verbose mode.\n#export DH_VERBOSE=1\n\nbuild-arch: build\nbuild-indep: build\nbuild: build-stamp\nbuild-stamp:\n\tdh_testdir\n\n\t# Add here commands to compile the package.\n\t./configure --prefix=/usr \\\n\t--sysconfdir=/etc \\\n\t--localstatedir=/var \\\n\t--docdir=/usr/share/doc/scalaris \\\n\t--disable-cpp\n#    --prefix=%{_prefix} \\\n#    --exec-prefix=%{_exec_prefix} \\\n#    --bindir=%{_bindir} \\\n#    --sbindir=%{_sbindir} \\\n#    --sysconfdir=%{_sysconfdir} \\\n#    --datadir=%{_datadir} \\\n#    --includedir=%{_includedir} \\\n#    --libdir=%{_libdir} \\\n#    --libexecdir=%{_libexecdir} \\\n#    --localstatedir=%{_localstatedir} \\\n#    --sharedstatedir=%{_sharedstatedir} \\\n#    --mandir=%{_mandir} \\\n#    --infodir=%{_infodir} \\\n#    --docdir=%{_docdir}/scalaris\n\tmake all\n\tmake doc\n\n\ttouch build-stamp\n\nclean:\n\tdh_testdir\n\tdh_testroot\n\trm -f build-stamp\n\n\t# Add here commands to clean up after the build process.\n\trm -rf build_dir\n\n\tdh_clean\n\nBUILD_DIR_MAIN:=$(CURDIR)/debian/scalaris\nBUILD_DIR_DOC:=$(CURDIR)/debian/scalaris-doc\ninstall: build\n\tdh_testdir\n\tdh_testroot\n\tdh_prep\n\tdh_installdirs\n\n\t# Add here commands to install the package into debian/gentoo.\n\tmake install DESTDIR=$(BUILD_DIR_MAIN)\n\tmake install-doc DESTDIR=$(BUILD_DIR_MAIN)\n\t# move some doc to the doc sub-package:\n\tmkdir -p $(BUILD_DIR_DOC)/usr/share/doc/scalaris\n\tmv $(BUILD_DIR_MAIN)/usr/share/doc/scalaris/erlang \\\n\t   $(BUILD_DIR_DOC)/usr/share/doc/scalaris/\n\tmv $(BUILD_DIR_MAIN)/usr/share/doc/scalaris/user-dev-guide.pdf \\\n\t   $(BUILD_DIR_DOC)/usr/share/doc/scalaris/\n\tmkdir -p $(BUILD_DIR_DOC)/usr/lib/scalaris/docroot\n\tmv $(BUILD_DIR_MAIN)/usr/lib/scalaris/docroot/doc \\\n\t   $(BUILD_DIR_DOC)/usr/lib/scalaris/docroot/\n\t\n\t# handle config files:\n\tcp $(BUILD_DIR_MAIN)/etc/scalaris/scalaris.cfg \\\n\t   $(BUILD_DIR_MAIN)/etc/scalaris/scalaris.cfg.example\n\n# Build architecture-independent files here.\nbinary-indep: build install\n\tdh_testdir\n\tdh_testroot\n#\tdh_installdebconf\n\tdh_installdocs\n\tdh_installexamples\n\tdh_installmenu\n#\tdh_installlogrotate\n#\tdh_installemacsen\n#\tdh_installpam\n#\tdh_installmime\n#\tdh_installinit\n\tdh_installcron\n\tdh_installman\n\tdh_installinfo\n#\tdh_undocumented\n\tdh_installchangelogs\n\tdh_link\n\tdh_strip\n\tdh_compress\n\tdh_fixperms\n#\tdh_makeshlibs\n\tdh_installdeb\n#\tdh_perl\n\tdh_shlibdeps\n\tdh_gencontrol\n\tdh_md5sums\n\tdh_builddeb\n\n# Build architecture-dependent files here.\nbinary-arch: build install\n\t# We have nothing to do by default.\n\nbinary: binary-indep binary-arch\n.PHONY: build clean binary-indep binary-arch binary install\n"
  },
  {
    "path": "contrib/packages/main/debian.scalaris.postinst",
    "content": "#!/bin/bash\n\nset -e\n\nscalaris_user=scalaris\nscalaris_group=scalaris\nscalaris_home=/var/lib/scalaris\n\n# use automatic handling for now...\n# # Copy config file if not existing yet\n# \n#   if [ ! -f /etc/scalaris/scalaris.cfg ]; then\n#     cp /etc/scalaris/scalaris.cfg.example /etc/scalaris/scalaris.cfg\n#   fi\n\ncreate_scalaris_user() {\n  getent group $scalaris_group >/dev/null || addgroup --quiet --system $scalaris_group\n  getent passwd $scalaris_user >/dev/null || mkdir -p $scalaris_home && adduser --quiet --system --ingroup $scalaris_group --home $scalaris_home --no-create-home --shell /sbin/nologin $scalaris_user && chown $scalaris_user:$scalaris_group $scalaris_home\n  chown -R $scalaris_user:$scalaris_group /var/log/scalaris\n}\n\nif grep -e '^cookie=\\w\\+' /etc/scalaris/scalarisctl.conf > /dev/null 2>&1; then\n  echo $RANDOM\"-\"$RANDOM\"-\"$RANDOM\"-\"$RANDOM >> /etc/scalaris/scalarisctl.conf\nfi\n\ncreate_scalaris_user\n\necho \"Installing Scalaris RC-links...\"\nupdate-rc.d scalaris defaults > /dev/null\n\necho \"Re-starting Scalaris service if started...\"\nif which invoke-rc.d >/dev/null 2>&1; then\n  invoke-rc.d scalaris try-restart\nelse\n  /etc/init.d/scalaris try-restart\nfi\n\n#DEBHELPER#\n"
  },
  {
    "path": "contrib/packages/main/debian.scalaris.postrm",
    "content": "#!/bin/bash\nset -e\n\nif [ \"$1\" = \"purge\" ]; then\n  echo \"Removing Scalaris RC-links\"\n  update-rc.d -f scalaris remove > /dev/null\nfi\n\n#DEBHELPER#\n"
  },
  {
    "path": "contrib/packages/main/debian.scalaris.prerm",
    "content": "#!/bin/bash\nset -e\n\n# stop before remove\nif [ \"$1\" = \"remove\" -o \"$1\" = \"deconfigure\" ]; then\n  echo \"Stopping Scalaris service...\"\n  if which invoke-rc.d >/dev/null 2>&1; then\n    invoke-rc.d scalaris stop\n  else\n    /etc/init.d/scalaris stop\n  fi\nfi\n\n#DEBHELPER#\n"
  },
  {
    "path": "contrib/packages/main/debian.source.lintian-overrides",
    "content": "scalaris source: source-is-missing docroot/ext-base.js\nscalaris source: source-is-missing docroot/ext-all.js\nscalaris source: source-is-missing docroot/tablesort/jquery.tablesorter.min.js\nscalaris source: source-is-missing docroot/flot/jquery.min.js\nscalaris source: source-is-missing docroot/flot/jquery.flot.threshold.min.js\nscalaris source: source-is-missing docroot/flot/jquery.flot.symbol.min.js\nscalaris source: source-is-missing docroot/flot/jquery.flot.stack.min.js\nscalaris source: source-is-missing docroot/flot/jquery.flot.selection.min.js\nscalaris source: source-is-missing docroot/flot/jquery.flot.resize.min.js\nscalaris source: source-is-missing docroot/flot/jquery.flot.pie.min.js\nscalaris source: source-is-missing docroot/flot/jquery.flot.navigate.min.js\nscalaris source: source-is-missing docroot/flot/jquery.flot.min.js\nscalaris source: source-is-missing docroot/flot/jquery.flot.image.min.js\nscalaris source: source-is-missing docroot/flot/jquery.flot.fillbetween.min.js\nscalaris source: source-is-missing docroot/flot/jquery.flot.errorbars.min.js\nscalaris source: source-is-missing docroot/flot/jquery.flot.crosshair.min.js\nscalaris source: source-is-missing docroot/flot/jquery.colorhelpers.min.js\nscalaris source: source-is-missing docroot/flot/excanvas.min.js\nscalaris source: source-is-missing contrib/wikipedia/contrib/libsqlite4java-linux-i386.so\nscalaris source: source-is-missing contrib/wikipedia/contrib/libsqlite4java-linux-amd64.so\n"
  },
  {
    "path": "contrib/packages/main/install",
    "content": "post_install() {\n\tgetent group scalaris &>/dev/null || groupadd -r scalaris\n\tgetent passwd scalaris &>/dev/null || mkdir -p /var/lib/scalaris && useradd -r -g scalaris -d /var/lib/scalaris -M -s /bin/false -c \"user for scalaris\" scalaris && chown scalaris:scalaris /var/lib/scalaris\n\tchown -R scalaris:scalaris /var/log/scalaris\n\ttrue\n}\n\npost_remove() {\n\tgetent passwd scalaris &>/dev/null && userdel scalaris\n\tgetent group scalaris &>/dev/null && groupdel scalaris\n\ttrue\n}\n"
  },
  {
    "path": "contrib/packages/main/scalaris-rpmlintrc",
    "content": "addFilter(\"non-standard-uid.*scalaris\")\naddFilter(\"non-standard-gid.*scalaris\")\naddFilter(\"file-contains-current-date.*/usr/share/doc/packages/scalaris/erlang/.*\\.html\")\naddFilter(\"file-contains-date-and-time.*/usr/share/doc/packages/scalaris/erlang/.*\\.html\")\naddFilter(\"zero-length.*/etc/scalaris/scalaris.local.cfg\")\n"
  },
  {
    "path": "contrib/packages/main/scalaris.changes",
    "content": "-------------------------------------------------------------------\nMon Feb 29 16:00:00 UTC 2016 - kruber@zib.de\n\n- Scalaris 0.9.0 (codename \"Vriesea scalaris\")\n  (partly supported by the EU project IES Cities http://iescities.eu/)\n  * Packaging:\n    - add a libscalaris-dev[el] package with the C++ API (rpm, deb)\n    - Maven: let Maven also create a source jar file for the Java-API\n    - Maven: allow proper snapshot versions for the maven repository at\n             https://scalaris-team.github.io/scalaris/maven\n  * API:\n    - CPP-API: provide a new C++ client library\n    - DataNucleus: provide a new storage back-end for Scalaris\n    - add a Scalaris adapter for YCSB\n    - new functions in all APIs for generic routing table operations\n  * Business Logic:\n    - rrepair: considerably improve the set reconciliation algorithms' performance\n               and run more processes in parallel\n    - rrepair: further reduce the reconciliation costs\n    - rrepair: re-use left-over failure probability from phase 1 for phase 2\n    - rrepair: increase accuracy in several formulaes of the set reconciliation\n               algorithms\n    - rrepair: support creating multiple Merkle trees for fewer message rounds\n               ('rr_merkle_num_trees' configuration variable)\n    - rrepair: remove support for aligning bit sizes (no improvements there)\n    - transactions: speed up write ops when the transaction log already contains\n                    a read op on the same key\n    - RM: optimise the traffic of state exchanges during ring maintenance\n    - RT: improve routing parallelism by handling more messages in the routing\n          process itself instead of inside the dht_node process\n    - RT: improve performance along the routing path\n    - support for dynamic replication factors given by the 'replication_factor'\n      configuration variable\n    - pid_groups: shuffle pids in find_a/1 for more better distributed (parallel)\n                  requests\n  * Infrastructure:\n    - pid_groups: group names are now atoms instead of strings\n                  (allows faster access)\n    - scalarisctl: do not set any default Erlang scheduler flags (allow changes\n                   through the ERL_SCHED_FLAGS environment variable)\n    - extend the example scripts for running Scalaris on a SLURM cluster\n    - add SLURM scripts for running Basho Bench\n    - use configure to select the routing table type (--with-rt=rt_chord)\n    - change hanoidb detection to \"configure --enable-hanoidb=<dir>\"\n    - change erlang_js detection to \"configure --enable-erlang-js=<dir>\"\n    - dht_node_monitor: switch messages on/off via configuration variable\n                        'dht_node_monitor'\n    - msg_delay: remove extraneous error messages for trigger messages of killed\n                 processes\n    - prbr: allow write_filter to return value to qwrite calling process\n    - update yaws to version 2.0.2\n    - support for Erlang R14B04 up to 18.2.3 and current otp master\n  * Tests:\n    - allow running single test groups via \"make <test suite>:<group name>_GROUP\"\n  * Tools:\n    - gen_component: allow processes to provide their own start functions\n    - trace_mpath: write whole message of log_info to the tex output\n  * Bugs:\n    - fix cleanup of some special cases with transactions\n    - fix several math issues and increase the math functions' performance\n    - fix some errors which occurred in large scale deployments\n    - fix several warnings reported by Coverity Scan\n    - fix numerous more bugs\n\n-------------------------------------------------------------------\nFri Jul 24 10:00:00 UTC 2015 - kruber@zib.de\n\n- Scalaris 0.8.2\n  (partly supported by the EU project IES Cities http://iescities.eu/)\n  * Business Logic:\n    - rrepair: minor performance improvements\n  * Bugs:\n    - rrepair: fix a rare crash due to hash collisions\n    - rrepair: fix calculation of the next MaxItemCount in the merkle tree\n               repair protocol (was one level behind)\n    - tests: fix rare crashes in some unit tests\n    - fix a few more minor bugs\n\n-------------------------------------------------------------------\nSun Jul 12 10:00:00 UTC 2015 - kruber@zib.de\n\n- Scalaris 0.8.1\n  (partly supported by the EU project IES Cities http://iescities.eu/)\n  * Business Logic:\n    - rrepair: allow specifying the minimum number of hash bits to use\n               to create non-approximate algorithms\n  * Bugs:\n    - rrepair: fix a badarith if rr_recon_version_bits is set to 'variable'\n    - rrepair: fix a bug in the accuracy of the evaluation\n    - fix two crashes in gossip\n\n-------------------------------------------------------------------\nFri Jul 10 10:00:00 UTC 2015 - kruber@zib.de\n\n- Scalaris 0.8.0 (codename \"Picoides scalaris\")\n  (partly supported by the EU project IES Cities http://iescities.eu/\n  and the EIT ICT Labs project MCData)\n  * Packaging:\n    - also install docs on Arch Linux\n    - fix lintian errors and warnings for Debian-based packages\n    - fix permissions on the log directory (owned by the 'scalaris' user)\n      and the config files (not owned by the 'scalaris' user anymore!)\n    - adapt packages for newer distributions\n  * API:\n    - Java-API: integrate new OtpErlang library (1.5.12 from Erlang 17.4)\n    - Python-API: do not include a shebang with a pre-defined path to python\n                  anymore\n    - Python-API: fix unclosed sockets in various tests\n    - Python-API: increase python{2,3} test speed\n    - Ruby-API: try to fallback to the normal json module if the gem is missing\n    - Ruby-API: considerably increase the ruby test speed\n    - remove the Publish/Subscribe API\n  * Business Logic:\n    - rrepair: faster and more accurate bloom filter repair protocol\n    - rrepair: faster, more accurate and less bandwidth-consuming merkle tree\n               repair protocol\n    - rrepair: reduce memory footprint of merkle_tree\n    - rrepair: more accurate and less bandwidth-consuming trivial repair\n               protocol\n    - rrepair: add a new simple hash ('shash') algorithm\n    - rrepair: switch to (the more efficient) push-only resolve phase for\n               synchronisation\n    - rrepair: add replica repair algorithm evaluation scripts\n    - RM: faster cyclon cache integration, especially during node jumps\n    - comm_layer: envelops can be nested\n    - comm_layer: allow send options for comm:send_local/3 and msg_delay\n                  (currently only ?quiet supported)\n  * Infrastructure:\n    - FD: simplify the API, support envelopes and remove own cookie handling\n    - FD: extend the failure detector with generic notifications (not just\n          crashes) to subscribed nodes\n    - DB: add an (experimental) mnesia back-end\n    - DB: add an (experimental) hanoidb back-end\n    - DB: make the back-end configurable via the scalaris.cfg option\n          'db_backend'\n    - improve the speed of several utility functions\n    - scalarisctl: add '-l' parameter for specifying the logdir\n    - scalarisctl: do not use the '-s' parameter anymore, instead specify the\n                   start type with the new '-t' parameter\n    - scalarisctl: allow starting several dht_nodes per vm with given keys via\n                   the new parameter '-j'\n    - update yaws to version 1.99\n    - make \"$HOME/.scalaris/log\" the default log dir\n    - support for Erlang R14B04 up to 18.0.2 and current otp master\n    - add example scripts for running Scalaris on a SLURM cluster\n    - move sources to github: https://github.com/scalaris-team/scalaris\n    - new project homepage: http://scalaris.zib.de\n  * Tests:\n    - reduce log spam in proto_sched tests and only print proto_sched results\n      when the tests fail\n    - extended and improved various test suites\n  * Tools:\n    - proto_sched: support callbacks\n    - proto_sched: add support for user msg shepherds\n    - proto_sched: better error handling when sending messages to dead\n                   (local) processes\n    - tester: add value creator and type checker for orddict:orddict()\n    - tester: full support for Erlang 18.0\n    - let 'make dialyzer' run over test as well and ignore some false-positives\n      for Erlang >= 18.0 using the '-dialyzer()' attribute\n  * Bugs:\n    - slide: fix some race conditions with 'jump' operations\n    - JSON-API: fix test_and_set not working correctly inside req_list\n    - fix numerous more bugs\n\n-------------------------------------------------------------------\nMon Oct 23 16:15:00 UTC 2014 - kruber@zib.de\n\n- Scalaris 0.7.2\n  (partly supported by the EU project IES Cities http://iescities.eu/\n  and the EIT ICT Labs project MCData)\n  * Packaging:\n    - fix ArchLinux packages with newest Java versions\n  * Demonstrator \"Wiki on Scalaris\":\n    - fix the separate count key optimisation not using Scalaris' increment\n      operation\n  * Business Logic:\n    - rrepair: let the trivial algorithm assume the worst case in order to always\n               meet the configured \"recon probability of one error\" (p1e)\n    - rrepair: fix the trivial algorithm having an effectively doubled p1e\n    - rrepair: fix the bloom algorithm having an effectively tripled p1e\n    - rrepair: allow disabling byte-alignment\n  * Bugs:\n    - fix a few minor bugs\n\n-------------------------------------------------------------------\nMon Sep 29 15:00:00 UTC 2014 - kruber@zib.de\n\n- Scalaris 0.7.1\n  (partly supported by the EU project IES Cities http://iescities.eu/\n  and the EIT ICT Labs project MCData)\n  * Packaging:\n    - add support for new distribution versions\n    - support systemd and SELinux\n    - include daemon for monitoring Scalaris through JMX\n  * API:\n    - Java-API: integrate new OtpErlang library (1.5.10 from Erlang 17.3)\n  * Demonstrator \"Wiki on Scalaris\":\n    - fix storing template back-links for magic words\n    - fix ARTICLE_COUNT partitioning with hashes\n  * Business Logic:\n    - more robust (still experimental) support for active load balancing with\n      Karger and Ruhl's algorithm including more flexible \"load\" definitions\n    - rm_tman: less overhead by only sending unknown nodes to neighbours\n    - slide: better support for slide aborts\n    - node join: less overhead during joins, especially in setups with huge\n                 lists of known nodes\n    - cyclon: move to new gossip framework\n    - vivaldi: move to new gossip framework\n    - monitor: move performance monitor to the basic_services group\n               (once per VM)\n  * Infrastructure:\n    - several smaller performance optimisations\n    - support for Erlang R13B01 up to 17.3 and current otp master\n  * Tests:\n    - some new unit tests and higher test coverage\n  * Documentation:\n    - add rrepair sequence diagrams\n  * Bugs:\n    - fix numerous bugs\n\n-------------------------------------------------------------------\nMon Apr 28 10:00:00 UTC 2014 - kruber@zib.de\n\n- Scalaris 0.7.0 (codename \"Stauroderus scalaris\")\n  * API:\n    - Java-API: integrate new OtpErlang library (1.5.9 from Erlang 17.0)\n  * Demonstrator \"Wiki on Scalaris\"\n    (supported by 4CaaSt http://www.4caast.eu/\n     and Contrail http://contrail-project.eu):\n    - add support for reading 7z dumps\n  * Business Logic\n    (partly supported by the EU project IES Cities http://iescities.eu/\n     and the EIT ICT Labs project MCData):\n    - add an experimental Map-Reduce framework on top of Scalaris\n    - add experimental support for active load balancing with Karger and\n      Ruhl's algorithm including more flexible \"load\" definitions\n    - rrepair: completely new merkle tree sync protocol with an order of\n               magnitude lower traffic costs (use dynamic signature sizes based\n               on a \"recon probability of one error\" (p1e) in the leaf nodes)\n    - rrepair: add a trivial probabilistic reconciliation protocol using the\n               \"recon probability of one error\" (p1e)\n    - rrepair: replace bloom_fpr parameter with a generic \"recon probability of\n               one error\" (p1e)\n    - rrepair: add a trivial reconciliation phase for the differences identified\n               by bloom\n    - rrepair: more efficient resolve also including a list of keys to request\n    - gossip: completely new (more flexible) gossip framework\n    - slide: add support for 'jump' operations\n    - slide: do not create a timer for each received message (use a periodic\n             cleanup)\n    - comm_layer: tune TCP connection parameters to increase throughput and\n                  latency (no delay_send)\n    - comm_layer: close idle TCP connections, support for no_keep_alive\n                  connections (used by cyclon, for example)\n    - comm_layer: try to bundle more messages\n    - node join: less overhead during joins, especially in small rings\n    - rm_tman: less overhead by not adding nodes in the cache to the dn_cache\n    - rt_chord: skip trying to contact nodes in the own range\n    - rt_chord: stop stabilize when the own node is reached\n  * Infrastructure:\n    - intervals: more compact (and transfer-friendly) representation\n    - trigger: replace modules with msg_delay:send_trigger/2\n    - config: replace implementation with a public ets table\n    - tune the garbage collection of some core processes and periodically\n      garbage collect all processes\n    - reduce the number of generated atoms and the number of ets tables used\n    - several smaller performance optimisations\n    - support for Erlang R13B01 up to 17.0 and current otp master\n  * Tests:\n    - improve test coverage (manual tests and random tests)\n    - change test definitions and allow groups of tests, e.g.\n      test (default), test-skipped,\n      all_TESTS, all_with_cover_TESTS, performance_TESTS, proto_sched_TESTS,\n      type_check_TESTS\n  * Documentation:\n    - add rrepair documentation\n  * Tools:\n    - proto_sched: new API, better determinism, more strict self-control\n    - proto_sched: support for short-lived processes\n    - proto_sched: more information in get_infos\n    - trace_mpath: more flexible LaTeX exports with more detailed message info\n    - trace_mpath: support meaningful PID names when tracing remote messages\n    - tester: full support for Erlang 17.0\n    - make: re-compile after Emakefile changes\n  * Bugs:\n    - rrepair: fix unnecessary feedback for KVV items of the same version\n    - bulkowner: some fixes for (gracefully) leaving nodes\n    - fix numerous more (less severe) bugs\n\n-------------------------------------------------------------------\nFri Oct 11 13:00:00 UTC 2013 - kruber@zib.de\n\n- Scalaris 0.6.1\n  * Packaging:\n    - add ArchLinux ruby API packages\n  * API:\n    - Java-API: add Maven build support\n    - Java-API: add CircularByteArrayOutputStream#clear()\n    - Java-API: fix ConnectionPool#getConnection(timeout) throwing\n                IllegalMonitorStateException if a single ConnectionPool is used\n                by multiple threads and no more connections are available\n    - Java-API: add a \"-monitor\" command line parameter\n    - api_monitor: return the latency and stddev values of the micro-benchmark\n                   executed by monitor_perf for node and service performance\n  * Demonstrator \"Wiki on Scalaris\" (supported by 4CaaSt http://www.4caast.eu/\n                                     and Contrail http://contrail-project.eu):\n    - separate list counters from their list partitions for a better data layout\n    - fix high memory use of the Scalaris import if the import is slow\n    - use tomcat 7.0.42\n  * Business Logic:\n    - rrepair: reduce overhead of ART reconciliation\n    - rrepair: allow resolving of multiple merkle node leaves with a single\n               resolve request\n    - rrepair: don't create resolve requests for empty intervals\n    - rrepair: reduce overhead of update_key_entry requests (use a single\n               request with all the data instead of one request for each item)\n    - rrepair: allow arbitrary intervals in interval_upd and interval_upd_send\n               resolve requests again\n    - rrepair: when hashing merkle_tree/ART children, also include the\n               represented interval (fixes indistinguishable empty leaf nodes\n               in ART)\n    - tx_tm_rtm: re-enable takeover by rtms on tm crash\n    - rt_chord: only re-build the RT if the pred or succ processes change or the\n                own new node ID is not between the new pred and succ any more\n  * Infrastructure:\n    - DB: improve performance of fold[lr] implementations by a factor of 2\n    - FD: get rid of annoying, wrongly raised log warnings\n    - RM: provide a more generic Reason for RM subscriptions\n    - RM: remove trigger infections\n    - RT: don't trigger an update when a slide finishes\n    - RT, FRT: reduce the number of messages sent on lookups\n    - log: don't exit with a badmatch if our error_logger is not the only one\n  * Tests:\n    - add protocol scheduler tests for slide, join and leave\n  * Documentation:\n    - user-dev-guide: add section on scalarisctl checkinstallation\n    - user-dev-guide: clarify the section about how to set up Scalaris\n  * Tools:\n    - proto scheduler: continue in case of send errors\n    - top: improve process messages output\n  * Bugs:\n    - node join: fix not being able to join a system with passive load balancer\n                 if the number of items in the DB is too high\n    - tx_tm_rtm: fix wrong asserts\n    - fix node resposibility check not always including message forward and\n      db_range intervals\n    - tx_tp: add missing snapshot number in tp_do_commit_abort message\n    - bulkowner: respect the nodes' DB ranges and forward requests for\n                 non-responsible ranges\n    - dn_cache: fix reporting wrong PIDs back as zombies after node reboots\n    - fix some more (less severe) bugs\n\n-------------------------------------------------------------------\nFri Aug 16 21:20:13 UTC 2013 - kruber@zib.de\n\n- Scalaris 0.6.0 (codename \"Conus scalaris\")\n  * Packaging:\n    - add ArchLinux packages\n    - add support for new distribution versions\n  * API:\n    - no more timeouts in client APIs\n    - Java-API: re-worked the request and result list handling\n      -> move result processing to the operation classes\n    - Java-API: better support for custom operations\n    - Java-API: support the new partial reads:\n                ReadRandomFromListOp and ReadSublistOp\n    - Java-API: compile with \"vars\" debug info\n    - Java-API: integrate new OtpErlang library (1.5.8 from R16B) with fixed\n                support for compressed binaries\n    - Java-API: add back-ports from the Wiki on Scalaris demonstrator:\n      * list-change operations: ScalarisChangeListOp and\n                                ScalarisListAppendRemoveOp\n      * MultiMap classes are now in de.zib.tools\n      * CircularByteArrayOutputStream\n    - Java-API: fix hostname issues with Erlang and Java\n    - Java-API: slightly changed the delete API\n    - JSON-API: add API for auto-scale requests\n    - Python-API: add API for auto-scale requests\n    - Python-API: use default socket timeout\n    - Ruby-API: use default socket timeout\n    - all APIs: support lists of composite types\n  * Demonstrator \"Wiki on Scalaris\" (supported by 4CaaSt http://www.4caast.eu/\n                                     and Contrail http://contrail-project.eu):\n    - allow monitoring via JMX in the FourCaastMonitoringPlugin\n    - support for getting random articles via the new partial read op\n    - new optimisation scheme \"Buckets with Write Cache\" - uses a single big\n      list to read from and the rest of the buckets to write to\n    - improve import and dump-processing (faster, more memory-efficient)\n    - add on-the-fly conversion to the different optimisation schemes during\n      import (only one prepared DB dump needed now)\n    - several UI enhancements and rendering fixes\n    - update bliki lib (includes code ported to upstream)\n    - add auto-import ability\n    - use tomcat 7.0.33\n  * Business Logic:\n    - replace common message tags with integers to reduce bandwidth\n    - more flexible read operations (easier to extend)\n    - add support for the following partial reads: random_from_list and sublist\n    - save bandwidth by not returning the full value for write operations\n      (only the version is required)\n    - new DB back-end implementation with a smaller and cleaner interface\n    - faster DB get_chunk processing\n    - tx: allow overwriting old/outdated DB entries\n    - tx: allow overwriting old/outdated write-locked entries\n    - tx: allow setting write lock on old/outdated read-locked entries\n    - tx: always reply when the majority replied during read\n    - tx: make sure that if not_found is reported to the user (while reading),\n          a write cannot go through if it is not also based on not_found\n    - tx: committing a test_and_set op on a non-existing entry now fails as well\n          (the op itself already returned the failure)\n    - tx: add a 2s delay to wait for slow learner_decide answers before cleaning\n          up (results in a faster state cleanup after the fourth response)\n    - tx: small performance improvements in several modules\n    - rm: only add alive, non-leaving nodes\n    - rm: if a predecessor crashes, start repairing the range (rrepair)\n    - rrepair: stabilised rrepair (not considered experimental any more)\n    - rrepair: also update entries with existing but outdated WriteLocks\n    - rrepair: several performance improvements\n               (bloom, merkle_tree, art and rrepair processes in general)\n    - rrepair: re-design of rr_recon\n    - rrepair: don't offload heavy work onto the dht_node (increases\n               responsiveness of the dht_node process during replica repair)\n    - rrepair: improve db_generator tool and random_bias binomial distribution\n               used for tests\n    - rrepair: support differently configured nodes (use the same reconciliation\n               structure parameters)\n    - rrepair: de-activate self-repair (a node with multiple copies of the same\n               items does not need a reconciliation structure to repair some of\n               them)\n    - rrepair: activate rrepair periodically every 10 minutes with a probability\n               of 33%\n    - slide v2.0: fewer message to initiate a slide\n    - slide v2.0: generic (asynchronous) call-backs for different ring\n                  maintenance algorithms\n    - slide v2.0: re-work handling of planned next operations, e.g. used by\n                  incremental slides\n    - slide v2.0: don't directly work on the DB any more (there may be more\n                  data needed to slide) - let dht_node_state decide\n    - slide v2.0: activate incremental join and leave operations\n    - slide v2.0: actively report graceful node shutdown to the local FD of the\n                  leaving node to inform subscribers\n    - slide v2.0: code clean-up\n    - slide v2.0: some fixes for incremental slides\n    - slide v2.0: more robust in general\n    - more smooth node joins by also reporting when a join is not possible due\n      to a running slide at the existing node\n    - passive load balancing: random selection of (equally qualified) nodes\n    - add new routing algorithms FRT-Chord (flexible routing tables) and\n      GFRT-Chord (supports proximity routing and data centers) as alternatives\n      to Chord (see rt_frtchord and rt_gfrtchord modules)\n    - add auto-scale framework, e.g. for cloud environments (supported by\n      Contrail http://contrail-project.eu/) which is able to scale the\n      deployment to maintain a given target latency of executed transactions\n    - cache config reads in the process dictionary for better performance\n    - cyclon: if the cache is empty, try one of the nodes in known_hosts\n    - add support for consistent snapshots (experimental)\n  * Infrastructure:\n    - add a daemon to monitor Scalaris via JMX\n    - disable message compression (only client values are compressed - the rest\n      is too expensive, at least on GbE)\n    - support for distributions with python3 available as \"python\" and\n      python2 as \"python2\"\n    - support for Ruby 1.9\n    - yaws 1.96 (with patch to compile on otp master and a patch to fix a\n      performance regression)\n    - support for Erlang R13B01 up to R16B01 and current otp master\n  * Tests:\n    - add test suite to find memory leaks\n    - let \"make test\" run the major test suites and \"make test-skipped\" for\n      some more (time-consuming) tests\n    - clean-up ring after timetrap timeout failures via common test hook\n    - new ?compare macro for custom comparison functions\n    - higher test coverage with more random-testing via the \"tester\"\n  * Documentation:\n    - user-dev-guide: add user tutorial on using scalaris\n    - user-dev-guide: add a section about the slide protocol\n    - user-dev-guide: extended description of scientific background\n    - add replica repair sequence diagrams\n    - better code descriptions\n  * Tools:\n    - gen_component: synchronous breakpoint set and delete for more\n                     deterministic usage\n    - trace_mpath: allow selective tracing via filter fun\n    - trace_mpath: fix several triggers becoming infected by trace_mpath\n                   resulting in infinite tracing\n    - trace_mpath: improve latex output of traces\n    - tester: copy dictionary to worker threads\n    - tester: add support for more types, e.g. neg_integer(), gb_rees\n    - tester: better type check error reporting\n    - tester: print tester last calls when aborting unit tests\n              (timeout or exception)\n    - tester: add support for constraints in type specs (\"when is_subtype(A,B)\")\n    - web debug interface: add cluster graph visualisation\n    - web debug interface: display vivaldi distance\n    - web debug interface: add IP addresses and ports to the ring charts and\n                           tables\n    - web debug interface: allow navigating to the web interfaces of shown nodes\n    - top: support for showing messages in message queue of an inspected PID\n    - top: support for showing larger dictionary values\n    - allow recursive reply_as envelopes\n    - experimental protocol scheduler to check protocols with random message\n      interleavings (see proto_sched module)\n  * Bugs:\n    - fix RM handling of (out-dated) nodes with the same ID as newly added nodes\n    - fix ganglia integration not working any more\n    - restore the ability to start nodes at a specific key via\n      \"scalarisctl -k <key> ...\"\n    - fix some memory leaks in the tx system\n    - fix statistics of comm_connection (not send in some cases, not\n      overflow-aware)\n    - use /bin/bash instead of /bin/sh which may not result in a bash session\n    - fix init.d scripts not checking for existing processes correctly\n    - fix dc_clustering\n    - fix numerous other bugs\n\n-------------------------------------------------------------------\nThu Oct 11 10:30:00 UTC 2012 - kruber@zib.de\n\n- Scalaris 0.5.0 (codename \"Saperda scalaris\")\n  * Packaging:\n    - new init.d script to start Scalaris\n    - added chef scripts to deploy Scalaris nodes\n    - improved Windows start scripts (support for R15B01 and R15B02, don't\n      close command prompt window immediately after shutdown if\n      double-clicked)\n    - more flexible scalarisctl (arbitrary parameter order, allow setting\n      cookie, ports and number of nodes in VM via parameters, allow using\n      screen for daemonised sessions, allow graceful leave via \"gstop\"\n      command, new \"status\" command)\n    - support for new linux distributions (Fedora 17, Ubuntu 12.04,\n      openSUSE 12.2)\n    - let scalarisctl checkinstallation also perform runtime tests for the\n      APIs\n  * API:\n    - allow Scalaris monitoring via JMX through the Java API\n    - added an executor-service to the Java-API (de.zib.scalaris.executor.*)\n    - added a node discovery daemon to the Java-API\n    - allow compressed communication between the Java-API and Erlang for\n      increased performance, especially if the two are on separate nodes\n    - added VM management support to the JSON- and Python-API\n    - added transaction log filtering to the Java-API, i.e. only sent the\n      needed parts of the tlog back to Erlang and re-combine the result\n    - fixed api_tx:req_list_commit_each/1 not running requests in parallel\n      -> do not assure any order of requests, even if on same key!\n  * Demonstrator \"Wiki on Scalaris\"\n    (supported by 4CaaSt http://www.4caast.eu/):\n    - allow different partitioned data models for better performance and\n      scalability\n    - allow logging of user requests\n    - added support for checking whether another article exists (approximate)\n      -> show link colours based on this check\n    - added check for bad page titles\n    - allow SERVERNAME and SERVERPATH in config for setups with load\n      balancers\n    - reduced memory footprint of Wiki data in Scalaris\n    - support for newer wiki xml dumps\n    - added support for using a MediaWiki-like SQLite-DB backend for e.g.\n      filtering\n    - improved overall performance\n    - several rendering fixes\n  * Business Logic:\n    - added (experimental) support for replica repair (disabled by default)\n      (thanks to Maik Lange)\n    - added monitoring of memory statistics (also available via web\n      interface)\n    - better error reporting in the failure detector\n    - reduced message overhead by UIDs and message/tuple tags\n    - reduced overall message size of transactions:\n      * do not include the (uncompressed) value in messages of the read phase\n        of write operations\n      * do not include the value in init_TP messages\n    - allow VM-flag \"first\" to be set via config file\n    - gather overall connection statistics in comm_stats (also available via\n      web interface)\n    - reduced erroneous failure messages on node shutdown\n    - integrated comm_layer into comm_server\n    - better scalability in pid_groups (find processes round-robin in\n      find_a/1)\n    - several changes to improve overall performance and/or CPU time at the\n      nodes\n  * Tests:\n    - support for more types in the runtime type-checker\n    - verify several API functions via runtime type-checker (also test\n      private functions if possible!)\n  * Tools:\n    - distributed protocol visualisation via trace_mpath (text-based or\n      latex-file for graphical presentation)\n    - better profiling via top for Erlang processes\n    - better debugging, e.g. via ASCII supervisor-tree rendering in verbose\n      mode\n  * Bugs:\n    - fixed memory leaks in read and write operations\n    - fixed memory leaks in tx_tm_rtm\n    - prevent potential endless loops in tx_tm_rtm\n    - fixed inform RTMs sometimes informing the wrong RTMs\n    - fixed numerous other bugs\n\n-------------------------------------------------------------------\nThu Mar 22 09:51:00 UTC 2012 - kruber@zib.de\n\n- Scalaris 0.4.1\n  * Packaging:\n    - new official ConPaaS packages (http://www.conpaas.eu/)\n    - install rubygem dependencies in Debian postinstall scripts for Ruby API\n    - improved Windows start scripts (if set, uses the ERLANG_HOME\n      environment variable to find Erlang, otherwise searches for Erlang in\n      common paths)\n  * Bugs:\n    - better tx cleanup (should fix rare occurance of duplicate client\n      inform)\n    - forward additional parameters of the start scripts to new syntax of\n      scalarisctl\n\n-------------------------------------------------------------------\nWed Jan 24 23:48:00 UTC 2012 - kruber@zib.de\n\n- Scalaris 0.4.0 (codename \"Pomacea scalaris\")\n  * API:\n    - new functions for incremental data change:\n      test_and_set: check for a provided old value before setting a new one\n      add_on_nr: increment a numeric value\n      add_del_on_list: append or delete entries from a list value\n    - added VM API to manage Scalaris nodes inside an Erlang virtual machine\n    - added monitoring API to retrieve some live metrics\n    - added a connection pool convenience class (Java, Python)\n  * Demonstrator \"Wiki on Scalaris\"\n    (supported by 4CaaSt http://www.4caast.eu/):\n    - improved performance of page edits\n    - improved performance of Wikipedia dump loading\n    - several rendering fixes\n  * Business Logic:\n    - improved handling of large values by reducing overhead of transaction\n      log handling (empty TLog after commit), no copy of value in TLog\n      returned to user after read requests)\n    - eliminated timeouts in data hand-over protocol (relies on fd now)\n    - added a DB subscribe mechanism, e.g. to become informed when locks\n      are freed\n    - fixed a strong consistency issue in the tx protocol\n    - gather some run-time statistics and expose them via the APIs and the\n      web debug interface\n  * Infrastructure:\n    - support for Erlang 15B\n    - fd now also uses feedback from TCP layer\n    - made message sending more flexible (gets an option list)\n    - added and corrected several Erlang type specifications\n    - added scripts to create Scalaris images for OpenNebula\n    - added tools for using Scalaris as the Database as a Service\n      component in ConPaaS (http://www.conpaas.eu/) which is part of the\n      EU project Contrail (http://contrail-project.eu/)\n    - added a separate communication channel for priority messages, e.g. fd\n      (reduces falsely reported node crashes under heavy load)\n  * Tests:\n    - added runtime type-checker for random testing extended unittests\n  * Documentation:\n    - updated documentation to extended APIs\n  * Bugs:\n    - fixed numerous bugs\n\n-------------------------------------------------------------------\nFri Jul 15 15:01:00 UTC 2011 - kruber@zib.de\n\n- Scalaris 0.3.0 (codename \"Rhinechis Scalaris\")\n  * API\n    - new API with interoperable bindings to Java, Python, Ruby, and JSON\n    - support for several data types, including strings, integers, JSON\n      objects, binary objects.\n    - new transaction interface with support for bundled requests for better\n      latency.\n    - separate APIs to access the raw DHT, a DHT with replication, and the\n      transactional DHT\n  * Demonstrator\n    - added Wikipedia-hosting using Scalaris as demonstrator application\n  * Business Logic\n    - fault-tolerant startup: start Scalaris when a quorum of the known_hosts\n      becomes available (option -q in bin/scalarisctl)\n    - perform data hand-over when nodes join/gracefully leave (also works when\n      transactions are executed concurrently)\n    - added passive load balancing (when a node joins a ring, it samples\n      several other nodes and joins at the node that balances the number of\n      stored items the most)\n    - completely rewritten transaction layer (more modular, more extendible,\n      less latency)\n    - modularized / reimplemented Paxos algorithm, so the algorithm can also\n      be used outside transactions (e.g. used for quorum-startup)\n    - switched almost all components to our component framework\n     'gen_component'\n    - added gossiping for estimating e.g. the number of nodes or the average\n      load in a ring\n    - more reliable unreliable look-up\n    - better ring start-up on slow networks\n  * Infrastructure\n    - Vivaldi and topology inference\n    - support for Erlang 13B01 and newer\n    - faster TCP/IP communication between Scalaris nodes\n    - completely rewritten failure detector framework for more accurate\n      detection of node failures\n    - added numerous Erlang type specifications\n    - extended unittests\n  * Tests\n    - added own random testing framework that reads type specifications and\n      scans the source code for constants to generate proper random test-data\n    - extended gen_component with breakpoint-support for debugging and testing\n      (perform deterministic pseudo-random message interleaving tests)\n    - added numerous unittests\n    - added language-binding interoperability tests\n  * Documentation\n    - extended, but - as always - by far not enough...\n  * Bugs\n    - fixed countless bugs\n"
  },
  {
    "path": "contrib/packages/main/scalaris.dsc",
    "content": "Format: 1.0\nSource: scalaris\nVersion: 0.9.0+git-1\nDEBTRANSFORM-RELEASE: 1\nBinary: scalaris\nMaintainer: Nico Kruber <kruber@zib.de>\nArchitecture: all\nStandards-Version: 3.9.6\nBuild-Depends: debhelper (>= 4.1.16), sudo, screen, erlang-dev (>= 14.b.4), erlang-parsetools (>= 14.b.4), erlang-tools (>= 14.b.4), erlang-edoc (>= 14.b.4), erlang-crypto (>= 14.b.4), pkg-config\nFiles:\n 6ea6383ef9251ff5ac641df84ddbcba0 2008366 scalaris.orig.tar.gz\n 2fecf324a32123b08cefc0f047bca5ee 0 scalaris.diff.tar.gz\n"
  },
  {
    "path": "contrib/packages/main/scalaris.spec",
    "content": "# norootforbuild\n\n%define pkg_version 0.9.0+git\n%define scalaris_user scalaris\n%define scalaris_group scalaris\n%define scalaris_home /var/lib/scalaris\nName:           scalaris\nSummary:        Scalable Distributed key-value store\nVersion:        %{pkg_version}\nRelease:        1\nLicense:        Apache-2.0\nGroup:          Productivity/Databases/Servers\nURL:            http://scalaris.zib.de\nSource0:        %{name}-%{version}.tar.gz\nSource99:       scalaris-rpmlintrc\nSource100:      checkout.sh\nBuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-build\nBuildArch:      noarch\nBuildRequires:  screen\nRequires:       screen net-tools\n\n##########################################################################################\n## Fedora, RHEL or CentOS\n##########################################################################################\n%if 0%{?fedora_version} || 0%{?rhel_version} || 0%{?centos_version}\nBuildRequires:  erlang-erts >= R14B04, erlang-kernel, erlang-stdlib, erlang-compiler, erlang-crypto, erlang-edoc, erlang-inets, erlang-parsetools, erlang-ssl, erlang-tools, erlang-xmerl, erlang-os_mon\nRequires:       erlang-erts >= R14B04, erlang-kernel, erlang-stdlib, erlang-compiler, erlang-crypto, erlang-inets, erlang-ssl, erlang-xmerl, erlang-os_mon\nRequires:       which\n%if 0%{?fedora_version} >= 19\nBuildRequires:  erlang-js\nRequires:       erlang-js\n%endif\n%if 0%{?fedora_version} >= 22\nBuildRequires:  hostname\nRequires:       hostname\n%endif\nBuildRequires:  pkgconfig\nRequires(pre):  shadow-utils\nRequires(pre):  /usr/sbin/groupadd /usr/sbin/useradd /bin/mkdir /bin/chown\n%if 0%{?fedora_version} >= 19 || 0%{?rhel_version} >= 700 || 0%{?centos_version} >= 700\n# https://fedoraproject.org/wiki/Packaging:Systemd?rd=Packaging:Guidelines:Systemd\n%define with_systemd 1\nBuildRequires:  systemd\nRequires:       systemd\nBuildRequires:  selinux-policy-devel\nRequires:       policycoreutils, libselinux-utils\nRequires(post): selinux-policy-base, policycoreutils\nRequires(postun): policycoreutils\n%else\n%define with_systemd 0\n%if 0%{?rhel_version} >= 600 || 0%{?centos_version} >= 600\n# provides runuser for the init.d script\nBuildRequires:  util-linux-ng >= 2.17\nRequires:       util-linux-ng >= 2.17\n%endif\n%endif\nBuildRequires:  sudo\nRequires:       sudo\n%endif\n\n###########################################################################################\n# SuSE, openSUSE\n###########################################################################################\n%if 0%{?suse_version}\nBuildRequires:  erlang >= R14B04\nRequires:       erlang >= R14B04\n%if 0%{?suse_version} >= 1310\nRequires:       which\n%else\nRequires:       util-linux\n%endif\n%if 0%{?suse_version} >= 1110\nBuildRequires:  erlang-erlang_js\nRequires:       erlang-erlang_js\n%endif\nBuildRequires:  pkg-config\nSuggests:       %{name}-java, %{name}-doc\nRequires(pre):  pwdutils\nPreReq:         /usr/sbin/groupadd /usr/sbin/useradd /bin/mkdir /bin/chown\nRequires(pre):  %insserv_prereq\n# keep systemd disabled for openSUSE 13.1 due to the runuser bug (see below)\n%if 0%{?suse_version} > 1310\n# https://en.opensuse.org/openSUSE:Systemd_packaging_guidelines\n%define with_systemd 1\nBuildRequires:  systemd\n%{?systemd_requires}\n%else\n%define with_systemd 0\n%endif\nBuildRequires:  sudo\nRequires:       sudo\n%endif\n\n%description\nScalaris is a scalable, transactional, distributed key-value store. It\ncan be used for building scalable services. Scalaris uses a structured\noverlay with a non-blocking Paxos commit protocol for transaction\nprocessing with strong consistency over replicas. Scalaris is\nimplemented in Erlang.\n\n%package doc\nSummary:    Documentation for Scalaris\nGroup:      Documentation/Other\nRequires:   %{name} == %{version}-%{release}\n\n%description doc\nDocumentation for Scalaris including its User-Dev-Guide.\n\n%prep\n%setup -q -n %{name}-%{version}\n\n%build\n# NOTE: disable runuser on openSUSE 13.1 because of the following bug\n# see https://bugzilla.novell.com/show_bug.cgi?id=892079\n./configure --prefix=%{_prefix} \\\n    --exec-prefix=%{_exec_prefix} \\\n    --bindir=%{_bindir} \\\n    --sbindir=%{_sbindir} \\\n    --sysconfdir=%{_sysconfdir} \\\n    --datadir=%{_datadir} \\\n    --includedir=%{_includedir} \\\n    --libdir=%{_libdir} \\\n    --libexecdir=%{_libexecdir} \\\n    --localstatedir=%{_localstatedir} \\\n    --sharedstatedir=%{_sharedstatedir} \\\n    --mandir=%{_mandir} \\\n    --infodir=%{_infodir} \\\n%if 0%{?suse_version} == 1310\n    --disable-runuser \\\n%endif\n%if 0%{?with_systemd}\n    --with-systemd=%{_unitdir} \\\n%endif\n    --docdir=%{_docdir}/scalaris \\\n    --disable-cpp\nmake all\nmake doc\n\n%install\nrm -rf $RPM_BUILD_ROOT\nmake install DESTDIR=$RPM_BUILD_ROOT\nmake install-doc DESTDIR=$RPM_BUILD_ROOT\n%if 0%{?with_systemd}\n%if 0%{?fedora_version} || 0%{?rhel_version} || 0%{?centos_version}\n# setup SELinux policies\ncd contrib/systemd\nsed -e \"s|/var/lib/scalaris|%{scalaris_home}|g\" \\\n    -i scalaris.fc\nmake -f /usr/share/selinux/devel/Makefile\ninstall -d %{buildroot}%{_datadir}/selinux/packages\ninstall -m 644 scalaris.pp %{buildroot}%{_datadir}/selinux/packages/\ncd -\n%endif\n%endif\n\n%pre\n# note: use \"-r\" instead of \"--system\" for old systems like CentOS5, RHEL5\ngetent group %{scalaris_group} >/dev/null || groupadd -r %{scalaris_group}\ngetent passwd %{scalaris_user} >/dev/null || mkdir -p %{scalaris_home} && useradd -r -g %{scalaris_group} -d %{scalaris_home} -M -s /sbin/nologin -c \"user for scalaris\" %{scalaris_user} && chown %{scalaris_user}:%{scalaris_group} %{scalaris_home}\n\n%if 0%{?suse_version}\n%if 0%{?with_systemd}\n%service_add_pre scalaris.service scalaris-first.service\n%endif\n%endif\n\nexit 0\n\n%post\nif grep -e '^cookie=\\w\\+' %{_sysconfdir}/scalaris/scalarisctl.conf > /dev/null 2>&1; then\n  echo $RANDOM\"-\"$RANDOM\"-\"$RANDOM\"-\"$RANDOM >> %{_sysconfdir}/scalaris/scalarisctl.conf\nfi\n\n%if 0%{?suse_version}\n%if 0%{?with_systemd}\n%service_add_post scalaris.service scalaris-first.service\n%else\n%fillup_and_insserv -f scalaris scalaris-first\n%endif\n%endif\n\n%if 0%{?fedora_version} || 0%{?rhel_version} || 0%{?centos_version}\n%if 0%{?with_systemd}\nsemodule -i %{_datadir}/selinux/packages/scalaris.pp || :\nif [ \"$1\" -le 1 ] ; then # First install\n  semanage port -a -t scalaris_port_t -p tcp 14194-14198 || :\n  semanage port -a -t scalaris_port_t -p tcp 8000-8004 || :\nfi\n/sbin/restorecon -R %{scalaris_home} || :\n%systemd_post scalaris.service scalaris-first.service\n%else\n/sbin/chkconfig --add scalaris\n%endif\n%endif\n\n%preun\n%if 0%{?suse_version}\n%if 0%{?with_systemd}\n%service_del_preun scalaris.service scalaris-first.service\n%else\n%stop_on_removal scalaris scalaris-first\n%endif\n%endif\n\n%if 0%{?fedora_version} || 0%{?rhel_version} || 0%{?centos_version}\n%if 0%{?with_systemd}\n%systemd_preun scalaris.service scalaris-first.service\n%else\n  if [ \"$1\" -eq 0 ] ; then # final removal\n    /sbin/service scalaris stop >/dev/null 2>&1\n    /sbin/chkconfig --del scalaris\n  fi\n%endif\n%endif\n\n%postun\n%if 0%{?suse_version}\n%if 0%{?with_systemd}\n%service_del_postun scalaris.service scalaris-first.service\n%else\n%restart_on_update scalaris scalaris-first\n%insserv_cleanup\n%endif\n%endif\n\n%if 0%{?fedora_version} || 0%{?rhel_version} || 0%{?centos_version}\n%if 0%{?with_systemd}\n%systemd_postun_with_restart scalaris.service scalaris-first.service\nif [ \"$1\" -eq 0 ]; then # final removal\n  semanage port -d -t scalaris_port_t -p tcp 14194-14198 2>/dev/null || :\n  semanage port -d -t scalaris_port_t -p tcp 8000-8004 2>/dev/null || :\n  semodule -r scalaris || :\n  /sbin/restorecon -R %{scalaris_home} || :\nfi\n%else\nif [ \"$1\" -ge 1 ] ; then  # pkg was updated -> restart\n  /sbin/service scalaris try-restart >/dev/null 2>&1 || :\nfi\n%endif\n%endif\n\n%clean\nrm -rf $RPM_BUILD_ROOT\n\n%files\n%defattr(-,root,root,-)\n%dir %{_docdir}/scalaris\n%{_docdir}/scalaris/AUTHORS\n%{_docdir}/scalaris/README.md\n%{_docdir}/scalaris/LICENSE\n%{_docdir}/scalaris/ChangeLog\n%{_bindir}/scalarisctl\n%{_prefix}/lib/scalaris\n%exclude %{_prefix}/lib/scalaris/docroot/doc\n%attr(-,scalaris,scalaris) %{_localstatedir}/log/scalaris\n%if 0%{?with_systemd}\n%{_unitdir}/scalaris.service\n%{_unitdir}/scalaris-first.service\n%dir %{_sysconfdir}/conf.d\n%config(noreplace) %{_sysconfdir}/conf.d/scalaris\n%config(noreplace) %{_sysconfdir}/conf.d/scalaris-first\n%if 0%{?fedora_version} || 0%{?rhel_version} || 0%{?centos_version}\n%attr(0600,root,root) %{_datadir}/selinux/packages/scalaris.pp\n%endif\n%else\n%{_sysconfdir}/init.d/scalaris\n%{_sysconfdir}/init.d/scalaris-first\n%{_sbindir}/rcscalaris\n%config(noreplace) %{_sysconfdir}/scalaris/initd.conf\n%config(noreplace) %{_sysconfdir}/scalaris/initd-first.conf\n%endif\n%dir %{_sysconfdir}/scalaris\n%config %{_sysconfdir}/scalaris/scalaris.cfg\n%config(noreplace) %{_sysconfdir}/scalaris/scalaris.local.cfg\n%config %{_sysconfdir}/scalaris/scalaris.local.cfg.example\n%config(noreplace) %{_sysconfdir}/scalaris/scalarisctl.conf\n\n%files doc\n%defattr(-,root,root,-)\n%doc %{_docdir}/scalaris/erlang\n%doc %{_docdir}/scalaris/user-dev-guide.pdf\n%{_prefix}/lib//scalaris/docroot/doc\n\n%changelog\n"
  },
  {
    "path": "contrib/packages/scalaris-pattern.xml",
    "content": "<!-- See http://en.opensuse.org/Build_Service_Tutorial#Create_Patterns -->\n<!-- See https://github.com/openSUSE/libzypp/blob/master/zypp/parser/yum/schema/patterns.rng -->\n\n<pattern\n xmlns=\"http://novell.com/package/metadata/suse/pattern\"\n xmlns:rpm=\"http://linux.duke.edu/metadata/rpm\">\n    <name>Scalaris</name>\n    <summary>Scalaris, a distributed key-value store</summary>\n    <description>Scalaris is a scalable, transactional, distributed key-value store. It can be used for building scalable Web 2.0 services.\n\nScalaris uses a structured overlay with a non-blocking Paxos commit protocol for transaction processing with strong consistency over replicas. Scalaris is implemented in Erlang.</description>\n    <uservisible/>\n    <category lang=\"en\">Database</category>\n    <category lang=\"de\">Datenbank</category>\n    <rpm:requires>\n      <rpm:entry name=\"scalaris\"/>\n    </rpm:requires>\n    <rpm:recommends>\n      <rpm:entry name=\"scalaris-java\"/>\n      <rpm:entry name=\"python-scalaris\"/>\n      <rpm:entry name=\"python3-scalaris\"/>\n      <rpm:entry name=\"ruby-scalaris\"/>\n    </rpm:recommends>\n    <rpm:suggests>\n      <rpm:entry name=\"scalaris-doc\"/>\n    </rpm:suggests>\n</pattern>\n"
  },
  {
    "path": "contrib/scalaris-start-local.sh",
    "content": "#!/bin/bash\n\n\n################################################################################\n#  Start a Scalaris ring on a local machine with multiple VMs and multiple dht\n#  nodes per VM. Details see print_usage().\n#\n#  For frequent add somethin like the following to your shell configuration:\n#  function starts() {\n#      $SCALARIS_DIR/contrib/scalaris_start_local -asw $1 $2\n#  }\n#  alias stops=\"$SCALARIS_DIR/contrib/scalaris_start_local.sh stop\"\n################################################################################\n\nSCALARIS_DIR=${SCALARIS_DIR:-$HOME/scalaris}\nETCDIR=$SCALARIS_DIR/bin\nBINDIR=$SCALARIS_DIR/bin\nBEAMDIR=$SCALARIS_DIR/ebin\nSHUFFLE_NODE_IDS=0\nATTACH=${ATTACH:-false}\nWAIT_FOR_STABLE_RING=${WAIT_FOR_STABLE_RING=-false}\nDHT_NODES_PER_VM=${DHT_NODES_PER_VM:-1}\n\nfunction start_servers() {\n    echo \"starting Scalaris with $VMS_PER_NODE VMs and $DHT_NODES_PER_VM dht nodes per vm...\"\n\n    let NR_OF_DHT_NODES=$VMS_PER_NODE\\*$DHT_NODES_PER_VM\n    let NR_OF_VMS=$VMS_PER_NODE\n\n    KEYLIST=\"\"\n    if [ $SHUFFLE_NODE_IDS -eq 1 ]\n    then\n        KEYLIST=`erl -name bench_ -pa $BEAMDIR -noinput -eval \"L = util:lists_split(util:shuffle(api_dht_raw:split_ring($NR_OF_DHT_NODES)), $NR_OF_VMS), io:format('~p', [L]), halt(0).\"`\n    else\n        KEYLIST=`erl -name bench_ -pa $BEAMDIR -noinput -eval \"L = util:lists_split(api_dht_raw:split_ring($NR_OF_DHT_NODES), $NR_OF_VMS), io:format('~p', [L]), halt(0).\"`\n    fi\n\n    PORT=14195\n    YAWSPORT=8000\n\n    VM_IDX=1\n    JOIN_KEYS=`erl -name bench_ -noinput -eval \"L = lists:nth($VM_IDX, $KEYLIST), io:format('~p', [L]), halt(0).\"`\n    $BINDIR/scalarisctl -j \"$JOIN_KEYS\" -n first -p $PORT -y $YAWSPORT --nodes-per-vm $DHT_NODES_PER_VM --screen -d -m -t first start\n    let PORT+=1\n    let YAWSPORT+=1\n    let VM_IDX+=1\n    for NODE in $(seq 2 $VMS_PER_NODE); do\n        JOIN_KEYS=`erl -name bench_ -noinput -eval \"L = lists:nth($VM_IDX, $KEYLIST), io:format('~p', [L]), halt(0).\"`\n        $BINDIR/scalarisctl -j \"$JOIN_KEYS\" -n node$PORT -p $PORT -y $YAWSPORT --nodes-per-vm $DHT_NODES_PER_VM --screen -d -t joining start\n        let PORT+=1\n        let YAWSPORT+=1\n        let VM_IDX+=1\n    done\n}\n\nfunction wait_for_stable_ring {\n    echo \"waiting for stable ring...\"\n    let NR_OF_NODES=$VMS_PER_NODE\\*$DHT_NODES_PER_VM\n\n    # wait for the first VM to start\n    NR_OF_FIRSTS=`epmd -names | grep 'name first at port' | wc -l`\n    while [ $NR_OF_FIRSTS -ne 1 ]\n    do\n        NR_OF_FIRSTS=`epmd -names | grep 'name first at port' | wc -l`\n    done\n    # wait for the first VM to initialize\n    erl -name bench_ -noinput -eval \"A = rpc:call('first@`hostname -f`', api_vm, wait_for_scalaris_to_start, []), io:format('\\tserver started: ~p~n', [A]), halt(0).\"\n    # wait for the ring to stabilize\n    erl -name bench_ -noinput -eval \"A = rpc:call('first@`hostname -f`', admin, wait_for_stable_ring, [$NR_OF_NODES]), io:format('\\tring stable: ~p~n', [A]), halt(0).\"\n}\n\nfunction kill_old_nodes() {\n    NO_OF_SESSIONS=$(screen -ls | grep Detached | grep scalaris_ | wc -l)\n    if [[ $NO_OF_SESSIONS -gt 0 ]]; then\n        echo \"killing $NO_OF_SESSIONS VMs\"\n        screen -ls | grep Detached | grep scalaris_ | cut -d. -f1 | awk '{print $1}' | xargs -r kill\n    fi\n}\n\nprint_usage() {\ncat << EOF\n\nSynopsis\n  $(basename $0) [-w] [-h] [stop | <no of vms> [ <no_of_dht_nodes> ]\n  Start a Scalaris ring on a local machine with multiple WMs and multiple DHT\n  nodes per VM.\n  The nodes are placed evenly across the ring. Still running ring will be killed\n  automatically.\n\n -a\n    attach to the screen session of the first node\n\n -s\n    shuffle ids\n\n -w\n    Wait for a stable ring.\n\n -h\n    Print this help.\n\n  -v\n    Enabble verbose mode, i.e. bash debugging ('set -x').\nEOF\n}\n\n############\n#   MAIN   #\n############\n\n# show usage if invoked without options/arguments\nif [ $# -eq 0 ]; then\n  print_usage\n  exit 1\nfi\n\n\n# parse options\nwhile getopts :a:hsvw opt; do\n  case $opt in\n    a)\n        ATTACH=true\n        ATTACH_NODE=$OPTARG\n        ;;\n    s)\n        SHUFFLE_NODE_IDS=1\n        ;;\n    w)\n        WAIT_FOR_STABLE_RING=true\n        ;;\n    h)\n        print_usage\n        exit 1\n        ;;\n    v)\n      set -x\n      ;;\n    \\?)\n      echo \"Invalid option: -$OPTARG\" >&2\n      exit 1\n      ;;\n    :)\n      echo \"Option -$OPTARG requires an argument.\" >&2\n      exit 1\n      ;;\n  esac\ndone\n\n# shift the arguments parsed by getopts away\nshift $((OPTIND-1))\n\n# get the mass arguments (either stop or <no of vms> <no of dht nodes per vm>\nif [[ -n $1 ]]; then\n    if [[ $1 = \"stop\"  ]]; then\n        kill_old_nodes\n        exit 0\n    else\n        VMS_PER_NODE=$1\n    fi\nfi\nif [[ -n $2 ]]; then\n    DHT_NODES_PER_VM=$2\nfi\n\nkill_old_nodes\nstart_servers\n\nif $WAIT_FOR_STABLE_RING; then\n    wait_for_stable_ring\nfi\n\n# attach to node given be -a\nif $ATTACH; then\n    screen -r $(screen -ls | grep scalaris_$ATTACH_NODE | cut -d. -f1 | awk '{print $1}')\nfi\n\n"
  },
  {
    "path": "contrib/slurm/README",
    "content": "# Scalaris on SLURM\n\nScripts to run Scalaris on SLURM. The idea is to completely script all performed\ntasks on the cumulus cluster. On a frontend machine (e.g. cumulus.zib.de) a script\nsets the desired parameters and enqueues the tasks via `sbatch`. For an example\nsee `bench.sh`.\n\nThe queued tasks execute another script on the nodes\nof the cumulus cluster (cumu01-00 - cumu01-15 and cumu02-00-cumu02-15), performing\nthe actual tasks. For examples see `example-job-script` and `increment-bench.slurm`.\n\nThe rest of the scripts are scripts for default tasks, like starting and stopping\nScalaris.\n\n# Scripts\n\n## bench.sh\n* runs a whole series of benchmarks\n* to start, execute `bench.sh`, e.g. on `cumulus.zib.de`\n        $ ./bench.sh\n* runs in the context of cumulus.zib.de\n* adjust number of nodes, VMs per Node, number of DHT nodes per vm,\n  number of repetitions (iterations) in the script\n* calls `sbatch` with `increment-bench.slurm` as script\n* the output of `increment-bench.slurm` is returned to the calling `sbatch` command\n    and saved in the directory from which `bench.sh` was called\n* the name of the outputfile is defined in `bench.sh` in the `sbatch` call with\n    the '-o' parameter and has the default form of\n\n    ```slurm-<ErlangVersion>-<Nodes>-<VMs/Node>-<DHTNodes/VM>-<jobid>.out```\n\n\n## increment-bench.slurm\n* runs in the context of exactly one node of the cumulus cluster\n* sets up a Scalaris ring through calling `start-scalaris.sh` with the number of\n    nodes, vms and dht-nodes specified in `bench.sh`\n* executes the set of comands specified in the designated area with one set\n    of parameters from `bench.sh`\n* shuts down the Scalaris ring via stop-scalaris.sh\n\n## start-scalaris\n* starts a Scalaris ring\n\n## stop-scalaris\n* stops a Scalaris ring\n\n## env.sh\n* provides default values for environment variables controlling Scalaris and Slurm\n\n# Web Interface\n\nThe web interface of a running Scalaris node can be accessed through\n\n    `http://cumu01-<Node-Nr>.zib.de:8000/index.yaws`\ne.g.\n    [http://cumu01-n03.zib.de:8000/index.yaws](http://cumu01-03.zib.de:8000/index.yaws)\n\nfrom every PC that has access to the cumulus cluster.\n\n# Access nodes running Scalaris\n* `ssh`-ing into a node is not possible with slurm\n* start the cmd or script with the `--share` instead of ``--exclusive` (default)\n    * Beware: Anyone can access this node now\n* an interactive shell can now be opened on the respective node with\n\n        $ srun -p CUMU -A csr --nodelist=cumu01-00 --pty bash\n\n* attach to the screen session running the Scalaris node\n\n# Canceling jobs\n\nA job can be canceled with `scancel <job-id>` by the user. The cmd or script of\nthe canceled job will first get a SIGTERM and later a SIGKILL by slurm.\n\nIt is possible to start processes through the cmd or script which are not\nterminated when the job is terminated. For Scalaris, this is true for example\nfor all the Erlang VMs and epmd. These have to be stopped manually on every node\nin case of cancellation.\n\nA script (`cleanup.sh`) is provided for this purpose. The script needs to be\nexecuted on all nodes used by the canceled\njob, run e.g.\n\n        $ srun -p CUMU -A csr --nodelist=\"cumu01-03,cumu01-04\" cleanup.sh\n\nto cleanup the nodes cumu01-[03,04]. The list of used nodes can be found in the\nlogfile of the cancelled job.\n\nThe `start-scalaris.sh` script also cancels still running Scalaris nodes, so if\nScalaris is started on the same ring immediately afterwards, running the cleanup\nscript is not necessary.\n\n# Quick and Dirty\n\nRun a script on a two-node setup\n\n    $ sbatch -N2 example-job-script.slurm\nMost of the parameters will be set through default values in `env.sh`.\n\nIndividual parameters can be set on a per call basis, e.g. specify the number of\nnodes per VM (this would lead to 16 Scalaris nodes, 2 machines with 8 Erlang-VMs each)\n\n    $ VMS_PER_NODE=\"8\" sbatch -N2 example-job-script.slurm\n"
  },
  {
    "path": "contrib/slurm/basho-bench.sh",
    "content": "#!/bin/bash\n\n###############################################################################\n# Author: Jens V. Fischer\n# Date: 11.11.2015\n#\n# Running a benchmark series on a Scalaris slurm installation with basho bench.\n# The scripts reads the basho-bench.cfg, sets up a Scalaris ring using the slurm\n# cluster management tool, using the slurm script collection of Scalaris with\n# basho_bench.slurm as slurm script. It then waits for the ring to set up and\n# starts multiple basho bench intances (\"load # generators\") on one or multiple\n# machines by calling start-basho-bench.sh.\n#\n# Call:\n# \t./basho-bench.sh\n#\n# Configuration:\n#   All configuration settings, including documentation and default values can\n#   be found in basho-bench.cfg.\n#   A quick-config section is provided below for overriding values from the\n#   configuration file. This is meant for easy manipulation of the most commonly\n#   used configuration parameters.\n###############################################################################\n\ntrap 'trap_cleanup' SIGTERM SIGINT\n\n# QUICK-CONFIG ===============\n\n# Values defined here override settings from the configuration file\n\n# REPETITIONS=2\n# DURATION=5\n# LOAD_GENERATORS=4\n#\n# PARTITION=\"CUMU\"\n# TIMEOUT=15\n# SLEEP1=30\n# SLEEP2=30\n#\n# SCALARIS_LOCAL=true\n# COLLECTL=true\n#\n# # size scalability series (only uncomment 'size' or 'load')\n# KIND='size'\n# NODES_SERIES=\"1 2 4 8 16 32\"\n# VMS_PER_NODE_SERIES=\"1\"\n# export ERL_SCHED_FLAGS=\"+S 32\"\n# LOAD_LEVEL=5\n\n# vary value sizes\n# KIND='value'\n# WORKERS_PER_LG=44\n# VALUE_SIZES=\"2 4 8 16 32 64 128 256 512 768 1024 1280 1536 1792 2048\"\n# NODES=32\n\n# load scalability series\n# KIND='load'\n# NODES=32\n# VMS_PER_NODE=1\n# WORKERS_PER_LG_SERIES=\"1 2 4 8 16 32 64 128 256 512 1024 2048\"\n\n# KIND=\"lgs\"\n# LOAD_GENERATORS_SERIES=\"4 6 8 10 12 15\"\n# WORKERS=\"8400\"\n# NODES=32\n# VMS_PER_NODE=4\n# export ERL_SCHED_FLAGS=\"+S 8\"\n\n#=============================\n\nmain() {\n    is_lg_external\n    if ! $EXTERNAL_LG ; then\n        LG_HOSTS=`scontrol show hostnames`\n        LG_HOSTS=($LG_HOSTS)\n    fi\n    source $(pwd)/config/basho-bench.cfg\n    check_wdir\n    check_result_dir\n    setup_logging\n    print_env\n    check_compile\n\n    if [[ $KIND == \"size\" ]]; then\n        main_size\n    elif [[ $KIND == \"load\" ]]; then\n        main_load\n    elif [[ $KIND == \"value\" ]]; then\n        main_value\n    elif [[ $KIND == \"lgs\" ]]; then\n        main_lgs\n    else\n        log error \"Unknown kind of benchmark, exiting\"\n        exit 1\n    fi\n}\n\nmain_lgs(){\n    for LOAD_GENERATORS in $LOAD_GENERATORS_SERIES; do\n        WORKERS_PER_LG=$((WORKERS/LOAD_GENERATORS))\n        log info \"starting load benchmark with $LOAD_GENERATORS LOAD_GENERATORS\"\n        log info \"WORKERS=$WORKERS\"\n        log info \"WORKERS_PER_LG=$WORKERS_PER_LG\"\n\n        PREFIX=\"lgs$(printf \"%04i\" $LOAD_GENERATORS)\"\n        repeat_benchmark\n    done\n}\n\nmain_value() {\n    for VALUE_SIZE in $VALUE_SIZES; do\n        local value=$(printf \"%04i\" $VALUE_SIZE)\n        PREFIX=\"value$value\"\n        log info \"starting value benchmark with $VALUE_SIZE\"\n        repeat_benchmark\n    done\n}\n\nmain_size(){\n    for NODES in $NODES_SERIES; do\n        if ((NODES==1)); then\n            NODELIST=\"\"\n        elif ((NODES==2)); then\n            NODELIST=\"cumu01-00,cumu02-00\"\n        else\n            half=$((NODES/2-1))\n            NODELIST=\"cumu01-[00-$half],cumu02-[00-$half]\"\n        fi\n\n        for VMS_PER_NODE in $VMS_PER_NODE_SERIES; do\n            local ringsize=$((NODES*VMS_PER_NODE*DHT_NODES_PER_VM))\n            WORKERS=$((ringsize*LOAD_LEVEL))\n            WORKERS_PER_LG=$((WORKERS/LOAD_GENERATORS))\n            log info \"RINGSIZE=$ringsize\"\n            log info \"WORKERS=$WORKERS\"\n            log info \"WORKERS_PER_LG=$WORKERS_PER_LG\"\n\n            ringsize=$(printf \"%04i\" $ringsize)\n            PREFIX=\"size$ringsize\"\n\n            repeat_benchmark\n        done\n    done\n}\n\nmain_load(){\n    for WORKERS_PER_LG in $WORKERS_PER_LG_SERIES; do\n        WORKERS=$((WORKERS_PER_LG*LOAD_GENERATORS))\n\n        var=1\n        for OPS in $OPERATIONS_SERIES; do\n            log info \"WORKERS=$WORKERS\"\n            log info \"WORKERS_PER_LG=$WORKERS_PER_LG\"\n\n            OPERATIONS=$OPS\n            log info \"OPERATIONS=$OPERATIONS\"\n\n            WORKERS=$(printf \"%04i\" $WORKERS)\n            PREFIX=\"load$WORKERS-$var\"\n            log info \"starting load benchmark with $WORKERS ($WORKERS_PER_LG*$LOAD_GENERATORS)\"\n\n            let \"var++\"\n            repeat_benchmark\n        done\n    done\n}\n\nrepeat_benchmark() {\n    for run in $(seq 1 $REPETITIONS); do\n\n        NAME=\"${PREFIX}-r$run\"\n        mkdir ${WD}/${NAME}\n        setup_directories\n        create_result_dir\n\n        # setup Scalaris log path\n        SLOGPATH=${SLOGPATH/?NAME/$NAME}  # replace \"?NAME\" with \"$NAME\"\n        SCALARISCTL_PARAMS=\"$SLOGPATH $SCTL_PARAMS\"  # Prepend log path\n        echo ${!SCALARISCTL_PARAMS@}=$SCALARISCTL_PARAMS\n\n        COLLECTL_DIR=$WD/$NAME/collectl\n        echo ${!COLLECTL_DIR@}=$COLLECTL_DIR\n\n        log info \"starting repetition $run...\"\n        [[ $COLLECTL = true ]] && start_collectl\n        [[ $TOPLOG = true ]] && start_toplog\n        start_scalaris\n\n        wait_for_scalaris_startup\n        build_hostlist\n\n        test_ring\n        run_bbench\n        test_ring\n\n        stop_scalaris\n        rm_lockfile\n\n        log info \"sleeping for $SLEEP1 seconds\"; sleep $SLEEP1\n        [[ $COLLECTL = true ]] && stop_collectl\n        [[ $TOPLOG = true ]] && stop_toplog\n    done\n\n    if (( SLEEP2 > 0 )); then\n        log info \"sleeping for $SLEEP2 seconds\"\n        sleep $SLEEP2\n    fi\n    collect_bbench_results\n\n}\n\n\n#=====================\n# FUNCTIONS\n#=====================\n\ncheck_wdir() {\n    # check if WD exists\n    if [[ ! -d $WD ]]; then\n        mkdir -p $WD\n    else\n        # check if WD is empty\n        if [ \"$(ls $WD)\" ]; then\n            log info \"Working directory ($WD) is not empty, containing the following files/dirs:\"\n            ls -l1 $WD\n            read -p \"Delete all files? \" -n 1 -r\n            echo    # move to a new line\n            if [[ $REPLY =~ ^[Yy]$ ]]; then\n                rm -r $WD/*\n            else\n                log error \"aborting...\"\n                exit 1\n            fi\n        fi\n    fi\n}\n\nsetup_logging(){\n    LOGFILE=\"$WD/bbench-suite-$(date +%y.%m.%d-%H:%M:%S).log\"\n    log info \"writing output also to $LOGFILE\"\n    # w/o -i option to tee, signal trapping does NOT work!\n    exec &>> >(tee -i $LOGFILE)\n}\n\nsetup_directories(){\n    if [[ $COLLECTL = true && ! -d $WD/$NAME/collectl ]]; then\n        mkdir -p $WD/$NAME/collectl\n    fi\n}\n\n\nprint_env(){\n    echo KIND=$KIND\n    if [[ $KIND == \"load\" ]]; then\n        echo RINGSIZE=$((NODES*VMS_PER_NODE*DHT_NODES_PER_VM))\n        echo NODES=$NODES\n        echo VMS_PER_NODE=$VMS_PER_NODE\n        echo WORKERS_PER_LG_SERIES=$WORKERS_PER_LG_SERIES\n    elif [[ $KIND == \"size\" ]]; then\n        echo NODES_SERIES=$NODES_SERIES\n        echo VMS_PER_NODE_SERIES=$VMS_PER_NODE_SERIES\n        echo LOAD_LEVEL=$LOAD_LEVEL\n    elif [[ $KIND == \"value\" ]]; then\n        echo RINGSIZE=$((NODES*VMS_PER_NODE*DHT_NODES_PER_VM))\n        echo WORKERS_PER_LG=$WORKERS_PER_LG\n        echo VALUE_SIZES=$VALUE_SIZES\n    fi\n    echo \"ERL_SCHED_FLAGS=$ERL_SCHED_FLAGS\"\n    echo TIMEOUT=$TIMEOUT\n    echo REPETITIONS=$REPETITIONS\n    echo DURATION=$DURATION\n    echo LOAD_GENERATORS=$LOAD_GENERATORS\n    echo LG_HOSTS=${LG_HOSTS[@]}\n    echo SLEEP1=$SLEEP1\n    echo SLEEP2=$SLEEP2\n    echo \"COLLECTL=$COLLECTL\"\n    echo \"PARTITION=$PARTITION\"\n    echo \"VALUE_SIZE=$VALUE_SIZE\"\n}\n\ncheck_compile(){\n    pushd $SCALARIS_DIR >/dev/null\n    local res=$(erl -pa contrib/yaws -pa ebin -noinput +B -eval 'R=make:all([noexec]), halt(0).')\n    popd >/dev/null\n    if [[ -n $res ]]; then\n        log error \"Scalaris binaries do not match source version:\"\n        echo $res\n        exit 1\n    fi\n}\n\nlog(){\n    local level=$1\n    local message=$2\n    printf \"%s %s\\n\" \"$(tag $level)\" \"$message\"\n}\n\ntag(){\n    local level=$1\n    printf \"[bbench] %s  [%s]\" \"$(date +%H:%M:%S)\" \"$level\"\n}\n\nstart_collectl() {\n    export COLLECTL_SUBSYSTEMS\n    export COLLECTL_INTERVAL\n    export COLLECTL_FLUSH\n    # start collectl at the load generators\n    for host in ${LG_HOSTS[@]}; do\n        log info \"starting collectl on $host\"\n        if [[ $(hostname -f) = $host ]]; then\n            collectl $COLLECTL_SUBSYSTEMS $COLLECTL_INTERVAL $COLLECTL_FLUSH -f $WD/$NAME/collectl/lg_$host 2>/dev/null &\n        else\n            if [[ $EXTERNAL_LG = true ]]; then\n                ssh $host collectl $COLLECTL_SUBSYSTEMS $COLLECTL_INTERVAL $COLLECTL_FLUSH -f $WD/$NAME/collectl/lg_$host 2>/dev/null &\n            else\n                srun --nodelist=$host -N1 bash -c \"collectl $COLLECTL_SUBSYSTEMS $COLLECTL_INTERVAL $COLLECTL_FLUSH -f $WD/$NAME/collectl/lg_$host 2>/dev/null &\"\n            fi\n        fi\n    done\n}\n\nstop_collectl(){\n    # stop collectl on load generators (collectl on slurm nodes are killed by the watchdog)\n    for host in ${LG_HOSTS[@]}; do\n        log info \"killing collectl on $host\"\n        if [[ $(hostname -f) = $host ]]; then\n            pkill -f lg_$host\n        else\n            if [[ $EXTERNAL_LG = true ]]; then\n                ssh $host pkill -f lg_$host\n            else\n                srun --nodelist=$host -N1 bash -c \"pkill -f lg_$host\"\n            fi\n        fi\n    done\n}\n\nstart_toplog() {\n    # start toplog at the load generators\n    for host in ${LG_HOSTS[@]}; do\n        log info \"starting toplog on $host\"\n        if [[ $(hostname -f) = $host ]]; then\n            $SCALARIS_DIR/contrib/slurm/util/toplog.sh \"$WD/$NAME\" &\n        else\n            if [[ $EXTERNAL_LG = true ]]; then\n                ssh $host $SCALARIS_DIR/contrib/slurm/util/toplog.sh \"$WD/$NAME\" &\n            else\n                srun --nodelist=$host -N1 bash -c \"$SCALARIS_DIR/contrib/slurm/util/toplog.sh \"$WD/$NAME\" &\"\n            fi\n        fi\n    done\n}\n\nstop_toplog(){\n    # stop toplog on load generators\n    for host in ${LG_HOSTS[@]}; do\n        log info \"killing toplog on $host\"\n        if [[ $(hostname -f) = $host ]]; then\n            pkill -f toplog.sh\n        else\n            if [[ $EXTERNAL_LG = true ]]; then\n                ssh $host pkill -f toplog.sh\n            else\n                srun --nodelist=$host -N1 bash -c \"pkill -f toplog.sh\"\n            fi\n        fi\n    done\n}\n\n# Setup up a scalaris ring on with slurm on cumulus\nstart_scalaris() {\n    log info \"starting scalaris...\"\n    # setup environment\n    [[ -n $VMS_PER_NODE ]] && export VMS_PER_NODE\n    [[ -n $WATCHDOG_INTERVAL ]] && export WATCHDOG_INTERVAL\n    [[ -n $DHT_NODES_PER_VM ]] && export DHT_NODES_PER_VM\n    [[ -n $SHUFFLE_NODE_IDS ]] && export SHUFFLE_NODE_IDS\n    [[ -n $WD ]] && export WD\n    [[ -n $COLLECTL ]] && export COLLECTL\n    [[ -n $COLLECTL_DIR ]] && export COLLECTL_DIR\n    [[ -n $SCALARIS_LOCAL ]] && export SCALARIS_LOCAL\n    [[ -n $SCALARISCTL_PARAMS ]] && export SCALARISCTL_PARAMS\n    [[ -n $NAME ]] && export NAME\n    [[ -n $ERL_SCHED_FLAGS ]] && export ERL_SCHED_FLAGS\n\n    # start sbatch command and capture output\n    # the ${var:+...} expands only, if the variable is set and non-empty\n    RET=$( sbatch -A csr -o $WD/$NAME/slurm-%j.out \\\n            ${PARTITION:+-p $PARTITION} \\\n            ${NODES:+-N $NODES} \\\n            ${NODELIST:+ --nodelist=$NODELIST} \\\n            ${TIMEOUT:+ -t $TIMEOUT} \\\n            basho-bench.slurm\n         )\n\n    # get the job id from the output of sbatch\n    REGEX=\"Submitted batch job ([[:digit:]]*)\"\n    if [[ $RET =~ $REGEX ]]; then\n        SLURM_JOBID=${BASH_REMATCH[1]}\n    else\n        exit 1\n    fi\n    local nodes=\"$(($NODES*$VMS_PER_NODE*$DHT_NODES_PER_VM)) ($NODES*$VMS_PER_NODE*$DHT_NODES_PER_VM)\"\n    log info \"submitted batch job $SLURM_JOBID to start scalaris with $nodes\"\n}\n\nwait_for_scalaris_startup() {\n    LOCKFILE=\"${WD}/${SLURM_JOBID}.lock\"\n    echo -n \"$(tag info) waiting for scalaris to start\"\n    timer=0\n    until [[ -e $LOCKFILE ]]; do\n        ((timer++))\n        # display status every 5 seconds\n        if ((timer%5==0)); then\n            echo -ne \".\"\n        fi\n        sleep 1\n    done\n    echo \": ok (${timer}s)\"\n}\n\ntest_ring() {\n    local retries=$1\n    local res=0\n    [[ -z \"$retries\" ]] && retries=0\n    local ringsize=$((NODES*VMS_PER_NODE*DHT_NODES_PER_VM))\n    log info \"testing ring\"\n    erl -setcookie \"chocolate chip cookie\" -name bench_ -noinput -eval \\\n        \"A = rpc:call($FIRST, admin, number_of_nodes, []),\n         case A of\n             $ringsize -> halt(0);\n             _ -> io:format('number_of_nodes: ~p~n', [A]), halt(1)\n         end.\"\n    res=$((res+=$?))\n    erl -setcookie \"chocolate chip cookie\" -name bench_ -noinput -eval \\\n        \"A = rpc:call($FIRST, admin, check_ring, []),\n         case A of\n             ok -> halt(0);\n             Error -> io:format('check_ring: ~p~n', [Error]), halt(1)\n         end.\"\n    res=$((res+=$?))\n    erl -setcookie \"chocolate chip cookie\" -name bench_ -noinput -eval \\\n        \"A = rpc:call($FIRST, admin, check_ring_deep, []),\n         case A of\n             ok -> halt(0);\n             Error -> io:format('check_ring_deep: ~p~n', [Error]), halt(1)\n         end.\"\n    res=$((res+=$?))\n\n\n    if  [[ $res -eq 0 ]]; then\n        log info \"testing ring was successful\"\n    else\n        if (( retries++ >= 2 )); then\n            log error \"test_ring failed, after $retries retries. Aborting...\"\n            shutdown\n            kill_bbench\n            exit 1\n        else\n            local sleeptime=20\n            log error \"testing ring failed, retrying in $sleeptime seconds...\"\n            sleep $sleeptime\n            test_ring $retries\n        fi\n    fi\n}\n\nstop_scalaris(){\n    log info \"stopping scalaris\"\n    scancel $SLURM_JOBID\n}\n\nbuild_hostlist() {\n    local counter=0\n    declare -a hosts\n    NODELIST=$(scontrol show job $SLURM_JOBID | grep \" NodeList\" | awk -F= '{print $2}')\n    for host in $(scontrol show hostnames $NODELIST); do\n        counter=$(($counter+1))\n        max_port=$((14194+VMS_PER_NODE))\n        for port in $(seq 14195 $max_port); do\n            if (( ${#hosts[@]} == 0 )); then\n                hosts+=(\"'first@${host}.zib.de'\")\n            else\n                hosts+=(\"'node${port}@${host}.zib.de'\")\n            fi\n        done\n    done\n    FIRST=${hosts[0]}\n    HOSTLIST=$(join \"${hosts[@]}\")\n}\n\njoin() {\n    local IFS=\",\"\n    echo \"$*\"\n}\n\nwrite_config() {\n    local max_key=$((NODES*2**17))\n    local config=${WD}/${NAME}/lg${PARALLEL_ID}.config\n    cat >  $config <<EOF\n{rng_seed, $RANDOM_SEED}.\n{mode, $MODE}.\n{duration, $DURATION}.\n{concurrent, $WORKERS_PER_LG}.\n{operations, [{put,2}, {get, 8}]}.\n{driver, basho_bench_driver_scalaris}.\n{key_generator, {int_to_str, {uniform_int, $max_key}}}.\n%%{key_generator, {int_to_str, {uniform_int, 16#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF}}}.\n%% size in Bytes\n{value_generator, {fixed_bin, $VALUE_SIZE}}.\n{scalarisclient_mynode, ['benchclient${PARALLEL_ID}']}.\n{scalarisclient_cookie, 'chocolate chip cookie'}.\n\n{report_interval, 1}.\n{log_level, info}.\n\n{scalarisclient_nodes, [$HOSTLIST]}.\nEOF\n}\n\nrun_bbench() {\n    declare -a lg_pids # the process id's of the load generators\n    local no_of_hosts=${#LG_HOSTS[*]}\n    local c # counter for indexing the LG_HOSTS array\n\n    for i in $(seq 1 $LOAD_GENERATORS); do\n        PARALLEL_ID=$i\n        RANDOM_SEED=\"{$((7*$i)), $((11*$i)), $((5*$i))}\"\n        write_config\n\n        # build args. The ${var:+...} expands only, if the variable is set and non-empty\n        local arg1=${SLURM_JOBID:+\"--jobid=$SLURM_JOBID\"}\n        local arg2=${PARALLEL_ID:+\"--parallel_id=$PARALLEL_ID\"}\n        local arg3=${WD:+\"--wd=$WD\"}\n        local arg4=${NAME:+\"--name=$NAME\"}\n        local arg5=${BBENCH_DIR:+\"--bbdir=$BBENCH_DIR\"}\n        local arg6=${RESULT_DIR:+\"--rdir=$RESULT_DIR\"}\n        declare -a args=($arg1 $arg2 $arg3 $arg4 $arg5 $arg6)\n\n        # get current host and (post)increment counter\n        host=${LG_HOSTS[$((c++ % no_of_hosts))]}\n\n        if [[ $(hostname -f) = $host ]]; then\n            $SCALARIS_DIR/contrib/slurm/util/start-basho-bench.sh ${args[@]} &\n            lg_pids[$i]=$!\n        else\n            # using -t (pseudo-tty allocation) allows to terminate children of the\n            # ssh cmd at the remote node through kill the ssh process at the local node\n            if [[ $EXTERNAL_LG = true ]]; then\n                ssh -t -t $host $SCALARIS_DIR/contrib/slurm/util/start-basho-bench.sh ${args[@]} &\n            else\n                ARGSTRING=$(printf '%s ' \"${args[@]}\")\n                echo \"Starting LG on $host (Total nummber of LG hosts $no_of_hosts)\"\n                echo \"Argstring = $ARGSTRING\"\n                srun --nodelist=$host -N1 bash -c \"$SCALARIS_DIR/contrib/slurm/util/start-basho-bench.sh $ARGSTRING\" &\n            fi\n            lg_pids[$i]=$!\n        fi\n    done\n\n    # wait for load generators to finish\n    for pid in \"${lg_pids[@]}\"; do\n        wait $pid\n    done\n}\n\nis_lg_external() {\n    if  [ -n ${SLURM_NODELIST} ]; then\n        EXTERNAL_LG=false\n    else\n        EXTERNAL_LG=true\n    fi\n}\n\n\ncheck_result_dir() {\n    for host in ${LG_HOSTS[@]}; do\n        local res=0\n\n        if [[ $(hostname -f) = $host ]]; then\n            $SCALARIS_DIR/contrib/slurm/util/checkdir.sh $RESULT_DIR\n            res=$((res+=$?))\n        else\n            if [[ $EXTERNAL_LG = true ]]; then\n                ssh -t -t $host \"$SCALARIS_DIR/contrib/slurm/util/checkdir.sh $RESULT_DIR\"\n            else\n                srun --nodelist=$host -N1 bash -c \"$SCALARIS_DIR/contrib/slurm/util/checkdir.sh $RESULT_DIR\"\n            fi\n            res=$((res+=$?))\n        fi\n\n        if [[ $res -ne 0 ]]; then\n            log error \"Result dir ($RESULT_DIR) on $host not empty, aborting\"\n            exit 1\n        fi\n    done\n\n}\n\ncreate_result_dir() {\n    for host in ${LG_HOSTS[@]}; do\n        log info \"creating result dir on $host\"\n        if [[ $(hostname -f) = $host ]]; then\n            mkdir -p $RESULT_DIR/$NAME\n        else\n            if [[ $EXTERNAL_LG = true ]]; then\n                ssh -t -t $host \"bash -c \\\"mkdir -p $RESULT_DIR/$NAME\\\"\"\n            else\n                srun --nodelist=$host -N1 bash -c \"mkdir -p $RESULT_DIR/$NAME\"\n            fi\n        fi\n    done\n}\n\ncollect_bbench_results() {\n    for host in ${LG_HOSTS[@]}; do\n        log info \"collecting bbench results from $host\"\n        if [[ $(hostname -f) = $host ]]; then\n            rsync -ayhx --progress $RESULT_DIR/ $WD/\n            if [[ $? == 0 ]]; then\n                log info \"deleting $RESULT_DIR/$PREFIX* on $host\"\n                rm -r $RESULT_DIR/$PREFIX*\n            fi\n        else\n            if [[ $EXTERNAL_LG = true ]]; then\n                ssh -t -t $host \"bash -c \\\"rsync -ayhx --progress $RESULT_DIR/ $WD/\\\"\"\n                if [[ $? == 0 ]]; then\n                    log info \"deleting $RESULT_DIR/$PREFIX* on $host\"\n                    ssh -t -t $host \"bash -c \\\"rm -r $RESULT_DIR/$PREFIX*\\\"\"\n                fi\n            else\n                srun --nodelist=$host -N1 bash -c \"rsync -ayhx --progress $RESULT_DIR/ $WD/\"\n                if [[ $? == 0 ]]; then\n                    log info \"deleting $RESULT_DIR/$PREFIX* on $host\"\n                    srun --nodelist=$host -N1 bash -c \"rm -r $RESULT_DIR/$PREFIX*\"\n                fi\n            fi\n        fi\n    done\n}\n\nkill_bbench(){\n    log info \"killing bbench...\"\n\n    # kill all load generators (or their ssh processes respectively)\n    for pid in \"${lg_pids[@]}\"; do\n        kill $pid\n    done\n}\n\ntrap_cleanup(){\n    log info \"received SIGTERM, cleaning up...\"\n    kill_bbench\n    shutdown\n    # kill all remaining children of the current script\n    PGID=$(ps -o pgid= $$ | grep -o [0-9]*)\n    setsid kill -9 -- -$PGID\n    exit 1\n}\n\nshutdown(){\n    stop_scalaris\n    [[ $COLLECTL = true ]] && stop_collectl\n    [[ $TOPLOG = true ]] && stop_toplog\n    collect_bbench_results\n    rm_lockfile\n}\n\nrm_lockfile() {\n    # remove lockfile\n    local lockfile=\"${WD}/${SLURM_JOBID}.lock\"\n    rm -f $lockfile\n}\n\nmain\n"
  },
  {
    "path": "contrib/slurm/basho-bench.slurm",
    "content": "#!/bin/bash -l\n\n################################################################################\n#\n#   BASHO BENCH SLURM SCRIPT\n#\n#   Starts a Scalaris ring and sleeps for 365d, canceled through\n#   the basho-bench.sh script.\n#\n################################################################################\n\n#SBATCH -J scalaris\n#SBATCH -N 1\n#SBATCH -p CUMU\n#SBATCH -A csr\n#SBATCH --exclusive\n#SBATCH --time=00:10:00\n\n# setup Scalaris\nsource /usr/share/modules/init/bash\nsource $(pwd)/config/env.sh\nprint_env\necho \"Nodelist: $SLURM_NODELIST\"\n$(pwd)/util/start-scalaris.sh\n\n# check that the WD (working dir) is set\nif [[ -z $WD ]]; then\n    echo \"WD not set, stopping servers\"\n    $(pwd)/util/stop-scalaris.sh\n    exit 1\nfi\n\n# create the lock file\n# (indicates successfully started scalaris ring to basho-basho.sh)\necho \"scalaris started\" > $WD/${SLURM_JOBID}.lock\n\n\nSLEEPTIME=\"365d\"\necho \"sleeping for $SLEEPTIME, need to cancel manually\"\nsleep $SLEEPTIME\n\necho \"stopping servers\"\n$(pwd)/util/stop-scalaris.sh\necho \"stopped servers\"\n"
  },
  {
    "path": "contrib/slurm/basho_slurm_lg.slurm",
    "content": "#!/bin/bash -l\n\n# Use this sbatch script to start the bash-bench benchmarks whilst using\n# slurm nodes for load generation (calling basho-bench.sh directly will use\n# external load generators as defined by config/basho-bench.cfg)\n\n#SBATCH -J load_gen_basho\n#SBATCH -p CUMU\n#SBATCH -N 1\n#SBATCH -A csr\n#SBATCH -t 10:00:00\n#SBATCH --exclusive\n#SBATCH -o example.out\n\n# override of parameters in config/basho-bench.cfg\nexport LOAD_GENERATORS=1 # load generator are distributed over nodelist\n\nexport KIND='load'\nexport DURATION=2\nexport NODES=3 # number of scalaris nodes\nexport VMS_PER_NODE=1\nexport WORKERS_PER_LG_SERIES=\"1 2 4 8 16 32 64 128 256 512 1024 2028\"\nexport OPERATIONS_SERIES=\"[{put,0},{get,1}] [{put,1},{get,19}] [{put,1},{get,9}] [{put,1},{get,1}] [{put,1},{get,0}]\"\n\nexport WD=\"$CUMUSCRATCH/$USER/example_work_dir_dir\"\nexport RESULT_DIR=\"/local/$USER/example_local_dir\"\n\n\n$(pwd)/basho-bench.sh\n"
  },
  {
    "path": "contrib/slurm/cleanup.sh",
    "content": "#!/bin/bash\n\n# Script to clean up the remaining processes from Scalaris on a slurm node after\n# a job was canceled\n\nsource $(pwd)/env.sh\n\nHOSTNAME=\"$(hostname)\"\n\nfunction test_success() {\n    if [ $? -eq 0 ]; then\n        echo \"ok\"\n    else\n        echo \"failed\"\n    fi\n}\n\n# find all the screen sessions running a scalaris node and kill them\nSCALARIS_SESSIONS=$(screen -ls | grep Detached | grep scalaris_ | awk '{print $1}')\nif [[ -z $SCALARIS_SESSIONS ]]; then\n    echo \"[$HOSTNAME] no Scalaris sessions of $(whoami) running\"\nelse\n    echo \"[$HOSTNAME] Running Scalaris sessions:\"\n    for SESSION in $SCALARIS_SESSIONS; do\n        echo -e \"\\t$SESSION\"\n    done\n    echo -n \"[$HOSTNAME] killing scalaris session... \"\n    screen -ls | grep Detached | grep scalaris_ | cut -d. -f1 | awk '{print $1}' | xargs kill\n    test_success\nfi\n\n# kill the epmd\nEPMD=$(pgrep epmd)\nif [[ -z $EPMD ]]; then\n    echo \"[$HOSTNAME] no epmd running\"\nelse\n    echo -n \"[$HOSTNAME] epmd running, killing the epmd process... \"\n    kill $EPMD\n    test_success\nfi\n\ntest_foreign_beams\n\n\n\n\n"
  },
  {
    "path": "contrib/slurm/config/basho-bench.cfg",
    "content": "\n################################################################################\n#\n# Author: Jens V. Fischer\n# Date: 11.11.2015\n#\n# CONFIGURATION FOR basho-bench.sh\n#\n# Default configuration settings and documention for basho-bench.sh.\n#\n# All variables are only set, if the variable was unset or empty, i.e. variables\n# set before sourcing this config (e.g. in the quick-config section) or passed\n# with the call to the script are *not* overwritten.\n#\n# This file is sourced in basho-bench.sh -> main().\n#\n################################################################################\n\n#----------------------------\n# basho-bench.sh Configuration\n#----------------------------\n\n# Series of scalaris ring sizes to use.\n# If a list of values is given, bbench will be repeated for given every value.\n# NODES_SERIES=${NODES_SERIES:-\"2 4 8 16 32\"}\nNODES_SERIES=${NODES_SERIES:-\"4\"}\n\n# Series of VMs per node to use.\n# This determines the number of Erlang VMs which are started per slurm node.\n# If a list of values is given, bbench will be repeated for every given value.\n# VMS_PER_NODE_SERIES=${VMS_PER_NODE_SERIES:-\"01 02 04 08\"}\nVMS_PER_NODE_SERIES=${VMS_PER_NODE_SERIES:-\"1\"}\n\n# number of times the basho benchmark is repeated\nREPETITIONS=${REPETITIONS:-1}\n\n# number of workers per load generator\n# workers / dht_node = load_generators * workers_base\nWORKERS_PER_LG_SERIES=${WORKERS_PER_LG_SERIES:-4}\n\n# a load generator is a Erlang VM running a basho bench instance\n# there can be multiple load generators on one machine\n# load generators can also distributed to multiple machines\nLOAD_GENERATORS=${LOAD_GENERATORS:-2}\n\n# hosts for the load generators. LOAD_GENERATORS should be a multiple of number of hosts\nDEFAULT_HOSTS=( \"$(hostname -f)\" )\nLG_HOSTS=(${LG_HOSTS[@]:-${DEFAULT_HOSTS[@]}})\n\n# start collectl on the load generators and the scalaris nodes\n# output will be written to WD/collectl\nCOLLECTL=${COLLECTL:-false}\nCOLLECTL_SUBSYSTEMS=${COLLECTL_SUBSYSTEMS:-\"-s cCmMnNdD\"}\nCOLLECTL_INTERVAL=${COLLECTL_INTERVAL:-\"-i 10\"}\nCOLLECTL_FLUSH=${COLLECTL_FLUSH:-\"-F 0\"}\n\n# log top 5 processes (beware: ressource intensive)\nTOPLOG=${TOPLOG:-false}\n\n# Prefix for naming the basho bench folder\nPREFIX=${PREFIX:-\"results-\"}\n\n# The working directory for this script (logs, results and tmp files go here)\n# (Beware of file quotas for this directory, large benchmark series can produce\n# large amounts of (small) files.)\nWD=${WD:-\"$CUMUSCRATCH/$USER/bbench\"}\n\n# Local directory for the results of Basho Bench. Basho Bench will write files\n# here. After finishing the benchmarks (i.e. repetitions) of one level of the\n# independent variable (load, size, etc) the results are synced back to the WD\n# using rsync. This allows to write all the large amounts of small result files\n# to a non-NFS directory (preferably on a fast storage device) during the benchmark.\nRESULT_DIR=${RESULT_DIR:-\"/local/$USER/bbench\"}\n\n# directory of the scalaris repository\nSCALARIS_DIR=${SCALARIS_DIR:-\"$HOME/scalaris\"}\n\n# time two wait between benchmark repetitions\nSLEEP1=${SLEEP1:-5}\n\n# time two wait between different ring sizes\nSLEEP2=${SLEEP2:-1}\n\n\n#-------------------------------------------------------------------------------\n# Scalaris Configuration\n#\n# These are parameters passed to the scalaris or slurm scripts. Most of them are\n# set dynamically by basho-bench.sh (these are commented out), they are described\n# here for documentation purposes.\n#-------------------------------------------------------------------------------\n\n# Number of physical machines to allocate\nNODES=\"${NODES:-16}\"\n\n# A list of slurm nodes to allocate (has to be compatible with NODES)\n# NODELIST=\"cumu01-[00-15]\"\n\n# Erlang VMs per machine\nVMS_PER_NODE=\"${VMS_PER_NODE:-1}\"\n\n# DHT nodes per Erlang VM\nDHT_NODES_PER_VM=\"${DHT_NODES_PER_VM:-1}\"\n\n# Shuffle the ids instead of assigning them in ring order\nSHUFFLE_NODE_IDS=${SHUFFLE_NODE_IDS:-1}\n\n# Interval the watchdog (cleans up cancelled jobs) checks the job status\nWATCHDOG_INTERVAL=${WATCHDOG_INTERVAL:-5}\n\n# Flags for the Erlang Scheduler\n# ERL_SCHED_FLAGS=\"+S 2\"\nERL_SCHED_FLAGS=${ERL_SCHED_FLAGS:-\"\"}\n\n# Partition to use for slurm\nPARTITION=${PARTITION:-\"CUMU\"}\n\n# slurm timeout\nTIMEOUT=${TIMEOUT:-10}\n\n# directory for the collectl logs\n# COLLECTL_DIR=${COLLECTL_DIR:-$WD/collectl}\n\n# set collectl arguments\nCOLLECTL_SUBSYSTEMS=${COLLECTL_SUBSYSTEMS:-\"-s cmnd\"}\nCOLLECTL_INTERVAL=${COLLECTL_INTERVAL:-\"-i 10\"}\nCOLLECTL_FLUSH=${COLLECTL_FLUSH:-\"-F 0\"}\n\n# directory for the Scalaris source\nSCALARIS_SRC=${SCALARIS_SRC:-$HOME/scalaris}\n\n# set to true to run Scalaris in a local directory instead of the nfs volume\nSCALARIS_LOCAL=${SCALARIS_LOCAL:-false}\n\n#-------------------------------------------------------------------------------\n# Basho Bench Configuration\n#\n# These are parameters written to the basho bench configuration file. Most of\n# them are set dynamically by basho-bench.sh (these are commented out), they are\n# described here for documentation purposes.\n#-------------------------------------------------------------------------------\n\n# The directory of the basho benchmark installation\nBBENCH_DIR=${BBENCH_DIR:-\"$HOME/basho_bench\"}\n\n# Random seed for the basho bench\n# (usually dynnamically created per load generator)\n# RANDOM_SEED=\"{42, 23, 122}\"\n\n# mode (max or rate in operations/sec)\n# Beware: rate is per worker per load generator\n# total rate = # load generators * worker base * ringsize * rate\nMODE=${MODE:-\"{rate, max}\"}\nMODE=${MODE:-\"{rate, 150}\"}\n\n# duration in minutes\nDURATION=${DURATION:-1}\n\n# path for Scalaris' log files (\"-l\" param to scalarisctl)\n# ?NAME will be expanded to the name of the respective benchmark\n# the expanded string will be prepended to \"$SCALARISCTL_PARAMS\" and passed to\n# slurm scripts setting up Scalaris\nSLOGPATH=${SLOGPATH:-\"-l $WD/?NAME/logs\"}\n\n# place Scalaris logs in a local folder (to avoid potential bottlenecks due to NFS)\n# and tag with the datetime of the benchmark.\n# SLOGPATH=${SLOGPATH:-\"-l /local/$USER/slog/$(date +%y%m%d-%H%M)-?NAME\"}\n\n# additional params passed to the scalarisctl\n# don't use \"-l\" or \"-c\", see above\nSCTL_PARAMS=${SCTL_PARAMS:-\"\"}\n\n# value size in bytes\nVALUE_SIZE=${VALUE_SIZE:-512}\n"
  },
  {
    "path": "contrib/slurm/config/env.sh",
    "content": "export SCALARIS_LOCAL=${SCALARIS_LOCAL:-false}\nexport SCALARIS_SRC=${SCALARIS_SRC:-$HOME/scalaris}\n# exports EPMD=... with to the Erlang version set through ./configure\nexport $(grep 'EPMD=' $SCALARIS_SRC/bin/scalarisctl)\nif [[ $SCALARIS_LOCAL = true ]]; then\n    export SCALARIS_DIR=\"/local/$(whoami)/scalaris\"\nelse\n    export SCALARIS_DIR=${SCALARIS_DIR:-$HOME/scalaris}\nfi\nexport VMS_PER_NODE=${VMS_PER_NODE:-1}\nexport DHT_NODES_PER_VM=${DHT_NODES_PER_VM:-1}\nexport ERL_SCHED_FLAGS=${ERL_SCHED_FLAGS:-\"\"}\nexport SCALARISCTL_PARAMS=${SCALARISCTL_PARAMS:-\"\"} # additional params for scalarisctl\n\nexport COLLECTL=${COLLECTL:-\"false\"}\n\nexport ETCDIR=$SCALARIS_DIR/bin\nexport BINDIR=$SCALARIS_DIR/bin\nexport BEAMDIR=$SCALARIS_DIR/ebin\nexport COLLECTL_DIR=${COLLECTL_DIR:-\"$(pwd)/collectl/\"}\n\n# collectl arguments\nexport COLLECTL_SUBSYSTEMS=${COLLECTL_SUBSYSTEMS:-\"-s cmnd\"}\nexport COLLECTL_INTERVAL=${COLLECTL_INTERVAL:-\"-i 10\"}\nexport COLLECTL_FLUSH=${COLLECTL_FLUSH:-\"-F 0\"}\n\nexport SHUFFLE_NODE_IDS=1\nexport WATCHDOG_INTERVAL=10\n\nfunction cleanup() {\n    echo \"Nodelist of the cancelled job: $SLURM_NODELIST\"\n    echo -e \"Use:\n    srun -p CUMU -A csr --nodelist='$SLURM_NODELIST' cleanup.sh\"\n    echo \"to clean up the nodes manually\"\n\n    # comment in for automatic cleanup\n    # for NODE in $NODELIST; do\n        # sbatch --job-name cleanup -p CUMU -A csr --nodelist=\"$NODE\" -o cleanup-%j.out cleanup.sh\n    # done\n    exit 1\n}\n\nfunction test_foreign_beams() {\n    BEAM=$(pgrep -a beam)\n    if [[ -n $BEAM ]]; then\n        USER=$(ps -e -o user,comm | grep beam | awk '{print $1}' | sort | uniq | xargs echo)\n        echo \"There are Erlang VMs from $USER still running, please contact $USER for cleanup:\"\n        echo \"$(ps -e -o user,pid,start_time,comm | awk 'NR == 1 {print} /beam/ {print}')\"\n        echo \"pgrep -a beam output: $BEAM\"\n        echo \"sleeping for 30 seconds, then retrying...\"\n        sleep 30\n    fi\n    BEAM=$(pgrep -a beam)\n    if [[ -n $BEAM ]]; then\n        USER=$(ps -e -o user,comm | grep beam | awk '{print $1}' | sort | uniq | xargs echo)\n        echo \"There are Erlang VMs from $USER still running, please contact $USER for cleanup:\"\n        echo \"$(ps -e -o user,pid,start_time,comm | awk 'NR == 1 {print} /beam/ {print}')\"\n        echo \"pgrep -a beam output: $BEAM\"\n        return 1\n    fi\n    return 0\n}\n\nfunction print_env() {\n    echo \"Nodes: $(($SLURM_JOB_NUM_NODES*$VMS_PER_NODE*$DHT_NODES_PER_VM))($SLURM_JOB_NUM_NODES*$VMS_PER_NODE*$DHT_NODES_PER_VM)\"\n    echo \"erl schedular flags: $ERL_SCHED_FLAGS\"\n    erl_binary=$(grep \"^ERL=\" $SCALARIS_DIR/bin/scalarisctl | awk -F= '{print $2}')\n    erl_version_file=$($erl_binary -eval 'io:format(\"~s\", [filename:join([code:root_dir(), \"releases\", erlang:system_info(otp_release), \"OTP_VERSION\"])]), halt()'  -noshell)\n    erl_version=$(cat $erl_version_file)\n    echo \"Erlang binary: $erl_binary\"\n    echo \"Erlang version: $erl_version\"\n    HEAD=$(git rev-parse --short HEAD)\n    echo \"Current git-HEAD: $HEAD\"\n    echo \"Slurm-JOBID: $SLURM_JOB_ID\"\n    echo \"Number of DHT Nodes: $(($SLURM_JOB_NUM_NODES*$VMS_PER_NODE)) (Nodes: $SLURM_JOB_NUM_NODES; VMs per Node: $VMS_PER_NODE)\"\n    echo \"SCALARIS_DIR=$SCALARIS_DIR\"\n    if [[ -n $WD && -n $NAME ]]; then\n        printenv > $WD/$NAME/slurm-${SLURM_JOB_ID}.env\n    else\n        printenv > slurm-${SLURM_JOB_ID}.env\n    fi\n}\n\ncheck_compile(){\n    pushd $SCALARIS_DIR >/dev/null\n    local res=$(erl -pa contrib/yaws -pa ebin -noinput +B -eval 'R=make:all([noexec]), halt(0).')\n    popd >/dev/null\n    if [[ -n $res ]]; then\n        echo \"Scalaris binaries do not match source version:\"\n        echo $res\n        echo \"exiting...\"\n        exit 1\n    fi\n}\n\nexport -f cleanup\nexport -f test_foreign_beams\nexport -f print_env\nexport -f check_compile\n\ntrap cleanup SIGTERM\n\n"
  },
  {
    "path": "contrib/slurm/count-channels.slurm",
    "content": "#!/bin/bash -l\n\n# -o: output log file: %j for the job ID, %N for the name of the first executing node\n# Change the path of the output logfile\n\n#SBATCH -J scalaris\n#SBATCH -N 2\n#SBATCH -p CSR\n#SBATCH -A csr\n#SBATCH --exclusive\n\nsource /usr/share/modules/init/bash\nsource $(pwd)/config/env.sh\n\n#$BINDIR/scalarisctl checkinstallation\n\necho $(date)\n$(pwd)/util/start-scalaris.sh\n\n#############################################\n#                                           #\n#     Place your commands between here      #\n#                                           #\n#############################################\n\necho \"Nodelist: $SLURM_NODELIST\"\nsleep 3\nHEAD=$(git rev-parse --short HEAD)\nJOBID=$SLURM_JOB_ID\nNO_OF_NODES=$SLURM_JOB_NUM_NODES\necho \"current HEAD: $HEAD\"\n\nfor i in {1..8}; do\n    KEYS=1\n    ITERS=1\n    echo \"$(date) running bench:increment($KEYS, $ITERS)...\"\n    METRICS=\"{value, {mean_troughput_overall, Mean}} = lists:keysearch(mean_troughput_overall, 1, Res), {value, {avg_latency_overall, Latency}} = lists:keysearch(avg_latency_overall, 1, Res)\"\n    LOGSTRING_INC=\"io:format('result data inc:~p:~p~n', [Mean, Latency])\"\n    erl -setcookie \"chocolate chip cookie\" -name bench_ -noinput -eval \"{ok, Res} = rpc:call('first@`hostname -f`', bench, increment, [$KEYS, $ITERS]), $METRICS, $LOGSTRING_INC, halt(0).\"\n    sleep 1\ndone\n\nsleep 3\n\necho \"$(date) no of channels\"\n\necho \"HEAD; JOBID; NO_OF_NODES; VMS_PER_NODE; PID; NO_OF_CH\"\n# erl -setcookie \"chocolate chip cookie\" -name bench_ -noinput -eval \"N = rpc:call('first@`hostname -f`', comm_stats, get_no_of_ch, []), io:format('number of channels: ~w~n', [N]), halt(0).\"\nerl -setcookie \"chocolate chip cookie\" -name bench_ -noinput -eval \"{no_of_channels, CommServer, NoOfCh} = rpc:call('first@`hostname -f`', comm_stats, get_no_of_ch, []), io:format('$HEAD; $JOBID; $NO_OF_NODES; $VMS_PER_NODE; ~w; ~w;~n', [CommServer, NoOfCh]), halt(0).\"\nPORT=14196\nfor TASKSPERNODE in `seq 2 $VMS_PER_NODE`; do\n    erl -setcookie \"chocolate chip cookie\" -name bench_ -noinput -eval \"{no_of_channels, CommServer, NoOfCh} = rpc:call('node$PORT@`hostname -f`', comm_stats, get_no_of_ch, []), io:format('$HEAD; $JOBID; $NO_OF_NODES; $VMS_PER_NODE; ~w; ~w;~n', [CommServer, NoOfCh]), halt(0).\"\n    let PORT+=1\ndone\n\nTAILNODES=`scontrol show hostnames | tail -n +2`\nfor NODE in $TAILNODES; do\n    PORT=14195\n    for TASKSPERNODE in `seq 1 $VMS_PER_NODE`; do\n        # erl -setcookie \"chocolate chip cookie\" -name bench_ -noinput -eval \"N = rpc:call('node$PORT@$NODE.zib.de', comm_stats, get_no_of_ch, []), io:format('number of channels: ~w~n', [N]), halt(0).\"\n        erl -setcookie \"chocolate chip cookie\" -name bench_ -noinput -eval \"{no_of_channels, CommServer, NoOfCh} = rpc:call('node$PORT@$NODE.zib.de', comm_stats, get_no_of_ch, []), io:format('$HEAD; $JOBID; $NO_OF_NODES; $VMS_PER_NODE; ~w; ~w;~n', [CommServer, NoOfCh]), halt(0).\"\n        let PORT+=1\n    done\ndone\n\n\n#############################################\n#                                           #\n#     and here                              #\n#                                           #\n#############################################\n\necho \"stopping servers\"\n$(pwd)/util/stop-scalaris.sh\necho \"stopped servers\"\n"
  },
  {
    "path": "contrib/slurm/example-job-script.slurm",
    "content": "#!/bin/bash -l\n\n# -o: output log file: %j for the job ID, %N for the name of the first executing node\n# Change the path of the output logfile\n\n#SBATCH -J bench-script\n#SBATCH -N 2\n#SBATCH -p CUMU\n#SBATCH -A csr\n#SBATCH -t 00:03:00\n#SBATCH --exclusive\n\nsource $(pwd)/config/env.sh\n\n#$BINDIR/scalarisctl checkinstallation\ncheck_compile\n$(pwd)/util/start-scalaris.sh\n\n#############################################\n#                                           #\n#     Place your commands between here      #\n#                                           #\n#############################################\n\nsleep 15\n\n\n#############################################\n#                                           #\n#     and here                              #\n#                                           #\n#############################################\n\n$(pwd)/util/stop-scalaris.sh\n"
  },
  {
    "path": "contrib/slurm/increment-bench.sh",
    "content": "#!/bin/bash\n\nfor NNODES in \"2\" #  1 2 3 4 5 6 7 8 12 16 24 32\ndo\n    for VMS_PER_NODE in \"2\" #  1 2 4 8\n    do\n        for DHT_NODES_PER_VM in \"1\" #  2 4 8\n        do\n            export VMS_PER_NODE=$VMS_PER_NODE\n            export DHT_NODES_PER_VM=$DHT_NODES_PER_VM\n            for ITERATION in `seq 1 20`\n            do\n                #sbatch --dependency=singleton --job-name scalaris-benchmark -N $NNODES increment-bench.slurm\n                sbatch --dependency=singleton --job-name scalaris-benchmark -N $NNODES -o slurm-$NNODES-$VMS_PER_NODE-$DHT_NODES_PER_VM-%j.out increment-bench.slurm\n            done\n        done\n    done\ndone\n"
  },
  {
    "path": "contrib/slurm/increment-bench.slurm",
    "content": "#!/bin/bash\n\n# -o: output log file: %j for the job ID, %N for the name of the first executing node\n# Change the path of the output logfile\n\n#SBATCH -J bench-script\n#SBATCH -N 2\n#SBATCH -p CUMU\n#SBATCH -A csr\n#SBATCH --time=00:10:00\n#SBATCH --exclusive\n\nsource /usr/share/modules/init/bash\nsource $(pwd)/config/env.sh\n\ncheck_compile\nprint_env\n$(pwd)/util/start-scalaris.sh\n\n#############################################\n#                                           #\n#     Place your commands between here      #\n#                                           #\n#############################################\n\nMETRICS=\"{value, {mean_troughput_overall, Mean}} = lists:keysearch(mean_troughput_overall, 1, Res), {value, {avg_latency_overall, Latency}} = lists:keysearch(avg_latency_overall, 1, Res)\"\nLOGSTRING_INC=\"io:format('result data inc:~p:~p~n', [Mean, Latency])\"\nLOGSTRING_QR=\"io:format('result data qr:~p:~p~n', [Mean, Latency])\"\n\nTHREADS=1024\nITERATIONS=4\necho \"running bench:increment($THREADS, $ITERATIONS)...\"\nerl -setcookie \"chocolate chip cookie\" -name bench_ -noinput -eval \"{ok, Res} = rpc:call('first@`hostname -f`', bench, increment, [$THREADS, $ITERATIONS]), $METRICS, $LOGSTRING_INC, halt(0).\"\n\nTHREADS=2048\nITERATIONS=10\necho \"running bench:increment($THREADS, $ITERATIONS)...\"\nerl -setcookie \"chocolate chip cookie\" -name bench_ -noinput -eval \"{ok, Res} = rpc:call('first@`hostname -f`', bench, quorum_read, [$THREADS, $ITERATIONS]), $METRICS, $LOGSTRING_QR, halt(0).\"\n\n#############################################\n#                                           #\n#     and here                              #\n#                                           #\n#############################################\n\necho \"stopping servers\"\n$(pwd)/util/stop-scalaris.sh\n"
  },
  {
    "path": "contrib/slurm/util/checkdir.sh",
    "content": "#!/bin/bash\n\nWD=$1\nif [[ -z $WD ]]; then\n    exit 1\nfi\n\nif [ \"$(ls $WD)\" ]; then\n    echo \"[error] Directory ($WD) on $(hostname -s) is not empty, containing the following files/dirs:\"\n    ls -l1 $WD\n    exit 1\nfi\n\n"
  },
  {
    "path": "contrib/slurm/util/collect_crashdumps.sh",
    "content": "#!/bin/bash\n\n###############################################################################\n# Author: Jens V. Fischer\n#\n# Sync all crashdumps from the local direcotry to the given subdirectory.\n#\n# Call:\n# \t./collect_crashdumps.sh <jobid> <dir>\n#\n###############################################################################\n\njobid=${1?\"no jobid\"}\n# subdir to put the crashdumps in\ncsub_dir=${2?\"no dir\"}\n\n# local dir (on the slurm nodes) of the scalaris installation\nexport SLOCAL_DIR=\"/local/bzcfisch\"\n\n# base dir for where to sync the crash dumps\ncbase_dir=\"/scratch/bzcfisch\"\n\nexport CDIR=\"$cbase_dir/$csub_dir\"\n\nnodelist=$(echo cumu{01,02}-{00..15})\n\nfor node in $nodelist; do\n    srun --jobid=$jobid -A csr -p CUMU -N1 --nodelist=$node bash <<'EOF'\necho \"$(hostname)\"\nrsync -ayhx --progress --executability $SLOCAL_DIR/scalaris/ebin/erl_crash.dump $CDIR/$(hostname -s)_crash.dump\necho \"rsync finished with exit code $?\"\nEOF\ndone\n\n"
  },
  {
    "path": "contrib/slurm/util/crashreport.awk",
    "content": "# Get all crashreports including reports spreading multiple lines\n\nBEGIN {\n    crashreport=0\n    timestamp_pattern=\"^20[[:digit:]]{2}-[[:digit:]]{2}-[[:digit:]]{2}\"\n}\n\n# FILENAME == last_filename { last_filename=FILENAME}\n# FILENAME != last_filename { last_filename=FILENAME; print FILENAME}\n\ncrashreport == 1 && $0 ~ timestamp_pattern { crashreport = 0; print \"\"} # leave crash report\ncrashreport == 0 && /FD/ { crashreport = 1; print $0 } # enter crash report\ncrashreport == 1 && $0 !~ timestamp_pattern { print \"\\t\", $0 } # during crash report\n"
  },
  {
    "path": "contrib/slurm/util/crashwatcher.sh",
    "content": "#!/bin/bash\n\n###############################################################################\n# Author: Jens V. Fischer\n#\n# Watch for crash reports in the Scalaris log files. Send kill -SIGUSR1 to all\n# Scalaris nodes to create a crashdump. Scalaris should run in a local dir,\n# otherwise all nodes try to write into the same crashdump.\n#\n# Call:\n# \t./crashwatcher <jobid> <no_of_nodes>\n#\n###############################################################################\n\n\nJOBID=$1\nNODES=$2\n\nWD=\"$HOME/bbench\"\n\n[[ -z $JOBID ]] && { echo \"no jobid, exiting...\"; exit 1; }\necho \"JOBID: $JOBID\"\n\n[[ -z $NODES ]] && { echo \"no number of nodes given, exiting...\"; exit 1; }\necho \"NODES: $NODES\"\n\nwatch() {\n    find $WD -iname scalaris_log4erl\\* | xargs grep crash\n}\n\nkill_bbench(){\n    # kill all remaining basho_bench processes\n    ssh bzcfisch@buildbot2.zib.de bash -c \"'pkill -f basho_bench'\"\n    pkill -f basho_bench\n}\n\nwait_for_crash() {\n    echo -n \"waiting for crash notice in Scalaris log files\"\n    timer=0\n    until watch; do\n        ((timer++))\n        # display status every 5 seconds\n        if ((timer%5==0)); then\n            echo -ne \".\"\n        fi\n        sleep 1\n    done\n    echo \": ok (${timer}s)\"\n}\n\nsend_sigusr1() {\n    echo \"sending SIGUSR1\"\n    srun --jobid=$JOBID -p CUMU -A csr -N $NODES bash <<'EOF'\necho -n \"$(hostname). \"\nps -e -o user,pid,start_time,comm | grep beam | awk '{print $2}' | xargs -r kill -SIGUSR1\necho \"ret: $?\"\nEOF\n}\n\nwait_for_crash\nkill_bbench\n\n# sleeptime=\"10m\"\n# echo \"$(date +%y.%m.%d-%H:%M:%S): sleeping for $sleeptime\"\n# sleep $sleeptime\n\nsend_sigusr1\n\n"
  },
  {
    "path": "contrib/slurm/util/start-basho-bench.sh",
    "content": "#!/bin/bash\n\nmain() {\n    parse_args \"$@\"\n\n    # prepend tag to every output (stdout and stderr)\n    TAG=\"[lg_${PARALELL_ID}]\"\n    exec &> >( while read line; do echo \"${TAG} ${line}\"; done)\n\n    # check_running\n    start_bbench\n}\n\nparse_args() {\n    for i in \"$@\"; do\n        case $i in\n            --jobid=*)\n                # get only the part after the \"=\" with bash substring removal\n                JOBID=\"${i#*=}\"\n                shift\n                ;;\n            --wd=*)\n                WD=\"${i#*=}\"\n                shift\n                ;;\n            --rdir=*)\n                RESULT_DIR=\"${i#*=}\"\n                shift\n                ;;\n            --bbdir=*)\n                BBENCH_DIR=\"${i#*=}\"\n                shift\n                ;;\n            --vms_per_node=*)\n                VMS_PER_NODE=\"${i#*=}\"\n                shift\n                ;;\n            --parallel_id=*)\n                PARALELL_ID=\"${i#*=}\"\n                shift\n                ;;\n            --name=*)\n                BBENCH_NAME=\"${i#*=}\"\n                shift\n                ;;\n            *)\n                echo \"unknown option, exiting\" # unknown option\n                exit 1\n                ;;\n        esac\n    done\n    [[ -z $JOBID ]] && { log error \"--jobid not specified\"; res=$((ret+=1)); }\n    [[ -z $WD ]] && { log error \"--wd not specified\"; res=$((ret+=1)); }\n    [[ -z $RESULT_DIR ]] && { log error \"--rdir not specified\"; res=$((ret+=1)); }\n    [[ -z $BBENCH_DIR ]] && { log error \"--bbdir not specified\"; res=$((ret+=1)); }\n    [[ -z $BBENCH_NAME ]] && { log error \"name not specified\"; res=$((ret+=1)); }\n    [[ -z $PARALELL_ID ]] && { log error \"--parallel_id not specified\"; res=$((ret+=1)); }\n    ((res>0)) && { log error \"parameter missing, exiting\"; exit 1; }\n}\n\ncheck_running(){\n    # check for running bbench instances\n    ppids=$(pgrep -af basho_bench | grep -v $JOBID)\n    if [[ -n $ppids ]]; then\n        echo \"[error] benchmarks still running, exiting...\"\n        pgrep -af basho_bench | grep -v $JOBID\n        exit 1\n    fi\n}\n\nstart_bbench() {\n    local config=\"${WD}/${BBENCH_NAME}/lg${PARALELL_ID}.config\"\n    $BBENCH_DIR/basho_bench -n lg${PARALELL_ID} -N vm${PARALELL_ID} \\\n        -C 'chocolate chip cookie' --results-dir $RESULT_DIR/$BBENCH_NAME $config\n}\n\nlog(){\n    local level=$1\n    local message=$2\n    printf \"[bbench] %s  [%s] %s\\n\" \"$(date +%H:%M:%S)\" \"$level\" \"$message\"\n}\n\nmain \"$@\"\n\n"
  },
  {
    "path": "contrib/slurm/util/start-scalaris.sh",
    "content": "#!/bin/bash\n\nfunction fix_known_hosts() {\n    # When using a local dir, the known hosts need to be written to the source dir,\n    # not the local dir. They are then synced to the local dir.\n    local old_etcdir=$ETCDIR\n    ETCDIR=$SCALARIS_SRC/bin\n\n    let NR_OF_NODES=$SLURM_JOB_NUM_NODES\\*$VMS_PER_NODE\n    if [ -e $ETCDIR/scalaris.local.cfg ]\n    then\n        mv $ETCDIR/scalaris.local.cfg .\n    else\n        touch scalaris.local.cfg\n    fi\n    NODEIDX=1\n    echo \"{known_hosts, [\" >> $ETCDIR/scalaris.local.cfg\n    for NODE in `scontrol show hostnames`; do\n        PORT=14195\n        for TASKSPERNODE in `seq 1 $VMS_PER_NODE`; do\n            IP=`host $NODE | cut -d ' ' -f 4`\n            echo -n \"{{\" >> $ETCDIR/scalaris.local.cfg\n            echo -n $IP | sed s/\\\\./\\,/g >> $ETCDIR/scalaris.local.cfg\n            echo -n \"},$PORT,service_per_vm}\" >> $ETCDIR/scalaris.local.cfg\n            if [ \"$NODEIDX\" -ne \"$NR_OF_NODES\" ]\n            then\n                echo \",\" >> $ETCDIR/scalaris.local.cfg\n            fi\n            let PORT+=1\n            let NODEIDX+=1\n        done\n    done\n    echo \"]}.\" >> $ETCDIR/scalaris.local.cfg\n\n    ## fix mgmt_server\n    HEADNODE=`scontrol show hostnames | head -n1`\n    echo -n \"{mgmt_server, {{\" >> $ETCDIR/scalaris.local.cfg\n    IP=`host $HEADNODE | cut -d ' ' -f 4`\n    echo -n $IP | sed s/\\\\./\\,/g >> $ETCDIR/scalaris.local.cfg\n    echo \"}, 14195, mgmt_server}}.\" >> $ETCDIR/scalaris.local.cfg\n\n    # restore old ETCDIR\n    ETCDIR=$old_etcdir\n}\n\nfunction kill_old_nodes() {\n    srun -N$SLURM_JOB_NUM_NODES bash -c \"screen -ls | grep Detached | grep scalaris_ | cut -d. -f1 | awk '{print $1}' | xargs -r kill\"\n    test_foreign_beams\n    if [[ $? -ne 0 ]]; then\n        scancel $SLURM_JOBID\n    fi\n}\n\nfunction sync_scalaris_dir() {\n    srun -N $SLURM_NNODES ./util/sync_scalaris_to_local_dir.sh\n}\n\nfunction start_servers() {\n    HEADNODE=`scontrol show hostnames | head -n1`\n    TAILNODES=`scontrol show hostnames | tail -n +2`\n\n    let NR_OF_DHT_NODES=$SLURM_JOB_NUM_NODES\\*$VMS_PER_NODE\\*$DHT_NODES_PER_VM\n    let NR_OF_VMS=$SLURM_JOB_NUM_NODES\\*$VMS_PER_NODE\n\n    KEYLIST=\"\"\n    if [ $SHUFFLE_NODE_IDS -eq 1 ]\n    then\n        KEYLIST=`erl -name bench_ -pa $BEAMDIR -noinput -eval \"L = util:lists_split(util:shuffle(api_dht_raw:split_ring($NR_OF_DHT_NODES)), $NR_OF_VMS), io:format('~p', [L]), halt(0).\"`\n    else\n        KEYLIST=`erl -name bench_ -pa $BEAMDIR -noinput -eval \"L = util:lists_split(api_dht_raw:split_ring($NR_OF_DHT_NODES), $NR_OF_VMS), io:format('~p', [L]), halt(0).\"`\n    fi\n    export KEYLIST # for start-vm.sh\n\n    VM_IDX=1\n    JOIN_KEYS=`erl -name bench_ -noinput -eval \"L = lists:nth($VM_IDX, $KEYLIST), io:format('~p', [L]), halt(0).\"`\n    # start first node on head node\n    $BINDIR/scalarisctl -c \"chocolate chip cookie\" -j \"$JOIN_KEYS\" -n first -p 14195 -y 8000 --nodes-per-vm $DHT_NODES_PER_VM --screen -d -m -t first  ${SCALARISCTL_PARAMS:+$SCALARISCTL_PARAMS} start\n    let VM_IDX+=1\n\n    ## @todo use auto-binding\n    # start vms at all the tail nodes\n    srun -k -r1 -N$((SLURM_NNODES-1)) --cpu_bind=none --ntasks-per-node=${VMS_PER_NODE} ./util/start-vm.sh\n\n    # start remaining VMs on head node\n    PORT=14196\n    YAWSPORT=8001\n    for TASKSPERNODE in `seq 2 $VMS_PER_NODE`; do\n        JOIN_KEYS=`erl -name bench_ -noinput -eval \"L = lists:nth($VM_IDX, $KEYLIST), io:format('~p', [L]), halt(0).\"`\n        $BINDIR/scalarisctl -c \"chocolate chip cookie\" -j \"$JOIN_KEYS\" -n node$PORT -p $PORT -y $YAWSPORT --nodes-per-vm $DHT_NODES_PER_VM --screen -d -t joining ${SCALARISCTL_PARAMS:+$SCALARISCTL_PARAMS} start\n        let VM_IDX+=1\n        let PORT+=1\n        let YAWSPORT+=1\n    done\n}\n\nfunction wait_for_servers_to_start {\n    let NR_OF_NODES=$SLURM_JOB_NUM_NODES\\*$VMS_PER_NODE\\*$DHT_NODES_PER_VM\n    for NODE in `scontrol show hostnames`; do\n        RUNNING_NODES=`srun --nodelist=$NODE -N1 --ntasks-per-node=1 $EPMD -names | grep \" at port \" | wc -l`\n        while [ $RUNNING_NODES -ne $VMS_PER_NODE ]\n        do\n            RUNNING_NODES=`srun --nodelist=$NODE -N1 --ntasks-per-node=1 $EPMD -names | grep \" at port \" | wc -l`\n        done\n    done\n\n    # wait for the first VM to start\n    NR_OF_FIRSTS=`$EPMD -names | grep 'name first at port' | wc -l`\n    while [ $NR_OF_FIRSTS -ne 1 ]\n    do\n        NR_OF_FIRSTS=`$EPMD -names | grep 'name first at port' | wc -l`\n    done\n    # wait for the first VM to initialize\n    erl -setcookie \"chocolate chip cookie\" -name bench_ -noinput -eval \"A = rpc:call('first@`hostname -f`', api_vm, wait_for_scalaris_to_start, []), io:format('waited for scalaris: ~p~n', [A]), halt(0).\"\n    # wait for the ring to stabilize\n    erl -setcookie \"chocolate chip cookie\" -name bench_ -noinput -eval \"A = rpc:call('first@`hostname -f`', admin, wait_for_stable_ring, [$NR_OF_NODES]), io:format('waited for the ring: ~p~n', [A]), halt(0).\"\n}\n\nfunction start_watchdog() {\n    # start watchdog\n    srun -N$SLURM_NNODES --nodelist=$SLURM_JOB_NODELIST screen -S scalaris_watchdog_${SLURM_JOBID} -d -m ./util/watchdog.sh\n}\n\nfunction start_collectl(){\n    # start collectl on all allocated nodes\n\n    if [[ ! -d $COLLECTL_DIR ]]; then\n        # create directory if necessary\n        mkdir -p $COLLECTL_DIR\n    fi\n\n    # collectl will be started in a screen session which will be cleaned up by the watchdog\n    srun -N$SLURM_NNODES screen -S \"scalaris_collectl_SLURM_JOBID_${SLURM_JOBID}\" -d -m \\\n        bash -c \"collectl $COLLECTL_SUBSYSTEMS $COLLECTL_INTERVAL $COLLECTL_FLUSH -f $COLLECTL_DIR; sleep 365d\"\n}\n\nfix_known_hosts\n[[ $SCALARIS_LOCAL = true ]] && sync_scalaris_dir\nkill_old_nodes\nstart_watchdog\n[[ $COLLECTL = true ]] && start_collectl\nd1=$(date '+%s')\nstart_servers\nwait_for_servers_to_start\nd2=$(date '+%s')\necho \"starting $(($SLURM_JOB_NUM_NODES*$VMS_PER_NODE*$DHT_NODES_PER_VM))($SLURM_JOB_NUM_NODES*$VMS_PER_NODE*$DHT_NODES_PER_VM) nodes took $((d2-d1)) seconds\"\n"
  },
  {
    "path": "contrib/slurm/util/start-vm.sh",
    "content": "#!/bin/bash\n\n# DHT_NODES_PER_VM, VMS_PER_NODE, KEYLIST inherited from environment\nBASE_PORT=14195\nBASE_YAWSPORT=8000\nBASE_VM_IDX=$VMS_PER_NODE\nLOCAL_OFFSET=${SLURM_LOCALID} # Node local task ID for the process within a job\nGLOBAL_OFFSET=${SLURM_PROCID} # relative process ID of the current process, 0 based\n\nstart_vm(){\n    # uncomment (incl line 24) to restrict the number of schedulers for one node only\n    # local old_erl_sched_flags=$ERL_SCHED_FLAGS\n    # [[ $GLOBAL_OFFSET = 0 ]] && ERL_SCHED_FLAGS=\"+S 1\"\n\n    # Delay joining depending on the local offset. Necessary for starting very large rings.\n    local sleeptime=$((LOCAL_OFFSET*2))\n    sleep $sleeptime\n\n    VM_IDX=$((BASE_VM_IDX+GLOBAL_OFFSET+1))\n    JOIN_KEYS=`erl -name bench_${GLOBAL_OFFSET} -noinput -eval \"L = lists:nth($VM_IDX, $KEYLIST), io:format('~p', [L]), halt(0).\"`\n    PORT=$((BASE_PORT+LOCAL_OFFSET))\n    YAWSPORT=$((BASE_YAWSPORT+LOCAL_OFFSET))\n    $BINDIR/scalarisctl -c \"chocolate chip cookie\" -j \"$JOIN_KEYS\" -n node$PORT -p $PORT -y $YAWSPORT --nodes-per-vm $DHT_NODES_PER_VM --screen -d -t joining ${SCALARISCTL_PARAMS:+$SCALARISCTL_PARAMS} start\n    # ERL_SCHED_FLAGS=$old_erl_sched_flags\n}\n\nstart_vm\n"
  },
  {
    "path": "contrib/slurm/util/stop-scalaris.sh",
    "content": "#!/bin/bash\n\nsrun -N$SLURM_JOB_NUM_NODES  --ntasks-per-node=1 bash -c \"screen -ls | grep Detached | grep scalaris_ | grep \"SLURM_JOBID_${SLURM_JOB_ID}\" | cut -d. -f1 | awk '{print $1}' | xargs -r kill\"\nmv scalaris.local.cfg $ETCDIR\n"
  },
  {
    "path": "contrib/slurm/util/sync_scalaris_to_local_dir.sh",
    "content": "#!/bin/bash\n\n# SCALARIS_SRC and SCALARIS_DIR inherited from environment\n\necho \"syncing scalaris to $SCALARIS_DIR on $(hostname -f)\"\n[[ -d $SCALARIS_DIR ]] || mkdir -p $SCALARIS_DIR\nrsync -ayhxq --executability --delete-after $SCALARIS_SRC/ $SCALARIS_DIR/\n(( $? == 0 )) || exit 1\n"
  },
  {
    "path": "contrib/slurm/util/toplog.sh",
    "content": "#!/bin/bash\n\n# echo \"PID: $$\"\n\n# set -x\nLOG=\"$(hostname -s)_top.log\"\nSLEEP=1\n\nif [[ -z $? ]]; then\n    echo \"[error] No basedir as first argument, exiting\"\n    exit 1\nelse\n    BASEDIR=$1\nfi\n\n\nwhile true; do\n    echo \"$(date +%y.%m.%d-%H:%M:%S):\" >> $BASEDIR/$LOG\n    top -bn1 | awk 'NR>6 && NR <12{print $0}' >> $BASEDIR/$LOG\n    echo \"\" >> $BASEDIR/$LOG\n    sleep 5\ndone\n"
  },
  {
    "path": "contrib/slurm/util/watchdog.sh",
    "content": "#!/bin/bash\n\nfunction cleanup_node(){\n    screen -ls | grep Detached | grep scalaris_ | grep \"SLURM_JOBID_${SLURM_JOB_ID}\" | cut -d. -f1 | awk '{print $1}' | xargs -r kill\n\n    return 0\n}\n\nIS_RUNNING=(\"RUNNING\")\nSACCT_RES=$(sacct -j $SLURM_JOB_ID -b)\nif [[ $? -eq 0 ]]; then\n    IS_RUNNING=($(echo \"$SACCT_RES\" | grep $SLURM_JOB_ID | awk '{print $2}'))\nfi\nwhile [[ ${IS_RUNNING[0]} == \"RUNNING\" ]] || [[ ${IS_RUNNING[0]} == \"PENDING\" ]] ; do\n    sleep $WATCHDOG_INTERVAL\n    SACCT_RES=$(sacct -j $SLURM_JOB_ID -b)\n    if [[ $? -eq 0 ]]; then\n        IS_RUNNING=($(echo \"$SACCT_RES\" | grep $SLURM_JOB_ID | awk '{print $2}'))\n    fi\ndone\n\ncleanup_node\n\n"
  },
  {
    "path": "contrib/systemd/.gitignore",
    "content": "/*.service\n/scalaris.pp\n/tmp\n"
  },
  {
    "path": "contrib/systemd/README",
    "content": "NOTE:\nFor a full SELinux policy, the typical Scalaris ports need to be added to the scalaris_port_t type:\n\nsemanage port -a -t scalaris_port_t -p tcp 14194-14198\nsemanage port -a -t scalaris_port_t -p tcp 8000-8004\n"
  },
  {
    "path": "contrib/systemd/scalaris-first.conf",
    "content": "SCALARIS_NODE=node\nSCALARIS_ADDITIONAL_PARAMETERS=-p 14195 -y 8000 -m -t joining\n"
  },
  {
    "path": "contrib/systemd/scalaris.conf",
    "content": "SCALARIS_NODE=node1\nSCALARIS_ADDITIONAL_PARAMETERS=-p 14196 -y 8001 -t first\n"
  },
  {
    "path": "contrib/systemd/scalaris.fc",
    "content": "/var/lib/scalaris(/.*)?  gen_context(system_u:object_r:scalaris_file_t,s0)\n"
  },
  {
    "path": "contrib/systemd/scalaris.if",
    "content": "## <summary>Scalaris node</summary>\n"
  },
  {
    "path": "contrib/systemd/scalaris.service.in",
    "content": "[Unit]\nDescription=Scalaris node\nRequires=remote-fs.target\nAfter=remote-fs.target\n\n[Service]\nEnvironmentFile=@sysconfdir@/conf.d/scalaris\nExecStart=@bindir@/scalarisctl -n ${SCALARIS_NODE} -e \"-noshell\" -l @localstatedir@/log/scalaris $SCALARIS_ADDITIONAL_PARAMETERS start\nExecStop=@bindir@/scalarisctl -n ${SCALARIS_NODE} -e \"-noshell\" -l @localstatedir@/log/scalaris $SCALARIS_ADDITIONAL_PARAMETERS gstop\n\nWorkingDirectory=@localstatedir@/log/scalaris\n\nUser=scalaris\nGroup=scalaris\n"
  },
  {
    "path": "contrib/systemd/scalaris.te",
    "content": "module scalaris 0.9.0_git;\n\nrequire {\n        attribute port_type;\n        attribute file_type;\n        type initrc_var_log_t;\n        type rabbitmq_beam_t;\n        type var_t;\n        type var_log_t;\n        type initrc_var_log_t;\n        type http_port_t;\n        class tcp_socket { name_bind name_connect };\n        class dir { getattr search read write add_name create };\n        class file { getattr setattr read write open create };\n}\n\ntype scalaris_port_t, port_type;\ntype scalaris_file_t, file_type;\n\n#============= rabbitmq_beam_t ==============\nallow rabbitmq_beam_t scalaris_port_t:tcp_socket name_bind;\nallow rabbitmq_beam_t scalaris_port_t:tcp_socket name_connect;\n\nallow rabbitmq_beam_t initrc_var_log_t:dir getattr;\nallow rabbitmq_beam_t var_t:dir read;\nallow rabbitmq_beam_t var_log_t:dir { getattr search read write add_name create };\nallow rabbitmq_beam_t var_log_t:file { getattr read write open create };\nallow rabbitmq_beam_t initrc_var_log_t:dir { getattr search read write add_name create };\nallow rabbitmq_beam_t initrc_var_log_t:file { getattr read write open create };\nallow rabbitmq_beam_t scalaris_file_t:dir { getattr search read write add_name create };\nallow rabbitmq_beam_t scalaris_file_t:file { getattr setattr read write open create };\n"
  },
  {
    "path": "contrib/wikipedia/.gitignore",
    "content": "/bin\n/.settings\n/.classpath\n/.project\n/*pages-articles.xml*\n/*pages-articles*.db\n/*pages-meta-history.xml*\n/*pages-meta-history*.db\n/*pages-meta-current.xml*\n/*pages-meta-current*.db\n/scalaris-wiki.war\n/scalaris-wiki-plugin-api.jar\n/.ant-targets-build.xml\n/build\n/demo*.txt\n/*.log\n"
  },
  {
    "path": "contrib/wikipedia/README",
    "content": "== QUICKSTART ==\n\n1) run 'ant download' to get a wikipedia dump of the simple wiktionary (or download a dump yourself)\n2) start at least one Scalaris node, e.g. by running '../../bin/scalarisctl -f -s -d start' (for a deamonized first node)\n3) import the dump using either of the two following options:\n3a) run 'ant import-xml' and follow the instructions to import data into Scalaris\n3b) run 'ant prepare' to create a prepared dump file for faster dump loads and then use 'ant import-db' with the created DB to import data into Scalaris\n4) run 'ant run' and follow the instructions to get a webserver up and running\n\n== Creating a scalaris-wiki.war with pre-defined config ==\n\n1) place your desired scalaris.properties in simplewiki/WEB-INF/\n2) run 'ant dist'\n\nNote: Properties inside simplewiki/WEB-INF/scalaris.properties overwrite config parameters provided during 'ant run'!\n"
  },
  {
    "path": "contrib/wikipedia/build.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<project basedir=\".\" default=\"build\" name=\"WikiOnScalaris\">\n  <!-- version of the OtpErlang jar file to bundle and compile against (OtpErlang-<version>.jar) -->\n  <property name=\"opterlang.version\" value=\"1.6.1\" />\n  <!-- debug level to use during compilation -->\n  <property name=\"debuglevel\" value=\"source,lines,vars\" />\n\n  <!-- All libraries needed to run the import task -->\n  <path id=\"import.classpath\">\n    <pathelement location=\"contrib/scalaris.jar\"/>\n    <pathelement location=\"contrib/OtpErlang-${opterlang.version}.jar\"/>\n    <pathelement location=\"contrib/commons-compress-1.6.jar\"/>\n    <pathelement location=\"contrib/xz-1.4.jar\"/>\n    <pathelement location=\"contrib/commons-codec-1.6.jar\"/>\n    <pathelement location=\"contrib/commons-lang-2.6.jar\"/>\n    <pathelement location=\"contrib/commons-logging-1.1.1.jar\"/>\n    <pathelement location=\"contrib/bliki-core-3.0.20-SNAPSHOT.jar\"/>\n    <pathelement location=\"contrib/java-bloomfilter-1.0.jar\"/>\n    <pathelement location=\"contrib/sqlite4java.jar\"/>\n  </path>\n\n  <property name=\"tomcat.home\" value=\"contrib/apache-tomcat\" />\n\n  <!-- All libraries needed to build the sources -->\n  <path id=\"build.classpath\">\n    <path refid=\"import.classpath\"/>\n    <pathelement location=\"${tomcat.home}/lib/servlet-api.jar\"/>\n  </path>\n\n  <!-- Create build directory and copy static files -->\n  <macrodef name=\"scalaris.check.jars\">\n    <sequential>\n      <condition property=\"scalaris.jars.available.contrib\">\n        <and>\n          <available file=\"contrib/scalaris.jar\" type=\"file\" />\n          <available file=\"contrib/OtpErlang-${opterlang.version}.jar\" type=\"file\" />\n        </and>\n      </condition>\n      <condition property=\"scalaris.src.available\">\n        <available file=\"../../java-api/build.xml\" type=\"file\" />\n      </condition>\n      <condition property=\"scalaris.src.jars.available\">\n        <and>\n          <available file=\"../../java-api/scalaris.jar\" type=\"file\" />\n          <available file=\"../../java-api/lib/OtpErlang-${opterlang.version}.jar\" type=\"file\" />\n        </and>\n      </condition>\n    </sequential>\n  </macrodef>\n  <target name=\"scalaris.check.jars\">\n    <scalaris.check.jars />\n  </target>\n  <target name=\"scalaris.build.jars\" depends=\"scalaris.check.jars\" if=\"scalaris.src.available\">\n    <ant dir=\"../../java-api/\"/>\n    <scalaris.check.jars />\n  </target>\n  <target name=\"scalaris.get.jars\" depends=\"scalaris.check.jars,scalaris.build.jars\" if=\"scalaris.src.available\">\n    <fail unless=\"scalaris.src.jars.available\" message=\"missing ../../java-api/scalaris.jar and ../../java-api/lib/OtpErlang-${opterlang.version}.jar\" />\n    <copy todir=\"contrib\" flatten=\"true\" preservelastmodified=\"true\">\n      <fileset dir=\"../../java-api/\">\n        <include name=\"scalaris.jar\" />\n        <include name=\"lib/OtpErlang-${opterlang.version}.jar\" />\n      </fileset>\n    </copy>\n    <scalaris.check.jars />\n  </target>\n  <target name=\"scalaris.update.jars\" depends=\"scalaris.get.jars\">\n    <scalaris.check.jars />\n    <fail unless=\"scalaris.jars.available.contrib\" message=\"missing scalaris.jar and OtpErlang-${opterlang.version}.jar and no sources found in ../../java-api/\" />\n  </target>\n\n  <target name=\"init\" depends=\"scalaris.update.jars\">\n    <mkdir dir=\"scalaris-wiki/WEB-INF/classes\"/>\n    <mkdir dir=\"scalaris-wiki/WEB-INF/lib\"/>\n    <copy todir=\"scalaris-wiki/WEB-INF/lib\" flatten=\"true\" preservelastmodified=\"true\">\n      <fileset dir=\"contrib\" includes=\"*.jar\"/>\n      <fileset dir=\"contrib\" includes=\"*.so\"/>\n      <fileset dir=\"contrib\" includes=\"*.jnilib\"/>\n      <fileset dir=\"contrib\" includes=\"*.dll\"/>\n    </copy>\n  </target>\n\n  <!-- Compile Servlets, Beans and other java files -->\n  <target name=\"build\" depends=\"init\">\n    <echo message=\"Download Wiktionary dump:                'ant download'\"/>\n    <echo message=\"Import dump into Scalaris (interactive): 'ant import-xml'\"/>\n    <echo message=\"Import dump into Scalaris (batch):       'ant import-xml -Ddata=&lt;file&gt; -Dmax_revisions=&lt;number&gt; -Dmin_time=&lt;date&gt; -Dmax_time=&lt;date&gt; -Dscalaris.node=&lt;node&gt; -Dscalaris.cookie=&lt;cookie&gt;'\"/>\n    <echo message=\"Import prepared DB into Scalaris (int.): 'ant import-db'\"/>\n    <echo message=\"Import prepared DB into Scalaris (batch):'ant import-db' -Ddata=&lt;file&gt; -Dnumber_of_importers=&lt;number&gt; -Dmy_import_number=&lt;number&gt; -Dweb_xml=&lt;patch to web.xml&gt; -Dscalaris.node=&lt;node&gt; -Dscalaris.cookie=&lt;cookie&gt;\"/>\n    <echo message=\"Run tomcat web server:                   'ant run [-Dscalaris.node=&lt;node&gt;] [-Dscalaris.cookie=&lt;cookie&gt;]'\"/>\n    <javac encoding=\"UTF-8\" debug=\"true\" debuglevel=\"${debuglevel}\" destdir=\"scalaris-wiki/WEB-INF/classes\" includeAntRuntime=\"false\">\n      <src path=\"src\"/>\n      <classpath refid=\"build.classpath\"/>\n    </javac>\n  </target>\n\n  <!-- Deletes the generated class files and the temporary files (compiled classes, jsps, logs,...) -->\n  <target name=\"clean\">\n    <delete dir=\"scalaris-wiki/WEB-INF/classes\"/>\n    <delete dir=\"scalaris-wiki/WEB-INF/lib\"/>\n    <delete file=\"scalaris-wiki.war\"/>\n    <delete file=\"scalaris-wiki-plugin-api.jar\"/>\n    <delete>\n      <fileset dir=\"contrib\" includes=\"scalaris.jar, OtpErlang-*.jar\"/>\n    </delete>\n  </target>\n  \n  <target name=\"run\" depends=\"run-tomcat\" />\n\n  <!-- starts a tomcat server which provides the wiki website -->\n  <target name=\"run-tomcat\" depends=\"build,scalaris.properties\">\n    <java classname=\"org.apache.catalina.startup.Bootstrap\" fork=\"true\">\n      <classpath>\n        <pathelement location=\"${tomcat.home}/bin/bootstrap.jar\"/>\n        <pathelement location=\"${tomcat.home}/bin/tomcat-juli.jar\"/>\n      </classpath>\n      <sysproperty key=\"catalina.home\" value=\"${tomcat.home}\" />\n      <sysproperty key=\"scalaris.node\" value=\"${scalaris.node}\" />\n      <sysproperty key=\"scalaris.cookie\" value=\"${scalaris.cookie}\" />\n    </java>\n  </target>\n\n  <target name=\"import.get.data_xml_sql\" unless=\"data\">\n    <input addproperty=\"data\" defaultvalue=\"simplewiktionary-latest-pages-meta-history.xml.bz2\" message=\"Wikipedia dump file (*.xml, *.xml.gz, *.xml.bz2, *.db)\"/>\n  </target>\n\n  <target name=\"import.get.data_xml\" unless=\"data\">\n    <input addproperty=\"data\" defaultvalue=\"simplewiktionary-latest-pages-meta-history.xml.bz2\" message=\"Wikipedia dump file (*.xml, *.xml.gz, *.xml.bz2)\"/>\n  </target>\n\n  <target name=\"import.get.data_sql\" unless=\"data\">\n    <input addproperty=\"data\" defaultvalue=\"simplewiktionary-latest-pages-meta-history.xml.db\" message=\"Wikipedia dump file (*.db)\"/>\n  </target>\n\n  <target name=\"import.get.allowed_pages\" unless=\"allowed_pages\">\n    <input addproperty=\"allowed_pages\" defaultvalue=\"\" message=\"Text file with additional page titles to include (one per line)\"/>\n  </target>\n\n  <target name=\"import.get.whitelist\" unless=\"whitelist\">\n    <input addproperty=\"whitelist\" defaultvalue=\"\" message=\"Text file with page titles to import (one per line)\"/>\n  </target>\n\n  <target name=\"import.get.max_revisions\" unless=\"max_revisions\">\n    <input addproperty=\"max_revisions\" defaultvalue=\"2\" message=\"Maximum number of revisions per page starting with the most recent one (-1 for all revisions)\"/>\n  </target>\n\n  <target name=\"import.get.min_time\" unless=\"min_time\">\n    <input addproperty=\"min_time\" defaultvalue=\"\" message=\"Import at most one revision older than this date in ISO8601 format, e.g. 2004-01-07T08:09:29Z, (empty for all revisions)\"/>\n  </target>\n\n  <target name=\"import.get.max_time\" unless=\"max_time\">\n    <input addproperty=\"max_time\" defaultvalue=\"\" message=\"Maximum date of revisions to import in ISO8601 format, e.g. 2004-01-07T08:09:29Z, (empty for all revisions)\"/>\n  </target>\n\n  <target name=\"import.get.recursion_lvl\" unless=\"recursion_lvl\">\n    <input addproperty=\"recursion_lvl\" defaultvalue=\"1\" message=\"Recursively import pages this deep\"/>\n  </target>\n\n  <target name=\"import.get.category1\" unless=\"category1\">\n    <input addproperty=\"category1\" defaultvalue=\"\" message=\"Import pages in this category\"/>\n  </target>\n\n  <target name=\"import.get.category2\" unless=\"category2\">\n    <input addproperty=\"category2\" defaultvalue=\"\" message=\"Import pages in this category\"/>\n  </target>\n\n  <target name=\"import.get.category3\" unless=\"category3\">\n    <input addproperty=\"category3\" defaultvalue=\"\" message=\"Import pages in this category\"/>\n  </target>\n  \n  <property name=\"category4\" value=\"\" />\n  <property name=\"category5\" value=\"\" />\n  <property name=\"category6\" value=\"\" />\n  <property name=\"category7\" value=\"\" />\n  <property name=\"category8\" value=\"\" />\n  <property name=\"category9\" value=\"\" />\n  <property name=\"category10\" value=\"\" />\n\n  <target name=\"import.get.filtered_pagelist\" unless=\"filtered_pagelist\">\n    <input addproperty=\"filtered_pagelist\" defaultvalue=\"${data}-filtered_pagelist.txt\" message=\"Write filtered pagelist to this file (will be created or overwritten)\"/>\n  </target>\n\n  <target name=\"import.get.prepare_db\" unless=\"prepare_db\">\n    <input addproperty=\"prepare_db\" defaultvalue=\"${data}-1.db\" message=\"Write pre-processed K/V pairs to this database (file must not exist yet!)\"/>\n  </target>\n\n  <target name=\"download\">\n    <get usetimestamp=\"true\" dest=\"simplewiktionary-latest-pages-meta-history.xml.bz2\" src=\"http://dumps.wikimedia.org/simplewiktionary/latest/simplewiktionary-latest-pages-meta-history.xml.bz2\"/>\n  </target>\n\n  <target name=\"scalaris.properties.get.node\" unless=\"scalaris.node\">\n    <input addproperty=\"scalaris.node\" defaultvalue=\"firstnode@localhost,node1@localhost,node2@localhost,node3@localhost,node4@localhost\" message=\"URL to a scalaris node\"/>\n  </target>\n  <target name=\"scalaris.properties.get.cookie\" unless=\"scalaris.cookie\">\n    <input addproperty=\"scalaris.cookie\" defaultvalue=\"chocolate chip cookie\" message=\"Cookie the scalaris node uses for connections\"/>\n  </target>\n\n  <target name=\"scalaris.properties\" depends=\"scalaris.properties.get.node,scalaris.properties.get.cookie\" />\n\n  <target name=\"import-xml\" depends=\"build,import.get.data_xml_sql,import.get.max_revisions,import.get.min_time,import.get.max_time,import.get.whitelist,scalaris.properties\">\n    <fail unless=\"data\">You must specify the data property for 'ant import-xml'.</fail>\n    <fail unless=\"max_revisions\">You must specify the max_revisions property for 'ant import-xml'.</fail>\n    <fail unless=\"max_time\">You must specify the max_time property for 'ant import-xml'.</fail>\n    <fail unless=\"whitelist\">You must specify the whitelist property for 'ant import-xml'.</fail>\n    <java classname=\"de.zib.scalaris.examples.wikipedia.data.xml.Main\" failonerror=\"true\">\n      <sysproperty key=\"scalaris.node\" value=\"${scalaris.node}\" />\n      <sysproperty key=\"scalaris.cookie\" value=\"${scalaris.cookie}\" />\n      <arg value=\"${data}\"/>\n      <arg value=\"import-xml\"/>\n      <arg value=\"${max_revisions}\"/>\n      <arg value=\"${min_time}\"/>\n      <arg value=\"${max_time}\"/>\n      <arg value=\"${whitelist}\"/>\n      <classpath>\n        <pathelement location=\"scalaris-wiki/WEB-INF/classes\"/>\n        <path refid=\"import.classpath\"/>\n        <pathelement path=\"${java.class.path}\"/>\n      </classpath>\n    </java>\n  </target>\n\n  <target name=\"import.get.number_of_importers\" unless=\"number_of_importers\">\n    <input addproperty=\"number_of_importers\" defaultvalue=\"1\" message=\"Total number of (independent) import jobs\"/>\n  </target>\n\n  <target name=\"import.get.my_import_number\" unless=\"my_import_number\">\n    <input addproperty=\"my_import_number\" defaultvalue=\"1\" message=\"This import's job number (1 &lt;= my_import_number &lt;= number_of_importers)\"/>\n  </target>\n\n  <target name=\"import.get.web_xml\" unless=\"web_xml\">\n    <input addproperty=\"web_xml\" defaultvalue=\"\" message=\"Path to web.xml containing servlet optimisation options (leave empty for no DB conversion)\"/>\n  </target>\n  \n  <target name=\"import-db\" depends=\"build,import.get.data_sql,import.get.number_of_importers,import.get.my_import_number,import.get.web_xml,scalaris.properties\">\n    <fail unless=\"data\">You must specify the data property for 'ant import-db'.</fail>\n    <fail unless=\"number_of_importers\">You must specify the number_of_importers property for 'ant import-db'.</fail>\n    <fail unless=\"my_import_number\">You must specify the my_import_number property for 'ant import-db'.</fail>\n    <fail unless=\"web_xml\">You must specify the web_xml property for 'ant import-db'.</fail>\n    <java classname=\"de.zib.scalaris.examples.wikipedia.data.xml.Main\" failonerror=\"true\">\n      <sysproperty key=\"scalaris.node\" value=\"${scalaris.node}\" />\n      <sysproperty key=\"scalaris.cookie\" value=\"${scalaris.cookie}\" />\n      <arg value=\"${data}\"/>\n      <arg value=\"import-db\"/>\n      <arg value=\"${number_of_importers}\"/>\n      <arg value=\"${my_import_number}\"/>\n      <arg value=\"${web_xml}\"/>\n      <classpath>\n        <pathelement location=\"scalaris-wiki/WEB-INF/classes\"/>\n        <path refid=\"import.classpath\"/>\n        <pathelement path=\"${java.class.path}\"/>\n      </classpath>\n    </java>\n  </target>\n  \n  <target name=\"prepare\" depends=\"build,import.get.data_xml,import.get.max_revisions,import.get.min_time,import.get.max_time,import.get.whitelist,import.get.prepare_db\">\n    <fail unless=\"data\">You must specify the data property for 'ant prepare'.</fail>\n    <fail unless=\"max_revisions\">You must specify the max_revisions property for 'ant prepare'.</fail>\n    <fail unless=\"max_time\">You must specify the max_time property for 'ant prepare'.</fail>\n    <fail unless=\"whitelist\">You must specify the whitelist property for 'ant prepare'.</fail>\n    <fail unless=\"prepare_db\">You must specify the prepare_db property for 'ant prepare'.</fail>\n    <java classname=\"de.zib.scalaris.examples.wikipedia.data.xml.Main\" failonerror=\"true\">\n      <arg value=\"${data}\"/>\n      <arg value=\"prepare\"/>\n      <arg value=\"${max_revisions}\"/>\n      <arg value=\"${min_time}\"/>\n      <arg value=\"${max_time}\"/>\n      <arg value=\"${whitelist}\"/>\n      <arg value=\"${prepare_db}\"/>\n      <classpath>\n        <pathelement location=\"scalaris-wiki/WEB-INF/classes\"/>\n        <path refid=\"import.classpath\"/>\n        <pathelement path=\"${java.class.path}\"/>\n      </classpath>\n    </java>\n  </target>\n  \n  <target name=\"xml2db\" depends=\"build,import.get.data_xml,import.get.max_revisions,import.get.min_time,import.get.max_time,import.get.whitelist,import.get.prepare_db\">\n    <fail unless=\"data\">You must specify the data property for 'ant xml2db'.</fail>\n    <fail unless=\"max_revisions\">You must specify the max_revisions property for 'ant xml2db'.</fail>\n    <fail unless=\"max_time\">You must specify the max_time property for 'ant xml2db'.</fail>\n    <fail unless=\"whitelist\">You must specify the whitelist property for 'ant xml2db'.</fail>\n    <fail unless=\"prepare_db\">You must specify the prepare_db property for 'ant xml2db'.</fail>\n    <java classname=\"de.zib.scalaris.examples.wikipedia.data.xml.Main\" failonerror=\"true\">\n      <arg value=\"${data}\"/>\n      <arg value=\"xml2db\"/>\n      <arg value=\"${max_revisions}\"/>\n      <arg value=\"${min_time}\"/>\n      <arg value=\"${max_time}\"/>\n      <arg value=\"${whitelist}\"/>\n      <arg value=\"${prepare_db}\"/>\n      <classpath>\n        <pathelement location=\"scalaris-wiki/WEB-INF/classes\"/>\n        <path refid=\"import.classpath\"/>\n        <pathelement path=\"${java.class.path}\"/>\n      </classpath>\n    </java>\n  </target>\n  \n  <target name=\"dumpdb-addlinks\" depends=\"build,import.get.data_sql\">\n    <fail unless=\"data\">You must specify the data property for 'ant dumpdb-addlinks'.</fail>\n    <java classname=\"de.zib.scalaris.examples.wikipedia.data.xml.Main\" failonerror=\"true\">\n      <arg value=\"${data}\"/>\n      <arg value=\"dumpdb-addlinks\"/>\n      <classpath>\n        <pathelement location=\"scalaris-wiki/WEB-INF/classes\"/>\n        <path refid=\"import.classpath\"/>\n        <pathelement path=\"${java.class.path}\"/>\n      </classpath>\n    </java>\n  </target>\n\n  <target name=\"dumpdb-filter\" depends=\"build,import.get.data_sql,import.get.recursion_lvl,import.get.filtered_pagelist,import.get.allowed_pages,import.get.category1,import.get.category2,import.get.category3\">\n    <fail unless=\"data\">You must specify the data property for 'ant dumpdb-filter'.</fail>\n    <fail unless=\"filtered_pagelist\">You must specify the filtered_pagelist property for 'ant dumpdb-filter'.</fail>\n    <fail unless=\"allowed_pages\">You must specify the allowed_pages property for 'ant dumpdb-filter'.</fail>\n    <fail unless=\"category1\">You must specify the category1 property for 'ant dumpdb-filter'.</fail>\n    <fail unless=\"category2\">You must specify the category2 property for 'ant dumpdb-filter'.</fail>\n    <fail unless=\"category3\">You must specify the category3 property for 'ant dumpdb-filter'.</fail>\n    <java classname=\"de.zib.scalaris.examples.wikipedia.data.xml.Main\" failonerror=\"true\">\n      <sysproperty key=\"scalaris.node\" value=\"${scalaris.node}\" />\n      <sysproperty key=\"scalaris.cookie\" value=\"${scalaris.cookie}\" />\n      <arg value=\"${data}\"/>\n      <arg value=\"dumpdb-filter\"/>\n      <arg value=\"${recursion_lvl}\"/>\n      <arg value=\"${filtered_pagelist}\"/>\n      <arg value=\"${allowed_pages}\"/>\n      <arg value=\"${category1}\"/>\n      <arg value=\"${category2}\"/>\n      <arg value=\"${category3}\"/>\n      <arg value=\"${category4}\"/>\n      <arg value=\"${category5}\"/>\n      <arg value=\"${category6}\"/>\n      <arg value=\"${category7}\"/>\n      <arg value=\"${category8}\"/>\n      <arg value=\"${category9}\"/>\n      <arg value=\"${category10}\"/>\n      <classpath>\n        <pathelement location=\"scalaris-wiki/WEB-INF/classes\"/>\n        <path refid=\"import.classpath\"/>\n        <pathelement path=\"${java.class.path}\"/>\n      </classpath>\n    </java>\n  </target>\n\n  <target name=\"filter\" depends=\"build,import.get.data_xml,import.get.recursion_lvl,import.get.max_time,import.get.filtered_pagelist,import.get.allowed_pages,import.get.category1,import.get.category2,import.get.category3\">\n    <fail unless=\"data\">You must specify the data property for 'ant filter'.</fail>\n    <fail unless=\"max_time\">You must specify the max_time property for 'ant filter'.</fail>\n    <fail unless=\"filtered_pagelist\">You must specify the filtered_pagelist property for 'ant filter'.</fail>\n    <fail unless=\"allowed_pages\">You must specify the allowed_pages property for 'ant filter'.</fail>\n    <fail unless=\"category1\">You must specify the category1 property for 'ant filter'.</fail>\n    <fail unless=\"category2\">You must specify the category2 property for 'ant filter'.</fail>\n    <fail unless=\"category3\">You must specify the category3 property for 'ant filter'.</fail>\n    <java classname=\"de.zib.scalaris.examples.wikipedia.data.xml.Main\" failonerror=\"true\">\n      <sysproperty key=\"scalaris.node\" value=\"${scalaris.node}\" />\n      <sysproperty key=\"scalaris.cookie\" value=\"${scalaris.cookie}\" />\n      <arg value=\"${data}\"/>\n      <arg value=\"filter\"/>\n      <arg value=\"${recursion_lvl}\"/>\n      <arg value=\"${max_time}\"/>\n      <arg value=\"${filtered_pagelist}\"/>\n      <arg value=\"${allowed_pages}\"/>\n      <arg value=\"${category1}\"/>\n      <arg value=\"${category2}\"/>\n      <arg value=\"${category3}\"/>\n      <arg value=\"${category4}\"/>\n      <arg value=\"${category5}\"/>\n      <arg value=\"${category6}\"/>\n      <arg value=\"${category7}\"/>\n      <arg value=\"${category8}\"/>\n      <arg value=\"${category9}\"/>\n      <arg value=\"${category10}\"/>\n      <classpath>\n        <pathelement location=\"scalaris-wiki/WEB-INF/classes\"/>\n        <path refid=\"import.classpath\"/>\n        <pathelement path=\"${java.class.path}\"/>\n      </classpath>\n    </java>\n  </target>\n\n  <target name=\"import.get.db_read\" unless=\"db_read\">\n    <input addproperty=\"db_read\" defaultvalue=\"simplewiktionary-latest-pages-meta-history.xml-1.db\" message=\"prepared SQLite dump file to read from\"/>\n  </target>\n\n  <target name=\"import.get.db_write\" unless=\"db_write\" depends=\"import.get.db_read\">\n    <input addproperty=\"db_write\" defaultvalue=\"${db_read}-new.db\" message=\"prepared SQLite dump file to write to\"/>\n  </target>\n\n  <target name=\"import.get.db_write_options\" unless=\"db_write_options\">\n    <input addproperty=\"db_write_options\" defaultvalue=\"ALL:APPEND_INCREMENT|PAGE_LIST:APPEND_INCREMENT_BUCKETS_WITH_HASH(10)\" message=\"new optimisation scheme\"/>\n  </target>\n  \n  <target name=\"convert\" depends=\"build,import.get.db_read,import.get.db_write,import.get.db_write_options\">\n    <fail unless=\"db_read\">You must specify the data property for 'ant convert'.</fail>\n    <fail unless=\"db_write\">You must specify the db_write property for 'ant convert'.</fail>\n    <fail unless=\"db_write_options\">You must specify the db_write_options property for 'ant convert'.</fail>\n    <java classname=\"de.zib.scalaris.examples.wikipedia.data.xml.Main\" failonerror=\"true\">\n      <arg value=\"${db_read}\"/>\n      <arg value=\"convert\"/>\n      <arg value=\"${db_write}\"/>\n      <arg value=\"${db_write_options}\"/>\n      <classpath>\n        <pathelement location=\"scalaris-wiki/WEB-INF/classes\"/>\n        <path refid=\"import.classpath\"/>\n        <pathelement path=\"${java.class.path}\"/>\n      </classpath>\n    </java>\n  </target>\n\n  <target name=\"dist\" depends=\"build\">\n    <jar jarfile=\"scalaris-wiki.war\" update=\"false\">\n      <fileset dir=\"scalaris-wiki\" excludes=\"WEB-INF/dumps/**, **/.gitignore\" />\n      <fileset dir=\"scalaris-wiki\" includes=\"WEB-INF/dumps/empty-*.xml\" />\n    </jar>\n  </target>\n\n  <target name=\"plugin-api\" depends=\"build\">\n    <jar jarfile=\"scalaris-wiki-plugin-api.jar\" update=\"false\">\n      <fileset dir=\"scalaris-wiki/WEB-INF/classes\"\n        includes=\"de/zib/scalaris/examples/wikipedia/*.class,\n        de/zib/scalaris/examples/wikipedia/data/*.class,\n        de/zib/scalaris/examples/wikipedia/plugin/*.class\"\n      excludes=\"de/zib/scalaris/examples/wikipedia/ScalarisDataHandler*.class,\n        de/zib/scalaris/examples/wikipedia/plugin/PluginClassLoader*.class\"/>\n    </jar>\n  </target>\n</project>\n"
  },
  {
    "path": "contrib/wikipedia/contrib/.gitignore",
    "content": "/scalaris.jar\n/OtpErlang-*.jar\n/info.bliki.wiki\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/.gitignore",
    "content": "/logs\n/work\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/LICENSE",
    "content": "\n                                 Apache License\n                           Version 2.0, January 2004\n                        http://www.apache.org/licenses/\n\n   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n   1. Definitions.\n\n      \"License\" shall mean the terms and conditions for use, reproduction,\n      and distribution as defined by Sections 1 through 9 of this document.\n\n      \"Licensor\" shall mean the copyright owner or entity authorized by\n      the copyright owner that is granting the License.\n\n      \"Legal Entity\" shall mean the union of the acting entity and all\n      other entities that control, are controlled by, or are under common\n      control with that entity. For the purposes of this definition,\n      \"control\" means (i) the power, direct or indirect, to cause the\n      direction or management of such entity, whether by contract or\n      otherwise, or (ii) ownership of fifty percent (50%) or more of the\n      outstanding shares, or (iii) beneficial ownership of such entity.\n\n      \"You\" (or \"Your\") shall mean an individual or Legal Entity\n      exercising permissions granted by this License.\n\n      \"Source\" form shall mean the preferred form for making modifications,\n      including but not limited to software source code, documentation\n      source, and configuration files.\n\n      \"Object\" form shall mean any form resulting from mechanical\n      transformation or translation of a Source form, including but\n      not limited to compiled object code, generated documentation,\n      and conversions to other media types.\n\n      \"Work\" shall mean the work of authorship, whether in Source or\n      Object form, made available under the License, as indicated by a\n      copyright notice that is included in or attached to the work\n      (an example is provided in the Appendix below).\n\n      \"Derivative Works\" shall mean any work, whether in Source or Object\n      form, that is based on (or derived from) the Work and for which the\n      editorial revisions, annotations, elaborations, or other modifications\n      represent, as a whole, an original work of authorship. For the purposes\n      of this License, Derivative Works shall not include works that remain\n      separable from, or merely link (or bind by name) to the interfaces of,\n      the Work and Derivative Works thereof.\n\n      \"Contribution\" shall mean any work of authorship, including\n      the original version of the Work and any modifications or additions\n      to that Work or Derivative Works thereof, that is intentionally\n      submitted to Licensor for inclusion in the Work by the copyright owner\n      or by an individual or Legal Entity authorized to submit on behalf of\n      the copyright owner. For the purposes of this definition, \"submitted\"\n      means any form of electronic, verbal, or written communication sent\n      to the Licensor or its representatives, including but not limited to\n      communication on electronic mailing lists, source code control systems,\n      and issue tracking systems that are managed by, or on behalf of, the\n      Licensor for the purpose of discussing and improving the Work, but\n      excluding communication that is conspicuously marked or otherwise\n      designated in writing by the copyright owner as \"Not a Contribution.\"\n\n      \"Contributor\" shall mean Licensor and any individual or Legal Entity\n      on behalf of whom a Contribution has been received by Licensor and\n      subsequently incorporated within the Work.\n\n   2. Grant of Copyright License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      copyright license to reproduce, prepare Derivative Works of,\n      publicly display, publicly perform, sublicense, and distribute the\n      Work and such Derivative Works in Source or Object form.\n\n   3. Grant of Patent License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      (except as stated in this section) patent license to make, have made,\n      use, offer to sell, sell, import, and otherwise transfer the Work,\n      where such license applies only to those patent claims licensable\n      by such Contributor that are necessarily infringed by their\n      Contribution(s) alone or by combination of their Contribution(s)\n      with the Work to which such Contribution(s) was submitted. If You\n      institute patent litigation against any entity (including a\n      cross-claim or counterclaim in a lawsuit) alleging that the Work\n      or a Contribution incorporated within the Work constitutes direct\n      or contributory patent infringement, then any patent licenses\n      granted to You under this License for that Work shall terminate\n      as of the date such litigation is filed.\n\n   4. Redistribution. You may reproduce and distribute copies of the\n      Work or Derivative Works thereof in any medium, with or without\n      modifications, and in Source or Object form, provided that You\n      meet the following conditions:\n\n      (a) You must give any other recipients of the Work or\n          Derivative Works a copy of this License; and\n\n      (b) You must cause any modified files to carry prominent notices\n          stating that You changed the files; and\n\n      (c) You must retain, in the Source form of any Derivative Works\n          that You distribute, all copyright, patent, trademark, and\n          attribution notices from the Source form of the Work,\n          excluding those notices that do not pertain to any part of\n          the Derivative Works; and\n\n      (d) If the Work includes a \"NOTICE\" text file as part of its\n          distribution, then any Derivative Works that You distribute must\n          include a readable copy of the attribution notices contained\n          within such NOTICE file, excluding those notices that do not\n          pertain to any part of the Derivative Works, in at least one\n          of the following places: within a NOTICE text file distributed\n          as part of the Derivative Works; within the Source form or\n          documentation, if provided along with the Derivative Works; or,\n          within a display generated by the Derivative Works, if and\n          wherever such third-party notices normally appear. The contents\n          of the NOTICE file are for informational purposes only and\n          do not modify the License. You may add Your own attribution\n          notices within Derivative Works that You distribute, alongside\n          or as an addendum to the NOTICE text from the Work, provided\n          that such additional attribution notices cannot be construed\n          as modifying the License.\n\n      You may add Your own copyright statement to Your modifications and\n      may provide additional or different license terms and conditions\n      for use, reproduction, or distribution of Your modifications, or\n      for any such Derivative Works as a whole, provided Your use,\n      reproduction, and distribution of the Work otherwise complies with\n      the conditions stated in this License.\n\n   5. Submission of Contributions. Unless You explicitly state otherwise,\n      any Contribution intentionally submitted for inclusion in the Work\n      by You to the Licensor shall be under the terms and conditions of\n      this License, without any additional terms or conditions.\n      Notwithstanding the above, nothing herein shall supersede or modify\n      the terms of any separate license agreement you may have executed\n      with Licensor regarding such Contributions.\n\n   6. Trademarks. This License does not grant permission to use the trade\n      names, trademarks, service marks, or product names of the Licensor,\n      except as required for reasonable and customary use in describing the\n      origin of the Work and reproducing the content of the NOTICE file.\n\n   7. Disclaimer of Warranty. Unless required by applicable law or\n      agreed to in writing, Licensor provides the Work (and each\n      Contributor provides its Contributions) on an \"AS IS\" BASIS,\n      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n      implied, including, without limitation, any warranties or conditions\n      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n      PARTICULAR PURPOSE. You are solely responsible for determining the\n      appropriateness of using or redistributing the Work and assume any\n      risks associated with Your exercise of permissions under this License.\n\n   8. Limitation of Liability. In no event and under no legal theory,\n      whether in tort (including negligence), contract, or otherwise,\n      unless required by applicable law (such as deliberate and grossly\n      negligent acts) or agreed to in writing, shall any Contributor be\n      liable to You for damages, including any direct, indirect, special,\n      incidental, or consequential damages of any character arising as a\n      result of this License or out of the use or inability to use the\n      Work (including but not limited to damages for loss of goodwill,\n      work stoppage, computer failure or malfunction, or any and all\n      other commercial damages or losses), even if such Contributor\n      has been advised of the possibility of such damages.\n\n   9. Accepting Warranty or Additional Liability. While redistributing\n      the Work or Derivative Works thereof, You may choose to offer,\n      and charge a fee for, acceptance of support, warranty, indemnity,\n      or other liability obligations and/or rights consistent with this\n      License. However, in accepting such obligations, You may act only\n      on Your own behalf and on Your sole responsibility, not on behalf\n      of any other Contributor, and only if You agree to indemnify,\n      defend, and hold each Contributor harmless for any liability\n      incurred by, or claims asserted against, such Contributor by reason\n      of your accepting any such warranty or additional liability.\n\n   END OF TERMS AND CONDITIONS\n\n   APPENDIX: How to apply the Apache License to your work.\n\n      To apply the Apache License to your work, attach the following\n      boilerplate notice, with the fields enclosed by brackets \"[]\"\n      replaced with your own identifying information. (Don't include\n      the brackets!)  The text should be enclosed in the appropriate\n      comment syntax for the file format. We also recommend that a\n      file or class name and description of purpose be included on the\n      same \"printed page\" as the copyright notice for easier\n      identification within third-party archives.\n\n   Copyright [yyyy] [name of copyright owner]\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\n\n\n\nAPACHE TOMCAT SUBCOMPONENTS:\n\nApache Tomcat includes a number of subcomponents with separate copyright notices\nand license terms. Your use of these subcomponents is subject to the terms and\nconditions of the following licenses.\n\n\nFor the ecj-x.x.x.jar component:\n\nEclipse Public License - v 1.0\n\nTHE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE PUBLIC\nLICENSE (\"AGREEMENT\"). ANY USE, REPRODUCTION OR DISTRIBUTION OF THE PROGRAM\nCONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.\n\n1. DEFINITIONS\n\n\"Contribution\" means:\n\na) in the case of the initial Contributor, the initial code and documentation\ndistributed under this Agreement, and\n\nb) in the case of each subsequent Contributor:\n\ni) changes to the Program, and\n\nii) additions to the Program;\n\nwhere such changes and/or additions to the Program originate from and are\ndistributed by that particular Contributor. A Contribution 'originates' from a\nContributor if it was added to the Program by such Contributor itself or anyone\nacting on such Contributor's behalf. Contributions do not include additions to\nthe Program which: (i) are separate modules of software distributed in\nconjunction with the Program under their own license agreement, and (ii) are not\nderivative works of the Program.\n\n\"Contributor\" means any person or entity that distributes the Program.\n\n\"Licensed Patents\" mean patent claims licensable by a Contributor which are\nnecessarily infringed by the use or sale of its Contribution alone or when\ncombined with the Program.\n\n\"Program\" means the Contributions distributed in accordance with this Agreement.\n\n\"Recipient\" means anyone who receives the Program under this Agreement,\nincluding all Contributors.\n\n2. GRANT OF RIGHTS\n\na) Subject to the terms of this Agreement, each Contributor hereby grants\nRecipient a non-exclusive, worldwide, royalty-free copyright license to\nreproduce, prepare derivative works of, publicly display, publicly perform,\ndistribute and sublicense the Contribution of such Contributor, if any, and such\nderivative works, in source code and object code form.\n\nb) Subject to the terms of this Agreement, each Contributor hereby grants\nRecipient a non-exclusive, worldwide, royalty-free patent license under Licensed\nPatents to make, use, sell, offer to sell, import and otherwise transfer the\nContribution of such Contributor, if any, in source code and object code form.\nThis patent license shall apply to the combination of the Contribution and the\nProgram if, at the time the Contribution is added by the Contributor, such\naddition of the Contribution causes such combination to be covered by the\nLicensed Patents. The patent license shall not apply to any other combinations\nwhich include the Contribution. No hardware per se is licensed hereunder.\n\nc) Recipient understands that although each Contributor grants the licenses to\nits Contributions set forth herein, no assurances are provided by any\nContributor that the Program does not infringe the patent or other intellectual\nproperty rights of any other entity. Each Contributor disclaims any liability to\nRecipient for claims brought by any other entity based on infringement of\nintellectual property rights or otherwise. As a condition to exercising the\nrights and licenses granted hereunder, each Recipient hereby assumes sole\nresponsibility to secure any other intellectual property rights needed, if any.\nFor example, if a third party patent license is required to allow Recipient to\ndistribute the Program, it is Recipient's responsibility to acquire that license\nbefore distributing the Program.\n\nd) Each Contributor represents that to its knowledge it has sufficient copyright\nrights in its Contribution, if any, to grant the copyright license set forth in\nthis Agreement.\n\n3. REQUIREMENTS\n\nA Contributor may choose to distribute the Program in object code form under its\nown license agreement, provided that:\n\na) it complies with the terms and conditions of this Agreement; and\n\nb) its license agreement:\n\ni) effectively disclaims on behalf of all Contributors all warranties and\nconditions, express and implied, including warranties or conditions of title and\nnon-infringement, and implied warranties or conditions of merchantability and\nfitness for a particular purpose;\n\nii) effectively excludes on behalf of all Contributors all liability for\ndamages, including direct, indirect, special, incidental and consequential\ndamages, such as lost profits;\n\niii) states that any provisions which differ from this Agreement are offered by\nthat Contributor alone and not by any other party; and\n\niv) states that source code for the Program is available from such Contributor,\nand informs licensees how to obtain it in a reasonable manner on or through a\nmedium customarily used for software exchange.\n\nWhen the Program is made available in source code form:\n\na) it must be made available under this Agreement; and\n\nb) a copy of this Agreement must be included with each copy of the Program.\n\nContributors may not remove or alter any copyright notices contained within the\nProgram.\n\nEach Contributor must identify itself as the originator of its Contribution, if\nany, in a manner that reasonably allows subsequent Recipients to identify the\noriginator of the Contribution.\n\n4. COMMERCIAL DISTRIBUTION\n\nCommercial distributors of software may accept certain responsibilities with\nrespect to end users, business partners and the like. While this license is\nintended to facilitate the commercial use of the Program, the Contributor who\nincludes the Program in a commercial product offering should do so in a manner\nwhich does not create potential liability for other Contributors. Therefore, if\na Contributor includes the Program in a commercial product offering, such\nContributor (\"Commercial Contributor\") hereby agrees to defend and indemnify\nevery other Contributor (\"Indemnified Contributor\") against any losses, damages\nand costs (collectively \"Losses\") arising from claims, lawsuits and other legal\nactions brought by a third party against the Indemnified Contributor to the\nextent caused by the acts or omissions of such Commercial Contributor in\nconnection with its distribution of the Program in a commercial product\noffering. The obligations in this section do not apply to any claims or Losses\nrelating to any actual or alleged intellectual property infringement. In order\nto qualify, an Indemnified Contributor must: a) promptly notify the Commercial\nContributor in writing of such claim, and b) allow the Commercial Contributor\nto control, and cooperate with the Commercial Contributor in, the defense and\nany related settlement negotiations. The Indemnified Contributor may\nparticipate in any such claim at its own expense.\n\nFor example, a Contributor might include the Program in a commercial product\noffering, Product X. That Contributor is then a Commercial Contributor. If that\nCommercial Contributor then makes performance claims, or offers warranties\nrelated to Product X, those performance claims and warranties are such\nCommercial Contributor's responsibility alone. Under this section, the\nCommercial Contributor would have to defend claims against the other\nContributors related to those performance claims and warranties, and if a court\nrequires any other Contributor to pay any damages as a result, the Commercial\nContributor must pay those damages.\n\n5. NO WARRANTY\n\nEXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED ON AN\n\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR\nIMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF TITLE,\nNON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Each\nRecipient is solely responsible for determining the appropriateness of using and\ndistributing the Program and assumes all risks associated with its exercise of\nrights under this Agreement , including but not limited to the risks and costs\nof program errors, compliance with applicable laws, damage to or loss of data,\nprograms or equipment, and unavailability or interruption of operations.\n\n6. DISCLAIMER OF LIABILITY\n\nEXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT NOR ANY\nCONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL,\nSPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST\nPROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,\nSTRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\nOUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS\nGRANTED HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.\n\n7. GENERAL\n\nIf any provision of this Agreement is invalid or unenforceable under applicable\nlaw, it shall not affect the validity or enforceability of the remainder of the\nterms of this Agreement, and without further action by the parties hereto, such\nprovision shall be reformed to the minimum extent necessary to make such\nprovision valid and enforceable.\n\nIf Recipient institutes patent litigation against any entity (including a\ncross-claim or counterclaim in a lawsuit) alleging that the Program itself\n(excluding combinations of the Program with other software or hardware)\ninfringes such Recipient's patent(s), then such Recipient's rights granted under\nSection 2(b) shall terminate as of the date such litigation is filed.\n\nAll Recipient's rights under this Agreement shall terminate if it fails to\ncomply with any of the material terms or conditions of this Agreement and does\nnot cure such failure in a reasonable period of time after becoming aware of\nsuch noncompliance. If all Recipient's rights under this Agreement terminate,\nRecipient agrees to cease use and distribution of the Program as soon as\nreasonably practicable. However, Recipient's obligations under this Agreement\nand any licenses granted by Recipient relating to the Program shall continue and\nsurvive.\n\nEveryone is permitted to copy and distribute copies of this Agreement, but in\norder to avoid inconsistency the Agreement is copyrighted and may only be\nmodified in the following manner. The Agreement Steward reserves the right to\npublish new versions (including revisions) of this Agreement from time to time.\nNo one other than the Agreement Steward has the right to modify this Agreement.\nThe Eclipse Foundation is the initial Agreement Steward. The Eclipse Foundation\nmay assign the responsibility to serve as the Agreement Steward to a suitable\nseparate entity. Each new version of the Agreement will be given a\ndistinguishing version number. The Program (including Contributions) may always\nbe distributed subject to the version of the Agreement under which it was\nreceived. In addition, after a new version of the Agreement is published,\nContributor may elect to distribute the Program (including its Contributions)\nunder the new version. Except as expressly stated in Sections 2(a) and 2(b)\nabove, Recipient receives no rights or licenses to the intellectual property of\nany Contributor under this Agreement, whether expressly, by implication,\nestoppel or otherwise. All rights in the Program not expressly granted under\nthis Agreement are reserved.\n\nThis Agreement is governed by the laws of the State of New York and the\nintellectual property laws of the United States of America. No party to this\nAgreement will bring a legal action under this Agreement more than one year\nafter the cause of action arose. Each party waives its rights to a jury trial in\nany resulting litigation.\n\n\nFor the Windows Installer component:\n\n    * All NSIS source code, plug-ins, documentation, examples, header files and\n       graphics, with the exception of the compression modules and where\n       otherwise noted, are licensed under the zlib/libpng license.\n    * The zlib compression module for NSIS is licensed under the zlib/libpng\n       license.\n    * The bzip2 compression module for NSIS is licensed under the bzip2 license.\n    * The lzma compression module for NSIS is licensed under the Common Public\n       License version 1.0.\n\nzlib/libpng license\n\nThis software is provided 'as-is', without any express or implied warranty. In\nno event will the authors be held liable for any damages arising from the use of\nthis software.\n\nPermission is granted to anyone to use this software for any purpose, including\ncommercial applications, and to alter it and redistribute it freely, subject to\nthe following restrictions:\n\n   1. The origin of this software must not be misrepresented; you must not claim\n       that you wrote the original software. If you use this software in a\n       product, an acknowledgment in the product documentation would be\n       appreciated but is not required.\n   2. Altered source versions must be plainly marked as such, and must not be\n       misrepresented as being the original software.\n   3. This notice may not be removed or altered from any source distribution.\n\nbzip2 license\n\nRedistribution and use in source and binary forms, with or without modification,\nare permitted provided that the following conditions are met:\n\n   1. Redistributions of source code must retain the above copyright notice,\n       this list of conditions and the following disclaimer.\n   2. The origin of this software must not be misrepresented; you must not claim\n       that you wrote the original software. If you use this software in a\n       product, an acknowledgment in the product documentation would be\n       appreciated but is not required.\n   3. Altered source versions must be plainly marked as such, and must not be\n       misrepresented as being the original software.\n   4. The name of the author may not be used to endorse or promote products\n       derived from this software without specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS AND ANY EXPRESS OR IMPLIED\nWARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF\nMERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT\nSHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,\nEXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT\nOF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS\nINTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN\nCONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING\nIN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY\nOF SUCH DAMAGE.\n\nJulian Seward, Cambridge, UK.\n\njseward@acm.org\nCommon Public License version 1.0\n\nTHE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS COMMON PUBLIC\nLICENSE (\"AGREEMENT\"). ANY USE, REPRODUCTION OR DISTRIBUTION OF THE PROGRAM\nCONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.\n\n1. DEFINITIONS\n\n\"Contribution\" means:\n\na) in the case of the initial Contributor, the initial code and documentation\ndistributed under this Agreement, and b) in the case of each subsequent\nContributor:\n\ni) changes to the Program, and\n\nii) additions to the Program;\n\nwhere such changes and/or additions to the Program originate from and are\ndistributed by that particular Contributor. A Contribution 'originates' from a\nContributor if it was added to the Program by such Contributor itself or anyone\nacting on such Contributor's behalf. Contributions do not include additions to\nthe Program which: (i) are separate modules of software distributed in\nconjunction with the Program under their own license agreement, and (ii) are not\nderivative works of the Program.\n\n\"Contributor\" means any person or entity that distributes the Program.\n\n\"Licensed Patents \" mean patent claims licensable by a Contributor which are\nnecessarily infringed by the use or sale of its Contribution alone or when\ncombined with the Program.\n\n\"Program\" means the Contributions distributed in accordance with this Agreement.\n\n\"Recipient\" means anyone who receives the Program under this Agreement,\nincluding all Contributors.\n\n2. GRANT OF RIGHTS\n\na) Subject to the terms of this Agreement, each Contributor hereby grants\nRecipient a non-exclusive, worldwide, royalty-free copyright license to\nreproduce, prepare derivative works of, publicly display, publicly perform,\ndistribute and sublicense the Contribution of such Contributor, if any, and such\nderivative works, in source code and object code form.\n\nb) Subject to the terms of this Agreement, each Contributor hereby grants\nRecipient a non-exclusive, worldwide, royalty-free patent license under Licensed\nPatents to make, use, sell, offer to sell, import and otherwise transfer the\nContribution of such Contributor, if any, in source code and object code form.\nThis patent license shall apply to the combination of the Contribution and the\nProgram if, at the time the Contribution is added by the Contributor, such\naddition of the Contribution causes such combination to be covered by the\nLicensed Patents. The patent license shall not apply to any other combinations\nwhich include the Contribution. No hardware per se is licensed hereunder.\n\nc) Recipient understands that although each Contributor grants the licenses to\nits Contributions set forth herein, no assurances are provided by any\nContributor that the Program does not infringe the patent or other intellectual\nproperty rights of any other entity. Each Contributor disclaims any liability to\nRecipient for claims brought by any other entity based on infringement of\nintellectual property rights or otherwise. As a condition to exercising the\nrights and licenses granted hereunder, each Recipient hereby assumes sole\nresponsibility to secure any other intellectual property rights needed, if any.\nFor example, if a third party patent license is required to allow Recipient to\ndistribute the Program, it is Recipient's responsibility to acquire that license\nbefore distributing the Program.\n\nd) Each Contributor represents that to its knowledge it has sufficient copyright\nrights in its Contribution, if any, to grant the copyright license set forth in\nthis Agreement.\n\n3. REQUIREMENTS\n\nA Contributor may choose to distribute the Program in object code form under its\nown license agreement, provided that:\n\na) it complies with the terms and conditions of this Agreement; and\n\nb) its license agreement:\n\ni) effectively disclaims on behalf of all Contributors all warranties and\nconditions, express and implied, including warranties or conditions of title and\nnon-infringement, and implied warranties or conditions of merchantability and\nfitness for a particular purpose;\n\nii) effectively excludes on behalf of all Contributors all liability for\ndamages, including direct, indirect, special, incidental and consequential\ndamages, such as lost profits;\n\niii) states that any provisions which differ from this Agreement are offered by\nthat Contributor alone and not by any other party; and\n\niv) states that source code for the Program is available from such Contributor,\nand informs licensees how to obtain it in a reasonable manner on or through a\nmedium customarily used for software exchange.\n\nWhen the Program is made available in source code form:\n\na) it must be made available under this Agreement; and\n\nb) a copy of this Agreement must be included with each copy of the Program.\n\nContributors may not remove or alter any copyright notices contained within the\nProgram.\n\nEach Contributor must identify itself as the originator of its Contribution, if\nany, in a manner that reasonably allows subsequent Recipients to identify the\noriginator of the Contribution.\n\n4. COMMERCIAL DISTRIBUTION\n\nCommercial distributors of software may accept certain responsibilities with\nrespect to end users, business partners and the like. While this license is\nintended to facilitate the commercial use of the Program, the Contributor who\nincludes the Program in a commercial product offering should do so in a manner\nwhich does not create potential liability for other Contributors. Therefore, if\na Contributor includes the Program in a commercial product offering, such\nContributor (\"Commercial Contributor\") hereby agrees to defend and indemnify\nevery other Contributor (\"Indemnified Contributor\") against any losses, damages\nand costs (collectively \"Losses\") arising from claims, lawsuits and other legal\nactions brought by a third party against the Indemnified Contributor to the\nextent caused by the acts or omissions of such Commercial Contributor in\nconnection with its distribution of the Program in a commercial product\noffering. The obligations in this section do not apply to any claims or Losses\nrelating to any actual or alleged intellectual property infringement. In order\nto qualify, an Indemnified Contributor must: a) promptly notify the Commercial\nContributor in writing of such claim, and b) allow the Commercial Contributor to\ncontrol, and cooperate with the Commercial Contributor in, the defense and any\nrelated settlement negotiations. The Indemnified Contributor may participate in\nany such claim at its own expense.\n\nFor example, a Contributor might include the Program in a commercial product\noffering, Product X. That Contributor is then a Commercial Contributor. If that\nCommercial Contributor then makes performance claims, or offers warranties\nrelated to Product X, those performance claims and warranties are such\nCommercial Contributor's responsibility alone. Under this section, the\nCommercial Contributor would have to defend claims against the other\nContributors related to those performance claims and warranties, and if a court\nrequires any other Contributor to pay any damages as a result, the Commercial\nContributor must pay those damages.\n\n5. NO WARRANTY\n\nEXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED ON AN\n\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR\nIMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF TITLE,\nNON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Each\nRecipient is solely responsible for determining the appropriateness of using and\ndistributing the Program and assumes all risks associated with its exercise of\nrights under this Agreement, including but not limited to the risks and costs of\nprogram errors, compliance with applicable laws, damage to or loss of data,\nprograms or equipment, and unavailability or interruption of operations.\n\n6. DISCLAIMER OF LIABILITY\n\nEXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT NOR ANY\nCONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL,\nSPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST\nPROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,\nSTRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\nOUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS\nGRANTED HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.\n\n7. GENERAL\n\nIf any provision of this Agreement is invalid or unenforceable under applicable\nlaw, it shall not affect the validity or enforceability of the remainder of the\nterms of this Agreement, and without further action by the parties hereto, such\nprovision shall be reformed to the minimum extent necessary to make such\nprovision valid and enforceable.\n\nIf Recipient institutes patent litigation against a Contributor with respect to\na patent applicable to software (including a cross-claim or counterclaim in a\nlawsuit), then any patent licenses granted by that Contributor to such Recipient\nunder this Agreement shall terminate as of the date such litigation is filed. In\naddition, if Recipient institutes patent litigation against any entity\n(including a cross-claim or counterclaim in a lawsuit) alleging that the Program\nitself (excluding combinations of the Program with other software or hardware)\ninfringes such Recipient's patent(s), then such Recipient's rights granted under\nSection 2(b) shall terminate as of the date such litigation is filed.\n\nAll Recipient's rights under this Agreement shall terminate if it fails to\ncomply with any of the material terms or conditions of this Agreement and does\nnot cure such failure in a reasonable period of time after becoming aware of\nsuch noncompliance. If all Recipient's rights under this Agreement terminate,\nRecipient agrees to cease use and distribution of the Program as soon as\nreasonably practicable. However, Recipient's obligations under this Agreement\nand any licenses granted by Recipient relating to the Program shall continue and\nsurvive.\n\nEveryone is permitted to copy and distribute copies of this Agreement, but in\norder to avoid inconsistency the Agreement is copyrighted and may only be\nmodified in the following manner. The Agreement Steward reserves the right to\npublish new versions (including revisions) of this Agreement from time to time.\nNo one other than the Agreement Steward has the right to modify this Agreement.\nIBM is the initial Agreement Steward. IBM may assign the responsibility to serve\nas the Agreement Steward to a suitable separate entity. Each new version of the\nAgreement will be given a distinguishing version number. The Program (including\nContributions) may always be distributed subject to the version of the Agreement\nunder which it was received. In addition, after a new version of the Agreement\nis published, Contributor may elect to distribute the Program (including its\nContributions) under the new version. Except as expressly stated in Sections\n2(a) and 2(b) above, Recipient receives no rights or licenses to the\nintellectual property of any Contributor under this Agreement, whether\nexpressly, by implication, estoppel or otherwise. All rights in the Program not\nexpressly granted under this Agreement are reserved.\n\nThis Agreement is governed by the laws of the State of New York and the\nintellectual property laws of the United States of America. No party to this\nAgreement will bring a legal action under this Agreement more than one year\nafter the cause of action arose. Each party waives its rights to a jury trial in\nany resulting litigation.\n\nSpecial exception for LZMA compression module\n\nIgor Pavlov and Amir Szekely, the authors of the LZMA compression module for\nNSIS, expressly permit you to statically or dynamically link your code (or bind\nby name) to the files from the LZMA compression module for NSIS without\nsubjecting your linked code to the terms of the Common Public license version\n1.0. Any modifications or additions to files from the LZMA compression module\nfor NSIS, however, are subject to the terms of the Common Public License version\n1.0.\n\n\nFor the following XML Schemas for Java EE Deployment Descriptors:\n - javaee_5.xsd\n - javaee_web_services_1_2.xsd\n - javaee_web_services_client_1_2.xsd\n - javaee_6.xsd\n - javaee_web_services_1_3.xsd\n - javaee_web_services_client_1_3.xsd\n - jsp_2_2.xsd\n - web-app_3_0.xsd\n - web-common_3_0.xsd\n - web-fragment_3_0.xsd\n\nCOMMON DEVELOPMENT AND DISTRIBUTION LICENSE (CDDL) Version 1.0\n\n1. Definitions.\n\n   1.1. Contributor. means each individual or entity that creates or contributes\n        to the creation of Modifications.\n\n   1.2. Contributor Version. means the combination of the Original Software,\n        prior Modifications used by a Contributor (if any), and the\n        Modifications made by that particular Contributor.\n\n   1.3. Covered Software. means (a) the Original Software, or (b) Modifications,\n        or (c) the combination of files containing Original Software with files\n        containing Modifications, in each case including portions thereof.\n\n   1.4. Executable. means the Covered Software in any form other than Source\n        Code.\n\n   1.5. Initial Developer. means the individual or entity that first makes\n        Original Software available under this License.\n\n   1.6. Larger Work. means a work which combines Covered Software or portions\n        thereof with code not governed by the terms of this License.\n\n   1.7. License. means this document.\n\n   1.8. Licensable. means having the right to grant, to the maximum extent\n        possible, whether at the time of the initial grant or subsequently\n        acquired, any and all of the rights conveyed herein.\n\n   1.9. Modifications. means the Source Code and Executable form of any of the\n        following:\n\n        A. Any file that results from an addition to, deletion from or\n           modification of the contents of a file containing Original Software\n           or previous Modifications;\n\n        B. Any new file that contains any part of the Original Software or\n           previous Modification; or\n\n        C. Any new file that is contributed or otherwise made available under\n           the terms of this License.\n\n   1.10. Original Software. means the Source Code and Executable form of\n         computer software code that is originally released under this License.\n\n   1.11. Patent Claims. means any patent claim(s), now owned or hereafter\n         acquired, including without limitation, method, process, and apparatus\n         claims, in any patent Licensable by grantor.\n\n   1.12. Source Code. means (a) the common form of computer software code in\n         which modifications are made and (b) associated documentation included\n         in or with such code.\n\n   1.13. You. (or .Your.) means an individual or a legal entity exercising\n         rights under, and complying with all of the terms of, this License. For\n         legal entities, .You. includes any entity which controls, is controlled\n         by, or is under common control with You. For purposes of this\n         definition, .control. means (a) the power, direct or indirect, to cause\n         the direction or management of such entity, whether by contract or\n         otherwise, or (b) ownership of more than fifty percent (50%) of the\n         outstanding shares or beneficial ownership of such entity.\n\n2. License Grants.\n\n      2.1. The Initial Developer Grant.\n\n      Conditioned upon Your compliance with Section 3.1 below and subject to\n      third party intellectual property claims, the Initial Developer hereby\n      grants You a world-wide, royalty-free, non-exclusive license:\n\n        (a) under intellectual property rights (other than patent or trademark)\n            Licensable by Initial Developer, to use, reproduce, modify, display,\n            perform, sublicense and distribute the Original Software (or\n            portions thereof), with or without Modifications, and/or as part of\n            a Larger Work; and\n\n        (b) under Patent Claims infringed by the making, using or selling of\n            Original Software, to make, have made, use, practice, sell, and\n            offer for sale, and/or otherwise dispose of the Original Software\n            (or portions thereof).\n\n        (c) The licenses granted in Sections 2.1(a) and (b) are effective on the\n            date Initial Developer first distributes or otherwise makes the\n            Original Software available to a third party under the terms of this\n            License.\n\n        (d) Notwithstanding Section 2.1(b) above, no patent license is granted:\n            (1) for code that You delete from the Original Software, or (2) for\n            infringements caused by: (i) the modification of the Original\n            Software, or (ii) the combination of the Original Software with\n            other software or devices.\n\n    2.2. Contributor Grant.\n\n    Conditioned upon Your compliance with Section 3.1 below and subject to third\n    party intellectual property claims, each Contributor hereby grants You a\n    world-wide, royalty-free, non-exclusive license:\n\n        (a) under intellectual property rights (other than patent or trademark)\n            Licensable by Contributor to use, reproduce, modify, display,\n            perform, sublicense and distribute the Modifications created by such\n            Contributor (or portions thereof), either on an unmodified basis,\n            with other Modifications, as Covered Software and/or as part of a\n            Larger Work; and\n\n        (b) under Patent Claims infringed by the making, using, or selling of\n            Modifications made by that Contributor either alone and/or in\n            combination with its Contributor Version (or portions of such\n            combination), to make, use, sell, offer for sale, have made, and/or\n            otherwise dispose of: (1) Modifications made by that Contributor (or\n            portions thereof); and (2) the combination of Modifications made by\n            that Contributor with its Contributor Version (or portions of such\n            combination).\n\n        (c) The licenses granted in Sections 2.2(a) and 2.2(b) are effective on\n            the date Contributor first distributes or otherwise makes the\n            Modifications available to a third party.\n\n        (d) Notwithstanding Section 2.2(b) above, no patent license is granted:\n            (1) for any code that Contributor has deleted from the Contributor\n            Version; (2) for infringements caused by: (i) third party\n            modifications of Contributor Version, or (ii) the combination of\n            Modifications made by that Contributor with other software (except\n            as part of the Contributor Version) or other devices; or (3) under\n            Patent Claims infringed by Covered Software in the absence of\n            Modifications made by that Contributor.\n\n3. Distribution Obligations.\n\n      3.1. Availability of Source Code.\n      Any Covered Software that You distribute or otherwise make available in\n      Executable form must also be made available in Source Code form and that\n      Source Code form must be distributed only under the terms of this License.\n      You must include a copy of this License with every copy of the Source Code\n      form of the Covered Software You distribute or otherwise make available.\n      You must inform recipients of any such Covered Software in Executable form\n      as to how they can obtain such Covered Software in Source Code form in a\n      reasonable manner on or through a medium customarily used for software\n      exchange.\n\n      3.2. Modifications.\n      The Modifications that You create or to which You contribute are governed\n      by the terms of this License. You represent that You believe Your\n      Modifications are Your original creation(s) and/or You have sufficient\n      rights to grant the rights conveyed by this License.\n\n      3.3. Required Notices.\n      You must include a notice in each of Your Modifications that identifies\n      You as the Contributor of the Modification. You may not remove or alter\n      any copyright, patent or trademark notices contained within the Covered\n      Software, or any notices of licensing or any descriptive text giving\n      attribution to any Contributor or the Initial Developer.\n\n      3.4. Application of Additional Terms.\n      You may not offer or impose any terms on any Covered Software in Source\n      Code form that alters or restricts the applicable version of this License\n      or the recipients. rights hereunder. You may choose to offer, and to\n      charge a fee for, warranty, support, indemnity or liability obligations to\n      one or more recipients of Covered Software. However, you may do so only on\n      Your own behalf, and not on behalf of the Initial Developer or any\n      Contributor. You must make it absolutely clear that any such warranty,\n      support, indemnity or liability obligation is offered by You alone, and\n      You hereby agree to indemnify the Initial Developer and every Contributor\n      for any liability incurred by the Initial Developer or such Contributor as\n      a result of warranty, support, indemnity or liability terms You offer.\n\n      3.5. Distribution of Executable Versions.\n      You may distribute the Executable form of the Covered Software under the\n      terms of this License or under the terms of a license of Your choice,\n      which may contain terms different from this License, provided that You are\n      in compliance with the terms of this License and that the license for the\n      Executable form does not attempt to limit or alter the recipient.s rights\n      in the Source Code form from the rights set forth in this License. If You\n      distribute the Covered Software in Executable form under a different\n      license, You must make it absolutely clear that any terms which differ\n      from this License are offered by You alone, not by the Initial Developer\n      or Contributor. You hereby agree to indemnify the Initial Developer and\n      every Contributor for any liability incurred by the Initial Developer or\n      such Contributor as a result of any such terms You offer.\n\n      3.6. Larger Works.\n      You may create a Larger Work by combining Covered Software with other code\n      not governed by the terms of this License and distribute the Larger Work\n      as a single product. In such a case, You must make sure the requirements\n      of this License are fulfilled for the Covered Software.\n\n4. Versions of the License.\n\n      4.1. New Versions.\n      Sun Microsystems, Inc. is the initial license steward and may publish\n      revised and/or new versions of this License from time to time. Each\n      version will be given a distinguishing version number. Except as provided\n      in Section 4.3, no one other than the license steward has the right to\n      modify this License.\n\n      4.2. Effect of New Versions.\n      You may always continue to use, distribute or otherwise make the Covered\n      Software available under the terms of the version of the License under\n      which You originally received the Covered Software. If the Initial\n      Developer includes a notice in the Original Software prohibiting it from\n      being distributed or otherwise made available under any subsequent version\n      of the License, You must distribute and make the Covered Software\n      available under the terms of the version of the License under which You\n      originally received the Covered Software. Otherwise, You may also choose\n      to use, distribute or otherwise make the Covered Software available under\n      the terms of any subsequent version of the License published by the\n      license steward.\n\n      4.3. Modified Versions.\n      When You are an Initial Developer and You want to create a new license for\n      Your Original Software, You may create and use a modified version of this\n      License if You: (a) rename the license and remove any references to the\n      name of the license steward (except to note that the license differs from\n      this License); and (b) otherwise make it clear that the license contains\n      terms which differ from this License.\n\n5. DISCLAIMER OF WARRANTY.\n\n   COVERED SOFTWARE IS PROVIDED UNDER THIS LICENSE ON AN .AS IS. BASIS, WITHOUT\n   WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, WITHOUT\n   LIMITATION, WARRANTIES THAT THE COVERED SOFTWARE IS FREE OF DEFECTS,\n   MERCHANTABLE, FIT FOR A PARTICULAR PURPOSE OR NON-INFRINGING. THE ENTIRE RISK\n   AS TO THE QUALITY AND PERFORMANCE OF THE COVERED SOFTWARE IS WITH YOU. SHOULD\n   ANY COVERED SOFTWARE PROVE DEFECTIVE IN ANY RESPECT, YOU (NOT THE INITIAL\n   DEVELOPER OR ANY OTHER CONTRIBUTOR) ASSUME THE COST OF ANY NECESSARY\n   SERVICING, REPAIR OR CORRECTION. THIS DISCLAIMER OF WARRANTY CONSTITUTES AN\n   ESSENTIAL PART OF THIS LICENSE. NO USE OF ANY COVERED SOFTWARE IS AUTHORIZED\n   HEREUNDER EXCEPT UNDER THIS DISCLAIMER.\n\n6. TERMINATION.\n\n      6.1. This License and the rights granted hereunder will terminate\n           automatically if You fail to comply with terms herein and fail to\n           cure such breach within 30 days of becoming aware of the breach.\n           Provisions which, by their nature, must remain in effect beyond the\n           termination of this License shall survive.\n\n      6.2. If You assert a patent infringement claim (excluding declaratory\n           judgment actions) against Initial Developer or a Contributor (the\n           Initial Developer or Contributor against whom You assert such claim\n           is referred to as .Participant.) alleging that the Participant\n           Software (meaning the Contributor Version where the Participant is a\n           Contributor or the Original Software where the Participant is the\n           Initial Developer) directly or indirectly infringes any patent, then\n           any and all rights granted directly or indirectly to You by such\n           Participant, the Initial Developer (if the Initial Developer is not\n           the Participant) and all Contributors under Sections 2.1 and/or 2.2\n           of this License shall, upon 60 days notice from Participant terminate\n           prospectively and automatically at the expiration of such 60 day\n           notice period, unless if within such 60 day period You withdraw Your\n           claim with respect to the Participant Software against such\n           Participant either unilaterally or pursuant to a written agreement\n           with Participant.\n\n      6.3. In the event of termination under Sections 6.1 or 6.2 above, all end\n           user licenses that have been validly granted by You or any\n           distributor hereunder prior to termination (excluding licenses\n           granted to You by any distributor) shall survive termination.\n\n7. LIMITATION OF LIABILITY.\n\n   UNDER NO CIRCUMSTANCES AND UNDER NO LEGAL THEORY, WHETHER TORT (INCLUDING\n   NEGLIGENCE), CONTRACT, OR OTHERWISE, SHALL YOU, THE INITIAL DEVELOPER, ANY\n   OTHER CONTRIBUTOR, OR ANY DISTRIBUTOR OF COVERED SOFTWARE, OR ANY SUPPLIER OF\n   ANY OF SUCH PARTIES, BE LIABLE TO ANY PERSON FOR ANY INDIRECT, SPECIAL,\n   INCIDENTAL, OR CONSEQUENTIAL DAMAGES OF ANY CHARACTER INCLUDING, WITHOUT\n   LIMITATION, DAMAGES FOR LOST PROFITS, LOSS OF GOODWILL, WORK STOPPAGE,\n   COMPUTER FAILURE OR MALFUNCTION, OR ANY AND ALL OTHER COMMERCIAL DAMAGES OR\n   LOSSES, EVEN IF SUCH PARTY SHALL HAVE BEEN INFORMED OF THE POSSIBILITY OF\n   SUCH DAMAGES. THIS LIMITATION OF LIABILITY SHALL NOT APPLY TO LIABILITY FOR\n   DEATH OR PERSONAL INJURY RESULTING FROM SUCH PARTY.S NEGLIGENCE TO THE EXTENT\n   APPLICABLE LAW PROHIBITS SUCH LIMITATION. SOME JURISDICTIONS DO NOT ALLOW THE\n   EXCLUSION OR LIMITATION OF INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO THIS\n   EXCLUSION AND LIMITATION MAY NOT APPLY TO YOU.\n\n8. U.S. GOVERNMENT END USERS.\n\n   The Covered Software is a .commercial item,. as that term is defined in 48\n   C.F.R. 2.101 (Oct. 1995), consisting of .commercial computer software. (as\n   that term is defined at 48 C.F.R. ? 252.227-7014(a)(1)) and commercial\n   computer software documentation. as such terms are used in 48 C.F.R. 12.212\n   (Sept. 1995). Consistent with 48 C.F.R. 12.212 and 48 C.F.R. 227.7202-1\n   through 227.7202-4 (June 1995), all U.S. Government End Users acquire Covered\n   Software with only those rights set forth herein. This U.S. Government Rights\n   clause is in lieu of, and supersedes, any other FAR, DFAR, or other clause or\n   provision that addresses Government rights in computer software under this\n   License.\n\n9. MISCELLANEOUS.\n\n   This License represents the complete agreement concerning subject matter\n   hereof. If any provision of this License is held to be unenforceable, such\n   provision shall be reformed only to the extent necessary to make it\n   enforceable. This License shall be governed by the law of the jurisdiction\n   specified in a notice contained within the Original Software (except to the\n   extent applicable law, if any, provides otherwise), excluding such\n   jurisdiction's conflict-of-law provisions. Any litigation relating to this\n   License shall be subject to the jurisdiction of the courts located in the\n   jurisdiction and venue specified in a notice contained within the Original\n   Software, with the losing party responsible for costs, including, without\n   limitation, court costs and reasonable attorneys. fees and expenses. The\n   application of the United Nations Convention on Contracts for the\n   International Sale of Goods is expressly excluded. Any law or regulation\n   which provides that the language of a contract shall be construed against\n   the drafter shall not apply to this License. You agree that You alone are\n   responsible for compliance with the United States export administration\n   regulations (and the export control laws and regulation of any other\n   countries) when You use, distribute or otherwise make available any Covered\n   Software.\n\n10. RESPONSIBILITY FOR CLAIMS.\n\n   As between Initial Developer and the Contributors, each party is responsible\n   for claims and damages arising, directly or indirectly, out of its\n   utilization of rights under this License and You agree to work with Initial\n   Developer and Contributors to distribute such responsibility on an equitable\n   basis. Nothing herein is intended or shall be deemed to constitute any\n   admission of liability.\n\n   NOTICE PURSUANT TO SECTION 9 OF THE COMMON DEVELOPMENT AND DISTRIBUTION\n   LICENSE (CDDL)\n\n   The code released under the CDDL shall be governed by the laws of the State\n   of California (excluding conflict-of-law provisions). Any litigation relating\n   to this License shall be subject to the jurisdiction of the Federal Courts of\n   the Northern District of California and the state courts of the State of\n   California, with venue lying in Santa Clara County, California.\n\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/NOTICE",
    "content": "Apache Tomcat\nCopyright 1999-2013 The Apache Software Foundation\n\nThis product includes software developed by\nThe Apache Software Foundation (http://www.apache.org/).\n\nThe Windows Installer is built with the Nullsoft\nScriptable Install System (NSIS), which is\nopen source software.  The original software and\nrelated information is available at\nhttp://nsis.sourceforge.net.\n\nJava compilation software for JSP pages is provided by Eclipse,\nwhich is open source software.  The original software and\nrelated information is available at\nhttp://www.eclipse.org.\n\nFor the bayeux implementation\nThe org.apache.cometd.bayeux API is derivative work originating at the Dojo Foundation\n* Copyright 2007-2008 Guy Molinari\n* Copyright 2007-2008 Filip Hanik\n* Copyright 2007 Dojo Foundation\n* Copyright 2007 Mort Bay Consulting Pty. Ltd.\n\nThe original XML Schemas for Java EE Deployment Descriptors:\n - javaee_5.xsd\n - javaee_web_services_1_2.xsd\n - javaee_web_services_client_1_2.xsd\n - javaee_6.xsd\n - javaee_web_services_1_3.xsd\n - javaee_web_services_client_1_3.xsd\n - jsp_2_2.xsd\n - web-app_3_0.xsd\n - web-common_3_0.xsd\n - web-fragment_3_0.xsd\nmay be obtained from http://java.sun.com/xml/ns/javaee/\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/RELEASE-NOTES",
    "content": "================================================================================\n  Licensed to the Apache Software Foundation (ASF) under one or more\n  contributor license agreements.  See the NOTICE file distributed with\n  this work for additional information regarding copyright ownership.\n  The ASF licenses this file to You under the Apache License, Version 2.0\n  (the \"License\"); you may not use this file except in compliance with\n  the License.  You may obtain a copy of the License at\n\n      http://www.apache.org/licenses/LICENSE-2.0\n\n  Unless required by applicable law or agreed to in writing, software\n  distributed under the License is distributed on an \"AS IS\" BASIS,\n  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n  See the License for the specific language governing permissions and\n  limitations under the License.\n================================================================================\n\n$Id: RELEASE-NOTES 1189163 2011-10-26 12:19:26Z kkolinko $\n\n\n                     Apache Tomcat Version 7.0.42\n                            Release Notes\n\n\n=========\nCONTENTS:\n=========\n\n* Dependency Changes\n* API Stability\n* JNI Based Applications\n* Bundled APIs\n* Web application reloading and static fields in shared libraries\n* Tomcat on Linux\n* Enabling SSI and CGI Support\n* Security manager URLs\n* Symlinking static resources\n* Viewing the Tomcat Change Log\n* Cryptographic software notice\n* When all else fails\n\n\n===================\nDependency Changes:\n===================\nTomcat 7.0 is designed to run on Java SE 6 and later.\n\nIn addition, Tomcat 7.0 uses the Eclipse JDT Java compiler for\ncompiling JSP pages.  This means you no longer need to have the complete\nJava Development Kit (JDK) to run Tomcat, but a Java Runtime Environment\n(JRE) is sufficient.  The Eclipse JDT Java compiler is bundled with the\nbinary Tomcat distributions.  Tomcat can also be configured to use the\ncompiler from the JDK to compile JSPs, or any other Java compiler supported\nby Apache Ant.\n\n\n==============\nAPI Stability:\n==============\nThe public interfaces for the following classes are fixed and will not be\nchanged at all during the remaining lifetime of the 7.x series:\n- javax/**/*\n\nThe public interfaces for the following classes may be added to in order to\nresolve bugs and/or add new features. No existing interface will be removed or\nchanged although it may be deprecated.\n- org/apache/catalina/*\n- org/apache/catalina/comet/*\n\nNote: As Tomcat 7 matures, the above list will be added to. The list is not\n      considered complete at this time.\n\nThe remaining classes are considered part of the Tomcat internals and may change\nwithout notice between point releases.\n\n\n=======================\nJNI Based Applications:\n=======================\nApplications that require native libraries must ensure that the libraries have\nbeen loaded prior to use.  Typically, this is done with a call like:\n\n  static {\n    System.loadLibrary(\"path-to-library-file\");\n  }\n\nin some class.  However, the application must also ensure that the library is\nnot loaded more than once.  If the above code were placed in a class inside\nthe web application (i.e. under /WEB-INF/classes or /WEB-INF/lib), and the\napplication were reloaded, the loadLibrary() call would be attempted a second\ntime.\n\nTo avoid this problem, place classes that load native libraries outside of the\nweb application, and ensure that the loadLibrary() call is executed only once\nduring the lifetime of a particular JVM.\n\n\n=============\nBundled APIs:\n=============\nA standard installation of Tomcat 7.0 makes all of the following APIs available\nfor use by web applications (by placing them in \"lib\"):\n* annotations-api.jar (Annotations package)\n* catalina.jar (Tomcat Catalina implementation)\n* catalina-ant.jar (Tomcat Catalina Ant tasks)\n* catalina-ha.jar (High availability package)\n* catalina-tribes.jar (Group communication)\n* ecj-4.2.2.jar (Eclipse JDT Java compiler)\n* el-api.jar (EL 2.2 API)\n* jasper.jar (Jasper 2 Compiler and Runtime)\n* jasper-el.jar (Jasper 2 EL implementation)\n* jsp-api.jar (JSP 2.2 API)\n* servlet-api.jar (Servlet 3.0 API)\n* tomcat-api.jar (Interfaces shared by Catalina and Jasper)\n* tomcat-coyote.jar (Tomcat connectors and utility classes)\n* tomcat-dbcp.jar (package renamed database connection pool based on Commons DBCP)\n\nYou can make additional APIs available to all of your web applications by\nputting unpacked classes into a \"classes\" directory (not created by default),\nor by placing them in JAR files in the \"lib\" directory.\n\nTo override the XML parser implementation or interfaces, use the endorsed\nmechanism of the JVM. The default configuration defines JARs located in\n\"endorsed\" as endorsed.\n\n\n================================================================\nWeb application reloading and static fields in shared libraries:\n================================================================\nSome shared libraries (many are part of the JDK) keep references to objects\ninstantiated by the web application. To avoid class loading related problems\n(ClassCastExceptions, messages indicating that the classloader\nis stopped, etc.), the shared libraries state should be reinitialized.\n\nSomething which might help is to avoid putting classes which would be\nreferenced by a shared static field in the web application classloader,\nand putting them in the shared classloader instead (JARs should be put in the\n\"lib\" folder, and classes should be put in the \"classes\" folder).\n\n\n================\nTomcat on Linux:\n================\nGLIBC 2.2 / Linux 2.4 users should define an environment variable:\nexport LD_ASSUME_KERNEL=2.2.5\n\nRedhat Linux 9.0 users should use the following setting to avoid\nstability problems:\nexport LD_ASSUME_KERNEL=2.4.1\n\nThere are some Linux bugs reported against the NIO sendfile behavior, make sure you\nhave a JDK that is up to date, or disable sendfile behavior in the Connector.<br/>\n6427312: (fc) FileChannel.transferTo() throws IOException \"system call interrupted\"<br/>\n5103988: (fc) FileChannel.transferTo should return -1 for EAGAIN instead throws IOException<br/>\n6253145: (fc) FileChannel.transferTo on Linux fails when going beyond 2GB boundary<br/>\n6470086: (fc) FileChannel.transferTo(2147483647, 1, channel) cause \"Value too large\" exception<br/>\n\n\n=============================\nEnabling SSI and CGI Support:\n=============================\nBecause of the security risks associated with CGI and SSI available\nto web applications, these features are disabled by default.  \n\nTo enable and configure CGI support, please see the cgi-howto.html page.\n\nTo enable and configue SSI support, please see the ssi-howto.html page.\n\n\n======================\nSecurity manager URLs:\n======================\nIn order to grant security permissions to JARs located inside the\nweb application repository, use URLs of of the following format\nin your policy file:\n\nfile:${catalina.base}/webapps/examples/WEB-INF/lib/driver.jar\n\n\n============================\nSymlinking static resources:\n============================\nBy default, Unix symlinks will not work when used in a web application to link\nresources located outside the web application root directory.\n\nThis behavior is optional, and the \"allowLinking\" flag may be used to disable\nthe check.\n\n\n==============================\nViewing the Tomcat Change Log:\n==============================\nSee changelog.html in this directory.\n\n\n=============================\nCryptographic software notice\n=============================\nThis distribution includes cryptographic software.  The country in\nwhich you currently reside may have restrictions on the import,\npossession, use, and/or re-export to another country, of\nencryption software.  BEFORE using any encryption software, please\ncheck your country's laws, regulations and policies concerning the\nimport, possession, or use, and re-export of encryption software, to\nsee if this is permitted.  See <http://www.wassenaar.org/> for more\ninformation.\n\nThe U.S. Government Department of Commerce, Bureau of Industry and\nSecurity (BIS), has classified this software as Export Commodity\nControl Number (ECCN) 5D002.C.1, which includes information security\nsoftware using or performing cryptographic functions with asymmetric\nalgorithms.  The form and manner of this Apache Software Foundation\ndistribution makes it eligible for export under the License Exception\nENC Technology Software Unrestricted (TSU) exception (see the BIS\nExport Administration Regulations, Section 740.13) for both object\ncode and source code.\n\nThe following provides more details on the included cryptographic\nsoftware:\n  - Tomcat includes code designed to work with JSSE\n  - Tomcat includes code designed to work with OpenSSL\n\n\n====================\nWhen all else fails:\n====================\nSee the FAQ\nhttp://tomcat.apache.org/faq/\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/RUNNING.txt",
    "content": "================================================================================\n  Licensed to the Apache Software Foundation (ASF) under one or more\n  contributor license agreements.  See the NOTICE file distributed with\n  this work for additional information regarding copyright ownership.\n  The ASF licenses this file to You under the Apache License, Version 2.0\n  (the \"License\"); you may not use this file except in compliance with\n  the License.  You may obtain a copy of the License at\n\n      http://www.apache.org/licenses/LICENSE-2.0\n\n  Unless required by applicable law or agreed to in writing, software\n  distributed under the License is distributed on an \"AS IS\" BASIS,\n  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n  See the License for the specific language governing permissions and\n  limitations under the License.\n================================================================================\n\n$Id: RUNNING.txt 1456725 2013-03-14 23:48:06Z kkolinko $\n\n            ===================================================\n            Running The Apache Tomcat 7.0 Servlet/JSP Container\n            ===================================================\n\nApache Tomcat 7.0 requires a Java Standard Edition Runtime\nEnvironment (JRE) version 6 or later.\n\n=============================\nRunning With JRE 6 Or Later\n=============================\n\n(1) Download and Install a Java SE Runtime Environment (JRE)\n\n(1.1) Download a Java SE Runtime Environment (JRE),\n      release version 6 or later, from\n      http://www.oracle.com/technetwork/java/javase/downloads/index.html\n\n(1.2) Install the JRE according to the instructions included with the\n      release.\n\n      You may also use a full Java Development Kit (JDK) rather than just\n      a JRE.\n\n\n(2) Download and Install Apache Tomcat\n\n(2.1) Download a binary distribution of Tomcat from:\n\n      http://tomcat.apache.org/\n\n(2.2) Unpack the binary distribution so that it resides in its own\n      directory (conventionally named \"apache-tomcat-[version]\").\n\n      For the purposes of the remainder of this document, the name\n      \"CATALINA_HOME\" is used to refer to the full pathname of that\n      directory.\n\nNOTE:  As an alternative to downloading a binary distribution, you can\ncreate your own from the Tomcat source code, as described in\n\"BUILDING.txt\".  You can either\n\n  a)  Do the full \"release\" build and find the created distribution in the\n      \"output/release\" directory and then proceed with unpacking as above, or\n\n  b)  Do a simple build and use the \"output/build\" directory as\n      \"CATALINA_HOME\".  Be warned that there are some differences between the\n      contents of the \"output/build\" directory and a full \"release\"\n      distribution.\n\n\n(3) Configure Environment Variables\n\nTomcat is a Java application and does not use environment variables. The\nvariables are used by the Tomcat startup scripts. The scripts use the variables\nto prepare the command that starts Tomcat.\n\n(3.1) Set CATALINA_HOME (required) and CATALINA_BASE (optional)\n\nThe CATALINA_HOME environment variable should be set to the location of the\nroot directory of the \"binary\" distribution of Tomcat.\n\nAn example was given in (2.2) above.\n\nThe Tomcat startup scripts have some logic to set this variable\nautomatically if it is absent, based on the location of the startup script\nin *nix and on the current directory in Windows. That logic might not work\nin all circumstances, so setting the variable explicitly is recommended.\n\nThe CATALINA_BASE environment variable specifies location of the root\ndirectory of the \"active configuration\" of Tomcat. It is optional. It\ndefaults to be equal to CATALINA_HOME.\n\nUsing distinct values for the CATALINA_HOME and CATALINA_BASE variables is\nrecommended to simplify further upgrades and maintenance. It is documented\nin the \"Multiple Tomcat Instances\" section below.\n\n\n(3.2) Set JRE_HOME or JAVA_HOME (required)\n\nThese variables are used to specify location of a Java Runtime\nEnvironment or of a Java Development Kit that is used to start Tomcat.\n\nThe JRE_HOME variable is used to specify location of a JRE. The JAVA_HOME\nvariable is used to specify location of a JDK.\n\nUsing JAVA_HOME provides access to certain additional startup options that\nare not allowed when JRE_HOME is used.\n\nIf both JRE_HOME and JAVA_HOME are specified, JRE_HOME is used.\n\nThe recommended place to specify these variables is a \"setenv\" script. See\nbelow.\n\n\n(3.3) Other variables (optional)\n\nOther environment variables exist, besides the four described above.\nSee the comments at the top of catalina.bat or catalina.sh scripts for\nthe list and a description of each of them.\n\nOne frequently used variable is CATALINA_OPTS. It allows specification of\nadditional options for the java command that starts Tomcat.\n\nSee the Java documentation for the options that affect the Java Runtime\nEnvironment.\n\nSee the \"System Properties\" page in the Tomcat Configuration Reference for\nthe system properties that are specific to Tomcat.\n\nA similar variable is JAVA_OPTS. It is used less frequently. It allows\nspecification of options that are used both to start and to stop Tomcat as well\nas for other commands.\n\nNote: Do not use JAVA_OPTS to specify memory limits. You do not need much\nmemory for a small process that is used to stop Tomcat. Those settings\nbelong to CATALINA_OPTS.\n\nAnother frequently used variable is CATALINA_PID (on *nix only). It\nspecifies the location of the file where process id of the forked Tomcat\njava process will be written. This setting is optional. It will enable the\nfollowing features:\n\n *  better protection against duplicate start attempts and\n *  allows forceful termination of Tomcat process when it does not react to\n    the standard shutdown command.\n\n\n(3.4) Using the \"setenv\" script (optional, recommended)\n\nApart from CATALINA_HOME and CATALINA_BASE, all environment variables can\nbe specified in the \"setenv\" script. The script is placed either into\nCATALINA_BASE/bin or into CATALINA_HOME/bin directory and is named\nsetenv.bat (on Windows) or setenv.sh (on *nix). The file has to be\nreadable.\n\nBy default the setenv script file is absent. If the script file is present\nboth in CATALINA_BASE and in CATALINA_HOME, the one in CATALINA_BASE is\npreferred.\n\nFor example, to configure the JRE_HOME and CATALINA_PID variables you can\ncreate the following script file:\n\nOn Windows, %CATALINA_BASE%\\bin\\setenv.bat:\n\n  set \"JRE_HOME=%ProgramFiles%\\Java\\jre6\"\n  exit /b 0\n\nOn *nix, $CATALINA_BASE/bin/setenv.sh:\n\n  JRE_HOME=/usr/java/latest\n  CATALINA_PID=\"$CATALINA_BASE/tomcat.pid\"\n\n\nThe CATALINA_HOME and CATALINA_BASE variables cannot be configured in the\nsetenv script, because they are used to locate that file.\n\nAll the environment variables described here and the \"setenv\" script are\nused only if you use the standard scripts to launch Tomcat. For example, if\nyou have installed Tomcat as a service on Windows, the service wrapper\nlaunches Java directly and does not use the script files.\n\n\n(4) Start Up Tomcat\n\n(4.1) Tomcat can be started by executing one of the following commands:\n\n  On Windows:\n\n      %CATALINA_HOME%\\bin\\startup.bat\n\n    or\n\n      %CATALINA_HOME%\\bin\\catalina.bat start\n\n  On *nix:\n\n      $CATALINA_HOME/bin/startup.sh\n\n    or\n\n      $CATALINA_HOME/bin/catalina.sh start\n\n(4.2) After startup, the default web applications included with Tomcat will be\n      available by visiting:\n\n      http://localhost:8080/\n\n(4.3) Further information about configuring and running Tomcat can be found in\n      the documentation included here, as well as on the Tomcat web site:\n\n      http://tomcat.apache.org/\n\n\n(5) Shut Down Tomcat\n\n(5.1) Tomcat can be shut down by executing one of the following commands:\n\n  On Windows:\n\n      %CATALINA_HOME%\\bin\\shutdown.bat\n\n    or\n\n      %CATALINA_HOME%\\bin\\catalina.bat stop\n\n  On *nix:\n\n      $CATALINA_HOME/bin/shutdown.sh\n\n    or\n\n      $CATALINA_HOME/bin/catalina.sh stop\n\n==================================================\nAdvanced Configuration - Multiple Tomcat Instances\n==================================================\n\nIn many circumstances, it is desirable to have a single copy of a Tomcat\nbinary distribution shared among multiple users on the same server.  To make\nthis possible, you can set the CATALINA_BASE environment variable to the\ndirectory that contains the files for your 'personal' Tomcat instance.\n\nWhen running with a separate CATALINA_HOME and CATALINA_BASE, the files\nand directories are split as following:\n\nIn CATALINA_BASE:\n\n * bin  - Only the following files:\n\n           * setenv.sh (*nix) or setenv.bat (Windows),\n           * tomcat-juli.jar\n\n          The setenv scripts were described above. The tomcat-juli library\n          is documented in the Logging chapter in the User Guide.\n\n * conf - Server configuration files (including server.xml)\n\n * lib  - Libraries and classes, as explained below\n\n * logs - Log and output files\n\n * webapps - Automatically loaded web applications\n\n * work - Temporary working directories for web applications\n\n * temp - Directory used by the JVM for temporary files (java.io.tmpdir)\n\n\nIn CATALINA_HOME:\n\n * bin  - Startup and shutdown scripts\n\n          The following files will be used only if they are absent in\n          CATALINA_BASE/bin:\n\n          setenv.sh (*nix), setenv.bat (Windows), tomcat-juli.jar\n\n * lib  - Libraries and classes, as explained below\n\n * endorsed - Libraries that override standard \"Endorsed Standards\"\n              libraries provided by JRE. See Classloading documentation\n              in the User Guide for details.\n\n              By default this \"endorsed\" directory is absent.\n\nIn the default configuration the JAR libraries and classes both in\nCATALINA_BASE/lib and in CATALINA_HOME/lib will be added to the common\nclasspath, but the ones in CATALINA_BASE will be added first and thus will\nbe searched first.\n\nThe idea is that you may leave the standard Tomcat libraries in\nCATALINA_HOME/lib and add other ones such as database drivers into\nCATALINA_BASE/lib.\n\nIn general it is advised to never share libraries between web applications,\nbut put them into WEB-INF/lib directories inside the applications. See\nClassloading documentation in the User Guide for details.\n\n\nIt might be useful to note that the values of CATALINA_HOME and\nCATALINA_BASE can be referenced in the XML configuration files processed\nby Tomcat as ${catalina.home} and ${catalina.base} respectively.\n\nFor example, the standard manager web application can be kept in\nCATALINA_HOME/webapps/manager and loaded into CATALINA_BASE by using\nthe following trick:\n\n * Copy the CATALINA_HOME/webapps/manager/META-INF/context.xml\n   file as CATALINA_BASE/conf/Catalina/localhost/manager.xml\n\n * Add docBase attribute as shown below.\n\nThe file will look like the following:\n\n  <?xml version=\"1.0\" encoding=\"UTF-8\"?>\n  <Context docBase=\"${catalina.home}/webapps/manager\"\n    antiResourceLocking=\"false\" privileged=\"true\" >\n    <Valve className=\"org.apache.catalina.valves.RemoteAddrValve\"\n         allow=\"127\\.0\\.0\\.1\" />\n  </Context>\n\nSee Deployer chapter in User Guide and Context and Host chapters in the\nConfiguration Reference for more information on contexts and web\napplication deployment.\n\n\n================\nTroubleshooting\n================\n\nThere are only really 2 things likely to go wrong during the stand-alone\nTomcat install:\n\n(1) The most common hiccup is when another web server (or any process for that\n    matter) has laid claim to port 8080.  This is the default HTTP port that\n    Tomcat attempts to bind to at startup.  To change this, open the file:\n\n       $CATALINA_HOME/conf/server.xml\n\n    and search for '8080'.  Change it to a port that isn't in use, and is\n    greater than 1024, as ports less than or equal to 1024 require superuser\n    access to bind under UNIX.\n\n    Restart Tomcat and you're in business.  Be sure that you replace the \"8080\"\n    in the URL you're using to access Tomcat.  For example, if you change the\n    port to 1977, you would request the URL http://localhost:1977/ in your\n    browser.\n\n(2) The 'localhost' machine isn't found.  This could happen if you're behind a\n    proxy.  If that's the case, make sure the proxy configuration for your\n    browser knows that you shouldn't be going through the proxy to access the\n    \"localhost\".\n\n    In Firefox, this is under Tools/Preferences -> Advanced/Network ->\n    Connection -> Settings..., and in Internet Explorer it is Tools ->\n    Internet Options -> Connections -> LAN Settings.\n\n\n====================\nOptional Components\n====================\n\nThe following optional components may be included with the Apache Tomcat binary\ndistribution. If they are not included, you can install them separately.\n\n 1. Apache Tomcat Native library\n\n 2. Apache Commons Daemon service launcher\n\nBoth of them are implemented in C language and as such have to be compiled\ninto binary code. The binary code will be specific for a platform and CPU\narchitecture and it must match the Java Runtime Environment executables\nthat will be used to launch Tomcat.\n\nThe Windows-specific binary distributions of Apache Tomcat include binary\nfiles for these components. On other platforms you would have to look for\nbinary versions elsewhere or compile them yourself.\n\nIf you are new to Tomcat, do not bother with these components to start with.\nIf you do use them, do not forget to read their documentation.\n\n\nApache Tomcat Native library\n-----------------------------\n\nIt is a library that allows to use the \"Apr\" variant of HTTP and AJP\nprotocol connectors in Apache Tomcat. It is built around OpenSSL and Apache\nPortable Runtime (APR) libraries. Those are the same libraries as used by\nApache HTTPD Server project.\n\nThis feature was especially important in the old days when Java performance\nwas poor. It is less important nowadays, but it is still used and respected\nby many. See Tomcat documentation for more details.\n\nFor further reading:\n\n - Apache Tomcat documentation\n\n    * Documentation for APR/Native library in the Tomcat User's Guide\n\n      http://tomcat.apache.org/tomcat-7.0-doc/apr.html\n\n    * Documentation for the HTTP and AJP protocol connectors in the Tomcat\n      Configuration Reference\n\n      http://tomcat.apache.org/tomcat-7.0-doc/config/http.html\n\n      http://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html\n\n - Apache Tomcat Native project home\n\n      http://tomcat.apache.org/native-doc/\n\n - Other projects\n\n    * OpenSSL\n\n      http://openssl.org/\n\n    * Apache Portable Runtime\n\n      http://apr.apache.org/\n\n    * Apache HTTP Server\n\n      http://httpd.apache.org/\n\nTo disable Apache Tomcat Native library:\n\n - To disable Apache Tomcat Native library when it is installed, or\n - To remove the warning that is logged during Tomcat startup when the\n   library is not installed:\n\n   Edit the \"conf/server.xml\" file and remove \"AprLifecycleListener\" from\n   it.\n\nThe binary file of Apache Tomcat Native library is usually named\n\n  - \"tcnative-1.dll\" on Windows\n  - \"libtcnative-1.so\" on *nix systems\n\n\nApache Commons Daemon\n----------------------\n\nApache Commons Daemon project provides wrappers that can be used to\ninstall Apache Tomcat as a service on Windows or as a daemon on *nix\nsystems.\n\nThe Windows-specific implementation of Apache Commons Daemon is called\n\"procrun\". The *nix-specific one is called \"jsvc\".\n\nFor further reading:\n\n - Apache Commons Daemon project\n\n      http://commons.apache.org/daemon/\n\n - Apache Tomcat documentation\n\n    * Installing Apache Tomcat\n\n      http://tomcat.apache.org/tomcat-7.0-doc/setup.html\n\n    * Windows service HOW-TO\n\n      http://tomcat.apache.org/tomcat-7.0-doc/windows-service-howto.html\n\nThe binary files of Apache Commons Daemon in Apache Tomcat distributions\nfor Windows are named:\n\n  - \"tomcat7.exe\"\n  - \"tomcat7w.exe\"\n\nThese files are renamed copies of \"prunsrv.exe\" and \"prunmgr.exe\" from\nApache Commons Daemon distribution. The file names have a meaning: they are\nused as the service name to register the service in Windows, as well as the\nkey name to store distinct configuration for this installation of\n\"procrun\". If you would like to install several instances of Tomcat 7.0\nin parallel, you have to further rename those files, using the same naming\nscheme.\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/catalina-tasks.xml",
    "content": "<?xml version=\"1.0\"?>\n<!--\n Licensed to the Apache Software Foundation (ASF) under one or more\n  contributor license agreements.  See the NOTICE file distributed with\n  this work for additional information regarding copyright ownership.\n  The ASF licenses this file to You under the Apache License, Version 2.0\n  (the \"License\"); you may not use this file except in compliance with\n  the License.  You may obtain a copy of the License at\n\n      http://www.apache.org/licenses/LICENSE-2.0\n\n  Unless required by applicable law or agreed to in writing, software\n  distributed under the License is distributed on an \"AS IS\" BASIS,\n  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n  See the License for the specific language governing permissions and\n  limitations under the License.\n-->\n<!--\n  XML file for importing Catalina ant tasks.\n  <import file=\"${catalina.home}/bin/catalina-tasks.xml\"/>\n-->\n\n<project name=\"catalina-tasks\">\n  <description>Catalina Ant Manager, JMX and JSPC Tasks</description>\n  <!-- set catalina.home if it's not already set -->\n  <dirname property=\"catalina.home.bin.dir\" file=\"${ant.file.catalina-tasks}\"/>\n  <property name=\"catalina.home\" value=\"${catalina.home.bin.dir}/..\"/>\n  <typedef resource=\"org/apache/catalina/ant/catalina.tasks\">\n    <classpath>\n      <fileset file=\"${catalina.home}/bin/tomcat-juli.jar\"/>\n      <fileset file=\"${catalina.home}/lib/tomcat-api.jar\"/>\n      <fileset file=\"${catalina.home}/lib/tomcat-util.jar\"/>\n      <fileset file=\"${catalina.home}/lib/jasper.jar\"/>\n      <fileset file=\"${catalina.home}/lib/jasper-el.jar\"/>\n      <fileset file=\"${catalina.home}/lib/el-api.jar\"/>\n      <fileset file=\"${catalina.home}/lib/jsp-api.jar\"/>\n      <fileset file=\"${catalina.home}/lib/servlet-api.jar\"/>\n      <fileset file=\"${catalina.home}/lib/catalina-ant.jar\"/>\n      <fileset file=\"${catalina.home}/lib/tomcat-coyote.jar\"/>\n    </classpath>\n  </typedef>\n  <typedef resource=\"org/apache/catalina/ant/jmx/jmxaccessor.tasks\">\n    <classpath>\n      <fileset file=\"${catalina.home}/lib/catalina-ant.jar\"/>\n    </classpath>\n  </typedef>\n</project>\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/catalina.bat",
    "content": "@echo off\r\nrem Licensed to the Apache Software Foundation (ASF) under one or more\r\nrem contributor license agreements.  See the NOTICE file distributed with\r\nrem this work for additional information regarding copyright ownership.\r\nrem The ASF licenses this file to You under the Apache License, Version 2.0\r\nrem (the \"License\"); you may not use this file except in compliance with\r\nrem the License.  You may obtain a copy of the License at\r\nrem\r\nrem     http://www.apache.org/licenses/LICENSE-2.0\r\nrem\r\nrem Unless required by applicable law or agreed to in writing, software\r\nrem distributed under the License is distributed on an \"AS IS\" BASIS,\r\nrem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nrem See the License for the specific language governing permissions and\r\nrem limitations under the License.\r\n\r\nif \"%OS%\" == \"Windows_NT\" setlocal\r\nrem ---------------------------------------------------------------------------\r\nrem Start/Stop Script for the CATALINA Server\r\nrem\r\nrem Environment Variable Prerequisites\r\nrem\r\nrem   Do not set the variables in this script. Instead put them into a script\r\nrem   setenv.bat in CATALINA_BASE/bin to keep your customizations separate.\r\nrem\r\nrem   CATALINA_HOME   May point at your Catalina \"build\" directory.\r\nrem\r\nrem   CATALINA_BASE   (Optional) Base directory for resolving dynamic portions\r\nrem                   of a Catalina installation.  If not present, resolves to\r\nrem                   the same directory that CATALINA_HOME points to.\r\nrem\r\nrem   CATALINA_OPTS   (Optional) Java runtime options used when the \"start\",\r\nrem                   \"run\" or \"debug\" command is executed.\r\nrem                   Include here and not in JAVA_OPTS all options, that should\r\nrem                   only be used by Tomcat itself, not by the stop process,\r\nrem                   the version command etc.\r\nrem                   Examples are heap size, GC logging, JMX ports etc.\r\nrem\r\nrem   CATALINA_TMPDIR (Optional) Directory path location of temporary directory\r\nrem                   the JVM should use (java.io.tmpdir).  Defaults to\r\nrem                   %CATALINA_BASE%\\temp.\r\nrem\r\nrem   JAVA_HOME       Must point at your Java Development Kit installation.\r\nrem                   Required to run the with the \"debug\" argument.\r\nrem\r\nrem   JRE_HOME        Must point at your Java Runtime installation.\r\nrem                   Defaults to JAVA_HOME if empty. If JRE_HOME and JAVA_HOME\r\nrem                   are both set, JRE_HOME is used.\r\nrem\r\nrem   JAVA_OPTS       (Optional) Java runtime options used when any command\r\nrem                   is executed.\r\nrem                   Include here and not in CATALINA_OPTS all options, that\r\nrem                   should be used by Tomcat and also by the stop process,\r\nrem                   the version command etc.\r\nrem                   Most options should go into CATALINA_OPTS.\r\nrem\r\nrem   JAVA_ENDORSED_DIRS (Optional) Lists of of semi-colon separated directories\r\nrem                   containing some jars in order to allow replacement of APIs\r\nrem                   created outside of the JCP (i.e. DOM and SAX from W3C).\r\nrem                   It can also be used to update the XML parser implementation.\r\nrem                   Defaults to $CATALINA_HOME/endorsed.\r\nrem\r\nrem   JPDA_TRANSPORT  (Optional) JPDA transport used when the \"jpda start\"\r\nrem                   command is executed. The default is \"dt_socket\".\r\nrem\r\nrem   JPDA_ADDRESS    (Optional) Java runtime options used when the \"jpda start\"\r\nrem                   command is executed. The default is 8000.\r\nrem\r\nrem   JPDA_SUSPEND    (Optional) Java runtime options used when the \"jpda start\"\r\nrem                   command is executed. Specifies whether JVM should suspend\r\nrem                   execution immediately after startup. Default is \"n\".\r\nrem\r\nrem   JPDA_OPTS       (Optional) Java runtime options used when the \"jpda start\"\r\nrem                   command is executed. If used, JPDA_TRANSPORT, JPDA_ADDRESS,\r\nrem                   and JPDA_SUSPEND are ignored. Thus, all required jpda\r\nrem                   options MUST be specified. The default is:\r\nrem\r\nrem                   -agentlib:jdwp=transport=%JPDA_TRANSPORT%,\r\nrem                       address=%JPDA_ADDRESS%,server=y,suspend=%JPDA_SUSPEND%\r\nrem\r\nrem   LOGGING_CONFIG  (Optional) Override Tomcat's logging config file\r\nrem                   Example (all one line)\r\nrem                   set LOGGING_CONFIG=\"-Djava.util.logging.config.file=%CATALINA_BASE%\\conf\\logging.properties\"\r\nrem\r\nrem   LOGGING_MANAGER (Optional) Override Tomcat's logging manager\r\nrem                   Example (all one line)\r\nrem                   set LOGGING_MANAGER=\"-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager\"\r\nrem\r\nrem   TITLE           (Optional) Specify the title of Tomcat window. The default\r\nrem                   TITLE is Tomcat if it's not specified.\r\nrem                   Example (all one line)\r\nrem                   set TITLE=Tomcat.Cluster#1.Server#1 [%DATE% %TIME%]\r\nrem\r\nrem\r\nrem\r\nrem $Id: catalina.bat 1344732 2012-05-31 14:08:02Z kkolinko $\r\nrem ---------------------------------------------------------------------------\r\n\r\nrem Suppress Terminate batch job on CTRL+C\r\nif not \"\"%1\"\" == \"\"run\"\" goto mainEntry\r\nif \"%TEMP%\" == \"\" goto mainEntry\r\nif exist \"%TEMP%\\%~nx0.run\" goto mainEntry\r\necho Y>\"%TEMP%\\%~nx0.run\"\r\nif not exist \"%TEMP%\\%~nx0.run\" goto mainEntry\r\necho Y>\"%TEMP%\\%~nx0.Y\"\r\ncall \"%~f0\" %* <\"%TEMP%\\%~nx0.Y\"\r\nrem Use provided errorlevel\r\nset RETVAL=%ERRORLEVEL%\r\ndel /Q \"%TEMP%\\%~nx0.Y\" >NUL 2>&1\r\nexit /B %RETVAL%\r\n:mainEntry\r\ndel /Q \"%TEMP%\\%~nx0.run\" >NUL 2>&1\r\n\r\nrem Guess CATALINA_HOME if not defined\r\nset \"CURRENT_DIR=%cd%\"\r\nif not \"%CATALINA_HOME%\" == \"\" goto gotHome\r\nset \"CATALINA_HOME=%CURRENT_DIR%\"\r\nif exist \"%CATALINA_HOME%\\bin\\catalina.bat\" goto okHome\r\ncd ..\r\nset \"CATALINA_HOME=%cd%\"\r\ncd \"%CURRENT_DIR%\"\r\n:gotHome\r\n\r\nif exist \"%CATALINA_HOME%\\bin\\catalina.bat\" goto okHome\r\necho The CATALINA_HOME environment variable is not defined correctly\r\necho This environment variable is needed to run this program\r\ngoto end\r\n:okHome\r\n\r\nrem Copy CATALINA_BASE from CATALINA_HOME if not defined\r\nif not \"%CATALINA_BASE%\" == \"\" goto gotBase\r\nset \"CATALINA_BASE=%CATALINA_HOME%\"\r\n:gotBase\r\n\r\nrem Ensure that any user defined CLASSPATH variables are not used on startup,\r\nrem but allow them to be specified in setenv.bat, in rare case when it is needed.\r\nset CLASSPATH=\r\n\r\nrem Get standard environment variables\r\nif not exist \"%CATALINA_BASE%\\bin\\setenv.bat\" goto checkSetenvHome\r\ncall \"%CATALINA_BASE%\\bin\\setenv.bat\"\r\ngoto setenvDone\r\n:checkSetenvHome\r\nif exist \"%CATALINA_HOME%\\bin\\setenv.bat\" call \"%CATALINA_HOME%\\bin\\setenv.bat\"\r\n:setenvDone\r\n\r\nrem Get standard Java environment variables\r\nif exist \"%CATALINA_HOME%\\bin\\setclasspath.bat\" goto okSetclasspath\r\necho Cannot find \"%CATALINA_HOME%\\bin\\setclasspath.bat\"\r\necho This file is needed to run this program\r\ngoto end\r\n:okSetclasspath\r\ncall \"%CATALINA_HOME%\\bin\\setclasspath.bat\" %1\r\nif errorlevel 1 goto end\r\n\r\nrem Add on extra jar file to CLASSPATH\r\nrem Note that there are no quotes as we do not want to introduce random\r\nrem quotes into the CLASSPATH\r\nif \"%CLASSPATH%\" == \"\" goto emptyClasspath\r\nset \"CLASSPATH=%CLASSPATH%;\"\r\n:emptyClasspath\r\nset \"CLASSPATH=%CLASSPATH%%CATALINA_HOME%\\bin\\bootstrap.jar\"\r\n\r\nif not \"%CATALINA_TMPDIR%\" == \"\" goto gotTmpdir\r\nset \"CATALINA_TMPDIR=%CATALINA_BASE%\\temp\"\r\n:gotTmpdir\r\n\r\nrem Add tomcat-juli.jar to classpath\r\nrem tomcat-juli.jar can be over-ridden per instance\r\nif not exist \"%CATALINA_BASE%\\bin\\tomcat-juli.jar\" goto juliClasspathHome\r\nset \"CLASSPATH=%CLASSPATH%;%CATALINA_BASE%\\bin\\tomcat-juli.jar\"\r\ngoto juliClasspathDone\r\n:juliClasspathHome\r\nset \"CLASSPATH=%CLASSPATH%;%CATALINA_HOME%\\bin\\tomcat-juli.jar\"\r\n:juliClasspathDone\r\n\r\nif not \"%LOGGING_CONFIG%\" == \"\" goto noJuliConfig\r\nset LOGGING_CONFIG=-Dnop\r\nif not exist \"%CATALINA_BASE%\\conf\\logging.properties\" goto noJuliConfig\r\nset LOGGING_CONFIG=-Djava.util.logging.config.file=\"%CATALINA_BASE%\\conf\\logging.properties\"\r\n:noJuliConfig\r\nset JAVA_OPTS=%JAVA_OPTS% %LOGGING_CONFIG%\r\n\r\nif not \"%LOGGING_MANAGER%\" == \"\" goto noJuliManager\r\nset LOGGING_MANAGER=-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager\r\n:noJuliManager\r\nset JAVA_OPTS=%JAVA_OPTS% %LOGGING_MANAGER%\r\n\r\nrem ----- Execute The Requested Command ---------------------------------------\r\n\r\necho Using CATALINA_BASE:   \"%CATALINA_BASE%\"\r\necho Using CATALINA_HOME:   \"%CATALINA_HOME%\"\r\necho Using CATALINA_TMPDIR: \"%CATALINA_TMPDIR%\"\r\nif \"\"%1\"\" == \"\"debug\"\" goto use_jdk\r\necho Using JRE_HOME:        \"%JRE_HOME%\"\r\ngoto java_dir_displayed\r\n:use_jdk\r\necho Using JAVA_HOME:       \"%JAVA_HOME%\"\r\n:java_dir_displayed\r\necho Using CLASSPATH:       \"%CLASSPATH%\"\r\n\r\nset _EXECJAVA=%_RUNJAVA%\r\nset MAINCLASS=org.apache.catalina.startup.Bootstrap\r\nset ACTION=start\r\nset SECURITY_POLICY_FILE=\r\nset DEBUG_OPTS=\r\nset JPDA=\r\n\r\nif not \"\"%1\"\" == \"\"jpda\"\" goto noJpda\r\nset JPDA=jpda\r\nif not \"%JPDA_TRANSPORT%\" == \"\" goto gotJpdaTransport\r\nset JPDA_TRANSPORT=dt_socket\r\n:gotJpdaTransport\r\nif not \"%JPDA_ADDRESS%\" == \"\" goto gotJpdaAddress\r\nset JPDA_ADDRESS=8000\r\n:gotJpdaAddress\r\nif not \"%JPDA_SUSPEND%\" == \"\" goto gotJpdaSuspend\r\nset JPDA_SUSPEND=n\r\n:gotJpdaSuspend\r\nif not \"%JPDA_OPTS%\" == \"\" goto gotJpdaOpts\r\nset JPDA_OPTS=-agentlib:jdwp=transport=%JPDA_TRANSPORT%,address=%JPDA_ADDRESS%,server=y,suspend=%JPDA_SUSPEND%\r\n:gotJpdaOpts\r\nshift\r\n:noJpda\r\n\r\nif \"\"%1\"\" == \"\"debug\"\" goto doDebug\r\nif \"\"%1\"\" == \"\"run\"\" goto doRun\r\nif \"\"%1\"\" == \"\"start\"\" goto doStart\r\nif \"\"%1\"\" == \"\"stop\"\" goto doStop\r\nif \"\"%1\"\" == \"\"configtest\"\" goto doConfigTest\r\nif \"\"%1\"\" == \"\"version\"\" goto doVersion\r\n\r\necho Usage:  catalina ( commands ... )\r\necho commands:\r\necho   debug             Start Catalina in a debugger\r\necho   debug -security   Debug Catalina with a security manager\r\necho   jpda start        Start Catalina under JPDA debugger\r\necho   run               Start Catalina in the current window\r\necho   run -security     Start in the current window with security manager\r\necho   start             Start Catalina in a separate window\r\necho   start -security   Start in a separate window with security manager\r\necho   stop              Stop Catalina\r\necho   configtest        Run a basic syntax check on server.xml\r\necho   version           What version of tomcat are you running?\r\ngoto end\r\n\r\n:doDebug\r\nshift\r\nset _EXECJAVA=%_RUNJDB%\r\nset DEBUG_OPTS=-sourcepath \"%CATALINA_HOME%\\..\\..\\java\"\r\nif not \"\"%1\"\" == \"\"-security\"\" goto execCmd\r\nshift\r\necho Using Security Manager\r\nset \"SECURITY_POLICY_FILE=%CATALINA_BASE%\\conf\\catalina.policy\"\r\ngoto execCmd\r\n\r\n:doRun\r\nshift\r\nif not \"\"%1\"\" == \"\"-security\"\" goto execCmd\r\nshift\r\necho Using Security Manager\r\nset \"SECURITY_POLICY_FILE=%CATALINA_BASE%\\conf\\catalina.policy\"\r\ngoto execCmd\r\n\r\n:doStart\r\nshift\r\nif not \"%OS%\" == \"Windows_NT\" goto noTitle\r\nif \"%TITLE%\" == \"\" set TITLE=Tomcat\r\nset _EXECJAVA=start \"%TITLE%\" %_RUNJAVA%\r\ngoto gotTitle\r\n:noTitle\r\nset _EXECJAVA=start %_RUNJAVA%\r\n:gotTitle\r\nif not \"\"%1\"\" == \"\"-security\"\" goto execCmd\r\nshift\r\necho Using Security Manager\r\nset \"SECURITY_POLICY_FILE=%CATALINA_BASE%\\conf\\catalina.policy\"\r\ngoto execCmd\r\n\r\n:doStop\r\nshift\r\nset ACTION=stop\r\nset CATALINA_OPTS=\r\ngoto execCmd\r\n\r\n:doConfigTest\r\nshift\r\nset ACTION=configtest\r\nset CATALINA_OPTS=\r\ngoto execCmd\r\n\r\n:doVersion\r\n%_EXECJAVA% -classpath \"%CATALINA_HOME%\\lib\\catalina.jar\" org.apache.catalina.util.ServerInfo\r\ngoto end\r\n\r\n\r\n:execCmd\r\nrem Get remaining unshifted command line arguments and save them in the\r\nset CMD_LINE_ARGS=\r\n:setArgs\r\nif \"\"%1\"\"==\"\"\"\" goto doneSetArgs\r\nset CMD_LINE_ARGS=%CMD_LINE_ARGS% %1\r\nshift\r\ngoto setArgs\r\n:doneSetArgs\r\n\r\nrem Execute Java with the applicable properties\r\nif not \"%JPDA%\" == \"\" goto doJpda\r\nif not \"%SECURITY_POLICY_FILE%\" == \"\" goto doSecurity\r\n%_EXECJAVA% %JAVA_OPTS% %CATALINA_OPTS% %DEBUG_OPTS% -Djava.endorsed.dirs=\"%JAVA_ENDORSED_DIRS%\" -classpath \"%CLASSPATH%\" -Dcatalina.base=\"%CATALINA_BASE%\" -Dcatalina.home=\"%CATALINA_HOME%\" -Djava.io.tmpdir=\"%CATALINA_TMPDIR%\" %MAINCLASS% %CMD_LINE_ARGS% %ACTION%\r\ngoto end\r\n:doSecurity\r\n%_EXECJAVA% %JAVA_OPTS% %CATALINA_OPTS% %DEBUG_OPTS% -Djava.endorsed.dirs=\"%JAVA_ENDORSED_DIRS%\" -classpath \"%CLASSPATH%\" -Djava.security.manager -Djava.security.policy==\"%SECURITY_POLICY_FILE%\" -Dcatalina.base=\"%CATALINA_BASE%\" -Dcatalina.home=\"%CATALINA_HOME%\" -Djava.io.tmpdir=\"%CATALINA_TMPDIR%\" %MAINCLASS% %CMD_LINE_ARGS% %ACTION%\r\ngoto end\r\n:doJpda\r\nif not \"%SECURITY_POLICY_FILE%\" == \"\" goto doSecurityJpda\r\n%_EXECJAVA% %JAVA_OPTS% %CATALINA_OPTS% %JPDA_OPTS% %DEBUG_OPTS% -Djava.endorsed.dirs=\"%JAVA_ENDORSED_DIRS%\" -classpath \"%CLASSPATH%\" -Dcatalina.base=\"%CATALINA_BASE%\" -Dcatalina.home=\"%CATALINA_HOME%\" -Djava.io.tmpdir=\"%CATALINA_TMPDIR%\" %MAINCLASS% %CMD_LINE_ARGS% %ACTION%\r\ngoto end\r\n:doSecurityJpda\r\n%_EXECJAVA% %JAVA_OPTS% %CATALINA_OPTS% %JPDA_OPTS% %DEBUG_OPTS% -Djava.endorsed.dirs=\"%JAVA_ENDORSED_DIRS%\" -classpath \"%CLASSPATH%\" -Djava.security.manager -Djava.security.policy==\"%SECURITY_POLICY_FILE%\" -Dcatalina.base=\"%CATALINA_BASE%\" -Dcatalina.home=\"%CATALINA_HOME%\" -Djava.io.tmpdir=\"%CATALINA_TMPDIR%\" %MAINCLASS% %CMD_LINE_ARGS% %ACTION%\r\ngoto end\r\n\r\n:end\r\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/catalina.sh",
    "content": "#!/bin/sh\n\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements.  See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License.  You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# -----------------------------------------------------------------------------\n# Control Script for the CATALINA Server\n#\n# Environment Variable Prerequisites\n#\n#   Do not set the variables in this script. Instead put them into a script\n#   setenv.sh in CATALINA_BASE/bin to keep your customizations separate.\n#\n#   CATALINA_HOME   May point at your Catalina \"build\" directory.\n#\n#   CATALINA_BASE   (Optional) Base directory for resolving dynamic portions\n#                   of a Catalina installation.  If not present, resolves to\n#                   the same directory that CATALINA_HOME points to.\n#\n#   CATALINA_OUT    (Optional) Full path to a file where stdout and stderr\n#                   will be redirected.\n#                   Default is $CATALINA_BASE/logs/catalina.out\n#\n#   CATALINA_OPTS   (Optional) Java runtime options used when the \"start\",\n#                   \"run\" or \"debug\" command is executed.\n#                   Include here and not in JAVA_OPTS all options, that should\n#                   only be used by Tomcat itself, not by the stop process,\n#                   the version command etc.\n#                   Examples are heap size, GC logging, JMX ports etc.\n#\n#   CATALINA_TMPDIR (Optional) Directory path location of temporary directory\n#                   the JVM should use (java.io.tmpdir).  Defaults to\n#                   $CATALINA_BASE/temp.\n#\n#   JAVA_HOME       Must point at your Java Development Kit installation.\n#                   Required to run the with the \"debug\" argument.\n#\n#   JRE_HOME        Must point at your Java Runtime installation.\n#                   Defaults to JAVA_HOME if empty. If JRE_HOME and JAVA_HOME\n#                   are both set, JRE_HOME is used.\n#\n#   JAVA_OPTS       (Optional) Java runtime options used when any command\n#                   is executed.\n#                   Include here and not in CATALINA_OPTS all options, that\n#                   should be used by Tomcat and also by the stop process,\n#                   the version command etc.\n#                   Most options should go into CATALINA_OPTS.\n#\n#   JAVA_ENDORSED_DIRS (Optional) Lists of of colon separated directories\n#                   containing some jars in order to allow replacement of APIs\n#                   created outside of the JCP (i.e. DOM and SAX from W3C).\n#                   It can also be used to update the XML parser implementation.\n#                   Defaults to $CATALINA_HOME/endorsed.\n#\n#   JPDA_TRANSPORT  (Optional) JPDA transport used when the \"jpda start\"\n#                   command is executed. The default is \"dt_socket\".\n#\n#   JPDA_ADDRESS    (Optional) Java runtime options used when the \"jpda start\"\n#                   command is executed. The default is 8000.\n#\n#   JPDA_SUSPEND    (Optional) Java runtime options used when the \"jpda start\"\n#                   command is executed. Specifies whether JVM should suspend\n#                   execution immediately after startup. Default is \"n\".\n#\n#   JPDA_OPTS       (Optional) Java runtime options used when the \"jpda start\"\n#                   command is executed. If used, JPDA_TRANSPORT, JPDA_ADDRESS,\n#                   and JPDA_SUSPEND are ignored. Thus, all required jpda\n#                   options MUST be specified. The default is:\n#\n#                   -agentlib:jdwp=transport=$JPDA_TRANSPORT,\n#                       address=$JPDA_ADDRESS,server=y,suspend=$JPDA_SUSPEND\n#\n#   CATALINA_PID    (Optional) Path of the file which should contains the pid\n#                   of the catalina startup java process, when start (fork) is\n#                   used\n#\n#   LOGGING_CONFIG  (Optional) Override Tomcat's logging config file\n#                   Example (all one line)\n#                   LOGGING_CONFIG=\"-Djava.util.logging.config.file=$CATALINA_BASE/conf/logging.properties\"\n#\n#   LOGGING_MANAGER (Optional) Override Tomcat's logging manager\n#                   Example (all one line)\n#                   LOGGING_MANAGER=\"-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager\"\n#\n# $Id: catalina.sh 1498485 2013-07-01 14:37:43Z markt $\n# -----------------------------------------------------------------------------\n\n# OS specific support.  $var _must_ be set to either true or false.\ncygwin=false\ndarwin=false\nos400=false\ncase \"`uname`\" in\nCYGWIN*) cygwin=true;;\nDarwin*) darwin=true;;\nOS400*) os400=true;;\nesac\n\n# resolve links - $0 may be a softlink\nPRG=\"$0\"\n\nwhile [ -h \"$PRG\" ]; do\n  ls=`ls -ld \"$PRG\"`\n  link=`expr \"$ls\" : '.*-> \\(.*\\)$'`\n  if expr \"$link\" : '/.*' > /dev/null; then\n    PRG=\"$link\"\n  else\n    PRG=`dirname \"$PRG\"`/\"$link\"\n  fi\ndone\n\n# Get standard environment variables\nPRGDIR=`dirname \"$PRG\"`\n\n# Only set CATALINA_HOME if not already set\n[ -z \"$CATALINA_HOME\" ] && CATALINA_HOME=`cd \"$PRGDIR/..\" >/dev/null; pwd`\n\n# Copy CATALINA_BASE from CATALINA_HOME if not already set\n[ -z \"$CATALINA_BASE\" ] && CATALINA_BASE=\"$CATALINA_HOME\"\n\n# Ensure that any user defined CLASSPATH variables are not used on startup,\n# but allow them to be specified in setenv.sh, in rare case when it is needed.\nCLASSPATH=\n\nif [ -r \"$CATALINA_BASE/bin/setenv.sh\" ]; then\n  . \"$CATALINA_BASE/bin/setenv.sh\"\nelif [ -r \"$CATALINA_HOME/bin/setenv.sh\" ]; then\n  . \"$CATALINA_HOME/bin/setenv.sh\"\nfi\n\n# For Cygwin, ensure paths are in UNIX format before anything is touched\nif $cygwin; then\n  [ -n \"$JAVA_HOME\" ] && JAVA_HOME=`cygpath --unix \"$JAVA_HOME\"`\n  [ -n \"$JRE_HOME\" ] && JRE_HOME=`cygpath --unix \"$JRE_HOME\"`\n  [ -n \"$CATALINA_HOME\" ] && CATALINA_HOME=`cygpath --unix \"$CATALINA_HOME\"`\n  [ -n \"$CATALINA_BASE\" ] && CATALINA_BASE=`cygpath --unix \"$CATALINA_BASE\"`\n  [ -n \"$CLASSPATH\" ] && CLASSPATH=`cygpath --path --unix \"$CLASSPATH\"`\nfi\n\n# For OS400\nif $os400; then\n  # Set job priority to standard for interactive (interactive - 6) by using\n  # the interactive priority - 6, the helper threads that respond to requests\n  # will be running at the same priority as interactive jobs.\n  COMMAND='chgjob job('$JOBNAME') runpty(6)'\n  system $COMMAND\n\n  # Enable multi threading\n  export QIBM_MULTI_THREADED=Y\nfi\n\n# Get standard Java environment variables\nif $os400; then\n  # -r will Only work on the os400 if the files are:\n  # 1. owned by the user\n  # 2. owned by the PRIMARY group of the user\n  # this will not work if the user belongs in secondary groups\n  . \"$CATALINA_HOME\"/bin/setclasspath.sh\nelse\n  if [ -r \"$CATALINA_HOME\"/bin/setclasspath.sh ]; then\n    . \"$CATALINA_HOME\"/bin/setclasspath.sh\n  else\n    echo \"Cannot find $CATALINA_HOME/bin/setclasspath.sh\"\n    echo \"This file is needed to run this program\"\n    exit 1\n  fi\nfi\n\n# Add on extra jar files to CLASSPATH\nif [ ! -z \"$CLASSPATH\" ] ; then\n  CLASSPATH=\"$CLASSPATH\":\nfi\nCLASSPATH=\"$CLASSPATH\"\"$CATALINA_HOME\"/bin/bootstrap.jar\n\nif [ -z \"$CATALINA_OUT\" ] ; then\n  CATALINA_OUT=\"$CATALINA_BASE\"/logs/catalina.out\nfi\n\nif [ -z \"$CATALINA_TMPDIR\" ] ; then\n  # Define the java.io.tmpdir to use for Catalina\n  CATALINA_TMPDIR=\"$CATALINA_BASE\"/temp\nfi\n\n# Add tomcat-juli.jar to classpath\n# tomcat-juli.jar can be over-ridden per instance\nif [ -r \"$CATALINA_BASE/bin/tomcat-juli.jar\" ] ; then\n  CLASSPATH=$CLASSPATH:$CATALINA_BASE/bin/tomcat-juli.jar\nelse\n  CLASSPATH=$CLASSPATH:$CATALINA_HOME/bin/tomcat-juli.jar\nfi\n\n# Bugzilla 37848: When no TTY is available, don't output to console\nhave_tty=0\nif [ \"`tty`\" != \"not a tty\" ]; then\n    have_tty=1\nfi\n\n# For Cygwin, switch paths to Windows format before running java\nif $cygwin; then\n  JAVA_HOME=`cygpath --absolute --windows \"$JAVA_HOME\"`\n  JRE_HOME=`cygpath --absolute --windows \"$JRE_HOME\"`\n  CATALINA_HOME=`cygpath --absolute --windows \"$CATALINA_HOME\"`\n  CATALINA_BASE=`cygpath --absolute --windows \"$CATALINA_BASE\"`\n  CATALINA_TMPDIR=`cygpath --absolute --windows \"$CATALINA_TMPDIR\"`\n  CLASSPATH=`cygpath --path --windows \"$CLASSPATH\"`\n  JAVA_ENDORSED_DIRS=`cygpath --path --windows \"$JAVA_ENDORSED_DIRS\"`\nfi\n\n# Set juli LogManager config file if it is present and an override has not been issued\nif [ -z \"$LOGGING_CONFIG\" ]; then\n  if [ -r \"$CATALINA_BASE\"/conf/logging.properties ]; then\n    LOGGING_CONFIG=\"-Djava.util.logging.config.file=$CATALINA_BASE/conf/logging.properties\"\n  else\n    # Bugzilla 45585\n    LOGGING_CONFIG=\"-Dnop\"\n  fi\nfi\n\nif [ -z \"$LOGGING_MANAGER\" ]; then\n  LOGGING_MANAGER=\"-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager\"\nfi\n\n# Uncomment the following line to make the umask available when using the\n# org.apache.catalina.security.SecurityListener\n#JAVA_OPTS=\"$JAVA_OPTS -Dorg.apache.catalina.security.SecurityListener.UMASK=`umask`\"\n\n# ----- Execute The Requested Command -----------------------------------------\n\n# Bugzilla 37848: only output this if we have a TTY\nif [ $have_tty -eq 1 ]; then\n  echo \"Using CATALINA_BASE:   $CATALINA_BASE\"\n  echo \"Using CATALINA_HOME:   $CATALINA_HOME\"\n  echo \"Using CATALINA_TMPDIR: $CATALINA_TMPDIR\"\n  if [ \"$1\" = \"debug\" ] ; then\n    echo \"Using JAVA_HOME:       $JAVA_HOME\"\n  else\n    echo \"Using JRE_HOME:        $JRE_HOME\"\n  fi\n  echo \"Using CLASSPATH:       $CLASSPATH\"\n  if [ ! -z \"$CATALINA_PID\" ]; then\n    echo \"Using CATALINA_PID:    $CATALINA_PID\"\n  fi\nfi\n\nif [ \"$1\" = \"jpda\" ] ; then\n  if [ -z \"$JPDA_TRANSPORT\" ]; then\n    JPDA_TRANSPORT=\"dt_socket\"\n  fi\n  if [ -z \"$JPDA_ADDRESS\" ]; then\n    JPDA_ADDRESS=\"8000\"\n  fi\n  if [ -z \"$JPDA_SUSPEND\" ]; then\n    JPDA_SUSPEND=\"n\"\n  fi\n  if [ -z \"$JPDA_OPTS\" ]; then\n    JPDA_OPTS=\"-agentlib:jdwp=transport=$JPDA_TRANSPORT,address=$JPDA_ADDRESS,server=y,suspend=$JPDA_SUSPEND\"\n  fi\n  CATALINA_OPTS=\"$CATALINA_OPTS $JPDA_OPTS\"\n  shift\nfi\n\nif [ \"$1\" = \"debug\" ] ; then\n  if $os400; then\n    echo \"Debug command not available on OS400\"\n    exit 1\n  else\n    shift\n    if [ \"$1\" = \"-security\" ] ; then\n      if [ $have_tty -eq 1 ]; then\n        echo \"Using Security Manager\"\n      fi\n      shift\n      exec \"$_RUNJDB\" \"$LOGGING_CONFIG\" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \\\n        -Djava.endorsed.dirs=\"$JAVA_ENDORSED_DIRS\" -classpath \"$CLASSPATH\" \\\n        -sourcepath \"$CATALINA_HOME\"/../../java \\\n        -Djava.security.manager \\\n        -Djava.security.policy==\"$CATALINA_BASE\"/conf/catalina.policy \\\n        -Dcatalina.base=\"$CATALINA_BASE\" \\\n        -Dcatalina.home=\"$CATALINA_HOME\" \\\n        -Djava.io.tmpdir=\"$CATALINA_TMPDIR\" \\\n        org.apache.catalina.startup.Bootstrap \"$@\" start\n    else\n      exec \"$_RUNJDB\" \"$LOGGING_CONFIG\" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \\\n        -Djava.endorsed.dirs=\"$JAVA_ENDORSED_DIRS\" -classpath \"$CLASSPATH\" \\\n        -sourcepath \"$CATALINA_HOME\"/../../java \\\n        -Dcatalina.base=\"$CATALINA_BASE\" \\\n        -Dcatalina.home=\"$CATALINA_HOME\" \\\n        -Djava.io.tmpdir=\"$CATALINA_TMPDIR\" \\\n        org.apache.catalina.startup.Bootstrap \"$@\" start\n    fi\n  fi\n\nelif [ \"$1\" = \"run\" ]; then\n\n  shift\n  if [ \"$1\" = \"-security\" ] ; then\n    if [ $have_tty -eq 1 ]; then\n      echo \"Using Security Manager\"\n    fi\n    shift\n    eval exec \\\"$_RUNJAVA\\\" \\\"$LOGGING_CONFIG\\\" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \\\n      -Djava.endorsed.dirs=\\\"$JAVA_ENDORSED_DIRS\\\" -classpath \\\"$CLASSPATH\\\" \\\n      -Djava.security.manager \\\n      -Djava.security.policy==\\\"$CATALINA_BASE/conf/catalina.policy\\\" \\\n      -Dcatalina.base=\\\"$CATALINA_BASE\\\" \\\n      -Dcatalina.home=\\\"$CATALINA_HOME\\\" \\\n      -Djava.io.tmpdir=\\\"$CATALINA_TMPDIR\\\" \\\n      org.apache.catalina.startup.Bootstrap \"$@\" start\n  else\n    eval exec \\\"$_RUNJAVA\\\" \\\"$LOGGING_CONFIG\\\" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \\\n      -Djava.endorsed.dirs=\\\"$JAVA_ENDORSED_DIRS\\\" -classpath \\\"$CLASSPATH\\\" \\\n      -Dcatalina.base=\\\"$CATALINA_BASE\\\" \\\n      -Dcatalina.home=\\\"$CATALINA_HOME\\\" \\\n      -Djava.io.tmpdir=\\\"$CATALINA_TMPDIR\\\" \\\n      org.apache.catalina.startup.Bootstrap \"$@\" start\n  fi\n\nelif [ \"$1\" = \"start\" ] ; then\n\n  if [ ! -z \"$CATALINA_PID\" ]; then\n    if [ -f \"$CATALINA_PID\" ]; then\n      if [ -s \"$CATALINA_PID\" ]; then\n        echo \"Existing PID file found during start.\"\n        if [ -r \"$CATALINA_PID\" ]; then\n          PID=`cat \"$CATALINA_PID\"`\n          ps -p $PID >/dev/null 2>&1\n          if [ $? -eq 0 ] ; then\n            echo \"Tomcat appears to still be running with PID $PID. Start aborted.\"\n            exit 1\n          else\n            echo \"Removing/clearing stale PID file.\"\n            rm -f \"$CATALINA_PID\" >/dev/null 2>&1\n            if [ $? != 0 ]; then\n              if [ -w \"$CATALINA_PID\" ]; then\n                cat /dev/null > \"$CATALINA_PID\"\n              else\n                echo \"Unable to remove or clear stale PID file. Start aborted.\"\n                exit 1\n              fi\n            fi\n          fi\n        else\n          echo \"Unable to read PID file. Start aborted.\"\n          exit 1\n        fi\n      else\n        rm -f \"$CATALINA_PID\" >/dev/null 2>&1\n        if [ $? != 0 ]; then\n          if [ ! -w \"$CATALINA_PID\" ]; then\n            echo \"Unable to remove or write to empty PID file. Start aborted.\"\n            exit 1\n          fi\n        fi\n      fi\n    fi\n  fi\n\n  shift\n  touch \"$CATALINA_OUT\"\n  if [ \"$1\" = \"-security\" ] ; then\n    if [ $have_tty -eq 1 ]; then\n      echo \"Using Security Manager\"\n    fi\n    shift\n    eval \\\"$_RUNJAVA\\\" \\\"$LOGGING_CONFIG\\\" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \\\n      -Djava.endorsed.dirs=\\\"$JAVA_ENDORSED_DIRS\\\" -classpath \\\"$CLASSPATH\\\" \\\n      -Djava.security.manager \\\n      -Djava.security.policy==\\\"$CATALINA_BASE/conf/catalina.policy\\\" \\\n      -Dcatalina.base=\\\"$CATALINA_BASE\\\" \\\n      -Dcatalina.home=\\\"$CATALINA_HOME\\\" \\\n      -Djava.io.tmpdir=\\\"$CATALINA_TMPDIR\\\" \\\n      org.apache.catalina.startup.Bootstrap \"$@\" start \\\n      >> \"$CATALINA_OUT\" 2>&1 \"&\"\n\n  else\n    eval \\\"$_RUNJAVA\\\" \\\"$LOGGING_CONFIG\\\" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \\\n      -Djava.endorsed.dirs=\\\"$JAVA_ENDORSED_DIRS\\\" -classpath \\\"$CLASSPATH\\\" \\\n      -Dcatalina.base=\\\"$CATALINA_BASE\\\" \\\n      -Dcatalina.home=\\\"$CATALINA_HOME\\\" \\\n      -Djava.io.tmpdir=\\\"$CATALINA_TMPDIR\\\" \\\n      org.apache.catalina.startup.Bootstrap \"$@\" start \\\n      >> \"$CATALINA_OUT\" 2>&1 \"&\"\n\n  fi\n\n  if [ ! -z \"$CATALINA_PID\" ]; then\n    echo $! > \"$CATALINA_PID\"\n  fi\n\nelif [ \"$1\" = \"stop\" ] ; then\n\n  shift\n\n  SLEEP=5\n  if [ ! -z \"$1\" ]; then\n    echo $1 | grep \"[^0-9]\" >/dev/null 2>&1\n    if [ $? -gt 0 ]; then\n      SLEEP=$1\n      shift\n    fi\n  fi\n\n  FORCE=0\n  if [ \"$1\" = \"-force\" ]; then\n    shift\n    FORCE=1\n  fi\n\n  if [ ! -z \"$CATALINA_PID\" ]; then\n    if [ -f \"$CATALINA_PID\" ]; then\n      if [ -s \"$CATALINA_PID\" ]; then\n        kill -0 `cat \"$CATALINA_PID\"` >/dev/null 2>&1\n        if [ $? -gt 0 ]; then\n          echo \"PID file found but no matching process was found. Stop aborted.\"\n          exit 1\n        fi\n      else\n        echo \"PID file is empty and has been ignored.\"\n      fi\n    else\n      echo \"\\$CATALINA_PID was set but the specified file does not exist. Is Tomcat running? Stop aborted.\"\n      exit 1\n    fi\n  fi\n\n  eval \\\"$_RUNJAVA\\\" $LOGGING_MANAGER $JAVA_OPTS \\\n    -Djava.endorsed.dirs=\\\"$JAVA_ENDORSED_DIRS\\\" -classpath \\\"$CLASSPATH\\\" \\\n    -Dcatalina.base=\\\"$CATALINA_BASE\\\" \\\n    -Dcatalina.home=\\\"$CATALINA_HOME\\\" \\\n    -Djava.io.tmpdir=\\\"$CATALINA_TMPDIR\\\" \\\n    org.apache.catalina.startup.Bootstrap \"$@\" stop\n\n  if [ ! -z \"$CATALINA_PID\" ]; then\n    if [ -f \"$CATALINA_PID\" ]; then\n      while [ $SLEEP -ge 0 ]; do\n        kill -0 `cat \"$CATALINA_PID\"` >/dev/null 2>&1\n        if [ $? -gt 0 ]; then\n          rm -f \"$CATALINA_PID\" >/dev/null 2>&1\n          if [ $? != 0 ]; then\n            if [ -w \"$CATALINA_PID\" ]; then\n              cat /dev/null > \"$CATALINA_PID\"\n            else\n              echo \"Tomcat stopped but the PID file could not be removed or cleared.\"\n            fi\n          fi\n          break\n        fi\n        if [ $SLEEP -gt 0 ]; then\n          sleep 1\n        fi\n        if [ $SLEEP -eq 0 ]; then\n          if [ $FORCE -eq 0 ]; then\n            echo \"Tomcat did not stop in time. PID file was not removed.\"\n          fi\n        fi\n        SLEEP=`expr $SLEEP - 1 `\n      done\n    fi\n  fi\n\n  KILL_SLEEP_INTERVAL=5\n  if [ $FORCE -eq 1 ]; then\n    if [ -z \"$CATALINA_PID\" ]; then\n      echo \"Kill failed: \\$CATALINA_PID not set\"\n    else\n      if [ -f \"$CATALINA_PID\" ]; then\n        PID=`cat \"$CATALINA_PID\"`\n        echo \"Killing Tomcat with the PID: $PID\"\n        kill -9 $PID\n        while [ $KILL_SLEEP_INTERVAL -ge 0 ]; do\n            kill -0 `cat \"$CATALINA_PID\"` >/dev/null 2>&1\n            if [ $? -gt 0 ]; then\n                rm -f \"$CATALINA_PID\" >/dev/null 2>&1\n                if [ $? != 0 ]; then\n                    echo \"Tomcat was killed but the PID file could not be removed.\"\n                fi\n                break\n            fi\n            if [ $KILL_SLEEP_INTERVAL -gt 0 ]; then\n                sleep 1\n            fi\n            KILL_SLEEP_INTERVAL=`expr $KILL_SLEEP_INTERVAL - 1 `\n        done\n        if [ $KILL_SLEEP_INTERVAL -gt 0 ]; then\n            echo \"Tomcat has not been killed completely yet. The process might be waiting on some system call or might be UNINTERRUPTIBLE.\"\n        fi\n      fi\n    fi\n  fi\n\nelif [ \"$1\" = \"configtest\" ] ; then\n\n    eval \\\"$_RUNJAVA\\\" $LOGGING_MANAGER $JAVA_OPTS \\\n      -Djava.endorsed.dirs=\\\"$JAVA_ENDORSED_DIRS\\\" -classpath \\\"$CLASSPATH\\\" \\\n      -Dcatalina.base=\\\"$CATALINA_BASE\\\" \\\n      -Dcatalina.home=\\\"$CATALINA_HOME\\\" \\\n      -Djava.io.tmpdir=\\\"$CATALINA_TMPDIR\\\" \\\n      org.apache.catalina.startup.Bootstrap configtest\n    result=$?\n    if [ $result -ne 0 ]; then\n        echo \"Configuration error detected!\"\n    fi\n    exit $result\n\nelif [ \"$1\" = \"version\" ] ; then\n\n    \"$_RUNJAVA\"   \\\n      -classpath \"$CATALINA_HOME/lib/catalina.jar\" \\\n      org.apache.catalina.util.ServerInfo\n\nelse\n\n  echo \"Usage: catalina.sh ( commands ... )\"\n  echo \"commands:\"\n  if $os400; then\n    echo \"  debug             Start Catalina in a debugger (not available on OS400)\"\n    echo \"  debug -security   Debug Catalina with a security manager (not available on OS400)\"\n  else\n    echo \"  debug             Start Catalina in a debugger\"\n    echo \"  debug -security   Debug Catalina with a security manager\"\n  fi\n  echo \"  jpda start        Start Catalina under JPDA debugger\"\n  echo \"  run               Start Catalina in the current window\"\n  echo \"  run -security     Start in the current window with security manager\"\n  echo \"  start             Start Catalina in a separate window\"\n  echo \"  start -security   Start in a separate window with security manager\"\n  echo \"  stop              Stop Catalina, waiting up to 5 seconds for the process to end\"\n  echo \"  stop n            Stop Catalina, waiting up to n seconds for the process to end\"\n  echo \"  stop -force       Stop Catalina, wait up to 5 seconds and then use kill -KILL if still running\"\n  echo \"  stop n -force     Stop Catalina, wait up to n seconds and then use kill -KILL if still running\"\n  echo \"  configtest        Run a basic syntax check on server.xml - check exit code for result\"\n  echo \"  version           What version of tomcat are you running?\"\n  echo \"Note: Waiting for the process to end and use of the -force option require that \\$CATALINA_PID is defined\"\n  exit 1\n\nfi\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/configtest.bat",
    "content": "@echo off\r\nrem Licensed to the Apache Software Foundation (ASF) under one or more\r\nrem contributor license agreements.  See the NOTICE file distributed with\r\nrem this work for additional information regarding copyright ownership.\r\nrem The ASF licenses this file to You under the Apache License, Version 2.0\r\nrem (the \"License\"); you may not use this file except in compliance with\r\nrem the License.  You may obtain a copy of the License at\r\nrem\r\nrem     http://www.apache.org/licenses/LICENSE-2.0\r\nrem\r\nrem Unless required by applicable law or agreed to in writing, software\r\nrem distributed under the License is distributed on an \"AS IS\" BASIS,\r\nrem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nrem See the License for the specific language governing permissions and\r\nrem limitations under the License.\r\n\r\nif \"%OS%\" == \"Windows_NT\" setlocal\r\nrem ---------------------------------------------------------------------------\r\nrem Configuration test script for the CATALINA Server\r\nrem\r\nrem $Id: configtest.bat 1137560 2011-06-20 09:27:44Z rjung $\r\nrem ---------------------------------------------------------------------------\r\n\r\nrem Guess CATALINA_HOME if not defined\r\nset \"CURRENT_DIR=%cd%\"\r\nif not \"%CATALINA_HOME%\" == \"\" goto gotHome\r\nset \"CATALINA_HOME=%CURRENT_DIR%\"\r\nif exist \"%CATALINA_HOME%\\bin\\catalina.bat\" goto okHome\r\ncd ..\r\nset \"CATALINA_HOME=%cd%\"\r\ncd \"%CURRENT_DIR%\"\r\n:gotHome\r\nif exist \"%CATALINA_HOME%\\bin\\catalina.bat\" goto okHome\r\necho The CATALINA_HOME environment variable is not defined correctly\r\necho This environment variable is needed to run this program\r\ngoto end\r\n:okHome\r\n\r\nset \"EXECUTABLE=%CATALINA_HOME%\\bin\\catalina.bat\"\r\n\r\nrem Check that target executable exists\r\nif exist \"%EXECUTABLE%\" goto okExec\r\necho Cannot find \"%EXECUTABLE%\"\r\necho This file is needed to run this program\r\ngoto end\r\n:okExec\r\n\r\nrem Get remaining unshifted command line arguments and save them in the\r\nset CMD_LINE_ARGS=\r\n:setArgs\r\nif \"\"%1\"\"==\"\"\"\" goto doneSetArgs\r\nset CMD_LINE_ARGS=%CMD_LINE_ARGS% %1\r\nshift\r\ngoto setArgs\r\n:doneSetArgs\r\n\r\ncall \"%EXECUTABLE%\" configtest %CMD_LINE_ARGS%\r\n\r\n:end\r\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/configtest.sh",
    "content": "#!/bin/sh\n\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements.  See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License.  You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# -----------------------------------------------------------------------------\n# Configuration Test Script for the CATALINA Server\n#\n# $Id: configtest.sh 1202062 2011-11-15 06:50:02Z mturk $\n# -----------------------------------------------------------------------------\n\n# Better OS/400 detection: see Bugzilla 31132\nos400=false\ncase \"`uname`\" in\nOS400*) os400=true;;\nesac\n\n# resolve links - $0 may be a softlink\nPRG=\"$0\"\n\nwhile [ -h \"$PRG\" ] ; do\n  ls=`ls -ld \"$PRG\"`\n  link=`expr \"$ls\" : '.*-> \\(.*\\)$'`\n  if expr \"$link\" : '/.*' > /dev/null; then\n    PRG=\"$link\"\n  else\n    PRG=`dirname \"$PRG\"`/\"$link\"\n  fi\ndone\n\nPRGDIR=`dirname \"$PRG\"`\nEXECUTABLE=catalina.sh\n\n# Check that target executable exists\nif $os400; then\n  # -x will Only work on the os400 if the files are:\n  # 1. owned by the user\n  # 2. owned by the PRIMARY group of the user\n  # this will not work if the user belongs in secondary groups\n  eval\nelse\n  if [ ! -x \"$PRGDIR\"/\"$EXECUTABLE\" ]; then\n    echo \"Cannot find $PRGDIR/$EXECUTABLE\"\n    echo \"The file is absent or does not have execute permission\"\n    echo \"This file is needed to run this program\"\n    exit 1\n  fi\nfi\n\nexec \"$PRGDIR\"/\"$EXECUTABLE\" configtest \"$@\"\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/cpappend.bat",
    "content": "@echo off\r\nrem Licensed to the Apache Software Foundation (ASF) under one or more\r\nrem contributor license agreements.  See the NOTICE file distributed with\r\nrem this work for additional information regarding copyright ownership.\r\nrem The ASF licenses this file to You under the Apache License, Version 2.0\r\nrem (the \"License\"); you may not use this file except in compliance with\r\nrem the License.  You may obtain a copy of the License at\r\nrem\r\nrem     http://www.apache.org/licenses/LICENSE-2.0\r\nrem\r\nrem Unless required by applicable law or agreed to in writing, software\r\nrem distributed under the License is distributed on an \"AS IS\" BASIS,\r\nrem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nrem See the License for the specific language governing permissions and\r\nrem limitations under the License.\r\n\r\nrem ---------------------------------------------------------------------------\r\nrem Append to CLASSPATH\r\nrem\r\nrem $Id: cpappend.bat 562770 2007-08-04 22:13:58Z markt $\r\nrem ---------------------------------------------------------------------------\r\n\r\nrem Process the first argument\r\nif \"\"%1\"\" == \"\"\"\" goto end\r\nset CLASSPATH=%CLASSPATH%;%1\r\nshift\r\n\r\nrem Process the remaining arguments\r\n:setArgs\r\nif \"\"%1\"\" == \"\"\"\" goto doneSetArgs\r\nset CLASSPATH=%CLASSPATH% %1\r\nshift\r\ngoto setArgs\r\n:doneSetArgs\r\n:end\r\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/daemon.sh",
    "content": "#!/bin/sh\n\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements.  See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License.  You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n# -----------------------------------------------------------------------------\n# Commons Daemon wrapper script.\n#\n# $Id: daemon.sh 1202058 2011-11-15 06:37:12Z mturk $\n# -----------------------------------------------------------------------------\n#\n# resolve links - $0 may be a softlink\nARG0=\"$0\"\nwhile [ -h \"$ARG0\" ]; do\n  ls=`ls -ld \"$ARG0\"`\n  link=`expr \"$ls\" : '.*-> \\(.*\\)$'`\n  if expr \"$link\" : '/.*' > /dev/null; then\n    ARG0=\"$link\"\n  else\n    ARG0=\"`dirname $ARG0`/$link\"\n  fi\ndone\nDIRNAME=\"`dirname $ARG0`\"\nPROGRAM=\"`basename $ARG0`\"\nwhile [ \".$1\" != . ]\ndo\n  case \"$1\" in\n    --java-home )\n        JAVA_HOME=\"$2\"\n        shift; shift;\n        continue\n    ;;\n    --catalina-home )\n        CATALINA_HOME=\"$2\"\n        shift; shift;\n        continue\n    ;;\n    --catalina-base )\n        CATALINA_BASE=\"$2\"\n        shift; shift;\n        continue\n    ;;\n    --catalina-pid )\n        CATALINA_PID=\"$2\"\n        shift; shift;\n        continue\n    ;;\n    --tomcat-user )\n        TOMCAT_USER=\"$2\"\n        shift; shift;\n        continue\n    ;;\n    * )\n        break\n    ;;\n  esac\ndone\n# OS specific support (must be 'true' or 'false').\ncygwin=false;\ndarwin=false;\ncase \"`uname`\" in\n    CYGWIN*)\n        cygwin=true\n        ;;\n    Darwin*)\n        darwin=true\n        ;;\nesac\n\n# Use the maximum available, or set MAX_FD != -1 to use that\ntest \".$MAX_FD\" = . && MAX_FD=\"maximum\"\n# Setup parameters for running the jsvc\n#\ntest \".$TOMCAT_USER\" = . && TOMCAT_USER=tomcat\n# Set JAVA_HOME to working JDK or JRE\n# JAVA_HOME=/opt/jdk-1.6.0.22\n# If not set we'll try to guess the JAVA_HOME\n# from java binary if on the PATH\n#\nif [ -z \"$JAVA_HOME\" ]; then\n    JAVA_BIN=\"`which java 2>/dev/null || type java 2>&1`\"\n    test -x \"$JAVA_BIN\" && JAVA_HOME=\"`dirname $JAVA_BIN`\"\n    test \".$JAVA_HOME\" != . && JAVA_HOME=`cd \"$JAVA_HOME/..\" >/dev/null; pwd`\nelse\n    JAVA_BIN=\"$JAVA_HOME/bin/java\"\nfi\n\n# Only set CATALINA_HOME if not already set\ntest \".$CATALINA_HOME\" = . && CATALINA_HOME=`cd \"$DIRNAME/..\" >/dev/null; pwd`\ntest \".$CATALINA_BASE\" = . && CATALINA_BASE=\"$CATALINA_HOME\"\ntest \".$CATALINA_MAIN\" = . && CATALINA_MAIN=org.apache.catalina.startup.Bootstrap\ntest \".$JSVC\" = . && JSVC=\"$CATALINA_BASE/bin/jsvc\"\n\n# Ensure that any user defined CLASSPATH variables are not used on startup,\n# but allow them to be specified in setenv.sh, in rare case when it is needed.\nCLASSPATH=\nJAVA_OPTS=\nif [ -r \"$CATALINA_BASE/bin/setenv.sh\" ]; then\n  . \"$CATALINA_BASE/bin/setenv.sh\"\nelif [ -r \"$CATALINA_HOME/bin/setenv.sh\" ]; then\n  . \"$CATALINA_HOME/bin/setenv.sh\"\nfi\n\n# Add on extra jar files to CLASSPATH\ntest \".$CLASSPATH\" != . && CLASSPATH=\"${CLASSPATH}:\"\nCLASSPATH=\"$CLASSPATH$CATALINA_HOME/bin/bootstrap.jar:$CATALINA_HOME/bin/commons-daemon.jar\"\n\ntest \".$CATALINA_OUT\" = . && CATALINA_OUT=\"$CATALINA_BASE/logs/catalina-daemon.out\"\ntest \".$CATALINA_TMP\" = . && CATALINA_TMP=\"$CATALINA_BASE/temp\"\n\n# Add tomcat-juli.jar to classpath\n# tomcat-juli.jar can be over-ridden per instance\nif [ -r \"$CATALINA_BASE/bin/tomcat-juli.jar\" ] ; then\n  CLASSPATH=\"$CLASSPATH:$CATALINA_BASE/bin/tomcat-juli.jar\"\nelse\n  CLASSPATH=\"$CLASSPATH:$CATALINA_HOME/bin/tomcat-juli.jar\"\nfi\n# Set juli LogManager config file if it is present and an override has not been issued\nif [ -z \"$LOGGING_CONFIG\" ]; then\n  if [ -r \"$CATALINA_BASE/conf/logging.properties\" ]; then\n    LOGGING_CONFIG=\"-Djava.util.logging.config.file=$CATALINA_BASE/conf/logging.properties\"\n  else\n    # Bugzilla 45585\n    LOGGING_CONFIG=\"-Dnop\"\n  fi\nfi\n\ntest \".$LOGGING_MANAGER\" = . && LOGGING_MANAGER=\"-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager\"\nJAVA_OPTS=\"$JAVA_OPTS $LOGGING_MANAGER\"\n\n# Set -pidfile\ntest \".$CATALINA_PID\" = . && CATALINA_PID=\"$CATALINA_BASE/logs/catalina-daemon.pid\"\n\n# Increase the maximum file descriptors if we can\nif [ \"$cygwin\" = \"false\" ]; then\n    MAX_FD_LIMIT=`ulimit -H -n`\n    if [ \"$?\" -eq 0 ]; then\n        # Darwin does not allow RLIMIT_INFINITY on file soft limit\n        if [ \"$darwin\" = \"true\" -a \"$MAX_FD_LIMIT\" = \"unlimited\" ]; then\n            MAX_FD_LIMIT=`/usr/sbin/sysctl -n kern.maxfilesperproc`\n        fi\n        test \".$MAX_FD\" = \".maximum\" && MAX_FD=\"$MAX_FD_LIMIT\"\n        ulimit -n $MAX_FD\n        if [ \"$?\" -ne 0 ]; then\n            echo \"$PROGRAM: Could not set maximum file descriptor limit: $MAX_FD\"\n        fi\n    else\n        echo \"$PROGRAM: Could not query system maximum file descriptor limit: $MAX_FD_LIMIT\"\n    fi\nfi\n\n# ----- Execute The Requested Command -----------------------------------------\ncase \"$1\" in\n    run     )\n      shift\n      \"$JSVC\" $* \\\n      $JSVC_OPTS \\\n      -java-home \"$JAVA_HOME\" \\\n      -pidfile \"$CATALINA_PID\" \\\n      -wait 10 \\\n      -nodetach \\\n      -outfile \"&1\" \\\n      -errfile \"&2\" \\\n      -classpath \"$CLASSPATH\" \\\n      \"$LOGGING_CONFIG\" $JAVA_OPTS $CATALINA_OPTS \\\n      -Djava.endorsed.dirs=\"$JAVA_ENDORSED_DIRS\" \\\n      -Dcatalina.base=\"$CATALINA_BASE\" \\\n      -Dcatalina.home=\"$CATALINA_HOME\" \\\n      -Djava.io.tmpdir=\"$CATALINA_TMP\" \\\n      $CATALINA_MAIN\n      exit $?\n    ;;\n    start   )\n      \"$JSVC\" $JSVC_OPTS \\\n      -java-home \"$JAVA_HOME\" \\\n      -user $TOMCAT_USER \\\n      -pidfile \"$CATALINA_PID\" \\\n      -wait 10 \\\n      -outfile \"$CATALINA_OUT\" \\\n      -errfile \"&1\" \\\n      -classpath \"$CLASSPATH\" \\\n      \"$LOGGING_CONFIG\" $JAVA_OPTS $CATALINA_OPTS \\\n      -Djava.endorsed.dirs=\"$JAVA_ENDORSED_DIRS\" \\\n      -Dcatalina.base=\"$CATALINA_BASE\" \\\n      -Dcatalina.home=\"$CATALINA_HOME\" \\\n      -Djava.io.tmpdir=\"$CATALINA_TMP\" \\\n      $CATALINA_MAIN\n      exit $?\n    ;;\n    stop    )\n      \"$JSVC\" $JSVC_OPTS \\\n      -stop \\\n      -pidfile \"$CATALINA_PID\" \\\n      -classpath \"$CLASSPATH\" \\\n      -Djava.endorsed.dirs=\"$JAVA_ENDORSED_DIRS\" \\\n      -Dcatalina.base=\"$CATALINA_BASE\" \\\n      -Dcatalina.home=\"$CATALINA_HOME\" \\\n      -Djava.io.tmpdir=\"$CATALINA_TMP\" \\\n      $CATALINA_MAIN\n      exit $?\n    ;;\n    version  )\n      \"$JSVC\" \\\n      -java-home \"$JAVA_HOME\" \\\n      -pidfile \"$CATALINA_PID\" \\\n      -classpath \"$CLASSPATH\" \\\n      -errfile \"&2\" \\\n      -version \\\n      -check \\\n      $CATALINA_MAIN\n      if [ \"$?\" = 0 ]; then\n        \"$JAVA_BIN\" \\\n        -classpath \"$CATALINA_HOME/lib/catalina.jar\" \\\n        org.apache.catalina.util.ServerInfo\n      fi\n      exit $?\n    ;;\n    *       )\n      echo \"Unknown command: \\`$1'\"\n      echo \"Usage: $PROGRAM ( commands ... )\"\n      echo \"commands:\"\n      echo \"  run               Start Tomcat without detaching from console\"\n      echo \"  start             Start Tomcat\"\n      echo \"  stop              Stop Tomcat\"\n      echo \"  version           What version of commons daemon and Tomcat\"\n      echo \"                    are you running?\"\n      exit 1\n    ;;\nesac\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/digest.bat",
    "content": "@echo off\r\nrem Licensed to the Apache Software Foundation (ASF) under one or more\r\nrem contributor license agreements.  See the NOTICE file distributed with\r\nrem this work for additional information regarding copyright ownership.\r\nrem The ASF licenses this file to You under the Apache License, Version 2.0\r\nrem (the \"License\"); you may not use this file except in compliance with\r\nrem the License.  You may obtain a copy of the License at\r\nrem\r\nrem     http://www.apache.org/licenses/LICENSE-2.0\r\nrem\r\nrem Unless required by applicable law or agreed to in writing, software\r\nrem distributed under the License is distributed on an \"AS IS\" BASIS,\r\nrem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nrem See the License for the specific language governing permissions and\r\nrem limitations under the License.\r\n\r\nif \"%OS%\" == \"Windows_NT\" setlocal\r\nrem ---------------------------------------------------------------------------\r\nrem Script to digest password using the algorithm specified\r\nrem\r\nrem $Id: digest.bat 1137559 2011-06-20 09:27:30Z rjung $\r\nrem ---------------------------------------------------------------------------\r\n\r\nrem Guess CATALINA_HOME if not defined\r\nset \"CURRENT_DIR=%cd%\"\r\nif not \"%CATALINA_HOME%\" == \"\" goto gotHome\r\nset \"CATALINA_HOME=%CURRENT_DIR%\"\r\nif exist \"%CATALINA_HOME%\\bin\\tool-wrapper.bat\" goto okHome\r\ncd ..\r\nset \"CATALINA_HOME=%cd%\"\r\ncd \"%CURRENT_DIR%\"\r\n:gotHome\r\nif exist \"%CATALINA_HOME%\\bin\\tool-wrapper.bat\" goto okHome\r\necho The CATALINA_HOME environment variable is not defined correctly\r\necho This environment variable is needed to run this program\r\ngoto end\r\n:okHome\r\n\r\nset \"EXECUTABLE=%CATALINA_HOME%\\bin\\tool-wrapper.bat\"\r\n\r\nrem Check that target executable exists\r\nif exist \"%EXECUTABLE%\" goto okExec\r\necho Cannot find \"%EXECUTABLE%\"\r\necho This file is needed to run this program\r\ngoto end\r\n:okExec\r\n\r\nrem Get remaining unshifted command line arguments and save them in the\r\nset CMD_LINE_ARGS=\r\n:setArgs\r\nif \"\"%1\"\"==\"\"\"\" goto doneSetArgs\r\nset CMD_LINE_ARGS=%CMD_LINE_ARGS% %1\r\nshift\r\ngoto setArgs\r\n:doneSetArgs\r\n\r\ncall \"%EXECUTABLE%\" -server org.apache.catalina.realm.RealmBase %CMD_LINE_ARGS%\r\n\r\n:end\r\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/digest.sh",
    "content": "#!/bin/sh\n\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements.  See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License.  You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# -----------------------------------------------------------------------------\n# Script to digest password using the algorithm specified\n#\n# $Id: digest.sh 1202062 2011-11-15 06:50:02Z mturk $\n# -----------------------------------------------------------------------------\n\n# Better OS/400 detection: see Bugzilla 31132\nos400=false\ncase \"`uname`\" in\nOS400*) os400=true;;\nesac\n\n# resolve links - $0 may be a softlink\nPRG=\"$0\"\n\nwhile [ -h \"$PRG\" ] ; do\n  ls=`ls -ld \"$PRG\"`\n  link=`expr \"$ls\" : '.*-> \\(.*\\)$'`\n  if expr \"$link\" : '/.*' > /dev/null; then\n    PRG=\"$link\"\n  else\n    PRG=`dirname \"$PRG\"`/\"$link\"\n  fi\ndone\n\nPRGDIR=`dirname \"$PRG\"`\nEXECUTABLE=tool-wrapper.sh\n\n# Check that target executable exists\nif $os400; then\n  # -x will Only work on the os400 if the files are:\n  # 1. owned by the user\n  # 2. owned by the PRIMARY group of the user\n  # this will not work if the user belongs in secondary groups\n  eval\nelse\n  if [ ! -x \"$PRGDIR\"/\"$EXECUTABLE\" ]; then\n    echo \"Cannot find $PRGDIR/$EXECUTABLE\"\n    echo \"The file is absent or does not have execute permission\"\n    echo \"This file is needed to run this program\"\n    exit 1\n  fi\nfi\n\nexec \"$PRGDIR\"/\"$EXECUTABLE\" -server org.apache.catalina.realm.RealmBase \"$@\"\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/setclasspath.bat",
    "content": "@echo off\r\nrem Licensed to the Apache Software Foundation (ASF) under one or more\r\nrem contributor license agreements.  See the NOTICE file distributed with\r\nrem this work for additional information regarding copyright ownership.\r\nrem The ASF licenses this file to You under the Apache License, Version 2.0\r\nrem (the \"License\"); you may not use this file except in compliance with\r\nrem the License.  You may obtain a copy of the License at\r\nrem\r\nrem     http://www.apache.org/licenses/LICENSE-2.0\r\nrem\r\nrem Unless required by applicable law or agreed to in writing, software\r\nrem distributed under the License is distributed on an \"AS IS\" BASIS,\r\nrem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nrem See the License for the specific language governing permissions and\r\nrem limitations under the License.\r\n\r\nrem ---------------------------------------------------------------------------\r\nrem Set JAVA_HOME or JRE_HOME if not already set, ensure any provided settings\r\nrem are valid and consistent with the selected start-up options and set up the\r\nrem endorsed directory.\r\nrem\r\nrem $Id: setclasspath.bat 1202062 2011-11-15 06:50:02Z mturk $\r\nrem ---------------------------------------------------------------------------\r\n\r\nrem Make sure prerequisite environment variables are set\r\n\r\nrem In debug mode we need a real JDK (JAVA_HOME)\r\nif \"\"%1\"\" == \"\"debug\"\" goto needJavaHome\r\n\r\nrem Otherwise either JRE or JDK are fine\r\nif not \"%JRE_HOME%\" == \"\" goto gotJreHome\r\nif not \"%JAVA_HOME%\" == \"\" goto gotJavaHome\r\necho Neither the JAVA_HOME nor the JRE_HOME environment variable is defined\r\necho At least one of these environment variable is needed to run this program\r\ngoto exit\r\n\r\n:needJavaHome\r\nrem Check if we have a usable JDK\r\nif \"%JAVA_HOME%\" == \"\" goto noJavaHome\r\nif not exist \"%JAVA_HOME%\\bin\\java.exe\" goto noJavaHome\r\nif not exist \"%JAVA_HOME%\\bin\\javaw.exe\" goto noJavaHome\r\nif not exist \"%JAVA_HOME%\\bin\\jdb.exe\" goto noJavaHome\r\nif not exist \"%JAVA_HOME%\\bin\\javac.exe\" goto noJavaHome\r\nset \"JRE_HOME=%JAVA_HOME%\"\r\ngoto okJava\r\n\r\n:noJavaHome\r\necho The JAVA_HOME environment variable is not defined correctly.\r\necho It is needed to run this program in debug mode.\r\necho NB: JAVA_HOME should point to a JDK not a JRE.\r\ngoto exit\r\n\r\n:gotJavaHome\r\nrem No JRE given, use JAVA_HOME as JRE_HOME\r\nset \"JRE_HOME=%JAVA_HOME%\"\r\n\r\n:gotJreHome\r\nrem Check if we have a usable JRE\r\nif not exist \"%JRE_HOME%\\bin\\java.exe\" goto noJreHome\r\nif not exist \"%JRE_HOME%\\bin\\javaw.exe\" goto noJreHome\r\ngoto okJava\r\n\r\n:noJreHome\r\nrem Needed at least a JRE\r\necho The JRE_HOME environment variable is not defined correctly\r\necho This environment variable is needed to run this program\r\ngoto exit\r\n\r\n:okJava\r\nrem Don't override the endorsed dir if the user has set it previously\r\nif not \"%JAVA_ENDORSED_DIRS%\" == \"\" goto gotEndorseddir\r\nrem Set the default -Djava.endorsed.dirs argument\r\nset \"JAVA_ENDORSED_DIRS=%CATALINA_HOME%\\endorsed\"\r\n:gotEndorseddir\r\n\r\nrem Set standard command for invoking Java.\r\nrem Note that NT requires a window name argument when using start.\r\nrem Also note the quoting as JAVA_HOME may contain spaces.\r\nset _RUNJAVA=\"%JRE_HOME%\\bin\\java\"\r\nset _RUNJDB=\"%JAVA_HOME%\\bin\\jdb\"\r\n\r\ngoto end\r\n\r\n:exit\r\nexit /b 1\r\n\r\n:end\r\nexit /b 0\r\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/setclasspath.sh",
    "content": "#!/bin/sh\n\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements.  See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License.  You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# -----------------------------------------------------------------------------\n#  Set JAVA_HOME or JRE_HOME if not already set, ensure any provided settings\n#  are valid and consistent with the selected start-up options and set up the\n#  endorsed directory.\n#\n#  $Id: setclasspath.sh 1430568 2013-01-08 22:08:57Z schultz $\n# -----------------------------------------------------------------------------\n\n# Make sure prerequisite environment variables are set\nif [ -z \"$JAVA_HOME\" -a -z \"$JRE_HOME\" ]; then\n  if $darwin; then\n    # Bugzilla 54390\n    if [ -x '/usr/libexec/java_home' ] ; then\n      export JAVA_HOME=`/usr/libexec/java_home`\n    # Bugzilla 37284 (reviewed).\n    elif [ -d \"/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home\" ]; then\n      export JAVA_HOME=\"/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home\"\n    fi\n  else\n    JAVA_PATH=`which java 2>/dev/null`\n    if [ \"x$JAVA_PATH\" != \"x\" ]; then\n      JAVA_PATH=`dirname $JAVA_PATH 2>/dev/null`\n      JRE_HOME=`dirname $JAVA_PATH 2>/dev/null`\n    fi\n    if [ \"x$JRE_HOME\" = \"x\" ]; then\n      # XXX: Should we try other locations?\n      if [ -x /usr/bin/java ]; then\n        JRE_HOME=/usr\n      fi\n    fi\n  fi\n  if [ -z \"$JAVA_HOME\" -a -z \"$JRE_HOME\" ]; then\n    echo \"Neither the JAVA_HOME nor the JRE_HOME environment variable is defined\"\n    echo \"At least one of these environment variable is needed to run this program\"\n    exit 1\n  fi\nfi\nif [ -z \"$JAVA_HOME\" -a \"$1\" = \"debug\" ]; then\n  echo \"JAVA_HOME should point to a JDK in order to run in debug mode.\"\n  exit 1\nfi\nif [ -z \"$JRE_HOME\" ]; then\n  JRE_HOME=\"$JAVA_HOME\"\nfi\n\n# If we're running under jdb, we need a full jdk.\nif [ \"$1\" = \"debug\" ] ; then\n  if [ \"$os400\" = \"true\" ]; then\n    if [ ! -x \"$JAVA_HOME\"/bin/java -o ! -x \"$JAVA_HOME\"/bin/javac ]; then\n      echo \"The JAVA_HOME environment variable is not defined correctly\"\n      echo \"This environment variable is needed to run this program\"\n      echo \"NB: JAVA_HOME should point to a JDK not a JRE\"\n      exit 1\n    fi\n  else\n    if [ ! -x \"$JAVA_HOME\"/bin/java -o ! -x \"$JAVA_HOME\"/bin/jdb -o ! -x \"$JAVA_HOME\"/bin/javac ]; then\n      echo \"The JAVA_HOME environment variable is not defined correctly\"\n      echo \"This environment variable is needed to run this program\"\n      echo \"NB: JAVA_HOME should point to a JDK not a JRE\"\n      exit 1\n    fi\n  fi\nfi\n\n# Don't override the endorsed dir if the user has set it previously\nif [ -z \"$JAVA_ENDORSED_DIRS\" ]; then\n  # Set the default -Djava.endorsed.dirs argument\n  JAVA_ENDORSED_DIRS=\"$CATALINA_HOME\"/endorsed\nfi\n\n# Set standard commands for invoking Java.\n_RUNJAVA=\"$JRE_HOME\"/bin/java\nif [ \"$os400\" != \"true\" ]; then\n  _RUNJDB=\"$JAVA_HOME\"/bin/jdb\nfi\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/shutdown.bat",
    "content": "@echo off\r\nrem Licensed to the Apache Software Foundation (ASF) under one or more\r\nrem contributor license agreements.  See the NOTICE file distributed with\r\nrem this work for additional information regarding copyright ownership.\r\nrem The ASF licenses this file to You under the Apache License, Version 2.0\r\nrem (the \"License\"); you may not use this file except in compliance with\r\nrem the License.  You may obtain a copy of the License at\r\nrem\r\nrem     http://www.apache.org/licenses/LICENSE-2.0\r\nrem\r\nrem Unless required by applicable law or agreed to in writing, software\r\nrem distributed under the License is distributed on an \"AS IS\" BASIS,\r\nrem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nrem See the License for the specific language governing permissions and\r\nrem limitations under the License.\r\n\r\nif \"%OS%\" == \"Windows_NT\" setlocal\r\nrem ---------------------------------------------------------------------------\r\nrem Stop script for the CATALINA Server\r\nrem\r\nrem $Id: shutdown.bat 895392 2010-01-03 14:02:31Z kkolinko $\r\nrem ---------------------------------------------------------------------------\r\n\r\nrem Guess CATALINA_HOME if not defined\r\nset \"CURRENT_DIR=%cd%\"\r\nif not \"%CATALINA_HOME%\" == \"\" goto gotHome\r\nset \"CATALINA_HOME=%CURRENT_DIR%\"\r\nif exist \"%CATALINA_HOME%\\bin\\catalina.bat\" goto okHome\r\ncd ..\r\nset \"CATALINA_HOME=%cd%\"\r\ncd \"%CURRENT_DIR%\"\r\n:gotHome\r\nif exist \"%CATALINA_HOME%\\bin\\catalina.bat\" goto okHome\r\necho The CATALINA_HOME environment variable is not defined correctly\r\necho This environment variable is needed to run this program\r\ngoto end\r\n:okHome\r\n\r\nset \"EXECUTABLE=%CATALINA_HOME%\\bin\\catalina.bat\"\r\n\r\nrem Check that target executable exists\r\nif exist \"%EXECUTABLE%\" goto okExec\r\necho Cannot find \"%EXECUTABLE%\"\r\necho This file is needed to run this program\r\ngoto end\r\n:okExec\r\n\r\nrem Get remaining unshifted command line arguments and save them in the\r\nset CMD_LINE_ARGS=\r\n:setArgs\r\nif \"\"%1\"\"==\"\"\"\" goto doneSetArgs\r\nset CMD_LINE_ARGS=%CMD_LINE_ARGS% %1\r\nshift\r\ngoto setArgs\r\n:doneSetArgs\r\n\r\ncall \"%EXECUTABLE%\" stop %CMD_LINE_ARGS%\r\n\r\n:end\r\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/shutdown.sh",
    "content": "#!/bin/sh\n\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements.  See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License.  You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# -----------------------------------------------------------------------------\n# Stop script for the CATALINA Server\n#\n# $Id: shutdown.sh 1202062 2011-11-15 06:50:02Z mturk $\n# -----------------------------------------------------------------------------\n\n# Better OS/400 detection: see Bugzilla 31132\nos400=false\ncase \"`uname`\" in\nOS400*) os400=true;;\nesac\n\n# resolve links - $0 may be a softlink\nPRG=\"$0\"\n\nwhile [ -h \"$PRG\" ] ; do\n  ls=`ls -ld \"$PRG\"`\n  link=`expr \"$ls\" : '.*-> \\(.*\\)$'`\n  if expr \"$link\" : '/.*' > /dev/null; then\n    PRG=\"$link\"\n  else\n    PRG=`dirname \"$PRG\"`/\"$link\"\n  fi\ndone\n\nPRGDIR=`dirname \"$PRG\"`\nEXECUTABLE=catalina.sh\n\n# Check that target executable exists\nif $os400; then\n  # -x will Only work on the os400 if the files are:\n  # 1. owned by the user\n  # 2. owned by the PRIMARY group of the user\n  # this will not work if the user belongs in secondary groups\n  eval\nelse\n  if [ ! -x \"$PRGDIR\"/\"$EXECUTABLE\" ]; then\n    echo \"Cannot find $PRGDIR/$EXECUTABLE\"\n    echo \"The file is absent or does not have execute permission\"\n    echo \"This file is needed to run this program\"\n    exit 1\n  fi\nfi\n\nexec \"$PRGDIR\"/\"$EXECUTABLE\" stop \"$@\"\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/startup.bat",
    "content": "@echo off\r\nrem Licensed to the Apache Software Foundation (ASF) under one or more\r\nrem contributor license agreements.  See the NOTICE file distributed with\r\nrem this work for additional information regarding copyright ownership.\r\nrem The ASF licenses this file to You under the Apache License, Version 2.0\r\nrem (the \"License\"); you may not use this file except in compliance with\r\nrem the License.  You may obtain a copy of the License at\r\nrem\r\nrem     http://www.apache.org/licenses/LICENSE-2.0\r\nrem\r\nrem Unless required by applicable law or agreed to in writing, software\r\nrem distributed under the License is distributed on an \"AS IS\" BASIS,\r\nrem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nrem See the License for the specific language governing permissions and\r\nrem limitations under the License.\r\n\r\nif \"%OS%\" == \"Windows_NT\" setlocal\r\nrem ---------------------------------------------------------------------------\r\nrem Start script for the CATALINA Server\r\nrem\r\nrem $Id: startup.bat 895392 2010-01-03 14:02:31Z kkolinko $\r\nrem ---------------------------------------------------------------------------\r\n\r\nrem Guess CATALINA_HOME if not defined\r\nset \"CURRENT_DIR=%cd%\"\r\nif not \"%CATALINA_HOME%\" == \"\" goto gotHome\r\nset \"CATALINA_HOME=%CURRENT_DIR%\"\r\nif exist \"%CATALINA_HOME%\\bin\\catalina.bat\" goto okHome\r\ncd ..\r\nset \"CATALINA_HOME=%cd%\"\r\ncd \"%CURRENT_DIR%\"\r\n:gotHome\r\nif exist \"%CATALINA_HOME%\\bin\\catalina.bat\" goto okHome\r\necho The CATALINA_HOME environment variable is not defined correctly\r\necho This environment variable is needed to run this program\r\ngoto end\r\n:okHome\r\n\r\nset \"EXECUTABLE=%CATALINA_HOME%\\bin\\catalina.bat\"\r\n\r\nrem Check that target executable exists\r\nif exist \"%EXECUTABLE%\" goto okExec\r\necho Cannot find \"%EXECUTABLE%\"\r\necho This file is needed to run this program\r\ngoto end\r\n:okExec\r\n\r\nrem Get remaining unshifted command line arguments and save them in the\r\nset CMD_LINE_ARGS=\r\n:setArgs\r\nif \"\"%1\"\"==\"\"\"\" goto doneSetArgs\r\nset CMD_LINE_ARGS=%CMD_LINE_ARGS% %1\r\nshift\r\ngoto setArgs\r\n:doneSetArgs\r\n\r\ncall \"%EXECUTABLE%\" start %CMD_LINE_ARGS%\r\n\r\n:end\r\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/startup.sh",
    "content": "#!/bin/sh\n\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements.  See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License.  You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# -----------------------------------------------------------------------------\n# Start Script for the CATALINA Server\n#\n# $Id: startup.sh 1202062 2011-11-15 06:50:02Z mturk $\n# -----------------------------------------------------------------------------\n\n# Better OS/400 detection: see Bugzilla 31132\nos400=false\ncase \"`uname`\" in\nOS400*) os400=true;;\nesac\n\n# resolve links - $0 may be a softlink\nPRG=\"$0\"\n\nwhile [ -h \"$PRG\" ] ; do\n  ls=`ls -ld \"$PRG\"`\n  link=`expr \"$ls\" : '.*-> \\(.*\\)$'`\n  if expr \"$link\" : '/.*' > /dev/null; then\n    PRG=\"$link\"\n  else\n    PRG=`dirname \"$PRG\"`/\"$link\"\n  fi\ndone\n\nPRGDIR=`dirname \"$PRG\"`\nEXECUTABLE=catalina.sh\n\n# Check that target executable exists\nif $os400; then\n  # -x will Only work on the os400 if the files are:\n  # 1. owned by the user\n  # 2. owned by the PRIMARY group of the user\n  # this will not work if the user belongs in secondary groups\n  eval\nelse\n  if [ ! -x \"$PRGDIR\"/\"$EXECUTABLE\" ]; then\n    echo \"Cannot find $PRGDIR/$EXECUTABLE\"\n    echo \"The file is absent or does not have execute permission\"\n    echo \"This file is needed to run this program\"\n    exit 1\n  fi\nfi\n\nexec \"$PRGDIR\"/\"$EXECUTABLE\" start \"$@\"\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/tool-wrapper.bat",
    "content": "@echo off\r\nrem Licensed to the Apache Software Foundation (ASF) under one or more\r\nrem contributor license agreements.  See the NOTICE file distributed with\r\nrem this work for additional information regarding copyright ownership.\r\nrem The ASF licenses this file to You under the Apache License, Version 2.0\r\nrem (the \"License\"); you may not use this file except in compliance with\r\nrem the License.  You may obtain a copy of the License at\r\nrem\r\nrem     http://www.apache.org/licenses/LICENSE-2.0\r\nrem\r\nrem Unless required by applicable law or agreed to in writing, software\r\nrem distributed under the License is distributed on an \"AS IS\" BASIS,\r\nrem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nrem See the License for the specific language governing permissions and\r\nrem limitations under the License.\r\n\r\nif \"%OS%\" == \"Windows_NT\" setlocal\r\nrem ---------------------------------------------------------------------------\r\nrem Wrapper script for command line tools\r\nrem\r\nrem Environment Variable Prerequisites\r\nrem\r\nrem   CATALINA_HOME   May point at your Catalina \"build\" directory.\r\nrem\r\nrem   TOOL_OPTS       (Optional) Java runtime options.\r\nrem\r\nrem   JAVA_HOME       Must point at your Java Development Kit installation.\r\nrem                   Using JRE_HOME instead works as well.\r\nrem\r\nrem   JRE_HOME        Must point at your Java Runtime installation.\r\nrem                   Defaults to JAVA_HOME if empty. If JRE_HOME and JAVA_HOME\r\nrem                   are both set, JRE_HOME is used.\r\nrem\r\nrem   JAVA_OPTS       (Optional) Java runtime options.\r\nrem\r\nrem   JAVA_ENDORSED_DIRS (Optional) Lists of of semi-colon separated directories\r\nrem                   containing some jars in order to allow replacement of APIs\r\nrem                   created outside of the JCP (i.e. DOM and SAX from W3C).\r\nrem                   It can also be used to update the XML parser implementation.\r\nrem                   Defaults to $CATALINA_HOME/endorsed.\r\nrem\r\nrem $Id: tool-wrapper.bat 1138835 2011-06-23 11:27:57Z rjung $\r\nrem ---------------------------------------------------------------------------\r\n\r\nrem Guess CATALINA_HOME if not defined\r\nset \"CURRENT_DIR=%cd%\"\r\nif not \"%CATALINA_HOME%\" == \"\" goto gotHome\r\nset \"CATALINA_HOME=%CURRENT_DIR%\"\r\nif exist \"%CATALINA_HOME%\\bin\\tool-wrapper.bat\" goto okHome\r\ncd ..\r\nset \"CATALINA_HOME=%cd%\"\r\ncd \"%CURRENT_DIR%\"\r\n:gotHome\r\nif exist \"%CATALINA_HOME%\\bin\\tool-wrapper.bat\" goto okHome\r\necho The CATALINA_HOME environment variable is not defined correctly\r\necho This environment variable is needed to run this program\r\ngoto end\r\n:okHome\r\n\r\nrem Ensure that any user defined CLASSPATH variables are not used on startup,\r\nrem but allow them to be specified in setenv.bat, in rare case when it is needed.\r\nset CLASSPATH=\r\n\r\nrem Get standard environment variables\r\nif exist \"%CATALINA_HOME%\\bin\\setenv.bat\" call \"%CATALINA_HOME%\\bin\\setenv.bat\"\r\n\r\nrem Get standard Java environment variables\r\nif exist \"%CATALINA_HOME%\\bin\\setclasspath.bat\" goto okSetclasspath\r\necho Cannot find \"%CATALINA_HOME%\\bin\\setclasspath.bat\"\r\necho This file is needed to run this program\r\ngoto end\r\n:okSetclasspath\r\ncall \"%CATALINA_HOME%\\bin\\setclasspath.bat\" %1\r\nif errorlevel 1 goto end\r\n\r\nrem Add on extra jar files to CLASSPATH\r\nrem Note that there are no quotes as we do not want to introduce random\r\nrem quotes into the CLASSPATH\r\nif \"%CLASSPATH%\" == \"\" goto emptyClasspath\r\nset \"CLASSPATH=%CLASSPATH%;\"\r\n:emptyClasspath\r\nset \"CLASSPATH=%CLASSPATH%%CATALINA_HOME%\\bin\\bootstrap.jar;%CATALINA_HOME%\\bin\\tomcat-juli.jar;%CATALINA_HOME%\\lib\\servlet-api.jar\"\r\n\r\nset JAVA_OPTS=%JAVA_OPTS% -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager\r\n\r\nrem Get remaining unshifted command line arguments and save them in the\r\nset CMD_LINE_ARGS=\r\n:setArgs\r\nif \"\"%1\"\"==\"\"\"\" goto doneSetArgs\r\nset CMD_LINE_ARGS=%CMD_LINE_ARGS% %1\r\nshift\r\ngoto setArgs\r\n:doneSetArgs\r\n\r\n%_RUNJAVA% %JAVA_OPTS% %TOOL_OPTS% -Djava.endorsed.dirs=\"%JAVA_ENDORSED_DIRS%\" -classpath \"%CLASSPATH%\" -Dcatalina.home=\"%CATALINA_HOME%\" org.apache.catalina.startup.Tool %CMD_LINE_ARGS%\r\n\r\n:end\r\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/tool-wrapper.sh",
    "content": "#!/bin/sh\n\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements.  See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License.  You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# -----------------------------------------------------------------------------\n# Wrapper script for command line tools\n#\n# Environment Variable Prerequisites\n#\n#   CATALINA_HOME   May point at your Catalina \"build\" directory.\n#\n#   TOOL_OPTS       (Optional) Java runtime options.\n#\n#   JAVA_HOME       Must point at your Java Development Kit installation.\n#                   Using JRE_HOME instead works as well.\n#\n#   JRE_HOME        Must point at your Java Runtime installation.\n#                   Defaults to JAVA_HOME if empty. If JRE_HOME and JAVA_HOME\n#                   are both set, JRE_HOME is used.\n#\n#   JAVA_OPTS       (Optional) Java runtime options.\n#\n#   JAVA_ENDORSED_DIRS (Optional) Lists of of colon separated directories\n#                   containing some jars in order to allow replacement of APIs\n#                   created outside of the JCP (i.e. DOM and SAX from W3C).\n#                   It can also be used to update the XML parser implementation.\n#                   Defaults to $CATALINA_HOME/endorsed.\n#\n# $Id: tool-wrapper.sh 1138835 2011-06-23 11:27:57Z rjung $\n# -----------------------------------------------------------------------------\n\n# OS specific support.  $var _must_ be set to either true or false.\ncygwin=false\ndarwin=false\nos400=false\ncase \"`uname`\" in\nCYGWIN*) cygwin=true;;\nDarwin*) darwin=true;;\nOS400*) os400=true;;\nesac\n\n# resolve links - $0 may be a softlink\nPRG=\"$0\"\n\nwhile [ -h \"$PRG\" ]; do\n  ls=`ls -ld \"$PRG\"`\n  link=`expr \"$ls\" : '.*-> \\(.*\\)$'`\n  if expr \"$link\" : '/.*' > /dev/null; then\n    PRG=\"$link\"\n  else\n    PRG=`dirname \"$PRG\"`/\"$link\"\n  fi\ndone\n\n# Get standard environment variables\nPRGDIR=`dirname \"$PRG\"`\n\n# Only set CATALINA_HOME if not already set\n[ -z \"$CATALINA_HOME\" ] && CATALINA_HOME=`cd \"$PRGDIR/..\" >/dev/null; pwd`\n\n# Ensure that any user defined CLASSPATH variables are not used on startup,\n# but allow them to be specified in setenv.sh, in rare case when it is needed.\nCLASSPATH=\n\nif [ -r \"$CATALINA_HOME/bin/setenv.sh\" ]; then\n  . \"$CATALINA_HOME/bin/setenv.sh\"\nfi\n\n# For Cygwin, ensure paths are in UNIX format before anything is touched\nif $cygwin; then\n  [ -n \"$JAVA_HOME\" ] && JAVA_HOME=`cygpath --unix \"$JAVA_HOME\"`\n  [ -n \"$JRE_HOME\" ] && JRE_HOME=`cygpath --unix \"$JRE_HOME\"`\n  [ -n \"$CATALINA_HOME\" ] && CATALINA_HOME=`cygpath --unix \"$CATALINA_HOME\"`\n  [ -n \"$CLASSPATH\" ] && CLASSPATH=`cygpath --path --unix \"$CLASSPATH\"`\nfi\n\n# For OS400\nif $os400; then\n  # Set job priority to standard for interactive (interactive - 6) by using\n  # the interactive priority - 6, the helper threads that respond to requests\n  # will be running at the same priority as interactive jobs.\n  COMMAND='chgjob job('$JOBNAME') runpty(6)'\n  system $COMMAND\n\n  # Enable multi threading\n  export QIBM_MULTI_THREADED=Y\nfi\n\n# Get standard Java environment variables\nif $os400; then\n  # -r will Only work on the os400 if the files are:\n  # 1. owned by the user\n  # 2. owned by the PRIMARY group of the user\n  # this will not work if the user belongs in secondary groups\n  . \"$CATALINA_HOME\"/bin/setclasspath.sh\nelse\n  if [ -r \"$CATALINA_HOME\"/bin/setclasspath.sh ]; then\n    . \"$CATALINA_HOME\"/bin/setclasspath.sh\n  else\n    echo \"Cannot find $CATALINA_HOME/bin/setclasspath.sh\"\n    echo \"This file is needed to run this program\"\n    exit 1\n  fi\nfi\n\n# Add on extra jar files to CLASSPATH\nif [ ! -z \"$CLASSPATH\" ] ; then\n  CLASSPATH=\"$CLASSPATH\":\nfi\nCLASSPATH=\"$CLASSPATH\"\"$CATALINA_HOME\"/bin/bootstrap.jar:\"$CATALINA_HOME\"/bin/tomcat-juli.jar:\"$CATALINA_HOME\"/lib/servlet-api.jar\n\n# For Cygwin, switch paths to Windows format before running java\nif $cygwin; then\n  JAVA_HOME=`cygpath --absolute --windows \"$JAVA_HOME\"`\n  JRE_HOME=`cygpath --absolute --windows \"$JRE_HOME\"`\n  CATALINA_HOME=`cygpath --absolute --windows \"$CATALINA_HOME\"`\n  CLASSPATH=`cygpath --path --windows \"$CLASSPATH\"`\n  JAVA_ENDORSED_DIRS=`cygpath --path --windows \"$JAVA_ENDORSED_DIRS\"`\nfi\n\nJAVA_OPTS=\"$JAVA_OPTS -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager\"\n\n# ----- Execute The Requested Command -----------------------------------------\n\nexec \"$_RUNJAVA\" $JAVA_OPTS $TOOL_OPTS \\\n  -Djava.endorsed.dirs=\"$JAVA_ENDORSED_DIRS\" -classpath \"$CLASSPATH\" \\\n  -Dcatalina.home=\"$CATALINA_HOME\" \\\n  org.apache.catalina.startup.Tool \"$@\"\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/version.bat",
    "content": "@echo off\r\nrem Licensed to the Apache Software Foundation (ASF) under one or more\r\nrem contributor license agreements.  See the NOTICE file distributed with\r\nrem this work for additional information regarding copyright ownership.\r\nrem The ASF licenses this file to You under the Apache License, Version 2.0\r\nrem (the \"License\"); you may not use this file except in compliance with\r\nrem the License.  You may obtain a copy of the License at\r\nrem\r\nrem     http://www.apache.org/licenses/LICENSE-2.0\r\nrem\r\nrem Unless required by applicable law or agreed to in writing, software\r\nrem distributed under the License is distributed on an \"AS IS\" BASIS,\r\nrem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\nrem See the License for the specific language governing permissions and\r\nrem limitations under the License.\r\n\r\nif \"%OS%\" == \"Windows_NT\" setlocal\r\nrem ---------------------------------------------------------------------------\r\nrem Version script for the CATALINA Server\r\nrem\r\nrem $Id: version.bat 895392 2010-01-03 14:02:31Z kkolinko $\r\nrem ---------------------------------------------------------------------------\r\n\r\nrem Guess CATALINA_HOME if not defined\r\nset \"CURRENT_DIR=%cd%\"\r\nif not \"%CATALINA_HOME%\" == \"\" goto gotHome\r\nset \"CATALINA_HOME=%CURRENT_DIR%\"\r\nif exist \"%CATALINA_HOME%\\bin\\catalina.bat\" goto okHome\r\ncd ..\r\nset \"CATALINA_HOME=%cd%\"\r\ncd \"%CURRENT_DIR%\"\r\n:gotHome\r\nif exist \"%CATALINA_HOME%\\bin\\catalina.bat\" goto okHome\r\necho The CATALINA_HOME environment variable is not defined correctly\r\necho This environment variable is needed to run this program\r\ngoto end\r\n:okHome\r\n\r\nset \"EXECUTABLE=%CATALINA_HOME%\\bin\\catalina.bat\"\r\n\r\nrem Check that target executable exists\r\nif exist \"%EXECUTABLE%\" goto okExec\r\necho Cannot find \"%EXECUTABLE%\"\r\necho This file is needed to run this program\r\ngoto end\r\n:okExec\r\n\r\nrem Get remaining unshifted command line arguments and save them in the\r\nset CMD_LINE_ARGS=\r\n:setArgs\r\nif \"\"%1\"\"==\"\"\"\" goto doneSetArgs\r\nset CMD_LINE_ARGS=%CMD_LINE_ARGS% %1\r\nshift\r\ngoto setArgs\r\n:doneSetArgs\r\n\r\ncall \"%EXECUTABLE%\" version %CMD_LINE_ARGS%\r\n\r\n:end\r\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/bin/version.sh",
    "content": "#!/bin/sh\n\n# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements.  See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License.  You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# -----------------------------------------------------------------------------\n# Version Script for the CATALINA Server\n#\n# $Id: version.sh 1202062 2011-11-15 06:50:02Z mturk $\n# -----------------------------------------------------------------------------\n\n# Better OS/400 detection: see Bugzilla 31132\nos400=false\ncase \"`uname`\" in\nOS400*) os400=true;;\nesac\n\n# resolve links - $0 may be a softlink\nPRG=\"$0\"\n\nwhile [ -h \"$PRG\" ] ; do\n  ls=`ls -ld \"$PRG\"`\n  link=`expr \"$ls\" : '.*-> \\(.*\\)$'`\n  if expr \"$link\" : '/.*' > /dev/null; then\n    PRG=\"$link\"\n  else\n    PRG=`dirname \"$PRG\"`/\"$link\"\n  fi\ndone\n\nPRGDIR=`dirname \"$PRG\"`\nEXECUTABLE=catalina.sh\n\n# Check that target executable exists\nif $os400; then\n  # -x will Only work on the os400 if the files are:\n  # 1. owned by the user\n  # 2. owned by the PRIMARY group of the user\n  # this will not work if the user belongs in secondary groups\n  eval\nelse\n  if [ ! -x \"$PRGDIR\"/\"$EXECUTABLE\" ]; then\n    echo \"Cannot find $PRGDIR/$EXECUTABLE\"\n    echo \"The file is absent or does not have execute permission\"\n    echo \"This file is needed to run this program\"\n    exit 1\n  fi\nfi\n\nexec \"$PRGDIR\"/\"$EXECUTABLE\" version \"$@\"\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/conf/catalina.policy",
    "content": "// Licensed to the Apache Software Foundation (ASF) under one or more\n// contributor license agreements.  See the NOTICE file distributed with\n// this work for additional information regarding copyright ownership.\n// The ASF licenses this file to You under the Apache License, Version 2.0\n// (the \"License\"); you may not use this file except in compliance with\n// the License.  You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n// ============================================================================\n// catalina.policy - Security Policy Permissions for Tomcat 7\n//\n// This file contains a default set of security policies to be enforced (by the\n// JVM) when Catalina is executed with the \"-security\" option.  In addition\n// to the permissions granted here, the following additional permissions are\n// granted to each web application:\n//\n// * Read access to the web application's document root directory\n// * Read, write and delete access to the web application's working directory\n//\n// $Id: catalina.policy 1460221 2013-03-23 20:17:29Z kkolinko $\n// ============================================================================\n\n\n// ========== SYSTEM CODE PERMISSIONS =========================================\n\n\n// These permissions apply to javac\ngrant codeBase \"file:${java.home}/lib/-\" {\n        permission java.security.AllPermission;\n};\n\n// These permissions apply to all shared system extensions\ngrant codeBase \"file:${java.home}/jre/lib/ext/-\" {\n        permission java.security.AllPermission;\n};\n\n// These permissions apply to javac when ${java.home] points at $JAVA_HOME/jre\ngrant codeBase \"file:${java.home}/../lib/-\" {\n        permission java.security.AllPermission;\n};\n\n// These permissions apply to all shared system extensions when\n// ${java.home} points at $JAVA_HOME/jre\ngrant codeBase \"file:${java.home}/lib/ext/-\" {\n        permission java.security.AllPermission;\n};\n\n\n// ========== CATALINA CODE PERMISSIONS =======================================\n\n\n// These permissions apply to the daemon code\ngrant codeBase \"file:${catalina.home}/bin/commons-daemon.jar\" {\n        permission java.security.AllPermission;\n};\n\n// These permissions apply to the logging API\n// Note: If tomcat-juli.jar is in ${catalina.base} and not in ${catalina.home},\n// update this section accordingly.\n//  grant codeBase \"file:${catalina.base}/bin/tomcat-juli.jar\" {..}\ngrant codeBase \"file:${catalina.home}/bin/tomcat-juli.jar\" {\n        permission java.io.FilePermission\n         \"${java.home}${file.separator}lib${file.separator}logging.properties\", \"read\";\n\n        permission java.io.FilePermission\n         \"${catalina.base}${file.separator}conf${file.separator}logging.properties\", \"read\";\n        permission java.io.FilePermission\n         \"${catalina.base}${file.separator}logs\", \"read, write\";\n        permission java.io.FilePermission\n         \"${catalina.base}${file.separator}logs${file.separator}*\", \"read, write\";\n\n        permission java.lang.RuntimePermission \"shutdownHooks\";\n        permission java.lang.RuntimePermission \"getClassLoader\";\n        permission java.lang.RuntimePermission \"setContextClassLoader\";\n\n        permission java.util.logging.LoggingPermission \"control\";\n\n        permission java.util.PropertyPermission \"java.util.logging.config.class\", \"read\";\n        permission java.util.PropertyPermission \"java.util.logging.config.file\", \"read\";\n        permission java.util.PropertyPermission \"catalina.base\", \"read\";\n\n        // Note: To enable per context logging configuration, permit read access to\n        // the appropriate file. Be sure that the logging configuration is\n        // secure before enabling such access.\n        // E.g. for the examples web application (uncomment and unwrap\n        // the following to be on a single line):\n        // permission java.io.FilePermission \"${catalina.base}${file.separator}\n        //  webapps${file.separator}examples${file.separator}WEB-INF\n        //  ${file.separator}classes${file.separator}logging.properties\", \"read\";\n};\n\n// These permissions apply to the server startup code\ngrant codeBase \"file:${catalina.home}/bin/bootstrap.jar\" {\n        permission java.security.AllPermission;\n};\n\n// These permissions apply to the servlet API classes\n// and those that are shared across all class loaders\n// located in the \"lib\" directory\ngrant codeBase \"file:${catalina.home}/lib/-\" {\n        permission java.security.AllPermission;\n};\n\n\n// If using a per instance lib directory, i.e. ${catalina.base}/lib,\n// then the following permission will need to be uncommented\n// grant codeBase \"file:${catalina.base}/lib/-\" {\n//         permission java.security.AllPermission;\n// };\n\n\n// ========== WEB APPLICATION PERMISSIONS =====================================\n\n\n// These permissions are granted by default to all web applications\n// In addition, a web application will be given a read FilePermission\n// and JndiPermission for all files and directories in its document root.\ngrant {\n    // Required for JNDI lookup of named JDBC DataSource's and\n    // javamail named MimePart DataSource used to send mail\n    permission java.util.PropertyPermission \"java.home\", \"read\";\n    permission java.util.PropertyPermission \"java.naming.*\", \"read\";\n    permission java.util.PropertyPermission \"javax.sql.*\", \"read\";\n\n    // OS Specific properties to allow read access\n    permission java.util.PropertyPermission \"os.name\", \"read\";\n    permission java.util.PropertyPermission \"os.version\", \"read\";\n    permission java.util.PropertyPermission \"os.arch\", \"read\";\n    permission java.util.PropertyPermission \"file.separator\", \"read\";\n    permission java.util.PropertyPermission \"path.separator\", \"read\";\n    permission java.util.PropertyPermission \"line.separator\", \"read\";\n\n    // JVM properties to allow read access\n    permission java.util.PropertyPermission \"java.version\", \"read\";\n    permission java.util.PropertyPermission \"java.vendor\", \"read\";\n    permission java.util.PropertyPermission \"java.vendor.url\", \"read\";\n    permission java.util.PropertyPermission \"java.class.version\", \"read\";\n    permission java.util.PropertyPermission \"java.specification.version\", \"read\";\n    permission java.util.PropertyPermission \"java.specification.vendor\", \"read\";\n    permission java.util.PropertyPermission \"java.specification.name\", \"read\";\n\n    permission java.util.PropertyPermission \"java.vm.specification.version\", \"read\";\n    permission java.util.PropertyPermission \"java.vm.specification.vendor\", \"read\";\n    permission java.util.PropertyPermission \"java.vm.specification.name\", \"read\";\n    permission java.util.PropertyPermission \"java.vm.version\", \"read\";\n    permission java.util.PropertyPermission \"java.vm.vendor\", \"read\";\n    permission java.util.PropertyPermission \"java.vm.name\", \"read\";\n\n    // Required for OpenJMX\n    permission java.lang.RuntimePermission \"getAttribute\";\n\n    // Allow read of JAXP compliant XML parser debug\n    permission java.util.PropertyPermission \"jaxp.debug\", \"read\";\n\n    // All JSPs need to be able to read this package\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.tomcat\";\n\n    // Precompiled JSPs need access to these packages.\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.jasper.el\";\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.jasper.runtime\";\n    permission java.lang.RuntimePermission\n     \"accessClassInPackage.org.apache.jasper.runtime.*\";\n\n    // Precompiled JSPs need access to these system properties.\n    permission java.util.PropertyPermission\n     \"org.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER\", \"read\";\n    permission java.util.PropertyPermission\n     \"org.apache.el.parser.COERCE_TO_ZERO\", \"read\";\n\n    // The cookie code needs these.\n    permission java.util.PropertyPermission\n     \"org.apache.catalina.STRICT_SERVLET_COMPLIANCE\", \"read\";\n    permission java.util.PropertyPermission\n     \"org.apache.tomcat.util.http.ServerCookie.STRICT_NAMING\", \"read\";\n    permission java.util.PropertyPermission\n     \"org.apache.tomcat.util.http.ServerCookie.FWD_SLASH_IS_SEPARATOR\", \"read\";\n\n    // Applications using Comet need to be able to access this package\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.catalina.comet\";\n\n    // Applications using WebSocket need to be able to access this package\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.catalina.websocket\";\n};\n\n\n// The Manager application needs access to the following packages to support the\n// session display functionality. These settings support the following\n// configurations:\n// - default CATALINA_HOME == CATALINA_BASE\n// - CATALINA_HOME != CATALINA_BASE, per instance Manager in CATALINA_BASE\n// - CATALINA_HOME != CATALINA_BASE, shared Manager in CATALINA_HOME\ngrant codeBase \"file:${catalina.base}/webapps/manager/-\" {\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.catalina\";\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.catalina.ha.session\";\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.catalina.manager\";\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.catalina.manager.util\";\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.catalina.util\";\n};\ngrant codeBase \"file:${catalina.home}/webapps/manager/-\" {\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.catalina\";\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.catalina.ha.session\";\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.catalina.manager\";\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.catalina.manager.util\";\n    permission java.lang.RuntimePermission \"accessClassInPackage.org.apache.catalina.util\";\n};\n\n// You can assign additional permissions to particular web applications by\n// adding additional \"grant\" entries here, based on the code base for that\n// application, /WEB-INF/classes/, or /WEB-INF/lib/ jar files.\n//\n// Different permissions can be granted to JSP pages, classes loaded from\n// the /WEB-INF/classes/ directory, all jar files in the /WEB-INF/lib/\n// directory, or even to individual jar files in the /WEB-INF/lib/ directory.\n//\n// For instance, assume that the standard \"examples\" application\n// included a JDBC driver that needed to establish a network connection to the\n// corresponding database and used the scrape taglib to get the weather from\n// the NOAA web server.  You might create a \"grant\" entries like this:\n//\n// The permissions granted to the context root directory apply to JSP pages.\n// grant codeBase \"file:${catalina.base}/webapps/examples/-\" {\n//      permission java.net.SocketPermission \"dbhost.mycompany.com:5432\", \"connect\";\n//      permission java.net.SocketPermission \"*.noaa.gov:80\", \"connect\";\n// };\n//\n// The permissions granted to the context WEB-INF/classes directory\n// grant codeBase \"file:${catalina.base}/webapps/examples/WEB-INF/classes/-\" {\n// };\n//\n// The permission granted to your JDBC driver\n// grant codeBase \"jar:file:${catalina.base}/webapps/examples/WEB-INF/lib/driver.jar!/-\" {\n//      permission java.net.SocketPermission \"dbhost.mycompany.com:5432\", \"connect\";\n// };\n// The permission granted to the scrape taglib\n// grant codeBase \"jar:file:${catalina.base}/webapps/examples/WEB-INF/lib/scrape.jar!/-\" {\n//      permission java.net.SocketPermission \"*.noaa.gov:80\", \"connect\";\n// };\n\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/conf/catalina.properties",
    "content": "# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements.  See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License.  You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n#\n# List of comma-separated packages that start with or equal this string\n# will cause a security exception to be thrown when\n# passed to checkPackageAccess unless the\n# corresponding RuntimePermission (\"accessClassInPackage.\"+package) has\n# been granted.\npackage.access=sun.,org.apache.catalina.,org.apache.coyote.,org.apache.tomcat.,org.apache.jasper.\n#\n# List of comma-separated packages that start with or equal this string\n# will cause a security exception to be thrown when\n# passed to checkPackageDefinition unless the\n# corresponding RuntimePermission (\"defineClassInPackage.\"+package) has\n# been granted.\n#\n# by default, no packages are restricted for definition, and none of\n# the class loaders supplied with the JDK call checkPackageDefinition.\n#\npackage.definition=sun.,java.,org.apache.catalina.,org.apache.coyote.,org.apache.tomcat.,org.apache.jasper.\n\n#\n#\n# List of comma-separated paths defining the contents of the \"common\"\n# classloader. Prefixes should be used to define what is the repository type.\n# Path may be relative to the CATALINA_HOME or CATALINA_BASE path or absolute.\n# If left as blank,the JVM system loader will be used as Catalina's \"common\"\n# loader.\n# Examples:\n#     \"foo\": Add this folder as a class repository\n#     \"foo/*.jar\": Add all the JARs of the specified folder as class\n#                  repositories\n#     \"foo/bar.jar\": Add bar.jar as a class repository\ncommon.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar\n\n#\n# List of comma-separated paths defining the contents of the \"server\"\n# classloader. Prefixes should be used to define what is the repository type.\n# Path may be relative to the CATALINA_HOME or CATALINA_BASE path or absolute.\n# If left as blank, the \"common\" loader will be used as Catalina's \"server\"\n# loader.\n# Examples:\n#     \"foo\": Add this folder as a class repository\n#     \"foo/*.jar\": Add all the JARs of the specified folder as class\n#                  repositories\n#     \"foo/bar.jar\": Add bar.jar as a class repository\nserver.loader=\n\n#\n# List of comma-separated paths defining the contents of the \"shared\"\n# classloader. Prefixes should be used to define what is the repository type.\n# Path may be relative to the CATALINA_BASE path or absolute. If left as blank,\n# the \"common\" loader will be used as Catalina's \"shared\" loader.\n# Examples:\n#     \"foo\": Add this folder as a class repository\n#     \"foo/*.jar\": Add all the JARs of the specified folder as class\n#                  repositories\n#     \"foo/bar.jar\": Add bar.jar as a class repository\n# Please note that for single jars, e.g. bar.jar, you need the URL form\n# starting with file:.\nshared.loader=\n\n# List of JAR files that should not be scanned using the JarScanner\n# functionality. This is typically used to scan JARs for configuration\n# information. JARs that do not contain such information may be excluded from\n# the scan to speed up the scanning process. This is the default list. JARs on\n# this list are excluded from all scans. Scan specific lists (to exclude JARs\n# from individual scans) follow this. The list must be a comma separated list of\n# JAR file names.\n# The JARs listed below include:\n# - Tomcat Bootstrap JARs\n# - Tomcat API JARs\n# - Catalina JARs\n# - Jasper JARs\n# - Tomcat JARs\n# - Common non-Tomcat JARs\ntomcat.util.scan.DefaultJarScanner.jarsToSkip=\\\nbootstrap.jar,commons-daemon.jar,tomcat-juli.jar,\\\nannotations-api.jar,el-api.jar,jsp-api.jar,servlet-api.jar,\\\ncatalina.jar,catalina-ant.jar,catalina-ha.jar,catalina-tribes.jar,\\\njasper.jar,jasper-el.jar,ecj-*.jar,\\\ntomcat-api.jar,tomcat-util.jar,tomcat-coyote.jar,tomcat-dbcp.jar,\\\ntomcat-jni.jar,tomcat-spdy.jar,\\\ntomcat-i18n-en.jar,tomcat-i18n-es.jar,tomcat-i18n-fr.jar,tomcat-i18n-ja.jar,\\\ntomcat-juli-adapters.jar,catalina-jmx-remote.jar,catalina-ws.jar,\\\ntomcat-jdbc.jar,\\\ntools.jar,\\\ncommons-beanutils*.jar,commons-codec*.jar,commons-collections*.jar,\\\ncommons-dbcp*.jar,commons-digester*.jar,commons-fileupload*.jar,\\\ncommons-httpclient*.jar,commons-io*.jar,commons-lang*.jar,commons-logging*.jar,\\\ncommons-math*.jar,commons-pool*.jar,\\\njstl.jar,\\\ngeronimo-spec-jaxrpc*.jar,wsdl4j*.jar,\\\nant.jar,ant-junit*.jar,aspectj*.jar,jmx.jar,h2*.jar,hibernate*.jar,httpclient*.jar,\\\njmx-tools.jar,jta*.jar,log4j*.jar,mail*.jar,slf4j*.jar,\\\nxercesImpl.jar,xmlParserAPIs.jar,xml-apis.jar,\\\njunit.jar,junit-*.jar,ant-launcher.jar\n\n# Additional JARs (over and above the default JARs listed above) to skip when\n# scanning for Servlet 3.0 pluggability features. These features include web\n# fragments, annotations, SCIs and classes that match @HandlesTypes. The list\n# must be a comma separated list of JAR file names.\norg.apache.catalina.startup.ContextConfig.jarsToSkip=\n\n# Additional JARs (over and above the default JARs listed above) to skip when\n# scanning for TLDs. The list must be a comma separated list of JAR file names.\norg.apache.catalina.startup.TldConfig.jarsToSkip=\n\n#\n# String cache configuration.\ntomcat.util.buf.StringCache.byte.enabled=true\n#tomcat.util.buf.StringCache.char.enabled=true\n#tomcat.util.buf.StringCache.trainThreshold=500000\n#tomcat.util.buf.StringCache.cacheSize=5000\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/conf/context.xml",
    "content": "<?xml version='1.0' encoding='utf-8'?>\n<!--\n  Licensed to the Apache Software Foundation (ASF) under one or more\n  contributor license agreements.  See the NOTICE file distributed with\n  this work for additional information regarding copyright ownership.\n  The ASF licenses this file to You under the Apache License, Version 2.0\n  (the \"License\"); you may not use this file except in compliance with\n  the License.  You may obtain a copy of the License at\n\n      http://www.apache.org/licenses/LICENSE-2.0\n\n  Unless required by applicable law or agreed to in writing, software\n  distributed under the License is distributed on an \"AS IS\" BASIS,\n  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n  See the License for the specific language governing permissions and\n  limitations under the License.\n-->\n<!-- The contents of this file will be loaded for each web application -->\n<Context>\n\n    <!-- Default set of monitored resources -->\n    <WatchedResource>WEB-INF/web.xml</WatchedResource>\n\n    <!-- Uncomment this to disable session persistence across Tomcat restarts -->\n    <!--\n    <Manager pathname=\"\" />\n    -->\n\n    <!-- Uncomment this to enable Comet connection tacking (provides events\n         on session expiration as well as webapp lifecycle) -->\n    <!--\n    <Valve className=\"org.apache.catalina.valves.CometConnectionManagerValve\" />\n    -->\n\n</Context>"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/conf/logging.properties",
    "content": "# Licensed to the Apache Software Foundation (ASF) under one or more\n# contributor license agreements.  See the NOTICE file distributed with\n# this work for additional information regarding copyright ownership.\n# The ASF licenses this file to You under the Apache License, Version 2.0\n# (the \"License\"); you may not use this file except in compliance with\n# the License.  You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nhandlers = 1catalina.org.apache.juli.FileHandler, 2localhost.org.apache.juli.FileHandler, 3manager.org.apache.juli.FileHandler, 4host-manager.org.apache.juli.FileHandler, java.util.logging.ConsoleHandler\n\n.handlers = 1catalina.org.apache.juli.FileHandler, java.util.logging.ConsoleHandler\n\n############################################################\n# Handler specific properties.\n# Describes specific configuration info for Handlers.\n############################################################\n\n1catalina.org.apache.juli.FileHandler.level = FINE\n1catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\n1catalina.org.apache.juli.FileHandler.prefix = catalina.\n\n2localhost.org.apache.juli.FileHandler.level = FINE\n2localhost.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\n2localhost.org.apache.juli.FileHandler.prefix = localhost.\n\n3manager.org.apache.juli.FileHandler.level = FINE\n3manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\n3manager.org.apache.juli.FileHandler.prefix = manager.\n\n4host-manager.org.apache.juli.FileHandler.level = FINE\n4host-manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\n4host-manager.org.apache.juli.FileHandler.prefix = host-manager.\n\njava.util.logging.ConsoleHandler.level = FINE\njava.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter\n\n\n############################################################\n# Facility specific properties.\n# Provides extra control for each logger.\n############################################################\n\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost].level = INFO\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost].handlers = 2localhost.org.apache.juli.FileHandler\n\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].level = INFO\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].handlers = 3manager.org.apache.juli.FileHandler\n\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager].level = INFO\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager].handlers = 4host-manager.org.apache.juli.FileHandler\n\n# For example, set the org.apache.catalina.util.LifecycleBase logger to log\n# each component that extends LifecycleBase changing state:\n#org.apache.catalina.util.LifecycleBase.level = FINE\n\n# To see debug messages in TldLocationsCache, uncomment the following line:\n#org.apache.jasper.compiler.TldLocationsCache.level = FINE\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/conf/server.xml",
    "content": "<?xml version='1.0' encoding='utf-8'?>\n<!--\n  Licensed to the Apache Software Foundation (ASF) under one or more\n  contributor license agreements.  See the NOTICE file distributed with\n  this work for additional information regarding copyright ownership.\n  The ASF licenses this file to You under the Apache License, Version 2.0\n  (the \"License\"); you may not use this file except in compliance with\n  the License.  You may obtain a copy of the License at\n\n      http://www.apache.org/licenses/LICENSE-2.0\n\n  Unless required by applicable law or agreed to in writing, software\n  distributed under the License is distributed on an \"AS IS\" BASIS,\n  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n  See the License for the specific language governing permissions and\n  limitations under the License.\n-->\n<!-- Note:  A \"Server\" is not itself a \"Container\", so you may not\n     define subcomponents such as \"Valves\" at this level.\n     Documentation at /docs/config/server.html\n -->\n<Server port=\"8005\" shutdown=\"SHUTDOWN\">\n  <!-- Security listener. Documentation at /docs/config/listeners.html\n  <Listener className=\"org.apache.catalina.security.SecurityListener\" />\n  -->\n  <!--APR library loader. Documentation at /docs/apr.html -->\n  <Listener className=\"org.apache.catalina.core.AprLifecycleListener\" SSLEngine=\"on\" />\n  <!--Initialize Jasper prior to webapps are loaded. Documentation at /docs/jasper-howto.html -->\n  <Listener className=\"org.apache.catalina.core.JasperListener\" />\n  <!-- Prevent memory leaks due to use of particular java/javax APIs-->\n  <Listener className=\"org.apache.catalina.core.JreMemoryLeakPreventionListener\" />\n  <Listener className=\"org.apache.catalina.mbeans.GlobalResourcesLifecycleListener\" />\n  <Listener className=\"org.apache.catalina.core.ThreadLocalLeakPreventionListener\" />\n\n  <!-- Global JNDI resources\n       Documentation at /docs/jndi-resources-howto.html\n  -->\n  <GlobalNamingResources>\n    <!-- Editable user database that can also be used by\n         UserDatabaseRealm to authenticate users\n    -->\n    <Resource name=\"UserDatabase\" auth=\"Container\"\n              type=\"org.apache.catalina.UserDatabase\"\n              description=\"User database that can be updated and saved\"\n              factory=\"org.apache.catalina.users.MemoryUserDatabaseFactory\"\n              pathname=\"conf/tomcat-users.xml\" />\n  </GlobalNamingResources>\n\n  <!-- A \"Service\" is a collection of one or more \"Connectors\" that share\n       a single \"Container\" Note:  A \"Service\" is not itself a \"Container\",\n       so you may not define subcomponents such as \"Valves\" at this level.\n       Documentation at /docs/config/service.html\n   -->\n  <Service name=\"Catalina\">\n\n    <!--The connectors can use a shared executor, you can define one or more named thread pools-->\n    <!--\n    <Executor name=\"tomcatThreadPool\" namePrefix=\"catalina-exec-\"\n        maxThreads=\"150\" minSpareThreads=\"4\"/>\n    -->\n\n\n    <!-- A \"Connector\" represents an endpoint by which requests are received\n         and responses are returned. Documentation at :\n         Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)\n         Java AJP  Connector: /docs/config/ajp.html\n         APR (HTTP/AJP) Connector: /docs/apr.html\n         Define a non-SSL HTTP/1.1 Connector on port 8080\n    -->\n    <Connector port=\"8080\" protocol=\"HTTP/1.1\"\n               connectionTimeout=\"20000\"\n               maxHttpHeaderSize=\"262144\"\n               redirectPort=\"8443\" URIEncoding=\"UTF-8\" />\n    <!-- A \"Connector\" using the shared thread pool-->\n    <!--\n    <Connector executor=\"tomcatThreadPool\"\n               port=\"8080\" protocol=\"HTTP/1.1\"\n               connectionTimeout=\"20000\"\n               redirectPort=\"8443\" />\n    -->\n    <!-- Define a SSL HTTP/1.1 Connector on port 8443\n         This connector uses the JSSE configuration, when using APR, the\n         connector should be using the OpenSSL style configuration\n         described in the APR documentation -->\n    <!--\n    <Connector port=\"8443\" protocol=\"HTTP/1.1\" SSLEnabled=\"true\"\n               maxThreads=\"150\" scheme=\"https\" secure=\"true\"\n               clientAuth=\"false\" sslProtocol=\"TLS\" />\n    -->\n\n    <!-- Define an AJP 1.3 Connector on port 8009 -->\n    <Connector port=\"8009\" protocol=\"AJP/1.3\" redirectPort=\"8443\" />\n\n\n    <!-- An Engine represents the entry point (within Catalina) that processes\n         every request.  The Engine implementation for Tomcat stand alone\n         analyzes the HTTP headers included with the request, and passes them\n         on to the appropriate Host (virtual host).\n         Documentation at /docs/config/engine.html -->\n\n    <!-- You should set jvmRoute to support load-balancing via AJP ie :\n    <Engine name=\"Catalina\" defaultHost=\"localhost\" jvmRoute=\"jvm1\">\n    -->\n    <Engine name=\"Catalina\" defaultHost=\"localhost\">\n\n      <!--For clustering, please take a look at documentation at:\n          /docs/cluster-howto.html  (simple how to)\n          /docs/config/cluster.html (reference documentation) -->\n      <!--\n      <Cluster className=\"org.apache.catalina.ha.tcp.SimpleTcpCluster\"/>\n      -->\n\n      <!-- Use the LockOutRealm to prevent attempts to guess user passwords\n           via a brute-force attack -->\n      <Realm className=\"org.apache.catalina.realm.LockOutRealm\">\n        <!-- This Realm uses the UserDatabase configured in the global JNDI\n             resources under the key \"UserDatabase\".  Any edits\n             that are performed against this UserDatabase are immediately\n             available for use by the Realm.  -->\n        <Realm className=\"org.apache.catalina.realm.UserDatabaseRealm\"\n               resourceName=\"UserDatabase\"/>\n      </Realm>\n\n      <Host name=\"localhost\"  appBase=\"webapps\"\n            unpackWARs=\"true\" autoDeploy=\"true\">\n\n        <!-- SingleSignOn valve, share authentication between web applications\n             Documentation at: /docs/config/valve.html -->\n        <!--\n        <Valve className=\"org.apache.catalina.authenticator.SingleSignOn\" />\n        -->\n\n        <!-- Access log processes all example.\n             Documentation at: /docs/config/valve.html\n             Note: The pattern used is equivalent to using pattern=\"common\" -->\n        <Valve className=\"org.apache.catalina.valves.AccessLogValve\" directory=\"logs\"\n               prefix=\"localhost_access_log.\" suffix=\".txt\"\n               pattern=\"%h %l %u %t &quot;%r&quot; %s %b\" />\n\n      </Host>\n    </Engine>\n  </Service>\n</Server>\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/conf/tomcat-users.xml",
    "content": "<?xml version='1.0' encoding='utf-8'?>\n<!--\n  Licensed to the Apache Software Foundation (ASF) under one or more\n  contributor license agreements.  See the NOTICE file distributed with\n  this work for additional information regarding copyright ownership.\n  The ASF licenses this file to You under the Apache License, Version 2.0\n  (the \"License\"); you may not use this file except in compliance with\n  the License.  You may obtain a copy of the License at\n\n      http://www.apache.org/licenses/LICENSE-2.0\n\n  Unless required by applicable law or agreed to in writing, software\n  distributed under the License is distributed on an \"AS IS\" BASIS,\n  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n  See the License for the specific language governing permissions and\n  limitations under the License.\n-->\n<tomcat-users>\n<!--\n  NOTE:  By default, no user is included in the \"manager-gui\" role required\n  to operate the \"/manager/html\" web application.  If you wish to use this app,\n  you must define such a user - the username and password are arbitrary.\n-->\n<!--\n  NOTE:  The sample user and role entries below are wrapped in a comment\n  and thus are ignored when reading this file. Do not forget to remove\n  <!.. ..> that surrounds them.\n-->\n<!--\n  <role rolename=\"tomcat\"/>\n  <role rolename=\"role1\"/>\n  <user username=\"tomcat\" password=\"tomcat\" roles=\"tomcat\"/>\n  <user username=\"both\" password=\"tomcat\" roles=\"tomcat,role1\"/>\n  <user username=\"role1\" password=\"tomcat\" roles=\"role1\"/>\n-->\n</tomcat-users>\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/conf/web.xml",
    "content": "<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?>\n<!--\n  Licensed to the Apache Software Foundation (ASF) under one or more\n  contributor license agreements.  See the NOTICE file distributed with\n  this work for additional information regarding copyright ownership.\n  The ASF licenses this file to You under the Apache License, Version 2.0\n  (the \"License\"); you may not use this file except in compliance with\n  the License.  You may obtain a copy of the License at\n\n      http://www.apache.org/licenses/LICENSE-2.0\n\n  Unless required by applicable law or agreed to in writing, software\n  distributed under the License is distributed on an \"AS IS\" BASIS,\n  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n  See the License for the specific language governing permissions and\n  limitations under the License.\n-->\n<web-app xmlns=\"http://java.sun.com/xml/ns/javaee\"\n  xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n  xsi:schemaLocation=\"http://java.sun.com/xml/ns/javaee\n                      http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd\"\n  version=\"3.0\">\n\n  <!-- ======================== Introduction ============================== -->\n  <!-- This document defines default values for *all* web applications      -->\n  <!-- loaded into this instance of Tomcat.  As each application is         -->\n  <!-- deployed, this file is processed, followed by the                    -->\n  <!-- \"/WEB-INF/web.xml\" deployment descriptor from your own               -->\n  <!-- applications.                                                        -->\n  <!--                                                                      -->\n  <!-- WARNING:  Do not configure application-specific resources here!      -->\n  <!-- They should go in the \"/WEB-INF/web.xml\" file in your application.   -->\n\n\n  <!-- ================== Built In Servlet Definitions ==================== -->\n\n\n  <!-- The default servlet for all web applications, that serves static     -->\n  <!-- resources.  It processes all requests that are not mapped to other   -->\n  <!-- servlets with servlet mappings (defined either here or in your own   -->\n  <!-- web.xml file).  This servlet supports the following initialization   -->\n  <!-- parameters (default values are in square brackets):                  -->\n  <!--                                                                      -->\n  <!--   debug               Debugging detail level for messages logged     -->\n  <!--                       by this servlet.  [0]                          -->\n  <!--                                                                      -->\n  <!--   fileEncoding        Encoding to be used to read static resources   -->\n  <!--                       [platform default]                             -->\n  <!--                                                                      -->\n  <!--   input               Input buffer size (in bytes) when reading      -->\n  <!--                       resources to be served.  [2048]                -->\n  <!--                                                                      -->\n  <!--   listings            Should directory listings be produced if there -->\n  <!--                       is no welcome file in this directory?  [false] -->\n  <!--                       WARNING: Listings for directories with many    -->\n  <!--                       entries can be slow and may consume            -->\n  <!--                       significant proportions of server resources.   -->\n  <!--                                                                      -->\n  <!--   output              Output buffer size (in bytes) when writing     -->\n  <!--                       resources to be served.  [2048]                -->\n  <!--                                                                      -->\n  <!--   readonly            Is this context \"read only\", so HTTP           -->\n  <!--                       commands like PUT and DELETE are               -->\n  <!--                       rejected?  [true]                              -->\n  <!--                                                                      -->\n  <!--   readmeFile          File to display together with the directory    -->\n  <!--                       contents. [null]                               -->\n  <!--                                                                      -->\n  <!--   sendfileSize        If the connector used supports sendfile, this  -->\n  <!--                       represents the minimal file size in KB for     -->\n  <!--                       which sendfile will be used. Use a negative    -->\n  <!--                       value to always disable sendfile.  [48]        -->\n  <!--                                                                      -->\n  <!--   useAcceptRanges     Should the Accept-Ranges header be included    -->\n  <!--                       in responses where appropriate? [true]         -->\n  <!--                                                                      -->\n  <!--  For directory listing customization. Checks localXsltFile, then     -->\n  <!--  globalXsltFile, then defaults to original behavior.                 -->\n  <!--                                                                      -->\n  <!--   localXsltFile       Make directory listings an XML doc and         -->\n  <!--                       pass the result to this style sheet residing   -->\n  <!--                       in that directory. This overrides              -->\n  <!--                       contextXsltFile and globalXsltFile[null]       -->\n  <!--                                                                      -->\n  <!--   contextXsltFile     Make directory listings an XML doc and         -->\n  <!--                       pass the result to this style sheet which is   -->\n  <!--                       relative to the context root. This overrides   -->\n  <!--                       globalXsltFile[null]                           -->\n  <!--                                                                      -->\n  <!--   globalXsltFile      Site wide configuration version of             -->\n  <!--                       localXsltFile This argument is expected        -->\n  <!--                       to be a physical file. [null]                  -->\n  <!--                                                                      -->\n  <!--                                                                      -->\n\n    <servlet>\n        <servlet-name>default</servlet-name>\n        <servlet-class>org.apache.catalina.servlets.DefaultServlet</servlet-class>\n        <init-param>\n            <param-name>debug</param-name>\n            <param-value>0</param-value>\n        </init-param>\n        <init-param>\n            <param-name>listings</param-name>\n            <param-value>false</param-value>\n        </init-param>\n        <load-on-startup>1</load-on-startup>\n    </servlet>\n\n\n  <!-- The JSP page compiler and execution servlet, which is the mechanism  -->\n  <!-- used by Tomcat to support JSP pages.  Traditionally, this servlet    -->\n  <!-- is mapped to the URL pattern \"*.jsp\".  This servlet supports the     -->\n  <!-- following initialization parameters (default values are in square    -->\n  <!-- brackets):                                                           -->\n  <!--                                                                      -->\n  <!--   checkInterval       If development is false and checkInterval is   -->\n  <!--                       greater than zero, background compilations are -->\n  <!--                       enabled. checkInterval is the time in seconds  -->\n  <!--                       between checks to see if a JSP page (and its   -->\n  <!--                       dependent files) needs to  be recompiled. [0]  -->\n  <!--                                                                      -->\n  <!--   classdebuginfo      Should the class file be compiled with         -->\n  <!--                       debugging information?  [true]                 -->\n  <!--                                                                      -->\n  <!--   classpath           What class path should I use while compiling   -->\n  <!--                       generated servlets?  [Created dynamically      -->\n  <!--                       based on the current web application]          -->\n  <!--                                                                      -->\n  <!--   compiler            Which compiler Ant should use to compile JSP   -->\n  <!--                       pages.  See the jasper documentation for more  -->\n  <!--                       information.                                   -->\n  <!--                                                                      -->\n  <!--   compilerSourceVM    Compiler source VM. [1.6]                      -->\n  <!--                                                                      -->\n  <!--   compilerTargetVM    Compiler target VM. [1.6]                      -->\n  <!--                                                                      -->\n  <!--   development         Is Jasper used in development mode? If true,   -->\n  <!--                       the frequency at which JSPs are checked for    -->\n  <!--                       modification may be specified via the          -->\n  <!--                       modificationTestInterval parameter. [true]     -->\n  <!--                                                                      -->\n  <!--   displaySourceFragment                                              -->\n  <!--                       Should a source fragment be included in        -->\n  <!--                       exception messages? [true]                     -->\n  <!--                                                                      -->\n  <!--   dumpSmap            Should the SMAP info for JSR45 debugging be    -->\n  <!--                       dumped to a file? [false]                      -->\n  <!--                       False if suppressSmap is true                  -->\n  <!--                                                                      -->\n  <!--   enablePooling       Determines whether tag handler pooling is      -->\n  <!--                       enabled. This is a compilation option. It will -->\n  <!--                       not alter the behaviour of JSPs that have      -->\n  <!--                       already been compiled. [true]                  -->\n  <!--                                                                      -->\n  <!--   engineOptionsClass  Allows specifying the Options class used to    -->\n  <!--                       configure Jasper. If not present, the default  -->\n  <!--                       EmbeddedServletOptions will be used.           -->\n  <!--                                                                      -->\n  <!--   errorOnUseBeanInvalidClassAttribute                                -->\n  <!--                       Should Jasper issue an error when the value of -->\n  <!--                       the class attribute in an useBean action is    -->\n  <!--                       not a valid bean class?  [true]                -->\n  <!--                                                                      -->\n  <!--   fork                Tell Ant to fork compiles of JSP pages so that -->\n  <!--                       a separate JVM is used for JSP page compiles   -->\n  <!--                       from the one Tomcat is running in. [true]      -->\n  <!--                                                                      -->\n  <!--   genStringAsCharArray                                               -->\n  <!--                       Should text strings be generated as char       -->\n  <!--                       arrays, to improve performance in some cases?  -->\n  <!--                       [false]                                        -->\n  <!--                                                                      -->\n  <!--   ieClassId           The class-id value to be sent to Internet      -->\n  <!--                       Explorer when using <jsp:plugin> tags.         -->\n  <!--                       [clsid:8AD9C840-044E-11D1-B3E9-00805F499D93]   -->\n  <!--                                                                      -->\n  <!--   javaEncoding        Java file encoding to use for generating java  -->\n  <!--                       source files. [UTF8]                           -->\n  <!--                                                                      -->\n  <!--   keepgenerated       Should we keep the generated Java source code  -->\n  <!--                       for each page instead of deleting it? [true]   -->\n  <!--                                                                      -->\n  <!--   mappedfile          Should we generate static content with one     -->\n  <!--                       print statement per input line, to ease        -->\n  <!--                       debugging?  [true]                             -->\n  <!--                                                                      -->\n  <!--   maxLoadedJsps       The maximum number of JSPs that will be loaded -->\n  <!--                       for a web application. If more than this       -->\n  <!--                       number of JSPs are loaded, the least recently  -->\n  <!--                       used JSPs will be unloaded so that the number  -->\n  <!--                       of JSPs loaded at any one time does not exceed -->\n  <!--                       this limit. A value of zero or less indicates  -->\n  <!--                       no limit. [-1]                                 -->\n  <!--                                                                      -->\n  <!--   jspIdleTimeout      The amount of time in seconds a JSP can be     -->\n  <!--                       idle before it is unloaded. A value of zero    -->\n  <!--                       or less indicates never unload. [-1]           -->\n  <!--                                                                      -->\n  <!--   modificationTestInterval                                           -->\n  <!--                       Causes a JSP (and its dependent files) to not  -->\n  <!--                       be checked for modification during the         -->\n  <!--                       specified time interval (in seconds) from the  -->\n  <!--                       last time the JSP was checked for              -->\n  <!--                       modification. A value of 0 will cause the JSP  -->\n  <!--                       to be checked on every access.                 -->\n  <!--                       Used in development mode only. [4]             -->\n  <!--                                                                      -->\n  <!--   recompileOnFail     If a JSP compilation fails should the          -->\n  <!--                       modificationTestInterval be ignored and the    -->\n  <!--                       next access trigger a re-compilation attempt?  -->\n  <!--                       Used in development mode only and is disabled  -->\n  <!--                       by default as compilation may be expensive and -->\n  <!--                       could lead to excessive resource usage.        -->\n  <!--                       [false]                                        -->\n  <!--                                                                      -->\n  <!--   scratchdir          What scratch directory should we use when      -->\n  <!--                       compiling JSP pages?  [default work directory  -->\n  <!--                       for the current web application]               -->\n  <!--                                                                      -->\n  <!--   suppressSmap        Should the generation of SMAP info for JSR45   -->\n  <!--                       debugging be suppressed?  [false]              -->\n  <!--                                                                      -->\n  <!--   trimSpaces          Should white spaces in template text between   -->\n  <!--                       actions or directives be trimmed?  [false]     -->\n  <!--                                                                      -->\n  <!--   xpoweredBy          Determines whether X-Powered-By response       -->\n  <!--                       header is added by generated servlet.  [false] -->\n\n    <servlet>\n        <servlet-name>jsp</servlet-name>\n        <servlet-class>org.apache.jasper.servlet.JspServlet</servlet-class>\n        <init-param>\n            <param-name>fork</param-name>\n            <param-value>false</param-value>\n        </init-param>\n        <init-param>\n            <param-name>xpoweredBy</param-name>\n            <param-value>false</param-value>\n        </init-param>\n        <load-on-startup>3</load-on-startup>\n    </servlet>\n\n\n  <!-- NOTE: An SSI Filter is also available as an alternative SSI          -->\n  <!-- implementation. Use either the Servlet or the Filter but NOT both.   -->\n  <!--                                                                      -->\n  <!-- Server Side Includes processing servlet, which processes SSI         -->\n  <!-- directives in HTML pages consistent with similar support in web      -->\n  <!-- servers like Apache.  Traditionally, this servlet is mapped to the   -->\n  <!-- URL pattern \"*.shtml\".  This servlet supports the following          -->\n  <!-- initialization parameters (default values are in square brackets):   -->\n  <!--                                                                      -->\n  <!--   buffered            Should output from this servlet be buffered?   -->\n  <!--                       (0=false, 1=true)  [0]                         -->\n  <!--                                                                      -->\n  <!--   debug               Debugging detail level for messages logged     -->\n  <!--                       by this servlet.  [0]                          -->\n  <!--                                                                      -->\n  <!--   expires             The number of seconds before a page with SSI   -->\n  <!--                       directives will expire.  [No default]          -->\n  <!--                                                                      -->\n  <!--   isVirtualWebappRelative                                            -->\n  <!--                       Should \"virtual\" paths be interpreted as       -->\n  <!--                       relative to the context root, instead of       -->\n  <!--                       the server root? [false]                       -->\n  <!--                                                                      -->\n  <!--   inputEncoding       The encoding to assume for SSI resources if    -->\n  <!--                       one is not available from the resource.        -->\n  <!--                       [Platform default]                             -->\n  <!--                                                                      -->\n  <!--   outputEncoding      The encoding to use for the page that results  -->\n  <!--                       from the SSI processing. [UTF-8]               -->\n  <!--                                                                      -->\n  <!--   allowExec           Is use of the exec command enabled? [false]    -->\n\n<!--\n    <servlet>\n        <servlet-name>ssi</servlet-name>\n        <servlet-class>\n          org.apache.catalina.ssi.SSIServlet\n        </servlet-class>\n        <init-param>\n          <param-name>buffered</param-name>\n          <param-value>1</param-value>\n        </init-param>\n        <init-param>\n          <param-name>debug</param-name>\n          <param-value>0</param-value>\n        </init-param>\n        <init-param>\n          <param-name>expires</param-name>\n          <param-value>666</param-value>\n        </init-param>\n        <init-param>\n          <param-name>isVirtualWebappRelative</param-name>\n          <param-value>false</param-value>\n        </init-param>\n        <load-on-startup>4</load-on-startup>\n    </servlet>\n-->\n\n\n  <!-- Common Gateway Includes (CGI) processing servlet, which supports     -->\n  <!-- execution of external applications that conform to the CGI spec      -->\n  <!-- requirements.  Typically, this servlet is mapped to the URL pattern  -->\n  <!-- \"/cgi-bin/*\", which means that any CGI applications that are         -->\n  <!-- executed must be present within the web application.  This servlet   -->\n  <!-- supports the following initialization parameters (default values     -->\n  <!-- are in square brackets):                                             -->\n  <!--                                                                      -->\n  <!--   cgiPathPrefix        The CGI search path will start at             -->\n  <!--                        webAppRootDir + File.separator + this prefix. -->\n  <!--                        [WEB-INF/cgi]                                 -->\n  <!--                                                                      -->\n  <!--   debug                Debugging detail level for messages logged    -->\n  <!--                        by this servlet.  [0]                         -->\n  <!--                                                                      -->\n  <!--   executable           Name of the executable used to run the        -->\n  <!--                        script. [perl]                                -->\n  <!--                                                                      -->\n  <!--   parameterEncoding    Name of parameter encoding to be used with    -->\n  <!--                        CGI servlet.                                  -->\n  <!--                        [System.getProperty(\"file.encoding\",\"UTF-8\")] -->\n  <!--                                                                      -->\n  <!--   passShellEnvironment Should the shell environment variables (if    -->\n  <!--                        any) be passed to the CGI script? [false]     -->\n  <!--                                                                      -->\n  <!--   stderrTimeout        The time (in milliseconds) to wait for the    -->\n  <!--                        reading of stderr to complete before          -->\n  <!--                        terminating the CGI process. [2000]           -->\n\n<!--\n    <servlet>\n        <servlet-name>cgi</servlet-name>\n        <servlet-class>org.apache.catalina.servlets.CGIServlet</servlet-class>\n        <init-param>\n          <param-name>debug</param-name>\n          <param-value>0</param-value>\n        </init-param>\n        <init-param>\n          <param-name>cgiPathPrefix</param-name>\n          <param-value>WEB-INF/cgi</param-value>\n        </init-param>\n         <load-on-startup>5</load-on-startup>\n    </servlet>\n-->\n\n\n  <!-- ================ Built In Servlet Mappings ========================= -->\n\n\n  <!-- The servlet mappings for the built in servlets defined above.  Note  -->\n  <!-- that, by default, the CGI and SSI servlets are *not* mapped.  You    -->\n  <!-- must uncomment these mappings (or add them to your application's own -->\n  <!-- web.xml deployment descriptor) to enable these services              -->\n\n    <!-- The mapping for the default servlet -->\n    <servlet-mapping>\n        <servlet-name>default</servlet-name>\n        <url-pattern>/</url-pattern>\n    </servlet-mapping>\n\n    <!-- The mappings for the JSP servlet -->\n    <servlet-mapping>\n        <servlet-name>jsp</servlet-name>\n        <url-pattern>*.jsp</url-pattern>\n        <url-pattern>*.jspx</url-pattern>\n    </servlet-mapping>\n\n    <!-- The mapping for the SSI servlet -->\n<!--\n    <servlet-mapping>\n        <servlet-name>ssi</servlet-name>\n        <url-pattern>*.shtml</url-pattern>\n    </servlet-mapping>\n-->\n\n    <!-- The mapping for the CGI Gateway servlet -->\n\n<!--\n    <servlet-mapping>\n        <servlet-name>cgi</servlet-name>\n        <url-pattern>/cgi-bin/*</url-pattern>\n    </servlet-mapping>\n-->\n\n\n  <!-- ================== Built In Filter Definitions ===================== -->\n\n  <!-- A filter that sets character encoding that is used to decode -->\n  <!-- parameters in a POST request -->\n<!--\n    <filter>\n        <filter-name>setCharacterEncodingFilter</filter-name>\n        <filter-class>org.apache.catalina.filters.SetCharacterEncodingFilter</filter-class>\n        <init-param>\n            <param-name>encoding</param-name>\n            <param-value>UTF-8</param-value>\n        </init-param>\n        <async-supported>true</async-supported>\n    </filter>\n-->\n\n  <!-- A filter that triggers request parameters parsing and rejects the    -->\n  <!-- request if some parameters were skipped because of parsing errors or -->\n  <!-- request size limitations.                                            -->\n<!--\n    <filter>\n        <filter-name>failedRequestFilter</filter-name>\n        <filter-class>\n          org.apache.catalina.filters.FailedRequestFilter\n        </filter-class>\n        <async-supported>true</async-supported>\n    </filter>\n-->\n\n\n  <!-- NOTE: An SSI Servlet is also available as an alternative SSI         -->\n  <!-- implementation. Use either the Servlet or the Filter but NOT both.   -->\n  <!--                                                                      -->\n  <!-- Server Side Includes processing filter, which processes SSI          -->\n  <!-- directives in HTML pages consistent with similar support in web      -->\n  <!-- servers like Apache.  Traditionally, this filter is mapped to the    -->\n  <!-- URL pattern \"*.shtml\", though it can be mapped to \"*\" as it will     -->\n  <!-- selectively enable/disable SSI processing based on mime types. For   -->\n  <!-- this to work you will need to uncomment the .shtml mime type         -->\n  <!-- definition towards the bottom of this file.                          -->\n  <!-- The contentType init param allows you to apply SSI processing to JSP -->\n  <!-- pages, javascript, or any other content you wish.  This filter       -->\n  <!-- supports the following initialization parameters (default values are -->\n  <!-- in square brackets):                                                 -->\n  <!--                                                                      -->\n  <!--   contentType         A regex pattern that must be matched before    -->\n  <!--                       SSI processing is applied.                     -->\n  <!--                       [text/x-server-parsed-html(;.*)?]              -->\n  <!--                                                                      -->\n  <!--   debug               Debugging detail level for messages logged     -->\n  <!--                       by this servlet.  [0]                          -->\n  <!--                                                                      -->\n  <!--   expires             The number of seconds before a page with SSI   -->\n  <!--                       directives will expire.  [No default]          -->\n  <!--                                                                      -->\n  <!--   isVirtualWebappRelative                                            -->\n  <!--                       Should \"virtual\" paths be interpreted as       -->\n  <!--                       relative to the context root, instead of       -->\n  <!--                       the server root? [false]                       -->\n  <!--                                                                      -->\n  <!--   allowExec           Is use of the exec command enabled? [false]    -->\n\n<!--\n    <filter>\n        <filter-name>ssi</filter-name>\n        <filter-class>\n          org.apache.catalina.ssi.SSIFilter\n        </filter-class>\n        <init-param>\n          <param-name>contentType</param-name>\n          <param-value>text/x-server-parsed-html(;.*)?</param-value>\n        </init-param>\n        <init-param>\n          <param-name>debug</param-name>\n          <param-value>0</param-value>\n        </init-param>\n        <init-param>\n          <param-name>expires</param-name>\n          <param-value>666</param-value>\n        </init-param>\n        <init-param>\n          <param-name>isVirtualWebappRelative</param-name>\n          <param-value>false</param-value>\n        </init-param>\n    </filter>\n-->\n\n\n  <!-- ==================== Built In Filter Mappings ====================== -->\n\n  <!-- The mapping for the Set Character Encoding Filter -->\n<!--\n    <filter-mapping>\n        <filter-name>setCharacterEncodingFilter</filter-name>\n        <url-pattern>/*</url-pattern>\n    </filter-mapping>\n-->\n\n  <!-- The mapping for the Failed Request Filter -->\n<!--\n    <filter-mapping>\n        <filter-name>failedRequestFilter</filter-name>\n        <url-pattern>/*</url-pattern>\n    </filter-mapping>\n-->\n\n  <!-- The mapping for the SSI Filter -->\n<!--\n    <filter-mapping>\n        <filter-name>ssi</filter-name>\n        <url-pattern>*.shtml</url-pattern>\n    </filter-mapping>\n-->\n\n\n  <!-- ==================== Default Session Configuration ================= -->\n  <!-- You can set the default session timeout (in minutes) for all newly   -->\n  <!-- created sessions by modifying the value below.                       -->\n\n    <session-config>\n        <session-timeout>30</session-timeout>\n    </session-config>\n\n\n  <!-- ===================== Default MIME Type Mappings =================== -->\n  <!-- When serving static resources, Tomcat will automatically generate    -->\n  <!-- a \"Content-Type\" header based on the resource's filename extension,  -->\n  <!-- based on these mappings.  Additional mappings can be added here (to  -->\n  <!-- apply to all web applications), or in your own application's web.xml -->\n  <!-- deployment descriptor.                                               -->\n\n    <mime-mapping>\n        <extension>123</extension>\n        <mime-type>application/vnd.lotus-1-2-3</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>3dml</extension>\n        <mime-type>text/vnd.in3d.3dml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>3ds</extension>\n        <mime-type>image/x-3ds</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>3g2</extension>\n        <mime-type>video/3gpp2</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>3gp</extension>\n        <mime-type>video/3gpp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>7z</extension>\n        <mime-type>application/x-7z-compressed</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>aab</extension>\n        <mime-type>application/x-authorware-bin</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>aac</extension>\n        <mime-type>audio/x-aac</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>aam</extension>\n        <mime-type>application/x-authorware-map</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>aas</extension>\n        <mime-type>application/x-authorware-seg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>abs</extension>\n        <mime-type>audio/x-mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>abw</extension>\n        <mime-type>application/x-abiword</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ac</extension>\n        <mime-type>application/pkix-attr-cert</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>acc</extension>\n        <mime-type>application/vnd.americandynamics.acc</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ace</extension>\n        <mime-type>application/x-ace-compressed</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>acu</extension>\n        <mime-type>application/vnd.acucobol</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>acutc</extension>\n        <mime-type>application/vnd.acucorp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>adp</extension>\n        <mime-type>audio/adpcm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>aep</extension>\n        <mime-type>application/vnd.audiograph</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>afm</extension>\n        <mime-type>application/x-font-type1</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>afp</extension>\n        <mime-type>application/vnd.ibm.modcap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ahead</extension>\n        <mime-type>application/vnd.ahead.space</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ai</extension>\n        <mime-type>application/postscript</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>aif</extension>\n        <mime-type>audio/x-aiff</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>aifc</extension>\n        <mime-type>audio/x-aiff</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>aiff</extension>\n        <mime-type>audio/x-aiff</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>aim</extension>\n        <mime-type>application/x-aim</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>air</extension>\n        <mime-type>application/vnd.adobe.air-application-installer-package+zip</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ait</extension>\n        <mime-type>application/vnd.dvb.ait</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ami</extension>\n        <mime-type>application/vnd.amiga.ami</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>anx</extension>\n        <mime-type>application/annodex</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>apk</extension>\n        <mime-type>application/vnd.android.package-archive</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>appcache</extension>\n        <mime-type>text/cache-manifest</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>application</extension>\n        <mime-type>application/x-ms-application</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>apr</extension>\n        <mime-type>application/vnd.lotus-approach</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>arc</extension>\n        <mime-type>application/x-freearc</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>art</extension>\n        <mime-type>image/x-jg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>asc</extension>\n        <mime-type>application/pgp-signature</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>asf</extension>\n        <mime-type>video/x-ms-asf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>asm</extension>\n        <mime-type>text/x-asm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>aso</extension>\n        <mime-type>application/vnd.accpac.simply.aso</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>asx</extension>\n        <mime-type>video/x-ms-asf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>atc</extension>\n        <mime-type>application/vnd.acucorp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>atom</extension>\n        <mime-type>application/atom+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>atomcat</extension>\n        <mime-type>application/atomcat+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>atomsvc</extension>\n        <mime-type>application/atomsvc+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>atx</extension>\n        <mime-type>application/vnd.antix.game-component</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>au</extension>\n        <mime-type>audio/basic</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>avi</extension>\n        <mime-type>video/x-msvideo</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>avx</extension>\n        <mime-type>video/x-rad-screenplay</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>aw</extension>\n        <mime-type>application/applixware</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>axa</extension>\n        <mime-type>audio/annodex</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>axv</extension>\n        <mime-type>video/annodex</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>azf</extension>\n        <mime-type>application/vnd.airzip.filesecure.azf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>azs</extension>\n        <mime-type>application/vnd.airzip.filesecure.azs</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>azw</extension>\n        <mime-type>application/vnd.amazon.ebook</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>bat</extension>\n        <mime-type>application/x-msdownload</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>bcpio</extension>\n        <mime-type>application/x-bcpio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>bdf</extension>\n        <mime-type>application/x-font-bdf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>bdm</extension>\n        <mime-type>application/vnd.syncml.dm+wbxml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>bed</extension>\n        <mime-type>application/vnd.realvnc.bed</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>bh2</extension>\n        <mime-type>application/vnd.fujitsu.oasysprs</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>bin</extension>\n        <mime-type>application/octet-stream</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>blb</extension>\n        <mime-type>application/x-blorb</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>blorb</extension>\n        <mime-type>application/x-blorb</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>bmi</extension>\n        <mime-type>application/vnd.bmi</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>bmp</extension>\n        <mime-type>image/bmp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>body</extension>\n        <mime-type>text/html</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>book</extension>\n        <mime-type>application/vnd.framemaker</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>box</extension>\n        <mime-type>application/vnd.previewsystems.box</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>boz</extension>\n        <mime-type>application/x-bzip2</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>bpk</extension>\n        <mime-type>application/octet-stream</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>btif</extension>\n        <mime-type>image/prs.btif</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>bz</extension>\n        <mime-type>application/x-bzip</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>bz2</extension>\n        <mime-type>application/x-bzip2</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>c</extension>\n        <mime-type>text/x-c</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>c11amc</extension>\n        <mime-type>application/vnd.cluetrust.cartomobile-config</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>c11amz</extension>\n        <mime-type>application/vnd.cluetrust.cartomobile-config-pkg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>c4d</extension>\n        <mime-type>application/vnd.clonk.c4group</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>c4f</extension>\n        <mime-type>application/vnd.clonk.c4group</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>c4g</extension>\n        <mime-type>application/vnd.clonk.c4group</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>c4p</extension>\n        <mime-type>application/vnd.clonk.c4group</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>c4u</extension>\n        <mime-type>application/vnd.clonk.c4group</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cab</extension>\n        <mime-type>application/vnd.ms-cab-compressed</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>caf</extension>\n        <mime-type>audio/x-caf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cap</extension>\n        <mime-type>application/vnd.tcpdump.pcap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>car</extension>\n        <mime-type>application/vnd.curl.car</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cat</extension>\n        <mime-type>application/vnd.ms-pki.seccat</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cb7</extension>\n        <mime-type>application/x-cbr</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cba</extension>\n        <mime-type>application/x-cbr</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cbr</extension>\n        <mime-type>application/x-cbr</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cbt</extension>\n        <mime-type>application/x-cbr</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cbz</extension>\n        <mime-type>application/x-cbr</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cc</extension>\n        <mime-type>text/x-c</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cct</extension>\n        <mime-type>application/x-director</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ccxml</extension>\n        <mime-type>application/ccxml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cdbcmsg</extension>\n        <mime-type>application/vnd.contact.cmsg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cdf</extension>\n        <mime-type>application/x-cdf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cdkey</extension>\n        <mime-type>application/vnd.mediastation.cdkey</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cdmia</extension>\n        <mime-type>application/cdmi-capability</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cdmic</extension>\n        <mime-type>application/cdmi-container</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cdmid</extension>\n        <mime-type>application/cdmi-domain</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cdmio</extension>\n        <mime-type>application/cdmi-object</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cdmiq</extension>\n        <mime-type>application/cdmi-queue</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cdx</extension>\n        <mime-type>chemical/x-cdx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cdxml</extension>\n        <mime-type>application/vnd.chemdraw+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cdy</extension>\n        <mime-type>application/vnd.cinderella</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cer</extension>\n        <mime-type>application/pkix-cert</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cfs</extension>\n        <mime-type>application/x-cfs-compressed</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cgm</extension>\n        <mime-type>image/cgm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>chat</extension>\n        <mime-type>application/x-chat</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>chm</extension>\n        <mime-type>application/vnd.ms-htmlhelp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>chrt</extension>\n        <mime-type>application/vnd.kde.kchart</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cif</extension>\n        <mime-type>chemical/x-cif</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cii</extension>\n        <mime-type>application/vnd.anser-web-certificate-issue-initiation</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cil</extension>\n        <mime-type>application/vnd.ms-artgalry</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cla</extension>\n        <mime-type>application/vnd.claymore</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>class</extension>\n        <mime-type>application/java</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>clkk</extension>\n        <mime-type>application/vnd.crick.clicker.keyboard</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>clkp</extension>\n        <mime-type>application/vnd.crick.clicker.palette</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>clkt</extension>\n        <mime-type>application/vnd.crick.clicker.template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>clkw</extension>\n        <mime-type>application/vnd.crick.clicker.wordbank</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>clkx</extension>\n        <mime-type>application/vnd.crick.clicker</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>clp</extension>\n        <mime-type>application/x-msclip</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cmc</extension>\n        <mime-type>application/vnd.cosmocaller</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cmdf</extension>\n        <mime-type>chemical/x-cmdf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cml</extension>\n        <mime-type>chemical/x-cml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cmp</extension>\n        <mime-type>application/vnd.yellowriver-custom-menu</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cmx</extension>\n        <mime-type>image/x-cmx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cod</extension>\n        <mime-type>application/vnd.rim.cod</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>com</extension>\n        <mime-type>application/x-msdownload</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>conf</extension>\n        <mime-type>text/plain</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cpio</extension>\n        <mime-type>application/x-cpio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cpp</extension>\n        <mime-type>text/x-c</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cpt</extension>\n        <mime-type>application/mac-compactpro</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>crd</extension>\n        <mime-type>application/x-mscardfile</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>crl</extension>\n        <mime-type>application/pkix-crl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>crt</extension>\n        <mime-type>application/x-x509-ca-cert</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cryptonote</extension>\n        <mime-type>application/vnd.rig.cryptonote</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>csh</extension>\n        <mime-type>application/x-csh</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>csml</extension>\n        <mime-type>chemical/x-csml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>csp</extension>\n        <mime-type>application/vnd.commonspace</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>css</extension>\n        <mime-type>text/css</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cst</extension>\n        <mime-type>application/x-director</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>csv</extension>\n        <mime-type>text/csv</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cu</extension>\n        <mime-type>application/cu-seeme</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>curl</extension>\n        <mime-type>text/vnd.curl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cww</extension>\n        <mime-type>application/prs.cww</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cxt</extension>\n        <mime-type>application/x-director</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>cxx</extension>\n        <mime-type>text/x-c</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dae</extension>\n        <mime-type>model/vnd.collada+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>daf</extension>\n        <mime-type>application/vnd.mobius.daf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dart</extension>\n        <mime-type>application/vnd.dart</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dataless</extension>\n        <mime-type>application/vnd.fdsn.seed</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>davmount</extension>\n        <mime-type>application/davmount+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dbk</extension>\n        <mime-type>application/docbook+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dcr</extension>\n        <mime-type>application/x-director</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dcurl</extension>\n        <mime-type>text/vnd.curl.dcurl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dd2</extension>\n        <mime-type>application/vnd.oma.dd2+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ddd</extension>\n        <mime-type>application/vnd.fujixerox.ddd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>deb</extension>\n        <mime-type>application/x-debian-package</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>def</extension>\n        <mime-type>text/plain</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>deploy</extension>\n        <mime-type>application/octet-stream</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>der</extension>\n        <mime-type>application/x-x509-ca-cert</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dfac</extension>\n        <mime-type>application/vnd.dreamfactory</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dgc</extension>\n        <mime-type>application/x-dgc-compressed</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dib</extension>\n        <mime-type>image/bmp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dic</extension>\n        <mime-type>text/x-c</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dir</extension>\n        <mime-type>application/x-director</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dis</extension>\n        <mime-type>application/vnd.mobius.dis</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dist</extension>\n        <mime-type>application/octet-stream</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>distz</extension>\n        <mime-type>application/octet-stream</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>djv</extension>\n        <mime-type>image/vnd.djvu</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>djvu</extension>\n        <mime-type>image/vnd.djvu</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dll</extension>\n        <mime-type>application/x-msdownload</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dmg</extension>\n        <mime-type>application/x-apple-diskimage</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dmp</extension>\n        <mime-type>application/vnd.tcpdump.pcap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dms</extension>\n        <mime-type>application/octet-stream</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dna</extension>\n        <mime-type>application/vnd.dna</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>doc</extension>\n        <mime-type>application/msword</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>docm</extension>\n        <mime-type>application/vnd.ms-word.document.macroenabled.12</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>docx</extension>\n        <mime-type>application/vnd.openxmlformats-officedocument.wordprocessingml.document</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dot</extension>\n        <mime-type>application/msword</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dotm</extension>\n        <mime-type>application/vnd.ms-word.template.macroenabled.12</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dotx</extension>\n        <mime-type>application/vnd.openxmlformats-officedocument.wordprocessingml.template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dp</extension>\n        <mime-type>application/vnd.osgi.dp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dpg</extension>\n        <mime-type>application/vnd.dpgraph</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dra</extension>\n        <mime-type>audio/vnd.dra</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dsc</extension>\n        <mime-type>text/prs.lines.tag</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dssc</extension>\n        <mime-type>application/dssc+der</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dtb</extension>\n        <mime-type>application/x-dtbook+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dtd</extension>\n        <mime-type>application/xml-dtd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dts</extension>\n        <mime-type>audio/vnd.dts</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dtshd</extension>\n        <mime-type>audio/vnd.dts.hd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dump</extension>\n        <mime-type>application/octet-stream</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dv</extension>\n        <mime-type>video/x-dv</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dvb</extension>\n        <mime-type>video/vnd.dvb.file</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dvi</extension>\n        <mime-type>application/x-dvi</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dwf</extension>\n        <mime-type>model/vnd.dwf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dwg</extension>\n        <mime-type>image/vnd.dwg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dxf</extension>\n        <mime-type>image/vnd.dxf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dxp</extension>\n        <mime-type>application/vnd.spotfire.dxp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>dxr</extension>\n        <mime-type>application/x-director</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ecelp4800</extension>\n        <mime-type>audio/vnd.nuera.ecelp4800</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ecelp7470</extension>\n        <mime-type>audio/vnd.nuera.ecelp7470</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ecelp9600</extension>\n        <mime-type>audio/vnd.nuera.ecelp9600</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ecma</extension>\n        <mime-type>application/ecmascript</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>edm</extension>\n        <mime-type>application/vnd.novadigm.edm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>edx</extension>\n        <mime-type>application/vnd.novadigm.edx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>efif</extension>\n        <mime-type>application/vnd.picsel</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ei6</extension>\n        <mime-type>application/vnd.pg.osasli</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>elc</extension>\n        <mime-type>application/octet-stream</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>emf</extension>\n        <mime-type>application/x-msmetafile</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>eml</extension>\n        <mime-type>message/rfc822</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>emma</extension>\n        <mime-type>application/emma+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>emz</extension>\n        <mime-type>application/x-msmetafile</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>eol</extension>\n        <mime-type>audio/vnd.digital-winds</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>eot</extension>\n        <mime-type>application/vnd.ms-fontobject</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>eps</extension>\n        <mime-type>application/postscript</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>epub</extension>\n        <mime-type>application/epub+zip</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>es3</extension>\n        <mime-type>application/vnd.eszigno3+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>esa</extension>\n        <mime-type>application/vnd.osgi.subsystem</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>esf</extension>\n        <mime-type>application/vnd.epson.esf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>et3</extension>\n        <mime-type>application/vnd.eszigno3+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>etx</extension>\n        <mime-type>text/x-setext</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>eva</extension>\n        <mime-type>application/x-eva</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>evy</extension>\n        <mime-type>application/x-envoy</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>exe</extension>\n        <mime-type>application/octet-stream</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>exi</extension>\n        <mime-type>application/exi</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ext</extension>\n        <mime-type>application/vnd.novadigm.ext</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ez</extension>\n        <mime-type>application/andrew-inset</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ez2</extension>\n        <mime-type>application/vnd.ezpix-album</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ez3</extension>\n        <mime-type>application/vnd.ezpix-package</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>f</extension>\n        <mime-type>text/x-fortran</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>f4v</extension>\n        <mime-type>video/x-f4v</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>f77</extension>\n        <mime-type>text/x-fortran</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>f90</extension>\n        <mime-type>text/x-fortran</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fbs</extension>\n        <mime-type>image/vnd.fastbidsheet</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fcdt</extension>\n        <mime-type>application/vnd.adobe.formscentral.fcdt</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fcs</extension>\n        <mime-type>application/vnd.isac.fcs</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fdf</extension>\n        <mime-type>application/vnd.fdf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fe_launch</extension>\n        <mime-type>application/vnd.denovo.fcselayout-link</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fg5</extension>\n        <mime-type>application/vnd.fujitsu.oasysgp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fgd</extension>\n        <mime-type>application/x-director</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fh</extension>\n        <mime-type>image/x-freehand</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fh4</extension>\n        <mime-type>image/x-freehand</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fh5</extension>\n        <mime-type>image/x-freehand</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fh7</extension>\n        <mime-type>image/x-freehand</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fhc</extension>\n        <mime-type>image/x-freehand</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fig</extension>\n        <mime-type>application/x-xfig</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>flac</extension>\n        <mime-type>audio/flac</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fli</extension>\n        <mime-type>video/x-fli</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>flo</extension>\n        <mime-type>application/vnd.micrografx.flo</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>flv</extension>\n        <mime-type>video/x-flv</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>flw</extension>\n        <mime-type>application/vnd.kde.kivio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>flx</extension>\n        <mime-type>text/vnd.fmi.flexstor</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fly</extension>\n        <mime-type>text/vnd.fly</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fm</extension>\n        <mime-type>application/vnd.framemaker</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fnc</extension>\n        <mime-type>application/vnd.frogans.fnc</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>for</extension>\n        <mime-type>text/x-fortran</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fpx</extension>\n        <mime-type>image/vnd.fpx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>frame</extension>\n        <mime-type>application/vnd.framemaker</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fsc</extension>\n        <mime-type>application/vnd.fsc.weblaunch</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fst</extension>\n        <mime-type>image/vnd.fst</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ftc</extension>\n        <mime-type>application/vnd.fluxtime.clip</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fti</extension>\n        <mime-type>application/vnd.anser-web-funds-transfer-initiation</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fvt</extension>\n        <mime-type>video/vnd.fvt</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fxp</extension>\n        <mime-type>application/vnd.adobe.fxp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fxpl</extension>\n        <mime-type>application/vnd.adobe.fxp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>fzs</extension>\n        <mime-type>application/vnd.fuzzysheet</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>g2w</extension>\n        <mime-type>application/vnd.geoplan</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>g3</extension>\n        <mime-type>image/g3fax</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>g3w</extension>\n        <mime-type>application/vnd.geospace</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gac</extension>\n        <mime-type>application/vnd.groove-account</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gam</extension>\n        <mime-type>application/x-tads</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gbr</extension>\n        <mime-type>application/rpki-ghostbusters</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gca</extension>\n        <mime-type>application/x-gca-compressed</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gdl</extension>\n        <mime-type>model/vnd.gdl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>geo</extension>\n        <mime-type>application/vnd.dynageo</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gex</extension>\n        <mime-type>application/vnd.geometry-explorer</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ggb</extension>\n        <mime-type>application/vnd.geogebra.file</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ggt</extension>\n        <mime-type>application/vnd.geogebra.tool</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ghf</extension>\n        <mime-type>application/vnd.groove-help</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gif</extension>\n        <mime-type>image/gif</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gim</extension>\n        <mime-type>application/vnd.groove-identity-message</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gml</extension>\n        <mime-type>application/gml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gmx</extension>\n        <mime-type>application/vnd.gmx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gnumeric</extension>\n        <mime-type>application/x-gnumeric</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gph</extension>\n        <mime-type>application/vnd.flographit</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gpx</extension>\n        <mime-type>application/gpx+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gqf</extension>\n        <mime-type>application/vnd.grafeq</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gqs</extension>\n        <mime-type>application/vnd.grafeq</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gram</extension>\n        <mime-type>application/srgs</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gramps</extension>\n        <mime-type>application/x-gramps-xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gre</extension>\n        <mime-type>application/vnd.geometry-explorer</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>grv</extension>\n        <mime-type>application/vnd.groove-injector</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>grxml</extension>\n        <mime-type>application/srgs+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gsf</extension>\n        <mime-type>application/x-font-ghostscript</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gtar</extension>\n        <mime-type>application/x-gtar</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gtm</extension>\n        <mime-type>application/vnd.groove-tool-message</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gtw</extension>\n        <mime-type>model/vnd.gtw</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gv</extension>\n        <mime-type>text/vnd.graphviz</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gxf</extension>\n        <mime-type>application/gxf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gxt</extension>\n        <mime-type>application/vnd.geonext</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>gz</extension>\n        <mime-type>application/x-gzip</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>h</extension>\n        <mime-type>text/x-c</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>h261</extension>\n        <mime-type>video/h261</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>h263</extension>\n        <mime-type>video/h263</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>h264</extension>\n        <mime-type>video/h264</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>hal</extension>\n        <mime-type>application/vnd.hal+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>hbci</extension>\n        <mime-type>application/vnd.hbci</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>hdf</extension>\n        <mime-type>application/x-hdf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>hh</extension>\n        <mime-type>text/x-c</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>hlp</extension>\n        <mime-type>application/winhlp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>hpgl</extension>\n        <mime-type>application/vnd.hp-hpgl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>hpid</extension>\n        <mime-type>application/vnd.hp-hpid</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>hps</extension>\n        <mime-type>application/vnd.hp-hps</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>hqx</extension>\n        <mime-type>application/mac-binhex40</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>htc</extension>\n        <mime-type>text/x-component</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>htke</extension>\n        <mime-type>application/vnd.kenameaapp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>htm</extension>\n        <mime-type>text/html</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>html</extension>\n        <mime-type>text/html</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>hvd</extension>\n        <mime-type>application/vnd.yamaha.hv-dic</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>hvp</extension>\n        <mime-type>application/vnd.yamaha.hv-voice</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>hvs</extension>\n        <mime-type>application/vnd.yamaha.hv-script</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>i2g</extension>\n        <mime-type>application/vnd.intergeo</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>icc</extension>\n        <mime-type>application/vnd.iccprofile</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ice</extension>\n        <mime-type>x-conference/x-cooltalk</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>icm</extension>\n        <mime-type>application/vnd.iccprofile</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ico</extension>\n        <mime-type>image/x-icon</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ics</extension>\n        <mime-type>text/calendar</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ief</extension>\n        <mime-type>image/ief</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ifb</extension>\n        <mime-type>text/calendar</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ifm</extension>\n        <mime-type>application/vnd.shana.informed.formdata</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>iges</extension>\n        <mime-type>model/iges</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>igl</extension>\n        <mime-type>application/vnd.igloader</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>igm</extension>\n        <mime-type>application/vnd.insors.igm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>igs</extension>\n        <mime-type>model/iges</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>igx</extension>\n        <mime-type>application/vnd.micrografx.igx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>iif</extension>\n        <mime-type>application/vnd.shana.informed.interchange</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>imp</extension>\n        <mime-type>application/vnd.accpac.simply.imp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ims</extension>\n        <mime-type>application/vnd.ms-ims</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>in</extension>\n        <mime-type>text/plain</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ink</extension>\n        <mime-type>application/inkml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>inkml</extension>\n        <mime-type>application/inkml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>install</extension>\n        <mime-type>application/x-install-instructions</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>iota</extension>\n        <mime-type>application/vnd.astraea-software.iota</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ipfix</extension>\n        <mime-type>application/ipfix</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ipk</extension>\n        <mime-type>application/vnd.shana.informed.package</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>irm</extension>\n        <mime-type>application/vnd.ibm.rights-management</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>irp</extension>\n        <mime-type>application/vnd.irepository.package+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>iso</extension>\n        <mime-type>application/x-iso9660-image</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>itp</extension>\n        <mime-type>application/vnd.shana.informed.formtemplate</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ivp</extension>\n        <mime-type>application/vnd.immervision-ivp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ivu</extension>\n        <mime-type>application/vnd.immervision-ivu</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jad</extension>\n        <mime-type>text/vnd.sun.j2me.app-descriptor</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jam</extension>\n        <mime-type>application/vnd.jam</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jar</extension>\n        <mime-type>application/java-archive</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>java</extension>\n        <mime-type>text/x-java-source</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jisp</extension>\n        <mime-type>application/vnd.jisp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jlt</extension>\n        <mime-type>application/vnd.hp-jlyt</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jnlp</extension>\n        <mime-type>application/x-java-jnlp-file</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>joda</extension>\n        <mime-type>application/vnd.joost.joda-archive</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jpe</extension>\n        <mime-type>image/jpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jpeg</extension>\n        <mime-type>image/jpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jpg</extension>\n        <mime-type>image/jpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jpgm</extension>\n        <mime-type>video/jpm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jpgv</extension>\n        <mime-type>video/jpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jpm</extension>\n        <mime-type>video/jpm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>js</extension>\n        <mime-type>application/javascript</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jsf</extension>\n        <mime-type>text/plain</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>json</extension>\n        <mime-type>application/json</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jsonml</extension>\n        <mime-type>application/jsonml+json</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>jspf</extension>\n        <mime-type>text/plain</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>kar</extension>\n        <mime-type>audio/midi</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>karbon</extension>\n        <mime-type>application/vnd.kde.karbon</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>kfo</extension>\n        <mime-type>application/vnd.kde.kformula</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>kia</extension>\n        <mime-type>application/vnd.kidspiration</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>kml</extension>\n        <mime-type>application/vnd.google-earth.kml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>kmz</extension>\n        <mime-type>application/vnd.google-earth.kmz</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>kne</extension>\n        <mime-type>application/vnd.kinar</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>knp</extension>\n        <mime-type>application/vnd.kinar</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>kon</extension>\n        <mime-type>application/vnd.kde.kontour</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>kpr</extension>\n        <mime-type>application/vnd.kde.kpresenter</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>kpt</extension>\n        <mime-type>application/vnd.kde.kpresenter</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>kpxx</extension>\n        <mime-type>application/vnd.ds-keypoint</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ksp</extension>\n        <mime-type>application/vnd.kde.kspread</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ktr</extension>\n        <mime-type>application/vnd.kahootz</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ktx</extension>\n        <mime-type>image/ktx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ktz</extension>\n        <mime-type>application/vnd.kahootz</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>kwd</extension>\n        <mime-type>application/vnd.kde.kword</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>kwt</extension>\n        <mime-type>application/vnd.kde.kword</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>lasxml</extension>\n        <mime-type>application/vnd.las.las+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>latex</extension>\n        <mime-type>application/x-latex</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>lbd</extension>\n        <mime-type>application/vnd.llamagraphics.life-balance.desktop</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>lbe</extension>\n        <mime-type>application/vnd.llamagraphics.life-balance.exchange+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>les</extension>\n        <mime-type>application/vnd.hhe.lesson-player</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>lha</extension>\n        <mime-type>application/x-lzh-compressed</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>link66</extension>\n        <mime-type>application/vnd.route66.link66+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>list</extension>\n        <mime-type>text/plain</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>list3820</extension>\n        <mime-type>application/vnd.ibm.modcap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>listafp</extension>\n        <mime-type>application/vnd.ibm.modcap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>lnk</extension>\n        <mime-type>application/x-ms-shortcut</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>log</extension>\n        <mime-type>text/plain</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>lostxml</extension>\n        <mime-type>application/lost+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>lrf</extension>\n        <mime-type>application/octet-stream</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>lrm</extension>\n        <mime-type>application/vnd.ms-lrm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ltf</extension>\n        <mime-type>application/vnd.frogans.ltf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>lvp</extension>\n        <mime-type>audio/vnd.lucent.voice</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>lwp</extension>\n        <mime-type>application/vnd.lotus-wordpro</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>lzh</extension>\n        <mime-type>application/x-lzh-compressed</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m13</extension>\n        <mime-type>application/x-msmediaview</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m14</extension>\n        <mime-type>application/x-msmediaview</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m1v</extension>\n        <mime-type>video/mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m21</extension>\n        <mime-type>application/mp21</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m2a</extension>\n        <mime-type>audio/mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m2v</extension>\n        <mime-type>video/mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m3a</extension>\n        <mime-type>audio/mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m3u</extension>\n        <mime-type>audio/x-mpegurl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m3u8</extension>\n        <mime-type>application/vnd.apple.mpegurl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m4a</extension>\n        <mime-type>audio/mp4</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m4b</extension>\n        <mime-type>audio/mp4</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m4r</extension>\n        <mime-type>audio/mp4</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m4u</extension>\n        <mime-type>video/vnd.mpegurl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>m4v</extension>\n        <mime-type>video/mp4</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ma</extension>\n        <mime-type>application/mathematica</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mac</extension>\n        <mime-type>image/x-macpaint</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mads</extension>\n        <mime-type>application/mads+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mag</extension>\n        <mime-type>application/vnd.ecowin.chart</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>maker</extension>\n        <mime-type>application/vnd.framemaker</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>man</extension>\n        <mime-type>text/troff</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mar</extension>\n        <mime-type>application/octet-stream</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mathml</extension>\n        <mime-type>application/mathml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mb</extension>\n        <mime-type>application/mathematica</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mbk</extension>\n        <mime-type>application/vnd.mobius.mbk</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mbox</extension>\n        <mime-type>application/mbox</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mc1</extension>\n        <mime-type>application/vnd.medcalcdata</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mcd</extension>\n        <mime-type>application/vnd.mcd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mcurl</extension>\n        <mime-type>text/vnd.curl.mcurl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mdb</extension>\n        <mime-type>application/x-msaccess</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mdi</extension>\n        <mime-type>image/vnd.ms-modi</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>me</extension>\n        <mime-type>text/troff</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mesh</extension>\n        <mime-type>model/mesh</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>meta4</extension>\n        <mime-type>application/metalink4+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>metalink</extension>\n        <mime-type>application/metalink+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mets</extension>\n        <mime-type>application/mets+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mfm</extension>\n        <mime-type>application/vnd.mfmp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mft</extension>\n        <mime-type>application/rpki-manifest</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mgp</extension>\n        <mime-type>application/vnd.osgeo.mapguide.package</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mgz</extension>\n        <mime-type>application/vnd.proteus.magazine</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mid</extension>\n        <mime-type>audio/midi</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>midi</extension>\n        <mime-type>audio/midi</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mie</extension>\n        <mime-type>application/x-mie</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mif</extension>\n        <mime-type>application/x-mif</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mime</extension>\n        <mime-type>message/rfc822</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mj2</extension>\n        <mime-type>video/mj2</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mjp2</extension>\n        <mime-type>video/mj2</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mk3d</extension>\n        <mime-type>video/x-matroska</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mka</extension>\n        <mime-type>audio/x-matroska</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mks</extension>\n        <mime-type>video/x-matroska</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mkv</extension>\n        <mime-type>video/x-matroska</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mlp</extension>\n        <mime-type>application/vnd.dolby.mlp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mmd</extension>\n        <mime-type>application/vnd.chipnuts.karaoke-mmd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mmf</extension>\n        <mime-type>application/vnd.smaf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mmr</extension>\n        <mime-type>image/vnd.fujixerox.edmics-mmr</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mng</extension>\n        <mime-type>video/x-mng</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mny</extension>\n        <mime-type>application/x-msmoney</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mobi</extension>\n        <mime-type>application/x-mobipocket-ebook</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mods</extension>\n        <mime-type>application/mods+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mov</extension>\n        <mime-type>video/quicktime</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>movie</extension>\n        <mime-type>video/x-sgi-movie</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mp1</extension>\n        <mime-type>audio/mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mp2</extension>\n        <mime-type>audio/mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mp21</extension>\n        <mime-type>application/mp21</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mp2a</extension>\n        <mime-type>audio/mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mp3</extension>\n        <mime-type>audio/mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mp4</extension>\n        <mime-type>video/mp4</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mp4a</extension>\n        <mime-type>audio/mp4</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mp4s</extension>\n        <mime-type>application/mp4</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mp4v</extension>\n        <mime-type>video/mp4</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpa</extension>\n        <mime-type>audio/mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpc</extension>\n        <mime-type>application/vnd.mophun.certificate</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpe</extension>\n        <mime-type>video/mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpeg</extension>\n        <mime-type>video/mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpega</extension>\n        <mime-type>audio/x-mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpg</extension>\n        <mime-type>video/mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpg4</extension>\n        <mime-type>video/mp4</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpga</extension>\n        <mime-type>audio/mpeg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpkg</extension>\n        <mime-type>application/vnd.apple.installer+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpm</extension>\n        <mime-type>application/vnd.blueice.multipass</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpn</extension>\n        <mime-type>application/vnd.mophun.application</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpp</extension>\n        <mime-type>application/vnd.ms-project</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpt</extension>\n        <mime-type>application/vnd.ms-project</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpv2</extension>\n        <mime-type>video/mpeg2</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mpy</extension>\n        <mime-type>application/vnd.ibm.minipay</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mqy</extension>\n        <mime-type>application/vnd.mobius.mqy</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mrc</extension>\n        <mime-type>application/marc</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mrcx</extension>\n        <mime-type>application/marcxml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ms</extension>\n        <mime-type>text/troff</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mscml</extension>\n        <mime-type>application/mediaservercontrol+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mseed</extension>\n        <mime-type>application/vnd.fdsn.mseed</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mseq</extension>\n        <mime-type>application/vnd.mseq</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>msf</extension>\n        <mime-type>application/vnd.epson.msf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>msh</extension>\n        <mime-type>model/mesh</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>msi</extension>\n        <mime-type>application/x-msdownload</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>msl</extension>\n        <mime-type>application/vnd.mobius.msl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>msty</extension>\n        <mime-type>application/vnd.muvee.style</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mts</extension>\n        <mime-type>model/vnd.mts</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mus</extension>\n        <mime-type>application/vnd.musician</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>musicxml</extension>\n        <mime-type>application/vnd.recordare.musicxml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mvb</extension>\n        <mime-type>application/x-msmediaview</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mwf</extension>\n        <mime-type>application/vnd.mfer</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mxf</extension>\n        <mime-type>application/mxf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mxl</extension>\n        <mime-type>application/vnd.recordare.musicxml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mxml</extension>\n        <mime-type>application/xv+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mxs</extension>\n        <mime-type>application/vnd.triscape.mxs</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>mxu</extension>\n        <mime-type>video/vnd.mpegurl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>n-gage</extension>\n        <mime-type>application/vnd.nokia.n-gage.symbian.install</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>n3</extension>\n        <mime-type>text/n3</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>nb</extension>\n        <mime-type>application/mathematica</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>nbp</extension>\n        <mime-type>application/vnd.wolfram.player</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>nc</extension>\n        <mime-type>application/x-netcdf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ncx</extension>\n        <mime-type>application/x-dtbncx+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>nfo</extension>\n        <mime-type>text/x-nfo</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ngdat</extension>\n        <mime-type>application/vnd.nokia.n-gage.data</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>nitf</extension>\n        <mime-type>application/vnd.nitf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>nlu</extension>\n        <mime-type>application/vnd.neurolanguage.nlu</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>nml</extension>\n        <mime-type>application/vnd.enliven</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>nnd</extension>\n        <mime-type>application/vnd.noblenet-directory</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>nns</extension>\n        <mime-type>application/vnd.noblenet-sealer</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>nnw</extension>\n        <mime-type>application/vnd.noblenet-web</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>npx</extension>\n        <mime-type>image/vnd.net-fpx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>nsc</extension>\n        <mime-type>application/x-conference</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>nsf</extension>\n        <mime-type>application/vnd.lotus-notes</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ntf</extension>\n        <mime-type>application/vnd.nitf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>nzb</extension>\n        <mime-type>application/x-nzb</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>oa2</extension>\n        <mime-type>application/vnd.fujitsu.oasys2</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>oa3</extension>\n        <mime-type>application/vnd.fujitsu.oasys3</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>oas</extension>\n        <mime-type>application/vnd.fujitsu.oasys</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>obd</extension>\n        <mime-type>application/x-msbinder</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>obj</extension>\n        <mime-type>application/x-tgif</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>oda</extension>\n        <mime-type>application/oda</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- OpenDocument Database -->\n        <extension>odb</extension>\n        <mime-type>application/vnd.oasis.opendocument.database</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- OpenDocument Chart -->\n        <extension>odc</extension>\n        <mime-type>application/vnd.oasis.opendocument.chart</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- OpenDocument Formula -->\n        <extension>odf</extension>\n        <mime-type>application/vnd.oasis.opendocument.formula</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>odft</extension>\n        <mime-type>application/vnd.oasis.opendocument.formula-template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- OpenDocument Drawing -->\n        <extension>odg</extension>\n        <mime-type>application/vnd.oasis.opendocument.graphics</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- OpenDocument Image -->\n        <extension>odi</extension>\n        <mime-type>application/vnd.oasis.opendocument.image</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- OpenDocument Master Document -->\n        <extension>odm</extension>\n        <mime-type>application/vnd.oasis.opendocument.text-master</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- OpenDocument Presentation -->\n        <extension>odp</extension>\n        <mime-type>application/vnd.oasis.opendocument.presentation</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- OpenDocument Spreadsheet -->\n        <extension>ods</extension>\n        <mime-type>application/vnd.oasis.opendocument.spreadsheet</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- OpenDocument Text -->\n        <extension>odt</extension>\n        <mime-type>application/vnd.oasis.opendocument.text</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>oga</extension>\n        <mime-type>audio/ogg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ogg</extension>\n        <mime-type>audio/ogg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ogv</extension>\n        <mime-type>video/ogg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- xiph mime types -->\n        <extension>ogx</extension>\n        <mime-type>application/ogg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>omdoc</extension>\n        <mime-type>application/omdoc+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>onepkg</extension>\n        <mime-type>application/onenote</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>onetmp</extension>\n        <mime-type>application/onenote</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>onetoc</extension>\n        <mime-type>application/onenote</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>onetoc2</extension>\n        <mime-type>application/onenote</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>opf</extension>\n        <mime-type>application/oebps-package+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>opml</extension>\n        <mime-type>text/x-opml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>oprc</extension>\n        <mime-type>application/vnd.palm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>org</extension>\n        <mime-type>application/vnd.lotus-organizer</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>osf</extension>\n        <mime-type>application/vnd.yamaha.openscoreformat</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>osfpvg</extension>\n        <mime-type>application/vnd.yamaha.openscoreformat.osfpvg+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>otc</extension>\n        <mime-type>application/vnd.oasis.opendocument.chart-template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>otf</extension>\n        <mime-type>application/x-font-otf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- OpenDocument Drawing Template -->\n        <extension>otg</extension>\n        <mime-type>application/vnd.oasis.opendocument.graphics-template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- HTML Document Template -->\n        <extension>oth</extension>\n        <mime-type>application/vnd.oasis.opendocument.text-web</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>oti</extension>\n        <mime-type>application/vnd.oasis.opendocument.image-template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- OpenDocument Presentation Template -->\n        <extension>otp</extension>\n        <mime-type>application/vnd.oasis.opendocument.presentation-template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- OpenDocument Spreadsheet Template -->\n        <extension>ots</extension>\n        <mime-type>application/vnd.oasis.opendocument.spreadsheet-template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- OpenDocument Text Template -->\n        <extension>ott</extension>\n        <mime-type>application/vnd.oasis.opendocument.text-template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>oxps</extension>\n        <mime-type>application/oxps</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>oxt</extension>\n        <mime-type>application/vnd.openofficeorg.extension</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>p</extension>\n        <mime-type>text/x-pascal</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>p10</extension>\n        <mime-type>application/pkcs10</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>p12</extension>\n        <mime-type>application/x-pkcs12</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>p7b</extension>\n        <mime-type>application/x-pkcs7-certificates</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>p7c</extension>\n        <mime-type>application/pkcs7-mime</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>p7m</extension>\n        <mime-type>application/pkcs7-mime</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>p7r</extension>\n        <mime-type>application/x-pkcs7-certreqresp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>p7s</extension>\n        <mime-type>application/pkcs7-signature</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>p8</extension>\n        <mime-type>application/pkcs8</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pas</extension>\n        <mime-type>text/x-pascal</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>paw</extension>\n        <mime-type>application/vnd.pawaafile</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pbd</extension>\n        <mime-type>application/vnd.powerbuilder6</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pbm</extension>\n        <mime-type>image/x-portable-bitmap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pcap</extension>\n        <mime-type>application/vnd.tcpdump.pcap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pcf</extension>\n        <mime-type>application/x-font-pcf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pcl</extension>\n        <mime-type>application/vnd.hp-pcl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pclxl</extension>\n        <mime-type>application/vnd.hp-pclxl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pct</extension>\n        <mime-type>image/pict</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pcurl</extension>\n        <mime-type>application/vnd.curl.pcurl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pcx</extension>\n        <mime-type>image/x-pcx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pdb</extension>\n        <mime-type>application/vnd.palm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pdf</extension>\n        <mime-type>application/pdf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pfa</extension>\n        <mime-type>application/x-font-type1</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pfb</extension>\n        <mime-type>application/x-font-type1</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pfm</extension>\n        <mime-type>application/x-font-type1</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pfr</extension>\n        <mime-type>application/font-tdpfr</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pfx</extension>\n        <mime-type>application/x-pkcs12</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pgm</extension>\n        <mime-type>image/x-portable-graymap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pgn</extension>\n        <mime-type>application/x-chess-pgn</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pgp</extension>\n        <mime-type>application/pgp-encrypted</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pic</extension>\n        <mime-type>image/pict</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pict</extension>\n        <mime-type>image/pict</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pkg</extension>\n        <mime-type>application/octet-stream</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pki</extension>\n        <mime-type>application/pkixcmp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pkipath</extension>\n        <mime-type>application/pkix-pkipath</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>plb</extension>\n        <mime-type>application/vnd.3gpp.pic-bw-large</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>plc</extension>\n        <mime-type>application/vnd.mobius.plc</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>plf</extension>\n        <mime-type>application/vnd.pocketlearn</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pls</extension>\n        <mime-type>audio/x-scpls</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pml</extension>\n        <mime-type>application/vnd.ctc-posml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>png</extension>\n        <mime-type>image/png</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pnm</extension>\n        <mime-type>image/x-portable-anymap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pnt</extension>\n        <mime-type>image/x-macpaint</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>portpkg</extension>\n        <mime-type>application/vnd.macports.portpkg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pot</extension>\n        <mime-type>application/vnd.ms-powerpoint</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>potm</extension>\n        <mime-type>application/vnd.ms-powerpoint.template.macroenabled.12</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>potx</extension>\n        <mime-type>application/vnd.openxmlformats-officedocument.presentationml.template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ppam</extension>\n        <mime-type>application/vnd.ms-powerpoint.addin.macroenabled.12</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ppd</extension>\n        <mime-type>application/vnd.cups-ppd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ppm</extension>\n        <mime-type>image/x-portable-pixmap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pps</extension>\n        <mime-type>application/vnd.ms-powerpoint</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ppsm</extension>\n        <mime-type>application/vnd.ms-powerpoint.slideshow.macroenabled.12</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ppsx</extension>\n        <mime-type>application/vnd.openxmlformats-officedocument.presentationml.slideshow</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ppt</extension>\n        <mime-type>application/vnd.ms-powerpoint</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pptm</extension>\n        <mime-type>application/vnd.ms-powerpoint.presentation.macroenabled.12</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pptx</extension>\n        <mime-type>application/vnd.openxmlformats-officedocument.presentationml.presentation</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pqa</extension>\n        <mime-type>application/vnd.palm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>prc</extension>\n        <mime-type>application/x-mobipocket-ebook</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pre</extension>\n        <mime-type>application/vnd.lotus-freelance</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>prf</extension>\n        <mime-type>application/pics-rules</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ps</extension>\n        <mime-type>application/postscript</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>psb</extension>\n        <mime-type>application/vnd.3gpp.pic-bw-small</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>psd</extension>\n        <mime-type>image/vnd.adobe.photoshop</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>psf</extension>\n        <mime-type>application/x-font-linux-psf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pskcxml</extension>\n        <mime-type>application/pskc+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ptid</extension>\n        <mime-type>application/vnd.pvi.ptid1</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pub</extension>\n        <mime-type>application/x-mspublisher</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pvb</extension>\n        <mime-type>application/vnd.3gpp.pic-bw-var</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pwn</extension>\n        <mime-type>application/vnd.3m.post-it-notes</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pya</extension>\n        <mime-type>audio/vnd.ms-playready.media.pya</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>pyv</extension>\n        <mime-type>video/vnd.ms-playready.media.pyv</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>qam</extension>\n        <mime-type>application/vnd.epson.quickanime</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>qbo</extension>\n        <mime-type>application/vnd.intu.qbo</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>qfx</extension>\n        <mime-type>application/vnd.intu.qfx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>qps</extension>\n        <mime-type>application/vnd.publishare-delta-tree</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>qt</extension>\n        <mime-type>video/quicktime</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>qti</extension>\n        <mime-type>image/x-quicktime</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>qtif</extension>\n        <mime-type>image/x-quicktime</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>qwd</extension>\n        <mime-type>application/vnd.quark.quarkxpress</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>qwt</extension>\n        <mime-type>application/vnd.quark.quarkxpress</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>qxb</extension>\n        <mime-type>application/vnd.quark.quarkxpress</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>qxd</extension>\n        <mime-type>application/vnd.quark.quarkxpress</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>qxl</extension>\n        <mime-type>application/vnd.quark.quarkxpress</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>qxt</extension>\n        <mime-type>application/vnd.quark.quarkxpress</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ra</extension>\n        <mime-type>audio/x-pn-realaudio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ram</extension>\n        <mime-type>audio/x-pn-realaudio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rar</extension>\n        <mime-type>application/x-rar-compressed</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ras</extension>\n        <mime-type>image/x-cmu-raster</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rcprofile</extension>\n        <mime-type>application/vnd.ipunplugged.rcprofile</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rdf</extension>\n        <mime-type>application/rdf+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rdz</extension>\n        <mime-type>application/vnd.data-vision.rdz</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rep</extension>\n        <mime-type>application/vnd.businessobjects</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>res</extension>\n        <mime-type>application/x-dtbresource+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rgb</extension>\n        <mime-type>image/x-rgb</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rif</extension>\n        <mime-type>application/reginfo+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rip</extension>\n        <mime-type>audio/vnd.rip</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ris</extension>\n        <mime-type>application/x-research-info-systems</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rl</extension>\n        <mime-type>application/resource-lists+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rlc</extension>\n        <mime-type>image/vnd.fujixerox.edmics-rlc</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rld</extension>\n        <mime-type>application/resource-lists-diff+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rm</extension>\n        <mime-type>application/vnd.rn-realmedia</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rmi</extension>\n        <mime-type>audio/midi</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rmp</extension>\n        <mime-type>audio/x-pn-realaudio-plugin</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rms</extension>\n        <mime-type>application/vnd.jcp.javame.midlet-rms</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rmvb</extension>\n        <mime-type>application/vnd.rn-realmedia-vbr</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rnc</extension>\n        <mime-type>application/relax-ng-compact-syntax</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>roa</extension>\n        <mime-type>application/rpki-roa</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>roff</extension>\n        <mime-type>text/troff</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rp9</extension>\n        <mime-type>application/vnd.cloanto.rp9</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rpss</extension>\n        <mime-type>application/vnd.nokia.radio-presets</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rpst</extension>\n        <mime-type>application/vnd.nokia.radio-preset</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rq</extension>\n        <mime-type>application/sparql-query</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rs</extension>\n        <mime-type>application/rls-services+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rsd</extension>\n        <mime-type>application/rsd+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rss</extension>\n        <mime-type>application/rss+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rtf</extension>\n        <mime-type>application/rtf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>rtx</extension>\n        <mime-type>text/richtext</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>s</extension>\n        <mime-type>text/x-asm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>s3m</extension>\n        <mime-type>audio/s3m</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>saf</extension>\n        <mime-type>application/vnd.yamaha.smaf-audio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sbml</extension>\n        <mime-type>application/sbml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sc</extension>\n        <mime-type>application/vnd.ibm.secure-container</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>scd</extension>\n        <mime-type>application/x-msschedule</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>scm</extension>\n        <mime-type>application/vnd.lotus-screencam</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>scq</extension>\n        <mime-type>application/scvp-cv-request</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>scs</extension>\n        <mime-type>application/scvp-cv-response</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>scurl</extension>\n        <mime-type>text/vnd.curl.scurl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sda</extension>\n        <mime-type>application/vnd.stardivision.draw</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sdc</extension>\n        <mime-type>application/vnd.stardivision.calc</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sdd</extension>\n        <mime-type>application/vnd.stardivision.impress</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sdkd</extension>\n        <mime-type>application/vnd.solent.sdkm+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sdkm</extension>\n        <mime-type>application/vnd.solent.sdkm+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sdp</extension>\n        <mime-type>application/sdp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sdw</extension>\n        <mime-type>application/vnd.stardivision.writer</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>see</extension>\n        <mime-type>application/vnd.seemail</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>seed</extension>\n        <mime-type>application/vnd.fdsn.seed</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sema</extension>\n        <mime-type>application/vnd.sema</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>semd</extension>\n        <mime-type>application/vnd.semd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>semf</extension>\n        <mime-type>application/vnd.semf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ser</extension>\n        <mime-type>application/java-serialized-object</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>setpay</extension>\n        <mime-type>application/set-payment-initiation</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>setreg</extension>\n        <mime-type>application/set-registration-initiation</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sfd-hdstx</extension>\n        <mime-type>application/vnd.hydrostatix.sof-data</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sfs</extension>\n        <mime-type>application/vnd.spotfire.sfs</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sfv</extension>\n        <mime-type>text/x-sfv</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sgi</extension>\n        <mime-type>image/sgi</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sgl</extension>\n        <mime-type>application/vnd.stardivision.writer-global</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sgm</extension>\n        <mime-type>text/sgml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sgml</extension>\n        <mime-type>text/sgml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sh</extension>\n        <mime-type>application/x-sh</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>shar</extension>\n        <mime-type>application/x-shar</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>shf</extension>\n        <mime-type>application/shf+xml</mime-type>\n    </mime-mapping>\n    <!--\n    <mime-mapping>\n        <extension>shtml</extension>\n        <mime-type>text/x-server-parsed-html</mime-type>\n    </mime-mapping>\n    -->\n    <mime-mapping>\n        <extension>sid</extension>\n        <mime-type>image/x-mrsid-image</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sig</extension>\n        <mime-type>application/pgp-signature</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sil</extension>\n        <mime-type>audio/silk</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>silo</extension>\n        <mime-type>model/mesh</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sis</extension>\n        <mime-type>application/vnd.symbian.install</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sisx</extension>\n        <mime-type>application/vnd.symbian.install</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sit</extension>\n        <mime-type>application/x-stuffit</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sitx</extension>\n        <mime-type>application/x-stuffitx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>skd</extension>\n        <mime-type>application/vnd.koan</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>skm</extension>\n        <mime-type>application/vnd.koan</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>skp</extension>\n        <mime-type>application/vnd.koan</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>skt</extension>\n        <mime-type>application/vnd.koan</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sldm</extension>\n        <mime-type>application/vnd.ms-powerpoint.slide.macroenabled.12</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sldx</extension>\n        <mime-type>application/vnd.openxmlformats-officedocument.presentationml.slide</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>slt</extension>\n        <mime-type>application/vnd.epson.salt</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sm</extension>\n        <mime-type>application/vnd.stepmania.stepchart</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>smf</extension>\n        <mime-type>application/vnd.stardivision.math</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>smi</extension>\n        <mime-type>application/smil+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>smil</extension>\n        <mime-type>application/smil+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>smv</extension>\n        <mime-type>video/x-smv</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>smzip</extension>\n        <mime-type>application/vnd.stepmania.package</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>snd</extension>\n        <mime-type>audio/basic</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>snf</extension>\n        <mime-type>application/x-font-snf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>so</extension>\n        <mime-type>application/octet-stream</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>spc</extension>\n        <mime-type>application/x-pkcs7-certificates</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>spf</extension>\n        <mime-type>application/vnd.yamaha.smaf-phrase</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>spl</extension>\n        <mime-type>application/x-futuresplash</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>spot</extension>\n        <mime-type>text/vnd.in3d.spot</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>spp</extension>\n        <mime-type>application/scvp-vp-response</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>spq</extension>\n        <mime-type>application/scvp-vp-request</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>spx</extension>\n        <mime-type>audio/ogg</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sql</extension>\n        <mime-type>application/x-sql</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>src</extension>\n        <mime-type>application/x-wais-source</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>srt</extension>\n        <mime-type>application/x-subrip</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sru</extension>\n        <mime-type>application/sru+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>srx</extension>\n        <mime-type>application/sparql-results+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ssdl</extension>\n        <mime-type>application/ssdl+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sse</extension>\n        <mime-type>application/vnd.kodak-descriptor</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ssf</extension>\n        <mime-type>application/vnd.epson.ssf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ssml</extension>\n        <mime-type>application/ssml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>st</extension>\n        <mime-type>application/vnd.sailingtracker.track</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>stc</extension>\n        <mime-type>application/vnd.sun.xml.calc.template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>std</extension>\n        <mime-type>application/vnd.sun.xml.draw.template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>stf</extension>\n        <mime-type>application/vnd.wt.stf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sti</extension>\n        <mime-type>application/vnd.sun.xml.impress.template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>stk</extension>\n        <mime-type>application/hyperstudio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>stl</extension>\n        <mime-type>application/vnd.ms-pki.stl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>str</extension>\n        <mime-type>application/vnd.pg.format</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>stw</extension>\n        <mime-type>application/vnd.sun.xml.writer.template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sub</extension>\n        <mime-type>text/vnd.dvb.subtitle</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sus</extension>\n        <mime-type>application/vnd.sus-calendar</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>susp</extension>\n        <mime-type>application/vnd.sus-calendar</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sv4cpio</extension>\n        <mime-type>application/x-sv4cpio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sv4crc</extension>\n        <mime-type>application/x-sv4crc</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>svc</extension>\n        <mime-type>application/vnd.dvb.service</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>svd</extension>\n        <mime-type>application/vnd.svd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>svg</extension>\n        <mime-type>image/svg+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>svgz</extension>\n        <mime-type>image/svg+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>swa</extension>\n        <mime-type>application/x-director</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>swf</extension>\n        <mime-type>application/x-shockwave-flash</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>swi</extension>\n        <mime-type>application/vnd.aristanetworks.swi</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sxc</extension>\n        <mime-type>application/vnd.sun.xml.calc</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sxd</extension>\n        <mime-type>application/vnd.sun.xml.draw</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sxg</extension>\n        <mime-type>application/vnd.sun.xml.writer.global</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sxi</extension>\n        <mime-type>application/vnd.sun.xml.impress</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sxm</extension>\n        <mime-type>application/vnd.sun.xml.math</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>sxw</extension>\n        <mime-type>application/vnd.sun.xml.writer</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>t</extension>\n        <mime-type>text/troff</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>t3</extension>\n        <mime-type>application/x-t3vm-image</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>taglet</extension>\n        <mime-type>application/vnd.mynfc</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tao</extension>\n        <mime-type>application/vnd.tao.intent-module-archive</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tar</extension>\n        <mime-type>application/x-tar</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tcap</extension>\n        <mime-type>application/vnd.3gpp2.tcap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tcl</extension>\n        <mime-type>application/x-tcl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>teacher</extension>\n        <mime-type>application/vnd.smart.teacher</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tei</extension>\n        <mime-type>application/tei+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>teicorpus</extension>\n        <mime-type>application/tei+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tex</extension>\n        <mime-type>application/x-tex</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>texi</extension>\n        <mime-type>application/x-texinfo</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>texinfo</extension>\n        <mime-type>application/x-texinfo</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>text</extension>\n        <mime-type>text/plain</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tfi</extension>\n        <mime-type>application/thraud+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tfm</extension>\n        <mime-type>application/x-tex-tfm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tga</extension>\n        <mime-type>image/x-tga</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>thmx</extension>\n        <mime-type>application/vnd.ms-officetheme</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tif</extension>\n        <mime-type>image/tiff</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tiff</extension>\n        <mime-type>image/tiff</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tmo</extension>\n        <mime-type>application/vnd.tmobile-livetv</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>torrent</extension>\n        <mime-type>application/x-bittorrent</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tpl</extension>\n        <mime-type>application/vnd.groove-tool-template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tpt</extension>\n        <mime-type>application/vnd.trid.tpt</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tr</extension>\n        <mime-type>text/troff</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tra</extension>\n        <mime-type>application/vnd.trueapp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>trm</extension>\n        <mime-type>application/x-msterminal</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tsd</extension>\n        <mime-type>application/timestamped-data</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>tsv</extension>\n        <mime-type>text/tab-separated-values</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ttc</extension>\n        <mime-type>application/x-font-ttf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ttf</extension>\n        <mime-type>application/x-font-ttf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ttl</extension>\n        <mime-type>text/turtle</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>twd</extension>\n        <mime-type>application/vnd.simtech-mindmapper</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>twds</extension>\n        <mime-type>application/vnd.simtech-mindmapper</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>txd</extension>\n        <mime-type>application/vnd.genomatix.tuxedo</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>txf</extension>\n        <mime-type>application/vnd.mobius.txf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>txt</extension>\n        <mime-type>text/plain</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>u32</extension>\n        <mime-type>application/x-authorware-bin</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>udeb</extension>\n        <mime-type>application/x-debian-package</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ufd</extension>\n        <mime-type>application/vnd.ufdl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ufdl</extension>\n        <mime-type>application/vnd.ufdl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ulw</extension>\n        <mime-type>audio/basic</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ulx</extension>\n        <mime-type>application/x-glulx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>umj</extension>\n        <mime-type>application/vnd.umajin</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>unityweb</extension>\n        <mime-type>application/vnd.unity</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uoml</extension>\n        <mime-type>application/vnd.uoml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uri</extension>\n        <mime-type>text/uri-list</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uris</extension>\n        <mime-type>text/uri-list</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>urls</extension>\n        <mime-type>text/uri-list</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>ustar</extension>\n        <mime-type>application/x-ustar</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>utz</extension>\n        <mime-type>application/vnd.uiq.theme</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uu</extension>\n        <mime-type>text/x-uuencode</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uva</extension>\n        <mime-type>audio/vnd.dece.audio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvd</extension>\n        <mime-type>application/vnd.dece.data</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvf</extension>\n        <mime-type>application/vnd.dece.data</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvg</extension>\n        <mime-type>image/vnd.dece.graphic</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvh</extension>\n        <mime-type>video/vnd.dece.hd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvi</extension>\n        <mime-type>image/vnd.dece.graphic</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvm</extension>\n        <mime-type>video/vnd.dece.mobile</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvp</extension>\n        <mime-type>video/vnd.dece.pd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvs</extension>\n        <mime-type>video/vnd.dece.sd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvt</extension>\n        <mime-type>application/vnd.dece.ttml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvu</extension>\n        <mime-type>video/vnd.uvvu.mp4</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvv</extension>\n        <mime-type>video/vnd.dece.video</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvva</extension>\n        <mime-type>audio/vnd.dece.audio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvvd</extension>\n        <mime-type>application/vnd.dece.data</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvvf</extension>\n        <mime-type>application/vnd.dece.data</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvvg</extension>\n        <mime-type>image/vnd.dece.graphic</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvvh</extension>\n        <mime-type>video/vnd.dece.hd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvvi</extension>\n        <mime-type>image/vnd.dece.graphic</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvvm</extension>\n        <mime-type>video/vnd.dece.mobile</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvvp</extension>\n        <mime-type>video/vnd.dece.pd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvvs</extension>\n        <mime-type>video/vnd.dece.sd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvvt</extension>\n        <mime-type>application/vnd.dece.ttml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvvu</extension>\n        <mime-type>video/vnd.uvvu.mp4</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvvv</extension>\n        <mime-type>video/vnd.dece.video</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvvx</extension>\n        <mime-type>application/vnd.dece.unspecified</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvvz</extension>\n        <mime-type>application/vnd.dece.zip</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvx</extension>\n        <mime-type>application/vnd.dece.unspecified</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>uvz</extension>\n        <mime-type>application/vnd.dece.zip</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vcard</extension>\n        <mime-type>text/vcard</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vcd</extension>\n        <mime-type>application/x-cdlink</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vcf</extension>\n        <mime-type>text/x-vcard</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vcg</extension>\n        <mime-type>application/vnd.groove-vcard</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vcs</extension>\n        <mime-type>text/x-vcalendar</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vcx</extension>\n        <mime-type>application/vnd.vcx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vis</extension>\n        <mime-type>application/vnd.visionary</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>viv</extension>\n        <mime-type>video/vnd.vivo</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vob</extension>\n        <mime-type>video/x-ms-vob</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vor</extension>\n        <mime-type>application/vnd.stardivision.writer</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vox</extension>\n        <mime-type>application/x-authorware-bin</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vrml</extension>\n        <mime-type>model/vrml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vsd</extension>\n        <mime-type>application/vnd.visio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vsf</extension>\n        <mime-type>application/vnd.vsf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vss</extension>\n        <mime-type>application/vnd.visio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vst</extension>\n        <mime-type>application/vnd.visio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vsw</extension>\n        <mime-type>application/vnd.visio</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vtu</extension>\n        <mime-type>model/vnd.vtu</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>vxml</extension>\n        <mime-type>application/voicexml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>w3d</extension>\n        <mime-type>application/x-director</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wad</extension>\n        <mime-type>application/x-doom</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wav</extension>\n        <mime-type>audio/x-wav</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wax</extension>\n        <mime-type>audio/x-ms-wax</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- Wireless Bitmap -->\n        <extension>wbmp</extension>\n        <mime-type>image/vnd.wap.wbmp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wbs</extension>\n        <mime-type>application/vnd.criticaltools.wbs+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wbxml</extension>\n        <mime-type>application/vnd.wap.wbxml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wcm</extension>\n        <mime-type>application/vnd.ms-works</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wdb</extension>\n        <mime-type>application/vnd.ms-works</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wdp</extension>\n        <mime-type>image/vnd.ms-photo</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>weba</extension>\n        <mime-type>audio/webm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>webm</extension>\n        <mime-type>video/webm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>webp</extension>\n        <mime-type>image/webp</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wg</extension>\n        <mime-type>application/vnd.pmi.widget</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wgt</extension>\n        <mime-type>application/widget</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wks</extension>\n        <mime-type>application/vnd.ms-works</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wm</extension>\n        <mime-type>video/x-ms-wm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wma</extension>\n        <mime-type>audio/x-ms-wma</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wmd</extension>\n        <mime-type>application/x-ms-wmd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wmf</extension>\n        <mime-type>application/x-msmetafile</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- WML Source -->\n        <extension>wml</extension>\n        <mime-type>text/vnd.wap.wml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- Compiled WML -->\n        <extension>wmlc</extension>\n        <mime-type>application/vnd.wap.wmlc</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- WML Script Source -->\n        <extension>wmls</extension>\n        <mime-type>text/vnd.wap.wmlscript</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <!-- Compiled WML Script -->\n        <extension>wmlsc</extension>\n        <mime-type>application/vnd.wap.wmlscriptc</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wmv</extension>\n        <mime-type>video/x-ms-wmv</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wmx</extension>\n        <mime-type>video/x-ms-wmx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wmz</extension>\n        <mime-type>application/x-msmetafile</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>woff</extension>\n        <mime-type>application/x-font-woff</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wpd</extension>\n        <mime-type>application/vnd.wordperfect</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wpl</extension>\n        <mime-type>application/vnd.ms-wpl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wps</extension>\n        <mime-type>application/vnd.ms-works</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wqd</extension>\n        <mime-type>application/vnd.wqd</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wri</extension>\n        <mime-type>application/x-mswrite</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wrl</extension>\n        <mime-type>model/vrml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wsdl</extension>\n        <mime-type>application/wsdl+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wspolicy</extension>\n        <mime-type>application/wspolicy+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wtb</extension>\n        <mime-type>application/vnd.webturbo</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>wvx</extension>\n        <mime-type>video/x-ms-wvx</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>x32</extension>\n        <mime-type>application/x-authorware-bin</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>x3d</extension>\n        <mime-type>model/x3d+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>x3db</extension>\n        <mime-type>model/x3d+binary</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>x3dbz</extension>\n        <mime-type>model/x3d+binary</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>x3dv</extension>\n        <mime-type>model/x3d+vrml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>x3dvz</extension>\n        <mime-type>model/x3d+vrml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>x3dz</extension>\n        <mime-type>model/x3d+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xaml</extension>\n        <mime-type>application/xaml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xap</extension>\n        <mime-type>application/x-silverlight-app</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xar</extension>\n        <mime-type>application/vnd.xara</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xbap</extension>\n        <mime-type>application/x-ms-xbap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xbd</extension>\n        <mime-type>application/vnd.fujixerox.docuworks.binder</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xbm</extension>\n        <mime-type>image/x-xbitmap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xdf</extension>\n        <mime-type>application/xcap-diff+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xdm</extension>\n        <mime-type>application/vnd.syncml.dm+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xdp</extension>\n        <mime-type>application/vnd.adobe.xdp+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xdssc</extension>\n        <mime-type>application/dssc+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xdw</extension>\n        <mime-type>application/vnd.fujixerox.docuworks</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xenc</extension>\n        <mime-type>application/xenc+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xer</extension>\n        <mime-type>application/patch-ops-error+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xfdf</extension>\n        <mime-type>application/vnd.adobe.xfdf</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xfdl</extension>\n        <mime-type>application/vnd.xfdl</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xht</extension>\n        <mime-type>application/xhtml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xhtml</extension>\n        <mime-type>application/xhtml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xhvml</extension>\n        <mime-type>application/xv+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xif</extension>\n        <mime-type>image/vnd.xiff</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xla</extension>\n        <mime-type>application/vnd.ms-excel</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xlam</extension>\n        <mime-type>application/vnd.ms-excel.addin.macroenabled.12</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xlc</extension>\n        <mime-type>application/vnd.ms-excel</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xlf</extension>\n        <mime-type>application/x-xliff+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xlm</extension>\n        <mime-type>application/vnd.ms-excel</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xls</extension>\n        <mime-type>application/vnd.ms-excel</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xlsb</extension>\n        <mime-type>application/vnd.ms-excel.sheet.binary.macroenabled.12</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xlsm</extension>\n        <mime-type>application/vnd.ms-excel.sheet.macroenabled.12</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xlsx</extension>\n        <mime-type>application/vnd.openxmlformats-officedocument.spreadsheetml.sheet</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xlt</extension>\n        <mime-type>application/vnd.ms-excel</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xltm</extension>\n        <mime-type>application/vnd.ms-excel.template.macroenabled.12</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xltx</extension>\n        <mime-type>application/vnd.openxmlformats-officedocument.spreadsheetml.template</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xlw</extension>\n        <mime-type>application/vnd.ms-excel</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xm</extension>\n        <mime-type>audio/xm</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xml</extension>\n        <mime-type>application/xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xo</extension>\n        <mime-type>application/vnd.olpc-sugar</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xop</extension>\n        <mime-type>application/xop+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xpi</extension>\n        <mime-type>application/x-xpinstall</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xpl</extension>\n        <mime-type>application/xproc+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xpm</extension>\n        <mime-type>image/x-xpixmap</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xpr</extension>\n        <mime-type>application/vnd.is-xpr</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xps</extension>\n        <mime-type>application/vnd.ms-xpsdocument</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xpw</extension>\n        <mime-type>application/vnd.intercon.formnet</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xpx</extension>\n        <mime-type>application/vnd.intercon.formnet</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xsl</extension>\n        <mime-type>application/xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xslt</extension>\n        <mime-type>application/xslt+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xsm</extension>\n        <mime-type>application/vnd.syncml+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xspf</extension>\n        <mime-type>application/xspf+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xul</extension>\n        <mime-type>application/vnd.mozilla.xul+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xvm</extension>\n        <mime-type>application/xv+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xvml</extension>\n        <mime-type>application/xv+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xwd</extension>\n        <mime-type>image/x-xwindowdump</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xyz</extension>\n        <mime-type>chemical/x-xyz</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>xz</extension>\n        <mime-type>application/x-xz</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>yang</extension>\n        <mime-type>application/yang</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>yin</extension>\n        <mime-type>application/yin+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>z</extension>\n        <mime-type>application/x-compress</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>Z</extension>\n        <mime-type>application/x-compress</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>z1</extension>\n        <mime-type>application/x-zmachine</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>z2</extension>\n        <mime-type>application/x-zmachine</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>z3</extension>\n        <mime-type>application/x-zmachine</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>z4</extension>\n        <mime-type>application/x-zmachine</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>z5</extension>\n        <mime-type>application/x-zmachine</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>z6</extension>\n        <mime-type>application/x-zmachine</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>z7</extension>\n        <mime-type>application/x-zmachine</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>z8</extension>\n        <mime-type>application/x-zmachine</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>zaz</extension>\n        <mime-type>application/vnd.zzazz.deck+xml</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>zip</extension>\n        <mime-type>application/zip</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>zir</extension>\n        <mime-type>application/vnd.zul</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>zirz</extension>\n        <mime-type>application/vnd.zul</mime-type>\n    </mime-mapping>\n    <mime-mapping>\n        <extension>zmm</extension>\n        <mime-type>application/vnd.handheld-entertainment+xml</mime-type>\n    </mime-mapping>\n\n  <!-- ==================== Default Welcome File List ===================== -->\n  <!-- When a request URI refers to a directory, the default servlet looks  -->\n  <!-- for a \"welcome file\" within that directory and, if present, to the   -->\n  <!-- corresponding resource URI for display.                              -->\n  <!-- If no welcome files are present, the default servlet either serves a -->\n  <!-- directory listing (see default servlet configuration on how to       -->\n  <!-- customize) or returns a 404 status, depending on the value of the    -->\n  <!-- listings setting.                                                    -->\n  <!--                                                                      -->\n  <!-- If you define welcome files in your own application's web.xml        -->\n  <!-- deployment descriptor, that list *replaces* the list configured      -->\n  <!-- here, so be sure to include any of the default values that you wish  -->\n  <!-- to use within your application.                                       -->\n\n    <welcome-file-list>\n        <welcome-file>index.html</welcome-file>\n        <welcome-file>index.htm</welcome-file>\n        <welcome-file>index.jsp</welcome-file>\n    </welcome-file-list>\n\n</web-app>\n"
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/temp/safeToDelete.tmp",
    "content": ""
  },
  {
    "path": "contrib/wikipedia/contrib/apache-tomcat/webapps/ROOT/index.jsp",
    "content": "<%@ page language=\"java\" contentType=\"text/html; charset=UTF-8\" pageEncoding=\"UTF-8\"%>\n<%\nresponse.sendRedirect(\"scalaris-wiki\");\n%>\n"
  },
  {
    "path": "contrib/wikipedia/contrib/update-tomcat.txt",
    "content": "1) download & extract the new tomcat (binary distribution)\n2) remove <new-tomcat-dir>/webapps/*\n3) move to new tomcat-dir:\n<old-tomcat-dir>/webapps/*\n<old-tomcat-dir>/.gitignore\n<old-tomcat-dir>/logs/.gitignore\n<old-tomcat-dir>/work/.gitignore\n4) in <new-tomcat-dir>/conf/server.xml:\n4a) search for the active \"Connector\" tag and add the following to its properties:\nURIEncoding=\"UTF-8\"\n-> sets default parameter encoding to UTF-8 (required by the wiki app)\nmaxHttpHeaderSize=\"262144\"\n-> support redirect URLs with a lot of logged involved and failed keys\n5) replace <old-tomcat-dir> with <new-tomcat-dir>\n"
  },
  {
    "path": "contrib/wikipedia/epl-v10.htm",
    "content": "<?xml version=\"1.0\" encoding=\"ISO-8859-1\" ?>\n<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n\n<head>\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=ISO-8859-1\" />\n<title>Eclipse Public License - Version 1.0</title>\n<style type=\"text/css\">\n  body {\n    size: 8.5in 11.0in;\n    margin: 0.25in 0.5in 0.25in 0.5in;\n    tab-interval: 0.5in;\n    }\n  p {  \t\n    margin-left: auto;\n    margin-top:  0.5em;\n    margin-bottom: 0.5em;\n    }\n  p.list {\n  \tmargin-left: 0.5in;\n    margin-top:  0.05em;\n    margin-bottom: 0.05em;\n    }\n  </style>\n\n</head>\n\n<body lang=\"EN-US\">\n\n<h2>Eclipse Public License - v 1.0</h2>\n\n<p>THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE\nPUBLIC LICENSE (&quot;AGREEMENT&quot;). ANY USE, REPRODUCTION OR\nDISTRIBUTION OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS\nAGREEMENT.</p>\n\n<p><b>1. DEFINITIONS</b></p>\n\n<p>&quot;Contribution&quot; means:</p>\n\n<p class=\"list\">a) in the case of the initial Contributor, the initial\ncode and documentation distributed under this Agreement, and</p>\n<p class=\"list\">b) in the case of each subsequent Contributor:</p>\n<p class=\"list\">i) changes to the Program, and</p>\n<p class=\"list\">ii) additions to the Program;</p>\n<p class=\"list\">where such changes and/or additions to the Program\noriginate from and are distributed by that particular Contributor. A\nContribution 'originates' from a Contributor if it was added to the\nProgram by such Contributor itself or anyone acting on such\nContributor's behalf. Contributions do not include additions to the\nProgram which: (i) are separate modules of software distributed in\nconjunction with the Program under their own license agreement, and (ii)\nare not derivative works of the Program.</p>\n\n<p>&quot;Contributor&quot; means any person or entity that distributes\nthe Program.</p>\n\n<p>&quot;Licensed Patents&quot; mean patent claims licensable by a\nContributor which are necessarily infringed by the use or sale of its\nContribution alone or when combined with the Program.</p>\n\n<p>&quot;Program&quot; means the Contributions distributed in accordance\nwith this Agreement.</p>\n\n<p>&quot;Recipient&quot; means anyone who receives the Program under\nthis Agreement, including all Contributors.</p>\n\n<p><b>2. GRANT OF RIGHTS</b></p>\n\n<p class=\"list\">a) Subject to the terms of this Agreement, each\nContributor hereby grants Recipient a non-exclusive, worldwide,\nroyalty-free copyright license to reproduce, prepare derivative works\nof, publicly display, publicly perform, distribute and sublicense the\nContribution of such Contributor, if any, and such derivative works, in\nsource code and object code form.</p>\n\n<p class=\"list\">b) Subject to the terms of this Agreement, each\nContributor hereby grants Recipient a non-exclusive, worldwide,\nroyalty-free patent license under Licensed Patents to make, use, sell,\noffer to sell, import and otherwise transfer the Contribution of such\nContributor, if any, in source code and object code form. This patent\nlicense shall apply to the combination of the Contribution and the\nProgram if, at the time the Contribution is added by the Contributor,\nsuch addition of the Contribution causes such combination to be covered\nby the Licensed Patents. The patent license shall not apply to any other\ncombinations which include the Contribution. No hardware per se is\nlicensed hereunder.</p>\n\n<p class=\"list\">c) Recipient understands that although each Contributor\ngrants the licenses to its Contributions set forth herein, no assurances\nare provided by any Contributor that the Program does not infringe the\npatent or other intellectual property rights of any other entity. Each\nContributor disclaims any liability to Recipient for claims brought by\nany other entity based on infringement of intellectual property rights\nor otherwise. As a condition to exercising the rights and licenses\ngranted hereunder, each Recipient hereby assumes sole responsibility to\nsecure any other intellectual property rights needed, if any. For\nexample, if a third party patent license is required to allow Recipient\nto distribute the Program, it is Recipient's responsibility to acquire\nthat license before distributing the Program.</p>\n\n<p class=\"list\">d) Each Contributor represents that to its knowledge it\nhas sufficient copyright rights in its Contribution, if any, to grant\nthe copyright license set forth in this Agreement.</p>\n\n<p><b>3. REQUIREMENTS</b></p>\n\n<p>A Contributor may choose to distribute the Program in object code\nform under its own license agreement, provided that:</p>\n\n<p class=\"list\">a) it complies with the terms and conditions of this\nAgreement; and</p>\n\n<p class=\"list\">b) its license agreement:</p>\n\n<p class=\"list\">i) effectively disclaims on behalf of all Contributors\nall warranties and conditions, express and implied, including warranties\nor conditions of title and non-infringement, and implied warranties or\nconditions of merchantability and fitness for a particular purpose;</p>\n\n<p class=\"list\">ii) effectively excludes on behalf of all Contributors\nall liability for damages, including direct, indirect, special,\nincidental and consequential damages, such as lost profits;</p>\n\n<p class=\"list\">iii) states that any provisions which differ from this\nAgreement are offered by that Contributor alone and not by any other\nparty; and</p>\n\n<p class=\"list\">iv) states that source code for the Program is available\nfrom such Contributor, and informs licensees how to obtain it in a\nreasonable manner on or through a medium customarily used for software\nexchange.</p>\n\n<p>When the Program is made available in source code form:</p>\n\n<p class=\"list\">a) it must be made available under this Agreement; and</p>\n\n<p class=\"list\">b) a copy of this Agreement must be included with each\ncopy of the Program.</p>\n\n<p>Contributors may not remove or alter any copyright notices contained\nwithin the Program.</p>\n\n<p>Each Contributor must identify itself as the originator of its\nContribution, if any, in a manner that reasonably allows subsequent\nRecipients to identify the originator of the Contribution.</p>\n\n<p><b>4. COMMERCIAL DISTRIBUTION</b></p>\n\n<p>Commercial distributors of software may accept certain\nresponsibilities with respect to end users, business partners and the\nlike. While this license is intended to facilitate the commercial use of\nthe Program, the Contributor who includes the Program in a commercial\nproduct offering should do so in a manner which does not create\npotential liability for other Contributors. Therefore, if a Contributor\nincludes the Program in a commercial product offering, such Contributor\n(&quot;Commercial Contributor&quot;) hereby agrees to defend and\nindemnify every other Contributor (&quot;Indemnified Contributor&quot;)\nagainst any losses, damages and costs (collectively &quot;Losses&quot;)\narising from claims, lawsuits and other legal actions brought by a third\nparty against the Indemnified Contributor to the extent caused by the\nacts or omissions of such Commercial Contributor in connection with its\ndistribution of the Program in a commercial product offering. The\nobligations in this section do not apply to any claims or Losses\nrelating to any actual or alleged intellectual property infringement. In\norder to qualify, an Indemnified Contributor must: a) promptly notify\nthe Commercial Contributor in writing of such claim, and b) allow the\nCommercial Contributor to control, and cooperate with the Commercial\nContributor in, the defense and any related settlement negotiations. The\nIndemnified Contributor may participate in any such claim at its own\nexpense.</p>\n\n<p>For example, a Contributor might include the Program in a commercial\nproduct offering, Product X. That Contributor is then a Commercial\nContributor. If that Commercial Contributor then makes performance\nclaims, or offers warranties related to Product X, those performance\nclaims and warranties are such Commercial Contributor's responsibility\nalone. Under this section, the Commercial Contributor would have to\ndefend claims against the other Contributors related to those\nperformance claims and warranties, and if a court requires any other\nContributor to pay any damages as a result, the Commercial Contributor\nmust pay those damages.</p>\n\n<p><b>5. NO WARRANTY</b></p>\n\n<p>EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS\nPROVIDED ON AN &quot;AS IS&quot; BASIS, WITHOUT WARRANTIES OR CONDITIONS\nOF ANY KIND, EITHER EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION,\nANY WARRANTIES OR CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY\nOR FITNESS FOR A PARTICULAR PURPOSE. Each Recipient is solely\nresponsible for determining the appropriateness of using and\ndistributing the Program and assumes all risks associated with its\nexercise of rights under this Agreement , including but not limited to\nthe risks and costs of program errors, compliance with applicable laws,\ndamage to or loss of data, programs or equipment, and unavailability or\ninterruption of operations.</p>\n\n<p><b>6. DISCLAIMER OF LIABILITY</b></p>\n\n<p>EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT\nNOR ANY CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT,\nINCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING\nWITHOUT LIMITATION LOST PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF\nLIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING\nNEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OR\nDISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS GRANTED\nHEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.</p>\n\n<p><b>7. GENERAL</b></p>\n\n<p>If any provision of this Agreement is invalid or unenforceable under\napplicable law, it shall not affect the validity or enforceability of\nthe remainder of the terms of this Agreement, and without further action\nby the parties hereto, such provision shall be reformed to the minimum\nextent necessary to make such provision valid and enforceable.</p>\n\n<p>If Recipient institutes patent litigation against any entity\n(including a cross-claim or counterclaim in a lawsuit) alleging that the\nProgram itself (excluding combinations of the Program with other\nsoftware or hardware) infringes such Recipient's patent(s), then such\nRecipient's rights granted under Section 2(b) shall terminate as of the\ndate such litigation is filed.</p>\n\n<p>All Recipient's rights under this Agreement shall terminate if it\nfails to comply with any of the material terms or conditions of this\nAgreement and does not cure such failure in a reasonable period of time\nafter becoming aware of such noncompliance. If all Recipient's rights\nunder this Agreement terminate, Recipient agrees to cease use and\ndistribution of the Program as soon as reasonably practicable. However,\nRecipient's obligations under this Agreement and any licenses granted by\nRecipient relating to the Program shall continue and survive.</p>\n\n<p>Everyone is permitted to copy and distribute copies of this\nAgreement, but in order to avoid inconsistency the Agreement is\ncopyrighted and may only be modified in the following manner. The\nAgreement Steward reserves the right to publish new versions (including\nrevisions) of this Agreement from time to time. No one other than the\nAgreement Steward has the right to modify this Agreement. The Eclipse\nFoundation is the initial Agreement Steward. The Eclipse Foundation may\nassign the responsibility to serve as the Agreement Steward to a\nsuitable separate entity. Each new version of the Agreement will be\ngiven a distinguishing version number. The Program (including\nContributions) may always be distributed subject to the version of the\nAgreement under which it was received. In addition, after a new version\nof the Agreement is published, Contributor may elect to distribute the\nProgram (including its Contributions) under the new version. Except as\nexpressly stated in Sections 2(a) and 2(b) above, Recipient receives no\nrights or licenses to the intellectual property of any Contributor under\nthis Agreement, whether expressly, by implication, estoppel or\notherwise. All rights in the Program not expressly granted under this\nAgreement are reserved.</p>\n\n<p>This Agreement is governed by the laws of the State of New York and\nthe intellectual property laws of the United States of America. No party\nto this Agreement will bring a legal action under this Agreement more\nthan one year after the cause of action arose. Each party waives its\nrights to a jury trial in any resulting litigation.</p>\n\n</body>\n\n</html>\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/META-INF/MANIFEST.MF",
    "content": "Manifest-Version: 1.0\r\nClass-Path: \r\n\r\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/WEB-INF/.gitignore",
    "content": "/classes\n/lib\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/WEB-INF/dumps/.gitignore",
    "content": "/*\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/WEB-INF/dumps/empty-de.xml",
    "content": "<mediawiki xmlns=\"http://www.mediawiki.org/xml/export-0.4/\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://www.mediawiki.org/xml/export-0.4/ http://www.mediawiki.org/xml/export-0.4.xsd\" version=\"0.4\" xml:lang=\"de\">\n  <siteinfo>\n    <sitename>Wikipedia</sitename>\n    <base>http://de.wikipedia.org/wiki/Wikipedia:Hauptseite</base>\n    <generator>MediaWiki 1.16alpha-wmf</generator>\n    <case>first-letter</case>\n    <namespaces>\n      <namespace key=\"-2\">Media</namespace>\n      <namespace key=\"-1\">Spezial</namespace>\n      <namespace key=\"0\" />\n      <namespace key=\"1\">Diskussion</namespace>\n      <namespace key=\"2\">Benutzer</namespace>\n      <namespace key=\"3\">Benutzer Diskussion</namespace>\n      <namespace key=\"4\">Wikipedia</namespace>\n      <namespace key=\"5\">Wikipedia Diskussion</namespace>\n      <namespace key=\"6\">Datei</namespace>\n      <namespace key=\"7\">Datei Diskussion</namespace>\n      <namespace key=\"8\">MediaWiki</namespace>\n      <namespace key=\"9\">MediaWiki Diskussion</namespace>\n      <namespace key=\"10\">Vorlage</namespace>\n      <namespace key=\"11\">Vorlage Diskussion</namespace>\n      <namespace key=\"12\">Hilfe</namespace>\n      <namespace key=\"13\">Hilfe Diskussion</namespace>\n      <namespace key=\"14\">Kategorie</namespace>\n      <namespace key=\"15\">Kategorie Diskussion</namespace>\n      <namespace key=\"100\">Portal</namespace>\n      <namespace key=\"101\">Portal Diskussion</namespace>\n    </namespaces>\n  </siteinfo>\n</mediawiki>\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/WEB-INF/dumps/empty-en.xml",
    "content": "<mediawiki xmlns=\"http://www.mediawiki.org/xml/export-0.5/\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://www.mediawiki.org/xml/export-0.5/ http://www.mediawiki.org/xml/export-0.5.xsd\" version=\"0.5\" xml:lang=\"en\">\n  <siteinfo>\n    <sitename>Wikipedia</sitename>\n    <base>http://en.wikipedia.org/wiki/Main_Page</base>\n    <generator></generator>\n    <case>first-letter</case>\n    <namespaces>\n      <namespace key=\"-2\" case=\"first-letter\">Media</namespace>\n      <namespace key=\"-1\" case=\"first-letter\">Special</namespace>\n      <namespace key=\"0\" case=\"first-letter\" />\n      <namespace key=\"1\" case=\"first-letter\">Talk</namespace>\n      <namespace key=\"2\" case=\"first-letter\">User</namespace>\n      <namespace key=\"3\" case=\"first-letter\">User talk</namespace>\n      <namespace key=\"4\" case=\"first-letter\">Wikipedia</namespace>\n      <namespace key=\"5\" case=\"first-letter\">Wikipedia talk</namespace>\n      <namespace key=\"6\" case=\"first-letter\">File</namespace>\n      <namespace key=\"7\" case=\"first-letter\">File talk</namespace>\n      <namespace key=\"8\" case=\"first-letter\">MediaWiki</namespace>\n      <namespace key=\"9\" case=\"first-letter\">MediaWiki talk</namespace>\n      <namespace key=\"10\" case=\"first-letter\">Template</namespace>\n      <namespace key=\"11\" case=\"first-letter\">Template talk</namespace>\n      <namespace key=\"12\" case=\"first-letter\">Help</namespace>\n      <namespace key=\"13\" case=\"first-letter\">Help talk</namespace>\n      <namespace key=\"14\" case=\"first-letter\">Category</namespace>\n      <namespace key=\"15\" case=\"first-letter\">Category talk</namespace>\n    </namespaces>\n  </siteinfo>\n</mediawiki>\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/WEB-INF/dumps/empty-es.xml",
    "content": "<mediawiki xmlns=\"http://www.mediawiki.org/xml/export-0.4/\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://www.mediawiki.org/xml/export-0.4/ http://www.mediawiki.org/xml/export-0.4.xsd\" version=\"0.4\" xml:lang=\"es\">\n  <siteinfo>\n    <sitename>Wikipedia</sitename>\n    <base>http://es.wikipedia.org/wiki/Wikipedia:Portada</base>\n    <generator>MediaWiki 1.16alpha-wmf</generator>\n    <case>first-letter</case>\n    <namespaces>\n      <namespace key=\"-2\">Media</namespace>\n      <namespace key=\"-1\">Especial</namespace>\n      <namespace key=\"0\" />\n      <namespace key=\"1\">Discusión</namespace>\n      <namespace key=\"2\">Usuario</namespace>\n      <namespace key=\"3\">Usuario Discusión</namespace>\n      <namespace key=\"4\">Wikipedia</namespace>\n      <namespace key=\"5\">Wikipedia Discusión</namespace>\n      <namespace key=\"6\">Archivo</namespace>\n      <namespace key=\"7\">Archivo Discusión</namespace>\n      <namespace key=\"8\">MediaWiki</namespace>\n      <namespace key=\"9\">MediaWiki Discusión</namespace>\n      <namespace key=\"10\">Plantilla</namespace>\n      <namespace key=\"11\">Plantilla Discusión</namespace>\n      <namespace key=\"12\">Ayuda</namespace>\n      <namespace key=\"13\">Ayuda Discusión</namespace>\n      <namespace key=\"14\">Categoría</namespace>\n      <namespace key=\"15\">Categoría Discusión</namespace>\n      <namespace key=\"100\">Portal</namespace>\n      <namespace key=\"101\">Portal Discusión</namespace>\n      <namespace key=\"102\">Wikiproyecto</namespace>\n      <namespace key=\"103\">Wikiproyecto Discusión</namespace>\n      <namespace key=\"104\">Anexo</namespace>\n      <namespace key=\"105\">Anexo Discusión</namespace>\n    </namespaces>\n  </siteinfo>\n</mediawiki>\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/WEB-INF/plugins/.gitignore",
    "content": "/*\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/WEB-INF/web.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<web-app version=\"2.5\" xmlns=\"http://java.sun.com/xml/ns/javaee\"\n  xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n  xsi:schemaLocation=\"http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd\">\n    <display-name>WikiOnScalaris</display-name>\n    <servlet>\n        <description>\n        </description>\n        <display-name>WikiServletScalaris</display-name>\n        <servlet-name>WikiServletScalaris</servlet-name>\n        <servlet-class>de.zib.scalaris.examples.wikipedia.bliki.WikiServletScalaris</servlet-class>\n        <load-on-startup>0</load-on-startup>\n        <init-param>\n            <param-name>SERVERNAME</param-name>\n            <param-value>localhost:8080</param-value>\n        </init-param>\n        <init-param>\n            <param-name>LOG_USER_REQS</param-name>\n            <!-- keep records of the last x minutes since the last request-->\n            <param-value>0</param-value>\n        </init-param>\n        <init-param>\n            <param-name>SCALARIS_NODE_DISCOVERY</param-name>\n            <!-- use the node discovery daemon to look for new Scalaris nodes (fixed delay between runs in seconds (0 to disable)-->\n            <param-value>60</param-value>\n        </init-param>\n        <init-param>\n            <param-name>SERVERPATH</param-name>\n            <param-value>/scalaris-wiki/wiki</param-value>\n        </init-param>\n        <init-param>\n            <param-name>WIKI_USE_BACKLINKS</param-name>\n            <param-value>true</param-value>\n        </init-param>\n        <init-param>\n            <param-name>WIKI_SAVEPAGE_RETRIES</param-name>\n            <param-value>3</param-value>\n        </init-param>\n        <init-param>\n            <param-name>WIKI_SAVEPAGE_RETRY_DELAY</param-name>\n            <param-value>10</param-value>\n        </init-param>\n        <init-param>\n            <param-name>WIKI_PAGES_CACHE_IMPL</param-name>\n            <!-- available implementations: BLOOM | FULL_SET -->\n            <param-value>FULL_SET</param-value>\n        </init-param>\n        <init-param>\n            <param-name>WIKI_REBUILD_PAGES_CACHE</param-name>\n            <param-value>600</param-value>\n        </init-param>\n        <init-param>\n            <param-name>WIKI_STORE_CONTRIBUTIONS</param-name>\n            <param-value>OUTSIDE_TX</param-value>\n        </init-param>\n        <init-param>\n            <!-- IMPORTANT: When changing this file, leave both -name and -value on the same line.\n                            Otherwise 'sed' will fail to match it. -->\n            <param-name>WIKI_OPTIMISATIONS</param-name>\n<!--\noptimisations are executed in the same order as mentioned\nformat: \"|\"-separated list of the following (no spaces!): \n <operation>:<optimisation>([<parameter>[,<parameter>[,<parameter>]...]])\navailable operations:\n to set all (except PAGE_COUNT and CATEGORY_PAGE_COUNT) to one optimisation, use:\n  ALL\n otherwise use values from de.zib.scalaris.examples.wikipedia.ScalarisOpType:\n  PAGE_LIST,\n  PAGE_COUNT (if set to buckets, PAGE_LIST must also use buckets for now!),\n  CATEGORY_PAGE_LIST,\n  CATEGORY_PAGE_COUNT (if set to buckets, CATEGORY_PAGE_LIST must also use buckets for now!),\n  TEMPLATE_PAGE_LIST,\n  BACKLINK_PAGE_LIST,\n  SHORTREV_LIST,\n  ARTICLE_COUNT,\n  PAGE,\n  REVISION,\n  CONTRIBUTION,\n  EDIT_STAT\navailable optimisations (parameter types depend on the implementation):\n see implementations of de.zib.scalaris.examples.wikipedia.Options.ListOptimisation\n  TRADITIONAL\n  APPEND_INCREMENT\n  APPEND_INCREMENT_PARTIALREAD\n  APPEND_INCREMENT_BUCKETS_RANDOM(int buckets)\n  APPEND_INCREMENT_PARTIALREAD_BUCKETS_RANDOM(int buckets)\n  APPEND_INCREMENT_BUCKETS_WITH_HASH(int buckets)\n  APPEND_INCREMENT_PARTIALREAD_BUCKETS_WITH_HASH(int buckets)\n  APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM(int readBuckets, int writeBuckets)\n  APPEND_INCREMENT_PARTIALREAD_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM(int readBuckets, int writeBuckets)\n  APPEND_INCREMENT_BUCKETS_WITH_WCACHE_HASH(int readBuckets, int writeBuckets)\n\nexample:\n <param-value>ALL:APPEND_INCREMENT|PAGE_LIST:APPEND_INCREMENT_BUCKETS_WITH_HASH(10)</param-value>\n-->\n            <param-value>ALL:APPEND_INCREMENT</param-value>\n        </init-param>\n<!--\n        <init-param>\n            <param-name>4CaaSt.accounting</param-name>\n            <param-value>URL</param-value>\n        </init-param>\n-->\n    </servlet>\n    <servlet-mapping>\n        <servlet-name>WikiServletScalaris</servlet-name>\n        <url-pattern>/wiki</url-pattern>\n    </servlet-mapping>\n    <welcome-file-list>\n        <welcome-file>index.jsp</welcome-file>\n    </welcome-file-list>\n    \n  <!-- A filter that sets character encoding that is used to decode -->\n  <!-- parameters in a POST request -->\n    <filter>\n        <filter-name>setCharacterEncodingFilter</filter-name>\n        <filter-class>de.zib.scalaris.examples.wikipedia.tomcat.SetCharacterEncodingFilter</filter-class>\n        <init-param>\n            <param-name>encoding</param-name>\n            <param-value>UTF-8</param-value>\n        </init-param>\n        <async-supported>true</async-supported>\n    </filter>\n\n  <!-- A filter that triggers request parameters parsing and rejects the    -->\n  <!-- request if some parameters were skipped because of parsing errors or -->\n  <!-- request size limitations.                                            -->\n    <filter>\n        <filter-name>failedRequestFilter</filter-name>\n        <filter-class>\n          de.zib.scalaris.examples.wikipedia.tomcat.FailedRequestFilter\n        </filter-class>\n        <async-supported>true</async-supported>\n    </filter>\n    \n  <!-- The mapping for the Set Character Encoding Filter -->\n    <filter-mapping>\n        <filter-name>setCharacterEncodingFilter</filter-name>\n        <url-pattern>/*</url-pattern>\n    </filter-mapping>\n    \n  <!-- The mapping for the Failed Request Filter -->\n    <filter-mapping>\n        <filter-name>failedRequestFilter</filter-name>\n        <url-pattern>/*</url-pattern>\n    </filter-mapping>\n</web-app>\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/index.jsp",
    "content": "<%@page language=\"java\" contentType=\"text/html; charset=UTF-8\" pageEncoding=\"UTF-8\"%>\n<jsp:forward page=\"wiki\"/>\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/page.jsp",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\" ?>\n<%@page import=\"de.zib.scalaris.examples.wikipedia.bliki.WikiServlet\"%>\n<%@page import=\"de.zib.scalaris.examples.wikipedia.InvolvedKey\"%>\n<%@page import=\"java.net.URL\"%>\n<%@page language=\"java\" contentType=\"text/html; charset=UTF-8\" pageEncoding=\"UTF-8\"\n import=\"java.util.Calendar,java.util.Locale,java.text.DateFormat,java.text.SimpleDateFormat,java.util.TimeZone,java.util.Iterator,java.util.Map,java.util.List,org.apache.commons.lang.StringUtils,org.apache.commons.lang.StringEscapeUtils,java.net.URLEncoder\"%>\n<% String req_render = request.getParameter(\"render\"); %>\n<jsp:useBean id=\"pageBean\" type=\"de.zib.scalaris.examples.wikipedia.bliki.WikiPageBean\" scope=\"request\" />\n<jsp:useBean id=\"servlet\" type=\"de.zib.scalaris.examples.wikipedia.WikiServletContext\" scope=\"request\" />\n<% /* created page based on https://secure.wikimedia.org/wiktionary/simple/wiki/relief */ %>\n<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n<html lang=\"${ pageBean.wikiLang }\" dir=\"${ pageBean.wikiLangDir }\" xmlns=\"http://www.w3.org/1999/xhtml\">\n<head>\n<% final String safePageTitle = StringEscapeUtils.escapeHtml(URLEncoder.encode(pageBean.getTitle(), \"UTF-8\")); %>\n<% final String andServiceUser = pageBean.getServiceUser().isEmpty() ? \"\" : \"&amp;service_user=\" + pageBean.getServiceUser(); %>\n<title>${ pageBean.title } - ${ pageBean.wikiTitle }</title>\n<!--<% if (!pageBean.getError().isEmpty()) { %>\n<error>${ pageBean.error }</error>\n<% } %>-->\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" />\n<meta http-equiv=\"Content-Style-Type\" content=\"text/css\" />\n<link rel=\"alternate\" type=\"application/x-wiki\" title=\"change this page\" href=\"wiki?title=<%= safePageTitle %>&amp;action=edit<%= andServiceUser %>\">\n<link rel=\"edit\" title=\"change this page\" href=\"wiki?title=<%= safePageTitle %>&amp;action=edit<%= andServiceUser %>\">\n<% /* \n<meta name=\"generator\" content=\"MediaWiki 1.17wmf1\" />\n<link rel=\"apple-touch-icon\" href=\"http://simple.wiktionary.org/apple-touch-icon.png\">\n*/ %>\n<link rel=\"shortcut icon\" href=\"favicon-wikipedia.ico\" />\n<% /*\n<link rel=\"search\" type=\"application/opensearchdescription+xml\" href=\"https://secure.wikimedia.org/wiktionary/simple/w/opensearch_desc.php\" title=\"Wiktionary (simple)\">\n<link rel=\"EditURI\" type=\"application/rsd+xml\" href=\"https://secure.wikimedia.org/wiktionary/simple/w/api.php?action=rsd\">\n*/ %>\n<link rel=\"copyright\" href=\"http://creativecommons.org/licenses/by-sa/3.0/\" />\n<% /*\n<link rel=\"alternate\" type=\"application/atom+xml\" title=\"Wiktionary Atom feed\" href=\"https://secure.wikimedia.org/wiktionary/simple/w/index.php?title=Special:RecentChanges&amp;feed=atom\">\n*/ %>\n<link rel=\"stylesheet\" href=\"skins/load_002.css\" type=\"text/css\" media=\"all\" />\n<style type=\"text/css\" media=\"all\">.suggestions{overflow:hidden;position:absolute;top:0px;left:0px;width:0px;border:none;z-index:99;padding:0;margin:-1px -1px 0 0} html > body .suggestions{margin:-1px 0 0 0}.suggestions-special{position:relative;background-color:Window;font-size:0.8em;cursor:pointer;border:solid 1px #aaaaaa;padding:0;margin:0;margin-top:-2px;display:none;padding:0.25em 0.25em;line-height:1.25em}.suggestions-results{background-color:white;background-color:Window;font-size:0.8em;cursor:pointer;border:solid 1px #aaaaaa;padding:0;margin:0}.suggestions-result{color:black;color:WindowText;margin:0;line-height:1.5em;padding:0.01em 0.25em;text-align:left}.suggestions-result-current{background-color:#4C59A6;background-color:Highlight;color:white;color:HighlightText}.suggestions-special .special-label{font-size:0.8em;color:gray;text-align:left}.suggestions-special .special-query{color:black;font-style:italic;text-align:left}.suggestions-special .special-hover{background-color:silver}.suggestions-result-current .special-label,.suggestions-result-current .special-query{color:white;color:HighlightText}.autoellipsis-matched,.highlight{font-weight:bold}</style>\n<style type=\"text/css\" media=\"all\">#mw-panel.collapsible-nav div.portal{background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAIwAAAABCAMAAAA7MLYKAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAEtQTFRF29vb2tra4ODg6urq5OTk4uLi6+vr7e3t7Ozs8PDw5+fn4+Pj4eHh3d3d39/f6Ojo5eXl6enp8fHx8/Pz8vLy7+/v3Nzc2dnZ2NjYnErj7QAAAD1JREFUeNq0wQUBACAMALDj7hf6JyUFGxzEnYhC9GaNPG1xVffGDErk/iCigLl1XV2xM49lfAxEaSM+AQYA9HMKuv4liFQAAAAASUVORK5CYII=);background-image:url(https://secure.wikimedia.org/w/extensions-1.17/Vector/modules/./images/portal-break.png?2011-02-12T21:25:00Z)!ie;background-position:left top;background-repeat:no-repeat;padding:0.25em 0 !important;margin:-11px 9px 10px 11px}#mw-panel.collapsible-nav div.portal h5{color:#4D4D4D;font-weight:normal;background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQBAMAAADt3eJSAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAA9QTFRFeXl53d3dmpqasbGx////GU0iEgAAAAV0Uk5T/////wD7tg5TAAAAK0lEQVQI12NwgQIG0hhCDAwMTCJAhqMCA4MiWEoIJABiOCooQhULi5BqMgB2bh4svs8t+QAAAABJRU5ErkJggg==) left center no-repeat;background:url(https://secure.wikimedia.org/w/extensions-1.17/Vector/modules/./images/open.png?2011-02-12T21:25:00Z) left center no-repeat!ie;padding:4px 0 3px 1.5em;margin-bottom:0px}#mw-panel.collapsible-nav div.collapsed h5{color:#0645AD;background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAMAAAAoLQ9TAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAAxQTFRF3d3deXl5////nZ2dQA6SoAAAAAN0Uk5T//8A18oNQQAAADNJREFUeNpiYEIDDMQKMKALMDOgCTDCRWACcBG4AEwEIcDITEAFuhnotmC4g4EEzwEEGAADqgHmQSPJKgAAAABJRU5ErkJggg==) left center no-repeat;background:url(https://secure.wikimedia.org/w/extensions-1.17/Vector/modules/./images/closed-ltr.png?2011-02-12T21:25:00Z) left center no-repeat!ie;margin-bottom:0px}#mw-panel.collapsible-nav div h5:hover{cursor:pointer;text-decoration:none}#mw-panel.collapsible-nav div.collapsed h5:hover{text-decoration:underline}#mw-panel.collapsible-nav div.portal div.body{background:none !important;padding-top:0px;display:none}#mw-panel.collapsible-nav div.persistent div.body{display:block}#mw-panel.collapsible-nav div.first h5{display:none}#mw-panel.collapsible-nav div.persistent h5{background:none !important;padding-left:0.7em;cursor:default}#mw-panel.collapsible-nav div.portal div.body ul li{padding:0.25em 0}#mw-panel.collapsible-nav div.first{background-image:none;margin-top:0px}#mw-panel.collapsible-nav div.persistent div.body{margin-left:0.5em}</style>\n<meta name=\"ResourceLoaderDynamicStyles\" content=\"\" />\n<link rel=\"stylesheet\" href=\"skins/load.css\" type=\"text/css\" media=\"all\" />\n<style type=\"text/css\" media=\"all\">a.new,#quickbar a.new{color:#ba0000}</style>\n</head>\n<body class=\"mediawiki ltr ns-0 ns-subject skin-vector\">\n        <div id=\"mw-page-base\" class=\"noprint\"></div>\n        <div id=\"mw-head-base\" class=\"noprint\"></div>\n        <!-- content -->\n        <div id=\"content\">\n            <a id=\"top\"></a>\n            <div id=\"mw-js-message\" style=\"display:none;\"></div>\n            <!-- sitenotice -->\n            <div id=\"siteNotice\"><!-- centralNotice loads here -->${ pageBean.notice }</div>\n            <!-- /sitenotice -->\n            <!-- firstHeading -->\n            <h1 id=\"firstHeading\" class=\"firstHeading\">${ pageBean.error }${ pageBean.title }</h1>\n            <!-- /firstHeading -->\n            <!-- bodyContent -->\n            <div id=\"bodyContent\">\n                <!-- tagline -->\n                <div id=\"siteSub\">From <%= pageBean.getWikiNamespace().getMeta() %></div>\n                <!-- /tagline -->\n                <!-- subtitle -->\n                <div id=\"contentSub\">\n                ${ pageBean.contentSub }\n                </div>\n                <!-- /subtitle -->\n                <!-- jumpto -->\n                <div id=\"jump-to-nav\">\n                    Jump to: <a href=\"#mw-head\">navigation</a>,\n                    <a href=\"#p-search\">search</a>\n                </div>\n                <!-- /jumpto -->\n                <!-- bodytext -->\n\n${ pageBean.page }\n\n<% if (!pageBean.getSubCategories().isEmpty()) {\n    int rest = pageBean.getSubCategories().size();\n%>\n\n<div id=\"mw-subcategories\">\n<h2>Subcategories</h2>\n<p>This category has the following <%= rest %> subcategories.</p>\n<%\nint[] columnCount = new int[3];\nif (rest > 15) {\n    columnCount[0] = rest / 3;\n    rest -= columnCount[0];\n} else {\n    columnCount[0] = rest;\n    rest = 0;\n}\ncolumnCount[1] = rest / 2;\ncolumnCount[2] = rest - columnCount[1];\n\nIterator<String> iter = pageBean.getSubCategories().iterator();\n%>\n\n<table width=\"100%\"><tbody><tr valign=\"top\">\n<% for (int i = 0; i < 3; ++i) { %>\n<td>\n\n<h3> </h3>\n<ul>\n<% for (int j = 0; j < columnCount[i]; ++j) {\n    final String subCat = iter.next();\n    final String safeFullSubCat = StringEscapeUtils.escapeHtml(pageBean.getWikiNamespace().getCategory() + \":\" + subCat);\n%>\n<li>\n  <div class=\"CategoryTreeSection\">\n\t<div class=\"CategoryTreeItem\">\n<% /*\n\t  <span class=\"CategoryTreeEmptyBullet\">[<b>×</b>] </span>\n*/ %>\n\t  <a class=\"CategoryTreeLabel  CategoryTreeLabelNs14 CategoryTreeLabelCategory\" href=\"wiki?title=<%=safeFullSubCat%><%= andServiceUser %>\"><%=subCat%></a>\n<% /* \n\t  <span title=\"contains 0 subcategories, 1745 pages, and 0 files\">(0)</span>\n*/ %>\n\t</div>\n    <div class=\"CategoryTreeChildren\" style=\"display: none;\"></div>\n  </div>\n</li>\n<% } %>\n</ul></td>\n<% } %>\n</tr>\n</table>\n</div>\n<% } %>\n\n<% if (!pageBean.getCategoryPages().isEmpty()) {\n    int rest = pageBean.getCategoryPages().size(); %>\n\n<div id=\"mw-pages\">\n<h2>Pages in category \"${ pageBean.title }\"</h2>\n<p>The following <%= rest %> pages are in this category.</p>\n<%\nint[] columnCount = new int[3];\nif (rest > 15) {\n    columnCount[0] = rest / 3;\n    rest -= columnCount[0];\n} else {\n    columnCount[0] = rest;\n    rest = 0;\n}\ncolumnCount[1] = rest / 2;\ncolumnCount[2] = rest - columnCount[1];\n\nIterator<String> iter = pageBean.getCategoryPages().iterator();\n%>\n\n<table width=\"100%\"><tbody><tr valign=\"top\">\n<% for (int i = 0; i < 3; ++i) { %>\n<td>\n\n<h3> </h3>\n<ul>\n<% for (int j = 0; j < columnCount[i]; ++j) {\n    final String catPage = iter.next();\n    final String safeCatPage = StringEscapeUtils.escapeHtml(catPage);\n%>\n<li><a href=\"wiki?title=<%=safeCatPage%><%= andServiceUser %>\" title=\"<%=safeCatPage%>\"><%=catPage%></a></li>\n<% } %>\n</ul></td>\n<% } %>\n</tr>\n</table>\n</div>\n<% } %>\n\n                <!-- /bodytext -->\n<% if (req_render == null || !req_render.equals(\"0\")) { %>\n                <!-- catlinks -->\n<% if (!pageBean.getCategories().isEmpty()) { %>\n                <div id=\"catlinks\" class=\"catlinks\">\n                <div id=\"mw-normal-catlinks\">\n                <a href=\"wiki?title=Special:Categories<%= andServiceUser %>\" title=\"Special:Categories\">Categories</a>:\n<%\n    for (Iterator<String> iterator = pageBean.getCategories().iterator(); iterator.hasNext();) {\n        final String category = iterator.next();\n        final String safeCategory = StringEscapeUtils.escapeHtml(category);\n        final String safeFullCatName = pageBean.getWikiNamespace().getCategory() + \":\" + safeCategory;\n        out.print(\"<span dir=\\\"\" + pageBean.getWikiLangDir() + \"\\\"><a href=\\\"wiki?title=\" + safeFullCatName + andServiceUser + \"\\\" title=\\\"\" + safeFullCatName + \"\\\">\" + category + \"</a></span>\");\n\n        if (iterator.hasNext()) {\n            out.print(\" | \");\n        }\n    }\n%>\n                </div>\n                </div>\n<% } // if (!pageBean.getCategories().isEmpty()) %>\n                <!-- /catlinks -->\n<% } %>\n                <div class=\"visualClear\"></div>\n            </div>\n            <!-- /bodyContent -->\n        </div>\n        <!-- /content -->\n        <!-- header -->\n        <div id=\"mw-head\" class=\"noprint\">\n            \n<!-- 0 -->\n<div id=\"p-personal\" class=\"\">\n    <h5>Personal tools</h5>\n    <ul>\n                    <li id=\"pt-login\"><a href=\"wiki?title=Special:UserLogin&amp;returnto=<%= safePageTitle %><%= andServiceUser %>\" title=\"You are encouraged to log in; however, it is not mandatory [o]\" accesskey=\"o\">Log in / create account</a></li>\n    </ul>\n</div>\n\n<!-- /0 -->\n            <div id=\"left-navigation\">\n                \n<!-- 0 -->\n<div id=\"p-namespaces\" class=\"vectorTabs\">\n    <h5>Namespaces</h5>\n    <ul>\n    <%\n    final String mainSelected = pageBean.getWikiNamespace().isTalkPage(pageBean.getTitle()) ? \"\" : \" class=\\\"selected\\\"\";\n    final String talkSelected = !pageBean.getWikiNamespace().isTalkPage(pageBean.getTitle()) ? \"\" : \" class=\\\"selected\\\"\";\n    %>\n                    <li id=\"ca-nstab-main\"<%= mainSelected %>><span><a href=\"wiki?title=<%= StringEscapeUtils.escapeHtml(pageBean.getWikiNamespace().getPageNameFromTalkPage(pageBean.getTitle())) %><%= andServiceUser %>\" title=\"View the content page [c]\" accesskey=\"c\">Page</a></span></li>\n                    <li id=\"ca-talk\"<%= talkSelected %>><span><a href=\"wiki?title=<%= StringEscapeUtils.escapeHtml(pageBean.getWikiNamespace().getTalkPageFromPageName(pageBean.getTitle())) %><%= andServiceUser %>\" title=\"Discussion about the content page [t]\" accesskey=\"t\">Talk</a></span></li>\n    </ul>\n</div>\n\n<!-- /0 -->\n\n<!-- 1 -->\n<div id=\"p-variants\" class=\"vectorMenu emptyPortlet\">\n        <h5><span>Variants</span><a href=\"#\"></a></h5>\n    <div class=\"menu\">\n        <ul>\n        </ul>\n    </div>\n</div>\n\n<!-- /1 -->\n            </div>\n            <div id=\"right-navigation\">\n                \n<!-- 0 -->\n<div id=\"p-views\" class=\"vectorTabs\">\n    <h5>Views</h5>\n    <ul>\n          <% if (!pageBean.isNotAvailable()) { %>\n                    <li id=\"ca-view\" class=\"selected\"><span><a href=\"wiki?title=<%= safePageTitle %><%= andServiceUser %>\">Read</a></span></li>\n          <% if (!pageBean.isEditRestricted()) { %>\n                    <li id=\"ca-edit\"><span><a href=\"wiki?title=<%= safePageTitle %>&amp;action=edit&amp;oldid=${ pageBean.version }<%= andServiceUser %>\" title=\"You can edit this page. Please use the preview button before saving [e]\" accesskey=\"e\">Change</a></span></li>\n          <% } else {%>\n                    <li id=\"ca-viewsource\"><span><a href=\"wiki?title=<%= safePageTitle %>&amp;action=edit&amp;oldid=${ pageBean.version }<%= andServiceUser %>\" title=\"This page is protected. You can view its source [e]\" accesskey=\"e\">View source</a></span></li>\n          <% } %>\n                    <li id=\"ca-history\" class=\"collapsible \"><span><a href=\"wiki?title=<%= safePageTitle %>&amp;action=history<%= andServiceUser %>\" title=\"Past revisions of this page [h]\" accesskey=\"h\">View history</a></span></li>\n          <% } else {%>\n          <% if (!pageBean.isEditRestricted()) { %>\n                    <li id=\"ca-edit\"><span><a href=\"wiki?title=<%= safePageTitle %>&amp;action=edit&amp;oldid=${ pageBean.version }<%= andServiceUser %>\" title=\"You can edit this page. Please use the preview button before saving [e]\" accesskey=\"e\">Start</a></span></li>\n          <% }%>\n          <% } %>\n    </ul>\n</div>\n\n<!-- /0 -->\n\n<!-- 1 -->\n<div id=\"p-cactions\" class=\"vectorMenu emptyPortlet\">\n    <h5><span>Actions</span><a href=\"#\"></a></h5>\n    <div class=\"menu\">\n        <ul>\n        </ul>\n    </div>\n</div>\n\n<!-- /1 -->\n\n<!-- 2 -->\n<div id=\"p-search\">\n    <h5><label for=\"searchInput\">Search</label></h5>\n    <form action=\"wiki?\" id=\"searchform\">\n        <input name=\"title\" value=\"Special:Search\" type=\"hidden\" />\n        <div id=\"simpleSearch\">\n            <input type=\"hidden\" value=\"${ pageBean.serviceUser }\" name=\"service_user\"/>\n            <input autocomplete=\"off\" placeholder=\"Search\" tabindex=\"1\" id=\"searchInput\" name=\"search\" title=\"Search <%= pageBean.getWikiNamespace().getMeta() %> [f]\" accesskey=\"f\" type=\"text\" />\n            <button id=\"searchButton\" type=\"submit\" name=\"button\" title=\"Search the pages for this text\"><img src=\"skins/search-ltr.png\" alt=\"Search\" /></button>\n        </div>\n    </form>\n</div>\n\n<!-- /2 -->\n            </div>\n        </div>\n        <!-- /header -->\n        <!-- panel -->\n            <div id=\"mw-panel\" class=\"noprint collapsible-nav\">\n                <!-- logo -->\n                    <div id=\"p-logo\"><a style=\"background-image: url(&quot;images/Wikipedia.png&quot;);\" href=\"wiki?title=Main Page<%= andServiceUser %>\" title=\"Visit the main page\"></a></div>\n                <!-- /logo -->\n                \n<!-- navigation -->\n<div class=\"portal first persistent\" id=\"p-navigation\">\n    <h5>Links</h5>\n    <div class=\"body\">\n                <ul>\n                    <li id=\"n-mainpage\"><a href=\"wiki?title=Main Page<%= andServiceUser %>\" title=\"Visit the main page [z]\" accesskey=\"z\">Main Page</a></li>\n                    <li id=\"n-recentchanges\"><a href=\"wiki?title=Special:RecentChanges<%= andServiceUser %>\" title=\"The list of recent changes in the wiki [r]\" accesskey=\"r\">New changes</a></li>\n                    <li id=\"n-randompage\"><a href=\"wiki?title=Special:Random<%= andServiceUser %>\" title=\"Load a random page [x]\" accesskey=\"x\">Show any entry</a></li>\n                    <li id=\"n-help\"><a href=\"wiki?title=Help:Contents<%= andServiceUser %>\" title=\"The place to find out\">Help</a></li>\n                </ul>\n            </div>\n</div>\n\n<!-- /navigation -->\n\n<!-- SEARCH -->\n\n<!-- /SEARCH -->\n\n<!-- TOOLBOX -->\n<div class=\"portal expanded\" id=\"p-tb\">\n    <h5 tabindex=\"2\">Toolbox</h5>\n    <div style=\"display: block;\" class=\"body\">\n        <ul>\n          <% if (de.zib.scalaris.examples.wikipedia.Options.getInstance().WIKI_USE_BACKLINKS) { %>\n                    <li id=\"t-whatlinkshere\"><a href=\"wiki?title=Special:WhatLinksHere&target=<%= safePageTitle %><%= andServiceUser %>\" title=\"List of all wiki pages that link here [j]\" accesskey=\"j\">What links here</a></li>\n          <% } %>\n          <% if (!pageBean.isNotAvailable()) { %>\n<% /*\n                    <li id=\"t-recentchangeslinked\"><a href=\"wiki?title=Special:RecentChangesLinked&target=safePageTitle<%= andServiceUser >\" title=\"Recent changes in pages linked from this page [k]\" accesskey=\"k\">Related changes</a></li>\n*/ %>\n          <% } %>\n                    <li id=\"t-specialpages\"><a href=\"wiki?title=Special:SpecialPages<%= andServiceUser %>\" title=\"List of all special pages [q]\" accesskey=\"q\">Special pages</a></li>\n<% /*\n                    <li id=\"t-print\"><a href=\"wiki?title=safePageTitle&amp;printable=yes<%= andServiceUser >\" rel=\"alternate\" title=\"Printable version of this page [p]\" accesskey=\"p\">Page for printing</a></li>\n*/ %>\n          <% if (!pageBean.isNotAvailable()) { %>\n                    <li id=\"t-permalink\"><a href=\"wiki?title=<%= safePageTitle %>&amp;oldid=${ pageBean.version }<%= andServiceUser %>\" title=\"Permanent link to this revision of the page\">Permanent link</a></li>\n          <% } %>\n<% /*\n                    <li><span><a href=\"javascript:adddefinition()\">Add definition</a></span></li>\n                    <li id=\"newimagebutton\"><span><a href=\"javascript:addimage()\">Add image</a></span></li>\n*/ %>\n        </ul>\n    </div>\n</div>\n\n<!-- /TOOLBOX -->\n\n<!-- LANGUAGES -->\n<!-- /LANGUAGES -->\n<!-- RENDERER -->\n<div class=\"portal expanded\" id=\"p-renderer\">\n    <h5 tabindex=\"2\">Renderer</h5>\n    <div style=\"display: block;\" class=\"body\">\n        <ul>\n                    <li id=\"t-renderer-default\">\n            <% if (req_render == null || req_render.equals(\"1\")) { %>\n                        Default\n            <% } else { %>\n                        <a href=\"wiki?title=<%= safePageTitle %>&render=1<%= andServiceUser %>\" title=\"Default renderer (gwtwiki)\">Default</a></li>\n            <% } %>\n                    </li>\n                    <li id=\"t-renderer-none\">\n            <% if (req_render != null && req_render.equals(\"0\")) { %>\n                        Plain\n            <% } else { %>\n                        <a href=\"wiki?title=<%= safePageTitle %>&render=0<%= andServiceUser %>\" title=\"No renderer (plain wiki text)\">Plain</a></li>\n            <% } %>\n                    </li>\n        </ul>\n    </div>\n</div>\n\n<!-- /RENDERER -->\n            </div>\n        <!-- /panel -->\n        <!-- footer -->\n        <div id=\"footer\">\n                <ul id=\"footer-info\">\n        <% if (!pageBean.isNotAvailable()) {\n          final DateFormat dfm = new SimpleDateFormat(\"d MMMMM yyyy', at 'HH:mm\");\n          dfm.setTimeZone(TimeZone.getTimeZone(\"GMT\")); // time presented by Wikipedia is UTC/GMT, not the local time of the user viewing the page\n          %>\n                    <li id=\"footer-info-lastmod\"> This page was last modified on <%= dfm.format(pageBean.getDate().getTime()) %>.</li>\n                    <li id=\"footer-info-copyright\">\n                        Text is available under the <a href=\"http://creativecommons.org/licenses/by-sa/3.0/\">Creative Commons Attribution/Share-Alike License</a>;\n                        additional terms may apply.\n                        See <a href=\"wiki?title=Terms of Use<%= andServiceUser %>\">Terms of Use</a> for details.</li>\n        <% } %>\n                </ul>\n                <ul id=\"footer-places\">\n                    <li id=\"footer-places-privacy\"><a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:Privacy policy<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:Privacy policy\">Privacy policy</a></li>\n                    <li id=\"footer-places-about\"><a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:About<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:About\">About <%= pageBean.getWikiNamespace().getMeta() %></a></li>\n                    <li id=\"footer-places-disclaimer\"><a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:General disclaimer<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:General disclaimer\">Disclaimers</a></li>\n                </ul>\n                <ul id=\"footer-icons\" class=\"noprint\">\n                </ul>\n                <div style=\"clear:both\"></div>\n                <div style=\"font-size:0.7em\">DB Timings:\n                <pre id=\"db_timings\" style=\"padding:0;line-height:0.7em\">\n<% for (Map.Entry<String,List<Long>> stats : pageBean.getStats().entrySet()) { %>\n<%= stats.getKey() %>: <%= stats.getValue().toString() %>\n<% } %>\n                </pre></div>\n                <div style=\"font-size:0.7em\">Involved DB keys:\n                <pre id=\"involved_keys\" style=\"padding:0;line-height:0.7em\">\n<% for (InvolvedKey involvedKey : pageBean.getInvolvedKeys()) { %>\n<%= involvedKey.toString() %>\n<% } %>\n                </pre></div>\n                <div style=\"font-size:0.7em\">Page save attempts: <span id=\"save_attempts\">${ pageBean.saveAttempts }</span></div>\n                <div style=\"font-size:0.7em\">Failed keys when saving page:\n                <pre id=\"save_failed_keys\" style=\"padding:0;line-height:0.7em\">\n<% for (Map.Entry<Integer,List<String>> failedKeys : pageBean.getFailedKeys().entrySet()) { %>\n<%= failedKeys.getKey() %>: <%= StringUtils.join(failedKeys.getValue(), \" # \") %>\n<% } %>\n                </pre></div>\n<% long renderTime = (System.currentTimeMillis() - pageBean.getStartTime()); %>\n                <div style=\"clear:both\"></div>\n                <div style=\"font-size:0.7em\">More timings:\n                <pre id=\"other_timings\" style=\"padding:0;line-height:0.7em\">\n\nserver: <%= renderTime %>\n\n                </pre></div>\n        </div>\n        <!-- /footer -->\n<% servlet.storeUserReq(pageBean, renderTime); %>\n</body>\n</html>\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/pageEdit.jsp",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\" ?>\n<%@page import=\"de.zib.scalaris.examples.wikipedia.bliki.MyScalarisMagicWord\"%>\n<%@page import=\"de.zib.scalaris.examples.wikipedia.bliki.MySQLiteMagicWord\"%>\n<%@page import=\"de.zib.scalaris.examples.wikipedia.bliki.MyMagicWord\"%>\n<%@page import=\"de.zib.scalaris.examples.wikipedia.bliki.WikiServlet\"%>\n<%@page import=\"de.zib.scalaris.examples.wikipedia.InvolvedKey\"%>\n<%@page language=\"java\" contentType=\"text/html; charset=UTF-8\" pageEncoding=\"UTF-8\"\n import=\"java.util.Calendar,java.util.Locale,java.text.DateFormat,java.text.SimpleDateFormat,java.util.TimeZone,java.util.Iterator,java.util.Map,java.util.List,org.apache.commons.lang.StringUtils,org.apache.commons.lang.StringEscapeUtils,java.net.URLEncoder\"%>\n<jsp:useBean id=\"pageBean\" type=\"de.zib.scalaris.examples.wikipedia.bliki.WikiPageEditBean\" scope=\"request\" />\n<jsp:useBean id=\"servlet\" type=\"de.zib.scalaris.examples.wikipedia.WikiServletContext\" scope=\"request\" />\n<% /* created page based on https://secure.wikimedia.org/wiktionary/simple/wiki/relief */ %>\n<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n<html lang=\"${ pageBean.wikiLang }\" dir=\"${ pageBean.wikiLangDir }\" xmlns=\"http://www.w3.org/1999/xhtml\">\n<head>\n<% final String safePageTitle = StringEscapeUtils.escapeHtml(URLEncoder.encode(pageBean.getTitle(), \"UTF-8\")); %>\n<% final String andServiceUser = pageBean.getServiceUser().isEmpty() ? \"\" : \"&amp;service_user=\" + pageBean.getServiceUser(); %>\n<title>Changing ${ pageBean.title } - ${ pageBean.wikiTitle }</title>\n<!--<% if (!pageBean.getError().isEmpty()) { %>\n<error>${ pageBean.error }</error>\n<% } %>-->\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" />\n<meta http-equiv=\"Content-Style-Type\" content=\"text/css\" />\n<meta name=\"robots\" content=\"noindex,nofollow\" />\n<link rel=\"alternate\" type=\"application/x-wiki\" title=\"change this page\" href=\"wiki?title=<%= safePageTitle %>&amp;action=edit<%= andServiceUser %>\">\n<link rel=\"edit\" title=\"change this page\" href=\"wiki?title=<%= safePageTitle %>&amp;action=edit<%= andServiceUser %>\">\n<% /*\n<meta name=\"generator\" content=\"MediaWiki 1.17wmf1\" />\n<link rel=\"apple-touch-icon\" href=\"http://simple.wiktionary.org/apple-touch-icon.png\">\n*/ %>\n<link rel=\"shortcut icon\" href=\"favicon-wikipedia.ico\" />\n<% /*\n<link rel=\"search\" type=\"application/opensearchdescription+xml\" href=\"https://secure.wikimedia.org/wiktionary/simple/w/opensearch_desc.php\" title=\"Wiktionary (simple)\">\n<link rel=\"EditURI\" type=\"application/rsd+xml\" href=\"https://secure.wikimedia.org/wiktionary/simple/w/api.php?action=rsd\">\n*/ %>\n<link rel=\"copyright\" href=\"http://creativecommons.org/licenses/by-sa/3.0/\" />\n<% /*\n<link rel=\"alternate\" type=\"application/atom+xml\" title=\"Wiktionary Atom feed\" href=\"https://secure.wikimedia.org/wiktionary/simple/w/index.php?title=Special:RecentChanges&amp;feed=atom\">\n*/ %>\n<link rel=\"stylesheet\" href=\"skins/load_002.css\" type=\"text/css\" media=\"all\" />\n<style type=\"text/css\" media=\"all\">.suggestions{overflow:hidden;position:absolute;top:0px;left:0px;width:0px;border:none;z-index:99;padding:0;margin:-1px -1px 0 0} html > body .suggestions{margin:-1px 0 0 0}.suggestions-special{position:relative;background-color:Window;font-size:0.8em;cursor:pointer;border:solid 1px #aaaaaa;padding:0;margin:0;margin-top:-2px;display:none;padding:0.25em 0.25em;line-height:1.25em}.suggestions-results{background-color:white;background-color:Window;font-size:0.8em;cursor:pointer;border:solid 1px #aaaaaa;padding:0;margin:0}.suggestions-result{color:black;color:WindowText;margin:0;line-height:1.5em;padding:0.01em 0.25em;text-align:left}.suggestions-result-current{background-color:#4C59A6;background-color:Highlight;color:white;color:HighlightText}.suggestions-special .special-label{font-size:0.8em;color:gray;text-align:left}.suggestions-special .special-query{color:black;font-style:italic;text-align:left}.suggestions-special .special-hover{background-color:silver}.suggestions-result-current .special-label,.suggestions-result-current .special-query{color:white;color:HighlightText}.autoellipsis-matched,.highlight{font-weight:bold}</style>\n<style type=\"text/css\" media=\"all\">#mw-panel.collapsible-nav div.portal{background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAIwAAAABCAMAAAA7MLYKAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAEtQTFRF29vb2tra4ODg6urq5OTk4uLi6+vr7e3t7Ozs8PDw5+fn4+Pj4eHh3d3d39/f6Ojo5eXl6enp8fHx8/Pz8vLy7+/v3Nzc2dnZ2NjYnErj7QAAAD1JREFUeNq0wQUBACAMALDj7hf6JyUFGxzEnYhC9GaNPG1xVffGDErk/iCigLl1XV2xM49lfAxEaSM+AQYA9HMKuv4liFQAAAAASUVORK5CYII=);background-image:url(https://secure.wikimedia.org/w/extensions-1.17/Vector/modules/./images/portal-break.png?2011-02-12T21:25:00Z)!ie;background-position:left top;background-repeat:no-repeat;padding:0.25em 0 !important;margin:-11px 9px 10px 11px}#mw-panel.collapsible-nav div.portal h5{color:#4D4D4D;font-weight:normal;background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQBAMAAADt3eJSAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAA9QTFRFeXl53d3dmpqasbGx////GU0iEgAAAAV0Uk5T/////wD7tg5TAAAAK0lEQVQI12NwgQIG0hhCDAwMTCJAhqMCA4MiWEoIJABiOCooQhULi5BqMgB2bh4svs8t+QAAAABJRU5ErkJggg==) left center no-repeat;background:url(https://secure.wikimedia.org/w/extensions-1.17/Vector/modules/./images/open.png?2011-02-12T21:25:00Z) left center no-repeat!ie;padding:4px 0 3px 1.5em;margin-bottom:0px}#mw-panel.collapsible-nav div.collapsed h5{color:#0645AD;background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAMAAAAoLQ9TAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAAxQTFRF3d3deXl5////nZ2dQA6SoAAAAAN0Uk5T//8A18oNQQAAADNJREFUeNpiYEIDDMQKMKALMDOgCTDCRWACcBG4AEwEIcDITEAFuhnotmC4g4EEzwEEGAADqgHmQSPJKgAAAABJRU5ErkJggg==) left center no-repeat;background:url(https://secure.wikimedia.org/w/extensions-1.17/Vector/modules/./images/closed-ltr.png?2011-02-12T21:25:00Z) left center no-repeat!ie;margin-bottom:0px}#mw-panel.collapsible-nav div h5:hover{cursor:pointer;text-decoration:none}#mw-panel.collapsible-nav div.collapsed h5:hover{text-decoration:underline}#mw-panel.collapsible-nav div.portal div.body{background:none !important;padding-top:0px;display:none}#mw-panel.collapsible-nav div.persistent div.body{display:block}#mw-panel.collapsible-nav div.first h5{display:none}#mw-panel.collapsible-nav div.persistent h5{background:none !important;padding-left:0.7em;cursor:default}#mw-panel.collapsible-nav div.portal div.body ul li{padding:0.25em 0}#mw-panel.collapsible-nav div.first{background-image:none;margin-top:0px}#mw-panel.collapsible-nav div.persistent div.body{margin-left:0.5em}</style>\n<meta name=\"ResourceLoaderDynamicStyles\" content=\"\" />\n<link rel=\"stylesheet\" href=\"skins/load.css\" type=\"text/css\" media=\"all\" />\n<style type=\"text/css\" media=\"all\">a.new,#quickbar a.new{color:#ba0000}</style>\n</head>\n<body class=\"mediawiki ltr ns-0 ns-subject skin-vector\">\n        <div id=\"mw-page-base\" class=\"noprint\"></div>\n        <div id=\"mw-head-base\" class=\"noprint\"></div>\n        <!-- content -->\n        <div id=\"content\">\n            <a id=\"top\"></a>\n            <div id=\"mw-js-message\" style=\"display:none;\"></div>\n            <!-- sitenotice -->\n            <div id=\"siteNotice\"><!-- centralNotice loads here -->${ pageBean.notice }</div>\n            <!-- /sitenotice -->\n            <!-- firstHeading -->\n            <h1 id=\"firstHeading\" class=\"firstHeading\">\n          <% if (!pageBean.isEditRestricted()) { %>\n            ${ pageBean.error }${ pageBean.title }\n          <% } else {%>\n            ${ pageBean.error }View source\n          <% } %>\n            </h1>\n            <!-- /firstHeading -->\n            <!-- bodyContent -->\n            <div id=\"bodyContent\">\n                <!-- tagline -->\n                <div id=\"siteSub\">From <%= pageBean.getWikiNamespace().getMeta() %></div>\n                <!-- /tagline -->\n                <!-- subtitle -->\n                <div id=\"contentSub\">\n          <% if (pageBean.isEditRestricted()) { %>\n                for <a href=\"wiki?title=<%= safePageTitle %><%= andServiceUser %>\" title=\"<%= safePageTitle %>\">${ pageBean.title }</a>\n          <% } %>\n                </div>\n                <!-- /subtitle -->\n                <!-- jumpto -->\n                <div id=\"jump-to-nav\">\n                    Jump to: <a href=\"#mw-head\">navigation</a>,\n                    <a href=\"#p-search\">search</a>\n                </div>\n                <!-- /jumpto -->\n                <!-- bodytext -->\n          <% if (pageBean.isEditRestricted()) { %>\n                <p>You do not have permission to edit this page, for the following reason:</p>\n                <div class=\"permissions-errors\">\n                <p>This page has been protected to prevent editing.</p>\n                </div>\n                <p>You can view and copy the source of this page:</p>\n                <textarea id=\"wpTextbox1\" name=\"wpTextbox1\" cols=\"80\" rows=\"25\" readonly=\"readonly\">${ pageBean.page }</textarea>\n                <div class=\"templatesUsed\"></div>\n                <p id=\"mw-returnto\">Return to <a href=\"wiki?title=<%= safePageTitle %><%= andServiceUser %>\" title=\"<%= safePageTitle %>\">${ pageBean.title }</a>.</p>\n          <% } else {%>\n          <% if (pageBean.isNewPage()) { %>\n\t\t\t\t<div class=\"mw-newarticletextanon\">\n\t\t\t\t<p>You have followed a link to a page that does not exist yet.\n\t\t\t\tTo create the page, start typing in the box below (see the <a href=\"wiki?title=Help:Contents<%= andServiceUser %>\" title=\"Help:Contents\">help page</a> for more info).\n\t\t\t\tIf you are here by mistake, click your browser's <b>back</b> button.\n\t\t\t\t</p>\n\t\t\t\t</div>\n          <% }%>\n                <div id=\"mw-anon-edit-warning\">\n          <% if (pageBean.getPreview().isEmpty()) { %>\n\t\t\t\tYou are not <a href=\"wiki?title=Special:UserLogin<%= andServiceUser %>\" title=\"Special:UserLogin\">logged in</a>. Your <a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:What_is_an_IP_address%3F<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:What is an IP address?\">IP address</a> will be recorded in this page's <span class=\"plainlinks\"> <a href=\"wiki?title=<%= safePageTitle %>&amp;action=history<%= andServiceUser %>\" class=\"external text\" rel=\"nofollow\">edit history</a></span>.\n          <% } else { %>\n\t\t\t\t<i>You are not logged in. Saving will record your IP address in this page's edit history.</i>\n          <% }%>\n                </div>\n          <% if (!pageBean.getPreview().isEmpty()) { %>\n\t\t\t\t<div id=\"wikiPreview\" class=\"ontop\">\n\t\t\t\t<div class=\"previewnote\">\n\t\t\t\t<h2 id=\"mw-previewheader\">Preview</h2>\n\t\t\t\t<p><b>Remember that this is only a preview.</b>\n\t\t\t\tYour changes have not yet been saved!\n\t\t\t\t</p><hr>\n\t\t\t\t</div>\n\n${ pageBean.preview }\n\n                </div>\n          <% } else { %>\n\t\t\t\t<div id=\"wikiPreview\" class=\"ontop\" style=\"display: none;\">\n\t\t\t\t</div>\n          <% }%>\n\n\t\t\t\t<div id='toolbar'></div>\n<% \n// not supported (TODO: check if this is still the case):\n//<form id=\"editform\" name=\"editform\" method=\"post\" action=\"wiki?title=safePageTitle&action=submit\" enctype=\"multipart/form-data\">\n%>\n\t\t\t\t<form id=\"editform\" name=\"editform\" method=\"post\" action=\"wiki?title=<%= safePageTitle %>&action=submit\">\n\t\t\t\t<div id=\"antispam-container\" style=\"display: none;\">\n\t\t\t\t<label for=\"wpAntispam\">Anti-spam check.\n\t\t\t\tDo <b>NOT</b> fill this in!</label> <input type=\"text\" name=\"wpAntispam\" id=\"wpAntispam\" value=\"\" />\n\t\t\t\t</div>\n\t            <input type=\"hidden\" value=\"<%= pageBean.getVersion() %>\" name=\"oldVersion\"/>\n                <input type=\"hidden\" value=\"${ pageBean.serviceUser }\" name=\"service_user\"/>\n\t\t\t\t<textarea tabindex=\"1\" accesskey=\",\" id=\"wpTextbox1\" cols=\"80\" rows=\"25\" style=\"\" name=\"wpTextbox1\">${ pageBean.page }</textarea>\n\t\t\t\t<div id=\"editpage-copywarn\">\n\t\t\t\t<p>By saving, you agree to irrevocably release your contribution under the <a href=\"http://creativecommons.org/licenses/by-sa/3.0/\" class=\"external text\" rel=\"nofollow\">Creative Commons Attribution/Share-Alike License 3.0</a> and the <a href=\"http://www.gnu.org/copyleft/fdl.html\" class=\"external text\" rel=\"nofollow\">GFDL</a>.\n\t\t\t\tYou agree to be credited by re-users, at minimum, through a hyperlink or URL to the page you are contributing to.\n\t\t\t\tSee the <a href=\"wiki?title=Terms of Use<%= andServiceUser %>\" rel=\"nofollow\">Terms of Use</a> for details.\n\t\t\t\t</p>\n\t\t\t\t</div>\n\t\t\t\t<div class='editOptions'>\n\t\t\t\t<span class=\"mw-summary\" id=\"wpSummaryLabel\">\n\t\t\t\t<label for=\"wpSummary\"><a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:Edit summary<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:Edit summary\" class=\"mw-redirect\"><span title=\"Briefly describe the changes you have made here\">Edit summary</span></a>\n\t\t\t\t</label>\n\t\t\t\t</span>\n\t\t\t\t<input class=\"mw-summary\" id=\"wpSummary\" maxlength=\"200\" tabindex=\"1\" size=\"60\" title=\"Enter a short summary [b]\" accesskey=\"b\" type=\"text\" value=\"\" name=\"wpSummary\" />\n\t\t\t\t<div class='editCheckboxes'>\n\t\t\t\t</div>\n\t\t\t\t<div class='editButtons'>\n\t\t\t\t<input id=\"wpSave\" name=\"wpSave\" type=\"submit\" tabindex=\"3\" value=\"Save page\" accesskey=\"s\" title=\"Save your changes [s]\" />\n\t\t\t\t<input id=\"wpPreview\" name=\"wpPreview\" type=\"submit\" tabindex=\"4\" value=\"Show preview\" accesskey=\"p\" title=\"Preview your changes, please use this before saving! [p]\" />\n\t\t\t\t<input disabled=\"disabled\" id=\"wpDiff\" name=\"wpDiff\" type=\"submit\" tabindex=\"5\" value=\"Show changes\" accesskey=\"v\" title=\"Show which changes you made to the text [v]\" />\n\t\t\t\t<span class='editHelp'>\n\t\t\t\t<a href=\"wiki?title=<%= safePageTitle %><%= andServiceUser %>\" title=\"<%= safePageTitle %>\" id=\"mw-editform-cancel\">Cancel</a> | \n\t\t\t\t<a target=\"helpwindow\" href=\"wiki?title=Help:Editing<%= andServiceUser %>\">Editing help</a> (opens in new window)</span>\n\t\t\t\t</div><!-- editButtons -->\n\t\t\t\t</div><!-- editOptions -->\n\t\t\t\t<div class=\"mw-tos-summary\"><p>If you do not want your writing to be edited and redistributed at will, then do not submit it here.\n\t\t\t\tIf you did not write this yourself, it must be available under terms consistent with the <a href=\"wiki?title=Terms of Use<%= andServiceUser %>\" rel=\"nofollow\">Terms of Use</a>, and you agree to follow any relevant licensing requirements.\n\t\t\t\t</p></div>\n\t\t\t\t<div class=\"mw-editTools\"></div>\n\t\t\t\t<% if (! (pageBean.getTemplates().isEmpty() && pageBean.getIncludes().isEmpty())) { %>\n\t\t\t\t<div class='templatesUsed'>\n\t\t\t\t<div class=\"mw-templatesUsedExplanation\">\n\t\t\t\t<p>Templates used on this page:</p>\n\t\t\t\t</div>\n\t\t\t\t<ul>\n                <% for (String include : pageBean.getIncludes()) {\n                    String safeInclude = StringEscapeUtils.escapeHtml(URLEncoder.encode(include, \"UTF-8\"));\n                    %>\n                <li><a href=\"wiki?title=<%= safeInclude %>\" title=\"<%= include %>\"><%= include %></a> (<a href=\"wiki?title=<%= safeInclude %>&amp;action=edit\" title=\"<%= include %>\">edit</a>) </li>\n                <% } %>\n\t\t\t\t<% for (String template0 : pageBean.getTemplates()) {\n\t\t\t\t    String template = pageBean.getWikiNamespace().getTemplate() + \":\" + template0;\n\t\t\t\t    String safeTemplate = StringEscapeUtils.escapeHtml(URLEncoder.encode(template, \"UTF-8\"));\n\t\t\t\t    %>\n                <li><a href=\"wiki?title=<%= safeTemplate %>\" title=\"<%= template %>\"><%= template %></a> (<a href=\"wiki?title=<%= safeTemplate %>&amp;action=edit\" title=\"<%= template %>\">edit</a>) </li>\n                <% } %>\n\t\t\t\t</ul>\n\t\t\t\t</div>\n\t\t\t\t<% } %>\n\t\t\t\t<div class='hiddencats'>\n\t\t\t\t</div>\n\t\t\t\t</form>\n          <% } %>\n                <div class=\"printfooter\">Retrieved from \"<a href=\"wiki?title=<%= safePageTitle %><%= andServiceUser %>\">wiki?title=${ pageBean.title }</a>\"</div>\n                <!-- /bodytext -->\n                <!-- catlinks -->\n                <div id='catlinks' class='catlinks catlinks-allhidden'></div>\n                <!-- /catlinks -->\n                <div class=\"visualClear\"></div>\n            </div>\n            <!-- /bodyContent -->\n        </div>\n        <!-- /content -->\n        <!-- header -->\n        <div id=\"mw-head\" class=\"noprint\">\n            \n<!-- 0 -->\n<div id=\"p-personal\" class=\"\">\n    <h5>Personal tools</h5>\n    <ul>\n                    <li id=\"pt-login\"><a href=\"wiki?title=Special:UserLogin&amp;returnto=<%= safePageTitle %>&amp;returntoquery=action%3Dedit<%= andServiceUser %>\" title=\"You are encouraged to log in; however, it is not mandatory [o]\" accesskey=\"o\">Log in / create account</a></li>\n    </ul>\n</div>\n\n<!-- /0 -->\n            <div id=\"left-navigation\">\n                \n<!-- 0 -->\n<div id=\"p-namespaces\" class=\"vectorTabs\">\n    <h5>Namespaces</h5>\n    <ul>\n    <%\n    String mainSelected = pageBean.getWikiNamespace().isTalkPage(pageBean.getTitle()) ? \"\" : \" class=\\\"selected\\\"\";\n    String talkSelected = !pageBean.getWikiNamespace().isTalkPage(pageBean.getTitle()) ? \"\" : \" class=\\\"selected\\\"\";\n    %>\n                    <li id=\"ca-nstab-main\"<%= mainSelected %>><span><a href=\"wiki?title=<%= StringEscapeUtils.escapeHtml(pageBean.getWikiNamespace().getPageNameFromTalkPage(pageBean.getTitle())) %><%= andServiceUser %>\" title=\"View the content page [c]\" accesskey=\"c\">Page</a></span></li>\n                    <li id=\"ca-talk\"<%= talkSelected %>><span><a href=\"wiki?title=<%= StringEscapeUtils.escapeHtml(pageBean.getWikiNamespace().getTalkPageFromPageName(pageBean.getTitle())) %><%= andServiceUser %>\" title=\"Discussion about the content page [t]\" accesskey=\"t\">Talk</a></span></li>\n    </ul>\n</div>\n\n<!-- /0 -->\n\n<!-- 1 -->\n<div id=\"p-variants\" class=\"vectorMenu emptyPortlet\">\n        <h5><span>Variants</span><a href=\"#\"></a></h5>\n    <div class=\"menu\">\n        <ul>\n        </ul>\n    </div>\n</div>\n\n<!-- /1 -->\n            </div>\n            <div id=\"right-navigation\">\n                \n<!-- 0 -->\n<div id=\"p-views\" class=\"vectorTabs\">\n    <h5>Views</h5>\n    <ul>\n          <% if (!pageBean.isNewPage()) { %>\n                    <li id=\"ca-view\"><span><a href=\"wiki?title=<%= safePageTitle %><%= andServiceUser %>\">Read</a></span></li>\n          <% if (!pageBean.isEditRestricted()) { %>\n                    <li id=\"ca-edit\" class=\"selected\"><span><a href=\"wiki?title=<%= safePageTitle %>&amp;action=edit&amp;oldid=${ pageBean.version }<%= andServiceUser %>\" title=\"You can edit this page. Please use the preview button before saving\">Change</a></span></li>\n          <% } else {%>\n                    <li id=\"ca-viewsource\" class=\"selected\"><span><a href=\"wiki?title=<%= safePageTitle %>&amp;action=edit&amp;oldid=${ pageBean.version }<%= andServiceUser %>\" title=\"This page is protected. You can view its source [e]\" accesskey=\"e\">View source</a></span></li>\n          <% } %>\n                    <li id=\"ca-history\" class=\"collapsible \"><span><a href=\"wiki?title=<%= safePageTitle %>&amp;action=history<%= andServiceUser %>\" title=\"Past revisions of this page [h]\" accesskey=\"h\">View history</a></span></li>\n          <% } else {%>\n                    <li id=\"ca-edit\" class=\"selected\"><span><a href=\"wiki?title=<%= safePageTitle %>&amp;action=edit&amp;oldid=${ pageBean.version }<%= andServiceUser %>\" title=\"You can edit this page. Please use the preview button before saving [e]\" accesskey=\"e\">Start</a></span></li>\n          <% } %>\n    </ul>\n</div>\n\n<!-- /0 -->\n\n<!-- 1 -->\n<div id=\"p-cactions\" class=\"vectorMenu emptyPortlet\">\n    <h5><span>Actions</span><a href=\"#\"></a></h5>\n    <div class=\"menu\">\n        <ul>\n        </ul>\n    </div>\n</div>\n\n<!-- /1 -->\n\n<!-- 2 -->\n<div id=\"p-search\">\n    <h5><label for=\"searchInput\">Search</label></h5>\n    <form action=\"wiki?\" id=\"searchform\">\n        <input name=\"title\" value=\"Special:Search\" type=\"hidden\" />\n        <div id=\"simpleSearch\">\n            <input type=\"hidden\" value=\"${ pageBean.serviceUser }\" name=\"service_user\"/>\n            <input autocomplete=\"off\" placeholder=\"Search\" tabindex=\"1\" id=\"searchInput\" name=\"search\" title=\"Search <%= pageBean.getWikiNamespace().getMeta() %> [f]\" accesskey=\"f\" type=\"text\" />\n            <button id=\"searchButton\" type=\"submit\" name=\"button\" title=\"Search the pages for this text\"><img src=\"skins/search-ltr.png\" alt=\"Search\" /></button>\n        </div>\n    </form>\n</div>\n\n<!-- /2 -->\n            </div>\n        </div>\n        <!-- /header -->\n        <!-- panel -->\n            <div id=\"mw-panel\" class=\"noprint collapsible-nav\">\n                <!-- logo -->\n                    <div id=\"p-logo\"><a style=\"background-image: url(&quot;images/Wikipedia.png&quot;);\" href=\"wiki?title=Main Page<%= andServiceUser %>\" title=\"Visit the main page\"></a></div>\n                <!-- /logo -->\n                \n<!-- navigation -->\n<div class=\"portal first persistent\" id=\"p-navigation\">\n    <h5>Links</h5>\n    <div class=\"body\">\n                <ul>\n                    <li id=\"n-mainpage\"><a href=\"wiki?title=Main Page<%= andServiceUser %>\" title=\"Visit the main page [z]\" accesskey=\"z\">Main Page</a></li>\n                    <li id=\"n-recentchanges\"><a href=\"wiki?title=Special:RecentChanges<%= andServiceUser %>\" title=\"The list of recent changes in the wiki [r]\" accesskey=\"r\">New changes</a></li>\n                    <li id=\"n-randompage\"><a href=\"wiki?title=Special:Random<%= andServiceUser %>\" title=\"Load a random page [x]\" accesskey=\"x\">Show any entry</a></li>\n                    <li id=\"n-help\"><a href=\"wiki?title=Help:Contents<%= andServiceUser %>\" title=\"The place to find out\">Help</a></li>\n                </ul>\n            </div>\n</div>\n\n<!-- /navigation -->\n\n<!-- SEARCH -->\n\n<!-- /SEARCH -->\n\n<!-- TOOLBOX -->\n<div class=\"portal expanded\" id=\"p-tb\">\n    <h5 tabindex=\"2\">Toolbox</h5>\n    <div style=\"display: block;\" class=\"body\">\n        <ul>\n          <% if (de.zib.scalaris.examples.wikipedia.Options.getInstance().WIKI_USE_BACKLINKS) { %>\n                    <li id=\"t-whatlinkshere\"><a href=\"wiki?title=Special:WhatLinksHere&target=<%= safePageTitle %><%= andServiceUser %>\" title=\"List of all wiki pages that link here [j]\" accesskey=\"j\">What links here</a></li>\n          <% } %>\n          <% if (!pageBean.isNewPage()) { %>\n<% /*\n                    <li id=\"t-recentchangeslinked\"><a href=\"wiki?title=Special:RecentChangesLinked&target=safePageTitle<%= andServiceUser >\" title=\"Recent changes in pages linked from this page [k]\" accesskey=\"k\">Related changes</a></li>\n*/ %>\n          <% } %>\n                    <li id=\"t-specialpages\"><a href=\"wiki?title=Special:SpecialPages<%= andServiceUser %>\" title=\"List of all special pages [q]\" accesskey=\"q\">Special pages</a></li>\n        </ul>\n    </div>\n</div>\n\n<!-- /TOOLBOX -->\n\n<!-- LANGUAGES -->\n\n<!-- /LANGUAGES -->\n<!-- RENDERER -->\n\n<!-- /RENDERER -->\n            </div>\n        <!-- /panel -->\n        <!-- footer -->\n        <div id=\"footer\">\n                <ul id=\"footer-places\">\n                    <li id=\"footer-places-privacy\"><a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:Privacy policy<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:Privacy policy\">Privacy policy</a></li>\n                    <li id=\"footer-places-about\"><a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:About<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:About\">About <%= pageBean.getWikiNamespace().getMeta() %></a></li>\n                    <li id=\"footer-places-disclaimer\"><a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:General disclaimer<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:General disclaimer\">Disclaimers</a></li>\n                </ul>\n                <ul id=\"footer-icons\" class=\"noprint\">\n                </ul>\n                <div style=\"clear:both\"></div>\n                <div style=\"font-size:0.7em\">DB Timings:\n                <pre id=\"db_timings\" style=\"padding:0;line-height:0.7em\">\n<% for (Map.Entry<String,List<Long>> stats : pageBean.getStats().entrySet()) { %>\n<%= stats.getKey() %>: <%= stats.getValue().toString() %>\n<% } %>\n                </pre></div>\n                <div style=\"font-size:0.7em\">Involved DB keys:\n                <pre id=\"involved_keys\" style=\"padding:0;line-height:0.7em\">\n<% for (InvolvedKey involvedKey : pageBean.getInvolvedKeys()) { %>\n<%= involvedKey.toString() %>\n<% } %>\n                </pre></div>\n                <div style=\"font-size:0.7em\">Page save attempts: <span id=\"save_attempts\">${ pageBean.saveAttempts }</span></div>\n                <div style=\"font-size:0.7em\">Failed keys when saving page:\n                <pre id=\"save_failed_keys\" style=\"padding:0;line-height:0.7em\">\n<% for (Map.Entry<Integer,List<String>> failedKeys : pageBean.getFailedKeys().entrySet()) { %>\n<%= failedKeys.getKey() %>: <%= StringUtils.join(failedKeys.getValue(), \" # \") %>\n<% } %>\n                </pre></div>\n<% long renderTime = (System.currentTimeMillis() - pageBean.getStartTime()); %>\n                <div style=\"clear:both\"></div>\n                <div style=\"font-size:0.7em\">More timings:\n                <pre id=\"other_timings\" style=\"padding:0;line-height:0.7em\">\n\nserver: <%= renderTime %>\n\n                </pre></div>\n        </div>\n        <!-- /footer -->\n<% servlet.storeUserReq(pageBean, renderTime); %>\n</body>\n</html>\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/pageHistory.jsp",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\" ?>\n<%@page import=\"de.zib.scalaris.examples.wikipedia.bliki.WikiServlet\"%>\n<%@page import=\"de.zib.scalaris.examples.wikipedia.InvolvedKey\"%>\n<%@page language=\"java\" contentType=\"text/html; charset=UTF-8\" pageEncoding=\"UTF-8\"\n import=\"java.util.Calendar,java.util.Locale,java.text.DateFormat,java.text.SimpleDateFormat,java.util.TimeZone,java.util.Iterator,de.zib.scalaris.examples.wikipedia.data.Revision,de.zib.scalaris.examples.wikipedia.data.ShortRevision,org.apache.commons.lang.StringEscapeUtils,java.util.Map,java.util.List,java.net.URLEncoder\"%>\n<% String req_render = request.getParameter(\"render\"); %>\n<jsp:useBean id=\"pageBean\" type=\"de.zib.scalaris.examples.wikipedia.bliki.WikiPageBean\" scope=\"request\" />\n<jsp:useBean id=\"servlet\" type=\"de.zib.scalaris.examples.wikipedia.WikiServletContext\" scope=\"request\" />\n<% /* created page based on https://secure.wikimedia.org/wiktionary/simple/wiki/relief */ %>\n<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n<html lang=\"${ pageBean.wikiLang }\" dir=\"${ pageBean.wikiLangDir }\" xmlns=\"http://www.w3.org/1999/xhtml\">\n<head>\n<% final String safePageTitle = StringEscapeUtils.escapeHtml(URLEncoder.encode(pageBean.getTitle(), \"UTF-8\")); %>\n<% final String andServiceUser = pageBean.getServiceUser().isEmpty() ? \"\" : \"&amp;service_user=\" + pageBean.getServiceUser(); %>\n<title>Revision history of \"${ pageBean.title }\" - ${ pageBean.wikiTitle }</title>\n<!--<% if (!pageBean.getError().isEmpty()) { %>\n<error>${ pageBean.error }</error>\n<% } %>-->\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" />\n<meta http-equiv=\"Content-Style-Type\" content=\"text/css\" />\n<meta name=\"robots\" content=\"noindex,nofollow\" />\n<link rel=\"alternate\" type=\"application/x-wiki\" title=\"change this page\" href=\"wiki?title=<%= safePageTitle %>&amp;action=edit<%= andServiceUser %>\">\n<link rel=\"edit\" title=\"change this page\" href=\"wiki?title=<%= safePageTitle %>&amp;action=edit<%= andServiceUser %>\">\n<% /*\n<meta name=\"generator\" content=\"MediaWiki 1.17wmf1\" />\n<link rel=\"apple-touch-icon\" href=\"http://simple.wiktionary.org/apple-touch-icon.png\">\n*/ %>\n<link rel=\"shortcut icon\" href=\"favicon-wikipedia.ico\" />\n<% /*\n<link rel=\"search\" type=\"application/opensearchdescription+xml\" href=\"https://secure.wikimedia.org/wiktionary/simple/w/opensearch_desc.php\" title=\"Wiktionary (simple)\">\n<link rel=\"EditURI\" type=\"application/rsd+xml\" href=\"https://secure.wikimedia.org/wiktionary/simple/w/api.php?action=rsd\">\n*/ %>\n<link rel=\"copyright\" href=\"http://creativecommons.org/licenses/by-sa/3.0/\" />\n<% /*\n<link rel=\"alternate\" type=\"application/atom+xml\" title=\"&quot;safePageTitle&quot; Atom feed\" href=\"wiki?title=safePageTitle&amp;feed=atom&amp;action=history<%= andServiceUser >\" />\n<link rel=\"alternate\" type=\"application/atom+xml\" title=\"Wiktionary Atom feed\" href=\"wiki?title=Special:RecentChanges&amp;feed=atom<%= andServiceUser >\">\n*/ %>\n<link rel=\"stylesheet\" href=\"skins/load_002.css\" type=\"text/css\" media=\"all\" />\n<style type=\"text/css\" media=\"all\">.suggestions{overflow:hidden;position:absolute;top:0px;left:0px;width:0px;border:none;z-index:99;padding:0;margin:-1px -1px 0 0} html > body .suggestions{margin:-1px 0 0 0}.suggestions-special{position:relative;background-color:Window;font-size:0.8em;cursor:pointer;border:solid 1px #aaaaaa;padding:0;margin:0;margin-top:-2px;display:none;padding:0.25em 0.25em;line-height:1.25em}.suggestions-results{background-color:white;background-color:Window;font-size:0.8em;cursor:pointer;border:solid 1px #aaaaaa;padding:0;margin:0}.suggestions-result{color:black;color:WindowText;margin:0;line-height:1.5em;padding:0.01em 0.25em;text-align:left}.suggestions-result-current{background-color:#4C59A6;background-color:Highlight;color:white;color:HighlightText}.suggestions-special .special-label{font-size:0.8em;color:gray;text-align:left}.suggestions-special .special-query{color:black;font-style:italic;text-align:left}.suggestions-special .special-hover{background-color:silver}.suggestions-result-current .special-label,.suggestions-result-current .special-query{color:white;color:HighlightText}.autoellipsis-matched,.highlight{font-weight:bold}</style>\n<style type=\"text/css\" media=\"all\">#mw-panel.collapsible-nav div.portal{background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAIwAAAABCAMAAAA7MLYKAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAEtQTFRF29vb2tra4ODg6urq5OTk4uLi6+vr7e3t7Ozs8PDw5+fn4+Pj4eHh3d3d39/f6Ojo5eXl6enp8fHx8/Pz8vLy7+/v3Nzc2dnZ2NjYnErj7QAAAD1JREFUeNq0wQUBACAMALDj7hf6JyUFGxzEnYhC9GaNPG1xVffGDErk/iCigLl1XV2xM49lfAxEaSM+AQYA9HMKuv4liFQAAAAASUVORK5CYII=);background-image:url(https://secure.wikimedia.org/w/extensions-1.17/Vector/modules/./images/portal-break.png?2011-02-12T21:25:00Z)!ie;background-position:left top;background-repeat:no-repeat;padding:0.25em 0 !important;margin:-11px 9px 10px 11px}#mw-panel.collapsible-nav div.portal h5{color:#4D4D4D;font-weight:normal;background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQBAMAAADt3eJSAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAA9QTFRFeXl53d3dmpqasbGx////GU0iEgAAAAV0Uk5T/////wD7tg5TAAAAK0lEQVQI12NwgQIG0hhCDAwMTCJAhqMCA4MiWEoIJABiOCooQhULi5BqMgB2bh4svs8t+QAAAABJRU5ErkJggg==) left center no-repeat;background:url(https://secure.wikimedia.org/w/extensions-1.17/Vector/modules/./images/open.png?2011-02-12T21:25:00Z) left center no-repeat!ie;padding:4px 0 3px 1.5em;margin-bottom:0px}#mw-panel.collapsible-nav div.collapsed h5{color:#0645AD;background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAMAAAAoLQ9TAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAAxQTFRF3d3deXl5////nZ2dQA6SoAAAAAN0Uk5T//8A18oNQQAAADNJREFUeNpiYEIDDMQKMKALMDOgCTDCRWACcBG4AEwEIcDITEAFuhnotmC4g4EEzwEEGAADqgHmQSPJKgAAAABJRU5ErkJggg==) left center no-repeat;background:url(https://secure.wikimedia.org/w/extensions-1.17/Vector/modules/./images/closed-ltr.png?2011-02-12T21:25:00Z) left center no-repeat!ie;margin-bottom:0px}#mw-panel.collapsible-nav div h5:hover{cursor:pointer;text-decoration:none}#mw-panel.collapsible-nav div.collapsed h5:hover{text-decoration:underline}#mw-panel.collapsible-nav div.portal div.body{background:none !important;padding-top:0px;display:none}#mw-panel.collapsible-nav div.persistent div.body{display:block}#mw-panel.collapsible-nav div.first h5{display:none}#mw-panel.collapsible-nav div.persistent h5{background:none !important;padding-left:0.7em;cursor:default}#mw-panel.collapsible-nav div.portal div.body ul li{padding:0.25em 0}#mw-panel.collapsible-nav div.first{background-image:none;margin-top:0px}#mw-panel.collapsible-nav div.persistent div.body{margin-left:0.5em}</style>\n<meta name=\"ResourceLoaderDynamicStyles\" content=\"\" />\n<link rel=\"stylesheet\" href=\"skins/load.css\" type=\"text/css\" media=\"all\" />\n<style type=\"text/css\" media=\"all\">a.new,#quickbar a.new{color:#ba0000}</style>\n</head>\n<body class=\"mediawiki ltr ns-0 ns-subject skin-vector\">\n        <div id=\"mw-page-base\" class=\"noprint\"></div>\n        <div id=\"mw-head-base\" class=\"noprint\"></div>\n        <!-- content -->\n        <div id=\"content\">\n            <a id=\"top\"></a>\n            <div id=\"mw-js-message\" style=\"display:none;\"></div>\n            <!-- sitenotice -->\n            <div id=\"siteNotice\"><!-- centralNotice loads here -->${ pageBean.notice }</div>\n            <!-- /sitenotice -->\n            <!-- firstHeading -->\n            <h1 id=\"firstHeading\" class=\"firstHeading\">${ pageBean.error }Revision history of \"${ pageBean.title }\"</h1>\n            <!-- /firstHeading -->\n            <!-- bodyContent -->\n            <div id=\"bodyContent\">\n                <!-- tagline -->\n                <div id=\"siteSub\">From <%= pageBean.getWikiNamespace().getMeta() %></div>\n                <!-- /tagline -->\n                <!-- subtitle -->\n                <div id=\"contentSub\">\n                <!-- <a href=\"wiki?title=Special:Log&amp;page=<%= safePageTitle %><%= andServiceUser %>\" title=\"Special:Log\">View logs for this page</a> -->\n                </div>\n                <!-- /subtitle -->\n                <!-- jumpto -->\n                <div id=\"jump-to-nav\">\n                    Jump to: <a href=\"#mw-head\">navigation</a>,\n                    <a href=\"#p-search\">search</a>\n                </div>\n                <!-- /jumpto -->\n                <!-- bodytext -->\n            <% if (!pageBean.isNotAvailable()) { %>\n                <form action=\"/wiktionary/simple/w/index.php\" method=\"get\" id=\"mw-history-searchform\"><fieldset id=\"mw-history-search\">\n<legend>Browse history</legend>\n<input type=\"hidden\" value=\"<%= safePageTitle %>\" name=\"title\" />\n<input type=\"hidden\" value=\"history\" name=\"action\" />\n<label for=\"year\">From year (and earlier):</label>\n<input disabled=\"disabled\" name=\"year\" size=\"4\" value=\"\" id=\"year\" maxlength=\"4\" />\n<label for=\"month\">From month (and earlier):</label>\n<select disabled=\"disabled\" id=\"month\" name=\"month\" class=\"mw-month-selector\">\n    <option value=\"-1\">all</option>\n\t<option value=\"1\">January</option>\n\t<option value=\"2\">February</option>\n\t<option value=\"3\">March</option>\n\t<option value=\"4\">April</option>\n\t<option value=\"5\">May</option>\n\t<option value=\"6\">June</option>\n\t<option value=\"7\">July</option>\n\t<option value=\"8\">August</option>\n\t<option value=\"9\">September</option>\n\t<option value=\"10\">October</option>\n\t<option value=\"11\">November</option>\n\t<option value=\"12\">December</option>\n</select>&#160;<input disabled=\"disabled\" name=\"deleted\" type=\"checkbox\" value=\"1\" id=\"mw-show-deleted-only\" />&#160;<label for=\"mw-show-deleted-only\">Deleted only</label>\n<input disabled=\"disabled\" type=\"submit\" value=\"Go\" />\n</fieldset></form>\n<div class=\"mw-history-legend\">\n<p>Diff selection: mark the radio boxes of the revisions to compare and hit enter or the button at the bottom.<br/>\nLegend: <b>(cur)</b> = difference with latest revision, <b>(prev)</b> = difference with preceding revision, <b>m</b> = minor edit.\n</p>\n</div>\n(Latest | Earliest) View (newer 50  |  older 50) (20 | 50 | 100 | 250 | 500)\n<form action=\"/wiktionary/simple/w/index.php\" id=\"mw-history-compare\">\n<input type=\"hidden\" value=\"relief\" name=\"title\" />\n<input type=\"hidden\" value=\"historysubmit\" name=\"action\" />\n<div><input disabled=\"disabled\" type=\"submit\" value=\"Compare selected revisions\" class=\"historysubmit\" />\n</div>\n<ul id=\"pagehistory\">\n\n<%\nif (req_render == null || !req_render.equals(\"0\")) {\n\tDateFormat dfm = new SimpleDateFormat(\"HH:mm, d MMMMM yyyy\");\n\tdfm.setTimeZone(TimeZone.getTimeZone(\"GMT\")); // time presented by Wikipedia is UTC/GMT, not the local time of the user viewing the page\n\tfor (ShortRevision rev : pageBean.getRevisions()) {\n\t    \n\t    // IP:\n\t    /* <a href=\"/wiktionary/simple/wiki/Special:Contributions/70.254.248.130\" title=\"Special:Contributions/70.254.248.130\" class=\"mw-userlink\">70.254.248.130</a>\n\t        <span class=\"mw-usertoollinks\">(<a href=\"/wiktionary/simple/w/index.php?title=User_talk:70.254.248.130&amp;action=edit&amp;redlink=1\" class=\"new\" title=\"User talk:70.254.248.130 (page does not exist)\">talk</a>)</span>\n\t    // username:\n\t    /* <a href=\"/wiktionary/simple/wiki/User:Interwicket\" title=\"User:Interwicket\" class=\"mw-userlink\">Interwicket</a>\n\t        <span class=\"mw-usertoollinks\">(<a href=\"/wiktionary/simple/wiki/User_talk:Interwicket\" title=\"User talk:Interwicket\">talk</a> |\n\t                                        <a href=\"/wiktionary/simple/wiki/Special:Contributions/Interwicket\" title=\"Special:Contributions/Interwicket\">changes</a>)\n\t        </span>\n\t    */\n\t    String usertools;\n\t    if (rev.getContributor().getIp().isEmpty() || !rev.getContributor().getUser().isEmpty()) {\n\t        String username = rev.getContributor().getUser();\n\t        usertools = \"<a href=\\\"wiki?title=User:\" + username + andServiceUser + \"\\\" title=\\\"User:\" + username + \"\\\" class=\\\"mw-userlink\\\">\" + username + \"</a> \" +\n\t                \"<span class=\\\"mw-usertoollinks\\\">(<a href=\\\"wiki?title=\" + pageBean.getWikiNamespace().getUser_talk() + \":\" + username + andServiceUser + \"\\\" title=\\\"User talk:\" + username + \"\\\">talk</a> | \" +\n\t                \"<a href=\\\"wiki?title=Special:Contributions/\" + username + andServiceUser + \"\\\" title=\\\"Special:Contributions/\" + username + \"\\\">changes</a>)</span>\";\n\t    } else {\n\t        String ip = rev.getContributor().getIp();\n\t        usertools = \"<a href=\\\"wiki?title=Special:Contributions/\" + ip + \"\\\" title=\\\"Special:Contributions/\" + ip + andServiceUser + \"\\\" class=\\\"mw-userlink\\\">\" + ip + \"</a> \" +\n\t                \"<span class=\\\"mw-usertoollinks\\\">(<a href=\\\"wiki?title=\" + pageBean.getWikiNamespace().getUser_talk() + \":\" + ip + andServiceUser + \"\\\" title=\\\"User talk:\" + ip + \"\\\" class=\\\"mw-usertoollinks\\\">talk</a>)</span>\";\n\t    }\n\t    \n\t    out.print(\"<li><span class=\\\"mw-history-histlinks\\\">(cur | prev) </span> \");\n\t    out.print(\"<a href=\\\"wiki?title=\" + safePageTitle + \"&amp;oldid=\" + rev.getId() + andServiceUser + \"\\\" title=\\\"\" + safePageTitle + \"\\\">\" + dfm.format(Revision.stringToCalendar(rev.getTimestamp()).getTime()) + \"</a> \");\n\t    out.print(\"<span class=\\\"history-user\\\">\" + usertools + \"</span> \");\n\t    if (rev.isMinor()) {\n\t        out.print(\"<abbr class=\\\"minor\\\" title=\\\"This is a minor edit\\\">m</abbr> \");\n\t    }\n\t    out.print(\"<span class=\\\"history-size\\\">(3,205 bytes)</span> \");\n\t    if (!rev.getComment().isEmpty()) {\n\t        // TODO: parse comment as wiki text\n\t        out.print(\"<span class=\\\"comment\\\">(\" + rev.getComment() + \")</span> \");\n\t    }\n\t    out.print(\"(<span class=\\\"mw-history-undo\\\">undo</span>)\");\n\t    out.println(\"</li>\");\n\t}\n} else {\n    for (ShortRevision rev : pageBean.getRevisions()) {\n        out.println(\"<pre>\"\n                + StringEscapeUtils.escapeHtml(rev.getTimestamp())\n                + \" (\" + StringEscapeUtils.escapeHtml(rev.getContributor().toString()) + \"),\"\n                + \" minor: \" + rev.isMinor() + \",\"\n                + \" size: \" + rev.getSize() + \"\\n\"\n                + StringEscapeUtils.escapeHtml(rev.getComment())+ \"</pre>\");\n    }\n}\n%>\n\n${ pageBean.page }\n\n</ul>\n<div><input disabled=\"disabled\" type=\"submit\" value=\"Compare selected revisions\" class=\"historysubmit\" title=\"See the differences between the two selected revisions of this page [v]\" accesskey=\"v\" />\n</div></form>\n(Latest | Earliest) View (newer 50  |  older 50) (20 | 50 | 100 | 250 | 500)\n            <% } else { %>\n            <p>There is no edit history for this page.</p> \n            <% } %>\n<div class=\"printfooter\">\nRetrieved from \"<a href=\"wiki?title=<%= safePageTitle %><%= andServiceUser %>\">wiki?title=${ pageBean.title }</a>\"</div>\n                <!-- /bodytext -->\n                <!-- catlinks -->\n                <div id='catlinks' class='catlinks catlinks-allhidden'></div>\n                <!-- /catlinks -->\n                <div class=\"visualClear\"></div>\n            </div>\n            <!-- /bodyContent -->\n        </div>\n        <!-- /content -->\n        <!-- header -->\n        <div id=\"mw-head\" class=\"noprint\">\n            \n<!-- 0 -->\n<div id=\"p-personal\" class=\"\">\n    <h5>Personal tools</h5>\n    <ul>\n                    <li id=\"pt-login\"><a href=\"wiki?title=Special:UserLogin&amp;returnto=<%= safePageTitle %><%= andServiceUser %>\" title=\"You are encouraged to log in; however, it is not mandatory [o]\" accesskey=\"o\">Log in / create account</a></li>\n    </ul>\n</div>\n\n<!-- /0 -->\n            <div id=\"left-navigation\">\n                \n<!-- 0 -->\n<div id=\"p-namespaces\" class=\"vectorTabs\">\n    <h5>Namespaces</h5>\n    <ul>\n    <%\n    String mainSelected = pageBean.getWikiNamespace().isTalkPage(pageBean.getTitle()) ? \"\" : \" class=\\\"selected\\\"\";\n    String talkSelected = !pageBean.getWikiNamespace().isTalkPage(pageBean.getTitle()) ? \"\" : \" class=\\\"selected\\\"\";\n    %>\n                    <li id=\"ca-nstab-main\"<%= mainSelected %>><span><a href=\"wiki?title=<%= StringEscapeUtils.escapeHtml(pageBean.getWikiNamespace().getPageNameFromTalkPage(pageBean.getTitle())) %><%= andServiceUser %>\" title=\"View the content page [c]\" accesskey=\"c\">Page</a></span></li>\n                    <li id=\"ca-talk\"<%= talkSelected %>><span><a href=\"wiki?title=<%= StringEscapeUtils.escapeHtml(pageBean.getWikiNamespace().getTalkPageFromPageName(pageBean.getTitle())) %><%= andServiceUser %>\" title=\"Discussion about the content page [t]\" accesskey=\"t\">Talk</a></span></li>\n    </ul>\n</div>\n\n<!-- /0 -->\n\n<!-- 1 -->\n<div id=\"p-variants\" class=\"vectorMenu emptyPortlet\">\n        <h5><span>Variants</span><a href=\"#\"></a></h5>\n    <div class=\"menu\">\n        <ul>\n        </ul>\n    </div>\n</div>\n\n<!-- /1 -->\n            </div>\n            <div id=\"right-navigation\">\n                \n<!-- 0 -->\n<div id=\"p-views\" class=\"vectorTabs\">\n    <h5>Views</h5>\n    <ul>\n          <% if (!pageBean.isNotAvailable()) { %>\n                    <li id=\"ca-view\"><span><a href=\"wiki?title=<%= safePageTitle %><%= andServiceUser %>\">Read</a></span></li>\n          <% if (!pageBean.isEditRestricted()) { %>\n                    <li id=\"ca-edit\"><span><a href=\"wiki?title=<%= safePageTitle %>&amp;action=edit&amp;oldid=${ pageBean.version }<%= andServiceUser %>\" title=\"You can edit this page. Please use the preview button before saving [e]\" accesskey=\"e\">Change</a></span></li>\n          <% } else {%>\n                    <li id=\"ca-viewsource\"><span><a href=\"wiki?title=<%= safePageTitle %>&amp;action=edit&amp;oldid=${ pageBean.version }<%= andServiceUser %>\" title=\"This page is protected. You can view its source [e]\" accesskey=\"e\">View source</a></span></li>\n          <% } %>\n                    <li id=\"ca-history\" class=\"collapsible selected\"><span><a href=\"wiki?title=<%= safePageTitle %>&amp;action=history<%= andServiceUser %>\" title=\"Past revisions of this page [h]\" accesskey=\"h\">View history</a></span></li>\n          <% } else {%>\n                    <li id=\"ca-edit\"><span><a href=\"wiki?title=<%= safePageTitle %>&amp;action=edit&amp;oldid=${ pageBean.version }<%= andServiceUser %>\" title=\"You can edit this page. Please use the preview button before saving [e]\" accesskey=\"e\">Start</a></span></li>\n          <% } %>\n    </ul>\n</div>\n\n<!-- /0 -->\n\n<!-- 1 -->\n<div id=\"p-cactions\" class=\"vectorMenu emptyPortlet\">\n    <h5><span>Actions</span><a href=\"#\"></a></h5>\n    <div class=\"menu\">\n        <ul>\n        </ul>\n    </div>\n</div>\n\n<!-- /1 -->\n\n<!-- 2 -->\n<div id=\"p-search\">\n    <h5><label for=\"searchInput\">Search</label></h5>\n    <form action=\"wiki?\" id=\"searchform\">\n        <input name=\"title\" value=\"Special:Search\" type=\"hidden\" />\n        <div id=\"simpleSearch\">\n            <input type=\"hidden\" value=\"${ pageBean.serviceUser }\" name=\"service_user\"/>\n            <input autocomplete=\"off\" placeholder=\"Search\" tabindex=\"1\" id=\"searchInput\" name=\"search\" title=\"Search <%= pageBean.getWikiNamespace().getMeta() %> [f]\" accesskey=\"f\" type=\"text\" />\n            <button id=\"searchButton\" type=\"submit\" name=\"button\" title=\"Search the pages for this text\"><img src=\"skins/search-ltr.png\" alt=\"Search\" /></button>\n        </div>\n    </form>\n</div>\n\n<!-- /2 -->\n            </div>\n        </div>\n        <!-- /header -->\n        <!-- panel -->\n            <div id=\"mw-panel\" class=\"noprint collapsible-nav\">\n                <!-- logo -->\n                    <div id=\"p-logo\"><a style=\"background-image: url(&quot;images/Wikipedia.png&quot;);\" href=\"wiki?title=Main Page<%= andServiceUser %>\" title=\"Visit the main page\"></a></div>\n                <!-- /logo -->\n                \n<!-- navigation -->\n<div class=\"portal first persistent\" id=\"p-navigation\">\n    <h5>Links</h5>\n    <div class=\"body\">\n                <ul>\n                    <li id=\"n-mainpage\"><a href=\"wiki?title=Main Page<%= andServiceUser %>\" title=\"Visit the main page [z]\" accesskey=\"z\">Main Page</a></li>\n                    <li id=\"n-recentchanges\"><a href=\"wiki?title=Special:RecentChanges<%= andServiceUser %>\" title=\"The list of recent changes in the wiki [r]\" accesskey=\"r\">New changes</a></li>\n                    <li id=\"n-randompage\"><a href=\"wiki?title=Special:Random<%= andServiceUser %>\" title=\"Load a random page [x]\" accesskey=\"x\">Show any entry</a></li>\n                    <li id=\"n-help\"><a href=\"wiki?title=Help:Contents<%= andServiceUser %>\" title=\"The place to find out\">Help</a></li>\n                </ul>\n            </div>\n</div>\n\n<!-- /navigation -->\n\n<!-- SEARCH -->\n\n<!-- /SEARCH -->\n\n<!-- TOOLBOX -->\n<div class=\"portal expanded\" id=\"p-tb\">\n    <h5 tabindex=\"2\">Toolbox</h5>\n    <div style=\"display: block;\" class=\"body\">\n        <ul>\n          <% if (de.zib.scalaris.examples.wikipedia.Options.getInstance().WIKI_USE_BACKLINKS) { %>\n                    <li id=\"t-whatlinkshere\"><a href=\"wiki?title=Special:WhatLinksHere&target=<%= safePageTitle %><%= andServiceUser %>\" title=\"List of all wiki pages that link here [j]\" accesskey=\"j\">What links here</a></li>\n          <% } %>\n          <% if (!pageBean.isNotAvailable()) { %>\n<% /*\n                    <li id=\"t-recentchangeslinked\"><a href=\"wiki?title=Special:RecentChangesLinked&target=safePageTitle<%= andServiceUser >\" title=\"Recent changes in pages linked from this page [k]\" accesskey=\"k\">Related changes</a></li>\n*/ %>\n          <% } %>\n<% /*\n                    <li id=\"feedlinks\"><a id=\"feed-atom\" href=\"wiki?title=safePageTitle&amp;feed=atom&amp;action=history<%= andServiceUser >\" rel=\"alternate\" type=\"application/atom+xml\" class=\"feedlink\" title=\"Atom feed for this page\">Atom</a></li>\n*/ %>\n                    <li id=\"t-specialpages\"><a href=\"wiki?title=Special:SpecialPages<%= andServiceUser %>\" title=\"List of all special pages [q]\" accesskey=\"q\">Special pages</a></li>\n        </ul>\n    </div>\n</div>\n\n<!-- /TOOLBOX -->\n\n<!-- LANGUAGES -->\n\n<!-- /LANGUAGES -->\n<!-- RENDERER -->\n<div class=\"portal expanded\" id=\"p-renderer\">\n    <h5 tabindex=\"2\">Renderer</h5>\n    <div style=\"display: block;\" class=\"body\">\n        <ul>\n                    <li id=\"t-renderer-default\">\n            <% if (req_render == null || req_render.equals(\"1\")) { %>\n                        Default\n            <% } else { %>\n                        <a href=\"wiki?title=<%= safePageTitle %>&action=history&render=1<%= andServiceUser %>\" title=\"Default renderer (gwtwiki)\">Default</a></li>\n            <% } %>\n                    </li>\n                    <li id=\"t-renderer-none\">\n            <% if (req_render != null && req_render.equals(\"0\")) { %>\n                        Plain\n            <% } else { %>\n                        <a href=\"wiki?title=<%= safePageTitle %>&action=history&render=0<%= andServiceUser %>\" title=\"No renderer (plain wiki text)\">Plain</a></li>\n            <% } %>\n                    </li>\n        </ul>\n    </div>\n</div>\n\n\n<!-- /RENDERER -->\n            </div>\n        <!-- /panel -->\n        <!-- footer -->\n        <div id=\"footer\">\n                <ul id=\"footer-info\">\n        <% if (!pageBean.isNotAvailable()) {\n          DateFormat dfm = new SimpleDateFormat(\"d MMMMM yyyy', at 'HH:mm\");\n          dfm.setTimeZone(TimeZone.getTimeZone(\"GMT\")); // time presented by Wikipedia is UTC/GMT, not the local time of the user viewing the page\n          %>\n                    <li id=\"footer-info-lastmod\"> This page was last modified on <%= dfm.format(pageBean.getDate().getTime()) %>.</li>\n                    <li id=\"footer-info-copyright\">\n                        Text is available under the <a href=\"http://creativecommons.org/licenses/by-sa/3.0/\">Creative Commons Attribution/Share-Alike License</a>;\n                        additional terms may apply.\n                        See <a href=\"wiki?title=Terms of Use<%= andServiceUser %>\">Terms of Use</a> for details.</li>\n        <% } %>\n                </ul>\n                <ul id=\"footer-places\">\n                    <li id=\"footer-places-privacy\"><a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:Privacy policy<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:Privacy policy\">Privacy policy</a></li>\n                    <li id=\"footer-places-about\"><a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:About<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:About\">About <%= pageBean.getWikiNamespace().getMeta() %></a></li>\n                    <li id=\"footer-places-disclaimer\"><a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:General disclaimer<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:General disclaimer\">Disclaimers</a></li>\n                </ul>\n                <ul id=\"footer-icons\" class=\"noprint\">\n                </ul>\n                <div style=\"clear:both\"></div>\n                <div style=\"font-size:0.7em\">DB Timings:\n                <pre id=\"db_timings\" style=\"padding:0;line-height:0.7em\">\n<% for (Map.Entry<String,List<Long>> stats : pageBean.getStats().entrySet()) { %>\n<%= stats.getKey() %>: <%= stats.getValue().toString() %>\n<% } %>\n                </pre></div>\n                <div style=\"font-size:0.7em\">Involved DB keys:\n                <pre id=\"involved_keys\" style=\"padding:0;line-height:0.7em\">\n<% for (InvolvedKey involvedKey : pageBean.getInvolvedKeys()) { %>\n<%= involvedKey.toString() %>\n<% } %>\n                </pre></div>\n<% long renderTime = (System.currentTimeMillis() - pageBean.getStartTime()); %>\n                <div style=\"clear:both\"></div>\n                <div style=\"font-size:0.7em\">More timings:\n                <pre id=\"other_timings\" style=\"padding:0;line-height:0.7em\">\n\nserver: <%= renderTime %>\n\n                </pre></div>\n        </div>\n        <!-- /footer -->\n<% servlet.storeUserReq(pageBean, renderTime); %>\n</body>\n</html>\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/pageSpecial_pagelist.jsp",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\" ?>\n<%@page import=\"de.zib.scalaris.examples.wikipedia.bliki.WikiServlet\"%>\n<%@page import=\"de.zib.scalaris.examples.wikipedia.InvolvedKey\"%>\n<%@page language=\"java\" contentType=\"text/html; charset=UTF-8\" pageEncoding=\"UTF-8\"\n import=\"java.util.Calendar,java.util.Locale,java.text.DateFormat,java.text.SimpleDateFormat,java.util.TimeZone,java.util.Iterator,de.zib.scalaris.examples.wikipedia.bliki.WikiPageListBean,java.util.Map,java.util.List,org.apache.commons.lang.StringEscapeUtils,java.net.URLEncoder\"%>\n<% String req_render = request.getParameter(\"render\"); %>\n<jsp:useBean id=\"pageBean\" type=\"de.zib.scalaris.examples.wikipedia.bliki.WikiPageListBean\" scope=\"request\" />\n<jsp:useBean id=\"servlet\" type=\"de.zib.scalaris.examples.wikipedia.WikiServletContext\" scope=\"request\" />\n<% /* created page based on https://secure.wikimedia.org/wiktionary/simple/wiki/relief */ %>\n<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\n<html lang=\"${ pageBean.wikiLang }\" dir=\"${ pageBean.wikiLangDir }\" xmlns=\"http://www.w3.org/1999/xhtml\">\n<head>\n<%\nfinal String safePageTitle = StringEscapeUtils.escapeHtml(URLEncoder.encode(pageBean.getTitle(), \"UTF-8\"));\nfinal String pageTitleWithPars = pageBean.titleWithParameters();\nfinal String safePageTitleWithPars = StringEscapeUtils.escapeHtml(pageTitleWithPars);\n\nfinal String safeFromTitle = StringEscapeUtils.escapeHtml(pageBean.getFromPage());\nfinal String safeToTitle = StringEscapeUtils.escapeHtml(pageBean.getToPage());\nfinal String safeTargetTitle = StringEscapeUtils.escapeHtml(pageBean.getTarget());\nfinal String safePrefixTitle = StringEscapeUtils.escapeHtml(pageBean.getPrefix());\nfinal String safeSearchTitle = StringEscapeUtils.escapeHtml(pageBean.getSearch());\nfinal String andServiceUser = pageBean.getServiceUser().isEmpty() ? \"\" : \"&amp;service_user=\" + pageBean.getServiceUser();\n%>\n<title>${ pageBean.pageHeading } - ${ pageBean.wikiTitle }</title>\n<!--<% if (!pageBean.getError().isEmpty()) { %>\n<error>${ pageBean.error }</error>\n<% } %>-->\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" />\n<meta http-equiv=\"Content-Style-Type\" content=\"text/css\" />\n<% /* \n<meta name=\"generator\" content=\"MediaWiki 1.17wmf1\" />\n<link rel=\"apple-touch-icon\" href=\"http://simple.wiktionary.org/apple-touch-icon.png\">\n*/ %>\n<link rel=\"shortcut icon\" href=\"favicon-wikipedia.ico\" />\n<% /*\n<link rel=\"search\" type=\"application/opensearchdescription+xml\" href=\"https://secure.wikimedia.org/wiktionary/simple/w/opensearch_desc.php\" title=\"Wiktionary (simple)\">\n<link rel=\"EditURI\" type=\"application/rsd+xml\" href=\"https://secure.wikimedia.org/wiktionary/simple/w/api.php?action=rsd\">\n*/ %>\n<link rel=\"copyright\" href=\"http://creativecommons.org/licenses/by-sa/3.0/\" />\n<% /*\n<link rel=\"alternate\" type=\"application/atom+xml\" title=\"Wiktionary Atom feed\" href=\"https://secure.wikimedia.org/wiktionary/simple/w/index.php?title=Special:RecentChanges&amp;feed=atom\">\n*/ %>\n<link rel=\"stylesheet\" href=\"skins/load_002.css\" type=\"text/css\" media=\"all\" />\n<style type=\"text/css\" media=\"all\">.suggestions{overflow:hidden;position:absolute;top:0px;left:0px;width:0px;border:none;z-index:99;padding:0;margin:-1px -1px 0 0} html > body .suggestions{margin:-1px 0 0 0}.suggestions-special{position:relative;background-color:Window;font-size:0.8em;cursor:pointer;border:solid 1px #aaaaaa;padding:0;margin:0;margin-top:-2px;display:none;padding:0.25em 0.25em;line-height:1.25em}.suggestions-results{background-color:white;background-color:Window;font-size:0.8em;cursor:pointer;border:solid 1px #aaaaaa;padding:0;margin:0}.suggestions-result{color:black;color:WindowText;margin:0;line-height:1.5em;padding:0.01em 0.25em;text-align:left}.suggestions-result-current{background-color:#4C59A6;background-color:Highlight;color:white;color:HighlightText}.suggestions-special .special-label{font-size:0.8em;color:gray;text-align:left}.suggestions-special .special-query{color:black;font-style:italic;text-align:left}.suggestions-special .special-hover{background-color:silver}.suggestions-result-current .special-label,.suggestions-result-current .special-query{color:white;color:HighlightText}.autoellipsis-matched,.highlight{font-weight:bold}</style>\n<style type=\"text/css\" media=\"all\">#mw-panel.collapsible-nav div.portal{background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAIwAAAABCAMAAAA7MLYKAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAEtQTFRF29vb2tra4ODg6urq5OTk4uLi6+vr7e3t7Ozs8PDw5+fn4+Pj4eHh3d3d39/f6Ojo5eXl6enp8fHx8/Pz8vLy7+/v3Nzc2dnZ2NjYnErj7QAAAD1JREFUeNq0wQUBACAMALDj7hf6JyUFGxzEnYhC9GaNPG1xVffGDErk/iCigLl1XV2xM49lfAxEaSM+AQYA9HMKuv4liFQAAAAASUVORK5CYII=);background-image:url(https://secure.wikimedia.org/w/extensions-1.17/Vector/modules/./images/portal-break.png?2011-02-12T21:25:00Z)!ie;background-position:left top;background-repeat:no-repeat;padding:0.25em 0 !important;margin:-11px 9px 10px 11px}#mw-panel.collapsible-nav div.portal h5{color:#4D4D4D;font-weight:normal;background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQBAMAAADt3eJSAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAA9QTFRFeXl53d3dmpqasbGx////GU0iEgAAAAV0Uk5T/////wD7tg5TAAAAK0lEQVQI12NwgQIG0hhCDAwMTCJAhqMCA4MiWEoIJABiOCooQhULi5BqMgB2bh4svs8t+QAAAABJRU5ErkJggg==) left center no-repeat;background:url(https://secure.wikimedia.org/w/extensions-1.17/Vector/modules/./images/open.png?2011-02-12T21:25:00Z) left center no-repeat!ie;padding:4px 0 3px 1.5em;margin-bottom:0px}#mw-panel.collapsible-nav div.collapsed h5{color:#0645AD;background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAMAAAAoLQ9TAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAAxQTFRF3d3deXl5////nZ2dQA6SoAAAAAN0Uk5T//8A18oNQQAAADNJREFUeNpiYEIDDMQKMKALMDOgCTDCRWACcBG4AEwEIcDITEAFuhnotmC4g4EEzwEEGAADqgHmQSPJKgAAAABJRU5ErkJggg==) left center no-repeat;background:url(https://secure.wikimedia.org/w/extensions-1.17/Vector/modules/./images/closed-ltr.png?2011-02-12T21:25:00Z) left center no-repeat!ie;margin-bottom:0px}#mw-panel.collapsible-nav div h5:hover{cursor:pointer;text-decoration:none}#mw-panel.collapsible-nav div.collapsed h5:hover{text-decoration:underline}#mw-panel.collapsible-nav div.portal div.body{background:none !important;padding-top:0px;display:none}#mw-panel.collapsible-nav div.persistent div.body{display:block}#mw-panel.collapsible-nav div.first h5{display:none}#mw-panel.collapsible-nav div.persistent h5{background:none !important;padding-left:0.7em;cursor:default}#mw-panel.collapsible-nav div.portal div.body ul li{padding:0.25em 0}#mw-panel.collapsible-nav div.first{background-image:none;margin-top:0px}#mw-panel.collapsible-nav div.persistent div.body{margin-left:0.5em}</style>\n<meta name=\"ResourceLoaderDynamicStyles\" content=\"\" />\n<link rel=\"stylesheet\" href=\"skins/load.css\" type=\"text/css\" media=\"all\" />\n<style type=\"text/css\" media=\"all\">a.new,#quickbar a.new{color:#ba0000}</style>\n</head>\n<body class=\"mediawiki ltr ns-0 ns-subject skin-vector\">\n        <div id=\"mw-page-base\" class=\"noprint\"></div>\n        <div id=\"mw-head-base\" class=\"noprint\"></div>\n        <!-- content -->\n        <div id=\"content\">\n            <a id=\"top\"></a>\n            <div id=\"mw-js-message\" style=\"display:none;\"></div>\n            <!-- sitenotice -->\n            <div id=\"siteNotice\"><!-- centralNotice loads here -->${ pageBean.notice }</div>\n            <!-- /sitenotice -->\n            <!-- firstHeading -->\n            <h1 id=\"firstHeading\" class=\"firstHeading\">${ pageBean.error }${ pageBean.pageHeading }</h1>\n            <!-- /firstHeading -->\n            <!-- bodyContent -->\n            <div id=\"bodyContent\">\n                <!-- tagline -->\n                <div id=\"siteSub\">From <%= pageBean.getWikiNamespace().getMeta() %></div>\n                <!-- /tagline -->\n                <!-- subtitle -->\n                <div id=\"contentSub\">\n                </div>\n                <!-- /subtitle -->\n                <!-- jumpto -->\n                <div id=\"jump-to-nav\">\n                    Jump to: <a href=\"#mw-head\">navigation</a>,\n                    <a href=\"#p-search\">search</a>\n                </div>\n                <!-- /jumpto -->\n                <!-- bodytext -->\n\n${ pageBean.page }\n\n                <% if (pageBean.getFormType() != WikiPageListBean.FormType.NoForm ) { %>\n                <table class=\"mw-allpages-table-form\">\n                  <tr>\n                    <td>\n                      <div class=\"namespaceoptions\">\n                        <form method=\"get\" action=\"wiki\">\n                          <input type=\"hidden\" value=\"${ pageBean.title }\" name=\"title\" />\n                          <fieldset>\n                            <legend>${ pageBean.formTitle }</legend>\n                            <table id=\"nsselect\" class=\"allpages\">\n                      <% if (pageBean.getFormType() == WikiPageListBean.FormType.FromToForm ) { %>\n                              <tr>\n                                <td class='mw-label'><label for=\"nsfrom\">Display pages starting at:</label> </td>\n                                <td class='mw-input'><input name=\"from\" size=\"30\" value=\"<%= safeFromTitle %>\" id=\"nsfrom\" />  </td>\n                              </tr>\n                              <tr>\n                                <td class='mw-label'><label for=\"nsto\">Display pages ending at:</label> </td>\n                                <td class='mw-input'><input name=\"to\" size=\"30\" value=\"<%= safeToTitle %>\" id=\"nsto\" />      </td>\n                              </tr>\n                      <% } else if (pageBean.getFormType() == WikiPageListBean.FormType.TargetPageForm) { %>\n                              <tr>\n                                <td class='mw-label'><label for=\"target\">Page:</label> </td>\n                                <td class='mw-input'><input name=\"target\" size=\"30\" value=\"<%= safeTargetTitle %>\" id=\"nstarget\" /> <input type=\"submit\" value=\"Go\" />  </td>\n                              </tr>\n                      <% } else if (pageBean.getFormType() == WikiPageListBean.FormType.PagePrefixForm) { %>\n                              <tr>\n                                <td class='mw-label'><label for=\"prefix\">Display pages with prefix:</label> </td>\n                                <td class='mw-input'><input name=\"prefix\" size=\"30\" value=\"<%= safePrefixTitle %>\" id=\"nsprefix\" />  </td>\n                              </tr>\n                      <% } else if (pageBean.getFormType() == WikiPageListBean.FormType.PageSearchForm) { %>\n                              <tr>\n                                <td class='mw-label'><label for=\"search\">Display pages containing:</label> </td>\n                                <td class='mw-input'><input name=\"search\" size=\"30\" value=\"<%= safeSearchTitle %>\" id=\"nsprefix\" />  </td>\n                              </tr>\n                      <% } %>\n                      <% if (pageBean.getFormType() != WikiPageListBean.FormType.TargetPageForm) { %>\n                              <tr>\n                                <td class='mw-label'><label for=\"namespace\">Namespace:</label>  </td>\n                                <td class='mw-input'>\n                                  <select id=\"namespace\" name=\"namespace\" class=\"namespaceselector\">\n                      <% for (int i = 0; i <= 15; ++i) {\n                          String namespace = (i == 0) ? \"Article\" : pageBean.getWikiNamespace().getNamespaceByNumber(i);\n                          String selected = (pageBean.getNamespaceId() == i) ? \" selected=\\\"selected\\\"\" : \"\";\n                      %>\n                                    <option value=\"<%= i %>\"<%= selected %>><%= namespace %></option>\n                      <% } %>\n                                  </select>\n                                  <input type=\"submit\" value=\"Go\" />\n                                </td>\n                              </tr>\n                      <% } %>\n                            </table>\n                          </fieldset>\n                        </form>\n                      </div>\n                    </td>\n<% /*               <td class=\"mw-allpages-nav\"><a href=\"wiki?title=Special:AllPages<%= andServiceUser >\" title=\"Special:AllPages\">All pages</a></td> */ %>\n                  </tr>\n                </table>\n                <% } %>\n\n<% if (pageBean.getFormType() == WikiPageListBean.FormType.PageSearchForm) { %>\n  <p class=\"mw-search-createlink\">\n  <%\n      if (pageBean.isFoundFullMatch()) {\n  %>\n    <b>There is a page named \"<a href=\"wiki?title=<%= safeSearchTitle %><%= andServiceUser %>\" title=\"<%= safeSearchTitle %>\"><%= pageBean.getSearch() %></a>\" on this wiki.</b>\n  <% } else {%>\n    <b>Create the page \"<a href=\"wiki?title=<%= safeSearchTitle %>&amp;action=edit<%= andServiceUser %>\" class=\"new\" title=\"<%= safeSearchTitle %>\"><%= pageBean.getSearch() %></a>\" on this wiki!</b>\n  <% } %>\n  </p>\n<% } %>\n                <table class=\"mw-allpages-table-chunk\">\n<% if (!pageBean.getPages().isEmpty()) {\n    Iterator<String> iter = pageBean.getPages().iterator();\n    while (iter.hasNext()) {\n        String value = iter.next();\n%>\n                  <tr>\n                    <td style=\"width:33%\">\n                      <a href=\"wiki?title=<%= value %><%= andServiceUser %>\"><%= value %></a>\n          <% if (de.zib.scalaris.examples.wikipedia.Options.getInstance().WIKI_USE_BACKLINKS) { %> \n                      <span class=\"mw-whatlinkshere-tools\">(<a href=\"wiki?title=Special:WhatLinksHere&amp;target=<%= value %><%= andServiceUser %>\" title=\"Special:WhatLinksHere\">← links</a>)</span>\n          <% } %>\n                    </td>\n                    <td style=\"width:33%\">\n<%      if (iter.hasNext()) {\n            value = iter.next();\n%>\n                      <a href=\"wiki?title=<%= value %><%= andServiceUser %>\"><%= value %></a> \n          <% if (de.zib.scalaris.examples.wikipedia.Options.getInstance().WIKI_USE_BACKLINKS) { %>\n                      <span class=\"mw-whatlinkshere-tools\">(<a href=\"wiki?title=Special:WhatLinksHere&amp;target=<%= value %><%= andServiceUser %>\" title=\"Special:WhatLinksHere\">← links</a>)</span>\n          <% } %>\n<%      } %>\n                    </td>\n                    <td style=\"width:33%\">\n<%      if (iter.hasNext()) {\n            value = iter.next();\n%>\n                      <a href=\"wiki?title=<%= value %><%= andServiceUser %>\"><%= value %></a>\n          <% if (de.zib.scalaris.examples.wikipedia.Options.getInstance().WIKI_USE_BACKLINKS) { %> \n                      <span class=\"mw-whatlinkshere-tools\">(<a href=\"wiki?title=Special:WhatLinksHere&amp;target=<%= value %><%= andServiceUser %>\" title=\"Special:WhatLinksHere\">← links</a>)</span>\n          <% } %>\n<%      } %>\n                    </td>\n                  </tr>\n<%\n    }\n  }\n%>\n                </table>\n                <hr />\n<% /*           <p class=\"mw-allpages-nav\"><a href=\"wiki?title=Special:AllPages<%= andServiceUser >\" title=\"Special:AllPages\">All pages</a></p> */ %>\n                <div class=\"printfooter\">\n                Retrieved from \"<a href=\"wiki?title=<%= pageTitleWithPars %><%= andServiceUser %>\">wiki?title=${ pageBean.title }</a>\"</div>\n                <!-- /bodytext -->\n<% if (req_render == null || !req_render.equals(\"0\")) { %>\n                <!-- catlinks -->\n                <div id=\"catlinks\" class=\"catlinks catlinks-allhidden\"></div>\n                <!-- /catlinks -->\n<% } %>\n                <div class=\"visualClear\"></div>\n            </div>\n            <!-- /bodyContent -->\n        </div>\n        <!-- /content -->\n        <!-- header -->\n        <div id=\"mw-head\" class=\"noprint\">\n            \n<!-- 0 -->\n<div id=\"p-personal\" class=\"\">\n    <h5>Personal tools</h5>\n    <ul>\n                    <li id=\"pt-login\"><a href=\"wiki?title=Special:UserLogin&amp;returnto=<%= safePageTitleWithPars %><%= andServiceUser %>\" title=\"You are encouraged to log in; however, it is not mandatory [o]\" accesskey=\"o\">Log in / create account</a></li>\n    </ul>\n</div>\n\n<!-- /0 -->\n            <div id=\"left-navigation\">\n                \n<!-- 0 -->\n<div id=\"p-namespaces\" class=\"vectorTabs\">\n    <h5>Namespaces</h5>\n    <ul>\n                    <li id=\"ca-special\" class=\"selected\"><span><a href=\"wiki?title=<%= pageTitleWithPars %><%= andServiceUser %>\" >Special page</a></span></li>\n    </ul>\n</div>\n\n<!-- /0 -->\n\n<!-- 1 -->\n<div id=\"p-variants\" class=\"vectorMenu emptyPortlet\">\n        <h5><span>Variants</span><a href=\"#\"></a></h5>\n    <div class=\"menu\">\n        <ul>\n        </ul>\n    </div>\n</div>\n\n<!-- /1 -->\n            </div>\n            <div id=\"right-navigation\">\n                \n<!-- 0 -->\n<div id=\"p-views\" class=\"vectorTabs emptyPortlet\">\n    <h5>Views</h5>\n    <ul>\n    </ul>\n</div>\n\n<!-- /0 -->\n\n<!-- 1 -->\n<div id=\"p-cactions\" class=\"vectorMenu emptyPortlet\">\n    <h5><span>Actions</span><a href=\"#\"></a></h5>\n    <div class=\"menu\">\n        <ul>\n        </ul>\n    </div>\n</div>\n\n<!-- /1 -->\n\n<!-- 2 -->\n<div id=\"p-search\">\n    <h5><label for=\"searchInput\">Search</label></h5>\n    <form action=\"wiki?\" id=\"searchform\">\n        <input name=\"title\" value=\"Special:Search\" type=\"hidden\" />\n        <div id=\"simpleSearch\">\n            <input type=\"hidden\" value=\"${ pageBean.serviceUser }\" name=\"service_user\"/>\n            <input autocomplete=\"off\" placeholder=\"Search\" tabindex=\"1\" id=\"searchInput\" name=\"search\" title=\"Search Wikipedia [f]\" accesskey=\"f\" type=\"text\" />\n            <button id=\"searchButton\" type=\"submit\" name=\"button\" title=\"Search the pages for this text\"><img src=\"skins/search-ltr.png\" alt=\"Search\" /></button>\n        </div>\n    </form>\n</div>\n\n<!-- /2 -->\n            </div>\n        </div>\n        <!-- /header -->\n        <!-- panel -->\n            <div id=\"mw-panel\" class=\"noprint collapsible-nav\">\n                <!-- logo -->\n                    <div id=\"p-logo\"><a style=\"background-image: url(&quot;images/Wikipedia.png&quot;);\" href=\"wiki?title=Main Page<%= andServiceUser %>\" title=\"Visit the main page\"></a></div>\n                <!-- /logo -->\n                \n<!-- navigation -->\n<div class=\"portal first persistent\" id=\"p-navigation\">\n    <h5>Links</h5>\n    <div class=\"body\">\n                <ul>\n                    <li id=\"n-mainpage\"><a href=\"wiki?title=Main Page<%= andServiceUser %>\" title=\"Visit the main page [z]\" accesskey=\"z\">Main Page</a></li>\n                    <li id=\"n-recentchanges\"><a href=\"wiki?title=Special:RecentChanges<%= andServiceUser %>\" title=\"The list of recent changes in the wiki [r]\" accesskey=\"r\">New changes</a></li>\n                    <li id=\"n-randompage\"><a href=\"wiki?title=Special:Random<%= andServiceUser %>\" title=\"Load a random page [x]\" accesskey=\"x\">Show any entry</a></li>\n                    <li id=\"n-help\"><a href=\"wiki?title=Help:Contents<%= andServiceUser %>\" title=\"The place to find out\">Help</a></li>\n                </ul>\n            </div>\n</div>\n\n<!-- /navigation -->\n\n<!-- SEARCH -->\n\n<!-- /SEARCH -->\n\n<!-- TOOLBOX -->\n<div class=\"portal expanded\" id=\"p-tb\">\n    <h5 tabindex=\"2\">Toolbox</h5>\n    <div style=\"display: block;\" class=\"body\">\n        <ul>\n                    <li id=\"t-specialpages\"><a href=\"wiki?title=Special:SpecialPages<%= andServiceUser %>\" title=\"List of all special pages [q]\" accesskey=\"q\">Special pages</a></li>\n        </ul>\n    </div>\n</div>\n\n<!-- /TOOLBOX -->\n\n<!-- LANGUAGES -->\n<!-- /LANGUAGES -->\n            </div>\n        <!-- /panel -->\n        <!-- footer -->\n        <div id=\"footer\">\n                <ul id=\"footer-places\">\n                    <li id=\"footer-places-privacy\"><a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:Privacy policy<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:Privacy policy\">Privacy policy</a></li>\n                    <li id=\"footer-places-about\"><a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:About<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:About\">About <%= pageBean.getWikiNamespace().getMeta() %></a></li>\n                    <li id=\"footer-places-disclaimer\"><a href=\"wiki?title=<%= pageBean.getWikiNamespace().getMeta() %>:General disclaimer<%= andServiceUser %>\" title=\"<%= pageBean.getWikiNamespace().getMeta() %>:General disclaimer\">Disclaimers</a></li>\n                </ul>\n                <ul id=\"footer-icons\" class=\"noprint\">\n                </ul>\n                <div style=\"clear:both\"></div>\n                <div style=\"font-size:0.7em\">DB Timings:\n                <pre id=\"db_timings\" style=\"padding:0;line-height:0.7em\">\n<% for (Map.Entry<String,List<Long>> stats : pageBean.getStats().entrySet()) { %>\n<%= stats.getKey() %>: <%= stats.getValue().toString() %>\n<% } %>\n                </pre></div>\n                <div style=\"font-size:0.7em\">Involved DB keys:\n                <pre id=\"involved_keys\" style=\"padding:0;line-height:0.7em\">\n<% for (InvolvedKey involvedKey : pageBean.getInvolvedKeys()) { %>\n<%= involvedKey.toString() %>\n<% } %>\n                </pre></div>\n<% long renderTime = (System.currentTimeMillis() - pageBean.getStartTime()); %>\n                <div style=\"clear:both\"></div>\n                <div style=\"font-size:0.7em\">More timings:\n                <pre id=\"other_timings\" style=\"padding:0;line-height:0.7em\">\n\nserver: <%= renderTime %>\n\n                </pre></div>\n        </div>\n        <!-- /footer -->\n<% servlet.storeUserReq(pageBean, renderTime); %>\n</body>\n</html>\n"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/skins/load.css",
    "content": ".interwiki-completelist{font-weight:bold} #t-upload{display:none} ol.references{font-size:100%}.references-small{font-size:90%}    .references-2column{font-size:90%;-moz-column-count:2;column-count:2}.same-bg{background:none } table.navbox{background-color:#f9f9f9;border:1px solid #aaa;clear:both;font-size:90%;margin:1em 0em 0em;padding:5px;text-align:center;width:100%}table.navbox th{background-color:#ccf;padding-left:1em;padding-right:1em}table.navbox tr:not(:first-child) th{background-color:#ddf}@media print{.navbox{display:none}} .float-left{clear: left;float: left;} .float-right{clear: right;float: right;margin: 1em 0 1em 1em;} .infobox{border:1px solid #aaa;background-color:#f9f9f9;color:black;margin-bottom:0.5em;margin-left:1em;padding:0.2em;float:right;clear:right}.infobox td,.infobox th{vertical-align:top}.infobox caption{font-size:larger;margin-left:inherit}.infobox.bordered{border-collapse:collapse}.infobox.bordered td,.infobox.bordered th{border:1px solid #aaa}.infobox.bordered .borderless td,.infobox.bordered .borderless th{border:0}.infobox.sisterproject{width:20em;font-size:90%}@media print{.infobox.sisterproject{display:none}} .infobox.bordered .mergedtoprow td,.infobox.bordered .mergedtoprow th{border:0;border-top:1px solid #aaa;border-right:1px solid #aaa}.infobox.bordered .mergedrow td,.infobox.bordered .mergedrow th{border:0;border-right:1px solid #aaa} .infobox.geography{text-align:left;border-collapse:collapse;line-height:1.2em;font-size:90%}.infobox.geography td,.infobox.geography th{border-top:solid 1px #aaa;padding:0.4em 0.2em 0.4em 0.8em}.infobox.geography .mergedtoprow td,.infobox.geography .mergedtoprow th{border-top:solid 1px #aaa;padding:0.4em 0.2em 0.2em 0.8em}.infobox.geography .mergedrow td,.infobox.geography .mergedrow th{border:0;padding:0 0.2em 0.2em 0.8em}.infobox.geography .mergedbottomrow td,.infobox.geography .mergedbottomrow th{border-top:0;border-bottom:solid 1px #aaa;padding:0 0.2em 0.4em 0.8em}.infobox.geography .maptable td,.infobox.geography .maptable th{border:0;padding:0 0 0 0} .notice{margin:1em;padding:0.2em}#disambig{border-top:1px solid #ccc;border-bottom:1px solid #ccc}#spoiler{border-top:2px solid #ddd;border-bottom:2px solid #ddd} .Talk-Notice{border:1px solid #C0C090;background-color:#F8EABA;margin-bottom:3px;width:85%;border-spacing:3px;margin-left:auto;margin-right:auto}.Talk-Notice:after{content:\"The CSS for this template should be changed. See [[Wikipedia:Template Standardisation]].\"} .Talk-Notice td{background:inherit} table.persondata{border:1px solid #aaa;display:none;speak:none}.persondata-label{color:#aaa}  .Use_Default_Date_Convention{display:inline}.Use_AD_and_BC{display:none}.Use_BCE_and_CE{display:none}     .audiolink a{background:url(11px-Loudspeaker.svg.png) center left no-repeat !important;padding-left:16px !important;padding-right:0 !important} div.listenlist{background:url(30px-Gnome-speakernotes.png);padding-left:40px}div.videolist,div.multivideolist{background:url(40px-Tango-video-x-generic.png);padding-left:50px} div.medialist{min-height:50px;margin:1em;background-position:top left;background-repeat:no-repeat}div.medialist ul{list-style-type:none;list-style-image:none;margin:0}div.medialist ul li{padding-bottom:0.5em}div.medialist ul li li{font-size:91%;padding-bottom:0} div.columns-2 div.column{float:left;width:50%;min-width:300px}div.columns-3 div.column{float:left;width:33.3%;min-width:200px}div.columns-4 div.column{float:left;width:25%;min-width:150px}div.columns-5 div.column{float:left;width:20%;min-width:120px} .messagebox{border:1px solid #aaa;background-color:#f9f9f9;width:80%;margin:0 auto 1em auto;padding:.2em}.messagebox.merge{border:1px solid #c0b8cc;background-color:#f0e5ff;text-align:center}.messagebox.cleanup{border:1px solid #9f9fff;background-color:#efefff;text-align:center}.messagebox.standard-talk{border:1px solid #c0c090;background-color:#f8eaba} #file img{background:url(Checker-16x16.png) repeat} .IPA{font-family:\"Chrysanthi Unicode\",\"Doulos SIL\",Gentium,GentiumAlt,Code2000,\"TITUS Cyberbit Basic\",\"DejaVu Sans\",\"Bitstream Cyberbit\",\"Arial Unicode MS\",\"Lucida Sans Unicode\",\"Hiragino Kaku Gothic Pro\",\"Matrix Unicode\";font-family :inherit}.Unicode{font-family:Code2000,\"TITUS Cyberbit Basic\",\"Doulos SIL\",\"Chrysanthi Unicode\",\"Bitstream Cyberbit\",\"Bitstream CyberBase\",Thryomanes,Gentium,GentiumAlt,\"Lucida Grande\",\"Arial Unicode MS\",\"Microsoft Sans Serif\",\"Lucida Sans Unicode\";font-family :inherit}.latinx{font-family:Code2000,\"TITUS Cyberbit Basic\",\"Microsoft Sans Serif\";font-family :inherit}.polytonic{font-family:Athena,Gentium,\"Palatino Linotype\",\"Arial Unicode MS\",\"Lucida Sans Unicode\",\"Lucida Grande\",Code2000;font-family :inherit}.mufi{font-family:Alphabetum,Cardo,LeedsUni,Junicode,\"TITUS Cyberbit Basic\",ALPHA-Demo}#wpSave{font-weight:bold} .hiddenStructure{display:inline ! important;color:#f00;background-color:#0f0} .IPA a:link,.IPA a:visited{text-decoration:none} @media print{#privacy,#about,#disclaimer{display:none}}#EnWpMpBook{background-image:url(MP-open-book.png)}#EnWpMpSearch{background:url(MP-magnifying-glass.png) no-repeat top right}#EnWpMpSearchInner{float:right;width:20em;text-align:center}#EnWpMpBook2{background-image:url(MP-open-book2.png)}.messagebox.small-talk{width:238px;font-size:85%;float:right;clear:both;margin:0 0 1em 1em;line-height:1.25em;background:#F8EABA} div.Boxmerge,div.NavFrame{margin:0px;padding:2px;border:1px solid #aaa;text-align:center;border-collapse:collapse;font-size:95%}div.Boxmerge div.NavFrame{border-style:none;border-style:hidden}div.NavFrame + div.NavFrame{border-top-style:none;border-top-style:hidden}div.NavPic{background-color:#fff;margin:0px;padding:2px;float:left}div.NavFrame div.NavHead{height:1.6em;font-weight:bold;font-size:100%;background-color:#efefef;position:relative}div.NavFrame p{font-size:100%}div.NavFrame div.NavContent{font-size:100%}div.NavFrame div.NavContent p{font-size:100%}div.NavEnd{margin:0px;padding:0px;line-height:1px;clear:both}a.NavToggle{position:absolute;top:0px;right:3px;font-weight:normal;font-size:smaller}.dablink{font-style:italic;padding-left:2em} .horizontal ul{padding:0;margin:0}.horizontal li{padding:0 0.6em 0 0.4em;display:inline;border-right:1px solid}.horizontal li:last-child{border-right:none;padding-right:0} .geo-default{display:inline}.geo-nondefault{display:none}.geo-dms{display:inline}.geo-dec{display:inline}.geo-multi-punct{display:none}.longitude .latitude{white-space:nowrap} .geo{}  .nonumtoc .tocnumber{display:none}.nonumtoc #toc ul,.nonumtoc .toc ul{line-height:1.5em;list-style:none;margin:.3em 0 0;padding:0}.nonumtoc #toc ul ul,.nonumtoc .toc ul ul{margin:0 0 0 2em} .toclimit-2 .toclevel-2,.toclimit-3 .toclevel-3,.toclimit-4 .toclevel-4,.toclimit-5 .toclevel-5,.toclimit-6 .toclevel-6,.toclimit-7 .toclevel-7{display:none } .listify td{display:list-item}.listify tr{display:block}.listify table{display:block} blockquote.templatequote{margin-top:0}blockquote.templatequote div.templatequotecite{line-height:1em;text-align:left;padding-left:2em;margin-top:0}blockquote.templatequote div.templatequotecite cite{font-size:smaller}div.user-block{padding:5px;border:1px solid #A9A9A9;background-color:#FFEFD5} .nowraplinks a{white-space:nowrap} .template-documentation{clear:both;margin:1em 0 0 0;border:1px solid #aaa;background-color:#ecfcf4;padding:5px}.thumbinner{min-width:100px} div.Boxmerge,div.NavFrame{margin:0px;padding:2px;border:1px solid #aaaaaa;text-align:center;border-collapse:collapse;font-size:95%;overflow:auto;width:auto}div.Boxmerge div.NavFrame{border-style:none;border-style:hidden}div.NavFrame + div.NavFrame{border-top-style:none;border-top-style:hidden}div.NavPic{background-color:#ffffff;margin:0px;padding:2px;float:left}div.NavFrame div.NavHead{height:1.6em;font-weight:bold;font-size:100%;background-color:#efefef;padding-left:10px}div.NavFrame p{font-size:100%}div.NavFrame div.NavContent{font-size:100%}div.NavFrame div.NavContent p{font-size:100%}div.NavEnd{margin:0px;padding:0px;line-height:1px;clear:both}span.NavToggle{cursor:pointer;float:right;font-weight:normal;font-size:smaller}ul.onyms-collapse{padding-left:3em}  @media print{  }@media handheld{  }\n\n/* cache key: simplewiktionary:resourceloader:filter:minify-css:5:c173a8df4339cd6ea4511c7460c22249 */"
  },
  {
    "path": "contrib/wikipedia/scalaris-wiki/skins/load_002.css",
    "content": "@media print{  a.stub,a.new{color:#ba0000;text-decoration:none}#toc{border:1px solid #aaaaaa;background-color:#f9f9f9;padding:5px}.tocindent{margin-left:2em}.tocline{margin-bottom:0px} div.floatright{float:right;clear:right;position:relative;margin:0.5em 0 0.8em 1.4em}div.floatright p{font-style:italic}div.floatleft{float:left;clear:left;position:relative;margin:0.5em 1.4em 0.8em 0}div.floatleft p{font-style:italic} div.thumb{border:none;width:auto;margin-top:0.5em;margin-bottom:0.8em;background-color:transparent}div.thumbinner{border:1px solid #cccccc;padding:3px !important;background-color:White;font-size:94%;text-align:center;overflow:hidden}html .thumbimage{border:1px solid #cccccc}html .thumbcaption{border:none;text-align:left;line-height:1.4em;padding:3px !important;font-size:94%}div.magnify{display:none}div.tright{float:right;clear:right;margin:0.5em 0 0.8em 1.4em}div.tleft{float:left;clear:left;margin:0.5em 1.4em 0.8em 0}img.thumbborder{border:1px solid #dddddd} table.rimage{float:right;width:1pt;position:relative;margin-left:1em;margin-bottom:1em;text-align:center}body{background:White; color:Black;margin:0;padding:0}.noprint,div#jump-to-nav,div.top,div#column-one,#colophon,.editsection,.toctoggle,.tochidden,div#f-poweredbyico,div#f-copyrightico,li#viewcount,li#about,li#disclaimer,li#privacy,#footer-places,#mw-hidden-catlinks{ display:none}ul{list-style-type:square}#content{background:none;border:none ! important;padding:0 ! important;margin:0 ! important}#footer{background :white;color :black;border-top:1px solid black}h1,h2,h3,h4,h5,h6{font-weight:bold}p,.documentDescription{margin:1em 0 ! important;line-height:1.2em}.tocindent p{margin:0 0 0 0 ! important}pre{border:1pt dashed black;white-space:pre;font-size:8pt;overflow:auto;padding:1em 0;background:white;color:black}table.listing,table.listing td{border:1pt solid black;border-collapse:collapse}a{color:Black !important;background:none !important;padding:0 !important}a:link,a:visited{color:#520;background:transparent;text-decoration:underline}#content a.external.text:after,#content a.external.autonumber:after{ content:\" (\" attr(href) \") \"}#globalWrapper{width:100% !important;min-width:0 !important}#content{background:white;color:black}#column-content{margin:0 !important}#column-content #content{padding:1em;margin:0 !important} a,a.external,a.new,a.stub{color:black ! important;text-decoration:none ! important} a,a.external,a.new,a.stub{color:inherit ! important;text-decoration:inherit ! important}img{border:none;vertical-align:middle} span.texhtml{font-family:serif}#siteNotice{display:none} table.gallery{border:1px solid #ccc;margin:2px;padding:2px;background-color:white;border-collapse:collapse}table.gallery tr{vertical-align:top}table.gallery td{vertical-align:top;padding:1px;border:1px solid #ccc}div.gallerybox{margin:2px}div.gallerybox div.thumb{text-align:center;border:1px solid #ccc;margin:2px}div.gallerytext{overflow:hidden;font-size:94%;padding:2px 4px} table.diff{background:white}td.diff-otitle{background:#ffffff}td.diff-ntitle{background:#ffffff}td.diff-addedline{background:#ccffcc;font-size:smaller;border:solid 2px black}td.diff-deletedline{background:#ffffaa;font-size:smaller;border:dotted 2px black}td.diff-context{background:#eeeeee;font-size:smaller}.diffchange{color:silver;font-weight:bold;text-decoration:underline} table.wikitable{margin:1em 1em 1em 0;border:1px #aaa solid;background:white;border-collapse:collapse}.wikitable th,.wikitable td{border:1px #aaa solid;padding:0.2em}.wikitable th{text-align:center;background:white;font-weight:bold}.wikitable caption{font-weight:bold}a.sortheader{margin:0px 0.3em} .wikitable,.thumb,img{page-break-inside:avoid}h2,h3,h4,h5,h6,h7{page-break-after:avoid}p{widows:3;orphans:3}}@media screen{  .mw-plusminus-pos{color:#006400} .mw-plusminus-neg{color:#8b0000} .mw-plusminus-null{color:#aaa}  .allpagesredirect,.redirect-in-category,.watchlistredir{font-style:italic} span.comment{font-style:italic}span.changedby{font-size:95%} .texvc{direction:ltr;unicode-bidi:embed}img.tex{vertical-align:middle}span.texhtml{font-family:serif}  #wikiPreview.ontop{margin-bottom:1em} #editform,#toolbar,#wpTextbox1{clear:both}div#mw-js-message{margin:1em 5%;padding:0.5em 2.5%;border:solid 1px #ddd;background-color:#fcfcfc} .editsection{float:right;margin-left:5px} h2#filehistory{clear:both}table.filehistory th,table.filehistory td{vertical-align:top}table.filehistory th{text-align:left}table.filehistory td.mw-imagepage-filesize,table.filehistory th.mw-imagepage-filesize{white-space:nowrap}table.filehistory td.filehistory-selected{font-weight:bold} li span.deleted,span.history-deleted{text-decoration:line-through;color:#888;font-style:italic} .not-patrolled{background-color:#ffa}.unpatrolled{font-weight:bold;color:red}div.patrollink{font-size:75%;text-align:right} td.mw-label{text-align:right}td.mw-input{text-align:left}td.mw-submit{text-align:left}td.mw-label{vertical-align:top}.prefsection td.mw-label{width:20%}.prefsection table{width:100%}td.mw-submit{white-space:nowrap}table.mw-htmlform-nolabel td.mw-label{width:0 !important}tr.mw-htmlform-vertical-label td.mw-label{text-align:left !important}input#wpSummary{width:80%} .thumbcaption{text-align:left}.magnify{float:right} .mw-hidden-cats-hidden{display:none}.catlinks-allhidden{display:none} p.mw-ipb-conveniencelinks,p.mw-protect-editreasons,p.mw-filedelete-editreasons,p.mw-delete-editreasons,p.mw-revdel-editreasons{font-size:90%;float:right} .searchresults{}.searchresults p{margin-left:0.4em;margin-top:1em;margin-bottom:1.2em}div.searchresult{font-size:95%;width:38em}.mw-search-results{margin-left:0.4em}.mw-search-results li{padding-bottom:1em;list-style:none;list-style-image:none}.mw-search-results li a{font-size:108%}.mw-search-result-data{color:green;font-size:97%}.mw-search-formheader{background-color:#f3f3f3;margin-top:1em;border:1px solid silver}.mw-search-formheader div.search-types{float:left;padding-left:0.25em}.mw-search-formheader div.search-types ul{margin:0 !important;padding:0 !important;list-style:none !important}.mw-search-formheader div.search-types ul li{float:left;margin:0;padding:0}.mw-search-formheader div.search-types ul li a{display:block;padding:0.5em}.mw-search-formheader div.search-types ul li.current a{color:#333333;cursor:default}.mw-search-formheader div.search-types ul li.current a:hover{text-decoration:none}.mw-search-formheader div.results-info{float:right;padding:0.5em;padding-right:0.75em}.mw-search-formheader div.results-info ul{margin:0 !important;padding:0 !important;list-style:none !important}.mw-search-formheader div.results-info ul li{float:right;margin:0;padding:0}fieldset#mw-searchoptions{margin:0;padding-left:0.75em !important;padding-right:0.75em !important;padding-bottom:0.5em !important;padding-top:0.5em !important;border:none;background-color:#f9f9f9;border:1px solid silver !important;border-top-width:0 !important}fieldset#mw-searchoptions legend{display:none}fieldset#mw-searchoptions h4{padding:0;margin:0;float:left}fieldset#mw-searchoptions div#mw-search-togglebox{float:right}fieldset#mw-searchoptions div#mw-search-togglebox label{margin-right:0.25em}fieldset#mw-searchoptions div#mw-search-togglebox input{margin-left:0.25em}fieldset#mw-searchoptions table{float:left;margin-right:3em}fieldset#mw-searchoptions table td{padding-right:1em}fieldset#mw-searchoptions div.divider{clear:both;border-bottom:1px solid #DDDDDD;padding-top:0.5em;margin-bottom:0.5em}td#mw-search-menu{padding-left:6em;font-size:85%}div#mw-search-interwiki{float:right;width:18em;border-style:solid;border-color:#AAAAAA;border-width:1px;margin-top:2ex}div#mw-search-interwiki li{font-size:95%}.mw-search-interwiki-more{float:right;font-size:90%}div#mw-search-interwiki-caption{text-align:center;font-weight:bold;font-size:95%}.mw-search-interwiki-project{font-size:97%;text-align:left;padding-left:0.2em;padding-right:0.15em;padding-bottom:0.2em;padding-top:0.15em;background-color:#ececec;border-top:1px solid #BBBBBB}span.searchalttitle{font-size:95%}div.searchdidyoumean{font-size:127%;margin-top:0.8em; color:#c00}div.searchdidyoumean em{font-weight:bold}.searchmatch{font-weight:bold} td#mw-search-togglebox{text-align:right}table#mw-search-powertable{width:100%}form#powersearch{clear:both} .mw-userrights-disabled{color:#888}table.mw-userrights-groups * td,table.mw-userrights-groups * th{padding-right:1.5em} .os-suggest{overflow:auto;overflow-x:hidden;position:absolute;top:0px;left:0px;width:0px;background-color:white;background-color:Window;border-style:solid;border-color:#AAAAAA;border-width:1px;z-index:99;font-size:95%}table.os-suggest-results{font-size:95%;cursor:pointer;border:0;border-collapse:collapse;width:100%}.os-suggest-result,.os-suggest-result-hl{white-space:nowrap;background-color:white;background-color:Window;color:black;color:WindowText;padding:2px}.os-suggest-result-hl,.os-suggest-result-hl-webkit{background-color:#4C59A6;color:white}.os-suggest-result-hl{ background-color:Highlight;color:HighlightText}.os-suggest-toggle{position:relative;left:1ex;font-size:65%}.os-suggest-toggle-def{position:absolute;top:0px;left:0px;font-size:65%;visibility:hidden}  .autocomment{color:gray}#pagehistory .history-user{margin-left:0.4em;margin-right:0.2em}#pagehistory span.minor{font-weight:bold}#pagehistory li{border:1px solid white}#pagehistory li.selected{background-color:#f9f9f9;border:1px dashed #aaa} .newpage,.minor,.bot{font-weight:bold} .mw-uctop{font-weight:bold} table.mw-listgrouprights-table tr{vertical-align:top}.listgrouprights-revoked{text-decoration:line-through} td.mw-statistics-numbers{text-align:right} h4.mw-specialpagesgroup{background-color:#dcdcdc;padding:2px;margin:.3em 0em 0em 0em}.mw-specialpagerestricted{font-weight:bold}#shared-image-dup,#shared-image-conflict{font-style:italic} table.mw-emailuser-table{width:98%}td#mw-emailuser-sender,td#mw-emailuser-recipient{font-weight:bold} table.mw-allpages-table-form,table.mw-allpages-table-chunk{width:100%}td.mw-allpages-alphaindexline{text-align:right}td.mw-allpages-nav,p.mw-allpages-nav{text-align:right;font-size:smaller;margin-bottom:1em}table.mw-allpages-table-form tr{vertical-align:top} table#mw-prefixindex-list-table,table#mw-prefixindex-nav-table{width:98%}td#mw-prefixindex-nav-form{font-size:smaller;margin-bottom:1em;text-align:right;vertical-align:top} div.mw-warning-with-logexcerpt{padding:3px;margin-bottom:3px;border:2px solid #2F6FAB;clear:both}div.mw-warning-with-logexcerpt ul li{font-size:90%} span.mw-revdelundel-link,strong.mw-revdelundel-link{font-size:90%}span.mw-revdelundel-hidden,input.mw-revdelundel-hidden{visibility:hidden}td.mw-revdel-checkbox,th.mw-revdel-checkbox{padding-right:10px;text-align:center} a.feedlink{background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAMCAYAAABWdVznAAAABmJLR0QA/wD/AP+gvaeTAAAAB3RJTUUH2AkOCjkSL9xYhAAAAc9JREFUKJE90LFrU1EYQPHzffe+l/iSVkXTmNiANBU7iE5OLrbSVYKIiy5dnARB3FwEB5dOOhQKuthJEEHRsUXBoosO0lKKEYRa29LWQk3S5L53r0PVv+D8OPJlolrrr1ZmI7F1BFEjqBXECGJAjSBCaLddc7u5Mmb7q5U5007rWh5E9rYR/xsTBBXBWMVEglqRpGiGhcE5G6kdyugxcGsGyRdJ15ZwC29IF55jNEWt8K+aFOMhc+dC7Z6SITjC7ga2MkI8cpH41Dhh7RPa20Gt4toZac+IqhFMTpG0hVt8RetJg967SaTvGLnGNKZ0EtfOcB1P5jyqVjCRkIzfpnjtMYXrT2FrCff6JqhFRx/gnCXtZHgXUFHQSGg/u4Gbf4T2lYkvTaFGce8fIgePY09fwXU8Pg3sk2JFu5v4lQ+4FxPge+j5u3Q+v8TvrBKfbZB1PT4LqJh9Uv7yFLmrM2i+gPs4jRyqIaUz7C2+xZZOEA4cJaSgaAhqhbC1DK0N3K9NusvzAHB4GLf+HQBJBsiCD7J6/9zXI2VbVyv/b6Sdv1e6nrTryboB7wVbyjXt1rcfo0Frs4UkqvtUJHMBjyVEAcSjFiQJwRvf3F3/OfYH/dDFWrCooaIAAAAASUVORK5CYII=) center left no-repeat;background:url(feed-icon.png) center left no-repeat!ie;padding-left:16px} .plainlinks a{background:none !important;padding:0 !important} table.wikitable{margin:1em 1em 1em 0;background:#f9f9f9;border:1px #aaa solid;border-collapse:collapse}.wikitable th,.wikitable td{border:1px #aaa solid;padding:0.2em}.wikitable th{background:#f2f2f2;text-align:center}.wikitable caption{font-weight:bold} table.collapsed tr.collapsable{display:none} .success{color:green;font-size:larger}.error{color:red;font-size:larger}.errorbox,.successbox{font-size:larger;border:2px solid;padding:.5em 1em;float:left;margin-bottom:2em;color:#000}.errorbox{border-color:red;background-color:#fff2f2}.successbox{border-color:green;background-color:#dfd}.errorbox h2,.successbox h2{font-size:1em;font-weight:bold;display:inline;margin:0 .5em 0 0;border:none} .previewnote{color:#c00;margin-bottom:1em}.previewnote p{text-indent:3em;margin:0.8em 0}.visualClear{clear:both}#mw_trackbacks{border:solid 1px #bbbbff;background-color:#eeeeff;padding:0.2em} .TablePager{min-width:80%;border-collapse:collapse}.TablePager_nav a{text-decoration:none}.TablePager,.TablePager td,.TablePager th{border:1px solid #aaaaaa;padding:0 0.15em 0 0.15em}.TablePager th{background-color:#eeeeff}.TablePager td{background-color:#ffffff}.TablePager tr:hover td{background-color:#eeeeff}.imagelist td,.imagelist th{white-space:nowrap}.imagelist .TablePager_col_links{background-color:#eeeeff}.imagelist .TablePager_col_img_description{white-space:normal}.imagelist th.TablePager_sort{background-color:#ccccff} #mw-allmessagestable .allmessages-customised td.am_default{background-color:#fcffc4}#mw-allmessagestable tr.allmessages-customised:hover td.am_default{background-color:#faff90}#mw-allmessagestable td.am_actual{background-color:#e2ffe2}#mw-allmessagestable tr.allmessages-customised:hover + tr.allmessages-customised td.am_actual{background-color:#b1ffb1} ul#filetoc{text-align:center;border:1px solid #aaaaaa;background-color:#f9f9f9;padding:5px;font-size:95%;margin-bottom:0.5em;margin-left:0;margin-right:0}#filetoc li{display:inline;list-style-type:none;padding-right:2em} table.mw_metadata{font-size:0.8em;margin-left:0.5em;margin-bottom:0.5em;width:300px}table.mw_metadata caption{font-weight:bold}table.mw_metadata th{font-weight:normal}table.mw_metadata td{padding:0.1em}table.mw_metadata{border:none;border-collapse:collapse}table.mw_metadata td,table.mw_metadata th{text-align:center;border:1px solid #aaaaaa;padding-left:0.1em;padding-right:0.1em}table.mw_metadata th{background-color:#f9f9f9}table.mw_metadata td{background-color:#fcfcfc}  li.gallerybox{vertical-align:top;background-color:#f9f9f9;border:solid 2px white;display:-moz-inline-box}ul.gallery,li.gallerybox{display:inline-block;zoom:1;*display:inline}ul.gallery{margin:2px;padding:2px;background-color:white;display:block}li.gallerycaption{font-weight:bold;text-align:center;display:block;word-wrap:break-word}li.gallerybox div.thumb{text-align:center;border:1px solid #ccc;margin:2px}div.gallerytext{overflow:hidden;font-size:94%;padding:2px 4px;word-wrap:break-word}table.mw-enhanced-rc{border:0;border-spacing:0}td.mw-enhanced-rc{white-space:nowrap;padding:0;vertical-align:top;font-family:monospace }#mw-addcategory-prompt{display:inline;margin-left:1em}#mw-addcategory-prompt input{margin-left:0.5em;margin-right:0.5em}.mw-remove-category{padding:8px;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA4AAAAOCAYAAAAfSC3RAAAACXBIWXMAAAsTAAALEwEAmpwYAAAKT2lDQ1BQaG90b3Nob3AgSUNDIHByb2ZpbGUAAHjanVNnVFPpFj333vRCS4iAlEtvUhUIIFJCi4AUkSYqIQkQSoghodkVUcERRUUEG8igiAOOjoCMFVEsDIoK2AfkIaKOg6OIisr74Xuja9a89+bN/rXXPues852zzwfACAyWSDNRNYAMqUIeEeCDx8TG4eQuQIEKJHAAEAizZCFz/SMBAPh+PDwrIsAHvgABeNMLCADATZvAMByH/w/qQplcAYCEAcB0kThLCIAUAEB6jkKmAEBGAYCdmCZTAKAEAGDLY2LjAFAtAGAnf+bTAICd+Jl7AQBblCEVAaCRACATZYhEAGg7AKzPVopFAFgwABRmS8Q5ANgtADBJV2ZIALC3AMDOEAuyAAgMADBRiIUpAAR7AGDIIyN4AISZABRG8lc88SuuEOcqAAB4mbI8uSQ5RYFbCC1xB1dXLh4ozkkXKxQ2YQJhmkAuwnmZGTKBNA/g88wAAKCRFRHgg/P9eM4Ors7ONo62Dl8t6r8G/yJiYuP+5c+rcEAAAOF0ftH+LC+zGoA7BoBt/qIl7gRoXgugdfeLZrIPQLUAoOnaV/Nw+H48PEWhkLnZ2eXk5NhKxEJbYcpXff5nwl/AV/1s+X48/Pf14L7iJIEyXYFHBPjgwsz0TKUcz5IJhGLc5o9H/LcL//wd0yLESWK5WCoU41EScY5EmozzMqUiiUKSKcUl0v9k4t8s+wM+3zUAsGo+AXuRLahdYwP2SycQWHTA4vcAAPK7b8HUKAgDgGiD4c93/+8//UegJQCAZkmScQAAXkQkLlTKsz/HCAAARKCBKrBBG/TBGCzABhzBBdzBC/xgNoRCJMTCQhBCCmSAHHJgKayCQiiGzbAdKmAv1EAdNMBRaIaTcA4uwlW4Dj1wD/phCJ7BKLyBCQRByAgTYSHaiAFiilgjjggXmYX4IcFIBBKLJCDJiBRRIkuRNUgxUopUIFVIHfI9cgI5h1xGupE7yAAygvyGvEcxlIGyUT3UDLVDuag3GoRGogvQZHQxmo8WoJvQcrQaPYw2oefQq2gP2o8+Q8cwwOgYBzPEbDAuxsNCsTgsCZNjy7EirAyrxhqwVqwDu4n1Y8+xdwQSgUXACTYEd0IgYR5BSFhMWE7YSKggHCQ0EdoJNwkDhFHCJyKTqEu0JroR+cQYYjIxh1hILCPWEo8TLxB7iEPENyQSiUMyJ7mQAkmxpFTSEtJG0m5SI+ksqZs0SBojk8naZGuyBzmULCAryIXkneTD5DPkG+Qh8lsKnWJAcaT4U+IoUspqShnlEOU05QZlmDJBVaOaUt2ooVQRNY9aQq2htlKvUYeoEzR1mjnNgxZJS6WtopXTGmgXaPdpr+h0uhHdlR5Ol9BX0svpR+iX6AP0dwwNhhWDx4hnKBmbGAcYZxl3GK+YTKYZ04sZx1QwNzHrmOeZD5lvVVgqtip8FZHKCpVKlSaVGyovVKmqpqreqgtV81XLVI+pXlN9rkZVM1PjqQnUlqtVqp1Q61MbU2epO6iHqmeob1Q/pH5Z/YkGWcNMw09DpFGgsV/jvMYgC2MZs3gsIWsNq4Z1gTXEJrHN2Xx2KruY/R27iz2qqaE5QzNKM1ezUvOUZj8H45hx+Jx0TgnnKKeX836K3hTvKeIpG6Y0TLkxZVxrqpaXllirSKtRq0frvTau7aedpr1Fu1n7gQ5Bx0onXCdHZ4/OBZ3nU9lT3acKpxZNPTr1ri6qa6UbobtEd79up+6Ynr5egJ5Mb6feeb3n+hx9L/1U/W36p/VHDFgGswwkBtsMzhg8xTVxbzwdL8fb8VFDXcNAQ6VhlWGX4YSRudE8o9VGjUYPjGnGXOMk423GbcajJgYmISZLTepN7ppSTbmmKaY7TDtMx83MzaLN1pk1mz0x1zLnm+eb15vft2BaeFostqi2uGVJsuRaplnutrxuhVo5WaVYVVpds0atna0l1rutu6cRp7lOk06rntZnw7Dxtsm2qbcZsOXYBtuutm22fWFnYhdnt8Wuw+6TvZN9un2N/T0HDYfZDqsdWh1+c7RyFDpWOt6azpzuP33F9JbpL2dYzxDP2DPjthPLKcRpnVOb00dnF2e5c4PziIuJS4LLLpc+Lpsbxt3IveRKdPVxXeF60vWdm7Obwu2o26/uNu5p7ofcn8w0nymeWTNz0MPIQ+BR5dE/C5+VMGvfrH5PQ0+BZ7XnIy9jL5FXrdewt6V3qvdh7xc+9j5yn+M+4zw33jLeWV/MN8C3yLfLT8Nvnl+F30N/I/9k/3r/0QCngCUBZwOJgUGBWwL7+Hp8Ib+OPzrbZfay2e1BjKC5QRVBj4KtguXBrSFoyOyQrSH355jOkc5pDoVQfujW0Adh5mGLw34MJ4WHhVeGP45wiFga0TGXNXfR3ENz30T6RJZE3ptnMU85ry1KNSo+qi5qPNo3ujS6P8YuZlnM1VidWElsSxw5LiquNm5svt/87fOH4p3iC+N7F5gvyF1weaHOwvSFpxapLhIsOpZATIhOOJTwQRAqqBaMJfITdyWOCnnCHcJnIi/RNtGI2ENcKh5O8kgqTXqS7JG8NXkkxTOlLOW5hCepkLxMDUzdmzqeFpp2IG0yPTq9MYOSkZBxQqohTZO2Z+pn5mZ2y6xlhbL+xW6Lty8elQfJa7OQrAVZLQq2QqboVFoo1yoHsmdlV2a/zYnKOZarnivN7cyzytuQN5zvn//tEsIS4ZK2pYZLVy0dWOa9rGo5sjxxedsK4xUFK4ZWBqw8uIq2Km3VT6vtV5eufr0mek1rgV7ByoLBtQFr6wtVCuWFfevc1+1dT1gvWd+1YfqGnRs+FYmKrhTbF5cVf9go3HjlG4dvyr+Z3JS0qavEuWTPZtJm6ebeLZ5bDpaql+aXDm4N2dq0Dd9WtO319kXbL5fNKNu7g7ZDuaO/PLi8ZafJzs07P1SkVPRU+lQ27tLdtWHX+G7R7ht7vPY07NXbW7z3/T7JvttVAVVN1WbVZftJ+7P3P66Jqun4lvttXa1ObXHtxwPSA/0HIw6217nU1R3SPVRSj9Yr60cOxx++/p3vdy0NNg1VjZzG4iNwRHnk6fcJ3/ceDTradox7rOEH0x92HWcdL2pCmvKaRptTmvtbYlu6T8w+0dbq3nr8R9sfD5w0PFl5SvNUyWna6YLTk2fyz4ydlZ19fi753GDborZ752PO32oPb++6EHTh0kX/i+c7vDvOXPK4dPKy2+UTV7hXmq86X23qdOo8/pPTT8e7nLuarrlca7nuer21e2b36RueN87d9L158Rb/1tWeOT3dvfN6b/fF9/XfFt1+cif9zsu72Xcn7q28T7xf9EDtQdlD3YfVP1v+3Njv3H9qwHeg89HcR/cGhYPP/pH1jw9DBY+Zj8uGDYbrnjg+OTniP3L96fynQ89kzyaeF/6i/suuFxYvfvjV69fO0ZjRoZfyl5O/bXyl/erA6xmv28bCxh6+yXgzMV70VvvtwXfcdx3vo98PT+R8IH8o/2j5sfVT0Kf7kxmTk/8EA5jz/GMzLdsAAAAgY0hSTQAAeiUAAICDAAD5/wAAgOkAAHUwAADqYAAAOpgAABdvkl/FRgAAAfJJREFUeF5tkk9rE0EYxp/JzE72X1LIwXrwYD2LEoR4Eq/qUfFb2JP6QewlYA+GCIVekiXWROu1tM2lNgaxqdhYQ1OhMZDshijZndfNxuSyDgzzMPye+R3eARFFu1Qqod3+ntvd3eu8rVZbxWIxG+YrZcfZdBxn6+DjwXKYF/wiFAqFOy/W1lreeETDoUu1d7VxfX//98VFj9yRRy/X17+8KhTuxopPnz3fqbzZouanJlWrNSoWX9Nho0Hb77fp9EeHNjY26cnq6s6cT+Dfkprmrly7isFggEwmg6Suw3WHuHR5GUoFUCBIoblzXswDhUtKCcMwYFk6zpMasjezsGwLigh79TqIgWJFAGziK/T6PfT7wNHnI+Ru5yKzH/jgEGAAixsZ4PsBdN0ETSZIL6UxU6jIKDgAxhA3KiAIfCSlhj9QMGwLLMGiexYogPMIixcZgaKiDM0KtmFOXwISgB9MwBMc7L9FmnIEaciwHBoNHcFsXtHJOQcRxYtcCAkoSKEBJoNp2UDEsVCsIkZKqc35xRy73e5xMjkdhYWlVBop24YQYra5gCYllKJ2rJhK2ZVG49Drnv+EN/IiS6/3C2dnXZycfMPX41Zf00QlNg7TND88uH/vRrlcXjFM8xYDrjPGOIBm+Jvqjx89PM3n8505/xfZkwoy8Sv0egAAAABJRU5ErkJggg==);background-image:url(remove.png)!ie;background-position:center center;background-repeat:no-repeat}.mw-ajax-addcategory{padding-left:20px;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAMCAYAAABWdVznAAAACXBIWXMAAAsTAAALEwEAmpwYAAAKT2lDQ1BQaG90b3Nob3AgSUNDIHByb2ZpbGUAAHjanVNnVFPpFj333vRCS4iAlEtvUhUIIFJCi4AUkSYqIQkQSoghodkVUcERRUUEG8igiAOOjoCMFVEsDIoK2AfkIaKOg6OIisr74Xuja9a89+bN/rXXPues852zzwfACAyWSDNRNYAMqUIeEeCDx8TG4eQuQIEKJHAAEAizZCFz/SMBAPh+PDwrIsAHvgABeNMLCADATZvAMByH/w/qQplcAYCEAcB0kThLCIAUAEB6jkKmAEBGAYCdmCZTAKAEAGDLY2LjAFAtAGAnf+bTAICd+Jl7AQBblCEVAaCRACATZYhEAGg7AKzPVopFAFgwABRmS8Q5ANgtADBJV2ZIALC3AMDOEAuyAAgMADBRiIUpAAR7AGDIIyN4AISZABRG8lc88SuuEOcqAAB4mbI8uSQ5RYFbCC1xB1dXLh4ozkkXKxQ2YQJhmkAuwnmZGTKBNA/g88wAAKCRFRHgg/P9eM4Ors7ONo62Dl8t6r8G/yJiYuP+5c+rcEAAAOF0ftH+LC+zGoA7BoBt/qIl7gRoXgugdfeLZrIPQLUAoOnaV/Nw+H48PEWhkLnZ2eXk5NhKxEJbYcpXff5nwl/AV/1s+X48/Pf14L7iJIEyXYFHBPjgwsz0TKUcz5IJhGLc5o9H/LcL//wd0yLESWK5WCoU41EScY5EmozzMqUiiUKSKcUl0v9k4t8s+wM+3zUAsGo+AXuRLahdYwP2SycQWHTA4vcAAPK7b8HUKAgDgGiD4c93/+8//UegJQCAZkmScQAAXkQkLlTKsz/HCAAARKCBKrBBG/TBGCzABhzBBdzBC/xgNoRCJMTCQhBCCmSAHHJgKayCQiiGzbAdKmAv1EAdNMBRaIaTcA4uwlW4Dj1wD/phCJ7BKLyBCQRByAgTYSHaiAFiilgjjggXmYX4IcFIBBKLJCDJiBRRIkuRNUgxUopUIFVIHfI9cgI5h1xGupE7yAAygvyGvEcxlIGyUT3UDLVDuag3GoRGogvQZHQxmo8WoJvQcrQaPYw2oefQq2gP2o8+Q8cwwOgYBzPEbDAuxsNCsTgsCZNjy7EirAyrxhqwVqwDu4n1Y8+xdwQSgUXACTYEd0IgYR5BSFhMWE7YSKggHCQ0EdoJNwkDhFHCJyKTqEu0JroR+cQYYjIxh1hILCPWEo8TLxB7iEPENyQSiUMyJ7mQAkmxpFTSEtJG0m5SI+ksqZs0SBojk8naZGuyBzmULCAryIXkneTD5DPkG+Qh8lsKnWJAcaT4U+IoUspqShnlEOU05QZlmDJBVaOaUt2ooVQRNY9aQq2htlKvUYeoEzR1mjnNgxZJS6WtopXTGmgXaPdpr+h0uhHdlR5Ol9BX0svpR+iX6AP0dwwNhhWDx4hnKBmbGAcYZxl3GK+YTKYZ04sZx1QwNzHrmOeZD5lvVVgqtip8FZHKCpVKlSaVGyovVKmqpqreqgtV81XLVI+pXlN9rkZVM1PjqQnUlqtVqp1Q61MbU2epO6iHqmeob1Q/pH5Z/YkGWcNMw09DpFGgsV/jvMYgC2MZs3gsIWsNq4Z1gTXEJrHN2Xx2KruY/R27iz2qqaE5QzNKM1ezUvOUZj8H45hx+Jx0TgnnKKeX836K3hTvKeIpG6Y0TLkxZVxrqpaXllirSKtRq0frvTau7aedpr1Fu1n7gQ5Bx0onXCdHZ4/OBZ3nU9lT3acKpxZNPTr1ri6qa6UbobtEd79up+6Ynr5egJ5Mb6feeb3n+hx9L/1U/W36p/VHDFgGswwkBtsMzhg8xTVxbzwdL8fb8VFDXcNAQ6VhlWGX4YSRudE8o9VGjUYPjGnGXOMk423GbcajJgYmISZLTepN7ppSTbmmKaY7TDtMx83MzaLN1pk1mz0x1zLnm+eb15vft2BaeFostqi2uGVJsuRaplnutrxuhVo5WaVYVVpds0atna0l1rutu6cRp7lOk06rntZnw7Dxtsm2qbcZsOXYBtuutm22fWFnYhdnt8Wuw+6TvZN9un2N/T0HDYfZDqsdWh1+c7RyFDpWOt6azpzuP33F9JbpL2dYzxDP2DPjthPLKcRpnVOb00dnF2e5c4PziIuJS4LLLpc+Lpsbxt3IveRKdPVxXeF60vWdm7Obwu2o26/uNu5p7ofcn8w0nymeWTNz0MPIQ+BR5dE/C5+VMGvfrH5PQ0+BZ7XnIy9jL5FXrdewt6V3qvdh7xc+9j5yn+M+4zw33jLeWV/MN8C3yLfLT8Nvnl+F30N/I/9k/3r/0QCngCUBZwOJgUGBWwL7+Hp8Ib+OPzrbZfay2e1BjKC5QRVBj4KtguXBrSFoyOyQrSH355jOkc5pDoVQfujW0Adh5mGLw34MJ4WHhVeGP45wiFga0TGXNXfR3ENz30T6RJZE3ptnMU85ry1KNSo+qi5qPNo3ujS6P8YuZlnM1VidWElsSxw5LiquNm5svt/87fOH4p3iC+N7F5gvyF1weaHOwvSFpxapLhIsOpZATIhOOJTwQRAqqBaMJfITdyWOCnnCHcJnIi/RNtGI2ENcKh5O8kgqTXqS7JG8NXkkxTOlLOW5hCepkLxMDUzdmzqeFpp2IG0yPTq9MYOSkZBxQqohTZO2Z+pn5mZ2y6xlhbL+xW6Lty8elQfJa7OQrAVZLQq2QqboVFoo1yoHsmdlV2a/zYnKOZarnivN7cyzytuQN5zvn//tEsIS4ZK2pYZLVy0dWOa9rGo5sjxxedsK4xUFK4ZWBqw8uIq2Km3VT6vtV5eufr0mek1rgV7ByoLBtQFr6wtVCuWFfevc1+1dT1gvWd+1YfqGnRs+FYmKrhTbF5cVf9go3HjlG4dvyr+Z3JS0qavEuWTPZtJm6ebeLZ5bDpaql+aXDm4N2dq0Dd9WtO319kXbL5fNKNu7g7ZDuaO/PLi8ZafJzs07P1SkVPRU+lQ27tLdtWHX+G7R7ht7vPY07NXbW7z3/T7JvttVAVVN1WbVZftJ+7P3P66Jqun4lvttXa1ObXHtxwPSA/0HIw6217nU1R3SPVRSj9Yr60cOxx++/p3vdy0NNg1VjZzG4iNwRHnk6fcJ3/ceDTradox7rOEH0x92HWcdL2pCmvKaRptTmvtbYlu6T8w+0dbq3nr8R9sfD5w0PFl5SvNUyWna6YLTk2fyz4ydlZ19fi753GDborZ752PO32oPb++6EHTh0kX/i+c7vDvOXPK4dPKy2+UTV7hXmq86X23qdOo8/pPTT8e7nLuarrlca7nuer21e2b36RueN87d9L158Rb/1tWeOT3dvfN6b/fF9/XfFt1+cif9zsu72Xcn7q28T7xf9EDtQdlD3YfVP1v+3Njv3H9qwHeg89HcR/cGhYPP/pH1jw9DBY+Zj8uGDYbrnjg+OTniP3L96fynQ89kzyaeF/6i/suuFxYvfvjV69fO0ZjRoZfyl5O/bXyl/erA6xmv28bCxh6+yXgzMV70VvvtwXfcdx3vo98PT+R8IH8o/2j5sfVT0Kf7kxmTk/8EA5jz/GMzLdsAAAAgY0hSTQAAeiUAAICDAAD5/wAAgOkAAHUwAADqYAAAOpgAABdvkl/FRgAAAcFJREFUeF4lkUtOVEEYRs9fVffVNNCghNegcUIwColhosEH0aGrYAdq4hjYgYkLYCUmTFyAUWNgIJooQcUW6HC7b9269ctjfk6+k3yy9vIxqgoCRLCp2Ry90d4wajnr9XdqX2+LkStGRDCoIF4Iw4bq3JOPZfNLz7vd5UeL3WIkn6+GFRcSTWhQBQMKUaCGxkdsy4ViMSXOesRJ0EbRqFxHgEEBL1dwVVXEQeMzMvrpKXVdew1CEyLXC4rTLG4ls8xMFxPR9aeb1kK2/o9jPfE9ik72bCpMvk2LxF5IpizLI1nZXP7Ze/p97tbqPCuySm09B78PqD8qnbRDe6LNeDFG88tw8P7w0F2m/P1Qcnryhb25fWIE2UuYym+SdhPysZxBXuKHkWAD8uD12laVDmaaxEeJtqnbfv3b+P6d0duWpU/3Pyd/il1ytdFHU535I+dCspWXI5hgoDYMZvtvintf77YXcsJufFfu9180WcRYIWk5nIoSTYPaCKr4zKeMRrLJFNuyqXUWSQJYQLg+TrmUAmoVG4xzvRz9YVAfnUnAGouIoIA8fPUEjQoCKGgSN887ZxtJ4siOR3aoZBsBABXlP6kY44hxREHRAAAAAElFTkSuQmCC);background-image:url(add.png)!ie;background-position:left center;background-repeat:no-repeat}.mw-ajax-loader{background-image:url(data:image/gif;base64,R0lGODlhIAAgAPMAAP///wAAAMbGxoSEhLa2tpqamjY2NlZWVtjY2OTk5Ly8vB4eHgQEBAAAAAAAAAAAACH/C05FVFNDQVBFMi4wAwEAAAAh/hpDcmVhdGVkIHdpdGggYWpheGxvYWQuaW5mbwAh+QQJCgAAACwAAAAAIAAgAAAE5xDISWlhperN52JLhSSdRgwVo1ICQZRUsiwHpTJT4iowNS8vyW2icCF6k8HMMBkCEDskxTBDAZwuAkkqIfxIQyhBQBFvAQSDITM5VDW6XNE4KagNh6Bgwe60smQUB3d4Rz1ZBApnFASDd0hihh12BkE9kjAJVlycXIg7CQIFA6SlnJ87paqbSKiKoqusnbMdmDC2tXQlkUhziYtyWTxIfy6BE8WJt5YJvpJivxNaGmLHT0VnOgSYf0dZXS7APdpB309RnHOG5gDqXGLDaC457D1zZ/V/nmOM82XiHRLYKhKP1oZmADdEAAAh+QQJCgAAACwAAAAAIAAgAAAE6hDISWlZpOrNp1lGNRSdRpDUolIGw5RUYhhHukqFu8DsrEyqnWThGvAmhVlteBvojpTDDBUEIFwMFBRAmBkSgOrBFZogCASwBDEY/CZSg7GSE0gSCjQBMVG023xWBhklAnoEdhQEfyNqMIcKjhRsjEdnezB+A4k8gTwJhFuiW4dokXiloUepBAp5qaKpp6+Ho7aWW54wl7obvEe0kRuoplCGepwSx2jJvqHEmGt6whJpGpfJCHmOoNHKaHx61WiSR92E4lbFoq+B6QDtuetcaBPnW6+O7wDHpIiK9SaVK5GgV543tzjgGcghAgAh+QQJCgAAACwAAAAAIAAgAAAE7hDISSkxpOrN5zFHNWRdhSiVoVLHspRUMoyUakyEe8PTPCATW9A14E0UvuAKMNAZKYUZCiBMuBakSQKG8G2FzUWox2AUtAQFcBKlVQoLgQReZhQlCIJesQXI5B0CBnUMOxMCenoCfTCEWBsJColTMANldx15BGs8B5wlCZ9Po6OJkwmRpnqkqnuSrayqfKmqpLajoiW5HJq7FL1Gr2mMMcKUMIiJgIemy7xZtJsTmsM4xHiKv5KMCXqfyUCJEonXPN2rAOIAmsfB3uPoAK++G+w48edZPK+M6hLJpQg484enXIdQFSS1u6UhksENEQAAIfkECQoAAAAsAAAAACAAIAAABOcQyEmpGKLqzWcZRVUQnZYg1aBSh2GUVEIQ2aQOE+G+cD4ntpWkZQj1JIiZIogDFFyHI0UxQwFugMSOFIPJftfVAEoZLBbcLEFhlQiqGp1Vd140AUklUN3eCA51C1EWMzMCezCBBmkxVIVHBWd3HHl9JQOIJSdSnJ0TDKChCwUJjoWMPaGqDKannasMo6WnM562R5YluZRwur0wpgqZE7NKUm+FNRPIhjBJxKZteWuIBMN4zRMIVIhffcgojwCF117i4nlLnY5ztRLsnOk+aV+oJY7V7m76PdkS4trKcdg0Zc0tTcKkRAAAIfkECQoAAAAsAAAAACAAIAAABO4QyEkpKqjqzScpRaVkXZWQEximw1BSCUEIlDohrft6cpKCk5xid5MNJTaAIkekKGQkWyKHkvhKsR7ARmitkAYDYRIbUQRQjWBwJRzChi9CRlBcY1UN4g0/VNB0AlcvcAYHRyZPdEQFYV8ccwR5HWxEJ02YmRMLnJ1xCYp0Y5idpQuhopmmC2KgojKasUQDk5BNAwwMOh2RtRq5uQuPZKGIJQIGwAwGf6I0JXMpC8C7kXWDBINFMxS4DKMAWVWAGYsAdNqW5uaRxkSKJOZKaU3tPOBZ4DuK2LATgJhkPJMgTwKCdFjyPHEnKxFCDhEAACH5BAkKAAAALAAAAAAgACAAAATzEMhJaVKp6s2nIkolIJ2WkBShpkVRWqqQrhLSEu9MZJKK9y1ZrqYK9WiClmvoUaF8gIQSNeF1Er4MNFn4SRSDARWroAIETg1iVwuHjYB1kYc1mwruwXKC9gmsJXliGxc+XiUCby9ydh1sOSdMkpMTBpaXBzsfhoc5l58Gm5yToAaZhaOUqjkDgCWNHAULCwOLaTmzswadEqggQwgHuQsHIoZCHQMMQgQGubVEcxOPFAcMDAYUA85eWARmfSRQCdcMe0zeP1AAygwLlJtPNAAL19DARdPzBOWSm1brJBi45soRAWQAAkrQIykShQ9wVhHCwCQCACH5BAkKAAAALAAAAAAgACAAAATrEMhJaVKp6s2nIkqFZF2VIBWhUsJaTokqUCoBq+E71SRQeyqUToLA7VxF0JDyIQh/MVVPMt1ECZlfcjZJ9mIKoaTl1MRIl5o4CUKXOwmyrCInCKqcWtvadL2SYhyASyNDJ0uIiRMDjI0Fd30/iI2UA5GSS5UDj2l6NoqgOgN4gksEBgYFf0FDqKgHnyZ9OX8HrgYHdHpcHQULXAS2qKpENRg7eAMLC7kTBaixUYFkKAzWAAnLC7FLVxLWDBLKCwaKTULgEwbLA4hJtOkSBNqITT3xEgfLpBtzE/jiuL04RGEBgwWhShRgQExHBAAh+QQJCgAAACwAAAAAIAAgAAAE7xDISWlSqerNpyJKhWRdlSAVoVLCWk6JKlAqAavhO9UkUHsqlE6CwO1cRdCQ8iEIfzFVTzLdRAmZX3I2SfZiCqGk5dTESJeaOAlClzsJsqwiJwiqnFrb2nS9kmIcgEsjQydLiIlHehhpejaIjzh9eomSjZR+ipslWIRLAgMDOR2DOqKogTB9pCUJBagDBXR6XB0EBkIIsaRsGGMMAxoDBgYHTKJiUYEGDAzHC9EACcUGkIgFzgwZ0QsSBcXHiQvOwgDdEwfFs0sDzt4S6BK4xYjkDOzn0unFeBzOBijIm1Dgmg5YFQwsCMjp1oJ8LyIAACH5BAkKAAAALAAAAAAgACAAAATwEMhJaVKp6s2nIkqFZF2VIBWhUsJaTokqUCoBq+E71SRQeyqUToLA7VxF0JDyIQh/MVVPMt1ECZlfcjZJ9mIKoaTl1MRIl5o4CUKXOwmyrCInCKqcWtvadL2SYhyASyNDJ0uIiUd6GGl6NoiPOH16iZKNlH6KmyWFOggHhEEvAwwMA0N9GBsEC6amhnVcEwavDAazGwIDaH1ipaYLBUTCGgQDA8NdHz0FpqgTBwsLqAbWAAnIA4FWKdMLGdYGEgraigbT0OITBcg5QwPT4xLrROZL6AuQAPUS7bxLpoWidY0JtxLHKhwwMJBTHgPKdEQAACH5BAkKAAAALAAAAAAgACAAAATrEMhJaVKp6s2nIkqFZF2VIBWhUsJaTokqUCoBq+E71SRQeyqUToLA7VxF0JDyIQh/MVVPMt1ECZlfcjZJ9mIKoaTl1MRIl5o4CUKXOwmyrCInCKqcWtvadL2SYhyASyNDJ0uIiUd6GAULDJCRiXo1CpGXDJOUjY+Yip9DhToJA4RBLwMLCwVDfRgbBAaqqoZ1XBMHswsHtxtFaH1iqaoGNgAIxRpbFAgfPQSqpbgGBqUD1wBXeCYp1AYZ19JJOYgH1KwA4UBvQwXUBxPqVD9L3sbp2BNk2xvvFPJd+MFCN6HAAIKgNggY0KtEBAAh+QQJCgAAACwAAAAAIAAgAAAE6BDISWlSqerNpyJKhWRdlSAVoVLCWk6JKlAqAavhO9UkUHsqlE6CwO1cRdCQ8iEIfzFVTzLdRAmZX3I2SfYIDMaAFdTESJeaEDAIMxYFqrOUaNW4E4ObYcCXaiBVEgULe0NJaxxtYksjh2NLkZISgDgJhHthkpU4mW6blRiYmZOlh4JWkDqILwUGBnE6TYEbCgevr0N1gH4At7gHiRpFaLNrrq8HNgAJA70AWxQIH1+vsYMDAzZQPC9VCNkDWUhGkuE5PxJNwiUK4UfLzOlD4WvzAHaoG9nxPi5d+jYUqfAhhykOFwJWiAAAIfkECQoAAAAsAAAAACAAIAAABPAQyElpUqnqzaciSoVkXVUMFaFSwlpOCcMYlErAavhOMnNLNo8KsZsMZItJEIDIFSkLGQoQTNhIsFehRww2CQLKF0tYGKYSg+ygsZIuNqJksKgbfgIGepNo2cIUB3V1B3IvNiBYNQaDSTtfhhx0CwVPI0UJe0+bm4g5VgcGoqOcnjmjqDSdnhgEoamcsZuXO1aWQy8KAwOAuTYYGwi7w5h+Kr0SJ8MFihpNbx+4Erq7BYBuzsdiH1jCAzoSfl0rVirNbRXlBBlLX+BP0XJLAPGzTkAuAOqb0WT5AH7OcdCm5B8TgRwSRKIHQtaLCwg1RAAAOwAAAAAAAAAAAA==);background-image:url(ajax-loader.gif)!ie;background-position:center center;background-repeat:no-repeat;padding:16px;position:relative;top:-16px}.mw-small-spinner{padding:10px !important;margin-right:0.6em;background-image:url(data:image/gif;base64,R0lGODlhFAAUAPUyAAEBAQICAgMDAwQEBAcHBwkJCSIiIigoKCwsLDQ0ND8/P0REREVFRU1NTVJSUlVVVVZWVl1dXWNjY25ubnBwcHR0dHh4eISEhIWFhYeHh4mJiZKSkpaWlpubm6Wlpaqqqra2tre3t7i4uLm5ubq6uru7u7+/v8DAwMLCwsPDw8TExMbGxsfHx8jIyMnJycrKys7OztDQ0P///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkKADIAIf4aQ3JlYXRlZCB3aXRoIGFqYXhsb2FkLmluZm8AIf8LTkVUU0NBUEUyLjADAQAAACwAAAAAFAAUAAAGlUCZcDhMOIhIImcyJBSEqVRy6EBUhIGADEYyTYcHhZCgTZG+w4fBIgsQZCSp8Cx8NIYKCbElXJFIMXUMDBEeX38pMEgPDBRfKytfG2hJHxoXGRmUIJwgKx2ZmJudipSmfXxTfolEMGZ0U69yMX+RMqlCLbAmcnBDZjKcMn62aHHBIFCwUyYkisJbf2hRQtAygadbxUlBACH5BAkKADcALAAAAAAUABQAhRISEhQUFBYWFh0dHR4eHiEhISIiIiMjIykpKSwsLC0tLS8vLzY2Njo6Oj8/P0FBQUhISEpKSlRUVFdXV2RkZGZmZm1tbW9vb3Nzc35+fn9/f4eHh4mJiYyMjJGRkZSUlJiYmJ2dnZ6enqOjo6SkpLa2tre3t7i4uLm5ubq6uru7u7y8vL+/v8DAwMLCwsPDw8TExMbGxsfHx8jIyMrKys7OztDQ0P///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAaQwJtwOIxciEhiKDMsFISLRXJYeTBvBIIQgJgOHxJh9rYAeKmNDbZwCTCGqiHIM5RgiBFhTKWyCUkdHCExXnwvNUgfHSNeMy+FZ0k1KiWVkZWVMTGYJZeYiJGhQi8zXnuHRDUvKitnqyuPN5MqhDelQ3tDsHBDqzeWe7VnrL+dN76uKoiWsnyutcw3fqKywklBACH5BAkKADkALAAAAAAUABQAhQoKChAQEBERESgoKCoqKiwsLC4uLjU1NTY2Njc3Nzo6Ojw8PD09PT4+PkBAQENDQ0RERElJSVBQUFlZWWFhYWJiYmVlZXx8fIKCgoaGhoqKioyMjI+Pj5GRkZWVlZiYmJ6enqSkpKenp7Ozs7a2tre3t7i4uLm5ubq6uru7u7y8vL+/v8DAwMLCwsPDw8TExMXFxcbGxsfHx8jIyMrKys7OztDQ0NTU1NnZ2f///wAAAAAAAAAAAAAAAAAAAAAAAAaZwJxwOKRciEgibDRUMIQRSHII4pSEiYSw8JwKNR2sllHwDkMaZkLBOUSGDoywtRp6QsSJJ8cQCChzKSkwOF4EAAQWSC0pM14QgFM3Zkk1JZdXZpglMDCblJgpNZSkQy2OUzApK6NDNYx1XowpLUI2gjBCqLopQyu1Qr2BOZc5qrmUq8SZK8JezaPFObfOSS3AKZnTpUI1yFNBACH5BAkKADoALAAAAAAUABQAhQAAAAICAgMDAwQEBAcHBwkJCSIiIigoKCsrKzQ0ND4+Pj8/P0REREVFRU1NTVJSUlNTU1RUVFVVVVZWVlxcXGNjY25ubnFxcXR0dHh4eIODg4SEhIWFhYeHh4mJiZGRkZaWlpubm6Wlpaqqqra2tre3t7i4uLm5ubq6uru7u7y8vL+/v8DAwMLCwsPDw8TExMbGxsfHx8jIyMrKys7Ozs/Pz9DQ0NTU1NfX19nZ2f///wAAAAAAAAAAAAAAAAAAAAaZQJ1wOAyNiEhizjbseIQSSXLYSjF1Go2Q4ZgOUythp6OTMLxDWIqG7XwYlKEEJGyFhasZ8SIqHxAWdSkpMDheCwYMF0hVMF4VGV6GaEg0JJcklAEDnAkwmJloAaMFD5SnaY5TM2BsQzYtJHdTVSktQjaDqnppKUMrt0K+gjqXOmqqaGDFoVWUK2vMQjSDaC3BKaE6V6g0yUlBACH5BAkKADoALAAAAAAUABQAhRISEhQUFBYWFhwcHB0dHR4eHiEhISIiIiMjIysrKy8vLzc3Nzo6Oj8/P0FBQUhISFJSUlZWVldXV2VlZW1tbX9/f4KCgoeHh4mJiYuLi4yMjJCQkJSUlJiYmJmZmZqampycnJ2dnZ6enqOjo7CwsLa2tre3t7i4uLm5ubq6uru7u7y8vL+/v8DAwMLCwsPDw8TExMbGxsfHx8jIyMrKys7OztDQ0NTU1NfX19nZ2f///wAAAAAAAAAAAAAAAAAAAAaaQJ1wOCSViEhirjYMfYSfZ1LoUjF1UWFGMx2qXNAnJ9MdxqzYj0oTGk7aOhdryKIRRzHdxNGwUFUqMThdEAwRFUhVeVMUF12DZUg1JZRHZQMImA8xlZZdmZgUkUh+UzGLSQ4CChFENS4lc10LAAAKQjaAi3ZmKkMIt0K+fzqUOmeoXSpzxnHDXSxozTWAZS5gOiqeNqNDNclIQQAh+QQJCgBFACwAAAAAFAAUAIYKCgoQEBAREREnJycqKiosLCwuLi40NDQ1NTU2NjY3Nzc5OTk6Ojo8PDw9PT0+Pj4/Pz9ERERJSUlRUVFYWFhhYWFlZWVmZmZvb298fHyCgoKGhoaHh4eKioqNjY2SkpKVlZWenp6fn5+hoaGioqKjo6Onp6epqamrq6usrKyurq6vr6+xsbGzs7O2tra3t7e4uLi5ubm6urq7u7u8vLy/v7/AwMDCwsLDw8PExMTFxcXGxsbHx8fIyMjKysrOzs7Q0NDU1NTV1dXX19fZ2dn///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHqYBFgoODOz+EiIREh4IvL4IqKomDNzOMjoIkKJODMzeNjyginIWWRY47JJKCIjuCNzWDNT6EN0NFIhsdq5UzO7eTIBsfLYiVrpMmxZPApIQ/jpikBwvUFTvRj6TVBxAZzogenDe0kxQGEBjPNy+xkyQTBAYSgkAzvoLlghEEgxARnSjNOPXIAoAF4GbEkkZAgLMapqRZEACB1I1PRWZoK1JBAzhBP5BNCgQAIfkECQoAOgAsAAAAABQAFACFAQEBAgICAwMDBAQEBwcHCQkJIiIiKCgoLCwsNDQ0Pz8/RERERUVFTU1NUlJSVVVVVlZWXV1dY2Njbm5ucHBwdHR0eHh4gICAg4ODhISEhYWFh4eHiYmJkpKSlpaWm5ubpaWlqqqqtra2t7e3uLi4ubm5urq6u7u7vLy8v7+/wMDAwsLCw8PDxMTExsbGx8fHyMjIysrKy8vLzs7Oz8/P0NDQ1NTU1dXV19fX2dnZ////AAAAAAAAAAAAAAAAAAAABptAnXA4dM2ISGLuKByNhKtVcrg61ZpP3Uk6FW6xumq3ODo6cScXNQdNDVMxYgync52s0LuLPk2NVkxUaV0xgUh8Y0QzTk6JGBuPHy6MWV0bkBwhiUgdXStxUxQMDxiKKyNuUyARDKNCNXpCoEISCkMND0MngjoDAToWBrmJJ24BA0IKB4kpJ0cBvzoVCA5jUUIFBUMTHptCDgljQQAh+QQJCgA9ACwAAAAAFAAUAIUSEhIUFBQVFRUdHR0eHh4fHx8hISEiIiIjIyMpKSksLCwtLS0vLy82NjY6Ojo/Pz9BQUFISEhUVFRWVlZkZGRmZmZtbW1xcXFzc3N8fHx/f3+Hh4eJiYmLi4uMjIyQkJCUlJSYmJidnZ2enp6jo6Orq6u2tra3t7e4uLi5ubm6urq7u7u8vLy/v7/AwMDCwsLDw8PExMTGxsbHx8fIyMjKysrLy8vOzs7Q0NDU1NTV1dXX19fZ2dn///8AAAAAAAAGmsCecDiU3YhIIu8oPJ2Er1dy+Frhms/eSjoVbrG9ard4Ojp3KxmVB20NWzVibdeTrazQu4w+bZ1eTFRpXTWBSHxjRDdOTomMJzIyj45OK4aJWl0vcVMkHh8lii8nblMyIh0dIUI4ekIQRBgSQx+rXkMNARgHAz0bDhWYAAo9A709E7BjCQFCvEIaD8FdCsQ9CAdDGiKYQhYRY0EAIfkECQoAPQAsAAAAABQAFACFCgoKEBAQERERKCgoKioqLCwsLi4uNTU1NjY2Nzc3Ojo6PDw8PT09Pj4+QEBAQ0NDRERESUlJUVFRWlpaW1tbYWFhYmJiZWVlfHx8goKChoaGh4eHioqKjIyMlZWVmZmZnp6en5+foaGhp6enr6+vsrKys7Oztra2t7e3uLi4ubm5urq6u7u7v7+/wMDAwsLCw8PDxMTExcXFxsbGyMjIysrKy8vLzs7O0NDQ1NTU1dXV19fX2dnZ////AAAAAAAABpjAnnA4nN2ISCLvKEShhK9Xcvhi4ZrPHks6FW6xvWq3iDo6d6wZlQdtDVs1Ym3Xm7Gs0PuMPm2hXkxUaV01gUh8Y0Q3Tk6JjCgzM4+OjIaJPYhIFg+EW4YXAwADXTt2Xz0UAgIKPSITRCMeb1wZnEISBh0KBz0mGiGXBg09B7w9HxqJDQZCxU0cwF0QEUINrEIkapc9GK9dQQAh+QQJCgA8ACwAAAAAFAAUAIUCAgIDAwMEBAQGBgYHBwcJCQkiIiIoKCgrKys0NDQ+Pj4/Pz9ERERFRUVNTU1SUlJTU1NUVFRVVVVWVlZcXFxjY2Nubm5xcXF0dHR4eHiDg4OEhISFhYWHh4eJiYmRkZGWlpabm5ulpaWqqqq2tra3t7e4uLi5ubm6urq7u7u/v7/AwMDCwsLDw8PExMTFxcXGxsbIyMjKysrLy8vOzs7Pz8/Q0NDU1NTV1dXX19fY2NjZ2dn///8AAAAAAAAAAAAGlUCecDiE0YhI4u4oLJWELFZyyErZmk9eSjoVbrG8ardYOjpzKRh1B1UNVTKiLMeDpazQO4w+VZVYTFRpXTKBSGxjSBAFAAADiU5OMAkDA42QkYaJPHxJGRWEW4YYCwYLXTl2XzwWCAcRPCIXcm5CKlwgEEMUDR8dGjw0g4kNsL95iQ4NQsfBq1MRsDweHkM2iJsjIWNBACH5BAkKADkALAAAAAAUABQAhQICAgMDAwQEBAcHBwgICAkJCRISEhQUFBYWFiIiIisrKy8vLzQ0NDU1NTo6Oj8/P0FBQVJSUldXV2VlZW1tbX9/f4ODg4iIiImJiYyMjJGRkZSUlJWVlZeXl52dnZ6enqOjo7a2tre3t7i4uLm5ubq6uru7u7+/v8DAwMLCwsPDw8TExMXFxcbGxsfHx8jIyMrKysvLy87OztDQ0NTU1NXV1dfX19jY2NnZ2f///wAAAAAAAAAAAAAAAAAAAAAAAAaYwJxwOGzJiEgi7igUiYQoVHKIOs2az9xIOhVusblqtzg6Om2nVhEnXCSGJxgRZsu1RqNrbnEwNDhdJ1t6Q3wQXTBMSRZjSREEAgIEjU5OLQ2RAgCUTnmNSHVTFxSIW4o5FhIOEV04dydcFQ8QE3YgcydUXB+1Qh8ZeE8yI2qNF4BOUCONGhlgOTOwYx23OclCM2yfOTLFU0EAIfkECQoAPgAsAAAAABQAFACFCgoKEBAQERERJycnKioqLCwsLi4uNDQ0NTU1NjY2Nzc3OTk5Ojo6PDw8PT09Pj4+Pz8/Q0NDRERESUlJUVFRWFhYYWFhYmJiZWVlfHx8goKChoaGioqKi4uLjY2NkZGRlZWVnp6eoaGhp6enr6+vsbGxs7Oztra2t7e3uLi4ubm5urq6u7u7v7+/wMDAwsLCw8PDxMTExcXFxsbGyMjIysrKy8vLzs7O0NDQ1NTU1dXV19fX2NjY2dnZ////AAAABplAn3A4nN2ISKLmMkShhDBYcvgIYIRO4ao1HQYI2Cds1R0yAEznbjUbwnpCCWM4kBBrO99stTr6JgYGFCJdLSgwflQGFV01iUgeZUkZDwcMc2VOTjMXDAefkpooj5I+eVMmI40riEQmHxsgXTt7rEIkHBwhpjV3XEItUnq7Qns7WTdspbZZPi1kZc9hQji2XVHT1HClPjdtXUEAOw==);background-image:url(spinner.gif)!ie;background-position:center center;background-repeat:no-repeat} a.sortheader{margin:0 0.3em} ol:lang(bcc) li,ol:lang(bqi) li,ol:lang(fa) li,ol:lang(glk) li,ol:lang(kk-arab) li,ol:lang(mzn) li{list-style-type:-moz-persian;list-style-type:persian}ol:lang(ckb) li{list-style-type:-moz-arabic-indic;list-style-type:arabic-indic}ol:lang(bn) li{list-style-type:-moz-bengali;list-style-type:bengali}ol:lang(or) li{list-style-type:-moz-oriya;list-style-type:oriya} .mw-help-field-hint{display:none;padding:0px;padding-left:15px;margin-left:2px;margin-bottom:-8px;background-image:url(data:image/gif;base64,R0lGODlhCwALALMAAP///01NTZOTk1lZWefn57i4uJSUlPPz82VlZdDQ0HFxcaysrNvb28TExAAAAAAAACH5BAAAAAAALAAAAAALAAsAAAQrUIRJqQQ455nNNBgHJANBDAwgZsVwqIG2IEQYYwXy2lq/Kg3NqqeSVCqCCAA7);background-image:url(help-question.gif)!ie;background-position:left center;background-repeat:no-repeat;color:#0645ad;text-decoration:underline;cursor:pointer;font-size:.8em}.mw-help-field-hint:hover{background-image:url(data:image/gif;base64,R0lGODlhCwALALMAAAtop+7z+GCWwpW51oStz8rb6yZzrafF3bnR5Nzn8QBcoD91oABQmf///wAAAAAAACH/C1hNUCBEYXRhWE1QPD94cGFja2V0IGJlZ2luPSLvu78iIGlkPSJXNU0wTXBDZWhpSHpyZVN6TlRjemtjOWQiPz4gPHg6eG1wbWV0YSB4bWxuczp4PSJhZG9iZTpuczptZXRhLyIgeDp4bXB0az0iQWRvYmUgWE1QIENvcmUgNS4wLWMwNjAgNjEuMTM0Nzc3LCAyMDEwLzAyLzEyLTE3OjMyOjAwICAgICAgICAiPiA8cmRmOlJERiB4bWxuczpyZGY9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1ucyMiPiA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0iIiB4bWxuczp4bXBNTT0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wL21tLyIgeG1sbnM6c3RSZWY9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9zVHlwZS9SZXNvdXJjZVJlZiMiIHhtbG5zOnhtcD0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wLyIgeG1wTU06T3JpZ2luYWxEb2N1bWVudElEPSJ4bXAuZGlkOjAyODAxMTc0MDcyMDY4MTE5NkQ0QUQzRjI0NzRCNUQwIiB4bXBNTTpEb2N1bWVudElEPSJ4bXAuZGlkOjJBN0FFQTQwQjlGQzExREY5RDlBQTRBODQyMkJCMkFDIiB4bXBNTTpJbnN0YW5jZUlEPSJ4bXAuaWlkOjJBN0FFQTNGQjlGQzExREY5RDlBQTRBODQyMkJCMkFDIiB4bXA6Q3JlYXRvclRvb2w9IkFkb2JlIFBob3Rvc2hvcCBDUzUgTWFjaW50b3NoIj4gPHhtcE1NOkRlcml2ZWRGcm9tIHN0UmVmOmluc3RhbmNlSUQ9InhtcC5paWQ6RkM3RjExNzQwNzIwNjgxMTk1RkVBQ0ZBOEQxNTU5MkUiIHN0UmVmOmRvY3VtZW50SUQ9InhtcC5kaWQ6MDI4MDExNzQwNzIwNjgxMTk2RDRBRDNGMjQ3NEI1RDAiLz4gPC9yZGY6RGVzY3JpcHRpb24+IDwvcmRmOlJERj4gPC94OnhtcG1ldGE+IDw/eHBhY2tldCBlbmQ9InIiPz4B//79/Pv6+fj39vX08/Lx8O/u7ezr6uno5+bl5OPi4eDf3t3c29rZ2NfW1dTT0tHQz87NzMvKycjHxsXEw8LBwL++vby7urm4t7a1tLOysbCvrq2sq6qpqKempaSjoqGgn56dnJuamZiXlpWUk5KRkI+OjYyLiomIh4aFhIOCgYB/fn18e3p5eHd2dXRzcnFwb25tbGtqaWhnZmVkY2JhYF9eXVxbWllYV1ZVVFNSUVBPTk1MS0pJSEdGRURDQkFAPz49PDs6OTg3NjU0MzIxMC8uLSwrKikoJyYlJCMiISAfHh0cGxoZGBcWFRQTEhEQDw4NDAsKCQgHBgUEAwIBAAAh+QQAAAAAACwAAAAACwALAAAEK3CxSalsOOeZxRQY1yBKkihFI2aDEqiMRgBJGGMD8NpavxoHzaqnklQqiwgAOw==);background-image:url(help-question-hover.gif)!ie}.mw-help-field-data{display:block;background-color:#d6f3ff;padding:5px 8px 4px 8px;border:1px solid #5dc9f4;margin-left:20px}.tipsy{padding:5px 5px 10px;font-size:12px;position:absolute;z-index:100000;overflow:visible}.tipsy-inner{padding:5px 8px 4px 8px;background-color:#d6f3ff;color:black;border:1px solid #5dc9f4;max-width:300px;text-align:left}.tipsy-arrow{position:absolute;background:url(data:image/gif;base64,R0lGODlhDQANAMQAAPf399bz/9vu9m/O9NXy/8Pm9svp9pfd+YLW943X9LTn++z093XQ9WnM9OLw9p/c9YTU9InY9/T292DK9Jre+afj+rvq/Nzv9rjk9brl9cPt/ZLb+GbL9MLs/ZHb+KLh+iH5BAAAAAAALAAAAAANAA0AAAVK4BGMZBkcg2WW1lBEKxkVAFTFFQQAwkSYhIlgB3hQTJQHEbBodEiaxmIJyHhIGwwVIGEoAgqGZAswIAIIA3mX+CTWOwfHAd9dtiEAOw==) no-repeat top left;background:url(tipsy-arrow.gif) no-repeat top left!ie;width:13px;height:13px}.tipsy-se .tipsy-arrow{bottom:-2px;right:10px;background-position:0% 100%}}@media screen{  html,body{height:100%;margin:0;padding:0;font-family:sans-serif;font-size:1em}body{background-color:#f3f3f3;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAABRJREFUeF4FwTEBAAAAwJD1D+weGQD4APc0a6VeAAAAAElFTkSuQmCC);background-image:url(page-base.png)!ie} div#content{margin-left:10em;padding:1em;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAABhJREFUeF4FwTEBAAAAgjD7FzESWfjYdgwEoAJ4lTsaxgAAAABJRU5ErkJggg==);background-image:url(border.png)!ie;background-position:top left;background-repeat:repeat-y;background-color:white;color:black;direction:ltr} #mw-page-base{height:5em;background-color:white;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAsCAIAAAArRUU2AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAADpJREFUeF5VjUkOAEAIwoD//7lzGJd4MJHGSoBImkFETP67CdLldUd7KC6f8fv3+psd8znbtU5x354HaWQjOx76v7MAAAAASUVORK5CYII=);background-image:url(page-fade.png)!ie;background-position:bottom left;background-repeat:repeat-x}#mw-head-base{margin-top:-5em;margin-left:10em;height:5em;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAABhJREFUeF4FwTEBAAAAgjD7FzESWfjYdgwEoAJ4lTsaxgAAAABJRU5ErkJggg==);background-image:url(border.png)!ie;background-position:bottom left;background-repeat:repeat-x}div#mw-head{position:absolute;top:0;right:0;width:100%}div#mw-head h5{margin:0;padding:0} div.emptyPortlet{display:none} #p-personal{position:absolute;top:0;padding-left:10em;right:0.75em}#p-personal h5{display:none}#p-personal ul{list-style:none;margin:0;padding:0} #p-personal li{line-height:1.125em;float:left} #p-personal li{margin-left:0.75em;margin-top:0.5em;font-size:0.75em;white-space:nowrap} #left-navigation{position:absolute;left:10em;top:2.5em}#right-navigation{float:right;margin-top:2.5em} div.vectorTabs h5,div.vectorMenu h5 span{display:none}  div.vectorTabs{float:left;height:2.5em}div.vectorTabs{background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAuCAIAAABmjeQ9AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAERJREFUeF5lTtEKgEAMMv//j/O0IxlH9CA6N2WURAA/OHl5GeWAwUUHBcKV795FtTePxpmV3t9uv8Z3/cmvM88vzbbrAV/dQdX+eas3AAAAAElFTkSuQmCC);background-image:url(tab-break.png)!ie;background-position:bottom left;background-repeat:no-repeat;padding-left:1px} div.vectorTabs ul{float:left}div.vectorTabs ul{height:100%;list-style:none;margin:0;padding:0} div.vectorTabs ul li{float:left} div.vectorTabs ul li{line-height:1.125em;display:inline-block;height:100%;margin:0;padding:0;background-color:#f3f3f3;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAABkCAIAAADITs03AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAADxJREFUeF7litsRACAMwrD77+Q0rtGoV98r+MEFchhgkr4NnZyb3bk/LM/yMCjiH4wots/++hYR3iXLJVWUBS1AtOi2fwAAAABJRU5ErkJggg==);background-image:url(tab-normal-fade.png)!ie;background-position:bottom left;background-repeat:repeat-x;white-space:nowrap} div.vectorTabs ul > li{display:block}div.vectorTabs li.selected{background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAABkAQAAAABvV2fNAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAABVJREFUeF7dwQEBAAAAQCDTTfdD4WOJ5TIB3ib9EgAAAABJRU5ErkJggg==);background-image:url(tab-current-fade.png)!ie} div.vectorTabs li a{display:inline-block;height:1.9em;padding-left:0.5em;padding-right:0.5em;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAuCAIAAABmjeQ9AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAERJREFUeF5lTtEKgEAMMv//j/O0IxlH9CA6N2WURAA/OHl5GeWAwUUHBcKV795FtTePxpmV3t9uv8Z3/cmvM88vzbbrAV/dQdX+eas3AAAAAElFTkSuQmCC);background-image:url(tab-break.png)!ie;background-position:bottom right;background-repeat:no-repeat;color:#0645ad;cursor:pointer;font-size:0.8em} div.vectorTabs li > a{display:block} div.vectorTabs span a{display:inline-block;padding-top:1.25em}  div.vectorTabs span > a{float:left;display:block}div.vectorTabs li.selected a,div.vectorTabs li.selected a:visited{color:#333333;text-decoration:none}div.vectorTabs li.new a,div.vectorTabs li.new a:visited{color:#a55858}  div.vectorMenu{direction:ltr;float:left;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABYAAAAQCAMAAAAlM38UAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAA9QTFRFsbGxmpqa3d3deXl58/n79CzHcQAAAAV0Uk5T/////wD7tg5TAAAAMklEQVR42mJgwQoYBkqYiZEZAhiZUFRDxWGicEPA4nBRhNlAcYQokpVMDEwD6kuAAAMAyGMFQVv5ldcAAAAASUVORK5CYII=);background-image:url(arrow-down-icon.png)!ie;background-position:100% 60%;background-repeat:no-repeat;cursor:pointer} body.rtl div.vectorMenu{direction:rtl}  div#mw-head div.vectorMenu h5{float:left;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAuCAIAAABmjeQ9AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAERJREFUeF5lTtEKgEAMMv//j/O0IxlH9CA6N2WURAA/OHl5GeWAwUUHBcKV795FtTePxpmV3t9uv8Z3/cmvM88vzbbrAV/dQdX+eas3AAAAAElFTkSuQmCC);background-image:url(tab-break.png)!ie;background-repeat:no-repeat} div#mw-head div.vectorMenu h5{background-position:bottom left;margin-left:-1px} div#mw-head div.vectorMenu > h5{background-image:none}div#mw-head div.vectorMenu h4{display:inline-block;float:left;font-size:0.8em;padding-left:0.5em;padding-top:1.375em;font-weight:normal;border:none}  div.vectorMenu h5 a{display:inline-block;width:24px;height:2.5em;text-decoration:none;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAuCAIAAABmjeQ9AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAERJREFUeF5lTtEKgEAMMv//j/O0IxlH9CA6N2WURAA/OHl5GeWAwUUHBcKV795FtTePxpmV3t9uv8Z3/cmvM88vzbbrAV/dQdX+eas3AAAAAElFTkSuQmCC);background-image:url(tab-break.png)!ie;background-repeat:no-repeat} div.vectorMenu h5 a{background-position:bottom right} div.vectorMenu h5 > a{display:block}div.vectorMenu div.menu{position:relative;display:none;clear:both;text-align:left}  body.rtl div.vectorMenu div.menu{margin-left:24px}  body.rtl div.vectorMenu > div.menu{margin-left:auto}   body.rtl div.vectorMenu > div.menu,x:-moz-any-link{margin-left:23px}div.vectorMenu:hover div.menu{display:block}div.vectorMenu ul{position:absolute;background-color:white;border:solid 1px silver;border-top-width:0;list-style:none;list-style-image:none;list-style-type:none;padding:0;margin:0;margin-left:-1px;text-align:left} div.vectorMenu ul,x:-moz-any-link{min-width:5em} div.vectorMenu ul,x:-moz-any-link,x:default{min-width:0}div.vectorMenu li{padding:0;margin:0;text-align:left;line-height:1em} div.vectorMenu li a{display:inline-block;padding:0.5em;white-space:nowrap;color:#0645ad;cursor:pointer;font-size:0.8em} div.vectorMenu li > a{display:block}div.vectorMenu li.selected a,div.vectorMenu li.selected a:visited{color:#333333;text-decoration:none} #p-search h5{display:none} #p-search{float:left}#p-search{margin-right:0.5em;margin-left:0.5em}#p-search form,#p-search input{margin:0;margin-top:0.4em}div#simpleSearch{display:block;width:14em;height:1.4em;margin-top:0.65em;position:relative;min-height:1px; border:solid 1px #AAAAAA;color:black;background-color:white;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAQCAIAAABY/YLgAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAACZJREFUeF5diqERACAQgID95/3s+cFg4CDQzASkXl4jidvrCPzfA7puAx52W1pnAAAAAElFTkSuQmCC);background-image:url(search-fade.png)!ie;background-position:top left;background-repeat:repeat-x}div#simpleSearch label{ font-size:13px;top:0.25em;direction:ltr}div#simpleSearch input{color:black;direction:ltr}div#simpleSearch input:focus{outline:none}div#simpleSearch input.placeholder{color:#999999}div#simpleSearch input::-webkit-input-placeholder{color:#999999}div#simpleSearch input#searchInput{position:absolute;top:0;left:0;width:90%;margin:0;padding:0;padding-left:0.2em;padding-top:0.2em;padding-bottom:0.2em;outline:none;border:none; font-size:13px;background-color:transparent;direction:ltr}div#simpleSearch button#searchButton{position:absolute;width:10%;right:0;top:0;padding:0;padding-top:0.2em;padding-bottom:0.2em;padding-right:0.4em;margin:0;border:none;cursor:pointer;background-color:transparent} div#simpleSearch button#searchButton img{border:none;margin:0;margin-top:-3px;padding:0} div#simpleSearch button#searchButton > img{margin:0} div#mw-panel{position:absolute;top:160px;padding-top:1em;width:10em;left:0}div#mw-panel div.portal{padding-bottom:1.5em;direction:ltr}div#mw-panel div.portal h5{font-weight:normal;color:#444444;padding:0.25em;padding-top:0;padding-left:1.75em;cursor:default;border:none;font-size:0.75em}div#mw-panel div.portal div.body{margin:0;padding-top:0.5em;margin-left:1.25em;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAIwAAAABCAAAAAAphRnkAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAClJREFUeF61yMEJACAQxMCN/Xfr/yIsaAfOJxC2UTPWS6f5gABhUTedBz7fGPSonIP/AAAAAElFTkSuQmCC);background-image:url(portal-break.png)!ie;background-repeat:no-repeat;background-position:top left}div#mw-panel div.portal div.body ul{list-style:none;list-style-image:none;list-style-type:none;padding:0;margin:0}div#mw-panel div.portal div.body ul li{line-height:1.125em;padding:0;padding-bottom:0.5em;margin:0;overflow:hidden;font-size:0.75em}div#mw-panel div.portal div.body ul li a{color:#0645ad}div#mw-panel div.portal div.body ul li a:visited{color:#0b0080} div#footer{margin-left:10em;margin-top:0;padding:0.75em;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAABhJREFUeF4FwTEBAAAAgjD7FzESWfjYdgwEoAJ4lTsaxgAAAABJRU5ErkJggg==);background-image:url(border.png)!ie;background-position:top left;background-repeat:repeat-x;direction:ltr}div#footer ul{list-style:none;list-style-image:none;list-style-type:none;margin:0;padding:0}div#footer ul li{margin:0;padding:0;padding-top:0.5em;padding-bottom:0.5em;color:#333333;font-size:0.7em}div#footer #footer-icons{float:right} body.ltr div#footer #footer-places{float:left}div#footer #footer-info li{line-height:1.4em}div#footer #footer-icons li{float:left;margin-left:0.5em;line-height:2em}div#footer #footer-places li{float:left;margin-right:1em;line-height:2em} #p-logo{position:absolute;top:-160px;left:0;width:10em;height:160px}#p-logo a{display:block;width:10em;height:160px;background-repeat:no-repeat;background-position:center center;text-decoration:none}  #preftoc{ width:100%;float:left;clear:both;margin:0 !important;padding:0 !important;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAhCAQAAACysAk0AAAACXBIWXMAAABIAAAASABGyWs+AAAACXZwQWcAAAABAAAAIQBSEXtPAAAAAmJLR0QA/vCI/CkAAAAmSURBVAjXY2BgYPj3n+k/AwL9g5Fwxl8GJgYGpr+ogmgITQuSgQA1QiAL/go8LAAAACV0RVh0Y3JlYXRlLWRhdGUAMjAwOS0wOC0wOVQxOTowNTo0MSswMDowMCYO2tEAAAAldEVYdG1vZGlmeS1kYXRlADIwMDktMDgtMDlUMTk6MDU6NDErMDA6MDB5v6zlAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAABJRU5ErkJggg==);background-image:url(preferences-break.png)!ie;background-position:bottom left;background-repeat:no-repeat}#preftoc li{ float:left;margin:0;padding:0;padding-right:1px;height:2.25em;white-space:nowrap;list-style-type:none;list-style-image:none;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAhCAQAAACysAk0AAAACXBIWXMAAABIAAAASABGyWs+AAAACXZwQWcAAAABAAAAIQBSEXtPAAAAAmJLR0QA/vCI/CkAAAAmSURBVAjXY2BgYPj3n+k/AwL9g5Fwxl8GJgYGpr+ogmgITQuSgQA1QiAL/go8LAAAACV0RVh0Y3JlYXRlLWRhdGUAMjAwOS0wOC0wOVQxOTowNTo0MSswMDowMCYO2tEAAAAldEVYdG1vZGlmeS1kYXRlADIwMDktMDgtMDlUMTk6MDU6NDErMDA6MDB5v6zlAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAABJRU5ErkJggg==);background-image:url(preferences-break.png)!ie;background-position:bottom right;background-repeat:no-repeat} #preftoc li:first-child{margin-left:1px}#preftoc a,#preftoc a:active{display:inline-block;position:relative;color:#0645ad;padding:0.5em;text-decoration:none;background-image:none;font-size:0.9em}#preftoc a:hover,#preftoc a:focus{text-decoration:underline}#preftoc li.selected a{background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAhCAQAAACysAk0AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAACBJREFUeF5twskJAAAMAjD3H7mXfYogCQiQeun68Z2WPk0SQHDa/pxXAAAAAElFTkSuQmCC);background-image:url(preferences-fade.png)!ie;background-position:bottom;background-repeat:repeat-x;color:#333333;text-decoration:none}#preferences{float:left;width:100%;margin:0;margin-top:-2px;clear:both;border:solid 1px #cccccc;background-color:#f9f9f9;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAABRJREFUeF4FwTEBAAAAwJD1j+waGQD8APvyfoZlAAAAAElFTkSuQmCC);background-image:url(preferences-base.png)!ie}#preferences fieldset.prefsection{border:none;padding:0;margin:1em}#preferences fieldset.prefsection fieldset{border:none;border-top:solid 1px #cccccc}#preferences legend{color:#666666}#preferences fieldset.prefsection legend.mainLegend{display:none}#preferences td{padding-left:0.5em;padding-right:0.5em}#preferences td.htmlform-tip{font-size:x-small;padding:.2em 2em;color:#666666}#preferences div.mw-prefs-buttons{padding:1em}#preferences div.mw-prefs-buttons input{margin-right:0.25em} #userlogin,#userloginForm{border:solid 1px #cccccc;padding:1.2em;margin:.5em;float:left}#userlogin{min-width:20em;max-width:90%;width:40em} div#content{line-height:1.5em}#bodyContent{font-size:0.8em} a{text-decoration:none;color:#0645ad;background:none}a:visited{color:#0b0080}a:active{color:#faa700}a:hover,a:focus{text-decoration:underline}a.stub{color:#772233}a.new,#p-personal a.new{color:#ba0000}a.new:visited,#p-personal a.new:visited{color:#a55858} img{border:none;vertical-align:middle}hr{height:1px;color:#aaa;background-color:#aaa;border:0;margin:.2em 0 .2em 0} h1,h2,h3,h4,h5,h6{color:black;background:none;font-weight:normal;margin:0;overflow:hidden;padding-top:.5em;padding-bottom:.17em;border-bottom:1px solid #aaa;width:auto}h1{font-size:188%}h1 .editsection{font-size:53%}h2{font-size:150%}h2 .editsection{font-size:67%}h3,h4,h5,h6{border-bottom:none;font-weight:bold}h3{font-size:132%}h3 .editsection{font-size:76%;font-weight:normal}h4{font-size:116%}h4 .editsection{font-size:86%;font-weight:normal}h5{font-size:100%}h5 .editsection{font-weight:normal}h6{font-size:80%}h6 .editsection{font-size:125%;font-weight:normal}.editsection{float:right}p{margin:.4em 0 .5em 0;line-height:1.5em}p img{margin:0}abbr,acronym,.explain{border-bottom:1px dotted black;color:black;background:none;cursor:help}q{font-family:Times,\"Times New Roman\",serif;font-style:italic} pre,code,tt,kbd,samp{ font-family:monospace,\"Courier New\"}code{background-color:#f9f9f9}pre{padding:1em;border:1px dashed #2f6fab;color:black;background-color:#f9f9f9;line-height:1.1em}ul{line-height:1.5em;list-style-type:square;margin:.3em 0 0 1.5em;padding:0;list-style-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUAAAANCAYAAABhPKSIAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAACtJREFUeF7NjbEJAAAIw7zRu/w5ouBUBEeHDM2QGiA8kObBULuFcJbSXN8T78SqnpKltAIAAAAASUVORK5CYII=);list-style-image:url(bullet-icon.png)!ie}ol{line-height:1.5em;margin:.3em 0 0 3.2em;padding:0;list-style-image:none}li{margin-bottom:.1em}dt{font-weight:bold;margin-bottom:.1em}dl{margin-top:.2em;margin-bottom:.5em}dd{line-height:1.5em;margin-left:2em;margin-bottom:.1em} table{font-size:100%;color:black} fieldset{border:1px solid #2f6fab;margin:1em 0 1em 0;padding:0 1em 1em;line-height:1.5em}fieldset.nested{margin:0 0 0.5em 0;padding:0 0.5em 0.5em}legend{padding:.5em;font-size:95%}form{border:none;margin:0}textarea{width:100%;padding:.1em}select{vertical-align:top} #toc,.toc,.mw-warning{border:1px solid #aaa;background-color:#f9f9f9;padding:5px;font-size:95%}#toc h2,.toc h2{display:inline;border:none;padding:0;font-size:100%;font-weight:bold}#toc #toctitle,.toc #toctitle,#toc .toctitle,.toc .toctitle{text-align:center}#toc ul,.toc ul{list-style-type:none;list-style-image:none;margin-left:0;padding-left:0;text-align:left}#toc ul ul,.toc ul ul{margin:0 0 0 2em}#toc .toctoggle,.toc .toctoggle{font-size:94%} div.floatright,table.floatright{clear:right;float:right;position:relative;margin:0 0 .5em .5em;border:0}div.floatright p{font-style:italic}div.floatleft,table.floatleft{float:left;clear:left;position:relative;margin:0 .5em .5em 0;border:0}div.floatleft p{font-style:italic} div.thumb{margin-bottom:.5em;width:auto;background-color:transparent}div.thumbinner{border:1px solid #ccc;padding:3px !important;background-color:#f9f9f9;font-size:94%;text-align:center;overflow:hidden}html .thumbimage{border:1px solid #ccc}html .thumbcaption{border:none;text-align:left;line-height:1.4em;padding:3px !important;font-size:94%}div.magnify{float:right;border:none !important;background:none !important}div.magnify a,div.magnify img{display:block;border:none !important;background:none !important} div.tright{clear:right;float:right;margin:.5em 0 1.3em 1.4em} div.tleft{float:left;clear:left;margin:.5em 1.4em 1.3em 0}img.thumbborder{border:1px solid #dddddd}.hiddenStructure{display:none} .mw-warning{margin-left:50px;margin-right:50px;text-align:center} .usermessage{background-color:#ffce7b;border:1px solid #ffa500;color:black;font-weight:bold;margin:2em 0 1em;padding:.5em 1em;vertical-align:middle} #siteNotice{position:relative;text-align:center;font-size:0.8em;margin:0}#localNotice{margin-bottom:0.9em} .catlinks{border:1px solid #aaa;background-color:#f9f9f9;padding:5px;margin-top:1em;clear:both} #siteSub{display:none}#jump-to-nav{display:none}#contentSub,#contentSub2{font-size:84%;line-height:1.2em;margin:0 0 1.4em 1em;color:#7d7d7d;width:auto}span.subpages{display:block} .center{width:100%;text-align:center}*.center *{margin-left:auto;margin-right:auto} .small,.small *{font-size:94%}table.small{font-size:100%} h1,h2{margin-bottom:.6em}h3,h4,h5{margin-bottom:.3em}#firstHeading{padding-top:0;margin-top:0;padding-top:0;margin-bottom:0.1em;line-height:1.2em;font-size:1.6em;padding-bottom:0}div#content a.external,div#content a[href ^=\"gopher://\"]{background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAFZJREFUeF59z4EJADEIQ1F36k7u5E7ZKXeUQPACJ3wK7UNokVxVk9kHnQH7bY9hbDyDhNXgjpRLqFlo4M2GgfyJHhjq8V4agfrgPQX3JtJQGbofmCHgA/nAKks+JAjFAAAAAElFTkSuQmCC) center right no-repeat;background:url(external-link-ltr-icon.png) center right no-repeat!ie;padding-right:13px}div#content a[href ^=\"https://\"],.link-https{background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAIVJREFUeF6tjzsKg0AQhi09mimsFJLCzpNYCGKbK3gAtfUIljaCoKCCZIs8MMV2v+yCg8siWlh8zOtjhjEAEFmeIopDQtTrTJNEZIxhWysiNfULJFJjDzGnba/aBt4+wAuBzD+tg6a8SVkXf4GET96xmDxNzP39IvE/PPDtXIyVpYinv14A5F0laJ8oYFgAAAAASUVORK5CYII=) center right no-repeat;background:url(lock-icon.png) center right no-repeat!ie;padding-right:13px}div#content a[href ^=\"mailto:\"],.link-mailto{background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKBAMAAAB/HNKOAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAADBQTFRF////////iIqF9vb26urpycfDvb275eXj2djV+/v4srKy6efio6GcqKejsa6q8fDtVM9qIQAAAAF0Uk5TAEDm2GYAAABOSURBVHheBcExDkAwGIDRL43NpJOt6a9hMdVilP8gklqsHMJmt4qeyeI03oNSNkCrAIU/7YTWbwp0zz4rTXZHxF/9YA15HTG4+4NFRNofUBMMOBBNZngAAAAASUVORK5CYII=) center right no-repeat;background:url(mail-icon.png) center right no-repeat!ie;padding-right:13px}div#content a[href ^=\"news://\"]{background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAHtJREFUeF6NkEEKgCAQRT2w1wiiUxgk0SKiTe6i9oKeQXDhKSZmYAJRKeHh4j//DIp+6OAPJH6cXJRSZqSUQClViBjUKER8zXAbUhev+6Q7hMA0G1msNtIo5zxhrX3xzlNG4ravYMwBMUZsKsBsXjQIABCTHlsfTXuj8wCN3T2QBjtcwQAAAABJRU5ErkJggg==) center right no-repeat;background:url(news-icon.png) center right no-repeat!ie;padding-right:13px}div#content a[href ^=\"ftp://\"],.link-ftp{background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAQAAAAnOwc2AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAExJREFUeF5VyEEKwCAMAMH83o/0LT6kFHqQYqkevG1jIITs3kaQgn+A7A29ujnw5NKrsaPCrTegBBrRMzYeXkbGzsdkZRwsPWMUmEd+CkSgVeVp2OkAAAAASUVORK5CYII=) center right no-repeat;background:url(file-icon.png) center right no-repeat!ie;padding-right:13px}div#content a[href ^=\"irc://\"],div#content a.extiw[href ^=\"irc://\"],.link-irc{background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAHRJREFUeF590E0KgCAQBWAvH0TXigI3ZccQ/8H91ExqKNrAW8j7kFG27SvMyzQM9s8whuBnENdQSllFKdWFWFC01pQQwhASMMaAtXYIMQScc/0dxSXyIaPq1ZzzF6JOsKBTHOC9hxgjoQLbf2tRgekWKka5AShBSepvauUSAAAAAElFTkSuQmCC) center right no-repeat;background:url(talk-icon.png) center right no-repeat!ie;padding-right:13px}div#content a.external[href $=\".ogg\"],div#content a.external[href $=\".OGG\"],div#content a.external[href $=\".mid\"],div#content a.external[href $=\".MID\"],div#content a.external[href $=\".midi\"],div#content a.external[href $=\".MIDI\"],div#content a.external[href $=\".mp3\"],div#content a.external[href $=\".MP3\"],div#content a.external[href $=\".wav\"],div#content a.external[href $=\".WAV\"],div#content a.external[href $=\".wma\"],div#content a.external[href $=\".WMA\"],.link-audio{background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKBAMAAAB/HNKOAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAADBQTFRF////dX8qyNF7eYMzwsxrsr9G8PHrm6Jrt7uakJVmn6OB1duat8NQi5YzhI4ykZR07gQraQAAAAF0Uk5TAEDm2GYAAABJSURBVHheNcSxDUBQFIbR727glxvKl3dHsIHCGESrNIIR7KE1hQ1MoDSCiMhJDixSDWVEhuZbei/sf/Jqbdn28+jxYe4u7CaND+p5C05J6bE1AAAAAElFTkSuQmCC) center right no-repeat;background:url(audio-icon.png) center right no-repeat!ie;padding-right:13px}div#content a.external[href $=\".ogm\"],div#content a.external[href $=\".OGM\"],div#content a.external[href $=\".avi\"],div#content a.external[href $=\".AVI\"],div#content a.external[href $=\".mpeg\"],div#content a.external[href $=\".MPEG\"],div#content a.external[href $=\".mpg\"],div#content a.external[href $=\".MPG\"],.link-video{background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAAAAACoWZBhAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAAJ0Uk5TAP9bkSK1AAAAXElEQVR4Xi2NMQoFMQgFvbpgHUj5LvF6K7sFQXKFsOew2G/xuylmGPn62Wb76U+ayHsTbDnrQMNrHdkZRChyi730KvK1QUWVD47gzoCOMBkXPSZrIuumseW/iKU/eKdG9xXBa10AAAAASUVORK5CYII=) center right no-repeat;background:url(video-icon.png) center right no-repeat!ie;padding-right:13px}div#content a.external[href $=\".pdf\"],div#content a.external[href $=\".PDF\"],div#content a.external[href *=\".pdf#\"],div#content a.external[href *=\".PDF#\"],div#content a.external[href *=\".pdf?\"],div#content a.external[href *=\".PDF?\"],.link-document{background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAQAAAAnOwc2AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAE5JREFUeF5lykEKgDAMBdF/+17Es/QkiosiCBURXIzJooZohmweX6gwmkCeI+Oqc2C1FnvnF2ejlQYU0tLK2NjY6f/l8V12Ti7uhFFgDj19b58EwXuqkAAAAABJRU5ErkJggg==) center right no-repeat;background:url(document-icon.png) center right no-repeat!ie;padding-right:13px} div#content a.extiw,div#content a.extiw:active{color:#36b;background:none;padding:0}div#content a.external{color:#36b}div#content .printfooter{display:none} #pt-userpage,#pt-anonuserpage,#pt-login{background:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAoAAAALCAMAAABxsOwqAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAGBQTFRF////R2uZMmGd5rp2boqvMFyS9unUTXWr5KQ/N2mn/Pr42Jg14Zstl6a80K93XXiVooVQ57VnpLnW4a1a36ZMr7Cwgpm5PXGw78mO+O/g1t7n5Orxk6zMvMne7/Dw6+zt5K6gdAAAAAF0Uk5TAEDm2GYAAABdSURBVHheHcpFAgMxEANBDZhhmQP//2Uc90V1EICrLuti0YqnyORrbLSrWPHJ/ukn8bkzbrnVD/MwpjQOb+B2hyvbQe7BTiG8ZmYqUOLglA0rCrcJxpiCz049/f4A9ZID96SyhDEAAAAASUVORK5CYII=) left top no-repeat;background:url(user-icon.png) left top no-repeat!ie;padding-left:15px !important;text-transform:none}.redirectText{font-size: 140%;}.redirectMsg img {vertical-align: text-bottom;}.toccolours{border:1px solid #aaa;background-color:#f9f9f9;padding:5px;font-size:95%}#bodyContent{position:relative;width:100%}#mw-js-message{font-size:0.8em}div#bodyContent{line-height:1.5em} #ca-unwatch.icon,#ca-watch.icon{margin-right:1px}#ca-unwatch.icon a,#ca-watch.icon a{margin:0;padding:0;outline:none;display:block;width:26px; padding-top:3.1em;margin-top:0; margin-top:-0.8em !ie;height:0;overflow:hidden;background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAFgAAAAQCAMAAAClQEgHAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAwBQTFRFoNb/+vr52tXLdcP/ltL/ysKt39rPrdz/xtDT8vLwwrJxodr/vqxjzdnr8v7+ntb/uuX/i87/ytTV9fb3zuz+8eOU+fr8zvH/wuX/ecT/hbrnj7XBltb/m9T/h8z/jtH+c8H/vq53lL/Ovq109vb1/v7rx8CuhcT0xLJlot3/2PL/kc//59N3s9//v7KGbL7/mNf//v395NSLmdr//Pz7ccT/wbOIZ7v/ybZk6OzzpNf/icPu0cm2g8n/p9n/9fTzva1ouuL/samQwu3/scfhfrbj8e/q4+bnyLJQ6u3tqtr/irbG7PH5fcz/0d7ww+r/zcuL6Obh9f7/hsv/s8+r+e2rw7J2rb3C+fj2icTy1O7/jrvO1s++vMyUz8zGocna6+rn8vDtlLK8aL3/d8X///vV7fz/vraklMr039nMtNjqp97/o9362ejN4vb/zcN7sN3/vfP/bsD/1dLNhK2+yLeIkrri28drz7tp5N7TiK26grXi3trTccP/vub/rd//+v//qM7fyeHMztmq5PT8u6t0/f3+/P39ksDwk8HQtMTH3fn/kdH/ltLpxb5o0dzsnND6ssXbzun5rdru2+Dh5+vsz9nadrrx1eLz+fLM7/z/w71z//zPgMz/8eWrwtKT9vn8jbPCf7vs1N3pua1terXo1O3tqtfWwOX51cV5dMH/vtmy28p8fLXR4efx9emq///8z9TbzrxowLOP5ea57///nq2xy7xo///5frrnwrSP9PX2+vv7ztzwvd3P2vH5r9z/8/X4nMrlsN//qLq9wa5zh7fikdf/tuL9zbpo3tnQ1u/kx8rL+/z8kq+6+vLGkqKq6f//oMrfxuf8xbVwqLvSh7vq8PP3ltD6d8P/v7Ngx7dqwbFt/PGyk8jv9vf3zLhofqy/wdPqyeTc0vH9//3kxun5i7O/x87X09mr1sRzmK3C3dnQz9XXmLvg///6uc7ozLpq7O7u//zc7evoyfT/+/z+mtf/9e25zcJt7ezowMXGu8nM+Pn5////8/n77InDmQAAAQB0Uk5T////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////AFP3ByUAAAJbSURBVHjaYvgPAb+3/ccK4tKxi7+4SkA9A5SWnuyJVSGXDidWcSnN6/jVQw0+s//fBWzqin9scMImvtF/62us6rfwOaEYXPLvXyM2pzn8+7dGBYt4dZ5WhjA29d3i+Sowg/d4TjuX9+9fQwnH1L8oqhIrN5//909VOdPNBEXcZ8Y7CQlHEam9Pt/Q1O/KrXNcdhSsnuFIZ3zWpLR/QMAnkhWvJA1TxVqx0mheKkj883qjVx9LYeIukRkT2P3rCtgPCGTfiLTuQKjPD3iZK1DAzv64OWD27VIG9+h/SOASB0xhwklk8XImmLilOp+IhK6XFQODrCyD+D1euPoHF50FDoPFZWQKfzIx/N/9PAiuP3oKwmOMYU9hwu8tAhHiYteidO34WbRtFZg1d65DVn+6HiJem3MrEBTGZ6taIPqDvN1RwkxxJkRcVeMLivixEwwsgpLmRfKulqjqm/jB4r08vyCp4tMhiAFPOFCj2L4cIh7KhCp+UJ1bjjlZ/6Y8L5r6PmOQuGkIEzS5vV0BMWBWOKrCGlGIeCorqvhieTlm5pRVkgYuaOpj5zLXmiqkLGeFGhwOTBRRl4EmJKEqVJsDdC3Q8B16qOITs4MNegS/B3OXoanf53s8JNbYN0cPanDSPy3vP0JVz/4tRFVo9u+uRcwbZdF/d1DFy8S5Fz3qr5ZxdkVT/3W1Rsyp1vmFS6AGP1TqAolzSK+9j6KQZ5MNiGK64sGIIr7U+gOI4pWaLoaqfjtEPRdIPdDgdiFY5hRCyaWGbDDz2CKQxdv8YOb5LcCtnuE/jQBAgAEAQlFsBT+lqfQAAAAASUVORK5CYII=);background-image:url(watch-icons.png)!ie}#ca-unwatch.icon a{background-position:-43px 60%}#ca-watch.icon a{background-position:5px 60%}#ca-unwatch.icon a:hover,#ca-unwatch.icon a:focus{background-position:-67px 60%}#ca-watch.icon a:hover,#ca-watch.icon a:focus{background-position:-19px 60%}#ca-unwatch.icon a.loading,#ca-watch.icon a.loading{background-image:url(data:image/gif;base64,R0lGODlhEAAQAMQfANra2uLi4vDw8PLy8ujo6Ozs7NbW1vj4+Pb29s7Oztzc3NTU1O7u7uDg4NHR0erq6v39/d7e3vz8/Pv7+/7+/tPT09jY2Pr6+tnZ2efn5/X19eXl5ebm5vT09P///////yH/C05FVFNDQVBFMi4wAwEAAAAh+QQJAwAfACwAAAAAEAAQAAAFa+Anjl9QkShacVqabp2XuKjjecHhStrjHDdIgtORiCyWSEZwud0mg0zEUhkYnNhsY/O5OCRZrEEwgvzCkgqZhGiEB1wUgRGeAFKApqcjcJ5QCx4aFQEECX1/JAlJJBsVFRMkEBkXLhyVNJkhACH5BAkDAB8ALAAAAAAQABAAAAV74CeO4hUQZEoGhqGqWzQtEnlYRCYMGSB5BkTKQCgUOBGPkjBIdQDKqBLhaJI4D6l0gylMRg6IVkmhNBIjxWBM8XAwHNFAIdYWDA0SRhNtKy0CJAUVEAcRAQJkFikZDg4EBB0RDR4dGCkIEhAjFBsBDwovKo0BoioFQiMhACH5BAkDAB8ALAAAAAAQABAAAAWB4CeO5HeU33OVl5IIpYEFh/QR1rYNZSMUAYVBwfBYbKRJwwPxFDxQjAbloECvHgMEBUBgPZTApjSxeL+eQGDUsQwkaGhBcUBYinGI5GBIEBwEGhxwVwwLFgoRHQwECgIADRFXBgUfEygfEBEDTmuYIxAJFAYwnyMFABVbpiMYGSghACH5BAkDAB8ALAAAAAAQABAAAAV+4CdKjWieKOJs6De1U5Zhg4YcmaG0kXcElQDtEWkZPMgMBGlofQDIqK9pmhAADClSEDBtAICJROvR7EQGx5LsgQAOogKm0LhQ2IDRQRJRFKIHAh4XAXknEw5REQsRBgAOEigRFBQEERofAgJiKBoZAgsXTicUDgYDoygNXU4hACH5BAUDAB8ALAAAAAAQABAAAAV54Cd+EFBNY6p+hgCssOERGwSP3eZBgUIEG0xhdGFpPMjChjNoRD6XIGBDQVo9FIcogZnsrlbLQNRQfMEewVN0ERAaaE9AoDoECGj76lBBTxQwDlYBEQweGwwqEDIHCwIbBgAAFioUBgUOdCIaBRwrBhUHNykQY6MfIQA7);background-image:url(watch-icon-loading.gif)!ie;background-position:5px 60%}#ca-unwatch.icon a span,#ca-watch.icon a span{display:none}div.vectorTabs ul{background-image:url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAuCAIAAABmjeQ9AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAERJREFUeF5lTtEKgEAMMv//j/O0IxlH9CA6N2WURAA/OHl5GeWAwUUHBcKV795FtTePxpmV3t9uv8Z3/cmvM88vzbbrAV/dQdX+eas3AAAAAElFTkSuQmCC);background-image:url(tab-break.png)!ie;background-position:right bottom;background-repeat:no-repeat} p.mw-ipb-conveniencelinks,p.mw-protect-editreasons,p.mw-filedelete-editreasons,p.mw-delete-editreasons{float:right} .tipsy{font-size:0.8em}}\n\n/* cache key: simplewiktionary:resourceloader:filter:minify-css:5:149524299ac89fe33bfc7202b008da3d */"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/ErlangConverter.java",
    "content": "package de.zib.scalaris.examples.wikipedia;\n\nimport de.zib.scalaris.ErlangValue;\n\n/**\n * Interface converting an {@link ErlangValue} to a custom type.\n * \n * @author Nico Kruber, kruber@zib.de\n *\n * @param <T> the type to convert to\n */\npublic interface ErlangConverter<T> {\n    /**\n     * Converts the given {@link ErlangValue} to the desired type\n     * \n     * @param v  the value to convert\n     * \n     * @return the converted value\n     * \n     * @throws ClassCastException if the value can not be converted\n     */\n    public T convert(ErlangValue v) throws ClassCastException;\n}"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/InvolvedKey.java",
    "content": "/**\n *  Copyright 2012-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia;\n\nimport java.util.Collection;\nimport java.util.List;\n\n\n\n/**\n * POD object for involved keys.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class InvolvedKey {\n    \n    /**\n     * Type of operation for the key.\n     */\n    public static enum OP {\n        /**\n         * The key was read.\n         */\n        NOOP(\"\"),\n        /**\n         * The key was read.\n         */\n        READ(\"r:\"),\n        /**\n         * The key was written.\n         */\n        WRITE(\"w:\");\n\n        private final String text;\n\n        OP(String text) {\n            this.text = text;\n        }\n\n        /**\n         * Converts the enum to text.\n         */\n        public String toString() {\n            return this.text;\n        }\n\n        /**\n         * Tries to convert a text to the according enum value.\n         * \n         * @param text the text to convert\n         * \n         * @return the according enum value\n         */\n        public static OP fromString(String text) {\n            if (text != null) {\n                for (OP b : OP.values()) {\n                    if (text.equalsIgnoreCase(b.text)) {\n                        return b;\n                    }\n                }\n            }\n            throw new IllegalArgumentException(\"No constant with text \" + text\n                    + \" found\");\n        }\n    }\n    \n    /**\n     * The operation performed at the key.\n     */\n    final public OP op;\n    /**\n     * The involved key.\n     */\n    final public String key;\n    \n    /**\n     * Constructor of a no-op key.\n     */\n    public InvolvedKey() {\n        this.op = OP.NOOP;\n        this.key = \"\";\n    }\n    \n    /**\n     * Constructor.\n     * \n     * @param op\n     *            the operation performed at the key\n     * @param key\n     *            the involved key\n     */\n    public InvolvedKey(final OP op, final String key) {\n        this.op = op;\n        this.key = key;\n    }\n    \n    @Override\n    public String toString() {\n        if (op == OP.NOOP && key.isEmpty()) {\n            return \"\";\n        }\n        return op.toString() + key.toString();\n    }\n    \n    /**\n     * Tries to parse an involved key from the given string.\n     * \n     * @param text\n     *            the string to parse from\n     * \n     * @return an {@link InvolvedKey} object\n     * \n     * @throws IllegalArgumentException\n     *             if the string is incorrect\n     */\n    public static InvolvedKey fromString(String text) throws IllegalArgumentException {\n        assert text != null;\n        if (text.isEmpty()) {\n            return new InvolvedKey();\n        } else if (text.length() >= 2) {\n            final OP op = OP.fromString(text.substring(0, 2));\n            final String key = text.substring(2);\n            return new InvolvedKey(op, key);\n        } else {\n            throw new IllegalArgumentException(\"No valid involvedKey: \" + text);\n        }\n    }\n    \n    /**\n     * Tries to parse multiple involved keys from the given collection and adds\n     * them the the list of involved keys.\n     * \n     * @param involvedKeys\n     *            list of involved keys\n     * @param keysStr\n     *            the collection to parse strings from\n     * @param includeNoops\n     *            whether to include noop keys\n     * \n     * @throws IllegalArgumentException\n     *             if the string is incorrect\n     */\n    public static void addInvolvedKeys(List<InvolvedKey> involvedKeys,\n            Collection<? extends String> keysStr, boolean includeNoops)\n            throws IllegalArgumentException {\n        assert involvedKeys != null;\n        assert keysStr != null;\n        \n        for (String text : keysStr) {\n            final InvolvedKey involvedKey = InvolvedKey.fromString(text);\n            if (includeNoops || !(involvedKey.op == OP.NOOP && involvedKey.key.isEmpty())) {\n                involvedKeys.add(involvedKey);\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/MyScalarisOpExecWrapper.java",
    "content": "package de.zib.scalaris.examples.wikipedia;\n\nimport java.util.ArrayList;\nimport java.util.Arrays;\nimport java.util.HashMap;\nimport java.util.List;\nimport java.util.Map.Entry;\nimport java.util.Random;\n\nimport de.zib.scalaris.examples.wikipedia.Options.APPEND_INCREMENT_BUCKETS;\nimport de.zib.scalaris.examples.wikipedia.Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE;\nimport de.zib.scalaris.examples.wikipedia.Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY;\nimport de.zib.scalaris.examples.wikipedia.Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM;\nimport de.zib.scalaris.examples.wikipedia.Options.IAppendIncrement;\nimport de.zib.scalaris.examples.wikipedia.Options.IBuckets;\nimport de.zib.scalaris.examples.wikipedia.Options.IReadBuckets;\nimport de.zib.scalaris.examples.wikipedia.Options.Optimisation;\nimport de.zib.scalaris.executor.ScalarisIncrementOp1;\nimport de.zib.scalaris.executor.ScalarisIncrementOp2;\nimport de.zib.scalaris.executor.ScalarisListAppendRemoveOp1;\nimport de.zib.scalaris.executor.ScalarisListAppendRemoveOp2;\nimport de.zib.scalaris.executor.ScalarisOpExecutor;\nimport de.zib.scalaris.executor.ScalarisWriteOp;\nimport de.zib.tools.LinkedMultiHashMap;\n\n/**\n * Wraps {@link ScalarisOpExecutor} and adds the different operations\n * based on the current configuration.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MyScalarisOpExecWrapper {\n    protected final ScalarisOpExecutor executor;\n    /**\n     * Creates a new wrapper with the given executor.\n     * \n     * @param executor the executor to use.\n     */\n    public MyScalarisOpExecWrapper(MyScalarisTxOpExecutor executor) {\n        this.executor = executor;\n    }\n\n    /**\n     * Creates a new write operation.\n     * \n     * @param opType    the type of the operation\n     * @param key       the key to write the value to\n     * @param value     the value to write\n     * \n     * @param <T>       type of the value\n     */\n    public <T> void addWrite(ScalarisOpType opType, String key, T value) {\n        switch (opType) {\n        default:\n            executor.addOp(new ScalarisWriteOp<T>(key, value));\n        }\n    }\n\n    /**\n     * Creates a new list append operation.\n     * \n     * @param opType      the type of the operation\n     * @param key         the key to append the value to\n     * @param toAdd       the value to add\n     * @param countOpType the type of the countKey operation (may be the same\n     *                    as <tt>opType</tt> or <tt>null</tt> if\n     *                    <tt>countKey</tt> is)\n     * @param countKey    the key for the counter of the entries in the list\n     *                    (may be <tt>null</tt>)\n     * \n     * @param <T>         type of the value to add\n     */\n    @SuppressWarnings(\"unchecked\")\n    public <T> void addAppend(ScalarisOpType opType, String key, T toAdd, ScalarisOpType countOpType, String countKey) {\n        addAppendRemove(opType, key, Arrays.asList(toAdd), new ArrayList<T>(0), countOpType, countKey);\n    }\n\n    /**\n     * Creates a new list remove operation.\n     * \n     * @param opType      the type of the operation\n     * @param key         the key to remove the value from\n     * @param toRemove    the value to remove\n     * @param countOpType the type of the countKey operation (may be the same\n     *                    as <tt>opType</tt> or <tt>null</tt> if\n     *                    <tt>countKey</tt> is)\n     * @param countKey    the key for the counter of the entries in the list\n     *                    (may be <tt>null</tt>)\n     * \n     * @param <T>         type of the value to remove\n     */\n    @SuppressWarnings(\"unchecked\")\n    public <T> void addRemove(ScalarisOpType opType, String key, T toRemove, ScalarisOpType countOpType, String countKey) {\n        addAppendRemove(opType, key, new ArrayList<T>(0), Arrays.asList(toRemove), countOpType, countKey);\n    }\n\n    /**\n     * Creates a new number increment operation.\n     * \n     * @param opType    the type of the operation\n     * @param key       the key of the value to increment\n     * @param toAdd     the value to increment by\n     * @param belongsTo the object this number belongs to, e.g. a page title to\n     *                  determine the partition the value gets added to or\n     *                  removed from\n     * \n     * @param <T>       type of the value to add\n     */\n    public <T extends Number, U> void addIncrement(final ScalarisOpType opType,\n            final String key, final T toAdd, final U belongsTo) {\n        final Optimisation optimisation = Options.getInstance().OPTIMISATIONS.get(opType);\n        if (optimisation instanceof APPEND_INCREMENT_BUCKETS) {\n            final APPEND_INCREMENT_BUCKETS optimisation2 = (APPEND_INCREMENT_BUCKETS) optimisation;\n            final String key2 = key + optimisation2.getBucketString(belongsTo);\n            executor.addOp(new ScalarisIncrementOp2<T>(key2, toAdd));\n        } else if (optimisation instanceof APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY) {\n            final APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY optimisation2 = (APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY) optimisation;\n            final String key2 = key + optimisation2.getWriteBucketString();\n            executor.addOp(new ScalarisIncrementOp2<T>(key2, toAdd));\n        } else if (optimisation instanceof APPEND_INCREMENT_BUCKETS_WITH_WCACHE) {\n            final APPEND_INCREMENT_BUCKETS_WITH_WCACHE optimisation2 = (APPEND_INCREMENT_BUCKETS_WITH_WCACHE) optimisation;\n            final String key2 = key + optimisation2.getWriteBucketString(belongsTo);\n            executor.addOp(new ScalarisIncrementOp2<T>(key2, toAdd));\n        } else if (optimisation instanceof IAppendIncrement) {\n            executor.addOp(new ScalarisIncrementOp2<T>(key, toAdd));\n        } else {\n            executor.addOp(new ScalarisIncrementOp1<T>(key, toAdd));\n        }\n    }\n\n    /**\n     * @return the executor\n     */\n    public ScalarisOpExecutor getExecutor() {\n        return executor;\n    }\n\n    /**\n     * Creates a new append+remove operation.\n     * \n     * @param opType      the type of the operation\n     * @param key         the key to append/remove the values to/from\n     * @param toAdd       the values to add\n     * @param toRemove    the values to remove\n     * @param countOpType the type of the countKey operation (may be the same\n     *                    as <tt>opType</tt> or <tt>null</tt> if\n     *                    <tt>countKey</tt> is)\n     * @param countKey    the key for the counter of the entries in the list\n     *                    (may be <tt>null</tt>)\n     * \n     * @param <T>         type of the value to remove\n     */\n    public <T> void addAppendRemove(final ScalarisOpType opType,\n            final String key, final List<T> toAdd, final List<T> toRemove,\n            final ScalarisOpType countOpType, String countKey) {\n        final Optimisation countOptimisation = Options.getInstance().OPTIMISATIONS.get(countOpType);\n\n        if (countKey != null && countOptimisation != null) {\n            // separate optimisation for the count key\n            int countInc = toAdd.size() - toRemove.size();\n            if (countOptimisation instanceof IBuckets) {\n                Random rand = new Random();\n                String bucketStr;\n                if (countOptimisation instanceof IReadBuckets) {\n                    final IReadBuckets optimisation2 = (IReadBuckets) countOptimisation;\n                    final int writeBuckets = optimisation2.getBuckets() - optimisation2.getReadBuckets();\n                    final int bucket = optimisation2.getReadBuckets() + rand.nextInt(writeBuckets);\n                    bucketStr = \":\" + bucket;\n                } else if (countOptimisation instanceof APPEND_INCREMENT_BUCKETS) {\n                    final APPEND_INCREMENT_BUCKETS optimisation2 = (APPEND_INCREMENT_BUCKETS) countOptimisation;\n                    bucketStr = \":\" + rand.nextInt(optimisation2.getBuckets());\n                } else {\n                    throw new RuntimeException(\"unsupported optimisation: \" + countOptimisation);\n                }\n                if (countOptimisation instanceof IAppendIncrement) {\n                    executor.addOp(new ScalarisIncrementOp2<Integer>(countKey + bucketStr, countInc));\n                } else {\n                    executor.addOp(new ScalarisIncrementOp1<Integer>(countKey + bucketStr, countInc));\n                }\n            } else if (countOptimisation instanceof IAppendIncrement) {\n                executor.addOp(new ScalarisIncrementOp2<Integer>(countKey, countInc));\n            } else {\n                executor.addOp(new ScalarisIncrementOp1<Integer>(countKey, countInc));\n            }\n            // prevent the code below to write to a counter:\n            countKey = null;\n        }\n        \n        final Optimisation optimisation = Options.getInstance().OPTIMISATIONS.get(opType);\n        if (optimisation instanceof APPEND_INCREMENT_BUCKETS) {\n            final APPEND_INCREMENT_BUCKETS optimisation2 = (APPEND_INCREMENT_BUCKETS) optimisation;\n            final HashMap<String, String> countKeys = new HashMap<String, String>(toAdd.size() + toRemove.size());\n            final LinkedMultiHashMap<String, T> kvAdd = new LinkedMultiHashMap<String, T>();\n            final LinkedMultiHashMap<String, T> kvRemove = new LinkedMultiHashMap<String, T>();\n            for (T t : toAdd) {\n                final String bucketStr = optimisation2.getBucketString(t);\n                final String key2 = key + bucketStr;\n                if (countKey != null) {\n                    countKeys.put(key2, countKey + bucketStr);\n                } else {\n                    countKeys.put(key2, null);\n                }\n                kvAdd.put1(key2, t);\n            }\n            for (T t : toRemove) {\n                final String bucketStr = optimisation2.getBucketString(t);\n                final String key2 = key + bucketStr;\n                if (countKey != null) {\n                    countKeys.put(key2, countKey + bucketStr);\n                } else {\n                    countKeys.put(key2, null);\n                }\n                kvRemove.put1(key2, t);\n            }\n            for (Entry<String, String> entry : countKeys.entrySet()) {\n                final String key2 = entry.getKey();\n                List<T> toAdd2 = kvAdd.get(key2);\n                if (toAdd2 == null) {\n                    toAdd2 = new ArrayList<T>(0);\n                }\n                List<T> toRemove2 = kvRemove.get(key2);\n                if (toRemove2 == null) {\n                    toRemove2 = new ArrayList<T>(0);\n                }\n                executor.addOp(new ScalarisListAppendRemoveOp2<T>(key2, toAdd2, toRemove2, entry.getValue()));\n            }\n        } else if (optimisation instanceof APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY) {\n            final APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY optimisation2 = (APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY) optimisation;\n            final String bucketStr = optimisation2.getWriteBucketString();\n            final String key2 = key + bucketStr;\n            String countKey2 = null;\n            if (countKey != null) {\n                countKey2 = countKey + bucketStr;\n            }\n            if (optimisation instanceof APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM\n                    && !toRemove.isEmpty()) {\n                System.err.println(\"deleting entries is not supported with APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM\");\n            }\n            executor.addOp(new ScalarisListAppendRemoveOp2<T>(key2, toAdd, toRemove, countKey2));\n        } else if (optimisation instanceof APPEND_INCREMENT_BUCKETS_WITH_WCACHE) {\n            final APPEND_INCREMENT_BUCKETS_WITH_WCACHE optimisation2 = (APPEND_INCREMENT_BUCKETS_WITH_WCACHE) optimisation;\n            final HashMap<String, String> countKeys = new HashMap<String, String>(2 * (toAdd.size() + toRemove.size()));\n            final LinkedMultiHashMap<String, Object> kvAdd = new LinkedMultiHashMap<String, Object>();\n            final LinkedMultiHashMap<String, Object> kvRemove = new LinkedMultiHashMap<String, Object>();\n            for (T t : toAdd) {\n                // add to add write-bucket\n                final String bucketStr = optimisation2.getWriteBucketString(t);\n                final String key2 = key + bucketStr;\n                if (countKey != null) {\n                    countKeys.put(key2, countKey + bucketStr);\n                } else {\n                    countKeys.put(key2, null);\n                }\n                kvAdd.put1(key2, optimisation2.makeAdd(t));\n                // also need to remove any previous delete op from the write-bucket (if present!)\n                kvRemove.put1(key2, optimisation2.makeDelete(t));\n            }\n            for (T t : toRemove) {\n                // add to delete write-bucket\n                final String bucketStr = optimisation2.getWriteBucketString(t);\n                final String key2 = key + bucketStr;\n                if (countKey != null) {\n                    countKeys.put(key2, countKey + bucketStr);\n                } else {\n                    countKeys.put(key2, null);\n                }\n                kvAdd.put1(key2, optimisation2.makeDelete(t));\n                // also need to remove any previous add op from the write-bucket (if present!)\n                kvRemove.put1(key2, optimisation2.makeAdd(t));\n            }\n            for (Entry<String, String> entry : countKeys.entrySet()) {\n                final String key2 = entry.getKey();\n                List<Object> toAdd2 = kvAdd.get(key2);\n                if (toAdd2 == null) {\n                    toAdd2 = new ArrayList<Object>(0);\n                }\n                List<Object> toRemove2 = kvRemove.get(key2);\n                if (toRemove2 == null) {\n                    toRemove2 = new ArrayList<Object>(0);\n                }\n                executor.addOp(new ScalarisListAppendRemoveOp2<Object>(key2, toAdd2, toRemove2, entry.getValue()));\n            }\n        } else if (optimisation instanceof IAppendIncrement) {\n            executor.addOp(new ScalarisListAppendRemoveOp2<T>(key, toAdd, toRemove, countKey));\n        } else {\n            executor.addOp(new ScalarisListAppendRemoveOp1<T>(key, toAdd, toRemove, countKey));\n        }\n    }\n}"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/MyScalarisSingleOpExecutor.java",
    "content": "/**\n *  Copyright 2012-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia;\n\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.TransactionSingleOp;\nimport de.zib.scalaris.TransactionSingleOp.ResultList;\nimport de.zib.scalaris.UnknownException;\nimport de.zib.scalaris.executor.ScalarisOp;\nimport de.zib.scalaris.executor.ScalarisSingleOpExecutor;\n\n/**\n * Executes multiple {@link ScalarisOp} operations in multiple phases only\n * sending requests to Scalaris once per work phase.\n * \n * In addition to {@link ScalarisSingleOpExecutor}, also collects info about all\n * involved keys.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MyScalarisSingleOpExecutor extends ScalarisSingleOpExecutor {\n    protected final List<InvolvedKey> involvedKeys;\n\n    /**\n     * Creates a new executor.\n     * \n     * @param scalaris_single\n     *            the Scalaris connection to use\n     * @param involvedKeys\n     *            list of all involved keys\n     */\n    public MyScalarisSingleOpExecutor(TransactionSingleOp scalaris_single,\n            List<InvolvedKey> involvedKeys) {\n        super(scalaris_single);\n        this.involvedKeys = involvedKeys;\n    }\n\n    /**\n     * Executes the given requests and records all involved keys.\n     * \n     * @param requests\n     *            a request list to execute\n     * \n     * @return the results from executing the requests\n     * \n     * @throws OtpErlangException\n     *             if an error occurred verifying a result from previous\n     *             operations\n     * @throws UnknownException\n     *             if an error occurred verifying a result from previous\n     *             operations\n     */\n    @Override\n    protected ResultList executeRequests(RequestList requests)\n            throws ConnectionException, AbortException,\n            UnknownException {\n        ScalarisDataHandler.addInvolvedKeys(involvedKeys, requests.getRequests());\n        return super.executeRequests(requests);\n    }\n\n    /**\n     * @return the involvedKeys\n     */\n    public List<InvolvedKey> getInvolvedKeys() {\n        return involvedKeys;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/MyScalarisTxOpExecutor.java",
    "content": "/**\n *  Copyright 2012-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia;\n\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.Transaction;\nimport de.zib.scalaris.Transaction.ResultList;\nimport de.zib.scalaris.UnknownException;\nimport de.zib.scalaris.executor.ScalarisOp;\nimport de.zib.scalaris.executor.ScalarisTxOpExecutor;\n\n/**\n * Executes multiple {@link ScalarisOp} operations in multiple phases only\n * sending requests to Scalaris once per work phase.\n * \n * In addition to {@link ScalarisTxOpExecutor}, also collects info about all\n * involved keys.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MyScalarisTxOpExecutor extends ScalarisTxOpExecutor {\n    protected final List<InvolvedKey> involvedKeys;\n\n    /**\n     * Creates a new executor.\n     * \n     * @param scalaris_tx\n     *            the Scalaris connection to use\n     * @param involvedKeys\n     *            list of all involved keys\n     */\n    public MyScalarisTxOpExecutor(Transaction scalaris_tx,\n            List<InvolvedKey> involvedKeys) {\n        super(scalaris_tx);\n        this.involvedKeys = involvedKeys;\n    }\n\n    /**\n     * Executes the given requests and records all involved keys.\n     * \n     * @param requests\n     *            a request list to execute\n     * \n     * @return the results from executing the requests\n     * \n     * @throws OtpErlangException\n     *             if an error occurred verifying a result from previous\n     *             operations\n     * @throws UnknownException\n     *             if an error occurred verifying a result from previous\n     *             operations\n     */\n    @Override\n    protected ResultList executeRequests(RequestList requests)\n            throws ConnectionException, AbortException,\n            UnknownException {\n        ScalarisDataHandler.addInvolvedKeys(involvedKeys, requests.getRequests());\n        return super.executeRequests(requests);\n    }\n\n    /**\n     * @return the involvedKeys\n     */\n    public List<InvolvedKey> getInvolvedKeys() {\n        return involvedKeys;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/NamespaceUtils.java",
    "content": "package de.zib.scalaris.examples.wikipedia;\n\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\n\n/**\n * Interface for namespace implementations using a {@link SiteInfo} backend.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic interface NamespaceUtils {\n\n    /**\n     * Gets the siteinfo object used for this namespace.\n     * \n     * @return the siteinfo\n     */\n    public abstract SiteInfo getSiteinfo();\n\n    /**\n     * Gets the talk page's name corresponding to the given page name.\n     * \n     * @param pageName\n     *            a page name, i.e. a title\n     * \n     * @return the name of the talk page\n     */\n    public abstract String getTalkPageFromPageName(String pageName);\n\n    /**\n     * Gets the content page's name corresponding to the given talk page name.\n     * \n     * @param talkPageName\n     *            a talk page name, i.e. a title\n     * \n     * @return the name of the content page\n     */\n    public abstract String getPageNameFromTalkPage(String talkPageName);\n\n    /**\n     * Checks whether the page's name is in a talk space.\n     * \n     * @param pageName\n     *            a page name, i.e. a title\n     * \n     * @return <tt>true</tt> if the page is a talk page, <tt>false</tt>\n     *         otherwise\n     */\n    public abstract boolean isTalkPage(String pageName);\n\n}"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/Options.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia;\n\nimport java.io.BufferedReader;\nimport java.io.FileInputStream;\nimport java.io.InputStreamReader;\nimport java.util.ArrayList;\nimport java.util.Arrays;\nimport java.util.EnumMap;\nimport java.util.HashMap;\nimport java.util.HashSet;\nimport java.util.List;\nimport java.util.Map;\nimport java.util.Random;\nimport java.util.Set;\nimport java.util.regex.Matcher;\nimport java.util.regex.Pattern;\n\nimport org.xml.sax.Attributes;\nimport org.xml.sax.InputSource;\nimport org.xml.sax.SAXException;\nimport org.xml.sax.XMLReader;\nimport org.xml.sax.helpers.DefaultHandler;\nimport org.xml.sax.helpers.XMLReaderFactory;\n\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.examples.wikipedia.bliki.ExistingPagesCache;\nimport de.zib.scalaris.examples.wikipedia.bliki.ExistingPagesCache.ExistingPagesCacheBloom;\nimport de.zib.scalaris.examples.wikipedia.bliki.ExistingPagesCache.ExistingPagesCacheFull;\n\n\n/**\n * @author Nico Kruber, kruber@zib.de\n *\n */\npublic class Options {\n    \n    private final static Options instance = new Options();\n    protected static final Pattern CONFIG_SINGLE_OPTIMISATION = Pattern.compile(\"([a-zA-Z_0-9]*):([a-zA-Z_0-9]*)(?:\\\\(([a-zA-Z_0-9,]*)\\\\))?\");\n\n    /**\n     * The name of the server (part of the URL), e.g. <tt>en.wikipedia.org</tt>.\n     */\n    public String SERVERNAME = \"localhost:8080\";\n    /**\n     * The path on the server (part of the URL), e.g. <tt>/wiki</tt>.\n     */\n    public String SERVERPATH = \"/scalaris-wiki/wiki\";\n    \n    /**\n     * Whether to support back-links (\"what links here?\") or not.\n     */\n    public boolean WIKI_USE_BACKLINKS = true;\n    \n    /**\n     * How often to re-try a \"sage page\" operation in case of failures, e.g.\n     * concurrent edits.\n     * \n     * @see #WIKI_SAVEPAGE_RETRY_DELAY\n     */\n    public int WIKI_SAVEPAGE_RETRIES = 0;\n    \n    /**\n     * How long to wait after a failed \"sage page\" operation before trying\n     * again (in milliseconds).\n     * \n     * @see #WIKI_SAVEPAGE_RETRIES\n     */\n    public int WIKI_SAVEPAGE_RETRY_DELAY = 10;\n    \n    /**\n     * Which implementation to use for the pages cache.\n     * \n     * @see #WIKI_REBUILD_PAGES_CACHE\n     */\n    public Class<? extends ExistingPagesCache> WIKI_PAGES_CACHE_IMPL = ExistingPagesCacheBloom.class;\n    \n    /**\n     * How often to re-create the pages cache with the existing pages (in\n     * seconds). The pages cache will be disabled if a value less than or equal\n     * to 0 is provided.\n     * \n     * @see #WIKI_PAGES_CACHE_IMPL\n     */\n    public int WIKI_REBUILD_PAGES_CACHE = 10 * 60;\n    \n    /**\n     * Whether and how to store user contributions in the DB.\n     */\n    public STORE_CONTRIB_TYPE WIKI_STORE_CONTRIBUTIONS = STORE_CONTRIB_TYPE.OUTSIDE_TX;\n    \n    /**\n     * Optimisations to use for the different Scalaris operations.\n     */\n    final public EnumMap<ScalarisOpType, Optimisation> OPTIMISATIONS = new EnumMap<ScalarisOpType, Options.Optimisation>(\n            ScalarisOpType.class);\n    \n    /**\n     * Store user requests in a log for the last x minutes before the last\n     * request.\n     */\n    public int LOG_USER_REQS = 0;\n    \n    /**\n     * Time (in seconds) between executions of the node discovery daemon of\n     * {@link de.zib.scalaris.NodeDiscovery} to look for new Scalaris nodes (\n     * <tt>0</tt> to disable).\n     */\n    public int SCALARIS_NODE_DISCOVERY = 60;\n    \n    /**\n     * Creates a new default option object.\n     */\n    public Options() {\n        for (ScalarisOpType op : ScalarisOpType.values()) {\n            OPTIMISATIONS.put(op, new APPEND_INCREMENT());\n        }\n        OPTIMISATIONS.put(ScalarisOpType.PAGE_COUNT, null);\n        OPTIMISATIONS.put(ScalarisOpType.CATEGORY_PAGE_COUNT, null);\n    }\n\n    /**\n     * Gets the static instance used throughout the wiki implementation.\n     * \n     * @return the instance\n     */\n    public static Options getInstance() {\n        return instance;\n    }\n    \n    /**\n     * Type of storing user contributions in the DB.\n     */\n    public static enum STORE_CONTRIB_TYPE {\n        /**\n         * Do not store user contributions.\n         */\n        NONE(\"NONE\"),\n        /**\n         * Store user contributions outside the main transaction used during\n         * save.\n         */\n        OUTSIDE_TX(\"OUTSIDE_TX\");\n\n        private final String text;\n\n        STORE_CONTRIB_TYPE(String text) {\n            this.text = text;\n        }\n\n        /**\n         * Converts the enum to text.\n         */\n        public String toString() {\n            return this.text;\n        }\n\n        /**\n         * Tries to convert a text to the according enum value.\n         * \n         * @param text the text to convert\n         * \n         * @return the according enum value\n         */\n        public static STORE_CONTRIB_TYPE fromString(String text) {\n            if (text != null) {\n                for (STORE_CONTRIB_TYPE b : STORE_CONTRIB_TYPE.values()) {\n                    if (text.equalsIgnoreCase(b.text)) {\n                        return b;\n                    }\n                }\n            }\n            throw new IllegalArgumentException(\"No constant with text \" + text\n                    + \" found\");\n        }\n    }\n    \n    /**\n     * Indicates a generic optimisation implementation.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static interface Optimisation {\n    }\n    \n    /**\n     * Indicates that the traditional read/write operations of Scalaris should\n     * be used, i.e. no append/increment.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class TRADITIONAL implements Optimisation {\n        @Override\n        public String toString() {\n            return \"TRADITIONAL\";\n        }\n    }\n\n    /**\n     * Indicates that the new append and increment operations of Scalaris should\n     * be used, i.e.\n     * {@link de.zib.scalaris.Transaction#addDelOnList(String, java.util.List, java.util.List)}\n     * and {@link de.zib.scalaris.Transaction#addOnNr(String, Object)}.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static interface IAppendIncrement {\n    }\n\n    /**\n     * Indicates that the new append and increment operations of Scalaris should\n     * be used, i.e.\n     * {@link de.zib.scalaris.Transaction#addDelOnList(String, java.util.List, java.util.List)}\n     * and {@link de.zib.scalaris.Transaction#addOnNr(String, Object)}.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class APPEND_INCREMENT implements Optimisation,\n            IAppendIncrement {\n        @Override\n        public String toString() {\n            return \"APPEND_INCREMENT\";\n        }\n    }\n\n    /**\n     * Indicates that the new partial reads for random elements of a list\n     * {@link de.zib.scalaris.operations.ReadRandomFromListOp} and sublists\n     * {@link de.zib.scalaris.operations.ReadSublistOp} should be used.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static interface IPartialRead {\n    }\n\n    /**\n     * Indicates that the new append and increment operations of Scalaris should\n     * be used, i.e.\n     * {@link de.zib.scalaris.Transaction#addDelOnList(String, java.util.List, java.util.List)}\n     * and {@link de.zib.scalaris.Transaction#addOnNr(String, Object)} as well\n     * as partial reads for random elements of a list\n     * {@link de.zib.scalaris.operations.ReadRandomFromListOp} and sublists\n     * {@link de.zib.scalaris.operations.ReadSublistOp}.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class APPEND_INCREMENT_PARTIALREAD extends APPEND_INCREMENT\n            implements IPartialRead {\n        @Override\n        public String toString() {\n            return \"APPEND_INCREMENT_PARTIALREAD\";\n        }\n    }\n\n    /**\n     * Indicates that the optimisation partitions data into buckets.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static interface IBuckets {\n        /**\n         * Gets the number of available buckets\n         * \n         * @return number of buckets\n         */\n        public abstract int getBuckets();\n    }\n\n    /**\n     * Indicates that the optimisation partitions data into buckets split into\n     * read-buckets and other buckets.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static interface IReadBuckets extends IBuckets {\n        /**\n         * Gets the number of available read-buckets.\n         * \n         * @return number of buckets not used for the write caches etc.\n         */\n        public abstract int getReadBuckets();\n    }\n\n    /**\n     * Indicates that the new append and increment operations of Scalaris should\n     * be used and list values should be distributed among several partitions,\n     * i.e. buckets.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static abstract class APPEND_INCREMENT_BUCKETS implements\n            Optimisation, IAppendIncrement, IBuckets {\n        final protected int buckets;\n        \n        /**\n         * Constructor.\n         * \n         * @param buckets\n         *            number of available buckets\n         */\n        public APPEND_INCREMENT_BUCKETS(int buckets) {\n            this.buckets = buckets;\n        }\n\n        public int getBuckets() {\n            return buckets;\n        }\n        \n        /**\n         * Gets the string to append to the key in order to point to the bucket\n         * for the given value.\n         * \n         * @param value\n         *            the value to check the bucket for\n         * \n         * @return the bucket string, e.g. \":0\"\n         */\n        abstract public <T> String getBucketString(final T value);\n    }\n\n    /**\n     * Indicates that the new append and increment operations of Scalaris should\n     * be used and list values should be randomly distributed among several\n     * partitions, i.e. buckets.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class APPEND_INCREMENT_BUCKETS_RANDOM extends\n            APPEND_INCREMENT_BUCKETS {\n        final static protected Random rand = new Random();\n        /**\n         * Constructor.\n         * \n         * @param buckets\n         *            number of available buckets\n         */\n        public APPEND_INCREMENT_BUCKETS_RANDOM(int buckets) {\n            super(buckets);\n        }\n\n        @Override\n        public <T> String getBucketString(final T value) {\n            return \":\" + rand.nextInt(buckets);\n        }\n        \n        @Override\n        public String toString() {\n            return \"APPEND_INCREMENT_BUCKETS_RANDOM(\" + buckets + \")\";\n        }\n    }\n\n    /**\n     * Indicates that the new append, increment and partial read operations of\n     * Scalaris should be used and list values should be randomly distributed\n     * among several partitions, i.e. buckets.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class APPEND_INCREMENT_PARTIALREAD_BUCKETS_RANDOM extends\n            APPEND_INCREMENT_BUCKETS_RANDOM implements IPartialRead {\n        /**\n         * Constructor.\n         * \n         * @param buckets\n         *            number of available buckets\n         */\n        public APPEND_INCREMENT_PARTIALREAD_BUCKETS_RANDOM(int buckets) {\n            super(buckets);\n        }\n        \n        @Override\n        public String toString() {\n            return \"APPEND_INCREMENT_PARTIALREAD_BUCKETS_RANDOM(\" + buckets + \")\";\n        }\n    }\n    \n    /**\n     * Indicates that the new append and increment operations of Scalaris should\n     * be used and list values should be distributed among several partitions,\n     * i.e. buckets, depending on the value's hash.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class APPEND_INCREMENT_BUCKETS_WITH_HASH extends\n            APPEND_INCREMENT_BUCKETS {\n        /**\n         * Constructor.\n         * \n         * @param buckets\n         *            number of available buckets\n         */\n        public APPEND_INCREMENT_BUCKETS_WITH_HASH(int buckets) {\n            super(buckets);\n        }\n        \n        @Override\n        public <T> String getBucketString(final T value) {\n            return \":\" + Math.abs((value.hashCode() % buckets));\n        }\n        \n        @Override\n        public String toString() {\n            return \"APPEND_INCREMENT_BUCKETS_WITH_HASH(\" + buckets + \")\";\n        }\n    }\n\n    /**\n     * Indicates that the new append, increment and partial read operations of\n     * Scalaris should be used and list values should be distributed among\n     * several partitions, i.e. buckets, depending on the value's hash.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class APPEND_INCREMENT_PARTIALREAD_BUCKETS_WITH_HASH extends\n            APPEND_INCREMENT_BUCKETS_WITH_HASH implements IPartialRead {\n        /**\n         * Constructor.\n         * \n         * @param buckets\n         *            number of available buckets\n         */\n        public APPEND_INCREMENT_PARTIALREAD_BUCKETS_WITH_HASH(int buckets) {\n            super(buckets);\n        }\n        \n        @Override\n        public String toString() {\n            return \"APPEND_INCREMENT_PARTIALREAD_BUCKETS_WITH_HASH(\" + buckets + \")\";\n        }\n    }\n\n    /**\n     * Indicates that the new append and increment operations of Scalaris should\n     * be used and list values should be distributed among several partitions,\n     * i.e. buckets, using a set of read-buckets and another set as a small\n     * write cache (write-buckets).\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static abstract class APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY\n            implements Optimisation, IAppendIncrement, IReadBuckets {\n        final protected int readBuckets;\n        final protected int writeBuckets;\n        \n        /**\n         * Constructor.\n         * \n         * @param readBuckets\n         *            number of available read buckets\n         * @param writeBuckets\n         *            number of available read buckets\n         */\n        public APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY(int readBuckets,\n                int writeBuckets) {\n            assert (readBuckets >= 1 && writeBuckets >= 1);\n            this.readBuckets = readBuckets;\n            this.writeBuckets = writeBuckets;\n        }\n\n        public int getBuckets() {\n            return readBuckets + writeBuckets;\n        }\n\n        public int getReadBuckets() {\n            return readBuckets;\n        }\n\n        /**\n         * Gets the string to append to the key in order to point to a\n         * read-bucket for the given value.\n         * \n         * @return the bucket string, e.g. \":0\"\n         */\n        abstract public <T> String getReadBucketString();\n        \n        /**\n         * Gets the string to append to the key in order to point to a\n         * write-bucket for the given value.\n         * \n         * @return the bucket string, e.g. \":0\"\n         */\n        abstract public <T> String getWriteBucketString();\n    }\n\n    /**\n     * Indicates that the new append and increment operations of Scalaris should\n     * be used and list values should be distributed among several partitions,\n     * i.e. buckets, using a set of read-buckets and another set as a small\n     * write cache (write-buckets). Supports only list add operations (no\n     * removal!)\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM extends\n            APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY {\n        final static protected Random rand = new Random();\n\n        /**\n         * Constructor.\n         * \n         * @param readBuckets\n         *            number of available read buckets\n         * @param writeBuckets\n         *            number of available write buckets\n         */\n        public APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM(\n                int readBuckets, int writeBuckets) {\n            super(readBuckets, writeBuckets);\n        }\n\n        @Override\n        public <T> String getReadBucketString() {\n            return \":\" + (rand.nextInt(readBuckets));\n        }\n\n        @Override\n        public <T> String getWriteBucketString() {\n            return \":\" + (rand.nextInt(writeBuckets) + readBuckets);\n        }\n        \n        @Override\n        public String toString() {\n            return \"APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM(\"\n                    + readBuckets + \",\" + writeBuckets + \")\";\n        }\n    }\n\n    /**\n     * Indicates that the new append, increment and partial read operations of\n     * Scalaris should be used and list values should be distributed among\n     * several partitions, i.e. buckets, using a set of read-buckets and another\n     * set as a small write cache (write-buckets). Supports only list add\n     * operations (no removal!)\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class APPEND_INCREMENT_PARTIALREAD_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM\n            extends APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM\n            implements IPartialRead {\n        /**\n         * Constructor.\n         * \n         * @param readBuckets\n         *            number of available read buckets\n         * @param writeBuckets\n         *            number of available write buckets\n         */\n        public APPEND_INCREMENT_PARTIALREAD_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM(\n                int readBuckets, int writeBuckets) {\n            super(readBuckets, writeBuckets);\n        }\n\n        @Override\n        public String toString() {\n            return \"APPEND_INCREMENT_PARTIALREAD_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM(\"\n                    + readBuckets + \",\" + writeBuckets + \")\";\n        }\n    }\n\n    /**\n     * Indicates that the new append and increment operations of Scalaris should\n     * be used and list values should be distributed among several partitions,\n     * i.e. buckets, using a set of read-buckets and another set as a small\n     * write cache (write-buckets). In contrast to\n     * {@link APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY} this class also\n     * supports add and delete operations by tagging values with\n     * {@link #makeAdd(Object)} and {@link #makeDelete(Object)}, respectively,\n     * when writing to any of the write buckets.\n     * \n     * Make sure to only use the tagged values when writing values to a write\n     * bucket! Use {@link WriteCacheDiffConv} to get a converter that\n     * creates a {@link WriteCacheDiff} object which can then be used to create\n     * a global view of the whole stored list including the write cache.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static abstract class APPEND_INCREMENT_BUCKETS_WITH_WCACHE implements\n            Optimisation, IAppendIncrement, IReadBuckets {\n        final protected int readBuckets;\n        final protected int writeBuckets;\n\n        /**\n         * Constructor.\n         * \n         * @param readBuckets\n         *            number of available read buckets\n         * @param writeBuckets\n         *            number of available write buckets for new/deleted elements\n         *            (compared to the data in the read buckets);\n         *            two different write buckets of this size will be used\n         */\n        public APPEND_INCREMENT_BUCKETS_WITH_WCACHE(int readBuckets, int writeBuckets) {\n            assert (readBuckets >= 1 && writeBuckets >= 1);\n            this.readBuckets = readBuckets;\n            this.writeBuckets = writeBuckets;\n        }\n\n        public int getBuckets() {\n            return readBuckets + writeBuckets;\n        }\n\n        public int getReadBuckets() {\n            return readBuckets;\n        }\n\n        /**\n         * Gets the number write buckets.\n         * \n         * @return number of buckets to use for each write bucket type (add and\n         *         del)\n         */\n        public int getWriteBuckets() {\n            return writeBuckets;\n        }\n\n        /**\n         * Hashes the given value to a bucket.\n         * \n         * @param value\n         *            the value to check the bucket for\n         * @param buckets\n         *            number of available buckets for this hash operation\n         * @param offset\n         *            first bucket number\n         * \n         * @return the bucket, e.g. 0\n         */\n        abstract protected <T> int hashToBucket(T value, int buckets, int offset);\n\n        /**\n         * Gets the string to append to the key in order to point to a\n         * read-bucket for the given value.\n         * \n         * @param value\n         *            the value to check the bucket for\n         * \n         * @return the bucket string, e.g. \":0\"\n         */\n        public <T> String getReadBucketString(final T value) {\n            return \":\" + hashToBucket(value, readBuckets, 0);\n        }\n\n        /**\n         * Gets the string to append to the key in order to point to a\n         * write-bucket for the given value.\n         * \n         * @param value\n         *            the value to check the bucket for\n         * \n         * @return the bucket string, e.g. \":0\"\n         */\n        public <T> String getWriteBucketString(final T value) {\n            return \":\" + hashToBucket(value, writeBuckets, readBuckets);\n        }\n        \n        /**\n         * Tags this value as an \"add\" operation to be written to the write\n         * cache.\n         * \n         * @param value\n         *            the value to tag\n         * \n         * @return the value to use when writing to the write cache\n         */\n        public final <T> Object makeAdd(final T value) {\n            return Arrays.asList(1, value);\n        }\n\n        /**\n         * Tags this value as a \"delete\" operation to be written to the write\n         * cache.\n         * \n         * @param value\n         *            the value to tag\n         * \n         * @return the value to use when writing to the write cache\n         */\n        public final <T> Object makeDelete(final T value) {\n            return Arrays.asList(-1, value);\n        }\n\n        /**\n         * Result object that represents the write cache.\n         * \n         * @param <T>\n         *            the element type\n         * \n         * @see WriteCacheDiffConv\n         */\n        public static class WriteCacheDiff<T> {\n            /**\n             * List of elements to add to the read-buckets.\n             */\n            final public List<T> toAdd;\n            /**\n             * List of elements to remove from the read-buckets.\n             */\n            final public Set<T> toDelete;\n            \n            protected WriteCacheDiff(List<T> toAdd, Set<T> toDelete) {\n                this.toAdd = toAdd;\n                this.toDelete = toDelete;\n            }\n        }\n        \n        /**\n         * Converter that converts the write cache to a {@link WriteCacheDiff}\n         * object.\n         * \n         * @author Nico Kruber, kruber@zib.de\n         * \n         * @param <T>\n         *            element type to convert\n         */\n        public static class WriteCacheDiffConv<T>\n                implements\n                ErlangConverter<Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE.WriteCacheDiff<T>> {\n            private final ErlangConverter<T> elemConv;\n\n            /**\n             * Creates a converter to use for simplified access to the write\n             * cache.\n             * \n             * @param elemConv\n             *            converter for a single entry of the list\n             */\n            public WriteCacheDiffConv(ErlangConverter<T> elemConv) {\n                this.elemConv = elemConv;\n            }\n\n            @Override\n            public WriteCacheDiff<T> convert(ErlangValue v)\n                    throws ClassCastException {\n                final List<ErlangValue> listValue = v.listValue();\n                final ArrayList<T> toAdd = new ArrayList<T>(listValue.size());\n                final HashSet<T> toDelete = new HashSet<T>(listValue.size());\n                for(ErlangValue elem : listValue) {\n                    List<ErlangValue> diffObj = elem.listValue();\n                    if (diffObj.size() != 2) {\n                        throw new ClassCastException();\n                    }\n                    int tag = diffObj.get(0).intValue();\n                    T value = elemConv.convert(diffObj.get(1));\n                    switch (tag) {\n                        case 1:\n                            toAdd.add(value);\n                            break;\n                        case -1:\n                            toDelete.add(value);\n                            break;\n                        default:\n                            throw new ClassCastException();\n                    }\n                }\n                return new WriteCacheDiff<T>(toAdd, toDelete);\n            }\n        }\n    }\n\n    /**\n     * Indicates that the new append and increment operations of Scalaris should\n     * be used and list values should be distributed among several partitions,\n     * i.e. buckets, using a set of read-buckets and another set as a small\n     * write cache (write-buckets) for added and removed values. The values will\n     * be added to the write cache based on their hash.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class APPEND_INCREMENT_BUCKETS_WITH_WCACHE_HASH extends\n            APPEND_INCREMENT_BUCKETS_WITH_WCACHE {\n        final static protected Random rand = new Random();\n\n        /**\n         * Constructor.\n         * \n         * @param readBuckets\n         *            number of available read buckets\n         * @param writeBuckets\n         *            number of available write buckets for new/deleted elements\n         *            (compared to the data in the read buckets);\n         *            two different write buckets of this size will be used\n         */\n        public APPEND_INCREMENT_BUCKETS_WITH_WCACHE_HASH(int readBuckets,\n                int writeBuckets) {\n            super(readBuckets, writeBuckets);\n        }\n\n        @Override\n        protected <T> int hashToBucket(T value, int buckets, int offset) {\n            return Math.abs((value.hashCode() % buckets)) + offset;\n        }\n\n        @Override\n        public String toString() {\n            return \"APPEND_INCREMENT_BUCKETS_WITH_WCACHE_HASH(\" + readBuckets\n                    + \",\" + writeBuckets + \")\";\n        }\n    }\n\n    \n    /**\n     * Parses the given option strings into their appropriate properties.\n     * \n     * @param options\n     *            the {@link Options} object to parse into\n     * @param SERVERNAME\n     *            {@link Options#SERVERNAME}\n     * @param SERVERPATH\n     *            {@link Options#SERVERPATH}\n     * @param WIKI_USE_BACKLINKS\n     *            {@link Options#WIKI_USE_BACKLINKS}\n     * @param WIKI_SAVEPAGE_RETRIES\n     *            {@link Options#WIKI_SAVEPAGE_RETRIES}\n     * @param WIKI_SAVEPAGE_RETRY_DELAY\n     *            {@link Options#WIKI_SAVEPAGE_RETRY_DELAY}\n     * @param WIKI_PAGES_CACHE_IMPL\n     *            {@link Options#WIKI_PAGES_CACHE_IMPL}\n     * @param WIKI_REBUILD_PAGES_CACHE\n     *            {@link Options#WIKI_REBUILD_PAGES_CACHE}\n     * @param WIKI_STORE_CONTRIBUTIONS\n     *            {@link Options#WIKI_STORE_CONTRIBUTIONS}\n     * @param OPTIMISATIONS\n     *            {@link Options#OPTIMISATIONS}\n     * @param LOG_USER_REQS\n     *            {@link Options#LOG_USER_REQS}\n     * @param SCALARIS_NODE_DISCOVERY\n     *            {@link Options#SCALARIS_NODE_DISCOVERY}\n     */\n    public static void parseOptions(Options options, final String SERVERNAME, final String SERVERPATH,\n            final String WIKI_USE_BACKLINKS,\n            final String WIKI_SAVEPAGE_RETRIES,\n            final String WIKI_SAVEPAGE_RETRY_DELAY,\n            final String WIKI_PAGES_CACHE_IMPL,\n            final String WIKI_REBUILD_PAGES_CACHE,\n            final String WIKI_STORE_CONTRIBUTIONS, final String OPTIMISATIONS,\n            final String LOG_USER_REQS, final String SCALARIS_NODE_DISCOVERY) {\n        if (SERVERNAME != null) {\n            options.SERVERNAME = SERVERNAME;\n        }\n        if (SERVERPATH != null) {\n            options.SERVERPATH = SERVERPATH;\n        }\n        if (WIKI_USE_BACKLINKS != null) {\n            options.WIKI_USE_BACKLINKS = Boolean.parseBoolean(WIKI_USE_BACKLINKS);\n        }\n        if (WIKI_SAVEPAGE_RETRIES != null) {\n            options.WIKI_SAVEPAGE_RETRIES = Integer.parseInt(WIKI_SAVEPAGE_RETRIES);\n        }\n        if (WIKI_SAVEPAGE_RETRY_DELAY != null) {\n            options.WIKI_SAVEPAGE_RETRY_DELAY = Integer.parseInt(WIKI_SAVEPAGE_RETRY_DELAY);\n        }\n        if (WIKI_PAGES_CACHE_IMPL != null) {\n            if (WIKI_PAGES_CACHE_IMPL.equals(\"BLOOM\")) {\n                options.WIKI_PAGES_CACHE_IMPL = ExistingPagesCacheBloom.class;\n            } else if (WIKI_PAGES_CACHE_IMPL.equals(\"FULL_SET\")) {\n                options.WIKI_PAGES_CACHE_IMPL = ExistingPagesCacheFull.class;\n            } else {\n                System.err.println(\"unknown WIKI_PAGES_CACHE_IMPL found: \" + WIKI_PAGES_CACHE_IMPL);\n            }\n        }\n        if (WIKI_REBUILD_PAGES_CACHE != null) {\n            options.WIKI_REBUILD_PAGES_CACHE = Integer.parseInt(WIKI_REBUILD_PAGES_CACHE);\n        }\n        if (WIKI_STORE_CONTRIBUTIONS != null) {\n            options.WIKI_STORE_CONTRIBUTIONS = STORE_CONTRIB_TYPE.fromString(WIKI_STORE_CONTRIBUTIONS);\n        }\n        if (OPTIMISATIONS != null) {\n            for (String singleOpt : OPTIMISATIONS.split(\"\\\\|\")) {\n                final Matcher matcher = CONFIG_SINGLE_OPTIMISATION.matcher(singleOpt);\n                if (matcher.matches()) {\n                    final String operationStr = matcher.group(1);\n                    if (operationStr.equals(\"ALL\")) {\n                        Optimisation optimisation = parseOptimisationString(matcher);\n                        if (optimisation == null) {\n                            // fall back if not parsed correctly:\n                            optimisation = new APPEND_INCREMENT();\n                        }\n                        for (ScalarisOpType op : ScalarisOpType.values()) {\n                            if (!(op.equals(ScalarisOpType.PAGE_COUNT) ||\n                                    op.equals(ScalarisOpType.CATEGORY_PAGE_COUNT))) {\n                                options.OPTIMISATIONS.put(op, optimisation);\n                            }\n                        }\n                    } else {\n                        ScalarisOpType operation = ScalarisOpType.fromString(operationStr);\n                        Optimisation optimisation = parseOptimisationString(matcher);\n                        if (optimisation != null) {\n                            options.OPTIMISATIONS.put(operation, optimisation);\n                        }\n                    }\n                }\n            }\n        }\n        if (LOG_USER_REQS != null) {\n            options.LOG_USER_REQS = Integer.parseInt(LOG_USER_REQS);\n        }\n        if (SCALARIS_NODE_DISCOVERY != null) {\n            options.SCALARIS_NODE_DISCOVERY = Integer.parseInt(SCALARIS_NODE_DISCOVERY);\n        }\n    }\n\n    /**\n     * Parses an optimisation string into an {@link Optimisation} object.\n     * \n     * @param matcher\n     *            matcher (1st group: group of keys to apply to, 2nd group:\n     *            optimisation class, 3rd group: optimisation parameters\n     * \n     * @return an {@link Optimisation} implementation or <tt>null</tt> if no\n     *         matching optimisation was found\n     * @throws NumberFormatException\n     *             if an integer parameter was wrong\n     */\n    public static Optimisation parseOptimisationString(final Matcher matcher)\n            throws NumberFormatException {\n        String optimisationStr = matcher.group(2);\n        String parameterStr = matcher.group(3);\n        if (optimisationStr.equals(\"TRADITIONAL\") && parameterStr == null) {\n            return new Options.TRADITIONAL();\n        } else if (optimisationStr.equals(\"APPEND_INCREMENT\")\n                && parameterStr == null) {\n            return new Options.APPEND_INCREMENT();\n        } else if (optimisationStr.equals(\"APPEND_INCREMENT_PARTIALREAD\")\n                && parameterStr == null) {\n            return new Options.APPEND_INCREMENT_PARTIALREAD();\n        } else if (optimisationStr.equals(\"APPEND_INCREMENT_BUCKETS_RANDOM\")\n                && parameterStr != null) {\n            String[] parameters = parameterStr.split(\",\");\n            return new Options.APPEND_INCREMENT_BUCKETS_RANDOM(\n                    Integer.parseInt(parameters[0]));\n        } else if (optimisationStr.equals(\"APPEND_INCREMENT_PARTIALREAD_BUCKETS_RANDOM\")\n                && parameterStr != null) {\n            String[] parameters = parameterStr.split(\",\");\n            return new Options.APPEND_INCREMENT_PARTIALREAD_BUCKETS_RANDOM(\n                    Integer.parseInt(parameters[0]));\n        } else if (optimisationStr.equals(\"APPEND_INCREMENT_BUCKETS_WITH_HASH\")\n                && parameterStr != null) {\n            String[] parameters = parameterStr.split(\",\");\n            return new Options.APPEND_INCREMENT_BUCKETS_WITH_HASH(\n                    Integer.parseInt(parameters[0]));\n        } else if (optimisationStr.equals(\"APPEND_INCREMENT_PARTIALREAD_BUCKETS_WITH_HASH\")\n                && parameterStr != null) {\n            String[] parameters = parameterStr.split(\",\");\n            return new Options.APPEND_INCREMENT_PARTIALREAD_BUCKETS_WITH_HASH(\n                    Integer.parseInt(parameters[0]));\n        } else if (optimisationStr.equals(\"APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM\")\n                && parameterStr != null) {\n            String[] parameters = parameterStr.split(\",\");\n            return new Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM(\n                    Integer.parseInt(parameters[0]),\n                    Integer.parseInt(parameters[1]));\n        } else if (optimisationStr.equals(\"APPEND_INCREMENT_PARTIALREAD_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM\")\n                && parameterStr != null) {\n            String[] parameters = parameterStr.split(\",\");\n            return new Options.APPEND_INCREMENT_PARTIALREAD_BUCKETS_WITH_WCACHE_ADDONLY_RANDOM(\n                    Integer.parseInt(parameters[0]),\n                    Integer.parseInt(parameters[1]));\n        } else if (optimisationStr.equals(\"APPEND_INCREMENT_BUCKETS_WITH_WCACHE_HASH\") && parameterStr != null) {\n            String[] parameters = parameterStr.split(\",\");\n            return new Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE_HASH(\n                    Integer.parseInt(parameters[0]),\n                    Integer.parseInt(parameters[1]));\n        }\n        System.err.println(\"unknown optimisation found: \" + matcher.group());\n        return null;\n    }\n\n    protected static class WebXmlInitParamHandler extends DefaultHandler {\n        public final Map<String, String> initParams = new HashMap<String, String>();\n        private final Map<String, String> curInitParams = new HashMap<String, String>();\n        protected StringBuilder curString = new StringBuilder();\n        protected String curInitParamName = null;\n        protected String curInitParamValue = null;\n        private boolean inWebApp = false;\n        private boolean inServlet = false;\n        private boolean inWikiServlet = false;\n        private boolean inInitParam = false;\n        private boolean parseContent = false;\n        \n\n        /* (non-Javadoc)\n         * @see org.xml.sax.helpers.DefaultHandler#startElement(java.lang.String, java.lang.String, java.lang.String, org.xml.sax.Attributes)\n         */\n        @Override\n        public void startElement(String uri, String localName, String qName,\n                Attributes attributes) throws SAXException {\n            /*\n             * <web-app ...>\n             *  <display-name>...</display-name>\n             *  <servlet>\n             *   <description>...</description>\n             *   <display-name>...</display-name>\n             *   <servlet-name>...</servlet-name>\n             *   <servlet-class>de.zib.scalaris.examples.wikipedia.bliki.WikiServletScalaris</servlet-class>\n             *   <init-param>\n             *    <param-name>\n             *     SERVERNAME|LOG_USER_REQS|SCALARIS_NODE_DISCOVERY|SERVERPATH|\n             *     WIKI_USE_BACKLINKS|WIKI_SAVEPAGE_RETRIES|WIKI_SAVEPAGE_RETRY_DELAY|\n             *     WIKI_PAGES_CACHE_IMPL|WIKI_REBUILD_PAGES_CACHE|WIKI_STORE_CONTRIBUTIONS|\n             *     WIKI_OPTIMISATIONS|...\n             *    </param-name>\n             *    <param-value>...</param-value>\n             *   </init-param>\n             *  </servlet>\n             *  ...\n             * </web-app>\n             */\n            if (inWebApp) {\n                if (inServlet) {\n                    if (inInitParam) {\n                        if (localName.equals(\"param-name\")) {\n                            parseContent();\n                        } else if (localName.equals(\"param-value\")) {\n                            parseContent();\n                        } else {\n                            throw new SAXException(\"unknown tag in <init-param>: \" + localName);\n                        }\n                    } else if (localName.equals(\"servlet-class\")) {\n                        parseContent();\n                    } else if (localName.equals(\"init-param\")) {\n                        inInitParam = true;\n                    }\n                    // ignore other tags\n                } else if (localName.equals(\"servlet\")) {\n                    inServlet = true;\n                }\n                // ignore other tags\n            } else if (localName.equals(\"web-app\")) {\n                inWebApp = true;\n            } else {\n                throw new SAXException(\"unknown tag in root: \" + localName);\n            }\n        }\n\n        private void parseContent() {\n            parseContent = true;\n            curString.setLength(0);\n        }\n\n        /* (non-Javadoc)\n         * @see org.xml.sax.helpers.DefaultHandler#characters(char[], int, int)\n         */\n        @Override\n        public void characters(char[] ch, int start, int length)\n                throws SAXException {\n            if (parseContent) {\n                curString.append(ch, start, length);\n            }\n        }\n\n        /* (non-Javadoc)\n         * @see org.xml.sax.helpers.DefaultHandler#endElement(java.lang.String, java.lang.String, java.lang.String)\n         */\n        @Override\n        public void endElement(String uri, String localName, String qName)\n                throws SAXException {\n            if (inWebApp) {\n                if (inServlet) {\n                    if (inInitParam) {\n                        if (localName.equals(\"param-name\")) {\n                            parseContent = false;\n                            curInitParamName = curString.toString();\n                        } else if (localName.equals(\"param-value\")) {\n                            parseContent = false;\n                            curInitParamValue = curString.toString();\n                        } else if (localName.equals(\"init-param\")) {\n                            inInitParam = false;\n                            curInitParams.put(curInitParamName, curInitParamValue);\n                        } else {\n                            throw new SAXException(\"unknown tag in <init-param>: \" + localName);\n                        }\n                    } else if (localName.equals(\"servlet-class\")) {\n                        parseContent = false;\n                        String servletClass = curString.toString();\n                        if (servletClass.equals(\"de.zib.scalaris.examples.wikipedia.bliki.WikiServletScalaris\")) {\n                            inWikiServlet = true;\n                        }\n                    } else if (localName.equals(\"init-param\")) {\n                        throw new SAXException(\"closing </init-param> without matching start\");\n                    } else if (localName.equals(\"servlet\")) {\n                        if (inWikiServlet) {\n                            initParams.putAll(curInitParams);\n                        }\n                        inServlet = false;\n                        inWikiServlet = false;\n                    }\n                    // ignore other tags\n                } else if (localName.equals(\"servlet\")) {\n                    throw new SAXException(\"closing </servlet> without matching start\");\n                }\n                // ignore other tags\n            } else if (localName.equals(\"web-app\")) {\n                inWebApp = false;\n            } else {\n                throw new SAXException(\"unknown tag in root: \" + localName);\n            }\n        }\n        \n    }\n\n    /**\n     * Parses the given input file as a <tt>web.xml</tt> servlet descriptor for\n     * servlet configuration options.\n     * \n     * @param options\n     *            the {@link Options} object to parse into\n     * @param filename\n     *            the name of the file to parse\n     */\n    public static void parseOptions(Options options, String filename) {\n        WebXmlInitParamHandler handler;\n        try {\n            FileInputStream is = new FileInputStream(filename);\n            BufferedReader br = new BufferedReader(new InputStreamReader(is, \"UTF-8\"));\n            handler = new WebXmlInitParamHandler();\n            XMLReader reader = XMLReaderFactory.createXMLReader();\n            reader.setContentHandler(handler);\n            reader.parse(new InputSource(br));\n            parseOptions(options,\n                    handler.initParams.get(\"SERVERNAME\"),\n                    handler.initParams.get(\"SERVERPATH\"),\n                    handler.initParams.get(\"WIKI_USE_BACKLINKS\"),\n                    handler.initParams.get(\"WIKI_SAVEPAGE_RETRIES\"),\n                    handler.initParams.get(\"WIKI_SAVEPAGE_RETRY_DELAY\"),\n                    handler.initParams.get(\"WIKI_PAGES_CACHE_IMPL\"),\n                    handler.initParams.get(\"WIKI_REBUILD_PAGES_CACHE\"),\n                    handler.initParams.get(\"WIKI_STORE_CONTRIBUTIONS\"),\n                    handler.initParams.get(\"WIKI_OPTIMISATIONS\"),\n                    handler.initParams.get(\"LOG_USER_REQS\"),\n                    handler.initParams.get(\"SCALARIS_NODE_DISCOVERY\"));\n        } catch (Exception e) {\n            System.err.println(\"parsing failed: \" + e.getMessage());\n            e.printStackTrace();\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/PageHistoryResult.java",
    "content": "package de.zib.scalaris.examples.wikipedia;\n\nimport java.util.List;\n\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.ShortRevision;\n\n/**\n * Result of an operation getting the page history.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class PageHistoryResult extends Result {\n    /**\n     * The retrieved page on success (or <tt>null</tt>).\n     */\n    public Page page = null;\n    /**\n     * The retrieved (short) revisions on success (or <tt>null</tt>).\n     */\n    public List<ShortRevision> revisions = null;\n    /**\n     * Whether the page exists or not.\n     */\n    public boolean not_existing = false;\n\n    /**\n     * Creates a successful result with an empty message and the given\n     * revisions.\n     * \n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param page\n     *            the retrieved page\n     * @param revisions\n     *            the retrieved (short) revisions\n     */\n    public PageHistoryResult(List<InvolvedKey> involvedKeys, Page page, List<ShortRevision> revisions) {\n        super(involvedKeys);\n        this.page = page;\n        this.revisions = revisions;\n    }\n\n    /**\n     * Creates a successful result with an empty message and the given\n     * revisions.\n     * \n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param page\n     *            the retrieved page\n     * @param revisions\n     *            the retrieved (short) revisions\n     * @param name\n     *            the name of the operation (for the stats - see {@link #stats})\n     * @param time\n     *            time in milliseconds for this operation\n     */\n    public PageHistoryResult(List<InvolvedKey> involvedKeys, Page page, List<ShortRevision> revisions,\n            String name, long time) {\n        super(involvedKeys);\n        this.page = page;\n        this.revisions = revisions;\n        addStat(name, time);\n    }\n\n    /**\n     * Creates a new custom result.\n     * \n     * @param success\n     *            the success status\n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param message\n     *            the message to use\n     * @param connectFailed\n     *            whether the connection to the DB failed or not\n     */\n    public PageHistoryResult(boolean success, List<InvolvedKey> involvedKeys,\n            String message, boolean connectFailed) {\n        super(success, involvedKeys, message, connectFailed);\n    }\n\n    /**\n     * Creates a new custom result.\n     * \n     * @param success\n     *            the success status\n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param message\n     *            the message to use\n     * @param connectFailed\n     *            whether the connection to the DB failed or not\n     * @param name\n     *            the name of the operation (for the stats - see {@link #stats})\n     * @param time\n     *            time in milliseconds for this operation\n     */\n    public PageHistoryResult(boolean success, List<InvolvedKey> involvedKeys,\n            String message, boolean connectFailed, String name, long time) {\n        super(success, involvedKeys, message, connectFailed);\n        addStat(name, time);\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/Result.java",
    "content": "package de.zib.scalaris.examples.wikipedia;\n\nimport java.util.ArrayList;\nimport java.util.List;\n\nimport de.zib.tools.LinkedMultiHashMap;\n\n\n/**\n * Common result class with a public member containing the result and a\n * message.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class Result {\n    /**\n     * Whether an operation was a success or not.\n     */\n    public boolean success;\n    /**\n     * An additional message (mostly used with unsuccessful operations).\n     */\n    public String message;\n    /**\n     * Indicates whether the connection to the DB failed or not.\n     */\n    public boolean connect_failed;\n    /**\n     * Time in milliseconds for this operation (one entry for each call to the\n     * DB).\n     */\n    public LinkedMultiHashMap<String, Long> stats = new LinkedMultiHashMap<String, Long>();\n    /**\n     * All keys that have been read or written during the operation.\n     */\n    public List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n    \n    /**\n     * Creates a successful result with an empty message.\n     * \n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     */\n    public Result(List<InvolvedKey> involvedKeys) {\n        this.success = true;\n        this.involvedKeys = involvedKeys;\n        this.message = \"\";\n        this.connect_failed = false;\n    }\n    \n    /**\n     * Creates a new custom result.\n     * \n     * @param success\n     *            the success status\n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param message\n     *            the message to use\n     * @param connectFailed\n     *            whether the connection to the DB failed or not\n     */\n    public Result(boolean success, List<InvolvedKey> involvedKeys, String message, boolean connectFailed) {\n        this.success = success;\n        this.involvedKeys = involvedKeys;\n        this.message = message;\n        this.connect_failed = connectFailed;\n        assert(!connect_failed || !success);\n    }\n\n    /**\n     * Adds the time needed to retrieve the given page to the collected\n     * statistics.\n     * \n     * @param name\n     *            the name of the operation\n     * @param time\n     *            time in milliseconds for this operation\n     */\n    public void addStat(String name, long time) {\n        stats.put1(name, time);\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/RevisionResult.java",
    "content": "package de.zib.scalaris.examples.wikipedia;\n\nimport java.util.List;\n\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\n\n/**\n * Result of an operation getting a revision.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class RevisionResult extends Result {\n    /**\n     * Normalised page title.\n     */\n    final public NormalisedTitle normalisedTitle;\n    /**\n     * Revision on success.\n     */\n    public Revision revision = null;\n    /**\n     * Page on success (if retrieved).\n     */\n    public Page page = null;\n    /**\n     * whether the page exists or not (if <tt>true</tt>, {@link Result#success}\n     * must also be <tt>true</tt>).\n     */\n    public boolean page_not_existing = false;\n    /**\n     * Whether the requested revision exists or not (if <tt>true</tt>,\n     * {@link Result#success} must also be <tt>true</tt>).\n     */\n    public boolean rev_not_existing = false;\n    \n    /**\n     * Creates a new successful result with the given revision.\n     * \n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param normalisedTitle\n     *            normalised page title\n     * @param page\n     *            the retrieved page\n     * @param revision\n     *            the retrieved revision\n     */\n    public RevisionResult(List<InvolvedKey> involvedKeys, NormalisedTitle normalisedTitle, Page page,\n            Revision revision) {\n        super(involvedKeys);\n        this.normalisedTitle = normalisedTitle;\n        this.page = page;\n        this.revision = revision;\n    }\n    \n    /**\n     * Creates a new successful result with the given revision.\n     * \n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param normalisedTitle\n     *            normalised page title\n     * @param page\n     *            the retrieved page\n     * @param revision\n     *            the retrieved revision\n     * @param statName\n     *            the name of the operation (for the stats - see {@link #stats})\n     * @param time\n     *            time in milliseconds for this operation\n     */\n    public RevisionResult(List<InvolvedKey> involvedKeys, NormalisedTitle normalisedTitle, Page page,\n            Revision revision, String statName, long time) {\n        super(involvedKeys);\n        this.normalisedTitle = normalisedTitle;\n        this.page = page;\n        this.revision = revision;\n        addStat(statName, time);\n    }\n    \n    /**\n     * Creates a new custom result.\n     * \n     * @param success\n     *            the success status\n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param message\n     *            the message to use\n     * @param connectFailed\n     *            whether the connection to the DB failed or not\n     * @param normalisedTitle\n     *            normalised page title\n     * @param page\n     *            page on success (if retrieved)\n     * @param revision\n     *            revision on success\n     * @param page_not_existing\n     *            whether the page exists or not (if <tt>true</tt>,\n     *            {@link Result#success} must also be <tt>true</tt>)\n     * @param rev_not_existing\n     *            whether the requested revision exists or not (if <tt>true</tt>,\n     *            {@link Result#success} must also be <tt>true</tt>)\n     */\n    public RevisionResult(boolean success, List<InvolvedKey> involvedKeys,\n            String message, boolean connectFailed, NormalisedTitle normalisedTitle, Page page,\n            Revision revision, boolean page_not_existing,\n            boolean rev_not_existing) {\n        super(success, involvedKeys, message, connectFailed);\n        this.normalisedTitle = normalisedTitle;\n        this.page = page;\n        this.revision = revision;\n        this.page_not_existing = page_not_existing;\n        this.rev_not_existing = rev_not_existing;\n        assert(!rev_not_existing || !success);\n        assert(!page_not_existing || !success);\n    }\n    \n    /**\n     * Creates a new custom result.\n     * \n     * @param success\n     *            the success status\n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param message\n     *            the message to use\n     * @param connectFailed\n     *            whether the connection to the DB failed or not\n     * @param normalisedTitle\n     *            normalised page title\n     * @param page\n     *            page on success (if retrieved)\n     * @param revision\n     *            revision on success\n     * @param page_not_existing\n     *            whether the page exists or not (if <tt>true</tt>,\n     *            {@link Result#success} must also be <tt>true</tt>)\n     * @param rev_not_existing\n     *            whether the requested revision exists or not (if <tt>true</tt>,\n     *            {@link Result#success} must also be <tt>true</tt>)\n     * @param name\n     *            the name of the operation (for the stats - see {@link #stats})\n     * @param time\n     *            time in milliseconds for this operation\n     */\n    public RevisionResult(boolean success, List<InvolvedKey> involvedKeys,\n            String message, boolean connectFailed, NormalisedTitle normalisedTitle, Page page,\n            Revision revision, boolean page_not_existing,\n            boolean rev_not_existing, String name, long time) {\n        super(success, involvedKeys, message, connectFailed);\n        this.normalisedTitle = normalisedTitle;\n        this.page = page;\n        this.revision = revision;\n        this.page_not_existing = page_not_existing;\n        this.rev_not_existing = rev_not_existing;\n        assert(!rev_not_existing || !success);\n        assert(!page_not_existing || !success);\n        addStat(name, time);\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/SQLiteDataHandler.java",
    "content": "/**\n * \n */\npackage de.zib.scalaris.examples.wikipedia;\n\nimport java.io.File;\nimport java.math.BigInteger;\nimport java.util.ArrayList;\nimport java.util.List;\n\nimport com.almworks.sqlite4java.SQLiteConnection;\nimport com.almworks.sqlite4java.SQLiteException;\nimport com.almworks.sqlite4java.SQLiteStatement;\n\nimport de.zib.scalaris.examples.wikipedia.bliki.MyNamespace;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.data.Contributor;\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\n\n/**\n * Retrieves and writes values from/to a SQLite DB.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class SQLiteDataHandler {\n\n    /**\n     * Opens a connection to a database and sets some default PRAGMAs for better\n     * performance in our case.\n     * \n     * @param fileName\n     *            the name of the DB file\n     * @param readOnly\n     *            whether to open the DB read-only or not\n     * @param cacheSize\n     *            cache size to set for the DB connection (default if\n     *            <tt>null</tt>)\n     * \n     * @return the DB connection\n     * \n     * @throws SQLiteException\n     *             if the connection fails or a pragma could not be set\n     */\n    public static SQLiteConnection openDB(String fileName, boolean readOnly, Long cacheSize) throws SQLiteException {\n        SQLiteConnection db = new SQLiteConnection(new File(fileName));\n        if (readOnly) {\n            db.openReadonly();\n        } else {\n            db.open(true);\n        }\n        // set cache_size:\n        if (cacheSize != null) {\n            final SQLiteStatement stmt = db.prepare(\"PRAGMA page_size;\");\n            if (stmt.step()) {\n                long pageSize = stmt.columnLong(0);\n                db.exec(\"PRAGMA cache_size = \" + (cacheSize / pageSize) + \";\");\n            }\n            stmt.dispose();\n        }\n        db.exec(\"PRAGMA synchronous = OFF;\");\n        db.exec(\"PRAGMA journal_mode = OFF;\");\n        //        db.exec(\"PRAGMA locking_mode = EXCLUSIVE;\");\n        db.exec(\"PRAGMA case_sensitive_like = true;\"); \n        db.exec(\"PRAGMA encoding = 'UTF-8';\"); \n        db.exec(\"PRAGMA temp_store = MEMORY;\"); \n        return db;\n    }\n\n    /**\n     * Opens a connection to a database and sets some default PRAGMAs for better\n     * performance in our case.\n     * \n     * @param fileName\n     *            the name of the DB file\n     * @param readOnly\n     *            whether to open the DB read-only or not\n     * \n     * @return the DB connection\n     * \n     * @throws SQLiteException\n     *             if the connection fails or a pragma could not be set\n     */\n    public static SQLiteConnection openDB(String fileName, boolean readOnly) throws SQLiteException {\n        return openDB(fileName, readOnly, null);\n    }\n\n    /**\n     * Wrapper for several prepared statements of a single SQLite connection.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class Connection {\n        /**\n         * The DB connection.\n         */\n        public final SQLiteConnection db;\n        \n        /**\n         * Allows retrieval of the latest revision of a page.\n         */\n        public SQLiteStatement stmtGetLatestRev;\n        \n        /**\n         * Allows retrieval of the total number of pages.\n         */\n        public SQLiteStatement stmtGetPageCount;\n        \n        /**\n         * Allows retrieval of the total number of articles.\n         */\n        public SQLiteStatement stmtGetArticleCount;\n        \n        /**\n         * Allows retrieval of the total number of pages inside a category.\n         */\n        public SQLiteStatement stmtGetPagesInCatCount;\n        \n        /**\n         * Allows retrieval of the list of pages inside a category.\n         */\n        public SQLiteStatement stmtGetPagesInCat;\n        \n        /**\n         * Allows retrieval of the list of pages using a template.\n         */\n        public SQLiteStatement stmtGetPagesInTpl;\n        \n        /**\n         * Allows retrieval of the list of pages linking to another page.\n         */\n        public SQLiteStatement stmtGetPagesLinkingTo;\n        \n        /**\n         * Creates all prepared statements and stores them in the object's\n         * members.\n         * \n         * @param connection\n         *            the SQLite connection to use\n         * \n         * @throws SQLiteException\n         *             if a prepared statement fails\n         */\n        public Connection(SQLiteConnection connection) throws SQLiteException {\n            this.db = connection;\n            initStmts();\n        }\n        \n        /**\n         * Creates all prepared statements and stores them in the object's\n         * members.\n         * \n         * @param fileName\n         *            the name of the DB file\n         * \n         * @throws SQLiteException\n         *             if a prepared statement fails\n         */\n        public Connection(String fileName) throws SQLiteException {\n            this.db = openDB(fileName, true);\n            initStmts();\n        }\n\n        protected void initStmts() throws SQLiteException {\n            stmtGetLatestRev = this.db.prepare(\"SELECT * FROM page \"\n                    + \"INNER JOIN revision ON page_latest == rev_id \"\n                    + \"INNER JOIN text ON rev_text_id == old_id \"\n                    + \"WHERE page_namespace == ? AND page_title == ?;\");\n            stmtGetPageCount = this.db\n                    .prepare(\"SELECT ss_total_pages FROM site_stats WHERE ss_row_id == 1;\");\n            stmtGetArticleCount = this.db\n                    .prepare(\"SELECT ss_good_articles FROM site_stats WHERE ss_row_id == 1;\");\n            stmtGetPagesInCatCount = this.db\n                    .prepare(\"SELECT COUNT(*) FROM categorylinks \"\n                            + \"WHERE cl_to = (SELECT page_id from page where page_namespace = \"\n                            + MyNamespace.CATEGORY_NAMESPACE_KEY\n                            + \" AND page_title = ?);\");\n            stmtGetPagesInCat = this.db\n                    .prepare(\"SELECT page_namespace, page_title FROM categorylinks \"\n                            + \"INNER JOIN page on cl_from = page_id \"\n                            + \"WHERE cl_to = (SELECT page_id from page where page_namespace = ? AND page_title = ?);\");\n            stmtGetPagesInTpl = this.db\n                    .prepare(\"SELECT page_namespace, page_title FROM templatelinks \"\n                            + \"INNER JOIN page on tl_from = page_id \"\n                            + \"WHERE tl_to = (SELECT page_id from page where page_namespace = ? AND page_title = ?);\");\n            stmtGetPagesLinkingTo = this.db\n                    .prepare(\"SELECT page_namespace, page_title FROM pagelinks \"\n                            + \"INNER JOIN page on pl_from = page_id \"\n                            + \"WHERE pl_to = (SELECT page_id from page where page_namespace = ? AND page_title = ?);\");\n        }\n        \n        /**\n         * Disposes all statements as well as the DB connection to free\n         * resources.\n         */\n        public void dispose() {\n            stmtGetLatestRev.dispose();\n            stmtGetPageCount.dispose();\n            stmtGetArticleCount.dispose();\n            stmtGetPagesInCatCount.dispose();\n            stmtGetPagesInCat.dispose();\n            db.dispose();\n        }\n    }\n    \n    /**\n     * Retrieves the current, i.e. most up-to-date, version of a page from\n     * the SQLite DB.\n     * \n     * @param connection\n     *            the connection to the SQLite DB\n     * @param title\n     *            the title of the page\n     * @param nsObject\n     *            the namespace for page title de-normalisation\n     * \n     * @return a result object with the page and revision on success\n     */\n    public static RevisionResult getRevision(Connection connection,\n            NormalisedTitle title, MyNamespace nsObject) {\n        final long timeAtStart = System.currentTimeMillis();\n        Page page = null;\n        Revision revision = null;\n        final List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        final String statName = \"PAGE:\" + title.toString();\n        if (connection == null) {\n            return new RevisionResult(false, involvedKeys,\n                    \"no connection to SQLite DB\", true, title, page, revision, false,\n                    false, statName, System.currentTimeMillis() - timeAtStart);\n        }\n        \n        final SQLiteStatement stmt = connection.stmtGetLatestRev;\n        try {\n            stmt.bind(1, title.namespace).bind(2, title.title);\n            if (stmt.step()) {\n                page = new Page();\n                page.setTitle(title.denormalise(nsObject));\n                revision = new Revision();\n                for (int i = 0; i < stmt.columnCount(); i++) {\n                    String columnName = stmt.getColumnName(i);\n                    if (columnName.equals(\"page_id\")) {\n                        page.setId(stmt.columnInt(i));\n                    } else if (columnName.equals(\"page_restrictions\")) {\n                        page.setRestrictions(Page.restrictionsFromString(stmt.columnString(i), \",\"));\n                    } else if (columnName.equals(\"page_is_redirect\")) {\n                        page.setRedirect(stmt.columnInt(i) != 0);\n                    } else if (columnName.equals(\"rev_id\")) {\n                        revision.setId(stmt.columnInt(i));\n                    } else if (columnName.equals(\"rev_comment\")) {\n                        revision.setComment(stmt.columnString(i));\n                    } else if (columnName.equals(\"rev_user_text\")) {\n                        Contributor contributor = new Contributor();\n                        contributor.setIp(stmt.columnString(i));\n                        revision.setContributor(contributor);\n                    } else if (columnName.equals(\"rev_timestamp\")) {\n                        revision.setTimestamp(stmt.columnString(i));\n                    } else if (columnName.equals(\"rev_minor_edit\")) {\n                        revision.setMinor(stmt.columnInt(i) != 0);\n                    } else if (columnName.equals(\"old_text\")) {\n                        revision.setPackedText(stmt.columnBlob(i));\n                    }\n                }\n                page.setCurRev(revision);\n            } else {\n                return new RevisionResult(false, involvedKeys,\n                        \"page \\\"\" + statName + \"\\\" not found\", false,\n                        title, page, revision, true, false, statName,\n                        System.currentTimeMillis() - timeAtStart);\n            }\n            // there should only be one data item\n            if (stmt.step()) {\n                return new RevisionResult(false, involvedKeys,\n                        \"more than one result\", false, title, page, revision, false,\n                        false, statName, System.currentTimeMillis() - timeAtStart);\n            }\n\n            return new RevisionResult(involvedKeys, title, page, revision, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        } catch (SQLiteException e) {\n            return new RevisionResult(false, involvedKeys, \"SQLite exception: \"\n                    + e.getMessage(), false, title, page, revision, false,\n                    false, statName, System.currentTimeMillis()\n                            - timeAtStart);\n        } finally {\n            try {\n                stmt.reset();\n            } catch (SQLiteException e) {\n            }\n        }\n    }\n\n    /**\n     * Retrieves the number of all available pages from the SQLite DB.\n     * \n     * @param connection\n     *            the connection to the SQLite DB\n     * \n     * @return a result object with the number of pages on success\n     */\n    public static ValueResult<BigInteger> getPageCount(Connection connection) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"PAGE_COUNT\";\n        final List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        if (connection == null) {\n            return new ValueResult<BigInteger>(false, involvedKeys,\n                    \"no connection to SQLite DB\", true, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n        return getInteger(connection.stmtGetPageCount, null, timeAtStart,\n                statName, involvedKeys);\n    }\n\n    /**\n     * Retrieves the number of available articles, i.e. pages in the main\n     * namespace, from the SQLite DB.\n     * \n     * @param connection\n     *            the connection to the SQLite DB\n     * \n     * @return a result object with the number of articles on success\n     */\n    public static ValueResult<BigInteger> getArticleCount(Connection connection) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"ARTICLE_COUNT\";\n        final List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        if (connection == null) {\n            return new ValueResult<BigInteger>(false, involvedKeys,\n                    \"no connection to SQLite DB\", true, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n        return getInteger(connection.stmtGetArticleCount, null, timeAtStart,\n                statName, involvedKeys);\n    }\n\n    /**\n     * Retrieves the number of all pages in the given category from the SQLite DB.\n     * \n     * @param connection\n     *            the connection to the SQLite DB\n     * @param title\n     *            the title of the category\n     * \n     * @return a result object with the number of pages on success\n     */\n    public static ValueResult<BigInteger> getPagesInCategoryCount(\n            Connection connection, NormalisedTitle title) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"CAT_CNT:\" + title;\n        final List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        if (connection == null) {\n            return new ValueResult<BigInteger>(false, involvedKeys,\n                    \"no connection to SQLite DB\", true, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n        return getInteger(connection.stmtGetPagesInCatCount, title.title,\n                timeAtStart, statName, involvedKeys);\n    }\n\n    /**\n     * Retrieves a number from the SQLite DB using a prepared statement with the\n     * given bindings and a single result column.\n     * \n     * @param stmt\n     *            the statement to evaluate\n     * @param bind1\n     *            string to bind to parameter 1 (if not <tt>null</tt>)\n     * @param timeAtStart\n     *            the start time of the method using this method\n     * @param statName\n     *            name for the time measurement statistics\n     * @param involvedKeys\n     *            list of all involved keys\n     * \n     * @return a result object with the number of pages on success\n     */\n    public static ValueResult<BigInteger> getInteger(\n            final SQLiteStatement stmt, String bind1, final long timeAtStart,\n            final String statName, List<InvolvedKey> involvedKeys) {\n        try {\n            if (bind1 != null) {\n                stmt.bind(1, bind1);\n            }\n            if (stmt.step()) {\n                BigInteger number = BigInteger.valueOf(stmt.columnLong(0));\n                return new ValueResult<BigInteger>(involvedKeys, number, statName,\n                        System.currentTimeMillis() - timeAtStart);\n            }\n            return new ValueResult<BigInteger>(\n                    false,\n                    involvedKeys,\n                    \"no result reading \" + statName,\n                    false, statName, System.currentTimeMillis() - timeAtStart);\n        } catch (SQLiteException e) {\n            return new ValueResult<BigInteger>(false, involvedKeys,\n                    e.getClass().getCanonicalName() + \" reading \" + statName + \": \"\n                            + e.getMessage(), false, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        } finally {\n            try {\n                stmt.reset();\n            } catch (SQLiteException e) {\n            }\n        }\n    }\n\n    /**\n     * Retrieves a list of pages in the given category from the SQLite DB.\n     * \n     * @param connection\n     *            the connection to the SQLite DB\n     * @param title\n     *            the title of the category\n     * \n     * @return a result object with the page list on success\n     */\n    public static ValueResult<List<NormalisedTitle>> getPagesInCategory(\n            Connection connection, NormalisedTitle title) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"CAT_LIST:\" + title;\n        final List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        if (connection == null) {\n            return new ValueResult<List<NormalisedTitle>>(false, involvedKeys,\n                    \"no connection to SQLite DB\", true, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n        return getPageList(connection.stmtGetPagesInCat, null, title.title,\n                timeAtStart, statName, involvedKeys);\n    }\n\n    /**\n     * Retrieves a list of pages using the given template from the SQLite DB.\n     * \n     * @param connection\n     *            the connection to the SQLite DB\n     * @param title\n     *            the title of the template\n     * \n     * @return a result object with the page list on success\n     */\n    public static ValueResult<List<NormalisedTitle>> getPagesInTemplate(\n            Connection connection, NormalisedTitle title) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"TPL_LIST:\" + title;\n        final List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        if (connection == null) {\n            return new ValueResult<List<NormalisedTitle>>(false, involvedKeys,\n                    \"no connection to SQLite DB\", true, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n        return getPageList(connection.stmtGetPagesInTpl, title.namespace,\n                title.title, timeAtStart, statName, involvedKeys);\n    }\n\n    /**\n     * Retrieves a list of pages linking to the given page from the SQLite DB.\n     * \n     * @param connection\n     *            the connection to the SQLite DB\n     * @param title\n     *            the title of the page\n     * \n     * @return a result object with the page list on success\n     */\n    public static ValueResult<List<NormalisedTitle>> getPagesLinkingTo(\n            Connection connection, NormalisedTitle title) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"LINKS:\" + title;\n        final List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        if (Options.getInstance().WIKI_USE_BACKLINKS) {\n            if (connection == null) {\n                return new ValueResult<List<NormalisedTitle>>(false, involvedKeys,\n                        \"no connection to SQLite DB\", true, statName,\n                        System.currentTimeMillis() - timeAtStart);\n            }\n            return getPageList(connection.stmtGetPagesLinkingTo,\n                    title.namespace, title.title, timeAtStart, statName,\n                    involvedKeys);\n        } else {\n            return new ValueResult<List<NormalisedTitle>>(involvedKeys,\n                    new ArrayList<NormalisedTitle>(0));\n        }\n    }\n\n    /**\n     * Retrieves a list of pages from the SQLite DB using a prepared statement\n     * with the given bindings and two result columns, the namespace and the\n     * title.\n     * \n     * @param stmt\n     *            the statement to evaluate\n     * @param namespace\n     *            bound to parameter 1 if not <tt>null</tt>\n     * @param title\n     *            bound to parameter 2 if <tt>namespace</tt> is not <tt>null</tt>,\n     *            otherwise to parameter 1\n     * @param timeAtStart\n     *            the start time of the method using this method\n     * @param statName\n     *            name for the time measurement statistics\n     * @param involvedKeys\n     *            list of all involved keys\n     * \n     * @return a result object with the number of pages on success\n     */\n    public static ValueResult<List<NormalisedTitle>> getPageList(\n            final SQLiteStatement stmt, Integer namespace, String title, final long timeAtStart,\n            final String statName, List<InvolvedKey> involvedKeys) {\n        // namespace != null -> title != null:\n        assert (namespace == null || title != null);\n        try {\n            ArrayList<NormalisedTitle> result = new ArrayList<NormalisedTitle>();\n            if (namespace != null) {\n                stmt.bind(1, namespace);\n                stmt.bind(2, title);\n            } else {\n                stmt.bind(1, title);\n            }\n            while (stmt.step()) {\n                result.add(new NormalisedTitle(stmt.columnInt(0), stmt.columnString(1)));\n            }\n            return new ValueResult<List<NormalisedTitle>>(involvedKeys, result, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        } catch (SQLiteException e) {\n            return new ValueResult<List<NormalisedTitle>>(false, involvedKeys,\n                    e.getClass().getCanonicalName() + \" reading \" + statName + \": \"\n                            + e.getMessage(), false, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        } finally {\n            try {\n                stmt.reset();\n            } catch (SQLiteException e) {\n            }\n        }\n    }\n\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/SavePageResult.java",
    "content": "package de.zib.scalaris.examples.wikipedia;\n\nimport java.math.BigInteger;\nimport java.util.ArrayList;\nimport java.util.List;\n\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.ShortRevision;\n\n/**\n * Result of an operation saving a page, i.e. adding a new revision.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class SavePageResult extends Result {\n    /**\n     * Old version of the page (may be null).\n     */\n    public Page oldPage = null;\n    /**\n     * New version of the page (may be null).\n     */\n    public Page newPage = null;\n    /**\n     * New list of (short) revisions (may be null).\n     */\n    public List<ShortRevision> newShortRevs = null;\n    /**\n     * New number of page edists (may be null).\n     */\n    public BigInteger pageEdits = null;\n    /**\n     * In cases of failed page-save commits, contains a list of failed keys.\n     */\n    public List<String> failedKeys = new ArrayList<String>();\n    \n    /**\n     * Creates a new successful result.\n     * \n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param oldPage\n     *            old version of the page (may be null)\n     * @param newPage\n     *            new version of the page (may be null)\n     * @param newShortRevs\n     *            new list of (short) revisions (may be null)\n     * @param pageEdits\n     *            new number of page edits (may be null)\n     */\n    public SavePageResult(List<InvolvedKey> involvedKeys, Page oldPage, Page newPage,\n            List<ShortRevision> newShortRevs, BigInteger pageEdits) {\n        super(involvedKeys);\n        this.oldPage = oldPage;\n        this.newPage = newPage;\n        this.newShortRevs = newShortRevs;\n        this.pageEdits = pageEdits;\n    }\n    \n    /**\n     * Creates a new successful result.\n     * \n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param oldPage\n     *            old version of the page (may be null)\n     * @param newPage\n     *            new version of the page (may be null)\n     * @param newShortRevs\n     *            new list of (short) revisions (may be null)\n     * @param pageEdits\n     *            new number of page edits (may be null)\n     * @param name\n     *            the name of the operation (for the stats - see {@link #stats})\n     * @param time\n     *            time in milliseconds for this operation\n     */\n    public SavePageResult(List<InvolvedKey> involvedKeys, Page oldPage, Page newPage,\n            List<ShortRevision> newShortRevs, BigInteger pageEdits,\n            String name, long time) {\n        super(involvedKeys);\n        this.oldPage = oldPage;\n        this.newPage = newPage;\n        this.newShortRevs = newShortRevs;\n        this.pageEdits = pageEdits;\n        addStat(name, time);\n    }\n    \n    /**\n     * Creates a new custom result.\n     * \n     * @param success\n     *            the success status\n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param message\n     *            the message to use\n     * @param connectFailed\n     *            whether the connection to the DB failed or not\n     * @param oldPage\n     *            old version of the page (may be null)\n     * @param newPage\n     *            new version of the page (may be null)\n     * @param newShortRevs\n     *            new list of (short) revisions (may be null)\n     * @param pageEdits\n     *            new number of page edits (may be null)\n     */\n    public SavePageResult(boolean success, List<InvolvedKey> involvedKeys,\n            String message, boolean connectFailed, Page oldPage, Page newPage,\n            List<ShortRevision> newShortRevs, BigInteger pageEdits) {\n        super(success, involvedKeys, message, connectFailed);\n        this.oldPage = oldPage;\n        this.newPage = newPage;\n        this.newShortRevs = newShortRevs;\n        this.pageEdits = pageEdits;\n    }\n    \n    /**\n     * Creates a new custom result.\n     * \n     * @param success\n     *            the success status\n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param message\n     *            the message to use\n     * @param connectFailed\n     *            whether the connection to the DB failed or not\n     * @param oldPage\n     *            old version of the page (may be null)\n     * @param newPage\n     *            new version of the page (may be null)\n     * @param newShortRevs\n     *            new list of (short) revisions (may be null)\n     * @param pageEdits\n     *            new number of page edits (may be null)\n     * @param name\n     *            the name of the operation (for the stats - see {@link #stats})\n     * @param time\n     *            time in milliseconds for this operation\n     */\n    public SavePageResult(boolean success, List<InvolvedKey> involvedKeys,\n            String message, boolean connectFailed, Page oldPage, Page newPage,\n            List<ShortRevision> newShortRevs, BigInteger pageEdits,\n            String name, long time) {\n        super(success, involvedKeys, message, connectFailed);\n        this.oldPage = oldPage;\n        this.newPage = newPage;\n        this.newShortRevs = newShortRevs;\n        this.pageEdits = pageEdits;\n        addStat(name, time);\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/ScalarisDataHandler.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia;\n\nimport java.math.BigInteger;\nimport java.util.ArrayList;\nimport java.util.Arrays;\nimport java.util.Collection;\nimport java.util.List;\nimport java.util.Random;\n\nimport com.ericsson.otp.erlang.OtpErlangString;\n\nimport de.zib.scalaris.Connection;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.ErlangValue.ListElementConverter;\nimport de.zib.scalaris.ScalarisVM;\nimport de.zib.scalaris.TransactionSingleOp;\nimport de.zib.scalaris.UnknownException;\nimport de.zib.scalaris.examples.wikipedia.Options.Optimisation;\nimport de.zib.scalaris.examples.wikipedia.Options.STORE_CONTRIB_TYPE;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyNamespace.NamespaceEnum;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.data.Contribution;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\nimport de.zib.scalaris.operations.Operation;\nimport de.zib.scalaris.operations.PartialReadOp;\nimport de.zib.scalaris.operations.ReadOp;\n\n/**\n * Retrieves and writes values from/to Scalaris.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class ScalarisDataHandler {\n    \n    /**\n     * Gets the key to store {@link SiteInfo} objects at.\n     * \n     * @return Scalaris key\n     */\n    public final static String getSiteInfoKey() {\n        return \"siteinfo\";\n    }\n    \n    /**\n     * Gets the key to store the list of pages in the given namespace at.\n     * \n     * @param namespace  the namespace ID\n     * \n     * @return Scalaris key\n     */\n    public final static String getPageListKey(int namespace) {\n        return \"pages:\" + namespace;\n    }\n    \n    /**\n     * Gets the key to store the number of pages at.\n     * \n     * @param namespace  the namespace ID\n     * \n     * @return Scalaris key\n     */\n    public final static String getPageCountKey(int namespace) {\n        return getPageListKey(namespace) + \":count\";\n    }\n    \n    /**\n     * Gets the key to store the number of articles, i.e. pages in the main\n     * namespace, at.\n     * \n     * @return Scalaris key\n     */\n    public final static String getArticleCountKey() {\n        return \"articles:count\";\n    }\n    \n    /**\n     * Gets the key to store the number of page edits.\n     * \n     * @return Scalaris key\n     */\n    public final static String getStatsPageEditsKey() {\n        return \"stats:pageedits\";\n    }\n    \n    /**\n     * Gets the key to store the list of contributions of a user.\n     * \n     * @param contributor  the user name or IP address of the user who created\n     *                     the revision\n     * \n     * @return Scalaris key\n     */\n    public final static String getContributionListKey(String contributor) {\n        return contributor + \":user:contrib\";\n    }\n\n    /**\n     * Retrieves the Scalaris version string.\n     * \n     * @param connection\n     *            the connection to the DB\n     * \n     * @return a result object with the version string on success\n     */\n    public final static ValueResult<String> getDbVersion(Connection connection) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"SCALARIS_VERSION\";\n        List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        if (connection == null) {\n            return new ValueResult<String>(false, involvedKeys,\n                    \"no connection to Scalaris\", true, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n\n        String node = connection.getRemote().toString();\n        try {\n            ScalarisVM scalarisVm = new ScalarisVM(node);\n            String version = scalarisVm.getVersion();\n            return new ValueResult<String>(involvedKeys, version, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        } catch (ConnectionException e) {\n            return new ValueResult<String>(false, involvedKeys,\n                    \"no connection to Scalaris node \\\"\" + node + \"\\\"\", true,\n                    statName, System.currentTimeMillis() - timeAtStart);\n        } catch (UnknownException e) {\n            return new ValueResult<String>(false, involvedKeys,\n                    e.getClass().getCanonicalName() + \" reading Scalaris version from node \\\"\"\n                            + node + \"\\\"\", true, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n    }\n\n    /**\n     * Retrieves a list of all available pages from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * \n     * @return a result object with the page list on success\n     */\n    public final static ValueResult<List<NormalisedTitle>> getPageList(Connection connection) {\n        final long timeAtStart = System.currentTimeMillis();\n        ArrayList<String> scalaris_keys = new ArrayList<String>(NamespaceEnum.values().length);\n        for (NamespaceEnum ns : NamespaceEnum.values()) {\n            scalaris_keys.add(getPageListKey(ns.getId()));\n        }\n        return getPageList2(connection, ScalarisOpType.PAGE_LIST,\n                scalaris_keys, false, timeAtStart, \"page list\");\n    }\n\n    /**\n     * Retrieves a list of available pages in the given namespace from Scalaris.\n     * \n     * @param namespace\n     *            the namespace ID\n     * @param connection\n     *            the connection to Scalaris\n     * \n     * @return a result object with the page list on success\n     */\n    public final static ValueResult<List<NormalisedTitle>> getPageList(int namespace, Connection connection) {\n        final long timeAtStart = System.currentTimeMillis();\n        return getPageList2(connection, ScalarisOpType.PAGE_LIST,\n                Arrays.asList(getPageListKey(namespace)), false, timeAtStart,\n                \"page list:\" + namespace);\n    }\n\n    /**\n     * Retrieves a list of pages linking to the given page from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param contributor\n     *            the user name or IP address of the user who created the\n     *            revision\n     * \n     * @return a result object with the page list on success\n     */\n    public final static ValueResult<List<Contribution>> getContributions(\n            Connection connection, String contributor) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"CONTRIB:\" + contributor;\n        if (Options.getInstance().WIKI_STORE_CONTRIBUTIONS != STORE_CONTRIB_TYPE.NONE) {\n            ValueResult<List<Contribution>> result = getPageList3(connection,\n                    ScalarisOpType.CONTRIBUTION,\n                    Arrays.asList(getContributionListKey(contributor)), false,\n                    timeAtStart, statName,\n                    new ErlangConverter<List<Contribution>>() {\n                        @Override\n                        public List<Contribution> convert(ErlangValue v)\n                                throws ClassCastException {\n                            return v.jsonListValue(Contribution.class);\n                        }\n                    },\n                    new ErlangConverter<Contribution>() {\n                        @Override\n                        public Contribution convert(ErlangValue v)\n                                throws ClassCastException {\n                            return v.jsonValue(Contribution.class);\n                        }\n                    });\n            if (result.success && result.value == null) {\n                result.value = new ArrayList<Contribution>(0);\n            }\n            return result;\n        } else {\n            return new ValueResult<List<Contribution>>(\n                    new ArrayList<InvolvedKey>(0), new ArrayList<Contribution>(0));\n        }\n    }\n\n    /**\n     * Retrieves a list of pages from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param opType\n     *            operation type indicating what is being read\n     * @param scalaris_keys\n     *            the keys under which the page list is stored in Scalaris\n     * @param failNotFound\n     *            whether the operation should fail if the key is not found or\n     *            not\n     * @param timeAtStart\n     *            the start time of the method using this method\n     * @param statName\n     *            name for the time measurement statistics\n     * \n     * @return a result object with the page list on success\n     */\n    protected final static ValueResult<List<NormalisedTitle>> getPageList2(\n            Connection connection, ScalarisOpType opType,\n            Collection<String> scalaris_keys, boolean failNotFound,\n            final long timeAtStart, String statName) {\n        ValueResult<List<NormalisedTitle>> result = getPageList3(connection,\n                opType, scalaris_keys, failNotFound, timeAtStart, statName,\n                new ErlangConverter<List<NormalisedTitle>>() {\n                    @Override\n                    public List<NormalisedTitle> convert(ErlangValue v)\n                            throws ClassCastException {\n                        return v.listValue(new ListElementConverter<NormalisedTitle>() {\n                            public NormalisedTitle convert(final int i,\n                                    final ErlangValue v) {\n                                return NormalisedTitle.fromNormalised(v\n                                        .stringValue());\n                            }\n                        });\n                    }\n                },\n                new ErlangConverter<NormalisedTitle>() {\n                    @Override\n                    public NormalisedTitle convert(ErlangValue v)\n                            throws ClassCastException {\n                        return NormalisedTitle.fromNormalised(v\n                                        .stringValue());\n                    }\n                });\n        if (result.success && result.value == null) {\n            result.value = new ArrayList<NormalisedTitle>(0);\n        }\n        return result;\n    }\n\n    /**\n     * Retrieves a list of pages from Scalaris.\n     * \n     * @param <T>\n     *            list type\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param opType\n     *            operation type indicating what is being read\n     * @param scalaris_keys\n     *            the keys under which the page list is stored in Scalaris\n     * @param failNotFound\n     *            whether the operation should fail if no key is found (in which\n     *            case the value is <tt>null</tt>) or not\n     * @param timeAtStart\n     *            the start time of the method using this method\n     * @param statName\n     *            name for the time measurement statistics\n     * @param listConv\n     *            converter to make an {@link ErlangValue} to a {@link List} of\n     *            <tt>T</tt>\n     * @param elemConv\n     *            converter to make an {@link ErlangValue} to a <tt>T</tt>\n     * \n     * @return a result object with the page list on success\n     */\n    protected final static <T> ValueResult<List<T>> getPageList3(\n            Connection connection, ScalarisOpType opType,\n            Collection<String> scalaris_keys, boolean failNotFound,\n            final long timeAtStart, String statName,\n            ErlangConverter<List<T>> listConv, ErlangConverter<T> elemConv) {\n        List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        \n        if (connection == null) {\n            return new ValueResult<List<T>>(false, involvedKeys,\n                    \"no connection to Scalaris\", true, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n        \n        final MyScalarisSingleOpExecutor executor = new MyScalarisSingleOpExecutor(\n                new TransactionSingleOp(connection), involvedKeys);\n\n        final ScalarisReadListOp1<T> readOp = new ScalarisReadListOp1<T>(\n                scalaris_keys, Options.getInstance().OPTIMISATIONS.get(opType),\n                listConv, elemConv, failNotFound);\n        executor.addOp(readOp);\n        try {\n            executor.run();\n        } catch (Exception e) {\n            return new ValueResult<List<T>>(false, involvedKeys,\n                    e.getClass().getCanonicalName() + \" reading page list at \\\"\"\n                            + involvedKeys.toString() + \"\\\" from Scalaris: \"\n                            + e.getMessage(), e instanceof ConnectionException,\n                    statName, System.currentTimeMillis() - timeAtStart);\n        }\n        \n        return new ValueResult<List<T>>(involvedKeys, readOp.getValue(), statName,\n                System.currentTimeMillis() - timeAtStart);\n    }\n\n    /**\n     * Retrieves the number of all available pages from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * \n     * @return a result object with the number of pages on success\n     */\n    public final static ValueResult<BigInteger> getPageCount(Connection connection) {\n        final long timeAtStart = System.currentTimeMillis();\n        ArrayList<String> scalaris_keys = new ArrayList<String>(NamespaceEnum.values().length);\n        for (NamespaceEnum ns : NamespaceEnum.values()) {\n            scalaris_keys.add(getPageCountKey(ns.getId()));\n        }\n        final String statName = \"PAGE_COUNT\";\n        return getInteger2(connection, ScalarisOpType.PAGE_COUNT, scalaris_keys,\n                false, timeAtStart, statName);\n    }\n\n    /**\n     * Retrieves the number of available pages in the given namespace from\n     * Scalaris.\n     * \n     * @param namespace\n     *            the namespace ID\n     * @param connection\n     *            the connection to Scalaris\n     * \n     * @return a result object with the number of pages on success\n     */\n    public final static ValueResult<BigInteger> getPageCount(int namespace, Connection connection) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"PAGE_COUNT:\" + namespace;\n        return getInteger2(connection, ScalarisOpType.PAGE_COUNT,\n                getPageCountKey(namespace), false, timeAtStart, statName);\n    }\n\n    /**\n     * Retrieves the number of available articles, i.e. pages in the main\n     * namespace, from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * \n     * @return a result object with the number of articles on success\n     */\n    public final static ValueResult<BigInteger> getArticleCount(Connection connection) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"ARTICLE_COUNT\";\n        return getInteger2(connection, ScalarisOpType.ARTICLE_COUNT,\n                getArticleCountKey(), false, timeAtStart, statName);\n    }\n\n    /**\n     * Retrieves the number of available articles, i.e. pages in the main\n     * namespace, from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * \n     * @return a result object with the number of articles on success\n     */\n    public final static ValueResult<BigInteger> getStatsPageEdits(Connection connection) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"PAGE_EDITS\";\n        return getInteger2(connection, ScalarisOpType.EDIT_STAT,\n                getStatsPageEditsKey(), false, timeAtStart, statName);\n    }\n\n    /**\n     * Retrieves a random page title from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param random\n     *            the random number generator to use\n     * \n     * @return a result object with the page list on success\n     */\n    public final static ValueResult<NormalisedTitle> getRandomArticle(Connection connection, Random random) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"RANDOM_PAGE\";\n        \n        final Optimisation optimisation = Options.getInstance().OPTIMISATIONS.get(ScalarisOpType.PAGE_LIST);\n        final ErlangConverter<List<ErlangValue>> listConv = new ErlangConverter<List<ErlangValue>>() {\n            @Override\n            public List<ErlangValue> convert(ErlangValue v)\n                    throws ClassCastException {\n                return v.listValue();\n            }\n        };\n        final ErlangConverter<ErlangValue> elemConv = new ErlangConverter<ErlangValue>() {\n            @Override\n            public ErlangValue convert(ErlangValue v) {\n                return v;\n            }\n        };\n        final List<String> scalarisKeys = Arrays.asList(getPageListKey(NamespaceEnum.MAIN_NAMESPACE_KEY.getId()));\n        List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        \n        if (connection == null) {\n            return new ValueResult<NormalisedTitle>(false, involvedKeys,\n                    \"no connection to Scalaris\", true, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n\n        // first try reading from one bucket only, if it fails, try to read from all\n        for (boolean readOnlyOneBucket : new boolean[] {true, false}) {\n            final MyScalarisSingleOpExecutor executor = new MyScalarisSingleOpExecutor(\n                    new TransactionSingleOp(connection), involvedKeys);\n\n            final ScalarisReadRandomListEntryOp1<ErlangValue> readOp = new ScalarisReadRandomListEntryOp1<ErlangValue>(\n                    scalarisKeys, optimisation, readOnlyOneBucket, elemConv,\n                    listConv, false, random);\n            executor.addOp(readOp);\n            try {\n                executor.run();\n            } catch (Exception e) {\n                return new ValueResult<NormalisedTitle>(false, involvedKeys,\n                        e.getClass().getCanonicalName() + \" reading page list at \\\"\"\n                                + involvedKeys.toString() + \"\\\" from Scalaris: \"\n                                + e.getMessage(), e instanceof ConnectionException,\n                        statName, System.currentTimeMillis() - timeAtStart);\n            }\n            \n            // return if successful, otherwise fall back and read the whole list\n            // as with no optimisation\n            if (readOp.getValue() != null) {\n                return new ValueResult<NormalisedTitle>(involvedKeys,\n                        NormalisedTitle.fromNormalised(readOp.getValue()\n                                .stringValue()), statName,\n                        System.currentTimeMillis() - timeAtStart);\n            }\n        }\n        return new ValueResult<NormalisedTitle>(false, involvedKeys,\n                \"unable to retrieve random page (no articles?)\", false,\n                statName, System.currentTimeMillis() - timeAtStart);\n    }\n\n    /**\n     * Retrieves an integral number from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param scalaris_key\n     *            the key under which the number is stored in Scalaris\n     * @param failNotFound\n     *            whether the operation should fail if the key is not found or\n     *            not\n     * @param timeAtStart\n     *            the start time of the method using this method\n     * @param statName\n     *            name for the time measurement statistics\n     * \n     * @return a result object with the number on success\n     */\n    protected final static ValueResult<BigInteger> getInteger2(\n            Connection connection, ScalarisOpType opType, String scalaris_key,\n            boolean failNotFound, final long timeAtStart, String statName) {\n        return getInteger2(connection, opType, Arrays.asList(scalaris_key),\n                failNotFound, timeAtStart, statName);\n    }\n\n    /**\n     * Retrieves an integral number from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param opType\n     *            operation type indicating what is being read\n     * @param scalaris_keys\n     *            the keys under which the number is stored in Scalaris\n     * @param failNotFound\n     *            whether the operation should fail if the key is not found or\n     *            not\n     * @param timeAtStart\n     *            the start time of the method using this method\n     * @param statName\n     *            name for the time measurement statistics\n     * \n     * @return a result object with the number on success\n     */\n    protected final static ValueResult<BigInteger> getInteger2(\n            Connection connection, ScalarisOpType opType,\n            Collection<String> scalaris_keys, boolean failNotFound,\n            final long timeAtStart, String statName) {\n        List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        if (connection == null) {\n            return new ValueResult<BigInteger>(false, involvedKeys,\n                    \"no connection to Scalaris\", true, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n        \n        final MyScalarisSingleOpExecutor executor = new MyScalarisSingleOpExecutor(\n                new TransactionSingleOp(connection), involvedKeys);\n\n        Optimisation optimisation = Options.getInstance().OPTIMISATIONS.get(opType);\n        if (optimisation == null) {\n            switch (opType) {\n                case PAGE_COUNT:\n                    // fall back to PAGE_LIST optimisation as in this case, the counter\n                    // follows the partitions\n                    optimisation = Options.getInstance().OPTIMISATIONS.get(ScalarisOpType.PAGE_LIST);\n                    break;\n                case CATEGORY_PAGE_COUNT:\n                    // fall back to CATEGORY_PAGE_LIST optimisation as in this case, the counter\n                    // follows the partitions\n                    optimisation = Options.getInstance().OPTIMISATIONS.get(ScalarisOpType.CATEGORY_PAGE_LIST);\n                    break;\n                default:\n                    break;\n            }\n        }\n        final ScalarisReadNumberOp1 readOp = new ScalarisReadNumberOp1(scalaris_keys,\n                optimisation, failNotFound);\n        executor.addOp(readOp);\n        try {\n            executor.run();\n        } catch (Exception e) {\n            return new ValueResult<BigInteger>(false, involvedKeys,\n                    e.getClass().getCanonicalName() + \" reading page list at \\\"\"\n                            + involvedKeys.toString() + \"\\\" from Scalaris: \"\n                            + e.getMessage(), e instanceof ConnectionException,\n                    statName, System.currentTimeMillis() - timeAtStart);\n        }\n        \n        return new ValueResult<BigInteger>(involvedKeys, readOp.getValue(), statName,\n                System.currentTimeMillis() - timeAtStart);\n    }\n\n    /**\n     * Adds all keys from the given operation list to the list of involved keys.\n     * \n     * @param involvedKeys\n     *            list of involved keys\n     * @param ops\n     *            new operations\n     */\n    public static void addInvolvedKeys(List<InvolvedKey> involvedKeys, Collection<? extends Operation> ops) {\n        assert involvedKeys != null;\n        assert ops != null;\n        for (Operation op : ops) {\n            final OtpErlangString key = op.getKey();\n            if (key != null) {\n                if (op instanceof ReadOp) {\n                    involvedKeys.add(new InvolvedKey(InvolvedKey.OP.READ, key.stringValue()));\n                } else if (op instanceof PartialReadOp) {\n                    involvedKeys.add(new InvolvedKey(InvolvedKey.OP.READ, key.stringValue()));\n                } else {\n                    involvedKeys.add(new InvolvedKey(InvolvedKey.OP.WRITE, key.stringValue()));\n                }\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/ScalarisDataHandlerNormalised.java",
    "content": "/**\n *  Copyright 2012-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia;\n\nimport java.math.BigInteger;\nimport java.util.ArrayList;\nimport java.util.Arrays;\nimport java.util.Collection;\nimport java.util.HashMap;\nimport java.util.List;\n\nimport de.zib.scalaris.Connection;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.Transaction;\nimport de.zib.scalaris.TransactionSingleOp;\nimport de.zib.scalaris.examples.wikipedia.InvolvedKey.OP;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\nimport de.zib.scalaris.examples.wikipedia.data.ShortRevision;\nimport de.zib.scalaris.executor.ScalarisOp;\nimport de.zib.scalaris.executor.ScalarisReadOp;\nimport de.zib.scalaris.operations.ReadOp;\n\n/**\n * @author Nico Kruber, kruber@zib.de\n */\npublic class ScalarisDataHandlerNormalised extends ScalarisDataHandler {\n    \n    /**\n     * Gets the key to store {@link Revision} objects at.\n     * \n     * @param title     the title of the page\n     * @param id        the id of the revision\n     * \n     * @return Scalaris key\n     */\n    public final static String getRevKey(NormalisedTitle title, int id) {\n        return title + \":rev:\" + id;\n    }\n    \n    /**\n     * Gets the key to store {@link Page} objects at.\n     * \n     * @param title     the title of the page\n     * \n     * @return Scalaris key\n     */\n    public final static String getPageKey(NormalisedTitle title) {\n        return title + \":page\";\n    }\n    \n    /**\n     * Gets the key to store the list of revisions of a page at.\n     * \n     * @param title     the title of the page\n     * \n     * @return Scalaris key\n     */\n    public final static String getRevListKey(NormalisedTitle title) {\n        return title + \":revs\";\n    }\n    \n    /**\n     * Gets the key to store the list of pages belonging to a category at.\n     * \n     * @param title     the category title (including <tt>Category:</tt>)\n     * \n     * @return Scalaris key\n     */\n    public final static String getCatPageListKey(NormalisedTitle title) {\n        return title + \":cpages\";\n    }\n    \n    /**\n     * Gets the key to store the number of pages belonging to a category at.\n     * \n     * @param title     the category title (including <tt>Category:</tt>)\n     * \n     * @return Scalaris key\n     */\n    public final static String getCatPageCountKey(NormalisedTitle title) {\n        return title + \":cpages:count\";\n    }\n    \n    /**\n     * Gets the key to store the list of pages using a template at.\n     * \n     * @param title     the template title (including <tt>Template:</tt>)\n     * \n     * @return Scalaris key\n     */\n    public final static String getTplPageListKey(NormalisedTitle title) {\n        return title + \":tpages\";\n    }\n    \n    /**\n     * Gets the key to store the list of pages linking to the given title.\n     * \n     * @param title     the page's title\n     * \n     * @return Scalaris key\n     */\n    public final static String getBackLinksPageListKey(NormalisedTitle title) {\n        return title + \":blpages\";\n    }\n    \n    /**\n     * Retrieves a page's history from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the page\n     * \n     * @return a result object with the page history on success\n     */\n    public static PageHistoryResult getPageHistory(Connection connection,\n            NormalisedTitle title) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"HISTORY:\" + title;\n        List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        if (connection == null) {\n            return new PageHistoryResult(false, involvedKeys, \"no connection to Scalaris\",\n                    true, statName, System.currentTimeMillis() - timeAtStart);\n        }\n        \n        TransactionSingleOp scalaris_single = new TransactionSingleOp(connection);\n        TransactionSingleOp.RequestList requests = new TransactionSingleOp.RequestList();\n        requests.addOp(new ReadOp(getPageKey(title)));\n        requests.addOp(new ReadOp(getRevListKey(title)));\n        \n        TransactionSingleOp.ResultList results;\n        try {\n            addInvolvedKeys(involvedKeys, requests.getRequests());\n            results = scalaris_single.req_list(requests);\n        } catch (Exception e) {\n            return new PageHistoryResult(false, involvedKeys,\n                    e.getClass().getCanonicalName() + \" reading \\\"\" + getPageKey(title)\n                            + \"\\\" or \\\"\" + getRevListKey(title)\n                            + \"\\\" from Scalaris: \" + e.getMessage(),\n                    e instanceof ConnectionException, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n        \n        Page page;\n        try {\n            page = results.processReadAt(0).jsonValue(Page.class);\n        } catch (NotFoundException e) {\n            PageHistoryResult result = new PageHistoryResult(false,\n                    involvedKeys, \"page not found at \\\"\" + getPageKey(title)\n                            + \"\\\"\", false, statName, System.currentTimeMillis()\n                            - timeAtStart);\n            result.not_existing = true;\n            return result;\n        } catch (Exception e) {\n            return new PageHistoryResult(false, involvedKeys,\n                    e.getClass().getCanonicalName() + \" reading \\\"\" + getPageKey(title)\n                            + \"\\\" from Scalaris: \" + e.getMessage(),\n                    e instanceof ConnectionException, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n\n        List<ShortRevision> revisions;\n        try {\n            revisions = results.processReadAt(1).jsonListValue(ShortRevision.class);\n        } catch (NotFoundException e) {\n            PageHistoryResult result = new PageHistoryResult(false,\n                    involvedKeys, \"revision list \\\"\" + getRevListKey(title)\n                            + \"\\\" does not exist\", false, statName,\n                    System.currentTimeMillis() - timeAtStart);\n            result.not_existing = true;\n            return result;\n        } catch (Exception e) {\n            return new PageHistoryResult(false, involvedKeys,\n                    e.getClass().getCanonicalName() + \" reading \\\"\" + getRevListKey(title)\n                            + \"\\\" from Scalaris: \" + e.getMessage(),\n                    e instanceof ConnectionException, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n        return new PageHistoryResult(involvedKeys, page, revisions, statName,\n                System.currentTimeMillis() - timeAtStart);\n    }\n\n    /**\n     * Retrieves the current, i.e. most up-to-date, version of a page from\n     * Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the page\n     * \n     * @return a result object with the page and revision on success\n     */\n    public static RevisionResult getRevision(Connection connection,\n            NormalisedTitle title) {\n        return getRevision(connection, title, -1);\n    }\n\n    /**\n     * Retrieves the given version of a page from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the page\n     * @param id\n     *            the id of the version\n     * \n     * @return a result object with the page and revision on success\n     */\n    public static RevisionResult getRevision(Connection connection,\n            NormalisedTitle title, int id) {\n        final long timeAtStart = System.currentTimeMillis();\n        Page page = null;\n        Revision revision = null;\n        List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        final String statName = \"PAGE:\" + title.toString();\n        if (connection == null) {\n            return new RevisionResult(false, involvedKeys,\n                    \"no connection to Scalaris\", true, title, page, revision, false,\n                    false, statName, System.currentTimeMillis() - timeAtStart);\n        }\n        \n        TransactionSingleOp scalaris_single;\n        String scalaris_key;\n        \n        scalaris_single = new TransactionSingleOp(connection);\n\n        scalaris_key = getPageKey(title);\n        try {\n            involvedKeys.add(new InvolvedKey(OP.READ, scalaris_key));\n            page = scalaris_single.read(scalaris_key).jsonValue(Page.class);\n        } catch (NotFoundException e) {\n            return new RevisionResult(false, involvedKeys,\n                    \"page not found at \\\"\" + scalaris_key + \"\\\"\", false, title,\n                    page, revision, true, false, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        } catch (Exception e) {\n            return new RevisionResult(false, involvedKeys,\n                    e.getClass().getCanonicalName() + \" reading \\\"\" + scalaris_key\n                            + \"\\\" from Scalaris: \" + e.getMessage(),\n                    e instanceof ConnectionException, title, page, revision, false,\n                    false, statName, System.currentTimeMillis() - timeAtStart);\n        }\n\n        // load requested version if it is not the current one cached in the Page object\n        if (id != page.getCurRev().getId() && id >= 0) {\n            scalaris_key = getRevKey(title, id);\n            try {\n                involvedKeys.add(new InvolvedKey(OP.READ, scalaris_key));\n                revision = scalaris_single.read(scalaris_key).jsonValue(Revision.class);\n            } catch (NotFoundException e) {\n                return new RevisionResult(false, involvedKeys,\n                        \"revision not found at \\\"\" + scalaris_key + \"\\\"\",\n                        false, title, page, revision, false, true, statName,\n                        System.currentTimeMillis() - timeAtStart);\n            } catch (Exception e) {\n                return new RevisionResult(false, involvedKeys,\n                        e.getClass().getCanonicalName() + \" reading \\\"\" + scalaris_key\n                                + \"\\\" from Scalaris: \" + e.getMessage(),\n                        e instanceof ConnectionException, title, page,\n                        revision, false, false, statName,\n                        System.currentTimeMillis() - timeAtStart);\n            }\n        } else {\n            revision = page.getCurRev();\n        }\n\n        return new RevisionResult(involvedKeys, title, page, revision, statName,\n                System.currentTimeMillis() - timeAtStart);\n    }\n\n    /**\n     * Retrieves the current version of all given pages from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param titles\n     *            the titles of the pages\n     * @param statName\n     *            name of the statistic to collect\n     * \n     * @return a result object with the pages and revisions on success\n     */\n    public static ValueResult<List<RevisionResult>> getRevisions(Connection connection,\n            Collection<NormalisedTitle> titles, final String statName) {\n        final long timeAtStart = System.currentTimeMillis();\n        List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        if (connection == null) {\n            return new ValueResult<List<RevisionResult>>(false, involvedKeys,\n                    \"no connection to Scalaris\", true, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n        \n        TransactionSingleOp scalaris_single = new TransactionSingleOp(connection);\n        final MyScalarisSingleOpExecutor executor0 = new MyScalarisSingleOpExecutor(\n                scalaris_single, involvedKeys);\n        HashMap<ScalarisReadOp, NormalisedTitle> opToTitleN = new HashMap<ScalarisReadOp, NormalisedTitle>(titles.size());\n        for (NormalisedTitle title : titles) {\n            ScalarisReadOp readOp = new ScalarisReadOp(getPageKey(title));\n            executor0.addOp(readOp);\n            opToTitleN.put(readOp, title);\n        }\n        try {\n            executor0.run();\n        } catch (Exception e) {\n            return new ValueResult<List<RevisionResult>>(false, involvedKeys,\n                    e.getClass().getCanonicalName() + \" reading \" + titles + \" from Scalaris\",\n                    e instanceof ConnectionException, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n        \n        List<RevisionResult> results = new ArrayList<RevisionResult>(titles.size());\n        for (ScalarisOp op : executor0.getOps()) {\n            if (op instanceof ScalarisReadOp) {\n                ScalarisReadOp readOp = (ScalarisReadOp) op;\n                RevisionResult curResult;\n                if (readOp.getValue() != null) {\n                    Page page = readOp.getValue().jsonValue(Page.class);\n                    curResult = new RevisionResult(involvedKeys,\n                            opToTitleN.get(readOp), page, page.getCurRev());\n                    \n                } else {\n                    curResult = new RevisionResult(false, involvedKeys,\n                            \"page not found at \\\"\" + readOp.getKey() + \"\\\"\",\n                            false, opToTitleN.get(readOp), null, null, true, false);\n                }\n                results.add(curResult);\n            }\n        }\n        return new ValueResult<List<RevisionResult>>(involvedKeys, results,\n                statName, System.currentTimeMillis() - timeAtStart);\n    }\n\n    /**\n     * Retrieves a list of pages in the given category from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the category\n     * \n     * @return a result object with the page list on success\n     */\n    public static ValueResult<List<NormalisedTitle>> getPagesInCategory(Connection connection,\n            NormalisedTitle title) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"CAT_LIST:\" + title;\n        return getPageList2(connection, ScalarisOpType.CATEGORY_PAGE_LIST,\n                Arrays.asList(getCatPageListKey(title)), false,\n                timeAtStart, statName);\n    }\n\n    /**\n     * Retrieves a list of pages using the given template from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the template\n     * \n     * @return a result object with the page list on success\n     */\n    public static ValueResult<List<NormalisedTitle>> getPagesInTemplate(Connection connection,\n            NormalisedTitle title) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"TPL_LIST:\" + title;\n        return getPageList2(connection, ScalarisOpType.TEMPLATE_PAGE_LIST,\n                Arrays.asList(getTplPageListKey(title)), false,\n                timeAtStart, statName);\n    }\n\n    /**\n     * Retrieves a list of pages using the given templates from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param titles\n     *            the titles of the templates\n     * @param pageTitle\n     *            the title of the page to retrieve the list for (will be\n     *            included in the statname)\n     * \n     * @return a result object with the page list on success\n     */\n    public static ValueResult<List<NormalisedTitle>> getPagesInTemplates(Connection connection,\n            List<NormalisedTitle> titles, String pageTitle) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"TPL_LISTS_FOR:\" + pageTitle;\n        ArrayList<String> scalarisKeys = new ArrayList<String>(titles.size());\n        for (NormalisedTitle title : titles) {\n            scalarisKeys.add(getTplPageListKey(title));\n        }\n        return getPageList2(connection, ScalarisOpType.TEMPLATE_PAGE_LIST,\n                scalarisKeys, false, timeAtStart, statName);\n    }\n\n    /**\n     * Retrieves a list of pages linking to the given page from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the page\n     * \n     * @return a result object with the page list on success\n     */\n    public static ValueResult<List<NormalisedTitle>> getPagesLinkingTo(Connection connection,\n            NormalisedTitle title) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"LINKS:\" + title;\n        if (Options.getInstance().WIKI_USE_BACKLINKS) {\n            return getPageList2(connection, ScalarisOpType.BACKLINK_PAGE_LIST,\n                    Arrays.asList(getBackLinksPageListKey(title)),\n                    false, timeAtStart, statName);\n        } else {\n            return new ValueResult<List<NormalisedTitle>>(new ArrayList<InvolvedKey>(0),\n                    new ArrayList<NormalisedTitle>(0));\n        }\n    }\n\n    /**\n     * Retrieves the number of pages in the given category from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the category\n     * \n     * @return a result object with the number of pages on success\n     */\n    public static ValueResult<BigInteger> getPagesInCategoryCount(\n            Connection connection, NormalisedTitle title) {\n        final long timeAtStart = System.currentTimeMillis();\n        final String statName = \"CAT_CNT:\" + title;\n        return getInteger2(connection, ScalarisOpType.CATEGORY_PAGE_COUNT,\n                getCatPageCountKey(title), false, timeAtStart,\n                statName);\n    }\n    \n    /**\n     * Updates a list of pages by removing and/or adding new page titles.\n     * \n     * @param scalaris_tx\n     *            connection to Scalaris\n     * @param opType\n     *            operation type indicating what is being updated\n     * @param pageList_key\n     *            Scalaris key for the page list\n     * @param countOpType\n     *            operation type indicating what is being updated for\n     *            updating the count key (if there is no count key, this may\n     *            be <tt>null</tt>)\n     * @param pageCount_key\n     *            Scalaris key for the number of pages in the list (may be null\n     *            if not used)\n     * @param entriesToAdd\n     *            list of (normalised) page names to add to the list\n     * @param entriesToRemove\n     *            list of (normalised) page names to remove from the list\n     * @param statName\n     *            name for the time measurement statistics\n     * \n     * @return the result of the operation\n     */\n    public static ValueResult<Integer> updatePageList(Transaction scalaris_tx,\n            ScalarisOpType opType, String pageList_key,\n            ScalarisOpType countOpType, String pageCount_key,\n            List<NormalisedTitle> entriesToAdd, List<NormalisedTitle> entriesToRemove,\n            final String statName) {\n        final long timeAtStart = System.currentTimeMillis();\n        List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n\n        try {\n            final MyScalarisTxOpExecutor executor0 = new MyScalarisTxOpExecutor(\n                    scalaris_tx, involvedKeys);\n            executor0.setCommitLast(true);\n            MyScalarisOpExecWrapper executor = new MyScalarisOpExecWrapper(\n                    executor0);\n\n            executor.addAppendRemove(opType, pageList_key,\n                    normList2normStringList(entriesToAdd),\n                    normList2normStringList(entriesToRemove),\n                    countOpType, pageCount_key);\n            \n            executor.getExecutor().run();\n            return new ValueResult<Integer>(involvedKeys, null, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        } catch (Exception e) {\n            return new ValueResult<Integer>(false, involvedKeys,\n                    e.getClass().getCanonicalName() + \" updating \\\"\" + pageList_key\n                            + \"\\\" and \\\"\" + pageCount_key + \"\\\" in Scalaris: \"\n                            + e.getMessage(), e instanceof ConnectionException,\n                    statName, System.currentTimeMillis() - timeAtStart);\n        }\n    }\n\n    /**\n     * Converts a list of {@link NormalisedTitle} objects to a list of\n     * normalised page title strings.\n     * \n     * @param list\n     *            the {@link NormalisedTitle} list\n     * \n     * @return a string list\n     */\n    public static List<String> normList2normStringList(\n            Collection<? extends NormalisedTitle> list) {\n        ArrayList<String> entriesToAddStr = new ArrayList<String>(list.size());\n        for (NormalisedTitle nt : list) {\n            entriesToAddStr.add(nt.toString());\n        }\n        return entriesToAddStr;\n    }\n\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/ScalarisDataHandlerUnnormalised.java",
    "content": "/**\n *  Copyright 2012-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia;\n\nimport java.math.BigInteger;\nimport java.util.ArrayList;\nimport java.util.Arrays;\nimport java.util.Collection;\nimport java.util.HashSet;\nimport java.util.LinkedHashMap;\nimport java.util.List;\nimport java.util.Map;\nimport java.util.Set;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.Connection;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.Transaction;\nimport de.zib.scalaris.examples.wikipedia.Options.STORE_CONTRIB_TYPE;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyNamespace;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyWikiModel;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.data.Contribution;\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\nimport de.zib.scalaris.examples.wikipedia.data.ShortRevision;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\nimport de.zib.scalaris.operations.ReadOp;\n\n/**\n * @author Nico Kruber, kruber@zib.de\n *\n */\npublic class ScalarisDataHandlerUnnormalised extends ScalarisDataHandler {\n    \n    /**\n     * Gets the key to store {@link Revision} objects at.\n     * \n     * @param title     the title of the page\n     * @param id        the id of the revision\n     * @param nsObject  the namespace for page title normalisation\n     * \n     * @return Scalaris key\n     */\n    public static String getRevKey(String title, int id, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getRevKey(NormalisedTitle.fromUnnormalised(title, nsObject), id);\n    }\n    \n    /**\n     * Gets the key to store {@link Page} objects at.\n     * \n     * @param title     the title of the page\n     * @param nsObject  the namespace for page title normalisation\n     * \n     * @return Scalaris key\n     */\n    public final static String getPageKey(String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getPageKey(NormalisedTitle.fromUnnormalised(title, nsObject));\n    }\n    \n    /**\n     * Gets the key to store the list of revisions of a page at.\n     * \n     * @param title     the title of the page\n     * @param nsObject  the namespace for page title normalisation\n     * \n     * @return Scalaris key\n     */\n    public final static String getRevListKey(String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getRevListKey(NormalisedTitle.fromUnnormalised(title, nsObject));\n    }\n    \n    /**\n     * Gets the key to store the list of pages belonging to a category at.\n     * \n     * @param title     the category title (including <tt>Category:</tt>)\n     * @param nsObject  the namespace for page title normalisation\n     * \n     * @return Scalaris key\n     */\n    public final static String getCatPageListKey(String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getCatPageListKey(NormalisedTitle.fromUnnormalised(title, nsObject));\n    }\n\n    /**\n     * Gets the key to store the number of pages belonging to a category at.\n     * \n     * @param title     the category title (including <tt>Category:</tt>)\n     * @param nsObject  the namespace for page title normalisation\n     * \n     * @return Scalaris key\n     */\n    public final static String getCatPageCountKey(String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getCatPageCountKey(NormalisedTitle.fromUnnormalised(title, nsObject));\n    }\n\n    /**\n     * Gets the key to store the list of pages using a template at.\n     * \n     * @param title     the template title (including <tt>Template:</tt>)\n     * @param nsObject  the namespace for page title normalisation\n     * \n     * @return Scalaris key\n     */\n    public final static String getTplPageListKey(String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getTplPageListKey(NormalisedTitle.fromUnnormalised(title, nsObject));\n    }\n    \n    /**\n     * Gets the key to store the list of pages linking to the given title.\n     * \n     * @param title     the page's title\n     * @param nsObject  the namespace for page title normalisation\n     * \n     * @return Scalaris key\n     */\n    public final static String getBackLinksPageListKey(String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getBackLinksPageListKey(NormalisedTitle.fromUnnormalised(title, nsObject));\n    }\n    \n    /**\n     * Retrieves a page's history from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the page\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return a result object with the page history on success\n     */\n    public static PageHistoryResult getPageHistory(Connection connection,\n            String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getPageHistory(connection, NormalisedTitle.fromUnnormalised(title, nsObject));\n    }\n\n    /**\n     * Retrieves the current, i.e. most up-to-date, version of a page from\n     * Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the page\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return a result object with the page and revision on success\n     */\n    public static RevisionResult getRevision(Connection connection,\n            String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getRevision(connection, NormalisedTitle.fromUnnormalised(title, nsObject));\n    }\n\n    /**\n     * Retrieves the given version of a page from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the page\n     * @param id\n     *            the id of the version\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return a result object with the page and revision on success\n     */\n    public static RevisionResult getRevision(Connection connection,\n            String title, int id, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getRevision(connection, NormalisedTitle.fromUnnormalised(title, nsObject), id);\n    }\n\n    /**\n     * Retrieves the current version of all given pages from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param titles\n     *            the titles of the pages\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * @param statName\n     *            name of the statistic to collect\n     * \n     * @return a result object with the pages and revisions on success\n     */\n    public static ValueResult<List<RevisionResult>> getRevisions(Connection connection,\n            Collection<String> titles, final String statName, final MyNamespace nsObject) {\n        final ArrayList<NormalisedTitle> normalisedTitles = new ArrayList<NormalisedTitle>(titles.size());\n        MyWikiModel.normalisePageTitles(titles, nsObject, normalisedTitles);\n        return ScalarisDataHandlerNormalised.getRevisions(connection, normalisedTitles, statName);\n    }\n\n    /**\n     * Retrieves a list of pages in the given category from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the category\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return a result object with the page list on success\n     */\n    public static ValueResult<List<NormalisedTitle>> getPagesInCategory(Connection connection,\n            String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getPagesInCategory(connection, NormalisedTitle.fromUnnormalised(title, nsObject));\n    }\n\n    /**\n     * Retrieves a list of pages using the given template from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the template\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return a result object with the page list on success\n     */\n    public static ValueResult<List<NormalisedTitle>> getPagesInTemplate(Connection connection,\n            String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getPagesInTemplate(connection, NormalisedTitle.fromUnnormalised(title, nsObject));\n    }\n\n    /**\n     * Retrieves a list of pages linking to the given page from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the page\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return a result object with the page list on success\n     */\n    public static ValueResult<List<NormalisedTitle>> getPagesLinkingTo(Connection connection,\n            String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getPagesLinkingTo(connection, NormalisedTitle.fromUnnormalised(title, nsObject));\n    }\n\n    /**\n     * Retrieves the number of pages in the given category from Scalaris.\n     * \n     * @param connection\n     *            the connection to Scalaris\n     * @param title\n     *            the title of the category\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return a result object with the number of pages on success\n     */\n    public static ValueResult<BigInteger> getPagesInCategoryCount(\n            Connection connection, String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerNormalised.getPagesInCategoryCount(connection, NormalisedTitle.fromUnnormalised(title, nsObject));\n    }\n\n    /**\n     * Saves or edits a page with the given parameters\n     * \n     * @param connection\n     *            the connection to use\n     * @param title0\n     *            the (unnormalised) title of the page\n     * @param newRev\n     *            the new revision to add\n     * @param prevRevId\n     *            the version of the previously existing revision or <tt>-1</tt>\n     *            if there was no previous revision\n     * @param restrictions\n     *            new restrictions of the page or <tt>null</tt> if they should\n     *            not be changed\n     * @param siteinfo\n     *            information about the wikipedia (used for parsing categories\n     *            and templates)\n     * @param username\n     *            name of the user editing the page (for enforcing restrictions)\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return success status\n     */\n    public static SavePageResult savePage(final Connection connection, final String title0,\n            final Revision newRev, final int prevRevId, final Map<String, String> restrictions,\n            final SiteInfo siteinfo, final String username, final MyNamespace nsObject) {\n        long timeAtStart = System.currentTimeMillis();\n        final String statName = \"SAVE:\" + title0;\n        Page oldPage = null;\n        Page newPage = null;\n        List<ShortRevision> newShortRevs = null;\n        BigInteger pageEdits = null;\n        List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n        if (connection == null) {\n            return new SavePageResult(false, involvedKeys,\n                    \"no connection to Scalaris\", true, oldPage, newPage,\n                    newShortRevs, pageEdits, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n        \n        final NormalisedTitle normTitle = NormalisedTitle.fromUnnormalised(title0, nsObject);\n        final String normTitleStr = normTitle.toString();\n        Transaction scalaris_tx = new Transaction(connection);\n\n        // check that the current version is still up-to-date:\n        // read old version first, then write\n        String pageInfoKey = getPageKey(title0, nsObject);\n        \n        Transaction.RequestList requests = new Transaction.RequestList();\n        requests.addOp(new ReadOp(pageInfoKey));\n        \n        Transaction.ResultList results;\n        try {\n            addInvolvedKeys(involvedKeys, requests.getRequests());\n            results = scalaris_tx.req_list(requests);\n        } catch (Exception e) {\n            return new SavePageResult(false, involvedKeys,\n                    e.getClass().getCanonicalName() + \" getting page info (\" + pageInfoKey\n                            + \") from Scalaris: \" + e.getMessage(),\n                    e instanceof ConnectionException, oldPage, newPage,\n                    newShortRevs, pageEdits, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n\n        int oldRevId;\n        try {\n            oldPage = results.processReadAt(0).jsonValue(Page.class);\n            newPage = new Page(oldPage.getTitle(), oldPage.getId(),\n                    oldPage.isRedirect(), new LinkedHashMap<String, String>(\n                            oldPage.getRestrictions()), newRev);\n            oldRevId = oldPage.getCurRev().getId();\n        } catch (NotFoundException e) {\n            // this is ok and means that the page did not exist yet\n            newPage = new Page(title0, 1, false,\n                    new LinkedHashMap<String, String>(), newRev);\n            oldRevId = 0;\n        } catch (Exception e) {\n            return new SavePageResult(false, involvedKeys,\n                    e.getClass().getCanonicalName() + \" reading \\\"\" + pageInfoKey\n                            + \"\\\" from Scalaris: \" + e.getMessage(),\n                    e instanceof ConnectionException, oldPage, newPage,\n                    newShortRevs, pageEdits, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n        newRev.setId(oldRevId + 1);\n        \n        if (!newPage.checkEditAllowed(username)) {\n            return new SavePageResult(false, involvedKeys,\n                    \"operation not allowed: edit is restricted\", false,\n                    oldPage, newPage, newShortRevs, pageEdits,\n                    statName, System.currentTimeMillis() - timeAtStart);\n        }\n        \n        /*\n         * if prevRevId is greater than 0, it must match the old revision,\n         * if it is -1, then there should not be an old page\n         */\n        if ((prevRevId > 0 && prevRevId != oldRevId) || (prevRevId == -1 && oldPage != null)) {\n            return new SavePageResult(false, involvedKeys, \"curRev(\" + oldRevId\n                    + \") != oldRev(\" + prevRevId + \")\", false, oldPage,\n                    newPage, newShortRevs, pageEdits, statName,\n                    System.currentTimeMillis() - timeAtStart);\n        }\n\n        // write:\n        // get previous categories, templates and backlinks:\n        final MyWikiModel wikiModel = new MyWikiModel(\"\", \"\", nsObject);\n        wikiModel.setNamespaceName(nsObject.getNamespaceByNumber(normTitle.namespace));\n        wikiModel.setPageName(normTitle.title);\n        Set<String> oldCats;\n        Set<String> oldTpls;\n        Set<String> oldLnks;\n        if (oldPage != null && oldPage.getCurRev() != null) {\n            // get a list of previous categories and templates:\n            wikiModel.setUp();\n            final long timeAtRenderStart = System.currentTimeMillis();\n            wikiModel.renderPageWithCache(null, oldPage.getCurRev().unpackedText());\n            timeAtStart += (System.currentTimeMillis() - timeAtRenderStart);\n            // note: no need to normalise the pages, we will do so during the write/read key generation\n            oldCats = wikiModel.getCategories().keySet();\n            oldTpls = wikiModel.getTemplatesNoMagicWords();\n            if (Options.getInstance().WIKI_USE_BACKLINKS) {\n                oldLnks = wikiModel.getLinks();\n            } else {\n                // use empty link lists to turn back-links off\n                oldLnks = new HashSet<String>();\n            }\n            wikiModel.tearDown();\n        } else {\n            oldCats = new HashSet<String>();\n            oldTpls = new HashSet<String>();\n            oldLnks = new HashSet<String>();\n        }\n        // get new categories and templates\n        wikiModel.setUp();\n        do {\n            final long timeAtRenderStart = System.currentTimeMillis();\n            wikiModel.renderPageWithCache(null, newRev.unpackedText());\n            timeAtStart += (System.currentTimeMillis() - timeAtRenderStart);\n        } while (false);\n        newPage.setRedirect(wikiModel.getRedirectLink() != null);\n        if (restrictions != null) {\n            newPage.setRestrictions(restrictions);\n        }\n        \n        // note: do not tear down the wiki model - the following statements\n        // still need it and it will be removed at the end of the method anyway\n        // note: no need to normalise the pages, we will do so during the write/read key generation\n        final Set<String> newCats = wikiModel.getCategories().keySet();\n        Difference catDiff = new Difference(oldCats, newCats,\n                new Difference.GetPageListAndCountKey() {\n                    @Override\n                    public String getPageListKey(String name) {\n                        return getCatPageListKey(\n                                wikiModel.getCategoryNamespace() + \":\" + name,\n                                nsObject);\n                    }\n\n                    @Override\n                    public String getPageCountKey(String name) {\n                        return getCatPageCountKey(\n                                wikiModel.getCategoryNamespace() + \":\" + name,\n                                nsObject);\n                    }\n                }, ScalarisOpType.CATEGORY_PAGE_LIST, ScalarisOpType.CATEGORY_PAGE_COUNT);\n        final Set<String> newTpls = wikiModel.getTemplatesNoMagicWords();\n        Difference tplDiff = new Difference(oldTpls, newTpls,\n                new Difference.GetPageListKey() {\n                    @Override\n                    public String getPageListKey(String name) {\n                        return getTplPageListKey(\n                                wikiModel.getTemplateNamespace() + \":\" + name,\n                                nsObject);\n                    }\n                }, ScalarisOpType.TEMPLATE_PAGE_LIST, null);\n        // use empty link lists to turn back-links off\n        final Set<String> newLnks = Options.getInstance().WIKI_USE_BACKLINKS ? wikiModel.getLinks() : new HashSet<String>();\n        Difference lnkDiff = new Difference(oldLnks, newLnks,\n                new Difference.GetPageListKey() {\n                    @Override\n                    public String getPageListKey(String name) {\n                        return getBackLinksPageListKey(name, nsObject);\n                    }\n                }, ScalarisOpType.BACKLINK_PAGE_LIST, null);\n        \n\n        // now save the changes:\n        do {\n            final MyScalarisTxOpExecutor executor0 = new MyScalarisTxOpExecutor(\n                    scalaris_tx, involvedKeys);\n            executor0.setCommitLast(true);\n            MyScalarisOpExecWrapper executor = new MyScalarisOpExecWrapper(\n                    executor0);\n\n            int articleCountChange = 0;\n            final boolean wasArticle = (oldPage != null)\n                    && MyWikiModel.isArticle(normTitle.namespace, oldLnks, oldCats);\n            final boolean isArticle = (normTitle.namespace == 0)\n                    && MyWikiModel.isArticle(normTitle.namespace, newLnks, newCats);\n            if (wasArticle == isArticle) {\n                articleCountChange = 0;\n            } else if (!wasArticle) {\n                articleCountChange = 1;\n            } else if (!isArticle) {\n                articleCountChange = -1;\n            }\n\n            //  PAGE LISTS UPDATE, step 1: append to / remove from old lists\n            executor.addAppend(ScalarisOpType.SHORTREV_LIST, getRevListKey(title0, nsObject), new ShortRevision(newRev), null, null);\n            if (articleCountChange != 0) {\n                executor.addIncrement(ScalarisOpType.ARTICLE_COUNT, getArticleCountKey(), articleCountChange, normTitleStr);\n            }\n\n            // write differences (categories, templates, backlinks)\n            catDiff.addScalarisOps(executor, normTitleStr);\n            tplDiff.addScalarisOps(executor, normTitleStr);\n            lnkDiff.addScalarisOps(executor, normTitleStr);\n\n            // new page? -> add to page/article lists\n            if (oldPage == null) {\n                final String pageListKey = getPageListKey(normTitle.namespace);\n                final String pageCountKey = getPageCountKey(normTitle.namespace);\n                executor.addAppend(ScalarisOpType.PAGE_LIST, pageListKey, normTitleStr, ScalarisOpType.PAGE_COUNT, pageCountKey);\n            }\n\n            executor.addWrite(ScalarisOpType.PAGE, getPageKey(title0, nsObject), newPage);\n            if (oldPage != null) {\n                executor.addWrite(ScalarisOpType.REVISION, getRevKey(title0, oldPage.getCurRev().getId(), nsObject), oldPage.getCurRev());\n            }\n\n            //  PAGE LISTS UPDATE, step 2: execute and evaluate operations\n            try {\n                executor.getExecutor().run();\n            } catch (Exception e) {\n                SavePageResult result = new SavePageResult(false, involvedKeys,\n                        e.getClass().getCanonicalName() + \" writing page \\\"\" + title0\n                                + \"\\\" to Scalaris: \" + e.getMessage(),\n                        e instanceof ConnectionException, oldPage, newPage,\n                        newShortRevs, pageEdits, statName,\n                        System.currentTimeMillis() - timeAtStart);\n                if (e instanceof AbortException) {\n                    result.failedKeys.addAll(((AbortException) e).getFailedKeys());\n                }\n                return result;\n            }\n        } while (false);\n        \n        if (Options.getInstance().WIKI_STORE_CONTRIBUTIONS == STORE_CONTRIB_TYPE.OUTSIDE_TX) {\n            addContribution(scalaris_tx, oldPage, newPage, involvedKeys);\n        }\n        \n        increasePageEditStat(scalaris_tx, involvedKeys);\n        \n        return new SavePageResult(involvedKeys, oldPage, newPage, newShortRevs,\n                pageEdits, statName, System.currentTimeMillis() - timeAtStart);\n    }\n    \n    /**\n     * Increases the number of overall page edits statistic.\n     * \n     * @param scalaris_tx\n     *            the transaction object to use\n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     */\n    private static void increasePageEditStat(\n            Transaction scalaris_tx, List<InvolvedKey> involvedKeys) {\n        // increase number of page edits (for statistics)\n        // as this is not that important, use a separate transaction and do not\n        // fail if updating the value fails\n        final MyScalarisTxOpExecutor executor0 = new MyScalarisTxOpExecutor(\n                scalaris_tx, involvedKeys);\n        executor0.setCommitLast(true);\n        MyScalarisOpExecWrapper executor = new MyScalarisOpExecWrapper(\n                executor0);\n        \n        executor.addIncrement(ScalarisOpType.EDIT_STAT, getStatsPageEditsKey(), 1, getStatsPageEditsKey());\n        try {\n            executor.getExecutor().run();\n        } catch (Exception e) {\n        }\n    }\n\n    /**\n     * Adds a contribution to the list of contributions of the user.\n     * \n     * @param scalaris_tx\n     *            the transaction object to use\n     * @param oldPage\n     *            the old page object or <tt>null</tt> if there was no old page\n     * @param newPage\n     *            the newly created page object\n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     */\n    private static void addContribution(\n            Transaction scalaris_tx, Page oldPage, Page newPage, List<InvolvedKey> involvedKeys) {\n        // as this is not that important, use a separate transaction and do not\n        // fail if updating the value fails\n        final MyScalarisTxOpExecutor executor0 = new MyScalarisTxOpExecutor(\n                scalaris_tx, involvedKeys);\n        executor0.setCommitLast(true);\n        MyScalarisOpExecWrapper executor = new MyScalarisOpExecWrapper(\n                executor0);\n\n        String scalaris_key = getContributionListKey(newPage.getCurRev().getContributor().toString());\n        executor.addAppend(ScalarisOpType.CONTRIBUTION, scalaris_key,\n                Arrays.asList(new Contribution(oldPage, newPage)), null, null);\n        try {\n            executor.getExecutor().run();\n        } catch (Exception e) {\n        }\n    }\n    \n    /**\n     * Handles differences of sets.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    private static class Difference {\n        public Set<String> onlyOld;\n        public Set<String> onlyNew;\n        @SuppressWarnings(\"unchecked\")\n        private Set<String>[] changes = new Set[2];\n        private GetPageListKey keyGen;\n        final private ScalarisOpType opType;\n        final private ScalarisOpType countOpType;\n        \n        /**\n         * Creates a new object calculating differences of two sets.\n         * \n         * @param oldSet\n         *            the old set\n         * @param newSet\n         *            the new set\n         * @param keyGen\n         *            object creating the Scalaris key for the page lists (based\n         *            on a set entry)\n         * @param opType\n         *            operation type indicating what is being updated\n         * @param countOpType\n         *            operation type indicating what is being updated for\n         *            updating count keys (if there are no count keys, this may\n         *            be <tt>null</tt>)\n         */\n        public Difference(Set<String> oldSet, Set<String> newSet,\n                GetPageListKey keyGen, ScalarisOpType opType,\n                ScalarisOpType countOpType) {\n            this.onlyOld = new HashSet<String>(oldSet);\n            this.onlyNew = new HashSet<String>(newSet);\n            this.onlyOld.removeAll(newSet);\n            this.onlyNew.removeAll(oldSet);\n            this.changes[0] = this.onlyOld;\n            this.changes[1] = this.onlyNew;\n            this.keyGen = keyGen;\n            this.opType = opType;\n            this.countOpType = countOpType;\n        }\n\n        static public interface GetPageListKey {\n            /**\n             * Gets the Scalaris key for a page list for the given article's\n             * name.\n             * \n             * @param name the name of an article\n             * @return the key for Scalaris\n             */\n            public abstract String getPageListKey(String name);\n        }\n        \n        static public interface GetPageListAndCountKey extends GetPageListKey {\n            /**\n             * Gets the Scalaris key for a page list counter for the given\n             * article's name.\n             * \n             * @param name the name of an article\n             * @return the key for Scalaris\n             */\n            public abstract String getPageCountKey(String name);\n        }\n        \n        /**\n         * Adds the appropriate list append operations to the given executor.\n         * \n         * @param executor  executor performing the Scalaris operations\n         * @param title     (normalised) page name to update\n         */\n        public void addScalarisOps(MyScalarisOpExecWrapper executor,\n                String title) {\n            String scalaris_key;\n            GetPageListAndCountKey keyCountGen = null;\n            if (keyGen instanceof GetPageListAndCountKey) {\n                assert(null != countOpType);\n                keyCountGen = (GetPageListAndCountKey) keyGen;\n            }\n            // remove from old page list\n            for (String name: onlyOld) {\n                scalaris_key = keyGen.getPageListKey(name);\n//                System.out.println(scalaris_key + \" -= \" + title);\n                String scalaris_countKey = keyCountGen == null ? null : keyCountGen.getPageCountKey(name);\n                executor.addRemove(opType, scalaris_key, title, countOpType, scalaris_countKey);\n            }\n            // add to new page list\n            for (String name: onlyNew) {\n                scalaris_key = keyGen.getPageListKey(name);\n//              System.out.println(scalaris_key + \" += \" + title);\n                String scalaris_countKey = keyCountGen == null ? null : keyCountGen.getPageCountKey(name);\n                executor.addAppend(opType, scalaris_key, title, countOpType, scalaris_countKey);\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/ScalarisOpType.java",
    "content": "package de.zib.scalaris.examples.wikipedia;\n\n/**\n * Different types of DB operations.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic enum ScalarisOpType {\n    /**\n     * Operation involving a (central) page list.\n     */\n    PAGE_LIST(\"PAGE_LIST\"),\n    /**\n     * Operation involving a counter of a (central) page list.\n     */\n    PAGE_COUNT(\"PAGE_COUNT\"),\n    /**\n     * Operation involving a category page list.\n     */\n    CATEGORY_PAGE_LIST(\"CATEGORY_PAGE_LIST\"),\n    /**\n     * Operation involving a counter of a category page list.\n     */\n    CATEGORY_PAGE_COUNT(\"CATEGORY_PAGE_COUNT\"),\n    /**\n     * Operation involving a template page list.\n     */\n    TEMPLATE_PAGE_LIST(\"TEMPLATE_PAGE_LIST\"),\n    /**\n     * Operation involving a backlink page list.\n     */\n    BACKLINK_PAGE_LIST(\"BACKLINK_PAGE_LIST\"),\n    /**\n     * Operation involving a list of (short) revisions.\n     */\n    SHORTREV_LIST(\"SHORTREV_LIST\"),\n    /**\n     * Operation involving the article counter.\n     */\n    ARTICLE_COUNT(\"ARTICLE_COUNT\"),\n    /**\n     * Operation involving a wiki page.\n     */\n    PAGE(\"PAGE\"),\n    /**\n     * Operation involving a wiki page revision.\n     */\n    REVISION(\"REVISION\"),\n    /**\n     * Operation involving a contribution.\n     */\n    CONTRIBUTION(\"CONTRIBUTION\"),\n    /**\n     * Operation involving the edit stats.\n     */\n    EDIT_STAT(\"EDIT_STAT\");\n\n    private final String text;\n\n    ScalarisOpType(String text) {\n        this.text = text;\n    }\n\n    /**\n     * Converts the enum to text.\n     */\n    public String toString() {\n        return this.text;\n    }\n\n    /**\n     * Tries to convert a text to the according enum value.\n     * \n     * @param text the text to convert\n     * \n     * @return the according enum value\n     */\n    public static ScalarisOpType fromString(String text) {\n        if (text != null) {\n            for (ScalarisOpType b : ScalarisOpType.values()) {\n                if (text.equalsIgnoreCase(b.text)) {\n                    return b;\n                }\n            }\n        }\n        throw new IllegalArgumentException(\"No constant with text \" + text\n                + \" found\");\n    }\n}"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/ScalarisReadListOp1.java",
    "content": "package de.zib.scalaris.examples.wikipedia;\n\nimport java.util.ArrayList;\nimport java.util.Collection;\nimport java.util.HashSet;\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.ResultList;\nimport de.zib.scalaris.UnknownException;\nimport de.zib.scalaris.examples.wikipedia.Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE;\nimport de.zib.scalaris.examples.wikipedia.Options.IBuckets;\nimport de.zib.scalaris.examples.wikipedia.Options.IReadBuckets;\nimport de.zib.scalaris.examples.wikipedia.Options.Optimisation;\nimport de.zib.scalaris.examples.wikipedia.Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE.WriteCacheDiff;\nimport de.zib.scalaris.examples.wikipedia.Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE.WriteCacheDiffConv;\nimport de.zib.scalaris.executor.ScalarisOp;\nimport de.zib.scalaris.operations.ReadOp;\n\n/**\n * Implements a list read operation.\n *\n * @param <T> the type of objects in the list\n *\n * @author Nico Kruber, kruber@zib.de\n */\npublic class ScalarisReadListOp1<T> implements ScalarisOp {\n    final Collection<String> keys;\n    final int buckets;\n    final ErlangConverter<List<T>> listConv;\n    final ErlangConverter<T> elemConv;\n    final boolean failNotFound;\n    final ArrayList<T> value = new ArrayList<T>();\n    final private Optimisation optimisation;\n\n    /**\n     * Creates a new list read operation.\n     * \n     * @param keys\n     *            the keys under which the list is stored in Scalaris\n     * @param optimisation\n     *            the list optimisation to use\n     * @param listConv\n     *            converter to make an {@link ErlangValue} to a {@link List} of\n     *            <tt>T</tt>\n     * @param elemConv\n     *            converter to make an {@link ErlangValue} to a <tt>T</tt>\n     * @param failNotFound\n     *            whether to re-throw the {@link NotFoundException} if no list\n     *            key was found\n     */\n    public ScalarisReadListOp1(final Collection<String> keys,\n            final Optimisation optimisation, ErlangConverter<List<T>> listConv,\n            ErlangConverter<T> elemConv, boolean failNotFound) {\n        this.keys = keys;\n        if (optimisation instanceof IBuckets) {\n            this.buckets = ((IBuckets) optimisation).getBuckets();\n        } else {\n            this.buckets = 1;\n        }\n        this.listConv = listConv;\n        this.elemConv = elemConv;\n        this.failNotFound = failNotFound;\n        this.optimisation = optimisation;\n    }\n\n    public int workPhases() {\n        return 1;\n    }\n\n    public final int doPhase(final int phase, final int firstOp,\n            final ResultList results, final RequestList requests)\n            throws OtpErlangException, UnknownException,\n            IllegalArgumentException {\n        switch (phase) {\n            case 0: return prepareRead(requests);\n            case 1: return checkRead(firstOp, results);\n            default:\n                throw new IllegalArgumentException(\"No phase \" + phase);\n        }\n    }\n\n    /**\n     * Adds a read operation for the list to the request list.\n     *\n     * @param requests the request list\n     *\n     * @return <tt>0</tt> (no operation processed since no results are used)\n     */\n    protected int prepareRead(final RequestList requests) {\n        for (String key : keys) {\n            if (!(optimisation instanceof IBuckets)) {\n                requests.addOp(new ReadOp(key));\n            } else {\n                for (int i = 0; i < buckets; ++i) {\n                    requests.addOp(new ReadOp(key + \":\" + i));\n                }\n            }\n        }\n        return 0;\n    }\n\n    /**\n     * Verifies the read operation(s) and creates the full list.\n     *\n     * @param firstOp   the first operation to process inside the result list\n     * @param results   the result list\n     * @param requests  the request list\n     *\n     * @return number of processed operations\n     */\n    protected int checkRead(int firstOp, final ResultList results) throws OtpErlangException,\n            UnknownException {\n        int notFound = 0;\n        NotFoundException lastNotFound = null;\n        ErlangConverter<WriteCacheDiff<T>> writeCacheDiffConv = null;\n        HashSet<T> toDelete = null;\n        if (optimisation instanceof APPEND_INCREMENT_BUCKETS_WITH_WCACHE) {\n            writeCacheDiffConv = new WriteCacheDiffConv<T>(elemConv);\n            toDelete = new HashSet<T>();\n        }\n        for (int x = 0; x < keys.size(); ++x) {\n            for (int i = 0; i < buckets; ++i) {\n                try {\n                    ErlangValue result = results.processReadAt(firstOp++);\n                    if (writeCacheDiffConv != null && i >= ((IReadBuckets) optimisation).getReadBuckets()) {\n                        WriteCacheDiff<T> diff = writeCacheDiffConv.convert(result);\n                        if (value.isEmpty() && !diff.toAdd.isEmpty()) {\n                            // assume each bucket has the same size\n                            // different keys may have different list sizes though\n                            // -> use automatic capacity increase for them\n                            value.ensureCapacity(diff.toAdd.size() * (buckets - i));\n                        }\n                        value.addAll(diff.toAdd);\n                        toDelete.addAll(diff.toDelete);\n                    } else {\n                        final List<T> list = listConv.convert(result);\n                        if (value.isEmpty() && !list.isEmpty()) {\n                            // assume each bucket has the same size\n                            // different keys may have different list sizes though\n                            // -> use automatic capacity increase for them\n                            value.ensureCapacity(list.size() * (buckets - i));\n                        }\n                        value.addAll(list);\n                    }\n                } catch (NotFoundException e) {\n                    ++notFound;\n                    lastNotFound = e;\n                }\n            }\n        }\n        if (toDelete != null && !toDelete.isEmpty()) {\n            value.removeAll(toDelete);\n        }\n        if (failNotFound && notFound == (keys.size() * buckets)) {\n            throw lastNotFound;\n        }\n        return keys.size() * buckets;\n    }\n\n    /**\n     * The full list that has been read (if {@link #failNotFound} is not set and\n     * none of the given keys was found, an empty list is returned).\n     * \n     * @return the value from Scalaris or an empty list\n     */\n    public List<T> getValue() {\n        return value;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/ScalarisReadNumberOp1.java",
    "content": "package de.zib.scalaris.examples.wikipedia;\n\nimport java.math.BigInteger;\nimport java.util.Collection;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.ResultList;\nimport de.zib.scalaris.UnknownException;\nimport de.zib.scalaris.examples.wikipedia.Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE;\nimport de.zib.scalaris.examples.wikipedia.Options.IBuckets;\nimport de.zib.scalaris.examples.wikipedia.Options.Optimisation;\nimport de.zib.scalaris.executor.ScalarisOp;\nimport de.zib.scalaris.operations.ReadOp;\n\n/**\n * Implements a number read operation.\n *\n * @author Nico Kruber, kruber@zib.de\n */\npublic class ScalarisReadNumberOp1 implements ScalarisOp {\n    final Collection<String> keys;\n    final int buckets;\n    final boolean failNotFound;\n    BigInteger value = BigInteger.ZERO;\n    final private Optimisation optimisation;\n\n    /**\n     * Creates a new number read operation.\n     * \n     * @param keys\n     *            the keys under which the number is stored in Scalaris\n     * @param optimisation\n     *            the list optimisation to use\n     * @param failNotFound\n     *            whether to re-throw the {@link NotFoundException} if no list\n     *            key was found\n     */\n    public ScalarisReadNumberOp1(final Collection<String> keys,\n            final Optimisation optimisation, boolean failNotFound) {\n        this.keys = keys;\n        if (optimisation instanceof APPEND_INCREMENT_BUCKETS_WITH_WCACHE) {\n            APPEND_INCREMENT_BUCKETS_WITH_WCACHE opt2 = (APPEND_INCREMENT_BUCKETS_WITH_WCACHE) optimisation;\n            // counters only exist in the read and write-add buckets\n            this.buckets = opt2.getReadBuckets() + opt2.getWriteBuckets();\n        } else if (optimisation instanceof IBuckets) {\n            this.buckets = ((IBuckets) optimisation).getBuckets();\n        } else {\n            this.buckets = 1;\n        }\n        this.failNotFound = failNotFound;\n        this.optimisation = optimisation;\n    }\n\n    public int workPhases() {\n        return 1;\n    }\n\n    public final int doPhase(final int phase, final int firstOp,\n            final ResultList results, final RequestList requests)\n            throws OtpErlangException, UnknownException,\n            IllegalArgumentException {\n        switch (phase) {\n            case 0: return prepareRead(requests);\n            case 1: return checkRead(firstOp, results);\n            default:\n                throw new IllegalArgumentException(\"No phase \" + phase);\n        }\n    }\n\n    /**\n     * Adds a read operation for the number to the request list.\n     *\n     * @param requests the request list\n     *\n     * @return <tt>0</tt> (no operation processed since no results are used)\n     */\n    protected int prepareRead(final RequestList requests) {\n        for (String key : keys) {\n            if (!(optimisation instanceof IBuckets)) {\n                requests.addOp(new ReadOp(key));\n            } else {\n                for (int i = 0; i < buckets; ++i) {\n                    requests.addOp(new ReadOp(key + \":\" + i));\n                }\n            }\n        }\n        return 0;\n    }\n\n    /**\n     * Verifies the read operation(s) and creates the full number.\n     *\n     * @param firstOp   the first operation to process inside the result list\n     * @param results   the result list\n     * @param requests  the request list\n     *\n     * @return number of processed operations\n     */\n    protected int checkRead(int firstOp, final ResultList results) throws OtpErlangException,\n            UnknownException {\n        int notFound = 0;\n        NotFoundException lastNotFound = null;\n        for (int x = 0; x < keys.size(); ++x) {\n            for (int i = 0; i < buckets; ++i) {\n                try {\n                    value = value.add(results.processReadAt(firstOp++).bigIntValue());\n                } catch (NotFoundException e) {\n                    ++notFound;\n                    lastNotFound = e;\n                }\n            }\n        }\n        if (failNotFound && notFound == (keys.size() * buckets)) {\n            throw lastNotFound;\n        }\n        return keys.size() * buckets;\n    }\n\n    /**\n     * The (assembled) number that has been read (if {@link #failNotFound} is\n     * not set and none of the given keys was found, {@link BigInteger#ZERO} is\n     * returned).\n     * \n     * @return the value from Scalaris or {@link BigInteger#ZERO}\n     */\n    public BigInteger getValue() {\n        return value;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/ScalarisReadRandomListEntryOp1.java",
    "content": "package de.zib.scalaris.examples.wikipedia;\n\nimport java.security.InvalidParameterException;\nimport java.util.ArrayList;\nimport java.util.Collection;\nimport java.util.HashSet;\nimport java.util.List;\nimport java.util.Random;\nimport java.util.TreeMap;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\nimport de.zib.scalaris.EmptyListException;\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.ResultList;\nimport de.zib.scalaris.UnknownException;\nimport de.zib.scalaris.examples.wikipedia.Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE;\nimport de.zib.scalaris.examples.wikipedia.Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY;\nimport de.zib.scalaris.examples.wikipedia.Options.IBuckets;\nimport de.zib.scalaris.examples.wikipedia.Options.IPartialRead;\nimport de.zib.scalaris.examples.wikipedia.Options.Optimisation;\nimport de.zib.scalaris.executor.ScalarisOp;\nimport de.zib.scalaris.operations.ReadOp;\nimport de.zib.scalaris.operations.ReadRandomFromListOp;\n\n/**\n * Implements a random list entry read operation.\n *\n * @param <T> the type of objects in the list\n *\n * @author Nico Kruber, kruber@zib.de\n */\npublic class ScalarisReadRandomListEntryOp1<T> implements ScalarisOp {\n    final Collection<String> keys;\n    final int bucketsPerKey;\n    final ErlangConverter<List<T>> listConv;\n    final ErlangConverter<T> elemConv;\n    final boolean failNotFound;\n    T value = null;\n    final Random random;\n    final private Optimisation optimisation;\n    final private boolean readOnlyOneBucket;\n\n    /**\n     * Creates a new (random) list entry read operation.\n     * \n     * Note: {@link APPEND_INCREMENT_BUCKETS_WITH_WCACHE} is not supported at\n     * the moment!\n     * \n     * @param keys\n     *            the keys under which the list is stored in Scalaris\n     * @param optimisation\n     *            the list optimisation to use\n     * @param readOnlyOneBucket\n     *            try to read fron a single bucket only (if supported by the\n     *            optimisation) - in contrast to reading values from all of them\n     * @param elemConv\n     *            converter to make an {@link ErlangValue} to a <tt>T</tt>\n     * @param listConv\n     *            converter to make an {@link ErlangValue} to a {@link List} of\n     *            <tt>T</tt>\n     * @param failNotFound\n     *            whether to re-throw the {@link NotFoundException} if a list\n     *            key was not found\n     * @param random\n     *            the random number generator to use\n     */\n    public ScalarisReadRandomListEntryOp1(final Collection<String> keys,\n            final Optimisation optimisation, boolean readOnlyOneBucket,\n            ErlangConverter<T> elemConv, ErlangConverter<List<T>> listConv,\n            boolean failNotFound, Random random) {\n        if (optimisation instanceof APPEND_INCREMENT_BUCKETS_WITH_WCACHE) {\n            throw new InvalidParameterException(\n                    \"APPEND_INCREMENT_BUCKETS_WITH_WCACHE not supported\");\n        }\n        this.keys = keys;\n        this.listConv = listConv;\n        this.elemConv = elemConv;\n        this.failNotFound = failNotFound;\n        this.random = random;\n        this.optimisation = optimisation;\n        this.readOnlyOneBucket = readOnlyOneBucket;\n\n        // keep in sync with prepareRead()!\n        if (!(optimisation instanceof IBuckets)) {\n            this.bucketsPerKey = 1;\n        } else if (!readOnlyOneBucket) {\n            this.bucketsPerKey = ((IBuckets) optimisation).getBuckets();\n        } else if (optimisation instanceof APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY) {\n            this.bucketsPerKey = 2;\n        } else {\n            this.bucketsPerKey = 1;\n        }\n    }\n\n    public int workPhases() {\n        return 1;\n    }\n\n    public final int doPhase(final int phase, final int firstOp,\n            final ResultList results, final RequestList requests)\n            throws OtpErlangException, UnknownException,\n            IllegalArgumentException {\n        switch (phase) {\n            case 0: return prepareRead(requests);\n            case 1: return checkRead(firstOp, results);\n            default:\n                throw new IllegalArgumentException(\"No phase \" + phase);\n        }\n    }\n\n    /**\n     * Adds a read operation for the list to the request list.\n     *\n     * @param requests the request list\n     *\n     * @return <tt>0</tt> (no operation processed since no results are used)\n     */\n    protected int prepareRead(final RequestList requests) {\n        for (String key : keys) {\n            HashSet<String> bucketKeys = new HashSet<String>(bucketsPerKey);\n            if (!(optimisation instanceof IBuckets)) {\n                bucketKeys.add(key);\n            } else if (!readOnlyOneBucket) {\n                for (int i = 0; i < bucketsPerKey; ++i) {\n                    bucketKeys.add(key + \":\" + i);\n                }\n            } else if (optimisation instanceof APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY) {\n                APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY optimisation2 = (APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY) optimisation;\n                // read a single read-bucket and a single write-bucket\n                int readBuckets = optimisation2.getReadBuckets();\n                bucketKeys.add(key + \":\" + random.nextInt(readBuckets));\n                bucketKeys.add(key + \":\" + (random.nextInt(optimisation2.getBuckets() - readBuckets) + readBuckets));\n            } else {\n                bucketKeys.add(key + \":\" + random.nextInt(((IBuckets) optimisation).getBuckets()));\n            }\n            assert(bucketsPerKey == bucketKeys.size());\n            if (optimisation instanceof IPartialRead) {\n                for (String bucketKey : bucketKeys) {\n                    requests.addOp(new ReadRandomFromListOp(bucketKey));\n                }\n            } else {\n                for (String bucketKey : bucketKeys) {\n                    requests.addOp(new ReadOp(bucketKey));\n                }\n            }\n        }\n        return 0;\n    }\n\n    /**\n     * Verifies the read operation(s) and creates the full list.\n     *\n     * @param firstOp   the first operation to process inside the result list\n     * @param results   the result list\n     * @param requests  the request list\n     *\n     * @return number of processed operations\n     */\n    protected int checkRead(int firstOp, final ResultList results) throws OtpErlangException,\n            UnknownException {\n        int notFound = 0;\n        NotFoundException lastNotFound = null;\n        if (optimisation instanceof IPartialRead) {\n            /*\n             * each partition may contain a different number of keys\n             * -> collect all partition's random values and calculate the final\n             * random value based on the number of entries in the partition's\n             * lists\n             * -> in the created valueMap, each random value stands for a range\n             * of keys equal to the number of list elements of its partition; we\n             * finally draw a random number in the overall range and use the\n             * read random value responsible for that\n             */\n            TreeMap<Integer, T> valueMap = new TreeMap<Integer, T>();\n            Integer listLen = 0;\n            for (int x = 0; x < keys.size(); ++x) {\n                for (int i = 0; i < bucketsPerKey; ++i) {\n                    try {\n                        ReadRandomFromListOp.Result res = ((ReadRandomFromListOp) results.get(firstOp++)).processResult();\n                        listLen += res.listLength;\n                        valueMap.put(listLen, elemConv.convert(res.randomElement));\n                    } catch (NotFoundException e) {\n                        ++notFound;\n                        lastNotFound = e;\n                    } catch (EmptyListException e) {\n                        // this is ok - we simply ignore this partition\n                    }\n                }\n            }\n            if (failNotFound && notFound == (keys.size() * bucketsPerKey)) {\n                throw lastNotFound;\n            }\n            if (listLen != 0) {\n                /*\n                 * note: tailMap returns the tail including the given key.\n                 * Since a value stored at 1 has a range of 1, we must increase\n                 * the randomly drawn integer\n                 */\n                value = valueMap.tailMap(random.nextInt(listLen) + 1).values().iterator().next();\n            }\n        } else {\n            List<T> valueList = new ArrayList<T>();\n            for (int x = 0; x < keys.size(); ++x) {\n                for (int i = 0; i < bucketsPerKey; ++i) {\n                    try {\n                        valueList.addAll(listConv.convert(results.processReadAt(firstOp++)));\n                    } catch (NotFoundException e) {\n                        ++notFound;\n                        lastNotFound = e;\n                    }\n                }\n            }\n            if (failNotFound && notFound == (keys.size() * bucketsPerKey)) {\n                throw lastNotFound;\n            }\n            if (!valueList.isEmpty()) {\n                value = valueList.get(random.nextInt(valueList.size()));\n            }\n        }\n        return keys.size();\n    }\n\n    /**\n     * A random element of the list that has been read (if {@link #failNotFound}\n     * is not set and none of the given keys was found / the chosen buckets are\n     * empty, <tt>null</tt> is returned).\n     * \n     * @return the value from Scalaris or <tt>null</tt>\n     */\n    public T getValue() {\n        return value;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/ValueResult.java",
    "content": "package de.zib.scalaris.examples.wikipedia;\n\nimport java.util.List;\n\n/**\n * Result of an operation getting a single value.\n * \n * @author Nico Kruber, kruber@zib.de\n * \n * @param <T>\n */\npublic class ValueResult<T> extends Result {\n    /**\n     * The retrieved value (may be null, e.g. if unsuccessful).\n     */\n    public T value = null;\n\n    /**\n     * Creates a new successful result with the given page list.\n     * \n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param number\n     *            the retrieved number\n     */\n    public ValueResult(List<InvolvedKey> involvedKeys, T number) {\n        super(involvedKeys);\n        this.value = number;\n    }\n\n    /**\n     * Creates a new successful result with the given value.\n     * \n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param value\n     *            the retrieved value\n     * @param name\n     *            the name of the operation (for the stats - see {@link #stats})\n     * @param time\n     *            time in milliseconds for this operation\n     */\n    public ValueResult(List<InvolvedKey> involvedKeys, T value, String name,\n            long time) {\n        super(involvedKeys);\n        this.value = value;\n        addStat(name, time);\n    }\n\n    /**\n     * Creates a new custom result (value = <tt>null</tt>).\n     * \n     * @param success\n     *            the success status\n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param message\n     *            the message to use\n     * @param connectFailed\n     *            whether the connection to the DB failed or not\n     */\n    public ValueResult(boolean success, List<InvolvedKey> involvedKeys,\n            String message, boolean connectFailed) {\n        super(success, involvedKeys, message, connectFailed);\n    }\n\n    /**\n     * Creates a new custom result (value = <tt>null</tt>).\n     * \n     * @param success\n     *            the success status\n     * @param involvedKeys\n     *            all keys that have been read or written during the operation\n     * @param message\n     *            the message to use\n     * @param connectFailed\n     *            whether the connection to the DB failed or not\n     * @param name\n     *            the name of the operation (for the stats - see {@link #stats})\n     * @param time\n     *            time in milliseconds for this operation\n     */\n    public ValueResult(boolean success, List<InvolvedKey> involvedKeys,\n            String message, boolean connectFailed, String name, long time) {\n        super(success, involvedKeys, message, connectFailed);\n        addStat(name, time);\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/WikiServletContext.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia;\n\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiPageBeanBase;\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiServlet;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\nimport de.zib.scalaris.examples.wikipedia.plugin.WikiEventHandler;\n\n/**\n * Interface for classes accessing the {@link WikiServlet} class without the\n * need to include the servlet API.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic interface WikiServletContext {\n    /**\n     * Gets the namespace of the wiki.\n     * \n     * @return the namespace\n     */\n    public abstract NamespaceUtils getNamespace();\n\n    /**\n     * Gets the siteinfo of the wiki.\n     * \n     * @return the siteinfo\n     */\n    public abstract SiteInfo getSiteinfo();\n\n    /**\n     * Gets the version of the wiki servlet.\n     * \n     * @return the version\n     */\n    public abstract String getVersion();\n\n    /**\n     * Gets the version of the DB used by the wiki servlet.\n     * \n     * @return the version\n     */\n    public abstract String getDbVersion();\n\n    /**\n     * Gets the version of the Server running the wiki servlet.\n     * \n     * @return the version\n     */\n    public abstract String getServerVersion();\n\n    /**\n     * Gets the version of the bliki rendering library used by the wiki servlet.\n     * \n     * @return the version\n     */\n    public abstract String getBlikiVersion();\n\n    /**\n     * Gets the base URL for links to articles relative to the servlet's context\n     * path.\n     * \n     * @param page\n     *            the page to get the URL for\n     * \n     * @return the linkbaseurl\n     */\n    public abstract String getLinkbaseurl(WikiPageBeanBase page);\n\n    /**\n     * Gets the base URL for links to images relative to the servlet's context\n     * path.\n     * \n     * @param page\n     *            the page to get the URL for\n     * \n     * @return the imagebaseurl\n     */\n    public abstract String getImagebaseurl(WikiPageBeanBase page);\n\n    /**\n     * Adds the given event handler to the list of event handlers.\n     * \n     * @param handler\n     *            the event handler to add\n     */\n    public abstract void registerEventHandler(WikiEventHandler handler);\n    \n    /**\n     * Called at the end of each <tt>jsp</tt> for storing user requests.\n     * \n     * Adds a user request to the user request log if enabled by setting\n     * {@link Options#LOG_USER_REQS} to a value larger than <tt>0</tt>. Also\n     * calls {@link WikiEventHandler#onPageView(WikiPageBeanBase, Object)} for\n     * each registered event handler no matter what\n     * {@link Options#LOG_USER_REQS} is set.\n     * \n     * @param page\n     *            some info on the shown page\n     * @param servertime\n     *            time spend in the web server\n     */\n    public abstract void storeUserReq(WikiPageBeanBase page, long servertime);\n}"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/ExistingPagesCache.java",
    "content": "/**\n *  Copyright 2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport java.security.InvalidParameterException;\nimport java.util.Collection;\nimport java.util.EnumMap;\nimport java.util.HashSet;\nimport java.util.Set;\n\nimport com.skjegstad.utils.BloomFilter;\n\nimport de.zib.scalaris.examples.wikipedia.Options;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyNamespace.NamespaceEnum;\n\n/**\n * Base class for a cache for set of the existing pages. This class provides no\n * cached list - use {@link #createCache(Collection)} to get an actual cache\n * implementation.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class ExistingPagesCache {\n    /**\n     * Empty cache implementation which does not add elements when\n     * {@link #add(NormalisedTitle)} or {@link #addAll(Collection)} are called.\n     */\n    public static ExistingPagesCache NULL_CACHE = new ExistingPagesCache();\n    \n    /**\n     * Constructor\n     */\n    private ExistingPagesCache() {\n    }\n\n    /**\n     * Gets an empty cache implementation.\n     * \n     * @param size\n     *            the size of the cache to create\n     * \n     * @return a pages cache of the given size\n     */\n    public static ExistingPagesCache createCache(int size) {\n        Class<? extends ExistingPagesCache> clazz = Options.getInstance().WIKI_PAGES_CACHE_IMPL;\n        // note: cannot use reflection as the constructors are not public (and should not be)\n        if (clazz.equals(ExistingPagesCacheBloom.class)) {\n            return new ExistingPagesCacheBloom(size);\n        } else if (clazz.equals(ExistingPagesCacheFull.class)) {\n            return new ExistingPagesCacheFull(size);\n        } else {\n            throw new InvalidParameterException(\"unknown pages cache class: \" + clazz.getCanonicalName());\n        }\n    }\n\n    /**\n     * Gets an cache implementation with the given elements.\n     * \n     * @param elements\n     *            the elements to add to the cache\n     * \n     * @return a pages cache\n     */\n    public static ExistingPagesCache createCache(\n            Collection<? extends NormalisedTitle> elements) {\n        Class<? extends ExistingPagesCache> clazz = Options.getInstance().WIKI_PAGES_CACHE_IMPL;\n        // note: cannot use reflection as the constructors are not public (and should not be)\n        if (clazz.equals(ExistingPagesCacheBloom.class)) {\n            return new ExistingPagesCacheBloom(elements);\n        } else if (clazz.equals(ExistingPagesCacheFull.class)) {\n            return new ExistingPagesCacheFull(elements);\n        } else {\n            throw new InvalidParameterException(\"unknown pages cache class: \" + clazz.getCanonicalName());\n        }\n    }\n\n    /**\n     * Add the given page title to the pages cache.\n     * \n     * @param element\n     *            page title to add\n     */\n    public void add(NormalisedTitle element) {\n    }\n\n    /**\n     * Adds all elements from a Collection to the pages cache.\n     * \n     * @param elements\n     *            collection of elements\n     */\n    public void addAll(Collection<? extends NormalisedTitle> elements) {\n    }\n\n    /**\n     * Gets whether the pages cache implementation supports\n     * {@link #contains(NormalisedTitle)}.\n     * \n     * @return support for {@link #contains(NormalisedTitle)}\n     */\n    public boolean hasContains() {\n        return false;\n    }\n    \n    /**\n     * Checks whether the given title is contained in the cache.\n     * \n     * Be sure to check for this capability with {@link #hasContains()}!\n     * \n     * @param element\n     *            the page title to check\n     * \n     * @return whether the title is in the cache or not\n     * \n     * @see #hasContains()\n     */\n    public boolean contains(NormalisedTitle element) {\n        return false;\n    }\n\n    /**\n     * Gets whether the pages cache implementation supports\n     * {@link #getList(NamespaceEnum)}.\n     * \n     * @return support for {@link #getList(NamespaceEnum)}\n     */\n    public boolean hasFullList() {\n        return false;\n    }\n    \n    /**\n     * Gets the page titles in the given namespace.\n     * \n     * Be sure to check for this capability with {@link #hasFullList()}!\n     * \n     * @param ns\n     *            the namespace to get page titles for\n     * \n     * @return a set of page titles\n     * \n     * @see #hasFullList()\n     */\n    public Set<NormalisedTitle> getList(NamespaceEnum ns) {\n        return new HashSet<NormalisedTitle>(0);\n    }\n\n    /**\n     * Existing pages cache using bloom filters.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class ExistingPagesCacheBloom extends ExistingPagesCache {\n        /**\n         * False positive rate of the bloom filter for the existing pages checks.\n         */\n        protected static final double existingPagesFPR = 0.1;\n\n        protected final BloomFilter<NormalisedTitle> bloom;\n\n        protected ExistingPagesCacheBloom(int size) {\n            this.bloom = new BloomFilter<NormalisedTitle>(existingPagesFPR, size);\n        }\n\n        protected ExistingPagesCacheBloom(\n                Collection<? extends NormalisedTitle> elements) {\n            this(Math.max(100, elements.size() + Math.max(10, elements.size() / 10)));\n            bloom.addAll(elements);\n        }\n\n        @Override\n        public void add(NormalisedTitle element) {\n            bloom.add(element);\n        }\n\n        @Override\n        public void addAll(Collection<? extends NormalisedTitle> elements) {\n            bloom.addAll(elements);\n        }\n\n        @Override\n        public boolean hasContains() {\n            return true;\n        }\n        \n        @Override\n        public boolean contains(NormalisedTitle element) {\n            return bloom.contains(element);\n        }\n    }\n\n    /**\n     * Existing pages cache using a hash set to cache the full list.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class ExistingPagesCacheFull extends ExistingPagesCache {\n        protected final EnumMap<NamespaceEnum, Set<NormalisedTitle>> cache = new EnumMap<NamespaceEnum, Set<NormalisedTitle>>(\n                NamespaceEnum.class);\n\n        protected ExistingPagesCacheFull(int size) {\n            for (NamespaceEnum ns : NamespaceEnum.values()) {\n                cache.put(ns, new HashSet<NormalisedTitle>());\n            }\n        }\n\n        protected ExistingPagesCacheFull(\n                Collection<? extends NormalisedTitle> elements) {\n            this(elements.size());\n            addAll(elements);\n        }\n\n        @Override\n        public void add(NormalisedTitle element) {\n            cache.get(NamespaceEnum.fromId(element.namespace)).add(element);\n        }\n\n        @Override\n        public void addAll(Collection<? extends NormalisedTitle> elements) {\n            for (NormalisedTitle element : elements) {\n                add(element);\n            }\n        }\n\n        @Override\n        public boolean hasContains() {\n            return true;\n        }\n        \n        @Override\n        public boolean contains(NormalisedTitle element) {\n            return cache.get(NamespaceEnum.fromId(element.namespace)).contains(element);\n        }\n\n        @Override\n        public boolean hasFullList() {\n            return true;\n        }\n\n        @Override\n        public Set<NormalisedTitle> getList(NamespaceEnum ns) {\n            return cache.get(ns);\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MyConfiguration.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport info.bliki.wiki.model.Configuration;\nimport info.bliki.wiki.template.ITemplateFunction;\n\nimport java.net.URL;\nimport java.util.Map;\nimport java.util.TreeMap;\n\n/**\n * Wiki configuration with its own interwiki and template maps.\n * \n * Note: {@link Configuration} uses a static interwiki and template function\n * maps valid for all instances.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MyConfiguration extends Configuration {\n    /**\n     * Map from the interwiki shortcut to the real Interwiki-URL\n     */\n    protected final Map<String, String> INTERWIKI_MAP = new TreeMap<String, String>(String.CASE_INSENSITIVE_ORDER);\n\n    /**\n     * Map the template's function name to the TemplateFunction implementation\n     */\n    protected final Map<String, ITemplateFunction> TEMPLATE_FUNCTION_MAP = new TreeMap<String, ITemplateFunction>(\n            String.CASE_INSENSITIVE_ORDER);\n\n    /**\n     * Creates a wiki configuration with its own interwiki map.\n     */\n    public MyConfiguration() {\n        INTERWIKI_MAP.putAll(Configuration.INTERWIKI_MAP);\n        TEMPLATE_FUNCTION_MAP.putAll(Configuration.TEMPLATE_FUNCTION_MAP);\n    }\n\n    /**\n     * Creates a wiki configuration with its own interwiki map. Removes the\n     * interwiki mapping to the base of this wiki.\n     * \n     * @param namespace\n     *            the namespace to use\n     */\n    public MyConfiguration(MyNamespace namespace) {\n        INTERWIKI_MAP.putAll(Configuration.INTERWIKI_MAP);\n        try {\n            /*\n             * simple.wikipedia.org\n             * simple.wiktionary.org\n             */\n            final String base = namespace.getSiteinfo().getBase();\n            if (!base.isEmpty()) {\n                String baseUrl = new URL(base).getHost();\n                String domain = baseUrl.split(\"\\\\.\")[1];\n                INTERWIKI_MAP.remove(domain);\n            }\n        } catch (Exception e) {\n            // ignore if the URL is not valid\n            e.printStackTrace();\n        }\n        TEMPLATE_FUNCTION_MAP.putAll(Configuration.TEMPLATE_FUNCTION_MAP);\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.model.Configuration#getInterwikiMap()\n     */\n    @Override\n    public Map<String, String> getInterwikiMap() {\n        return INTERWIKI_MAP;\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.model.Configuration#addInterwikiLink(java.lang.String, java.lang.String)\n     */\n    @Override\n    public String addInterwikiLink(String key, String value) {\n        return INTERWIKI_MAP.put(key, value);\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.model.Configuration#addTemplateFunction(java.lang.String, info.bliki.wiki.template.ITemplateFunction)\n     */\n    @Override\n    public ITemplateFunction addTemplateFunction(String key,\n            ITemplateFunction value) {\n        return TEMPLATE_FUNCTION_MAP.put(key, value);\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.model.Configuration#getTemplateMap()\n     */\n    @Override\n    public Map<String, ITemplateFunction> getTemplateMap() {\n        return TEMPLATE_FUNCTION_MAP;\n    }\n\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MyFullurl.java",
    "content": "// adapted from info.bliki.wiki.template.Fullurl which is licensed under\n// Eclipse Public License 1.0\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport info.bliki.wiki.model.IWikiModel;\nimport info.bliki.wiki.template.Fullurl;\n\nimport java.io.UnsupportedEncodingException;\nimport java.util.List;\n\n/**\n * A template parser function for <code>{{fullurl: ... }}</code> syntax.\n * Note: Falls back to {@link Fullurl} if the model is not a {@link MyWikiModel}\n * model.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MyFullurl extends Fullurl {\n    /**\n     * Single static instance of a {@link MyFullurl} object.\n     */\n    public final static MyFullurl CONST = new MyFullurl();\n    \n    /**\n     * Constructor\n     */\n    public MyFullurl() {\n        super();\n    }\n\n    @Override\n    public String parseFunction(List<String> list, IWikiModel model,\n            char[] src, int beginIndex, int endIndex, boolean isSubst)\n            throws UnsupportedEncodingException {\n        final String server = MyMagicWord.processMagicWord(\"{{SERVER}}\", \"\", model, false);\n        final String localurl = MyLocalurl.CONST.parseFunction(list, model,\n                src, beginIndex, endIndex, isSubst);\n        return server + localurl;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MyIfexistTemplateFun.java",
    "content": "/**\n *  Copyright 2012-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport info.bliki.wiki.model.IWikiModel;\nimport info.bliki.wiki.template.ITemplateFunction;\nimport info.bliki.wiki.template.Ifexist;\n\nimport java.util.List;\n\n/**\n * A template parser function for <code>{{ #ifexist: ... }}</code> syntax. See\n * <a href=\"http://www.mediawiki.org/wiki/Help:Extension:ParserFunctions\">\n * Mediwiki's Help:Extension:ParserFunctions</a>\n * \n * Based on {@link Ifexist} but every title in the Media, Image or File\n * namespace is assumed to exist.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MyIfexistTemplateFun extends Ifexist {\n    /**\n     * Static instance of this template function parser.\n     */\n    public final static ITemplateFunction CONST = new MyIfexistTemplateFun();\n\n    /**\n     * Constructor.\n     */\n    public MyIfexistTemplateFun() {\n        super();\n    }\n\n    @Override\n    public String parseFunction(List<String> list, IWikiModel model,\n            char[] src, int beginIndex, int endIndex, boolean isSubst) {\n        if (model instanceof MyWikiModel) {\n            MyWikiModel myModel = (MyWikiModel) model;\n            if (list.size() > 1) {\n                final String[] wikiTopicName = myModel.splitNsTitle(isSubst ? list.get(0) : parseTrim(list.get(0), model));\n                if (myModel.isImageNamespace(wikiTopicName[0])\n                        || myModel.isMediaNamespace(wikiTopicName[0])\n                        || model.getRawWikiContent(wikiTopicName[0], wikiTopicName[1], null) != null) {\n                    return isSubst ? list.get(1) : parseTrim(list.get(1), model);\n                } else { // non-existing page\n                    if (list.size() >= 3) {\n                        return isSubst ? list.get(2) : parseTrim(list.get(2), model);\n                    }\n                }\n            }\n            return null;\n        } else {\n            return super.parseFunction(list, model, src, beginIndex, endIndex, isSubst);\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MyLocalurl.java",
    "content": "// adapted from info.bliki.wiki.template.Localurl which is licensed under\n// Eclipse Public License 1.0\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport info.bliki.api.Connector;\nimport info.bliki.wiki.model.IWikiModel;\nimport info.bliki.wiki.template.Localurl;\n\nimport java.io.UnsupportedEncodingException;\nimport java.net.URLEncoder;\nimport java.util.List;\n\n/**\n * A template parser function for <code>{{localurl: ... }}</code> syntax\n * Note: Falls back to {@link Localurl} if the model is not a {@link MyWikiModel}\n * model.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MyLocalurl extends Localurl {\n    /**\n     * Single static instance of a {@link MyFullurl} object.\n     */\n    public final static MyLocalurl CONST = new MyLocalurl();\n\n    /**\n     * Constructor\n     */\n    public MyLocalurl() {\n        super();\n    }\n    \n    @Override\n    public String parseFunction(List<String> list, IWikiModel model,\n            char[] src, int beginIndex, int endIndex, boolean isSubst)\n            throws UnsupportedEncodingException {\n        if (model instanceof MyWikiModel) {\n            MyWikiModel myModel = (MyWikiModel) model;\n            if (list.size() > 0) {\n                String arg0 = isSubst ? list.get(0) : parseTrim(list.get(0), model);\n                final String title = URLEncoder.encode(Character.toUpperCase(arg0.charAt(0)) + \"\", Connector.UTF8_CHARSET)\n                        + URLEncoder.encode(arg0.substring(1), Connector.UTF8_CHARSET);\n                if (arg0.length() > 0 && list.size() == 1) {\n                    String result = myModel.getWikiBaseURL().replace(\n                            \"${title}\", title);\n                    return result;\n                }\n                StringBuilder builder = new StringBuilder(arg0.length() + 32);\n                builder.append(myModel.getWikiBaseURL().replace(\"${title}\",\n                        title));\n                for (int i = 1; i < list.size(); i++) {\n                    builder.append(\"&\");\n                    builder.append(isSubst ? list.get(i) : parseTrim(list.get(i), model));\n                }\n                return builder.toString();\n            }\n            return null;\n        } else {\n            return super.parseFunction(list, model, src, beginIndex, endIndex, isSubst);\n        }\n    }\n\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MyMagicWord.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport info.bliki.wiki.filter.MagicWord;\n\nimport java.util.HashSet;\n\nimport de.zib.scalaris.examples.wikipedia.Options;\n\n/**\n * Gets values for magic words not handled by {@link MagicWord}.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MyMagicWord extends MagicWord {\n\n    private static HashSet<String> MY_MAGIC_WORDS = new HashSet<String>(100);\n\n    // private HashMap parameterValues = new HashMap();\n\n    static {\n        // statistics\n        MY_MAGIC_WORDS.add(MAGIC_CURRENT_VERSION);\n        // page values\n        MY_MAGIC_WORDS.add(MAGIC_PAGE_SIZE);\n        MY_MAGIC_WORDS.add(MAGIC_SITE_NAME);\n        MY_MAGIC_WORDS.add(MAGIC_SERVER);\n        MY_MAGIC_WORDS.add(MAGIC_SCRIPT_PATH);\n        MY_MAGIC_WORDS.add(MAGIC_SERVER_NAME);\n        MY_MAGIC_WORDS.add(MAGIC_DISPLAY_TITLE);\n        MY_MAGIC_WORDS.add(MAGIC_DEFAULT_SORT);\n    }\n\n    /**\n     * Determines if a template name corresponds to a magic word (does not\n     * recognise magic words with parameters, e.g. <tt>TALKPAGENAME:Title</tt> -\n     * use {@link #extractMagicWordPart(String)} for a full title and check for\n     * magic word with its result).\n     * \n     * @param name\n     *            the potential magic word\n     * @return <tt>true</tt> if <tt>name</tt> was a magic word, <tt>false</tt>\n     *         otherwise\n     */\n    public static boolean isMagicWord(String name) {\n        return MagicWord.isMagicWord(name) || isMyMagicWord(name);\n    }\n\n    /**\n     * Extract the potential magic word part from a given title name.\n     * \n     * @param title\n     *            the template's name without the namespace, e.g. a magic word\n     *            including its parameters\n     * \n     * @return the magic word or another string\n     *\n     * @see MyWikiModel#processMagicWord(String)\n     * @see #isMagicWord(String)\n     */\n    public static String extractMagicWordPart(String title) {\n        int index = title.indexOf(':');\n        String magicWord = title;\n        if (index > 0) {\n            // if it is a magic word, the first part is the word itself, the second its parameters\n            magicWord = title.substring(0, index);\n        }\n        return magicWord;\n    }\n\n    /**\n     * Determines if a template name corresponds to a magic word that is handled\n     * by this class instead of {@link MagicWord}.\n     * \n     * @param name\n     *            the template name\n     * \n     * @return whether this class should be favoured over {@link MagicWord} for\n     *         parsing this template\n     */\n    public static boolean isMyMagicWord(String name) {\n        return MY_MAGIC_WORDS.contains(name);\n    }\n\n    /**\n     * Process a magic word, returning the value corresponding to the magic\n     * word.\n     * \n     * @param name\n     *            the template name, i.e. a magic word\n     * @param parameter\n     *            the template parameters\n     * @param model\n     *            the currently used model\n     * @param hasParameter\n     *            whether a parameter was given or not (cannot distinguish from\n     *            <tt>parameter</tt> value alone)\n     * \n     * @return the value of the magic word\n     * \n     * @see <a\n     *      href=\"http://meta.wikimedia.org/wiki/Help:Magic_words\">http://meta.wikimedia.org/wiki/Help:Magic_words</a>\n     */\n    public static String processMagicWord(String name, String parameter, MyWikiModel model, boolean hasParameter) {\n        if (!isMyMagicWord(name)) {\n            return MagicWord.processMagicWord(name, parameter, model, hasParameter);\n        }\n        \n        // check whether numbers should be printed in raw format and\n        // remove this tag from the parameter string:\n        boolean rawNumber = false;\n        if (parameter.equals(\"R\")) {\n            parameter = \"\";\n            rawNumber = true;\n        } else if (parameter.endsWith(\"|R\")) {\n            parameter = parameter.substring(0, parameter.length() - 2);\n            rawNumber = true;\n        }\n        \n        /*\n         * Technical metadata / site\n         */\n        if (name.equals(MAGIC_SITE_NAME)) {\n            return model.getNamespace().getSiteinfo().getSitename();\n        } else if (name.equals(MAGIC_SERVER)) {\n            // {{SERVER}} (//en.wikipedia.org)\n            return \"//\" + Options.getInstance().SERVERNAME;\n        } else if (name.equals(MAGIC_SERVER_NAME)) {\n            // {{SERVERNAME}} (en.wikipedia.org)\n            return Options.getInstance().SERVERNAME;\n//            {{DIRMARK}}\n//            {{DIRECTIONMARK}}\n//        } else if (name.equals(MAGIC_SCRIPT_PATH)) {\n//            // TODO: implement\n//            return null;\n//            {{STYLEPATH}}\n        } else if (name.equals(MAGIC_CURRENT_VERSION)) {\n            return WikiServlet.version;\n//            {{CONTENTLANGUAGE}}\n//            {{CONTENTLANG}}\n//          {{PROTECTIONLEVEL:action}}\n        /*\n         * Technical metadata / Affects page content / Behavior switches\n         */\n        } else if (name.equals(MAGIC_DISPLAY_TITLE)) {\n            return \"\";\n        } else if (name.equals(MAGIC_DEFAULT_SORT)) {\n            return \"\";\n//            {{DEFAULTSORTKEY:sortkey}}\n//            {{DEFAULTCATEGORYSORT:sortkey}}\n\n        /*\n         * Technical metadata / Latest revision to current page\n         */\n        } else if (name.equals(MAGIC_PAGE_SIZE)) {\n            final String revText = model.retrievePage(parameter, null, false);\n            int size = 0;\n            if (revText != null) {\n                size = revText.getBytes().length;\n            }\n            return model.formatStatisticNumber(rawNumber, size);\n        }\n        \n        return name;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MyNamespace.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport info.bliki.Messages;\nimport info.bliki.wiki.namespaces.Namespace;\n\nimport java.util.ListResourceBundle;\nimport java.util.Locale;\nimport java.util.Map;\nimport java.util.ResourceBundle;\n\nimport de.zib.scalaris.examples.wikipedia.NamespaceUtils;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\n\n/**\n * Namespace implementation using a {@link SiteInfo} backend.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MyNamespace extends Namespace implements NamespaceUtils {\n    private SiteInfo siteinfo;\n    \n    /**\n     * Provides all namespace values as an enum type.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static enum NamespaceEnum {\n        /**\n         * Alias for direct links to media files.\n         */\n        MEDIA_NAMESPACE_KEY(MyNamespace.MEDIA_NAMESPACE_KEY),\n        /**\n         * Holds special pages.\n         */\n        SPECIAL_NAMESPACE_KEY(MyNamespace.SPECIAL_NAMESPACE_KEY),\n        /**\n         * \"Real\" content; articles. Has no prefix.\n         */\n        MAIN_NAMESPACE_KEY(MyNamespace.MAIN_NAMESPACE_KEY),\n        /**\n         * Talk pages of \"Real\" content\n         */\n        TALK_NAMESPACE_KEY(MyNamespace.TALK_NAMESPACE_KEY),\n        /**\n         * \n         */\n        USER_NAMESPACE_KEY(MyNamespace.USER_NAMESPACE_KEY),\n        /**\n         * Talk pages for User Pages\n         */\n        USER_TALK_NAMESPACE_KEY(MyNamespace.USER_TALK_NAMESPACE_KEY),\n        /**\n         * Information about the wiki. Prefix is the same as $wgSitename of the PHP\n         * installation.\n         */\n        PROJECT_NAMESPACE_KEY(MyNamespace.PROJECT_NAMESPACE_KEY),\n        /**\n         * \n         */\n        PROJECT_TALK_NAMESPACE_KEY(MyNamespace.PROJECT_TALK_NAMESPACE_KEY),\n        /**\n         * Media description pages.\n         */\n        FILE_NAMESPACE_KEY(MyNamespace.FILE_NAMESPACE_KEY),\n        /**\n         * \n         */\n        FILE_TALK_NAMESPACE_KEY(MyNamespace.FILE_TALK_NAMESPACE_KEY),\n        /**\n         * Site interface customisation. Protected.\n         */\n        MEDIAWIKI_NAMESPACE_KEY(MyNamespace.MEDIAWIKI_NAMESPACE_KEY),\n        /**\n         * \n         */\n        MEDIAWIKI_TALK_NAMESPACE_KEY(MyNamespace.MEDIAWIKI_TALK_NAMESPACE_KEY),\n        /**\n         * Template pages.\n         */\n        TEMPLATE_NAMESPACE_KEY(MyNamespace.TEMPLATE_NAMESPACE_KEY),\n        /**\n         * \n         */\n        TEMPLATE_TALK_NAMESPACE_KEY(MyNamespace.TEMPLATE_TALK_NAMESPACE_KEY),\n        /**\n         * Help pages.\n         */\n        HELP_NAMESPACE_KEY(MyNamespace.HELP_NAMESPACE_KEY),\n        /**\n         * \n         */\n        HELP_TALK_NAMESPACE_KEY(MyNamespace.HELP_TALK_NAMESPACE_KEY),\n        /**\n         * Category description pages.\n         */\n        CATEGORY_NAMESPACE_KEY(MyNamespace.CATEGORY_NAMESPACE_KEY),\n        /**\n         * Talk pages for portal pages.\n         */\n        CATEGORY_TALK_NAMESPACE_KEY(MyNamespace.CATEGORY_TALK_NAMESPACE_KEY),\n        /**\n         * Portal pages.\n         */\n        PORTAL_NAMESPACE_KEY(MyNamespace.PORTAL_NAMESPACE_KEY),\n        /**\n         * \n         */\n        PORTAL_TALK_NAMESPACE_KEY(MyNamespace.PORTAL_TALK_NAMESPACE_KEY);\n        \n        private final int id;\n        NamespaceEnum(int id) {\n            this.id = id;\n        }\n        \n        /**\n         * Returns the namespace's ID.\n         * \n         * @return the ID of the namespace\n         */\n        public int getId() {\n            return id;\n        }\n\n        /**\n         * Tries to convert a namespace ID to the according enum value.\n         * \n         * @param id  the ID to convert\n         * \n         * @return the according enum value\n         */\n        public static NamespaceEnum fromId(int id) {\n            // manual 'switch' is less elegant but faster than going through all\n            // values and comparing the IDs\n            switch (id) {\n                case -2: return MEDIA_NAMESPACE_KEY;\n                case -1: return SPECIAL_NAMESPACE_KEY;\n                case 0:  return MAIN_NAMESPACE_KEY;\n                case 1:  return TALK_NAMESPACE_KEY;\n                case 2:  return USER_NAMESPACE_KEY;\n                case 3:  return USER_TALK_NAMESPACE_KEY;\n                case 4:  return PROJECT_NAMESPACE_KEY;\n                case 5:  return PROJECT_TALK_NAMESPACE_KEY;\n                case 6:  return FILE_NAMESPACE_KEY;\n                case 7:  return FILE_TALK_NAMESPACE_KEY;\n                case 8:  return MEDIAWIKI_NAMESPACE_KEY;\n                case 9:  return MEDIAWIKI_TALK_NAMESPACE_KEY;\n                case 10: return TEMPLATE_NAMESPACE_KEY;\n                case 11: return TEMPLATE_TALK_NAMESPACE_KEY;\n                case 12: return HELP_NAMESPACE_KEY;\n                case 13: return HELP_TALK_NAMESPACE_KEY;\n                case 14: return CATEGORY_NAMESPACE_KEY;\n                case 15: return CATEGORY_TALK_NAMESPACE_KEY;\n                case 100: return PORTAL_NAMESPACE_KEY;\n                case 101: return PORTAL_TALK_NAMESPACE_KEY;\n                default: throw new IllegalArgumentException(\n                        \"No constant with id \" + id + \" found\");\n            }\n        }\n    }\n    \n    /**\n     * Resource bundle derived from a given {@link SiteInfo} object.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class MyResourceBundle extends ListResourceBundle {\n        protected SiteInfo siteinfo;\n        protected ResourceBundle langResBundle = null;\n        protected ResourceBundle enBundle;\n        \n        /**\n         * Creates a new bundle using the namespace contents from the given\n         * siteinfo.\n         * \n         * @param siteinfo\n         *            the siteinfo\n         */\n        public MyResourceBundle(SiteInfo siteinfo) {\n            super();\n            this.siteinfo = siteinfo;\n            Locale locale = siteinfo.extractLolace();\n            if (locale != null) {\n                langResBundle = Messages.getResourceBundle(locale);\n            }\n            enBundle = Messages.getResourceBundle(Locale.ENGLISH);\n        }\n        \n        @Override\n        protected Object[][] getContents() {\n            /*\n             * from: http://www.mediawiki.org/wiki/Manual:Namespace#Built-in_namespaces\n             * Index    Name    Purpose     Comments\n             * 0   Main    \"Real\" content; articles    Has no prefix\n             * 1   Talk    Talk pages of \"Real\" content    \n             * 2   User    User pages\n             * 3   User talk   Talk pages for User Pages   \n             * 4   Project     Information about the wiki  Prefix is the same as $wgSitename[1]\n             * 5   Project talk    \n             * 6   File    Media description pages\n             * 7   File talk   \n             * 8   MediaWiki   Site interface customisation    Protected\n             * 9   MediaWiki talk  \n             * 10  Template    Template pages\n             * 11  Template talk   \n             * 12  Help    Help pages\n             * 13  Help talk   \n             * 14  Category    Category description pages\n             * 15  Category talk\n             * 100 Portal\n             * 101 Portal talk\n             * -1  Special     Holds special pages\n             * -2  Media   Alias for direct links to media files\n             */\n            return new Object[][] {\n                    {Messages.WIKI_API_MEDIA1, getNsPref(Namespace.MEDIA_NAMESPACE_KEY, Messages.WIKI_API_MEDIA1)},\n                    {Messages.WIKI_API_SPECIAL1, getNsPref(Namespace.SPECIAL_NAMESPACE_KEY, Messages.WIKI_API_SPECIAL1)},\n                    // Main - nothing to add for this\n                    {Messages.WIKI_API_TALK1, getNsPref(Namespace.TALK_NAMESPACE_KEY, Messages.WIKI_API_TALK1)},\n                    {Messages.WIKI_API_USER1, getNsPref(Namespace.USER_NAMESPACE_KEY, Messages.WIKI_API_USER1)},\n                    {Messages.WIKI_API_USERTALK1, getNsPref(Namespace.USER_TALK_NAMESPACE_KEY, Messages.WIKI_API_USERTALK1)},\n                    {Messages.WIKI_API_META1, getNsPref(Namespace.PROJECT_NAMESPACE_KEY, Messages.WIKI_API_META1)},\n                    {Messages.WIKI_API_METATALK1, getNsPref(Namespace.PROJECT_TALK_NAMESPACE_KEY, Messages.WIKI_API_METATALK1)},\n                    {Messages.WIKI_API_IMAGE1, getNsPref(Namespace.FILE_NAMESPACE_KEY, Messages.WIKI_API_IMAGE1)},\n                    {Messages.WIKI_API_IMAGETALK1, getNsPref(Namespace.FILE_TALK_NAMESPACE_KEY, Messages.WIKI_API_IMAGETALK1)},\n                    {Messages.WIKI_API_MEDIAWIKI1, getNsPref(Namespace.MEDIAWIKI_NAMESPACE_KEY, Messages.WIKI_API_MEDIAWIKI1)},\n                    {Messages.WIKI_API_MEDIAWIKITALK1, getNsPref(Namespace.MEDIAWIKI_TALK_NAMESPACE_KEY, Messages.WIKI_API_MEDIAWIKITALK1)},\n                    {Messages.WIKI_API_TEMPLATE1, getNsPref(Namespace.TEMPLATE_NAMESPACE_KEY, Messages.WIKI_API_TEMPLATE1)},\n                    {Messages.WIKI_API_TEMPLATETALK1, getNsPref(Namespace.TEMPLATE_TALK_NAMESPACE_KEY, Messages.WIKI_API_TEMPLATETALK1)},\n                    {Messages.WIKI_API_HELP1, getNsPref(Namespace.HELP_NAMESPACE_KEY, Messages.WIKI_API_HELP1)},\n                    {Messages.WIKI_API_HELPTALK1, getNsPref(Namespace.HELP_TALK_NAMESPACE_KEY, Messages.WIKI_API_HELPTALK1)},\n                    {Messages.WIKI_API_CATEGORY1, getNsPref(Namespace.CATEGORY_NAMESPACE_KEY, Messages.WIKI_API_CATEGORY1)},\n                    {Messages.WIKI_API_CATEGORYTALK1, getNsPref(Namespace.CATEGORY_TALK_NAMESPACE_KEY, Messages.WIKI_API_CATEGORYTALK1)},\n                    {Messages.WIKI_API_PORTAL1, getNsPref(Namespace.PORTAL_NAMESPACE_KEY, Messages.WIKI_API_PORTAL1)},\n                    {Messages.WIKI_API_PORTALTALK1, getNsPref(Namespace.PORTAL_TALK_NAMESPACE_KEY, Messages.WIKI_API_PORTALTALK1)},\n                    {Messages.WIKI_TAGS_TOC_CONTENT, langResBundle == null ? \"Contents\" : Messages.getString(langResBundle, Messages.WIKI_TAGS_TOC_CONTENT, \"Contents\")}\n            };\n        }\n\n        /**\n         * Gets the namespace prefix of the given key.\n         * \n         * @param key\n         *            the key of the namespace to retrieve\n         * @param msgkey\n         *            the key used by {@link Messages}\n         * @return namespace name\n         */\n        private String getNsPref(Integer key, String msgkey) {\n            final Map<String, String> nsMap = siteinfo.getNamespaces().get(key.toString());\n            if (nsMap != null) {\n                return nsMap.get(SiteInfo.NAMESPACE_PREFIX);\n            }\n            // note: the result MUST NOT BE NULL!\n            // otherwise Messages#getString() will return <tt>\"!\" + key + \"!\"</tt>\n            String fallBackName = null;\n            if (langResBundle != null) {\n                fallBackName = Messages.getString(langResBundle, msgkey, null);\n            }\n            if (fallBackName == null) {\n                fallBackName = Messages.getString(enBundle, msgkey, null);\n            }\n            return fallBackName;\n        }\n    }\n\n    /**\n     * Creates a default namespace as in {@link Namespace#Namespace()}.\n     */\n    public MyNamespace() {\n        super();\n    }\n    \n    /**\n     * Creates a wikipedia namespace based on the given site info.\n     * \n     * @param siteinfo\n     *            the site info\n     * \n     */\n    public MyNamespace(SiteInfo siteinfo) {\n        super(new MyResourceBundle(siteinfo));\n        this.siteinfo = siteinfo;\n        Locale locale = siteinfo.extractLolace();\n        if (locale != null) {\n            initializeAliases(Messages.getResourceBundle(locale));\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.bliki.NamespaceUtils#getSiteinfo()\n     */\n    @Override\n    public SiteInfo getSiteinfo() {\n        return siteinfo;\n    }\n    \n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.bliki.NamespaceUtils#getTalkPageFromPageName(java.lang.String)\n     */\n    @Override\n    public String getTalkPageFromPageName(String pageName) {\n        String[] pnSplit = splitNsTitle(pageName);\n        String talkspace = getTalkspace(pnSplit[0]);\n        if (talkspace == null) {\n            return pnSplit[1];\n        } else {\n            return talkspace + \":\" + pnSplit[1];\n        }\n    }\n    \n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.bliki.NamespaceUtils#getPageNameFromTalkPage(java.lang.String)\n     */\n    @Override\n    public String getPageNameFromTalkPage(String talkPageName) {\n        String[] pnSplit = splitNsTitle(talkPageName);\n        String namespace = pnSplit[0];\n        String talkspace = getTalkspace(namespace);\n        if (talkspace == null || !namespace.equals(talkspace)) {\n            return talkPageName;\n        } else {\n            return pnSplit[1];\n        }\n    }\n    \n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.bliki.NamespaceUtils#isTalkPage(java.lang.String)\n     */\n    @Override\n    public boolean isTalkPage(String pageName) {\n        String namespace = splitNsTitle(pageName)[0];\n        String talkspace = getTalkspace(namespace);\n        if (talkspace == null || !namespace.equals(talkspace)) {\n            return false;\n        } else {\n            return true;\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MyParsingIfTemplateFun.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport info.bliki.wiki.model.IWikiModel;\nimport info.bliki.wiki.template.AbstractTemplateFunction;\nimport info.bliki.wiki.template.ITemplateFunction;\nimport info.bliki.wiki.template.If;\n\nimport java.util.List;\n\n/**\n * A template parser function for <code>{{ #if: ... }}</code> syntax.\n * Also works for <tt>#iferror</tt>, <tt>#ifeq</tt>, <tt>#ifexist</tt>,\n * <tt>#ifexpr</tt>.\n * \n * Based on {@link If} but does not evaluate the condition. This is useful\n * during filtering since the condition needs to be evaluated at runtime which\n * may yield another result.\n */\npublic class MyParsingIfTemplateFun extends AbstractTemplateFunction {\n    /**\n     * Static instance of this template function parser.\n     */\n    public final static ITemplateFunction CONST = new MyParsingIfTemplateFun();\n\n    /**\n     * Constructor.\n     */\n    public MyParsingIfTemplateFun() {\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.template.AbstractTemplateFunction#parseFunction(java.util.List, info.bliki.wiki.model.IWikiModel, char[], int, int)\n     */\n    @Override\n    public String parseFunction(List<String> list, IWikiModel model,\n            char[] src, int beginIndex, int endIndex, boolean isSubst) {\n        if (list.size() > 1) {\n            if (!isSubst) {\n                parseTrim(list.get(0), model);\n            }\n            // parse both parameters\n            String result = isSubst ? list.get(1) : parseTrim(list.get(1), model);\n            if (list.size() >= 3) {\n                result += isSubst ? list.get(2) : parseTrim(list.get(2), model);\n            }\n            return result;\n        }\n        return null;\n    }\n\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MyParsingSwitchTemplateFun.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport info.bliki.wiki.model.IWikiModel;\nimport info.bliki.wiki.template.AbstractTemplateFunction;\nimport info.bliki.wiki.template.ITemplateFunction;\nimport info.bliki.wiki.template.Switch;\n\nimport java.util.List;\n\n/**\n * A template parser function for <code>{{ #switch: ... }}</code> syntax.\n * \n * Based on {@link Switch} but does not evaluate the condition. This is useful\n * during filtering since the condition needs to be evaluated at runtime which\n * may yield another result.\n */\npublic class MyParsingSwitchTemplateFun extends AbstractTemplateFunction {\n    /**\n     * Static instance of this template function parser.\n     */\n    public final static ITemplateFunction CONST = new MyParsingSwitchTemplateFun();\n\n    /**\n     * Constructor.\n     */\n    public MyParsingSwitchTemplateFun() {\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.template.AbstractTemplateFunction#parseFunction(java.util.List, info.bliki.wiki.model.IWikiModel, char[], int, int)\n     */\n    @Override\n    public String parseFunction(List<String> list, IWikiModel model,\n            char[] src, int beginIndex, int endIndex, boolean isSubst) {\n        if (list.size() > 2) {\n            if (!isSubst) {\n                parseTrim(list.get(0), model);\n            }\n            StringBuilder result = new StringBuilder();\n            for (int i = 1; i < list.size(); i++) {\n                String temp = isSubst ? list.get(i) : parseTrim(list.get(i), model);\n                int index = temp.indexOf('=');\n                String leftHandSide;\n                if (index >= 0) {\n                    result.append(temp.substring(index + 1).trim());\n                    leftHandSide = temp.substring(0, index).trim();\n                } else {\n                    leftHandSide = temp.trim();\n                }\n                String parsedLHS = isSubst ? leftHandSide.trim() : parseTrim(leftHandSide, model);\n                if (index < 0 && i == list.size() - 1) {\n                    result.append(parsedLHS);\n                }\n            }\n            return result.toString();\n        }\n        return null;\n    }\n\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MyParsingWikiModel.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport java.util.HashSet;\nimport java.util.Map;\n\n/**\n * Wiki model which should be used during parsing of xml dumps.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MyParsingWikiModel extends MyWikiModel {\n\n    /**\n     * Creates a new wiki model to render wiki text.\n     * \n     * @param imageBaseURL\n     *            base url pointing to images - can contain ${image} for\n     *            replacement\n     * @param linkBaseURL\n     *            base url pointing to links - can contain ${title} for\n     *            replacement\n     * @param namespace\n     *            namespace of the wiki\n     */\n    public MyParsingWikiModel(String imageBaseURL, String linkBaseURL,\n            MyNamespace namespace) {\n        super(imageBaseURL, linkBaseURL, namespace);\n        addTemplateFunction(\"#if\", MyParsingIfTemplateFun.CONST);\n        addTemplateFunction(\"#iferror\", MyParsingIfTemplateFun.CONST);\n        addTemplateFunction(\"#ifeq\", MyParsingIfTemplateFun.CONST);\n        addTemplateFunction(\"#ifexist\", MyParsingIfTemplateFun.CONST);\n        addTemplateFunction(\"#ifexpr\", MyParsingIfTemplateFun.CONST);\n        addTemplateFunction(\"#switch\", MyParsingSwitchTemplateFun.CONST);\n    }\n\n    /**\n     * Creates a stub template content that has as many parameters as given by\n     * the template call. This allows parsing of template contents hidden in\n     * parameters.\n     * \n     * @param name\n     *            the template's name without the namespace\n     * @param parameter\n     *            the parameters of the template\n     * @param followRedirect\n     *            whether to follow a redirect or not (ignored)\n     * \n     * @return the template's contents\n     */\n    @Override\n    protected String retrievePage(String namespace, String articleName,\n            Map<String, String> templateParameters, boolean followRedirect) {\n        if (isTemplateNamespace(namespace)) {\n            int index = articleName.indexOf(':');\n            if (index > 0) {\n                String magicWord = articleName.substring(0, index);\n                String parameter = articleName.substring(index + 1).trim();\n                if (magicWord.equals(MyScalarisMagicWord.MAGIC_PAGES_IN_CATEGORY)\n                        || magicWord.equals(MyScalarisMagicWord.MAGIC_PAGES_IN_CAT)) {\n//                  {{PAGESINCATEGORY:categoryname}}\n//                  {{PAGESINCAT:categoryname}}\n                    // -> also add as an include\n                    addInclude(createFullPageName(getCategoryNamespace(), parameter));\n                    return \"\";\n                }\n            }\n            if (templateParameters != null) {\n                StringBuilder result = new StringBuilder(8 * templateParameters.size());\n                for (int i = 1; i <= templateParameters.size(); ++i) {\n                    result.append(\"{{{\");\n                    result.append(i);\n                    result.append(\"}}}\\n\");\n                }\n                return result.toString();\n            }\n        }\n        return null;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.bliki.MyWikiModel#setUp()\n     */\n    @Override\n    public void setUp() {\n        super.setUp();\n        includes = new HashSet<String>();\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.bliki.MyWikiModel#appendRedirectLink(java.lang.String)\n     */\n    @Override\n    public boolean appendRedirectLink(String redirectLink) {\n        // count redirect as include, too:\n        addInclude(redirectLink);\n        return super.appendRedirectLink(redirectLink);\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MySQLiteMagicWord.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport java.math.BigInteger;\nimport java.util.HashSet;\n\nimport de.zib.scalaris.examples.wikipedia.SQLiteDataHandler;\nimport de.zib.scalaris.examples.wikipedia.ValueResult;\n\n/**\n * Gets values for magic words not handled by {@link MyMagicWord} which need\n * to be retrieved from the SQLite DB.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MySQLiteMagicWord extends MyMagicWord {\n\n    private static HashSet<String> MY_MAGIC_WORDS = new HashSet<String>(20);\n\n    // private HashMap parameterValues = new HashMap();\n\n    static {\n        // statistics\n        MY_MAGIC_WORDS.add(MAGIC_NUMBER_ARTICLES);\n        MY_MAGIC_WORDS.add(MAGIC_NUMBER_PAGES);\n        MY_MAGIC_WORDS.add(MAGIC_NUMBER_FILES);\n        MY_MAGIC_WORDS.add(MAGIC_PAGES_IN_CATEGORY);\n        MY_MAGIC_WORDS.add(MAGIC_PAGES_IN_CAT);\n        MY_MAGIC_WORDS.add(MAGIC_NUMBER_USERS);\n        MY_MAGIC_WORDS.add(MAGIC_NUMBER_ADMINS);\n        MY_MAGIC_WORDS.add(MAGIC_PAGES_IN_NAMESPACE_NS);\n        MY_MAGIC_WORDS.add(MAGIC_PAGES_IN_NAMESPACE);\n        // page values\n        MY_MAGIC_WORDS.add(MAGIC_REVISION_ID);\n        MY_MAGIC_WORDS.add(MAGIC_REVISION_DAY);\n        MY_MAGIC_WORDS.add(MAGIC_REVISION_DAY2);\n        MY_MAGIC_WORDS.add(MAGIC_REVISION_MONTH);\n        MY_MAGIC_WORDS.add(MAGIC_REVISION_YEAR);\n        MY_MAGIC_WORDS.add(MAGIC_REVISION_TIMESTAMP);\n    }\n    \n    public static boolean isMagicWord(String name) {\n        return MyMagicWord.isMagicWord(name) || isMyMagicWord(name);\n    }\n\n    /**\n     * Determines if a template name corresponds to a magic word that is handled\n     * by this class instead of {@link MyMagicWord}.\n     * \n     * @param name\n     *            the template name\n     * \n     * @return whether this class should be favoured over {@link MyMagicWord} for\n     *         parsing this template\n     */\n    public static boolean isMyMagicWord(String name) {\n        return MY_MAGIC_WORDS.contains(name);\n    }\n\n    /**\n     * Process a magic word, returning the value corresponding to the magic\n     * word.\n     * \n     * @param name\n     *            the template name, i.e. a magic word\n     * @param origParameter\n     *            the template parameters\n     * @param model\n     *            the currently used model name\n     * @param hasParameter\n     *            whether a parameter was given or not (cannot distinguish from\n     *            <tt>parameter</tt> value alone)\n     * \n     * @return the value of the magic word\n     * \n     * @see <a\n     *      href=\"http://meta.wikimedia.org/wiki/Help:Magic_words\">http://meta.wikimedia.org/wiki/Help:Magic_words</a>\n     */\n    public static String processMagicWord(final String name,\n            final String origParameter, final MySQLiteWikiModel model, boolean hasParameter) {\n        if (!isMyMagicWord(name)) {\n            return MyMagicWord.processMagicWord(name, origParameter, model, hasParameter);\n        }\n        \n        // check whether numbers should be printed in raw format and\n        // remove this tag from the parameter string:\n        boolean rawNumber = false;\n        String parameter;\n        if (origParameter.equals(\"R\")) {\n            parameter = \"\";\n            rawNumber = true;\n        } else if (origParameter.endsWith(\"|R\")) {\n            parameter = origParameter.substring(0, origParameter.length() - 2);\n            rawNumber = true;\n        } else {\n            parameter = origParameter;\n        }\n        \n        /*\n         * Technical metadata / Latest revision to current page\n         */\n//      } else if (name.equals(MAGIC_REVISION_ID)) {\n//          // TODO: implement\n//          return null;\n//      } else if (name.equals(MAGIC_REVISION_DAY)) {\n//          // TODO: implement\n//          return null;\n//      } else if (name.equals(MAGIC_REVISION_DAY2)) {\n//          // TODO: implement\n//          return null;\n//      } else if (name.equals(MAGIC_REVISION_MONTH)) {\n//          // TODO: implement\n//          return null;\n//      } else if (name.equals(MAGIC_REVISION_YEAR)) {\n//          // TODO: implement\n//          return null;\n//      } else if (name.equals(MAGIC_REVISION_TIMESTAMP)) {\n//          // TODO: implement\n//          return null;\n//          {{REVISIONUSER}}\n//          {{PROTECTIONLEVEL:action}}\n\n        /*\n         * Statistics / Entire wiki\n         */\n        if (name.equals(MAGIC_NUMBER_PAGES)) {\n            ValueResult<BigInteger> pageCountResult = SQLiteDataHandler.getPageCount(model.connection);\n            model.addStats(pageCountResult.stats);\n            model.addInvolvedKeys(pageCountResult.involvedKeys);\n            if (pageCountResult.success) {\n                return model.formatStatisticNumber(rawNumber, pageCountResult.value);\n            }\n        } else if (name.equals(MAGIC_NUMBER_ARTICLES)) {\n            ValueResult<BigInteger> pageCountResult = SQLiteDataHandler\n                    .getArticleCount(model.connection);\n            model.addStats(pageCountResult.stats);\n            model.addInvolvedKeys(pageCountResult.involvedKeys);\n            if (pageCountResult.success) {\n                return model.formatStatisticNumber(rawNumber, pageCountResult.value);\n            }\n        } else if (name.equals(MAGIC_NUMBER_FILES)) {\n            // we currently do not store files:\n            return model.formatStatisticNumber(rawNumber, 0);\n//            {{NUMBEROFEDITS}}\n//            {{NUMBEROFVIEWS}}\n        } else if (name.equals(MAGIC_NUMBER_USERS) || name.equals(MAGIC_NUMBER_ADMINS)) {\n            // we currently do not support users/admins:\n            return model.formatStatisticNumber(rawNumber, 0);\n//            {{NUMBEROFACTIVEUSERS}}\n        } else if (name.equals(MAGIC_PAGES_IN_CATEGORY) || name.equals(MAGIC_PAGES_IN_CAT)) {\n            NormalisedTitle category = new NormalisedTitle(\n                    MyNamespace.CATEGORY_NAMESPACE_KEY,\n                    MyWikiModel.normaliseName(parameter.trim()));\n            ValueResult<BigInteger> catListResult = SQLiteDataHandler\n                    .getPagesInCategoryCount(model.connection, category);\n            model.addStats(catListResult.stats);\n            model.addInvolvedKeys(catListResult.involvedKeys);\n            if (catListResult.success) {\n                return model.formatStatisticNumber(rawNumber, catListResult.value);\n            }\n//            {{NUMBERINGROUP:groupname}}\n//            {{NUMINGROUP:groupname}}\n//        } else if (name.equals(MAGIC_PAGES_IN_NAMESPACE) || name.equals(MAGIC_PAGES_IN_NAMESPACE_NS)) {\n//            // TODO: implement\n//            return null;\n        }\n        \n        return name;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MySQLiteWikiModel.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport com.almworks.sqlite4java.SQLiteException;\n\nimport de.zib.scalaris.examples.wikipedia.RevisionResult;\nimport de.zib.scalaris.examples.wikipedia.SQLiteDataHandler;\nimport de.zib.scalaris.examples.wikipedia.SQLiteDataHandler.Connection;\n\n/**\n * Wiki model using SQLite to fetch (new) data, e.g. templates.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MySQLiteWikiModel extends MyWikiModel {\n    final protected Connection connection;\n    \n    /**\n     * Creates a new wiki model to render wiki text using the given connection\n     * to a SQLite DB.\n     * \n     * @param imageBaseURL\n     *            base url pointing to images - can contain ${image} for\n     *            replacement\n     * @param linkBaseURL\n     *            base url pointing to links - can contain ${title} for\n     *            replacement\n     * @param connection\n     *            the database connection\n     * @param namespace\n     *            namespace of the wiki\n     * \n     * @throws SQLiteException\n     *             if the DB connection fails\n     */\n    public MySQLiteWikiModel(String imageBaseURL, String linkBaseURL, Connection connection, MyNamespace namespace) throws SQLiteException {\n        super(imageBaseURL, linkBaseURL, namespace);\n        assert (connection != null);\n        this.connection = connection;\n    }\n\n    /**\n     * Determines if a template name corresponds to a magic word using\n     * {@link MySQLiteMagicWord#isMagicWord(String)}.\n     * \n     * @param name\n     *            the template name\n     * \n     * @return whether the template is a magic word or not\n     */\n    @Override\n    protected boolean isMagicWord(String name) {\n        return MySQLiteMagicWord.isMagicWord(name);\n    }\n\n    /**\n     * Retrieves the contents of the given magic word from the SQLite DB.\n     * \n     * @param templateName\n     *            the template's name without the namespace, e.g. a magic word\n     *            including its parameters\n     * @param magicWord\n     *            the magic word alone\n     * @param parameter\n     *            the parameters of the magic word\n     * @param hasParameter\n     *            whether a parameter was given or not (cannot distinguish from\n     *            <tt>parameter</tt> value alone)\n     * \n     * @return the contents of the magic word (see\n     *         {@link MySQLiteMagicWord#processMagicWord(String, String, info.bliki.wiki.model.IWikiModel)})\n     */\n    @Override\n    protected String retrieveMagicWord(String articleName, String magicWord,\n            String parameter, boolean hasParameter) {\n        return MySQLiteMagicWord.processMagicWord(magicWord, parameter, this, hasParameter);\n    }\n    \n    @Override\n    protected boolean hasDBConnection() {\n        return connection != null;\n    }\n\n    @Override\n    protected RevisionResult getRevFromDB(NormalisedTitle pageName) {\n        return SQLiteDataHandler.getRevision(connection, pageName, getNamespace());\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MyScalarisMagicWord.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport java.math.BigInteger;\nimport java.util.HashSet;\n\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandler;\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandlerNormalised;\nimport de.zib.scalaris.examples.wikipedia.ValueResult;\n\n/**\n * Gets values for magic words not handled by {@link MyMagicWord} which need\n * to be retrieved from Scalaris.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MyScalarisMagicWord extends MyMagicWord {\n\n    private static HashSet<String> MY_MAGIC_WORDS = new HashSet<String>(20);\n\n    // private HashMap parameterValues = new HashMap();\n\n    static {\n        // statistics\n        MY_MAGIC_WORDS.add(MAGIC_NUMBER_ARTICLES);\n        MY_MAGIC_WORDS.add(MAGIC_NUMBER_PAGES);\n        MY_MAGIC_WORDS.add(MAGIC_NUMBER_FILES);\n        MY_MAGIC_WORDS.add(MAGIC_PAGES_IN_CATEGORY);\n        MY_MAGIC_WORDS.add(MAGIC_PAGES_IN_CAT);\n        MY_MAGIC_WORDS.add(MAGIC_NUMBER_USERS);\n        MY_MAGIC_WORDS.add(MAGIC_NUMBER_ADMINS);\n        MY_MAGIC_WORDS.add(MAGIC_PAGES_IN_NAMESPACE_NS);\n        MY_MAGIC_WORDS.add(MAGIC_PAGES_IN_NAMESPACE);\n        // page values\n        MY_MAGIC_WORDS.add(MAGIC_REVISION_ID);\n        MY_MAGIC_WORDS.add(MAGIC_REVISION_DAY);\n        MY_MAGIC_WORDS.add(MAGIC_REVISION_DAY2);\n        MY_MAGIC_WORDS.add(MAGIC_REVISION_MONTH);\n        MY_MAGIC_WORDS.add(MAGIC_REVISION_YEAR);\n        MY_MAGIC_WORDS.add(MAGIC_REVISION_TIMESTAMP);\n    }\n    \n    public static boolean isMagicWord(String name) {\n        return MyMagicWord.isMagicWord(name) || isMyMagicWord(name);\n    }\n\n    /**\n     * Determines if a template name corresponds to a magic word that is handled\n     * by this class instead of {@link MyMagicWord}.\n     * \n     * @param name\n     *            the template name\n     * \n     * @return whether this class should be favoured over {@link MyMagicWord} for\n     *         parsing this template\n     */\n    public static boolean isMyMagicWord(String name) {\n        return MY_MAGIC_WORDS.contains(name);\n    }\n\n    /**\n     * Process a magic word, returning the value corresponding to the magic\n     * word.\n     * \n     * @param name\n     *            the template name, i.e. a magic word\n     * @param origParameter\n     *            the template parameters\n     * @param model\n     *            the currently used model\n     * @param hasParameter\n     *            whether a parameter was given or not (cannot distinguish from\n     *            <tt>parameter</tt> value alone)\n     * \n     * @return the value of the magic word\n     * \n     * @see <a\n     *      href=\"http://meta.wikimedia.org/wiki/Help:Magic_words\">http://meta.wikimedia.org/wiki/Help:Magic_words</a>\n     */\n    public static String processMagicWord(final String name,\n            final String origParameter, final MyScalarisWikiModel model, boolean hasParameter) {\n        if (!isMyMagicWord(name)) {\n            return MyMagicWord.processMagicWord(name, origParameter, model, hasParameter);\n        }\n        \n        // check whether numbers should be printed in raw format and\n        // remove this tag from the parameter string:\n        boolean rawNumber = false;\n        String parameter;\n        if (origParameter.equals(\"R\")) {\n            parameter = \"\";\n            rawNumber = true;\n        } else if (origParameter.endsWith(\"|R\")) {\n            parameter = origParameter.substring(0, origParameter.length() - 2);\n            rawNumber = true;\n        } else {\n            parameter = origParameter;\n        }\n\n        /*\n         * Technical metadata / Latest revision to current page\n         */\n//      } else if (name.equals(MAGIC_REVISION_ID)) {\n//          // TODO: implement\n//          return null;\n//      } else if (name.equals(MAGIC_REVISION_DAY)) {\n//          // TODO: implement\n//          return null;\n//      } else if (name.equals(MAGIC_REVISION_DAY2)) {\n//          // TODO: implement\n//          return null;\n//      } else if (name.equals(MAGIC_REVISION_MONTH)) {\n//          // TODO: implement\n//          return null;\n//      } else if (name.equals(MAGIC_REVISION_YEAR)) {\n//          // TODO: implement\n//          return null;\n//      } else if (name.equals(MAGIC_REVISION_TIMESTAMP)) {\n//          // TODO: implement\n//          return null;\n//          {{REVISIONUSER}}\n//          {{PROTECTIONLEVEL:action}}\n        \n        /*\n         * Statistics / Entire wiki\n         */\n        if (name.equals(MAGIC_NUMBER_PAGES)) {\n            ValueResult<BigInteger> pageCountResult = ScalarisDataHandler.getPageCount(model.connection);\n            model.addStats(pageCountResult.stats);\n            model.addInvolvedKeys(pageCountResult.involvedKeys);\n            if (pageCountResult.success) {\n                return model.formatStatisticNumber(rawNumber, pageCountResult.value);\n            }\n        } else if (name.equals(MAGIC_NUMBER_ARTICLES)) {\n            ValueResult<BigInteger> pageCountResult = ScalarisDataHandler\n                    .getArticleCount(model.connection);\n            model.addStats(pageCountResult.stats);\n            model.addInvolvedKeys(pageCountResult.involvedKeys);\n            if (pageCountResult.success) {\n                return model.formatStatisticNumber(rawNumber, pageCountResult.value);\n            }\n        } else if (name.equals(MAGIC_NUMBER_FILES)) {\n            // we currently do not store files:\n            return model.formatStatisticNumber(rawNumber, 0);\n//            {{NUMBEROFEDITS}}\n//            {{NUMBEROFVIEWS}}\n        } else if (name.equals(MAGIC_NUMBER_USERS) || name.equals(MAGIC_NUMBER_ADMINS)) {\n            // we currently do not support users/admins:\n            return model.formatStatisticNumber(rawNumber, 0);\n//            {{NUMBEROFACTIVEUSERS}}\n        } else if (name.equals(MAGIC_PAGES_IN_CATEGORY) || name.equals(MAGIC_PAGES_IN_CAT)) {\n            NormalisedTitle category = new NormalisedTitle(\n                    MyNamespace.CATEGORY_NAMESPACE_KEY,\n                    MyWikiModel.normaliseName(parameter.trim()));\n            ValueResult<BigInteger> catListResult = ScalarisDataHandlerNormalised\n                    .getPagesInCategoryCount(model.connection, category);\n            model.addStats(catListResult.stats);\n            model.addInvolvedKeys(catListResult.involvedKeys);\n            if (catListResult.success) {\n                return model.formatStatisticNumber(rawNumber, catListResult.value);\n            }\n//            {{NUMBERINGROUP:groupname}}\n//            {{NUMINGROUP:groupname}}\n//        } else if (name.equals(MAGIC_PAGES_IN_NAMESPACE) || name.equals(MAGIC_PAGES_IN_NAMESPACE_NS)) {\n//            // TODO: implement\n//            return null;\n        }\n        \n        return name;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MyScalarisWikiModel.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport de.zib.scalaris.Connection;\nimport de.zib.scalaris.examples.wikipedia.RevisionResult;\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandlerNormalised;\n\n/**\n * Wiki model using Scalaris to fetch (new) data, e.g. templates.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MyScalarisWikiModel extends MyWikiModel {\n    protected Connection connection;\n    /**\n     * Creates a new wiki model to render wiki text using the given connection\n     * to Scalaris.\n     * \n     * @param imageBaseURL\n     *            base url pointing to images - can contain ${image} for\n     *            replacement\n     * @param linkBaseURL\n     *            base url pointing to links - can contain ${title} for\n     *            replacement\n     * @param connection\n     *            connection to Scalaris\n     * @param namespace\n     *            namespace of the wiki\n     */\n    public MyScalarisWikiModel(String imageBaseURL, String linkBaseURL, Connection connection, MyNamespace namespace) {\n        super(imageBaseURL, linkBaseURL, namespace);\n        this.connection = connection;\n    }\n\n    /**\n     * Determines if a template name corresponds to a magic word using\n     * {@link MyScalarisMagicWord#isMagicWord(String)}.\n     * \n     * @param name\n     *            the template name\n     * \n     * @return whether the template is a magic word or not\n     */\n    @Override\n    protected boolean isMagicWord(String name) {\n        return MyScalarisMagicWord.isMagicWord(name);\n    }\n\n    /**\n     * Retrieves the contents of the given magic word from Scalaris.\n     * \n     * @param templateName\n     *            the template's name without the namespace, e.g. a magic word\n     *            including its parameters\n     * @param magicWord\n     *            the magic word alone\n     * @param parameter\n     *            the parameters of the magic word\n     * @param hasParameter\n     *            whether a parameter was given or not (cannot distinguish from\n     *            <tt>parameter</tt> value alone)\n     * \n     * @return the contents of the magic word (see\n     *         {@link MyScalarisMagicWord#processMagicWord(String, String, info.bliki.wiki.model.IWikiModel)})\n     */\n    @Override\n    protected String retrieveMagicWord(String articleName, String magicWord,\n            String parameter, boolean hasParameter) {\n        return MyScalarisMagicWord.processMagicWord(magicWord, parameter, this, hasParameter);\n    }\n    \n    @Override\n    protected boolean hasDBConnection() {\n        return connection != null;\n    }\n\n    @Override\n    protected RevisionResult getRevFromDB(NormalisedTitle pageName) {\n        return ScalarisDataHandlerNormalised.getRevision(connection, pageName);\n    }\n\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/MyWikiModel.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport info.bliki.htmlcleaner.TagNode;\nimport info.bliki.wiki.filter.Encoder;\nimport info.bliki.wiki.filter.HTMLConverter;\nimport info.bliki.wiki.filter.ITextConverter;\nimport info.bliki.wiki.model.Configuration;\nimport info.bliki.wiki.model.WikiModel;\nimport info.bliki.wiki.namespaces.Namespace;\nimport info.bliki.wiki.tags.IgnoreTag;\n\nimport java.io.UnsupportedEncodingException;\nimport java.net.URLEncoder;\nimport java.text.NumberFormat;\nimport java.util.ArrayList;\nimport java.util.Collection;\nimport java.util.EnumMap;\nimport java.util.HashMap;\nimport java.util.HashSet;\nimport java.util.List;\nimport java.util.Locale;\nimport java.util.Map;\nimport java.util.Map.Entry;\nimport java.util.Set;\nimport java.util.regex.Matcher;\nimport java.util.regex.Pattern;\n\nimport org.apache.commons.lang.StringEscapeUtils;\n\nimport de.zib.scalaris.examples.wikipedia.InvolvedKey;\nimport de.zib.scalaris.examples.wikipedia.RevisionResult;\nimport de.zib.tools.LinkedMultiHashMap;\n\n/**\n * Wiki model fixing some bugs of {@link WikiModel} and adding some\n * functionality.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class MyWikiModel extends WikiModel {\n\n    /**\n     * Interwiki links pointing to other wikis in the web\n     */\n    private static final String[] INTERLANGUAGE_STRINGS = { \"en\", \"de\", \"fr\",\n            \"it\", \"pl\", \"es\", \"ja\", \"ru\", \"nl\", \"pt\", \"sv\", \"zh\", \"ca\", \"uk\",\n            \"no\", \"fi\", \"hu\", \"cs\", \"ro\", \"tr\", \"ko\", \"vi\", \"da\", \"ar\", \"eo\",\n            \"sr\", \"id\", \"lt\", \"vo\", \"sk\", \"he\", \"fa\", \"bg\", \"sl\", \"eu\", \"war\",\n            \"lmo\", \"et\", \"hr\", \"new\", \"te\", \"nn\", \"th\", \"gl\", \"el\", \"ceb\",\n            \"simple\", \"ms\", \"ht\", \"bs\", \"bpy\", \"lb\", \"ka\", \"is\", \"sq\", \"la\",\n            \"br\", \"hi\", \"az\", \"bn\", \"mk\", \"mr\", \"sh\", \"tl\", \"cy\", \"io\", \"pms\",\n            \"lv\", \"ta\", \"su\", \"oc\", \"jv\", \"nap\", \"nds\", \"scn\", \"be\", \"ast\",\n            \"ku\", \"wa\", \"af\", \"be-x-old\", \"an\", \"ksh\", \"szl\", \"fy\", \"frr\",\n            \"yue\", \"ur\", \"ia\", \"ga\", \"yi\", \"sw\", \"als\", \"hy\", \"am\", \"roa-rup\",\n            \"map-bms\", \"bh\", \"co\", \"cv\", \"dv\", \"nds-nl\", \"fo\", \"fur\", \"glk\",\n            \"gu\", \"ilo\", \"kn\", \"pam\", \"csb\", \"kk\", \"km\", \"lij\", \"li\", \"ml\",\n            \"gv\", \"mi\", \"mt\", \"nah\", \"ne\", \"nrm\", \"se\", \"nov\", \"qu\", \"os\",\n            \"pi\", \"pag\", \"ps\", \"pdc\", \"rm\", \"bat-smg\", \"sa\", \"gd\", \"sco\", \"sc\",\n            \"si\", \"tg\", \"roa-tara\", \"tt\", \"to\", \"tk\", \"hsb\", \"uz\", \"vec\",\n            \"fiu-vro\", \"wuu\", \"vls\", \"yo\", \"diq\", \"zh-min-nan\", \"zh-classical\",\n            \"frp\", \"lad\", \"bar\", \"bcl\", \"kw\", \"mn\", \"haw\", \"ang\", \"ln\", \"ie\",\n            \"wo\", \"tpi\", \"ty\", \"crh\", \"jbo\", \"ay\", \"zea\", \"eml\", \"ky\", \"ig\",\n            \"or\", \"mg\", \"cbk-zam\", \"kg\", \"arc\", \"rmy\", \"gn\", \"mo (closed)\",\n            \"so\", \"kab\", \"ks\", \"stq\", \"ce\", \"udm\", \"mzn\", \"pap\", \"cu\", \"sah\",\n            \"tet\", \"sd\", \"lo\", \"ba\", \"pnb\", \"iu\", \"na\", \"got\", \"bo\", \"dsb\",\n            \"chr\", \"cdo\", \"hak\", \"om\", \"my\", \"sm\", \"ee\", \"pcd\", \"ug\", \"as\",\n            \"ti\", \"av\", \"bm\", \"zu\", \"pnt\", \"nv\", \"cr\", \"pih\", \"ss\", \"ve\", \"bi\",\n            \"rw\", \"ch\", \"arz\", \"xh\", \"kl\", \"ik\", \"bug\", \"dz\", \"ts\", \"tn\", \"kv\",\n            \"tum\", \"xal\", \"st\", \"tw\", \"bxr\", \"ak\", \"ab\", \"ny\", \"fj\", \"lbe\",\n            \"ki\", \"za\", \"ff\", \"lg\", \"sn\", \"ha\", \"sg\", \"ii\", \"cho\", \"rn\", \"mh\",\n            \"chy\", \"ng\", \"kj\", \"ho\", \"mus\", \"kr\", \"hz\", \"mwl\", \"pa\", \"ace\",\n            \"bat-smg\", \"bjn\", \"cbk-zam\", \"cdo\", \"ceb\", \"crh\", \"dsb\", \"eml\",\n            \"ext\", \"fiu-vro\", \"frp\", \"frr\", \"gag\", \"gan\", \"hak\", \"kaa\", \"kab\",\n            \"kbd\", \"koi\", \"krc\", \"ksh\", \"lad\", \"lbe\", \"lmo\", \"map-bms\", \"mdf\",\n            \"mrj\", \"mwl\", \"nap\", \"nds-nl\", \"nrm\", \"pcd\", \"pfl\", \"pih\", \"pnb\", \"rmy\",\n            \"roa-tara\", \"rue\", \"sah\", \"sco\", \"stq\", \"szl\", \"udm\",\n            \"war\", \"wuu\", \"xmf\", \"zea\", \"zh-classical\", \"zh-yue\",\n            \"ckb\", \"hif\", \"mhr\", \"myv\", \"srn\"};\n    \n    protected static final Set<String> INTERLANGUAGE_KEYS;\n    \n    /**\n     * Localised prefixes for special pages, e.g. \"Special\". \n     */\n    public static final Map<String, String> SPECIAL_PREFIX = new HashMap<String, String>();\n    /**\n     * Localised suffixes for special pages, e.g. \"AllPages\".\n     */\n    public static final Map<String, EnumMap<SpecialPage, String>> SPECIAL_SUFFIX = new HashMap<String, EnumMap<SpecialPage, String>>();\n\n    /**\n     * Enum for all available special pages.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static enum SpecialPage {\n        /**\n         * Redirect to random page.\n         */\n        SPECIAL_RANDOM,\n        /**\n         * List of all pages.\n         */\n        SPECIAL_ALLPAGES,\n        /**\n         * List of all pages with a given prefix.\n         */\n        SPECIAL_PREFIXINDEX,\n        /**\n         * Search for a page.\n         */\n        SPECIAL_SEARCH,\n        /**\n         * Pages linking to another page.\n         */\n        SPECIAL_WHATLINKSHERE,\n        /**\n         * List of available special pages.\n         */\n        SPECIAL_SPECIALPAGES,\n        /**\n         * Some statistics.\n         */\n        SPECIAL_STATS,\n        /**\n         * Version information.\n         */\n        SPECIAL_VERSION;\n    }\n    \n    /**\n     * Cache of processed magic words.\n     */\n    protected Map<String, String> magicWordCache = new HashMap<String, String>();\n\n    protected LinkedMultiHashMap<String, Long> stats = new LinkedMultiHashMap<String, Long>();\n    \n    /**\n     * All keys that have been read or written during the current operation.\n     */\n    protected final List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n    \n    protected ExistingPagesCache existingPages = ExistingPagesCache.NULL_CACHE;\n\n    protected Map<NormalisedTitle, String> pageCache = new HashMap<NormalisedTitle, String>();\n\n    /**\n     * Text of the page to render, i.e. given to\n     * {@link #renderPageWithCache(String)} or\n     * {@link #renderPageWithCache(ITextConverter, String)}.\n     */\n    private String renderWikiText = null;\n    \n    protected static final Pattern MATCH_WIKI_FORBIDDEN_TITLE_CHARS =\n            Pattern.compile(\"^.*?([\\\\p{Cc}\\\\p{Cn}\\\\p{Co}#<>\\\\[\\\\]|{}\\\\n\\\\r]).*$\", Pattern.CASE_INSENSITIVE | Pattern.DOTALL);\n\n    /**\n     * Pattern to check whether a wikitext is redirecting or not.\n     */\n    public static final Pattern MATCH_WIKI_REDIRECT = Pattern.compile(\"^\\\\s*#REDIRECT[ ]?\\\\[\\\\[:?([^\\\\]#]*)[^\\\\]]*\\\\]\\\\].*$\", Pattern.CASE_INSENSITIVE | Pattern.DOTALL);\n\n    static {\n        // BEWARE: fields in Configuration are static -> this changes all configurations!\n        Configuration.DEFAULT_CONFIGURATION.addTemplateFunction(\"fullurl\", MyFullurl.CONST);\n        Configuration.DEFAULT_CONFIGURATION.addTemplateFunction(\"localurl\", MyLocalurl.CONST);\n        \n        // do not put these into the HTML text (they are not rendered anyway)\n        Configuration.DEFAULT_CONFIGURATION.addTokenTag(\"inputbox\", new IgnoreTag(\"inputbox\"));\n        Configuration.DEFAULT_CONFIGURATION.addTokenTag(\"imagemap\", new IgnoreTag(\"imagemap\"));\n        Configuration.DEFAULT_CONFIGURATION.addTokenTag(\"timeline\", new IgnoreTag(\"timeline\"));\n        \n        Configuration.AVOID_PAGE_BREAK_IN_TABLE = false;\n        \n        // allow style attributes:\n        TagNode.addAllowedAttribute(\"style\");\n        \n        // create set of keys for interlanguage wiki links\n        // also add missing hsb interlanguage link:\n        Map<String, String> interWikiMap = Configuration.DEFAULT_CONFIGURATION.getInterwikiMap();\n        INTERLANGUAGE_KEYS = new HashSet<String>(INTERLANGUAGE_STRINGS.length);\n        for (String lang : INTERLANGUAGE_STRINGS) {\n            INTERLANGUAGE_KEYS.add(lang);\n            // if there is no interwiki link for it, create one and guess the URL:\n            if (!interWikiMap.containsKey(lang)) {\n                Configuration.DEFAULT_CONFIGURATION.addInterwikiLink(lang, \"http://\" + lang + \".wiktionary.org/wiki/?${title}\");\n            }\n        }\n        \n        // localised special pages titles (prefix + suffix)\n        // BEWARE: keep SPECIAL_PREFIX and SPECIAL_SUFFIX in sync!\n        SPECIAL_PREFIX.put(\"en\", \"Special\");\n        SPECIAL_PREFIX.put(\"simple\", \"Special\");\n        SPECIAL_PREFIX.put(\"de\", \"Spezial\");\n        SPECIAL_PREFIX.put(\"bar\", \"Spezial\");\n        SPECIAL_PREFIX.put(\"es\", \"Especial\");\n        SPECIAL_PREFIX.put(\"pl\", \"Specjalna\");\n        // BEWARE: include normalised page titles!\n\n        EnumMap<SpecialPage, String> SPECIAL_SUFFIX_EN = new EnumMap<SpecialPage, String>(SpecialPage.class);\n        SPECIAL_SUFFIX_EN.put(SpecialPage.SPECIAL_RANDOM, \"Random\");\n        SPECIAL_SUFFIX_EN.put(SpecialPage.SPECIAL_ALLPAGES, \"AllPages\");\n        SPECIAL_SUFFIX_EN.put(SpecialPage.SPECIAL_PREFIXINDEX, \"PrefixIndex\");\n        SPECIAL_SUFFIX_EN.put(SpecialPage.SPECIAL_SEARCH, \"Search\");\n        SPECIAL_SUFFIX_EN.put(SpecialPage.SPECIAL_WHATLINKSHERE, \"WhatLinksHere\");\n        SPECIAL_SUFFIX_EN.put(SpecialPage.SPECIAL_SPECIALPAGES, \"SpecialPages\");\n        SPECIAL_SUFFIX_EN.put(SpecialPage.SPECIAL_STATS, \"Statistics\");\n        SPECIAL_SUFFIX_EN.put(SpecialPage.SPECIAL_VERSION, \"Version\");\n\n        EnumMap<SpecialPage, String> SPECIAL_SUFFIX_DE = new EnumMap<SpecialPage, String>(SpecialPage.class);\n        SPECIAL_SUFFIX_DE.put(SpecialPage.SPECIAL_RANDOM, \"Zufällige Seite\");\n        SPECIAL_SUFFIX_DE.put(SpecialPage.SPECIAL_ALLPAGES, \"Alle Seiten\");\n        SPECIAL_SUFFIX_DE.put(SpecialPage.SPECIAL_PREFIXINDEX, \"Präfixindex\");\n        SPECIAL_SUFFIX_DE.put(SpecialPage.SPECIAL_SEARCH, \"Suche\");\n        SPECIAL_SUFFIX_DE.put(SpecialPage.SPECIAL_WHATLINKSHERE, \"Linkliste\");\n        SPECIAL_SUFFIX_DE.put(SpecialPage.SPECIAL_SPECIALPAGES, \"Spezialseiten\");\n        SPECIAL_SUFFIX_DE.put(SpecialPage.SPECIAL_STATS, \"Statistik\");\n        SPECIAL_SUFFIX_DE.put(SpecialPage.SPECIAL_VERSION, \"Version\");\n\n        EnumMap<SpecialPage, String> SPECIAL_SUFFIX_ES = new EnumMap<SpecialPage, String>(SpecialPage.class);\n        SPECIAL_SUFFIX_ES.put(SpecialPage.SPECIAL_RANDOM, \"Aleatoria\");\n        SPECIAL_SUFFIX_ES.put(SpecialPage.SPECIAL_ALLPAGES, \"Todas\");\n        SPECIAL_SUFFIX_ES.put(SpecialPage.SPECIAL_PREFIXINDEX, \"PáginasPorPrefijo\");\n        SPECIAL_SUFFIX_ES.put(SpecialPage.SPECIAL_SEARCH, \"Buscar\");\n        SPECIAL_SUFFIX_ES.put(SpecialPage.SPECIAL_WHATLINKSHERE, \"LoQueEnlazaAquí\");\n        SPECIAL_SUFFIX_ES.put(SpecialPage.SPECIAL_SPECIALPAGES, \"PáginasEspeciales\");\n        SPECIAL_SUFFIX_ES.put(SpecialPage.SPECIAL_STATS, \"Estadísticas\");\n        SPECIAL_SUFFIX_ES.put(SpecialPage.SPECIAL_VERSION, \"Versión\");\n        \n        SPECIAL_SUFFIX.put(\"en\", SPECIAL_SUFFIX_EN);\n        SPECIAL_SUFFIX.put(\"simple\", SPECIAL_SUFFIX_EN);\n        SPECIAL_SUFFIX.put(\"de\", SPECIAL_SUFFIX_DE);\n        SPECIAL_SUFFIX.put(\"bar\", SPECIAL_SUFFIX_DE);\n        SPECIAL_SUFFIX.put(\"es\", SPECIAL_SUFFIX_ES);\n    }\n    \n    /**\n     * Creates a new wiki model to render wiki text fixing some bugs of\n     * {@link WikiModel}.\n     * \n     * @param imageBaseURL\n     *            base url pointing to images - can contain ${image} for\n     *            replacement\n     * @param linkBaseURL\n     *            base url pointing to links - can contain ${title} for\n     *            replacement\n     * @param namespace\n     *            namespace of the wiki\n     */\n    public MyWikiModel(String imageBaseURL, String linkBaseURL, MyNamespace namespace) {\n        super(new MyConfiguration(namespace), null, namespace, imageBaseURL, linkBaseURL);\n        addTemplateFunction(\"#ifexist\", MyIfexistTemplateFun.CONST);\n    }\n    \n    /* (non-Javadoc)\n     * @see info.bliki.wiki.model.AbstractWikiModel#getRawWikiContent(java.lang.String, java.lang.String, java.util.Map)\n     */\n    @Override\n    public String getRawWikiContent(String namespace, String articleName,\n            Map<String, String> templateParameters) {\n        if (isTemplateNamespace(namespace)) {\n            String processedMagicWord = null;\n            processedMagicWord = processMagicWord(articleName);\n            if (processedMagicWord != null) {\n                return processedMagicWord;\n            }\n        }\n        \n        if (!isValidTitle(createFullPageName(namespace, articleName))) {\n            return null;\n        }\n\n        // (ugly) fix for template parameter replacement if no parameters given,\n        // e.g. \"{{noun}}\" in the simple English Wiktionary\n        if (templateParameters != null && templateParameters.isEmpty()) {\n            templateParameters.put(\"\", null);\n        }\n\n        // note: cannot cache templates here since the text returned by the\n        // implementation-specific retrievePage() method may depend in the exact\n        // parameters or not\n        return retrievePage(namespace, articleName, templateParameters);\n    }\n\n    /**\n     * Checks whether the given template name is a magic word and if this is the\n     * case, processes it and returns its value.\n     * \n     * Retrieves magic word contents using\n     * {@link #retrieveMagicWord(String, String, String)} and caches the\n     * contents in {@link #magicWordCache}.\n     * \n     * @param templateName\n     *            the template's name without the namespace, e.g. a magic word\n     *            including its parameters\n     * \n     * @return the contents of the magic word or <tt>null</tt> if the template\n     *         is no magic word\n     */\n    private String processMagicWord(String templateName) {\n        int index = templateName.indexOf(':');\n        String magicWord = templateName;\n        String parameter = \"\";\n        boolean hasParameter = false;\n        if (index > 0) {\n            hasParameter = true;\n            // if it is a magic word, the first part is the word itself, the second its parameters\n            magicWord = templateName.substring(0, index);\n            parameter = templateName.substring(index + 1).trim();\n        }\n        \n        if (isMagicWord(magicWord)) {\n            // cache values for magic words:\n            if (magicWordCache.containsKey(templateName)) {\n                return magicWordCache.get(templateName);\n            } else {\n                String value = retrieveMagicWord(templateName, magicWord, parameter, hasParameter);\n                magicWordCache.put(templateName, value);\n                return value;\n            }\n        } else {\n            return null;\n        }\n    }\n\n    /**\n     * Determines if a template name corresponds to a magic word using\n     * {@link MyMagicWord#isMagicWord(String)} (does not recognise magic\n     * words with parameters, e.g. <tt>TALKPAGENAME:Title</tt>).\n     * \n     * @param name\n     *            the template name (without the template namespace)\n     * \n     * @return whether the template is a magic word or not\n     */\n    protected boolean isMagicWord(String name) {\n        return MyMagicWord.isMagicWord(name);\n    }\n\n    /**\n     * Determines if a template name corresponds to a magic word using\n     * {@link #isMagicWord(String)} (also recognises magic\n     * words with parameters, e.g. <tt>TALKPAGENAME:Title</tt>).\n     * \n     * @param name\n     *            the template name (without the template namespace)\n     * \n     * @return whether the template is a magic word or not\n     */\n    public final boolean isMagicWordFull(String name) {\n        return isMagicWord(MyMagicWord.extractMagicWordPart(name));\n    }\n\n    /**\n     * Gets the names of all included pages in the template namespace (excluding\n     * magic words).\n     * \n     * @return page names without the template namespace prefix\n     * @see #getTemplates()\n     */\n    public final Set<String> getTemplatesNoMagicWords() {\n        final Set<String> result = new HashSet<String>(templates.size());\n        // remove magic words:\n        for (String template : templates) {\n            if (!isMagicWordFull(template)) {\n                result.add(template);\n            }\n        }\n        return result;\n    }\n    \n    /**\n     * Retrieves the contents of the given magic word using\n     * {@link MyMagicWord#processMagicWord(String, String, info.bliki.wiki.model.IWikiModel)}.\n     * \n     * @param templateName\n     *            the template's name without the namespace, e.g. a magic word\n     *            including its parameters\n     * @param magicWord\n     *            the magic word alone\n     * @param parameter\n     *            the parameters of the magic word name\n     * @param hasParameter\n     *            whether a parameter was given or not (cannot distinguish from\n     *            <tt>parameter</tt> value alone)\n     * \n     * @return the contents of the magic word\n     */\n    protected String retrieveMagicWord(String templateName, String magicWord,\n            String parameter, boolean hasParameter) {\n        return MyMagicWord.processMagicWord(templateName, parameter, this, hasParameter);\n    }\n    \n    /**\n     * Renders the \"redirect to\" content in case no auto-redirection is used.\n     * \n     * @param pageTitle\n     *            the title of the page being redirected to\n     * \n     * @return HTML redirect note\n     */\n    public String renderRedirectPage(String pageTitle) {\n        final String redirectUrl = getWikiBaseURL().replace(\"${title}\", pageTitle);\n        final String safeRedirectTitle = StringEscapeUtils.escapeHtml(pageTitle);\n        return \"<div class=\\\"redirectMsg\\\">\"\n                + \"<img src=\\\"skins/redirectltr.png\\\" alt=\\\"#REDIRECT\\\" />\"\n                + \"<span class=\\\"redirectText\\\">\"\n                + \"<a href=\\\"\" + redirectUrl + \"\\\" title=\\\"\" + safeRedirectTitle + \"\\\">\" + pageTitle + \"</a>\"\n                + \"</span></div>\";\n    }\n    \n    /**\n     * Creates the full pagename including the namespace (if non-empty).\n     * \n     * @param namespace\n     *            the namespace of a page\n     * @param articleName\n     *            the name of a page\n     * \n     * @return the full name of the page\n     */\n    public static String createFullPageName(String namespace, String articleName) {\n        String pageName;\n        if (namespace.isEmpty()) {\n            pageName = articleName;\n        } else {\n            pageName = namespace + \":\" + articleName;\n        }\n        return pageName;\n    }\n    \n    /**\n     * Retrieves the contents of the given page.\n     * \n     * @param namespace\n     *            the namespace of the page\n     * @param articleName\n     *            the (unnormalised) page's name without the namespace\n     * @param templateParameters\n     *            template parameters if the page is a template, <tt>null</tt>\n     *            otherwise\n     * \n     * @return <tt>null</tt>\n     * @see #retrievePage(String, String, Map, boolean)\n     */\n    final protected String retrievePage(String namespace, String articleName,\n            Map<String, String> templateParameters) {\n        return retrievePage(namespace, articleName, templateParameters, true);\n    }\n\n    /**\n     * Retrieves the contents of the given page.\n     * \n     * @param pageName0\n     *            the unnormalised name of the page\n     * @param templateParameters\n     *            template parameters if the page is a template, <tt>null</tt>\n     *            otherwise\n     * @param followRedirect\n     *            whether to follow a redirect or not (at most one redirect\n     *            should be followed)\n     * \n     * @return the page's contents or <tt>null</tt> if no connection exists\n     * @see #retrievePage(String, String, Map, boolean)\n     */\n    final protected String retrievePage(String pageName0,\n            Map<String, String> templateParameters, boolean followRedirect) {\n        String[] parts = splitNsTitle(pageName0);\n        return retrievePage(parts[0], parts[1], templateParameters, followRedirect);\n    }\n\n    /**\n     * Retrieves the contents of the given page.\n     * \n     * If {@link #hasDBConnection()} is <tt>true</tt>, uses\n     * {@link #getRevFromDB(NormalisedTitle)} to get the content from the DB. If\n     * <tt>followRedirect</tt> is set, resolves redirects by including the\n     * redirected content instead.\n     * \n     * Caches retrieved pages in {@link #pageCache}.\n     * \n     * @param namespace\n     *            the namespace of the page\n     * @param articleName\n     *            the (unnormalised) page's name without the namespace\n     * @param templateParameters\n     *            template parameters if the page is a template, <tt>null</tt>\n     *            otherwise\n     * @param followRedirect\n     *            whether to follow a redirect or not (at most one redirect\n     *            should be followed)\n     * \n     * @return the page's contents or <tt>null</tt> if no connection exists\n     */\n    protected String retrievePage(String namespace, String articleName,\n            Map<String, String> templateParameters, boolean followRedirect) {\n\n        if (articleName.isEmpty()) {\n            return null;\n        }\n        \n        // normalise page name:\n        NormalisedTitle pageName = normalisePageTitle(namespace, articleName);\n        if (pageCache.containsKey(pageName)) {\n            return pageCache.get(pageName);\n        } else if (hasDBConnection()) {\n            String text = null;\n            // System.out.println(\"retrievePage(\" + namespace + \", \" + articleName + \")\");\n            RevisionResult getRevResult = getRevFromDB(pageName);\n            addStats(getRevResult.stats);\n            addInvolvedKeys(getRevResult.involvedKeys);\n            if (getRevResult.success) {\n                text = getRevResult.revision.unpackedText();\n                if (getRevResult.page.isRedirect()) {\n                    final Matcher matcher = MATCH_WIKI_REDIRECT.matcher(text);\n                    if (matcher.matches()) {\n                        String[] redirFullName = splitNsTitle(matcher.group(1));\n                        if (followRedirect) {\n                            // see https://secure.wikimedia.org/wikipedia/en/wiki/Help:Redirect#Transclusion\n                            String redirText = retrievePage(redirFullName[0], redirFullName[1], templateParameters, false);\n                            if (redirText != null && !redirText.isEmpty()) {\n                                text = redirText;\n                            } else {\n                                text = \"<ol><li>REDIRECT [[\"\n                                        + createFullPageName(redirFullName[0],\n                                                redirFullName[1]) + \"]]</li></ol>\";\n                            }\n                        } else {\n                            // we must disarm the redirect here!\n                            text = \"<ol><li>REDIRECT [[\"\n                                    + createFullPageName(redirFullName[0],\n                                            redirFullName[1]) + \"]]</li></ol>\";\n                        }\n                    } else {\n                        // we must disarm the redirect here!\n                        System.err\n                                .println(\"Couldn't parse the redirect title of \\\"\"\n                                        + createFullPageName(namespace, articleName)\n                                        + \"\\\" in \\\"\"\n                                        + createFullPageName(\n                                                getNamespaceName(),\n                                                getPageName())\n                                        + \"\\\" from: \"\n                                        + text.substring(0, Math.min(100, text.length())));\n                        text = null;\n                    }\n                }\n            } else {\n                // NOTE: must return null for non-existing pages in order for #ifexist to work correctly!\n                // System.err.println(getRevResult.message);\n                // text = \"<b>ERROR: template \" + pageName + \" not available: \" + getRevResult.message + \"</b>\";\n            }\n            pageCache.put(pageName, text);\n            return text;\n        }\n        return null;\n    }\n    \n    protected boolean hasDBConnection() {\n        return false;\n    }\n    \n    protected RevisionResult getRevFromDB(NormalisedTitle title) {\n        return new RevisionResult(false, new ArrayList<InvolvedKey>(),\n                \"no DB connection\", true, title, null, null, false,\n                false, title.toString(), 0l);\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.model.AbstractWikiModel#encodeTitleToUrl(java.lang.String, boolean)\n     */\n    @Override\n    public String encodeTitleToUrl(String wikiTitle, boolean firstCharacterAsUpperCase) {\n        try {\n            // some links may contain '_' which needs to be translated back to ' ':\n            wikiTitle = wikiTitle.replace('_', ' ');\n            return URLEncoder.encode(wikiTitle, \"UTF-8\");\n        } catch (UnsupportedEncodingException e) {\n            return super.encodeTitleToUrl(wikiTitle, firstCharacterAsUpperCase);\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.model.WikiModel#getNamespace()\n     */\n    @Override\n    public MyNamespace getNamespace() {\n        return (MyNamespace) super.getNamespace();\n    }\n\n    /**\n     * Formats the given number using the wiki's locale.\n     * \n     * Note: Currently, the English locale is always used.\n     * \n     * @param rawNumber\n     *            whether the raw number should be returned\n     * @param number\n     *            the number\n     * \n     * @return the formatted number\n     */\n    public String formatStatisticNumber(boolean rawNumber, Number number) {\n        if (rawNumber) {\n            return number.toString();\n        } else {\n            // TODO: use locale from Wiki\n            NumberFormat nf = NumberFormat.getNumberInstance(Locale.ENGLISH);\n            nf.setGroupingUsed(true);\n            return nf.format(number);\n        }\n    }\n\n    /**\n     * Splits the given full title at the first colon.\n     * \n     * @param fullTitle\n     *            the (full) title including a namespace (if present)\n     * \n     * @return a 2-element array with the two components - the first may be\n     *         empty if no colon is found\n     */\n    protected static String[] splitAtColon(String fullTitle) {\n        int colonIndex = fullTitle.indexOf(':');\n        if (colonIndex != (-1)) {\n            return new String[] { fullTitle.substring(0, colonIndex),\n                    fullTitle.substring(colonIndex + 1) };\n        }\n        return new String[] { \"\", fullTitle };\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.model.AbstractWikiModel#appendRedirectLink(java.lang.String)\n     */\n    @Override\n    public boolean appendRedirectLink(String redirectLink) {\n        // do not add redirection if we are parsing a template:\n        if (getRecursionLevel() == 0) {\n            // remove \"#section\" from redirect links (this form of redirects is unsupported)\n            if (redirectLink != null) {\n                return super.appendRedirectLink(redirectLink.replaceFirst(\"#.*$\", \"\"));\n            }\n            return super.appendRedirectLink(null);\n        }\n        return true;\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.model.AbstractWikiModel#appendInterWikiLink(java.lang.String, java.lang.String, java.lang.String)\n     */\n    @Override\n    public void appendInterWikiLink(String namespace, String title, String linkText) {\n        appendInterWikiLink(namespace, title, linkText, true);\n    }\n    \n    protected void appendInterWikiLink(String namespace, String title, String linkText, boolean ignoreInterLang) {\n        if (INTERLANGUAGE_KEYS.contains(namespace)) {\n            // also check if this is an inter wiki link to an external wiki in another language\n            // -> only ignore inter language links to the same wiki\n            String namespace2 = splitAtColon(title)[0];\n            if (!ignoreInterLang || (!namespace2.isEmpty() && isInterWiki(namespace2))) {\n                // bliki is not able to parse language-specific interwiki links\n                // -> use default language\n                super.appendInterWikiLink(namespace2, title, linkText);\n            } else {\n                // ignore interlanguage keys\n            }\n        } else {\n            super.appendInterWikiLink(namespace, title, linkText);\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.model.WikiModel#addLink(java.lang.String)\n     */\n    @Override\n    public void addLink(String topicName) {\n        /*\n         * do not add links like [[:w:nl:User:WinContro|Dutch Wikipedia]] to\n         * the internal links\n         */\n        String[] nsTitle = splitAtColon(topicName);\n        if (!nsTitle[0].isEmpty() && isInterWiki(nsTitle[0])) {\n            appendInterWikiLink(nsTitle[0], nsTitle[1], \"\");\n        } else {\n            super.addLink(topicName);\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.model.WikiModel#appendInternalLink(java.lang.String, java.lang.String, java.lang.String, java.lang.String, boolean)\n     */\n    @Override\n    public void appendInternalLink(String topic0, String hashSection, String topicDescription,\n            String cssClass, boolean parseRecursive) {\n        /*\n         * convert links like [[:w:nl:User:WinContro|Dutch Wikipedia]] to\n         * external links if the link is an interwiki link\n         */\n        String[] nsTitle = splitAtColon(topic0);\n        if (!nsTitle[0].isEmpty() && isInterWiki(nsTitle[0])) {\n            appendInterWikiLink(nsTitle[0], nsTitle[1], topicDescription, nsTitle[1].isEmpty() && topicDescription.equals(topic0));\n        } else {\n            boolean pageExists = true;\n            if (existingPages.hasContains()) {\n                pageExists = existingPages.contains(normalisePageTitle(topic0));\n            }\n            super.appendInternalLink(topic0, hashSection, topicDescription, cssClass, parseRecursive, pageExists);\n        }\n    }\n\n    /**\n     * Normalises the given string, i.e. capitalises the first letter and\n     * replaces underscores with spaces.\n     * \n     * @param value\n     *            the string\n     * \n     * @return a normalised string\n     */\n    public static String normaliseName(final String value) {\n        return Encoder.normaliseTitle(value, true, ' ', true);\n    }\n    \n    /**\n     * Normalises the given page title by capitalising its first letter after\n     * the namespace.\n     * \n     * @param title\n     *            the original page title\n     * \n     * @return the normalised page title\n     */\n    public NormalisedTitle normalisePageTitle(final String title) {\n        return NormalisedTitle.fromUnnormalised(title, getNamespace());\n    }\n    \n    /**\n     * Normalises the given page title by capitalising its first letter after\n     * the namespace.\n     * \n     * @param maybeNs\n     *            the namespace of the page\n     * @param articleName\n     *            the (unnormalised) page's name without the namespace\n     * \n     * @return the normalised page title\n     */\n    public NormalisedTitle normalisePageTitle(final String maybeNs, final String articleName) {\n        return NormalisedTitle.fromUnnormalised(maybeNs, articleName, getNamespace());\n    }\n    \n    /**\n     * Normalises the given page title by capitalising its first letter after\n     * the namespace.\n     * \n     * @param <T>\n     * \n     * @param titles\n     *            the original page titles\n     * @param normalisedTitles\n     *            the container to write the normalised titles to\n     * \n     * @return the normalised page titles\n     */\n    public <T extends Collection<NormalisedTitle>> T normalisePageTitles(final Collection<String> titles, T normalisedTitles) {\n        return MyWikiModel.<T>normalisePageTitles(titles, getNamespace(), normalisedTitles);\n    }\n    \n    /**\n     * Normalises the given page titles by capitalising their first letter after\n     * the namespace.\n     * \n     * @param <T>\n     * \n     * @param titles\n     *            the original page titles\n     * @param nsObject\n     *            the namespace for determining how to split the title\n     * @param normalisedTitles\n     *            the container to write the normalised titles to\n     * \n     * @return the normalised page titles\n     */\n    public static <T extends Collection<NormalisedTitle>> T normalisePageTitles(final Collection<String> titles, final MyNamespace nsObject, T normalisedTitles) {\n        for (String title: titles) {\n            normalisedTitles.add(NormalisedTitle.fromUnnormalised(title, nsObject));\n        }\n        return normalisedTitles;\n    }\n    \n    /**\n     * De-normalises the given page titles.\n     * \n     * @param <T>\n     * \n     * @param titles\n     *            the normalised page titles\n     * @param denormalisedTitles\n     *            the container to write the de-normalised titles to\n     * \n     * @return the original page title\n     */\n    public <T extends Collection<String>> T denormalisePageTitles(final Collection<NormalisedTitle> titles, T denormalisedTitles) {\n        return MyWikiModel.<T>denormalisePageTitles(titles, getNamespace(), denormalisedTitles);\n    }\n    \n    /**\n     * De-normalises the given page titles.\n     * \n     * @param <T>\n     * \n     * @param titles\n     *            the normalised page titles\n     * @param nsObject\n     *            the namespace for determining how to split the title\n     * @param denormalisedTitles\n     *            the container to write the de-normalised titles to\n     * \n     * @return the original page title\n     */\n    public static <T extends Collection<String>> T denormalisePageTitles(final Collection<NormalisedTitle> titles, final MyNamespace nsObject, T denormalisedTitles) {\n        for (NormalisedTitle title: titles) {\n            denormalisedTitles.add(title.denormalise(nsObject));\n        }\n        return denormalisedTitles;\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.model.WikiModel#setUp()\n     */\n    @Override\n    public void setUp() {\n        super.setUp();\n        magicWordCache = new HashMap<String, String>();\n        pageCache = new HashMap<NormalisedTitle, String>();\n        if (renderWikiText != null) {\n            pageCache.put(normalisePageTitle(getPageName()), renderWikiText );\n        }\n    }\n\n    /**\n     * Checks whether the given namespace is a valid media namespace.\n     * \n     * @param namespace\n     *            the namespace to check\n     * \n     * @return <tt>true</tt> if it is one of the two media namespace strings\n     */\n    public boolean isMediaNamespace(String namespace) {\n        return namespace.equalsIgnoreCase(fNamespace.getMedia()) || namespace.equalsIgnoreCase(fNamespace.getMedia2());\n    }\n\n    /**\n     * Gets information about the time needed to look up pages.\n     * \n     * @return a mapping of page titles to retrieval times\n     */\n    public Map<String, List<Long>> getStats() {\n        return stats;\n    }\n\n    /**\n     * Adds the time needed to retrieve the given page to the collected\n     * statistics.\n     * \n     * @param title\n     *            the title of the page\n     * @param value\n     *            the number of milliseconds it took to retrieve the page\n     */\n    public void addStat(String title, long value) {\n        stats.put1(title, value);\n    }\n\n    /**\n     * Adds the time needed to retrieve the given page to the collected\n     * statistics.\n     * \n     * @param title\n     *            the title of the page\n     * @param value\n     *            multiple number of milliseconds it took to retrieve the page\n     */\n    public void addStats(String title, List<Long> value) {\n        stats.put(title, value);\n    }\n\n    /**\n     * Adds the time needed to retrieve the given page to the collected\n     * statistics.\n     * \n     * @param values\n     *            a mapping between page titles and the number of milliseconds\n     *            it took to retrieve the page\n     */\n    public void addStats(Map<String, List<Long>> values) {\n        stats.putAll(values);\n    }\n    \n    /**\n     * Adds an involved key to the collected statistics.\n     * \n     * @param key\n     *            the key to add\n     */\n    public void addInvolvedKey(InvolvedKey key) {\n        involvedKeys.add(key);\n    }\n    \n    /**\n     * Adds a number of involved keys to the collected statistics.\n     * \n     * @param keys\n     *            the keys to add\n     */\n    public void addInvolvedKeys(Collection<? extends InvolvedKey> keys) {\n        involvedKeys.addAll(keys);\n    }\n\n    /**\n     * Gets the list of all keys that have been read or written during the\n     * current operation.\n     * \n     * @return the involvedKeys\n     */\n    public List<InvolvedKey> getInvolvedKeys() {\n        return involvedKeys;\n    }\n    \n    /**\n     * @return the existingPages\n     */\n    public ExistingPagesCache getExistingPages() {\n        return existingPages;\n    }\n\n    /**\n     * @param existingPages the existingPages to set\n     */\n    public void setExistingPages(ExistingPagesCache existingPages) {\n        this.existingPages = existingPages;\n    }\n    \n    /**\n     * The following characters are forbidden in page titles:\n     * <tt># &lt; &gt; [ ] | { }</tt>. Any line breaks and non-printable unicode\n     * characters are also forbidden here.\n     * \n     * @param title\n     *            the title to check\n     * \n     * @return <tt>true</tt> if the title contains a forbidden character,\n     *         <tt>false</tt> otherwise\n     * \n     * @see #MATCH_WIKI_FORBIDDEN_TITLE_CHARS\n     */\n    public static boolean isValidTitle(String title) {\n        if (title == null || title.isEmpty() || title.length() >= 256) {\n            return false;\n        }\n        final Matcher matcher = MATCH_WIKI_FORBIDDEN_TITLE_CHARS.matcher(title);\n        return !matcher.matches();\n    }\n\n    /**\n     * Determines whether a page with the given properties is an article.\n     * \n     * A new page in the main namespace will be counted as an article if it\n     * contains at least one wiki link or is categorised to at least one\n     * category.\n     * \n     * @param namespace\n     *            the ID of the namespace of the page\n     * @param links\n     *            the links in this page\n     * @param categories\n     *            categories of this page\n     * \n     * @return whether the page is an article or not\n     * \n     * @see <a\n     *      href=\"https://www.mediawiki.org/wiki/Manual:Article_count\">MediaWiki\n     *      explanation</a>\n     */\n    public static boolean isArticle(int namespace, Collection<String> links,\n            Collection<String> categories) {\n        return (namespace == 0) && (!links.isEmpty() || !categories.isEmpty());\n    }\n    \n    /**\n     * Gets all localised variants for the given special page.\n     * \n     * @param page\n     *            the special page\n     * \n     * @return localised variants including the English names\n     */\n    public static Collection<String> getLocalisedSpecialPageNames(SpecialPage page) {\n        ArrayList<String> result = new ArrayList<String>();\n        \n        for (Entry<String, String> prefix : SPECIAL_PREFIX.entrySet()) {\n            EnumMap<SpecialPage, String> localisedSuffix = SPECIAL_SUFFIX.get(prefix.getKey());\n            if (localisedSuffix != null) {\n                result.add(MyWikiModel.createFullPageName(prefix.getValue(), localisedSuffix.get(page)));\n            }\n            // Also add the English suffix for the localised prefix.\n            EnumMap<SpecialPage, String> englishSuffix = SPECIAL_SUFFIX.get(\"en\");\n            if (englishSuffix != null) {\n                result.add(MyWikiModel.createFullPageName(prefix.getValue(), englishSuffix.get(page)));\n            }\n        }\n        return result;\n    }\n    \n    /**\n     * Gets the localised variants for the given special page.\n     * \n     * @param page\n     *            the special page\n     * @param language\n     *            the language to get the variants for\n     * \n     * @return localised variants including the English names\n     */\n    public static Collection<String> getLocalisedSpecialPageNames(SpecialPage page, String language) {\n        ArrayList<String> result = new ArrayList<String>(4);\n        \n        String localisedPrefix = SPECIAL_PREFIX.get(\"en\");\n        EnumMap<SpecialPage, String> localisedSuffix = SPECIAL_SUFFIX.get(\"en\");\n        result.add(MyWikiModel.createFullPageName(localisedPrefix, localisedSuffix.get(page)));\n        \n        localisedPrefix = SPECIAL_PREFIX.get(language);\n        localisedSuffix = SPECIAL_SUFFIX.get(language);\n        if (localisedPrefix != null && localisedSuffix != null) {\n            result.add(MyWikiModel.createFullPageName(localisedPrefix, localisedSuffix.get(page)));\n        }\n        return result;\n    }\n\n    /* (non-Javadoc)\n     * @see info.bliki.wiki.model.AbstractWikiModel#isImageNamespace(java.lang.String)\n     */\n    @Override\n    public boolean isImageNamespace(String namespace) {\n        return super.isImageNamespace(namespace) ||\n                ((MyNamespace) fNamespace).getNumberByName(namespace) == Namespace.FILE_NAMESPACE_KEY;\n    }\n\n    /**\n     * Renders the raw Wikipedia text into a string for a given converter\n     * (renders the wiki text as if a template topic will be displayed\n     * directly).\n     * \n     * @param converter\n     *            a text converter. <b>Note</b> the converter may be\n     *            <code>null</code>, if you only would like to analyze the raw\n     *            wiki text and don't need to convert. This speeds up the\n     *            parsing process.\n     * @param rawWikiText\n     *            a raw wiki text\n     * @return <code>null</code> if an IOException occurs or\n     *         <code>converter==null</code>\n     * \n     * @see info.bliki.wiki.model.AbstractWikiModel#render(info.bliki.wiki.filter.ITextConverter,\n     *      java.lang.String, boolean)\n     */\n    public String renderPageWithCache(ITextConverter converter, String rawWikiText) {\n        renderWikiText = rawWikiText;\n        return super.render(converter, rawWikiText, true);\n    }\n\n    /**\n     * Renders the raw Wikipedia text into an HTML string and use the default\n     * HTMLConverter (renders the wiki text as if a template topic will be\n     * displayed directly).\n     * \n     * @param rawWikiText\n     *            a raw wiki text\n     * @return <code>null</code> if an IOException occurs\n     * \n     * @see info.bliki.wiki.model.AbstractWikiModel#render(java.lang.String,\n     *      boolean)\n     */\n    public String renderPageWithCache(String rawWikiText) {\n        renderWikiText = rawWikiText;\n        return super.render(new HTMLConverter(), rawWikiText, true);\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/NormalisedTitle.java",
    "content": "package de.zib.scalaris.examples.wikipedia.bliki;\n\n/**\n * Represents a normalised title, split into its two components: namespace\n * and page title.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class NormalisedTitle {\n    /**\n     * The namespace number.\n     */\n    public final Integer namespace;\n    /**\n     * The page title without the namespace.\n     */\n    public final String title;\n    \n    /**\n     * Constructor.\n     * \n     * @param namespace\n     *            namespace number\n     * @param title\n     *            page title without the namespace\n     */\n    public NormalisedTitle(Integer namespace, String title) {\n        assert (namespace != null);\n        assert (title != null);\n        this.namespace = namespace;\n        this.title = title;\n    }\n    \n    /**\n     * Creates the full page name with namespace and title.\n     * \n     * @return <tt>namespace:title</tt>\n     */\n    @Override\n    public String toString() {\n        return namespace + \":\" + title;\n    }\n    \n    /**\n     * Creates the full page name with namespace and title.\n     * \n     * @param normTitleStr\n     *            a normalised title of the form <tt>namespace:title</tt>\n     * \n     * @return a {@link NormalisedTitle} object\n     * \n     * @throws IllegalArgumentException\n     *             if the parameter string was not a normalised title\n     */\n    public static NormalisedTitle fromNormalised(String normTitleStr) throws IllegalArgumentException {\n        int colonIndex = normTitleStr.indexOf(':');\n        if (colonIndex == (-1)) {\n            throw new IllegalArgumentException(\n                    \"no normalised title string: \" + normTitleStr);\n        }\n\n        try {\n            final Integer ns = Integer.parseInt(normTitleStr.substring(0, colonIndex));\n            final String title = normTitleStr.substring(colonIndex + 1);\n            return new NormalisedTitle(ns, title);\n        } catch (NumberFormatException e) {\n            throw new IllegalArgumentException(\n                    \"no normalised title string: \" + normTitleStr);\n        }\n    }\n    \n    /**\n     * Normalises the given page title by capitalising its first letter after\n     * the namespace.\n     * \n     * @param title\n     *            a unnormalised title\n     * @param nsObject\n     *            the namespace for determining how to split the title\n     * \n     * @return a {@link NormalisedTitle} object\n     */\n    public static NormalisedTitle fromUnnormalised(String title, final MyNamespace nsObject) {\n        String[] parts = nsObject.splitNsTitle(title);\n        return new NormalisedTitle(nsObject.getNumberByName(parts[0]), parts[1]);\n    }\n    \n    /**\n     * Normalises the given page title by capitalising its first letter after\n     * the namespace.\n     * \n     * @param maybeNs\n     *            the namespace of the page\n     * @param articleName\n     *            the (unnormalised) page's name without the namespace\n     * @param nsObject\n     *            the namespace for determining how to split the title\n     * \n     * @return the normalised page title\n     */\n    public static NormalisedTitle fromUnnormalised(final String maybeNs, final String articleName, final MyNamespace nsObject) {\n        Integer nsNumber = nsObject.getNumberByName(maybeNs);\n        if (nsNumber == null) {\n            nsNumber = MyNamespace.MAIN_NAMESPACE_KEY;\n        }\n        return new NormalisedTitle(nsNumber, MyWikiModel.normaliseName(articleName));\n    }\n    \n    /**\n     * Gets a de-normalised version of the page title, including the\n     * namespace.\n     * \n     * @param nsObject\n     *            the namespace object for determining the string of the\n     *            namespace id\n     * \n     * @return de-normalised <tt>namespace:title</tt> or <tt>title</tt>\n     */\n    public String denormalise(final MyNamespace nsObject) {\n        return MyWikiModel.createFullPageName(nsObject.getNamespaceByNumber(namespace),\n                title);\n    }\n    \n    @Override\n    public boolean equals(Object obj) {\n        if (this == obj) {\n            return true;\n        }\n        if (!(obj instanceof NormalisedTitle)) {\n            return false;\n        }\n        \n        NormalisedTitle obj2 = (NormalisedTitle) obj;\n        return this.namespace.equals(obj2.namespace) &&\n                this.title.equals(obj2.title);\n    }\n    \n    @Override\n    public int hashCode() {\n        return toString().hashCode();\n    }\n}"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/WikiPageBean.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\n\nimport java.util.Calendar;\nimport java.util.Collection;\nimport java.util.GregorianCalendar;\nimport java.util.LinkedHashSet;\nimport java.util.LinkedList;\nimport java.util.List;\nimport java.util.Set;\n\nimport de.zib.scalaris.examples.wikipedia.data.ShortRevision;\n\n/**\n * Bean with the content to display in the jsp. \n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class WikiPageBean extends WikiPageBeanBase {\n    private Set<String> categories = new LinkedHashSet<String>();\n    /**\n     * signals that the requested page was not available\n     * (maybe a fallback-page is shown, but the original one does not exist)\n     */\n    private boolean notAvailable = false;\n    \n    /**\n     * represents the date of the revision (last page change)\n     */\n    private Calendar date = new GregorianCalendar();\n    \n    private List<ShortRevision> revisions = new LinkedList<ShortRevision>();\n    \n    private Collection<String> subCategories = new LinkedList<String>();\n    private Collection<String> categoryPages = new LinkedList<String>();\n    \n    private String contentSub = \"\";\n\n    /**\n     * Creates an empty page bean.\n     */\n    public WikiPageBean() {\n        super();\n    }\n\n    /**\n     * Creates a new (empty) bean with the given start time.\n     * \n     * @param serviceUser\n     *            service user\n     * @param startTime\n     *            the time when the request reached the servlet (in ms)\n     */\n    public WikiPageBean(String serviceUser, long startTime) {\n        super(serviceUser, startTime);\n    }\n    \n    /**\n     * Creates a page bean from a given {@link WikiPageBeanBase}.\n     * \n     * @param other\n     *            the page bean to copy properties from\n     */\n    public WikiPageBean(WikiPageBeanBase other) {\n        super(other);\n    }\n\n    /**\n     * returns whether the originally requested page is available\n     * \n     * @return the availability status\n     */\n    public boolean isNotAvailable() {\n        return notAvailable;\n    }\n\n    /**\n     * sets that the originally requested page is not available\n     * \n     * @param notAvailable the status to set\n     */\n    public void setNotAvailable(boolean notAvailable) {\n        this.notAvailable = notAvailable;\n    }\n\n    /**\n     * returns the date of the currently shown revision\n     * \n     * @return the date\n     */\n    public Calendar getDate() {\n        return date;\n    }\n\n    /**\n     * sets the 'last changed' date of the page\n     * \n     * @param date the date\n     */\n    public void setDate(Calendar date) {\n        this.date = date;\n    }\n\n    /**\n     * @return the categories\n     */\n    public Set<String> getCategories() {\n        return categories;\n    }\n\n    /**\n     * @param categories the categories to set\n     */\n    public void setCategories(Set<String> categories) {\n        this.categories = categories;\n    }\n\n    /**\n     * @return the revisions\n     */\n    public List<ShortRevision> getRevisions() {\n        return revisions;\n    }\n\n    /**\n     * @param revisions the revisions to set\n     */\n    public void setRevisions(List<ShortRevision> revisions) {\n        this.revisions = revisions;\n    }\n\n    /**\n     * @return the subCategories\n     */\n    public Collection<String> getSubCategories() {\n        return subCategories;\n    }\n\n    /**\n     * @param subCategories the subCategories to set\n     */\n    public void setSubCategories(Collection<String> subCategories) {\n        this.subCategories = subCategories;\n    }\n\n    /**\n     * @return the categoryPages\n     */\n    public Collection<String> getCategoryPages() {\n        return categoryPages;\n    }\n\n    /**\n     * @param categoryPages the categoryPages to set\n     */\n    public void setCategoryPages(Collection<String> categoryPages) {\n        this.categoryPages = categoryPages;\n    }\n    \n    /**\n     * @return the contentSub\n     */\n    public String getContentSub() {\n        return contentSub;\n    }\n\n    /**\n     * @param contentSub the contentSub to set\n     */\n    public void setContentSub(String contentSub) {\n        this.contentSub = contentSub;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/WikiPageBeanBase.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport java.util.ArrayList;\nimport java.util.List;\nimport java.util.Map;\n\nimport de.zib.scalaris.examples.wikipedia.InvolvedKey;\nimport de.zib.tools.LinkedMultiHashMap;\n\n/**\n * Bean with common content to display in a jsp. \n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class WikiPageBeanBase {\n\n    /**\n     * the service user\n     */\n    private String serviceUser = \"\";\n    /**\n     * the title of the site\n     */\n    private String title = \"\";\n    /**\n     * Version of the page (the revision id)\n     */\n    private int version = 0;\n    private String notice = \"\";\n    private String error = \"\";\n    private String wikiTitle = \"Wikipedia\";\n    private String wikiLang = \"en\";\n    private String wikiLangDir = \"ltr\";\n    private MyNamespace wikiNamespace = new MyNamespace();\n    private boolean isEditRestricted = false;\n    protected LinkedMultiHashMap<String, Long> stats = new LinkedMultiHashMap<String, Long>();\n    private long startTime;\n    \n    /**\n     * the content of the site\n     */\n    private String page = \"\";\n    /**\n     * number of attempts of saving a wiki page\n     */\n    private int saveAttempts = 0;\n    /**\n     * All keys that have been read or written during the current operation.\n     */\n    public List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n    /**\n     * In cases of failed page-save commits, contains a list of failed keys for\n     * each save attempt.\n     */\n    protected LinkedMultiHashMap<Integer, String> failedKeys = new LinkedMultiHashMap<Integer, String>();\n\n    /**\n     * Creates a new (empty) bean.\n     */\n    public WikiPageBeanBase() {\n        super();\n        startTime = System.currentTimeMillis();\n    }\n\n    /**\n     * Creates a new (empty) bean with the given start time.\n     * \n     * @param serviceUser\n     *            service user\n     * @param startTime\n     *            the time when the request reached the servlet (in ms)\n     */\n    public WikiPageBeanBase(String serviceUser, long startTime) {\n        super();\n        this.serviceUser = serviceUser;\n        this.startTime = startTime;\n    }\n    \n    /**\n     * Creates a page bean from a given {@link WikiPageBeanBase}.\n     * \n     * @param other\n     *            the page bean to copy properties from\n     */\n    public WikiPageBeanBase(WikiPageBeanBase other) {\n        super();\n        this.serviceUser = other.serviceUser;\n        this.title = other.title;\n        version = other.version;\n        notice = other.notice;\n        error = other.error;\n        wikiTitle = other.wikiTitle;\n        wikiLang = other.wikiLang;\n        wikiLangDir = other.wikiLangDir;\n        wikiNamespace = other.wikiNamespace;\n        isEditRestricted = other.isEditRestricted;\n        stats = new LinkedMultiHashMap<String, Long>(other.stats);\n        startTime = other.startTime;\n    }\n\n    /**\n     * gets the page content\n     * \n     * @return the content\n     */\n    public String getPage() {\n        return page;\n    }\n\n    /**\n     * sets the page content\n     * \n     * @param page\n     *            the content\n     */\n    public void setPage(String page) {\n        this.page = page;\n    }\n\n    /**\n     * @return the notice\n     */\n    public String getNotice() {\n        return notice;\n    }\n\n    /**\n     * @param notice the notice to set\n     */\n    public void setNotice(String notice) {\n        this.notice = notice;\n    }\n\n    /**\n     * gets the page title\n     * \n     * @return the title\n     */\n    public String getTitle() {\n        return title;\n    }\n\n    /**\n     * sets the page title\n     * \n     * @param title\n     *            the title\n     */\n    public void setTitle(String title) {\n        this.title = title;\n    }\n\n    /**\n     * returns the wiki version of the page\n     * \n     * @return the version\n     */\n    public int getVersion() {\n        return version;\n    }\n\n    /**\n     * sets the wiki version of the page\n     * \n     * @param version the version to set\n     */\n    public void setVersion(int version) {\n        this.version = version;\n    }\n\n    /**\n     * @return the wikiLang\n     */\n    public String getWikiLang() {\n        return wikiLang;\n    }\n\n    /**\n     * @param wikiLang the wikiLang to set\n     */\n    public void setWikiLang(String wikiLang) {\n        this.wikiLang = wikiLang;\n    }\n\n    /**\n     * @return the wikiLangDir\n     */\n    public String getWikiLangDir() {\n        return wikiLangDir;\n    }\n\n    /**\n     * @param wikiLangDir the wikiLangDir to set\n     */\n    public void setWikiLangDir(String wikiLangDir) {\n        this.wikiLangDir = wikiLangDir;\n    }\n\n    /**\n     * @return the wikiTitle\n     */\n    public String getWikiTitle() {\n        return wikiTitle;\n    }\n\n    /**\n     * @param wikiTitle the wikiTitle to set\n     */\n    public void setWikiTitle(String wikiTitle) {\n        this.wikiTitle = wikiTitle;\n    }\n\n    /**\n     * @return the wikiTalkNamespace\n     */\n    public MyNamespace getWikiNamespace() {\n        return wikiNamespace;\n    }\n\n    /**\n     * @param wikiNamespace the wikiTalkNamespace to set\n     */\n    public void setWikiNamespace(MyNamespace wikiNamespace) {\n        this.wikiNamespace = wikiNamespace;\n    }\n\n    /**\n     * @return the isEditRestricted\n     */\n    public boolean isEditRestricted() {\n        return isEditRestricted;\n    }\n\n    /**\n     * @param isEditRestricted the isEditRestricted to set\n     */\n    public void setEditRestricted(boolean isEditRestricted) {\n        this.isEditRestricted = isEditRestricted;\n    }\n\n    /**\n     * Gets information about the time needed to look up pages.\n     * \n     * @return a mapping of page titles to retrieval times\n     */\n    public Map<String, List<Long>> getStats() {\n        return stats;\n    }\n\n    /**\n     * @param stats the stats to set\n     */\n    public void setStats(Map<String, List<Long>> stats) {\n        this.stats = new LinkedMultiHashMap<String, Long>(stats);\n    }\n\n    /**\n     * Adds the time needed to retrieve the given page to the collected\n     * statistics.\n     * \n     * @param title\n     *            the title of the page\n     * @param value\n     *            the number of milliseconds it took to retrieve the page\n     */\n    public void addStat(String title, long value) {\n        stats.put1(title, value);\n    }\n\n    /**\n     * Adds the time needed to retrieve the given page to the collected\n     * statistics.\n     * \n     * @param title\n     *            the title of the page\n     * @param value\n     *            multiple number of milliseconds it took to retrieve the page\n     */\n    public void addStats(String title, List<Long> value) {\n        stats.put(title, value);\n    }\n\n    /**\n     * Adds the time needed to retrieve the given page to the collected\n     * statistics.\n     * \n     * @param values\n     *            a mapping between page titles and the number of milliseconds\n     *            it took to retrieve the page\n     */\n    public void addStats(Map<String, List<Long>> values) {\n        stats.putAll(values);\n    }\n\n    /**\n     * @return the error\n     */\n    public String getError() {\n        return error;\n    }\n\n    /**\n     * @param error the error to set\n     */\n    public void setError(String error) {\n        this.error = error;\n    }\n\n    /**\n     * @return the saveAttempts\n     */\n    public int getSaveAttempts() {\n        return saveAttempts;\n    }\n\n    /**\n     * @param saveAttempts the saveAttempts to set\n     */\n    public void setSaveAttempts(int saveAttempts) {\n        this.saveAttempts = saveAttempts;\n    }\n\n    /**\n     * @return the failedKeys\n     */\n    public LinkedMultiHashMap<Integer, String> getFailedKeys() {\n        return failedKeys;\n    }\n\n    /**\n     * @param failedKeys the failedKeys to set\n     */\n    public void setFailedKeys(LinkedMultiHashMap<Integer, String> failedKeys) {\n        this.failedKeys = failedKeys;\n    }\n\n    /**\n     * @return the involvedKeys\n     */\n    public List<InvolvedKey> getInvolvedKeys() {\n        return involvedKeys;\n    }\n\n    /**\n     * @param involvedKeys the involvedKeys to set\n     */\n    public void setInvolvedKeys(List<InvolvedKey> involvedKeys) {\n        this.involvedKeys = involvedKeys;\n    }\n\n    /**\n     * @return the startTime\n     */\n    public long getStartTime() {\n        return startTime;\n    }\n\n    /**\n     * @param startTime the startTime to set\n     */\n    public void setStartTime(long startTime) {\n        this.startTime = startTime;\n    }\n\n    /**\n     * @return the serviceUser\n     */\n    public String getServiceUser() {\n        return serviceUser;\n    }\n\n    /**\n     * @param serviceUser the serviceUser to set\n     */\n    public void setServiceUser(String serviceUser) {\n        this.serviceUser = serviceUser;\n    }\n}"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/WikiPageEditBean.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport java.util.HashSet;\nimport java.util.Set;\n\n/**\n * Bean with the content to display in the jsp (only for editing articles).\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class WikiPageEditBean extends WikiPageBeanBase {\n    /**\n     * the page did not exist before\n     */\n    private boolean isNewPage = false;\n    \n    /**\n     * the preview part of the site\n     */\n    private String preview = \"\";\n    \n    /**\n     * set of templates for the review content\n     */\n    private Set<String> templates = new HashSet<String>();\n    \n    /**\n     * set of transcludes (other than templates) for the review content\n     */\n    private Set<String> includes = new HashSet<String>();\n    \n    /**\n     * the summary field of the site\n     */\n    private String summary = \"\";\n    \n    /**\n     * Creates a new (empty) bean.\n     */\n    public WikiPageEditBean() {\n        super();\n    }\n\n    /**\n     * Creates a new (empty) bean with the given start time.\n     * \n     * @param serviceUser\n     *            service user\n     * @param startTime\n     *            the time when the request reached the servlet (in ms)\n     */\n    public WikiPageEditBean(String serviceUser, long startTime) {\n        super(serviceUser, startTime);\n    }\n\n    /**\n     * Creates a page bean from a given {@link WikiPageBeanBase}.\n     * \n     * @param other\n     *            the page bean to copy properties from\n     */\n    public WikiPageEditBean(WikiPageBeanBase other) {\n        super(other);\n    }\n\n    /**\n     * @return the preview\n     */\n    public String getPreview() {\n        return preview;\n    }\n\n    /**\n     * @param preview the preview to set\n     */\n    public void setPreview(String preview) {\n        this.preview = preview;\n    }\n\n    /**\n     * @return the summary\n     */\n    public String getSummary() {\n        return summary;\n    }\n\n    /**\n     * @param summary the summary to set\n     */\n    public void setSummary(String summary) {\n        this.summary = summary;\n    }\n\n    /**\n     * @return the isNewPage\n     */\n    public boolean isNewPage() {\n        return isNewPage;\n    }\n\n    /**\n     * @param isNewPage the isNewPage to set\n     */\n    public void setNewPage(boolean isNewPage) {\n        this.isNewPage = isNewPage;\n    }\n\n    /**\n     * @param templates\n     */\n    public void setTemplates(Set<String> templates) {\n        this.templates = templates;\n    }\n\n    /**\n     * @return the templates\n     */\n    public Set<String> getTemplates() {\n        return templates;\n    }\n\n    /**\n     * @param includes\n     */\n    public void setIncludes(Set<String> includes) {\n        this.includes = includes;\n    }\n\n    /**\n     * @return the includes\n     */\n    public Set<String> getIncludes() {\n        return includes;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/WikiPageListBean.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport java.io.UnsupportedEncodingException;\nimport java.net.URLEncoder;\nimport java.util.Collection;\nimport java.util.LinkedList;\n\n/**\n * Generic bean for lists of pages, i.e. page titles.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class WikiPageListBean extends WikiPageBeanBase {\n    /**\n     * Distinguishes between different form types on the page lists.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public enum FormType {\n        /**\n         * Do not show a form.\n         */\n        NoForm,\n        /**\n         * Form with From and To fields.\n         */\n        FromToForm,\n        /**\n         * Form for a single page.\n         */\n        TargetPageForm,\n        /**\n         * Form for pages with a given prefix.\n         */\n        PagePrefixForm,\n        /**\n         * Form for pages with a given substring (for searching).\n         */\n        PageSearchForm\n    }\n    private Collection<String> pages = new LinkedList<String>();\n\n    private String fromPage = \"\";\n    private String toPage = \"\";\n    private String formTitle = \"Pages\";\n    private FormType formType = FormType.NoForm;\n    /**\n     * Title for the heading of the page.\n     */\n    private String pageHeading = \"\";\n    \n    private String target = \"\";\n    private String prefix = \"\";\n    private String search = \"\";\n    private boolean foundFullMatch = false;\n    private int namespaceId = 0;\n    private boolean showAllPages = false;\n\n    /**\n     * Creates a new (empty) bean.\n     */\n    public WikiPageListBean() {\n        super();\n    }\n\n    /**\n     * Creates a new (empty) bean with the given start time.\n     * \n     * @param serviceUser\n     *            service user\n     * @param startTime\n     *            the time when the request reached the servlet (in ms)\n     */\n    public WikiPageListBean(String serviceUser, long startTime) {\n        super(serviceUser, startTime);\n    }\n\n    /**\n     * Creates a page bean from a given {@link WikiPageBeanBase}.\n     * \n     * @param other\n     *            the page bean to copy properties from\n     */\n    public WikiPageListBean(WikiPageBeanBase other) {\n        super(other);\n    }\n\n    /**\n     * @return the subCategories\n     */\n    public Collection<String> getPages() {\n        return pages;\n    }\n\n    /**\n     * @param pages the pages to set\n     */\n    public void setPages(Collection<String> pages) {\n        this.pages = pages;\n    }\n\n    /**\n     * @return the fromChar\n     */\n    public String getFromPage() {\n        return fromPage;\n    }\n\n    /**\n     * @param fromPage the fromPage to set\n     */\n    public void setFromPage(String fromPage) {\n        this.fromPage = fromPage;\n    }\n\n    /**\n     * @return the toChar\n     */\n    public String getToPage() {\n        return toPage;\n    }\n\n    /**\n     * @param toPage the toPage to set\n     */\n    public void setToPage(String toPage) {\n        this.toPage = toPage;\n    }\n\n    /**\n     * @return the formTitle\n     */\n    public String getFormTitle() {\n        return formTitle;\n    }\n\n    /**\n     * @param formTitle the formTitle to set\n     */\n    public void setFormTitle(String formTitle) {\n        this.formTitle = formTitle;\n    }\n    \n    /**\n     * @return the formType\n     */\n    public FormType getFormType() {\n        return formType;\n    }\n\n    /**\n     * @param formType the formType to set\n     */\n    public void setFormType(FormType formType) {\n        this.formType = formType;\n    }\n\n    /**\n     * @return the pageTitle\n     */\n    public String getPageHeading() {\n        return pageHeading;\n    }\n\n    /**\n     * @param pageHeading the pageTitle to set\n     */\n    public void setPageHeading(String pageHeading) {\n        this.pageHeading = pageHeading;\n    }\n\n    /**\n     * @return the target\n     */\n    public String getTarget() {\n        return target;\n    }\n\n    /**\n     * @param target the target to set\n     */\n    public void setTarget(String target) {\n        this.target = target;\n    }\n\n    /**\n     * @return the prefix\n     */\n    public String getPrefix() {\n        return prefix;\n    }\n\n    /**\n     * @param prefix the prefix to set\n     */\n    public void setPrefix(String prefix) {\n        this.prefix = prefix;\n    }\n\n    /**\n     * @return the namespaceId\n     */\n    public int getNamespaceId() {\n        return namespaceId;\n    }\n\n    /**\n     * @param namespaceId the namespaceId to set\n     */\n    public void setNamespaceId(int namespaceId) {\n        this.namespaceId = namespaceId;\n    }\n\n    /**\n     * @return the search\n     */\n    public String getSearch() {\n        return search;\n    }\n\n    /**\n     * @param search the search to set\n     */\n    public void setSearch(String search) {\n        this.search = search;\n    }\n\n    /**\n     * @return the showAllPages\n     */\n    public boolean isShowAllPages() {\n        return showAllPages;\n    }\n\n    /**\n     * @param showAllPages the showAllPages to set\n     */\n    public void setShowAllPages(boolean showAllPages) {\n        this.showAllPages = showAllPages;\n    }\n    \n    /**\n     * Gets a version of the title string with all parameters needed to\n     * re-create the form.\n     * \n     * Note: Form parameters are URL-encoded, the \"&\" connecting them are not!\n     * \n     * @return a title string with all parameters to be used in a URL\n     */\n    public String titleWithParameters() {\n        try {\n        String title = URLEncoder.encode(getTitle(), \"UTF-8\");\n            switch (formType) {\n                case PageSearchForm:\n                    if (search.isEmpty()) {\n                        return title + \"&namespace=\" + namespaceId;\n                    } else {\n                        return title \n                                + \"&search=\" + URLEncoder.encode(search, \"UTF-8\")\n                                + \"&namespace=\" + namespaceId;\n                    }\n                case FromToForm:\n                    if (!showAllPages && fromPage.isEmpty() && toPage.isEmpty()) {\n                        return title + \"&namespace=\" + namespaceId;\n                    } else {\n                        return title\n                                + \"&from=\" + URLEncoder.encode(fromPage, \"UTF-8\")\n                                + \"&to=\" + URLEncoder.encode(toPage, \"UTF-8\")\n                                + \"&namespace=\" + namespaceId;\n                    }\n                case PagePrefixForm:\n                    if (!showAllPages && prefix.isEmpty()) {\n                        return title + \"&namespace=\" + namespaceId;\n                    } else {\n                        return title\n                                + \"&prefix=\" + URLEncoder.encode(prefix, \"UTF-8\")\n                                + \"&namespace=\" + namespaceId;\n                    }\n                case TargetPageForm:\n                    if (!showAllPages && target.isEmpty()) {\n                        return title;\n                    } else {\n                        return title\n                                + \"&target=\" + URLEncoder.encode(target, \"UTF-8\");\n                    }\n                default:\n            }\n            return title;\n        } catch (UnsupportedEncodingException e) {\n            return \"\";\n        }\n    }\n\n    /**\n     * @return the searchFoundMatch\n     */\n    public boolean isFoundFullMatch() {\n        return foundFullMatch;\n    }\n\n    /**\n     * @param foundFullMatch the foundFullMatch to set\n     */\n    public void setFoundFullMatch(boolean foundFullMatch) {\n        this.foundFullMatch = foundFullMatch;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/WikiServlet.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport info.bliki.api.Connector;\nimport info.bliki.api.User;\nimport info.bliki.wiki.model.Configuration;\n\nimport java.io.IOException;\nimport java.io.PrintWriter;\nimport java.io.UnsupportedEncodingException;\nimport java.math.BigInteger;\nimport java.net.URLEncoder;\nimport java.util.ArrayList;\nimport java.util.Arrays;\nimport java.util.Calendar;\nimport java.util.EnumMap;\nimport java.util.Enumeration;\nimport java.util.HashMap;\nimport java.util.Iterator;\nimport java.util.LinkedHashMap;\nimport java.util.LinkedList;\nimport java.util.List;\nimport java.util.Map;\nimport java.util.Map.Entry;\nimport java.util.Random;\nimport java.util.Set;\nimport java.util.TimeZone;\nimport java.util.TreeSet;\nimport java.util.concurrent.Executors;\nimport java.util.concurrent.ScheduledExecutorService;\nimport java.util.concurrent.TimeUnit;\nimport java.util.regex.Matcher;\nimport java.util.regex.Pattern;\n\nimport javax.servlet.RequestDispatcher;\nimport javax.servlet.Servlet;\nimport javax.servlet.ServletConfig;\nimport javax.servlet.ServletException;\nimport javax.servlet.http.HttpServlet;\nimport javax.servlet.http.HttpServletRequest;\nimport javax.servlet.http.HttpServletResponse;\n\nimport org.apache.commons.lang.StringEscapeUtils;\nimport org.apache.commons.lang.StringUtils;\n\nimport de.zib.scalaris.examples.wikipedia.InvolvedKey;\nimport de.zib.scalaris.examples.wikipedia.NamespaceUtils;\nimport de.zib.scalaris.examples.wikipedia.Options;\nimport de.zib.scalaris.examples.wikipedia.PageHistoryResult;\nimport de.zib.scalaris.examples.wikipedia.RevisionResult;\nimport de.zib.scalaris.examples.wikipedia.SavePageResult;\nimport de.zib.scalaris.examples.wikipedia.ValueResult;\nimport de.zib.scalaris.examples.wikipedia.WikiServletContext;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyNamespace.NamespaceEnum;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyWikiModel.SpecialPage;\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiPageListBean.FormType;\nimport de.zib.scalaris.examples.wikipedia.data.Contributor;\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\nimport de.zib.scalaris.examples.wikipedia.data.xml.WikiDump;\nimport de.zib.scalaris.examples.wikipedia.plugin.PluginClassLoader;\nimport de.zib.scalaris.examples.wikipedia.plugin.WikiEventHandler;\nimport de.zib.scalaris.examples.wikipedia.plugin.WikiPlugin;\nimport de.zib.scalaris.examples.wikipedia.tomcat.URLParamEncoder;\nimport de.zib.tools.CircularByteArrayOutputStream;\n\n/**\n * Servlet for handling wiki page display and editing.\n * \n * @param <Connection> connection to a DB\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic abstract class WikiServlet<Connection> extends HttpServlet implements\n        Servlet, WikiServletContext, WikiServletDataHandler<Connection> {\n    protected static final String MAIN_PAGE = \"Main Page\";\n    protected static final int IMPORT_REDIRECT_EVERY = 5; // seconds\n\n    private static final long serialVersionUID = 1L;\n    \n    /**\n     * Version of the \"Wikipedia on Scalaris\" example implementation.\n     */\n    public static final String version = \"0.9.0+git\";\n\n    protected SiteInfo siteinfo = null;\n    protected MyNamespace namespace = null;\n    \n    protected boolean initialized = false;\n\n    protected static final Pattern MATCH_WIKI_AUTOIMPORT_FILE = Pattern.compile(\".*\\\\.db.auto$\");\n    protected static final Pattern MATCH_WIKI_IMPORT_FILE = Pattern.compile(\".*((\\\\.xml(\\\\.gz|\\\\.bz2)?)|\\\\.db)$\");\n    protected static final Pattern MATCH_WIKI_IMAGE_PX = Pattern.compile(\"^[0-9]*px-\");\n    protected static final Pattern MATCH_WIKI_IMAGE_SVG_PNG = Pattern.compile(\"\\\\.svg\\\\.png$\");\n    /*\n     * http://simple.wiktionary.org/wiki/Main_Page\n     * http://bar.wikipedia.org/wiki/Hauptseitn\n     * https://secure.wikimedia.org/wikipedia/en/wiki/Main_Page\n     */\n    protected static final Pattern MATCH_WIKI_SITE_BASE = Pattern.compile(\"^(http[s]?://.+)(/wiki/.*)$\");\n    \n    protected String currentImport = \"\";\n\n    protected static CircularByteArrayOutputStream importLog = null;\n    protected WikiDump importHandler = null;\n    \n    protected List<WikiEventHandler> eventHandlers = new LinkedList<WikiEventHandler>();\n    \n    protected ExistingPagesCache existingPages = ExistingPagesCache.createCache(100);\n\n    protected static final EnumMap<SpecialPage, String> SPECIAL_SUFFIX_EN = MyWikiModel.SPECIAL_SUFFIX.get(\"en\");\n    protected EnumMap<SpecialPage, String> SPECIAL_SUFFIX_LANG;\n\n    /**\n     * Full list of normalised special page titles for {@link #existingPages}. \n     */\n    protected final List<NormalisedTitle> specialPages;\n    \n    protected static LinkedList<Map<String, Object>>[] userReqLogs = null;\n    protected static int curReqLog = 0;\n    protected static int curReqLogStartTime = 0;\n\n    /**\n     * Creates the servlet. \n     */\n    public WikiServlet() {\n        super();\n        SPECIAL_SUFFIX_LANG = SPECIAL_SUFFIX_EN;\n        specialPages = new ArrayList<NormalisedTitle>(SPECIAL_SUFFIX_EN.size() * 2);\n        // add English names here - the localised versions will be added when\n        // the siteinfo is loaded\n        for (String suffix : SPECIAL_SUFFIX_EN.values()) {\n            // note: if non-english namespace, \"Special\" won't be recognised\n            // during normalisation -> leave it unnormalised\n            specialPages.add(new NormalisedTitle(\n                    MyNamespace.MAIN_NAMESPACE_KEY,\n                    MyWikiModel.createFullPageName(\n                            MyWikiModel.SPECIAL_PREFIX.get(\"en\"),\n                            suffix)));\n        }\n    }\n\n    /**\n     * Servlet initialisation: imports options from the servlet info, and\n     * initialises it.\n     */\n    @SuppressWarnings(\"unchecked\")\n    @Override\n    public final void init(ServletConfig config) throws ServletException {\n        super.init(config);\n        readOptionsFromConfig(config);\n        \n        init2(config);\n        \n        loadSiteInfo();\n        loadPlugins(config);\n        startExistingPagesUpdate();\n        existingPages.addAll(specialPages);\n        if (Options.getInstance().LOG_USER_REQS > 0) {\n            userReqLogs = new LinkedList[Options.getInstance().LOG_USER_REQS];\n            for (int i = 0; i < userReqLogs.length; ++i) {\n                userReqLogs[i] = new LinkedList<Map<String, Object>>();\n            }\n            // integer resolution should be enough\n            curReqLogStartTime = (int) (System.currentTimeMillis() / 1000);\n        }\n        \n        startAutoImport();\n    }\n\n    /**\n     * Start automatically importing data right after initialisation in\n     * {@link #init()} (if implemented in sub-class).\n     */\n    protected void startAutoImport() {\n    }\n\n    /**\n     * Extracts servlet parameters from its config into the {@link Options}\n     * class.\n     * \n     * @param config\n     *            servlet config\n     */\n    protected void readOptionsFromConfig(ServletConfig config) {\n        final Options options = Options.getInstance();\n        Options.parseOptions(options,\n                config.getInitParameter(\"SERVERNAME\"),\n                config.getInitParameter(\"SERVERPATH\"),\n                config.getInitParameter(\"WIKI_USE_BACKLINKS\"),\n                config.getInitParameter(\"WIKI_SAVEPAGE_RETRIES\"),\n                config.getInitParameter(\"WIKI_SAVEPAGE_RETRY_DELAY\"),\n                config.getInitParameter(\"WIKI_PAGES_CACHE_IMPL\"),\n                config.getInitParameter(\"WIKI_REBUILD_PAGES_CACHE\"),\n                config.getInitParameter(\"WIKI_STORE_CONTRIBUTIONS\"),\n                config.getInitParameter(\"WIKI_OPTIMISATIONS\"),\n                config.getInitParameter(\"LOG_USER_REQS\"),\n                config.getInitParameter(\"SCALARIS_NODE_DISCOVERY\"));\n        System.out.println(\"Effective optimisations: \" + options.OPTIMISATIONS.toString());\n    }\n    \n    /**\n     * Servlet initialisation, phase 2: this is executed directly after\n     * importing the servlet config in {@link #init(ServletConfig)}. Overwrite\n     * in sub-classes if needed, e.g. to setup the DB connection.\n     */\n    protected void init2(ServletConfig config) throws ServletException {\n    }\n    \n    /**\n     * Loads the siteinfo object.\n     * \n     * @return <tt>true</tt> on success,\n     *         <tt>false</tt> if not found or no connection available\n     */\n    abstract protected boolean loadSiteInfo();\n    \n    /**\n     * Load all plugins from the plugin directory\n     * <tt>&lt;ServletContextDir&gt;/WEB-INF/plugins</tt>.\n     * \n     * @param config\n     *            servlet config\n     * \n     * @return <tt>true</tt> on success, <tt>false</tt> otherwise\n     */\n    @SuppressWarnings(\"unchecked\")\n    protected synchronized boolean loadPlugins(ServletConfig config) {\n        final String pluginDir = getServletContext().getRealPath(\"/WEB-INF/plugins\");\n        try {\n            PluginClassLoader pcl = new PluginClassLoader(pluginDir, new Class[] {WikiPlugin.class});\n            List<Class<?>> plugins = pcl.getClasses(WikiPlugin.class);\n            if (plugins != null) {\n                for (Class<?> clazz: plugins) {\n                    WikiPlugin plugin;\n                    try {\n                        plugin = ((Class<WikiPlugin>) clazz).newInstance();\n                        plugin.init(this, config);\n                    } catch (Exception e) {\n                        System.err.println(\"failed to load plugin \" + clazz.getCanonicalName());\n                        e.printStackTrace();\n                        continue;\n                    }\n                }\n            }\n        } catch (IOException e) {\n            System.err.println(\"failed to load plugins\");\n            e.printStackTrace();\n            return false;\n        }\n        return true;\n    }\n\n    /**\n     * Starts the service updating the bloom filter for existing pages.\n     */\n    protected void startExistingPagesUpdate() {\n        final int rebuildDelay = Options.getInstance().WIKI_REBUILD_PAGES_CACHE;\n        if (rebuildDelay > 0) {\n            updateExistingPages();\n            ScheduledExecutorService ses = Executors.newScheduledThreadPool(1);\n            ses.scheduleWithFixedDelay(new Runnable() {\n                @Override\n                public void run() {\n                    updateExistingPages();\n                }\n            }, rebuildDelay, rebuildDelay, TimeUnit.SECONDS);\n        }\n    }\n    \n    /**\n     * Sets localised special page names by using the information provided by\n     * the {@link #siteinfo} object.\n     * \n     * Call this method inside {@link #loadSiteInfo()} in implementing classes.\n     */\n    protected void setLocalisedSpecialPageNames() {\n        if (initialized) {\n            String lang = siteinfo.extractLang();\n            final EnumMap<SpecialPage, String> specialSuffixLang = MyWikiModel.SPECIAL_SUFFIX.get(lang);\n            if (specialSuffixLang != null) {\n                SPECIAL_SUFFIX_LANG = specialSuffixLang;\n            }\n        }\n        // note: we always need to add these suffixes, even if lang == \"en\"\n        // because English suffixes have only been added as un-normalised titles\n        // before!\n        for (String suffix : SPECIAL_SUFFIX_LANG.values()) {\n            specialPages.add(new NormalisedTitle(MyNamespace.SPECIAL_NAMESPACE_KEY, suffix));\n        }\n        existingPages.addAll(specialPages);\n    }\n\n    /**\n     * Sets up the connection to the DB server.\n     * \n     * In case of errors, the <tt>error</tt> and <tt>notice</tt> attributes of\n     * the <tt>request</tt> object are set appropriately if not <tt>null</tt>.\n     * \n     * @param request\n     *            the request to the servlet (may be <tt>null</tt>)\n     * \n     * @return a valid connection of <tt>null</tt> if an error occurred\n     */\n    abstract protected Connection getConnection(HttpServletRequest request);\n\n    /**\n     * Releases the connection to the DB server, e.g. closes it.\n     * \n     * @param request\n     *            the request to the servlet\n     * @param conn\n     *            the connection to release\n     */\n    abstract protected void releaseConnection(HttpServletRequest request, Connection conn);\n\n    @Override\n    public void destroy() {\n    }\n\n    /*\n     * (non-Javadoc)\n     * \n     * @see javax.servlet.http.HttpServlet#doGet(HttpServletRequest request,\n     *      HttpServletResponse response)\n     */\n    @Override\n    protected void doGet(HttpServletRequest request,\n            HttpServletResponse response) throws ServletException, IOException {\n        long startTime = System.currentTimeMillis();\n        String image = request.getParameter(\"get_image\");\n        if (image != null) {\n            showImage(request, response, image);\n            return;\n        }\n\n        final String serviceUser = getParam(request, \"service_user\");\n        \n        Connection connection = getConnection(request);\n        if (connection == null) {\n            response.setStatus(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);\n            showEmptyPage(request, response, connection, new WikiPageBean(serviceUser, startTime)); // should forward to another page\n            return; // return just in case\n        }\n        \n        for (WikiEventHandler handler: eventHandlers) {\n            if (!handler.checkAccess(serviceUser, request, connection)) {\n                // access not allowed\n                showEmptyPage(request, response, connection, new WikiPageBean(serviceUser, startTime));\n                return;\n            }\n        }\n        \n        try {\n            if (!initialized && !loadSiteInfo() || !currentImport.isEmpty()) {\n                showImportPage(request, response, connection, new WikiPageBean(serviceUser, startTime)); // should forward to another page\n                return; // return just in case\n            }\n            request.setCharacterEncoding(\"UTF-8\");\n            response.setCharacterEncoding(\"UTF-8\");\n\n            // show empty page for testing purposes if a parameter called \"test\" exists:\n            if (request.getParameter(\"test\") != null) {\n                showEmptyPage(request, response, connection, new WikiPageBean(\n                        serviceUser, startTime));\n                return;\n            }\n            \n            // if the \"search\" parameter exists, show the search\n            String req_search = request.getParameter(\"search\");\n            if (req_search != null) {\n                handleSearch(request, response, null, req_search, connection,\n                        new WikiPageListBean(serviceUser, startTime));\n                return;\n            }\n\n            // get parameters:\n            String req_title = request.getParameter(\"title\");\n            if (req_title == null) {\n                req_title = MAIN_PAGE;\n            }\n\n            String req_action = request.getParameter(\"action\");\n\n            if (!MyWikiModel.isValidTitle(req_title)) {\n                handleViewPageBadTitle(request, response, connection, new WikiPageBean(serviceUser, startTime));\n            } else if (req_title.startsWith(\"Special:\")) {\n                handleViewSpecialPage(request, response, req_title, req_search, \"Special:\".length(), connection, startTime, serviceUser);\n            } else if (req_title.startsWith(namespace.getSpecial() + \":\")) {\n                handleViewSpecialPage(request, response, req_title, req_search, namespace.getSpecial().length() + 1, connection, startTime, serviceUser);\n            } else if (req_action == null || req_action.equals(\"view\")) {\n                handleViewPage(request, response, req_title, connection, new WikiPageBean(serviceUser, startTime));\n            } else if (req_action.equals(\"history\")) {\n                handleViewPageHistory(request, response, req_title, connection, new WikiPageBean(serviceUser, startTime));\n            } else if (req_action.equals(\"edit\")) {\n                handleEditPage(request, response, req_title, connection, new WikiPageEditBean(serviceUser, startTime));\n            } else {\n                // default: show page\n                handleViewPage(request, response, req_title, connection, new WikiPageBean(serviceUser, startTime));\n            }\n\n            // if the request has not been forwarded, print a general error\n            response.setContentType(\"text/html\");\n            PrintWriter out = response.getWriter();\n            out.write(\"An unknown error occured, please contact your administrator. A server restart may be required.\");\n            out.close();\n        } finally {\n            releaseConnection(request, connection);\n        }\n    }\n\n    /**\n     * Shows a special page.\n     * \n     * @param request\n     *            the HTTP request\n     * @param response\n     *            the response object\n     * @param title\n     *            the requested title, e.g. \"Special:Search\"\n     * @param req_search\n     *            search string\n     * @param prefixLength\n     *            length of the (localised) \"Special:\" prefix string\n     * @param connection\n     *            connection to the database\n     * @param startTime\n     *            the time when the request reached the servlet (in ms)\n     * \n     * @throws ServletException\n     * @throws IOException\n     */\n    protected void handleViewSpecialPage(HttpServletRequest request,\n            HttpServletResponse response, String title, String req_search,\n            int prefixLength, Connection connection, long startTime,\n            final String serviceUser)\n            throws IOException, ServletException {\n        final String plainTitle = title.substring(prefixLength);\n        // a \"/\" which separates the special page title from its parameters\n        final String plainTitleToSlash = plainTitle.split(\"/\")[0];\n        if (plainTitle.equalsIgnoreCase(SPECIAL_SUFFIX_EN.get(SpecialPage.SPECIAL_RANDOM))\n                || plainTitle.equalsIgnoreCase(SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_RANDOM))) {\n            handleViewRandomPage(request, response, title, connection, new WikiPageBean(serviceUser, startTime));\n        } else if (plainTitleToSlash.equalsIgnoreCase(SPECIAL_SUFFIX_EN.get(SpecialPage.SPECIAL_ALLPAGES))\n                || plainTitleToSlash.equalsIgnoreCase(SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_ALLPAGES))) {\n            handleSpecialAllPages(request, response, title, connection, new WikiPageListBean(serviceUser, startTime));\n        } else if (plainTitleToSlash.equalsIgnoreCase(SPECIAL_SUFFIX_EN.get(SpecialPage.SPECIAL_PREFIXINDEX))\n                || plainTitleToSlash.equalsIgnoreCase(SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_PREFIXINDEX))) {\n            handleSpecialPrefix(request, response, title, connection, new WikiPageListBean(serviceUser, startTime));\n        } else if (plainTitleToSlash.equalsIgnoreCase(SPECIAL_SUFFIX_EN.get(SpecialPage.SPECIAL_SEARCH))\n                || plainTitleToSlash.equalsIgnoreCase(SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_SEARCH))) {\n            handleSearch(request, response, title, req_search, connection, new WikiPageListBean(serviceUser, startTime));\n        } else if (plainTitleToSlash.equalsIgnoreCase(SPECIAL_SUFFIX_EN.get(SpecialPage.SPECIAL_WHATLINKSHERE))\n                || plainTitleToSlash.equalsIgnoreCase(SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_WHATLINKSHERE))) {\n            handleSpecialWhatLinksHere(request, response, title, connection, new WikiPageListBean(serviceUser, startTime));\n        } else if (plainTitle.equalsIgnoreCase(SPECIAL_SUFFIX_EN.get(SpecialPage.SPECIAL_SPECIALPAGES))\n                || plainTitle.equalsIgnoreCase(SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_SPECIALPAGES))) {\n            handleViewSpecialPages(request, response, connection, new WikiPageListBean(serviceUser, startTime));\n        } else if (plainTitle.equalsIgnoreCase(SPECIAL_SUFFIX_EN.get(SpecialPage.SPECIAL_STATS))\n                || plainTitle.equalsIgnoreCase(SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_STATS))) {\n            handleViewSpecialStatistics(request, response, connection, new WikiPageListBean(serviceUser, startTime));\n        } else if (plainTitle.equalsIgnoreCase(SPECIAL_SUFFIX_EN.get(SpecialPage.SPECIAL_VERSION))\n                || plainTitle.equalsIgnoreCase(SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_VERSION))) {\n            handleViewSpecialVersion(request, response, connection, new WikiPageListBean(serviceUser, startTime));\n        } else {\n            // no such special page\n            handleViewPageNotExisting(request, response, title, connection, new WikiPageBean(serviceUser, startTime));\n        }\n    }\n\n    /**\n     * Shows the page search for the given search string.\n     * \n     * @param request\n     *            the HTTP request\n     * @param response\n     *            the response object\n     * @param title\n     *            the requested title, e.g. \"Special:Search\"\n     * @param req_search\n     *            search string\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     * \n     * @throws ServletException\n     * @throws IOException\n     */\n    private void handleSearch(HttpServletRequest request,\n            HttpServletResponse response, String title, String req_search,\n            Connection connection, WikiPageListBean page)\n            throws ServletException, IOException {\n        // note: req_search can only be null if Special:Search is shown\n        if (req_search == null) {\n            int slashIndex = title.indexOf('/');\n            if (slashIndex != (-1)) {\n                req_search = title.substring(slashIndex + 1);\n            } else {\n                req_search = \"\";\n            }\n        }\n        // use default namespace (id 0) for invalid values\n        int nsId = parseInt(request.getParameter(\"namespace\"), 0);\n        page.setPageHeading(\"Search\");\n        page.setFormTitle(\"Search results\");\n        page.setFormType(FormType.PageSearchForm);\n        ValueResult<List<NormalisedTitle>> result;\n        page.setSearch(req_search);\n        page.setTitle(MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_SEARCH)));\n        page.setShowAllPages(false);\n        if (req_search.isEmpty()) {\n            result = new ValueResult<List<NormalisedTitle>>(new ArrayList<InvolvedKey>(0), new ArrayList<NormalisedTitle>(0));\n        } else {\n            if (existingPages.hasFullList()) {\n                final long timeAtStart = System.currentTimeMillis();\n                final String statName = \"page list:\" + nsId;\n                final List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n                final Set<NormalisedTitle> pages = existingPages.getList(NamespaceEnum.fromId(nsId));\n                result = new ValueResult<List<NormalisedTitle>>(involvedKeys, new ArrayList<NormalisedTitle>(pages),\n                        statName, System.currentTimeMillis() - timeAtStart);\n            } else {\n                result = getPageList(nsId, connection);\n            }\n        }\n        page.setNamespaceId(nsId);\n        page.addStats(result.stats);\n        page.getInvolvedKeys().addAll(result.involvedKeys);\n        handleViewSpecialPageList(request, response, result, connection, page);\n    }\n\n    /**\n     * @param request\n     *            the HTTP request\n     * @param response\n     *            the response object\n     * @param title\n     *            the requested title, e.g. \"Special:AllPages\"\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     *\n     * @throws ServletException\n     * @throws IOException\n     */\n    private void handleSpecialAllPages(HttpServletRequest request,\n            HttpServletResponse response, String title,\n            Connection connection, WikiPageListBean page) throws ServletException, IOException {\n        String req_from = request.getParameter(\"from\");\n        if (req_from == null) {\n            int slashIndex = title.indexOf('/');\n            if (slashIndex != (-1)) {\n                req_from = title.substring(slashIndex + 1);\n            }\n        }\n        String req_to = request.getParameter(\"to\");\n        // use default namespace (id 0) for invalid values\n        int nsId = parseInt(request.getParameter(\"namespace\"), 0);\n        page.setPageHeading(\"All pages\");\n        page.setFormTitle(\"All pages\");\n        page.setFormType(FormType.FromToForm);\n        page.setTitle(MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_ALLPAGES)));\n        ValueResult<List<NormalisedTitle>> result;\n        if (req_from == null && req_to == null) {\n            page.setShowAllPages(false);\n            page.setFromPage(\"\");\n            page.setToPage(\"\");\n            result = new ValueResult<List<NormalisedTitle>>(new ArrayList<InvolvedKey>(0), new ArrayList<NormalisedTitle>(0));\n        } else {\n            page.setShowAllPages(true);\n            if (req_from == null) {\n                req_from = \"\"; // start with first page\n            }\n            if (req_to == null) {\n                req_to = \"\"; // stop at last page\n            }\n            page.setFromPage(req_from);\n            page.setToPage(req_to);\n            if (existingPages.hasFullList()) {\n                final long timeAtStart = System.currentTimeMillis();\n                final String statName = \"page list:\" + nsId;\n                final List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n                final Set<NormalisedTitle> pages = existingPages.getList(NamespaceEnum.fromId(nsId));\n                result = new ValueResult<List<NormalisedTitle>>(involvedKeys, new ArrayList<NormalisedTitle>(pages),\n                        statName, System.currentTimeMillis() - timeAtStart);\n            } else {\n                result = getPageList(nsId, connection);\n            }\n        }\n        page.addStats(result.stats);\n        page.getInvolvedKeys().addAll(result.involvedKeys);\n        page.setNamespaceId(nsId);\n        handleViewSpecialPageList(request, response, result, connection, page);\n    }\n\n    /**\n     * @param request\n     *            the HTTP request\n     * @param response\n     *            the response object\n     * @param title\n     *            the requested title, e.g. \"Special:PrefixIndex\"\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     *\n     * @throws ServletException\n     * @throws IOException\n     */\n    private void handleSpecialPrefix(HttpServletRequest request,\n            HttpServletResponse response, String title,\n            Connection connection, WikiPageListBean page) throws ServletException, IOException {\n        String req_prefix = request.getParameter(\"prefix\");\n        if (req_prefix == null) {\n            int slashIndex = title.indexOf('/');\n            if (slashIndex != (-1)) {\n                req_prefix = title.substring(slashIndex + 1);\n            }\n        }\n        // use default namespace (id 0) for invalid values\n        int nsId = parseInt(request.getParameter(\"namespace\"), 0);\n        page.setPageHeading(\"All pages\");\n        page.setFormTitle(\"All pages\");\n        page.setFormType(FormType.PagePrefixForm);\n        page.setTitle(MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_PREFIXINDEX)));\n        ValueResult<List<NormalisedTitle>> result;\n        if (req_prefix == null) {\n            page.setShowAllPages(false);\n            page.setPrefix(\"\");\n            result = new ValueResult<List<NormalisedTitle>>(new ArrayList<InvolvedKey>(0), new ArrayList<NormalisedTitle>(0));\n        } else {\n            page.setShowAllPages(true);\n            page.setPrefix(req_prefix);\n            if (existingPages.hasFullList()) {\n                final long timeAtStart = System.currentTimeMillis();\n                final String statName = \"page list:\" + nsId;\n                final List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n                final Set<NormalisedTitle> pages = existingPages.getList(NamespaceEnum.fromId(nsId));\n                result = new ValueResult<List<NormalisedTitle>>(involvedKeys, new ArrayList<NormalisedTitle>(pages),\n                        statName, System.currentTimeMillis() - timeAtStart);\n            } else {\n                result = getPageList(nsId, connection);\n            }\n        }\n        page.addStats(result.stats);\n        page.getInvolvedKeys().addAll(result.involvedKeys);\n        page.setNamespaceId(nsId);\n        handleViewSpecialPageList(request, response, result, connection, page);\n    }\n\n    /**\n     * @param request\n     *            the HTTP request\n     * @param response\n     *            the response object\n     * @param title\n     *            the requested title, e.g. \"Special:WhatLinksHere\"\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     *\n     * @throws ServletException\n     * @throws IOException\n     */\n    private void handleSpecialWhatLinksHere(HttpServletRequest request,\n            HttpServletResponse response, String title, Connection connection,\n            WikiPageListBean page) throws ServletException, IOException {\n        String req_target = request.getParameter(\"target\");\n        if (req_target == null) {\n            // maybe we got the name separated with a '/' in the title:\n            int slashIndex = title.indexOf('/');\n            if (slashIndex != (-1)) {\n                req_target = title.substring(slashIndex + 1);\n            }\n        }\n        page.setFormTitle(\"What links here\");\n        page.setFormType(FormType.TargetPageForm);\n        page.setTitle(MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_WHATLINKSHERE)));\n        ValueResult<List<NormalisedTitle>> result;\n        if (req_target == null) {\n            page.setShowAllPages(false);\n            page.setPageHeading(\"Pages that link to a selected page\");\n            page.setTarget(\"\");\n            result = new ValueResult<List<NormalisedTitle>>(new ArrayList<InvolvedKey>(0), new ArrayList<NormalisedTitle>(0));\n        } else {\n            page.setShowAllPages(true);\n            page.setPageHeading(\"Pages that link to \\\"\" + req_target + \"\\\"\");\n            page.setTarget(req_target);\n            result = getPagesLinkingTo(connection, req_target, namespace);\n        }\n        page.addStats(result.stats);\n        page.getInvolvedKeys().addAll(result.involvedKeys);\n        handleViewSpecialPageList(request, response, result, connection, page);\n    }\n\n    /*\n     * (non-Javadoc)\n     * \n     * @see javax.servlet.http.HttpServlet#doPost(HttpServletRequest request,\n     *      HttpServletResponse response)\n     */\n    @Override\n    protected void doPost(HttpServletRequest request,\n            HttpServletResponse response) throws ServletException, IOException {\n        long startTime = System.currentTimeMillis();\n        Connection connection = getConnection(request);\n        final String serviceUser = getParam(request, \"service_user\");\n        if (connection == null) {\n            showEmptyPage(request, response, connection, new WikiPageBean(\n                    serviceUser, startTime)); // should forward to another page\n            return; // return just in case\n        }\n        \n        for (WikiEventHandler handler: eventHandlers) {\n            if (!handler.checkAccess(serviceUser, request, connection)) {\n                // access not allowed\n                showEmptyPage(request, response, connection, new WikiPageBean(\n                        serviceUser, startTime));\n                return;\n            }\n        }\n        \n        try {\n            if (!initialized && !loadSiteInfo() || !currentImport.isEmpty()) {\n                showImportPage(request, response, connection, new WikiPageBean(\n                        serviceUser, startTime)); // should forward to another page\n                return; // return just in case\n            }\n            request.setCharacterEncoding(\"UTF-8\");\n            response.setCharacterEncoding(\"UTF-8\");\n\n            String req_title = request.getParameter(\"title\");\n            if (!MyWikiModel.isValidTitle(req_title)) {\n                handleViewPageBadTitle(request, response, connection,\n                        new WikiPageBean(serviceUser, startTime));\n            } else {\n                handleEditPageSubmitted(request, response,\n                        request.getParameter(\"title\"), connection,\n                        new WikiPageEditBean(serviceUser, startTime));\n            }\n\n            // if the request has not been forwarded, print a general error\n            response.setContentType(\"text/html\");\n            PrintWriter out = response.getWriter();\n            out.write(\"An unknown error occured, please contact your administrator. \"\n                    + \"A server restart may be required.\");\n            out.close();\n        } finally {\n            releaseConnection(request, connection);\n        }\n    }\n\n    /**\n     * Gets a random page and forwards the user to the site with this page.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param title\n     *            the requested title (mostly \"Special:Random\")\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     * \n     * @throws IOException\n     *             if the forward fails\n     * @throws ServletException\n     */\n    private void handleViewRandomPage(HttpServletRequest request,\n            HttpServletResponse response, String title, Connection connection,\n            WikiPageBean page) throws IOException, ServletException {\n        page.setTitle(title);\n        final Random random = new Random();\n        ValueResult<NormalisedTitle> result;\n        if (existingPages.hasFullList()) {\n            final long timeAtStart = System.currentTimeMillis();\n            final String statName = \"RANDOM_PAGE\";\n            final List<InvolvedKey> involvedKeys = new ArrayList<InvolvedKey>();\n            final Set<NormalisedTitle> pages = existingPages.getList(NamespaceEnum.MAIN_NAMESPACE_KEY);\n            final int randIdx = random.nextInt(pages.size());\n            // no real other option to get a random element from a set :(\n            int i = 0;\n            NormalisedTitle randValue = null;\n            for (Iterator<NormalisedTitle> iterator = pages.iterator(); iterator.hasNext();) {\n                if (i == randIdx) {\n                    randValue = iterator.next();\n                    break;\n                } else {\n                    iterator.next();\n                }\n                ++i;\n            }\n            result = new ValueResult<NormalisedTitle>(involvedKeys, randValue,\n                    statName, System.currentTimeMillis() - timeAtStart);\n        } else {\n            result = getRandomArticle(connection, random);\n        }\n        page.addStats(result.stats);\n        page.getInvolvedKeys().addAll(result.involvedKeys);\n        final String serviceUser = page.getServiceUser().isEmpty() ? \"\" : \"&service_user=\" + page.getServiceUser();\n        String redirectUrl;\n        ArrayList<Long> times = new ArrayList<Long>();\n        for (List<Long> time : result.stats.values()) {\n            times.addAll(time);\n        }\n        if (result.success) {\n            StringBuilder redirectUrl0 = new StringBuilder(256);\n            redirectUrl0.append(\"?title=\");\n            redirectUrl0.append(URLEncoder.encode(result.value.denormalise(namespace), \"UTF-8\"));\n            redirectUrl0.append(\"&random_times=\" + StringUtils.join(times, \"%2C\"));\n            redirectUrl0.append(\"&involved_keys=\" + URLEncoder.encode(StringUtils.join(page.getInvolvedKeys(), \" # \"), \"UTF-8\"));\n            redirectUrl = \"http://\" + Options.getInstance().SERVERNAME\n                    + Options.getInstance().SERVERPATH\n                    + response.encodeRedirectURL(redirectUrl0.toString())\n                    + serviceUser;\n        } else if (result.connect_failed) {\n            setParam_error(request, \"ERROR: DB connection failed\");\n            showEmptyPage(request, response, connection, page);\n            return;\n        } else {\n            StringBuilder redirectUrl0 = new StringBuilder(256);\n            redirectUrl0.append(\"?title=\");\n            redirectUrl0.append(URLEncoder.encode(MAIN_PAGE, \"UTF-8\"));\n            redirectUrl0.append(\"&random_times=\" + StringUtils.join(times, \"%2C\"));\n            redirectUrl0.append(\"&involved_keys=\" + URLEncoder.encode(StringUtils.join(page.getInvolvedKeys(), \" # \"), \"UTF-8\"));\n            redirectUrl0.append(\"&notice=\" + URLParamEncoder.encode(\"error: can not view random page: <pre>\" + result.message + \"</pre>\"));\n            redirectUrl = \"http://\" + Options.getInstance().SERVERNAME\n                    + Options.getInstance().SERVERPATH\n                    + response.encodeRedirectURL(redirectUrl0.toString())\n                    + serviceUser;\n        }\n        for (WikiEventHandler handler: eventHandlers) {\n            handler.onViewRandomPage(page, result, connection);\n        }\n        response.sendRedirect(redirectUrl + \"&server_time=\" + (System.currentTimeMillis() - page.getStartTime()));\n    }\n\n    /**\n     * Shows the contents of the page page with the given <tt>title</tt>.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param title\n     *            the title of the page to show\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     * \n     * @throws IOException\n     * @throws ServletException\n     */\n    private void handleViewPage(HttpServletRequest request,\n            HttpServletResponse response, String title, Connection connection,\n            WikiPageBean page) throws ServletException, IOException {\n        // get revision id to load:\n        int req_oldid = getParam_oldid(request);\n\n        RevisionResult result = getRevision(connection, title, req_oldid, namespace);\n        page.addStats(result.stats);\n        page.getInvolvedKeys().addAll(result.involvedKeys);\n        handleViewPage2(request, response, title, connection, page, req_oldid,\n                result);\n    }\n\n    private void handleViewPage2(HttpServletRequest request,\n            HttpServletResponse response, String title, Connection connection,\n            WikiPageBean page, int req_oldid, RevisionResult result)\n            throws ServletException, IOException {\n        if (result.connect_failed) {\n            setParam_error(request, \"ERROR: DB connection failed\");\n            showEmptyPage(request, response, connection, page);\n            return;\n        } else if (result.page_not_existing) {\n            handleViewPageNotExisting(request, response, title, connection, page);\n            return;\n        } else if (result.rev_not_existing) {\n            if (result.page != null) {\n                result.revision = result.page.getCurRev();\n                result.success = true;\n            }\n            addToParam_notice(request, \"revision \" + req_oldid + \" not found - loaded current revision instead\");\n        }\n        \n        if (result.success) {\n            // get renderer\n            int render = getParam_renderer(request);\n            final boolean noRedirect = getParam(request, \"redirect\").equals(\"no\");\n            renderRevision(result.page.getTitle(), result, render, request,\n                    connection, page, noRedirect,\n                    getWikiModel(connection, page), true);\n            \n            if (!result.page.checkEditAllowed(\"\")) {\n                page.setEditRestricted(true);\n            }\n\n            forwardToPageJsp(request, response, connection, page, \"page.jsp\");\n        } else {\n            setParam_error(request, \"ERROR: revision unavailable\");\n            addToParam_notice(request, \"error: unknown error getting page \" + title + \":\" + req_oldid + \": <pre>\" + result.message + \"</pre>\");\n            showEmptyPage(request, response, connection, page);\n        }\n    }\n    \n    /**\n     * Creates a {@link WikiPageBean} object with the rendered content of a\n     * given revision.\n     * \n     * @param title\n     *            the title of the article to render\n     * @param result\n     *            the revision to render (must be successful and contain a\n     *            revision)\n     * @param renderer\n     *            the renderer to use (0=plain text, 1=Bliki)\n     * @param request\n     *            the request object\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page (the rendered content will be added to\n     *            this object)\n     * @param noRedirect\n     *            if <tt>true</tt>, a redirect will be shown as such, otherwise\n     *            the content of the redirected page will be show\n     * @param wikiModel\n     *            the wiki model to use\n     * @param topLevel\n     *            if this function is called from inside\n     *            {@link #renderRevision()}, this will be <tt>false</tt>,\n     *            otherwise always use <tt>true</tt>\n     */\n    private void renderRevision(final String title,\n            final RevisionResult result, final int renderer,\n            final HttpServletRequest request, final Connection connection,\n            final WikiPageBean page, final boolean noRedirect,\n            final MyWikiModel wikiModel, final boolean topLevel) {\n        // set the page's contents according to the renderer used\n        // (categories are included in the content string, so they only\n        // need special handling the wiki renderer is used)\n        NormalisedTitle titleN = NormalisedTitle.fromUnnormalised(title, namespace);\n        wikiModel.setNamespaceName(namespace.getNamespaceByNumber(titleN.namespace));\n        wikiModel.setPageName(titleN.title);\n        if (renderer > 0) {\n            String mainText = wikiModel.renderPageWithCache(result.revision.unpackedText());\n            if (titleN.namespace.equals(MyNamespace.CATEGORY_NAMESPACE_KEY)) {\n                ValueResult<List<NormalisedTitle>> catPagesResult = getPagesInCategory(connection, titleN);\n                page.addStats(catPagesResult.stats);\n                page.getInvolvedKeys().addAll(catPagesResult.involvedKeys);\n                if (catPagesResult.success) {\n                    final TreeSet<String> subCategories = new TreeSet<String>(String.CASE_INSENSITIVE_ORDER);\n                    final TreeSet<String> categoryPages = new TreeSet<String>(String.CASE_INSENSITIVE_ORDER);\n                    final List<NormalisedTitle> tplPages = new ArrayList<NormalisedTitle>(catPagesResult.value.size());\n\n                    for (NormalisedTitle pageInCat: catPagesResult.value) {\n                        if (pageInCat.namespace.equals(MyNamespace.CATEGORY_NAMESPACE_KEY)) {\n                            subCategories.add(pageInCat.title);\n                        } else if (pageInCat.namespace.equals(MyNamespace.TEMPLATE_NAMESPACE_KEY)) {\n                            tplPages.add(pageInCat);\n                            categoryPages.add(pageInCat.denormalise(namespace));\n                        } else {\n                            categoryPages.add(pageInCat.denormalise(namespace));\n                        }\n                    }\n                    if (!tplPages.isEmpty()) {\n                        // all pages using a template are in the category of the template, too\n                        ValueResult<List<NormalisedTitle>> tplResult = getPagesInTemplates(connection, tplPages, title);\n                        page.addStats(tplResult.stats);\n                        page.getInvolvedKeys().addAll(tplResult.involvedKeys);\n                        if (tplResult.success) {\n                            for (NormalisedTitle pageInTplOfCat: tplResult.value) {\n                                if (pageInTplOfCat.namespace.equals(MyNamespace.CATEGORY_NAMESPACE_KEY)) {\n                                    subCategories.add(pageInTplOfCat.title);\n                                } else if (pageInTplOfCat.namespace.equals(MyNamespace.TEMPLATE_NAMESPACE_KEY)) {\n                                    // TODO: go into recursion?! -> for now, just add the template\n//                                  tplPages.add(pageInTplOfCat);\n                                    categoryPages.add(pageInTplOfCat.denormalise(namespace));\n                                } else {\n                                    categoryPages.add(pageInTplOfCat.denormalise(namespace));\n                                }\n                            }\n                        } else {\n                            if (tplResult.connect_failed) {\n                                setParam_error(request, \"ERROR: DB connection failed\");\n                            } else {\n                                setParam_error(request, \"ERROR: template page lists unavailable\");\n                            }\n                            addToParam_notice(request, \"error getting pages using templates: \" + tplResult.message);\n                        }\n                    }\n                    page.setSubCategories(subCategories);\n                    page.setCategoryPages(categoryPages);\n                } else {\n                    if (catPagesResult.connect_failed) {\n                        setParam_error(request, \"ERROR: DB connection failed\");\n                    } else {\n                        setParam_error(request, \"ERROR: category page list unavailable\");\n                    }\n                    addToParam_notice(request, \"error getting category pages: \" + catPagesResult.message);\n                }\n            }\n            page.setTitle(title);\n            page.setVersion(result.revision.getId());\n            String redirectedPageName = wikiModel.getRedirectLink();\n            if (redirectedPageName != null) {\n                if (noRedirect) {\n                    if (topLevel) {\n                        page.setContentSub(\"Redirect page\");\n                    }\n                    mainText = wikiModel.renderRedirectPage(redirectedPageName);\n                    page.setDate(Revision.stringToCalendar(result.revision.getTimestamp()));\n                } else {\n                    final String safeTitle = StringEscapeUtils.escapeHtml(title);\n                    final String redirectUrl = wikiModel.getWikiBaseURL().replace(\"${title}\", title);\n                    page.setContentSub(\"(Redirected from <a href=\\\"\" + redirectUrl + \"&redirect=no\\\" title=\\\"\" + safeTitle + \"\\\">\" + title + \"</a>)\");\n                    // add the content from the page directed to:\n                    wikiModel.tearDown();\n                    wikiModel.setUp();\n                    \n                    RevisionResult redirectResult = getRevision(connection,\n                            redirectedPageName, namespace);\n                    page.addStats(redirectResult.stats);\n                    page.getInvolvedKeys().addAll(redirectResult.involvedKeys);\n                    if (redirectResult.success) {\n                        renderRevision(redirectedPageName, redirectResult,\n                                renderer, request, connection, page, true,\n                                wikiModel, false);\n                        return;\n                    } else {\n                        // non-existing/non-successful page is like redirect=no\n                        mainText = wikiModel.renderRedirectPage(redirectedPageName);\n                        page.setDate(Revision.stringToCalendar(result.revision.getTimestamp()));\n                    }\n                }\n            } else {\n                setSubPageNav(title, page, wikiModel);\n            }\n            page.setPage(mainText);\n            page.setCategories(wikiModel.getCategories().keySet());\n            page.addStats(wikiModel.getStats());\n            page.getInvolvedKeys().addAll(wikiModel.getInvolvedKeys());\n        } else if (renderer == 0) {\n            // for debugging, show all parameters:\n            StringBuilder sb = new StringBuilder();\n            for (Enumeration<?> req_pars = request.getParameterNames(); req_pars.hasMoreElements();) {\n                String element = (String) req_pars.nextElement();\n                sb.append(element + \" = \");\n                sb.append(request.getParameter(element) + \"\\n\");\n            }\n            sb.append(\"\\n\\n\");\n            for (Enumeration<?> headers = request.getHeaderNames(); headers.hasMoreElements();) {\n                String element = (String) headers.nextElement();\n                sb.append(element + \" = \");\n                sb.append(request.getHeader(element) + \"\\n\");\n            }\n            page.setPage(\"<p>WikiText:<pre>\"\n                    + StringEscapeUtils.escapeHtml(result.revision.unpackedText()) + \"</pre></p>\" +\n                    \"<p>Version:<pre>\"\n                    + StringEscapeUtils.escapeHtml(String.valueOf(result.revision.getId())) + \"</pre></p>\" +\n                    \"<p>Last change:<pre>\"\n                    + StringEscapeUtils.escapeHtml(result.revision.getTimestamp()) + \"</pre></p>\" +\n                    \"<p>Request Parameters:<pre>\"\n                    + StringEscapeUtils.escapeHtml(sb.toString()) + \"</pre></p>\");\n            page.setTitle(title);\n            page.setVersion(result.revision.getId());\n            page.setDate(Revision.stringToCalendar(result.revision.getTimestamp()));\n        }\n        page.setNotice(getParam_notice(request));\n        page.setError(getParam_error(request));\n        page.setWikiTitle(siteinfo.getSitename());\n        page.setWikiNamespace(namespace);\n    }\n\n    /**\n     * For sub-pages set a navigation to higher-level pages via\n     * {@link WikiPageBean#setContentSub(String)} into the page bean.\n     * \n     * @param title\n     *            the title of the article to render\n     * @param page\n     *            the bean for the page\n     * @param wikiModel\n     *            the wiki model to get the base URL from\n     */\n    protected void setSubPageNav(String title, WikiPageBean page,\n            MyWikiModel wikiModel) {\n        final String wikiBaseURL = wikiModel.getWikiBaseURL();\n        setSubPageNav(title, page, wikiBaseURL);\n    }\n\n    /**\n     * For sub-pages set a navigation to higher-level pages via\n     * {@link WikiPageBean#setContentSub(String)} into the page bean.\n     * \n     * @param title\n     *            the title of the article to render\n     * @param page\n     *            the bean for the page\n     * @param wikiBaseURL\n     *            base url for links\n     */\n    protected void setSubPageNav(String title, WikiPageBean page,\n            final String wikiBaseURL) {\n        String[] parts = title.split(\"/\");\n        if (parts.length > 1) {\n            String fullPart = null;\n            StringBuilder contentSub = new StringBuilder();\n            for (int i = 0; i < parts.length - 1; ++i) {\n                String part = parts[i];\n                if (i == 0) {\n                    // first?\n                    fullPart = part;\n                } else {\n                    contentSub.append(\" · \");\n                    fullPart += \"/\" + part;\n                }\n                contentSub.append(\"<a href=\\\"\");\n                contentSub.append(wikiBaseURL.replace(\"${title}\", fullPart));\n                contentSub.append(\"\\\" title=\\\"\");\n                contentSub.append(fullPart);\n                contentSub.append(\"\\\">\");\n                contentSub.append(fullPart);\n                contentSub.append(\"</a>‎\");\n            }\n            if (contentSub.length() > 0) {\n                page.setContentSub(\"<span class=\\\"subpages\\\">&lt; \" + contentSub.toString() + \"</span>\");\n            }\n        }\n    }\n    \n    /**\n     * Shows the \"Page not available\" message the wiki returns in case a page\n     * does not exist.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param title\n     *            the original title of the page that does not exist\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     * \n     * @throws IOException \n     * @throws ServletException \n     */\n    private void handleViewPageNotExisting(HttpServletRequest request,\n            HttpServletResponse response, String title, Connection connection,\n            WikiPageBean page) throws ServletException, IOException {\n        // get renderer\n        int render = getParam_renderer(request);\n        String notExistingTitle = \"MediaWiki:Noarticletext\";\n\n        RevisionResult result = getRevision(connection, notExistingTitle, namespace);\n        page.addStats(result.stats);\n        page.getInvolvedKeys().addAll(result.involvedKeys);\n        \n        if (result.success) {\n            renderRevision(title, result, render, request, connection, page,\n                    false, getWikiModel(connection, page), true);\n        } else if (result.connect_failed) {\n            setParam_error(request, \"ERROR: DB connection failed\");\n            showEmptyPage(request, response, connection, page);\n            return;\n        } else {\n//            addToParam_notice(request, \"error: unknown error getting page \" + notExistingTitle + \": <pre>\" + result.message + \"</pre>\");\n            page.setPage(\"Page not available.\");\n            page.setError(getParam_error(request));\n            page.setTitle(title);\n        }\n        setSubPageNav(title, page, getLinkbaseurl(page));\n        // re-set version (we are only showing this page due to a non-existing page)\n        page.setVersion(-1);\n        page.setNotAvailable(true);\n        page.setNotice(getParam_notice(request));\n        page.setWikiTitle(siteinfo.getSitename());\n        page.setWikiNamespace(namespace);\n\n        forwardToPageJsp(request, response, connection, page, \"page.jsp\");\n    }\n\n    /**\n     * Forwards a request to the <tt>page.jsp</tt> with the given page bean.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     * \n     * @throws ServletException\n     * @throws IOException\n     */\n    protected void forwardToPageJsp(HttpServletRequest request,\n            HttpServletResponse response, Connection connection,\n            WikiPageBeanBase page, String jsp) throws ServletException, IOException {\n        final String[] pageSaveTimes = getParam(request, \"save_times\").split(\",\");\n        for (String pageSaveTime : pageSaveTimes) {\n            final int pageSaveTimeInt = parseInt(pageSaveTime, -1);\n            if (pageSaveTimeInt >= 0) {\n                final String statName = \"SAVE:\" + page.getTitle();\n                page.addStat(statName, pageSaveTimeInt);\n            }\n        }\n        final int pageSaveServerTime = parseInt(getParam(request, \"server_time\"), -1);\n        if (pageSaveServerTime >= 0) {\n            page.addStat(\"server_time (last op)\", pageSaveServerTime);\n        }\n        page.setSaveAttempts(parseInt(getParam(request, \"save_attempts\"), 0));\n        for (int i = 1; i <= Options.getInstance().WIKI_SAVEPAGE_RETRIES; ++i) {\n            final String failedKeysPar = getParam(request, \"failed_keys\" + i);\n            if (!failedKeysPar.isEmpty()) {\n                final List<String> pageSaveFailedKeys = Arrays.asList(failedKeysPar.split(\" # \"));\n                page.getFailedKeys().put(i, pageSaveFailedKeys);\n            }\n        }\n        final String involvedKeysPar = getParam(request, \"involved_keys\");\n        if (!involvedKeysPar.isEmpty()) {\n            page.getInvolvedKeys().add(new InvolvedKey());\n            InvolvedKey.addInvolvedKeys(page.getInvolvedKeys(), Arrays.asList(involvedKeysPar.split(\" # \")), true);\n        }\n        final String[] pageRandomTimes = getParam(request, \"random_times\").split(\",\");\n        for (String pageRandomTime : pageRandomTimes) {\n            final int pageRandomTimeInt = parseInt(pageRandomTime, -1);\n            if (pageRandomTimeInt >= 0) {\n                final String statName = \"RANDOM_PAGE (last op)\";\n                page.addStat(statName, pageRandomTimeInt);\n            }\n        }\n        // forward the request and the bean to the jsp:\n        request.setAttribute(\"pageBean\", page);\n        request.setAttribute(\"servlet\", this);\n        RequestDispatcher dispatcher = request.getRequestDispatcher(jsp);\n        dispatcher.forward(request, response);\n    }\n    \n    /**\n     * Shows the \"Bad Title\" message the wiki returns in case a page title is\n     * invalid.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     * \n     * @throws IOException \n     * @throws ServletException \n     */\n    private void handleViewPageBadTitle(HttpServletRequest request,\n            HttpServletResponse response, Connection connection,\n            WikiPageBean page) throws ServletException, IOException {\n        final String title = \"Bad title\";\n        // get renderer\n        final int render = getParam_renderer(request);\n        final String badTitleKey = \"MediaWiki:Badtitletext\";\n\n        RevisionResult result = getRevision(connection, badTitleKey, namespace);\n        page.addStats(result.stats);\n        page.getInvolvedKeys().addAll(result.involvedKeys);\n        \n        if (result.success) {\n            renderRevision(title, result, render, request, connection, page,\n                    false, getWikiModel(connection, page), true);\n        } else if (result.connect_failed) {\n            setParam_error(request, \"ERROR: DB connection failed\");\n            showEmptyPage(request, response, connection, page);\n            return;\n        } else {\n//            addToParam_notice(request, \"error: unknown error getting page \" + badTitleTitle + \": <pre>\" + result.message + \"</pre>\");\n            page.setPage(\"The requested page title is invalid. It may be empty, contain unsupported characters, or include a non-local or incorrectly linked interwiki prefix. You may be able to locate the desired page by searching for its name (with interwiki prefix, if any) in the search box.\");\n            page.setError(getParam_error(request));\n            page.setTitle(title);\n        }\n        // re-set version (we are only showing this page due to a non-existing page)\n        page.setVersion(-1);\n        page.setEditRestricted(true);\n        page.setNotAvailable(true);\n        page.setNotice(getParam_notice(request));\n        page.setWikiTitle(siteinfo.getSitename());\n        page.setWikiNamespace(namespace);\n\n        forwardToPageJsp(request, response, connection, page, \"page.jsp\");\n    }\n\n    /**\n     * Shows the page containing the history information of an article with the\n     * given <tt>title</tt>.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param title\n     *            the title of the page\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     * \n     * @throws IOException \n     * @throws ServletException \n     */\n    private void handleViewPageHistory(HttpServletRequest request,\n            HttpServletResponse response, String title, Connection connection,\n            WikiPageBean page) throws ServletException, IOException {\n        PageHistoryResult result = getPageHistory(connection, title, namespace);\n        page.addStats(result.stats);\n        page.getInvolvedKeys().addAll(result.involvedKeys);\n        if (result.connect_failed) {\n            setParam_error(request, \"ERROR: DB connection failed\");\n            showEmptyPage(request, response, connection, page);\n            return;\n        } else if (result.not_existing) {\n            handleViewPageNotExisting(request, response, title, connection, page);\n            return;\n        }\n\n        page.setTitle(title);\n        if (result.success) {\n            page.setNotice(getParam_notice(request));\n            page.setRevisions(result.revisions);\n            if (!result.page.checkEditAllowed(\"\")) {\n                page.setEditRestricted(true);\n            }\n\n            page.setError(getParam_error(request));\n            if (!result.revisions.isEmpty()) { \n                page.setVersion(result.revisions.get(0).getId());\n            }\n            page.setWikiTitle(siteinfo.getSitename());\n            page.setWikiNamespace(namespace);\n\n            forwardToPageJsp(request, response, connection, page, \"pageHistory.jsp\");\n        } else {\n            setParam_error(request, \"ERROR: revision list unavailable\");\n            addToParam_notice(request, \"error: unknown error getting revision list for page \" + title + \": <pre>\" + result.message + \"</pre>\");\n            showEmptyPage(request, response, connection, page);\n        }\n    }\n\n    /**\n     * Shows a page containing a list of article names.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param result\n     *            result from reading the page list\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     * \n     * @throws IOException\n     * @throws ServletException\n     */\n    private void handleViewSpecialPageList(HttpServletRequest request,\n            HttpServletResponse response, ValueResult<List<NormalisedTitle>> result,\n            Connection connection, WikiPageListBean page)\n            throws ServletException, IOException {\n        if (result.success) {\n            final TreeSet<String> pageList = new TreeSet<String>(String.CASE_INSENSITIVE_ORDER);\n            MyWikiModel.denormalisePageTitles(result.value, namespace, pageList);\n            page.setNotice(getParam_notice(request));\n            String nsPrefix = namespace.getNamespaceByNumber(page.getNamespaceId());\n            if (!nsPrefix.isEmpty()) {\n                nsPrefix += \":\";\n            }\n            final String prefix = nsPrefix + page.getPrefix();\n            final String from = page.getFromPage();\n            final String fullFrom = nsPrefix + page.getFromPage();\n            final String to = page.getToPage();\n            final String fullTo = nsPrefix + page.getToPage();\n            final String search = page.getSearch().toLowerCase();\n            final String searchTitle = MyWikiModel.normaliseName(page.getSearch());\n            boolean foundMatch = false;\n            if (!prefix.isEmpty() || !from.isEmpty() || !to.isEmpty() || !search.isEmpty()) {\n                // only show pages with this prefix:\n                for (Iterator<String> it = pageList.iterator(); it.hasNext(); ) {\n                    final String cur = it.next();\n                    // case-insensitive \"startsWith\" check:\n                    if (!cur.regionMatches(true, 0, prefix, 0, prefix.length())) {\n                        it.remove();\n                    } else if (!from.isEmpty() && cur.compareToIgnoreCase(fullFrom) <= 0) {\n                        it.remove();\n                    } else if (!to.isEmpty() && cur.compareToIgnoreCase(fullTo) > 0) {\n                        it.remove();\n                    } else if (!search.isEmpty() && !cur.toLowerCase().contains(search)) {\n                        it.remove();\n                    } else if (!search.isEmpty() && cur.equals(searchTitle)) {\n                        foundMatch = true;\n                    }\n                }\n            }\n            page.setPages(pageList);\n            page.setFoundFullMatch(foundMatch);\n            page.setWikiTitle(siteinfo.getSitename());\n            page.setWikiNamespace(namespace);\n            \n            forwardToPageJsp(request, response, connection, page, \"pageSpecial_pagelist.jsp\");\n        } else {\n            if (result.connect_failed) {\n                setParam_error(request, \"ERROR: DB connection failed\");\n            } else {\n                setParam_error(request, \"ERROR: page list unavailable\");\n                addToParam_notice(request, \"error: unknown error getting page list for \" + page.getTitle() + \": <pre>\" + result.message + \"</pre>\");\n            }\n            showEmptyPage(request, response, connection, page);\n            return;\n        }\n        page.setError(getParam_error(request));\n        page.setTitle(page.getTitle());\n    }\n\n    /**\n     * Shows the overview of all available special pages.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     *\n     * @throws ServletException\n     * @throws IOException\n     */\n    private void handleViewSpecialPages(HttpServletRequest request,\n            HttpServletResponse response, Connection connection,\n            WikiPageListBean page) throws ServletException, IOException {\n        final String title = MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_SPECIALPAGES));\n        page.setPageHeading(\"Special pages\");\n        page.setTitle(title);\n        \n        Map<String /*group*/, Map<String /*title*/, String /*description*/>> specialPages = new LinkedHashMap<String, Map<String, String>>();\n        Map<String, String> curSpecialPages;\n        // Lists of pages\n        curSpecialPages = new LinkedHashMap<String, String>();\n        curSpecialPages.put(MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_ALLPAGES)), \"All pages\");\n        curSpecialPages.put(MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_PREFIXINDEX)), \"All pages with prefix\");\n        specialPages.put(\"Lists of pages\", curSpecialPages);\n        // Wiki data and tools\n        curSpecialPages = new LinkedHashMap<String, String>();\n        curSpecialPages.put(MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_STATS)), \"Statistics\");\n        curSpecialPages.put(MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_VERSION)), \"Version\");\n        specialPages.put(\"Wiki data and tools\", curSpecialPages);\n        // Redirecting special pages\n        curSpecialPages = new LinkedHashMap<String, String>();\n        curSpecialPages.put(MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_SEARCH)), \"Search\");\n        curSpecialPages.put(MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_RANDOM)), \"Show any page\");\n        specialPages.put(\"Redirecting special pages\", curSpecialPages);\n        // Page tools\n        curSpecialPages = new LinkedHashMap<String, String>();\n        if (Options.getInstance().WIKI_USE_BACKLINKS) {\n            curSpecialPages.put(MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_WHATLINKSHERE)), \"What links here\");\n        }\n        specialPages.put(\"Page tools\", curSpecialPages);\n\n        StringBuilder content = new StringBuilder();\n        final String serviceUser = page.getServiceUser().isEmpty() ? \"\" : \"&service_user=\" + page.getServiceUser();\n        for (Entry<String, Map<String, String>> specialPagesInGroup: specialPages.entrySet()) {\n            String groupName = specialPagesInGroup.getKey();\n            int i = 0;\n            Iterator<Entry<String, String>> it = specialPagesInGroup.getValue().entrySet().iterator();\n            \n            content.append(\"<h4 class=\\\"mw-specialpagesgroup\\\"> <span class=\\\"mw-headline\\\">\" + groupName + \"</span></h4>\\n\");\n            content.append(\"<table style=\\\"width: 100%;\\\" class=\\\"mw-specialpages-table\\\">\\n\");\n            content.append(\" <tbody>\\n\");\n            content.append(\"  <tr>\\n\");\n            content.append(\"   <td style=\\\"width: 30%; vertical-align: top;\\\">\\n\");\n            content.append(\"    <ul>\\n\");\n            int pagesInFirst = specialPagesInGroup.getValue().size() / 2 + specialPagesInGroup.getValue().size() % 2;\n            for (; i < pagesInFirst; ++i) {\n                Entry<String, String> pageInFirst = it.next();\n                content.append(\"<li><a href=\\\"wiki?title=\" + pageInFirst.getKey() + serviceUser + \"\\\" title=\\\"\" + pageInFirst.getKey() + \"\\\">\" + pageInFirst.getValue() + \"</a></li>\\n\");\n            }\n            content.append(\"    </ul>\\n\");\n            content.append(\"   </td>\\n\");\n            content.append(\"   <td style=\\\"width: 10%;\\\"></td>\\n\");\n            content.append(\"   <td style=\\\"width: 30%;\\\">\\n\");\n            content.append(\"    <ul>\\n\");\n            while(it.hasNext()) {\n                Entry<String, String> pageInSecond = it.next();\n                content.append(\"<li><a href=\\\"wiki?title=\" + pageInSecond.getKey() + serviceUser + \"\\\" title=\\\"\" + pageInSecond.getKey() + \"\\\">\" + pageInSecond.getValue() + \"</a></li>\\n\");\n            }\n            content.append(\"    </ul>\\n\");\n            content.append(\"   </td>\\n\");\n            content.append(\"   <td style=\\\"width: 30%;\\\"></td>\\n\");\n            content.append(\"  </tr>\\n\");\n            content.append(\" </tbody>\\n\");\n            content.append(\"</table>\\n\");\n        }\n        \n        page.setPage(content.toString());\n        // abuse #handleViewSpecialPageList here:\n        ValueResult<List<NormalisedTitle>> result = new ValueResult<List<NormalisedTitle>>(\n                new ArrayList<InvolvedKey>(0),\n                new ArrayList<NormalisedTitle>(0));\n        handleViewSpecialPageList(request, response, result, connection, page);\n    }\n\n    /**\n     * Shows several statistics about the running Wiki instance.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     *\n     * @throws ServletException\n     * @throws IOException\n     */\n    private void handleViewSpecialStatistics(HttpServletRequest request,\n            HttpServletResponse response, Connection connection,\n            WikiPageListBean page) throws ServletException, IOException {\n        MyWikiModel wikiModel = getWikiModel(connection, page);\n        final String title = MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_STATS));\n        page.setPageHeading(\"Statistics\");\n        page.setTitle(title);\n\n        String articleCountStr;\n        do {\n            ValueResult<BigInteger> articleCount = getArticleCount(connection);\n            page.addStats(articleCount.stats);\n            page.getInvolvedKeys().addAll(articleCount.involvedKeys);\n            if (articleCount.success) {\n                articleCountStr = wikiModel.formatStatisticNumber(false, articleCount.value);\n            } else {\n                articleCountStr = \"n/a\";\n            }\n        } while(false);\n        \n        String pageCountStr;\n        String pageEditsStr;\n        String pageEditsPerPageStr;\n        do {\n            ValueResult<BigInteger> pageCount = getPageCount(connection);\n            page.addStats(pageCount.stats);\n            page.getInvolvedKeys().addAll(pageCount.involvedKeys);\n            if (pageCount.success) {\n                pageCountStr = wikiModel.formatStatisticNumber(false, pageCount.value);\n                \n                ValueResult<BigInteger> pageEdits = getStatsPageEdits(connection);\n                page.addStats(pageEdits.stats);\n                page.getInvolvedKeys().addAll(pageEdits.involvedKeys);\n                if (pageEdits.success) {\n                    pageEditsStr = wikiModel.formatStatisticNumber(false, pageEdits.value);\n                    if (pageCount.value.equals(BigInteger.valueOf(0))) {\n                        pageEditsPerPageStr = wikiModel.formatStatisticNumber(false, 0.0);\n                    } else {\n                        pageEditsPerPageStr = wikiModel.formatStatisticNumber(false, pageEdits.value.doubleValue() / pageCount.value.doubleValue());\n                    }\n                } else {\n                    pageEditsStr = \"n/a\";\n                    pageEditsPerPageStr = \"n/a\";\n                }\n            } else {\n                pageCountStr = \"n/a\";\n                pageEditsStr = \"n/a\";\n                pageEditsPerPageStr = \"n/a\";\n            }\n        } while (false);\n        BigInteger uploadedFiles = BigInteger.valueOf(0); // currently not supported       \n        \n        Map<String /*group*/, Map<String /*name*/, String /*value*/>> specialPages = new LinkedHashMap<String, Map<String, String>>();\n        Map<String, String> curStats;\n        // Page statistics\n        curStats = new LinkedHashMap<String, String>();\n        curStats.put(\"Content pages\", articleCountStr);\n        curStats.put(\"Pages<br><small class=\\\"mw-statistic-desc\\\"> (All pages in the wiki, including talk pages, redirects, etc.)</small>\",\n                pageCountStr);\n        curStats.put(\"Uploaded files\", wikiModel.formatStatisticNumber(false, uploadedFiles));\n        specialPages.put(\"Page statistics\", curStats);\n        // Edit statistics\n        curStats = new LinkedHashMap<String, String>();\n        curStats.put(\"Page edits since Wikipedia was set up\", pageEditsStr);\n        curStats.put(\"Average changes per page\", pageEditsPerPageStr);\n        specialPages.put(\"Edit statistics\", curStats);\n        // User statistics\n        curStats = new LinkedHashMap<String, String>();\n        specialPages.put(\"User statistics\", curStats);\n\n        StringBuilder content = new StringBuilder();\n        content.append(\"<table class=\\\"wikitable mw-statistics-table\\\">\\n\");\n        content.append(\" <tbody>\\n\");\n        for (Entry<String, Map<String, String>> specialPagesInGroup: specialPages.entrySet()) {\n            String groupName = specialPagesInGroup.getKey();\n            Iterator<Entry<String, String>> it = specialPagesInGroup.getValue().entrySet().iterator();\n            \n            content.append(\"  <tr>\\n\");\n            content.append(\"   <th colspan=\\\"2\\\">\" + groupName + \"</th>\\n\");\n            content.append(\"  </tr>\\n\");\n            \n            while(it.hasNext()) {\n                Entry<String, String> stat = it.next();\n                content.append(\"  <tr class=\\\"mw-statistics\\\">\\n\");\n                content.append(\"   <td>\" + stat.getKey() + \"</td>\\n\");\n                content.append(\"   <td class=\\\"mw-statistics-numbers\\\">\" + stat.getValue() + \"</td>\\n\");\n                content.append(\"  </tr>\\n\");\n            }\n        }\n        content.append(\" </tbody>\\n\");\n        content.append(\"</table>\\n\");\n        \n        page.setPage(content.toString());\n        page.addStats(wikiModel.getStats());\n        page.getInvolvedKeys().addAll(wikiModel.getInvolvedKeys());\n        // abuse #handleViewSpecialPageList here:\n        ValueResult<List<NormalisedTitle>> result = new ValueResult<List<NormalisedTitle>>(\n                new ArrayList<InvolvedKey>(0),\n                new ArrayList<NormalisedTitle>(0));\n        handleViewSpecialPageList(request, response, result, connection, page);\n    }\n\n    /**\n     * Shows version information about the running Wiki instance.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     *\n     * @throws ServletException\n     * @throws IOException\n     */\n    private void handleViewSpecialVersion(HttpServletRequest request,\n            HttpServletResponse response, Connection connection,\n            WikiPageListBean page) throws ServletException, IOException {\n        final String title = MyWikiModel.createFullPageName(namespace.getSpecial(), SPECIAL_SUFFIX_LANG.get(SpecialPage.SPECIAL_VERSION));\n        page.setPageHeading(\"Version\");\n        page.setTitle(title);\n\n        StringBuilder content = new StringBuilder();\n        content.append(\"<h2 id=\\\"mw-version-license\\\"> <span class=\\\"mw-headline\\\" id=\\\"License\\\">License</span></h2>\\n\");\n        content.append(\"<div>\\n\");\n        content.append(\"<p>This wiki is powered by <b><a href=\\\"http://code.google.com/p/scalaris/\\\" class=\\\"external text\\\" rel=\\\"nofollow\\\">Scalaris</a></b>, copyright © 2013 Zuse Institute Berlin</p>\\n\");\n        content.append(\"<p>\\n\");\n        content.append(\" Licensed under the Apache License, Version 2.0 (the \\\"License\\\");</br>\\n\");\n        content.append(\" you may not use this software except in compliance with the License.</br>\\n\");\n        content.append(\" &nbsp;</br>\\n\");\n        content.append(\" You may obtain a copy of the License at</br>\\n\");\n        content.append(\" <a href=\\\"http://www.apache.org/licenses/LICENSE-2.0\\\" class=\\\"external text\\\" rel=\\\"nofollow\\\">http://www.apache.org/licenses/LICENSE-2.0</a>\\n\");\n        content.append(\"</p>\\n\");\n        content.append(\"<p>\\n\");\n        content.append(\" Unless required by applicable law or agreed to in writing, software</br>\\n\");\n        content.append(\" distributed under the License is distributed on an \\\"AS IS\\\" BASIS,</br>\\n\");\n        content.append(\" WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</br>\\n\");\n        content.append(\" See the License for the specific language governing permissions and</br>\\n\");\n        content.append(\" limitations under the License.\\n\");\n        content.append(\"</p>\\n\");\n        content.append(\"</div>\");\n        \n        content.append(\"<h2 id=\\\"mw-version-software\\\"> <span class=\\\"mw-headline\\\" id=\\\"Installed_software\\\">Installed software</span></h2>\\n\");\n        content.append(\"<table class=\\\"wikitable\\\" id=\\\"sv-software\\\">\\n\");\n        content.append(\" <tbody>\\n\");\n        content.append(\"  <tr>\\n\");\n        content.append(\"   <th>Product</th>\\n\");\n        content.append(\"   <th>Version</th>\\n\");\n        content.append(\"  </tr>\\n\");\n        content.append(\"  <tr>\\n\");\n        content.append(\"   <td><a href=\\\"https://code.google.com/p/scalaris/\\\" class=\\\"external text\\\" rel=\\\"nofollow\\\">Scalaris Wiki Example</a></td>\\n\");\n        content.append(\"   <td>\" + version + \"</td>\\n\");\n        content.append(\"  </tr>\\n\");\n        content.append(\"  <tr>\\n\");\n        content.append(\"   <td><a href=\\\"https://code.google.com/p/scalaris/\\\" class=\\\"external text\\\" rel=\\\"nofollow\\\">Scalaris</a></td>\\n\");\n        content.append(\"   <td>\" + getDbVersionStr(connection) + \"</td>\\n\");\n        content.append(\"  </tr>\\n\");\n        content.append(\"  <tr>\\n\");\n        content.append(\"   <td>Server</td>\\n\");\n        content.append(\"   <td>\" + getServerVersion() + \"</td>\\n\");\n        content.append(\"  </tr>\\n\");\n        content.append(\"  <tr>\\n\");\n        content.append(\"   <td><a href=\\\"https://code.google.com/p/gwtwiki/\\\" class=\\\"external text\\\" rel=\\\"nofollow\\\">bliki renderer</a></td>\\n\");\n        content.append(\"   <td>\" + getBlikiVersion() + \"</td>\\n\");\n        content.append(\"  </tr>\\n\");\n        content.append(\" </tbody>\\n\");\n        content.append(\"</table>\\n\");\n        \n        content.append(\"<h2 id=\\\"mw-version-ext\\\"> <span class=\\\"mw-headline\\\" id=\\\"Installed_extensions\\\">Installed extensions</span></h2>\\n\");\n        content.append(\"<table class=\\\"wikitable\\\" id=\\\"sv-ext\\\">\\n\");\n        content.append(\" <tbody>\\n\");\n        content.append(\"  <tr>\\n\");\n        content.append(\"   <th colspan=\\\"4\\\" id=\\\"sv-credits-specialpage\\\">Event handlers</th>\\n\");\n        content.append(\"  </tr>\\n\");\n        if (eventHandlers.isEmpty()) {\n            content.append(\"  <tr>\\n\");\n            content.append(\"   <td>-</td>\\n\");\n            content.append(\"   <td>-</td>\\n\");\n            content.append(\"   <td>-</td>\\n\");\n            content.append(\"   <td>-</td>\\n\");\n            content.append(\"  </tr>\\n\");\n        }\n        for (WikiEventHandler eventHandler : eventHandlers) {\n            content.append(\"  <tr>\\n\");\n            content.append(\"   <td><em><a class=\\\"external text\\\" href=\\\"\" + eventHandler.getURL() + \"\\\">\" + eventHandler.getName() + \"</a> (Version \" + eventHandler.getVersion() + \")</em></td>\\n\");\n            content.append(\"   <td><code title=\\\" \" + eventHandler.getClass().getCanonicalName() + \"\\\">\" + eventHandler.getClass().getSimpleName() + \"</code></td>\\n\");\n            content.append(\"   <td>\" + eventHandler.getDescription() + \"</td>\\n\");\n            content.append(\"   <td>\" + eventHandler.getAuthor() + \"</td>\\n\");\n            content.append(\"  </tr>\\n\");\n        }\n        content.append(\" </tbody>\\n\");\n        content.append(\"</table>\\n\");\n        \n        page.setPage(content.toString());\n        // abuse #handleViewSpecialPageList here:\n        ValueResult<List<NormalisedTitle>> result = new ValueResult<List<NormalisedTitle>>(\n                new ArrayList<InvolvedKey>(0),\n                new ArrayList<NormalisedTitle>(0));\n        handleViewSpecialPageList(request, response, result, connection, page);\n    }\n    \n    /**\n     * Shows an empty page for testing purposes.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param connection\n     *            connection to the database\n     * @param page\n     *            the bean for the page\n     * \n     * @throws IOException \n     * @throws ServletException \n     */\n    private void showEmptyPage(HttpServletRequest request,\n            HttpServletResponse response, Connection connection,\n            WikiPageBeanBase page) throws ServletException, IOException {\n        page.setNotice(getParam_notice(request));\n        page.setError(getParam_error(request));\n        forwardToPageJsp(request, response, connection, new WikiPageBean(page), \"page.jsp\");\n    }\n    \n    /**\n     * Shows a page for importing a DB dump (if implemented by the sub-class).\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param page\n     *            the bean for the page\n     * \n     * @throws IOException \n     * @throws ServletException \n     */\n    protected synchronized void showImportPage(HttpServletRequest request,\n            HttpServletResponse response, Connection connection,\n            WikiPageBean page) throws ServletException, IOException {\n    }\n\n    /**\n     * Shows the edit page form for an article with the given <tt>title</tt>.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param title\n     *            the title of the article to show\n     * @param page\n     *            the bean for the page\n     *            \n     * @throws IOException \n     * @throws ServletException \n     */\n    private void handleEditPage(HttpServletRequest request,\n            HttpServletResponse response, String title, Connection connection,\n            WikiPageEditBean page) throws ServletException, IOException {\n        // get revision id to load:\n        int req_oldid = getParam_oldid(request);\n\n        RevisionResult result = getRevision(connection, title, req_oldid, namespace);\n        \n        if (getParam_redlink(request) && !result.page_not_existing) {\n            handleViewPage2(request, response, title, connection,\n                    new WikiPageBean(page), req_oldid, result);\n            return;\n        }\n        \n        page.addStats(result.stats);\n        page.getInvolvedKeys().addAll(result.involvedKeys);\n        if (result.connect_failed) {\n            setParam_error(request, \"ERROR: DB connection failed\");\n            showEmptyPage(request, response, connection, page);\n            return;\n        } else if (result.rev_not_existing) {\n            result = getRevision(connection, title, namespace);\n            page.addStats(result.stats);\n            page.getInvolvedKeys().addAll(result.involvedKeys);\n            addToParam_notice(request, \"revision \" + req_oldid + \" not found - loaded current revision instead\");\n        }\n\n        if (result.page_not_existing) {\n            page.setVersion(-1);\n            page.setNewPage(true);\n        } else if (result.rev_not_existing) {\n            // DB corrupt\n            setParam_error(request, \"ERROR: revision unavailable\");\n            addToParam_notice(request, \"error: unknown error getting current revision of page \\\"\" + title + \"\\\": <pre>\" + result.message + \"</pre>\");\n            showEmptyPage(request, response, connection, page);\n            return;\n        }\n        if (result.success) {\n            if (!result.page.checkEditAllowed(\"\")) {\n                page.setEditRestricted(true);\n            }\n            page.setPage(StringEscapeUtils.escapeHtml(result.revision.unpackedText()));\n            page.setVersion(result.revision.getId());\n        } else if (!page.isNewPage()) {\n            page.setEditRestricted(true);\n        }\n\n        // set the textarea's contents:\n        page.setNotice(getParam_notice(request));\n        page.setError(getParam_error(request));\n        page.setTitle(title);\n        page.setWikiTitle(siteinfo.getSitename());\n        page.setWikiNamespace(namespace);\n\n        forwardToPageJsp(request, response, connection, page, \"pageEdit.jsp\");\n    }\n    \n    /**\n     * Shows a preview of the edit operation submitted or saves the page with\n     * the given <tt>title</tt> depending on what button the user clicked.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param title\n     *            the title of the article to show\n     * @param page\n     *            the bean for the page\n     * \n     * @throws IOException \n     * @throws UnsupportedEncodingException \n     * @throws ServletException \n     */\n    private void handleEditPageSubmitted(HttpServletRequest request,\n            HttpServletResponse response, String title, Connection connection,\n            WikiPageEditBean page) throws UnsupportedEncodingException,\n            IOException, ServletException {\n        String content = request.getParameter(\"wpTextbox1\");\n        String summary = request.getParameter(\"wpSummary\");\n        int oldVersion = parseInt(request.getParameter(\"oldVersion\"), -1);\n        boolean minorChange = Boolean.parseBoolean(request.getParameter(\"minor\"));\n\n        // save page or preview+edit page?\n        if (request.getParameter(\"wpSave\") != null) {\n            // save page\n            Contributor contributor = new Contributor();\n            contributor.setIp(request.getRemoteAddr());\n            String timestamp = Revision.calendarToString(Calendar.getInstance(TimeZone.getTimeZone(\"UTC\")));\n            Revision newRev = new Revision(-1, timestamp, minorChange, contributor, summary);\n            newRev.setUnpackedText(content);\n\n            SavePageResult result;\n            int retries = 0;\n            while (true) {\n                result = savePage(connection, title, newRev, oldVersion, null, siteinfo, \"\", namespace);\n                page.addStats(result.stats);\n                page.getInvolvedKeys().addAll(result.involvedKeys);\n                if (!result.failedKeys.isEmpty()) {\n                    page.getFailedKeys().put(retries + 1, result.failedKeys);\n                }\n                if (!result.success && retries < Options.getInstance().WIKI_SAVEPAGE_RETRIES) {\n                    // check for conflicting edit on same page, do not retry in this case\n                    final Page oldPage = result.oldPage;\n                    if (oldPage != null && oldPage.getCurRev().getId() != oldVersion) {\n                        break;\n                    }\n                    try {\n                        Thread.sleep(Options.getInstance().WIKI_SAVEPAGE_RETRY_DELAY);\n                    } catch (InterruptedException e) {\n                    }\n                    ++retries;\n                } else {\n                    break;\n                }\n            }\n            page.setSaveAttempts(retries + 1);\n            for (WikiEventHandler handler: eventHandlers) {\n                handler.onPageSaved(page, result, connection);\n            }\n            if (result.success) {\n                // successfully saved -> show page with a notice of the successful operation\n                // also actively update the bloom filter of existing pages\n                existingPages.add(NormalisedTitle.fromUnnormalised(title, namespace));\n                ArrayList<Long> times = new ArrayList<Long>();\n                for (List<Long> time : page.getStats().values()) {\n                    times.addAll(time);\n                }\n                // do not include the UTF-8-title directly into encodeRedirectURL since that's not \n                // encoding umlauts (maybe other special chars as well) correctly, e.g. ä -> %E4 instead of %C3%A4\n                StringBuilder redirectUrl = new StringBuilder(256);\n                redirectUrl.append(\"?title=\");\n                redirectUrl.append(URLEncoder.encode(title, \"UTF-8\"));\n                redirectUrl.append(\"&notice=successfully%20saved%20page\");\n                redirectUrl.append(\"&save_times=\" + StringUtils.join(times, \"%2C\"));\n                redirectUrl.append(\"&save_attempts=\" + page.getSaveAttempts());\n                for (Entry<Integer, List<String>> failedKeys : page.getFailedKeys().entrySet()) {\n                    redirectUrl.append(\"&failed_keys\" + failedKeys.getKey() + \"=\" + URLEncoder.encode(StringUtils.join(failedKeys.getValue(), \" # \"), \"UTF-8\"));\n                }\n                redirectUrl.append(\"&involved_keys=\" + URLEncoder.encode(StringUtils.join(page.getInvolvedKeys(), \" # \"), \"UTF-8\"));\n                redirectUrl.append(\"&server_time=\" + (System.currentTimeMillis() - page.getStartTime()));\n                if (result.newPage.isRedirect()) {\n                    redirectUrl.append(\"&redirect=no\");\n                }\n                final String serviceUser = page.getServiceUser().isEmpty() ? \"\" : \"&service_user=\" + page.getServiceUser();\n                response.sendRedirect(\"http://\"\n                        + Options.getInstance().SERVERNAME\n                        + Options.getInstance().SERVERPATH\n                        + response.encodeRedirectURL(redirectUrl.toString())\n                        + serviceUser);\n                return;\n            } else {\n                // set error message and show the edit page again (see below)\n                if (result.connect_failed) {\n                    setParam_error(request, \"ERROR: DB connection failed\");\n                } else {\n                    setParam_error(request, \"ERROR: conflicting edit\");\n                }\n                addToParam_notice(request, \"error: could not save page: <pre>\" + result.message + \"</pre>\");\n            }\n        }\n\n        // preview+edit page\n\n        page.setNotice(getParam_notice(request));\n        // set the textarea's contents:\n        page.setPage(StringEscapeUtils.escapeHtml(content));\n\n        MyWikiModel wikiModel = getWikiModel(connection, page);\n        String[] titleParts = wikiModel.splitNsTitle(title);\n        wikiModel.setNamespaceName(titleParts[0]);\n        wikiModel.setPageName(titleParts[1]);\n        page.setPreview(wikiModel.renderPageWithCache(content));\n        page.setIncludes(wikiModel.getIncludes());\n        page.setTemplates(wikiModel.getTemplatesNoMagicWords());\n        page.addStats(wikiModel.getStats());\n        page.getInvolvedKeys().addAll(wikiModel.getInvolvedKeys());\n        page.setPage(content);\n        page.setVersion(oldVersion);\n        page.setError(getParam_error(request));\n        page.setTitle(title);\n        page.setSummary(request.getParameter(\"wpSummary\"));\n        page.setWikiTitle(siteinfo.getSitename());\n        page.setWikiNamespace(namespace);\n\n        forwardToPageJsp(request, response, connection, page, \"pageEdit.jsp\");\n    }\n    \n    /**\n     * Shows a preview of the edit operation submitted or saves the page with\n     * the given <tt>title</tt> depending on what button the user clicked.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * @param title\n     *            the title of the article to show\n     * \n     * @throws IOException \n     * @throws UnsupportedEncodingException \n     * @throws ServletException \n     */\n    private void showImage(final HttpServletRequest request,\n            final HttpServletResponse response, final String image)\n            throws UnsupportedEncodingException, IOException, ServletException {\n        // we need to fix the image title first, e.g. a prefix with the desired size may exist\n        final String imageNoSize = MATCH_WIKI_IMAGE_PX.matcher(image).replaceFirst(\"\");\n        String realImageUrl = getWikiImageUrl(imageNoSize);\n        if (realImageUrl == null) {\n            // bliki may have created \".svg.png\" from an original \".svg\" image:\n            String new_image = MATCH_WIKI_IMAGE_SVG_PNG.matcher(imageNoSize).replaceFirst(\".svg\");\n            realImageUrl = getWikiImageUrl(new_image);\n            if (imageNoSize.equals(new_image) || realImageUrl == null) {\n                realImageUrl = response.encodeRedirectURL(\"images/image.png\");\n            }\n        }\n\n        for (WikiEventHandler handler: eventHandlers) {\n            handler.onImageRedirect(image, realImageUrl);\n        }\n        \n        response.sendRedirect(realImageUrl);\n    }\n\n    /**\n     * Retrieves the URL of an image from the Wikipedia related to the base URL\n     * of this wiki.\n     * \n     * @param image\n     *            the name of the image as created by the bliki engine\n     */\n    protected String getWikiImageUrl(String image) {\n        // add namespace - \"Image\" is a default alias for \"File\" in any language\n        image = new String(\"Image:\" + image);\n        String fullBaseUrl = siteinfo.getBase();\n        String baseUrl = \"http://en.wikipedia.org\";\n        Matcher matcher = MATCH_WIKI_SITE_BASE.matcher(fullBaseUrl);\n        if (matcher.matches()) {\n            baseUrl = matcher.group(1);\n        }\n        User user = new User(\"\", \"\", baseUrl + \"/w/api.php\");\n        Connector connector = new Connector();\n        user = connector.login(user);\n\n        // set image width thumb size to 400px\n        List<info.bliki.api.Page> pages = user.queryImageinfo(new String[] { image }, 400);\n        if (pages.size() == 1) {\n            info.bliki.api.Page imagePage = pages.get(0);\n//            System.out.println(\"IMG-THUMB-URL: \" + imagePage.getImageThumbUrl());\n//            System.out.println(\"IMG-URL: \" + imagePage.getImageUrl());\n\n            if (imagePage.getImageThumbUrl() != null && !imagePage.getImageThumbUrl().isEmpty()) {\n                return imagePage.getImageThumbUrl();\n            }\n        }\n        return null;\n    }\n    \n    abstract protected MyWikiModel getWikiModel(Connection connection, WikiPageBeanBase page);\n\n    /**\n     * Adds the given notice to the notice attribute.\n     * \n     * @param request\n     *            the http request\n     * @param notice\n     *            the notice to add\n     */\n    public static void addToParam_notice(HttpServletRequest request, String notice) {\n        String req_notice = getParam_notice(request);\n        String new_notice = req_notice.isEmpty() ? notice : req_notice + \"<br />\" + notice;\n        request.setAttribute(\"notice\", new_notice);\n    }\n\n    /**\n     * Returns the notice parameter.\n     * \n     * @param request\n     *            the http request\n     * \n     * @return the notice parameter or \"\"\n     * \n     * @see #getParam(HttpServletRequest, String)\n     */\n    public static String getParam_notice(HttpServletRequest request) {\n        return getParam(request, \"notice\");\n    }\n\n    /**\n     * Returns the given parameter (or attribute if the parameter does not\n     * exist, or an empty string if both are not present).\n     * \n     * @param request\n     *            the http request\n     * @param name\n     *            the name of the parameter\n     * \n     * @return the requested parameter or \"\"\n     */\n    public static String getParam(HttpServletRequest request, String name) {\n        String parValue = request.getParameter(name);\n        if (parValue == null) {\n            Object temp = request.getAttribute(name);\n            if (temp instanceof String) {\n                parValue = (String) temp; \n            }\n        }\n        if (parValue == null) {\n            return new String(\"\");\n        } else {\n            return parValue;\n        }\n    }\n\n    /**\n     * Returns the error parameter.\n     * \n     * @param request\n     *            the http request\n     * \n     * @return the error parameter or \"\"\n     * \n     * @see #getParam(HttpServletRequest, String)\n     */\n    public static String getParam_error(HttpServletRequest request) {\n        return getParam(request, \"error\");\n    }\n\n    /**\n     * Sets the error attribute. Once set, it cannot be changed with this\n     * method.\n     * \n     * @param request\n     *            the http request\n     * @param error\n     *            the error to set\n     */\n    public static void setParam_error(HttpServletRequest request, String error) {\n        String req_error = getParam(request, \"error\");\n        if (req_error.isEmpty()) {\n            request.setAttribute(\"error\", error + \" - \");\n        }\n    }\n\n    /**\n     * Get the revision id to load from the request object\n     * \n     * @param request\n     *            the http request\n     * @return the revision id or -1 on failure to parse\n     */\n    private static int getParam_oldid(HttpServletRequest request) {\n        String req_oldid = request.getParameter(\"oldid\");\n        return parseInt(req_oldid, -1);\n    }\n\n    /**\n     * Determines which renderer should be used by evaluating the render\n     * parameter of the request\n     * \n     * @param request\n     *            the http request\n     * @return the renderer id\n     */\n    private static int getParam_renderer(HttpServletRequest request) {\n        int render = 1;\n        String req_render = request.getParameter(\"render\");\n        if (req_render == null) {\n            // already set to 1, so the default renderer is used\n        } else if (req_render.equals(\"0\")) {\n            render = 0;\n        } else if (req_render.equals(\"-1\")) {\n            render = -1;\n        }\n        return render;\n    }\n\n    /**\n     * Check whether the page was reached by a \"redlink\" URL (in this case, if\n     * the edit page was requested but the page exists, show the page instead)\n     * \n     * @param request\n     *            the http request\n     * @return <tt>true</tt> if it was a redlink, <tt>false</tt> otherwise\n     */\n    private static boolean getParam_redlink(HttpServletRequest request) {\n        String req_redlink = request.getParameter(\"redlink\");\n        return parseInt(req_redlink, 0) == 1;\n    }\n    \n    protected final static int parseInt(String value, int def) {\n        if (value == null) {\n            return def;\n        }\n        try {\n            return Integer.parseInt(value);\n        } catch (NumberFormatException e) {\n            return def;\n        }\n    }\n    \n    protected final static Calendar parseDate(String value, Calendar def) {\n        if (value == null) {\n            return def;\n        }\n        try {\n            return Revision.stringToCalendar(value);\n        } catch (IllegalArgumentException e) {\n            return def;\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.bliki.WikiServletContext#getNamespace()\n     */\n    @Override\n    public final NamespaceUtils getNamespace() {\n        return namespace;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.bliki.WikiServletContext#getSiteinfo()\n     */\n    @Override\n    public SiteInfo getSiteinfo() {\n        return siteinfo;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.bliki.WikiServletContext#getVersion()\n     */\n    @Override\n    public String getVersion() {\n        return version;\n    }\n\n    @Override\n    public String getDbVersion() {\n        Connection connection = getConnection(null);\n        if (connection == null) {\n            return \"n/a\"; // return just in case\n        }\n        return getDbVersionStr(connection);\n    }\n\n    protected String getDbVersionStr(Connection connection) {\n        ValueResult<String> dbVersion = getDbVersion(connection);\n        if (dbVersion.success) {\n            return dbVersion.value;\n        } else {\n            return \"n/a\";\n        }\n    }\n\n    @Override\n    public String getServerVersion() {\n        return getServletContext().getServerInfo();\n    }\n\n    @Override\n    public String getBlikiVersion() {\n        return Configuration.BLIKI_VERSION;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.bliki.WikiServletContext#getLinkbaseurl(WikiPageBeanBase)\n     */\n    @Override\n    public String getLinkbaseurl(WikiPageBeanBase page) {\n        final String serviceUser = page.getServiceUser().isEmpty() ? \"\" : \"&service_user=\" + page.getServiceUser();\n        return Options.getInstance().SERVERPATH + \"?title=${title}\" + serviceUser;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.bliki.WikiServletContext#getImagebaseurl(WikiPageBeanBase)\n     */\n    @Override\n    public String getImagebaseurl(WikiPageBeanBase page) {\n        final String serviceUser = page.getServiceUser().isEmpty() ? \"\" : \"&service_user=\" + page.getServiceUser();\n        return Options.getInstance().SERVERPATH + \"?get_image=${image}\" + serviceUser;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.bliki.WikiServletContext#getImagebaseurl()\n     */\n    @Override\n    public synchronized void registerEventHandler(WikiEventHandler handler) {\n        eventHandlers.add(handler);\n    }\n\n    /**\n     * Updates the bloom filter of existing pages for quick checks.\n     */\n    protected void updateExistingPages() {\n        if (initialized) {\n            Connection connection = getConnection(null);\n            if (connection != null) {\n                try {\n                    ValueResult<List<NormalisedTitle>> result = getPageList(connection);\n                    if (result.success) {\n                        List<NormalisedTitle> pages = result.value;\n                        pages.addAll(specialPages);\n                        ExistingPagesCache filter = ExistingPagesCache.createCache(pages);\n                        existingPages = filter;\n                    }\n                } finally {\n                    releaseConnection(null, connection);\n                }\n            }\n        }\n    }\n    \n    @Override\n    public void storeUserReq(WikiPageBeanBase page, long servertime) {\n        long timestamp = page.getStartTime();\n        String serviceUser = page.getServiceUser();\n        if (!eventHandlers.isEmpty()) {\n            Connection connection = getConnection(null);\n            for (WikiEventHandler handler: eventHandlers) {\n                try {\n                    handler.onPageView(page, connection);\n                } catch (Exception e) {\n                    e.printStackTrace();\n                }\n            }\n            releaseConnection(null, connection);\n        }\n        if (userReqLogs != null && userReqLogs.length > 0) {\n            int timestamp_s = (int) (timestamp / 1000);\n            // build log entry:\n            Map<String, Object> entry = new HashMap<String, Object>();\n            entry.put(\"timestamp\", timestamp_s);\n            entry.put(\"serviceuser\", serviceUser == null ? \"\" : serviceUser);\n            entry.put(\"servertime\", servertime);\n            \n            // get the correct bucket:\n            LinkedList<Map<String, Object>> curReqLogList;\n            synchronized (userReqLogs) {\n                curReqLogList = userReqLogs[curReqLog];\n                int diffInMin = (timestamp_s - curReqLogStartTime) / 60;\n                if (diffInMin > 0) {\n                    if (diffInMin > userReqLogs.length) {\n                        for (LinkedList<Map<String, Object>> log : userReqLogs) {\n                            log.clear();\n                        }\n                        curReqLog = 0;\n                        curReqLogList = userReqLogs[curReqLog];\n                    } else  {\n                        while ((curReqLogStartTime + 60) < timestamp_s) {\n                            curReqLogList = userReqLogs[curReqLog];\n                            curReqLogList.clear();\n                            curReqLog = (++curReqLog) % userReqLogs.length;\n                        }\n                    }\n                    curReqLogStartTime += 60 * diffInMin;\n                }\n            }\n            curReqLogList.add(entry);\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/WikiServletDataHandler.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport java.math.BigInteger;\nimport java.util.List;\nimport java.util.Map;\nimport java.util.Random;\n\nimport de.zib.scalaris.examples.wikipedia.PageHistoryResult;\nimport de.zib.scalaris.examples.wikipedia.RevisionResult;\nimport de.zib.scalaris.examples.wikipedia.SavePageResult;\nimport de.zib.scalaris.examples.wikipedia.ValueResult;\nimport de.zib.scalaris.examples.wikipedia.data.Contribution;\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\n\n/**\n * Interface for data-retrieving methods used by {@link WikiServlet}.\n * \n * @param <Connection> connection to a DB\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic interface WikiServletDataHandler<Connection> {\n    /**\n     * Gets the key to store {@link SiteInfo} objects at.\n     * \n     * @return DB key\n     */\n    public String getSiteInfoKey();\n    \n    /**\n     * Gets the key to store the list of pages in the given namespace at.\n     * \n     * @param namespace  the namespace ID\n     * \n     * @return DB key\n     */\n    public String getPageListKey(int namespace);\n    \n    /**\n     * Gets the key to store the number of pages at.\n     * \n     * @param namespace  the namespace ID\n     * \n     * @return DB key\n     */\n    public String getPageCountKey(int namespace);\n    \n    /**\n     * Gets the key to store the number of articles, i.e. pages in the main\n     * namespace, at.\n     * \n     * @return DB key\n     */\n    public String getArticleCountKey();\n    \n    /**\n     * Gets the key to store {@link Revision} objects at.\n     * \n     * @param title\n     *            the title of the page\n     * @param id\n     *            the id of the revision\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return DB key\n     */\n    public String getRevKey(String title, int id, final MyNamespace nsObject);\n    \n    /**\n     * Gets the key to store {@link Page} objects at.\n     * \n     * @param title\n     *            the title of the page\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return DB key\n     */\n    public String getPageKey(String title, final MyNamespace nsObject);\n    \n    /**\n     * Gets the key to store the list of revisions of a page at.\n     * \n     * @param title\n     *            the title of the page\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return DB key\n     */\n    public String getRevListKey(String title, final MyNamespace nsObject);\n    \n    /**\n     * Gets the key to store the list of pages belonging to a category at.\n     * \n     * @param title\n     *            the category title (including <tt>Category:</tt>)\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return DB key\n     */\n    public String getCatPageListKey(String title, final MyNamespace nsObject);\n    \n    /**\n     * Gets the key to store the number of pages belonging to a category at.\n     * \n     * @param title\n     *            the category title (including <tt>Category:</tt>)\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return DB key\n     */\n    public String getCatPageCountKey(String title, final MyNamespace nsObject);\n    \n    /**\n     * Gets the key to store the list of pages using a template at.\n     * \n     * @param title\n     *            the template title (including <tt>Template:</tt>)\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return DB key\n     */\n    public String getTplPageListKey(String title, final MyNamespace nsObject);\n    \n    /**\n     * Gets the key to store the list of pages linking to the given title.\n     * \n     * @param title\n     *            the page's title\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return DB key\n     */\n    public String getBackLinksPageListKey(String title, final MyNamespace nsObject);\n    \n    /**\n     * Gets the key to store the number of page edits.\n     * \n     * @return DB key\n     */\n    public String getStatsPageEditsKey();\n    \n    /**\n     * Gets the key to store the list of contributions of a user.\n     * \n     * @param contributor  the user name or IP address of the user who created\n     *                     the revision\n     * \n     * @return DB key\n     */\n    public String getContributionListKey(String contributor);\n\n    \n    /**\n     * Retrieves the back-ends version string.\n     * \n     * @param connection\n     *            the connection to the DB\n     * \n     * @return a result object with the version string on success\n     */\n    public ValueResult<String> getDbVersion(Connection connection);\n\n    /**\n     * Retrieves a page's history from the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * @param title\n     *            the title of the page\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return a result object with the page history on success\n     */\n    public PageHistoryResult getPageHistory(Connection connection,\n            String title, final MyNamespace nsObject);\n\n    /**\n     * Retrieves the current, i.e. most up-to-date, version of a page from\n     * the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * @param title\n     *            the title of the page\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return a result object with the page and revision on success\n     */\n    public RevisionResult getRevision(Connection connection, String title,\n            final MyNamespace nsObject);\n    \n    /**\n     * Retrieves the given version of a page from the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * @param title\n     *            the title of the page\n     * @param id\n     *            the id of the version\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return a result object with the page and revision on success\n     */\n    public RevisionResult getRevision(Connection connection, String title,\n            int id, final MyNamespace nsObject);\n    \n    /**\n     * Retrieves a list of all available pages from the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * \n     * @return a result object with the page list on success\n     */\n    public ValueResult<List<NormalisedTitle>> getPageList(Connection connection);\n    \n    /**\n     * Retrieves a list of available pages in the given namespace from the DB.\n     * \n     * @param namespace\n     *            the namespace ID\n     * @param connection\n     *            the connection to the DB\n     * \n     * @return a result object with the page list on success\n     */\n    public ValueResult<List<NormalisedTitle>> getPageList(int namespace, Connection connection);\n    \n    /**\n     * Retrieves a list of pages in the given category from the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * @param title\n     *            the title of the category\n     * \n     * @return a result object with the page list on success\n     */\n    public ValueResult<List<NormalisedTitle>> getPagesInCategory(Connection connection,\n            NormalisedTitle title);\n    \n    /**\n     * Retrieves a list of pages using the given template from the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * @param title\n     *            the title of the template\n     * \n     * @return a result object with the page list on success\n     */\n    public ValueResult<List<NormalisedTitle>> getPagesInTemplate(Connection connection,\n            NormalisedTitle title);\n\n    /**\n     * Retrieves a list of pages using the given templates from the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * @param titles\n     *            the titles of the templates\n     * @param pageTitle\n     *            the title of the page to retrieve the list for (will be\n     *            included in the statname)\n     * \n     * @return a result object with the page list on success\n     */\n    public ValueResult<List<NormalisedTitle>> getPagesInTemplates(Connection connection,\n            List<NormalisedTitle> titles, String pageTitle);\n    \n    /**\n     * Retrieves a list of pages linking to the given page from the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * @param title\n     *            the title of the page\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return a result object with the page list on success\n     */\n    public ValueResult<List<NormalisedTitle>> getPagesLinkingTo(Connection connection,\n            String title, final MyNamespace nsObject);\n\n    /**\n     * Retrieves a list of pages linking to the given page from the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * @param contributor\n     *            the user name or IP address of the user who created the\n     *            revision\n     * \n     * @return a result object with the page list on success\n     */\n    public ValueResult<List<Contribution>> getContributions(\n            Connection connection, String contributor);\n    \n    /**\n     * Retrieves the number of all available pages from the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * \n     * @return a result object with the number of pages on success\n     */\n    public ValueResult<BigInteger> getPageCount(Connection connection);\n    \n    /**\n     * Retrieves the number of available pages in the given namespace from the\n     * DB.\n     * \n     * @param namespace\n     *            the namespace ID\n     * @param connection\n     *            the connection to the DB\n     * \n     * @return a result object with the number of pages on success\n     */\n    public ValueResult<BigInteger> getPageCount(int namespace, Connection connection);\n    \n    /**\n     * Retrieves the number of available articles, i.e. pages in the main\n     * namespace, from the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * \n     * @return a result object with the number of articles on success\n     */\n    public ValueResult<BigInteger> getArticleCount(Connection connection);\n\n    /**\n     * Retrieves the number of pages in the given category from the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * @param title\n     *            the title of the category\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return a result object with the number of pages on success\n     */\n    public ValueResult<BigInteger> getPagesInCategoryCount(Connection connection,\n            String title, final MyNamespace nsObject);\n    \n    /**\n     * Retrieves the number of available articles, i.e. pages in the main\n     * namespace, from the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * \n     * @return a result object with the number of articles on success\n     */\n    public ValueResult<BigInteger> getStatsPageEdits(Connection connection);\n    \n    /**\n     * Retrieves a random page title from the DB.\n     * \n     * @param connection\n     *            the connection to the DB\n     * @param random\n     *            the random number generator to use\n     * \n     * @return a result object with the page list on success\n     */\n    public ValueResult<NormalisedTitle> getRandomArticle(Connection connection, Random random);\n    \n    /**\n     * Saves or edits a page with the given parameters\n     * \n     * @param connection\n     *            the connection to use\n     * @param title\n     *            the title of the page\n     * @param newRev\n     *            the new revision to add\n     * @param prevRevId\n     *            the version of the previously existing revision or <tt>-1</tt>\n     *            if there was no previous revision or <tt>-2</tt> if the\n     *            previous revision is unknown and should be determined during\n     *            the save\n     * @param restrictions\n     *            new restrictions of the page or <tt>null</tt> if they should\n     *            not be changed\n     * @param siteinfo\n     *            information about the wikipedia (used for parsing categories\n     *            and templates)\n     * @param username\n     *            name of the user editing the page (for enforcing restrictions)\n     * @param nsObject\n     *            the namespace for page title normalisation\n     * \n     * @return success status\n     */\n    public SavePageResult savePage(Connection connection, String title,\n            Revision newRev, int prevRevId, Map<String, String> restrictions,\n            SiteInfo siteinfo, String username, final MyNamespace nsObject);\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/bliki/WikiServletScalaris.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.bliki;\n\nimport java.io.File;\nimport java.io.FilenameFilter;\nimport java.io.IOException;\nimport java.io.InputStream;\nimport java.io.PrintStream;\nimport java.math.BigInteger;\nimport java.util.Arrays;\nimport java.util.Calendar;\nimport java.util.List;\nimport java.util.Map;\nimport java.util.Properties;\nimport java.util.Random;\nimport java.util.TreeSet;\n\nimport javax.servlet.ServletConfig;\nimport javax.servlet.ServletException;\nimport javax.servlet.http.HttpServletRequest;\nimport javax.servlet.http.HttpServletResponse;\n\nimport org.xml.sax.InputSource;\nimport org.xml.sax.XMLReader;\nimport org.xml.sax.helpers.XMLReaderFactory;\n\nimport de.zib.scalaris.Connection;\nimport de.zib.scalaris.ConnectionFactory;\nimport de.zib.scalaris.ConnectionPool;\nimport de.zib.scalaris.NodeDiscovery;\nimport de.zib.scalaris.TransactionSingleOp;\nimport de.zib.scalaris.examples.wikipedia.Options;\nimport de.zib.scalaris.examples.wikipedia.PageHistoryResult;\nimport de.zib.scalaris.examples.wikipedia.RevisionResult;\nimport de.zib.scalaris.examples.wikipedia.SavePageResult;\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandlerNormalised;\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandlerUnnormalised;\nimport de.zib.scalaris.examples.wikipedia.ValueResult;\nimport de.zib.scalaris.examples.wikipedia.data.Contribution;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\nimport de.zib.scalaris.examples.wikipedia.data.xml.SAXParsingInterruptedException;\nimport de.zib.scalaris.examples.wikipedia.data.xml.WikiDump;\nimport de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpHandler;\nimport de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpPreparedSQLiteToScalaris;\nimport de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpToScalarisHandler;\nimport de.zib.tools.CircularByteArrayOutputStream;\n\n/**\n * Wiki servlet connecting to Scalaris.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class WikiServletScalaris extends WikiServlet<Connection> {\n\n    private static final long serialVersionUID = 1L;\n    private static final int CONNECTION_POOL_SIZE = 200;\n    private static final int MAX_WAIT_FOR_CONNECTION = 10000; // 10s\n    \n    private ConnectionPool cPool;\n    protected NodeDiscovery nodeDiscovery;\n    private boolean autoImport;\n\n    /**\n     * Default constructor creating the servlet.\n     */\n    public WikiServletScalaris() {\n        super();\n    }\n\n    /**\n     * Servlet initialisation: creates the connection to the erlang node and\n     * imports site information.\n     */\n    @Override\n    public void init2(ServletConfig config) throws ServletException {\n        super.init2(config);\n        Properties properties = new Properties();\n        try {\n            InputStream fis = config.getServletContext().getResourceAsStream(\"/WEB-INF/scalaris.properties\");\n            if (fis != null) {\n                properties.load(fis);\n                properties.setProperty(\"PropertyLoader.loadedfile\", \"/WEB-INF/scalaris.properties\");\n                fis.close();\n            } else {\n                properties = null;\n            }\n        } catch (IOException e) {\n//            e.printStackTrace();\n            properties = null;\n        }\n        \n        ConnectionFactory cFactory;\n        if (properties != null) {\n            cFactory = new ConnectionFactory(properties);\n        } else {\n            cFactory = new ConnectionFactory();\n            cFactory.setClientName(\"wiki\");\n        }\n        Random random = new Random();\n        String clientName = new BigInteger(128, random).toString(16);\n        cFactory.setClientName(cFactory.getClientName() + '_' + clientName);\n        cFactory.setClientNameAppendUUID(true);\n//        cFactory.setConnectionPolicy(new RoundRobinConnectionPolicy(cFactory.getNodes()));\n\n        cPool = new ConnectionPool(cFactory, CONNECTION_POOL_SIZE);\n        if (Options.getInstance().SCALARIS_NODE_DISCOVERY > 0) {\n            nodeDiscovery = new NodeDiscovery(cPool);\n            nodeDiscovery.startWithFixedDelay(Options.getInstance().SCALARIS_NODE_DISCOVERY);\n        }\n    }\n\n    @Override\n    protected void startAutoImport() {\n        String dumpsPath = getServletContext().getRealPath(\"/WEB-INF/dumps\");\n        if (!initialized && !loadSiteInfo() || !currentImport.isEmpty()) {\n            String req_import = null;\n            // get auto-import dumps:\n            File dumpsDir = new File(dumpsPath);\n            if (dumpsDir.isDirectory()) {\n                List<String> autoImportFiles = Arrays.asList(dumpsDir.list(new FilenameFilter() {\n                    @Override\n                    public boolean accept(File dir, String name) {\n                        return MATCH_WIKI_AUTOIMPORT_FILE.matcher(name).matches();\n                    }\n                }));\n                if (!autoImportFiles.isEmpty()) {\n                    // use the first auto-import file\n                    req_import = autoImportFiles.get(0);\n                    // remove .auto from filename:\n                    req_import = req_import.substring(0, req_import.length() - \".auto\".length());\n                    startImport(dumpsPath, req_import, 2, null);\n                    autoImport = true;\n                }\n            }\n        }\n    }\n    \n    /**\n     * Loads the siteinfo object from Scalaris.\n     * \n     * @return <tt>true</tt> on success,\n     *         <tt>false</tt> if not found or no connection available\n     */\n    @Override\n    protected synchronized boolean loadSiteInfo() {\n        TransactionSingleOp scalaris_single;\n        try {\n            Connection conn = cPool.getConnection(MAX_WAIT_FOR_CONNECTION);\n            if (conn == null) {\n                System.err.println(\"Could not get a connection to Scalaris for siteinfo, waited \" + MAX_WAIT_FOR_CONNECTION + \"ms\");\n                return false;\n            }\n            scalaris_single = new TransactionSingleOp(conn);\n            try {\n                siteinfo = scalaris_single.read(\"siteinfo\").jsonValue(SiteInfo.class);\n                // TODO: fix siteinfo's base url\n                namespace = new MyNamespace(siteinfo);\n                initialized = true;\n                setLocalisedSpecialPageNames();\n            } catch (Exception e) {\n                // no warning here - this probably is an empty wiki\n                return false;\n            }\n        } catch (Exception e) {\n            System.out.println(e);\n            e.printStackTrace();\n            return false;\n        }\n        return true;\n    }\n\n    /**\n     * Sets up the connection to the Scalaris erlang node once on the server.\n     * \n     * In case of errors, the <tt>error</tt> and <tt>notice</tt> attributes of\n     * the <tt>request</tt> object are set appropriately if not <tt>null</tt>.\n     * \n     * @param request\n     *            the request to the servlet (may be <tt>null</tt>)\n     * \n     * @return a valid connection of <tt>null</tt> if an error occurred\n     */\n    @Override\n    protected Connection getConnection(HttpServletRequest request) {\n        try {\n            Connection conn = cPool.getConnection(MAX_WAIT_FOR_CONNECTION);\n            if (conn == null) {\n                System.err.println(\"Could not get a connection to Scalaris, waited \" + MAX_WAIT_FOR_CONNECTION + \"ms\");\n                if (request != null) {\n                    setParam_error(request, \"ERROR: DB unavailable\");\n                    addToParam_notice(request, \"error: <pre>Could not get a connection to Scalaris, waited \" + MAX_WAIT_FOR_CONNECTION + \"ms</pre>\");\n                }\n                return null;\n            }\n            return conn;\n        } catch (Exception e) {\n            if (request != null) {\n                setParam_error(request, \"ERROR: DB unavailable\");\n                addToParam_notice(request, \"error: <pre>\" + e.getMessage() + \"</pre>\");\n            } else {\n                System.out.println(e);\n                e.printStackTrace();\n            }\n            return null;\n        }\n    }\n\n    /**\n     * Releases the connection back into the Scalaris connection pool.\n     * \n     * @param request\n     *            the request to the servlet or <tt>null</tt> if there is none\n     * @param conn\n     *            the connection to release\n     */\n    @Override\n    protected void releaseConnection(HttpServletRequest request, Connection conn) {\n        cPool.releaseConnection(conn);\n    }\n    \n    /**\n     * Shows a page for importing a DB dump.\n     * \n     * @param request\n     *            the request of the current operation\n     * @param response\n     *            the response of the current operation\n     * \n     * @throws IOException \n     * @throws ServletException \n     */\n    @Override\n    protected synchronized void showImportPage(HttpServletRequest request,\n            HttpServletResponse response, Connection connection,\n            WikiPageBean page) throws ServletException, IOException {\n        page.setNotAvailable(true);\n        \n        StringBuilder content = new StringBuilder();\n        String dumpsPath = getServletContext().getRealPath(\"/WEB-INF/dumps\");\n        final String serviceUser = page.getServiceUser().isEmpty() ? \"\" : \"&service_user=\" + page.getServiceUser();\n        \n        if (currentImport.isEmpty() && importHandler == null) {\n            TreeSet<String> availableDumps = new TreeSet<String>();\n            File dumpsDir = new File(dumpsPath);\n            if (dumpsDir.isDirectory()) {\n                availableDumps.addAll(Arrays.asList(dumpsDir.list(new FilenameFilter() {\n                    @Override\n                    public boolean accept(File dir, String name) {\n                        return MATCH_WIKI_IMPORT_FILE.matcher(name).matches();\n                    }\n                })));\n            }\n\n            // get parameters:\n            String req_import = request.getParameter(\"import\");\n            if (req_import == null || !availableDumps.contains(req_import)) {\n                content.append(\"<h2>Please select a wiki dump to import</h2>\\n\");\n                \n                content.append(\"<form method=\\\"get\\\" action=\\\"wiki\\\">\\n\");\n                if (!page.getServiceUser().isEmpty()) {\n                    content.append(\"<input type=\\\"hidden\\\" value=\\\"\" + page.getServiceUser() + \"\\\" name=\\\"service_user\\\"/>\");\n                }\n                content.append(\"<p>\\n\");\n                content.append(\"  <select name=\\\"import\\\" size=\\\"10\\\" style=\\\"width:500px;\\\">\\n\");\n                for (String dump: availableDumps) {\n                    content.append(\"   <option>\" + dump + \"</option>\\n\");\n                }\n                content.append(\"  </select>\\n\");\n                content.append(\" </p>\\n\");\n                content.append(\" <p>Maximum number of revisions per page: <input name=\\\"max_revisions\\\" size=\\\"2\\\" value=\\\"2\\\" /></br><span style=\\\"font-size:80%\\\">(<tt>-1</tt> to import everything)</span></p>\\n\");\n                content.append(\" <p>No entry newer than: <input name=\\\"max_time\\\" size=\\\"20\\\" value=\\\"\\\" /></br><span style=\\\"font-size:80%\\\">(ISO8601 format, e.g. <tt>2004-01-07T08:09:29Z</tt> - leave empty to import everything)</span></p>\\n\");\n                content.append(\" <input type=\\\"submit\\\" value=\\\"Import\\\" />\\n\");\n                content.append(\"</form>\\n\");\n                content.append(\"<p>Note: You will be re-directed to the main page when the import finishes.</p>\");\n            } else {\n                content.append(\"<h2>Importing \\\"\" + req_import + \"\\\"...</h2>\\n\");\n                try {\n                    int maxRevisions = parseInt(request.getParameter(\"max_revisions\"), 2);\n                    Calendar maxTime = parseDate(request.getParameter(\"max_time\"), null);\n                    startImport(dumpsPath, req_import, maxRevisions, maxTime);\n                    response.setHeader(\"Refresh\", \"2; url = wiki?import=\" + currentImport + serviceUser + \"#refresh\");\n                    content.append(\"<p>Current log file (refreshed automatically every \" + IMPORT_REDIRECT_EVERY + \" seconds):</p>\\n\");\n                    content.append(\"<pre>\");\n                    content.append(\"starting import...\\n\");\n                    content.append(\"</pre>\");\n                    content.append(\"<p><a name=\\\"refresh\\\" href=\\\"wiki?import=\" + currentImport + serviceUser + \"#refresh\\\">refresh</a></p>\");\n                    if (importHandler.hasStopSupport()) {\n                        content.append(\"<p><a href=\\\"wiki?stop_import=\" + currentImport + serviceUser + \"\\\">stop</a> (WARNING: pages may be incomplete due to missing templates)</p>\");\n                    }\n                } catch (Exception e) {\n                    setParam_error(request, \"ERROR: import failed\");\n                    addToParam_notice(request, \"error: <pre>\" + e.getMessage() + \"</pre>\");\n                    currentImport = \"\";\n                }\n            }\n        } else if (!currentImport.isEmpty() && importHandler != null) {\n            content.append(\"<h2>Importing \\\"\" + currentImport + \"\\\"...</h2>\\n\");\n            \n            String req_stop_import = request.getParameter(\"stop_import\");\n            boolean stopImport = false;\n            if (importHandler.hasStopSupport() && req_stop_import != null && !req_stop_import.isEmpty()) {\n                stopImport = true;\n                importHandler.stopParsing();\n                content.append(\"<p>Current log file:</p>\\n\");\n            } else {\n                response.setHeader(\"Refresh\", IMPORT_REDIRECT_EVERY + \"; url = wiki?import=\" + currentImport + serviceUser + \"#refresh\");\n                content.append(\"<p>Current log file (refreshed automatically every \" + IMPORT_REDIRECT_EVERY + \" seconds):</p>\\n\");\n            }\n            content.append(\"<pre>\");\n            String log = importLog.toString();\n            int start = log.indexOf(\"\\n\");\n            if (start != -1) { \n                content.append(log.substring(start));\n            }\n            content.append(\"</pre>\");\n            if (!stopImport) {\n                content.append(\"<p><a name=\\\"refresh\\\" href=\\\"wiki?import=\" + currentImport + serviceUser + \"#refresh\\\">refresh</a></p>\");\n                if (importHandler.hasStopSupport()) {\n                    content.append(\"<p><a href=\\\"wiki?stop_import=\" + currentImport + serviceUser + \"\\\">stop</a> (WARNING: pages may be incomplete due to missing templates)</p>\");\n                }\n            } else {\n                content.append(\"<p>Import has been stopped by the user. Return to <a href=\\\"wiki?title=\" + MAIN_PAGE + serviceUser + \"\\\">\" + MAIN_PAGE + \"</a>.</p>\");\n            }\n        } else if (!currentImport.isEmpty() && importHandler == null) {\n            content.append(\"<h2>Import of \\\"\" + currentImport + \"\\\" finished</h2>\\n\");\n            content.append(\"<p>Current log file:</p>\\n\");\n            content.append(\"<pre>\");\n            String log = importLog.toString();\n            int start = log.indexOf(\"\\n\");\n            if (start != -1) { \n                content.append(log.substring(start));\n            }\n            content.append(\"</pre>\");\n\n            String req_stop_import = request.getParameter(\"stop_import\");\n            if (req_stop_import != null && !req_stop_import.isEmpty()) {\n                synchronized (WikiServletScalaris.this) {\n                    importLog.close();\n                    WikiServletScalaris.this.currentImport = \"\";\n                }\n                response.setHeader(\"Refresh\", \"1; url = wiki?title=\" + MAIN_PAGE + serviceUser + \"\");\n                content.append(\"<p>If not re-directed automatically: Return to <a href=\\\"wiki?title=\" + MAIN_PAGE + serviceUser + \"\\\">\" + MAIN_PAGE + \"</a></p>\\n\");\n            } else {\n                content.append(\"<p><a href=\\\"wiki?stop_import=\" + currentImport + serviceUser + \"\\\">clear log and return to Main Page</a></p>\");\n            }\n        }\n\n        page.setNotice(WikiServlet.getParam_notice(request));\n        page.setError(getParam_error(request));\n        page.setTitle(\"Import Wiki dump\");\n        page.setPage(content.toString());\n\n        forwardToPageJsp(request, response, connection, page, \"page.jsp\");\n    }\n\n    private void startImport(String dumpsPath, String req_import,\n            int maxRevisions, Calendar maxTime) throws RuntimeException {\n        currentImport = req_import;\n        importLog = new CircularByteArrayOutputStream(1024 * 1024);\n        PrintStream ps = new PrintStream(importLog);\n        ps.println(\"starting import...\");\n        String fileName = dumpsPath + File.separator + req_import;\n        if (fileName.endsWith(\".db\")) {\n            importHandler = new WikiDumpPreparedSQLiteToScalaris(fileName, Options.getInstance(), 1, 1, cPool.getConnectionFactory());\n        } else {\n            importHandler = new WikiDumpToScalarisHandler(\n                    de.zib.scalaris.examples.wikipedia.data.xml.Main.blacklist,\n                    null, maxRevisions, null, maxTime, cPool.getConnectionFactory());\n        }\n        importHandler.setMsgOut(ps);\n        this.new ImportThread(importHandler, fileName, ps).start();\n    }\n    \n    private class ImportThread extends Thread {\n        private WikiDump handler;\n        private String fileName;\n        private PrintStream ps;\n        \n        public ImportThread(WikiDump handler, String fileName, PrintStream ps) {\n            this.handler = handler;\n            this.fileName = fileName;\n            this.ps = ps;\n        }\n        /* (non-Javadoc)\n         * @see java.lang.Thread#run()\n         */\n        @Override\n        public void run() {\n            InputSource[] is = null;\n            try {\n                handler.setUp();\n                if (handler instanceof WikiDumpHandler) {\n                    WikiDumpHandler xmlHandler = (WikiDumpHandler) handler;\n                    XMLReader reader = XMLReaderFactory.createXMLReader();\n                    reader.setContentHandler(xmlHandler);\n                    is = de.zib.scalaris.examples.wikipedia.data.xml.Main.getFileReader(fileName);\n                    for (InputSource source : is) {\n                        reader.parse(source);\n                    }\n                    xmlHandler.new ReportAtShutDown().reportAtEnd();\n                    ps.println(\"import finished\");\n                } else if (handler instanceof WikiDumpPreparedSQLiteToScalaris) {\n                    WikiDumpPreparedSQLiteToScalaris sqlHandler =\n                            (WikiDumpPreparedSQLiteToScalaris) handler;\n                    sqlHandler.writeToScalaris();\n                    sqlHandler.new ReportAtShutDown().reportAtEnd();\n                }\n            } catch (Exception e) {\n                if (e instanceof SAXParsingInterruptedException) {\n                    // this is ok - we told the parser to stop\n                } else {\n                    e.printStackTrace(ps);\n                }\n            } finally {\n                handler.tearDown();\n                if (is != null) {\n                    try {\n                        for (InputSource source : is) {\n                            source.getCharacterStream().close();\n                        }\n                    } catch (IOException e) {\n                        // don't care\n                    }\n                }\n            }\n            synchronized (WikiServletScalaris.this) {\n                WikiServletScalaris.this.importHandler = null;\n                WikiServletScalaris.this.updateExistingPages();\n                if (WikiServletScalaris.this.autoImport) {\n                    WikiServletScalaris.this.currentImport = \"\";\n                }\n            }\n        }\n    }\n    \n    @Override\n    protected MyScalarisWikiModel getWikiModel(Connection connection, WikiPageBeanBase page) {\n        final MyScalarisWikiModel model = new MyScalarisWikiModel(getImagebaseurl(page),\n                getLinkbaseurl(page), connection, namespace);\n        model.setExistingPages(existingPages);\n        return model;\n    }\n\n    @Override\n    public String getSiteInfoKey() {\n        return ScalarisDataHandlerUnnormalised.getSiteInfoKey();\n    }\n\n    @Override\n    public String getPageListKey(int namespace) {\n        return ScalarisDataHandlerUnnormalised.getPageListKey(namespace);\n    }\n\n    @Override\n    public String getPageCountKey(int namespace) {\n        return ScalarisDataHandlerUnnormalised.getPageCountKey(namespace);\n    }\n\n    @Override\n    public String getArticleCountKey() {\n        return ScalarisDataHandlerUnnormalised.getArticleCountKey();\n    }\n\n    @Override\n    public String getRevKey(String title, int id, final MyNamespace nsObject) {\n        return ScalarisDataHandlerUnnormalised.getRevKey(title, id, nsObject);\n    }\n\n    @Override\n    public String getPageKey(String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerUnnormalised.getPageKey(title, nsObject);\n    }\n\n    @Override\n    public String getRevListKey(String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerUnnormalised.getRevListKey(title, nsObject);\n    }\n\n    @Override\n    public String getCatPageListKey(String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerUnnormalised.getCatPageListKey(title, nsObject);\n    }\n\n    @Override\n    public String getCatPageCountKey(String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerUnnormalised.getCatPageCountKey(title, nsObject);\n    }\n\n    @Override\n    public String getTplPageListKey(String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerUnnormalised.getTplPageListKey(title, nsObject);\n    }\n\n    @Override\n    public String getBackLinksPageListKey(String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerUnnormalised.getBackLinksPageListKey(title, nsObject);\n    }\n\n    @Override\n    public String getStatsPageEditsKey() {\n        return ScalarisDataHandlerUnnormalised.getStatsPageEditsKey();\n    }\n\n    @Override\n    public String getContributionListKey(String contributor) {\n        return ScalarisDataHandlerUnnormalised.getContributionListKey(contributor);\n    }\n\n    @Override\n    public ValueResult<String> getDbVersion(Connection connection) {\n        return ScalarisDataHandlerUnnormalised.getDbVersion(connection);\n    }\n\n    @Override\n    public PageHistoryResult getPageHistory(Connection connection, String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerUnnormalised.getPageHistory(connection, title, nsObject);\n    }\n\n    @Override\n    public RevisionResult getRevision(Connection connection, String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerUnnormalised.getRevision(connection, title, nsObject);\n    }\n\n    @Override\n    public RevisionResult getRevision(Connection connection, String title, int id, final MyNamespace nsObject) {\n        return ScalarisDataHandlerUnnormalised.getRevision(connection, title, id, nsObject);\n    }\n\n    @Override\n    public ValueResult<List<NormalisedTitle>> getPageList(Connection connection) {\n        return ScalarisDataHandlerUnnormalised.getPageList(connection);\n    }\n\n    @Override\n    public ValueResult<List<NormalisedTitle>> getPageList(int namespace, Connection connection) {\n        return ScalarisDataHandlerUnnormalised.getPageList(namespace, connection);\n    }\n\n    @Override\n    public ValueResult<List<NormalisedTitle>> getPagesInCategory(Connection connection, NormalisedTitle title) {\n        return ScalarisDataHandlerNormalised.getPagesInCategory(connection, title);\n    }\n\n    @Override\n    public ValueResult<List<NormalisedTitle>> getPagesInTemplate(Connection connection, NormalisedTitle title) {\n        return ScalarisDataHandlerNormalised.getPagesInTemplate(connection, title);\n    }\n\n    @Override\n    public ValueResult<List<NormalisedTitle>> getPagesInTemplates(Connection connection, List<NormalisedTitle> titles, String pageTitle) {\n        return ScalarisDataHandlerNormalised.getPagesInTemplates(connection, titles, pageTitle);\n    }\n\n    @Override\n    public ValueResult<List<NormalisedTitle>> getPagesLinkingTo(Connection connection, String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerUnnormalised.getPagesLinkingTo(connection, title, nsObject);\n    }\n\n    @Override\n    public ValueResult<List<Contribution>> getContributions(\n            Connection connection, String contributor) {\n        return ScalarisDataHandlerUnnormalised.getContributions(connection, contributor);\n    }\n\n    @Override\n    public ValueResult<BigInteger> getPageCount(Connection connection) {\n        return ScalarisDataHandlerUnnormalised.getPageCount(connection);\n    }\n\n    @Override\n    public ValueResult<BigInteger> getPageCount(int namespace, Connection connection) {\n        return ScalarisDataHandlerUnnormalised.getPageCount(namespace, connection);\n    }\n\n    @Override\n    public ValueResult<BigInteger> getArticleCount(Connection connection) {\n        return ScalarisDataHandlerUnnormalised.getArticleCount(connection);\n    }\n\n    @Override\n    public ValueResult<BigInteger> getPagesInCategoryCount(Connection connection, String title, final MyNamespace nsObject) {\n        return ScalarisDataHandlerUnnormalised.getPagesInCategoryCount(connection, title, nsObject);\n    }\n\n    @Override\n    public ValueResult<BigInteger> getStatsPageEdits(Connection connection) {\n        return ScalarisDataHandlerUnnormalised.getStatsPageEdits(connection);\n    }\n\n    @Override\n    public ValueResult<NormalisedTitle> getRandomArticle(Connection connection, Random random) {\n        return ScalarisDataHandlerUnnormalised.getRandomArticle(connection, random);\n    }\n\n    @Override\n    public SavePageResult savePage(Connection connection, String title,\n            Revision newRev, int prevRevId, Map<String, String> restrictions,\n            SiteInfo siteinfo, String username, final MyNamespace nsObject) {\n        return ScalarisDataHandlerUnnormalised.savePage(connection, title, newRev,\n                prevRevId, restrictions, siteinfo, username, nsObject);\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/Contribution.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data;\n\nimport java.io.Serializable;\n\n/**\n * Represents a revision of a page.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class Contribution implements Serializable {\n    /**\n     * Version for serialisation.\n     */\n    private static final long serialVersionUID = 1L;\n    \n    /**\n     * the edited revision's ID (note: 1 means a new page was created)\n     */\n    protected int id;\n\n    /**\n     * the edited revision's date of creation\n     */\n    protected String timestamp;\n    \n    /**\n     * whether the change is a minor change or not\n     */\n    protected boolean minor;\n\n    /**\n     * the revision's contributor\n     */\n    protected String pageName;\n\n    /**\n     * the comment of the revision\n     */\n    protected String comment;\n\n    /**\n     * the size of the content (text) of the revision before the edit\n     */\n    protected int sizeBefore;\n\n    /**\n     * the size of the content (text) of the revision after the edit\n     */\n    protected int sizeAfter;\n\n    /**\n     * Creates an empty short revision.\n     */\n    public Contribution() {\n        this.id = 0;\n        this.timestamp = \"\";\n        this.minor = false;\n        this.pageName = \"\";\n        this.comment = \"\";\n        this.sizeBefore = 0;\n        this.sizeAfter = 0;\n    }\n\n    /**\n     * Creates a new short revision from the given revision.\n     * \n     * @param oldPage\n     *            the page object before the edit (including the revision)\n     * @param newPage\n     *            the page object after the edit (including the revision)\n     */\n    public Contribution(Page oldPage, Page newPage) {\n        this.id = newPage.getCurRev().getId();\n        this.timestamp = newPage.getCurRev().getTimestamp();\n        this.minor = newPage.getCurRev().isMinor();\n        this.pageName = newPage.getTitle();\n        this.comment = newPage.getCurRev().getComment();\n        if (oldPage != null) {\n            this.sizeBefore = oldPage.getCurRev().unpackedText().length();\n        } else {\n            this.sizeBefore = 0;\n        }\n        this.sizeAfter = newPage.getCurRev().unpackedText().length();\n    }\n\n    /**\n     * @return the id\n     */\n    public int getId() {\n        return id;\n    }\n\n    /**\n     * @return the timestamp\n     */\n    public String getTimestamp() {\n        return timestamp;\n    }\n\n    /**\n     * @return the pageName\n     */\n    public String getPageName() {\n        return pageName;\n    }\n\n    /**\n     * @return the comment\n     */\n    public String getComment() {\n        return comment;\n    }\n\n    /**\n     * @return the sizeBefore\n     */\n    public int getSizeBefore() {\n        return sizeBefore;\n    }\n\n    /**\n     * @return the sizeAfter\n     */\n    public int getSizeAfter() {\n        return sizeAfter;\n    }\n\n    /**\n     * Gets whether the change is a minor change or not.\n     * \n     * @return <tt>true</tt> if the revision is a minor change, <tt>false</tt>\n     *         otherwise\n     */\n    public boolean isMinor() {\n        return minor;\n    }\n\n    /**\n     * @param id the id to set\n     */\n    public void setId(int id) {\n        this.id = id;\n    }\n\n    /**\n     * @param timestamp the timestamp to set\n     */\n    public void setTimestamp(String timestamp) {\n        this.timestamp = timestamp;\n    }\n\n    /**\n     * @param minor the minorChange to set\n     */\n    public void setMinor(boolean minor) {\n        this.minor = minor;\n    }\n\n    /**\n     * @param pageName the pageName to set\n     */\n    public void setPageName(String pageName) {\n        this.pageName = pageName;\n    }\n\n    /**\n     * @param comment the comment to set\n     */\n    public void setComment(String comment) {\n        this.comment = comment;\n    }\n\n    /**\n     * @param sizeBefore the sizeBefore to set\n     */\n    public void setSizeBefore(int sizeBefore) {\n        this.sizeBefore = sizeBefore;\n    }\n\n    /**\n     * @param sizeAfter the sizeAfter to set\n     */\n    public void setSizeAfter(int sizeAfter) {\n        this.sizeAfter = sizeAfter;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/Contributor.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data;\n\nimport java.io.Serializable;\n\n/**\n * Contributor known as a registered user or as an IP address.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class Contributor implements Serializable {\n    /**\n     * Version for serialisation.\n     */\n    private static final long serialVersionUID = 1L;\n    \n    /**\n     * IP address or custom name of a contributor, e.g. \"127.0.0.1\" or\n     * \"Incubator import\".\n     */\n    \n    protected String ip = \"\";\n    \n    /**\n     * User name (if known).\n     */\n    protected String user = \"\";\n    \n    /**\n     * User ID (if known).\n     */\n    protected int id = -1;\n\n    /**\n     * Creates a new (empty) contributor. Use the setters to make it a valid\n     * contributor.\n     */\n    public Contributor() {\n    }\n\n    /**\n     * Gets the IP address of the contributor.\n     * \n     * @return the ip\n     */\n    public String getIp() {\n        return ip;\n    }\n\n    /**\n     * Gets the contributor's username.\n     * \n     * @return the username\n     */\n    public String getUser() {\n        return user;\n    }\n\n    /**\n     * Gets the contributor's ID.\n     * \n     * @return the id\n     */\n    public int getId() {\n        return id;\n    }\n\n    /**\n     * Sets the IP of the contributor.\n     * \n     * @param ip the ip to set\n     */\n    public void setIp(String ip) {\n        this.ip = ip;\n    }\n\n    /**\n     * Sets the user name.\n     * \n     * @param user the user to set\n     */\n    public void setUser(String user) {\n        this.user = user;\n    }\n\n    /**\n     * Sets the user ID.\n     * \n     * @param id the id to set\n     */\n    public void setId(int id) {\n        this.id = id;\n    }\n    \n    public String toString() {\n        if (ip.isEmpty() || !user.isEmpty()) {\n            return user;\n        } else {\n            return ip;\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/Page.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data;\n\nimport java.io.Serializable;\nimport java.util.LinkedHashMap;\nimport java.util.Map;\nimport java.util.Map.Entry;\n\n/**\n * Represents a page including its revisions.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class Page implements Serializable {\n    /**\n     * Version for serialisation.\n     */\n    private static final long serialVersionUID = 1L;\n\n    /**\n     * The page's title.\n     */\n    protected String title = \"\";\n\n    /**\n     * The page's ID.\n     */\n    protected int id = -1;\n    \n    /**\n     * Whether the page's newest revision redirects or not.\n     */\n    protected boolean redirect = false;\n    \n    /**\n     * Page restrictions, e.g. for moving/editing the page.\n     */\n    protected Map<String, String> restrictions = new LinkedHashMap<String, String>();\n    \n    /**\n     * Current revision (cached).\n     */\n    protected Revision curRev = null;\n\n    /**\n     * Creates a new page with default values (this page is invalid until all of\n     * them have been set!).\n     */\n    public Page() {\n    }\n\n    /**\n     * Creates a new page with the given title and ID and a single revision.\n     * \n     * @param title\n     *            the title of the page\n     * @param id\n     *            the id of the page\n     * @param redirect\n     *            whether the page's newest revision redirects or not\n     * @param restrictions\n     *            page restrictions\n     * @param curRev\n     *            current revision\n     */\n    public Page(String title, int id, boolean redirect, Map<String, String> restrictions, Revision curRev) {\n        this.title = title;\n        this.id = id;\n        this.redirect = redirect;\n        this.restrictions = restrictions;\n        this.curRev = curRev;\n    }\n\n    /**\n     * Gets the page's title.\n     * \n     * @return the title of the page\n     */\n    public String getTitle() {\n        return title;\n    }\n\n    /**\n     * Gets the page's ID.\n     * \n     * @return the id\n     */\n    public int getId() {\n        return id;\n    }\n\n    /**\n     * Gets whether the page's newest revision redirects or not.\n     * \n     * @return <tt>true</tt> if redirecting, otherwise <tt>false</tt>\n     */\n    public boolean isRedirect() {\n        return redirect;\n    }\n\n    /**\n     * Gets all page restrictions, e.g. for moving/editing the page.\n     * \n     * @return the restrictions\n     */\n    public Map<String, String> getRestrictions() {\n        return restrictions;\n    }\n\n    /**\n     * Gets the current revision.\n     * \n     * @return the curRev\n     */\n    public Revision getCurRev() {\n        return curRev;\n    }\n\n    /**\n     * Sets the current revision.\n     * \n     * @param curRev the curRev to set\n     */\n    public void setCurRev(Revision curRev) {\n        this.curRev = curRev;\n    }\n\n    /**\n     * Sets the title of the page.\n     * \n     * @param title the title to set\n     */\n    public void setTitle(String title) {\n        this.title = title;\n    }\n\n    /**\n     * Sets the page's ID.\n     * \n     * @param id the id to set\n     */\n    public void setId(int id) {\n        this.id = id;\n    }\n\n    /**\n     * Sets whether the page is a redirect.\n     * \n     * @param redirect the redirect to set\n     */\n    public void setRedirect(boolean redirect) {\n        this.redirect = redirect;\n    }\n\n    /**\n     * Sets page restrictions.\n     * \n     * @param restrictions the restrictions to set\n     */\n    public void setRestrictions(Map<String, String> restrictions) {\n        this.restrictions = restrictions;\n    }\n\n    /**\n     * Checks if a user is allows to edit the given page.\n     * \n     * @param username\n     *            the name of a user\n     * \n     * @return whether edit is allowed for the user or not\n     */\n    public boolean checkEditAllowed(String username) {\n        // System.out.println(result.page.getRestrictions());\n        String all = restrictions.get(\"all\");\n        if (all != null && !all.equals(username)) {\n            return false;\n        }\n        String edit = restrictions.get(\"edit\");\n        if (edit != null && !edit.equals(username)) {\n            return false;\n        }\n        return true;\n    }\n\n    /**\n     * Converts a restrictions map to its corresponding string.\n     * \n     * @param restrictions\n     *            a map of restrictions\n     * \n     * @return a string of the form <tt>key1=value1,key2=value2</tt>\n     */\n    public static String restrictionsToString(Map<String, String> restrictions) {\n        String restrictionsStr;\n        if (restrictions.isEmpty()) {\n            restrictionsStr = \"\";\n        } else {\n            StringBuilder sb = new StringBuilder();\n            for (Entry<String, String> restr : restrictions.entrySet()) {\n                sb.append(restr.getKey());\n                sb.append('=');\n                sb.append(restr.getValue());\n                sb.append(',');\n            }\n            restrictionsStr = sb.substring(0, sb.length() - 1);\n        }\n        return restrictionsStr;\n    }\n\n    /**\n     * Converts a restrictions string to its corresponding map.\n     * \n     * @param restrictionsStr\n     *            a string of the form <tt>key1=value1,key2=value2</tt>\n     * \n     * @return a map of restrictions\n     */\n    public static Map<String, String> restrictionsFromString(String restrictionsStr) {\n        return restrictionsFromString(restrictionsStr, \":\");\n    }\n\n    /**\n     * Converts a restrictions string to its corresponding map.\n     * \n     * @param restrictionsStr\n     *            a string of the form\n     *            <tt>key1=value1&lt;SEPARATOR&gt;key2=value2</tt>\n     * @param separator\n     *            the separator between key value pairs\n     * \n     * @return a map of restrictions\n     */\n    public static Map<String, String> restrictionsFromString(\n            String restrictionsStr, final String separator) {\n        LinkedHashMap<String, String> restrictions_map = new LinkedHashMap<String, String>();\n        if (!restrictionsStr.isEmpty()) {\n            String[] restrictions_array = restrictionsStr.split(separator);\n            for (int i = 0; i < restrictions_array.length; ++i) {\n                String[] restriction = restrictions_array[i].split(\"=\");\n                if (restriction.length == 2) {\n                    restrictions_map.put(restriction[0], restriction[1]);\n                } else if (restriction.length == 1) {\n                    restrictions_map.put(\"all\", restriction[0]);\n                } else {\n                    System.err.println(\"Unknown restriction: \" + restrictions_array[i]);\n                }\n            }\n        }\n        return restrictions_map;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/Revision.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data;\n\nimport java.io.ByteArrayInputStream;\nimport java.io.ByteArrayOutputStream;\nimport java.io.IOException;\nimport java.io.Serializable;\nimport java.util.Calendar;\nimport java.util.zip.GZIPInputStream;\nimport java.util.zip.GZIPOutputStream;\n\nimport org.apache.commons.codec.binary.Base64;\n\n/**\n * Represents a revision of a page.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class Revision implements Serializable {\n    /**\n     * Version for serialisation.\n     */\n    private static final long serialVersionUID = 1L;\n\n    /**\n     * the revision's ID\n     */\n    protected int id = 0;\n\n    /**\n     * the revision's date of creation\n     */\n    protected String timestamp = \"\";\n\n    /**\n     * whether the change is a minor change or not\n     */\n    protected boolean minor = false;\n\n    /**\n     * the revision's contributor\n     */\n    protected Contributor contributor = new Contributor();\n\n    /**\n     * the comment of the revision\n     */\n    protected String comment = \"\";\n\n    /**\n     * the content (text) of the revision (compressed)\n     */\n    protected byte[] pText = packText(\"\");\n\n    /**\n     * Creates a new revision with invalid data. Use the setters to make it a\n     * valid revision.\n     */\n    public Revision() {\n    }\n\n    /**\n     * Creates a new revision with the given data.\n     * \n     * @param id\n     *            the id of the revision\n     * @param timestamp\n     *            the time the revision was created\n     * @param minorChange\n     *            whether the change is a minor change or not\n     * @param contributor\n     *            the contributor of the revision\n     * @param comment\n     *            a comment entered when the revision was created\n     */\n    public Revision(int id, String timestamp, boolean minorChange,\n            Contributor contributor, String comment) {\n        this.id = id;\n        this.timestamp = timestamp;\n        this.minor = minorChange;\n        this.contributor = contributor;\n        this.comment = comment;\n    }\n\n    /**\n     * @return the id\n     */\n    public int getId() {\n        return id;\n    }\n\n    /**\n     * @return the timestamp\n     */\n    public String getTimestamp() {\n        return timestamp;\n    }\n\n    /**\n     * @return the contributor\n     */\n    public Contributor getContributor() {\n        return contributor;\n    }\n\n    /**\n     * @return the comment\n     */\n    public String getComment() {\n        return comment;\n    }\n\n    /**\n     * Gets the compressed revision text.\n     * \n     * @return the text\n     */\n    public String getB64pText() {\n        Base64 b64 = new Base64(0);\n        return b64.encodeToString(pText);\n    }\n\n    /**\n     * Gets the compressed revision text.\n     * \n     * @return the text\n     */\n    public byte[] packedText() {\n        return pText;\n    }\n\n    /**\n     * Gets the un-compressed revision text.\n     * \n     * @return the text\n     */\n    public String unpackedText() {\n        return unpackText(pText);\n    }\n\n    /**\n     * De-compresses the given text and returns it as a string.\n     * \n     * @param text\n     *            the compressed text\n     * \n     * @return the de-compressed text\n     * \n     * @throws RuntimeException\n     *             if de-compressing the text did not work\n     */\n    protected static String unpackText(byte[] text) throws RuntimeException {\n        try {\n            ByteArrayOutputStream unpacked = new ByteArrayOutputStream();\n            ByteArrayInputStream bis = new ByteArrayInputStream(text);\n            GZIPInputStream gis = new GZIPInputStream(bis);\n            byte[] bbuf = new byte[256];\n            int read = 0;\n            while ((read = gis.read(bbuf)) >= 0) {\n                unpacked.write(bbuf, 0, read);\n            }\n            gis.close();\n            return new String(unpacked.toString(\"UTF-8\"));\n        } catch (IOException e) {\n            throw new RuntimeException(e);\n        }\n    }\n\n    /**\n     * Compresses the given text and returns it as a byte array.\n     * \n     * @param text\n     *            the un-compressed text\n     * \n     * @return the compressed text\n     * \n     * @throws RuntimeException\n     *             if compressing the text did not work\n     */\n    protected static byte[] packText(String text) throws RuntimeException {\n        try {\n            ByteArrayOutputStream bos = new ByteArrayOutputStream();\n            GZIPOutputStream gos = new GZIPOutputStream(bos);\n            gos.write(text.getBytes(\"UTF-8\"));\n            gos.flush();\n            gos.close();\n            return bos.toByteArray();\n        } catch (IOException e) {\n            throw new RuntimeException(e);\n        }\n    }\n\n    /**\n     * Gets whether the change is a minor change or not.\n     * \n     * @return <tt>true</tt> if the revision is a minor change, <tt>false</tt>\n     *         otherwise\n     */\n    public boolean isMinor() {\n        return minor;\n    }\n\n    /**\n     * @param id\n     *            the id to set\n     */\n    public void setId(int id) {\n        this.id = id;\n    }\n\n    /**\n     * @param timestamp\n     *            the timestamp to set\n     */\n    public void setTimestamp(String timestamp) {\n        this.timestamp = timestamp;\n    }\n\n    /**\n     * @param minor\n     *            the minorChange to set\n     */\n    public void setMinor(boolean minor) {\n        this.minor = minor;\n    }\n\n    /**\n     * @param contributor\n     *            the contributor to set\n     */\n    public void setContributor(Contributor contributor) {\n        this.contributor = contributor;\n    }\n\n    /**\n     * @param comment\n     *            the comment to set\n     */\n    public void setComment(String comment) {\n        this.comment = comment;\n    }\n\n    /**\n     * @param text\n     *            the base64-encoded packed text to set\n     */\n    public void setB64pText(String text) {\n        Base64 b64 = new Base64(0);\n        this.pText = b64.decode(text);\n    }\n\n    /**\n     * @param text\n     *            the (packed) text to set\n     */\n    public void setPackedText(byte[] text) {\n        this.pText = text;\n    }\n\n    /**\n     * @param text\n     *            the (unpacked) text to set\n     */\n    public void setUnpackedText(String text) {\n        this.pText = packText(text);\n    }\n\n    /**\n     * Converts a timestamp in ISO8601 format to a {@link Calendar} object.\n     * \n     * @param timestamp\n     *            the timestamp to convert\n     * \n     * @return a {@link Calendar} with the same date\n     */\n    public static Calendar stringToCalendar(String timestamp) {\n        return javax.xml.bind.DatatypeConverter.parseDateTime(timestamp\n                .toString());\n    }\n\n    /**\n     * Converts a {@link Calendar} object to a timestamp in ISO8601 format.\n     * \n     * @param calendar\n     *            the calendar to convert\n     * \n     * @return a timestamp string with the same date\n     */\n    public static String calendarToString(Calendar calendar) {\n        return javax.xml.bind.DatatypeConverter.printDateTime(calendar);\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/ShortRevision.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data;\n\nimport java.io.Serializable;\nimport java.util.LinkedList;\nimport java.util.List;\n\n/**\n * Represents a revision of a page.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class ShortRevision implements Serializable {\n    /**\n     * Version for serialisation.\n     */\n    private static final long serialVersionUID = 1L;\n    \n    /**\n     * the revision's ID\n     */\n    protected int id;\n\n    /**\n     * the revision's date of creation\n     */\n    protected String timestamp;\n    \n    /**\n     * whether the change is a minor change or not\n     */\n    protected boolean minor;\n\n    /**\n     * the revision's contributor\n     */\n    protected Contributor contributor;\n\n    /**\n     * the comment of the revision\n     */\n    protected String comment;\n\n    /**\n     * the size of the content (text) of the revision\n     */\n    protected int size;\n\n    /**\n     * Creates an empty short revision.\n     */\n    public ShortRevision() {\n        this.id = 0;\n        this.timestamp = \"\";\n        this.minor = false;\n        this.contributor = new Contributor();\n        this.comment = \"\";\n        this.size = 0;\n    }\n\n    /**\n     * Creates a new short revision from the given revision.\n     * \n     * @param revision\n     *            the revision to describe\n     */\n    public ShortRevision(Revision revision) {\n        this.id = revision.getId();\n        this.timestamp = revision.getTimestamp();\n        this.minor = revision.isMinor();\n        this.contributor = revision.getContributor();\n        this.comment = revision.getComment();\n        this.size = revision.unpackedText().length();\n    }\n\n    /**\n     * @return the id\n     */\n    public int getId() {\n        return id;\n    }\n\n    /**\n     * @return the timestamp\n     */\n    public String getTimestamp() {\n        return timestamp;\n    }\n\n    /**\n     * @return the contributor\n     */\n    public Contributor getContributor() {\n        return contributor;\n    }\n\n    /**\n     * @return the comment\n     */\n    public String getComment() {\n        return comment;\n    }\n\n    /**\n     * @return the size of the revision's text\n     */\n    public int getSize() {\n        return size;\n    }\n\n    /**\n     * Gets whether the change is a minor change or not.\n     * \n     * @return <tt>true</tt> if the revision is a minor change, <tt>false</tt>\n     *         otherwise\n     */\n    public boolean isMinor() {\n        return minor;\n    }\n\n    /**\n     * @param id the id to set\n     */\n    public void setId(int id) {\n        this.id = id;\n    }\n\n    /**\n     * @param timestamp the timestamp to set\n     */\n    public void setTimestamp(String timestamp) {\n        this.timestamp = timestamp;\n    }\n\n    /**\n     * @param minor the minorChange to set\n     */\n    public void setMinor(boolean minor) {\n        this.minor = minor;\n    }\n\n    /**\n     * @param contributor the contributor to set\n     */\n    public void setContributor(Contributor contributor) {\n        this.contributor = contributor;\n    }\n\n    /**\n     * @param comment the comment to set\n     */\n    public void setComment(String comment) {\n        this.comment = comment;\n    }\n\n    /**\n     * @param size the size to set\n     */\n    public void setSize(int size) {\n        this.size = size;\n    }\n    \n    /**\n     * Converts a list of {@link Revision} objects into a list of\n     * {@link ShortRevision} objects.\n     * \n     * @param revisions\n     *            the revision list to convert\n     * \n     * @return a short revision list\n     */\n    public static List<ShortRevision> fromRevisions(final List<Revision> revisions) {\n        List<ShortRevision> result = new LinkedList<ShortRevision>();\n        for (Revision rev : revisions) {\n            result.add(new ShortRevision(rev));\n        }\n        return result;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/SiteInfo.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data;\n\nimport java.io.Serializable;\nimport java.util.HashMap;\nimport java.util.Locale;\nimport java.util.Map;\nimport java.util.regex.Matcher;\nimport java.util.regex.Pattern;\n\n/**\n * Represents generic site information.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class SiteInfo implements Serializable {\n    /**\n     * Version for serialisation.\n     */\n    private static final long serialVersionUID = 1L;\n    \n    protected String base;\n    protected String sitename;\n    protected String generator;\n    protected String caseStr;\n    /**\n     * Maps namespace keys to a map with the following two entries:\n     * <ul>\n     * <li><tt>{@link #NAMESPACE_PREFIX}</tt>: prefix of the namespace</li>\n     * <li><tt>{@link #NAMESPACE_CASE}</tt>: case of the namespace, e.g. \"first-letter\"</li>\n     * </ul>\n     */\n    protected Map<String, Map<String, String>> namespaces;\n    \n    /**\n     * Key for getting the namespace prefix in the maps contained in\n     * {@link #namespaces}.\n     * \n     * @see #getNamespaces()\n     */\n    public final static String NAMESPACE_PREFIX = \"prefix\";\n\n    /**\n     * Key for getting the namespace case in the maps contained in\n     * {@link #namespaces}.\n     * \n     * @see #getNamespaces()\n     */\n    public final static String NAMESPACE_CASE = \"case\";\n    \n    protected static final Pattern MATCH_WIKI_SITE_LANG = Pattern.compile(\"^http[s]?://([^.]+).*$\");\n\n    /**\n     * Creates a site info object with the given data.\n     */\n    public SiteInfo() {\n        this.base = \"\";\n        this.sitename = \"\";\n        this.generator = \"\";\n        this.caseStr = \"\";\n        this.namespaces = new HashMap<String, Map<String, String>>();\n    }\n\n    /**\n     * Creates a site info object with the given data.\n     * \n     * @param base\n     *            the url of the main site\n     * @param sitename\n     *            the name of the site\n     * @param generator\n     *            the generator of the site (MediaWiki version string)\n     * @param caseStr\n     *            the case option of the site\n     * @param namespaces\n     *            the namespaces of the site\n     */\n    public SiteInfo(String base, String sitename, String generator, String caseStr, Map<String, Map<String, String>> namespaces) {\n        this.base = base;\n        this.sitename = sitename;\n        this.generator = generator;\n        this.caseStr = caseStr;\n        this.namespaces = namespaces;\n    }\n\n    /**\n     * Gets the base URL of the site.\n     * \n     * @return the base URL\n     */\n    public String getBase() {\n        return base;\n    }\n\n    /**\n     * Sets the base URL of the site.\n     * \n     * @param base the base URL to set\n     */\n    public void setBase(String base) {\n        this.base = base;\n    }\n    \n    /**\n     * Extract the language string from {@link #base}. Assumes <tt>en</tt> if no\n     * match is found.\n     * \n     * @return Wikipedia language code\n     * @see #getBase()\n     */\n    public String extractLang() {\n        String lang = \"en\";\n        Matcher matcher = MATCH_WIKI_SITE_LANG.matcher(base);\n        if (matcher.matches()) {\n            lang = matcher.group(1);\n        }\n        return lang;\n    }\n    \n    /**\n     * Extract the locale from the language string from {@link #base}.\n     * \n     * @return locale or <tt>null</tt> if the found language is no valid language code\n     * @see #extractLang()\n     */\n    public Locale extractLolace() {\n        String lang = extractLang();\n        String country = \"\";\n        int idx = lang.indexOf('_'); // e.g. de_DE\n        if (idx >= 0) {\n            lang = lang.substring(0, idx);\n            country = lang.substring(idx + 1);\n        }\n        try {\n            Locale locale = new Locale(lang, country);\n            // test that the locale is working:\n            locale.getLanguage();\n            locale.getCountry();\n            return locale;\n        } catch (Exception e) {\n        }\n        return null;\n    }\n\n    /**\n     * Gets the site's name.\n     * \n     * @return the sitename\n     */\n    public String getSitename() {\n        return sitename;\n    }\n\n    /**\n     * Sets the site's name.\n     * \n     * @param sitename the sitename to set\n     */\n    public void setSitename(String sitename) {\n        this.sitename = sitename;\n    }\n\n    /**\n     * Gets the site's generator (MediaWiki version string).\n     * \n     * @return the generator\n     */\n    public String getGenerator() {\n        return generator;\n    }\n\n    /**\n     * Sets the site's generator (MediaWiki version string).\n     * \n     * @param generator the generator to set\n     */\n    public void setGenerator(String generator) {\n        this.generator = generator;\n    }\n\n    /**\n     * Gets the namespace mapping.\n     * \n     * Maps namespace keys to a map with the following two entries:\n     * <ul>\n     * <li><tt>{@link #NAMESPACE_PREFIX}</tt>: prefix of the namespace</li>\n     * <li><tt>{@link #NAMESPACE_CASE}</tt>: case of the namespace, e.g. \"first-letter\"</li>\n     * </ul>\n     * \n     * @return the namespace\n     */\n    public Map<String, Map<String, String>> getNamespaces() {\n        return namespaces;\n    }\n\n    /**\n     * Sets the namespace mapping.\n     * \n     * @param namespaces the namespace to set\n     */\n    public void setNamespaces(Map<String, Map<String, String>> namespaces) {\n        this.namespaces = namespaces;\n    }\n\n    /**\n     * Gets the case option of the site.\n     * \n     * @return the case\n     */\n    public String getCase() {\n        return caseStr;\n    }\n\n    /**\n     * Sets the case option of the site.\n     * \n     * @param caseStr the case to set\n     */\n    public void setCase(String caseStr) {\n        this.caseStr = caseStr;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/Main.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport java.io.BufferedReader;\nimport java.io.File;\nimport java.io.FileInputStream;\nimport java.io.FileNotFoundException;\nimport java.io.FileReader;\nimport java.io.FileWriter;\nimport java.io.IOException;\nimport java.io.InputStream;\nimport java.io.InputStreamReader;\nimport java.io.PrintWriter;\nimport java.util.Arrays;\nimport java.util.Calendar;\nimport java.util.Collection;\nimport java.util.HashMap;\nimport java.util.HashSet;\nimport java.util.LinkedList;\nimport java.util.Map;\nimport java.util.Set;\nimport java.util.SortedSet;\n\nimport java.util.zip.GZIPInputStream;\nimport org.apache.commons.compress.compressors.bzip2.BZip2CompressorInputStream;\nimport de.zib.scalaris.examples.wikipedia.data.xml.util.SevenZInputStream;\n\nimport org.xml.sax.InputSource;\nimport org.xml.sax.SAXException;\nimport org.xml.sax.XMLReader;\nimport org.xml.sax.helpers.XMLReaderFactory;\n\nimport de.zib.scalaris.examples.wikipedia.Options;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\nimport de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpHandler.ReportAtShutDown;\n\n/**\n * Provides abilities to read an xml wiki dump file and write Wiki pages to\n * Scalaris.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class Main {\n    /**\n     * Default blacklist - pages with these names are not imported\n     */\n    public final static Set<String> blacklist = new HashSet<String>();\n    \n    private static enum ImportType {\n        IMPORT_XML,\n        IMPORT_DB,\n        PREPARE_DB,\n        XML_2_DB\n    }\n    \n    /**\n     * The main function of the application. Some articles are blacklisted and\n     * will not be processed (see implementation for a list of them).\n     * \n     * @param args\n     *            first argument should be the xml file to convert.\n     */\n    public static void main(String[] args) {\n        try {\n            String filename = args[0];\n            \n            if (args.length > 1) {\n                if (args[1].equals(\"filter\")) {\n                    doFilter(filename, Arrays.copyOfRange(args, 2, args.length));\n                } else if (args[1].equals(\"import-xml\")) {\n                    doImport(filename, Arrays.copyOfRange(args, 2, args.length), ImportType.IMPORT_XML);\n                } else if (args[1].equals(\"import-db\")) {\n                    doImport(filename, Arrays.copyOfRange(args, 2, args.length), ImportType.IMPORT_DB);\n                } else if (args[1].equals(\"prepare\")) {\n                    doImport(filename, Arrays.copyOfRange(args, 2, args.length), ImportType.PREPARE_DB);\n                } else if (args[1].equals(\"xml2db\")) {\n                    doImport(filename, Arrays.copyOfRange(args, 2, args.length), ImportType.XML_2_DB);\n                } else if (args[1].equals(\"convert\")) {\n                    doConvert(filename, Arrays.copyOfRange(args, 2, args.length));\n                } else if (args[1].equals(\"dumpdb-addlinks\")) {\n                    doDumpdbAddlinks(filename, Arrays.copyOfRange(args, 2, args.length));\n                } else if (args[1].equals(\"dumpdb-filter\")) {\n                    doDumpdbFilter(filename, Arrays.copyOfRange(args, 2, args.length));\n                }\n            }\n        } catch (SAXException e) {\n            System.err.println(e.getMessage());\n            System.exit(-1);\n        } catch (FileNotFoundException e) {\n            e.printStackTrace();\n            System.exit(-1);\n        } catch (IOException e) {\n            e.printStackTrace();\n            System.exit(-1);\n        }\n    }\n\n    /**\n     * Imports all pages in the Wikipedia XML dump from the given file to Scalaris.\n     * \n     * @param filename\n     * @param args\n     * @param type\n     * \n     * @throws RuntimeException\n     * @throws IOException\n     * @throws SAXException\n     * @throws FileNotFoundException\n     */\n    private static void doImport(String filename, String[] args, ImportType type) throws RuntimeException, IOException,\n            SAXException, FileNotFoundException {\n        int i = 0;\n        \n        if (type == ImportType.IMPORT_DB) {\n            int numberOfImporters = 1;\n            if (args.length > i) {\n                try {\n                    numberOfImporters = Integer.parseInt(args[i]);\n                } catch (NumberFormatException e) {\n                    System.err.println(\"no number: \" + args[i]);\n                    System.exit(-1);\n                }\n            }\n            ++i;\n            \n            int myNumber = 1;\n            if (args.length > i) {\n                try {\n                    myNumber = Integer.parseInt(args[i]);\n                } catch (NumberFormatException e) {\n                    System.err.println(\"no number: \" + args[i]);\n                    System.exit(-1);\n                }\n            }\n            ++i;\n            \n            Options options = null;\n            if (args.length > i) {\n                if (args[i].length() > 0) {\n                    options = Options.getInstance();\n                    Options.parseOptions(options, args[i]);\n                }\n            }\n            ++i;\n            \n            if (filename.endsWith(\".db\") && numberOfImporters > 0 && myNumber > 0) {\n                WikiDumpHandler.println(System.out, \"wiki import from \" + filename);\n                WikiDumpHandler.println(System.out, \" importers   : \" + numberOfImporters);\n                WikiDumpHandler.println(System.out, \" my import nr: \" + myNumber);\n                WikiDumpPreparedSQLiteToScalaris handler =\n                        new WikiDumpPreparedSQLiteToScalaris(filename, options, numberOfImporters, myNumber);\n                handler.setUp();\n                WikiDumpPreparedSQLiteToScalaris.ReportAtShutDown shutdownHook = handler.new ReportAtShutDown();\n                Runtime.getRuntime().addShutdownHook(shutdownHook);\n                handler.writeToScalaris();\n                handler.tearDown();\n                shutdownHook.reportAtEnd();\n                Runtime.getRuntime().removeShutdownHook(shutdownHook);\n                exitCheckHandler(handler);\n            } else {\n                System.err.println(\"incorrect command line parameters\");\n                System.exit(-1);\n            }\n            return;\n        }\n        \n        int maxRevisions = -1;\n        if (args.length > i) {\n            try {\n                maxRevisions = Integer.parseInt(args[i]);\n            } catch (NumberFormatException e) {\n                System.err.println(\"no number: \" + args[i]);\n                System.exit(-1);\n            }\n        }\n        ++i;\n        \n        // a timestamp in ISO8601 format\n        Calendar minTime = null;\n        if (args.length > i && !args[i].isEmpty()) {\n            try {\n                minTime = Revision.stringToCalendar(args[i]);\n            } catch (IllegalArgumentException e) {\n                System.err.println(\"no date in ISO8601: \" + args[i]);\n                System.exit(-1);\n            }\n        }\n        ++i;\n        \n        // a timestamp in ISO8601 format\n        Calendar maxTime = null;\n        if (args.length > i && !args[i].isEmpty()) {\n            try {\n                maxTime = Revision.stringToCalendar(args[i]);\n            } catch (IllegalArgumentException e) {\n                System.err.println(\"no date in ISO8601: \" + args[i]);\n                System.exit(-1);\n            }\n        }\n        ++i;\n        \n        Set<String> whitelist = null;\n        String whitelistFile = \"\";\n        if (args.length > i && !args[i].isEmpty()) {\n            whitelistFile = args[i];\n            whitelist = new HashSet<String>();\n            addFromFile(whitelist, whitelistFile);\n            if (whitelist.isEmpty()) {\n                whitelist = null;\n            }\n        }\n        ++i;\n\n        if (type == ImportType.PREPARE_DB || type == ImportType.XML_2_DB) {\n            // only prepare the import to Scalaris, i.e. pre-process K/V pairs?\n            String dbFileName = \"\";\n            if (args.length > i && !args[i].isEmpty()) {\n                dbFileName = args[i];\n            } else {\n                System.err.println(\"need a DB file name for prepare; arguments given: \" + Arrays.toString(args));\n                System.exit(-1);\n            }\n            ++i;\n            WikiDumpHandler.println(System.out, \"wiki prepare file \" + dbFileName);\n            WikiDumpHandler.println(System.out, \" wiki dump     : \" + filename);\n            WikiDumpHandler.println(System.out, \" white list    : \" + whitelistFile);\n            WikiDumpHandler.println(System.out, \" max revisions : \" + maxRevisions);\n            WikiDumpHandler.println(System.out, \" min time      : \" + (minTime == null ? \"null\" : Revision.calendarToString(minTime)));\n            WikiDumpHandler.println(System.out, \" max time      : \" + (maxTime == null ? \"null\" : Revision.calendarToString(maxTime)));\n            WikiDumpHandler handler = null;\n            switch (type) {\n                case PREPARE_DB:\n                    handler = new WikiDumpPrepareSQLiteForScalarisHandler(\n                            blacklist, whitelist, maxRevisions, minTime,\n                            maxTime, dbFileName);\n                    break;\n                case XML_2_DB:\n                    handler = new WikiDumpXml2SQLite(blacklist, whitelist,\n                            maxRevisions, minTime, maxTime, dbFileName);\n                    break;\n                default:\n                    throw new RuntimeException();\n            }\n            runXmlHandler(handler, getFileReader(filename));\n        } else if (type == ImportType.IMPORT_XML) {\n            WikiDumpHandler.println(System.out, \"wiki import from \" + filename);\n            WikiDumpHandler.println(System.out, \" white list    : \" + whitelistFile);\n            WikiDumpHandler.println(System.out, \" max revisions : \" + maxRevisions);\n            WikiDumpHandler.println(System.out, \" min time      : \" + (minTime == null ? \"null\" : Revision.calendarToString(minTime)));\n            WikiDumpHandler.println(System.out, \" max time      : \" + (maxTime == null ? \"null\" : Revision.calendarToString(maxTime)));\n            WikiDumpHandler handler = new WikiDumpToScalarisHandler(\n                    blacklist, whitelist, maxRevisions, minTime, maxTime);\n            runXmlHandler(handler, getFileReader(filename));\n        }\n    }\n\n    /**\n     * Exits from the VM if the handler had an error in the previous import\n     * step.\n     * \n     * @param handler the handler used for the import\n     */\n    protected static void exitCheckHandler(WikiDump handler) {\n        if (handler.isErrorDuringImport()) {\n            System.exit(-1);\n        }\n    }\n\n    /**\n     * Converts the default prepared SQLite dump to a different optimisation scheme.\n     * \n     * @param dbReadFileName\n     * @param args \n     * \n     * @throws RuntimeException\n     * @throws IOException\n     * @throws SAXException\n     * @throws FileNotFoundException\n     */\n    private static void doConvert(String dbReadFileName, String[] args)\n            throws RuntimeException, FileNotFoundException {\n        \n        int i = 0;\n        String dbWriteFileName;\n        if (args.length > i) {\n            dbWriteFileName = args[i];\n        } else {\n            System.err.println(\"need an new DB file name for convert; arguments given: \" + Arrays.toString(args));\n            System.exit(-1);\n            return;\n        }\n        ++i;\n        \n        String dbWriteOptionsStr;\n        Options dbWriteOptions = new Options();\n        if (args.length > i) {\n            dbWriteOptionsStr = args[i];\n            Options.parseOptions(dbWriteOptions, null, null, null, null, null, null, null, null, dbWriteOptionsStr, null, null);\n        } else {\n            System.err.println(\"need a new optimisation scheme for convert; arguments given: \" + Arrays.toString(args));\n            System.exit(-1);\n            return;\n        }\n        ++i;\n\n        WikiDumpHandler.println(System.out, \"converting\");\n        WikiDumpHandler.println(System.out, \" from    : \" + dbReadFileName);\n        WikiDumpHandler.println(System.out, \" to      : \" + dbWriteFileName);\n        WikiDumpHandler.println(System.out, \" options : \" + dbWriteOptionsStr);\n        \n        WikiDumpConvertPreparedSQLite handler = new WikiDumpConvertPreparedSQLite(dbReadFileName, dbWriteFileName, dbWriteOptions);\n        handler.setUp();\n        WikiDumpConvertPreparedSQLite.ReportAtShutDown shutdownHook = handler.new ReportAtShutDown();\n        Runtime.getRuntime().addShutdownHook(shutdownHook);\n        handler.convertObjects();\n        handler.tearDown();\n        shutdownHook.reportAtEnd();\n        Runtime.getRuntime().removeShutdownHook(shutdownHook);\n        exitCheckHandler(handler);\n    }\n\n    /**\n     * processed a SQLite DB created from xml2db and parses all links, i.e.\n     * interconnections between pages of the wiki.\n     * \n     * @param dbReadFileName\n     * @param args \n     * \n     * @throws RuntimeException\n     * @throws IOException\n     * @throws SAXException\n     * @throws FileNotFoundException\n     */\n    private static void doDumpdbAddlinks(String dbReadFileName, String[] args)\n            throws RuntimeException {\n        WikiDumpHandler.println(System.out, \"adding links to: \" + dbReadFileName);\n        \n        WikiDumpSQLiteLinkTables handler = new WikiDumpSQLiteLinkTables(dbReadFileName);\n        handler.setUp();\n        WikiDumpSQLiteLinkTables.ReportAtShutDown shutdownHook = handler.new ReportAtShutDown();\n        Runtime.getRuntime().addShutdownHook(shutdownHook);\n        handler.processLinks();\n        handler.tearDown();\n        shutdownHook.reportAtEnd();\n        Runtime.getRuntime().removeShutdownHook(shutdownHook);\n        exitCheckHandler(handler);\n    }\n\n    /**\n     * Filters all pages in the Wikipedia XML2DB dump from the given file and\n     * creates a list of page names belonging to certain categories.\n     * \n     * @param filename\n     * @param args\n     * \n     * @throws RuntimeException\n     * @throws IOException\n     * @throws SAXException\n     * @throws FileNotFoundException\n     */\n    private static void doDumpdbFilter(String filename, String[] args) throws RuntimeException, IOException,\n            SAXException, FileNotFoundException {\n        int i = 0;\n        int recursionLvl = 1;\n        if (args.length > i) {\n            try {\n                recursionLvl = Integer.parseInt(args[i]);\n            } catch (NumberFormatException e) {\n                System.err.println(\"no number: \" + args[i]);\n                System.exit(-1);\n            }\n        }\n        ++i;\n\n        String pageListFileName = \"\";\n        if (args.length > i && !args[i].isEmpty()) {\n            pageListFileName = args[i];\n        } else {\n            System.err.println(\"need a pagelist file name for filter; arguments given: \" + Arrays.toString(args));\n            System.exit(-1);\n        }\n        ++i;\n        \n        Set<String> allowedPages0 = new HashSet<String>();\n        allowedPages0.add(\"Main Page\");\n        String allowedPagesFileName = \"\";\n        if (args.length > i && !args[i].isEmpty()) {\n            allowedPagesFileName = args[i];\n            addFromFile(allowedPages0, allowedPagesFileName);\n        }\n        ++i;\n        \n        LinkedList<String> rootCategories = new LinkedList<String>();\n        if (args.length > i) {\n            for (String rCat : Arrays.asList(args).subList(i, args.length)) {\n                if (!rCat.isEmpty()) {\n                    rootCategories.add(rCat);\n                }\n            }\n        }\n        WikiDumpHandler.println(System.out, \"filtering by categories \" + rootCategories.toString() + \" ...\");\n        WikiDumpHandler.println(System.out, \" wiki dump     : \" + filename);\n        WikiDumpHandler.println(System.out, \" allowed pages : \" + allowedPagesFileName);\n        WikiDumpHandler.println(System.out, \" recursion lvl : \" + recursionLvl);\n        \n        WikiDumpHandler.println(System.out, \"creating list of pages to import (recursion level: \" + recursionLvl + \") ...\");\n        Set<String> allowedCats0 = new HashSet<String>(rootCategories);\n        \n        WikiDumpSQLiteLinkTables handler = new WikiDumpSQLiteLinkTables(filename);\n        handler.setUp();\n        SortedSet<String> pages = handler.getPagesInCategories(allowedCats0,\n                allowedPages0, recursionLvl, false);\n        handler.tearDown();\n\n        do {\n            FileWriter outFile = new FileWriter(pageListFileName);\n            PrintWriter out = new PrintWriter(outFile);\n            for (String page : pages) {\n                out.println(page);\n            }\n            out.close();\n        } while(false);\n        exitCheckHandler(handler);\n    }\n\n    /**\n     * @param handler\n     * @param file\n     * @throws SAXException\n     * @throws IOException\n     */\n    private static void runXmlHandler(WikiDumpHandler handler, InputSource[] files)\n            throws SAXException, IOException {\n        XMLReader reader = XMLReaderFactory.createXMLReader();\n        handler.setUp();\n        ReportAtShutDown shutdownHook = handler.new ReportAtShutDown();\n        Runtime.getRuntime().addShutdownHook(shutdownHook);\n        reader.setContentHandler(handler);\n        for (InputSource file : files) {\n            reader.parse(file);\n        }\n        handler.tearDown();\n        shutdownHook.reportAtEnd();\n        Runtime.getRuntime().removeShutdownHook(shutdownHook);\n        exitCheckHandler(handler);\n    }\n\n    /**\n     * Filters all pages in the Wikipedia XML dump from the given file and\n     * creates a list of page names belonging to certain categories.\n     * \n     * @param filename\n     * @param args\n     * \n     * @throws RuntimeException\n     * @throws IOException\n     * @throws SAXException\n     * @throws FileNotFoundException\n     */\n    private static void doFilter(String filename, String[] args) throws RuntimeException, IOException,\n            SAXException, FileNotFoundException {\n        int i = 0;\n        int recursionLvl = 1;\n        if (args.length > i) {\n            try {\n                recursionLvl = Integer.parseInt(args[i]);\n            } catch (NumberFormatException e) {\n                System.err.println(\"no number: \" + args[i]);\n                System.exit(-1);\n            }\n        }\n        ++i;\n        \n        // a timestamp in ISO8601 format\n        Calendar maxTime = null;\n        if (args.length > i && !args[i].isEmpty()) {\n            try {\n                maxTime = Revision.stringToCalendar(args[i]);\n            } catch (IllegalArgumentException e) {\n                System.err.println(\"no date in ISO8601: \" + args[i]);\n                System.exit(-1);\n            }\n        }\n        ++i;\n\n        String pageListFileName = \"\";\n        if (args.length > i && !args[i].isEmpty()) {\n            pageListFileName = args[i];\n        } else {\n            System.err.println(\"need a pagelist file name for filter; arguments given: \" + Arrays.toString(args));\n            System.exit(-1);\n        }\n        ++i;\n        \n        Set<String> allowedPages = new HashSet<String>();\n        allowedPages.add(\"Main Page\");\n        String allowedPagesFileName = \"\";\n        if (args.length > i && !args[i].isEmpty()) {\n            allowedPagesFileName = args[i];\n            addFromFile(allowedPages, allowedPagesFileName);\n        }\n        ++i;\n        \n        LinkedList<String> rootCategories = new LinkedList<String>();\n        if (args.length > i) {\n            for (String rCat : Arrays.asList(args).subList(i, args.length)) {\n                if (!rCat.isEmpty()) {\n                    rootCategories.add(rCat);\n                }\n            }\n        }\n        WikiDumpHandler.println(System.out, \"filtering by categories \" + rootCategories.toString() + \" ...\");\n        WikiDumpHandler.println(System.out, \" wiki dump     : \" + filename);\n        WikiDumpHandler.println(System.out, \" max time      : \" + maxTime);\n        WikiDumpHandler.println(System.out, \" allowed pages : \" + allowedPagesFileName);\n        WikiDumpHandler.println(System.out, \" recursion lvl : \" + recursionLvl);\n        SortedSet<String> pages = getPageList(filename, maxTime, allowedPages,\n                rootCategories, recursionLvl);\n\n        do {\n            FileWriter outFile = new FileWriter(pageListFileName);\n            PrintWriter out = new PrintWriter(outFile);\n            for (String page : pages) {\n                out.println(page);\n            }\n            out.close();\n        } while(false);\n    }\n\n    private static void addFromFile(Collection<String> container,\n            String fileName) throws FileNotFoundException, IOException {\n        FileReader inFile = new FileReader(fileName);\n        BufferedReader br = new BufferedReader(inFile);\n        String line;\n        while ((line = br.readLine()) != null) {\n            if (!line.isEmpty()) {\n                container.add(line);\n            }\n        }\n        br.close();\n    }\n\n    /**\n     * Extracts all allowed pages in the given root categories as well as those\n     * pages explicitly mentioned in a list of allowed pages.\n     * \n     * Gets the category and template trees from a file, i.e.\n     * <tt>filename + \"-trees.db\"</tt>, or if this does not exist, builds the\n     * trees and stores them to this file.\n     * \n     * @param filename\n     *            the name of the xml wiki dump file\n     * @param maxTime\n     *            the maximum time of a revision to use for category parsing\n     * @param allowedPages\n     *            all allowed pages (un-normalised page titles)\n     * @param rootCategories\n     *            all allowed categories (un-normalised page titles)\n     * @param recursionLvl\n     *            recursion level to include pages linking to\n     * \n     * @return a set of (de-normalised) page titles\n     * \n     * @throws RuntimeException\n     * @throws FileNotFoundException\n     * @throws IOException\n     * @throws SAXException\n     */\n    protected static SortedSet<String> getPageList(String filename, Calendar maxTime,\n            Set<String> allowedPages, LinkedList<String> rootCategories,\n            int recursionLvl) throws RuntimeException, FileNotFoundException,\n            IOException, SAXException {\n        Map<NormalisedTitle, Set<NormalisedTitle>> templateTree = new HashMap<NormalisedTitle, Set<NormalisedTitle>>();\n        Map<NormalisedTitle, Set<NormalisedTitle>> includeTree = new HashMap<NormalisedTitle, Set<NormalisedTitle>>();\n        Map<NormalisedTitle, Set<NormalisedTitle>> referenceTree = new HashMap<NormalisedTitle, Set<NormalisedTitle>>();\n\n        File trees = new File(filename + \"-trees.db\");\n        if (trees.exists()) {\n            // read trees from tree file\n            WikiDumpHandler.println(System.out, \"reading category tree from \" + trees.getAbsolutePath() + \" ...\");\n            WikiDumpGetCategoryTreeHandler.readTrees(trees.getAbsolutePath(),\n                    templateTree, includeTree, referenceTree);\n        } else {\n            // build trees from xml file\n            // need to get all subcategories recursively, as they must be\n            // included as well\n            WikiDumpHandler.println(System.out, \"building category tree from \" + filename + \" ...\");\n            WikiDumpGetCategoryTreeHandler handler = new WikiDumpGetCategoryTreeHandler(\n                    blacklist, null, maxTime, trees.getPath());\n            runXmlHandler(handler, getFileReader(filename));\n            WikiDumpGetCategoryTreeHandler.readTrees(trees.getAbsolutePath(),\n                    templateTree, includeTree, referenceTree);\n        }\n\n        WikiDumpHandler.println(System.out, \"creating list of pages to import (recursion level: \" + recursionLvl + \") ...\");\n        Set<String> allowedCats = new HashSet<String>(rootCategories);\n\n        return WikiDumpGetCategoryTreeHandler.getPagesInCategories(\n                trees.getAbsolutePath(), allowedCats, allowedPages, recursionLvl,\n                templateTree, includeTree, referenceTree, System.out, false);\n    }\n    \n    /**\n     * Gets appropriate file reader(s) for the given file(s).\n     * \n     * @param filename\n     *            the name of the file.\n     *            multiple files are separated using a pipe\n     * \n     * @return a file reader array\n     * \n     * @throws FileNotFoundException\n     * @throws IOException\n     */\n    public static InputSource[] getFileReader(String filename) throws FileNotFoundException, IOException {\n        String[] files = filename.split(\"\\\\|\");\n        InputSource[] sources = new InputSource[files.length];\n\n        for (int i = 0; i < files.length; i++) {\n\n            String file = files[i];\n            InputStream is;\n\n            if (file.endsWith(\".xml.gz\")) {\n                is = new GZIPInputStream(new FileInputStream(file));\n            } else if (file.endsWith(\".xml.bz2\")) {\n                is = new BZip2CompressorInputStream(new FileInputStream(file));\n            } else if (file.endsWith(\".xml.7z\")) {\n                is = new SevenZInputStream(new File(file));\n            } else if (file.endsWith(\".xml\")) {\n                is = new FileInputStream(file);\n            } else {\n                System.err.println(\"Unsupported file: \" + file\n                        + \". Supported: *.xml.gz, *.xml.bz2, *.xml.7z, *.xml\");\n                System.exit(-1);\n                return null; // will never be reached but is necessary to keep javac happy\n            }\n\n            BufferedReader br = new BufferedReader(new InputStreamReader(is, \"UTF-8\"));\n            sources[i] = new InputSource(br);\n        }\n\n        return sources;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/SAXParsingInterruptedException.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport org.xml.sax.SAXException;\n\n/**\n * Exception that is thrown if a {@link WikiDumpHandler} instance is told to\n * stop parsing while parsing a file.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class SAXParsingInterruptedException extends SAXException {\n\n    /**\n     * Generated version for serialisation.\n     */\n    private static final long serialVersionUID = -4208711186203540999L;\n\n    /**\n     * Create a new SAXException.\n     */\n    public SAXParsingInterruptedException() {\n        super();\n    }\n\n    /**\n     * Create a new SAXException.\n     * \n     * @param message\n     *            the error or warning message\n     */\n    public SAXParsingInterruptedException(String message) {\n        super(message);\n    }\n\n    /**\n     * Create a new SAXException wrapping an existing exception.\n     * \n     * The existing exception will be embedded in the new one, and its message\n     * will become the default message for the SAXException.\n     * \n     * @param e\n     *            the exception to be wrapped in a SAXException\n     */\n    public SAXParsingInterruptedException(Exception e) {\n        super(e);\n    }\n\n    /**\n     * Create a new SAXException from an existing exception.\n     * \n     * The existing exception will be embedded in the new one, but the new\n     * exception will have its own message.\n     * \n     * @param message\n     *            the detail message\n     * @param e\n     *            the exception to be wrapped in a SAXException\n     */\n    public SAXParsingInterruptedException(String message, Exception e) {\n        super(message, e);\n    }\n\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/WikiDump.java",
    "content": "package de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport java.io.PrintStream;\n\n/**\n * Interface for common WikiDump methods\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic interface WikiDump {\n\n    /**\n     * Gets the time the import started.\n     * \n     * @return the time the import started (in milliseconds)\n     */\n    public abstract long getTimeAtStart();\n\n    /**\n     * Gets the time the import finished.\n     * \n     * @return the time the import finished (in milliseconds)\n     */\n    public abstract long getTimeAtEnd();\n\n    /**\n     * Gets the number of imported pages.\n     * \n     * @return the number of pages imported into Scalaris\n     */\n    public abstract int getImportCount();\n\n    /**\n     * Sets the output writer to write status messages to (defaults to\n     * System.out).\n     * \n     * @param msgOut\n     *            the msgOut to set\n     */\n    public abstract void setMsgOut(PrintStream msgOut);\n\n    /**\n     * Whether {@link #stopParsing()} is supported or not.\n     * \n     * @return stop parsing support\n     */\n    public abstract boolean hasStopSupport();\n\n    /**\n     * Tells the import to stop (may not be supported by an implementation!).\n     */\n    public abstract void stopParsing();\n\n    /**\n     * Method to be called before using the handler.\n     */\n    public abstract void setUp();\n\n    /**\n     * Method to be called after using the handler (to clean up).\n     */\n    public abstract void tearDown();\n\n    /**\n     * Prints a message to the chosen output stream (includes a timestamp).\n     * \n     * @param message\n     *            the message to print\n     * \n     * @see #setMsgOut(PrintStream)\n     */\n    public void print(String message);\n\n    /**\n     * Prints a message to the chosen output stream (includes a timestamp).\n     * Includes a newline character at the end.\n     * \n     * @param message\n     *            the message to print\n     * \n     * @see #setMsgOut(PrintStream)\n     */\n    public void println(String message);\n\n    /**\n     * Indicates an error with the given message. The implementation decides where and if to print it.\n     * \n     * @param message\n     *            the error message\n     */\n    public void error(String message);\n    \n    /**\n     * Whether an error occurred during import.\n     * \n     * @return <tt>true</tt> if an error occurred, <tt>false</tt> otherwise\n     */\n    public boolean isErrorDuringImport();\n\n}"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/WikiDumpConvertPreparedSQLite.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport java.io.FileNotFoundException;\nimport java.io.IOException;\nimport java.io.PrintStream;\nimport java.text.NumberFormat;\nimport java.util.ArrayList;\nimport java.util.Collection;\nimport java.util.HashMap;\nimport java.util.List;\nimport java.util.Locale;\nimport java.util.Map.Entry;\nimport java.util.concurrent.ArrayBlockingQueue;\nimport java.util.regex.Matcher;\nimport java.util.regex.Pattern;\n\nimport com.almworks.sqlite4java.SQLiteConnection;\nimport com.almworks.sqlite4java.SQLiteException;\nimport com.almworks.sqlite4java.SQLiteStatement;\n\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.examples.wikipedia.Options;\nimport de.zib.scalaris.examples.wikipedia.Options.APPEND_INCREMENT_BUCKETS;\nimport de.zib.scalaris.examples.wikipedia.Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE;\nimport de.zib.scalaris.examples.wikipedia.Options.APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY;\nimport de.zib.scalaris.examples.wikipedia.Options.IBuckets;\nimport de.zib.scalaris.examples.wikipedia.Options.IReadBuckets;\nimport de.zib.scalaris.examples.wikipedia.Options.Optimisation;\nimport de.zib.scalaris.examples.wikipedia.Options.STORE_CONTRIB_TYPE;\nimport de.zib.scalaris.examples.wikipedia.SQLiteDataHandler;\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandler;\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandlerUnnormalised;\nimport de.zib.scalaris.examples.wikipedia.ScalarisOpType;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyNamespace;\n\n/**\n * Provides abilities to read a prepared SQLite wiki dump file and convert its\n * contents to a different optimisation scheme writing the converted data to a\n * new SQLite db.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class WikiDumpConvertPreparedSQLite implements WikiDump {\n    private static final int PRINT_SCALARIS_KV_PAIRS_EVERY = 5000;\n    protected ArrayBlockingQueue<Runnable> sqliteJobs = new ArrayBlockingQueue<Runnable>(10);\n    SQLiteWorker sqliteWorker = new SQLiteWorker();\n    protected boolean errorDuringImport = false;\n    \n    /**\n     * The time at the start of an import operation.\n     */\n    private long timeAtStart = 0;\n    /**\n     * The time at the end of an import operation.\n     */\n    private long timeAtEnd = 0;\n    /**\n     * The number of (successfully) processed K/V pairs.\n     */\n    protected int importedKeys = 0;\n    \n    protected PrintStream msgOut = System.out;\n    \n    protected boolean stop = false;\n\n    SQLiteConnection dbRead = null;\n    SQLiteConnection dbWrite = null;\n    protected SQLiteStatement stRead = null;\n    protected SQLiteStatement stWrite = null;\n    \n    final String dbReadFileName;\n    final String dbWriteFileName;\n\n    final Options dbWriteOptions;\n    \n    /**\n     * Constructor.\n     * \n     * @param dbReadFileName\n     *            the name of the database file to read from\n     * @param dbWriteFileName\n     *            the name of the database file to write to\n     * @param dbWriteOptions\n     *            optimisation scheme of the DB to write\n     * \n     * @throws RuntimeException\n     *             if the connection to Scalaris fails\n     */\n    public WikiDumpConvertPreparedSQLite(String dbReadFileName,\n            String dbWriteFileName, Options dbWriteOptions)\n            throws RuntimeException {\n        this.dbReadFileName = dbReadFileName;\n        this.dbWriteFileName = dbWriteFileName;\n        this.dbWriteOptions = dbWriteOptions;\n    }\n\n    /**\n     * Sets the time the import started.\n     */\n    final protected void importStart() {\n        timeAtStart = System.currentTimeMillis();\n    }\n\n    /**\n     * Sets the time the import finished.\n     */\n    final protected void importEnd() {\n        timeAtEnd = System.currentTimeMillis();\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#getTimeAtStart()\n     */\n    @Override\n    public long getTimeAtStart() {\n        return timeAtStart;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#getTimeAtEnd()\n     */\n    @Override\n    public long getTimeAtEnd() {\n        return timeAtEnd;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#getPageCount()\n     */\n    @Override\n    public int getImportCount() {\n        return importedKeys;\n    }\n\n    @Override\n    public boolean isErrorDuringImport() {\n        return errorDuringImport;\n    }\n\n    @Override\n    public void error(String message) {\n        System.err.println(message);\n        errorDuringImport = true;\n    }\n\n    /**\n     * Reports the speed of the import (pages/s) and may be used as a shutdown\n     * handler.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public class ReportAtShutDown extends Thread {\n        public void run() {\n            reportAtEnd();\n        }\n\n        /**\n         * Sets the import end time and reports the overall speed.\n         */\n        public void reportAtEnd() {\n            // import may have been interrupted - get an end time in this case\n            if (timeAtEnd == 0) {\n                importEnd();\n            }\n            final long timeTaken = timeAtEnd - timeAtStart;\n            final double speed = (((double) importedKeys) * 1000) / timeTaken;\n            NumberFormat nf = NumberFormat.getNumberInstance(Locale.ENGLISH);\n            nf.setGroupingUsed(true);\n            println(\"Finished conversion (\" + nf.format(speed) + \" keys/s)\");\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#setMsgOut(java.io.PrintStream)\n     */\n    @Override\n    public void setMsgOut(PrintStream msgOut) {\n        this.msgOut = msgOut;\n    }\n\n    /**\n     * Whether {@link #stopParsing()} is supported or not.\n     * \n     * @return no stop parsing support\n     */\n    @Override\n    public boolean hasStopSupport() {\n        return false;\n    }\n\n    /**\n     * Tells the import to stop (not supported since this may not result in a\n     * consistent view).\n     */\n    @Override\n    public void stopParsing() {\n//        this.stop = true;\n    }\n    \n    protected class SQLiteWorker extends Thread {\n        boolean stopWhenQueueEmpty = false;\n        boolean initialised = false;\n        \n        @Override\n        public void run() {\n            try {\n                // set up DB:\n                try {\n                    dbWrite = SQLiteDataHandler.openDB(dbWriteFileName, false);\n                    dbWrite.exec(\"CREATE TABLE objects(scalaris_key STRING PRIMARY KEY ASC, scalaris_value);\");\n                    stWrite = WikiDumpPrepareSQLiteForScalarisHandler.createWriteStmt(dbWrite);\n                } catch (SQLiteException e) {\n                    throw new RuntimeException(e);\n                }\n                initialised = true;\n\n                // take jobs\n\n                while(!(sqliteJobs.isEmpty() && stopWhenQueueEmpty)) {\n                    Runnable job;\n                    try {\n                        job = sqliteJobs.take();\n                    } catch (InterruptedException e) {\n                        throw new RuntimeException(e);\n                    }\n                    job.run();\n                }\n\n            } finally {\n                if (stWrite != null) {\n                    stWrite.dispose();\n                }\n                if (dbWrite != null) {\n                    dbWrite.dispose();\n                }\n                initialised = false;\n            }\n        }\n    }\n    \n    protected void addSQLiteJob(Runnable job) throws RuntimeException {\n        try {\n            sqliteJobs.put(job);\n        } catch (InterruptedException e) {\n            throw new RuntimeException(e);\n        }\n    }\n    \n    static class SQLiteWriteBytesJob implements Runnable {\n        final String key;\n        final byte[] value;\n        protected final SQLiteStatement stWrite;\n        private final WikiDump importer;\n        \n        public SQLiteWriteBytesJob(WikiDump importer, String key, byte[] value,\n                SQLiteStatement stWrite) {\n            this.importer = importer;\n            this.key = key;\n            this.value = value;\n            this.stWrite = stWrite;\n        }\n        \n        @Override\n        public void run() {\n            try {\n                try {\n                    stWrite.bind(1, key).bind(2, value).stepThrough();\n                } finally {\n                    stWrite.reset();\n                }\n            } catch (SQLiteException e) {\n                importer.error(\"write of \" + key + \" failed (sqlite error: \" + e.toString() + \")\");\n            }\n        }\n    }\n    \n    protected static class KVPair<T> {\n        public final String key;\n        public final T value;\n        \n        public KVPair(String key, T value) {\n            this.key = key;\n            this.value = value;\n        }\n    }\n    \n    static class SQLiteCopyList implements Runnable {\n        protected final String key;\n        protected final String countKey;\n        protected final byte[] value;\n        protected final SQLiteStatement stWrite;\n        private final WikiDump importer;\n        \n        public SQLiteCopyList(WikiDump importer, String key, byte[] value,\n                String countKey, SQLiteStatement stWrite) {\n            this.importer = importer;\n            this.key = key;\n            this.value = value;\n            this.stWrite = stWrite;\n            this.countKey = countKey;\n        }\n        \n        public static Collection<KVPair<Object>> splitOp(String key, String countKey, byte[] value) throws ClassNotFoundException, ClassCastException, IOException {\n            List<KVPair<Object>> result = new ArrayList<KVPair<Object>>(2);\n            // write list\n            result.add(new KVPair<Object>(key, value));\n            // write count (if available)\n            if (countKey != null) {\n                int listSize = ErlangValue.otpObjectToOtpList(\n                        WikiDumpPrepareSQLiteForScalarisHandler\n                                .objectFromBytes2(value).value()).arity();\n                result.add(new KVPair<Object>(countKey, listSize));\n            }\n            \n            return result;\n        }\n        \n        @Override\n        public void run() {\n            try {\n                Collection<KVPair<Object>> operations = splitOp(key, countKey, value);\n                for (KVPair<Object> kvPair : operations) {\n                    try {\n                        byte[] valueB;\n                        if (kvPair.value instanceof byte[]) {\n                            valueB = (byte[]) kvPair.value;\n                        } else {\n                            valueB = WikiDumpPrepareSQLiteForScalarisHandler.objectToBytes(kvPair.value);\n                        }\n                        stWrite.bind(1, kvPair.key).bind(2, valueB).stepThrough();\n                    } catch (SQLiteException e) {\n                        importer.error(\"write of \" + kvPair.key + \" failed (sqlite error: \" + e.toString() + \")\");\n                    } finally {\n                        try {\n                            stWrite.reset();\n                        } catch (SQLiteException e) {\n                            importer.error(\"failed to reset write statement (error: \" + e.toString() + \")\");\n                        }\n                    }\n                }\n            } catch (IOException e) {\n                importer.error(\"split of \" + key + \" failed (error: \" + e.toString() + \")\");\n            } catch (ClassNotFoundException e) {\n                importer.error(\"split of \" + key + \" failed (error: \" + e.toString() + \")\");\n            } catch (ClassCastException e) {\n                importer.error(\"split of \" + key + \" failed (error: \" + e.toString() + \")\");\n            }\n        }\n    }\n    \n    static class SQLiteWriteBucketListJob implements Runnable {\n        protected final String key;\n        protected final String countKey;\n        protected List<ErlangValue> value;\n        protected final SQLiteStatement stWrite;\n        protected final IBuckets optimisation;\n        private final WikiDump importer;\n        \n        public SQLiteWriteBucketListJob(WikiDump importer, String key,\n                List<ErlangValue> value, String countKey,\n                SQLiteStatement stWrite,\n                IBuckets optimisation) {\n            this.importer = importer;\n            this.key = key;\n            this.value = value;\n            this.stWrite = stWrite;\n            this.optimisation = optimisation;\n            this.countKey = countKey;\n        }\n\n        protected static HashMap<String, List<ErlangValue>> splitList(IBuckets optimisation, List<ErlangValue> value)\n                throws RuntimeException, ClassNotFoundException, IOException {\n            // split lists:\n            int bucketsToUse;\n            if (optimisation instanceof IReadBuckets) {\n                bucketsToUse = ((IReadBuckets) optimisation).getReadBuckets();\n            } else {\n                bucketsToUse = optimisation.getBuckets();\n            }\n            HashMap<String, List<ErlangValue>> newLists = new HashMap<String, List<ErlangValue>>(bucketsToUse);\n            for (ErlangValue obj : value) {\n                String keyAppend2;\n                if (optimisation instanceof APPEND_INCREMENT_BUCKETS) {\n                    keyAppend2 = ((APPEND_INCREMENT_BUCKETS) optimisation).getBucketString(obj);\n                } else if (optimisation instanceof APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY) {\n                    keyAppend2 = ((APPEND_INCREMENT_BUCKETS_WITH_WCACHE_ADDONLY) optimisation).getReadBucketString();\n                } else if (optimisation instanceof APPEND_INCREMENT_BUCKETS_WITH_WCACHE) {\n                    keyAppend2 = ((APPEND_INCREMENT_BUCKETS_WITH_WCACHE) optimisation).getReadBucketString(obj);\n                } else {\n                    throw new RuntimeException(\"unsupported optimisation: \" + optimisation);\n                }\n                List<ErlangValue> valueAtKey2 = newLists.get(keyAppend2);\n                if (valueAtKey2 == null) {\n                    // ideal size of each bucket: value.size() / bucketsToUse\n                    // ->  add 10% to include variance without having to increase the array list\n                    valueAtKey2 = new ArrayList<ErlangValue>(value.size() / bucketsToUse\n                            + ((value.size() / bucketsToUse) / 10));\n                    newLists.put(keyAppend2, valueAtKey2);\n                }\n                valueAtKey2.add(obj);\n            }\n            return newLists;\n        }\n        \n        @Override\n        public void run() {\n            try {\n                HashMap<String, List<ErlangValue>> newLists = splitList(optimisation, value);\n                for (Entry<String, List<ErlangValue>> newList : newLists.entrySet()) {\n                    try {\n                        // write list\n                        final String key2 = key + newList.getKey();\n                        try {\n                            stWrite.bind(1, key2).bind(2, WikiDumpPrepareSQLiteForScalarisHandler.objectToBytes(newList.getValue())).stepThrough();\n                        } catch (SQLiteException e) {\n                            importer.error(\"write of \" + key2 + \" failed (sqlite error: \" + e.toString() + \")\");\n                        } catch (IOException e) {\n                            importer.error(\"write of \" + key2 + \" failed (error: \" + e.toString() + \")\");\n                        }\n                        // write count (if available)\n                        if (countKey != null) {\n                            final String countKey2 = countKey + newList.getKey();\n                            try {\n                                stWrite.reset();\n                                stWrite.bind(1, countKey2).bind(2, WikiDumpPrepareSQLiteForScalarisHandler.objectToBytes(newList.getValue().size())).stepThrough();\n                            } catch (SQLiteException e) {\n                                importer.error(\"write of \" + countKey2 + \" failed (sqlite error: \" + e.toString() + \")\");\n                            } catch (IOException e) {\n                                importer.error(\"write of \" + countKey2 + \" failed (error: \" + e.toString() + \")\");\n                            }\n                        }\n                    } finally {\n                        try {\n                            stWrite.reset();\n                        } catch (SQLiteException e) {\n                            importer.error(\"failed to reset write statement (error: \" + e.toString() + \")\");\n                        }\n                    }\n                }\n            } catch (Exception e) {\n                importer.error(\"write of \" + key + \" failed (error: \" + e.toString() + \")\");\n                return;\n            }\n        }\n    }\n    \n    static class SQLiteWriteBucketCounterJob implements Runnable {\n        protected final String key;\n        protected final int value;\n        protected final SQLiteStatement stWrite;\n        protected final IBuckets optimisation;\n        private final WikiDump importer;\n        \n        public SQLiteWriteBucketCounterJob(WikiDump importer, String key,\n                int value, SQLiteStatement stWrite,\n                IBuckets optimisation) {\n            this.importer = importer;\n            this.key = key;\n            this.value = value;\n            this.stWrite = stWrite;\n            this.optimisation = optimisation;\n        }\n        \n        public static Collection<KVPair<Integer>> splitCounter(IBuckets optimisation, String key, int value) {\n            // cannot partition a counter without its original values,\n            // however, it does not matter which counter is how large\n            // -> make all bucket counters (almost) the same value\n            int avg = value;\n            int bucketsToUse;\n            if (optimisation instanceof IReadBuckets) {\n                bucketsToUse = ((IReadBuckets) optimisation).getReadBuckets();\n            } else if (optimisation instanceof APPEND_INCREMENT_BUCKETS) {\n                bucketsToUse = optimisation.getBuckets();\n            } else {\n                throw new RuntimeException(\"unsupported optimisation: \" + optimisation);\n            }\n            List<KVPair<Integer>> result = new ArrayList<KVPair<Integer>>(bucketsToUse);\n            avg = value / bucketsToUse;\n            int rest = value; \n            \n            for (int i = 0; i < bucketsToUse; ++i) {\n                final String key2 = key + \":\" + i;\n                int curValue;\n                curValue = (i == bucketsToUse - 1) ? rest : avg;\n                rest -= curValue;\n                result.add(new KVPair<Integer>(key2, curValue));\n            }\n            assert(rest == 0);\n            \n            return result;\n        }\n        \n        @Override\n        public void run() {\n            Collection<KVPair<Integer>> operations = splitCounter(optimisation, key, value);\n            for (KVPair<Integer> kvPair : operations) {\n                try {\n                    stWrite.bind(1, kvPair.key).bind(2, WikiDumpPrepareSQLiteForScalarisHandler.objectToBytes(kvPair.value)).stepThrough();\n                } catch (SQLiteException e) {\n                    importer.error(\"write of \" + kvPair.key + \" failed (sqlite error: \" + e.toString() + \")\");\n                } catch (IOException e) {\n                    importer.error(\"write of \" + kvPair.key + \" failed (error: \" + e.toString() + \")\");\n                } finally {\n                    try {\n                        stWrite.reset();\n                    } catch (SQLiteException e) {\n                        importer.error(\"failed to reset write statement (error: \" + e.toString() + \")\");\n                    }\n                }\n            }\n        }\n    }\n\n    protected static enum ListOrCountOp {\n        NONE, LIST, COUNTER;\n    }\n    \n    static {\n        // BEWARE: keep in sync with ScalarisDataHandler!\n        assert ScalarisDataHandler.getPageListKey(0).equals(\"pages:0\");\n        assert ScalarisDataHandler.getPageListKey(-2).equals(\"pages:-2\");\n        assert ScalarisDataHandler.getPageCountKey(0).equals(\"pages:0:count\");\n        assert ScalarisDataHandlerUnnormalised.getRevKey(\"foobar\", 0, new MyNamespace()).equals(\"foobar:rev:0\");\n        assert ScalarisDataHandlerUnnormalised.getPageKey(\"foobar\", new MyNamespace()).equals(\"foobar:page\");\n        assert ScalarisDataHandlerUnnormalised.getRevListKey(\"foobar\", new MyNamespace()).equals(\"foobar:revs\");\n        assert ScalarisDataHandlerUnnormalised.getCatPageListKey(\"foobar\", new MyNamespace()).equals(\"foobar:cpages\");\n        assert ScalarisDataHandlerUnnormalised.getCatPageCountKey(\"foobar\", new MyNamespace()).equals(\"foobar:cpages:count\");\n        assert ScalarisDataHandlerUnnormalised.getTplPageListKey(\"foobar\", new MyNamespace()).equals(\"foobar:tpages\");\n        assert ScalarisDataHandlerUnnormalised.getBackLinksPageListKey(\"foobar\", new MyNamespace()).equals(\"foobar:blpages\");\n        assert ScalarisDataHandler.getContributionListKey(\"foobar\").equals(\"foobar:user:contrib\");\n    }\n    \n    protected static final Pattern pageListPattern = Pattern.compile(\"^pages:([+-]?[0-9]+)$\", Pattern.DOTALL);\n    protected static final Pattern pageCountPattern = Pattern.compile(\"^pages:([+-]?[0-9]+):count$\", Pattern.DOTALL);\n    protected static final Pattern revPattern = Pattern.compile(\"^(.*):rev:([0-9]+)$\", Pattern.DOTALL);\n    protected static final Pattern pagePattern = Pattern.compile(\"^(.*):page$\", Pattern.DOTALL);\n    protected static final Pattern revListPattern = Pattern.compile(\"^(.*):revs$\", Pattern.DOTALL);\n    protected static final Pattern catPageListPattern = Pattern.compile(\"^(.*):cpages$\", Pattern.DOTALL);\n    protected static final Pattern catPageCountPattern = Pattern.compile(\"^(.*):cpages:count$\", Pattern.DOTALL);\n    protected static final Pattern tplPageListPattern = Pattern.compile(\"^(.*):tpages$\", Pattern.DOTALL);\n    protected static final Pattern backLinksPageListPattern = Pattern.compile(\"^(.*):blpages$\", Pattern.DOTALL);\n    protected static final Pattern contributionListPattern = Pattern.compile(\"^(.*):user:contrib$\", Pattern.DOTALL);\n    \n    protected static class ConvertOp {\n        ListOrCountOp listOrCount = ListOrCountOp.NONE;\n        String countKey = null;\n        Optimisation optimisation = null;\n        Optimisation countKeyOptimisation = null;\n    }\n    \n    /**\n     * Starts the conversion.\n     * \n     * @throws RuntimeException\n     * @throws FileNotFoundException\n     */\n    public void convertObjects() throws RuntimeException, FileNotFoundException {\n        importStart();\n        try {\n            try {                \n                while (stRead.step()) {\n                    ++importedKeys;\n                    String key = stRead.columnString(0);\n                    byte[] value = stRead.columnBlob(1);\n                    \n                    ConvertOp convOp = getConvertOp(key, dbWriteOptions);\n                    if (convOp == null) {\n                        println(\"unknown key: \" + key);\n                        // use defaults and continue anyway...\n                        convOp = new ConvertOp();\n                    }\n                    \n                    if (convOp.optimisation instanceof IBuckets) {\n                        IBuckets optimisation = (IBuckets) convOp.optimisation;\n                        switch (convOp.listOrCount) {\n                            case LIST:\n                                List<ErlangValue> listVal = WikiDumpPrepareSQLiteForScalarisHandler\n                                        .objectFromBytes2(value).listValue();\n                                String countKey;\n                                if (convOp.countKeyOptimisation == null) {\n                                    // integrated counter\n                                    countKey = convOp.countKey;\n                                } else {\n                                    // separate counter\n                                    countKey = null;\n                                    int listSize = listVal.size();\n                                    if (convOp.countKeyOptimisation instanceof IBuckets) {\n                                        // similar to \"case COUNTER\" below:\n                                        final int countValue = listSize;\n                                        addSQLiteJob(new SQLiteWriteBucketCounterJob(\n                                                this, convOp.countKey, countValue, stWrite,\n                                                (IBuckets) convOp.countKeyOptimisation));\n                                    } else {\n                                        // copy counter\n                                        final byte[] countValue = WikiDumpPrepareSQLiteForScalarisHandler\n                                                .objectToBytes(listSize);\n                                        addSQLiteJob(new SQLiteWriteBytesJob(\n                                                this, convOp.countKey,\n                                                countValue, stWrite));\n                                    }\n                                }\n                                addSQLiteJob(new SQLiteWriteBucketListJob(\n                                        this, key, listVal, countKey,\n                                        stWrite, optimisation));\n                                break;\n                            case COUNTER:\n                                final int countValue = WikiDumpPrepareSQLiteForScalarisHandler\n                                        .objectFromBytes2(value).intValue();\n                                addSQLiteJob(new SQLiteWriteBucketCounterJob(\n                                        this, key, countValue, stWrite,\n                                        optimisation));\n                                break;\n                            default:\n                                break;\n                        }\n                    } else if (convOp.optimisation != null ) {\n                        assert (convOp.countKeyOptimisation == null);\n                        if (convOp.listOrCount == ListOrCountOp.LIST) {\n                            addSQLiteJob(new SQLiteCopyList(this, key, value, convOp.countKey, stWrite));\n                        } else {\n                            addSQLiteJob(new SQLiteWriteBytesJob(this, key, value, stWrite));\n                        }\n                    }\n                    if ((importedKeys % PRINT_SCALARIS_KV_PAIRS_EVERY) == 0) {\n                        println(\"processed keys: \" + importedKeys);\n                    }\n                }\n            } finally {\n                stRead.reset();\n            }\n        } catch (FileNotFoundException e) {\n            throw e;\n        } catch (SQLiteException e) {\n            error(\"read failed (sqlite error: \" + e.toString() + \")\");\n            throw new RuntimeException(e);\n        } catch (ClassNotFoundException e) {\n            error(\"read failed (sqlite error: \" + e.toString() + \")\");\n            throw new RuntimeException(e);\n        } catch (IOException e) {\n            error(\"read failed (error: \" + e.toString() + \")\");\n            throw new RuntimeException(e);\n        }\n    }\n\n    /**\n     * Parses the key and gets the conversion op needed for a plain DB dump to a\n     * dump with the given optimisations.\n     * \n     * @param key\n     *            the key to write to\n     * @param dbWriteOptions\n     *            optimisation options\n     * @return convert operation or <tt>null</tt> if the key signature is\n     *         unknown\n     * \n     * @throws NumberFormatException\n     */\n    public static ConvertOp getConvertOp(String key, Options dbWriteOptions) throws NumberFormatException {\n        ConvertOp convOp = new ConvertOp();\n        ScalarisOpType opType = null;\n        ScalarisOpType opTypeCount = null;\n\n        final Matcher pageListMatcher = pageListPattern.matcher(key);\n        final Matcher pageCountMatcher = pageCountPattern.matcher(key);\n        final Matcher revMatcher = revPattern.matcher(key);\n        final Matcher pageMatcher = pagePattern.matcher(key);\n        final Matcher revListMatcher = revListPattern.matcher(key);\n        final Matcher catPageListMatcher = catPageListPattern.matcher(key);\n        final Matcher catPageCountMatcher = catPageCountPattern.matcher(key);\n        final Matcher tplPageListMatcher = tplPageListPattern.matcher(key);\n        final Matcher backLinksPageListMatcher = backLinksPageListPattern.matcher(key);\n        final Matcher contributionListMatcher = contributionListPattern.matcher(key);\n        \n        if (key.equals(ScalarisDataHandler.getSiteInfoKey())) {\n            convOp.optimisation = new Options.TRADITIONAL();\n        } else if (pageListMatcher.matches()) {\n            int namespace = Integer.parseInt(pageListMatcher.group(1));\n            convOp.countKey = ScalarisDataHandler.getPageCountKey(namespace);\n            opType = ScalarisOpType.PAGE_LIST;\n            opTypeCount = ScalarisOpType.PAGE_COUNT;\n            convOp.listOrCount = ListOrCountOp.LIST;\n        } else if (pageCountMatcher.matches()) {\n            // ignore (written during page list partitioning (see above)\n            convOp.listOrCount = ListOrCountOp.COUNTER;\n        } else if (key.equals(ScalarisDataHandler.getArticleCountKey())) {\n            convOp.countKey = null;\n            opType = ScalarisOpType.ARTICLE_COUNT;\n            convOp.listOrCount = ListOrCountOp.COUNTER;\n        } else if (revMatcher.matches()) {\n            opType = ScalarisOpType.REVISION;\n        } else if (pageMatcher.matches()) {\n            opType = ScalarisOpType.PAGE;\n        } else if (revListMatcher.matches()) {\n            convOp.countKey = null;\n            opType = ScalarisOpType.SHORTREV_LIST;\n            convOp.listOrCount = ListOrCountOp.LIST;\n        } else if (catPageListMatcher.matches()) {\n            String title = catPageListMatcher.group(1);\n            convOp.countKey = title + \":cpages:count\";\n            opType = ScalarisOpType.CATEGORY_PAGE_LIST;\n            opTypeCount = ScalarisOpType.CATEGORY_PAGE_COUNT;\n            convOp.listOrCount = ListOrCountOp.LIST;\n        } else if (catPageCountMatcher.matches()) {\n            // ignore (written during page list partitioning (see above)\n            convOp.listOrCount = ListOrCountOp.COUNTER;\n        } else if (tplPageListMatcher.matches()) {\n            convOp.countKey = null;\n            opType = ScalarisOpType.TEMPLATE_PAGE_LIST;\n            convOp.listOrCount = ListOrCountOp.LIST;\n        } else if (backLinksPageListMatcher.matches()) {\n            // omit if disabled\n            if (dbWriteOptions.WIKI_USE_BACKLINKS) {\n                convOp.countKey = null;\n                opType = ScalarisOpType.BACKLINK_PAGE_LIST;\n                convOp.listOrCount = ListOrCountOp.LIST;\n            }\n        } else if (key.matches(ScalarisDataHandler.getStatsPageEditsKey())) {\n            convOp.countKey = null;\n            opType = ScalarisOpType.EDIT_STAT;\n            convOp.listOrCount = ListOrCountOp.COUNTER;\n        } else if (contributionListMatcher.matches()) {\n            // omit if disabled\n            if (dbWriteOptions.WIKI_STORE_CONTRIBUTIONS != STORE_CONTRIB_TYPE.NONE) {\n                convOp.countKey = null;\n                opType = ScalarisOpType.CONTRIBUTION;\n                convOp.listOrCount = ListOrCountOp.LIST;\n            }\n        } else {\n            return null;\n        }\n\n        if (opType != null) {\n            convOp.optimisation = dbWriteOptions.OPTIMISATIONS.get(opType);\n        }\n        if (opTypeCount != null) {\n            convOp.countKeyOptimisation = dbWriteOptions.OPTIMISATIONS.get(opTypeCount);\n        }\n        return convOp;\n    }\n\n    /**\n     * Sets up the directory to write files to as well as the Scalaris\n     * connection.\n     * \n     * @throws RuntimeException\n     *             if the directory could not be created\n     */\n    @Override\n    public void setUp() {\n        try {\n            dbRead = SQLiteDataHandler.openDB(dbReadFileName, true, null);\n            stRead = dbRead.prepare(\"SELECT scalaris_key, scalaris_value FROM objects\");\n        } catch (SQLiteException e) {\n            error(\"Cannot read database: \" + dbReadFileName);\n            throw new RuntimeException(e);\n        }\n        println(\"Converting prepared SQLite wiki dump ...\");\n        sqliteWorker.start();\n        // wait for worker to initialise the DB and the prepared statements\n        while (!sqliteWorker.initialised) {\n            try {\n                Thread.sleep(100);\n            } catch (InterruptedException e) {\n                throw new RuntimeException(e);\n            }\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see java.lang.Object#finalize()\n     */\n    @Override\n    protected void finalize() throws Throwable {\n        super.finalize();\n        tearDown();\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#tearDown()\n     */\n    @Override\n    public void tearDown() {\n        sqliteWorker.stopWhenQueueEmpty = true;\n        addSQLiteJob(new WikiDumpPrepareSQLiteForScalarisHandler.SQLiteNoOpJob());\n        // wait for worker to close the DB\n        try {\n            sqliteWorker.join();\n        } catch (InterruptedException e) {\n            throw new RuntimeException(e);\n        }\n        if (stRead != null) {\n            stRead.dispose();\n        }\n        if (dbRead != null) {\n            dbRead.dispose();\n        }\n        importEnd();\n    }\n\n    /**\n     * Prints a message to the chosen output stream (includes a timestamp).\n     * \n     * @param message\n     *            the message to print\n     * \n     * @see #setMsgOut(PrintStream)\n     */\n    public void print(String message) {\n        WikiDumpHandler.print(msgOut, message);\n    }\n\n    /**\n     * Prints a message to the chosen output stream (includes a timestamp).\n     * Includes a newline character at the end.\n     * \n     * @param message\n     *            the message to print\n     * \n     * @see #setMsgOut(PrintStream)\n     */\n    public void println(String message) {\n        WikiDumpHandler.println(msgOut, message);\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/WikiDumpGetCategoryTreeHandler.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport java.io.FileNotFoundException;\nimport java.io.PrintStream;\nimport java.util.ArrayList;\nimport java.util.Arrays;\nimport java.util.Calendar;\nimport java.util.Collection;\nimport java.util.HashSet;\nimport java.util.LinkedList;\nimport java.util.List;\nimport java.util.Map;\nimport java.util.Set;\nimport java.util.SortedSet;\nimport java.util.TreeSet;\nimport java.util.concurrent.ArrayBlockingQueue;\n\nimport com.almworks.sqlite4java.SQLiteConnection;\nimport com.almworks.sqlite4java.SQLiteException;\nimport com.almworks.sqlite4java.SQLiteStatement;\n\nimport de.zib.scalaris.examples.wikipedia.SQLiteDataHandler;\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandlerNormalised;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyNamespace;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyWikiModel;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\n\n/**\n * Provides abilities to read an xml wiki dump file and create a category (and\n * template) tree.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class WikiDumpGetCategoryTreeHandler extends WikiDumpHandler {\n    private static final int PRINT_PAGES_EVERY = 400;\n    protected String dbFileName;\n    protected SQLiteConnection db = null;\n    protected SQLiteStatement stGetPageId = null;\n    protected SQLiteStatement stWritePages = null;\n    protected SQLiteStatement stWriteCategories = null;\n    protected SQLiteStatement stWriteTemplates = null;\n    protected SQLiteStatement stWriteIncludes = null;\n    protected SQLiteStatement stWriteRedirects = null;\n    protected SQLiteStatement stWriteLinks = null;\n    protected long nextPageId = 0l;\n    protected ArrayBlockingQueue<SQLiteJob> sqliteJobs = new ArrayBlockingQueue<SQLiteJob>(100);\n    SQLiteWorker sqliteWorker = new SQLiteWorker();\n    \n    /**\n     * Sets up a SAX XmlHandler extracting all categories from all pages except\n     * the ones in a blacklist to stdout.\n     * \n     * @param blacklist\n     *            a number of page titles to ignore\n     * @param minTime\n     *            minimum time a revision should have (only one revision older\n     *            than this will be imported) - <tt>null/tt> imports all\n     *            revisions\n     * @param maxTime\n     *            maximum time a revision should have (newer revisions are\n     *            omitted) - <tt>null/tt> imports all revisions\n     *            (useful to create dumps of a wiki at a specific point in time)\n     * @param dbFileName\n     *            the name of the directory to write categories, templates,\n     *            inclusions etc to\n     * \n     * @throws RuntimeException\n     *             if the creation of the SQLite DB fails\n     */\n    public WikiDumpGetCategoryTreeHandler(Set<String> blacklist,\n            Calendar minTime, Calendar maxTime, String dbFileName)\n            throws RuntimeException {\n        super(blacklist, null, 1, minTime, maxTime);\n        this.dbFileName = dbFileName;\n    }\n    \n    protected void addSQLiteJob(SQLiteJob job) throws RuntimeException {\n        try {\n            sqliteJobs.put(job);\n        } catch (InterruptedException e) {\n            throw new RuntimeException(e);\n        }\n    }\n\n    protected void writeValue(SQLiteStatement stmt, NormalisedTitle key, NormalisedTitle value)\n            throws RuntimeException {\n        addSQLiteJob(new SQLiteWriteValuesJob(stmt, key, value));\n    }\n\n    protected void writeValues(SQLiteStatement stmt, NormalisedTitle key, Collection<? extends NormalisedTitle> values)\n            throws RuntimeException {\n        addSQLiteJob(new SQLiteWriteValuesJob(stmt, key, values));\n    }\n\n    protected void writeSiteInfo(SiteInfo siteInfo)\n            throws RuntimeException {\n        addSQLiteJob(new SQLiteWriteSiteInfoJob(siteInfo));\n    }\n\n    static SiteInfo readSiteInfo(SQLiteConnection db) throws RuntimeException {\n        SQLiteStatement stmt = null;\n        try {\n            stmt = db.prepare(\"SELECT value FROM properties WHERE key == ?\");\n            return WikiDumpPrepareSQLiteForScalarisHandler.readObject2(stmt, \"siteinfo\").jsonValue(SiteInfo.class);\n        } catch (SQLiteException e) {\n            throw new RuntimeException(e);\n        } catch (FileNotFoundException e) {\n            throw new RuntimeException(e);\n        } finally {\n            if (stmt != null) {\n                stmt.dispose();\n            }\n        }\n    }\n    \n    protected static void updateMap(\n            Map<NormalisedTitle, Set<NormalisedTitle>> map,\n            NormalisedTitle key, NormalisedTitle addToValue) {\n        Set<NormalisedTitle> oldValue = map.get(key);\n        if (oldValue == null) {\n            oldValue = new HashSet<NormalisedTitle>();\n            map.put(key, oldValue);\n        }\n        oldValue.add(addToValue);\n    }\n\n    /**\n     * Exports the given siteinfo (nothing to do here).\n     * \n     * @param revisions\n     *            the siteinfo to export\n     */\n    @Override\n    protected void export(XmlSiteInfo siteinfo_xml) {\n        writeSiteInfo(siteinfo_xml.getSiteInfo());\n    }\n\n    /**\n     * Builds the category tree.\n     * \n     * @param page_xml\n     *            the page object extracted from XML\n     */\n    @Override\n    protected void export(XmlPage page_xml) {\n        Page page = page_xml.getPage();\n\n        if (page.getCurRev() != null && wikiModel != null) {\n            wikiModel.setUp();\n            final NormalisedTitle normTitle = wikiModel.normalisePageTitle(page.getTitle());\n            wikiModel.setNamespaceName(wikiModel.getNamespace().getNamespaceByNumber(normTitle.namespace));\n            wikiModel.setPageName(normTitle.title);\n            wikiModel.renderPageWithCache(null, page.getCurRev().unpackedText());\n            \n            // categories:\n            do {\n                final Set<String> pageCategories_raw = wikiModel.getCategories().keySet();\n                ArrayList<NormalisedTitle> pageCategories = new ArrayList<NormalisedTitle>(pageCategories_raw.size());\n                for (String cat_raw: pageCategories_raw) {\n                    NormalisedTitle category = new NormalisedTitle(\n                            MyNamespace.CATEGORY_NAMESPACE_KEY,\n                            MyWikiModel.normaliseName(cat_raw));\n                    pageCategories.add(category);\n                }\n                writeValues(stWriteCategories, normTitle, pageCategories);\n            } while(false);\n            \n            // templates:\n            do {\n                final Set<String> pageTemplates_raw = wikiModel.getTemplatesNoMagicWords();\n                ArrayList<NormalisedTitle> pageTemplates = new ArrayList<NormalisedTitle>(pageTemplates_raw.size());\n                for (String tpl_raw: pageTemplates_raw) {\n                    NormalisedTitle template = new NormalisedTitle(\n                            MyNamespace.TEMPLATE_NAMESPACE_KEY,\n                            MyWikiModel.normaliseName(tpl_raw));\n                    pageTemplates.add(template);\n                }\n                writeValues(stWriteTemplates, normTitle, pageTemplates);\n            } while (false);\n            \n            // includes:\n            do {\n                Set<String> pageIncludes = wikiModel.getIncludes();\n                if (!pageIncludes.isEmpty()) {\n                    // make sure, the pageIncludes set is not changed anymore (deferred processing in the thread taking place):\n                    ArrayList<NormalisedTitle> normPageIncludes = new ArrayList<NormalisedTitle>(pageIncludes.size());\n                    wikiModel.normalisePageTitles(pageIncludes, normPageIncludes);\n                    writeValues(stWriteIncludes, normTitle, normPageIncludes);\n                }\n            } while (false);\n            \n            // redirections:\n            do {\n                String pageRedirLink = wikiModel.getRedirectLink();\n                if (pageRedirLink != null) {\n                    writeValue(stWriteRedirects, normTitle, wikiModel.normalisePageTitle(pageRedirLink));\n                }\n            } while(false);\n            \n            // links:\n            do {\n                Set<String> pageLinks = wikiModel.getLinks();\n                if (!pageLinks.isEmpty()) {\n                    // make sure, the pageIncludes set is not changed anymore (deferred processing in the thread taking place):\n                    ArrayList<NormalisedTitle> normPageLinks = new ArrayList<NormalisedTitle>(pageLinks.size());\n                    wikiModel.normalisePageTitles(pageLinks, normPageLinks);\n                    writeValues(stWriteLinks, normTitle, normPageLinks);\n                }\n            } while(false);\n            \n            wikiModel.tearDown();\n        }\n        ++pageCount;\n        // only export page list every UPDATE_PAGELIST_EVERY pages:\n        if ((pageCount % PRINT_PAGES_EVERY) == 0) {\n            println(\"processed pages: \" + pageCount);\n        }\n    }\n\n    /**\n     * Gets all sub categories that belong to a given root category\n     * (recursively).\n     * \n     * @param tree\n     *            the tree of categories or templates as created by\n     *            {@link #readTrees(String, Map, Map, Map)}\n     * @param root\n     *            a root category or template\n     * \n     * @return a set of all sub categories/templates; also includes the root\n     */\n    public static Set<NormalisedTitle> getAllChildren(Map<NormalisedTitle, Set<NormalisedTitle>> tree, NormalisedTitle root) {\n        return getAllChildren(tree, new LinkedList<NormalisedTitle>(Arrays.asList(root)));\n    }\n    \n    /**\n     * Gets all sub categories that belong to any of the given root categories\n     * (recursively).\n     * \n     * @param tree\n     *            the tree of categories or templates as created by\n     *            {@link #readTrees(String, Map, Map, Map)}\n     * @param roots\n     *            a list of root categories or templates\n     * \n     * @return a set of all sub categories; also includes the rootCats\n     */\n    public static Set<NormalisedTitle> getAllChildren(Map<NormalisedTitle, Set<NormalisedTitle>> tree, List<NormalisedTitle> roots) {\n        HashSet<NormalisedTitle> allChildren = new HashSet<NormalisedTitle>(roots);\n        while (!roots.isEmpty()) {\n            NormalisedTitle curChild = roots.remove(0);\n            Set<NormalisedTitle> subChilds = tree.get(curChild);\n            if (subChilds != null) {\n                // only add new children to the root list\n                // (remove already processed ones)\n                // -> prevents endless loops in circles\n                Set<NormalisedTitle> newChilds = new HashSet<NormalisedTitle>(subChilds);\n                newChilds.removeAll(allChildren);\n                allChildren.addAll(newChilds);\n                roots.addAll(newChilds);\n            }\n        }\n        return allChildren;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpHandler#setUp()\n     */\n    @Override\n    public void setUp() {\n        super.setUp();\n        sqliteWorker.start();\n        // wait for worker to initialise the DB and the prepared statements\n        while (!sqliteWorker.initialised) {\n            try {\n                Thread.sleep(100);\n            } catch (InterruptedException e) {\n                throw new RuntimeException(e);\n            }\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpHandler#tearDown()\n     */\n    @Override\n    public void tearDown() {\n        super.tearDown();\n        sqliteWorker.stopWhenQueueEmpty = true;\n        addSQLiteJob(new SQLiteNoOpJob());\n        // wait for worker to close the DB\n        try {\n            sqliteWorker.join();\n        } catch (InterruptedException e) {\n            throw new RuntimeException(e);\n        }\n        importEnd();\n    }\n    \n    /**\n     * Reads the given parameter trees from the DB file.\n     * \n     * @param dbFileName\n     *            name of the DB file\n     * @param templateTree\n     *            information about the templates and their dependencies\n     * @param includeTree\n     *            information about page includes\n     * @param referenceTree\n     *            information about references to a page\n     * \n     * @throws RuntimeException if any error occurs\n     */\n    public static void readTrees(\n            String dbFileName,\n            Map<NormalisedTitle, Set<NormalisedTitle>> templateTree,\n            Map<NormalisedTitle, Set<NormalisedTitle>> includeTree,\n            Map<NormalisedTitle, Set<NormalisedTitle>> referenceTree)\n            throws RuntimeException {\n        SQLiteConnection db = null;\n        SQLiteStatement stmt = null;\n        try {\n            db = SQLiteDataHandler.openDB(dbFileName, true);\n            stmt = db\n                    .prepare(\"SELECT page.title, tpl.title FROM \" +\n                            \"templates INNER JOIN pages AS page ON templates.title == page.id \" +\n                            \"INNER JOIN pages AS tpl ON templates.template == tpl.id \" +\n                            \"WHERE page.title LIKE '\" + (new NormalisedTitle(MyNamespace.TEMPLATE_NAMESPACE_KEY, \"\")).toString() + \"%';\");\n            while (stmt.step()) {\n                NormalisedTitle pageTitle = NormalisedTitle.fromNormalised(stmt.columnString(0));\n                NormalisedTitle template = NormalisedTitle.fromNormalised(stmt.columnString(1));\n                updateMap(templateTree, pageTitle, template);\n            }\n            stmt.dispose();\n            stmt = db\n                    .prepare(\"SELECT page.title, incl.title FROM \" +\n                            \"includes INNER JOIN pages AS page ON includes.title == page.id \" +\n                            \"INNER JOIN pages AS incl ON includes.include == incl.id;\");\n            while (stmt.step()) {\n                NormalisedTitle pageTitle = NormalisedTitle.fromNormalised(stmt.columnString(0));\n                NormalisedTitle include = NormalisedTitle.fromNormalised(stmt.columnString(1));\n                updateMap(includeTree, pageTitle, include);\n            }\n            stmt.dispose();\n            stmt = db\n                    .prepare(\"SELECT page.title, redir.title FROM \" +\n                            \"redirects INNER JOIN pages AS page ON redirects.title == page.id \" +\n                            \"INNER JOIN pages AS redir ON redirects.redirect == redir.id;\");\n            while (stmt.step()) {\n                NormalisedTitle pageTitle = NormalisedTitle.fromNormalised(stmt.columnString(0));\n                NormalisedTitle redirect = NormalisedTitle.fromNormalised(stmt.columnString(1));\n                updateMap(referenceTree, redirect, pageTitle);\n            }\n        } catch (SQLiteException e) {\n            System.err.println(\"read of category tree failed (sqlite error: \" + e.toString() + \")\");\n            throw new RuntimeException(e);\n        } finally {\n            if (stmt != null) {\n                stmt.dispose();\n            }\n            if (db != null) {\n                db.dispose();\n            }\n        }\n    }\n    \n    /**\n     * Extracts all pages in the given categories from the given DB.\n     * \n     * @param dbFileName\n     *            name of the DB file\n     * @param allowedCats0\n     *            include all pages in these categories (un-normalised page\n     *            titles)\n     * @param allowedPages0\n     *            a number of pages to include, also parses these pages for more\n     *            links (un-normalised page titles)\n     * @param depth\n     *            follow links this deep\n     * @param templateTree\n     *            information about the templates and their dependencies\n     * @param includeTree\n     *            information about page includes\n     * @param referenceTree\n     *            information about references to a page\n     * @param msgOut\n     *            the output stream to write status messages to\n     * @param normalised\n     *            whether the pages should be returned as normalised page titles\n     *            or not\n     * \n     * @return a (sorted) set of page titles\n     * \n     * @throws RuntimeException\n     *             if any error occurs\n     */\n    public static SortedSet<String> getPagesInCategories(String dbFileName,\n            Set<String> allowedCats0, Set<String> allowedPages0, int depth,\n            Map<NormalisedTitle, Set<NormalisedTitle>> templateTree,\n            Map<NormalisedTitle, Set<NormalisedTitle>> includeTree,\n            Map<NormalisedTitle, Set<NormalisedTitle>> referenceTree,\n            PrintStream msgOut, boolean normalised) throws RuntimeException {\n        SQLiteConnection db = null;\n        try {\n            // set 1GB cache_size:\n            db = SQLiteDataHandler.openDB(dbFileName, true, 1024l*1024l*1024l);\n            db.exec(\"CREATE TEMPORARY TABLE currentPages(id INTEGER PRIMARY KEY ASC);\");\n            SiteInfo siteInfo = readSiteInfo(db);\n            MyNamespace namespace = new MyNamespace(siteInfo);\n            ArrayList<NormalisedTitle> allowedCats = new ArrayList<NormalisedTitle>(allowedCats0.size());\n            MyWikiModel.normalisePageTitles(allowedCats0, namespace, allowedCats);\n\n            Set<NormalisedTitle> allowedCatsFull = getSubCategories(allowedCats0, allowedCats, db,\n                    templateTree, includeTree, referenceTree, msgOut, namespace);\n\n            ArrayList<NormalisedTitle> allowedPages = new ArrayList<NormalisedTitle>(allowedPages0.size());\n            MyWikiModel.normalisePageTitles(allowedPages0, namespace, allowedPages);\n\n            Set<NormalisedTitle> currentPages = new HashSet<NormalisedTitle>();\n            currentPages.addAll(allowedPages);\n            currentPages.addAll(allowedCatsFull);\n            currentPages.addAll(getPagesDirectlyInCategories(allowedCatsFull, db));\n\n            Set<NormalisedTitle> normalisedPages = getRecursivePages(currentPages, depth, db,\n                    templateTree, includeTree, referenceTree, msgOut);\n            \n            // no need to drop table - we set temporary tables to be in-memory only\n//            db.exec(\"DROP TABLE currentPages;\");\n\n            // note: need to sort case-sensitively (wiki is only case-insensitive at the first char)\n            final TreeSet<String> pages = new TreeSet<String>();\n            if (normalised) {\n                pages.addAll(ScalarisDataHandlerNormalised.normList2normStringList(normalisedPages));\n            } else {\n                MyWikiModel.denormalisePageTitles(normalisedPages, namespace, pages);\n            }\n            return pages;\n        } catch (SQLiteException e) {\n            System.err.println(\"read of pages in categories failed (sqlite error: \" + e.toString() + \")\");\n            throw new RuntimeException(e);\n        } finally {\n            if (db != null) {\n                db.dispose();\n            }\n        }\n    }\n    \n    /**\n     * Gets all sub-categories for the given ones from an SQLite database.\n     * \n     * Note: needs a (temporary) currentPages table to be set up before this\n     * call.\n     * \n     * @param allowedCats0\n     *            include all pages in these categories (un-normalised page\n     *            titles)\n     * @param allowedCats\n     *            include all pages in these categories (normalised page titles)\n     * @param db\n     *            connection to the SQLite database\n     * @param templateTree\n     *            information about the templates and their dependencies\n     * @param includeTree\n     *            information about page includes\n     * @param referenceTree\n     *            information about references to a page\n     * @param msgOut\n     *            the output stream to write status messages to\n     * \n     * @return the set of the given categories and all their sub-categories\n     *         (normalised)\n     * \n     * @throws SQLiteException\n     *             if an error occurs\n     */\n    private static Set<NormalisedTitle> getSubCategories(Collection<? extends String> allowedCats0,\n            Collection<? extends NormalisedTitle> allowedCats, SQLiteConnection db,\n            Map<NormalisedTitle, Set<NormalisedTitle>> templateTree,\n            Map<NormalisedTitle, Set<NormalisedTitle>> includeTree,\n            Map<NormalisedTitle, Set<NormalisedTitle>> referenceTree, PrintStream msgOut,\n            MyNamespace nsObject) throws SQLiteException {\n        Set<NormalisedTitle> allowedCatsFull = new HashSet<NormalisedTitle>();\n        Set<NormalisedTitle> currentPages = new HashSet<NormalisedTitle>();\n        Set<NormalisedTitle> newPages = new HashSet<NormalisedTitle>();\n        \n        if (!allowedCats.isEmpty()) {\n            SQLiteStatement stmt = null;\n            try {\n                // need to extend the category set by all sub-categories:\n                currentPages.addAll(allowedCats);\n\n                println(msgOut, \" determining sub-categories of \" + allowedCats0.toString() + \"\");\n                do {\n                    stmt = db.prepare(\"INSERT INTO currentPages (id) SELECT pages.id FROM pages WHERE pages.title == ?;\");\n                    for (NormalisedTitle pageTitle : currentPages) {\n                        addToPages(allowedCatsFull, newPages, pageTitle, includeTree, referenceTree);\n                        // beware: add pageTitle to allowedCatsFull _AFTER_ adding its dependencies\n                        // (otherwise the dependencies won't be added)\n                        allowedCatsFull.add(pageTitle);\n                        stmt.bind(1, pageTitle.toString()).stepThrough().reset();\n                    }\n                    stmt.dispose();\n\n                    println(msgOut, \"  adding sub-categories of \" + currentPages.size() + \" categories or templates\");\n                    // add all categories the page belongs to\n                    stmt = db\n                            .prepare(\"SELECT page.title FROM categories \" +\n                                    \"INNER JOIN currentPages AS cp ON categories.category == cp.id \" +\n                                    \"INNER JOIN pages AS page ON categories.title == page.id \" +\n                                    // \"INNER JOIN pages AS cat ON categories.category == cat.id\" +\n                                    \"WHERE page.title LIKE '\" + MyNamespace.NamespaceEnum.CATEGORY_NAMESPACE_KEY.getId() + \":%';\");\n                    while (stmt.step()) {\n                        NormalisedTitle pageCategory = NormalisedTitle.fromNormalised(stmt.columnString(0));\n                        addToPages(allowedCatsFull, newPages, pageCategory, includeTree, referenceTree);\n                    }\n                    stmt.dispose();\n                    println(msgOut, \"  adding sub-templates or sub-categories of \" + currentPages.size() + \" categories or templates\");\n                    // add all templates (and their requirements) of the pages\n                    stmt = db\n                            .prepare(\"SELECT page.title FROM templates \" +\n                                    \"INNER JOIN currentPages AS cp ON templates.template == cp.id \" +\n                                    \"INNER JOIN pages AS page ON templates.title == page.id \" +\n                                    // \"INNER JOIN pages AS tpl ON templates.template == tpl.id\" +\n                                    \"WHERE page.title LIKE '\" + MyNamespace.NamespaceEnum.CATEGORY_NAMESPACE_KEY.getId() + \":%' OR \"\n                                    + \"page.title LIKE '\" + MyNamespace.NamespaceEnum.TEMPLATE_NAMESPACE_KEY.getId() + \":%';\");\n                    while (stmt.step()) {\n                        NormalisedTitle pageTemplate = NormalisedTitle.fromNormalised(stmt.columnString(0));\n                        Set<NormalisedTitle> tplChildren = WikiDumpGetCategoryTreeHandler.getAllChildren(templateTree, pageTemplate);\n                        addToPages(allowedCatsFull, newPages, tplChildren, includeTree, referenceTree);\n                    }\n                    stmt.dispose();\n                    db.exec(\"DELETE FROM currentPages;\");\n                    if (newPages.isEmpty()) {\n                        break;\n                    } else {\n                        println(msgOut, \" adding \" + newPages.size() + \" dependencies\");\n                        currentPages = newPages;\n                        newPages = new HashSet<NormalisedTitle>();\n                    }\n                } while (true);\n            } finally {\n                if (stmt != null) {\n                    stmt.dispose();\n                }\n            }\n        }\n        return allowedCatsFull;\n    }\n    \n    /**\n     * Gets all pages in the given category set from an SQLite database. Note:\n     * sub-categories are not taken into account.\n     * \n     * @param allowedCats\n     *            include all pages in these categories (normalised page\n     *            titles)\n     * @param db\n     *            connection to the SQLite database\n     * \n     * @return a set of pages (directly) in the given categories (normalised\n     *         page titles)\n     * \n     * @throws SQLiteException\n     *             if an error occurs\n     */\n    private static Set<NormalisedTitle> getPagesDirectlyInCategories(Set<NormalisedTitle> allowedCats,\n            SQLiteConnection db) throws SQLiteException {\n        Set<NormalisedTitle> currentPages = new HashSet<NormalisedTitle>();\n\n        // note: allowedCatsFull can contain categories or templates\n        if (!allowedCats.isEmpty()) {\n            SQLiteStatement stmt = null;\n            try {\n                // select all pages belonging to any of the allowed categories:\n                stmt = db\n                        .prepare(\"SELECT page.title, cat.title FROM \" +\n                                \"categories INNER JOIN pages AS page ON categories.title == page.id \" +\n                                \"INNER JOIN pages AS cat ON categories.category == cat.id;\");\n                while (stmt.step()) {\n                    NormalisedTitle pageTitle = NormalisedTitle.fromNormalised(stmt.columnString(0));\n                    NormalisedTitle pageCategory = NormalisedTitle.fromNormalised(stmt.columnString(1));\n                    if (allowedCats.contains(pageCategory)) {\n                        currentPages.add(pageTitle);\n                    }\n                }\n                stmt.dispose();\n                // select all pages belonging to any of the allowed templates:\n                stmt = db\n                        .prepare(\"SELECT page.title, tpl.title FROM \" +\n                                \"templates INNER JOIN pages AS page ON templates.title == page.id \" +\n                                \"INNER JOIN pages AS tpl ON templates.template == tpl.id;\");\n                while (stmt.step()) {\n                    NormalisedTitle pageTitle = NormalisedTitle.fromNormalised(stmt.columnString(0));\n                    NormalisedTitle pageTemplate = NormalisedTitle.fromNormalised(stmt.columnString(1));\n                    if (allowedCats.contains(pageTemplate)) {\n                        currentPages.add(pageTitle);\n                    }\n                }\n            } finally {\n                if (stmt != null) {\n                    stmt.dispose();\n                }\n            }\n        }\n        return currentPages;\n    }\n    \n    /**\n     * Gets all pages and their dependencies from an SQLite database, follows\n     * links recursively.\n     * \n     * Note: needs a (temporary) currentPages table to be set up before this\n     * call.\n     * \n     * @param currentPages\n     *            parse these pages recursively (normalised page titles)\n     * @param depth\n     *            follow links this deep\n     * @param db\n     *            connection to the SQLite database\n     * @param templateTree\n     *            information about the templates and their dependencies\n     * @param includeTree\n     *            information about page includes\n     * @param referenceTree\n     *            information about references to a page\n     * @param msgOut\n     *            the output stream to write status messages to\n     * \n     * @return a set of normalised page titles\n     * \n     * @throws SQLiteException\n     *             if an error occurs\n     */\n    private static Set<NormalisedTitle> getRecursivePages(Set<NormalisedTitle> currentPages,\n            int depth, SQLiteConnection db, Map<NormalisedTitle, Set<NormalisedTitle>> templateTree,\n            Map<NormalisedTitle, Set<NormalisedTitle>> includeTree,\n            Map<NormalisedTitle, Set<NormalisedTitle>> referenceTree,\n            PrintStream msgOut)\n            throws SQLiteException {\n        Set<NormalisedTitle> allPages = new HashSet<NormalisedTitle>();\n        Set<NormalisedTitle> newPages = new HashSet<NormalisedTitle>();\n        Set<NormalisedTitle> pageLinks = new HashSet<NormalisedTitle>();\n        SQLiteStatement stmt = null;\n        final ArrayList<NormalisedTitle> autoIncluded = new ArrayList<NormalisedTitle>(10000);\n        try {\n            println(msgOut, \"adding all mediawiki pages\");\n            // add all auto-included pages\n            stmt = db.prepare(\"SELECT pages.title FROM pages WHERE pages.title LIKE '\"\n                              + MyNamespace.NamespaceEnum.MEDIAWIKI_NAMESPACE_KEY.getId()\n                              + \":%';\");\n            while (stmt.step()) {\n                autoIncluded.add(NormalisedTitle.fromNormalised(stmt.columnString(0)));\n            }\n            currentPages.addAll(autoIncluded);\n            stmt.dispose();\n            \n            while(depth >= 0) {\n                println(msgOut, \"recursion level: \" + depth);\n                println(msgOut, \" adding \" + currentPages.size() + \" pages\");\n                do {\n                    stmt = db.prepare(\"INSERT INTO currentPages (id) SELECT pages.id FROM pages WHERE pages.title == ?;\");\n                    for (NormalisedTitle pageTitle : currentPages) {\n                        addToPages(allPages, newPages, pageTitle, includeTree, referenceTree);\n                        // beware: add pageTitle to pages _AFTER_ adding its dependencies\n                        // (otherwise the dependencies won't be added)\n                        allPages.add(pageTitle);\n                        stmt.bind(1, pageTitle.toString()).stepThrough().reset();\n                    }\n                    stmt.dispose();\n\n                    println(msgOut, \"  adding categories of \" + currentPages.size() + \" pages\");\n                    // add all categories the page belongs to\n                    stmt = db\n                            .prepare(\"SELECT cat.title FROM categories \" +\n                                    \"INNER JOIN currentPages AS cp ON categories.title == cp.id \" +\n                                    // \"INNER JOIN pages AS page ON categories.title == page.id \" +\n                                    \"INNER JOIN pages AS cat ON categories.category == cat.id;\");\n                    while (stmt.step()) {\n                        NormalisedTitle pageCategory = NormalisedTitle.fromNormalised(stmt.columnString(0));\n                        addToPages(allPages, newPages, pageCategory, includeTree, referenceTree);\n                    }\n                    stmt.dispose();\n                    println(msgOut, \"  adding templates of \" + currentPages.size() + \" pages\");\n                    // add all templates (and their requirements) of the pages\n                    stmt = db\n                            .prepare(\"SELECT tpl.title FROM templates \" +\n                                    \"INNER JOIN currentPages AS cp ON templates.title == cp.id \" +\n                                    // \"INNER JOIN pages AS page ON templates.title == page.id \" +\n                                    \"INNER JOIN pages AS tpl ON templates.template == tpl.id;\");\n                    while (stmt.step()) {\n                        NormalisedTitle pageTemplate = NormalisedTitle.fromNormalised(stmt.columnString(0));\n                        Set<NormalisedTitle> tplChildren = WikiDumpGetCategoryTreeHandler.getAllChildren(templateTree, pageTemplate);\n                        addToPages(allPages, newPages, tplChildren, includeTree, referenceTree);\n                    }\n                    stmt.dispose();\n                    println(msgOut, \"  adding links of \" + currentPages.size() + \" pages\");\n                    // add all links of the pages for further processing\n                    stmt = db\n                            .prepare(\"SELECT lnk.title FROM links \" +\n                                    \"INNER JOIN currentPages AS cp ON links.title == cp.id \" +\n                                    // \"INNER JOIN pages AS page ON links.title == page.id \" +\n                                    \"INNER JOIN pages AS lnk ON links.link == lnk.id;\");\n                    while (stmt.step()) {\n                        String pageLink = stmt.columnString(0);\n                        if (!pageLink.isEmpty()) { // there may be empty links\n                            pageLinks.add(NormalisedTitle.fromNormalised(pageLink));\n                        }\n                    }\n                    stmt.dispose();\n                    db.exec(\"DELETE FROM currentPages;\");\n                    if (newPages.isEmpty()) {\n                        break;\n                    } else {\n                        println(msgOut, \" adding \" + newPages.size() + \" dependencies\");\n                        currentPages = newPages;\n                        newPages = new HashSet<NormalisedTitle>();\n                    }\n                } while (true);\n                // for the next recursion:\n                currentPages = pageLinks;\n                pageLinks = new HashSet<NormalisedTitle>();\n                --depth;\n            }\n        } finally {\n            if (stmt != null) {\n                stmt.dispose();\n            }\n        }\n        // these pages will be automatically included even if not in the page\n        // list -> do not include them for a smaller more readable page list\n        allPages.removeAll(autoIncluded);\n        return allPages;\n    }\n    \n    static protected void addToPages(Set<NormalisedTitle> pages, Set<NormalisedTitle> newPages,\n            NormalisedTitle title,\n            Map<NormalisedTitle, Set<NormalisedTitle>> includeTree,\n            Map<NormalisedTitle, Set<NormalisedTitle>> referenceTree) {\n        if (!pages.contains(title) && newPages.add(title)) {\n            // title not yet in pages -> add includes, redirects and pages redirecting to this page\n            addToPages(pages, newPages,\n                    WikiDumpGetCategoryTreeHandler.getAllChildren(includeTree,\n                            title), includeTree, referenceTree); // also has redirects\n            addToPages(pages, newPages,\n                    WikiDumpGetCategoryTreeHandler.getAllChildren(\n                            referenceTree, title), includeTree, referenceTree);\n        }\n    }\n    \n    static protected void addToPages(Set<NormalisedTitle> pages, Set<NormalisedTitle> newPages,\n            Collection<? extends NormalisedTitle> titles,\n            Map<NormalisedTitle, Set<NormalisedTitle>> includeTree,\n            Map<NormalisedTitle, Set<NormalisedTitle>> referenceTree) {\n        for (NormalisedTitle title : titles) {\n            addToPages(pages, newPages, title, includeTree, referenceTree);\n        }\n    }\n    \n    protected class SQLiteWorker extends Thread {\n        boolean stopWhenQueueEmpty = false;\n        boolean initialised = false;\n        \n        @Override\n        public void run() {\n            try {\n                // set up DB:\n                try {\n                    // set 1GB cache_size:\n                    db = SQLiteDataHandler.openDB(dbFileName, false, 1024l*1024l*1024l);\n                    db.exec(\"CREATE TABLE pages(id INTEGER PRIMARY KEY ASC, title STRING);\");\n                    db.exec(\"CREATE INDEX page_titles ON pages(title);\");\n                    db.exec(\"CREATE TABLE categories(title INTEGER, category INTEGER);\");\n                    db.exec(\"CREATE TABLE templates(title INTEGER, template INTEGER);\");\n                    db.exec(\"CREATE TABLE includes(title INTEGER, include INTEGER);\");\n                    db.exec(\"CREATE TABLE redirects(title INTEGER, redirect INTEGER);\");\n                    db.exec(\"CREATE TABLE links(title INTEGER, link INTEGER);\");\n                    db.exec(\"CREATE TABLE properties(key STRING PRIMARY KEY ASC, value);\");\n                    stGetPageId = db.prepare(\"SELECT id FROM pages WHERE title == ?;\");\n                    stWritePages = db.prepare(\"INSERT INTO pages (id, title) VALUES (?, ?);\");\n                    stWriteCategories = db.prepare(\"INSERT INTO categories (title, category) VALUES (?, ?);\");\n                    stWriteTemplates = db.prepare(\"INSERT INTO templates (title, template) VALUES (?, ?);\");\n                    stWriteIncludes = db.prepare(\"INSERT INTO includes (title, include) VALUES (?, ?);\");\n                    stWriteRedirects = db.prepare(\"INSERT INTO redirects (title, redirect) VALUES (?, ?);\");\n                    stWriteLinks = db.prepare(\"INSERT INTO links (title, link) VALUES (?, ?);\");\n                } catch (SQLiteException e) {\n                    throw new RuntimeException(e);\n                }\n                initialised = true;\n\n                // take jobs\n\n                while(!(sqliteJobs.isEmpty() && stopWhenQueueEmpty)) {\n                    SQLiteJob job;\n                    try {\n                        job = sqliteJobs.take();\n                    } catch (InterruptedException e) {\n                        throw new RuntimeException(e);\n                    }\n                    job.run();\n                }\n                try {\n                    db.exec(\"CREATE INDEX cat_titles ON categories(title);\");\n                    db.exec(\"CREATE INDEX tpl_titles ON templates(title);\");\n                    db.exec(\"CREATE INDEX incl_titles ON includes(title);\");\n                    db.exec(\"CREATE INDEX redir_titles ON redirects(title);\");\n                    db.exec(\"CREATE INDEX lnk_titles ON links(title);\");\n                } catch (SQLiteException e) {\n                    throw new RuntimeException(e);\n                }\n            } finally {\n                if (stGetPageId != null) {\n                    stGetPageId.dispose();\n                }\n                if (stWritePages != null) {\n                    stWritePages.dispose();\n                }\n                if (stWriteCategories != null) {\n                    stWriteCategories.dispose();\n                }\n                if (stWriteTemplates != null) {\n                    stWriteTemplates.dispose();\n                }\n                if (stWriteIncludes != null) {\n                    stWriteIncludes.dispose();\n                }\n                if (stWriteRedirects != null) {\n                    stWriteRedirects.dispose();\n                }\n                if (stWriteLinks != null) {\n                    stWriteLinks.dispose();\n                }\n                if (db != null) {\n                    db.dispose();\n                }\n                initialised = false;\n            }\n        }\n    }\n    \n    protected static interface SQLiteJob {\n        public abstract void run();\n    };\n    \n    protected static class SQLiteNoOpJob implements SQLiteJob {\n        @Override\n        public void run() {\n        }\n    }\n    \n    protected class SQLiteWriteValuesJob implements SQLiteJob {\n        SQLiteStatement stmt;\n        NormalisedTitle key;\n        Collection<? extends NormalisedTitle> values;\n        \n        public SQLiteWriteValuesJob(SQLiteStatement stmt, NormalisedTitle key, NormalisedTitle value) {\n            this.stmt = stmt;\n            this.key = key;\n            this.values = Arrays.asList(value);\n        }\n        \n        public SQLiteWriteValuesJob(SQLiteStatement stmt, NormalisedTitle key, Collection<? extends NormalisedTitle> values) {\n            this.stmt = stmt;\n            this.key = key;\n            this.values = values;\n        }\n\n        protected long pageToId(NormalisedTitle pageTitle) throws RuntimeException {\n            String pageTitle2 = pageTitle.toString();\n            try {\n                long pageId = -1;\n                // try to find the page id in the pages table:\n                try {\n                    stGetPageId.bind(1, pageTitle2);\n                    if (stGetPageId.step()) {\n                        pageId = stGetPageId.columnLong(0);\n                    }\n                } finally {\n                    stGetPageId.reset();\n                }\n                // page not found yet -> add to pages table:\n                if (pageId == -1) {\n                    pageId = nextPageId++;\n                    try {\n                        stWritePages.bind(1, pageId).bind(2, pageTitle2).stepThrough();\n                    } finally {\n                        stWritePages.reset();\n                    }\n                }\n                return pageId;\n            } catch (SQLiteException e) {\n                error(\"write of \" + pageTitle2 + \" failed (sqlite error: \" + e.toString() + \")\");\n                throw new RuntimeException(e);\n            }\n        }\n        \n        @Override\n        public void run() {\n            long key_id = pageToId(key);\n            ArrayList<Long> values_id = new ArrayList<Long>(values.size());\n            for (NormalisedTitle value : values) {\n                values_id.add(pageToId(value));\n            }\n            try {\n                try {\n                    stmt.bind(1, key_id);\n                    for (Long value_id : values_id) {\n                        stmt.bind(2, value_id).stepThrough().reset(false);\n                    }\n                } finally {\n                    stmt.reset();\n                }\n            } catch (SQLiteException e) {\n                error(\"write of \" + key + \" failed (sqlite error: \" + e.toString() + \")\");\n                throw new RuntimeException(e);\n            }\n        }\n    }\n    \n    protected class SQLiteWriteSiteInfoJob implements SQLiteJob {\n        SiteInfo siteInfo;\n        \n        public SQLiteWriteSiteInfoJob(SiteInfo siteInfo) {\n            this.siteInfo = siteInfo;\n        }\n        \n        @Override\n        public void run() {\n            SQLiteStatement stmt = null;\n            try {\n                stmt = db.prepare(\"REPLACE INTO properties (key, value) VALUES (?, ?);\");\n                WikiDumpPrepareSQLiteForScalarisHandler.writeObject(stmt, \"siteinfo\", siteInfo);\n            } catch (SQLiteException e) {\n                throw new RuntimeException(e);\n            } finally {\n                if (stmt != null) {\n                    stmt.dispose();\n                }\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/WikiDumpHandler.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport java.io.PrintStream;\nimport java.text.NumberFormat;\nimport java.util.Calendar;\nimport java.util.Date;\nimport java.util.HashSet;\nimport java.util.Locale;\nimport java.util.Set;\n\nimport org.xml.sax.Attributes;\nimport org.xml.sax.SAXException;\nimport org.xml.sax.helpers.DefaultHandler;\n\nimport de.zib.scalaris.examples.wikipedia.bliki.MyNamespace;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyParsingWikiModel;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\nimport de.zib.scalaris.examples.wikipedia.data.xml.XmlPage.CheckSkipRevisions;\n\n/**\n * Provides abilities to read an xml wiki dump file and write its contents to\n * the standard output.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic abstract class WikiDumpHandler extends DefaultHandler implements WikiDump {\n    private boolean inSiteInfo = false;\n    private boolean inPage = false;\n\n    private XmlSiteInfo currentSiteInfo = null;\n    private XmlPage currentPage;\n\n    final private Set<String> blacklist0;\n    final private Set<String> whitelist0;\n    private Set<NormalisedTitle> blacklist = null;\n    private Set<NormalisedTitle> whitelist = null;\n    \n    protected MyParsingWikiModel wikiModel;\n    \n    /**\n     * The time at the start of an import operation.\n     */\n    private long timeAtStart = 0;\n    /**\n     * The time at the end of an import operation.\n     */\n    private long timeAtEnd = 0;\n    /**\n     * The number of (successfully) processed pages.\n     */\n    protected int pageCount = 0;\n    \n    protected PrintStream msgOut = System.out;\n    \n    protected boolean stop = false;\n    \n    protected boolean errorDuringImport = false;\n\n    /**\n     * Sets up a SAX XmlHandler exporting all parsed pages except the ones in a\n     * blacklist.\n     * \n     * If a whitelist is given, a skip revision handler is set which ignores\n     * pages not in the whitelist. See\n     * {@link #setPageCheckSkipRevisions(CheckSkipRevisions)}.\n     * \n     * @param blacklist\n     *            a number of page titles to ignore (un-normalised page titles),\n     *            may be <tt>null</tt>\n     * @param whitelist\n     *            only import these pages (un-normalised page titles), if\n     *            <tt>null</tt>, import all pages\n     * @param maxRevisions\n     *            maximum number of revisions per page (starting with the most\n     *            recent) - <tt>-1/tt> imports all revisions\n     *            (useful to speed up the import / reduce the DB size)\n     * @param minTime\n     *            minimum time a revision should have (only one revision older\n     *            than this will be imported) - <tt>null/tt> imports all\n     *            revisions\n     * @param maxTime\n     *            maximum time a revision should have (newer revisions are\n     *            omitted) - <tt>null/tt> imports all revisions (useful to\n     *            create dumps of a wiki at a specific point in time)\n     */\n    public WikiDumpHandler(Set<String> blacklist, Set<String> whitelist, int maxRevisions, Calendar minTime, Calendar maxTime) {\n        this.blacklist0 = blacklist;\n        this.whitelist0 = whitelist;\n        currentPage = new XmlPage(maxRevisions, minTime, maxTime);\n        // if a whitelist is given, do not render any other page:\n        if (whitelist != null) {\n            currentPage.setCheckSkipRevisions(new CheckSkipRevisions() {\n                @Override\n                public boolean skipRevisions(String pageTitle) {\n                    return !inWhiteList(wikiModel.normalisePageTitle(pageTitle));\n                }\n            });\n        }\n    }\n    \n    /**\n     * Checks whether the given page title is in the whitelist.\n     * \n     * @param pageTitle\n     *            the title of a page\n     * \n     * @return whether the page should be imported or not\n     */\n    private boolean inWhiteList(NormalisedTitle pageTitle) {\n        return whitelist == null\n                || pageTitle.namespace.equals(MyNamespace.MEDIAWIKI_NAMESPACE_KEY)\n                || whitelist.contains(pageTitle);\n    }\n    \n    /**\n     * Checks whether the given page title is in the blacklist.\n     * \n     * @param pageTitle\n     *            the title of a page\n     * \n     * @return whether the page should be skipped or not\n     */\n    private boolean inBlackList(NormalisedTitle pageTitle) {\n        return blacklist != null && blacklist.contains(pageTitle);\n    }\n\n    /**\n     * Called to when a starting element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     * @param attributes\n     *            The attributes attached to the element. If there are no\n     *            attributes, it shall be an empty Attributes object.\n     */\n    @Override\n    public void startElement(String uri, String localName, String qName,\n            Attributes attributes) throws SAXException {\n        if (stop) {\n            throw new SAXParsingInterruptedException();\n        }\n        // out.println(localName);\n\n        /*\n         * <siteinfo> <sitename>Wikipedia</sitename>\n         * <base>http://bar.wikipedia.org/wiki/Hauptseitn</base>\n         * <generator>MediaWiki 1.11alpha</generator> <case>first-letter</case>\n         * <namespaces> <namespace key=\"-2\">Media</namespace> ... </namespaces>\n         * </siteinfo> <page></page> ...\n         */\n        if (localName.equals(\"mediawiki\")) {\n            importStart();\n        } else if (localName.equals(\"siteinfo\")) {\n            inSiteInfo = true;\n            currentSiteInfo = new XmlSiteInfo();\n            currentSiteInfo.startSiteInfo(uri, localName, qName, attributes);\n        } else if (localName.equals(\"page\")) {\n            inPage = true;\n            currentPage.reset();\n            currentPage.startPage(uri, localName, qName, attributes);\n        } else if (inSiteInfo) {\n            currentSiteInfo.startElement(uri, localName, qName, attributes);\n        } else if (inPage) {\n            currentPage.startElement(uri, localName, qName, attributes);\n        }\n    }\n\n    /**\n     * Called to process character data.\n     * \n     * Note: a SAX driver is free to chunk the character data any way it wants,\n     * so you cannot count on all of the character data content of an element\n     * arriving in a single characters event.\n     * \n     * @param ch\n     *            The characters.\n     * @param start\n     *            The start position in the character array.\n     * @param length\n     *            The number of characters to use from the character array.\n     */\n    @Override\n    public void characters(char[] ch, int start, int length)\n            throws SAXException {\n        // out.println(new String(ch, start, length));\n        if (inSiteInfo) {\n            currentSiteInfo.characters(ch, start, length);\n        }\n        if (inPage) {\n            currentPage.characters(ch, start, length);\n        }\n    }\n\n    /**\n     * Called to when an ending element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     */\n    @Override\n    public void endElement(String uri, String localName, String qName)\n            throws SAXException {\n        if (inSiteInfo) {\n            if (localName.equals(\"siteinfo\")) {\n                inSiteInfo = false;\n                currentSiteInfo.endSiteInfo(uri, localName, qName);\n                setUpWikiModel(currentSiteInfo.getSiteInfo());\n                export(currentSiteInfo);\n                currentSiteInfo = null;\n            } else {\n                currentSiteInfo.endElement(uri, localName, qName);\n            }\n        } else if (inPage) {\n            if (localName.equals(\"page\")) {\n                inPage = false;\n                currentPage.endPage(uri, localName, qName);\n                if (currentPage.getPage() != null) {\n                    final NormalisedTitle normTitle = wikiModel\n                            .normalisePageTitle(currentPage.getPage().getTitle());\n                    if (!inBlackList(normTitle) && inWhiteList(normTitle)) {\n                        export(currentPage);\n                    }\n                }\n                currentPage.reset();\n            } else {\n                currentPage.endElement(uri, localName, qName);\n            }\n        } else if (localName.equals(\"mediawiki\")) {\n            importEnd();\n        }\n    }\n    \n    private void setUpWikiModel(SiteInfo siteinfo) {\n        wikiModel = new MyParsingWikiModel(\"\", \"\", new MyNamespace(siteinfo));\n        // we are now able to normalise the page titles in the whitelist:\n        if (whitelist0 != null) {\n            this.whitelist = new HashSet<NormalisedTitle>(whitelist0.size());\n            wikiModel.normalisePageTitles(whitelist0, whitelist);\n            // we don't need the original page titles anymore:\n            this.whitelist0.clear();\n        }\n        if (blacklist0 != null) {\n            this.blacklist = new HashSet<NormalisedTitle>(blacklist0.size());\n            wikiModel.normalisePageTitles(blacklist0, blacklist);\n            // we don't need the original page titles anymore:\n            this.blacklist0.clear();\n        }\n    }\n\n    /**\n     * Exports the given siteinfo.\n     * \n     * @param siteinfo\n     *            the siteinfo to export\n     */\n    protected abstract void export(XmlSiteInfo siteinfo);\n\n    /**\n     * Exports the given page (including all revisions).\n     * \n     * @param page\n     *            the page to export\n     */\n    protected abstract void export(XmlPage page);\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#setUp()\n     */\n    @Override\n    public void setUp() {\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#tearDown()\n     */\n    @Override\n    public void tearDown() {\n    }\n\n    /**\n     * Sets the time the import started.\n     */\n    final protected void importStart() {\n        timeAtStart = System.currentTimeMillis();\n    }\n\n    /**\n     * Sets the time the import finished.\n     */\n    final protected void importEnd() {\n        timeAtEnd = System.currentTimeMillis();\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#getTimeAtStart()\n     */\n    @Override\n    public long getTimeAtStart() {\n        return timeAtStart;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#getTimeAtEnd()\n     */\n    @Override\n    public long getTimeAtEnd() {\n        return timeAtEnd;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#getPageCount()\n     */\n    @Override\n    public int getImportCount() {\n        return pageCount;\n    }\n\n    @Override\n    public boolean isErrorDuringImport() {\n        return errorDuringImport;\n    }\n\n    @Override\n    public void error(String message) {\n        System.err.println(message);\n        errorDuringImport = true;\n    }\n\n    /**\n     * Reports the speed of the import (pages/s) and may be used as a shutdown\n     * handler.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public class ReportAtShutDown extends Thread {\n        @Override\n        public void run() {\n            reportAtEnd();\n        }\n\n        /**\n         * Sets the import end time and reports the overall speed.\n         */\n        public void reportAtEnd() {\n            // import may have been interrupted - get an end time in this case\n            if (timeAtEnd == 0) {\n                importEnd();\n            }\n            final long timeTaken = timeAtEnd - timeAtStart;\n            final double speed = (((double) pageCount) * 1000) / timeTaken;\n            NumberFormat nf = NumberFormat.getNumberInstance(Locale.ENGLISH);\n            nf.setGroupingUsed(true);\n            println(\"Finished import (\" + nf.format(speed) + \" pages/s)\");\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#setMsgOut(java.io.PrintStream)\n     */\n    @Override\n    public void setMsgOut(PrintStream msgOut) {\n        this.msgOut = msgOut;\n    }\n\n    /**\n     * Prints a message to the chosen output stream (includes a timestamp).\n     * \n     * @param message\n     *            the message to print\n     * \n     * @see #setMsgOut(PrintStream)\n     */\n    public void print(String message) {\n        print(msgOut, message);\n    }\n\n    /**\n     * Prints a message to the chosen output stream (includes a timestamp).\n     * Includes a newline character at the end.\n     * \n     * @param message\n     *            the message to print\n     * \n     * @see #setMsgOut(PrintStream)\n     */\n    public void println(String message) {\n        println(msgOut, message);\n    }\n\n    /**\n     * Prints a message to the chosen output stream (includes a timestamp).\n     * \n     * @param msgOut\n     *            the output stream to write to\n     * @param message\n     *            the message to print\n     * \n     * @see #setMsgOut(PrintStream)\n     */\n    static public void print(PrintStream msgOut, String message) {\n        msgOut.print(\"[\" + (new Date()).toString() + \"] \" + message);\n    }\n\n    /**\n     * Prints a message to the chosen output stream (includes a timestamp).\n     * Includes a newline character at the end.\n     * \n     * @param msgOut\n     *            the output stream to write to\n     * @param message\n     *            the message to print\n     * \n     * @see #setMsgOut(PrintStream)\n     */\n    static public void println(PrintStream msgOut, String message) {\n        msgOut.println(\"[\" + (new Date()).toString() + \"] \" + message);\n    }\n\n    /**\n     * @param checkSkipRevisions the checkSkipRevisions to set\n     */\n    protected void setPageCheckSkipRevisions(CheckSkipRevisions checkSkipRevisions) {\n        this.currentPage.setCheckSkipRevisions(checkSkipRevisions);\n    }\n\n    /**\n     * Whether {@link #stopParsing()} is supported or not.\n     * \n     * @return stop parsing support\n     */\n    @Override\n    public boolean hasStopSupport() {\n        return true;\n    }\n\n    /**\n     * Tells the parser to stop at the next starting element.\n     */\n    @Override\n    public void stopParsing() {\n        this.stop = true;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/WikiDumpPageHandler.java",
    "content": "package de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport java.util.ArrayList;\nimport java.util.Calendar;\nimport java.util.Collections;\nimport java.util.EnumMap;\nimport java.util.List;\nimport java.util.Set;\n\nimport de.zib.scalaris.examples.wikipedia.bliki.MyNamespace;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyNamespace.NamespaceEnum;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyWikiModel;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\nimport de.zib.scalaris.examples.wikipedia.data.ShortRevision;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\nimport de.zib.tools.MultiHashMap;\n\n/**\n * Intermediate class between XML processing of a Wiki dump and custom\n * exporting of read {@link SiteInfo} and {@link Page} objects, including all\n * revisions and the list of short revisions.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic abstract class WikiDumpPageHandler extends WikiDumpHandler {\n    protected static final int UPDATE_PAGELIST_EVERY = 500;\n    protected static final int NEW_CATS_HASH_DEF_SIZE = 100;\n    protected static final int NEW_TPLS_HASH_DEF_SIZE = 100;\n    protected static final int NEW_BLNKS_HASH_DEF_SIZE = 100;\n\n    protected EnumMap<NamespaceEnum, ArrayList<NormalisedTitle>> newPages;\n    protected int articleCount = 0;\n    protected MultiHashMap<NormalisedTitle, NormalisedTitle> newCategories;\n    protected MultiHashMap<NormalisedTitle, NormalisedTitle> newTemplates;\n    protected MultiHashMap<NormalisedTitle, NormalisedTitle> newBackLinks;\n\n    /**\n     * Sets up a SAX XmlHandler exporting all parsed pages except the ones in a\n     * blacklist.\n     * \n     * @param blacklist\n     *            a number of page titles to ignore\n     * @param whitelist\n     *            only import these pages\n     * @param maxRevisions\n     *            maximum number of revisions per page (starting with the most\n     *            recent) - <tt>-1/tt> imports all revisions\n     *            (useful to speed up the import / reduce the DB size)\n     * @param minTime\n     *            minimum time a revision should have (only one revision older\n     *            than this will be imported) - <tt>null/tt> imports all\n     *            revisions\n     * @param maxTime\n     *            maximum time a revision should have (newer revisions are\n     *            omitted) - <tt>null/tt> imports all revisions\n     *            (useful to create dumps of a wiki at a specific point in time)\n     */\n    public WikiDumpPageHandler(Set<String> blacklist, Set<String> whitelist,\n            int maxRevisions, Calendar minTime, Calendar maxTime) {\n        super(blacklist, whitelist, maxRevisions, minTime, maxTime);\n        initNewPagesList();\n        initLinkLists();\n    }\n\n    /**\n     * Initialises the {@link #newPages} member.\n     */\n    protected void initNewPagesList() {\n        newPages = createNewPagesList();\n    }\n\n    /**\n     * Creates a new EnumMap similar to the {@link #newPages} member.\n     * \n     * @return a map with wiki namespace keys and lists of strings as values\n     */\n    public static EnumMap<NamespaceEnum, ArrayList<NormalisedTitle>> createNewPagesList() {\n        EnumMap<NamespaceEnum, ArrayList<NormalisedTitle>> result =\n                new EnumMap<NamespaceEnum, ArrayList<NormalisedTitle>>(NamespaceEnum.class);\n        for(NamespaceEnum ns : NamespaceEnum.values()) {\n            result.put(ns, new ArrayList<NormalisedTitle>(UPDATE_PAGELIST_EVERY));\n        }\n        return result;\n    }\n\n    /**\n     * Initialises the {@link #newCategories}, {@link #newTemplates} and\n     * {@link #newBackLinks} members.\n     */\n    protected void initLinkLists() {\n        newCategories = new MultiHashMap<NormalisedTitle, NormalisedTitle>(NEW_CATS_HASH_DEF_SIZE);\n        newTemplates = new MultiHashMap<NormalisedTitle, NormalisedTitle>(NEW_TPLS_HASH_DEF_SIZE);\n        newBackLinks = new MultiHashMap<NormalisedTitle, NormalisedTitle>(NEW_BLNKS_HASH_DEF_SIZE);\n    }\n\n    /**\n     * Exports the given siteinfo to Scalaris\n     * \n     * @param revisions\n     *            the siteinfo to export\n     */\n    @Override\n    protected void export(XmlSiteInfo siteinfo_xml) {\n        doExport(siteinfo_xml.getSiteInfo());\n    }\n\n    /**\n     * Exports the given page (including all revisions) to Scalaris\n     * \n     * @param page_xml\n     *            the page object extracted from XML\n     */\n    @Override\n    protected void export(XmlPage page_xml) {\n        Page page = page_xml.getPage();\n        ++pageCount;\n        \n        if (page.getCurRev() != null) {\n            List<Revision> revisions = page_xml.getRevisions();\n            List<ShortRevision> revisions_short = ShortRevision.fromRevisions(revisions);\n            Collections.sort(revisions, Collections.reverseOrder(new byRevId()));\n            Collections.sort(revisions_short, Collections.reverseOrder(new byShortRevId()));\n\n            assert(wikiModel != null);\n            final NormalisedTitle normTitle = wikiModel.normalisePageTitle(page.getTitle());\n            if (!revisions.isEmpty()) {\n                wikiModel.setUp();\n                wikiModel.setNamespaceName(wikiModel.getNamespace().getNamespaceByNumber(normTitle.namespace));\n                wikiModel.setPageName(normTitle.title);\n                wikiModel.renderPageWithCache(null, revisions.get(0).unpackedText());\n                for (String cat_raw: wikiModel.getCategories().keySet()) {\n                    NormalisedTitle category = new NormalisedTitle(\n                            MyNamespace.CATEGORY_NAMESPACE_KEY,\n                            MyWikiModel.normaliseName(cat_raw));\n                    newCategories.put1(category, normTitle);\n                }\n                for (String tpl_raw: wikiModel.getTemplatesNoMagicWords()) {\n                    NormalisedTitle template = new NormalisedTitle(\n                            MyNamespace.TEMPLATE_NAMESPACE_KEY,\n                            MyWikiModel.normaliseName(tpl_raw));\n                    newTemplates.put1(template, normTitle);\n                }\n                for (String link: wikiModel.getLinks()) {\n                    newBackLinks.put1(wikiModel.normalisePageTitle(link),\n                            normTitle);\n                }\n                if (MyWikiModel.isArticle(normTitle.namespace, wikiModel\n                        .getLinks(), wikiModel.getCategories().keySet())) {\n                    ++articleCount;\n                }\n                wikiModel.tearDown();\n            }\n    \n            doExport(page, revisions, revisions_short, normTitle);\n        }\n        if ((pageCount % UPDATE_PAGELIST_EVERY) == 0) {\n            println(\"processed pages: \" + pageCount);\n        }\n    }\n\n    abstract protected void doExport(SiteInfo siteInfo);\n\n    abstract protected void doExport(Page page, List<Revision> revisions,\n            List<ShortRevision> revisions_short, NormalisedTitle title);\n\n    /**\n     * Provides a comparator for sorting {@link Revision} objects by their IDs.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    protected static class byRevId implements java.util.Comparator<Revision> {\n        /**\n         * Compares its two arguments for order. Returns a negative integer,\n         * zero, or a positive integer as the first argument is less than, equal\n         * to, or greater than the second.\n         * \n         * @param rev1\n         *            the first revision to be compared.\n         * @param rev2\n         *            the second revision to be compared.\n         * \n         * @return a negative integer, zero, or a positive integer as the first\n         *         argument is less than, equal to, or greater than the second.\n         */\n        @Override\n        public int compare(Revision rev1, Revision rev2) {\n            return (rev1.getId() - rev2.getId());\n        }\n    }\n\n    /**\n     * Provides a comparator for sorting {@link Revision} objects by their IDs.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    protected static class byShortRevId implements java.util.Comparator<ShortRevision> {\n        /**\n         * Compares its two arguments for order. Returns a negative integer,\n         * zero, or a positive integer as the first argument is less than, equal\n         * to, or greater than the second.\n         * \n         * @param rev1\n         *            the first revision to be compared.\n         * @param rev2\n         *            the second revision to be compared.\n         * \n         * @return a negative integer, zero, or a positive integer as the first\n         *         argument is less than, equal to, or greater than the second.\n         */\n        @Override\n        public int compare(ShortRevision rev1, ShortRevision rev2) {\n            return (rev1.getId() - rev2.getId());\n        }\n    }\n}"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/WikiDumpPrepareSQLiteForScalarisHandler.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport java.io.FileNotFoundException;\nimport java.io.IOException;\nimport java.util.ArrayList;\nimport java.util.Calendar;\nimport java.util.Collection;\nimport java.util.EnumMap;\nimport java.util.List;\nimport java.util.Map.Entry;\nimport java.util.Set;\nimport java.util.concurrent.ArrayBlockingQueue;\n\nimport com.almworks.sqlite4java.SQLiteConnection;\nimport com.almworks.sqlite4java.SQLiteException;\nimport com.almworks.sqlite4java.SQLiteStatement;\nimport com.ericsson.otp.erlang.OtpErlangDecodeException;\nimport com.ericsson.otp.erlang.OtpErlangObject;\n\nimport de.zib.scalaris.CommonErlangObjects;\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.examples.wikipedia.SQLiteDataHandler;\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandler;\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandlerNormalised;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyNamespace.NamespaceEnum;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\nimport de.zib.scalaris.examples.wikipedia.data.ShortRevision;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\nimport de.zib.tools.MultiHashMap;\n\n/**\n * Provides abilities to read an xml wiki dump file and prepare its contents\n * for Scalaris by creating key/value pairs in a local SQLite db.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class WikiDumpPrepareSQLiteForScalarisHandler extends WikiDumpPageHandler {\n    protected SQLiteConnection db = null;\n    protected SQLiteStatement stGetTmpPageId = null;\n    protected SQLiteStatement stWriteTmpPages = null;\n    protected SQLiteStatement stWriteTmpCategories = null;\n    protected SQLiteStatement stWriteTmpTemplates = null;\n    protected SQLiteStatement stWriteTmpLinks = null;\n    protected SQLiteStatement stRead = null;\n    protected SQLiteStatement stWrite = null;\n    protected long nextPageId = 0l;\n    protected String dbFileName;\n    protected ArrayBlockingQueue<Runnable> sqliteJobs = new ArrayBlockingQueue<Runnable>(UPDATE_PAGELIST_EVERY);\n    SQLiteWorker sqliteWorker = new SQLiteWorker();\n    \n    /**\n     * Sets up a SAX XmlHandler exporting all parsed pages except the ones in a\n     * blacklist to Scalaris but with an additional pre-process phase.\n     * \n     * @param blacklist\n     *            a number of page titles to ignore\n     * @param whitelist\n     *            only import these pages\n     * @param maxRevisions\n     *            maximum number of revisions per page (starting with the most\n     *            recent) - <tt>-1/tt> imports all revisions\n     *            (useful to speed up the import / reduce the DB size)\n     * @param minTime\n     *            minimum time a revision should have (only one revision older\n     *            than this will be imported) - <tt>null/tt> imports all\n     *            revisions\n     * @param maxTime\n     *            maximum time a revision should have (newer revisions are\n     *            omitted) - <tt>null/tt> imports all revisions\n     *            (useful to create dumps of a wiki at a specific point in time)\n     * @param dbFileName\n     *            the name of the database file to write to\n     * \n     * @throws RuntimeException\n     *             if the connection to Scalaris fails\n     */\n    public WikiDumpPrepareSQLiteForScalarisHandler(Set<String> blacklist,\n            Set<String> whitelist, int maxRevisions, Calendar minTime,\n            Calendar maxTime, String dbFileName) throws RuntimeException {\n        super(blacklist, whitelist, maxRevisions, minTime, maxTime);\n        this.dbFileName = dbFileName;\n    }\n\n    static SQLiteStatement createReadStmt(SQLiteConnection db) throws SQLiteException {\n        return db.prepare(\"SELECT scalaris_value FROM objects WHERE scalaris_key == ?\");\n    }\n\n    static SQLiteStatement createWriteStmt(SQLiteConnection db) throws SQLiteException {\n        return db.prepare(\"REPLACE INTO objects (scalaris_key, scalaris_value) VALUES (?, ?);\");\n    }\n\n    /**\n     * @param <T>\n     * @param siteinfo\n     * @param key\n     * @throws IOException\n     * @throws FileNotFoundException\n     */\n    static <T> void writeObject(SQLiteStatement stWrite, String key, T value)\n            throws RuntimeException {\n        WikiDumpXml2SQLite.writeObject(stWrite, key,\n                CommonErlangObjects.encode(ErlangValue.convertToErlang(value)));\n    }\n\n    /**\n     * Encodes the given object to the erlang value used by Scalaris and\n     * converts this value to bytes for use by SQLite.\n     * \n     * @param value\n     *            the object to convert\n     * \n     * @return the byte array\n     * \n     * @throws IOException\n     * \n     * @see {@link #objectFromBytes(byte[])}\n     */\n    static <T> byte[] objectToBytes(T value) throws IOException {\n        return WikiDumpXml2SQLite.objectToBytes(CommonErlangObjects\n                .encode(ErlangValue.convertToErlang(value)));\n    }\n\n    /**\n     * Reads an encoded Scalaris value from a (compressed) byte array.\n     * \n     * @param value  the byte array to get the object from\n     * \n     * @return the encoded object\n     * \n     * @throws IOException\n     * @throws ClassNotFoundException\n     * \n     * @see #objectToBytes(Object)\n     */\n    static OtpErlangObject objectFromBytes(byte[] value) throws IOException,\n            ClassNotFoundException {\n        return WikiDumpXml2SQLite.<OtpErlangObject>objectFromBytes(value);\n    }\n\n    /**\n     * Reads a compressed and encoded Scalaris value from a (compressed) byte array.\n     * \n     * @param value  the byte array to get the object from\n     * \n     * @return the decoded object\n     * \n     * @throws IOException\n     * @throws ClassNotFoundException\n     * \n     * @see #objectToBytes(Object)\n     */\n    static ErlangValue objectFromBytes2(byte[] value) throws IOException,\n            ClassNotFoundException {\n        OtpErlangObject value2;\n        try {\n            value2 = CommonErlangObjects.decode(objectFromBytes(value));\n        } catch (OtpErlangDecodeException e) {\n            throw new RuntimeException(e);\n        }\n        return new ErlangValue(value2);\n    }\n\n    /**\n     * Reads an Erlang-encoded, byte-encoded object from a SQLite DB.\n     * \n     * @param stRead\n     *            read statement\n     * @param key\n     *            key to read from\n     * \n     * @return the Erlang-encoded object\n     * \n     * @throws IOException\n     * @throws FileNotFoundException\n     */\n    static OtpErlangObject readObject(SQLiteStatement stRead, String key)\n            throws RuntimeException, FileNotFoundException {\n        return WikiDumpXml2SQLite.<OtpErlangObject>readObject(stRead, key);\n    }\n\n    /**\n     * Reads an Erlang-encoded, byte-encoded object from a SQLite DB.\n     * \n     * @param stRead\n     *            read statement\n     * @param key\n     *            key to read from\n     * \n     * @return the decoded object\n     * \n     * @throws IOException\n     * @throws FileNotFoundException\n     */\n    static ErlangValue readObject2(SQLiteStatement stRead, String key)\n            throws RuntimeException, FileNotFoundException {\n        try {\n            OtpErlangObject value = CommonErlangObjects.decode(readObject(\n                    stRead, key));\n            return new ErlangValue(value);\n        } catch (FileNotFoundException e) {\n            throw e;\n        } catch (OtpErlangDecodeException e) {\n            throw new RuntimeException(e);\n        }\n    }\n    \n    protected void addSQLiteJob(Runnable job) throws RuntimeException {\n        try {\n            sqliteJobs.put(job);\n        } catch (InterruptedException e) {\n            throw new RuntimeException(e);\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpPrepareForScalarisHandler#setUp()\n     */\n    @Override\n    public void setUp() {\n        super.setUp();\n        println(\"Pre-processing pages to key/value pairs...\");\n        sqliteWorker.start();\n        // wait for worker to initialise the DB and the prepared statements\n        while (!sqliteWorker.initialised) {\n            try {\n                Thread.sleep(100);\n            } catch (InterruptedException e) {\n                throw new RuntimeException(e);\n            }\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpHandler#tearDown()\n     */\n    @Override\n    public void tearDown() {\n        super.tearDown();\n        updatePageList();\n        updateLinkLists();\n        addSQLiteJob(new SQLiteConvertTmpPageLists2Job());\n        sqliteWorker.stopWhenQueueEmpty = true;\n        addSQLiteJob(new SQLiteNoOpJob());\n        // wait for worker to close the DB\n        try {\n            sqliteWorker.join();\n        } catch (InterruptedException e) {\n            throw new RuntimeException(e);\n        }\n        importEnd();\n    }\n\n    /* (non-Javadoc)\n     * @see java.lang.Object#finalize()\n     */\n    @Override\n    protected void finalize() throws Throwable {\n        super.finalize();\n        db.dispose();\n    }\n\n    @Override\n    protected void doExport(SiteInfo siteinfo) throws RuntimeException {\n        addSQLiteJob(new SQLiteWriteSiteInfoJob(siteinfo, stWrite));\n    }\n\n    @Override\n    protected void doExport(Page page, List<Revision> revisions,\n            List<ShortRevision> revisions_short, NormalisedTitle title)\n            throws UnsupportedOperationException {\n        for (Revision rev : revisions) {\n            if (rev.getId() != page.getCurRev().getId()) {\n                addSQLiteJob(new SQLiteWriteObjectJob<Revision>(\n                        ScalarisDataHandlerNormalised.getRevKey(title,\n                                rev.getId()), rev, stWrite));\n            }\n        }\n        addSQLiteJob(new SQLiteWriteObjectJob<List<ShortRevision>>(\n                ScalarisDataHandlerNormalised.getRevListKey(title),\n                revisions_short, stWrite));\n        addSQLiteJob(new SQLiteWriteObjectJob<Page>(\n                ScalarisDataHandlerNormalised.getPageKey(title), page, stWrite));\n\n        // note: do not normalise page titles (this will be done later)\n        newPages.get(NamespaceEnum.fromId(title.namespace)).add(title);\n        // only export page list every UPDATE_PAGELIST_EVERY pages:\n        if ((pageCount % UPDATE_PAGELIST_EVERY) == 0) {\n            updatePageList();\n        }\n        // limit the number of changes in SQLiteUpdatePageLists2Job to\n        // UPDATE_PAGELIST_EVERY\n        if ((newCategories.size() + newTemplates.size() + newBackLinks.size()) >= UPDATE_PAGELIST_EVERY) {\n            updateLinkLists();\n        }\n    }\n    \n    protected void updatePageList() {\n        addSQLiteJob(new SQLiteUpdatePageLists1Job(newPages, articleCount));\n        initNewPagesList();\n    }\n    \n    protected void updateLinkLists() {\n        addSQLiteJob(new SQLiteUpdateTmpPageLists2Job(newCategories, newTemplates, newBackLinks));\n        initLinkLists();\n    }\n    \n    protected class SQLiteWorker extends Thread {\n        boolean stopWhenQueueEmpty = false;\n        boolean initialised = false;\n        \n        @Override\n        public void run() {\n            try {\n                // set up DB:\n                try {\n                    // set 1GB cache_size:\n                    db = SQLiteDataHandler.openDB(dbFileName, false, 1024l*1024l*1024l);\n                    db.exec(\"CREATE TABLE objects(scalaris_key STRING PRIMARY KEY ASC, scalaris_value);\");\n                    db.exec(\"CREATE TEMPORARY TABLE pages(id INTEGER PRIMARY KEY ASC, title STRING);\");\n                    db.exec(\"CREATE INDEX page_titles ON pages(title);\");\n                    db.exec(\"CREATE TEMPORARY TABLE categories(category INTEGER, page INTEGER);\");\n                    db.exec(\"CREATE TEMPORARY TABLE templates(template INTEGER, page INTEGER);\");\n                    db.exec(\"CREATE TEMPORARY TABLE links(lnkDest INTEGER, lnkSrc INTEGER);\");\n                    stGetTmpPageId = db.prepare(\"SELECT id FROM pages WHERE title == ?;\");\n                    stWriteTmpPages = db.prepare(\"INSERT INTO pages (id, title) VALUES (?, ?);\");\n                    stWriteTmpCategories = db.prepare(\"INSERT INTO categories (category, page) VALUES (?, ?);\");\n                    stWriteTmpTemplates = db.prepare(\"INSERT INTO templates (template, page) VALUES (?, ?);\");\n                    stWriteTmpLinks = db.prepare(\"INSERT INTO links (lnkDest, lnkSrc) VALUES (?, ?);\");\n                    stRead = createReadStmt(db);\n                    stWrite = createWriteStmt(db);\n                } catch (SQLiteException e) {\n                    throw new RuntimeException(e);\n                }\n                initialised = true;\n\n                // take jobs\n\n                while(!(sqliteJobs.isEmpty() && stopWhenQueueEmpty)) {\n                    Runnable job;\n                    try {\n                        job = sqliteJobs.take();\n                    } catch (InterruptedException e) {\n                        throw new RuntimeException(e);\n                    }\n                    job.run();\n                }\n\n            } finally {\n                if (stGetTmpPageId != null) {\n                    stGetTmpPageId.dispose();\n                }\n                if (stWriteTmpPages != null) {\n                    stWriteTmpPages.dispose();\n                }\n                if (stWriteTmpCategories != null) {\n                    stWriteTmpCategories.dispose();\n                }\n                if (stWriteTmpTemplates != null) {\n                    stWriteTmpTemplates.dispose();\n                }\n                if (stWriteTmpLinks != null) {\n                    stWriteTmpLinks.dispose();\n                }\n                if (stRead != null) {\n                    stRead.dispose();\n                }\n                if (stWrite != null) {\n                    stWrite.dispose();\n                }\n                if (db != null) {\n                    try {\n                        db.exec(\"DROP TABLE pages;\");\n                        db.exec(\"DROP TABLE categories;\");\n                        db.exec(\"DROP TABLE templates;\");\n                        db.exec(\"DROP TABLE links;\");\n                    } catch (SQLiteException e) {\n                        throw new RuntimeException(e);\n                    }\n                    db.dispose();\n                }\n                initialised = false;\n            }\n        }\n    }\n    \n    static class SQLiteNoOpJob implements Runnable {\n        @Override\n        public void run() {\n        }\n    }\n    \n    static class SQLiteWriteObjectJob<T> implements Runnable {\n        protected final String key;\n        protected final T value;\n        protected final SQLiteStatement stWrite;\n        \n        public SQLiteWriteObjectJob(String key, T value, SQLiteStatement stWrite) {\n            this.key = key;\n            this.value = value;\n            this.stWrite = stWrite;\n        }\n        \n        @Override\n        public void run() {\n            writeObject(stWrite, key, value);\n        }\n    }\n    \n    abstract protected class SQLiteUpdatePageListsJob implements Runnable {\n\n        protected OtpErlangObject readObject(String key)\n                throws RuntimeException, FileNotFoundException {\n            // Note: need to qualify static function call due to\n            // http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6302954\n            return WikiDumpPrepareSQLiteForScalarisHandler.readObject(stRead, key);\n        }\n\n        protected ErlangValue readObject2(String key)\n                throws RuntimeException, FileNotFoundException {\n            // Note: need to qualify static function call due to\n            // http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6302954\n            return WikiDumpPrepareSQLiteForScalarisHandler.readObject2(stRead, key);\n        }\n\n        protected <T> void writeObject(String key, T value)\n                throws RuntimeException {\n            WikiDumpPrepareSQLiteForScalarisHandler.writeObject(stWrite, key, value);\n        }\n        \n        /**\n         * Converts a page to an integer ID and inserts the name into the\n         * (temporary) page table.\n         * \n         * @param pageTitle\n         *            page title\n         * \n         * @return the ID of the page\n         * \n         * @throws RuntimeException\n         */\n        protected long pageToId(NormalisedTitle pageTitle) throws RuntimeException {\n            String pageTitle2 = pageTitle.toString();\n            try {\n                long pageId = -1;\n                // try to find the page id in the pages table:\n                try {\n                    stGetTmpPageId.bind(1, pageTitle2);\n                    if (stGetTmpPageId.step()) {\n                        pageId = stGetTmpPageId.columnLong(0);\n                    }\n                } finally {\n                    stGetTmpPageId.reset();\n                }\n                // page not found yet -> add to pages table:\n                if (pageId == -1) {\n                    pageId = nextPageId++;\n                    try {\n                        stWriteTmpPages.bind(1, pageId).bind(2, pageTitle2).stepThrough();\n                    } finally {\n                        stWriteTmpPages.reset();\n                    }\n                }\n                return pageId;\n            } catch (SQLiteException e) {\n                error(\"write of \" + pageTitle + \" failed (sqlite error: \" + e.toString() + \")\");\n                throw new RuntimeException(e);\n            }\n        }\n    }\n    \n    /**\n     * Updates the pages and articles page lists.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    protected class SQLiteUpdatePageLists1Job extends SQLiteUpdatePageListsJob {\n        EnumMap<NamespaceEnum, ArrayList<NormalisedTitle>> newPages;\n        int articleCount;\n        \n        /**\n         * Writes the page and article list to the DB.\n         * \n         * @param newPages\n         *            list of page titles\n         * @param articleCount\n         *            number of articles\n         */\n        public SQLiteUpdatePageLists1Job(EnumMap<NamespaceEnum, ArrayList<NormalisedTitle>> newPages, int articleCount) {\n            this.newPages = newPages;\n            this.articleCount = articleCount;\n        }\n        \n        @Override\n        public void run() {\n            String scalaris_key;\n            \n            // list of pages:\n            for(NamespaceEnum ns : NamespaceEnum.values()) {\n                scalaris_key = ScalarisDataHandler.getPageListKey(ns.getId());\n                final List<String> curNewPages = ScalarisDataHandlerNormalised\n                        .normList2normStringList(newPages.get(ns));\n                List<String> pageList;\n                try {\n                    pageList = readObject2(scalaris_key).stringListValue();\n                    pageList.addAll(curNewPages);\n                } catch (FileNotFoundException e) {\n                    pageList = curNewPages;\n                }\n                writeObject(scalaris_key, pageList);\n                writeObject(ScalarisDataHandler.getPageCountKey(ns.getId()), pageList.size());\n            }\n            \n            // number articles:\n            writeObject(ScalarisDataHandler.getArticleCountKey(), articleCount);\n        }\n    }\n\n    /**\n     * Updates the categories, templates and backlinks page lists.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    protected class SQLiteUpdateTmpPageLists2Job extends SQLiteUpdatePageListsJob {\n        MultiHashMap<NormalisedTitle, NormalisedTitle> newCategories;\n        MultiHashMap<NormalisedTitle, NormalisedTitle> newTemplates;\n        MultiHashMap<NormalisedTitle, NormalisedTitle> newBackLinks;\n        \n        /**\n         * Writes (temporary) page list mappings to the DB.\n         * \n         * @param newCategories\n         *            category mappings\n         * @param newTemplates\n         *            template mappings\n         * @param newBackLinks\n         *            link mappings\n         */\n        public SQLiteUpdateTmpPageLists2Job(\n                MultiHashMap<NormalisedTitle, NormalisedTitle> newCategories,\n                MultiHashMap<NormalisedTitle, NormalisedTitle> newTemplates,\n                MultiHashMap<NormalisedTitle, NormalisedTitle> newBackLinks) {\n            this.newCategories = newCategories;\n            this.newTemplates = newTemplates;\n            this.newBackLinks = newBackLinks;\n        }\n        \n        protected void addToList(SQLiteStatement stmt, NormalisedTitle key, Collection<? extends NormalisedTitle> values) {\n            long key_id = pageToId(key);\n            ArrayList<Long> values_id = new ArrayList<Long>(values.size());\n            for (NormalisedTitle value : values) {\n                values_id.add(pageToId(value));\n            }\n            try {\n                try {\n                    stmt.bind(1, key_id);\n                    for (Long value_id : values_id) {\n                        stmt.bind(2, value_id).stepThrough().reset(false);\n                    }\n                } finally {\n                    stmt.reset();\n                }\n            } catch (SQLiteException e) {\n                error(\"write of \" + key + \" failed (sqlite error: \" + e.toString() + \")\");\n                throw new RuntimeException(e);\n            }\n        }\n        \n        @Override\n        public void run() {\n            // list of pages in each category:\n            for (Entry<NormalisedTitle, List<NormalisedTitle>> category: newCategories.entrySet()) {\n                addToList(stWriteTmpCategories, category.getKey(), category.getValue());\n            }\n        \n            // list of pages a template is used in:\n            for (Entry<NormalisedTitle, List<NormalisedTitle>> template: newTemplates.entrySet()) {\n                addToList(stWriteTmpTemplates, template.getKey(), template.getValue());\n            }\n            \n            // list of pages linking to other pages:\n            for (Entry<NormalisedTitle, List<NormalisedTitle>> backlinks: newBackLinks.entrySet()) {\n                addToList(stWriteTmpLinks, backlinks.getKey(), backlinks.getValue());\n            }\n        }\n    }\n    \n    private static enum ListType {\n        CAT_LIST, TPL_LIST, LNK_LIST;\n    }\n\n    /**\n     * Updates the categories, templates and backlinks page lists.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    protected class SQLiteConvertTmpPageLists2Job extends SQLiteUpdatePageListsJob {\n        public SQLiteConvertTmpPageLists2Job() {\n        }\n        \n        @Override\n        public void run() {\n            // NOTE: need to normalise every page title!!\n            SQLiteStatement stmt = null;\n            try {\n                // list of pages in each category:\n                do {\n                    println(\"  creating category lists\");\n                    stmt = db.prepare(\"SELECT cat.title, page.title FROM categories as categories \" +\n                            \"INNER JOIN pages AS cat ON categories.category == cat.id \" +\n                            \"INNER JOIN pages AS page ON categories.page == page.id ORDER BY cat.title;\");\n                    writeToScalarisKV(stmt, ListType.CAT_LIST);\n                    stmt.dispose();\n                } while (false);\n\n                // list of pages a template is used in:\n                do {\n                    println(\"  creating template lists\");\n                    stmt = db.prepare(\"SELECT tpl.title, page.title FROM templates as templates \" +\n                            \"INNER JOIN pages AS tpl ON templates.template == tpl.id \" +\n                            \"INNER JOIN pages AS page ON templates.page == page.id ORDER BY tpl.title;\");\n                    writeToScalarisKV(stmt, ListType.TPL_LIST);\n                    stmt.dispose();\n                } while (false);\n\n                // list of pages linking to other pages:\n                do {\n                    println(\"  creating backlink lists\");\n                    stmt = db.prepare(\"SELECT lnkDest.title, lnkSrc.title FROM links as links \" +\n                            \"INNER JOIN pages AS lnkDest ON links.lnkDest == lnkDest.id \" +\n                            \"INNER JOIN pages AS lnkSrc ON links.lnkSrc == lnkSrc.id ORDER BY lnkDest.title;\");\n                    writeToScalarisKV(stmt, ListType.LNK_LIST);\n                    stmt.dispose();\n                } while (false);\n            } catch (SQLiteException e) {\n                error(\"sqlite error: \" + e.toString());\n                throw new RuntimeException(e);\n            } finally {\n                if (stmt != null) {\n                    stmt.dispose();\n                }\n            }\n        }\n        \n        /**\n         * Aggregates page lists from a 2-column table mapping a list key to a\n         * list content.\n         * \n         * @param stmt\n         *            the statement to read from\n         * @param listType\n         *            the type of list\n         * \n         * @throws SQLiteException\n         * @throws RuntimeException\n         */\n        private void writeToScalarisKV(SQLiteStatement stmt, ListType listType) throws SQLiteException, RuntimeException {\n            String listKey = null;\n            List<String> pageList = new ArrayList<String>(1000);\n            while (stmt.step()) {\n                // note: both titles are already normalised!\n                final String stmtKey = stmt.columnString(0);\n                final String stmtPage = stmt.columnString(1);\n                if (listKey == null) {\n                    // first row\n                    listKey = stmtKey;\n                    pageList.add(stmtPage);\n                } else if (listKey.equals(stmtKey)) {\n                    // next item with the same list key\n                    pageList.add(stmtPage);\n                } else {\n                    // new item, i.e. different list key\n                    // -> write old list, then accumulate new\n                    writeToScalarisKV(NormalisedTitle.fromNormalised(listKey), pageList, listType);\n                    listKey = stmtKey;\n                    pageList.add(stmtPage);\n                }\n            }\n            if (listKey != null && !pageList.isEmpty()) {\n                writeToScalarisKV(NormalisedTitle.fromNormalised(listKey), pageList, listType);\n            }\n        }\n\n        private void writeToScalarisKV(NormalisedTitle listKey, List<String> pageList,\n                ListType listType) throws RuntimeException {\n            switch (listType) {\n            case CAT_LIST:\n                writeCatToScalarisKV(listKey, pageList);\n                break;\n            case TPL_LIST:\n                writeTplToScalarisKV(listKey, pageList);\n                break;\n            case LNK_LIST:\n                writeLnkToScalarisKV(listKey, pageList);\n                break;\n            }\n        }\n\n        private void writeCatToScalarisKV(NormalisedTitle category0,\n                List<String> catPageList) throws RuntimeException {\n            // note: titles in the page list are already normalised!\n            String scalaris_key;\n            scalaris_key = ScalarisDataHandlerNormalised.getCatPageListKey(category0);\n            writeObject(scalaris_key, catPageList);\n            scalaris_key = ScalarisDataHandlerNormalised.getCatPageCountKey(category0);\n            writeObject(scalaris_key, catPageList.size());\n            catPageList.clear();\n        }\n\n        private void writeTplToScalarisKV(NormalisedTitle template0,\n                List<String> tplPageList) throws RuntimeException {\n            // note: titles in the page list are already normalised!\n            String scalaris_key;\n            scalaris_key = ScalarisDataHandlerNormalised.getTplPageListKey(template0);\n            writeObject(scalaris_key, tplPageList);\n            tplPageList.clear();\n        }\n\n        private void writeLnkToScalarisKV(NormalisedTitle linkDest0,\n                List<String> backLinksPageList) throws RuntimeException {\n            // note: titles in the page list are already normalised!\n            String scalaris_key;\n            scalaris_key = ScalarisDataHandlerNormalised.getBackLinksPageListKey(linkDest0);\n            writeObject(scalaris_key, backLinksPageList);\n            backLinksPageList.clear();\n        }\n    }\n    \n    protected static class SQLiteWriteSiteInfoJob implements Runnable {\n        SiteInfo siteInfo;\n        protected SQLiteStatement stWrite;\n        \n        public SQLiteWriteSiteInfoJob(SiteInfo siteInfo, SQLiteStatement stWrite) {\n            this.siteInfo = siteInfo;\n            this.stWrite = stWrite;\n        }\n        \n        @Override\n        public void run() {\n            String key = ScalarisDataHandler.getSiteInfoKey();\n            writeObject(stWrite, key, siteInfo);\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/WikiDumpPreparedSQLiteToScalaris.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport java.io.IOException;\nimport java.io.PrintStream;\nimport java.math.BigInteger;\nimport java.text.NumberFormat;\nimport java.util.Collection;\nimport java.util.HashMap;\nimport java.util.List;\nimport java.util.Locale;\nimport java.util.Map.Entry;\nimport java.util.Random;\nimport java.util.concurrent.ArrayBlockingQueue;\nimport java.util.concurrent.ExecutorService;\nimport java.util.concurrent.TimeUnit;\n\nimport com.almworks.sqlite4java.SQLiteConnection;\nimport com.almworks.sqlite4java.SQLiteException;\nimport com.almworks.sqlite4java.SQLiteStatement;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\nimport de.zib.scalaris.CommonErlangObjects;\nimport de.zib.scalaris.Connection;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.ConnectionFactory;\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.RoundRobinConnectionPolicy;\nimport de.zib.scalaris.TransactionSingleOp;\nimport de.zib.scalaris.examples.wikipedia.Options;\nimport de.zib.scalaris.examples.wikipedia.SQLiteDataHandler;\nimport de.zib.scalaris.examples.wikipedia.Options.IBuckets;\nimport de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpConvertPreparedSQLite.ConvertOp;\nimport de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpConvertPreparedSQLite.KVPair;\nimport de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpConvertPreparedSQLite.ListOrCountOp;\nimport de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpConvertPreparedSQLite.SQLiteCopyList;\nimport de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpConvertPreparedSQLite.SQLiteWriteBucketCounterJob;\nimport de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpConvertPreparedSQLite.SQLiteWriteBucketListJob;\nimport de.zib.scalaris.operations.WriteOp;\n\n/**\n * Provides abilities to read an xml wiki dump file and write its contents to\n * Scalaris by pre-processing key/value pairs into a local SQLite db.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class WikiDumpPreparedSQLiteToScalaris implements WikiDump {\n    private static final int MAX_SCALARIS_CONNECTIONS = Runtime.getRuntime().availableProcessors() * 4;\n    private static final int REQUEST_BUNDLE_SIZE = 10;\n    private static final int PRINT_SCALARIS_KV_PAIRS_EVERY = 5000;\n    private ArrayBlockingQueue<TransactionSingleOp> scalaris_single = new ArrayBlockingQueue<TransactionSingleOp>(MAX_SCALARIS_CONNECTIONS);\n    \n    private ExecutorService executor = WikiDumpToScalarisHandler\n            .createExecutor(MAX_SCALARIS_CONNECTIONS);\n    \n    protected TransactionSingleOp.RequestList requests = new TransactionSingleOp.RequestList();\n    \n    /**\n     * The time at the start of an import operation.\n     */\n    private long timeAtStart = 0;\n    /**\n     * The time at the end of an import operation.\n     */\n    private long timeAtEnd = 0;\n    /**\n     * The number of (successfully) processed K/V pairs.\n     */\n    protected int importedKeys = 0;\n    \n    protected PrintStream msgOut = System.out;\n    \n    protected boolean stop = false;\n    \n    protected SQLiteConnection db = null;\n    \n    protected final String dbFileName;\n    protected final int numberOfImporters;\n    protected final int myNumber;\n    protected final ConnectionFactory cFactory;\n    protected boolean errorDuringImport = false;\n    protected final Options dbWriteOptions;\n\n    /**\n     * Sets up a SAX XmlHandler exporting all parsed pages except the ones in a\n     * blacklist to Scalaris but with an additional pre-process phase.\n     * \n     * @param dbFileName\n     *            the name of the database file to read from\n     * @param dbWriteOptions\n     *            options that specify which optimisations should be applied in\n     *            the Scalaris DB\n     * @param numberOfImporters\n     *            number of (independent) import jobs\n     * @param myNumber\n     *            my own import job number (1 &lt;= <tt>myNumber</tt> &lt;= <tt>numberOfImporters</tt>)\n     * \n     * @throws RuntimeException\n     *             if the connection to Scalaris fails\n     * @throws IllegalArgumentException\n     *             if <tt>myNumber</tt> is not greater than 0 or is greater than\n     *             <tt>numberOfImporters</tt>\n     */\n    public WikiDumpPreparedSQLiteToScalaris(String dbFileName, Options dbWriteOptions, int numberOfImporters, int myNumber) throws RuntimeException {\n        this.dbFileName = dbFileName;\n        this.cFactory = new ConnectionFactory();\n        Random random = new Random();\n        String clientName = new BigInteger(128, random).toString(16);\n        this.cFactory.setClientName(\"wiki_import_\" + clientName);\n        this.cFactory.setClientNameAppendUUID(true);\n        this.cFactory.setConnectionPolicy(\n                new RoundRobinConnectionPolicy(this.cFactory.getNodes()));\n        this.numberOfImporters = numberOfImporters;\n        this.myNumber = myNumber;\n        if (myNumber > numberOfImporters || myNumber < 1) {\n            throw new IllegalArgumentException(\"not 1 <= myNumber (\" + myNumber + \") <= numberOfImporters (\" + numberOfImporters + \")\");\n        }\n        this.dbWriteOptions = dbWriteOptions;\n    }\n\n    /**\n     * Sets up a SAX XmlHandler exporting all parsed pages except the ones in a\n     * blacklist to Scalaris but with an additional pre-process phase.\n     * \n     * @param dbFileName\n     *            the name of the database file to read from\n     * @param dbWriteOptions\n     *            options that specify which optimisations should be applied in\n     *            the Scalaris DB\n     * @param numberOfImporters\n     *            number of (independent) import jobs\n     * @param myNumber\n     *            my own import job number (1 &lt;= <tt>myNumber</tt> &lt;= <tt>numberOfImporters</tt>)\n     * @param cFactory\n     *            the connection factory to use for creating new connections\n     * \n     * @throws RuntimeException\n     *             if the connection to Scalaris fails\n     * @throws IllegalArgumentException\n     *             if <tt>myNumber</tt> is not greater than 0 or is greater than\n     *             <tt>numberOfImporters</tt>\n     */\n    public WikiDumpPreparedSQLiteToScalaris(String dbFileName, Options dbWriteOptions, int numberOfImporters, int myNumber, ConnectionFactory cFactory) throws RuntimeException {\n        this.dbFileName = dbFileName;\n        this.cFactory = cFactory;\n        this.numberOfImporters = numberOfImporters;\n        this.myNumber = myNumber;\n        if (myNumber > numberOfImporters || myNumber < 1) {\n            throw new IllegalArgumentException(\"not 1 <= myNumber (\" + myNumber + \") <= numberOfImporters (\" + numberOfImporters + \")\");\n        }\n        this.dbWriteOptions = dbWriteOptions;\n    }\n\n    /**\n     * Sets the time the import started.\n     */\n    final protected void importStart() {\n        timeAtStart = System.currentTimeMillis();\n    }\n\n    /**\n     * Sets the time the import finished.\n     */\n    final protected void importEnd() {\n        timeAtEnd = System.currentTimeMillis();\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#getTimeAtStart()\n     */\n    @Override\n    public long getTimeAtStart() {\n        return timeAtStart;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#getTimeAtEnd()\n     */\n    @Override\n    public long getTimeAtEnd() {\n        return timeAtEnd;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#getPageCount()\n     */\n    @Override\n    public int getImportCount() {\n        return importedKeys;\n    }\n\n    /**\n     * Reports the speed of the import (pages/s) and may be used as a shutdown\n     * handler.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public class ReportAtShutDown extends Thread {\n        public void run() {\n            reportAtEnd();\n        }\n\n        /**\n         * Sets the import end time and reports the overall speed.\n         */\n        public void reportAtEnd() {\n            // import may have been interrupted - get an end time in this case\n            if (timeAtEnd == 0) {\n                importEnd();\n            }\n            final long timeTaken = timeAtEnd - timeAtStart;\n            final double speed = (((double) importedKeys) * 1000) / timeTaken;\n            NumberFormat nf = NumberFormat.getNumberInstance(Locale.ENGLISH);\n            nf.setGroupingUsed(true);\n            println(\"Finished import (\" + nf.format(speed) + \" pages/s)\");\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#setMsgOut(java.io.PrintStream)\n     */\n    @Override\n    public void setMsgOut(PrintStream msgOut) {\n        this.msgOut = msgOut;\n    }\n\n    /**\n     * Whether {@link #stopParsing()} is supported or not.\n     * \n     * @return no stop parsing support\n     */\n    @Override\n    public boolean hasStopSupport() {\n        return false;\n    }\n\n    /**\n     * Tells the import to stop (not supported since this may not result in a\n     * consistent view).\n     */\n    @Override\n    public void stopParsing() {\n//        this.stop = true;\n    }\n\n    /**\n     * Writes all K/V pairs to Scalaris.\n     * \n     * Note that this process can not be stopped as the resulting view may not\n     * be consistent.\n     */\n    public void writeToScalaris() {\n        println(\"Importing key/value pairs to Scalaris...\");\n        SQLiteStatement st = null;\n        try {\n            importStart();\n            if (numberOfImporters > 1) {\n                final SQLiteStatement countStmt = db.prepare(\"SELECT COUNT(*) FROM objects;\");\n                try {\n                    if (countStmt.step()) {\n                        final long stepSize = countStmt.columnLong(0) / numberOfImporters;\n                        final long offset = (myNumber - 1) * stepSize;\n                        final long limit = (myNumber == numberOfImporters) ? -1 : stepSize;\n                        st = db.prepare(\"SELECT scalaris_key, scalaris_value FROM objects LIMIT \" + limit + \" OFFSET \" + offset + \";\");\n                    } else {\n                        throw new RuntimeException(\"cannot count table objects\");\n                    }\n                } finally {\n                    countStmt.dispose();\n                }\n                \n            } else {\n                st = db.prepare(\"SELECT scalaris_key, scalaris_value FROM objects;\");\n            }\n            while (st.step()) {\n                String key = st.columnString(0);\n                byte[] value = st.columnBlob(1);\n                try {\n                    writeToScalaris(key, value);\n                } catch (ClassNotFoundException e) {\n                    error(\"read of \" + key + \" failed (error: \" + e.toString() + \")\");\n                    throw new RuntimeException(e);\n                } catch (IOException e) {\n                    error(\"read of \" + key + \" failed (error: \" + e.toString() + \")\");\n                    throw new RuntimeException(e);\n                }\n            }\n            // some requests may be left over\n            Runnable worker = new WikiDumpToScalarisHandler.MyScalarisSingleRunnable(\n                    this, requests, scalaris_single, \"\");\n            executor.execute(worker);\n            requests = new TransactionSingleOp.RequestList();\n            executor.shutdown();\n            boolean shutdown = false;\n            while (!shutdown) {\n                try {\n                    shutdown = executor.awaitTermination(1, TimeUnit.MINUTES);\n                } catch (InterruptedException e) {\n                }\n            }\n            importEnd();\n        } catch (SQLiteException e) {\n            e.printStackTrace();\n        } finally {\n            if (st != null) {\n                st.dispose();\n            }\n        }\n    }\n    \n    protected void writeToScalaris(String key, byte[] value) throws ClassNotFoundException, IOException {\n        ++importedKeys;\n        if (dbWriteOptions == null) {\n            OtpErlangObject valueOtp = WikiDumpPrepareSQLiteForScalarisHandler\n                    .objectFromBytes(value);\n            requests.addOp(new WriteCompressedOp(key, valueOtp));\n        } else {\n            ConvertOp convOp = WikiDumpConvertPreparedSQLite.getConvertOp(key, dbWriteOptions);\n            if (convOp == null) {\n                println(\"unknown key: \" + key);\n                // use defaults and continue anyway...\n                convOp = new ConvertOp();\n            }\n\n            if (convOp.optimisation instanceof IBuckets) {\n                IBuckets optimisation = (IBuckets) convOp.optimisation;\n                switch (convOp.listOrCount) {\n                    case LIST:\n                        try {\n                            List<ErlangValue> listVal = WikiDumpPrepareSQLiteForScalarisHandler\n                                    .objectFromBytes2(value).listValue();\n                            HashMap<String, List<ErlangValue>> newLists = SQLiteWriteBucketListJob\n                                    .splitList(optimisation, listVal);\n                            for (Entry<String, List<ErlangValue>> newList : newLists.entrySet()) {\n                                // write list\n                                final String key2 = key + newList.getKey();\n                                requests.addOp(new WriteOp(key2, newList.getValue()));\n                                // write count (if available)\n                                if (convOp.countKey != null && convOp.countKeyOptimisation == null) {\n                                    // integrated counter\n                                    final String countKey2 = convOp.countKey + newList.getKey();\n                                    requests.addOp(new WriteOp(countKey2, newList.getValue().size()));\n                                }\n                            }\n                            if (convOp.countKey != null && convOp.countKeyOptimisation != null) {\n                                // separate counter:\n                                int listSize = listVal.size();\n                                if (convOp.countKeyOptimisation instanceof IBuckets) {\n                                    // similar to \"case COUNTER\" below:\n                                    Collection<KVPair<Integer>> newCounters = SQLiteWriteBucketCounterJob\n                                            .splitCounter(\n                                                    (IBuckets) convOp.countKeyOptimisation,\n                                                    convOp.countKey, listSize);\n                                    for (KVPair<Integer> kvPair : newCounters) {\n                                        requests.addOp(new WriteOp(kvPair.key, kvPair.value));\n                                    }\n                                } else {\n                                    // copy counter\n                                    requests.addOp(new WriteOp(convOp.countKey, listSize));\n                                }\n                            }\n                        } catch (Exception e) {\n                            println(\"write of \" + key + \" failed (error: \" + e.toString() + \")\");\n                            return;\n                        }\n                        break;\n                    case COUNTER:\n                        int counter = WikiDumpPrepareSQLiteForScalarisHandler\n                                .objectFromBytes2(value).intValue();\n                        Collection<KVPair<Integer>> newCounters = SQLiteWriteBucketCounterJob\n                                .splitCounter(optimisation, key, counter);\n                        for (KVPair<Integer> kvPair : newCounters) {\n                            requests.addOp(new WriteOp(kvPair.key, kvPair.value));\n                        }\n                        break;\n                    default:\n                        break;\n                }\n            } else if (convOp.optimisation != null ) {\n                assert (convOp.countKeyOptimisation == null);\n                if (convOp.listOrCount == ListOrCountOp.LIST) {\n                    Collection<KVPair<Object>> operations = SQLiteCopyList.splitOp(key, convOp.countKey, value);\n                    for (KVPair<Object> kvPair : operations) {\n                        OtpErlangObject valueOtpCompressed;\n                        if (kvPair.value instanceof byte[]) {\n                            valueOtpCompressed = WikiDumpPrepareSQLiteForScalarisHandler\n                                    .objectFromBytes((byte[]) kvPair.value);\n                        } else {\n                            valueOtpCompressed = CommonErlangObjects\n                                    .encode(ErlangValue.convertToErlang(kvPair.value));\n                        }\n                        requests.addOp(new WriteCompressedOp(kvPair.key, valueOtpCompressed));\n                    }\n                } else {\n                    // write object as is\n                    OtpErlangObject valueOtp = WikiDumpPrepareSQLiteForScalarisHandler\n                            .objectFromBytes(value);\n                    requests.addOp(new WriteCompressedOp(key, valueOtp));\n                }\n            }\n        }\n        // bundle requests:\n        if (requests.size() >= REQUEST_BUNDLE_SIZE) {\n            Runnable worker = new WikiDumpToScalarisHandler.MyScalarisSingleRunnable(\n                    this, requests, scalaris_single, \"keys up to \" + key);\n            executor.execute(worker);\n            requests = new TransactionSingleOp.RequestList();\n        }\n        if ((importedKeys % PRINT_SCALARIS_KV_PAIRS_EVERY) == 0) {\n            // wait for all threads to finish (otherwise we would take a lot of\n            // memory, especially if the connection to Scalaris is slow)\n            executor.shutdown();\n            boolean shutdown = false;\n            while (!shutdown) {\n                try {\n                    shutdown = executor.awaitTermination(1, TimeUnit.MINUTES);\n                } catch (InterruptedException e) {\n                }\n            }\n            executor = WikiDumpToScalarisHandler.createExecutor(MAX_SCALARIS_CONNECTIONS);\n            \n            println(\"imported K/V pairs to Scalaris: \" + importedKeys);\n        }\n    }\n    \n    /**\n     * Similar to {@link WriteOp} but assumes that the value is already encoded\n     * by {@link CommonErlangObjects#encode(OtpErlangObject)} and that the\n     * connection to Scalaris which uses this operation uses compressed values.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class WriteCompressedOp extends WriteOp {\n        /**\n         * Constructor\n         *\n         * @param key\n         *            the key to write the value to\n         * @param value\n         *            the value to write\n         */\n        public WriteCompressedOp(OtpErlangString key, OtpErlangObject value) {\n            super(key, value);\n        }\n        /**\n         * Constructor\n         *\n         * @param key\n         *            the key to write the value to\n         * @param value\n         *            the value to write\n         */\n        public WriteCompressedOp(String key, OtpErlangObject value) {\n            super(new OtpErlangString(key), value);\n        }\n\n        /* (non-Javadoc)\n         * @see de.zib.scalaris.operations.WriteOp#getErlang(boolean)\n         */\n        @Override\n        public OtpErlangObject getErlang(boolean compressed) {\n            assert compressed;\n            return new OtpErlangTuple(new OtpErlangObject[] {\n                    CommonErlangObjects.writeAtom, key, value });\n        }\n        \n    }\n\n    /* (non-Javadoc)\n     * @see java.lang.Object#finalize()\n     */\n    @Override\n    protected void finalize() throws Throwable {\n        super.finalize();\n        tearDown();\n    }\n\n    /**\n     * Sets up the directory to write files to as well as the Scalaris\n     * connection.\n     * \n     * @throws RuntimeException\n     *             if the directory could not be created\n     */\n    @Override\n    public void setUp() {\n        try {\n            db = SQLiteDataHandler.openDB(dbFileName, true, null);\n        } catch (SQLiteException e) {\n            error(\"Cannot read database: \" + dbFileName);\n            throw new RuntimeException(e);\n        }\n\n        try {\n            for (int i = 0; i < MAX_SCALARIS_CONNECTIONS; ++i) {\n                Connection connection = cFactory.createConnection(\n                        \"wiki_import_\" + myNumber, true);\n                scalaris_single.put(new TransactionSingleOp(connection));\n            }\n        } catch (ConnectionException e) {\n            error(\"Connection to Scalaris failed\");\n            throw new RuntimeException(e);\n        } catch (InterruptedException e) {\n            error(\"Interrupted while setting up multiple connections to Scalaris\");\n            throw new RuntimeException(e);\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#tearDown()\n     */\n    @Override\n    public void tearDown() {\n        if (db != null) {\n            db.dispose();\n        }\n    }\n\n    /**\n     * Prints a message to the chosen output stream (includes a timestamp).\n     * \n     * @param message\n     *            the message to print\n     * \n     * @see #setMsgOut(PrintStream)\n     */\n    public void print(String message) {\n        WikiDumpHandler.print(msgOut, message);\n    }\n\n    /**\n     * Prints a message to the chosen output stream (includes a timestamp).\n     * Includes a newline character at the end.\n     * \n     * @param message\n     *            the message to print\n     * \n     * @see #setMsgOut(PrintStream)\n     */\n    public void println(String message) {\n        WikiDumpHandler.println(msgOut, message);\n    }\n\n    @Override\n    public boolean isErrorDuringImport() {\n        return errorDuringImport;\n    }\n\n    @Override\n    public void error(String message) {\n        System.err.println(message);\n        errorDuringImport = true;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/WikiDumpSQLiteLinkTables.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport java.io.PrintStream;\nimport java.text.NumberFormat;\nimport java.util.ArrayList;\nimport java.util.Collection;\nimport java.util.HashSet;\nimport java.util.Locale;\nimport java.util.Set;\nimport java.util.SortedSet;\nimport java.util.TreeSet;\n\nimport com.almworks.sqlite4java.SQLiteConnection;\nimport com.almworks.sqlite4java.SQLiteException;\nimport com.almworks.sqlite4java.SQLiteStatement;\n\nimport de.zib.scalaris.examples.wikipedia.RevisionResult;\nimport de.zib.scalaris.examples.wikipedia.SQLiteDataHandler;\nimport de.zib.scalaris.examples.wikipedia.SQLiteDataHandler.Connection;\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandlerNormalised;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyNamespace;\nimport de.zib.scalaris.examples.wikipedia.bliki.MySQLiteWikiModel;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyWikiModel;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\n\n/**\n * Provides abilities to read a XML2SQLite wiki dump file and renders each page\n * to create the link tables, i.e. templatelinks, pagelinks, categorylinks,\n * redirect.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class WikiDumpSQLiteLinkTables implements WikiDump {\n    private static final int PRINT_PAGES_EVERY = 500;\n    /**\n     * The time at the start of an import operation.\n     */\n    private long timeAtStart = 0;\n    /**\n     * The time at the end of an import operation.\n     */\n    private long timeAtEnd = 0;\n    /**\n     * The number of (successfully) processed pages.\n     */\n    protected int importedPages = 0;\n    \n    protected PrintStream msgOut = System.out;\n    \n    protected boolean stop = false;\n\n    Connection connection = null;\n    protected SQLiteStatement stGetPages = null;\n    \n    final String dbFileName;\n    private SQLiteStatement stWriteCat = null;\n    private SQLiteStatement stWriteTpl = null;\n    private SQLiteStatement stWriteLnk = null;\n    private SQLiteStatement stWriteRedirect = null;\n    private SQLiteStatement stWriteStats = null;\n    protected boolean errorDuringImport = false;\n    \n    /**\n     * Constructor.\n     * \n     * @param dbFileName\n     *            the name of the database file\n     * \n     * @throws RuntimeException\n     *             if the connection to Scalaris fails\n     */\n    public WikiDumpSQLiteLinkTables(String dbFileName)\n            throws RuntimeException {\n        this.dbFileName = dbFileName;\n    }\n\n    /**\n     * Sets the time the import started.\n     */\n    final protected void importStart() {\n        timeAtStart = System.currentTimeMillis();\n    }\n\n    /**\n     * Sets the time the import finished.\n     */\n    final protected void importEnd() {\n        timeAtEnd = System.currentTimeMillis();\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#getTimeAtStart()\n     */\n    @Override\n    public long getTimeAtStart() {\n        return timeAtStart;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#getTimeAtEnd()\n     */\n    @Override\n    public long getTimeAtEnd() {\n        return timeAtEnd;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#getPageCount()\n     */\n    @Override\n    public int getImportCount() {\n        return importedPages;\n    }\n\n    @Override\n    public boolean isErrorDuringImport() {\n        return errorDuringImport ;\n    }\n\n    @Override\n    public void error(String message) {\n        System.err.println(message);\n        errorDuringImport = true;\n    }\n\n    /**\n     * Reports the speed of the import (pages/s) and may be used as a shutdown\n     * handler.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public class ReportAtShutDown extends Thread {\n        public void run() {\n            reportAtEnd();\n        }\n\n        /**\n         * Sets the import end time and reports the overall speed.\n         */\n        public void reportAtEnd() {\n            // import may have been interrupted - get an end time in this case\n            if (timeAtEnd == 0) {\n                importEnd();\n            }\n            final long timeTaken = timeAtEnd - timeAtStart;\n            final double speed = (((double) importedPages) * 1000) / timeTaken;\n            NumberFormat nf = NumberFormat.getNumberInstance(Locale.ENGLISH);\n            nf.setGroupingUsed(true);\n            println(\"Finished conversion (\" + nf.format(speed) + \" pages/s)\");\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#setMsgOut(java.io.PrintStream)\n     */\n    @Override\n    public void setMsgOut(PrintStream msgOut) {\n        this.msgOut = msgOut;\n    }\n\n    /**\n     * Whether {@link #stopParsing()} is supported or not.\n     * \n     * @return no stop parsing support\n     */\n    @Override\n    public boolean hasStopSupport() {\n        return false;\n    }\n\n    /**\n     * Tells the import to stop (not supported since this may not result in a\n     * consistent view).\n     */\n    @Override\n    public void stopParsing() {\n//        this.stop = true;\n    }\n    \n    protected void updateLinks3(Integer key, NormalisedTitle value, SQLiteStatement stWrite, String tableName) {\n        try {\n            try {\n                stWrite.bind(1, key).bind(2, value.namespace)\n                        .bind(3, value.title).stepThrough();\n            } finally {\n                stWrite.reset();\n            }\n        } catch (SQLiteException e) {\n            error(\"write of \" + tableName + \".\" + key + \" failed (sqlite error: \" + e.toString() + \")\");\n            e.printStackTrace();\n        }\n    }\n    \n    protected void updateLinks2(Integer key, NormalisedTitle value, SQLiteStatement stWrite, String tableName) {\n        try {\n            try {\n                stWrite.bind(1, key).bind(2, value.title).stepThrough();\n            } finally {\n                stWrite.reset();\n            }\n        } catch (SQLiteException e) {\n            error(\"write of \" + tableName + \".\" + key + \" failed (sqlite error: \" + e.toString() + \")\");\n            e.printStackTrace();\n        }\n    }\n\n    protected void updateSiteStats(long editCount, int articleCount) {\n        try {\n            try {\n                stWriteStats.bind(1, editCount).bind(2, articleCount).stepThrough();\n            } finally {\n                stWriteStats.reset();\n            }\n        } catch (SQLiteException e) {\n            error(\"write of sitestats failed (sqlite error: \" + e.toString() + \")\");\n            e.printStackTrace();\n        }\n    }\n    \n    /**\n     * Parses all wiki pages in the DB and (re-)creates the link tables.\n     * \n     * @throws RuntimeException\n     */\n    public void processLinks() throws RuntimeException {\n        println(\"Populating link tables...\");\n        importStart();\n        try {\n            try {\n                SiteInfo siteInfo = WikiDumpXml2SQLite.readSiteInfo(connection.db);\n                MyNamespace namespace = new MyNamespace(siteInfo);\n                MySQLiteWikiModel wikiModel = new MySQLiteWikiModel(\"\", \"\", connection, namespace);\n                long editCount = 0l;\n                int articleCount = 0;\n\n                while (stGetPages.step()) {\n                    NormalisedTitle normTitle = new NormalisedTitle(\n                            stGetPages.columnInt(0), stGetPages.columnString(1));\n                    \n                    RevisionResult getRevResult = SQLiteDataHandler\n                            .getRevision(connection, normTitle, namespace);\n                    \n                    if (!getRevResult.success) {\n                        error(\"read of current revision failed (error: \" + getRevResult.message + \")\");\n                        throw new RuntimeException();\n                    }\n                    \n                    final Page page = getRevResult.page;\n                    final Revision revision = getRevResult.revision;\n\n                    wikiModel.setUp();\n                    wikiModel.setNamespaceName(wikiModel.getNamespace().getNamespaceByNumber(normTitle.namespace));\n                    wikiModel.setPageName(normTitle.title);\n                    wikiModel.renderPageWithCache(null, revision.unpackedText());\n\n                    String redirLink_raw = wikiModel.getRedirectLink();\n                    if (redirLink_raw != null) {\n                        NormalisedTitle redirLink = wikiModel.normalisePageTitle(redirLink_raw);\n                        updateLinks3(page.getId(), redirLink, stWriteRedirect, \"redirect\");\n                    }\n                    \n                    // TODO: what if the page was a redirect?\n                    for (String cat_raw: wikiModel.getCategories().keySet()) {\n                        NormalisedTitle category = new NormalisedTitle(\n                                MyNamespace.CATEGORY_NAMESPACE_KEY,\n                                MyWikiModel.normaliseName(cat_raw));\n                        updateLinks2(page.getId(), category, stWriteCat, \"categorylinks\");\n                    }\n                    for (String tpl_raw: wikiModel.getTemplatesNoMagicWords()) {\n                        NormalisedTitle template = new NormalisedTitle(\n                                MyNamespace.TEMPLATE_NAMESPACE_KEY,\n                                MyWikiModel.normaliseName(tpl_raw));\n                        updateLinks3(page.getId(), template, stWriteTpl, \"templatelinks\");\n                    }\n                    for (String incl_raw: wikiModel.getIncludes()) {\n                        NormalisedTitle include = wikiModel.normalisePageTitle(incl_raw);\n                        updateLinks3(page.getId(), include, stWriteTpl, \"templatelinks\");\n                    }\n                    for (String link_raw: wikiModel.getLinks()) {\n                        NormalisedTitle link = wikiModel.normalisePageTitle(link_raw);\n                        updateLinks3(page.getId(), link, stWriteLnk, \"pagelinks\");\n                    }\n                    if (MyWikiModel.isArticle(normTitle.namespace, wikiModel\n                            .getLinks(), wikiModel.getCategories().keySet())) {\n                        ++articleCount;\n                    }\n                    \n                    wikiModel.tearDown();\n                    ++importedPages;\n                    // only export page list every UPDATE_PAGELIST_EVERY pages:\n                    if ((importedPages % PRINT_PAGES_EVERY) == 0) {\n                        println(\"processed pages: \" + importedPages);\n                    }\n                }\n                updateSiteStats(editCount, articleCount);\n            } finally {\n                stGetPages.reset();\n            }\n        } catch (SQLiteException e) {\n            error(\"read failed (sqlite error: \" + e.toString() + \")\");\n            throw new RuntimeException(e);\n        }\n    }\n\n    /**\n     * Sets up the directory to write files to as well as the Scalaris\n     * connection.\n     * \n     * @throws RuntimeException\n     *             if the directory could not be created\n     */\n    @Override\n    public void setUp() {\n        // note: this needs to be executed from the thread that works on the DB!\n        try {\n            println(\"Creating link tables...\");\n            final SQLiteConnection db = SQLiteDataHandler.openDB(dbFileName, false, null);\n            \n            /**\n             * Track page-to-page hyperlinks within the wiki.\n             */\n            final String createPageLinksTable = \"CREATE TABLE IF NOT EXISTS pagelinks (\"\n                    + \"pl_from int unsigned NOT NULL default 0,\"\n                    + \"pl_to int unsigned NOT NULL default 0\"\n                    + \");\";\n            db.exec(createPageLinksTable);\n            // create index here to check at insertion:\n            db.exec(\"CREATE UNIQUE INDEX IF NOT EXISTS pl_from ON pagelinks (pl_from,pl_to);\");\n            \n            /**\n             * Track template inclusions.\n             */\n            final String createTemplateLinksTable = \"CREATE TABLE IF NOT EXISTS templatelinks (\"\n                    + \"tl_from int unsigned NOT NULL default 0,\"\n                    + \"tl_to int unsigned NOT NULL default 0\"\n                    + \");\";\n            db.exec(createTemplateLinksTable);\n            // create index here to check at insertion:\n            db.exec(\"CREATE UNIQUE INDEX IF NOT EXISTS tl_from ON templatelinks (tl_from,tl_to);\");\n        \n            /**\n             * Track category inclusions *used inline*.\n             * This tracks a single level of category membership.\n             */\n            final String createCategoryLinksTable = \"CREATE TABLE IF NOT EXISTS categorylinks (\"\n                    + \"cl_from int unsigned NOT NULL default 0,\"\n                    + \"cl_to int unsigned NOT NULL default 0\"\n                    + \");\";\n            db.exec(createCategoryLinksTable);\n            // create index here to check at insertion:\n            db.exec(\"CREATE UNIQUE INDEX IF NOT EXISTS cl_from ON categorylinks (cl_from,cl_to);\");\n            \n            /**\n             * For each redirect, this table contains exactly one row defining its target.\n             */\n            final String createRedirectsTable = \"CREATE TABLE IF NOT EXISTS redirect (\"\n                    + \"rd_from int unsigned NOT NULL default 0 PRIMARY KEY,\"\n                    + \"rd_to int unsigned NOT NULL default 0\"\n                    + \");\";\n            db.exec(createRedirectsTable);\n        \n            /**\n             * Contains a single row with some aggregate info on the state of the site.\n             */\n            final String createSiteStatsTable = \"CREATE TABLE IF NOT EXISTS site_stats (\"\n                    + \"ss_row_id int unsigned NOT NULL,\"\n                    + \"ss_total_views bigint unsigned default 0,\"\n                    + \"ss_total_edits bigint unsigned default 0,\"\n                    + \"ss_good_articles bigint unsigned default 0,\"\n                    + \"ss_total_pages bigint default '-1'\"\n                    + \");\";\n            db.exec(createSiteStatsTable);\n            // create index here to check at insertion:\n            db.exec(\"CREATE UNIQUE INDEX IF NOT EXISTS ss_row_id ON site_stats (ss_row_id);\");\n        \n            stGetPages = db.prepare(\"SELECT page_namespace, page_title FROM page\");\n            stWriteCat = db.prepare(\"REPLACE INTO categorylinks \"\n                    + \"(cl_from, cl_to) SELECT ?, page_id FROM page \"\n                    + \"WHERE page_namespace == \"\n                    + MyNamespace.CATEGORY_NAMESPACE_KEY\n                    + \" AND page_title == ?;\");\n            stWriteTpl = db.prepare(\"REPLACE INTO templatelinks \"\n                    + \"(tl_from, tl_to) SELECT ?, page_id FROM page \"\n                    + \"WHERE page_namespace == ? AND page_title == ?;\");\n            stWriteLnk = db.prepare(\"REPLACE INTO pagelinks \"\n                    + \"(pl_from, pl_to) SELECT ?, page_id FROM page \"\n                    + \"WHERE page_namespace == ? AND page_title == ?;\");\n            stWriteRedirect = db.prepare(\"REPLACE INTO redirect \"\n                    + \"(rd_from, rd_to) SELECT ?, page_id FROM page \"\n                    + \"WHERE page_namespace == ? AND page_title == ?;\");\n            stWriteStats = db.prepare(\"REPLACE INTO site_stats \"\n                    + \"(ss_row_id, ss_total_views, ss_total_edits, ss_good_articles, ss_total_pages) \"\n                    + \"SELECT 1, 0, ?, ?, COUNT(*) FROM page;\");\n        \n            connection = new Connection(db);\n        } catch (SQLiteException e) {\n            throw new RuntimeException(e);\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see java.lang.Object#finalize()\n     */\n    @Override\n    protected void finalize() throws Throwable {\n        super.finalize();\n        tearDown();\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDump#tearDown()\n     */\n    @Override\n    public void tearDown() {\n        if (stGetPages != null) {\n            stGetPages.dispose();\n        }\n        if (connection != null) {\n            connection.dispose();\n        }\n        if (stWriteCat != null) {\n            stWriteCat.dispose();\n        }\n        if (stWriteTpl != null) {\n            stWriteTpl.dispose();\n        }\n        if (stWriteLnk != null) {\n            stWriteLnk.dispose();\n        }\n        if (stWriteRedirect != null) {\n            stWriteRedirect.dispose();\n        }\n        if (stWriteStats != null) {\n            stWriteStats.dispose();\n        }\n        importEnd();\n    }\n\n    /**\n     * Prints a message to the chosen output stream (includes a timestamp).\n     * \n     * @param message\n     *            the message to print\n     * \n     * @see #setMsgOut(PrintStream)\n     */\n    public void print(String message) {\n        WikiDumpHandler.print(msgOut, message);\n    }\n\n    /**\n     * Prints a message to the chosen output stream (includes a timestamp).\n     * Includes a newline character at the end.\n     * \n     * @param message\n     *            the message to print\n     * \n     * @see #setMsgOut(PrintStream)\n     */\n    public void println(String message) {\n        WikiDumpHandler.println(msgOut, message);\n    }\n\n    /**\n     * Extracts all pages in the given categories from the given DB.\n     * \n     * @param allowedCats0\n     *            include all pages in these categories (un-normalised page\n     *            titles)\n     * @param allowedPages0\n     *            a number of pages to include, also parses these pages for more\n     *            links (un-normalised page titles)\n     * @param depth\n     *            follow links this deep\n     * @param normalised\n     *            whether the pages should be returned as normalised page titles\n     *            or not\n     * \n     * @return a (sorted) set of page titles\n     * \n     * @throws RuntimeException\n     *             if any error occurs\n     */\n    public SortedSet<String> getPagesInCategories(\n            Collection<String> allowedCats0, Collection<String> allowedPages0, int depth,\n            boolean normalised) throws RuntimeException {\n        SiteInfo siteInfo = WikiDumpXml2SQLite.readSiteInfo(connection.db);\n        MyNamespace namespace = new MyNamespace(siteInfo);\n        ArrayList<NormalisedTitle> allowedCats = new ArrayList<NormalisedTitle>(allowedCats0.size());\n        MyWikiModel.normalisePageTitles(allowedCats0, namespace, allowedCats);\n        ArrayList<NormalisedTitle> allowedPages = new ArrayList<NormalisedTitle>(allowedPages0.size());\n        MyWikiModel.normalisePageTitles(allowedPages0, namespace, allowedPages);\n        return getPagesInCategories2(allowedCats, allowedPages, depth, normalised);\n    }\n    \n    /**\n     * Extracts all pages in the given categories from the given DB.\n     * \n     * @param allowedCats\n     *            include all pages in these categories (normalised page\n     *            titles)\n     * @param allowedPages\n     *            a number of pages to include, also parses these pages for more\n     *            links (normalised page titles)\n     * @param depth\n     *            follow links this deep\n     * @param normalised\n     *            whether the pages should be returned as normalised page titles\n     *            or not\n     * \n     * @return a (sorted) set of page titles\n     * \n     * @throws RuntimeException\n     *             if any error occurs\n     */\n    public SortedSet<String> getPagesInCategories2(\n            Collection<NormalisedTitle> allowedCats, Collection<NormalisedTitle> allowedPages, int depth,\n            boolean normalised) throws RuntimeException {\n        try {\n            connection.db.exec(\"CREATE TEMPORARY TABLE currentpages(cp_id INTEGER PRIMARY KEY ASC);\");\n            SiteInfo siteInfo = WikiDumpXml2SQLite.readSiteInfo(connection.db);\n            MyNamespace namespace = new MyNamespace(siteInfo);\n\n            Set<NormalisedTitle> allowedCatsFull = getSubCategories(\n                    allowedCats, namespace);\n\n            Set<NormalisedTitle> currentPages = new HashSet<NormalisedTitle>();\n            currentPages.addAll(allowedPages);\n            currentPages.addAll(allowedCatsFull);\n            currentPages.addAll(getPagesDirectlyInCategories(allowedCatsFull));\n\n            Set<NormalisedTitle> normalisedPages = getRecursivePages(currentPages, depth);\n            \n            // no need to drop table - we set temporary tables to be in-memory only\n//            connection.db.exec(\"DROP TABLE currentpages;\");\n\n            // note: need to sort case-sensitively (wiki is only case-insensitive at the first char)\n            final TreeSet<String> pages = new TreeSet<String>();\n            if (normalised) {\n                pages.addAll(ScalarisDataHandlerNormalised.normList2normStringList(normalisedPages));\n            } else {\n                MyWikiModel.denormalisePageTitles(normalisedPages, namespace, pages);\n            }\n            return pages;\n        } catch (SQLiteException e) {\n            error(\"read of pages in categories failed (sqlite error: \" + e.toString() + \")\");\n            throw new RuntimeException(e);\n        }\n    }\n    \n    /**\n     * Gets all sub-categories for the given ones from an SQLite database.\n     * \n     * Note: needs a (temporary) <tt>currentpages</tt> table to be set up before this\n     * call.\n     * \n     * @param allowedCats\n     *            include all pages in these categories (normalised page titles)\n     * \n     * @return the set of the given categories and all their sub-categories\n     *         (normalised)\n     * \n     * @throws SQLiteException\n     *             if an error occurs\n     */\n    private Set<NormalisedTitle> getSubCategories(\n            Collection<? extends NormalisedTitle> allowedCats,\n            MyNamespace nsObject) throws SQLiteException {\n        if (!allowedCats.isEmpty()) {\n            SQLiteStatement stmt = null;\n            SQLiteStatement stmtCount = null;\n            try {\n                println(\" determining sub-categories of \" + allowedCats.toString() + \"\");\n                // first insert all categories:\n                stmt = connection.db.prepare(\"REPLACE INTO currentpages (cp_id) SELECT page_id FROM page WHERE page_title == ?;\");\n                for (NormalisedTitle pageTitle : allowedCats) {\n                    if (pageTitle.namespace.equals(MyNamespace.CATEGORY_NAMESPACE_KEY)) {\n                        stmt.bind(1, pageTitle.toString()).stepThrough().reset();\n                    }\n                }\n                stmt.dispose();\n\n                // then recursively determine all sub categories:\n                stmt = connection.db\n                        .prepare(\"REPLACE INTO currentpages (cp_id) SELECT cl_from FROM categorylinks \"\n                                + \"INNER JOIN currentpages on cl_to == cp_id \"\n                                + \"INNER JOIN page ON cl_from == page_id WHERE page_namespace == \"\n                                + MyNamespace.CATEGORY_NAMESPACE_KEY + \";\");\n                stmtCount = connection.db\n                        .prepare(\"SELECT COUNT(*) FROM currentpages;\");\n                // note: not necessary to use the correct value at first\n                int oldCount = 0;\n                int newCount = 0;\n                do {\n                    oldCount = newCount;\n                    stmt.stepThrough().reset();\n                    if (stmtCount.step()) {\n                        newCount = stmtCount.columnInt(0);\n                    }\n                    stmtCount.reset();\n                    println(\"  added \" + newCount + \" categories\");\n                } while (oldCount != newCount);\n                stmt.dispose();\n                stmtCount.dispose();\n\n                // now read back all categories:\n                stmt = connection.db.prepare(\"SELECT page_title FROM currentpages INNER JOIN page on page_id == cp_id;\");\n                Set<NormalisedTitle> allowedCatsFull = new HashSet<NormalisedTitle>(newCount);\n                while(stmt.step()) {\n                    String title = stmt.columnString(0);\n                    allowedCatsFull.add(new NormalisedTitle(MyNamespace.CATEGORY_NAMESPACE_KEY, title));\n                }\n                stmt.reset();\n                connection.db.exec(\"DELETE FROM currentpages;\");\n                return allowedCatsFull;\n            } finally {\n                if (stmt != null) {\n                    stmt.dispose();\n                }\n                if (stmtCount != null) {\n                    stmtCount.dispose();\n                }\n            }\n        } else {\n            return new HashSet<NormalisedTitle>();\n        }\n    }\n    \n    /**\n     * Gets all pages in the given category set from an SQLite database. Note:\n     * sub-categories are not taken into account.\n     * \n     * @param allowedCats\n     *            include all pages in these categories (normalised page\n     *            titles)\n     * @param db\n     *            connection to the SQLite database\n     * \n     * @return a set of pages (directly) in the given categories (normalised\n     *         page titles)\n     * \n     * @throws SQLiteException\n     *             if an error occurs\n     */\n    private Set<NormalisedTitle> getPagesDirectlyInCategories(Set<NormalisedTitle> allowedCats) throws SQLiteException {\n        Set<NormalisedTitle> currentPages = new HashSet<NormalisedTitle>();\n\n        if (!allowedCats.isEmpty()) {\n            SQLiteStatement stmt = null;\n            try {\n                println(\" determining pages in any of the sub-categories\");\n                // first insert all categories:\n                stmt = connection.db\n                        .prepare(\"REPLACE INTO currentpages (cp_id) \"\n                                + \"SELECT page_id FROM page WHERE page_namespace == \"\n                                + MyNamespace.CATEGORY_NAMESPACE_KEY\n                                + \" page_title == ?;\");\n                for (NormalisedTitle pageTitle : allowedCats) {\n                    if (pageTitle.namespace.equals(MyNamespace.CATEGORY_NAMESPACE_KEY)) {\n                        stmt.bind(1, pageTitle.toString()).stepThrough().reset();\n                    }\n                }\n                stmt.dispose();\n                \n                // select all pages belonging to any of the allowed categories:\n                stmt = connection.db\n                        .prepare(\"SELECT page_namespace, page_title FROM currentpages \" +\n                                \"INNER JOIN categorylinks ON cl_to == cp_id \" +\n                                \"INNER JOIN page ON cl_from == page_id;\");\n                while (stmt.step()) {\n                    int namespace = stmt.columnInt(0);\n                    String title = stmt.columnString(1);\n                    currentPages.add(new NormalisedTitle(namespace, title));\n                }\n                stmt.dispose();\n                connection.db.exec(\"DELETE FROM currentpages;\");\n            } finally {\n                if (stmt != null) {\n                    stmt.dispose();\n                }\n            }\n        }\n        return currentPages;\n    }\n    \n    /**\n     * Gets all pages and their dependencies from an SQLite database, follows\n     * links recursively.\n     * \n     * Note: needs a (temporary) currentPages table to be set up before this\n     * call.\n     * \n     * @param currentPages\n     *            parse these pages recursively (normalised page titles)\n     * @param depth\n     *            follow links this deep\n     * \n     * @return a set of normalised page titles\n     * \n     * @throws SQLiteException\n     *             if an error occurs\n     */\n    private Set<NormalisedTitle> getRecursivePages(\n            Set<NormalisedTitle> currentPages, int depth)\n            throws SQLiteException {\n        Set<NormalisedTitle> allPages = new HashSet<NormalisedTitle>(100000);\n        SQLiteStatement stmt = null;\n        try {\n            println(\"adding all mediawiki pages\");\n            // add all auto-included pages\n            stmt = connection.db\n                    .prepare(\"REPLACE INTO currentpages (cp_id) SELECT page_id FROM page WHERE page_namespace == \"\n                            + MyNamespace.MEDIAWIKI_NAMESPACE_KEY + \";\");\n            stmt.stepThrough().reset();\n            stmt.dispose();\n\n            println(\"adding \" + currentPages.size() + \" pages\");\n            stmt = connection.db.prepare(\"REPLACE INTO currentpages (cp_id) SELECT page_id FROM page WHERE page_namespace == ? AND page_title == ?;\");\n            for (NormalisedTitle page : currentPages) {\n                stmt.bind(1, page.namespace).bind(2, page.title).stepThrough().reset();\n            }\n            stmt.dispose();\n            allPages.addAll(currentPages);\n            while(depth >= 0) {\n                println(\"recursion level: \" + depth);\n\n                println(\" adding categories of \" + allPages.size() + \" pages\");\n                // add all categories the page belongs to\n                stmt = connection.db\n                        .prepare(\"REPLACE INTO currentpages (cp_id) SELECT cl_to FROM categorylinks \"\n                                + \"INNER JOIN currentpages on cl_from == cp_id;\");\n                stmt.stepThrough().dispose();\n\n                println(\" adding templates of \" + allPages.size() + \" pages\");\n                // add all templates (and their requirements) of the pages\n                stmt = connection.db\n                        .prepare(\"REPLACE INTO currentpages (cp_id) SELECT tl_to FROM templatelinks \"\n                                + \"INNER JOIN currentpages on tl_from == cp_id;\");\n                stmt.stepThrough().dispose();\n\n                // now read back all pages (except auto-included ones):\n                stmt = connection.db\n                        .prepare(\"SELECT page_namespace,page_title FROM currentpages \"\n                                + \"INNER JOIN page on page_id == cp_id \" \n                                + \"WHERE page_namespace != \" + MyNamespace.MEDIAWIKI_NAMESPACE_KEY + \" ;\");\n                while(stmt.step()) {\n                    int namespace = stmt.columnInt(0);\n                    String title = stmt.columnString(1);\n                    allPages.add(new NormalisedTitle(namespace, title));\n                }\n                stmt.reset();\n\n                if (depth > 1) {\n                    println(\" adding links of \" + allPages.size() + \" pages\");\n                    // add all links of the pages for further processing\n                    stmt = connection.db\n                            .prepare(\"REPLACE INTO currentpages (cp_id) SELECT pl_to FROM pagelinks \"\n                                    + \"INNER JOIN currentpages on pl_from == cp_id;\");\n                    stmt.stepThrough().dispose();\n                }\n\n                --depth;\n            }\n            connection.db.exec(\"DELETE FROM currentPages;\");\n        } finally {\n            if (stmt != null) {\n                stmt.dispose();\n            }\n        }\n        return allPages;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/WikiDumpToScalarisHandler.java",
    "content": "/**\n *  Copyright 2007-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport java.util.Calendar;\nimport java.util.LinkedList;\nimport java.util.List;\nimport java.util.Map.Entry;\nimport java.util.Set;\nimport java.util.concurrent.ArrayBlockingQueue;\nimport java.util.concurrent.ExecutorService;\nimport java.util.concurrent.ThreadPoolExecutor;\nimport java.util.concurrent.TimeUnit;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.Connection;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.ConnectionFactory;\nimport de.zib.scalaris.Transaction;\nimport de.zib.scalaris.TransactionSingleOp;\nimport de.zib.scalaris.UnknownException;\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandler;\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandlerNormalised;\nimport de.zib.scalaris.examples.wikipedia.ScalarisDataHandlerUnnormalised;\nimport de.zib.scalaris.examples.wikipedia.ScalarisOpType;\nimport de.zib.scalaris.examples.wikipedia.ValueResult;\nimport de.zib.scalaris.examples.wikipedia.bliki.MyNamespace.NamespaceEnum;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\nimport de.zib.scalaris.examples.wikipedia.data.ShortRevision;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\nimport de.zib.scalaris.operations.WriteOp;\n\n/**\n * Provides abilities to read an xml wiki dump file and write its contents to\n * the Scalaris.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class WikiDumpToScalarisHandler extends WikiDumpPageHandler {\n    private static final int MAX_SCALARIS_CONNECTIONS = Runtime.getRuntime().availableProcessors() * 2;\n    private ArrayBlockingQueue<TransactionSingleOp> scalaris_single = new ArrayBlockingQueue<TransactionSingleOp>(MAX_SCALARIS_CONNECTIONS);\n    private ArrayBlockingQueue<Transaction> scalaris_tx = new ArrayBlockingQueue<Transaction>(MAX_SCALARIS_CONNECTIONS);\n    private ExecutorService executor = createExecutor(MAX_SCALARIS_CONNECTIONS);\n    private ExecutorService pageListExecutor = createExecutor(1);\n\n    /**\n     * Sets up a SAX XmlHandler exporting all parsed pages except the ones in a\n     * blacklist to stdout.\n     * \n     * @param blacklist\n     *            a number of page titles to ignore\n     * @param whitelist\n     *            only import these pages\n     * @param maxRevisions\n     *            maximum number of revisions per page (starting with the most\n     *            recent) - <tt>-1/tt> imports all revisions\n     *            (useful to speed up the import / reduce the DB size)\n     * @param minTime\n     *            minimum time a revision should have (only one revision older\n     *            than this will be imported) - <tt>null/tt> imports all\n     *            revisions\n     * @param maxTime\n     *            maximum time a revision should have (newer revisions are\n     *            omitted) - <tt>null/tt> imports all revisions\n     *            (useful to create dumps of a wiki at a specific point in time)\n     * \n     * @throws RuntimeException\n     *             if the connection to Scalaris fails\n     */\n    public WikiDumpToScalarisHandler(Set<String> blacklist,\n            Set<String> whitelist, int maxRevisions, Calendar minTime,\n            Calendar maxTime) throws RuntimeException {\n        super(blacklist, whitelist, maxRevisions, minTime, maxTime);\n        init(ConnectionFactory.getInstance());\n    }\n\n    /**\n     * Sets up a SAX XmlHandler exporting all parsed pages except the ones in a\n     * blacklist to stdout.\n     * \n     * @param blacklist\n     *            a number of page titles to ignore\n     * @param whitelist\n     *            only import these pages\n     * @param maxRevisions\n     *            maximum number of revisions per page (starting with the most\n     *            recent) - <tt>-1/tt> imports all revisions\n     *            (useful to speed up the import / reduce the DB size)\n     * @param minTime\n     *            minimum time a revision should have (only one revision older\n     *            than this will be imported) - <tt>null/tt> imports all\n     *            revisions\n     * @param maxTime\n     *            maximum time a revision should have (newer revisions are\n     *            omitted) - <tt>null/tt> imports all revisions\n     *            (useful to create dumps of a wiki at a specific point in time)\n     * @param cFactory\n     *            the connection factory to use for creating new connections\n     * \n     * @throws RuntimeException\n     *             if the connection to Scalaris fails\n     */\n    public WikiDumpToScalarisHandler(Set<String> blacklist,\n            Set<String> whitelist, int maxRevisions, Calendar maxTime,\n            Calendar minTime, ConnectionFactory cFactory)\n            throws RuntimeException {\n        super(blacklist, whitelist, maxRevisions, minTime, maxTime);\n        init(cFactory);\n    }\n\n    /**\n     * Sets up connections to Scalaris.\n     * \n     * @param cFactory\n     *            the connection factory to use for creating new connections\n     * \n     * @throws RuntimeException\n     *             if the connection to Scalaris fails\n     */\n    private void init(ConnectionFactory cFactory) throws RuntimeException {\n        try {\n            for (int i = 0; i < MAX_SCALARIS_CONNECTIONS; ++i) {\n                Connection connection = cFactory.createConnection(\n                        \"wiki_import\", true);\n                scalaris_single.put(new TransactionSingleOp(connection));\n                connection = cFactory.createConnection(\n                        \"wiki_import\", true);\n                scalaris_tx.put(new Transaction(connection));\n            }\n        } catch (ConnectionException e) {\n            error(\"Connection to Scalaris failed\");\n            throw new RuntimeException(e);\n        } catch (InterruptedException e) {\n            error(\"Interrupted while setting up multiple connections to Scalaris\");\n            throw new RuntimeException(e);\n        }\n    }\n\n    /**\n     * Creates a new {@link ThreadPoolExecutor} used during the import to\n     * process import jobs.\n     * \n     * @param nThreads\n     *            the (fixed) number of threads to use\n     * \n     * @return a {@link ThreadPoolExecutor} with a bounded queue of length\n     *         <tt>nThreads * 10</tt> that runs the job in the calling task if\n     *         the queue is full\n     */\n    public static ThreadPoolExecutor createExecutor(int nThreads) {\n        return new ThreadPoolExecutor(nThreads, nThreads, 0L,\n                TimeUnit.MILLISECONDS, new ArrayBlockingQueue<Runnable>(\n                        nThreads * 10),\n                new ThreadPoolExecutor.CallerRunsPolicy());\n    }\n\n    @Override\n    protected void doExport(SiteInfo siteinfo) throws RuntimeException {\n        String key = ScalarisDataHandler.getSiteInfoKey();\n        TransactionSingleOp scalaris_single;\n        try {\n            scalaris_single = this.scalaris_single.take();\n        } catch (InterruptedException e) {\n            error(\"write of \" + key + \" interrupted while getting connection to Scalaris\");\n            throw new RuntimeException(e);\n        }\n        try {\n            scalaris_single.write(key, siteinfo);\n        } catch (ConnectionException e) {\n            error(\"write of \" + key + \" failed with connection error\");\n        } catch (AbortException e) {\n            error(\"write of \" + key + \" failed with abort\");\n        } catch (UnknownException e) {\n            error(\"write of \" + key + \" failed with unknown\");\n        }\n        if (scalaris_single != null) {\n            try {\n                this.scalaris_single.put(scalaris_single);\n            } catch (InterruptedException e) {\n                error(\"Interrupted while putting back a connection to Scalaris\");\n                throw new RuntimeException(e);\n            }\n        }\n    }\n    \n    @Override\n    protected void doExport(Page page, List<Revision> revisions,\n            List<ShortRevision> revisions_short, NormalisedTitle title)\n            throws UnsupportedOperationException {\n        // do not make the translog too full -> write revisions beforehand,\n        // ignore the (rest of the) page if a failure occured\n        TransactionSingleOp.RequestList requests = new TransactionSingleOp.RequestList();\n        for (Revision rev : revisions) {\n            if (rev.getId() != page.getCurRev().getId()) {\n                String key = ScalarisDataHandlerUnnormalised.getRevKey(page.getTitle(), rev.getId(), wikiModel.getNamespace());\n                requests.addOp(new WriteOp(key, rev));\n            }\n        }\n        requests.addOp(new WriteOp(ScalarisDataHandlerUnnormalised.getRevListKey(page.getTitle(), wikiModel.getNamespace()), revisions_short));\n        requests.addOp(new WriteOp(ScalarisDataHandlerUnnormalised.getPageKey(page.getTitle(), wikiModel.getNamespace()), page));\n        Runnable worker = new MyScalarisSingleRunnable(this, requests,\n                scalaris_single, \"revisions and page of \" + page.getTitle());\n        executor.execute(worker);\n        newPages.get(NamespaceEnum.fromId(title.namespace)).add(title);\n        // only export page list every UPDATE_PAGELIST_EVERY pages:\n        if ((pageCount % UPDATE_PAGELIST_EVERY) == 0) {\n            updatePageLists();\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpHandler#tearDown()\n     */\n    @Override\n    public void tearDown() {\n        super.tearDown();\n        updatePageLists();\n        importEnd();\n    }\n    \n    private void updatePageLists() {\n        String scalaris_key;\n        Runnable worker;\n        \n        // list of pages:\n        for(NamespaceEnum ns : NamespaceEnum.values()) {\n            scalaris_key = ScalarisDataHandler.getPageListKey(ns.getId());\n            worker = new MyScalarisAddToPageListRunnable(this, scalaris_key,\n                    newPages.get(ns), scalaris_tx, ScalarisOpType.PAGE_LIST,\n                    ScalarisDataHandler.getPageCountKey(ns.getId()),\n                    ScalarisOpType.PAGE_COUNT);\n            pageListExecutor.execute(worker);\n        }\n        initNewPagesList();\n        \n        // articles count:\n        TransactionSingleOp.RequestList requests = new TransactionSingleOp.RequestList();\n        requests.addOp(new WriteOp(ScalarisDataHandler.getArticleCountKey(), articleCount));\n        worker = new MyScalarisSingleRunnable(this, requests, scalaris_single, \"ARTICLE_COUNT\");\n        pageListExecutor.execute(worker);\n        \n        // list of pages in each category:\n        for (Entry<NormalisedTitle, List<NormalisedTitle>> category: newCategories.entrySet()) {\n            scalaris_key = ScalarisDataHandlerNormalised.getCatPageListKey(category.getKey());\n            worker = new MyScalarisAddToPageListRunnable(this, scalaris_key,\n                    category.getValue(), scalaris_tx, ScalarisOpType.CATEGORY_PAGE_LIST,\n                    ScalarisDataHandlerNormalised.getCatPageCountKey(category.getKey()),\n                    ScalarisOpType.CATEGORY_PAGE_COUNT);\n            pageListExecutor.execute(worker);\n        }\n\n        // list of pages a template is used in:\n        for (Entry<NormalisedTitle, List<NormalisedTitle>> template: newTemplates.entrySet()) {\n            scalaris_key = ScalarisDataHandlerNormalised.getTplPageListKey(template.getKey());\n            worker = new MyScalarisAddToPageListRunnable(this, scalaris_key,\n                    template.getValue(), scalaris_tx,\n                    ScalarisOpType.TEMPLATE_PAGE_LIST, null, null);\n            pageListExecutor.execute(worker);\n        }\n        \n        // list of pages linking to other pages:\n        for (Entry<NormalisedTitle, List<NormalisedTitle>> backlinks: newBackLinks.entrySet()) {\n            scalaris_key = ScalarisDataHandlerNormalised.getBackLinksPageListKey(backlinks.getKey());\n            worker = new MyScalarisAddToPageListRunnable(this, scalaris_key,\n                    backlinks.getValue(), scalaris_tx,\n                    ScalarisOpType.BACKLINK_PAGE_LIST, null, null);\n            pageListExecutor.execute(worker);\n        }\n        initLinkLists();\n\n        executor.shutdown();\n        pageListExecutor.shutdown();\n        boolean shutdown = false;\n        while (!shutdown) {\n            try {\n                shutdown = executor.awaitTermination(1, TimeUnit.MINUTES)\n                        && pageListExecutor.awaitTermination(1, TimeUnit.MINUTES);\n            } catch (InterruptedException e) {\n            }\n        }\n        executor = createExecutor(MAX_SCALARIS_CONNECTIONS);\n        pageListExecutor = createExecutor(1);\n    }\n\n    @Override\n    public boolean isErrorDuringImport() {\n        return errorDuringImport;\n    }\n\n    @Override\n    public void error(String message) {\n        System.err.println(message);\n        errorDuringImport = true;\n    }\n\n    /**\n     * Processes write requests to Scalaris in a separate thread. Takes one of\n     * the available {@link #scalarisSingleQueue} connections.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    static class MyScalarisSingleRunnable implements Runnable {\n        private final TransactionSingleOp.RequestList requests;\n        private final ArrayBlockingQueue<TransactionSingleOp> scalarisSingleQueue;\n        private final String note;\n        private final WikiDump importer;\n        private TransactionSingleOp scalaris_single;\n        \n        public MyScalarisSingleRunnable(WikiDump importer,\n                TransactionSingleOp.RequestList requests,\n                ArrayBlockingQueue<TransactionSingleOp> scalarisSingleQueue,\n                String note) {\n            this.importer = importer;\n            this.requests = requests;\n            this.scalarisSingleQueue = scalarisSingleQueue;\n            this.note = note;\n            try {\n                this.scalaris_single = this.scalarisSingleQueue.take();\n            } catch (InterruptedException e) {\n                this.importer.error(\"write of \" + note + \" interrupted while getting connection to Scalaris\");\n                throw new RuntimeException(e);\n            }\n        }\n        \n        @Override\n        public void run() {\n            try {\n                TransactionSingleOp.ResultList results = scalaris_single.req_list(requests);\n                for (int i = 0; i < results.size(); ++i) {\n                    results.processWriteAt(i);\n                }\n            } catch (ConnectionException e) {\n                importer.error(\"write of \" + note + \" failed with connection error\");\n            } catch (AbortException e) {\n                importer.error(\"write of \" + note + \" failed with abort\");\n            }\n            if (scalaris_single != null) {\n                try {\n                    this.scalarisSingleQueue.put(scalaris_single);\n                } catch (InterruptedException e) {\n                    importer.error(\"Interrupted while putting back a connection to Scalaris\");\n                    throw new RuntimeException(e);\n                }\n            }\n        }\n    }\n\n    /**\n     * Processes page list update requests to Scalaris in a separate thread.\n     * Takes one of the available {@link #scalarisTxQueue} connections.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    private static class MyScalarisAddToPageListRunnable implements Runnable {\n        private final String scalaris_pageList_key;\n        private final List<NormalisedTitle> newEntries;\n        private final ArrayBlockingQueue<Transaction> scalarisTxQueue;\n        private final String scalaris_pageCount_key;\n        private final ScalarisOpType opType;\n        private final ScalarisOpType countOpType;\n        private final WikiDump importer;\n        private Transaction scalaris_tx;\n        \n        public MyScalarisAddToPageListRunnable(WikiDump importer,\n                String scalaris_pageList_key, List<NormalisedTitle> newEntries,\n                ArrayBlockingQueue<Transaction> scalarisTxQueue,\n                ScalarisOpType opType, String scalaris_pageCount_key,\n                ScalarisOpType countOpType) {\n            this.importer = importer;\n            this.scalaris_pageList_key = scalaris_pageList_key;\n            this.newEntries = newEntries;\n            this.scalarisTxQueue = scalarisTxQueue;\n            this.opType = opType;\n            this.scalaris_pageCount_key = scalaris_pageCount_key;\n            this.countOpType = countOpType;\n            try {\n                this.scalaris_tx = this.scalarisTxQueue.take();\n            } catch (InterruptedException e) {\n                this.importer.error(\"update of \" + scalaris_pageList_key + \" and \" + scalaris_pageCount_key + \" interrupted while getting connection to Scalaris\");\n                throw new RuntimeException(e);\n            }\n        }\n        \n        @Override\n        public void run() {\n            ValueResult<Integer> result = ScalarisDataHandlerNormalised.updatePageList(\n                    scalaris_tx, opType, scalaris_pageList_key,\n                    countOpType, scalaris_pageCount_key, newEntries,\n                    new LinkedList<NormalisedTitle>(), \"\");\n            if (!result.success) {\n                importer.error(result.message);\n            }\n            \n            if (scalaris_tx != null) {\n                try {\n                    this.scalarisTxQueue.put(scalaris_tx);\n                } catch (InterruptedException e) {\n                    importer.error(\"update of \" + scalaris_pageList_key + \" and \" + scalaris_pageCount_key + \" interrupted while putting back a connection to Scalaris\");\n                    throw new RuntimeException(e);\n                }\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/WikiDumpXml2SQLite.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport java.io.ByteArrayInputStream;\nimport java.io.ByteArrayOutputStream;\nimport java.io.FileNotFoundException;\nimport java.io.IOException;\nimport java.io.ObjectInputStream;\nimport java.io.ObjectOutputStream;\nimport java.util.Calendar;\nimport java.util.List;\nimport java.util.Set;\nimport java.util.concurrent.ArrayBlockingQueue;\nimport java.util.zip.GZIPInputStream;\nimport java.util.zip.GZIPOutputStream;\n\nimport com.almworks.sqlite4java.SQLiteConnection;\nimport com.almworks.sqlite4java.SQLiteException;\nimport com.almworks.sqlite4java.SQLiteStatement;\n\nimport de.zib.scalaris.examples.wikipedia.SQLiteDataHandler;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\n\n/**\n * Provides abilities to read an xml wiki dump file and write its contents into\n * a local SQLite db.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class WikiDumpXml2SQLite extends WikiDumpHandler {\n    private static final int PRINT_PAGES_EVERY = 500;\n    protected SQLiteConnection db = null;\n    protected SQLiteStatement stWritePage = null;\n    protected SQLiteStatement stWriteRevision = null;\n    protected SQLiteStatement stWriteText = null;\n    final protected String dbFileName;\n    protected long nextPageId = 0l;\n    protected ArrayBlockingQueue<SQLiteJob> sqliteJobs = new ArrayBlockingQueue<SQLiteJob>(10);\n    SQLiteWorker sqliteWorker = new SQLiteWorker();\n    private SiteInfo siteInfo = null;\n    \n    /**\n     * Sets up a SAX XmlHandler exporting all parsed pages except the ones in a\n     * blacklist to Scalaris but with an additional pre-process phase.\n     * \n     * @param blacklist\n     *            a number of page titles to ignore\n     * @param whitelist\n     *            only import these pages\n     * @param maxRevisions\n     *            maximum number of revisions per page (starting with the most\n     *            recent) - <tt>-1/tt> imports all revisions\n     *            (useful to speed up the import / reduce the DB size)\n     * @param minTime\n     *            minimum time a revision should have (only one revision older\n     *            than this will be imported) - <tt>null/tt> imports all\n     *            revisions\n     * @param maxTime\n     *            maximum time a revision should have (newer revisions are\n     *            omitted) - <tt>null/tt> imports all revisions\n     *            (useful to create dumps of a wiki at a specific point in time)\n     * @param dbFileName\n     *            the name of the database file to write to\n     * \n     * @throws RuntimeException\n     *             if the connection to Scalaris fails\n     */\n    public WikiDumpXml2SQLite(Set<String> blacklist,\n            Set<String> whitelist, int maxRevisions, Calendar minTime,\n            Calendar maxTime, String dbFileName) throws RuntimeException {\n        super(blacklist, whitelist, maxRevisions, minTime, maxTime);\n        this.dbFileName = dbFileName;\n    }\n    \n    protected void addSQLiteJob(SQLiteJob job) throws RuntimeException {\n        try {\n            sqliteJobs.put(job);\n        } catch (InterruptedException e) {\n            throw new RuntimeException(e);\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpPrepareForScalarisHandler#setUp()\n     */\n    @Override\n    public void setUp() {\n        super.setUp();\n        println(\"Pre-processing pages from XML to SQLite DB...\");\n        sqliteWorker.start();\n        // wait for worker to initialise the DB and the prepared statements\n        while (!sqliteWorker.initialised) {\n            try {\n                Thread.sleep(100);\n            } catch (InterruptedException e) {\n                throw new RuntimeException(e);\n            }\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.data.xml.WikiDumpHandler#tearDown()\n     */\n    @Override\n    public void tearDown() {\n        super.tearDown();\n        sqliteWorker.stopWhenQueueEmpty = true;\n        addSQLiteJob(new SQLiteNoOpJob());\n        // wait for worker to close the DB\n        try {\n            sqliteWorker.join();\n        } catch (InterruptedException e) {\n            throw new RuntimeException(e);\n        }\n        importEnd();\n    }\n\n    /* (non-Javadoc)\n     * @see java.lang.Object#finalize()\n     */\n    @Override\n    protected void finalize() throws Throwable {\n        super.finalize();\n        db.dispose();\n    }\n\n    @Override\n    protected void export(XmlSiteInfo siteinfo_xml) {\n        this.siteInfo  = siteinfo_xml.getSiteInfo();\n    }\n\n    /**\n     * @param <T>\n     * @param siteinfo\n     * @param key\n     * @throws IOException\n     * @throws FileNotFoundException\n     */\n    static <T> void writeObject(SQLiteStatement stWrite, String key, T value)\n            throws RuntimeException {\n        try {\n            try {\n                stWrite.bind(1, key).bind(2, objectToBytes(value)).stepThrough();\n            } finally {\n                stWrite.reset();\n            }\n        } catch (SQLiteException e) {\n            System.err.println(\"write of \" + key + \" failed (sqlite error: \" + e.toString() + \")\");\n            throw new RuntimeException(e);\n        } catch (IOException e) {\n            System.err.println(\"write of \" + key + \" failed\");\n            throw new RuntimeException(e);\n        }\n    }\n\n    /**\n     * Reads a byte-encoded object from a SQLite DB.\n     * \n     * Note: may need to qualify static function call due to\n     * http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6302954\n     * <tt>WikiDumpXml2SQLite.<T>readObject(stRead, key)</tt>\n     * \n     * @param <T>\n     *            the type to convert the object to\n     * \n     * @param stRead\n     *            read statement\n     * @param key\n     *            key to read from\n     * \n     * @return the decoded object\n     * \n     * @throws IOException\n     * @throws FileNotFoundException\n     */\n    static <T> T readObject(SQLiteStatement stRead, String key)\n            throws RuntimeException, FileNotFoundException {\n        try {\n            try {\n                stRead.bind(1, key);\n                if (stRead.step()) {\n                    // there should only be one result\n                    byte[] value = stRead.columnBlob(0);\n                    return WikiDumpXml2SQLite.<T>objectFromBytes(value);\n                } else {\n                    throw new FileNotFoundException();\n                }\n            } finally {\n                stRead.reset();\n            }\n        } catch (FileNotFoundException e) {\n            throw e;\n        } catch (SQLiteException e) {\n            System.err.println(\"read of \" + key + \" failed (sqlite error: \" + e.toString() + \")\");\n            throw new RuntimeException(e);\n        } catch (ClassNotFoundException e) {\n            System.err.println(\"read of \" + key + \" failed (error: \" + e.toString() + \")\");\n            throw new RuntimeException(e);\n        } catch (IOException e) {\n            System.err.println(\"read of \" + key + \" failed (error: \" + e.toString() + \")\");\n            throw new RuntimeException(e);\n        }\n    }\n\n    /**\n     * Converts this value to bytes for use by SQLite.\n     * \n     * @param value\n     *            the object to convert\n     * \n     * @return the byte array\n     * \n     * @throws IOException\n     * \n     * @see {@link #objectFromBytes(byte[])}\n     */\n    static <T> byte[] objectToBytes(T value) throws IOException {\n        ByteArrayOutputStream bos = new ByteArrayOutputStream();\n        ObjectOutputStream oos = new ObjectOutputStream(new GZIPOutputStream(bos));\n        oos.writeObject(value);\n        oos.flush();\n        oos.close();\n        return bos.toByteArray();\n    }\n\n    /**\n     * Reads an object from a (compressed) byte array.\n     * \n     * Note: may need to qualify static function call due to\n     *  http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6302954\n     * <tt>WikiDumpXml2SQLite.<T>objectFromBytes(value)</tt>\n     * \n     * @param <T>\n     *            the type to convert the object to\n     * \n     * @param value\n     *            the byte array to get the object from\n     * \n     * @return the encoded object\n     * \n     * @throws IOException\n     * @throws ClassNotFoundException\n     * \n     * @see #objectToBytes(Object)\n     */\n    static <T> T objectFromBytes(byte[] value) throws IOException,\n            ClassNotFoundException {\n        ObjectInputStream ois = new ObjectInputStream(\n                new GZIPInputStream(new ByteArrayInputStream(value)));\n        @SuppressWarnings(\"unchecked\")\n        T result = (T) ois.readObject();\n        ois.close();\n        return result;\n    }\n\n    /**\n     * Reads the site info object from the given DB.\n     * \n     * @param db\n     *            the DB which was previously prepared with this class\n     * \n     * @return a site info object\n     * \n     * @throws RuntimeException\n     */\n    public static SiteInfo readSiteInfo(SQLiteConnection db) throws RuntimeException {\n        SQLiteStatement stmt = null;\n        try {\n            stmt = db.prepare(\"SELECT value FROM properties WHERE key == ?\");\n            return WikiDumpXml2SQLite.<SiteInfo>readObject(stmt, \"siteinfo\");\n        } catch (SQLiteException e) {\n            throw new RuntimeException(e);\n        } catch (FileNotFoundException e) {\n            throw new RuntimeException(e);\n        } finally {\n            if (stmt != null) {\n                stmt.dispose();\n            }\n        }\n    }\n\n    @Override\n    protected void export(XmlPage page_xml) {\n        ++pageCount;\n        final Page page = page_xml.getPage();\n        if (page.getCurRev() != null) {\n            addSQLiteJob(new SQLiteWritePageJob(page, page_xml.getRevisions()));\n        }\n        if ((pageCount % PRINT_PAGES_EVERY) == 0) {\n            println(\"processed pages: \" + pageCount);\n        }\n    }\n    \n    protected class SQLiteWorker extends Thread {\n        boolean stopWhenQueueEmpty = false;\n        boolean initialised = false;\n        \n        @Override\n        public void run() {\n            try {\n                // set up DB:\n                try {\n                    // set 1GB cache_size:\n                    db = SQLiteDataHandler.openDB(dbFileName, false, 1024l*1024l*1024l);\n\n                    /**\n                     * Table storing the siteinfo object.\n                     */\n                    db.exec(\"CREATE TABLE properties(key STRING PRIMARY KEY ASC, value);\");\n                    \n                    /**\n                     * Core of the wiki: each page has an entry here which identifies\n                     * it by title and contains some essential metadata.\n                     */\n                    final String createPageTable = \"CREATE TABLE page (\"\n                            + \"page_id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,\"\n                            + \"page_namespace int NOT NULL,\"\n                            + \"page_title varchar(255) NOT NULL,\"\n                            + \"page_restrictions tinyblob NOT NULL,\"\n                            + \"page_is_redirect tinyint unsigned NOT NULL default 0,\"\n                            + \"page_latest int unsigned NOT NULL,\"\n                            + \"page_len int unsigned NOT NULL\"\n                            + \");\";\n                    db.exec(createPageTable);\n                    // create index here as we need it during import:\n                    db.exec(\"CREATE UNIQUE INDEX name_title ON page (page_namespace,page_title);\");\n                    \n                    /**\n                     * Every edit of a page creates also a revision row.\n                     * This stores metadata about the revision, and a reference\n                     * to the text storage backend.\n                     */\n                    final String createRevTable = \"CREATE TABLE revision (\"\n                            + \"rev_id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,\"\n                            + \"rev_page int unsigned NOT NULL,\"\n                            + \"rev_text_id int unsigned NOT NULL,\"\n                            + \"rev_comment tinyblob NOT NULL,\"\n                            + \"rev_user_text varchar(255) NOT NULL default '',\"\n                            + \"rev_timestamp binary(14) NOT NULL default '',\"\n                            + \"rev_minor_edit tinyint unsigned NOT NULL default 0,\"\n                            + \"rev_len int unsigned\"\n                            + \");\";\n                    db.exec(createRevTable);\n                    \n                    /**\n                     * Holds text of individual page revisions.\n                     */\n                    final String createTextTable = \"CREATE TABLE text (\"\n                            + \"old_id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,\"\n                            + \"old_text mediumblob NOT NULL,\"\n                            + \"old_flags tinyblob NOT NULL\"\n                            + \");\";\n                    db.exec(createTextTable);\n                    \n                    stWritePage = db\n                            .prepare(\"INSERT INTO page \"\n                                    + \"(page_id, page_namespace, page_title, page_restrictions, page_is_redirect, page_latest, page_len) \"\n                                    + \"VALUES (?, ?, ?, ?, ?, ?, ?);\");\n                    stWriteRevision = db\n                            .prepare(\"INSERT INTO revision \"\n                                    + \"(rev_id, rev_page, rev_text_id, rev_comment, rev_user_text, rev_timestamp, rev_minor_edit, rev_len) \"\n                                    + \"VALUES (?, ?, ?, ?, ?, ?, ?, ?);\");\n                    stWriteText = db.prepare(\"INSERT INTO text \"\n                            + \"(old_text, old_flags) \"\n                            + \"VALUES (?, 'utf8,gzip')\");\n                } catch (SQLiteException e) {\n                    throw new RuntimeException(e);\n                }\n                initialised = true;\n                \n                // take jobs\n                while(!(sqliteJobs.isEmpty() && stopWhenQueueEmpty)) {\n                    SQLiteJob job;\n                    try {\n                        job = sqliteJobs.take();\n                    } catch (InterruptedException e) {\n                        throw new RuntimeException(e);\n                    }\n                    job.run();\n                }\n\n                // write siteinfo object last (may not have been initialised at the beginning):\n                SQLiteStatement stmt = null;\n                try {\n                    stmt = db.prepare(\"REPLACE INTO properties (key, value) VALUES (?, ?);\");\n                    writeObject(stmt, \"siteinfo\", siteInfo);\n                } catch (SQLiteException e) {\n                    throw new RuntimeException(e);\n                } finally {\n                    if (stmt != null) {\n                        stmt.dispose();\n                    }\n                }\n                \n                // create indices at last (improves performance)\n                try {\n                    db.exec(\"CREATE INDEX page_len ON page (page_len);\");\n                    db.exec(\"CREATE UNIQUE INDEX rev_page_id ON revision (rev_page, rev_id);\");\n                    db.exec(\"CREATE INDEX rev_timestamp ON revision (rev_timestamp);\");\n                    db.exec(\"CREATE INDEX page_timestamp ON revision (rev_page,rev_timestamp);\");\n                    db.exec(\"CREATE INDEX usertext_timestamp ON revision (rev_user_text,rev_timestamp);\");\n                } catch (SQLiteException e) {\n                    throw new RuntimeException(e);\n                }\n            } finally {\n                if (stWritePage != null) {\n                    stWritePage.dispose();\n                }\n                if (stWriteRevision != null) {\n                    stWriteRevision.dispose();\n                }\n                if (stWriteText != null) {\n                    stWriteText.dispose();\n                }\n                if (db != null) {\n                    db.dispose();\n                }\n                initialised = false;\n            }\n        }\n    }\n    \n    protected static interface SQLiteJob {\n        public abstract void run();\n    }\n    \n    protected static class SQLiteNoOpJob implements SQLiteJob {\n        @Override\n        public void run() {\n        }\n    }\n    \n    protected class SQLiteWritePageJob implements SQLiteJob {\n        Page page;\n        List<Revision> revisions;\n        \n        public SQLiteWritePageJob(Page page, List<Revision> revisions) {\n            this.page = page;\n            this.revisions = revisions;\n        }\n        \n        @Override\n        public void run() {\n            NormalisedTitle normTitle = NormalisedTitle.fromUnnormalised(page.getTitle(), wikiModel.getNamespace());\n            // check if page exists:\n            try {\n                final SQLiteStatement stmt = db.prepare(\"SELECT page_id FROM page WHERE page_namespace == ? AND page_title = ?;\");\n                stmt.bind(1, normTitle.namespace).bind(2, normTitle.title);\n                if (stmt.step()) {\n                    // exists\n                    error(\"duplicate page in dump: \"\n                            + page.getTitle() + \" (=\" + normTitle.toString()\n                            + \")\");\n                    return;\n                }\n                stmt.dispose();\n            } catch (SQLiteException e) {\n                error(\"existance check of \" + page.getTitle() + \" failed (sqlite error: \" + e.toString() + \")\");\n                throw new RuntimeException(e);\n            }\n            \n            \n            String pageRestrictions = Page.restrictionsToString(page.getRestrictions());\n            final int pageId = page.getId();\n            try {\n                try {\n                    stWritePage.bind(1, pageId).bind(2, normTitle.namespace)\n                            .bind(3, normTitle.title).bind(4, pageRestrictions)\n                            .bind(5, page.isRedirect() ? 1 : 0)\n                            .bind(6, page.getCurRev().getId())\n                            .bind(7, page.getCurRev().unpackedText().getBytes().length);\n                    stWritePage.stepThrough();\n                } finally {\n                    stWritePage.reset();\n                }\n                for (Revision rev : revisions) {\n                    int revTextId = -1;\n                    try {\n                        stWriteText.bind(1, rev.packedText());\n                        stWriteText.stepThrough();\n                        // get the text's ID:\n                        final SQLiteStatement stmt = db.prepare(\"SELECT last_insert_rowid();\");\n                        if (stmt.step()) {\n                            revTextId = stmt.columnInt(0);\n                        }\n                        stmt.dispose();\n                    } catch (SQLiteException e) {\n                        error(\"write of text for \" + page.getTitle() + \", rev \" + rev.getId() + \" failed (sqlite error: \" + e.toString() + \")\");\n                        throw new RuntimeException(e);\n                    } finally {\n                        stWriteText.reset();\n                    }\n                    try {\n                        stWriteRevision.bind(1, rev.getId()).bind(2, pageId)\n                                .bind(3, revTextId)\n                                .bind(4, rev.getComment())\n                                .bind(5, rev.getContributor().toString())\n                                .bind(6, rev.getTimestamp())\n                                .bind(7, rev.isMinor() ? 1 : 0)\n                                .bind(8, rev.unpackedText().getBytes().length);\n                        stWriteRevision.stepThrough();\n                    } catch (SQLiteException e) {\n                        error(\"write of \" + page.getTitle() + \", rev \" + rev.getId() + \" failed (sqlite error: \" + e.toString() + \")\");\n                        throw new RuntimeException(e);\n                    } finally {\n                        stWriteRevision.reset();\n                    }\n                }\n            } catch (SQLiteException e) {\n                error(\"write of \" + page.getTitle() + \" failed (sqlite error: \" + e.toString() + \")\");\n                throw new RuntimeException(e);\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/XmlContributor.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport org.xml.sax.Attributes;\nimport org.xml.sax.helpers.DefaultHandler;\n\nimport de.zib.scalaris.examples.wikipedia.data.Contributor;\n\n/**\n * Contributor known as a registered user for use by an XML reader.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class XmlContributor extends DefaultHandler {\n    protected StringBuilder currentString = new StringBuilder();\n    protected String user;\n    protected String id;\n    protected String ip;\n    \n    protected Contributor final_contributor;\n\n    /**\n     * Creates a new contributor with a (temporarily) empty username and ID.\n     * \n     */\n    public XmlContributor() {\n        super();\n        init();\n    }\n\n    /**\n     * (Re-) Initialises all instance variables.\n     */\n    private void init() {\n        currentString.setLength(0);\n        user = \"\";\n        id = \"\";\n        ip = \"\";\n        final_contributor = null;\n    }\n    \n    /**\n     * Resets all instance variables. Afterwards, the object has the same state\n     * as a newly created one.\n     */\n    public void reset() {\n        init();\n    }\n    \n    /**\n     * Called to when a starting contributor element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     * @param attributes\n     *            The attributes attached to the element. If there are no\n     *            attributes, it shall be an empty Attributes object.\n     */\n    public void startContributor(String uri, String localName, String qName,\n            Attributes attributes) {\n        // nothing to do\n    }\n    \n    /**\n     * Called to when a starting element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     * @param attributes\n     *            The attributes attached to the element. If there are no\n     *            attributes, it shall be an empty Attributes object.\n     */\n    @Override\n    public void startElement(String uri, String localName, String qName,\n            Attributes attributes) {\n        // System.out.println(localName);\n        currentString.setLength(0);\n        /*\n         * <ip>127.0.0.1</ip>\n         * \n         * or\n         * \n         * <username>Melancholie</username> <id>12</id>\n         */\n        if (localName.equals(\"username\")) {\n        } else if (localName.equals(\"id\")) {\n        } else if (localName.equals(\"ip\")) {\n        } else {\n            System.err.println(\"unknown contributor tag: \" + localName);\n        }\n    }\n\n    /**\n     * Called to process character data.\n     * \n     * Note: a SAX driver is free to chunk the character data any way it wants,\n     * so you cannot count on all of the character data content of an element\n     * arriving in a single characters event.\n     * \n     * @param ch\n     *            The characters.\n     * @param start\n     *            The start position in the character array.\n     * @param length\n     *            The number of characters to use from the character array.\n     */\n    @Override\n    public void characters(char[] ch, int start, int length) {\n        // System.out.println(new String(ch, start, length));\n        currentString.append(ch, start, length);\n    }\n\n    /**\n     * Called to when an ending contributor element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     */\n    public void endContributor(String uri, String localName, String qName) {\n//        System.out.println(\"ip: \" + ip + \", user: \" + user + \", id: \" + id);\n        final_contributor = new Contributor();\n        final_contributor.setIp(ip);\n        if (!id.isEmpty()) {\n            final_contributor.setId(Integer.parseInt(id));\n        }\n        final_contributor.setUser(user);\n    }\n\n    /**\n     * Called to when an ending element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     */\n    @Override\n    public void endElement(String uri, String localName, String qName) {\n        if (localName.equals(\"username\")) {\n            user = currentString.toString();\n        } else if (localName.equals(\"id\")) {\n            id = currentString.toString();\n        } else if (localName.equals(\"ip\")) {\n            ip = currentString.toString();\n        }\n    }\n\n    /**\n     * Converts the {@link XmlContributor} object to a {@link Contributor}\n     * object. If no properties have been set, a {@link Contributor} with the\n     * IP Address \"unknown\" is returned.\n     * \n     * @return the contributor of a revision\n     */\n    public Contributor getContributor() {\n        if (final_contributor == null) {\n            Contributor contributor = new Contributor();\n            contributor.setIp(\"unknown\");\n            return contributor;\n        } else {\n            return final_contributor;\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/XmlPage.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport java.util.Calendar;\nimport java.util.LinkedList;\nimport java.util.List;\nimport java.util.Map;\nimport java.util.TreeMap;\n\nimport org.xml.sax.Attributes;\nimport org.xml.sax.helpers.DefaultHandler;\n\nimport de.zib.scalaris.examples.wikipedia.bliki.MyWikiModel;\nimport de.zib.scalaris.examples.wikipedia.data.Page;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\n\n/**\n * Represents a page including its revisions for use by an XML reader.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class XmlPage extends DefaultHandler {\n    protected StringBuilder currentString = new StringBuilder();\n    /**\n     * The page's title.\n     */\n    protected String title;\n\n    /**\n     * The page's ID.\n     */\n    protected String id;\n\n    /**\n     * The page's restrictions, e.g. for moving/editing the page.\n     */\n    protected String restrictions;\n    \n    /**\n     * Whether the page's newest revision redirects or not.\n     */\n    protected boolean redirect;\n    \n    /**\n     * All revisions of the page.\n     */\n    protected TreeMap<Integer, Revision> revisions = new TreeMap<Integer, Revision>();\n    \n    protected boolean inPage_title;\n    protected boolean inPage_id;\n    protected boolean inPage_restrictions;\n    protected boolean inRevision;\n    \n    protected CheckSkipRevisions checkSkipRevisions = null;\n    protected boolean skipRevisions;\n    \n    protected XmlRevision currentRevision = new XmlRevision();\n    protected String lastRevText = null;\n    \n    protected Page final_page;\n\n    /**\n     * Maximum number of revisions per page (starting with the most recent) -\n     * <tt>-1/tt> imports all revisions.\n     */\n    protected int maxRevisions;\n    \n    /**\n     * Maximum time a revision should have (newer revisions are omitted) -\n     * <tt>null/tt> imports all revisions.\n     */\n    protected Calendar maxTime;\n    \n    /**\n     * Minimum time a revision should have (only one revision older than this\n     * will be imported) - <tt>null/tt> imports all revisions.\n     */\n    protected Calendar minTime;\n\n    /**\n     * Creates a new page with an empty title, id and no revision.\n     * \n     * @param maxRevisions\n     *            maximum number of revisions per page (starting with the most\n     *            recent) - <tt>-1/tt> imports all revisions\n     * @param minTime\n     *            minimum time a revision should have (only one revision older\n     *            than this will be imported) - <tt>null/tt> imports all\n     *            revisions\n     * @param maxTime\n     *            maximum time a revision should have (newer revisions are\n     *            omitted) - <tt>null/tt> imports all revisions\n     */\n    public XmlPage(int maxRevisions, Calendar minTime, Calendar maxTime) {\n        super();\n        this.maxRevisions = maxRevisions;\n        this.minTime = minTime;\n        this.maxTime = maxTime;\n        init();\n    }\n    \n    /**\n     * (Re-) Initialises all instance variables.\n     */\n    private void init() {\n        currentString.setLength(0);\n        title = \"\";\n        id = \"\";\n        restrictions = \"\";\n        redirect = false;\n        revisions.clear();\n        inPage_title = false;\n        inPage_id = false;\n        inPage_restrictions = false;\n        inRevision = false;\n        skipRevisions = false;\n        currentRevision.reset();\n        final_page = null;\n    }\n    \n    /**\n     * Resets all instance variables. Afterwards, the object has the same state\n     * as a newly created one with the given {@link #maxRevisions},\n     * {@link #maxTime} and {@link #minTime}.\n     */\n    public void reset() {\n        init();\n    }\n\n    /**\n     * Called to when a starting page element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     * @param attributes\n     *            The attributes attached to the element. If there are no\n     *            attributes, it shall be an empty Attributes object.\n     */\n    public void startPage(String uri, String localName, String qName,\n            Attributes attributes) {\n        // nothing to do\n    }       \n\n    /**\n     * Called to when a starting element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     * @param attributes\n     *            The attributes attached to the element. If there are no\n     *            attributes, it shall be an empty Attributes object.\n     */\n    @Override\n    public void startElement(String uri, String localName, String qName,\n            Attributes attributes) {\n        // System.out.println(localName);\n        \n        if (inRevision) {\n            if (!skipRevisions) {\n                currentRevision.startElement(uri, localName, qName, attributes);\n            }\n        } else {\n            currentString.setLength(0);\n            /*\n             * <title>Main Page</title> <id>1</id> <revision></revision> ...\n             */\n            if (localName.equals(\"title\")) {\n            } else if (localName.equals(\"id\")) {\n            } else if (localName.equals(\"restrictions\")) {\n            } else if (localName.equals(\"redirect\")) {\n                redirect = true;\n            } else if (localName.equals(\"revision\")) {\n                inRevision = true;\n                // check whether all revisions of this page should be skipped\n                if (!skipRevisions && checkSkipRevisions != null\n                        && checkSkipRevisions.skipRevisions(title)) {\n                    skipRevisions = true;\n                }\n                if (!skipRevisions) {\n                    currentRevision.reset();\n                    currentRevision.startRevision(uri, localName, qName, attributes);\n                }\n            } else if (localName.equals(\"ns\")) {\n            } else if (localName.equals(\"sha1\")) {\n            } else {\n                System.err.println(\"unknown page tag: \" + localName);\n            }\n        }\n    }\n\n    /**\n     * Called to process character data.\n     * \n     * Note: a SAX driver is free to chunk the character data any way it wants,\n     * so you cannot count on all of the character data content of an element\n     * arriving in a single characters event.\n     * \n     * @param ch\n     *            The characters.\n     * @param start\n     *            The start position in the character array.\n     * @param length\n     *            The number of characters to use from the character array.\n     */\n    @Override\n    public void characters(char[] ch, int start, int length) {\n        // System.out.println(new String(ch, start, length));\n\n        if (inRevision) {\n            if (!skipRevisions) {\n                currentRevision.characters(ch, start, length);\n            }\n        } else {\n            currentString.append(ch, start, length);\n        }\n    }\n\n    /**\n     * Called to when an ending element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     */\n    public void endPage(String uri, String localName, String qName) {\n        /* \n         * parse page restrictions - examples:\n         * <restrictions>edit=sysop:move=sysop</restrictions>\n         * <restrictions>sysop</restrictions>\n         */\n        Map<String, String> restrictions_map = Page\n                .restrictionsFromString(restrictions);\n        // get current revision (the largest one):\n        Revision curRev = null;\n        if (!revisions.isEmpty()) {\n            curRev = revisions.lastEntry().getValue();\n            curRev.setUnpackedText(lastRevText);\n        }\n        // NOTE: We cannot rely on the page info from the XML dump since our\n        //       current revision may be different to the latest available one!\n        // -> update redirect with info from curRev\n        redirect = MyWikiModel.MATCH_WIKI_REDIRECT.matcher(lastRevText).matches();\n        final_page = new Page(title,\n                Integer.parseInt(id), redirect, restrictions_map, curRev);\n    }\n\n    /**\n     * Called to when an ending element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     */\n    @Override\n    public void endElement(String uri, String localName, String qName) {\n        if (inRevision) {\n            if (localName.equals(\"revision\")) {\n                inRevision = false;\n                if (!skipRevisions) {\n                    currentRevision.endRevision(uri, localName, qName);\n                    Revision curRev = currentRevision.getRevision();\n                    // check rev not too new:\n                    if (maxTime == null ||\n                            !Revision.stringToCalendar(curRev.getTimestamp()).after(maxTime)) {\n                        // check rev not too old:\n                        if (minTime != null &&\n                                Revision.stringToCalendar(curRev.getTimestamp()).compareTo(minTime) <= 0) {\n                            // keep only the newest (old) revision\n                            revisions.clear();\n                        }\n                        if (maxRevisions != (-1) && revisions.size() >= maxRevisions) {\n                            revisions.remove(revisions.firstKey());\n                        }\n                        if (!revisions.isEmpty()) {\n                            revisions.lastEntry().getValue().setUnpackedText(lastRevText);\n                        }\n                        revisions.put(curRev.getId(), curRev);\n                        lastRevText = currentRevision.getText();\n                    }\n                }\n            } else {\n                if (!skipRevisions) {\n                    currentRevision.endElement(uri, localName, qName);\n                }\n            }\n        } else {\n            if (localName.equals(\"title\")) {\n                title = currentString.toString();\n            } else if (localName.equals(\"id\")) {\n                id = currentString.toString();\n            } else if (localName.equals(\"restrictions\")) {\n                restrictions = currentString.toString();\n            } else if (localName.equals(\"redirect\")) {\n                // nothing to do\n            } else if (localName.equals(\"ns\")) {\n                // nothing to do\n            } else if (localName.equals(\"sha1\")) {\n                // nothing to do\n            }\n        }\n    }\n    \n    /**\n     * Translates the {@link XmlPage} object to a {@link Page} object.\n     * Throws in case of a malformed XML file.\n     * \n     * @return the page\n     */\n    public Page getPage() {\n        return final_page;\n    }\n\n    /**\n     * Gets all revisions of this page.\n     * \n     * @return the revisions\n     */\n    public List<Revision> getRevisions() {\n        return new LinkedList<Revision>(revisions.values());\n    }\n\n    /**\n     * @param checkSkipRevisions the checkSkipRevisions to set\n     */\n    public void setCheckSkipRevisions(CheckSkipRevisions checkSkipRevisions) {\n        this.checkSkipRevisions = checkSkipRevisions;\n    }\n    \n    /**\n     * Functor to check whether to skip parsing revisions based on a page's\n     * title.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static interface CheckSkipRevisions {\n        /**\n         * Checks whether to skip parsing all revisions of a page.\n         * \n         * @param pageTitle\n         *            the title of the page\n         * \n         * @return <tt>true</tt> if revisions should not be parsed,\n         *         <tt>false</tt> otherwise\n         */\n        public abstract boolean skipRevisions(String pageTitle);\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/XmlRevision.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport org.xml.sax.Attributes;\nimport org.xml.sax.helpers.DefaultHandler;\n\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\n\n/**\n * Represents a revision of a page for use by an XML reader.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class XmlRevision extends DefaultHandler {\n    protected StringBuilder currentString = new StringBuilder();\n    /**\n     * the revision's ID\n     */\n    protected String id;\n\n    /**\n     * the revision's date of creation\n     */\n    protected String timestamp;\n    \n    /**\n     * whether the change is a minor change or not\n     */\n    protected boolean minorChange;\n\n    /**\n     * the revision's contributor\n     */\n    protected XmlContributor currentContributor = new XmlContributor();\n\n    /**\n     * the comment of the revision\n     */\n    protected String comment;\n    \n    /**\n     * the content (text) of the revision\n     */\n    protected String text;\n    \n    private boolean inRevision_contributor;\n    \n    protected Revision final_revision;\n\n    /**\n     * Creates a new revision with an empty id, timestamp, contributor,\n     * comment and text.\n     */\n    public XmlRevision() {\n        super();\n        init();\n    }\n\n    /**\n     * (Re-) Initialises all instance variables.\n     */\n    private void init() {\n        currentString.setLength(0);\n        id = \"\";\n        timestamp = \"\";\n        minorChange = false;\n        currentContributor.reset();\n        comment = \"\";\n        text = \"\";\n        inRevision_contributor = false;\n        final_revision = null;\n    }\n    \n    /**\n     * Resets all instance variables. Afterwards, the object has the same state\n     * as a newly created one.\n     */\n    public void reset() {\n        init();\n    }\n\n    /**\n     * Called to when a starting revision element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     * @param attributes\n     *            The attributes attached to the element. If there are no\n     *            attributes, it shall be an empty Attributes object.\n     */\n    public void startRevision(String uri, String localName, String qName,\n            Attributes attributes) {\n        // nothing to do\n    }\n\n    /**\n     * Called to when a starting element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     * @param attributes\n     *            The attributes attached to the element. If there are no\n     *            attributes, it shall be an empty Attributes object.\n     */\n    @Override\n    public void startElement(String uri, String localName, String qName,\n            Attributes attributes) {\n        // System.out.println(localName);\n\n        if (inRevision_contributor) {\n            currentContributor.startElement(uri, localName, qName, attributes);\n        } else {\n            currentString.setLength(0);\n            /*\n             * <id>2</id> <timestamp>2006-10-01T11:36:54Z</timestamp>\n             * <contributor> ... </contributor> <comment>...</comment>\n             * <text>...</text>\n             */\n            if (localName.equals(\"id\")) {\n            } else if (localName.equals(\"timestamp\")) {\n            } else if (localName.equals(\"minor\")) {\n                minorChange = true;\n            } else if (localName.equals(\"contributor\")) {\n                inRevision_contributor = true;\n                currentContributor.reset();\n                currentContributor.startContributor(uri, localName, qName, attributes);\n            } else if (localName.equals(\"comment\")) {\n            } else if (localName.equals(\"text\")) {\n            } else if (localName.equals(\"sha1\")) {\n            } else {\n                System.err.println(\"unknown revision tag: \" + localName);\n            }\n        }\n    }\n\n    /**\n     * Called to process character data.\n     * \n     * Note: a SAX driver is free to chunk the character data any way it wants,\n     * so you cannot count on all of the character data content of an element\n     * arriving in a single characters event.\n     * \n     * @param ch\n     *            The characters.\n     * @param start\n     *            The start position in the character array.\n     * @param length\n     *            The number of characters to use from the character array.\n     */\n    @Override\n    public void characters(char[] ch, int start, int length) {\n        // System.out.println(new String(ch, start, length));\n\n        if (inRevision_contributor) {\n            currentContributor.characters(ch, start, length);\n        } else {\n            currentString.append(ch, start, length);\n        }\n    }\n\n    /**\n     * Called to when an ending revision element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     */\n    public void endRevision(String uri, String localName, String qName) {\n        final_revision = new Revision(Integer.parseInt(id), timestamp,\n                minorChange, currentContributor.getContributor(), comment);\n    }\n\n    /**\n     * Called to when an ending element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     */\n    @Override\n    public void endElement(String uri, String localName, String qName) {\n        if (inRevision_contributor) {\n            if (localName.equals(\"contributor\")) {\n                currentContributor.endContributor(uri, localName, qName);\n                inRevision_contributor = false;\n            } else {\n                currentContributor.endElement(uri, localName, qName);\n            }\n        } else {\n            if (localName.equals(\"id\")) {\n                id = currentString.toString();\n            } else if (localName.equals(\"timestamp\")) {\n                timestamp = currentString.toString();\n            } else if (localName.equals(\"minor\")) {\n                // nothing to do\n            } else if (localName.equals(\"contributor\")) {\n                inRevision_contributor = false;\n            } else if (localName.equals(\"comment\")) {\n                comment = currentString.toString();\n            } else if (localName.equals(\"text\")) {\n                text = currentString.toString();\n            }\n        }\n    }\n\n    /**\n     * Converts the {@link XmlRevision} object to a {@link Revision} object.\n     * Throws in case of a malformed XML file.\n     * \n     * @return the revision of a page\n     */\n    public Revision getRevision() {\n        return final_revision;\n    }\n\n    /**\n     * Gets the revision's (uncompressed) text.\n     * \n     * @return the text\n     */\n    public String getText() {\n        return text;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/XmlSiteInfo.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.data.xml;\n\nimport java.util.HashMap;\nimport java.util.Map;\n\nimport org.xml.sax.Attributes;\nimport org.xml.sax.helpers.DefaultHandler;\n\nimport de.zib.scalaris.examples.wikipedia.data.SiteInfo;\n\n/**\n * Represents generic site information for use by an XML reader.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class XmlSiteInfo extends DefaultHandler {\n    protected StringBuilder currentString = new StringBuilder();\n    \n    protected String base = \"\";\n    protected String sitename = \"\";\n    protected String generator = \"\";\n    protected String caseStr = \"\";\n    protected Map<String, Map<String, String>> namespaces = new HashMap<String, Map<String, String>>();\n    protected String currentNamespaceKey = null;\n    protected Map<String, String> currentNamespace = null;\n    \n    protected boolean inSiteInfo_namespaces = false;\n    \n    protected SiteInfo final_siteinfo = null;\n\n    /**\n     * Creates an empty site info object.\n     */\n    public XmlSiteInfo() {\n        super();\n    }\n    \n    /**\n     * Called to when a starting siteinfo element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     * @param attributes\n     *            The attributes attached to the element. If there are no\n     *            attributes, it shall be an empty Attributes object.\n     */\n    public void startSiteInfo(String uri, String localName, String qName,\n            Attributes attributes) {\n        // nothing to do\n    }\n    \n    /**\n     * Called to when a starting element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     * @param attributes\n     *            The attributes attached to the element. If there are no\n     *            attributes, it shall be an empty Attributes object.\n     */\n    @Override\n    public void startElement(String uri, String localName, String qName,\n            Attributes attributes) {\n        // System.out.println(localName);\n        currentString.setLength(0);\n\n        /*\n         * <sitename>Wikipedia</sitename>\n         * <base>http://bar.wikipedia.org/wiki/Hauptseitn</base>\n         * <generator>MediaWiki 1.17wmf1</generator>\n         * <case>first-letter</case>\n         * <namespaces>\n         *   <namespace key=\"-2\" case=\"first-letter\">Medium</namespace>\n         *   <namespace key=\"-1\" case=\"first-letter\">Spezial</namespace>\n         *   <namespace key=\"0\" case=\"first-letter\" />\n         *   <namespace key=\"1\" case=\"first-letter\">Diskussion</namespace>\n         *   <namespace key=\"2\" case=\"first-letter\">Benutzer</namespace>\n         *   <namespace key=\"3\" case=\"first-letter\">Benutzer Diskussion</namespace>\n         *   <namespace key=\"4\" case=\"first-letter\">Wikipedia</namespace>\n         *   <namespace key=\"5\" case=\"first-letter\">Wikipedia Diskussion</namespace>\n         *   <namespace key=\"6\" case=\"first-letter\">Datei</namespace>\n         *   <namespace key=\"7\" case=\"first-letter\">Datei Diskussion</namespace>\n         *   <namespace key=\"8\" case=\"first-letter\">MediaWiki</namespace>\n         *   <namespace key=\"9\" case=\"first-letter\">MediaWiki Diskussion</namespace>\n         *   <namespace key=\"10\" case=\"first-letter\">Vorlage</namespace>\n         *   <namespace key=\"11\" case=\"first-letter\">Vorlage Diskussion</namespace>\n         *   <namespace key=\"12\" case=\"first-letter\">Hilfe</namespace>\n         *   <namespace key=\"13\" case=\"first-letter\">Hilfe Diskussion</namespace>\n         *   <namespace key=\"14\" case=\"first-letter\">Kategorie</namespace>\n         *   <namespace key=\"15\" case=\"first-letter\">Kategorie Diskussion</namespace>\n         *   <namespace key=\"100\" case=\"first-letter\">Portal</namespace>\n         *   <namespace key=\"101\" case=\"first-letter\">Portal Diskussion</namespace>\n         * </namespaces>\n         */\n        if (localName.equals(\"sitename\")) {\n        } else if (localName.equals(\"base\")) {\n        } else if (localName.equals(\"generator\")) {\n        } else if (localName.equals(\"case\")) {\n        } else if (localName.equals(\"namespaces\")) {\n            inSiteInfo_namespaces = true;\n        } else if (inSiteInfo_namespaces && localName.equals(\"namespace\")) {\n            currentNamespace = new HashMap<String, String>();\n            currentNamespace.put(SiteInfo.NAMESPACE_CASE, attributes.getValue(\"\", \"case\"));\n            currentNamespaceKey = attributes.getValue(\"\", \"key\");\n        } else {\n            System.err.println(\"unknown siteinfo tag: \" + localName);\n        }\n    }\n\n    /**\n     * Called to process character data.\n     * \n     * Note: a SAX driver is free to chunk the character data any way it wants,\n     * so you cannot count on all of the character data content of an element\n     * arriving in a single characters event.\n     * \n     * @param ch\n     *            The characters.\n     * @param start\n     *            The start position in the character array.\n     * @param length\n     *            The number of characters to use from the character array.\n     */\n    public void characters(char[] ch, int start, int length) {\n        // System.out.println(new String(ch, start, length));\n        currentString.append(ch, start, length);\n    }\n\n    /**\n     * Called to when an ending siteinfo element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     */\n    public void endSiteInfo(String uri, String localName, String qName) {\n        final_siteinfo = new SiteInfo(base, sitename, generator, caseStr, namespaces);\n    }\n\n    /**\n     * Called to when an ending element is encountered.\n     * \n     * @param uri\n     *            The Namespace URI, or the empty string if the element has no\n     *            Namespace URI or if Namespace processing is not being\n     *            performed.\n     * @param localName\n     *            The local name (without prefix), or the empty string if\n     *            Namespace processing is not being performed.\n     * @param qName\n     *            The qualified name (with prefix), or the empty string if\n     *            qualified names are not available.\n     */\n    public void endElement(String uri, String localName, String qName) {\n        if (localName.equals(\"sitename\")) {\n            sitename = currentString.toString();\n        } else if (localName.equals(\"base\")) {\n            base = currentString.toString();\n        } else if (localName.equals(\"generator\")) {\n            generator = currentString.toString();\n        } else if (localName.equals(\"case\")) {\n            caseStr = currentString.toString();\n        } else if (localName.equals(\"namespaces\")) {\n            inSiteInfo_namespaces = false;\n        } else if (localName.equals(\"namespace\")) {\n            currentNamespace.put(SiteInfo.NAMESPACE_PREFIX, currentString.toString());\n            namespaces.put(currentNamespaceKey, currentNamespace);\n        }\n    }\n\n    /**\n     * Converts the {@link XmlSiteInfo} object to a {@link SiteInfo} object.\n     * Throws in case of a malformed XML file.\n     * \n     * @return the siteinfo of a wiki\n     */\n    public SiteInfo getSiteInfo() {\n        return final_siteinfo;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/data/xml/util/SevenZInputStream.java",
    "content": "package de.zib.scalaris.examples.wikipedia.data.xml.util;\n\nimport java.io.File;\nimport java.io.IOException;\n\nimport org.apache.commons.compress.archivers.ArchiveInputStream;\nimport org.apache.commons.compress.archivers.sevenz.SevenZArchiveEntry;\nimport org.apache.commons.compress.archivers.sevenz.SevenZFile;\n\n/**\n * Provides an InputStream interface to a SevenZFile  \n */\n\npublic class SevenZInputStream extends ArchiveInputStream {\n\n    private SevenZFile archive;\n    private SevenZArchiveEntry currentEntry;\n    \n    public SevenZInputStream(File file) throws IOException {\n        archive = new SevenZFile(file);\n        currentEntry = archive.getNextEntry();\n    }\n\n    @Override\n    public int read(byte[] b, int off, int len) throws IOException {\n        if(currentEntry != null) {\n            return archive.read(b, off, len);\n        } else {\n            return -1;\n        }\n    }\n\n    public SevenZArchiveEntry getCurrentEntry() {\n        return currentEntry;\n    }\n    \n    @Override\n    public SevenZArchiveEntry getNextEntry() throws IOException {\n        currentEntry = archive.getNextEntry();\n        return currentEntry;\n    }\n    \n    @Override\n    public void close() throws IOException {\n        currentEntry = null;\n        archive.close();        \n    }\n\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/plugin/PluginClassLoader.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.plugin;\n\nimport java.io.File;\nimport java.io.FileFilter;\nimport java.io.FileInputStream;\nimport java.io.FileNotFoundException;\nimport java.io.IOException;\nimport java.io.InputStream;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport java.net.URLClassLoader;\nimport java.util.Arrays;\nimport java.util.Collection;\nimport java.util.Enumeration;\nimport java.util.HashMap;\nimport java.util.LinkedList;\nimport java.util.List;\nimport java.util.Map;\nimport java.util.jar.JarEntry;\nimport java.util.jar.JarFile;\nimport java.util.regex.Pattern;\n\nimport javax.tools.JavaCompiler;\nimport javax.tools.JavaFileObject;\nimport javax.tools.StandardJavaFileManager;\nimport javax.tools.ToolProvider;\n\n/**\n * A class loader capable of loading plug-in classes from a given plug-in\n * directory.\n * \n * <p>\n * It allows for an efficient retrieval of plug-ins by means of an interface.\n * Three requirements have to be met:\n * <ul>\n * <li>the plug-in needs to be assignable from any of the given plug-in interfaces\n * </li>\n * <li>the policy needs to be in the plug-in directory, its dependencies need to\n * be placed in the <tt>%lt;pluginname&gt;</tt> sub-directory of the plug-in\n * directory.</li>\n * </ul>\n * </p>\n * \n * @author Jan Stender, stender@zib.de\n * @author Nico Kruber, kruber@zib.de\n */\npublic class PluginClassLoader extends ClassLoader {\n\n    private final Map<String, Class<?>> cache = new HashMap<String, Class<?>>();\n\n    private final Map<Class<?>, List<Class<?>>> pluginMap = new HashMap<Class<?>, List<Class<?>>>();\n\n    private final Class<?>[] pluginInterfaces;\n\n    private File pluginDir;\n\n    private File[] jarFiles = new File[0];\n\n    private List<File> subDirs = new LinkedList<File>();\n\n    private static final Pattern REMOVE_JAR_EXT = Pattern.compile(\"\\\\.jar$\");\n\n    private static final Pattern REMOVE_JAVA_EXT = Pattern.compile(\"\\\\.java$\");\n\n    private static final Pattern REMOVE_CLASS_EXT = Pattern.compile(\"\\\\.class$\");\n\n    /**\n     * Instantiates a new plug-in class loader.\n     * \n     * @param pluginDirPath\n     *            the path for the directory with all (compiled) plug-ins\n     * @param pluginInterfaces\n     *            the plug-in interfaces (any plug-in implementing any of these\n     *            interfaces will be loaded)\n     * @throws IOException\n     *             if an error occurs while initialising the plug-in interfaces\n     */\n    public PluginClassLoader(String pluginDirPath, Class<?>[] pluginInterfaces)\n            throws IOException {\n        this.pluginDir = new File(pluginDirPath);\n        this.pluginInterfaces = pluginInterfaces;\n        init();\n    }\n    \n    /**\n     * Initialises the class loader. Each class in the\n     * directory is loaded and checked for assignability to one of the given\n     * plug-in interfaces. If the check is successful, the class is added to a\n     * map, from which it can be retrieved.\n     * \n     * @throws IOException\n     *             if an I/O error occurs while loading any of the classes\n     */\n    public void init() throws IOException {\n        if (pluginDir.exists()) {\n            // get all Java files recursively\n            File[] javaFiles = pluginDir.listFiles(new FileFilter() {\n                public boolean accept(File pathname) {\n                    return pathname.getPath().endsWith(\".java\");\n                }\n            });\n            \n            // compile all Java files\n            if (javaFiles.length != 0) {\n                JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();\n                if (compiler != null) {\n                    List<File> pluginJars = new LinkedList<File>();\n                    for (File javaFile: javaFiles) {\n                        String filename = REMOVE_JAVA_EXT.matcher(javaFile.getName()).replaceAll(\"\");\n                        pluginJars.addAll(Arrays.asList(getPluginJars(filename)));\n                    } \n                    String cp = buildClassPath(pluginJars);\n                    List<String> options = Arrays.asList(\"-cp\", cp);\n                    StandardJavaFileManager fileManager = compiler.getStandardFileManager(null, null, null);\n\n                    Iterable<? extends JavaFileObject> compilationUnits = fileManager\n                            .getJavaFileObjectsFromFiles(Arrays.asList(javaFiles));\n                    if (!compiler.getTask(null, fileManager, null, options, null, compilationUnits).call()) {\n                        // note: an error is already printed to the log\n                    }\n\n                    fileManager.close();\n                }\n            }\n            \n            \n            // retrieve all policies from class files\n            File[] classFiles = pluginDir.listFiles(new FileFilter() {\n                public boolean accept(File pathname) {\n                    return pathname.getPath().endsWith(\".class\");\n                }\n            });\n            \n            for (File cls : classFiles) {\n                String filename = REMOVE_CLASS_EXT.matcher(cls.getName()).replaceAll(\"\");\n                subDirs.addAll(Arrays.asList(getPluginJars(filename)));\n                try {\n                    String className = cls.getAbsolutePath().substring(\n                        pluginDir.getAbsolutePath().length() + 1,\n                        cls.getAbsolutePath().length() - \".class\".length()).replace(File.separatorChar, '.');\n                    if (cache.containsKey(className)) {\n                        continue;\n                    }\n                    \n                    // load the class\n                    Class<?> clazz = loadFromStream(new FileInputStream(cls));\n                    \n                    // check whether the class refers to a plug-in; if so,\n                    // cache it\n                    checkClass(clazz);\n                    \n                } catch (LinkageError err) {\n                    // ignore linkage errors\n                } catch (Exception err) {\n                    err.printStackTrace(); // TODO: remove\n                }\n            }\n            \n            // get all JAR files\n            jarFiles = pluginDir.listFiles(new FileFilter() {\n                public boolean accept(File pathname) {\n                    return pathname.getPath().endsWith(\".jar\");\n                }\n            });\n\n            // retrieve all policies from JAR files, add appropriate subdirs to\n            // the list of directories\n            for (File jar : jarFiles) {\n                String subdir = REMOVE_JAR_EXT.matcher(jar.getName()).replaceAll(\"\");\n                subDirs.addAll(Arrays.asList(getPluginJars(subdir)));\n                JarFile jarFile = new JarFile(jar);\n\n                Enumeration<JarEntry> entries = jarFile.entries();\n                while (entries.hasMoreElements()) {\n                    JarEntry entry = entries.nextElement();\n                    if (entry.getName().endsWith(\".class\")) {\n                        try {\n                            // load the class\n                            Class<?> clazz = loadFromStream(jarFile.getInputStream(entry));\n\n                            // check whether the class refers to a plug-in; if\n                            // so, cache it\n                            checkClass(clazz);\n                        } catch (IOException err) {\n                            err.printStackTrace(); // TODO: remove\n                        } catch (LinkageError err) {\n                            // ignore\n                        }\n                    }\n                }\n                jarFile.close();\n            }\n        }\n    }\n\n    /**\n     * Uses the class path defined in the system class path, used by this class'\n     * class loader and system class loader to build a class path for\n     * compilation. Adds all given jars as well.\n     * \n     * @param addJars\n     *            jar files to add to the class path\n     * \n     * @return the class path as a string\n     */\n    private String buildClassPath(Collection<File> addJars) {\n        StringBuilder cp = new StringBuilder();\n        cp.append(System.getProperty(\"java.class.path\" ));\n        \n        for (ClassLoader cl: new ClassLoader[] {this.getClass().getClassLoader(), getSystemClassLoader()}) {\n            if (cl != null && cl instanceof URLClassLoader) {\n                // note: we don't create new class loaders here!\n                @SuppressWarnings(\"resource\")\n                URLClassLoader clu = (URLClassLoader) cl;\n                for (URL url: clu.getURLs()) {\n                    cp.append(File.pathSeparatorChar);\n                    cp.append(url.getPath());\n                }\n            }\n        }\n        for (File jarFile: addJars) {\n            cp.append(File.pathSeparatorChar);\n            cp.append(jarFile.getAbsolutePath());\n        }\n        return cp.toString();\n    }\n\n    /**\n     * Gets all jar files in the sub-directory the given plugin can place\n     * dependencies in.\n     * \n     * @param pluginName\n     *            the name of the plug-in (determines the sub-directory's name)\n     * \n     * @return all jar files (may be an empty array)\n     */\n    private File[] getPluginJars(String pluginName) {\n        File clsFile = new File(pluginDir + File.separator + pluginName);\n        if (clsFile.isDirectory()) {\n            // add the sub-directory itself\n            subDirs.add(clsFile);\n\n            // add all JAR files in the sub-directory\n            File[] jarsInSubDir = clsFile.listFiles(new FileFilter() {\n                public boolean accept(File pathname) {\n                    return pathname.getPath().endsWith(\".jar\");\n                }\n            });\n            return jarsInSubDir;\n        }\n        return new File[0];\n    }\n\n    @Override\n    public Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {\n        // first, check whether the class is cached\n        if (cache.containsKey(name)) {\n            return cache.get(name);\n        }\n\n        // if not cached, try to resolve the class by means of the system\n        // class loader\n        try {\n            return getClass().getClassLoader().loadClass(name);\n        } catch (ClassNotFoundException exc) {\n        }\n\n        if (!pluginDir.exists()) {\n            throw new ClassNotFoundException(\"the plug-in directory does not exist\");\n        }\n\n        // if the class could not be loaded by the system class loader, try\n        // to load it from an external JAR file\n        URL[] urls = new URL[subDirs.size() + jarFiles.length];\n        try {\n            int i = 0;\n            // add the plug-ins' directories for dependencies\n            for (File subdir: subDirs) {\n                urls[i++] = subdir.toURI().toURL();\n            }\n            for (File jarFile: jarFiles) {\n                // add the plug-in's directory for dependencies\n                urls[i++] = jarFile.toURI().toURL();\n            }\n        } catch (MalformedURLException err) {\n        }\n        \n        return new URLClassLoader(urls) {\n            @Override\n            public URL getResource(String name) {\n                URL resource = super.getResource(name);\n                if (resource != null) {\n                    return resource;\n                }\n                return PluginClassLoader.this.getResource(name);\n            }\n\n            @Override\n            public InputStream getResourceAsStream(String name) {\n                InputStream stream = super.getResourceAsStream(name);\n                if (stream != null) {\n                    return stream;\n                }\n                return PluginClassLoader.this.getResourceAsStream(name);\n            }\n\n        }.loadClass(name);\n    }\n\n    /**\n     * Returns a list of all plug-ins for a given interface.\n     * \n     * @param pluginInterface\n     *            the plug-in interface\n     * \n     * @return a list of plug-in classes where each is assignable from the given\n     *         interface\n     */\n    public List<Class<?>> getClasses(Class<?> pluginInterface) {\n        return pluginMap.get(pluginInterface);\n    }\n\n    @Override\n    public URL getResource(String name) {\n        // first, try to get the resource from the parent class loader\n        URL resource = super.getResource(name);\n        if (resource != null) {\n            return resource;\n        }\n\n        // then try to get the resource from a plug-in's folder\n        for (File subDir: subDirs) {\n            File file = new File(subDir.getAbsolutePath() + File.separator + name);\n            if (file.exists()) {\n                try {\n                    return file.toURI().toURL();\n                } catch (MalformedURLException e) {\n                    return null;\n                }\n            }\n        }\n\n        return null;\n    }\n\n    @Override\n    public InputStream getResourceAsStream(String name) {\n\n        // first, try to get the stream from the parent class loader\n        InputStream stream = super.getResourceAsStream(name);\n        if (stream != null) {\n            return stream;\n        }\n\n        // then try to get the resource from a plug-in's folder\n        for (File subDir: subDirs) {\n            File file = new File(subDir.getAbsolutePath() + File.separator + name);\n            try {\n                return new FileInputStream(file);\n            } catch (FileNotFoundException exc) {\n                return null;\n            }\n        }\n\n        return null;\n    }\n\n    private Class<?> loadFromStream(InputStream in) throws IOException {\n        // load the binary class content\n        byte[] classData = new byte[in.available()];\n        in.read(classData);\n        in.close();\n\n        Class<?> clazz = defineClass(null, classData, 0, classData.length);\n        cache.put(clazz.getName(), clazz);\n\n        return clazz;\n    }\n\n    /**\n     * Check whether the class matches any of the plug-in interfaces and adds\n     * the class to the appropriate list(s) of assignable classes.\n     * \n     * @param clazz the class to check\n     */\n    private void checkClass(Class<?> clazz) {\n        for (Class<?> ifc : pluginInterfaces) {\n            if (ifc.isAssignableFrom(clazz)) {\n                List<Class<?>> assignableClasses = pluginMap.get(ifc);\n                if (assignableClasses == null) {\n                    assignableClasses = new LinkedList<Class<?>>();\n                    pluginMap.put(ifc, assignableClasses);\n                }\n                assignableClasses.add(clazz);\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/plugin/WikiEventHandler.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.plugin;\n\nimport javax.servlet.http.HttpServletRequest;\n\nimport de.zib.scalaris.examples.wikipedia.SavePageResult;\nimport de.zib.scalaris.examples.wikipedia.ValueResult;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiPageBean;\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiPageBeanBase;\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiPageEditBean;\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiServlet;\n\n/**\n * Simple handler of events in the\n * {@link de.zib.scalaris.examples.wikipedia.bliki.WikiServlet} class.\n * \n * Note: this API is not stable and will probably change in future.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic interface WikiEventHandler {\n    /**\n     * Will be called after the user submitted a new revision to be stored in\n     * the wiki.\n     * \n     * Note that if the operation was not successful,\n     * {@link #onPageView(WikiPageBeanBase, Object)} will also be called!\n     * \n     * @param page\n     *            some info on the edited page\n     * @param result\n     *            the result of the operation (may not be successful)\n     * @param connection\n     *            connection to the data store\n     */\n    public <Connection> void onPageSaved(WikiPageEditBean page, SavePageResult result,\n            Connection connection);\n\n    /**\n     * Will be called for a page view. This also includes empty pages,\n     * NoArticleText, BadTitle, unsuccessful data connections, page edit/preview\n     * pages, special pages, page history pages etc.\n     * \n     * @param page\n     *            some info on the shown page\n     * @param connection\n     *            connection to the data store (may be <tt>null</tt>!)\n     */\n    public <Connection> void onPageView(WikiPageBeanBase page, Connection connection);\n\n    /**\n     * Will be called during a request for an image.\n     * \n     * @param image\n     *            the plain image name from the rendering\n     * @param realImageUrl\n     *            the translated URL pointing to the wikipedia\n     */\n    public void onImageRedirect(String image, String realImageUrl);\n\n    /**\n     * Will be called when a random page should be shown.\n     * \n     * @param page\n     *            some info about the call\n     * @param result\n     *            DB query result\n     * @param connection\n     *            connection to the data store\n     */\n    public <Connection> void onViewRandomPage(WikiPageBean page,\n            ValueResult<NormalisedTitle> result, Connection connection);\n\n    /**\n     * Checks whether the given user is allowed to access the wiki.\n     * \n     * @param <Connection>\n     * \n     * @param serviceUser\n     *            the provided username\n     * @param request\n     *            the request object (allows reading parameters or writing\n     *            status messages, e.g. errors using\n     *            {@link WikiServlet#setParam_error(HttpServletRequest, String)}\n     * @param connection\n     *            connection to the data store\n     * \n     * @return <tt>true</tt> if access is allowed, <tt>false</tt> otherwise\n     */\n    public <Connection> boolean checkAccess(String serviceUser, HttpServletRequest request,\n            Connection connection);\n\n    /**\n     * Gets a descriptive name of this event handler.\n     * \n     * @return a (HTML) string\n     */\n    public String getName();\n\n    /**\n     * Gets a URL where to get the plugin from or get more information about it.\n     * \n     * @return a URL\n     */\n    public String getURL();\n\n    /**\n     * Gets a descriptive version number of this event handler.\n     * \n     * @return a (HTML) string\n     */\n    public String getVersion();\n\n    /**\n     * Gets a brief description of what this event handler is doing.\n     * \n     * @return a (HTML) string\n     */\n    public String getDescription();\n\n    /**\n     * Gets the name of the author(s) of this event handler.\n     * \n     * @return a (HTML) string\n     */\n    public String getAuthor();\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/plugin/WikiPlugin.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.plugin;\n\nimport javax.servlet.ServletConfig;\n\nimport de.zib.scalaris.examples.wikipedia.WikiServletContext;\n\n/**\n * Simple plug-in interface for plug-ins in the {@link de.zib.scalaris.examples.wikipedia.bliki.WikiServlet} class.\n * \n * Note: this API is not stable and will probably change in future.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic interface WikiPlugin {\n    /**\n     * Initialises the plugin.\n     * \n     * @param servlet\n     *            the servlet using the plugin\n     * @param config\n     *            servlet config object\n     */\n    public void init(WikiServletContext servlet, ServletConfig config);\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/plugin/fourcaast/FourCaastAccounting.java",
    "content": "/**\n *  Copyright 2012-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.plugin.fourcaast;\n\nimport java.io.BufferedReader;\nimport java.io.IOException;\nimport java.io.InputStreamReader;\nimport java.io.OutputStreamWriter;\nimport java.net.HttpURLConnection;\nimport java.net.ProtocolException;\nimport java.net.URL;\nimport java.util.Calendar;\nimport java.util.GregorianCalendar;\nimport java.util.List;\nimport java.util.Map.Entry;\n\nimport javax.servlet.http.HttpServletRequest;\n\nimport org.apache.commons.lang.StringEscapeUtils;\n\nimport de.zib.scalaris.examples.wikipedia.SavePageResult;\nimport de.zib.scalaris.examples.wikipedia.ValueResult;\nimport de.zib.scalaris.examples.wikipedia.WikiServletContext;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiPageBean;\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiPageBeanBase;\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiPageEditBean;\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiServlet;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\nimport de.zib.scalaris.examples.wikipedia.plugin.WikiEventHandler;\n\n/**\n * Registers each page view with the accounting server.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class FourCaastAccounting implements WikiEventHandler {\n    protected final WikiServletContext servlet;\n    protected final String meteringInstructionId = \"de.zib.scalaris.examples.wikipedia.pages_viewed\";\n    protected final String eventName = \"Pages viewed\";\n    protected final URL accountingServer;\n\n    /**\n     * Creates the plugin and stores the wiki servlet context.\n     * \n     * @param servlet\n     *            servlet context\n     * @param accountingServer\n     *            the URL pointing to the accounting server\n     */\n    public FourCaastAccounting(WikiServletContext servlet, URL accountingServer) {\n        this.servlet = servlet;\n        this.accountingServer = accountingServer;\n    }\n\n    @Override\n    public <Connection> void onPageSaved(WikiPageEditBean page,\n            SavePageResult result, Connection connection) {\n        // if unsuccessful, #onPageEditPreview is called, too\n        if (result.success) {\n            pushSdrToServer(page);\n        }\n    }\n\n    @Override\n    public <Connection> void onPageView(WikiPageBeanBase page,\n            Connection connection) {\n        pushSdrToServer(page);\n    }\n\n    @Override\n    public void onImageRedirect(String image, String realImageUrl) {\n    }\n\n    @Override\n    public <Connection> void onViewRandomPage(WikiPageBean page,\n            ValueResult<NormalisedTitle> result, Connection connection) {\n        pushSdrToServer(page);\n    }\n\n    @Override\n    public <Connection> boolean checkAccess(String serviceUser,\n            HttpServletRequest request, Connection connection) {\n        // TODO check in the data store?\n        if (serviceUser.isEmpty()) {\n            WikiServlet.setParam_error(request, \"Access forbidden for user \\\"\"\n                    + serviceUser + \"\\\"\");\n            return false;\n        }\n        return true;\n    }\n    \n    protected String createSdr(final WikiPageBeanBase page) {\n        final Calendar now = GregorianCalendar.getInstance();\n        final long serverTime = System.currentTimeMillis() - page.getStartTime();\n        long dbTime = 0l;\n        for (Entry<String, List<Long>> stats : page.getStats().entrySet()) {\n            String statName = stats.getKey();\n            for (Long stat : stats.getValue()) {\n                if (statName.endsWith(\" (last op)\")) {\n                    // this is from a previous operation that lead to the\n                    // current page view, e.g. during random page view\n                    // -> exclude it here (it has already been accounted for)\n                } else {\n                    dbTime += stat;\n                }\n            }\n        }\n        return \"{\" + \n                \"\\\"TenantID\\\": \\\"\" + StringEscapeUtils.escapeJava(page.getServiceUser()) + \"\\\",\" +\n                \"\\\"MeteringInstructionId\\\": \\\"\" + meteringInstructionId + \"\\\",\" +\n                \"\\\"Timestamp\\\": \\\"\" + Revision.calendarToString(now) + \"\\\",\" +\n                \"\\\"RecordType\\\": \\\"NamedEvent\\\",\" +\n                \"\\\"EventName\\\": \\\"\" + eventName + \"\\\",\" +\n                \"\\\"Info\\\": {\" +\n                \"\\\"Title\\\": \\\"\" + StringEscapeUtils.escapeJava(page.getTitle()) + \"\\\",\" +\n                \"\\\"DBTime\\\": \\\"\" + dbTime + \"\\\",\" +\n                \"\\\"ServerTime\\\": \\\"\" + serverTime + \"\\\"\" +\n                \"}\"+\n                \"}\";\n    }\n    \n    protected void pushSdrToServer(final WikiPageBeanBase page) {\n        if (!page.getServiceUser().isEmpty()) {\n            final String sdr = createSdr(page);\n//            System.out.println(\"Sending sdr...\\n\" + sdr);\n            try {\n                HttpURLConnection urlConn = (HttpURLConnection) accountingServer\n                        .openConnection();\n                urlConn.setDoOutput(true);\n                urlConn.setRequestMethod(\"PUT\");\n                OutputStreamWriter out = new OutputStreamWriter(\n                        urlConn.getOutputStream());\n                out.write(sdr);\n                out.close();\n                \n                //read the result from the server (necessary for the request to be send!)\n                BufferedReader in = new BufferedReader(new InputStreamReader(\n                        urlConn.getInputStream()));\n                StringBuilder sb = new StringBuilder();\n                String line;\n                while ((line = in.readLine()) != null) {\n                    sb.append(line + '\\n');\n                }\n//                System.out.println(sb.toString());\n                in.close();\n            } catch (ProtocolException e) {\n                throw new RuntimeException(e);\n            } catch (IOException e) {\n                e.printStackTrace();\n            }\n        }\n\n    }\n\n    @Override\n    public String getName() {\n        return \"4CaaSt accounting\";\n    }\n    \n    @Override\n    public String getURL() {\n        return \"https://code.google.com/p/scalaris/\";\n    }\n\n    @Override\n    public String getVersion() {\n        return \"0.1\";\n    }\n\n    @Override\n    public String getDescription() {\n        return \"Registers page-view events with an accounting server.\";\n    }\n\n    @Override\n    public String getAuthor() {\n        return \"Nico Kruber\";\n    }\n\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/plugin/fourcaast/FourCaastAccountingPlugin.java",
    "content": "/**\n *  Copyright 2012-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.plugin.fourcaast;\n\nimport java.net.MalformedURLException;\nimport java.net.URL;\n\nimport javax.servlet.ServletConfig;\n\nimport de.zib.scalaris.examples.wikipedia.WikiServletContext;\nimport de.zib.scalaris.examples.wikipedia.plugin.WikiPlugin;\n\n/**\n * Registers the 4CaaSt accounting plugin.\n *\n * @author Nico Kruber, kruber@zib.de\n */\npublic class FourCaastAccountingPlugin implements WikiPlugin {\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.plugin.WikiPlugin#init(de.zib.scalaris.examples.wikipedia.WikiServletContext, javax.servlet.ServletConfig)\n     */\n    @Override\n    public void init(WikiServletContext servlet, ServletConfig config) {\n        System.out.println(\"Activating 4CaaSt accounting plugin...\");\n        final String accountingServer = config.getInitParameter(\"4CaaSt.accounting\");\n        if (accountingServer != null) {\n            try {\n                servlet.registerEventHandler(new FourCaastAccounting(servlet,\n                        new URL(accountingServer)));\n            } catch (MalformedURLException e) {\n                System.err.println(\"Invalid URL in \\\"4CaaSt.accounting\\\": \" + e.getMessage());\n            }\n        } else {\n            System.err.println(\"Accounting plugin activated but no \\\"4CaaSt.accounting\\\" parameter present - disabled plugin\");\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/plugin/fourcaast/FourCaastMonitoring.java",
    "content": "/**\n *  Copyright 2012-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.plugin.fourcaast;\n\nimport java.util.Calendar;\nimport java.util.GregorianCalendar;\nimport java.util.List;\nimport java.util.Map.Entry;\n\nimport javax.servlet.http.HttpServletRequest;\n\nimport org.apache.commons.lang.StringEscapeUtils;\n\nimport de.zib.scalaris.examples.wikipedia.SavePageResult;\nimport de.zib.scalaris.examples.wikipedia.ValueResult;\nimport de.zib.scalaris.examples.wikipedia.WikiServletContext;\nimport de.zib.scalaris.examples.wikipedia.bliki.NormalisedTitle;\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiPageBean;\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiPageBeanBase;\nimport de.zib.scalaris.examples.wikipedia.bliki.WikiPageEditBean;\nimport de.zib.scalaris.examples.wikipedia.data.Revision;\nimport de.zib.scalaris.examples.wikipedia.plugin.WikiEventHandler;\n\n/**\n * Monitors page views and aggregates certain attributes.\n * \n * @author Nico Kruber, kruber@zib.de\n */\npublic class FourCaastMonitoring implements WikiEventHandler, FourCaastMonitoringMBean {\n    protected final WikiServletContext servlet;\n    final static protected Record NULL_RECORD = new Record(\"\", \"\", \"\", 0l, 0l, 0l);\n    protected Record lastRecord = NULL_RECORD; \n\n    /**\n     * Creates the plugin and stores the wiki servlet context.\n     * \n     * @param servlet\n     *            servlet context\n     */\n    public FourCaastMonitoring(WikiServletContext servlet) {\n        this.servlet = servlet;\n        resetMonitoringStats();\n    }\n\n    @Override\n    public <Connection> void onPageSaved(WikiPageEditBean page,\n            SavePageResult result, Connection connection) {\n        // if unsuccessful, #onPageEditPreview is called, too\n        if (result.success) {\n            extractMonitoringStats(page);\n        }\n    }\n\n    @Override\n    public <Connection> void onPageView(WikiPageBeanBase page,\n            Connection connection) {\n        extractMonitoringStats(page);\n    }\n\n    @Override\n    public void onImageRedirect(String image, String realImageUrl) {\n    }\n\n    @Override\n    public <Connection> void onViewRandomPage(WikiPageBean page,\n            ValueResult<NormalisedTitle> result, Connection connection) {\n        extractMonitoringStats(page);\n    }\n\n    @Override\n    public <Connection> boolean checkAccess(String serviceUser,\n            HttpServletRequest request, Connection connection) {\n        return true;\n    }\n    \n    protected synchronized void extractMonitoringStats(final WikiPageBeanBase page) {\n        final Calendar now = GregorianCalendar.getInstance();\n        long dbTime = 0l;\n        for (Entry<String, List<Long>> stats : page.getStats().entrySet()) {\n            String statName = stats.getKey();\n            for (Long stat : stats.getValue()) {\n                if (statName.endsWith(\" (last op)\")) {\n                    // this is from a previous operation that lead to the\n                    // current page view, e.g. during random page view\n                    // -> exclude it here (there has already been a monitoring call for it)\n                } else {\n                    dbTime += stat;\n                }\n            }\n        }\n        final long serverTime = System.currentTimeMillis() - page.getStartTime();\n        final long renderTime = serverTime - dbTime;\n        lastRecord = new Record(page.getServiceUser(), Revision.calendarToString(now),\n                StringEscapeUtils.escapeJava(page.getTitle()),\n                serverTime, dbTime, renderTime);\n    }\n    \n    @Override\n    public\n    synchronized void resetMonitoringStats() {\n        lastRecord = NULL_RECORD;\n    }\n\n    @Override\n    public String getName() {\n        return \"4CaaSt monitoring\";\n    }\n    \n    @Override\n    public String getURL() {\n        return \"https://code.google.com/p/scalaris/\";\n    }\n\n    @Override\n    public String getVersion() {\n        return \"0.1\";\n    }\n\n    @Override\n    public String getDescription() {\n        return \"Exposes several timing values via JMX.\";\n    }\n\n    @Override\n    public String getAuthor() {\n        return \"Nico Kruber\";\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.plugin.fourcaast.FourCaastMonitoringMBean#getServletVersion()\n     */\n    @Override\n    public String getServletVersion() {\n        return servlet.getVersion();\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.plugin.fourcaast.FourCaastMonitoringMBean#getDbVersion()\n     */\n    @Override\n    public String getDbVersion() {\n        return servlet.getDbVersion();\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.plugin.fourcaast.FourCaastMonitoringMBean#getServerVersion()\n     */\n    @Override\n    public String getServerVersion() {\n        return servlet.getServerVersion();\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.plugin.fourcaast.FourCaastMonitoringMBean#getBlikiVersion()\n     */\n    @Override\n    public String getBlikiVersion() {\n        return servlet.getBlikiVersion();\n    }\n\n    /**\n     * @return the lastServiceUser\n     */\n    @Override\n    public String getLastServiceUser() {\n        return lastRecord.serviceUser;\n    }\n\n    /**\n     * @return the lastTimestamp\n     */\n    @Override\n    public String getLastTimestamp() {\n        return lastRecord.timestamp;\n    }\n\n    /**\n     * @return the lastTitle\n     */\n    @Override\n    public String getLastTitle() {\n        return lastRecord.title;\n    }\n\n    /**\n     * @return the lastDbTime\n     */\n    @Override\n    public long getLastDbTime() {\n        return lastRecord.dbTime;\n    }\n\n    /**\n     * @return the lastServerTime\n     */\n    @Override\n    public long getLastServerTime() {\n        return lastRecord.serverTime;\n    }\n\n    /**\n     * @return the lastRenderTime\n     */\n    @Override\n    public long getLastRenderTime() {\n        return lastRecord.renderTime;\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/plugin/fourcaast/FourCaastMonitoringMBean.java",
    "content": "package de.zib.scalaris.examples.wikipedia.plugin.fourcaast;\n\n/**\n * Provides methods to monitor the Wiki servlet via JMX.\n *\n * @author Nico Kruber, kruber@zib.de\n */\npublic interface FourCaastMonitoringMBean {\n\n    /**\n     * Class holding a single page view record.\n     * \n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static class Record {\n        /**\n         * Creates a new record.\n         * \n         * @param serviceUser\n         * @param timestamp\n         * @param title\n         * @param serverTime\n         * @param dbTime\n         * @param renderTime\n         */\n        public Record(String serviceUser, String timestamp, String title,\n                long serverTime, long dbTime, long renderTime) {\n            this.serviceUser = serviceUser;\n            this.timestamp = timestamp;\n            this.title = title;\n            this.serverTime = serverTime;\n            this.dbTime = dbTime;\n            this.renderTime = renderTime;\n        }\n\n        protected String serviceUser;\n        protected String timestamp;\n        protected String title;\n        protected long dbTime;\n        protected long serverTime;\n        protected long renderTime;\n    }\n    \n    public abstract String getServletVersion();\n\n    public abstract String getDbVersion();\n\n    public abstract String getServerVersion();\n\n    public abstract String getBlikiVersion();\n\n    public abstract long getLastRenderTime();\n\n    public abstract long getLastDbTime();\n\n    public abstract String getLastTitle();\n\n    public abstract String getLastTimestamp();\n\n    public abstract String getLastServiceUser();\n\n    public abstract void resetMonitoringStats();\n\n    public abstract long getLastServerTime();\n\n}"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/plugin/fourcaast/FourCaastMonitoringPlugin.java",
    "content": "/**\n *  Copyright 2012-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.plugin.fourcaast;\n\nimport java.lang.management.ManagementFactory;\n\nimport javax.management.InstanceAlreadyExistsException;\nimport javax.management.MBeanRegistrationException;\nimport javax.management.MBeanServer;\nimport javax.management.MalformedObjectNameException;\nimport javax.management.NotCompliantMBeanException;\nimport javax.management.ObjectName;\nimport javax.servlet.ServletConfig;\n\nimport de.zib.scalaris.examples.wikipedia.WikiServletContext;\nimport de.zib.scalaris.examples.wikipedia.plugin.WikiPlugin;\n\n/**\n * Registers the 4CaaSt monitoring plugin.\n *\n * @author Nico Kruber, kruber@zib.de\n */\npublic class FourCaastMonitoringPlugin implements WikiPlugin {\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.plugin.WikiPlugin#init(de.zib.scalaris.examples.wikipedia.WikiServletContext, javax.servlet.ServletConfig)\n     */\n    @Override\n    public void init(WikiServletContext servlet, ServletConfig config) {\n        System.out.println(\"Activating 4CaaSt monitoring plugin...\");\n        final FourCaastMonitoring monitor = new FourCaastMonitoring(servlet);\n        servlet.registerEventHandler(monitor);\n        try {\n            final MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();\n            final ObjectName nodeMonitorName = new ObjectName(\"de.zib.scalaris:type=FourCaastMonitoring\");\n            final FourCaastMonitoring nodeMonitorMbean = monitor;\n            mbs.registerMBean(nodeMonitorMbean, nodeMonitorName);\n        } catch (final MalformedObjectNameException e) {\n            throw new RuntimeException(e);\n        } catch (final InstanceAlreadyExistsException e) {\n            throw new RuntimeException(e);\n        } catch (final MBeanRegistrationException e) {\n            throw new RuntimeException(e);\n        } catch (final NotCompliantMBeanException e) {\n            throw new RuntimeException(e);\n        }\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/tomcat/FailedRequestFilter.java",
    "content": "/*\n * Licensed to the Apache Software Foundation (ASF) under one or more\n * contributor license agreements.  See the NOTICE file distributed with\n * this work for additional information regarding copyright ownership.\n * The ASF licenses this file to You under the Apache License, Version 2.0\n * (the \"License\"); you may not use this file except in compliance with\n * the License.  You may obtain a copy of the License at\n *\n *      http://www.apache.org/licenses/LICENSE-2.0\n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n * See the License for the specific language governing permissions and\n * limitations under the License.\n */\npackage de.zib.scalaris.examples.wikipedia.tomcat;\n\nimport java.io.IOException;\nimport java.lang.reflect.Field;\nimport java.util.Enumeration;\n\nimport javax.servlet.Filter;\nimport javax.servlet.FilterChain;\nimport javax.servlet.FilterConfig;\nimport javax.servlet.ServletException;\nimport javax.servlet.ServletRequest;\nimport javax.servlet.ServletResponse;\nimport javax.servlet.http.HttpServletResponse;\n\n/**\n * Filter that will reject requests if there was a failure during parameter\n * parsing. This filter can be used to ensure that none parameter values\n * submitted by client are lost.\n *\n * <p>\n * Note that it has side effect that it triggers parameter parsing and thus\n * consumes the body for POST requests. Parameter parsing does check content\n * type of the request, so there should not be problems with addresses that use\n * <code>request.getInputStream()</code> and <code>request.getReader()</code>,\n * if requests parsed by them do not use standard value for content mime-type.\n */\npublic class FailedRequestFilter implements Filter {\n    String PARAMETER_PARSE_FAILED_ATTR = null;\n    \n    @Override\n    public void init(FilterConfig filterConfig) throws ServletException {\n        Enumeration<String> paramNames = filterConfig.getInitParameterNames();\n        while (paramNames.hasMoreElements()) {\n            String paramName = paramNames.nextElement();\n            String msg = \"The property \" + paramName\n                    + \" is not defined for filters of type \"\n                    + this.getClass().getName();\n            throw new ServletException(msg);\n        }\n        // apache tomcat:\n        try {\n            Class<?> globals = Class.forName(\"org.apache.catalina.Globals\");\n            Field f = globals.getDeclaredField(\"PARAMETER_PARSE_FAILED_ATTR\");\n            PARAMETER_PARSE_FAILED_ATTR = (String) f.get(null);\n        } catch (ClassNotFoundException e) {\n        } catch (SecurityException e) {\n            throw new ServletException(e);\n        } catch (NoSuchFieldException e) {\n            throw new ServletException(e);\n        } catch (IllegalAccessException e) {\n            throw new ServletException(e);\n        }\n    }\n\n    @Override\n    public void doFilter(ServletRequest request, ServletResponse response,\n            FilterChain chain) throws IOException, ServletException {\n        if (!isGoodRequest(request)) {\n            ((HttpServletResponse) response)\n                    .sendError(HttpServletResponse.SC_BAD_REQUEST);\n            return;\n        }\n        chain.doFilter(request, response);\n    }\n\n    private boolean isGoodRequest(ServletRequest request) throws ServletException {\n        // Trigger parsing of parameters\n        try {\n            request.getParameter(\"none\");\n            // Detect failure (apache tomcat)\n            if (PARAMETER_PARSE_FAILED_ATTR != null\n                    && request.getAttribute(PARAMETER_PARSE_FAILED_ATTR) != null) {\n                return false;\n            }\n            return true;\n        } catch (IllegalArgumentException e) {\n            // Detect failure (jetty)\n            return false;\n        }\n    }\n\n    @Override\n    public void destroy() {\n        // NOOP\n    }\n\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/tomcat/SetCharacterEncodingFilter.java",
    "content": "/*\n* Licensed to the Apache Software Foundation (ASF) under one or more\n* contributor license agreements.  See the NOTICE file distributed with\n* this work for additional information regarding copyright ownership.\n* The ASF licenses this file to You under the Apache License, Version 2.0\n* (the \"License\"); you may not use this file except in compliance with\n* the License.  You may obtain a copy of the License at\n*\n*     http://www.apache.org/licenses/LICENSE-2.0\n*\n* Unless required by applicable law or agreed to in writing, software\n* distributed under the License is distributed on an \"AS IS\" BASIS,\n* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n* See the License for the specific language governing permissions and\n* limitations under the License.\n*/\npackage de.zib.scalaris.examples.wikipedia.tomcat;\n\nimport java.io.IOException;\nimport java.util.Enumeration;\n\nimport javax.servlet.Filter;\nimport javax.servlet.FilterChain;\nimport javax.servlet.FilterConfig;\nimport javax.servlet.ServletException;\nimport javax.servlet.ServletRequest;\nimport javax.servlet.ServletResponse;\n\n\n/**\n * <p>Example filter that sets the character encoding to be used in parsing the\n * incoming request, either unconditionally or only if the client did not\n * specify a character encoding.  Configuration of this filter is based on\n * the following initialization parameters:</p>\n * <ul>\n * <li><strong>encoding</strong> - The character encoding to be configured\n *     for this request, either conditionally or unconditionally based on\n *     the <code>ignore</code> initialization parameter.  This parameter\n *     is required, so there is no default.</li>\n * <li><strong>ignore</strong> - If set to \"true\", any character encoding\n *     specified by the client is ignored, and the value returned by the\n *     <code>selectEncoding()</code> method is set.  If set to \"false,\n *     <code>selectEncoding()</code> is called <strong>only</strong> if the\n *     client has not already specified an encoding.  By default, this\n *     parameter is set to \"false\".</li>\n * </ul>\n *\n * <p>Although this filter can be used unchanged, it is also easy to\n * subclass it and make the <code>selectEncoding()</code> method more\n * intelligent about what encoding to choose, based on characteristics of\n * the incoming request (such as the values of the <code>Accept-Language</code>\n * and <code>User-Agent</code> headers, or a value stashed in the current\n * user's session.</p>\n */\npublic class SetCharacterEncodingFilter implements Filter {\n\n\n    // ----------------------------------------------------- Instance Variables\n\n    /**\n     * The default character encoding to set for requests that pass through\n     * this filter.\n     */\n    private String encoding = null;\n    public void setEncoding(String encoding) { this.encoding = encoding; }\n    public String getEncoding() { return encoding; }\n\n\n    /**\n     * Should a character encoding specified by the client be ignored?\n     */\n    private boolean ignore = false;\n    public void setIgnore(boolean ignore) { this.ignore = ignore; }\n    public boolean isIgnore() { return ignore; }\n\n\n    // --------------------------------------------------------- Public Methods\n    \n    @Override\n    public void init(FilterConfig filterConfig) throws ServletException {\n        Enumeration<String> paramNames = filterConfig.getInitParameterNames();\n        while (paramNames.hasMoreElements()) {\n            final String paramName = paramNames.nextElement();\n            final String paramValue = filterConfig.getInitParameter(paramName);\n            if (paramName.equals(\"encoding\")) {\n                setEncoding(paramValue);\n            } else if (paramName.equals(\"ignore\")) {\n                setIgnore(Boolean.valueOf(paramValue));\n            } else {\n//                String msg = \"The property \" + paramName\n//                        + \" is not defined for filters of type \"\n//                        + this.getClass().getName();\n//                throw new ServletException(msg);\n            }\n        }\n    }\n\n    /**\n     * Select and set (if specified) the character encoding to be used to\n     * interpret request parameters for this request.\n     *\n     * @param request The servlet request we are processing\n     * @param response The servlet response we are creating\n     * @param chain The filter chain we are processing\n     *\n     * @exception IOException if an input/output error occurs\n     * @exception ServletException if a servlet error occurs\n     */\n    @Override\n    public void doFilter(ServletRequest request, ServletResponse response,\n                         FilterChain chain)\n        throws IOException, ServletException {\n\n        // Conditionally select and set the character encoding to be used\n        if (ignore || (request.getCharacterEncoding() == null)) {\n            String characterEncoding = selectEncoding(request);\n            if (characterEncoding != null) {\n                request.setCharacterEncoding(characterEncoding);\n            }\n        }\n\n        // Pass control on to the next filter\n        chain.doFilter(request, response);\n    }\n\n\n    // ------------------------------------------------------ Protected Methods\n\n\n    /**\n     * Select an appropriate character encoding to be used, based on the\n     * characteristics of the current request and/or filter initialization\n     * parameters.  If no character encoding should be set, return\n     * <code>null</code>.\n     * <p>\n     * The default implementation unconditionally returns the value configured\n     * by the <strong>encoding</strong> initialization parameter for this\n     * filter.\n     *\n     * @param request The servlet request we are processing\n     */\n    protected String selectEncoding(ServletRequest request) {\n        return this.encoding;\n    }\n    @Override\n    public void destroy() {\n        // NOOP\n    }\n}\n"
  },
  {
    "path": "contrib/wikipedia/src/de/zib/scalaris/examples/wikipedia/tomcat/URLParamEncoder.java",
    "content": "package de.zib.scalaris.examples.wikipedia.tomcat;\n\nimport java.io.UnsupportedEncodingException;\nimport java.net.URLEncoder;\n\npublic class URLParamEncoder {\n\n    public static String encode(String s) throws UnsupportedEncodingException {\n        StringBuilder b = new StringBuilder(URLEncoder.encode(s, \"UTF-8\"));\n        for (int i = 0; i < s.length(); i++) {\n            char c = s.charAt(i);\n            switch (c) {\n            case '?':\n                b.append(\"%3F\");\n                break;\n            case '&':\n                b.append(\"%26\");\n                break;\n            default:\n                b.append(c);\n            }\n        }\n        return b.toString();\n    }\n}\n"
  },
  {
    "path": "contrib/yaws/LICENSE",
    "content": "\nCopyright (c) 2006, Claes Wikstrom, klacke@hyber.org\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions are met:\n\n    * Redistributions of source code must retain the above copyright\n      notice, this list of conditions and the following disclaimer.\n    * Redistributions in binary form must reproduce the above copyright\n      notice, this list of conditions and the following disclaimer in the\n      documentation and/or other materials provided with the distribution.\n    * Neither the name of \"Yaws\" nor the names of its contributors may be\n      used to endorse or promote products derived from this software without\n      specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\"\nAND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\nIMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\nARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE\nLIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR\nCONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF\nSUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS\nINTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN\nCONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)\nARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE\nPOSSIBILITY OF SUCH DAMAGE.\n"
  },
  {
    "path": "contrib/yaws/include/soap-envelope.hrl",
    "content": "%% HRL file generated by ERLSOM\n%%\n%% It is possible to change the name of the record fields.\n%%\n%% It is possible to add default values, but be aware that these will\n%% only be used when *writing* an xml document.\n\n-record('soap:UpgradeType', {anyAttribs, 'SupportedEnvelope'}).\n-record('soap:SupportedEnvType', {anyAttribs, 'qname'}).\n-record('soap:NotUnderstoodType', {anyAttribs, 'qname'}).\n-record('soap:detail', {anyAttribs, choice}).\n-record('soap:subcode', {anyAttribs, 'Value', 'Subcode'}).\n-record('soap:faultcode', {anyAttribs, 'Value', 'Subcode'}).\n-record('soap:reasontext', {anyAttribs, 'P:lang', '#text'}).\n-record('soap:faultreason', {anyAttribs, 'Text'}).\n-record('soap:Fault', {anyAttribs, 'Code', 'Reason', 'Node', 'Role', 'Detail'}).\n-record('soap:Body', {anyAttribs, choice}).\n-record('soap:Header', {anyAttribs, choice}).\n-record('soap:Envelope', {anyAttribs, 'Header', 'Body'}).\n"
  },
  {
    "path": "contrib/yaws/include/soap.hrl",
    "content": "-record(wsdl, {operations, model, module}).\n-record(port, {service, port, binding, address}).\n-record(operation, {service, port, operation, binding, address, action}).\n-record('soap:detail', {anyAttribs, choice}).\n-record('soap:Fault', {anyAttribs, 'faultcode', 'faultstring', 'faultactor', 'detail'}).\n-record('soap:Body', {anyAttribs, choice}).\n-record('soap:Header', {anyAttribs, choice}).\n-record('soap:Envelope', {anyAttribs, 'Header', 'Body', choice}).\n-record('wsdl:tExtensibilityElement', {anyAttribs, 'wsdl:required'}).\n-record('wsdl:tPort', {anyAttribs, 'name', 'binding', 'documentation', choice}).\n-record('wsdl:tService', {anyAttribs, 'name', 'documentation', choice, 'port'}).\n-record('wsdl:tBindingOperation', {anyAttribs, 'name', 'documentation', choice, 'input', 'output', 'fault'}).\n-record('wsdl:tBindingOperationFault', {anyAttribs, 'name', 'documentation', choice}).\n-record('wsdl:tBindingOperationMessage', {anyAttribs, 'name', 'documentation', choice}).\n-record('wsdl:tBinding', {anyAttribs, 'name', 'type', 'documentation', choice, 'operation'}).\n-record('wsdl:tFault', {anyAttribs, 'name', 'message', 'documentation'}).\n-record('wsdl:tParam', {anyAttribs, 'name', 'message', 'documentation'}).\n-record('wsdl:solicit-response-or-notification-operation', {anyAttribs, 'output', 'solicit-response-or-notification-operation/SEQ2'}).\n-record('wsdl:solicit-response-or-notification-operation/SEQ2', {anyAttribs, 'input', 'fault'}).\n-record('wsdl:request-response-or-one-way-operation', {anyAttribs, 'input', 'request-response-or-one-way-operation/SEQ1'}).\n-record('wsdl:request-response-or-one-way-operation/SEQ1', {anyAttribs, 'output', 'fault'}).\n-record('wsdl:tOperation', {anyAttribs, 'name', 'parameterOrder', 'documentation', any, choice}).\n-record('wsdl:tPortType', {anyAttribs, 'name', 'documentation', 'operation'}).\n-record('wsdl:tPart', {anyAttribs, 'name', 'element', 'type', 'documentation'}).\n-record('wsdl:tMessage', {anyAttribs, 'name', 'documentation', choice, 'part'}).\n-record('wsdl:tTypes', {anyAttribs, 'documentation', choice}).\n-record('wsdl:tImport', {anyAttribs, 'namespace', 'location', 'documentation'}).\n-record('wsdl:tDefinitions', {anyAttribs, 'targetNamespace', 'name', 'documentation', any, choice}).\n-record('wsdl:anyTopLevelOptionalElement-service', {anyAttribs, 'service'}).\n-record('wsdl:anyTopLevelOptionalElement-binding', {anyAttribs, 'binding'}).\n-record('wsdl:anyTopLevelOptionalElement-portType', {anyAttribs, 'portType'}).\n-record('wsdl:anyTopLevelOptionalElement-message', {anyAttribs, 'message'}).\n-record('wsdl:anyTopLevelOptionalElement-types', {anyAttribs, 'types'}).\n-record('wsdl:anyTopLevelOptionalElement-import', {anyAttribs, 'import'}).\n-record('wsdl:anyTopLevelOptionalElement', {anyAttribs, choice}).\n-record('wsdl:tExtensibleDocumented', {anyAttribs, 'documentation', choice}).\n-record('wsdl:tExtensibleAttributesDocumented', {anyAttribs, 'documentation'}).\n-record('wsdl:tDocumented', {anyAttribs, 'documentation'}).\n-record('wsdl:tDocumentation-any', {anyAttribs, choice}).\n-record('wsdl:tDocumentation', {anyAttribs, choice}).\n-record('soap:tBinding', {anyAttribs, 'wsdl:required', 'transport', 'style'}).\n-record('soap:tOperation', {anyAttribs, 'wsdl:required', 'soapAction', 'style'}).\n-record('soap:tBody', {anyAttribs, 'wsdl:required', 'parts', 'namespace', 'use', 'encodingStyle'}).\n-record('soap:tFaultRes', {anyAttribs, 'wsdl:required', 'parts', 'namespace', 'use', 'encodingStyle'}).\n-record('soap:tFault', {anyAttribs, 'wsdl:required', 'parts', 'namespace', 'use', 'encodingStyle', 'name'}).\n-record('soap:tHeader', {anyAttribs, 'wsdl:required', 'namespace', 'encodingStyle', 'use', 'part', 'message', 'headerfault'}).\n-record('soap:tHeaderFault', {anyAttribs, 'namespace', 'encodingStyle', 'use', 'part', 'message'}).\n-record('soap:tAddress', {anyAttribs, 'wsdl:required', 'location'}).\n"
  },
  {
    "path": "contrib/yaws/include/wsdl11soap12.hrl",
    "content": "%% HRL file generated by ERLSOM\n%%\n%% It is possible to change the name of the record fields.\n%%\n%% It is possible to add default values, but be aware that these will\n%% only be used when *writing* an xml document.\n\n-record('soap:tAddress', {anyAttribs, 'wsdl:required', 'location'}).\n-record('soap:tHeaderFault', {anyAttribs, 'message', 'part', 'use', 'encodingStyle', 'namespace'}).\n-record('soap:tHeader', {anyAttribs, 'wsdl:required', 'message', 'part', 'use', 'encodingStyle', 'namespace', 'headerfault'}).\n-record('soap:tFault', {anyAttribs, 'wsdl:required', 'parts', 'encodingStyle', 'use', 'namespace', 'name'}).\n-record('soap:tFaultRes', {anyAttribs, 'wsdl:required', 'parts', 'encodingStyle', 'use', 'namespace'}).\n-record('soap:tBody', {anyAttribs, 'wsdl:required', 'parts', 'encodingStyle', 'use', 'namespace'}).\n-record('soap:tOperation', {anyAttribs, 'wsdl:required', 'soapAction', 'soapActionRequired', 'style'}).\n-record('soap:tBinding', {anyAttribs, 'wsdl:required', 'transport', 'style'}).\n-record('soap:tExtensibilityElementOpenAttrs', {anyAttribs, 'wsdl:required'}).\n-record('wsdl:tDocumentation', {anyAttribs, choice}).\n-record('wsdl:tDocumented', {anyAttribs, 'documentation'}).\n-record('wsdl:tExtensibleAttributesDocumented', {anyAttribs, 'documentation'}).\n-record('wsdl:tExtensibleDocumented', {anyAttribs, 'documentation', choice}).\n-record('wsdl:anyTopLevelOptionalElement', {anyAttribs, choice}).\n-record('wsdl:tDefinitions', {anyAttribs, 'targetNamespace', 'name', 'documentation', choice, choice1}).\n-record('wsdl:tImport', {anyAttribs, 'namespace', 'location', 'documentation'}).\n-record('wsdl:tTypes', {anyAttribs, 'documentation', choice}).\n-record('wsdl:tMessage', {anyAttribs, 'name', 'documentation', choice, 'part'}).\n-record('wsdl:tPart', {anyAttribs, 'name', 'element', 'type', 'documentation'}).\n-record('wsdl:tPortType', {anyAttribs, 'name', 'documentation', 'operation'}).\n-record('wsdl:tOperation', {anyAttribs, 'name', 'parameterOrder', 'documentation', choice, choice1}).\n-record('wsdl:request-response-or-one-way-operation/SEQ1', {anyAttribs, 'output', 'fault'}).\n-record('wsdl:request-response-or-one-way-operation', {anyAttribs, 'input', 'request-response-or-one-way-operation/SEQ1'}).\n-record('wsdl:solicit-response-or-notification-operation/SEQ2', {anyAttribs, 'input', 'fault'}).\n-record('wsdl:solicit-response-or-notification-operation', {anyAttribs, 'output', 'solicit-response-or-notification-operation/SEQ2'}).\n-record('wsdl:tParam', {anyAttribs, 'name', 'message', 'documentation'}).\n-record('wsdl:tFault', {anyAttribs, 'name', 'message', 'documentation'}).\n-record('wsdl:tBinding', {anyAttribs, 'name', 'type', 'documentation', choice, 'operation'}).\n-record('wsdl:tBindingOperationMessage', {anyAttribs, 'name', 'documentation', choice}).\n-record('wsdl:tBindingOperationFault', {anyAttribs, 'name', 'documentation', choice}).\n-record('wsdl:tBindingOperation', {anyAttribs, 'name', 'documentation', choice, 'input', 'output', 'fault'}).\n-record('wsdl:tService', {anyAttribs, 'name', 'documentation', choice, 'port'}).\n-record('wsdl:tPort', {anyAttribs, 'name', 'binding', 'documentation', choice}).\n-record('wsdl:tExtensibilityElement', {anyAttribs, 'wsdl:required'}).\n"
  },
  {
    "path": "contrib/yaws/include/yaws.hrl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws.hrl\n%%% Author  : Claes Wikstrom <klacke@hyber.org>\n%%% Purpose :\n%%% Created : 16 Jan 2002 by Claes Wikstrom <klacke@hyber.org>\n%%%----------------------------------------------------------------------\n\n-author('klacke@hyber.org').\n\n\n\n%% flags for gconfs\n-define(GC_TTY_TRACE,                        1).\n-define(GC_DEBUG,                            2).\n-define(GC_COPY_ERRLOG,                      4).\n-define(GC_BACKWARDS_COMPAT_PARSE,           8).\n-define(GC_LOG_RESOLVE_HOSTNAME,            16).\n-define(GC_FAIL_ON_BIND_ERR,                32).\n-define(GC_PICK_FIRST_VIRTHOST_ON_NOMATCH,  64).\n-define(GC_USE_FDSRV,                      128).\n-define(GC_USE_ERLANG_SENDFILE,            256).\n-define(GC_USE_YAWS_SENDFILE,              512).\n\n\n\n-define(GC_DEF, ?GC_FAIL_ON_BIND_ERR).\n\n-define(gc_has_tty_trace(GC),\n        ((GC#gconf.flags band ?GC_TTY_TRACE) /= 0)).\n-define(gc_has_debug(GC),\n        ((GC#gconf.flags band ?GC_DEBUG) /= 0)).\n-define(gc_has_copy_errlog(GC),\n        ((GC#gconf.flags band ?GC_COPY_ERRLOG) /= 0)).\n-define(gc_log_has_resolve_hostname(GC),\n        ((GC#gconf.flags band ?GC_LOG_RESOLVE_HOSTNAME) /= 0)).\n-define(gc_fail_on_bind_err(GC),\n        ((GC#gconf.flags band ?GC_FAIL_ON_BIND_ERR) /= 0)).\n-define(gc_pick_first_virthost_on_nomatch(GC),\n        ((GC#gconf.flags band ?GC_PICK_FIRST_VIRTHOST_ON_NOMATCH) /= 0)).\n-define(gc_use_erlang_sendfile(GC),\n        ((GC#gconf.flags band ?GC_USE_ERLANG_SENDFILE) /= 0)).\n-define(gc_use_yaws_sendfile(GC),\n        ((GC#gconf.flags band ?GC_USE_YAWS_SENDFILE) /= 0)).\n\n-define(gc_set_tty_trace(GC, Bool),\n        GC#gconf{flags = yaws:flag(GC#gconf.flags,?GC_TTY_TRACE, Bool)}).\n-define(gc_set_debug(GC, Bool),\n        GC#gconf{flags = yaws:flag(GC#gconf.flags, ?GC_DEBUG, Bool)}).\n-define(gc_set_copy_errlog(GC, Bool),\n        GC#gconf{flags = yaws:flag(GC#gconf.flags, ?GC_COPY_ERRLOG, Bool)}).\n-define(gc_log_set_resolve_hostname(GC, Bool),\n        GC#gconf{flags = yaws:flag(GC#gconf.flags,\n                                   ?GC_LOG_RESOLVE_HOSTNAME, Bool)}).\n-define(gc_set_fail_on_bind_err(GC, Bool),\n        GC#gconf{flags = yaws:flag(GC#gconf.flags,?GC_FAIL_ON_BIND_ERR,Bool)}).\n-define(gc_set_pick_first_virthost_on_nomatch(GC, Bool),\n        GC#gconf{flags = yaws:flag(GC#gconf.flags,\n                                   ?GC_PICK_FIRST_VIRTHOST_ON_NOMATCH,Bool)}).\n-define(gc_set_use_erlang_sendfile(GC, Bool),\n        GC#gconf{flags = yaws:flag(GC#gconf.flags,?GC_USE_ERLANG_SENDFILE,Bool)}).\n-define(gc_set_use_yaws_sendfile(GC, Bool),\n        GC#gconf{flags = yaws:flag(GC#gconf.flags,?GC_USE_YAWS_SENDFILE,Bool)}).\n\n\n%% global conf\n-record(gconf,{\n          yaws_dir,                       % topdir of Yaws installation\n          trace,                          % false | {true,http} | {true,traffic}\n          flags = ?GC_DEF,                % boolean flags\n          logdir,\n          ebin_dir = [],\n          src_dir  = [],\n          runmods  = [],                  % runmods for entire server\n          keepalive_timeout    = 30000,\n          keepalive_maxuses    = nolimit, % nolimit or non negative integer\n          max_num_cached_files = 400,\n          max_num_cached_bytes = 1000000, % 1 MEG\n          max_size_cached_file = 8000,\n          max_connections      = nolimit, % max number of TCP connections\n\n          %% Override default connection handler processes spawn options for\n          %% performance/memory tuning.\n          %% [] | [{fullsweep_after,Number}, {min_heap_size, Size}]\n          %% other options such as monitor, link are ignored.\n          process_options = [],\n\n          large_file_chunk_size = 10240,\n          mnesia_dir            = [],\n          log_wrap_size         = 10000000, % wrap logs after 10M\n          cache_refresh_secs    = 30,       % seconds  (auto zero when debug)\n          include_dir           = [],       % list of inc dirs for .yaws files\n          phpexe = \"/usr/bin/php-cgi\",      % cgi capable php executable\n\n          yaws,                % server string\n          id = \"default\",      % string identifying this instance of yaws\n\n          enable_soap = false, % start yaws_soap_srv iff true\n\n          %% a list of\n          %% {{Mod, Func}, WsdlFile, Prefix} | {{Mod, Func}, WsdlFile}\n          %% automatically setup in yaws_soap_srv init.\n          soap_srv_mods = [],\n\n          ysession_mod = yaws_session_server, % storage module for ysession\n          acceptor_pool_size = 8,             % size of acceptor proc pool\n\n          mime_types_info,                    % undefined | #mime_types_info{}\n          nslookup_pref = [inet],             % [inet | inet6]\n          ysession_idle_timeout = 2*60*1000,  % default 2 minutes\n          ysession_long_timeout = 60*60*1000, % default 1 hour\n\n          sni = disable % disable | enable | strict\n         }).\n\n-record(ssl, {\n          keyfile,\n          certfile,\n          verify = verify_none,\n          fail_if_no_peer_cert,\n          depth = 1,\n          password,\n          cacertfile,\n          dhfile,\n          ciphers,\n          cachetimeout,\n          secure_renegotiate = false,\n          client_renegotiation = case yaws_dynopts:have_ssl_client_renegotiation() of\n                                     true  -> true;\n                                     false -> undefined\n                                 end,\n          honor_cipher_order = case yaws_dynopts:have_ssl_honor_cipher_order() of\n                                   true  -> true;\n                                   false -> undefined\n                               end,\n          protocol_version,\n          require_sni = false\n         }).\n\n\n%% flags for sconfs\n-define(SC_ACCESS_LOG,             1).\n-define(SC_AUTH_LOG,               2).\n-define(SC_ADD_PORT,               4).\n-define(SC_STATISTICS,             8).\n-define(SC_TILDE_EXPAND,          16).\n-define(SC_DIR_LISTINGS,          32).\n-define(SC_DEFLATE,               64).\n-define(SC_DIR_ALL_ZIP,          128).\n-define(SC_DAV,                  256).\n-define(SC_FCGI_TRACE_PROTOCOL,  512).\n-define(SC_FCGI_LOG_APP_ERROR,  1024).\n-define(SC_FORWARD_PROXY,       2048).\n-define(SC_AUTH_SKIP_DOCROOT,   4096).\n\n\n\n-define(SC_DEF, ?SC_ACCESS_LOG bor ?SC_ADD_PORT bor ?SC_AUTH_LOG).\n\n-define(sc_has_access_log(SC),\n        (((SC)#sconf.flags band ?SC_ACCESS_LOG) /= 0)).\n-define(sc_has_auth_log(SC),\n        (((SC)#sconf.flags band ?SC_AUTH_LOG) /= 0)).\n-define(sc_has_add_port(SC),\n        (((SC)#sconf.flags band ?SC_ADD_PORT) /= 0)).\n-define(sc_has_statistics(SC),\n        (((SC)#sconf.flags band ?SC_STATISTICS) /= 0)).\n-define(sc_has_tilde_expand(SC),\n        (((SC)#sconf.flags band ?SC_TILDE_EXPAND) /= 0)).\n-define(sc_has_dir_listings(SC),\n        (((SC)#sconf.flags band ?SC_DIR_LISTINGS) /= 0)).\n-define(sc_has_deflate(SC),\n        (((SC)#sconf.flags band ?SC_DEFLATE) /= 0)).\n-define(sc_has_dir_all_zip(SC),\n        (((SC)#sconf.flags band ?SC_DIR_ALL_ZIP) /= 0)).\n-define(sc_has_dav(SC),\n        (((SC)#sconf.flags band ?SC_DAV) /= 0)).\n-define(sc_fcgi_trace_protocol(SC),\n        (((SC)#sconf.flags band ?SC_FCGI_TRACE_PROTOCOL) /= 0)).\n-define(sc_fcgi_log_app_error(SC),\n        (((SC)#sconf.flags band ?SC_FCGI_LOG_APP_ERROR) /= 0)).\n-define(sc_forward_proxy(SC),\n        (((SC)#sconf.flags band ?SC_FORWARD_PROXY) /= 0)).\n-define(sc_auth_skip_docroot(SC),\n        (((SC)#sconf.flags band ?SC_AUTH_SKIP_DOCROOT) /= 0)).\n\n\n-define(sc_set_access_log(SC, Bool),\n        SC#sconf{flags = yaws:flag(SC#sconf.flags, ?SC_ACCESS_LOG, Bool)}).\n-define(sc_set_auth_log(SC, Bool),\n        SC#sconf{flags = yaws:flag(SC#sconf.flags, ?SC_AUTH_LOG, Bool)}).\n-define(sc_set_add_port(SC, Bool),\n        SC#sconf{flags = yaws:flag(SC#sconf.flags, ?SC_ADD_PORT, Bool)}).\n-define(sc_set_statistics(SC, Bool),\n        SC#sconf{flags = yaws:flag(SC#sconf.flags, ?SC_STATISTICS, Bool)}).\n-define(sc_set_tilde_expand(SC, Bool),\n        SC#sconf{flags = yaws:flag(SC#sconf.flags, ?SC_TILDE_EXPAND, Bool)}).\n-define(sc_set_dir_listings(SC, Bool),\n        SC#sconf{flags = yaws:flag(SC#sconf.flags, ?SC_DIR_LISTINGS, Bool)}).\n-define(sc_set_deflate(SC, Bool),\n        SC#sconf{flags = yaws:flag(SC#sconf.flags, ?SC_DEFLATE, Bool)}).\n-define(sc_set_dir_all_zip(SC, Bool),\n        SC#sconf{flags = yaws:flag(SC#sconf.flags, ?SC_DIR_ALL_ZIP, Bool)}).\n-define(sc_set_dav(SC, Bool),\n        SC#sconf{flags = yaws:flag(SC#sconf.flags, ?SC_DAV, Bool)}).\n-define(sc_set_fcgi_trace_protocol(SC, Bool),\n        SC#sconf{flags = yaws:flag(SC#sconf.flags, ?SC_FCGI_TRACE_PROTOCOL,\n                                   Bool)}).\n-define(sc_set_fcgi_log_app_error(SC, Bool),\n        SC#sconf{flags = yaws:flag(SC#sconf.flags, ?SC_FCGI_LOG_APP_ERROR,\n                                   Bool)}).\n-define(sc_set_forward_proxy(SC, Bool),\n        SC#sconf{flags = yaws:flag(SC#sconf.flags, ?SC_FORWARD_PROXY, Bool)}).\n-define(sc_set_auth_skip_docroot(SC, Bool),\n        SC#sconf{flags = yaws:flag(SC#sconf.flags,?SC_AUTH_SKIP_DOCROOT,Bool)}).\n\n\n%% server conf\n%% we cannot compare sconfs directly due to the ets field in #sconf{} use\n%% yaws_config:eq_sconfs/2\n-record(sconf, {\n          port = 8000,                  % which port is this server listening to\n          flags = ?SC_DEF,\n          redirect_map=[],              % a list of\n                                        % {Prefix, #url{}, append|noappend}\n                                        % #url{} can be partially populated\n\n          rhost,                        % forced redirect host (+ optional port)\n          rmethod,                      % forced redirect method\n          docroot,                      % path to the docs\n          xtra_docroots = [],           % if we have additional pseudo docroots\n          listen = [{127,0,0,1}],       % bind to this IP, {0,0,0,0} is possible\n          servername = \"localhost\",     % servername is what Host: header is\n          serveralias = [],             % Alternate names for this vhost\n          yaws,                         % server string for this vhost\n          ets,                          % local store for this server\n          ssl,                          % undefined | #ssl{}\n          authdirs = [],                % [{docroot, [#auth{}]}]\n          partial_post_size = 10240,\n\n          %% An item in the appmods list  can be either of the\n          %% following, this is all due to backwards compat issues.\n          %% 1.  an atom - this is the equivalent to {atom, atom}\n          %% 2 . A two tuple {Path, Mod}\n          %% 3 A three tuple {Path, Mod, [ExcludeDir ....]}\n          appmods = [],\n\n          expires = [],\n          errormod_401 = yaws_outmod,   % the default 401 error module\n          errormod_404 = yaws_outmod,   % the default 404 error module\n          errormod_crash = yaws_outmod, % use the same module for crashes\n          arg_rewrite_mod = yaws,\n          logger_mod = yaws_log,        % access/auth logging module\n          opaque = [],                  % useful in embedded mode\n          start_mod,                    % user provided module to be started\n          allowed_scripts = [yaws,php,cgi,fcgi],\n          tilde_allowed_scripts = [],\n          index_files = [\"index.yaws\", \"index.html\", \"index.php\"],\n          revproxy = [],\n          soptions = [{listen_opts, [{backlog, 1024}]}],\n          extra_cgi_vars = [],\n          stats,                        % raw traffic statistics\n          fcgi_app_server,              % FastCGI application server {host,port}\n          php_handler = {cgi, \"/usr/bin/php-cgi\"},\n          shaper,\n          deflate_options,              % undefined | #deflate{}\n          mime_types_info,              % undefined | #mime_types_info{}\n                                        % if undefined, global config is used\n          dispatch_mod                  % custom dispatch module\n         }).\n\n\n%% Auth conf - from server conf and .yaws_auth\n-record(auth, {\n          dir     = [],\n          docroot = [],\n          files   = [],\n          realm   = \"\",\n          type    = \"Basic\",\n          headers = [],    % headers to send on 401\n          users   = [],    % list of {User, Password} tuples\n          acl     = none,  % list of allowed/denies IPs or none\n          mod     = [],    % authentication module callback\n          outmod  = [],    % module to handles 401 unauthorized messages\n          pam     = false  % should we use pam to auth a user\n         }).\n\n\n%% Macro used to list default compressible mime-types\n-define(DEFAULT_COMPRESSIBLE_MIME_TYPES, [\n                                          {\"text\", all},\n                                          {\"application\", \"rtf\"},\n                                          {\"application\", \"msword\"},\n                                          {\"application\", \"postscript\"},\n                                          {\"application\", \"pdf\"},\n                                          {\"application\", \"x-dvi\"},\n                                          {\"application\", \"javascript\"},\n                                          {\"application\", \"x-javascript\"}\n                                         ]).\n\n%% Internal record used to initialize a zlib stream for compression\n-record(deflate, {\n          min_compress_size = nolimit, % nolimit or non negative integer\n                                       % (in bytes)\n          compression_level = default, % none | default | best_compression |\n                                       % best_speed | 0..9\n          window_size       = -15,     % -15..-9\n          mem_level         = 8,       % 1..9\n          strategy          = default, % default | filtered | huffman_only\n          use_gzip_static   = false,\n\n          %% [{Type, undefined|SubType}] | all\n          mime_types = ?DEFAULT_COMPRESSIBLE_MIME_TYPES\n         }).\n\n\n%% Internal record used to set information about mime-types\n-record(mime_types_info, {\n          mime_types_file, % an absolute filename path\n          types    = [],   % a map between mime-types and extensions\n          charsets = [],   % a map between charsets and extensions\n          default_type = \"text/plain\",\n          default_charset\n         }).\n\n\n%% this internal record is used and returned by the URL path parser\n-record(urltype, {\n          type,          % error | yaws | regular | directory | forbidden |\n                         % appmod\n          finfo,\n          path     = [],\n          fullpath = [], % deep list (WHY?)\n          dir      = [], % relative dir where the path leads to\n                         % flat | unflat need flat for authentication\n          data,          % type-specific\n                         % e.g: Binary | FileDescriptor | DirListing | undefined\n          deflate,       % undefined | Binary | dynamic\n          mime = \"text/html\",\n          getpath,       % as GET'ed by client\n          pathinfo\n         }).\n\n\n\n\n%% this record is constructed as we build up the outgoing headers\n-record(outh, {\n          status,              % int status code\n\n          doclose,             % bool\n          chunked,             % bool\n          exceedmaxuses=false, % bool, true if hit keep-alive max uses\n          encoding=decide,     % decide, identity, deflate\n          contlen,             % integer\n          act_contlen,         % actual content length for dynamic pages\n                               % and the total set of out headers we can have as\n                               % actual strings\n          connection,\n          server,\n          location,\n          cache_control,\n          expires,\n          date,\n          allow,\n          last_modified,\n          etag,\n          set_cookie,\n          content_range,\n          content_length,\n          content_type,\n          content_encoding,\n          transfer_encoding,\n          www_authenticate,\n          vary,\n          other                % misc other headers\n         }).\n\n\n%% forward and reverse proxy config info\n-record(proxy_cfg, {\n          prefix,\n          url,\n          intercept_mod\n         }).\n\n\n%% as read by application:get_env()\n-record(env, {\n          debug,\n          trace,\n          traceoutput,\n          conf,\n          runmod,\n          embedded,\n          id\n         }).\n\n%% Typically used in error printouts as in:\n%% error_logger:format(\"Err ~p at ~p~n\", [Reason, ?stack()])\n-define(stack(), try throw(1) catch _:_ -> erlang:get_stacktrace() end).\n\n\n%%% The following is for emacs, do not remove\n%%% Local Variables:\n%%% comment-column: 36\n%%% End:\n"
  },
  {
    "path": "contrib/yaws/include/yaws_api.hrl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_api.hrl\n%%% Author  : Claes Wikstrom <klacke@hyber.org>\n%%% Purpose :\n%%% Created : 24 Jan 2002 by Claes Wikstrom <klacke@hyber.org>\n%%%----------------------------------------------------------------------\n\n-author('klacke@hyber.org').\n\n-record(arg, {\n          clisock,        % the socket leading to the peer client\n          client_ip_port, % {ClientIp, ClientPort} tuple\n          headers,        % headers\n          req,            % request (possibly rewritten)\n          orig_req,       % original request\n          clidata,        % The client data (as a binary in POST requests)\n          server_path,    % The normalized server path\n                          % (pre-querystring part of URI)\n          querydata,      % For URIs of the form ...?querydata\n                          %  equiv of cgi QUERY_STRING\n          appmoddata,     % (deprecated - use pathinfo instead) the remainder\n                          % of the path leading up to the query\n          docroot,        % Physical base location of data for this request\n          docroot_mount,  % virtual directory e.g /myapp/ that the docroot\n                          %  refers to.\n          fullpath,       % full deep path to yaws file\n          cont,           % Continuation for chunked multipart uploads\n          state,          % State for use by users of the out/1 callback\n          pid,            % pid of the yaws worker process\n          opaque,         % useful to pass static data\n          appmod_prepath, % (deprecated - use prepath instead) path in front\n                          %  of: <appmod><appmoddata>\n          prepath,        % Path prior to 'dynamic' segment of URI.\n                          %  ie http://some.host/<prepath>/<script-point>/d/e\n                          % where <script-point> is an appmod mount point,\n                          % or .yaws,.php,.cgi,.fcgi etc script file.\n          pathinfo        % Set to '/d/e' when calling c.yaws for the request\n                          % http://some.host/a/b/c.yaws/d/e\n                          %  equiv of cgi PATH_INFO\n         }).\n\n\n-record(http_request, {method,\n                       path,\n                       version}).\n\n-record(http_response, {version,\n                        status,\n                        phrase}).\n\n-record(rewrite_response, {status,\n                           headers = [],\n                           content = <<>>}).\n\n-record(headers, {\n          connection,\n          accept,\n          host,\n          if_modified_since,\n          if_match,\n          if_none_match,\n          if_range,\n          if_unmodified_since,\n          range,\n          referer,\n          user_agent,\n          accept_ranges,\n          cookie = [],\n          keep_alive,\n          location,\n          content_length,\n          content_type,\n          content_encoding,\n          authorization,\n          transfer_encoding,\n          x_forwarded_for,\n          other = []   % misc other headers\n         }).\n\n\n\n\n-record(url,\n        {scheme,          % undefined means not set\n         host,            % undefined means not set\n         port,            % undefined means not set\n         path = [],\n         querypart = []}).\n\n\n-record(setcookie, {key,\n                    value,\n                    quoted = false,\n                    domain,\n                    max_age,\n                    expires,\n                    path,\n                    secure = false,\n                    http_only = false,\n                    extensions = []}).\n\n\n-record(cookie, {key,\n                 value,\n                 quoted = false}).\n\n\n-record(redir_self, {\n          host,        % string() - our own host\n          scheme,      % http | https\n          scheme_str,  % \"https://\"  | \"http://\"\n          port,        % integer()  - our own port\n          port_str     % \"\" | \":<int>\" - the optional port part\n                       %                 to append to the url\n         }).\n\n%% Corresponds to the frame sections as in\n%% http://tools.ietf.org/html/rfc6455#section-5.2\n%% plus 'data' and 'ws_state'. Used for incoming frames.\n-record(ws_frame_info, {\n          fin,\n          rsv,\n          opcode,\n          masked,\n          masking_key,\n          length,\n          payload,\n          data,        % The unmasked payload. Makes payload redundant.\n          ws_state     % The ws_state after unframing this frame.\n                       % This is useful for the endpoint to know what type of\n                       % fragment a potentially fragmented message is.\n         }).\n\n%% Used for outgoing frames. No checks are done on the validity of a frame. This\n%% is the application's responsability to send valid frames.\n-record(ws_frame, {\n          fin = true,\n          rsv = 0,\n          opcode,\n          payload = <<>>\n         }).\n\n%%----------------------------------------------------------------------\n%% The state of a WebSocket connection.\n%% This is held by the ws owner process and passed in calls to yaws_api.\n%%----------------------------------------------------------------------\n-type frag_type() :: text\n                   | binary\n                   | none.  % The WebSocket is not expecting continuation\n                            % of any fragmented message.\n-record(ws_state, {\n          vsn :: integer(),                     % WebSocket version number\n          sock,                                 % gen_tcp or gen_ssl socket\n          frag_type :: frag_type()\n         }).\n"
  },
  {
    "path": "contrib/yaws/include/yaws_dav.hrl",
    "content": "-ifndef(_YAWS_DAV).\n-define(_YAWS_DAV, true).\n\n-record(resource,{\n            name,               % normalized name of resource\n            info                % file_info record of mapped file\n        }).\n-record(upload, {\n            fd,\n            tempname,\n            filename\n        }).\n        \n-endif.\n"
  },
  {
    "path": "contrib/yaws/include/yaws_lock.hrl",
    "content": "-ifndef(_YAWS_LOCK).\n-define(_YAWS_LOCK, true).\n\n-define(LOCK_LIFETIME, 900). % lock lifetime in seconds: 15 minutes\n-define(CLEANUP_INTERVAL, 60). % cleanup interval in seconds: 1 minute\n\n-record(lock,{\n            path=undefined,     % resource path\n            id=undefined,       % uid\n            owner=anonymous,    % lock owner if submitted\n            depth=infinity,     % 0|infinity\n            scope=exclusive,    % exclusive|shared\n            type=write,         % write\n            timeout=0,          % ?LOCK_LIFETIME or shorter\n            timestamp=0         % erlang:now()\n        }).\n\n-endif.\n"
  },
  {
    "path": "contrib/yaws/include/yaws_soap.hrl",
    "content": "-record(wsdl, {operations, model, module}).\n-record(port, {service, port, binding, address}).\n-record(operation, {service, port, operation, binding, address, action}).\n"
  },
  {
    "path": "contrib/yaws/src/haxe.erl",
    "content": "%%% Copyright (c) 2005-2006, A2Z Development USA, Inc.  All Rights Reserved.\n%%%\n%%% The contents of this file are subject to the Erlang Public License,\n%%% Version 1.1, (the \"License\"); you may not use this file except in\n%%% compliance with the License. You should have received a copy of the\n%%% Erlang Public License along with this software. If not, it can be\n%%% retrieved via the world wide web at http://www.erlang.org/.\n%%%\n%%% Software distributed under the License is distributed on an \"AS IS\"\n%%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See\n%%% the License for the specific language governing rights and limitations\n%%% under the License.\n%%%\n%%% The Initial Developer of the Original Code is A2Z Development USA, Inc.\n%%% All Rights Reserved.\n\n%% This code was originally created for serializing/deserializing\n%% Erlang terms in JSON format. It was hacked to handle haXe\n%% (http://www.haxe.org) serialization/deserialization.\n%%\n%% Modified by Yariv Sadan (yarivvv@gmail.com)\n\n-module(haxe).\n-export([encode/1, encode/2, decode_string/1, decode_string/2,\n         decode/1, decode/2, decode_next/1]).\n-export([get_left_over_chars/1]).\n-export([is_obj/1, obj_new/0, obj_fetch/2, obj_find/2, obj_is_key/2]).\n-export([test/0]).\n-export([obj_store/3, obj_from_list/1, obj_fold/3]).\n\n-author(\"Jim Larson <jalarson@amazon.com>, Robert Wai-Chi Chu <robchu@amazon.com>\").\n-author(\"Gaspar Chilingarov <nm@web.am>, Gurgen Tumanyan <barbarian@armkb.com>\").\n-author(\"Yariv Sadan <yarivvv@gmail.com>\").\n\n%%% This module translates haXe types into the following Erlang types:\n%%%\n%%%     haXe                         Erlang\n%%%     ----                         ------\n%%%     int, float                   number\n%%%     string (ascii, utf8)         string\n%%%     string (base64)              binary\n%%%     array                        {array, ElementList}\n%%%     true, false, null            atoms 't', 'f', and 'n'\n%%%     Number.NaN                   atom 'nan'\n%%%     Number.NEGATIVE_INFINITY     atom 'neg_infinity'\n%%%     Number.POSITIVE_INFINITY     atom 'infinity'\n%%%     object                       tagged proplist with string (or atom) keys\n%%%                                  (e.g. {struct, [{foo, \"bar\"}, {baz, 4}]} )\n%%%     class object                 NOT SUPPORTED\n%%%     enum                         NOT SUPPORTED\n%%%     reference                    {ref, Idx}, Idx -> int()\n%%%     exception                    {exception, Obj}, Obj is any\n%%%                                  of the haXe types above\n%%%\n%%% Classs objects and Enums are currently not supported\n%%% because simulating such types in Erlang is quite cumbersome, plus\n%%% anonymous objects should be sufficient for most RPC needs.\n%%% (If you strongly believe otherwise, contact me at yarivvv@gmail.com\n%%% and I will consider adding class/enum support in a future version).\n%%%\n%%% References are handled transparently by the encoding/decoding\n%%% functions. The decoding functions automatically expand\n%%% references, and the encoding functions automatically serialize\n%%% a string as a reference if an equal string has already been\n%%% serialized (note: the haXe serializer can reference strings,\n%%% arrays and objects, but the Erlang encoder only references\n%%% strings due to the lack of address comparison in Erlang).\n%%%\n%%% If you wish to avoid expensive string comparisons\n%%% in the encoder, or to have references to non-string objects,\n%%% you can define a reference explicitly as a\n%%% {ref, Idx} tuple. However, this usage is confusing and\n%%% error prone, so it's better to avoid it except in extreme\n%%% cases.\n\n-define(L(Obj), io:format(\"~s:~w: ~w\\n\", [?MODULE, ?LINE, Obj])).\n\nencode({ref, _Idx} = Ref) -> encode_basic(Ref);\n\n\nencode(Obj) -> encode(Obj, true, 2).\n\n%% The second parameter is a list of options.\n%% The following options are available:\n%%    {use_cache, boolean()}      -- default = true\n%%    {haxe_version, Vsn}         -- default = 2, can be 1 or 2\n%%\n%% The haxe_version is a rough indication of what haXe version we\n%% should encode to. This does not really capture the different\n%% encodings used in the different haXe 1.x versions, see the\n%% token_identifier function in this module for details.\n%%\n%% There second parameter can also be a boolean for backwards compatibility,\n%% in which case the latest haxe_version (2) is assumed.\nencode(Obj, Opts) when is_list(Opts) ->\n    UseCache = proplists:get_value(use_cache, Opts, true),\n    Vsn = proplists:get_value(haxe_version, Opts, 2),\n    encode(Obj, UseCache, Vsn);\n% function clauses for backwards compatibility:\nencode(Obj, true=UseCache)  -> encode(Obj, UseCache, 2);\nencode(Obj, false=UseCache) -> encode(Obj, UseCache, 2).\n\n\n%% The second parameter, if a boolean, indicates whether the cache should\n%% be used. By default, it's turned on (this follows the default configuration\n%% in haXe).\nencode(Obj, true, Vsn) ->\n    {Result, _Cache} = encode(Obj, dict:new(), Vsn),\n    Result;\nencode(Obj, false, Vsn) ->\n    {Result, _Cache} = encode(Obj, undefined, Vsn),\n    Result;\nencode(L, Cache, _Vsn=1) when is_list(L) ->\n    case is_string(L) of\n        yes -> encode_string1(L, $s, Cache);\n        unicode -> encode_string1(xmerl_ucs:to_utf8(L), $j, Cache);\n        no -> exit({error, {haxe_encode, {not_string, L}}})\n    end;\nencode(L, Cache, _Vsn=2) when is_list(L) ->\n    case is_string(L) of\n        no -> exit({error, {haxe_encode, {not_string, L}}});\n        _  -> L2 = yaws_api:url_encode(xmerl_ucs:to_utf8(L)),\n              encode_string1(L2, $y, Cache)\n    end;\nencode({array, Props}, Cache, Vsn) -> encode_array(Props, Cache, Vsn);\nencode({struct, Props}, Cache, Vsn) -> encode_object(Props, Cache, Vsn);\nencode({exception, E}, Cache, Vsn) ->\n    {Result, Cache2} = encode(E, Cache, Vsn),\n    {[$x | Result], Cache2};\nencode(Term, Cache, _Vsn) ->\n    case encode_basic(Term) of\n        {error, _Err} = Err ->\n            exit(Err);\n        Result ->\n            {Result, Cache}\n    end.\n\nencode_basic(Term) ->\n    case Term of\n        true -> \"t\";\n        false -> \"f\";\n        null -> \"n\";\n        undefined -> \"n\";\n        {ref, Idx} -> [$r | integer_to_list(Idx)];\n        nan -> \"k\";\n        infinity -> \"p\";\n        neg_infinity -> \"m\";\n        0 -> \"z\";\n        I when is_integer(I) -> [$i | integer_to_list(I)];\n        F when is_float(F) -> [$d | io_lib:format(\"~g\", [F])];\n        B when is_binary(B) -> S = binary_to_list(base64:encode(B)),\n                               [$s, integer_to_list(length(S)), $: | S];\n        _ -> {error, {bad_term, Term}}\n    end.\n\n\n%% Find a string in the list of previously encoded string\n%% end return {true, {ref, Idx}}, if the search succeeded,\n%% where Idx is the value associated with the string.\n%% Otherwise, return 'false'.\nfind_ref(_Str, undefined) ->\n    false;\nfind_ref(Str, CacheDict) ->\n    case dict:find(Str, CacheDict) of\n        error ->\n            false;\n        {ok, Idx} ->\n            {true, Idx}\n    end.\n\n%% Encode an Erlang string to haXe.\nencode_string(Str, Cache, 1=_Vsn) -> encode_string1(Str, $s, Cache);\nencode_string(Str, Cache, 2=_Vsn) -> encode_string1(Str, $y, Cache).\n\nencode_string1(Str, FirstChar, Cache) ->\n    case find_ref(Str, Cache) of\n        false ->\n            {ok, Result} = encode_string2(Str, FirstChar),\n            NewCache =\n                case Cache of\n                    undefined ->\n                        undefined;\n                    _ ->\n                        dict:store(Str, dict:size(Cache), Cache)\n                end,\n            {Result, NewCache};\n        {true, Idx} ->\n            {[$R | integer_to_list(Idx)], Cache}\n    end.\nencode_string2(S, $y) ->\n    {ok, [$y, integer_to_list(length(S)), $: | S]};\nencode_string2(S, FirstChar) ->\n    encode_string2(S, FirstChar, []).\nencode_string2([], FirstChar, Acc) ->\n    Str = lists:reverse(Acc),\n    Len = integer_to_list(length(Str)),\n    {ok, [FirstChar, Len, $: | Str]};\nencode_string2([C | Cs], FirstChar, Acc) ->\n    case C of\n        $\\\\ -> encode_string2(Cs, FirstChar, [$\\\\, $\\\\ | Acc]);\n        $\\n -> encode_string2(Cs, FirstChar, [$n, $\\\\ | Acc]);\n        $\\r -> encode_string2(Cs, FirstChar, [$r, $\\\\ | Acc]);\n        C when C =< 16#FFFF -> encode_string2(Cs, FirstChar, [C | Acc]);\n        _ -> exit({error, {haxe_encode, {bad_char, C}}})\n    end.\n\nencode_object(Props, Cache, Vsn) ->\n    {Result, Cache2} = encode_object_rest(Props, Cache, Vsn),\n    {[$o | Result], Cache2}.\nencode_object_rest(Props, Cache, Vsn) ->\n    {EncodedProps, Cache1} =\n        lists:foldl(\n          fun({Key, Value}, {Acc, Cache2}) ->\n                  {EncodedKey, Cache3} =\n                      case Key of\n                          L when is_list(L) ->\n                              encode_string(L, Cache2, Vsn);\n                          A when is_atom(A) ->\n                              encode_string(atom_to_list(A), Cache2, Vsn);\n                          _ ->\n                              exit({error, {haxe_encode, {bad_key, Key}}})\n                      end,\n                  {EncodedVal, Cache4} = encode(Value, Cache3, Vsn),\n                  case Acc of\n                      [] ->\n                          {[[EncodedKey, EncodedVal]], Cache4};\n                      _  ->\n                          {[[EncodedKey, EncodedVal] | Acc], Cache4}\n                  end\n          end,\n\n          {[], Cache},\n          Props),\n\n    Result = [lists:reverse(EncodedProps), $g],\n    {Result, Cache1}.\n\nencode_array(Props, Cache, Vsn) ->\n    {NullCount, Arr, Cache1} = lists:foldl(\n          fun\n              (Elem, {NullCount, Arr, Cache2}) when Elem == null;\n                                                    Elem == undefined ->\n                  {NullCount+1, Arr, Cache2};\n              (Elem, {0, Arr, Cache2}) ->\n                  {Encoded, Cache3} = encode(Elem, Cache2, Vsn),\n                  {0, [Encoded | Arr], Cache3};\n              (Elem, {NullCount, Arr, Cache2}) ->\n                  {Encoded, Cache3}  = encode(Elem, Cache2, Vsn),\n                  {0, [Encoded, encode_nulls(NullCount) | Arr], Cache3}\n          end,\n          {0, [$a], Cache}, Props),\n    {lists:reverse([$h , encode_nulls(NullCount) | Arr]), Cache1}.\n\nencode_nulls(0) -> [];\nencode_nulls(1) -> [$n];\nencode_nulls(Num) -> [$u | integer_to_list(Num)].\n\n%%% SCANNING\n%%%\n%%% Scanning funs return either:\n%%%    {done, Result, LeftOverChars}\n%%% if a complete token is recognized, or\n%%%    {more, Continuation}\n%%% if more input is needed.\n%%% Result is {ok, Term}, 'eof', or {error, Reason}.\n%%% Here, the Continuation is a simple Erlang string.\n%%%\n%%% Currently, error handling is rather crude - errors are recognized\n%%% by match failures.  EOF is handled only by number scanning, where\n%%% it can delimit a number, and otherwise causes a match failure.\n%%%\n\ntoken_vsn1([]) -> {more, []};\ntoken_vsn1(eof) -> {done, eof, []};\ntoken_vsn1([C | Rest]) ->\n    case token_identifier(1, C) of\n        int -> scan_int(Rest);\n        float -> scan_float(Rest);\n        string -> scan_string(Rest);\n        utf8 -> scan_utf8(Rest);\n        error -> {done, {error, invalid_token}, Rest};\n        Token -> {done, {ok, Token}, Rest}\n    end.\n\ntoken_vsn2([]) -> {more, []};\ntoken_vsn2(eof) -> {done, eof, []};\ntoken_vsn2([C | Rest]) ->\n    case token_identifier(2, C) of\n        int -> scan_int(Rest);\n        float -> scan_float(Rest);\n        string -> scan_string(Rest);\n        url_encoded_string -> scan_url_encoded_string(Rest);\n        base64_bytes -> scan_base64_bytes(Rest);\n        error -> {done, {error, invalid_token}, Rest};\n        Token -> {done, {ok, Token}, Rest}\n    end.\n\ntoken_identifier(Vsn, C) ->\n    case C of\n        $i -> int;\n        $d -> float;\n        $s -> case Vsn of\n                  1 -> string;\n                  2 -> base64_bytes %% redefined to bytes (base64) in haXe 2.0\n              end;\n        $j -> case Vsn of\n                  1 -> utf8;\n                  2 -> enum  %% redefined from utf8 to enum in haXe 1.16\n              end;\n        $n -> null;\n        $t -> true;\n        $f -> false;\n        $z -> 0;\n        $k -> nan;\n        $p -> infinity;\n        $m -> neg_infinity;\n        $a -> array;\n        $h -> array_end;\n        $o -> obj;\n        $c -> class;\n        $g -> obj_end;\n        $r -> ref;\n        $R -> str_ref;\n        $x -> exception;\n        $y -> case Vsn of\n                  1 -> error;\n                  2 -> url_encoded_string %% utf8; added/defined in haXe 1.11\n              end;\n\n        %% this token is only valid as an element\n        %% of an array\n        $u -> null_seq;\n        _ -> error\n    end.\n\nscan_utf8(Chars) ->\n    scan_chars(Chars, utf8).\n\nscan_string(Chars) ->\n    scan_chars(Chars, string).\n\nscan_url_encoded_string(Chars) ->\n    scan_chars(Chars, url_encoded).\n\nscan_base64_bytes(Chars) ->\n    scan_chars(Chars, base64_bytes).\n\nscan_chars(Chars, Type) ->\n    case scan_int(Chars) of\n        {done, {ok, _NumBytes}, []} ->\n            {more, Chars};\n        {done, {ok, _NumBytes}, [C | _Rest]} when C /= $: ->\n            {done, {error, bad_char, C}, Chars};\n        {done, {ok, NumBytes}, [_ | Rest]} when length(Rest) >= NumBytes ->\n            case Type of\n                utf8 ->\n                    NewStr = xmerl_ucs:from_utf8(lists:sublist(Rest, NumBytes)),\n                    scan_chars(NewStr, [], length(NewStr));\n                string ->\n                    scan_chars(Rest, [], NumBytes);\n                url_encoded ->\n                    {S, Rest2} = split_str_after(Rest, NumBytes),\n                    S2 = xmerl_ucs:from_utf8(yaws_api:url_decode_with_encoding(S, latin1)),\n                    {done, {ok, S2}, Rest2};\n                base64_bytes ->\n                    {S, Rest2} = split_str_after(Rest, NumBytes),\n                    {done, {ok, base64:decode(S)}, Rest2}\n            end;\n        {done, {ok, _NumBytes}, _Rest} ->\n            {more, Chars};\n        Other ->\n            Other\n    end.\n\nscan_chars(Rest, A, 0) ->\n    {done, {ok, lists:reverse(A)}, Rest};\nscan_chars([$\\\\] = Chars, _A, 0) ->\n    {done, {error, missing_escape_character}, Chars};\nscan_chars([$\\\\, C | Rest], A, NumLeft) ->\n    scan_chars(Rest, [esc_to_char(C) | A], NumLeft - 2);\nscan_chars([C | Rest], A, NumLeft) ->\n    scan_chars(Rest, [C | A], NumLeft - 1).\n\nesc_to_char(C) ->\n    case C of\n        $n -> $\\n;\n        $r -> $\\r;\n        $\\\\ -> $\\\\\n    end.\n\nsplit_str_after(S, AfterNumBytes) ->\n    {string:substr(S, 1, AfterNumBytes), string:substr(S, AfterNumBytes + 1)}.\n\n\nscan_float(Chars) ->\n    scan_number(Chars, float).\n\nscan_int(Chars) ->\n    scan_number(Chars, int).\n\nscan_number([], _Type) -> {more, []};\nscan_number(eof, _Type) -> {done, {error, incomplete_number}, []};\nscan_number([$-, $- | _Rest] = Input, _Type) ->\n    {done, {error, invalid_number}, Input};\nscan_number([$- | Ds] = Input, Type) ->\n    case scan_number(Ds, Type) of\n        {more, _Cont} -> {more, Input};\n        {done, {ok, N}, CharList} -> {done, {ok, -1 * N}, CharList};\n        {done, Other, Chars} -> {done, Other, Chars}\n    end;\nscan_number([D | Ds] = Input, Type) when D >= $0, D =< $9 ->\n    scan_number(Ds, D - $0, Input, Type).\n\n%% Numbers don't have a terminator, so stop at the first non-digit,\n%% and ask for more if we run out.\n\nscan_number([], _Num, X, _Type) -> {more, X};\nscan_number(eof, Num, _X, _Type) -> {done, {ok, Num}, eof};\nscan_number([$.], _Num, X, float) -> {more, X};\nscan_number([$., D | Ds], Num, X, float) when D >= $0, D =< $9 ->\n    scan_fraction([D | Ds], Num, X);\nscan_number([D | Ds], Num, X, Type) when Num > 0, D >= $0, D =< $9 ->\n    % Note that nonzero numbers can't start with \"0\".\n    scan_number(Ds, 10 * Num + (D - $0), X, Type);\nscan_number([D | Ds], Num, X, float) when D == $E; D == $e ->\n    scan_exponent_begin(Ds, float(Num), X);\nscan_number([D | _] = Ds, Num, _X, _Type) when D < $0; D > $9 ->\n    {done, {ok, Num}, Ds}.\n\nscan_fraction(Ds, I, X) -> scan_fraction(Ds, [], I, X).\nscan_fraction([], _Fs, _I, X) -> {more, X};\nscan_fraction(eof, Fs, I, _X) ->\n    R = I + list_to_float(\"0.\" ++ lists:reverse(Fs)),\n    {done, {ok, R}, eof};\nscan_fraction([D | Ds], Fs, I, X) when D >= $0, D =< $9 ->\n    scan_fraction(Ds, [D | Fs], I, X);\nscan_fraction([D | Ds], Fs, I, X) when D == $E; D == $e ->\n    R = I + list_to_float(\"0.\" ++ lists:reverse(Fs)),\n    scan_exponent_begin(Ds, R, X);\nscan_fraction(Rest, Fs, I, _X) ->\n    R = I + list_to_float(\"0.\" ++ lists:reverse(Fs)),\n    {done, {ok, R}, Rest}.\n\nscan_exponent_begin(Ds, R, X) ->\n    scan_exponent_begin(Ds, [], R, X).\nscan_exponent_begin([], _Es, _R, X) -> {more, X};\nscan_exponent_begin(eof, _Es, _R, X) -> {done, {error, missing_exponent}, X};\nscan_exponent_begin([D | Ds], Es, R, X) when D == $-;\n                                             D == $+;\n                                             D >= $0, D =< $9 ->\n    scan_exponent(Ds, [D | Es], R, X).\n\nscan_exponent([], _Es, _R, X) -> {more, X};\nscan_exponent(eof, Es, R, _X) ->\n    X = R * math:pow(10, list_to_integer(lists:reverse(Es))),\n    {done, {ok, X}, eof};\nscan_exponent([D | Ds], Es, R, X) when D >= $0, D =< $9 ->\n    scan_exponent(Ds, [D | Es], R, X);\nscan_exponent(Rest, Es, R, _X) ->\n    X = R * math:pow(10, list_to_integer(lists:reverse(Es))),\n    {done, {ok, X}, Rest}.\n\n%%% PARSING\n%%%\n%%% The decode function takes a char list as input, but\n%%% interprets the end of the list as only an end to the available\n%%% input, and returns a \"continuation\" requesting more input.\n%%% When additional characters are available, they, and the\n%%% continuation, are fed into decode/2.  You can use the atom 'eof'\n%%% as a character to signal a true end to the input stream, and\n%%% possibly flush out an unfinished number.  The decode_string/1\n%%% function appends 'eof' to its input and calls decode/1.\n%%%\n%%% Parsing and scanning errors are handled only by match failures.\n%%% The external caller must take care to wrap the call in a \"catch\"\n%%% or \"try\" if better error-handling is desired.  Eventually parse\n%%% or scan errors will be returned explicitly with a description,\n%%% and someday with line numbers too.\n%%%\n%%% The parsing code uses a continuation-passing style to allow\n%%% for the parsing to suspend at any point and be resumed when\n%%% more input is available.\n%%% See http://en.wikipedia.org/wiki/Continuation_passing_style\n\n\n%% Return the first haXe value decoded from the input string.\n%% The string must contain at least one complete haXe value.\n%%\n%% The second (optional) parameter can be either:\n%% - a list of options:\n%%     {use_cache, boolean()}   -- default = true\n%%     {haxe_version, Vsn}      -- default = 2, can be 1 or 2\n%% - a continuation, if {more,Continuation} was previously\n%%   returned and you now have more characters\n%% - a boolean specifying whether to use the cache,\n%%   for backwards compatibility\n%%\n%% The haxe_version is a rough indication of what haXe version we\n%% should expect to decode. This does not really capture the different\n%% encodings used in the different haXe 1.x versions, see the\n%% token_identifier function in this module for details.\n%%\n%% There second parameter can also be a boolean for backwards compatibility,\n%% in which case the latest haxe_version (2) is assumed.\n%%\n%% Caching is on by default, but it can be turned\n%% off for performance and lower memory consumption depending\n%% on the content of the messages.\ndecode_string(CharList) ->\n    decode_string(CharList, [{use_cache, true}]).\ndecode_string(CharList, Opts) ->\n    {done, V, _} = decode(CharList ++ [eof], Opts),\n    V.\n\n%% Attempt to decode a haXe value from the input string\n%% and continuation, using an empty initial continuation.\n%% Return {done, Result, Continuation} if a value is recognized,\n%% or {more, Continuation} if more input characters are needed.\n%% The Result can be {ok, Value}, eof, or {error, Reason}.\n%% The Continuation is then fed as an argument to decode/2 when\n%% more input is available.\n%% Use the atom 'eof' instead of a char list to signal\n%% a true end to the input, and may flush a final number.\n%%\n%% Use the get_left_over_chars/1 function to retrieve any non-consumed\n%% characters in the continuation.\n\ndecode(CharList) ->\n    decode_o(CharList, [{use_cache, true}]).\n\n-record(cont,{chars,\n              cache,\n              tokenizer}).\n\ndecode(CharList, Opts) when is_list(Opts) ->\n    decode_o(CharList, Opts);\ndecode(CharList, Cont) when is_record(Cont, cont) -> %% Continuation case\n    #cont{chars = Chars, cache = Cache, tokenizer = Tokenizer} = Cont,\n    decode2(Chars ++ CharList, Cache, Tokenizer);\ndecode(CharList, true) ->  %% function clause for backwards compatibility\n    decode_o(CharList, [{use_cache, true}]);\ndecode(CharList, false) -> %% function clause for backwards compatibility\n    decode_o(CharList, [{use_cache, false}]).\n\ndecode_o(CharList, Opts) ->\n    %% Note: our Cache for decoding is the tuple\n    %% {ObjCache::list(), StringCache::dict()}\n    Cache = case proplists:get_value(use_cache, Opts, true) of\n                true  -> {[], dict:new()};\n                false -> undefined\n            end,\n    Tokenizer = case proplists:get_value(haxe_version, Opts, 2) of\n                    2 -> fun token_vsn2/1;\n                    1 -> fun token_vsn1/1 %% haxe uses \"old_serializer\"\n                end,\n    decode2(CharList, Cache, Tokenizer).\n\n\ndecode2(CharList, Cache, Tokenizer) ->\n    Cont = #cont{chars = CharList,\n                 cache = Cache,\n                 tokenizer = Tokenizer},\n    get_token(Cont, fun first_cont_fun/2).\n\n\ndecode_next(Cont) ->\n    get_token(Cont, fun first_cont_fun/2).\n\nget_left_over_chars(#cont{chars=Cs}) ->\n    Cs.\n\n\nfirst_cont_fun(eof, Cs) -> {done, eof, Cs};\nfirst_cont_fun(T, Cs) ->\n    parse_value(T, Cs,\n                fun(V, C2) ->\n                        {done, {ok, V}, C2}\n                end).\n\n%% Continuation Kt must accept (TokenOrEof, {Chars, Cache, Tokenizer})\n\nget_token(C = #cont{chars=Chars, tokenizer=Tokenizer}, Kt) ->\n    case Tokenizer(Chars) of\n        {done, {ok, T}, Rest} -> Kt(T, C#cont{chars=Rest});\n        {done, eof, Rest} -> Kt(eof, C#cont{chars=Rest});\n        {done, {error, Reason}, Rest} -> {done, {error, Reason},\n                                          C#cont{chars=Rest}};\n        {more, X} -> {more, C#cont{chars=X}}\n    end.\n\n%% Continuation Kv must accept (Value, {Chars, Cache, Tokenizer})\n\nparse_value(Token, C, Kv) ->\n    parse_value(Token, C, Kv, false).\nparse_value(Token, #cont{cache=Cache} = C, Kv, AcceptNullSeq)->\n    case Token of\n        eof -> {done, {error, premature_eof}, C};\n        T when T == null; T == true; T == false; T == nan;\n               T == infinity; T == neg_infinity ->\n            Kv(T, C);\n        obj -> parse_object(C, Kv);\n        class -> parse_class(C, Kv);\n        array -> parse_array(C, Kv);\n        enum -> parse_enum(C, Kv);\n        exception -> parse_exception(C, Kv);\n        ref -> parse_ref(C, Kv);\n        str_ref -> parse_ref(C, Kv, true);\n        null_seq when AcceptNullSeq -> parse_null_seq(C, Kv);\n        Str when is_list(Str) -> Kv(Str, C#cont{cache=put_str(Str, Cache)});\n        Num when is_number(Num) -> Kv(Num, C);\n        B   when is_binary(B) -> Kv(B, C#cont{cache=put_str(B, Cache)});\n        X -> {done, {error, syntax_error,X}, C}\n    end.\n\nparse_class(Chars, _Kv) ->\n    {done, {error, class_objects_not_supported}, Chars}.\n\nparse_object(Chars, Kv) ->\n    parse_object(Chars, Kv, obj_new()).\n\nparse_object(C, Kv, Obj) ->\n    get_token(C,\n              fun(T, C2=#cont{cache=Cache}) ->\n                      case T of\n                          obj_end ->\n                              NewCache =\n                                  put_obj(Obj, Cache),\n                              Kv(Obj, C2#cont{cache=NewCache});\n                          _ ->\n                              NewCache =\n                                  put_obj(placeholder, Cache),\n                              parse_object_field(\n                                Obj, T,\n                                C2#cont{cache=NewCache}, Cache, Kv)\n                      end\n              end).\n\nput_obj(_xObj, undefined) ->\n    undefined;\nput_obj(Obj, {ObjCache, StrCache}) ->\n    {[Obj | ObjCache], StrCache}.\n\nput_str(_Str, undefined) ->\n    undefined;\nput_str(Str, {ObjCache, StrCache}) ->\n    {ObjCache, dict:store(dict:size(StrCache), Str, StrCache)}.\n\nparse_object_field(_Obj, eof, C, _OrigCache, _Kv) ->\n    {done, {error, premature_eof}, C};\n\n%% if the key is a reference, we deference it and continue\nparse_object_field(Obj, RefType, C, OrigCache, Kv)\n  when RefType == ref; RefType == str_ref ->\n    parse_ref(C,\n              fun(Key, C1) ->\n                      parse_object_val(Obj, Key, C1, OrigCache, Kv)\n              end,\n             RefType == str_ref);\n\n%% if the key is a string, we put it in the cache and continue\nparse_object_field(Obj, Field, C=#cont{cache=Cache}, OrigCache, Kv)\n  when is_list(Field) ->\n    NewCache = put_str(Field, Cache),\n    parse_object_val(Obj, Field, C#cont{cache=NewCache}, OrigCache, Kv).\n\nparse_object_val(Obj, Field, C, OrigCache, Kv)\n  when is_list(Field) ->\n    get_token(C,\n              fun(T, C2) ->\n                      parse_value\n                        (T, C2,\n                         fun(Val, C3) ->\n                                 Obj2 = obj_store(Field, Val, Obj),\n                                 parse_object_next(Obj2, C3, OrigCache, Kv)\n                         end)\n              end);\nparse_object_val(_Obj, Field, C, _OrigCache, _Kv) ->\n    {done, {error, {member_name_not_string, Field}}, C}.\n\nparse_object_next({struct, Props} = Obj, C, OrigCache, Kv) ->\n    get_token(C,\n              fun\n                  (obj_end, C1 = #cont{cache=Cache}) ->\n                      Obj1 = {struct, lists:reverse(Props)},\n                      Cache1 = append_new_elems(\n                                    Cache, Obj1, OrigCache),\n                      Kv(Obj1, C1#cont{cache=Cache1});\n                  (eof, C1) ->\n                      {done, {error, premature_eof}, C1};\n                  (T, C1) ->\n                      parse_object_field(Obj, T, C1, OrigCache, Kv)\n              end).\n\n\nappend_new_elems(undefined, _Obj, _Cache) ->\n    undefined;\nappend_new_elems({TempObjCache, NewStrCache}, Obj,\n                 {ObjCache, _OrigStrCache}) ->\n    ObjCache1 = [Obj | ObjCache],\n    NumNewElts = length(TempObjCache) - length(ObjCache1),\n    NewElts = lists:sublist(TempObjCache, NumNewElts),\n    {NewElts ++ ObjCache1, NewStrCache}.\n\n\nparse_array(C = #cont{cache=Cache}, Kv) ->\n\n    %% We need to put a temporary placeholder in the cache\n    %% to comply with haXe's indexing scheme, which assumes\n    %% the array is put in the cache *before* its members.\n    %% When we finish parsing the array, we collect the new\n    %% cache entries and put them in the old cache after\n    %% first inserting the fully parsed array.\n    %%\n    %% If it sounds backwards, well, it is! :)\n\n    parse_array([], C#cont{cache=put_obj(placeholder, Cache)},\n                Cache, Kv).\n\nparse_array(Elems, C, OrigCache, Kv) ->\n    get_token(C,\n              fun\n                  (eof, C1) -> {done, {error, premature_eof}, C1};\n                  (array_end, C1=#cont{cache=Cache1}) ->\n                      Arr = {array, lists:reverse(Elems)},\n                      Cache2 = append_new_elems(\n                                 Cache1, Arr, OrigCache),\n                      Kv(Arr, C1#cont{cache=Cache2});\n                  (T, C1) ->\n                      parse_array_tok(Elems, T, C1, OrigCache, Kv)\n              end).\n\nparse_array_tok(Elems, T, Cont, OrigCache, Kv) ->\n    parse_value(T, Cont,\n                fun({null_seq, Nulls}, C1) ->\n                        parse_array(Nulls ++ Elems, C1, OrigCache, Kv);\n                   (V, C1) ->\n                        parse_array([V | Elems], C1, OrigCache, Kv)\n                end,\n               true).\n\nparse_enum(Chars, _Kv) ->\n    {done, {error, enums_not_supported}, Chars}.\n\n%% it's safe to assume we'll never have to parse exceptions\n%% on the server side, but this function is here for completeness\nparse_exception(Cont, Kv) ->\n    get_token(Cont,\n              fun(T, C1) ->\n                      parse_value(T, C1,\n                                  fun(Val, C2) ->\n                                          Kv({exception, Val}, C2)\n                                  end)\n              end).\n\n\n%%% The next three functions help with storing references\n%%% to deserialized objects for future lookup during decoding\n\nparse_ref(Cont, Kv) ->\n    parse_ref(Cont, Kv, false).\n\nparse_ref(C=#cont{chars=Chars}, Kv, IsStrRef) ->\n    case scan_int(Chars) of\n        {done, {ok, Idx}, Chars1} ->\n            parse_ref2(IsStrRef, C#cont{chars=Chars1}, Idx, Kv);\n        Other ->\n            Other\n    end.\n\nparse_ref2(_IsStrRef, #cont{cache=undefined} = Cont, _Idx, _Kv) ->\n    {done, {error, references_disabled}, Cont};\n\nparse_ref2(false, #cont{cache={ObjCache, _StrCache}} = Cont, Idx, Kv) ->\n    if Idx > length(ObjCache) ->\n            {done, {error, {ref_idx_out_of_bounds, Idx}}, Cont};\n       true ->\n            Val = lists:nth(length(ObjCache) - Idx, ObjCache),\n            if\n                Val == placeholder ->\n                    {done, {error, {illegal_ref, Idx}}, Cont};\n                true ->\n                    Kv(Val, Cont)\n            end\n    end;\n\nparse_ref2(true, #cont{cache={_ObjCache, StrCache}} = Cont, Idx, Kv) ->\n    Cond = Idx + 1 > dict:size(StrCache),\n    if Cond ->\n            {done, {error, {str_ref_idx_out_of_bounds, Idx}}, Cont};\n       true ->\n            {ok, Val} = dict:find(Idx, StrCache),\n            Kv(Val, Cont)\n    end.\n\n\n\nparse_null_seq(Cont = #cont{chars=Chars}, Kv) ->\n    case scan_int(Chars) of\n        {done, {ok, Num}, C1} when Num > 0 ->\n            Cont1 = Cont#cont{chars=C1},\n            Kv({null_seq, lists:duplicate(Num, null)}, Cont1);\n        Other ->\n            Other\n    end.\n\n%%% OBJECTS\n%%%\n%%% We'll use tagged property lists as the internal representation\n%%% of haXe objects.  Unordered lists perform worse than trees for\n%%% lookup and modification of members, but we expect objects to be\n%%% have only a few members.  Lists also print better.\n\nis_obj(_) ->\n    false.\n\n%% create a simple haXe object\nobj_new() ->\n    {struct, []}.\n\n%% Fetch an object member's value, expecting it to be in the object.\n%% Return value, runtime error if no member found with that name.\nobj_fetch(Key, {struct, Props}) ->\n    case proplists:get_value(Key, Props) of\n        undefined ->\n            exit({struct_no_key, Key});\n        Value ->\n            Value\n    end.\n\n%% Fetch an object member's value, or indicate that there is no such member.\n%% Return {ok, Value} or 'error'.\n\nobj_find(Key, {struct, Props}) ->\n    case proplists:get_value(Key, Props) of\n        undefined ->\n            error;\n        Value ->\n            {ok, Value}\n    end.\n\nobj_is_key(Key, {struct, Props}) ->\n    proplists:is_defined(Key, Props).\n\n%% Store a new member in an object.  Returns a new object.\n\nobj_store(KeyStr, Value, {struct, Props}) ->\n    Key = list_to_atom(KeyStr),\n    NewProps = [{Key, Value} | proplists:delete(Key, Props)],\n    {struct, NewProps}.\n\n%% Create an object from a list of Key/Value pairs.\n\nobj_from_list(Props) ->\n    {struct, {Props}}.\n\n%% Fold Fun across object, with initial accumulator Acc.\n%% Fun should take (Value, Acc) as arguments and return Acc.\n\nobj_fold(Fun, Acc, {struct, Props}) ->\n    lists:foldl(Fun, Acc, Props).\n\nis_string([]) -> yes;\nis_string(List) -> is_string(List, non_unicode).\n\nis_string([C|Rest], non_unicode) when C >= 0, C =< 255 ->\n    is_string(Rest, non_unicode);\nis_string([C|Rest], _) when C =< 65000 -> is_string(Rest, unicode);\nis_string([], non_unicode) -> yes;\nis_string([], unicode) -> unicode;\nis_string(_, _) -> no.\n\n\ntest() ->\n    Tests = [\n             {1, \"i1\"},\n             {1.1, \"d1.10000\"},\n             {\"foo\", \"y3:foo\"},\n             %% todo test utf8\n             {null, \"n\"},\n             {true, \"t\"},\n             {false, \"f\"},\n             {0, \"z\"},\n             {nan, \"k\"},\n             {infinity, \"p\"},\n             {neg_infinity, \"m\"},\n             {{array, [1,2,3]}, \"ai1i2i3h\"},\n             {{array, [null]}, \"anh\"},\n             {{array, [null, null]}, \"au2h\"},\n             {{array, [3, 4, null, null, null, 5, null]}, \"ai3i4u3i5nh\"},\n             {{struct, [{foo, \"bar\"}, {baz, \"boing\"}]},\n              \"oy3:fooy3:bary3:bazy5:boingg\"},\n             {{array, [\"foo\", \"bar\", \"foo\",\n                       {struct, [{bar, \"baz\"}, {foo, 123}]}]},\n              \"ay3:fooy3:bary3:foooy3:bary3:bazy3:fooi123gh\"},\n             {{exception, \"bad\"}, \"xy3:bad\"}\n            ],\n\n    {Passed, Failed} = run_tests(Tests, false),\n\n    Tests1 = [\n              {{array, [\"foo\", \"bar\", \"foo\"]}, \"ay3:fooy3:barR0h\"},\n              {{struct, [{foo, \"bar\"}, {bar, \"foo\"}]}, \"oy3:fooy3:barR1R0g\"},\n              {{array, [\"foo\", \"bar\", \"foo\",\n                        {struct, [{bar, \"baz\"}, {foo, 123}]}]},\n               \"ay3:fooy3:barR0oR1y3:bazR0i123gh\"}\n             ],\n    {Passed1, Failed1} = run_tests(Tests1, true),\n\n    Tests2 = [{\"abc\"++[16#e5,16#e4,16#f6],   %% a-ring,a-diaeresis,o-diaeresis\n               \"y21:abc%C3%A5%C3%A4%C3%B6\"}, %% url-encoded utf8\n              {<<\"abc\">>,\n               \"s4:YWJj\"}],\n    {Passed2, Failed2} = run_tests(Tests2, [{haxe_version,2}]),\n\n    %% version1 encode/decode (simple utf8)\n    %% This is mostly based on looking at the code of the haxe\n    %% Serializer.hx and Unserializer.hx (as of cvs-tag v1-10)\n    Tests3 = [{\"abc\\n\"++[16#1e5,16#1e4,16#1f6],\n\t       \"j11:abc\\\\n\"++[16#c7,16#a5,16#c7,16#a4,16#c7,16#b6]},\n              {\"abc\",\n               \"s3:abc\"}],\n    {Passed3, Failed3} = run_tests(Tests3, [{haxe_version,1}]),\n\n    io:format(\"passed: ~w, failed: ~w\\n\",\n              [Passed + Passed1 + Passed2 + Passed3,\n               Failed + Failed1 + Failed2 + Failed3]).\n\nrun_tests(Tests, EnableReferences) ->\n    lists:foldl(\n      fun({Term, Str}, Agg) ->\n              Encoded = lists:flatten(encode(Term,EnableReferences)),\n              {ok, Decoded} = decode_string(Str,EnableReferences),\n              Check = fun(Val1, Val2, {P, F}) ->\n                              case Val1 == Val2 of\n                                  true -> {P + 1, F};\n                                  _ -> {P, F + 1}\n                              end\n                      end,\n              Agg1 = Check(Str, Encoded, Agg),\n              Agg2 = Check(Term, Decoded, Agg1),\n              io:format(\"~s == ~s\\n~w\\n~w == ~w\\n~w\\n\\n\",\n                        [Str, Encoded, Str == Encoded,\n                         Term, Decoded, Term == Decoded]),\n              Agg2\n      end,\n      {0, 0}, Tests).\n"
  },
  {
    "path": "contrib/yaws/src/json.erl",
    "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%% WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED\n%%% WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED\n%%%\n%%% Use module json2.erl instead\n%%%\n%%% This module is deprecated. It uses list_to_atom and so could potentially\n%%% fill the atom table. It also fails to pass its own internal tests due to\n%%% changes made years ago outside the context of Yaws.\n%%%\n%%% Do not report problems with this module, as they will not be fixed. You\n%%% should instead convert your code to use the json2 module.\n%%%\n%%% WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED\n%%% WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%(%% Copyright (c) 2005-2006, A2Z Development USA, Inc.  All Rights Reserved.\n%%%\n%%% The contents of this file are subject to the Erlang Public License,\n%%% Version 1.1, (the \"License\"); you may not use this file except in\n%%% compliance with the License. You should have received a copy of the\n%%% Erlang Public License along with this software. If not, it can be\n%%% retrieved via the world wide web at http://www.erlang.org/.\n%%%\n%%% Software distributed under the License is distributed on an \"AS IS\"\n%%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See\n%%% the License for the specific language governing rights and limitations\n%%% under the License.\n%%%\n%%% The Initial Developer of the Original Code is A2Z Development USA, Inc.\n%%% All Rights Reserved.\n\n-module(json).\n-deprecated(module).\n-export([encode/1, decode_string/1, decode/2]).\n-export([is_obj/1, obj_new/0, obj_fetch/2, obj_find/2, obj_is_key/2]).\n-export([obj_store/3, obj_from_list/1, obj_fold/3]).\n-export([test/0]).\n-author(\"Jim Larson <jalarson@amazon.com>, Robert Wai-Chi Chu <robchu@amazon.com>\").\n-author(\"Gaspar Chilingarov <nm@web.am>, Gurgen Tumanyan <barbarian@armkb.com>\").\n\n%%% JavaScript Object Notation (\"JSON\", http://www.json.org) is a simple\n%%% data syntax meant as a lightweight alternative to other representations,\n%%% such as XML.  JSON is natively supported by JavaScript, but many\n%%% other languages have conversion libraries available.\n%%%\n%%% This module translates JSON types into the following Erlang types:\n%%%\n%%%     JSON                    Erlang\n%%%     ----                    ------\n%%%     number                  number\n%%%     string                  string\n%%%     array                   {array, ElementList}\n%%%     object                  tagged proplist with string (or atom) keys (i.e. {struct, PropList} )\n%%%     true, false, null       atoms 'true', 'false', and 'null'\n%%%\n%%% Character Sets: the external representation, and the internal\n%%% representation of strings, are lists of UTF-16 code units.\n%%% The encoding of supplementary characters, as well as\n%%% transcoding to other schemes, such as UTF-8, can be provided\n%%% by other modules.  (See discussion at\n%%% http://groups.yahoo.com/group/json/message/52)\n%%%\n%%%######################################################################\n%%% UPD by Gaspar: for this moment utf-8 encoding inplemented by default\n%%%                if incoming character list have symbols with codes\n%%%                > 255\n%%%######################################################################\n%%%\n%%% Numbers: Thanks to Erlang's bignums, JSON-encoded integers of any\n%%% size can be parsed.  Conversely, extremely large integers may\n%%% be JSON-encoded.  This may cause problems for interoperability\n%%% with JSON parsers which can't handle arbitrary-sized integers.\n%%% Erlang's floats are of fixed precision and limited range, so\n%%% syntactically valid JSON floating-point numbers could silently\n%%% lose precision or noisily cause an overflow.  However, most\n%%% other JSON libraries are likely to behave in the same way.\n%%%\n%%% Strings: If we represented JSON string data as Erlang binaries,\n%%% we would have to choose a particular unicode format.  Instead,\n%%% we use lists of UTF-16 code units, which applications may then\n%%% change to binaries in their application-preferred manner.\n%%%\n%%% Arrays: Because of the string decision above, and Erlang's\n%%% lack of a distinguished string datatype, JSON arrays map\n%%% to Erlang tuples.  Consider utilities like tuple_fold/3\n%%% to deal with tuples in their native form.\n%%%######################################################################\n%%% UPD by Gaspar: array changed to {array, ArrayElementList}\n%%%                ArrayElementList -> list\n%%%                to provide compatibility to xmlrpc module\n%%%######################################################################\n%%%\n%%% Objects: Though not explicitly stated in the JSON \"spec\",\n%%% JSON's JavaScript heritage mandates that member names must\n%%% be unique within an object.  The object/tuple ambiguity is\n%%% not a problem, since the atom 'struct' is not an\n%%% allowable value.  Object keys may be atoms or strings on\n%%% encoding but are always decoded as strings.\n%%%\n%%%######################################################################\n%%% UPD by Gaspar: struct changed to {array, PropList}\n%%%                object keys always decoded to atoms to\n%%%                provide full compatility with xmlrpc module\n%%%######################################################################\n%%%\n\n%%% ENCODING\n\n%% Encode an erlang number, string, tuple, or object to JSON syntax, as a\n%% possibly deep list of UTF-16 code units, throwing a runtime error in the\n%% case of un-convertible input.\n%% Note: object keys may be either strings or atoms.\n\nencode(true) -> \"true\";\nencode(false) -> \"false\";\nencode(null) -> \"null\";\nencode(undefined) -> \"null\";\nencode(B) when is_binary(B) -> encode_string(B);\nencode(I) when is_integer(I) -> integer_to_list(I);\nencode(F) when is_float(F) -> float_to_list(F);\nencode(L) when is_list(L) ->\n    case is_string(L) of\n        yes -> encode_string(L);\n        unicode -> encode_string(xmerl_ucs:to_utf8(L));\n        no -> encode({array, L})\n    end;\nencode({array, Props}) when is_list(Props) -> encode_array(Props);\nencode({struct, Props} = T) when is_list(Props) -> encode_object(T);\nencode(Bad) -> exit({json_encode, {bad_term, Bad}}).\n\n%% Encode an Erlang string to JSON.\n%% Accumulate strings in reverse.\n\nencode_string(B) when is_binary(B) -> encode_string(binary_to_list(B));\nencode_string(S) -> encode_string(S, [$\"]).\n\nencode_string([], Acc) -> lists:reverse([$\" | Acc]);\nencode_string([C | Cs], Acc) ->\n    case C of\n        $\" -> encode_string(Cs, [$\", $\\\\ | Acc]);\n        % (don't escape solidus on encode)\n        $\\\\ -> encode_string(Cs, [$\\\\, $\\\\ | Acc]);\n        $\\b -> encode_string(Cs, [$b, $\\\\ | Acc]);      % note missing \\\n        $\\f -> encode_string(Cs, [$f, $\\\\ | Acc]);\n        $\\n -> encode_string(Cs, [$n, $\\\\ | Acc]);\n        $\\r -> encode_string(Cs, [$r, $\\\\ | Acc]);\n        $\\t -> encode_string(Cs, [$t, $\\\\ | Acc]);\n        C when C >= 0, C < $\\s ->\n            % Control characters must be unicode-encoded.\n            Hex = lists:flatten(io_lib:format(\"~4.16.0b\", [C])),\n            encode_string(Cs, lists:reverse(Hex) ++ \"u\\\\\" ++ Acc); % \"\n        C when C =< 16#FFFF -> encode_string(Cs, [C | Acc]);\n        _ -> exit({json_encode, {bad_char, C}})\n    end.\n\n%% Encode an Erlang object as a JSON object, allowing string or atom keys.\n%% Note that order is irrelevant in both internal and external object\n%% representations.  Nevertheless, the output will respect the order\n%% of the input.\n\nencode_object({struct, _Props} = Obj) ->\n    M = obj_fold(fun({Key, Value}, Acc) ->\n        S = case Key of\n                B when is_binary(B) -> encode_string(B);\n                L when is_list(L) ->\n                    case is_string(L) of\n                        yes -> encode_string(L);\n                        unicode -> encode_string(xmerl_ucs:to_utf8(L));\n                        no -> exit({json_encode, {bad_key, Key}})\n                    end;\n                A when is_atom(A) -> encode_string(atom_to_list(A));\n                _ -> exit({json_encode, {bad_key, Key}})\n            end,\n        V = encode(Value),\n        case Acc of\n            [] -> [S, $:, V];\n            _ -> [Acc, $,, S, $:, V]\n        end\n    end, [], Obj),\n    [${, M, $}].\n\n%% Encode an Erlang tuple as a JSON array.\n%% Order *is* significant in a JSON array!\n\nencode_array(T) ->\n    M = lists:foldl(fun(E, Acc) ->\n        V = encode(E),\n        case Acc of\n            [] -> V;\n            _ -> [Acc, $,, V]\n        end\n    end, [], T),\n    [$[, M, $]].\n\n%%% SCANNING\n%%%\n%%% Scanning funs return either:\n%%%    {done, Result, LeftOverChars}\n%%% if a complete token is recognized, or\n%%%    {more, Continuation}\n%%% if more input is needed.\n%%% Result is {ok, Term}, 'eof', or {error, Reason}.\n%%% Here, the Continuation is a simple Erlang string.\n%%%\n%%% Currently, error handling is rather crude - errors are recognized\n%%% by match failures.  EOF is handled only by number scanning, where\n%%% it can delimit a number, and otherwise causes a match failure.\n%%%\n%%% Tokens are one of the following\n%%% JSON string -> erlang string\n%%% JSON number -> erlang number\n%%% true, false, null -> erlang atoms\n%%% { } [ ] : , -> lcbrace rcbrace lsbrace rsbrace colon comma\n\ntoken([]) -> {more, []};\ntoken(eof) -> {done, eof, []};\n\ntoken(\"true\" ++ Rest) -> {done, {ok, true}, Rest};\ntoken(\"tru\")    -> {more, \"tru\"};\ntoken(\"tr\")     -> {more, \"tr\"};\ntoken(\"t\")      -> {more, \"t\"};\n\ntoken(\"false\" ++ Rest) -> {done, {ok, false}, Rest};\ntoken(\"fals\")   -> {more, \"fals\"};\ntoken(\"fal\")    -> {more, \"fal\"};\ntoken(\"fa\")     -> {more, \"fa\"};\ntoken(\"f\")      -> {more, \"f\"};\n\ntoken(\"null\" ++ Rest) -> {done, {ok, null}, Rest};\ntoken(\"nul\")    -> {more, \"nul\"};\ntoken(\"nu\")     -> {more, \"nu\"};\ntoken(\"n\")      -> {more, \"n\"};\n\ntoken([C | Cs] = Input) ->\n    case C of\n        $\\s -> token(Cs);       % eat whitespace\n        $\\t -> token(Cs);       % eat whitespace\n        $\\n -> token(Cs);       % eat whitespace\n        $\\r -> token(Cs);       % eat whitespace\n        $\" -> scan_string(Input);\n        $- -> scan_number(Input);\n        D when D >= $0, D =< $9-> scan_number(Input);\n        ${ -> {done, {ok, lcbrace}, Cs};\n        $} -> {done, {ok, rcbrace}, Cs};\n        $[ -> {done, {ok, lsbrace}, Cs};\n        $] -> {done, {ok, rsbrace}, Cs};\n        $: -> {done, {ok, colon}, Cs};\n        $, -> {done, {ok, comma}, Cs};\n        $/ -> case scan_comment(Cs) of\n            {more, X} -> {more, X};\n            {done, _, Chars} -> token(Chars)\n        end;\n        _ -> {done, {error, {bad_char, C}}, Cs}\n    end.\n\nscan_string([$\" | Cs] = Input) ->\n    scan_string(Cs, [], Input).\n\n%% Accumulate in reverse order, save original start-of-string for continuation.\n\nscan_string([], _, X) -> {more, X};\nscan_string(eof, _, X) -> {done, {error, missing_close_quote}, X};\nscan_string([$\" | Rest], A, _) -> {done, {ok, lists:reverse(A)}, Rest};\nscan_string([$\\\\], _, X) -> {more, X};\nscan_string([$\\\\, $u, U1, U2, U3, U4 | Rest], A, X) ->\n    scan_string(Rest, [uni_char([U1, U2, U3, U4]) | A], X);\nscan_string([$\\\\, $u | _], _, X) -> {more, X};\nscan_string([$\\\\, C | Rest], A, X) ->\n    scan_string(Rest, [esc_to_char(C) | A], X);\nscan_string([C | Rest], A, X) ->\n    scan_string(Rest, [C | A], X).\n\n%% Given a list of hex characters, convert to the corresponding integer.\n\nuni_char(HexList) ->\n    erlang:list_to_integer(HexList, 16).\n\nesc_to_char($\") -> $\";\nesc_to_char($/) -> $/;\nesc_to_char($\\\\) -> $\\\\;\nesc_to_char($b) -> $\\b;\nesc_to_char($f) -> $\\f;\nesc_to_char($n) -> $\\n;\nesc_to_char($r) -> $\\r;\nesc_to_char($t) -> $\\t.\n\nscan_number([]) -> {more, []};\nscan_number(eof) -> {done, {error, incomplete_number}, []};\nscan_number([$-, $- | _Ds]) -> {done, {error, invalid_number}, []};\nscan_number([$- | Ds] = Input) ->\n    case scan_number(Ds) of\n        {more, _Cont} -> {more, Input};\n        {done, {ok, N}, CharList} -> {done, {ok, -1 * N}, CharList};\n        {done, Other, Chars} -> {done, Other, Chars}\n    end;\nscan_number([D | Ds] = Input) when D >= $0, D =< $9 ->\n    scan_number(Ds, D - $0, Input).\n\n%% Numbers don't have a terminator, so stop at the first non-digit,\n%% and ask for more if we run out.\n\nscan_number([], _A, X) -> {more, X};\nscan_number(eof, A, _X) -> {done, {ok, A}, eof};\nscan_number([$.], _A, X) -> {more, X};\nscan_number([$., D | Ds], A, X) when D >= $0, D =< $9 ->\n    scan_fraction([D | Ds], A, X);\nscan_number([D | Ds], A, X) when A > 0, D >= $0, D =< $9 ->\n    % Note that nonzero numbers can't start with \"0\".\n    scan_number(Ds, 10 * A + (D - $0), X);\nscan_number([D | Ds], A, X) when D == $E; D == $e ->\n    scan_exponent_begin(Ds, integer_to_list(A) ++ \".0\", X);\nscan_number([D | _] = Ds, A, _X) when D < $0; D > $9 ->\n    {done, {ok, A}, Ds}.\n\nscan_fraction(Ds, I, X) -> scan_fraction(Ds, [], I, X).\n\nscan_fraction([], _Fs, _I, X) -> {more, X};\nscan_fraction(eof, Fs, I, _X) ->\n    R = list_to_float(lists:append([integer_to_list(I), \".\",\n                                    lists:reverse(Fs)])),\n    {done, {ok, R}, eof};\nscan_fraction([D | Ds], Fs, I, X) when D >= $0, D =< $9 ->\n    scan_fraction(Ds, [D | Fs], I, X);\nscan_fraction([D | Ds], Fs, I, X) when D == $E; D == $e ->\n    R = lists:append([integer_to_list(I), \".\", lists:reverse(Fs)]),\n    scan_exponent_begin(Ds, R, X);\nscan_fraction(Rest, Fs, I, _X) ->\n    R = list_to_float(lists:append([integer_to_list(I), \".\",\n                                    lists:reverse(Fs)])),\n    {done, {ok, R}, Rest}.\n\nscan_exponent_begin(Ds, R, X) ->\n    scan_exponent_begin(Ds, [], R, X).\n\nscan_exponent_begin([], _Es, _R, X) -> {more, X};\nscan_exponent_begin(eof, _Es, _R, X) -> {done, {error, missing_exponent}, X};\nscan_exponent_begin([D | Ds], Es, R, X) when D == $-;\n                                             D == $+;\n                                             D >= $0, D =< $9 ->\n    scan_exponent(Ds, [D | Es], R, X).\n\nscan_exponent([], _Es, _R, X) -> {more, X};\nscan_exponent(eof, Es, R, _X) ->\n    X = list_to_float(lists:append([R, \"e\", lists:reverse(Es)])),\n    {done, {ok, X}, eof};\nscan_exponent([D | Ds], Es, R, X) when D >= $0, D =< $9 ->\n    scan_exponent(Ds, [D | Es], R, X);\nscan_exponent(Rest, Es, R, _X) ->\n    X = list_to_float(lists:append([R, \"e\", lists:reverse(Es)])),\n    {done, {ok, X}, Rest}.\n\nscan_comment([]) -> {more, \"/\"};\nscan_comment(eof) -> {done, eof, []};\nscan_comment([$/ | Rest]) -> scan_cpp_comment(Rest);\nscan_comment([$* | Rest]) -> scan_c_comment(Rest).\n\n%% Ignore up to next CR or LF.  If the line ends in CRLF,\n%% the LF will be treated as separate whitespace, which is\n%% okay since it will also be ignored.\n\nscan_cpp_comment([]) -> {more, \"//\"};\nscan_cpp_comment(eof) -> {done, eof, []};\nscan_cpp_comment([$\\r | Rest]) -> {done, [], Rest};\nscan_cpp_comment([$\\n | Rest]) -> {done, [], Rest};\nscan_cpp_comment([_ | Rest]) -> scan_cpp_comment(Rest).\n\nscan_c_comment([]) -> {more, \"/*\"};\nscan_c_comment(eof) -> {done, eof, []};\nscan_c_comment([$*]) -> {more, \"/**\"};\nscan_c_comment([$*, $/ | Rest]) -> {done, [], Rest};\nscan_c_comment([_ | Rest]) -> scan_c_comment(Rest).\n\n%%% PARSING\n%%%\n%%% The decode function takes a char list as input, but\n%%% interprets the end of the list as only an end to the available\n%%% input, and returns a \"continuation\" requesting more input.\n%%% When additional characters are available, they, and the\n%%% continuation, are fed into decode/2.  You can use the atom 'eof'\n%%% as a character to signal a true end to the input stream, and\n%%% possibly flush out an unfinished number.  The decode_string/1\n%%% function appends 'eof' to its input and calls decode/1.\n%%%\n%%% Parsing and scanning errors are handled only by match failures.\n%%% The external caller must take care to wrap the call in a \"catch\"\n%%% or \"try\" if better error-handling is desired.  Eventually parse\n%%% or scan errors will be returned explicitly with a description,\n%%% and someday with line numbers too.\n%%%\n%%% The parsing code uses a continuation-passing style to allow\n%%% for the parsing to suspend at any point and be resumed when\n%%% more input is available.\n%%% See http://en.wikipedia.org/wiki/Continuation_passing_style\n\n%% Return the first JSON value decoded from the input string.\n%% The string must contain at least one complete JSON value.\n\ndecode_string(CharList) ->\n    {done, V, _} = decode([], CharList ++ eof),\n    V.\n\n%% Attempt to decode a JSON value from the input string\n%% and continuation, using empty list for the initial continuation.\n%% Return {done, Result, LeftoverChars} if a value is recognized,\n%% or {more, Continuation} if more input characters are needed.\n%% The Result can be {ok, Value}, eof, or {error, Reason}.\n%% The Continuation is then fed as an argument to decode/2 when\n%% more input is available.\n%% Use the atom 'eof' instead of a char list to signal\n%% a true end to the input, and may flush a final number.\n\ndecode([], CharList) ->\n    decode(first_continuation(), CharList);\n\ndecode(Continuation, CharList) ->\n    {OldChars, Kt} = Continuation,\n    get_token(OldChars ++ CharList, Kt).\n\nfirst_continuation() ->\n    {[], fun\n        (eof, Cs) ->\n                {done, eof, Cs};\n        (T, Cs) ->\n            parse_value(T, Cs, fun(V, C2) ->\n                {done, {ok, V}, C2}\n            end)\n    end}.\n\n%% Continuation Kt must accept (TokenOrEof, Chars)\n\nget_token(Chars, Kt) ->\n    case token(Chars) of\n        {done, {ok, T}, Rest} -> Kt(T, Rest);\n        {done, eof, Rest} -> Kt(eof, Rest);\n        {done, {error, Reason}, Rest} -> {done, {error, Reason}, Rest};\n        {more, X} -> {more, {X, Kt}}\n    end.\n\n%% Continuation Kv must accept (Value, Chars)\n\nparse_value(eof, C, _Kv) -> {done, {error, premature_eof}, C};\nparse_value(true, C, Kv) -> Kv(true, C);\nparse_value(false, C, Kv) -> Kv(false, C);\nparse_value(null, C, Kv) -> Kv(null, C);\nparse_value(S, C, Kv) when is_list(S) -> Kv(S, C);\nparse_value(N, C, Kv) when is_number(N) -> Kv(N, C);\nparse_value(lcbrace, C, Kv) -> parse_object(C, Kv);\nparse_value(lsbrace, C, Kv) -> parse_array(C, Kv);\nparse_value(_, C, _Kv) -> {done, {error, syntax_error}, C}.\n\n%% Continuation Kv must accept (Value, Chars)\n\nparse_object(Chars, Kv) ->\n    get_token(Chars, fun(T, C2) ->\n        Obj = obj_new(),\n        case T of\n            rcbrace -> Kv(Obj, C2);             % empty object\n            _ -> parse_object(Obj, T, C2, Kv)   % token must be string\n        end\n    end).\n\nparse_object(_Obj, eof, C, _Kv) ->\n    {done, {error, premature_eof}, C};\n\nparse_object(Obj, S, C, Kv) when is_list(S) ->    % S is member name\n    get_token(C, fun\n        (colon, C2) ->\n            parse_object2(Obj, S, C2, Kv);\n        (T, C2) ->\n            {done, {error, {expecting_colon, T}}, C2}\n    end);\n\nparse_object(_Obj, M, C, _Kv) ->\n    {done, {error, {member_name_not_string, M}}, C}.\n\nparse_object2(Obj, S, C, Kv) ->\n    get_token(C, fun\n        (eof, C2) ->\n            {done, {error, premature_eof}, C2};\n        (T, C2) ->\n            parse_value(T, C2, fun(V, C3) ->    % V is member value\n                Obj2 = obj_store(S, V, Obj),\n                get_token(C3, fun\n                    (rcbrace, C4) ->    % \"}\" end of object\n                                                {struct, PropList1} = Obj2,\n                        Kv({struct, lists:reverse(PropList1)}, C4);\n                    (comma, C4) ->              % \",\" another member follows\n                        get_token(C4, fun(T3, C5) ->\n                            parse_object(Obj2, T3, C5, Kv)\n                        end);\n                    (eof, C4) ->\n                        {done, {error, premature_eof}, C4};\n                    (T2, C4) ->\n                        {done, {error, {expecting_comma_or_curly, T2}}, C4}\n                end)\n            end)\n    end).\n\n%% Continuation Kv must accept (Value, Chars)\n\nparse_array(C, Kv) ->\n    get_token(C, fun\n        (eof, C2) -> {done, {error, premature_eof}, C2};\n        (rsbrace, C2) -> Kv({array, []}, C2);  % empty array\n        (T, C2) -> parse_array([], T, C2, Kv)\n    end).\n\nparse_array(E, T, C, Kv) ->\n    parse_value(T, C, fun(V, C2) ->\n        E2 = [V | E],\n        get_token(C2, fun\n            (rsbrace, C3) ->        % \"]\" end of array\n                Kv({array, lists:reverse(E2)}, C3);\n\n            (comma, C3) ->          % \",\" another value follows\n                get_token(C3, fun(T3, C4) ->\n                    parse_array(E2, T3, C4, Kv)\n                end);\n            (eof, C3) ->\n                {done, {error, premature_eof}, C3};\n            (T2, C3) ->\n                {done, {error, {expecting_comma_or_close_array, T2}}, C3}\n        end)\n    end).\n\n%%% OBJECTS\n%%%\n%%% We'll use tagged property lists as the internal representation\n%%% of JSON objects.  Unordered lists perform worse than trees for\n%%% lookup and modification of members, but we expect objects to be\n%%% have only a few members.  Lists also print better.\n\n%% Is this a proper JSON object representation?\n\nis_obj({struct, Props}) when is_list(Props) ->\n    lists:all(fun\n        ({Member, _Value}) when is_atom(Member); is_list(Member) -> true;\n        (_) -> false\n    end, Props);\n\nis_obj(_) ->\n    false.\n\n%% Create a new, empty object.\n\nobj_new() ->\n    {struct, []}.\n\n%% Fetch an object member's value, expecting it to be in the object.\n%% Return value, runtime error if no member found with that name.\n\nobj_fetch(Key, {struct, Props}) when is_list(Props) ->\n    case proplists:get_value(Key, Props) of\n        undefined ->\n            exit({struct_no_key, Key});\n        Value ->\n            Value\n    end.\n\n%% Fetch an object member's value, or indicate that there is no such member.\n%% Return {ok, Value} or 'error'.\n\nobj_find(Key, {struct, Props}) when is_list(Props) ->\n    case proplists:get_value(Key, Props) of\n        undefined ->\n            error;\n        Value ->\n            {ok, Value}\n    end.\n\nobj_is_key(Key, {struct, Props}) ->\n    proplists:is_defined(Key, Props).\n\n%% Store a new member in an object.  Returns a new object.\n\nobj_store(KeyList, Value, {struct, Props}) when is_list(Props) ->\n        Key = try list_to_atom(KeyList)\n              catch error:badarg -> KeyList\n              end,\n    {struct, [{Key, Value} | proplists:delete(Key, Props)]}.\n\n%% Create an object from a list of Key/Value pairs.\n\nobj_from_list(Props) ->\n    Obj = {struct, Props},\n    case is_obj(Obj) of\n        true -> Obj;\n        false -> exit(json_bad_object)\n    end.\n\n%% Fold Fun across object, with initial accumulator Acc.\n%% Fun should take (Value, Acc) as arguments and return Acc.\n\nobj_fold(Fun, Acc, {struct, Props}) ->\n    lists:foldl(Fun, Acc, Props).\n\nis_string([]) -> yes;\nis_string(List) -> is_string(List, non_unicode).\n\nis_string([C|Rest], non_unicode) when is_integer(C), C >= 0, C =< 255 ->\n    is_string(Rest, non_unicode);\nis_string([C|Rest], _) when is_integer(C), C>= 0, C =< 65000 ->\n    is_string(Rest, unicode);\nis_string([], non_unicode) -> yes;\nis_string([], unicode) -> unicode;\nis_string(_, _) -> no.\n\n\n%%% TESTING\n%%%\n%%% We can't expect to round-trip from JSON -> Erlang -> JSON,\n%%% due to the degrees of freedom in the JSON syntax: whitespace,\n%%% and ordering of object members.  We can, however, expect to\n%%% round-trip from Erlang -> JSON -> Erlang, so the JSON parsing\n%%% tests will in fact test the Erlang equivalence of the\n%%% JSON -> Erlang -> JSON -> Erlang coding chain.\n\n%% Test driver.  Return 'ok' or {failed, Failures}.\n\ntest() ->\n    E2Js = e2j_test_vec(),\n    Failures = lists:foldl(fun({E, J}, Fs) ->\n        case (catch test_e2j(E, J)) of\n            ok ->\n                case (catch round_trip(E)) of\n                    ok ->\n                        case (catch round_trip_one_char(E)) of\n                            ok -> Fs;\n                            Reason -> [{round_trip_one_char, E, Reason} | Fs]\n                        end;\n                    Reason ->\n                        [{round_trip, E, Reason} | Fs]\n                end;\n            Reason ->\n                [{erlang_to_json, E, J, Reason} | Fs]\n        end;\n    (end_of_tests, Fs) -> Fs end, [], E2Js),\n    case Failures of\n        [] -> ok;\n        _ -> {failed, Failures}\n    end.\n\n%% Test for conversion from Erlang to JSON.  Note that unequal strings\n%% may represent equal JSON data, due to discretionary whitespace,\n%% object member order, trailing zeroes in floating point, etc.\n%% Legitimate changes to the encoding routines may require tweaks to\n%% the reference JSON strings in e2j_test_vec().\n\ntest_e2j(E, J) ->\n    J2 = lists:flatten(encode(E)),\n    J = J2,                                     % raises error if unequal\n    ok.\n\n%% Test that Erlang -> JSON -> Erlang round-trip yields equivalent term.\n\nround_trip(E) ->\n    J2 = lists:flatten(encode(E)),\n    {ok, E2} = decode_string(J2),\n    true = equiv(E, E2),                        % raises error if false\n    ok.\n\n%% Round-trip with one character at a time to test all continuations.\n\nround_trip_one_char(E) ->\n    J = lists:flatten(encode(E)),\n    {done, {ok, E2}, _} = lists:foldl(fun(C, Ret) ->\n        case Ret of\n            {done, _, _} -> Ret;\n            {more, Cont} -> decode(Cont, [C])\n        end\n    end, {more, first_continuation()}, J ++ [eof]),\n    true = equiv(E, E2),                        % raises error if false\n    ok.\n\n%% Test for equivalence of Erlang terms.\n%% Due to arbitrary order of construction, equivalent objects might\n%% compare unequal as erlang terms, so we need to carefully recurse\n%% through aggregates (tuples and objects).\n\nequiv({struct, Props1}, {struct, Props2}) ->\n    equiv_object(Props1, Props2);\nequiv(T1, T2) when is_tuple(T1), is_tuple(T2) ->\n    equiv_tuple(T1, T2);\nequiv(N1, N2) when is_number(N1), is_number(N2) -> N1 == N2;\nequiv(S1, S2) when is_list(S1), is_list(S2)     -> S1 == S2;\nequiv(true, true) -> true;\nequiv(false, false) -> true;\nequiv(null, null) -> true.\n\n%% Object representation and traversal order is unknown.\n%% Use the sledgehammer and sort property lists.\n\nequiv_object(Props1, Props2) ->\n    L1 = lists:keysort(1, Props1),\n    L2 = lists:keysort(1, Props2),\n    Pairs = lists:zip(L1, L2),\n    true = lists:all(fun({{K1, V1}, {K2, V2}}) ->\n        equiv(K1, K2) and equiv(V1, V2)\n    end, Pairs).\n\n%% Recursively compare tuple elements for equivalence.\n\nequiv_tuple({}, {}) ->\n    true;\nequiv_tuple(T1, T2) when size(T1) == size(T2) ->\n    S = size(T1),\n    lists:all(fun(I) ->\n        equiv(element(I, T1), element(I, T2))\n    end, lists:seq(1, S)).\n\ne2j_test_vec() -> [\n    {1, \"1\"},\n    {3.1416, \"3.14160\"}, % text representation may truncate, trail zeroes\n    {-1, \"-1\"},\n    {-3.1416, \"-3.14160\"},\n    {12.0e10, \"1.20000e+11\"},\n    {1.234E+10, \"1.23400e+10\"},\n    {-1.234E-10, \"-1.23400e-10\"},\n    {\"foo\", \"\\\"foo\\\"\"},\n    {\"foo\" ++ [500] ++ \"bar\", [$\", $f, $o, $o, 500, $b, $a, $r, $\"]},\n    {\"foo\" ++ [5] ++ \"bar\", \"\\\"foo\\\\u0005bar\\\"\"},\n    {\"\", \"\\\"\\\"\"},\n    {[], \"\\\"\\\"\"},\n    {\"\\n\\n\\n\", \"\\\"\\\\n\\\\n\\\\n\\\"\"},\n    {obj_new(), \"{}\"},\n    {obj_from_list([{\"foo\", \"bar\"}]), \"{\\\"foo\\\":\\\"bar\\\"}\"},\n    {obj_from_list([{\"foo\", \"bar\"}, {\"baz\", 123}]),\n     \"{\\\"foo\\\":\\\"bar\\\",\\\"baz\\\":123}\"},\n    {{}, \"[]\"},\n    {{{}}, \"[[]]\"},\n    {{1, \"foo\"}, \"[1,\\\"foo\\\"]\"},\n\n    % json array in a json object\n    {obj_from_list([{\"foo\", {123}}]),\n     \"{\\\"foo\\\":[123]}\"},\n\n    % json object in a json object\n    {obj_from_list([{\"foo\", obj_from_list([{\"bar\", true}])}]),\n     \"{\\\"foo\\\":{\\\"bar\\\":true}}\"},\n\n    % fold evaluation order\n    {obj_from_list([{\"foo\", {}},\n                     {\"bar\", obj_from_list([{\"baz\", true}])},\n                     {\"alice\", \"bob\"}]),\n     \"{\\\"foo\\\":[],\\\"bar\\\":{\\\"baz\\\":true},\\\"alice\\\":\\\"bob\\\"}\"},\n\n    % json object in a json array\n    {{-123, \"foo\", obj_from_list([{\"bar\", {}}]), null},\n     \"[-123,\\\"foo\\\",{\\\"bar\\\":[]},null]\"},\n\n    end_of_tests\n].\n\n%%% TODO:\n%%%\n%%% Measure the overhead of the CPS-based parser by writing a conventional\n%%% scanner-parser that expects all input to be available.\n%%%\n%%% JSON has dropped comments - disable their parsing.\n%%%\n%%% Allow a compile-time option to decode object member names as atoms,\n%%% to reduce the internal representation overheads when communicating\n%%% with trusted peers.\n"
  },
  {
    "path": "contrib/yaws/src/json2.erl",
    "content": "%%% Copyright (c) 2005-2006, A2Z Development USA, Inc.  All Rights Reserved.\n%%%\n%%% The contents of this file are subject to the Erlang Public License,\n%%% Version 1.1, (the \"License\"); you may not use this file except in\n%%% compliance with the License. You should have received a copy of the\n%%% Erlang Public License along with this software. If not, it can be\n%%% retrieved via the world wide web at http://www.erlang.org/.\n%%%\n%%% Software distributed under the License is distributed on an \"AS IS\"\n%%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See\n%%% the License for the specific language governing rights and limitations\n%%% under the License.\n%%%\n%%% The Initial Developer of the Original Code is A2Z Development USA, Inc.\n%%% All Rights Reserved.\n\n-module(json2).\n-export([encode/1, decode_string/1, decode/2]).\n-export([is_obj/1, obj_new/0, obj_fetch/2, obj_find/2, obj_is_key/2]).\n-export([obj_store/3, obj_from_list/1, obj_fold/3]).\n-export([test/0]).\n-author(\"Jim Larson <jalarson@amazon.com>, Robert Wai-Chi Chu <robchu@amazon.com>\").\n-author(\"Gaspar Chilingarov <nm@web.am>, Gurgen Tumanyan <barbarian@armkb.com>\").\n-author(\"Steve Vinoski <vinoski@ieee.org>\").\n\n%%% JavaScript Object Notation (\"JSON\", http://www.json.org) is a simple\n%%% data syntax meant as a lightweight alternative to other representations,\n%%% such as XML.  JSON is natively supported by JavaScript, but many\n%%% other languages have conversion libraries available.\n%%%\n%%% This module translates JSON types into the following Erlang types:\n%%%\n%%%     JSON                    Erlang\n%%%     ----                    ------\n%%%     number                  number\n%%%     string                  string\n%%%     array                   {array, ElementList}\n%%%     object                  tagged proplist with string keys (i.e. {struct, PropList} )\n%%%     true, false, null       atoms 'true', 'false', and 'null'\n%%%\n%%% Character Sets: the external representation, and the internal\n%%% representation of strings, are lists of UTF-8 code units.\n%%%\n%%% Numbers: Thanks to Erlang's bignums, JSON-encoded integers of any\n%%% size can be parsed.  Conversely, extremely large integers may\n%%% be JSON-encoded.  This may cause problems for interoperability\n%%% with JSON parsers which can't handle arbitrary-sized integers.\n%%% Erlang's floats are of fixed precision and limited range, so\n%%% syntactically valid JSON floating-point numbers could silently\n%%% lose precision or noisily cause an overflow.  However, most\n%%% other JSON libraries are likely to behave in the same way.\n%%%\n%%% Strings: If we represented JSON string data as Erlang binaries,\n%%% we would have to choose a particular unicode format.  Instead,\n%%% we use lists of UTF-16 code units, which applications may then\n%%% change to binaries in their application-preferred manner.\n%%%\n%%% Arrays: Because of the string decision above, and Erlang's\n%%% lack of a distinguished string datatype, JSON arrays map\n%%% to {array, ArrayElementList}, where ArrayElementList -> list.\n%%%\n%%% Objects: Though not explicitly stated in the JSON \"spec\",\n%%% JSON's JavaScript heritage mandates that member names must\n%%% be unique within an object.  The object/tuple ambiguity is\n%%% not a problem, since the atom 'struct' is not an\n%%% allowable value.  Object keys may be atoms or strings on\n%%% encoding but are always decoded as strings.\n%%%\n\n%%% ENCODING\n\n%% Encode an erlang number, string, tuple, or object to JSON syntax, as a\n%% possibly deep list of UTF-8 code units, throwing a runtime error in the\n%% case of un-convertible input.\n%% Note: object keys may be either strings or atoms.\n\nencode(true) -> \"true\";\nencode(false) -> \"false\";\nencode(null) -> \"null\";\nencode(undefined) -> \"null\";\nencode(B) when is_binary(B) -> encode_string(B);\nencode(I) when is_integer(I) -> integer_to_list(I);\nencode(F) when is_float(F) -> float_to_list(F);\nencode(L) when is_list(L) ->\n    case is_string(L) of\n        yes -> encode_string(L);\n        unicode -> encode_string(xmerl_ucs:to_utf8(L));\n        no -> encode({array, L})\n    end;\nencode({array, Props}) when is_list(Props) -> encode_array(Props);\nencode({struct, Props} = T) when is_list(Props) -> encode_object(T);\nencode(Bad) -> exit({json_encode, {bad_term, Bad}}).\n\n%% Encode an Erlang string to JSON.\n%% Accumulate strings in reverse.\n\nencode_string(B) when is_binary(B) -> encode_string(binary_to_list(B));\nencode_string(S) -> encode_string(S, [$\"]).\n\nencode_string([], Acc) -> lists:reverse([$\" | Acc]);\nencode_string([C | Cs], Acc) ->\n    case C of\n        $\" -> encode_string(Cs, [$\", $\\\\ | Acc]);\n        % (don't escape solidus on encode)\n        $\\\\ -> encode_string(Cs, [$\\\\, $\\\\ | Acc]);\n        $\\b -> encode_string(Cs, [$b, $\\\\ | Acc]);      % note missing \\\n        $\\f -> encode_string(Cs, [$f, $\\\\ | Acc]);\n        $\\n -> encode_string(Cs, [$n, $\\\\ | Acc]);\n        $\\r -> encode_string(Cs, [$r, $\\\\ | Acc]);\n        $\\t -> encode_string(Cs, [$t, $\\\\ | Acc]);\n        C when C >= 0, C < $\\s ->\n            % Control characters must be unicode-encoded.\n            Hex = lists:flatten(io_lib:format(\"~4.16.0b\", [C])),\n            encode_string(Cs, lists:reverse(Hex) ++ \"u\\\\\" ++ Acc); % \"\n        C when C =< 16#FFFF -> encode_string(Cs, [C | Acc]);\n        _ -> exit({json_encode, {bad_char, C}})\n    end.\n\n%% Encode an Erlang object as a JSON object, allowing string or atom keys.\n%% Note that order is irrelevant in both internal and external object\n%% representations.  Nevertheless, the output will respect the order\n%% of the input.\n\nencode_object({struct, _Props} = Obj) ->\n    M = obj_fold(fun({Key, Value}, Acc) ->\n        S = case Key of\n                B when is_binary(B) -> encode_string(B);\n                L when is_list(L) ->\n                    case is_string(L) of\n                        yes -> encode_string(L);\n                        unicode -> encode_string(xmerl_ucs:to_utf8(L));\n                        no -> exit({json_encode, {bad_key, Key}})\n                    end;\n                A when is_atom(A) -> encode_string(atom_to_list(A));\n                _ -> exit({json_encode, {bad_key, Key}})\n            end,\n        V = encode(Value),\n        case Acc of\n            [] -> [S, $:, V];\n            _ -> [Acc, $,, S, $:, V]\n        end\n    end, [], Obj),\n    [${, M, $}].\n\n%% Encode an Erlang tuple as a JSON array.\n%% Order *is* significant in a JSON array!\n\nencode_array(T) ->\n    M = lists:foldl(fun(E, Acc) ->\n        V = encode(E),\n        case Acc of\n            [] -> V;\n            _ -> [Acc, $,, V]\n        end\n    end, [], T),\n    [$[, M, $]].\n\n%%% SCANNING\n%%%\n%%% Scanning funs return either:\n%%%    {done, Result, LeftOverChars}\n%%% if a complete token is recognized, or\n%%%    {more, Continuation}\n%%% if more input is needed.\n%%% Result is {ok, Term}, 'eof', or {error, Reason}.\n%%% Here, the Continuation is a simple Erlang string.\n%%%\n%%% Currently, error handling is rather crude - errors are recognized\n%%% by match failures.  EOF is handled only by number scanning, where\n%%% it can delimit a number, and otherwise causes a match failure.\n%%%\n%%% Tokens are one of the following\n%%% JSON string -> erlang string\n%%% JSON number -> erlang number\n%%% true, false, null -> erlang atoms\n%%% { } [ ] : , -> lcbrace rcbrace lsbrace rsbrace colon comma\n-define(is_hex(X), ((X >= $a andalso X =< $f) orelse\n                    (X >= $A andalso X =< $F) orelse\n                    (X >= $0 andalso X =< $9))).\n\n-define(is_high_surrogate(U1, U2, U3, U4), (U1 == $d orelse U1 == $D) andalso\n                                           (U2 == $8 orelse U2 == $9 orelse\n                                            U2 == $a orelse U2 == $A orelse\n                                            U2 == $b orelse U2 == $B) andalso\n                                           ?is_hex(U3) andalso ?is_hex(U4)).\n\n-define(is_low_surrogate(U1, U2, U3, U4), (U1 == $d orelse U1 == $D) andalso\n                                          (U2 == $c orelse U2 == $C orelse\n                                           U2 == $d orelse U2 == $D orelse\n                                           U2 == $e orelse U2 == $E orelse\n                                           U2 == $f orelse U2 == $F) andalso\n                                          ?is_hex(U3) andalso ?is_hex(U4)).\n\n\ntoken([]) -> {more, []};\ntoken(eof) -> {done, eof, []};\n\ntoken(\"true\" ++ Rest) -> {done, {ok, true}, Rest};\ntoken(\"tru\")    -> {more, \"tru\"};\ntoken(\"tr\")     -> {more, \"tr\"};\ntoken(\"t\")      -> {more, \"t\"};\n\ntoken(\"false\" ++ Rest) -> {done, {ok, false}, Rest};\ntoken(\"fals\")   -> {more, \"fals\"};\ntoken(\"fal\")    -> {more, \"fal\"};\ntoken(\"fa\")     -> {more, \"fa\"};\ntoken(\"f\")      -> {more, \"f\"};\n\ntoken(\"null\" ++ Rest) -> {done, {ok, null}, Rest};\ntoken(\"nul\")    -> {more, \"nul\"};\ntoken(\"nu\")     -> {more, \"nu\"};\ntoken(\"n\")      -> {more, \"n\"};\n\ntoken([C | Cs] = Input) ->\n    case C of\n        $\\s -> token(Cs);       % eat whitespace\n        $\\t -> token(Cs);       % eat whitespace\n        $\\n -> token(Cs);       % eat whitespace\n        $\\r -> token(Cs);       % eat whitespace\n        $\" -> scan_string(Input);\n        $- -> scan_number(Input);\n        D when D >= $0, D =< $9-> scan_number(Input);\n        ${ -> {done, {ok, lcbrace}, Cs};\n        $} -> {done, {ok, rcbrace}, Cs};\n        $[ -> {done, {ok, lsbrace}, Cs};\n        $] -> {done, {ok, rsbrace}, Cs};\n        $: -> {done, {ok, colon}, Cs};\n        $, -> {done, {ok, comma}, Cs};\n        _ -> {done, {error, {bad_char, C}}, Cs}\n    end.\n\nscan_string([$\" | Cs] = Input) ->\n    scan_string(Cs, [], Input).\n\n%% Accumulate in reverse order, save original start-of-string for continuation.\nscan_string([], _, X) ->\n    {more, X};\nscan_string(eof, _, X) ->\n    {done, {error, missing_close_quote}, X};\nscan_string([$\" | Rest], A, _) ->\n    {done, {ok, lists:reverse(A)}, Rest};\nscan_string([$\\\\], _, X) ->\n    {more, X};\nscan_string([$\\\\,$u,U1,U2,U3,U4|Rest0],  A, X)\n  when ?is_high_surrogate(U1,U2,U3,U4) ->\n    case Rest0 of\n        [$\\\\,$u,V1,V2,V3,V4|Rest] when ?is_low_surrogate(V1,V2,V3,V4) ->\n            High = erlang:list_to_integer([U1,U2,U3,U4], 16),\n            Low  = erlang:list_to_integer([V1,V2,V3,V4], 16),\n            Codepoint = (High - 16#d800) * 16#400 + (Low - 16#dc00) + 16#10000,\n            scan_string(Rest, [Codepoint|A], X);\n\n        [$\\\\,$u,V1,V2,V3,V4|_] ->\n            Bad = [$\\\\,$u,U1,U2,U3,U4,$\\\\,$u,V1,V2,V3,V4],\n            {done, {error, {bad_surrogate_pair, Bad}}, X};\n\n        [$\\\\,$u,_,_,_|eof] -> {done, {error, missing_close_quote}, X};\n        [$\\\\,$u,_,_|eof]   -> {done, {error, missing_close_quote}, X};\n        [$\\\\,$u,_|eof]     -> {done, {error, missing_close_quote}, X};\n        [$\\\\,$u|eof]       -> {done, {error, missing_close_quote}, X};\n        [$\\\\|eof]          -> {done, {error, missing_close_quote}, X};\n        eof                -> {done, {error, missing_close_quote}, X};\n        [$\\\\,$u|_]         -> {more, X};\n        [$\\\\]              -> {more, X};\n        []                 -> {more, X};\n\n        _ ->\n            Bad = [$\\\\,$u,U1,U2,U3,U4],\n            {done, {error, {bad_utf8_char, Bad}}, X}\n    end;\nscan_string([$\\\\,$u,U1,U2,U3,U4|Rest], A, X)\n  when ?is_hex(U1) andalso ?is_hex(U2) andalso ?is_hex(U3) andalso ?is_hex(U4) ->\n    case erlang:list_to_integer([U1,U2,U3,U4], 16) of\n        Codepoint when Codepoint > 0 andalso\n                       (Codepoint < 16#d800 orelse Codepoint > 16#dfff) ->\n            scan_string(Rest, [Codepoint | A], X);\n        _ ->\n            Bad = [$\\\\,$u,U1,U2,U3,U4],\n            {done, {error, {bad_utf8_char, Bad}}, X}\n    end;\nscan_string([$\\\\,$u|Rest], _, X) ->\n    case Rest of\n        [U1,U2,U3,U4|_] ->\n            Bad = [$\\\\,$u,U1,U2,U3,U4],\n            {done, {error, {bad_utf8_char, Bad}}, X};\n\n        [_,_,_|eof] -> {done, {error, missing_close_quote}, X};\n        [_,_|eof]   -> {done, {error, missing_close_quote}, X};\n        [_|eof]     -> {done, {error, missing_close_quote}, X};\n        eof         -> {done, {error, missing_close_quote}, X};\n\n        _           -> {more, X}\n    end;\nscan_string([$\\\\,C|Rest], A, X) ->\n    case esc_to_char(C) of\n        {error, E} -> {done, {error, E}, X};\n        C1         -> scan_string(Rest, [C1|A], X)\n    end;\n\nscan_string([C|Rest0], A, X) when C >= 16#d800 andalso C =< 16#dfff ->\n    case Rest0 of\n        [D|Rest1] when D >= 16#dc00 andalso D =< 16#dfff ->\n            Codepoint = (C - 16#d800) * 16#400 + (D - 16#dc00) + 16#10000,\n            scan_string(Rest1, [Codepoint|A], X);\n        [D|_] ->\n            {done, {error, {bad_surrogate_pair, [C,D]}}, X};\n        _ ->\n            scan_string(Rest0, [C|A], X)\n    end;\n\n\nscan_string([C|Rest], A, X) when C == 16#20 orelse C == 16#21 orelse\n                                 (C >= 16#23 andalso C =< 16#5B) orelse\n                                 (C >= 16#5D andalso C =< 16#10FFFF) ->\n    scan_string(Rest, [C|A], X);\nscan_string(_, _, X) ->\n    {done, {error, invalid_string}, X}.\n\n\n\nesc_to_char($\") -> $\";\nesc_to_char($/) -> $/;\nesc_to_char($\\\\) -> $\\\\;\nesc_to_char($b) -> $\\b;\nesc_to_char($f) -> $\\f;\nesc_to_char($n) -> $\\n;\nesc_to_char($r) -> $\\r;\nesc_to_char($t) -> $\\t;\nesc_to_char(C) -> {error, {bad_char, C}}.\n\nscan_number([]) -> {more, []};\nscan_number(eof) -> {done, {error, incomplete_number}, []};\nscan_number([$-, $- | _Ds]) -> {done, {error, invalid_number}, []};\nscan_number([$- | Ds] = Input) ->\n    case scan_number(Ds) of\n        {more, _Cont} -> {more, Input};\n        {done, {ok, N}, CharList} -> {done, {ok, -1 * N}, CharList};\n        {done, Other, Chars} -> {done, Other, Chars}\n    end;\nscan_number([D | Ds] = Input) when D >= $0, D =< $9 ->\n    scan_number(Ds, D - $0, Input);\nscan_number(Input) ->\n    {done, {error, invalid_number}, Input}.\n\n\n%% Numbers don't have a terminator, so stop at the first non-digit,\n%% and ask for more if we run out.\n\nscan_number([], _A, X) -> {more, X};\nscan_number(eof, A, _X) -> {done, {ok, A}, eof};\nscan_number([$.], _A, X) -> {more, X};\nscan_number([$., D | Ds], A, X) when D >= $0, D =< $9 ->\n    scan_fraction([D | Ds], A, X);\nscan_number([D | Ds], A, X) when A > 0, D >= $0, D =< $9 ->\n    % Note that nonzero numbers can't start with \"0\".\n    scan_number(Ds, 10 * A + (D - $0), X);\nscan_number([D | Ds], A, X) when D == $E; D == $e ->\n    scan_exponent_begin(Ds, integer_to_list(A) ++ \".0\", X);\nscan_number([D | _] = Ds, A, _X) when D < $0; D > $9 ->\n    {done, {ok, A}, Ds};\nscan_number(_, _, X) ->\n    {done, {error, invalid_number}, X}.\n\nscan_fraction(Ds, I, X) -> scan_fraction(Ds, [], I, X).\n\nscan_fraction([], _Fs, _I, X) -> {more, X};\nscan_fraction(eof, Fs, I, _X) ->\n    try\n        R = list_to_float(lists:append([integer_to_list(I), \".\",\n                                        lists:reverse(Fs)])),\n        {done, {ok, R}, eof}\n    catch\n        _:_ -> {done, {error, number_overflow}, eof}\n    end;\nscan_fraction([D | Ds], Fs, I, X) when D >= $0, D =< $9 ->\n    scan_fraction(Ds, [D | Fs], I, X);\nscan_fraction([D | Ds], Fs, I, X) when D == $E; D == $e ->\n    R = lists:append([integer_to_list(I), \".\", lists:reverse(Fs)]),\n    scan_exponent_begin(Ds, R, X);\nscan_fraction(Rest, Fs, I, _X) ->\n    try\n        R = list_to_float(lists:append([integer_to_list(I), \".\",\n                                        lists:reverse(Fs)])),\n        {done, {ok, R}, Rest}\n    catch\n        _:_ -> {done, {error, number_overflow}, []}\n    end.\n\nscan_exponent_begin(Ds, R, X) ->\n    scan_exponent_begin(Ds, [], R, X).\n\nscan_exponent_begin([], _Es, _R, X) -> {more, X};\nscan_exponent_begin(eof, _Es, _R, X) -> {done, {error, missing_exponent}, X};\nscan_exponent_begin([D | Ds], Es, R, X) when D == $-;\n                                             D == $+;\n                                             D >= $0, D =< $9 ->\n    scan_exponent(Ds, [D | Es], R, X);\nscan_exponent_begin(_, _Es, _R, X) -> {done, {error, invalid_exponent}, X}.\n\n\nscan_exponent([], _Es, _R, X) -> {more, X};\nscan_exponent(eof, Es, R, _X) ->\n    try\n        X = list_to_float(lists:append([R, \"e\", lists:reverse(Es)])),\n        {done, {ok, X}, eof}\n    catch\n        _:_ -> {done, {error, number_overflow}, eof}\n    end;\nscan_exponent([D | Ds], Es, R, X) when D >= $0, D =< $9 ->\n    scan_exponent(Ds, [D | Es], R, X);\nscan_exponent(Rest, Es, R, _X) ->\n    try\n        X = list_to_float(lists:append([R, \"e\", lists:reverse(Es)])),\n        {done, {ok, X}, Rest}\n    catch\n        _:_ -> {done, {error, number_overflow}, Rest}\n    end.\n\n%%% PARSING\n%%%\n%%% The decode function takes a char list as input, but\n%%% interprets the end of the list as only an end to the available\n%%% input, and returns a \"continuation\" requesting more input.\n%%% When additional characters are available, they, and the\n%%% continuation, are fed into decode/2.  You can use the atom 'eof'\n%%% as a character to signal a true end to the input stream, and\n%%% possibly flush out an unfinished number.  The decode_string/1\n%%% function appends 'eof' to its input and calls decode/1.\n%%%\n%%% Parsing and scanning errors are handled only by match failures.\n%%% The external caller must take care to wrap the call in a \"catch\"\n%%% or \"try\" if better error-handling is desired.  Eventually parse\n%%% or scan errors will be returned explicitly with a description,\n%%% and someday with line numbers too.\n%%%\n%%% The parsing code uses a continuation-passing style to allow\n%%% for the parsing to suspend at any point and be resumed when\n%%% more input is available.\n%%% See http://en.wikipedia.org/wiki/Continuation_passing_style\n\n%% Return the first JSON value decoded from the input string.\n%% The string must contain at least one complete JSON value.\n\ndecode_string(CharList) ->\n    {done, V, _} = decode([], CharList ++ eof),\n    V.\n\n%% Attempt to decode a JSON value from the input string\n%% and continuation, using empty list for the initial continuation.\n%% Return {done, Result, LeftoverChars} if a value is recognized,\n%% or {more, Continuation} if more input characters are needed.\n%% The Result can be {ok, Value}, eof, or {error, Reason}.\n%% The Continuation is then fed as an argument to decode/2 when\n%% more input is available.\n%% Use the atom 'eof' instead of a char list to signal\n%% a true end to the input, and may flush a final number.\n\ndecode([], CharList) ->\n    decode(first_continuation(), CharList);\n\ndecode(Continuation, CharList) ->\n    {OldChars, Kt} = Continuation,\n    get_token(OldChars ++ CharList, Kt).\n\nfirst_continuation() ->\n    {[], fun\n             (eof, Cs) ->\n                 {done, eof, Cs};\n             (T, Cs) ->\n                 Fun = fun(V, eof)   -> {done, {ok, V}, eof};\n                          (V, [])    -> {done, {ok, V}, []};\n                          (V, [eof]) -> {done, {ok, V}, [eof]};\n                          (_, Cs2)   -> {done, {error, invalid_trailing_data}, Cs2}\n                       end,\n                 parse_value(T, Cs, Fun)\n    end}.\n\n%% Continuation Kt must accept (TokenOrEof, Chars)\n\nget_token(Chars, Kt) ->\n    case token(Chars) of\n        {done, {ok, T}, Rest} -> Kt(T, Rest);\n        {done, eof, Rest} -> Kt(eof, Rest);\n        {done, {error, Reason}, Rest} -> {done, {error, Reason}, Rest};\n        {more, X} -> {more, {X, Kt}}\n    end.\n\n%% Continuation Kv must accept (Value, Chars)\n\nparse_value(eof, C, _Kv) -> {done, {error, premature_eof}, C};\nparse_value(true, C, Kv) -> Kv(true, C);\nparse_value(false, C, Kv) -> Kv(false, C);\nparse_value(null, C, Kv) -> Kv(null, C);\nparse_value(S, C, Kv) when is_list(S) -> Kv(S, C);\nparse_value(N, C, Kv) when is_number(N) -> Kv(N, C);\nparse_value(lcbrace, C, Kv) -> parse_object(C, Kv);\nparse_value(lsbrace, C, Kv) -> parse_array(C, Kv);\nparse_value(_, C, _Kv) -> {done, {error, syntax_error}, C}.\n\n%% Continuation Kv must accept (Value, Chars)\n\nparse_object(Chars, Kv) ->\n    get_token(Chars, fun(T, C2) ->\n        Obj = obj_new(),\n        case T of\n            rcbrace -> Kv(Obj, C2);             % empty object\n            _ -> parse_object(Obj, T, C2, Kv)   % token must be string\n        end\n    end).\n\nparse_object(_Obj, eof, C, _Kv) ->\n    {done, {error, premature_eof}, C};\n\nparse_object(Obj, S, C, Kv) when is_list(S) ->    % S is member name\n    get_token(C, fun\n        (colon, C2) ->\n            parse_object2(Obj, S, C2, Kv);\n        (T, C2) ->\n            {done, {error, {expecting_colon, T}}, C2}\n    end);\n\nparse_object(_Obj, M, C, _Kv) ->\n    {done, {error, {member_name_not_string, M}}, C}.\n\nparse_object2(Obj, S, C, Kv) ->\n    get_token(C, fun\n        (eof, C2) ->\n            {done, {error, premature_eof}, C2};\n        (T, C2) ->\n            parse_value(T, C2, fun(V, C3) ->    % V is member value\n                Obj2 = obj_store(S, V, Obj),\n                get_token(C3, fun\n                    (rcbrace, C4) ->    % \"}\" end of object\n                                                {struct, PropList1} = Obj2,\n                        Kv({struct, lists:reverse(PropList1)}, C4);\n                    (comma, C4) ->              % \",\" another member follows\n                        get_token(C4, fun(T3, C5) ->\n                            parse_object(Obj2, T3, C5, Kv)\n                        end);\n                    (eof, C4) ->\n                        {done, {error, premature_eof}, C4};\n                    (T2, C4) ->\n                        {done, {error, {expecting_comma_or_curly, T2}}, C4}\n                end)\n            end)\n    end).\n\n%% Continuation Kv must accept (Value, Chars)\n\nparse_array(C, Kv) ->\n    get_token(C, fun\n        (eof, C2) -> {done, {error, premature_eof}, C2};\n        (rsbrace, C2) -> Kv({array, []}, C2);  % empty array\n        (T, C2) -> parse_array([], T, C2, Kv)\n    end).\n\nparse_array(E, T, C, Kv) ->\n    parse_value(T, C, fun(V, C2) ->\n        E2 = [V | E],\n        get_token(C2, fun\n            (rsbrace, C3) ->        % \"]\" end of array\n                Kv({array, lists:reverse(E2)}, C3);\n\n            (comma, C3) ->          % \",\" another value follows\n                get_token(C3, fun(T3, C4) ->\n                    parse_array(E2, T3, C4, Kv)\n                end);\n            (eof, C3) ->\n                {done, {error, premature_eof}, C3};\n            (T2, C3) ->\n                {done, {error, {expecting_comma_or_close_array, T2}}, C3}\n        end)\n    end).\n\n%%% OBJECTS\n%%%\n%%% We'll use tagged property lists as the internal representation\n%%% of JSON objects.  Unordered lists perform worse than trees for\n%%% lookup and modification of members, but we expect objects to be\n%%% have only a few members.  Lists also print better.\n\n%% Is this a proper JSON object representation?\n\nis_obj({struct, Props}) when is_list(Props) ->\n    lists:all(fun\n        ({Member, _Value}) when is_atom(Member); is_list(Member) -> true;\n        (_) -> false\n    end, Props);\n\nis_obj(_) ->\n    false.\n\n%% Create a new, empty object.\n\nobj_new() ->\n    {struct, []}.\n\n%% Fetch an object member's value, expecting it to be in the object.\n%% Return value, runtime error if no member found with that name.\n\nobj_fetch(Key, {struct, Props}) when is_list(Props) ->\n    case proplists:get_value(Key, Props) of\n        undefined ->\n            exit({struct_no_key, Key});\n        Value ->\n            Value\n    end.\n\n%% Fetch an object member's value, or indicate that there is no such member.\n%% Return {ok, Value} or 'error'.\n\nobj_find(Key, {struct, Props}) when is_list(Props) ->\n    case proplists:get_value(Key, Props) of\n        undefined ->\n            error;\n        Value ->\n            {ok, Value}\n    end.\n\nobj_is_key(Key, {struct, Props}) ->\n    proplists:is_defined(Key, Props).\n\n%% Store a new member in an object.  Returns a new object.\n\nobj_store(Key, Value, {struct, Props}) when is_list(Props) ->\n    {struct, [{Key, Value} | proplists:delete(Key, Props)]}.\n\n%% Create an object from a list of Key/Value pairs.\n\nobj_from_list(Props) ->\n    Obj = {struct, Props},\n    case is_obj(Obj) of\n        true -> Obj;\n        false -> exit(json_bad_object)\n    end.\n\n%% Fold Fun across object, with initial accumulator Acc.\n%% Fun should take (Value, Acc) as arguments and return Acc.\n\nobj_fold(Fun, Acc, {struct, Props}) ->\n    lists:foldl(Fun, Acc, Props).\n\nis_string([]) -> yes;\nis_string(List) -> is_string(List, non_unicode).\n\nis_string([C|Rest], non_unicode) when is_integer(C), C >= 0, C =< 255 ->\n    is_string(Rest, non_unicode);\nis_string([C|Rest], _) when is_integer(C), C =< 65000 ->\n    is_string(Rest, unicode);\nis_string([], non_unicode) -> yes;\nis_string([], unicode) -> unicode;\nis_string(_, _) -> no.\n\n\n%%% TESTING\n%%%\n%%% We can't expect to round-trip from JSON -> Erlang -> JSON,\n%%% due to the degrees of freedom in the JSON syntax: whitespace,\n%%% and ordering of object members.  We can, however, expect to\n%%% round-trip from Erlang -> JSON -> Erlang, so the JSON parsing\n%%% tests will in fact test the Erlang equivalence of the\n%%% JSON -> Erlang -> JSON -> Erlang coding chain.\n\n%% Test driver.  Return 'ok' or {failed, Failures}.\n\ntest() ->\n    E2Js = e2j_test_vec(),\n    Failures =\n        lists:foldl(\n          fun({E, J}, Fs) ->\n                  case (catch test_e2j(E, J)) of\n                      ok ->\n                          case (catch round_trip(E)) of\n                              ok ->\n                                  case (catch round_trip_one_char(E)) of\n                                      ok ->\n                                          Fs;\n                                      Reason ->\n                                          [{round_trip_one_char, E, Reason} |\n                                           Fs]\n                                  end;\n                              Reason ->\n                                  [{round_trip, E, Reason} | Fs]\n                          end;\n                      Reason ->\n                          [{erlang_to_json, E, J, Reason} | Fs]\n                  end;\n             (end_of_tests, Fs) ->\n                  Fs\n          end, [], E2Js),\n    case Failures of\n        [] -> ok;\n        _ -> {failed, Failures}\n    end.\n\n%% Test for conversion from Erlang to JSON.  Note that unequal strings\n%% may represent equal JSON data, due to discretionary whitespace,\n%% object member order, trailing zeroes in floating point, etc.\n%% Legitimate changes to the encoding routines may require tweaks to\n%% the reference JSON strings in e2j_test_vec().\n\n%% This clause handles floats specially due to the need for fuzzy matching\n%% to avoid slight differences due to conversions. Rather than direct\n%% comparison as done in the more general clause below, here we allow a\n%% small relative difference between expected and actual.\ntest_e2j(E, J) when is_float(E) ->\n    J2 = lists:flatten(encode(E)),\n    E2 = list_to_float(J2),\n    E1 = list_to_float(J),\n    Rel = abs(E2 - E1)/E,\n    true = Rel < 0.005,\n    ok;\ntest_e2j(E, J) ->\n    J2 = lists:flatten(encode(E)),\n    J = J2,                                     % raises error if unequal\n    ok.\n\n%% Test that Erlang -> JSON -> Erlang round-trip yields equivalent term.\n\nround_trip(E) ->\n    J2 = lists:flatten(encode(E)),\n    {ok, E2} = decode_string(J2),\n    true = equiv(E, E2),                        % raises error if false\n    ok.\n\n%% Round-trip with one character at a time to test all continuations.\n\nround_trip_one_char(E) ->\n    J = lists:flatten(encode(E)),\n    {done, {ok, E2}, _} = lists:foldl(fun(C, Ret) ->\n        case Ret of\n            {done, _, _} -> Ret;\n            {more, Cont} -> decode(Cont, [C])\n        end\n    end, {more, first_continuation()}, J ++ [eof]),\n    true = equiv(E, E2),                        % raises error if false\n    ok.\n\n%% Test for equivalence of Erlang terms.\n%% Due to arbitrary order of construction, equivalent objects might\n%% compare unequal as erlang terms, so we need to carefully recurse\n%% through aggregates (arrays and objects).\n\nequiv({struct, Props1}, {struct, Props2}) ->\n    equiv_object(Props1, Props2);\nequiv({array, ArrayList1}, {array, ArrayList2}) ->\n    equiv_array(ArrayList1, ArrayList2);\nequiv(N1, N2) when is_number(N1), is_number(N2) -> N1 == N2;\nequiv(S1, S2) when is_list(S1), is_list(S2) ->\n    case {is_string(S1), is_string(S2)} of\n        {unicode, unicode} ->\n            xmerl_ucs:to_utf8(S1) == xmerl_ucs:to_utf8(S2);\n        {unicode, _} ->\n            xmerl_ucs:to_utf8(S1) == S2;\n        {_, unicode} ->\n            S1 == xmerl_ucs:to_utf8(S2);\n        _ ->\n            S1 == S2\n    end;\nequiv(true, true) -> true;\nequiv(false, false) -> true;\nequiv(null, null) -> true.\n\n%% Object representation and traversal order is unknown.\n%% Use the sledgehammer and sort property lists.\n\nequiv_object(Props1, Props2) ->\n    L1 = lists:keysort(1, Props1),\n    L2 = lists:keysort(1, Props2),\n    Pairs = lists:zip(L1, L2),\n    true = lists:all(fun({{K1, V1}, {K2, V2}}) ->\n        equiv(K1, K2) and equiv(V1, V2)\n    end, Pairs).\n\n%% Recursively compare array elements for equivalence.\n\nequiv_array([], []) ->\n    true;\nequiv_array(A1, A2) when length(A1) == length(A2) ->\n    lists:all(fun({E1,E2}) ->\n                      equiv(E1, E2)\n              end, lists:zip(A1, A2)).\n\ne2j_test_vec() -> [\n    {1, \"1\"},\n    {3.1416, \"3.14160\"}, % text representation may truncate, trail zeroes\n    {-1, \"-1\"},\n    {-3.1416, \"-3.14160\"},\n    {12.0e10, \"1.20000e+11\"},\n    {1.234E+10, \"1.23400e+10\"},\n    {-1.234E-10, \"-1.23400e-10\"},\n    {\"foo\", \"\\\"foo\\\"\"},\n    {\"foo\" ++ [500] ++ \"bar\", [$\", $f, $o, $o, $\\307, $\\264, $b, $a, $r, $\"]},\n    {\"foo\" ++ [5] ++ \"bar\", \"\\\"foo\\\\u0005bar\\\"\"},\n    {\"\", \"\\\"\\\"\"},\n    {[], \"\\\"\\\"\"},\n    {\"\\n\\n\\n\", \"\\\"\\\\n\\\\n\\\\n\\\"\"},\n    {obj_new(), \"{}\"},\n    {obj_from_list([{\"foo\", \"bar\"}]), \"{\\\"foo\\\":\\\"bar\\\"}\"},\n    {obj_from_list([{\"foo\", \"bar\"}, {\"baz\", 123}]),\n     \"{\\\"foo\\\":\\\"bar\\\",\\\"baz\\\":123}\"},\n    {{array, []}, \"[]\"},\n    {{array, [{array, []}]}, \"[[]]\"},\n    {{array, [1, \"foo\"]}, \"[1,\\\"foo\\\"]\"},\n\n    % json array in a json object\n    {obj_from_list([{\"foo\", {array, [123]}}]),\n     \"{\\\"foo\\\":[123]}\"},\n\n    % json object in a json object\n    {obj_from_list([{\"foo\", obj_from_list([{\"bar\", true}])}]),\n     \"{\\\"foo\\\":{\\\"bar\\\":true}}\"},\n\n    % fold evaluation order\n    {obj_from_list([{\"foo\", {array, []}},\n                     {\"bar\", obj_from_list([{\"baz\", true}])},\n                     {\"alice\", \"bob\"}]),\n     \"{\\\"foo\\\":[],\\\"bar\\\":{\\\"baz\\\":true},\\\"alice\\\":\\\"bob\\\"}\"},\n\n    % json object in a json array\n    {{array, [-123, \"foo\", obj_from_list([{\"bar\", {array, []}}]), null]},\n     \"[-123,\\\"foo\\\",{\\\"bar\\\":[]},null]\"},\n\n    end_of_tests\n].\n\n%%% TODO:\n%%%\n%%% Measure the overhead of the CPS-based parser by writing a conventional\n%%% scanner-parser that expects all input to be available.\n%%%\n%%% Allow a compile-time option to decode object member names as atoms,\n%%% to reduce the internal representation overheads when communicating\n%%% with trusted peers.\n"
  },
  {
    "path": "contrib/yaws/src/jsonrpc.erl",
    "content": "%% Copyright (C) 2006 Gaspar Chilingarov <nm@web.am>\n%%                    Gurgen Tumanyan <barbarian@armkb.com>\n%% All rights reserved.\n%%\n%%\n%% Redistribution and use in source and binary forms, with or without\n%% modification, are permitted provided that the following conditions\n%% are met:\n%%\n%% 1. Redistributions of source code must retain the above copyright\n%%    notice, this list of conditions and the following disclaimer.\n%% 2. Redistributions in binary form must reproduce the above\n%%    copyright notice, this list of conditions and the following\n%%    disclaimer in the documentation and/or other materials provided\n%%    with the distribution.\n%%\n%% THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS\n%% OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED\n%% WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n%% ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY\n%% DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n%% DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE\n%% GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS\n%% INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,\n%% WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING\n%% NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS\n%% SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n\n-module(jsonrpc).\n-author(\"Gaspar Chilingarov <nm@web.am>, Gurgen Tumanyan <barbarian@armkb.com>\").\n-export([call/3]).\n-export([s/2]).        % extract element from proplist\n\n%%%\n%%% call function calls json-rpc method on remote host\n%%%\n%%% URL - remote server url (may use https)\n%%% Options - option list to be passed to http:request\n%%% (ssl options ot timeout, for example)\n%%% Payload -> {call, MethodName, Args} tuple\n%%% MethodName -> atom\n%%% Args -> list\n%%%\ncall(URL, Options, Payload) ->\n    try\n        {ok, CallPayloadDeep} = encode_call_payload(Payload),\n        CallPayload = lists:flatten(CallPayloadDeep),\n        {ok, Response} = httpc:request(\n                           post,\n                           {URL,[{\"Content-Length\",length(CallPayload)}],\n                            \"application/x-www-form-urlencoded\",CallPayload},\n                           Options, []),\n\n        RespBody = if\n                       (size(Response) == 2) or (size(Response) == 3) ->\n                           element(size(Response), Response)\n                   end,\n        decode_call_payload(RespBody)\n    catch\n        error:Err->\n            error_logger:error_report([{'json_rpc:call', error},\n                                       {error, Err},\n                                       {stack, erlang:get_stacktrace()}]),\n            {error,Err}\n    end.\n\n%%%\n%%% json-rpc.org defines such structure for making call\n%%%\nencode_call_payload({call, Method, Args}) when is_list(Args) ->\n    %% id makes sense when there are many requests in same\n    %% communication channel and replies can come in random\n    %% order here it can be changed to something less expensive\n    ID = element(3, yaws:unique_triple()),\n    Struct =  json2:encode({struct, [{\"jsonrpc\", \"2.0\"},\n                                     {method, Method},\n                                     {params, {array, Args}},\n                                     {id, ID}]}),\n    {ok, Struct}.\n\n%%%\n%%% decode response structure\n%%%\ndecode_call_payload(JSonStr) ->\n    {ok, JSON} = json2:decode_string(JSonStr),\n    Result = s(JSON, result),\n    Error = s(JSON, error),\n    case Error of\n        undefined ->\n            {ok,{response,[Result]}}; % make it compliant with xmlrpc response\n        Error ->\n            {error, Error}\n    end.\n\n%%% lookup element in proplist\ns({struct, List}, ElemName) ->\n    s(List, ElemName);\ns(List, ElemName) when is_list(List) ->\n    case lists:keysearch(ElemName,1,List) of\n        {value,{ElemName,Val}} ->\n            Val;\n        false when is_atom(ElemName) ->\n            ElemList = atom_to_list(ElemName),\n            case lists:keysearch(ElemList,1,List) of\n                {value,{ElemList,Val}} ->\n                    Val;\n                _ ->\n                    undefined\n            end;\n        _ ->\n            undefined\n    end.\n\n\n% vim: tabstop=4 ft=erlang\n"
  },
  {
    "path": "contrib/yaws/src/mime_type_c.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : mime_type_c.erl\n%%% Author  : Claes Wikstrom <klacke@hyber.org>\n%%% Purpose :\n%%% Created : 10 Jul 2002 by Claes Wikstrom <klacke@hyber.org>\n%%%----------------------------------------------------------------------\n\n-module(mime_type_c).\n-author('klacke@hyber.org').\n\n-export([generate/0, generate/3]).\n\n-include(\"../include/yaws.hrl\").\n-include(\"yaws_charset.hrl\").\n\n\n-define(MIME_TYPES_FILE, filename:join(yaws:get_priv_dir(), \"mime.types\")).\n-define(DEFAULT_MIME_TYPE, \"text/plain\").\n\n%% This function is used during Yaws' compilation. To rebuild/reload mime_types\n%% module, generate/3 _MUST_ be used.\ngenerate() ->\n    AppDir = yaws:get_app_dir(),\n    GInfo  = #mime_types_info{\n                mime_types_file = filename:join(AppDir, \"priv/mime.types\"),\n                default_charset = ?YAWS_CHARSET\n               },\n    ModFile = filename:join(AppDir,  \"src/mime_types.erl\"),\n\n    case generate(ModFile, GInfo, []) of\n        ok ->\n            erlang:halt(0);\n        {error, Reason} ->\n            error_logger:format(\"Cannot write module ~p: ~p\\n\",\n                                [ModFile, file:format_error(Reason)]),\n            erlang:halt(1)\n    end.\n\n\n%% GInfo      ::= #mime_types_info{}\n%% SInfoMap   ::= [{{ServerName, Port}, #mime_types_info{}}]\n%% ServerName ::= string() | atom()\ngenerate(ModFile, GInfo, SInfoMap) ->\n    case file:open(ModFile, [write]) of\n        {ok, Fd} ->\n            TypesData = [create_mime_types_data(Name, Info) ||\n                            {Name, Info} <- [{global, GInfo}|SInfoMap] ],\n\n            %% Generate module Header\n            IncDir  = yaws:get_inc_dir(),\n            IncFile = filename:join(IncDir, \"yaws.hrl\"),\n            Include = \"-include(\\\"\"++IncFile++\"\\\").\",\n            io:format(Fd,\n                      \"-module(mime_types).~n~n\"\n                      \"-export([default_type/0, default_type/1]).~n\"\n                      \"-export([t/1, revt/1]).~n\"\n                      \"-export([t/2, revt/2]).~n~n\"\n                      \"~s~n~n\", [Include]),\n\n\n            %% Generate default_type/0, t/1 and revt/1\n            io:format(Fd,\n                      \"default_type() -> default_type(global).~n\"\n                      \"t(Ext) -> t(global, Ext).~n\"\n                      \"revt(Ext) -> revt(global, Ext).~n~n\", []),\n\n            %% Generate default_type/1\n            io:format(Fd, \"default_type(#sconf{servername=SN, port=P}) -> \"\n                      \"default_type({SN,P});~n\", []),\n            lists:foreach(fun({Name, _, DefaultType, DefaultCharset}) ->\n                                  generate_default_type(Fd, Name, DefaultType,\n                                                        DefaultCharset)\n                          end, TypesData),\n            io:format(Fd, \"default_type(_) -> default_type(global).~n~n\", []),\n\n            %% Generate t/2 function\n            io:format(Fd, \"t(#sconf{servername=SN, port=P}, Ext) -> \"\n                      \"t({SN,P}, Ext);~n\", []),\n            lists:foreach(fun({Name, MimeTypes, DefaultType, DefaultCharset}) ->\n                                  generate_t(Fd, Name, MimeTypes,\n                                             DefaultType, DefaultCharset)\n                          end, TypesData),\n            io:format(Fd, \"t(_, Ext) -> t(global, Ext).~n~n\", []),\n\n            %% Generate revt/2 function\n            io:format(Fd,\n                      \"revt(#sconf{servername=SN, port=P}, Ext) -> \"\n                      \"revt({SN,P}, Ext);~n\",\n                      []),\n            lists:foreach(fun({Name, MimeTypes, DefaultType, DefaultCharset}) ->\n                                  generate_revt(Fd, Name, MimeTypes,\n                                                DefaultType, DefaultCharset)\n                          end, TypesData),\n            io:format(Fd, \"revt(_, RExt) -> revt(global, RExt).~n\", []),\n\n            file:close(Fd),\n            ok;\n\n        {error, Reason} ->\n            {error, Reason}\n    end.\n\n\n%% ----\ncreate_mime_types_data(Name, Info) ->\n    Charsets = Info#mime_types_info.charsets,\n    DefaultC = case Info#mime_types_info.default_charset of\n                   undefined -> \"\";\n                   DCharset  -> \"; charset=\" ++ DCharset\n               end,\n\n    Map = case Info#mime_types_info.mime_types_file of\n              undefined -> read_mime_types_file(?MIME_TYPES_FILE);\n              File      -> read_mime_types_file(File)\n          end,\n    TypesData =\n        lists:foldl(fun({Ext, MimeType}, Acc) ->\n                            ExtType = get_ext_type(Ext),\n                            Charset = case lists:keyfind(Ext, 1, Charsets) of\n                                          {_,C} ->\n                                              \"; charset=\" ++ C;\n                                          false ->\n                                              case MimeType of\n                                                  \"text/\"++_ -> DefaultC;\n                                                  _          -> \"\"\n                                              end\n                                      end,\n                            lists:keystore(Ext, 1, Acc,\n                                           {Ext, ExtType, MimeType, Charset})\n                    end, [], Map ++ Info#mime_types_info.types),\n    {Name, TypesData, Info#mime_types_info.default_type, DefaultC}.\n\n\n%% ----\ngenerate_default_type(Fd, Name, DefaultType, DefaultCharset) ->\n    io:format(Fd, \"default_type(~p) -> \\\"~s~s\\\";~n\",\n              [Name, DefaultType, DefaultCharset]).\n\n\n%% ----\ngenerate_t(Fd, Name, [], \"text/\"++_=DefaultType, DefaultCharset) ->\n    io:format(Fd, \"t(~p, _) -> {regular, \\\"~s~s\\\"};~n\",\n              [Name, DefaultType, DefaultCharset]);\ngenerate_t(Fd, Name, [], DefaultType, _) ->\n    io:format(Fd, \"t(~p, _) -> {regular, \\\"~s\\\"};~n\", [Name, DefaultType]);\ngenerate_t(Fd, Name, [{Ext,ExtType,MimeType,Charset}|Rest],\n           DefaultType, DefaultCharset) ->\n    case string:to_upper(Ext) of\n        Ext ->\n            io:format(Fd, \"t(~p, ~p) -> {~p, \\\"~s~s\\\"};~n\",\n                      [Name, Ext, ExtType, MimeType, Charset]);\n        UExt ->\n            io:format(Fd,\n                      \"t(~p, ~p) -> {~p, \\\"~s~s\\\"};~n\"\n                      \"t(~p, ~p) -> {~p, \\\"~s~s\\\"};~n\",\n                      [Name, Ext, ExtType, MimeType, Charset,\n                       Name, UExt, ExtType, MimeType, Charset])\n    end,\n    generate_t(Fd, Name, Rest, DefaultType, DefaultCharset).\n\n\n%% ----\ngenerate_revt(Fd, Name, [], \"text/\"++_=DefaultType, DefaultCharset) ->\n    io:format(Fd,\n              \"revt(~p, RExt) -> {regular, lists:reverse(RExt), \\\"~s~s\\\"};~n\",\n              [Name, DefaultType, DefaultCharset]);\ngenerate_revt(Fd, Name, [], DefaultType, _) ->\n    io:format(Fd,\n              \"revt(~p, RExt) -> {regular, lists:reverse(RExt), \\\"~s\\\"};~n\",\n              [Name, DefaultType]);\ngenerate_revt(Fd, Name, [{Ext,ExtType,MimeType,Charset}|Rest],\n           DefaultType, DefaultCharset) ->\n    RExt = lists:reverse(Ext),\n    case string:to_upper(Ext) of\n        Ext ->\n            io:format(Fd,\n                      \"revt(~p, ~p) -> {~p, ~p, \\\"~s~s\\\"};~n\",\n                      [Name, RExt, ExtType, Ext, MimeType, Charset]);\n        UExt ->\n            RUExt = lists:reverse(UExt),\n            io:format(Fd,\n                      \"revt(~p, ~p) -> {~p, ~p, \\\"~s~s\\\"};~n\"\n                      \"revt(~p, ~p) -> {~p, ~p, \\\"~s~s\\\"};~n\",\n                      [Name, RExt, ExtType, Ext, MimeType, Charset,\n                       Name, RUExt, ExtType, UExt, MimeType, Charset])\n    end,\n    generate_revt(Fd, Name, Rest, DefaultType, DefaultCharset).\n\n%% ----\nread_mime_types_file(File) ->\n    case file:open(File, [read]) of\n        {ok, Io} ->\n            %% Define mime-types for special extensions. It could be overridden\n            Acc0 = [{E, \"text/html\"} || E <- get_special_exts()],\n            read_mime_types_file(Io, 1, file:read_line(Io), Acc0);\n        {error, Reason} ->\n            error_logger:format(\"Cannot read ~p: ~p\\n\",\n                                [File, file:format_error(Reason)]),\n            []\n    end.\n\nread_mime_types_file(Io, _, eof, Acc) ->\n    file:close(Io),\n    lists:reverse(Acc);\nread_mime_types_file(Io, Lno, {error, Reason}, Acc) ->\n    file:close(Io),\n    error_logger:format(\"read mime-types config failed at line ~p: ~p\\n\",\n                        [Lno, file:format_error(Reason)]),\n    lists:reverse(Acc);\nread_mime_types_file(Io, Lno, {ok, [$#|_]}, Acc) ->\n    read_mime_types_file(Io, Lno+1, file:read_line(Io), Acc);\nread_mime_types_file(Io, Lno, {ok, [$\\s|_]}, Acc) ->\n    read_mime_types_file(Io, Lno+1, file:read_line(Io), Acc);\nread_mime_types_file(Io, Lno, {ok, Line}, Acc0) ->\n    case string:tokens(Line,\"\\r\\n\\s\\t\\0\\f\") of\n        []  ->\n            read_mime_types_file(Io, Lno+1, file:read_line(Io), Acc0);\n        [_] ->\n            read_mime_types_file(Io, Lno+1, file:read_line(Io), Acc0);\n        [MimeType | Exts] ->\n            Acc1 = lists:foldl(fun(Ext, Acc) ->\n                                       lists:keystore(Ext, 1, Acc,\n                                                      {Ext, MimeType})\n                               end, Acc0, Exts),\n            read_mime_types_file(Io, Lno+1, file:read_line(Io), Acc1)\n    end.\n\n\n%% ----\nget_special_exts() -> [\"yaws\", \"php\", \"cgi\", \"fcgi\"].\n\nget_ext_type(\"yaws\") -> yaws;\nget_ext_type(\"php\")  -> php;\nget_ext_type(\"cgi\")  -> cgi;\nget_ext_type(\"fcgi\") -> fcgi;\nget_ext_type(_)      -> regular.\n\n"
  },
  {
    "path": "contrib/yaws/src/mime_types.erl",
    "content": "-module(mime_types).\n\n-export([default_type/0, default_type/1]).\n-export([t/1, revt/1]).\n-export([t/2, revt/2]).\n\n-include(\"yaws.hrl\").\n\ndefault_type() -> default_type(global).\nt(Ext) -> t(global, Ext).\nrevt(Ext) -> revt(global, Ext).\n\ndefault_type(#sconf{servername=SN, port=P}) -> default_type({SN,P});\ndefault_type(global) -> \"text/plain\";\ndefault_type(_) -> default_type(global).\n\nt(#sconf{servername=SN, port=P}, Ext) -> t({SN,P}, Ext);\nt(global, \"ice\") -> {regular, \"x-conference/x-cooltalk\"};\nt(global, \"ICE\") -> {regular, \"x-conference/x-cooltalk\"};\nt(global, \"smv\") -> {regular, \"video/x-smv\"};\nt(global, \"SMV\") -> {regular, \"video/x-smv\"};\nt(global, \"movie\") -> {regular, \"video/x-sgi-movie\"};\nt(global, \"MOVIE\") -> {regular, \"video/x-sgi-movie\"};\nt(global, \"avi\") -> {regular, \"video/x-msvideo\"};\nt(global, \"AVI\") -> {regular, \"video/x-msvideo\"};\nt(global, \"wvx\") -> {regular, \"video/x-ms-wvx\"};\nt(global, \"WVX\") -> {regular, \"video/x-ms-wvx\"};\nt(global, \"wmx\") -> {regular, \"video/x-ms-wmx\"};\nt(global, \"WMX\") -> {regular, \"video/x-ms-wmx\"};\nt(global, \"wmv\") -> {regular, \"video/x-ms-wmv\"};\nt(global, \"WMV\") -> {regular, \"video/x-ms-wmv\"};\nt(global, \"wm\") -> {regular, \"video/x-ms-wm\"};\nt(global, \"WM\") -> {regular, \"video/x-ms-wm\"};\nt(global, \"vob\") -> {regular, \"video/x-ms-vob\"};\nt(global, \"VOB\") -> {regular, \"video/x-ms-vob\"};\nt(global, \"asx\") -> {regular, \"video/x-ms-asf\"};\nt(global, \"ASX\") -> {regular, \"video/x-ms-asf\"};\nt(global, \"asf\") -> {regular, \"video/x-ms-asf\"};\nt(global, \"ASF\") -> {regular, \"video/x-ms-asf\"};\nt(global, \"mng\") -> {regular, \"video/x-mng\"};\nt(global, \"MNG\") -> {regular, \"video/x-mng\"};\nt(global, \"mks\") -> {regular, \"video/x-matroska\"};\nt(global, \"MKS\") -> {regular, \"video/x-matroska\"};\nt(global, \"mk3d\") -> {regular, \"video/x-matroska\"};\nt(global, \"MK3D\") -> {regular, \"video/x-matroska\"};\nt(global, \"mkv\") -> {regular, \"video/x-matroska\"};\nt(global, \"MKV\") -> {regular, \"video/x-matroska\"};\nt(global, \"m4v\") -> {regular, \"video/x-m4v\"};\nt(global, \"M4V\") -> {regular, \"video/x-m4v\"};\nt(global, \"flv\") -> {regular, \"video/x-flv\"};\nt(global, \"FLV\") -> {regular, \"video/x-flv\"};\nt(global, \"fli\") -> {regular, \"video/x-fli\"};\nt(global, \"FLI\") -> {regular, \"video/x-fli\"};\nt(global, \"f4v\") -> {regular, \"video/x-f4v\"};\nt(global, \"F4V\") -> {regular, \"video/x-f4v\"};\nt(global, \"webm\") -> {regular, \"video/webm\"};\nt(global, \"WEBM\") -> {regular, \"video/webm\"};\nt(global, \"viv\") -> {regular, \"video/vnd.vivo\"};\nt(global, \"VIV\") -> {regular, \"video/vnd.vivo\"};\nt(global, \"uvvu\") -> {regular, \"video/vnd.uvvu.mp4\"};\nt(global, \"UVVU\") -> {regular, \"video/vnd.uvvu.mp4\"};\nt(global, \"uvu\") -> {regular, \"video/vnd.uvvu.mp4\"};\nt(global, \"UVU\") -> {regular, \"video/vnd.uvvu.mp4\"};\nt(global, \"pyv\") -> {regular, \"video/vnd.ms-playready.media.pyv\"};\nt(global, \"PYV\") -> {regular, \"video/vnd.ms-playready.media.pyv\"};\nt(global, \"m4u\") -> {regular, \"video/vnd.mpegurl\"};\nt(global, \"M4U\") -> {regular, \"video/vnd.mpegurl\"};\nt(global, \"mxu\") -> {regular, \"video/vnd.mpegurl\"};\nt(global, \"MXU\") -> {regular, \"video/vnd.mpegurl\"};\nt(global, \"fvt\") -> {regular, \"video/vnd.fvt\"};\nt(global, \"FVT\") -> {regular, \"video/vnd.fvt\"};\nt(global, \"dvb\") -> {regular, \"video/vnd.dvb.file\"};\nt(global, \"DVB\") -> {regular, \"video/vnd.dvb.file\"};\nt(global, \"uvvv\") -> {regular, \"video/vnd.dece.video\"};\nt(global, \"UVVV\") -> {regular, \"video/vnd.dece.video\"};\nt(global, \"uvv\") -> {regular, \"video/vnd.dece.video\"};\nt(global, \"UVV\") -> {regular, \"video/vnd.dece.video\"};\nt(global, \"uvvs\") -> {regular, \"video/vnd.dece.sd\"};\nt(global, \"UVVS\") -> {regular, \"video/vnd.dece.sd\"};\nt(global, \"uvs\") -> {regular, \"video/vnd.dece.sd\"};\nt(global, \"UVS\") -> {regular, \"video/vnd.dece.sd\"};\nt(global, \"uvvp\") -> {regular, \"video/vnd.dece.pd\"};\nt(global, \"UVVP\") -> {regular, \"video/vnd.dece.pd\"};\nt(global, \"uvp\") -> {regular, \"video/vnd.dece.pd\"};\nt(global, \"UVP\") -> {regular, \"video/vnd.dece.pd\"};\nt(global, \"uvvm\") -> {regular, \"video/vnd.dece.mobile\"};\nt(global, \"UVVM\") -> {regular, \"video/vnd.dece.mobile\"};\nt(global, \"uvm\") -> {regular, \"video/vnd.dece.mobile\"};\nt(global, \"UVM\") -> {regular, \"video/vnd.dece.mobile\"};\nt(global, \"uvvh\") -> {regular, \"video/vnd.dece.hd\"};\nt(global, \"UVVH\") -> {regular, \"video/vnd.dece.hd\"};\nt(global, \"uvh\") -> {regular, \"video/vnd.dece.hd\"};\nt(global, \"UVH\") -> {regular, \"video/vnd.dece.hd\"};\nt(global, \"mov\") -> {regular, \"video/quicktime\"};\nt(global, \"MOV\") -> {regular, \"video/quicktime\"};\nt(global, \"qt\") -> {regular, \"video/quicktime\"};\nt(global, \"QT\") -> {regular, \"video/quicktime\"};\nt(global, \"ogv\") -> {regular, \"video/ogg\"};\nt(global, \"OGV\") -> {regular, \"video/ogg\"};\nt(global, \"m2v\") -> {regular, \"video/mpeg\"};\nt(global, \"M2V\") -> {regular, \"video/mpeg\"};\nt(global, \"m1v\") -> {regular, \"video/mpeg\"};\nt(global, \"M1V\") -> {regular, \"video/mpeg\"};\nt(global, \"mpe\") -> {regular, \"video/mpeg\"};\nt(global, \"MPE\") -> {regular, \"video/mpeg\"};\nt(global, \"mpg\") -> {regular, \"video/mpeg\"};\nt(global, \"MPG\") -> {regular, \"video/mpeg\"};\nt(global, \"mpeg\") -> {regular, \"video/mpeg\"};\nt(global, \"MPEG\") -> {regular, \"video/mpeg\"};\nt(global, \"mpg4\") -> {regular, \"video/mp4\"};\nt(global, \"MPG4\") -> {regular, \"video/mp4\"};\nt(global, \"mp4v\") -> {regular, \"video/mp4\"};\nt(global, \"MP4V\") -> {regular, \"video/mp4\"};\nt(global, \"mp4\") -> {regular, \"video/mp4\"};\nt(global, \"MP4\") -> {regular, \"video/mp4\"};\nt(global, \"ts\") -> {regular, \"video/mp2t\"};\nt(global, \"TS\") -> {regular, \"video/mp2t\"};\nt(global, \"mjp2\") -> {regular, \"video/mj2\"};\nt(global, \"MJP2\") -> {regular, \"video/mj2\"};\nt(global, \"mj2\") -> {regular, \"video/mj2\"};\nt(global, \"MJ2\") -> {regular, \"video/mj2\"};\nt(global, \"jpgm\") -> {regular, \"video/jpm\"};\nt(global, \"JPGM\") -> {regular, \"video/jpm\"};\nt(global, \"jpm\") -> {regular, \"video/jpm\"};\nt(global, \"JPM\") -> {regular, \"video/jpm\"};\nt(global, \"jpgv\") -> {regular, \"video/jpeg\"};\nt(global, \"JPGV\") -> {regular, \"video/jpeg\"};\nt(global, \"h264\") -> {regular, \"video/h264\"};\nt(global, \"H264\") -> {regular, \"video/h264\"};\nt(global, \"h263\") -> {regular, \"video/h263\"};\nt(global, \"H263\") -> {regular, \"video/h263\"};\nt(global, \"h261\") -> {regular, \"video/h261\"};\nt(global, \"H261\") -> {regular, \"video/h261\"};\nt(global, \"3g2\") -> {regular, \"video/3gpp2\"};\nt(global, \"3G2\") -> {regular, \"video/3gpp2\"};\nt(global, \"3gp\") -> {regular, \"video/3gpp\"};\nt(global, \"3GP\") -> {regular, \"video/3gpp\"};\nt(global, \"vcf\") -> {regular, \"text/x-vcard\"};\nt(global, \"VCF\") -> {regular, \"text/x-vcard\"};\nt(global, \"vcs\") -> {regular, \"text/x-vcalendar\"};\nt(global, \"VCS\") -> {regular, \"text/x-vcalendar\"};\nt(global, \"uu\") -> {regular, \"text/x-uuencode\"};\nt(global, \"UU\") -> {regular, \"text/x-uuencode\"};\nt(global, \"sfv\") -> {regular, \"text/x-sfv\"};\nt(global, \"SFV\") -> {regular, \"text/x-sfv\"};\nt(global, \"etx\") -> {regular, \"text/x-setext\"};\nt(global, \"ETX\") -> {regular, \"text/x-setext\"};\nt(global, \"pas\") -> {regular, \"text/x-pascal\"};\nt(global, \"PAS\") -> {regular, \"text/x-pascal\"};\nt(global, \"p\") -> {regular, \"text/x-pascal\"};\nt(global, \"P\") -> {regular, \"text/x-pascal\"};\nt(global, \"opml\") -> {regular, \"text/x-opml\"};\nt(global, \"OPML\") -> {regular, \"text/x-opml\"};\nt(global, \"nfo\") -> {regular, \"text/x-nfo\"};\nt(global, \"NFO\") -> {regular, \"text/x-nfo\"};\nt(global, \"java\") -> {regular, \"text/x-java-source\"};\nt(global, \"JAVA\") -> {regular, \"text/x-java-source\"};\nt(global, \"f90\") -> {regular, \"text/x-fortran\"};\nt(global, \"F90\") -> {regular, \"text/x-fortran\"};\nt(global, \"f77\") -> {regular, \"text/x-fortran\"};\nt(global, \"F77\") -> {regular, \"text/x-fortran\"};\nt(global, \"for\") -> {regular, \"text/x-fortran\"};\nt(global, \"FOR\") -> {regular, \"text/x-fortran\"};\nt(global, \"f\") -> {regular, \"text/x-fortran\"};\nt(global, \"F\") -> {regular, \"text/x-fortran\"};\nt(global, \"dic\") -> {regular, \"text/x-c\"};\nt(global, \"DIC\") -> {regular, \"text/x-c\"};\nt(global, \"hh\") -> {regular, \"text/x-c\"};\nt(global, \"HH\") -> {regular, \"text/x-c\"};\nt(global, \"h\") -> {regular, \"text/x-c\"};\nt(global, \"H\") -> {regular, \"text/x-c\"};\nt(global, \"cpp\") -> {regular, \"text/x-c\"};\nt(global, \"CPP\") -> {regular, \"text/x-c\"};\nt(global, \"cxx\") -> {regular, \"text/x-c\"};\nt(global, \"CXX\") -> {regular, \"text/x-c\"};\nt(global, \"cc\") -> {regular, \"text/x-c\"};\nt(global, \"CC\") -> {regular, \"text/x-c\"};\nt(global, \"c\") -> {regular, \"text/x-c\"};\nt(global, \"C\") -> {regular, \"text/x-c\"};\nt(global, \"asm\") -> {regular, \"text/x-asm\"};\nt(global, \"ASM\") -> {regular, \"text/x-asm\"};\nt(global, \"s\") -> {regular, \"text/x-asm\"};\nt(global, \"S\") -> {regular, \"text/x-asm\"};\nt(global, \"wmls\") -> {regular, \"text/vnd.wap.wmlscript\"};\nt(global, \"WMLS\") -> {regular, \"text/vnd.wap.wmlscript\"};\nt(global, \"wml\") -> {regular, \"text/vnd.wap.wml\"};\nt(global, \"WML\") -> {regular, \"text/vnd.wap.wml\"};\nt(global, \"sl\") -> {regular, \"text/vnd.wap.sl\"};\nt(global, \"SL\") -> {regular, \"text/vnd.wap.sl\"};\nt(global, \"si\") -> {regular, \"text/vnd.wap.si\"};\nt(global, \"SI\") -> {regular, \"text/vnd.wap.si\"};\nt(global, \"jad\") -> {regular, \"text/vnd.sun.j2me.app-descriptor\"};\nt(global, \"JAD\") -> {regular, \"text/vnd.sun.j2me.app-descriptor\"};\nt(global, \"spot\") -> {regular, \"text/vnd.in3d.spot\"};\nt(global, \"SPOT\") -> {regular, \"text/vnd.in3d.spot\"};\nt(global, \"3dml\") -> {regular, \"text/vnd.in3d.3dml\"};\nt(global, \"3DML\") -> {regular, \"text/vnd.in3d.3dml\"};\nt(global, \"gv\") -> {regular, \"text/vnd.graphviz\"};\nt(global, \"GV\") -> {regular, \"text/vnd.graphviz\"};\nt(global, \"flx\") -> {regular, \"text/vnd.fmi.flexstor\"};\nt(global, \"FLX\") -> {regular, \"text/vnd.fmi.flexstor\"};\nt(global, \"fly\") -> {regular, \"text/vnd.fly\"};\nt(global, \"FLY\") -> {regular, \"text/vnd.fly\"};\nt(global, \"scurl\") -> {regular, \"text/vnd.curl.scurl\"};\nt(global, \"SCURL\") -> {regular, \"text/vnd.curl.scurl\"};\nt(global, \"mcurl\") -> {regular, \"text/vnd.curl.mcurl\"};\nt(global, \"MCURL\") -> {regular, \"text/vnd.curl.mcurl\"};\nt(global, \"dcurl\") -> {regular, \"text/vnd.curl.dcurl\"};\nt(global, \"DCURL\") -> {regular, \"text/vnd.curl.dcurl\"};\nt(global, \"curl\") -> {regular, \"text/vnd.curl\"};\nt(global, \"CURL\") -> {regular, \"text/vnd.curl\"};\nt(global, \"vcard\") -> {regular, \"text/vcard\"};\nt(global, \"VCARD\") -> {regular, \"text/vcard\"};\nt(global, \"urls\") -> {regular, \"text/uri-list\"};\nt(global, \"URLS\") -> {regular, \"text/uri-list\"};\nt(global, \"uris\") -> {regular, \"text/uri-list\"};\nt(global, \"URIS\") -> {regular, \"text/uri-list\"};\nt(global, \"uri\") -> {regular, \"text/uri-list\"};\nt(global, \"URI\") -> {regular, \"text/uri-list\"};\nt(global, \"ttl\") -> {regular, \"text/turtle\"};\nt(global, \"TTL\") -> {regular, \"text/turtle\"};\nt(global, \"me\") -> {regular, \"text/troff\"};\nt(global, \"ME\") -> {regular, \"text/troff\"};\nt(global, \"man\") -> {regular, \"text/troff\"};\nt(global, \"MAN\") -> {regular, \"text/troff\"};\nt(global, \"roff\") -> {regular, \"text/troff\"};\nt(global, \"ROFF\") -> {regular, \"text/troff\"};\nt(global, \"tr\") -> {regular, \"text/troff\"};\nt(global, \"TR\") -> {regular, \"text/troff\"};\nt(global, \"t\") -> {regular, \"text/troff\"};\nt(global, \"T\") -> {regular, \"text/troff\"};\nt(global, \"tsv\") -> {regular, \"text/tab-separated-values\"};\nt(global, \"TSV\") -> {regular, \"text/tab-separated-values\"};\nt(global, \"sgm\") -> {regular, \"text/sgml\"};\nt(global, \"SGM\") -> {regular, \"text/sgml\"};\nt(global, \"sgml\") -> {regular, \"text/sgml\"};\nt(global, \"SGML\") -> {regular, \"text/sgml\"};\nt(global, \"rtx\") -> {regular, \"text/richtext\"};\nt(global, \"RTX\") -> {regular, \"text/richtext\"};\nt(global, \"dsc\") -> {regular, \"text/prs.lines.tag\"};\nt(global, \"DSC\") -> {regular, \"text/prs.lines.tag\"};\nt(global, \"in\") -> {regular, \"text/plain\"};\nt(global, \"IN\") -> {regular, \"text/plain\"};\nt(global, \"log\") -> {regular, \"text/plain\"};\nt(global, \"LOG\") -> {regular, \"text/plain\"};\nt(global, \"list\") -> {regular, \"text/plain\"};\nt(global, \"LIST\") -> {regular, \"text/plain\"};\nt(global, \"def\") -> {regular, \"text/plain\"};\nt(global, \"DEF\") -> {regular, \"text/plain\"};\nt(global, \"conf\") -> {regular, \"text/plain\"};\nt(global, \"CONF\") -> {regular, \"text/plain\"};\nt(global, \"text\") -> {regular, \"text/plain\"};\nt(global, \"TEXT\") -> {regular, \"text/plain\"};\nt(global, \"txt\") -> {regular, \"text/plain\"};\nt(global, \"TXT\") -> {regular, \"text/plain\"};\nt(global, \"n3\") -> {regular, \"text/n3\"};\nt(global, \"N3\") -> {regular, \"text/n3\"};\nt(global, \"htm\") -> {regular, \"text/html\"};\nt(global, \"HTM\") -> {regular, \"text/html\"};\nt(global, \"html\") -> {regular, \"text/html\"};\nt(global, \"HTML\") -> {regular, \"text/html\"};\nt(global, \"csv\") -> {regular, \"text/csv\"};\nt(global, \"CSV\") -> {regular, \"text/csv\"};\nt(global, \"css\") -> {regular, \"text/css\"};\nt(global, \"CSS\") -> {regular, \"text/css\"};\nt(global, \"ifb\") -> {regular, \"text/calendar\"};\nt(global, \"IFB\") -> {regular, \"text/calendar\"};\nt(global, \"ics\") -> {regular, \"text/calendar\"};\nt(global, \"ICS\") -> {regular, \"text/calendar\"};\nt(global, \"appcache\") -> {regular, \"text/cache-manifest\"};\nt(global, \"APPCACHE\") -> {regular, \"text/cache-manifest\"};\nt(global, \"x3dz\") -> {regular, \"model/x3d+xml\"};\nt(global, \"X3DZ\") -> {regular, \"model/x3d+xml\"};\nt(global, \"x3dvz\") -> {regular, \"model/x3d+vrml\"};\nt(global, \"X3DVZ\") -> {regular, \"model/x3d+vrml\"};\nt(global, \"x3dv\") -> {regular, \"model/x3d+vrml\"};\nt(global, \"X3DV\") -> {regular, \"model/x3d+vrml\"};\nt(global, \"x3dbz\") -> {regular, \"model/x3d+binary\"};\nt(global, \"X3DBZ\") -> {regular, \"model/x3d+binary\"};\nt(global, \"x3db\") -> {regular, \"model/x3d+binary\"};\nt(global, \"X3DB\") -> {regular, \"model/x3d+binary\"};\nt(global, \"vrml\") -> {regular, \"model/vrml\"};\nt(global, \"VRML\") -> {regular, \"model/vrml\"};\nt(global, \"wrl\") -> {regular, \"model/vrml\"};\nt(global, \"WRL\") -> {regular, \"model/vrml\"};\nt(global, \"vtu\") -> {regular, \"model/vnd.vtu\"};\nt(global, \"VTU\") -> {regular, \"model/vnd.vtu\"};\nt(global, \"mts\") -> {regular, \"model/vnd.mts\"};\nt(global, \"MTS\") -> {regular, \"model/vnd.mts\"};\nt(global, \"gtw\") -> {regular, \"model/vnd.gtw\"};\nt(global, \"GTW\") -> {regular, \"model/vnd.gtw\"};\nt(global, \"gdl\") -> {regular, \"model/vnd.gdl\"};\nt(global, \"GDL\") -> {regular, \"model/vnd.gdl\"};\nt(global, \"dwf\") -> {regular, \"model/vnd.dwf\"};\nt(global, \"DWF\") -> {regular, \"model/vnd.dwf\"};\nt(global, \"dae\") -> {regular, \"model/vnd.collada+xml\"};\nt(global, \"DAE\") -> {regular, \"model/vnd.collada+xml\"};\nt(global, \"silo\") -> {regular, \"model/mesh\"};\nt(global, \"SILO\") -> {regular, \"model/mesh\"};\nt(global, \"mesh\") -> {regular, \"model/mesh\"};\nt(global, \"MESH\") -> {regular, \"model/mesh\"};\nt(global, \"msh\") -> {regular, \"model/mesh\"};\nt(global, \"MSH\") -> {regular, \"model/mesh\"};\nt(global, \"iges\") -> {regular, \"model/iges\"};\nt(global, \"IGES\") -> {regular, \"model/iges\"};\nt(global, \"igs\") -> {regular, \"model/iges\"};\nt(global, \"IGS\") -> {regular, \"model/iges\"};\nt(global, \"mime\") -> {regular, \"message/rfc822\"};\nt(global, \"MIME\") -> {regular, \"message/rfc822\"};\nt(global, \"eml\") -> {regular, \"message/rfc822\"};\nt(global, \"EML\") -> {regular, \"message/rfc822\"};\nt(global, \"xwd\") -> {regular, \"image/x-xwindowdump\"};\nt(global, \"XWD\") -> {regular, \"image/x-xwindowdump\"};\nt(global, \"xpm\") -> {regular, \"image/x-xpixmap\"};\nt(global, \"XPM\") -> {regular, \"image/x-xpixmap\"};\nt(global, \"xbm\") -> {regular, \"image/x-xbitmap\"};\nt(global, \"XBM\") -> {regular, \"image/x-xbitmap\"};\nt(global, \"tga\") -> {regular, \"image/x-tga\"};\nt(global, \"TGA\") -> {regular, \"image/x-tga\"};\nt(global, \"rgb\") -> {regular, \"image/x-rgb\"};\nt(global, \"RGB\") -> {regular, \"image/x-rgb\"};\nt(global, \"ppm\") -> {regular, \"image/x-portable-pixmap\"};\nt(global, \"PPM\") -> {regular, \"image/x-portable-pixmap\"};\nt(global, \"pgm\") -> {regular, \"image/x-portable-graymap\"};\nt(global, \"PGM\") -> {regular, \"image/x-portable-graymap\"};\nt(global, \"pbm\") -> {regular, \"image/x-portable-bitmap\"};\nt(global, \"PBM\") -> {regular, \"image/x-portable-bitmap\"};\nt(global, \"pnm\") -> {regular, \"image/x-portable-anymap\"};\nt(global, \"PNM\") -> {regular, \"image/x-portable-anymap\"};\nt(global, \"pct\") -> {regular, \"image/x-pict\"};\nt(global, \"PCT\") -> {regular, \"image/x-pict\"};\nt(global, \"pic\") -> {regular, \"image/x-pict\"};\nt(global, \"PIC\") -> {regular, \"image/x-pict\"};\nt(global, \"pcx\") -> {regular, \"image/x-pcx\"};\nt(global, \"PCX\") -> {regular, \"image/x-pcx\"};\nt(global, \"sid\") -> {regular, \"image/x-mrsid-image\"};\nt(global, \"SID\") -> {regular, \"image/x-mrsid-image\"};\nt(global, \"ico\") -> {regular, \"image/x-icon\"};\nt(global, \"ICO\") -> {regular, \"image/x-icon\"};\nt(global, \"fh7\") -> {regular, \"image/x-freehand\"};\nt(global, \"FH7\") -> {regular, \"image/x-freehand\"};\nt(global, \"fh5\") -> {regular, \"image/x-freehand\"};\nt(global, \"FH5\") -> {regular, \"image/x-freehand\"};\nt(global, \"fh4\") -> {regular, \"image/x-freehand\"};\nt(global, \"FH4\") -> {regular, \"image/x-freehand\"};\nt(global, \"fhc\") -> {regular, \"image/x-freehand\"};\nt(global, \"FHC\") -> {regular, \"image/x-freehand\"};\nt(global, \"fh\") -> {regular, \"image/x-freehand\"};\nt(global, \"FH\") -> {regular, \"image/x-freehand\"};\nt(global, \"cmx\") -> {regular, \"image/x-cmx\"};\nt(global, \"CMX\") -> {regular, \"image/x-cmx\"};\nt(global, \"ras\") -> {regular, \"image/x-cmu-raster\"};\nt(global, \"RAS\") -> {regular, \"image/x-cmu-raster\"};\nt(global, \"3ds\") -> {regular, \"image/x-3ds\"};\nt(global, \"3DS\") -> {regular, \"image/x-3ds\"};\nt(global, \"webp\") -> {regular, \"image/webp\"};\nt(global, \"WEBP\") -> {regular, \"image/webp\"};\nt(global, \"xif\") -> {regular, \"image/vnd.xiff\"};\nt(global, \"XIF\") -> {regular, \"image/vnd.xiff\"};\nt(global, \"wbmp\") -> {regular, \"image/vnd.wap.wbmp\"};\nt(global, \"WBMP\") -> {regular, \"image/vnd.wap.wbmp\"};\nt(global, \"npx\") -> {regular, \"image/vnd.net-fpx\"};\nt(global, \"NPX\") -> {regular, \"image/vnd.net-fpx\"};\nt(global, \"wdp\") -> {regular, \"image/vnd.ms-photo\"};\nt(global, \"WDP\") -> {regular, \"image/vnd.ms-photo\"};\nt(global, \"mdi\") -> {regular, \"image/vnd.ms-modi\"};\nt(global, \"MDI\") -> {regular, \"image/vnd.ms-modi\"};\nt(global, \"rlc\") -> {regular, \"image/vnd.fujixerox.edmics-rlc\"};\nt(global, \"RLC\") -> {regular, \"image/vnd.fujixerox.edmics-rlc\"};\nt(global, \"mmr\") -> {regular, \"image/vnd.fujixerox.edmics-mmr\"};\nt(global, \"MMR\") -> {regular, \"image/vnd.fujixerox.edmics-mmr\"};\nt(global, \"fst\") -> {regular, \"image/vnd.fst\"};\nt(global, \"FST\") -> {regular, \"image/vnd.fst\"};\nt(global, \"fpx\") -> {regular, \"image/vnd.fpx\"};\nt(global, \"FPX\") -> {regular, \"image/vnd.fpx\"};\nt(global, \"fbs\") -> {regular, \"image/vnd.fastbidsheet\"};\nt(global, \"FBS\") -> {regular, \"image/vnd.fastbidsheet\"};\nt(global, \"dxf\") -> {regular, \"image/vnd.dxf\"};\nt(global, \"DXF\") -> {regular, \"image/vnd.dxf\"};\nt(global, \"dwg\") -> {regular, \"image/vnd.dwg\"};\nt(global, \"DWG\") -> {regular, \"image/vnd.dwg\"};\nt(global, \"sub\") -> {regular, \"text/vnd.dvb.subtitle\"};\nt(global, \"SUB\") -> {regular, \"text/vnd.dvb.subtitle\"};\nt(global, \"djv\") -> {regular, \"image/vnd.djvu\"};\nt(global, \"DJV\") -> {regular, \"image/vnd.djvu\"};\nt(global, \"djvu\") -> {regular, \"image/vnd.djvu\"};\nt(global, \"DJVU\") -> {regular, \"image/vnd.djvu\"};\nt(global, \"uvvg\") -> {regular, \"image/vnd.dece.graphic\"};\nt(global, \"UVVG\") -> {regular, \"image/vnd.dece.graphic\"};\nt(global, \"uvg\") -> {regular, \"image/vnd.dece.graphic\"};\nt(global, \"UVG\") -> {regular, \"image/vnd.dece.graphic\"};\nt(global, \"uvvi\") -> {regular, \"image/vnd.dece.graphic\"};\nt(global, \"UVVI\") -> {regular, \"image/vnd.dece.graphic\"};\nt(global, \"uvi\") -> {regular, \"image/vnd.dece.graphic\"};\nt(global, \"UVI\") -> {regular, \"image/vnd.dece.graphic\"};\nt(global, \"psd\") -> {regular, \"image/vnd.adobe.photoshop\"};\nt(global, \"PSD\") -> {regular, \"image/vnd.adobe.photoshop\"};\nt(global, \"tif\") -> {regular, \"image/tiff\"};\nt(global, \"TIF\") -> {regular, \"image/tiff\"};\nt(global, \"tiff\") -> {regular, \"image/tiff\"};\nt(global, \"TIFF\") -> {regular, \"image/tiff\"};\nt(global, \"svgz\") -> {regular, \"image/svg+xml\"};\nt(global, \"SVGZ\") -> {regular, \"image/svg+xml\"};\nt(global, \"svg\") -> {regular, \"image/svg+xml\"};\nt(global, \"SVG\") -> {regular, \"image/svg+xml\"};\nt(global, \"sgi\") -> {regular, \"image/sgi\"};\nt(global, \"SGI\") -> {regular, \"image/sgi\"};\nt(global, \"btif\") -> {regular, \"image/prs.btif\"};\nt(global, \"BTIF\") -> {regular, \"image/prs.btif\"};\nt(global, \"png\") -> {regular, \"image/png\"};\nt(global, \"PNG\") -> {regular, \"image/png\"};\nt(global, \"ktx\") -> {regular, \"image/ktx\"};\nt(global, \"KTX\") -> {regular, \"image/ktx\"};\nt(global, \"jpe\") -> {regular, \"image/jpeg\"};\nt(global, \"JPE\") -> {regular, \"image/jpeg\"};\nt(global, \"jpg\") -> {regular, \"image/jpeg\"};\nt(global, \"JPG\") -> {regular, \"image/jpeg\"};\nt(global, \"jpeg\") -> {regular, \"image/jpeg\"};\nt(global, \"JPEG\") -> {regular, \"image/jpeg\"};\nt(global, \"ief\") -> {regular, \"image/ief\"};\nt(global, \"IEF\") -> {regular, \"image/ief\"};\nt(global, \"gif\") -> {regular, \"image/gif\"};\nt(global, \"GIF\") -> {regular, \"image/gif\"};\nt(global, \"g3\") -> {regular, \"image/g3fax\"};\nt(global, \"G3\") -> {regular, \"image/g3fax\"};\nt(global, \"cgm\") -> {regular, \"image/cgm\"};\nt(global, \"CGM\") -> {regular, \"image/cgm\"};\nt(global, \"bmp\") -> {regular, \"image/bmp\"};\nt(global, \"BMP\") -> {regular, \"image/bmp\"};\nt(global, \"xyz\") -> {regular, \"chemical/x-xyz\"};\nt(global, \"XYZ\") -> {regular, \"chemical/x-xyz\"};\nt(global, \"csml\") -> {regular, \"chemical/x-csml\"};\nt(global, \"CSML\") -> {regular, \"chemical/x-csml\"};\nt(global, \"cml\") -> {regular, \"chemical/x-cml\"};\nt(global, \"CML\") -> {regular, \"chemical/x-cml\"};\nt(global, \"cmdf\") -> {regular, \"chemical/x-cmdf\"};\nt(global, \"CMDF\") -> {regular, \"chemical/x-cmdf\"};\nt(global, \"cif\") -> {regular, \"chemical/x-cif\"};\nt(global, \"CIF\") -> {regular, \"chemical/x-cif\"};\nt(global, \"cdx\") -> {regular, \"chemical/x-cdx\"};\nt(global, \"CDX\") -> {regular, \"chemical/x-cdx\"};\nt(global, \"xm\") -> {regular, \"audio/xm\"};\nt(global, \"XM\") -> {regular, \"audio/xm\"};\nt(global, \"wav\") -> {regular, \"audio/x-wav\"};\nt(global, \"WAV\") -> {regular, \"audio/x-wav\"};\nt(global, \"rmp\") -> {regular, \"audio/x-pn-realaudio-plugin\"};\nt(global, \"RMP\") -> {regular, \"audio/x-pn-realaudio-plugin\"};\nt(global, \"ra\") -> {regular, \"audio/x-realaudio\"};\nt(global, \"RA\") -> {regular, \"audio/x-realaudio\"};\nt(global, \"ram\") -> {regular, \"audio/x-pn-realaudio\"};\nt(global, \"RAM\") -> {regular, \"audio/x-pn-realaudio\"};\nt(global, \"wma\") -> {regular, \"audio/x-ms-wma\"};\nt(global, \"WMA\") -> {regular, \"audio/x-ms-wma\"};\nt(global, \"wax\") -> {regular, \"audio/x-ms-wax\"};\nt(global, \"WAX\") -> {regular, \"audio/x-ms-wax\"};\nt(global, \"m3u\") -> {regular, \"audio/x-mpegurl\"};\nt(global, \"M3U\") -> {regular, \"audio/x-mpegurl\"};\nt(global, \"mka\") -> {regular, \"audio/x-matroska\"};\nt(global, \"MKA\") -> {regular, \"audio/x-matroska\"};\nt(global, \"flac\") -> {regular, \"audio/x-flac\"};\nt(global, \"FLAC\") -> {regular, \"audio/x-flac\"};\nt(global, \"caf\") -> {regular, \"audio/x-caf\"};\nt(global, \"CAF\") -> {regular, \"audio/x-caf\"};\nt(global, \"aifc\") -> {regular, \"audio/x-aiff\"};\nt(global, \"AIFC\") -> {regular, \"audio/x-aiff\"};\nt(global, \"aiff\") -> {regular, \"audio/x-aiff\"};\nt(global, \"AIFF\") -> {regular, \"audio/x-aiff\"};\nt(global, \"aif\") -> {regular, \"audio/x-aiff\"};\nt(global, \"AIF\") -> {regular, \"audio/x-aiff\"};\nt(global, \"aac\") -> {regular, \"audio/x-aac\"};\nt(global, \"AAC\") -> {regular, \"audio/x-aac\"};\nt(global, \"weba\") -> {regular, \"audio/webm\"};\nt(global, \"WEBA\") -> {regular, \"audio/webm\"};\nt(global, \"rip\") -> {regular, \"audio/vnd.rip\"};\nt(global, \"RIP\") -> {regular, \"audio/vnd.rip\"};\nt(global, \"ecelp9600\") -> {regular, \"audio/vnd.nuera.ecelp9600\"};\nt(global, \"ECELP9600\") -> {regular, \"audio/vnd.nuera.ecelp9600\"};\nt(global, \"ecelp7470\") -> {regular, \"audio/vnd.nuera.ecelp7470\"};\nt(global, \"ECELP7470\") -> {regular, \"audio/vnd.nuera.ecelp7470\"};\nt(global, \"ecelp4800\") -> {regular, \"audio/vnd.nuera.ecelp4800\"};\nt(global, \"ECELP4800\") -> {regular, \"audio/vnd.nuera.ecelp4800\"};\nt(global, \"pya\") -> {regular, \"audio/vnd.ms-playready.media.pya\"};\nt(global, \"PYA\") -> {regular, \"audio/vnd.ms-playready.media.pya\"};\nt(global, \"lvp\") -> {regular, \"audio/vnd.lucent.voice\"};\nt(global, \"LVP\") -> {regular, \"audio/vnd.lucent.voice\"};\nt(global, \"dtshd\") -> {regular, \"audio/vnd.dts.hd\"};\nt(global, \"DTSHD\") -> {regular, \"audio/vnd.dts.hd\"};\nt(global, \"dts\") -> {regular, \"audio/vnd.dts\"};\nt(global, \"DTS\") -> {regular, \"audio/vnd.dts\"};\nt(global, \"dra\") -> {regular, \"audio/vnd.dra\"};\nt(global, \"DRA\") -> {regular, \"audio/vnd.dra\"};\nt(global, \"eol\") -> {regular, \"audio/vnd.digital-winds\"};\nt(global, \"EOL\") -> {regular, \"audio/vnd.digital-winds\"};\nt(global, \"uvva\") -> {regular, \"audio/vnd.dece.audio\"};\nt(global, \"UVVA\") -> {regular, \"audio/vnd.dece.audio\"};\nt(global, \"uva\") -> {regular, \"audio/vnd.dece.audio\"};\nt(global, \"UVA\") -> {regular, \"audio/vnd.dece.audio\"};\nt(global, \"sil\") -> {regular, \"audio/silk\"};\nt(global, \"SIL\") -> {regular, \"audio/silk\"};\nt(global, \"s3m\") -> {regular, \"audio/s3m\"};\nt(global, \"S3M\") -> {regular, \"audio/s3m\"};\nt(global, \"spx\") -> {regular, \"audio/ogg\"};\nt(global, \"SPX\") -> {regular, \"audio/ogg\"};\nt(global, \"ogg\") -> {regular, \"audio/ogg\"};\nt(global, \"OGG\") -> {regular, \"audio/ogg\"};\nt(global, \"oga\") -> {regular, \"audio/ogg\"};\nt(global, \"OGA\") -> {regular, \"audio/ogg\"};\nt(global, \"m3a\") -> {regular, \"audio/mpeg\"};\nt(global, \"M3A\") -> {regular, \"audio/mpeg\"};\nt(global, \"m2a\") -> {regular, \"audio/mpeg\"};\nt(global, \"M2A\") -> {regular, \"audio/mpeg\"};\nt(global, \"mp3\") -> {regular, \"audio/mpeg\"};\nt(global, \"MP3\") -> {regular, \"audio/mpeg\"};\nt(global, \"mp2a\") -> {regular, \"audio/mpeg\"};\nt(global, \"MP2A\") -> {regular, \"audio/mpeg\"};\nt(global, \"mp2\") -> {regular, \"audio/mpeg\"};\nt(global, \"MP2\") -> {regular, \"audio/mpeg\"};\nt(global, \"mpga\") -> {regular, \"audio/mpeg\"};\nt(global, \"MPGA\") -> {regular, \"audio/mpeg\"};\nt(global, \"mp4a\") -> {regular, \"audio/mp4\"};\nt(global, \"MP4A\") -> {regular, \"audio/mp4\"};\nt(global, \"rmi\") -> {regular, \"audio/midi\"};\nt(global, \"RMI\") -> {regular, \"audio/midi\"};\nt(global, \"kar\") -> {regular, \"audio/midi\"};\nt(global, \"KAR\") -> {regular, \"audio/midi\"};\nt(global, \"midi\") -> {regular, \"audio/midi\"};\nt(global, \"MIDI\") -> {regular, \"audio/midi\"};\nt(global, \"mid\") -> {regular, \"audio/midi\"};\nt(global, \"MID\") -> {regular, \"audio/midi\"};\nt(global, \"snd\") -> {regular, \"audio/basic\"};\nt(global, \"SND\") -> {regular, \"audio/basic\"};\nt(global, \"au\") -> {regular, \"audio/basic\"};\nt(global, \"AU\") -> {regular, \"audio/basic\"};\nt(global, \"adp\") -> {regular, \"audio/adpcm\"};\nt(global, \"ADP\") -> {regular, \"audio/adpcm\"};\nt(global, \"zip\") -> {regular, \"application/zip\"};\nt(global, \"ZIP\") -> {regular, \"application/zip\"};\nt(global, \"yin\") -> {regular, \"application/yin+xml\"};\nt(global, \"YIN\") -> {regular, \"application/yin+xml\"};\nt(global, \"yang\") -> {regular, \"application/yang\"};\nt(global, \"YANG\") -> {regular, \"application/yang\"};\nt(global, \"xvm\") -> {regular, \"application/xv+xml\"};\nt(global, \"XVM\") -> {regular, \"application/xv+xml\"};\nt(global, \"xvml\") -> {regular, \"application/xv+xml\"};\nt(global, \"XVML\") -> {regular, \"application/xv+xml\"};\nt(global, \"xhvml\") -> {regular, \"application/xv+xml\"};\nt(global, \"XHVML\") -> {regular, \"application/xv+xml\"};\nt(global, \"mxml\") -> {regular, \"application/xv+xml\"};\nt(global, \"MXML\") -> {regular, \"application/xv+xml\"};\nt(global, \"xspf\") -> {regular, \"application/xspf+xml\"};\nt(global, \"XSPF\") -> {regular, \"application/xspf+xml\"};\nt(global, \"xslt\") -> {regular, \"application/xslt+xml\"};\nt(global, \"XSLT\") -> {regular, \"application/xslt+xml\"};\nt(global, \"xpl\") -> {regular, \"application/xproc+xml\"};\nt(global, \"XPL\") -> {regular, \"application/xproc+xml\"};\nt(global, \"xop\") -> {regular, \"application/xop+xml\"};\nt(global, \"XOP\") -> {regular, \"application/xop+xml\"};\nt(global, \"dtd\") -> {regular, \"application/xml-dtd\"};\nt(global, \"DTD\") -> {regular, \"application/xml-dtd\"};\nt(global, \"xsl\") -> {regular, \"text/xml\"};\nt(global, \"XSL\") -> {regular, \"text/xml\"};\nt(global, \"xml\") -> {regular, \"text/xml\"};\nt(global, \"XML\") -> {regular, \"text/xml\"};\nt(global, \"xht\") -> {regular, \"application/xhtml+xml\"};\nt(global, \"XHT\") -> {regular, \"application/xhtml+xml\"};\nt(global, \"xhtml\") -> {regular, \"application/xhtml+xml\"};\nt(global, \"XHTML\") -> {regular, \"application/xhtml+xml\"};\nt(global, \"xenc\") -> {regular, \"application/xenc+xml\"};\nt(global, \"XENC\") -> {regular, \"application/xenc+xml\"};\nt(global, \"xdf\") -> {regular, \"application/xcap-diff+xml\"};\nt(global, \"XDF\") -> {regular, \"application/xcap-diff+xml\"};\nt(global, \"xaml\") -> {regular, \"application/xaml+xml\"};\nt(global, \"XAML\") -> {regular, \"application/xaml+xml\"};\nt(global, \"z8\") -> {regular, \"application/x-zmachine\"};\nt(global, \"Z8\") -> {regular, \"application/x-zmachine\"};\nt(global, \"z7\") -> {regular, \"application/x-zmachine\"};\nt(global, \"Z7\") -> {regular, \"application/x-zmachine\"};\nt(global, \"z6\") -> {regular, \"application/x-zmachine\"};\nt(global, \"Z6\") -> {regular, \"application/x-zmachine\"};\nt(global, \"z5\") -> {regular, \"application/x-zmachine\"};\nt(global, \"Z5\") -> {regular, \"application/x-zmachine\"};\nt(global, \"z4\") -> {regular, \"application/x-zmachine\"};\nt(global, \"Z4\") -> {regular, \"application/x-zmachine\"};\nt(global, \"z3\") -> {regular, \"application/x-zmachine\"};\nt(global, \"Z3\") -> {regular, \"application/x-zmachine\"};\nt(global, \"z2\") -> {regular, \"application/x-zmachine\"};\nt(global, \"Z2\") -> {regular, \"application/x-zmachine\"};\nt(global, \"z1\") -> {regular, \"application/x-zmachine\"};\nt(global, \"Z1\") -> {regular, \"application/x-zmachine\"};\nt(global, \"xz\") -> {regular, \"application/x-xz\"};\nt(global, \"XZ\") -> {regular, \"application/x-xz\"};\nt(global, \"xpi\") -> {regular, \"application/x-xpinstall\"};\nt(global, \"XPI\") -> {regular, \"application/x-xpinstall\"};\nt(global, \"xlf\") -> {regular, \"application/x-xliff+xml\"};\nt(global, \"XLF\") -> {regular, \"application/x-xliff+xml\"};\nt(global, \"fig\") -> {regular, \"application/x-xfig\"};\nt(global, \"FIG\") -> {regular, \"application/x-xfig\"};\nt(global, \"crt\") -> {regular, \"application/x-x509-ca-cert\"};\nt(global, \"CRT\") -> {regular, \"application/x-x509-ca-cert\"};\nt(global, \"der\") -> {regular, \"application/x-x509-ca-cert\"};\nt(global, \"DER\") -> {regular, \"application/x-x509-ca-cert\"};\nt(global, \"src\") -> {regular, \"application/x-wais-source\"};\nt(global, \"SRC\") -> {regular, \"application/x-wais-source\"};\nt(global, \"ustar\") -> {regular, \"application/x-ustar\"};\nt(global, \"USTAR\") -> {regular, \"application/x-ustar\"};\nt(global, \"ms\") -> {regular, \"text/troff\"};\nt(global, \"MS\") -> {regular, \"text/troff\"};\nt(global, \"obj\") -> {regular, \"application/x-tgif\"};\nt(global, \"OBJ\") -> {regular, \"application/x-tgif\"};\nt(global, \"texi\") -> {regular, \"application/x-texinfo\"};\nt(global, \"TEXI\") -> {regular, \"application/x-texinfo\"};\nt(global, \"texinfo\") -> {regular, \"application/x-texinfo\"};\nt(global, \"TEXINFO\") -> {regular, \"application/x-texinfo\"};\nt(global, \"tfm\") -> {regular, \"application/x-tex-tfm\"};\nt(global, \"TFM\") -> {regular, \"application/x-tex-tfm\"};\nt(global, \"tex\") -> {regular, \"application/x-tex\"};\nt(global, \"TEX\") -> {regular, \"application/x-tex\"};\nt(global, \"tcl\") -> {regular, \"application/x-tcl\"};\nt(global, \"TCL\") -> {regular, \"application/x-tcl\"};\nt(global, \"tar\") -> {regular, \"application/x-tar\"};\nt(global, \"TAR\") -> {regular, \"application/x-tar\"};\nt(global, \"gam\") -> {regular, \"application/x-tads\"};\nt(global, \"GAM\") -> {regular, \"application/x-tads\"};\nt(global, \"t3\") -> {regular, \"application/x-t3vm-image\"};\nt(global, \"T3\") -> {regular, \"application/x-t3vm-image\"};\nt(global, \"sv4crc\") -> {regular, \"application/x-sv4crc\"};\nt(global, \"SV4CRC\") -> {regular, \"application/x-sv4crc\"};\nt(global, \"sv4cpio\") -> {regular, \"application/x-sv4cpio\"};\nt(global, \"SV4CPIO\") -> {regular, \"application/x-sv4cpio\"};\nt(global, \"srt\") -> {regular, \"application/x-subrip\"};\nt(global, \"SRT\") -> {regular, \"application/x-subrip\"};\nt(global, \"sitx\") -> {regular, \"application/x-stuffitx\"};\nt(global, \"SITX\") -> {regular, \"application/x-stuffitx\"};\nt(global, \"sit\") -> {regular, \"application/x-stuffit\"};\nt(global, \"SIT\") -> {regular, \"application/x-stuffit\"};\nt(global, \"sql\") -> {regular, \"application/x-sql\"};\nt(global, \"SQL\") -> {regular, \"application/x-sql\"};\nt(global, \"xap\") -> {regular, \"application/x-silverlight-app\"};\nt(global, \"XAP\") -> {regular, \"application/x-silverlight-app\"};\nt(global, \"swf\") -> {regular, \"application/x-shockwave-flash\"};\nt(global, \"SWF\") -> {regular, \"application/x-shockwave-flash\"};\nt(global, \"shar\") -> {regular, \"application/x-shar\"};\nt(global, \"SHAR\") -> {regular, \"application/x-shar\"};\nt(global, \"sh\") -> {regular, \"application/x-sh\"};\nt(global, \"SH\") -> {regular, \"application/x-sh\"};\nt(global, \"rpm\") -> {regular, \"application/x-rpm\"};\nt(global, \"RPM\") -> {regular, \"application/x-rpm\"};\nt(global, \"ris\") -> {regular, \"application/x-research-info-systems\"};\nt(global, \"RIS\") -> {regular, \"application/x-research-info-systems\"};\nt(global, \"rar\") -> {regular, \"application/x-rar-compressed\"};\nt(global, \"RAR\") -> {regular, \"application/x-rar-compressed\"};\nt(global, \"p7r\") -> {regular, \"application/x-pkcs7-certreqresp\"};\nt(global, \"P7R\") -> {regular, \"application/x-pkcs7-certreqresp\"};\nt(global, \"spc\") -> {regular, \"application/x-pkcs7-certificates\"};\nt(global, \"SPC\") -> {regular, \"application/x-pkcs7-certificates\"};\nt(global, \"p7b\") -> {regular, \"application/x-pkcs7-certificates\"};\nt(global, \"P7B\") -> {regular, \"application/x-pkcs7-certificates\"};\nt(global, \"pfx\") -> {regular, \"application/x-pkcs12\"};\nt(global, \"PFX\") -> {regular, \"application/x-pkcs12\"};\nt(global, \"p12\") -> {regular, \"application/x-pkcs12\"};\nt(global, \"P12\") -> {regular, \"application/x-pkcs12\"};\nt(global, \"nzb\") -> {regular, \"application/x-nzb\"};\nt(global, \"NZB\") -> {regular, \"application/x-nzb\"};\nt(global, \"pac\") -> {regular, \"application/x-ns-proxy-autoconfig\"};\nt(global, \"PAC\") -> {regular, \"application/x-ns-proxy-autoconfig\"};\nt(global, \"cdf\") -> {regular, \"application/x-netcdf\"};\nt(global, \"CDF\") -> {regular, \"application/x-netcdf\"};\nt(global, \"nc\") -> {regular, \"application/x-netcdf\"};\nt(global, \"NC\") -> {regular, \"application/x-netcdf\"};\nt(global, \"wri\") -> {regular, \"application/x-mswrite\"};\nt(global, \"WRI\") -> {regular, \"application/x-mswrite\"};\nt(global, \"trm\") -> {regular, \"application/x-msterminal\"};\nt(global, \"TRM\") -> {regular, \"application/x-msterminal\"};\nt(global, \"scd\") -> {regular, \"application/x-msschedule\"};\nt(global, \"SCD\") -> {regular, \"application/x-msschedule\"};\nt(global, \"pub\") -> {regular, \"application/x-mspublisher\"};\nt(global, \"PUB\") -> {regular, \"application/x-mspublisher\"};\nt(global, \"mny\") -> {regular, \"application/x-msmoney\"};\nt(global, \"MNY\") -> {regular, \"application/x-msmoney\"};\nt(global, \"emz\") -> {regular, \"application/x-msmetafile\"};\nt(global, \"EMZ\") -> {regular, \"application/x-msmetafile\"};\nt(global, \"emf\") -> {regular, \"application/x-msmetafile\"};\nt(global, \"EMF\") -> {regular, \"application/x-msmetafile\"};\nt(global, \"wmf\") -> {regular, \"application/x-msmetafile\"};\nt(global, \"WMF\") -> {regular, \"application/x-msmetafile\"};\nt(global, \"m14\") -> {regular, \"application/x-msmediaview\"};\nt(global, \"M14\") -> {regular, \"application/x-msmediaview\"};\nt(global, \"m13\") -> {regular, \"application/x-msmediaview\"};\nt(global, \"M13\") -> {regular, \"application/x-msmediaview\"};\nt(global, \"mvb\") -> {regular, \"application/x-msmediaview\"};\nt(global, \"MVB\") -> {regular, \"application/x-msmediaview\"};\nt(global, \"msi\") -> {regular, \"application/x-msdownload\"};\nt(global, \"MSI\") -> {regular, \"application/x-msdownload\"};\nt(global, \"bat\") -> {regular, \"application/x-msdownload\"};\nt(global, \"BAT\") -> {regular, \"application/x-msdownload\"};\nt(global, \"com\") -> {regular, \"application/x-msdownload\"};\nt(global, \"COM\") -> {regular, \"application/x-msdownload\"};\nt(global, \"dll\") -> {regular, \"application/x-msdownload\"};\nt(global, \"DLL\") -> {regular, \"application/x-msdownload\"};\nt(global, \"exe\") -> {regular, \"application/x-msdownload\"};\nt(global, \"EXE\") -> {regular, \"application/x-msdownload\"};\nt(global, \"clp\") -> {regular, \"application/x-msclip\"};\nt(global, \"CLP\") -> {regular, \"application/x-msclip\"};\nt(global, \"crd\") -> {regular, \"application/x-mscardfile\"};\nt(global, \"CRD\") -> {regular, \"application/x-mscardfile\"};\nt(global, \"obd\") -> {regular, \"application/x-msbinder\"};\nt(global, \"OBD\") -> {regular, \"application/x-msbinder\"};\nt(global, \"mdb\") -> {regular, \"application/x-msaccess\"};\nt(global, \"MDB\") -> {regular, \"application/x-msaccess\"};\nt(global, \"xbap\") -> {regular, \"application/x-ms-xbap\"};\nt(global, \"XBAP\") -> {regular, \"application/x-ms-xbap\"};\nt(global, \"wmz\") -> {regular, \"application/x-msmetafile\"};\nt(global, \"WMZ\") -> {regular, \"application/x-msmetafile\"};\nt(global, \"wmd\") -> {regular, \"application/x-ms-wmd\"};\nt(global, \"WMD\") -> {regular, \"application/x-ms-wmd\"};\nt(global, \"lnk\") -> {regular, \"application/x-ms-shortcut\"};\nt(global, \"LNK\") -> {regular, \"application/x-ms-shortcut\"};\nt(global, \"application\") -> {regular, \"application/x-ms-application\"};\nt(global, \"APPLICATION\") -> {regular, \"application/x-ms-application\"};\nt(global, \"mobi\") -> {regular, \"application/x-mobipocket-ebook\"};\nt(global, \"MOBI\") -> {regular, \"application/x-mobipocket-ebook\"};\nt(global, \"prc\") -> {regular, \"application/x-mobipocket-ebook\"};\nt(global, \"PRC\") -> {regular, \"application/x-mobipocket-ebook\"};\nt(global, \"mie\") -> {regular, \"application/x-mie\"};\nt(global, \"MIE\") -> {regular, \"application/x-mie\"};\nt(global, \"lha\") -> {regular, \"application/x-lzh-compressed\"};\nt(global, \"LHA\") -> {regular, \"application/x-lzh-compressed\"};\nt(global, \"lzh\") -> {regular, \"application/x-lzh-compressed\"};\nt(global, \"LZH\") -> {regular, \"application/x-lzh-compressed\"};\nt(global, \"latex\") -> {regular, \"application/x-latex\"};\nt(global, \"LATEX\") -> {regular, \"application/x-latex\"};\nt(global, \"kil\") -> {regular, \"application/x-killustrator\"};\nt(global, \"KIL\") -> {regular, \"application/x-killustrator\"};\nt(global, \"jnlp\") -> {regular, \"application/x-java-jnlp-file\"};\nt(global, \"JNLP\") -> {regular, \"application/x-java-jnlp-file\"};\nt(global, \"iso\") -> {regular, \"application/x-iso9660-image\"};\nt(global, \"ISO\") -> {regular, \"application/x-iso9660-image\"};\nt(global, \"install\") -> {regular, \"application/x-install-instructions\"};\nt(global, \"INSTALL\") -> {regular, \"application/x-install-instructions\"};\nt(global, \"hdf\") -> {regular, \"application/x-hdf\"};\nt(global, \"HDF\") -> {regular, \"application/x-hdf\"};\nt(global, \"tgz\") -> {regular, \"application/x-gzip\"};\nt(global, \"TGZ\") -> {regular, \"application/x-gzip\"};\nt(global, \"gz\") -> {regular, \"application/x-gzip\"};\nt(global, \"GZ\") -> {regular, \"application/x-gzip\"};\nt(global, \"gtar\") -> {regular, \"application/x-gtar\"};\nt(global, \"GTAR\") -> {regular, \"application/x-gtar\"};\nt(global, \"gramps\") -> {regular, \"application/x-gramps-xml\"};\nt(global, \"GRAMPS\") -> {regular, \"application/x-gramps-xml\"};\nt(global, \"gnumeric\") -> {regular, \"application/x-gnumeric\"};\nt(global, \"GNUMERIC\") -> {regular, \"application/x-gnumeric\"};\nt(global, \"ulx\") -> {regular, \"application/x-glulx\"};\nt(global, \"ULX\") -> {regular, \"application/x-glulx\"};\nt(global, \"gca\") -> {regular, \"application/x-gca-compressed\"};\nt(global, \"GCA\") -> {regular, \"application/x-gca-compressed\"};\nt(global, \"spl\") -> {regular, \"application/x-futuresplash\"};\nt(global, \"SPL\") -> {regular, \"application/x-futuresplash\"};\nt(global, \"arc\") -> {regular, \"application/x-freearc\"};\nt(global, \"ARC\") -> {regular, \"application/x-freearc\"};\nt(global, \"afm\") -> {regular, \"application/x-font-type1\"};\nt(global, \"AFM\") -> {regular, \"application/x-font-type1\"};\nt(global, \"pfm\") -> {regular, \"application/x-font-type1\"};\nt(global, \"PFM\") -> {regular, \"application/x-font-type1\"};\nt(global, \"pfb\") -> {regular, \"application/x-font-type1\"};\nt(global, \"PFB\") -> {regular, \"application/x-font-type1\"};\nt(global, \"pfa\") -> {regular, \"application/x-font-type1\"};\nt(global, \"PFA\") -> {regular, \"application/x-font-type1\"};\nt(global, \"ttc\") -> {regular, \"application/x-font-ttf\"};\nt(global, \"TTC\") -> {regular, \"application/x-font-ttf\"};\nt(global, \"ttf\") -> {regular, \"application/x-font-ttf\"};\nt(global, \"TTF\") -> {regular, \"application/x-font-ttf\"};\nt(global, \"snf\") -> {regular, \"application/x-font-snf\"};\nt(global, \"SNF\") -> {regular, \"application/x-font-snf\"};\nt(global, \"pcf\") -> {regular, \"application/x-font-pcf\"};\nt(global, \"PCF\") -> {regular, \"application/x-font-pcf\"};\nt(global, \"otf\") -> {regular, \"application/x-font-otf\"};\nt(global, \"OTF\") -> {regular, \"application/x-font-otf\"};\nt(global, \"psf\") -> {regular, \"application/x-font-linux-psf\"};\nt(global, \"PSF\") -> {regular, \"application/x-font-linux-psf\"};\nt(global, \"gsf\") -> {regular, \"application/x-font-ghostscript\"};\nt(global, \"GSF\") -> {regular, \"application/x-font-ghostscript\"};\nt(global, \"bdf\") -> {regular, \"application/x-font-bdf\"};\nt(global, \"BDF\") -> {regular, \"application/x-font-bdf\"};\nt(global, \"eva\") -> {regular, \"application/x-eva\"};\nt(global, \"EVA\") -> {regular, \"application/x-eva\"};\nt(global, \"evy\") -> {regular, \"application/x-envoy\"};\nt(global, \"EVY\") -> {regular, \"application/x-envoy\"};\nt(global, \"dvi\") -> {regular, \"application/x-dvi\"};\nt(global, \"DVI\") -> {regular, \"application/x-dvi\"};\nt(global, \"res\") -> {regular, \"application/x-dtbresource+xml\"};\nt(global, \"RES\") -> {regular, \"application/x-dtbresource+xml\"};\nt(global, \"dtb\") -> {regular, \"application/x-dtbook+xml\"};\nt(global, \"DTB\") -> {regular, \"application/x-dtbook+xml\"};\nt(global, \"ncx\") -> {regular, \"application/x-dtbncx+xml\"};\nt(global, \"NCX\") -> {regular, \"application/x-dtbncx+xml\"};\nt(global, \"wad\") -> {regular, \"application/x-doom\"};\nt(global, \"WAD\") -> {regular, \"application/x-doom\"};\nt(global, \"swa\") -> {regular, \"application/x-director\"};\nt(global, \"SWA\") -> {regular, \"application/x-director\"};\nt(global, \"fgd\") -> {regular, \"application/x-director\"};\nt(global, \"FGD\") -> {regular, \"application/x-director\"};\nt(global, \"w3d\") -> {regular, \"application/x-director\"};\nt(global, \"W3D\") -> {regular, \"application/x-director\"};\nt(global, \"cxt\") -> {regular, \"application/x-director\"};\nt(global, \"CXT\") -> {regular, \"application/x-director\"};\nt(global, \"cct\") -> {regular, \"application/x-director\"};\nt(global, \"CCT\") -> {regular, \"application/x-director\"};\nt(global, \"cst\") -> {regular, \"application/x-director\"};\nt(global, \"CST\") -> {regular, \"application/x-director\"};\nt(global, \"dxr\") -> {regular, \"application/x-director\"};\nt(global, \"DXR\") -> {regular, \"application/x-director\"};\nt(global, \"dcr\") -> {regular, \"application/x-director\"};\nt(global, \"DCR\") -> {regular, \"application/x-director\"};\nt(global, \"dir\") -> {regular, \"application/x-director\"};\nt(global, \"DIR\") -> {regular, \"application/x-director\"};\nt(global, \"dgc\") -> {regular, \"application/x-dgc-compressed\"};\nt(global, \"DGC\") -> {regular, \"application/x-dgc-compressed\"};\nt(global, \"udeb\") -> {regular, \"application/x-debian-package\"};\nt(global, \"UDEB\") -> {regular, \"application/x-debian-package\"};\nt(global, \"deb\") -> {regular, \"application/x-debian-package\"};\nt(global, \"DEB\") -> {regular, \"application/x-debian-package\"};\nt(global, \"csh\") -> {regular, \"application/x-csh\"};\nt(global, \"CSH\") -> {regular, \"application/x-csh\"};\nt(global, \"cpio\") -> {regular, \"application/x-cpio\"};\nt(global, \"CPIO\") -> {regular, \"application/x-cpio\"};\nt(global, \"nsc\") -> {regular, \"application/x-conference\"};\nt(global, \"NSC\") -> {regular, \"application/x-conference\"};\nt(global, \"pgn\") -> {regular, \"application/x-chess-pgn\"};\nt(global, \"PGN\") -> {regular, \"application/x-chess-pgn\"};\nt(global, \"chat\") -> {regular, \"application/x-chat\"};\nt(global, \"CHAT\") -> {regular, \"application/x-chat\"};\nt(global, \"cfs\") -> {regular, \"application/x-cfs-compressed\"};\nt(global, \"CFS\") -> {regular, \"application/x-cfs-compressed\"};\nt(global, \"vcd\") -> {regular, \"application/x-cdlink\"};\nt(global, \"VCD\") -> {regular, \"application/x-cdlink\"};\nt(global, \"cb7\") -> {regular, \"application/x-cbr\"};\nt(global, \"CB7\") -> {regular, \"application/x-cbr\"};\nt(global, \"cbz\") -> {regular, \"application/x-cbr\"};\nt(global, \"CBZ\") -> {regular, \"application/x-cbr\"};\nt(global, \"cbt\") -> {regular, \"application/x-cbr\"};\nt(global, \"CBT\") -> {regular, \"application/x-cbr\"};\nt(global, \"cba\") -> {regular, \"application/x-cbr\"};\nt(global, \"CBA\") -> {regular, \"application/x-cbr\"};\nt(global, \"cbr\") -> {regular, \"application/x-cbr\"};\nt(global, \"CBR\") -> {regular, \"application/x-cbr\"};\nt(global, \"boz\") -> {regular, \"application/x-bzip2\"};\nt(global, \"BOZ\") -> {regular, \"application/x-bzip2\"};\nt(global, \"bz2\") -> {regular, \"application/x-bzip2\"};\nt(global, \"BZ2\") -> {regular, \"application/x-bzip2\"};\nt(global, \"bz\") -> {regular, \"application/x-bzip\"};\nt(global, \"BZ\") -> {regular, \"application/x-bzip\"};\nt(global, \"blorb\") -> {regular, \"application/x-blorb\"};\nt(global, \"BLORB\") -> {regular, \"application/x-blorb\"};\nt(global, \"blb\") -> {regular, \"application/x-blorb\"};\nt(global, \"BLB\") -> {regular, \"application/x-blorb\"};\nt(global, \"torrent\") -> {regular, \"application/x-bittorrent\"};\nt(global, \"TORRENT\") -> {regular, \"application/x-bittorrent\"};\nt(global, \"bcpio\") -> {regular, \"application/x-bcpio\"};\nt(global, \"BCPIO\") -> {regular, \"application/x-bcpio\"};\nt(global, \"aas\") -> {regular, \"application/x-authorware-seg\"};\nt(global, \"AAS\") -> {regular, \"application/x-authorware-seg\"};\nt(global, \"aam\") -> {regular, \"application/x-authorware-map\"};\nt(global, \"AAM\") -> {regular, \"application/x-authorware-map\"};\nt(global, \"vox\") -> {regular, \"application/x-authorware-bin\"};\nt(global, \"VOX\") -> {regular, \"application/x-authorware-bin\"};\nt(global, \"u32\") -> {regular, \"application/x-authorware-bin\"};\nt(global, \"U32\") -> {regular, \"application/x-authorware-bin\"};\nt(global, \"x32\") -> {regular, \"application/x-authorware-bin\"};\nt(global, \"X32\") -> {regular, \"application/x-authorware-bin\"};\nt(global, \"aab\") -> {regular, \"application/x-authorware-bin\"};\nt(global, \"AAB\") -> {regular, \"application/x-authorware-bin\"};\nt(global, \"dmg\") -> {regular, \"application/x-apple-diskimage\"};\nt(global, \"DMG\") -> {regular, \"application/x-apple-diskimage\"};\nt(global, \"ace\") -> {regular, \"application/x-ace-compressed\"};\nt(global, \"ACE\") -> {regular, \"application/x-ace-compressed\"};\nt(global, \"abw\") -> {regular, \"application/x-abiword\"};\nt(global, \"ABW\") -> {regular, \"application/x-abiword\"};\nt(global, \"7z\") -> {regular, \"application/x-7z-compressed\"};\nt(global, \"7Z\") -> {regular, \"application/x-7z-compressed\"};\nt(global, \"wspolicy\") -> {regular, \"application/wspolicy+xml\"};\nt(global, \"WSPOLICY\") -> {regular, \"application/wspolicy+xml\"};\nt(global, \"wsdl\") -> {regular, \"application/wsdl+xml\"};\nt(global, \"WSDL\") -> {regular, \"application/wsdl+xml\"};\nt(global, \"hlp\") -> {regular, \"application/winhlp\"};\nt(global, \"HLP\") -> {regular, \"application/winhlp\"};\nt(global, \"wgt\") -> {regular, \"application/widget\"};\nt(global, \"WGT\") -> {regular, \"application/widget\"};\nt(global, \"vxml\") -> {regular, \"application/voicexml+xml\"};\nt(global, \"VXML\") -> {regular, \"application/voicexml+xml\"};\nt(global, \"zaz\") -> {regular, \"application/vnd.zzazz.deck+xml\"};\nt(global, \"ZAZ\") -> {regular, \"application/vnd.zzazz.deck+xml\"};\nt(global, \"zirz\") -> {regular, \"application/vnd.zul\"};\nt(global, \"ZIRZ\") -> {regular, \"application/vnd.zul\"};\nt(global, \"zir\") -> {regular, \"application/vnd.zul\"};\nt(global, \"ZIR\") -> {regular, \"application/vnd.zul\"};\nt(global, \"cmp\") -> {regular, \"application/vnd.yellowriver-custom-menu\"};\nt(global, \"CMP\") -> {regular, \"application/vnd.yellowriver-custom-menu\"};\nt(global, \"spf\") -> {regular, \"application/vnd.yamaha.smaf-phrase\"};\nt(global, \"SPF\") -> {regular, \"application/vnd.yamaha.smaf-phrase\"};\nt(global, \"saf\") -> {regular, \"application/vnd.yamaha.smaf-audio\"};\nt(global, \"SAF\") -> {regular, \"application/vnd.yamaha.smaf-audio\"};\nt(global, \"osfpvg\") -> {regular, \"application/vnd.yamaha.openscoreformat.osfpvg+xml\"};\nt(global, \"OSFPVG\") -> {regular, \"application/vnd.yamaha.openscoreformat.osfpvg+xml\"};\nt(global, \"osf\") -> {regular, \"application/vnd.yamaha.openscoreformat\"};\nt(global, \"OSF\") -> {regular, \"application/vnd.yamaha.openscoreformat\"};\nt(global, \"hvp\") -> {regular, \"application/vnd.yamaha.hv-voice\"};\nt(global, \"HVP\") -> {regular, \"application/vnd.yamaha.hv-voice\"};\nt(global, \"hvs\") -> {regular, \"application/vnd.yamaha.hv-script\"};\nt(global, \"HVS\") -> {regular, \"application/vnd.yamaha.hv-script\"};\nt(global, \"hvd\") -> {regular, \"application/vnd.yamaha.hv-dic\"};\nt(global, \"HVD\") -> {regular, \"application/vnd.yamaha.hv-dic\"};\nt(global, \"xfdl\") -> {regular, \"application/vnd.xfdl\"};\nt(global, \"XFDL\") -> {regular, \"application/vnd.xfdl\"};\nt(global, \"xar\") -> {regular, \"application/vnd.xara\"};\nt(global, \"XAR\") -> {regular, \"application/vnd.xara\"};\nt(global, \"stf\") -> {regular, \"application/vnd.wt.stf\"};\nt(global, \"STF\") -> {regular, \"application/vnd.wt.stf\"};\nt(global, \"wqd\") -> {regular, \"application/vnd.wqd\"};\nt(global, \"WQD\") -> {regular, \"application/vnd.wqd\"};\nt(global, \"wpd\") -> {regular, \"application/vnd.wordperfect\"};\nt(global, \"WPD\") -> {regular, \"application/vnd.wordperfect\"};\nt(global, \"nbp\") -> {regular, \"application/vnd.wolfram.player\"};\nt(global, \"NBP\") -> {regular, \"application/vnd.wolfram.player\"};\nt(global, \"wtb\") -> {regular, \"application/vnd.webturbo\"};\nt(global, \"WTB\") -> {regular, \"application/vnd.webturbo\"};\nt(global, \"wmlsc\") -> {regular, \"application/vnd.wap.wmlscriptc\"};\nt(global, \"WMLSC\") -> {regular, \"application/vnd.wap.wmlscriptc\"};\nt(global, \"wmlc\") -> {regular, \"application/vnd.wap.wmlc\"};\nt(global, \"WMLC\") -> {regular, \"application/vnd.wap.wmlc\"};\nt(global, \"wbxml\") -> {regular, \"application/vnd.wap.wbxml\"};\nt(global, \"WBXML\") -> {regular, \"application/vnd.wap.wbxml\"};\nt(global, \"slc\") -> {regular, \"application/vnd.wap.slc\"};\nt(global, \"SLC\") -> {regular, \"application/vnd.wap.slc\"};\nt(global, \"sic\") -> {regular, \"application/vnd.wap.sic\"};\nt(global, \"SIC\") -> {regular, \"application/vnd.wap.sic\"};\nt(global, \"vsf\") -> {regular, \"application/vnd.vsf\"};\nt(global, \"VSF\") -> {regular, \"application/vnd.vsf\"};\nt(global, \"vis\") -> {regular, \"application/vnd.visionary\"};\nt(global, \"VIS\") -> {regular, \"application/vnd.visionary\"};\nt(global, \"vsw\") -> {regular, \"application/vnd.visio\"};\nt(global, \"VSW\") -> {regular, \"application/vnd.visio\"};\nt(global, \"vss\") -> {regular, \"application/vnd.visio\"};\nt(global, \"VSS\") -> {regular, \"application/vnd.visio\"};\nt(global, \"vst\") -> {regular, \"application/vnd.visio\"};\nt(global, \"VST\") -> {regular, \"application/vnd.visio\"};\nt(global, \"vsd\") -> {regular, \"application/vnd.visio\"};\nt(global, \"VSD\") -> {regular, \"application/vnd.visio\"};\nt(global, \"vcx\") -> {regular, \"application/vnd.vcx\"};\nt(global, \"VCX\") -> {regular, \"application/vnd.vcx\"};\nt(global, \"uoml\") -> {regular, \"application/vnd.uoml+xml\"};\nt(global, \"UOML\") -> {regular, \"application/vnd.uoml+xml\"};\nt(global, \"unityweb\") -> {regular, \"application/vnd.unity\"};\nt(global, \"UNITYWEB\") -> {regular, \"application/vnd.unity\"};\nt(global, \"umj\") -> {regular, \"application/vnd.umajin\"};\nt(global, \"UMJ\") -> {regular, \"application/vnd.umajin\"};\nt(global, \"utz\") -> {regular, \"application/vnd.uiq.theme\"};\nt(global, \"UTZ\") -> {regular, \"application/vnd.uiq.theme\"};\nt(global, \"ufdl\") -> {regular, \"application/vnd.ufdl\"};\nt(global, \"UFDL\") -> {regular, \"application/vnd.ufdl\"};\nt(global, \"ufd\") -> {regular, \"application/vnd.ufdl\"};\nt(global, \"UFD\") -> {regular, \"application/vnd.ufdl\"};\nt(global, \"tra\") -> {regular, \"application/vnd.trueapp\"};\nt(global, \"TRA\") -> {regular, \"application/vnd.trueapp\"};\nt(global, \"mxs\") -> {regular, \"application/vnd.triscape.mxs\"};\nt(global, \"MXS\") -> {regular, \"application/vnd.triscape.mxs\"};\nt(global, \"tpt\") -> {regular, \"application/vnd.trid.tpt\"};\nt(global, \"TPT\") -> {regular, \"application/vnd.trid.tpt\"};\nt(global, \"tmo\") -> {regular, \"application/vnd.tmobile-livetv\"};\nt(global, \"TMO\") -> {regular, \"application/vnd.tmobile-livetv\"};\nt(global, \"dmp\") -> {regular, \"application/vnd.tcpdump.pcap\"};\nt(global, \"DMP\") -> {regular, \"application/vnd.tcpdump.pcap\"};\nt(global, \"cap\") -> {regular, \"application/vnd.tcpdump.pcap\"};\nt(global, \"CAP\") -> {regular, \"application/vnd.tcpdump.pcap\"};\nt(global, \"pcap\") -> {regular, \"application/vnd.tcpdump.pcap\"};\nt(global, \"PCAP\") -> {regular, \"application/vnd.tcpdump.pcap\"};\nt(global, \"tao\") -> {regular, \"application/vnd.tao.intent-module-archive\"};\nt(global, \"TAO\") -> {regular, \"application/vnd.tao.intent-module-archive\"};\nt(global, \"xdm\") -> {regular, \"application/vnd.syncml.dm+xml\"};\nt(global, \"XDM\") -> {regular, \"application/vnd.syncml.dm+xml\"};\nt(global, \"bdm\") -> {regular, \"application/vnd.syncml.dm+wbxml\"};\nt(global, \"BDM\") -> {regular, \"application/vnd.syncml.dm+wbxml\"};\nt(global, \"xsm\") -> {regular, \"application/vnd.syncml+xml\"};\nt(global, \"XSM\") -> {regular, \"application/vnd.syncml+xml\"};\nt(global, \"sisx\") -> {regular, \"application/vnd.symbian.install\"};\nt(global, \"SISX\") -> {regular, \"application/vnd.symbian.install\"};\nt(global, \"sis\") -> {regular, \"application/vnd.symbian.install\"};\nt(global, \"SIS\") -> {regular, \"application/vnd.symbian.install\"};\nt(global, \"svd\") -> {regular, \"application/vnd.svd\"};\nt(global, \"SVD\") -> {regular, \"application/vnd.svd\"};\nt(global, \"susp\") -> {regular, \"application/vnd.sus-calendar\"};\nt(global, \"SUSP\") -> {regular, \"application/vnd.sus-calendar\"};\nt(global, \"sus\") -> {regular, \"application/vnd.sus-calendar\"};\nt(global, \"SUS\") -> {regular, \"application/vnd.sus-calendar\"};\nt(global, \"stw\") -> {regular, \"application/vnd.sun.xml.writer.template\"};\nt(global, \"STW\") -> {regular, \"application/vnd.sun.xml.writer.template\"};\nt(global, \"sxg\") -> {regular, \"application/vnd.sun.xml.writer.global\"};\nt(global, \"SXG\") -> {regular, \"application/vnd.sun.xml.writer.global\"};\nt(global, \"sxw\") -> {regular, \"application/vnd.sun.xml.writer\"};\nt(global, \"SXW\") -> {regular, \"application/vnd.sun.xml.writer\"};\nt(global, \"sxm\") -> {regular, \"application/vnd.sun.xml.math\"};\nt(global, \"SXM\") -> {regular, \"application/vnd.sun.xml.math\"};\nt(global, \"sti\") -> {regular, \"application/vnd.sun.xml.impress.template\"};\nt(global, \"STI\") -> {regular, \"application/vnd.sun.xml.impress.template\"};\nt(global, \"sxi\") -> {regular, \"application/vnd.sun.xml.impress\"};\nt(global, \"SXI\") -> {regular, \"application/vnd.sun.xml.impress\"};\nt(global, \"std\") -> {regular, \"application/vnd.sun.xml.draw.template\"};\nt(global, \"STD\") -> {regular, \"application/vnd.sun.xml.draw.template\"};\nt(global, \"sxd\") -> {regular, \"application/vnd.sun.xml.draw\"};\nt(global, \"SXD\") -> {regular, \"application/vnd.sun.xml.draw\"};\nt(global, \"stc\") -> {regular, \"application/vnd.sun.xml.calc.template\"};\nt(global, \"STC\") -> {regular, \"application/vnd.sun.xml.calc.template\"};\nt(global, \"sxc\") -> {regular, \"application/vnd.sun.xml.calc\"};\nt(global, \"SXC\") -> {regular, \"application/vnd.sun.xml.calc\"};\nt(global, \"sm\") -> {regular, \"application/vnd.stepmania.stepchart\"};\nt(global, \"SM\") -> {regular, \"application/vnd.stepmania.stepchart\"};\nt(global, \"smzip\") -> {regular, \"application/vnd.stepmania.package\"};\nt(global, \"SMZIP\") -> {regular, \"application/vnd.stepmania.package\"};\nt(global, \"sgl\") -> {regular, \"application/vnd.stardivision.writer-global\"};\nt(global, \"SGL\") -> {regular, \"application/vnd.stardivision.writer-global\"};\nt(global, \"vor\") -> {regular, \"application/vnd.stardivision.writer\"};\nt(global, \"VOR\") -> {regular, \"application/vnd.stardivision.writer\"};\nt(global, \"sdw\") -> {regular, \"application/vnd.stardivision.writer\"};\nt(global, \"SDW\") -> {regular, \"application/vnd.stardivision.writer\"};\nt(global, \"smf\") -> {regular, \"application/vnd.stardivision.math\"};\nt(global, \"SMF\") -> {regular, \"application/vnd.stardivision.math\"};\nt(global, \"sdd\") -> {regular, \"application/vnd.stardivision.impress\"};\nt(global, \"SDD\") -> {regular, \"application/vnd.stardivision.impress\"};\nt(global, \"sda\") -> {regular, \"application/vnd.stardivision.draw\"};\nt(global, \"SDA\") -> {regular, \"application/vnd.stardivision.draw\"};\nt(global, \"sdc\") -> {regular, \"application/vnd.stardivision.calc\"};\nt(global, \"SDC\") -> {regular, \"application/vnd.stardivision.calc\"};\nt(global, \"sfs\") -> {regular, \"application/vnd.spotfire.sfs\"};\nt(global, \"SFS\") -> {regular, \"application/vnd.spotfire.sfs\"};\nt(global, \"dxp\") -> {regular, \"application/vnd.spotfire.dxp\"};\nt(global, \"DXP\") -> {regular, \"application/vnd.spotfire.dxp\"};\nt(global, \"sdkd\") -> {regular, \"application/vnd.solent.sdkm+xml\"};\nt(global, \"SDKD\") -> {regular, \"application/vnd.solent.sdkm+xml\"};\nt(global, \"sdkm\") -> {regular, \"application/vnd.solent.sdkm+xml\"};\nt(global, \"SDKM\") -> {regular, \"application/vnd.solent.sdkm+xml\"};\nt(global, \"teacher\") -> {regular, \"application/vnd.smart.teacher\"};\nt(global, \"TEACHER\") -> {regular, \"application/vnd.smart.teacher\"};\nt(global, \"mmf\") -> {regular, \"application/vnd.smaf\"};\nt(global, \"MMF\") -> {regular, \"application/vnd.smaf\"};\nt(global, \"twds\") -> {regular, \"application/vnd.simtech-mindmapper\"};\nt(global, \"TWDS\") -> {regular, \"application/vnd.simtech-mindmapper\"};\nt(global, \"twd\") -> {regular, \"application/vnd.simtech-mindmapper\"};\nt(global, \"TWD\") -> {regular, \"application/vnd.simtech-mindmapper\"};\nt(global, \"ipk\") -> {regular, \"application/vnd.shana.informed.package\"};\nt(global, \"IPK\") -> {regular, \"application/vnd.shana.informed.package\"};\nt(global, \"iif\") -> {regular, \"application/vnd.shana.informed.interchange\"};\nt(global, \"IIF\") -> {regular, \"application/vnd.shana.informed.interchange\"};\nt(global, \"itp\") -> {regular, \"application/vnd.shana.informed.formtemplate\"};\nt(global, \"ITP\") -> {regular, \"application/vnd.shana.informed.formtemplate\"};\nt(global, \"ifm\") -> {regular, \"application/vnd.shana.informed.formdata\"};\nt(global, \"IFM\") -> {regular, \"application/vnd.shana.informed.formdata\"};\nt(global, \"semf\") -> {regular, \"application/vnd.semf\"};\nt(global, \"SEMF\") -> {regular, \"application/vnd.semf\"};\nt(global, \"semd\") -> {regular, \"application/vnd.semd\"};\nt(global, \"SEMD\") -> {regular, \"application/vnd.semd\"};\nt(global, \"sema\") -> {regular, \"application/vnd.sema\"};\nt(global, \"SEMA\") -> {regular, \"application/vnd.sema\"};\nt(global, \"see\") -> {regular, \"application/vnd.seemail\"};\nt(global, \"SEE\") -> {regular, \"application/vnd.seemail\"};\nt(global, \"st\") -> {regular, \"application/vnd.sailingtracker.track\"};\nt(global, \"ST\") -> {regular, \"application/vnd.sailingtracker.track\"};\nt(global, \"link66\") -> {regular, \"application/vnd.route66.link66+xml\"};\nt(global, \"LINK66\") -> {regular, \"application/vnd.route66.link66+xml\"};\nt(global, \"rmvb\") -> {regular, \"application/vnd.rn-realmedia-vbr\"};\nt(global, \"RMVB\") -> {regular, \"application/vnd.rn-realmedia-vbr\"};\nt(global, \"rm\") -> {regular, \"application/vnd.rn-realmedia\"};\nt(global, \"RM\") -> {regular, \"application/vnd.rn-realmedia\"};\nt(global, \"cod\") -> {regular, \"application/vnd.rim.cod\"};\nt(global, \"COD\") -> {regular, \"application/vnd.rim.cod\"};\nt(global, \"cryptonote\") -> {regular, \"application/vnd.rig.cryptonote\"};\nt(global, \"CRYPTONOTE\") -> {regular, \"application/vnd.rig.cryptonote\"};\nt(global, \"musicxml\") -> {regular, \"application/vnd.recordare.musicxml+xml\"};\nt(global, \"MUSICXML\") -> {regular, \"application/vnd.recordare.musicxml+xml\"};\nt(global, \"mxl\") -> {regular, \"application/vnd.recordare.musicxml\"};\nt(global, \"MXL\") -> {regular, \"application/vnd.recordare.musicxml\"};\nt(global, \"bed\") -> {regular, \"application/vnd.realvnc.bed\"};\nt(global, \"BED\") -> {regular, \"application/vnd.realvnc.bed\"};\nt(global, \"qxb\") -> {regular, \"application/vnd.quark.quarkxpress\"};\nt(global, \"QXB\") -> {regular, \"application/vnd.quark.quarkxpress\"};\nt(global, \"qxl\") -> {regular, \"application/vnd.quark.quarkxpress\"};\nt(global, \"QXL\") -> {regular, \"application/vnd.quark.quarkxpress\"};\nt(global, \"qwt\") -> {regular, \"application/vnd.quark.quarkxpress\"};\nt(global, \"QWT\") -> {regular, \"application/vnd.quark.quarkxpress\"};\nt(global, \"qwd\") -> {regular, \"application/vnd.quark.quarkxpress\"};\nt(global, \"QWD\") -> {regular, \"application/vnd.quark.quarkxpress\"};\nt(global, \"qxt\") -> {regular, \"application/vnd.quark.quarkxpress\"};\nt(global, \"QXT\") -> {regular, \"application/vnd.quark.quarkxpress\"};\nt(global, \"qxd\") -> {regular, \"application/vnd.quark.quarkxpress\"};\nt(global, \"QXD\") -> {regular, \"application/vnd.quark.quarkxpress\"};\nt(global, \"ptid\") -> {regular, \"application/vnd.pvi.ptid1\"};\nt(global, \"PTID\") -> {regular, \"application/vnd.pvi.ptid1\"};\nt(global, \"qps\") -> {regular, \"application/vnd.publishare-delta-tree\"};\nt(global, \"QPS\") -> {regular, \"application/vnd.publishare-delta-tree\"};\nt(global, \"mgz\") -> {regular, \"application/vnd.proteus.magazine\"};\nt(global, \"MGZ\") -> {regular, \"application/vnd.proteus.magazine\"};\nt(global, \"box\") -> {regular, \"application/vnd.previewsystems.box\"};\nt(global, \"BOX\") -> {regular, \"application/vnd.previewsystems.box\"};\nt(global, \"pbd\") -> {regular, \"application/vnd.powerbuilder6\"};\nt(global, \"PBD\") -> {regular, \"application/vnd.powerbuilder6\"};\nt(global, \"plf\") -> {regular, \"application/vnd.pocketlearn\"};\nt(global, \"PLF\") -> {regular, \"application/vnd.pocketlearn\"};\nt(global, \"wg\") -> {regular, \"application/vnd.pmi.widget\"};\nt(global, \"WG\") -> {regular, \"application/vnd.pmi.widget\"};\nt(global, \"efif\") -> {regular, \"application/vnd.picsel\"};\nt(global, \"EFIF\") -> {regular, \"application/vnd.picsel\"};\nt(global, \"ei6\") -> {regular, \"application/vnd.pg.osasli\"};\nt(global, \"EI6\") -> {regular, \"application/vnd.pg.osasli\"};\nt(global, \"str\") -> {regular, \"application/vnd.pg.format\"};\nt(global, \"STR\") -> {regular, \"application/vnd.pg.format\"};\nt(global, \"paw\") -> {regular, \"application/vnd.pawaafile\"};\nt(global, \"PAW\") -> {regular, \"application/vnd.pawaafile\"};\nt(global, \"oprc\") -> {regular, \"application/vnd.palm\"};\nt(global, \"OPRC\") -> {regular, \"application/vnd.palm\"};\nt(global, \"pqa\") -> {regular, \"application/vnd.palm\"};\nt(global, \"PQA\") -> {regular, \"application/vnd.palm\"};\nt(global, \"pdb\") -> {regular, \"application/vnd.palm\"};\nt(global, \"PDB\") -> {regular, \"application/vnd.palm\"};\nt(global, \"esa\") -> {regular, \"application/vnd.osgi.subsystem\"};\nt(global, \"ESA\") -> {regular, \"application/vnd.osgi.subsystem\"};\nt(global, \"dp\") -> {regular, \"application/vnd.osgi.dp\"};\nt(global, \"DP\") -> {regular, \"application/vnd.osgi.dp\"};\nt(global, \"mgp\") -> {regular, \"application/vnd.osgeo.mapguide.package\"};\nt(global, \"MGP\") -> {regular, \"application/vnd.osgeo.mapguide.package\"};\nt(global, \"dotx\") -> {regular, \"application/vnd.openxmlformats-officedocument.wordprocessingml.template\"};\nt(global, \"DOTX\") -> {regular, \"application/vnd.openxmlformats-officedocument.wordprocessingml.template\"};\nt(global, \"docx\") -> {regular, \"application/vnd.openxmlformats-officedocument.wordprocessingml.document\"};\nt(global, \"DOCX\") -> {regular, \"application/vnd.openxmlformats-officedocument.wordprocessingml.document\"};\nt(global, \"xltx\") -> {regular, \"application/vnd.openxmlformats-officedocument.spreadsheetml.template\"};\nt(global, \"XLTX\") -> {regular, \"application/vnd.openxmlformats-officedocument.spreadsheetml.template\"};\nt(global, \"xlsx\") -> {regular, \"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\"};\nt(global, \"XLSX\") -> {regular, \"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\"};\nt(global, \"potx\") -> {regular, \"application/vnd.openxmlformats-officedocument.presentationml.template\"};\nt(global, \"POTX\") -> {regular, \"application/vnd.openxmlformats-officedocument.presentationml.template\"};\nt(global, \"ppsx\") -> {regular, \"application/vnd.openxmlformats-officedocument.presentationml.slideshow\"};\nt(global, \"PPSX\") -> {regular, \"application/vnd.openxmlformats-officedocument.presentationml.slideshow\"};\nt(global, \"sldx\") -> {regular, \"application/vnd.openxmlformats-officedocument.presentationml.slide\"};\nt(global, \"SLDX\") -> {regular, \"application/vnd.openxmlformats-officedocument.presentationml.slide\"};\nt(global, \"pptx\") -> {regular, \"application/vnd.openxmlformats-officedocument.presentationml.presentation\"};\nt(global, \"PPTX\") -> {regular, \"application/vnd.openxmlformats-officedocument.presentationml.presentation\"};\nt(global, \"oxt\") -> {regular, \"application/vnd.openofficeorg.extension\"};\nt(global, \"OXT\") -> {regular, \"application/vnd.openofficeorg.extension\"};\nt(global, \"dd2\") -> {regular, \"application/vnd.oma.dd2+xml\"};\nt(global, \"DD2\") -> {regular, \"application/vnd.oma.dd2+xml\"};\nt(global, \"xo\") -> {regular, \"application/vnd.olpc-sugar\"};\nt(global, \"XO\") -> {regular, \"application/vnd.olpc-sugar\"};\nt(global, \"oth\") -> {regular, \"application/vnd.oasis.opendocument.text-web\"};\nt(global, \"OTH\") -> {regular, \"application/vnd.oasis.opendocument.text-web\"};\nt(global, \"ott\") -> {regular, \"application/vnd.oasis.opendocument.text-template\"};\nt(global, \"OTT\") -> {regular, \"application/vnd.oasis.opendocument.text-template\"};\nt(global, \"odm\") -> {regular, \"application/vnd.oasis.opendocument.text-master\"};\nt(global, \"ODM\") -> {regular, \"application/vnd.oasis.opendocument.text-master\"};\nt(global, \"odt\") -> {regular, \"application/vnd.oasis.opendocument.text\"};\nt(global, \"ODT\") -> {regular, \"application/vnd.oasis.opendocument.text\"};\nt(global, \"ots\") -> {regular, \"application/vnd.oasis.opendocument.spreadsheet-template\"};\nt(global, \"OTS\") -> {regular, \"application/vnd.oasis.opendocument.spreadsheet-template\"};\nt(global, \"ods\") -> {regular, \"application/vnd.oasis.opendocument.spreadsheet\"};\nt(global, \"ODS\") -> {regular, \"application/vnd.oasis.opendocument.spreadsheet\"};\nt(global, \"otp\") -> {regular, \"application/vnd.oasis.opendocument.presentation-template\"};\nt(global, \"OTP\") -> {regular, \"application/vnd.oasis.opendocument.presentation-template\"};\nt(global, \"odp\") -> {regular, \"application/vnd.oasis.opendocument.presentation\"};\nt(global, \"ODP\") -> {regular, \"application/vnd.oasis.opendocument.presentation\"};\nt(global, \"oti\") -> {regular, \"application/vnd.oasis.opendocument.image-template\"};\nt(global, \"OTI\") -> {regular, \"application/vnd.oasis.opendocument.image-template\"};\nt(global, \"odi\") -> {regular, \"application/vnd.oasis.opendocument.image\"};\nt(global, \"ODI\") -> {regular, \"application/vnd.oasis.opendocument.image\"};\nt(global, \"otg\") -> {regular, \"application/vnd.oasis.opendocument.graphics-template\"};\nt(global, \"OTG\") -> {regular, \"application/vnd.oasis.opendocument.graphics-template\"};\nt(global, \"odg\") -> {regular, \"application/vnd.oasis.opendocument.graphics\"};\nt(global, \"ODG\") -> {regular, \"application/vnd.oasis.opendocument.graphics\"};\nt(global, \"odft\") -> {regular, \"application/vnd.oasis.opendocument.formula-template\"};\nt(global, \"ODFT\") -> {regular, \"application/vnd.oasis.opendocument.formula-template\"};\nt(global, \"odf\") -> {regular, \"application/vnd.oasis.opendocument.formula\"};\nt(global, \"ODF\") -> {regular, \"application/vnd.oasis.opendocument.formula\"};\nt(global, \"odb\") -> {regular, \"application/vnd.oasis.opendocument.database\"};\nt(global, \"ODB\") -> {regular, \"application/vnd.oasis.opendocument.database\"};\nt(global, \"otc\") -> {regular, \"application/vnd.oasis.opendocument.chart-template\"};\nt(global, \"OTC\") -> {regular, \"application/vnd.oasis.opendocument.chart-template\"};\nt(global, \"odc\") -> {regular, \"application/vnd.oasis.opendocument.chart\"};\nt(global, \"ODC\") -> {regular, \"application/vnd.oasis.opendocument.chart\"};\nt(global, \"ext\") -> {regular, \"application/vnd.novadigm.ext\"};\nt(global, \"EXT\") -> {regular, \"application/vnd.novadigm.ext\"};\nt(global, \"edx\") -> {regular, \"application/vnd.novadigm.edx\"};\nt(global, \"EDX\") -> {regular, \"application/vnd.novadigm.edx\"};\nt(global, \"edm\") -> {regular, \"application/vnd.novadigm.edm\"};\nt(global, \"EDM\") -> {regular, \"application/vnd.novadigm.edm\"};\nt(global, \"rpss\") -> {regular, \"application/vnd.nokia.radio-presets\"};\nt(global, \"RPSS\") -> {regular, \"application/vnd.nokia.radio-presets\"};\nt(global, \"rpst\") -> {regular, \"application/vnd.nokia.radio-preset\"};\nt(global, \"RPST\") -> {regular, \"application/vnd.nokia.radio-preset\"};\nt(global, \"n-gage\") -> {regular, \"application/vnd.nokia.n-gage.symbian.install\"};\nt(global, \"N-GAGE\") -> {regular, \"application/vnd.nokia.n-gage.symbian.install\"};\nt(global, \"ngdat\") -> {regular, \"application/vnd.nokia.n-gage.data\"};\nt(global, \"NGDAT\") -> {regular, \"application/vnd.nokia.n-gage.data\"};\nt(global, \"nnw\") -> {regular, \"application/vnd.noblenet-web\"};\nt(global, \"NNW\") -> {regular, \"application/vnd.noblenet-web\"};\nt(global, \"nns\") -> {regular, \"application/vnd.noblenet-sealer\"};\nt(global, \"NNS\") -> {regular, \"application/vnd.noblenet-sealer\"};\nt(global, \"nnd\") -> {regular, \"application/vnd.noblenet-directory\"};\nt(global, \"NND\") -> {regular, \"application/vnd.noblenet-directory\"};\nt(global, \"nitf\") -> {regular, \"application/vnd.nitf\"};\nt(global, \"NITF\") -> {regular, \"application/vnd.nitf\"};\nt(global, \"ntf\") -> {regular, \"application/vnd.nitf\"};\nt(global, \"NTF\") -> {regular, \"application/vnd.nitf\"};\nt(global, \"nlu\") -> {regular, \"application/vnd.neurolanguage.nlu\"};\nt(global, \"NLU\") -> {regular, \"application/vnd.neurolanguage.nlu\"};\nt(global, \"taglet\") -> {regular, \"application/vnd.mynfc\"};\nt(global, \"TAGLET\") -> {regular, \"application/vnd.mynfc\"};\nt(global, \"msty\") -> {regular, \"application/vnd.muvee.style\"};\nt(global, \"MSTY\") -> {regular, \"application/vnd.muvee.style\"};\nt(global, \"mus\") -> {regular, \"application/vnd.musician\"};\nt(global, \"MUS\") -> {regular, \"application/vnd.musician\"};\nt(global, \"mseq\") -> {regular, \"application/vnd.mseq\"};\nt(global, \"MSEQ\") -> {regular, \"application/vnd.mseq\"};\nt(global, \"xps\") -> {regular, \"application/vnd.ms-xpsdocument\"};\nt(global, \"XPS\") -> {regular, \"application/vnd.ms-xpsdocument\"};\nt(global, \"wpl\") -> {regular, \"application/vnd.ms-wpl\"};\nt(global, \"WPL\") -> {regular, \"application/vnd.ms-wpl\"};\nt(global, \"wdb\") -> {regular, \"application/vnd.ms-works\"};\nt(global, \"WDB\") -> {regular, \"application/vnd.ms-works\"};\nt(global, \"wcm\") -> {regular, \"application/vnd.ms-works\"};\nt(global, \"WCM\") -> {regular, \"application/vnd.ms-works\"};\nt(global, \"wks\") -> {regular, \"application/vnd.ms-works\"};\nt(global, \"WKS\") -> {regular, \"application/vnd.ms-works\"};\nt(global, \"wps\") -> {regular, \"application/vnd.ms-works\"};\nt(global, \"WPS\") -> {regular, \"application/vnd.ms-works\"};\nt(global, \"dotm\") -> {regular, \"application/vnd.ms-word.template.macroenabled.12\"};\nt(global, \"DOTM\") -> {regular, \"application/vnd.ms-word.template.macroenabled.12\"};\nt(global, \"docm\") -> {regular, \"application/vnd.ms-word.document.macroenabled.12\"};\nt(global, \"DOCM\") -> {regular, \"application/vnd.ms-word.document.macroenabled.12\"};\nt(global, \"mpt\") -> {regular, \"application/vnd.ms-project\"};\nt(global, \"MPT\") -> {regular, \"application/vnd.ms-project\"};\nt(global, \"mpp\") -> {regular, \"application/vnd.ms-project\"};\nt(global, \"MPP\") -> {regular, \"application/vnd.ms-project\"};\nt(global, \"potm\") -> {regular, \"application/vnd.ms-powerpoint.template.macroenabled.12\"};\nt(global, \"POTM\") -> {regular, \"application/vnd.ms-powerpoint.template.macroenabled.12\"};\nt(global, \"ppsm\") -> {regular, \"application/vnd.ms-powerpoint.slideshow.macroenabled.12\"};\nt(global, \"PPSM\") -> {regular, \"application/vnd.ms-powerpoint.slideshow.macroenabled.12\"};\nt(global, \"sldm\") -> {regular, \"application/vnd.ms-powerpoint.slide.macroenabled.12\"};\nt(global, \"SLDM\") -> {regular, \"application/vnd.ms-powerpoint.slide.macroenabled.12\"};\nt(global, \"pptm\") -> {regular, \"application/vnd.ms-powerpoint.presentation.macroenabled.12\"};\nt(global, \"PPTM\") -> {regular, \"application/vnd.ms-powerpoint.presentation.macroenabled.12\"};\nt(global, \"ppam\") -> {regular, \"application/vnd.ms-powerpoint.addin.macroenabled.12\"};\nt(global, \"PPAM\") -> {regular, \"application/vnd.ms-powerpoint.addin.macroenabled.12\"};\nt(global, \"pot\") -> {regular, \"application/vnd.ms-powerpoint\"};\nt(global, \"POT\") -> {regular, \"application/vnd.ms-powerpoint\"};\nt(global, \"pps\") -> {regular, \"application/vnd.ms-powerpoint\"};\nt(global, \"PPS\") -> {regular, \"application/vnd.ms-powerpoint\"};\nt(global, \"ppt\") -> {regular, \"application/vnd.ms-powerpoint\"};\nt(global, \"PPT\") -> {regular, \"application/vnd.ms-powerpoint\"};\nt(global, \"stl\") -> {regular, \"application/vnd.ms-pki.stl\"};\nt(global, \"STL\") -> {regular, \"application/vnd.ms-pki.stl\"};\nt(global, \"cat\") -> {regular, \"application/vnd.ms-pki.seccat\"};\nt(global, \"CAT\") -> {regular, \"application/vnd.ms-pki.seccat\"};\nt(global, \"thmx\") -> {regular, \"application/vnd.ms-officetheme\"};\nt(global, \"THMX\") -> {regular, \"application/vnd.ms-officetheme\"};\nt(global, \"lrm\") -> {regular, \"application/vnd.ms-lrm\"};\nt(global, \"LRM\") -> {regular, \"application/vnd.ms-lrm\"};\nt(global, \"ims\") -> {regular, \"application/vnd.ms-ims\"};\nt(global, \"IMS\") -> {regular, \"application/vnd.ms-ims\"};\nt(global, \"chm\") -> {regular, \"application/vnd.ms-htmlhelp\"};\nt(global, \"CHM\") -> {regular, \"application/vnd.ms-htmlhelp\"};\nt(global, \"eot\") -> {regular, \"application/vnd.ms-fontobject\"};\nt(global, \"EOT\") -> {regular, \"application/vnd.ms-fontobject\"};\nt(global, \"xltm\") -> {regular, \"application/vnd.ms-excel.template.macroenabled.12\"};\nt(global, \"XLTM\") -> {regular, \"application/vnd.ms-excel.template.macroenabled.12\"};\nt(global, \"xlsm\") -> {regular, \"application/vnd.ms-excel.sheet.macroenabled.12\"};\nt(global, \"XLSM\") -> {regular, \"application/vnd.ms-excel.sheet.macroenabled.12\"};\nt(global, \"xlsb\") -> {regular, \"application/vnd.ms-excel.sheet.binary.macroenabled.12\"};\nt(global, \"XLSB\") -> {regular, \"application/vnd.ms-excel.sheet.binary.macroenabled.12\"};\nt(global, \"xlam\") -> {regular, \"application/vnd.ms-excel.addin.macroenabled.12\"};\nt(global, \"XLAM\") -> {regular, \"application/vnd.ms-excel.addin.macroenabled.12\"};\nt(global, \"xlw\") -> {regular, \"application/vnd.ms-excel\"};\nt(global, \"XLW\") -> {regular, \"application/vnd.ms-excel\"};\nt(global, \"xlt\") -> {regular, \"application/vnd.ms-excel\"};\nt(global, \"XLT\") -> {regular, \"application/vnd.ms-excel\"};\nt(global, \"xlc\") -> {regular, \"application/vnd.ms-excel\"};\nt(global, \"XLC\") -> {regular, \"application/vnd.ms-excel\"};\nt(global, \"xla\") -> {regular, \"application/vnd.ms-excel\"};\nt(global, \"XLA\") -> {regular, \"application/vnd.ms-excel\"};\nt(global, \"xlm\") -> {regular, \"application/vnd.ms-excel\"};\nt(global, \"XLM\") -> {regular, \"application/vnd.ms-excel\"};\nt(global, \"xls\") -> {regular, \"application/vnd.ms-excel\"};\nt(global, \"XLS\") -> {regular, \"application/vnd.ms-excel\"};\nt(global, \"cab\") -> {regular, \"application/vnd.ms-cab-compressed\"};\nt(global, \"CAB\") -> {regular, \"application/vnd.ms-cab-compressed\"};\nt(global, \"cil\") -> {regular, \"application/vnd.ms-artgalry\"};\nt(global, \"CIL\") -> {regular, \"application/vnd.ms-artgalry\"};\nt(global, \"xul\") -> {regular, \"application/vnd.mozilla.xul+xml\"};\nt(global, \"XUL\") -> {regular, \"application/vnd.mozilla.xul+xml\"};\nt(global, \"mpc\") -> {regular, \"application/vnd.mophun.certificate\"};\nt(global, \"MPC\") -> {regular, \"application/vnd.mophun.certificate\"};\nt(global, \"mpn\") -> {regular, \"application/vnd.mophun.application\"};\nt(global, \"MPN\") -> {regular, \"application/vnd.mophun.application\"};\nt(global, \"txf\") -> {regular, \"application/vnd.mobius.txf\"};\nt(global, \"TXF\") -> {regular, \"application/vnd.mobius.txf\"};\nt(global, \"plc\") -> {regular, \"application/vnd.mobius.plc\"};\nt(global, \"PLC\") -> {regular, \"application/vnd.mobius.plc\"};\nt(global, \"msl\") -> {regular, \"application/vnd.mobius.msl\"};\nt(global, \"MSL\") -> {regular, \"application/vnd.mobius.msl\"};\nt(global, \"mqy\") -> {regular, \"application/vnd.mobius.mqy\"};\nt(global, \"MQY\") -> {regular, \"application/vnd.mobius.mqy\"};\nt(global, \"mbk\") -> {regular, \"application/vnd.mobius.mbk\"};\nt(global, \"MBK\") -> {regular, \"application/vnd.mobius.mbk\"};\nt(global, \"dis\") -> {regular, \"application/vnd.mobius.dis\"};\nt(global, \"DIS\") -> {regular, \"application/vnd.mobius.dis\"};\nt(global, \"daf\") -> {regular, \"application/vnd.mobius.daf\"};\nt(global, \"DAF\") -> {regular, \"application/vnd.mobius.daf\"};\nt(global, \"mif\") -> {regular, \"application/vnd.mif\"};\nt(global, \"MIF\") -> {regular, \"application/vnd.mif\"};\nt(global, \"igx\") -> {regular, \"application/vnd.micrografx.igx\"};\nt(global, \"IGX\") -> {regular, \"application/vnd.micrografx.igx\"};\nt(global, \"flo\") -> {regular, \"application/vnd.micrografx.flo\"};\nt(global, \"FLO\") -> {regular, \"application/vnd.micrografx.flo\"};\nt(global, \"mfm\") -> {regular, \"application/vnd.mfmp\"};\nt(global, \"MFM\") -> {regular, \"application/vnd.mfmp\"};\nt(global, \"mwf\") -> {regular, \"application/vnd.mfer\"};\nt(global, \"MWF\") -> {regular, \"application/vnd.mfer\"};\nt(global, \"cdkey\") -> {regular, \"application/vnd.mediastation.cdkey\"};\nt(global, \"CDKEY\") -> {regular, \"application/vnd.mediastation.cdkey\"};\nt(global, \"mc1\") -> {regular, \"application/vnd.medcalcdata\"};\nt(global, \"MC1\") -> {regular, \"application/vnd.medcalcdata\"};\nt(global, \"mcd\") -> {regular, \"application/vnd.mcd\"};\nt(global, \"MCD\") -> {regular, \"application/vnd.mcd\"};\nt(global, \"portpkg\") -> {regular, \"application/vnd.macports.portpkg\"};\nt(global, \"PORTPKG\") -> {regular, \"application/vnd.macports.portpkg\"};\nt(global, \"lwp\") -> {regular, \"application/vnd.lotus-wordpro\"};\nt(global, \"LWP\") -> {regular, \"application/vnd.lotus-wordpro\"};\nt(global, \"scm\") -> {regular, \"application/vnd.lotus-screencam\"};\nt(global, \"SCM\") -> {regular, \"application/vnd.lotus-screencam\"};\nt(global, \"org\") -> {regular, \"application/vnd.lotus-organizer\"};\nt(global, \"ORG\") -> {regular, \"application/vnd.lotus-organizer\"};\nt(global, \"nsf\") -> {regular, \"application/vnd.lotus-notes\"};\nt(global, \"NSF\") -> {regular, \"application/vnd.lotus-notes\"};\nt(global, \"pre\") -> {regular, \"application/vnd.lotus-freelance\"};\nt(global, \"PRE\") -> {regular, \"application/vnd.lotus-freelance\"};\nt(global, \"apr\") -> {regular, \"application/vnd.lotus-approach\"};\nt(global, \"APR\") -> {regular, \"application/vnd.lotus-approach\"};\nt(global, \"123\") -> {regular, \"application/vnd.lotus-1-2-3\"};\nt(global, \"lbe\") -> {regular, \"application/vnd.llamagraphics.life-balance.exchange+xml\"};\nt(global, \"LBE\") -> {regular, \"application/vnd.llamagraphics.life-balance.exchange+xml\"};\nt(global, \"lbd\") -> {regular, \"application/vnd.llamagraphics.life-balance.desktop\"};\nt(global, \"LBD\") -> {regular, \"application/vnd.llamagraphics.life-balance.desktop\"};\nt(global, \"lasxml\") -> {regular, \"application/vnd.las.las+xml\"};\nt(global, \"LASXML\") -> {regular, \"application/vnd.las.las+xml\"};\nt(global, \"sse\") -> {regular, \"application/vnd.kodak-descriptor\"};\nt(global, \"SSE\") -> {regular, \"application/vnd.kodak-descriptor\"};\nt(global, \"skm\") -> {regular, \"application/x-koan\"};\nt(global, \"SKM\") -> {regular, \"application/x-koan\"};\nt(global, \"skt\") -> {regular, \"application/x-koan\"};\nt(global, \"SKT\") -> {regular, \"application/x-koan\"};\nt(global, \"skd\") -> {regular, \"application/x-koan\"};\nt(global, \"SKD\") -> {regular, \"application/x-koan\"};\nt(global, \"skp\") -> {regular, \"application/x-koan\"};\nt(global, \"SKP\") -> {regular, \"application/x-koan\"};\nt(global, \"knp\") -> {regular, \"application/vnd.kinar\"};\nt(global, \"KNP\") -> {regular, \"application/vnd.kinar\"};\nt(global, \"kne\") -> {regular, \"application/vnd.kinar\"};\nt(global, \"KNE\") -> {regular, \"application/vnd.kinar\"};\nt(global, \"kia\") -> {regular, \"application/vnd.kidspiration\"};\nt(global, \"KIA\") -> {regular, \"application/vnd.kidspiration\"};\nt(global, \"htke\") -> {regular, \"application/vnd.kenameaapp\"};\nt(global, \"HTKE\") -> {regular, \"application/vnd.kenameaapp\"};\nt(global, \"kwt\") -> {regular, \"application/x-kword\"};\nt(global, \"KWT\") -> {regular, \"application/x-kword\"};\nt(global, \"kwd\") -> {regular, \"application/x-kword\"};\nt(global, \"KWD\") -> {regular, \"application/x-kword\"};\nt(global, \"ksp\") -> {regular, \"application/x-kspread\"};\nt(global, \"KSP\") -> {regular, \"application/x-kspread\"};\nt(global, \"kpt\") -> {regular, \"application/x-kpresenter\"};\nt(global, \"KPT\") -> {regular, \"application/x-kpresenter\"};\nt(global, \"kpr\") -> {regular, \"application/x-kpresenter\"};\nt(global, \"KPR\") -> {regular, \"application/x-kpresenter\"};\nt(global, \"kon\") -> {regular, \"application/vnd.kde.kontour\"};\nt(global, \"KON\") -> {regular, \"application/vnd.kde.kontour\"};\nt(global, \"flw\") -> {regular, \"application/vnd.kde.kivio\"};\nt(global, \"FLW\") -> {regular, \"application/vnd.kde.kivio\"};\nt(global, \"kfo\") -> {regular, \"application/vnd.kde.kformula\"};\nt(global, \"KFO\") -> {regular, \"application/vnd.kde.kformula\"};\nt(global, \"chrt\") -> {regular, \"application/x-kchart\"};\nt(global, \"CHRT\") -> {regular, \"application/x-kchart\"};\nt(global, \"karbon\") -> {regular, \"application/vnd.kde.karbon\"};\nt(global, \"KARBON\") -> {regular, \"application/vnd.kde.karbon\"};\nt(global, \"ktr\") -> {regular, \"application/vnd.kahootz\"};\nt(global, \"KTR\") -> {regular, \"application/vnd.kahootz\"};\nt(global, \"ktz\") -> {regular, \"application/vnd.kahootz\"};\nt(global, \"KTZ\") -> {regular, \"application/vnd.kahootz\"};\nt(global, \"joda\") -> {regular, \"application/vnd.joost.joda-archive\"};\nt(global, \"JODA\") -> {regular, \"application/vnd.joost.joda-archive\"};\nt(global, \"jisp\") -> {regular, \"application/vnd.jisp\"};\nt(global, \"JISP\") -> {regular, \"application/vnd.jisp\"};\nt(global, \"rms\") -> {regular, \"application/vnd.jcp.javame.midlet-rms\"};\nt(global, \"RMS\") -> {regular, \"application/vnd.jcp.javame.midlet-rms\"};\nt(global, \"jam\") -> {regular, \"application/vnd.jam\"};\nt(global, \"JAM\") -> {regular, \"application/vnd.jam\"};\nt(global, \"fcs\") -> {regular, \"application/vnd.isac.fcs\"};\nt(global, \"FCS\") -> {regular, \"application/vnd.isac.fcs\"};\nt(global, \"xpr\") -> {regular, \"application/vnd.is-xpr\"};\nt(global, \"XPR\") -> {regular, \"application/vnd.is-xpr\"};\nt(global, \"irp\") -> {regular, \"application/vnd.irepository.package+xml\"};\nt(global, \"IRP\") -> {regular, \"application/vnd.irepository.package+xml\"};\nt(global, \"rcprofile\") -> {regular, \"application/vnd.ipunplugged.rcprofile\"};\nt(global, \"RCPROFILE\") -> {regular, \"application/vnd.ipunplugged.rcprofile\"};\nt(global, \"qfx\") -> {regular, \"application/vnd.intu.qfx\"};\nt(global, \"QFX\") -> {regular, \"application/vnd.intu.qfx\"};\nt(global, \"qbo\") -> {regular, \"application/vnd.intu.qbo\"};\nt(global, \"QBO\") -> {regular, \"application/vnd.intu.qbo\"};\nt(global, \"i2g\") -> {regular, \"application/vnd.intergeo\"};\nt(global, \"I2G\") -> {regular, \"application/vnd.intergeo\"};\nt(global, \"xpx\") -> {regular, \"application/vnd.intercon.formnet\"};\nt(global, \"XPX\") -> {regular, \"application/vnd.intercon.formnet\"};\nt(global, \"xpw\") -> {regular, \"application/vnd.intercon.formnet\"};\nt(global, \"XPW\") -> {regular, \"application/vnd.intercon.formnet\"};\nt(global, \"igm\") -> {regular, \"application/vnd.insors.igm\"};\nt(global, \"IGM\") -> {regular, \"application/vnd.insors.igm\"};\nt(global, \"ivu\") -> {regular, \"application/vnd.immervision-ivu\"};\nt(global, \"IVU\") -> {regular, \"application/vnd.immervision-ivu\"};\nt(global, \"ivp\") -> {regular, \"application/vnd.immervision-ivp\"};\nt(global, \"IVP\") -> {regular, \"application/vnd.immervision-ivp\"};\nt(global, \"igl\") -> {regular, \"application/vnd.igloader\"};\nt(global, \"IGL\") -> {regular, \"application/vnd.igloader\"};\nt(global, \"icm\") -> {regular, \"application/vnd.iccprofile\"};\nt(global, \"ICM\") -> {regular, \"application/vnd.iccprofile\"};\nt(global, \"icc\") -> {regular, \"application/vnd.iccprofile\"};\nt(global, \"ICC\") -> {regular, \"application/vnd.iccprofile\"};\nt(global, \"sc\") -> {regular, \"application/vnd.ibm.secure-container\"};\nt(global, \"SC\") -> {regular, \"application/vnd.ibm.secure-container\"};\nt(global, \"irm\") -> {regular, \"application/vnd.ibm.rights-management\"};\nt(global, \"IRM\") -> {regular, \"application/vnd.ibm.rights-management\"};\nt(global, \"list3820\") -> {regular, \"application/vnd.ibm.modcap\"};\nt(global, \"LIST3820\") -> {regular, \"application/vnd.ibm.modcap\"};\nt(global, \"listafp\") -> {regular, \"application/vnd.ibm.modcap\"};\nt(global, \"LISTAFP\") -> {regular, \"application/vnd.ibm.modcap\"};\nt(global, \"afp\") -> {regular, \"application/vnd.ibm.modcap\"};\nt(global, \"AFP\") -> {regular, \"application/vnd.ibm.modcap\"};\nt(global, \"mpy\") -> {regular, \"application/vnd.ibm.minipay\"};\nt(global, \"MPY\") -> {regular, \"application/vnd.ibm.minipay\"};\nt(global, \"x3d\") -> {regular, \"model/x3d+xml\"};\nt(global, \"X3D\") -> {regular, \"model/x3d+xml\"};\nt(global, \"sfd-hdstx\") -> {regular, \"application/vnd.hydrostatix.sof-data\"};\nt(global, \"SFD-HDSTX\") -> {regular, \"application/vnd.hydrostatix.sof-data\"};\nt(global, \"pclxl\") -> {regular, \"application/vnd.hp-pclxl\"};\nt(global, \"PCLXL\") -> {regular, \"application/vnd.hp-pclxl\"};\nt(global, \"pcl\") -> {regular, \"application/vnd.hp-pcl\"};\nt(global, \"PCL\") -> {regular, \"application/vnd.hp-pcl\"};\nt(global, \"jlt\") -> {regular, \"application/vnd.hp-jlyt\"};\nt(global, \"JLT\") -> {regular, \"application/vnd.hp-jlyt\"};\nt(global, \"hps\") -> {regular, \"application/vnd.hp-hps\"};\nt(global, \"HPS\") -> {regular, \"application/vnd.hp-hps\"};\nt(global, \"hpid\") -> {regular, \"application/vnd.hp-hpid\"};\nt(global, \"HPID\") -> {regular, \"application/vnd.hp-hpid\"};\nt(global, \"hpgl\") -> {regular, \"application/vnd.hp-hpgl\"};\nt(global, \"HPGL\") -> {regular, \"application/vnd.hp-hpgl\"};\nt(global, \"les\") -> {regular, \"application/vnd.hhe.lesson-player\"};\nt(global, \"LES\") -> {regular, \"application/vnd.hhe.lesson-player\"};\nt(global, \"hbci\") -> {regular, \"application/vnd.hbci\"};\nt(global, \"HBCI\") -> {regular, \"application/vnd.hbci\"};\nt(global, \"zmm\") -> {regular, \"application/vnd.handheld-entertainment+xml\"};\nt(global, \"ZMM\") -> {regular, \"application/vnd.handheld-entertainment+xml\"};\nt(global, \"hal\") -> {regular, \"application/vnd.hal+xml\"};\nt(global, \"HAL\") -> {regular, \"application/vnd.hal+xml\"};\nt(global, \"vcg\") -> {regular, \"application/vnd.groove-vcard\"};\nt(global, \"VCG\") -> {regular, \"application/vnd.groove-vcard\"};\nt(global, \"tpl\") -> {regular, \"application/vnd.groove-tool-template\"};\nt(global, \"TPL\") -> {regular, \"application/vnd.groove-tool-template\"};\nt(global, \"gtm\") -> {regular, \"application/vnd.groove-tool-message\"};\nt(global, \"GTM\") -> {regular, \"application/vnd.groove-tool-message\"};\nt(global, \"grv\") -> {regular, \"application/vnd.groove-injector\"};\nt(global, \"GRV\") -> {regular, \"application/vnd.groove-injector\"};\nt(global, \"gim\") -> {regular, \"application/vnd.groove-identity-message\"};\nt(global, \"GIM\") -> {regular, \"application/vnd.groove-identity-message\"};\nt(global, \"ghf\") -> {regular, \"application/vnd.groove-help\"};\nt(global, \"GHF\") -> {regular, \"application/vnd.groove-help\"};\nt(global, \"gac\") -> {regular, \"application/vnd.groove-account\"};\nt(global, \"GAC\") -> {regular, \"application/vnd.groove-account\"};\nt(global, \"gqs\") -> {regular, \"application/vnd.grafeq\"};\nt(global, \"GQS\") -> {regular, \"application/vnd.grafeq\"};\nt(global, \"gqf\") -> {regular, \"application/vnd.grafeq\"};\nt(global, \"GQF\") -> {regular, \"application/vnd.grafeq\"};\nt(global, \"kmz\") -> {regular, \"application/vnd.google-earth.kmz\"};\nt(global, \"KMZ\") -> {regular, \"application/vnd.google-earth.kmz\"};\nt(global, \"kml\") -> {regular, \"application/vnd.google-earth.kml+xml\"};\nt(global, \"KML\") -> {regular, \"application/vnd.google-earth.kml+xml\"};\nt(global, \"gmx\") -> {regular, \"application/vnd.gmx\"};\nt(global, \"GMX\") -> {regular, \"application/vnd.gmx\"};\nt(global, \"g3w\") -> {regular, \"application/vnd.geospace\"};\nt(global, \"G3W\") -> {regular, \"application/vnd.geospace\"};\nt(global, \"g2w\") -> {regular, \"application/vnd.geoplan\"};\nt(global, \"G2W\") -> {regular, \"application/vnd.geoplan\"};\nt(global, \"gxt\") -> {regular, \"application/vnd.geonext\"};\nt(global, \"GXT\") -> {regular, \"application/vnd.geonext\"};\nt(global, \"gre\") -> {regular, \"application/vnd.geometry-explorer\"};\nt(global, \"GRE\") -> {regular, \"application/vnd.geometry-explorer\"};\nt(global, \"gex\") -> {regular, \"application/vnd.geometry-explorer\"};\nt(global, \"GEX\") -> {regular, \"application/vnd.geometry-explorer\"};\nt(global, \"ggt\") -> {regular, \"application/vnd.geogebra.tool\"};\nt(global, \"GGT\") -> {regular, \"application/vnd.geogebra.tool\"};\nt(global, \"ggb\") -> {regular, \"application/vnd.geogebra.file\"};\nt(global, \"GGB\") -> {regular, \"application/vnd.geogebra.file\"};\nt(global, \"txd\") -> {regular, \"application/vnd.genomatix.tuxedo\"};\nt(global, \"TXD\") -> {regular, \"application/vnd.genomatix.tuxedo\"};\nt(global, \"fzs\") -> {regular, \"application/vnd.fuzzysheet\"};\nt(global, \"FZS\") -> {regular, \"application/vnd.fuzzysheet\"};\nt(global, \"xbd\") -> {regular, \"application/vnd.fujixerox.docuworks.binder\"};\nt(global, \"XBD\") -> {regular, \"application/vnd.fujixerox.docuworks.binder\"};\nt(global, \"xdw\") -> {regular, \"application/vnd.fujixerox.docuworks\"};\nt(global, \"XDW\") -> {regular, \"application/vnd.fujixerox.docuworks\"};\nt(global, \"ddd\") -> {regular, \"application/vnd.fujixerox.ddd\"};\nt(global, \"DDD\") -> {regular, \"application/vnd.fujixerox.ddd\"};\nt(global, \"bh2\") -> {regular, \"application/vnd.fujitsu.oasysprs\"};\nt(global, \"BH2\") -> {regular, \"application/vnd.fujitsu.oasysprs\"};\nt(global, \"fg5\") -> {regular, \"application/vnd.fujitsu.oasysgp\"};\nt(global, \"FG5\") -> {regular, \"application/vnd.fujitsu.oasysgp\"};\nt(global, \"oa3\") -> {regular, \"application/vnd.fujitsu.oasys3\"};\nt(global, \"OA3\") -> {regular, \"application/vnd.fujitsu.oasys3\"};\nt(global, \"oa2\") -> {regular, \"application/vnd.fujitsu.oasys2\"};\nt(global, \"OA2\") -> {regular, \"application/vnd.fujitsu.oasys2\"};\nt(global, \"oas\") -> {regular, \"application/vnd.fujitsu.oasys\"};\nt(global, \"OAS\") -> {regular, \"application/vnd.fujitsu.oasys\"};\nt(global, \"fsc\") -> {regular, \"application/vnd.fsc.weblaunch\"};\nt(global, \"FSC\") -> {regular, \"application/vnd.fsc.weblaunch\"};\nt(global, \"ltf\") -> {regular, \"application/vnd.frogans.ltf\"};\nt(global, \"LTF\") -> {regular, \"application/vnd.frogans.ltf\"};\nt(global, \"fnc\") -> {regular, \"application/vnd.frogans.fnc\"};\nt(global, \"FNC\") -> {regular, \"application/vnd.frogans.fnc\"};\nt(global, \"book\") -> {regular, \"application/vnd.framemaker\"};\nt(global, \"BOOK\") -> {regular, \"application/vnd.framemaker\"};\nt(global, \"maker\") -> {regular, \"application/vnd.framemaker\"};\nt(global, \"MAKER\") -> {regular, \"application/vnd.framemaker\"};\nt(global, \"frame\") -> {regular, \"application/vnd.framemaker\"};\nt(global, \"FRAME\") -> {regular, \"application/vnd.framemaker\"};\nt(global, \"fm\") -> {regular, \"application/vnd.framemaker\"};\nt(global, \"FM\") -> {regular, \"application/vnd.framemaker\"};\nt(global, \"ftc\") -> {regular, \"application/vnd.fluxtime.clip\"};\nt(global, \"FTC\") -> {regular, \"application/vnd.fluxtime.clip\"};\nt(global, \"gph\") -> {regular, \"application/vnd.flographit\"};\nt(global, \"GPH\") -> {regular, \"application/vnd.flographit\"};\nt(global, \"dataless\") -> {regular, \"application/vnd.fdsn.seed\"};\nt(global, \"DATALESS\") -> {regular, \"application/vnd.fdsn.seed\"};\nt(global, \"seed\") -> {regular, \"application/vnd.fdsn.seed\"};\nt(global, \"SEED\") -> {regular, \"application/vnd.fdsn.seed\"};\nt(global, \"mseed\") -> {regular, \"application/vnd.fdsn.mseed\"};\nt(global, \"MSEED\") -> {regular, \"application/vnd.fdsn.mseed\"};\nt(global, \"fdf\") -> {regular, \"application/vnd.fdf\"};\nt(global, \"FDF\") -> {regular, \"application/vnd.fdf\"};\nt(global, \"ez3\") -> {regular, \"application/vnd.ezpix-package\"};\nt(global, \"EZ3\") -> {regular, \"application/vnd.ezpix-package\"};\nt(global, \"ez2\") -> {regular, \"application/vnd.ezpix-album\"};\nt(global, \"EZ2\") -> {regular, \"application/vnd.ezpix-album\"};\nt(global, \"et3\") -> {regular, \"application/vnd.eszigno3+xml\"};\nt(global, \"ET3\") -> {regular, \"application/vnd.eszigno3+xml\"};\nt(global, \"es3\") -> {regular, \"application/vnd.eszigno3+xml\"};\nt(global, \"ES3\") -> {regular, \"application/vnd.eszigno3+xml\"};\nt(global, \"ssf\") -> {regular, \"application/vnd.epson.ssf\"};\nt(global, \"SSF\") -> {regular, \"application/vnd.epson.ssf\"};\nt(global, \"slt\") -> {regular, \"application/vnd.epson.salt\"};\nt(global, \"SLT\") -> {regular, \"application/vnd.epson.salt\"};\nt(global, \"qam\") -> {regular, \"application/vnd.epson.quickanime\"};\nt(global, \"QAM\") -> {regular, \"application/vnd.epson.quickanime\"};\nt(global, \"msf\") -> {regular, \"application/vnd.epson.msf\"};\nt(global, \"MSF\") -> {regular, \"application/vnd.epson.msf\"};\nt(global, \"esf\") -> {regular, \"application/vnd.epson.esf\"};\nt(global, \"ESF\") -> {regular, \"application/vnd.epson.esf\"};\nt(global, \"nml\") -> {regular, \"application/vnd.enliven\"};\nt(global, \"NML\") -> {regular, \"application/vnd.enliven\"};\nt(global, \"mag\") -> {regular, \"application/vnd.ecowin.chart\"};\nt(global, \"MAG\") -> {regular, \"application/vnd.ecowin.chart\"};\nt(global, \"geo\") -> {regular, \"application/vnd.dynageo\"};\nt(global, \"GEO\") -> {regular, \"application/vnd.dynageo\"};\nt(global, \"svc\") -> {regular, \"application/vnd.dvb.service\"};\nt(global, \"SVC\") -> {regular, \"application/vnd.dvb.service\"};\nt(global, \"ait\") -> {regular, \"application/vnd.dvb.ait\"};\nt(global, \"AIT\") -> {regular, \"application/vnd.dvb.ait\"};\nt(global, \"kpxx\") -> {regular, \"application/vnd.ds-keypoint\"};\nt(global, \"KPXX\") -> {regular, \"application/vnd.ds-keypoint\"};\nt(global, \"dfac\") -> {regular, \"application/vnd.dreamfactory\"};\nt(global, \"DFAC\") -> {regular, \"application/vnd.dreamfactory\"};\nt(global, \"dpg\") -> {regular, \"application/vnd.dpgraph\"};\nt(global, \"DPG\") -> {regular, \"application/vnd.dpgraph\"};\nt(global, \"mlp\") -> {regular, \"application/vnd.dolby.mlp\"};\nt(global, \"MLP\") -> {regular, \"application/vnd.dolby.mlp\"};\nt(global, \"dna\") -> {regular, \"application/vnd.dna\"};\nt(global, \"DNA\") -> {regular, \"application/vnd.dna\"};\nt(global, \"fe_launch\") -> {regular, \"application/vnd.denovo.fcselayout-link\"};\nt(global, \"FE_LAUNCH\") -> {regular, \"application/vnd.denovo.fcselayout-link\"};\nt(global, \"uvvz\") -> {regular, \"application/vnd.dece.zip\"};\nt(global, \"UVVZ\") -> {regular, \"application/vnd.dece.zip\"};\nt(global, \"uvz\") -> {regular, \"application/vnd.dece.zip\"};\nt(global, \"UVZ\") -> {regular, \"application/vnd.dece.zip\"};\nt(global, \"uvvx\") -> {regular, \"application/vnd.dece.unspecified\"};\nt(global, \"UVVX\") -> {regular, \"application/vnd.dece.unspecified\"};\nt(global, \"uvx\") -> {regular, \"application/vnd.dece.unspecified\"};\nt(global, \"UVX\") -> {regular, \"application/vnd.dece.unspecified\"};\nt(global, \"uvvt\") -> {regular, \"application/vnd.dece.ttml+xml\"};\nt(global, \"UVVT\") -> {regular, \"application/vnd.dece.ttml+xml\"};\nt(global, \"uvt\") -> {regular, \"application/vnd.dece.ttml+xml\"};\nt(global, \"UVT\") -> {regular, \"application/vnd.dece.ttml+xml\"};\nt(global, \"uvvd\") -> {regular, \"application/vnd.dece.data\"};\nt(global, \"UVVD\") -> {regular, \"application/vnd.dece.data\"};\nt(global, \"uvd\") -> {regular, \"application/vnd.dece.data\"};\nt(global, \"UVD\") -> {regular, \"application/vnd.dece.data\"};\nt(global, \"uvvf\") -> {regular, \"application/vnd.dece.data\"};\nt(global, \"UVVF\") -> {regular, \"application/vnd.dece.data\"};\nt(global, \"uvf\") -> {regular, \"application/vnd.dece.data\"};\nt(global, \"UVF\") -> {regular, \"application/vnd.dece.data\"};\nt(global, \"rdz\") -> {regular, \"application/vnd.data-vision.rdz\"};\nt(global, \"RDZ\") -> {regular, \"application/vnd.data-vision.rdz\"};\nt(global, \"dart\") -> {regular, \"application/vnd.dart\"};\nt(global, \"DART\") -> {regular, \"application/vnd.dart\"};\nt(global, \"pcurl\") -> {regular, \"application/vnd.curl.pcurl\"};\nt(global, \"PCURL\") -> {regular, \"application/vnd.curl.pcurl\"};\nt(global, \"car\") -> {regular, \"application/vnd.curl.car\"};\nt(global, \"CAR\") -> {regular, \"application/vnd.curl.car\"};\nt(global, \"ppd\") -> {regular, \"application/vnd.cups-ppd\"};\nt(global, \"PPD\") -> {regular, \"application/vnd.cups-ppd\"};\nt(global, \"pml\") -> {regular, \"application/vnd.ctc-posml\"};\nt(global, \"PML\") -> {regular, \"application/vnd.ctc-posml\"};\nt(global, \"wbs\") -> {regular, \"application/vnd.criticaltools.wbs+xml\"};\nt(global, \"WBS\") -> {regular, \"application/vnd.criticaltools.wbs+xml\"};\nt(global, \"clkw\") -> {regular, \"application/vnd.crick.clicker.wordbank\"};\nt(global, \"CLKW\") -> {regular, \"application/vnd.crick.clicker.wordbank\"};\nt(global, \"clkt\") -> {regular, \"application/vnd.crick.clicker.template\"};\nt(global, \"CLKT\") -> {regular, \"application/vnd.crick.clicker.template\"};\nt(global, \"clkp\") -> {regular, \"application/vnd.crick.clicker.palette\"};\nt(global, \"CLKP\") -> {regular, \"application/vnd.crick.clicker.palette\"};\nt(global, \"clkk\") -> {regular, \"application/vnd.crick.clicker.keyboard\"};\nt(global, \"CLKK\") -> {regular, \"application/vnd.crick.clicker.keyboard\"};\nt(global, \"clkx\") -> {regular, \"application/vnd.crick.clicker\"};\nt(global, \"CLKX\") -> {regular, \"application/vnd.crick.clicker\"};\nt(global, \"cmc\") -> {regular, \"application/vnd.cosmocaller\"};\nt(global, \"CMC\") -> {regular, \"application/vnd.cosmocaller\"};\nt(global, \"cdbcmsg\") -> {regular, \"application/vnd.contact.cmsg\"};\nt(global, \"CDBCMSG\") -> {regular, \"application/vnd.contact.cmsg\"};\nt(global, \"csp\") -> {regular, \"application/vnd.commonspace\"};\nt(global, \"CSP\") -> {regular, \"application/vnd.commonspace\"};\nt(global, \"c11amz\") -> {regular, \"application/vnd.cluetrust.cartomobile-config-pkg\"};\nt(global, \"C11AMZ\") -> {regular, \"application/vnd.cluetrust.cartomobile-config-pkg\"};\nt(global, \"c11amc\") -> {regular, \"application/vnd.cluetrust.cartomobile-config\"};\nt(global, \"C11AMC\") -> {regular, \"application/vnd.cluetrust.cartomobile-config\"};\nt(global, \"c4u\") -> {regular, \"application/vnd.clonk.c4group\"};\nt(global, \"C4U\") -> {regular, \"application/vnd.clonk.c4group\"};\nt(global, \"c4p\") -> {regular, \"application/vnd.clonk.c4group\"};\nt(global, \"C4P\") -> {regular, \"application/vnd.clonk.c4group\"};\nt(global, \"c4f\") -> {regular, \"application/vnd.clonk.c4group\"};\nt(global, \"C4F\") -> {regular, \"application/vnd.clonk.c4group\"};\nt(global, \"c4d\") -> {regular, \"application/vnd.clonk.c4group\"};\nt(global, \"C4D\") -> {regular, \"application/vnd.clonk.c4group\"};\nt(global, \"c4g\") -> {regular, \"application/vnd.clonk.c4group\"};\nt(global, \"C4G\") -> {regular, \"application/vnd.clonk.c4group\"};\nt(global, \"rp9\") -> {regular, \"application/vnd.cloanto.rp9\"};\nt(global, \"RP9\") -> {regular, \"application/vnd.cloanto.rp9\"};\nt(global, \"cla\") -> {regular, \"application/vnd.claymore\"};\nt(global, \"CLA\") -> {regular, \"application/vnd.claymore\"};\nt(global, \"cdy\") -> {regular, \"application/vnd.cinderella\"};\nt(global, \"CDY\") -> {regular, \"application/vnd.cinderella\"};\nt(global, \"mmd\") -> {regular, \"application/vnd.chipnuts.karaoke-mmd\"};\nt(global, \"MMD\") -> {regular, \"application/vnd.chipnuts.karaoke-mmd\"};\nt(global, \"cdxml\") -> {regular, \"application/vnd.chemdraw+xml\"};\nt(global, \"CDXML\") -> {regular, \"application/vnd.chemdraw+xml\"};\nt(global, \"rep\") -> {regular, \"application/vnd.businessobjects\"};\nt(global, \"REP\") -> {regular, \"application/vnd.businessobjects\"};\nt(global, \"bmi\") -> {regular, \"application/vnd.bmi\"};\nt(global, \"BMI\") -> {regular, \"application/vnd.bmi\"};\nt(global, \"mpm\") -> {regular, \"application/vnd.blueice.multipass\"};\nt(global, \"MPM\") -> {regular, \"application/vnd.blueice.multipass\"};\nt(global, \"aep\") -> {regular, \"application/vnd.audiograph\"};\nt(global, \"AEP\") -> {regular, \"application/vnd.audiograph\"};\nt(global, \"iota\") -> {regular, \"application/vnd.astraea-software.iota\"};\nt(global, \"IOTA\") -> {regular, \"application/vnd.astraea-software.iota\"};\nt(global, \"swi\") -> {regular, \"application/vnd.aristanetworks.swi\"};\nt(global, \"SWI\") -> {regular, \"application/vnd.aristanetworks.swi\"};\nt(global, \"m3u8\") -> {regular, \"application/vnd.apple.mpegurl\"};\nt(global, \"M3U8\") -> {regular, \"application/vnd.apple.mpegurl\"};\nt(global, \"mpkg\") -> {regular, \"application/vnd.apple.installer+xml\"};\nt(global, \"MPKG\") -> {regular, \"application/vnd.apple.installer+xml\"};\nt(global, \"atx\") -> {regular, \"application/vnd.antix.game-component\"};\nt(global, \"ATX\") -> {regular, \"application/vnd.antix.game-component\"};\nt(global, \"fti\") -> {regular, \"application/vnd.anser-web-funds-transfer-initiation\"};\nt(global, \"FTI\") -> {regular, \"application/vnd.anser-web-funds-transfer-initiation\"};\nt(global, \"cii\") -> {regular, \"application/vnd.anser-web-certificate-issue-initiation\"};\nt(global, \"CII\") -> {regular, \"application/vnd.anser-web-certificate-issue-initiation\"};\nt(global, \"apk\") -> {regular, \"application/vnd.android.package-archive\"};\nt(global, \"APK\") -> {regular, \"application/vnd.android.package-archive\"};\nt(global, \"ami\") -> {regular, \"application/vnd.amiga.ami\"};\nt(global, \"AMI\") -> {regular, \"application/vnd.amiga.ami\"};\nt(global, \"acc\") -> {regular, \"application/vnd.americandynamics.acc\"};\nt(global, \"ACC\") -> {regular, \"application/vnd.americandynamics.acc\"};\nt(global, \"azw\") -> {regular, \"application/vnd.amazon.ebook\"};\nt(global, \"AZW\") -> {regular, \"application/vnd.amazon.ebook\"};\nt(global, \"azs\") -> {regular, \"application/vnd.airzip.filesecure.azs\"};\nt(global, \"AZS\") -> {regular, \"application/vnd.airzip.filesecure.azs\"};\nt(global, \"azf\") -> {regular, \"application/vnd.airzip.filesecure.azf\"};\nt(global, \"AZF\") -> {regular, \"application/vnd.airzip.filesecure.azf\"};\nt(global, \"ahead\") -> {regular, \"application/vnd.ahead.space\"};\nt(global, \"AHEAD\") -> {regular, \"application/vnd.ahead.space\"};\nt(global, \"xfdf\") -> {regular, \"application/vnd.adobe.xfdf\"};\nt(global, \"XFDF\") -> {regular, \"application/vnd.adobe.xfdf\"};\nt(global, \"xdp\") -> {regular, \"application/vnd.adobe.xdp+xml\"};\nt(global, \"XDP\") -> {regular, \"application/vnd.adobe.xdp+xml\"};\nt(global, \"fxpl\") -> {regular, \"application/vnd.adobe.fxp\"};\nt(global, \"FXPL\") -> {regular, \"application/vnd.adobe.fxp\"};\nt(global, \"fxp\") -> {regular, \"application/vnd.adobe.fxp\"};\nt(global, \"FXP\") -> {regular, \"application/vnd.adobe.fxp\"};\nt(global, \"fcdt\") -> {regular, \"application/vnd.adobe.formscentral.fcdt\"};\nt(global, \"FCDT\") -> {regular, \"application/vnd.adobe.formscentral.fcdt\"};\nt(global, \"air\") -> {regular, \"application/vnd.adobe.air-application-installer-package+zip\"};\nt(global, \"AIR\") -> {regular, \"application/vnd.adobe.air-application-installer-package+zip\"};\nt(global, \"acutc\") -> {regular, \"application/vnd.acucorp\"};\nt(global, \"ACUTC\") -> {regular, \"application/vnd.acucorp\"};\nt(global, \"atc\") -> {regular, \"application/vnd.acucorp\"};\nt(global, \"ATC\") -> {regular, \"application/vnd.acucorp\"};\nt(global, \"acu\") -> {regular, \"application/vnd.acucobol\"};\nt(global, \"ACU\") -> {regular, \"application/vnd.acucobol\"};\nt(global, \"imp\") -> {regular, \"application/vnd.accpac.simply.imp\"};\nt(global, \"IMP\") -> {regular, \"application/vnd.accpac.simply.imp\"};\nt(global, \"aso\") -> {regular, \"application/vnd.accpac.simply.aso\"};\nt(global, \"ASO\") -> {regular, \"application/vnd.accpac.simply.aso\"};\nt(global, \"pwn\") -> {regular, \"application/vnd.3m.post-it-notes\"};\nt(global, \"PWN\") -> {regular, \"application/vnd.3m.post-it-notes\"};\nt(global, \"tcap\") -> {regular, \"application/vnd.3gpp2.tcap\"};\nt(global, \"TCAP\") -> {regular, \"application/vnd.3gpp2.tcap\"};\nt(global, \"pvb\") -> {regular, \"application/vnd.3gpp.pic-bw-var\"};\nt(global, \"PVB\") -> {regular, \"application/vnd.3gpp.pic-bw-var\"};\nt(global, \"psb\") -> {regular, \"application/vnd.3gpp.pic-bw-small\"};\nt(global, \"PSB\") -> {regular, \"application/vnd.3gpp.pic-bw-small\"};\nt(global, \"plb\") -> {regular, \"application/vnd.3gpp.pic-bw-large\"};\nt(global, \"PLB\") -> {regular, \"application/vnd.3gpp.pic-bw-large\"};\nt(global, \"tsd\") -> {regular, \"application/timestamped-data\"};\nt(global, \"TSD\") -> {regular, \"application/timestamped-data\"};\nt(global, \"tfi\") -> {regular, \"application/thraud+xml\"};\nt(global, \"TFI\") -> {regular, \"application/thraud+xml\"};\nt(global, \"teicorpus\") -> {regular, \"application/tei+xml\"};\nt(global, \"TEICORPUS\") -> {regular, \"application/tei+xml\"};\nt(global, \"tei\") -> {regular, \"application/tei+xml\"};\nt(global, \"TEI\") -> {regular, \"application/tei+xml\"};\nt(global, \"ssml\") -> {regular, \"application/ssml+xml\"};\nt(global, \"SSML\") -> {regular, \"application/ssml+xml\"};\nt(global, \"ssdl\") -> {regular, \"application/ssdl+xml\"};\nt(global, \"SSDL\") -> {regular, \"application/ssdl+xml\"};\nt(global, \"sru\") -> {regular, \"application/sru+xml\"};\nt(global, \"SRU\") -> {regular, \"application/sru+xml\"};\nt(global, \"grxml\") -> {regular, \"application/srgs+xml\"};\nt(global, \"GRXML\") -> {regular, \"application/srgs+xml\"};\nt(global, \"gram\") -> {regular, \"application/srgs\"};\nt(global, \"GRAM\") -> {regular, \"application/srgs\"};\nt(global, \"srx\") -> {regular, \"application/sparql-results+xml\"};\nt(global, \"SRX\") -> {regular, \"application/sparql-results+xml\"};\nt(global, \"rq\") -> {regular, \"application/sparql-query\"};\nt(global, \"RQ\") -> {regular, \"application/sparql-query\"};\nt(global, \"smil\") -> {regular, \"application/smil+xml\"};\nt(global, \"SMIL\") -> {regular, \"application/smil+xml\"};\nt(global, \"smi\") -> {regular, \"application/smil+xml\"};\nt(global, \"SMI\") -> {regular, \"application/smil+xml\"};\nt(global, \"shf\") -> {regular, \"application/shf+xml\"};\nt(global, \"SHF\") -> {regular, \"application/shf+xml\"};\nt(global, \"setreg\") -> {regular, \"application/set-registration-initiation\"};\nt(global, \"SETREG\") -> {regular, \"application/set-registration-initiation\"};\nt(global, \"setpay\") -> {regular, \"application/set-payment-initiation\"};\nt(global, \"SETPAY\") -> {regular, \"application/set-payment-initiation\"};\nt(global, \"sdp\") -> {regular, \"application/sdp\"};\nt(global, \"SDP\") -> {regular, \"application/sdp\"};\nt(global, \"spp\") -> {regular, \"application/scvp-vp-response\"};\nt(global, \"SPP\") -> {regular, \"application/scvp-vp-response\"};\nt(global, \"spq\") -> {regular, \"application/scvp-vp-request\"};\nt(global, \"SPQ\") -> {regular, \"application/scvp-vp-request\"};\nt(global, \"scs\") -> {regular, \"application/scvp-cv-response\"};\nt(global, \"SCS\") -> {regular, \"application/scvp-cv-response\"};\nt(global, \"scq\") -> {regular, \"application/scvp-cv-request\"};\nt(global, \"SCQ\") -> {regular, \"application/scvp-cv-request\"};\nt(global, \"sbml\") -> {regular, \"application/sbml+xml\"};\nt(global, \"SBML\") -> {regular, \"application/sbml+xml\"};\nt(global, \"rtf\") -> {regular, \"text/rtf\"};\nt(global, \"RTF\") -> {regular, \"text/rtf\"};\nt(global, \"rss\") -> {regular, \"application/rss+xml\"};\nt(global, \"RSS\") -> {regular, \"application/rss+xml\"};\nt(global, \"rsd\") -> {regular, \"application/rsd+xml\"};\nt(global, \"RSD\") -> {regular, \"application/rsd+xml\"};\nt(global, \"roa\") -> {regular, \"application/rpki-roa\"};\nt(global, \"ROA\") -> {regular, \"application/rpki-roa\"};\nt(global, \"mft\") -> {regular, \"application/rpki-manifest\"};\nt(global, \"MFT\") -> {regular, \"application/rpki-manifest\"};\nt(global, \"gbr\") -> {regular, \"application/rpki-ghostbusters\"};\nt(global, \"GBR\") -> {regular, \"application/rpki-ghostbusters\"};\nt(global, \"rs\") -> {regular, \"application/rls-services+xml\"};\nt(global, \"RS\") -> {regular, \"application/rls-services+xml\"};\nt(global, \"rld\") -> {regular, \"application/resource-lists-diff+xml\"};\nt(global, \"RLD\") -> {regular, \"application/resource-lists-diff+xml\"};\nt(global, \"rl\") -> {regular, \"application/resource-lists+xml\"};\nt(global, \"RL\") -> {regular, \"application/resource-lists+xml\"};\nt(global, \"rnc\") -> {regular, \"application/relax-ng-compact-syntax\"};\nt(global, \"RNC\") -> {regular, \"application/relax-ng-compact-syntax\"};\nt(global, \"rif\") -> {regular, \"application/reginfo+xml\"};\nt(global, \"RIF\") -> {regular, \"application/reginfo+xml\"};\nt(global, \"rdf\") -> {regular, \"application/rdf+xml\"};\nt(global, \"RDF\") -> {regular, \"application/rdf+xml\"};\nt(global, \"pskcxml\") -> {regular, \"application/pskc+xml\"};\nt(global, \"PSKCXML\") -> {regular, \"application/pskc+xml\"};\nt(global, \"cww\") -> {regular, \"application/prs.cww\"};\nt(global, \"CWW\") -> {regular, \"application/prs.cww\"};\nt(global, \"ps\") -> {regular, \"application/postscript\"};\nt(global, \"PS\") -> {regular, \"application/postscript\"};\nt(global, \"eps\") -> {regular, \"application/postscript\"};\nt(global, \"EPS\") -> {regular, \"application/postscript\"};\nt(global, \"ai\") -> {regular, \"application/postscript\"};\nt(global, \"AI\") -> {regular, \"application/postscript\"};\nt(global, \"pls\") -> {regular, \"application/pls+xml\"};\nt(global, \"PLS\") -> {regular, \"application/pls+xml\"};\nt(global, \"pki\") -> {regular, \"application/pkixcmp\"};\nt(global, \"PKI\") -> {regular, \"application/pkixcmp\"};\nt(global, \"pkipath\") -> {regular, \"application/pkix-pkipath\"};\nt(global, \"PKIPATH\") -> {regular, \"application/pkix-pkipath\"};\nt(global, \"crl\") -> {regular, \"application/pkix-crl\"};\nt(global, \"CRL\") -> {regular, \"application/pkix-crl\"};\nt(global, \"cer\") -> {regular, \"application/pkix-cert\"};\nt(global, \"CER\") -> {regular, \"application/pkix-cert\"};\nt(global, \"ac\") -> {regular, \"application/pkix-attr-cert\"};\nt(global, \"AC\") -> {regular, \"application/pkix-attr-cert\"};\nt(global, \"p8\") -> {regular, \"application/pkcs8\"};\nt(global, \"P8\") -> {regular, \"application/pkcs8\"};\nt(global, \"p7s\") -> {regular, \"application/pkcs7-signature\"};\nt(global, \"P7S\") -> {regular, \"application/pkcs7-signature\"};\nt(global, \"p7c\") -> {regular, \"application/pkcs7-mime\"};\nt(global, \"P7C\") -> {regular, \"application/pkcs7-mime\"};\nt(global, \"p7m\") -> {regular, \"application/pkcs7-mime\"};\nt(global, \"P7M\") -> {regular, \"application/pkcs7-mime\"};\nt(global, \"p10\") -> {regular, \"application/pkcs10\"};\nt(global, \"P10\") -> {regular, \"application/pkcs10\"};\nt(global, \"prf\") -> {regular, \"application/pics-rules\"};\nt(global, \"PRF\") -> {regular, \"application/pics-rules\"};\nt(global, \"sig\") -> {regular, \"application/pgp-signature\"};\nt(global, \"SIG\") -> {regular, \"application/pgp-signature\"};\nt(global, \"asc\") -> {regular, \"application/pgp-signature\"};\nt(global, \"ASC\") -> {regular, \"application/pgp-signature\"};\nt(global, \"pgp\") -> {regular, \"application/pgp-encrypted\"};\nt(global, \"PGP\") -> {regular, \"application/pgp-encrypted\"};\nt(global, \"pdf\") -> {regular, \"application/pdf\"};\nt(global, \"PDF\") -> {regular, \"application/pdf\"};\nt(global, \"xer\") -> {regular, \"application/patch-ops-error+xml\"};\nt(global, \"XER\") -> {regular, \"application/patch-ops-error+xml\"};\nt(global, \"oxps\") -> {regular, \"application/oxps\"};\nt(global, \"OXPS\") -> {regular, \"application/oxps\"};\nt(global, \"onepkg\") -> {regular, \"application/onenote\"};\nt(global, \"ONEPKG\") -> {regular, \"application/onenote\"};\nt(global, \"onetmp\") -> {regular, \"application/onenote\"};\nt(global, \"ONETMP\") -> {regular, \"application/onenote\"};\nt(global, \"onetoc2\") -> {regular, \"application/onenote\"};\nt(global, \"ONETOC2\") -> {regular, \"application/onenote\"};\nt(global, \"onetoc\") -> {regular, \"application/onenote\"};\nt(global, \"ONETOC\") -> {regular, \"application/onenote\"};\nt(global, \"omdoc\") -> {regular, \"application/omdoc+xml\"};\nt(global, \"OMDOC\") -> {regular, \"application/omdoc+xml\"};\nt(global, \"ogx\") -> {regular, \"application/ogg\"};\nt(global, \"OGX\") -> {regular, \"application/ogg\"};\nt(global, \"opf\") -> {regular, \"application/oebps-package+xml\"};\nt(global, \"OPF\") -> {regular, \"application/oebps-package+xml\"};\nt(global, \"oda\") -> {regular, \"application/oda\"};\nt(global, \"ODA\") -> {regular, \"application/oda\"};\nt(global, \"deploy\") -> {regular, \"application/octet-stream\"};\nt(global, \"DEPLOY\") -> {regular, \"application/octet-stream\"};\nt(global, \"elc\") -> {regular, \"application/octet-stream\"};\nt(global, \"ELC\") -> {regular, \"application/octet-stream\"};\nt(global, \"dump\") -> {regular, \"application/octet-stream\"};\nt(global, \"DUMP\") -> {regular, \"application/octet-stream\"};\nt(global, \"bpk\") -> {regular, \"application/octet-stream\"};\nt(global, \"BPK\") -> {regular, \"application/octet-stream\"};\nt(global, \"pkg\") -> {regular, \"application/octet-stream\"};\nt(global, \"PKG\") -> {regular, \"application/octet-stream\"};\nt(global, \"distz\") -> {regular, \"application/octet-stream\"};\nt(global, \"DISTZ\") -> {regular, \"application/octet-stream\"};\nt(global, \"dist\") -> {regular, \"application/octet-stream\"};\nt(global, \"DIST\") -> {regular, \"application/octet-stream\"};\nt(global, \"so\") -> {regular, \"application/octet-stream\"};\nt(global, \"SO\") -> {regular, \"application/octet-stream\"};\nt(global, \"mar\") -> {regular, \"application/octet-stream\"};\nt(global, \"MAR\") -> {regular, \"application/octet-stream\"};\nt(global, \"lrf\") -> {regular, \"application/octet-stream\"};\nt(global, \"LRF\") -> {regular, \"application/octet-stream\"};\nt(global, \"dms\") -> {regular, \"application/octet-stream\"};\nt(global, \"DMS\") -> {regular, \"application/octet-stream\"};\nt(global, \"bin\") -> {regular, \"application/octet-stream\"};\nt(global, \"BIN\") -> {regular, \"application/octet-stream\"};\nt(global, \"mxf\") -> {regular, \"application/mxf\"};\nt(global, \"MXF\") -> {regular, \"application/mxf\"};\nt(global, \"dot\") -> {regular, \"application/msword\"};\nt(global, \"DOT\") -> {regular, \"application/msword\"};\nt(global, \"doc\") -> {regular, \"application/msword\"};\nt(global, \"DOC\") -> {regular, \"application/msword\"};\nt(global, \"mp4s\") -> {regular, \"application/mp4\"};\nt(global, \"MP4S\") -> {regular, \"application/mp4\"};\nt(global, \"mp21\") -> {regular, \"application/mp21\"};\nt(global, \"MP21\") -> {regular, \"application/mp21\"};\nt(global, \"m21\") -> {regular, \"application/mp21\"};\nt(global, \"M21\") -> {regular, \"application/mp21\"};\nt(global, \"mods\") -> {regular, \"application/mods+xml\"};\nt(global, \"MODS\") -> {regular, \"application/mods+xml\"};\nt(global, \"mets\") -> {regular, \"application/mets+xml\"};\nt(global, \"METS\") -> {regular, \"application/mets+xml\"};\nt(global, \"meta4\") -> {regular, \"application/metalink4+xml\"};\nt(global, \"META4\") -> {regular, \"application/metalink4+xml\"};\nt(global, \"metalink\") -> {regular, \"application/metalink+xml\"};\nt(global, \"METALINK\") -> {regular, \"application/metalink+xml\"};\nt(global, \"mscml\") -> {regular, \"application/mediaservercontrol+xml\"};\nt(global, \"MSCML\") -> {regular, \"application/mediaservercontrol+xml\"};\nt(global, \"mbox\") -> {regular, \"application/mbox\"};\nt(global, \"MBOX\") -> {regular, \"application/mbox\"};\nt(global, \"mathml\") -> {regular, \"application/mathml+xml\"};\nt(global, \"MATHML\") -> {regular, \"application/mathml+xml\"};\nt(global, \"mb\") -> {regular, \"application/mathematica\"};\nt(global, \"MB\") -> {regular, \"application/mathematica\"};\nt(global, \"nb\") -> {regular, \"application/mathematica\"};\nt(global, \"NB\") -> {regular, \"application/mathematica\"};\nt(global, \"ma\") -> {regular, \"application/mathematica\"};\nt(global, \"MA\") -> {regular, \"application/mathematica\"};\nt(global, \"mrcx\") -> {regular, \"application/marcxml+xml\"};\nt(global, \"MRCX\") -> {regular, \"application/marcxml+xml\"};\nt(global, \"mrc\") -> {regular, \"application/marc\"};\nt(global, \"MRC\") -> {regular, \"application/marc\"};\nt(global, \"mads\") -> {regular, \"application/mads+xml\"};\nt(global, \"MADS\") -> {regular, \"application/mads+xml\"};\nt(global, \"cpt\") -> {regular, \"application/mac-compactpro\"};\nt(global, \"CPT\") -> {regular, \"application/mac-compactpro\"};\nt(global, \"hqx\") -> {regular, \"application/mac-binhex40\"};\nt(global, \"HQX\") -> {regular, \"application/mac-binhex40\"};\nt(global, \"lostxml\") -> {regular, \"application/lost+xml\"};\nt(global, \"LOSTXML\") -> {regular, \"application/lost+xml\"};\nt(global, \"jsonml\") -> {regular, \"application/jsonml+json\"};\nt(global, \"JSONML\") -> {regular, \"application/jsonml+json\"};\nt(global, \"json\") -> {regular, \"application/json\"};\nt(global, \"JSON\") -> {regular, \"application/json\"};\nt(global, \"js\") -> {regular, \"application/x-javascript\"};\nt(global, \"JS\") -> {regular, \"application/x-javascript\"};\nt(global, \"class\") -> {regular, \"application/java-vm\"};\nt(global, \"CLASS\") -> {regular, \"application/java-vm\"};\nt(global, \"ser\") -> {regular, \"application/java-serialized-object\"};\nt(global, \"SER\") -> {regular, \"application/java-serialized-object\"};\nt(global, \"jar\") -> {regular, \"application/java-archive\"};\nt(global, \"JAR\") -> {regular, \"application/java-archive\"};\nt(global, \"ipfix\") -> {regular, \"application/ipfix\"};\nt(global, \"IPFIX\") -> {regular, \"application/ipfix\"};\nt(global, \"inkml\") -> {regular, \"application/inkml+xml\"};\nt(global, \"INKML\") -> {regular, \"application/inkml+xml\"};\nt(global, \"ink\") -> {regular, \"application/inkml+xml\"};\nt(global, \"INK\") -> {regular, \"application/inkml+xml\"};\nt(global, \"stk\") -> {regular, \"application/hyperstudio\"};\nt(global, \"STK\") -> {regular, \"application/hyperstudio\"};\nt(global, \"gxf\") -> {regular, \"application/gxf\"};\nt(global, \"GXF\") -> {regular, \"application/gxf\"};\nt(global, \"gpx\") -> {regular, \"application/gpx+xml\"};\nt(global, \"GPX\") -> {regular, \"application/gpx+xml\"};\nt(global, \"gml\") -> {regular, \"application/gml+xml\"};\nt(global, \"GML\") -> {regular, \"application/gml+xml\"};\nt(global, \"woff\") -> {regular, \"application/font-woff\"};\nt(global, \"WOFF\") -> {regular, \"application/font-woff\"};\nt(global, \"pfr\") -> {regular, \"application/font-tdpfr\"};\nt(global, \"PFR\") -> {regular, \"application/font-tdpfr\"};\nt(global, \"exi\") -> {regular, \"application/exi\"};\nt(global, \"EXI\") -> {regular, \"application/exi\"};\nt(global, \"epub\") -> {regular, \"application/epub+zip\"};\nt(global, \"EPUB\") -> {regular, \"application/epub+zip\"};\nt(global, \"emma\") -> {regular, \"application/emma+xml\"};\nt(global, \"EMMA\") -> {regular, \"application/emma+xml\"};\nt(global, \"ecma\") -> {regular, \"application/ecmascript\"};\nt(global, \"ECMA\") -> {regular, \"application/ecmascript\"};\nt(global, \"xdssc\") -> {regular, \"application/dssc+xml\"};\nt(global, \"XDSSC\") -> {regular, \"application/dssc+xml\"};\nt(global, \"dssc\") -> {regular, \"application/dssc+der\"};\nt(global, \"DSSC\") -> {regular, \"application/dssc+der\"};\nt(global, \"dbk\") -> {regular, \"application/docbook+xml\"};\nt(global, \"DBK\") -> {regular, \"application/docbook+xml\"};\nt(global, \"davmount\") -> {regular, \"application/davmount+xml\"};\nt(global, \"DAVMOUNT\") -> {regular, \"application/davmount+xml\"};\nt(global, \"cu\") -> {regular, \"application/cu-seeme\"};\nt(global, \"CU\") -> {regular, \"application/cu-seeme\"};\nt(global, \"cdmiq\") -> {regular, \"application/cdmi-queue\"};\nt(global, \"CDMIQ\") -> {regular, \"application/cdmi-queue\"};\nt(global, \"cdmio\") -> {regular, \"application/cdmi-object\"};\nt(global, \"CDMIO\") -> {regular, \"application/cdmi-object\"};\nt(global, \"cdmid\") -> {regular, \"application/cdmi-domain\"};\nt(global, \"CDMID\") -> {regular, \"application/cdmi-domain\"};\nt(global, \"cdmic\") -> {regular, \"application/cdmi-container\"};\nt(global, \"CDMIC\") -> {regular, \"application/cdmi-container\"};\nt(global, \"cdmia\") -> {regular, \"application/cdmi-capability\"};\nt(global, \"CDMIA\") -> {regular, \"application/cdmi-capability\"};\nt(global, \"ccxml\") -> {regular, \"application/ccxml+xml\"};\nt(global, \"CCXML\") -> {regular, \"application/ccxml+xml\"};\nt(global, \"atomsvc\") -> {regular, \"application/atomsvc+xml\"};\nt(global, \"ATOMSVC\") -> {regular, \"application/atomsvc+xml\"};\nt(global, \"atomcat\") -> {regular, \"application/atomcat+xml\"};\nt(global, \"ATOMCAT\") -> {regular, \"application/atomcat+xml\"};\nt(global, \"atom\") -> {regular, \"application/atom+xml\"};\nt(global, \"ATOM\") -> {regular, \"application/atom+xml\"};\nt(global, \"aw\") -> {regular, \"application/applixware\"};\nt(global, \"AW\") -> {regular, \"application/applixware\"};\nt(global, \"ez\") -> {regular, \"application/andrew-inset\"};\nt(global, \"EZ\") -> {regular, \"application/andrew-inset\"};\nt(global, \"fcgi\") -> {fcgi, \"text/html\"};\nt(global, \"FCGI\") -> {fcgi, \"text/html\"};\nt(global, \"cgi\") -> {cgi, \"text/html\"};\nt(global, \"CGI\") -> {cgi, \"text/html\"};\nt(global, \"php\") -> {php, \"text/html\"};\nt(global, \"PHP\") -> {php, \"text/html\"};\nt(global, \"yaws\") -> {yaws, \"text/html\"};\nt(global, \"YAWS\") -> {yaws, \"text/html\"};\nt(global, _) -> {regular, \"text/plain\"};\nt(_, Ext) -> t(global, Ext).\n\nrevt(#sconf{servername=SN, port=P}, Ext) -> revt({SN,P}, Ext);\nrevt(global, \"eci\") -> {regular, \"ice\", \"x-conference/x-cooltalk\"};\nrevt(global, \"ECI\") -> {regular, \"ICE\", \"x-conference/x-cooltalk\"};\nrevt(global, \"vms\") -> {regular, \"smv\", \"video/x-smv\"};\nrevt(global, \"VMS\") -> {regular, \"SMV\", \"video/x-smv\"};\nrevt(global, \"eivom\") -> {regular, \"movie\", \"video/x-sgi-movie\"};\nrevt(global, \"EIVOM\") -> {regular, \"MOVIE\", \"video/x-sgi-movie\"};\nrevt(global, \"iva\") -> {regular, \"avi\", \"video/x-msvideo\"};\nrevt(global, \"IVA\") -> {regular, \"AVI\", \"video/x-msvideo\"};\nrevt(global, \"xvw\") -> {regular, \"wvx\", \"video/x-ms-wvx\"};\nrevt(global, \"XVW\") -> {regular, \"WVX\", \"video/x-ms-wvx\"};\nrevt(global, \"xmw\") -> {regular, \"wmx\", \"video/x-ms-wmx\"};\nrevt(global, \"XMW\") -> {regular, \"WMX\", \"video/x-ms-wmx\"};\nrevt(global, \"vmw\") -> {regular, \"wmv\", \"video/x-ms-wmv\"};\nrevt(global, \"VMW\") -> {regular, \"WMV\", \"video/x-ms-wmv\"};\nrevt(global, \"mw\") -> {regular, \"wm\", \"video/x-ms-wm\"};\nrevt(global, \"MW\") -> {regular, \"WM\", \"video/x-ms-wm\"};\nrevt(global, \"bov\") -> {regular, \"vob\", \"video/x-ms-vob\"};\nrevt(global, \"BOV\") -> {regular, \"VOB\", \"video/x-ms-vob\"};\nrevt(global, \"xsa\") -> {regular, \"asx\", \"video/x-ms-asf\"};\nrevt(global, \"XSA\") -> {regular, \"ASX\", \"video/x-ms-asf\"};\nrevt(global, \"fsa\") -> {regular, \"asf\", \"video/x-ms-asf\"};\nrevt(global, \"FSA\") -> {regular, \"ASF\", \"video/x-ms-asf\"};\nrevt(global, \"gnm\") -> {regular, \"mng\", \"video/x-mng\"};\nrevt(global, \"GNM\") -> {regular, \"MNG\", \"video/x-mng\"};\nrevt(global, \"skm\") -> {regular, \"mks\", \"video/x-matroska\"};\nrevt(global, \"SKM\") -> {regular, \"MKS\", \"video/x-matroska\"};\nrevt(global, \"d3km\") -> {regular, \"mk3d\", \"video/x-matroska\"};\nrevt(global, \"D3KM\") -> {regular, \"MK3D\", \"video/x-matroska\"};\nrevt(global, \"vkm\") -> {regular, \"mkv\", \"video/x-matroska\"};\nrevt(global, \"VKM\") -> {regular, \"MKV\", \"video/x-matroska\"};\nrevt(global, \"v4m\") -> {regular, \"m4v\", \"video/x-m4v\"};\nrevt(global, \"V4M\") -> {regular, \"M4V\", \"video/x-m4v\"};\nrevt(global, \"vlf\") -> {regular, \"flv\", \"video/x-flv\"};\nrevt(global, \"VLF\") -> {regular, \"FLV\", \"video/x-flv\"};\nrevt(global, \"ilf\") -> {regular, \"fli\", \"video/x-fli\"};\nrevt(global, \"ILF\") -> {regular, \"FLI\", \"video/x-fli\"};\nrevt(global, \"v4f\") -> {regular, \"f4v\", \"video/x-f4v\"};\nrevt(global, \"V4F\") -> {regular, \"F4V\", \"video/x-f4v\"};\nrevt(global, \"mbew\") -> {regular, \"webm\", \"video/webm\"};\nrevt(global, \"MBEW\") -> {regular, \"WEBM\", \"video/webm\"};\nrevt(global, \"viv\") -> {regular, \"viv\", \"video/vnd.vivo\"};\nrevt(global, \"VIV\") -> {regular, \"VIV\", \"video/vnd.vivo\"};\nrevt(global, \"uvvu\") -> {regular, \"uvvu\", \"video/vnd.uvvu.mp4\"};\nrevt(global, \"UVVU\") -> {regular, \"UVVU\", \"video/vnd.uvvu.mp4\"};\nrevt(global, \"uvu\") -> {regular, \"uvu\", \"video/vnd.uvvu.mp4\"};\nrevt(global, \"UVU\") -> {regular, \"UVU\", \"video/vnd.uvvu.mp4\"};\nrevt(global, \"vyp\") -> {regular, \"pyv\", \"video/vnd.ms-playready.media.pyv\"};\nrevt(global, \"VYP\") -> {regular, \"PYV\", \"video/vnd.ms-playready.media.pyv\"};\nrevt(global, \"u4m\") -> {regular, \"m4u\", \"video/vnd.mpegurl\"};\nrevt(global, \"U4M\") -> {regular, \"M4U\", \"video/vnd.mpegurl\"};\nrevt(global, \"uxm\") -> {regular, \"mxu\", \"video/vnd.mpegurl\"};\nrevt(global, \"UXM\") -> {regular, \"MXU\", \"video/vnd.mpegurl\"};\nrevt(global, \"tvf\") -> {regular, \"fvt\", \"video/vnd.fvt\"};\nrevt(global, \"TVF\") -> {regular, \"FVT\", \"video/vnd.fvt\"};\nrevt(global, \"bvd\") -> {regular, \"dvb\", \"video/vnd.dvb.file\"};\nrevt(global, \"BVD\") -> {regular, \"DVB\", \"video/vnd.dvb.file\"};\nrevt(global, \"vvvu\") -> {regular, \"uvvv\", \"video/vnd.dece.video\"};\nrevt(global, \"VVVU\") -> {regular, \"UVVV\", \"video/vnd.dece.video\"};\nrevt(global, \"vvu\") -> {regular, \"uvv\", \"video/vnd.dece.video\"};\nrevt(global, \"VVU\") -> {regular, \"UVV\", \"video/vnd.dece.video\"};\nrevt(global, \"svvu\") -> {regular, \"uvvs\", \"video/vnd.dece.sd\"};\nrevt(global, \"SVVU\") -> {regular, \"UVVS\", \"video/vnd.dece.sd\"};\nrevt(global, \"svu\") -> {regular, \"uvs\", \"video/vnd.dece.sd\"};\nrevt(global, \"SVU\") -> {regular, \"UVS\", \"video/vnd.dece.sd\"};\nrevt(global, \"pvvu\") -> {regular, \"uvvp\", \"video/vnd.dece.pd\"};\nrevt(global, \"PVVU\") -> {regular, \"UVVP\", \"video/vnd.dece.pd\"};\nrevt(global, \"pvu\") -> {regular, \"uvp\", \"video/vnd.dece.pd\"};\nrevt(global, \"PVU\") -> {regular, \"UVP\", \"video/vnd.dece.pd\"};\nrevt(global, \"mvvu\") -> {regular, \"uvvm\", \"video/vnd.dece.mobile\"};\nrevt(global, \"MVVU\") -> {regular, \"UVVM\", \"video/vnd.dece.mobile\"};\nrevt(global, \"mvu\") -> {regular, \"uvm\", \"video/vnd.dece.mobile\"};\nrevt(global, \"MVU\") -> {regular, \"UVM\", \"video/vnd.dece.mobile\"};\nrevt(global, \"hvvu\") -> {regular, \"uvvh\", \"video/vnd.dece.hd\"};\nrevt(global, \"HVVU\") -> {regular, \"UVVH\", \"video/vnd.dece.hd\"};\nrevt(global, \"hvu\") -> {regular, \"uvh\", \"video/vnd.dece.hd\"};\nrevt(global, \"HVU\") -> {regular, \"UVH\", \"video/vnd.dece.hd\"};\nrevt(global, \"vom\") -> {regular, \"mov\", \"video/quicktime\"};\nrevt(global, \"VOM\") -> {regular, \"MOV\", \"video/quicktime\"};\nrevt(global, \"tq\") -> {regular, \"qt\", \"video/quicktime\"};\nrevt(global, \"TQ\") -> {regular, \"QT\", \"video/quicktime\"};\nrevt(global, \"vgo\") -> {regular, \"ogv\", \"video/ogg\"};\nrevt(global, \"VGO\") -> {regular, \"OGV\", \"video/ogg\"};\nrevt(global, \"v2m\") -> {regular, \"m2v\", \"video/mpeg\"};\nrevt(global, \"V2M\") -> {regular, \"M2V\", \"video/mpeg\"};\nrevt(global, \"v1m\") -> {regular, \"m1v\", \"video/mpeg\"};\nrevt(global, \"V1M\") -> {regular, \"M1V\", \"video/mpeg\"};\nrevt(global, \"epm\") -> {regular, \"mpe\", \"video/mpeg\"};\nrevt(global, \"EPM\") -> {regular, \"MPE\", \"video/mpeg\"};\nrevt(global, \"gpm\") -> {regular, \"mpg\", \"video/mpeg\"};\nrevt(global, \"GPM\") -> {regular, \"MPG\", \"video/mpeg\"};\nrevt(global, \"gepm\") -> {regular, \"mpeg\", \"video/mpeg\"};\nrevt(global, \"GEPM\") -> {regular, \"MPEG\", \"video/mpeg\"};\nrevt(global, \"4gpm\") -> {regular, \"mpg4\", \"video/mp4\"};\nrevt(global, \"4GPM\") -> {regular, \"MPG4\", \"video/mp4\"};\nrevt(global, \"v4pm\") -> {regular, \"mp4v\", \"video/mp4\"};\nrevt(global, \"V4PM\") -> {regular, \"MP4V\", \"video/mp4\"};\nrevt(global, \"4pm\") -> {regular, \"mp4\", \"video/mp4\"};\nrevt(global, \"4PM\") -> {regular, \"MP4\", \"video/mp4\"};\nrevt(global, \"st\") -> {regular, \"ts\", \"video/mp2t\"};\nrevt(global, \"ST\") -> {regular, \"TS\", \"video/mp2t\"};\nrevt(global, \"2pjm\") -> {regular, \"mjp2\", \"video/mj2\"};\nrevt(global, \"2PJM\") -> {regular, \"MJP2\", \"video/mj2\"};\nrevt(global, \"2jm\") -> {regular, \"mj2\", \"video/mj2\"};\nrevt(global, \"2JM\") -> {regular, \"MJ2\", \"video/mj2\"};\nrevt(global, \"mgpj\") -> {regular, \"jpgm\", \"video/jpm\"};\nrevt(global, \"MGPJ\") -> {regular, \"JPGM\", \"video/jpm\"};\nrevt(global, \"mpj\") -> {regular, \"jpm\", \"video/jpm\"};\nrevt(global, \"MPJ\") -> {regular, \"JPM\", \"video/jpm\"};\nrevt(global, \"vgpj\") -> {regular, \"jpgv\", \"video/jpeg\"};\nrevt(global, \"VGPJ\") -> {regular, \"JPGV\", \"video/jpeg\"};\nrevt(global, \"462h\") -> {regular, \"h264\", \"video/h264\"};\nrevt(global, \"462H\") -> {regular, \"H264\", \"video/h264\"};\nrevt(global, \"362h\") -> {regular, \"h263\", \"video/h263\"};\nrevt(global, \"362H\") -> {regular, \"H263\", \"video/h263\"};\nrevt(global, \"162h\") -> {regular, \"h261\", \"video/h261\"};\nrevt(global, \"162H\") -> {regular, \"H261\", \"video/h261\"};\nrevt(global, \"2g3\") -> {regular, \"3g2\", \"video/3gpp2\"};\nrevt(global, \"2G3\") -> {regular, \"3G2\", \"video/3gpp2\"};\nrevt(global, \"pg3\") -> {regular, \"3gp\", \"video/3gpp\"};\nrevt(global, \"PG3\") -> {regular, \"3GP\", \"video/3gpp\"};\nrevt(global, \"fcv\") -> {regular, \"vcf\", \"text/x-vcard\"};\nrevt(global, \"FCV\") -> {regular, \"VCF\", \"text/x-vcard\"};\nrevt(global, \"scv\") -> {regular, \"vcs\", \"text/x-vcalendar\"};\nrevt(global, \"SCV\") -> {regular, \"VCS\", \"text/x-vcalendar\"};\nrevt(global, \"uu\") -> {regular, \"uu\", \"text/x-uuencode\"};\nrevt(global, \"UU\") -> {regular, \"UU\", \"text/x-uuencode\"};\nrevt(global, \"vfs\") -> {regular, \"sfv\", \"text/x-sfv\"};\nrevt(global, \"VFS\") -> {regular, \"SFV\", \"text/x-sfv\"};\nrevt(global, \"xte\") -> {regular, \"etx\", \"text/x-setext\"};\nrevt(global, \"XTE\") -> {regular, \"ETX\", \"text/x-setext\"};\nrevt(global, \"sap\") -> {regular, \"pas\", \"text/x-pascal\"};\nrevt(global, \"SAP\") -> {regular, \"PAS\", \"text/x-pascal\"};\nrevt(global, \"p\") -> {regular, \"p\", \"text/x-pascal\"};\nrevt(global, \"P\") -> {regular, \"P\", \"text/x-pascal\"};\nrevt(global, \"lmpo\") -> {regular, \"opml\", \"text/x-opml\"};\nrevt(global, \"LMPO\") -> {regular, \"OPML\", \"text/x-opml\"};\nrevt(global, \"ofn\") -> {regular, \"nfo\", \"text/x-nfo\"};\nrevt(global, \"OFN\") -> {regular, \"NFO\", \"text/x-nfo\"};\nrevt(global, \"avaj\") -> {regular, \"java\", \"text/x-java-source\"};\nrevt(global, \"AVAJ\") -> {regular, \"JAVA\", \"text/x-java-source\"};\nrevt(global, \"09f\") -> {regular, \"f90\", \"text/x-fortran\"};\nrevt(global, \"09F\") -> {regular, \"F90\", \"text/x-fortran\"};\nrevt(global, \"77f\") -> {regular, \"f77\", \"text/x-fortran\"};\nrevt(global, \"77F\") -> {regular, \"F77\", \"text/x-fortran\"};\nrevt(global, \"rof\") -> {regular, \"for\", \"text/x-fortran\"};\nrevt(global, \"ROF\") -> {regular, \"FOR\", \"text/x-fortran\"};\nrevt(global, \"f\") -> {regular, \"f\", \"text/x-fortran\"};\nrevt(global, \"F\") -> {regular, \"F\", \"text/x-fortran\"};\nrevt(global, \"cid\") -> {regular, \"dic\", \"text/x-c\"};\nrevt(global, \"CID\") -> {regular, \"DIC\", \"text/x-c\"};\nrevt(global, \"hh\") -> {regular, \"hh\", \"text/x-c\"};\nrevt(global, \"HH\") -> {regular, \"HH\", \"text/x-c\"};\nrevt(global, \"h\") -> {regular, \"h\", \"text/x-c\"};\nrevt(global, \"H\") -> {regular, \"H\", \"text/x-c\"};\nrevt(global, \"ppc\") -> {regular, \"cpp\", \"text/x-c\"};\nrevt(global, \"PPC\") -> {regular, \"CPP\", \"text/x-c\"};\nrevt(global, \"xxc\") -> {regular, \"cxx\", \"text/x-c\"};\nrevt(global, \"XXC\") -> {regular, \"CXX\", \"text/x-c\"};\nrevt(global, \"cc\") -> {regular, \"cc\", \"text/x-c\"};\nrevt(global, \"CC\") -> {regular, \"CC\", \"text/x-c\"};\nrevt(global, \"c\") -> {regular, \"c\", \"text/x-c\"};\nrevt(global, \"C\") -> {regular, \"C\", \"text/x-c\"};\nrevt(global, \"msa\") -> {regular, \"asm\", \"text/x-asm\"};\nrevt(global, \"MSA\") -> {regular, \"ASM\", \"text/x-asm\"};\nrevt(global, \"s\") -> {regular, \"s\", \"text/x-asm\"};\nrevt(global, \"S\") -> {regular, \"S\", \"text/x-asm\"};\nrevt(global, \"slmw\") -> {regular, \"wmls\", \"text/vnd.wap.wmlscript\"};\nrevt(global, \"SLMW\") -> {regular, \"WMLS\", \"text/vnd.wap.wmlscript\"};\nrevt(global, \"lmw\") -> {regular, \"wml\", \"text/vnd.wap.wml\"};\nrevt(global, \"LMW\") -> {regular, \"WML\", \"text/vnd.wap.wml\"};\nrevt(global, \"ls\") -> {regular, \"sl\", \"text/vnd.wap.sl\"};\nrevt(global, \"LS\") -> {regular, \"SL\", \"text/vnd.wap.sl\"};\nrevt(global, \"is\") -> {regular, \"si\", \"text/vnd.wap.si\"};\nrevt(global, \"IS\") -> {regular, \"SI\", \"text/vnd.wap.si\"};\nrevt(global, \"daj\") -> {regular, \"jad\", \"text/vnd.sun.j2me.app-descriptor\"};\nrevt(global, \"DAJ\") -> {regular, \"JAD\", \"text/vnd.sun.j2me.app-descriptor\"};\nrevt(global, \"tops\") -> {regular, \"spot\", \"text/vnd.in3d.spot\"};\nrevt(global, \"TOPS\") -> {regular, \"SPOT\", \"text/vnd.in3d.spot\"};\nrevt(global, \"lmd3\") -> {regular, \"3dml\", \"text/vnd.in3d.3dml\"};\nrevt(global, \"LMD3\") -> {regular, \"3DML\", \"text/vnd.in3d.3dml\"};\nrevt(global, \"vg\") -> {regular, \"gv\", \"text/vnd.graphviz\"};\nrevt(global, \"VG\") -> {regular, \"GV\", \"text/vnd.graphviz\"};\nrevt(global, \"xlf\") -> {regular, \"flx\", \"text/vnd.fmi.flexstor\"};\nrevt(global, \"XLF\") -> {regular, \"FLX\", \"text/vnd.fmi.flexstor\"};\nrevt(global, \"ylf\") -> {regular, \"fly\", \"text/vnd.fly\"};\nrevt(global, \"YLF\") -> {regular, \"FLY\", \"text/vnd.fly\"};\nrevt(global, \"lrucs\") -> {regular, \"scurl\", \"text/vnd.curl.scurl\"};\nrevt(global, \"LRUCS\") -> {regular, \"SCURL\", \"text/vnd.curl.scurl\"};\nrevt(global, \"lrucm\") -> {regular, \"mcurl\", \"text/vnd.curl.mcurl\"};\nrevt(global, \"LRUCM\") -> {regular, \"MCURL\", \"text/vnd.curl.mcurl\"};\nrevt(global, \"lrucd\") -> {regular, \"dcurl\", \"text/vnd.curl.dcurl\"};\nrevt(global, \"LRUCD\") -> {regular, \"DCURL\", \"text/vnd.curl.dcurl\"};\nrevt(global, \"lruc\") -> {regular, \"curl\", \"text/vnd.curl\"};\nrevt(global, \"LRUC\") -> {regular, \"CURL\", \"text/vnd.curl\"};\nrevt(global, \"dracv\") -> {regular, \"vcard\", \"text/vcard\"};\nrevt(global, \"DRACV\") -> {regular, \"VCARD\", \"text/vcard\"};\nrevt(global, \"slru\") -> {regular, \"urls\", \"text/uri-list\"};\nrevt(global, \"SLRU\") -> {regular, \"URLS\", \"text/uri-list\"};\nrevt(global, \"siru\") -> {regular, \"uris\", \"text/uri-list\"};\nrevt(global, \"SIRU\") -> {regular, \"URIS\", \"text/uri-list\"};\nrevt(global, \"iru\") -> {regular, \"uri\", \"text/uri-list\"};\nrevt(global, \"IRU\") -> {regular, \"URI\", \"text/uri-list\"};\nrevt(global, \"ltt\") -> {regular, \"ttl\", \"text/turtle\"};\nrevt(global, \"LTT\") -> {regular, \"TTL\", \"text/turtle\"};\nrevt(global, \"em\") -> {regular, \"me\", \"text/troff\"};\nrevt(global, \"EM\") -> {regular, \"ME\", \"text/troff\"};\nrevt(global, \"nam\") -> {regular, \"man\", \"text/troff\"};\nrevt(global, \"NAM\") -> {regular, \"MAN\", \"text/troff\"};\nrevt(global, \"ffor\") -> {regular, \"roff\", \"text/troff\"};\nrevt(global, \"FFOR\") -> {regular, \"ROFF\", \"text/troff\"};\nrevt(global, \"rt\") -> {regular, \"tr\", \"text/troff\"};\nrevt(global, \"RT\") -> {regular, \"TR\", \"text/troff\"};\nrevt(global, \"t\") -> {regular, \"t\", \"text/troff\"};\nrevt(global, \"T\") -> {regular, \"T\", \"text/troff\"};\nrevt(global, \"vst\") -> {regular, \"tsv\", \"text/tab-separated-values\"};\nrevt(global, \"VST\") -> {regular, \"TSV\", \"text/tab-separated-values\"};\nrevt(global, \"mgs\") -> {regular, \"sgm\", \"text/sgml\"};\nrevt(global, \"MGS\") -> {regular, \"SGM\", \"text/sgml\"};\nrevt(global, \"lmgs\") -> {regular, \"sgml\", \"text/sgml\"};\nrevt(global, \"LMGS\") -> {regular, \"SGML\", \"text/sgml\"};\nrevt(global, \"xtr\") -> {regular, \"rtx\", \"text/richtext\"};\nrevt(global, \"XTR\") -> {regular, \"RTX\", \"text/richtext\"};\nrevt(global, \"csd\") -> {regular, \"dsc\", \"text/prs.lines.tag\"};\nrevt(global, \"CSD\") -> {regular, \"DSC\", \"text/prs.lines.tag\"};\nrevt(global, \"ni\") -> {regular, \"in\", \"text/plain\"};\nrevt(global, \"NI\") -> {regular, \"IN\", \"text/plain\"};\nrevt(global, \"gol\") -> {regular, \"log\", \"text/plain\"};\nrevt(global, \"GOL\") -> {regular, \"LOG\", \"text/plain\"};\nrevt(global, \"tsil\") -> {regular, \"list\", \"text/plain\"};\nrevt(global, \"TSIL\") -> {regular, \"LIST\", \"text/plain\"};\nrevt(global, \"fed\") -> {regular, \"def\", \"text/plain\"};\nrevt(global, \"FED\") -> {regular, \"DEF\", \"text/plain\"};\nrevt(global, \"fnoc\") -> {regular, \"conf\", \"text/plain\"};\nrevt(global, \"FNOC\") -> {regular, \"CONF\", \"text/plain\"};\nrevt(global, \"txet\") -> {regular, \"text\", \"text/plain\"};\nrevt(global, \"TXET\") -> {regular, \"TEXT\", \"text/plain\"};\nrevt(global, \"txt\") -> {regular, \"txt\", \"text/plain\"};\nrevt(global, \"TXT\") -> {regular, \"TXT\", \"text/plain\"};\nrevt(global, \"3n\") -> {regular, \"n3\", \"text/n3\"};\nrevt(global, \"3N\") -> {regular, \"N3\", \"text/n3\"};\nrevt(global, \"mth\") -> {regular, \"htm\", \"text/html\"};\nrevt(global, \"MTH\") -> {regular, \"HTM\", \"text/html\"};\nrevt(global, \"lmth\") -> {regular, \"html\", \"text/html\"};\nrevt(global, \"LMTH\") -> {regular, \"HTML\", \"text/html\"};\nrevt(global, \"vsc\") -> {regular, \"csv\", \"text/csv\"};\nrevt(global, \"VSC\") -> {regular, \"CSV\", \"text/csv\"};\nrevt(global, \"ssc\") -> {regular, \"css\", \"text/css\"};\nrevt(global, \"SSC\") -> {regular, \"CSS\", \"text/css\"};\nrevt(global, \"bfi\") -> {regular, \"ifb\", \"text/calendar\"};\nrevt(global, \"BFI\") -> {regular, \"IFB\", \"text/calendar\"};\nrevt(global, \"sci\") -> {regular, \"ics\", \"text/calendar\"};\nrevt(global, \"SCI\") -> {regular, \"ICS\", \"text/calendar\"};\nrevt(global, \"ehcacppa\") -> {regular, \"appcache\", \"text/cache-manifest\"};\nrevt(global, \"EHCACPPA\") -> {regular, \"APPCACHE\", \"text/cache-manifest\"};\nrevt(global, \"zd3x\") -> {regular, \"x3dz\", \"model/x3d+xml\"};\nrevt(global, \"ZD3X\") -> {regular, \"X3DZ\", \"model/x3d+xml\"};\nrevt(global, \"zvd3x\") -> {regular, \"x3dvz\", \"model/x3d+vrml\"};\nrevt(global, \"ZVD3X\") -> {regular, \"X3DVZ\", \"model/x3d+vrml\"};\nrevt(global, \"vd3x\") -> {regular, \"x3dv\", \"model/x3d+vrml\"};\nrevt(global, \"VD3X\") -> {regular, \"X3DV\", \"model/x3d+vrml\"};\nrevt(global, \"zbd3x\") -> {regular, \"x3dbz\", \"model/x3d+binary\"};\nrevt(global, \"ZBD3X\") -> {regular, \"X3DBZ\", \"model/x3d+binary\"};\nrevt(global, \"bd3x\") -> {regular, \"x3db\", \"model/x3d+binary\"};\nrevt(global, \"BD3X\") -> {regular, \"X3DB\", \"model/x3d+binary\"};\nrevt(global, \"lmrv\") -> {regular, \"vrml\", \"model/vrml\"};\nrevt(global, \"LMRV\") -> {regular, \"VRML\", \"model/vrml\"};\nrevt(global, \"lrw\") -> {regular, \"wrl\", \"model/vrml\"};\nrevt(global, \"LRW\") -> {regular, \"WRL\", \"model/vrml\"};\nrevt(global, \"utv\") -> {regular, \"vtu\", \"model/vnd.vtu\"};\nrevt(global, \"UTV\") -> {regular, \"VTU\", \"model/vnd.vtu\"};\nrevt(global, \"stm\") -> {regular, \"mts\", \"model/vnd.mts\"};\nrevt(global, \"STM\") -> {regular, \"MTS\", \"model/vnd.mts\"};\nrevt(global, \"wtg\") -> {regular, \"gtw\", \"model/vnd.gtw\"};\nrevt(global, \"WTG\") -> {regular, \"GTW\", \"model/vnd.gtw\"};\nrevt(global, \"ldg\") -> {regular, \"gdl\", \"model/vnd.gdl\"};\nrevt(global, \"LDG\") -> {regular, \"GDL\", \"model/vnd.gdl\"};\nrevt(global, \"fwd\") -> {regular, \"dwf\", \"model/vnd.dwf\"};\nrevt(global, \"FWD\") -> {regular, \"DWF\", \"model/vnd.dwf\"};\nrevt(global, \"ead\") -> {regular, \"dae\", \"model/vnd.collada+xml\"};\nrevt(global, \"EAD\") -> {regular, \"DAE\", \"model/vnd.collada+xml\"};\nrevt(global, \"olis\") -> {regular, \"silo\", \"model/mesh\"};\nrevt(global, \"OLIS\") -> {regular, \"SILO\", \"model/mesh\"};\nrevt(global, \"hsem\") -> {regular, \"mesh\", \"model/mesh\"};\nrevt(global, \"HSEM\") -> {regular, \"MESH\", \"model/mesh\"};\nrevt(global, \"hsm\") -> {regular, \"msh\", \"model/mesh\"};\nrevt(global, \"HSM\") -> {regular, \"MSH\", \"model/mesh\"};\nrevt(global, \"segi\") -> {regular, \"iges\", \"model/iges\"};\nrevt(global, \"SEGI\") -> {regular, \"IGES\", \"model/iges\"};\nrevt(global, \"sgi\") -> {regular, \"igs\", \"model/iges\"};\nrevt(global, \"SGI\") -> {regular, \"IGS\", \"model/iges\"};\nrevt(global, \"emim\") -> {regular, \"mime\", \"message/rfc822\"};\nrevt(global, \"EMIM\") -> {regular, \"MIME\", \"message/rfc822\"};\nrevt(global, \"lme\") -> {regular, \"eml\", \"message/rfc822\"};\nrevt(global, \"LME\") -> {regular, \"EML\", \"message/rfc822\"};\nrevt(global, \"dwx\") -> {regular, \"xwd\", \"image/x-xwindowdump\"};\nrevt(global, \"DWX\") -> {regular, \"XWD\", \"image/x-xwindowdump\"};\nrevt(global, \"mpx\") -> {regular, \"xpm\", \"image/x-xpixmap\"};\nrevt(global, \"MPX\") -> {regular, \"XPM\", \"image/x-xpixmap\"};\nrevt(global, \"mbx\") -> {regular, \"xbm\", \"image/x-xbitmap\"};\nrevt(global, \"MBX\") -> {regular, \"XBM\", \"image/x-xbitmap\"};\nrevt(global, \"agt\") -> {regular, \"tga\", \"image/x-tga\"};\nrevt(global, \"AGT\") -> {regular, \"TGA\", \"image/x-tga\"};\nrevt(global, \"bgr\") -> {regular, \"rgb\", \"image/x-rgb\"};\nrevt(global, \"BGR\") -> {regular, \"RGB\", \"image/x-rgb\"};\nrevt(global, \"mpp\") -> {regular, \"ppm\", \"image/x-portable-pixmap\"};\nrevt(global, \"MPP\") -> {regular, \"PPM\", \"image/x-portable-pixmap\"};\nrevt(global, \"mgp\") -> {regular, \"pgm\", \"image/x-portable-graymap\"};\nrevt(global, \"MGP\") -> {regular, \"PGM\", \"image/x-portable-graymap\"};\nrevt(global, \"mbp\") -> {regular, \"pbm\", \"image/x-portable-bitmap\"};\nrevt(global, \"MBP\") -> {regular, \"PBM\", \"image/x-portable-bitmap\"};\nrevt(global, \"mnp\") -> {regular, \"pnm\", \"image/x-portable-anymap\"};\nrevt(global, \"MNP\") -> {regular, \"PNM\", \"image/x-portable-anymap\"};\nrevt(global, \"tcp\") -> {regular, \"pct\", \"image/x-pict\"};\nrevt(global, \"TCP\") -> {regular, \"PCT\", \"image/x-pict\"};\nrevt(global, \"cip\") -> {regular, \"pic\", \"image/x-pict\"};\nrevt(global, \"CIP\") -> {regular, \"PIC\", \"image/x-pict\"};\nrevt(global, \"xcp\") -> {regular, \"pcx\", \"image/x-pcx\"};\nrevt(global, \"XCP\") -> {regular, \"PCX\", \"image/x-pcx\"};\nrevt(global, \"dis\") -> {regular, \"sid\", \"image/x-mrsid-image\"};\nrevt(global, \"DIS\") -> {regular, \"SID\", \"image/x-mrsid-image\"};\nrevt(global, \"oci\") -> {regular, \"ico\", \"image/x-icon\"};\nrevt(global, \"OCI\") -> {regular, \"ICO\", \"image/x-icon\"};\nrevt(global, \"7hf\") -> {regular, \"fh7\", \"image/x-freehand\"};\nrevt(global, \"7HF\") -> {regular, \"FH7\", \"image/x-freehand\"};\nrevt(global, \"5hf\") -> {regular, \"fh5\", \"image/x-freehand\"};\nrevt(global, \"5HF\") -> {regular, \"FH5\", \"image/x-freehand\"};\nrevt(global, \"4hf\") -> {regular, \"fh4\", \"image/x-freehand\"};\nrevt(global, \"4HF\") -> {regular, \"FH4\", \"image/x-freehand\"};\nrevt(global, \"chf\") -> {regular, \"fhc\", \"image/x-freehand\"};\nrevt(global, \"CHF\") -> {regular, \"FHC\", \"image/x-freehand\"};\nrevt(global, \"hf\") -> {regular, \"fh\", \"image/x-freehand\"};\nrevt(global, \"HF\") -> {regular, \"FH\", \"image/x-freehand\"};\nrevt(global, \"xmc\") -> {regular, \"cmx\", \"image/x-cmx\"};\nrevt(global, \"XMC\") -> {regular, \"CMX\", \"image/x-cmx\"};\nrevt(global, \"sar\") -> {regular, \"ras\", \"image/x-cmu-raster\"};\nrevt(global, \"SAR\") -> {regular, \"RAS\", \"image/x-cmu-raster\"};\nrevt(global, \"sd3\") -> {regular, \"3ds\", \"image/x-3ds\"};\nrevt(global, \"SD3\") -> {regular, \"3DS\", \"image/x-3ds\"};\nrevt(global, \"pbew\") -> {regular, \"webp\", \"image/webp\"};\nrevt(global, \"PBEW\") -> {regular, \"WEBP\", \"image/webp\"};\nrevt(global, \"fix\") -> {regular, \"xif\", \"image/vnd.xiff\"};\nrevt(global, \"FIX\") -> {regular, \"XIF\", \"image/vnd.xiff\"};\nrevt(global, \"pmbw\") -> {regular, \"wbmp\", \"image/vnd.wap.wbmp\"};\nrevt(global, \"PMBW\") -> {regular, \"WBMP\", \"image/vnd.wap.wbmp\"};\nrevt(global, \"xpn\") -> {regular, \"npx\", \"image/vnd.net-fpx\"};\nrevt(global, \"XPN\") -> {regular, \"NPX\", \"image/vnd.net-fpx\"};\nrevt(global, \"pdw\") -> {regular, \"wdp\", \"image/vnd.ms-photo\"};\nrevt(global, \"PDW\") -> {regular, \"WDP\", \"image/vnd.ms-photo\"};\nrevt(global, \"idm\") -> {regular, \"mdi\", \"image/vnd.ms-modi\"};\nrevt(global, \"IDM\") -> {regular, \"MDI\", \"image/vnd.ms-modi\"};\nrevt(global, \"clr\") -> {regular, \"rlc\", \"image/vnd.fujixerox.edmics-rlc\"};\nrevt(global, \"CLR\") -> {regular, \"RLC\", \"image/vnd.fujixerox.edmics-rlc\"};\nrevt(global, \"rmm\") -> {regular, \"mmr\", \"image/vnd.fujixerox.edmics-mmr\"};\nrevt(global, \"RMM\") -> {regular, \"MMR\", \"image/vnd.fujixerox.edmics-mmr\"};\nrevt(global, \"tsf\") -> {regular, \"fst\", \"image/vnd.fst\"};\nrevt(global, \"TSF\") -> {regular, \"FST\", \"image/vnd.fst\"};\nrevt(global, \"xpf\") -> {regular, \"fpx\", \"image/vnd.fpx\"};\nrevt(global, \"XPF\") -> {regular, \"FPX\", \"image/vnd.fpx\"};\nrevt(global, \"sbf\") -> {regular, \"fbs\", \"image/vnd.fastbidsheet\"};\nrevt(global, \"SBF\") -> {regular, \"FBS\", \"image/vnd.fastbidsheet\"};\nrevt(global, \"fxd\") -> {regular, \"dxf\", \"image/vnd.dxf\"};\nrevt(global, \"FXD\") -> {regular, \"DXF\", \"image/vnd.dxf\"};\nrevt(global, \"gwd\") -> {regular, \"dwg\", \"image/vnd.dwg\"};\nrevt(global, \"GWD\") -> {regular, \"DWG\", \"image/vnd.dwg\"};\nrevt(global, \"bus\") -> {regular, \"sub\", \"text/vnd.dvb.subtitle\"};\nrevt(global, \"BUS\") -> {regular, \"SUB\", \"text/vnd.dvb.subtitle\"};\nrevt(global, \"vjd\") -> {regular, \"djv\", \"image/vnd.djvu\"};\nrevt(global, \"VJD\") -> {regular, \"DJV\", \"image/vnd.djvu\"};\nrevt(global, \"uvjd\") -> {regular, \"djvu\", \"image/vnd.djvu\"};\nrevt(global, \"UVJD\") -> {regular, \"DJVU\", \"image/vnd.djvu\"};\nrevt(global, \"gvvu\") -> {regular, \"uvvg\", \"image/vnd.dece.graphic\"};\nrevt(global, \"GVVU\") -> {regular, \"UVVG\", \"image/vnd.dece.graphic\"};\nrevt(global, \"gvu\") -> {regular, \"uvg\", \"image/vnd.dece.graphic\"};\nrevt(global, \"GVU\") -> {regular, \"UVG\", \"image/vnd.dece.graphic\"};\nrevt(global, \"ivvu\") -> {regular, \"uvvi\", \"image/vnd.dece.graphic\"};\nrevt(global, \"IVVU\") -> {regular, \"UVVI\", \"image/vnd.dece.graphic\"};\nrevt(global, \"ivu\") -> {regular, \"uvi\", \"image/vnd.dece.graphic\"};\nrevt(global, \"IVU\") -> {regular, \"UVI\", \"image/vnd.dece.graphic\"};\nrevt(global, \"dsp\") -> {regular, \"psd\", \"image/vnd.adobe.photoshop\"};\nrevt(global, \"DSP\") -> {regular, \"PSD\", \"image/vnd.adobe.photoshop\"};\nrevt(global, \"fit\") -> {regular, \"tif\", \"image/tiff\"};\nrevt(global, \"FIT\") -> {regular, \"TIF\", \"image/tiff\"};\nrevt(global, \"ffit\") -> {regular, \"tiff\", \"image/tiff\"};\nrevt(global, \"FFIT\") -> {regular, \"TIFF\", \"image/tiff\"};\nrevt(global, \"zgvs\") -> {regular, \"svgz\", \"image/svg+xml\"};\nrevt(global, \"ZGVS\") -> {regular, \"SVGZ\", \"image/svg+xml\"};\nrevt(global, \"gvs\") -> {regular, \"svg\", \"image/svg+xml\"};\nrevt(global, \"GVS\") -> {regular, \"SVG\", \"image/svg+xml\"};\nrevt(global, \"igs\") -> {regular, \"sgi\", \"image/sgi\"};\nrevt(global, \"IGS\") -> {regular, \"SGI\", \"image/sgi\"};\nrevt(global, \"fitb\") -> {regular, \"btif\", \"image/prs.btif\"};\nrevt(global, \"FITB\") -> {regular, \"BTIF\", \"image/prs.btif\"};\nrevt(global, \"gnp\") -> {regular, \"png\", \"image/png\"};\nrevt(global, \"GNP\") -> {regular, \"PNG\", \"image/png\"};\nrevt(global, \"xtk\") -> {regular, \"ktx\", \"image/ktx\"};\nrevt(global, \"XTK\") -> {regular, \"KTX\", \"image/ktx\"};\nrevt(global, \"epj\") -> {regular, \"jpe\", \"image/jpeg\"};\nrevt(global, \"EPJ\") -> {regular, \"JPE\", \"image/jpeg\"};\nrevt(global, \"gpj\") -> {regular, \"jpg\", \"image/jpeg\"};\nrevt(global, \"GPJ\") -> {regular, \"JPG\", \"image/jpeg\"};\nrevt(global, \"gepj\") -> {regular, \"jpeg\", \"image/jpeg\"};\nrevt(global, \"GEPJ\") -> {regular, \"JPEG\", \"image/jpeg\"};\nrevt(global, \"fei\") -> {regular, \"ief\", \"image/ief\"};\nrevt(global, \"FEI\") -> {regular, \"IEF\", \"image/ief\"};\nrevt(global, \"fig\") -> {regular, \"gif\", \"image/gif\"};\nrevt(global, \"FIG\") -> {regular, \"GIF\", \"image/gif\"};\nrevt(global, \"3g\") -> {regular, \"g3\", \"image/g3fax\"};\nrevt(global, \"3G\") -> {regular, \"G3\", \"image/g3fax\"};\nrevt(global, \"mgc\") -> {regular, \"cgm\", \"image/cgm\"};\nrevt(global, \"MGC\") -> {regular, \"CGM\", \"image/cgm\"};\nrevt(global, \"pmb\") -> {regular, \"bmp\", \"image/bmp\"};\nrevt(global, \"PMB\") -> {regular, \"BMP\", \"image/bmp\"};\nrevt(global, \"zyx\") -> {regular, \"xyz\", \"chemical/x-xyz\"};\nrevt(global, \"ZYX\") -> {regular, \"XYZ\", \"chemical/x-xyz\"};\nrevt(global, \"lmsc\") -> {regular, \"csml\", \"chemical/x-csml\"};\nrevt(global, \"LMSC\") -> {regular, \"CSML\", \"chemical/x-csml\"};\nrevt(global, \"lmc\") -> {regular, \"cml\", \"chemical/x-cml\"};\nrevt(global, \"LMC\") -> {regular, \"CML\", \"chemical/x-cml\"};\nrevt(global, \"fdmc\") -> {regular, \"cmdf\", \"chemical/x-cmdf\"};\nrevt(global, \"FDMC\") -> {regular, \"CMDF\", \"chemical/x-cmdf\"};\nrevt(global, \"fic\") -> {regular, \"cif\", \"chemical/x-cif\"};\nrevt(global, \"FIC\") -> {regular, \"CIF\", \"chemical/x-cif\"};\nrevt(global, \"xdc\") -> {regular, \"cdx\", \"chemical/x-cdx\"};\nrevt(global, \"XDC\") -> {regular, \"CDX\", \"chemical/x-cdx\"};\nrevt(global, \"mx\") -> {regular, \"xm\", \"audio/xm\"};\nrevt(global, \"MX\") -> {regular, \"XM\", \"audio/xm\"};\nrevt(global, \"vaw\") -> {regular, \"wav\", \"audio/x-wav\"};\nrevt(global, \"VAW\") -> {regular, \"WAV\", \"audio/x-wav\"};\nrevt(global, \"pmr\") -> {regular, \"rmp\", \"audio/x-pn-realaudio-plugin\"};\nrevt(global, \"PMR\") -> {regular, \"RMP\", \"audio/x-pn-realaudio-plugin\"};\nrevt(global, \"ar\") -> {regular, \"ra\", \"audio/x-realaudio\"};\nrevt(global, \"AR\") -> {regular, \"RA\", \"audio/x-realaudio\"};\nrevt(global, \"mar\") -> {regular, \"ram\", \"audio/x-pn-realaudio\"};\nrevt(global, \"MAR\") -> {regular, \"RAM\", \"audio/x-pn-realaudio\"};\nrevt(global, \"amw\") -> {regular, \"wma\", \"audio/x-ms-wma\"};\nrevt(global, \"AMW\") -> {regular, \"WMA\", \"audio/x-ms-wma\"};\nrevt(global, \"xaw\") -> {regular, \"wax\", \"audio/x-ms-wax\"};\nrevt(global, \"XAW\") -> {regular, \"WAX\", \"audio/x-ms-wax\"};\nrevt(global, \"u3m\") -> {regular, \"m3u\", \"audio/x-mpegurl\"};\nrevt(global, \"U3M\") -> {regular, \"M3U\", \"audio/x-mpegurl\"};\nrevt(global, \"akm\") -> {regular, \"mka\", \"audio/x-matroska\"};\nrevt(global, \"AKM\") -> {regular, \"MKA\", \"audio/x-matroska\"};\nrevt(global, \"calf\") -> {regular, \"flac\", \"audio/x-flac\"};\nrevt(global, \"CALF\") -> {regular, \"FLAC\", \"audio/x-flac\"};\nrevt(global, \"fac\") -> {regular, \"caf\", \"audio/x-caf\"};\nrevt(global, \"FAC\") -> {regular, \"CAF\", \"audio/x-caf\"};\nrevt(global, \"cfia\") -> {regular, \"aifc\", \"audio/x-aiff\"};\nrevt(global, \"CFIA\") -> {regular, \"AIFC\", \"audio/x-aiff\"};\nrevt(global, \"ffia\") -> {regular, \"aiff\", \"audio/x-aiff\"};\nrevt(global, \"FFIA\") -> {regular, \"AIFF\", \"audio/x-aiff\"};\nrevt(global, \"fia\") -> {regular, \"aif\", \"audio/x-aiff\"};\nrevt(global, \"FIA\") -> {regular, \"AIF\", \"audio/x-aiff\"};\nrevt(global, \"caa\") -> {regular, \"aac\", \"audio/x-aac\"};\nrevt(global, \"CAA\") -> {regular, \"AAC\", \"audio/x-aac\"};\nrevt(global, \"abew\") -> {regular, \"weba\", \"audio/webm\"};\nrevt(global, \"ABEW\") -> {regular, \"WEBA\", \"audio/webm\"};\nrevt(global, \"pir\") -> {regular, \"rip\", \"audio/vnd.rip\"};\nrevt(global, \"PIR\") -> {regular, \"RIP\", \"audio/vnd.rip\"};\nrevt(global, \"0069plece\") -> {regular, \"ecelp9600\", \"audio/vnd.nuera.ecelp9600\"};\nrevt(global, \"0069PLECE\") -> {regular, \"ECELP9600\", \"audio/vnd.nuera.ecelp9600\"};\nrevt(global, \"0747plece\") -> {regular, \"ecelp7470\", \"audio/vnd.nuera.ecelp7470\"};\nrevt(global, \"0747PLECE\") -> {regular, \"ECELP7470\", \"audio/vnd.nuera.ecelp7470\"};\nrevt(global, \"0084plece\") -> {regular, \"ecelp4800\", \"audio/vnd.nuera.ecelp4800\"};\nrevt(global, \"0084PLECE\") -> {regular, \"ECELP4800\", \"audio/vnd.nuera.ecelp4800\"};\nrevt(global, \"ayp\") -> {regular, \"pya\", \"audio/vnd.ms-playready.media.pya\"};\nrevt(global, \"AYP\") -> {regular, \"PYA\", \"audio/vnd.ms-playready.media.pya\"};\nrevt(global, \"pvl\") -> {regular, \"lvp\", \"audio/vnd.lucent.voice\"};\nrevt(global, \"PVL\") -> {regular, \"LVP\", \"audio/vnd.lucent.voice\"};\nrevt(global, \"dhstd\") -> {regular, \"dtshd\", \"audio/vnd.dts.hd\"};\nrevt(global, \"DHSTD\") -> {regular, \"DTSHD\", \"audio/vnd.dts.hd\"};\nrevt(global, \"std\") -> {regular, \"dts\", \"audio/vnd.dts\"};\nrevt(global, \"STD\") -> {regular, \"DTS\", \"audio/vnd.dts\"};\nrevt(global, \"ard\") -> {regular, \"dra\", \"audio/vnd.dra\"};\nrevt(global, \"ARD\") -> {regular, \"DRA\", \"audio/vnd.dra\"};\nrevt(global, \"loe\") -> {regular, \"eol\", \"audio/vnd.digital-winds\"};\nrevt(global, \"LOE\") -> {regular, \"EOL\", \"audio/vnd.digital-winds\"};\nrevt(global, \"avvu\") -> {regular, \"uvva\", \"audio/vnd.dece.audio\"};\nrevt(global, \"AVVU\") -> {regular, \"UVVA\", \"audio/vnd.dece.audio\"};\nrevt(global, \"avu\") -> {regular, \"uva\", \"audio/vnd.dece.audio\"};\nrevt(global, \"AVU\") -> {regular, \"UVA\", \"audio/vnd.dece.audio\"};\nrevt(global, \"lis\") -> {regular, \"sil\", \"audio/silk\"};\nrevt(global, \"LIS\") -> {regular, \"SIL\", \"audio/silk\"};\nrevt(global, \"m3s\") -> {regular, \"s3m\", \"audio/s3m\"};\nrevt(global, \"M3S\") -> {regular, \"S3M\", \"audio/s3m\"};\nrevt(global, \"xps\") -> {regular, \"spx\", \"audio/ogg\"};\nrevt(global, \"XPS\") -> {regular, \"SPX\", \"audio/ogg\"};\nrevt(global, \"ggo\") -> {regular, \"ogg\", \"audio/ogg\"};\nrevt(global, \"GGO\") -> {regular, \"OGG\", \"audio/ogg\"};\nrevt(global, \"ago\") -> {regular, \"oga\", \"audio/ogg\"};\nrevt(global, \"AGO\") -> {regular, \"OGA\", \"audio/ogg\"};\nrevt(global, \"a3m\") -> {regular, \"m3a\", \"audio/mpeg\"};\nrevt(global, \"A3M\") -> {regular, \"M3A\", \"audio/mpeg\"};\nrevt(global, \"a2m\") -> {regular, \"m2a\", \"audio/mpeg\"};\nrevt(global, \"A2M\") -> {regular, \"M2A\", \"audio/mpeg\"};\nrevt(global, \"3pm\") -> {regular, \"mp3\", \"audio/mpeg\"};\nrevt(global, \"3PM\") -> {regular, \"MP3\", \"audio/mpeg\"};\nrevt(global, \"a2pm\") -> {regular, \"mp2a\", \"audio/mpeg\"};\nrevt(global, \"A2PM\") -> {regular, \"MP2A\", \"audio/mpeg\"};\nrevt(global, \"2pm\") -> {regular, \"mp2\", \"audio/mpeg\"};\nrevt(global, \"2PM\") -> {regular, \"MP2\", \"audio/mpeg\"};\nrevt(global, \"agpm\") -> {regular, \"mpga\", \"audio/mpeg\"};\nrevt(global, \"AGPM\") -> {regular, \"MPGA\", \"audio/mpeg\"};\nrevt(global, \"a4pm\") -> {regular, \"mp4a\", \"audio/mp4\"};\nrevt(global, \"A4PM\") -> {regular, \"MP4A\", \"audio/mp4\"};\nrevt(global, \"imr\") -> {regular, \"rmi\", \"audio/midi\"};\nrevt(global, \"IMR\") -> {regular, \"RMI\", \"audio/midi\"};\nrevt(global, \"rak\") -> {regular, \"kar\", \"audio/midi\"};\nrevt(global, \"RAK\") -> {regular, \"KAR\", \"audio/midi\"};\nrevt(global, \"idim\") -> {regular, \"midi\", \"audio/midi\"};\nrevt(global, \"IDIM\") -> {regular, \"MIDI\", \"audio/midi\"};\nrevt(global, \"dim\") -> {regular, \"mid\", \"audio/midi\"};\nrevt(global, \"DIM\") -> {regular, \"MID\", \"audio/midi\"};\nrevt(global, \"dns\") -> {regular, \"snd\", \"audio/basic\"};\nrevt(global, \"DNS\") -> {regular, \"SND\", \"audio/basic\"};\nrevt(global, \"ua\") -> {regular, \"au\", \"audio/basic\"};\nrevt(global, \"UA\") -> {regular, \"AU\", \"audio/basic\"};\nrevt(global, \"pda\") -> {regular, \"adp\", \"audio/adpcm\"};\nrevt(global, \"PDA\") -> {regular, \"ADP\", \"audio/adpcm\"};\nrevt(global, \"piz\") -> {regular, \"zip\", \"application/zip\"};\nrevt(global, \"PIZ\") -> {regular, \"ZIP\", \"application/zip\"};\nrevt(global, \"niy\") -> {regular, \"yin\", \"application/yin+xml\"};\nrevt(global, \"NIY\") -> {regular, \"YIN\", \"application/yin+xml\"};\nrevt(global, \"gnay\") -> {regular, \"yang\", \"application/yang\"};\nrevt(global, \"GNAY\") -> {regular, \"YANG\", \"application/yang\"};\nrevt(global, \"mvx\") -> {regular, \"xvm\", \"application/xv+xml\"};\nrevt(global, \"MVX\") -> {regular, \"XVM\", \"application/xv+xml\"};\nrevt(global, \"lmvx\") -> {regular, \"xvml\", \"application/xv+xml\"};\nrevt(global, \"LMVX\") -> {regular, \"XVML\", \"application/xv+xml\"};\nrevt(global, \"lmvhx\") -> {regular, \"xhvml\", \"application/xv+xml\"};\nrevt(global, \"LMVHX\") -> {regular, \"XHVML\", \"application/xv+xml\"};\nrevt(global, \"lmxm\") -> {regular, \"mxml\", \"application/xv+xml\"};\nrevt(global, \"LMXM\") -> {regular, \"MXML\", \"application/xv+xml\"};\nrevt(global, \"fpsx\") -> {regular, \"xspf\", \"application/xspf+xml\"};\nrevt(global, \"FPSX\") -> {regular, \"XSPF\", \"application/xspf+xml\"};\nrevt(global, \"tlsx\") -> {regular, \"xslt\", \"application/xslt+xml\"};\nrevt(global, \"TLSX\") -> {regular, \"XSLT\", \"application/xslt+xml\"};\nrevt(global, \"lpx\") -> {regular, \"xpl\", \"application/xproc+xml\"};\nrevt(global, \"LPX\") -> {regular, \"XPL\", \"application/xproc+xml\"};\nrevt(global, \"pox\") -> {regular, \"xop\", \"application/xop+xml\"};\nrevt(global, \"POX\") -> {regular, \"XOP\", \"application/xop+xml\"};\nrevt(global, \"dtd\") -> {regular, \"dtd\", \"application/xml-dtd\"};\nrevt(global, \"DTD\") -> {regular, \"DTD\", \"application/xml-dtd\"};\nrevt(global, \"lsx\") -> {regular, \"xsl\", \"text/xml\"};\nrevt(global, \"LSX\") -> {regular, \"XSL\", \"text/xml\"};\nrevt(global, \"lmx\") -> {regular, \"xml\", \"text/xml\"};\nrevt(global, \"LMX\") -> {regular, \"XML\", \"text/xml\"};\nrevt(global, \"thx\") -> {regular, \"xht\", \"application/xhtml+xml\"};\nrevt(global, \"THX\") -> {regular, \"XHT\", \"application/xhtml+xml\"};\nrevt(global, \"lmthx\") -> {regular, \"xhtml\", \"application/xhtml+xml\"};\nrevt(global, \"LMTHX\") -> {regular, \"XHTML\", \"application/xhtml+xml\"};\nrevt(global, \"cnex\") -> {regular, \"xenc\", \"application/xenc+xml\"};\nrevt(global, \"CNEX\") -> {regular, \"XENC\", \"application/xenc+xml\"};\nrevt(global, \"fdx\") -> {regular, \"xdf\", \"application/xcap-diff+xml\"};\nrevt(global, \"FDX\") -> {regular, \"XDF\", \"application/xcap-diff+xml\"};\nrevt(global, \"lmax\") -> {regular, \"xaml\", \"application/xaml+xml\"};\nrevt(global, \"LMAX\") -> {regular, \"XAML\", \"application/xaml+xml\"};\nrevt(global, \"8z\") -> {regular, \"z8\", \"application/x-zmachine\"};\nrevt(global, \"8Z\") -> {regular, \"Z8\", \"application/x-zmachine\"};\nrevt(global, \"7z\") -> {regular, \"z7\", \"application/x-zmachine\"};\nrevt(global, \"7Z\") -> {regular, \"Z7\", \"application/x-zmachine\"};\nrevt(global, \"6z\") -> {regular, \"z6\", \"application/x-zmachine\"};\nrevt(global, \"6Z\") -> {regular, \"Z6\", \"application/x-zmachine\"};\nrevt(global, \"5z\") -> {regular, \"z5\", \"application/x-zmachine\"};\nrevt(global, \"5Z\") -> {regular, \"Z5\", \"application/x-zmachine\"};\nrevt(global, \"4z\") -> {regular, \"z4\", \"application/x-zmachine\"};\nrevt(global, \"4Z\") -> {regular, \"Z4\", \"application/x-zmachine\"};\nrevt(global, \"3z\") -> {regular, \"z3\", \"application/x-zmachine\"};\nrevt(global, \"3Z\") -> {regular, \"Z3\", \"application/x-zmachine\"};\nrevt(global, \"2z\") -> {regular, \"z2\", \"application/x-zmachine\"};\nrevt(global, \"2Z\") -> {regular, \"Z2\", \"application/x-zmachine\"};\nrevt(global, \"1z\") -> {regular, \"z1\", \"application/x-zmachine\"};\nrevt(global, \"1Z\") -> {regular, \"Z1\", \"application/x-zmachine\"};\nrevt(global, \"zx\") -> {regular, \"xz\", \"application/x-xz\"};\nrevt(global, \"ZX\") -> {regular, \"XZ\", \"application/x-xz\"};\nrevt(global, \"ipx\") -> {regular, \"xpi\", \"application/x-xpinstall\"};\nrevt(global, \"IPX\") -> {regular, \"XPI\", \"application/x-xpinstall\"};\nrevt(global, \"flx\") -> {regular, \"xlf\", \"application/x-xliff+xml\"};\nrevt(global, \"FLX\") -> {regular, \"XLF\", \"application/x-xliff+xml\"};\nrevt(global, \"gif\") -> {regular, \"fig\", \"application/x-xfig\"};\nrevt(global, \"GIF\") -> {regular, \"FIG\", \"application/x-xfig\"};\nrevt(global, \"trc\") -> {regular, \"crt\", \"application/x-x509-ca-cert\"};\nrevt(global, \"TRC\") -> {regular, \"CRT\", \"application/x-x509-ca-cert\"};\nrevt(global, \"red\") -> {regular, \"der\", \"application/x-x509-ca-cert\"};\nrevt(global, \"RED\") -> {regular, \"DER\", \"application/x-x509-ca-cert\"};\nrevt(global, \"crs\") -> {regular, \"src\", \"application/x-wais-source\"};\nrevt(global, \"CRS\") -> {regular, \"SRC\", \"application/x-wais-source\"};\nrevt(global, \"ratsu\") -> {regular, \"ustar\", \"application/x-ustar\"};\nrevt(global, \"RATSU\") -> {regular, \"USTAR\", \"application/x-ustar\"};\nrevt(global, \"sm\") -> {regular, \"ms\", \"text/troff\"};\nrevt(global, \"SM\") -> {regular, \"MS\", \"text/troff\"};\nrevt(global, \"jbo\") -> {regular, \"obj\", \"application/x-tgif\"};\nrevt(global, \"JBO\") -> {regular, \"OBJ\", \"application/x-tgif\"};\nrevt(global, \"ixet\") -> {regular, \"texi\", \"application/x-texinfo\"};\nrevt(global, \"IXET\") -> {regular, \"TEXI\", \"application/x-texinfo\"};\nrevt(global, \"ofnixet\") -> {regular, \"texinfo\", \"application/x-texinfo\"};\nrevt(global, \"OFNIXET\") -> {regular, \"TEXINFO\", \"application/x-texinfo\"};\nrevt(global, \"mft\") -> {regular, \"tfm\", \"application/x-tex-tfm\"};\nrevt(global, \"MFT\") -> {regular, \"TFM\", \"application/x-tex-tfm\"};\nrevt(global, \"xet\") -> {regular, \"tex\", \"application/x-tex\"};\nrevt(global, \"XET\") -> {regular, \"TEX\", \"application/x-tex\"};\nrevt(global, \"lct\") -> {regular, \"tcl\", \"application/x-tcl\"};\nrevt(global, \"LCT\") -> {regular, \"TCL\", \"application/x-tcl\"};\nrevt(global, \"rat\") -> {regular, \"tar\", \"application/x-tar\"};\nrevt(global, \"RAT\") -> {regular, \"TAR\", \"application/x-tar\"};\nrevt(global, \"mag\") -> {regular, \"gam\", \"application/x-tads\"};\nrevt(global, \"MAG\") -> {regular, \"GAM\", \"application/x-tads\"};\nrevt(global, \"3t\") -> {regular, \"t3\", \"application/x-t3vm-image\"};\nrevt(global, \"3T\") -> {regular, \"T3\", \"application/x-t3vm-image\"};\nrevt(global, \"crc4vs\") -> {regular, \"sv4crc\", \"application/x-sv4crc\"};\nrevt(global, \"CRC4VS\") -> {regular, \"SV4CRC\", \"application/x-sv4crc\"};\nrevt(global, \"oipc4vs\") -> {regular, \"sv4cpio\", \"application/x-sv4cpio\"};\nrevt(global, \"OIPC4VS\") -> {regular, \"SV4CPIO\", \"application/x-sv4cpio\"};\nrevt(global, \"trs\") -> {regular, \"srt\", \"application/x-subrip\"};\nrevt(global, \"TRS\") -> {regular, \"SRT\", \"application/x-subrip\"};\nrevt(global, \"xtis\") -> {regular, \"sitx\", \"application/x-stuffitx\"};\nrevt(global, \"XTIS\") -> {regular, \"SITX\", \"application/x-stuffitx\"};\nrevt(global, \"tis\") -> {regular, \"sit\", \"application/x-stuffit\"};\nrevt(global, \"TIS\") -> {regular, \"SIT\", \"application/x-stuffit\"};\nrevt(global, \"lqs\") -> {regular, \"sql\", \"application/x-sql\"};\nrevt(global, \"LQS\") -> {regular, \"SQL\", \"application/x-sql\"};\nrevt(global, \"pax\") -> {regular, \"xap\", \"application/x-silverlight-app\"};\nrevt(global, \"PAX\") -> {regular, \"XAP\", \"application/x-silverlight-app\"};\nrevt(global, \"fws\") -> {regular, \"swf\", \"application/x-shockwave-flash\"};\nrevt(global, \"FWS\") -> {regular, \"SWF\", \"application/x-shockwave-flash\"};\nrevt(global, \"rahs\") -> {regular, \"shar\", \"application/x-shar\"};\nrevt(global, \"RAHS\") -> {regular, \"SHAR\", \"application/x-shar\"};\nrevt(global, \"hs\") -> {regular, \"sh\", \"application/x-sh\"};\nrevt(global, \"HS\") -> {regular, \"SH\", \"application/x-sh\"};\nrevt(global, \"mpr\") -> {regular, \"rpm\", \"application/x-rpm\"};\nrevt(global, \"MPR\") -> {regular, \"RPM\", \"application/x-rpm\"};\nrevt(global, \"sir\") -> {regular, \"ris\", \"application/x-research-info-systems\"};\nrevt(global, \"SIR\") -> {regular, \"RIS\", \"application/x-research-info-systems\"};\nrevt(global, \"rar\") -> {regular, \"rar\", \"application/x-rar-compressed\"};\nrevt(global, \"RAR\") -> {regular, \"RAR\", \"application/x-rar-compressed\"};\nrevt(global, \"r7p\") -> {regular, \"p7r\", \"application/x-pkcs7-certreqresp\"};\nrevt(global, \"R7P\") -> {regular, \"P7R\", \"application/x-pkcs7-certreqresp\"};\nrevt(global, \"cps\") -> {regular, \"spc\", \"application/x-pkcs7-certificates\"};\nrevt(global, \"CPS\") -> {regular, \"SPC\", \"application/x-pkcs7-certificates\"};\nrevt(global, \"b7p\") -> {regular, \"p7b\", \"application/x-pkcs7-certificates\"};\nrevt(global, \"B7P\") -> {regular, \"P7B\", \"application/x-pkcs7-certificates\"};\nrevt(global, \"xfp\") -> {regular, \"pfx\", \"application/x-pkcs12\"};\nrevt(global, \"XFP\") -> {regular, \"PFX\", \"application/x-pkcs12\"};\nrevt(global, \"21p\") -> {regular, \"p12\", \"application/x-pkcs12\"};\nrevt(global, \"21P\") -> {regular, \"P12\", \"application/x-pkcs12\"};\nrevt(global, \"bzn\") -> {regular, \"nzb\", \"application/x-nzb\"};\nrevt(global, \"BZN\") -> {regular, \"NZB\", \"application/x-nzb\"};\nrevt(global, \"cap\") -> {regular, \"pac\", \"application/x-ns-proxy-autoconfig\"};\nrevt(global, \"CAP\") -> {regular, \"PAC\", \"application/x-ns-proxy-autoconfig\"};\nrevt(global, \"fdc\") -> {regular, \"cdf\", \"application/x-netcdf\"};\nrevt(global, \"FDC\") -> {regular, \"CDF\", \"application/x-netcdf\"};\nrevt(global, \"cn\") -> {regular, \"nc\", \"application/x-netcdf\"};\nrevt(global, \"CN\") -> {regular, \"NC\", \"application/x-netcdf\"};\nrevt(global, \"irw\") -> {regular, \"wri\", \"application/x-mswrite\"};\nrevt(global, \"IRW\") -> {regular, \"WRI\", \"application/x-mswrite\"};\nrevt(global, \"mrt\") -> {regular, \"trm\", \"application/x-msterminal\"};\nrevt(global, \"MRT\") -> {regular, \"TRM\", \"application/x-msterminal\"};\nrevt(global, \"dcs\") -> {regular, \"scd\", \"application/x-msschedule\"};\nrevt(global, \"DCS\") -> {regular, \"SCD\", \"application/x-msschedule\"};\nrevt(global, \"bup\") -> {regular, \"pub\", \"application/x-mspublisher\"};\nrevt(global, \"BUP\") -> {regular, \"PUB\", \"application/x-mspublisher\"};\nrevt(global, \"ynm\") -> {regular, \"mny\", \"application/x-msmoney\"};\nrevt(global, \"YNM\") -> {regular, \"MNY\", \"application/x-msmoney\"};\nrevt(global, \"zme\") -> {regular, \"emz\", \"application/x-msmetafile\"};\nrevt(global, \"ZME\") -> {regular, \"EMZ\", \"application/x-msmetafile\"};\nrevt(global, \"fme\") -> {regular, \"emf\", \"application/x-msmetafile\"};\nrevt(global, \"FME\") -> {regular, \"EMF\", \"application/x-msmetafile\"};\nrevt(global, \"fmw\") -> {regular, \"wmf\", \"application/x-msmetafile\"};\nrevt(global, \"FMW\") -> {regular, \"WMF\", \"application/x-msmetafile\"};\nrevt(global, \"41m\") -> {regular, \"m14\", \"application/x-msmediaview\"};\nrevt(global, \"41M\") -> {regular, \"M14\", \"application/x-msmediaview\"};\nrevt(global, \"31m\") -> {regular, \"m13\", \"application/x-msmediaview\"};\nrevt(global, \"31M\") -> {regular, \"M13\", \"application/x-msmediaview\"};\nrevt(global, \"bvm\") -> {regular, \"mvb\", \"application/x-msmediaview\"};\nrevt(global, \"BVM\") -> {regular, \"MVB\", \"application/x-msmediaview\"};\nrevt(global, \"ism\") -> {regular, \"msi\", \"application/x-msdownload\"};\nrevt(global, \"ISM\") -> {regular, \"MSI\", \"application/x-msdownload\"};\nrevt(global, \"tab\") -> {regular, \"bat\", \"application/x-msdownload\"};\nrevt(global, \"TAB\") -> {regular, \"BAT\", \"application/x-msdownload\"};\nrevt(global, \"moc\") -> {regular, \"com\", \"application/x-msdownload\"};\nrevt(global, \"MOC\") -> {regular, \"COM\", \"application/x-msdownload\"};\nrevt(global, \"lld\") -> {regular, \"dll\", \"application/x-msdownload\"};\nrevt(global, \"LLD\") -> {regular, \"DLL\", \"application/x-msdownload\"};\nrevt(global, \"exe\") -> {regular, \"exe\", \"application/x-msdownload\"};\nrevt(global, \"EXE\") -> {regular, \"EXE\", \"application/x-msdownload\"};\nrevt(global, \"plc\") -> {regular, \"clp\", \"application/x-msclip\"};\nrevt(global, \"PLC\") -> {regular, \"CLP\", \"application/x-msclip\"};\nrevt(global, \"drc\") -> {regular, \"crd\", \"application/x-mscardfile\"};\nrevt(global, \"DRC\") -> {regular, \"CRD\", \"application/x-mscardfile\"};\nrevt(global, \"dbo\") -> {regular, \"obd\", \"application/x-msbinder\"};\nrevt(global, \"DBO\") -> {regular, \"OBD\", \"application/x-msbinder\"};\nrevt(global, \"bdm\") -> {regular, \"mdb\", \"application/x-msaccess\"};\nrevt(global, \"BDM\") -> {regular, \"MDB\", \"application/x-msaccess\"};\nrevt(global, \"pabx\") -> {regular, \"xbap\", \"application/x-ms-xbap\"};\nrevt(global, \"PABX\") -> {regular, \"XBAP\", \"application/x-ms-xbap\"};\nrevt(global, \"zmw\") -> {regular, \"wmz\", \"application/x-msmetafile\"};\nrevt(global, \"ZMW\") -> {regular, \"WMZ\", \"application/x-msmetafile\"};\nrevt(global, \"dmw\") -> {regular, \"wmd\", \"application/x-ms-wmd\"};\nrevt(global, \"DMW\") -> {regular, \"WMD\", \"application/x-ms-wmd\"};\nrevt(global, \"knl\") -> {regular, \"lnk\", \"application/x-ms-shortcut\"};\nrevt(global, \"KNL\") -> {regular, \"LNK\", \"application/x-ms-shortcut\"};\nrevt(global, \"noitacilppa\") -> {regular, \"application\", \"application/x-ms-application\"};\nrevt(global, \"NOITACILPPA\") -> {regular, \"APPLICATION\", \"application/x-ms-application\"};\nrevt(global, \"ibom\") -> {regular, \"mobi\", \"application/x-mobipocket-ebook\"};\nrevt(global, \"IBOM\") -> {regular, \"MOBI\", \"application/x-mobipocket-ebook\"};\nrevt(global, \"crp\") -> {regular, \"prc\", \"application/x-mobipocket-ebook\"};\nrevt(global, \"CRP\") -> {regular, \"PRC\", \"application/x-mobipocket-ebook\"};\nrevt(global, \"eim\") -> {regular, \"mie\", \"application/x-mie\"};\nrevt(global, \"EIM\") -> {regular, \"MIE\", \"application/x-mie\"};\nrevt(global, \"ahl\") -> {regular, \"lha\", \"application/x-lzh-compressed\"};\nrevt(global, \"AHL\") -> {regular, \"LHA\", \"application/x-lzh-compressed\"};\nrevt(global, \"hzl\") -> {regular, \"lzh\", \"application/x-lzh-compressed\"};\nrevt(global, \"HZL\") -> {regular, \"LZH\", \"application/x-lzh-compressed\"};\nrevt(global, \"xetal\") -> {regular, \"latex\", \"application/x-latex\"};\nrevt(global, \"XETAL\") -> {regular, \"LATEX\", \"application/x-latex\"};\nrevt(global, \"lik\") -> {regular, \"kil\", \"application/x-killustrator\"};\nrevt(global, \"LIK\") -> {regular, \"KIL\", \"application/x-killustrator\"};\nrevt(global, \"plnj\") -> {regular, \"jnlp\", \"application/x-java-jnlp-file\"};\nrevt(global, \"PLNJ\") -> {regular, \"JNLP\", \"application/x-java-jnlp-file\"};\nrevt(global, \"osi\") -> {regular, \"iso\", \"application/x-iso9660-image\"};\nrevt(global, \"OSI\") -> {regular, \"ISO\", \"application/x-iso9660-image\"};\nrevt(global, \"llatsni\") -> {regular, \"install\", \"application/x-install-instructions\"};\nrevt(global, \"LLATSNI\") -> {regular, \"INSTALL\", \"application/x-install-instructions\"};\nrevt(global, \"fdh\") -> {regular, \"hdf\", \"application/x-hdf\"};\nrevt(global, \"FDH\") -> {regular, \"HDF\", \"application/x-hdf\"};\nrevt(global, \"zgt\") -> {regular, \"tgz\", \"application/x-gzip\"};\nrevt(global, \"ZGT\") -> {regular, \"TGZ\", \"application/x-gzip\"};\nrevt(global, \"zg\") -> {regular, \"gz\", \"application/x-gzip\"};\nrevt(global, \"ZG\") -> {regular, \"GZ\", \"application/x-gzip\"};\nrevt(global, \"ratg\") -> {regular, \"gtar\", \"application/x-gtar\"};\nrevt(global, \"RATG\") -> {regular, \"GTAR\", \"application/x-gtar\"};\nrevt(global, \"spmarg\") -> {regular, \"gramps\", \"application/x-gramps-xml\"};\nrevt(global, \"SPMARG\") -> {regular, \"GRAMPS\", \"application/x-gramps-xml\"};\nrevt(global, \"ciremung\") -> {regular, \"gnumeric\", \"application/x-gnumeric\"};\nrevt(global, \"CIREMUNG\") -> {regular, \"GNUMERIC\", \"application/x-gnumeric\"};\nrevt(global, \"xlu\") -> {regular, \"ulx\", \"application/x-glulx\"};\nrevt(global, \"XLU\") -> {regular, \"ULX\", \"application/x-glulx\"};\nrevt(global, \"acg\") -> {regular, \"gca\", \"application/x-gca-compressed\"};\nrevt(global, \"ACG\") -> {regular, \"GCA\", \"application/x-gca-compressed\"};\nrevt(global, \"lps\") -> {regular, \"spl\", \"application/x-futuresplash\"};\nrevt(global, \"LPS\") -> {regular, \"SPL\", \"application/x-futuresplash\"};\nrevt(global, \"cra\") -> {regular, \"arc\", \"application/x-freearc\"};\nrevt(global, \"CRA\") -> {regular, \"ARC\", \"application/x-freearc\"};\nrevt(global, \"mfa\") -> {regular, \"afm\", \"application/x-font-type1\"};\nrevt(global, \"MFA\") -> {regular, \"AFM\", \"application/x-font-type1\"};\nrevt(global, \"mfp\") -> {regular, \"pfm\", \"application/x-font-type1\"};\nrevt(global, \"MFP\") -> {regular, \"PFM\", \"application/x-font-type1\"};\nrevt(global, \"bfp\") -> {regular, \"pfb\", \"application/x-font-type1\"};\nrevt(global, \"BFP\") -> {regular, \"PFB\", \"application/x-font-type1\"};\nrevt(global, \"afp\") -> {regular, \"pfa\", \"application/x-font-type1\"};\nrevt(global, \"AFP\") -> {regular, \"PFA\", \"application/x-font-type1\"};\nrevt(global, \"ctt\") -> {regular, \"ttc\", \"application/x-font-ttf\"};\nrevt(global, \"CTT\") -> {regular, \"TTC\", \"application/x-font-ttf\"};\nrevt(global, \"ftt\") -> {regular, \"ttf\", \"application/x-font-ttf\"};\nrevt(global, \"FTT\") -> {regular, \"TTF\", \"application/x-font-ttf\"};\nrevt(global, \"fns\") -> {regular, \"snf\", \"application/x-font-snf\"};\nrevt(global, \"FNS\") -> {regular, \"SNF\", \"application/x-font-snf\"};\nrevt(global, \"fcp\") -> {regular, \"pcf\", \"application/x-font-pcf\"};\nrevt(global, \"FCP\") -> {regular, \"PCF\", \"application/x-font-pcf\"};\nrevt(global, \"fto\") -> {regular, \"otf\", \"application/x-font-otf\"};\nrevt(global, \"FTO\") -> {regular, \"OTF\", \"application/x-font-otf\"};\nrevt(global, \"fsp\") -> {regular, \"psf\", \"application/x-font-linux-psf\"};\nrevt(global, \"FSP\") -> {regular, \"PSF\", \"application/x-font-linux-psf\"};\nrevt(global, \"fsg\") -> {regular, \"gsf\", \"application/x-font-ghostscript\"};\nrevt(global, \"FSG\") -> {regular, \"GSF\", \"application/x-font-ghostscript\"};\nrevt(global, \"fdb\") -> {regular, \"bdf\", \"application/x-font-bdf\"};\nrevt(global, \"FDB\") -> {regular, \"BDF\", \"application/x-font-bdf\"};\nrevt(global, \"ave\") -> {regular, \"eva\", \"application/x-eva\"};\nrevt(global, \"AVE\") -> {regular, \"EVA\", \"application/x-eva\"};\nrevt(global, \"yve\") -> {regular, \"evy\", \"application/x-envoy\"};\nrevt(global, \"YVE\") -> {regular, \"EVY\", \"application/x-envoy\"};\nrevt(global, \"ivd\") -> {regular, \"dvi\", \"application/x-dvi\"};\nrevt(global, \"IVD\") -> {regular, \"DVI\", \"application/x-dvi\"};\nrevt(global, \"ser\") -> {regular, \"res\", \"application/x-dtbresource+xml\"};\nrevt(global, \"SER\") -> {regular, \"RES\", \"application/x-dtbresource+xml\"};\nrevt(global, \"btd\") -> {regular, \"dtb\", \"application/x-dtbook+xml\"};\nrevt(global, \"BTD\") -> {regular, \"DTB\", \"application/x-dtbook+xml\"};\nrevt(global, \"xcn\") -> {regular, \"ncx\", \"application/x-dtbncx+xml\"};\nrevt(global, \"XCN\") -> {regular, \"NCX\", \"application/x-dtbncx+xml\"};\nrevt(global, \"daw\") -> {regular, \"wad\", \"application/x-doom\"};\nrevt(global, \"DAW\") -> {regular, \"WAD\", \"application/x-doom\"};\nrevt(global, \"aws\") -> {regular, \"swa\", \"application/x-director\"};\nrevt(global, \"AWS\") -> {regular, \"SWA\", \"application/x-director\"};\nrevt(global, \"dgf\") -> {regular, \"fgd\", \"application/x-director\"};\nrevt(global, \"DGF\") -> {regular, \"FGD\", \"application/x-director\"};\nrevt(global, \"d3w\") -> {regular, \"w3d\", \"application/x-director\"};\nrevt(global, \"D3W\") -> {regular, \"W3D\", \"application/x-director\"};\nrevt(global, \"txc\") -> {regular, \"cxt\", \"application/x-director\"};\nrevt(global, \"TXC\") -> {regular, \"CXT\", \"application/x-director\"};\nrevt(global, \"tcc\") -> {regular, \"cct\", \"application/x-director\"};\nrevt(global, \"TCC\") -> {regular, \"CCT\", \"application/x-director\"};\nrevt(global, \"tsc\") -> {regular, \"cst\", \"application/x-director\"};\nrevt(global, \"TSC\") -> {regular, \"CST\", \"application/x-director\"};\nrevt(global, \"rxd\") -> {regular, \"dxr\", \"application/x-director\"};\nrevt(global, \"RXD\") -> {regular, \"DXR\", \"application/x-director\"};\nrevt(global, \"rcd\") -> {regular, \"dcr\", \"application/x-director\"};\nrevt(global, \"RCD\") -> {regular, \"DCR\", \"application/x-director\"};\nrevt(global, \"rid\") -> {regular, \"dir\", \"application/x-director\"};\nrevt(global, \"RID\") -> {regular, \"DIR\", \"application/x-director\"};\nrevt(global, \"cgd\") -> {regular, \"dgc\", \"application/x-dgc-compressed\"};\nrevt(global, \"CGD\") -> {regular, \"DGC\", \"application/x-dgc-compressed\"};\nrevt(global, \"bedu\") -> {regular, \"udeb\", \"application/x-debian-package\"};\nrevt(global, \"BEDU\") -> {regular, \"UDEB\", \"application/x-debian-package\"};\nrevt(global, \"bed\") -> {regular, \"deb\", \"application/x-debian-package\"};\nrevt(global, \"BED\") -> {regular, \"DEB\", \"application/x-debian-package\"};\nrevt(global, \"hsc\") -> {regular, \"csh\", \"application/x-csh\"};\nrevt(global, \"HSC\") -> {regular, \"CSH\", \"application/x-csh\"};\nrevt(global, \"oipc\") -> {regular, \"cpio\", \"application/x-cpio\"};\nrevt(global, \"OIPC\") -> {regular, \"CPIO\", \"application/x-cpio\"};\nrevt(global, \"csn\") -> {regular, \"nsc\", \"application/x-conference\"};\nrevt(global, \"CSN\") -> {regular, \"NSC\", \"application/x-conference\"};\nrevt(global, \"ngp\") -> {regular, \"pgn\", \"application/x-chess-pgn\"};\nrevt(global, \"NGP\") -> {regular, \"PGN\", \"application/x-chess-pgn\"};\nrevt(global, \"tahc\") -> {regular, \"chat\", \"application/x-chat\"};\nrevt(global, \"TAHC\") -> {regular, \"CHAT\", \"application/x-chat\"};\nrevt(global, \"sfc\") -> {regular, \"cfs\", \"application/x-cfs-compressed\"};\nrevt(global, \"SFC\") -> {regular, \"CFS\", \"application/x-cfs-compressed\"};\nrevt(global, \"dcv\") -> {regular, \"vcd\", \"application/x-cdlink\"};\nrevt(global, \"DCV\") -> {regular, \"VCD\", \"application/x-cdlink\"};\nrevt(global, \"7bc\") -> {regular, \"cb7\", \"application/x-cbr\"};\nrevt(global, \"7BC\") -> {regular, \"CB7\", \"application/x-cbr\"};\nrevt(global, \"zbc\") -> {regular, \"cbz\", \"application/x-cbr\"};\nrevt(global, \"ZBC\") -> {regular, \"CBZ\", \"application/x-cbr\"};\nrevt(global, \"tbc\") -> {regular, \"cbt\", \"application/x-cbr\"};\nrevt(global, \"TBC\") -> {regular, \"CBT\", \"application/x-cbr\"};\nrevt(global, \"abc\") -> {regular, \"cba\", \"application/x-cbr\"};\nrevt(global, \"ABC\") -> {regular, \"CBA\", \"application/x-cbr\"};\nrevt(global, \"rbc\") -> {regular, \"cbr\", \"application/x-cbr\"};\nrevt(global, \"RBC\") -> {regular, \"CBR\", \"application/x-cbr\"};\nrevt(global, \"zob\") -> {regular, \"boz\", \"application/x-bzip2\"};\nrevt(global, \"ZOB\") -> {regular, \"BOZ\", \"application/x-bzip2\"};\nrevt(global, \"2zb\") -> {regular, \"bz2\", \"application/x-bzip2\"};\nrevt(global, \"2ZB\") -> {regular, \"BZ2\", \"application/x-bzip2\"};\nrevt(global, \"zb\") -> {regular, \"bz\", \"application/x-bzip\"};\nrevt(global, \"ZB\") -> {regular, \"BZ\", \"application/x-bzip\"};\nrevt(global, \"brolb\") -> {regular, \"blorb\", \"application/x-blorb\"};\nrevt(global, \"BROLB\") -> {regular, \"BLORB\", \"application/x-blorb\"};\nrevt(global, \"blb\") -> {regular, \"blb\", \"application/x-blorb\"};\nrevt(global, \"BLB\") -> {regular, \"BLB\", \"application/x-blorb\"};\nrevt(global, \"tnerrot\") -> {regular, \"torrent\", \"application/x-bittorrent\"};\nrevt(global, \"TNERROT\") -> {regular, \"TORRENT\", \"application/x-bittorrent\"};\nrevt(global, \"oipcb\") -> {regular, \"bcpio\", \"application/x-bcpio\"};\nrevt(global, \"OIPCB\") -> {regular, \"BCPIO\", \"application/x-bcpio\"};\nrevt(global, \"saa\") -> {regular, \"aas\", \"application/x-authorware-seg\"};\nrevt(global, \"SAA\") -> {regular, \"AAS\", \"application/x-authorware-seg\"};\nrevt(global, \"maa\") -> {regular, \"aam\", \"application/x-authorware-map\"};\nrevt(global, \"MAA\") -> {regular, \"AAM\", \"application/x-authorware-map\"};\nrevt(global, \"xov\") -> {regular, \"vox\", \"application/x-authorware-bin\"};\nrevt(global, \"XOV\") -> {regular, \"VOX\", \"application/x-authorware-bin\"};\nrevt(global, \"23u\") -> {regular, \"u32\", \"application/x-authorware-bin\"};\nrevt(global, \"23U\") -> {regular, \"U32\", \"application/x-authorware-bin\"};\nrevt(global, \"23x\") -> {regular, \"x32\", \"application/x-authorware-bin\"};\nrevt(global, \"23X\") -> {regular, \"X32\", \"application/x-authorware-bin\"};\nrevt(global, \"baa\") -> {regular, \"aab\", \"application/x-authorware-bin\"};\nrevt(global, \"BAA\") -> {regular, \"AAB\", \"application/x-authorware-bin\"};\nrevt(global, \"gmd\") -> {regular, \"dmg\", \"application/x-apple-diskimage\"};\nrevt(global, \"GMD\") -> {regular, \"DMG\", \"application/x-apple-diskimage\"};\nrevt(global, \"eca\") -> {regular, \"ace\", \"application/x-ace-compressed\"};\nrevt(global, \"ECA\") -> {regular, \"ACE\", \"application/x-ace-compressed\"};\nrevt(global, \"wba\") -> {regular, \"abw\", \"application/x-abiword\"};\nrevt(global, \"WBA\") -> {regular, \"ABW\", \"application/x-abiword\"};\nrevt(global, \"z7\") -> {regular, \"7z\", \"application/x-7z-compressed\"};\nrevt(global, \"Z7\") -> {regular, \"7Z\", \"application/x-7z-compressed\"};\nrevt(global, \"ycilopsw\") -> {regular, \"wspolicy\", \"application/wspolicy+xml\"};\nrevt(global, \"YCILOPSW\") -> {regular, \"WSPOLICY\", \"application/wspolicy+xml\"};\nrevt(global, \"ldsw\") -> {regular, \"wsdl\", \"application/wsdl+xml\"};\nrevt(global, \"LDSW\") -> {regular, \"WSDL\", \"application/wsdl+xml\"};\nrevt(global, \"plh\") -> {regular, \"hlp\", \"application/winhlp\"};\nrevt(global, \"PLH\") -> {regular, \"HLP\", \"application/winhlp\"};\nrevt(global, \"tgw\") -> {regular, \"wgt\", \"application/widget\"};\nrevt(global, \"TGW\") -> {regular, \"WGT\", \"application/widget\"};\nrevt(global, \"lmxv\") -> {regular, \"vxml\", \"application/voicexml+xml\"};\nrevt(global, \"LMXV\") -> {regular, \"VXML\", \"application/voicexml+xml\"};\nrevt(global, \"zaz\") -> {regular, \"zaz\", \"application/vnd.zzazz.deck+xml\"};\nrevt(global, \"ZAZ\") -> {regular, \"ZAZ\", \"application/vnd.zzazz.deck+xml\"};\nrevt(global, \"zriz\") -> {regular, \"zirz\", \"application/vnd.zul\"};\nrevt(global, \"ZRIZ\") -> {regular, \"ZIRZ\", \"application/vnd.zul\"};\nrevt(global, \"riz\") -> {regular, \"zir\", \"application/vnd.zul\"};\nrevt(global, \"RIZ\") -> {regular, \"ZIR\", \"application/vnd.zul\"};\nrevt(global, \"pmc\") -> {regular, \"cmp\", \"application/vnd.yellowriver-custom-menu\"};\nrevt(global, \"PMC\") -> {regular, \"CMP\", \"application/vnd.yellowriver-custom-menu\"};\nrevt(global, \"fps\") -> {regular, \"spf\", \"application/vnd.yamaha.smaf-phrase\"};\nrevt(global, \"FPS\") -> {regular, \"SPF\", \"application/vnd.yamaha.smaf-phrase\"};\nrevt(global, \"fas\") -> {regular, \"saf\", \"application/vnd.yamaha.smaf-audio\"};\nrevt(global, \"FAS\") -> {regular, \"SAF\", \"application/vnd.yamaha.smaf-audio\"};\nrevt(global, \"gvpfso\") -> {regular, \"osfpvg\", \"application/vnd.yamaha.openscoreformat.osfpvg+xml\"};\nrevt(global, \"GVPFSO\") -> {regular, \"OSFPVG\", \"application/vnd.yamaha.openscoreformat.osfpvg+xml\"};\nrevt(global, \"fso\") -> {regular, \"osf\", \"application/vnd.yamaha.openscoreformat\"};\nrevt(global, \"FSO\") -> {regular, \"OSF\", \"application/vnd.yamaha.openscoreformat\"};\nrevt(global, \"pvh\") -> {regular, \"hvp\", \"application/vnd.yamaha.hv-voice\"};\nrevt(global, \"PVH\") -> {regular, \"HVP\", \"application/vnd.yamaha.hv-voice\"};\nrevt(global, \"svh\") -> {regular, \"hvs\", \"application/vnd.yamaha.hv-script\"};\nrevt(global, \"SVH\") -> {regular, \"HVS\", \"application/vnd.yamaha.hv-script\"};\nrevt(global, \"dvh\") -> {regular, \"hvd\", \"application/vnd.yamaha.hv-dic\"};\nrevt(global, \"DVH\") -> {regular, \"HVD\", \"application/vnd.yamaha.hv-dic\"};\nrevt(global, \"ldfx\") -> {regular, \"xfdl\", \"application/vnd.xfdl\"};\nrevt(global, \"LDFX\") -> {regular, \"XFDL\", \"application/vnd.xfdl\"};\nrevt(global, \"rax\") -> {regular, \"xar\", \"application/vnd.xara\"};\nrevt(global, \"RAX\") -> {regular, \"XAR\", \"application/vnd.xara\"};\nrevt(global, \"fts\") -> {regular, \"stf\", \"application/vnd.wt.stf\"};\nrevt(global, \"FTS\") -> {regular, \"STF\", \"application/vnd.wt.stf\"};\nrevt(global, \"dqw\") -> {regular, \"wqd\", \"application/vnd.wqd\"};\nrevt(global, \"DQW\") -> {regular, \"WQD\", \"application/vnd.wqd\"};\nrevt(global, \"dpw\") -> {regular, \"wpd\", \"application/vnd.wordperfect\"};\nrevt(global, \"DPW\") -> {regular, \"WPD\", \"application/vnd.wordperfect\"};\nrevt(global, \"pbn\") -> {regular, \"nbp\", \"application/vnd.wolfram.player\"};\nrevt(global, \"PBN\") -> {regular, \"NBP\", \"application/vnd.wolfram.player\"};\nrevt(global, \"btw\") -> {regular, \"wtb\", \"application/vnd.webturbo\"};\nrevt(global, \"BTW\") -> {regular, \"WTB\", \"application/vnd.webturbo\"};\nrevt(global, \"cslmw\") -> {regular, \"wmlsc\", \"application/vnd.wap.wmlscriptc\"};\nrevt(global, \"CSLMW\") -> {regular, \"WMLSC\", \"application/vnd.wap.wmlscriptc\"};\nrevt(global, \"clmw\") -> {regular, \"wmlc\", \"application/vnd.wap.wmlc\"};\nrevt(global, \"CLMW\") -> {regular, \"WMLC\", \"application/vnd.wap.wmlc\"};\nrevt(global, \"lmxbw\") -> {regular, \"wbxml\", \"application/vnd.wap.wbxml\"};\nrevt(global, \"LMXBW\") -> {regular, \"WBXML\", \"application/vnd.wap.wbxml\"};\nrevt(global, \"cls\") -> {regular, \"slc\", \"application/vnd.wap.slc\"};\nrevt(global, \"CLS\") -> {regular, \"SLC\", \"application/vnd.wap.slc\"};\nrevt(global, \"cis\") -> {regular, \"sic\", \"application/vnd.wap.sic\"};\nrevt(global, \"CIS\") -> {regular, \"SIC\", \"application/vnd.wap.sic\"};\nrevt(global, \"fsv\") -> {regular, \"vsf\", \"application/vnd.vsf\"};\nrevt(global, \"FSV\") -> {regular, \"VSF\", \"application/vnd.vsf\"};\nrevt(global, \"siv\") -> {regular, \"vis\", \"application/vnd.visionary\"};\nrevt(global, \"SIV\") -> {regular, \"VIS\", \"application/vnd.visionary\"};\nrevt(global, \"wsv\") -> {regular, \"vsw\", \"application/vnd.visio\"};\nrevt(global, \"WSV\") -> {regular, \"VSW\", \"application/vnd.visio\"};\nrevt(global, \"ssv\") -> {regular, \"vss\", \"application/vnd.visio\"};\nrevt(global, \"SSV\") -> {regular, \"VSS\", \"application/vnd.visio\"};\nrevt(global, \"tsv\") -> {regular, \"vst\", \"application/vnd.visio\"};\nrevt(global, \"TSV\") -> {regular, \"VST\", \"application/vnd.visio\"};\nrevt(global, \"dsv\") -> {regular, \"vsd\", \"application/vnd.visio\"};\nrevt(global, \"DSV\") -> {regular, \"VSD\", \"application/vnd.visio\"};\nrevt(global, \"xcv\") -> {regular, \"vcx\", \"application/vnd.vcx\"};\nrevt(global, \"XCV\") -> {regular, \"VCX\", \"application/vnd.vcx\"};\nrevt(global, \"lmou\") -> {regular, \"uoml\", \"application/vnd.uoml+xml\"};\nrevt(global, \"LMOU\") -> {regular, \"UOML\", \"application/vnd.uoml+xml\"};\nrevt(global, \"bewytinu\") -> {regular, \"unityweb\", \"application/vnd.unity\"};\nrevt(global, \"BEWYTINU\") -> {regular, \"UNITYWEB\", \"application/vnd.unity\"};\nrevt(global, \"jmu\") -> {regular, \"umj\", \"application/vnd.umajin\"};\nrevt(global, \"JMU\") -> {regular, \"UMJ\", \"application/vnd.umajin\"};\nrevt(global, \"ztu\") -> {regular, \"utz\", \"application/vnd.uiq.theme\"};\nrevt(global, \"ZTU\") -> {regular, \"UTZ\", \"application/vnd.uiq.theme\"};\nrevt(global, \"ldfu\") -> {regular, \"ufdl\", \"application/vnd.ufdl\"};\nrevt(global, \"LDFU\") -> {regular, \"UFDL\", \"application/vnd.ufdl\"};\nrevt(global, \"dfu\") -> {regular, \"ufd\", \"application/vnd.ufdl\"};\nrevt(global, \"DFU\") -> {regular, \"UFD\", \"application/vnd.ufdl\"};\nrevt(global, \"art\") -> {regular, \"tra\", \"application/vnd.trueapp\"};\nrevt(global, \"ART\") -> {regular, \"TRA\", \"application/vnd.trueapp\"};\nrevt(global, \"sxm\") -> {regular, \"mxs\", \"application/vnd.triscape.mxs\"};\nrevt(global, \"SXM\") -> {regular, \"MXS\", \"application/vnd.triscape.mxs\"};\nrevt(global, \"tpt\") -> {regular, \"tpt\", \"application/vnd.trid.tpt\"};\nrevt(global, \"TPT\") -> {regular, \"TPT\", \"application/vnd.trid.tpt\"};\nrevt(global, \"omt\") -> {regular, \"tmo\", \"application/vnd.tmobile-livetv\"};\nrevt(global, \"OMT\") -> {regular, \"TMO\", \"application/vnd.tmobile-livetv\"};\nrevt(global, \"pmd\") -> {regular, \"dmp\", \"application/vnd.tcpdump.pcap\"};\nrevt(global, \"PMD\") -> {regular, \"DMP\", \"application/vnd.tcpdump.pcap\"};\nrevt(global, \"pac\") -> {regular, \"cap\", \"application/vnd.tcpdump.pcap\"};\nrevt(global, \"PAC\") -> {regular, \"CAP\", \"application/vnd.tcpdump.pcap\"};\nrevt(global, \"pacp\") -> {regular, \"pcap\", \"application/vnd.tcpdump.pcap\"};\nrevt(global, \"PACP\") -> {regular, \"PCAP\", \"application/vnd.tcpdump.pcap\"};\nrevt(global, \"oat\") -> {regular, \"tao\", \"application/vnd.tao.intent-module-archive\"};\nrevt(global, \"OAT\") -> {regular, \"TAO\", \"application/vnd.tao.intent-module-archive\"};\nrevt(global, \"mdx\") -> {regular, \"xdm\", \"application/vnd.syncml.dm+xml\"};\nrevt(global, \"MDX\") -> {regular, \"XDM\", \"application/vnd.syncml.dm+xml\"};\nrevt(global, \"mdb\") -> {regular, \"bdm\", \"application/vnd.syncml.dm+wbxml\"};\nrevt(global, \"MDB\") -> {regular, \"BDM\", \"application/vnd.syncml.dm+wbxml\"};\nrevt(global, \"msx\") -> {regular, \"xsm\", \"application/vnd.syncml+xml\"};\nrevt(global, \"MSX\") -> {regular, \"XSM\", \"application/vnd.syncml+xml\"};\nrevt(global, \"xsis\") -> {regular, \"sisx\", \"application/vnd.symbian.install\"};\nrevt(global, \"XSIS\") -> {regular, \"SISX\", \"application/vnd.symbian.install\"};\nrevt(global, \"sis\") -> {regular, \"sis\", \"application/vnd.symbian.install\"};\nrevt(global, \"SIS\") -> {regular, \"SIS\", \"application/vnd.symbian.install\"};\nrevt(global, \"dvs\") -> {regular, \"svd\", \"application/vnd.svd\"};\nrevt(global, \"DVS\") -> {regular, \"SVD\", \"application/vnd.svd\"};\nrevt(global, \"psus\") -> {regular, \"susp\", \"application/vnd.sus-calendar\"};\nrevt(global, \"PSUS\") -> {regular, \"SUSP\", \"application/vnd.sus-calendar\"};\nrevt(global, \"sus\") -> {regular, \"sus\", \"application/vnd.sus-calendar\"};\nrevt(global, \"SUS\") -> {regular, \"SUS\", \"application/vnd.sus-calendar\"};\nrevt(global, \"wts\") -> {regular, \"stw\", \"application/vnd.sun.xml.writer.template\"};\nrevt(global, \"WTS\") -> {regular, \"STW\", \"application/vnd.sun.xml.writer.template\"};\nrevt(global, \"gxs\") -> {regular, \"sxg\", \"application/vnd.sun.xml.writer.global\"};\nrevt(global, \"GXS\") -> {regular, \"SXG\", \"application/vnd.sun.xml.writer.global\"};\nrevt(global, \"wxs\") -> {regular, \"sxw\", \"application/vnd.sun.xml.writer\"};\nrevt(global, \"WXS\") -> {regular, \"SXW\", \"application/vnd.sun.xml.writer\"};\nrevt(global, \"mxs\") -> {regular, \"sxm\", \"application/vnd.sun.xml.math\"};\nrevt(global, \"MXS\") -> {regular, \"SXM\", \"application/vnd.sun.xml.math\"};\nrevt(global, \"its\") -> {regular, \"sti\", \"application/vnd.sun.xml.impress.template\"};\nrevt(global, \"ITS\") -> {regular, \"STI\", \"application/vnd.sun.xml.impress.template\"};\nrevt(global, \"ixs\") -> {regular, \"sxi\", \"application/vnd.sun.xml.impress\"};\nrevt(global, \"IXS\") -> {regular, \"SXI\", \"application/vnd.sun.xml.impress\"};\nrevt(global, \"dts\") -> {regular, \"std\", \"application/vnd.sun.xml.draw.template\"};\nrevt(global, \"DTS\") -> {regular, \"STD\", \"application/vnd.sun.xml.draw.template\"};\nrevt(global, \"dxs\") -> {regular, \"sxd\", \"application/vnd.sun.xml.draw\"};\nrevt(global, \"DXS\") -> {regular, \"SXD\", \"application/vnd.sun.xml.draw\"};\nrevt(global, \"cts\") -> {regular, \"stc\", \"application/vnd.sun.xml.calc.template\"};\nrevt(global, \"CTS\") -> {regular, \"STC\", \"application/vnd.sun.xml.calc.template\"};\nrevt(global, \"cxs\") -> {regular, \"sxc\", \"application/vnd.sun.xml.calc\"};\nrevt(global, \"CXS\") -> {regular, \"SXC\", \"application/vnd.sun.xml.calc\"};\nrevt(global, \"ms\") -> {regular, \"sm\", \"application/vnd.stepmania.stepchart\"};\nrevt(global, \"MS\") -> {regular, \"SM\", \"application/vnd.stepmania.stepchart\"};\nrevt(global, \"pizms\") -> {regular, \"smzip\", \"application/vnd.stepmania.package\"};\nrevt(global, \"PIZMS\") -> {regular, \"SMZIP\", \"application/vnd.stepmania.package\"};\nrevt(global, \"lgs\") -> {regular, \"sgl\", \"application/vnd.stardivision.writer-global\"};\nrevt(global, \"LGS\") -> {regular, \"SGL\", \"application/vnd.stardivision.writer-global\"};\nrevt(global, \"rov\") -> {regular, \"vor\", \"application/vnd.stardivision.writer\"};\nrevt(global, \"ROV\") -> {regular, \"VOR\", \"application/vnd.stardivision.writer\"};\nrevt(global, \"wds\") -> {regular, \"sdw\", \"application/vnd.stardivision.writer\"};\nrevt(global, \"WDS\") -> {regular, \"SDW\", \"application/vnd.stardivision.writer\"};\nrevt(global, \"fms\") -> {regular, \"smf\", \"application/vnd.stardivision.math\"};\nrevt(global, \"FMS\") -> {regular, \"SMF\", \"application/vnd.stardivision.math\"};\nrevt(global, \"dds\") -> {regular, \"sdd\", \"application/vnd.stardivision.impress\"};\nrevt(global, \"DDS\") -> {regular, \"SDD\", \"application/vnd.stardivision.impress\"};\nrevt(global, \"ads\") -> {regular, \"sda\", \"application/vnd.stardivision.draw\"};\nrevt(global, \"ADS\") -> {regular, \"SDA\", \"application/vnd.stardivision.draw\"};\nrevt(global, \"cds\") -> {regular, \"sdc\", \"application/vnd.stardivision.calc\"};\nrevt(global, \"CDS\") -> {regular, \"SDC\", \"application/vnd.stardivision.calc\"};\nrevt(global, \"sfs\") -> {regular, \"sfs\", \"application/vnd.spotfire.sfs\"};\nrevt(global, \"SFS\") -> {regular, \"SFS\", \"application/vnd.spotfire.sfs\"};\nrevt(global, \"pxd\") -> {regular, \"dxp\", \"application/vnd.spotfire.dxp\"};\nrevt(global, \"PXD\") -> {regular, \"DXP\", \"application/vnd.spotfire.dxp\"};\nrevt(global, \"dkds\") -> {regular, \"sdkd\", \"application/vnd.solent.sdkm+xml\"};\nrevt(global, \"DKDS\") -> {regular, \"SDKD\", \"application/vnd.solent.sdkm+xml\"};\nrevt(global, \"mkds\") -> {regular, \"sdkm\", \"application/vnd.solent.sdkm+xml\"};\nrevt(global, \"MKDS\") -> {regular, \"SDKM\", \"application/vnd.solent.sdkm+xml\"};\nrevt(global, \"rehcaet\") -> {regular, \"teacher\", \"application/vnd.smart.teacher\"};\nrevt(global, \"REHCAET\") -> {regular, \"TEACHER\", \"application/vnd.smart.teacher\"};\nrevt(global, \"fmm\") -> {regular, \"mmf\", \"application/vnd.smaf\"};\nrevt(global, \"FMM\") -> {regular, \"MMF\", \"application/vnd.smaf\"};\nrevt(global, \"sdwt\") -> {regular, \"twds\", \"application/vnd.simtech-mindmapper\"};\nrevt(global, \"SDWT\") -> {regular, \"TWDS\", \"application/vnd.simtech-mindmapper\"};\nrevt(global, \"dwt\") -> {regular, \"twd\", \"application/vnd.simtech-mindmapper\"};\nrevt(global, \"DWT\") -> {regular, \"TWD\", \"application/vnd.simtech-mindmapper\"};\nrevt(global, \"kpi\") -> {regular, \"ipk\", \"application/vnd.shana.informed.package\"};\nrevt(global, \"KPI\") -> {regular, \"IPK\", \"application/vnd.shana.informed.package\"};\nrevt(global, \"fii\") -> {regular, \"iif\", \"application/vnd.shana.informed.interchange\"};\nrevt(global, \"FII\") -> {regular, \"IIF\", \"application/vnd.shana.informed.interchange\"};\nrevt(global, \"pti\") -> {regular, \"itp\", \"application/vnd.shana.informed.formtemplate\"};\nrevt(global, \"PTI\") -> {regular, \"ITP\", \"application/vnd.shana.informed.formtemplate\"};\nrevt(global, \"mfi\") -> {regular, \"ifm\", \"application/vnd.shana.informed.formdata\"};\nrevt(global, \"MFI\") -> {regular, \"IFM\", \"application/vnd.shana.informed.formdata\"};\nrevt(global, \"fmes\") -> {regular, \"semf\", \"application/vnd.semf\"};\nrevt(global, \"FMES\") -> {regular, \"SEMF\", \"application/vnd.semf\"};\nrevt(global, \"dmes\") -> {regular, \"semd\", \"application/vnd.semd\"};\nrevt(global, \"DMES\") -> {regular, \"SEMD\", \"application/vnd.semd\"};\nrevt(global, \"ames\") -> {regular, \"sema\", \"application/vnd.sema\"};\nrevt(global, \"AMES\") -> {regular, \"SEMA\", \"application/vnd.sema\"};\nrevt(global, \"ees\") -> {regular, \"see\", \"application/vnd.seemail\"};\nrevt(global, \"EES\") -> {regular, \"SEE\", \"application/vnd.seemail\"};\nrevt(global, \"ts\") -> {regular, \"st\", \"application/vnd.sailingtracker.track\"};\nrevt(global, \"TS\") -> {regular, \"ST\", \"application/vnd.sailingtracker.track\"};\nrevt(global, \"66knil\") -> {regular, \"link66\", \"application/vnd.route66.link66+xml\"};\nrevt(global, \"66KNIL\") -> {regular, \"LINK66\", \"application/vnd.route66.link66+xml\"};\nrevt(global, \"bvmr\") -> {regular, \"rmvb\", \"application/vnd.rn-realmedia-vbr\"};\nrevt(global, \"BVMR\") -> {regular, \"RMVB\", \"application/vnd.rn-realmedia-vbr\"};\nrevt(global, \"mr\") -> {regular, \"rm\", \"application/vnd.rn-realmedia\"};\nrevt(global, \"MR\") -> {regular, \"RM\", \"application/vnd.rn-realmedia\"};\nrevt(global, \"doc\") -> {regular, \"cod\", \"application/vnd.rim.cod\"};\nrevt(global, \"DOC\") -> {regular, \"COD\", \"application/vnd.rim.cod\"};\nrevt(global, \"etonotpyrc\") -> {regular, \"cryptonote\", \"application/vnd.rig.cryptonote\"};\nrevt(global, \"ETONOTPYRC\") -> {regular, \"CRYPTONOTE\", \"application/vnd.rig.cryptonote\"};\nrevt(global, \"lmxcisum\") -> {regular, \"musicxml\", \"application/vnd.recordare.musicxml+xml\"};\nrevt(global, \"LMXCISUM\") -> {regular, \"MUSICXML\", \"application/vnd.recordare.musicxml+xml\"};\nrevt(global, \"lxm\") -> {regular, \"mxl\", \"application/vnd.recordare.musicxml\"};\nrevt(global, \"LXM\") -> {regular, \"MXL\", \"application/vnd.recordare.musicxml\"};\nrevt(global, \"deb\") -> {regular, \"bed\", \"application/vnd.realvnc.bed\"};\nrevt(global, \"DEB\") -> {regular, \"BED\", \"application/vnd.realvnc.bed\"};\nrevt(global, \"bxq\") -> {regular, \"qxb\", \"application/vnd.quark.quarkxpress\"};\nrevt(global, \"BXQ\") -> {regular, \"QXB\", \"application/vnd.quark.quarkxpress\"};\nrevt(global, \"lxq\") -> {regular, \"qxl\", \"application/vnd.quark.quarkxpress\"};\nrevt(global, \"LXQ\") -> {regular, \"QXL\", \"application/vnd.quark.quarkxpress\"};\nrevt(global, \"twq\") -> {regular, \"qwt\", \"application/vnd.quark.quarkxpress\"};\nrevt(global, \"TWQ\") -> {regular, \"QWT\", \"application/vnd.quark.quarkxpress\"};\nrevt(global, \"dwq\") -> {regular, \"qwd\", \"application/vnd.quark.quarkxpress\"};\nrevt(global, \"DWQ\") -> {regular, \"QWD\", \"application/vnd.quark.quarkxpress\"};\nrevt(global, \"txq\") -> {regular, \"qxt\", \"application/vnd.quark.quarkxpress\"};\nrevt(global, \"TXQ\") -> {regular, \"QXT\", \"application/vnd.quark.quarkxpress\"};\nrevt(global, \"dxq\") -> {regular, \"qxd\", \"application/vnd.quark.quarkxpress\"};\nrevt(global, \"DXQ\") -> {regular, \"QXD\", \"application/vnd.quark.quarkxpress\"};\nrevt(global, \"ditp\") -> {regular, \"ptid\", \"application/vnd.pvi.ptid1\"};\nrevt(global, \"DITP\") -> {regular, \"PTID\", \"application/vnd.pvi.ptid1\"};\nrevt(global, \"spq\") -> {regular, \"qps\", \"application/vnd.publishare-delta-tree\"};\nrevt(global, \"SPQ\") -> {regular, \"QPS\", \"application/vnd.publishare-delta-tree\"};\nrevt(global, \"zgm\") -> {regular, \"mgz\", \"application/vnd.proteus.magazine\"};\nrevt(global, \"ZGM\") -> {regular, \"MGZ\", \"application/vnd.proteus.magazine\"};\nrevt(global, \"xob\") -> {regular, \"box\", \"application/vnd.previewsystems.box\"};\nrevt(global, \"XOB\") -> {regular, \"BOX\", \"application/vnd.previewsystems.box\"};\nrevt(global, \"dbp\") -> {regular, \"pbd\", \"application/vnd.powerbuilder6\"};\nrevt(global, \"DBP\") -> {regular, \"PBD\", \"application/vnd.powerbuilder6\"};\nrevt(global, \"flp\") -> {regular, \"plf\", \"application/vnd.pocketlearn\"};\nrevt(global, \"FLP\") -> {regular, \"PLF\", \"application/vnd.pocketlearn\"};\nrevt(global, \"gw\") -> {regular, \"wg\", \"application/vnd.pmi.widget\"};\nrevt(global, \"GW\") -> {regular, \"WG\", \"application/vnd.pmi.widget\"};\nrevt(global, \"fife\") -> {regular, \"efif\", \"application/vnd.picsel\"};\nrevt(global, \"FIFE\") -> {regular, \"EFIF\", \"application/vnd.picsel\"};\nrevt(global, \"6ie\") -> {regular, \"ei6\", \"application/vnd.pg.osasli\"};\nrevt(global, \"6IE\") -> {regular, \"EI6\", \"application/vnd.pg.osasli\"};\nrevt(global, \"rts\") -> {regular, \"str\", \"application/vnd.pg.format\"};\nrevt(global, \"RTS\") -> {regular, \"STR\", \"application/vnd.pg.format\"};\nrevt(global, \"wap\") -> {regular, \"paw\", \"application/vnd.pawaafile\"};\nrevt(global, \"WAP\") -> {regular, \"PAW\", \"application/vnd.pawaafile\"};\nrevt(global, \"crpo\") -> {regular, \"oprc\", \"application/vnd.palm\"};\nrevt(global, \"CRPO\") -> {regular, \"OPRC\", \"application/vnd.palm\"};\nrevt(global, \"aqp\") -> {regular, \"pqa\", \"application/vnd.palm\"};\nrevt(global, \"AQP\") -> {regular, \"PQA\", \"application/vnd.palm\"};\nrevt(global, \"bdp\") -> {regular, \"pdb\", \"application/vnd.palm\"};\nrevt(global, \"BDP\") -> {regular, \"PDB\", \"application/vnd.palm\"};\nrevt(global, \"ase\") -> {regular, \"esa\", \"application/vnd.osgi.subsystem\"};\nrevt(global, \"ASE\") -> {regular, \"ESA\", \"application/vnd.osgi.subsystem\"};\nrevt(global, \"pd\") -> {regular, \"dp\", \"application/vnd.osgi.dp\"};\nrevt(global, \"PD\") -> {regular, \"DP\", \"application/vnd.osgi.dp\"};\nrevt(global, \"pgm\") -> {regular, \"mgp\", \"application/vnd.osgeo.mapguide.package\"};\nrevt(global, \"PGM\") -> {regular, \"MGP\", \"application/vnd.osgeo.mapguide.package\"};\nrevt(global, \"xtod\") -> {regular, \"dotx\", \"application/vnd.openxmlformats-officedocument.wordprocessingml.template\"};\nrevt(global, \"XTOD\") -> {regular, \"DOTX\", \"application/vnd.openxmlformats-officedocument.wordprocessingml.template\"};\nrevt(global, \"xcod\") -> {regular, \"docx\", \"application/vnd.openxmlformats-officedocument.wordprocessingml.document\"};\nrevt(global, \"XCOD\") -> {regular, \"DOCX\", \"application/vnd.openxmlformats-officedocument.wordprocessingml.document\"};\nrevt(global, \"xtlx\") -> {regular, \"xltx\", \"application/vnd.openxmlformats-officedocument.spreadsheetml.template\"};\nrevt(global, \"XTLX\") -> {regular, \"XLTX\", \"application/vnd.openxmlformats-officedocument.spreadsheetml.template\"};\nrevt(global, \"xslx\") -> {regular, \"xlsx\", \"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\"};\nrevt(global, \"XSLX\") -> {regular, \"XLSX\", \"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\"};\nrevt(global, \"xtop\") -> {regular, \"potx\", \"application/vnd.openxmlformats-officedocument.presentationml.template\"};\nrevt(global, \"XTOP\") -> {regular, \"POTX\", \"application/vnd.openxmlformats-officedocument.presentationml.template\"};\nrevt(global, \"xspp\") -> {regular, \"ppsx\", \"application/vnd.openxmlformats-officedocument.presentationml.slideshow\"};\nrevt(global, \"XSPP\") -> {regular, \"PPSX\", \"application/vnd.openxmlformats-officedocument.presentationml.slideshow\"};\nrevt(global, \"xdls\") -> {regular, \"sldx\", \"application/vnd.openxmlformats-officedocument.presentationml.slide\"};\nrevt(global, \"XDLS\") -> {regular, \"SLDX\", \"application/vnd.openxmlformats-officedocument.presentationml.slide\"};\nrevt(global, \"xtpp\") -> {regular, \"pptx\", \"application/vnd.openxmlformats-officedocument.presentationml.presentation\"};\nrevt(global, \"XTPP\") -> {regular, \"PPTX\", \"application/vnd.openxmlformats-officedocument.presentationml.presentation\"};\nrevt(global, \"txo\") -> {regular, \"oxt\", \"application/vnd.openofficeorg.extension\"};\nrevt(global, \"TXO\") -> {regular, \"OXT\", \"application/vnd.openofficeorg.extension\"};\nrevt(global, \"2dd\") -> {regular, \"dd2\", \"application/vnd.oma.dd2+xml\"};\nrevt(global, \"2DD\") -> {regular, \"DD2\", \"application/vnd.oma.dd2+xml\"};\nrevt(global, \"ox\") -> {regular, \"xo\", \"application/vnd.olpc-sugar\"};\nrevt(global, \"OX\") -> {regular, \"XO\", \"application/vnd.olpc-sugar\"};\nrevt(global, \"hto\") -> {regular, \"oth\", \"application/vnd.oasis.opendocument.text-web\"};\nrevt(global, \"HTO\") -> {regular, \"OTH\", \"application/vnd.oasis.opendocument.text-web\"};\nrevt(global, \"tto\") -> {regular, \"ott\", \"application/vnd.oasis.opendocument.text-template\"};\nrevt(global, \"TTO\") -> {regular, \"OTT\", \"application/vnd.oasis.opendocument.text-template\"};\nrevt(global, \"mdo\") -> {regular, \"odm\", \"application/vnd.oasis.opendocument.text-master\"};\nrevt(global, \"MDO\") -> {regular, \"ODM\", \"application/vnd.oasis.opendocument.text-master\"};\nrevt(global, \"tdo\") -> {regular, \"odt\", \"application/vnd.oasis.opendocument.text\"};\nrevt(global, \"TDO\") -> {regular, \"ODT\", \"application/vnd.oasis.opendocument.text\"};\nrevt(global, \"sto\") -> {regular, \"ots\", \"application/vnd.oasis.opendocument.spreadsheet-template\"};\nrevt(global, \"STO\") -> {regular, \"OTS\", \"application/vnd.oasis.opendocument.spreadsheet-template\"};\nrevt(global, \"sdo\") -> {regular, \"ods\", \"application/vnd.oasis.opendocument.spreadsheet\"};\nrevt(global, \"SDO\") -> {regular, \"ODS\", \"application/vnd.oasis.opendocument.spreadsheet\"};\nrevt(global, \"pto\") -> {regular, \"otp\", \"application/vnd.oasis.opendocument.presentation-template\"};\nrevt(global, \"PTO\") -> {regular, \"OTP\", \"application/vnd.oasis.opendocument.presentation-template\"};\nrevt(global, \"pdo\") -> {regular, \"odp\", \"application/vnd.oasis.opendocument.presentation\"};\nrevt(global, \"PDO\") -> {regular, \"ODP\", \"application/vnd.oasis.opendocument.presentation\"};\nrevt(global, \"ito\") -> {regular, \"oti\", \"application/vnd.oasis.opendocument.image-template\"};\nrevt(global, \"ITO\") -> {regular, \"OTI\", \"application/vnd.oasis.opendocument.image-template\"};\nrevt(global, \"ido\") -> {regular, \"odi\", \"application/vnd.oasis.opendocument.image\"};\nrevt(global, \"IDO\") -> {regular, \"ODI\", \"application/vnd.oasis.opendocument.image\"};\nrevt(global, \"gto\") -> {regular, \"otg\", \"application/vnd.oasis.opendocument.graphics-template\"};\nrevt(global, \"GTO\") -> {regular, \"OTG\", \"application/vnd.oasis.opendocument.graphics-template\"};\nrevt(global, \"gdo\") -> {regular, \"odg\", \"application/vnd.oasis.opendocument.graphics\"};\nrevt(global, \"GDO\") -> {regular, \"ODG\", \"application/vnd.oasis.opendocument.graphics\"};\nrevt(global, \"tfdo\") -> {regular, \"odft\", \"application/vnd.oasis.opendocument.formula-template\"};\nrevt(global, \"TFDO\") -> {regular, \"ODFT\", \"application/vnd.oasis.opendocument.formula-template\"};\nrevt(global, \"fdo\") -> {regular, \"odf\", \"application/vnd.oasis.opendocument.formula\"};\nrevt(global, \"FDO\") -> {regular, \"ODF\", \"application/vnd.oasis.opendocument.formula\"};\nrevt(global, \"bdo\") -> {regular, \"odb\", \"application/vnd.oasis.opendocument.database\"};\nrevt(global, \"BDO\") -> {regular, \"ODB\", \"application/vnd.oasis.opendocument.database\"};\nrevt(global, \"cto\") -> {regular, \"otc\", \"application/vnd.oasis.opendocument.chart-template\"};\nrevt(global, \"CTO\") -> {regular, \"OTC\", \"application/vnd.oasis.opendocument.chart-template\"};\nrevt(global, \"cdo\") -> {regular, \"odc\", \"application/vnd.oasis.opendocument.chart\"};\nrevt(global, \"CDO\") -> {regular, \"ODC\", \"application/vnd.oasis.opendocument.chart\"};\nrevt(global, \"txe\") -> {regular, \"ext\", \"application/vnd.novadigm.ext\"};\nrevt(global, \"TXE\") -> {regular, \"EXT\", \"application/vnd.novadigm.ext\"};\nrevt(global, \"xde\") -> {regular, \"edx\", \"application/vnd.novadigm.edx\"};\nrevt(global, \"XDE\") -> {regular, \"EDX\", \"application/vnd.novadigm.edx\"};\nrevt(global, \"mde\") -> {regular, \"edm\", \"application/vnd.novadigm.edm\"};\nrevt(global, \"MDE\") -> {regular, \"EDM\", \"application/vnd.novadigm.edm\"};\nrevt(global, \"sspr\") -> {regular, \"rpss\", \"application/vnd.nokia.radio-presets\"};\nrevt(global, \"SSPR\") -> {regular, \"RPSS\", \"application/vnd.nokia.radio-presets\"};\nrevt(global, \"tspr\") -> {regular, \"rpst\", \"application/vnd.nokia.radio-preset\"};\nrevt(global, \"TSPR\") -> {regular, \"RPST\", \"application/vnd.nokia.radio-preset\"};\nrevt(global, \"egag-n\") -> {regular, \"n-gage\", \"application/vnd.nokia.n-gage.symbian.install\"};\nrevt(global, \"EGAG-N\") -> {regular, \"N-GAGE\", \"application/vnd.nokia.n-gage.symbian.install\"};\nrevt(global, \"tadgn\") -> {regular, \"ngdat\", \"application/vnd.nokia.n-gage.data\"};\nrevt(global, \"TADGN\") -> {regular, \"NGDAT\", \"application/vnd.nokia.n-gage.data\"};\nrevt(global, \"wnn\") -> {regular, \"nnw\", \"application/vnd.noblenet-web\"};\nrevt(global, \"WNN\") -> {regular, \"NNW\", \"application/vnd.noblenet-web\"};\nrevt(global, \"snn\") -> {regular, \"nns\", \"application/vnd.noblenet-sealer\"};\nrevt(global, \"SNN\") -> {regular, \"NNS\", \"application/vnd.noblenet-sealer\"};\nrevt(global, \"dnn\") -> {regular, \"nnd\", \"application/vnd.noblenet-directory\"};\nrevt(global, \"DNN\") -> {regular, \"NND\", \"application/vnd.noblenet-directory\"};\nrevt(global, \"ftin\") -> {regular, \"nitf\", \"application/vnd.nitf\"};\nrevt(global, \"FTIN\") -> {regular, \"NITF\", \"application/vnd.nitf\"};\nrevt(global, \"ftn\") -> {regular, \"ntf\", \"application/vnd.nitf\"};\nrevt(global, \"FTN\") -> {regular, \"NTF\", \"application/vnd.nitf\"};\nrevt(global, \"uln\") -> {regular, \"nlu\", \"application/vnd.neurolanguage.nlu\"};\nrevt(global, \"ULN\") -> {regular, \"NLU\", \"application/vnd.neurolanguage.nlu\"};\nrevt(global, \"telgat\") -> {regular, \"taglet\", \"application/vnd.mynfc\"};\nrevt(global, \"TELGAT\") -> {regular, \"TAGLET\", \"application/vnd.mynfc\"};\nrevt(global, \"ytsm\") -> {regular, \"msty\", \"application/vnd.muvee.style\"};\nrevt(global, \"YTSM\") -> {regular, \"MSTY\", \"application/vnd.muvee.style\"};\nrevt(global, \"sum\") -> {regular, \"mus\", \"application/vnd.musician\"};\nrevt(global, \"SUM\") -> {regular, \"MUS\", \"application/vnd.musician\"};\nrevt(global, \"qesm\") -> {regular, \"mseq\", \"application/vnd.mseq\"};\nrevt(global, \"QESM\") -> {regular, \"MSEQ\", \"application/vnd.mseq\"};\nrevt(global, \"spx\") -> {regular, \"xps\", \"application/vnd.ms-xpsdocument\"};\nrevt(global, \"SPX\") -> {regular, \"XPS\", \"application/vnd.ms-xpsdocument\"};\nrevt(global, \"lpw\") -> {regular, \"wpl\", \"application/vnd.ms-wpl\"};\nrevt(global, \"LPW\") -> {regular, \"WPL\", \"application/vnd.ms-wpl\"};\nrevt(global, \"bdw\") -> {regular, \"wdb\", \"application/vnd.ms-works\"};\nrevt(global, \"BDW\") -> {regular, \"WDB\", \"application/vnd.ms-works\"};\nrevt(global, \"mcw\") -> {regular, \"wcm\", \"application/vnd.ms-works\"};\nrevt(global, \"MCW\") -> {regular, \"WCM\", \"application/vnd.ms-works\"};\nrevt(global, \"skw\") -> {regular, \"wks\", \"application/vnd.ms-works\"};\nrevt(global, \"SKW\") -> {regular, \"WKS\", \"application/vnd.ms-works\"};\nrevt(global, \"spw\") -> {regular, \"wps\", \"application/vnd.ms-works\"};\nrevt(global, \"SPW\") -> {regular, \"WPS\", \"application/vnd.ms-works\"};\nrevt(global, \"mtod\") -> {regular, \"dotm\", \"application/vnd.ms-word.template.macroenabled.12\"};\nrevt(global, \"MTOD\") -> {regular, \"DOTM\", \"application/vnd.ms-word.template.macroenabled.12\"};\nrevt(global, \"mcod\") -> {regular, \"docm\", \"application/vnd.ms-word.document.macroenabled.12\"};\nrevt(global, \"MCOD\") -> {regular, \"DOCM\", \"application/vnd.ms-word.document.macroenabled.12\"};\nrevt(global, \"tpm\") -> {regular, \"mpt\", \"application/vnd.ms-project\"};\nrevt(global, \"TPM\") -> {regular, \"MPT\", \"application/vnd.ms-project\"};\nrevt(global, \"ppm\") -> {regular, \"mpp\", \"application/vnd.ms-project\"};\nrevt(global, \"PPM\") -> {regular, \"MPP\", \"application/vnd.ms-project\"};\nrevt(global, \"mtop\") -> {regular, \"potm\", \"application/vnd.ms-powerpoint.template.macroenabled.12\"};\nrevt(global, \"MTOP\") -> {regular, \"POTM\", \"application/vnd.ms-powerpoint.template.macroenabled.12\"};\nrevt(global, \"mspp\") -> {regular, \"ppsm\", \"application/vnd.ms-powerpoint.slideshow.macroenabled.12\"};\nrevt(global, \"MSPP\") -> {regular, \"PPSM\", \"application/vnd.ms-powerpoint.slideshow.macroenabled.12\"};\nrevt(global, \"mdls\") -> {regular, \"sldm\", \"application/vnd.ms-powerpoint.slide.macroenabled.12\"};\nrevt(global, \"MDLS\") -> {regular, \"SLDM\", \"application/vnd.ms-powerpoint.slide.macroenabled.12\"};\nrevt(global, \"mtpp\") -> {regular, \"pptm\", \"application/vnd.ms-powerpoint.presentation.macroenabled.12\"};\nrevt(global, \"MTPP\") -> {regular, \"PPTM\", \"application/vnd.ms-powerpoint.presentation.macroenabled.12\"};\nrevt(global, \"mapp\") -> {regular, \"ppam\", \"application/vnd.ms-powerpoint.addin.macroenabled.12\"};\nrevt(global, \"MAPP\") -> {regular, \"PPAM\", \"application/vnd.ms-powerpoint.addin.macroenabled.12\"};\nrevt(global, \"top\") -> {regular, \"pot\", \"application/vnd.ms-powerpoint\"};\nrevt(global, \"TOP\") -> {regular, \"POT\", \"application/vnd.ms-powerpoint\"};\nrevt(global, \"spp\") -> {regular, \"pps\", \"application/vnd.ms-powerpoint\"};\nrevt(global, \"SPP\") -> {regular, \"PPS\", \"application/vnd.ms-powerpoint\"};\nrevt(global, \"tpp\") -> {regular, \"ppt\", \"application/vnd.ms-powerpoint\"};\nrevt(global, \"TPP\") -> {regular, \"PPT\", \"application/vnd.ms-powerpoint\"};\nrevt(global, \"lts\") -> {regular, \"stl\", \"application/vnd.ms-pki.stl\"};\nrevt(global, \"LTS\") -> {regular, \"STL\", \"application/vnd.ms-pki.stl\"};\nrevt(global, \"tac\") -> {regular, \"cat\", \"application/vnd.ms-pki.seccat\"};\nrevt(global, \"TAC\") -> {regular, \"CAT\", \"application/vnd.ms-pki.seccat\"};\nrevt(global, \"xmht\") -> {regular, \"thmx\", \"application/vnd.ms-officetheme\"};\nrevt(global, \"XMHT\") -> {regular, \"THMX\", \"application/vnd.ms-officetheme\"};\nrevt(global, \"mrl\") -> {regular, \"lrm\", \"application/vnd.ms-lrm\"};\nrevt(global, \"MRL\") -> {regular, \"LRM\", \"application/vnd.ms-lrm\"};\nrevt(global, \"smi\") -> {regular, \"ims\", \"application/vnd.ms-ims\"};\nrevt(global, \"SMI\") -> {regular, \"IMS\", \"application/vnd.ms-ims\"};\nrevt(global, \"mhc\") -> {regular, \"chm\", \"application/vnd.ms-htmlhelp\"};\nrevt(global, \"MHC\") -> {regular, \"CHM\", \"application/vnd.ms-htmlhelp\"};\nrevt(global, \"toe\") -> {regular, \"eot\", \"application/vnd.ms-fontobject\"};\nrevt(global, \"TOE\") -> {regular, \"EOT\", \"application/vnd.ms-fontobject\"};\nrevt(global, \"mtlx\") -> {regular, \"xltm\", \"application/vnd.ms-excel.template.macroenabled.12\"};\nrevt(global, \"MTLX\") -> {regular, \"XLTM\", \"application/vnd.ms-excel.template.macroenabled.12\"};\nrevt(global, \"mslx\") -> {regular, \"xlsm\", \"application/vnd.ms-excel.sheet.macroenabled.12\"};\nrevt(global, \"MSLX\") -> {regular, \"XLSM\", \"application/vnd.ms-excel.sheet.macroenabled.12\"};\nrevt(global, \"bslx\") -> {regular, \"xlsb\", \"application/vnd.ms-excel.sheet.binary.macroenabled.12\"};\nrevt(global, \"BSLX\") -> {regular, \"XLSB\", \"application/vnd.ms-excel.sheet.binary.macroenabled.12\"};\nrevt(global, \"malx\") -> {regular, \"xlam\", \"application/vnd.ms-excel.addin.macroenabled.12\"};\nrevt(global, \"MALX\") -> {regular, \"XLAM\", \"application/vnd.ms-excel.addin.macroenabled.12\"};\nrevt(global, \"wlx\") -> {regular, \"xlw\", \"application/vnd.ms-excel\"};\nrevt(global, \"WLX\") -> {regular, \"XLW\", \"application/vnd.ms-excel\"};\nrevt(global, \"tlx\") -> {regular, \"xlt\", \"application/vnd.ms-excel\"};\nrevt(global, \"TLX\") -> {regular, \"XLT\", \"application/vnd.ms-excel\"};\nrevt(global, \"clx\") -> {regular, \"xlc\", \"application/vnd.ms-excel\"};\nrevt(global, \"CLX\") -> {regular, \"XLC\", \"application/vnd.ms-excel\"};\nrevt(global, \"alx\") -> {regular, \"xla\", \"application/vnd.ms-excel\"};\nrevt(global, \"ALX\") -> {regular, \"XLA\", \"application/vnd.ms-excel\"};\nrevt(global, \"mlx\") -> {regular, \"xlm\", \"application/vnd.ms-excel\"};\nrevt(global, \"MLX\") -> {regular, \"XLM\", \"application/vnd.ms-excel\"};\nrevt(global, \"slx\") -> {regular, \"xls\", \"application/vnd.ms-excel\"};\nrevt(global, \"SLX\") -> {regular, \"XLS\", \"application/vnd.ms-excel\"};\nrevt(global, \"bac\") -> {regular, \"cab\", \"application/vnd.ms-cab-compressed\"};\nrevt(global, \"BAC\") -> {regular, \"CAB\", \"application/vnd.ms-cab-compressed\"};\nrevt(global, \"lic\") -> {regular, \"cil\", \"application/vnd.ms-artgalry\"};\nrevt(global, \"LIC\") -> {regular, \"CIL\", \"application/vnd.ms-artgalry\"};\nrevt(global, \"lux\") -> {regular, \"xul\", \"application/vnd.mozilla.xul+xml\"};\nrevt(global, \"LUX\") -> {regular, \"XUL\", \"application/vnd.mozilla.xul+xml\"};\nrevt(global, \"cpm\") -> {regular, \"mpc\", \"application/vnd.mophun.certificate\"};\nrevt(global, \"CPM\") -> {regular, \"MPC\", \"application/vnd.mophun.certificate\"};\nrevt(global, \"npm\") -> {regular, \"mpn\", \"application/vnd.mophun.application\"};\nrevt(global, \"NPM\") -> {regular, \"MPN\", \"application/vnd.mophun.application\"};\nrevt(global, \"fxt\") -> {regular, \"txf\", \"application/vnd.mobius.txf\"};\nrevt(global, \"FXT\") -> {regular, \"TXF\", \"application/vnd.mobius.txf\"};\nrevt(global, \"clp\") -> {regular, \"plc\", \"application/vnd.mobius.plc\"};\nrevt(global, \"CLP\") -> {regular, \"PLC\", \"application/vnd.mobius.plc\"};\nrevt(global, \"lsm\") -> {regular, \"msl\", \"application/vnd.mobius.msl\"};\nrevt(global, \"LSM\") -> {regular, \"MSL\", \"application/vnd.mobius.msl\"};\nrevt(global, \"yqm\") -> {regular, \"mqy\", \"application/vnd.mobius.mqy\"};\nrevt(global, \"YQM\") -> {regular, \"MQY\", \"application/vnd.mobius.mqy\"};\nrevt(global, \"kbm\") -> {regular, \"mbk\", \"application/vnd.mobius.mbk\"};\nrevt(global, \"KBM\") -> {regular, \"MBK\", \"application/vnd.mobius.mbk\"};\nrevt(global, \"sid\") -> {regular, \"dis\", \"application/vnd.mobius.dis\"};\nrevt(global, \"SID\") -> {regular, \"DIS\", \"application/vnd.mobius.dis\"};\nrevt(global, \"fad\") -> {regular, \"daf\", \"application/vnd.mobius.daf\"};\nrevt(global, \"FAD\") -> {regular, \"DAF\", \"application/vnd.mobius.daf\"};\nrevt(global, \"fim\") -> {regular, \"mif\", \"application/vnd.mif\"};\nrevt(global, \"FIM\") -> {regular, \"MIF\", \"application/vnd.mif\"};\nrevt(global, \"xgi\") -> {regular, \"igx\", \"application/vnd.micrografx.igx\"};\nrevt(global, \"XGI\") -> {regular, \"IGX\", \"application/vnd.micrografx.igx\"};\nrevt(global, \"olf\") -> {regular, \"flo\", \"application/vnd.micrografx.flo\"};\nrevt(global, \"OLF\") -> {regular, \"FLO\", \"application/vnd.micrografx.flo\"};\nrevt(global, \"mfm\") -> {regular, \"mfm\", \"application/vnd.mfmp\"};\nrevt(global, \"MFM\") -> {regular, \"MFM\", \"application/vnd.mfmp\"};\nrevt(global, \"fwm\") -> {regular, \"mwf\", \"application/vnd.mfer\"};\nrevt(global, \"FWM\") -> {regular, \"MWF\", \"application/vnd.mfer\"};\nrevt(global, \"yekdc\") -> {regular, \"cdkey\", \"application/vnd.mediastation.cdkey\"};\nrevt(global, \"YEKDC\") -> {regular, \"CDKEY\", \"application/vnd.mediastation.cdkey\"};\nrevt(global, \"1cm\") -> {regular, \"mc1\", \"application/vnd.medcalcdata\"};\nrevt(global, \"1CM\") -> {regular, \"MC1\", \"application/vnd.medcalcdata\"};\nrevt(global, \"dcm\") -> {regular, \"mcd\", \"application/vnd.mcd\"};\nrevt(global, \"DCM\") -> {regular, \"MCD\", \"application/vnd.mcd\"};\nrevt(global, \"gkptrop\") -> {regular, \"portpkg\", \"application/vnd.macports.portpkg\"};\nrevt(global, \"GKPTROP\") -> {regular, \"PORTPKG\", \"application/vnd.macports.portpkg\"};\nrevt(global, \"pwl\") -> {regular, \"lwp\", \"application/vnd.lotus-wordpro\"};\nrevt(global, \"PWL\") -> {regular, \"LWP\", \"application/vnd.lotus-wordpro\"};\nrevt(global, \"mcs\") -> {regular, \"scm\", \"application/vnd.lotus-screencam\"};\nrevt(global, \"MCS\") -> {regular, \"SCM\", \"application/vnd.lotus-screencam\"};\nrevt(global, \"gro\") -> {regular, \"org\", \"application/vnd.lotus-organizer\"};\nrevt(global, \"GRO\") -> {regular, \"ORG\", \"application/vnd.lotus-organizer\"};\nrevt(global, \"fsn\") -> {regular, \"nsf\", \"application/vnd.lotus-notes\"};\nrevt(global, \"FSN\") -> {regular, \"NSF\", \"application/vnd.lotus-notes\"};\nrevt(global, \"erp\") -> {regular, \"pre\", \"application/vnd.lotus-freelance\"};\nrevt(global, \"ERP\") -> {regular, \"PRE\", \"application/vnd.lotus-freelance\"};\nrevt(global, \"rpa\") -> {regular, \"apr\", \"application/vnd.lotus-approach\"};\nrevt(global, \"RPA\") -> {regular, \"APR\", \"application/vnd.lotus-approach\"};\nrevt(global, \"321\") -> {regular, \"123\", \"application/vnd.lotus-1-2-3\"};\nrevt(global, \"ebl\") -> {regular, \"lbe\", \"application/vnd.llamagraphics.life-balance.exchange+xml\"};\nrevt(global, \"EBL\") -> {regular, \"LBE\", \"application/vnd.llamagraphics.life-balance.exchange+xml\"};\nrevt(global, \"dbl\") -> {regular, \"lbd\", \"application/vnd.llamagraphics.life-balance.desktop\"};\nrevt(global, \"DBL\") -> {regular, \"LBD\", \"application/vnd.llamagraphics.life-balance.desktop\"};\nrevt(global, \"lmxsal\") -> {regular, \"lasxml\", \"application/vnd.las.las+xml\"};\nrevt(global, \"LMXSAL\") -> {regular, \"LASXML\", \"application/vnd.las.las+xml\"};\nrevt(global, \"ess\") -> {regular, \"sse\", \"application/vnd.kodak-descriptor\"};\nrevt(global, \"ESS\") -> {regular, \"SSE\", \"application/vnd.kodak-descriptor\"};\nrevt(global, \"mks\") -> {regular, \"skm\", \"application/x-koan\"};\nrevt(global, \"MKS\") -> {regular, \"SKM\", \"application/x-koan\"};\nrevt(global, \"tks\") -> {regular, \"skt\", \"application/x-koan\"};\nrevt(global, \"TKS\") -> {regular, \"SKT\", \"application/x-koan\"};\nrevt(global, \"dks\") -> {regular, \"skd\", \"application/x-koan\"};\nrevt(global, \"DKS\") -> {regular, \"SKD\", \"application/x-koan\"};\nrevt(global, \"pks\") -> {regular, \"skp\", \"application/x-koan\"};\nrevt(global, \"PKS\") -> {regular, \"SKP\", \"application/x-koan\"};\nrevt(global, \"pnk\") -> {regular, \"knp\", \"application/vnd.kinar\"};\nrevt(global, \"PNK\") -> {regular, \"KNP\", \"application/vnd.kinar\"};\nrevt(global, \"enk\") -> {regular, \"kne\", \"application/vnd.kinar\"};\nrevt(global, \"ENK\") -> {regular, \"KNE\", \"application/vnd.kinar\"};\nrevt(global, \"aik\") -> {regular, \"kia\", \"application/vnd.kidspiration\"};\nrevt(global, \"AIK\") -> {regular, \"KIA\", \"application/vnd.kidspiration\"};\nrevt(global, \"ekth\") -> {regular, \"htke\", \"application/vnd.kenameaapp\"};\nrevt(global, \"EKTH\") -> {regular, \"HTKE\", \"application/vnd.kenameaapp\"};\nrevt(global, \"twk\") -> {regular, \"kwt\", \"application/x-kword\"};\nrevt(global, \"TWK\") -> {regular, \"KWT\", \"application/x-kword\"};\nrevt(global, \"dwk\") -> {regular, \"kwd\", \"application/x-kword\"};\nrevt(global, \"DWK\") -> {regular, \"KWD\", \"application/x-kword\"};\nrevt(global, \"psk\") -> {regular, \"ksp\", \"application/x-kspread\"};\nrevt(global, \"PSK\") -> {regular, \"KSP\", \"application/x-kspread\"};\nrevt(global, \"tpk\") -> {regular, \"kpt\", \"application/x-kpresenter\"};\nrevt(global, \"TPK\") -> {regular, \"KPT\", \"application/x-kpresenter\"};\nrevt(global, \"rpk\") -> {regular, \"kpr\", \"application/x-kpresenter\"};\nrevt(global, \"RPK\") -> {regular, \"KPR\", \"application/x-kpresenter\"};\nrevt(global, \"nok\") -> {regular, \"kon\", \"application/vnd.kde.kontour\"};\nrevt(global, \"NOK\") -> {regular, \"KON\", \"application/vnd.kde.kontour\"};\nrevt(global, \"wlf\") -> {regular, \"flw\", \"application/vnd.kde.kivio\"};\nrevt(global, \"WLF\") -> {regular, \"FLW\", \"application/vnd.kde.kivio\"};\nrevt(global, \"ofk\") -> {regular, \"kfo\", \"application/vnd.kde.kformula\"};\nrevt(global, \"OFK\") -> {regular, \"KFO\", \"application/vnd.kde.kformula\"};\nrevt(global, \"trhc\") -> {regular, \"chrt\", \"application/x-kchart\"};\nrevt(global, \"TRHC\") -> {regular, \"CHRT\", \"application/x-kchart\"};\nrevt(global, \"nobrak\") -> {regular, \"karbon\", \"application/vnd.kde.karbon\"};\nrevt(global, \"NOBRAK\") -> {regular, \"KARBON\", \"application/vnd.kde.karbon\"};\nrevt(global, \"rtk\") -> {regular, \"ktr\", \"application/vnd.kahootz\"};\nrevt(global, \"RTK\") -> {regular, \"KTR\", \"application/vnd.kahootz\"};\nrevt(global, \"ztk\") -> {regular, \"ktz\", \"application/vnd.kahootz\"};\nrevt(global, \"ZTK\") -> {regular, \"KTZ\", \"application/vnd.kahootz\"};\nrevt(global, \"adoj\") -> {regular, \"joda\", \"application/vnd.joost.joda-archive\"};\nrevt(global, \"ADOJ\") -> {regular, \"JODA\", \"application/vnd.joost.joda-archive\"};\nrevt(global, \"psij\") -> {regular, \"jisp\", \"application/vnd.jisp\"};\nrevt(global, \"PSIJ\") -> {regular, \"JISP\", \"application/vnd.jisp\"};\nrevt(global, \"smr\") -> {regular, \"rms\", \"application/vnd.jcp.javame.midlet-rms\"};\nrevt(global, \"SMR\") -> {regular, \"RMS\", \"application/vnd.jcp.javame.midlet-rms\"};\nrevt(global, \"maj\") -> {regular, \"jam\", \"application/vnd.jam\"};\nrevt(global, \"MAJ\") -> {regular, \"JAM\", \"application/vnd.jam\"};\nrevt(global, \"scf\") -> {regular, \"fcs\", \"application/vnd.isac.fcs\"};\nrevt(global, \"SCF\") -> {regular, \"FCS\", \"application/vnd.isac.fcs\"};\nrevt(global, \"rpx\") -> {regular, \"xpr\", \"application/vnd.is-xpr\"};\nrevt(global, \"RPX\") -> {regular, \"XPR\", \"application/vnd.is-xpr\"};\nrevt(global, \"pri\") -> {regular, \"irp\", \"application/vnd.irepository.package+xml\"};\nrevt(global, \"PRI\") -> {regular, \"IRP\", \"application/vnd.irepository.package+xml\"};\nrevt(global, \"eliforpcr\") -> {regular, \"rcprofile\", \"application/vnd.ipunplugged.rcprofile\"};\nrevt(global, \"ELIFORPCR\") -> {regular, \"RCPROFILE\", \"application/vnd.ipunplugged.rcprofile\"};\nrevt(global, \"xfq\") -> {regular, \"qfx\", \"application/vnd.intu.qfx\"};\nrevt(global, \"XFQ\") -> {regular, \"QFX\", \"application/vnd.intu.qfx\"};\nrevt(global, \"obq\") -> {regular, \"qbo\", \"application/vnd.intu.qbo\"};\nrevt(global, \"OBQ\") -> {regular, \"QBO\", \"application/vnd.intu.qbo\"};\nrevt(global, \"g2i\") -> {regular, \"i2g\", \"application/vnd.intergeo\"};\nrevt(global, \"G2I\") -> {regular, \"I2G\", \"application/vnd.intergeo\"};\nrevt(global, \"xpx\") -> {regular, \"xpx\", \"application/vnd.intercon.formnet\"};\nrevt(global, \"XPX\") -> {regular, \"XPX\", \"application/vnd.intercon.formnet\"};\nrevt(global, \"wpx\") -> {regular, \"xpw\", \"application/vnd.intercon.formnet\"};\nrevt(global, \"WPX\") -> {regular, \"XPW\", \"application/vnd.intercon.formnet\"};\nrevt(global, \"mgi\") -> {regular, \"igm\", \"application/vnd.insors.igm\"};\nrevt(global, \"MGI\") -> {regular, \"IGM\", \"application/vnd.insors.igm\"};\nrevt(global, \"uvi\") -> {regular, \"ivu\", \"application/vnd.immervision-ivu\"};\nrevt(global, \"UVI\") -> {regular, \"IVU\", \"application/vnd.immervision-ivu\"};\nrevt(global, \"pvi\") -> {regular, \"ivp\", \"application/vnd.immervision-ivp\"};\nrevt(global, \"PVI\") -> {regular, \"IVP\", \"application/vnd.immervision-ivp\"};\nrevt(global, \"lgi\") -> {regular, \"igl\", \"application/vnd.igloader\"};\nrevt(global, \"LGI\") -> {regular, \"IGL\", \"application/vnd.igloader\"};\nrevt(global, \"mci\") -> {regular, \"icm\", \"application/vnd.iccprofile\"};\nrevt(global, \"MCI\") -> {regular, \"ICM\", \"application/vnd.iccprofile\"};\nrevt(global, \"cci\") -> {regular, \"icc\", \"application/vnd.iccprofile\"};\nrevt(global, \"CCI\") -> {regular, \"ICC\", \"application/vnd.iccprofile\"};\nrevt(global, \"cs\") -> {regular, \"sc\", \"application/vnd.ibm.secure-container\"};\nrevt(global, \"CS\") -> {regular, \"SC\", \"application/vnd.ibm.secure-container\"};\nrevt(global, \"mri\") -> {regular, \"irm\", \"application/vnd.ibm.rights-management\"};\nrevt(global, \"MRI\") -> {regular, \"IRM\", \"application/vnd.ibm.rights-management\"};\nrevt(global, \"0283tsil\") -> {regular, \"list3820\", \"application/vnd.ibm.modcap\"};\nrevt(global, \"0283TSIL\") -> {regular, \"LIST3820\", \"application/vnd.ibm.modcap\"};\nrevt(global, \"pfatsil\") -> {regular, \"listafp\", \"application/vnd.ibm.modcap\"};\nrevt(global, \"PFATSIL\") -> {regular, \"LISTAFP\", \"application/vnd.ibm.modcap\"};\nrevt(global, \"pfa\") -> {regular, \"afp\", \"application/vnd.ibm.modcap\"};\nrevt(global, \"PFA\") -> {regular, \"AFP\", \"application/vnd.ibm.modcap\"};\nrevt(global, \"ypm\") -> {regular, \"mpy\", \"application/vnd.ibm.minipay\"};\nrevt(global, \"YPM\") -> {regular, \"MPY\", \"application/vnd.ibm.minipay\"};\nrevt(global, \"d3x\") -> {regular, \"x3d\", \"model/x3d+xml\"};\nrevt(global, \"D3X\") -> {regular, \"X3D\", \"model/x3d+xml\"};\nrevt(global, \"xtsdh-dfs\") -> {regular, \"sfd-hdstx\", \"application/vnd.hydrostatix.sof-data\"};\nrevt(global, \"XTSDH-DFS\") -> {regular, \"SFD-HDSTX\", \"application/vnd.hydrostatix.sof-data\"};\nrevt(global, \"lxlcp\") -> {regular, \"pclxl\", \"application/vnd.hp-pclxl\"};\nrevt(global, \"LXLCP\") -> {regular, \"PCLXL\", \"application/vnd.hp-pclxl\"};\nrevt(global, \"lcp\") -> {regular, \"pcl\", \"application/vnd.hp-pcl\"};\nrevt(global, \"LCP\") -> {regular, \"PCL\", \"application/vnd.hp-pcl\"};\nrevt(global, \"tlj\") -> {regular, \"jlt\", \"application/vnd.hp-jlyt\"};\nrevt(global, \"TLJ\") -> {regular, \"JLT\", \"application/vnd.hp-jlyt\"};\nrevt(global, \"sph\") -> {regular, \"hps\", \"application/vnd.hp-hps\"};\nrevt(global, \"SPH\") -> {regular, \"HPS\", \"application/vnd.hp-hps\"};\nrevt(global, \"diph\") -> {regular, \"hpid\", \"application/vnd.hp-hpid\"};\nrevt(global, \"DIPH\") -> {regular, \"HPID\", \"application/vnd.hp-hpid\"};\nrevt(global, \"lgph\") -> {regular, \"hpgl\", \"application/vnd.hp-hpgl\"};\nrevt(global, \"LGPH\") -> {regular, \"HPGL\", \"application/vnd.hp-hpgl\"};\nrevt(global, \"sel\") -> {regular, \"les\", \"application/vnd.hhe.lesson-player\"};\nrevt(global, \"SEL\") -> {regular, \"LES\", \"application/vnd.hhe.lesson-player\"};\nrevt(global, \"icbh\") -> {regular, \"hbci\", \"application/vnd.hbci\"};\nrevt(global, \"ICBH\") -> {regular, \"HBCI\", \"application/vnd.hbci\"};\nrevt(global, \"mmz\") -> {regular, \"zmm\", \"application/vnd.handheld-entertainment+xml\"};\nrevt(global, \"MMZ\") -> {regular, \"ZMM\", \"application/vnd.handheld-entertainment+xml\"};\nrevt(global, \"lah\") -> {regular, \"hal\", \"application/vnd.hal+xml\"};\nrevt(global, \"LAH\") -> {regular, \"HAL\", \"application/vnd.hal+xml\"};\nrevt(global, \"gcv\") -> {regular, \"vcg\", \"application/vnd.groove-vcard\"};\nrevt(global, \"GCV\") -> {regular, \"VCG\", \"application/vnd.groove-vcard\"};\nrevt(global, \"lpt\") -> {regular, \"tpl\", \"application/vnd.groove-tool-template\"};\nrevt(global, \"LPT\") -> {regular, \"TPL\", \"application/vnd.groove-tool-template\"};\nrevt(global, \"mtg\") -> {regular, \"gtm\", \"application/vnd.groove-tool-message\"};\nrevt(global, \"MTG\") -> {regular, \"GTM\", \"application/vnd.groove-tool-message\"};\nrevt(global, \"vrg\") -> {regular, \"grv\", \"application/vnd.groove-injector\"};\nrevt(global, \"VRG\") -> {regular, \"GRV\", \"application/vnd.groove-injector\"};\nrevt(global, \"mig\") -> {regular, \"gim\", \"application/vnd.groove-identity-message\"};\nrevt(global, \"MIG\") -> {regular, \"GIM\", \"application/vnd.groove-identity-message\"};\nrevt(global, \"fhg\") -> {regular, \"ghf\", \"application/vnd.groove-help\"};\nrevt(global, \"FHG\") -> {regular, \"GHF\", \"application/vnd.groove-help\"};\nrevt(global, \"cag\") -> {regular, \"gac\", \"application/vnd.groove-account\"};\nrevt(global, \"CAG\") -> {regular, \"GAC\", \"application/vnd.groove-account\"};\nrevt(global, \"sqg\") -> {regular, \"gqs\", \"application/vnd.grafeq\"};\nrevt(global, \"SQG\") -> {regular, \"GQS\", \"application/vnd.grafeq\"};\nrevt(global, \"fqg\") -> {regular, \"gqf\", \"application/vnd.grafeq\"};\nrevt(global, \"FQG\") -> {regular, \"GQF\", \"application/vnd.grafeq\"};\nrevt(global, \"zmk\") -> {regular, \"kmz\", \"application/vnd.google-earth.kmz\"};\nrevt(global, \"ZMK\") -> {regular, \"KMZ\", \"application/vnd.google-earth.kmz\"};\nrevt(global, \"lmk\") -> {regular, \"kml\", \"application/vnd.google-earth.kml+xml\"};\nrevt(global, \"LMK\") -> {regular, \"KML\", \"application/vnd.google-earth.kml+xml\"};\nrevt(global, \"xmg\") -> {regular, \"gmx\", \"application/vnd.gmx\"};\nrevt(global, \"XMG\") -> {regular, \"GMX\", \"application/vnd.gmx\"};\nrevt(global, \"w3g\") -> {regular, \"g3w\", \"application/vnd.geospace\"};\nrevt(global, \"W3G\") -> {regular, \"G3W\", \"application/vnd.geospace\"};\nrevt(global, \"w2g\") -> {regular, \"g2w\", \"application/vnd.geoplan\"};\nrevt(global, \"W2G\") -> {regular, \"G2W\", \"application/vnd.geoplan\"};\nrevt(global, \"txg\") -> {regular, \"gxt\", \"application/vnd.geonext\"};\nrevt(global, \"TXG\") -> {regular, \"GXT\", \"application/vnd.geonext\"};\nrevt(global, \"erg\") -> {regular, \"gre\", \"application/vnd.geometry-explorer\"};\nrevt(global, \"ERG\") -> {regular, \"GRE\", \"application/vnd.geometry-explorer\"};\nrevt(global, \"xeg\") -> {regular, \"gex\", \"application/vnd.geometry-explorer\"};\nrevt(global, \"XEG\") -> {regular, \"GEX\", \"application/vnd.geometry-explorer\"};\nrevt(global, \"tgg\") -> {regular, \"ggt\", \"application/vnd.geogebra.tool\"};\nrevt(global, \"TGG\") -> {regular, \"GGT\", \"application/vnd.geogebra.tool\"};\nrevt(global, \"bgg\") -> {regular, \"ggb\", \"application/vnd.geogebra.file\"};\nrevt(global, \"BGG\") -> {regular, \"GGB\", \"application/vnd.geogebra.file\"};\nrevt(global, \"dxt\") -> {regular, \"txd\", \"application/vnd.genomatix.tuxedo\"};\nrevt(global, \"DXT\") -> {regular, \"TXD\", \"application/vnd.genomatix.tuxedo\"};\nrevt(global, \"szf\") -> {regular, \"fzs\", \"application/vnd.fuzzysheet\"};\nrevt(global, \"SZF\") -> {regular, \"FZS\", \"application/vnd.fuzzysheet\"};\nrevt(global, \"dbx\") -> {regular, \"xbd\", \"application/vnd.fujixerox.docuworks.binder\"};\nrevt(global, \"DBX\") -> {regular, \"XBD\", \"application/vnd.fujixerox.docuworks.binder\"};\nrevt(global, \"wdx\") -> {regular, \"xdw\", \"application/vnd.fujixerox.docuworks\"};\nrevt(global, \"WDX\") -> {regular, \"XDW\", \"application/vnd.fujixerox.docuworks\"};\nrevt(global, \"ddd\") -> {regular, \"ddd\", \"application/vnd.fujixerox.ddd\"};\nrevt(global, \"DDD\") -> {regular, \"DDD\", \"application/vnd.fujixerox.ddd\"};\nrevt(global, \"2hb\") -> {regular, \"bh2\", \"application/vnd.fujitsu.oasysprs\"};\nrevt(global, \"2HB\") -> {regular, \"BH2\", \"application/vnd.fujitsu.oasysprs\"};\nrevt(global, \"5gf\") -> {regular, \"fg5\", \"application/vnd.fujitsu.oasysgp\"};\nrevt(global, \"5GF\") -> {regular, \"FG5\", \"application/vnd.fujitsu.oasysgp\"};\nrevt(global, \"3ao\") -> {regular, \"oa3\", \"application/vnd.fujitsu.oasys3\"};\nrevt(global, \"3AO\") -> {regular, \"OA3\", \"application/vnd.fujitsu.oasys3\"};\nrevt(global, \"2ao\") -> {regular, \"oa2\", \"application/vnd.fujitsu.oasys2\"};\nrevt(global, \"2AO\") -> {regular, \"OA2\", \"application/vnd.fujitsu.oasys2\"};\nrevt(global, \"sao\") -> {regular, \"oas\", \"application/vnd.fujitsu.oasys\"};\nrevt(global, \"SAO\") -> {regular, \"OAS\", \"application/vnd.fujitsu.oasys\"};\nrevt(global, \"csf\") -> {regular, \"fsc\", \"application/vnd.fsc.weblaunch\"};\nrevt(global, \"CSF\") -> {regular, \"FSC\", \"application/vnd.fsc.weblaunch\"};\nrevt(global, \"ftl\") -> {regular, \"ltf\", \"application/vnd.frogans.ltf\"};\nrevt(global, \"FTL\") -> {regular, \"LTF\", \"application/vnd.frogans.ltf\"};\nrevt(global, \"cnf\") -> {regular, \"fnc\", \"application/vnd.frogans.fnc\"};\nrevt(global, \"CNF\") -> {regular, \"FNC\", \"application/vnd.frogans.fnc\"};\nrevt(global, \"koob\") -> {regular, \"book\", \"application/vnd.framemaker\"};\nrevt(global, \"KOOB\") -> {regular, \"BOOK\", \"application/vnd.framemaker\"};\nrevt(global, \"rekam\") -> {regular, \"maker\", \"application/vnd.framemaker\"};\nrevt(global, \"REKAM\") -> {regular, \"MAKER\", \"application/vnd.framemaker\"};\nrevt(global, \"emarf\") -> {regular, \"frame\", \"application/vnd.framemaker\"};\nrevt(global, \"EMARF\") -> {regular, \"FRAME\", \"application/vnd.framemaker\"};\nrevt(global, \"mf\") -> {regular, \"fm\", \"application/vnd.framemaker\"};\nrevt(global, \"MF\") -> {regular, \"FM\", \"application/vnd.framemaker\"};\nrevt(global, \"ctf\") -> {regular, \"ftc\", \"application/vnd.fluxtime.clip\"};\nrevt(global, \"CTF\") -> {regular, \"FTC\", \"application/vnd.fluxtime.clip\"};\nrevt(global, \"hpg\") -> {regular, \"gph\", \"application/vnd.flographit\"};\nrevt(global, \"HPG\") -> {regular, \"GPH\", \"application/vnd.flographit\"};\nrevt(global, \"sselatad\") -> {regular, \"dataless\", \"application/vnd.fdsn.seed\"};\nrevt(global, \"SSELATAD\") -> {regular, \"DATALESS\", \"application/vnd.fdsn.seed\"};\nrevt(global, \"dees\") -> {regular, \"seed\", \"application/vnd.fdsn.seed\"};\nrevt(global, \"DEES\") -> {regular, \"SEED\", \"application/vnd.fdsn.seed\"};\nrevt(global, \"deesm\") -> {regular, \"mseed\", \"application/vnd.fdsn.mseed\"};\nrevt(global, \"DEESM\") -> {regular, \"MSEED\", \"application/vnd.fdsn.mseed\"};\nrevt(global, \"fdf\") -> {regular, \"fdf\", \"application/vnd.fdf\"};\nrevt(global, \"FDF\") -> {regular, \"FDF\", \"application/vnd.fdf\"};\nrevt(global, \"3ze\") -> {regular, \"ez3\", \"application/vnd.ezpix-package\"};\nrevt(global, \"3ZE\") -> {regular, \"EZ3\", \"application/vnd.ezpix-package\"};\nrevt(global, \"2ze\") -> {regular, \"ez2\", \"application/vnd.ezpix-album\"};\nrevt(global, \"2ZE\") -> {regular, \"EZ2\", \"application/vnd.ezpix-album\"};\nrevt(global, \"3te\") -> {regular, \"et3\", \"application/vnd.eszigno3+xml\"};\nrevt(global, \"3TE\") -> {regular, \"ET3\", \"application/vnd.eszigno3+xml\"};\nrevt(global, \"3se\") -> {regular, \"es3\", \"application/vnd.eszigno3+xml\"};\nrevt(global, \"3SE\") -> {regular, \"ES3\", \"application/vnd.eszigno3+xml\"};\nrevt(global, \"fss\") -> {regular, \"ssf\", \"application/vnd.epson.ssf\"};\nrevt(global, \"FSS\") -> {regular, \"SSF\", \"application/vnd.epson.ssf\"};\nrevt(global, \"tls\") -> {regular, \"slt\", \"application/vnd.epson.salt\"};\nrevt(global, \"TLS\") -> {regular, \"SLT\", \"application/vnd.epson.salt\"};\nrevt(global, \"maq\") -> {regular, \"qam\", \"application/vnd.epson.quickanime\"};\nrevt(global, \"MAQ\") -> {regular, \"QAM\", \"application/vnd.epson.quickanime\"};\nrevt(global, \"fsm\") -> {regular, \"msf\", \"application/vnd.epson.msf\"};\nrevt(global, \"FSM\") -> {regular, \"MSF\", \"application/vnd.epson.msf\"};\nrevt(global, \"fse\") -> {regular, \"esf\", \"application/vnd.epson.esf\"};\nrevt(global, \"FSE\") -> {regular, \"ESF\", \"application/vnd.epson.esf\"};\nrevt(global, \"lmn\") -> {regular, \"nml\", \"application/vnd.enliven\"};\nrevt(global, \"LMN\") -> {regular, \"NML\", \"application/vnd.enliven\"};\nrevt(global, \"gam\") -> {regular, \"mag\", \"application/vnd.ecowin.chart\"};\nrevt(global, \"GAM\") -> {regular, \"MAG\", \"application/vnd.ecowin.chart\"};\nrevt(global, \"oeg\") -> {regular, \"geo\", \"application/vnd.dynageo\"};\nrevt(global, \"OEG\") -> {regular, \"GEO\", \"application/vnd.dynageo\"};\nrevt(global, \"cvs\") -> {regular, \"svc\", \"application/vnd.dvb.service\"};\nrevt(global, \"CVS\") -> {regular, \"SVC\", \"application/vnd.dvb.service\"};\nrevt(global, \"tia\") -> {regular, \"ait\", \"application/vnd.dvb.ait\"};\nrevt(global, \"TIA\") -> {regular, \"AIT\", \"application/vnd.dvb.ait\"};\nrevt(global, \"xxpk\") -> {regular, \"kpxx\", \"application/vnd.ds-keypoint\"};\nrevt(global, \"XXPK\") -> {regular, \"KPXX\", \"application/vnd.ds-keypoint\"};\nrevt(global, \"cafd\") -> {regular, \"dfac\", \"application/vnd.dreamfactory\"};\nrevt(global, \"CAFD\") -> {regular, \"DFAC\", \"application/vnd.dreamfactory\"};\nrevt(global, \"gpd\") -> {regular, \"dpg\", \"application/vnd.dpgraph\"};\nrevt(global, \"GPD\") -> {regular, \"DPG\", \"application/vnd.dpgraph\"};\nrevt(global, \"plm\") -> {regular, \"mlp\", \"application/vnd.dolby.mlp\"};\nrevt(global, \"PLM\") -> {regular, \"MLP\", \"application/vnd.dolby.mlp\"};\nrevt(global, \"and\") -> {regular, \"dna\", \"application/vnd.dna\"};\nrevt(global, \"AND\") -> {regular, \"DNA\", \"application/vnd.dna\"};\nrevt(global, \"hcnual_ef\") -> {regular, \"fe_launch\", \"application/vnd.denovo.fcselayout-link\"};\nrevt(global, \"HCNUAL_EF\") -> {regular, \"FE_LAUNCH\", \"application/vnd.denovo.fcselayout-link\"};\nrevt(global, \"zvvu\") -> {regular, \"uvvz\", \"application/vnd.dece.zip\"};\nrevt(global, \"ZVVU\") -> {regular, \"UVVZ\", \"application/vnd.dece.zip\"};\nrevt(global, \"zvu\") -> {regular, \"uvz\", \"application/vnd.dece.zip\"};\nrevt(global, \"ZVU\") -> {regular, \"UVZ\", \"application/vnd.dece.zip\"};\nrevt(global, \"xvvu\") -> {regular, \"uvvx\", \"application/vnd.dece.unspecified\"};\nrevt(global, \"XVVU\") -> {regular, \"UVVX\", \"application/vnd.dece.unspecified\"};\nrevt(global, \"xvu\") -> {regular, \"uvx\", \"application/vnd.dece.unspecified\"};\nrevt(global, \"XVU\") -> {regular, \"UVX\", \"application/vnd.dece.unspecified\"};\nrevt(global, \"tvvu\") -> {regular, \"uvvt\", \"application/vnd.dece.ttml+xml\"};\nrevt(global, \"TVVU\") -> {regular, \"UVVT\", \"application/vnd.dece.ttml+xml\"};\nrevt(global, \"tvu\") -> {regular, \"uvt\", \"application/vnd.dece.ttml+xml\"};\nrevt(global, \"TVU\") -> {regular, \"UVT\", \"application/vnd.dece.ttml+xml\"};\nrevt(global, \"dvvu\") -> {regular, \"uvvd\", \"application/vnd.dece.data\"};\nrevt(global, \"DVVU\") -> {regular, \"UVVD\", \"application/vnd.dece.data\"};\nrevt(global, \"dvu\") -> {regular, \"uvd\", \"application/vnd.dece.data\"};\nrevt(global, \"DVU\") -> {regular, \"UVD\", \"application/vnd.dece.data\"};\nrevt(global, \"fvvu\") -> {regular, \"uvvf\", \"application/vnd.dece.data\"};\nrevt(global, \"FVVU\") -> {regular, \"UVVF\", \"application/vnd.dece.data\"};\nrevt(global, \"fvu\") -> {regular, \"uvf\", \"application/vnd.dece.data\"};\nrevt(global, \"FVU\") -> {regular, \"UVF\", \"application/vnd.dece.data\"};\nrevt(global, \"zdr\") -> {regular, \"rdz\", \"application/vnd.data-vision.rdz\"};\nrevt(global, \"ZDR\") -> {regular, \"RDZ\", \"application/vnd.data-vision.rdz\"};\nrevt(global, \"trad\") -> {regular, \"dart\", \"application/vnd.dart\"};\nrevt(global, \"TRAD\") -> {regular, \"DART\", \"application/vnd.dart\"};\nrevt(global, \"lrucp\") -> {regular, \"pcurl\", \"application/vnd.curl.pcurl\"};\nrevt(global, \"LRUCP\") -> {regular, \"PCURL\", \"application/vnd.curl.pcurl\"};\nrevt(global, \"rac\") -> {regular, \"car\", \"application/vnd.curl.car\"};\nrevt(global, \"RAC\") -> {regular, \"CAR\", \"application/vnd.curl.car\"};\nrevt(global, \"dpp\") -> {regular, \"ppd\", \"application/vnd.cups-ppd\"};\nrevt(global, \"DPP\") -> {regular, \"PPD\", \"application/vnd.cups-ppd\"};\nrevt(global, \"lmp\") -> {regular, \"pml\", \"application/vnd.ctc-posml\"};\nrevt(global, \"LMP\") -> {regular, \"PML\", \"application/vnd.ctc-posml\"};\nrevt(global, \"sbw\") -> {regular, \"wbs\", \"application/vnd.criticaltools.wbs+xml\"};\nrevt(global, \"SBW\") -> {regular, \"WBS\", \"application/vnd.criticaltools.wbs+xml\"};\nrevt(global, \"wklc\") -> {regular, \"clkw\", \"application/vnd.crick.clicker.wordbank\"};\nrevt(global, \"WKLC\") -> {regular, \"CLKW\", \"application/vnd.crick.clicker.wordbank\"};\nrevt(global, \"tklc\") -> {regular, \"clkt\", \"application/vnd.crick.clicker.template\"};\nrevt(global, \"TKLC\") -> {regular, \"CLKT\", \"application/vnd.crick.clicker.template\"};\nrevt(global, \"pklc\") -> {regular, \"clkp\", \"application/vnd.crick.clicker.palette\"};\nrevt(global, \"PKLC\") -> {regular, \"CLKP\", \"application/vnd.crick.clicker.palette\"};\nrevt(global, \"kklc\") -> {regular, \"clkk\", \"application/vnd.crick.clicker.keyboard\"};\nrevt(global, \"KKLC\") -> {regular, \"CLKK\", \"application/vnd.crick.clicker.keyboard\"};\nrevt(global, \"xklc\") -> {regular, \"clkx\", \"application/vnd.crick.clicker\"};\nrevt(global, \"XKLC\") -> {regular, \"CLKX\", \"application/vnd.crick.clicker\"};\nrevt(global, \"cmc\") -> {regular, \"cmc\", \"application/vnd.cosmocaller\"};\nrevt(global, \"CMC\") -> {regular, \"CMC\", \"application/vnd.cosmocaller\"};\nrevt(global, \"gsmcbdc\") -> {regular, \"cdbcmsg\", \"application/vnd.contact.cmsg\"};\nrevt(global, \"GSMCBDC\") -> {regular, \"CDBCMSG\", \"application/vnd.contact.cmsg\"};\nrevt(global, \"psc\") -> {regular, \"csp\", \"application/vnd.commonspace\"};\nrevt(global, \"PSC\") -> {regular, \"CSP\", \"application/vnd.commonspace\"};\nrevt(global, \"zma11c\") -> {regular, \"c11amz\", \"application/vnd.cluetrust.cartomobile-config-pkg\"};\nrevt(global, \"ZMA11C\") -> {regular, \"C11AMZ\", \"application/vnd.cluetrust.cartomobile-config-pkg\"};\nrevt(global, \"cma11c\") -> {regular, \"c11amc\", \"application/vnd.cluetrust.cartomobile-config\"};\nrevt(global, \"CMA11C\") -> {regular, \"C11AMC\", \"application/vnd.cluetrust.cartomobile-config\"};\nrevt(global, \"u4c\") -> {regular, \"c4u\", \"application/vnd.clonk.c4group\"};\nrevt(global, \"U4C\") -> {regular, \"C4U\", \"application/vnd.clonk.c4group\"};\nrevt(global, \"p4c\") -> {regular, \"c4p\", \"application/vnd.clonk.c4group\"};\nrevt(global, \"P4C\") -> {regular, \"C4P\", \"application/vnd.clonk.c4group\"};\nrevt(global, \"f4c\") -> {regular, \"c4f\", \"application/vnd.clonk.c4group\"};\nrevt(global, \"F4C\") -> {regular, \"C4F\", \"application/vnd.clonk.c4group\"};\nrevt(global, \"d4c\") -> {regular, \"c4d\", \"application/vnd.clonk.c4group\"};\nrevt(global, \"D4C\") -> {regular, \"C4D\", \"application/vnd.clonk.c4group\"};\nrevt(global, \"g4c\") -> {regular, \"c4g\", \"application/vnd.clonk.c4group\"};\nrevt(global, \"G4C\") -> {regular, \"C4G\", \"application/vnd.clonk.c4group\"};\nrevt(global, \"9pr\") -> {regular, \"rp9\", \"application/vnd.cloanto.rp9\"};\nrevt(global, \"9PR\") -> {regular, \"RP9\", \"application/vnd.cloanto.rp9\"};\nrevt(global, \"alc\") -> {regular, \"cla\", \"application/vnd.claymore\"};\nrevt(global, \"ALC\") -> {regular, \"CLA\", \"application/vnd.claymore\"};\nrevt(global, \"ydc\") -> {regular, \"cdy\", \"application/vnd.cinderella\"};\nrevt(global, \"YDC\") -> {regular, \"CDY\", \"application/vnd.cinderella\"};\nrevt(global, \"dmm\") -> {regular, \"mmd\", \"application/vnd.chipnuts.karaoke-mmd\"};\nrevt(global, \"DMM\") -> {regular, \"MMD\", \"application/vnd.chipnuts.karaoke-mmd\"};\nrevt(global, \"lmxdc\") -> {regular, \"cdxml\", \"application/vnd.chemdraw+xml\"};\nrevt(global, \"LMXDC\") -> {regular, \"CDXML\", \"application/vnd.chemdraw+xml\"};\nrevt(global, \"per\") -> {regular, \"rep\", \"application/vnd.businessobjects\"};\nrevt(global, \"PER\") -> {regular, \"REP\", \"application/vnd.businessobjects\"};\nrevt(global, \"imb\") -> {regular, \"bmi\", \"application/vnd.bmi\"};\nrevt(global, \"IMB\") -> {regular, \"BMI\", \"application/vnd.bmi\"};\nrevt(global, \"mpm\") -> {regular, \"mpm\", \"application/vnd.blueice.multipass\"};\nrevt(global, \"MPM\") -> {regular, \"MPM\", \"application/vnd.blueice.multipass\"};\nrevt(global, \"pea\") -> {regular, \"aep\", \"application/vnd.audiograph\"};\nrevt(global, \"PEA\") -> {regular, \"AEP\", \"application/vnd.audiograph\"};\nrevt(global, \"atoi\") -> {regular, \"iota\", \"application/vnd.astraea-software.iota\"};\nrevt(global, \"ATOI\") -> {regular, \"IOTA\", \"application/vnd.astraea-software.iota\"};\nrevt(global, \"iws\") -> {regular, \"swi\", \"application/vnd.aristanetworks.swi\"};\nrevt(global, \"IWS\") -> {regular, \"SWI\", \"application/vnd.aristanetworks.swi\"};\nrevt(global, \"8u3m\") -> {regular, \"m3u8\", \"application/vnd.apple.mpegurl\"};\nrevt(global, \"8U3M\") -> {regular, \"M3U8\", \"application/vnd.apple.mpegurl\"};\nrevt(global, \"gkpm\") -> {regular, \"mpkg\", \"application/vnd.apple.installer+xml\"};\nrevt(global, \"GKPM\") -> {regular, \"MPKG\", \"application/vnd.apple.installer+xml\"};\nrevt(global, \"xta\") -> {regular, \"atx\", \"application/vnd.antix.game-component\"};\nrevt(global, \"XTA\") -> {regular, \"ATX\", \"application/vnd.antix.game-component\"};\nrevt(global, \"itf\") -> {regular, \"fti\", \"application/vnd.anser-web-funds-transfer-initiation\"};\nrevt(global, \"ITF\") -> {regular, \"FTI\", \"application/vnd.anser-web-funds-transfer-initiation\"};\nrevt(global, \"iic\") -> {regular, \"cii\", \"application/vnd.anser-web-certificate-issue-initiation\"};\nrevt(global, \"IIC\") -> {regular, \"CII\", \"application/vnd.anser-web-certificate-issue-initiation\"};\nrevt(global, \"kpa\") -> {regular, \"apk\", \"application/vnd.android.package-archive\"};\nrevt(global, \"KPA\") -> {regular, \"APK\", \"application/vnd.android.package-archive\"};\nrevt(global, \"ima\") -> {regular, \"ami\", \"application/vnd.amiga.ami\"};\nrevt(global, \"IMA\") -> {regular, \"AMI\", \"application/vnd.amiga.ami\"};\nrevt(global, \"cca\") -> {regular, \"acc\", \"application/vnd.americandynamics.acc\"};\nrevt(global, \"CCA\") -> {regular, \"ACC\", \"application/vnd.americandynamics.acc\"};\nrevt(global, \"wza\") -> {regular, \"azw\", \"application/vnd.amazon.ebook\"};\nrevt(global, \"WZA\") -> {regular, \"AZW\", \"application/vnd.amazon.ebook\"};\nrevt(global, \"sza\") -> {regular, \"azs\", \"application/vnd.airzip.filesecure.azs\"};\nrevt(global, \"SZA\") -> {regular, \"AZS\", \"application/vnd.airzip.filesecure.azs\"};\nrevt(global, \"fza\") -> {regular, \"azf\", \"application/vnd.airzip.filesecure.azf\"};\nrevt(global, \"FZA\") -> {regular, \"AZF\", \"application/vnd.airzip.filesecure.azf\"};\nrevt(global, \"daeha\") -> {regular, \"ahead\", \"application/vnd.ahead.space\"};\nrevt(global, \"DAEHA\") -> {regular, \"AHEAD\", \"application/vnd.ahead.space\"};\nrevt(global, \"fdfx\") -> {regular, \"xfdf\", \"application/vnd.adobe.xfdf\"};\nrevt(global, \"FDFX\") -> {regular, \"XFDF\", \"application/vnd.adobe.xfdf\"};\nrevt(global, \"pdx\") -> {regular, \"xdp\", \"application/vnd.adobe.xdp+xml\"};\nrevt(global, \"PDX\") -> {regular, \"XDP\", \"application/vnd.adobe.xdp+xml\"};\nrevt(global, \"lpxf\") -> {regular, \"fxpl\", \"application/vnd.adobe.fxp\"};\nrevt(global, \"LPXF\") -> {regular, \"FXPL\", \"application/vnd.adobe.fxp\"};\nrevt(global, \"pxf\") -> {regular, \"fxp\", \"application/vnd.adobe.fxp\"};\nrevt(global, \"PXF\") -> {regular, \"FXP\", \"application/vnd.adobe.fxp\"};\nrevt(global, \"tdcf\") -> {regular, \"fcdt\", \"application/vnd.adobe.formscentral.fcdt\"};\nrevt(global, \"TDCF\") -> {regular, \"FCDT\", \"application/vnd.adobe.formscentral.fcdt\"};\nrevt(global, \"ria\") -> {regular, \"air\", \"application/vnd.adobe.air-application-installer-package+zip\"};\nrevt(global, \"RIA\") -> {regular, \"AIR\", \"application/vnd.adobe.air-application-installer-package+zip\"};\nrevt(global, \"ctuca\") -> {regular, \"acutc\", \"application/vnd.acucorp\"};\nrevt(global, \"CTUCA\") -> {regular, \"ACUTC\", \"application/vnd.acucorp\"};\nrevt(global, \"cta\") -> {regular, \"atc\", \"application/vnd.acucorp\"};\nrevt(global, \"CTA\") -> {regular, \"ATC\", \"application/vnd.acucorp\"};\nrevt(global, \"uca\") -> {regular, \"acu\", \"application/vnd.acucobol\"};\nrevt(global, \"UCA\") -> {regular, \"ACU\", \"application/vnd.acucobol\"};\nrevt(global, \"pmi\") -> {regular, \"imp\", \"application/vnd.accpac.simply.imp\"};\nrevt(global, \"PMI\") -> {regular, \"IMP\", \"application/vnd.accpac.simply.imp\"};\nrevt(global, \"osa\") -> {regular, \"aso\", \"application/vnd.accpac.simply.aso\"};\nrevt(global, \"OSA\") -> {regular, \"ASO\", \"application/vnd.accpac.simply.aso\"};\nrevt(global, \"nwp\") -> {regular, \"pwn\", \"application/vnd.3m.post-it-notes\"};\nrevt(global, \"NWP\") -> {regular, \"PWN\", \"application/vnd.3m.post-it-notes\"};\nrevt(global, \"pact\") -> {regular, \"tcap\", \"application/vnd.3gpp2.tcap\"};\nrevt(global, \"PACT\") -> {regular, \"TCAP\", \"application/vnd.3gpp2.tcap\"};\nrevt(global, \"bvp\") -> {regular, \"pvb\", \"application/vnd.3gpp.pic-bw-var\"};\nrevt(global, \"BVP\") -> {regular, \"PVB\", \"application/vnd.3gpp.pic-bw-var\"};\nrevt(global, \"bsp\") -> {regular, \"psb\", \"application/vnd.3gpp.pic-bw-small\"};\nrevt(global, \"BSP\") -> {regular, \"PSB\", \"application/vnd.3gpp.pic-bw-small\"};\nrevt(global, \"blp\") -> {regular, \"plb\", \"application/vnd.3gpp.pic-bw-large\"};\nrevt(global, \"BLP\") -> {regular, \"PLB\", \"application/vnd.3gpp.pic-bw-large\"};\nrevt(global, \"dst\") -> {regular, \"tsd\", \"application/timestamped-data\"};\nrevt(global, \"DST\") -> {regular, \"TSD\", \"application/timestamped-data\"};\nrevt(global, \"ift\") -> {regular, \"tfi\", \"application/thraud+xml\"};\nrevt(global, \"IFT\") -> {regular, \"TFI\", \"application/thraud+xml\"};\nrevt(global, \"suprociet\") -> {regular, \"teicorpus\", \"application/tei+xml\"};\nrevt(global, \"SUPROCIET\") -> {regular, \"TEICORPUS\", \"application/tei+xml\"};\nrevt(global, \"iet\") -> {regular, \"tei\", \"application/tei+xml\"};\nrevt(global, \"IET\") -> {regular, \"TEI\", \"application/tei+xml\"};\nrevt(global, \"lmss\") -> {regular, \"ssml\", \"application/ssml+xml\"};\nrevt(global, \"LMSS\") -> {regular, \"SSML\", \"application/ssml+xml\"};\nrevt(global, \"ldss\") -> {regular, \"ssdl\", \"application/ssdl+xml\"};\nrevt(global, \"LDSS\") -> {regular, \"SSDL\", \"application/ssdl+xml\"};\nrevt(global, \"urs\") -> {regular, \"sru\", \"application/sru+xml\"};\nrevt(global, \"URS\") -> {regular, \"SRU\", \"application/sru+xml\"};\nrevt(global, \"lmxrg\") -> {regular, \"grxml\", \"application/srgs+xml\"};\nrevt(global, \"LMXRG\") -> {regular, \"GRXML\", \"application/srgs+xml\"};\nrevt(global, \"marg\") -> {regular, \"gram\", \"application/srgs\"};\nrevt(global, \"MARG\") -> {regular, \"GRAM\", \"application/srgs\"};\nrevt(global, \"xrs\") -> {regular, \"srx\", \"application/sparql-results+xml\"};\nrevt(global, \"XRS\") -> {regular, \"SRX\", \"application/sparql-results+xml\"};\nrevt(global, \"qr\") -> {regular, \"rq\", \"application/sparql-query\"};\nrevt(global, \"QR\") -> {regular, \"RQ\", \"application/sparql-query\"};\nrevt(global, \"lims\") -> {regular, \"smil\", \"application/smil+xml\"};\nrevt(global, \"LIMS\") -> {regular, \"SMIL\", \"application/smil+xml\"};\nrevt(global, \"ims\") -> {regular, \"smi\", \"application/smil+xml\"};\nrevt(global, \"IMS\") -> {regular, \"SMI\", \"application/smil+xml\"};\nrevt(global, \"fhs\") -> {regular, \"shf\", \"application/shf+xml\"};\nrevt(global, \"FHS\") -> {regular, \"SHF\", \"application/shf+xml\"};\nrevt(global, \"gertes\") -> {regular, \"setreg\", \"application/set-registration-initiation\"};\nrevt(global, \"GERTES\") -> {regular, \"SETREG\", \"application/set-registration-initiation\"};\nrevt(global, \"yaptes\") -> {regular, \"setpay\", \"application/set-payment-initiation\"};\nrevt(global, \"YAPTES\") -> {regular, \"SETPAY\", \"application/set-payment-initiation\"};\nrevt(global, \"pds\") -> {regular, \"sdp\", \"application/sdp\"};\nrevt(global, \"PDS\") -> {regular, \"SDP\", \"application/sdp\"};\nrevt(global, \"pps\") -> {regular, \"spp\", \"application/scvp-vp-response\"};\nrevt(global, \"PPS\") -> {regular, \"SPP\", \"application/scvp-vp-response\"};\nrevt(global, \"qps\") -> {regular, \"spq\", \"application/scvp-vp-request\"};\nrevt(global, \"QPS\") -> {regular, \"SPQ\", \"application/scvp-vp-request\"};\nrevt(global, \"scs\") -> {regular, \"scs\", \"application/scvp-cv-response\"};\nrevt(global, \"SCS\") -> {regular, \"SCS\", \"application/scvp-cv-response\"};\nrevt(global, \"qcs\") -> {regular, \"scq\", \"application/scvp-cv-request\"};\nrevt(global, \"QCS\") -> {regular, \"SCQ\", \"application/scvp-cv-request\"};\nrevt(global, \"lmbs\") -> {regular, \"sbml\", \"application/sbml+xml\"};\nrevt(global, \"LMBS\") -> {regular, \"SBML\", \"application/sbml+xml\"};\nrevt(global, \"ftr\") -> {regular, \"rtf\", \"text/rtf\"};\nrevt(global, \"FTR\") -> {regular, \"RTF\", \"text/rtf\"};\nrevt(global, \"ssr\") -> {regular, \"rss\", \"application/rss+xml\"};\nrevt(global, \"SSR\") -> {regular, \"RSS\", \"application/rss+xml\"};\nrevt(global, \"dsr\") -> {regular, \"rsd\", \"application/rsd+xml\"};\nrevt(global, \"DSR\") -> {regular, \"RSD\", \"application/rsd+xml\"};\nrevt(global, \"aor\") -> {regular, \"roa\", \"application/rpki-roa\"};\nrevt(global, \"AOR\") -> {regular, \"ROA\", \"application/rpki-roa\"};\nrevt(global, \"tfm\") -> {regular, \"mft\", \"application/rpki-manifest\"};\nrevt(global, \"TFM\") -> {regular, \"MFT\", \"application/rpki-manifest\"};\nrevt(global, \"rbg\") -> {regular, \"gbr\", \"application/rpki-ghostbusters\"};\nrevt(global, \"RBG\") -> {regular, \"GBR\", \"application/rpki-ghostbusters\"};\nrevt(global, \"sr\") -> {regular, \"rs\", \"application/rls-services+xml\"};\nrevt(global, \"SR\") -> {regular, \"RS\", \"application/rls-services+xml\"};\nrevt(global, \"dlr\") -> {regular, \"rld\", \"application/resource-lists-diff+xml\"};\nrevt(global, \"DLR\") -> {regular, \"RLD\", \"application/resource-lists-diff+xml\"};\nrevt(global, \"lr\") -> {regular, \"rl\", \"application/resource-lists+xml\"};\nrevt(global, \"LR\") -> {regular, \"RL\", \"application/resource-lists+xml\"};\nrevt(global, \"cnr\") -> {regular, \"rnc\", \"application/relax-ng-compact-syntax\"};\nrevt(global, \"CNR\") -> {regular, \"RNC\", \"application/relax-ng-compact-syntax\"};\nrevt(global, \"fir\") -> {regular, \"rif\", \"application/reginfo+xml\"};\nrevt(global, \"FIR\") -> {regular, \"RIF\", \"application/reginfo+xml\"};\nrevt(global, \"fdr\") -> {regular, \"rdf\", \"application/rdf+xml\"};\nrevt(global, \"FDR\") -> {regular, \"RDF\", \"application/rdf+xml\"};\nrevt(global, \"lmxcksp\") -> {regular, \"pskcxml\", \"application/pskc+xml\"};\nrevt(global, \"LMXCKSP\") -> {regular, \"PSKCXML\", \"application/pskc+xml\"};\nrevt(global, \"wwc\") -> {regular, \"cww\", \"application/prs.cww\"};\nrevt(global, \"WWC\") -> {regular, \"CWW\", \"application/prs.cww\"};\nrevt(global, \"sp\") -> {regular, \"ps\", \"application/postscript\"};\nrevt(global, \"SP\") -> {regular, \"PS\", \"application/postscript\"};\nrevt(global, \"spe\") -> {regular, \"eps\", \"application/postscript\"};\nrevt(global, \"SPE\") -> {regular, \"EPS\", \"application/postscript\"};\nrevt(global, \"ia\") -> {regular, \"ai\", \"application/postscript\"};\nrevt(global, \"IA\") -> {regular, \"AI\", \"application/postscript\"};\nrevt(global, \"slp\") -> {regular, \"pls\", \"application/pls+xml\"};\nrevt(global, \"SLP\") -> {regular, \"PLS\", \"application/pls+xml\"};\nrevt(global, \"ikp\") -> {regular, \"pki\", \"application/pkixcmp\"};\nrevt(global, \"IKP\") -> {regular, \"PKI\", \"application/pkixcmp\"};\nrevt(global, \"htapikp\") -> {regular, \"pkipath\", \"application/pkix-pkipath\"};\nrevt(global, \"HTAPIKP\") -> {regular, \"PKIPATH\", \"application/pkix-pkipath\"};\nrevt(global, \"lrc\") -> {regular, \"crl\", \"application/pkix-crl\"};\nrevt(global, \"LRC\") -> {regular, \"CRL\", \"application/pkix-crl\"};\nrevt(global, \"rec\") -> {regular, \"cer\", \"application/pkix-cert\"};\nrevt(global, \"REC\") -> {regular, \"CER\", \"application/pkix-cert\"};\nrevt(global, \"ca\") -> {regular, \"ac\", \"application/pkix-attr-cert\"};\nrevt(global, \"CA\") -> {regular, \"AC\", \"application/pkix-attr-cert\"};\nrevt(global, \"8p\") -> {regular, \"p8\", \"application/pkcs8\"};\nrevt(global, \"8P\") -> {regular, \"P8\", \"application/pkcs8\"};\nrevt(global, \"s7p\") -> {regular, \"p7s\", \"application/pkcs7-signature\"};\nrevt(global, \"S7P\") -> {regular, \"P7S\", \"application/pkcs7-signature\"};\nrevt(global, \"c7p\") -> {regular, \"p7c\", \"application/pkcs7-mime\"};\nrevt(global, \"C7P\") -> {regular, \"P7C\", \"application/pkcs7-mime\"};\nrevt(global, \"m7p\") -> {regular, \"p7m\", \"application/pkcs7-mime\"};\nrevt(global, \"M7P\") -> {regular, \"P7M\", \"application/pkcs7-mime\"};\nrevt(global, \"01p\") -> {regular, \"p10\", \"application/pkcs10\"};\nrevt(global, \"01P\") -> {regular, \"P10\", \"application/pkcs10\"};\nrevt(global, \"frp\") -> {regular, \"prf\", \"application/pics-rules\"};\nrevt(global, \"FRP\") -> {regular, \"PRF\", \"application/pics-rules\"};\nrevt(global, \"gis\") -> {regular, \"sig\", \"application/pgp-signature\"};\nrevt(global, \"GIS\") -> {regular, \"SIG\", \"application/pgp-signature\"};\nrevt(global, \"csa\") -> {regular, \"asc\", \"application/pgp-signature\"};\nrevt(global, \"CSA\") -> {regular, \"ASC\", \"application/pgp-signature\"};\nrevt(global, \"pgp\") -> {regular, \"pgp\", \"application/pgp-encrypted\"};\nrevt(global, \"PGP\") -> {regular, \"PGP\", \"application/pgp-encrypted\"};\nrevt(global, \"fdp\") -> {regular, \"pdf\", \"application/pdf\"};\nrevt(global, \"FDP\") -> {regular, \"PDF\", \"application/pdf\"};\nrevt(global, \"rex\") -> {regular, \"xer\", \"application/patch-ops-error+xml\"};\nrevt(global, \"REX\") -> {regular, \"XER\", \"application/patch-ops-error+xml\"};\nrevt(global, \"spxo\") -> {regular, \"oxps\", \"application/oxps\"};\nrevt(global, \"SPXO\") -> {regular, \"OXPS\", \"application/oxps\"};\nrevt(global, \"gkpeno\") -> {regular, \"onepkg\", \"application/onenote\"};\nrevt(global, \"GKPENO\") -> {regular, \"ONEPKG\", \"application/onenote\"};\nrevt(global, \"pmteno\") -> {regular, \"onetmp\", \"application/onenote\"};\nrevt(global, \"PMTENO\") -> {regular, \"ONETMP\", \"application/onenote\"};\nrevt(global, \"2coteno\") -> {regular, \"onetoc2\", \"application/onenote\"};\nrevt(global, \"2COTENO\") -> {regular, \"ONETOC2\", \"application/onenote\"};\nrevt(global, \"coteno\") -> {regular, \"onetoc\", \"application/onenote\"};\nrevt(global, \"COTENO\") -> {regular, \"ONETOC\", \"application/onenote\"};\nrevt(global, \"codmo\") -> {regular, \"omdoc\", \"application/omdoc+xml\"};\nrevt(global, \"CODMO\") -> {regular, \"OMDOC\", \"application/omdoc+xml\"};\nrevt(global, \"xgo\") -> {regular, \"ogx\", \"application/ogg\"};\nrevt(global, \"XGO\") -> {regular, \"OGX\", \"application/ogg\"};\nrevt(global, \"fpo\") -> {regular, \"opf\", \"application/oebps-package+xml\"};\nrevt(global, \"FPO\") -> {regular, \"OPF\", \"application/oebps-package+xml\"};\nrevt(global, \"ado\") -> {regular, \"oda\", \"application/oda\"};\nrevt(global, \"ADO\") -> {regular, \"ODA\", \"application/oda\"};\nrevt(global, \"yolped\") -> {regular, \"deploy\", \"application/octet-stream\"};\nrevt(global, \"YOLPED\") -> {regular, \"DEPLOY\", \"application/octet-stream\"};\nrevt(global, \"cle\") -> {regular, \"elc\", \"application/octet-stream\"};\nrevt(global, \"CLE\") -> {regular, \"ELC\", \"application/octet-stream\"};\nrevt(global, \"pmud\") -> {regular, \"dump\", \"application/octet-stream\"};\nrevt(global, \"PMUD\") -> {regular, \"DUMP\", \"application/octet-stream\"};\nrevt(global, \"kpb\") -> {regular, \"bpk\", \"application/octet-stream\"};\nrevt(global, \"KPB\") -> {regular, \"BPK\", \"application/octet-stream\"};\nrevt(global, \"gkp\") -> {regular, \"pkg\", \"application/octet-stream\"};\nrevt(global, \"GKP\") -> {regular, \"PKG\", \"application/octet-stream\"};\nrevt(global, \"ztsid\") -> {regular, \"distz\", \"application/octet-stream\"};\nrevt(global, \"ZTSID\") -> {regular, \"DISTZ\", \"application/octet-stream\"};\nrevt(global, \"tsid\") -> {regular, \"dist\", \"application/octet-stream\"};\nrevt(global, \"TSID\") -> {regular, \"DIST\", \"application/octet-stream\"};\nrevt(global, \"os\") -> {regular, \"so\", \"application/octet-stream\"};\nrevt(global, \"OS\") -> {regular, \"SO\", \"application/octet-stream\"};\nrevt(global, \"ram\") -> {regular, \"mar\", \"application/octet-stream\"};\nrevt(global, \"RAM\") -> {regular, \"MAR\", \"application/octet-stream\"};\nrevt(global, \"frl\") -> {regular, \"lrf\", \"application/octet-stream\"};\nrevt(global, \"FRL\") -> {regular, \"LRF\", \"application/octet-stream\"};\nrevt(global, \"smd\") -> {regular, \"dms\", \"application/octet-stream\"};\nrevt(global, \"SMD\") -> {regular, \"DMS\", \"application/octet-stream\"};\nrevt(global, \"nib\") -> {regular, \"bin\", \"application/octet-stream\"};\nrevt(global, \"NIB\") -> {regular, \"BIN\", \"application/octet-stream\"};\nrevt(global, \"fxm\") -> {regular, \"mxf\", \"application/mxf\"};\nrevt(global, \"FXM\") -> {regular, \"MXF\", \"application/mxf\"};\nrevt(global, \"tod\") -> {regular, \"dot\", \"application/msword\"};\nrevt(global, \"TOD\") -> {regular, \"DOT\", \"application/msword\"};\nrevt(global, \"cod\") -> {regular, \"doc\", \"application/msword\"};\nrevt(global, \"COD\") -> {regular, \"DOC\", \"application/msword\"};\nrevt(global, \"s4pm\") -> {regular, \"mp4s\", \"application/mp4\"};\nrevt(global, \"S4PM\") -> {regular, \"MP4S\", \"application/mp4\"};\nrevt(global, \"12pm\") -> {regular, \"mp21\", \"application/mp21\"};\nrevt(global, \"12PM\") -> {regular, \"MP21\", \"application/mp21\"};\nrevt(global, \"12m\") -> {regular, \"m21\", \"application/mp21\"};\nrevt(global, \"12M\") -> {regular, \"M21\", \"application/mp21\"};\nrevt(global, \"sdom\") -> {regular, \"mods\", \"application/mods+xml\"};\nrevt(global, \"SDOM\") -> {regular, \"MODS\", \"application/mods+xml\"};\nrevt(global, \"stem\") -> {regular, \"mets\", \"application/mets+xml\"};\nrevt(global, \"STEM\") -> {regular, \"METS\", \"application/mets+xml\"};\nrevt(global, \"4atem\") -> {regular, \"meta4\", \"application/metalink4+xml\"};\nrevt(global, \"4ATEM\") -> {regular, \"META4\", \"application/metalink4+xml\"};\nrevt(global, \"knilatem\") -> {regular, \"metalink\", \"application/metalink+xml\"};\nrevt(global, \"KNILATEM\") -> {regular, \"METALINK\", \"application/metalink+xml\"};\nrevt(global, \"lmcsm\") -> {regular, \"mscml\", \"application/mediaservercontrol+xml\"};\nrevt(global, \"LMCSM\") -> {regular, \"MSCML\", \"application/mediaservercontrol+xml\"};\nrevt(global, \"xobm\") -> {regular, \"mbox\", \"application/mbox\"};\nrevt(global, \"XOBM\") -> {regular, \"MBOX\", \"application/mbox\"};\nrevt(global, \"lmhtam\") -> {regular, \"mathml\", \"application/mathml+xml\"};\nrevt(global, \"LMHTAM\") -> {regular, \"MATHML\", \"application/mathml+xml\"};\nrevt(global, \"bm\") -> {regular, \"mb\", \"application/mathematica\"};\nrevt(global, \"BM\") -> {regular, \"MB\", \"application/mathematica\"};\nrevt(global, \"bn\") -> {regular, \"nb\", \"application/mathematica\"};\nrevt(global, \"BN\") -> {regular, \"NB\", \"application/mathematica\"};\nrevt(global, \"am\") -> {regular, \"ma\", \"application/mathematica\"};\nrevt(global, \"AM\") -> {regular, \"MA\", \"application/mathematica\"};\nrevt(global, \"xcrm\") -> {regular, \"mrcx\", \"application/marcxml+xml\"};\nrevt(global, \"XCRM\") -> {regular, \"MRCX\", \"application/marcxml+xml\"};\nrevt(global, \"crm\") -> {regular, \"mrc\", \"application/marc\"};\nrevt(global, \"CRM\") -> {regular, \"MRC\", \"application/marc\"};\nrevt(global, \"sdam\") -> {regular, \"mads\", \"application/mads+xml\"};\nrevt(global, \"SDAM\") -> {regular, \"MADS\", \"application/mads+xml\"};\nrevt(global, \"tpc\") -> {regular, \"cpt\", \"application/mac-compactpro\"};\nrevt(global, \"TPC\") -> {regular, \"CPT\", \"application/mac-compactpro\"};\nrevt(global, \"xqh\") -> {regular, \"hqx\", \"application/mac-binhex40\"};\nrevt(global, \"XQH\") -> {regular, \"HQX\", \"application/mac-binhex40\"};\nrevt(global, \"lmxtsol\") -> {regular, \"lostxml\", \"application/lost+xml\"};\nrevt(global, \"LMXTSOL\") -> {regular, \"LOSTXML\", \"application/lost+xml\"};\nrevt(global, \"lmnosj\") -> {regular, \"jsonml\", \"application/jsonml+json\"};\nrevt(global, \"LMNOSJ\") -> {regular, \"JSONML\", \"application/jsonml+json\"};\nrevt(global, \"nosj\") -> {regular, \"json\", \"application/json\"};\nrevt(global, \"NOSJ\") -> {regular, \"JSON\", \"application/json\"};\nrevt(global, \"sj\") -> {regular, \"js\", \"application/x-javascript\"};\nrevt(global, \"SJ\") -> {regular, \"JS\", \"application/x-javascript\"};\nrevt(global, \"ssalc\") -> {regular, \"class\", \"application/java-vm\"};\nrevt(global, \"SSALC\") -> {regular, \"CLASS\", \"application/java-vm\"};\nrevt(global, \"res\") -> {regular, \"ser\", \"application/java-serialized-object\"};\nrevt(global, \"RES\") -> {regular, \"SER\", \"application/java-serialized-object\"};\nrevt(global, \"raj\") -> {regular, \"jar\", \"application/java-archive\"};\nrevt(global, \"RAJ\") -> {regular, \"JAR\", \"application/java-archive\"};\nrevt(global, \"xifpi\") -> {regular, \"ipfix\", \"application/ipfix\"};\nrevt(global, \"XIFPI\") -> {regular, \"IPFIX\", \"application/ipfix\"};\nrevt(global, \"lmkni\") -> {regular, \"inkml\", \"application/inkml+xml\"};\nrevt(global, \"LMKNI\") -> {regular, \"INKML\", \"application/inkml+xml\"};\nrevt(global, \"kni\") -> {regular, \"ink\", \"application/inkml+xml\"};\nrevt(global, \"KNI\") -> {regular, \"INK\", \"application/inkml+xml\"};\nrevt(global, \"kts\") -> {regular, \"stk\", \"application/hyperstudio\"};\nrevt(global, \"KTS\") -> {regular, \"STK\", \"application/hyperstudio\"};\nrevt(global, \"fxg\") -> {regular, \"gxf\", \"application/gxf\"};\nrevt(global, \"FXG\") -> {regular, \"GXF\", \"application/gxf\"};\nrevt(global, \"xpg\") -> {regular, \"gpx\", \"application/gpx+xml\"};\nrevt(global, \"XPG\") -> {regular, \"GPX\", \"application/gpx+xml\"};\nrevt(global, \"lmg\") -> {regular, \"gml\", \"application/gml+xml\"};\nrevt(global, \"LMG\") -> {regular, \"GML\", \"application/gml+xml\"};\nrevt(global, \"ffow\") -> {regular, \"woff\", \"application/font-woff\"};\nrevt(global, \"FFOW\") -> {regular, \"WOFF\", \"application/font-woff\"};\nrevt(global, \"rfp\") -> {regular, \"pfr\", \"application/font-tdpfr\"};\nrevt(global, \"RFP\") -> {regular, \"PFR\", \"application/font-tdpfr\"};\nrevt(global, \"ixe\") -> {regular, \"exi\", \"application/exi\"};\nrevt(global, \"IXE\") -> {regular, \"EXI\", \"application/exi\"};\nrevt(global, \"bupe\") -> {regular, \"epub\", \"application/epub+zip\"};\nrevt(global, \"BUPE\") -> {regular, \"EPUB\", \"application/epub+zip\"};\nrevt(global, \"amme\") -> {regular, \"emma\", \"application/emma+xml\"};\nrevt(global, \"AMME\") -> {regular, \"EMMA\", \"application/emma+xml\"};\nrevt(global, \"amce\") -> {regular, \"ecma\", \"application/ecmascript\"};\nrevt(global, \"AMCE\") -> {regular, \"ECMA\", \"application/ecmascript\"};\nrevt(global, \"cssdx\") -> {regular, \"xdssc\", \"application/dssc+xml\"};\nrevt(global, \"CSSDX\") -> {regular, \"XDSSC\", \"application/dssc+xml\"};\nrevt(global, \"cssd\") -> {regular, \"dssc\", \"application/dssc+der\"};\nrevt(global, \"CSSD\") -> {regular, \"DSSC\", \"application/dssc+der\"};\nrevt(global, \"kbd\") -> {regular, \"dbk\", \"application/docbook+xml\"};\nrevt(global, \"KBD\") -> {regular, \"DBK\", \"application/docbook+xml\"};\nrevt(global, \"tnuomvad\") -> {regular, \"davmount\", \"application/davmount+xml\"};\nrevt(global, \"TNUOMVAD\") -> {regular, \"DAVMOUNT\", \"application/davmount+xml\"};\nrevt(global, \"uc\") -> {regular, \"cu\", \"application/cu-seeme\"};\nrevt(global, \"UC\") -> {regular, \"CU\", \"application/cu-seeme\"};\nrevt(global, \"qimdc\") -> {regular, \"cdmiq\", \"application/cdmi-queue\"};\nrevt(global, \"QIMDC\") -> {regular, \"CDMIQ\", \"application/cdmi-queue\"};\nrevt(global, \"oimdc\") -> {regular, \"cdmio\", \"application/cdmi-object\"};\nrevt(global, \"OIMDC\") -> {regular, \"CDMIO\", \"application/cdmi-object\"};\nrevt(global, \"dimdc\") -> {regular, \"cdmid\", \"application/cdmi-domain\"};\nrevt(global, \"DIMDC\") -> {regular, \"CDMID\", \"application/cdmi-domain\"};\nrevt(global, \"cimdc\") -> {regular, \"cdmic\", \"application/cdmi-container\"};\nrevt(global, \"CIMDC\") -> {regular, \"CDMIC\", \"application/cdmi-container\"};\nrevt(global, \"aimdc\") -> {regular, \"cdmia\", \"application/cdmi-capability\"};\nrevt(global, \"AIMDC\") -> {regular, \"CDMIA\", \"application/cdmi-capability\"};\nrevt(global, \"lmxcc\") -> {regular, \"ccxml\", \"application/ccxml+xml\"};\nrevt(global, \"LMXCC\") -> {regular, \"CCXML\", \"application/ccxml+xml\"};\nrevt(global, \"cvsmota\") -> {regular, \"atomsvc\", \"application/atomsvc+xml\"};\nrevt(global, \"CVSMOTA\") -> {regular, \"ATOMSVC\", \"application/atomsvc+xml\"};\nrevt(global, \"tacmota\") -> {regular, \"atomcat\", \"application/atomcat+xml\"};\nrevt(global, \"TACMOTA\") -> {regular, \"ATOMCAT\", \"application/atomcat+xml\"};\nrevt(global, \"mota\") -> {regular, \"atom\", \"application/atom+xml\"};\nrevt(global, \"MOTA\") -> {regular, \"ATOM\", \"application/atom+xml\"};\nrevt(global, \"wa\") -> {regular, \"aw\", \"application/applixware\"};\nrevt(global, \"WA\") -> {regular, \"AW\", \"application/applixware\"};\nrevt(global, \"ze\") -> {regular, \"ez\", \"application/andrew-inset\"};\nrevt(global, \"ZE\") -> {regular, \"EZ\", \"application/andrew-inset\"};\nrevt(global, \"igcf\") -> {fcgi, \"fcgi\", \"text/html\"};\nrevt(global, \"IGCF\") -> {fcgi, \"FCGI\", \"text/html\"};\nrevt(global, \"igc\") -> {cgi, \"cgi\", \"text/html\"};\nrevt(global, \"IGC\") -> {cgi, \"CGI\", \"text/html\"};\nrevt(global, \"php\") -> {php, \"php\", \"text/html\"};\nrevt(global, \"PHP\") -> {php, \"PHP\", \"text/html\"};\nrevt(global, \"sway\") -> {yaws, \"yaws\", \"text/html\"};\nrevt(global, \"SWAY\") -> {yaws, \"YAWS\", \"text/html\"};\nrevt(global, RExt) -> {regular, lists:reverse(RExt), \"text/plain\"};\nrevt(_, RExt) -> revt(global, RExt).\n"
  },
  {
    "path": "contrib/yaws/src/yaws.app.src",
    "content": "{application,yaws,\n [{description,\"yaws WWW server\"},\n  {vsn, {cmd, \"sed -ne '1,/^YAWS_VSN=/s/^YAWS_VSN=//p' vsn.mk 2>/dev/null\"}},\n  {modules,[]},\n  {registered, []},\n  {mod,{yaws_app,[]}},\n  {env, [\n         % {config, undefined},           % undefined | filename()\n         % {debug, false},                % true | false\n         % {trace, false},                % http | traffic | false\n         % {traceoutput, false},          % true | false\n         % {conf, \"/etc/yaws.conf\"},      % string()\n         % {runmod, mymodule},            % atom()\n         % {embedded, false},             % true | false\n         % {id, \"default\"},               % string()\n         % {pam_service, \"system-auth\"},  % string()\n         % {pam_use_acct, true},          % true | false\n         % {pam_use_sess, true}           % true | false\n        ]},\n  {applications,[kernel,stdlib@APPDEPS@]}]}.\n"
  },
  {
    "path": "contrib/yaws/src/yaws.app.src.script",
    "content": "%% -*- mode: erlang;erlang-indent-level: 4;indent-tabs-mode: nil -*-\n%% ex: ft=erlang ts=4 sw=4 et\n\nAppDeps = case os:getenv(\"YAWS_APPDEPS\") of\n              false ->\n                  [];\n              AppDeps0 ->\n                  string:tokens(AppDeps0, \" ,\")\n          end,\nAppDepsList = \",\"++string:join(AppDeps, \",\"),\n\n%% generate src/yaws_appdeps.hrl from src/yaws_appdeps.hrl.in\nDir = filename:dirname(SCRIPT),\nInFile = filename:join(Dir, \"yaws_appdeps.hrl.in\"),\nOutFile = filename:join(Dir, \"yaws_appdeps.hrl\"),\n{ok, AppHrl0} = file:read_file(InFile),\nAppHrl = binary:replace(AppHrl0, <<\"@APPDEPS@\">>, list_to_binary(AppDepsList)),\nok = file:write_file(OutFile, AppHrl),\n\n%% replace @APPDEPS@ in application dependencies with any apps specified in\n%% the YAWS_APPDEPS OS env var\n[{application,yaws,Cfg}] = CONFIG,\n{applications,Apps0} = lists:keyfind(applications, 1, Cfg),\nApps = lists:flatmap(fun('stdlib@APPDEPS@') ->\n                             [stdlib]++[list_to_atom(A) || A <- AppDeps];\n                        (A) ->\n                             [A]\n                     end, Apps0),\n[{application,yaws,lists:keyreplace(applications, 1, Cfg,\n                                    {applications, Apps})}].\n"
  },
  {
    "path": "contrib/yaws/src/yaws.appup.src",
    "content": "{\"%VSN%\",\n  [\n  ],\n  [\n  ]}.\n"
  },
  {
    "path": "contrib/yaws/src/yaws.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws.erl\n%%% Author  : Claes Wikstrom <klacke@bluetail.com>\n%%% Purpose :\n%%% Created : 16 Jan 2002 by Claes Wikstrom <klacke@bluetail.com>\n%%%----------------------------------------------------------------------\n\n-module(yaws).\n-author('klacke@bluetail.com').\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_appdeps.hrl\").\n-include(\"yaws_debug.hrl\").\n\n-include_lib(\"kernel/include/file.hrl\").\n-export([start/0, stop/0, hup/0, hup/1, restart/0, modules/0, load/0]).\n-export([start_embedded/1, start_embedded/2, start_embedded/3, start_embedded/4,\n         add_server/2, create_gconf/2, create_sconf/2, setup_sconf/2]).\n\n-export([gconf_yaws_dir/1, gconf_trace/1, gconf_flags/1, gconf_logdir/1,\n         gconf_ebin_dir/1, gconf_src_dir/1, gconf_runmods/1,\n         gconf_keepalive_timeout/1, gconf_keepalive_maxuses/1,\n         gconf_max_num_cached_files/1, gconf_max_num_cached_bytes/1,\n         gconf_max_size_cached_file/1, gconf_max_connections/1,\n         gconf_process_options/1, gconf_large_file_chunk_size/1,\n         gconf_mnesia_dir/1, gconf_log_wrap_size/1, gconf_cache_refresh_secs/1,\n         gconf_include_dir/1, gconf_phpexe/1, gconf_yaws/1, gconf_id/1,\n         gconf_enable_soap/1, gconf_soap_srv_mods/1, gconf_ysession_mod/1,\n         gconf_acceptor_pool_size/1, gconf_mime_types_info/1,\n         gconf_nslookup_pref/1,\n         gconf_ysession_idle_timeout/1, gconf_ysession_long_timeout/1,\n         gconf_sni/1]).\n\n-export([gconf_yaws_dir/2, gconf_trace/2, gconf_flags/2, gconf_logdir/2,\n         gconf_ebin_dir/2, gconf_src_dir/2, gconf_runmods/2,\n         gconf_keepalive_timeout/2, gconf_keepalive_maxuses/2,\n         gconf_max_num_cached_files/2, gconf_max_num_cached_bytes/2,\n         gconf_max_size_cached_file/2, gconf_max_connections/2,\n         gconf_process_options/2, gconf_large_file_chunk_size/2,\n         gconf_mnesia_dir/2, gconf_log_wrap_size/2, gconf_cache_refresh_secs/2,\n         gconf_include_dir/2, gconf_phpexe/2, gconf_yaws/2, gconf_id/2,\n         gconf_enable_soap/2, gconf_soap_srv_mods/2, gconf_ysession_mod/2,\n         gconf_acceptor_pool_size/2, gconf_mime_types_info/2,\n         gconf_nslookup_pref/2,\n         gconf_ysession_idle_timeout/2, gconf_ysession_long_timeout/2,\n         gconf_sni/2]).\n\n-export([sconf_port/1, sconf_flags/1, sconf_redirect_map/1, sconf_rhost/1,\n         sconf_rmethod/1, sconf_docroot/1, sconf_xtra_docroots/1,\n         sconf_listen/1, sconf_servername/1, sconf_serveralias/1, sconf_yaws/1,\n         sconf_ets/1, sconf_ssl/1, sconf_authdirs/1, sconf_partial_post_size/1,\n         sconf_appmods/1, sconf_expires/1, sconf_errormod_401/1,\n         sconf_errormod_404/1, sconf_arg_rewrite_mode/1, sconf_logger_mod/1,\n         sconf_opaque/1, sconf_start_mod/1, sconf_allowed_scripts/1,\n         sconf_tilde_allowed_scripts/1, sconf_index_files/1, sconf_revproxy/1,\n         sconf_spotions/1, sconf_extra_cgi_vars/1, sconf_stats/1,\n         sconf_fcgi_app_server/1, sconf_php_handler/1, sconf_shaper/1,\n         sconf_deflate_options/1, sconf_mime_types_info/1,\n         sconf_dispatch_mod/1]).\n\n-export([sconf_port/2, sconf_flags/2, sconf_redirect_map/2, sconf_rhost/2,\n         sconf_rmethod/2, sconf_docroot/2, sconf_xtra_docroots/2,\n         sconf_listen/2, sconf_servername/2, sconf_serveralias/2, sconf_yaws/2,\n         sconf_ets/2, sconf_ssl/2, sconf_authdirs/2, sconf_partial_post_size/2,\n         sconf_appmods/2, sconf_expires/2, sconf_errormod_401/2,\n         sconf_errormod_404/2, sconf_arg_rewrite_mode/2, sconf_logger_mod/2,\n         sconf_opaque/2, sconf_start_mod/2, sconf_allowed_scripts/2,\n         sconf_tilde_allowed_scripts/2, sconf_index_files/2, sconf_revproxy/2,\n         sconf_spotions/2, sconf_extra_cgi_vars/2, sconf_stats/2,\n         sconf_fcgi_app_server/2, sconf_php_handler/2, sconf_shaper/2,\n         sconf_deflate_options/2, sconf_mime_types_info/2,\n         sconf_dispatch_mod/2]).\n\n-export([new_auth/0,\n         auth_dir/1, auth_dir/2,\n         auth_docroot/1, auth_docroot/2,\n         auth_files/1, auth_files/2,\n         auth_realm/1, auth_realm/2,\n         auth_type/1, auth_type/2,\n         auth_headers/1, auth_headers/2,\n         auth_users/1, auth_users/2,\n         auth_acl/1, auth_acl/2,\n         auth_mod/1, auth_mod/2,\n         auth_outmod/1, auth_outmod/2,\n         auth_pam/1, auth_pam/2]).\n\n-export([new_ssl/0,\n         ssl_keyfile/1, ssl_keyfile/2,\n         ssl_certfile/1, ssl_certfile/2,\n         ssl_verify/1, ssl_verify/2,\n         ssl_fail_if_no_peer_cert/1, ssl_fail_if_no_peer_cert/2,\n         ssl_depth/1, ssl_depth/2,\n         ssl_password/1, ssl_password/2,\n         ssl_cacertfile/1, ssl_cacertfile/2,\n         ssl_dhfile/1, ssl_dhfile/2,\n         ssl_ciphers/1, ssl_ciphers/2,\n         ssl_cachetimeout/1, ssl_cachetimeout/2,\n         ssl_secure_renegotiate/1, ssl_secure_renegotiate/2,\n         ssl_client_renegotiation/1, ssl_client_renegotiation/2,\n         ssl_protocol_version/1, ssl_protocol_version/2,\n         ssl_honor_cipher_order/1, ssl_honor_cipher_order/2,\n         ssl_require_sni/1, ssl_require_sni/2]).\n\n-export([new_deflate/0,\n         deflate_min_compress_size/1, deflate_min_compress_size/2,\n         deflate_compression_level/1, deflate_compression_level/2,\n         deflate_window_size/1, deflate_window_size/2,\n         deflate_mem_level/1, deflate_mem_level/2,\n         deflate_strategy/1, deflate_strategy/2,\n         deflate_use_gzip_static/1, deflate_use_gzip_static/2,\n         deflate_mime_types/1, deflate_mime_types/2]).\n\n-export([new_mime_types_info/0,\n         mime_types_info_mime_types_file/1, mime_types_info_mime_types_file/2,\n         mime_types_info_types/1, mime_types_info_types/2,\n         mime_types_info_charsets/1, mime_types_info_charsets/2,\n         mime_types_info_default_type/1, mime_types_info_default_type/2,\n         mime_types_info_default_charset/1, mime_types_info_default_charset/2]).\n\n-export([first/2, elog/2, filesize/1, upto/2, to_string/1, to_list/1,\n         integer_to_hex/1, hex_to_integer/1, string_to_hex/1, hex_to_string/1,\n         is_modified_p/2, flag/3, dohup/1, is_ssl/1, address/0, is_space/1,\n         setopts/3, eat_crnl/2, get_chunk_num/2, get_chunk_header/2,\n         get_chunk/4, get_chunk_trailer/2, list_to_uue/1, uue_to_list/1,\n         printversion/0, strip_spaces/1, strip_spaces/2,\n         month/1, mk2/1, home/0, arg_rewrite/1, to_lowerchar/1, to_lower/1,\n         funreverse/2, is_prefix/2, split_sep/2, join_sep/2, accepts_gzip/2,\n         upto_char/2, deepmap/2, ticker/2, ticker/3, unique_triple/0, get_time_tuple/0,\n         parse_qvalue/1, parse_auth/1]).\n\n-export([outh_set_status_code/1,\n         outh_set_non_cacheable/1,\n         outh_set_content_type/1,\n         outh_set_content_encoding/1,\n         outh_set_cookie/1,\n         outh_set_static_headers/3, outh_set_static_headers/4,\n         outh_set_304_headers/3,\n         outh_set_dyn_headers/3,\n         outh_set_connection/1,\n         outh_set_content_length/1,\n         outh_set_dcc/2,\n         outh_set_transfer_encoding_off/0,\n         outh_set_auth/1,\n         outh_set_vary/1,\n         outh_clear_headers/0,\n         outh_fix_doclose/0,\n         dcc/2]).\n\n-export([make_allow_header/0, make_allow_header/1,\n         make_server_header/0,\n         make_last_modified_header/1,\n         make_location_header/1,\n         make_etag_header/1,\n         make_content_range_header/1,\n         make_content_length_header/1,\n         make_content_encoding_header/1,\n         make_connection_close_header/1,\n         make_transfer_encoding_chunked_header/1,\n         make_www_authenticate_header/1,\n         make_etag/1,\n         make_content_type_header/1,\n         make_date_header/0,\n         make_vary_header/1]).\n\n-export([outh_get_status_code/0,\n         outh_get_contlen/0,\n         outh_get_act_contlen/0,\n         outh_inc_act_contlen/1,\n         outh_get_doclose/0,\n         outh_get_chunked/0,\n         outh_get_content_encoding/0,\n         outh_get_content_encoding_header/0,\n         outh_get_content_type/0,\n         outh_get_vary_fields/0,\n         outh_serialize/0]).\n\n-export([accumulate_header/1, headers_to_str/1,\n         getuid/0,\n         user_to_home/1,\n         uid_to_name/1,\n         exists/1,\n         mkdir/1]).\n\n-export([tcp_connect/3, tcp_connect/4, ssl_connect/3, ssl_connect/4]).\n\n-export([do_recv/3, do_recv/4, cli_recv/3,\n         gen_tcp_send/2,\n         http_get_headers/2]).\n\n-export([sconf_to_srvstr/1,\n         redirect_host/2, redirect_port/1,\n         redirect_scheme_port/1, redirect_scheme/1,\n         tmpdir/0, tmpdir/1, mktemp/1, split_at/2, insert_at/3,\n         id_dir/1, ctl_file/1]).\n\n-export([parse_ipmask/1, match_ipmask/2, find_private_port/0]).\n\n-export([get_app_dir/0, get_ebin_dir/0, get_priv_dir/0,\n         get_inc_dir/0]).\n\n%% Internal\n-export([local_time_as_gmt_string/1, universal_time_as_string/1,\n         stringdate_to_datetime/1]).\n\nstart() ->\n    ok = start_app_deps(),\n    application:start(yaws, permanent).\n\nstop() ->\n    application:stop(yaws).\n\n\n%%% Quick and easy way of starting Yaws in embedded mode.  No need for any\n%%% start-script switches and no dependencies to Yaws header files. Just call\n%%% start_embedded/N and you are in the air.\nstart_embedded(DocRoot) ->\n    start_embedded(DocRoot, []).\n\nstart_embedded(DocRoot, SL) when is_list(DocRoot),is_list(SL) ->\n    start_embedded(DocRoot, SL, []).\n\nstart_embedded(DocRoot, SL, GL) when is_list(DocRoot),is_list(SL),is_list(GL) ->\n    start_embedded(DocRoot, SL, GL, \"default\").\nstart_embedded(DocRoot, SL, GL, Id)\n  when is_list(DocRoot), is_list(SL), is_list(GL) ->\n    ok = start_app_deps(),\n    {ok, SCList, GC, _} = yaws_api:embedded_start_conf(DocRoot, SL, GL, Id),\n    ok = application:start(yaws, permanent),\n    yaws_config:add_yaws_soap_srv(GC),\n    yaws_api:setconf(GC, SCList),\n    ok.\n\nadd_server(DocRoot, SL) when is_list(DocRoot),is_list(SL) ->\n    SC  = create_sconf(DocRoot, SL),\n    %% Change #auth in authdirs to {Dir, #auth} if needed\n    Fun = fun\n              (A = #auth{dir = [Dir]}, Acc) -> [{Dir, A}| Acc];\n              (A, Acc)                      -> [A| Acc]\n          end,\n    Authdirs = lists:foldr(Fun, [], SC#sconf.authdirs),\n    SC1 = yaws_config:add_yaws_auth(SC#sconf{authdirs = Authdirs}),\n    yaws_config:add_sconf(SC1).\n\ncreate_gconf(GL, Id) when is_list(GL) ->\n    Debug = case application:get_env(yaws, debug) of\n                undefined -> false;\n                {ok, D}   -> D\n            end,\n    setup_gconf(GL, yaws_config:make_default_gconf(Debug, Id)).\n\ncreate_sconf(DocRoot, SL) when is_list(DocRoot), is_list(SL) ->\n    SC = yaws_config:make_default_sconf(DocRoot,\n                                        lkup(servername, SL, undefined),\n                                        lkup(port, SL, undefined)),\n    SL1 = lists:keydelete(port, 1, lists:keydelete(servername, 1, SL)),\n    setup_sconf(SL1, SC).\n\nstart_app_deps() ->\n    Deps = split_sep(?YAWS_APPDEPS, $,),\n    catch lists:foldl(fun(App0, Acc) ->\n                              App = list_to_existing_atom(App0),\n                              case application:start(App, permanent) of\n                                  ok -> Acc;\n                                  {error,{already_started,App}} -> Acc;\n                                  Else -> throw(Else)\n                              end\n                      end, ok, Deps).\n\n%%% Access functions for the GCONF and SCONF records.\n%% Getters\ngconf_yaws_dir             (#gconf{yaws_dir              = X}) -> X.\ngconf_trace                (#gconf{trace                 = X}) -> X.\ngconf_flags                (#gconf{flags                 = X}) -> X.\ngconf_logdir               (#gconf{logdir                = X}) -> X.\ngconf_ebin_dir             (#gconf{ebin_dir              = X}) -> X.\ngconf_src_dir              (#gconf{src_dir               = X}) -> X.\ngconf_runmods              (#gconf{runmods               = X}) -> X.\ngconf_keepalive_timeout    (#gconf{keepalive_timeout     = X}) -> X.\ngconf_keepalive_maxuses    (#gconf{keepalive_maxuses     = X}) -> X.\ngconf_max_num_cached_files (#gconf{max_num_cached_files  = X}) -> X.\ngconf_max_num_cached_bytes (#gconf{max_num_cached_bytes  = X}) -> X.\ngconf_max_size_cached_file (#gconf{max_size_cached_file  = X}) -> X.\ngconf_max_connections      (#gconf{max_connections       = X}) -> X.\ngconf_process_options      (#gconf{process_options       = X}) -> X.\ngconf_large_file_chunk_size(#gconf{large_file_chunk_size = X}) -> X.\ngconf_mnesia_dir           (#gconf{mnesia_dir            = X}) -> X.\ngconf_log_wrap_size        (#gconf{log_wrap_size         = X}) -> X.\ngconf_cache_refresh_secs   (#gconf{cache_refresh_secs    = X}) -> X.\ngconf_include_dir          (#gconf{include_dir           = X}) -> X.\ngconf_phpexe               (#gconf{phpexe                = X}) -> X.\ngconf_yaws                 (#gconf{yaws                  = X}) -> X.\ngconf_id                   (#gconf{id                    = X}) -> X.\ngconf_enable_soap          (#gconf{enable_soap           = X}) -> X.\ngconf_soap_srv_mods        (#gconf{soap_srv_mods         = X}) -> X.\ngconf_ysession_mod         (#gconf{ysession_mod          = X}) -> X.\ngconf_acceptor_pool_size   (#gconf{acceptor_pool_size    = X}) -> X.\ngconf_mime_types_info      (#gconf{mime_types_info       = X}) -> X.\ngconf_nslookup_pref        (#gconf{nslookup_pref         = X}) -> X.\ngconf_ysession_idle_timeout(#gconf{ysession_idle_timeout = X}) -> X.\ngconf_ysession_long_timeout(#gconf{ysession_long_timeout = X}) -> X.\ngconf_sni                  (#gconf{sni                   = X}) -> X.\n\n%% Setters\ngconf_yaws_dir             (S, X) -> S#gconf{yaws_dir              = X}.\ngconf_trace                (S, X) -> S#gconf{trace                 = X}.\ngconf_flags                (S, X) -> S#gconf{flags                 = X}.\ngconf_logdir               (S, X) -> S#gconf{logdir                = X}.\ngconf_ebin_dir             (S, X) -> S#gconf{ebin_dir              = X}.\ngconf_src_dir              (S, X) -> S#gconf{src_dir               = X}.\ngconf_runmods              (S, X) -> S#gconf{runmods               = X}.\ngconf_keepalive_timeout    (S, X) -> S#gconf{keepalive_timeout     = X}.\ngconf_keepalive_maxuses    (S, X) -> S#gconf{keepalive_maxuses     = X}.\ngconf_max_num_cached_files (S, X) -> S#gconf{max_num_cached_files  = X}.\ngconf_max_num_cached_bytes (S, X) -> S#gconf{max_num_cached_bytes  = X}.\ngconf_max_size_cached_file (S, X) -> S#gconf{max_size_cached_file  = X}.\ngconf_max_connections      (S, X) -> S#gconf{max_connections       = X}.\ngconf_process_options      (S, X) -> S#gconf{process_options       = X}.\ngconf_large_file_chunk_size(S, X) -> S#gconf{large_file_chunk_size = X}.\ngconf_mnesia_dir           (S, X) -> S#gconf{mnesia_dir            = X}.\ngconf_log_wrap_size        (S, X) -> S#gconf{log_wrap_size         = X}.\ngconf_cache_refresh_secs   (S, X) -> S#gconf{cache_refresh_secs    = X}.\ngconf_include_dir          (S, X) -> S#gconf{include_dir           = X}.\ngconf_phpexe               (S, X) -> S#gconf{phpexe                = X}.\ngconf_yaws                 (S, X) -> S#gconf{yaws                  = X}.\ngconf_id                   (S, X) -> S#gconf{id                    = X}.\ngconf_enable_soap          (S, X) -> S#gconf{enable_soap           = X}.\ngconf_soap_srv_mods        (S, X) -> S#gconf{soap_srv_mods         = X}.\ngconf_ysession_mod         (S, X) -> S#gconf{ysession_mod          = X}.\ngconf_acceptor_pool_size   (S, X) -> S#gconf{acceptor_pool_size    = X}.\ngconf_mime_types_info      (S, X) -> S#gconf{mime_types_info       = X}.\ngconf_nslookup_pref        (S, X) -> S#gconf{nslookup_pref         = X}.\ngconf_ysession_idle_timeout(S, X) -> S#gconf{ysession_idle_timeout = X}.\ngconf_ysession_long_timeout(S, X) -> S#gconf{ysession_long_timeout = X}.\ngconf_sni                  (S, X) -> S#gconf{sni                   = X}.\n\n%% Getters\nsconf_port                 (#sconf{port                  = X}) -> X.\nsconf_flags                (#sconf{flags                 = X}) -> X.\nsconf_redirect_map         (#sconf{redirect_map          = X}) -> X.\nsconf_rhost                (#sconf{rhost                 = X}) -> X.\nsconf_rmethod              (#sconf{rmethod               = X}) -> X.\nsconf_docroot              (#sconf{docroot               = X}) -> X.\nsconf_xtra_docroots        (#sconf{xtra_docroots         = X}) -> X.\nsconf_listen               (#sconf{listen                = X}) -> X.\nsconf_servername           (#sconf{servername            = X}) -> X.\nsconf_serveralias          (#sconf{serveralias           = X}) -> X.\nsconf_yaws                 (#sconf{yaws                  = X}) -> X.\nsconf_ets                  (#sconf{ets                   = X}) -> X.\nsconf_ssl                  (#sconf{ssl                   = X}) -> X.\nsconf_authdirs             (#sconf{authdirs              = X}) -> X.\nsconf_partial_post_size    (#sconf{partial_post_size     = X}) -> X.\nsconf_appmods              (#sconf{appmods               = X}) -> X.\nsconf_expires              (#sconf{expires               = X}) -> X.\nsconf_errormod_401         (#sconf{errormod_401          = X}) -> X.\nsconf_errormod_404         (#sconf{errormod_404          = X}) -> X.\nsconf_arg_rewrite_mode     (#sconf{arg_rewrite_mod       = X}) -> X.\nsconf_logger_mod           (#sconf{logger_mod            = X}) -> X.\nsconf_opaque               (#sconf{opaque                = X}) -> X.\nsconf_start_mod            (#sconf{start_mod             = X}) -> X.\nsconf_allowed_scripts      (#sconf{allowed_scripts       = X}) -> X.\nsconf_tilde_allowed_scripts(#sconf{tilde_allowed_scripts = X}) -> X.\nsconf_index_files          (#sconf{index_files           = X}) -> X.\nsconf_revproxy             (#sconf{revproxy              = X}) -> X.\nsconf_spotions             (#sconf{soptions              = X}) -> X.\nsconf_extra_cgi_vars       (#sconf{extra_cgi_vars        = X}) -> X.\nsconf_stats                (#sconf{stats                 = X}) -> X.\nsconf_fcgi_app_server      (#sconf{fcgi_app_server       = X}) -> X.\nsconf_php_handler          (#sconf{php_handler           = X}) -> X.\nsconf_shaper               (#sconf{shaper                = X}) -> X.\nsconf_deflate_options      (#sconf{deflate_options       = X}) -> X.\nsconf_mime_types_info      (#sconf{mime_types_info       = X}) -> X.\nsconf_dispatch_mod         (#sconf{dispatch_mod          = X}) -> X.\n\n%% Setters\nsconf_port                 (S, X) -> S#sconf{port                  = X}.\nsconf_flags                (S, X) -> S#sconf{flags                 = X}.\nsconf_redirect_map         (S, X) -> S#sconf{redirect_map          = X}.\nsconf_rhost                (S, X) -> S#sconf{rhost                 = X}.\nsconf_rmethod              (S, X) -> S#sconf{rmethod               = X}.\nsconf_docroot              (S, X) -> S#sconf{docroot               = X}.\nsconf_xtra_docroots        (S, X) -> S#sconf{xtra_docroots         = X}.\nsconf_listen               (S, X) -> S#sconf{listen                = X}.\nsconf_servername           (S, X) -> S#sconf{servername            = X}.\nsconf_serveralias          (S, X) -> S#sconf{serveralias           = X}.\nsconf_yaws                 (S, X) -> S#sconf{yaws                  = X}.\nsconf_ets                  (S, X) -> S#sconf{ets                   = X}.\nsconf_ssl                  (S, X) -> S#sconf{ssl                   = X}.\nsconf_authdirs             (S, X) -> S#sconf{authdirs              = X}.\nsconf_partial_post_size    (S, X) -> S#sconf{partial_post_size     = X}.\nsconf_appmods              (S, X) -> S#sconf{appmods               = X}.\nsconf_expires              (S, X) -> S#sconf{expires               = X}.\nsconf_errormod_401         (S, X) -> S#sconf{errormod_401          = X}.\nsconf_errormod_404         (S, X) -> S#sconf{errormod_404          = X}.\nsconf_arg_rewrite_mode     (S, X) -> S#sconf{arg_rewrite_mod       = X}.\nsconf_logger_mod           (S, X) -> S#sconf{logger_mod            = X}.\nsconf_opaque               (S, X) -> S#sconf{opaque                = X}.\nsconf_start_mod            (S, X) -> S#sconf{start_mod             = X}.\nsconf_allowed_scripts      (S, X) -> S#sconf{allowed_scripts       = X}.\nsconf_tilde_allowed_scripts(S, X) -> S#sconf{tilde_allowed_scripts = X}.\nsconf_index_files          (S, X) -> S#sconf{index_files           = X}.\nsconf_revproxy             (S, X) -> S#sconf{revproxy              = X}.\nsconf_spotions             (S, X) -> S#sconf{soptions              = X}.\nsconf_extra_cgi_vars       (S, X) -> S#sconf{extra_cgi_vars        = X}.\nsconf_stats                (S, X) -> S#sconf{stats                 = X}.\nsconf_fcgi_app_server      (S, X) -> S#sconf{fcgi_app_server       = X}.\nsconf_php_handler          (S, X) -> S#sconf{php_handler           = X}.\nsconf_shaper               (S, X) -> S#sconf{shaper                = X}.\nsconf_deflate_options      (S, X) -> S#sconf{deflate_options       = X}.\nsconf_mime_types_info      (S, X) -> S#sconf{mime_types_info       = X}.\nsconf_dispatch_mod         (S, X) -> S#sconf{dispatch_mod          = X}.\n\n\n%% Access functions for the AUTH record.\nnew_auth() -> #auth{}.\n\nauth_dir    (#auth{dir     = X}) -> X.\nauth_docroot(#auth{docroot = X}) -> X.\nauth_files  (#auth{files   = X}) -> X.\nauth_realm  (#auth{realm   = X}) -> X.\nauth_type   (#auth{type    = X}) -> X.\nauth_headers(#auth{headers = X}) -> X.\nauth_users  (#auth{users   = X}) -> X.\nauth_acl    (#auth{acl     = X}) -> X.\nauth_mod    (#auth{mod     = X}) -> X.\nauth_outmod (#auth{outmod  = X}) -> X.\nauth_pam    (#auth{pam     = X}) -> X.\n\nauth_dir    (A, Dir)     -> A#auth{dir     = Dir}.\nauth_docroot(A, DocRoot) -> A#auth{docroot = DocRoot}.\nauth_files  (A, Files)   -> A#auth{files   = Files}.\nauth_realm  (A, Realm)   -> A#auth{realm   = Realm}.\nauth_type   (A, Type)    -> A#auth{type    = Type}.\nauth_headers(A, Headers) -> A#auth{headers = Headers}.\nauth_users  (A, Users)   -> A#auth{users   = Users}.\nauth_acl    (A, Acl)     -> A#auth{acl     = Acl}.\nauth_mod    (A, Mod)     -> A#auth{mod     = Mod}.\nauth_outmod (A, Outmod)  -> A#auth{outmod  = Outmod}.\nauth_pam    (A, Pam)     -> A#auth{pam     = Pam}.\n\n\nsetup_authdirs(SL, DefaultAuthDirs) ->\n    case [A || {auth, A} <- SL] of\n        [] -> DefaultAuthDirs;\n        As -> [setup_auth(A) || A <- As]\n    end.\n\nsetup_auth(#auth{}=Auth) ->\n    Auth;\nsetup_auth(AuthProps) ->\n    Auth = #auth{},\n    #auth{dir     = lkup(dir,     AuthProps, Auth#auth.dir),\n          docroot = lkup(docroot, AuthProps, Auth#auth.docroot),\n          files   = lkup(files,   AuthProps, Auth#auth.files),\n          realm   = lkup(realm,   AuthProps, Auth#auth.realm),\n          type    = lkup(type,    AuthProps, Auth#auth.type),\n          headers = lkup(headers, AuthProps, Auth#auth.headers),\n          users   = lkup(users,   AuthProps, Auth#auth.users),\n          acl     = lkup(acl,     AuthProps, Auth#auth.acl),\n          mod     = lkup(mod,     AuthProps, Auth#auth.mod),\n          outmod  = lkup(outmod,  AuthProps, Auth#auth.outmod),\n          pam     = lkup(pam,     AuthProps, Auth#auth.pam)}.\n\n\n%% Access functions for the SSL record.\nnew_ssl() -> #ssl{}.\n\nssl_keyfile             (#ssl{keyfile              = X}) -> X.\nssl_certfile            (#ssl{certfile             = X}) -> X.\nssl_verify              (#ssl{verify               = X}) -> X.\nssl_fail_if_no_peer_cert(#ssl{fail_if_no_peer_cert = X}) -> X.\nssl_depth               (#ssl{depth                = X}) -> X.\nssl_password            (#ssl{password             = X}) -> X.\nssl_cacertfile          (#ssl{cacertfile           = X}) -> X.\nssl_dhfile              (#ssl{dhfile               = X}) -> X.\nssl_ciphers             (#ssl{ciphers              = X}) -> X.\nssl_cachetimeout        (#ssl{cachetimeout         = X}) -> X.\nssl_secure_renegotiate  (#ssl{secure_renegotiate   = X}) -> X.\nssl_client_renegotiation(#ssl{client_renegotiation = X}) -> X.\nssl_protocol_version    (#ssl{protocol_version     = X}) -> X.\nssl_honor_cipher_order  (#ssl{honor_cipher_order   = X}) -> X.\nssl_require_sni         (#ssl{require_sni          = X}) -> X.\n\nssl_keyfile             (S, File)    -> S#ssl{keyfile              = File}.\nssl_certfile            (S, File)    -> S#ssl{certfile             = File}.\nssl_verify              (S, Verify)  -> S#ssl{verify               = Verify}.\nssl_fail_if_no_peer_cert(S, Bool)    -> S#ssl{fail_if_no_peer_cert = Bool}.\nssl_depth               (S, Depth)   -> S#ssl{depth                = Depth}.\nssl_password            (S, Pass)    -> S#ssl{password             = Pass}.\nssl_cacertfile          (S, File)    -> S#ssl{cacertfile           = File}.\nssl_dhfile              (S, File)    -> S#ssl{dhfile               = File}.\nssl_ciphers             (S, Ciphers) -> S#ssl{ciphers              = Ciphers}.\nssl_cachetimeout        (S, Timeout) -> S#ssl{cachetimeout         = Timeout}.\nssl_secure_renegotiate  (S, Bool)    -> S#ssl{secure_renegotiate   = Bool}.\nssl_protocol_version    (S, Vsns)    -> S#ssl{protocol_version     = Vsns}.\nssl_require_sni         (S, Bool)    -> S#ssl{require_sni          = Bool}.\nssl_honor_cipher_order  (S, Bool) ->\n    case yaws_dynopts:have_ssl_honor_cipher_order() of\n        true  -> S#ssl{honor_cipher_order   = Bool};\n        false -> S\n    end.\nssl_client_renegotiation(S, Bool) ->\n    case yaws_dynopts:have_ssl_client_renegotiation() of\n        true  -> S#ssl{client_renegotiation = Bool};\n        false -> S\n    end.\n\nsetup_ssl(SL, DefaultSSL) ->\n    case lkup(ssl, SL, undefined) of\n        undefined ->\n            DefaultSSL;\n        SSL when is_record(SSL, ssl) ->\n            SSL;\n        SSLProps when is_list(SSLProps) ->\n            SSL = #ssl{},\n            #ssl{keyfile              = lkup(keyfile, SSLProps,\n                                             SSL#ssl.keyfile),\n                 certfile             = lkup(certfile, SSLProps,\n                                             SSL#ssl.certfile),\n                 verify               = lkup(verify, SSLProps, SSL#ssl.verify),\n                 fail_if_no_peer_cert = lkup(fail_if_no_peer_cert, SSLProps,\n                                             SSL#ssl.fail_if_no_peer_cert),\n                 depth                = lkup(depth, SSLProps, SSL#ssl.depth),\n                 password             = lkup(password, SSLProps,\n                                             SSL#ssl.password),\n                 cacertfile           = lkup(cacertfile, SSLProps,\n                                             SSL#ssl.cacertfile),\n                 dhfile               = lkup(dhfile, SSLProps,\n                                             SSL#ssl.dhfile),\n                 ciphers              = lkup(ciphers, SSLProps,\n                                             SSL#ssl.ciphers),\n                 cachetimeout         = lkup(cachetimeout, SSLProps,\n                                             SSL#ssl.cachetimeout),\n                 secure_renegotiate   = lkup(secure_renegotiate, SSLProps,\n                                             SSL#ssl.secure_renegotiate),\n                 client_renegotiation = lkup(client_renegotiation, SSLProps,\n                                             SSL#ssl.client_renegotiation),\n                 honor_cipher_order   = lkup(honor_cipher_order, SSLProps,\n                                             SSL#ssl.honor_cipher_order),\n                 protocol_version     = lkup(protocol_version, SSLProps,\n                                             undefined),\n                 require_sni          = lkup(require_sni, SSLProps,\n                                             SSL#ssl.require_sni)}\n    end.\n\n\n%% Access functions for the DEFLATE record.\nnew_deflate() -> #deflate{}.\n\ndeflate_min_compress_size(#deflate{min_compress_size = X}) -> X.\ndeflate_compression_level(#deflate{compression_level = X}) -> X.\ndeflate_window_size      (#deflate{window_size       = X}) -> X.\ndeflate_mem_level        (#deflate{mem_level         = X}) -> X.\ndeflate_strategy         (#deflate{strategy          = X}) -> X.\ndeflate_use_gzip_static  (#deflate{use_gzip_static   = X}) -> X.\ndeflate_mime_types       (#deflate{mime_types        = X}) -> X.\n\ndeflate_min_compress_size(D, Min)   -> D#deflate{min_compress_size = Min}.\ndeflate_compression_level(D, Level) -> D#deflate{compression_level = Level}.\ndeflate_window_size      (D, Size)  -> D#deflate{window_size       = Size}.\ndeflate_mem_level        (D, Level) -> D#deflate{mem_level         = Level}.\ndeflate_strategy         (D, Strat) -> D#deflate{strategy          = Strat}.\ndeflate_use_gzip_static  (D, Bool)  -> D#deflate{use_gzip_static   = Bool}.\ndeflate_mime_types       (D, Types) -> D#deflate{mime_types        = Types}.\n\n\nsetup_deflate(SL, DefaultDeflate) ->\n    case lkup(deflate_options, SL, undefined) of\n        undefined ->\n            DefaultDeflate;\n        D when is_record(D, deflate) ->\n            D;\n        DProps when is_list(DProps) ->\n            D = #deflate{},\n            #deflate{min_compress_size = lkup(min_compress_size, DProps,\n                                              D#deflate.min_compress_size),\n                     compression_level = lkup(compression_level, DProps,\n                                              D#deflate.compression_level),\n                     window_size       = lkup(window_size, DProps,\n                                              D#deflate.window_size),\n                     mem_level         = lkup(mem_level, DProps,\n                                              D#deflate.mem_level),\n                     strategy          = lkup(strategy, DProps,\n                                              D#deflate.strategy),\n                     use_gzip_static   = lkup(use_gzip_static, DProps,\n                                              D#deflate.use_gzip_static),\n                     mime_types        = lkup(mime_types, DProps,\n                                              D#deflate.mime_types)}\n    end.\n\n%% Access functions to MIME_TYPES_INFO record.\nnew_mime_types_info() -> #mime_types_info{}.\n\nmime_types_info_mime_types_file(#mime_types_info{mime_types_file = X}) -> X.\nmime_types_info_types          (#mime_types_info{types           = X}) -> X.\nmime_types_info_charsets       (#mime_types_info{charsets        = X}) -> X.\nmime_types_info_default_type   (#mime_types_info{default_type    = X}) -> X.\nmime_types_info_default_charset(#mime_types_info{default_charset = X}) -> X.\n\nmime_types_info_mime_types_file(M, File) ->\n    M#mime_types_info{mime_types_file = File}.\nmime_types_info_types(M, Types) ->\n    M#mime_types_info{types = Types}.\nmime_types_info_charsets(M, Charsets) ->\n    M#mime_types_info{charsets = Charsets}.\nmime_types_info_default_type(M, Type) ->\n    M#mime_types_info{default_type = Type}.\nmime_types_info_default_charset(M, Charset) ->\n    M#mime_types_info{default_charset = Charset}.\n\n\nsetup_mime_types_info(SL, DefaultMTI) ->\n    case lkup(mime_types_info, SL, undefined) of\n        undefined ->\n            DefaultMTI;\n        M when is_record(M, mime_types_info) ->\n            M;\n        MProps when is_list(MProps) ->\n            M = #mime_types_info{},\n            #mime_types_info{mime_types_file =\n                                 lkup(mime_types_file, MProps,\n                                      M#mime_types_info.mime_types_file),\n                             types           = lkup(types, MProps,\n                                                    M#mime_types_info.types),\n                             charsets        = lkup(charsets, MProps,\n                                                    M#mime_types_info.charsets),\n                             default_type    =\n                                 lkup(default_type, MProps,\n                                      M#mime_types_info.default_type),\n                             default_charset =\n                                 lkup(default_charset, MProps,\n                                      M#mime_types_info.default_charset)}\n    end.\n\n\n%% Setup global configuration\nsetup_gconf([], GC) -> GC;\nsetup_gconf(GL, GC) ->\n    #gconf{yaws_dir              = lkup(yaws_dir, GL, GC#gconf.yaws_dir),\n           trace                 = lkup(trace, GL, GC#gconf.trace),\n           flags                 = set_gc_flags(lkup(flags, GL, []),\n                                                GC#gconf.flags),\n           logdir                = lkup(logdir, GL, GC#gconf.logdir),\n           ebin_dir              = lkup(ebin_dir, GL, GC#gconf.ebin_dir),\n           src_dir               = lkup(src_dir, GL, GC#gconf.src_dir),\n           runmods               = lkup(runmods, GL, GC#gconf.runmods),\n           keepalive_timeout     = lkup(keepalive_timeout, GL,\n                                        GC#gconf.keepalive_timeout),\n           keepalive_maxuses     = lkup(keepalive_maxuses, GL,\n                                        GC#gconf.keepalive_maxuses),\n           max_num_cached_files  = lkup(max_num_cached_files, GL,\n                                        GC#gconf.max_num_cached_files),\n           max_num_cached_bytes  = lkup(max_num_cached_bytes, GL,\n                                        GC#gconf.max_num_cached_bytes),\n           max_size_cached_file  = lkup(max_size_cached_file, GL,\n                                        GC#gconf.max_size_cached_file),\n           max_connections       = lkup(max_connections, GL,\n                                        GC#gconf.max_connections),\n           process_options       = lkup(process_options, GL,\n                                        GC#gconf.process_options),\n           large_file_chunk_size = lkup(large_file_chunk_size, GL,\n                                        GC#gconf.large_file_chunk_size),\n           mnesia_dir            = lkup(mnesia_dir, GL, GC#gconf.mnesia_dir),\n           log_wrap_size         = lkup(log_wrap_size, GL,\n                                        GC#gconf.log_wrap_size),\n           cache_refresh_secs    = lkup(cache_refresh_secs, GL,\n                                        GC#gconf.cache_refresh_secs),\n           include_dir           = lkup(include_dir, GL, GC#gconf.include_dir),\n           phpexe                = lkup(phpexe, GL, GC#gconf.phpexe),\n           yaws                  = lkup(yaws, GL, GC#gconf.yaws),\n           id                    = lkup(id, GL, GC#gconf.id),\n           enable_soap           = lkup(enable_soap, GL, GC#gconf.enable_soap),\n           soap_srv_mods         = lkup(soap_srv_mods, GL,\n                                        GC#gconf.soap_srv_mods),\n           ysession_mod          = lkup(ysession_mod, GL,\n                                        GC#gconf.ysession_mod),\n           acceptor_pool_size    = lkup(acceptor_pool_size, GL,\n                                        GC#gconf.acceptor_pool_size),\n           mime_types_info       = setup_mime_types_info(\n                                     GL, GC#gconf.mime_types_info\n                                    ),\n           nslookup_pref         = lkup(nslookup_pref, GL,\n                                        GC#gconf.nslookup_pref),\n           sni                   = lkup(sni, GL, GC#gconf.sni)\n          }.\n\nset_gc_flags([{tty_trace, Bool}|T], Flags) ->\n    set_gc_flags(T, flag(Flags,?GC_TTY_TRACE, Bool));\nset_gc_flags([{debug, Bool}|T], Flags) ->\n    set_gc_flags(T, flag(Flags, ?GC_DEBUG, Bool));\nset_gc_flags([{copy_errlog, Bool}|T], Flags) ->\n    set_gc_flags(T, flag(Flags, ?GC_COPY_ERRLOG, Bool));\nset_gc_flags([{copy_error_log, Bool}|T], Flags) ->\n    set_gc_flags(T, flag(Flags, ?GC_COPY_ERRLOG, Bool));\nset_gc_flags([{backwards_compat_parse, Bool}|T], Flags) ->\n    set_gc_flags(T, flag(Flags, ?GC_BACKWARDS_COMPAT_PARSE, Bool));\nset_gc_flags([{log_resolve_hostname, Bool}|T], Flags) ->\n    set_gc_flags(T, flag(Flags, ?GC_LOG_RESOLVE_HOSTNAME, Bool));\nset_gc_flags([{fail_on_bind_err, Bool}|T], Flags) ->\n    set_gc_flags(T, flag(Flags,?GC_FAIL_ON_BIND_ERR,Bool));\nset_gc_flags([{pick_first_virthost_on_nomatch, Bool}|T], Flags) ->\n    set_gc_flags(T, flag(Flags, ?GC_PICK_FIRST_VIRTHOST_ON_NOMATCH,Bool));\nset_gc_flags([{use_erlang_sendfile, Bool}|T], Flags) ->\n    set_gc_flags(T, flag(Flags,?GC_USE_ERLANG_SENDFILE,Bool));\nset_gc_flags([{use_yaws_sendfile, Bool}|T], Flags) ->\n    set_gc_flags(T, flag(Flags,?GC_USE_YAWS_SENDFILE,Bool));\nset_gc_flags([_|T], Flags) ->\n    set_gc_flags(T, Flags);\nset_gc_flags([], Flags) ->\n    Flags.\n\n\n%% Setup vhost configuration\nsetup_sconf(SL, SC) ->\n    #sconf{port                  = lkup(port, SL, SC#sconf.port),\n           flags                 = set_sc_flags(lkup(flags, SL, []),\n                                                SC#sconf.flags),\n           redirect_map          = lkup(redirect_map, SL,\n                                        SC#sconf.redirect_map),\n           rhost                 = lkup(rhost, SL, SC#sconf.rhost),\n           rmethod               = lkup(rmethod, SL, SC#sconf.rmethod),\n           docroot               = lkup(docroot, SL, SC#sconf.docroot),\n           xtra_docroots         = lkup(xtra_docroots, SL,\n                                        SC#sconf.xtra_docroots),\n           listen                = lkup(listen, SL, SC#sconf.listen),\n           servername            = lkup(servername, SL, SC#sconf.servername),\n           serveralias           = lkup(serveralias, SL, SC#sconf.serveralias),\n           yaws                  = lkup(yaws, SL, SC#sconf.yaws),\n           ets                   = lkup(ets, SL, SC#sconf.ets),\n           ssl                   = setup_ssl(SL, SC#sconf.ssl),\n           authdirs              = setup_authdirs(SL, SC#sconf.authdirs),\n           partial_post_size     = lkup(partial_post_size, SL,\n                                        SC#sconf.partial_post_size),\n           appmods               = lkup(appmods, SL, SC#sconf.appmods),\n           expires               = lkup(expires, SL, SC#sconf.expires),\n           errormod_401          = lkup(errormod_401, SL,\n                                        SC#sconf.errormod_401),\n           errormod_404          = lkup(errormod_404, SL,\n                                        SC#sconf.errormod_404),\n           errormod_crash        = lkup(errormod_crash, SL,\n                                        SC#sconf.errormod_crash),\n           arg_rewrite_mod       = lkup(arg_rewrite_mod, SL,\n                                        SC#sconf.arg_rewrite_mod),\n           logger_mod            = lkup(logger_mod, SL, SC#sconf.logger_mod),\n           opaque                = lkup(opaque, SL, SC#sconf.opaque),\n           start_mod             = lkup(start_mod, SL, SC#sconf.start_mod),\n           allowed_scripts       = lkup(allowed_scripts, SL,\n                                        SC#sconf.allowed_scripts),\n           tilde_allowed_scripts = lkup(tilde_allowed_scripts, SL,\n                                        SC#sconf.tilde_allowed_scripts),\n           index_files           = lkup(index_files, SL, SC#sconf.index_files),\n           revproxy              = lkup(revproxy, SL, SC#sconf.revproxy),\n           soptions              = lkup(soptions, SL, SC#sconf.soptions),\n           extra_cgi_vars        = lkup(extra_cgi_vars, SL,\n                                        SC#sconf.extra_cgi_vars),\n           stats                 = lkup(stats, SL, SC#sconf.stats),\n           fcgi_app_server       = lkup(fcgi_app_server, SL,\n                                        SC#sconf.fcgi_app_server),\n           php_handler           = lkup(php_handler, SL, SC#sconf.php_handler),\n           shaper                = lkup(shaper, SL, SC#sconf.shaper),\n           deflate_options       = setup_deflate(SL, SC#sconf.deflate_options),\n           mime_types_info       = setup_mime_types_info(\n                                     SL, SC#sconf.mime_types_info\n                                    ),\n           dispatch_mod          = lkup(dispatchmod, SL, SC#sconf.dispatch_mod)\n          }.\n\nset_sc_flags([{access_log, Bool}|T], Flags) ->\n    set_sc_flags(T, flag(Flags, ?SC_ACCESS_LOG, Bool));\nset_sc_flags([{auth_log, Bool}|T], Flags) ->\n    set_sc_flags(T, flag(Flags, ?SC_AUTH_LOG, Bool));\nset_sc_flags([{add_port, Bool}|T], Flags) ->\n    set_sc_flags(T, flag(Flags, ?SC_ADD_PORT, Bool));\nset_sc_flags([{statistics, Bool}|T], Flags) ->\n    set_sc_flags(T, flag(Flags, ?SC_STATISTICS, Bool));\nset_sc_flags([{tilde_expand, Bool}|T], Flags) ->\n    set_sc_flags(T, flag(Flags, ?SC_TILDE_EXPAND, Bool));\nset_sc_flags([{dir_listings, Bool}|T], Flags) ->\n    set_sc_flags(T, flag(Flags, ?SC_DIR_LISTINGS, Bool));\nset_sc_flags([{deflate, Bool}|T], Flags) ->\n    set_sc_flags(T, flag(Flags, ?SC_DEFLATE, Bool));\nset_sc_flags([{dir_all_zip, Bool}|T], Flags) ->\n    set_sc_flags(T, flag(Flags, ?SC_DIR_ALL_ZIP, Bool));\nset_sc_flags([{dav, Bool}|T], Flags) ->\n    set_sc_flags(T, flag(Flags, ?SC_DAV, Bool));\nset_sc_flags([{fcgi_trace_protocol, Bool}|T], Flags) ->\n    set_sc_flags(T, flag(Flags, ?SC_FCGI_TRACE_PROTOCOL, Bool));\nset_sc_flags([{fcgi_log_app_error, Bool}|T], Flags) ->\n    set_sc_flags(T, flag(Flags, ?SC_FCGI_LOG_APP_ERROR, Bool));\nset_sc_flags([{forward_proxy, Bool}|T], Flags) ->\n    set_sc_flags(T, flag(Flags, ?SC_FORWARD_PROXY, Bool));\nset_sc_flags([{auth_skip_docroot, Bool}|T], Flags) ->\n    set_sc_flags(T, flag(Flags, ?SC_AUTH_SKIP_DOCROOT, Bool));\nset_sc_flags([_Unknown|T], Flags) ->\n    error_logger:format(\"Unknown and unhandled flag ~p~n\", [_Unknown]),\n    set_sc_flags(T, Flags);\nset_sc_flags([], Flags) ->\n    Flags.\n\nlkup(Key, List, Def) ->\n    case lists:keyfind(Key, 1, List) of\n        {_,Value} -> Value;\n        _         -> Def\n    end.\n\n\n\nhup() ->\n    dohup(undefined).\n\nhup(Sock) ->\n    spawn(fun() ->\n                  group_leader(whereis(user), self()),\n                  dohup(Sock)\n          end).\n\ndohup(Sock) ->\n    Env = yaws_sup:get_app_args(),\n    Res = try yaws_config:load(Env) of\n              {ok, Gconf, Sconfs} -> yaws_api:setconf(Gconf, Sconfs);\n              Err                 -> Err\n          catch\n              _:X ->\n                  X\n          end,\n    gen_event:notify(yaws_event_manager, {yaws_hupped, Res}),\n    yaws_log:rotate(Res),\n    case Sock of\n        undefined ->\n            {yaws_hupped, Res};\n        _  ->\n            gen_tcp:send(Sock, io_lib:format(\"hupped: ~p~n\", [Res])),\n            gen_tcp:close(Sock)\n    end.\n\n\n\n%%% misc funcs\nfirst(_F, []) ->\n    false;\nfirst(F, [H|T]) ->\n    case F(H) of\n        {ok, Val} -> {ok, Val, H};\n        ok        -> {ok, ok, H};\n        _         -> first(F, T)\n    end.\n\n\nelog(F, As) ->\n    error_logger:format(F, As).\n\n\nfilesize(Fname) ->\n    case file:read_file_info(Fname) of\n        {ok, FI} when FI#file_info.type == regular ->\n            {ok, FI#file_info.size};\n        {ok, FI} ->\n            {error,  FI#file_info.type};\n        Err ->\n            Err\n    end.\n\n\nupto(_I, [])    -> [];\nupto(0,  _)     -> \" ....\";\nupto(_I, [0|_]) -> \" ....\";\nupto(I,  [H|T]) -> [H|upto(I-1, T)].\n\n\nto_string(X) when is_float(X)   -> io_lib:format(\"~.2.0f\",[X]);\nto_string(X) when is_integer(X) -> erlang:integer_to_list(X);\nto_string(X) when is_atom(X)    -> atom_to_list(X);\nto_string(X)                    -> lists:concat([X]).\n\n\nto_list(L) when is_list(L) -> L;\nto_list(A) when is_atom(A) -> atom_to_list(A).\n\n\ninteger_to_hex(I) ->\n    case catch erlang:integer_to_list(I, 16) of\n        {'EXIT', _} -> old_integer_to_hex(I);\n        Int         -> Int\n    end.\n\n\nold_integer_to_hex(I) when I < 10 ->\n    integer_to_list(I);\nold_integer_to_hex(I) when I < 16 ->\n    [I-10+$A];\nold_integer_to_hex(I) when I >= 16 ->\n    N = trunc(I/16),\n    old_integer_to_hex(N) ++ old_integer_to_hex(I rem 16).\n\n\nhex_to_integer(Hex) ->\n    erlang:list_to_integer(Hex, 16).\n\n\nstring_to_hex(String) ->\n    HEXC = fun (D) when D > 9 -> $a + D - 10;\n               (D)            -> $0 + D\n           end,\n    lists:foldr(fun(E, Acc) -> [HEXC(E div 16),HEXC(E rem 16)|Acc] end,\n                [], String).\n\n\nhex_to_string(Hex) ->\n    DEHEX = fun (H) when H >= $a -> H - $a + 10;\n                (H) when H >= $A -> H - $A + 10;\n                (H) ->              H - $0\n            end,\n    {String, _} =\n        lists:foldr(fun (E, {Acc, nolow}) -> {Acc, DEHEX(E)};\n                        (E, {Acc, LO})    -> {[DEHEX(E)*16+LO|Acc], nolow}\n                    end, {[], nolow}, Hex),\n    String.\n\n\n\nuniversal_time_as_string() ->\n    universal_time_as_string(calendar:universal_time()).\nuniversal_time_as_string(UTime) ->\n    time_to_string(UTime, \"GMT\").\nlocal_time_as_gmt_string(LocalTime) ->\n    time_to_string(erlang:localtime_to_universaltime(LocalTime), \"GMT\").\n\n\ntime_to_string({{Year, Month, Day}, {Hour, Min, Sec}}, Zone) ->\n    [day(Year, Month, Day), \", \",\n     mk2(Day), \" \", month(Month), \" \", erlang:integer_to_list(Year), \" \",\n     mk2(Hour), \":\", mk2(Min), \":\", mk2(Sec), \" \", Zone].\n\nmk2(I) when I < 10 -> [$0 | erlang:integer_to_list(I)];\nmk2(I)             -> erlang:integer_to_list(I).\n\nday(Year, Month, Day) ->\n    int_to_wd(calendar:day_of_the_week(Year, Month, Day)).\n\nint_to_wd(1) -> \"Mon\";\nint_to_wd(2) -> \"Tue\";\nint_to_wd(3) -> \"Wed\";\nint_to_wd(4) -> \"Thu\";\nint_to_wd(5) -> \"Fri\";\nint_to_wd(6) -> \"Sat\";\nint_to_wd(7) -> \"Sun\".\n\nmonth(1)  -> \"Jan\";\nmonth(2)  -> \"Feb\";\nmonth(3)  -> \"Mar\";\nmonth(4)  -> \"Apr\";\nmonth(5)  -> \"May\";\nmonth(6)  -> \"Jun\";\nmonth(7)  -> \"Jul\";\nmonth(8)  -> \"Aug\";\nmonth(9)  -> \"Sep\";\nmonth(10) -> \"Oct\";\nmonth(11) -> \"Nov\";\nmonth(12) -> \"Dec\".\n\nmonth_str_to_int(\"Jan\") -> 1;\nmonth_str_to_int(\"Feb\") -> 2;\nmonth_str_to_int(\"Mar\") -> 3;\nmonth_str_to_int(\"Apr\") -> 4;\nmonth_str_to_int(\"May\") -> 5;\nmonth_str_to_int(\"Jun\") -> 6;\nmonth_str_to_int(\"Jul\") -> 7;\nmonth_str_to_int(\"Aug\") -> 8;\nmonth_str_to_int(\"Sep\") -> 9;\nmonth_str_to_int(\"Oct\") -> 10;\nmonth_str_to_int(\"Nov\") -> 11;\nmonth_str_to_int(\"Dec\") -> 12.\n\n\nstringdate_to_datetime([$ |T]) ->\n    stringdate_to_datetime(T);\nstringdate_to_datetime([_D1, _D2, _D3, $\\,, $ |Tail]) ->\n    stringdate_to_datetime1(Tail).\n\nstringdate_to_datetime1([A, B, $\\s |T]) ->\n    stringdate_to_datetime2(T, erlang:list_to_integer([A,B]));\nstringdate_to_datetime1([A, $\\s |T]) ->\n    stringdate_to_datetime2(T, erlang:list_to_integer([A])).\n\nstringdate_to_datetime2([M1, M2, M3, $\\s , Y1, Y2, Y3, Y4, $\\s,\n                         H1, H2, $:, Min1, Min2,$:,\n                         S1, S2,$\\s ,$G, $M, $T|_], Day) ->\n    {{erlang:list_to_integer([Y1,Y2,Y3,Y4]),\n      month_str_to_int([M1, M2, M3]), Day},\n     {erlang:list_to_integer([H1, H2]),\n      erlang:list_to_integer([Min1, Min2]),\n      erlang:list_to_integer([S1, S2])}}.\n\n\n%% used by If-Modified-Since header code\nis_modified_p(FI, UTC_string) ->\n    case catch stringdate_to_datetime(UTC_string) of\n        {'EXIT', _ } ->\n            true;\n        UTC ->\n            MtimeUTC = erlang:localtime_to_universaltime(FI#file_info.mtime),\n            (MtimeUTC > UTC)\n    end.\n\n\nticker(Time, Msg) ->\n    ticker(Time, self(), Msg).\nticker(Time, To, Msg ) ->\n    spawn_link(fun() ->\n                       process_flag(trap_exit, true),\n                       yaws_ticker:ticker(Time, To, Msg)\n               end).\n\n\naddress() ->\n    Sc = get(sc),\n    ?F(\"<address> ~s Server at ~s </address>\",\n       [case Sc#sconf.yaws of\n            undefined -> (get(gc))#gconf.yaws;\n            Signature -> Signature\n        end, Sc#sconf.servername]).\n\n\nis_space($\\s) -> true;\nis_space($\\r) -> true;\nis_space($\\n) -> true;\nis_space($\\t) -> true;\nis_space(_)   -> false.\n\n\nstrip_spaces(String) ->\n    strip_spaces(String, both).\n\nstrip_spaces(String, left) ->\n    drop_spaces(String);\nstrip_spaces(String, right) ->\n    lists:reverse(drop_spaces(lists:reverse(String)));\nstrip_spaces(String, both) ->\n    strip_spaces(drop_spaces(String), right).\n\ndrop_spaces([]) ->\n    [];\ndrop_spaces(YS=[X|XS]) ->\n    case is_space(X) of\n        true  -> drop_spaces(XS);\n        false -> YS\n    end.\n\n\n%%% basic uuencode and decode functionality\nlist_to_uue(L) -> list_to_uue(L, []).\n\nlist_to_uue([], Out) ->\n    lists:reverse([$\\n,enc(0)|Out]);\nlist_to_uue(L, Out) ->\n    {L45, L1} = get_45(L),\n    Encoded = encode_line(L45),\n    list_to_uue(L1, lists:reverse(Encoded, Out)).\n\nuue_to_list(L) ->\n    uue_to_list(L, []).\n\nuue_to_list([], Out) ->\n    lists:reverse(Out);\nuue_to_list(L, Out) ->\n    {Decoded, L1} = decode_line(L),\n    uue_to_list(L1, lists:reverse(Decoded, Out)).\n\nencode_line(L) ->\n    [enc(length(L))|encode_line1(L)].\n\nencode_line1([C0, C1, C2|T]) ->\n    Char1 = enc(C0 bsr 2),\n    Char2 = enc((C0 bsl 4) band 8#60 bor (C1 bsr 4) band 8#17),\n    Char3 = enc((C1 bsl 2) band 8#74 bor (C2 bsr 6) band 8#3),\n    Char4 = enc(C2 band 8#77),\n    [Char1,Char2,Char3,Char4|encode_line1(T)];\nencode_line1([C1, C2]) ->\n    encode_line1([C1, C2, 0]);\nencode_line1([C]) ->\n    encode_line1([C,0,0]);\nencode_line1([]) ->\n    [$\\n].\n\ndecode_line([H|T]) ->\n    case dec(H) of\n        0   -> {[], []};\n        Len -> decode_line(T, Len, [])\n    end.\n\ndecode_line([P0,P1,P2,P3|T], N, Out) when N >= 3->\n    Char1 = 16#FF band ((dec(P0) bsl 2) bor (dec(P1) bsr 4)),\n    Char2 = 16#FF band ((dec(P1) bsl 4) bor (dec(P2) bsr 2)),\n    Char3 = 16#FF band ((dec(P2) bsl 6) bor dec(P3)),\n    decode_line(T, N-3, [Char3,Char2,Char1|Out]);\ndecode_line([P0,P1,P2,_|T], 2, Out) ->\n    Char1  = 16#FF band ((dec(P0) bsl 2) bor (dec(P1) bsr 4)),\n    Char2  = 16#FF band ((dec(P1) bsl 4) bor (dec(P2) bsr 2)),\n    {lists:reverse([Char2,Char1|Out]), tl(T)};\ndecode_line([P0,P1,_,_|T], 1, Out) ->\n    Char1  = 16#FF band ((dec(P0) bsl 2) bor (dec(P1) bsr 4)),\n    {lists:reverse([Char1|Out]), tl(T)};\ndecode_line(T, 0, Out) ->\n    {lists:reverse(Out), tl(T)}.\n\nget_45(L) -> get_45(L, 45, []).\n\nget_45(L, 0, F)     -> {lists:reverse(F), L};\nget_45([], _N, L)   -> {lists:reverse(L), []};\nget_45([H|T], N, L) -> get_45(T, N-1, [H|L]).\n\n\n%% enc/1 is the basic 1 character encoding function to make a char printing\n%% dec/1 is the inverse\nenc(0) -> $`;\nenc(C) -> (C band 8#77) + $ .\n\ndec(Char) -> (Char - $ ) band 8#77.\n\n\nprintversion() ->\n    io:format(\"Yaws ~s~n\", [yaws_generated:version()]),\n    init:stop().\n\n%% our default arg rewriter does of course nothing\narg_rewrite(A) ->\n    A.\n\nis_ssl(#sconf{ssl = undefined})                -> nossl;\nis_ssl(#sconf{ssl = S}) when is_record(S, ssl) -> ssl.\n\n\nto_lowerchar(C) when C >= $A, C =< $Z -> C+($a-$A);\nto_lowerchar(C)                       -> C.\n\nto_lower([])                           -> [];\nto_lower([C|Cs]) when C >= $A, C =< $Z -> [C+($a-$A)|to_lower(Cs)];\nto_lower([C|Cs])                       -> [C|to_lower(Cs)];\nto_lower(A) when is_atom(A)            -> to_lower(atom_to_list(A)).\n\n\nfunreverse(List, Fun) ->\n    funreverse(List, Fun, []).\n\nfunreverse([H|T], Fun, Ack) -> funreverse(T, Fun, [Fun(H)|Ack]);\nfunreverse([], _Fun, Ack)   -> Ack.\n\n%% is arg1 a prefix of arg2\nis_prefix([H|T1], [H|T2]) -> is_prefix(T1, T2);\nis_prefix([], T)          -> {true, T};\nis_prefix(_,_)            -> false.\n\n\n%% Split a string of words separated by Sep into a list of words and\n%% strip off white space.\n%%\n%% HTML semantics are used, such that empty words are omitted.\nsplit_sep(undefined, _Sep) ->\n    [];\nsplit_sep(L, Sep) ->\n    case drop_spaces(L) of\n        []      -> [];\n        [Sep|T] -> split_sep(T, Sep);\n        [C|T]   -> split_sep(T, Sep, [C], [])\n    end.\n\nsplit_sep([], _Sep, AccL) ->\n    lists:reverse(AccL);\nsplit_sep([Sep|T], Sep, AccL) ->\n    split_sep(T, Sep, AccL);\nsplit_sep([C|T], Sep, AccL) ->\n    split_sep(T, Sep, [C], AccL).\n\nsplit_sep([], _Sep, AccW, AccL) ->\n    lists:reverse([lists:reverse(drop_spaces(AccW))|AccL]);\nsplit_sep([Sep|Tail], Sep, AccW, AccL) ->\n    split_sep(drop_spaces(Tail), Sep, [lists:reverse(drop_spaces(AccW))|AccL]);\nsplit_sep([C|Tail], Sep, AccW, AccL) ->\n    split_sep(Tail, Sep, [C|AccW], AccL).\n\n\n%% Join strings with separator. Same as string:join in later\n%% versions of Erlang. Separator is expected to be a list.\njoin_sep([], Sep) when is_list(Sep) ->\n    [];\njoin_sep([H|T], Sep) ->\n    H ++ lists:append([Sep ++ X || X <- T]).\n\n%% Provide a unique 3-tuple of positive integers.\nunique_triple() -> yaws_dynopts:unique_triple().\n\n%% Get a current time 3-tuple.\nget_time_tuple() -> yaws_dynopts:get_time_tuple().\n\n%% header parsing\nparse_qval(S) ->\n    parse_qval([], S).\n\nparse_qval(A, \";q=\"++Q) -> {lists:reverse(A), parse_qvalue(Q)};\nparse_qval(A, \"\")       -> {lists:reverse(A), 1000};\nparse_qval(A, [C|T])    -> parse_qval([C|A], T).\n\nparse_qvalue(\"0\")              -> 0;\nparse_qvalue(\"0.\")             -> 0;\nparse_qvalue(\"1\")              -> 1000;\nparse_qvalue(\"1.\")             -> 1000;\nparse_qvalue(\"1.0\")            -> 1000;\nparse_qvalue(\"1.00\")           -> 1000;\nparse_qvalue(\"1.000\")          -> 1000;\nparse_qvalue(\"0.\"++[D1])       -> three_digits_to_integer(D1,$0,$0);\nparse_qvalue(\"0.\"++[D1,D2])    -> three_digits_to_integer(D1,D2,$0);\nparse_qvalue(\"0.\"++[D1,D2,D3]) -> three_digits_to_integer(D1,D2,D3);\nparse_qvalue(_)                -> 0. %% error\n\nthree_digits_to_integer(D1, D2, D3) ->\n    100*(D1-$0)+10*(D2-$0)+D3-$0.\n\n\n%% Gzip encoding\naccepts_gzip(H, Mime) ->\n    case [Val || {_,_,'Accept-Encoding',_,Val}<- H#headers.other] of\n        [] ->\n            false;\n        [_|_]=AcceptEncoding0 ->\n            AcceptEncoding = join_sep(AcceptEncoding0, \",\"),\n            EncList = [parse_qval(X) || X <- split_sep(AcceptEncoding, $,)],\n            case [Q || {\"gzip\",Q} <- EncList] ++ [Q || {\"*\",Q} <- EncList] of\n                [] ->\n                    false;\n                [Q|_] ->\n                    (Q > 100) %% just for fun\n                        and not has_buggy_gzip(H#headers.user_agent, Mime)\n            end\n    end.\n\n%%% Advice partly taken from Apache's documentation of `mod_deflate'.\n\n%% Only Netscape 4.06-4.08 is really broken.\nhas_buggy_gzip(\"Mozilla/4.06\"++_, _) ->\n    true;\nhas_buggy_gzip(\"Mozilla/4.07\"++_, _) ->\n    true;\nhas_buggy_gzip(\"Mozilla/4.08\"++_, _) ->\n    true;\n\n%% Everything else handles at least HTML.\nhas_buggy_gzip(_, \"text/html\") ->\n    false;\nhas_buggy_gzip(UserAgent, Mime) ->\n    UA = parse_ua(UserAgent),\n    in_ua(fun(\"Mozilla/4\"++_) ->\n                  %% Netscape 4.x may choke on anything not HTML.\n                  case Mime of\n                      %% IE doesn't, but some versions are said to have issues\n                      %% with plugins.\n                      \"application/pdf\" ->\n                          true;\n                      _ -> not in_comment(\n                                 fun(\"MSIE\"++_) -> true;\n                                    (_)         -> false\n                                 end, UA)\n                  end;\n             (\"w3m\"++_) ->\n                  %% W3m does not decompress when saving.\n                  true;\n             (\"Opera\") ->\n                  %% Opera 6 does not uncompress downloads.\n                  in_ua(fun(\"6.\"++_) -> true;\n                           (_)       -> false\n                        end, UA);\n             (\"Opera/6.\"++_) ->\n                  true;\n             (_) ->\n                  false\n          end, UA).\n\n\n%%% Parsing of User-Agent header.\n%%% Yes, this looks a bit like overkill.\ntokenize_ua([], Acc) ->\n    lists:reverse(Acc);\ntokenize_ua([$\\\\ , C|T], Acc) ->\n    tokenize_ua(T, [C|Acc]);\ntokenize_ua([$(|T], Acc) ->\n    tokenize_ua(T, [popen | Acc]);\ntokenize_ua([$)|T], Acc) ->\n    tokenize_ua(T, [pclose | Acc]);\ntokenize_ua([C|T], Acc) ->\n    tokenize_ua(T, [C|Acc]).\n\nparse_ua(Line) ->\n    case catch parse_ua_l(tokenize_ua(Line, [])) of\n        {'EXIT', _} -> [];\n        Res         -> Res\n    end.\n\nparse_ua_l(Line) ->\n    case drop_spaces(Line) of\n        [] ->\n            [];\n        [popen|T] ->\n            {Comment, Tail} = parse_comment(T),\n            [Comment | parse_ua_l(Tail)];\n        [pclose|T] ->\n            %% Error, ignore\n            parse_ua_l(T);\n        L ->\n            {UA, Tail} = parse_ua1(L),\n            [UA | parse_ua_l(Tail)]\n    end.\n\nparse_comment(L) ->\n    parse_comment(L, [], []).\n\nparse_comment([], _, _) ->\n    %% Error\n    {error, []};\nparse_comment([pclose|T], CAcc, CsAcc) ->\n    {{comment, lists:reverse([lists:reverse(CAcc)|CsAcc])}, T};\nparse_comment([popen|T], CAcc, CsAcc) ->\n    {Comment, Tail} = parse_comment(T),\n    parse_comment(drop_spaces(Tail), [], [Comment, lists:reverse(CAcc)|CsAcc]);\nparse_comment([$;|T], CAcc, CsAcc) ->\n    parse_comment(drop_spaces(T), [], [lists:reverse(CAcc)|CsAcc]);\nparse_comment([C|T], CAcc, CsAcc) ->\n    parse_comment(T, [C|CAcc], CsAcc).\n\n\nparse_ua1(L) ->\n    parse_ua1(L, []).\n\nparse_ua1([], Acc) ->\n    {{ua,lists:reverse(Acc)}, []};\nparse_ua1([popen|T], Acc) ->\n    {{ua, lists:reverse(Acc)}, [popen|T]};\nparse_ua1([pclose|T], _Acc) ->\n    {error, T};\nparse_ua1([$ |T], Acc) ->\n    {{ua, lists:reverse(Acc)}, T};\nparse_ua1([C|T], Acc) ->\n    parse_ua1(T, [C|Acc]).\n\n\nin_ua(Pred, L) ->\n    lists:any(fun({ua, UA}) -> Pred(UA);\n                 (_)        -> false\n              end, L).\n\nin_comment(_Pred, []) ->\n    false;\nin_comment(Pred, [{comment, Cs}|T]) ->\n    case in_comment_l(Pred, Cs) of\n        true  -> true;\n        false -> in_comment(Pred, T)\n    end;\nin_comment(Pred, [_|T]) ->\n    in_comment(Pred, T).\n\n\nin_comment_l(Pred, Cs) ->\n    lists:any(fun({comment, Cs1}) -> in_comment_l(Pred, Cs1);\n                 (error)          -> false;\n                 (L)              -> Pred(L)\n              end, Cs).\n\n\n%% imperative out header management\nouth_set_status_code(Code) ->\n    put(outh, (get(outh))#outh{status = Code}),\n    ok.\n\nouth_set_non_cacheable(_Version) ->\n    put(outh, (get(outh))#outh{cache_control = \"Cache-Control: no-cache\\r\\n\"}),\n    ok.\n\nouth_set_content_type(Mime) ->\n    put(outh, (get(outh))#outh{content_type = make_content_type_header(Mime)}),\n    ok.\n\nouth_set_content_encoding(Encoding) ->\n    ContentEncoding = case Encoding of\n                          identity -> undefined;\n                          deflate  -> make_content_encoding_header(Encoding)\n                      end,\n    put(outh, (get(outh))#outh{encoding         = Encoding,\n                               content_encoding = ContentEncoding}),\n    ok.\n\nouth_set_cookie(C) ->\n    put(outh, (get(outh))#outh{set_cookie = [\"Set-Cookie: \", C, \"\\r\\n\"]}),\n    ok.\n\n\nouth_clear_headers() ->\n    H = get(outh),\n    put(outh, #outh{status     = H#outh.status,\n                    doclose    = true,\n                    chunked    = false,\n                    connection = make_connection_close_header(true)}),\n    ok.\n\n\nouth_set_static_headers(Req, UT, Headers) ->\n    outh_set_static_headers(Req, UT, Headers, all).\n\nouth_set_static_headers(Req, UT, Headers, Range) ->\n    H = get(outh),\n    FIL = (UT#urltype.finfo)#file_info.size,\n    {DoClose0, Chunked0} = dcc(Req, Headers),\n    {DoDeflate, Length}\n        = case Range of\n              all ->\n                  case UT#urltype.deflate of\n                      DB when is_binary(DB) -> % cached\n                          %% Remove charset\n                          [Mime|_] = yaws:split_sep(UT#urltype.mime, $;),\n                          case accepts_gzip(Headers, Mime) of\n                              true  -> {true, size(DB)};\n                              false -> {false, FIL}\n                          end;\n                      undefined ->\n                          {false, FIL};\n                      dynamic ->\n                          %% Remove charset\n                          [Mime|_] = yaws:split_sep(UT#urltype.mime, $;),\n                          case accepts_gzip(Headers, Mime) of\n                              true  -> {true, undefined};\n                              false -> {false, FIL}\n                          end\n                  end;\n              {fromto, From, To, _} ->\n                  {false, To - From + 1}\n          end,\n    Encoding = case DoDeflate of\n                   true  -> decide;\n                   false -> identity\n               end,\n    Chunked = Chunked0 and (Length == undefined),\n    DoClose = if\n                  DoClose0 == true ->\n                      true;\n                  ((Length == undefined) and not Chunked) ->\n                      %% We cannot keep the connection alive, because the client\n                      %% has no way of knowing the end of the content data.\n                      true;\n                  DoClose0 == keep_alive ->\n                      keep_alive;\n                  true ->\n                      DoClose0\n              end,\n\n    H2 = H#outh{\n           status            = case Range of\n                                   all               -> 200;\n                                   {fromto, _, _, _} -> 206\n                               end,\n           chunked           = Chunked,\n           encoding          = Encoding,\n           date              = make_date_header(),\n           server            = make_server_header(),\n           last_modified     = make_last_modified_header(UT#urltype.finfo),\n           etag              = make_etag_header(UT#urltype.finfo),\n           content_range     = make_content_range_header(Range),\n           content_length    = make_content_length_header(Length),\n           content_type      = make_content_type_header(UT#urltype.mime),\n           content_encoding  = make_content_encoding_header(Encoding),\n           transfer_encoding = make_transfer_encoding_chunked_header(Chunked),\n           connection        = make_connection_close_header(DoClose),\n           doclose           = DoClose,\n           contlen           = Length\n          },\n    %% store finfo to set last_modified, expires and cache_control headers\n    %% during #outh{} serialization.\n    put(file_info, UT#urltype.finfo),\n    put(outh, H2).\n\nouth_set_304_headers(Req, UT, Headers) ->\n    H = get(outh),\n    {DoClose, _Chunked} = dcc(Req, Headers),\n    H2 = H#outh{\n           status         = 304,\n           chunked        = false,\n           date           = make_date_header(),\n           server         = make_server_header(),\n           last_modified  = make_last_modified_header(UT#urltype.finfo),\n           etag           = make_etag_header(UT#urltype.finfo),\n           content_length = make_content_length_header(0),\n           connection     = make_connection_close_header(DoClose),\n           doclose        = DoClose,\n           contlen        = 0\n          },\n    %% store finfo to set last_modified, expires and cache_control headers\n    %% during #outh{} serialization.\n    put(file_info, UT#urltype.finfo),\n    put(outh, H2).\n\nouth_set_dyn_headers(Req, Headers, UT) ->\n    H = get(outh),\n    {DoClose, Chunked} = dcc(Req, Headers),\n    H2 = H#outh{\n           status            = 200,\n           date              = make_date_header(),\n           server            = make_server_header(),\n           connection        = make_connection_close_header(DoClose),\n           content_type      = make_content_type_header(UT#urltype.mime),\n           doclose           = DoClose,\n           chunked           = Chunked,\n           transfer_encoding = make_transfer_encoding_chunked_header(Chunked)},\n    %% store finfo to set last_modified, expires and cache_control headers\n    %% during #outh{} serialization.\n    put(file_info, UT#urltype.finfo),\n    put(outh, H2).\n\n\nouth_set_connection(What) ->\n    H = get(outh),\n    H2 = H#outh{connection = make_connection_close_header(What),\n                doclose    = What},\n    put(outh, H2),\n    ok.\n\n\nouth_set_content_length(Int) ->\n    H  = get(outh),\n    H2 = H#outh{\n           content_length = make_content_length_header(Int),\n           contlen        = Int\n          },\n    put(outh, H2).\n\n\n\nouth_set_dcc(Req, Headers) ->\n    H = get(outh),\n    {DoClose, Chunked} = dcc(Req, Headers),\n    H2 = H#outh{\n           connection        = make_connection_close_header(DoClose),\n           doclose           = DoClose,\n           chunked           = Chunked,\n           transfer_encoding = make_transfer_encoding_chunked_header(Chunked)\n          },\n    put(outh, H2).\n\n\n%% can only turn if off, not on.\n%% if it allready is off, it's off because the cli headers forced us.\nouth_set_transfer_encoding_off() ->\n    H  = get(outh),\n    H2 = H#outh{\n           chunked           = false,\n           transfer_encoding = make_transfer_encoding_chunked_header(false)\n          },\n    put(outh, H2).\n\nouth_set_auth([]) ->\n    ok;\n\nouth_set_auth(Headers) ->\n    H  = get(outh),\n    H2 = case H#outh.www_authenticate of\n             undefined ->\n                 H#outh{www_authenticate = Headers};\n             _ ->\n                 H#outh{www_authenticate = H#outh.www_authenticate ++ Headers}\n         end,\n    put(outh, H2).\n\nouth_set_vary(Fields) ->\n    put(outh, (get(outh))#outh{vary = make_vary_header(Fields)}),\n    ok.\n\nouth_fix_doclose() ->\n    H = get(outh),\n    if\n        (H#outh.doclose /= true)    andalso\n        (H#outh.contlen==undefined) andalso\n        (H#outh.chunked == false) ->\n            put(outh, H#outh{doclose    = true,\n                             connection = make_connection_close_header(true)});\n        true ->\n            ok\n    end.\n\n\ndcc(Req, Headers) ->\n    H = get(outh),\n    DoClose = case Req#http_request.version of\n                  _ when H#outh.exceedmaxuses == true ->\n                      true; %% too many keepalives\n                  {1, 0} ->\n                      case Headers#headers.connection of\n                          \"close\"      -> true;\n                          \"Keep-Alive\" -> keep_alive;\n                          _            -> true\n                      end;\n                  {1, 1} ->\n                      Headers#headers.connection == \"close\";\n                  {0,9} ->\n                      true\n              end,\n    Chunked = case Req#http_request.version of\n                  {1, 0} -> false;\n                  {1,1}  -> true;\n                  {0,9}  ->  false\n              end,\n    {DoClose, Chunked}.\n\n\n\n\n\n%%\n%% The following all make_ function return an actual header string\n%%\nmake_allow_header() ->\n    make_allow_header([]).\nmake_allow_header(Options) ->\n    case Options of\n        [] ->\n            [\"Allow: GET, POST, OPTIONS, HEAD\\r\\n\"];\n        _ ->\n            [\"Allow: \",\n             lists:foldl(fun(M, \"\") -> atom_to_list(M);\n                            (M, Acc) -> atom_to_list(M) ++ \", \" ++ Acc\n                         end, \"\", lists:reverse(Options)),\n             \"\\r\\n\"]\n    end.\nmake_server_header() ->\n    Sc = get(sc),\n    Signature = case Sc#sconf.yaws of\n                    undefined -> (get(gc))#gconf.yaws;\n                    S         -> S\n                end,\n    case Signature of\n        \"\" ->\n            [];\n        _ ->\n            [\"Server: \", Signature, \"\\r\\n\"]\n    end.\n\nmake_last_modified_header(FI) ->\n    Then = FI#file_info.mtime,\n    [\"Last-Modified: \", local_time_as_gmt_string(Then), \"\\r\\n\"].\n\n\nmake_expires_header(all, FI) ->\n    SC = get(sc),\n    case lists:keyfind(all, 1, SC#sconf.expires) of\n        {_, EType, TTL} -> make_expires_header(EType, TTL, FI);\n        false           -> {undefined, undefined}\n    end;\nmake_expires_header({Type,all}, FI) ->\n    SC = get(sc),\n    case lists:keyfind({Type,all}, 1, SC#sconf.expires) of\n        {_, EType, TTL} -> make_expires_header(EType, TTL, FI);\n        false           -> make_expires_header(all, FI)\n    end;\nmake_expires_header({Type,SubType}, FI) ->\n    SC = get(sc),\n    case lists:keyfind({Type,SubType}, 1, SC#sconf.expires) of\n        {_, EType, TTL} -> make_expires_header(EType, TTL, FI);\n        false           -> make_expires_header({Type,all}, FI)\n    end;\nmake_expires_header(MT0, FI) ->\n    SC = get(sc),\n    %% Use split_sep to remove charset\n    case yaws:split_sep(MT0, $;) of\n        [] -> {undefined, undefined};\n        [MT1|_] ->\n            case lists:keyfind(MT1, 1, SC#sconf.expires) of\n                {_, EType, TTL} ->\n                    make_expires_header(EType, TTL, FI);\n                false ->\n                    case split_sep(MT1, $/) of\n                        [Type, SubType] ->\n                            make_expires_header({Type,SubType}, FI);\n                        false ->\n                            make_expires_header(all, FI)\n                    end\n            end\n    end.\n\n\nmake_expires_header(always, _TTL, _FI) ->\n    {[\"Expires: \", \"Thu, 01 Jan 1970 00:00:00 GMT\\r\\n\"],\n     [\"Cache-Control: \", \"private, no-cache, no-store, must-revalidate, max-age=0, proxy-revalidate, s-maxage=0\\r\\n\"]};\nmake_expires_header(access, TTL, _FI) ->\n    Secs = calendar:datetime_to_gregorian_seconds(erlang:universaltime()),\n    ExpireTime = calendar:gregorian_seconds_to_datetime(Secs+TTL),\n    {[\"Expires: \", universal_time_as_string(ExpireTime), \"\\r\\n\"],\n     [\"Cache-Control: \", \"max-age=\", erlang:integer_to_list(TTL), \"\\r\\n\"]};\nmake_expires_header(modify, TTL, FI) ->\n    %% mtime is local here\n    Secs1 = calendar:datetime_to_gregorian_seconds(FI#file_info.mtime),\n    Secs2 = calendar:datetime_to_gregorian_seconds(erlang:localtime()),\n    ExpireTime = calendar:gregorian_seconds_to_datetime(Secs1+TTL),\n    MaxAge     = erlang:max(0, TTL - (Secs2 - Secs1)),\n    {[\"Expires: \", local_time_as_gmt_string(ExpireTime), \"\\r\\n\"],\n     [\"Cache-Control: \", \"max-age=\", erlang:integer_to_list(MaxAge), \"\\r\\n\"]}.\n\n\nmake_location_header(Where) ->\n    [\"Location: \", Where, \"\\r\\n\"].\n\n\nmake_etag_header(FI) ->\n    ETag = make_etag(FI),\n    [\"Etag: \", ETag, \"\\r\\n\"].\n\nmake_etag(FI) ->\n    Stamp = {FI#file_info.size, FI#file_info.mtime},\n    ETag = integer_to_list(erlang:phash2(Stamp, 16#100000000), 19),\n    lists:flatten([$\", ETag, $\"]).\n\n\nmake_content_type_header(no_content_type) ->\n    undefined;\nmake_content_type_header(MimeType) ->\n    [\"Content-Type: \", MimeType, \"\\r\\n\"].\n\n\nmake_content_range_header(all) ->\n    undefined;\nmake_content_range_header({fromto, From, To, Tot}) ->\n    [\"Content-Range: bytes \",\n     erlang:integer_to_list(From), $-, erlang:integer_to_list(To),\n     $/, erlang:integer_to_list(Tot), $\\r, $\\n].\n\nmake_content_length_header(Size) when is_integer(Size) ->\n    [\"Content-Length: \", erlang:integer_to_list(Size), \"\\r\\n\"];\nmake_content_length_header(FI) when is_record(FI, file_info) ->\n    Size = FI#file_info.size,\n    [\"Content-Length: \", erlang:integer_to_list(Size), \"\\r\\n\"];\nmake_content_length_header(_) ->\n    undefined.\n\nmake_content_encoding_header(deflate) ->\n    \"Content-Encoding: gzip\\r\\n\";\nmake_content_encoding_header(_) ->\n    undefined.\n\nmake_connection_close_header(true) ->\n    \"Connection: close\\r\\n\";\nmake_connection_close_header(false) ->\n    undefined;\nmake_connection_close_header(keep_alive) ->\n    \"Connection: Keep-Alive\\r\\n\".\n\nmake_transfer_encoding_chunked_header(true) ->\n    \"Transfer-Encoding: chunked\\r\\n\";\nmake_transfer_encoding_chunked_header(false) ->\n    undefined.\n\nmake_www_authenticate_header({realm, Realm}) ->\n    [\"WWW-Authenticate: Basic realm=\\\"\", Realm, [\"\\\"\\r\\n\"]];\n\nmake_www_authenticate_header(Method) ->\n    [\"WWW-Authenticate: \", Method, [\"\\r\\n\"]].\n\nmake_date_header() ->\n    N = element(2, os:timestamp()),\n    case get(date_header) of\n        {_Str, Secs} when (Secs+10) < N ->\n            H = [\"Date: \", universal_time_as_string(), \"\\r\\n\"],\n            put(date_header, {H, N}),\n            H;\n        {Str, _Secs} ->\n            Str;\n        undefined ->\n            H = [\"Date: \", universal_time_as_string(), \"\\r\\n\"],\n            put(date_header, {H, N}),\n            H\n    end.\n\nmake_vary_header(Fields) ->\n    case lists:member(\"*\", Fields) of\n        true  -> [\"Vary: \", \"*\", \"\\r\\n\"];\n        false -> [\"Vary: \", join_sep(Fields, \", \"), \"\\r\\n\"]\n    end.\n\n\n\n%% access functions into the outh record\nouth_get_status_code() ->\n    (get(outh))#outh.status.\n\nouth_get_contlen() ->\n    (get(outh))#outh.contlen.\n\nouth_get_act_contlen() ->\n    (get(outh))#outh.act_contlen.\n\nouth_inc_act_contlen(Int) ->\n    O = get(outh),\n    L = case O#outh.act_contlen of\n            undefined -> Int;\n            Len       -> Len+Int\n        end,\n    put(outh, O#outh{act_contlen = L}),\n    L.\n\nouth_get_doclose() ->\n    (get(outh))#outh.doclose.\n\nouth_get_chunked() ->\n    (get(outh))#outh.chunked.\n\nouth_get_content_encoding() ->\n    (get(outh))#outh.encoding.\n\nouth_get_content_encoding_header() ->\n    (get(outh))#outh.content_encoding.\n\nouth_get_content_type() ->\n    case (get(outh))#outh.content_type of\n        undefined    -> undefined;\n        [_, Mime, _] -> Mime\n    end.\n\nouth_get_vary_fields() ->\n    case (get(outh))#outh.vary of\n        undefined      -> [];\n        [_, Fields, _] -> split_sep(Fields, $,)\n    end.\n\nouth_serialize() ->\n    H = get(outh),\n    Code = case H#outh.status of\n               undefined -> 200;\n               Int       -> Int\n           end,\n    StatusLine = [\"HTTP/1.1 \", erlang:integer_to_list(Code), \" \",\n                  yaws_api:code_to_phrase(Code), \"\\r\\n\"],\n    GC=get(gc),\n    if ?gc_has_debug(GC) -> yaws_debug:check_headers(H);\n       true              -> ok\n    end,\n    ContentEnc = case H#outh.content_encoding of\n                     undefined -> make_content_encoding_header(H#outh.encoding);\n                     CE        -> CE\n                 end,\n    {Expires, CacheControl} =\n        case erase(file_info) of\n            undefined ->\n                {H#outh.expires, H#outh.cache_control};\n            FI ->\n                {E, CC} = case {H#outh.expires, H#outh.cache_control} of\n                              {undefined, undefined} ->\n                                  CT = outh_get_content_type(),\n                                  make_expires_header(CT, FI);\n                              _ ->\n                                  {H#outh.expires, H#outh.cache_control}\n                          end,\n                {E, CC}\n        end,\n\n    %% Add 'Accept-Encoding' in the 'Vary:' header if the compression is enabled\n    %% or if the response is compressed _AND_ if the response has a non-empty\n    %% body.\n    Vary = case get(sc) of\n               undefined -> undefined;\n               SC ->\n                   case (?sc_has_deflate(SC) orelse H#outh.encoding == deflate) of\n                       true when H#outh.contlen /= undefined, H#outh.contlen /= 0;\n                                 H#outh.act_contlen /= undefined,\n                                 H#outh.act_contlen /= 0 ->\n                           Fields = outh_get_vary_fields(),\n                           Fun    = fun(\"*\") -> true;\n                                       (F)   -> (to_lower(F) == \"accept-encoding\")\n                                    end,\n                           case lists:any(Fun, Fields) of\n                               true  -> H#outh.vary;\n                               false -> make_vary_header([\"Accept-Encoding\"|Fields])\n                           end;\n                       _ ->\n                           H#outh.vary\n                   end\n           end,\n\n    Headers = [noundef(H#outh.connection),\n               noundef(H#outh.server),\n               noundef(H#outh.location),\n               noundef(H#outh.date),\n               noundef(H#outh.allow),\n               noundef(H#outh.last_modified),\n               noundef(Expires),\n               noundef(CacheControl),\n               noundef(H#outh.etag),\n               noundef(H#outh.content_range),\n               noundef(H#outh.content_length),\n               noundef(H#outh.content_type),\n               noundef(ContentEnc),\n               noundef(H#outh.set_cookie),\n               noundef(H#outh.transfer_encoding),\n               noundef(H#outh.www_authenticate),\n               noundef(Vary),\n               noundef(H#outh.other)],\n    {StatusLine, Headers}.\n\n\nnoundef(undefined) -> [];\nnoundef(Str)       -> Str.\n\n\n\naccumulate_header({X, erase}) when is_atom(X) ->\n    erase_header(X);\n\n\n%% special headers\naccumulate_header({connection, What}) ->\n    DC = case What of\n             \"close\" -> true;\n             _       -> false\n         end,\n    H = get(outh),\n    put(outh, H#outh{connection = [\"Connection: \", What, \"\\r\\n\"],\n                     doclose    = DC});\naccumulate_header({\"Connection\", What}) ->\n    accumulate_header({connection, What});\n\naccumulate_header({server, What}) ->\n    put(outh, (get(outh))#outh{server = [\"Server: \", What, \"\\r\\n\"]});\naccumulate_header({\"Server\", What}) ->\n    accumulate_header({server, What});\n\naccumulate_header({location, What}) ->\n    put(outh, (get(outh))#outh{location = [\"Location: \", What, \"\\r\\n\"]});\naccumulate_header({\"Location\", What}) ->\n    accumulate_header({location, What});\n\naccumulate_header({cache_control, What}) ->\n    put(outh, (get(outh))#outh{cache_control = [\"Cache-Control: \", What,\n                                                \"\\r\\n\"]});\naccumulate_header({\"Cache-Control\", What}) ->\n    accumulate_header({cache_control, What});\n\naccumulate_header({expires, What}) ->\n    put(outh, (get(outh))#outh{expires = [\"Expires: \", What, \"\\r\\n\"]});\naccumulate_header({\"Expires\", What}) ->\n    accumulate_header({expires, What});\n\naccumulate_header({date, What}) ->\n    put(outh, (get(outh))#outh{date = [\"Date: \", What, \"\\r\\n\"]});\naccumulate_header({\"Date\", What}) ->\n    accumulate_header({date, What});\n\naccumulate_header({allow, What}) ->\n    put(outh, (get(outh))#outh{date = [\"Allow: \", What, \"\\r\\n\"]});\naccumulate_header({\"Allow\", What}) ->\n    accumulate_header({allow, What});\n\naccumulate_header({last_modified, What}) ->\n    put(outh, (get(outh))#outh{last_modified = [\"Last-Modified: \", What,\n                                                \"\\r\\n\"]});\naccumulate_header({\"Last-Modified\", What}) ->\n    accumulate_header({last_modified, What});\n\naccumulate_header({etag, What}) ->\n    put(outh, (get(outh))#outh{etag = [\"Etag: \", What, \"\\r\\n\"]});\naccumulate_header({\"Etag\", What}) ->\n    accumulate_header({etag, What});\n\naccumulate_header({set_cookie, What}) ->\n    O = get(outh),\n    Old = case O#outh.set_cookie of\n              undefined -> \"\";\n              X         -> X\n          end,\n    put(outh, O#outh{set_cookie = [\"Set-Cookie: \", What, \"\\r\\n\"|Old]});\naccumulate_header({\"Set-Cookie\", What}) ->\n    accumulate_header({set_cookie, What});\n\naccumulate_header({content_range, What}) ->\n    put(outh, (get(outh))#outh{content_range = [\"Content-Range: \", What,\n                                                \"\\r\\n\"]});\naccumulate_header({\"Content-Range\", What}) ->\n    accumulate_header({content_range, What});\n\naccumulate_header({content_type, What}) ->\n    put(outh, (get(outh))#outh{content_type = [\"Content-Type: \", What,\n                                               \"\\r\\n\"]});\naccumulate_header({\"Content-Type\", What}) ->\n    accumulate_header({content_type, What});\n\naccumulate_header({content_encoding, What}) ->\n    case What of\n        \"identity\" ->\n            put(outh, (get(outh))#outh{encoding         = identity,\n                                       content_encoding = undefined});\n        _ ->\n            put(outh, (get(outh))#outh{encoding         = deflate,\n                                       content_encoding = [\"Content-Encoding: \",\n                                                           What, \"\\r\\n\"]})\n    end;\naccumulate_header({\"Content-Encoding\", What}) ->\n    accumulate_header({content_encoding, What});\n\naccumulate_header({content_length, Len}) when is_integer(Len) ->\n    H = get(outh),\n    put(outh, H#outh{\n                chunked           = false,\n                transfer_encoding = undefined,\n                contlen           = Len,\n                act_contlen       = 0,\n                content_length    = make_content_length_header(Len)});\naccumulate_header({\"Content-Length\", Len}) ->\n    case Len of\n        I when is_integer(I) ->\n            accumulate_header({content_length, I});\n        L when is_list(L) ->\n            accumulate_header({content_length, erlang:list_to_integer(L)})\n    end;\n\naccumulate_header({transfer_encoding, What}) ->\n    put(outh, (get(outh))#outh{chunked           = true,\n                               contlen           = 0,\n                               transfer_encoding = [\"Transfer-Encoding: \", What,\n                                                    \"\\r\\n\"]});\naccumulate_header({\"Transfer-Encoding\", What}) ->\n    accumulate_header({transfer_encoding, What});\n\naccumulate_header({www_authenticate, What}) ->\n    put(outh, (get(outh))#outh{www_authenticate = [\"WWW-Authenticate: \", What,\n                                                   \"\\r\\n\"]});\naccumulate_header({\"WWW-Authenticate\", What}) ->\n    accumulate_header({www_authenticate, What});\n\naccumulate_header({vary, What}) ->\n    put(outh, (get(outh))#outh{vary = [\"Vary: \", What, \"\\r\\n\"]});\naccumulate_header({\"Vary\", What}) ->\n    accumulate_header({vary, What});\n\n%% non-special headers (which may be special in a future Yaws version)\naccumulate_header({Name, What}) when is_list(Name) ->\n    H = get(outh),\n    Old = case H#outh.other of\n              undefined -> [];\n              V         -> V\n          end,\n    H2 = H#outh{other = [Name, \": \", What, \"\\r\\n\", Old]},\n    put(outh, H2);\n\n\n\n%% backwards compatible clause\naccumulate_header(Data) when is_list(Data) ->\n    Str = lists:flatten(Data),\n    accumulate_header(split_header(Str)).\n\nsplit_header(Str)           ->\n    split_header(Str, []).\n\nsplit_header([], A)         -> {lists:reverse(A), \"\"};\nsplit_header([$:, $ |W], A) -> {lists:reverse(A), W};\nsplit_header([$:|W], A)     -> {lists:reverse(A), W};\nsplit_header([C|S], A)      -> split_header(S, [C|A]).\n\n\nerase_header(connection) ->\n    put(outh, (get(outh))#outh{connection=undefined, doclose=false});\nerase_header(server) ->\n    put(outh, (get(outh))#outh{server=undefined});\nerase_header(cache_control) ->\n    put(outh, (get(outh))#outh{cache_control=undefined});\nerase_header(expires) ->\n    put(outh, (get(outh))#outh{expires=undefined});\nerase_header(date) ->\n    put(outh, (get(outh))#outh{date=undefined});\nerase_header(allow) ->\n    put(outh, (get(outh))#outh{allow=undefined});\nerase_header(last_modified) ->\n    put(outh, (get(outh))#outh{last_modified=undefined});\nerase_header(etag) ->\n    put(outh, (get(outh))#outh{etag=undefined});\nerase_header(set_cookie) ->\n    put(outh, (get(outh))#outh{set_cookie=undefined});\nerase_header(content_range) ->\n    put(outh, (get(outh))#outh{content_range=undefined});\nerase_header(content_length) ->\n    put(outh, (get(outh))#outh{contlen=0, content_length=undefined});\nerase_header(content_type) ->\n    put(outh, (get(outh))#outh{content_type=undefined});\nerase_header(content_encoding) ->\n    put(outh, (get(outh))#outh{encoding=decide, content_encoding=undefined});\nerase_header(transfer_encoding) ->\n    put(outh, (get(outh))#outh{chunked           = false,\n                               act_contlen       = 0,\n                               transfer_encoding = undefined});\nerase_header(www_authenticate) ->\n    put(outh, (get(outh))#outh{www_authenticate=undefined});\nerase_header(location) ->\n    put(outh, (get(outh))#outh{location=undefined});\nerase_header(vary) ->\n    put(outh, (get(outh))#outh{vary=undefined}).\n\ngetuid() ->\n    case os:type() of\n        {win32, _} ->\n            {ok, \"0\"};\n        _ ->\n            load_setuid_drv(),\n            P = open_port({spawn, \"setuid_drv g\"},[]),\n            receive\n                {P, {data, \"ok \" ++ IntList}} ->\n                    {ok, IntList}\n            end\n    end.\n\nuser_to_home(User) ->\n    case os:type() of\n        {win32, _} ->\n            \".\";\n        _ ->\n            load_setuid_drv(),\n            P = open_port({spawn, \"setuid_drv \" ++ [$h|User]}, []),\n            receive\n                {P, {data, \"ok \" ++ Home}} ->\n                    Home\n            end\n    end.\n\n\nuid_to_name(Uid) ->\n    load_setuid_drv(),\n    P = open_port({spawn, \"setuid_drv \" ++\n                       [$n|erlang:integer_to_list(Uid)]}, []),\n    receive\n        {P, {data, \"ok \" ++ Name}} ->\n            Name\n    end.\n\nload_setuid_drv() ->\n    Path = filename:join(get_priv_dir(), \"lib\"),\n    case erl_ddll:load_driver(Path, \"setuid_drv\") of\n        ok ->\n            ok;\n        {error, Reason} ->\n            error_logger:format(\"Failed to load setuid_drv (from ~p) : ~p\",\n                                [Path, erl_ddll:format_error(Reason)]),\n            exit(normal)\n    end.\n\nexists(F) ->\n    case file:open(F, [read, raw]) of\n        {ok, Fd} ->\n            file:close(Fd),\n            ok;\n        _ ->\n            false\n    end.\n\n\nmkdir(Path) ->\n    [Hd|Parts] = filename:split(Path),\n    mkdir([Hd], Parts).\nmkdir(Ack, []) ->\n    ensure_exist(filename:join(Ack));\nmkdir(Ack, [H|T]) ->\n    ensure_exist(filename:join(Ack ++ [H])),\n    mkdir(Ack ++ [H], T).\n\nensure_exist(Path) ->\n    case file:read_file_info(Path) of\n        {ok, _} ->\n            ok;\n        _ ->\n            case file:make_dir(Path) of\n                ok ->\n                    ok;\n                ERR ->\n                    error_logger:format(\"Failed to mkdir ~p: ~p~n\", [Path, ERR])\n            end\n    end.\n\n%%\n%%\n%% TCP/SSL connection with a configurable IPv4/IPv6 preference on NS lookup.\n%%\n%%\n\ntcp_connect(Host, Port, Options) ->\n    tcp_connect(Host, Port, Options, infinity).\n\ntcp_connect(Host, Port, Options, Timeout) ->\n    parse_ipaddr_and_connect(tcp, Host, Port, Options, Timeout).\n\nssl_connect(Host, Port, Options) ->\n    ssl_connect(Host, Port, Options, infinity).\n\nssl_connect(Host, Port, Options, Timeout) ->\n    parse_ipaddr_and_connect(ssl, Host, Port, Options, Timeout).\n\nparse_ipaddr_and_connect(Proto, IP, Port, Options, Timeout)\nwhen is_tuple(IP) ->\n    %% The caller handled name resolution himself.\n    filter_tcpoptions_and_connect(Proto, undefined,\n      IP, Port, Options, Timeout);\nparse_ipaddr_and_connect(Proto, [$[ | Rest], Port, Options, Timeout) ->\n    %% yaws_api:parse_url/1 keep the \"[...]\" enclosing an IPv6 address.\n    %% Remove them now, and parse the address.\n    IP = string:strip(Rest, right, $]),\n    parse_ipaddr_and_connect(Proto, IP, Port, Options, Timeout);\nparse_ipaddr_and_connect(Proto, Host, Port, Options, Timeout) ->\n    %% First, try to parse an IP address, because inet:getaddr/2 could\n    %% return nxdomain if the family doesn't match the IP address\n    %% format.\n    case inet:parse_strict_address(Host) of\n        {ok, IP} ->\n            filter_tcpoptions_and_connect(Proto, undefined,\n                                          IP, Port, Options, Timeout);\n        {error, einval} ->\n            NsLookupPref = get_nslookup_pref(Options),\n            filter_tcpoptions_and_connect(Proto, NsLookupPref,\n                                          Host, Port, Options, Timeout)\n    end.\n\nfilter_tcpoptions_and_connect(Proto, NsLookupPref,\n  Host, Port, Options, Timeout) ->\n    %% Now that we have IP addresses, remove family from the TCP options,\n    %% because calling gen_tcp:connect/3 with {127,0,0,1} and [inet6]\n    %% would return {error, nxdomain otherwise}.\n    OptionsWithoutFamily = lists:filter(fun\n          (inet)  -> false;\n          (inet6) -> false;\n          (_)     -> true\n      end, Options),\n    resolve_and_connect(Proto, NsLookupPref, Host, Port, OptionsWithoutFamily, Timeout).\n\nresolve_and_connect(Proto, _, IP, Port, Options, Timeout)\nwhen is_tuple(IP) ->\n    do_connect(Proto, IP, Port, Options, Timeout);\nresolve_and_connect(Proto, [Family | Rest], Host, Port, Options, Timeout) ->\n    Result = case inet:getaddr(Host, Family) of\n        {ok, IP} -> do_connect(Proto, IP, Port, Options, Timeout);\n        R        -> R\n    end,\n    case Result of\n        {ok, Socket} ->\n            {ok, Socket};\n        {error, _} when length(Rest) >= 1 ->\n            %% If the connection fails here, ignore the error and\n            %% continue with the next address family.\n            resolve_and_connect(Proto, Rest, Host, Port, Options, Timeout);\n        {error, Reason} ->\n            %% This was the last IP address in the list, return the\n            %% connection error.\n            {error, Reason}\n    end.\n\ndo_connect(Proto, IP, Port, Options, Timeout) ->\n    case Proto of\n        tcp -> gen_tcp:connect(IP, Port, Options, Timeout);\n        ssl -> ssl:connect(IP, Port, Options, Timeout)\n    end.\n\n%% If the caller specified inet or inet6 in the TCP options, prefer\n%% this to the global nslookup_pref parameter.\n%%\n%% This can be used in processes which can't use get(gc) to get the\n%% global conf: if they are given the global conf, they can get\n%% nslookup_pref value and add it the TCP options.\n%%\n%% If neither TCP options specify the family, nor the global conf is\n%% accessible, use default value declared in #gconf definition.\nget_nslookup_pref(TcpOptions) ->\n    get_nslookup_pref(TcpOptions, []).\n\nget_nslookup_pref([inet | Rest], Result) ->\n    get_nslookup_pref(Rest, [inet | Result]);\nget_nslookup_pref([inet6 | Rest], Result) ->\n    get_nslookup_pref(Rest, [inet6 | Result]);\nget_nslookup_pref([_ | Rest], Result) ->\n    get_nslookup_pref(Rest, Result);\nget_nslookup_pref([], []) ->\n    case get(gc) of\n        undefined -> gconf_nslookup_pref(#gconf{});\n        GC        -> gconf_nslookup_pref(GC)\n    end;\nget_nslookup_pref([], Result) ->\n    lists:reverse(Result).\n\n%%\n%%\n%% http/tcp send receive functions\n%%\n%%\ndo_recv(Sock, Num, nossl) ->\n    gen_tcp:recv(Sock, Num, (get(gc))#gconf.keepalive_timeout);\ndo_recv(Sock, Num, ssl) ->\n    ssl:recv(Sock, Num, (get(gc))#gconf.keepalive_timeout).\ndo_recv(Sock, Num, nossl, Timeout) ->\n    gen_tcp:recv(Sock, Num, Timeout);\ndo_recv(Sock, Num, ssl, Timeout) ->\n    ssl:recv(Sock, Num, Timeout).\n\ncli_recv(S, Num, SslBool) ->\n    Res = do_recv(S, Num, SslBool),\n    cli_recv_trace(yaws_trace:get_type(get(gc)), Res),\n    Res.\n\ncli_recv_trace(undefined, _) ->\n    ok;\ncli_recv_trace(Trace, Res) ->\n    case Res of\n        {ok, Val} when is_tuple(Val) ->\n            yaws_trace:write(from_client, ?F(\"~p~n\", [Val]));\n        {error, What} ->\n            yaws_trace:write(from_client, ?F(\"~p~n\", [What]));\n        {ok, http_eoh} ->\n            ok;\n        {ok, Val} when Trace == traffic ->\n            yaws_trace:write(from_client, Val);\n        _ ->\n            ok\n    end.\n\n\n\ngen_tcp_send(S, Data) ->\n    SC = get(sc),\n    Res = case SC of\n              undefined ->\n                  case catch ssl:sockname(S) of\n                      {ok, _} -> ssl:send(S, Data);\n                      _ -> gen_tcp:send(S, Data)\n                  end;\n              _ ->\n                  case SC#sconf.ssl of\n                      undefined -> gen_tcp:send(S, Data);\n                      _SSL      -> ssl:send(S, Data)\n                  end\n          end,\n    case ?gc_has_debug((get(gc))) of\n        false ->\n            case Res of\n                ok ->\n                    case SC of\n                        undefined -> ok;\n                        _ ->\n                            yaws_stats:sent(iolist_size(Data))\n                    end,\n                    ok;\n                _Err ->\n                    exit(normal)   %% keep quiet\n            end;\n        true ->\n            case Res of\n                ok ->\n                    case SC of\n                        undefined -> ok;\n                        _ ->\n                            yaws_stats:sent(iolist_size(Data))\n                    end,\n                    ?Debug(\"Sent ~p~n\", [yaws_debug:nobin(Data)]),\n                    ok;\n                Err ->\n                    {B2, Size} = strip(Data),\n                    yaws_debug:derror(\"Failed to send ~w bytes:~n~p \"\n                                      \"on socket ~p: ~p~n~p~n\",\n                                      [Size, B2, S, Err,\n                                       yaws_debug:nobin(Data)]),\n                    erlang:error(Err)\n            end\n    end.\n\n\nstrip(Data) ->\n    L = list_to_binary([Data]),\n    case L of\n        <<Head:50/binary, _/binary>> ->\n            {binary_to_list(<<Head/binary, \".....\">>), size(L)};\n        _ ->\n            {binary_to_list(L), size(L)}\n    end.\n\n\n\n%% This is the api function\n%% return {Req, Headers}\n%%     or closed\nhttp_get_headers(CliSock, SSL) ->\n    do_http_get_headers(CliSock, SSL).\n\n\nheaders_to_str(Headers) ->\n    lists:map(fun(H) -> [H, \"\\r\\n\"] end, yaws_api:reformat_header(Headers)).\n\n\nsetopts(Sock, Opts, nossl) ->\n    ok = inet:setopts(Sock, Opts);\nsetopts(Sock, Opts, ssl) ->\n    ok = ssl:setopts(Sock, Opts).\n\ndo_http_get_headers(CliSock, SSL) ->\n    case http_recv_request(CliSock,SSL) of\n        bad_request ->\n            {#http_request{method=bad_request, version={0,9}}, #headers{}};\n        closed ->\n            closed;\n        R ->\n            %% Http request received. Store the current time. it will be usefull\n            %% to get the time taken to serve the request.\n            put(request_start_time, os:timestamp()),\n            case http_collect_headers(CliSock, R,  #headers{}, SSL, 0) of\n                {error, _}=Error ->\n                    Error;\n                H ->\n                    {R, H}\n            end\n    end.\n\n\nhttp_recv_request(CliSock, SSL) ->\n    setopts(CliSock, [{packet, http}, {packet_size, 16#4000}], SSL),\n    case do_recv(CliSock, 0,  SSL) of\n        {ok, R} when is_record(R, http_request) ->\n            R;\n        {ok, R} when is_record(R, http_response) ->\n            R;\n        {_, {http_error, \"\\r\\n\"}} ->\n            http_recv_request(CliSock, SSL);\n        {_, {http_error, \"\\n\"}} ->\n            http_recv_request(CliSock,SSL);\n        {_, {http_error, _}} ->\n            bad_request;\n        {error, closed} ->\n            closed;\n        {error, timeout} ->\n            closed;\n        _Other ->\n            error_logger:format(\"Unhandled reply fr. do_recv() ~p~n\", [_Other]),\n            exit(normal)\n    end.\n\nhttp_collect_headers(CliSock, Req, H, SSL, Count) when Count < 1000 ->\n    setopts(CliSock, [{packet, httph}, {packet_size, 16#4000}], SSL),\n    Recv = do_recv(CliSock, 0, SSL),\n    case Recv of\n        {ok, {http_header,  _Num, 'Host', _, Host}} ->\n            NewHostH = case H#headers.host of\n                           undefined ->\n                               H#headers{host = Host};\n                           {Hosts} ->\n                               H#headers{host = {[Host | Hosts]}};\n                           CurrentHost ->\n                               H#headers{host = {[Host, CurrentHost]}}\n                       end,\n            http_collect_headers(CliSock, Req, NewHostH, SSL, Count+1);\n        {ok, {http_header, _Num, 'Connection', _, Conn}} ->\n            http_collect_headers(CliSock, Req,\n                                 H#headers{connection = Conn},SSL, Count+1);\n        {ok, {http_header, _Num, 'Accept', _, Accept}} ->\n            http_collect_headers(CliSock, Req, H#headers{accept = Accept},\n                                 SSL, Count+1);\n        {ok, {http_header, _Num, 'If-Modified-Since', _, X}} ->\n            http_collect_headers(CliSock, Req,\n                                 H#headers{if_modified_since = X},SSL, Count+1);\n        {ok, {http_header, _Num, 'If-Match', _, X}} ->\n            http_collect_headers(CliSock, Req, H#headers{if_match = X},\n                                 SSL, Count+1);\n        {ok, {http_header, _Num, 'If-None-Match', _, X}} ->\n            http_collect_headers(CliSock, Req,\n                                 H#headers{if_none_match = X},SSL, Count+1);\n        {ok, {http_header, _Num, 'If-Range', _, X}} ->\n            http_collect_headers(CliSock, Req, H#headers{if_range = X},\n                                 SSL, Count+1);\n        {ok, {http_header, _Num, 'If-Unmodified-Since', _, X}} ->\n            http_collect_headers(CliSock, Req,\n                                 H#headers{if_unmodified_since = X},SSL,\n                                 Count+1);\n        {ok, {http_header, _Num, 'Range', _, X}} ->\n            http_collect_headers(CliSock, Req, H#headers{range = X},\n                                 SSL, Count+1);\n        {ok, {http_header, _Num, 'Referer',_, X}} ->\n            http_collect_headers(CliSock, Req, H#headers{referer = X},\n                                 SSL, Count+1);\n        {ok, {http_header, _Num, 'User-Agent', _, X}} ->\n            http_collect_headers(CliSock, Req, H#headers{user_agent = X},\n                                 SSL, Count+1);\n        {ok, {http_header, _Num, 'Accept-Ranges', _, X}} ->\n            http_collect_headers(CliSock, Req,\n                                 H#headers{accept_ranges = X},SSL, Count+1);\n        {ok, {http_header, _Num, 'Cookie', _, X}} ->\n            http_collect_headers(CliSock, Req,\n                                 H#headers{cookie = [X|H#headers.cookie]},\n                                 SSL, Count+1);\n        {ok, {http_header, _Num, 'Keep-Alive', _, X}} ->\n            http_collect_headers(CliSock, Req, H#headers{keep_alive = X},\n                                 SSL, Count+1);\n        {ok, {http_header, _Num, 'Content-Length', _, X}} ->\n            http_collect_headers(CliSock, Req,\n                                 H#headers{content_length = X},SSL,\n                                 Count+1);\n        {ok, {http_header, _Num, 'Content-Type', _, X}} ->\n            http_collect_headers(CliSock, Req,\n                                 H#headers{content_type = X},SSL, Count+1);\n        {ok, {http_header, _Num, 'Content-Encoding', _, X}} ->\n            http_collect_headers(CliSock, Req,\n                                 H#headers{content_encoding = X},SSL, Count+1);\n        {ok, {http_header, _Num, 'Transfer-Encoding', _, X}} ->\n            http_collect_headers(CliSock, Req,\n                                 H#headers{transfer_encoding=X},SSL, Count+1);\n        {ok, {http_header, _Num, 'Location', _, X}} ->\n            http_collect_headers(CliSock, Req, H#headers{location=X},\n                                 SSL, Count+1);\n        {ok, {http_header, _Num, 'Authorization', _, X}} ->\n            http_collect_headers(CliSock, Req,\n                                 H#headers{authorization = parse_auth(X)},\n                                 SSL, Count+1);\n        {ok, {http_header, _Num, 'X-Forwarded-For', _, X}} ->\n            case H#headers.x_forwarded_for of\n                undefined ->\n                    http_collect_headers(CliSock, Req, H#headers{x_forwarded_for=X},\n                                         SSL, Count+1);\n                PrevXF ->\n                    NewXF = join_sep([PrevXF,X], \", \"),\n                    http_collect_headers(CliSock, Req, H#headers{x_forwarded_for=NewXF},\n                                         SSL, Count+1)\n            end;\n        {ok, http_eoh} ->\n            H;\n\n        %% these are here to be a little forgiving to\n        %% bad (typically test script) clients\n        {_, {http_error, \"\\r\\n\"}} ->\n            http_collect_headers(CliSock, Req, H,SSL, Count+1);\n        {_, {http_error, \"\\n\"}} ->\n            http_collect_headers(CliSock, Req, H,SSL, Count+1);\n\n        %% auxiliary headers we don't have builtin support for\n        {ok, X} ->\n            ?Debug(\"OTHER header ~p~n\", [X]),\n            http_collect_headers(CliSock, Req,\n                                 H#headers{other=[X|H#headers.other]},\n                                 SSL, Count+1);\n        _Err ->\n            exit(normal)\n\n    end;\nhttp_collect_headers(_CliSock, Req, _H, _SSL, _Count)  ->\n    {error, {too_many_headers, Req}}.\n\n\n\nparse_auth(Orig = \"Basic \" ++ Auth64) ->\n    case decode_base64(Auth64) of\n        {error, _Err} ->\n            {undefined, undefined, Orig};\n        Auth ->\n            case string:tokens(Auth, \":\") of\n                [User, Pass ] ->\n                    {User, Pass, Orig};\n                [User, Pass0 | Extra] ->\n                    %% password can contain :\n                    Pass = join_sep([Pass0 | Extra], \":\"),\n                    {User, Pass, Orig};\n                _ ->\n                    {undefined, undefined, Orig}\n            end\n    end;\nparse_auth(Orig = \"Negotiate \" ++ _Auth64) ->\n    {undefined, undefined, Orig};\nparse_auth(Orig) ->\n    {undefined, undefined, Orig}.\n\n\ndecode_base64([]) ->\n    [];\ndecode_base64(Auth64) ->\n    decode_base64(Auth64, []).\ndecode_base64([], Acc) ->\n    lists:reverse(Acc);\ndecode_base64([Sextet1,Sextet2,$=,$=|Rest], Acc) ->\n    Bits2x6 =\n        (d(Sextet1) bsl 18) bor\n        (d(Sextet2) bsl 12),\n    Octet1 = Bits2x6 bsr 16,\n    decode_base64(Rest, [Octet1|Acc]);\ndecode_base64([Sextet1,Sextet2,Sextet3,$=|Rest], Acc) ->\n    Bits3x6 =\n        (d(Sextet1) bsl 18) bor\n        (d(Sextet2) bsl 12) bor\n        (d(Sextet3) bsl 6),\n    Octet1 = Bits3x6 bsr 16,\n    Octet2 = (Bits3x6 bsr 8) band 16#ff,\n    decode_base64(Rest, [Octet2,Octet1|Acc]);\ndecode_base64([Sextet1,Sextet2,Sextet3,Sextet4|Rest], Acc) ->\n    Bits4x6 =\n        (d(Sextet1) bsl 18) bor\n        (d(Sextet2) bsl 12) bor\n        (d(Sextet3) bsl 6) bor\n        d(Sextet4),\n    Octet1 = Bits4x6 bsr 16,\n    Octet2 = (Bits4x6 bsr 8) band 16#ff,\n    Octet3 = Bits4x6 band 16#ff,\n    decode_base64(Rest, [Octet3,Octet2,Octet1|Acc]);\ndecode_base64(_CatchAll, _Acc) ->\n    {error, bad_base64}.\n\nd(X) when X >= $A, X =<$Z -> X-65;\nd(X) when X >= $a, X =<$z -> X-71;\nd(X) when X >= $0, X =<$9 -> X+4;\nd($+)                     -> 62;\nd($/)                     -> 63;\nd(_)                      -> 63.\n\n\nflag(CurFlag, Bit, true)  -> CurFlag bor Bit;\nflag(CurFlag, Bit, false) -> CurFlag band (bnot Bit).\n\n\n%% misc debug funcs .... use from cli only\nrestart() ->\n    stop(),\n    load(),\n    start().\n\n\nmodules() ->\n    application:load(yaws),\n    M = case application:get_all_key(yaws) of\n            {ok, L} ->\n                case lists:keysearch(modules, 1, L) of\n                    {value, {modules, Mods}} -> Mods;\n                    _                        -> []\n                end;\n            _ ->\n                []\n        end,\n    M.\n\n\nload() ->\n    load(modules()).\nload(M) ->\n    lists:foreach(fun(Mod) ->\n                          ?Debug(\"Load ~p~n\", [Mod]),\n                          c:l(Mod)\n                  end, M).\n\n\n\nupto_char(Char, [Char|_]) ->\n    [];\nupto_char(Char, [H|T]) when is_integer(H) ->\n    [H|upto_char(Char, T)];\nupto_char(_, []) ->\n    [];\n%% deep lists\nupto_char(Char, [H|T]) when is_list(H) ->\n    case lists:member(Char ,H) of\n        true  -> upto_char(Char, H);\n        false -> [H, upto_char(Char, T)]\n    end.\n\n\n%% map over deep list and maintain\n%% list structure as is\ndeepmap(Fun, [H|T]) when is_list(H) ->\n    [deepmap(Fun, H) | deepmap(Fun, T)];\ndeepmap(Fun, [H|T]) ->\n    [Fun(H) | deepmap(Fun,T)];\ndeepmap(_Fun, []) ->\n    [].\n\n\nsconf_to_srvstr(SC) ->\n    redirect_scheme(SC) ++ redirect_host(SC,undefined).\n\nredirect_scheme(SC) ->\n    case {SC#sconf.ssl,SC#sconf.rmethod} of\n        {_, Method} when is_list(Method) -> Method++\"://\";\n        {undefined, _}                   -> \"http://\";\n        {_SSl, _}                        -> \"https://\"\n    end.\n\nredirect_host(SC, HostHdr) ->\n    case SC#sconf.rhost of\n        undefined ->\n            if HostHdr == undefined ->\n                    ServerName  = SC#sconf.servername,\n                    SnameNoPort = case string:chr(ServerName, $:) of\n                                      0 -> ServerName;\n                                      N -> lists:sublist(ServerName, N-1)\n                                  end,\n                    SnameNoPort ++ redirect_port(SC);\n               true ->\n                    HostHdr\n            end;\n        _ ->\n            SC#sconf.rhost\n    end.\n\nredirect_port(SC) ->\n    case {SC#sconf.rmethod, SC#sconf.ssl, SC#sconf.port} of\n        {\"https\", _, 443}    -> \"\";\n        {\"http\", _, 80}      -> \"\";\n        {_, undefined, 80}   -> \"\";\n        {_, undefined, Port} -> [$:|erlang:integer_to_list(Port)];\n        {_, _SSL, 443}       -> \"\";\n        {_, _SSL, Port}      -> [$:|erlang:integer_to_list(Port)]\n    end.\n\nredirect_scheme_port(SC) ->\n    Scheme   = redirect_scheme(SC),\n    PortPart = redirect_port(SC),\n    {Scheme, PortPart}.\n\ntmpdir() ->\n    tmpdir(filename:join([home(), \".yaws\"])).\ntmpdir(DefaultTmpDir) ->\n    case os:type() of\n        {win32,_} ->\n            case os:getenv(\"TEMP\") of\n                false ->\n                    case os:getenv(\"TMP\") of\n                        %%\n                        %% No temporary path set?\n                        %% Then try standard paths.\n                        %%\n                        false ->\n                            case file:read_file_info(\"C:/WINNT/Temp\") of\n                                {error, _} -> \"C:/WINDOWS/Temp\";\n                                {ok, _}    -> \"C:/WINNT/Temp\"\n                            end;\n                        PathTMP ->\n                            PathTMP\n                    end;\n                PathTEMP ->\n                    PathTEMP\n            end;\n        _ ->\n            DefaultTmpDir\n    end.\n\n%% mktemp function borrowed from Klacke's misc module\n%% Modified to use tmpdir/1 so it works on Windows too.\n%% Note that mktemp/2 could be exported too, but no Yaws\n%% code needs it, yet anyway.\nmktemp(Template) ->\n    mktemp(Template, file).\n\nmktemp(Template, Ret) ->\n    Tdir = tmpdir(\"/tmp\"),\n    Max = 1000,\n    mktemp(Tdir, Template, Ret, 0, Max, \"\").\n\nmktemp(Dir, Template, Ret, I, Max, Suffix) when I < Max ->\n    {X,Y,Z} = unique_triple(),\n    PostFix = erlang:integer_to_list(X) ++ \"-\" ++\n        erlang:integer_to_list(Y) ++ \"-\" ++\n        erlang:integer_to_list(Z),\n    F = filename:join(Dir, Template ++ [$_ | PostFix] ++ Suffix),\n    filelib:ensure_dir(F),\n    case file:open(F, [read, raw]) of\n        {error, enoent} when Ret == file ->\n            {ok, F};\n        {error, enoent} when Ret == fd ->\n            case file:open(F, [read, write, raw]) of\n                {ok, Fd} ->\n                    file:delete(F),\n                    {ok, Fd};\n                Err ->\n                    Err\n            end;\n        {error, enoent} when Ret == binfd ->\n            case file:open(F, [read, write, raw, binary]) of\n                {ok, Fd} ->\n                    file:delete(F),\n                    {ok, Fd};\n                Err ->\n                    Err\n            end;\n        {ok, Fd} ->\n            file:close(Fd),\n            mktemp(Dir, Template, Ret, I+1, Max, Suffix);\n        _Err ->\n            mktemp(Dir, Template, Ret, I+1, Max, Suffix)\n    end;\nmktemp(_Dir, _Template, _Ret, _I, _Max, _Suffix) ->\n    {error, too_many}.\n\n\n%% This feature is usable together with\n%% privbind and authbind on linux\nhome() ->\n    case os:getenv(\"YAWSHOME\") of\n        false -> os:getenv(\"HOME\");\n        DIR   -> DIR\n    end.\n\nid_dir(Id) ->\n    filename:join([tmpdir(), \"yaws\", to_list(Id)]).\n\nctl_file(Id) ->\n    filename:join([id_dir(Id), \"CTL\"]).\n\n\neat_crnl(Fd,SSL) ->\n    setopts(Fd, [{packet, line}],SSL),\n    case do_recv(Fd,0, SSL) of\n        {ok, <<13,10>>} -> ok;\n        {ok, [13,10]}   -> ok;\n        _               -> exit(normal)\n    end.\n\n\nget_chunk_num(Fd, SSL) ->\n    {N, _} = get_chunk_header(Fd, SSL),\n    N.\n\nget_chunk_header(Fd, SSL) ->\n    case do_recv(Fd, 0, SSL) of\n        {ok, Data} ->\n            Line = if is_binary(Data) -> binary_to_list(Data);\n                      true            -> Data\n                   end,\n            ?Debug(\"Get chunk num from line ~p~n\",[Line]),\n            {N, Exts} = split_at(Line, $;),\n            {erlang:list_to_integer(strip_spaces(N),16), strip_spaces(Exts)};\n        {error, _Rsn} ->\n            exit(normal)\n    end.\n\n\nget_chunk(_Fd, N, N, _) ->\n    [];\nget_chunk(Fd, N, Asz,SSL) ->\n    case do_recv(Fd, N, SSL) of\n        {ok, Bin} ->\n            SZ = size(Bin),\n            [Bin|get_chunk(Fd, N, SZ+Asz,SSL)];\n        _ ->\n            exit(normal)\n    end.\n\nget_chunk_trailer(Fd, SSL) ->\n    Hdrs = #headers{},\n    case http_collect_headers(Fd, undefined, Hdrs, SSL, 0) of\n        {error,_} -> exit(normal);\n        Hdrs      -> <<>>;\n        NewHdrs   -> {<<>>, NewHdrs}\n    end.\n\n%% split inputstring at first occurrence of Char\nsplit_at(String, Char) ->\n    split_at(String, Char, []).\nsplit_at([H|T], H, Ack) ->\n    {lists:reverse(Ack), T};\nsplit_at([H|T], Char, Ack) ->\n    split_at(T, Char, [H|Ack]);\nsplit_at([], _Char, Ack) ->\n    {lists:reverse(Ack), []}.\n\n%% insert an elemant at a given position into a list\ninsert_at(Elm, 0, Ls) ->\n    Ls ++ [Elm];\ninsert_at(Elm, Pos, Ls) ->\n    insert_at(Elm, Pos, Ls, []).\n\ninsert_at(Elm, _, [], Res) ->\n    lists:reverse([Elm|Res]);\ninsert_at(Elm, 1, Ls, Res) ->\n    lists:reverse([Elm|Res]) ++ Ls;\ninsert_at(Elm, Pos, [H|T], Res) ->\n    insert_at(Elm, Pos-1, T, [H|Res]).\n\n\n\n%% Parse an Ip address or an Ip address range\n%% Return Ip || {IpMin, IpMax} where:\n%%     Ip, IpMin, IpMax ::= ip_address()\nparse_ipmask(Str) when is_list(Str) ->\n    case string:tokens(Str, [$/]) of\n        [IpStr] ->\n            case inet_parse:address(IpStr) of\n                {ok, Ip}        -> Ip;\n                {error, Reason} -> throw({error, Reason})\n            end;\n        [IpStr, NetMask] ->\n            {Type, IpInt} = ip_to_integer(IpStr),\n            MaskInt       = netmask_to_integer(Type, NetMask),\n            case netmask_to_wildcard(Type, MaskInt) of\n                0 ->\n                    integer_to_ip(Type, IpInt);\n                Wildcard when Type =:= ipv4 ->\n                    NetAddr   = (IpInt band MaskInt),\n                    Broadcast = NetAddr + Wildcard,\n                    IpMin     = NetAddr + 1,\n                    IpMax     = Broadcast - 1,\n                    {integer_to_ip(ipv4, IpMin), integer_to_ip(ipv4, IpMax)};\n                Wildcard when Type =:= ipv6 ->\n                    NetAddr   = (IpInt band MaskInt),\n                    IpMin = NetAddr,\n                    IpMax = NetAddr + Wildcard,\n                    {integer_to_ip(ipv6, IpMin), integer_to_ip(ipv6, IpMax)}\n            end;\n        _ ->\n            throw({error, einval})\n    end;\nparse_ipmask(_) ->\n    throw({error, einval}).\n\n\n-define(MAXBITS_IPV4, 32).\n-define(MASK_IPV4,    16#FFFFFFFF).\n-define(MAXBITS_IPV6, 128).\n-define(MASK_IPV6,    16#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF).\n\nip_to_integer(Str) when is_list(Str) ->\n    case inet_parse:address(Str) of\n        {ok, Ip}        -> ip_to_integer(Ip);\n        {error, Reason} -> throw({error, Reason})\n    end;\nip_to_integer({N1,N2,N3,N4}) ->\n    <<Int:32>> = <<N1:8, N2:8, N3:8, N4:8>>,\n    if\n        (Int bsr ?MAXBITS_IPV4) == 0 -> {ipv4, Int};\n        true -> throw({error, einval})\n    end;\nip_to_integer({N1,N2,N3,N4,N5,N6,N7,N8}) ->\n    <<Int:128>> = <<N1:16, N2:16, N3:16, N4:16, N5:16, N6:16, N7:16, N8:16>>,\n    if\n        (Int bsr ?MAXBITS_IPV6) == 0 -> {ipv6, Int};\n        true -> throw({error, einval})\n    end;\nip_to_integer(_) ->\n    throw({error, einval}).\n\ninteger_to_ip(ipv4, I) when is_integer(I), I =< ?MASK_IPV4 ->\n    <<N1:8, N2:8, N3:8, N4:8>> = <<I:32>>,\n    {N1, N2, N3, N4};\ninteger_to_ip(ipv6, I) when is_integer(I), I =< ?MASK_IPV6 ->\n    <<N1:16, N2:16, N3:16, N4:16, N5:16, N6:16, N7:16, N8:16>> = <<I:128>>,\n    {N1, N2, N3, N4, N5, N6, N7, N8};\ninteger_to_ip(_, _) ->\n    throw({error, einval}).\n\nnetmask_to_integer(Type, NetMask) ->\n    case catch erlang:list_to_integer(NetMask) of\n        I when is_integer(I) ->\n            case Type of\n                ipv4 -> (1 bsl ?MAXBITS_IPV4) - (1 bsl (?MAXBITS_IPV4 - I));\n                ipv6 -> (1 bsl ?MAXBITS_IPV6) - (1 bsl (?MAXBITS_IPV6 - I))\n            end;\n        _ ->\n            case ip_to_integer(NetMask) of\n                {Type, MaskInt} -> MaskInt;\n                _               -> throw({error, einval})\n            end\n    end.\n\nnetmask_to_wildcard(ipv4, Mask) -> ((1 bsl ?MAXBITS_IPV4) - 1) bxor Mask;\nnetmask_to_wildcard(ipv6, Mask) -> ((1 bsl ?MAXBITS_IPV6) - 1) bxor Mask.\n\n\n%% Compare an ip to another ip or a range of ips\nmatch_ipmask(Ip, Ip) ->\n    true;\nmatch_ipmask(Ip, {IpMin, IpMax}) ->\n    case compare_ips(Ip, IpMin) of\n        error -> false;\n        less  -> false;\n        _ ->\n            case compare_ips(Ip, IpMax) of\n                error   -> false;\n                greater -> false;\n                _       -> true\n            end\n    end;\nmatch_ipmask(_, _) ->\n    false.\n\ncompare_ips({A,B,C,D},          {A,B,C,D})                       -> equal;\ncompare_ips({A,B,C,D,E,F,G,H},  {A,B,C,D,E,F,G,H})               -> equal;\ncompare_ips({A,B,C,D1},         {A,B,C,D2})         when D1 < D2 -> less;\ncompare_ips({A,B,C,D1},         {A,B,C,D2})         when D1 > D2 -> greater;\ncompare_ips({A,B,C1,_},         {A,B,C2,_})         when C1 < C2 -> less;\ncompare_ips({A,B,C1,_},         {A,B,C2,_})         when C1 > C2 -> greater;\ncompare_ips({A,B1,_,_},         {A,B2,_,_})         when B1 < B2 -> less;\ncompare_ips({A,B1,_,_},         {A,B2,_,_})         when B1 > B2 -> greater;\ncompare_ips({A1,_,_,_},         {A2,_,_,_})         when A1 < A2 -> less;\ncompare_ips({A1,_,_,_},         {A2,_,_,_})         when A1 > A2 -> greater;\ncompare_ips({A,B,C,D,E,F,G,H1}, {A,B,C,D,E,F,G,H2}) when H1 < H2 -> less;\ncompare_ips({A,B,C,D,E,F,G,H1}, {A,B,C,D,E,F,G,H2}) when H1 > H2 -> greater;\ncompare_ips({A,B,C,D,E,F,G1,_}, {A,B,C,D,E,F,G2,_}) when G1 < G2 -> less;\ncompare_ips({A,B,C,D,E,F,G1,_}, {A,B,C,D,E,F,G2,_}) when G1 > G2 -> greater;\ncompare_ips({A,B,C,D,E,F1,_,_}, {A,B,C,D,E,F2,_,_}) when F1 < F2 -> less;\ncompare_ips({A,B,C,D,E,F1,_,_}, {A,B,C,D,E,F2,_,_}) when F1 > F2 -> greater;\ncompare_ips({A,B,C,D,E1,_,_,_}, {A,B,C,D,E2,_,_,_}) when E1 < E2 -> less;\ncompare_ips({A,B,C,D,E1,_,_,_}, {A,B,C,D,E2,_,_,_}) when E1 > E2 -> greater;\ncompare_ips({A,B,C,D1,_,_,_,_}, {A,B,C,D2,_,_,_,_}) when D1 < D2 -> less;\ncompare_ips({A,B,C,D1,_,_,_,_}, {A,B,C,D2,_,_,_,_}) when D1 > D2 -> greater;\ncompare_ips({A,B,C1,_,_,_,_,_}, {A,B,C2,_,_,_,_,_}) when C1 < C2 -> less;\ncompare_ips({A,B,C1,_,_,_,_,_}, {A,B,C2,_,_,_,_,_}) when C1 > C2 -> greater;\ncompare_ips({A,B1,_,_,_,_,_,_}, {A,B2,_,_,_,_,_,_}) when B1 < B2 -> less;\ncompare_ips({A,B1,_,_,_,_,_,_}, {A,B2,_,_,_,_,_,_}) when B1 > B2 -> greater;\ncompare_ips({A1,_,_,_,_,_,_,_}, {A2,_,_,_,_,_,_,_}) when A1 < A2 -> less;\ncompare_ips({A1,_,_,_,_,_,_,_}, {A2,_,_,_,_,_,_,_}) when A1 > A2 -> greater;\ncompare_ips(_,                  _)                               -> error.\n\n\n%% Use  IANA range for dynamic or private ports (49152 -> 65535)\nfind_private_port() ->\n    Start = case application:get_env(kernel, next_yaws_private_port) of\n                {ok, Port} -> Port;\n                undefined  -> 49152\n            end,\n    End = 65535,\n    find_private_port(Start, End).\n\nfind_private_port(Port, End) when Port > End ->\n    {error, notfound};\nfind_private_port(Port, End) ->\n    case gen_tcp:listen(Port, [{ip, {0,0,0,0}}]) of\n        {ok, Sock} ->\n            gen_tcp:close(Sock),\n            application:set_env(kernel, next_yaws_private_port, Port+1),\n            {ok, Port};\n        {error, _} ->\n            find_private_port(Port+1, End)\n    end.\n\n%% ----\nget_app_subdir(SubDir) when is_atom(SubDir) ->\n    filename:join(get_app_dir(), atom_to_list(SubDir)).\n\nget_app_dir() ->\n    case application:get_env(yaws, app_dir) of\n        {ok, AppDir} ->\n            AppDir;\n        undefined ->\n            Path = case code:which(?MODULE) of\n                       cover_compiled -> code:where_is_file(\"yaws.beam\");\n                       Dir            -> Dir\n                   end,\n            AppDir = filename:absname(filename:dirname(filename:dirname(Path))),\n            application:set_env(yaws, app_dir, AppDir),\n            AppDir\n    end.\n\nget_ebin_dir() ->\n    get_app_subdir(ebin).\n\nget_priv_dir() ->\n    get_app_subdir(priv).\n\nget_inc_dir() ->\n    get_app_subdir(include).\n"
  },
  {
    "path": "contrib/yaws/src/yaws_api.erl",
    "content": "%%----------------------------------------------------------------------\n%%% File    : yaws_api.erl\n%%% Author  : Claes Wikstrom <klacke@hyber.org>\n%%% Purpose :\n%%% Created : 24 Jan 2002 by Claes Wikstrom <klacke@hyber.org>\n%%%----------------------------------------------------------------------\n\n-module(yaws_api).\n-author('klacke@hyber.org').\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_debug.hrl\").\n\n\n\n-export([parse_query/1, parse_post/1,\n         parse_multipart_post/1, parse_multipart_post/2,\n         parse_multipart/2, parse_multipart/3]).\n-export([code_to_phrase/1, ssi/2, redirect/1]).\n-export([setcookie/2, setcookie/3, setcookie/4, setcookie/5, setcookie/6]).\n-deprecated([{setcookie, 2, eventually},\n             {setcookie, 3, eventually},\n             {setcookie, 4, eventually},\n             {setcookie, 5, eventually},\n             {setcookie, 6, eventually}]).\n-export([set_cookie/3]).\n-export([pre_ssi_files/2,  pre_ssi_string/1, pre_ssi_string/2,\n         set_content_type/1,\n         htmlize/1, htmlize_char/1, f/2, fl/1]).\n-export([find_cookie_val/2, secs/0,\n         url_decode/1, url_decode_q_split/1, url_decode_with_encoding/2,\n         url_encode/1, parse_url/1, parse_url/2, format_url/1,\n         format_partial_url/2]).\n-export([is_absolute_URI/1]).\n-export([path_norm/1, path_norm_reverse/1,\n         sanitize_file_name/1]).\n-export([get_line/1, mime_type/1, mime_type/2]).\n-export([stream_chunk_deliver/2, stream_chunk_deliver_blocking/2,\n         stream_chunk_end/1]).\n-export([stream_process_deliver/2, stream_process_deliver_chunk/2,\n         stream_process_deliver_final_chunk/2, stream_process_end/2]).\n-export([websocket_send/2, websocket_close/1, websocket_close/2]).\n-export([get_sslsocket/1]).\n-export([new_cookie_session/1, new_cookie_session/2, new_cookie_session/3,\n         cookieval_to_opaque/1, request_url/1,\n         print_cookie_sessions/0,\n         replace_cookie_session/2, replace_cookie_session/3,\n         delete_cookie_session/1]).\n\n-export([getconf/0,\n         setconf/2,\n         get_listen_port/1,\n         embedded_start_conf/1, embedded_start_conf/2,\n         embedded_start_conf/3, embedded_start_conf/4]).\n\n-export([set_status_code/1, reformat_header/1, reformat_header/2,\n         reformat_request/1, reformat_response/1, reformat_url/1]).\n\n-export([set_trace/1,\n         set_tty_trace/1,\n         set_access_log/1]).\n\n-export([call_cgi/2, call_cgi/3]).\n\n-export([call_fcgi_responder/1, call_fcgi_responder/2,\n         call_fcgi_authorizer/1, call_fcgi_authorizer/2]).\n\n-export([ehtml_expand/1, ehtml_expander/1, ehtml_apply/2,\n         ehtml_expander_test/0]).\n\n-export([parse_set_cookie/1, parse_cookie/1, format_set_cookie/1,\n         format_cookie/1, postvar/2, queryvar/2, getvar/2]).\n\n-export([binding/1,binding_exists/1,\n         dir_listing/1, dir_listing/2, redirect_self/1]).\n\n-export([arg_clisock/1, arg_client_ip_port/1, arg_headers/1, arg_req/1,\n         arg_orig_req/1, arg_clidata/1, arg_server_path/1, arg_querydata/1,\n         arg_appmoddata/1, arg_docroot/1, arg_docroot_mount/1, arg_fullpath/1,\n         arg_cont/1, arg_state/1, arg_pid/1, arg_opaque/1, arg_appmod_prepath/1,\n         arg_prepath/1,\n         arg_pathinfo/1]).\n-export([http_request_method/1, http_request_path/1, http_request_version/1,\n         http_response_version/1, http_response_status/1,\n         http_response_phrase/1,\n         headers_connection/1, headers_accept/1, headers_host/1,\n         headers_if_modified_since/1, headers_if_match/1,\n         headers_if_none_match/1,\n         headers_if_range/1, headers_if_unmodified_since/1, headers_range/1,\n         headers_referer/1, headers_user_agent/1, headers_accept_ranges/1,\n         headers_cookie/1, headers_keep_alive/1, headers_location/1,\n         headers_content_length/1, headers_content_type/1,\n         headers_content_encoding/1, headers_authorization/1,\n         headers_transfer_encoding/1, headers_x_forwarded_for/1,\n         headers_other/1]).\n\n-export([set_header/2, set_header/3, merge_header/2, merge_header/3,\n         get_header/2, get_header/3, delete_header/2]).\n\n-import(lists, [flatten/1, reverse/1]).\n\n%% These are a bunch of accessor functions that are useful inside\n%% yaws scripts.\n\narg_clisock(#arg{clisock = X}) -> X.\narg_client_ip_port(#arg{client_ip_port = X}) -> X.\narg_headers(#arg{headers = X}) -> X.\narg_req(#arg{req = X}) -> X.\narg_orig_req(#arg{orig_req = X}) -> X.\narg_clidata(#arg{clidata = X}) -> X.\narg_server_path(#arg{server_path = X}) -> X.\narg_querydata(#arg{querydata = X}) -> X.\narg_appmoddata(#arg{appmoddata = X}) -> X.\narg_docroot(#arg{docroot = X}) -> X.\narg_docroot_mount(#arg{docroot_mount = X}) -> X.\narg_fullpath(#arg{fullpath = X}) -> X.\narg_cont(#arg{cont = X}) -> X.\narg_state(#arg{state = X}) -> X.\narg_pid(#arg{pid = X}) -> X.\narg_opaque(#arg{opaque = X}) -> X.\narg_appmod_prepath(#arg{appmod_prepath = X}) -> X.\narg_prepath(#arg{prepath = X}) -> X.\narg_pathinfo(#arg{pathinfo = X}) ->  X.\n\nhttp_request_method(#http_request{method = X}) -> X.\nhttp_request_path(#http_request{path = X}) -> X.\nhttp_request_version(#http_request{version = X}) -> X.\n\nhttp_response_version(#http_response{version = X}) -> X.\nhttp_response_status(#http_response{status = X}) -> X.\nhttp_response_phrase(#http_response{phrase = X}) -> X.\n\nheaders_connection(#headers{connection = X}) -> X.\nheaders_accept(#headers{accept = X}) -> X.\nheaders_host(#headers{host = X}) -> X.\nheaders_if_modified_since(#headers{if_modified_since = X}) -> X.\nheaders_if_match(#headers{if_match = X}) -> X.\nheaders_if_none_match(#headers{if_none_match = X}) -> X.\nheaders_if_range(#headers{if_range = X}) -> X.\nheaders_if_unmodified_since(#headers{if_unmodified_since = X}) -> X.\nheaders_range(#headers{range = X}) -> X.\nheaders_referer(#headers{referer = X}) -> X.\nheaders_user_agent(#headers{user_agent = X}) -> X.\nheaders_accept_ranges(#headers{accept_ranges = X}) -> X.\nheaders_cookie(#headers{cookie = X}) -> X.\nheaders_keep_alive(#headers{keep_alive = X}) -> X.\nheaders_location(#headers{location = X}) -> X.\nheaders_content_length(#headers{content_length = X}) -> X.\nheaders_content_type(#headers{content_type = X}) -> X.\nheaders_content_encoding(#headers{content_encoding = X}) -> X.\nheaders_authorization(#headers{authorization = X}) -> X.\nheaders_transfer_encoding(#headers{transfer_encoding = X}) -> X.\nheaders_x_forwarded_for(#headers{x_forwarded_for = X}) -> X.\nheaders_other(#headers{other = X}) -> X.\n\n\n%% parse the command line query data\nparse_query(Arg) ->\n    case get(query_parse) of\n        undefined ->\n            Res = case Arg#arg.querydata of\n                      [] -> [];\n                      D  -> parse_post_data_urlencoded(D)\n                  end,\n            put(query_parse, Res),\n            Res;\n        Res ->\n            Res\n    end.\n\n%% parse url encoded POST data\nparse_post(Arg) ->\n    case get(post_parse) of\n        undefined ->\n            H = Arg#arg.headers,\n            Res = case H#headers.content_type of\n                      \"application/x-www-form-urlencoded\"++_ ->\n                          case Arg#arg.clidata of\n                              [] -> [];\n                              D  -> parse_post_data_urlencoded(D)\n                          end;\n                      _ ->\n                          []\n                  end,\n            put(post_parse, Res),\n            Res;\n        Res ->\n            Res\n    end.\n\n\n%%\n%% Changed implementation of multipart form data. There is a new config\n%% parameter called\n%%\n%%      partial_post_size\n%%\n%% which if set to an integer value\n%% will cause the content of the post content to be sent to the out/1\n%% function in chunks of this size.\n%%\n%% It is possible to get the server to maintain a state on behalf of the\n%% out/1 user by returning {get_more, Cont, State}.\n%%\n%%\n%% yaws_api:parse_multipart_post/1 will return either:\n%%\n%% {cont, Cont, Res} where Res is new result(s) from this segment. This\n%% indicates that there is more data to come and the out/1 function\n%% should return {get_more, Cont, User_state} where User_state might\n%% usefully be a File Descriptor.\n%%\n%% {result, Res} if this is the last (or only) segment.\n%%\n%% or {error, Reason} if an error occurred during the parsing.\n%%\n%% Res is a list of {head, {Name, Hdrs}} | {part_body, Binary} | {body, Binary}\n%%\n%% Example usage could be:\n%%\n%% <erl>\n%%\n%% out(A) ->\n%%        case yaws_api:parse_multipart_post(A) of\n%%             {cont, Cont, Res} ->\n%%                    St = handle_res(A, Res),\n%%                    {get_more, Cont, St};\n%%             {result, Res} ->\n%%                    handle_res(A, Res),\n%%                    {html, f(\"<pre>Done </pre>\",[])};\n%%             {error, Reason} ->\n%%                    {html, f(\"An error occured: ~p\", [Reason])}\n%%        end.\n%%\n%% handle_res(A, [{head, {Name, Hdrs}}|T]) ->\n%%      io:format(\"head:~p~n\",[Name]),\n%%      handle_res(A, T);\n%% handle_res(A, [{part_body, Data}|T]) ->\n%%      io:format(\"part_body:~p~n\",[Data]),\n%%      handle_res(A, T);\n%% handle_res(A, [{body, Data}|T]) ->\n%%      io:format(\"body:~p~n\",[Data]),\n%%      handle_res(A, T);\n%% handle_res(A, []) ->\n%%      io:format(\"End_res~n\").\n%%\n%% </erl>\n\nparse_multipart_post(Arg) ->\n    parse_multipart_post(Arg, [list]).\nparse_multipart_post(Arg, Options) ->\n    H = Arg#arg.headers,\n    case H#headers.content_type of\n        undefined ->\n            {error, no_content_type};\n        \"multipart/form-data\"++Line ->\n            case Arg#arg.cont of\n                {cont, Cont} ->\n                    parse_multipart(un_partial(Arg#arg.clidata), {cont, Cont});\n                undefined ->\n                    LineArgs = parse_arg_line(Line),\n                    {value, {_, Boundary}} = lists:keysearch(\"boundary\", 1,\n                                                             LineArgs),\n                    parse_multipart(un_partial(Arg#arg.clidata),\n                                    Boundary, Options)\n            end;\n        _Other ->\n            {error, no_multipart_form_data}\n    end.\n\nun_partial({partial, Bin}) ->\n    Bin;\nun_partial(Bin) ->\n    Bin.\n\nparse_arg_line(Line) ->\n    parse_arg_line(Line, []).\n\nparse_arg_line([],Acc) -> Acc;\nparse_arg_line([$ |Line], Acc) ->\n    parse_arg_line(Line, Acc);\nparse_arg_line([$;|Line], Acc) ->\n    {KV,Rest} = parse_arg_key(Line, [], []),\n    parse_arg_line(Rest, [KV|Acc]).\n\n%%\n\nparse_arg_key([], Key, Value) ->\n    make_parse_line_reply(Key, Value, []);\nparse_arg_key([$;|Line], Key, Value) ->\n    make_parse_line_reply(Key, Value, [$;|Line]);\nparse_arg_key([$ |Line], Key, Value) ->\n    parse_arg_key(Line, Key, Value);\nparse_arg_key([$=|Line], Key, Value) ->\n    parse_arg_value(Line, Key, Value, false, false);\nparse_arg_key([C|Line], Key, Value) ->\n    parse_arg_key(Line, [C|Key], Value).\n\n%%\n%% We need to deal with quotes and initial spaces here.\n%% parse_arg_value(String, Key, ValueAcc, InQuoteBool, InValueBool)\n%%\n\nparse_arg_value([], Key, Value, _, _) ->\n    make_parse_line_reply(Key, Value, []);\nparse_arg_value([$\\\\,$\"], Key, Value, _, _) ->\n    make_parse_line_reply(Key, [$\\\\|Value], []);\nparse_arg_value([$\\\\,$\"|Line], Key, Value, Quote, Begun) ->\n    parse_arg_value(Line, Key, [$\"|Value], Quote, Begun);\nparse_arg_value([$\"|Line], Key, Value, false, _) ->\n    parse_arg_value(Line, Key, Value, true, true);\nparse_arg_value([$\"], Key, Value, true, _) ->\n    make_parse_line_reply(Key, Value, []);\nparse_arg_value([$\",$;|Line], Key, Value, true, _) ->\n    make_parse_line_reply(Key, Value, [$;|Line]);\nparse_arg_value([$;|Line], Key, Value, false, _) ->\n    make_parse_line_reply(Key, Value, [$;|Line]);\nparse_arg_value([$ |Line], Key, Value, false, true) ->\n    make_parse_line_reply(Key, Value, Line);\nparse_arg_value([$ |Line], Key, Value, false, false) ->\n    parse_arg_value(Line, Key, Value, false, false);\nparse_arg_value([C|Line], Key, Value, Quote, _) ->\n    parse_arg_value(Line, Key, [C|Value], Quote, true).\n\n\n%%\n\nmake_parse_line_reply(Key, Value, Rest) ->\n    {{yaws:funreverse(Key, fun yaws:to_lowerchar/1),\n      lists:reverse(Value)}, Rest}.\n\n\n-record(mp_parse_state, {\n          state,\n          boundary_ctx,\n          boundary_len,\n          hdr_end_ctx,\n          old_data,\n          data_type\n         }).\n\n%% Stateful parser of multipart data - allows easy re-entry\nparse_multipart(Data, St) ->\n    parse_multipart(Data, St, [list]).\nparse_multipart(Data, St, Options) ->\n    case parse_multi(Data, St, Options) of\n        {cont, St2, Res} -> {cont, {cont, St2}, lists:reverse(Res)};\n        {result, Res}    -> {result, lists:reverse(Res)};\n        {error, Reason}  -> {error, Reason}\n    end.\n\nparse_multi(Data, #mp_parse_state{state=boundary}=ParseState, Acc) ->\n    %% Find the beginning of the next part or the last boundary\n    case binary:match(Data, ParseState#mp_parse_state.boundary_ctx) of\n        {Pos, Len} ->\n            %% If Pos != 0, ignore data preceding the boundary\n            case Data of\n                <<_:Pos/binary, Rest/binary>> when size(Rest) < Len+2 ->\n                    %% Not enough data to tell if it is the last boundary or not\n                    {cont, ParseState#mp_parse_state{old_data=Rest}, Acc};\n                <<_:Pos/binary, _:Len/binary, \"\\r\\n\", Rest/binary>> ->\n                    %% It is not the last boundary, so parse the next part\n                    NPState = ParseState#mp_parse_state{state=start_headers},\n                    parse_multi(Rest, NPState, Acc);\n                <<_:Pos/binary, _:Len/binary, \"--\\r\\n\", _/binary>> ->\n                    %% Match on the last boundary and ignore remaining data\n                    {result, Acc};\n                <<_:Pos/binary, Boundary:Len/binary, \"--\", Rest/binary>> when size(Rest) < 2 ->\n                    %% Partial match on the last boundary; need more data\n\t\t    {cont, ParseState#mp_parse_state{old_data = <<Boundary/binary, \"--\", Rest/binary>>}, Acc};\n                _ ->\n                    {error, malformed_multipart_post}\n            end;\n        nomatch ->\n            %% No boundary found, request more data. Here we keep just enough\n            %% data to match on the boundary the next time\n            DLen = size(Data),\n            BLen = ParseState#mp_parse_state.boundary_len,\n            SkipLen = erlang:max(DLen - BLen, 0),\n            KeepLen = erlang:min(BLen, DLen),\n            <<_:SkipLen/binary, OldData:KeepLen/binary>> = Data,\n            {cont, ParseState#mp_parse_state{old_data=OldData}, Acc}\n    end;\n\nparse_multi(Data, #mp_parse_state{state=start_headers}=ParseState, Acc) ->\n    parse_multi(Data, ParseState, Acc, [], []);\n\nparse_multi(Data, #mp_parse_state{state=body}=ParseState, Acc) ->\n    %% Find the end of this part (i.e the next boundary)\n    case binary:match(Data, ParseState#mp_parse_state.boundary_ctx) of\n        {Pos, _Len} ->\n            %% Extract the body and keep the boundary\n            <<Body:Pos/binary, Rest/binary>> = Data,\n            BodyData = case ParseState#mp_parse_state.data_type of\n                           list   -> binary_to_list(Body);\n                           binary -> Body\n                       end,\n            NAcc = [{body, BodyData}|Acc],\n            NParseState = ParseState#mp_parse_state{state=boundary},\n            parse_multi(Rest, NParseState, NAcc);\n        nomatch ->\n            %% No boundary found, request more data.\n            DLen = size(Data),\n            BLen = ParseState#mp_parse_state.boundary_len,\n            SkipLen = erlang:max(DLen - BLen, 0),\n            KeepLen = erlang:min(BLen, DLen),\n            <<PartData:SkipLen/binary, OldData:KeepLen/binary>> = Data,\n            NParseState = ParseState#mp_parse_state{state=body,\n                                                    old_data=OldData},\n            BodyData = case ParseState#mp_parse_state.data_type of\n                           list   -> binary_to_list(PartData);\n                           binary -> PartData\n                       end,\n            {cont, NParseState, [{part_body, BodyData}|Acc]}\n    end;\n\nparse_multi(Data, {cont, #mp_parse_state{old_data=OldData}=ParseState}, _) ->\n    %% Reentry point\n    NData = <<OldData/binary, Data/binary>>,\n    parse_multi(NData, ParseState, []);\n\nparse_multi(Data, Boundary, Options) ->\n    %% Initial entry point\n    FullBoundary = list_to_binary([\"\\r\\n--\", Boundary]),\n    BoundaryCtx  = binary:compile_pattern(FullBoundary),\n    HdrEndCtx    = binary:compile_pattern(<<\"\\r\\n\\r\\n\">>),\n    DataType     = lists:foldl(fun(_,      list)      -> list;\n\t\t\t\t  (list,   _)         -> list;\n\t\t\t\t  (binary, undefined) -> binary;\n\t\t\t\t  (_,      Acc)       -> Acc\n\t\t\t       end, undefined, Options),\n    ParseState = #mp_parse_state{state        = boundary,\n                                 boundary_ctx = BoundaryCtx,\n                                 boundary_len = size(FullBoundary),\n                                 hdr_end_ctx  = HdrEndCtx,\n                                 data_type    = DataType},\n    parse_multi(<<\"\\r\\n\", Data/binary>>, ParseState, []).\n\n\nparse_multi(Data, #mp_parse_state{state=start_headers}=ParseState,\n            Acc, [], []) ->\n    %% Find the end of headers for this part\n    case binary:match(Data, ParseState#mp_parse_state.hdr_end_ctx) of\n        {_Pos, _Len} ->\n            %% We have all headers, we can parse it\n            NParseState = ParseState#mp_parse_state{state=headers},\n            parse_multi(Data, NParseState, Acc, [], []);\n        nomatch ->\n            {cont, ParseState#mp_parse_state{old_data=Data}, Acc}\n    end;\nparse_multi(Data, #mp_parse_state{state=headers}=ParseState, Acc, Name, Hdrs) ->\n    case erlang:decode_packet(httph_bin, Data, [{packet_size, 16#4000}]) of\n        {ok, http_eoh, Rest} ->\n            %% All headers are parsed, get the body now\n            Head = case Name of\n                       [] -> lists:reverse(Hdrs);\n                       _  -> {Name, lists:reverse(Hdrs)}\n                   end,\n            NParseState = ParseState#mp_parse_state{state=body},\n            parse_multi(Rest, NParseState, [{head, Head}|Acc]);\n        {ok, {http_header, _, Hdr, _, HdrVal}, Rest} when is_atom(Hdr) ->\n            Header = {case Hdr of\n                          'Content-Type' -> content_type;\n                          Else           -> Else\n                      end,\n                      binary_to_list(HdrVal)},\n            parse_multi(Rest, ParseState, Acc, Name, [Header|Hdrs]);\n        {ok, {http_header, _, Hdr, _, HdrVal}, Rest} ->\n            HdrValStr = binary_to_list(HdrVal),\n            case yaws:to_lower(binary_to_list(Hdr)) of\n                \"content-disposition\" ->\n                    \"form-data\"++Params = HdrValStr,\n                    Parameters = parse_arg_line(Params),\n                    {_, NewName} = lists:keyfind(\"name\", 1, Parameters),\n                    parse_multi(Rest, ParseState, Acc,\n                                NewName, Parameters++Hdrs);\n                LowerHdr ->\n                    parse_multi(Rest, ParseState, Acc,\n                                Name, [{LowerHdr, HdrValStr}|Hdrs])\n            end;\n        _ ->\n            {error, malformed_multipart_post}\n    end.\n\n%% parse POST data when ENCTYPE is unset or\n%% Content-type: application/x-www-form-urlencoded\n%% Bin is the content of ARG#arg.clidata\n%% the alternative is\n%% Content-type: multipart/form-data; boundary=-------------------7cd1d6371ec\n%% which is used for file upload\n\nparse_post_data_urlencoded(Bin) ->\n    do_parse_spec(Bin, nokey, [], key).\n\n\n%% It will return a [{Key, Value}] list from the post data\n\ndo_parse_spec(<<$%, Hi:8, Lo:8, Tail/binary>>, Last, Cur, State)\n    when Hi /= $u ->\n    Hex = yaws:hex_to_integer([Hi, Lo]),\n    do_parse_spec(Tail, Last, [ Hex | Cur],  State);\n\ndo_parse_spec(<<$&, Tail/binary>>, _Last , Cur,  key) ->\n    [{lists:reverse(Cur), undefined} |\n     do_parse_spec(Tail, nokey, [], key)];  %% cont keymode\n\ndo_parse_spec(<<$&, Tail/binary>>, Last, Cur, value) ->\n    V = {Last, lists:reverse(Cur)},\n    [V | do_parse_spec(Tail, nokey, [], key)];\n\ndo_parse_spec(<<$+, Tail/binary>>, Last, Cur,  State) ->\n    do_parse_spec(Tail, Last, [$\\s|Cur], State);\n\ndo_parse_spec(<<$=, Tail/binary>>, _Last, Cur, key) ->\n    do_parse_spec(Tail, lists:reverse(Cur), [], value); %% change mode\n\ndo_parse_spec(<<$%, $u, A:8, B:8,C:8,D:8, Tail/binary>>,\n               Last, Cur, State) ->\n    %% non-standard encoding for Unicode characters: %uxxxx,\n    Hex = yaws:hex_to_integer([A,B,C,D]),\n    do_parse_spec(Tail, Last, [ Hex | Cur],  State);\n\ndo_parse_spec(<<H:8, Tail/binary>>, Last, Cur, State) ->\n    do_parse_spec(Tail, Last, [H|Cur], State);\ndo_parse_spec(<<>>, nokey, Cur, _State) ->\n    [{lists:reverse(Cur), undefined}];\ndo_parse_spec(<<>>, Last, Cur, _State) ->\n    [{Last, lists:reverse(Cur)}];\ndo_parse_spec(undefined,_,_,_) ->\n    [];\ndo_parse_spec(QueryList, Last, Cur, State) when is_list(QueryList) ->\n    do_parse_spec(list_to_binary(QueryList), Last, Cur, State).\n\n\ncode_to_phrase(100) -> \"Continue\";\ncode_to_phrase(101) -> \"Switching Protocols \";\ncode_to_phrase(102) -> \"Processing\";\ncode_to_phrase(200) -> \"OK\";\ncode_to_phrase(201) -> \"Created\";\ncode_to_phrase(202) -> \"Accepted\";\ncode_to_phrase(203) -> \"Non-Authoritative Information\";\ncode_to_phrase(204) -> \"No Content\";\ncode_to_phrase(205) -> \"Reset Content\";\ncode_to_phrase(206) -> \"Partial Content\";\ncode_to_phrase(207) -> \"Multi-Status\";\ncode_to_phrase(208) -> \"Already Reported\";\ncode_to_phrase(226) -> \"IM Used\";\ncode_to_phrase(300) -> \"Multiple Choices\";\ncode_to_phrase(301) -> \"Moved Permanently\";\ncode_to_phrase(302) -> \"Found\";\ncode_to_phrase(303) -> \"See Other\";\ncode_to_phrase(304) -> \"Not Modified\";\ncode_to_phrase(305) -> \"Use Proxy\";\ncode_to_phrase(306) -> \"(Unused)\";\ncode_to_phrase(307) -> \"Temporary Redirect\";\ncode_to_phrase(308) -> \"Permanent Redirect\";\ncode_to_phrase(400) -> \"Bad Request\";\ncode_to_phrase(401) -> \"Unauthorized\";\ncode_to_phrase(402) -> \"Payment Required\";\ncode_to_phrase(403) -> \"Forbidden\";\ncode_to_phrase(404) -> \"Not Found\";\ncode_to_phrase(405) -> \"Method Not Allowed\";\ncode_to_phrase(406) -> \"Not Acceptable\";\ncode_to_phrase(407) -> \"Proxy Authentication Required\";\ncode_to_phrase(408) -> \"Request Timeout\";\ncode_to_phrase(409) -> \"Conflict\";\ncode_to_phrase(410) -> \"Gone\";\ncode_to_phrase(411) -> \"Length Required\";\ncode_to_phrase(412) -> \"Precondition Failed\";\ncode_to_phrase(413) -> \"Request Entity Too Large\";\ncode_to_phrase(414) -> \"Request-URI Too Long\";\ncode_to_phrase(415) -> \"Unsupported Media Type\";\ncode_to_phrase(416) -> \"Requested Range Not Satisfiable\";\ncode_to_phrase(417) -> \"Expectation Failed\";\ncode_to_phrase(418) -> \"I'm a teapot\";\ncode_to_phrase(420) -> \"Enhance Your Calm\";\ncode_to_phrase(422) -> \"Unprocessable Entity\";\ncode_to_phrase(423) -> \"Locked\";\ncode_to_phrase(424) -> \"Failed Dependency\";\ncode_to_phrase(425) -> \"Unordered Collection\";\ncode_to_phrase(426) -> \"Upgrade Required\";\ncode_to_phrase(428) -> \"Precondition Required\";\ncode_to_phrase(429) -> \"Too Many Requests\";\ncode_to_phrase(431) -> \"Request Header Fields Too Large\";\ncode_to_phrase(451) -> \"Unavailable For Legal Reasons\";\ncode_to_phrase(500) -> \"Internal Server Error\";\ncode_to_phrase(501) -> \"Not Implemented\";\ncode_to_phrase(502) -> \"Bad Gateway\";\ncode_to_phrase(503) -> \"Service Unavailable\";\ncode_to_phrase(504) -> \"Gateway Timeout\";\ncode_to_phrase(505) -> \"HTTP Version Not Supported\";\ncode_to_phrase(506) -> \"Variant Also Negotiates\";\ncode_to_phrase(507) -> \"Insufficient Storage\";\ncode_to_phrase(508) -> \"Loop Detected\";\ncode_to_phrase(510) -> \"Not Extended\";\ncode_to_phrase(511) -> \"Network Authentication Required\";\n\n%% Below are some non-HTTP status codes from other protocol standards that\n%% we've seen used with HTTP in the wild, so we include them here. HTTP 1.1\n%% section 6.1.1 allows for this sort of extensibility, but we recommend\n%% sticking with the HTTP status codes above for maximal portability and\n%% interoperability.\n%%\ncode_to_phrase(452) -> \"Insufficient Storage Space\"; % from FTP (RFC 959)\ncode_to_phrase(453) -> \"Not Enough Bandwidth\".       % from RTSP (RFC 2326)\n\n\n\n%%\n%% server side include\n%%\n\nssi(DocRoot, Files) ->\n    L = lists:map(fun(F) ->\n                          case file:read_file([DocRoot ++ [$/|F]]) of\n                              {ok, Bin} ->\n                                  Bin;\n                              {error, Reason} ->\n                                  io_lib:format(\"Cannot include file ~p: ~p\",\n                                                [F, Reason])\n                          end\n                  end, Files),\n    {html, L}.\n\n\n%% include pre\npre_ssi_files(DocRoot, Files) ->\n    {html, L} = ssi(DocRoot, Files),\n    pre_ssi_string(L).\n\npre_ssi_string(Str) ->\n    pre_ssi_string(Str, \"box\").\n\npre_ssi_string(Str, Class) ->\n    {html, [\"<br><br>\\n<div class=\\\"\", Class, \"\\\"> <pre>\\n\",\n            htmlize_l(Str),\n            \"\\n</pre></div>\\n<br>\\n\\n\"]}.\n\n\n%% convenience\n\nf(Fmt, Args) ->\n    io_lib:format(Fmt, Args).\n\n\nfl([Fmt, Arg | Tail]) ->\n    [f(Fmt, Arg) | fl(Tail)];\nfl([]) ->\n    [].\n\n%% htmlize\nhtmlize(Bin) when is_binary(Bin) ->\n    list_to_binary(htmlize_l(binary_to_list(Bin)));\nhtmlize(List) when is_list(List) ->\n    htmlize_l(List).\n\n\n\nhtmlize_char($>) ->\n    <<\"&gt;\">>;\nhtmlize_char($<) ->\n    <<\"&lt;\">>;\nhtmlize_char($&) ->\n    <<\"&amp;\">>;\nhtmlize_char($\") ->\n    <<\"&quot;\">>;\nhtmlize_char(X) ->\n    X.\n\n\n%% htmlize list (usually much more efficient than above)\nhtmlize_l(List) ->\n    htmlize_l(List, []).\n\nhtmlize_l([], Acc) -> lists:reverse(Acc);\nhtmlize_l([$>|Tail], Acc) ->\n    htmlize_l(Tail, [$;,$t,$g,$&|Acc]);\nhtmlize_l([$<|Tail], Acc) ->\n    htmlize_l(Tail, [$;,$t,$l,$&|Acc]);\nhtmlize_l([$&|Tail], Acc) ->\n    htmlize_l(Tail, [$;,$p,$m,$a,$&|Acc]);\nhtmlize_l([$\"|Tail], Acc) ->\n    htmlize_l(Tail, [$; , $t, $o,  $u,  $q  ,$&|Acc]);\n\nhtmlize_l([X|Tail], Acc) when is_integer(X) ->\n    htmlize_l(Tail, [X|Acc]);\nhtmlize_l([X|Tail], Acc) when is_binary(X) ->\n    X2 = htmlize_l(binary_to_list(X)),\n    htmlize_l(Tail, [X2|Acc]);\nhtmlize_l([X|Tail], Ack) when is_list(X) ->\n    X2 = htmlize_l(X),\n    htmlize_l(Tail, [X2|Ack]).\n\n\n\nsecs() ->\n    {MS, S, _} = yaws:get_time_tuple(),\n    (MS * 1000000) + S.\n\ncookie_option(secure) ->\n    \"; Secure\";\ncookie_option(http_only) ->\n    \"; HttpOnly\";\ncookie_option(I) ->\n    throw({badarg, I}).\ncookie_option(expires, UTC) when is_tuple(UTC) ->\n    [\"; Expires=\" | yaws:universal_time_as_string(UTC)];\ncookie_option(max_age, Age) when is_integer(Age) ->\n    V = if Age < 0 -> \"0\"; true -> integer_to_list(Age) end,\n    [\"; Max-Age=\" | V];\ncookie_option(path, Path) when is_list(Path), Path =/= [] ->\n    [\"; Path=\" | Path];\ncookie_option(domain, Domain) when is_list(Domain), Domain =/= [] ->\n    [\"; Domain=\" | Domain];\ncookie_option(comment, Comment) when is_list(Comment), Comment=/= [] ->\n    [\"; Comment=\" | Comment];\ncookie_option(I, _) ->\n    throw({badarg, I}).\n\n%% @doc Generate a set_cookie header field tuple.\n%%      This function is more RFC6265 compliant than setcookie/6 and\n%%      therefore it deprecates setcookie/6 completely.\nset_cookie(Key, Value, Options)\n        when is_list(Key), is_list(Value), is_list(Options) ->\n    %% RFC6265 (4.1.1): Name=Value options must come first.\n    {NV,SV} = lists:foldl(fun\n        ({N,V}, {L1, L2}) -> {[cookie_option(N,V) | L1], L2};\n        (N,     {L1, L2}) -> {L1, [cookie_option(N) | L2]}\n    end, {[], []}, Options),\n    {header, {set_cookie, [Key, $=, Value, \"; Version=1\", NV | SV]}}.\n\nsetcookie(Name, Value) ->\n    {header, {set_cookie, f(\"~s=~s;\", [Name, Value])}}.\n\nsetcookie(Name, Value, Path) ->\n    {header, {set_cookie, f(\"~s=~s; path=~s\", [Name, Value, Path])}}.\n\nsetcookie(Name, Value, Path, Expire) ->\n    setcookie(Name, Value, Path,  Expire, [], []).\n\nsetcookie(Name, Value, Path, Expire, Domain) ->\n    setcookie(Name, Value, Path, Expire, Domain,[]).\n\nsetcookie(Name, Value, Path, Expire, Domain, Secure) ->\n    SetDomain = if Domain == [] -> \"\";\n                   true -> \" Domain=\"++Domain++\";\"\n                end,\n    SetExpire = if Expire == [] -> \"\";\n                   true -> \" Expires=\"++Expire++\";\"\n                end,\n    SetPath = if Path == [] -> \"/\";\n                 true -> Path\n              end,\n    SetSecure = if Secure == on -> \" secure;\";\n                   true -> \"\"\n                end,\n    {header, {set_cookie, f(\"~s=~s;~s~s~s Path=~s\",\n                            [Name,Value,SetDomain,SetExpire,\n                             SetSecure, SetPath])}}.\n\n\n%% This function can be passed the cookie we get in the Arg#arg.headers.cookies\n%% to search for a specific cookie\n%% return [] if not found\n%%        Str if found\n%% if several cookies with the same name are passed fron the browser,\n%% only the first match is returned\nfind_cookie_val(Name, #arg{}=A) ->\n    find_cookie_val(Name, (A#arg.headers)#headers.cookie);\nfind_cookie_val(Name, Cookies) ->\n    find_cookie_val2(yaws:to_lower(Name), Cookies).\n\nfind_cookie_val2(_, []) ->\n    [];\nfind_cookie_val2(Name, [Cookie|Rest]) ->\n    L = parse_cookie(Cookie),\n    case lists:keyfind(Name, #cookie.key, L) of\n        #cookie{value=undefined} -> [];\n        #cookie{value=Value}     -> Value;\n        false                    -> find_cookie_val2(Name, Rest)\n    end.\n\n\n%%\nurl_decode(Path) ->\n    url_decode_with_encoding(Path, file:native_name_encoding()).\n\nurl_decode_with_encoding(Path, Encoding) ->\n    {DecPath, QS} = url_decode(Path, []),\n    DecPath1 = case Encoding of\n                   latin1 ->\n                       DecPath;\n                   utf8 ->\n                       case unicode:characters_to_list(list_to_binary(DecPath)) of\n                           UTF8DecPath when is_list(UTF8DecPath) -> UTF8DecPath;\n                           _ -> DecPath\n                       end\n               end,\n    case QS of\n        [] -> lists:flatten(DecPath1);\n        _  -> lists:flatten([DecPath1, $?, QS])\n    end.\n\nurl_decode([], Acc) ->\n    {lists:reverse(Acc), []};\nurl_decode([$?|Tail], Acc) ->\n    %% Don't decode the query string here, that is parsed separately.\n    {lists:reverse(Acc), Tail};\nurl_decode([$%, Hi, Lo | Tail], Acc) ->\n    Hex = yaws:hex_to_integer([Hi, Lo]),\n    url_decode(Tail, [Hex|Acc]);\nurl_decode([H|T], Acc) when is_integer(H) ->\n    url_decode(T, [H|Acc]);\n%% deep lists\nurl_decode([H|T], Acc) when is_list(H) ->\n    case url_decode(H, Acc) of\n        {P1, []} ->\n            {P2, QS} = url_decode(T, []),\n            {[P1,P2], QS};\n        {P1, QS} ->\n            {P1, QS++T}\n    end.\n\n\npath_norm(Path) ->\n    path_norm_reverse(lists:reverse(Path)).\n\npath_norm_reverse(\"/\" ++ T) -> start_dir(0, \"/\", T);\npath_norm_reverse(       T) -> start_dir(0,  \"\", T).\n\nstart_dir(N, Path, [$\\\\|T]     ) -> start_dir(N, Path, [$/|T]);\nstart_dir(N, Path, \"..\"        ) -> rest_dir(N, Path, \"\");\nstart_dir(N, Path, \"/\"    ++ T ) -> start_dir(N    , Path, T);\nstart_dir(N, Path, \"./\"   ++ T ) -> start_dir(N    , Path, T);\nstart_dir(N, Path, \".\\\\\"  ++ T ) -> start_dir(N    , Path, T);\nstart_dir(N, Path, \"../\"  ++ T ) -> start_dir(N + 1, Path, T);\nstart_dir(N, Path, \"..\\\\\" ++ T ) -> start_dir(N + 1, Path, T);\nstart_dir(N, Path,           T ) -> rest_dir (N    , Path, T).\n\nrest_dir (_N, Path, []         ) -> case Path of\n                                        [] -> \"/\";\n                                        _  -> Path\n                                    end;\nrest_dir (0, Path, [ $/ | T ] ) -> start_dir(0    , [ $/ | Path ], T);\nrest_dir (N, Path, [ $/ | T ] ) -> start_dir(N - 1,        Path  , T);\nrest_dir (N, Path, [ $\\\\ | T ] ) -> rest_dir(N, Path, [$/|T]);\nrest_dir (0, Path, [  H | T ] ) -> rest_dir (0    , [  H | Path ], T);\nrest_dir (N, Path, [  _H | T ] ) -> rest_dir (N    ,        Path  , T).\n\n%% url decode the path and return {Path, QueryPart}\n\nurl_decode_q_split(Path) ->\n    {DecPath, QS} = url_decode_q_split(Path, []),\n    case file:native_name_encoding() of\n        latin1 ->\n            {DecPath, QS};\n        utf8 ->\n            case unicode:characters_to_list(list_to_binary(DecPath)) of\n                UTF8DecPath when is_list(UTF8DecPath) -> {UTF8DecPath, QS};\n                _ -> {DecPath, QS}\n            end\n    end.\n\nurl_decode_q_split([$%, Hi, Lo | Tail], Ack) ->\n    Hex = yaws:hex_to_integer([Hi, Lo]),\n    if Hex  == 0 -> exit(badurl);\n       true -> ok\n    end,\n    url_decode_q_split(Tail, [Hex|Ack]);\nurl_decode_q_split([$?|T], Ack) ->\n    %% Don't decode the query string here,\n    %% that is parsed separately.\n    {path_norm_reverse(Ack), T};\nurl_decode_q_split([H|T], Ack) when H /= 0 ->\n    url_decode_q_split(T, [H|Ack]);\nurl_decode_q_split([], Ack) ->\n    {path_norm_reverse(Ack), []}.\n\n\nurl_encode(URL) when is_list(URL) ->\n    Bin = case file:native_name_encoding() of\n              latin1 -> list_to_binary(URL);\n              utf8   -> unicode:characters_to_binary(URL)\n          end,\n    %% ReservedChars = \"!*'();:@&=+$,/?%#[]\",\n    UnreservedChars = sets:from_list(\"ABCDEFGHIJKLMNOPQRSTUVWXYZ\"\n                                     \"abcdefghijklmnopqrstuvwxyz\"\n                                     \"0123456789-_.~\"),\n    flatten([url_encode_byte(Byte, UnreservedChars) || <<Byte>> <= Bin]).\n\nurl_encode_byte($:, _) -> $:;  % FIXME: both : and / should be encoded, but\nurl_encode_byte($/, _) -> $/;  % too much code currently assumes they're not\nurl_encode_byte(Byte, UnreservedChars) ->\n    case sets:is_element(Byte, UnreservedChars) of\n        true -> Byte;\n        false ->\n            case yaws:integer_to_hex(Byte) of\n                [X, Y] -> [$%, X, Y];\n                [X]    -> [$%, $0, X]\n            end\n    end.\n\nredirect(Url) -> [{redirect, Url}].\n\nis_nb_space(X) ->\n    lists:member(X, [$\\s, $\\t]).\n\n%% ret: {line, Line, Trail} | {lastline, Line, Trail} | need_more\n\nget_line(L) ->\n    get_line(L, []).\nget_line(\"\\r\\n\\r\\n\" ++ Tail, Cur) ->\n    {lastline, lists:reverse(Cur), Tail};\nget_line(\"\\r\\n\" ++ Tail, Cur) when Tail /= []  ->\n    case is_nb_space(hd(Tail)) of\n        true ->  %% multiline ... continue\n            get_line(Tail, [$\\n, $\\r | Cur]);\n        false ->\n            {line, lists:reverse(Cur), Tail}\n    end;\nget_line(\"\\r\\n\", Cur)   ->\n    {line, lists:reverse(Cur), []};\nget_line([H|T], Cur) ->\n    get_line(T, [H|Cur]);\nget_line([], _) ->\n    need_more.\n\n\n\nmime_type(FileName) ->\n    mime_type(get(sc), FileName).\n\nmime_type(S, FileName) ->\n    case filename:extension(FileName) of\n        [_|T] -> element(2, mime_types:t(S, T));\n        []    -> element(2, mime_types:t(S, []))\n    end.\n\n\n%% Asynchronously delivery\nstream_chunk_deliver(YawsPid, Data) ->\n    YawsPid  ! {streamcontent, Data}.\n\n\n%% Use timeout here to guard against bug in the SSL application\n%% that apparently does not close the socket in between\n%% ssl_esock and erlang (FIN_WAIT2 resp. CLOSE_WAIT).\n%% Thus, the stream process hangs forever...\n-define(STREAM_GARBAGE_TIMEOUT, 3600000). % 1 hour\n\n%% Synchronous (on ultimate gen_tcp:send) delivery\n%% Returns: ok | {error, Rsn}\nstream_chunk_deliver_blocking(YawsPid, Data) ->\n    Ref = erlang:monitor(process, YawsPid),\n    YawsPid  ! {streamcontent_with_ack, self(), Data},\n    receive\n        {YawsPid, streamcontent_ack} ->\n            erlang:demonitor(Ref),\n            %% flush incase a DOWN message was sent before the demonitor call\n            receive\n                {'DOWN', Ref, _, _, _} ->\n                    ok\n            after 0 ->\n                    ok\n            end;\n        {'DOWN', Ref, _, _, Info} ->\n            {error, {ypid_crash, Info}}\n    after ?STREAM_GARBAGE_TIMEOUT ->\n            %% Killing (unless this function is caught) process tree but\n            %% NOTE that as this is probably due to the OTP SSL application\n            %% not managing to close the socket (FIN_WAIT2\n            %% resp. CLOSE_WAIT) the SSL process is not killed (it traps\n            %% exit signals) and thus we will leak one file descriptor.\n            error_logger:error_msg(\n              \"~p:stream_chunk_deliver_blocking/2 STREAM_GARBAGE_TIMEOUT \"\n              \"(default 1 hour). Killing ~p\", [?MODULE, YawsPid]),\n            erlang:error(stream_garbage_timeout, [YawsPid, Data])\n    end.\n\nstream_chunk_end(YawsPid) ->\n    YawsPid ! endofstreamcontent.\n\nstream_process_deliver({ssl, SslSock}, IoList) ->\n    ssl:send(SslSock, IoList);\nstream_process_deliver(Sock, IoList) ->\n    gen_tcp:send(Sock, IoList).\n\nstream_process_deliver_chunk(Sock, IoList) ->\n    Chunk = case erlang:iolist_size(IoList) of\n                0 ->\n                    stream_process_deliver_final_chunk(Sock, IoList);\n                S ->\n                    [yaws:integer_to_hex(S), \"\\r\\n\", IoList, \"\\r\\n\"]\n            end,\n    stream_process_deliver(Sock, Chunk).\n\nstream_process_deliver_final_chunk(Sock, IoList) ->\n    Chunk = case erlang:iolist_size(IoList) of\n                0 ->\n                    <<\"0\\r\\n\\r\\n\">>;\n                S ->\n                    [yaws:integer_to_hex(S), \"\\r\\n\", IoList, \"\\r\\n0\\r\\n\\r\\n\"]\n            end,\n    stream_process_deliver(Sock, Chunk).\n\nstream_process_end(closed, YawsPid) ->\n    YawsPid ! {endofstreamcontent, closed};\nstream_process_end({ssl, SslSock}, YawsPid) ->\n    ssl:controlling_process(SslSock, YawsPid),\n    YawsPid ! endofstreamcontent;\nstream_process_end(Sock, YawsPid) ->\n    gen_tcp:controlling_process(Sock, YawsPid),\n    YawsPid ! endofstreamcontent.\n\n\nwebsocket_send(#ws_state{}=WSState, {Type, Data}) ->\n    yaws_websockets:send(WSState, {Type, Data});\nwebsocket_send(#ws_state{}=WSState, #ws_frame{}=Frame) ->\n    yaws_websockets:send(WSState, Frame);\n%% Pid must be the process in control of the websocket connection.\nwebsocket_send(Pid, {Type, Data}) when is_pid(Pid) ->\n    yaws_websockets:send(Pid, {Type, Data});\nwebsocket_send(Pid, #ws_frame{}=Frame) when is_pid(Pid) ->\n    yaws_websockets:send(Pid, Frame).\n\nwebsocket_close(#ws_state{}=WSState) ->\n    yaws_websockets:close(WSState, normal);\nwebsocket_close(Pid) when is_pid(Pid) ->\n    yaws_websockets:close(Pid, normal).\nwebsocket_close(#ws_state{}=WSState, Reason) ->\n    yaws_websockets:close(WSState, Reason);\nwebsocket_close(Pid, Reason) when is_pid(Pid) ->\n    yaws_websockets:close(Pid, Reason).\n\n\n%% returns {ok, SSL socket} if an SSL socket, undefined otherwise\nget_sslsocket({ssl, SslSocket}) ->\n    {ok, SslSocket};\nget_sslsocket(_Socket) ->\n    undefined.\n\n%% Return new cookie string\nnew_cookie_session(Opaque) ->\n    yaws_session_server:new_session(Opaque).\n\nnew_cookie_session(Opaque, TTL) ->\n    yaws_session_server:new_session(Opaque, TTL).\n\nnew_cookie_session(Opaque, TTL, Cleanup) ->\n    yaws_session_server:new_session(Opaque, TTL, Cleanup).\n\n%% as returned in #ysession.cookie\ncookieval_to_opaque(CookieVal) ->\n    yaws_session_server:cookieval_to_opaque(CookieVal).\n\nprint_cookie_sessions() ->\n    yaws_session_server:print_sessions().\n\nreplace_cookie_session(Cookie, NewOpaque) ->\n    yaws_session_server:replace_session(Cookie, NewOpaque).\nreplace_cookie_session(Cookie, NewOpaque, Cleanup) ->\n    yaws_session_server:replace_session(Cookie, NewOpaque, Cleanup).\n\ndelete_cookie_session(Cookie) ->\n    yaws_session_server:delete_session(Cookie).\n\n\nlmap(F, [H|T]) ->\n    [lists:map(F, H) | lmap(F, T)];\nlmap(_, []) ->\n    [].\n\n\n%% interactively turn on|off tracing\nset_trace(Val) ->\n    Str = yaws_ctl:actl_trace(Val),\n    io:format(\"~s\", [Str]).\n\n\nset_access_log(Bool) ->\n    {ok, GC, Groups} = getconf(),\n    Groups2 = lmap(fun(SC) ->\n                           ?sc_set_access_log(SC, Bool)\n                   end, Groups),\n    setconf(GC, Groups2).\n\n\n%% interactively turn on|off tracing to the tty (as well)\n%% typically useful in embedded mode\nset_tty_trace(Bool) ->\n    yaws_trace:set_tty_trace(Bool).\n\n\n\nset_status_code(Code) ->\n    {status, Code}.\n\n\n\n\n%% returns [ Header1, Header2 .....]\nreformat_header(H) ->\n    FormatFun = fun(Hname, {multi, Values}) ->\n                        [lists:flatten(io_lib:format(\"~s: ~s\", [Hname, Val])) ||\n                            Val <- Values];\n                   (Hname, Str) ->\n                        lists:flatten(io_lib:format(\"~s: ~s\", [Hname, Str]))\n                end,\n    reformat_header(H, FormatFun).\nreformat_header(H, FormatFun) ->\n    lists:zf(fun({Hname, Str}) ->\n                     I =  FormatFun(Hname, Str),\n                     {true, I};\n                (undefined) ->\n                     false\n             end,\n             [\n              if H#headers.connection == undefined ->\n                      undefined;\n                 true ->\n                      {\"Connection\", H#headers.connection}\n              end,\n\n              if H#headers.accept == undefined ->\n                      undefined;\n                 true ->\n                      {\"Accept\", H#headers.accept}\n              end,\n              if H#headers.host == undefined ->\n                      undefined;\n                 true ->\n                      {\"Host\", H#headers.host}\n              end,\n              if H#headers.if_modified_since == undefined ->\n                      undefined;\n                 true ->\n                      {\"If-Modified-Since\", H#headers.if_modified_since}\n              end,\n              if H#headers.if_match == undefined ->\n                      undefined;\n                 true ->\n                      {\"If-Match\", H#headers.if_match}\n              end,\n              if H#headers.if_none_match == undefined ->\n                      undefined;\n                 true ->\n                      {\"If-None-Match\", H#headers.if_none_match}\n              end,\n\n\n              if H#headers.if_range == undefined ->\n                      undefined;\n                 true ->\n                      {\"If-Range\", H#headers.if_range}\n              end,\n              if H#headers.if_unmodified_since == undefined ->\n                      undefined;\n                 true ->\n                      {\"If-Unmodified-Since\", H#headers.if_unmodified_since}\n              end,\n              if H#headers.range == undefined ->\n                      undefined;\n                 true ->\n                      {\"Range\", H#headers.range}\n              end,\n              if H#headers.referer == undefined ->\n                      undefined;\n                 true ->\n                      {\"Referer\", H#headers.referer}\n              end,\n              if H#headers.user_agent == undefined ->\n                      undefined;\n                 true ->\n                      {\"User-Agent\", H#headers.user_agent}\n              end,\n              if H#headers.accept_ranges == undefined ->\n                      undefined;\n                 true ->\n                      {\"Accept-Ranges\", H#headers.accept_ranges}\n              end,\n              if H#headers.cookie == [] ->\n                      undefined;\n                 true ->\n                      {\"Cookie\", H#headers.cookie}\n              end,\n              if H#headers.keep_alive == undefined ->\n                      undefined;\n                 true ->\n                      {\"Keep-Alive\", H#headers.keep_alive}\n              end,\n              if H#headers.content_length == undefined ->\n                      undefined;\n                 true ->\n                      {\"Content-Length\", H#headers.content_length}\n              end,\n              if H#headers.content_type == undefined ->\n                      undefined;\n                 true ->\n                      {\"Content-Type\", H#headers.content_type}\n              end,\n              if H#headers.content_encoding == undefined ->\n                      undefined;\n                 true ->\n                      {\"Content-Encoding\", H#headers.content_encoding}\n              end,\n\n              if H#headers.authorization == undefined ->\n                      undefined;\n                 true ->\n                      {\"Authorization\", element(3, H#headers.authorization)}\n              end,\n              if H#headers.transfer_encoding == undefined ->\n                      undefined;\n                 true ->\n                      {\"Transfer-Encoding\", H#headers.transfer_encoding}\n              end,\n              if H#headers.location == undefined ->\n                      undefined;\n                 true ->\n                      {\"Location\", H#headers.location}\n              end,\n              if H#headers.x_forwarded_for == undefined ->\n                      undefined;\n                 true ->\n                      {\"X-Forwarded-For\", H#headers.x_forwarded_for}\n              end\n\n             ]\n            ) ++\n        lists:map(\n          fun({http_header,_,K,_,V}) ->\n                  FormatFun(K,V)\n          end, H#headers.other).\n\n\nset_header(#headers{}=Hdrs, {Header, Value}) ->\n    set_header(Hdrs, Header, Value).\n\nset_header(#headers{}=Hdrs, connection, Value) ->\n    Hdrs#headers{connection = Value};\nset_header(#headers{}=Hdrs, {lower, \"connection\"}, Value) ->\n    Hdrs#headers{connection = Value};\nset_header(#headers{}=Hdrs, accept, Value) ->\n    Hdrs#headers{accept = Value};\nset_header(#headers{}=Hdrs, {lower, \"accept\"}, Value) ->\n    Hdrs#headers{accept = Value};\nset_header(#headers{}=Hdrs, host, Value) ->\n    Hdrs#headers{host = Value};\nset_header(#headers{}=Hdrs, {lower, \"host\"}, Value) ->\n    Hdrs#headers{host = Value};\nset_header(#headers{}=Hdrs, if_modified_since, Value) ->\n    Hdrs#headers{if_modified_since = Value};\nset_header(#headers{}=Hdrs, {lower, \"if-modified-since\"}, Value) ->\n    Hdrs#headers{if_modified_since = Value};\nset_header(#headers{}=Hdrs, if_match, Value) ->\n    Hdrs#headers{if_match = Value};\nset_header(#headers{}=Hdrs, {lower, \"if-match\"}, Value) ->\n    Hdrs#headers{if_match = Value};\nset_header(#headers{}=Hdrs, if_none_match, Value) ->\n    Hdrs#headers{if_none_match = Value};\nset_header(#headers{}=Hdrs, {lower, \"if-none-match\"}, Value) ->\n    Hdrs#headers{if_none_match = Value};\nset_header(#headers{}=Hdrs, if_range, Value) ->\n    Hdrs#headers{if_range = Value};\nset_header(#headers{}=Hdrs, {lower, \"if-range\"}, Value) ->\n    Hdrs#headers{if_range = Value};\nset_header(#headers{}=Hdrs, if_unmodified_since, Value) ->\n    Hdrs#headers{if_unmodified_since = Value};\nset_header(#headers{}=Hdrs, {lower, \"if-unmodified-since\"}, Value) ->\n    Hdrs#headers{if_unmodified_since = Value};\nset_header(#headers{}=Hdrs, range, Value) ->\n    Hdrs#headers{range = Value};\nset_header(#headers{}=Hdrs, {lower, \"range\"}, Value) ->\n    Hdrs#headers{range = Value};\nset_header(#headers{}=Hdrs, referer, Value) ->\n    Hdrs#headers{referer = Value};\nset_header(#headers{}=Hdrs, {lower, \"referer\"}, Value) ->\n    Hdrs#headers{referer = Value};\nset_header(#headers{}=Hdrs, user_agent, Value) ->\n    Hdrs#headers{user_agent = Value};\nset_header(#headers{}=Hdrs, {lower, \"user-agent\"}, Value) ->\n    Hdrs#headers{user_agent = Value};\nset_header(#headers{}=Hdrs, accept_ranges, Value) ->\n    Hdrs#headers{accept_ranges = Value};\nset_header(#headers{}=Hdrs, {lower, \"accept-ranges\"}, Value) ->\n    Hdrs#headers{accept_ranges = Value};\nset_header(#headers{}=Hdrs, cookie, Value) ->\n    Hdrs#headers{cookie = Value};\nset_header(#headers{}=Hdrs, {lower, \"cookie\"}, Value) ->\n    Hdrs#headers{cookie = Value};\nset_header(#headers{}=Hdrs, keep_alive, Value) ->\n    Hdrs#headers{keep_alive = Value};\nset_header(#headers{}=Hdrs, {lower, \"keep-alive\"}, Value) ->\n    Hdrs#headers{keep_alive = Value};\nset_header(#headers{}=Hdrs, location, Value) ->\n    Hdrs#headers{location = Value};\nset_header(#headers{}=Hdrs, {lower, \"location\"}, Value) ->\n    Hdrs#headers{location = Value};\nset_header(#headers{}=Hdrs, content_length, Value) ->\n    Hdrs#headers{content_length = Value};\nset_header(#headers{}=Hdrs, {lower, \"content-length\"}, Value) ->\n    Hdrs#headers{content_length = Value};\nset_header(#headers{}=Hdrs, content_type, Value) ->\n    Hdrs#headers{content_type = Value};\nset_header(#headers{}=Hdrs, {lower, \"content-type\"}, Value) ->\n    Hdrs#headers{content_type = Value};\nset_header(#headers{}=Hdrs, content_encoding, Value) ->\n    Hdrs#headers{content_encoding = Value};\nset_header(#headers{}=Hdrs, {lower, \"content-encoding\"}, Value) ->\n    Hdrs#headers{content_encoding = Value};\nset_header(#headers{}=Hdrs, authorization, Value) ->\n    Hdrs#headers{authorization = Value};\nset_header(#headers{}=Hdrs, {lower, \"authorization\"}, Value) ->\n    Hdrs#headers{authorization = Value};\nset_header(#headers{}=Hdrs, transfer_encoding, Value) ->\n    Hdrs#headers{transfer_encoding = Value};\nset_header(#headers{}=Hdrs, {lower, \"transfer-encoding\"}, Value) ->\n    Hdrs#headers{transfer_encoding = Value};\nset_header(#headers{}=Hdrs, x_forwarded_for, Value) ->\n    Hdrs#headers{x_forwarded_for = Value};\nset_header(#headers{}=Hdrs, {lower, \"x-forwarded-for\"}, Value) ->\n    Hdrs#headers{x_forwarded_for = Value};\nset_header(#headers{}=Hdrs, Header, Value) when is_atom(Header) ->\n    set_header(Hdrs, atom_to_list(Header), Value);\nset_header(#headers{}=Hdrs, Header, Value) when is_binary(Header) ->\n    set_header(Hdrs, binary_to_list(Header), Value);\nset_header(#headers{}=Hdrs, Header, Val) when is_binary(Val) ->\n    set_header(Hdrs, {lower, string:to_lower(Header)}, binary_to_list(Val));\nset_header(#headers{other=Other}=Hdrs, {lower, Header}, undefined) ->\n    Handler = fun(_, true, Acc) ->\n                      Acc;\n                 (HdrVal, false, Acc) ->\n                      [HdrVal|Acc]\n              end,\n    NewOther = fold_others(Header, Handler, Other, []),\n    Hdrs#headers{other = lists:reverse(NewOther)};\nset_header(#headers{other=Other}=Hdrs, {lower, Header}, Val) ->\n    HdrName = erlang_header_name(Header),\n    Handler = fun({http_header, Int, _, Rsv, _}, true, {Acc, _}) ->\n                      {[{http_header, Int, HdrName, Rsv, Val}|Acc],true};\n                 (HdrVal, false, {Acc, Found}) ->\n                      {[HdrVal|Acc], Found}\n              end,\n    {NewOther0, Found} = fold_others(Header, Handler, Other, {[], false}),\n    NewOther = case Found of\n                   true ->\n                       NewOther0;\n                   false ->\n                       [{http_header, 0, HdrName, undefined, Val}|NewOther0]\n               end,\n    Hdrs#headers{other = lists:reverse(NewOther)};\nset_header(#headers{}=Hdrs, Header, undefined) ->\n    set_header(Hdrs, {lower, string:to_lower(Header)}, undefined);\nset_header(#headers{}=Hdrs, Header, Value) ->\n    set_header(Hdrs, {lower, string:to_lower(Header)}, Value).\n\nmerge_header(#headers{}=Hdrs, {Header, Value}) ->\n    merge_header(Hdrs, Header, Value).\n\nmerge_header(#headers{}=Hdrs, _Header, undefined) ->\n    Hdrs;\nmerge_header(#headers{}=Hdrs, Header, Value) when is_atom(Header) ->\n    merge_header(Hdrs, atom_to_list(Header), Value);\nmerge_header(#headers{}=Hdrs, Header, Value) when is_binary(Header) ->\n    merge_header(Hdrs, binary_to_list(Header), Value);\nmerge_header(#headers{}=Hdrs, Header, Value) when is_binary(Value) ->\n    merge_header(Hdrs, Header, binary_to_list(Value));\nmerge_header(Hdrs, {lower, \"set-cookie\"}=LHdr, Value) ->\n    NewValue = case get_header(Hdrs, LHdr) of\n                   undefined ->\n                       {multi, [Value]};\n                   {multi, MultiVal} ->\n                       {multi, MultiVal ++ [Value]};\n                   ExistingValue ->\n                       {multi, [ExistingValue, Value]}\n               end,\n    set_header(Hdrs, LHdr, NewValue);\nmerge_header(Hdrs, {lower, _Header}=LHdr, Value) ->\n    NewValue = case get_header(Hdrs, LHdr) of\n                   undefined ->\n                       Value;\n                   ExistingValue ->\n                       ExistingValue ++ \", \" ++ Value\n               end,\n    set_header(Hdrs, LHdr, NewValue);\nmerge_header(#headers{}=Hdrs, Header, Value) ->\n    merge_header(Hdrs, {lower, string:to_lower(Header)}, Value).\n\nget_header(#headers{}=Hdrs, connection) ->\n    Hdrs#headers.connection;\nget_header(#headers{}=Hdrs, {lower, \"connection\"}) ->\n    Hdrs#headers.connection;\nget_header(#headers{}=Hdrs, accept) ->\n    Hdrs#headers.accept;\nget_header(#headers{}=Hdrs, {lower, \"accept\"}) ->\n    Hdrs#headers.accept;\nget_header(#headers{}=Hdrs, host) ->\n    Hdrs#headers.host;\nget_header(#headers{}=Hdrs, {lower, \"host\"}) ->\n    Hdrs#headers.host;\nget_header(#headers{}=Hdrs, if_modified_since) ->\n    Hdrs#headers.if_modified_since;\nget_header(#headers{}=Hdrs, {lower, \"if-modified-since\"}) ->\n    Hdrs#headers.if_modified_since;\nget_header(#headers{}=Hdrs, if_match) ->\n    Hdrs#headers.if_match;\nget_header(#headers{}=Hdrs, {lower, \"if-match\"}) ->\n    Hdrs#headers.if_match;\nget_header(#headers{}=Hdrs, if_none_match) ->\n    Hdrs#headers.if_none_match;\nget_header(#headers{}=Hdrs, {lower, \"if-none-match\"}) ->\n    Hdrs#headers.if_none_match;\nget_header(#headers{}=Hdrs, if_range) ->\n    Hdrs#headers.if_range;\nget_header(#headers{}=Hdrs, {lower, \"if-range\"}) ->\n    Hdrs#headers.if_range;\nget_header(#headers{}=Hdrs, if_unmodified_since) ->\n    Hdrs#headers.if_unmodified_since;\nget_header(#headers{}=Hdrs, {lower, \"if-unmodified-since\"}) ->\n    Hdrs#headers.if_unmodified_since;\nget_header(#headers{}=Hdrs, range) ->\n    Hdrs#headers.range;\nget_header(#headers{}=Hdrs, {lower, \"range\"}) ->\n    Hdrs#headers.range;\nget_header(#headers{}=Hdrs, referer) ->\n    Hdrs#headers.referer;\nget_header(#headers{}=Hdrs, {lower, \"referer\"}) ->\n    Hdrs#headers.referer;\nget_header(#headers{}=Hdrs, user_agent) ->\n    Hdrs#headers.user_agent;\nget_header(#headers{}=Hdrs, {lower, \"user-agent\"}) ->\n    Hdrs#headers.user_agent;\nget_header(#headers{}=Hdrs, accept_ranges) ->\n    Hdrs#headers.accept_ranges;\nget_header(#headers{}=Hdrs, {lower, \"accept-ranges\"}) ->\n    Hdrs#headers.accept_ranges;\nget_header(#headers{}=Hdrs, cookie) ->\n    Hdrs#headers.cookie;\nget_header(#headers{}=Hdrs, {lower, \"cookie\"}) ->\n    Hdrs#headers.cookie;\nget_header(#headers{}=Hdrs, keep_alive) ->\n    Hdrs#headers.keep_alive;\nget_header(#headers{}=Hdrs, {lower, \"keep-alive\"}) ->\n    Hdrs#headers.keep_alive;\nget_header(#headers{}=Hdrs, location) ->\n    Hdrs#headers.location;\nget_header(#headers{}=Hdrs, {lower, \"location\"}) ->\n    Hdrs#headers.location;\nget_header(#headers{}=Hdrs, content_length) ->\n    Hdrs#headers.content_length;\nget_header(#headers{}=Hdrs, {lower, \"content-length\"}) ->\n    Hdrs#headers.content_length;\nget_header(#headers{}=Hdrs, content_type) ->\n    Hdrs#headers.content_type;\nget_header(#headers{}=Hdrs, {lower, \"content-type\"}) ->\n    Hdrs#headers.content_type;\nget_header(#headers{}=Hdrs, content_encoding) ->\n    Hdrs#headers.content_encoding;\nget_header(#headers{}=Hdrs, {lower, \"content-encoding\"}) ->\n    Hdrs#headers.content_encoding;\nget_header(#headers{}=Hdrs, authorization) ->\n    Hdrs#headers.authorization;\nget_header(#headers{}=Hdrs, {lower, \"authorization\"}) ->\n    Hdrs#headers.authorization;\nget_header(#headers{}=Hdrs, transfer_encoding) ->\n    Hdrs#headers.transfer_encoding;\nget_header(#headers{}=Hdrs, {lower, \"transfer-encoding\"}) ->\n    Hdrs#headers.transfer_encoding;\nget_header(#headers{}=Hdrs, x_forwarded_for) ->\n    Hdrs#headers.x_forwarded_for;\nget_header(#headers{}=Hdrs, {lower, \"x-forwarded-for\"}) ->\n    Hdrs#headers.x_forwarded_for;\nget_header(#headers{}=Hdrs, Header) when is_atom(Header) ->\n    get_header(Hdrs, atom_to_list(Header));\nget_header(#headers{}=Hdrs, Header) when is_binary(Header) ->\n    get_header(Hdrs, binary_to_list(Header));\nget_header(#headers{other = Other}, {lower, Header}) ->\n    Handler = fun({http_header, _, _, _, Value}, true, _Acc) ->\n                      throw(Value);\n                 (_, false, Acc) ->\n                      Acc\n              end,\n    catch fold_others(Header, Handler, Other, undefined);\nget_header(#headers{}=Hdrs, Header) ->\n    get_header(Hdrs, {lower, string:to_lower(Header)}).\n\nget_header(#headers{}=Hdrs, Header, Default) ->\n    case get_header(Hdrs, Header) of\n        undefined ->\n            Default;\n        Value ->\n            Value\n    end.\n\ndelete_header(#headers{}=Hdrs, Header) ->\n    set_header(Hdrs, Header, undefined).\n\n%% assumes that LowerHdr is already downcased\nfold_others(LowerHdr, Handler, Other, StartAcc) ->\n    lists:foldl(fun({http_header, _, Hdr, _, _}=HdrVal, Acc) ->\n                        HdrNm = string:to_lower(\n                                  if\n                                      is_atom(Hdr) -> atom_to_list(Hdr);\n                                      is_binary(Hdr) -> binary_to_list(Hdr);\n                                      true -> Hdr\n                                  end),\n                        Handler(HdrVal, HdrNm == LowerHdr, Acc)\n                end, StartAcc, Other).\n\nerlang_header_name(\"cache-control\")       -> 'Cache-Control';\nerlang_header_name(\"date\")                -> 'Date';\nerlang_header_name(\"pragma\")              -> 'Pragma';\nerlang_header_name(\"upgrade\")             -> 'Upgrade';\nerlang_header_name(\"via\")                 -> 'Via';\nerlang_header_name(\"accept-charset\")      -> 'Accept-Charset';\nerlang_header_name(\"accept-encoding\")     -> 'Accept-Encoding';\nerlang_header_name(\"accept-language\")     -> 'Accept-Language';\nerlang_header_name(\"from\")                -> 'From';\nerlang_header_name(\"max-forwards\")        -> 'Max-Forwards';\nerlang_header_name(\"proxy-authorization\") -> 'Proxy-Authorization';\nerlang_header_name(\"age\")                 -> 'Age';\nerlang_header_name(\"proxy-authenticate\")  -> 'Proxy-Authenticate';\nerlang_header_name(\"public\")              -> 'Public';\nerlang_header_name(\"retry-after\")         -> 'Retry-After';\nerlang_header_name(\"server\")              -> 'Server';\nerlang_header_name(\"vary\")                -> 'Vary';\nerlang_header_name(\"warning\")             -> 'Warning';\nerlang_header_name(\"www-authenticate\")    -> 'Www-Authenticate';\nerlang_header_name(\"allow\")               -> 'Allow';\nerlang_header_name(\"content-base\")        -> 'Content-Base';\nerlang_header_name(\"content-encoding\")    -> 'Content-Encoding';\nerlang_header_name(\"content-language\")    -> 'Content-Language';\nerlang_header_name(\"content-location\")    -> 'Content-Location';\nerlang_header_name(\"content-md5\")         -> 'Content-Md5';\nerlang_header_name(\"content-range\")       -> 'Content-Range';\nerlang_header_name(\"etag\")                -> 'Etag';\nerlang_header_name(\"expires\")             -> 'Expires';\nerlang_header_name(\"last-modified\")       -> 'Last-Modified';\nerlang_header_name(\"set-cookie\")          -> 'Set-Cookie';\nerlang_header_name(\"set-cookie2\")         -> 'Set-Cookie2';\nerlang_header_name(\"proxy-connection\")    -> 'Proxy-Connection';\nerlang_header_name(Name)                  -> capitalize_header(Name).\n\ncapitalize_header(Name) ->\n    %% Before R16B Erlang capitalized words inside header names only for\n    %% headers less than 20 characters long. In R16B that length was raised\n    %% to 50. Using decode_packet lets us be portable.\n    {ok, {http_header, _, Result, _, _}, _} =\n        erlang:decode_packet(httph, list_to_binary([Name, <<\": x\\r\\n\\r\\n\">>]),\n                             []),\n    Result.\n\nreformat_request(#http_request{method = bad_request}) ->\n    [\"Bad request\"];\nreformat_request(Req) ->\n    Path = case Req#http_request.path of\n               {abs_path, AbsPath} ->\n                   AbsPath;\n               {absoluteURI, _Scheme, _Host0, _Port, RawPath} ->\n                   RawPath\n           end,\n    {Maj, Min} = Req#http_request.version,\n    [yaws:to_list(Req#http_request.method), \" \", Path,\" HTTP/\",\n     integer_to_list(Maj),\".\", integer_to_list(Min)].\n\n\nreformat_response(Resp) ->\n    {Maj,Min} = Resp#http_response.version,\n    [\"HTTP/\",integer_to_list(Maj),\".\", integer_to_list(Min),\n     \" \", integer_to_list(Resp#http_response.status),\n     \" \", Resp#http_response.phrase].\n\n\n\n%% stringify the scheme://host[:port] part of a #url\nreformat_url(U) ->\n    [yaws:to_string(U#url.scheme),\n     \"://\",\n     U#url.host,\n     if\n         U#url.port == undefined ->\n             [];\n         true ->\n             [$: | integer_to_list(U#url.port)]\n     end].\n\nset_content_type(MimeType) ->\n    {header, {content_type, MimeType}}.\n\n\n%% returns a #url{} record\nparse_url(Str) ->\n    parse_url(Str, strict).\n\nparse_url(Str, Strict) ->\n    case Str of\n        \"http://\" ++ Rest ->\n            parse_url(host, Strict, #url{scheme = http}, Rest, []);\n        \"https://\" ++ Rest ->\n            parse_url(host, Strict, #url{scheme = https}, Rest, []);\n        \"ftp://\" ++ Rest ->\n            parse_url(host, Strict, #url{scheme = ftp}, Rest, []);\n        \"file://\" ++ Rest ->\n            parse_url(host, Strict, #url{scheme = file}, Rest, []);\n        _ when Strict == sloppy ->\n            parse_url(host, Strict, #url{scheme = undefined}, Str, [])\n    end.\n\n\nparse_url(host, Strict, U, Str, Ack) ->\n    case Str of\n        [] ->\n            U#url{host = lists:reverse(Ack),\n                  path = \"/\"\n                 };\n        [$/|Tail] ->\n            U2 = U#url{host = lists:reverse(Ack)},\n            parse_url(path, Strict, U2, Tail,\"/\");\n        [$:|T] ->\n            U2 = U#url{host = lists:reverse(Ack)},\n            parse_url(port, Strict, U2, T,[]);\n        [$[|T] ->\n            parse_url(ipv6, Strict, U, T, [$[]);\n        [H|T] ->\n            parse_url(host, Strict, U, T, [H|Ack])\n    end;\nparse_url(ipv6, Strict, U, Str, Ack) ->\n    case Str of\n        [$]] ->\n            U#url{host = lists:reverse([$]|Ack]),\n                  path = \"/\"\n                 };\n        [$], $/|T] ->\n            U2 = U#url{host = lists:reverse([$]|Ack])},\n            parse_url(path, Strict, U2, T,\"/\");\n        [$], $:|T] ->\n            U2 = U#url{host = lists:reverse([$]|Ack])},\n            parse_url(port, Strict, U2, T,[]);\n        [H|T] ->\n            parse_url(ipv6, Strict, U, T, [H|Ack])\n    end;\nparse_url(port, Strict, U, Str, Ack) ->\n    case Str of\n        [] ->\n            U#url{port = list_to_integer(lists:reverse(Ack)),\n                  path = \"/\"};\n        [$/|T] ->\n            U2 = U#url{port = list_to_integer(lists:reverse(Ack))},\n            parse_url(path, Strict, U2, T,\"/\");\n        [H|T] ->\n            parse_url(port, Strict, U,T,[H|Ack])\n    end;\nparse_url(path, Strict, U, Str, Ack) ->\n    case Str of\n        [] ->\n            U#url{path = lists:reverse(Ack)};\n        [$?|T] ->\n            U#url{path = lists:reverse(Ack),\n                  querypart = T};\n        [H|T] ->\n            parse_url(path, Strict, U, T, [H|Ack])\n    end.\n\n\n%% used to construct redir headers from partial URLs such\n%% as e.g. /foo/bar\n\nformat_partial_url(Url, SC) ->\n    [if\n         Url#url.scheme == undefined ->\n             yaws:redirect_scheme(SC);\n         true ->\n             yaws:to_string(Url#url.scheme) ++ \"://\"\n     end,\n     if\n         Url#url.host == undefined orelse Url#url.host == [] ->\n             yaws:redirect_host(SC, undefined);\n         true ->\n             Url#url.host\n     end,\n     if\n         Url#url.port == undefined ->\n             [];\n         true  ->\n             [$: | integer_to_list(Url#url.port)]\n     end,\n     Url#url.path,\n     if\n         Url#url.querypart == [] ->\n             [];\n         true ->\n             [$?|Url#url.querypart]\n     end\n    ].\n\n\nformat_url(Url) when is_record(Url, url) ->\n    [\n     if\n         Url#url.scheme == undefined ->\n             \"http://\";\n         true ->\n             yaws:to_string(Url#url.scheme) ++ \"://\"\n     end,\n     Url#url.host,\n     if\n         Url#url.port == undefined ->\n             [];\n         true  ->\n             [$: | integer_to_list(Url#url.port)]\n     end,\n     Url#url.path,\n     if\n         Url#url.querypart == [] ->\n             [];\n         true ->\n             [$?|Url#url.querypart]\n     end\n    ].\n\nis_absolute_URI([C|T]) when ((C>=$a) and (C=<$z)) or ((C>=$A) and (C=<$Z))->\n    is_abs_URI1(T);\nis_absolute_URI(_) ->\n    false.\n\nis_abs_URI1([$:|_]) ->\n    true;\nis_abs_URI1([C|T]) when\n((C>=$a) and (C=<$z))\nor ((C>=$A) and (C=<$Z))\nor ((C>=$0) and (C=<$9))\nor (C==$+) or (C==$-) or (C==$.) ->\n    is_abs_URI1(T);\nis_abs_URI1(_) ->\n    false.\n\n\n%% ------------------------------------------------------------\n%% simple erlang term representation of HTML:\n%% EHTML = [EHTML] | {Tag, Attrs, Body} | {Tag, Attrs} | {Tag} |\n%%         {Module, Fun, [Args]} | fun/0 |\n%%         binary() | character()\n%% Tag   = atom()\n%% Attrs = [{Key, Value}]\n%% Key   = atom()\n%% Value = string() | binary() | atom() | integer() | float() |\n%%         {Module, Fun, [Args]} | fun/0\n%% Body  = EHTML\n\nehtml_expand(Ch) when Ch >= 0, Ch =< 255 -> Ch; %yaws_api:htmlize_char(Ch);\nehtml_expand(Bin) when is_binary(Bin) -> Bin; % yaws_api:htmlize(Bin);\n\nehtml_expand({ssi,File, Del, Bs}) ->\n    case yaws_server:ssi(File, Del, Bs) of\n        {error, Rsn} ->\n            io_lib:format(\"ERROR: ~p~n\",[Rsn]);\n        X ->\n            X\n    end;\n\n%%!todo (low priority) - investigate whether tail-recursion would be of any\n%% benefit here instead of the current ehtml_expand(Body) recursion.\n%%                - provide a tail_recursive version & add a file in the\n%% benchmarks folder to measure it.\n                                                %\nehtml_expand({Tag}) ->\n    [\"<\", atom_to_list(Tag), ehtml_end_tag(Tag)];\nehtml_expand({pre_html, X}) -> X;\nehtml_expand({Mod, Fun, Args})\n  when is_atom(Mod), is_atom(Fun), is_list(Args) ->\n    ehtml_expand(Mod:Fun(Args));\nehtml_expand({Tag, Attrs}) ->\n    NL = ehtml_nl(Tag),\n    [NL, \"<\", atom_to_list(Tag), ehtml_attrs(Attrs), ehtml_end_tag(Tag)];\nehtml_expand({Tag, Attrs, Body}) when is_atom(Tag) ->\n    Ts = atom_to_list(Tag),\n    NL = ehtml_nl(Tag),\n    [NL, \"<\", Ts, ehtml_attrs(Attrs), \">\", ehtml_expand(Body), \"</\", Ts, \">\"];\nehtml_expand([H|T]) -> [ehtml_expand(H)|ehtml_expand(T)];\nehtml_expand([]) -> [];\nehtml_expand(Fun) when is_function(Fun) ->\n    ehtml_expand(Fun()).\n\n\nehtml_attrs([]) -> [];\nehtml_attrs([Attribute|Tail]) when is_atom(Attribute) ->\n    [[$ |atom_to_list(Attribute)]|ehtml_attrs(Tail)];\nehtml_attrs([Attribute|Tail]) when is_list(Attribute) ->\n    [\" \", Attribute|ehtml_attrs(Tail)];\nehtml_attrs([{Name, {Mod, Fun, Args}} | Tail])\n  when is_atom(Mod), is_atom(Fun), is_list(Args) ->\n    ehtml_attrs([{Name,  Mod:Fun(Args)} | Tail]);\nehtml_attrs([{Name, Value} | Tail]) when is_function(Value) ->\n    ehtml_attrs([{Name, Value()} | Tail]);\nehtml_attrs([{Name, Value} | Tail]) ->\n    ValueString = [$\", value2string(Value), $\"],\n    [[$ |atom_to_list(Name)], [$=|ValueString]|ehtml_attrs(Tail)];\nehtml_attrs([{check, Name, {Mod, Fun, Args}} | Tail])\n  when is_atom(Mod), is_atom(Fun), is_list(Args) ->\n    ehtml_attrs([{check, Name,  Mod:Fun(Args)} | Tail]);\nehtml_attrs([{check, Name, Value} | Tail]) when is_function(Value) ->\n    ehtml_attrs([{check, Name, Value()} | Tail]);\nehtml_attrs([{check, Name, Value} | Tail]) ->\n    Val = value2string(Value),\n    Q = case deepmember($\", Val) of\n            true -> $';\n            false -> $\"\n        end,\n    ValueString = [Q,Val,Q],\n    [[$ |atom_to_list(Name)], [$=|ValueString]|ehtml_attrs(Tail)].\n\nvalue2string(Atom) when is_atom(Atom) -> atom_to_list(Atom);\nvalue2string(String) when is_list(String) -> String;\nvalue2string(Binary) when is_binary(Binary) -> Binary;\nvalue2string(Integer) when is_integer(Integer) -> integer_to_list(Integer);\nvalue2string(Float) when is_float(Float) -> float_to_list(Float).\n\n\n\n%% Tags for which we must not add extra white space.\n%% FIXME: should there be anything more in this list?\n\nehtml_nl(a) -> [];\nehtml_nl(br) -> [];\nehtml_nl(span) -> [];\nehtml_nl(em) -> [];\nehtml_nl(strong) -> [];\nehtml_nl(dfn) -> [];\nehtml_nl(code) -> [];\nehtml_nl(samp) -> [];\nehtml_nl(kbd) -> [];\nehtml_nl(var) -> [];\nehtml_nl(cite) -> [];\nehtml_nl(abbr) -> [];\nehtml_nl(acronym) -> [];\nehtml_nl(q) -> [];\nehtml_nl(sub) -> [];\nehtml_nl(sup) -> [];\nehtml_nl(ins) -> [];\nehtml_nl(del) -> [];\nehtml_nl(img) -> [];\nehtml_nl(tt) -> [];\nehtml_nl(i) -> [];\nehtml_nl(b) -> [];\nehtml_nl(big) -> [];\nehtml_nl(small) -> [];\nehtml_nl(strike) -> [];\nehtml_nl(s) -> [];\nehtml_nl(u) -> [];\nehtml_nl(font) -> [];\nehtml_nl(basefont) -> [];\nehtml_nl(input) -> [];\nehtml_nl(button) -> [];\nehtml_nl(object) -> [];\nehtml_nl(_) -> \"\\n\".\n\n\n%% Void elements must not have an end tag (</tag>) in HTML5, while for most\n%% elements a proper end tag (<tag></tag>, not <tag />) is mandatory.\n%%\n%% http://www.w3.org/TR/html5/syntax.html#void-elements\n%% http://www.w3.org/TR/html5/syntax.html#syntax-tag-omission\n\n-define(self_closing, \" />\"). % slash ignored in HTML5\n\nehtml_end_tag(area) -> ?self_closing;\nehtml_end_tag(base) -> ?self_closing;\nehtml_end_tag(br) -> ?self_closing;\nehtml_end_tag(col) -> ?self_closing;\nehtml_end_tag(embed) -> ?self_closing;\nehtml_end_tag(hr) -> ?self_closing;\nehtml_end_tag(img) -> ?self_closing;\nehtml_end_tag(input) -> ?self_closing;\nehtml_end_tag(keygen) -> ?self_closing;\nehtml_end_tag(link) -> ?self_closing;\nehtml_end_tag(meta) -> ?self_closing;\nehtml_end_tag(param) -> ?self_closing;\nehtml_end_tag(source) -> ?self_closing;\nehtml_end_tag(track) -> ?self_closing;\nehtml_end_tag(wbr) -> ?self_closing;\nehtml_end_tag(Tag) -> [\"></\", atom_to_list(Tag), \">\"].\n\n\n%% ------------------------------------------------------------\n%% ehtml_expander/1: an EHTML optimizer\n%%\n%% This is an optimization for generating the same EHTML multiple times with\n%% only small differences, by using fast re-usable templates that contain\n%% variables. The variables are atoms starting with a dollar sign, like\n%% '$myvar'. There are two functions: ehtml_expander/1 to create an optimized\n%% EHTML template, then ehtml_apply/2 takes a template and a dictionary of\n%% variable values and generates the actual HTML.\n%%\n%% If you are spending a lot of time regenerating similar EHTML fragments then\n%% this is for you.\n%%\n%% Variables can appear in three places:\n%% - As a body element, where you would normally have a tag. The values of\n%%   these variables are expanded as EHTML.\n%% - As the name or value of an attribute. The values of these variables are\n%%   strings.\n%% - As the CDR of an attribute list. The values of these variables are\n%%   key-value lists of more attributes.\n%%\n%% See ehtml_expander_test/0 for an example.\n%%\n%% The approach is inspired by the way that Yaws already treats .yaws files,\n%% and the article ``A Hacker's Introduction To Partial Evaluation'' by Darius\n%% Bacon (cool guy), http://www.lisp-p.org/htdocs/peval/peval.cgi\n%%\n%% (For now I flatter myself that this is some kind of partial evaluator, but\n%% I don't really know :-) -luke)\n\nehtml_expander(X) ->\n    ehtml_expander_compress(flatten(ehtml_expander(X, [], [])), []).\n\n%% Returns a deep list of text and variable references (atoms)\n\n%% Text\nehtml_expander(Ch, Before, After) when Ch >= 0, Ch =< 255 ->\n    ehtml_expander_done(yaws_api:htmlize_char(Ch), Before, After);\nehtml_expander(Bin, Before, After) when is_binary(Bin) ->\n    ehtml_expander_done(yaws_api:htmlize(Bin), Before, After);\n\nehtml_expander({ssi,File, Del, Bs}, Before, After) ->\n    Str = case yaws_server:ssi(File, Del, Bs) of\n              {error, Rsn} ->\n                  io_lib:format(\"ERROR: ~p~n\",[Rsn]);\n              X ->\n                  X\n          end,\n    ehtml_expander_done(Str, Before, After);\n\nehtml_expander({pre_html, X}, Before, After) ->\n    ehtml_expander_done(X, Before, After);\n%% Tags\nehtml_expander({Tag}, Before, After) ->\n    ehtml_expander_done([\"<\", atom_to_list(Tag), ehtml_end_tag(Tag)],\n                        Before, After);\nehtml_expander({Tag, Attrs}, Before, After) ->\n    NL = ehtml_nl(Tag),\n    ehtml_expander_done([NL, \"<\", atom_to_list(Tag), ehtml_attrs(Attrs),\n                         ehtml_end_tag(Tag)],\n                        Before,\n                        After);\nehtml_expander({Tag, Attrs, Body}, Before, After) ->\n    ehtml_expander(Body,\n                   [[\"\\n<\", atom_to_list(Tag),\n                     ehtml_attrs_expander(Attrs), \">\"]|\n                    Before],\n                   [\"</\", atom_to_list(Tag), \">\"|After]);\n%% Variable references\nehtml_expander(Var, Before, After) when is_atom(Var) ->\n    [reverse(Before), {ehtml, ehtml_var_name(Var)}, After];\n%% Lists\nehtml_expander([H|T], Before, After) ->\n    ehtml_expander(T, [ehtml_expander(H, [], [])|Before], After);\nehtml_expander([], Before, After) ->\n    ehtml_expander_done(\"\", Before, After).\n\n%% Expander for attributes. The attribute name and value can each be a\n%% variable reference.\nehtml_attrs_expander([]) -> \"\";\nehtml_attrs_expander([{Var,Val}|T]) ->\n    [[\" \",\n      ehtml_attr_part_expander(Var),\n      \"=\",\n      \"\\\"\", ehtml_attr_part_expander(Val), \"\\\"\"]|\n     ehtml_attrs_expander(T)];\nehtml_attrs_expander([Var|T]) ->\n    [[\" \",\n      ehtml_attr_part_expander(Var)]|\n     ehtml_attrs_expander(T)];\nehtml_attrs_expander(Var) when is_atom(Var) ->\n    %% Var in the cdr of an attribute list\n    [{ehtml_attrs, ehtml_var_name(Var)}].\n\nehtml_attr_part_expander(A) when is_atom(A) ->\n    case atom_to_list(A) of\n        [$$|_Rest] -> {preformatted, ehtml_var_name(A)};\n        Other -> Other\n    end;\nehtml_attr_part_expander(I) when is_integer(I) -> integer_to_list(I);\nehtml_attr_part_expander(S) when is_list(S) -> S.\n\nehtml_expander_done(X, Before, After) -> [reverse([X|Before]), After].\n\n%% Compress an EHTML expander, converting all adjacent bits of text into\n%% binaries.\n%% Returns: [binary() | {ehtml, Var} | {preformatted, Var}, {ehtml_attrs, Var}]\n%% Var = atom()\nehtml_expander_compress([Tag|T], Acc) when is_tuple(Tag) ->\n    [list_to_binary(reverse(Acc)), Tag | ehtml_expander_compress(T, [])];\nehtml_expander_compress([], Acc) -> [list_to_binary(reverse(Acc))];\nehtml_expander_compress([H|T], Acc) when is_integer(H) ->\n    ehtml_expander_compress(T, [H|Acc]).\n\n%% Apply an expander with the variable bindings in Env.  Env is a list of\n%% {VarName, Value} tuples, where VarName is an atom and Value is an ehtml\n%% term.\nehtml_apply(Expander, Env) -> [ehtml_eval(X, Env) || X <- Expander].\n\nehtml_eval(Bin, _Env) when is_binary(Bin) -> Bin;\nehtml_eval({Type, Var}, Env) ->\n    case lists:keysearch(Var, 1, Env) of\n        false -> erlang:error({ehtml_unbound, Var});\n        {value, {Var, Val}} ->\n            case Type of\n                ehtml -> ehtml_expand(Val);\n                preformatted -> Val;\n                ehtml_attrs -> ehtml_attrs(Val)\n            end\n    end.\n\n%% Get the name part of a variable reference.\n%% e.g. ehtml_var_name('$foo') -> foo.\nehtml_var_name(A) when is_atom(A) ->\n    case atom_to_list(A) of\n        [$$|Rest] -> list_to_atom(Rest);\n        _Other -> erlang:error({bad_ehtml_var_name, A})\n    end.\n\nehtml_expander_test() ->\n    %% Expr is a template containing variables.\n    Expr = {html, [{title, '$title'}],\n            {body, [],\n             [{h1, [], '$heading'},\n              '$text']}},\n    %% Expand is an expander that can be used to quickly generate the HTML\n    %% specified in Expr.\n    Expand = ehtml_expander(Expr),\n    %% Bs{1,2} are lists of variable bindings to fill in the gaps in the\n    %% template. We can reuse the template on many sets of bindings, and this\n    %% is much faster than doing a full ehtml of the whole page each time.\n    Bs1 = [{title, \"First page\"},\n           {heading, \"Heading\"},\n           {text, {pre_html, \"<b>My text!</b>\"}}],\n    Bs2 = [{title, \"Second page\"},\n           {heading, \"Foobar\"},\n           {text, {b, [], \"My text again!\"}}],\n    %% Page1 and Page2 are generated from the template. They are I/O lists\n    %% (i.e. deep lists of strings and binaries, ready to ship)\n    Page1 = ehtml_apply(Expand, Bs1),\n    Page2 = ehtml_apply(Expand, Bs2),\n    %% We return the two pages as strings, plus the actual expander (which is\n    %% an \"opaque\" data structure, but maybe interesting to see.)\n    {binary_to_list(list_to_binary(Page1)),\n     binary_to_list(list_to_binary(Page2)),\n     Expand}.\n\n\n%% call_cgi calls the script `Scriptfilename' (full path).  If\n%% `Exefilename' is given, it is the executable to handle this,\n%% otherwise `Scriptfilame' is assumed to be executable itself.\n%%\n%% Note however, that these functions usually generate stream content.\n%% (If you have good use for a version generating {content, _, _}\n%% instead, contact carsten@codimi.de)\n%%\n%% Also note, that they may return `get_more' and expect to be called\n%% again.\n\ncall_cgi(Arg, Scriptfilename) ->\n    yaws_cgi:call_cgi(Arg, Scriptfilename).\n\ncall_cgi(Arg, Exefilename, Scriptfilename) ->\n    yaws_cgi:call_cgi(Arg, Exefilename, Scriptfilename).\n\n%% call_fci_responder issues a responder role call to the FastCGI\n%% application server. It returns the same return value as out/1.\n%%\n%% call_fci_authorizer issues a authorizer role call to the FastCGI\n%% application server. It returns:\n%%\n%% {denied, Out} : Access is denied. Out is the same return value as\n%% out/1.\n%%\n%% {allowed, Variables} : Access is allowed. Variables is a list of\n%% environment variables returned by the authorization server using\n%% Variable-XXX: YYY headers.\n%%\n%% Note: the FastCGI filter role is not yet supported.\n%%\n%% The following information is taken from the server configuration:\n%% - The hostname (or address) and port number of the application server.\n%% - Extra CGI variables.\n%% - Trace FastCGI protocol messages?\n%% - Log application server error messages?\n%%\n%% The caller can optionally provide an Options argument which supports\n%% the following options. These override the defaults taken from the\n%% server config.\n%%\n%% {app_server_host, string() | ip_address()} : The hostname or IP address\n%% of the application server.\n%%\n%% {app_server_port, int()} : The TCP port number of the application server.\n%%\n%% {path_info, string()} : Override the pathinfo string from Arg.\n%%\n%% {extra_env, [{string()|binary(), string()|binary()}]} : Extra\n%% environment variables to be passed to the application server, as a list\n%% of name-value pairs.\n%%\n%% trace_protocol : Trace FastCGI protocol messages.\n%%\n%% log_app_error : Log application errors (output to stderr and non-zero\n%% exit value).\n%%\ncall_fcgi_responder(Arg) ->\n    yaws_cgi:call_fcgi_responder(Arg).\n\ncall_fcgi_responder(Arg, Options) ->\n    yaws_cgi:call_fcgi_responder(Arg, Options).\n\ncall_fcgi_authorizer(Arg) ->\n    yaws_cgi:call_fcgi_authorizer(Arg).\n\ncall_fcgi_authorizer(Arg, Options) ->\n    yaws_cgi:call_fcgi_authorizer(Arg, Options).\n\n%%\n\ndeepmember(_C,[]) ->\n    false;\ndeepmember(C,[C|_Cs]) ->\n    true;\ndeepmember(C,[L|Cs]) when is_list(L) ->\n    case deepmember(C,L) of\n        true  -> true;\n        false -> deepmember(C,Cs)\n    end;\ndeepmember(C,[N|Cs]) when C /= N ->\n    deepmember(C, Cs);\ndeepmember(_C,<<>>) ->\n    false;\ndeepmember(C, <<C,_Cs/binary>>) ->\n    true;\ndeepmember(C, <<_,Cs/binary>>) ->\n    deepmember(C, Cs).\n\n\n%%  . Parse a Set-Cookie header, following the RFC6265:\n%%\n%% \"Set-Cookie: \" set-cookie-string\n%%    set-cookie-string = cookie-pair *( \";\" SP cookie-av )\n%%    cookie-pair       = cookie-name \"=\" cookie-value\n%%    cookie-name       = token\n%%    cookie-value      = *cookie-octet / ( DQUOTE *cookie-octet DQUOTE )\n%%    cookie-octet      = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E\n%%    token             = <token, defined in [RFC2616], Section 2.2>\n%%\n%%    cookie-av         = expires-av / max-age-av / domain-av / path-av /\n%%                        secure-av / httponly-av / extension-av\n%%    expires-av        = \"Expires=\" <rfc1123-date, defined in [RFC2616]>\n%%    max-age-av        = \"Max-Age=\" [1-9] *DIGIT\n%%    domain-av         = \"Domain=\" <subdomain> ; defined in [RFC1034]\n%%    path-av           = \"Path=\" <any CHAR except CTLs or \";\">\n%%    secure-av         = \"Secure\"\n%%    httponly-av       = \"HttpOnly\"\n%%    extension-av      = <any CHAR except CTLs or \";\">\n%%\n%% NOTE: in RFC2109 and RFC2965, multiple cookies, separated by comma, can be\n%% defined in a single header. So, To be backward compatible with these RFCs,\n%% comma is forbidden in 'path-av' and 'extension-av' except for double-quoted\n%% value.\n%%\n%%\n%%  . Parse a Cookie header, following the RFC6265:\n%%\n%% \"Cookie: \" cookie-string\n%%    cookie-string = cookie-pair *( \";\" SP cookie-pair )\n%%\n%% NOTE: To be backward compatible with RFCs, comma is considered as a cookie\n%% separator, like semicolon.\n%%\nparse_set_cookie(Str) ->\n    parse_set_cookie(Str, []).\n\nparse_set_cookie([], [SetCookie]) ->\n    SetCookie;\nparse_set_cookie([], SetCookies) ->\n    lists:reverse(SetCookies);\nparse_set_cookie(Str, SetCookies) ->\n    case do_parse_set_cookie(Str) of\n        {#setcookie{extensions=Exts}=C0, Rest} ->\n            C1 = C0#setcookie{extensions=lists:reverse(Exts)},\n            parse_set_cookie(Rest, [C1|SetCookies]);\n        error ->\n            []\n    end.\n\n\ndo_parse_set_cookie(Str) ->\n    {Key, Rest0} = parse_cookie_key(skip_space(Str), []),\n    case yaws:to_lower(Key) of\n        [] ->\n            error;\n        K ->\n            Cookie0 = #setcookie{key=K, quoted=false},\n            case skip_space(Rest0) of\n                [$=|Rest1] ->\n                    {V, Q, Rest2} = parse_cookie_value(skip_space(Rest1)),\n                    Cookie1 = Cookie0#setcookie{value=V, quoted=Q},\n                    parse_set_cookie_result(Cookie1, skip_space(Rest2));\n                [$;|Rest1] -> parse_set_cookie_options(Rest1, Cookie0);\n                [$,|Rest1] -> {Cookie0, Rest1};\n                []         -> {Cookie0, []};\n                _          -> error\n            end\n    end.\n\nparse_set_cookie_options(Str, Cookie0) ->\n    {Key, Rest0} = parse_cookie_key(skip_space(Str), []),\n    case yaws:to_lower(Key) of\n        [] ->\n            {Cookie0, Rest0};\n        \"domain\" ->\n            case skip_space(Rest0) of\n                [$=|Rest1] ->\n                    {V,_,Rest2} = parse_set_cookie_domain(skip_space(Rest1),[]),\n                    Cookie1 = Cookie0#setcookie{domain=V},\n                    parse_set_cookie_result(Cookie1, skip_space(Rest2));\n                [$;|Rest1] -> parse_set_cookie_options(Rest1, Cookie0);\n                [$,|Rest1] -> {Cookie0, Rest1};\n                []         -> {Cookie0, []};\n                _          -> error\n            end;\n        \"max-age\" ->\n            case skip_space(Rest0) of\n                [$=|Rest1] ->\n                    {V,_,Rest2} = parse_set_cookie_maxage(skip_space(Rest1),[]),\n                    Cookie1 = Cookie0#setcookie{max_age=V},\n                    parse_set_cookie_result(Cookie1, skip_space(Rest2));\n                [$;|Rest1] -> parse_set_cookie_options(Rest1, Cookie0);\n                [$,|Rest1] -> {Cookie0, Rest1};\n                []         -> {Cookie0, []};\n                _          -> error\n            end;\n        \"expires\" ->\n            case skip_space(Rest0) of\n                [$=|Rest1] ->\n                    {V, _, Rest2} = parse_set_cookie_expires(skip_space(Rest1)),\n                    Cookie1 = Cookie0#setcookie{expires=V},\n                    parse_set_cookie_result(Cookie1, skip_space(Rest2));\n                [$;|Rest1] -> parse_set_cookie_options(Rest1, Cookie0);\n                [$,|Rest1] -> {Cookie0, Rest1};\n                []         -> {Cookie0, []};\n                _          -> error\n            end;\n        \"path\" ->\n            case skip_space(Rest0) of\n                [$=|Rest1] ->\n                    {V, _, Rest2} = parse_cookie_value(skip_space(Rest1)),\n                    Cookie1 = Cookie0#setcookie{path=V},\n                    parse_set_cookie_result(Cookie1, skip_space(Rest2));\n                [$;|Rest1] -> parse_set_cookie_options(Rest1, Cookie0);\n                [$,|Rest1] -> {Cookie0, Rest1};\n                []         -> {Cookie0, []};\n                _          -> error\n            end;\n        \"secure\" ->\n            Cookie1 = Cookie0#setcookie{secure=true},\n            parse_set_cookie_result(Cookie1, skip_space(Rest0));\n        \"httponly\" ->\n            Cookie1 = Cookie0#setcookie{http_only=true},\n            parse_set_cookie_result(Cookie1, skip_space(Rest0));\n        K ->\n            Exts = Cookie0#setcookie.extensions,\n            case skip_space(Rest0) of\n                [$=|Rest1] ->\n                    {V, Q, Rest2} = parse_cookie_value(skip_space(Rest1)),\n                    Cookie1 = Cookie0#setcookie{extensions=[{K,V,Q}|Exts]},\n                    parse_set_cookie_result(Cookie1, skip_space(Rest2));\n                [$;|Rest1] ->\n                    Cookie1 = Cookie0#setcookie{\n                                extensions=[{K,undefined,false}|Exts]\n                               },\n                    parse_set_cookie_options(Rest1, Cookie1);\n                [$,|Rest1] ->\n                    Cookie1 = Cookie0#setcookie{\n                                extensions=[{K,undefined,false}|Exts]\n                               },\n                    {Cookie1, Rest1};\n                [] ->\n                    Cookie1 = Cookie0#setcookie{\n                                extensions=[{K,undefined,false}|Exts]\n                               },\n                    {Cookie1, []};\n                _ ->\n                    error\n            end\n    end.\n\n\nparse_set_cookie_domain([C|_]=Rest, []) when C < $A orelse C > $Z orelse\n                                             C < $a orelse C > $z orelse\n                                             C /= $. ->\n    parse_cookie_value(Rest);\nparse_set_cookie_domain([C|_]=Rest, [_|_]=Acc) when C < $0 orelse C > $9 orelse\n                                                    C < $A orelse C > $Z orelse\n                                                    C < $a orelse C > $z orelse\n                                                    C /= $. orelse C /= $- ->\n    {lists:reverse(Acc), false, Rest};\nparse_set_cookie_domain([], Acc) ->\n    {lists:reverse(Acc), false, []};\nparse_set_cookie_domain([C|T], Acc) ->\n    parse_set_cookie_domain(T, [C|Acc]).\n\n\nparse_set_cookie_maxage([C|_]=Rest, []) when C < $1 orelse C > $9 ->\n    parse_cookie_value(Rest);\nparse_set_cookie_maxage([C|_]=Rest, [_|_]=Acc) when C < $0 orelse C > $9 ->\n    {lists:reverse(Acc), false, Rest};\nparse_set_cookie_maxage([], Acc) ->\n    {lists:reverse(Acc), false, []};\nparse_set_cookie_maxage([C|T], Acc) ->\n    parse_set_cookie_maxage(T, [C|Acc]).\n\n\n%% First of all, try to parse valid rfc1123 date (faster), then use a regex\n%% (more permissive)\nparse_set_cookie_expires([D,A,Y,$,,$\\s,D1,D2,SEP,M,O,N,SEP,Y1,Y2,Y3,Y4,$\\s,\n                          H1,H2,$:,M1,M2,$:,S1,S2,$\\s,Z1,Z2,Z3|Rest])\n  when SEP =:= $- orelse SEP =:= $\\s ->\n    {[D,A,Y,$,,$\\s,D1,D2,SEP,M,O,N,SEP,Y1,Y2,Y3,Y4,$\\s,\n      H1,H2,$:,M1,M2,$:,S1,S2,$\\s,Z1,Z2,Z3], false, Rest};\nparse_set_cookie_expires(Str) ->\n    RE = \"^(\"\n        \"(?:[a-zA-Z]+,\\s+)?\"                    %% Week day\n        \"[0-9]+(?:\\s|-)[a-zA-Z]+(?:\\s|-)[0-9]+\" %% DD Month YYYY\n        \"\\s+[0-9]+:[0-9]+:[0-9]+\"               %% hh:mm:ss\n        \"(?:\\s+[a-zA-Z]+)?\"                     %% timezone\n        \")\"\n        \"(.*)$\",\n    case re:run(Str, RE, [{capture, all_but_first, list}, caseless]) of\n        {match, [Date, Rest]} -> {Date, false, Rest};\n        nomatch               -> parse_cookie_value(Str)\n    end.\n\n\nparse_set_cookie_result(Cookie, [$;|Rest]) ->\n    parse_set_cookie_options(Rest, Cookie);\nparse_set_cookie_result(Cookie, [$,|Rest]) ->\n    {Cookie, Rest};\nparse_set_cookie_result(Cookie, []) ->\n    {Cookie, []};\nparse_set_cookie_result(_, _) ->\n    error.\n\n\n%%\nparse_cookie(Str) ->\n    parse_cookie(Str, []).\n\nparse_cookie([], Cookies) ->\n    lists:reverse(Cookies);\nparse_cookie(Str, Cookies) ->\n    {Key, Rest0} = parse_cookie_key(skip_space(Str), []),\n    case yaws:to_lower(Key) of\n        [] ->\n            [];\n        K ->\n            case skip_space(Rest0) of\n                [$=|Rest1] ->\n                    {V, Q, Rest2} = parse_cookie_value(skip_space(Rest1)),\n                    C = #cookie{key=K, value=V, quoted=Q},\n                    case skip_space(Rest2) of\n                        [$;|Rest3] -> parse_cookie(Rest3, [C|Cookies]);\n                        [$,|Rest3] -> parse_cookie(Rest3, [C|Cookies]);\n                        []         -> parse_cookie([], [C|Cookies]);\n                        _          -> []\n                    end;\n                [$;|Rest1] -> parse_cookie(Rest1, [#cookie{key=K}|Cookies]);\n                [$,|Rest1] -> parse_cookie(Rest1, [#cookie{key=K}|Cookies]);\n                []         -> parse_cookie([], [#cookie{key=K}|Cookies]);\n                _          -> []\n            end\n    end.\n\n\n%%\n%% All CHAR except ('=' | ';' | ',' | SP | HT | CRLF | LF)\nparse_cookie_key([], Acc) ->\n    {lists:reverse(Acc), []};\nparse_cookie_key(T=[$=|_], Acc) ->\n    {lists:reverse(Acc), T};\nparse_cookie_key(T=[$;|_], Acc) ->\n    {lists:reverse(Acc), T};\nparse_cookie_key(T=[$,|_], Acc) ->\n    {lists:reverse(Acc), T};\nparse_cookie_key(T=[$\\s|_], Acc) ->\n    {lists:reverse(Acc), T};\nparse_cookie_key(T=[$\\t|_], Acc) ->\n    {lists:reverse(Acc), T};\nparse_cookie_key(T=[$\\r,$\\n|_], Acc) ->\n    {lists:reverse(Acc), T};\nparse_cookie_key(T=[$\\n|_], Acc) ->\n    {lists:reverse(Acc), T};\nparse_cookie_key([C|T], Acc) ->\n    parse_cookie_key(T, [C|Acc]).\n\n\n%%\nparse_cookie_value([$\"|T]) ->\n    parse_cookie_quoted(T,[]);\nparse_cookie_value(T) ->\n    parse_cookie_value(T,[]).\n\n%% All CHAR except (';' | ',' | SP | HT | CRLF | LF)\nparse_cookie_value([],Acc) ->\n    {lists:reverse(Acc), false, []};\nparse_cookie_value(T=[$;|_], Acc) ->\n    {lists:reverse(Acc), false, T};\nparse_cookie_value(T=[$,|_], Acc) ->\n    {lists:reverse(Acc), false, T};\nparse_cookie_value(T=[$\\s|_], Acc) ->\n    {lists:reverse(Acc), false, T};\nparse_cookie_value(T=[$\\t|_], Acc) ->\n    {lists:reverse(Acc), false, T};\nparse_cookie_value(T=[$\\r,$\\n|_], Acc) ->\n    {lists:reverse(Acc), false, T};\nparse_cookie_value(T=[$\\n|_], Acc) ->\n    {lists:reverse(Acc), false, T};\nparse_cookie_value([C|T], Acc) ->\n    parse_cookie_value(T, [C|Acc]).\n\n\n%% All CHAR except ('\"' | CTLs) but including LWS and escape DQUOTEs\n%%   CTL = any US-ASCII control character (octets 0 - 31) and DEL (127)\n%%   LWS = [CRLF] 1*( SP | HT )\nparse_cookie_quoted([], Acc) ->\n    {lists:reverse(Acc), true, []};\nparse_cookie_quoted([$\"|T], Acc) ->\n    {lists:reverse(Acc), true, T};\nparse_cookie_quoted([$\\\\,C|T], Acc) ->\n    parse_cookie_quoted(T,[C,$\\\\|Acc]);\nparse_cookie_quoted([$\\t|T], Acc) ->\n    parse_cookie_quoted(T,[$\\t|Acc]);\nparse_cookie_quoted([$\\r,$\\n,$\\s|T], Acc) ->\n    parse_cookie_quoted(T,[$\\s,$\\n,$\\r|Acc]);\nparse_cookie_quoted([$\\r,$\\n,$\\t|T], Acc) ->\n    parse_cookie_quoted(T,[$\\t,$\\n,$\\r|Acc]);\nparse_cookie_quoted([C|T], Acc) when C > 31 andalso C < 127 ->\n    parse_cookie_quoted(T,[C|Acc]);\nparse_cookie_quoted(T, Acc) ->\n    {lists:reverse(Acc), true, T}.\n\n\n%%\nformat_set_cookie(C) when C#setcookie.value == undefined ->\n    [C#setcookie.key|format_cookie_opts(C)];\nformat_set_cookie(C) when C#setcookie.quoted ->\n    [C#setcookie.key,$=,$\",C#setcookie.value,$\"|format_cookie_opts(C)];\nformat_set_cookie(C) ->\n    [C#setcookie.key,$=,C#setcookie.value|format_cookie_opts(C)].\n\n%%\nformat_cookie([Cookie]) ->\n    format_cookie(Cookie);\nformat_cookie([Cookie|Rest]) ->\n    [format_cookie(Cookie),$;,$\\s|format_cookie(Rest)];\nformat_cookie(#cookie{key=Key, value=undefined}) ->\n    Key;\nformat_cookie(#cookie{key=Key, value=Value, quoted=true}) ->\n    [Key,$=,$\",Value,$\"];\nformat_cookie(#cookie{key=Key, value=Value, quoted=false}) ->\n    [Key,$=,Value].\n\n%%\nformat_cookie_opts(C=#setcookie{}) ->\n    [\n     add_opt(\"Domain\",   C#setcookie.domain,    false),\n     add_opt(\"Max-Age\",  C#setcookie.max_age,   false),\n     add_opt(\"Expires\",  C#setcookie.expires,   false),\n     add_opt(\"Path\",     C#setcookie.path,      false),\n     add_opt(\"Secure\",   C#setcookie.secure,    false),\n     add_opt(\"HttpOnly\", C#setcookie.http_only, false)\n    ] ++ [add_opt(K,V,Q) || {K,V,Q} <- C#setcookie.extensions].\n\n\nadd_opt(_, undefined, _) -> [];\nadd_opt(_, false, _)     -> [];\nadd_opt(Key, true, _)    -> [$;,$\\s,Key];\nadd_opt(Key, Opt, true)  -> [$;,$\\s,Key,$=,$\",Opt,$\"];\nadd_opt(Key, Opt, false)  -> [$;,$\\s,Key,$=,Opt].\n\n\n%%\nskip_space([])          -> [];\nskip_space([$\\s|T])     -> skip_space(T);\nskip_space([$\\t|T])     -> skip_space(T);\nskip_space([$\\r,$\\n|T]) -> skip_space(T);\nskip_space([$\\n|T])     -> skip_space(T);\nskip_space(T)           -> T.\n\n\n%%\ngetvar(ARG,Key) when is_atom(Key) ->\n    getvar(ARG, atom_to_list(Key));\ngetvar(ARG,Key) ->\n    filter_parse(Key, yaws_api:parse_query(ARG), yaws_api:parse_post(ARG)).\n\n\nqueryvar(ARG,Key) when is_atom(Key) ->\n    queryvar(ARG, atom_to_list(Key));\nqueryvar(ARG, Key) ->\n    filter_parse(Key, yaws_api:parse_query(ARG), []).\n\npostvar(ARG, Key) when is_atom(Key) ->\n    postvar(ARG, atom_to_list(Key));\npostvar(ARG, Key) ->\n    filter_parse(Key, [], yaws_api:parse_post(ARG)).\n\nfilter_parse(Key, QueryParse, PostParse) ->\n    Fun = fun({K,V}) -> (Key == K andalso V /= undefined) end,\n    Values = lists:filter(Fun, QueryParse) ++ lists:filter(Fun, PostParse),\n    case Values of\n        [] -> undefined;\n        [{_, V}] -> {ok,V};\n        %% Multivalued case - return list of values\n        _  -> list_to_tuple(lists:map(fun({_,V}) -> V end, Values))\n    end.\n\n\nbinding(Key) ->\n    case get({binding, Key}) of\n        undefined -> erlang:error({unknown_binding, Key});\n        Value -> Value\n    end.\n\nbinding_exists(Key) ->\n    case get({binding, Key}) of\n        undefined -> false;\n        _ -> true\n    end.\n\n\n\n%% Return the parsed url that the client requested.\nrequest_url(ARG) ->\n    SC = get(sc),\n    Headers = ARG#arg.headers,\n    {abs_path, Path} = (ARG#arg.req)#http_request.path,\n    DecPath = url_decode(Path),\n    {P,Q} = yaws:split_at(DecPath, $?),\n    #url{scheme = case SC#sconf.ssl of\n                      undefined ->\n                          \"http\";\n                      _ ->\n                          \"https\"\n                  end,\n         host = case Headers#headers.host of\n                    undefined ->\n                        yaws:upto_char($:, SC#sconf.servername);\n                    HostHdr ->\n                        yaws:upto_char($:, HostHdr)\n                end,\n         port = case {SC#sconf.ssl, SC#sconf.port} of\n                    {_, 80} ->\n                        undefined;\n                    {_, 443} ->\n                        undefined;\n                    {_, Port} ->\n                        Port\n                end,\n         path = P,\n         querypart = Q}.\n\n\n\n%% remove sick characters\n\nsanitize_file_name(\"..\" ++ T) ->\n    sanitize_file_name([$.|T]);\nsanitize_file_name([H|T]) ->\n    case lists:member(H,  \" &;'`{}!\\\\?<>\\\"()$\") of\n        true ->\n            sanitize_file_name(T);\n        false ->\n            [H|sanitize_file_name(T)]\n    end;\nsanitize_file_name([]) ->\n    [].\n\n\n\n%% to be used in embedded mode, make it possible\n%% to pass a config to yaws from another data source\n%% than /etc/yaws/yaws.conf, for example from a database\n%% this code is also called by the server -h hup code\nsetconf(GC0, Groups0) ->\n    setconf(GC0, Groups0, true).\nsetconf(GC0, Groups0, CheckCertsChanged) ->\n    case CheckCertsChanged of\n        true ->\n            CertCheck = gen_server:call(yaws_server, check_certs, infinity),\n            case lists:member(yes, CertCheck) of\n                true ->\n                    application:stop(ssl),\n                    application:start(ssl);\n                false ->\n                    ok\n            end;\n        false ->\n            ok\n    end,\n\n    {GC, Groups1} = yaws_config:verify_upgrade_args(GC0, Groups0),\n    Groups2 = lists:map(fun(X) -> yaws_config:add_yaws_auth(X) end, Groups1),\n    {ok, OLDGC, OldGroups} = yaws_api:getconf(),\n    case {yaws_config:can_hard_gc(GC, OLDGC),\n          yaws_config:can_soft_setconf(GC, Groups2, OLDGC, OldGroups)} of\n        {true, true} ->\n            yaws_config:soft_setconf(GC, Groups2, OLDGC, OldGroups);\n        {true, false} ->\n            ok = yaws_config:hard_setconf(GC, Groups2);\n        _ ->\n            {error, need_restart}\n    end.\n\n%% return {ok, GC, Groups}.\ngetconf() ->\n    gen_server:call(yaws_server, getconf, infinity).\n\n%% return listen port number for the given sconf, useful if yaws is used in\n%% a test scenario where the configured port number is 0 (for requesting an\n%% ephemeral port)\nget_listen_port(SC) ->\n    yaws_server:listen_port(SC).\n\nembedded_start_conf(DocRoot) when is_list(DocRoot) ->\n    embedded_start_conf(DocRoot, []).\nembedded_start_conf(DocRoot, SL) when is_list(DocRoot), is_list(SL) ->\n    embedded_start_conf(DocRoot, SL, []).\nembedded_start_conf(DocRoot, SL, GL)\n  when is_list(DocRoot), is_list(SL), is_list(GL) ->\n    embedded_start_conf(DocRoot, SL, GL, \"default\").\nembedded_start_conf(DocRoot, SL, GL, Id)\n  when is_list(DocRoot), is_list(SL), is_list(GL) ->\n    case application:load(yaws) of\n        ok -> ok;\n        {error, {already_loaded,yaws}} -> ok;\n        _ -> exit(\"cannot load yaws\")\n    end,\n    ok = application:set_env(yaws, embedded, true),\n    ok = application:set_env(yaws, id, Id),\n    ChildSpecs = yaws_sup:child_specs(),\n    GC = yaws:create_gconf(GL, Id),\n    SCList  = case SL of\n                  [] ->\n                      [[]];\n                  [Cnf|_] when is_tuple(Cnf) ->\n                      [[yaws:create_sconf(DocRoot, SL)]];\n                  [Cnf|_] when is_list(Cnf) ->\n                      [[yaws:create_sconf(DocRoot, SLItem)] || SLItem <- SL]\n              end,\n    SoapChild = yaws_config:add_yaws_soap_srv(GC, false),\n\n    %% In case a server is started before any configuration has been set,\n    %% this makes it possible to get hold of the 'pending' configuration.\n    %% (see for example the start of the yaws_session_server)\n    ok = application:set_env(yaws, embedded_conf, [{sclist,SCList},{gc,GC}]),\n\n    yaws:mkdir(yaws:id_dir(Id)),\n    {ok, SCList, GC, ChildSpecs ++ SoapChild}.\n\n\n%% Function which is invoked typically from an index.yaws file\ndir_listing(Arg) ->\n    dir_listing(Arg, \".\").\ndir_listing(Arg, RelDir) ->\n    %% .yaws.auth\n    Dir0 = filename:dirname(Arg#arg.fullpath),\n    Dir = case RelDir of\n              \".\" -> Dir0;\n              _ -> filename:join([Dir0, RelDir])\n          end,\n    Req = Arg#arg.req,\n    case file:list_dir(Dir) of\n        {ok, Data0} ->\n            Data = Data0 -- [\".yaws.auth\", \"index.yaws\"],\n            yaws_ls:list_directory(Arg, Arg#arg.clisock, Data,\n                                   Dir,\n                                   Req,  false),\n            ok;\n        _Err ->\n            %% Just ignore errors ??, the programmer has to\n            %% make sure it's a valid path here\n            ok\n    end.\n\n%% Returns #redir_self{} record\nredirect_self(A) ->\n    SC = get(sc),\n    {Port, PortStr} =\n        case {SC#sconf.rmethod, SC#sconf.ssl, SC#sconf.port} of\n            {\"https\", _, 443} -> {443, \"\"};\n            {\"http\", _, 80} -> {80, \"\"};\n            {_, undefined, 80} -> {80, \"\"};\n            {_, undefined, Port2} ->\n                {port, [$:|integer_to_list(Port2)]};\n            {_, _SSL, 443} ->\n                {443, \"\"};\n            {_, _SSL, Port2} ->\n                {Port2, [$:|integer_to_list(Port2)]}\n        end,\n    H = A#arg.headers,\n    Host0 = yaws:redirect_host(get(sc), H#headers.host),\n    %% redirect host contains the port number - for mysterious reasons\n    Host = case string:tokens(Host0, \":\") of\n               [H0, _] -> H0;\n               [H1] -> H1\n           end,\n    {Scheme, SchemeStr} =\n        case {SC#sconf.ssl,SC#sconf.rmethod} of\n            {_, Method} when is_list(Method) ->\n                {list_to_atom(Method), Method++\"://\"};\n            {undefined,_} ->\n                {http, \"http://\"};\n            {_SSl,_} ->\n                {https, \"https://\"}\n        end,\n    #redir_self{host = Host,\n                scheme = Scheme,\n                scheme_str = SchemeStr,\n                port = Port,\n                port_str = PortStr}.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_app.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_app.erl\n%%% Author  : Claes Wikstrom <klacke@bluetail.com>\n%%% Purpose :\n%%% Created : 16 Jan 2002 by Claes Wikstrom <klacke@hyber.org>\n%%%----------------------------------------------------------------------\n\n-module(yaws_app).\n-author('klacke@hyber.org').\n\n\n-behaviour(application).\n-export([start/2,stop/1]).\n\n%% start\n\nstart(_Type, _StartArgs) ->\n  yaws_sup:start_link().\n\n%% stop\n\nstop(_State) ->\n  ok.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_appdeps.hrl",
    "content": "-define(YAWS_APPDEPS, \"\").\n"
  },
  {
    "path": "contrib/yaws/src/yaws_appmod_cgi.erl",
    "content": "%%% File    : yaws_appmod_cgi.erl\n%%% Author  : Claes  Wikstrom <klacke@hyber.org>\n%%% Description :\n%%% Created : 10 Mar 2008 by Claes  Wikstrom <klacke@hyber.org>\n\n-module(yaws_appmod_cgi).\n-export([out/1]).\n-include(\"../include/yaws_api.hrl\").\n\nout(Arg) ->\n    yaws_cgi:call_cgi(Arg,  lists:flatten(Arg#arg.fullpath)).\n\n\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_appmod_dav.erl",
    "content": "%%%-------------------------------------------------------------------\n%%% File    : yaws_appmod_dav.erl\n%%% Created : 11 Nov 2012\n%%% Purpose : WebDav module, RFC4918 class 1, 2, 3 compliant\n%%%\n%%%           To use, add the following configuration:\n%%%\n%%%               <server>\n%%%                   ...\n%%%                   dav = true\n%%%               </server>\n%%%\n%%%           This configuration is short for:\n%%%\n%%%               runmod = yaws_runmod_lock\n%%%               ...\n%%%               <server>\n%%%                   ...\n%%%                   appmods = </, yaws_appmod_dav>\n%%%               <server>\n%%%\n%%% Todo    : 1) Add header handling (most of the time not used by DAV\n%%%              clients):\n%%%              If-Match, If-Modified-Since, If-None-Match,\n%%%              If-Range, If-Unmodified-Since, TE\n%%%           2) POST on collections\n%%%-------------------------------------------------------------------\n\n-module(yaws_appmod_dav).\n\n%% for appmod:\n-export([out/1]).\n\n%% for replacement xmerl_xml:\n-export([export/1]).\n-import(xmerl_lib, [markup/3, empty_tag/2, export_text/1]).\n\n\n-ifdef(debug).\n-define(DEBUG(X), io:format(X)).\n-define(DEBUG(X,Y), io:format(X,Y)).\n-else.\n-define(DEBUG(X), true).\n-define(DEBUG(X,Y), true).\n-endif.\n\n-include(\"../include/yaws_dav.hrl\").\n-include(\"../include/yaws_lock.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"../include/yaws.hrl\").\n-include(\"yaws_debug.hrl\").\n-include_lib(\"xmerl/include/xmerl.hrl\").\n-include_lib(\"kernel/include/file.hrl\").\n\nout(A) ->\n    try\n        h_litmus(A),\n        %% handle the request\n        R = A#arg.req,\n        Method = R#http_request.method,\n        H = A#arg.headers,\n        C = if\n                H#headers.user_agent == undefined -> standard;\n                true ->\n                    Ms = string:str(H#headers.user_agent,\"Microsoft\"),\n                    if\n                        Ms > 0 -> microsoft;\n                        true -> standard\n                    end\n            end,\n        put(compatibility,C),\n        handle(Method,A)\n    catch\n        {Status,Precondition} ->\n            Response1 = [{'D:error', [{'xmlns:D',\"DAV:\"}],\n                          [{Precondition,[],[]}]}],\n            status(Status,{xml,Response1});\n        Status ->\n            status(Status);\n        _Error:{noproc,{gen_server,call,[yaws_runmod_lock|_Whatever]}} ->\n            Msg = \"Lock server not started. See documentation.~n\",\n            error_logger:error_msg(Msg),\n            Response = [{'D:error',[{'xmlns:D',\"DAV:\"}],[Msg]}],\n            status(500,{xml,Response});\n        _Error:Reason ->\n            error_logger:info_msg(\"unexpected error: ~p~n~p~n\",\n                                  [Reason,erlang:get_stacktrace()]),\n            Response = [{'D:error',[{'xmlns:D',\"DAV:\"}],[Reason]}],\n            status(500,{xml,Response})\n    end.\n\n%%------------------------------------------------------\n%% handle methods\n%%\nhandle('OPTIONS',A) ->\n    ?DEBUG(\"OPTIONS\"),\n    R = davresource0(A),\n    F = R#resource.info,\n    T = case F#file_info.type of\n            directory -> \"httpd/unix-directory\";\n            _ ->\n              Name = R#resource.name,\n              Ext = filename:extension(Name),\n              Ext1 = case Ext of\n                         [] -> \"\";\n                         _ -> tl(Ext)\n                     end,\n              {_Kind,Mimetype} = mime_types:t(Ext1),\n              Mimetype\n        end,\n    Headers = [{header,{\"Allow\",\n                        \"GET, POST, OPTIONS, HEAD, PUT, DELETE, \"\n                        \"PROPFIND, PROPPATCH, LOCK, UNLOCK, \"\n                        \"MKCOL, MOVE, COPY\"}}],\n    status(200,[{header,{\"Content-Type\",T}}|Headers],[]);\nhandle('HEAD',A) ->\n    ?DEBUG(\"HEAD ~p\",[A#arg.server_path]),\n    Path = davpath(A),\n    case file:read_file_info(Path) of\n        {ok, F} when (F#file_info.type == regular) ->\n            E = yaws:make_etag(F),\n            Name = A#arg.server_path,\n            Ext = filename:extension(Name),\n            Ext1 = case Ext of\n                       [] -> \"\";\n                       _ -> tl(Ext)\n                   end,\n            {_Kind,T} = mime_types:t(Ext1),\n            status(200,[{header,{\"Etag\",E}},{header,{\"Content-Type\",T}}]);\n        {ok, F} when (F#file_info.type == directory) ->\n            status(200,[{header,{\"Content-Type\",\"httpd/unix-directory\"}}]);\n        _ -> throw(404)\n    end;\nhandle('GET',A) ->\n    ?DEBUG(\"GET ~p\",[A#arg.server_path]),\n    Name = A#arg.server_path,\n    Path = A#arg.fullpath,\n    Pid = self(),\n    SC=get(sc),\n    PPS = SC#sconf.partial_post_size,\n    case file:read_file_info(Path) of\n        {ok,F} when F#file_info.type==directory ->\n            {ok,Dir} = file:list_dir(Path),\n            Listing = lists:foldl(\n                        fun(Fname,L) ->\n                            {ok,Finfo} = file:read_file_info(\n                                           filename:join(Path,Fname)),\n                            Ftype = case Finfo#file_info.type of\n                                        directory -> \"Coll:\";\n                                        _         -> \"     \"\n                                    end,\n                            Fsize = integer_to_list(Finfo#file_info.size),\n                            Ftime = yaws:local_time_as_gmt_string(\n                                      Finfo#file_info.mtime),\n                            Entry = io_lib:format(\"~s ~-20s ~10s  ~s~n\",\n                                                  [Ftype,Fname,Fsize,Ftime]),\n                            L++Entry\n                        end,\"\",lists:sort(Dir)),\n            Response = {ehtml,[\n                {html,[],[\n                    {head,[],[\n                        {title,[],Name}]\n                    },\n                    {body,[],[\n                        {h2,[],\"Index of \"++Name},\n                        {pre,[],Listing}]\n                    }]\n                }]\n            },\n            status(200,Response);\n        {ok,F} ->\n            Size = integer_to_list(F#file_info.size),\n            Ext = filename:extension(Name),\n            Ext1 = case Ext of\n                       [] -> \"\";\n                       _ -> tl(Ext)\n                   end,\n            {_Kind,Mimetype} = mime_types:t(Ext1),\n            H = [{header,{\"Content-Length\",Size}}],\n            {ok,Fd} = file:open(Path,[read,binary]),\n            case file:read(Fd,PPS) of\n                {ok,Data} when size(Data)<PPS ->\n                    ?DEBUG(\"only chunk~n\"),\n                    status(200,H,{content,Mimetype,Data});\n                {ok,Data} ->\n                    ?DEBUG(\"first chunk~n\"),\n                    spawn(fun() -> deliver_rest(Pid,Fd) end),\n                    status(200,H,{streamcontent,Mimetype,Data});\n                eof ->\n                    status(200,{content,\"application/octet-stream\",<<>>});\n                {error,Reason} ->\n                    Response = [{'D:error',[{'xmlns:D',\"DAV:\"}],[Reason]}],\n                    status(500,{xml,Response})\n            end;\n        {error,enoent} ->\n            status(404);\n        _Other ->\n            status(500)\n    end;\nhandle('POST',A) ->\n    ?DEBUG(\"POST ~p\",[A#arg.server_path]),\n    _Path = davpath(A),\n    %% TODO POST for collections: RFC5995\n    status(501);\nhandle(\"LOCK\",A) ->\n    %% TODO Multi resource lock (lock on collection) returns 207\n    %%      (multi-status) when failing\n    ?DEBUG(\"LOCK ~p\",[A#arg.server_path]),\n    Name = A#arg.server_path,\n    Path = davpath(A),\n    %% check if file/collection exists and create if not so\n    %% RFC4918 - 9.10.4\n    {Status,R} = case file:read_file_info(Path) of\n            {ok, F} when (F#file_info.type == directory) or\n                         (F#file_info.type == regular) ->\n                {200,#resource{ name = Name, info = F}};\n            {error,enoent} ->\n                case string:right(A#arg.server_path,1) of\n                    \"/\" ->\n                        ok = file:make_dir(Path);\n                    _ ->\n                        ok = file:write_file(Path,\"\")\n                end,\n                {ok, F} = file:read_file_info(Path),\n                {201,#resource{ name = Name, info = F}};\n            {error,_} -> throw(409)\n        end,\n    Req = binary_to_list(A#arg.clidata),\n    L = parse_lockinfo(Req),\n    %%Id = h_locktoken(A),\n    Timeout = h_timeout(A),\n    Depth = h_depth(A),\n    Id = h_if_refresh(A,Path),\n    case yaws_runmod_lock:lock(Path,L#lock{path=Path,id=Id,timeout=Timeout,\n                                           depth=Depth}) of\n        {ok,Id1} ->\n            {200,Result} = prop_get({'DAV:',lockdiscovery},A,R),\n            Response = [{'D:prop', [{'xmlns:D',\"DAV:\"}], [Result]}],\n            status(Status,[{header,\n                            {\"Lock-Token\",\"<opaquelocktoken:\"++Id1++\">\"}}],\n                   {xml,Response});\n        {error,locked} ->\n            status(423);\n        _ ->\n            throw(501)\n    end;\nhandle(\"UNLOCK\",A) ->\n    ?DEBUG(\"UNLOCK ~p\",[A#arg.server_path]),\n    Path = davpath(A),\n    Id = h_locktoken(A),\n    %%?DEBUG(\" Id=~p\",[Id]),\n    case yaws_runmod_lock:unlock(Path,Id) of\n        ok -> status(204);\n        not_found ->\n            Response = [{'D:error', [{'xmlns:D',\"DAV:\"}],\n                         [{'lock-token-matches-request-uri',[],[]}]}],\n            status(409,{xml,Response})\n    end;\nhandle('DELETE',A) ->\n    ?DEBUG(\"DELETE ~p\",[A#arg.server_path]),\n    Path = davpath(A),\n    h_if(A,Path),\n    R = davresource0(A),\n    %% use internal locking to be safe\n    case yaws_runmod_lock:lock(R#resource.name,\n                               #lock{depth=infinity,scope=exclusive}) of\n        {ok,Id} ->\n            F = filename:join(A#arg.docroot,[\"./\",R#resource.name]),\n            fs_rmrf(F),\n            yaws_runmod_lock:unlock(R#resource.name,Id),\n            status(200);\n        _ -> throw(423)\n    end;\nhandle('PUT',A) when A#arg.state == undefined ->\n    ?DEBUG(\"PUT ~p\",[A#arg.server_path]),\n    Path = davpath(A),\n    h_if(A,Path),\n    case filelib:is_dir(Path) of\n        true ->\n            throw(405);\n        false ->\n            TmpName = temp_name(Path),\n            X = file:open(TmpName, [raw,write]),\n            case X  of\n                {ok, Fd} ->\n                    State = #upload{fd=Fd, filename=Path, tempname=TmpName},\n                    case A#arg.clidata of\n                        {partial,Bin} ->\n                            file:write(Fd,Bin),\n                            {get_more,<<>>,State};\n                        Bin ->\n                            file:write(Fd,Bin),\n                            file:close(Fd),\n                            case file:rename(TmpName, Path) of\n                                ok -> status(201);\n                                _ -> status(409)\n                            end\n                    end;\n                {error,eexist} ->\n                    throw(405);\n                {error,enoent} -> throw(409);\n                {error,eisdir} -> throw(409);\n                {error,enospace} -> throw(507);\n                _ -> status(500)\n            end\n    end;\nhandle('PUT',A) ->\n    State = A#arg.state,\n    Fd = State#upload.fd,\n    case A#arg.clidata of\n        {partial,Bin} ->\n            file:write(Fd,Bin),\n            {get_more,<<>>,State};\n        Bin ->\n            file:write(Fd,Bin),\n            file:close(Fd),\n            FName = State#upload.filename,\n            TmpName = State#upload.tempname,\n            case file:rename(TmpName, FName) of\n                ok -> status(201);\n                _ -> status(409) % TODO delete temp file here?\n            end\n    end;\nhandle(\"MKCOL\",A) ->\n    ?DEBUG(\"MKCOL ~p\",[A#arg.server_path]),\n    Path = davpath(A),\n    if\n        %% RFC2518, 8.3.1\n        size(A#arg.clidata) > 0 -> throw(415);\n        true -> ok\n    end,\n    h_if(A,Path),\n    file_do(make_dir,[Path]),\n    status(201);\nhandle(\"COPY\",A) ->\n    ?DEBUG(\"COPY ~p\",[A#arg.server_path]),\n    From = A#arg.fullpath,\n    Dest = h_destination(A),\n    To = case {string:right(Dest,1),string:right(Dest,1)} of\n             {\"/\",_} -> Dest;\n             {_,\"/\"} -> filename:join(Dest,filename:basename(From));\n             _ -> Dest\n         end,\n    DoOverwrite = h_overwrite(A),\n    ToExists = exists(To),\n    if\n        DoOverwrite == false, ToExists == true ->\n            status(412);\n        true  ->\n            if ToExists == true ->\n                    h_if(A,To),\n                    fs_rmrf(To),\n                    fs_cp(From,To),\n                    status(204);\n               true ->\n                    fs_cp(From,To),\n                    status(201)\n                    %%status(201,[{'Location',To}],[])\n            end\n    end;\nhandle(\"MOVE\",A) ->\n    ?DEBUG(\"MOVE ~p\",[A#arg.server_path]),\n    From = davpath(A),\n    h_if(A,From),\n    Dest = h_destination(A),\n    To = case {string:right(Dest,1),string:right(Dest,1)} of\n             {\"/\",_} -> Dest;\n             {_,\"/\"} -> filename:join(Dest,filename:basename(From));\n             _ -> Dest\n         end,\n    DoOverwrite = h_overwrite(A),\n    ToExists = exists(To),\n    if\n        DoOverwrite == false, ToExists == true ->\n            status(412);\n        true  ->\n            if\n                ToExists == true ->\n                    h_if(A,To),\n                    fs_rmrf(To);\n                true ->\n                    ok\n            end,\n            case file:rename(From, To) of\n                ok when ToExists == true ->\n                    status(204);\n                ok ->\n                    status(201);\n                _ ->\n                    try\n                        fs_cp(From, To),\n                        fs_rmrf(From),\n                        status(201)\n                    catch\n                        throw:Status ->\n                            ?DEBUG(\" move from ~p to ~p failed: ~p\\n\",\n                                   [From, To, Status]),\n                            Response = [{'D:error', [{'xmlns:D',\"DAV:\"}],\n                                         [Status]}],\n                            status(Status,{xml,Response})\n                    end\n            end\n    end;\nhandle(\"PROPFIND\",A) ->\n    ?DEBUG(\"PROPFIND ~p\",[A#arg.server_path]),\n    Req = binary_to_list(A#arg.clidata),\n    Props = parse_propfind(Req),\n    R = davresource0(A),\n    case h_depth(A) of\n        0 ->\n            Response = {'D:response', [], propfind_response(Props,A,R)},\n            MultiStatus = [{'D:multistatus', [{'xmlns:D',\"DAV:\"}], [Response]}],\n            status(207,{xml,MultiStatus});\n        1 ->\n            R1 = davresource1(A),\n            Response = {'D:response', [], propfind_response(Props,A,R)},\n            Responses = [{'D:response', [],\n                          propfind_response(Props,A,Rx)} || Rx <- R1],\n            MultiStatus = [{'D:multistatus', [{'xmlns:D',\"DAV:\"}],\n                            [Response|Responses]}],\n            status(207,{xml,MultiStatus});\n        infinity ->\n            Response = [{'D:error', [{'xmlns:D',\"DAV:\"}],\n                         [{'propfind-finite-depth',[],[]}]}],\n            status(403,{xml,Response})\n    end;\nhandle(\"PROPPATCH\",A) ->\n    ?DEBUG(\"PROPPATCH ~p\",[A#arg.server_path]),\n    Path = davpath(A),\n    h_if(A,Path),\n    Req = binary_to_list(A#arg.clidata),\n    R = davresource0(A),\n    Update = parse_proppatch(Req),\n    Response = {'D:response',[],proppatch_response(Update,A,R)},\n    MultiStatus = [{'D:multistatus', [{'xmlns:D',\"DAV:\"}], [Response]}],\n    status(207,{xml,MultiStatus});\nhandle(_Other,_A) ->\n    status(405).\n\n\n%% --------------------------------------------------------\n%% File functions\n%%\n\n%% deliver chunked data\ndeliver_rest(Pid,Fd) ->\n    case file:read(Fd,10240) of\n        {ok, Data} ->\n            yaws_api:stream_chunk_deliver(Pid,Data),\n            deliver_rest(Pid,Fd);\n        eof ->\n            yaws_api:stream_chunk_end(Pid),\n            file:close(Fd);\n        {error,Reason} ->\n            exit(Reason)\n    end.\n\n%% do a recoverable/catchable file function\nfile_do(Func,Params) ->\n    Result = erlang:apply(file,Func,Params),\n    case Result of\n        ok -> ok;\n        {ok,X} -> {ok,X};\n        {ok,X1,X2} -> {ok,X1,X2};\n        eof -> eof;\n        {error,eexist} -> throw(405);\n        {error,enoent} -> throw(409);\n        {error,eisdir} -> throw(409);\n        {error,enospace} -> throw(507);\n        _Error -> ?DEBUG(\"file function returned ~p~n\",[_Error]),throw(500)\n    end.\n\n%% recursive remove, equivalent of rm -rf\nfs_rmrf(Path) ->\n    {ok, F} = file_do(read_file_info,[Path]),\n    case F#file_info.type of\n        directory ->\n            {ok, Dir} = file_do(list_dir,[Path]),\n            [ fs_rmrf(filename:join(Path,File)) || File <- Dir ],\n            ok = file:del_dir(Path);\n            %%file_do(del_dir,[Path]);\n        _ ->\n            ok = file:delete(Path)\n            %%file_do(delete,[Path])\n    end.\n\n%% recursive copy, equivalent of cp\nfs_cp(From,To) ->\n    %% All checks on existence of the destination have to be done before\n    %% so destination should not exist\n    {ok, F} = file:read_file_info(From),\n    case F#file_info.type of\n        directory ->\n            file_do(make_dir,[To]),\n            {ok, Dir} = file:list_dir(From),\n            [ fs_cp(filename:join(From,File),filename:join(To,File)) ||\n                File <- Dir ];\n        _ ->\n            file_do(copy,[From,To])\n    end.\n\n%% check existence\nexists(Path) ->\n    case file:read_file_info(Path) of\n        {ok, _} -> true;\n        _ -> false\n    end.\n\n%% generate a temporary filename as a dotted file with a timestamp\ntemp_name(F) ->\n    {A,B,C} = yaws:get_time_tuple(),\n    Path = filename:dirname(F),\n    File = filename:basename(F),\n    T0 = io_lib:format(\"~s/.~s.~p-~p-~p\",[Path,File,A,B,C]),\n    lists:flatten(T0).\n\n%% --------------------------------------------------------\n%% Property functions\n%%\n\n%% propfind response\npropfind_response(Props,A,R) ->\n    Url = yaws_api:url_encode(R#resource.name),\n    case Props of\n        [allprop] ->\n            AllProp = [ prop_get(N,A,R) || N <- allprops(R) ],\n            AllSorted = prop_sort(AllProp),\n            {200, Results} = lists:keyfind(200,1,AllSorted),\n            [{'D:href', [], [Url]},\n             {'D:propstat', [], [\n                {'D:prop', [], Results},{status, [],[\"HTTP/1.1 200 OK\"]}\n            ]}];\n        [propname] ->\n            Results = [ case NS of\n                        'DAV:' -> {list_to_atom(\"D:\"++atom_to_list(P)),[],[]};\n                        _ -> {P,[{'xmlns',NS}],[]}\n                        end\n                      || {NS,P} <-allprops(R) ],\n            [{'D:href', [], [Url]},\n             {'D:propstat', [], [\n                {'D:prop', [], Results},{status, [],[\"HTTP/1.1 200 OK\"]}\n            ]}];\n        PropsRequested ->\n            Results = [ prop_get(N,A,R) || {N,_} <- PropsRequested ],\n            ResultsSorted = prop_sort(Results),\n            [{'D:href', [], [Url]}|\n             [{'D:propstat', [], [\n                {'D:prop', [], PropsFound},prop_status(Status)\n             ]} || {Status,PropsFound} <- ResultsSorted ]\n            ]\n    end.\n\n%% proppatch response/\nproppatch_response(Update,A,R) ->\n    Url = yaws_api:url_encode(R#resource.name),\n    Results = proppatch_response(Update,A,R,[]),\n    ResultsSorted = prop_sort(lists:flatten(Results)),\n    [{'D:href', [], [Url]}|\n     [{'D:propstat', [], [\n        {'D:prop', [], PropsFound},prop_status(Status)\n     ]} || {Status,PropsFound} <- ResultsSorted ]\n    ].\nproppatch_response([H|T],A,R,Results) ->\n    Result = case H of\n                 {set,Props} -> [ prop_set(P,A,R,V) || {P,V} <- Props];\n                 {remove,Props} -> [ prop_remove(P,A,R) || {P,_V} <- Props]\n             end,\n    proppatch_response(T,A,R,[Result|Results]);\nproppatch_response([],_A,_R,Results) ->\n    Results.\n\nprop_sort(L) -> prop_sort(L,[]).\nprop_sort([H|T],R) ->\n    {Status,Prop} = H,\n    R1 = case lists:keyfind(Status,1,R) of\n        {Status, Props} -> lists:keystore(Status,1,R,{Status,[Prop|Props]});\n        false -> lists:keystore(Status,1,R,{Status,[Prop]})\n    end,\n    prop_sort(T,R1);\nprop_sort([],R) -> R.\n\n\nprop_status(Status) ->\n    {'D:status',[],[\"HTTP/1.1 \" ++ integer_to_list(Status) ++ \" \" ++\n                        yaws_api:code_to_phrase(Status)]}.\n\n\n%% Available props include namespace\n%% Available props can differ per resource\n%% For proposed Microsoft extensions see: draft-hopmann-collection-props-00.txt\n%%\nallprops(R) ->\n    F = R#resource.info,\n    C = get(compatibility),\n    %% default property set\n    P1 = [\n          {'http://yaws.hyber.org/',access},    % sample Yaws extension\n          {'DAV:',creationdate},\n          %%{'DAV:',getcontentlanguage},        % not supported in GET\n                                                % so omitted here as well\n          {'DAV:',getcontentlength},\n          {'DAV:',getcontenttype},\n          {'DAV:',getetag},\n          {'DAV:',getlastmodified},\n          {'DAV:',lockdiscovery}, % class 2 compliancy\n          %%{'DAV:','quota-avialable-bytes'}    % RFC4331\n                                         %{'DAV:','quota-used-bytes'} % RFC4331\n          {'DAV:',resourcetype},\n          {'DAV:',supportedlock} % class 2 compliancy\n         ],\n    %% properties depending on file type\n    P2 = case F#file_info.type of\n             directory when C==windows ->\n                 [\n                   {'DAV:',childcount} % Microsoft extension\n                 ];\n             %% The executable property is only shown for regular files\n             regular ->\n                 [\n                  {'http://apache.org/dav/props/',executable} % Apache extension\n                 ];\n             _ -> [\n                  ]\n         end,\n    %% compatibility properties\n    P3 = case C of\n             microsoft -> [\n                           %%{'DAV:',iscollection},\n                           {'DAV:',isfolder},\n                           {'DAV:',ishidden}\n                           %%{'DAV:',isreadonly},\n                           %%{'DAV:',isroot},\n                           %%{'DAV:',name},\n                          ];\n             _ -> [\n                   {'DAV:',displayname}\n                  ]\n         end,\n    P1++P2++P3.\n\nprop_get({'http://yaws.hyber.org/',access},_A,R) ->\n    F = R#resource.info,\n    A = F#file_info.access,\n    P = {access, [{xmlns,'http://yaws.hyber.org/'}], [atom_to_list(A)]},\n    {200, P};\nprop_get({'DAV:',childcount},A,_R) ->\n    Path=davpath(A),\n    L = case file:list_dir(Path) of\n            {ok, Files} -> length(Files);\n            _ -> 0\n        end,\n    P = {'D:childcount', [], [integer_to_list(L)]},\n    {200, P};\nprop_get({'DAV:',creationdate},_A,R) ->\n    F = R#resource.info,\n    D = F#file_info.ctime,\n    T = yaws:local_time_as_gmt_string(D),\n    P = {'D:creationdate', [], [lists:flatten(T)]},\n    {200, P};\nprop_get({'DAV:',displayname},_A,R) ->\n    case get(compatibility) of\n        microsoft ->\n            {404,{'D:displayname',[],[]}};\n        _ ->\n            Name = filename:basename(R#resource.name),\n            Xml = #xmlText{type=cdata,value=Name},\n            P = {'D:displayname', [], [Xml]},\n            {200, P}\n    end;\nprop_get({'http://apache.org/dav/props/',executable},_A,R) ->\n    F = R#resource.info,\n    case F#file_info.type of\n        directory -> {404,{executable,\n                           [{'xmlns',\"http://apache.org/dav/props/\"}], []}};\n        _ ->\n            %% TODO check on extension for Windows?\n            X = case F of\n                    #file_info{mode=Mode} when Mode band 8#111 =/= 0 -> \"T\";\n                    _ -> \"F\"\n                end,\n            {200, {executable, [{'xmlns',\"http://apache.org/dav/props/\"}], X}}\n    end;\nprop_get({'DAV:',getcontentlanguage},_A,_R) ->\n    P = {'D:getcontentlanguage', [], []},\n    {200, P};\nprop_get({'DAV:',getcontentlength},_A,R) ->\n    F = R#resource.info,\n    P = {'D:getcontentlength', [], [integer_to_list(F#file_info.size)]},\n    {200, P};\nprop_get({'DAV:',getcontenttype},_A,R) ->\n    F = R#resource.info,\n    Mediatype = case F#file_info.type of\n          directory ->\n              \"httpd/unix-directory\";\n              %%\"text/html\";\n              %% this should represent the mediatype of a GET on a collection\n          _ ->\n              Name = R#resource.name,\n              Ext = filename:extension(Name),\n              Ext1 = case Ext of\n                         [] -> \"\";\n                         _ -> tl(Ext)\n                     end,\n              {_Kind,Mimetype} = mime_types:t(Ext1),\n              Mimetype\n        end,\n    P = {'D:getcontenttype', [], [Mediatype]},\n    {200, P};\nprop_get({'DAV:',getetag},_A,R) ->\n    F = R#resource.info,\n    E = yaws:make_etag(F),\n    P = {'D:getetag', [], [E]},\n    {200, P};\nprop_get({'DAV:',getlastmodified},_A,R) ->\n    F = R#resource.info,\n    D = F#file_info.mtime,\n    T = yaws:local_time_as_gmt_string(D),\n    X = lists:flatten(T),\n    C = get(compatibility),\n    P = case C of\n            microsoft ->\n                {'getlastmodified',\n                 [{'xmlns:b',\n                   \"urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/\"},\n                  {'b:dt',\"dateTime.rfc1123\"}],[X]};\n            _ -> {'D:getlastmodified',[],[X]}\n        end,\n    {200, P};\nprop_get({'DAV:',isfolder},_A,R) ->\n    F = R#resource.info,\n    D = case F#file_info.type of\n            directory -> \"1\";\n            _ -> \"0\"\n        end,\n    P = {'D:isfolder', [], [D]},\n    {200, P};\nprop_get({'DAV:',ishidden},_A,R) ->\n    N = filename:basename(R#resource.name),\n    H = case N of\n            \".\"++_Rest -> \"1\"; % dotted file\n            _ -> \"0\"\n        end,\n    P = {'D:ishidden', [], [H]},\n    {200, P};\nprop_get({'DAV:',resourcetype},_A,R) ->\n    F = R#resource.info,\n    P = case F#file_info.type of\n            directory -> {'D:resourcetype', [], [{'D:collection',[],[]}]};\n            _ -> {'D:resourcetype', [], []}\n        end,\n    {200, P};\nprop_get({'DAV:',lockdiscovery},A,_R) ->\n    Path = davpath(A),\n    Locks = yaws_runmod_lock:discover(Path),\n    case Locks of\n        [] ->\n            {404,{'D:lockdiscovery',[],[]}};\n        _ ->\n            ActiveLocks =\n                [{'D:activelock',[],\n                  [{'D:lockscope',[],[prop_get_format(scope,Lock#lock.scope)]},\n                   {'D:locktype',[],[prop_get_format(type,Lock#lock.type)]},\n                   {'D:depth',[],[prop_get_format(depth,Lock#lock.depth)]},\n                   {'D:owner',[],[prop_get_format(owner,Lock#lock.owner)]},\n                   {'D:timeout',[],[prop_get_format(timeout,Lock#lock.timeout)]},\n                   {'D:locktoken',[],[prop_get_format(locktoken,Lock#lock.id)]},\n                   {'D:lockroot',[],[prop_get_format(lockroot,Lock#lock.path)]}\n                  ]}\n                 || Lock <- Locks ],\n            {200, {'D:lockdiscovery',[],ActiveLocks}}\n    end;\nprop_get({'DAV:',supportedlock},_A,_R) ->\n    P = {'D:supportedlock',[],[\n            {'D:lockentry',[],[\n                {'D:lockscope',[],[{'D:exclusive',[],[]}]},\n                {'D:locktype',[],[{'D:write',[],[]}]}\n            ]},\n            {'D:lockentry',[],[\n                {'D:lockscope',[],[{'D:shared',[],[]}]},\n                {'D:locktype',[],[{'D:write',[],[]}]}\n            ]}\n        ]},\n    {200, P};\nprop_get({'',_P},_A,_R) ->\n    throw(400);\nprop_get({NS,P},_A,_R) ->\n    {404,{P,[{'xmlns',NS}],[]}}.\n\n\nprop_set({'DAV:',creationdate},A,_R,V) ->\n    Path=davpath(A),\n    P = {'D:creationdate', [], []},\n    case file:read_file_info(Path) of\n        {ok,F0} ->\n            T = yaws:stringdate_to_datetime(V),\n            F1 = F0#file_info{ctime=T},\n            case file:write_file_info(Path,F1) of\n                ok ->\n                    {200, P};\n                {error,_} ->\n                    {409, P}\n            end;\n        {error,_} ->\n            {409, P}\n    end;\nprop_set({'DAV:',getlastmodified},A,_R,V) ->\n    Path=davpath(A),\n    P = {'D:getlastmodified', [], []},\n    case file:read_file_info(Path) of\n        {ok,F0} ->\n            T = yaws:stringdate_to_datetime(V),\n            F1 = F0#file_info{mtime=T},\n            case file:write_file_info(Path,F1) of\n                ok ->\n                    {200, P};\n                {error,_} ->\n                    {409, P}\n            end;\n        {error,_} ->\n            {409, P}\n    end;\n%%prop_set({'http://apache.org/dav/props/',executable},_A,R,_V) ->\n%%   {501,{P,[{'xmlns',NS}],[]}}; % Not yet implemented\nprop_set({'DAV:',getetag},_A,_R,_V) ->\n    {403,{'D:getetag',[],[{'cannot-modify-protected-property',[],[]}]}};\nprop_set({'DAV:',lockdiscovery},_A,_R,_V) ->\n    {403,{'D:lockdiscovery',[],[{'cannot-modify-protected-property',[],[]}]}};\nprop_set({'DAV:',resourcetype},_A,_R,_V) ->\n    {403,{'D:resourcetype',[],[{'cannot-modify-protected-property',[],[]}]}};\nprop_set({NS,P},_A,_R,_V) ->\n    {404,{P,[{'xmlns',NS}],[]}}.\n\n\nprop_remove({P,NS},_A,_R) ->\n    {403,{P,[{'xmlns',NS}],[]}}.\n\n\nprop_get_format(type,write) ->\n    {'D:write',[],[]};\nprop_get_format(scope,exclusive) ->\n    {'D:exclusive',[],[]};\nprop_get_format(scope,_) ->\n    {'D:shared',[],[]};\nprop_get_format(depth,infinity) ->\n    \"infinity\";\nprop_get_format(depth,Depth) ->\n    integer_to_list(Depth);\nprop_get_format(timeout,Timeout) ->\n    lists:flatten(io_lib:format(\"Second-~p\",[Timeout]));\nprop_get_format(locktoken,Id) ->\n    {'D:href',[],[\"opaquelocktoken:\"++Id]};\nprop_get_format(lockroot,Ref) ->\n    {'D:href',[],[Ref]};\nprop_get_format(owner,Owner) ->\n    Owner;\nprop_get_format(_Something,_) ->\n    ?DEBUG(\" did not expect ~p here ~n\",[_Something]),\n    throw(500).\n\n\n%% --------------------------------------------------------\n%% Resource mapping\n%%\n\ndavname(A) ->\n    A#arg.server_path.\n\ndavpath(A) ->\n    filename:join(A#arg.docroot,[\"./\",A#arg.server_path]).\n\n%% davresource0/1 - get resources with depth 0\ndavresource0(A) ->\n    Name = davname(A),\n    Path = davpath(A),\n    case file:read_file_info(Path) of\n        {ok, F} when (F#file_info.type == directory) or\n                     (F#file_info.type == regular) ->\n            #resource{ name = Name, info = F};\n        {error,_} -> throw(404)\n    end.\n%% davresource1/1 - get additional resources for depth 1\ndavresource1(A) ->\n    Coll = davname(A),\n    Path = davpath(A),\n    case file:read_file_info(Path) of\n        {ok, Dir} when Dir#file_info.type == directory ->\n            {ok, L} = file:list_dir(Path),\n            davresource1(A,Path,Coll,L,[]);\n        {ok, _Else} ->\n            []\n    end.\ndavresource1(_A,_Path,_Coll,[],Result) ->\n    Result;\ndavresource1(_A,Path,Coll,[Name|Rest],Result) ->\n    File = filename:join(Path,Name),\n    Ref = filename:join(Coll,Name),\n    {ok, Info} = file:read_file_info(File),\n    if\n        (Info#file_info.type == regular) or\n        (Info#file_info.type == directory) ->\n            Resource = #resource {name = Ref, info = Info},\n            davresource1(_A,Path,Coll,Rest,[Resource|Result]);\n        true ->\n            davresource1(_A,Path,Coll,Rest,Result)\n    end.\n\n\n%% --------------------------------------------------------\n%% Parse additional HTTP headers\n%%\n\nh_litmus(A) ->\n    Hs = (A#arg.headers)#headers.other,\n    case lists:keysearch(\"X-Litmus\", 3, Hs) of\n        {value, {_,_,\"X-Litmus\",_,_Test}} ->\n            ?DEBUG(\"~s - \",[_Test]);\n        _ ->\n            ok\n    end.\n\nh_depth(A) ->\n    Hs = (A#arg.headers)#headers.other,\n    case lists:keysearch(\"Depth\", 3, Hs) of\n        {value, {_,_,\"Depth\",_,Depth}} ->\n            h_depth_interpret(Depth);\n        _ ->\n            infinity\n    end.\nh_depth_interpret(\"infinity\") -> infinity;\nh_depth_interpret(\"1\") -> 1;\nh_depth_interpret(_) -> 0.\n\nh_destination(A) ->\n    Hs = (A#arg.headers)#headers.other,\n    case lists:keysearch(\"Destination\", 3, Hs) of\n        {value, {http_header,_,_,_,Dest}} ->\n            Url = yaws_api:parse_url(Dest),\n            {Path,_} = yaws_api:url_decode_q_split(Url#url.path),\n            ?DEBUG(\" TO ~p\",[Path]),\n            filename:join(A#arg.docroot,[\"./\",Path]);\n        _ ->\n            throw(501)\n    end.\n\nh_overwrite(A) ->\n    Hs = (A#arg.headers)#headers.other,\n    case lists:keysearch(\"Overwrite\", 3, Hs) of\n        {value, {http_header, _ , _, _, \"T\"}} ->\n            true;\n        _ ->\n            false\n    end.\n\nh_timeout(A) ->\n    Hs = (A#arg.headers)#headers.other,\n    case lists:keysearch(\"Timeout\", 3, Hs) of\n        {value, {_,_,\"Timeout\",_,T}} ->\n            case T of\n                \"Second-\"++TimeoutVal ->\n                    Val = case catch list_to_integer(TimeoutVal) of\n                              I when is_integer(I) -> I;\n                              _ -> ?LOCK_LIFETIME\n                          end,\n                    erlang:min(Val,?LOCK_LIFETIME);\n                _ -> ?LOCK_LIFETIME\n            end;\n        _ ->\n            ?LOCK_LIFETIME\n    end.\n\nh_locktoken(A) ->\n    Hs = (A#arg.headers)#headers.other,\n    C = get(compatibility),\n    case lists:keysearch(\"Lock-Token\", 3, Hs) of\n        {value, {_,_,\"Lock-Token\",_,URL}} ->\n            case URL of\n                %%\"<opaquelocktoken:\"++Token -> string:left(Token,36);\n                \"<opaquelocktoken:\"++Token ->\n                    T = parse_locktoken(Token),\n                    check_locktoken_format(T),\n                    T;\n                \"opaquelocktoken:\"++Token when C==microsoft -> Token;\n                _ -> URL\n            end;\n        _ ->\n            undefined\n    end.\nparse_locktoken([]) ->\n    [];\nparse_locktoken([H|_T]) when H==62 ->\n    [];\nparse_locktoken([H|T]) ->\n    [H|parse_locktoken(T)].\n\ncheck_locktoken_format(\"DAV:no-lock\") ->\n    ok;\ncheck_locktoken_format(Token) when length(Token)==36 ->\n    ok;\ncheck_locktoken_format(_Token) ->\n    throw(423).\n\n%%h_if_match(A,Path) ->\n%%    Hs = (A#arg.headers)#headers.other,\n%%    case lists:keysearch(\"If-Match\", 3, Hs) of\n%%        {value, {_,_,\"If-Match\",_,Tag}} ->\n%%            F = file:read_file_info(Path),\n%%            case yaws:make_etag(F) of\n%%                Tag -> ok;\n%%                _ -> throw(412)\n%%            end;\n%%        _ ->\n%%            ok\n%%    end.\n\nh_if_refresh(A,Path) ->\n    ?DEBUG(\" If(~p)\",[Path]),\n    _Locks = yaws_runmod_lock:discover(Path),\n    Hs = (A#arg.headers)#headers.other,\n    case lists:keysearch(\"If\", 3, Hs) of\n        {value, {_,_,\"If\",_,If}} ->\n            List = if_parse(If,untagged),\n            %%?DEBUG(\" ~p\",[List]),\n            case List of\n                [{_Resource,[{true,state,Locktoken}]}] -> Locktoken;\n                _ -> throw(412)\n            end;\n        _ ->\n            undefined\n    end.\n\nh_if(A,Path) ->\n    ?DEBUG(\" If(~p)\",[Path]),\n    Locks = yaws_runmod_lock:discover(Path),\n    Hs = (A#arg.headers)#headers.other,\n    {_L,I} = case lists:keysearch(\"If\", 3, Hs) of\n            {value, {_,_,\"If\",_,If}} ->\n                List = if_parse(If,untagged),\n                Value = if_eval(A,Locks,List),\n                {List,Value};\n            _ ->\n                {[],undefined}\n        end,\n    ?DEBUG(\" -> ~p (~p)\",[I,length(Locks)]),\n    ?DEBUG(\" If-header ~p (~p) evaluated to ~p~n\",[_L,length(Locks),I]),\n    case I of\n        undefined when length(Locks)>0 -> throw(423);\n        %%false when length(Locks)>0 -> throw(412);\n        false -> throw(412);\n        _ -> ok\n    end.\n\nif_parse([],_Resource) ->\n    [];\nif_parse(Line,Resource) when hd(Line)==32 ->\n    if_parse(tl(Line),Resource);\nif_parse(Line,untagged) when hd(Line)==60 -> % <\n    {Url,Rest} = if_parse_token(tl(Line),\"\"),\n    if_parse(Rest,Url);\nif_parse(Line,Resource) when hd(Line)==40 -> % (\n    {Condition,Rest} = if_parse_condition(tl(Line),[],true),\n    [{Resource,Condition}|if_parse(Rest,untagged)].\n\nif_parse_condition(Line,List,_Bool) when hd(Line)==41 -> % )\n    {List,tl(Line)};\nif_parse_condition(Line,List,Bool) when hd(Line)==32 -> % whitespace\n    if_parse_condition(tl(Line),List,Bool);\nif_parse_condition(\"Not\"++Line,List,_Bool) -> % negate\n    if_parse_condition(tl(Line),List,false);\nif_parse_condition(Line,List,Bool) when hd(Line)==60 -> % <\n    {Token,Rest} = if_parse_token(tl(Line),\"\"),\n    if_parse_condition(Rest,[{Bool,state,Token}|List],true);\nif_parse_condition(Line,List,Bool) when hd(Line)==91 -> % [\n    {Etag,Rest} = if_parse_etag(tl(Line),\"\"),\n    if_parse_condition(Rest,[{Bool,etag,Etag}|List],true).\n\nif_parse_token(Line,Buffer) when hd(Line)==62 -> % >\n    Uri = lists:reverse(Buffer),\n    Token1 = case Uri of\n                 \"opaquelocktoken:\"++Token ->\n                    check_locktoken_format(Token),\n                    Token;\n                 _ -> Uri\n             end,\n    {Token1,tl(Line)};\nif_parse_token([H|T],Buffer) ->\n    if_parse_token(T,[H|Buffer]).\n\nif_parse_etag(Line,Buffer) when hd(Line)==93 -> % ]\n    {lists:reverse(Buffer),tl(Line)};\nif_parse_etag([H|T],Buffer) ->\n    if_parse_etag(T,[H|Buffer]).\n\n%% if_eval(A,RequestPath,Conditions)\nif_eval(_A,_Locks,[]) ->\n    false;\nif_eval(A,Locks,[{Resource,AndList}|More]) ->\n    Target = case Resource of\n                 untagged -> davpath(A);\n                 _ ->\n                    Url = yaws_api:parse_url(Resource),\n                    filename:join(A#arg.docroot,[\"./\",Url#url.path])\n             end,\n    if_eval_condition(AndList,A,Target,Locks) orelse if_eval(A,Locks,More).\n\nif_eval_condition(AndList,A,Target,Locks) ->\n    if_eval_condition(AndList,true,false,A,Target,Locks).\n\nif_eval_condition([],Result,Valid,_A,_Target,_Locks) ->\n    Result and Valid;\nif_eval_condition([{false,Kind,Ref}|T],Result,Valid,A,Target,Locks) ->\n    not if_eval_condition([{true,Kind,Ref}|T],Result,Valid,A,Target,Locks);\nif_eval_condition([{true,state,Ref}|T],_Result,_Valid,A,Target,Locks) ->\n    Result1 = if_eval_locktoken(Target,Ref,Locks),\n    Valid1 = true,\n    Result1 andalso if_eval_condition(T,Result1,Valid1,A,Target,Locks);\nif_eval_condition([{true,etag,Ref}|T],_Result,Valid,A,Target,Locks) ->\n    {ok, F} = file:read_file_info(filename:join(A#arg.docroot,Target)),\n    E = yaws:make_etag(F),\n    Result1 = (E==Ref),\n    Valid1 = Valid,\n    Result1 andalso if_eval_condition(T,Result1,Valid1,A,Target,Locks).\n\n%% if_eval_locktoken(Target,Token,Locktokens) -> true|false\nif_eval_locktoken(_Target,_Token,[]) ->\n    false;\n%%if_eval_locktoken(_Target,\"DAV:no-lock\",[]) ->\n%%    true;\nif_eval_locktoken(Target,Token,[H|T]) ->\n    ((H#lock.path == Target) and (H#lock.id == Token))\n        orelse if_eval_locktoken(Target,Token,T).\n\n\n%% --------------------------------------------------------\n%% Parsing of XML elements (RFC4918)\n%%\n%% activelock\n-define(IS_ALLPROP(X), #xmlElement{expanded_name = {'DAV:',allprop}} = X).\n%% collection\n%% depth\n%% error\n-define(IS_EXCLUSIVE(X), #xmlElement{expanded_name = {'DAV:',exclusive}} = X).\n-define(IS_HREF(X), #xmlElement{expanded_name = {'DAV:',href}} = X).\n%% include % TODO: add this tag\n%% location\n%% lockentry\n-define(IS_LOCKINFO(X), #xmlElement{expanded_name = {'DAV:',lockinfo}} = X).\n%% lockroot\n-define(IS_LOCKSCOPE(X), #xmlElement{expanded_name = {'DAV:',lockscope}} = X).\n%% locktoken\n-define(IS_LOCKTYPE(X), #xmlElement{expanded_name = {'DAV:',locktype}} = X).\n%% multistatus\n-define(IS_OWNER(X), #xmlElement{expanded_name = {'DAV:',owner}} = X).\n-define(IS_PROP(X), #xmlElement{expanded_name = {'DAV:',prop}} = X).\n-define(IS_PROPERTYUPDATE(X),\n        #xmlElement{expanded_name = {'DAV:',propertyupdate}} = X).\n-define(IS_PROPFIND(X), #xmlElement{expanded_name = {'DAV:',propfind}} = X).\n-define(IS_PROPNAME(X), #xmlElement{expanded_name = {'DAV:',propname}} = X).\n%% propstat\n-define(IS_REMOVE(X), #xmlElement{expanded_name = {'DAV:',remove}} = X).\n%% response\n%% responsedescription\n-define(IS_SET(X), #xmlElement{expanded_name = {'DAV:',set}} = X).\n-define(IS_SHARED(X), #xmlElement{expanded_name = {'DAV:',shared}} = X).\n%% status\n%% timeout\n-define(IS_WRITE(X), #xmlElement{expanded_name = {'DAV:',write}} = X).\n\n-define(CONTENT(X), X#xmlElement.content).\n\n%% Parameter is always list\nparse_propfind([]) -> [allprop]; % RFC4918: no body then allprop, is [] no body?\nparse_propfind(L) ->\n    case catch xmerl_scan:string(L, [{namespace_conformant, true}]) of\n        {?IS_PROPFIND(X),_} ->\n            parse_propfind(?CONTENT(X),[]);\n        _Z ->\n            throw(400)\n    end.\nparse_propfind([?IS_PROPNAME(_H)|_T], _R) ->\n    [propname];\nparse_propfind([?IS_ALLPROP(_H)|_T], _R) ->\n    [allprop];\nparse_propfind([?IS_PROP(H)|_T], _R) when length(?CONTENT(H))==0 ->\n    [allprop]; % NetDrive uses empty <prop> element instead of <allprop>\nparse_propfind([?IS_PROP(H)|T], _R) ->\n    Props = parse_prop(?CONTENT(H)),\n    parse_propfind(T, Props);\nparse_propfind([_H|T], R) ->\n    parse_propfind(T, R);\nparse_propfind([], R) ->\n    R.\n\nparse_proppatch(L) ->\n    case catch xmerl_scan:string(L, [{namespace_conformant, true}]) of\n        {?IS_PROPERTYUPDATE(X),_} ->\n            parse_proppatch(?CONTENT(X),[]);\n        _Z ->\n            throw(400)\n    end.\nparse_proppatch([?IS_SET(H)|T],R) ->\n    Props = parse_setremove(?CONTENT(H)),\n    parse_proppatch(T,[{set,Props}|R]);\nparse_proppatch([?IS_REMOVE(H)|T],R) ->\n    Props = parse_setremove(?CONTENT(H)),\n    parse_proppatch(T,[{remove,Props}|R]);\nparse_proppatch([_H|T], R) ->\n    parse_proppatch(T, R);\nparse_proppatch([],R) ->\n    lists:reverse(R). % MUST proces in document order\n\nparse_setremove([?IS_PROP(X)]) ->\n    parse_prop(?CONTENT(X)).\n\nparse_prop(L) ->\n    parse_prop(L, []).\n\nparse_prop([H|T],L) ->\n    case H of\n        H when is_record(H,xmlElement) ->\n            %% check on supported namespaces:\n            %% - http://www.w3.org/TR/RC-xml-names#dt-prefix\n            %% - although strict, not very forgiving towards clients\n            %%NS = H#xmlElement.namespace,\n            %%case NS#xmlNamespace.default of\n            %%    \"\" ->\n            %%        throw(400);\n            %%    _ -> ok\n            %%end,\n            Value = case H#xmlElement.content of\n                        [C] when is_record(C,xmlText) -> C#xmlText.value;\n                        _ -> \"\"\n                    end,\n            parse_prop(T,[{H#xmlElement.expanded_name,Value}|L]);\n        _ ->\n            parse_prop(T,L)\n    end;\nparse_prop([], L) ->\n    L.\n\nparse_lockinfo([]) ->\n    #lock{};\nparse_lockinfo(L) ->\n    case catch xmerl_scan:string(L, [{namespace_conformant, true}]) of\n        {?IS_LOCKINFO(X),_} ->\n            parse_lockinfo(?CONTENT(X),#lock{});\n        _Z ->\n            throw(400)\n    end.\nparse_lockinfo([?IS_LOCKSCOPE(H)|T], D) ->\n    X = parse_lockscope(?CONTENT(H)),\n    parse_lockinfo(T,D#lock{scope=X});\nparse_lockinfo([?IS_LOCKTYPE(H)|T], D) ->\n    X = parse_locktype(?CONTENT(H)),\n    parse_lockinfo(T,D#lock{type=X});\nparse_lockinfo([?IS_OWNER(H)|T], D) ->\n    X = parse_owner(?CONTENT(H)),\n    parse_lockinfo(T,D#lock{owner=X});\nparse_lockinfo([_H|T],D) ->\n    parse_lockinfo(T,D); % skip spaces and comments, etc.\nparse_lockinfo([], D) ->\n    D.\n\nparse_lockscope([?IS_EXCLUSIVE(_H)|_T]) ->\n    exclusive;\nparse_lockscope([?IS_SHARED(_H)|_T]) ->\n    shared;\nparse_lockscope(_X) ->\n    throw(400).\n\nparse_locktype([?IS_WRITE(_H)|_T]) ->\n    write;\nparse_locktype(_) ->\n    throw(400).\n\nparse_owner(X) ->\n    Xml = xmerl:export_simple_content(X,xmerl_xml),\n    lists:flatten(Xml).\n\n%% --------------------------------------------------------\n%% Status output\n%%\n\nstatus(Status) ->\n    status(Status,[],[]).\nstatus(Status,Response) ->\n    status(Status,[],Response).\nstatus(Status,Headers,{xml,Response}) ->\n    Xml = xml_expand(Response),\n    status(Status,Headers,{content, \"application/xml; charset=\\\"utf-8\\\"\", Xml});\nstatus(Status,Headers,Response) ->\n    ?DEBUG(\" -> ~p~n\",[Status]),\n    H = case get(compatibility) of\n            microsoft -> [{header,{\"MS-Author-Via\",\"DAV\"}}|Headers];\n            _ -> Headers\n        end,\n    [{status, Status},{header,{\"DAV\",\"1, 2, 3\"}}|H] ++ [Response].\n\nxml_expand(L) ->\n    xml_expand(L, \"utf-8\").\nxml_expand(L, Cset) ->\n    Prolog = [\"<?xml version=\\\"1.0\\\" encoding=\\\"\",Cset,\"\\\" ?>\"],\n    %%Xml = xmerl:export_simple(L,xmerl_xml,[{prolog,Prolog}]),\n    %% MS requires \\r\\n at end of every XML response\n    case get(compatibility) of\n        microsoft ->\n            [Prolog,yaws_appmod_dav:export(L),\"\\r\\n\"];\n        _ ->\n            [Prolog,yaws_appmod_dav:export(L)]\n    end.\n\n%% --------------------------------------------------------\n%% XML output (xmlerl_xml does not support CDATA)\n%%\n\nexport([]) ->\n    [];\nexport([#xmlComment{}|T]) -> % for now I skip comments\n    export(T);\nexport([#xmlText{type=text, value=Text}|T]) ->\n    [export_text(Text),export(T)];\nexport([#xmlText{type=cdata, value=Text}|T]) ->\n    [\"<![CDATA[\",Text,\"]]>\",export(T)];\nexport([#xmlElement{name=Name,attributes=Attrs,content=Content}|T]) ->\n    export([{Name,Attrs,Content}|T]);\nexport([{Name,Attrs,Content}|T]) when is_atom(Name)->\n    Tag = atom_to_list(Name),\n    export([{Tag,Attrs,Content}|T]);\nexport([{Tag,Attrs,[]}|T]) when is_list(Tag)->\n    [\"<\",Tag,export_attrs(Attrs),\" />\",export(T)];\nexport([{Tag,Attrs,Content}|T]) when is_list(Tag)->\n    [\"<\",Tag,export_attrs(Attrs),\">\",\n     export_content(Content),\"</\",Tag,\">\",export(T)].\n\nexport_content([]) ->\n    \"\";\nexport_content([H|T]) when is_tuple(H) -> % tuples are XML records\n    export([H|T]);\nexport_content([H]) when is_number(H) ->\n    integer_to_list(H);\nexport_content([H]) when is_atom(H) ->\n    atom_to_list(H);\nexport_content(L) ->\n    L.\n\nexport_attrs([]) ->\n    [];\nexport_attrs([{Name,Value}|T]) ->\n    [\" \",export_id(Name),\"=\\\"\",export_id(Value),\"\\\"\",export_attrs(T)];\nexport_attrs([Attr|T]) ->\n    [\" \\\"\",export_id(Attr),\"\\\"\",export_attrs(T)].\n\nexport_id(Id) when is_atom(Id) ->\n    atom_to_list(Id);\nexport_id(Id) when is_number(Id) ->\n    integer_to_list(Id);\nexport_id(Id) when is_list(Id) ->\n    Id.\n\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_appmod_fcgi.erl",
    "content": "%%% File        : yaws_appmod_fcgi.erl\n%%% Author      : Bruno Rijsman <brunorijsman@hotmail.com>\n%%% Description : Application module for FastCGI virtual paths.\n%%% Created     : 9 Jul 2009\n\n-module(yaws_appmod_fcgi).\n-export([out/1]).\n-include(\"../include/yaws_api.hrl\").\n\nout(Arg) ->\n    yaws_cgi:call_fcgi_responder(Arg).\n\n\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_cgi.erl",
    "content": "-module(yaws_cgi).\n-author('carsten@codimi.de').\n-author('brunorijsman@hotmail.com').         %% Added support for FastCGI\n\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_debug.hrl\").\n-include(\"../include/yaws.hrl\").\n\n%%% Returns Out (i.e. same return values as out/1).\n%%%\n-export([call_cgi/5, call_cgi/4, call_cgi/3, call_cgi/2]).\n-export([call_fcgi_responder/2, call_fcgi_responder/1]).\n\n%%% Returns {allowed, Out} or {denied, Out}.\n%%%\n-export([call_fcgi_authorizer/2, call_fcgi_authorizer/1]).\n\n%%% Returns [{VariableName, VariableValue}].\n%%%\n-export([fcgi_extract_variables/1]).\n\n%%% TODO: Implement FastCGI filter role.\n\n-export([cgi_worker/7, fcgi_worker/6]).\n\n%%%=====================================================================\n%%% Code shared between CGI and FastCGI\n%%%=====================================================================\n\n-define(ASCII_NEW_LINE, 10).\n-define(ASCII_CARRIAGE_RETURN, 13).\n\n\nhandle_clidata(Arg, WorkerPid) ->\n    case Arg#arg.clidata of\n        undefined ->\n            end_of_clidata(Arg, WorkerPid);\n        {partial, Data} ->\n            send_clidata(WorkerPid, Data),\n            {get_more, cgicont, {cgistate, WorkerPid}};\n        Data when is_binary(Data) ->\n            send_clidata(WorkerPid, Data),\n            end_of_clidata(Arg, WorkerPid)\n    end.\n\n\nend_of_clidata(Arg, WorkerPid) ->\n    WorkerPid ! {self(), end_of_clidata},\n    get_from_worker(Arg, WorkerPid).\n\n\nsend_clidata(WorkerPid, Data) ->\n    WorkerPid ! {self(), clidata, Data},\n    receive\n        {WorkerPid, clidata_receipt} -> ok\n    end.\n\n\nget_from_worker(Arg, WorkerPid) ->\n    case get_resp(WorkerPid) of\n        {failure, Reason} ->\n            [{status, 500}, {html, io_lib:format(\"CGI failure: ~p\", [Reason])}];\n        {Headers, Data} ->\n            AllResps = lists:map(fun(X)-> do_header(Arg, X, Data) end, Headers),\n            %%\n            %% The CGI 1.1 spec (RFC 3875) requires a worker response\n            %% consisting of only a location header and optional extension\n            %% headers to be augmented with a 302 status code. Any other\n            %% worker response with a location header is handled normally.\n            %% Technically a response of the latter type MUST have a status\n            %% code in it, but we don't enforce that.\n            %%\n            {LocHdr, _ExtHdrs, TheRest} =\n                lists:foldl(\n                  fun({header, Line}=Hdr, {Loc, Ext, Rest}) ->\n                          {HdrLower, _HdrVal} = do_lower_header(Line),\n                          case HdrLower of\n                              \"location\" ->\n                                  {[Hdr], Ext, Rest};\n                              \"x-cgi-\"++_ ->\n                                  {Loc, [Hdr|Ext], Rest};\n                              _ ->\n                                  {Loc, Ext, [Hdr|Rest]}\n                          end;\n                     (Hdr, {Loc, Ext, Rest}) ->\n                          {Loc, Ext, [Hdr|Rest]}\n                  end, {[], [], []}, AllResps),\n            Next = case LocHdr of\n                       [] ->\n                           normal;\n                       [{header, _Location}] ->\n                           case TheRest of\n                               [] ->\n                                   location_add_302;\n                               _ ->\n                                   normal\n                           end\n                   end,\n            case Next of\n                normal ->\n                    {ContentResps, NotCtnt} =\n                        filter2(fun iscontent/1, AllResps),\n                    {RedirResps, Others} = filter2(fun isredirect/1, NotCtnt),\n                    case RedirResps of\n                        [R|_] ->\n                            WorkerPid ! {self(), no_data},\n                            Others ++ [R];\n                        [] ->\n                            case ContentResps of\n                                [C={streamcontent, _, _}|_] ->\n                                    WorkerPid ! {self(), stream_data},\n                                    Others++[C];\n                                [C={content, _, _}|_] ->\n                                    WorkerPid ! {self(), no_data},\n                                    Others++[C];\n                                [] ->\n                                    WorkerPid ! {self(), no_data},\n                                    Others\n                            end\n                    end;\n                location_add_302 ->\n                    WorkerPid ! {self(), no_data},\n                    AllResps++[{status, 302}]\n            end\n    end.\n\n\nfilter2(Pred, Xs) ->\n    filter2(Pred, Xs, [], []).\n\nfilter2(_Pred, [], Ts, Fs) ->\n    {lists:reverse(Ts), lists:reverse(Fs)};\nfilter2(Pred, [X|Xs], Ts, Fs) ->\n    case Pred(X) of\n        true ->\n            filter2(Pred, Xs, [X|Ts], Fs);\n        false ->\n            filter2(Pred, Xs, Ts, [X|Fs])\n    end.\n\n\niscontent({content, _, _}) ->\n    true;\niscontent({streamcontent, _, _}) ->\n    true;\niscontent(_) ->\n    false.\n\nisredirect({status, I}) when is_integer(I) , I > 301, I < 304; I =:= 307 ->\n    true;\nisredirect(_) ->\n    false.\n\ncheckdef(undefined) ->\n    \"\";\ncheckdef(L) ->\n    L.\n\n\ndeep_drop_prefix([], L) ->\n    L;\ndeep_drop_prefix([X|Xs], [X|Ys]) when is_integer(X) ->\n    deep_drop_prefix(Xs, Ys);\ndeep_drop_prefix([X|Xs], Ys) when is_list(X) ->\n    deep_drop_prefix(X++Xs, Ys);\ndeep_drop_prefix(Xs, [Y|Ys]) when is_list(Y) ->\n    deep_drop_prefix(Xs, Y++Ys);\ndeep_drop_prefix(_, _) ->\n    false.\n\n\nget_socket_peername({ssl, SslSocket}) ->\n    {ok, {IP, _Port}}=ssl:peername(SslSocket),\n    inet_parse:ntoa(IP);\nget_socket_peername(Socket) ->\n    {ok, {IP, _Port}}=inet:peername(Socket),\n    inet_parse:ntoa(IP).\n\n\nget_socket_sockname({ssl, SslSocket}) ->\n    {ok, {IP, _Port}}=ssl:sockname(SslSocket),\n    inet_parse:ntoa(IP);\nget_socket_sockname(Socket) ->\n    {ok, {IP, _Port}}=inet:sockname(Socket),\n    inet_parse:ntoa(IP).\n\n\nbuild_env(Arg, Scriptfilename, Pathinfo, ExtraEnv, SC) ->\n    H       = Arg#arg.headers,\n    Req     = Arg#arg.req,\n    OrigReq = Arg#arg.orig_req,\n\n    %% Use the original request to set REQUEST_URI\n    case OrigReq#http_request.path of\n        {abs_path, RequestURI} -> ok;\n        _ -> RequestURI = undefined\n    end,\n    {Maj,Min} = Req#http_request.version,\n    {Hostname, Hosttail}=lists:splitwith(fun(X)->X /= $: end,\n                                         checkdef(H#headers.host)),\n    Hostport = case Hosttail of\n                   [$: | P] -> P;\n                   [] -> integer_to_list(SC#sconf.port)\n               end,\n    PeerAddr = get_socket_peername(Arg#arg.clisock),\n    LocalAddr = get_socket_sockname(Arg#arg.clisock),\n\n    Scheme = (catch yaws:redirect_scheme(SC)),\n    %% Needed by trac, for redirs after POST\n    HttpsEnv  = case Scheme of\n                    \"https://\" -> [{\"HTTPS\", \"1\"}];\n                    _ ->[]\n                end,\n\n\n    %%Scriptname = deep_drop_prefix(Arg#arg.docroot, Arg#arg.fullpath),\n    %%SCRIPT_NAME is the path of the script relative to the root of the website.\n    %%just dropping docroot from the fullpath does not give the full SCRIPT_NAME\n    %% path if a 'vdir' is involved.\n    UriTail = deep_drop_prefix(Arg#arg.docroot, Arg#arg.fullpath),\n    case Arg#arg.docroot_mount of\n        \"/\" ->\n            %%no arg.docroot_mount means that arg.docroot\n            %% corresponds to the URI-root of the request \"/\"\n            Scriptname = UriTail;\n        Vdir ->\n            Scriptname = Vdir ++ string:strip(UriTail,left,$/)\n    end,\n\n    Pathinfo2 = checkdef(Pathinfo),\n    case Pathinfo2 of\n        \"\" ->\n            PathTranslated = \"\";\n        _ ->\n            %%determine what physical path the server would map Pathinfo2\n            %%to if it had received just Pathinfo2 in the request.\n            PathTranslated = yaws_server:mappath(SC,Arg,Pathinfo2)\n    end,\n\n\n    %%Pass auth info in environment - yes - including password in plain text.\n    %%REMOTE_USER always = AUTH_USER\n    %%!todo - LOGON_USER - same as AUTH_USER unless some auth filter has mapped\n    %%the user to another username under which to run the request.\n    case H#headers.authorization of\n        undefined ->\n            AuthEnv = [];\n        {undefined, _, _} ->\n            AuthEnv = [];\n        {User, Password, \"Basic \" ++ Auth64} ->\n            AuthEnv = [\n                       {\"HTTP_AUTHORIZATION\", \"Basic \" ++ Auth64},\n                       {\"AUTH_TYPE\", \"Basic\"},\n                       {\"AUTH_USER\", User},\n                       {\"REMOTE_USER\", User},\n                       {\"LOGON_USER\", User},\n                       {\"AUTH_PASSWORD\", Password}\n                      ];\n        {_User, _Password, _OrigString} ->\n            %%not attempting to pass through any auth info for\n            %% auth schemes that we don't yet handle\n            AuthEnv = []\n    end,\n\n    Extra_CGI_Vars = lists:flatmap(fun({Dir, Vars}) ->\n                                           case lists:prefix(Dir, Scriptname) of\n                                               true -> Vars;\n                                               false -> []\n                                           end\n                                   end,\n                                   SC#sconf.extra_cgi_vars),\n\n    %% Some versions of erlang:open_port can't handle query strings that\n    %% end with an equal sign. This is because the broken versions treat\n    %% environment variable strings ending with '=' as environment variable\n    %% names intended to be deleted from the environment, i.e. as if they\n    %% have no value. The result is that no QUERY_STRING environment\n    %% variable gets set for these cases. We work around this by appending\n    %% a & character to any query string that ends in =.\n    QueryString = case checkdef(Arg#arg.querydata) of\n                      \"\" ->\n                          \"\";\n                      QS ->\n                          case lists:reverse(QS) of\n                              [$= | _] ->\n                                  QS ++ \"&\";\n                              _ ->\n                                  QS\n                          end\n                  end,\n\n    %%todo - review. should AuthEnv entries be overridable by ExtraEnv or not?\n    %% we should define policy here rather than let through dupes.\n\n    ExtraEnv ++\n        HttpsEnv ++\n        AuthEnv ++\n        lists:filter(\n          fun({K, L}) when is_list(L) ->\n                  case lists:keysearch(K, 1, ExtraEnv) of\n                      false ->\n                          true;\n                      _ ->\n                          %% we have override in extraenv\n                          false\n                  end;\n             (_) ->\n                  false\n          end,\n          ([\n            {\"SERVER_SOFTWARE\", \"Yaws/\"++yaws_generated:version()},\n            {\"SERVER_NAME\", Hostname},\n            {\"HTTP_HOST\", checkdef(H#headers.host)},\n            {\"GATEWAY_INTERFACE\", \"CGI/1.1\"},\n            {\"SERVER_PROTOCOL\", \"HTTP/\" ++ integer_to_list(Maj) ++\n             \".\" ++ integer_to_list(Min)},\n            {\"SERVER_PORT\", Hostport},\n            {\"REQUEST_METHOD\", yaws:to_list(Req#http_request.method)},\n            {\"REQUEST_URI\", RequestURI},\n            {\"DOCUMENT_ROOT\",         Arg#arg.docroot},\n            {\"DOCUMENT_ROOT_MOUNT\", Arg#arg.docroot_mount},\n            %% SCRIPT_FILENAME is for PHP 4.3.2 and higher\n            %% see http://bugs.php.net/bug.php?id=28227\n            %% (Sergei Golovan).\n            {\"SCRIPT_FILENAME\", Scriptfilename},\n            %% {\"SCRIPT_TRANSLATED\", Scriptfilename},   %IIS6+\n            {\"PATH_INFO\",                Pathinfo2},\n            {\"PATH_TRANSLATED\",        PathTranslated},\n            %% <JMN_2007-02>\n            %%  CGI/1.1 spec says PATH_TRANSLATED should be NULL or unset\n            %% if PATH_INFO is NULL\n            %%  This is in contrast to IIS behaviour - and may break some apps.\n            %%  broken apps that expect it to always correspond to path of\n            %% script\n            %%  should be modified to use SCRIPT_FILENAME instead - or wrapped.\n            %% </JMN_2007-02>\n            %% --------------------\n            %%  <pre_2007-02_comments>\n            %%  This seems not to\n            %%  correspond to the\n            %%  documentation I have\n            %%  read, but it works\n            %%  with PHP.\n            %%\n            %%  (Not with PHP 4.3.10-16) from\n            %%  Debian sarge (Sergei Golovan).\n            %%  </pre_2007-02_comments>\n            %% ---------------------\n            {\"SCRIPT_NAME\", Scriptname},\n            {\"REMOTE_ADDR\", PeerAddr},\n            {\"REMOTE_HOST\", PeerAddr},  %%  We SHOULD send this\n            %%  Resolving DNS not practical for performance reasons\n            %%  - at least on 1st contact from a particular host.\n            %%  we could do background lookup so that it's available\n            %% for subsequent invocations,\n            %%  but it hardly seems worthwhile. We are permitted by the\n            %% CGI/1.1 spec to substitute REMOTE_ADDR\n            {\"SERVER_ADDR\", LocalAddr},   %% Apache compat\n            {\"LOCAL_ADDR\", LocalAddr},    %% IIS compat\n            {\"QUERY_STRING\", QueryString},\n            {\"CONTENT_TYPE\", H#headers.content_type},\n            {\"CONTENT_LENGTH\", H#headers.content_length},\n            {\"HTTP_ACCEPT\", H#headers.accept},\n            {\"HTTP_USER_AGENT\", H#headers.user_agent},\n            {\"HTTP_REFERER\", H#headers.referer},\n            {\"HTTP_IF_MODIFIED_SINCE\", H#headers.if_modified_since},\n            {\"HTTP_IF_MATCH\", H#headers.if_match},\n            {\"HTTP_IF_NONE_MATCH\", H#headers.if_none_match},\n            {\"HTTP_IF_UNMODIFIED_SINCE\", H#headers.if_unmodified_since},\n            {\"HTTP_COOKIE\", flatten_val(make_cookie_val(H#headers.cookie))}\n           ]++ other_headers(H#headers.other)\n          )) ++\n        Extra_CGI_Vars.\n\nother_headers(Headers) ->\n    lists:zf(fun({http_header,_,Var,_,Val}) ->\n                     case tohttp(Var) of\n                         \"HTTP_PROXY\" ->\n                             %% See http://httpoxy.org/\n                             false;\n                         HTTP ->\n                             {true, {HTTP,Val}}\n                     end\n             end, Headers).\n\ntohttp(X) ->\n    \"HTTP_\"++lists:map(fun tohttp_c/1, yaws:to_list(X)).\n\n\ntohttp_c($-) ->\n    $_;\n\ntohttp_c(C) when C >= $a , C =< $z ->\n    C - $a + $A;\n\ntohttp_c(C) ->\n    C.\n\n\nmake_cookie_val([]) ->\n    undefined;\nmake_cookie_val([C]) ->\n    C;\nmake_cookie_val([C|CS]) ->\n    [make_cookie_val(CS), $; | C].\n\n\n%%% Seems not to be necessary, but open_port documentation says that\n%%% value has to be a string.\n\nflatten_val(L) when is_list(L) ->\n    lists:flatten(L);\nflatten_val(X) ->\n    X.\n\n\nnotslash($/) ->\n    false;\nnotslash(_) ->\n    true.\n\n\npathof(F) ->\n    case lists:dropwhile(fun notslash/1, lists:reverse(F)) of\n        \"/\" ->\n            \"/\";\n        [$/ | Tail] -> lists:reverse(Tail)\n    end.\n\n\nexeof(F) ->\n    [$\\., $/|lists:reverse(lists:takewhile(fun notslash/1, lists:reverse(F)))].\n\n\ndo_header(_Arg, \"HTTP/1.\"++[_,_,N1,N2,N3|_], _) ->\n    {status, list_to_integer([N1,N2,N3])};\ndo_header(Arg, Header, Data) when is_list(Header) ->\n    {HdrLower, HdrVal} = do_lower_header(Header),\n    do_header(Arg, {HdrLower, yaws:join_sep(HdrVal, \":\"), Header}, Data);\ndo_header(_Arg, {\"content-type\", CT, _}, {partial_data, Data}) ->\n    {streamcontent, CT, Data};\ndo_header(_Arg, {\"content-type\", CT, _}, {all_data, Data}) ->\n    {content, CT, Data};\ndo_header(_Arg, {\"status\", [N1,N2,N3|_], _}, _) ->\n    {status, list_to_integer([N1,N2,N3])};\ndo_header(_Arg, {_, _, Line}, _) ->\n    {header, Line}.\n\ndo_lower_header(Header) ->\n    [HdrName | HdrVal] = yaws:split_sep(Header, $:),\n    HdrNmParts = [yaws:to_lower(H) || H <- yaws:split_sep(HdrName, $-)],\n    {yaws:join_sep(HdrNmParts, \"-\"), HdrVal}.\n\nget_resp(WorkerPid) ->\n    get_resp([], WorkerPid).\n\nget_resp(Hs, WorkerPid) ->\n    receive\n        {WorkerPid, header, H} ->\n            ?Debug(\"~p~n\", [{WorkerPid, header, H}]),\n            get_resp([H|Hs], WorkerPid);\n        {WorkerPid, all_data, Data} ->\n            ?Debug(\"~p~n\", [{WorkerPid, all_data, Data}]),\n            {Hs, {all_data, Data}};\n        {WorkerPid, partial_data, Data} ->\n            ?Debug(\"~p~n\", [{WorkerPid, partial_data, binary_to_list(Data)}]),\n            {Hs, {partial_data, Data}};\n        {WorkerPid, failure, Reason} ->\n            ?Debug(\"~p~n\", [{WorkerPid, failure, Reason}]),\n            {failure, Reason};\n        _Other ->\n            ?Debug(\"~p~n\", [_Other]),\n            get_resp(Hs, WorkerPid)\n    end.\n\n\nget_opt(Key, List, Default) ->\n    case lists:keysearch(Key, 1, List) of\n        {value, {_Key, Val}} -> Val;\n        _ -> Default\n    end.\n\n\n%%%==========================================================================\n%%% Code specific to CGI\n%%%==========================================================================\n\n%%%  TO DO:  Handle failure and timeouts.\n\n%%%  call_cgi calls the script `Scriptfilename' (full path).\n%%%  If `Exefilename' is given, it is the executable to handle this,\n%%%  otherwise `Scriptfilame' is assumed to be executable itself.\n%%%\n%%%  Corresponding to a URI of\n%%%     `http://somehost/some/dir/script.cgi/path/info',\n%%%  `Pathinfo' should be set to `/path/info'.\n\n%%%  These functions can be used from a `.yaws' file.\n%%%  Note however, that they usually generate stream content.\n\ncall_cgi(Arg, Scriptfilename) ->\n    call_cgi(Arg, undefined, Scriptfilename, undefined, []).\n\ncall_cgi(Arg, Exefilename, Scriptfilename) ->\n    call_cgi(Arg, Exefilename, Scriptfilename, undefined, []).\n\ncall_cgi(Arg, Exefilename, Scriptfilename, Pathinfo) ->\n    call_cgi(Arg, Exefilename, Scriptfilename, Pathinfo, []).\n\ncall_cgi(Arg, Exefilename, Scriptfilename, Pathinfo, ExtraEnv) ->\n    case Arg#arg.state of\n        {cgistate, WorkerPid} ->\n            case Arg#arg.cont of\n                cgicont ->\n                    handle_clidata(Arg, WorkerPid);\n                undefined ->\n                    ?Debug(\"Error while reading clidata: ~p~n\",\n                           [Arg#arg.clidata]),\n                    %%  Error, what to do?\n                    exit(normal)\n            end;\n        _ ->\n            WorkerPid = cgi_start_worker(Arg, Exefilename, Scriptfilename,\n                                      Pathinfo, ExtraEnv, get(sc)),\n            handle_clidata(Arg, WorkerPid)\n    end.\n\n\ncgi_start_worker(Arg, Exefilename, Scriptfilename, Pathinfo, ExtraEnv, SC) ->\n    ExeFN = case Exefilename of\n                undefined -> exeof(Scriptfilename);\n                \"\" -> exeof(Scriptfilename);\n                FN -> FN\n            end,\n    PI = case Pathinfo of\n             undefined -> Arg#arg.pathinfo;\n             OK -> OK\n         end,\n    WorkerPid = proc_lib:spawn(?MODULE, cgi_worker,\n                               [self(), Arg, ExeFN, Scriptfilename,\n                                PI, ExtraEnv, SC]),\n    WorkerPid.\n\n\n\ncgi_worker(Parent, Arg, Exefilename, Scriptfilename, Pathinfo, ExtraEnv0, SC) ->\n    ExtraEnv = lists:map(fun({K,V}) when is_binary(K), is_binary(V) ->\n                                 {binary_to_list(K), binary_to_list(V)};\n                            ({K,V}) when is_binary(K) ->\n                                 {binary_to_list(K), V};\n                            ({K,V}) when is_binary(V) ->\n                                 {K, binary_to_list(V)};\n                            (KV) -> KV\n                         end, ExtraEnv0),\n    Env = build_env(Arg, Scriptfilename, Pathinfo, ExtraEnv, SC),\n    ?Debug(\"~p~n\", [Env]),\n    CGIPort = open_port({spawn, Exefilename},\n                        [{env, Env},\n                         {cd, pathof(Scriptfilename)},\n                         exit_status,\n                         binary]),\n    cgi_pass_through_clidata(Parent, CGIPort),\n    cgi_do_work(Parent, Arg, CGIPort).\n\n\ncgi_pass_through_clidata(Parent, CGIPort) ->\n    receive\n        {Parent, clidata, Clidata} ->\n            ?Debug(\"Got clidata ~p~n\", [binary_to_list(Clidata)]),\n            Parent ! {self(), clidata_receipt},\n            CGIPort ! {self(), {command, Clidata}},\n            cgi_pass_through_clidata(Parent, CGIPort);\n        {Parent, end_of_clidata} ->\n            ?Debug(\"End of clidata~n\", []),\n            ok\n    end.\n\n\ncgi_do_work(Parent, Arg, Port) ->\n    cgi_header_loop(Parent, Arg, {start, Port}).\n\n\ncgi_header_loop(Parent, Arg, S) ->\n    Line = cgi_get_line(S),\n    ?Debug(\"Line = ~p~n\", [Line]),\n    case Line of\n        {failure, F} ->\n            Parent ! {self(), failure, F};\n        {[], T} ->\n            case T of\n                {middle, Data, Port} ->\n                    Parent ! {self(), partial_data, Data},\n                    receive\n                        {Parent, stream_data} ->\n                            cgi_data_loop(Arg#arg.pid, Port);\n                        {Parent, no_data} ->\n                            ok\n                    end;\n                {ending, Data, _} ->\n                    Parent ! {self(), all_data, Data},\n                    receive\n                        {Parent, stream_data} ->\n                            yaws_api:stream_chunk_end(Arg#arg.pid);\n                        {Parent, no_data} ->\n                            ok\n                    end\n            end;\n        {H, T} ->\n            Parent ! {self(), header, H},\n            cgi_header_loop(Parent, Arg, T)\n    end.\n\n\ncgi_data_loop(Pid, Port) ->\n    receive\n        {Port, {data,Data}} ->\n            ?Debug(\"~p~n\", [{data, binary_to_list(Data)}]),\n            yaws_api:stream_chunk_deliver_blocking(Pid, Data),\n            cgi_data_loop(Pid, Port);\n        {Port, {exit_status, _Status}} ->\n            ?Debug(\"~p~n\", [{exit_status, _Status}]),\n            yaws_api:stream_chunk_end(Pid);\n        _Other ->\n            ?Debug(\"~p~n\", [_Other]),\n            cgi_data_loop(Pid, Port)\n    end.\n\n\n\ncgi_get_line({start, Port}) ->\n    receive\n        {Port, {data,Data}} ->\n            cgi_get_line([], {middle, Data, Port});\n        {Port, {exit_status, 0}} ->\n            ?Debug(\"~p~n\", [{exit_status, 0}]),\n            cgi_get_line([], {ending, <<>>, Port});\n        {Port, {exit_status, Status}} when Status /=0 ->\n            ?Debug(\"~p~n\", [{exit_status, Status}]),\n            {failure, {exit_status, Status}};\n        _Other ->\n            ?Debug(\"~p~n\", [_Other]),\n            cgi_get_line({start, Port})\n    end;\ncgi_get_line(State) ->\n    cgi_get_line([], State).\n\ncgi_get_line(Acc, {S, <<?ASCII_NEW_LINE, Tail/binary>>, Port}) ->\n    {lists:reverse(Acc), {S, Tail, Port}};\ncgi_get_line(Acc, {S, <<?ASCII_CARRIAGE_RETURN, ?ASCII_NEW_LINE, Tail/binary>>,\n                   Port}) ->\n    {lists:reverse(Acc), {S, Tail, Port}};\ncgi_get_line(Acc, {middle, <<>>, Port}) ->\n    cgi_get_line(Acc, cgi_add_resp(<<>>, Port));\ncgi_get_line(Acc, {middle, <<?ASCII_CARRIAGE_RETURN>>, Port}) ->\n    %% We SHOULD test for CRLF.\n    %% Would be easier without.\n    cgi_get_line(Acc, cgi_add_resp(<<?ASCII_CARRIAGE_RETURN>>, Port));\ncgi_get_line(Acc, {ending, <<>>, Port}) ->\n    {lists:reverse(Acc), {ending, <<>>, Port}};\ncgi_get_line(Acc, {S, <<C, Tail/binary>>, Port}) ->\n    cgi_get_line([C|Acc], {S, Tail, Port}).\n\n\ncgi_add_resp(Bin, Port) ->\n    receive\n        {Port, {data,Data}} ->\n            {middle, <<Bin/binary, Data/binary>>, Port};\n        {Port, {exit_status, _Status}} ->\n            ?Debug(\"~p~n\", [{exit_status, _Status}]),\n            {ending, Bin, Port};\n        _Other ->\n            ?Debug(\"~p~n\", [_Other]),\n            cgi_add_resp(Bin, Port)\n    end.\n\n\n%%%===========================================================================\n%%% Code specific to FastCGI\n%%%===========================================================================\n\n-define(FCGI_VERSION_1, 1).\n\n-define(FCGI_TYPE_BEGIN_REQUEST, 1).\n%%% Not needed yet\n%%%-define(FCGI_TYPE_ABORT_REQUEST, 2).\n-define(FCGI_TYPE_END_REQUEST, 3).\n-define(FCGI_TYPE_PARAMS, 4).\n-define(FCGI_TYPE_STDIN, 5).\n-define(FCGI_TYPE_STDOUT, 6).\n-define(FCGI_TYPE_STDERR, 7).\n%%% Not needed yet\n%%%-define(FCGI_TYPE_DATA, 8).\n%%%-define(FCGI_TYPE_GET_VALUES, 9).\n%%%-define(FCGI_TYPE_GET_VALUES_RESULT, 10).\n-define(FCGI_TYPE_UNKNOWN_TYPE, 11).\n\nfcgi_type_name(?FCGI_TYPE_BEGIN_REQUEST) -> \"begin-request\";\n%%% Not needed yet\n%%%fcgi_type_name(?FCGI_TYPE_ABORT_REQUEST) -> \"abort-request\";\nfcgi_type_name(?FCGI_TYPE_END_REQUEST) -> \"end-request\";\nfcgi_type_name(?FCGI_TYPE_PARAMS) -> \"params\";\nfcgi_type_name(?FCGI_TYPE_STDIN) -> \"stdin\";\nfcgi_type_name(?FCGI_TYPE_STDOUT) -> \"stdout\";\nfcgi_type_name(?FCGI_TYPE_STDERR) -> \"stderr\";\n%%% Not needed yet\n%%%fcgi_type_name(?FCGI_TYPE_DATA) -> \"data\";\n%%%fcgi_type_name(?FCGI_TYPE_GET_VALUES) -> \"get_values\";\n%%%fcgi_type_name(?FCGI_TYPE_GET_VALUES_RESULT) -> \"get_values_result\";\nfcgi_type_name(?FCGI_TYPE_UNKNOWN_TYPE) -> \"unknown-type\".\n\n%%% The FCGI implementation does not support handling concurrent requests\n%%% over a connection; it creates a separate connection for each\n%%% request. Hence, all application records have the same request-id,\n%%% namely 1.\n%%%\n-define(FCGI_REQUEST_ID_MANAGEMENT, 0).\n-define(FCGI_REQUEST_ID_APPLICATION, 1).\n\n-define(FCGI_DONT_KEEP_CONN, 0).\n-define(FCGI_KEEP_CONN, 1).\n\n-define(FCGI_ROLE_RESPONDER, 1).\n-define(FCGI_ROLE_AUTHORIZER, 2).\n-define(FCGI_ROLE_FILTER, 3).\n\n-ifdef(debug).   % To avoid compile warning if debug is disabled.\nfcgi_role_name(?FCGI_ROLE_RESPONDER) -> \"responder\";\nfcgi_role_name(?FCGI_ROLE_AUTHORIZER) -> \"authorizer\";\nfcgi_role_name(?FCGI_ROLE_FILTER) -> \"filter\";\nfcgi_role_name(_) -> \"?\".\n-endif.\n\n-define(FCGI_STATUS_REQUEST_COMPLETE, 0).\n-define(FCGI_STATUS_CANT_MPX_CONN, 1).\n-define(FCGI_STATUS_OVERLOADED, 2).\n-define(FCGI_STATUS_UNKNOWN_ROLE, 3).\n\nfcgi_status_name(?FCGI_STATUS_REQUEST_COMPLETE) -> \"request-complete\";\nfcgi_status_name(?FCGI_STATUS_CANT_MPX_CONN) -> \"cannot-multiple-connection\";\nfcgi_status_name(?FCGI_STATUS_OVERLOADED) -> \"overloaded\";\nfcgi_status_name(?FCGI_STATUS_UNKNOWN_ROLE) -> \"unknown-role\";\nfcgi_status_name(_) -> \"?\".\n\n%%% Amount of time (in milliseconds) allowed to connect to the application\n%%% server.\n%%%\n-define(FCGI_CONNECT_TIMEOUT_MSECS, 10000).\n\n%%% Amount of time (in milliseconds) allowed for data to arrive when\n%%% reading the TCP connection to the application server.\n%%%\n-define(FCGI_READ_TIMEOUT_MSECS, 10000).\n\n%%% TODO: Implement a configurable timeout which applies to the whole\n%%% operation (as oposed to individual socket reads).\n\n-record(fcgi_worker_state, {\n            app_server_host,            % The hostname or IP address of\n                                        % the application server\n            app_server_port,            % The TCP port number of the\n                                        % application server\n            path_info,                  % The path info\n            env,                        % All environment variables to be passed\n                                        % to the application (incl the extras)\n            keep_connection,            % Delegate close authority to the\n                                        % application?\n            trace_protocol,             % If true, log info messages for sent\n                                        % and received FastCGI messages\n            log_app_error,              % If true, log error messages for\n                                        % application errors (stderr and\n                                        % non-zero exit)\n            role,                       % The role of the worker\n                                        % (responder, authorizer, filter)\n            parent_pid,                 % The PID of the parent process = the\n                                        % Yaws worker process\n            yaws_worker_pid,            % When doing chunked output, stream to\n                                        % this Yaws worker.\n            app_server_socket,          % The TCP socket to the FastCGI\n                                        % application server\n            stream_to_socket            % The TCP socket to the web browser\n                                        % (stream chunked delivery to\n                                        %  this socket)\n        }).\n\n\ncall_fcgi_responder(Arg) ->\n    call_fcgi_responder(Arg, []).\n\ncall_fcgi_responder(Arg, Options) ->\n    call_fcgi(?FCGI_ROLE_RESPONDER, Arg, Options).\n\n\ncall_fcgi_authorizer(Arg) ->\n    call_fcgi_authorizer(Arg, []).\n\ncall_fcgi_authorizer(Arg, Options) ->\n    Out = call_fcgi(?FCGI_ROLE_AUTHORIZER, Arg, Options),\n    case fcgi_is_access_allowed(Out) of\n        true ->\n            StrippedOut = strip_content_from_out(Out),\n            {allowed, StrippedOut};\n        false ->\n            {denied, Out}\n    end.\n\n\ncall_fcgi(Role, Arg, Options) ->\n    case Arg#arg.state of\n        {cgistate, WorkerPid} ->\n            case Arg#arg.cont of\n                cgicont ->\n                    ?Debug(\"Call FastCGI: continuation~n\", []),\n                    handle_clidata(Arg, WorkerPid)\n            end;\n        _ ->\n            ?Debug(\"Call FastCGI:~n\"\n                   \"  Role = ~p (~s)~n\"\n                   \"  Options = ~p~n\"\n                   \"  Arg = ~p~n\",\n                   [Role, fcgi_role_name(Role),\n                    Options,\n                    Arg]),\n            GlobalConf = get(gc),\n            ServerConf = get(sc),\n            WorkerPid = fcgi_start_worker(Role, Arg, GlobalConf, ServerConf,\n              Options),\n            handle_clidata(Arg, WorkerPid)\n    end.\n\n\nis_not_content({content, _MimeType, _Content}) -> false;\nis_not_content({streamcontent, _MimeType, _Content}) -> false;\nis_not_content(_) -> true.\n\n\nstrip_content_from_out(Out) ->\n    lists:filter(fun is_not_content/1, Out).\n\n\nfcgi_worker_fail(WorkerState, Reason) ->\n    ParentPid = WorkerState#fcgi_worker_state.parent_pid,\n    ParentPid ! {self(), failure, Reason},\n    error_logger:error_msg(\"FastCGI failure: ~p~n\", [Reason]),\n    %% exit normally to avoid filling log with crash messages\n    exit(normal).\n\nfcgi_worker_fail_if(true, WorkerState, Reason) ->\n    fcgi_worker_fail(WorkerState, Reason);\nfcgi_worker_fail_if(_Condition, _WorkerState, _Reason) ->\n    ok.\n\nfcgi_start_worker(Role, Arg, GlobalConf, ServerConf, Options) ->\n    proc_lib:spawn(?MODULE, fcgi_worker,\n                   [self(), Role, Arg, GlobalConf, ServerConf, Options]).\n\n\nfcgi_worker(ParentPid, Role, Arg, GlobalConf, ServerConf, Options) ->\n    {DefaultSvrHost, DefaultSvrPort} =\n        case ServerConf#sconf.fcgi_app_server of\n            undefined ->\n                {undefined, undefined};\n            Else ->\n                Else\n        end,\n    AppServerHost = get_opt(app_server_host, Options, DefaultSvrHost),\n    AppServerPort = get_opt(app_server_port, Options, DefaultSvrPort),\n    PreliminaryWorkerState = #fcgi_worker_state{parent_pid = ParentPid},\n    fcgi_worker_fail_if(AppServerHost == undefined, PreliminaryWorkerState,\n                        \"app server host must be configured\"),\n    fcgi_worker_fail_if(AppServerPort == undefined, PreliminaryWorkerState,\n                        \"app server port must be configured\"),\n    PathInfo = get_opt(path_info, Options, Arg#arg.pathinfo),\n    ScriptFileName = Arg#arg.fullpath,\n    ExtraEnv = get_opt(extra_env, Options, []),\n    Env = build_env(Arg, ScriptFileName, PathInfo, ExtraEnv, ServerConf),\n    TraceProtocol = get_opt(trace_protocol, Options,\n                            ?sc_fcgi_trace_protocol(ServerConf)),\n    LogAppError = get_opt(log_app_error, Options,\n                          ?sc_fcgi_log_app_error(ServerConf)),\n    TcpOptions = yaws:gconf_nslookup_pref(GlobalConf),\n    AppServerSocket =\n        fcgi_connect_to_application_server(PreliminaryWorkerState,\n                                           AppServerHost, AppServerPort,\n                                           TcpOptions),\n    ?Debug(\"Start FastCGI worker:~n\"\n           \"  Role = ~p (~s)~n\"\n           \"  AppServerHost = ~p~n\"\n           \"  AppServerPort = ~p~n\"\n           \"  PathInfo = ~p~n\"\n           \"  ExtraEnv = ~p~n\"\n           \"  TraceProtocol = ~p~n\"\n           \"  LogAppStderr = ~p~n\",\n           [Role, fcgi_role_name(Role),\n            AppServerHost,\n            AppServerPort,\n            PathInfo,\n            ExtraEnv,\n            TraceProtocol,\n            LogAppError]),\n    WorkerState = #fcgi_worker_state{\n      app_server_host = AppServerHost,\n      app_server_port = AppServerPort,\n      path_info = PathInfo,\n      env = Env,\n      keep_connection = false,                % Currently hard-coded; make\n                                              % configurable in the future?\n      trace_protocol = TraceProtocol,\n      log_app_error = LogAppError,\n      role = Role,\n      parent_pid = ParentPid,\n      yaws_worker_pid = Arg#arg.pid,\n      app_server_socket = AppServerSocket\n     },\n    fcgi_send_begin_request(WorkerState),\n    fcgi_send_params(WorkerState, Env),\n    fcgi_send_params(WorkerState, []),\n    fcgi_pass_through_client_data(WorkerState),\n    fcgi_header_loop(WorkerState),\n    gen_tcp:close(AppServerSocket),\n    ok.\n\n\nfcgi_pass_through_client_data(WorkerState) ->\n    ParentPid = WorkerState#fcgi_worker_state.parent_pid,\n    receive\n        {ParentPid, clidata, <<>>} ->\n            ParentPid ! {self(), clidata_receipt},\n            fcgi_pass_through_client_data(WorkerState);\n        {ParentPid, clidata, ClientData} ->\n            ParentPid ! {self(), clidata_receipt},\n            fcgi_send_stdin(WorkerState, ClientData),\n            fcgi_pass_through_client_data(WorkerState);\n        {ParentPid, end_of_clidata} ->\n            fcgi_send_stdin(WorkerState, <<>>)\n    end.\n\n\nfcgi_connect_to_application_server(WorkerState, Host, Port, TcpOptions) ->\n    Options = [binary, {packet, 0}, {active, false}, {nodelay, true} |\n      TcpOptions],\n    case yaws:tcp_connect(Host, Port, Options, ?FCGI_CONNECT_TIMEOUT_MSECS) of\n        {error, Reason} ->\n            fcgi_worker_fail(WorkerState,\n                             {\"connect to application server failed\", Reason});\n        {ok, Socket} ->\n            Socket\n    end.\n\n\nfcgi_send_begin_request(WorkerState) ->\n    %% Not needed yet -- keep_connection is currently hard-coded to false\n    %%KeepConnection = WorkerState#fcgi_worker_state.keep_connection,\n    %%Flags = case KeepConnection of\n    %%            true -> ?FCGI_KEEP_CONN;\n    %%            false -> ?FCGI_DONT_KEEP_CONN\n    %%        end,\n    Flags = ?FCGI_DONT_KEEP_CONN,\n    Role = WorkerState#fcgi_worker_state.role,\n    fcgi_send_record(WorkerState, ?FCGI_TYPE_BEGIN_REQUEST,\n                     ?FCGI_REQUEST_ID_APPLICATION, <<Role:16, Flags:8, 0:40>>).\n\n\nfcgi_send_params(WorkerState, NameValueList) ->\n    fcgi_send_record(WorkerState, ?FCGI_TYPE_PARAMS,\n                     ?FCGI_REQUEST_ID_APPLICATION, NameValueList).\n\n\nfcgi_send_stdin(WorkerState, Data) ->\n    fcgi_send_record(WorkerState, ?FCGI_TYPE_STDIN,\n                     ?FCGI_REQUEST_ID_APPLICATION, Data).\n\n\n%%% Not needed yet\n%%%\n%%% fcgi_send_data(ParentPid, Socket, Data) ->\n%%%     fcgi_send_record(ParentPid, Socket, ?FCGI_TYPE_DATA,\n%%%                      ?FCGI_REQUEST_ID_APPLICATION, Data).\n\n\n%%% Not needed yet\n%%%\n%%% fcgi_send_abort_request(ParentPid, Socket) ->\n%%%     fcgi_send_record(ParentPid, Socket, ?FCGI_TYPE_ABORT_REQUEST,\n%%%                      ?FCGI_REQUEST_ID_APPLICATION, <<>>).\n\n\nfcgi_data_to_string(Data) ->\n    fcgi_data_to_string(\"\", 0, \"\", \"\", Data).\n\nfcgi_data_to_string(LinesStr, Count, CharStr, HexStr, <<>>) ->\n    if\n        Count == 0 ->\n            LinesStr;\n        true ->\n            Padding = lists:duplicate(16 - Count, $ ),\n            LinesStr ++ \"\\n    \" ++ CharStr ++ Padding ++ \"  \" ++ HexStr\n    end;\nfcgi_data_to_string(LinesStr, Count, CharStr, HexStr,\n                    <<Byte:8, MoreData/binary>>) ->\n    Char = if\n        (Byte >= $!) and (Byte =< $~) ->\n            Byte;\n        true ->\n            $.\n    end,\n    Hex = io_lib:format(\"~2.16.0b \", [Byte]),\n    if\n        Count == 16 ->\n            fcgi_data_to_string(LinesStr ++ \"\\n    \" ++ CharStr ++ \"  \" ++\n                                HexStr, 1, [Char], Hex, MoreData);\n        true ->\n            fcgi_data_to_string(LinesStr, Count + 1, CharStr ++ [Char],\n                                HexStr ++ Hex, MoreData)\n    end.\n\n\nfcgi_trace_protocol(WorkerState, Action, Version, Type, RequestId,\n                    ContentLength, PaddingLength, Reserved, ContentData,\n                    PaddingData) ->\n    Trace = WorkerState#fcgi_worker_state.trace_protocol,\n    if\n        Trace ->\n            error_logger:info_msg(\n                \"~s FastCGI record:~n\"\n                \"  version = ~p~n\"\n                \"  type = ~p (~s)~n\"\n                \"  request-id = ~p~n\"\n                \"  content-length = ~p~n\"\n                \"  padding-length = ~p~n\"\n                \"  reserved = ~p~n\"\n                \"  content-data = ~s~n\"\n                \"  padding-data = ~s~n\",\n                [Action,\n                 Version,\n                 Type, fcgi_type_name(Type),\n                 RequestId,\n                 ContentLength,\n                 PaddingLength,\n                 Reserved,\n                 fcgi_data_to_string(ContentData),\n                 fcgi_data_to_string(PaddingData)]);\n        true ->\n            ok\n    end.\n\n\nfcgi_send_record(WorkerState, Type, RequestId, NameValueList) ->\n    EncodedRecord = fcgi_encode_record(WorkerState, Type, RequestId,\n                                       NameValueList),\n    AppServerSocket = WorkerState#fcgi_worker_state.app_server_socket,\n    case gen_tcp:send(AppServerSocket, EncodedRecord) of\n        {error, Reason} ->\n            fcgi_worker_fail(WorkerState,\n                             {\"send to application server failed\", Reason});\n        ok ->\n            ok\n    end.\n\n\nfcgi_encode_record(WorkerState, Type, RequestId, NameValueList)\n  when is_list(NameValueList) ->\n    fcgi_encode_record(WorkerState, Type, RequestId,\n                       fcgi_encode_name_value_list(NameValueList));\n\nfcgi_encode_record(WorkerState, Type, RequestId, ContentData)\n  when is_binary(ContentData), size(ContentData) > 65535  ->\n    <<Bin:65535/binary, Rest/binary>> = ContentData,\n    [fcgi_encode_record(WorkerState, Type, RequestId, Bin),\n     fcgi_encode_record(WorkerState, Type, RequestId, Rest)];\n\nfcgi_encode_record(WorkerState, Type, RequestId, ContentData)\n  when is_binary(ContentData) ->\n    Version = 1,\n    ContentLength = size(ContentData),\n    %% Add padding bytes (if needed) to content bytes to make\n    %% content plus padding a multiple of 8 bytes.\n    PaddingLength = if\n                        ContentLength rem 8 == 0 ->\n                            0;\n                        true ->\n                            8 - (ContentLength rem 8)\n                    end,\n    PaddingData = <<0:(PaddingLength * 8)>>,\n    Reserved = 0,\n    fcgi_trace_protocol(WorkerState, \"Send\", Version, Type, RequestId,\n                        ContentLength, PaddingLength, Reserved,\n                        ContentData, PaddingData),\n    <<Version:8,\n      Type:8,\n      RequestId:16,\n      ContentLength:16,\n      PaddingLength:8,\n      Reserved:8,\n      ContentData/binary,\n      PaddingData/binary>>.\n\n\nfcgi_encode_name_value_list(_NameValueList = []) ->\n    <<>>;\nfcgi_encode_name_value_list(_NameValueList = [{Name, Value} | Tail]) ->\n    <<(fcgi_encode_name_value(Name,Value))/binary,\n      (fcgi_encode_name_value_list(Tail))/binary>>.\n\n\nfcgi_encode_name_value(Name, _Value = undefined) ->\n    fcgi_encode_name_value(Name, \"\");\nfcgi_encode_name_value(Name0, Value0) ->\n    Name = unicode:characters_to_binary(Name0),\n    Value = unicode:characters_to_binary(Value0),\n    NameSize = byte_size(Name),\n    %% If name size is < 128, encode it as one byte with the high bit clear.\n    %% If the name size >= 128, encoded it as 4 bytes with the high bit set.\n    NameSizeData = if\n                       NameSize < 128 ->\n                           <<NameSize:8>>;\n                       true ->\n                           <<(NameSize bor 16#80000000):32>>\n                   end,\n    %% Same encoding for the value size.\n    ValueSize = byte_size(Value),\n    ValueSizeData = if\n                        ValueSize < 128 ->\n                            <<ValueSize:8>>;\n                        true ->\n                            <<(ValueSize bor 16#80000000):32>>\n                    end,\n    list_to_binary([<<NameSizeData/binary, ValueSizeData/binary>>, Name, Value]).\n\n\nfcgi_header_loop(WorkerState) ->\n    fcgi_header_loop(WorkerState, start).\n\nfcgi_header_loop(WorkerState, LineState) ->\n    Line = fcgi_get_line(WorkerState, LineState),\n    ParentPid = WorkerState#fcgi_worker_state.parent_pid,\n    case Line of\n        {failure, Reason} ->\n            ParentPid ! {self(), failure, Reason};\n        {_EmptyLine = [], NewLineState} ->\n            case NewLineState of\n                {middle, Data} ->\n                    case WorkerState#fcgi_worker_state.role of\n                        ?FCGI_ROLE_AUTHORIZER ->\n                            % For authorization we never stream to the client\n                            fcgi_collect_all_data_loop(WorkerState, Data);\n                        _ ->\n                            ParentPid ! {self(), partial_data, Data},\n                            receive\n                                {ParentPid, stream_data} ->\n                                    fcgi_stream_data_loop(WorkerState);\n                                {ParentPid, no_data} ->\n                                    ok\n                            end\n                    end;\n                {ending, Data} ->\n                    ParentPid ! {self(), all_data, Data},\n                    receive\n                        {ParentPid, stream_data} ->\n                            yaws_api:stream_chunk_end(\n                              WorkerState#fcgi_worker_state.yaws_worker_pid);\n                        {ParentPid, no_data} ->\n                            ok\n                    end\n            end;\n        {Header, NewLineState} ->\n            ParentPid ! {self(), header, Header},\n            fcgi_header_loop(WorkerState, NewLineState)\n    end.\n\n\nfcgi_get_line(WorkerState, start) ->\n    case fcgi_get_output(WorkerState) of\n        {data, Data} ->\n            fcgi_get_line(WorkerState, [], {middle, Data});\n        {exit_status, 0} ->\n            fcgi_get_line(WorkerState, [], {ending, <<>>});\n        {exit_status, Status} when Status /=0 ->\n            {failure, {exit_status, Status}}\n    end;\nfcgi_get_line(WorkerState, LineState) ->\n    fcgi_get_line(WorkerState, [], LineState).\n\nfcgi_get_line(_WorkerState, Acc, {State, <<?ASCII_NEW_LINE, Tail/binary>>}) ->\n    {lists:reverse(Acc), {State, Tail}};\nfcgi_get_line(_WorkerState, Acc, {State, <<?ASCII_CARRIAGE_RETURN,\n                                           ?ASCII_NEW_LINE, Tail/binary>>}) ->\n    {lists:reverse(Acc), {State, Tail}};\nfcgi_get_line(WorkerState, Acc, {middle, <<>>}) ->\n    fcgi_get_line(WorkerState, Acc, fcgi_add_resp(WorkerState, <<>>));\nfcgi_get_line(WorkerState, Acc, {middle, <<?ASCII_CARRIAGE_RETURN>>}) ->\n    fcgi_get_line(WorkerState, Acc, fcgi_add_resp(WorkerState,\n                                                  <<?ASCII_CARRIAGE_RETURN>>));\nfcgi_get_line(_WorkerState, Acc, {ending, <<>>}) ->\n    {lists:reverse(Acc), {ending, <<>>}};\nfcgi_get_line(WorkerState, Acc, {State, <<Char, Tail/binary>>}) ->\n    fcgi_get_line(WorkerState, [Char | Acc], {State, Tail}).\n\n\nfcgi_add_resp(WorkerState, OldData) ->\n    case fcgi_get_output(WorkerState) of\n        {data, NewData} ->\n            {middle, <<OldData/binary, NewData/binary>>};\n        {exit_status, _Status} ->\n            {ending, OldData}\n    end.\n\n\nfcgi_stream_data_loop(WorkerState) ->\n    YawsWorkerPid = WorkerState#fcgi_worker_state.yaws_worker_pid,\n    case catch fcgi_get_output(WorkerState) of\n        {data, Data} ->\n            yaws_api:stream_chunk_deliver_blocking(YawsWorkerPid, Data),\n            fcgi_stream_data_loop(WorkerState);\n        {exit_status, _Status} ->\n            yaws_api:stream_chunk_end(YawsWorkerPid);\n        {'EXIT', _Reason} ->\n            yaws_api:stream_chunk_end(YawsWorkerPid)\n    end.\n\n\nfcgi_collect_all_data_loop(WorkerState, Data) ->\n    YawsWorkerPid = WorkerState#fcgi_worker_state.yaws_worker_pid,\n    case fcgi_get_output(WorkerState) of\n        {data, MoreData} ->\n            NewData = <<Data/binary, MoreData/binary>>,\n            fcgi_collect_all_data_loop(WorkerState, NewData);\n        {exit_status, _Status} ->\n            ParentPid = WorkerState#fcgi_worker_state.parent_pid,\n            ParentPid ! {self(), all_data, Data},\n            receive\n                {ParentPid, stream_data} ->\n                    yaws_api:stream_chunk_end(YawsWorkerPid);\n                {ParentPid, no_data} ->\n                    ok\n            end\n    end.\n\n\nfcgi_get_output(WorkerState) ->\n    {Type, ContentData} = fcgi_receive_record(WorkerState),\n    case Type of\n        ?FCGI_TYPE_END_REQUEST ->\n            <<AppStatus:32/signed, ProtStatus:8, _Reserved:24>> = ContentData,\n            fcgi_worker_fail_if(ProtStatus < ?FCGI_STATUS_REQUEST_COMPLETE,\n                                WorkerState,\n                                {\"received unknown protocol status\",\n                                 ProtStatus}),\n            fcgi_worker_fail_if(ProtStatus > ?FCGI_STATUS_UNKNOWN_ROLE,\n                                WorkerState,\n                                {\"received unknown protocol status\",\n                                 ProtStatus}),\n            if\n                ProtStatus /= ?FCGI_STATUS_REQUEST_COMPLETE ->\n                    error_logger:error_msg(\"FastCGI protocol error: ~p (~s)~n\",\n                                           [ProtStatus,\n                                            fcgi_status_name(ProtStatus)]);\n                true ->\n                    ok\n            end,\n            if\n                (AppStatus /= 0),\n                (WorkerState#fcgi_worker_state.log_app_error) ->\n                    error_logger:error_msg(\n                      \"FastCGI application non-zero exit status: ~p~n\",\n                      [AppStatus]);\n                true ->\n                    ok\n            end,\n            {exit_status, AppStatus};\n        ?FCGI_TYPE_STDOUT ->\n            {data, ContentData};\n        ?FCGI_TYPE_STDERR ->\n            if\n                (ContentData /= <<>>),\n                (WorkerState#fcgi_worker_state.log_app_error) ->\n                    error_logger:error_msg(\n                      \"FastCGI application stderr output:~s~n\",\n                      [fcgi_data_to_string(ContentData)]);\n                true ->\n                    ok\n            end,\n            fcgi_get_output(WorkerState);\n        ?FCGI_TYPE_UNKNOWN_TYPE ->\n            <<UnknownType:8, _Reserved:56>> = ContentData,\n            fcgi_worker_fail(\n              WorkerState,\n              {\"application did not understand record type we sent\",\n               UnknownType})\n    end.\n\n\nfcgi_receive_record(WorkerState) ->\n    Header = fcgi_receive_binary(WorkerState, 8, ?FCGI_READ_TIMEOUT_MSECS),\n    <<Version:8, Type:8, RequestId:16, ContentLength:16,\n      PaddingLength:8, Reserved:8>> = Header,\n    fcgi_worker_fail_if(Version /= 1, WorkerState,\n                        {\"received unsupported version\", Version}),\n    case Type of\n        ?FCGI_TYPE_END_REQUEST ->\n            fcgi_worker_fail_if(RequestId /= ?FCGI_REQUEST_ID_APPLICATION,\n                                WorkerState,\n                                {\"unexpected request id\", RequestId}),\n            fcgi_worker_fail_if(ContentLength /= 8, WorkerState,\n                                {\"incorrect content length for end request\",\n                                 ContentLength}),\n            ok;\n        ?FCGI_TYPE_STDOUT ->\n            fcgi_worker_fail_if(RequestId /= ?FCGI_REQUEST_ID_APPLICATION,\n                                WorkerState,\n                                {\"unexpected request id\", RequestId}),\n            ok;\n        ?FCGI_TYPE_STDERR ->\n            fcgi_worker_fail_if(RequestId /= ?FCGI_REQUEST_ID_APPLICATION,\n                                WorkerState,\n                                {\"unexpected request id\", RequestId}),\n            ok;\n        ?FCGI_TYPE_UNKNOWN_TYPE ->\n            fcgi_worker_fail_if(RequestId /= ?FCGI_REQUEST_ID_MANAGEMENT,\n                                WorkerState,\n                                {\"unexpected request id\", RequestId}),\n            fcgi_worker_fail_if(ContentLength /= 8, WorkerState,\n                                {\"incorrect content length for unknown type\",\n                                 ContentLength}),\n            ok;\n        OtherType ->\n            throw({\"received unexpected type\", OtherType})\n    end,\n    ContentData = case ContentLength of\n                      0 ->\n                          <<>>;\n                      _ ->\n                          fcgi_receive_binary(WorkerState, ContentLength,\n                                              ?FCGI_READ_TIMEOUT_MSECS)\n                  end,\n    case PaddingLength of\n        0 ->\n            {Type, ContentData};\n        _ ->\n            PaddingData = fcgi_receive_binary(WorkerState, PaddingLength,\n                                              ?FCGI_READ_TIMEOUT_MSECS),\n            fcgi_trace_protocol(WorkerState, \"Receive\",\n                                Version, Type, RequestId,\n                                ContentLength, PaddingLength,\n                                Reserved, ContentData,\n                                PaddingData),\n            {Type, ContentData}\n    end.\n\n\nfcgi_receive_binary(_WorkerState, Length, _Timeout) when Length == 0 ->\n    <<>>;\nfcgi_receive_binary(WorkerState, Length, Timeout) ->\n    AppServerSocket = WorkerState#fcgi_worker_state.app_server_socket,\n    case gen_tcp:recv(AppServerSocket, Length, Timeout) of\n        {error, Reason} ->\n            fcgi_worker_fail(WorkerState,\n                             {\"recv from application server failed\", Reason});\n        {ok, Data} ->\n            Data\n    end.\n\n\n%%% Access is allowed if, and only if, the resonse from the authorizer\n%%% running on the application server contains a 200 OK status. Any other\n%%% status or absence of a status means access is denied.\n%%%\nfcgi_is_access_allowed([Head | Tail]) ->\n    fcgi_is_access_allowed(Head) orelse fcgi_is_access_allowed(Tail);\nfcgi_is_access_allowed({status, 200}) ->\n    true;\nfcgi_is_access_allowed(_AnythingElse) ->\n    false.\n\n\n%%% Look for headers of the form \"Variable-VAR_NAME: var value\"\n%%%\nfcgi_extract_variables([Head | Tail]) ->\n    fcgi_extract_variables(Head) ++ fcgi_extract_variables(Tail);\nfcgi_extract_variables({header, \"Variable-\" ++ Rest}) ->\n    [fcgi_split_header(Rest)];\nfcgi_extract_variables(_AnythingElse) ->\n    [].\n\n\nfcgi_split_header(Header) ->\n    fcgi_split_header(name, [], [], Header).\n\nfcgi_split_header(_, NameAcc, ValueAcc, \"\") ->\n    {string:strip(lists:reverse(NameAcc)),\n     string:strip(lists:reverse(ValueAcc))};\nfcgi_split_header(name, NameAcc, ValueAcc, [$: | MoreStr]) ->\n    fcgi_split_header(value, NameAcc, ValueAcc, MoreStr);\nfcgi_split_header(name, NameAcc, ValueAcc, [Char | MoreStr]) ->\n    fcgi_split_header(name, [Char | NameAcc], ValueAcc, MoreStr);\nfcgi_split_header(value, NameAcc, ValueAcc, [Char | MoreStr]) ->\n    fcgi_split_header(value, NameAcc, [Char | ValueAcc], MoreStr).\n"
  },
  {
    "path": "contrib/yaws/src/yaws_charset.hrl",
    "content": "-define(YAWS_CHARSET, undefined).\n"
  },
  {
    "path": "contrib/yaws/src/yaws_compile.erl",
    "content": "-module(yaws_compile).\n-author('klacke@hyber.org').\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_debug.hrl\").\n\n-export([compile_file/1, compile_file/2]).\n\n-record(comp, {gc, %% global conf\n               script,\n               hash,\n               line = 1,\n               nummod = 1,\n               check_script = false,\n               nberrors = 0,\n               numchars = 0,\n               verbatim,\n               erlcode,\n               comp_opts = []}).\n\n-spec compile_file(File) -> {ok, NbErrors, CodeSpec} when\n      File         :: file:filename(),\n      NbErrors     :: non_neg_integer(),\n      CodeSpec     :: [Data | Binding | YawsMod | Verbatim | Skip | Error],\n      Data         :: {data, NumChars},\n      Binding      :: {binding, NumSkipChars},\n      YawsMod      :: {mod, LineNo, YawsFile, NumSkipChars,  Mod, Func},\n      Verbatim     :: {verbatim, NumSkipChars, Html},\n      Skip         :: {skip, NumSkipChars},\n      Error        :: {error, NumSkipChars, Reason},\n      NumChars     :: non_neg_integer(),\n      NumSkipChars :: non_neg_integer(),\n      LineNo       :: non_neg_integer(),\n      YawsFile     :: file:filename(),\n      Mod          :: atom(),\n      Func         :: atom(),\n      Html         :: iolist(),\n      Reason       :: iolist().\n\n-define(IS_SPACE(C), (C =:= $\\s orelse C =:= $\\t orelse\n                      C =:= $\\r orelse C =:= $\\n)).\n\n-define(IS_BINDING_CHAR(C), ((C >= $a andalso C =< $z) orelse\n                             (C >= $A andalso C =< $Z) orelse\n                             (C >= $0 andalso C =< $9) orelse\n                             C =:= $_ orelse C =:= $- orelse C =:= $.)).\n\ncompile_file(File) ->\n    compile_file(File, []).\n\ncompile_file(File, Opts) ->\n    case file:read_file(File) of\n        {ok, Bin} ->\n            GC = get(gc),\n            CheckFlag = (get(check_yaws_script) =:= true),\n            Comp = #comp{gc             = GC,\n                         script         = File,\n                         hash           = integer_to_list(erlang:phash2(File)),\n                         check_script   = CheckFlag,\n                         comp_opts      = compile_opts(GC, Opts, CheckFlag)},\n            global:trans({{yaws, Comp#comp.hash}, self()},\n                         fun() -> do_compile_file(Bin, Comp) end,\n                         [node()], infinity);\n        {error, Reason} ->\n            S = ?F(\"failed to open '~s': ~p~n\",\n                   [File, file:format_error(Reason)]),\n            case get(check_yaws_script) of\n                true  -> io:put_chars(S);\n                _     -> yaws:elog(\"~s\", [S])\n            end,\n            {ok, 1, [{error, 0, S}]}\n    end.\n\ndo_compile_file(Bin, Comp) ->\n    ?Debug(\"Compile ~s~n\", [Comp#comp.script]),\n    {ok, Spec0, NewComp} = compile_file(Bin, Comp, init, []),\n    Spec1 = join_data(Spec0),\n    ?Debug(\"NbErrors: ~p - Spec: ~p~n\", [NewComp#comp.nberrors, Spec1]),\n    {ok, NewComp#comp.nberrors, Spec1}.\n\n\n%% Possible states:\n%%   * init,\n%%   * html_tag, html, html_close_tag,\n%%   * verbatim_tag, verbatim, verbatim_close_tag\n%%   * erl_tag, erl, erl_string, erl_atom, erl_close_tag\ncompile_file(<<>>, Comp, erl, Spec) ->\n    compile_erl(0, <<>>, Comp, Spec);\ncompile_file(<<>>, Comp, _State, Spec) ->\n    NewComp = Comp#comp{numchars = 0,\n                        verbatim = undefined,\n                        erlcode  = undefined},\n    NewSpec = if\n                  Comp#comp.numchars == 0 -> lists:reverse(Spec);\n                  true -> lists:reverse([{data, Comp#comp.numchars}|Spec])\n              end,\n    {ok, NewSpec, NewComp};\n\n%% init state\ncompile_file(<<$\\n,Bin/binary>>, Comp, init, Spec) ->\n    NewComp = Comp#comp{line     = Comp#comp.line+1,\n                        numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, init, Spec);\ncompile_file(<<C,Bin/binary>>, Comp, init, Spec) when ?IS_SPACE(C) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, init, Spec);\ncompile_file(<<\"<erl>\",Bin/binary>>, Comp, init, Spec) ->\n    %% accumulate all spaces and skip it iff erl tag if found.\n    ?Debug(\"Start parsing erlang block at line ~p\", [Comp#comp.line]),\n    ErlCode = {Comp#comp.line, Comp#comp.line, undefined, []},\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+5,\n                        erlcode  = ErlCode},\n    compile_file(Bin, NewComp, erl, Spec);\ncompile_file(<<\"<erl\",C,Bin/binary>>, Comp, init, Spec) when ?IS_SPACE(C) ->\n    %% accumulate all spaces and skip it iff erl tag if found.\n    ?Debug(\"Start parsing erlang block at line ~p\", [Comp#comp.line]),\n    ErlCode = {Comp#comp.line, Comp#comp.line, undefined, []},\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+4,\n                        erlcode  = ErlCode},\n    compile_file(<<C,Bin/binary>>, NewComp, erl_tag, Spec);\ncompile_file(Bin, Comp, init, Spec) ->\n    compile_file(Bin, Comp, html, Spec);\n\n%% html_tag state\ncompile_file(<<\"%%\",Bin/binary>>, Comp, html_tag, Spec) ->\n    case parse_binding(Bin) of\n        {ok, Skipped, Rest} ->\n            NewComp = Comp#comp{numchars = 0},\n            compile_file(Rest, NewComp, html_tag,\n                         [{binding, Skipped+2}, {data, Comp#comp.numchars}|Spec]);\n        not_found ->\n            NewComp = Comp#comp{numchars = Comp#comp.numchars+2},\n            compile_file(Bin, NewComp, html_tag, Spec)\n    end;\ncompile_file(<<$>,Bin/binary>>, Comp, html_tag, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, html, Spec);\ncompile_file(<<$\\n,Bin/binary>>, Comp, html_tag, Spec) ->\n    NewComp = Comp#comp{line     = Comp#comp.line+1,\n                        numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, html_tag, Spec);\ncompile_file(<<_,Bin/binary>>, Comp, html_tag, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, html_tag, Spec);\n\n%% html state\ncompile_file(<<\"<erl>\",Bin/binary>>, Comp, html, Spec) ->\n    ?Debug(\"Start parsing erlang block at line ~p\", [Comp#comp.line]),\n    ErlCode = {Comp#comp.line, Comp#comp.line, undefined, []},\n    NewComp = Comp#comp{numchars = 5,\n                        erlcode  = ErlCode},\n    compile_file(Bin, NewComp, erl, [{data, Comp#comp.numchars}|Spec]);\ncompile_file(<<\"<erl\",C,Bin/binary>>, Comp, html, Spec) when ?IS_SPACE(C) ->\n    ?Debug(\"Start parsing erlang block at line ~p\", [Comp#comp.line]),\n    ErlCode = {Comp#comp.line, Comp#comp.line, undefined, []},\n    NewComp = Comp#comp{numchars = 4,\n                        erlcode  = ErlCode},\n    compile_file(<<C, Bin/binary>>, NewComp, erl_tag,\n                 [{data, Comp#comp.numchars}|Spec]);\ncompile_file(<<\"<verbatim>\",Bin/binary>>, Comp, html, Spec) ->\n    ?Debug(\"Start parsing verbatim block at line ~p\", [Comp#comp.line]),\n    NewComp = Comp#comp{numchars = 10,\n                        verbatim = {[], []}},\n    compile_file(Bin, NewComp, verbatim, [{data, Comp#comp.numchars}|Spec]);\ncompile_file(<<\"<verbatim\",C,Bin/binary>>, Comp, html, Spec) when ?IS_SPACE(C) ->\n    ?Debug(\"Start parsing verbatim block at line ~p\", [Comp#comp.line]),\n    NewComp = Comp#comp{numchars = 9,\n                        verbatim = {[], []}},\n    compile_file(<<C, Bin/binary>>, NewComp, verbatim_tag,\n                 [{data, Comp#comp.numchars}|Spec]);\ncompile_file(<<\"</\",Bin/binary>>, Comp, html, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+2},\n    compile_file(Bin, NewComp, html_close_tag, Spec);\ncompile_file(<<$<,Bin/binary>>, Comp, html, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, html_tag, Spec);\ncompile_file(<<\"%%\",Bin/binary>>, Comp, html, Spec) ->\n    case parse_binding(Bin) of\n        {ok, Skipped, Rest} ->\n            ?Debug(\"Binding key '~s' parsed at line ~p\",\n                   [binary:part(Bin, 0, Skipped), Comp#comp.line]),\n            NewComp = Comp#comp{numchars = 0},\n            compile_file(Rest, NewComp, html,\n                         [{binding, Skipped+2}, {data, Comp#comp.numchars}|Spec]);\n        not_found ->\n            NewComp = Comp#comp{numchars = Comp#comp.numchars+2},\n            compile_file(Bin, NewComp, html, Spec)\n    end;\ncompile_file(<<$\\n,Bin/binary>>, Comp, html, Spec) ->\n    NewComp = Comp#comp{line     = Comp#comp.line+1,\n                        numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, html, Spec);\ncompile_file(<<_,Bin/binary>>, Comp, html, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, html, Spec);\n\n%% html_close_tag state\ncompile_file(<<\"%%\",Bin/binary>>, Comp, html_close_tag, Spec) ->\n    case parse_binding(Bin) of\n        {ok, Skipped, Rest} ->\n            NewComp = Comp#comp{numchars = 0},\n            compile_file(Rest, NewComp, html_close_tag,\n                         [{binding, Skipped+2}, {data, Comp#comp.numchars}|Spec]);\n        not_found ->\n            NewComp = Comp#comp{numchars = Comp#comp.numchars+2},\n            compile_file(Bin, NewComp, html_close_tag, Spec)\n    end;\ncompile_file(<<$\\n,Bin/binary>>, Comp, html_close_tag, Spec) ->\n    NewComp = Comp#comp{line     = Comp#comp.line+1,\n                        numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, html_close_tag, Spec);\ncompile_file(<<$>,Bin/binary>>, Comp, html_close_tag, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, html, Spec);\ncompile_file(<<_,Bin/binary>>, Comp, html_close_tag, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, html_close_tag, Spec);\n\n%% verbatim_tag state\ncompile_file(<<\"/>\",Bin/binary>>, Comp, verbatim_tag, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+2,\n                        verbatim = undefined},\n    %% TODO: empty verbatim ?\n    compile_file(Bin, NewComp, html, Spec);\ncompile_file(<<$>,Bin/binary>>, Comp, verbatim_tag, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, verbatim, Spec);\ncompile_file(<<$\\n,Bin/binary>>, Comp, verbatim_tag, Spec) ->\n    {Attrs, Data} = Comp#comp.verbatim,\n    NewComp = Comp#comp{line     = Comp#comp.line+1,\n                        numchars = Comp#comp.numchars+1,\n                        verbatim = {[$\\n|Attrs], Data}},\n    compile_file(Bin, NewComp, verbatim_tag, Spec);\ncompile_file(<<C,Bin/binary>>, Comp, verbatim_tag, Spec) ->\n    {Attrs, Data} = Comp#comp.verbatim,\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1,\n                        verbatim = {[C|Attrs], Data}},\n    compile_file(Bin, NewComp, verbatim_tag, Spec);\n\n%% verbatim state\ncompile_file(<<\"</verbatim>\",Bin/binary>>, Comp, verbatim, Spec) ->\n    compile_verbatim(11, Bin, Comp, Spec);\ncompile_file(<<\"</verbatim\",C,Bin/binary>>, Comp, verbatim, Spec) when ?IS_SPACE(C) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+10},\n    compile_file(<<C,Bin/binary>>, NewComp, verbatim_close_tag, Spec);\ncompile_file(<<$\\n,Bin/binary>>, Comp, verbatim, Spec) ->\n    {Attrs, Data} = Comp#comp.verbatim,\n    NewComp = Comp#comp{line     = Comp#comp.line+1,\n                        numchars = Comp#comp.numchars+1,\n                        verbatim = {Attrs, [$\\n|Data]}},\n    compile_file(Bin, NewComp, verbatim, Spec);\ncompile_file(<<C,Bin/binary>>, Comp, verbatim, Spec) ->\n    {Attrs, Data} = Comp#comp.verbatim,\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1,\n                        verbatim = {Attrs, [yaws_api:htmlize_char(C)|Data]}},\n    compile_file(Bin, NewComp, verbatim, Spec);\n\n%% verbatim_close_tag state\ncompile_file(<<$\\n,Bin/binary>>, Comp, verbatim_close_tag, Spec) ->\n    NewComp = Comp#comp{line     = Comp#comp.line+1,\n                        numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, verbatim_close_tag, Spec);\ncompile_file(<<$>,Bin/binary>>, Comp, verbatim_close_tag, Spec) ->\n    compile_verbatim(1, Bin, Comp, Spec);\ncompile_file(<<_,Bin/binary>>, Comp, verbatim_close_tag, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, verbatim_close_tag, Spec);\n\n%% erl_tag state\ncompile_file(<<\"/>\",Bin/binary>>, Comp, erl_tag, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+2,\n                        erlcode  = undefined},\n    %% TODO: empty erl ?\n    compile_file(Bin, NewComp, html, Spec);\ncompile_file(<<$>,Bin/binary>>, Comp, erl_tag, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, erl, Spec);\ncompile_file(<<C,\"module\",Bin/binary>>, Comp, erl_tag, Spec) when ?IS_SPACE(C) ->\n    case parse_tag_attribute(Bin) of\n        {ok, Lines, Skipped, ModName, Data} ->\n            {SLine, ELine, _, _} = Comp#comp.erlcode,\n            NewComp = Comp#comp{line     = Comp#comp.line+Lines,\n                                numchars = Comp#comp.numchars+Skipped+7,\n                                erlcode  = {SLine, ELine+Lines, ModName, []}},\n            compile_file(Data, NewComp, erl_tag, Spec);\n        not_found ->\n            %% ignore malformed module attribute\n            NewComp = Comp#comp{numchars = Comp#comp.numchars+7},\n            compile_file(Bin, NewComp, erl_tag, Spec)\n    end;\ncompile_file(<<$\\n,Bin/binary>>, Comp, erl_tag, Spec) ->\n    NewComp = Comp#comp{line     = Comp#comp.line+1,\n                        numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, erl_tag, Spec);\ncompile_file(<<_,Bin/binary>>, Comp, erl_tag, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, erl_tag, Spec);\n\n%% erl state\ncompile_file(<<\"</erl>\",Bin/binary>>, Comp, erl, Spec) ->\n    compile_erl(6, Bin, Comp, Spec);\ncompile_file(<<\"</erl\",C,Bin/binary>>, Comp, erl, Spec) when ?IS_SPACE(C) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+5},\n    compile_file(<<C,Bin/binary>>, NewComp, erl_close_tag, Spec);\ncompile_file(<<$\",Bin/binary>>, Comp, erl, Spec) ->\n    {SLine, ELine, ModName, Code} = Comp#comp.erlcode,\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1,\n                        erlcode  = {SLine, ELine, ModName, [$\"|Code]}},\n    compile_file(Bin, NewComp, erl_string, Spec);\ncompile_file(<<$',Bin/binary>>, Comp, erl, Spec) ->\n    {SLine, ELine, ModName, Code} = Comp#comp.erlcode,\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1,\n                        erlcode  = {SLine, ELine, ModName, [$'|Code]}},\n    compile_file(Bin, NewComp, erl_atom, Spec);\ncompile_file(<<$\\n,Bin/binary>>, Comp, erl, Spec) ->\n    {SLine, ELine, ModName, Code} = Comp#comp.erlcode,\n    NewComp = Comp#comp{line     = Comp#comp.line+1,\n                        numchars = Comp#comp.numchars+1,\n                        erlcode  = {SLine, ELine+1, ModName, [$\\n|Code]}},\n    compile_file(Bin, NewComp, erl, Spec);\ncompile_file(<<C,Bin/binary>>, Comp, erl, Spec) ->\n    {SLine, ELine, ModName, Code} = Comp#comp.erlcode,\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1,\n                        erlcode  = {SLine, ELine, ModName, [C|Code]}},\n    compile_file(Bin, NewComp, erl, Spec);\n\n%% erl_string state\ncompile_file(<<$\\n,Bin/binary>>, Comp, erl_string, Spec) ->\n    {SLine, ELine, ModName, Code} = Comp#comp.erlcode,\n    NewComp = Comp#comp{line     = Comp#comp.line+1,\n                        numchars = Comp#comp.numchars+1,\n                        erlcode  = {SLine, ELine+1, ModName, [$\\n|Code]}},\n    compile_file(Bin, NewComp, erl_string, Spec);\ncompile_file(<<$\\\\,$\",Bin/binary>>, Comp, erl_string, Spec) ->\n    {SLine, ELine, ModName, Code} = Comp#comp.erlcode,\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+2,\n                        erlcode  = {SLine, ELine, ModName, [$\",$\\\\|Code]}},\n    compile_file(Bin, NewComp, erl_string, Spec);\ncompile_file(<<$\",Bin/binary>>, Comp, erl_string, Spec) ->\n    {SLine, ELine, ModName, Code} = Comp#comp.erlcode,\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1,\n                        erlcode  = {SLine, ELine, ModName, [$\"|Code]}},\n    compile_file(Bin, NewComp, erl, Spec);\ncompile_file(<<C,Bin/binary>>, Comp, erl_string, Spec) ->\n    {SLine, ELine, ModName, Code} = Comp#comp.erlcode,\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1,\n                        erlcode  = {SLine, ELine, ModName, [C|Code]}},\n    compile_file(Bin, NewComp, erl_string, Spec);\n\n%% erl_atom state\ncompile_file(<<$\\n,Bin/binary>>, Comp, erl_atom, Spec) ->\n    {SLine, ELine, ModName, Code} = Comp#comp.erlcode,\n    NewComp = Comp#comp{line     = Comp#comp.line+1,\n                        numchars = Comp#comp.numchars+1,\n                        erlcode  = {SLine, ELine+1, ModName, [$\\n|Code]}},\n    compile_file(Bin, NewComp, erl_atom, Spec);\ncompile_file(<<$\\\\,$',Bin/binary>>, Comp, erl_atom, Spec) ->\n    {SLine, ELine, ModName, Code} = Comp#comp.erlcode,\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+2,\n                        erlcode  = {SLine, ELine, ModName, [$',$\\\\|Code]}},\n    compile_file(Bin, NewComp, erl_atom, Spec);\ncompile_file(<<$',Bin/binary>>, Comp, erl_atom, Spec) ->\n    {SLine, ELine, ModName, Code} = Comp#comp.erlcode,\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1,\n                        erlcode  = {SLine, ELine, ModName, [$'|Code]}},\n    compile_file(Bin, NewComp, erl, Spec);\ncompile_file(<<C,Bin/binary>>, Comp, erl_atom, Spec) ->\n    {SLine, ELine, ModName, Code} = Comp#comp.erlcode,\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1,\n                        erlcode  = {SLine, ELine, ModName, [C|Code]}},\n    compile_file(Bin, NewComp, erl_atom, Spec);\n\n%% erl_close_tag state\ncompile_file(<<$\\n,Bin/binary>>, Comp, erl_close_tag, Spec) ->\n    NewComp = Comp#comp{line     = Comp#comp.line+1,\n                        numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, erl_close_tag, Spec);\ncompile_file(<<$>,Bin/binary>>, Comp, erl_close_tag, Spec) ->\n    compile_erl(1, Bin, Comp, Spec);\ncompile_file(<<_,Bin/binary>>, Comp, erl_close_tag, Spec) ->\n    NewComp = Comp#comp{numchars = Comp#comp.numchars+1},\n    compile_file(Bin, NewComp, erl_close_tag, Spec).\n\ncompile_verbatim(N, Bin, Comp, Spec) ->\n    ?Debug(\"Stop parsing verbatim block at line ~p\", [Comp#comp.line]),\n    {Attrs, Data} = Comp#comp.verbatim,\n    NewComp = Comp#comp{numchars=0, verbatim=undefined},\n    Html = case Attrs of\n               [] ->\n                   [\"<pre>\", lists:reverse(Data), \"</pre>\"];\n               _ ->\n                   [\"<pre \",lists:reverse(Attrs),$>,lists:reverse(Data),\"</pre>\"]\n           end,\n    NewSpec = [{verbatim,Comp#comp.numchars+N,Html}|Spec],\n    compile_file(Bin, NewComp, html, NewSpec).\n\ncompile_erl(N, Bin, Comp, Spec) ->\n    ?Debug(\"Stop parsing erlang block at line ~p\", [Comp#comp.line]),\n    NewComp = Comp#comp{nummod   = Comp#comp.nummod+1,\n                        numchars = 0,\n                        erlcode  = undefined},\n    case handle_erlang_block(Comp) of\n        skip ->\n            S = {skip, Comp#comp.numchars+N},\n            compile_file(Bin, NewComp, html, [S|Spec]);\n        {ok, Mod, Fun} ->\n            {Line, _, _, _} = Comp#comp.erlcode,\n            S = {mod, Line, Comp#comp.script, Comp#comp.numchars+N,  Mod, Fun},\n            compile_file(Bin, NewComp, html, [S|Spec]);\n        {error, Reason} ->\n            S = case Comp#comp.check_script of\n                    true  ->\n                        io:put_chars(Reason),\n                        Reason;\n                    false ->\n                        yaws:elog(\"~s\", [Reason]),\n                        ?F(\"~n<pre>~n~s~n</pre>~n\", [Reason])\n                end,\n            compile_file(Bin, NewComp#comp{nberrors=NewComp#comp.nberrors+1},\n                         html, [{error, Comp#comp.numchars+N, S}|Spec])\n    end.\n\njoin_data([])                          -> [];\njoin_data([{data, I}, {data, J}|Tail]) -> join_data([{data, I+J}|Tail]);\njoin_data([H|T])                       -> [H|join_data(T)].\n\nparse_binding(<<>>) ->\n    not_found;\nparse_binding(<<\"%%\",_/binary>>) ->\n    not_found;\nparse_binding(Bin) ->\n    parse_binding(Bin, 0).\n\nparse_binding(<<\"%%\",Bin/binary>>, NumChars) ->\n    {ok, NumChars+2, Bin};\nparse_binding(<<C,Bin/binary>>, NumChars) when ?IS_BINDING_CHAR(C) ->\n    parse_binding(Bin, NumChars+1);\nparse_binding(_, _) ->\n    not_found.\n\nparse_tag_attribute(<<>>) ->\n    not_found;\nparse_tag_attribute(Bin) ->\n    case skip_space(Bin) of\n        {ok, Lines, Skipped, <<$=,Data/binary>>} ->\n            parse_tag_attribute(Data, Lines, Skipped+1);\n        _ ->\n            not_found\n    end.\n\nparse_tag_attribute(Bin, Lines, NumChars) ->\n    case skip_space(Bin) of\n        {ok, N, Skipped, <<$\",Data/binary>>} ->\n            parse_tag_attribute(Data, $\", [], Lines+N, NumChars+Skipped+1);\n        {ok, N, Skipped, <<$',Data/binary>>} ->\n            parse_tag_attribute(Data, $', [], Lines+N, NumChars+Skipped+1);\n         _ ->\n            not_found\n    end.\n\nparse_tag_attribute(<<>>, _Sep, _Value, _Lines, _NumChars) ->\n    not_found;\nparse_tag_attribute(<<Sep,Bin/binary>>, Sep, Value, Lines, NumChars) ->\n    {ok, Lines, NumChars+1, lists:reverse(Value), Bin};\nparse_tag_attribute(<<$\\\\,Sep,Bin/binary>>, Sep, Value, Lines, NumChars) ->\n    parse_tag_attribute(Bin, Sep, [Sep,$\\\\|Value], Lines, NumChars+2);\nparse_tag_attribute(<<C,Bin/binary>>, Sep, Value, Lines, NumChars) ->\n    parse_tag_attribute(Bin, Sep, [C|Value], Lines, NumChars+1).\n\nskip_space(Bin) ->\n    skip_space(Bin, 0, 0).\n\nskip_space(<<$\\n,Bin/binary>>, Lines, NumChars) ->\n    skip_space(Bin, Lines+1, NumChars);\nskip_space(<<C,Bin/binary>>, Lines, NumChars) when ?IS_SPACE(C) ->\n    skip_space(Bin, Lines, NumChars+1);\nskip_space(Bin, Lines, NumChars) ->\n    {ok, Lines, NumChars, Bin}.\n\nhandle_erlang_block(Comp) ->\n    {Line, _, ModName, _} = Comp#comp.erlcode,\n    case check_module_name(ModName, Comp) of\n        ok ->\n            {Module, File} = get_module_info(ModName, Comp),\n            ?Debug(\"Writting generated module ~s in file ~s~n\", [Module, File]),\n            case dump_erlang_block(File, Module, Comp) of\n                ok ->\n                    Res = compile_and_load_erlang_block(File, Comp),\n                    file:delete(File),\n                    Res;\n                {error, Reason} ->\n                    S = io_lib:format(\"Error in file ~s at line ~p~n\"\n                                      \"    Failed to create temp file '~s': ~s~n\",\n                                      [Comp#comp.script, Line,\n                                       File, file:format_error(Reason)]),\n                    {error, lists:flatten(S)}\n            end;\n        {error, Reason} ->\n            S = io_lib:format(\"Error in file ~s at line ~p~n\"\n                              \"    Cannot create generated module '~s': ~s~n\",\n                              [Comp#comp.script, Line,\n                               ModName, Reason]),\n            {error, lists:flatten(S)}\n    end.\n\nget_module_info(undefined, Comp) ->\n    N = integer_to_list(Comp#comp.nummod),\n    case Comp#comp.check_script of\n        true  ->\n            YFile   = filename:rootname(Comp#comp.script),\n            ModName = lists:flatten([YFile, \"_yaws_\", N]),\n            {filename:basename(ModName), ModName++\".erl\"};\n        false ->\n            Dir     = yaws:id_dir((Comp#comp.gc)#gconf.id),\n            ModName = lists:flatten([\"m_\", Comp#comp.hash, $_, N]),\n            {ModName, filename:join([Dir, ModName++\".erl\"])}\n    end;\nget_module_info(ModName, Comp) ->\n    case Comp#comp.check_script of\n        true  ->\n            Dir = filename:dirname(Comp#comp.script),\n            {ModName, filename:join([Dir, ModName++\".erl\"])};\n        false ->\n            Dir = yaws:id_dir((Comp#comp.gc)#gconf.id),\n            {ModName, filename:join([Dir, ModName++\".erl\"])}\n    end.\n\ncheck_module_name(undefined, _) ->\n    ok;\ncheck_module_name(\"\", _) ->\n    {error, \"empty module name\"};\ncheck_module_name(ModName, Comp) ->\n    Mod = list_to_atom(ModName),\n    case code:is_loaded(Mod) of\n        {file, _} ->\n            case lists:keyfind(yawsfile, 1, Mod:module_info(attributes)) of\n                {yawsfile, Script} when Script =:= Comp#comp.script ->\n                    ok;\n                {yawsfile, OtherSript} ->\n                    S = io_lib:format(\"try to override generated module \"\n                                      \"owned by script ~s\", [OtherSript]),\n                    {error, S};\n                false ->\n                    S = io_lib:format(\"try to override existing module\", []),\n                    {error, S}\n            end;\n        false ->\n            ok\n    end.\n\ndump_erlang_block(File, Module, Comp) ->\n    case file:open(File, [write]) of\n        {ok, Fd} ->\n            GC = Comp#comp.gc,\n            {SLine, ELine, _, Code} = Comp#comp.erlcode,\n            Data = [\n                    \"-module('\",Module,\"').\\n\",\n                    \"\\n\",\n                    \"-export([out/1]).\\n\",\n                    \"\\n\",\n                    \"-yawsfile(\\\"\",Comp#comp.script,\"\\\").\\n\",\n                    \"\\n\",\n                    \"%%\\n\",\n                    \"%% code between lines \",integer_to_list(SLine),\" and \",\n                    integer_to_list(ELine),\"\\n\",\n                    \"%%\\n\",\n                    \"\\n\",\n                    \"-import(yaws_api, [f/2, fl/1, postvar/2, queryvar/2]).\\n\",\n                    \"\\n\",\n                    \"-include(\\\"\",GC#gconf.yaws_dir,\"/include/yaws_api.hrl\\\").\\n\",\n                    \"\\n\",\n                    lists:reverse(Code)\n                   ],\n            file:write(Fd, Data),\n            file:close(Fd),\n            ok;\n        Else ->\n            Else\n    end.\n\ncompile_and_load_erlang_block(File, Comp) ->\n    case compile_erlang_block(File, Comp) of\n        {ok, Mod, Bin, Ws} ->\n            report_compile_warnings(Comp, Ws),\n            load_erlang_block(Mod, Bin, Comp);\n        {error, Es, Ws} ->\n            {error, format_compile_error(Comp, Es, Ws)}\n    end.\n\ncompile_erlang_block(File, Comp) ->\n    case compile:file(File, Comp#comp.comp_opts) of\n        {ok, Module} ->\n            {ok, Module, <<>>, []};\n        {ok, Module, Binary} when is_binary(Binary) ->\n            {ok, Module, Binary, []};\n        {ok, Module, Warnings} ->\n            {ok, Module, <<>>, Warnings};\n        {ok, Module, Binary, Warnings} ->\n            {ok, Module, Binary, Warnings};\n        {error, Errors, Warnings} ->\n            {error, Errors, Warnings}\n    end.\n\nload_erlang_block(_Mod, <<>>, _Comp) ->\n    skip;\nload_erlang_block(Mod, Bin, Comp) ->\n    {Line, _, _, _} = Comp#comp.erlcode,\n    case code:load_binary(Mod, \"\", Bin) of\n        {module, Mod} ->\n            case lists:member({out,1}, Mod:module_info(exports)) of\n                true ->\n                    {ok, Mod, out};\n                false ->\n                    S = io_lib:format(\"Error in file ~s at line ~p~n\"\n                                      \"    out/1 is not defined~n\",\n                                      [Comp#comp.script, Line]),\n                    {error, lists:flatten(S)}\n            end;\n        Err ->\n            S = io_lib:format(\"Error in file ~s at line ~p~n\"\n                              \"    Cannot load generated module ~p: ~p~n\",\n                              [Comp#comp.script, Line, Mod, Err]),\n            {error, lists:flatten(S)}\n    end.\n\ncompile_opts(GC, Opts0, CheckFlag) ->\n    ?Debug(\"Includes = ~p~n\", [GC#gconf.include_dir]),\n    I = lists:map(fun(Dir) -> {i, Dir} end, GC#gconf.include_dir),\n    Warnings = if\n                   CheckFlag == true -> [return_warnings];\n                   true              -> []\n               end,\n    Opts = [binary, return_errors] ++ Warnings ++ I ++ Opts0,\n    ?Debug(\"Compile options = ~p~n\", [Opts]),\n    Opts.\n\nformat_compile_error(Comp, Errors, Warnings) ->\n    S1 = report_errors(Comp,   Errors),\n    S2 = report_warnings(Comp, Warnings),\n    S = io_lib:format(\"Dynamic compile error:~n~s~s\", [S1, S2]),\n    lists:flatten(S).\n\nreport_compile_warnings(#comp{check_script=true}=Comp, Warnings) ->\n    io:put_chars(report_warnings(Comp, Warnings));\nreport_compile_warnings(_, _) ->\n    ok.\n\nline(N) -> N - 15.\n\n%% -----------------------------------------------------------------\n%% Adapted from compile.erl.\nreport_errors(C, Es0) ->\n    File = C#comp.script,\n    {StartLine, _, _, _} = C#comp.erlcode,\n    SLine = line(StartLine),\n    lists:flatmap(fun ({{_F,_L},Eds}) -> format_errors(File, SLine, Eds);\n                      ({_F,Eds})      -> format_errors(File, SLine, Eds)\n                  end, Es0).\n\nreport_warnings(C, Ws0) ->\n    File = C#comp.script,\n    {StartLine, _, _, _} = C#comp.erlcode,\n    SLine = line(StartLine),\n    lists:flatmap(fun({{_F,_L},Eds}) -> format_warnings(File, SLine, Eds);\n                     ({_F,Eds})      -> format_warnings(File, SLine, Eds)\n                  end, Ws0).\n\nformat_warnings(F, SLine, [{Line0,Mod,E}|Es]) ->\n    Line = Line0 + SLine,\n    M = io_lib:format(\"~s:~w: Warning: ~s~n\", [F,Line,Mod:format_error(E)]),\n    [M|format_warnings(F, SLine, Es)];\nformat_warnings(F, SLine, [{Mod,E}|Es]) ->\n    M = io_lib:format(\"~s: Warning: ~s~n\", [F,Mod:format_error(E)]),\n    [M|format_warnings(F, SLine, Es)];\nformat_warnings(_, _, []) ->\n    [].\n\nformat_errors(F, SLine, [{Line0,Mod,E}|Es]) ->\n    Line = erlang:max(0, Line0 + SLine),\n    M = io_lib:format(\"~s:~w: ~s~n\", [F,Line,Mod:format_error(E)]),\n    [M|format_errors(F, SLine, Es)];\nformat_errors(F, SLine, [{Mod,E}|Es]) ->\n    M = io_lib:format(\"~s: ~s~n\", [F,Mod:format_error(E)]),\n    [M|format_errors(F, SLine, Es)];\nformat_errors(_F, _SLine, []) ->\n    [].\n"
  },
  {
    "path": "contrib/yaws/src/yaws_config.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_config.erl\n%%% Author  : Claes Wikstrom <klacke@bluetail.com>\n%%% Purpose :\n%%% Created : 16 Jan 2002 by Claes Wikstrom <klacke@bluetail.com>\n%%%----------------------------------------------------------------------\n\n-module(yaws_config).\n-author('klacke@bluetail.com').\n\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_debug.hrl\").\n\n-include_lib(\"kernel/include/file.hrl\").\n\n-define(NEXTLINE, io_get_line(FD, '', [])).\n\n-export([load/1,\n         make_default_gconf/2, make_default_sconf/0, make_default_sconf/3,\n         add_sconf/1,\n         add_yaws_auth/1,\n         add_yaws_soap_srv/1, add_yaws_soap_srv/2,\n         load_mime_types_module/2,\n         compile_and_load_src_dir/1,\n         search_sconf/3, search_group/3,\n         update_sconf/4, delete_sconf/3,\n         eq_sconfs/2, soft_setconf/4, hard_setconf/2,\n         can_hard_gc/2, can_soft_setconf/4,\n         can_soft_gc/2, verify_upgrade_args/2, toks/2]).\n\n%% where to look for yaws.conf\npaths() ->\n    case application:get_env(yaws, config) of\n        undefined ->\n            case yaws:getuid() of\n                {ok, \"0\"} ->    %% root\n                    [yaws_generated:etcdir() ++ \"/yaws/yaws.conf\"];\n                _ -> %% developer\n                    [filename:join([yaws:home(), \"yaws.conf\"]),\n                     \"./yaws.conf\",\n                     yaws_generated:etcdir() ++ \"/yaws/yaws.conf\"]\n            end;\n        {ok, File} ->\n            [File]\n    end.\n\n\n\n%% load the config\n\nload(E = #env{conf = false}) ->\n    case yaws:first(fun(F) -> yaws:exists(F) end, paths()) of\n        false ->\n            {error, \"Can't find any config file \"};\n        {ok, _, File} ->\n            load(E#env{conf = {file, File}})\n    end;\nload(E) ->\n    {file, File} = E#env.conf,\n    error_logger:info_msg(\"Yaws: Using config file ~s~n\", [File]),\n    case file:open(File, [read]) of\n        {ok, FD} ->\n            GC = make_default_gconf(E#env.debug, E#env.id),\n            GC1 = if E#env.traceoutput == undefined ->\n                          GC;\n                     true ->\n                          ?gc_set_tty_trace(GC, E#env.traceoutput)\n                  end,\n            GC2 =  ?gc_set_debug(GC1, E#env.debug),\n            GC3 = GC2#gconf{trace = E#env.trace},\n            R = fload(FD, GC3),\n            ?Debug(\"FLOAD(~s): ~p\", [File, R]),\n            case R of\n                {ok, GC4, Cs} ->\n                    yaws:mkdir(yaws:tmpdir()),\n                    Cs1 = add_yaws_auth(Cs),\n                    add_yaws_soap_srv(GC4),\n                    validate_cs(GC4, Cs1);\n                Err ->\n                    Err\n            end;\n        Err ->\n            {error, ?F(\"Can't open config file ~s: ~p\", [File, Err])}\n    end.\n\n\nadd_yaws_soap_srv(GC) when GC#gconf.enable_soap == true ->\n    add_yaws_soap_srv(GC, true);\nadd_yaws_soap_srv(_GC) ->\n    [].\nadd_yaws_soap_srv(GC, false) when GC#gconf.enable_soap == true ->\n    [{yaws_soap_srv, {yaws_soap_srv, start_link, [GC#gconf.soap_srv_mods]},\n      permanent, 5000, worker, [yaws_soap_srv]}];\nadd_yaws_soap_srv(GC, true) when GC#gconf.enable_soap == true ->\n    Spec = add_yaws_soap_srv(GC, false),\n    case whereis(yaws_soap_srv) of\n        undefined ->\n            spawn(fun() -> supervisor:start_child(yaws_sup, hd(Spec)) end);\n        _ ->\n            ok\n    end,\n    Spec;\nadd_yaws_soap_srv(_GC, _Start) ->\n    [].\n\n\nadd_yaws_auth(#sconf{}=SC) ->\n    SC#sconf{authdirs = setup_auth(SC)};\nadd_yaws_auth(SCs) ->\n    [SC#sconf{authdirs = setup_auth(SC)} || SC <- SCs].\n\n\n%% We search and setup www authenticate for each directory\n%% specified as an auth directory or containing a .yaws_auth file.\n%% These are merged with server conf.\nsetup_auth(#sconf{docroot = Docroot, xtra_docroots = XtraDocroots,\n                  authdirs = Authdirs}=SC) ->\n    [begin\n         Authdirs1 = load_yaws_auth_from_docroot(D, ?sc_auth_skip_docroot(SC)),\n         Authdirs2 = load_yaws_auth_from_authdirs(Authdirs, D, []),\n         Authdirs3 = [A || A <- Authdirs1,\n                           not lists:keymember(A#auth.dir,#auth.dir,Authdirs2)],\n         Authdirs4 = ensure_auth_headers(Authdirs3 ++ Authdirs2),\n         start_pam(Authdirs4),\n         {D, Authdirs4}\n     end || D <- [Docroot|XtraDocroots] ].\n\n\nload_yaws_auth_from_docroot(_, true) ->\n    [];\nload_yaws_auth_from_docroot(undefined, _) ->\n    [];\nload_yaws_auth_from_docroot(Docroot, _) ->\n    Fun = fun (Path, Acc) ->\n                  %% Strip Docroot and then filename\n                  SP  = string:sub_string(Path, length(Docroot)+1),\n                  Dir = filename:dirname(SP),\n                  A = #auth{docroot=Docroot, dir=Dir},\n                  case catch load_yaws_auth_file(Path, A) of\n                      {ok, L} -> L ++ Acc;\n                      _Other  -> Acc\n                  end\n          end,\n    filelib:fold_files(Docroot, \"^.yaws_auth$\", true, Fun, []).\n\n\nload_yaws_auth_from_authdirs([], _, Acc) ->\n    lists:reverse(Acc);\nload_yaws_auth_from_authdirs([Auth = #auth{dir=Dir}| Rest], Docroot, Acc) ->\n    if\n        Auth#auth.docroot /= [] andalso Auth#auth.docroot /= Docroot ->\n            load_yaws_auth_from_authdirs(Rest, Docroot, Acc);\n        Auth#auth.docroot == [] ->\n            Auth1 = Auth#auth{dir=filename:nativename(Dir)},\n            F = fun(A) ->\n                        (A#auth.docroot == Docroot andalso\n                         A#auth.dir == Auth1#auth.dir)\n                end,\n            case lists:any(F, Acc) of\n                true ->\n                    load_yaws_auth_from_authdirs(Rest, Docroot, Acc);\n                false ->\n                    Acc1 = Acc ++ load_yaws_auth_from_authdir(Docroot, Auth1),\n                    load_yaws_auth_from_authdirs(Rest, Docroot, Acc1)\n            end;\n        true -> %% #auth.docroot == Docroot\n            Auth1 = Auth#auth{docroot=Docroot, dir=filename:nativename(Dir)},\n            F = fun(A) ->\n                        not (A#auth.docroot == [] andalso\n                             A#auth.dir == Auth1#auth.dir)\n                end,\n            Acc1 = lists:filter(F, Acc),\n            Acc2 = Acc1 ++ load_yaws_auth_from_authdir(Docroot, Auth1),\n            load_yaws_auth_from_authdirs(Rest, Docroot, Acc2)\n    end;\nload_yaws_auth_from_authdirs([{Docroot, Auths}|_], Docroot, Acc) ->\n    load_yaws_auth_from_authdirs(Auths, Docroot, Acc);\nload_yaws_auth_from_authdirs([_| Rest], Docroot, Acc) ->\n    load_yaws_auth_from_authdirs(Rest, Docroot, Acc).\n\n\nload_yaws_auth_from_authdir(Docroot, Auth) ->\n    Dir = case Auth#auth.dir of\n              \"/\" ++ R -> R;\n              _        -> Auth#auth.dir\n          end,\n    Path = filename:join([Docroot, Dir, \".yaws_auth\"]),\n    case catch load_yaws_auth_file(Path, Auth) of\n        {ok, Auths} -> Auths;\n        _           -> [Auth]\n    end.\n\n\nload_yaws_auth_file(Path, Auth) ->\n    case file:consult(Path) of\n        {ok, TermList} ->\n            error_logger:info_msg(\"Reading .yaws_auth ~s~n\", [Path]),\n            parse_yaws_auth_file(TermList, Auth);\n        {error, enoent} ->\n            {error, enoent};\n        Error ->\n            error_logger:format(\"Bad .yaws_auth file ~s ~p~n\", [Path, Error]),\n            Error\n    end.\n\n\nensure_auth_headers(Authdirs) ->\n    [add_auth_headers(Auth) || Auth <- Authdirs].\n\nadd_auth_headers(Auth = #auth{headers = []}) ->\n    %% Headers needs to be set\n    Realm   = Auth#auth.realm,\n    Headers = yaws:make_www_authenticate_header({realm, Realm}),\n    Auth#auth{headers = Headers};\nadd_auth_headers(Auth) ->\n    Auth.\n\n\nstart_pam([]) ->\n    ok;\nstart_pam([#auth{pam = false}|T]) ->\n    start_pam(T);\nstart_pam([A|T]) ->\n    case whereis(yaws_pam) of\n        undefined ->    % pam not started\n            Spec = {yaws_pam, {yaws_pam, start_link,\n                               [yaws:to_list(A#auth.pam),undefined,undefined]},\n                    permanent, 5000, worker, [yaws_pam]},\n            spawn(fun() -> supervisor:start_child(yaws_sup, Spec) end);\n        _ ->\n            start_pam(T)\n    end.\n\n\nparse_yaws_auth_file([], Auth=#auth{files=[]}) ->\n    {ok, [Auth]};\nparse_yaws_auth_file([], Auth=#auth{dir=Dir, files=Files}) ->\n    {ok, [Auth#auth{dir=filename:join(Dir, F), files=[F]} || F <- Files]};\n\nparse_yaws_auth_file([{realm, Realm}|T], Auth0) ->\n    parse_yaws_auth_file(T, Auth0#auth{realm = Realm});\n\nparse_yaws_auth_file([{pam, Pam}|T], Auth0)\n  when is_atom(Pam) ->\n    parse_yaws_auth_file(T, Auth0#auth{pam = Pam});\n\nparse_yaws_auth_file([{authmod, Authmod0}|T], Auth0)\n  when is_atom(Authmod0)->\n    Headers = try\n                  Authmod0:get_header() ++ Auth0#auth.headers\n              catch\n                  _:_ ->\n                      error_logger:format(\"Failed to ~p:get_header() \\n\",\n                                          [Authmod0]),\n                      Auth0#auth.headers\n              end,\n    parse_yaws_auth_file(T, Auth0#auth{mod = Authmod0, headers = Headers});\n\nparse_yaws_auth_file([{file, File}|T], Auth0) ->\n    Files = case File of\n                \"/\" ++ F -> [F|Auth0#auth.files];\n                _        -> [File|Auth0#auth.files]\n            end,\n    parse_yaws_auth_file(T, Auth0#auth{files=Files});\n\nparse_yaws_auth_file([{User, Password}|T], Auth0)\n  when is_list(User), is_list(Password) ->\n    Salt = yaws_dynopts:rand_bytes(32),\n    Hash = crypto:hash(sha256, [Salt, Password]),\n    Users = case lists:member({User, sha256, Salt, Hash}, Auth0#auth.users) of\n                true  -> Auth0#auth.users;\n                false -> [{User, sha256, Salt, Hash} | Auth0#auth.users]\n            end,\n    parse_yaws_auth_file(T, Auth0#auth{users = Users});\n\nparse_yaws_auth_file([{User, Algo, B64Hash}|T], Auth0)\n  when is_list(User), is_list(Algo), is_list(B64Hash) ->\n    case parse_auth_user(User, Algo, \"\", B64Hash) of\n        {ok, Res} ->\n            Users = case lists:member(Res, Auth0#auth.users) of\n                        true  -> Auth0#auth.users;\n                        false -> [Res | Auth0#auth.users]\n                    end,\n            parse_yaws_auth_file(T, Auth0#auth{users = Users});\n        {error, Reason} ->\n            error_logger:format(\"Failed to parse user line ~p: ~p~n\",\n                                [{User, Algo, B64Hash}, Reason]),\n            parse_yaws_auth_file(T, Auth0)\n    end;\n\nparse_yaws_auth_file([{User, Algo, B64Salt, B64Hash}|T], Auth0)\n  when is_list(User), is_list(Algo), is_list(B64Salt), is_list(B64Hash) ->\n    case parse_auth_user(User, Algo, B64Salt, B64Hash) of\n        {ok, Res} ->\n            Users = case lists:member(Res, Auth0#auth.users) of\n                        true  -> Auth0#auth.users;\n                        false -> [Res | Auth0#auth.users]\n                    end,\n            parse_yaws_auth_file(T, Auth0#auth{users = Users});\n        {error, Reason} ->\n            error_logger:format(\"Failed to parse user line ~p: ~p~n\",\n                                [{User, Algo, B64Hash, B64Hash}, Reason]),\n            parse_yaws_auth_file(T, Auth0)\n    end;\n\nparse_yaws_auth_file([{allow, all}|T], Auth0) ->\n    Auth1 = case Auth0#auth.acl of\n                none    -> Auth0#auth{acl={all, [], deny_allow}};\n                {_,D,O} -> Auth0#auth{acl={all, D, O}}\n            end,\n    parse_yaws_auth_file(T, Auth1);\n\nparse_yaws_auth_file([{allow, IPs}|T], Auth0) when is_list(IPs) ->\n    Auth1 = case Auth0#auth.acl of\n                none ->\n                    AllowIPs = parse_auth_ips(IPs, []),\n                    Auth0#auth{acl={AllowIPs, [], deny_allow}};\n                {all, _, _} ->\n                    Auth0;\n                {AllowIPs, DenyIPs, Order} ->\n                    AllowIPs2 = parse_auth_ips(IPs, []) ++ AllowIPs,\n                    Auth0#auth{acl={AllowIPs2, DenyIPs, Order}}\n            end,\n    parse_yaws_auth_file(T, Auth1);\n\nparse_yaws_auth_file([{deny, all}|T], Auth0) ->\n    Auth1 = case Auth0#auth.acl of\n                none    -> Auth0#auth{acl={[], all, deny_allow}};\n                {A,_,O} -> Auth0#auth{acl={A, all, O}}\n            end,\n    parse_yaws_auth_file(T, Auth1);\n\nparse_yaws_auth_file([{deny, IPs}|T], Auth0) when is_list(IPs) ->\n    Auth1 = case Auth0#auth.acl of\n                none ->\n                    DenyIPs = parse_auth_ips(IPs, []),\n                    Auth0#auth{acl={[], DenyIPs, deny_allow}};\n                {_, all, _} ->\n                    Auth0;\n                {AllowIPs, DenyIPs, Order} ->\n                    DenyIPs2 = parse_auth_ips(IPs, []) ++ DenyIPs,\n                    Auth0#auth{acl={AllowIPs, DenyIPs2, Order}}\n            end,\n    parse_yaws_auth_file(T, Auth1);\n\nparse_yaws_auth_file([{order, O}|T], Auth0)\n  when O == allow_deny; O == deny_allow ->\n    Auth1 = case Auth0#auth.acl of\n                none    -> Auth0#auth{acl={[], [], O}};\n                {A,D,_} -> Auth0#auth{acl={A, D, O}}\n            end,\n    parse_yaws_auth_file(T, Auth1).\n\n\n\n%% Create mime_types.erl, compile it and load it. If everything is ok,\n%% reload groups.\n%%\n%% If an error occured, the previously-loaded version (the first time, it's the\n%% static version) is kept.\nload_mime_types_module(GC, Groups) ->\n    GInfo  = GC#gconf.mime_types_info,\n    SInfos = [{{SC#sconf.servername, SC#sconf.port}, SC#sconf.mime_types_info}\n              || SC <- lists:flatten(Groups),\n                 SC#sconf.mime_types_info /= undefined],\n\n    case {is_dir(yaws:id_dir(GC#gconf.id)), is_dir(yaws:tmpdir(\"/tmp\"))} of\n        {true, _} ->\n            File = filename:join(yaws:id_dir(GC#gconf.id), \"mime_types.erl\"),\n            load_mime_types_module(File, GInfo, SInfos);\n        {_, true} ->\n            File = filename:join(yaws:tmpdir(\"/tmp\"), \"mime_types.erl\"),\n            load_mime_types_module(File, GInfo, SInfos);\n        _ ->\n            error_logger:format(\"Cannot write module mime_types.erl~n\"\n                                \"Keep the previously-loaded version~n\", [])\n    end,\n    lists:map(fun(Gp) ->\n                      [begin\n                           F   = fun(X) when is_atom(X) -> X;\n                                    (X) -> element(1, mime_types:t(SC, X))\n                                 end,\n                           TAS = SC#sconf.tilde_allowed_scripts,\n                           AS  = SC#sconf.allowed_scripts,\n                           SC#sconf{tilde_allowed_scripts=lists:map(F, TAS),\n                                    allowed_scripts=lists:map(F, AS)}\n                       end || SC <- Gp]\n              end, Groups).\n\nload_mime_types_module(_, undefined, []) ->\n    ok;\nload_mime_types_module(File, undefined, SInfos) ->\n    load_mime_types_module(File, #mime_types_info{}, SInfos);\nload_mime_types_module(File, GInfo, SInfos) ->\n    case mime_type_c:generate(File, GInfo, SInfos) of\n        ok ->\n            case compile:file(File, [binary]) of\n                {ok, ModName, Binary} ->\n                    case code:load_binary(ModName, [], Binary) of\n                        {module, ModName} ->\n                            ok;\n                        {error, What} ->\n                            error_logger:format(\n                              \"Cannot load module '~p': ~p~n\"\n                              \"Keep the previously-loaded version~n\",\n                              [ModName, What]\n                             )\n                    end;\n                _ ->\n                    error_logger:format(\"Compilation of '~p' failed~n\"\n                                        \"Keep the previously-loaded version~n\",\n                                        [File])\n            end;\n        {error, Reason} ->\n            error_logger:format(\"Cannot write module ~p: ~p~n\"\n                                \"Keep the previously-loaded version~n\",\n                                [File, Reason])\n    end.\n\n\n%% Compile modules found in the configured source directories, recursively.\ncompile_and_load_src_dir(GC) ->\n    Incs = lists:map(fun(Dir) -> {i, Dir} end, GC#gconf.include_dir),\n    Opts = [binary, return] ++ Incs,\n    lists:foreach(fun(D) -> compile_and_load_src_dir([], [D], Opts) end,\n                  GC#gconf.src_dir).\n\ncompile_and_load_src_dir(_Dir, [], _Opts) ->\n    ok;\ncompile_and_load_src_dir(Dir, [Entry0|Rest], Opts) ->\n    Entry1 = case Dir of\n                 [] -> Entry0;\n                 _  -> filename:join(Dir, Entry0)\n             end,\n    case filelib:is_dir(Entry1) of\n        true ->\n            case file:list_dir(Entry1) of\n                {ok, Files} ->\n                    compile_and_load_src_dir(Entry1, Files, Opts);\n                {error, Reason} ->\n                    error_logger:format(\"Failed to compile modules in ~p: ~s~n\",\n                                        [Entry1, file:format_error(Reason)])\n            end;\n        false ->\n            case filename:extension(Entry0) of\n                \".erl\" -> compile_module_src_dir(Entry1, Opts);\n                _      -> ok\n            end\n    end,\n    compile_and_load_src_dir(Dir, Rest, Opts).\n\n\ncompile_module_src_dir(File, Opts) ->\n    case catch compile:file(File, Opts) of\n        {ok, Mod, Bin} ->\n            error_logger:info_msg(\"Compiled ~p~n\", [File]),\n            load_src_dir(File, Mod, Bin);\n        {ok, Mod, Bin, []} ->\n            error_logger:info_msg(\"Compiled ~p [0 Errors - 0 Warnings]~n\", [File]),\n            load_src_dir(File, Mod, Bin);\n        {ok, Mod, Bin, Warnings} ->\n            WsMsg = [format_compile_warns(W,[]) || W <- Warnings],\n            error_logger:warning_msg(\"Compiled ~p [~p Errors - ~p Warnings]~n~s\",\n                                     [File,0,length(WsMsg),WsMsg]),\n            load_src_dir(File, Mod, Bin);\n        {error, [], Warnings} ->\n            WsMsg = [format_compile_warns(W,[]) || W <- Warnings],\n            error_logger:format(\"Failed to compile ~p \"\n                                \"[~p Errors - ~p Warnings]~n~s\"\n                                \"*** warnings being treated as errors~n\",\n                                [File,0,length(WsMsg),WsMsg]);\n        {error, Errors, Warnings} ->\n            WsMsg = [format_compile_warns(W,[]) || W <- Warnings],\n            EsMsg = [format_compile_errs(E,[])  || E <- Errors],\n            error_logger:format(\"Failed to compile ~p \"\n                                \"[~p Errors - ~p Warnings]~n~s~s\",\n                                [File,length(EsMsg),length(WsMsg),EsMsg,WsMsg]);\n        error ->\n            error_logger:format(\"Failed to compile ~p~n\", [File]);\n        {'EXIT', Reason} ->\n            error_logger:format(\"Failed to compile ~p: ~p~n\", [File, Reason])\n    end.\n\n\nload_src_dir(File, Mod, Bin) ->\n    case code:load_binary(Mod, File, Bin) of\n        {module, Mod}   -> ok;\n        {error, Reason} -> error_logger:format(\"Cannot load module ~p: ~p~n\",\n                                               [Mod, Reason])\n    end.\n\nformat_compile_warns({_, []}, Acc) ->\n    lists:reverse(Acc);\nformat_compile_warns({File, [{L,M,E}|Rest]}, Acc) ->\n    Msg = io_lib:format(\"    ~s:~w: Warning: ~s~n\", [File,L,M:format_error(E)]),\n    format_compile_warns({File, Rest}, [Msg|Acc]).\n\nformat_compile_errs({_, []}, Acc) ->\n    lists:reverse(Acc);\nformat_compile_errs({File, [{L,M,E}|Rest]}, Acc) ->\n    Msg = io_lib:format(\"    ~s:~w: ~s~n\", [File,L,M:format_error(E)]),\n    format_compile_errs({File, Rest}, [Msg|Acc]).\n\n\n\n%% This is the function that arranges sconfs into\n%% different server groups\nvalidate_cs(GC, Cs) ->\n    L = lists:map(fun(#sconf{listen=IP0}=SC0) ->\n                          SC = case is_tuple(IP0) of\n                                   false ->\n                                       {ok, IP} = inet_parse:address(IP0),\n                                       SC0#sconf{listen=IP};\n                                   true ->\n                                       SC0\n                               end,\n                              {{SC#sconf.listen, SC#sconf.port}, SC}\n                  end, Cs),\n    L2 = lists:map(fun(X) -> element(2, X) end, lists:keysort(1,L)),\n    L3 = arrange(L2, start, [], []),\n    case validate_groups(GC, L3) of\n        ok ->\n            {ok, GC, L3};\n        Err ->\n            Err\n    end.\n\n\nvalidate_groups(_, []) ->\n    ok;\nvalidate_groups(GC, [H|T]) ->\n    case (catch validate_group(GC, H)) of\n        ok ->\n            validate_groups(GC, T);\n        Err ->\n            Err\n    end.\n\nvalidate_group(GC, List) ->\n    [SC0|SCs] = List,\n\n    %% all servers with the same IP/Port must share the same tcp configuration\n    case lists:all(fun(SC) ->\n                           lists:keyfind(listen_opts, 1, SC#sconf.soptions) ==\n                               lists:keyfind(listen_opts, 1, SC0#sconf.soptions)\n                   end, SCs) of\n        true ->\n            ok;\n        false ->\n            throw({error, ?F(\"Servers in the same group must share the same tcp\"\n                             \" configuration: ~p\", [SC0#sconf.servername])})\n    end,\n\n    %% If the default servers (the first one) is not an SSL server:\n    %%    all servers  with the same IP/Port must be non-SSL server\n    %% If SNI is disabled or not supported:\n    %%    all servers with the same IP/Port must share the same SSL config\n    %% If SNI is enabled:\n    %%    TLS protocol must be supported by the default servers (the first one)\n    if\n        SC0#sconf.ssl == undefined ->\n            case lists:all(fun(SC) -> SC#sconf.ssl == SC0#sconf.ssl end, SCs) of\n                true  -> ok;\n                false ->\n                    throw({error, ?F(\"All servers in the same group than\"\n                                     \" ~p must have no SSL configuration\",\n                                     [SC0#sconf.servername])})\n            end;\n        GC#gconf.sni == disable ->\n            case lists:all(fun(SC) -> SC#sconf.ssl == SC0#sconf.ssl end, SCs) of\n                true  -> ok;\n                false ->\n                    throw({error, ?F(\"SNI is disabled, all servers in the same\"\n                                     \" group than ~p must share the same ssl\"\n                                     \" configuration\",\n                                     [SC0#sconf.servername])})\n            end;\n\n        true ->\n            Vs = case (SC0#sconf.ssl)#ssl.protocol_version of\n                     undefined -> proplists:get_value(available,ssl:versions());\n                     L         -> L\n                 end,\n            F = fun(V) -> lists:member(V, ['tlsv1.2','tlsv1.1',tlsv1]) end,\n            case lists:any(F, Vs) of\n                true -> ok;\n                false ->\n                    throw({error, ?F(\"SNI is enabled, the server ~p must enable\"\n                                     \" TLS protocol\", [SC0#sconf.servername])})\n            end\n    end,\n\n    %% all servernames in a group must be unique\n    SN = lists:sort([yaws:to_lower(X#sconf.servername) || X <- List]),\n    no_two_same(SN).\n\nno_two_same([H,H|_]) ->\n    throw({error,\n           ?F(\"Two servers in the same group cannot have same name ~p\",[H])});\nno_two_same([_H|T]) ->\n    no_two_same(T);\nno_two_same([]) ->\n    ok.\n\n\n\narrange([C|Tail], start, [], B) ->\n    C1 = set_server(C),\n    arrange(Tail, {in, C1}, [C1], B);\narrange([], _, [], B) ->\n    B;\narrange([], _, A, B) ->\n    [lists:reverse(A) | B];\narrange([C|Tail], {in, C0}, A, B) ->\n    C1 = set_server(C),\n    if\n        C1#sconf.listen == C0#sconf.listen,\n        C1#sconf.port == C0#sconf.port ->\n            arrange(Tail, {in, C0}, [C1|A], B);\n        true ->\n            arrange(Tail, {in, C1}, [C1], [lists:reverse(A)|B])\n    end.\n\n\nset_server(SC) ->\n    SC1 = if\n              SC#sconf.port == 0 ->\n                  {ok, P} = yaws:find_private_port(),\n                  SC#sconf{port=P};\n              true ->\n                  SC\n          end,\n    case {SC1#sconf.ssl, SC1#sconf.port, ?sc_has_add_port(SC1)} of\n        {undefined, 80, _} ->\n            SC1;\n        {undefined, Port, true} ->\n            add_port(SC1, Port);\n        {_SSL, 443, _} ->\n            SC1;\n        {_SSL, Port, true} ->\n            add_port(SC1, Port);\n        {_,_,_} ->\n            SC1\n    end.\n\n\nadd_port(SC, Port) ->\n    case string:tokens(SC#sconf.servername, \":\") of\n        [Srv, Prt] ->\n            case (catch list_to_integer(Prt)) of\n                {'EXIT', _} ->\n                    SC#sconf{servername =\n                                 Srv ++ [$:|integer_to_list(Port)]};\n                _Int ->\n                    SC\n            end;\n        [Srv] ->\n            SC#sconf{servername =   Srv ++ [$:|integer_to_list(Port)]}\n    end.\n\n\nmake_default_gconf(Debug, Id) ->\n    Y = yaws_dir(),\n    Flags = case yaws_sendfile:have_sendfile() of\n                true ->\n                    (?GC_COPY_ERRLOG bor ?GC_FAIL_ON_BIND_ERR bor\n                         ?GC_PICK_FIRST_VIRTHOST_ON_NOMATCH bor\n                         ?GC_USE_YAWS_SENDFILE);\n                false ->\n                    (?GC_COPY_ERRLOG bor ?GC_FAIL_ON_BIND_ERR bor\n                         ?GC_PICK_FIRST_VIRTHOST_ON_NOMATCH)\n            end,\n    #gconf{yaws_dir = Y,\n           ebin_dir = [filename:join([Y, \"examples/ebin\"])],\n           include_dir = [filename:join([Y, \"examples/include\"])],\n           trace = false,\n           logdir = \".\",\n           cache_refresh_secs = if\n                                    Debug == true ->\n                                        0;\n                                    true ->\n                                        30\n                                end,\n           flags = if Debug -> Flags bor ?GC_DEBUG;\n                      true  -> Flags\n                   end,\n\n           yaws = \"Yaws \" ++ yaws_generated:version(),\n           id = Id\n          }.\n\n%% Keep this function for backward compatibility. But no one is supposed to use\n%% it (yaws_config is an internal module, its api is private).\nmake_default_sconf() ->\n    make_default_sconf([], undefined, undefined).\n\nmake_default_sconf([], Servername, Port) ->\n    make_default_sconf(filename:join([yaws_dir(), \"www\"]), Servername, Port);\nmake_default_sconf(DocRoot, undefined, Port) ->\n    make_default_sconf(DocRoot, \"localhost\", Port);\nmake_default_sconf(DocRoot, Servername, undefined) ->\n    make_default_sconf(DocRoot, Servername, 8000);\nmake_default_sconf(DocRoot, Servername, Port) ->\n    AbsDocRoot = filename:absname(DocRoot),\n    case is_dir(AbsDocRoot) of\n        true ->\n            set_server(#sconf{port=Port, servername=Servername,\n                              listen={127,0,0,1},docroot=AbsDocRoot});\n        false ->\n            throw({error, ?F(\"Invalid docroot: directory ~s does not exist\",\n                             [AbsDocRoot])})\n    end.\n\n\nyaws_dir() ->\n    yaws:get_app_dir().\n\nstring_to_host_and_port(String) ->\n    HostPortRE = \"^(?:\\\\[([^\\\\]]+)\\\\]|([^:]+)):([0-9]+)$\",\n    REOptions = [{capture, all_but_first, list}],\n    case re:run(String, HostPortRE, REOptions) of\n        {match, [IPv6, HostOrIPv4, Port]} ->\n            case string:to_integer(Port) of\n                {Integer, []} when Integer >= 0, Integer =< 65535 ->\n                    case IPv6 of\n                        \"\" -> {ok, HostOrIPv4, Integer};\n                        _  -> {ok, IPv6, Integer}\n                    end;\n                _Else ->\n                    {error, ?F(\"~p is not a valid port number\", [Port])}\n            end;\n        nomatch ->\n            {error, ?F(\"bad host and port specifier, expected HOST:PORT; \"\n                \"use [IP]:PORT for IPv6 address\", [])}\n    end.\n\nstring_to_node_mod_fun(String) ->\n    case string:tokens(String, \":\") of\n        [Node, Mod, Fun] ->\n            {ok, list_to_atom(Node), list_to_atom(Mod), list_to_atom(Fun)};\n        [Mod, Fun] ->\n            {ok, list_to_atom(Mod), list_to_atom(Fun)};\n        _ ->\n            {error, ?F(\"bad external module specifier, \"\n                       \"expected NODE:MODULE:FUNCTION or MODULE:FUNCTION\", [])}\n    end.\n\n\n\n%% two states, global, server\nfload(FD, GC) ->\n    case catch fload(FD, GC, [], 1, ?NEXTLINE) of\n        {ok, GC1, Cs} -> {ok, GC1, lists:reverse(Cs)};\n        Err           -> Err\n    end.\n\n\nfload(FD, GC, Cs, _Lno, eof) ->\n    file:close(FD),\n    {ok, GC, Cs};\n\nfload(FD, GC, Cs, Lno, Chars) ->\n    case toks(Lno, Chars) of\n        [] ->\n            fload(FD, GC, Cs, Lno+1, ?NEXTLINE);\n\n        [\"subconfig\", '=', Name] ->\n            case subconfigfiles(FD, Name, Lno) of\n                {ok, Files} ->\n                    case fload_subconfigfiles(Files, global, GC, Cs) of\n                        {ok, GC1, Cs1} ->\n                            fload(FD, GC1, Cs1, Lno+1, ?NEXTLINE);\n                        Err ->\n                            Err\n                    end;\n                Err ->\n                    Err\n            end;\n\n        [\"subconfigdir\", '=', Name] ->\n            case subconfigdir(FD, Name, Lno) of\n                {ok, Files} ->\n                    case fload_subconfigfiles(Files, global, GC, Cs) of\n                        {ok, GC1, Cs1} ->\n                            fload(FD, GC1, Cs1, Lno+1, ?NEXTLINE);\n                        Err ->\n                            Err\n                    end;\n                Err ->\n                    Err\n            end;\n\n        [\"trace\", '=', Bstr] when GC#gconf.trace == false ->\n            case Bstr of\n                \"traffic\" ->\n                    fload(FD, GC#gconf{trace = {true, traffic}}, Cs,\n                          Lno+1, ?NEXTLINE);\n                \"http\" ->\n                    fload(FD, GC#gconf{trace = {true, http}}, Cs,\n                          Lno+1, ?NEXTLINE);\n                \"false\" ->\n                    fload(FD, GC#gconf{trace = false}, Cs, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect false|http|traffic at line ~w\",[Lno])}\n            end;\n        [\"trace\", '=', _Bstr] ->\n            %% don't overwrite setting from commandline\n            fload(FD, GC, Cs, Lno+1, ?NEXTLINE);\n\n\n        [\"logdir\", '=', Logdir] ->\n            Dir = case Logdir of\n                      \"+\" ++ D ->\n                          D1 = filename:absname(D),\n                          %% try to make the log directory if it doesn't exist\n                          yaws:mkdir(D1),\n                          D1;\n                      _ ->\n                          filename:absname(Logdir)\n                  end,\n            case is_dir(Dir) of\n                true ->\n                    put(logdir, Dir),\n                    fload(FD, GC#gconf{logdir = Dir}, Cs, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect directory at line ~w (logdir ~s)\",\n                               [Lno, Dir])}\n            end;\n\n        [\"ebin_dir\", '=', Ebindir] ->\n            Dir = filename:absname(Ebindir),\n            case warn_dir(\"ebin_dir\", Dir) of\n                true ->\n                    fload(FD, GC#gconf{ebin_dir = [Dir|GC#gconf.ebin_dir]}, Cs,\n                          Lno+1, ?NEXTLINE);\n                false ->\n                    fload(FD, GC, Cs, Lno+1, ?NEXTLINE)\n            end;\n\n        [\"src_dir\", '=', Srcdir] ->\n            Dir = filename:absname(Srcdir),\n            case warn_dir(\"src_dir\", Dir) of\n                true ->\n                    fload(FD, GC#gconf{src_dir = [Dir|GC#gconf.src_dir]}, Cs,\n                          Lno+1, ?NEXTLINE);\n                false ->\n                    fload(FD, GC, Cs, Lno+1, ?NEXTLINE)\n            end;\n\n        [\"runmod\", '=', Mod0] ->\n            Mod = list_to_atom(Mod0),\n            fload(FD, GC#gconf{runmods = [Mod|GC#gconf.runmods]}, Cs,\n                  Lno+1, ?NEXTLINE);\n\n        [\"enable_soap\", '=', Bool] ->\n            if (Bool == \"true\") ->\n                    fload(FD, GC#gconf{enable_soap = true}, Cs,\n                          Lno+1, ?NEXTLINE);\n               true ->\n                    fload(FD, GC#gconf{enable_soap = false}, Cs,\n                          Lno+1, ?NEXTLINE)\n            end;\n\n        [\"soap_srv_mods\", '=' | SoapSrvMods] ->\n            case parse_soap_srv_mods(SoapSrvMods, []) of\n                {ok, L} ->\n                    fload(FD, GC#gconf{soap_srv_mods = L}, Cs,\n                          Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"max_connections\", '=', Int] ->\n            case (catch list_to_integer(Int)) of\n                I when is_integer(I) ->\n                    fload(FD, GC#gconf{max_connections = I}, Cs,\n                          Lno+1, ?NEXTLINE);\n                _ when Int == \"nolimit\" ->\n                    fload(FD, GC, Cs, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n        [\"process_options\", '=', POpts] ->\n            case parse_process_options(POpts) of\n                {ok, ProcList} ->\n                    fload(FD, GC#gconf{process_options=ProcList}, Cs,\n                          Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"large_file_chunk_size\", '=', Int] ->\n            case (catch list_to_integer(Int)) of\n                I when is_integer(I) ->\n                    fload(FD, GC#gconf{large_file_chunk_size = I}, Cs,\n                          Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n        [\"large_file_sendfile\", '=', Method] ->\n            case set_sendfile_flags(GC, Method) of\n                {ok, GC1} ->\n                    fload(FD, GC1, Cs, Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"acceptor_pool_size\", '=', Int] ->\n            case catch list_to_integer(Int) of\n                I when is_integer(I), I >= 0 ->\n                    fload(FD, GC#gconf{acceptor_pool_size = I}, Cs,\n                          Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer >= 0 at line ~w\", [Lno])}\n            end;\n\n        [\"log_wrap_size\", '=', Int] ->\n            case (catch list_to_integer(Int)) of\n                I when is_integer(I) ->\n                    fload(FD, GC#gconf{log_wrap_size = I}, Cs,\n                          Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n        [\"log_resolve_hostname\", '=',  Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    fload(FD, ?gc_log_set_resolve_hostname(GC, Val), Cs,\n                          Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"fail_on_bind_err\", '=',  Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    fload(FD, ?gc_set_fail_on_bind_err(GC, Val), Cs,\n                          Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n\n        [\"include_dir\", '=', Incdir] ->\n            Dir = filename:absname(Incdir),\n            case warn_dir(\"include_dir\", Dir) of\n                true ->\n                    fload(FD, GC#gconf{include_dir= [Dir|GC#gconf.include_dir]},\n                          Cs, Lno+1, ?NEXTLINE);\n                false ->\n                    fload(FD, GC, Cs, Lno+1, ?NEXTLINE)\n\n            end;\n\n        [\"mnesia_dir\", '=', Mnesiadir] ->\n            Dir = filename:absname(Mnesiadir),\n            case is_dir(Dir) of\n                true ->\n                    put(mnesiadir, Dir),\n                    fload(FD, GC#gconf{mnesia_dir = Dir}, Cs, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect mnesia directory at line ~w\", [Lno])}\n            end;\n\n        [\"tmpdir\", '=', _TmpDir] ->\n            %% ignore\n            error_logger:format(\n              \"tmpdir in yaws.conf is no longer supported - ignoring\\n\",[]\n             ),\n            fload(FD, GC, Cs, Lno+1, ?NEXTLINE);\n\n        [\"keepalive_timeout\", '=', Val] ->\n            %% keep this bugger for backward compat for a while\n            case (catch list_to_integer(Val)) of\n                I when is_integer(I) ->\n                    fload(FD, GC#gconf{keepalive_timeout = I}, Cs,\n                          Lno+1, ?NEXTLINE);\n                _ when Val == \"infinity\" ->\n                    fload(FD, GC#gconf{keepalive_timeout = infinity}, Cs,\n                          Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n        [\"keepalive_maxuses\", '=', Int] ->\n            case (catch list_to_integer(Int)) of\n                I when is_integer(I) ->\n                    fload(FD, GC#gconf{keepalive_maxuses = I}, Cs,\n                          Lno+1, ?NEXTLINE);\n                _ when Int == \"nolimit\" ->\n                    %% nolimit is the default\n                    fload(FD, GC, Cs, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n        [\"php_exe_path\", '=' , PhpPath] ->\n            error_logger:format(\n              \"'php_exe_path' is deprecated, use 'php_handler' instead\\n\",\n              []),\n            case is_file(PhpPath) of\n                true ->\n                    fload(FD, GC#gconf{phpexe = PhpPath}, Cs, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect executable file at line ~w\", [Lno])}\n            end;\n\n        [\"read_timeout\", '=', _Val] ->\n            %% deprected, don't use\n            error_logger:format(\n              \"read_timeout in yaws.conf is no longer supported - ignoring\\n\",[]\n             ),\n            fload(FD, GC, Cs, Lno+1, ?NEXTLINE);\n\n        [\"max_num_cached_files\", '=', Val] ->\n            case (catch list_to_integer(Val)) of\n                I when is_integer(I) ->\n                    fload(FD, GC#gconf{max_num_cached_files = I}, Cs,\n                          Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n\n        [\"max_num_cached_bytes\", '=', Val] ->\n            case (catch list_to_integer(Val)) of\n                I when is_integer(I) ->\n                    fload(FD, GC#gconf{max_num_cached_bytes = I}, Cs,\n                          Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n\n        [\"max_size_cached_file\", '=', Val] ->\n            case (catch list_to_integer(Val)) of\n                I when is_integer(I) ->\n                    fload(FD, GC#gconf{max_size_cached_file = I}, Cs,\n                          Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n        [\"cache_refresh_secs\", '=', Val] ->\n            case (catch list_to_integer(Val)) of\n                I when is_integer(I), I >= 0 ->\n                    fload(FD, GC#gconf{cache_refresh_secs = I}, Cs,\n                          Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect 0 or positive integer at line ~w\",[Lno])}\n            end;\n\n\n        [\"copy_error_log\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    fload(FD, ?gc_set_copy_errlog(GC, Val), Cs,\n                          Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n\n        [\"auth_log\", '=', Bool] ->\n            error_logger:format(\n              \"'auth_log' global variable is deprecated and ignored.\"\n              \" it is now a per-server variable\", []),\n            case is_bool(Bool) of\n                {true, _Val} ->\n                    fload(FD, GC, Cs, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"id\", '=', String] when GC#gconf.id == undefined;\n                                 GC#gconf.id == \"default\" ->\n            fload(FD, GC#gconf{id=String}, Cs, Lno+1, ?NEXTLINE);\n        [\"id\", '=', String]  ->\n            error_logger:format(\"Ignoring 'id = ~p' setting at line ~p~n\",\n                                [String,Lno]),\n            fload(FD, GC, Cs, Lno+1, ?NEXTLINE);\n\n        [\"pick_first_virthost_on_nomatch\", '=',  Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    fload(FD, ?gc_set_pick_first_virthost_on_nomatch(GC,Val),\n                          Cs, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"use_fdsrv\", '=',  _Bool] ->\n            %% feature removed\n            error_logger:format(\n              \"use_fdsrv in yaws.conf is no longer supported - ignoring\\n\",[]\n             ),\n            fload(FD, GC, Cs, Lno+1, ?NEXTLINE);\n\n        [\"use_old_ssl\", '=',  _Bool] ->\n            %% feature removed\n            error_logger:format(\n              \"use_old_ssl in yaws.conf is no longer supported - ignoring\\n\",[]\n             ),\n            fload(FD, GC, Cs, Lno+1, ?NEXTLINE);\n\n        [\"use_large_ssl_pool\", '=',  _Bool] ->\n            %% just ignore - not relevant any longer\n            error_logger:format(\n              \"use_large_ssl_pool in yaws.conf is no longer supported\"\n              \" - ignoring\\n\", []\n             ),\n            fload(FD, GC, Cs, Lno+1, ?NEXTLINE);\n\n        [\"x_forwarded_for_log_proxy_whitelist\", '=' | _] ->\n            error_logger:format(\n              \"x_forwarded_for_log_proxy_whitelist in yaws.conf is no longer\"\n              \" supported - ignoring\\n\", []\n             ),\n            fload(FD, GC, Cs, Lno+1, ?NEXTLINE);\n\n        [\"ysession_mod\", '=', Mod_str] ->\n            Ysession_mod = list_to_atom(Mod_str),\n            fload(FD, GC#gconf{ysession_mod = Ysession_mod}, Cs,\n                  Lno+1, ?NEXTLINE);\n\n        [\"ysession_idle_timeout\", '=', YsessionIdle] ->\n            case (catch list_to_integer(YsessionIdle)) of\n                I when is_integer(I), I > 0 ->\n                    fload(FD, GC#gconf{ysession_idle_timeout = I}, Cs,\n                          Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect positive integer at line ~w\",[Lno])}\n            end;\n\n        [\"ysession_long_timeout\", '=', YsessionLong] ->\n            case (catch list_to_integer(YsessionLong)) of\n                I when is_integer(I), I > 0 ->\n                    fload(FD, GC#gconf{ysession_long_timeout = I}, Cs,\n                          Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect positive integer at line ~w\",[Lno])}\n            end;\n\n        [\"server_signature\", '=', Signature] ->\n            fload(FD, GC#gconf{yaws=Signature}, Cs, Lno+1, ?NEXTLINE);\n\n        [\"default_type\", '=', MimeType] ->\n            case parse_mime_types_info(default_type, MimeType,\n                                       GC#gconf.mime_types_info,\n                                       #mime_types_info{}) of\n                {ok, Info} ->\n                    fload(FD, GC#gconf{mime_types_info=Info}, Cs,\n                          Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"default_charset\", '=', Charset] ->\n            case parse_mime_types_info(default_charset, Charset,\n                                       GC#gconf.mime_types_info,\n                                       #mime_types_info{}) of\n                {ok, Info} ->\n                    fload(FD, GC#gconf{mime_types_info=Info}, Cs,\n                          Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"mime_types_file\", '=', File] ->\n            case parse_mime_types_info(mime_types_file, File,\n                                       GC#gconf.mime_types_info,\n                                       #mime_types_info{}) of\n                {ok, Info} ->\n                    fload(FD, GC#gconf{mime_types_info=Info}, Cs,\n                          Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"add_types\", '=' | NewTypes] ->\n            case parse_mime_types_info(add_types, NewTypes,\n                                       GC#gconf.mime_types_info,\n                                       #mime_types_info{}) of\n                {ok, Info} ->\n                    fload(FD, GC#gconf{mime_types_info=Info}, Cs,\n                          Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"add_charsets\", '=' | NewCharsets] ->\n            case parse_mime_types_info(add_charsets, NewCharsets,\n                                       GC#gconf.mime_types_info,\n                                       #mime_types_info{}) of\n                {ok, Info} ->\n                    fload(FD, GC#gconf{mime_types_info=Info}, Cs,\n                          Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"nslookup_pref\", '=' | Pref] ->\n            case parse_nslookup_pref(Pref) of\n                {ok, Families} ->\n                    fload(FD, GC#gconf{nslookup_pref = Families}, Cs,\n                          Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"sni\", '=', Sni] ->\n            if\n                Sni == \"disable\" ->\n                    fload(FD, GC#gconf{sni=disable}, Cs, Lno+1, ?NEXTLINE);\n\n                Sni == \"enable\" orelse Sni == \"strict\" ->\n                    case yaws_dynopts:have_ssl_sni() of\n                        true ->\n                            fload(FD, GC#gconf{sni=list_to_atom(Sni)}, Cs, Lno+1,\n                                  ?NEXTLINE);\n                        _ ->\n                            error_logger:info_msg(\"Warning, sni option is not\"\n                                                  \" supported at line ~w~n\", [Lno]),\n                            fload(FD, GC, Cs, Lno+1, ?NEXTLINE)\n                    end;\n                true ->\n                    {error, ?F(\"Expect disable|enable|strict at line ~w\",[Lno])}\n            end;\n\n        ['<', \"server\", Server, '>'] ->\n            C = #sconf{servername = Server, listen = [],\n                       php_handler = {cgi, GC#gconf.phpexe}},\n            fload(FD, server, GC, C, Cs, Lno+1, ?NEXTLINE);\n\n        [H|_] ->\n            {error, ?F(\"Unexpected tokens ~p at line ~w\", [H, Lno])};\n        Err ->\n            Err\n    end.\n\n\nfload(FD, server, _GC, _C, _Cs, Lno, eof) ->\n    file:close(FD),\n    {error, ?F(\"Unexpected end-of-file at line ~w\", [Lno])};\n\nfload(FD, server, GC, C, Cs, Lno, Chars) ->\n    case fload(FD, server, GC, C, Lno, Chars) of\n        {ok, _, _, Lno1, eof} ->\n            {error, ?F(\"Unexpected end-of-file at line ~w\", [Lno1])};\n        {ok, GC1, C1, Lno1, ['<', \"/server\", '>']} ->\n            HasDocroot =\n                case C1#sconf.docroot of\n                    undefined ->\n                        Tests =\n                            [fun() ->\n                                     lists:keymember(\"/\", #proxy_cfg.prefix,\n                                                     C1#sconf.revproxy)\n                             end,\n                             fun() ->\n                                     lists:keymember(\"/\", 1,\n                                                     C1#sconf.redirect_map)\n                             end,\n                             fun() ->\n                                     lists:foldl(fun(_, true) -> true;\n                                                    ({\"/\", _}, _Acc) -> true;\n                                                    (_, Acc) -> Acc\n                                                 end, false, C1#sconf.appmods)\n                             end,\n                             fun() ->\n                                     ?sc_forward_proxy(C1)\n                             end],\n                        lists:any(fun(T) -> T() end, Tests);\n                    _ ->\n                        true\n                end,\n            case HasDocroot of\n                true ->\n                    case C1#sconf.listen of\n                        [] ->\n                            C2 = C1#sconf{listen = {127,0,0,1}},\n                            fload(FD, GC1, [C2|Cs], Lno1+1, ?NEXTLINE);\n                        Ls ->\n                            Cs1 = [C1#sconf{listen=L} || L <- Ls] ++ Cs,\n                            fload(FD, GC1, Cs1, Lno1+1, ?NEXTLINE)\n                    end;\n                false ->\n                    {error,\n                     ?F(\"No valid docroot configured for virthost \"\n                        \"'~s' (port: ~w)\",\n                        [C1#sconf.servername, C1#sconf.port])}\n            end;\n        Err ->\n            Err\n    end.\n\n\nfload(FD, server, GC, C, Lno, eof) ->\n    file:close(FD),\n    {ok, GC, C, Lno, eof};\nfload(FD, _,  _GC, _C, Lno, eof) ->\n    file:close(FD),\n    {error, ?F(\"Unexpected end-of-file at line ~w\", [Lno])};\n\nfload(FD, server, GC, C, Lno, Chars) ->\n    case toks(Lno, Chars) of\n        [] ->\n            fload(FD, server, GC, C, Lno+1, ?NEXTLINE);\n\n        [\"subconfig\", '=', Name] ->\n            case subconfigfiles(FD, Name, Lno) of\n                {ok, Files} ->\n                    case fload_subconfigfiles(Files, server, GC, C) of\n                        {ok, GC1, C1} ->\n                            fload(FD, server, GC1, C1, Lno+1, ?NEXTLINE);\n                        Err ->\n                            Err\n                    end;\n                Err ->\n                    Err\n            end;\n\n        [\"subconfigdir\", '=', Name] ->\n            case subconfigdir(FD, Name, Lno) of\n                {ok, Files} ->\n                    case fload_subconfigfiles(Files, server, GC, C) of\n                        {ok, GC1, C1} ->\n                            fload(FD, server, GC1, C1, Lno+1, ?NEXTLINE);\n                        Err ->\n                            Err\n                    end;\n                Err ->\n                    Err\n            end;\n\n        [\"server_signature\", '=', Sig] ->\n            fload(FD, server, GC, C#sconf{yaws=Sig}, Lno+1, ?NEXTLINE);\n\n        [\"access_log\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = ?sc_set_access_log(C, Val),\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"auth_log\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = ?sc_set_auth_log(C, Val),\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"logger_mod\", '=', Module] ->\n            C1 = C#sconf{logger_mod = list_to_atom(Module)},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"dir_listings\", '=', StrVal] ->\n            case StrVal of\n                \"true\" ->\n                    C1 = ?sc_set_dir_listings(C, true),\n                    C2 = ?sc_set_dir_all_zip(C1, true),\n                    C3 = C2#sconf{appmods = [ {\"all.zip\", yaws_ls},\n                                              {\"all.tgz\", yaws_ls},\n                                              {\"all.tbz2\", yaws_ls}|\n                                              C2#sconf.appmods]},\n                    fload(FD, server, GC, C3, Lno+1, ?NEXTLINE);\n                \"true_nozip\" ->\n                    C1 = ?sc_set_dir_listings(C, true),\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                \"false\" ->\n                    C1 = ?sc_set_dir_listings(C, false),\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect true|true_nozip|false at line ~w\",[Lno])}\n            end;\n\n        [\"deflate\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = C#sconf{deflate_options=#deflate{}},\n                    C2 = ?sc_set_deflate(C1, Val),\n                    fload(FD, server, GC, C2, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"auth_skip_docroot\",'=',Bool] ->\n            case is_bool(Bool) of\n                {true,Val} ->\n                    C1 = ?sc_set_auth_skip_docroot(C, Val),\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"dav\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, true} ->\n                    %% Ever since WebDAV support was moved into an appmod,\n                    %% we must no longer set the dav flag in the\n                    %% sconf. Always turn it off instead.\n                    C1 = ?sc_set_dav(C, false),\n                    Runmods = GC#gconf.runmods,\n                    GC1 = case lists:member(yaws_runmod_lock, Runmods) of\n                              false ->\n                                  GC#gconf{runmods=[yaws_runmod_lock|Runmods]};\n                              true ->\n                                  GC\n                          end,\n                    DavAppmods = lists:keystore(yaws_appmod_dav, 2,\n                                                C1#sconf.appmods,\n                                                {\"/\",yaws_appmod_dav}),\n                    C2 = C1#sconf{appmods=DavAppmods},\n                    fload(FD, server, GC1, C2, Lno+1, ?NEXTLINE);\n                {true,false} ->\n                    C1 = ?sc_set_dav(C, false),\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"port\", '=', Val] ->\n            case (catch list_to_integer(Val)) of\n                I when is_integer(I) ->\n                    C1 = C#sconf{port = I},\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n        [\"rmethod\", '=', Val] ->\n            case Val of\n                \"http\" ->\n                    C1 = C#sconf{rmethod = Val},\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                \"https\" ->\n                    C1 = C#sconf{rmethod = Val},\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect http or https at line ~w\", [Lno])}\n            end;\n\n        [\"rhost\", '=', Val] ->\n            C1 = C#sconf{rhost = Val},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"listen\", '=', IP] ->\n            case inet_parse:address(IP) of\n                {error, _} ->\n                    {error, ?F(\"Expect IP address at line ~w:\", [Lno])};\n                {ok,Addr} ->\n                    Lstn = C#sconf.listen,\n                    C1 = if\n                             is_list(Lstn) ->\n                                 case lists:member(Addr, Lstn) of\n                                     false ->\n                                         C#sconf{listen = [Addr|Lstn]};\n                                     true ->\n                                         C\n                                 end;\n                             true ->\n                                 C#sconf{listen = [Addr, Lstn]}\n                         end,\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE)\n            end;\n\n        [\"listen_backlog\", '=', Val] ->\n            case (catch list_to_integer(Val)) of\n                B when is_integer(B) ->\n                    C1 = update_soptions(C, listen_opts, backlog, B),\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n        [\"servername\", '=', Name] ->\n            C1 = ?sc_set_add_port((C#sconf{servername = Name}),false),\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"serveralias\", '=' | Names] ->\n            C1 = C#sconf{serveralias = Names ++ C#sconf.serveralias},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [ '<', \"listen_opts\", '>'] ->\n            fload(FD, listen_opts, GC, C, Lno+1, ?NEXTLINE);\n\n        [\"docroot\", '=', Rootdir | XtraDirs] ->\n            RootDirs = lists:map(fun(R) -> filename:absname(R) end,\n                                 [Rootdir | XtraDirs]),\n            case lists:filter(fun(R) -> not is_dir(R) end, RootDirs) of\n                [] when C#sconf.docroot =:= undefined ->\n                    C1 = C#sconf{docroot = hd(RootDirs),\n                                 xtra_docroots = tl(RootDirs)},\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                [] ->\n                    XtraDocroots = RootDirs ++ C#sconf.xtra_docroots,\n                    C1 = C#sconf{xtra_docroots = XtraDocroots},\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                NoDirs ->\n                    error_logger:info_msg(\"Warning, Skip invalid docroots\"\n                                          \" at line ~w : ~s~n\",\n                                          [Lno, string:join(NoDirs, \", \")]),\n                    case lists:subtract(RootDirs, NoDirs) of\n                        [] ->\n                            fload(FD, server, GC, C, Lno+1, ?NEXTLINE);\n                        [H|T] when C#sconf.docroot =:= undefined ->\n                            C1 = C#sconf{docroot = H, xtra_docroots = T},\n                            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                        Ds ->\n                            XtraDocroots = Ds ++ C#sconf.xtra_docroots,\n                            C1 = C#sconf{xtra_docroots = XtraDocroots},\n                            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE)\n                    end\n            end;\n\n        [\"partial_post_size\",'=',Size] ->\n            case Size of\n                \"nolimit\" ->\n                    C1 = C#sconf{partial_post_size = nolimit},\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                Val ->\n                    case (catch list_to_integer(Val)) of\n                        I when is_integer(I) ->\n                            C1 = C#sconf{partial_post_size = I},\n                            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                        _ ->\n                            {error,\n                             ?F(\"Expect integer or 'nolimit' at line ~w\",\n                                [Lno])}\n                    end\n            end;\n\n        ['<', \"auth\", '>'] ->\n            C1 = C#sconf{authdirs=[#auth{}|C#sconf.authdirs]},\n            fload(FD, server_auth, GC, C1, Lno+1, ?NEXTLINE);\n\n        ['<', \"redirect\", '>'] ->\n            fload(FD, server_redirect, GC, C, Lno+1, ?NEXTLINE);\n\n        ['<', \"deflate\", '>'] ->\n            C1 = C#sconf{deflate_options=#deflate{mime_types=[]}},\n            fload(FD, server_deflate, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"default_server_on_this_ip\", '=', _Bool] ->\n            error_logger:format(\n              \"default_server_on_this_ip in yaws.conf is no longer\"\n              \" supported - ignoring\\n\", []\n             ),\n            fload(FD, server, GC, C, Lno+1, ?NEXTLINE);\n\n        [ '<', \"ssl\", '>'] ->\n            ssl_start(),\n            fload(FD, ssl, GC, C#sconf{ssl = #ssl{}}, Lno+1, ?NEXTLINE);\n\n        [\"appmods\", '=' | AppMods] ->\n            case parse_appmods(AppMods, []) of\n                {ok, L} ->\n                    C1 = C#sconf{appmods = L ++ C#sconf.appmods},\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"dispatchmod\", '=', DispatchMod] ->\n            C1 = C#sconf{dispatch_mod = list_to_atom(DispatchMod)},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"expires\", '=' | Expires] ->\n            case parse_expires(Expires, []) of\n                {ok, L} ->\n                    C1 = C#sconf{expires = L ++ C#sconf.expires},\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"errormod_404\", '=' , Module] ->\n            C1 = C#sconf{errormod_404 = list_to_atom(Module)},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"errormod_crash\", '=', Module] ->\n            C1 = C#sconf{errormod_crash = list_to_atom(Module)},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"errormod_401\", '=' , Module] ->\n            C1 = C#sconf{errormod_401 = list_to_atom(Module)},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"arg_rewrite_mod\", '=', Module] ->\n            C1 = C#sconf{arg_rewrite_mod = list_to_atom(Module)},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"tilde_expand\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = ?sc_set_tilde_expand(C,Val),\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        ['<', \"opaque\", '>'] ->\n            fload(FD, opaque, GC, C, Lno+1, ?NEXTLINE);\n\n        [\"start_mod\", '=' , Module] ->\n            C1 = C#sconf{start_mod = list_to_atom(Module)},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        ['<', \"rss\", '>'] ->\n            erase(rss_id),\n            put(rss, []),\n            fload(FD, rss, GC, C, Lno+1, ?NEXTLINE);\n\n        [\"tilde_allowed_scripts\", '=' | Suffixes] ->\n            C1 = C#sconf{tilde_allowed_scripts=Suffixes},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"allowed_scripts\", '=' | Suffixes] ->\n            C1 = C#sconf{allowed_scripts=Suffixes},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"index_files\", '=' | Files] ->\n            case parse_index_files(Files) of\n                ok ->\n                    C1 = C#sconf{index_files = Files},\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"revproxy\", '=' | Tail] ->\n            case parse_revproxy(Tail) of\n                {ok, RevProxy} ->\n                    C1 = C#sconf{revproxy = [RevProxy | C#sconf.revproxy]},\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                {error, url} ->\n                    {error, ?F(\"Bad url at line ~p\",[Lno])};\n                {error, syntax} ->\n                    {error, ?F(\"Bad revproxy syntax at line ~p\",[Lno])};\n                Error ->\n                    Error\n            end;\n\n        [\"fwdproxy\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = ?sc_set_forward_proxy(C, Val),\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        ['<', \"extra_cgi_vars\", \"dir\", '=', Dir, '>'] ->\n            C1 = C#sconf{extra_cgi_vars=[{Dir, []}|C#sconf.extra_cgi_vars]},\n            fload(FD, extra_cgi_vars, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"statistics\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = ?sc_set_statistics(C, Val),\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"fcgi_app_server\", '=' | Val] ->\n            HostPortSpec = case Val of\n                [HPS]                    -> HPS;\n                ['[', HSpec, ']', PSpec] -> \"[\" ++ HSpec ++ \"]\" ++ PSpec\n            end,\n            case string_to_host_and_port(HostPortSpec) of\n                {ok, Host, Port} ->\n                    C1 = C#sconf{fcgi_app_server = {Host, Port}},\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                {error, Reason} ->\n                    {error, ?F(\"Invalid fcgi_app_server ~p at line ~w: ~s\",\n                               [HostPortSpec, Lno, Reason])}\n            end;\n\n        [\"fcgi_trace_protocol\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = ?sc_set_fcgi_trace_protocol(C, Val),\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"fcgi_log_app_error\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = ?sc_set_fcgi_log_app_error(C, Val),\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"phpfcgi\", '=', HostPortSpec] ->\n            error_logger:format(\n              \"'phpfcgi' is deprecated, use 'php_handler' instead\\n\", []),\n            case string_to_host_and_port(HostPortSpec) of\n                {ok, Host, Port} ->\n                    C1 = C#sconf{php_handler = {fcgi, {Host, Port}}},\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                {error, Reason} ->\n                    {error,\n                     ?F(\"Invalid php fcgi server ~p at line ~w: ~s\",\n                        [HostPortSpec, Lno, Reason])}\n            end;\n\n        [\"php_handler\", '=' | PhpMod] ->\n            case parse_phpmod(PhpMod, GC#gconf.phpexe) of\n                {ok, I} ->\n                    C1 = C#sconf{php_handler = I},\n                    fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n                {error, Reason} ->\n                    {error,\n                     ?F(\"Invalide php_handler configuration at line ~w: ~s\",\n                        [Lno, Reason])}\n            end;\n\n        [\"shaper\", '=', Module] ->\n            C1 = C#sconf{shaper = list_to_atom(Module)},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n\n        [\"default_type\", '=', MimeType] ->\n            case parse_mime_types_info(default_type, MimeType,\n                                       C#sconf.mime_types_info,\n                                       GC#gconf.mime_types_info) of\n                {ok, Info} ->\n                    fload(FD, server, GC, C#sconf{mime_types_info=Info},\n                          Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"default_charset\", '=', Charset] ->\n            case parse_mime_types_info(default_charset, Charset,\n                                       C#sconf.mime_types_info,\n                                       GC#gconf.mime_types_info) of\n                {ok, Info} ->\n                    fload(FD, server, GC, C#sconf{mime_types_info=Info},\n                          Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"mime_types_file\", '=', File] ->\n            case parse_mime_types_info(mime_types_file, File,\n                                       C#sconf.mime_types_info,\n                                       GC#gconf.mime_types_info) of\n                {ok, Info} ->\n                    fload(FD, server, GC, C#sconf{mime_types_info=Info},\n                          Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"add_types\", '=' | NewTypes] ->\n            case parse_mime_types_info(add_types, NewTypes,\n                                       C#sconf.mime_types_info,\n                                       GC#gconf.mime_types_info) of\n                {ok, Info} ->\n                    fload(FD, server, GC, C#sconf{mime_types_info=Info},\n                          Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"add_charsets\", '=' | NewCharsets] ->\n            case parse_mime_types_info(add_charsets, NewCharsets,\n                                       C#sconf.mime_types_info,\n                                       GC#gconf.mime_types_info) of\n                {ok, Info} ->\n                    fload(FD, server, GC, C#sconf{mime_types_info=Info},\n                          Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        ['<', \"/server\", '>'] ->\n            {ok, GC, C, Lno, ['<', \"/server\", '>']};\n\n        [H|T] ->\n            {error, ?F(\"Unexpected input ~p at line ~w\", [[H|T], Lno])};\n        Err ->\n            Err\n    end;\n\n\nfload(FD, listen_opts, GC, C, Lno, Chars) ->\n    case toks(Lno, Chars) of\n        [] ->\n            fload(FD, listen_opts, GC, C, Lno+1, ?NEXTLINE);\n\n        [\"buffer\", '=', Int] ->\n            case (catch list_to_integer(Int)) of\n                B when is_integer(B) ->\n                    C1 = update_soptions(C, listen_opts, buffer, B),\n                    fload(FD, listen_opts, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n        [\"delay_send\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = update_soptions(C, listen_opts, delay_send, Val),\n                    fload(FD, listen_opts, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"linger\", '=', Val] ->\n            case (catch list_to_integer(Val)) of\n                I when is_integer(I) ->\n                    C1 = update_soptions(C, listen_opts, linger, {true, I}),\n                    fload(FD, listen_opts, GC, C1, Lno+1, ?NEXTLINE);\n                _ when Val == \"false\" ->\n                    C1 = update_soptions(C, listen_opts, linger, {false, 0}),\n                    fload(FD, listen_opts, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer|false at line ~w\", [Lno])}\n            end;\n\n        [\"nodelay\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = update_soptions(C, listen_opts, nodelay, Val),\n                    fload(FD, listen_opts, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"priority\", '=', Int] ->\n            case (catch list_to_integer(Int)) of\n                P when is_integer(P) ->\n                    C1 = update_soptions(C, listen_opts, priority, P),\n                    fload(FD, listen_opts, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n        [\"sndbuf\", '=', Int] ->\n            case (catch list_to_integer(Int)) of\n                I when is_integer(I) ->\n                    C1 = update_soptions(C, listen_opts, sndbuf, I),\n                    fload(FD, listen_opts, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n        [\"recbuf\", '=', Int] ->\n            case (catch list_to_integer(Int)) of\n                I when is_integer(I) ->\n                    C1 = update_soptions(C, listen_opts, recbuf, I),\n                    fload(FD, listen_opts, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer at line ~w\", [Lno])}\n            end;\n\n        [\"send_timeout\", '=', Val] ->\n            case (catch list_to_integer(Val)) of\n                I when is_integer(I) ->\n                    C1 = update_soptions(C, listen_opts, send_timeout, I),\n                    fload(FD, listen_opts, GC, C1, Lno+1, ?NEXTLINE);\n                _ when Val == \"infinity\" ->\n                    C1 = update_soptions(C, listen_opts, send_timeout,\n                                         infinity),\n                    fload(FD, listen_opts, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer|infinity at line ~w\", [Lno])}\n            end;\n\n        [\"send_timeout_close\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = update_soptions(C, listen_opts, send_timeout_close,\n                                         Val),\n                    fload(FD, listen_opts, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        ['<', \"/listen_opts\", '>'] ->\n            fload(FD, server, GC, C, Lno+1, ?NEXTLINE);\n\n        [H|T] ->\n            {error, ?F(\"Unexpected input ~p at line ~w\", [[H|T], Lno])};\n        Err ->\n            Err\n    end;\n\nfload(FD, ssl, GC, C, Lno, Chars) ->\n    case toks(Lno, Chars) of\n        [] ->\n            fload(FD, ssl, GC, C, Lno+1, ?NEXTLINE);\n\n        %% A bunch of ssl options\n\n        [\"keyfile\", '=', Val] ->\n            case is_file(Val) of\n                true ->\n                    C1 = C#sconf{ssl = (C#sconf.ssl)#ssl{keyfile = Val}},\n                    fload(FD, ssl, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect existing file at line ~w\", [Lno])}\n            end;\n\n        [\"certfile\", '=', Val] ->\n            case is_file(Val) of\n                true ->\n                    C1 = C#sconf{ssl = (C#sconf.ssl)#ssl{certfile = Val}},\n                    fload(FD, ssl, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect existing file at line ~w\", [Lno])}\n            end;\n\n        [\"cacertfile\", '=', Val] ->\n            case is_file(Val) of\n                true ->\n                    C1 = C#sconf{ssl = (C#sconf.ssl)#ssl{cacertfile = Val}},\n                    fload(FD, ssl, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect existing file at line ~w\", [Lno])}\n            end;\n\n        [\"dhfile\", '=', Val] ->\n            case is_file(Val) of\n                true ->\n                    C1 = C#sconf{ssl = (C#sconf.ssl)#ssl{dhfile = Val}},\n                    fload(FD, ssl, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect existing file at line ~w\", [Lno])}\n            end;\n\n        [\"verify\", '=', Val0] ->\n            Fail0 = (C#sconf.ssl)#ssl.fail_if_no_peer_cert,\n            {Val, Fail} = try\n                              case list_to_integer(Val0) of\n                                  0 -> {verify_none, Fail0};\n                                  1 -> {verify_peer, false};\n                                  2 -> {verify_peer, true};\n                                  _ -> {error, Fail0}\n                              end\n                          catch error:badarg ->\n                                  case list_to_atom(Val0) of\n                                      verify_none -> {verify_none, Fail0};\n                                      verify_peer -> {verify_peer, Fail0};\n                                      _           -> {error, Fail0}\n                                  end\n                          end,\n            case Val of\n                error ->\n                    {error, ?F(\"Expect integer or verify_none, \"\n                               \"verify_peer at line ~w\", [Lno])};\n                _ ->\n                    SSL = (C#sconf.ssl)#ssl{verify=Val,\n                                            fail_if_no_peer_cert=Fail},\n                    C1 = C#sconf{ssl=SSL},\n                    fload(FD, ssl, GC, C1, Lno+1, ?NEXTLINE)\n            end;\n\n        [\"fail_if_no_peer_cert\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = C#sconf{ssl = (C#sconf.ssl)#ssl{\n                                         fail_if_no_peer_cert = Val}},\n                    fload(FD, ssl, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"depth\", '=', Val0] ->\n            Val = (catch list_to_integer(Val0)),\n            case lists:member(Val, [0, 1,2,3,4,5,6,7]) of\n                true ->\n                    C1 = C#sconf{ssl = (C#sconf.ssl)#ssl{depth = Val}},\n                    fload(FD, ssl, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer 0..7 at line ~w\", [Lno])}\n            end;\n\n        [\"password\", '=', Val] ->\n            C1 = C#sconf{ssl = (C#sconf.ssl)#ssl{password = Val}},\n            fload(FD, ssl, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"ciphers\", '=', Val] ->\n            try\n                L = str2term(Val),\n                Ciphers = ssl:cipher_suites(),\n                case check_ciphers(L, Ciphers) of\n                    ok ->\n                        C1 = C#sconf{ssl = (C#sconf.ssl)#ssl{ciphers = L}},\n                        fload(FD, ssl, GC, C1, Lno+1, ?NEXTLINE);\n                    Err ->\n                        Err\n                end\n            catch _:_ ->\n                    {error, ?F(\"Bad cipherspec at line ~w\", [Lno])}\n            end;\n\n        [\"secure_renegotiate\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = C#sconf{ssl=(C#sconf.ssl)#ssl{secure_renegotiate=Val}},\n                    fload(FD, ssl, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        [\"client_renegotiation\", '=', Bool] ->\n            case yaws_dynopts:have_ssl_client_renegotiation() of\n                true ->\n                    case is_bool(Bool) of\n                        {true, Val} ->\n                            C1 = C#sconf{ssl=(C#sconf.ssl)#ssl{client_renegotiation=Val}},\n                            fload(FD, ssl, GC, C1, Lno+1, ?NEXTLINE);\n                        false ->\n                            {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n                    end;\n                _ ->\n                    error_logger:info_msg(\"Warning, client_renegotiation SSL \"\n                                          \"option is not supported \"\n                                          \"at line ~w~n\", [Lno]),\n                    fload(FD, ssl, GC, C, Lno+1, ?NEXTLINE)\n            end;\n\n        [\"honor_cipher_order\", '=', Bool] ->\n            case yaws_dynopts:have_ssl_honor_cipher_order() of\n                true ->\n                    case is_bool(Bool) of\n                        {true, Val} ->\n                            C2 = C#sconf{\n                                   ssl=(C#sconf.ssl)#ssl{honor_cipher_order=Val}\n                                  },\n                            fload(FD, ssl, GC, C2, Lno+1, ?NEXTLINE);\n                        false ->\n                            {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n                    end;\n                _ ->\n                    error_logger:info_msg(\"Warning, honor_cipher_order SSL \"\n                                          \"option is not supported \"\n                                          \"at line ~w~n\", [Lno]),\n                    fload(FD, ssl, GC, C, Lno+1, ?NEXTLINE)\n            end;\n\n        [\"protocol_version\", '=' | Vsns0] ->\n            try\n                Vsns = [list_to_existing_atom(V) || V <- Vsns0, not is_atom(V)],\n                C1 = C#sconf{\n                       ssl=(C#sconf.ssl)#ssl{protocol_version=Vsns}\n                      },\n                fload(FD, ssl, GC, C1, Lno+1, ?NEXTLINE)\n            catch _:_ ->\n                    {error, ?F(\"Bad ssl protocol_version at line ~w\", [Lno])}\n            end;\n\n        [\"require_sni\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    C1 = C#sconf{\n                           ssl=(C#sconf.ssl)#ssl{require_sni=Val}\n                          },\n                    fload(FD, ssl, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        ['<', \"/ssl\", '>'] ->\n            fload(FD, server, GC, C, Lno+1, ?NEXTLINE);\n\n        [H|T] ->\n            {error, ?F(\"Unexpected input ~p at line ~w\", [[H|T], Lno])};\n        Err ->\n            Err\n    end;\n\nfload(FD, server_auth, GC, C, Lno, Chars) ->\n    [Auth|AuthDirs] = C#sconf.authdirs,\n    case toks(Lno, Chars) of\n        [] ->\n            fload(FD, server_auth, GC, C, Lno+1, ?NEXTLINE);\n\n        [\"docroot\", '=', Docroot] ->\n            Auth1 = Auth#auth{docroot = filename:absname(Docroot)},\n            C1 = C#sconf{authdirs=[Auth1|AuthDirs]},\n            fload(FD, server_auth, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"dir\", '=', Dir] ->\n            case file:list_dir(Dir) of\n                {ok,_} when Dir /= \"/\" ->\n                    error_logger:info_msg(\"Warning, authdir must be set \"\n                                          \"relative docroot ~n\",[]);\n                _ ->\n                    ok\n            end,\n            Dir1 = yaws_api:path_norm(Dir),\n            Auth1 = Auth#auth{dir = [Dir1 | Auth#auth.dir]},\n            C1 = C#sconf{authdirs=[Auth1|AuthDirs]},\n            fload(FD, server_auth, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"realm\", '=', Realm] ->\n            Auth1 = Auth#auth{realm = Realm},\n            C1 = C#sconf{authdirs=[Auth1|AuthDirs]},\n            fload(FD, server_auth, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"authmod\", '=', Mod] ->\n            Mod1 = list_to_atom(Mod),\n            code:ensure_loaded(Mod1),\n            %% Add the auth header for the mod\n            H = try\n                    Mod1:get_header() ++ Auth#auth.headers\n                catch _:_ ->\n                        error_logger:format(\"Failed to ~p:get_header() \\n\",\n                                            [Mod1]),\n                        Auth#auth.headers\n                end,\n            Auth1 = Auth#auth{mod = Mod1, headers = H},\n            C1 = C#sconf{authdirs=[Auth1|AuthDirs]},\n            fload(FD, server_auth, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"user\", '=', User] ->\n            case parse_auth_user(User, Lno) of\n                {Name, Algo, Salt, Hash} ->\n                    Auth1 = Auth#auth{\n                              users = [{Name, Algo, Salt, Hash}|Auth#auth.users]\n                             },\n                    C1 = C#sconf{authdirs=[Auth1|AuthDirs]},\n                    fload(FD, server_auth, GC, C1, Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, Str}\n            end;\n\n        [\"allow\", '=', \"all\"] ->\n            Auth1 = case Auth#auth.acl of\n                        none    -> Auth#auth{acl={all, [], deny_allow}};\n                        {_,D,O} -> Auth#auth{acl={all, D, O}}\n                    end,\n            C1 = C#sconf{authdirs=[Auth1|AuthDirs]},\n            fload(FD, server_auth, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"allow\", '=' | IPs] ->\n            Auth1 = case Auth#auth.acl of\n                        none ->\n                            AllowIPs = parse_auth_ips(IPs, []),\n                            Auth#auth{acl={AllowIPs, [], deny_allow}};\n                        {all, _, _} ->\n                            Auth;\n                        {AllowIPs, DenyIPs, Order} ->\n                            AllowIPs1 = parse_auth_ips(IPs, []) ++ AllowIPs,\n                            Auth#auth{acl={AllowIPs1, DenyIPs, Order}}\n                    end,\n            C1 = C#sconf{authdirs=[Auth1|AuthDirs]},\n            fload(FD, server_auth, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"deny\", '=', \"all\"] ->\n            Auth1 = case Auth#auth.acl of\n                        none    -> Auth#auth{acl={[], all, deny_allow}};\n                        {A,_,O} -> Auth#auth{acl={A, all, O}}\n                    end,\n            C1 = C#sconf{authdirs=[Auth1|AuthDirs]},\n            fload(FD, server_auth, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"deny\", '=' | IPs] ->\n            Auth1 = case Auth#auth.acl of\n                        none ->\n                            DenyIPs = parse_auth_ips(IPs, []),\n                            Auth#auth{acl={[], DenyIPs, deny_allow}};\n                        {_, all, _} ->\n                            Auth;\n                        {AllowIPs, DenyIPs, Order} ->\n                            DenyIPs1 = parse_auth_ips(IPs, []) ++ DenyIPs,\n                            Auth#auth{acl={AllowIPs, DenyIPs1, Order}}\n                    end,\n            C1 = C#sconf{authdirs=[Auth1|AuthDirs]},\n            fload(FD, server_auth, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"order\", '=', \"allow\", ',', \"deny\"] ->\n            Auth1 = case Auth#auth.acl of\n                        none    -> Auth#auth{acl={[], [], allow_deny}};\n                        {A,D,_} -> Auth#auth{acl={A, D, allow_deny}}\n                    end,\n            C1 = C#sconf{authdirs=[Auth1|AuthDirs]},\n            fload(FD, server_auth, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"order\", '=', \"deny\", ',', \"allow\"] ->\n            Auth1 = case Auth#auth.acl of\n                        none    -> Auth#auth{acl={[], [], deny_allow}};\n                        {A,D,_} -> Auth#auth{acl={A, D, deny_allow}}\n                    end,\n            C1 = C#sconf{authdirs=[Auth1|AuthDirs]},\n            fload(FD, server_auth, GC, C1, Lno+1, ?NEXTLINE);\n\n        [\"pam\", \"service\", '=', Serv] ->\n            Auth1 = Auth#auth{pam=Serv},\n            C1 = C#sconf{authdirs=[Auth1|AuthDirs]},\n            fload(FD, server_auth, GC, C1, Lno+1, ?NEXTLINE);\n\n        ['<', \"/auth\", '>'] ->\n            Pam = Auth#auth.pam,\n            Users = Auth#auth.users,\n            Realm = Auth#auth.realm,\n            Auth1 =  case {Pam, Users} of\n                         {false, []} ->\n                             Auth;\n                         _ ->\n                             H = Auth#auth.headers ++\n                                 yaws:make_www_authenticate_header({realm, Realm}),\n                             Auth#auth{headers = H}\n                     end,\n            AuthDirs1 = case Auth1#auth.dir of\n                            [] -> [Auth1#auth{dir=\"/\"}|AuthDirs];\n                            Ds -> [Auth1#auth{dir=D} || D <- Ds] ++ AuthDirs\n                        end,\n            C1 = C#sconf{authdirs=AuthDirs1},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [H|T] ->\n            {error, ?F(\"Unexpected input ~p at line ~w\", [[H|T], Lno])};\n        Err ->\n            Err\n    end;\n\nfload(FD, server_redirect, GC, C, Lno, Chars) ->\n    RedirMap = C#sconf.redirect_map,\n    case toks(Lno, Chars) of\n        [] ->\n            fload(FD, server_redirect, GC, C, Lno+1, ?NEXTLINE);\n\n        [Path, '=', '=' | Rest] ->\n            %% \"Normalize\" Path\n            Path1 = filename:join([yaws_api:path_norm(Path)]),\n            case parse_redirect(Path1, Rest, noappend, Lno) of\n                {error, Str} ->\n                    {error, Str};\n                Redir ->\n                    C1 = C#sconf{redirect_map=RedirMap ++ [Redir]},\n                    fload(FD, server_redirect, GC, C1, Lno+1, ?NEXTLINE)\n            end;\n\n        [Path, '=' | Rest] ->\n            %% \"Normalize\" Path\n            Path1 = filename:join([yaws_api:path_norm(Path)]),\n            case parse_redirect(Path1, Rest, append, Lno) of\n                {error, Str} ->\n                    {error, Str};\n                Redir ->\n                    C1 = C#sconf{redirect_map=RedirMap ++ [Redir]},\n                    fload(FD, server_redirect, GC, C1, Lno+1, ?NEXTLINE)\n            end;\n\n        ['<', \"/redirect\", '>'] ->\n            fload(FD, server, GC, C, Lno+1, ?NEXTLINE);\n\n        [H|T] ->\n            {error, ?F(\"Unexpected input ~p at line ~w\", [[H|T], Lno])};\n        Err ->\n            Err\n    end;\n\nfload(FD, server_deflate, GC, C, Lno, Chars) ->\n    Deflate = C#sconf.deflate_options,\n    case toks(Lno, Chars) of\n        [] ->\n            fload(FD, server_deflate, GC, C, Lno+1, ?NEXTLINE);\n\n        [\"min_compress_size\", '=', CSize] ->\n            case (catch list_to_integer(CSize)) of\n                I when is_integer(I), I > 0 ->\n                    Deflate1 = Deflate#deflate{min_compress_size=I},\n                    C1 = C#sconf{deflate_options=Deflate1},\n                    fload(FD, server_deflate, GC, C1, Lno+1, ?NEXTLINE);\n                _ when CSize == \"nolimit\" ->\n                    Deflate1 = Deflate#deflate{min_compress_size=nolimit},\n                    C1 = C#sconf{deflate_options=Deflate1},\n                    fload(FD, server_deflate, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer > 0 at line ~w\", [Lno])}\n            end;\n\n        [\"mime_types\", '=' | MimeTypes] ->\n            case parse_compressible_mime_types(MimeTypes,\n                                               Deflate#deflate.mime_types) of\n                {ok, L} ->\n                    Deflate1 = Deflate#deflate{mime_types=L},\n                    C1 = C#sconf{deflate_options=Deflate1},\n                    fload(FD, server_deflate, GC, C1, Lno+1, ?NEXTLINE);\n                {error, Str} ->\n                    {error, ?F(\"~s at line ~w\", [Str, Lno])}\n            end;\n\n        [\"compression_level\", '=', CLevel] ->\n            L = try\n                    list_to_integer(CLevel)\n                catch error:badarg ->\n                        list_to_atom(CLevel)\n                end,\n            if\n                L =:= none; L =:= default;\n                L =:= best_compression; L =:= best_speed ->\n                    Deflate1 = Deflate#deflate{compression_level=L},\n                    C1 = C#sconf{deflate_options=Deflate1},\n                    fload(FD, server_deflate, GC, C1, Lno+1, ?NEXTLINE);\n                is_integer(L), L >= 0, L =< 9 ->\n                    Deflate1 = Deflate#deflate{compression_level=L},\n                    C1 = C#sconf{deflate_options=Deflate1},\n                    fload(FD, server_deflate, GC, C1, Lno+1, ?NEXTLINE);\n                true ->\n                    {error, ?F(\"Bad compression level at line ~w\", [Lno])}\n            end;\n\n        [\"window_size\", '=', WSize] ->\n            case (catch list_to_integer(WSize)) of\n                I when is_integer(I), I > 8, I < 16 ->\n                    Deflate1 = Deflate#deflate{window_size=I * -1},\n                    C1 = C#sconf{deflate_options=Deflate1},\n                    fload(FD, server_deflate, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error,\n                     ?F(\"Expect integer between 9..15 at line ~w\",\n                        [Lno])}\n            end;\n\n        [\"mem_level\", '=', MLevel] ->\n            case (catch list_to_integer(MLevel)) of\n                I when is_integer(I), I >= 1, I =< 9 ->\n                    Deflate1 = Deflate#deflate{mem_level=I},\n                    C1 = C#sconf{deflate_options=Deflate1},\n                    fload(FD, server_deflate, GC, C1, Lno+1, ?NEXTLINE);\n                _ ->\n                    {error, ?F(\"Expect integer between 1..9 at line ~w\", [Lno])}\n            end;\n\n        [\"strategy\", '=', Strategy] ->\n            if\n                Strategy =:= \"default\";\n                Strategy =:= \"filtered\";\n                Strategy =:= \"huffman_only\" ->\n                    Deflate1 = Deflate#deflate{strategy=list_to_atom(Strategy)},\n                    C1 = C#sconf{deflate_options=Deflate1},\n                    fload(FD, server_deflate, GC, C1, Lno+1, ?NEXTLINE);\n                true ->\n                    {error,\n                     ?F(\"Unknown strategy ~p at line ~w\", [Strategy, Lno])}\n            end;\n\n        [\"use_gzip_static\", '=', Bool] ->\n            case is_bool(Bool) of\n                {true, Val} ->\n                    Deflate1 = Deflate#deflate{use_gzip_static=Val},\n                    C1 = C#sconf{deflate_options=Deflate1},\n                    fload(FD, server_deflate, GC, C1, Lno+1, ?NEXTLINE);\n                false ->\n                    {error, ?F(\"Expect true|false at line ~w\", [Lno])}\n            end;\n\n        ['<', \"/deflate\", '>'] ->\n            Deflate1 = case Deflate#deflate.mime_types of\n                           [] ->\n                               Deflate#deflate{\n                                 mime_types = ?DEFAULT_COMPRESSIBLE_MIME_TYPES\n                                };\n                           _ ->\n                               Deflate\n                       end,\n            C1 = C#sconf{deflate_options = Deflate1},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [H|T] ->\n            {error, ?F(\"Unexpected input ~p at line ~w\", [[H|T], Lno])};\n        Err ->\n            Err\n    end;\n\nfload(FD, extra_cgi_vars, GC, C, Lno, Chars) ->\n    [{Dir, Vars}|EVars] = C#sconf.extra_cgi_vars,\n    case toks(Lno, Chars) of\n        [] ->\n            fload(FD, extra_cgi_vars, GC, C, Lno+1, ?NEXTLINE);\n\n        [Var, '=', Val] ->\n            C1 = C#sconf{extra_cgi_vars=[{Dir, [{Var, Val} | Vars]}|EVars]},\n            fload(FD, extra_cgi_vars, GC, C1, Lno+1, ?NEXTLINE);\n\n        ['<', \"/extra_cgi_vars\", '>'] ->\n            C1 = C#sconf{extra_cgi_vars = [EVars | C#sconf.extra_cgi_vars]},\n            fload(FD, server, GC, C1, Lno+1, ?NEXTLINE);\n\n        [H|T] ->\n            {error, ?F(\"Unexpected input ~p at line ~w\", [[H|T], Lno])};\n        Err ->\n            Err\n    end;\n\nfload(FD, rss, GC, C, Lno, Chars) ->\n    case toks(Lno, Chars) of\n        [] ->\n            fload(FD, rss, GC, C, Lno+1, ?NEXTLINE);\n\n        [\"rss_id\", '=', Value] ->   % mandatory !!\n            put(rss_id, list_to_atom(Value)),\n            fload(FD, rss, GC, C, Lno+1, ?NEXTLINE);\n\n        [\"rss_dir\", '=', Value] ->   % mandatory !!\n            put(rss, [{db_dir, Value} | get(rss)]),\n            fload(FD, rss, GC, C, Lno+1, ?NEXTLINE);\n\n        [\"rss_expire\", '=', Value] ->\n            put(rss, [{expire, Value} | get(rss)]),\n            fload(FD, rss, GC, C, Lno+1, ?NEXTLINE);\n\n        [\"rss_days\", '=', Value] ->\n            put(rss, [{days, Value} | get(rss)]),\n            fload(FD, rss, GC, C, Lno+1, ?NEXTLINE);\n\n        [\"rss_rm_exp\", '=', Value] ->\n            put(rss, [{rm_exp, Value} | get(rss)]),\n            fload(FD, rss, GC, C, Lno+1, ?NEXTLINE);\n\n        [\"rss_max\", '=', Value] ->\n            put(rss, [{rm_max, Value} | get(rss)]),\n            fload(FD, rss, GC, C, Lno+1, ?NEXTLINE);\n\n        ['<', \"/rss\", '>'] ->\n            case get(rss_id) of\n                undefined ->\n                    {error, ?F(\"No rss_id specified at line ~w\", [Lno])};\n                RSSid ->\n                    yaws_rss:open(RSSid, get(rss)),\n                    fload(FD, server, GC, C, Lno+1, ?NEXTLINE)\n            end;\n\n        [H|T] ->\n            {error, ?F(\"Unexpected input ~p at line ~w\", [[H|T], Lno])};\n        Err ->\n            Err\n    end;\n\nfload(FD, opaque, GC, C, Lno, Chars) ->\n    case toks(Lno, Chars) of\n        [] ->\n            fload(FD, opaque, GC, C, Lno+1, ?NEXTLINE);\n\n        [Key, '=', Value] ->\n            C1 = C#sconf{opaque = [{Key,Value} | C#sconf.opaque]},\n            fload(FD, opaque, GC, C1, Lno+1, ?NEXTLINE);\n\n        [Key, '='| Value] ->\n            String_value = lists:flatten(\n                             lists:map(\n                               fun(Item) when is_atom(Item) ->\n                                       atom_to_list(Item);\n                                  (Item) ->\n                                       Item\n                               end, Value)),\n            C1 = C#sconf{opaque = [{Key, String_value} | C#sconf.opaque]},\n            fload(FD, opaque, GC, C1, Lno+1, ?NEXTLINE);\n\n        ['<', \"/opaque\", '>'] ->\n            fload(FD, server, GC, C, Lno+1, ?NEXTLINE);\n\n        [H|T] ->\n            {error, ?F(\"Unexpected input ~p at line ~w\", [[H|T], Lno])};\n        Err ->\n            Err\n    end.\n\n\nis_bool(\"true\") ->\n    {true, true};\nis_bool(\"false\") ->\n    {true, false};\nis_bool(_) ->\n    false.\n\n\nwarn_dir(Type, Dir) ->\n    case is_dir(Dir) of\n        true ->\n            true;\n        false ->\n            error_logger:format(\"Config Warning: Directory ~s \"\n                                \"for ~s doesn't exist~n\",\n                                [Dir, Type]),\n            false\n    end.\n\nis_dir(Val) ->\n    case file:read_file_info(Val) of\n        {ok, FI} when FI#file_info.type == directory ->\n            true;\n        _ ->\n            false\n    end.\n\n\nis_file(Val) ->\n    case file:read_file_info(Val) of\n        {ok, FI} when FI#file_info.type == regular ->\n            true;\n        _ ->\n            false\n    end.\n\nis_wildcard(Val) ->\n    (lists:member($*, Val) orelse\n     lists:member($?, Val) orelse\n     (lists:member($[, Val) andalso lists:member($], Val)) orelse\n     (lists:member(${, Val) andalso lists:member($}, Val))).\n\n\n%% tokenizer\ntoks(Lno, Chars) ->\n    toks(Lno, Chars, free, [], []). % two accumulators\n\ntoks(Lno, [$#|_T], Mode, Ack, Tack) ->\n    toks(Lno, [], Mode, Ack, Tack);\n\ntoks(Lno, [H|T], free, Ack, Tack) ->\n    %%?Debug(\"Char=~p\", [H]),\n    case {is_quote(H), is_string_char([H|T]),is_special(H), yaws:is_space(H)} of\n        {_,_, _, true} ->\n            toks(Lno, T, free, Ack, Tack);\n        {_,_, true, _} ->\n            toks(Lno, T, free, [], [list_to_atom([H]) | Tack]);\n        {_,true, _,_} ->\n            toks(Lno, T, string, [H], Tack);\n        {_,utf8, _,_} ->\n            toks(Lno, tl(T), string, [H, hd(T)], Tack);\n        {true,_, _,_} ->\n            toks(Lno, T, quote, [], Tack);\n        {false, false, false, false} ->\n            {error, ?F(\"Unexpected character  <~p / ~c> at line ~w\",\n                       [H,H, Lno])}\n    end;\ntoks(Lno, [C|T], string, Ack, Tack) ->\n    case {is_backquote(C), is_quote(C), is_string_char([C|T]), is_special(C),\n          yaws:is_space(C)} of\n        {true, _, _, _,_} ->\n            toks(Lno, T, [backquote,string], Ack, Tack);\n        {_, _, true, _,_} ->\n            toks(Lno, T, string, [C|Ack], Tack);\n        {_, _, utf8, _,_} ->\n            toks(Lno, tl(T), string, [C, hd(T)|Ack], Tack);\n        {_, _, _, true, _} ->\n            toks(Lno, T, free, [], [list_to_atom([C]),lists:reverse(Ack)|Tack]);\n        {_, true, _, _, _} ->\n            toks(Lno, T, quote, [], [lists:reverse(Ack)|Tack]);\n        {_, _, _, _, true} ->\n            toks(Lno, T, free, [], [lists:reverse(Ack)|Tack]);\n        {false, false, false, false, false} ->\n            {error, ?F(\"Unexpected character  <~p / ~c> at line ~w\",\n                       [C, C, Lno])}\n    end;\ntoks(Lno, [C|T], quote, Ack, Tack) ->\n    case {is_quote(C), is_backquote(C)} of\n        {true, _} ->\n            toks(Lno, T, free, [], [lists:reverse(Ack)|Tack]);\n        {_, true} ->\n            toks(Lno, T, [backquote,quote], [C|Ack], Tack);\n        {false, false} ->\n            toks(Lno, T, quote, [C|Ack], Tack)\n    end;\ntoks(Lno, [C|T], [backquote,Mode], Ack, Tack) ->\n    toks(Lno, T, Mode, [C|Ack], Tack);\ntoks(_Lno, [], string, Ack, Tack) ->\n    lists:reverse([lists:reverse(Ack) | Tack]);\ntoks(_Lno, [], free, _,Tack) ->\n    lists:reverse(Tack).\n\nis_quote(34) -> true ;  %% $\" but emacs mode can't handle it\nis_quote(_)  -> false.\n\nis_backquote($\\\\) -> true ;\nis_backquote(_)  -> false.\n\nis_string_char([C|T]) ->\n    if\n        $a =< C, C =< $z ->\n            true;\n        $A =< C, C =< $Z ->\n            true;\n        $0 =< C, C =< $9 ->\n            true;\n        C == 195 , T /= [] ->\n            %% FIXME check that [C, hd(T)] really is a char ?? how\n            utf8;\n        true ->\n            lists:member(C, [$., $/, $:, $_, $-, $+, $~, $@, $*])\n    end.\n\nis_special(C) ->\n    lists:member(C, [$=, $[, $], ${, $}, $, ,$<, $>, $,]).\n\n%% parse the argument string PLString which can either be the undefined\n%% atom or a proplist. Currently the only supported keys are\n%% fullsweep_after, min_heap_size, and min_bin_vheap_size. Any other\n%% key/values are ignored.\nparse_process_options(PLString) ->\n    case erl_scan:string(PLString ++ \".\") of\n        {ok, PLTokens, _} ->\n            case erl_parse:parse_term(PLTokens) of\n                {ok, undefined} ->\n                    {ok, []};\n                {ok, []} ->\n                    {ok, []};\n                {ok, [Hd|_Tl]=PList} when is_atom(Hd); is_tuple(Hd) ->\n                    %% create new safe proplist of desired options\n                    {ok, proplists_int_copy([], PList, [fullsweep_after,\n                                                        min_heap_size,\n                                                        min_bin_vheap_size])};\n                _ ->\n                    {error, \"Expect undefined or proplist\"}\n            end;\n        _ ->\n            {error, \"Expect undefined or proplist\"}\n    end.\n\n%% copy proplist integer values for the given keys from the\n%% Src proplist to the Dest proplist. Ignored keys that are not\n%% found or have non-integer values. Returns the new Dest proplist.\nproplists_int_copy(Dest, _Src, []) ->\n    Dest;\nproplists_int_copy(Dest, Src, [Key|NextKeys]) ->\n    case proplists:get_value(Key, Src) of\n        Val when is_integer(Val) ->\n            proplists_int_copy([{Key, Val}|Dest], Src, NextKeys);\n        _ ->\n            proplists_int_copy(Dest, Src, NextKeys)\n    end.\n\nparse_soap_srv_mods(['<', Module, ',' , Handler, ',', WsdlFile, '>' | Tail],\n                    Ack) ->\n    case is_file(WsdlFile) of\n        true ->\n            S = { {list_to_atom(Module), list_to_atom(Handler)}, WsdlFile},\n            parse_soap_srv_mods(Tail, [S |Ack]);\n        false ->\n            {error, ?F(\"Bad wsdl file ~p\", [WsdlFile])}\n    end;\n\nparse_soap_srv_mods(['<', Module, ',' , Handler, ',', WsdlFile, ',',\n                     Prefix, '>' | Tail], Ack) ->\n    case is_file(WsdlFile) of\n        true ->\n            S = { {list_to_atom(Module), list_to_atom(Handler)},\n                  WsdlFile, Prefix},\n            parse_soap_srv_mods(Tail, [S |Ack]);\n        false ->\n            {error, ?F(\"Bad wsdl file ~p\", [WsdlFile])}\n    end;\n\nparse_soap_srv_mods([ SoapSrvMod | _Tail], _Ack) ->\n    {error, ?F(\"Bad soap_srv_mods syntax: ~p\", [SoapSrvMod])};\n\nparse_soap_srv_mods([], Ack) ->\n    {ok, Ack}.\n\nparse_appmods(['<', PathElem, ',' , AppMod, '>' | Tail], Ack) ->\n    S = {PathElem , list_to_atom(AppMod)},\n    parse_appmods(Tail, [S |Ack]);\n\nparse_appmods(['<', PathElem, ',' , AppMod, \"exclude_paths\" |Tail], Ack)->\n    Paths = lists:takewhile(fun(X) -> X /= '>' end,\n                            Tail),\n    Tail2 = lists:dropwhile(fun(X) -> X /= '>' end,\n                            Tail),\n    Tail3 = tl(Tail2),\n\n    S = {PathElem , list_to_atom(AppMod), lists:map(\n                                            fun(Str) ->\n                                                    string:tokens(Str, \"/\")\n                                            end, Paths)},\n    parse_appmods(Tail3, [S |Ack]);\n\n\nparse_appmods([AppMod | Tail], Ack) ->\n    %% just some simpleminded test to catch syntax errors in the config\n    case AppMod of\n        [Char] ->\n            case is_special(Char) of\n                true ->\n                    {error, \"Bad appmod syntax\"};\n                false ->\n                    S = {AppMod, list_to_atom(AppMod)},\n                    parse_appmods(Tail, [S | Ack])\n            end;\n        _ ->\n            S = {AppMod, list_to_atom(AppMod)},\n            parse_appmods(Tail, [S | Ack])\n    end;\n\nparse_appmods([], Ack) ->\n    {ok, Ack}.\n\n\nparse_revproxy([Prefix, Url]) ->\n    parse_revproxy_url(Prefix, Url);\nparse_revproxy([Prefix, Url, \"intercept_mod\", InterceptMod]) ->\n    case parse_revproxy_url(Prefix, Url) of\n        {ok, RP} ->\n            {ok, RP#proxy_cfg{intercept_mod = list_to_atom(InterceptMod)}};\n        Error ->\n            Error\n    end;\nparse_revproxy([Prefix, Proto, '[', IPv6, ']', Rest, \"intercept_mod\", InterceptMod]) ->\n    Url = Proto ++ \"[\" ++ IPv6 ++ \"]\" ++ Rest,\n    parse_revproxy([Prefix, Url, \"intercept_mod\", InterceptMod]);\nparse_revproxy([Prefix, Proto, '[', IPv6, ']', Rest]) ->\n    Url = Proto ++ \"[\" ++ IPv6 ++ \"]\" ++ Rest,\n    parse_revproxy([Prefix, Url]);\nparse_revproxy(_Other) ->\n    {error, syntax}.\n\nparse_revproxy_url(Prefix, Url) ->\n    case (catch yaws_api:parse_url(Url)) of\n        {'EXIT', _} ->\n            {error, url};\n        URL when URL#url.path == \"/\" ->\n            P = case lists:reverse(Prefix) of\n                    [$/|_Tail] ->\n                        Prefix;\n                    Other ->\n                        lists:reverse(Other)\n                end,\n            {ok, #proxy_cfg{prefix=P, url=URL}};\n        _URL ->\n            {error, \"Can't revproxy to a URL with a path \"}\n    end.\n\n\nparse_expires(['<', MimeType, ',' , Expire, '>' | Tail], Acc) ->\n    {EType, Value} =\n        case string:tokens(Expire, \"+\") of\n            [\"always\"] ->\n                {always, 0};\n            [Secs] ->\n                {access, (catch list_to_integer(Secs))};\n            [\"access\", Secs] ->\n                {access, (catch list_to_integer(Secs))};\n            [\"modify\", Secs] ->\n                {modify, (catch list_to_integer(Secs))};\n            _ ->\n                {error, \"Bad expires syntax\"}\n        end,\n    if\n        EType =:= error ->\n            {EType, Value};\n        not is_integer(Value) ->\n            {error, \"Bad expires syntax\"};\n        true ->\n            case parse_mime_type(MimeType) of\n                {ok, \"*\", \"*\"} ->\n                    E = {all, EType, Value},\n                    parse_expires(Tail, [E |Acc]);\n                {ok, Type, \"*\"} ->\n                    E = {{Type, all}, EType, Value},\n                    parse_expires(Tail, [E |Acc]);\n                {ok, _Type, _SubType} ->\n                    E = {MimeType, EType, Value},\n                    parse_expires(Tail, [E |Acc]);\n                Error ->\n                    Error\n            end\n    end;\nparse_expires([], Acc)->\n    {ok, Acc}.\n\n\nparse_phpmod(['<', \"cgi\", ',', DefaultPhpPath, '>'], DefaultPhpPath) ->\n    {ok, {cgi, DefaultPhpPath}};\nparse_phpmod(['<', \"cgi\", ',', PhpPath, '>'], _) ->\n    case is_file(PhpPath) of\n        true ->\n            {ok, {cgi, PhpPath}};\n        false ->\n            {error, ?F(\"~s is not a regular file\", [PhpPath])}\n    end;\nparse_phpmod(['<', \"fcgi\", ',', HostPortSpec, '>'], _) ->\n    case string_to_host_and_port(HostPortSpec) of\n        {ok, Host, Port} ->\n            {ok, {fcgi, {Host, Port}}};\n        {error, Reason} ->\n            {error, Reason}\n    end;\nparse_phpmod(['<', \"fcgi\", ',', '[', HostSpec, ']', PortSpec, '>'], _) ->\n    case string_to_host_and_port(\"[\" ++ HostSpec ++ \"]\" ++ PortSpec) of\n        {ok, Host, Port} ->\n            {ok, {fcgi, {Host, Port}}};\n        {error, Reason} ->\n            {error, Reason}\n    end;\nparse_phpmod(['<', \"extern\", ',', NodeModFunSpec, '>'], _) ->\n    case string_to_node_mod_fun(NodeModFunSpec) of\n        {ok, Node, Mod, Fun} ->\n            {ok, {extern, {Node,Mod,Fun}}};\n        {ok, Mod, Fun} ->\n            {ok, {extern, {Mod,Fun}}};\n        {error, Reason} ->\n            {error, Reason}\n    end.\n\n\nparse_compressible_mime_types(_, all) ->\n    {ok, all};\nparse_compressible_mime_types([\"all\"|_], _Acc) ->\n    {ok, all};\nparse_compressible_mime_types([\"defaults\"|Rest], Acc) ->\n    parse_compressible_mime_types(Rest, ?DEFAULT_COMPRESSIBLE_MIME_TYPES++Acc);\nparse_compressible_mime_types([',' | Rest], Acc) ->\n    parse_compressible_mime_types(Rest, Acc);\nparse_compressible_mime_types([MimeType | Rest], Acc) ->\n    case parse_mime_type(MimeType) of\n        {ok, \"*\", \"*\"} ->\n            {ok, all};\n        {ok, Type, \"*\"} ->\n            parse_compressible_mime_types(Rest, [{Type, all}|Acc]);\n        {ok, Type, SubType} ->\n            parse_compressible_mime_types(Rest, [{Type, SubType}|Acc]);\n        Error ->\n            Error\n    end;\nparse_compressible_mime_types([], Acc) ->\n    {ok, Acc}.\n\n\nparse_mime_type(MimeType) ->\n    Res = re:run(MimeType, \"^([-\\\\w\\+]+|\\\\*)/([-\\\\w\\+\\.]+|\\\\*)$\",\n                 [{capture, all_but_first, list}]),\n    case Res of\n        {match, [Type,SubType]} ->\n            {ok, Type, SubType};\n        nomatch ->\n            {error, \"Invalid MimeType\"}\n    end.\n\n\nparse_index_files([]) ->\n    ok;\nparse_index_files([Idx|Rest]) ->\n    case Idx of\n        [$/|_] when Rest /= [] ->\n            {error, \"Only the last index should be absolute\"};\n        _ ->\n            parse_index_files(Rest)\n    end.\n\nis_valid_mime_type(MimeType) ->\n    case re:run(MimeType, \"^[-\\\\w\\+]+/[-\\\\w\\+\\.]+$\", [{capture, none}]) of\n        match   -> true;\n        nomatch -> false\n    end.\n\nparse_mime_types(['<', MimeType, ',' | Tail], Acc0) ->\n    Exts      = lists:takewhile(fun(X) -> X /= '>' end, Tail),\n    [_|Tail2] = lists:dropwhile(fun(X) -> X /= '>' end, Tail),\n    Acc1 = lists:foldl(fun(E, Acc) ->\n                               lists:keystore(E, 1, Acc, {E, MimeType})\n                       end, Acc0, Exts),\n    case is_valid_mime_type(MimeType) of\n        true  -> parse_mime_types(Tail2, Acc1);\n        false -> {error, ?F(\"Invalid mime-type '~p'\", [MimeType])}\n    end;\nparse_mime_types([], Acc)->\n    {ok, lists:reverse(Acc)};\nparse_mime_types(_, _) ->\n    {error, \"Unexpected tokens\"}.\n\nparse_charsets(['<', Charset, ',' | Tail], Acc0) ->\n    Exts      = lists:takewhile(fun(X) -> X /= '>' end, Tail),\n    [_|Tail2] = lists:dropwhile(fun(X) -> X /= '>' end, Tail),\n    Acc1 = lists:foldl(fun(E, Acc) ->\n                               lists:keystore(E, 1, Acc, {E, Charset})\n                       end, Acc0, Exts),\n    parse_charsets(Tail2, Acc1);\nparse_charsets([], Acc)->\n    {ok, lists:reverse(Acc)};\nparse_charsets(_, _) ->\n    {error, \"Unexpected tokens\"}.\n\n\nparse_mime_types_info(Directive, Type, undefined, undefined) ->\n    parse_mime_types_info(Directive, Type, #mime_types_info{});\nparse_mime_types_info(Directive, Type, undefined, DefaultInfo) ->\n    parse_mime_types_info(Directive, Type, DefaultInfo);\nparse_mime_types_info(Directive, Type, Info, _) ->\n    parse_mime_types_info(Directive, Type, Info).\n\nparse_mime_types_info(default_type, Type, Info) ->\n    case is_valid_mime_type(Type) of\n        true  -> {ok, Info#mime_types_info{default_type=Type}};\n        false -> {error, ?F(\"Invalid mime-type '~p'\", [Type])}\n    end;\nparse_mime_types_info(default_charset, Charset, Info) ->\n    {ok, Info#mime_types_info{default_charset=Charset}};\nparse_mime_types_info(mime_types_file, File, Info) ->\n    {ok, Info#mime_types_info{mime_types_file=File}};\nparse_mime_types_info(add_types, NewTypes, Info) ->\n    case parse_mime_types(NewTypes, Info#mime_types_info.types) of\n        {ok, Types} -> {ok, Info#mime_types_info{types=Types}};\n        Error       -> Error\n    end;\nparse_mime_types_info(add_charsets, NewCharsets, Info) ->\n    case parse_charsets(NewCharsets, Info#mime_types_info.charsets) of\n        {ok, Charsets} -> {ok, Info#mime_types_info{charsets=Charsets}};\n        Error          -> Error\n    end.\n\n\nparse_nslookup_pref(Pref) ->\n    parse_nslookup_pref(Pref, []).\n\nparse_nslookup_pref(Empty, []) when Empty == [] orelse Empty == ['[', ']'] ->\n    %% Get default value, if nslookup_pref = [].\n    {ok, yaws:gconf_nslookup_pref(#gconf{})};\nparse_nslookup_pref([C, Family | Rest], Result)\n  when C == '[' orelse C == ',' ->\n    case Family of\n        \"inet\" ->\n            case lists:member(inet, Result) of\n                false -> parse_nslookup_pref(Rest, [inet | Result]);\n                true  -> parse_nslookup_pref(Rest, Result)\n            end;\n        \"inet6\" ->\n            case lists:member(inet6, Result) of\n                false -> parse_nslookup_pref(Rest, [inet6 | Result]);\n                true  -> parse_nslookup_pref(Rest, Result)\n            end;\n        _ ->\n            case Result of\n                [PreviousFamily | _] ->\n                    {error, ?F(\"Invalid nslookup_pref: invalid family or \"\n                        \"token '~s', after family '~s'\",\n                        [Family, PreviousFamily])};\n                [] ->\n                    {error, ?F(\"Invalid nslookup_pref: invalid family or \"\n                        \"token '~s'\", [Family])}\n            end\n    end;\nparse_nslookup_pref([']'], Result) ->\n    {ok, lists:reverse(Result)};\nparse_nslookup_pref([Invalid | _], []) ->\n    {error, ?F(\"Invalid nslookup_pref: unexpected token '~s'\", [Invalid])};\nparse_nslookup_pref([Invalid | _], [Family | _]) ->\n    {error, ?F(\"Invalid nslookup_pref: unexpected token '~s', \"\n        \"after family '~s'\", [Invalid, Family])}.\n\n\nparse_redirect(Path, [Code, URL], Mode, Lno) ->\n    case catch list_to_integer(Code) of\n        I when is_integer(I), I >= 300, I =< 399 ->\n            try yaws_api:parse_url(URL, sloppy) of\n                U when is_record(U, url) ->\n                    {Path, I, U, Mode}\n            catch _:_ ->\n                    {error, ?F(\"Bad redirect URL ~p at line ~w\", [URL, Lno])}\n            end;\n        I when is_integer(I), I >= 100, I =< 599 ->\n            %% Only relative path are authorized here\n            try yaws_api:parse_url(URL, sloppy) of\n                #url{scheme=undefined, host=[], port=undefined, path=P} ->\n                    {Path, I, P, Mode};\n                #url{} ->\n                    {error, ?F(\"Bad redirect rule at line ~w: \"\n                               \" Absolute URL is forbidden here\", [Lno])}\n            catch _:_ ->\n                    {error, ?F(\"Bad redirect URL ~p at line ~w\", [URL, Lno])}\n            end;\n        _ ->\n            {error, ?F(\"Bad status code ~p at line ~w\", [Code, Lno])}\n    end;\nparse_redirect(Path, [CodeOrUrl], Mode, Lno) ->\n    case catch list_to_integer(CodeOrUrl) of\n        I when is_integer(I), I >= 300, I =< 399 ->\n            {error, ?F(\"Bad redirect rule at line ~w: \"\n                       \"URL to redirect to is missing \", [Lno])};\n        I when is_integer(I), I >= 100, I =< 599 ->\n            {Path, I, undefined, Mode};\n        I when is_integer(I) ->\n            {error, ?F(\"Bad status code ~p at line ~w\", [CodeOrUrl, Lno])};\n        _ ->\n            try yaws_api:parse_url(CodeOrUrl, sloppy) of\n                #url{}=U ->\n                    {Path, 302, U, Mode}\n            catch _:_ ->\n                    {error, ?F(\"Bad redirect URL ~p at line ~w\",\n                               [CodeOrUrl, Lno])}\n            end\n    end;\nparse_redirect(_Path, _, _Mode, Lno) ->\n    {error, ?F(\"Bad redirect rule at line ~w\", [Lno])}.\n\n\nssl_start() ->\n    case catch ssl:start() of\n        ok ->\n            ok;\n        {error,{already_started,ssl}} ->\n            ok;\n        Err ->\n            error_logger:format(\"Failed to start ssl: ~p~n\", [Err])\n    end.\n\n\n\n%% search for an SC within Pairs that have the same, listen,port,ssl,severname\n%% Return {Pid, SC, Scs} or false\n%% Pairs is the pairs in yaws_server #state{}\nsearch_sconf(GC, NewSC, Pairs) ->\n    case lists:zf(\n           fun({Pid, Scs = [SC|_]}) ->\n                   case same_virt_srv(GC, NewSC, SC) of\n                       true ->\n                           case lists:keysearch(NewSC#sconf.servername,\n                                                #sconf.servername, Scs) of\n                               {value, Found} ->\n                                   {true, {Pid, Found, Scs}};\n                               false ->\n                                   false\n                           end;\n                       false ->\n                           false\n                   end\n           end, Pairs) of\n        [] ->\n            false;\n        [{Pid, Found, Scs}] ->\n            {Pid, Found, Scs};\n        _Other ->\n            error_logger:format(\"Fatal error, no two sconfs should \"\n                                \" ever be considered equal ..\",[]),\n            erlang:error(fatal_conf)\n    end.\n\n%% find the group a new SC would belong to\nsearch_group(GC, SC, Pairs) ->\n    Fun =  fun({Pid, [S|Ss]}) ->\n                   case same_virt_srv(GC, S, SC) of\n                       true ->\n                           {true, {Pid, [S|Ss]}};\n                       false ->\n                           false\n                   end\n           end,\n\n    lists:zf(Fun, Pairs).\n\n\n%% Return a new Pairs list with one SC updated\nupdate_sconf(Gc, NewSc, Pos, Pairs) ->\n    lists:map(\n      fun({Pid, Scs}) ->\n              case same_virt_srv(Gc, hd(Scs), NewSc) of\n                  true ->\n                      L2 = lists:keydelete(NewSc#sconf.servername,\n                                           #sconf.servername, Scs),\n                      {Pid, yaws:insert_at(NewSc, Pos, L2)};\n                  false ->\n                      {Pid, Scs}\n              end\n      end, Pairs).\n\n\n%% return a new pairs list with SC removed\ndelete_sconf(Gc, OldSc, Pairs) ->\n    lists:zf(\n      fun({Pid, Scs}) ->\n              case same_virt_srv(Gc, hd(Scs), OldSc) of\n                  true ->\n                      L2 = lists:keydelete(OldSc#sconf.servername,\n                                           #sconf.servername, Scs),\n                      {true, {Pid, L2}};\n                  false ->\n                      {true, {Pid, Scs}}\n              end\n\n      end, Pairs).\n\n\n\nsame_virt_srv(Gc, S, NewSc) when S#sconf.listen == NewSc#sconf.listen,\n                                 S#sconf.port == NewSc#sconf.port ->\n    if\n        Gc#gconf.sni == disable orelse\n        S#sconf.ssl == undefined orelse\n        NewSc#sconf.ssl == undefined ->\n            (S#sconf.ssl == NewSc#sconf.ssl);\n        true ->\n            true\n    end;\nsame_virt_srv(_,_,_) ->\n    false.\n\n\neq_sconfs(S1,S2) ->\n    (S1#sconf.port == S2#sconf.port andalso\n     S1#sconf.flags == S2#sconf.flags andalso\n     S1#sconf.redirect_map == S2#sconf.redirect_map andalso\n     S1#sconf.rhost == S2#sconf.rhost andalso\n     S1#sconf.rmethod == S2#sconf.rmethod andalso\n     S1#sconf.docroot == S2#sconf.docroot andalso\n     S1#sconf.xtra_docroots == S2#sconf.xtra_docroots andalso\n     S1#sconf.listen == S2#sconf.listen andalso\n     S1#sconf.servername == S2#sconf.servername andalso\n     S1#sconf.yaws == S2#sconf.yaws andalso\n     S1#sconf.ssl == S2#sconf.ssl andalso\n     S1#sconf.authdirs == S2#sconf.authdirs andalso\n     S1#sconf.partial_post_size == S2#sconf.partial_post_size andalso\n     S1#sconf.appmods == S2#sconf.appmods andalso\n     S1#sconf.expires == S2#sconf.expires andalso\n     S1#sconf.errormod_401 == S2#sconf.errormod_401 andalso\n     S1#sconf.errormod_404 == S2#sconf.errormod_404 andalso\n     S1#sconf.errormod_crash == S2#sconf.errormod_crash andalso\n     S1#sconf.arg_rewrite_mod == S2#sconf.arg_rewrite_mod andalso\n     S1#sconf.logger_mod == S2#sconf.logger_mod andalso\n     S1#sconf.opaque == S2#sconf.opaque andalso\n     S1#sconf.start_mod == S2#sconf.start_mod andalso\n     S1#sconf.allowed_scripts == S2#sconf.allowed_scripts andalso\n     S1#sconf.tilde_allowed_scripts == S2#sconf.tilde_allowed_scripts andalso\n     S1#sconf.index_files == S2#sconf.index_files andalso\n     S1#sconf.revproxy == S2#sconf.revproxy andalso\n     S1#sconf.soptions == S2#sconf.soptions andalso\n     S1#sconf.extra_cgi_vars == S2#sconf.extra_cgi_vars andalso\n     S1#sconf.stats == S2#sconf.stats andalso\n     S1#sconf.fcgi_app_server == S2#sconf.fcgi_app_server andalso\n     S1#sconf.php_handler == S2#sconf.php_handler andalso\n     S1#sconf.shaper == S2#sconf.shaper andalso\n     S1#sconf.deflate_options == S2#sconf.deflate_options andalso\n     S1#sconf.mime_types_info == S2#sconf.mime_types_info).\n\n\n\n\n%% This the version of setconf that perform a\n%% soft reconfig, it requires the args to be checked.\nsoft_setconf(GC, Groups, OLDGC, OldGroups) ->\n    if\n        GC /= OLDGC ->\n            yaws_trace:setup(GC),\n            update_gconf(GC);\n        true ->\n            ok\n    end,\n    compile_and_load_src_dir(GC),\n    Grps = load_mime_types_module(GC, Groups),\n    Rems = remove_old_scs(GC, lists:flatten(OldGroups), Grps),\n    Adds = soft_setconf_scs(GC, lists:flatten(Grps), 1, OldGroups),\n    lists:foreach(\n      fun({delete_sconf, SC}) ->\n              delete_sconf(SC);\n         ({add_sconf, N, SC}) ->\n              add_sconf(N, SC);\n         ({update_sconf, N, SC}) ->\n              update_sconf(N, SC)\n      end, Rems ++ Adds).\n\n\n\nhard_setconf(GC, Groups) ->\n    gen_server:call(yaws_server,{setconf, GC, Groups}, infinity).\n\n\nremove_old_scs(Gc, [Sc|Scs], NewGroups) ->\n    case find_group(Gc, Sc, NewGroups) of\n        false ->\n            [{delete_sconf, Sc} |remove_old_scs(Gc, Scs, NewGroups)];\n        {true, G} ->\n            case find_sc(Sc, G) of\n                false ->\n                    [{delete_sconf, Sc} | remove_old_scs(Gc, Scs, NewGroups)];\n                _ ->\n                    remove_old_scs(Gc, Scs, NewGroups)\n            end\n    end;\nremove_old_scs(_, [],_) ->\n    [].\n\nsoft_setconf_scs(Gc, [Sc|Scs], N, OldGroups) ->\n    case find_group(Gc, Sc, OldGroups) of\n        false ->\n            [{add_sconf,N,Sc} | soft_setconf_scs(Gc, Scs, N+1, OldGroups)];\n        {true, G} ->\n            case find_sc(Sc, G) of\n                false ->\n                    [{add_sconf,N,Sc} | soft_setconf_scs(Gc, Scs,N+1,OldGroups)];\n                {true, _OldSc} ->\n                    [{update_sconf,N,Sc} | soft_setconf_scs(Gc, Scs,N+1,OldGroups)]\n            end\n    end;\nsoft_setconf_scs(_,[], _, _) ->\n    [].\n\n\n%% checking code\n\ncan_hard_gc(New, Old) ->\n    if\n        Old == undefined ->\n            true;\n        New#gconf.yaws_dir == Old#gconf.yaws_dir,\n        New#gconf.runmods == Old#gconf.runmods,\n        New#gconf.logdir == Old#gconf.logdir ->\n            true;\n        true ->\n            false\n    end.\n\n\n\ncan_soft_setconf(NEWGC, NewGroups, OLDGC, OldGroups) ->\n    can_soft_gc(NEWGC, OLDGC) andalso\n        can_soft_sconf(NEWGC, lists:flatten(NewGroups), OldGroups).\n\ncan_soft_gc(G1, G2) ->\n    if\n        G1#gconf.flags == G2#gconf.flags,\n        G1#gconf.logdir == G2#gconf.logdir,\n        G1#gconf.log_wrap_size == G2#gconf.log_wrap_size,\n        G1#gconf.sni == G2#gconf.sni,\n        G1#gconf.id == G2#gconf.id ->\n            true;\n        true ->\n            false\n    end.\n\n\ncan_soft_sconf(Gc, [Sc|Scs], OldGroups) ->\n    case find_group(Gc, Sc, OldGroups) of\n        false ->\n            can_soft_sconf(Gc, Scs, OldGroups);\n        {true, G} ->\n            case find_sc(Sc, G) of\n                false ->\n                    can_soft_sconf(Gc, Scs, OldGroups);\n                {true, Old} when Old#sconf.start_mod /= Sc#sconf.start_mod ->\n                    false;\n                {true, Old} ->\n                    case\n                        {proplists:get_value(listen_opts, Old#sconf.soptions),\n                         proplists:get_value(listen_opts, Sc#sconf.soptions)} of\n                        {Opts, Opts} ->\n                            can_soft_sconf(Gc, Scs, OldGroups);\n                        _ ->\n                            false\n                    end\n            end\n    end;\ncan_soft_sconf(_, [], _) ->\n    true.\n\n\nfind_group(GC, SC, [G|Gs]) ->\n    case same_virt_srv(GC, SC, hd(G)) of\n        true ->\n            {true, G};\n        false ->\n            find_group(GC, SC, Gs)\n    end;\nfind_group(_,_,[]) ->\n    false.\n\nfind_sc(SC, [S|Ss]) ->\n    if SC#sconf.servername  == S#sconf.servername  ->\n            {true, S};\n       true ->\n            find_sc(SC, Ss)\n    end;\nfind_sc(_SC,[]) ->\n    false.\n\n\nverify_upgrade_args(GC, Groups0) when is_record(GC, gconf) ->\n    SCs0 = lists:flatten(Groups0),\n    case lists:all(fun(SC) -> is_record(SC, sconf) end, SCs0) of\n        true ->\n            %% Embedded code may give appmods as a list of strings, or\n            %% appmods can be {StringPathElem,ModAtom} or\n            %% {StringPathElem,ModAtom,ExcludePathsList} tuples. Handle\n            %% all possible variants here.\n            SCs1 = lists:map(\n                     fun(SC) ->\n                             SC#sconf{appmods =\n                                          lists:map(\n                                            fun({PE, Mod}) ->\n                                                    {PE, Mod};\n                                               ({PE,Mod,Ex}) ->\n                                                    {PE,Mod,Ex};\n                                               (AM) when is_list(AM) ->\n                                                    {AM,list_to_atom(AM)};\n                                               (AM) when is_atom(AM) ->\n                                                    {atom_to_list(AM), AM}\n                                            end,\n                                            SC#sconf.appmods)}\n                     end, SCs0),\n            case catch validate_cs(GC, SCs1) of\n                {ok, GC, Groups1} -> {GC, Groups1};\n                {error, Reason}   -> erlang:error(Reason);\n                _                 -> erlang:error(badgroups)\n            end;\n        false ->\n            erlang:error(badgroups)\n    end.\n\n\n\nadd_sconf(SC) ->\n    add_sconf(-1, SC).\n\nadd_sconf(Pos, SC0) ->\n    {ok, SC1} = gen_server:call(yaws_server, {add_sconf, Pos, SC0}, infinity),\n    ok = yaws_log:add_sconf(SC1),\n    {ok, SC1}.\n\nupdate_sconf(Pos, SC) ->\n    gen_server:call(yaws_server, {update_sconf, Pos, SC}, infinity).\n\ndelete_sconf(SC) ->\n    ok = gen_server:call(yaws_server, {delete_sconf, SC}, infinity),\n    ok = yaws_log:del_sconf(SC).\n\nupdate_gconf(GC) ->\n    ok = gen_server:call(yaws_server, {update_gconf, GC}, infinity).\n\n\nparse_auth_ips([], Result) ->\n    Result;\nparse_auth_ips([Str|Rest], Result) ->\n    try\n        parse_auth_ips(Rest, [yaws:parse_ipmask(Str)|Result])\n    catch\n        _:_ -> parse_auth_ips(Rest, Result)\n    end.\n\nparse_auth_user(User, Lno) ->\n    try\n        [Name, Passwd] = string:tokens(User, \":\"),\n        case re:run(Passwd, \"{([^}]+)}(?:\\\\$([^$]+)\\\\$)?(.+)\", [{capture,all_but_first,list}]) of\n            {match, [Algo, B64Salt, B64Hash]} ->\n                case parse_auth_user(Name, Algo, B64Salt, B64Hash) of\n                    {ok, Res} ->\n                        Res;\n                    {error, bad_algo} ->\n                        {error, ?F(\"Unsupported hash algorithm '~p' at line ~w\",\n                                   [Algo, Lno])};\n                    {error, bad_user} ->\n                        {error, ?F(\"Invalid user at line ~w\", [Lno])}\n                end;\n            _ ->\n                Salt = yaws_dynopts:rand_bytes(32),\n                {Name, sha256, Salt, crypto:hash(sha256, [Salt, Passwd])}\n        end\n    catch\n        _:_ ->\n            {error, ?F(\"Invalid user at line ~w\", [Lno])}\n    end.\n\nparse_auth_user(User, Algo, B64Salt, B64Hash) ->\n    try\n        if\n            Algo == \"md5\"    orelse Algo == \"sha\"    orelse\n            Algo == \"sha224\" orelse Algo == \"sha256\" orelse\n            Algo == \"sha384\" orelse Algo == \"sha512\" orelse\n            Algo == \"ripemd160\" ->\n                Salt = base64:decode(B64Salt),\n                Hash = base64:decode(B64Hash),\n                {ok, {User, list_to_atom(Algo), Salt, Hash}};\n            true ->\n                {error, unsupported_algo}\n        end\n    catch\n        _:_ -> {error, bad_user}\n    end.\n\n\nsubconfigfiles(FD, Name, Lno) ->\n    {ok, Config} = file:pid2name(FD),\n    ConfPath = filename:dirname(filename:absname(Config)),\n    File = filename:absname(Name, ConfPath),\n    case {is_file(File), is_wildcard(Name)} of\n        {true,_} ->\n            {ok, [File]};\n        {false,true} ->\n            Names = filelib:wildcard(Name, ConfPath),\n            Files = [filename:absname(N, ConfPath) || N <- lists:sort(Names)],\n            {ok, lists:filter(fun filter_subconfigfile/1, Files)};\n        {false,false} ->\n            {error, ?F(\"Expect filename or wildcard at line ~w\"\n                       \" (subconfig: ~s)\", [Lno, Name])}\n    end.\n\nsubconfigdir(FD, Name, Lno) ->\n    {ok, Config} = file:pid2name(FD),\n    ConfPath = filename:dirname(filename:absname(Config)),\n    Dir = filename:absname(Name, ConfPath),\n    case is_dir(Dir) of\n        true ->\n            case file:list_dir(Dir) of\n                {ok, Names} ->\n                    Files = [filename:absname(N, Dir) || N <- lists:sort(Names)],\n                    {ok, lists:filter(fun filter_subconfigfile/1, Files)};\n                {error, Error} ->\n                    {error, ?F(\"Directory ~s is not readable: ~s\",\n                               [Name, Error])}\n            end;\n        false ->\n            {error, ?F(\"Expect directory at line ~w (subconfdir: ~s)\",\n                       [Lno, Dir])}\n    end.\n\nfilter_subconfigfile(File) ->\n    case filename:basename(File) of\n        [$.|_] ->\n            error_logger:info_msg(\"Yaws: Ignore subconfig file ~s~n\", [File]),\n            false;\n        _ ->\n            true\n    end.\n\nfload_subconfigfiles([], global, GC, Cs) ->\n    {ok, GC, Cs};\nfload_subconfigfiles([File|Files], global, GC, Cs) ->\n    error_logger:info_msg(\"Yaws: Using global subconfig file ~s~n\", [File]),\n    case file:open(File, [read]) of\n        {ok, FD} ->\n            R = (catch fload(FD, GC, Cs, 1, ?NEXTLINE)),\n            ?Debug(\"FLOAD(~s): ~p\", [File, R]),\n            case R of\n                {ok, GC1, Cs1} -> fload_subconfigfiles(Files, global, GC1, Cs1);\n                Err            -> Err\n            end;\n        Err ->\n            {error, ?F(\"Can't open subconfig file ~s: ~p\", [File,Err])}\n    end;\nfload_subconfigfiles([], server, GC, C) ->\n    {ok, GC, C};\nfload_subconfigfiles([File|Files], server, GC, C) ->\n    error_logger:info_msg(\"Yaws: Using server subconfig file ~s~n\", [File]),\n    case file:open(File, [read]) of\n        {ok, FD} ->\n            R = (catch fload(FD, server, GC, C, 1, ?NEXTLINE)),\n            ?Debug(\"FLOAD(~s): ~p\", [File, R]),\n            case R of\n                {ok, GC1, C1, _, eof} ->\n                    fload_subconfigfiles(Files, server, GC1, C1);\n                {ok, _, _, Lno, ['<', \"/server\", '>']} ->\n                    {error, ?F(\"Unexpected closing tag in subconfgile ~s\"\n                               \" at line ~w \", [File, Lno])};\n                Err ->\n                    Err\n            end;\n        Err ->\n            {error, ?F(\"Can't open subconfig file ~s: ~p\", [File,Err])}\n    end.\n\n\nstr2term(Str0) ->\n    Str=Str0++\".\",\n    {ok,Tokens,_EndLine} = erl_scan:string(Str),\n    {ok,AbsForm} = erl_parse:parse_exprs(Tokens),\n    {value,Value,_Bs} = erl_eval:exprs(AbsForm, erl_eval:new_bindings()),\n    Value.\n\ncheck_ciphers([], _) ->\n    ok;\ncheck_ciphers([Spec|Specs], L) ->\n    case lists:member(Spec, L) of\n        true ->\n            check_ciphers(Specs, L);\n        false ->\n            {error, ?F(\"Bad cipherspec ~p\",[Spec])}\n    end;\ncheck_ciphers(X,_) ->\n    {error, ?F(\"Bad cipherspec ~p\",[X])}.\n\n\nio_get_line(FD, Prompt, Acc) ->\n    Next = io:get_line(FD, Prompt),\n    if\n        is_list(Next) ->\n            case lists:reverse(Next) of\n                [$\\n, $\\\\ |More] ->\n                    io_get_line(FD, Prompt, Acc ++ lists:reverse(More));\n                _ ->\n                    Acc ++ Next\n            end;\n        true ->\n            Next\n    end.\n\nupdate_soptions(SC, Name, Key, Value) ->\n    Opts0 = proplists:get_value(Name, SC#sconf.soptions),\n    Opts1 = lists:keystore(Key, 1, Opts0, {Key, Value}),\n    SOpts = lists:keystore(Name, 1, SC#sconf.soptions, {Name, Opts1}),\n    SC#sconf{soptions = SOpts}.\n\n\nset_sendfile_flags(GC, \"erlang\") ->\n    GC1 = ?gc_set_use_erlang_sendfile(GC, true),\n    {ok, ?gc_set_use_yaws_sendfile(GC1, false)};\nset_sendfile_flags(GC, \"yaws\") ->\n    GC1 = ?gc_set_use_erlang_sendfile(GC, false),\n    {ok, ?gc_set_use_yaws_sendfile(GC1, true)};\nset_sendfile_flags(GC, \"disable\") ->\n    GC1 = ?gc_set_use_erlang_sendfile(GC, false),\n    {ok, ?gc_set_use_yaws_sendfile(GC1, false)};\nset_sendfile_flags(_, _) ->\n    {error, \"Expect erlang|yaws|disable\"}.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_ctl.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_ctl.erl\n%%% Author  : Claes Wikstrom <klacke@bluetail.com>\n%%% Purpose :\n%%% Created : 29 Apr 2002 by Claes Wikstrom <klacke@bluetail.com>\n%%%----------------------------------------------------------------------\n\n\n%% some code to remoteley control a running yaws server\n\n-module(yaws_ctl).\n-author('klacke@bluetail.com').\n\n-include_lib(\"kernel/include/file.hrl\").\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_debug.hrl\").\n\n-export([start/2, actl_trace/1]).\n-export([ls/1,hup/1,stop/1,status/1,load/1,\n         check/1,trace/1, debug_dump/1, stats/1, running_config/1,\n         configtest/1, auth/1]).\n%% internal\n-export([run/1, aloop/3, handle_a/3]).\n\n\n%% assumes the appropriate file structures\n%% are already created with the right perms\n\nstart(_GC, FirstTime) when FirstTime == false ->\n    ok;\nstart(GC, true) ->\n    proc_lib:start_link(?MODULE, run, [GC]).\n\n\nrun(GC) ->\n    %% First check if there is already a Yaws system running\n    %% with the same sid.\n    case connect(GC#gconf.id) of\n        {ok, Sock, _Key} ->\n            %% Not good,\n            gen_tcp:close(Sock),\n            e(\"There is already a yaws system running with the same ~n\"\n              \" id <~p> on this computer and this user, ~n\"\n              \" set another id in the yaws conf file ~n\",\n              [GC#gconf.id]);\n        {error, eacces} ->\n            %% We're not allowed to open the ctl file\n            e(\"Error reading ~s, you don't have access rights to read it\",\n              [yaws:ctl_file(GC#gconf.id)]);\n        {error, _} ->\n            %% Fine, this should be the case\n            run_listen(GC)\n    end.\n\nrand() ->\n    case os:type() of\n        {win32, _} ->\n            {A1, A2, A3}=yaws:get_time_tuple(),\n            yaws_dynopts:random_seed(A1, A2, A3),\n            yaws_dynopts:random_uniform(1 bsl 64);\n        _ ->\n            try\n                crypto:start(),\n                crypto:rand_uniform(0, 1 bsl 64)\n            catch\n                _:_ ->\n                    error_logger:warning_msg(\"Running without crypto app\\n\"),\n                    {A1, A2, A3}=yaws:get_time_tuple(),\n                    yaws_dynopts:random_seed(A1, A2, A3),\n                    yaws_dynopts:random_uniform(1 bsl 64)\n            end\n    end.\n\n\n\nctl_args() ->\n    [{packet, 2},\n     {active, false},\n     binary,\n     {ip, {127,0,0,1}},\n     {reuseaddr, true}].\n\nrun_listen(GC) ->\n    case gen_tcp:listen(0, ctl_args()) of\n        {ok,  L} ->\n            case inet:sockname(L) of\n                {ok, {_, Port}} ->\n                    Key = rand(),\n                    case w_ctl_file(GC#gconf.id, Port, Key) of\n                        ok ->\n                            proc_lib:init_ack(ok),\n                            aloop(L, GC, Key);\n                        error ->\n                            e(\n                              \"Failed to create/manipulate the ctlfile ~n\"\n                              \"called ~s~n\"\n                              \"Either problems with permissions or \"\n                              \" earlier runs of yaws ~nwith the same id \"\n                              \" <~p> as this, check dir for perms~n\",\n                              [yaws:ctl_file(GC#gconf.id), GC#gconf.id])\n                    end;\n                Err ->\n                    e(\"Cannot get sockname for ctlsock: ~p\",[Err] )\n            end;\n        Err ->\n            e(\"Cannot listen on ctl socket, fatal: ~p\", [Err])\n    end.\n\n\ne(Fmt, Args) ->\n    proc_lib:init_ack({error, io_lib:format(Fmt, Args)}),\n    exit(normal).\n\n\n\n%% write the control file, set perms of the file\n%% so that only this user can read the file\n%% That way we're making sure different users\n%% cannot manipulate each others webservers\nw_ctl_file(Sid, Port, Key) ->\n    case catch\n             begin\n                 F = yaws:ctl_file(Sid),\n                 error_logger:info_msg(\"Ctlfile : ~s~n\", [F]),\n                 file:write_file(F, io_lib:format(\"~w.\", [{Port,Key}])),\n                 {ok, FI} = file:read_file_info(F),\n                 ok = file:write_file_info(F, FI#file_info{mode = 8#00600})\n             end of\n             {'EXIT', _} ->\n                 error;\n             _ ->\n                 ok\n         end.\n\n\n\naloop(L, GC, Key) ->\n    case gen_tcp:accept(L) of\n        {ok, A} ->\n            proc_lib:spawn (?MODULE, handle_a, [A, GC, Key]);\n        Err ->\n            error_logger:format(\"yaws_ctl failed to accept: ~p~n\",\n                                [Err]),\n            timer:sleep(2000),\n            ignore\n    end,\n    ?MODULE:aloop(L, GC, Key).\n\nhandle_a(A, GC, Key) ->\n    case gen_tcp:recv(A, 0) of\n        {ok, Data} ->\n            case catch binary_to_term(Data) of\n                {hup, Key} ->\n                    Res = yaws:dohup(A),\n                    Res;\n                {stop, Key} ->\n                    error_logger:info_msg(\"Stopping yaws\\n\",[]),\n                    gen_tcp:send(A, io_lib:format(\n                                      \"stopping yaws with id=~p\\n\",\n                                      [GC#gconf.id])),\n                    file:delete(yaws:ctl_file(GC#gconf.id)),\n                    init:stop();\n                {{trace, What}, Key} ->\n                    Res = actl_trace(What),\n                    gen_tcp:send(A, Res),\n                    gen_tcp:close(A);\n                {status, Key} ->\n                    a_status(A),\n                    gen_tcp:close(A);\n                {{load, Mods}, Key} ->\n                    a_load(A, Mods),\n                    gen_tcp:close(A);\n                {id, Key} ->\n                    a_id(A),\n                    gen_tcp:close(A);\n                {debug_dump, Key} ->\n                    a_debug_dump(A),\n                    gen_tcp:close(A);\n                {stats, Key} ->\n                    a_stats(A),\n                    gen_tcp:close(A);\n                {running_config, Key} ->\n                    a_running_config(A),\n                    gen_tcp:close(A);\n                {Other, Key} ->\n                    gen_tcp:send(A, io_lib:format(\"Other: ~p~n\", [Other])),\n                    gen_tcp:close(A);\n                _Other ->\n                    gen_tcp:close(A)\n\n            end;\n        {error, _} ->\n            gen_tcp:close(A)\n    end.\n\n\n%% We implement this by reloading a patched config\nactl_trace(What) ->\n    case lists:member(What, [traffic, http, off]) of\n        true ->\n            {ok, GC, SCs} = yaws_api:getconf(),\n            case GC#gconf.trace of\n                false when What /= off ->\n                    yaws_api:setconf(GC#gconf{trace = {true, What}},SCs),\n                    io_lib:format(\n                      \"Turning on trace of ~p to directory ~s~n\",\n                      [What,\n                       filename:join([GC#gconf.logdir,\n                                      yaws_trace:get_tracedir()])]);\n                false when What == off ->\n                    io_lib:format(\"Tracing is already turned off ~n\",[]);\n                {true, _} when What == off ->\n                    yaws_api:setconf(GC#gconf{trace = false},SCs),\n                    \"Turning trace off \\n\";\n                {true, What} ->\n                    io_lib:format(\"Trace of ~p is already turned on, use 'off' \"\n                                  \"to turn off~n\", [What]);\n                {true, _Other} ->\n                    yaws_api:setconf(GC#gconf{trace = {true, What}},SCs),\n                    io_lib:format(\n                      \"Turning on trace of ~p to directory ~s~n\",\n                      [What,\n                       filename:join([GC#gconf.logdir,\n                                      yaws_trace:get_tracedir()])])\n            end;\n        false ->\n            \"error: need one of http | traffic | off as argument\\n\"\n    end.\n\n\n\nf(Fmt, As) ->\n    io_lib:format(Fmt, As).\n\n\na_id(Sock) ->\n    ID = gen_server:call(yaws_server, id, infinity),\n    gen_tcp:send(Sock, ID),\n    ok.\n\na_status(Sock) ->\n    gen_tcp:send(Sock, a_status()).\na_status() ->\n    try\n        {UpTime, L} = yaws_server:stats(),\n        {Days, {Hours, Minutes, _Secs}} = UpTime,\n        UpStr = f(\"~n Uptime: ~w Days, ~w Hours, ~w Minutes  ~n\",\n                  [Days, Hours, Minutes]),\n\n        Header = f(\"IP Port Connections Sessions Requests~n\", []),\n        Lines  = lists:map(fun({IP0, Port, Conns, Sess, Reqs}) ->\n                                   IP = format_ip(IP0),\n                                   f(\"~s ~p ~p ~p ~p~n\",\n                                     [IP, Port, Conns, Sess, Reqs])\n                           end, L),\n        [Header, Lines, UpStr]\n    catch\n        _:Err ->\n            io_lib:format(\"Cannot get status ~p~n\", [Err])\n    end.\n\n\na_debug_dump(Sock) ->\n    gen_tcp:send(Sock, a_status()),\n    yaws_debug:do_debug_dump(Sock).\n\n\nvsn(IP) when size(IP) =:= 4 ->\n    \"(ipv4)\";\nvsn(IP) when size(IP) =:= 8 ->\n    \"(ipv6)\".\n\nformat_ip(IP) ->\n    inet_parse:ntoa(IP).\n\na_running_config(Sock) ->\n    gen_tcp:send(Sock, a_running_config()).\na_running_config() ->\n    {ok, GC, Groups} = yaws_server:getconf(),\n    GcStr = ?format_record(GC, gconf),\n    L = lists:map(fun(Group) ->\n                          [\"** GROUP ** \\n\",\n                           lists:map(\n                             fun(SC) ->\n                                     ?format_record(SC, sconf)\n                             end,\n                             Group)\n                          ]\n                  end, Groups),\n    [\"** GLOBAL CONF ** \\n\", GcStr, L].\n\na_stats(Sock) ->\n    gen_tcp:send(Sock, a_stats()).\na_stats() ->\n    {ok, _GC, Servers0} = yaws_server:getconf(),\n    Servers1 = lists:flatten(Servers0),\n    %% io:format(\"~p~n\", [Servers1]),\n    Servers2 = parse(Servers1),\n\n    case Servers2 of\n        [] ->\n            f(\"No statistics available~n\", []);\n\n        Servers2 ->\n            Stats = fstats(Servers2),\n            Header = f(\"Host IP Port Hits Sent~n\", []),\n\n            Lines = lists:map(fun({Host, IP0, Port, {Hits, Sent}}) ->\n                                      %% we don't use inet_parse:ntoa/1\n                                      %% since it's not documented\n                                      IP = format_ip(IP0),\n                                      IPVsn = vsn(IP0),\n                                      f(\"~s~s ~s ~p ~p ~p~n\",\n                                        [Host, IPVsn, IP, Port, Hits, Sent])\n                              end, Stats),\n            [Header, Lines]\n    end.\n\nparse(V) ->\n    parse(V, []).\nparse([], Acc) ->\n    Acc;\nparse([#sconf{stats=undefined}|Tail], Acc) ->\n    parse(Tail, Acc);\nparse([#sconf{listen=IP, port=Port, servername=Servername,\n              stats=Stats}|Tail], Acc) ->\n    Host = {Servername, IP, Port, Stats},\n    parse(Tail, [Host|Acc]).\n\nfstats(S) ->\n    fstats(S, []).\nfstats([], Acc) ->\n    lists:keysort(1, Acc);\nfstats([{IP, Port, Server, Stats}|Tail], Acc) ->\n    S = {IP, Port, Server, yaws_stats:get(Stats)},\n    fstats(Tail, [S|Acc]).\n\n\na_load(A, Mods) ->\n    case purge(Mods) of\n        ok ->\n            gen_tcp:send(A, f(\"~p~n\", [loadm(Mods)]));\n        Err ->\n            gen_tcp:send(A, f(\"~p~n\", [Err]))\n    end.\n\nloadm([]) ->\n    [];\nloadm([M|Ms]) ->\n    [code:load_file(M)|loadm(Ms)].\n\npurge(Ms) ->\n    case purge(Ms, []) of\n        [] -> ok;\n        L -> {cannot_purge, L}\n    end.\n\npurge([], Ack) ->\n    Ack;\npurge([M|Ms], Ack) ->\n    case code:soft_purge(M) of\n        true ->\n            purge(Ms, Ack);\n        false ->\n            purge(Ms, [M|Ack])\n    end.\n\nconnect(Sid) ->\n    connect_file(yaws:ctl_file(Sid)).\n\n\n%% The ctl file contains the port number the yaws server\n%% is listening at and secret key string.\n\nconnect_file(CtlFile) ->\n    case file:consult(CtlFile) of\n        {ok, [{Port, Key}]} ->\n            case gen_tcp:connect({127,0,0,1}, Port,\n                                 [{active, false},\n                                  {reuseaddr, true},\n                                  binary,\n                                  {packet, 2}], 2000) of\n                {ok, Socket} ->\n                    case inet:port(Socket) of\n                        {ok,Port} ->\n                            {error, erefused};\n                        _X ->\n                            {ok, Socket, Key}\n                    end;\n                Err ->\n                    Err\n            end;\n        {ok, Terms} ->\n            {error, {content, Terms}};\n        Err ->\n            Err\n    end.\n\n\n\nactl(SID, Term) ->\n    case connect(SID) of\n        {error, eacces} ->\n            io:format(\"Another user is using the yaws sid <~p>, ~n\"\n                      \"You are not allowd to read the file <~s>, ~n\"\n                      \"specify by <-I id> which yaws system you want\"\n                      \" to control~n\",\n                      [SID, yaws:ctl_file(SID)]),\n            timer:sleep(10),\n            erlang:halt(1);\n        {error, econnrefused} ->\n            io:format(\"No yaws system responds~n\",[]),\n            timer:sleep(10),\n            erlang:halt(2);\n        {error, {content,Terms}} ->\n            io:format(\"The ctlfile ~s is readable but its content~n\"\n                      \"~p~n\"\n                      \"isn't in YAWS control file format~n\",\n                      [yaws:ctl_file(SID),Terms]),\n            timer:sleep(10),\n            erlang:halt(2);\n        {error, Reason} ->\n            io:format(\"You failed to read the ctlfile ~s~n\"\n                      \"error was: <~p>~n\"\n                      \"specify by <-I id> which yaws system you want\"\n                      \" to control~n\",\n                      [yaws:ctl_file(SID), Reason]),\n            timer:sleep(10),\n            erlang:halt(3);\n        {ok, Socket, Key} ->\n            gen_tcp:send(Socket, term_to_binary({Term, Key})),\n            Ret = s_cmd(Socket, SID, 0),\n            timer:sleep(40), %% sucks bigtime, we have no good way to flush io\n            case Ret of\n                ok when Term == stop ->\n                    %% wait for Yaws node to truly stop.\n                    case gen_tcp:recv(Socket, 0) of\n                        {error, closed} ->\n                            erlang:halt(0);\n                        Other ->\n                            io:format(\"Stopping yaws: ~p~n\", [Other]),\n                            erlang:halt(3)\n                    end;\n                ok ->\n                    erlang:halt(0);\n                error ->\n                    erlang:halt(4)\n            end\n    end.\n\n\ns_cmd(Fd, SID, Count) ->\n    case gen_tcp:recv(Fd, 0) of\n        {ok, Bin} ->\n            io:format(\"~s\", [binary_to_list(Bin)]),\n            s_cmd(Fd, SID, Count+1);\n        {error, closed} when Count > 0 ->\n            gen_tcp:close(Fd);\n        Err ->\n            io_lib:format(\"yaws server for yaws id <~p> not \"\n                          \"responding: ~p ~n\", [SID, Err]),\n            error\n    end.\n\n\n\n%% List existing yaws nodes on this machine for this user\nls(_) ->\n    Dir = filename:join([yaws:tmpdir(), \"yaws\"]),\n    case file:list_dir(Dir) of\n        {ok, List} ->\n            io:format(\"~-15s~-10s~-10s~n\",\n                      [\"Id\", \"Status\", \"Owner\"]),\n            io:format(\"-------------------------------------~n\",[]),\n            lists:foreach(\n              fun(IdDir) ->\n                      lls(IdDir)\n              end, List);\n        _ ->\n            ok\n    end,\n    init:stop().\n\n\nlls(IdDir) ->\n    CtlFile = yaws:ctl_file(IdDir),\n    case {file:read_file_info(CtlFile),\n          file:read_file(CtlFile)} of\n        {{ok, FI}, {error, eacces}} ->\n            User = yaws:uid_to_name(FI#file_info.uid),\n            io:format(\"~-15s~-10s~-10s~n\",\n                      [IdDir, \"noaccess\", User]);\n        {{ok, FI}, {ok, _Bin}} ->\n            Running = case connect(IdDir) of\n                          {ok, Sock, _Key} ->\n                              gen_tcp:close(Sock),\n                              \"running\";\n                          {error, timeout} ->\n                              \"hanging??\";\n                          {error, eacces} ->\n                              \"noaccess\";\n                          _Err ->\n                              \"stopped\"\n                      end,\n            User = yaws:uid_to_name(FI#file_info.uid),\n            io:format(\"~-15s~-10s~-10s~n\",\n                      [IdDir, Running, User]);\n        _ ->\n            ok\n    end.\n\n\n\n%% send a hup (kindof) to the yaws server to make it\n%% reload its configuration and clear its caches\n\nhup([SID]) ->\n    actl(SID, hup).\n\n\n%% stop a daemon\nstop([SID]) ->\n    actl(SID, stop).\n\n%% query a daemon for status/stats\nstatus([SID]) ->\n    actl(SID, status).\n\nload(X) ->\n    [SID | Modules] = lists:reverse(X),\n    actl(SID, {load, Modules}).\n\ncheck([Id, File| IncludeDirs]) ->\n    GC = yaws_config:make_default_gconf(false, atom_to_list(Id)),\n    GC2 = GC#gconf{include_dir = lists:map(fun(X) -> atom_to_list(X) end,\n                                           IncludeDirs)},\n    put(gc, GC2),\n    put(check_yaws_script, true),\n    case yaws_compile:compile_file(atom_to_list(File)) of\n        {ok, 0, _Spec} ->\n            timer:sleep(100),erlang:halt(0);\n        _Other ->\n            timer:sleep(100),erlang:halt(1)\n    end.\n\n%% control a daemon http/traffic tracer\ntrace([What, SID]) ->\n    actl(SID, {trace, What}).\n\ndebug_dump([SID]) ->\n    actl(SID, debug_dump).\n\nstats([SID]) ->\n    actl(SID, stats).\nrunning_config([SID]) ->\n    actl(SID, running_config).\n\nconfigtest([File]) ->\n    Env = #env{debug = false, conf  = {file, File}},\n    case catch yaws_config:load(Env) of\n        {ok, _GC, _SCs} ->\n            io:format(\"Syntax OK~n\"),\n            timer:sleep(100),erlang:halt(0);\n        {error, Error} ->\n            io:format(\"Syntax error in file ~p:~n~s~n\", [File, Error]),\n            timer:sleep(100),erlang:halt(1);\n        Other ->\n            io:format(\"Syntax error in file ~p:~n~p~n\", [File, Other]),\n            timer:sleep(100),erlang:halt(1)\n    end.\n\nauth([User, Algo, Passwd]) ->\n    if\n        Algo == md5    orelse Algo == sha    orelse\n        Algo == sha224 orelse Algo == sha256 orelse\n        Algo == sha384 orelse Algo == sha512 orelse\n        Algo == ripemd160 ->\n            Salt    = yaws_dynopts:rand_bytes(32),\n            B64Salt = base64:encode(Salt),\n            Hash    = crypto:hash(Algo, [Salt, atom_to_list(Passwd)]),\n            B64Hash = base64:encode(Hash),\n            io:format(\"~nUser's credential successfully generated:~n\", []),\n            io:format(\"\\tPut this line in your Yaws config (in <auth> section):\"\n                      \" user = \\\"~s:{~s}$~s$~s\\\"~n~n\",\n                      [atom_to_list(User),atom_to_list(Algo),B64Salt,B64Hash]),\n            io:format(\"\\tOr in a .yaws_auth file:\"\n                      \" {\\\"~s\\\", \\\"~s\\\", \\\"~s\\\", \\\"~s\\\"}.~n\",\n                      [atom_to_list(User),atom_to_list(Algo),B64Salt,B64Hash]),\n            timer:sleep(100),erlang:halt(0);\n        true ->\n            io:format(\"Unsupported Hash algorithm ~p~n\"\n                      \"\\tUse: md5 | ripemd160 | sha | sha224 | sha256 | sha384 | sha512 ~n\", [Algo]),\n            timer:sleep(100),erlang:halt(1)\n    end.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_debug.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_debug.erl\n%%% Author  : Claes Wikstrom <klacke@hyber.org>\n%%% Purpose :\n%%% Created :  7 Feb 2002 by Claes Wikstrom <klacke@hyber.org>\n%%%----------------------------------------------------------------------\n\n-module(yaws_debug).\n-author('klacke@hyber.org').\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_debug.hrl\").\n-export([typecheck/3,\n         format_record/3,\n         assert/4,\n         format/2,format/3,\n         derror/2,\n         dinfo/2,\n         mktags/0,\n         xref/1,\n         pids/0,\n         eprof/0,\n         check_headers/1, nobin/1,\n         do_debug_dump/1\n        ]).\n\n\n\ntypecheck([{record, Rec, X} | Tail], File, Line) when is_atom(X),\n                                                      element(1, Rec) == X ->\n    typecheck(Tail, File, Line);\ntypecheck([{int, Int} |Tail], File, Line) when is_integer(Int) ->\n    typecheck(Tail, File, Line);\ntypecheck([Err|_], File, Line) ->\n    debug_format(user, \"TC ERROR ~s:~w:~n~p\",\n              [File, Line, Err]),\n    erlang:error(tcerr);\ntypecheck([], _,_) ->\n    ok.\n\n\n%% returns {record, RecName, [Field1, Val1} .....]\nformat_record(Record, Name, Fields) ->\n    case tuple_to_list(Record) of\n        [Name | Rest] ->\n            io_lib:format(\"record ~w\\n~s\", [Name,\n                                            format_record(Rest, Fields)]);\n        _X ->\n            ?Debug(\"Bad record ~p is not ~p~n\", [_X, Name]),\n            \"badrecord\"\n    end.\n\nformat_record([], []) ->\n    [];\nformat_record([Val|Vals], [F|Fs]) when is_integer(Val);\n                                       Val == [];\n                                       is_atom(Val);\n                                       is_float(Val)->\n    [io_lib:format(\"     ~w = ~w\\n\", [F,Val]),\n     format_record(Vals, Fs)];\nformat_record([Val|Vals], [F|Fs]) ->\n    case is_string(Val) of\n        true ->\n            [io_lib:format(\"     ~w = \\\"~s\\\"\\n\", [F,Val]),\n             format_record(Vals, Fs)];\n        false ->\n            [io_lib:format(\"     ~w = ~p~n\", [F, nobin(Val)]),\n             format_record(Vals, Fs)]\n    end.\n\nis_string(L) when is_list(L) ->\n    lists:filter(fun(X) when is_integer(X),\n                             $A < X, X < $z ->\n                         false;\n                    (_) ->\n                         true\n                 end,L) == [];\nis_string(_) ->\n    false.\n\n\n\nassert(equal,X,Y,_) when X==Y ->\n    ok;\nassert(neq,X,Y,_) when X/=Y ->\n    ok;\nassert(integer,X,_,_) when is_integer(X) ->\n    ok;\nassert(list,X,_,_) when is_list(X) ->\n    ok;\nassert({list,length,equal},X,Y,_) when is_list(X), length(X)==Y ->\n    ok;\nassert(greater,X,Y,_) when is_integer(X), is_integer(Y), X>Y ->\n    ok;\nassert(min,X,Y,_) when is_integer(X), is_integer(Y), X>=Y ->\n    ok;\nassert(lesser,X,Y,_) when is_integer(X), is_integer(Y), X<Y ->\n    ok;\nassert(max,X,Y,_) when is_integer(X), is_integer(Y), X=<Y ->\n    ok;\nassert(interval,X,{Min,Max},_) when is_integer(X), is_integer(Min),\n                                    is_integer(Max),\n                                    X>=Min, Max>=X ->\n    ok;\nassert('fun', Fun, _, Failure) ->\n    case catch Fun() of\n        true -> ok;\n        _Other -> fail(Failure)\n    end;\n\nassert(in,X,L,Failure) when is_list(L) ->\n    case lists:member(X,L) of\n        true -> ok;\n        _ -> fail(Failure)\n    end;\n\nassert(_,_,_,Failure) ->\n    fail(Failure).\n\nfail({assert,File,Line,Message}) ->\n    debug_format(user, \"Assertion FAILED ~p:~p, pid ~w exiting: ~p~n\",\n              [File, Line, self(), Message]),\n    erlang:error(assertion_failed);\nfail({alert,File,Line,Message}) ->\n    debug_format(user, \"Assert WARNING ~p:~p, pid ~w: ~p~n\",\n              [File, Line, self(), Message]),\n    ok;\nfail({{debug,Fstr}, File,Line,Fmt, Args}) ->\n    Str = lists:flatten(\n            io_lib:format(\"~s <~p> ~s:~p, pid ~w: ~n\",\n                          [Fstr, node(), filename:basename(File),\n                           Line, self()])),\n\n    case (catch debug_format(user, Str ++ Fmt ++ \"~n\", Args)) of\n        ok -> ok;\n        _ -> debug_format(user, \"ERROR ~p:~p: Pid ~w: (bad format)~n~p,~p~n\",\n                       [File, Line, self(), Fmt, Args]),\n\n             ok\n    end;\n\nfail({format, File,Line,Fmt,Args}) ->\n    case (catch debug_format(user, Fmt,Args)) of\n        ok -> ok;\n        _ ->\n            debug_format(user, \"ERROR ~p:~p: Pid ~w: (bad format)~n~p,~p~n\",\n                      [File, Line, self(), Fmt, Args]),\n\n            ok\n    end.\n\ndebug_format(_, F, D) ->\n    debug_format(F, D).\n\ndebug_format(F, A) ->\n    Str = case catch io_lib:format(\"yaws debug: \" ++ F, A) of\n        {'EXIT', Reason} ->\n            io_lib:format(\"yaws debug: F=~s A=~p (failed to format: ~p)\",\n                [F, A, Reason]);\n        Ok -> Ok\n    end,\n    error_logger:info_msg(Str),\n    catch io:format(F, A),\n    ok.\n\nformat(F, A) ->\n    format(get(gc), F, A).\nformat(GC, F, A) ->\n    case ?gc_has_debug(GC) of\n        true ->\n            error_logger:info_msg(\"yaws debug:\" ++ F, A);\n        false ->\n            ok\n    end.\n\nderror(F, A) ->\n    case ?gc_has_debug((get(gc))) of\n        true ->\n            error_logger:error_msg(\"yaws:\" ++ F, A);\n        false ->\n            ok\n    end.\n\ndinfo(F, A) ->\n    case ?gc_has_debug((get(gc))) of\n        true ->\n            error_logger:info_msg(\"yaws:\" ++ F, A);\n        false ->\n            ok\n    end.\n\n\nmktags() ->\n    tags:dirs([\".\"]),\n    init:stop().\n\n\n\nxref([Dir]) ->\n    debug_format(\"~p~n\", [xref:d(Dir)]),\n    init:stop().\n\n\npids() ->\n    lists:zf(\n      fun(P) ->\n              case process_info(P) of\n                  L when is_list(L) ->\n                      {value, {_, {M1, _,_}}} =\n                          lists:keysearch(current_function, 1, L),\n                      {value, {_, {M2, _,_}}} =\n                          lists:keysearch(initial_call, 1, L),\n                      S1= atom_to_list(M1),\n                      S2 = atom_to_list(M2),\n                      case {S1, S2} of\n                          {\"yaws\" ++ _, _} ->\n                              {true, P};\n                          {_, \"yaws\"++_} ->\n                              {true, P};\n                          _ ->\n                              false\n                      end;\n                  _ ->\n                      false\n              end\n      end,\n      processes()).\n\n\n\neprof() ->\n    eprof:start(),\n    eprof:profile(pids()),\n    debug_format(\"Ok run some traffic \\n\", []).\n\n\n\n-define(h_check(H, Field),\n        f_check(H#outh.Field, Field)).\n\nf_check(undefined, _Field) ->\n    ok;\nf_check(Str, Field) ->\n    case lists:reverse(lists:flatten(Str)) of\n        [$\\n, $\\r , H | _Tail] ->\n            case lists:member(H, [$\\n, $\\r]) of\n                true ->\n                    error_logger:format(\"Bad <~p> header:~n\"\n                                        \"  ~p~nToo many newlines\",\n                                        [Field, Str]),\n                    exit(normal);\n                false ->\n                    ok\n            end;\n        _Other ->\n            error_logger:format(\"Bad <~p> header:~n\"\n                                \"~p~nNot ending with CRNL~n\",\n                                [Field, Str]),\n            exit(normal)\n    end.\n\ncheck_headers(H) ->\n    ?h_check(H, connection),\n    ?h_check(H, server),\n    ?h_check(H, location),\n    ?h_check(H, cache_control),\n    ?h_check(H, date),\n    ?h_check(H, allow),\n    ?h_check(H, last_modified),\n    ?h_check(H, etag),\n    ?h_check(H, content_range),\n    ?h_check(H, content_length),\n    ?h_check(H, content_encoding),\n    ?h_check(H, set_cookie),\n    ?h_check(H, transfer_encoding),\n    ?h_check(H, www_authenticate),\n    check_other(H#outh.other).\n\n\ncheck_other(undefined) ->\n    ok;\ncheck_other(L0) ->\n    L = lists:flatten(L0),\n    case lists:dropwhile(fun(X) -> not lists:member(X, [\"\\r\\n\"]) end, L) of\n        [] ->\n            ok;\n        [$\\r, $\\n, H | _Tail] ->\n            case lists:member(H, [$\\n, $\\r]) of\n                true ->\n                    bad_other(L);\n                false ->\n                    ok\n            end;\n        _Other ->\n            bad_other(L)\n    end.\n\n\nbad_other(L) ->\n    Bad = lists:takewhile(\n            fun(X) -> not lists:member(X, [\"\\r\\n\"]) end, L),\n    error_logger:format(\"Bad header:~p~n\"\n                        \"Too many newlines\",\n                        [Bad]),\n    exit(normal).\n\n\n\n\nnobin(X) ->\n    case catch xnobin(X) of\n        {'EXIT', Reason} ->\n            error_logger:format(\"~p~n~p~n\", [X, Reason]),\n            erlang:error(Reason);\n        Res ->\n            Res\n    end.\n\n\nxnobin(B) when is_binary(B) ->\n    lists:flatten(io_lib:format(\"#Bin(~w)\", [size(B)]));\nxnobin(L) when is_list(L) ->\n    lists:map(fun(X) -> xnobin(X) end, L);\nxnobin(T) when is_tuple(T) ->\n    list_to_tuple(xnobin(tuple_to_list(T)));\nxnobin(X) ->\n    X.\n\n\n\n%%%%%%%%%%%%%%% debug dump %%%%%%%%%%%%%%%%%%%%%%%\n\n\ndo_debug_dump(Socket) ->\n    gen_version(Socket),\n    gen_sep(Socket),\n    %% keep proc status last, to report on hangs for the others\n    CollectOS = gen_os(Socket),\n    Collect = lists:foldl(fun({F, Str}, Acc) ->\n                                  Ret = collect(F, Socket, Str),\n                                  gen_sep(Socket),\n                                  [Ret|Acc]\n                          end,\n                          CollectOS,\n                          [{fun send_status/1, \"Yaws status\"},\n                           {fun send_inet/1,   \"Inet status\"},\n                           {proc_status_fun(), \"process status\"}]),\n    lists:foreach(fun(ok) ->\n                          ok;\n                     ({pid, Pid}) ->\n                          exit(Pid, shutdown)\n                  end, Collect),\n    ok.\n\n\ngen_version(Socket) ->\n    sock_format(Socket, \"Yawsvsn: ~p~n\", [yaws_generated:version()]).\n\ngen_os(Socket) ->\n    OSType = os:type(),\n    [gen_oscmd(Socket, \"uname -a\"),\n     gen_oscmd(Socket, \"ifconfig -a\"),\n     gen_oscmd(Socket, top_cmd(OSType)),\n     gen_oscmd(Socket, netstat_cmd(OSType))].\n\ngen_oscmd(Socket, Cmd) ->\n    F = fun(Sock) ->\n                sock_format(Sock, \"~s:~n~s~n\", [Cmd, os:cmd(Cmd)])\n        end,\n    Ret = collect(F, Socket, Cmd),\n    gen_sep(Socket),\n    Ret.\n\n%% FIXME The 'top -b -n 1' invocation is actually for version 3.2.x\n%% typically(?) found on Linux, while the 'top -b' is for (e.g.) version\n%% 3.5.x typically(?) found on *BSD. For the latter, '-b' itself means\n%% \"run only once\", '-n' is an alias for '-b', and '1' means show only\n%% one process. Obviously there is a problem if 3.2.x or equivalent ends\n%% up getting invoked w/o '-n 1', since it will loop forever...\ntop_cmd({unix, linux}) -> \"top -b -n 1\";\ntop_cmd({unix, sunos}) -> \"top -b -d 2 -s 1 || /usr/ucb/ps -auxww\";\ntop_cmd({unix, qnx})   -> \"pidin times; pidin pmem\";\ntop_cmd({unix, darwin}) -> \"top -o cpu -l 1\";\ntop_cmd(_)             -> \"top -b\".\n\nnetstat_cmd({unix, linux})   -> \"netstat -ant\";\nnetstat_cmd({unix, freebsd}) -> \"netstat -an -p tcp\";\nnetstat_cmd({unix, sunos})   -> \"netstat -an -P tcp\";\nnetstat_cmd(_)               -> \"netstat -an\".\n\ngen_sep(Socket) ->\n    sock_format(Socket,\"~n~s~n\", [lists:duplicate(40, $*)]).\n\n\nproc_status_fun() ->\n    fun(Fd) ->\n            sock_format(Fd, \"Process status:~n\", []),\n            i1(Fd, processes())\n    end.\n\ni1(Fd, Ps) ->\n    Alive = lists:filter(fun palive/1, Ps),\n    i2(Fd, Alive),\n    case lists:filter(fun pzombie/1, Ps) of\n        [] ->\n            ok;\n        Zombies ->\n            %% Zombies is not the same as Ps-Alive, since the remote\n            %% process that fetched Ps is included among Alive, but has\n            %% exited (for ni/0).\n            sock_format(Fd, \"\\nDead processes:\\n\", []),\n            i2(Fd, Zombies)\n    end.\n\ni2(Fd, Ps) ->\n    iformat(Fd, \"Pid\", \"Initial Call\", \"Current Function\", \"Reds\", \"Msgs\",\n            \"Heap\", \"Stack\"),\n    {Reds,Msgs,Heap,Stack,Susp1,Susp2,MemSusp,_} =\n        lists:foldl(fun display_info/2, {0,0,0,0,[],[],[],Fd}, Ps),\n    iformat(Fd, \"Total\", \"\", \"\", io_lib:write(Reds), io_lib:write(Msgs),\n            io_lib:write(Heap), io_lib:write(Stack)),\n    lists:foreach(fun(Susp) -> display_susp1(Fd, Susp) end, Susp1),\n    lists:foreach(fun(Susp) -> display_susp2(Fd, Susp) end, Susp2),\n    lists:foreach(fun(Susp) -> display_susp3(Fd, Susp) end, MemSusp).\n\n\npalive(Pid) ->\n    case process_info(Pid, status) of\n        undefined         -> false;\n        {status, exiting} -> false;\n        _                 -> true\n    end.\n\npzombie(Pid) ->\n    case process_info(Pid, status) of\n        undefined         -> false;\n        {status, exiting} -> true;\n        _                 -> false\n    end.\n\n\n-define(MEM_LARGE, 40000).\n\ndisplay_info(Pid, {R,M,H,St,S1,S2,S3,Fd}) ->\n    case process_info(Pid) of\n        undefined ->\n            {R, M};\n        Info ->\n            Call = initial_call(Info),\n            Curr = fetch(current_function, Info),\n            Reds = fetch(reductions, Info),\n            LM = fetch(message_queue_len, Info),\n            Heap = fetch(heap_size, Info),\n            Stack = fetch(stack_size, Info),\n            Mem = case process_info(Pid, memory) of\n                      undefined -> 0;\n                      {memory, Int} -> Int\n                  end,\n            iformat(Fd,\n                    io_lib:write(Pid),\n                    mfa_string(Call),\n                    mfa_string(Curr),\n                    io_lib:write(Reds),\n                    io_lib:write(LM),\n                    io_lib:write(Heap),\n                    io_lib:write(Stack)),\n            %% if it got msgs, it's suspicios\n            NS1 = if LM > 0 -> [{Pid, Reds, LM} | S1];\n                     true -> S1\n                  end,\n            %% if it's in gen:wait_resp* it's suspicios\n            NS2 = case Curr of\n                      {gen, wait_resp, _} -> [{Pid, Reds} | S2];\n                      {gen, wait_resp_mon, _} -> [{Pid, Reds} | S2];\n                      _ -> S2\n                  end,\n            %% If it is large .. it is suspicios\n            NS3 = if Mem > ?MEM_LARGE ->\n                          [{Pid, Mem} | S3];\n                     true ->\n                          S3\n                  end,\n            {R+Reds, M+LM, H+Heap,St+Stack, NS1, NS2, NS3, Fd}\n    end.\n\ndisplay_susp1(Fd, {Pid, Reds0, LM0}) ->\n    case process_info(Pid) of\n        undefined ->\n            ok;\n        Info ->\n            Reds1 = fetch(reductions, Info),\n            LM1 = fetch(message_queue_len, Info),\n            Msgs = fetch(messages, Info),\n            Bt = case process_info(Pid, backtrace) of\n                     {backtrace, Bin} ->\n                         binary_to_list(Bin);\n                     _ ->\n                         []\n                 end,\n            if LM1 > 0 ->\n                    %% still suspicious\n                    sock_format(Fd,\n                                \"*** Suspicious *** : ~-12w, Qlen = ~4w/~-4w, \"\n                                \"Reds = ~12w/~-12w\\n\",\n                                [Pid, LM0, LM1, Reds0, Reds1]),\n                    lists:foreach(\n                      fun(Msg) -> sock_format(Fd, \"  ~p\\n\",[Msg]) end,\n                      Msgs),\n                    gen_sep(Fd),\n                    sock_format(Fd, \"\\n\\n\\n\\n*** Backtrace *** for ~w\\n~s\\n\",\n                                [Pid,Bt]);\n               true ->\n                    ok\n            end\n    end.\n\ndisplay_susp2(Fd, {Pid, Reds0}) ->\n    case process_info(Pid, reductions) of\n        undefined ->\n            ok;\n        {reductions, Reds0} ->\n            %% it hasn't done any work... print bt\n            case process_info(Pid, backtrace) of\n                {backtrace, Bin} ->\n                    gen_sep(Fd),\n                    sock_format(Fd, \"\\n\\n\\n\\n*** Backtrace (gen_wait) \"\n                                \"*** for ~w\\n~s\\n\",\n                                [Pid, binary_to_list(Bin)]);\n                _ ->\n                    ok\n            end;\n        _ ->\n            ok\n    end.\n\ndisplay_susp3(Fd, {Pid, _Mem}) ->\n    case {process_info(Pid, memory), process_info(Pid, current_function)} of\n        {undefined, _} ->\n            ok;\n        {_, {current_function,{yaws_debug,display_susp3,2}}} ->\n            ok;\n        {{memory, Mem2}, _} when Mem2 > ?MEM_LARGE ->\n            %% it's still too big\n            case process_info(Pid, backtrace) of\n                {backtrace, Bin} ->\n                    gen_sep(Fd),\n                    sock_format(Fd,\n                                \"\\n\\n\\n\\n*** Backtrace (mem=~p) \"\n                                \"*** for ~w\\n~p~n~s\\n\",\n                                [Mem2, Pid, process_info(Pid),\n                                 binary_to_list(Bin)]);\n                _ ->\n                    ok\n            end;\n        _ ->\n            ok\n    end.\n\n\n\ninitial_call(Info)  ->\n    case fetch(initial_call, Info) of\n        {proc_lib, init_p, 5} ->\n            proc_lib:translate_initial_call(Info);\n        ICall ->\n            ICall\n    end.\n\nmfa_string({M, F, A}) ->\n    io_lib:format(\"~w:~w/~w\", [M, F, A]);\nmfa_string(X) ->\n    io_lib:write(X).\n\nfetch(Key, Info) ->\n    case lists:keysearch(Key, 1, Info) of\n        {value, {_, Val}} -> Val;\n        false -> 0\n    end.\n\niformat(Fd, A1, A2, A3, A4, A5, A6, A7) ->\n    sock_format(Fd, \"~-12s ~-23s ~-23s ~12s ~4s ~12s ~10s\\n\",\n                [A1,A2,A3,A4,A5,A6,A7]).\n\nsock_format(Sock, Fmt, Args) ->\n    gen_tcp:send(Sock, io_lib:format(Fmt, Args)).\n\n-define(COLLECT_TIMEOUT, 10000).\n\n\n%% purpose of this collect function is to not hang, remember a probable\n%% reason for running debug-dump is that the system is in a\n%% corrupt state.\n\ncollect(F, Sock, User) ->\n    SELF = self(),\n    Pid = spawn(fun() ->\n                        F(Sock),\n                        SELF ! {self(), ok},\n                        timer:sleep(infinity)\n                end),\n    Ref = erlang:monitor(process, Pid),\n    receive\n        {Pid, ok} ->\n            erlang:demonitor(Ref),\n            exit(Pid, shutdown),\n            ok;\n        Down = {'DOWN', Ref, _,_,_} ->\n            sock_format(Sock, \"*** Failed to collect ~s: ~p~n\", [User, Down]),\n            ok\n    after ?COLLECT_TIMEOUT ->\n            erlang:demonitor(Ref),\n            sock_format(Sock, \"*** Failed to collect ~s: timeout~n\", [User]),\n            {pid, Pid}  % Let it hang for proc status, exit after\n    end.\n\nsend_status(Sock) ->\n    {InitStatus, _} = init:get_status(),\n    sock_format(Sock, \"vsn: ~s\\n\", [yaws_generated:version()]),\n    sock_format(Sock, \"status: ~p\\n\", [InitStatus]),\n    ok.\n\nsend_inet(Sock) ->\n    Chars = capture_io(fun() -> inet:i() end),\n    sock_format(Sock, \"inet:i() output ~n~s~n\", [Chars]),\n    ok.\n\n\n%% This function runs a Fun that is producing IO through\n%% io:format() and collects the IO and retuns the IO as a char list\n%% Returns io_list() | {timeout, io_list()}\n%%\ncapture_io(Fun) ->\n    do_capture_io(Fun).\n\n%% capture_io(Fun, MilliSecTimeout) ->\n%%    {ok, Tref} = timer:send_after(MilliSecTimeout, capio_timeout),\n%%    Chars = do_capture_io(Fun),\n%%    timer:cancel(Tref),\n%%    Chars.\n\ndo_capture_io(Fun) ->\n    Pid = spawn(fun() ->\n                        receive run -> ok end,\n                        Fun()\n                end),\n    Mref = erlang:monitor(process,Pid),\n    group_leader(self(), Pid),\n    Pid ! run,\n    collect_io(Pid, Mref, []).\n\ncollect_io(Pid, Mref, Ack) ->\n    receive\n        {'DOWN', Mref, _,_,_} ->\n            Ack;\n        {io_request, From, Me, {put_chars, M, F, A}} ->\n            From ! {io_reply, Me, ok},\n            collect_io(Pid, Mref, [Ack, apply(M,F, A)]);\n        {io_request, From, Me, {put_chars, unicode, M, F, A}} ->\n            From ! {io_reply, Me, ok},\n            collect_io(Pid, Mref, [Ack, apply(M,F, A)]);\n        capio_timeout ->\n            {timeout, Ack}\n    end.\n\n\n\n\n\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_debug.hrl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_debug.hrl\n%%% Author  : Claes Wikstrom <klacke@hyber.org>\n%%% Purpose :\n%%% Created :  7 Feb 2002 by Claes Wikstrom <klacke@hyber.org>\n%%%----------------------------------------------------------------------\n\n-author('klacke@hyber.org').\n\n\n\n-define(F(Format,Args),\n         lists:flatten(io_lib:format(Format,Args))).\n\n-define(f(L), lists:flatten(L)).\n\n-define(W(A), ?F(\"~w\", [A])).\n\n-define(format_record(Rec, Name),\n        yaws_debug:format_record(Rec, Name, record_info(fields, Name))).\n\n\n\n-define(Trace(What, Fmt, Args), if Trace == false ->\n                                        ok;\n                                   _ ->\n                                        yaws_debug:dtrace(What,Fmt,Args)\n                                end).\n\n\n\n-ifdef(debug).\n\n\n%% Possible Ops are, equal | neq | integer | list | {list, length, equal}\n%% | greater | min | max | interval | {in, X , List}\n\n\n\n\n-define(Dassert(X,Op,Y,Msg),\n        yaws_debug:assert(Op,X,Y,{assert,?FILE,?LINE,Msg})).\n\n\n-define(Dalert(X,Op,Y,Msg),\n        yaws_debug:assert(Op,X,Y,{alert,?FILE,?LINE,Msg})).\n\n-define(Deval(Expr),Expr).\n\n-define(Debug(F, A),\n        yaws_debug:assert([],0,0,{{debug,\"DEBUG\"}, ?FILE,?LINE,F, A})).\n\n\n%% ease of use, just do ?Dvar(Variable)\n-define(Dvar(Var), ?Debug(\"Var = ~p~n\", [Var])).\n\n-define(TC(L), yaws_debug:typecheck(L, ?FILE, ?LINE)).\n\n\n-define(Derror(Fmt,Args),\n        yaws_debug:assert([],0,0,{{debug,\"ERROR\"}, ?FILE,?LINE,Fmt,Args})).\n\n\n-define(Dformat(Fmt,Args),\n        yaws_debug:assert([],0,0,{format, ?FILE,?LINE,Fmt,Args})).\n\n\n-define(Dfunassert(Fun, Msg),\n        yaws_debug:assert('fun', Fun, 0, {assert,?FILE,?LINE,Msg})).\n\n-else. %% not debug_mode\n\n-define(DLOG(F, A), ?LOG(F, A)).\n-define(Dassert(X,Op,Y,Msg),debug_disabled).\n-define(Dalert(X,Op,Y,Msg),debug_disabled).\n-define(Deval(Expr),debug_disabled).\n-define(Debug(F, A),debug_disabled).\n-define(Dvar(Var), debug_disabled).\n-define(Dformat(Fmt,Args),debug_disabled).\n-define(Dfunassert(Fun, Msg), debug_disabled).\n-define(Derror(Fmt,Args),debug_disabled).\n-define(TC(L), debug_disabled).\n\n-endif. %% debug defined\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_dime.erl",
    "content": "%%%-------------------------------------------------------------------\n%%% File    : dime.erl\n%%% @author Anders Nygren <anders.nygren@gmail.com>\n%%% @doc Encoding and decoding of DIME messages.\n%%% The Direct Internet Message Encapsulation (DIME) specification\n%%% defines a mechanism for packaging binary data with SOAP messages.\n%%% http://bgp.potaroo.net/ietf/all-ids/draft-nielsen-dime-02.txt\n%%% Layout of a DIME encoded message is like this\n%%%<pre>\n%%%  0                   1                   2                   3\n%%%  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1\n%%% +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n%%% |         |M|M|C|       |       |                               |\n%%% | VERSION |B|E|F| TYPE_T| RESRVD|         OPTIONS_LENGTH        |\n%%% +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n%%% |            ID_LENGTH          |           TYPE_LENGTH         |\n%%% +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n%%% |                          DATA_LENGTH                          |\n%%% +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n%%% |                                                               /\n%%% /                     OPTIONS + PADDING                         /\n%%% /                                                               |\n%%% +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n%%% |                                                               /\n%%% /                          ID + PADDING                         /\n%%% /                                                               |\n%%% +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n%%% |                                                               /\n%%% /                        TYPE + PADDING                         /\n%%% /                                                               |\n%%% +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n%%% |                                                               /\n%%% /                        DATA + PADDING                         /\n%%% /                                                               |\n%%% +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\n%%%<pre>\n%%% @end\n%%%\n%%% Created :  7 Apr 2008 by Anders Nygren <>\n%%%-------------------------------------------------------------------\n-module(yaws_dime).\n\n%% API\n-export([encode/2,\n         decode/1,\n         pad_len/1]).\n\n\n-include_lib(\"kernel/include/file.hrl\").\n\n-define(VERSION1, 1).\n-define(T_UNCHANGED, 0).\n-define(T_MEDIA_TYPE, 1).\n-define(T_ABS_URI, 2).\n-define(T_UNKNOWN, 3).\n-define(T_NONE, 4).\n-define(SOAP_URI, \"http://schemas.xmlsoap.org/soap/envelope\").\n%%====================================================================\n%% API\n%%====================================================================\n%%--------------------------------------------------------------------\n%% @spec (Req, Attachments) ->binary()\n%% Description:\n%%--------------------------------------------------------------------\nencode(Req, []) ->\n    encode_part(1, 1, 0, ?T_ABS_URI, <<\"\">>, <<\"\">>, <<?SOAP_URI>>, Req);\nencode(Req, As) ->\n        list_to_binary([encode_part(1, 0, 0, ?T_ABS_URI,\n                                    <<\"\">>, <<\"\">>, <<?SOAP_URI>>, Req)|\n        encode_attachments(As)]).\n\nencode_attachments([{attachment, Id, Type, File}]) ->\n    [encode_part(0, 1, 0, ?T_ABS_URI, <<\"\">>, list_to_binary(Id),\n                 list_to_binary(Type), File)];\nencode_attachments([{attachment, Id, Type, File} | As]) ->\n    [encode_part(0, 0, 0, ?T_ABS_URI, <<\"\">>, list_to_binary(Id),\n                 list_to_binary(Type), File)|\n     encode_attachments(As)].\n\nencode_part(MB, ME, CF, TypeT, Opts, ID, Type, Data) ->\n    Opts_len = size_of(Opts),\n    Opts_pad = pad_len(Opts_len),\n    Id_len = size_of(ID),\n    Id_pad = pad_len(Id_len),\n    Type_len = size_of(Type),\n    Type_pad = pad_len(Type_len),\n    Data_len = size_of(Data),\n    Data_pad = pad_len(Data_len),\n    Data1 = get_data(Data),\n    <<?VERSION1:5, MB:1, ME:1, CF:1, TypeT:4, 0:4,\n     Opts_len:1/big-integer-unit:16,\n     Id_len:1/big-integer-unit:16,\n     Type_len:1/big-integer-unit:16,\n     Data_len:1/big-integer-unit:32,\n     Opts:Opts_len/binary-unit:8, 0:Opts_pad/integer-unit:8,\n     ID:Id_len/binary-unit:8,     0:Id_pad/integer-unit:8,\n     Type:Type_len/binary-unit:8, 0:Type_pad/integer-unit:8,\n     Data1:Data_len/binary-unit:8, 0:Data_pad/integer-unit:8>>.\n\npad_len(Len) ->\n    case Len rem 4 of\n        0 ->\n            0;\n        N ->\n            4-N\n    end.\n\nsize_of(X) when is_list(X) ->\n    length(X);\nsize_of(X) when is_binary(X)->\n    size(X);\nsize_of({file, File}) ->\n    {ok,R} = file:read_file_info(File),\n    R#file_info.size.\n\nget_data({file, File}) ->\n    {ok, Data} = file:read_file(File),\n    Data;\nget_data(Data) when is_list(Data) ->\n    list_to_binary(Data);\n\nget_data(Data) ->\n    Data.\n%%--------------------------------------------------------------------\n%% @spec (Msg::binary()) ->\n%% @doc Decode a DIME encoded message.\n%%--------------------------------------------------------------------\ndecode(Msg) ->\n    decode_recs(Msg, [], []).\n\ndecode_recs(<<>>, Acc, _Chunks) ->\n    Acc;\ndecode_recs(Msg, Acc, Chunks) ->\n    case {decode_rec(Msg), Chunks} of\n        %% A normal record\n        {{_ME=0, _CF=0, Opts, ID, Type, Chunk, More}, []} ->\n            decode_recs(More, [{Opts, ID, Type, Chunk}|Acc], []);\n\n        %% The last chunk of a block, but not the last record\n        {{_ME=0, _CF=0, Opts, ID, Type, Chunk, More}, Chunks} ->\n            Cs = lists:reverse([{Opts, ID, Type, Chunk}|Chunks]),\n            %% Only the first chunk has values for Opts, Id and Type\n            {ROpts, RID, RType, _Chunk} = hd(Cs),\n            Block = list_to_binary([Ch || {_Opts, _ID, _Type, Ch} <- Cs]),\n            error_logger:info_report([?MODULE, decode_rec, merging,\n                                      {id, ID},{type, Type},\n                                      {chunks, length(Cs)}]),\n            decode_recs(More, [{ROpts, RID, RType, Block}|Acc], []);\n\n        %% Last record, but not chunked\n        {{_ME=1, _CF=0, Opts, ID, Type, Data, _More}, []} ->\n            lists:reverse([{Opts, ID, Type, Data}|Acc]);\n\n        %% Last record, and last chunk of block\n        {{_ME=1, _CF=0, Opts, ID, Type, Chunk, _More}, Chunks} ->\n            Cs = lists:reverse([{Opts, ID, Type, Chunk}|Chunks]),\n            {ROpts, RID, RType, _Chunk} = hd(Cs),\n            Block = list_to_binary([Ch || {_Opts, _ID, _Type, Ch} <- Cs]),\n            error_logger:info_report([?MODULE, decode_rec, merging,\n                                      {id, ID},{type, Type},\n                                      {chunks, length(Cs)}]),\n            lists:reverse([{ROpts, RID, RType, Block}|Acc]);\n\n        %% First or intermediate chunk, but not the last\n        {{_ME=0, _CF=1, Opts, ID, Type, Data, More}, Chunks} ->\n            decode_recs(More, Acc, [{Opts, ID, Type, Data}|Chunks]);\n\n        %% Something wrong, ME=1 and CF=1\n        ErrorResult ->\n                error_logger:error_report([?MODULE, decode_recs,\n                                           {result, ErrorResult}])\n    end.\n\ndecode_rec(<<?VERSION1:5, _MB:1, ME:1, CF:1, _Type_T:4, _Res:4,\n            Opt_Len:1/big-integer-unit:16,\n            ID_Len:1/big-integer-unit:16,\n            Type_Len:1/big-integer-unit:16,\n            Data_Len:1/big-integer-unit:32,\n            Rest/binary>>=Block) ->\n    error_logger:info_report([?MODULE, decode_rec,\n                              {version,1},\n                              {mb,_MB},{me,ME},{cf,CF},{type_t,_Type_T},\n                              {res,_Res},{opt_len,Opt_Len},{id_len,ID_Len},\n                              {type_len,Type_Len},{date_len,Data_Len},\n                              {total_rec_size,byte_size(Block)}]),\n    Opt_pad = pad_len(Opt_Len),\n    Id_pad = pad_len(ID_Len),\n    Type_pad = pad_len(Type_Len),\n    Data_pad = pad_len(Data_Len),\n    Header = Opt_Len+Opt_pad+ID_Len+Id_pad+Type_Len+Type_pad,\n    DataTot = Data_Len+Data_pad,\n    case byte_size(Rest) of\n        N when N >= Header+DataTot ->\n            <<Opts:Opt_Len/binary-unit:8, _:Opt_pad/binary-unit:8,\n             ID:ID_Len/binary-unit:8,     _:Id_pad/binary-unit:8,\n             Type:Type_Len/binary-unit:8, _:Type_pad/binary-unit:8,\n             Data:Data_Len/binary-unit:8, _:Data_pad/binary-unit:8,\n             More/binary >> = Rest,\n            {ME, CF, Opts, ID, Type, Data, More};\n        N when N > Header->\n            <<Opts:Opt_Len/binary-unit:8, _:Opt_pad/binary-unit:8,\n             ID:ID_Len/binary-unit:8,     _:Id_pad/binary-unit:8,\n             Type:Type_Len/binary-unit:8, _:Type_pad/binary-unit:8,\n             Data/binary>> = Rest,\n            error_logger:error_report([?MODULE, decode_rec, short_record,\n                                       {need, Header+DataTot},\n                                       {has, N}]),\n            {ME, CF, Opts, ID, Type, Data, <<>>};\n        N ->\n            error_logger:error_report([?MODULE, decode_rec, short_record,\n                                       {need, Header+DataTot},\n                                       {has, N}]),\n            {error, not_enough_data}\n    end;\n\ndecode_rec(Bin) ->\n    error_logger:error_report([?MODULE, decode_rec, short_record,\n                               no_header,\n                               {has, byte_size(Bin)}]),\n    {error, no_header}.\n\n\n%%====================================================================\n%% Internal functions\n%%====================================================================\n"
  },
  {
    "path": "contrib/yaws/src/yaws_dynopts.erl",
    "content": "-module(yaws_dynopts).\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n\n-export([\n         have_ssl_honor_cipher_order/0,\n         have_ssl_client_renegotiation/0,\n         have_ssl_sni/0,\n         have_ssl_log_alert/0,\n         have_erlang_sendfile/0,\n         have_crypto_strong_rand_bytes/0,\n         have_erlang_now/0,\n         have_rand/0,\n\n         rand_bytes/1,\n         unique_triple/0,\n         get_time_tuple/0,\n         now_secs/0,\n         random_seed/3,\n         random_uniform/1,\n\n         generate/1,\n         is_generated/0\n        ]).\n\n-export([is_greater/2, is_less/2,\n         is_greater_or_equal/2, is_less_or_equal/2]).\n\n%% SSL option honor_cipher_order was added in release 17 (ERTS >= 6.0)\nhave_ssl_honor_cipher_order() ->\n    is_greater_or_equal(erlang:system_info(version), \"6.0\").\n\n%% SSL option client_renegotiation was added in release 18 (ERTS >= 7.0)\nhave_ssl_client_renegotiation() ->\n    is_greater_or_equal(erlang:system_info(version), \"7.0\").\n\n%% SSL sni support was added in release 18  (ERTS >= 7.0)\nhave_ssl_sni() ->\n    is_greater_or_equal(erlang:system_info(version), \"7.0\").\n\n%% SSL option log_alert SSL was added in R16B02 (ERTS >= 5.10.3)\nhave_ssl_log_alert() ->\n    is_greater_or_equal(erlang:system_info(version), \"5.10.3\").\n\n%% erlang:sendfile/5 is buggy for R15 & R16 releases (ERTS < 6.0)\nhave_erlang_sendfile() ->\n    is_greater_or_equal(erlang:system_info(version), \"6.0\").\n\n%% crypto:rand_bytes/1 is deprecated since releases 19 (ERTS >= 8.0)\nhave_crypto_strong_rand_bytes() ->\n    lists:member({strong_rand_bytes, 1}, crypto:module_info(exports)).\n\n%% erlang:now/0 is deprecated since releases 18 (ERTS >= 7.0)\nhave_erlang_now() ->\n    is_less(erlang:system_info(version), \"7.0\").\n\n%% random module is deprecated since releases 19 (ERTS >= 8.0)\nhave_rand() ->\n    (code:which(rand) /= non_existing).\n\nrand_bytes(N) ->\n    crypto:strong_rand_bytes(N).\n\nunique_triple() ->\n    erlang:now().\n\nget_time_tuple() ->\n    erlang:now().\n\nnow_secs() ->\n    {M,S,_} = erlang:now(),\n    (M*1000000)+S.\n\nrandom_seed(A,B,C) ->\n    case have_rand() of\n        true  -> rand:seed(exsplus, {A,B,C});\n        false -> (fun random:seed/3)(A,B,C)\n    end.\n\nrandom_uniform(N) ->\n    case have_rand() of\n        true  -> rand:uniform(N);\n        false -> (fun random:uniform/1)(N)\n    end.\n\nis_greater         (Vsn1, Vsn2) -> compare_version(Vsn1, Vsn2) == greater.\nis_less            (Vsn1, Vsn2) -> compare_version(Vsn1, Vsn2) == less.\nis_greater_or_equal(Vsn1, Vsn2) -> not is_less(Vsn1, Vsn2).\nis_less_or_equal   (Vsn1, Vsn2) -> not is_greater(Vsn1, Vsn2).\n\ncompare_version(Vsn, Vsn) ->\n    equal;\ncompare_version(Vsn1, Vsn2) ->\n    compare_version1(string:tokens(Vsn1, \".\"), string:tokens(Vsn2, \".\")).\n\ncompare_version1([], []) ->\n    equal;\ncompare_version1([X|Rest1], [X|Rest2]) ->\n    compare_version1(Rest1, Rest2);\ncompare_version1([X1], [X2]) ->\n    %% For last digit ignore everything after the \"-\", if any\n    Y1 = lists:takewhile(fun(X) -> X /= $- end, X1),\n    Y2 = lists:takewhile(fun(X) -> X /= $- end, X2),\n    compare_digit(Y1, Y2);\ncompare_version1([X1|Rest1], [X2|Rest2]) ->\n    case compare_digit(X1, X2) of\n        equal -> compare_version1(Rest1, Rest2);\n        Else  -> Else\n    end;\ncompare_version1(_X, []) ->\n    greater;\ncompare_version1([], _X) ->\n    less.\n\ncompare_digit(X, X) ->\n    equal;\ncompare_digit(X1, X2) when length(X1) > length(X2) ->\n    greater;\ncompare_digit(X1, X2) when length(X1) < length(X2) ->\n    less;\ncompare_digit(X1, X2) ->\n    case X1 > X2 of\n        true  -> greater;\n        false -> less\n    end.\n\n%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nis_generated() -> false.\n\ngenerate(GC) ->\n    code:ensure_loaded(crypto),\n    code:ensure_loaded(inet),\n    case {filelib:is_dir(yaws:id_dir(GC#gconf.id)),\n          filelib:is_dir(yaws:tmpdir(\"/tmp\"))} of\n        {true, _} ->\n            File = filename:join(yaws:id_dir(GC#gconf.id), \"yaws_dynopts.erl\"),\n            generate1(File);\n        {_, true} ->\n            File = filename:join(yaws:tmpdir(\"/tmp\"), \"yaws_dynopts.erl\"),\n            generate1(File);\n        _ ->\n            error_logger:format(\"Cannot write yaws_dynopts.erl~n\"\n                                \"Use the default version~n\", [])\n    end.\n\ngenerate1(ModFile) ->\n    case write_module(ModFile) of\n        ok ->\n            Opts = compile_options(),\n            case compile:file(ModFile, Opts) of\n                {ok, ModName, Binary} ->\n                    case code:load_binary(ModName, [], Binary) of\n                        {module, ModName} ->\n                            ok;\n                        {error, What} ->\n                            error_logger:format(\n                              \"Cannot load module '~p': ~p~n\"\n                              \"Use the default version~n\",\n                              [ModName, What]\n                             )\n                    end;\n                _ ->\n                    error_logger:format(\"Compilation of '~p' failed: ~p~n\"\n                                        \"Use the default version~n\",\n                                        [ModFile])\n            end;\n        {error, Reason} ->\n            error_logger:format(\"Cannot write ~p: ~p~n\"\n                                \"Use the default version~n\", [ModFile, Reason])\n    end.\n\nwrite_module(ModFile) ->\n    case file:open(ModFile, [write]) of\n        {ok, Fd} ->\n            io:format(Fd, source(), []),\n            file:close(Fd),\n            ok;\n        {error, Reason} ->\n            {error, Reason}\n    end.\n\ncompile_options() ->\n    [binary, report,\n     {d, 'HAVE_SSL_HONOR_CIPHER_ORDER',    have_ssl_honor_cipher_order()},\n     {d, 'HAVE_SSL_CLIENT_RENEGOTIATION',  have_ssl_client_renegotiation()},\n     {d, 'HAVE_SSL_SNI',                   have_ssl_sni()},\n     {d, 'HAVE_SSL_LOG_ALERT',             have_ssl_log_alert()},\n     {d, 'HAVE_ERLANG_SENDFILE',           have_erlang_sendfile()}\n    ]\n        ++\n        case have_crypto_strong_rand_bytes() of\n            true  -> [{d, 'HAVE_CRYPTO_STRONG_RAND_BYTES'}];\n            false -> []\n        end\n        ++\n        case have_erlang_now() of\n            true  -> [{d, 'HAVE_ERLANG_NOW'}];\n            false -> []\n        end\n        ++\n        case have_rand() of\n            true  -> [{d, 'HAVE_RAND'}];\n            false -> []\n        end.\n\nsource() ->\n    IncDir  = yaws:get_inc_dir(),\n    Src = [\n           \"-module(yaws_dynopts).\",\n           \"\",\n           \"-include(\\\"\" ++ filename:join(IncDir, \"yaws.hrl\") ++ \"\\\").\",\n           \"-include(\\\"\" ++ filename:join(IncDir, \"yaws_api.hrl\") ++ \"\\\").\",\n           \"\",\n           \"-export([\",\n           \"    have_ssl_honor_cipher_order/0,\",\n           \"    have_ssl_client_renegotiation/0,\",\n           \"    have_ssl_sni/0,\",\n           \"    have_ssl_log_alert/0,\",\n           \"    have_erlang_sendfile/0,\",\n           \"    have_crypto_strong_rand_bytes/0,\",\n           \"    have_erlang_now/0,\",\n           \"    have_rand/0,\"\n           \"\",\n           \"    rand_bytes/1,\",\n           \"    unique_triple/0,\",\n           \"    get_time_tuple/0,\",\n           \"    now_secs/0,\",\n           \"    random_seed/3,\",\n           \"    random_uniform/1,\",\n           \"\",\n           \"    generate/1,\",\n           \"    is_generated/0\",\n           \"   ]).\",\n           \"\",\n           \"\",\n           \"generate(_) -> ok.\",\n           \"is_generated() -> true.\",\n           \"\",\n           \"have_ssl_honor_cipher_order()   -> ?HAVE_SSL_HONOR_CIPHER_ORDER.\",\n           \"have_ssl_client_renegotiation() -> ?HAVE_SSL_CLIENT_RENEGOTIATION.\",\n           \"have_ssl_sni()                  -> ?HAVE_SSL_SNI.\",\n           \"have_ssl_log_alert()            -> ?HAVE_SSL_LOG_ALERT.\",\n           \"have_erlang_sendfile()          -> ?HAVE_ERLANG_SENDFILE.\",\n           \"\",\n           \"-ifdef(HAVE_CRYPTO_STRONG_RAND_BYTES).\",\n           \"have_crypto_strong_rand_bytes() -> true.\",\n           \"rand_bytes(N) -> crypto:strong_rand_bytes(N).\",\n           \"-else.\",\n           \"have_crypto_strong_rand_bytes() -> false.\",\n           \"rand_bytes(N) -> crypto:rand_bytes(N).\",\n           \"-endif.\",\n           \"\"\n           \"-ifdef(HAVE_ERLANG_NOW).\",\n           \"have_erlang_now() -> true.\",\n           \"unique_triple() ->\",\n           \"    now().\",\n           \"get_time_tuple() ->\",\n           \"    now().\",\n           \"now_secs() ->\",\n           \"    {M,S,_} = now(),\",\n           \"    (M*1000000)+S.\",\n           \"-else.\",\n           \"have_erlang_now() -> false.\",\n           \"unique_triple() ->\",\n           \"    {erlang:unique_integer([positive]),\",\n           \"     erlang:unique_integer([positive]),\",\n           \"     erlang:unique_integer([positive])}.\",\n           \"get_time_tuple() ->\",\n           \"    erlang:timestamp().\",\n           \"now_secs() ->\",\n           \"    {M,S,_} = erlang:timestamp(),\",\n           \"    (M*1000000)+S.\",\n           \"-endif.\",\n           \"\",\n           \"-ifdef(HAVE_RAND).\",\n           \"have_rand() -> true.\",\n           \"random_seed(A,B,C) -> rand:seed(exsplus, {A,B,C}).\",\n           \"random_uniform(N)  -> rand:uniform(N).\",\n           \"-else.\",\n           \"have_rand() -> false.\",\n           \"random_seed(A,B,C) -> random:seed(A,B,C).\",\n           \"random_uniform(N)  -> random:uniform(N).\",\n           \"-endif.\",\n           \"\"\n          ],\n    string:join(Src, \"\\n\").\n"
  },
  {
    "path": "contrib/yaws/src/yaws_exhtml.erl",
    "content": "%% -*- coding: latin-1 -*-\n%%%----------------------------------------------------------------------\n%%% File    : exhtml.erl\n%%% Author  : Joakim Greben <jocke@tail-f.com>\n%%% Purpose : Format ehtml as xhtml code with optional indentation support.\n%%% Created : 24 Apr 2006 by Joakim Greben <jocke@tail-f.com>\n%%%----------------------------------------------------------------------\n\n-module(yaws_exhtml).\n-export([check_xhtml/1]).\n-export([fformat/0, sformat/0]). % test\n-export([format/1, format/2, format/3]).\n-export([sformat/1, sformat/2]).\n-export([count_trailing_spaces/1]).\n\n-define(INDENT_LEVEL, 2).\n\nfformat() -> % test\n    HTML =\n        format(\n          [{table,\n            [{tr,\n              [{td,\n                [\"foo \", {em, [\"bar\"]}, \" now.\",\n                 {hr},\n                 {p, \"foo\"}]}]},\n             {tr,\n              [{td,\n                [\"foo \", {em, [\"bar\"]}, \" now.\"]}]}]}]),\n    io:format(\"~s~n\", [lists:flatten(HTML)]).\n\nsformat() -> % test\n    HTML =\n        sformat(\n          [{table,\n            [{tr,\n              [{td,\n                [\"foo \", {em, [\"bar\"]}, \" now.\",\n                 {hr},\n                 {p, \"foo\"}]}]},\n             {tr,\n              [{td,\n                [\"foo \", {em, [\"bar\"]}, \" now.\"]}]}]}]),\n    io:format(\"~s~n\", [lists:flatten(HTML)]).\n\ncheck_xhtml(XHTMLContent) when is_list(XHTMLContent) ->\n    check_xhtml(list_to_binary(XHTMLContent));\ncheck_xhtml(XHTMLContent) when is_binary(XHTMLContent) ->\n    {ok, Filename} = yaws:mktemp(\"confd\"),\n    ok = file:write_file(Filename, XHTMLContent),\n    DTD = filename:join(code:priv_dir(webgui), \"xhtml1-strict.dtd\"),\n    Cmd = \"xmllint --dtdvalid \"++DTD++\" -noout -nonet \"++Filename++\" 2>&1\",\n    case os:cmd(Cmd) of\n        \"\" ->\n            file:delete(Filename),\n            ok;\n        Reason ->\n            file:delete(Filename),\n            {error, Reason}\n    end.\n\nformat(Data) ->\n    tl(format(block, Data, fun value2string/1, 0, [])).\n\nformat(Data, N) ->\n    tl(format(block, Data,  fun value2string/1, N, lists:duplicate(N, $ ))).\n\nformat(Data, N, Value2StringF) ->\n    tl(format(block, Data, Value2StringF, N, lists:duplicate(N, $ ))).\n\nformat(Mode, Data, Value2StringF, N, Indent) when is_tuple(Data) ->\n    format(Mode, [Data], Value2StringF, N, Indent);\nformat(_Mode, [], _Value2StringF, _N, _Indent) -> [];\nformat(Mode, [{'$html', HTML}|Rest], Value2StringF, N, Indent) ->\n    [HTML|format(Mode, Rest, Value2StringF, N, Indent)];\nformat(Mode, [{Tag}|Rest], Value2StringF, N, Indent) ->\n    format(Mode, [{Tag, [], []}|Rest], Value2StringF, N, Indent);\nformat(Mode, [{Tag, Attrs}|Rest], Value2StringF, N, Indent) ->\n    format(Mode, [{Tag, Attrs, []}|Rest], Value2StringF, N, Indent);\nformat(Mode, [{Tag, Attrs, Body}|Rest], Value2StringF, N, Indent) ->\n    TagString = lowercase(tag_string(Tag)),\n    case {Mode, block_level(TagString), Body} of\n        {first_in_block, no, []} ->\n            [$\\n, Indent, $<, TagString,\n             format_attrs(Value2StringF, Attrs), $>, $<, $/,\n             TagString, $>|format(Mode, Rest, Value2StringF, N, Indent)];\n        {block, no, []} ->\n            [$<, TagString, format_attrs(Value2StringF, Attrs), $>, $<, $/,\n             TagString, $>|format(Mode, Rest, Value2StringF, N, Indent)];\n        {first_in_block, yes, _} ->\n            NextLevel = lists:duplicate(?INDENT_LEVEL, $ ),\n            [$\\n, Indent, $<, TagString,\n             format_attrs(Value2StringF, Attrs), $>,\n             format(first_in_block, Body, Value2StringF, N+?INDENT_LEVEL,\n                    [NextLevel, Indent]),\n             $\\n, Indent, $<, $/, TagString, $>|\n             format(block, Rest, Value2StringF, N, Indent)];\n        %% Block element in a block element.\n        {block, yes, _} ->\n            NextLevel = lists:duplicate(?INDENT_LEVEL, $ ),\n            [$\\n, Indent, $<, TagString,\n             format_attrs(Value2StringF, Attrs), $>,\n             format(first_in_block, Body, Value2StringF, N+?INDENT_LEVEL,\n                    [NextLevel, Indent]),\n             $\\n, Indent, $<, $/, TagString, $>|\n             format(block, Rest, Value2StringF, N, Indent)];\n        %% Inline element first in a block element.\n        {first_in_block, no, _} ->\n            [$\\n, Indent, $<, TagString, format_attrs(Value2StringF, Attrs),\n             $>,\n             format(inline, Body, Value2StringF, N, Indent),\n             $<, $/, TagString, $>|\n             format(block, Rest, Value2StringF, N, Indent)];\n        %% Inline element in a block or an inline element.\n        {_, no, _} ->\n            [$<, TagString, format_attrs(Value2StringF, Attrs), $>,\n             format(inline, Body, Value2StringF, N, Indent),\n             $<, $/, TagString, $>|\n             format(Mode, Rest, Value2StringF, N, Indent)]\n    end;\n%% Inline data first in a block element.\nformat(first_in_block, [String|Rest], Value2StringF, N, Indent)\n  when is_list(String) ->\n    [$\\n, Indent, String|format(block, Rest, Value2StringF, N, Indent)];\n%% Inline data in a block/inline element.\nformat(Mode, [String|Rest], Value2StringF, N, Indent) when is_list(String) ->\n    [String|format(Mode, Rest, Value2StringF, N, Indent)];\n%% PCDATA in first a block element.\nformat(first_in_block, Value, Value2StringF, _N, Indent) ->\n    [$\\n, Indent, Value2StringF(Value)];\n%% PCDATA in a block element.\nformat(block, Value, Value2StringF, _N, _Indent) ->\n    [Value2StringF(Value)];\n%% PCDATA in an inline element.\nformat(inline, Value, Value2StringF, _N, _Indent) ->\n    Value2StringF(Value).\n\ntag_string(TagAtom) when is_atom(TagAtom) -> atom_to_list(TagAtom);\ntag_string(TagString) -> TagString.\n\nlowercase(String) -> lowercase(String, []).\n\nlowercase([C|Cs], Acc) when C >= $A, C =< $Z ->\n    lowercase(Cs, [C+($a-$A)| Acc]);\nlowercase([C|Cs], Acc) -> lowercase(Cs, [C| Acc]);\nlowercase([], Acc) -> lists:reverse(Acc).\n\n%% The following are defined as block-level elements:\nblock_level(\"address\") -> yes;\nblock_level(\"blockquote\") -> yes;\nblock_level(\"center\") -> yes;\nblock_level(\"dir\") -> yes;\nblock_level(\"div\") -> yes;\nblock_level(\"dl\") -> yes;\nblock_level(\"fieldset\") -> yes;\nblock_level(\"form\") -> yes;\nblock_level(\"h1\") -> yes;\nblock_level(\"h2\") -> yes;\nblock_level(\"h3\") -> yes;\nblock_level(\"h4\") -> yes;\nblock_level(\"h5\") -> yes;\nblock_level(\"h6\") -> yes;\nblock_level(\"hr\") -> yes;\nblock_level(\"input\") -> yes;\nblock_level(\"isindex\") -> yes;\nblock_level(\"menu\") -> yes;\nblock_level(\"noframes\") -> yes;\nblock_level(\"noscript\") -> yes;\nblock_level(\"ol\") -> yes;\nblock_level(\"p\") -> yes;\nblock_level(\"pre\") -> yes;\nblock_level(\"table\") -> yes;\nblock_level(\"textarea\") -> no;\nblock_level(\"tbody\") -> yes;\nblock_level(\"ul\") -> yes;\nblock_level(\"select\") -> yes;\n%% The following elements may also be considered block-level elements since\n%% they may contain block-level elements:\nblock_level(\"dd\") -> yes;\nblock_level(\"dt\") -> yes;\nblock_level(\"frameset\") -> yes;\nblock_level(\"li\") -> yes;\nblock_level(\"td\") -> yes;\nblock_level(\"tfoot\") -> yes;\nblock_level(\"th\") -> yes;\nblock_level(\"thead\") -> yes;\nblock_level(\"tr\") -> yes;\n%% The following elements may be used as either block-level elements or\n%% inline elements. If used as inline elements (e.g., within another inline\n%% element or a P), these elements should not contain any block-level\n%% elements.\nblock_level(\"applet\") -> yes;\nblock_level(\"button\") -> yes;\nblock_level(\"del\") -> yes;\nblock_level(\"iframe\") -> yes;\nblock_level(\"ins\") -> yes;\nblock_level(\"map\") -> yes;\nblock_level(\"object\") -> yes;\nblock_level(\"script\") -> yes;\n%% All else are defined as inline elements:\nblock_level(_) -> no.\n\nformat_attrs(_Value2StringF, []) -> [];\nformat_attrs(Value2StringF, [{Name, Value}|Rest]) ->\n    [$ , lowercase(tag_string(Name)), $=, $\\\", Value2StringF(Value), $\\\"|\n     format_attrs(Value2StringF, Rest)].\n\nvalue2string(Atom) when is_atom(Atom) -> atom_to_list(Atom);\nvalue2string(Integer) when is_integer(Integer) -> integer_to_list(Integer);\nvalue2string(Float) when is_float(Float) -> float_to_list(Float);\nvalue2string(Binary) when is_binary(Binary) -> Binary;\nvalue2string(String) when is_list(String) -> String.\n\nsformat(Data) -> sformat(Data, fun value2string/1).\n\nsformat(Data, Value2StringF) when is_tuple(Data) ->\n    sformat([Data], Value2StringF);\nsformat([], _Value2StringF) -> [];\nsformat([{Tag}|Rest], Value2StringF) ->\n    sformat([{Tag, [], []}|Rest], Value2StringF);\nsformat([{Tag, Body}|Rest], Value2StringF) ->\n    sformat([{Tag, [], Body}|Rest], Value2StringF);\nsformat([{Tag, Attrs, []}|Rest], Value2StringF) ->\n    TagString = lowercase(tag_string(Tag)),\n    [$<, TagString, format_attrs(Value2StringF, Attrs), $/, $>|\n     sformat(Rest, Value2StringF)];\nsformat([{Tag, Attrs, Body}|Rest], Value2StringF) ->\n    TagString = lowercase(tag_string(Tag)),\n    [$<, TagString, format_attrs(Value2StringF, Attrs), $>,\n     sformat(Body, Value2StringF),\n     $<, $/, TagString, $>|\n     sformat(Rest, Value2StringF)];\nsformat([String|Rest], Value2StringF) when is_list(String) ->\n    [String|sformat(Rest, Value2StringF)];\nsformat(Value, Value2StringF) ->\n    Value2StringF(Value).\n\n-define(SZ, 16).\n\ncount_trailing_spaces(<<>>) ->\n    0;\ncount_trailing_spaces(Bin) ->\n    count_trailing_spaces(Bin, size(Bin), 0).\n\ncount_trailing_spaces(Bin, Stop, N) ->\n    Start = if Stop =< ?SZ ->\n                    1;\n               true ->\n                    Stop - ?SZ + 1\n            end,\n    L = binary_to_list(Bin, Start, Stop),\n    case spaces_in_list(L) of\n        ?SZ when Start == 1 ->\n            N + ?SZ;\n        ?SZ ->\n            %% keep going\n            count_trailing_spaces(Bin, Start-1, N + ?SZ);\n        M ->\n            N + M\n    end.\n\nspaces_in_list(L) ->\n    spaces_in_list(lists:reverse(L), 0).\n\nspaces_in_list([$\\s | T], N) ->\n    spaces_in_list(T, N+1);\nspaces_in_list(_, N) ->\n    N.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_generated.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_generated.template\n%%% Author  : Klacke <klacke@bluetail.com>\n%%% Purpose :\n%%% Created : 10 Jun 2002 by Klacke <klacke@bluetail.com>\n%%%----------------------------------------------------------------------\n\n%% generated code from some environment variables\n\n-module(yaws_generated).\n-author('klacke@bluetail.com').\n\n-export([version/0,\n         vardir/0,\n         etcdir/0]).\n\nversion() -> \"2.0.4\".\n\nvardir() ->  \"/usr/local/var\".\n\netcdir() -> \"/usr/local/etc\".\n\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_generated.template",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_generated.template\n%%% Author  : Klacke <klacke@bluetail.com>\n%%% Purpose :\n%%% Created : 10 Jun 2002 by Klacke <klacke@bluetail.com>\n%%%----------------------------------------------------------------------\n\n%% generated code from some environment variables\n\n-module(yaws_generated).\n-author('klacke@bluetail.com').\n\n-export([version/0,\n         vardir/0,\n         etcdir/0]).\n\nversion() -> \"%VSN%\".\n\nvardir() ->  \"%VARDIR%\".\n\netcdir() -> \"%ETCDIR%\".\n\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_html.erl",
    "content": "%%    -*- Erlang -*-\n%%    File:        parse_html.erl\n%%    Author:        Johan Bevemyr\n%%    Created:        Tue Nov 25 20:53:36 2003\n%%    Purpose:   Transform html to an erlang represention (ehtml)\n\n-module('yaws_html').\n-author('jb@bevemyr.com').\n\n-export([parse/1,parse/2,h2e/1]).\n\nparse(Name) ->\n    {ok, B} = file:read_file(Name),\n    h2e(binary_to_list(B)).\n\nparse(Name,Out) ->\n    {ok, B} = file:read_file(Name),\n    case h2e(binary_to_list(B)) of\n        {ehtml, [], Ehtml} ->\n            Cont = io_lib:format(\"~p\", [{ehtml, Ehtml}]),\n            file:write_file(Out, Cont);\n        Error ->\n            Error\n    end.\n\nh2e(Input) ->\n    Tokens = tokenize(Input, [], [], 1),\n    parse(Tokens, {ehtml,[],0}, [], []).\n\n%% parse(Tokens, Stack, Acc)\n\nparse([], {T,A,_L}, [], Acc) ->\n    {T, A, lists:reverse(Acc)};\nparse([], {T,A,L}, [{CTag,CAcc}|Stack], Acc) ->\n    io:format(\"Unterminated tag '~p' at line ~p\\n\", [T,L]),\n    parse([], CTag, Stack, [{T,A,lists:reverse(Acc)}|CAcc]);\nparse([{begin_tag,T,A,L}|Tokens], CTag, Stack, Acc) ->\n    case tag_type(T) of\n        leaf ->\n            parse(Tokens, CTag, Stack, [{T,A}|Acc]);\n        node ->\n            parse(Tokens, {T,A,L}, [{CTag,Acc}|Stack],[])\n    end;\n\nparse([{end_tag,T,[],_L}|Tokens], {T,A,_}, [{CTag,CAcc}|Stack], Acc) ->\n    E = case Acc of\n            [Single] ->\n                {T,A,Single};\n            _ ->\n                {T,A,lists:reverse(Acc)}\n        end,\n    parse(Tokens, CTag, Stack, [E|CAcc]);\n\nparse([{end_tag,T1,[],L1}|Tokens], CTag = {T2,_A,L2}, Stack, Acc) ->\n    case tag_type(T1) of\n        leaf -> %% ignore\n            parse(Tokens, CTag, Stack, Acc);\n        node ->\n            Msg = lists:flatten(io_lib:format(\n                                  \"expected '</~p>'  on line ~p, start \"\n                                  \"tag at line: ~p\", [T2,L1,L2])),\n            {error, Msg}\n    end;\n\nparse([{data, Data, _Line}|Tokens], CTag, Stack, Acc) ->\n    case skip_space(Data, 0) of\n        {[], _} ->\n            parse(Tokens, CTag, Stack, Acc);\n        _ ->\n            parse(Tokens, CTag, Stack, [Data|Acc])\n    end.\n%%\n\ntag_type(p)          -> leaf;\ntag_type(hr)         -> leaf;\ntag_type(input)      -> leaf;\ntag_type(base)       -> leaf;\ntag_type(img)        -> leaf;\ntag_type('!doctype') -> leaf;\ntag_type(meta)       -> leaf;\ntag_type(link)       -> leaf;\ntag_type(br)         -> leaf;\ntag_type(_)          -> node.\n\n%% tokenize(Input, DataAcc, TokenAcc, LineNr)\n\ntokenize([], [], Tokens, _Line) ->\n    lists:reverse(Tokens);\ntokenize([], Acc, Tokens, Line) ->\n    lists:reverse([{data, lists:reverse(Acc), Line}|Tokens]);\ntokenize([$<,$!,$-,$-|R0], Acc, Tokens, L0) ->\n    {R1, L1} = skip_comment(R0,L0),\n    tokenize(R1, Acc, Tokens, L1);\ntokenize([$<|R0], Acc, Tokens, L0) ->\n    {Tag,R1,L1} = scan_tag(R0,L0),\n    if\n        Acc == [] ->\n            next_token(Tag, R1, [Tag|Tokens], L1);\n        true ->\n            Data = {data,lists:reverse(Acc),L0},\n            next_token(Tag, R1, [Tag,Data|Tokens], L1)\n    end;\ntokenize([C=$\\n|R0], Acc, Tokens, L) ->\n    tokenize(R0, [C|Acc], Tokens, L+1);\ntokenize([C=$\\r|R0], Acc, Tokens, L) ->\n    tokenize(R0, [C|Acc], Tokens, L+1);\ntokenize([C|R0], Acc, Tokens, L) ->\n    tokenize(R0, [C|Acc], Tokens, L).\n\n%%\n\nnext_token({begin_tag, script, _, _}, R, Tokens, L) ->\n    {Data, R1, L1} = scan_endtag(R, \"script\", L),\n    tokenize(R1, [], [{data, Data, L}|Tokens], L1);\nnext_token({begin_tag, style, _, _}, R, Tokens, L) ->\n    {Data, R1, L1} = scan_endtag(R, \"style\", L),\n    tokenize(R1, [], [{data, Data, L}|Tokens], L1);\nnext_token(_Tag, R, Tokens, L) ->\n    tokenize(R, [], Tokens, L).\n\n%% '<' <id> <sp>+ [<id><sp>*['='<val>]]* ['/'] '>'\n\nscan_tag([$/|I], L) ->\n    {_R0,L0} = skip_space(I, L),\n    {Name,R1,L1} = scan_tag_name(I, L0),\n    {R2,L2} = skip_space(R1, L1),\n    {Args,R3,L3} = scan_tag_args(R2, L2),\n    {{end_tag,list_to_atom(lowercase(Name)),Args,L0}, R3, L3};\nscan_tag(I, L) ->\n    {_R0,L0} = skip_space(I, L),\n    {Name,R1,L1} = scan_tag_name(I, L0),\n    {R2,L2} = skip_space(R1, L1),\n    {Args,R3,L3} = scan_tag_args(R2, L2),\n    {{begin_tag,list_to_atom(lowercase(Name)),Args,L0}, R3, L3}.\n\n%%\n\nscan_tag_name(I, L) ->\n    scan_token(I, [], L).\n\n%%\n\nscan_tag_args(I, L) ->\n    scan_tag_args(I, [], L).\n\nscan_tag_args([], Acc, L) ->\n    {lists:reverse(Acc), [], L};\nscan_tag_args([$>|R], Acc, L) ->\n    {lists:reverse(Acc), R, L};\nscan_tag_args(R=[$<|_], Acc, L) ->  %%%% bad html\n    {lists:reverse(Acc), R, L};\nscan_tag_args(R0, Acc, L0) ->\n    {Name,R1,L1} = scan_value(R0, L0),\n    {R2, L2} = skip_space(R1, L1),\n    case R2 of\n        [$=|R3] ->\n            {R4,L4} = skip_space(R3, L2),\n            {Value,R5,L5} = scan_value(R4, L4),\n            {R6,L6} = skip_space(R5, L5),\n            OptName = list_to_atom(lowercase(Name)),\n            scan_tag_args(R6, [{OptName,Value}|Acc], L6);\n        _ ->\n            scan_tag_args(R2, [Name|Acc], L2)\n    end.\n\n%%\n\nscan_value([$\"|R], L) ->\n    scan_quote(R, [], $\", L);\nscan_value([$'|R], L) ->\n    scan_quote(R, [], $', L);\nscan_value(R, L) ->\n    scan_token(R, [], L).\n\n%%\n\nscan_token([], Acc, L) ->\n    {lists:reverse(Acc), [], L};\nscan_token(R=[$>|_], Acc, L) ->\n    {lists:reverse(Acc), R, L};\nscan_token(R=[$<|_], Acc, L) ->  %%% bad html\n    {lists:reverse(Acc), R, L};\nscan_token(R=[$=|_], Acc, L) ->  %% bad html\n    {lists:reverse(Acc), R, L};\nscan_token([C|R], Acc, L0) ->\n    case char_class(C) of\n        space ->\n            {lists:reverse(Acc), R, L0};\n        nl ->\n            {lists:reverse(Acc), R, L0+1};\n        _ ->\n            scan_token(R, [C|Acc], L0)\n    end.\n\n%%\n\nscan_quote([], Acc, _Q, L) ->\n    {lists:reverse(Acc), [], L};\nscan_quote([Q|R], Acc, Q, L) ->\n    {lists:reverse(Acc), R, L};\nscan_quote([C=$\\n|R], Acc, Q, L) ->\n    scan_quote(R, [C|Acc], Q, L+1);\nscan_quote([C=$\\r|R], Acc, Q, L) ->\n    scan_quote(R, [C|Acc], Q, L+1);\nscan_quote([C|R], Acc, Q, L) ->\n    scan_quote(R, [C|Acc], Q, L).\n\n%%\n\nscan_endtag(R, Tag, L) ->\n    scan_endtag(R, Tag, [], L).\n\nscan_endtag([], _Tag, Acc, L) ->\n    {lists:reverse(Acc), [], L};\nscan_endtag(R=[$<,$/|R0], Tag, Acc, L0) ->\n    case casecmp(Tag, R0) of\n        {true, R1} ->\n            {R2,_} = skip_space(R1,L0),\n            if hd(R2) == $> ->\n                    {lists:reverse(Acc), R, L0};\n               true ->\n                    scan_endtag(R0, Tag, Acc, L0)\n            end;\n        false ->\n            scan_endtag(R0, Tag, Acc, L0)\n    end;\nscan_endtag([C=$\\n|R], Tag, Acc, L) ->\n    scan_endtag(R, Tag, [C|Acc], L+1);\nscan_endtag([C=$\\r|R], Tag, Acc, L) ->\n    scan_endtag(R, Tag, [C|Acc], L+1);\nscan_endtag([C|R], Tag, Acc, L) ->\n    scan_endtag(R, Tag, [C|Acc], L).\n\n%%\n\ncasecmp([], R) -> {true, R};\ncasecmp([C1|T1], [C2|T2]) ->\n    C2low = lowercase_ch(C2),\n    if C1 == C2low -> casecmp(T1,T2);\n       true        -> false\n    end.\n\n%%\n\nchar_class($\\n) -> nl;\nchar_class($\\r) -> nl;\nchar_class($ )  -> space;\nchar_class($\\t) -> space;\nchar_class(C) when C >= $a, C =< $z -> alpha;\nchar_class(C) when C >= $A, C =< $Z -> alpha;\nchar_class(C) when C >= $0, C =< $9 -> digit;\nchar_class(_C)   -> other.\n\n%%\n\nskip_space([], L) ->\n    {[], L};\nskip_space(R = [C|R0], L) ->\n    case char_class(C) of\n        nl ->\n            skip_space(R0, L+1);\n        space ->\n            skip_space(R0, L);\n        _ ->\n            {R, L}\n    end.\n\n%%\n\nskip_comment([], L) ->          {[], L};\nskip_comment([$-,$-,$>|R],L) -> {R,L};\nskip_comment([$\\n|R],L) ->      skip_comment(R,L+1);\nskip_comment([$\\r|R],L) ->      skip_comment(R,L+1);\nskip_comment([_C|R],L) ->        skip_comment(R,L).\n\n%%\n\nlowercase(Str) ->\n    [lowercase_ch(S) || S <- Str].\n\nlowercase_ch(C) when C>=$A, C=<$Z -> C + 32;\nlowercase_ch(C) -> C.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_jsonrpc.erl",
    "content": "%% -*- coding: latin-1 -*-\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%% WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED\n%%% WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED\n%%%\n%%% Use module yaws_rpc.erl instead\n%%%\n%%% This module is deprecated.\n%%%\n%%% Do not report problems with this module, as they will not be fixed. You\n%%% should instead convert your code to use the yaws_rpc module.\n%%%\n%%% WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED\n%%% WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED WARNING DEPRECATED\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Copyright (C) 2003 Joakim Greben <jocke@gleipnir.com>.\n%% All rights reserved.\n%%\n%% Copyright (C) 2006 Gaspar Chilingarov <nm@web.am>\n%%                      Gurgen Tumanyan <barbarian@armkb.com>\n%% All rights reserved.\n%%\n%%\n%% Redistribution and use in source and binary forms, with or without\n%% modification, are permitted provided that the following conditions\n%% are met:\n%%\n%% 1. Redistributions of source code must retain the above copyright\n%%    notice, this list of conditions and the following disclaimer.\n%% 2. Redistributions in binary form must reproduce the above\n%%    copyright notice, this list of conditions and the following\n%%    disclaimer in the documentation and/or other materials provided\n%%    with the distribution.\n%%\n%% THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS\n%% OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED\n%% WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n%% ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY\n%% DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n%% DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE\n%% GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS\n%% INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,\n%% WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING\n%% NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS\n%% SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n\n-module(yaws_jsonrpc).\n-author(\"Gaspar Chilingarov <nm@web.am>, Gurgen Tumanyan <barbarian@armkb.com>\").\n-export([handler/2]).\n-export([handler_session/2, handler_session/3]).\n\n%%-define(debug, 1).\n-include(\"yaws_debug.hrl\").\n\n-include(\"../include/yaws_api.hrl\").\n\n%%% ######################################################################\n%%% public interface\n%%%\n\n%%%\n%%% use jsonrpc handler which can automagically start sessions if we need\n%%%\nhandler_session(Args, Handler) ->\n    handler_session(Args, Handler, 'SID').\n\n%%%\n%%% allow overriding session Cookie name\n%%%\nhandler_session(Args, Handler, SID_NAME) when is_atom(SID_NAME) ->\n    handler_session(Args, Handler, atom_to_list(SID_NAME));\n\nhandler_session(Args, Handler, SID_NAME) ->\n    handler(Args, Handler, {session, SID_NAME}).    % go to generic handler\n\n%%%\n%%% xmlrpc:handler compatible call\n%%% no session support will be available\nhandler(Args, Handler) ->\n    handler(Args, Handler, simple).\n\n\n%%% ######################################################################\n%%% private functions\n%%%\n\n%%% we should be called from yaws page or module\nhandler(Args, Handler, Type) when is_record(Args, arg) ->\n    case parse_request(Args) of\n        ok ->\n            handle_payload(Args, Handler, Type);\n        {status, StatusCode} ->        % cannot parse request\n            send(Args, StatusCode)\n    end.\n\n-define(ERROR_LOG(Reason),\n        error_logger:error_report({?MODULE, ?LINE, Reason})).\n\n%%%\n%%% check that request come in reasonable protocol version and reasonable method\n%%%\nparse_request(Args) ->\n    case {(Args#arg.req)#http_request.method,\n          (Args#arg.req)#http_request.version} of\n        {'POST', {1,0}} ->\n            ?Debug(\"HTTP Version 1.0~n\", []),\n            ok;\n        {'POST', {1,1}} ->\n            ?Debug(\"HTTP Version 1.1~n\", []),\n            ok;\n        {'POST', _HTTPVersion} -> {status, 505};\n        {_Method, {1,1}} -> {status, 501};\n        _ -> {status, 400}\n    end.\n\nhandle_payload(Args, Handler, Type) ->\n    Payload = binary_to_list(Args#arg.clidata),\n    ?Debug(\"jsonrpc plaintext call ~p ~n\", [Payload]),\n    case decode_handler_payload(Payload) of\n        {ok, DecodedPayload, ID} ->\n            ?Debug(\"json2erl decoded call ~p ~n\", [DecodedPayload]),\n            eval_payload(Args, Handler, DecodedPayload, Type, ID);\n        {error, Reason} ->\n            ?ERROR_LOG({html, json2erl, Payload, Reason}),\n            send(Args, 400)\n    end.\n\n%%%\n%%% call handler/3 and provide session support\neval_payload(Args, {M, F}, Payload, {session, CookieName},ID) ->\n    {SessionValue, Cookie} =\n        case yaws_api:find_cookie_val(CookieName,\n                                      (Args#arg.headers)#headers.cookie) of\n            [] ->      % have no session started, just call handler\n                {undefined, undefined};\n            Cookie2 -> % get old session data\n                case yaws_api:cookieval_to_opaque(Cookie2) of\n                    {ok, OP} ->\n                        yaws_api:cookieval_to_opaque(Cookie2),\n                        {OP, Cookie2};\n                    {error, _ErrMsg} -> % cannot get corresponding session\n                        {undefined, undefined}\n                end\n        end,\n\n    case catch M:F(Args#arg.state, Payload, SessionValue) of\n        {'EXIT', Reason} ->\n            ?ERROR_LOG({M, F, {'EXIT', Reason}}),\n            send(Args, 500);\n        {error, Reason} ->\n            ?ERROR_LOG({M, F, Reason}),\n            send(Args, 500);\n        {false, ResponsePayload} ->\n            %% do not have updates in session data\n            encode_send(Args, 200, ResponsePayload, [], ID);\n        {true, _NewTimeout, NewSessionValue, ResponsePayload} ->\n            %% be compatible with xmlrpc module\n            CO = case NewSessionValue of\n                     undefined when Cookie == undefined -> []; % nothing to do\n                     undefined -> % rpc handler requested session delete\n                         yaws_api:delete_cookie_session(Cookie), [];\n                     %% XXX: may be return set-cookie with empty val?\n                     _ ->\n                         %% any other value will stored in session\n                         case SessionValue of\n                             undefined ->\n                                 %% got session data and should start\n                                 %% new session now\n                                 Cookie1 = yaws_api:new_cookie_session(\n                                             NewSessionValue),\n                                 yaws_api:setcookie(CookieName, Cookie1, \"/\");\n                             _ ->\n                                 yaws_api:replace_cookie_session(\n                                   Cookie, NewSessionValue),\n                                 [] % nothing to add to yaws data\n                         end\n                 end,\n            encode_send(Args, 200, ResponsePayload, CO, ID)\n    end;\n\n%%%\n%%% call handler/2 without session support\n%%%\neval_payload(Args, {M, F}, Payload, simple, ID) ->\n    case catch M:F(Args#arg.state, Payload) of\n        {'EXIT', Reason} ->\n            ?ERROR_LOG({M, F, {'EXIT', Reason}}),\n            send(Args, 500);\n        {error, Reason} ->\n            ?ERROR_LOG({M, F, Reason}),\n            send(Args, 500);\n        {false, ResponsePayload} ->\n            encode_send(Args, 200, ResponsePayload, [],ID);\n        {true, _NewTimeout, _NewState, ResponsePayload} ->\n            encode_send(Args, 200, ResponsePayload, [],ID)\n    end.\n\n\n%%% XXX compatibility with XMLRPC handlers\n%%% XXX - potential bug here?\nencode_send(Args, StatusCode, [Payload], AddOn, ID) ->\n    encode_send(Args, StatusCode, Payload, AddOn, ID);\n\nencode_send(Args, StatusCode, Payload, AddOn, ID) ->\n    {ok, EncodedPayload} = encode_handler_payload(Payload, ID),\n    send(Args, StatusCode, EncodedPayload, AddOn).\n\nsend(Args, StatusCode) -> send(Args, StatusCode, \"\", []).\n\nsend(Args, StatusCode, Payload, AddOnData) when not is_list(AddOnData) ->\n    send(Args, StatusCode, Payload, [AddOnData]);\n\nsend(_Args, StatusCode, Payload, AddOnData) ->\n    A = [\n    {status, StatusCode},\n    {content, \"text/xml\", Payload},\n    {header, {content_length, lists:flatlength(Payload) }}\n    ] ++ AddOnData,\n    A.\n\n\nencode_handler_payload({response, [ErlStruct]},ID) ->\n    encode_handler_payload({response, ErlStruct}, ID);\n\nencode_handler_payload({response, ErlStruct},ID) ->\n    StructStr = json2:encode({struct, [{result, ErlStruct}, {id, ID}]}),\n    {ok, StructStr}.\n\ndecode_handler_payload(JSonStr) ->\n    try\n        {ok, JSON} = json2:decode_string(JSonStr),\n        Method = list_to_atom(jsonrpc:s(JSON, method)),\n        {array, Args} = jsonrpc:s(JSON, params),\n        ID = jsonrpc:s(JSON, id),\n\n        {ok, {call, Method, Args}, ID}\n    catch\n        error:Err -> {error, Err}\n    end.\n\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_log.erl",
    "content": "%%----------------------------------------------------------------------\n%%% File    : yaws_log.erl\n%%% Author  : Claes Wikstrom <klacke@hyber.org>\n%%% Purpose :\n%%% Created : 26 Jan 2002 by Claes Wikstrom <klacke@hyber.org>\n%%%----------------------------------------------------------------------\n\n-module(yaws_log).\n-author('klacke@hyber.org').\n-include_lib(\"kernel/include/file.hrl\").\n-include_lib(\"kernel/include/inet.hrl\").\n\n\n-behaviour(gen_server).\n\n%% External exports\n-export([start_link/0, reopen_logs/0]).\n\n%% gen_server callbacks\n-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2,\n         code_change/3]).\n\n%% API\n-export([accesslog/6,\n         setup/2,\n         authlog/4,\n         rotate/1,\n         add_sconf/1,\n         del_sconf/1]).\n\n%% yaws_logger callbacks\n-export([\n         open_log/3,\n         close_log/3,\n         wrap_log/4,\n         write_log/4\n        ]).\n\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_debug.hrl\").\n\n%% 1 meg log we wrap\n-define(WRAP_LOG_SIZE, 1000000).\n\n\n-record(state, {running,\n                dir,\n                now,\n                log_wrap_size = ?WRAP_LOG_SIZE,\n                copy_errlog,\n                resolve_hostnames = false}).\n\n\n%%%----------------------------------------------------------------------\n%%% API\n%%%----------------------------------------------------------------------\nstart_link() ->\n    gen_server:start_link({local, yaws_log}, yaws_log, [], []).\n\nsetup(GC, Sconfs) ->\n    gen_server:call(?MODULE, {setup, GC, Sconfs}, infinity).\n\nadd_sconf(SConf) ->\n    gen_server:call(yaws_log, {soft_add_sc, SConf}, infinity).\n\ndel_sconf(SConf) ->\n    gen_server:call(yaws_log, {soft_del_sc, SConf}, infinity).\n\naccesslog(SConf, Ip, Req, InH, OutH, Time) ->\n    catch yaws_logger:accesslog(SConf, Ip, Req, InH, OutH, Time).\n\nauthlog(SConf, IP, Path, Item) ->\n    catch yaws_logger:authlog(SConf, IP, Path, Item).\n\nrotate(Res) ->\n    gen_server:cast(?MODULE, {yaws_hupped, Res}).\n\n%% Useful for embeddded yaws when we don't want yaws to\n%% automatically wrap the logs.\nreopen_logs() ->\n    {ok, _GC, SCs} = yaws_api:getconf(),\n    gen_server:call(?MODULE, {reopen, SCs}).\n\n\n%%%----------------------------------------------------------------------\n%%% Callback functions from yaws_logger\n%%%----------------------------------------------------------------------\nopen_log(ServerName, Type, Dir) ->\n    FileName = case os:type() of\n                   {win32,_ } ->\n                       lists:map(fun($:) -> $.;\n                                    (C ) -> C\n                                 end, ServerName);\n                   _ ->\n                       ServerName\n               end,\n    A = filename:join([Dir, FileName ++ \".\" ++ atom_to_list(Type)]),\n    case file:open(A, [write, raw, append]) of\n        {ok, Fd} ->\n            {true, {Fd, A}};\n        _Err ->\n            error_logger:format(\"Cannot open ~p\",[A]),\n            false\n    end.\n\nclose_log(_ServerName, _Type, {Fd, _FileName}) ->\n    file:close(Fd).\n\nwrap_log(_ServerName, _Type, {Fd, FileName}, LogWrapSize) ->\n    case wrap_p(FileName, LogWrapSize) of\n        true ->\n            file:close(Fd),\n            Old = [FileName, \".old\"],\n            file:delete(Old),\n            file:rename(FileName, Old),\n            {ok, Fd2} = file:open(FileName, [write, raw]),\n            {Fd2, FileName};\n        false ->\n            {Fd, FileName};\n        enoent ->\n            %% Logfile disappeared,\n            error_logger:format(\"Logfile ~p disappeared - we reopen it\",\n                                [FileName]),\n            file:close(Fd),\n            {ok, Fd2} = file:open(FileName, [write, raw]),\n            {Fd2, FileName}\n    end.\n\nwrite_log(ServerName, Type, {Fd, _FileName}, Infos) ->\n    gen_server:cast(yaws_log, {ServerName, Type, Fd, Infos}).\n\n%%%----------------------------------------------------------------------\n%%% Callback functions from gen_server\n%%%----------------------------------------------------------------------\n\n%%----------------------------------------------------------------------\n%% Func: init/1\n%% Returns: {ok, State}          |\n%%          {ok, State, Timeout} |\n%%          ignore               |\n%%          {stop, Reason}\n%%----------------------------------------------------------------------\ninit([]) ->\n    process_flag(trap_exit, true),\n    ets:new(yaws_log, [named_table, set, protected, {keypos, 2}]),\n    {ok, #state{running = false, now = fmtnow()}}.\n\n%%----------------------------------------------------------------------\n%% Func: handle_call/3\n%% Returns: {reply, Reply, State}          |\n%%          {reply, Reply, State, Timeout} |\n%%          {noreply, State}               |\n%%          {noreply, State, Timeout}      |\n%%          {stop, Reason, Reply, State}   | (terminate/2 is called)\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%----------------------------------------------------------------------\nhandle_call({setup, GC, Sconfs}, _From, State)\n  when State#state.running == false ->\n    Dir = GC#gconf.logdir,\n    ?Debug(\"setup ~s~n\", [Dir]),\n    ElogFile = filename:join([Dir, \"report.log\"]),\n    Copy = if ?gc_has_copy_errlog(GC) ->\n                   gen_event:add_handler(error_logger, yaws_log_file_h,\n                                         ElogFile),\n                   true;\n              true ->\n                   false\n           end,\n    SCs = lists:flatten(Sconfs),\n    lists:foreach(fun(SC) ->\n                          yaws_logger:open_log(SC, auth, Dir),\n                          yaws_logger:open_log(SC, access, Dir)\n                  end, SCs),\n\n    S2 = State#state{running = true,\n                     dir  = Dir,\n                     now = fmtnow(),\n                     log_wrap_size = GC#gconf.log_wrap_size,\n                     copy_errlog = Copy,\n                     resolve_hostnames = ?gc_log_has_resolve_hostname(GC)},\n\n    yaws:ticker(3000, secs3),\n\n    if is_integer(GC#gconf.log_wrap_size) ->\n            yaws:ticker(10 * 60 * 1000, minute10);\n       true ->\n            ok\n    end,\n\n    {reply, ok, S2};\n\n\n\n%% We can't ever change logdir, we can however\n%% change logging opts for various servers\n\nhandle_call({setup, GC, Sconfs}, _From, State)\n  when State#state.running == true ->\n\n    Dir = State#state.dir,\n    ElogFile = filename:join([Dir, \"report.log\"]),\n    Copy = if ?gc_has_copy_errlog(GC), State#state.copy_errlog == false->\n                   gen_event:add_handler(error_logger, yaws_log_file_h,\n                                         ElogFile),\n                   true;\n              ?gc_has_copy_errlog(GC) ->\n                   true;\n              State#state.copy_errlog == true ->\n                   gen_event:delete_handler(error_logger, yaws_log_file_h,\n                                            normal),\n                   false;\n              true ->\n                   false\n           end,\n\n    %% close all files\n    yaws_logger:close_logs(),\n\n    %% reopen logfiles\n    SCs = lists:flatten(Sconfs),\n    lists:foreach(fun(SC) ->\n                          yaws_logger:open_log(SC, auth, Dir),\n                          yaws_logger:open_log(SC, access, Dir)\n                  end, SCs),\n\n    S2 = State#state{running = true,\n                     dir  = Dir,\n                     now = fmtnow(),\n                     log_wrap_size = GC#gconf.log_wrap_size,\n                     copy_errlog = Copy,\n                     resolve_hostnames = ?gc_log_has_resolve_hostname(GC)},\n\n    if\n        not is_integer(State#state.log_wrap_size),\n        is_integer(GC#gconf.log_wrap_size) ->\n            yaws:ticker(10 * 60 * 1000, minute10);\n       true ->\n            ok\n    end,\n    {reply, ok, S2};\n\n\n%% a virt server has been added\nhandle_call({soft_add_sc, SC}, _From, State) ->\n    yaws_logger:open_log(SC, auth, State#state.dir),\n    yaws_logger:open_log(SC, access, State#state.dir),\n    {reply, ok, State};\n\n%% a virt server has been deleted\nhandle_call({soft_del_sc, SC}, _From, State) ->\n    yaws_logger:close_log(SC, auth),\n    yaws_logger:close_log(SC, access),\n    {reply, ok, State};\n\n\nhandle_call(state, _From, State) ->\n    {reply, State, State};\n\nhandle_call({reopen, Sconfs}, _From, State) ->\n    Dir = State#state.dir,\n    %% close all files\n    yaws_logger:close_logs(),\n\n    %% reopen logfiles\n    SCs = lists:flatten(Sconfs),\n    lists:foreach(fun(SC) ->\n                          yaws_logger:open_log(SC, auth, Dir),\n                          yaws_logger:open_log(SC, access, Dir)\n                  end, SCs),\n    {reply, ok, State}.\n\n%%----------------------------------------------------------------------\n%% Func: handle_cast/2\n%% Returns: {noreply, State}          |\n%%          {noreply, State, Timeout} |\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%----------------------------------------------------------------------\nhandle_cast({_ServerName, access, Fd, {Ip, Req, InH, OutH, _}}, State) ->\n    case State#state.running of\n        true ->\n            Status = case OutH#outh.status of\n                         undefined -> \"-\";\n                         S         -> integer_to_list(S)\n                     end,\n            Len = case Req#http_request.method of\n                      'HEAD' ->\n                          \"-\";\n                      _ ->\n                          case OutH#outh.contlen of\n                              undefined ->\n                                  case OutH#outh.act_contlen of\n                                      undefined -> \"-\";\n                                      L         -> integer_to_list(L)\n                                  end;\n                              L ->\n                                  integer_to_list(L)\n                          end\n                  end,\n            Ver = case Req#http_request.version of\n                      {1,0} -> \"HTTP/1.0\";\n                      {1,1} -> \"HTTP/1.1\";\n                      {0,9} -> \"HTTP/0.9\";\n                      _     -> \"HTTP/X.X\"\n                  end,\n\n            Path      = yaws_server:safe_path(Req#http_request.path),\n            Meth      = yaws:to_list(Req#http_request.method),\n            Referer   = optional_header(InH#headers.referer),\n            UserAgent = optional_header(InH#headers.user_agent),\n            User      = case InH#headers.authorization of\n                            {U, _P, _OStr} -> U;\n                            _              -> \"-\"\n                        end,\n\n            Msg = fmt_access_log(State#state.now, fmt_ip(Ip, State), User,\n                                 [Meth, $\\s, Path, $\\s, Ver],\n                                 Status,  Len, Referer, UserAgent),\n            file:write(Fd, safe_log_data(Msg)),\n            {noreply, State};\n        false ->\n            {noreply, State}\n    end;\n\nhandle_cast({ServerName, auth, Fd, {Ip, Path, Item}}, State) ->\n    case State#state.running of\n        true ->\n            Host = fmt_ip(Ip, State),\n            Msg  = [Host, \" \", State#state.now, \" \", ServerName, \" \" ,\n                    \"\\\"\", Path,\"\\\"\",\n                   case Item of\n                       {ok, User}       -> [\" OK user=\", User];\n                       403              -> [\" 403\"];\n                       {401, Realm}     -> [\" 401 realm=\", Realm];\n                       {401, User, PWD} -> [\" 401 user=\", User, \" badpwd=\",PWD];\n                       _                -> \"\"\n                   end, \"\\n\"],\n            file:write(Fd, safe_log_data(Msg)),\n            {noreply, State};\n        false ->\n            {noreply,State}\n    end;\n\nhandle_cast({yaws_hupped, _}, State) ->\n    handle_info(minute10, State).\n\n\n%%----------------------------------------------------------------------\n%% Func: handle_info/2\n%% Returns: {noreply, State}          |\n%%          {noreply, State, Timeout} |\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%----------------------------------------------------------------------\nhandle_info(secs3, State) ->\n    {noreply, State#state{now = fmtnow()}};\n\n%% once every 10 minutes, check log sizes\nhandle_info(minute10, State) ->\n    yaws_logger:rotate(State#state.log_wrap_size),\n\n    case gen_event:call(error_logger, yaws_log_file_h, size, infinity) of\n        {ok, Size} when  State#state.log_wrap_size > 0,\n                       Size > State#state.log_wrap_size ->\n            gen_event:call(error_logger, yaws_log_file_h, wrap, infinity);\n        {error, enoent} ->\n            gen_event:call(error_logger, yaws_log_file_h, reopen, infinity);\n        _ ->\n            ok\n    end,\n    {noreply, State};\nhandle_info({'EXIT', _, _}, State) ->\n    {noreply, State}.\n\n\n\nwrap_p(Filename, LogWrapSize) ->\n    case file:read_file_info(Filename) of\n        {ok, FI} when FI#file_info.size > LogWrapSize, LogWrapSize > 0 ->\n            true;\n        {ok, _FI} ->\n            false;\n        {error, enoent} ->\n            enoent;\n        _ ->\n            false\n    end.\n\n\n\n%%----------------------------------------------------------------------\n%% Func: terminate/2\n%% Purpose: Shutdown the server\n%% Returns: any (ignored by gen_server)\n%%----------------------------------------------------------------------\nterminate(_Reason, _State) ->\n    gen_event:delete_handler(error_logger, yaws_log_file_h, normal),\n    yaws_logger:close_logs(),\n    ok.\n\n\n%%----------------------------------------------------------------------\n%% Func: code_change/3\n%% Purpose: Handle upgrade\n%% Returns: new State data\n%%----------------------------------------------------------------------\ncode_change(_OldVsn, Data, _Extra) ->\n    {ok, Data}.\n\n%%%----------------------------------------------------------------------\n%%% Internal functions\n%%%----------------------------------------------------------------------\noptional_header(Item) ->\n    case Item of\n        undefined -> \"-\";\n        Item -> Item\n    end.\n\nfmt_access_log(Time, Host, User, Req, Status,  Length, Referrer, UserAgent) ->\n    [Host, \" - \", User, [$\\s], Time, [$\\s, $\\\"], no_ctl(Req), [$\\\",$\\s],\n     Status, [$\\s], Length, [$\\s,$\"], Referrer, [$\",$\\s,$\"], UserAgent,\n     [$\",$\\n]].\n\n\n%% Odd security advisory that only affects webservers where users are\n%% somehow allowed to upload files that later can be downloaded.\n\nno_ctl([H|T]) when H < 32 ->\n    no_ctl(T);\nno_ctl([H|T]) ->\n    [H|no_ctl(T)];\nno_ctl([]) ->\n    [].\n\n\nfmt_ip(IP, State) when is_tuple(IP) ->\n    case State#state.resolve_hostnames of\n        true ->\n            case catch inet:gethostbyaddr(IP) of\n                {ok, HE} ->\n                    HE#hostent.h_name;\n                _ ->\n                    case catch inet_parse:ntoa(IP) of\n                        {'EXIT', _} -> \"unknownip\";\n                        Addr        -> Addr\n                    end\n            end;\n        false ->\n            case catch inet_parse:ntoa(IP) of\n                {'EXIT', _} -> \"unknownip\";\n                Addr        -> Addr\n            end\n    end;\nfmt_ip(unknown, _) ->\n    \"unknownip\";\nfmt_ip(undefined, _) ->\n    \"0.0.0.0\";\nfmt_ip(HostName, _) ->\n    HostName.\n\n\nfmtnow() ->\n    {{Year, Month, Day}, {Hour, Min, Sec}} =\n        calendar:now_to_local_time(yaws:get_time_tuple()),\n    [\"[\",fill_zero(Day,2),\"/\",yaws:month(Month),\"/\",integer_to_list(Year),\":\",\n     fill_zero(Hour,2),\":\",fill_zero(Min,2),\":\",\n     fill_zero(Sec,2),\" \",zone(),\"]\"].\n\n\nzone() ->\n    Time = erlang:universaltime(),\n    LocalTime = calendar:universal_time_to_local_time(Time),\n    DiffSecs = calendar:datetime_to_gregorian_seconds(LocalTime) -\n        calendar:datetime_to_gregorian_seconds(Time),\n    zone(DiffSecs div 3600, (DiffSecs rem 3600) div 60).\n\n\n\nzone(Hr, Min) when Hr < 0; Min < 0 ->\n    [$-, fill_zero(abs(Hr), 2), fill_zero(abs(Min), 2)];\nzone(Hr, Min) when Hr >= 0, Min >= 0 ->\n    [$+, fill_zero(abs(Hr), 2), fill_zero(abs(Min), 2)].\n\nfill_zero(N, Width) ->\n    left_fill(N, Width, $0).\n\nleft_fill(N, Width, Fill) when is_integer(N) ->\n    left_fill(integer_to_list(N), Width, Fill);\nleft_fill(N, Width, _Fill) when length(N) >= Width ->\n    N;\nleft_fill(N, Width, Fill) ->\n    left_fill([Fill|N], Width, Fill).\n\nsafe_log_data(Elements) ->\n    [ yaws:to_string(E) || E <- Elements ].\n\n\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_log_file_h.erl",
    "content": "%%%-------------------------------------------------------------------\n%%% File    : yaws_log_file_h.erl\n%%% Author  :  <klacke@hyber.org>\n%%% Description :\n%%%\n%%% Created : 11 Mar 2004 by  <klacke@hyber.org>\n%%%-------------------------------------------------------------------\n\n\n%% Just extending the error_logger_file_h abit,\n%% If they change the internals of that module, this module\n%% breaks, but then again, the otp crew doesn't ever appear to\n%% change anything that might break anything .....\n\n-module(yaws_log_file_h).\n-behaviour(gen_event).\n-include_lib(\"kernel/include/file.hrl\").\n\n-export([init/1,\n         handle_event/2, handle_call/2, handle_info/2,\n         terminate/2, code_change/3]).\n\n\n\n%% This one is used when we are started directly.\ninit(File) ->\n    process_flag(trap_exit, true),\n    {ok, [Major,Minor], _} = io_lib:fread(\"~d.~d\", erlang:system_info(version)),\n    case file:open(File, [append]) of\n        {ok, Fd} when {Major,Minor} < {7,1} ->  %% Pre 18.1\n            {ok, {Fd, File, []}};\n        {ok, Fd} ->                             %% Post 18.1\n            {ok, {st, Fd, File, [], unlimited}};\n        Error ->\n            error_logger:error_msg(\n              \"Failed to set Yaws error report handler: ~p~n\", [Error]\n             ),\n            Error\n    end.\n\n%% Pre 18.1\nhandle_call(reopen, {Fd, File, Prev}) ->\n    {ok, ok, {reopen(Fd,File), File, Prev}};\nhandle_call(wrap, {Fd, File, Prev}) ->\n    {ok, ok, {wrap(Fd,File), File, Prev}};\nhandle_call(size, {Fd, File, Prev}) ->\n    {ok, size(Fd,File), {Fd, File, Prev}};\n\n%% Post 18.1\nhandle_call(reopen, {st, Fd, File, Prev, Depth}) ->\n    {ok, ok, {st, reopen(Fd,File), File, Prev, Depth}};\nhandle_call(wrap, {st, Fd, File, Prev, Depth}) ->\n    {ok, ok, {st, wrap(Fd,File), File, Prev, Depth}};\nhandle_call(size, {st, Fd, File, Prev, Depth}) ->\n    {ok, size(Fd,File), {st, Fd, File, Prev, Depth}};\n\nhandle_call(X, S) ->\n    error_logger_file_h:handle_call(X,S).\n\n\nhandle_event(X, S) ->\n    error_logger_file_h:handle_event(X, S).\nhandle_info(X, S) ->\n    error_logger_file_h:handle_info(X, S).\n\n\nterminate(Reason, State) ->\n    error_logger_file_h:terminate(Reason, State).\n\ncode_change(_OldVsn, State, _Extra) ->\n    {ok, State}.\n\n\nreopen(Fd, File) ->\n    file:close(Fd),\n    {ok, Fd2} = file:open(File, [write,append]),\n    Fd2.\n\nwrap(Fd, File) ->\n    Old = File ++ \".old\",\n    file:delete(Old),\n    file:close(Fd),\n    file:rename(File, Old),\n    {ok, Fd2} = file:open(File, [write,append]),\n    Fd2.\n\nsize(Fd, File) ->\n    file:sync(Fd),\n    case file:read_file_info(File) of\n        {ok, FI} -> {ok, FI#file_info.size};\n        Error    -> Error\n    end.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_logger.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_logger.erl\n%%% Author  : Christopher Faulet <christopher@yakaz.com>\n%%% Purpose :\n%%% Created : 14 Dec 2010 by Christopher Faulet <christopher@yakaz.com>\n%%%----------------------------------------------------------------------\n\n-module(yaws_logger).\n-author('christopher@yakaz.com').\n-include_lib(\"kernel/include/file.hrl\").\n\n\n-export([behaviour_info/1]).\n\n%% API\n-export([\n         open_log/3,\n         close_log/2,\n         close_logs/0,\n         rotate/1,\n\n         accesslog/6,\n         authlog/4\n        ]).\n\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_debug.hrl\").\n\n\n-record(log, {id, amod, data}).\n\n%%%----------------------------------------------------------------------\n%%% API\n%%%----------------------------------------------------------------------\nbehaviour_info(callbacks) ->\n    [{open_log,3}, {close_log,3}, {wrap_log,4}, {write_log,4}];\nbehaviour_info(_Other) ->\n    undefined.\n\n\nopen_log(SConf, auth, Dir) when ?sc_has_auth_log(SConf) ->\n    do_open_log(SConf, auth, Dir);\nopen_log(SConf, access, Dir) when ?sc_has_access_log(SConf) ->\n    do_open_log(SConf, access, Dir);\nopen_log(_, _, _) ->\n    false.\n\n\nclose_log(SConf, Type) ->\n    case ets:lookup(yaws_log, {Type, SConf#sconf.servername}) of\n        [AL] ->\n            do_close_log(AL),\n            ets:delete(yaws_log, {Type, SConf#sconf.servername}),\n            ok;\n        [] ->\n            ok\n    end.\n\n\nclose_logs() ->\n    do_close_logs(ets:first(yaws_log)),\n    ets:delete_all_objects(yaws_log),\n    ok.\n\n\nrotate(LogWrapSize) ->\n    do_rotate(ets:first(yaws_log), LogWrapSize).\n\n\naccesslog(#sconf{servername=Srv}, Ip, Req, InH, OutH, Time) ->\n    case ets:lookup(yaws_log, {access, Srv}) of\n        [#log{amod=Mod, data=Data}] ->\n            catch Mod:write_log(Srv, access, Data, {Ip, Req, InH, OutH, Time});\n        _ ->\n            ok\n    end.\n\nauthlog(#sconf{servername=Srv}, IP, Path, Item) ->\n    case ets:lookup(yaws_log, {auth, Srv}) of\n        [#log{amod=Mod, data=Data}] ->\n            catch Mod:write_log(Srv, auth, Data, {IP, Path, Item});\n        _ ->\n            ok\n    end.\n\n%%%----------------------------------------------------------------------\n%%% Internal functions\n%%%----------------------------------------------------------------------\n\ndo_open_log(#sconf{servername=Srv, logger_mod=Mod}, Type, Dir) ->\n    Id = {Type, Srv},\n    case ets:lookup(yaws_log, Id) of\n        [] ->\n            case catch Mod:open_log(Srv, Type, Dir) of\n                {true, Data} ->\n                    AL = #log{id={Type, Srv}, amod=Mod, data=Data},\n                    ets:insert(yaws_log, AL),\n                    true;\n                _ ->\n                    false\n            end;\n        _ ->\n            %% Already exists. Might be the case that both http and https\n            %% has been enabled and we don't want to open the same log twice.\n            true\n    end.\n\n\ndo_close_log(#log{id={Type, Srv}, amod=Mod, data=Data}) ->\n    catch Mod:close_log(Srv, Type, Data).\n\n\ndo_close_logs('$end_of_table') ->\n    ok;\ndo_close_logs(Id) ->\n    [AL] = ets:lookup(yaws_log, Id),\n    do_close_log(AL),\n    do_close_logs(ets:next(yaws_log, Id)).\n\n\ndo_rotate('$end_of_table', _) ->\n    ok;\ndo_rotate(Id, LogWrapSize) ->\n    [#log{id={Type, Srv}, amod=Mod, data=Data}=AL] = ets:lookup(yaws_log, Id),\n    Data1 = Mod:wrap_log(Srv, Type, Data, LogWrapSize),\n    ets:insert(yaws_log, AL#log{data=Data1}),\n    do_rotate(ets:next(yaws_log, Id), LogWrapSize).\n"
  },
  {
    "path": "contrib/yaws/src/yaws_ls.erl",
    "content": "%% -*- coding: latin-1 -*-\n%%%----------------------------------------------------------------------\n%%% File    : yaws_ls.erl\n%%% Author  : Claes Wikstrom <klacke@hyber.org>\n%%% Purpose :\n%%% Created :  5 Feb 2002 by Claes Wikstrom <klacke@hyber.org>\n%%% Modified: 13 Jan 2004 by Martin Bjorklund <mbj@bluetail.com>\n%%% Modified:    Jan 2006 by Sbastien Bigot <sebastien.bigot@tremplin-utc.net>\n%%%----------------------------------------------------------------------\n\n-module(yaws_ls).\n-author('klacke@hyber.org').\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_debug.hrl\").\n\n-include_lib(\"kernel/include/file.hrl\").\n-export([list_directory/6, out/1]).\n\n%% Exports for EUNIT.\n-export([parse_query/1, trim/2]).\n\n-define(FILE_LEN_SZ, 45).\n\nlist_directory(_Arg, CliSock, List, DirName, Req, DoAllZip) ->\n    {abs_path, Path} = Req#http_request.path,\n    {DirStr, Pos, Direction, Qry} = parse_query(Path),\n    ?Debug(\"List=~p Dirname~p~n\", [List, DirName]),\n\n    Descriptions = read_descriptions(DirName),\n\n    L0 = lists:zf(\n           fun(F) ->\n                   File = DirName ++ [$/|F],\n                   FI = file:read_file_info(File),\n                   file_entry(FI, DirName, F, Qry,Descriptions)\n           end, List),\n\n    L1 = lists:keysort(Pos, L0),\n\n    L2 = if Direction == normal -> L1;\n            Direction == reverse -> lists:reverse(L1)\n         end,\n\n    L3 = [Html || {_, _, _, _, Html} <- L2],\n\n    Body = [ doc_head(DirStr),\n             dir_header(DirName,DirStr),\n             table_head(Direction),\n             parent_dir(),\n             if\n                 DoAllZip == true ->\n                     allzip();\n                 DoAllZip == true_nozip ->\n                     [];\n                 true ->\n                     []\n             end,\n\n             %%              if DoAllGZip == true -> alltgz() end,\n             %%              if DoAllBZip2 == true -> alltbz2() end,\n\n             %%              if DoAllZip == true -> alltgz() end,\n             %%              if DoAllZip == true -> alltbz2() end,\n\n             L3,\n             table_tail(),\n             dir_footer(DirName),%yaws:address(),\n             doc_tail()\n           ],\n\n    B = unicode:characters_to_binary(Body),\n\n    %% Always use UTF-8 encoded file names. So, set the UTF-8 charset in the\n    %% Content-Type header\n    NewCT = case yaws:outh_get_content_type() of\n                undefined ->\n                    \"text/html; charset=utf-8\";\n                CT0 ->\n                    [CT|_] = yaws:split_sep(CT0, $;),\n                    CT++\"; charset=utf-8\"\n            end,\n    yaws:outh_set_content_type(NewCT),\n\n    yaws_server:accumulate_content(B),\n    yaws_server:deliver_accumulated(CliSock),\n    yaws_server:done_or_continue().\n\nparse_query(Path) ->\n    case string:tokens(Path, [$?]) of\n        [DirStr, [PosC, $=, DirC] = Q] ->\n            Pos = case PosC of\n                      $m -> 2;\n                      $M -> 2; % last modified\n                      $s -> 3;\n                      $S -> 3; % size\n                      $d -> 4;\n                      $D -> 4; % description\n                      _  -> 1  % name (default)\n                  end,\n            Dir = case DirC of\n                      $r -> reverse;\n                      _  -> normal\n                  end,\n            {DirStr, Pos, Dir, \"/?\"++Q};\n        _ ->\n            {Path, 1, normal, \"/\"}\n    end.\n\nparse_description(Line) ->\n    L = string:strip(Line),\n    Pos = string:chr(L,$ ),\n    Filename = string:substr(L, 1, Pos-1),\n    D = string:substr(L,Pos+1),\n    Description = string:strip(D,left),\n    {Filename,Description}.\n\nread_descriptions(DirName) ->\n    File = filename:join(DirName, \"MANIFEST.txt\"),\n    case file:read_file(File) of\n        {ok,Bin} -> Lines = string:tokens(binary_to_list(Bin),\"\\n\"),\n                    lists:map(fun parse_description/1,Lines);\n        _ -> []\n    end.\n\nget_description(Name,Descriptions) ->\n    case lists:keysearch(Name,1,Descriptions) of\n        {value, {_,Description}} -> Description;\n        _ -> []\n    end.\n\ndoc_head(DirName) ->\n    EncDirName = file_display_name(yaws_api:url_decode(DirName)),\n    HtmlDirName = yaws_api:htmlize(EncDirName),\n    ?F(\"<!DOCTYPE html PUBLIC \\\"-//W3C//DTD XHTML 1.0 Strict//EN\\\" \\\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\\\">\\n\"\n       \"<html>\\n\"\n       \" <head>\\n\"\n       \"  <meta charset=\\\"utf-8\\\">\"\n       \"  <title>Index of ~ts</title>\\n\"\n       \"  <style type=\\\"text/css\\\">\\n\"\n       \"    img { border: 0; padding: 0 2px; vertical-align: text-bottom; }\\n\"\n       \"    td  { font-family: monospace; padding: 2px 3px; text-align:left;\\n\"\n       \"          vertical-align: bottom; white-space: pre; }\\n\"\n       \"    td:first-child { text-align: left; padding: 2px 10px 2px 3px; }\\n\"\n       \"    table { border: 0; }\\n\"\n       \"  </style>\\n\"\n       \"</head> \\n\"\n       \"<body>\\n\",\n       [list_to_binary(HtmlDirName)]\n      ).\n\ndoc_tail() ->\n    \"</body>\\n\"\n        \"</html>\\n\".\n\ntable_head(Direction) ->\n    NextDirection = if Direction == normal  -> \"r\";\n                       Direction == reverse -> \"n\"\n                    end,\n    [\"<table>\\n\"\n     \"  <tr>\\n\"\n     \"    <td><img src=\\\"/icons/blank.gif\\\" alt=\\\"&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\\\"/><a href=\\\"?N=\",NextDirection,\"\\\">Name</a></td>\\n\"\n     \"    <td><a href=\\\"?M=\",NextDirection,\"\\\">Last Modified</a></td>\\n\"\n     \"    <td><a href=\\\"?S=\",NextDirection,\"\\\">Size</a></td>\\n\"\n     \"    <td><a href=\\\"?D=\",NextDirection,\"\\\">Description</a></td>\\n\"\n     \"  </tr>\\n\"\n     \"  <tr><th colspan=\\\"4\\\"><hr/></th></tr>\\n\"].\n\ntable_tail() ->\n    \"  <tr><th colspan=\\\"4\\\"><hr/></th></tr>\\n\"\n        \"</table>\\n\".\n\n\ndir_footer(DirName) ->\n    File = DirName ++ [$/ | \"README.txt\"],\n    case file:read_file(File) of\n        {ok,Bin} -> \"<pre>\\n\" ++ binary_to_list(Bin) ++ \"</pre>\\n\";\n        _ -> yaws:address()\n    end.\n\ndir_header(DirName,DirStr) ->\n    File = DirName ++ [$/ | \"HEADER.txt\"],\n    case file:read_file(File) of\n        {ok,Bin} ->\n            \"<pre>\\n\" ++ binary_to_list(Bin) ++ \"</pre>\\n\";\n        _ ->\n            EncDirStr   = file_display_name(yaws_api:url_decode(DirStr)),\n            HtmlDirName = yaws_api:htmlize(EncDirStr),\n            ?F(\"<h1>Index of ~ts</h1>\\n\", [list_to_binary(HtmlDirName)])\n    end.\n\nparent_dir() ->\n    {Gif, Alt} = list_gif(directory,\".\"),\n    ?F(\"  <tr>\\n\"\n       \"    <td><img src=~p alt=~p/><a href=\\\"..\\\">Parent Directory</a></td>\\n\"\n       \"    <td></td>\\n\"\n       \"    <td>-</td>\\n\"\n       \"    <td></td>\\n\"\n       \"  </tr>\\n\",\n       [\"/icons/\" ++ Gif,\n        Alt\n       ]).\n\n%% FIXME: would be nice with a good size approx.  but it would require\n%% a deep scan of possibly the entire docroot, (and also some knowledge\n%% about zip's compression ratio in advance...)\nallzip() ->\n    {Gif, Alt} = list_gif(zip,\"\"),\n    ?F(\"  <tr>\\n\"\n       \"    <td><img src=~p alt=~p/><a href=\\\"all.zip\\\">all.zip</a></td>\\n\"\n       \"    <td></td>\\n\"\n       \"    <td>-</td>\\n\"\n       \"    <td>Build a zip archive of current directory</td>\\n\"\n       \"  </tr>\\n\",\n       [\"/icons/\" ++ Gif,\n        Alt]).\n\n%% alltgz() ->\n%%    {Gif, Alt} = list_gif(zip,\"\"),\n%%    ?F(\"  <tr>\\n\"\n%%       \"    <td><img src=~p alt=~p/><a href=\\\"all.tgz\\\">all.tgz</a></td>\\n\"\n%%       \"    <td></td>\\n\"\n%%       \"    <td>-</td>\\n\"\n%%       \"    <td>Build a gzip archive of current directory</td>\\n\"\n%%       \"  </tr>\\n\",\n%%       [\"/icons/\" ++ Gif,\n%%        Alt]).\n\n%% alltbz2() ->\n%%    {Gif, Alt} = list_gif(zip,\"\"),\n%%    ?F(\"  <tr>\\n\"\n%%       \"    <td><img src=~p alt=~p/><a href=\\\"all.tbz2\\\">all.tbz2</a></td>\\n\"\n%%       \"    <td></td>\\n\"\n%%       \"    <td>-</td>\\n\"\n%%       \"    <td>Build a bzip2 archive of current directory</td>\\n\"\n%%       \"  </tr>\\n\",\n%%       [\"/icons/\" ++ Gif,\n%%        Alt]).\n\nis_user_dir(SP) ->\n    case SP of\n        [$/,$~ | T] -> User = string:sub_word(T,1,$/),\n                       case catch yaws:user_to_home(User) of\n                           {'EXIT', _} ->\n                               false;\n                           Home ->\n                               {true,Home}\n                       end;\n        _ -> false\n    end.\n\nout(A) ->\n    SP = A#arg.server_path,\n    PP = A#arg.appmod_prepath,\n    Dir = case is_user_dir(SP) of\n              {true,Home} -> Home ++ \"/public_html\";\n              false -> A#arg.docroot\n          end ++ PP,\n\n    %%    {html,?F(\"<h2>~p</h2>\",[Dir])}.\n\n    YPid = self(),\n\n    Forbidden_Paths = accumulate_forbidden_paths(),\n    case filename:basename(A#arg.server_path) of\n        \"all.zip\" -> spawn_link(fun() -> zip(YPid, Dir, Forbidden_Paths) end),\n                     {streamcontent, \"application/zip\", \"\"}\n                     %%        \"all.tgz\" -> spawn_link(fun() -> tgz(YPid, Dir) end),\n                     %%                     {streamcontent, \"application/gzip\", \"\"};\n                     %%        \"all.tbz2\" -> spawn_link(fun() -> tbz2(YPid, Dir) end),\n                     %%                     {streamcontent, \"application/gzip\", \"\"}\n    end.\n\n\ngenerate_random_fn() ->\n    Bytes = try yaws_dynopts:rand_bytes(64) of\n                B when is_bitstring(B) ->\n                    B\n            catch _:_ ->\n                    %% for installations without crypto\n                    << <<(yaws_dynopt:random_uniform(256) - 1)>> || _ <- lists:seq(1,64) >>\n            end,\n    << Int:512/unsigned-big-integer >> = << Bytes/binary >>,\n    integer_to_list(Int).\n\nmktempfilename([]) ->\n    {error, no_temp_dir};\nmktempfilename([Dir|R]) ->\n    RandomFN = generate_random_fn(),\n    Filename = filename:join(Dir, RandomFN),\n    case file:open(Filename, [write]) of\n        {ok, FileHandle} ->\n            {ok, {Filename, FileHandle}};\n        _Else ->\n            mktempfilename(R)\n    end.\n\nmktempfilename() ->\n    %% TODO: Add code to determine the temporary directory on various\n    %% operating systems.\n    PossibleDirs = [\"/tmp\", \"/var/tmp\"],\n    mktempfilename(PossibleDirs).\n\nzip(YPid, Dir, ForbiddenPaths) ->\n    {ok, RE_ForbiddenNames} = re:compile(\"\\\\.yaws\\$\", [unicode]),\n    Files = dig_through_dir(Dir, ForbiddenPaths, RE_ForbiddenNames),\n    {ok, {Tempfile, TempfileH}} = mktempfilename(),\n    file:write(TempfileH, lists:foldl(fun(I, Acc) ->\n                                              [Acc, list_to_binary(file_display_name(I)), \"\\n\"]\n                                      end, [], Files)),\n    file:close(TempfileH),\n    process_flag(trap_exit, true),\n    %% TODO: find a way to directly pass the list of files to\n    %% zip. Erlang ports do not allow stdin to be closed\n    %% independently; however, zip needs stdin to be closed as an\n    %% indicator that the list of files is complete.\n    P = open_port({spawn, \"zip -q -1 - -@ < \" ++ Tempfile},\n                  [{cd, Dir},use_stdio, binary, exit_status]),\n    F = fun() ->\n                file:delete(Tempfile)\n        end,\n    stream_loop(YPid, P, F).\n\naccumulate_forbidden_paths() ->\n    SC = get(sc),\n    Auth = SC#sconf.authdirs,\n    lists:foldl(fun({Path, _Auth}, Acc) ->\n                        Acc ++ [Path]\n                end, [], Auth).\n\n\n%% tgz(YPid, Dir) ->\n%%    process_flag(trap_exit, true),\n%%    P = open_port({spawn, \"tar cz .\"},\n%%                  [{cd, Dir},use_stdio, binary, exit_status]),\n%%    stream_loop(YPid, P).\n\n%% tbz2(YPid, Dir) ->\n%%     process_flag(trap_exit, true),\n%%     P = open_port({spawn, \"tar cj .\"},\n%%                   [{cd, Dir},use_stdio, binary, exit_status]),\n%%     stream_loop(YPid, P).\n\ndir_contains_indexfile(_Dir, []) ->\n    false;\ndir_contains_indexfile(Dir, [File|R]) ->\n    case file:read_file_info(filename:join(Dir, File)) of\n        {ok, _} ->\n            true;\n        _Else ->\n            dir_contains_indexfile(Dir, R)\n    end.\n\ndir_contains_indexfile(Dir) ->\n    Indexfiles = [\".yaws.auth\", \"index.yaws\", \"index.html\", \"index.htm\"],\n    dir_contains_indexfile(Dir, Indexfiles).\n\ndig_through_dir(Basedirlen, Dir, ForbiddenPaths, RE_ForbiddenNames) ->\n    Dir1 = string:sub_string(Dir, Basedirlen),\n    case {lists:member(Dir1, ForbiddenPaths),\n          dir_contains_indexfile(Dir)} of\n        {true,_} ->\n            [];\n        {_,true} ->\n            [];\n        {false, false} ->\n            {ok, Files} = file:list_dir(Dir),\n            lists:foldl(fun(I, Acc) ->\n                                Filename = filename:join(Dir, I),\n                                case {file:read_file_info(Filename),\n                                      re:run(Filename, RE_ForbiddenNames)} of\n                                    {_, {match, _}} ->\n                                        Acc;\n                                    {{ok, #file_info{type=directory}}, _} ->\n                                        Acc ++ dig_through_dir(\n                                                 Basedirlen,\n                                                 Filename,\n                                                 ForbiddenPaths,\n                                                 RE_ForbiddenNames);\n                                    {{ok, #file_info{type=regular}}, _} ->\n                                        Acc ++ [string:sub_string(\n                                                  Filename, Basedirlen)];\n                                    _Else ->\n                                        Acc %% Ignore other files\n                                end\n                        end, [], Files)\n    end.\n\ndig_through_dir(Dir, ForbiddenPaths, RE_ForbiddenNames) ->\n    dig_through_dir(length(Dir) + 1,\n                    Dir,\n                    ForbiddenPaths,\n                    RE_ForbiddenNames).\n\nstream_loop(YPid, P, FinishedFun) ->\n    receive\n        {P, {data, Data}} ->\n            yaws_api:stream_chunk_deliver_blocking(YPid, Data),\n            stream_loop(YPid, P, FinishedFun);\n        {P, {exit_status, _}} ->\n            yaws_api:stream_chunk_end(YPid),\n            FinishedFun();\n        {'EXIT', YPid, Status} ->\n            FinishedFun(),\n            exit(Status);\n        Else ->\n            FinishedFun(),\n            error_logger:error_msg(\"Could not deliver zip file: ~p\\n\", [Else])\n    end.\n\nfile_entry({ok, FI}, _DirName, Name, Qry, Descriptions) ->\n    ?Debug(\"file_entry(~p) \", [Name]),\n    Ext = filename:extension(Name),\n    {Gif, Alt} = list_gif(FI#file_info.type, Ext),\n    QryStr = if FI#file_info.type == directory -> Qry;\n                true -> \"\"\n             end,\n\n    EncName  = file_display_name(Name),\n    Description = get_description(Name,Descriptions),\n\n    Entry =\n        ?F(\"  <tr>\\n\"\n           \"    <td><img src=~p alt=~p/><a href=~p title=\\\"~ts\\\">~ts</a></td>\\n\"\n           \"    <td>~s</td>\\n\"\n           \"    <td>~s</td>\\n\"\n           \"    <td>~s</td>\\n\"\n           \"  </tr>\\n\",\n           [\"/icons/\" ++ Gif,\n            Alt,\n            yaws_api:url_encode(Name) ++ QryStr,\n            list_to_binary(EncName),\n            list_to_binary(trim(EncName,?FILE_LEN_SZ)),\n            datestr(FI),\n            sizestr(FI),\n            Description]),\n    ?Debug(\"Entry:~p\", [Entry]),\n\n    {true, {EncName, FI#file_info.mtime, FI#file_info.size, Description, Entry}};\n\nfile_entry(_Err, _, _Name, _, _) ->\n    ?Debug(\"no entry for ~p: ~p\", [_Name, _Err]),\n    false.\n\ntrim(L,N) ->\n    trim(L,N,[]).\ntrim([_H1,_H2,_H3]=[H|T], 3=I, Acc) ->\n    trim(T, I-1, [H|Acc]);\ntrim([H1,H2,H3|_T], 3=_I, Acc) when H1 < 128, H2 < 128, H3 < 128 ->\n    lists:reverse(Acc) ++ \"..&gt;\";\ntrim([H1,H2,H3|_T], 3=_I, [H0|Acc]) ->\n    %% Drop UTF8 continuation bytes: 10xxxxxx\n    Hs0 = lists:dropwhile(fun(Byte) -> Byte bsr 6 == 2#10 end, [H3,H2,H1,H0]),\n    %% Drop UTF8 leading byte: 11xxxxxx\n    Hs = lists:dropwhile(fun(Byte) -> Byte bsr 6 == 2#11 end, Hs0),\n    lists:reverse(Hs++Acc) ++ \"..&gt;\";\ntrim([H|T], I, Acc) ->\n    trim(T, I-1, [H|Acc]);\ntrim([], _I, Acc) ->\n    lists:reverse(Acc).\n\n%% FI -> 16-Jan-2006 23:06\ndatestr(FI) ->\n    {{Year, Month, Day}, {Hour, Min, _}} = FI#file_info.mtime,\n    io_lib:format(\"~s-~s-~w ~s:~s\",\n                  [yaws:mk2(Day),yaws:month(Month),Year,\n                   yaws:mk2(Hour),yaws:mk2(Min)]).\n\nsizestr(FI) when FI#file_info.size > 1000000 ->\n    ?F(\"~.1fM\", [FI#file_info.size / 1000000]);\nsizestr(FI) when FI#file_info.size > 1000 ->\n    ?F(\"~wk\", [trunc(FI#file_info.size / 1000)]);\nsizestr(FI) when FI#file_info.size == 0 ->\n    ?F(\"0k\", []);\nsizestr(_FI) ->\n    ?F(\"1k\", []). % As apache does it...\n\nlist_gif(directory, \".\") ->\n    {\"back.gif\", \"[DIR]\"};\nlist_gif(regular, \".txt\") ->\n    {\"text.gif\", \"[TXT]\"};\nlist_gif(regular, \".c\") ->\n    {\"c.gif\", \"[&nbsp;&nbsp;&nbsp;]\"};\nlist_gif(regular, \".dvi\") ->\n    {\"dvi.gif\", \"[&nbsp;&nbsp;&nbsp;]\"};\nlist_gif(regular, \".pdf\") ->\n    {\"pdf.gif\", \"[&nbsp;&nbsp;&nbsp;]\"};\nlist_gif(regular, _) ->\n    {\"layout.gif\", \"[&nbsp;&nbsp;&nbsp;]\"};\nlist_gif(directory, _) ->\n    {\"dir.gif\", \"[DIR]\"};\nlist_gif(zip, _) ->\n    {\"compressed.gif\", \"[DIR]\"};\nlist_gif(_, _) ->\n    {\"unknown.gif\", \"[OTH]\"}.\n\n\n%% Assume that all file names are UTF-8 encoded. If the VM uses ISO-latin-1\n%% encoding, then no conversion is needed (file already returns the byte\n%% representation of file names). If the VM uses UTF-8, we need to do a little\n%% conversion to return the byte representation of file names.\nfile_display_name(Name) ->\n    case file:native_name_encoding() of\n        latin1 -> Name;\n        utf8   -> binary_to_list(unicode:characters_to_binary(Name))\n    end.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_multipart.erl",
    "content": "-module(yaws_multipart).\n-export([read_multipart_form/2]).\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n\n-record(upload, {\n          fd,\n          filename,\n          fixed_filename,\n          last = false,\n          param_name,\n          param_running_value,\n          params,\n          running_file_size = 0,\n          max_file_size,\n          no_temp_file,\n          temp_dir = yaws:tmpdir(\"/tmp\"),\n          temp_file,\n          headers = [],\n          data_type = list\n         }).\n\nread_multipart_form(A, Options) when A#arg.state == undefined ->\n    State = #upload{params = dict:new()},\n    NewState = read_options(Options,State),\n    multipart(A, NewState);\nread_multipart_form(A, _Options) ->\n    multipart(A, A#arg.state).\n\nread_options([], State) -> State;\nread_options([Option|Rest], State) ->\n    NewState = case Option of\n                   {max_file_size, SizeInBytes} ->\n                       State#upload{max_file_size = SizeInBytes};\n                   no_temp_file ->\n                       State#upload{no_temp_file = true};\n                   {temp_file, FullPath} ->\n                       State#upload{fixed_filename = FullPath};\n                   {temp_dir, Dir} ->\n                       true = filelib:is_dir(Dir),\n                       State#upload{temp_dir = Dir};\n                   list ->\n                       State#upload{data_type = list};\n                   binary ->\n                       State#upload{data_type = binary}\n               end,\n    read_options(Rest, NewState).\n\nmultipart(A, State) ->\n    Parse = yaws_api:parse_multipart_post(A, [State#upload.data_type]),\n    case Parse of\n        {cont, Cont, Res} ->\n            case add_file_chunk(A, Res, State) of\n                {done, NewState} ->\n                    {done, NewState#upload.params};\n                {cont, NewState} ->\n                    {get_more, Cont, NewState};\n                Error={error, _Reason} ->\n                    Error\n            end;\n        {result, Res} ->\n            case add_file_chunk(A, Res, State#upload{last=true}) of\n                {done, S2} ->\n                    {done,S2#upload.params};\n                Error={error, _Reason} ->\n                    Error\n            end;\n        Error={error, _Reason} ->\n            Error\n    end.\n\n\nadd_file_chunk(A, [{part_body, Data}|Res], State) ->\n    add_file_chunk(A, [{body, Data}|Res], State);\n\nadd_file_chunk(_A, [], State) when State#upload.last == true ->\n    {done, close_previous_param(State)};\n\nadd_file_chunk(_A, [], State) ->\n    {cont, State};\n\nadd_file_chunk(A, [{head, {_Name, Opts}}|Res], State ) ->\n    S1 = close_previous_param(State),\n    S2 = lists:foldl(\n           fun({\"filename\", Fname0}, RunningState) ->\n                   case create_temp_file(State) of\n                       [undefined, undefined] ->\n                           %% values will be stored in memory as\n                           %% dictated by state#upload.no_temp_file\n                           RunningState#upload{\n                             filename            = Fname0,\n                             param_running_value = undefined,\n                             running_file_size   = 0};\n                       [Fname, Fd] ->\n                           RunningState#upload{\n                             fd                  = Fd,\n                             filename            = Fname0,\n                             temp_file           = Fname,\n                             param_running_value = undefined,\n                             running_file_size   = 0}\n                   end;\n              ({\"name\", ParamName}, RunningState) ->\n                   RunningState#upload{\n                     param_name          = ParamName,\n                     param_running_value = undefined};\n              (HdrVal, RunningState) ->\n                   RunningState#upload{\n                     headers = [HdrVal | RunningState#upload.headers]}\n           end,\n           S1,\n           Opts),\n    add_file_chunk(A,Res,S2);\n\nadd_file_chunk(A, [{body, Data}|Res],State) when State#upload.fd /= undefined ->\n    NewSize = compute_new_size(State,Data),\n    Check   = check_param_size(State, NewSize),\n    case Check of\n        ok ->\n            ok = file:write(State#upload.fd, Data),\n            add_file_chunk(A, Res, State#upload{running_file_size = NewSize});\n        Error={error, _Reason} ->\n            Error\n    end;\n\nadd_file_chunk(A, [{body, Data}|Res], State) ->\n    NewSize = compute_new_size(State,Data),\n    Check   = check_param_size(State, NewSize),\n    case Check of\n        ok ->\n            NewState =\n                case State#upload.param_running_value of\n                    undefined ->\n                        State#upload{param_running_value = Data};\n                    PrevValue ->\n                        NewData = compute_new_value(PrevValue, Data),\n                        State#upload{param_running_value = NewData}\n                end,\n            add_file_chunk(A, Res,NewState#upload{running_file_size = NewSize});\n        Error={error, _Reason} ->\n            Error\n    end.\n\ncreate_temp_file(State) ->\n    case State#upload.no_temp_file of\n        undefined ->\n            FilePath =\n                case State#upload.fixed_filename of\n                    undefined ->\n                        {A, B, C} = yaws:unique_triple(),\n                        FileName = yaws:join_sep([\"yaws\",\n                                                  integer_to_list(A),\n                                                  integer_to_list(B),\n                                                  integer_to_list(C)], \"_\"),\n                        filename:join([State#upload.temp_dir, FileName]);\n                    Filename ->\n                        Filename\n                end,\n            {ok,Fd} = file:open(FilePath, [write,binary]),\n            [FilePath, Fd];\n        true ->\n            [undefined, undefined]\n    end\n        .\n\nclose_previous_param(#upload{param_name = undefined} = State) ->\n    State;\nclose_previous_param(#upload{param_name = ParamName} = State) ->\n    S2 = case State#upload.filename of\n             undefined ->\n                 ParamValue = State#upload.param_running_value,\n                 State#upload{\n                   params = dict:store(ParamName, ParamValue,\n                                       State#upload.params)};\n             _ ->\n                 ParamInfo = [{\"filename\", State#upload.filename}],\n                 ParamInfo2 = case State#upload.fd of\n                                  undefined ->\n                                      lists:append(\n                                        ParamInfo,\n                                        [{value,\n                                          State#upload.param_running_value}]);\n                                  Fd ->\n                                      file:close(Fd),\n                                      lists:append(ParamInfo,\n                                                   [{temp_file,\n                                                     State#upload.temp_file}])\n                              end,\n                 ParamInfo3 = lists:append(ParamInfo2, State#upload.headers),\n                 State#upload{\n                   filename = undefined,\n                   fd       = undefined,\n                   temp_file= undefined,\n                   running_file_size = 0,\n                   params   = dict:store(ParamName, ParamInfo3,\n                                         State#upload.params)\n                  }\n         end,\n    S2#upload{param_name          = undefined,\n              param_running_value = undefined,\n              headers             = []}.\n\ncompute_new_size(State, Data) ->\n    case Data of\n        undefined ->\n            State#upload.running_file_size;\n        _ ->\n            State#upload.running_file_size + iolist_size(Data)\n    end.\n\n\ncheck_param_size(State, NewSize) ->\n    case State#upload.max_file_size of\n        undefined -> ok;\n        MaxSizeInBytes ->\n            case NewSize > MaxSizeInBytes of\n                true ->\n                    {error, io_lib:format(\"~p is too large\",\n                                          [State#upload.param_name])};\n                false ->\n                    ok\n            end\n    end.\n\ncompute_new_value(PrevValue, NewData) ->\n    case NewData of\n        Data when is_binary(NewData) ->\n            <<PrevValue/binary, Data/binary>>;\n        Data when is_list(NewData) ->\n            PrevValue ++ Data\n    end.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_outmod.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_outmod.erl\n%%% Author  : Claes Wikstrom <klacke@hyber.org>\n%%% Purpose :\n%%% Created :  4 Nov 2002 by Claes Wikstrom <klacke@hyber.org>\n%%%----------------------------------------------------------------------\n\n-module(yaws_outmod).\n-author('klacke@hyber.org').\n\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n\n-export([out401/3,\n         out404/3,\n         out404/1,\n         out/1,\n         crashmsg/3]).\n\n\n%% The default error 404 error delivery module\n%% This function can be used to generate\n%% a special page on 404's (it doesn't even have to be a 404)\n\n\n\n\nout404(Arg) ->\n    out404(Arg, get(gc), get(sc)).\nout404(Arg, GC, SC) ->\n    Req = Arg#arg.orig_req,\n    {abs_path, Path} = Req#http_request.path,\n    B = not_found_body(Path, GC, SC),\n    [{status, 404},\n     {header, {content_type, \"text/html\"}},\n     {header, {connection, \"close\"}},\n     {html, B}].\n\n\n\n%% The default error 401 error delivery module\n%% This function can be used to generate\n%% a special page on 401's (it doesn't even have to be a 401)\nout401(_Arg, _Auth, _Realm) ->\n    {ehtml,\n     [{html,[],\n       [\n        {body, [],\n         [{h1,[], \"401 authentication needed\"}\n         ]\n        }\n       ]\n      }\n     ]\n    }.\n\n%% Deprecated, out401/3 will be used\nout(_Arg) ->\n     {ehtml,\n      [{html,[],\n        [\n         {body, [],\n          [{h1,[], \"401 authentication needed\"}\n          ]\n         }\n        ]\n       }\n      ]\n     }.\n\nnot_found_body(Path, _GC, _SC) ->\n    L = [\"<!DOCTYPE HTML PUBLIC \\\"-//IETF//DTD HTML 2.0//EN\\\">\"\n         \"<HTML><HEAD>\"\n         \"<TITLE>404 Not Found</TITLE>\"\n         \"</HEAD><BODY>\"\n         \"<H1>Not Found</H1>\"\n         \"The requested URL \",\n         yaws_api:htmlize(Path),\n         \" was not found on this server.<P>\"\n         \"<HR>\",\n         yaws:address(),\n         \"  </BODY></HTML>\"\n        ],\n    list_to_binary(L).\n\n\n\n\n%% possibility to customize crash messages,\n\n%% while developing\n%% it's extremely convenient to get the crash messages in the browser,\n%% however not in production :-)\n%% This function can only return an {ehtml, EH} or an {html, HTML}\n%% value, no status codes, no headers etc.\ncrashmsg(_Arg, _SC, L) ->\n    %% Hide user/password in auth structures\n    RE = \"{\\\"[^\\\"]+\\\"\\\\\\s*,\\\\\\s*(md5|ripemd160|sha|sha224|sha256|sha384|sha512)\\\\\\s*,\\\\\\s*[^}]+}\",\n    L1 = re:replace(L, RE, \"#####\", [global, noteol, {return, list}]),\n    error_logger:format(\"~s\", [L1]),\n    {ehtml,\n     [{html, [],\n       [{body, [],\n         [{h2, [], \"Internal error, yaws code crashed\"},\n          {br},\n          {hr},\n          {pre, [], yaws_api:htmlize(L1)},\n          {hr}]\n        }\n       ]\n      }\n     ]\n    }.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_pam.erl",
    "content": "%%%-------------------------------------------------------------------\n%%% File    : yaws_pam.erl\n%%% Author  :  <klacke@hyber.org>\n%%% Description :\n%%%\n%%% Created : 20 Dec 2005 by  <klacke@hyber.org>\n%%%-------------------------------------------------------------------\n-module(yaws_pam).\n-behaviour(gen_server).\n\n%%--------------------------------------------------------------------\n%% External exports\n-export([start_link/0,\n         start_link/3\n        ]).\n\n%% gen_server callbacks\n-export([init/1, handle_call/3, handle_cast/2, handle_info/2,\n         terminate/2, code_change/3]).\n-export([auth/2,\n         close/1]).\n\n\n-define(TO, (1000 * 60 * 3)).\n\n\n-record(user, {i,    %% sid\n               from, %% pid\n               ref}).%% monitorref\n\n-record(state, {i,\n                port,\n                mode,\n                sids = [], % active sessions [#user{}]\n                srv,\n                reqs = []  % outstandig requests [#user{}]\n               }).\n\n%%====================================================================\n%% External functions\n%%====================================================================\n%%--------------------------------------------------------------------\n%% Function: start_link/0\n%% Description: Starts the server\n%%--------------------------------------------------------------------\nstart_link() ->\n    gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).\nstart_link(Service, UseAccounting, UseSess) ->\n    Args = [Service, UseAccounting, UseSess],\n    gen_server:start_link({local, ?MODULE}, ?MODULE, Args, []).\n\nauth(User, Password) ->\n    case has_nul(User) or has_nul(Password) of\n        false ->\n            try  gen_server:call(?MODULE, {auth, User, Password}, 15000) of\n                 Ret ->\n                    Ret\n            catch _:_ ->\n                    {no, {\"auth\", \"timeout\"}}\n            end;\n        true ->\n            %% PAM can't handle embedded NUL (nor can the \"port protocol\")\n            %% - and it's probably a DOS attempt anyway, do a delay\n            timer:sleep(1000),\n            {no, {\"auth\", \"Authentication failure\"}}\n    end.\n\n%% yaws never use close, ... no session mgmt in yaws\nclose(Handle) ->\n    gen_server:call(?MODULE, {close, Handle}, infinity).\n\nhas_nul(<<B/binary>>) ->\n    lists:member(0, binary_to_list(B));\nhas_nul(L) ->\n    lists:member(0, L).\n\n\n%%====================================================================\n%% Server functions\n%%====================================================================\n\n%%--------------------------------------------------------------------\n%% Function: init/1\n%% Description: Initiates the server\n%% Returns: {ok, State}          |\n%%          {ok, State, Timeout} |\n%%          ignore               |\n%%          {stop, Reason}\n%%--------------------------------------------------------------------\n\ninit([]) ->\n    {Srv, Act,Sess} =\n        {okundef(application:get_env(pam_service)),\n         okundef(application:get_env(pam_use_acct)),\n         okundef(application:get_env(pam_use_sess))},\n    init([Srv, Act, Sess]);\n\ninit([undefined, _Act, _Sess]) ->\n    error_logger:format(\"pam: need service in pam environment\\n\", []),\n    {stop, noservice};\ninit([SRV, Act, Sess]) ->\n\n    %% we never want to use the accounting\n    %% in yaws\n\n    M1 = case Act of\n             undefined ->\n                 \"\";\n             true ->\n                 \"A\";\n             false ->\n                 []\n         end,\n    %% and we definitely never want to use the\n    %% the session capability in yaws since noone is never\n    %% ever going to close the session\n    M2 = case Sess of\n             undefined ->\n                 \"\";\n             true ->\n                 \"S\";\n             false ->\n                 []\n         end,\n    Mode = M1 ++ M2,\n\n    %% we're not starting the portprogram now, it's done\n    %% on demand.\n    {ok, #state{i = 0,\n                mode = Mode,\n                srv = SRV,\n                port = undefined,\n                sids = [],\n                reqs = []}}.\n\nokundef({ok,Val}) ->\n    Val;\nokundef(undefined) ->\n    undefined.\n\n\n%%--------------------------------------------------------------------\n%% Function: handle_call/3\n%% Description: Handling call messages\n%% Returns: {reply, Reply, State}          |\n%%          {reply, Reply, State, Timeout} |\n%%          {noreply, State}               |\n%%          {noreply, State, Timeout}      |\n%%          {stop, Reason, Reply, State}   | (terminate/2 is called)\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%--------------------------------------------------------------------\nhandle_call({auth, User, Password}, From, State0) ->\n    State = ensure_port(State0),\n    I = integer_to_list(State#state.i),\n    port_command(State#state.port, [$a, User, 0, Password, 0,\n                                    State#state.mode,0, I, 0]),\n    Ref = erlang:monitor(process, element(1, From)),\n    U = #user{i = State#state.i,\n              ref = Ref,\n              from = From},\n    R = [U| State#state.reqs],\n\n    {noreply, State#state{i = State#state.i + 1,\n                          reqs = R}, ?TO};\n\nhandle_call({close, _Sid}, _From, State = #state{port=undefined}) ->\n    {reply, ok, State};\nhandle_call({close, Sid}, _From, State = #state{port = Port}) ->\n    case lists:keysearch(Sid, #user.i, State#state.sids) of\n        {value, U} ->\n            erlang:demonitor(U#user.ref);\n        false ->\n            ok\n    end,\n    port_command(Port, [$c, integer_to_list(Sid), 0]),\n\n    {reply, ok, State#state{\n                  sids = lists:keydelete(Sid, #user.i, State#state.sids)},?TO}.\n\n\n%%--------------------------------------------------------------------\n%% Function: handle_cast/2\n%% Description: Handling cast messages\n%% Returns: {noreply, State}          |\n%%          {noreply, State, Timeout} |\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%--------------------------------------------------------------------\nhandle_cast(_Msg, State) ->\n    {noreply, State, ?TO}.\n\n%%--------------------------------------------------------------------\n%% Function: handle_info/2\n%% Description: Handling all non call/cast messages\n%% Returns: {noreply, State}          |\n%%          {noreply, State, Timeout} |\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%--------------------------------------------------------------------\nhandle_info(timeout, State) when\n      State#state.port /= undefined,\n      State#state.sids == [],\n      State#state.reqs == [] ->\n    unlink(State#state.port),\n    port_close(State#state.port),\n    {noreply, State#state{port = undefined}};\n\nhandle_info(timeout, State) ->\n    {noreply, State, ?TO};\n\nhandle_info({'EXIT', Port, _}, State = #state{port = Port}) ->\n    error_logger:format(\"epam port program died \\n\",[]),\n    lists:foreach(\n      fun(U) -> gen_server:reply(U#user.from, {no, \"epam died\"}) end,\n      State#state.reqs),\n    {noreply, State#state{sids = [],\n                          reqs = [],\n                          port = undefined\n                         }};\nhandle_info({'DOWN', MonitorRef, _Type, _Object, _Info}, State)\n  when State#state.port /= undefined ->\n    case lists:keysearch(MonitorRef, #user.ref, State#state.sids) of\n        {value, U} ->\n            port_command(State#state.port,\n                         [$c, integer_to_list(U#user.i), 0]),\n            S2 = lists:keydelete(MonitorRef, #user.ref, State#state.sids),\n            {noreply, State#state{sids = S2}, ?TO};\n        false ->\n            {noreply, State, ?TO}\n    end;\n\nhandle_info({'DOWN', _MonitorRef, _Type, _Object, _Info}, State) ->\n    {noreply, State};\n\nhandle_info({_Port, {data, Str}}, State) ->\n    case string:tokens(Str, \" \\n\") of\n        [\"pam\", IntStr | Reply] ->\n            I = list_to_integer(IntStr),\n            {value, U} = lists:keysearch(I, #user.i, State#state.reqs),\n            R = case reply(U#user.from, I, Reply) of\n                    yes ->\n                        [U |State#state.sids];\n                    no ->\n                        State#state.sids\n                end,\n            {noreply,\n             State#state{reqs = lists:keydelete(I,#user.i,State#state.reqs),\n                         sids = R}, ?TO};\n        _Other ->\n            error_logger:format(\"epam: ~s\", [Str]),\n            {noreply, State, ?TO}\n    end.\n\n\n\n%%--------------------------------------------------------------------\n%% Function: terminate/2\n%% Description: Shutdown the server\n%% Returns: any (ignored by gen_server)\n%%--------------------------------------------------------------------\nterminate(_Reason, _State) ->\n    ok.\n\n%%--------------------------------------------------------------------\n%% Func: code_change/3\n%% Purpose: Convert process state when code is changed\n%% Returns: {ok, NewState}\n%%--------------------------------------------------------------------\ncode_change(_OldVsn, State, _Extra) ->\n    {ok, State}.\n\n%%--------------------------------------------------------------------\n%%% Internal functions\n%%--------------------------------------------------------------------\n\n\nreply(From, Sid, [\"yes\"]) ->\n    gen_server:reply(From, {yes, Sid}),\n    yes;\nreply(From, _Sid, [\"no\", What |Reason ]) ->\n    gen_server:reply(From, {no, {What, fsp(Reason)}}),\n    no.\n\nfsp([]) -> [];\nfsp([X]) -> X;\nfsp([H|T]) -> H ++ \" \" ++ fsp(T).\n\n\n\nensure_port(S = #state{port = undefined, srv = Srv}) ->\n    Prg0 = filename:dirname(code:which(?MODULE)) ++\n        \"/../priv/epam \",\n\n    Prg = Prg0 ++ Srv,\n    P = open_port({spawn, Prg}, [{packet, 2}]),\n    receive\n        {P, {data, \"ok\"}} ->\n            S#state{port = P};\n        {P, {data, ErrStr}} ->\n            error_logger:format(\"epam: ~s~n\", [ErrStr]),\n            exit(noepam);\n        {'EXIT', P, _} ->\n            error_logger:format(\"yaws_pam: Cannot start epam\",[]),\n            exit(noepam)\n    end;\nensure_port(S)->\n    S.\n\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_revproxy.erl",
    "content": "%%%-------------------------------------------------------------------\n%%% File    : yaws_revproxy.erl\n%%% Author  :  <klacke@hyber.org>\n%%% Description : reverse proxy\n%%%\n%%% Created :  3 Dec 2003 by  <klacke@hyber.org>\n%%%-------------------------------------------------------------------\n-module(yaws_revproxy).\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_debug.hrl\").\n-export([out/1]).\n\n\n%% reverse proxy implementation.\n\n%% the revproxy internal state\n-record(revproxy, {srvsock,         %% the socket opened on the backend server\n                   type,            %% the socket type: ssl | nossl\n\n                   cliconn_status,  %% \"Connection:\" header value:\n                   srvconn_status,  %%   \"keep-alive' or \"close\"\n\n                   state,           %% revproxy state:\n                                    %%   sendheaders | sendcontent | sendchunk |\n                                    %%   recvheaders | recvcontent | recvchunk |\n                                    %%   terminate\n                   prefix,          %% The prefix to strip and add\n                   url,             %% the url we're proxying to\n                   r_meth,          %% what req method are we processing\n                   r_host,          %%   and value of Host: for the cli request\n\n                   resp,            %% response received from the server\n                   headers,         %%   and associated headers\n                   srvdata,         %% the server data\n                   is_chunked,      %% true if the response is chunked\n                   intercept_mod   %% revproxy request/response intercept module\n                  }).\n\n\n%% TODO: Activate proxy keep-alive with a new option ?\n-define(proxy_keepalive, false).\n\n\n%% Initialize the connection to the backend server. If an error occurred, return\n%% an error 404.\nout(#arg{req=Req, headers=Hdrs, state=#proxy_cfg{url=URL}=State}=Arg) ->\n    case connect(URL) of\n        {ok, Sock, Type} ->\n            ?Debug(\"Connection established on ~p: Socket=~p, Type=~p~n\",\n                   [URL, Sock, Type]),\n            RPState = #revproxy{srvsock       = Sock,\n                                type          = Type,\n                                state         = sendheaders,\n                                prefix        = State#proxy_cfg.prefix,\n                                url           = URL,\n                                r_meth        = Req#http_request.method,\n                                r_host        = Hdrs#headers.host,\n                                intercept_mod = State#proxy_cfg.intercept_mod},\n            out(Arg#arg{state=RPState});\n        _ERR ->\n            ?Debug(\"Connection failed: ~p~n\", [_ERR]),\n            out404(Arg)\n    end;\n\n\n%% Send the client request to the server then check if the request content is\n%% chunked or not\nout(#arg{state=#revproxy{}=RPState}=Arg)\n  when RPState#revproxy.state == sendheaders ->\n    ?Debug(\"Send request headers to backend server: ~n\"\n           \" - ~s~n\", [?format_record(Arg#arg.req, http_request)]),\n\n    Req     = rewrite_request(RPState,  Arg#arg.req),\n    Hdrs0   = Arg#arg.headers,\n    Hdrs    = rewrite_client_headers(RPState, Hdrs0),\n    {NewReq, NewHdrs} = case RPState#revproxy.intercept_mod of\n                            undefined ->\n                                {Req, Hdrs};\n                            InterceptMod ->\n                                case catch InterceptMod:rewrite_request(\n                                             Req, Hdrs) of\n                                    {ok, NewReq0, NewHdrs0} ->\n                                        {NewReq0, NewHdrs0};\n                                    InterceptError ->\n                                        error_logger:error_msg(\n                                          \"revproxy intercept module ~p:\"\n                                          \"rewrite_request failed: ~p~n\",\n                                          [InterceptMod, InterceptError]),\n                                        exit({error, intercept_mod})\n                                end\n                        end,\n    ReqStr  = yaws_api:reformat_request(NewReq),\n    HdrsStr = yaws:headers_to_str(NewHdrs),\n    case send(RPState, [ReqStr, \"\\r\\n\", HdrsStr, \"\\r\\n\"]) of\n        ok ->\n            case yaws:to_lower(Hdrs#headers.transfer_encoding) of\n                \"chunked\" ->\n                    ?Debug(\"Request content is chunked~n\", []),\n                    out(Arg#arg{state=RPState#revproxy{state=sendchunk}});\n                _ ->\n                    out(Arg#arg{state=RPState#revproxy{state=sendcontent}})\n            end;\n\n        {error, Reason} ->\n            ?Debug(\"TCP error: ~p~n\", [Reason]),\n            case Reason of\n                closed -> ok;\n                _      -> close(RPState)\n            end,\n            outXXX(500, Arg)\n    end;\n\n\n%% Send the request content to the server. Here the content is not chunked. But\n%% it can be split because of 'partial_post_size' value.\nout(#arg{state=RPState}=Arg) when RPState#revproxy.state == sendcontent ->\n    case Arg#arg.clidata of\n        {partial, Bin} ->\n            ?Debug(\"Send partial content to backend server: ~p bytes~n\",\n                   [size(Bin)]),\n            case send(RPState, Bin) of\n                ok ->\n                    {get_more, undefined, RPState};\n                {error, Reason} ->\n                    ?Debug(\"TCP error: ~p~n\", [Reason]),\n                    case Reason of\n                        closed -> ok;\n                        _      -> close(RPState)\n                    end,\n                    outXXX(500, Arg)\n            end;\n\n        Bin when is_binary(Bin), Bin /= <<>> ->\n            ?Debug(\"Send content to backend server: ~p bytes~n\", [size(Bin)]),\n            case send(RPState, Bin) of\n                ok ->\n                    RPState1 = RPState#revproxy{state=recvheaders},\n                    out(Arg#arg{state=RPState1});\n                {error, Reason} ->\n                    ?Debug(\"TCP error: ~p~n\", [Reason]),\n                    case Reason of\n                        closed -> ok;\n                        _      -> close(RPState)\n                    end,\n                    outXXX(500, Arg)\n            end;\n\n        _ ->\n            ?Debug(\"no content found~n\", []),\n            RPState1 = RPState#revproxy{state=recvheaders},\n            out(Arg#arg{state=RPState1})\n    end;\n\n\n%% Send the request content to the server. Here the content is chunked, so we\n%% must rebuild the chunk before sending it. Chunks can have different size than\n%% the original request because of 'partial_post_size' value.\nout(#arg{state=RPState}=Arg) when RPState#revproxy.state == sendchunk ->\n    case Arg#arg.clidata of\n        {partial, Bin} ->\n            ?Debug(\"Send chunked content to backend server: ~p bytes~n\",\n                   [size(Bin)]),\n            Res = send(RPState,\n                       [yaws:integer_to_hex(size(Bin)),\"\\r\\n\",Bin,\"\\r\\n\"]),\n            case Res of\n                ok ->\n                    {get_more, undefined, RPState};\n                {error, Reason} ->\n                    ?Debug(\"TCP error: ~p~n\", [Reason]),\n                    case Reason of\n                        closed -> ok;\n                        _      -> close(RPState)\n                    end,\n                    outXXX(500, Arg)\n            end;\n\n        <<>> ->\n            ?Debug(\"Send last chunk to backend server~n\", []),\n            case send(RPState, \"0\\r\\n\\r\\n\") of\n                ok ->\n                    RPState1 = RPState#revproxy{state=recvheaders},\n                    out(Arg#arg{state=RPState1});\n                {error, Reason} ->\n                    ?Debug(\"TCP error: ~p~n\", [Reason]),\n                    case Reason of\n                        closed -> ok;\n                        _      -> close(RPState)\n                    end,\n                    outXXX(500, Arg)\n            end\n    end;\n\n\n%% The request and its content were sent. Now, we try to read the response\n%% headers. Then we check if the response content is chunked or not.\nout(#arg{state=RPState}=Arg) when RPState#revproxy.state == recvheaders ->\n    Res = yaws:http_get_headers(RPState#revproxy.srvsock,\n                                RPState#revproxy.type),\n    case Res of\n        {error, {too_many_headers, _Resp}} ->\n            ?Debug(\"Response headers too large from backend server~n\", []),\n            close(RPState),\n            outXXX(500, Arg);\n\n        {Resp0, RespHdrs0} when is_record(Resp0, http_response) ->\n            ?Debug(\"Response headers received from backend server:~n\"\n                   \" - ~s~n - ~s~n\", [?format_record(Resp0, http_response),\n                                      ?format_record(RespHdrs0, headers)]),\n\n            {Resp, RespHdrs} =\n                case RPState#revproxy.intercept_mod of\n                    undefined ->\n                        {Resp0, RespHdrs0};\n                    InterceptMod ->\n                        case catch InterceptMod:rewrite_response(\n                                     Resp0, RespHdrs0) of\n                            {ok, NewResp, NewRespHdrs} ->\n                                {NewResp, NewRespHdrs};\n                            InterceptError ->\n                                error_logger:error_msg(\n                                  \"revproxy intercept module ~p:\"\n                                  \"rewrite_response failure: ~p~n\",\n                                  [InterceptMod, InterceptError]),\n                                exit({error, intercept_mod})\n                        end\n                end,\n\n            {CliConn, SrvConn} = get_connection_status(\n                                   (Arg#arg.req)#http_request.version,\n                                   Arg#arg.headers, RespHdrs\n                                  ),\n            RPState1 = RPState#revproxy{cliconn_status = CliConn,\n                                        srvconn_status = SrvConn,\n                                        resp           = Resp,\n                                        headers        = RespHdrs},\n            if\n                RPState1#revproxy.r_meth =:= 'HEAD' ->\n                    RPState2 = RPState1#revproxy{state=terminate},\n                    out(Arg#arg{state=RPState2});\n\n                Resp#http_response.status =:= 100 orelse\n                Resp#http_response.status =:= 204 orelse\n                Resp#http_response.status =:= 205 orelse\n                Resp#http_response.status =:= 304 orelse\n                Resp#http_response.status =:= 406 ->\n                    RPState2 = RPState1#revproxy{state=terminate},\n                    out(Arg#arg{state=RPState2});\n\n                true ->\n                    RPState2 =\n                        case {yaws:to_lower(RespHdrs#headers.transfer_encoding),\n                              RespHdrs#headers.content_length} of\n                            {\"chunked\", _} ->\n                                RPState1#revproxy{state=recvchunk};\n                            {_, undefined} ->\n                                RPState1#revproxy{\n                                  cliconn_status=\"close\",\n                                  srvconn_status=\"close\",\n                                  state=recvcontent};\n                            _ ->\n                                RPState1#revproxy{state=recvcontent}\n                        end,\n                    out(Arg#arg{state=RPState2})\n            end;\n\n        {_R, _H} ->\n            %% bad_request\n            ?Debug(\"Bad response received from backend server: ~p~n\", [_R]),\n            close(RPState),\n            outXXX(500, Arg);\n\n        closed ->\n            ?Debug(\"TCP error: ~p~n\", [closed]),\n            outXXX(500, Arg)\n    end;\n\n\n%% The response content is not chunked.\nout(#arg{state=RPState}=Arg) when RPState#revproxy.state == recvcontent ->\n    Len = case (RPState#revproxy.headers)#headers.content_length of\n              undefined -> undefined;\n              CLen      -> list_to_integer(CLen)\n          end,\n    SC=get(sc),\n    if\n        is_integer(Len) andalso Len =< SC#sconf.partial_post_size ->\n            case read(RPState, Len) of\n                {ok, Data} ->\n                    ?Debug(\"Response content received from the backend server: \"\n                           \"~p bytes~n\", [size(Data)]),\n                    RPState1 = RPState#revproxy{state      = terminate,\n                                                is_chunked = false,\n                                                srvdata    = {content, Data}},\n                    out(Arg#arg{state=RPState1});\n                {error, Reason} ->\n                    ?Debug(\"TCP error: ~p~n\", [Reason]),\n                    case Reason of\n                        closed -> ok;\n                        _      -> close(RPState)\n                    end,\n                    outXXX(500, Arg)\n            end;\n\n        is_integer(Len) ->\n            %% Here partial_post_size is always an integer\n            BlockSize  = SC#sconf.partial_post_size,\n            BlockCount = Len div BlockSize,\n            LastBlock  = Len rem BlockSize,\n            SrvData    = {block, BlockCount, BlockSize, LastBlock},\n            RPState1   = RPState#revproxy{state      = terminate,\n                                          is_chunked = true,\n                                          srvdata    = SrvData},\n            out(Arg#arg{state=RPState1});\n\n        true ->\n            SrvData  = {block, undefined, undefined, undefined},\n            RPState1 = RPState#revproxy{state      = terminate,\n                                        is_chunked = true,\n                                        srvdata    = SrvData},\n            out(Arg#arg{state=RPState1})\n    end;\n\n%% The response content is chunked. Read the first chunk here and spawn a\n%% process to read others.\nout(#arg{state=RPState}=Arg) when RPState#revproxy.state == recvchunk ->\n    case read_chunk(RPState) of\n        {ok, Data} ->\n            ?Debug(\"First chunk received from the backend server : \"\n                   \"~p bytes~n\", [size(Data)]),\n            RPState1 = RPState#revproxy{state      = terminate,\n                                        is_chunked = (Data /= <<>>),\n                                        srvdata    = {stream, Data}},\n            out(Arg#arg{state=RPState1});\n        {error, Reason} ->\n            ?Debug(\"TCP error: ~p~n\", [Reason]),\n            case Reason of\n                closed -> ok;\n                _      -> close(RPState)\n            end,\n            outXXX(500, Arg)\n    end;\n\n\n%% Now, we return the result and we let yaws_server deals with it. If it is\n%% possible, we try to cache the connection.\nout(#arg{state=RPState}=Arg) when RPState#revproxy.state == terminate ->\n    case RPState#revproxy.srvconn_status of\n        \"close\" when RPState#revproxy.is_chunked == false -> close(RPState);\n        \"close\" -> ok;\n        _  -> cache_connection(RPState)\n    end,\n\n    AllHdrs = [{header, H} || H <- yaws_api:reformat_header(\n                                     rewrite_server_headers(RPState)\n                                    )],\n    ?Debug(\"~p~n\", [AllHdrs]),\n\n    Res = [\n           {status, (RPState#revproxy.resp)#http_response.status},\n           {allheaders, AllHdrs}\n          ],\n    case RPState#revproxy.srvdata of\n        {content, <<>>} ->\n            Res;\n        {content, Data} ->\n            MimeType = (RPState#revproxy.headers)#headers.content_type,\n            Res ++ [{content, MimeType, Data}];\n\n        {stream, <<>>} ->\n            %% Chunked response with only the last empty chunk: do not spawn a\n            %% process to manage chunks\n            yaws_api:stream_chunk_end(self()),\n            MimeType = (RPState#revproxy.headers)#headers.content_type,\n            Res ++ [{streamcontent, MimeType, <<>>}];\n\n        {stream, Chunk} ->\n            Self = self(),\n            GC   = get(gc),\n            spawn(fun() -> put(gc, GC), recv_next_chunk(Self, Arg) end),\n            MimeType = (RPState#revproxy.headers)#headers.content_type,\n            Res ++ [{streamcontent, MimeType, Chunk}];\n\n        {block, BlockCnt, BlockSz, LastBlock} ->\n            GC  = get(gc),\n            Pid = spawn(fun() ->\n                                put(gc, GC),\n                                receive\n                                    {ok, YawsPid} ->\n                                        recv_blocks(YawsPid, Arg, BlockCnt,\n                                                    BlockSz, LastBlock);\n                                    {discard, YawsPid} ->\n                                        recv_blocks(YawsPid, Arg, 0, BlockSz, 0)\n                                end\n                        end),\n            MimeType = (RPState#revproxy.headers)#headers.content_type,\n            Res ++ [{streamcontent_from_pid, MimeType, Pid}];\n\n        _ ->\n            Res\n    end;\n\n\n%% Catch unexpected state by sending an error 500\nout(#arg{state=RPState}=Arg) ->\n    ?Debug(\"Unexpected revproxy state:~n - ~s~n\",\n           [?format_record(RPState, revproxy)]),\n    case RPState#revproxy.srvsock of\n        undefined -> ok;\n        _         -> close(RPState)\n    end,\n    outXXX(500, Arg).\n\n\n%%==========================================================================\nout404(Arg) ->\n    SC=get(sc),\n    (SC#sconf.errormod_404):out404(Arg,get(gc),SC).\n\n\noutXXX(Code, _Arg) ->\n    Content = [\"<html><h1>\", integer_to_list(Code), $\\ ,\n               yaws_api:code_to_phrase(Code), \"</h1></html>\"],\n    [\n     {status, Code},\n     {header, {connection, \"close\"}},\n     {content, \"text/html\", Content}\n    ].\n\n\n%%==========================================================================\n%% This function is used to read a chunk and to stream it to the client.\nrecv_next_chunk(YawsPid, #arg{state=RPState}=Arg) ->\n    case read_chunk(RPState) of\n        {ok, <<>>} ->\n            ?Debug(\"Last chunk received from the backend server~n\", []),\n            yaws_api:stream_chunk_end(YawsPid),\n            case RPState#revproxy.srvconn_status of\n                \"close\" -> close(RPState);\n                _       -> ok %% Cached by the main process\n            end;\n        {ok, Data} ->\n            ?Debug(\"Next chunk received from the backend server : \"\n                   \"~p bytes~n\", [size(Data)]),\n            yaws_api:stream_chunk_deliver(YawsPid, Data),\n            recv_next_chunk(YawsPid, Arg);\n        {error, Reason} ->\n            ?Debug(\"TCP error: ~p~n\", [Reason]),\n            yaws_api:stream_chunk_end(YawsPid),\n            case Reason of\n                closed -> ok;\n                _      -> close(RPState)\n            end\n    end.\n\n%%==========================================================================\n%% This function reads blocks from the server and streams them to the client.\nrecv_blocks(YawsPid, #arg{state=RPState}=Arg,\n            undefined, undefined, undefined) ->\n    case read(RPState) of\n        {ok, <<>>} ->\n            %% no data, wait 100 msec to avoid time-consuming loop and retry\n            timer:sleep(100),\n            recv_blocks(YawsPid, Arg, undefined, undefined, undefined);\n        {ok, Data} ->\n            ?Debug(\"Response content received from the backend server : \"\n                   \"~p bytes~n\", [size(Data)]),\n            ok = yaws_api:stream_process_deliver(Arg#arg.clisock, Data),\n            recv_blocks(YawsPid, Arg, undefined, undefined, undefined);\n        {error, closed} ->\n            yaws_api:stream_process_end(closed, YawsPid);\n        {error, _Reason} ->\n            ?Debug(\"TCP error: ~p~n\", [_Reason]),\n            yaws_api:stream_process_end(closed, YawsPid),\n            close(RPState)\n    end;\nrecv_blocks(YawsPid, #arg{state=RPState}=Arg, 0, _, 0) ->\n    yaws_api:stream_process_end(Arg#arg.clisock, YawsPid),\n    case RPState#revproxy.srvconn_status of\n        \"close\" -> close(RPState);\n        _       -> ok %% Cached by the main process\n    end;\nrecv_blocks(YawsPid, #arg{state=RPState}=Arg, 0, _, LastBlock) ->\n    Sock = Arg#arg.clisock,\n    case read(RPState, LastBlock) of\n        {ok, Data} ->\n            ?Debug(\"Response content received from the backend server : \"\n                   \"~p bytes~n\", [size(Data)]),\n            ok = yaws_api:stream_process_deliver(Sock, Data),\n            yaws_api:stream_process_end(Sock, YawsPid),\n            case RPState#revproxy.srvconn_status of\n                \"close\" -> close(RPState);\n                _       -> ok %% Cached by the main process\n            end;\n        {error, Reason} ->\n            ?Debug(\"TCP error: ~p~n\", [Reason]),\n            yaws_api:stream_process_end(closed, YawsPid),\n            case Reason of\n                closed -> ok;\n                _      -> close(RPState)\n            end\n    end;\nrecv_blocks(YawsPid, #arg{state=RPState}=Arg, BlockCnt, BlockSz, LastBlock) ->\n    case read(RPState, BlockSz) of\n        {ok, Data} ->\n            ?Debug(\"Response content received from the backend server : \"\n                   \"~p bytes~n\", [size(Data)]),\n            ok = yaws_api:stream_process_deliver(Arg#arg.clisock, Data),\n            recv_blocks(YawsPid, Arg, BlockCnt-1, BlockSz, LastBlock);\n        {error, Reason} ->\n            ?Debug(\"TCP error: ~p~n\", [Reason]),\n            yaws_api:stream_process_end(closed, YawsPid),\n            case Reason of\n                closed -> ok;\n                _      -> close(RPState)\n            end\n    end.\n\n%%==========================================================================\n%% TODO: find a better way to cache connections to backend servers. Here we can\n%% have 1 connection per gserv process for each backend server.\nget_cached_connection(URL) ->\n    Key = lists:flatten(yaws_api:reformat_url(URL)),\n    case erase(Key) of\n        undefined ->\n            undefined;\n        {Sock, nossl} ->\n            case gen_tcp:recv(Sock, 0, 1) of\n                {error, closed} ->\n                    ?Debug(\"Invalid cached connection~n\", []),\n                    undefined;\n                _ ->\n                    ?Debug(\"Found cached connection to ~s~n\", [Key]),\n                    {ok, Sock, nossl}\n            end;\n        {Sock, ssl} ->\n            case ssl:recv(Sock, 0, 1) of\n                {error, closed} ->\n                    ?Debug(\"Invalid cached connection~n\", []),\n                    undefined;\n                _ ->\n                    ?Debug(\"Found cached connection to ~s~n\", [Key]),\n                    {ok, Sock, ssl}\n            end\n    end.\n\ncache_connection(RPState) ->\n    Key = lists:flatten(yaws_api:reformat_url(RPState#revproxy.url)),\n    ?Debug(\"Cache connection to ~s~n\", [Key]),\n    InitDB0 = get(init_db),\n    InitDB1 = lists:keystore(\n                Key, 1, InitDB0,\n                {Key, {RPState#revproxy.srvsock, RPState#revproxy.type}}\n               ),\n    put(init_db, InitDB1),\n    ok.\n\n\n%%==========================================================================\nconnect(URL) ->\n    case get_cached_connection(URL) of\n        {ok, Sock, Type} -> {ok, Sock, Type};\n        undefined        -> do_connect(URL)\n    end.\n\ndo_connect(URL) ->\n    Opts = [\n            binary,\n            {packet,    raw},\n            {active,    false},\n            {reuseaddr, true}\n           ],\n    case URL#url.scheme of\n        http  ->\n            Port = case URL#url.port of\n                       undefined -> 80;\n                       P         -> P\n                   end,\n            case yaws:tcp_connect(URL#url.host, Port, Opts) of\n                {ok, S} -> {ok, S, nossl};\n                Err     -> Err\n            end;\n        https ->\n            Port = case URL#url.port of\n                       undefined -> 443;\n                       P         -> P\n                   end,\n            case yaws:ssl_connect(URL#url.host, Port, Opts) of\n                {ok, S} -> {ok, S, ssl};\n                Err     -> Err\n            end;\n        _ ->\n            {error, unsupported_protocol}\n    end.\n\nsend(#revproxy{srvsock=Sock, type=ssl}, Data) ->\n    ssl:send(Sock, Data);\nsend(#revproxy{srvsock=Sock, type=nossl}, Data) ->\n    gen_tcp:send(Sock, Data).\n\n\nread(#revproxy{srvsock=Sock, type=Type}) ->\n    yaws:setopts(Sock, [{packet, raw}, binary], Type),\n    yaws:do_recv(Sock, 0, Type).\n\nread(RPState, Len) ->\n    yaws:setopts(RPState#revproxy.srvsock, [{packet, raw}, binary],\n                 RPState#revproxy.type),\n    read(RPState, Len, []).\n\nread(_, 0, Data) ->\n    {ok, iolist_to_binary(lists:reverse(Data))};\nread(RPState = #revproxy{srvsock=Sock, type=Type}, Len, Data) ->\n    case yaws:do_recv(Sock, Len, Type) of\n        {ok, Bin}       -> read(RPState, Len-size(Bin), [Bin|Data]);\n        {error, Reason} -> {error, Reason}\n    end.\n\nread_chunk(#revproxy{srvsock=Sock, type=Type}) ->\n    try\n        yaws:setopts(Sock, [binary, {packet, line}], Type),\n        %% Ignore chunk extentions\n        {Len, _Exts} = yaws:get_chunk_header(Sock, Type),\n        yaws:setopts(Sock, [binary, {packet, raw}], Type),\n        if\n            Len == 0 ->\n                %% Ignore chunk trailer\n                yaws:get_chunk_trailer(Sock, Type),\n                {ok, <<>>};\n            true ->\n                B = yaws:get_chunk(Sock, Len, 0, Type),\n                ok = yaws:eat_crnl(Sock, Type),\n                {ok, iolist_to_binary(B)}\n        end\n    catch\n        _:Reason ->\n            {error, Reason}\n    end.\n\n\nclose(#revproxy{srvsock=Sock, type=ssl}) ->\n    ssl:close(Sock);\nclose(#revproxy{srvsock=Sock, type=nossl}) ->\n    gen_tcp:close(Sock).\n\n\nget_connection_status(Version, ReqHdrs, RespHdrs) ->\n    CliConn = case Version of\n                  {0,9} ->\n                      \"close\";\n                  {1, 0} ->\n                      case ReqHdrs#headers.connection of\n                          undefined -> \"close\";\n                          C1        -> yaws:to_lower(C1)\n                      end;\n                  {1, 1} ->\n                      case ReqHdrs#headers.connection of\n                          undefined -> \"keep-alive\";\n                          C1        -> yaws:to_lower(C1)\n                      end\n              end,\n    ?Debug(\"Client Connection header: ~p~n\", [CliConn]),\n\n    %% below, ignore dialyzer warning:\n    %% \"The pattern 'true' can never match the type 'false'\"\n    SrvConn = case ?proxy_keepalive of\n                  true ->\n                      case RespHdrs#headers.connection of\n                          undefined -> CliConn;\n                          C2        -> yaws:to_lower(C2)\n                      end;\n                  false ->\n                      \"close\"\n              end,\n    ?Debug(\"Server Connection header: ~p~n\", [SrvConn]),\n    {CliConn, SrvConn}.\n\n%%==========================================================================\nrewrite_request(RPState, Req) ->\n    ?Debug(\"Request path to rewrite:  ~p~n\", [Req#http_request.path]),\n    {abs_path, Path} = Req#http_request.path,\n    NewPath = strip_prefix(Path, RPState#revproxy.prefix),\n    ?Debug(\"New Request path: ~p~n\", [NewPath]),\n    Req#http_request{path = {abs_path, NewPath}}.\n\n\nrewrite_client_headers(RPState, Hdrs) ->\n    ?Debug(\"Host header to rewrite:  ~p~n\", [Hdrs#headers.host]),\n    Host = case Hdrs#headers.host of\n               undefined ->\n                   undefined;\n               _ ->\n                   ProxyUrl = RPState#revproxy.url,\n                   [ProxyUrl#url.host,\n                    case ProxyUrl#url.port of\n                        undefined -> [];\n                        P         -> [$:|integer_to_list(P)]\n                    end]\n           end,\n    ?Debug(\"New Host header: ~p~n\", [Host]),\n    Hdrs#headers{host = Host}.\n\n\nrewrite_server_headers(RPState) ->\n    Hdrs = RPState#revproxy.headers,\n    ?Debug(\"Location header to rewrite:  ~p~n\", [Hdrs#headers.location]),\n    Loc = case Hdrs#headers.location of\n              undefined ->\n                  undefined;\n              L ->\n                  ?Debug(\"parse_url(~p)~n\", [L]),\n                  LocUrl   = (catch yaws_api:parse_url(L)),\n                  ProxyUrl = RPState#revproxy.url,\n                  if\n                      LocUrl#url.scheme == ProxyUrl#url.scheme andalso\n                      LocUrl#url.host   == ProxyUrl#url.host   andalso\n                      LocUrl#url.port   == ProxyUrl#url.port ->\n                          rewrite_loc_url(RPState, LocUrl);\n\n                      element(1, L) == 'EXIT' ->\n                          rewrite_loc_rel(RPState, L);\n\n                      true ->\n                          L\n                  end\n          end,\n    ?Debug(\"New Location header: ~p~n\", [Loc]),\n\n    %% FIXME: And we also should do cookies here ...\n\n    Hdrs#headers{location = Loc, connection = RPState#revproxy.cliconn_status}.\n\n\n%% Rewrite a properly formatted location redir\nrewrite_loc_url(RPState, LocUrl) ->\n    SC=get(sc),\n    Scheme    = yaws:redirect_scheme(SC),\n    RedirHost = yaws:redirect_host(SC, RPState#revproxy.r_host),\n    [Scheme, RedirHost, slash_append(RPState#revproxy.prefix, LocUrl#url.path)].\n\n\n%% This is the case for broken webservers that reply with\n%% Location: /path\n%% or even worse, Location: path\nrewrite_loc_rel(RPState, Loc) ->\n    SC=get(sc),\n    Scheme    = yaws:redirect_scheme(SC),\n    RedirHost = yaws:redirect_host(SC, RPState#revproxy.r_host),\n    [Scheme, RedirHost, Loc].\n\n\n\nstrip_prefix(\"\", \"\") ->\n    \"/\";\nstrip_prefix(P, \"\") ->\n    P;\nstrip_prefix(P, \"/\") ->\n    P;\nstrip_prefix([H|T1], [H|T2]) ->\n    strip_prefix(T1, T2).\n\n\nslash_append(\"/\", [$/|T]) ->\n    [$/|T];\nslash_append(\"/\", T) ->\n    [$/|T];\nslash_append([], [$/|T]) ->\n    [$/|T];\nslash_append([], T) ->\n    [$/|T];\nslash_append([H|T], X) ->\n    [H | slash_append(T, X)].\n"
  },
  {
    "path": "contrib/yaws/src/yaws_rpc.erl",
    "content": "%% -*- coding: latin-1 -*-\n%% Copyright (C) 2003 Joakim Greben <jocke@gleipnir.com>.\n%% All rights reserved.\n%%\n%% Copyright (C) 2006 Gaspar Chilingarov <nm@web.am>\n%%                      Gurgen Tumanyan <barbarian@armkb.com>\n%% All rights reserved.\n%%\n%%\n%% Redistribution and use in source and binary forms, with or without\n%% modification, are permitted provided that the following conditions\n%% are met:\n%%\n%% 1. Redistributions of source code must retain the above copyright\n%%    notice, this list of conditions and the following disclaimer.\n%% 2. Redistributions in binary form must reproduce the above\n%%    copyright notice, this list of conditions and the following\n%%    disclaimer in the documentation and/or other materials provided\n%%    with the distribution.\n%%\n%% THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS\n%% OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED\n%% WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n%% ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY\n%% DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n%% DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE\n%% GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS\n%% INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,\n%% WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING\n%% NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS\n%% SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n\n%% NOTE: This module was originally called yaws_jsonrpc.\n%% It was hacked to transparently supports haXe remoting as well,\n%% hence its name was changed to the more generic 'yaws_rpc'.\n%%\n%% modified by Yariv Sadan (yarivvv@gmail.com)\n\n-module(yaws_rpc).\n-author(\"Gaspar Chilingarov <nm@web.am>, Gurgen Tumanyan <barbarian@armkb.com>\").\n-modified_by(\"Yariv Sadan <yarivvv@gmail.com>\").\n-modified_by(\"Steve Vinoski <vinoski@ieee.org>\").\n\n-export([handler/2]).\n-export([handler_session/2, handler_session/3]).\n\n%%-define(debug, 1).\n-include(\"yaws_debug.hrl\").\n-include(\"../include/yaws_api.hrl\").\n\n%%% ######################################################################\n%%% public interface\n%%%\n\n%%%\n%%% use rpc handler which can automagically start sessions if we need\n%%%\nhandler_session(Args, Handler) ->\n    handler_session(Args, Handler, 'SID').\n\n%%%\n%%% allow overriding session Cookie name\n%%%\nhandler_session(Args, Handler, SID_NAME) when is_atom(SID_NAME) ->\n    handler_session(Args, Handler, atom_to_list(SID_NAME));\n\nhandler_session(Args, Handler, SID_NAME) ->\n    handler(Args, Handler, {session, SID_NAME}).    % go to generic handler\n\n%%%\n%%% xmlrpc:handler compatible call\n%%% no session support will be available\nhandler(Args, Handler) ->\n    handler(Args, Handler, simple).\n\n\n%%% ######################################################################\n%%% private functions\n%%%\n\n%%% we should be called from yaws page or module\nhandler(Args, Handler, Type) when is_record(Args, arg) ->\n    case parse_request(Args) of\n        ok ->\n            handle_payload(Args, Handler, Type);\n        {status, StatusCode} ->        % cannot parse request\n            send(Args, StatusCode)\n    end.\n\n-define(ERROR_LOG(Reason),\n        error_logger:error_report({?MODULE, ?LINE, Reason})).\n\n-define(LOG(Reason), ?ERROR_LOG(Reason)).\n\n%%%\n%%% check that request come in reasonable protocol version and reasonable method\n%%%\nparse_request(Args) ->\n    Req = Args#arg.req,\n    case {Req#http_request.method, Req#http_request.version} of\n        {'POST', {1,0}} ->\n            ?Debug(\"HTTP Version 1.0~n\", []),\n            ok;\n        {'POST', {1,1}} ->\n            ?Debug(\"HTTP Version 1.1~n\", []),\n            ok;\n        {'POST', _HTTPVersion} ->\n            {status, 505};\n        {_Method, {1,1}} ->\n            {status, 501};\n        _ ->\n            {status, 400}\n    end.\n\nhandle_payload(Args, Handler, Type) ->\n    RpcType = recognize_rpc_type(Args),\n    %% haXe parameters are URL encoded\n    PL = unicode:characters_to_list(Args#arg.clidata),\n    {Payload,DecodedStr} =\n        case RpcType of\n            T when T==haxe; T==json ->\n                ?Debug(\"rpc ~p call ~p~n\", [T, PL]),\n                {PL, yaws_api:url_decode(PL)};\n            soap_dime ->\n                [{_,_,_,Req}|As] = yaws_dime:decode(Args#arg.clidata),\n                {Args#arg.clidata, {binary_to_list(Req), As}};\n            _ ->\n                ?Debug(\"rpc plaintext call ~p~n\", [PL]),\n                {PL, PL}\n        end,\n    case decode_handler_payload(RpcType, DecodedStr) of\n        Batch when RpcType == json, is_list(Batch) ->\n            BatchRes =\n                lists:foldl(\n                  fun(Req, Acc) ->\n                          Result = check_decoded_payload(Args, Handler,\n                                                         Req, Payload,\n                                                         Type, json),\n                          case Result of\n                              empty ->\n                                  Acc;\n                              {result, _Code, Send} ->\n                                  [Send|Acc];\n                              {send, S} ->\n                                  %% TODO: it would be better if\n                                  %% Result was never of the\n                                  %% {send, ...} variety because\n                                  %% it requires us to take the\n                                  %% content out via searching.\n                                  case lists:keysearch(content,1,S) of\n                                      {value, {content, _, Send}} ->\n                                          [Send|Acc];\n                                      _ ->\n                                          Acc\n                                  end\n                          end\n                  end, [], Batch),\n            case BatchRes of\n                [] ->\n                    %% all notifications, no replies\n                    send(Args, 200, json);\n                _ ->\n                    send(Args, 200,\n                         \"[\"++yaws:join_sep(lists:reverse(BatchRes),\",\")++\"]\",\n                         [], json)\n            end;\n        NonBatch ->\n            Result = check_decoded_payload(Args, Handler, NonBatch,\n                                           Payload, Type, RpcType),\n            case Result of\n                {send, Send} ->\n                    Send;\n                empty ->\n                    send(Args, 200, RpcType);\n                {result, Code, Send} ->\n                    send(Args, Code, Send, [], RpcType)\n            end\n    end.\n\ncheck_decoded_payload(Args, Handler, DecodedResult, Payload, Type, RpcType) ->\n    case DecodedResult of\n        {ok, DecodedPayload, ID} ->\n            ?Debug(\"client2erl decoded call ~p~n\", [DecodedPayload]),\n            eval_payload(Args, Handler, DecodedPayload, Type, ID, RpcType);\n        {error, Reason} ->\n            ?ERROR_LOG({html, client2erl, Payload, Reason}),\n            case RpcType of\n                json ->\n                    case Reason of\n                        {ErrCode, _ErrString} ->\n                            {result, 200, json_error(ErrCode)};\n                        ErrCode ->\n                            {result, 200, json_error(ErrCode)}\n                    end;\n                _ ->\n                    {send, send(Args, 400, RpcType)}\n            end\n    end.\n\n%%% Identify the RPC type. We first try to recognize haXe by the\n%%% \"X-Haxe-Remoting\" HTTP header, then the \"SOAPAction\" header,\n%%% and if those are absent we assume the request is JSON.\nrecognize_rpc_type(Args) ->\n    case (Args#arg.headers)#headers.content_type of\n        \"application/dime\" -> soap_dime;\n        _ ->\n            OtherHeaders = ((Args#arg.headers)#headers.other),\n            recognize_rpc_hdr([{X,Y,yaws:to_lower(Z),Q,W} ||\n                                  {X,Y,Z,Q,W} <- OtherHeaders])\n    end.\n\nrecognize_rpc_hdr([{_,_,\"x-haxe-remoting\",_,_}|_]) -> haxe;\nrecognize_rpc_hdr([{_,_,\"soapaction\",_,_}|_])      -> soap;\nrecognize_rpc_hdr([_|T])                           -> recognize_rpc_hdr(T);\nrecognize_rpc_hdr([])                              -> json.\n\n%%%\n%%% call handler/3 and provide session support\neval_payload(Args, {M, F}, Payload, {session, CookieName}, ID, RpcType) ->\n    {SessionValue, Cookie} =\n        case yaws_api:find_cookie_val(CookieName,\n                                      (Args#arg.headers)#headers.cookie) of\n            [] ->      %% have no session started, just call handler\n                {undefined, undefined};\n            Cookie2 -> %% get old session data\n                case yaws_api:cookieval_to_opaque(Cookie2) of\n                    {ok, OP} ->\n                        {OP, Cookie2};\n                    {error, _ErrMsg} -> % cannot get corresponding session\n                        {undefined, undefined}\n                end\n        end,\n    CbackFun = callback_fun(M, F, Args, Payload, SessionValue, RpcType),\n    case catch CbackFun() of\n        {'EXIT', {function_clause, _}} when RpcType == json ->\n            case ID of\n                undefined ->\n                    %% empty HTTP reply for notification\n                    empty;\n                _ ->\n                    {result, 200, json_error(-32601, ID)}\n            end;\n        {'EXIT', Reason} ->\n            ?ERROR_LOG({M, F, {'EXIT', Reason}}),\n            {send, send(Args, 500, RpcType)};\n        {error, Reason} ->\n            ?ERROR_LOG({M, F, Reason}),\n            {send, send(Args, 500, RpcType)};\n        {error, Reason, Rc} ->\n            ?ERROR_LOG({M, F, Reason}),\n            {send, send(Args, Rc, Reason, [], RpcType)};\n        {false, ResponsePayload} ->\n            %% do not have updates in session data\n            {send, encode_send(Args, 200, ResponsePayload, [], ID, RpcType)};\n        {false, ResponsePayload, RespCode} ->\n            %% do not have updates in session data\n            {send, encode_send(Args,RespCode,ResponsePayload,[],ID,RpcType)};\n        false ->   % soap or json-rpc notify\n            empty;\n        {true, _NewTimeout, NewSessionValue, ResponsePayload} ->\n            %% be compatible with xmlrpc module\n            CO = handle_cookie(Cookie, CookieName, SessionValue,\n                               NewSessionValue, M, F),\n            {send, encode_send(Args, 200, ResponsePayload, CO, ID, RpcType)};\n        {true, _NewTimeout, NewSessionValue, ResponsePayload, RespCode} ->\n            %% be compatible with xmlrpc module\n            CO = handle_cookie(Cookie, CookieName, SessionValue,\n                               NewSessionValue, M, F),\n            {send, encode_send(Args, RespCode,\n                               ResponsePayload, CO, ID, RpcType)}\n    end;\n\n%%%\n%%% call handler/2 without session support\n%%%\neval_payload(Args, {M, F}, Payload, simple, ID, RpcType) ->\n    case catch M:F(Args#arg.state, Payload) of\n        {'EXIT', Reason} ->\n            ?ERROR_LOG({M, F, {'EXIT', Reason}}),\n            {send, send(Args, 500)};\n        {error, Reason} ->\n            ?ERROR_LOG({M, F, Reason}),\n            {send, send(Args, 500)};\n        {false, ResponsePayload} ->\n            {send, encode_send(Args, 200, ResponsePayload, [], ID, RpcType)};\n        false -> % Soap notify\n            {send, send(Args, 200, RpcType)};\n        {true, _NewTimeout, _NewState, ResponsePayload} ->\n            {send, encode_send(Args, 200, ResponsePayload, [], ID, RpcType)}\n    end.\n\nhandle_cookie(Cookie, CookieName, SessionValue, NewSessionValue, M, F) ->\n    case NewSessionValue of\n        undefined when Cookie == undefined -> []; % nothing to do\n        undefined -> % rpc handler requested session delete\n            yaws_api:delete_cookie_session(Cookie), [];\n        %% XXX: may be return set-cookie with empty val?\n        _ ->\n            %% any other value will stored in session\n            case SessionValue of\n                undefined ->\n                    %% got session data and should start new session now\n                    Cookie1 = yaws_api:new_cookie_session(NewSessionValue),\n                    case get_expire(M, F) of\n                        false ->\n                            yaws_api:setcookie(CookieName, Cookie1, \"/\");\n                        %% return set_cookie header\n                        Expire ->\n                            yaws_api:setcookie(CookieName, Cookie1, \"/\",Expire)\n                            %% return set_cookie header\n                    end;\n                _ ->\n                    yaws_api:replace_cookie_session(Cookie, NewSessionValue),\n                    [] % nothing to add to yaws data\n            end\n    end.\n\n%%% Make it possible for callback module to set Cookie Expire string!\nget_expire(M, F) ->\n    case catch M:F(cookie_expire) of\n        Expire when is_list(Expire) -> Expire;\n        _                        -> false\n    end.\n\ncallback_fun(M, F, Args, Payload, SessionValue, RpcType)\n  when RpcType =:= soap; RpcType =:= soap_dime ->\n    fun() -> yaws_soap_srv:handler(Args, {M,F}, Payload, SessionValue) end;\ncallback_fun(M, F, Args, Payload, SessionValue, _RpcType) ->\n    fun() -> M:F(Args#arg.state, Payload, SessionValue) end.\n\n%%% XXX compatibility with XMLRPC handlers\n%%% XXX - potential bug here?\nencode_send(Args, StatusCode, [Payload], AddOn, ID, RpcType) ->\n    encode_send(Args, StatusCode, Payload, AddOn, ID, RpcType);\n\nencode_send(Args, StatusCode, Payload, AddOn, ID, RpcType) ->\n    ?Debug(\"rpc response ~p ~n\", [Payload]),\n    case encode_handler_payload(Payload, ID, RpcType) of\n        {ok, EncodedPayload, NewRpcType} ->\n            ?Debug(\"rpc encoded response ~p ~n\", [EncodedPayload]),\n            send(Args, StatusCode, EncodedPayload, AddOn, NewRpcType);\n        {ok, EncodedPayload} ->\n            ?Debug(\"rpc encoded response ~p ~n\", [EncodedPayload]),\n            send(Args, StatusCode, EncodedPayload, AddOn, RpcType)\n    end.\n\nsend(Args, StatusCode) ->\n    send(Args, StatusCode, json).\n\nsend(Args, StatusCode, RpcType) ->\n    send(Args, StatusCode, \"\", [], RpcType).\n\nsend(Args, StatusCode, Payload, AddOn, RpcType) when not is_list(AddOn) ->\n    send(Args, StatusCode, Payload, [AddOn], RpcType);\nsend(Args, StatusCode, Payload, AddOnData, RpcType) ->\n    [{status, StatusCode},\n     content_hdr(RpcType, Args, Payload),\n     {header, {content_length, lists:flatlength(Payload)}}] ++ AddOnData.\n\ncontent_hdr(json, _Args, Payload) -> {content, \"application/json\", Payload};\ncontent_hdr(soap, Args, Payload) ->\n    CallerContentType = (Args#arg.headers)#headers.content_type,\n    %% drop caller charset info if present, may not\n    %% be appropriate for the response\n    ContentType = hd(string:tokens(CallerContentType, \";\")),\n    {content, ContentType, Payload};\ncontent_hdr(_, _Args, Payload) -> {content, \"text/xml\", Payload}.\n\nencode_handler_payload({Xml,[]}, _ID, soap_dime) ->\n    {ok, Xml, soap};\nencode_handler_payload({Xml,As}, _ID, soap_dime) ->\n    EncodedPayload = yaws_dime:encode(Xml, As),\n    {ok, EncodedPayload};\nencode_handler_payload(Xml, _ID, soap_dime) ->\n    {ok, Xml, soap};\nencode_handler_payload({Xml,[]}, _ID, soap) ->\n    {ok, Xml};\nencode_handler_payload({Xml,As}, _ID, soap) ->\n    EncodedPayload = yaws_dime:encode(Xml, As),\n    {ok, EncodedPayload, soap_dime};\nencode_handler_payload(Xml, _ID, soap) ->\n    {ok, Xml};\nencode_handler_payload({error, [ErlStruct]}, ID, RpcType) ->\n    encode_handler_payload({error, ErlStruct}, ID, RpcType);\nencode_handler_payload({error, ErlStruct}, ID, RpcType) ->\n    StructStr =\n        case RpcType of\n            json -> json2:encode({struct, [{id, ID}, {error, ErlStruct},\n                                           {\"jsonrpc\", \"2.0\"}]});\n            haxe -> [$h, $x, $r | haxe:encode({exception, ErlStruct})]\n        end,\n    {ok, StructStr};\nencode_handler_payload({response, [ErlStruct]}, ID, RpcType) ->\n    encode_handler_payload({response, ErlStruct}, ID, RpcType);\nencode_handler_payload({response, ErlStruct}, ID, RpcType) ->\n    StructStr =\n        case RpcType of\n            json -> json2:encode({struct, [{result, ErlStruct}, {id, ID},\n                                           {\"jsonrpc\", \"2.0\"}]});\n            haxe -> [$h, $x, $r | haxe:encode(ErlStruct)]\n        end,\n    {ok, StructStr}.\n\ndecode_handler_payload(json, JSonStr) ->\n    try\n        {ok, Obj} = json2:decode_string(JSonStr),\n        decode_handler_payload_json(Obj)\n    catch\n        error:Err ->\n            ?ERROR_LOG({json_decode, JSonStr, Err}),\n            {error, {-32700, Err}}\n    end;\ndecode_handler_payload(haxe, [$_, $_, $x, $= | HaxeStr]) ->\n    try\n        {done, {ok, {array, [MethodName | _]}}, Cont} = haxe:decode(HaxeStr),\n        {done, {ok, Args}, _Cont2} = haxe:decode_next(Cont),\n\n        %% ID is undefined because haXe remoting doesn't automagically handle\n        %% sessions.\n        {ok, {call, list_to_atom(MethodName), Args}, undefined}\n    catch\n        error:Err -> {error, Err}\n    end;\ndecode_handler_payload(haxe, _HaxeStr) ->\n    {error, missing_haxe_prefix};\n\ndecode_handler_payload(soap_dime, Payload) ->\n    {ok, Payload, undefined};\ndecode_handler_payload(soap, Payload) ->\n    {ok, Payload, undefined}.\n\ndecode_handler_payload_json({struct, _}=Obj) ->\n    case jsonrpc:s(Obj, method) of\n        undefined ->\n            {error, -32600};\n        Method0 when is_list(Method0) ->\n            Method = case jsonrpc:s(Obj, jsonrpc) of\n                         \"2.0\" ->\n                             try\n                                 list_to_existing_atom(Method0)\n                             catch\n                                 error:badarg ->\n                                     Method0\n                             end;\n                         undefined ->\n                             list_to_atom(Method0)\n                     end,\n            Args = jsonrpc:s(Obj, params),\n            ArgsOk = case Args of\n                         {struct, _} -> true;\n                         {array, _} -> true;\n                         undefined -> true;\n                         _ -> false\n                     end,\n            case ArgsOk of\n                true ->\n                    ID = jsonrpc:s(Obj, id),\n                    CallOrNotify = case ID of\n                                       undefined ->\n                                           notification;\n                                       _ ->\n                                           call\n                                   end,\n                    {ok, {CallOrNotify, Method, Args}, ID};\n                false ->\n                    {error, -32602}\n            end;\n        _ ->\n            {error, -32600}\n    end;\ndecode_handler_payload_json({array, []}) ->\n    {error, -32600};\ndecode_handler_payload_json({array, Batch}) ->\n    [decode_handler_payload_json(Obj) || Obj <- Batch];\ndecode_handler_payload_json(_) ->\n    {error, -32600}.\n\njson_error(ErrCode) ->\n    json_error(ErrCode, null).\njson_error(ErrCode, Id) ->\n    Err = {struct, [{\"jsonrpc\", \"2.0\"},\n                    {\"id\", Id},\n                    {\"error\", {struct,\n                               [{\"code\", ErrCode},\n                                {\"message\", json_error_message(ErrCode)}]}}]},\n    json2:encode(Err).\n\njson_error_message(-32700) -> \"parse error\";\njson_error_message(-32600) -> \"invalid request\";\njson_error_message(-32601) -> \"method not found\";\njson_error_message(-32602) -> \"invalid params\";\njson_error_message(-32603) -> \"internal error\";\njson_error_message(Code) when Code >= -32099, Code =< -32000 -> \"server error\";\njson_error_message(_) -> \"json error\".\n"
  },
  {
    "path": "contrib/yaws/src/yaws_rss.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_rss.erl\n%%% Created : 15 Dec 2004 by Torbjorn Tornkvist <tobbe@tornkvist.org>\n%%%\n%%% @doc A Yaws RSS feed interface.\n%%%\n%%% @author  Torbjorn Tornkvist <tobbe@tornkvist.org>\n%%% @end\n%%%\n%%% $Id$\n%%%----------------------------------------------------------------------\n-module(yaws_rss).\n\n-behaviour(gen_server).\n\n%% External exports\n-export([start/0, start_link/0, open/1, open/2, close/0, close/2,\n         insert/5, insert/6, insert/7, retrieve/2]).\n\n-export([t_setup/0, t_exp/0, t_xopen/0]).\n\n%% gen_server callbacks\n-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2,\n         code_change/3]).\n\n-record(s, {\n          open_apps = [],    % activated applications\n          expire = false,    % false | days\n          rm_exp = false,    % remove expired items\n          max=infinite,      % maximum number of elements in DB\n          days=7,            % maximum number of days in DB\n          counter}).         % item counter\n\n-define(SERVER, ?MODULE).\n-define(DB, ?MODULE).\n-define(DB_FNAME, \"yaws_rss.dets\").\n-define(ITEM(App, Tag, Counter, Item), {{App, Tag, Counter}, Item}).\n\n%%%----------------------------------------------------------------------\n%%% API\n%%%----------------------------------------------------------------------\nstart_link() ->\n    gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).\n\nstart() ->\n    gen_server:start({local, ?SERVER}, ?MODULE, [], []).\n\n%%%\n%%% @spec open(App::atom()) ->\n%%%         {ok, DB::db()} | {error, string()}\n%%%\n%%% @type db(). An opaque handle leading to an RSS database.\n%%%\n%%% @doc See {@link open/2}\n%%% @end\nopen(App) ->\n    open(App, []).\n\n%%%\n%%% @spec open(App::atom(), Opts::list()) ->\n%%%         {ok, DB::db()} | {error, string()}\n%%%\n%%% @doc Open a RSS database.\n%%%      Per default <em>dets</em> is used as database,\n%%%      but by using the <em>db_mod</em> option it is\n%%%      possible to use your own database.<br/>\n%%%      These are the options:\n%%%      <p><dl>\n%%%\n%%%      <dt>{db_mod, Module}</dt>\n%%%      <dd>If specified, the following functions will be\n%%%      called:<ul>\n%%%      <li>Module:open(Opts)</li>\n%%%      <li>Module:insert(App,Tag,Title,Link,Desc,Creator,GregSec)</li>\n%%%      <li>Module:retrieve(App,Tag) -&gt; {Title, Link, Desc, Creator, GregSecs}</li>\n%%%      <li>Module:close(DbName)</li></ul>\n%%%      This means that the default DB won't be used, and\n%%%      no expiration handling will be done. Only the producing of\n%%%      XML will thus be done. Also, the whole <em>Opts</em> will be\n%%%      passed un-interpreted to the other DB module.</dd>\n%%%\n%%%      <dt>{db_dir, Dir}</dt>\n%%%      <dd>Specifies the directory where the database will be created.\n%%%      Default is: /tmp</dd>\n%%%\n%%%      <dt>{expire, Expire}</dt>\n%%%      <dd>Specifies what method to use to expire items. Possible values\n%%%      are: <em>false</em>, <em>days</em>, meaning\n%%%      never expire, expire after a number of days.\n%%%      Default is to never expire items.</dd>\n%%%\n%%%      <dt>{days, Number}</dt>\n%%%      <dd>Specifies the number of days befor an item is expired.\n%%%      Default is 7 days.</dd>\n%%%\n%%%      <dt>{rm_exp, Bool}</dt>\n%%%      <dd>Specifies if expired items should be removed from\n%%%      the database. Default is to not remove any items.</dd>\n%%%\n%%%      <dt>{max, Number}</dt>\n%%%      <dd>Specifies the maximum number of items that should\n%%%      be stored in the database. The default in <em>infinite</em></dd>\n%%%      </dl></p>\n%%%      <p>If no database exist, a new one will be created.\n%%%      The returned database handle is to be used with {@link close/1}.</p>\n%%% @end\n%%%\nopen(App, Opts) ->\n    %% This is called during read of yaws.conf during startup, so make sure this\n    %% server is up and running before invoking it\n    ok = wait_for_server(?SERVER),\n    gen_server:call(?SERVER, {open, App, Opts}, infinity).\n\n%%%\n%%% @spec close() -> ok | {error, string()}\n%%%\n%%% @doc Close the RSS database.\n%%% @end\nclose() ->\n    gen_server:call(?SERVER, {close, ?DB}, infinity).\n\n%%%\n%%% @spec close(DbMod::atom(), DbName::atom()) ->\n%%%          ok | {error, string()}\n%%%\n%%% @doc Close the user provided RSS database.\n%%%      A call to; <em>DbMod:close(DbName)</em> will be made.\n%%% @end\nclose(DBmod, DBname) ->\n    gen_server:call(?SERVER, {close, DBmod, DBname}, infinity).\n\n%%%\n%%% @spec insert(App::atom(), Tag::atom(), Title::string(),\n%%%              Link::string(), Desc::string()) ->\n%%%          ok | {error, string()}\n%%%\n%%% @doc Insert an RSS item into the <em>{App,Tag}</em> RSS feed.\n%%%      An application (App) can maintain several feeds each\n%%%      one refered to with a symbolic name (Tag).\n%%%      <em>Link</em> should be a URL pointing to the item.\n%%%      <p>In case another database backend is used, the\n%%%      <em>Tag</em> has the format: <em>{DbModule, OpaqueTag}</em>\n%%%      where <em>DbModule</em> is the database backend module\n%%%      to be called, and <em>OpaqueTag</em> the Tag that is\n%%%      used in <em>DbModule:insert(Tag, ...)</em></p>\n%%% @end\n%%%\ninsert(App, Tag, Title, Link, Desc) ->\n    insert(App, Tag, Title, Link, Desc, \"\").\n\n%%%\n%%% @spec insert(App::atom(), Tag::atom(), Title::string(),\n%%%              Link::string(), Desc::string(),\n%%%              Creator::string()) ->\n%%%          ok | {error, string()}\n%%%\n%%% @doc Works as {@link insert/5} but takes an extra argument\n%%%      <em>Creator</em> which may contains an identification\n%%%      of who created the item.\n%%% @end\ninsert(App, Tag, Title, Link, Desc, Creator) ->\n    GregSecs = calendar:datetime_to_gregorian_seconds({date(),time()}),\n    insert(App, Tag, Title, Link, Desc, Creator, GregSecs).\n\n%%%\n%%% @spec insert(App::atom(), Tag::atom(), Title::string(),\n%%%              Link::string(), Desc::string(),\n%%%              Creator::string(), GregSecs::integer()) ->\n%%%          ok | {error, string()}\n%%%\n%%% @doc Works as {@link insert/6} but takes an extra argument\n%%%      <em>GregSecs</em> which is the creation time of the item\n%%%      in Gregorian Seconds.\n%%% @end\ninsert(App, Tag, Title, Link, Desc, Creator, GregSecs) ->\n    Args = {App, Tag, Title, Link, Desc, Creator, GregSecs},\n    gen_server:call(?SERVER, {insert, Args}, infinity).\n\n\n%%%\n%%% @spec retrieve(App::atom(), Tag::atom()) ->\n%%%           {ok, RSSContent::iolist()} |{error, string()}\n%%%\n%%% @type ioList().  A deep list of strings and/or binaries.\n%%%\n%%% @doc Retrieve the <em>RSScontent</em> (in XML and all...)\n%%%      to be delivered to a RSS client.\n%%%      <p>In case another database backend is used, the\n%%%      <em>Tag</em> has the format: <em>{DbModule, OpaqueTag}</em>\n%%%      where <em>DbModule</em> is the database backend module\n%%%      to be called, and <em>OpaqueTag</em> the Tag that is\n%%%      used in <em>DbModule:retrieve(Tag)</em> which must return\n%%%      a list of tuples: <em>{Title, Link, Desc, Creator, GregSecs}</em></p>\n%%% @end\nretrieve(App, Tag) ->\n    gen_server:call(?SERVER, {retrieve, App, Tag}, infinity).\n\n%%%----------------------------------------------------------------------\n%%% Callback functions from gen_server\n%%%----------------------------------------------------------------------\n\n%%----------------------------------------------------------------------\n%% Func: init/1\n%% Returns: {ok, State}          |\n%%          {ok, State, Timeout} |\n%%          ignore               |\n%%          {stop, Reason}\n%%----------------------------------------------------------------------\ninit([]) ->\n    {ok, #s{}}.\n\n%%----------------------------------------------------------------------\n%% Func: handle_call/3\n%% Returns: {reply, Reply, State}          |\n%%          {reply, Reply, State, Timeout} |\n%%          {noreply, State}               |\n%%          {noreply, State, Timeout}      |\n%%          {stop, Reason, Reply, State}   | (terminate/2 is called)\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%----------------------------------------------------------------------\nhandle_call({open, App, Opts}, _From, State) ->\n    {NewState, Res} = do_open_dir(State, App, Opts),\n    {reply, Res, NewState};\n%%\nhandle_call({close, DB}, _From, State) ->\n    dets:close(DB),\n    {reply, ok, State};\n%%\nhandle_call({close, DBMod, DBname}, _From, State) ->\n    catch apply(DBMod, close, [DBname]),\n    {reply, ok, State};\n%%\nhandle_call({insert, Args}, _From, State) ->\n    {NewState, Res} = do_insert(State, Args),\n    {reply, Res, NewState};\n%%\nhandle_call({retrieve, App, Tag}, _From, State) ->\n    {NewState, Res} = do_retrieve(State, App, Tag),\n    {reply, Res, NewState}.\n\n%%----------------------------------------------------------------------\n%% Func: handle_cast/2\n%% Returns: {noreply, State}          |\n%%          {noreply, State, Timeout} |\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%----------------------------------------------------------------------\nhandle_cast(_Msg, State) ->\n    {noreply, State}.\n\n%%----------------------------------------------------------------------\n%% Func: handle_info/2\n%% Returns: {noreply, State}          |\n%%          {noreply, State, Timeout} |\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%----------------------------------------------------------------------\nhandle_info(_Info, State) ->\n    {noreply, State}.\n\n%%----------------------------------------------------------------------\n%% Func: terminate/2\n%% Purpose: Shutdown the server\n%% Returns: any (ignored by gen_server)\n%%----------------------------------------------------------------------\nterminate(_Reason, _State) ->\n    ok.\n\n%%----------------------------------------------------------------------\n%% Func: code_change/3\n%% Purpose: Handle upgrade\n%% Returns: new State data\n%%----------------------------------------------------------------------\ncode_change(_OldVsn, Data, _Extra) ->\n    {ok, Data}.\n\n%%%----------------------------------------------------------------------\n%%% Internal functions\n%%%----------------------------------------------------------------------\n\n%%%\n%%% Check what database store that should be used.\n%%% Per default 'dets' is used.\n%%%\ndo_open_dir(State, App, Opts) ->\n    case get_db_mod(Opts, dets) of\n        dets ->\n            File = get_db_file(Opts),\n            Expire = get_expire(Opts, #s.expire),\n            Max = get_max(Opts, #s.max),\n            Days = get_days(Opts, #s.days),\n            RmExp = get_rm_exp(Opts, #s.rm_exp),\n            case dets:is_dets_file(File) of\n                false ->\n                    {State, {error, \"not a proper dets file\"}};\n                _     ->\n                    case catch dets:open_file(?DB, [{file, File}]) of\n                        {ok,DB} = Res   ->\n                            {State#s{\n                               open_apps = u_insert(App, State#s.open_apps),\n                               expire = Expire,\n                               days = Days,\n                               rm_exp = RmExp,\n                               max = Max,\n                               counter = init_counter(DB)},\n                             Res};\n                        {error, _Reason} ->\n                            {State, {error, \"open dets file\"}}\n                    end\n            end;\n        DBmod ->\n            {State, catch apply(DBmod, open, Opts)}\n    end.\n\nget_db_file(Opts) ->\n    Dir = get_db_dir(Opts, \"/tmp\"),\n    Dir ++ \"/\" ++ a2l(?DB) ++ \".dets\".\n\ninit_counter(DB) ->\n    case dets:lookup(DB, counter) of\n        []            -> dets:insert(DB, {counter, 0}), 0;\n        [{counter,N}] -> N\n    end.\n\nset_counter(DB, N) ->\n    dets:insert(DB, {counter, N}).\n\ndo_insert(State, {App, {DbMod,Tag}, Title, Link, Desc, Creator, GregSecs}) ->\n    {State, catch apply(DbMod, insert, [App, Tag,Title,Link,\n                                        Desc,Creator,GregSecs])};\ndo_insert(State, {App, Tag, Title, Link, Desc, Creator, GregSecs}) ->\n    case lists:member(App, State#s.open_apps) of\n        true ->\n            Counter = if (State#s.max > 0) ->\n                              (State#s.counter + 1) rem State#s.max;\n                         true ->\n                              State#s.counter + 1\n                      end,\n            Item = {Title, Link, Desc, Creator, GregSecs},\n            Res = dets:insert(?DB, ?ITEM(App, Tag, Counter, Item)),\n            set_counter(?DB, Counter),\n            {State#s{counter = Counter}, Res};\n        false ->\n            {State, {error, \"no open DB\"}}\n    end.\n\n\ndo_retrieve(State, App, {DbMod,Tag}) ->\n    {State, catch apply(DbMod, retrieve, [App, Tag])};\ndo_retrieve(State, App, Tag) ->\n    case lists:member(App, State#s.open_apps) of\n        true ->\n            F = fun(?ITEM(Xa, Xt, _Counter, Item), Acc)\n                      when Xa == App, Xt == Tag ->\n                        [Item|Acc];\n                   (_, Acc) ->\n                        Acc\n                end,\n            Items = sort_items(expired(State, dets:foldl(F, [], ?DB))),\n            Xml = to_xml(Items),\n            {State, {ok, Xml}};\n        false ->\n            {State, {error, \"no open DB\"}}\n    end.\n\n\n\n-define(ONE_DAY, 86400).  % 24*60*60 seconds\n-define(X(GregSecs), {Title, Link, Desc, Creator, GregSecs}).\n\n%%% Filter away expired items !!\nexpired(State, List) when State#s.expire == days ->\n    Gs = calendar:datetime_to_gregorian_seconds({date(),time()}),\n    Old = Gs - (?ONE_DAY * State#s.days),\n    F = fun(?X(GregSecs), Acc) when GregSecs > Old ->\n                [?X(GregSecs) | Acc];\n           (_, Acc) ->\n                Acc\n        end,\n    lists:foldl(F, [], List);\nexpired(_State, List) ->\n    List.\n\n-undef(X).\n\n\n\n%%%\n%%% Sort on creation date !!\n%%% Item = {Title, Link, Desc, Creator, GregSecs},\n%%%\nsort_items(Is) ->\n    lists:keysort(5,Is).\n\n\nto_xml([{Title, Link, Desc, Creator, GregSecs}|Tail]) ->\n    Date = w3cdtf(GregSecs),\n    [[\"<item>\\n\",\n      \"<title>\", yaws_api:htmlize(Title), \"</title>\\n\",\n      \"<link>\", Link, \"</link>\\n\",\n      \"<guid>\", Link, \"</guid>\\n\",\n      \"<description>\", yaws_api:htmlize(Desc), \"</description>\\n\",\n      \"<dc:creator>\", Creator, \"</dc:creator>\\n\",\n      \"<dc:date>\", Date, \"</dc:date>\\n\",\n      \"</item>\\n\"] |\n     to_xml(Tail)];\nto_xml([]) ->\n    [].\n\n%%%\n%%% Create W3CDTF (http://www.w3.org/TR/NOTE-datetime) formatted date\n%%% w3cdtf(GregSecs) -> \"YYYY-MM-DDThh:mm:ssTZD\"\n%%%\nw3cdtf(GregSecs) ->    Date = calendar:gregorian_seconds_to_datetime(GregSecs),\n                       {{Y, Mo, D},{H, Mi, S}} = Date,\n                       [UDate|_] = calendar:local_time_to_universal_time_dst(\n                                     Date),\n                       {DiffD,{DiffH,DiffMi,_}}=calendar:time_difference(\n                                                  UDate,Date),\n                       w3cdtf_diff(Y, Mo, D, H, Mi, S, DiffD, DiffH, DiffMi).\n\n%%%  w3cdtf's helper function\nw3cdtf_diff(Y, Mo, D, H, Mi, S, _DiffD, DiffH, DiffMi)\n  when DiffH < 12,  DiffH /= 0 ->\n    i2l(Y) ++ \"-\" ++ add_zero(Mo) ++ \"-\" ++ add_zero(D) ++ \"T\" ++\n        add_zero(H) ++ \":\" ++ add_zero(Mi) ++ \":\"  ++\n        add_zero(S) ++ \"+\" ++ add_zero(DiffH) ++ \":\"  ++ add_zero(DiffMi);\n\nw3cdtf_diff(Y, Mo, D, H, Mi, S, DiffD, DiffH, DiffMi)\n  when DiffH > 12,  DiffD == 0 ->\n    i2l(Y) ++ \"-\" ++ add_zero(Mo) ++ \"-\" ++ add_zero(D) ++ \"T\" ++\n        add_zero(H) ++ \":\" ++ add_zero(Mi) ++ \":\"  ++\n        add_zero(S) ++ \"+\" ++ add_zero(DiffH) ++ \":\"  ++\n        add_zero(DiffMi);\n\nw3cdtf_diff(Y, Mo, D, H, Mi, S, DiffD, DiffH, DiffMi)\n  when DiffH > 12,  DiffD /= 0, DiffMi /= 0 ->\n    i2l(Y) ++ \"-\" ++ add_zero(Mo) ++ \"-\" ++ add_zero(D) ++ \"T\" ++\n        add_zero(H) ++ \":\" ++ add_zero(Mi) ++ \":\"  ++\n        add_zero(S) ++ \"-\" ++ add_zero(23-DiffH) ++\n        \":\" ++ add_zero(60-DiffMi);\n\nw3cdtf_diff(Y, Mo, D, H, Mi, S, DiffD, DiffH, DiffMi)\n  when DiffH > 12,  DiffD /= 0, DiffMi == 0 ->\n    i2l(Y) ++ \"-\" ++ add_zero(Mo) ++ \"-\" ++ add_zero(D) ++ \"T\" ++\n        add_zero(H) ++ \":\" ++ add_zero(Mi) ++ \":\"  ++\n        add_zero(S) ++ \"-\" ++ add_zero(24-DiffH) ++\n        \":\" ++ add_zero(DiffMi);\n\nw3cdtf_diff(Y, Mo, D, H, Mi, S, _DiffD, DiffH, _DiffMi) when DiffH == 0 ->\n    i2l(Y) ++ \"-\" ++ add_zero(Mo) ++ \"-\" ++ add_zero(D) ++ \"T\" ++\n        add_zero(H) ++ \":\" ++ add_zero(Mi) ++ \":\"  ++\n        add_zero(S) ++ \"Z\".\n\nadd_zero(I) when is_integer(I) -> add_zero(i2l(I));\nadd_zero([A])               -> [$0,A];\nadd_zero(L) when is_list(L)    -> L.\n\n\n\nget_db_mod(Opts, Def)  -> lkup(db_mod, Opts, Def).\nget_db_dir(Opts, Def)  -> lkup(db_dir, Opts, Def).\nget_expire(Opts, Def)  -> lkup(expire, Opts, Def).\nget_max(Opts, Def)     -> lkup(max, Opts, Def).\nget_days(Opts, Def)    -> lkup(days, Opts, Def).\nget_rm_exp(Opts, Def ) -> lkup(rm_exp, Opts, Def).\n\nlkup(Key, List, Def) ->\n    case lists:keysearch(Key, 1, List) of\n        {value,{_,Value}} -> Value;\n        _                 -> Def\n    end.\n\n\nu_insert(H, [H|T]) -> T;\nu_insert(E, [H|T]) -> [H|u_insert(E,T)];\nu_insert(E, [])    -> [E].\n\n\ni2l(I) when is_integer(I) -> integer_to_list(I).\n\na2l(A) when is_atom(A) -> atom_to_list(A).\n\n\n\n\nt_setup() ->\n    %%open([{db_file, \"yaws_rss.dets\"}, {max,7}]),\n    insert(test,xml,\"Normalizing XML, Part 2\",\n           \"http://www.xml.com/pub/a/2002/12/04/normalizing.html\",\n           \"In this second and final look at applying relational \"\n           \"normalization techniques to W3C XML Schema data modeling, \"\n           \"Will Provost discusses when not to normalize, the scope \"\n           \"of uniqueness and the fourth and fifth normal forms.\"),\n    insert(test,xml,\"The .NET Schema Object Model\",\n           \"http://www.xml.com/pub/a/2002/12/04/som.html\",\n           \"Priya Lakshminarayanan describes in detail the use of \"\n           \"the .NET Schema Object Model for programmatic manipulation \"\n           \"of W3C XML Schemas.\"),\n    insert(test,xml,\"SVG's Past and Promising Future\",\n           \"http://www.xml.com/pub/a/2002/12/04/svg.html\",\n           \"In this month's SVG column, Antoine Quint looks back at \"\n           \"SVG's journey through 2002 and looks forward to 2003.\").\n\n\nt_exp() ->\n    %%open([{db_file, \"yaws_rss.dets\"}, {expire,days}]),\n    insert(test,xml,\"Expired article\",\n           \"http://www.xml.com/pub/a/2002/12/04/normalizing.html\",\n           \"In this second and final look at applying relational \"\n           \"normalization techniques to W3C XML Schema data modeling, \"\n           \"Will Provost discusses when not to normalize, the scope \"\n           \"of uniqueness and the fourth and fifth normal forms.\",\n           \"tobbe\",\n           63269561882).  % 6/12-2004\n\nt_xopen() ->\n    open([{db_file, \"yaws_rss.dets\"},\n          {expire,days},\n          {days, 20}]).\n\nwait_for_server(Server) ->\n    wait_for_server(Server, 20).\n\nwait_for_server(_Server, 0) ->\n    {error, timeout};\nwait_for_server(Server, N) ->\n    case erlang:whereis(Server) of\n        undefined ->\n            receive after 500 -> ok end,\n            wait_for_server(Server, N-1);\n        _ ->\n            ok\n    end.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_runmod_lock.erl",
    "content": "%%----------------------------------------------------------------------\n%%% File    : yaws_runmod_lock.erl\n%%% Created : 11 Nov 2012 by tjeerd <tjeerd@yolt.nl>\n%%% Purpose : Generic locking server\n%%%           - lock a resource identified by its path\n%%%           - this path uses a forward slash (\"/\") as separator\n%%%           - use unique path for the resource (i.e. filesystem path)\n%%%           - unique: server wide, locking functionality can be shared\n%%%           - always use an absolute path starting with a slash\n%%%           - lock scope can be exclusive (default) or shared\n%%%           - depth can be 0 or infinity (default)\n%%%---------------------------------------------------------------------\n\n-module(yaws_runmod_lock).\n\n-define(DEBUG(X), io:format(X)).\n-define(DEBUG(X,Y), io:format(X,Y)).\n\n\n-include(\"../include/yaws_lock.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"../include/yaws.hrl\").\n-include(\"yaws_debug.hrl\").\n-include_lib(\"kernel/include/file.hrl\").\n\n-behaviour(gen_server).\n\n%% start/stop manually\n-export([start/0,stop/0]).\n\n%% gen_server callbacks\n-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2,\n         code_change/3]).\n\n%% API - main use\n%%\n%% lock(Path,Lock) -> {ok,Id}|{error,Reason}\n%% unlock(Path,Id) -> ok|{error,Reason}\n%% locked(Path) -> true|false\n%% check(Path,Id) -> ok|{error,Reason}\n%% discover(Path) -> [Lock]\n%% started() -> true|false\n%%\n-export([lock/2, unlock/2, locked/1, check/2, discover/1, clear/0, started/0]).\n\n%% API - console debugging\n%%\n%% report() -> Report\n%% dump() -> Dump\n%% clear_manual() -> ok\n%%\n-export([report/0, dump/0, cleanup_manual/0]).\n\nstart() -> gen_server:start_link({local,?MODULE},?MODULE,[],[]).\nstop() ->\n    gen_server:call(?MODULE,stop).\n\nlock(Path,Lock) ->\n    gen_server:call(?MODULE,{lock,Path,Lock}).\nunlock(Path,Id) ->\n    gen_server:call(?MODULE,{unlock,Path,Id}).\nlocked(Path) ->\n    gen_server:call(?MODULE,{locked,Path}).\ncheck(Path,Id) ->\n    gen_server:call(?MODULE,{check,Path,Id}).\ndiscover(Path) ->\n    gen_server:call(?MODULE,{discover,Path}).\nreport() ->\n    gen_server:call(?MODULE,report).\ndump() ->\n    gen_server:call(?MODULE,dump).\nclear() ->\n    gen_server:call(?MODULE,clear).\ncleanup_manual() ->\n    erlang:send(?MODULE,cleanup).\nstarted() ->\n    case erlang:whereis(?MODULE) of\n        undefined -> false;\n        _ -> true\n    end.\n\n%% init/1\ninit([]) ->\n    % Table is a tree consisting of {Name, Locks, Children} tuples\n    error_logger:info_msg(\"Initializing resource locking server ...~n\"),\n    Table = [],\n    erlang:send_after(?CLEANUP_INTERVAL*1000, self(), cleanup),\n    {ok, Table}.\n\n%% handle_call/3\nhandle_call({lock,Path,Lock}, _From, Table) ->\n    try\n        T0 = yaws:get_time_tuple(),\n        Id = case Lock#lock.id of\n                 undefined -> locktoken();\n                 _ -> Lock#lock.id\n             end,\n        %?DEBUG(\"create lock ~p for ~p~n\",[Id,Path]),\n        Lock1 = Lock#lock{path=Path,id=Id,timestamp=T0},\n        Path1 = filename:split([\"/\",Path]),\n        Table1 = do_lock(Path1,Lock1,Table),\n        {reply, {ok,Id}, Table1}\n    catch\n        Status -> {reply, {error, Status}, Table};\n        _Error:Reason ->\n            error_logger:error_msg(\"Unexpected error: ~p~n~p~n\",\n                                   [Reason,erlang:get_stacktrace()]),\n            {reply, {error, Reason}, Table}\n    end;\nhandle_call({unlock,Path,Id}, _From, Table) ->\n    % even if the lock is not found, its removal is succesfull\n    Path1 = filename:split([\"/\",Path]),\n    {Status,Table1} = do_unlock(Path1,Id,Table),\n    {reply, Status, Table1};\nhandle_call({locked,Path}, _From, Table) ->\n    L = filename:split([\"/\",Path]),\n    Lock = do_locked(L,Table),\n    {reply, Lock, Table};\nhandle_call({check,Path,Id}, _From, Table) ->\n    L = filename:split([\"/\",Path]),\n    Lock = do_check(L,Id,Table),\n    {reply, Lock, Table};\nhandle_call({discover,Path}, _From, Table) ->\n    L = filename:split([\"/\",Path]),\n    Lock = do_discover(L,Table),\n    {reply, Lock, Table};\nhandle_call(report, _From, Table) ->\n    do_report(Table),\n    {reply, ok, Table};\nhandle_call(dump, _From, Table) ->\n    {reply, Table, Table};\nhandle_call(stop, _From, Table) ->\n    {stop, normal, stopped, Table};\nhandle_call(clear, _From, _Table) ->\n    {reply, ok, []}.\n\n%% handle_cast/2\nhandle_cast(_Msg, State) ->\n    {noreply, State}.\n\n%% handle_info/2\nhandle_info(cleanup, Table) ->\n    erlang:send_after(?CLEANUP_INTERVAL*1000, self(), cleanup),\n    Table1 = do_cleanup(Table),\n    {noreply, Table1}.\n\n%% terminate/2\nterminate(_Reason, _State) ->\n    ok.\n\n%% code_change/3\ncode_change(_OldVsn, State, _Extra) ->\n    {ok, State}.\n\n%%----------------------------------------------------------------------\n%% do_lock(Lock,Path,Table) -> Table\n%%\ndo_lock([H],Lock,Table) ->\n    case lists:keysearch(H,1,Table) of\n        {value,{H,Locks,Children}} ->\n            case do_lock_check(Locks,Lock#lock.id) of\n                this ->\n                    % refresh lock only when same resource\n                    Locks1 = do_lock_refresh(Locks,Lock),\n                    lists:keyreplace(H,1,Table,{H,Locks1,Children});\n                {shared,_} when Lock#lock.scope == shared ->\n                    lists:keyreplace(H,1,Table,{H,[Lock|Locks],Children});\n                unlocked ->\n                    lists:keyreplace(H,1,Table,{H,[Lock],Children});\n                _ ->\n                    throw(locked)\n            end;\n        false ->\n            lists:keystore(H,1,Table,{H,[Lock],[]})\n    end;\ndo_lock([H|T],Lock,Table) ->\n    case lists:keysearch(H,1,Table) of\n        {value,{H,Locks,Children}} -> %%%% hier gebleven\n            case do_lock_check(Locks,Lock#lock.id) of\n                {_,infinity} when Lock#lock.scope == exclusive ->\n                    throw(locked);\n                _ ->\n                    lists:keyreplace(H,1,Table,{H,Locks,\n                                                do_lock(T,Lock,Children)})\n            end;\n        false ->\n            lists:keystore(H,1,Table,{H,[],do_lock(T,Lock,[])})\n    end.\n\n%% do_lock_check/2 returns\n%% - unlocked when no lock found or\n%% - {scope,depth} when locked\n%% - this when Id matches\ndo_lock_check(Locks,Id) ->\n    do_lock_check(Locks,Id,unlocked).\ndo_lock_check([],_Id,Result) ->\n    Result;\ndo_lock_check([H|_T],Id,_Result) when H#lock.id == Id ->\n    this;\ndo_lock_check([H|T],Id,_Result) ->\n    case H#lock.depth of\n        infinity -> {H#lock.scope,infinity};\n        0 -> do_lock_check(T,Id,{H#lock.scope,0})\n    end.\n\ndo_lock_refresh(Locks,Lock) ->\n    do_lock_refresh(Locks,Lock,[]).\ndo_lock_refresh([],_Lock,Result) ->\n    Result;\ndo_lock_refresh([H|T],Lock,Result) when H#lock.id == Lock#lock.id ->\n    do_lock_refresh(T,Lock,[Lock|Result]);\ndo_lock_refresh([H|T],Lock,Result) ->\n    do_lock_refresh(T,Lock,[H|Result]).\n\n%%----------------------------------------------------------------------\n%% do_unlock(Path,Id,Table) -> Table\n%%\ndo_unlock([H],Id,Table) ->\n    case lists:keysearch(H,1,Table) of\n        {value,{H,Locks,Children}} ->\n            {Status,Locks1} = do_unlock_id(Locks,Id),\n            case {Locks1,Children} of\n                {[],[]} ->\n                    {_,_,Return} = lists:keytake(H,1,Table),\n                    {Status,Return};\n                _ ->\n                    Result = lists:keyreplace(H,1,Table,{H,Locks1,Children}),\n                    {Status,Result}\n            end;\n        false ->\n            {not_found,Table}\n    end;\ndo_unlock([H|T],Id,Table) ->\n    case lists:keysearch(H,1,Table) of\n        {value,{H,Locks,Children}} ->\n            {Status,Children1} = do_unlock(T,Id,Children),\n            case {Locks,Children1} of\n                {[],[]} ->\n                    {_,_,Return} = lists:keytake(H,1,Table),\n                    {Status,Return};\n                _ ->\n                    Result = lists:keyreplace(H,1,Table,{H,Locks,Children1}),\n                    {Status,Result}\n            end;\n        false ->\n            {not_found,Table}\n    end.\n\ndo_unlock_id([],_Id) ->\n    {not_found,[]};\ndo_unlock_id([H|T],Id) ->\n    case H#lock.id of\n        Id ->\n            {ok,T};\n        _ ->\n            {Status,Result} = do_unlock_id(T,Id),\n            {Status,[H|Result]}\n    end.\n\n%%----------------------------------------------------------------------\n%% do_locked(Path,Table) -> Table\n%%\ndo_locked([H],Table) ->\n    case lists:keysearch(H,1,Table) of\n        {value,{H,Locks,_}} when length(Locks)>0 -> true;\n        _ -> false\n    end;\ndo_locked([H|T],Table) ->\n    case lists:keysearch(H,1,Table) of\n        {value,{H,_Locks,Children}} -> do_locked(T,Children);\n        _ -> false\n    end.\n\n%%----------------------------------------------------------------------\n%% do_check(Path,Id,Table) -> Table\n%%\ndo_check([H|T],Id,Table) ->\n    case lists:keysearch(H,1,Table) of\n        {value,{H,Locks,_}} when length(T)==0 -> do_check_locks(Locks,Id);\n        {value,{H,_Lock,Children}} -> do_check(T,Id,Children);\n        false -> {error,not_found}\n    end.\n\ndo_check_locks([],_Id) ->\n    {error,not_found};\ndo_check_locks([H|T],Id) ->\n    case H#lock.id of\n        Id -> ok;\n        _ -> do_check_locks(T,Id)\n    end.\n\n%%----------------------------------------------------------------------\n%% do_discover(Path,Table) -> Locks\n%%\ndo_discover([H|T],Table) ->\n    case lists:keysearch(H,1,Table) of\n        {value,{H,Locks,_}} when length(T)==0 ->\n            Locks;\n        {value,{H,Locks,Children}} ->\n            do_discover_depth_infinity(Locks)++do_discover(T,Children);\n        false ->\n            []\n    end.\n\ndo_discover_depth_infinity([]) ->\n    [];\ndo_discover_depth_infinity([H|T]) ->\n    Take = case H#lock.depth of\n               infinity -> [H];\n               _ -> []\n           end,\n    Take ++ do_discover_depth_infinity(T).\n\n%%----------------------------------------------------------------------\n%% do_report(Path) -> Report\n%%\ndo_report(Table) ->\n    case Table of\n        [] -> io:format(\"No locks.~n\",[]);\n        _ -> do_report([],Table)\n    end.\ndo_report(_Path,[]) ->\n    ok;\ndo_report(Path,[{Name,Locks,Children}|T]) ->\n    Resource = filename:join(Path,Name),\n    if\n        length(Locks)>0 ->\n            io:format(\"~p~n\",[Resource]),\n            do_report_locks(Locks);\n        true ->\n            []\n    end,\n    do_report(Resource,Children),\n    do_report(Path,T).\n\ndo_report_locks([]) ->\n    ok;\ndo_report_locks([H|T]) ->\n    io:format(\"... ~p lock with token ~p, scope ~p, depth ~p~n\",\n              [H#lock.type,H#lock.id,H#lock.scope,H#lock.depth]),\n    do_report_locks(T).\n\n%%----------------------------------------------------------------------\n%% do_cleanup(Table) -> Table\n%%\ndo_cleanup([]) ->\n    [];\ndo_cleanup([{Name,Locks,Children}|T]) ->\n    Locks1 = do_cleanup_locks(Locks),\n    Children1 = do_cleanup(Children),\n    if\n        (length(Locks1)==0) and (length(Children1)==0) ->\n            do_cleanup(T);\n        true ->\n            [{Name,Locks1,Children1}|do_cleanup(T)]\n    end.\n\ndo_cleanup_locks([]) ->\n    [];\ndo_cleanup_locks([H|T]) ->\n    T0 = H#lock.timestamp,\n    T1 = yaws:get_time_tuple(),\n    Delta = timer:now_diff(T1,T0),\n    if\n        Delta > (H#lock.timeout*1000000) ->\n            error_logger:info_msg(\"discarded lock ~p~n\",[H#lock.id]),\n            do_cleanup_locks(T);\n        true ->\n            [H|do_cleanup_locks(T)]\n    end.\n\n%%----------------------------------------------------------------------\nlocktoken() ->\n    % RFC4122 section 3 based UUID\n    Version = 1,\n    Variant = 2#10,\n    Now = {_, _, Micro} = yaws:get_time_tuple(),\n    Nowish = calendar:now_to_universal_time(Now),\n    Timestamp = calendar:datetime_to_gregorian_seconds(Nowish) * 1000000000,\n    <<TimeHi:12, TimeMid:16, TimeLow:32>> = <<Timestamp:60>>,\n    Clocksequence = <<Micro:14>>,\n    <<ClockseqHi:6, ClockseqLow:8>> = Clocksequence,\n    Node = get_hwaddr(),\n    UUID = <<TimeLow:32, TimeMid:16, Version:4, TimeHi:12,\n      Variant:2, ClockseqLow:8, ClockseqHi:6, Node/binary>>,\n    <<U0:32, U1:16, U2:16, U3:16, U4:48>> = UUID,\n    lists:flatten(io_lib:format(\"~8.16.0b-~4.16.0b-~4.16.0b-~4.16.0b-~12.16.0b\"\n                                ,[U0,U1,U2,U3,U4])).\n\n\nget_hwaddr() ->\n    get_hwaddr(erlang:function_exported(inet, getifaddrs, 0)).\nget_hwaddr(true) ->\n    {ok,Ifs} = inet:getifaddrs(),\n    Addrs = [ lists:keysearch(hwaddr,1,Attr) || {_If,Attr} <- Ifs ],\n    Addr = lists:max([ A || {value,{hwaddr,A}} <- Addrs ]),\n    list_to_binary(Addr);\nget_hwaddr(false) ->\n    %% this clause is for backward compatibility to R13\n    {ok, Ifs} = inet:getiflist(),\n    Addrs = lists:foldl(\n              fun([{hwaddr, HW}], Acc) -> [HW|Acc];\n                 ([], Acc) -> Acc\n              end, [],\n              [begin {ok, HW} =inet:ifget(If, [hwaddr]), HW end || If <- Ifs]),\n    HWAddrs = case Addrs of\n                  [] ->\n                      %% hwaddr doesn't work on Mac on R13.\n                      %% Fall back to ifconfig :(\n                      Ifconfig = os:cmd(\"/sbin/ifconfig -a\"),\n                      {ok, Pat} = re:compile(\"ether\\s+([0-9a-fA-F:]+)\"),\n                      {match, Matches} =\n                          re:run(Ifconfig, Pat, [global, {capture, [1]}]),\n                      HWs = [string:substr(Ifconfig, At+1, Len) ||\n                                [{At,Len}] <- Matches],\n                      [[erlang:list_to_integer(V, 16) ||\n                           V <- string:tokens(S, \":\")] || S <- HWs];\n                  _ ->\n                      Addrs\n              end,\n    list_to_binary(lists:max(HWAddrs)).\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_sendfile.erl",
    "content": "%%% File    : yaws_sendfile.erl\n%%% Author  : Steve Vinoski <vinoski@ieee.org>\n%%% Description : interface to sendfile linked-in driver for Yaws\n%%% Created :  9 Nov 2008 by Steve Vinoski <vinoski@ieee.org>\n\n-module(yaws_sendfile).\n-author('vinoski@ieee.org').\n\n-include(\"../include/yaws.hrl\").\n-include_lib(\"kernel/include/file.hrl\").\n\n-export([send/2, send/3, send/4]).\n-export([have_sendfile/0, have_erlang_sendfile/0, check_gc_flags/1]).\n\n%% export bytes_to_transfer to avoid warning when sendfile is disabled (or not\n%% supported)\n-export([bytes_to_transfer/3]).\n\n\n-ifdef(HAVE_SENDFILE).\n\n-behavior(gen_server).\n-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2,\n         code_change/3]).\n\n-export([start_link/0, stop/0]).\n\nhave_sendfile() -> true.\n\n-else.\n\nhave_sendfile() -> false.\n\n-endif.\n\nhave_erlang_sendfile() -> yaws_dynopts:have_erlang_sendfile().\n\ncheck_gc_flags(GC) ->\n    %% below, ignore dialyzer warning:\n    %% The pattern depends on the macro HAVE_ERLANG_SENDFILE\n    case have_erlang_sendfile() of\n        false when ?gc_use_erlang_sendfile(GC) ->\n            error_logger:error_msg(\"Cannot use file:sendfile/5: not supported, \"\n                                   \"gen_tcp:send/2 will be used instead.~n\",\n                                   []);\n        _ ->\n            ok\n    end,\n\n    %% below, ignore dialyzer warning:\n    %% The pattern depends on the macro HAVE_SENDFILE\n    case have_sendfile() of\n        false when ?gc_use_yaws_sendfile(GC) ->\n            error_logger:error_msg(\"Cannot use Yaws sendfile linked-in driver:\"\n                                   \" not supported, gen_tcp:send/2 will be used\"\n                                   \" instead.~n\",\n                                   []);\n        _ ->\n            ok\n    end.\n\n\nsend(Out, Filename) ->\n    send(Out, Filename, 0, all).\nsend(Out, Filename, Offset) ->\n    send(Out, Filename, Offset, all).\n\nsend(Out, Filename, Offset, Count) ->\n    GC             = get(gc),\n    ChunkSize      = GC#gconf.large_file_chunk_size,\n    ErlangSendFile = ?gc_use_erlang_sendfile(GC),\n    YawsSendFile   = ?gc_use_yaws_sendfile(GC),\n    if\n        ErlangSendFile ->\n            erlang_sendfile(Out, Filename, Offset, Count, ChunkSize);\n        YawsSendFile ->\n            yaws_sendfile(Out, Filename, Offset, Count, ChunkSize);\n        true ->\n            compat_send(Out, Filename, Offset, Count, ChunkSize)\n    end.\n\n\nbytes_to_transfer(Filename, Offset, Count) ->\n    case Count of\n        all ->\n            case file:read_file_info(Filename) of\n                {ok, #file_info{size = Size}} -> Size - Offset;\n                Error -> Error\n            end;\n        Count when is_integer(Count) ->\n            Count;\n        _ ->\n            {error, badarg}\n    end.\n\n\nerlang_sendfile(Out, Filename, Offset, Count, ChunkSize) ->\n    case have_erlang_sendfile() of\n        true ->\n            Count1 = bytes_to_transfer(Filename, Offset, Count),\n            case Count1 of\n                {error, _}=Error1 ->\n                    Error1;\n                _ ->\n                    case file:open(Filename, [raw, read, binary]) of\n                        {ok, RawFile} ->\n                            Res = file:sendfile(RawFile, Out, Offset, Count1,\n                                                [{chunk_size, ChunkSize}]),\n                            ok = file:close(RawFile),\n                            Res;\n                        Error2 ->\n                            Error2\n                    end\n            end;\n        false ->\n            compat_send(Out, Filename, Offset, Count, ChunkSize)\n     end.\n\n\n-ifdef(HAVE_SENDFILE).\n\nyaws_sendfile(Out, Filename, Offset, Count, ChunkSize) ->\n    Count1 = bytes_to_transfer(Filename, Offset, Count),\n    case Count1 of\n        {error, _}=Error ->\n            Error;\n        _ ->\n            case prim_inet:getfd(Out) of\n                {ok, SocketFd} ->\n                    do_send(Out, SocketFd, Filename, Offset, Count1, ChunkSize);\n                Error2 ->\n                    Error2\n            end\n    end.\n\n\n-record(state, {\n          port,                    % driver port\n          caller_tbl               % table mapping socket fd to caller\n         }).\n\nstart_link() ->\n    gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).\n\nstop() ->\n    gen_server:cast(?MODULE, stop).\n\ninit([]) ->\n    process_flag(trap_exit, true),\n    Shlib = \"yaws_sendfile_drv\",\n    Dir   = filename:join(yaws:get_priv_dir(), \"lib\"),\n    case erl_ddll:load_driver(Dir, Shlib) of\n        ok -> ok;\n        {error, already_loaded} -> ok;\n        _ -> exit({error, \"could not load driver \" ++ Shlib})\n    end,\n    Port = open_port({spawn, Shlib}, [binary]),\n    CallerTable = ets:new(yaws_sendfile, []),\n    {ok, #state{port = Port, caller_tbl = CallerTable}}.\n\nhandle_call({send, SocketFd, Msg}, From, State) ->\n    true = erlang:port_command(State#state.port, Msg),\n    true = ets:insert(State#state.caller_tbl, {SocketFd, From}),\n    {noreply, State};\nhandle_call(_Req, _From, State) ->\n    {reply, ok, State}.\n\nhandle_info({_, {data, <<Cnt:64, SocketFd:32, Res:8, Err/binary>>}}, State) ->\n    Reply = case Res of\n                1 ->\n                    {ok, Cnt};\n                0 ->\n                    {error,\n                     list_to_atom(\n                       lists:takewhile(fun(El) -> El =/= 0 end,\n                                       binary_to_list(Err)))}\n            end,\n    CallerTable = State#state.caller_tbl,\n    [{SocketFd, From}] = ets:lookup(CallerTable, SocketFd),\n    gen_server:reply(From, Reply),\n    ets:delete(CallerTable, SocketFd),\n    {noreply, State};\nhandle_info(_Info, State) ->\n    {noreply, State}.\n\nhandle_cast(stop, State) ->\n    {stop, State};\nhandle_cast(_, State) ->\n    {noreply, State}.\n\nterminate(_Reason, #state{port = Port, caller_tbl = CallerTable}) ->\n    erlang:port_close(Port),\n    receive {'EXIT', Port, _Reason} -> ok\n    after 0 -> ok\n    end,\n    ets:delete(CallerTable),\n    ok.\n\ncode_change(_OldVsn, Data, _Extra) ->\n    {ok, Data}.\n\ndo_send(_Out, _SocketFd, _Filename, _Offset, Count, _) when Count =< 0 ->\n    {ok, 0};\ndo_send(Out, SocketFd, Filename0, Offset, Count, ChunkSize) ->\n    Filename = case file:native_name_encoding() of\n                   latin1 -> Filename0;\n                   utf8 -> unicode:characters_to_binary(Filename0)\n               end,\n    Call = list_to_binary([<<Offset:64, Count:64, SocketFd:32>>,\n                           Filename, <<0:8>>]),\n    case gen_server:call(?MODULE, {send, SocketFd, Call}, infinity) of\n        {error, eoverflow} ->\n            compat_send(Out, Filename, Offset, Count, ChunkSize);\n        Else ->\n            Else\n    end.\n\n-else.\n\nyaws_sendfile(Out, Filename, Offset, Count, ChunkSize) ->\n    compat_send(Out, Filename, Offset, Count, ChunkSize).\n\n-endif.\n\n\n\ncompat_send(Out, Filename, Offset, Count0, ChunkSize) ->\n    Count = case Count0 of\n                0 -> all;\n                _ -> Count0\n            end,\n    case file:open(Filename, [read, binary, raw]) of\n        {ok, Fd} ->\n            file:position(Fd, {bof, Offset}),\n            Ret = loop_send(Fd, ChunkSize, file:read(Fd, ChunkSize), Out,\n                            Count, 0),\n            file:close(Fd),\n            Ret;\n        Err ->\n            Err\n    end.\n\nloop_send(Fd, ChunkSize, {ok, Bin}, Out, all, BytesSent) ->\n    case gen_tcp:send(Out, Bin) of\n        ok ->\n            loop_send(Fd, ChunkSize, file:read(Fd, ChunkSize), Out, all,\n                      BytesSent+size(Bin));\n        Err ->\n            Err\n    end;\nloop_send(_Fd, _ChunkSize, eof, _Out, _, BytesSent) ->\n    {ok, BytesSent};\nloop_send(Fd, ChunkSize, {ok, Bin}, Out, Count, BytesSent) ->\n    Sz = size(Bin),\n    if Sz < Count ->\n            case gen_tcp:send(Out, Bin) of\n                ok ->\n                    loop_send(Fd, ChunkSize, file:read(Fd, ChunkSize),\n                              Out, Count-Sz, BytesSent+Sz);\n                Err ->\n                    Err\n            end;\n       Sz == Count ->\n            case gen_tcp:send(Out, Bin) of\n                ok  -> {ok, BytesSent+Sz};\n                Err -> Err\n            end;\n       Sz > Count ->\n            <<Deliver:Count/binary , _/binary>> = Bin,\n            case gen_tcp:send(Out, Deliver) of\n                ok  -> {ok, BytesSent+Count};\n                Err -> Err\n            end\n    end;\nloop_send(_Fd, _, Err, _, _, _) ->\n    Err.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_server.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_server.erl\n%%% Author  : Claes Wikstrom <klacke@hyber.org>\n%%% Purpose :\n%%% Created : 16 Jan 2002 by Claes Wikstrom <klacke@hyber.org>\n%%%----------------------------------------------------------------------\n\n-module(yaws_server).\n-author('klacke@hyber.org').\n\n-behaviour(gen_server).\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_debug.hrl\").\n\n-include_lib(\"kernel/include/file.hrl\").\n\n-export([mappath/3, vdirpath/3]).\n\n\n%% External exports\n-export([start_link/1]).\n-export([safe_path/1]).\n\n%% gen_server callbacks\n-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2,\n         code_change/3]).\n-export([status/0,\n         getconf/0,\n         stats/0,\n         gs_status/0,\n         listen_port/1,\n         ssi/3,ssi/5,ssi/6\n        ]).\n\n%% internal exports\n-export([gserv/3,acceptor0/2, load_and_run/2, done_or_continue/0,\n         accumulate_content/1, deliver_accumulated/4, deliver_accumulated/1,\n         setup_dirs/1,\n         deliver_dyn_part/8, finish_up_dyn_file/2, gserv_loop/4\n        ]).\n\n%% exports for eunit usage\n-export([comp_sname/2, wildcomp_salias/2]).\n\n-export(['GET'/4,\n         'POST'/4,\n         'HEAD'/4,\n         'TRACE'/4,\n         'OPTIONS'/4,\n         'PUT'/4,\n         'DELETE'/4,\n         'PATCH'/4]).\n\n-import(lists, [member/2, foreach/2, map/2,\n                flatten/1, reverse/1]).\n\n-import(yaws_api, [ehtml_expand/1]).\n\n-record(gs, {gconf,\n             group,         %% list of #sconf{} s\n             ssl,           %% ssl | nossl\n             certinfo,      %% undefined | [{string(), #certinfo{}}]\n             l,             %% listen socket\n             connections = 0, %% number of TCP connections opened now\n             sessions = 0,  %% number of active HTTP sessions\n             reqs = 0}).    %% total number of processed HTTP requests\n\n\n-record(state, {gc,         %% Global conf #gc{} record\n                pairs,      %% [{GservPid, ScList}]\n                embedded    %% true if in embedded mode, false otherwise\n               }).\n\n%% undefined | mtime from #file_info\n-record(certinfo, {keyfile,\n                   certfile,\n                   cacertfile\n                  }).\n\n-define(elog(X,Y), error_logger:info_msg(\"*elog ~p:~p: \" X,\n                                         [?MODULE, ?LINE | Y])).\n\n\nstart_link(A) ->\n    gen_server:start_link({local, yaws_server}, yaws_server, A, []).\n\nstatus() ->\n    gen_server:call(?MODULE, status, 10000).\ngs_status() ->\n    [_|Pids] = gen_server:call(?MODULE, pids, 10000),\n    lists:map(\n      fun(P) ->\n              P ! {self(), status},\n              receive {P, Stat} -> Stat end\n      end, Pids).\ngetconf() ->\n    gen_server:call(?MODULE,getconf, infinity).\n\n%% Return the configured port number from the sconf or, if the port number\n%% is 0 indicating an ephemeral port, retrieve the actual port via sockname\nlisten_port(#sconf{}=SC) ->\n    try\n        lists:foldl(fun(#gs{group=SCs, l=Sock}, Acc) ->\n                            case lists:member(SC, SCs) of\n                                true ->\n                                    {ok, {_, Port}} =\n                                        case SC#sconf.ssl of\n                                            undefined ->\n                                                inet:sockname(Sock);\n                                            _ ->\n                                                ssl:sockname(Sock)\n                                        end,\n                                    %% throw the result to end the fold early\n                                    throw(Port);\n                                false ->\n                                    Acc\n                            end\n                    end, [], gs_status()),\n        {error, not_found}\n    catch\n        throw:Port ->\n            Port\n    end.\n\nstats() ->\n    {_S, Time} = status(),\n    Diff = calendar:time_difference(Time, calendar:local_time()),\n    L = [begin\n             SC = hd(GS#gs.group),\n             {SC#sconf.listen, SC#sconf.port,\n              GS#gs.connections, GS#gs.sessions, GS#gs.reqs}\n         end || GS <- gs_status()],\n    {Diff, L}.\n\n\nl2a(L) when is_list(L) -> list_to_atom(L);\nl2a(A) when is_atom(A) -> A.\n\n\n\n%%----------------------------------------------------------------------\n%% Func: init/1\n%% Returns: {ok, State}          |\n%%          {ok, State, Timeout} |\n%%          ignore               |\n%%          {stop, Reason}\n%%----------------------------------------------------------------------\n\ninit(Env) -> %% #env{Trace, TraceOut, Conf, RunMod, Embedded, Id}) ->\n    process_flag(trap_exit, true),\n    put(start_time, calendar:local_time()),  %% for uptime\n    case Env#env.embedded of\n        false ->\n            Config = (catch yaws_config:load(Env)),\n            case Config of\n                {ok, Gconf, Sconfs} ->\n                    erase(logdir),\n                    ?Debug(\"GC = ~s~n\", [?format_record(Gconf, gconf)]),\n                    lists:foreach(\n                      fun(Group) ->\n                              lists:foreach(\n                                fun(_SC) ->\n                                        ?Debug(\"SC = ~s~n\",\n                                               [?format_record(_SC, sconf)])\n                                end, Group)\n                      end, Sconfs),\n                    init2(Gconf, Sconfs, Env#env.runmod, Env#env.embedded, true);\n                {error, E} ->\n                    case erase(logdir) of\n                        undefined ->\n                            error_logger:error_msg(\"Yaws: Bad conf: ~p~n\",[E]),\n                            init:stop(),\n                            {stop, E};\n                        Dir ->\n                            GC = yaws_config:make_default_gconf(true,\n                                                                Env#env.id),\n                            yaws_log:setup(GC#gconf{logdir = Dir}, []),\n                            error_logger:error_msg(\"Yaws: bad conf: ~s \"\n                                                   \"terminating~n\",[E]),\n                            init:stop(),\n                            {stop, E}\n                    end;\n                EXIT ->\n                    error_logger:format(\"FATAL ~p~n\", [EXIT]),\n                    erlang:error(badconf)\n            end;\n        true ->\n            {ok, #state{gc = undefined,\n                        embedded = Env#env.embedded,\n                        pairs = []}}\n    end.\n\n\ninit2(GC, Sconfs, RunMod, Embedded, FirstTime) ->\n    put(gc, GC),\n    yaws_sendfile:check_gc_flags(GC),\n    case GC#gconf.mnesia_dir of\n        MD when length(MD) > 0 ->\n            yaws_debug:format(\"loading mnesia ~p~n\", [MD]),\n            application:set_env(mnesia,dir,MD),\n            mnesia:start();\n        _ ->\n            ok\n    end,\n    foreach(fun(D) ->\n                    yaws_debug:format(\"Add path ~p~n\", [D]),\n                    code:add_pathz(D)\n            end, GC#gconf.ebin_dir),\n    yaws_debug:format(\"Running with id=~p ~n\"\n                      \"~s\"\n                      \"Logging to directory ~p~n\",\n                      [GC#gconf.id,\n                       if ?gc_has_debug(GC) ->\n                               \"Running with debug checks \"\n                                   \"turned on (slower server) \\n\";\n                          true ->\n                               \"\"\n                       end,\n                       GC#gconf.logdir]),\n\n    case Embedded of\n        false ->\n            setup_dirs(GC),\n            case yaws_ctl:start(GC, FirstTime) of\n                ok ->\n                    ok;\n                {error, RSN} ->\n                    %% Must call init stop here otherwise heart\n                    %% will restart us\n                    error_logger:format(\"Failed to start: ~s~n\", [RSN]),\n                    init:stop(),\n                    receive nothing -> ok end\n            end;\n        true ->\n            ok\n    end,\n\n    runmod(RunMod, GC),\n    yaws_config:compile_and_load_src_dir(GC),\n    yaws_dynopts:generate(GC),\n    yaws_log:setup(GC, Sconfs),\n    yaws_trace:setup(GC),\n    L2 = lists:zf(fun(Group) -> start_group(GC, Group) end,\n                  yaws_config:load_mime_types_module(GC, Sconfs)),\n    {ok, #state{gc       = GC,\n                pairs    = L2,\n                embedded = Embedded}}.\n\n\n\nstart_group(GC, Group) ->\n    FailOnBind = ?gc_fail_on_bind_err(GC),\n    case proc_lib:start_link(?MODULE, gserv, [self(), GC, Group]) of\n        {error, F, A} when FailOnBind == false ->\n            error_logger:error_msg(F, A),\n            false;\n        {error, F, A} ->\n            error_logger:error_msg(F, A),\n            erlang:error(badbind);\n        {error, Reason} when FailOnBind == false ->\n            error_logger:error_msg(\"FATAL: ~p~n\", [Reason]),\n            false;\n        {error, Reason} ->\n            error_logger:error_msg(\"FATAL: ~p~n\", [Reason]),\n            erlang:error(badbind);\n        {Pid, SCs} ->\n            {true, {Pid, SCs}};\n        none ->\n            false\n    end.\n\n\n\n\n%%----------------------------------------------------------------------\n%% Func: handle_call/3\n%% Returns: {reply, Reply, State}          |\n%%          {reply, Reply, State, Timeout} |\n%%          {noreply, State}               |\n%%          {noreply, State, Timeout}      |\n%%          {stop, Reason, Reply, State}   | (terminate/2 is called)\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%----------------------------------------------------------------------\nhandle_call(status, _From, State) ->\n    Reply = {State, get(start_time)},\n    {reply, Reply, State};\n\nhandle_call(id, _From, State) ->\n    {reply, (State#state.gc)#gconf.id, State};\n\nhandle_call(pids, _From, State) ->  %% for gprof\n    L = map(fun(X) ->element(1, X) end, State#state.pairs),\n    {reply, [self() | L], State};\n\n\n%% This is a brutal restart of everything\nhandle_call({setconf, GC, Groups}, _From, State) ->\n    %% First off, terminate all currently running processes\n    Curr = map(fun(X) ->element(1, X) end, State#state.pairs),\n    foreach(fun(Pid) ->\n                    gserv_stop(Pid)\n            end, Curr),\n    {ok, State2} = init2(GC, Groups, undef, State#state.embedded, false),\n    {reply, ok, State2};\n\nhandle_call(getconf, _From, State) ->\n    Groups = map(fun({_Pid, SCs}) -> SCs end, State#state.pairs),\n    {reply, {ok, State#state.gc, Groups}, State};\n\n%% If cert has changed, server will stop implicitly\nhandle_call(check_certs, _From, State) ->\n    L = lists:map(fun({Pid, _SCs}) ->\n                          Pid ! {check_cert_changed, self()},\n                          receive {Pid, YesNo} ->\n                                  YesNo\n                          end\n                  end, State#state.pairs),\n    {reply, L, State};\n\nhandle_call({update_sconf, Pos, NewSc}, From, State) ->\n    case yaws_config:search_sconf(State#state.gc, NewSc, State#state.pairs) of\n        {Pid, OldSc, Group} ->\n            OldPos = string:str(Group, [OldSc]),\n            case (yaws_config:eq_sconfs(OldSc,NewSc) andalso OldPos == Pos) of\n                true ->\n                    error_logger:info_msg(\"Keeping conf for ~s intact\\n\",\n                                          [yaws:sconf_to_srvstr(OldSc)]),\n                    {reply, ok,  State};\n                false ->\n                    Pid ! {update_sconf, Pos, NewSc, OldSc, From, self()},\n                    receive\n                        {updated_sconf, Pid, NewSc2} ->\n                            P2 = yaws_config:update_sconf(State#state.gc,\n                                                          NewSc2, Pos,\n                                                          State#state.pairs),\n                            {noreply, State#state{pairs = P2}}\n                    after 2000 ->\n                            {reply, {error, \"Failed to update new conf\"}, State}\n                    end\n            end;\n        false ->\n            {reply, {error, \"No matching group\"}, State}\n    end;\n\n\nhandle_call({delete_sconf, Sc}, From, State) ->\n    case yaws_config:search_sconf(State#state.gc, Sc, State#state.pairs) of\n        {Pid, OldSc, Group} when length(Group) == 1 ->\n            error_logger:info_msg(\"Terminate whole ~s virt server group \\n\",\n                                  [yaws:sconf_to_srvstr(OldSc)]),\n            gserv_stop(Pid),\n            NewPairs = lists:keydelete(Pid, 1, State#state.pairs),\n            {reply, ok, State#state{pairs = NewPairs}};\n        {Pid, OldSc, _Group} ->\n            Pid ! {delete_sconf, OldSc, From},\n            P2 = yaws_config:delete_sconf(State#state.gc, OldSc,\n                                          State#state.pairs),\n            {noreply, State#state{pairs = P2}};\n        false ->\n            {reply, {error, \"No matching group\"}, State}\n    end;\n\nhandle_call({add_sconf, Pos, Sc}, From, State) ->\n    case yaws_config:search_group(State#state.gc, Sc, State#state.pairs) of\n        [{Pid, _Group}] ->\n            Pid ! {add_sconf, From, Pos, Sc, self()},\n            receive\n                {added_sconf, Pid, Sc2} ->\n                    P2 = yaws_config:update_sconf(State#state.gc,\n                                                  Sc2, Pos,\n                                                  State#state.pairs),\n                    {noreply, State#state{pairs = P2}}\n            after 2000 ->\n                    {reply, {error, \"Failed to add new conf\"}, State}\n            end;\n        [] ->\n            %% Need to create a new group\n            error_logger:info_msg(\"Creating new virt server ~s\\n\",\n                                  [yaws:sconf_to_srvstr(Sc)]),\n            GC = State#state.gc,\n            case start_group(GC, [Sc]) of\n                false ->\n                    {reply, {ok, Sc}, State};\n                {true, {_,[Sc2]}=Pair} ->\n                    P2 = [Pair | State#state.pairs],\n                    {reply, {ok, Sc2}, State#state{pairs = P2}}\n            end\n    end;\n\nhandle_call({update_gconf, GC}, _From, State) ->\n    lists:foreach(fun({Pid, _Group}) ->\n                          Pid ! {update_gconf, GC}\n                  end, State#state.pairs),\n    %% no need to tell yaws_log, new vals must be compatible\n    put(gc, GC),\n    {reply, ok, State#state{gc = GC}}.\n\n\n\n\n\n%%----------------------------------------------------------------------\n%% Func: handle_cast/2\n%% Returns: {noreply, State}          |\n%%          {noreply, State, Timeout} |\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%----------------------------------------------------------------------\nhandle_cast(_Msg, State) ->\n    {noreply, State}.\n\n%%----------------------------------------------------------------------\n%% Func: handle_info/2\n%% Returns: {noreply, State}          |\n%%          {noreply, State, Timeout} |\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%----------------------------------------------------------------------\nhandle_info({Pid, certchanged},  State) ->\n    {noreply, State#state{pairs = lists:keydelete(Pid, 1, State#state.pairs)}};\nhandle_info({'EXIT', Pid, Reason},  State) ->\n    case lists:keysearch(Pid, 1, State#state.pairs) of\n        {value, _} ->\n            %% one of our gservs died\n            error_logger:format(\"yaws: FATAL gserv died ~p~n\", [Reason]),\n            erlang:error(restartme);\n        false ->\n            ignore\n    end,\n    {noreply, State};\n\n\nhandle_info(_Msg, State) ->\n    ?Debug(\"GOT ~p~n\", [_Msg]),\n    {noreply, State}.\n\n%%----------------------------------------------------------------------\n%% Func: terminate/2\n%% Purpose: Shutdown the server\n%% Returns: any (ignored by gen_server)\n%%----------------------------------------------------------------------\nterminate(_Reason, State) ->\n    foreach(fun({Pid, _GP}) -> gserv_stop(Pid) end, State#state.pairs),\n    ok.\n\ndo_listen(GC, [SC0|_]=Group) ->\n    case SC0#sconf.ssl of\n        undefined ->\n            {nossl, undefined,\n             gen_tcp_listen(SC0#sconf.port, tcp_listen_opts(SC0))};\n        _ ->\n            CertInfo = lists:foldl(fun(#sconf{servername=SN, ssl=SSL}, Acc) ->\n                                           [{SN, certinfo(SSL)}|Acc]\n                                   end, [], Group),\n            {ssl, lists:reverse(CertInfo),\n             ssl_listen(SC0#sconf.port, ssl_listen_opts(GC, Group))}\n    end.\n\ncertinfo(SSL=#ssl{}) ->\n    #certinfo{\n       keyfile = if SSL#ssl.keyfile /= undefined ->\n                         case file:read_file_info(SSL#ssl.keyfile) of\n                             {ok, FI} ->\n                                 FI#file_info.mtime;\n                             _ ->\n                                 undefined\n                         end;\n                    true ->\n                         undefined\n                 end,\n       certfile = if SSL#ssl.certfile /= undefined ->\n                          case file:read_file_info(SSL#ssl.certfile) of\n                              {ok, FI} ->\n                                  FI#file_info.mtime;\n                              _ ->\n                                  undefined\n                          end;\n                     true ->\n                          undefined\n                  end,\n       cacertfile = if SSL#ssl.cacertfile /= undefined ->\n                            case file:read_file_info(SSL#ssl.cacertfile) of\n                                {ok, FI} ->\n                                    FI#file_info.mtime;\n                                _ ->\n                                    undefined\n                            end;\n                       true ->\n                            undefined\n                    end\n      }.\n\ngen_tcp_listen(Port, Opts) ->\n    ?Debug(\"TCP Listen ~p:~p~n\", [Port, Opts]),\n    gen_tcp:listen(Port, Opts).\n\nssl_listen(Port, Opts) ->\n    ?Debug(\"SSL Listen ~p:~p~n\", [Port, Opts]),\n    ssl:listen(Port, Opts).\n\ngserv(_Top, _, []) ->\n    proc_lib:init_ack(none);\n\n%% One server per IP we listen to\ngserv(Top, GC, Group0) ->\n    process_flag(trap_exit, true),\n    ?TC([{record, GC, gconf}]),\n    put(gc, GC),\n    put(top, Top),\n    Group1 = map(fun(SC) -> setup_ets(SC) end, Group0),\n    Group = map(fun(SC) -> start_stats(SC) end, Group1),\n    case do_listen(GC, Group) of\n        {SSLBOOL, CertInfo, {ok, Listen}} ->\n            lists:foreach(fun(SC) -> call_start_mod(SC) end, Group),\n            error_logger:info_msg(\n              \"Yaws: Listening to ~s:~w for <~p> virtual servers:~s~n\",\n              [inet_parse:ntoa((hd(Group))#sconf.listen),\n               (hd(Group))#sconf.port,\n               length(Group),\n               catch map(\n                       fun(S) ->\n                               io_lib:format(\"~n - ~s under ~s\",\n                                             [yaws:sconf_to_srvstr(S),\n                                              S#sconf.docroot])\n                       end, Group)\n              ]),\n            proc_lib:init_ack({self(), Group}),\n            GS = #gs{gconf = GC,\n                     group = Group,\n                     ssl = SSLBOOL,\n                     certinfo = CertInfo,\n                     l = Listen},\n            Last = initial_acceptor(GS),\n            gserv_loop(GS#gs{sessions = 1}, [], 0, Last);\n        {_,_,Err} ->\n            error_logger:format(\"Yaws: Failed to listen ~s:~w  : ~p~n\",\n                                [inet_parse:ntoa((hd(Group))#sconf.listen),\n                                 (hd(Group))#sconf.port, Err]),\n            proc_lib:init_ack({error, \"Can't listen to socket: ~p \",[Err]}),\n            exit(normal)\n    end.\n\n\n\nsetup_dirs(GC) ->\n    Dir = yaws:id_dir(GC#gconf.id),\n    Ctl = yaws:ctl_file(GC#gconf.id),\n    ok = filelib:ensure_dir(Ctl),\n    case file:list_dir(Dir) of\n        {ok, LL} ->\n            foreach(\n              fun(F) ->\n                      file:delete(filename:join([Dir, F]))\n              end, LL -- [\"CTL\"]);\n        {error, RSN} ->\n            error_logger:format(\"Failed to list ~p probably \"\n                                \"due to permission errs: ~p\",\n                                [Dir, RSN]),\n            erlang:error(RSN)\n    end.\n\n\nsetup_ets(SC) ->\n    E = ets:new(yaws_code, [public, set]),\n    ets:insert(E, {num_files, 0}),\n    ets:insert(E, {num_bytes, 0}),\n    SC#sconf{ets = E}.\n\nclear_ets_complete(SC) ->\n    case SC#sconf.ets of\n        undefined ->\n            setup_ets(SC);\n        E ->\n            ets:match_delete(E,'_'),\n            ets:insert(E, {num_files, 0}),\n            ets:insert(E, {num_bytes, 0}),\n            SC\n    end.\n\n\nstart_stats(SC) ->\n    case ?sc_has_statistics(SC) of\n        true ->\n            {ok, Pid} = yaws_stats:start_link(),\n            SC#sconf{stats = Pid};\n        false ->\n            SC\n    end.\n\nstop_stats(SC) ->\n    case SC#sconf.stats of\n        undefined ->\n            SC;\n        Pid when is_pid(Pid) ->\n            %% Unlink the stats process before stopping it to be sure to not\n            %% receive the {'EXIT..} message in gserv_loop.\n            unlink(Pid),\n            yaws_stats:stop(Pid),\n            SC#sconf{stats = undefined}\n    end.\n\ngserv_loop(GS, Ready, Rnum, Last) ->\n    receive\n        {From , status} ->\n            From ! {self(), GS},\n            ?MODULE:gserv_loop(GS, Ready, Rnum, Last);\n        {_From, next, Accepted} when Ready == [] ->\n            close_accepted_if_max(GS,Accepted),\n            New = acceptor(GS),\n            GS2 = GS#gs{sessions = GS#gs.sessions + 1,\n                        connections = GS#gs.connections + 1},\n            ?MODULE:gserv_loop(GS2, Ready, Rnum, New);\n        {_From, next, Accepted} ->\n            close_accepted_if_max(GS,Accepted),\n            [{_Then, R}|RS] = Ready,\n            R ! {self(), accept},\n            GS2 = GS#gs{connections=GS#gs.connections + 1},\n            ?MODULE:gserv_loop(GS2, RS, Rnum-1, R);\n        {_From, decrement} ->\n            GS2 = GS#gs{connections=GS#gs.connections - 1},\n            ?MODULE:gserv_loop(GS2, Ready, Rnum, Last);\n        {From, done_client, Int} ->\n            GS2 = if\n                      Int == 0 -> GS#gs{connections = GS#gs.connections - 1};\n                      Int > 0  -> GS#gs{reqs = GS#gs.reqs+Int,\n                                        connections = GS#gs.connections - 1}\n                  end,\n            PoolSize = (GS#gs.gconf)#gconf.acceptor_pool_size,\n            if\n                Rnum == PoolSize ->\n                    From ! {self(), stop},\n                    ?MODULE:gserv_loop(GS2, Ready, Rnum, Last);\n                Rnum < PoolSize ->\n                    %% cache this process for 10 secs\n                    ?MODULE:gserv_loop(GS2,\n                                       [{yaws:get_time_tuple(), From} | Ready],\n                                       Rnum+1, Last)\n            end;\n        {'EXIT', Pid, Reason} ->\n            case get(top) of\n                Pid when Reason /= shutdown ->\n                    error_logger:format(\"Top proc died, terminate gserv\",[]),\n                    {links, Ls} = process_info(self(), links),\n                    foreach(fun(X) -> unlink(X), exit(X, shutdown) end, Ls),\n                    exit(noserver);\n                Pid ->\n                    {links, Ls} = process_info(self(), links),\n                    foreach(fun(X) -> unlink(X), exit(X, shutdown) end, Ls),\n                    exit(normal);\n                Top when Reason == failaccept ->\n                    error_logger:format(\n                      \"Accept proc died, terminate gserv\",[]),\n                    {links, Ls} = process_info(self(), links),\n                    %% do not send exit signal to yaws_server process\n                    Ls1 = Ls -- [Top],\n                    foreach(fun(X) -> unlink(X), exit(X, shutdown) end, Ls1),\n                    exit(noserver);\n                _ ->\n                    GS2 = GS#gs{sessions = GS#gs.sessions - 1},\n                    if\n                        Pid == Last ->\n                            %% probably died due to new code loaded; if we\n                            %% don't start a new acceptor here we end up with\n                            %% no active acceptor\n                            error_logger:format(\"Last acceptor died (~p), \"\n                                                \"restart it\", [Reason]),\n                            New = acceptor(GS),\n                            ?MODULE:gserv_loop(GS2, Ready, Rnum, New);\n                       true ->\n                            case lists:keysearch(Pid, 2, Ready) of\n                                {value, _} ->\n                                    Ready1 = lists:keydelete(Pid, 2, Ready),\n                                    ?MODULE:gserv_loop(GS2, Ready1,\n                                                       Rnum-1, Last);\n                                false ->\n                                    ?MODULE:gserv_loop(GS2, Ready, Rnum, Last)\n                            end\n                    end\n            end;\n        {From, stop} ->\n            unlink(From),\n\n            %% Close the socket and stop acceptors\n            stop_ready(Ready, Last),\n            if\n                GS#gs.ssl == nossl -> gen_tcp:close(GS#gs.l);\n                GS#gs.ssl == ssl   -> ssl:close(GS#gs.l)\n            end,\n\n            %% Stop yaws_stats processes, if needed\n            foreach(fun(SC) -> stop_stats(SC) end, GS#gs.group),\n\n            %% Close softly all opened connections\n            {links, Ls1} = process_info(self(), links),\n            foreach(fun(X) when is_pid(X) ->\n                            unlink(X),\n                            X ! {self(), suspend},\n                            exit(X, shutdown);\n                       (_) -> ok\n                    end, Ls1),\n            WaitFun = fun(_, 0, Pids)     -> Pids;\n                         (_, _, [])       -> [];\n                         (F, Secs, Pids0) ->\n                              timer:sleep(1000),\n                              Pids1 = lists:filter(fun(X) when is_pid(X) ->\n                                                           is_process_alive(X);\n                                                      (_) ->\n                                                           false\n                                                   end, Pids0),\n                              F(F, Secs-1, Pids1)\n                      end,\n            Ls2 = WaitFun(WaitFun, 60, Ls1),\n\n            %% Kill all remaining connections\n            foreach(fun(X) -> exit(X, kill) end, Ls2),\n\n            From ! {self(), ok},\n            exit(normal);\n\n\n        %% This code will shutdown all ready procs as well as the\n        %% acceptor()\n        {update_sconf, Pos, NewSc, OldSc, From, Updater} ->\n            case lists:member(OldSc, GS#gs.group) of\n                false ->\n                    error_logger:error_msg(\"gserv: No found SC ~p/~p~n\",\n                                           [OldSc, GS#gs.group]),\n                    erlang:error(nosc);\n                true ->\n                    stop_stats(OldSc),\n                    NewSc1 = start_stats(NewSc),\n                    stop_ready(Ready, Last),\n                    NewSc2 = clear_ets_complete(\n                               NewSc1#sconf{ets = OldSc#sconf.ets}\n                              ),\n                    GS2 = GS#gs{group = yaws:insert_at(\n                                          NewSc2, Pos,\n                                          lists:delete(OldSc, GS#gs.group)\n                                         )},\n                    Ready2 = [],\n                    Updater ! {updated_sconf, self(), NewSc2},\n                    gen_server:reply(From, {ok, NewSc2}),\n                    error_logger:info_msg(\"Updating sconf for server ~s~n\",\n                                          [yaws:sconf_to_srvstr(NewSc2)]),\n                    New = acceptor(GS2),\n                    ?MODULE:gserv_loop(GS2, Ready2, 0, New)\n            end;\n\n        {delete_sconf, OldSc, From} ->\n            case lists:member(OldSc, GS#gs.group) of\n                false ->\n                    error_logger:error_msg(\"gserv: No found SC ~n\",[]),\n                    erlang:error(nosc);\n                true ->\n                    stop_ready(Ready, Last),\n                    GS2 = GS#gs{group =  lists:delete(OldSc,GS#gs.group)},\n                    Ready2 = [],\n                    stop_stats(OldSc),\n                    ets:delete(OldSc#sconf.ets),\n                    gen_server:reply(From, ok),\n                    error_logger:info_msg(\"Deleting sconf for server ~s~n\",\n                                          [yaws:sconf_to_srvstr(OldSc)]),\n                    New = acceptor(GS2),\n                    ?MODULE:gserv_loop(GS2, Ready2, 0, New)\n            end;\n\n        {add_sconf, From, Pos, SC0, Adder} ->\n            SC = start_stats(SC0),\n            stop_ready(Ready, Last),\n            SC2 = setup_ets(SC),\n            GS2 = GS#gs{group =  yaws:insert_at(SC2, Pos, GS#gs.group)},\n            Ready2 = [],\n            Adder ! {added_sconf, self(), SC2},\n            gen_server:reply(From, {ok, SC2}),\n            error_logger:info_msg(\"Adding sconf for server ~s~n\",\n                                  [yaws:sconf_to_srvstr(SC)]),\n            New = acceptor(GS2),\n            ?MODULE:gserv_loop(GS2, Ready2, 0, New);\n        {check_cert_changed, From} ->\n            Changed =\n                case GS#gs.ssl of\n                    ssl ->\n                        CertInfo = GS#gs.certinfo,\n                        lists:any(\n                          fun(#sconf{servername=SN}=SC) ->\n                                  certinfo(SC#sconf.ssl) /=\n                                      proplists:get_value(SN, CertInfo)\n                          end,\n                          GS#gs.group\n                         );\n                    nossl ->\n                        false\n                end,\n            if\n                Changed == false ->\n                    From ! {self(), no},\n                    ?MODULE:gserv_loop(GS, Ready, Rnum, Last);\n                Changed == true ->\n                    error_logger:info_msg(\n                      \"Stopping ~s due to cert change\\n\",\n                      [yaws:sconf_to_srvstr(hd(GS#gs.group))]),\n                    {links, Ls0} = process_info(self(), links),\n                    Ls = Ls0 -- [get(top)],\n                    foreach(fun(X) -> unlink(X), exit(X, shutdown) end, Ls),\n                    From ! {self(), yes},\n                    unlink(get(top)),\n                    get(top) ! {self(), certchanged},\n                    exit(normal)\n            end;\n        {update_gconf, GC} ->\n            stop_ready(Ready, Last),\n            GS2 = GS#gs{gconf = GC},\n            Ready2 = [],\n            put(gc, GC),\n            error_logger:info_msg(\"Updating gconf \\n\",[]),\n            New = acceptor(GS2),\n            ?MODULE:gserv_loop(GS2, Ready2, 0, New)\n    after (10 * 1000) ->\n            %% collect old procs, to save memory\n            {NowMega, NowSecs, _} = yaws:get_time_tuple(),\n            R2 = lists:filter(fun({{ThenMega, ThenSecs, _}, Pid}) ->\n                                      if\n                                          NowMega > ThenMega;\n                                          (NowSecs > (ThenSecs + 8)) ->\n                                              Pid ! {self(), stop},\n                                              false;\n                                          true ->\n                                              true\n                                      end\n                              end, Ready),\n            ?MODULE:gserv_loop(GS, R2, length(R2), Last)\n    end.\n\n\nstop_ready(Ready, Last) ->\n    error_logger:info_msg(\"stop_ready(~p, ~p)~n\", [Ready, Last]),\n    unlink(Last),\n    exit(Last, shutdown),\n\n    lists:foreach(\n      fun({_,Pid}) ->\n              Pid ! {self(), stop}\n      end, Ready).\n\ngserv_stop(Gpid) ->\n    case is_process_alive(Gpid) of\n        true ->\n            Gpid ! {self(), stop},\n            receive\n                {Gpid, ok} -> ok\n            end;\n        false ->\n            ok\n    end.\n\ncall_start_mod(SC) ->\n    case SC#sconf.start_mod of\n        undefined ->\n            ok;\n        Mod0 ->\n            Mod = l2a(Mod0),\n            case code:ensure_loaded(Mod) of\n                {module, Mod} ->\n                    error_logger:info_msg(\n                      \"Yaws: calling start_mod: ~p:start/1~n\", [Mod]),\n                    spawn(Mod, start, [SC]);\n                Err ->\n                    error_logger:format(\"Cannot load module ~p: ~p~n\",\n                                        [Mod,Err])\n            end\n    end.\n\nlisten_opts(SC) ->\n    InetType = if\n                   is_tuple( SC#sconf.listen), size( SC#sconf.listen) == 8 ->\n                       [inet6];\n                   true ->\n                       []\n               end,\n    [binary,\n     {ip, SC#sconf.listen},\n     {packet, http},\n     {packet_size, 16#4000},\n     {reuseaddr, true},\n     {active, false}\n     | proplists:get_value(listen_opts, SC#sconf.soptions, [])\n    ] ++ InetType.\n\ntcp_listen_opts(SC) ->\n    Opts = listen_opts(SC),\n    ?Debug(\"tcp listen options: ~p\", [Opts]),\n    Opts.\n\ncheck_sni_servername(_, [], Default) ->\n    Default;\ncheck_sni_servername(SN, [{SC,Opts}|Rest], Default) ->\n    case comp_sname(SN, SC#sconf.servername) of\n        true  ->\n            Opts;\n        false ->\n            Res = lists:any(fun(Alias) -> wildcomp_salias(SN, Alias) end,\n                            SC#sconf.serveralias),\n            case Res of\n                true  -> Opts;\n                false -> check_sni_servername(SN, Rest, Default)\n            end\n    end.\n\nssl_sni_opts(Group, DefaultSSLOpts) ->\n    SniOpts = lists:foldl(fun(SC, Acc) ->\n                                  [{SC, ssl_listen_opts(SC)}|Acc]\n                          end, [], Group),\n    SniFun = fun(SN) ->\n                     check_sni_servername(SN, lists:reverse(SniOpts),\n                                          DefaultSSLOpts)\n             end,\n    [{sni_fun, SniFun}].\n\nssl_listen_opts(GC, [SC0|_]=Group) ->\n    DefaultSSLOpts = ssl_listen_opts(SC0),\n    Opts0 = listen_opts(SC0) ++ DefaultSSLOpts,\n    Opts = case GC#gconf.sni of\n               disable -> Opts0;\n               _       -> Opts0 ++ ssl_sni_opts(Group, DefaultSSLOpts)\n           end,\n    ?Debug(\"ssl listen options: ~p\", [Opts]),\n    Opts.\n\nssl_listen_opts(#sconf{ssl=SSL}) ->\n    L = [if SSL#ssl.keyfile /= undefined ->\n                 {keyfile, SSL#ssl.keyfile};\n            true ->\n                 false\n         end,\n\n         if SSL#ssl.certfile /= undefined ->\n                 {certfile, SSL#ssl.certfile};\n            true ->\n                 false\n         end,\n\n         if SSL#ssl.cacertfile /= undefined  ->\n                 {cacertfile, SSL#ssl.cacertfile};\n            true ->\n                 false\n         end,\n\n         if SSL#ssl.dhfile /= undefined  ->\n                 {dhfile, SSL#ssl.dhfile};\n            true ->\n                 false\n         end,\n\n         if SSL#ssl.verify /= undefined ->\n                 {verify, SSL#ssl.verify};\n            true ->\n                 false\n         end,\n\n         if SSL#ssl.fail_if_no_peer_cert /= undefined ->\n                 {fail_if_no_peer_cert, SSL#ssl.fail_if_no_peer_cert};\n            true ->\n                 false\n         end,\n\n         if SSL#ssl.password /= undefined ->\n                 {password, SSL#ssl.password};\n            true ->\n                 false\n         end,\n         if SSL#ssl.ciphers /= undefined ->\n                 {ciphers, SSL#ssl.ciphers};\n            true ->\n                 false\n         end,\n         if SSL#ssl.protocol_version /= undefined ->\n                 {versions, SSL#ssl.protocol_version};\n            true ->\n                 false\n         end,\n         if SSL#ssl.depth /= undefined ->\n                 {depth, SSL#ssl.depth};\n            true ->\n                 false\n         end,\n         if SSL#ssl.secure_renegotiate /= undefined ->\n                 {secure_renegotiate, SSL#ssl.secure_renegotiate};\n            true ->\n                 false\n         end,\n         if SSL#ssl.client_renegotiation /= undefined ->\n                 {client_renegotiation, SSL#ssl.client_renegotiation};\n            true ->\n                 false\n         end,\n         if SSL#ssl.honor_cipher_order /= undefined ->\n                 {honor_cipher_order, SSL#ssl.honor_cipher_order};\n            true ->\n                 false\n         end,\n         case yaws_dynopts:have_ssl_log_alert() of\n             true  -> {log_alert, false};\n             false -> false\n         end\n        ],\n    [X || X <- L, X /= false].\n\ndo_accept(GS) when GS#gs.ssl == nossl ->\n    ?Debug(\"wait in accept ... ~n\",[]),\n    gen_tcp:accept(GS#gs.l);\ndo_accept(GS) when GS#gs.ssl == ssl ->\n    ssl:transport_accept(GS#gs.l).\n\n\ninitial_acceptor(GS) ->\n    acceptor(GS).\n\n\nacceptor(GS) ->\n    case (GS#gs.gconf)#gconf.process_options of\n        [] ->\n            proc_lib:spawn_link(?MODULE, acceptor0, [GS, self()]);\n        Opts ->\n            %% as we tightly controlled what is set in options, we can\n            %% blindly add \"link\" to get a linked process as per default\n            %% case and use the provided options.\n            proc_lib:spawn_opt(?MODULE, acceptor0, [GS, self()], [link | Opts])\n    end.\nacceptor0(GS, Top) ->\n    ?TC([{record, GS, gs}]),\n    put(gserv_pid, Top),\n    put(gc, GS#gs.gconf),\n    yaws_trace:open(),\n    X = do_accept(GS),\n    Top ! {self(), next, X},\n    case X of\n        {ok, Client} ->\n            if\n                GS#gs.ssl == ssl ->\n                    case ssl:ssl_accept(\n                           Client, (GS#gs.gconf)#gconf.keepalive_timeout\n                          ) of\n                        ok ->\n                            ok;\n                        {error, closed} ->\n                            Top ! {self(), decrement},\n                            exit(normal);\n                        {error, esslaccept} ->\n                            %% Don't log SSL esslaccept to error log since it\n                            %% seems this is what we get on portscans and\n                            %% similar\n                            ?Debug(\"SSL accept failed: ~p~n\", [esslaccept]),\n                            Top ! {self(), decrement},\n                            exit(normal);\n                        {error, Reason} ->\n                            error_logger:format(\"SSL accept failed: ~p~n\",\n                                                [Reason]),\n                            Top ! {self(), decrement},\n                            exit(normal)\n                    end;\n                true ->\n                    ok\n            end,\n            {IP,Port} = peername(Client, GS#gs.ssl),\n            put(trace_filter, yaws_trace:get_filter()),\n            Res = (catch aloop(Client, {IP,Port}, GS,  0)),\n            %% Skip closing the socket, as required by web sockets & stream\n            %% processes.\n            CloseSocket = (get(outh) =:= undefined) orelse\n                          (done_or_continue() =:= done),\n            case CloseSocket of\n                false -> ok;\n                true ->\n                    if\n                        GS#gs.ssl == nossl ->\n                            gen_tcp:close(Client);\n                        GS#gs.ssl == ssl ->\n                            ssl:close(Client)\n                    end\n            end,\n            case Res of\n                {ok, Int} when is_integer(Int) ->\n                    Top ! {self(), done_client, Int};\n                {'EXIT', normal} ->\n                    Top ! {self(), decrement},\n                    exit(normal);\n                {'EXIT', shutdown} ->\n                    exit(shutdown);\n                {'EXIT', {error, einval}} ->\n                    %% Typically clients that close their end of the socket\n                    %% don't log. Happens all the time.\n                    Top ! {self(), decrement},\n                    exit(normal);\n                {'EXIT', {{error, einval}, _}} ->\n                    Top ! {self(), decrement},\n                    exit(normal);\n                {'EXIT', {{badmatch, {error, einval}}, _}} ->\n                    Top ! {self(), decrement},\n                    exit(normal);\n                {'EXIT', {error, closed}} ->\n                    Top ! {self(), decrement},\n                    exit(normal);\n                {'EXIT', {{error, closed}, _}} ->\n                    Top ! {self(), decrement},\n                    exit(normal);\n                {'EXIT', {{error, econnreset},_}} ->\n                    Top ! {self(), decrement},\n                    exit(normal);\n                {'EXIT', Reason2} ->\n                    error_logger:error_msg(\"Yaws process died: ~p~n\",\n                                           [Reason2]),\n                    Top ! {self(), decrement},\n                    exit(shutdown)\n            end,\n\n            %% we cache processes\n            receive\n                {'EXIT', Top, Error} ->\n                    exit(Error);\n                {Top, stop} ->\n                    exit(normal);\n                {Top, accept} ->\n                    erase_transients(),\n                    acceptor0(GS, Top)\n            end;\n        {error, Reason} when ((Reason == timeout) or\n                              (Reason == einval) or\n                              (Reason == normal) or\n                              (Reason == econnaborted)) ->\n            %% The econnaborted is\n            %% caused by recieving a RST when a SYN or SYN+ACK was expected.\n            %% einval is reported to happen, it could be accept attempts\n            %% on just recently reconfigured servers through --hup ... ???\n            Top ! {self(), done_client, 0},\n            receive\n                {Top, stop} ->\n                    exit(normal);\n                {Top, accept} ->\n                    acceptor0(GS, Top)\n            end;\n        {error, closed} ->\n            %% This is what happens when we call yaws --stop\n            Top ! {self(), decrement},\n            exit(normal);\n        {error, Reason} when ((Reason == emfile) or\n                                                   (Reason == enfile)) ->\n            error_logger:format(\"yaws: Failed to accept - no more \"\n                                \"file descriptors - terminating: ~p~n\",\n                                [Reason]),\n            exit(failaccept);\n        ERR ->\n            %% When we fail to accept, the correct thing to do\n            %% is to terminate yaws as an application, if we're running\n            %% yaws as a standalone webserver, we want to restart the\n            %% entire webserver, preferably through heart\n            %% typical errors here are shortage of fds or memory\n            error_logger:format(\"yaws: Failed to accept - terminating: ~p~n\",\n                                [ERR]),\n            exit(failaccept)\n    end.\n\n\n\n%%%----------------------------------------------------------------------\n%%% Internal functions\n%%%----------------------------------------------------------------------\n\naloop(CliSock, {IP,Port}=IPPort, GS, Num) ->\n    case yaws_trace:get_type(GS#gs.gconf) of\n        undefined ->\n            ok;\n        _ when Num =:= 0 ->\n            yaws_trace:write(from_client,\n                             ?F(\"New (~p) connection from ~s:~w~n\",\n                                [GS#gs.ssl,inet_parse:ntoa(IP),Port]));\n        _ ->\n            ok\n    end,\n\n    process_flag(trap_exit, false),\n    init_db(),\n    SSL = GS#gs.ssl,\n    Head = yaws:http_get_headers(CliSock, SSL),\n    process_flag(trap_exit, true),\n    ?Debug(\"Head = ~p~n\", [Head]),\n    case Head of\n        {error, {too_many_headers, ReqTooMany}} ->\n            %% RFC 6585 status code 431\n            ?Debug(\"Request headers too large~n\", []),\n            case pick_sconf(GS#gs.gconf, #headers{}, GS#gs.group) of\n                undefined ->\n                    deliver_400(CliSock, ReqTooMany);\n                SC ->\n                    put(sc, SC),\n                    put(outh, #outh{}),\n                    deliver_431(CliSock, ReqTooMany)\n            end,\n            {ok, Num+1};\n        {Req0, H0} when Req0#http_request.method /= bad_request ->\n            {Req, H} = fix_abs_uri(Req0, H0),\n            ?Debug(\"{Req, H} = ~p~n\", [{Req, H}]),\n            case pick_sconf({GS#gs.ssl, CliSock}, GS#gs.gconf, H, GS#gs.group) of\n                undefined ->\n                    deliver_400(CliSock, Req),\n                    {ok, Num+1};\n                SC ->\n                    put(outh, #outh{}),\n                    put(sc, SC),\n                    DispatchResult = case SC#sconf.dispatch_mod of\n                                         undefined ->\n                                             continue;\n                                         DispatchMod ->\n                                             Arg = make_arg(SC, CliSock, IPPort,\n                                                            H, Req, undefined),\n                                             ok = yaws:setopts(CliSock,\n                                                               [{packet, raw},\n                                                                {active, false}],\n                                                               yaws:is_ssl(SC)),\n                                             DispatchMod:dispatch(Arg)\n                                     end,\n                    case DispatchResult of\n                        done ->\n                            erase_transients(),\n                            case exceed_keepalive_maxuses(GS, Num) of\n                                true  -> {ok, Num+1};\n                                false -> aloop(CliSock, IPPort, GS, Num+1)\n                            end;\n                        closed ->\n                            %% Dispatcher closed the socket\n                            erase_transients(),\n                            {ok, Num+1};\n                        continue ->\n                            ?Debug(\"SC: ~s\", [?format_record(SC, sconf)]),\n                            ?TC([{record, SC, sconf}]),\n                            ?Debug(\"Headers = ~s~n\", [?format_record(H, headers)]),\n                            ?Debug(\"Request = ~s~n\",\n                                   [?format_record(Req, http_request)]),\n                            run_trace_filter(GS, IP, Req, H),\n                            yaws_stats:hit(),\n                            check_keepalive_maxuses(GS, Num),\n                            Call = case yaws_shaper:check(SC, IP) of\n                                       allow ->\n                                           call_method(Req#http_request.method,CliSock,\n                                                       IPPort,Req,H);\n                                       {deny, Status, Msg} ->\n                                           deliver_xxx(CliSock, Req, Status, Msg)\n                                   end,\n                            Call2 = fix_keepalive_maxuses(Call),\n                            handle_method_result(Call2, CliSock, IPPort,\n                                                 GS, Req, H, Num)\n                    end\n            end;\n        closed ->\n            case yaws_trace:get_type(GS#gs.gconf) of\n                undefined -> ok;\n                _         -> yaws_trace:write(from_client, \"closed\\n\")\n            end,\n            {ok, Num};\n        _ ->\n            % not even HTTP traffic\n            exit(normal)\n    end.\n\n\nrun_trace_filter(GS, IP, Req, H) ->\n    case {yaws_trace:get_type(GS#gs.gconf), get(trace_filter)} of\n        {undefined, _} ->\n            ok;\n        {_, undefined} ->\n            RStr = yaws_api:reformat_request(Req),\n            HStr = yaws:headers_to_str(H),\n            yaws_trace:write(from_client, ?F(\"~s~n~s~n\", [RStr, HStr])),\n            ok;\n        {_, FilterFun} ->\n            case FilterFun(inet_parse:ntoa(IP),Req,H) of\n                true  ->\n                    RStr = yaws_api:reformat_request(Req),\n                    HStr = yaws:headers_to_str(H),\n                    yaws_trace:write(from_client, ?F(\"~s~n~s~n\", [RStr, HStr])),\n                    ok;\n                false ->\n                    put(gc, (GS#gs.gconf)#gconf{trace = false})\n            end\n    end.\n\n%% Checks how many times keepalive has been used and updates the\n%% process dictionary outh variable if required to say that the\n%% connection has exceeded its maxuses.\ncheck_keepalive_maxuses(GS, Num) ->\n    Flag = exceed_keepalive_maxuses(GS, Num),\n    put(outh, (get(outh))#outh{exceedmaxuses=Flag}).\n\nexceed_keepalive_maxuses(GS, Num) ->\n    case (GS#gs.gconf)#gconf.keepalive_maxuses of\n        nolimit          -> false;\n        0                -> false;\n        N when Num+1 < N -> false;\n        _N               -> true\n    end.\n\n%% Change to Res to 'done' if we've exceeded our maxuses.\nfix_keepalive_maxuses(Res) ->\n    case Res of\n        continue ->\n            case (get(outh))#outh.exceedmaxuses of\n                true ->\n                    done;  % no keepalive this time!\n                _ ->\n                    Res\n            end;\n        _ ->\n            Res\n    end.\n\n%% keep original dictionary but filter out eventual previous init_db\n%% in erase_transients/0\ninit_db() ->\n    put(init_db, lists:keydelete(init_db, 1, get())).\n\nerase_transients() ->\n    %% flush all messages.\n    %% If exit signal is received from the gserv process, rethrow it\n    Top = get(gserv_pid),\n    Fun = fun(G) -> receive\n                        {'EXIT', Top, Reason} -> exit(Reason);\n                        _X -> G(G)\n                    after 0 -> ok\n                    end\n          end,\n    Fun(Fun),\n    I = get(init_db),\n    if I == undefined ->\n            ok;\n       is_list(I) ->\n            erase(),\n            %% Need to keep init_db in case we do not enter aloop (i.e. init:db)\n            %% again as R12B-5 requires proc_lib keys in dict while exiting...\n            put(init_db, I),\n            lists:foreach(fun({K,V}) -> put(K,V) end, I)\n    end.\n\n\nhandle_method_result(Res, CliSock, {IP,Port}, GS, Req, H, Num) ->\n    case Res of\n        continue ->\n            yaws_shaper:update(get(sc), IP, Req),\n            maybe_access_log(IP, Req, H),\n            erase_transients(),\n            aloop(CliSock, {IP,Port}, GS, Num+1);\n        done ->\n            yaws_shaper:update(get(sc), IP, Req),\n            maybe_access_log(IP, Req, H),\n            erase_transients(),\n            {ok, Num+1};\n        {page, P} ->\n            %% Because the request is rewritten but the body is the same, we\n            %% keep post_parse and erase query_parse.\n            erase(query_parse),\n            put(outh, #outh{}),\n            case P of\n                {Options, Page} ->\n                    %% We got additional headers for the page to deliver.\n                    %%\n                    %% Might be useful for `Vary' or `Content-Location'.\n                    %%\n                    %% These headers are stored to be set later to preserve\n                    %% it during the next loop.\n                    put(page_options, Options);\n                Page ->\n                    ok\n            end,\n            %% `is_reentrant_request' flag is used to correctly identify the url\n            %% type\n            put(is_reentrant_request, true),\n\n            %% Renew #sconf{} to restore docroot/xtra_docroots fields\n            OldSC = get(sc),\n            NewSC = pick_sconf(GS#gs.gconf, H, GS#gs.group),\n            put(sc, NewSC#sconf{appmods = OldSC#sconf.appmods}),\n\n            %% Rewrite the request\n            NextReq = Req#http_request{path = {abs_path, Page}},\n\n            %% Renew #arg{}: keep clidata, state and cont\n            Arg0 = case get(yaws_arg) of\n                       undefined -> #arg{};\n                       A         -> A\n                   end,\n            Arg1 = make_arg(CliSock, {IP,Port}, H, NextReq, Arg0#arg.clidata),\n            Arg2 = Arg1#arg{orig_req = Arg0#arg.orig_req,\n                            cont     = Arg0#arg.cont,\n                            state    = Arg0#arg.state},\n\n\n            %% Get the number of bytes already read and do the reentrant call\n            CliDataPos = case get(client_data_pos) of\n                             undefined -> 0;\n                             Pos       -> Pos\n                         end,\n            Call  = handle_request(CliSock, Arg2, CliDataPos),\n            Call2 = fix_keepalive_maxuses(Call),\n            handle_method_result(Call2, CliSock, {IP,Port}, GS, NextReq, H, Num)\n    end.\n\n\npeername(CliSock, ssl) ->\n    case ssl:peername(CliSock) of\n        {ok, Res} -> Res;\n        _         -> {unknown, unknown}\n    end;\npeername(CliSock, nossl) ->\n    case inet:peername(CliSock) of\n        {ok, Res} -> Res;\n        _         -> {unknown, unknown}\n    end.\n\n\ndeepforeach(_F, []) ->\n    ok;\ndeepforeach(F, [H|T]) ->\n    deepforeach(F, H),\n    deepforeach(F, T);\ndeepforeach(F, X) ->\n    F(X).\n\n\nfix_abs_uri(Req, H) ->\n    case Req#http_request.path of\n        {absoluteURI, _Scheme, Host0, Port, RawPath} ->\n            Host = case Port of\n                       P when is_integer(P) ->\n                           Host0 ++ [$: | integer_to_list(P)];\n                                                % Is this ok?\n                       _ ->\n                           Host0\n                   end,\n            {Req#http_request{path={abs_path, RawPath}},\n             H#headers{host=Host}};\n        _ -> {Req, H}\n    end.\n\n\n%% Case-insensitive compare servername and ignore any optional :Port\n%% postfix. This is performance-sensitive code, so if you change it,\n%% measure it.\ncomp_sname([], []) ->\n    true;\ncomp_sname([$:|_], [$:|_]) ->\n    true;\ncomp_sname([$:|_], []) ->\n    true;\ncomp_sname([], [$:|_]) ->\n    true;\ncomp_sname([$:|_], _) ->\n    false;\ncomp_sname(_, [$:|_]) ->\n    false;\ncomp_sname([], _) ->\n    false;\ncomp_sname(_, []) ->\n    false;\ncomp_sname([C1|T1], [C2|T2]) ->\n    case string:to_lower(C1) == string:to_lower(C2) of\n        true  -> comp_sname(T1, T2);\n        false -> false\n    end.\n\n%% Same thing than comp_sname but here we compare a pattern containing\n%% wildcards:\n%%   - '*' matches any sequence of zero or more characters\n%%   - '?' matches one character unless that character is a period ('.')\nwildcomp_salias([], []) ->\n    true;\nwildcomp_salias([$:|_], [$:|_]) ->\n    true;\nwildcomp_salias([$:|_], []) ->\n    true;\nwildcomp_salias([], [$:|_]) ->\n    true;\nwildcomp_salias([$:|_], _) ->\n    false;\nwildcomp_salias(_, [$:|_]) ->\n    false;\nwildcomp_salias([], _) ->\n    false;\nwildcomp_salias(_, []) ->\n    false;\nwildcomp_salias([$.|_], [$?|_]) ->\n    false;\nwildcomp_salias([_|T1], [$?|T2]) ->\n    wildcomp_salias(T1, T2);\nwildcomp_salias(_, [$*]) ->\n    true;\nwildcomp_salias([_|T1]=Str, [$*|T2]=Pattern) ->\n    case wildcomp_salias(Str, T2) of\n        true  -> true;\n        false -> wildcomp_salias(T1, Pattern)\n    end;\nwildcomp_salias([C1|T1], [C2|T2]) ->\n    case string:to_lower(C1) == string:to_lower(C2) of\n        true  -> wildcomp_salias(T1, T2);\n        false -> false\n    end.\n\ncomp_sni(SniHost, Host) ->\n    case Host of\n        undefined -> false;\n        {_}       -> false;\n        _         -> comp_sname(SniHost, Host)\n    end.\n\npick_sconf({nossl, _}, GC, H, Group) ->\n    pick_sconf(GC, H, Group);\npick_sconf({ssl, _}, #gconf{sni=disable}=GC, H, Group) ->\n    pick_sconf(GC, H, Group);\npick_sconf({ssl, Sock}, GC, H, Group) ->\n    SniHost = unfdefined,\n%case ssl:connection_information(Sock, [sni_hostname]) of\n%                  {ok, [{sni_hostname, SN}]} -> SN;\n%                  _                          -> undefined\n%              end,\n    if\n        SniHost == undefined andalso GC#gconf.sni == strict ->\n            error_logger:format(\n              \"SSL Error: No Hostname was provided via SNI~n\", []\n             ),\n            undefined;\n        SniHost /= undefined ->\n            %% Host header must be defined to SniHost. Multiple Host headers are\n            %% not allowed here.\n            case comp_sni(SniHost, H#headers.host) of\n                true ->\n                    pick_sni_sconf(SniHost, GC, H, Group);\n                false ->\n                    error_logger:format(\n                      \"SSL Error: Hostname ~p provided via SNI and hostname ~p\"\n                      \" provided via HTTP are different~n\",\n                      [SniHost, H#headers.host]\n                     ),\n                    undefined\n            end;\n        true ->\n            pick_sni_sconf(SniHost, GC, H, Group)\n    end.\n\n%% Check is the server is visible without SNI or if the default server is\n%% visible when SNI hostname does not match.\npick_sni_sconf(SniHost, GC, H, Group) ->\n    SC = pick_sconf(GC, H, Group),\n    Flag = (SniHost == undefined orelse get(nomatch_virthost)),\n    if\n        Flag andalso SC /= undefined andalso\n        SC#sconf.ssl /= undefined andalso (SC#sconf.ssl)#ssl.require_sni ->\n            error_logger:format(\"server ~p require a (matching) SNI hostname~n\",\n                                [SC#sconf.servername]),\n            undefined;\n        true ->\n            SC\n    end.\n\npick_sconf(GC, H, Group) ->\n    case H#headers.host of\n        undefined when ?gc_pick_first_virthost_on_nomatch(GC) ->\n            put(nomatch_virthost, true),\n            hd(Group);\n        {[Host|_]} when ?gc_pick_first_virthost_on_nomatch(GC) ->\n            pick_host(GC, Host, Group, Group);\n        {_} ->\n            %% HTTP spec does not allow multiple Host headers\n            undefined;\n        Host ->\n            pick_host(GC, Host, Group, Group)\n    end.\n\n%% Compare Host against [] in case caller sends an empty Host header\npick_host(GC, Host, SCs, Group)\n  when Host == []; Host == undefined; SCs == [] ->\n    if\n        ?gc_pick_first_virthost_on_nomatch(GC) ->\n            put(nomatch_virthost, true),\n            hd(Group);\n        true ->\n            yaws_debug:format(\"Drop req since ~p doesn't match any \"\n                              \"servername \\n\", [Host]),\n            undefined\n    end;\npick_host(GC, Host, [SC|T], Group) ->\n    case comp_sname(Host, SC#sconf.servername) of\n        true  ->\n            SC;\n        false ->\n            Res = lists:any(fun(Alias) -> wildcomp_salias(Host, Alias) end,\n                            SC#sconf.serveralias),\n            case Res of\n                true  -> SC;\n                false -> pick_host(GC, Host, T, Group)\n            end\n    end.\n\nmaybe_auth_log(Item, ARG) ->\n    SC=get(sc),\n    case ?sc_has_auth_log(SC) of\n        false ->\n            ok;\n        true ->\n            Req = ARG#arg.req,\n            {IP,_} = ARG#arg.client_ip_port,\n            Path = safe_path(Req#http_request.path),\n            yaws_log:authlog(SC, IP, Path, Item)\n    end.\n\nmaybe_access_log(Ip, Req, H) ->\n    SC=get(sc),\n    case ?sc_has_access_log(SC) of\n        true ->\n            Time = timer:now_diff(yaws:get_time_tuple(), get(request_start_time)),\n            yaws_log:accesslog(SC, Ip, Req, H, get(outh), Time);\n        false ->\n            ignore\n    end.\n\nsafe_path({abs_path, Path}) -> Path;\nsafe_path(_)                -> \"/undecodable_path\".\n\n\n%% ret:  continue | done\n'GET'(CliSock, IPPort, Req, Head) ->\n    no_body_method(CliSock, IPPort, Req, Head).\n\n\n'POST'(CliSock, IPPort, Req, Head) ->\n    ?Debug(\"POST Req=~s~n H=~s~n\", [?format_record(Req, http_request),\n                                    ?format_record(Head, headers)]),\n    body_method(CliSock, IPPort, Req, Head).\n\n\nun_partial({partial, Bin}) ->\n    Bin;\nun_partial(Bin) ->\n    Bin.\n\n\ncall_method(Method, CliSock, IPPort, Req, H) ->\n    case Method of\n        F when is_atom(F) ->\n            ?MODULE:F(CliSock, IPPort, Req, H);\n        L when is_list(L) ->\n            handle_extension_method(L, CliSock, IPPort, Req, H)\n    end.\n\n\n'HEAD'(CliSock, IPPort, Req, Head) ->\n    put(acc_content, discard),\n    no_body_method(CliSock, IPPort, Req, Head).\n\nnot_implemented(CliSock, _IPPort, Req, Head) ->\n    SC=get(sc),\n    ok = yaws:setopts(CliSock, [{packet, raw}, binary], yaws:is_ssl(SC)),\n    flush(CliSock, Head#headers.content_length,\n          yaws:to_lower(Head#headers.transfer_encoding)),\n    deliver_501(CliSock, Req).\n\n\n'TRACE'(CliSock, IPPort, Req, Head) ->\n    not_implemented(CliSock, IPPort, Req, Head).\n\n'OPTIONS'(CliSock, IPPort, Req, Head) ->\n    case Req#http_request.path of\n        '*' ->\n            % Handle \"*\" as per RFC2616 section 5.1.2\n            deliver_options(CliSock, Req, ['GET', 'HEAD', 'OPTIONS',\n                                           'PUT', 'POST', 'DELETE']);\n        _ ->\n            no_body_method(CliSock, IPPort, Req, Head)\n    end.\n\n'PUT'(CliSock, IPPort, Req, Head) ->\n    ?Debug(\"PUT Req=~p~n H=~p~n\", [?format_record(Req, http_request),\n                                   ?format_record(Head, headers)]),\n    SC=get(sc),\n    case ?sc_has_dav(SC) of\n        true ->\n            %% body is handled by yaws_dav:put/1\n            ok = yaws:setopts(CliSock, [{packet, raw}, binary],\n                              yaws:is_ssl(SC)),\n            ARG = make_arg(CliSock, IPPort, Head, Req, undefined),\n            handle_request(CliSock, ARG, 0);\n        false ->\n            body_method(CliSock, IPPort, Req, Head)\n    end.\n\n'DELETE'(CliSock, IPPort, Req, Head) ->\n    body_method(CliSock, IPPort, Req, Head).\n\n'PATCH'(CliSock, IPPort, Req, Head) ->\n    ?Debug(\"PATCH Req=~p~n H=~p~n\", [?format_record(Req, http_request),\n                                     ?format_record(Head, headers)]),\n    body_method(CliSock, IPPort, Req, Head).\n\nbody_method(CliSock, IPPort, Req, Head) ->\n    SC=get(sc),\n    ok = yaws:setopts(CliSock, [{packet, raw}, binary], yaws:is_ssl(SC)),\n    PPS = SC#sconf.partial_post_size,\n    case yaws_api:get_header(Head, {lower, \"expect\"}) of\n        undefined ->\n            ok;\n        Value ->\n            case yaws:to_lower(Value) of\n                \"100-continue\" -> deliver_100(CliSock);\n                _ -> ok\n            end\n    end,\n    Res = case {yaws:to_lower(Head#headers.transfer_encoding),\n                Head#headers.content_length} of\n              {\"chunked\", _} ->\n                  get_chunked_client_data(CliSock, yaws:is_ssl(SC));\n              {_, undefined} ->\n                  <<>>;\n              {_, Len} ->\n                  Int_len = list_to_integer(Len),\n                  if\n                      Int_len < 0 ->\n                          {error, content_length_overflow};\n                      Int_len == 0 ->\n                          <<>>;\n                      PPS /= nolimit andalso PPS < Int_len ->\n                          {partial, get_client_data(CliSock, PPS, yaws:is_ssl(SC))};\n                      true ->\n                          get_client_data(CliSock, Int_len, yaws:is_ssl(SC))\n                  end\n          end,\n    case Res of\n        {error, Reason} ->\n            error_logger:format(\"Invalid Request: ~p~n\", [Reason]),\n            deliver_400(CliSock, Req);\n        Bin ->\n            ?Debug(\"Request data = ~s~n\", [binary_to_list(un_partial(Bin))]),\n            ARG = make_arg(CliSock, IPPort, Head, Req, Bin),\n            handle_request(CliSock, ARG, size(un_partial(Bin)))\n    end.\n\n\n\nno_body_method(CliSock, IPPort, Req, Head) ->\n    SC=get(sc),\n    ok = yaws:setopts(CliSock, [{packet, raw}, binary], yaws:is_ssl(SC)),\n    flush(CliSock, Head#headers.content_length,\n          yaws:to_lower(Head#headers.transfer_encoding)),\n    Head1 = Head#headers{content_length=undefined, transfer_encoding=undefined},\n    ARG = make_arg(CliSock, IPPort, Head1, Req, undefined),\n    handle_request(CliSock, ARG, 0).\n\n\nmake_arg(CliSock0, IPPort, Head, Req, Bin) ->\n    SC = get(sc),\n    make_arg(SC, CliSock0, IPPort, Head, Req, Bin).\nmake_arg(SC, CliSock0, IPPort, Head, Req, Bin) ->\n    CliSock = case yaws:is_ssl(SC) of\n                  nossl ->\n                      CliSock0;\n                  ssl ->\n                      {ssl, CliSock0}\n              end,\n    ARG = #arg{clisock = CliSock,\n               client_ip_port = IPPort,\n               headers = Head,\n               req = Req,\n               orig_req = Req,\n               opaque = SC#sconf.opaque,\n               pid = self(),\n               docroot = SC#sconf.docroot,\n               docroot_mount = \"/\",\n               clidata = Bin\n              },\n    apply(SC#sconf.arg_rewrite_mod, arg_rewrite, [ARG]).\n\n\n%% PATCH is not an extension method, but at this time the Erlang HTTP\n%% request line parser doesn't know about it so it comes back to us as a\n%% string rather than an atom, which causes call_method to call\n%% handle_extension_method. If and when the parser is updated to accept\n%% PATCH, we'll get it back as an atom and this following clause will be\n%% unnecessary.\nhandle_extension_method(\"PATCH\", CliSock, IPPort, Req, Head) ->\n    'PATCH'(CliSock, IPPort, Req#http_request{method = 'PATCH'}, Head);\nhandle_extension_method(_Method, CliSock, IPPort, Req, Head) ->\n    body_method(CliSock, IPPort, Req, Head).\n\n%% Return values:\n%% continue, done, {page, Page}\n\nhandle_request(CliSock, ARG, _N)\n  when is_record(ARG#arg.state, rewrite_response) ->\n    State = ARG#arg.state,\n    ?Debug(\"SrvReq=~s - RwResp=~s~n\",[?format_record(ARG#arg.req, http_request),\n                                      ?format_record(State, rewrite_response)]),\n    OutH = #outh{status  = State#rewrite_response.status,\n                 chunked = false,\n                 date    = yaws:make_date_header(),\n                 server  = yaws:make_server_header()},\n    put(outh, OutH),\n    deepforeach(fun(X) ->\n                        case X of\n                            {header, H} -> yaws:accumulate_header(H);\n                            _           -> ok\n                        end\n                end, State#rewrite_response.headers),\n    case State#rewrite_response.content of\n        <<>> ->\n            deliver_accumulated(CliSock);\n        _ ->\n            %% Define a default content type if needed\n            case yaws:outh_get_content_type() of\n                undefined ->\n                    Mime = mime_types:default_type(get(sc)),\n                    yaws:outh_set_content_type(Mime);\n                _ ->\n                    ok\n            end,\n            accumulate_content(State#rewrite_response.content),\n            deliver_accumulated(ARG, CliSock, undefined, final)\n    end,\n    done_or_continue();\n\nhandle_request(CliSock, ARG, N) ->\n    Req = ARG#arg.req,\n    ?Debug(\"SrvReq=~s~n\",[?format_record(Req, http_request)]),\n    case Req#http_request.path of\n        {abs_path, RawPath} ->\n            case (catch yaws_api:url_decode_q_split(RawPath)) of\n                {'EXIT', _} ->   %% weird broken cracker requests\n                    deliver_400(CliSock, Req);\n                {DecPath, QueryPart} ->\n                    %% http://<server><port><DecPath>?<QueryPart>\n                    %% DecPath is stored in arg.server_path and is equiv to\n                    %%SCRIPT_PATH + PATH_INFO  (where PATH_INFO may be empty)\n\n                    QueryString = case QueryPart of\n                                      [] ->\n                                          undefined;\n                                      _ ->\n                                          QueryPart\n                                  end,\n\n                    %% by this stage, ARG#arg.docroot_mount is either \"/\" ,\n                    %% or has been set by a rewrite module.\n                    %%!todo - retrieve 'vdir' definitions from main part of\n                    %% config file rather than\n                    %% rely on rewrite module to dig them out of opaque.\n\n                    ARGvdir = ARG#arg.docroot_mount,\n                    %% here we make sure that the conf file, or any rewrite mod\n                    %% wrote  nothing, or something sensible into\n                    %% arg.docroot_mount\n                    %% It must be empty, or of the form \"/path/\" where path\n                    %% may be  further slash-separated.\n                    %%\n                    %%!todo - review - is handle_request\n                    %% (which is presumably performance\n                    %%sensitive) really the place for sanity checks?\n                    %% Presumably this sort of check is trivial enough\n                    %% that it'll have negligible impact.\n\n                    VdirSanity = case ARGvdir of\n                                     \"/\" ->\n                                         sane;\n                                     [$/|_] ->\n                                         case string:right(ARGvdir,1) of\n                                             \"/\" when length(ARGvdir) > 2 ->\n                                                 sane;\n                                             _ ->\n                                                 loopy\n                                         end;\n                                     _ ->\n                                         loopy\n                                 end,\n\n\n                    case VdirSanity of\n                        loopy ->\n                            %%!todo - log somewhere?\n                            error_logger:format(\n                              \"BAD arg.docroot_mount data: '~p'\\n\",[ARGvdir]),\n                            deliver_xxx(CliSock, Req, 500),\n                            exit(normal);\n                        _ ->\n                            ok\n                    end,\n\n\n                    SC = get(sc),\n                    IsRev = is_revproxy(ARG, DecPath, SC),\n                    IsRedirect = is_redirect_map(DecPath,\n                                                 SC#sconf.redirect_map),\n\n                    case {IsRev, IsRedirect} of\n                        {_, {true, Redir}} ->\n                            ARG1 = ARG#arg{server_path = DecPath,\n                                           querydata   = QueryString},\n                            deliver_redirect_map(CliSock, Req, ARG1, Redir, N);\n                        {false, _} ->\n                            %%'main' branch so to speak. Most requests\n                            %% pass through here.\n                            UT = url_type(DecPath, ARG#arg.docroot,\n                                          ARG#arg.docroot_mount),\n\n                            ARG1 = ARG#arg{server_path = DecPath,\n                                           querydata   = QueryString,\n                                           fullpath    = UT#urltype.fullpath,\n                                           prepath     = UT#urltype.dir,\n                                           pathinfo    = UT#urltype.pathinfo},\n                            handle_normal_request(CliSock, ARG1, UT,\n                                                  SC#sconf.authdirs, N);\n                        {{true, PP}, _} ->\n                            UT = #urltype{type = appmod,\n                                          data = {yaws_revproxy, []}},\n                            ARG1 = ARG#arg{server_path = DecPath,\n                                           querydata   = QueryString,\n                                           state       = PP},\n                            handle_normal_request(CliSock, ARG1, UT,\n                                                  SC#sconf.authdirs, N)\n                    end\n            end;\n        {scheme, _Scheme, _RequestString} ->\n            deliver_501(CliSock, Req);\n        _ ->                                    % for completeness\n            deliver_400(CliSock, Req)\n    end.\n\n\nhandle_normal_request(CliSock, ARG, UT = #urltype{type=error}, _, N) ->\n    handle_ut(CliSock, ARG, UT, N);\nhandle_normal_request(CliSock, ARG, UT, Authdirs, N) ->\n    {IsAuth, ARG1} = case is_auth(ARG, Authdirs) of\n                         {true, User} -> {true, set_auth_user(ARG, User)};\n                         E            -> {E, ARG}\n                     end,\n    case IsAuth of\n        true ->\n            %%!todo - remove special treatment of appmod here. (after suitable\n            %% deprecation period) - prepath & pathinfo are applicable to other\n            %% types of dynamic url too replace: appmoddata with pathinfo &\n            %% appmod_prepath with prepath.\n            case UT#urltype.type of\n                appmod ->\n                    {_Mod, PathInfo} = UT#urltype.data,\n                    Appmoddata = case PathInfo of\n                                     undefined ->\n                                         undefined;\n                                     \"/\" ->\n                                         \"/\";\n                                     _ ->\n                                         lists:dropwhile(fun(C) -> C == $/ end,\n                                                         PathInfo)\n                                 end,\n                    ARG2 = ARG1#arg{appmoddata     = Appmoddata,\n                                    appmod_prepath = UT#urltype.dir};\n                _ ->\n                    ARG2 = ARG1\n            end,\n            handle_ut(CliSock, ARG2, UT, N);\n        false_403 ->\n            deliver_403(CliSock, ARG1#arg.orig_req);\n        {false, AuthMethods, Realm} ->\n            UT1 = #urltype{type = {unauthorized, AuthMethods, Realm},\n                           path = ARG1#arg.server_path},\n            handle_ut(CliSock, ARG1, UT1, N)\n    end.\n\n\nset_auth_user(ARG, User) ->\n    H = ARG#arg.headers,\n    Auth =\n        case H#headers.authorization of\n            {undefined, _, _} ->\n                {User, undefined, undefined};\n            {_User, Pass, Orig} ->\n                {User, Pass, Orig};\n            undefined ->\n                {User, undefined, undefined};\n            E ->\n                E\n        end,\n    H2 = H#headers{authorization = Auth},\n    ARG#arg{headers = H2}.\n\n\n\nfilter_auths(Auths, Req_dir) ->\n    case filter_auths(Auths, Req_dir, []) of\n        [] when Req_dir =:= \"/\" orelse Req_dir =:= \".\" ->\n            [];\n        [] ->\n            filter_auths(Auths, filename:dirname(Req_dir));\n        As ->\n            As\n    end.\n\nfilter_auths([], _, Auths) ->\n    lists:reverse(Auths);\nfilter_auths([A=#auth{dir=Req_dir}|T], Req_dir, Auths) ->\n    filter_auths(T, Req_dir, [A|Auths]);\nfilter_auths([_|T], Req_dir, Auths) ->\n    filter_auths(T, Req_dir, Auths).\n\n\n%% Call is_auth(...)/5 with a default value.\nis_auth(#arg{req=Req, orig_req=Req}=ARG, L) ->\n    case lists:keyfind(ARG#arg.docroot, 1, L) of\n        {_, Auths} -> is_req_auth(ARG, Auths, true);\n        false      -> true\n    end;\nis_auth(ARG, L) ->\n    case lists:keyfind(ARG#arg.docroot, 1, L) of\n        {_, Auths} -> is_req_auth(ARG, Auths, is_orig_req_auth(ARG,Auths,true));\n        false      -> true\n    end.\n\nis_orig_req_auth(#arg{orig_req=OrigReq, headers=H}=ARG, Auths, Ret) ->\n    case OrigReq#http_request.path of\n        {abs_path, RawPath} ->\n            case (catch yaws_api:url_decode_q_split(RawPath)) of\n                {'EXIT', _} ->\n                    Ret;\n                {DecPath, _} ->\n                    is_auth(ARG, DecPath, H, filter_auths(Auths, DecPath),\n                            {true, []})\n            end;\n        _ ->\n            Ret\n    end.\n\nis_req_auth(#arg{server_path=Req_dir, headers=H}=ARG, Auths, Ret) ->\n    case is_auth(ARG, Req_dir, H, filter_auths(Auths, Req_dir), {true, []}) of\n        true -> Ret;\n        Else -> Else\n    end.\n\n\n%% Either no authentication was done or all methods returned false\nis_auth(_ARG, _Req_dir, _H, [], {Ret, Auth_headers}) ->\n    yaws:outh_set_auth(Auth_headers),\n    Ret;\n\nis_auth(ARG, Req_dir, H, [Auth_methods|T], {_Ret, Auth_headers}) ->\n    Auth_H = H#headers.authorization,\n    case handle_auth(ARG, Auth_H, Auth_methods, false) of\n                %% If we auth using an authmod we need to return User\n                %% so that we can set it in ARG.\n        {false, A} ->\n            L = A#auth.headers,\n            Auth_methods1 = Auth_methods#auth{realm = A#auth.realm,\n                                              outmod = A#auth.outmod},\n            is_auth(ARG, Req_dir, H, T,\n                    {{false, Auth_methods1, A#auth.realm}, L ++ Auth_headers});\n        Is_auth -> %% true, {true, User} or false_403\n                    Is_auth\n    end.\n\nhandle_auth(#arg{client_ip_port={IP,_}}=ARG, Auth_H,\n            #auth{acl={AllowIPs, DenyIPs, Order}}=Auth_methods, Ret) ->\n    Fun  = fun(IpMask) -> yaws:match_ipmask(IP, IpMask) end,\n    Ret1 = case Auth_methods of\n               #auth{users=[],pam=false,mod=[]} -> true;\n               _                                -> Ret\n           end,\n    case {AllowIPs, DenyIPs, Order} of\n        {_, all, deny_allow} ->\n            case lists:any(Fun, AllowIPs) of\n                true ->\n                    handle_auth(ARG, Auth_H, Auth_methods#auth{acl=none}, Ret1);\n                false ->\n                    false_403\n            end;\n        {all, _, deny_allow} ->\n            handle_auth(ARG, Auth_H, Auth_methods#auth{acl=none}, Ret1);\n        {_, _, deny_allow} ->\n            case lists:any(Fun, DenyIPs) of\n                true ->\n                    case lists:any(Fun, AllowIPs) of\n                        true ->\n                            handle_auth(ARG, Auth_H,\n                                        Auth_methods#auth{acl=none},\n                                        Ret1);\n                        false ->\n                            false_403\n                    end;\n                false ->\n                    handle_auth(ARG, Auth_H, Auth_methods#auth{acl=none}, Ret1)\n            end;\n\n        {_, all, allow_deny} ->\n            false_403;\n        {all, _, allow_deny} ->\n            case lists:any(Fun, DenyIPs) of\n                true ->\n                    false_403;\n                false ->\n                    handle_auth(ARG, Auth_H, Auth_methods#auth{acl=none}, Ret1)\n            end;\n        {_, _, allow_deny} ->\n            case lists:any(Fun, AllowIPs) of\n                true ->\n                    case lists:any(Fun, DenyIPs) of\n                        true ->\n                            false_403;\n                        false ->\n                            handle_auth(ARG, Auth_H,\n                                        Auth_methods#auth{acl=none},\n                                        Ret1)\n                    end;\n                false ->\n                    false_403\n            end\n    end;\n\nhandle_auth(_ARG, _Auth_H, #auth{users=[],pam=false,mod=[]}, true) ->\n    true;\n\nhandle_auth(ARG, _Auth_H, Auth_methods=#auth{users=[],pam=false,mod=[]}, Ret) ->\n    maybe_auth_log({401, Auth_methods#auth.realm}, ARG),\n    {Ret, Auth_methods};\n\nhandle_auth(ARG, Auth_H, Auth_methods = #auth{mod = Mod}, Ret) when Mod /= [] ->\n    case catch Mod:auth(ARG, Auth_methods) of\n        {'EXIT', Reason} ->\n            L = ?F(\"authmod crashed ~n~p:auth(~p, ~n ~p) \\n\"\n                   \"Reason: ~p~n\"\n                   \"Stack: ~p~n\",\n                   [Mod, ARG, Auth_methods, Reason,\n                    erlang:get_stacktrace()]),\n            handle_crash(ARG, L),\n            CliSock = case yaws_api:get_sslsocket(ARG#arg.clisock) of\n                          {ok, SslSock} -> SslSock;\n                          undefined     -> ARG#arg.clisock\n                      end,\n            deliver_accumulated(CliSock),\n            exit(normal);\n\n        %% appmod means the auth headers are undefined, i.e. false.\n        %% TODO: change so that authmods simply return true/false\n        {true, User} ->\n            {true, User};\n        true ->\n            true;\n        false ->\n            handle_auth(ARG, Auth_H, Auth_methods#auth{mod = []},\n                        Ret);\n        {false, Realm} ->\n            handle_auth(ARG, Auth_H, Auth_methods#auth{mod=[], realm=Realm},\n                        Ret);\n        {appmod, Module} ->\n            handle_auth(ARG, Auth_H, Auth_methods#auth{mod=[], outmod=Module},\n                        Ret);\n        _ ->\n            maybe_auth_log(403, ARG),\n            false_403\n    end;\n\n%% if the headers are undefined we do not need to check Pam or Users\nhandle_auth(ARG, undefined, Auth_methods, Ret) ->\n    handle_auth(ARG, undefined, Auth_methods#auth{pam = false, users= []}, Ret);\n\nhandle_auth(ARG, {User, Password, OrigString},\n            Auth_methods = #auth{pam = Pam}, Ret) when Pam /= false ->\n    case yaws_pam:auth(User, Password) of\n        {yes, _} ->\n            maybe_auth_log({ok, User}, ARG),\n            true;\n        {no, _Rsn} ->\n            handle_auth(ARG, {User, Password, OrigString},\n                        Auth_methods#auth{pam = false}, Ret)\n    end;\n\nhandle_auth(ARG, {User, Password, OrigString},\n            Auth_methods = #auth{users = Users}, Ret) when Users /= [] ->\n    F = fun({U, A, S, H}) ->\n                (U == User andalso H == crypto:hash(A, [S,Password]))\n        end,\n    case lists:any(F, Users) of\n        true ->\n            maybe_auth_log({ok, User}, ARG),\n            true;\n        false ->\n            handle_auth(ARG, {User, Password, OrigString},\n                        Auth_methods#auth{users = []}, Ret)\n    end.\n\n\nis_revproxy(ARG, Path, SC = #sconf{revproxy = RevConf}) ->\n    IsFwd = ?sc_forward_proxy(SC),\n    %% Note: these are mututally exclusive.\n    case {IsFwd, RevConf} of\n        {false, []} ->\n            false;\n        {false, _} ->\n            is_revproxy1(Path, RevConf);\n        {true, _} ->\n            {true, #proxy_cfg{prefix=\"/\", url=fwdproxy_url(ARG)}}\n    end.\n\nis_revproxy1(_,[]) ->\n    false;\nis_revproxy1(Path, RevConf) ->\n    case lists:keyfind(Path, #proxy_cfg.prefix, RevConf) of\n        #proxy_cfg{}=R ->\n            {true, R};\n        false when Path =:= \"/\" orelse Path =:= \".\" ->\n            false;\n        false ->\n            is_revproxy1(filename:dirname(Path), RevConf)\n    end.\n\nis_redirect_map(_, []) ->\n    false;\nis_redirect_map(Path, RedirMap) ->\n    case lists:keyfind(Path, 1, RedirMap) of\n        {Path, _Code, _Url, _AppendMod}=E ->\n            {true, E};\n        false when Path =:= \"/\" orelse Path =:= \".\" ->\n            false;\n        false ->\n            is_redirect_map(filename:dirname(Path), RedirMap)\n    end.\n\n%% Find out what which module to call when urltype is unauthorized\n%% Precedence is:\n%% 1. SC#errormod401 if it's not default\n%% 2. outmod if defined\n%% 3. yaws_outmod\n\nget_unauthorized_outmod(_Req_dir, _Auth, Errormod401)\n  when Errormod401 /= yaws_outmod ->\n    Errormod401;\n\nget_unauthorized_outmod(_Req_dir, Auth, Errormod401) ->\n    case Auth#auth.outmod /= [] of\n        true  -> Auth#auth.outmod;\n        false -> Errormod401\n    end.\n\n\n\n%% Return values:\n%% continue, done, {page, Page}\n\nhandle_ut(CliSock, ARG, UT = #urltype{type = regular}, _N) ->\n    Req = ARG#arg.req,\n    H = ARG#arg.headers,\n\n    Regular_allowed   = ['GET', 'HEAD', 'OPTIONS'],\n    IsReentrantRequest = erase(is_reentrant_request),\n    if\n        %% Do not check http method for reentrant requests\n        IsReentrantRequest == true;\n        Req#http_request.method == 'GET';\n        Req#http_request.method == 'HEAD' ->\n            ETag = yaws:make_etag(UT#urltype.finfo),\n            Range = case H#headers.if_range of\n                        [34|_] = Range_etag when Range_etag /= ETag ->\n                            all;\n                        _ ->\n                            requested_range(\n                              H#headers.range,\n                              (UT#urltype.finfo)#file_info.size)\n                    end,\n            case Range of\n                error -> deliver_416(\n                           CliSock, Req,\n                           (UT#urltype.finfo)#file_info.size);\n                _ ->\n                    Do_deliver =\n                        case Req#http_request.method of\n                            'HEAD' -> fun() -> deliver_accumulated(CliSock),\n                                               done end;\n                            _      -> fun() -> deliver_file(CliSock, Req,\n                                                            UT, Range) end\n                        end,\n                    case H#headers.if_none_match of\n                        undefined ->\n                            case H#headers.if_match of\n                                undefined ->\n                                    case H#headers.if_modified_since of\n                                        undefined ->\n                                            yaws:outh_set_static_headers\n                                              (Req, UT, H, Range),\n                                            maybe_set_page_options(),\n                                            Do_deliver();\n                                        UTC_string ->\n                                            case yaws:is_modified_p(\n                                                   UT#urltype.finfo,\n                                                   UTC_string) of\n                                                true ->\n                                                    yaws:outh_set_static_headers\n                                                      (Req, UT, H, Range),\n                                                    maybe_set_page_options(),\n                                                    Do_deliver();\n                                                false ->\n                                                    yaws:outh_set_304_headers(\n                                                      Req, UT, H),\n                                                    maybe_set_page_options(),\n                                                    deliver_accumulated(\n                                                      CliSock),\n                                                    done_or_continue()\n                                            end\n                                    end;\n                                Line ->\n                                    case member(ETag,\n                                                yaws:split_sep(Line, $,)) of\n                                        true ->\n                                            yaws:outh_set_static_headers(\n                                              Req, UT, H, Range),\n                                            maybe_set_page_options(),\n                                            Do_deliver();\n                                        false ->\n                                            deliver_xxx(CliSock, Req, 412)\n                                    end\n                            end;\n                        Line ->\n                            case member(ETag,yaws:split_sep(Line, $,)) of\n                                true ->\n                                    yaws:outh_set_304_headers(Req, UT, H),\n                                    maybe_set_page_options(),\n                                    deliver_accumulated(CliSock),\n                                    done_or_continue();\n                                false ->\n                                    yaws:outh_set_static_headers\n                                      (Req, UT, H, Range),\n                                    Do_deliver()\n                            end\n                    end\n            end;\n        Req#http_request.method == 'OPTIONS' ->\n            deliver_options(CliSock, Req, Regular_allowed);\n        true ->\n            deliver_405(CliSock, Req, Regular_allowed)\n    end;\n\n\nhandle_ut(CliSock, ARG, UT = #urltype{type = yaws}, N) ->\n    Req = ARG#arg.req,\n    H = ARG#arg.headers,\n\n    ?Debug(\"UT = ~s~n\", [?format_record(UT, urltype)]),\n    Yaws_allowed = ['GET', 'POST', 'HEAD', 'OPTIONS'],\n    if\n        Req#http_request.method == 'GET';\n        Req#http_request.method == 'POST';\n        Req#http_request.method == 'HEAD' ->\n            yaws:outh_set_dyn_headers(Req, H, UT),\n            maybe_set_page_options(),\n            do_yaws(CliSock, ARG, UT, N);\n        Req#http_request.method == 'OPTIONS' ->\n            deliver_options(CliSock, Req, Yaws_allowed);\n        true ->\n            deliver_405(CliSock, Req, Yaws_allowed)\n    end;\n\nhandle_ut(CliSock, ARG, UT = #urltype{type = {unauthorized, Auth, Realm}}, N) ->\n    Req = ARG#arg.req,\n    H   = ARG#arg.headers,\n    SC  = get(sc),\n    yaws:outh_set_dyn_headers(Req, H, UT),\n\n    %% outh_set_dyn headers sets status to 200 by default\n    %% so we need to set it 401\n    yaws:outh_set_status_code(401),\n    Outmod = get_unauthorized_outmod(UT#urltype.path, Auth,\n                                     SC#sconf.errormod_401),\n    OutFun = fun (A) ->\n                     case catch Outmod:out401(A, Auth, Realm) of\n                         {'EXIT', {undef, _}} ->\n                             %% Possibly a deprecated warning\n                             Outmod:out(A);\n                         {'EXIT', Reason} ->\n                             exit(Reason);\n                         Result ->\n                             Result\n                     end\n             end,\n    DeliverFun = fun (A) -> finish_up_dyn_file(A, CliSock) end,\n    deliver_dyn_part(CliSock, 0, \"appmod\", N, ARG, UT, OutFun, DeliverFun);\n\nhandle_ut(CliSock, ARG, UT = #urltype{type = error}, N) ->\n    Req = ARG#arg.req,\n    H = ARG#arg.headers,\n    SC=get(sc),GC=get(gc),\n    case UT#urltype.type of\n        error when SC#sconf.xtra_docroots == [] ->\n            yaws:outh_set_dyn_headers(Req, H, UT),\n            deliver_dyn_part(CliSock,\n                             0, \"404\",\n                             N,\n                             ARG,UT,\n                             fun(A)->(SC#sconf.errormod_404):\n                                         out404(A,GC,SC)\n                             end,\n                             fun(A)->finish_up_dyn_file(A, CliSock)\n                             end\n                            );\n        error ->\n            SC2 = SC#sconf{docroot = hd(SC#sconf.xtra_docroots),\n                           xtra_docroots = tl(SC#sconf.xtra_docroots)},\n            put(sc, SC2),\n\n            %%!todo - review & change. rewriting the docroot and xtra_docroots\n            %% is not a good way to handle the xtra_docroot feature because\n            %% it makes less information available to the subsequent calls -\n            %% this is especially an issue for a nested ssi.\n            ARG2 = ARG#arg{docroot = SC2#sconf.docroot},\n            handle_request(CliSock, ARG2, N)\n    end;\n\n\nhandle_ut(CliSock, ARG, UT = #urltype{type = directory}, N) ->\n    Req = ARG#arg.req,\n    H = ARG#arg.headers,\n    SC=get(sc),\n\n    if (?sc_has_dir_listings(SC)) ->\n            Directory_allowed = ['GET', 'HEAD', 'OPTIONS'],\n            IsReentrantRequest = erase(is_reentrant_request),\n            if\n                %% Do not check http method for reentrant requests\n                IsReentrantRequest == true;\n                Req#http_request.method == 'GET';\n                Req#http_request.method == 'HEAD' ->\n                    yaws:outh_set_dyn_headers(Req, H, UT),\n                    maybe_set_page_options(),\n                    P = UT#urltype.fullpath,\n                    yaws_ls:list_directory(ARG, CliSock, UT#urltype.data,\n                                           P, Req,\n                                           ?sc_has_dir_all_zip(SC));\n                Req#http_request.method == 'OPTIONS' ->\n                    deliver_options(CliSock, Req, Directory_allowed);\n                true ->\n                    deliver_405(CliSock, Req, Directory_allowed)\n            end;\n       true ->\n            handle_ut(CliSock, ARG, #urltype{type = error}, N)\n    end;\n\n\nhandle_ut(CliSock, ARG, UT = #urltype{type = redir}, N) ->\n    Req = ARG#arg.req,\n    H = ARG#arg.headers,\n    yaws:outh_set_dyn_headers(Req, H, UT),\n    case yaws:outh_get_doclose() of\n        true  -> ok;\n        _     -> flush(CliSock, N, H#headers.content_length,\n                       yaws:to_lower(H#headers.transfer_encoding))\n    end,\n    deliver_302(CliSock, Req, ARG, UT#urltype.path);\n\nhandle_ut(CliSock, ARG, UT = #urltype{type = appmod}, N) ->\n    Req = ARG#arg.req,\n    H = ARG#arg.headers,\n    yaws:outh_set_dyn_headers(Req, H, UT),\n    maybe_set_page_options(),\n    {Mod,_} = UT#urltype.data,\n    deliver_dyn_part(CliSock,\n                     0, \"appmod\",\n                     N,\n                     ARG,UT,\n                     fun(A)->Mod:out(A) end,\n                     fun(A)->finish_up_dyn_file(A, CliSock) end\n                    );\n\n\nhandle_ut(CliSock, ARG, UT = #urltype{type = cgi}, N) ->\n    Req = ARG#arg.req,\n    H = ARG#arg.headers,\n    yaws:outh_set_dyn_headers(Req, H, UT),\n    maybe_set_page_options(),\n    deliver_dyn_part(CliSock,\n                     0, \"cgi\",\n                     N,\n                     ARG,UT,\n                     fun(A)->yaws_cgi:call_cgi(\n                               A,flatten(UT#urltype.fullpath))\n                     end,\n                     fun(A)->finish_up_dyn_file(A, CliSock) end\n                    );\n\nhandle_ut(CliSock, ARG, UT = #urltype{type = fcgi}, N) ->\n    error_logger:error_msg(\"*** handle_ut: type=fcgi~n\"),    %%@@@\n    Req = ARG#arg.req,\n    H = ARG#arg.headers,\n    yaws:outh_set_dyn_headers(Req, H, UT),\n    maybe_set_page_options(),\n    deliver_dyn_part(CliSock,\n                     0, \"fcgi\",\n                     N,\n                     ARG,UT,\n                     fun(A)->yaws_cgi:call_fcgi_responder(A)\n                     end,\n                     fun(A)->finish_up_dyn_file(A, CliSock)\n                     end\n                    );\n\nhandle_ut(CliSock, ARG, UT = #urltype{type = dav}, N) ->\n    Req = ARG#arg.req,\n    H = ARG#arg.headers,\n    Next =\n        case Req#http_request.method of\n            'OPTIONS' ->\n                options;\n            _ when Req#http_request.method == 'GET';\n                   Req#http_request.method == 'HEAD' ->\n                case prim_file:read_file_info(UT#urltype.fullpath) of\n                    {ok, FI} when FI#file_info.type == regular ->\n                        {regular, FI};\n                    _ ->\n                        error\n                end;\n            _Dav when Req#http_request.method == 'PUT';\n                      Req#http_request.method == 'DELETE';\n                      Req#http_request.method == \"PROPFIND\";\n                      Req#http_request.method == \"PROPPATCH\";\n                      Req#http_request.method == \"LOCK\";\n                      Req#http_request.method == \"UNLOCK\";\n                      Req#http_request.method == \"MOVE\";\n                      Req#http_request.method == \"COPY\";\n                      Req#http_request.method == \"MKCOL\" ->\n                dav;\n            _ ->\n                error\n        end,\n    case Next of\n        error ->\n            handle_ut(CliSock, ARG, #urltype{type = error}, N);\n        options ->\n            deliver_options(CliSock, Req, []);\n        {regular, Finfo} ->\n            handle_ut(CliSock, ARG, UT#urltype{type = regular,\n                                               finfo = Finfo}, N);\n        dav ->\n            yaws:outh_set_dyn_headers(Req, H, UT),\n            maybe_set_page_options(),\n            deliver_dyn_part(CliSock,\n                             0, \"dav\",\n                             N,\n                             ARG,UT,\n                             Next,\n                             fun(A) -> finish_up_dyn_file(A, CliSock) end\n                            )\n    end;\n\nhandle_ut(CliSock, ARG, UT = #urltype{type = php}, N) ->\n    Req = ARG#arg.req,\n    H = ARG#arg.headers,\n    SC=get(sc),\n    yaws:outh_set_dyn_headers(Req, H, UT),\n    maybe_set_page_options(),\n    Fun = case SC#sconf.php_handler of\n              {cgi, Exe} ->\n                  fun(A)->\n                          yaws_cgi:call_cgi(\n                            A,Exe,flatten(UT#urltype.fullpath)\n                           )\n                  end;\n              {fcgi, {PhpFcgiHost, PhpFcgiPort}} ->\n                  fun(A)->\n                          yaws_cgi:call_fcgi_responder(\n                            A, [{app_server_host, PhpFcgiHost},\n                                {app_server_port, PhpFcgiPort}]\n                           )\n                  end;\n              {extern, {PhpMod, PhpFun}} ->\n                  fun(A) ->\n                          PhpMod:PhpFun(A)\n                  end;\n              {extern, {PhpNode,PhpMod,PhpFun}} ->\n                  fun(A) ->\n                          %% Mod:Fun must return\n                          rpc:call(PhpNode, PhpMod, PhpFun, [A], infinity)\n                  end\n          end,\n    deliver_dyn_part(CliSock,\n                     0, \"php\",\n                     N,\n                     ARG,UT,\n                     Fun,\n                     fun(A)->finish_up_dyn_file(A, CliSock) end\n                    ).\n\ndone_or_continue() ->\n    case yaws:outh_get_doclose() of\n        true -> done;\n        false -> continue;\n        keep_alive -> continue;\n        undefined -> continue\n    end.\n\n%% we may have content,\n\nnew_redir_h(OH, Loc) ->\n    new_redir_h(OH, Loc, 302).\n\nnew_redir_h(OH, Loc, Status) ->\n    OH2 = OH#outh{status = Status,\n                  location = Loc},\n    put(outh, OH2).\n\n\n%% we must deliver a 302 if the browser asks for a dir\n%% without a trailing / in the HTTP req\n%% otherwise the relative urls in /dir/index.html will be broken.\n%% Note: Here Path is always decoded, so we must encode it\ndeliver_302(CliSock, _Req, Arg, Path) ->\n    ?Debug(\"in redir 302 \",[]),\n    H = get(outh),\n    SC=get(sc),\n    Scheme  = yaws:redirect_scheme(SC),\n    Headers = Arg#arg.headers,\n    EncPath = yaws_api:url_encode(Path),\n    RedirHost = yaws:redirect_host(SC, Headers#headers.host),\n\n    %% QueryString must be added\n    Loc = case Arg#arg.querydata of\n              undefined -> [\"Location: \", Scheme, RedirHost, EncPath, \"\\r\\n\"];\n              [] -> [\"Location: \", Scheme, RedirHost, EncPath, \"\\r\\n\"];\n              Q -> [\"Location: \", Scheme, RedirHost, EncPath, \"?\", Q, \"\\r\\n\"]\n          end,\n    new_redir_h(H, Loc),\n    deliver_accumulated(CliSock),\n    done_or_continue().\n\n\ndeliver_redirect_map(CliSock, Req, _Arg,\n                     {_Prefix, Code, undefined, _Mode}, _N) ->\n    %% Here Code is 1xx, 2xx, 4xx or 5xx\n    ?Debug(\"in redir ~p\", [Code]),\n    deliver_xxx(CliSock, Req, Code);\ndeliver_redirect_map(_CliSock, _Req, Arg,\n                     {_Prefix, Code, Path, Mode}, N) when is_list(Path) ->\n    %% Here Code is 1xx, 2xx, 4xx or 5xx\n    ?Debug(\"in redir ~p\", [Code]),\n    Path1 = if\n                Mode == append ->\n                    EncPath = yaws_api:url_encode(Arg#arg.server_path),\n                    filename:join([Path ++ EncPath]);\n                true -> %% noappend\n                    Path\n            end,\n    Page = case Arg#arg.querydata of\n               undefined -> Path1;\n               []        -> Path1;\n               Q         -> Path1 ++ \"?\" ++ Q\n           end,\n\n    %% Set variables used in handle_method_result/7\n    put(yaws_arg, Arg),\n    put(client_data_pos, N),\n    {page, {[{status, Code}], Page}};\ndeliver_redirect_map(CliSock, Req, Arg,\n                     {_Prefix, Code, URL, Mode}, N) when is_record(URL, url) ->\n    %% Here Code is 3xx\n    ?Debug(\"in redir ~p\", [Code]),\n    H = get(outh),\n    QueryData = case Arg#arg.querydata of\n                    undefined -> [];\n                    Q         -> Q\n                end,\n    LocPath = if\n                  Mode == append ->\n                      EncPath = yaws_api:url_encode(Arg#arg.server_path),\n                      Path1   = filename:join([URL#url.path ++ EncPath]),\n                      yaws_api:format_partial_url(\n                        URL#url{path=Path1,querypart=QueryData}, get(sc)\n                       );\n                  true -> %% noappend\n                      yaws_api:format_partial_url(URL#url{querypart=QueryData},\n                                                  get(sc))\n              end,\n    Loc = [\"Location: \", LocPath, \"\\r\\n\"],\n\n    {DoClose, _Chunked} = yaws:dcc(Req, Arg#arg.headers),\n    case DoClose of\n        true  -> ok;\n        _     -> flush(CliSock, N, (Arg#arg.headers)#headers.content_length,\n                       yaws:to_lower((Arg#arg.headers)#headers.transfer_encoding))\n    end,\n    new_redir_h(H#outh{\n                  connection = yaws:make_connection_close_header(DoClose),\n                  doclose    = DoClose,\n                  server     = yaws:make_server_header(),\n                  chunked    = false,\n                  date       = yaws:make_date_header()\n                 }, Loc, Code),\n    deliver_accumulated(CliSock),\n    done_or_continue().\n\n\ndeliver_options(CliSock, _Req, Options) ->\n    H = #outh{status = 200,\n              doclose = false,\n              chunked = false,\n              server = yaws:make_server_header(),\n              date = yaws:make_date_header(),\n              allow = yaws:make_allow_header(Options)},\n    put(outh, H),\n    deliver_accumulated(CliSock),\n    continue.\n\ndeliver_100(CliSock) ->\n    H = #outh{status = 100,\n              doclose = false,\n              chunked = false,\n              server = yaws:make_server_header(),\n              allow = yaws:make_allow_header()},\n    put(outh, H),\n    deliver_accumulated(CliSock),\n    continue.\n\n\ndeliver_xxx(CliSock, _Req, Code) ->\n    deliver_xxx(CliSock, _Req, Code, \"\").\ndeliver_xxx(CliSock, _Req, Code, ExtraHtml) ->\n    B = [\"<html><h1>\", integer_to_list(Code), $\\ ,\n         yaws_api:code_to_phrase(Code), \"</h1></html>\", ExtraHtml],\n    Sz = iolist_size(B),\n    Server = case get(sc) of\n                 undefined -> undefined;\n                 _ -> yaws:make_server_header()\n             end,\n    H = #outh{status = Code,\n              doclose = true,\n              chunked = false,\n              server = Server,\n              connection = yaws:make_connection_close_header(true),\n              content_length = yaws:make_content_length_header(Sz),\n              contlen = Sz,\n              content_type = yaws:make_content_type_header(\"text/html\")},\n    put(outh, H),\n    accumulate_content(B),\n    deliver_accumulated(CliSock),\n    done.\n\ndeliver_400(CliSock, Req) ->\n    deliver_xxx(CliSock, Req, 400).% Bad Request\n\ndeliver_403(CliSock, Req) ->\n    deliver_xxx(CliSock, Req, 403).        % Forbidden\n\ndeliver_405(CliSock, Req, Methods) ->\n    Methods_msg = lists:flatten(\n                    [\"<p>This resource allows \",\n                     yaws:join_sep([atom_to_list(M) || M <- Methods], \", \"),\n                     \"</p>\"]),\n    deliver_xxx(CliSock, Req, 405, Methods_msg).\n\ndeliver_416(CliSock, _Req, Tot) ->\n    B = [\"<html><h1>416 \", yaws_api:code_to_phrase(416), \"</h1></html>\"],\n    Sz = iolist_size(B),\n    H = #outh{status = 416,\n              doclose = true,\n              chunked = false,\n              server = yaws:make_server_header(),\n              connection = yaws:make_connection_close_header(true),\n              content_range = [\"Content-Range: */\",\n                               integer_to_list(Tot), $\\r, $\\n],\n              content_length = yaws:make_content_length_header(Sz),\n              contlen = Sz,\n              content_type = yaws:make_content_type_header(\"text/html\")},\n    put(outh, H),\n    accumulate_content(B),\n    deliver_accumulated(CliSock),\n    done.\n\ndeliver_431(CliSock, Req) ->\n    deliver_xxx(CliSock, Req, 431).\n\ndeliver_501(CliSock, Req) ->\n    deliver_xxx(CliSock, Req, 501). % Not implemented\n\ndo_yaws(CliSock, ARG, UT, N) ->\n    Key = UT#urltype.getpath, %% always flat\n    Mtime = mtime(UT#urltype.finfo),\n    SC=get(sc),\n    case ets:lookup(SC#sconf.ets, Key) of\n        [{_Key, spec, Mtime1, Spec, Es}] when Mtime1 == Mtime,\n                                              Es == 0 ->\n            deliver_dyn_file(CliSock, Spec, ARG, UT, N);\n        Other  ->\n            purge_old_mods(get(gc),Other),\n            {ok, NbErrs, Spec} = yaws_compile:compile_file(UT#urltype.fullpath),\n            ?Debug(\"Spec for file ~s is:~n~p~n\",[UT#urltype.fullpath, Spec]),\n            ets:insert(SC#sconf.ets, {Key, spec, Mtime, Spec, NbErrs}),\n            deliver_dyn_file(CliSock, Spec, ARG, UT, N)\n    end.\n\n\npurge_old_mods(_, []) ->\n    ok;\npurge_old_mods(_GC, [{_FileAtom, spec, _Mtime1, Spec, _}]) ->\n    foreach(\n      fun({mod, _, _, _,  Mod, _Func}) ->\n              code:purge(Mod),\n              code:purge(Mod);\n         (_) ->\n              ok\n      end, Spec).\n\n\nget_client_data(CliSock, Len, SSlBool) ->\n    get_client_data(CliSock, Len, [], SSlBool).\n\nget_client_data(_CliSock, 0, Bs, _SSlBool) ->\n    list_to_binary(Bs);\nget_client_data(CliSock, Len, Bs, SSlBool) ->\n    case yaws:cli_recv(CliSock, Len, SSlBool) of\n        {ok, B} ->\n            get_client_data(CliSock, Len-size(B), [Bs,B], SSlBool);\n        _Other ->\n            error_logger:format(\"get_client_data: ~p~n\", [_Other]),\n            exit(normal)\n    end.\n\n%% not nice to support this for ssl sockets\nget_chunked_client_data(CliSock,SSL) ->\n    SC  = get(sc),\n    Val = erase(current_chunk_size),\n    Len = if\n              Val =:= 0 ->\n                  %% Last chunk was already read.\n                  undefined;\n              Val =:= undefined ->\n                  yaws:setopts(CliSock, [binary, {packet, line}],SSL),\n                  %% Ignore chunk extentions\n                  {N, _Exts} = yaws:get_chunk_header(CliSock, SSL),\n                  yaws:setopts(CliSock, [binary, {packet, raw}],SSL),\n                  N;\n              true ->\n                  Val\n          end,\n    if\n        Len =:= undefined ->\n            %% Do nothing\n            put(current_chunk_size, 0),\n            <<>>;\n        Len == 0 ->\n            put(current_chunk_size, 0),\n            %% Ignore chunk trailer\n            yaws:get_chunk_trailer(CliSock, SSL),\n            <<>>;\n        Len =< SC#sconf.partial_post_size ->\n            B = yaws:get_chunk(CliSock, Len, 0, SSL),\n            yaws:eat_crnl(CliSock,SSL),\n            {partial, list_to_binary(B)};\n        true ->\n            B = yaws:get_chunk(CliSock, SC#sconf.partial_post_size, 0, SSL),\n            put(current_chunk_size, Len - SC#sconf.partial_post_size),\n            {partial, list_to_binary(B)}\n    end.\n\n%% Return values:\n%% continue, done, {page, Page}\n\ndeliver_dyn_part(CliSock,                       % essential params\n                 LineNo, YawsFile,              % for diagnostic output\n                 CliDataPos0,                   % for `get_more' and `flush'\n                 Arg,UT,\n                 YawsFun,                       % call YawsFun(Arg)\n                 DeliverCont                    % call DeliverCont(Arg)\n                                                % to continue normally\n                ) ->\n    %% Note: yaws_arg and client_data_pos are also used in\n    %% handle_method_result/7 when `{page, Page}' is returned\n    put(yaws_ut, UT),\n    put(yaws_arg, Arg),\n    put(client_data_pos, CliDataPos0),\n    OutReply = try\n                   Res = YawsFun(Arg),\n                   handle_out_reply(Res, LineNo, YawsFile, UT, Arg)\n               catch\n                   Class:Exc ->\n                       handle_out_reply({throw, Class, Exc}, LineNo,\n                                        YawsFile, UT, Arg)\n               end,\n    case OutReply of\n        {get_more, Cont, State} when element(1, Arg#arg.clidata) == partial  ->\n            CliDataPos1 = get(client_data_pos),\n            More = get_more_post_data(CliSock, CliDataPos1, Arg),\n            A2 = Arg#arg{clidata=More, cont=Cont, state=State},\n            deliver_dyn_part(\n              CliSock, LineNo, YawsFile, CliDataPos1+size(un_partial(More)),\n              A2, UT, YawsFun, DeliverCont\n             );\n        break ->\n            finish_up_dyn_file(Arg, CliSock);\n        {page, Page} ->\n            {page, Page};\n        Arg2 = #arg{} ->\n            DeliverCont(Arg2);\n        {streamcontent, _, _} ->\n            Priv = deliver_accumulated(Arg, CliSock, undefined, stream),\n            stream_loop_send(Priv, CliSock, 30000);\n        %% For other timeout values (other than 30 second)\n        {streamcontent_with_timeout, _, _, TimeOut} ->\n            Priv = deliver_accumulated(Arg, CliSock, undefined, stream),\n            stream_loop_send(Priv, CliSock, TimeOut);\n        {streamcontent_with_size, Sz, _, _} ->\n            Priv = deliver_accumulated(Arg, CliSock, Sz, stream),\n            stream_loop_send(Priv, CliSock, 30000);\n        {streamcontent_from_pid, _, Pid} ->\n            case yaws:outh_get_content_encoding() of\n                decide -> yaws:outh_set_content_encoding(identity);\n                _      -> ok\n            end,\n            Priv = deliver_accumulated(Arg, CliSock, undefined, stream),\n            wait_for_streamcontent_pid(Priv, CliSock, Pid);\n        {websocket, CallbackMod, Opts} ->\n            %% The handshake passes control over the socket to OwnerPid\n            %% and terminates the Yaws worker!\n            yaws_websockets:start(Arg, CallbackMod, Opts);\n        _ ->\n            DeliverCont(Arg)\n    end.\n\nfinish_up_dyn_file(Arg, CliSock) ->\n    deliver_accumulated(Arg, CliSock, undefined, final),\n    done_or_continue().\n\n\n\n%% do the header and continue\ndeliver_dyn_file(CliSock, Specs, ARG, UT, N) ->\n    Bin = ut_read(UT),\n    deliver_dyn_file(CliSock, Bin, Specs, ARG, UT, N).\n\ndeliver_dyn_file(CliSock, Bin, [H|T], Arg, UT, N) ->\n    ?Debug(\"deliver_dyn_file: ~p~n\", [H]),\n    case H of\n        {mod, LineNo, YawsFile, NumChars, Mod, out} ->\n            {_, Bin2} = skip_data(Bin, NumChars),\n            deliver_dyn_part(CliSock, LineNo, YawsFile,\n                             N, Arg, UT,\n                             fun(A)->Mod:out(A) end,\n                             fun(A)->deliver_dyn_file(CliSock,Bin2,T,A,UT,0)\n                             end);\n        {data, 0} ->\n            deliver_dyn_file(CliSock, Bin, T, Arg, UT, N);\n        {data, NumChars} ->\n            {Send, Bin2} = skip_data(Bin, NumChars),\n            accumulate_content(Send),\n            deliver_dyn_file(CliSock, Bin2, T, Arg, UT, N);\n        {skip, 0} ->\n            deliver_dyn_file(CliSock, Bin, T, Arg, UT, N);\n        {skip, NumChars} ->\n            {_, Bin2} = skip_data(Bin, NumChars),\n            deliver_dyn_file(CliSock, Bin2, T, Arg, UT, N);\n        {binding, NumChars} ->\n            {Send, Bin2} = skip_data(Bin, NumChars),\n            \"%%\"++Key = binary_to_list(Send),\n            Chunk =\n                case get({binding, Key--\"%%\"}) of\n                    undefined -> Send;\n                    Value -> Value\n                end,\n            accumulate_content(Chunk),\n            deliver_dyn_file(CliSock, Bin2, T, Arg, UT, N);\n        {error, NumChars, Str} ->\n            {_, Bin2} = skip_data(Bin, NumChars),\n            accumulate_content(Str),\n            deliver_dyn_file(CliSock, Bin2, T, Arg, UT, N);\n        {verbatim, NumChars, Data} ->\n            {_Send, Bin2} = skip_data(Bin, NumChars),\n            accumulate_content(Data),\n            deliver_dyn_file(CliSock, Bin2, T, Arg, UT, N);\n        yssi ->\n            ok\n    end;\ndeliver_dyn_file(CliSock, _Bin, [], ARG,_UT,_N) ->\n    ?Debug(\"deliver_dyn: done~n\", []),\n    finish_up_dyn_file(ARG, CliSock).\n\n\n-define(unflushed_timeout, 300).\n\nstream_loop_send(Priv, CliSock, IdleTimeout) ->\n    stream_loop_send(Priv, CliSock, unflushed, ?unflushed_timeout, IdleTimeout).\n\nstream_loop_send(Priv, CliSock, FlushStatus, CurTimeout, IdleTimeout) ->\n    receive\n        {streamcontent, Cont} ->\n            P = send_streamcontent_chunk(Priv, CliSock, Cont),\n            stream_loop_send(P, CliSock, unflushed,\n                             ?unflushed_timeout, IdleTimeout);\n        {streamcontent_with_ack, From, Cont} -> % acknowledge after send\n            P = send_streamcontent_chunk(Priv, CliSock, Cont),\n            From ! {self(), streamcontent_ack},\n            stream_loop_send(P, CliSock, unflushed,\n                             ?unflushed_timeout, IdleTimeout);\n        endofstreamcontent ->\n            end_streaming(Priv, CliSock)\n    after CurTimeout ->\n            case FlushStatus of\n                flushed ->\n                    erlang:error(stream_timeout);\n                unflushed ->\n                    P = sync_streamcontent(Priv, CliSock),\n                    stream_loop_send(P, CliSock, flushed,\n                                     IdleTimeout, IdleTimeout)\n            end\n    end.\n\nmake_chunk(Data) ->\n    case yaws:outh_get_chunked() of\n        true ->\n            case iolist_size(Data) of\n                0 ->\n                    empty;\n                S ->\n                    CRNL = crnl(),\n                    {S, [yaws:integer_to_hex(S), CRNL, Data, CRNL]}\n            end;\n        false ->\n            {iolist_size(Data), Data}\n    end.\n\nmake_final_chunk(Data) ->\n    case yaws:outh_get_chunked() of\n        true ->\n            CRNL = crnl(),\n            case iolist_size(Data) of\n                0 ->\n                    {0, [\"0\",CRNL,CRNL]};\n                S ->\n                    {S, [yaws:integer_to_hex(S), CRNL, Data, CRNL,\n                         \"0\", CRNL, CRNL]}\n            end;\n        false ->\n            {iolist_size(Data), Data}\n    end.\n\nsend_streamcontent_chunk(discard, _, _) ->\n    discard;\nsend_streamcontent_chunk(undefined, CliSock, Data) ->\n    case make_chunk(Data) of\n        empty -> ok;\n        {Size, Chunk} ->\n            ?Debug(\"send ~p bytes to ~p ~n\",\n                   [Size, CliSock]),\n            yaws:outh_inc_act_contlen(Size),\n            yaws:gen_tcp_send(CliSock, Chunk)\n    end,\n    undefined;\nsend_streamcontent_chunk({Z, Priv}, CliSock, Data) ->\n    ?Debug(\"send ~p bytes to ~p ~n\",\n           [iolist_size(Data), CliSock]),\n    {ok, P, D} = yaws_zlib:gzipDeflate(Z, Priv, iolist_to_binary(Data), none),\n    case make_chunk(D) of\n        empty -> ok;\n        {Size, Chunk} ->\n            yaws:outh_inc_act_contlen(Size),\n            yaws:gen_tcp_send(CliSock, Chunk)\n    end,\n    {Z, P}.\n\n\nsync_streamcontent(discard, _CliSock) ->\n    discard;\nsync_streamcontent(undefined, _CliSock) ->\n    undefined;\nsync_streamcontent({Z, Priv}, CliSock) ->\n    ?Debug(\"syncing~n\", []),\n    {ok, P, D} = yaws_zlib:gzipDeflate(Z, Priv, <<>>, sync),\n    case make_chunk(D) of\n        empty -> ok;\n        {Size, Chunk} ->\n            yaws:outh_inc_act_contlen(Size),\n            yaws:gen_tcp_send(CliSock, Chunk)\n    end,\n    {Z, P}.\n\n\nend_streaming(discard, _) ->\n    done_or_continue();\nend_streaming(undefined, CliSock) ->\n    ?Debug(\"end_streaming~n\", []),\n    {_, Chunk} = make_final_chunk(<<>>),\n    yaws:gen_tcp_send(CliSock, Chunk),\n    done_or_continue();\nend_streaming({Z, Priv}, CliSock) ->\n    ?Debug(\"end_streaming~n\", []),\n    {ok, _P, Data} = yaws_zlib:gzipDeflate(Z, Priv, <<>>, finish),\n    {Size, Chunk} = make_final_chunk(Data),\n    yaws:outh_inc_act_contlen(Size),\n    yaws:gen_tcp_send(CliSock, Chunk),\n    yaws_zlib:gzipEnd(Z),\n    zlib:close(Z),\n    done_or_continue().\n\n\n%% what about trailers ??\n%% vinoski -- I think trailers should be added as an optional argument to\n%% yaws_api:stream_chunk_end(). The end_streaming() function above could\n%% then easily deal with sending them.\n\nwait_for_streamcontent_pid(Priv, CliSock, ContentPid) ->\n    Ref = erlang:monitor(process, ContentPid),\n    case Priv of\n        discard ->\n            ContentPid ! {discard, self()};\n        _ ->\n            SC = get(sc),\n            case SC#sconf.ssl of\n                undefined ->\n                    gen_tcp:controlling_process(CliSock, ContentPid);\n                _ ->\n                    ssl:controlling_process(CliSock, ContentPid)\n            end,\n            ContentPid ! {ok, self()}\n    end,\n    receive\n        endofstreamcontent ->\n            demonitor_streamcontent_pid(Ref);\n        {endofstreamcontent, closed} ->\n            H = get(outh),\n            put(outh, H#outh{doclose = true}),\n            demonitor_streamcontent_pid(Ref);\n        {'DOWN', Ref, _, _, _} ->\n            ok\n    end,\n    done_or_continue().\n\ndemonitor_streamcontent_pid(Ref) ->\n    erlang:demonitor(Ref),\n    %% should just use demonitor [flush] option instead?\n    receive\n        {'DOWN', Ref, _, _, _} ->\n            ok\n    after 0 ->\n            ok\n    end.\n\nskip_data(Bin, Sz) ->\n    ?Debug(\"Skip data ~p bytes from\", [Sz]),\n    <<Head:Sz/binary, Tail/binary>> = Bin,\n    {Head, Tail}.\n\naccumulate_content(Data) ->\n    case get(acc_content) of\n        undefined ->\n            put(acc_content, [Data]);\n        discard ->\n            discard;\n        List ->\n            put(acc_content, [List, Data])\n    end.\n\n\n%% handle_out_reply(R, ...)\n%%\n%% R is a reply or a deep list of replies.  The special return values\n%% `streamcontent', `get_more_data' etc, which are not handled here\n%% completely but returned, have to be the last element of the list.\n\nhandle_out_reply(L, LineNo, YawsFile, UT, ARG) when is_list (L) ->\n    handle_out_reply_l(L, LineNo, YawsFile, UT, ARG, undefined);\n\n\n\n\n%% yssi, yaws include\nhandle_out_reply({yssi, Yfile}, LineNo, YawsFile, UT, ARG) ->\n    SC = get(sc),\n\n    %% special case for abs paths\n    UT2=case Yfile of\n            [$/|_] ->\n                url_type( Yfile, ARG#arg.docroot, ARG#arg.docroot_mount);\n            _Else ->\n                %%why lists:flatten? is urltype.dir ever nested more than\n                %% 1 level deep?\n                %%!todo - replace with conc_path if 1 level - or specify that\n                %% urltype fields should be written flat!\n                %% All this deep listing of relatively *short* strings\n                %seems unwieldy. just how much performance can it gain if we\n                %% end up using slower funcs like lists:flatten anyway?\n                %% review!.\n                url_type(lists:flatten(UT#urltype.dir) ++ [$/|Yfile],\n                         ARG#arg.docroot, ARG#arg.docroot_mount)\n        end,\n\n    case UT2#urltype.type of\n        yaws ->\n            Mtime = mtime(UT2#urltype.finfo),\n            Key = UT2#urltype.getpath,\n            CliSock = case yaws_api:get_sslsocket(ARG#arg.clisock) of\n                          {ok, SslSock} -> SslSock;\n                          undefined     -> ARG#arg.clisock\n                      end,\n            N = 0,\n            case ets:lookup(SC#sconf.ets, Key) of\n                [{_Key, spec, Mtime1, Spec, Es}] when Mtime1 == Mtime,\n                                                      Es == 0 ->\n                    deliver_dyn_file(CliSock, Spec ++ [yssi], ARG, UT2, N);\n                Other  ->\n                    purge_old_mods(get(gc), Other),\n                    {ok, NbErrs, Spec} =\n                        yaws_compile:compile_file(UT2#urltype.fullpath),\n                    ?Debug(\"Spec for file ~s is:~n~p~n\",\n                           [UT2#urltype.fullpath, Spec]),\n                    ets:insert(SC#sconf.ets, {Key, spec, Mtime, Spec, NbErrs}),\n                    deliver_dyn_file(CliSock, Spec ++ [yssi], ARG, UT2, N)\n            end;\n        error when SC#sconf.xtra_docroots /= [] ->\n            SC2 = SC#sconf{docroot = hd(SC#sconf.xtra_docroots),\n                           xtra_docroots = tl(SC#sconf.xtra_docroots)},\n            put(sc, SC2), ARG2 = ARG#arg{docroot = SC2#sconf.docroot},\n            Ret = handle_out_reply({yssi, Yfile}, LineNo, YawsFile, UT, ARG2),\n            put(sc, SC),\n            Ret;\n        _ ->\n            error_logger:format(\"Failed to yssi ~p~n\", [Yfile]),\n            ok\n    end;\n\n\nhandle_out_reply({html, Html}, _LineNo, _YawsFile,  _UT, _ARG) ->\n    accumulate_content(Html),\n    ok;\n\nhandle_out_reply({ehtml, E}, _LineNo, _YawsFile,  _UT, ARG) ->\n    case safe_ehtml_expand(E) of\n        {ok, Val} ->\n            accumulate_content(Val),\n            ok;\n        {error, ErrStr} ->\n            handle_crash(ARG, ErrStr)\n    end;\n\nhandle_out_reply({exhtml, E}, _LineNo, _YawsFile,  _UT, _A) ->\n    N = count_trailing_spaces(),\n    accumulate_content(yaws_exhtml:format(E, N)),\n    ok;\n\nhandle_out_reply({exhtml, Value2StringF, E}, _LineNo, _YawsFile,  _UT, _A) ->\n    N = count_trailing_spaces(),\n    accumulate_content(yaws_exhtml:format(E, N, Value2StringF)),\n    ok;\n\nhandle_out_reply({sexhtml, E}, _LineNo, _YawsFile,  _UT, _A) ->\n    accumulate_content(yaws_exhtml:sformat(E)),\n    ok;\n\nhandle_out_reply({sexhtml, Value2StringF, E},\n                 _LineNo, _YawsFile,  _UT, _A) ->\n    accumulate_content(yaws_exhtml:sformat(E, Value2StringF)),\n    ok;\n\nhandle_out_reply({content, MimeType, Cont}, _LineNo,_YawsFile, _UT, _ARG) ->\n    yaws:outh_set_content_type(MimeType),\n    accumulate_content(Cont),\n    ok;\n\nhandle_out_reply({streamcontent, MimeType, First},\n                 _LineNo,_YawsFile, _UT, _ARG) ->\n    yaws:outh_set_content_type(MimeType),\n    accumulate_content(First),\n    {streamcontent, MimeType, First};\n\nhandle_out_reply({streamcontent_with_timeout, MimeType, First, Timeout},\n                 _LineNo,_YawsFile, _UT, _ARG) ->\n    yaws:outh_set_content_type(MimeType),\n    accumulate_content(First),\n    {streamcontent_with_timeout, MimeType, First, Timeout};\n\nhandle_out_reply(Res = {page, _Page},\n                 _LineNo,_YawsFile, _UT, _ARG) ->\n    Res;\n\nhandle_out_reply({streamcontent_with_size, Sz, MimeType, First},\n                 _LineNo,_YawsFile, _UT, _ARG) ->\n    yaws:outh_set_content_type(MimeType),\n    accumulate_content(First),\n    {streamcontent_with_size, Sz, MimeType, First};\n\nhandle_out_reply({streamcontent_from_pid, MimeType, Pid},\n                 _LineNo,_YawsFile, _UT, _ARG) ->\n    yaws:outh_set_content_type(MimeType),\n    {streamcontent_from_pid, MimeType, Pid};\n\nhandle_out_reply({websocket, _CallbackMod, _Opts}=Reply,\n                 _LineNo,_YawsFile, _UT, _ARG) ->\n    yaws:accumulate_header({connection, erase}),\n    Reply;\n\nhandle_out_reply({header, H},  _LineNo, _YawsFile, _UT, _ARG) ->\n    yaws:accumulate_header(H),\n    ok;\n\nhandle_out_reply({allheaders, Hs}, _LineNo, _YawsFile, _UT, _ARG) ->\n    yaws:outh_clear_headers(),\n    foreach(fun({header, Head}) -> yaws:accumulate_header(Head) end, Hs),\n    ok;\n\nhandle_out_reply({status, Code},_LineNo,_YawsFile,_UT,_ARG)\n    when is_integer(Code) ->\n    yaws:outh_set_status_code(Code),\n    ok;\n\nhandle_out_reply({'EXIT', normal}, _LineNo, _YawsFile, _UT, _ARG) ->\n    exit(normal);\n\nhandle_out_reply({ssi, File, Delimiter, Bindings}, LineNo, YawsFile, UT, ARG) ->\n    case ssi(File, Delimiter, Bindings, UT, ARG) of\n        {error, Rsn} ->\n            L = ?F(\"yaws code at~s:~p had the following err:~n~p\",\n                   [YawsFile, LineNo, Rsn]),\n            handle_crash(ARG, L);\n        OutData ->\n            accumulate_content(OutData),\n            ok\n    end;\n\n\nhandle_out_reply(break, _LineNo, _YawsFile, _UT, _ARG) ->\n    break;\n\nhandle_out_reply({redirect_local, Path}, LN, YF, UT, ARG) ->\n    handle_out_reply({redirect_local, Path, 302}, LN, YF, UT, ARG);\n\n%% What about:\n%%\n%% handle_out_reply({redirect_local, Path, Status}, LineNo,\n%%                 YawsFile, SC, ARG) when string(Path) ->\n%%   handle_out_reply({redirect_local, {any_path, Path}, Status}, LineNo,\n%%                 YawsFile, SC, ARG);\n%%\n%% It would introduce a slight incompatibility with earlier versions,\n%% but might be desirable.\n\nhandle_out_reply({redirect_local, {any_path, URL}, Status}, LineNo,\n                 YawsFile, _UT, ARG) ->\n    PathType =\n        case yaws_api:is_absolute_URI(URL) of\n            true -> net_path;\n            false -> case URL of\n                         [$/|_] -> abs_path;\n                         _ -> rel_path\n                     end\n        end,\n    handle_out_reply({redirect_local, {PathType, URL}, Status}, LineNo,\n                     YawsFile, _UT, ARG);\n\nhandle_out_reply({redirect_local, {net_path, URL}, Status}, _LineNo,\n                 _YawsFile,  _UT, _ARG) ->\n    Loc = [\"Location: \", URL, \"\\r\\n\"],\n    new_redir_h(get(outh), Loc, Status),\n    ok;\n\nhandle_out_reply({redirect_local, Path0, Status}, _LineNo,_YawsFile,_UT, ARG) ->\n    SC=get(sc),\n    Path = case Path0 of\n               {abs_path, P} ->\n                   P;\n               {rel_path, P} ->\n                   {abs_path, RP} = (ARG#arg.req)#http_request.path,\n                   case string:rchr(RP, $/) of\n                       0 ->\n                           [$/|P];\n                       N ->\n                           [lists:sublist(RP, N),P]\n                   end;\n               P ->\n                   P\n           end,\n    Scheme = yaws:redirect_scheme(SC),\n    Headers = ARG#arg.headers,\n    HostPort = yaws:redirect_host(SC, Headers#headers.host),\n    Loc = [\"Location: \", Scheme, HostPort, Path, \"\\r\\n\"],\n    new_redir_h(get(outh), Loc, Status),\n    ok;\n\nhandle_out_reply({redirect, URL}, LN, YF, UT, ARG) ->\n    handle_out_reply({redirect, URL, 302}, LN, YF, UT, ARG);\n\nhandle_out_reply({redirect, URL, Status}, _LineNo, _YawsFile, _UT, _ARG) ->\n    Loc = [\"Location: \", URL, \"\\r\\n\"],\n    new_redir_h(get(outh), Loc, Status),\n    ok;\n\nhandle_out_reply({bindings, L}, _LineNo, _YawsFile, _UT, _ARG) ->\n    foreach(fun({Key, Value}) -> put({binding, Key}, Value) end, L),\n    ok;\n\nhandle_out_reply(ok, _LineNo, _YawsFile, _UT, _ARG) ->\n    ok;\n\nhandle_out_reply({'EXIT', Err}, LineNo, YawsFile, _UT, ARG) ->\n    L = ?F(\"~n~nERROR erlang  code  crashed:~n \"\n           \"File: ~s:~w~n\"\n           \"Reason: ~p~nReq: ~p~n\"\n           \"Stack: ~p~n\",\n           [YawsFile, LineNo, Err, ARG#arg.req, erlang:get_stacktrace()]),\n    handle_crash(ARG, L);\n\nhandle_out_reply({throw, Class, Exc}, LineNo, YawsFile, _UT, ARG) ->\n    L = ?F(\"~n~nERROR erlang code threw an uncaught exception:~n \"\n           \"File: ~s:~w~n\"\n           \"Class: ~p~nException: ~p~nReq: ~p~n\"\n           \"Stack: ~p~n\",\n           [YawsFile, LineNo, Class, Exc, ARG#arg.req,\n            erlang:get_stacktrace()]),\n    handle_crash(ARG, L);\n\nhandle_out_reply({get_more, Cont, State}, _LineNo, _YawsFile, _UT, _ARG) ->\n    {get_more, Cont, State};\n\nhandle_out_reply(Arg = #arg{},  _LineNo, _YawsFile, _UT, _ARG) ->\n    Arg;\n\nhandle_out_reply(flush, _LineNo, _YawsFile, _UT, ARG) ->\n    CliSock = case yaws_api:get_sslsocket(ARG#arg.clisock) of\n                  {ok, SslSock} -> SslSock;\n                  undefined     -> ARG#arg.clisock\n              end,\n    Hdrs = ARG#arg.headers,\n    CliDataPos0 = get(client_data_pos),\n    CliDataPos1 = flush(CliSock, CliDataPos0,\n                        Hdrs#headers.content_length,\n                        yaws:to_lower(Hdrs#headers.transfer_encoding)),\n    put(client_data_pos, CliDataPos1),\n    ok;\n\nhandle_out_reply(Reply, LineNo, YawsFile, _UT, ARG) ->\n    L =  ?F(\"yaws code at ~s:~p crashed or \"\n            \"ret bad val:~p ~nReq: ~p\",\n            [YawsFile, LineNo, Reply, ARG#arg.req]),\n    handle_crash(ARG, L).\n\n\n\nhandle_out_reply_l([Reply|T], LineNo, YawsFile, UT, ARG, Res) ->\n    case handle_out_reply(Reply, LineNo, YawsFile, UT, ARG) of\n        break ->\n            break;\n        {page, Page} ->\n            {page, Page};\n        {get_more, Cont, State} ->\n            {get_more, Cont, State};\n        {streamcontent,_,_}=Reply ->\n            Reply;\n        {streamcontent_with_timeout,_,_,_}=Reply ->\n            Reply;\n        {streamcontent_with_size,_,_,_}=Reply ->\n            Reply;\n        {streamcontent_from_pid,_,_}=Reply ->\n            Reply;\n        {websocket,_,_}=Reply ->\n            Reply;\n        ok ->\n            handle_out_reply_l(T, LineNo, YawsFile, UT, ARG, Res);\n        RetVal ->\n            %% XXX: if RetVal == #arg{}, replace ARG in recursion ?\n            handle_out_reply_l(T, LineNo, YawsFile, UT, ARG, RetVal)\n    end;\nhandle_out_reply_l([], _LineNo, _YawsFile, _UT, _ARG, Res) ->\n    Res.\n\n\ncount_trailing_spaces() ->\n    case get(acc_content) of\n        undefined -> 0;\n        discard -> 0;\n        List ->\n            Binary = first_binary(List),\n            yaws_exhtml:count_trailing_spaces(Binary)\n    end.\n\nfirst_binary([Binary|_]) when is_binary(Binary) -> Binary;\nfirst_binary([List|_Rest]) when is_list(List) -> first_binary(List).\n\n\n\n%% fast server side include with macrolike variable bindings expansion\n%%\n\n\nssi(File, Delimiter, Bindings) ->\n    ssi(File, Delimiter, Bindings, get(yaws_ut), get(yaws_arg), get (sc)).\n\n\nssi(File, Delimiter, Bindings, UT, ARG) ->\n    ssi(File, Delimiter, Bindings, UT, ARG, get(sc)).\n\n\nssi(File, Delimiter, Bindings, UT, ARG, SC) ->\n\n    Dir = UT#urltype.dir,\n    %%Dir here should be equiv to arg.prepath\n\n    Docroot = ARG#arg.docroot,\n    VirtualDir = ARG#arg.docroot_mount,\n\n\n    %%JMN - line below looks suspicious, why are we not keying on\n    %% {ssi, File, Dir} ???\n    %%Surely a name like header.inc may be present in various parts of the\n    %% hierarchy so Dir should form part of key.\n    Key = {ssi, File, Delimiter},\n\n    %%!todo - review rel_path & abs_path - define & document behaviour..\n    %% or remove them.\n\n    FullPath =\n        case File of\n            {rel_path, FileName} ->\n                [Docroot, [$/|Dir],[$/|FileName]];\n            {abs_path, FileName} ->\n                [Docroot, [$/|FileName]];\n            [$/|_] ->\n                %%absolute path - need to determine Docroot and any Vdir\n                %% that might apply\n                {Vdir, DR} = vdirpath(SC, ARG, File),\n\n                construct_fullpath(DR, File, Vdir);\n            _ ->\n                %%relative to the Docroot and VirtualDir that correspond\n                %% to the request.\n\n                construct_fullpath(Docroot, lists:flatten([Dir, [$/|File]]),\n                                   VirtualDir)\n        end,\n\n    Mtime = path_to_mtime(FullPath),\n    case ets:lookup(SC#sconf.ets, Key) of\n        [{_, Parts, Mtime}] ->\n            case (catch expand_parts(Parts, Bindings, [])) of\n                {'EXIT', ErrStr} ->\n                    {error, ErrStr};\n                Value ->\n                    Value\n            end;\n        _ ->\n            case prim_file:read_file(FullPath) of\n                {ok, Bin} ->\n                    D =delim_split_file(Delimiter,binary_to_list(Bin),data,[]),\n                    ets:insert(SC#sconf.ets,{Key,D, Mtime}),\n                    ssi(File, Delimiter, Bindings, UT, ARG, SC);\n                {error, _} when SC#sconf.xtra_docroots /= [] ->\n                    SC2 = SC#sconf{docroot = hd(SC#sconf.xtra_docroots),\n                                   xtra_docroots = tl(SC#sconf.xtra_docroots)},\n\n                    ARG2 = ARG#arg{\n                             docroot = hd(SC#sconf.xtra_docroots),\n                             docroot_mount = \"/\"\n                            },\n\n                    ssi(File, Delimiter, Bindings, UT, ARG2, SC2);\n                {error, Rsn} ->\n                    error_logger:format(\"Failed to read/ssi file ~p~n\",\n                                        [FullPath]),\n                    {error,Rsn}\n            end\n    end.\n\n\npath_to_mtime(FullPath) ->\n    case prim_file:read_file_info(FullPath) of\n        {ok, FI} ->\n            mtime(FI);\n        Err ->\n            Err\n    end.\n\n\nexpand_parts([{data, D} |T], Bs, Ack) ->\n    expand_parts(T, Bs, [D|Ack]);\nexpand_parts([{var, V} |T] , Bs, Ack) ->\n    case lists:keysearch(V, 1, Bs) of\n        {value, {_, {ehtml, E}}} ->\n            case safe_ehtml_expand(E) of\n                {ok, Val} ->\n                    expand_parts(T, Bs, [Val|Ack]);\n                {error, ErrStr} ->\n                    erlang:error(ErrStr)\n            end;\n        {value, {_, Val}} ->\n            expand_parts(T, Bs, [Val|Ack]);\n        false  ->\n            case get({binding,V}) of\n                undefined -> expand_parts(T, Bs, Ack);\n                Valb -> expand_parts(T, Bs, [Valb|Ack])\n            end\n    end;\nexpand_parts([], _,Ack) ->\n    lists:reverse(Ack).\n\n\n\ndelim_split_file([], Data, _, _Ack) ->\n    [{data, Data}];\ndelim_split_file(Del, Data, State, Ack) ->\n    case delim_split(Del, Del, Data, [], []) of\n        {H, []} when State == data ->\n            %% Ok, last chunk\n            lists:reverse([{data, list_to_binary(H)} | Ack]);\n        {H, T} when State == data ->\n            delim_split_file(Del, T, var, [{data, list_to_binary(H)}|Ack]);\n        {H, []} when State == var ->\n            lists:reverse([{var, H} | Ack]);\n        {H, T} when State == var ->\n            delim_split_file(Del, T, data, [{var, H}|Ack])\n    end.\n\n\ndelim_split([H|T], Odel, [H|T1], Ack, DAcc) ->\n    delim_split(T,Odel,T1,Ack, [H|DAcc]);\ndelim_split([], _Odel, T, Ack, _DAcc) ->\n    {lists:reverse(Ack),T};\ndelim_split([H|_T],Odel, [H1|T1], Ack, []) when H /= H1 ->\n    delim_split(Odel, Odel, T1, [H1|Ack], []);\ndelim_split([H|_T],Odel, [H1|T1], Ack, DAcc) when H /= H1 ->\n    delim_split(Odel, Odel, T1, [H1|DAcc++Ack], []);\ndelim_split(_,_,[],Ack,[]) ->\n    {lists:reverse(Ack),[]};\ndelim_split(_,_,[],Ack,DAcc) ->\n    {lists:reverse(DAcc++Ack),[]}.\n\n\n\n%% Erlang yaws code crashed, display either the\n%% actual crash or a customized error message\n\nhandle_crash(ARG, L) ->\n    ?Debug(\"handle_crash(~p)~n\", [L]),\n    SC=get(sc),\n    yaws:outh_set_status_code(500),\n    case catch apply(SC#sconf.errormod_crash, crashmsg, [ARG, SC, L]) of\n        {content,MimeType,Cont} ->\n            yaws:outh_set_content_type(MimeType),\n            accumulate_content(Cont),\n            break;\n        {html, Str} ->\n            accumulate_content(Str),\n            break;\n        {ehtml, Term} ->\n            case safe_ehtml_expand(Term) of\n                {error, Reason} ->\n                    yaws:elog(\"~s\", [Reason]),\n                    %% Aghhh, yet another user crash :-(\n                    T2 = [{h2, [], \"Internal error\"}, {hr},\n                          {p, [], \"Customized crash display code crashed !!!\"}],\n                    accumulate_content(ehtml_expand(T2)),\n                    break;\n                {ok, Out} ->\n                    accumulate_content(Out),\n                    break\n            end;\n        Other ->\n            yaws:elog(\"Bad return value from errmod_crash ~n~p~n\",[Other]),\n            T2 = [{h2, [], \"Internal error\"}, {hr},\n                  {p, [], \"Customized crash display code returned bad val\"}],\n            accumulate_content(ehtml_expand(T2)),\n            break\n\n    end.\n\n%% Ret: true | false | {data, Data}\ndecide_deflate(false, _, _, _, _, _, _) ->\n    ?Debug(\"Compression not supported by the server~n\", []),\n    false;\ndecide_deflate(_, _, _, _, _, identity, _) ->\n    ?Debug(\"No compression: Encoding=identity~n\", []),\n    false;\ndecide_deflate(_, _, _, _, _, deflate, _) ->\n    ?Debug(\"Compression already handled: Encoding=deflate~n\", []),\n    false;\ndecide_deflate(true, SC, Arg, Sz, Data, decide, Mode) ->\n    DOpts = SC#sconf.deflate_options,\n    if\n        Mode == final andalso size(Data) == 0 ->\n            ?Debug(\"No data to be compressed~n\",[]),\n            false;\n\n        Mode == final andalso\n        DOpts#deflate.min_compress_size /= nolimit andalso\n        size(Data) < DOpts#deflate.min_compress_size ->\n            ?Debug(\"Data too small to be compressed~n\",[]),\n            false;\n\n        is_integer(Sz) andalso\n        DOpts#deflate.min_compress_size /= nolimit andalso\n        Sz < DOpts#deflate.min_compress_size ->\n            ?Debug(\"Data too small to be compressed~n\",[]),\n            false;\n\n        true ->\n            Mime0     = yaws:outh_get_content_type(),\n            [Mime1|_] = yaws:split_sep(Mime0, $;), %% Remove charset\n            ?Debug(\"Check compression support: Mime-Type=~p~n\", [Mime1]),\n            case compressible_mime_type(Mime1, DOpts) of\n                true ->\n                    case (Arg =:= undefined\n                          orelse\n                          yaws:accepts_gzip(Arg#arg.headers, Mime1)) of\n                        true when Mode =:= final ->\n                            ?Debug(\"Compress data~n\", []),\n                            yaws:outh_set_content_encoding(deflate),\n                            {ok, DB} = yaws_zlib:gzip(Data, DOpts),\n                            {data, DB};\n                        true -> %% Mode == stream | {file,_,_}\n                            ?Debug(\"Compress streamed data~n\", []),\n                            yaws:outh_set_content_encoding(deflate),\n                            true;\n                        false ->\n                            ?Debug(\"Compression not supported by the client~n\",\n                                   []),\n                            yaws:outh_set_content_encoding(identity),\n                            false\n                    end;\n                false ->\n                    ?Debug(\"~p is not compressible~n\", [Mime1]),\n                    yaws:outh_set_content_encoding(identity),\n                    false\n            end\n    end.\n\n\n\ndeliver_accumulated(Sock) ->\n    case yaws:outh_get_content_encoding() of\n        decide -> yaws:outh_set_content_encoding(identity);\n        _      -> ok\n    end,\n    deliver_accumulated(undefined, Sock, undefined, final).\n\n%% Arg           = #arg{} | undefined\n%% ContentLength = Int    | undefined\n%% Mode          = final  | stream | {file, File, MTime}\n%%\n%% For Mode==final: (all content has been accumulated before calling\n%%                   deliver_accumulated)\n%%     Result: can be ignored\n%%\n%% For Mode==stream:\n%%     Result: opaque value to be threaded through\n%%             send_streamcontent_chunk / end_streaming\n%%\n%% For Mode=={file,File,MTime}:\n%%     Result: {gzfile, GzFile} is gzip_static option is enabled and if\n%%     GzFile exists. Else, same result than for Mode==stream\n\ndeliver_accumulated(Arg, Sock, ContentLength, Mode) ->\n    %% See if we must close the connection\n    receive\n        {_From, suspend} -> yaws:outh_set_connection(true)\n    after 0 -> ok\n    end,\n\n    Cont = case erase(acc_content) of\n               undefined -> [];\n               Cont2     -> Cont2\n           end,\n    {Result, Data} = case Cont of\n                         discard ->\n                             yaws:outh_set_transfer_encoding_off(),\n                             {discard, []};\n                         _ ->\n                             deflate_accumulated(Arg, iolist_to_binary(Cont),\n                                                 ContentLength, Mode)\n                     end,\n\n    {StatusLine, Headers} = yaws:outh_serialize(),\n    All = [StatusLine, Headers, crnl(), Data],\n    yaws:gen_tcp_send(Sock, All),\n    case yaws_trace:get_type(get(gc)) of\n        http      -> yaws_trace:write(from_server, [StatusLine, Headers]);\n        traffic   -> yaws_trace:write(from_server, All);\n        undefined -> ok\n    end,\n    Result.\n\ndeflate_accumulated(Arg, Content, ContentLength, Mode) ->\n    case get(sc) of\n        undefined ->\n            {undefined, Content};\n        SC ->\n            Enc   = yaws:outh_get_content_encoding(),\n            DOpts = SC#sconf.deflate_options,\n            {Result, Data, Size} =\n                case decide_deflate(?sc_has_deflate(SC), SC, Arg, ContentLength,\n                                    Content, Enc, Mode) of\n                    {data, Bin} ->\n                        %% implies Mode==final\n                        {undefined, Bin, iolist_size(Bin)};\n\n                    true when Mode == stream; DOpts#deflate.use_gzip_static == false ->\n                        Z = zlib:open(),\n                        {ok, Priv, Bin} =\n                            yaws_zlib:gzipDeflate(Z,yaws_zlib:gzipInit(Z,DOpts),\n                                                  Content,none),\n                        {{Z, Priv}, Bin, undefined};\n                    true ->\n                        %% implies Mode=={file,_,_} and use_gzip_static==true\n                        {file, File, MTime} = Mode,\n                        GzFile = File++\".gz\",\n                        case prim_file:read_file_info(GzFile) of\n                            {ok, FI} when FI#file_info.type == regular,\n                                          FI#file_info.mtime >= MTime ->\n                                {{gzfile, GzFile}, <<>>, FI#file_info.size};\n                            _ ->\n                                Z = zlib:open(),\n                                {ok, Priv, Bin} =\n                                    yaws_zlib:gzipDeflate(Z,yaws_zlib:gzipInit(Z,DOpts),\n                                                          Content,none),\n                                {{Z, Priv}, Bin, undefined}\n                        end;\n\n                    false when Mode == final ->\n                        {undefined, Content, iolist_size(Content)};\n                    false ->\n                        %% implies Mode=stream | {file,_,_}\n                        {undefined, Content, ContentLength}\n                end,\n            case Size of\n                undefined -> yaws:outh_fix_doclose();\n                _         -> yaws:accumulate_header({content_length, Size})\n            end,\n            case Mode of\n                final ->\n                    {Result, Data};\n                _ ->\n                    case make_chunk(Data) of\n                        empty ->\n                            {Result, []};\n                        {S, Chunk} ->\n                            yaws:outh_inc_act_contlen(S),\n                            {Result, Chunk}\n                    end\n            end\n    end.\n\nget_more_post_data(CliSock, PPS, ARG) ->\n    SC = get(sc),\n    N = SC#sconf.partial_post_size,\n    case {yaws:to_lower((ARG#arg.headers)#headers.transfer_encoding),\n          (ARG#arg.headers)#headers.content_length} of\n        {\"chunked\", _} ->\n            get_chunked_client_data(CliSock, yaws:is_ssl(SC));\n        {_, undefined} ->\n            <<>>;\n        {_, Len} ->\n            Int_len = list_to_integer(Len),\n            if N + PPS < Int_len ->\n                    Bin = get_client_data(CliSock, N, yaws:is_ssl(SC)),\n                    {partial, Bin};\n               true ->\n                    get_client_data(CliSock, Int_len - PPS, yaws:is_ssl(SC))\n            end\n    end.\n\n\nut_read(UT) ->\n    ?Debug(\"ut_read() UT.fullpath = ~p~n\", [UT#urltype.fullpath]),\n    CE = yaws:outh_get_content_encoding(),\n    if\n\n        (CE =:= identity) andalso is_binary(UT#urltype.data) ->\n            UT#urltype.data;\n        CE =:= identity ->\n            ?Debug(\"ut_read reading\\n\",[]),\n            {ok, Bin} = file:read_file(UT#urltype.fullpath),\n            ?Debug(\"ut_read read ~p\\n\",[size(Bin)]),\n            Bin;\n\n        (CE =:= decide) andalso is_binary(UT#urltype.deflate) ->\n            ?Debug(\"ut_read using deflated binary of size ~p~n\",\n                   [size(UT#urltype.deflate)]),\n            yaws:outh_set_content_encoding(deflate),\n            UT#urltype.deflate;\n        CE =:= decide andalso is_binary(UT#urltype.data) ->\n            UT#urltype.data;\n        CE =:= decide ->\n            ?Debug(\"ut_read reading\\n\",[]),\n            {ok, Bin} = file:read_file(UT#urltype.fullpath),\n            ?Debug(\"ut_read read ~p\\n\",[size(Bin)]),\n            Bin;\n\n        CE =:= deflate ->\n            ?Debug(\"ut_read using deflated binary of size ~p~n\",\n                   [size(UT#urltype.deflate)]),\n            UT#urltype.deflate\n    end.\n\n\nparse_range(L, Tot) ->\n    case catch parse_range_throw(L, Tot) of\n        {'EXIT', _} ->\n            error;\n        R -> R\n    end.\n\nparse_range_throw(L, Tot) ->\n    case lists:splitwith(fun(C)->C /= $- end, L) of\n        {FromS, [$-|ToS]} ->\n            case FromS of\n                [] -> case list_to_integer(ToS) of\n                          I when Tot >= I, I>0 ->\n                              {fromto, Tot-I, Tot-1, Tot}\n                      end;\n                _ -> case list_to_integer(FromS) of\n                         From when From>=0, From < Tot ->\n                             case ToS of\n                                 [] -> {fromto, From, Tot-1, Tot};\n                                 _ -> case list_to_integer(ToS) of\n                                          To when To<Tot ->\n                                              {fromto, From, To, Tot};\n                                          _ ->\n                                              {fromto, From, Tot-1, Tot}\n                                      end\n                             end\n                     end\n            end\n    end.\n\n\n%% This is not exactly what the RFC describes, but we do not want to\n%% deal with multipart/byteranges.\nunite_ranges(all, _) ->\n    all;\nunite_ranges(error, R) ->\n    R;\nunite_ranges(_, all) ->\n    all;\nunite_ranges(R, error) ->\n    R;\nunite_ranges({fromto, F0, T0, Tot},{fromto,F1,T1, Tot}) ->\n    {fromto,\n     if F0 >= F1 -> F1;\n        true -> F0\n     end,\n     if T0 >= T1 -> T0;\n        true -> T1\n     end,\n     Tot\n    }.\n\n\n%% ret:  all | error | {fromto, From, To, Tot}\nrequested_range(RangeHeader, TotalSize) ->\n    case yaws:split_sep(RangeHeader, $,) of\n        [\"bytes=\"++H|T] ->\n            lists:foldl(fun(L, R)->\n                                unite_ranges(parse_range(L, TotalSize), R)\n                        end, parse_range(H, TotalSize), T);\n        _ -> all\n    end.\n\n\ndeliver_file(CliSock, Req, UT, Range) ->\n    if\n        is_binary(UT#urltype.data) ->\n            %% cached\n            deliver_small_file(CliSock, Req, UT, Range);\n        true ->\n            deliver_large_file(CliSock, Req, UT, Range)\n    end.\n\ndeliver_small_file(CliSock, _Req, UT, Range) ->\n    Bin0 = ut_read(UT),\n    Bin = case Range of\n              all ->\n                  Bin0;\n              {fromto, From, To, _Tot} ->\n                  Length = To - From + 1,\n                  <<_:From/binary, Bin1:Length/binary, _/binary>> = Bin0,\n                  Bin1\n          end,\n    accumulate_content(Bin),\n    deliver_accumulated(CliSock),\n    done_or_continue().\n\ndeliver_large_file(CliSock,  _Req, UT, Range) ->\n    Sz = case Range of\n             all ->\n                 (UT#urltype.finfo)#file_info.size;\n             {fromto, From, To, _Tot} ->\n                 yaws:outh_set_content_encoding(identity),\n                 (To - From + 1)\n         end,\n    Mode = {file, UT#urltype.fullpath, mtime(UT#urltype.finfo)},\n    case deliver_accumulated(undefined, CliSock, Sz, Mode) of\n        discard -> ok;\n        Priv    -> send_file(CliSock, UT#urltype.fullpath, Range, Priv)\n    end,\n    done_or_continue().\n\n\nsend_file(CliSock, Path, all, undefined) when is_port(CliSock) ->\n    ?Debug(\"send_file(~p,~p,no ...)~n\", [CliSock, Path]),\n    Size = yaws_sendfile:send(CliSock, Path),\n    yaws_stats:sent(Size);\nsend_file(CliSock, Path, all, undefined) ->\n    ?Debug(\"send_file(~p,~p,no ...)~n\", [CliSock, Path]),\n    {ok, Fd} = file:open(Path, [raw, binary, read]),\n    send_file(CliSock, Fd, undefined);\nsend_file(CliSock, _, all, {gzfile, GzFile}) when is_port(CliSock) ->\n    ?Debug(\"send_file(~p,~p, ...)~n\", [CliSock, GzFile]),\n    Size = yaws_sendfile:send(CliSock, GzFile),\n    yaws_stats:sent(Size);\nsend_file(CliSock, _, all, {gzfile, GzFile}) ->\n    ?Debug(\"send_file(~p,~p, ...)~n\", [CliSock, GzFile]),\n    {ok, Fd} = file:open(GzFile, [raw, binary, read]),\n    send_file(CliSock, Fd, undefined);\nsend_file(CliSock, Path, all, Priv) ->\n    ?Debug(\"send_file(~p,~p, ...)~n\", [CliSock, Path]),\n    {ok, Fd} = file:open(Path, [raw, binary, read]),\n    send_file(CliSock, Fd, Priv);\nsend_file(CliSock, Path,  {fromto, From, To, _Tot}, _) when is_port(CliSock) ->\n    Size = yaws_sendfile:send(CliSock, Path, From, (To-From+1)),\n    yaws_stats:sent(Size);\nsend_file(CliSock, Path,  {fromto, From, To, _Tot}, _) ->\n    {ok, Fd} = file:open(Path, [raw, binary, read]),\n    file:position(Fd, {bof, From}),\n    send_file_range(CliSock, Fd, To - From + 1).\n\nsend_file(CliSock, Fd, Priv) ->\n    ?Debug(\"send_file(~p,~p, ...)~n\", [CliSock, Fd]),\n    case file:read(Fd, (get(gc))#gconf.large_file_chunk_size) of\n        {ok, Bin} ->\n            Priv1 = send_streamcontent_chunk(Priv, CliSock, Bin),\n            send_file(CliSock, Fd, Priv1);\n        eof ->\n            file:close(Fd),\n            end_streaming(Priv, CliSock)\n    end.\n\nsend_file_range(CliSock, Fd, Len) when Len > 0 ->\n    {ok, Bin} = file:read(Fd,\n                          case (get(gc))#gconf.large_file_chunk_size of\n                              S when S < Len -> S;\n                              _ -> Len\n                          end\n                         ),\n    send_streamcontent_chunk(undefined, CliSock, Bin),\n    send_file_range(CliSock, Fd, Len - size(Bin));\nsend_file_range(CliSock, Fd, 0) ->\n    file:close(Fd),\n    end_streaming(undefined, CliSock).\n\ncrnl() ->\n    \"\\r\\n\".\n\nnow_secs() -> yaws_dynopts:now_secs().\n\n%% a file cache,\nurl_type(GetPath, ArgDocroot, VirtualDir) ->\n    SC=get(sc),\n    GC=get(gc),\n    E = SC#sconf.ets,\n\n    %% In reentrant call, the cache can be disabled. It could be useful in case\n    %% of \"proxy\" appmod.\n    NoCache = case get(is_reentrant_request) of\n                  true ->\n                      case get(page_options) of\n                          undefined -> false;\n                          Opts      ->  proplists:get_bool(disable_cache, Opts)\n                      end;\n                  _ ->\n                      false\n              end,\n\n    case ets:lookup(E, {url, GetPath}) of\n        [] ->\n            UT = do_url_type(SC, GetPath, ArgDocroot, VirtualDir),\n            ?TC([{record, UT, urltype}]),\n            ?Debug(\"UT=~s\\n\", [?format_record(UT, urltype)]),\n            if\n                NoCache ->\n                    ?Debug(\"Cache disabled\\n\", []),\n                    UT;\n                true ->\n                    CF = cache_file(SC, GC, GetPath, UT),\n                    ?Debug(\"CF=~s\\n\", [?format_record(CF, urltype)]),\n                    CF\n            end;\n        [{_, When, UT}] ->\n            N = now_secs(),\n            Refresh = GC#gconf.cache_refresh_secs,\n            if\n                ((N-When) >= Refresh) ->\n                    ?Debug(\"Timed out entry for ~s ~p~n\",\n                           [GetPath, {When, N}]),\n                    %% more than 30 secs old entry\n                    UT2 = do_url_type(SC, GetPath, ArgDocroot, VirtualDir),\n                    case file_changed(UT, UT2) of\n                        true ->\n                            ?Debug(\"Recaching~n\", []),\n                            ets:delete(E, {url, GetPath}),\n                            ets:delete(E, {urlc, GetPath}),\n                            ets:update_counter(E, num_files, -1),\n                            ets:update_counter(E, num_bytes, -cache_size(UT)),\n                            cache_file(SC, GC, GetPath, UT2);\n                        false ->\n                            ?Debug(\"Using unchanged cached version~n\", []),\n                            (catch ets:update_counter(E, {urlc, GetPath}, 1)),\n                            UT\n                    end;\n                true ->\n                    ?Debug(\"Serve page from cache ~p\", [{When , N, N-When}]),\n                    (catch ets:update_counter(E, {urlc, GetPath}, 1)),\n                    UT\n            end\n    end.\n\n\nfile_changed(UT1, UT2) ->\n    case {UT1#urltype.type, UT2#urltype.type} of\n        {T, T} when T==yaws; T==regular->\n            F1 = UT1#urltype.finfo,\n            F2 = UT2#urltype.finfo,\n            {F1#file_info.inode, F1#file_info.mtime}\n                /= {F2#file_info.inode, F2#file_info.mtime};\n        _ ->\n            true % don't care too much\n    end.\n\n\n\ncache_size(UT) when is_binary(UT#urltype.deflate),\n                    is_binary(UT#urltype.data) ->\n    size(UT#urltype.deflate) + size(UT#urltype.data);\ncache_size(UT) when is_binary(UT#urltype.data) ->\n    size(UT#urltype.data);\ncache_size(_UT) ->\n    0.\n\n\n\n\ncache_file(_SC, GC, _Path, UT)\n  when GC#gconf.max_num_cached_files == 0;\n       GC#gconf.max_num_cached_bytes == 0;\n       GC#gconf.max_size_cached_file == 0 ->\n    UT;\ncache_file(SC, GC, Path, UT)\n  when ((UT#urltype.type == regular) or\n        ((UT#urltype.type == yaws) and (UT#urltype.pathinfo == undefined))) ->\n    E = SC#sconf.ets,\n    [{num_files, N}] = ets:lookup(E, num_files),\n    [{num_bytes, B}] = ets:lookup(E, num_bytes),\n    FI = UT#urltype.finfo,\n    ?Debug(\"FI=~s\\n\", [?format_record(FI, file_info)]),\n    if\n        N + 1 > GC#gconf.max_num_cached_files ->\n            error_logger:info_msg(\"Max NUM cached files reached for server ~p\",\n                                  [SC#sconf.servername]),\n            cleanup_cache(E, num),\n            cache_file(SC, GC, Path, UT);\n        FI#file_info.size < GC#gconf.max_size_cached_file,\n        FI#file_info.size < GC#gconf.max_num_cached_bytes,\n        B + FI#file_info.size > GC#gconf.max_num_cached_bytes ->\n            error_logger:info_msg(\"Max size cached bytes reached for server ~p\",\n                                  [SC#sconf.servername]),\n            cleanup_cache(E, size),\n            cache_file(SC, GC, Path, UT);\n        true ->\n            ?Debug(\"Check file size\\n\",[]),\n            if\n                FI#file_info.size > GC#gconf.max_size_cached_file;\n                FI#file_info.size > GC#gconf.max_num_cached_bytes ->\n                    ?Debug(\"Too large\\n\",[]),\n                    UT;\n                true ->\n                    ?Debug(\"File fits\\n\",[]),\n                    {ok, Bin} = prim_file:read_file(UT#urltype.fullpath),\n                    DOpts = SC#sconf.deflate_options,\n                    Deflated =\n                        if\n                            size(Bin) == 0 ->\n                                undefined;\n                            DOpts#deflate.min_compress_size /= nolimit,\n                            size(Bin) < DOpts#deflate.min_compress_size ->\n                                undefined;\n                            UT#urltype.deflate /= dynamic ->\n                                undefined;\n                            true ->\n                                {ok, DBL} = yaws_zlib:gzip(Bin, DOpts),\n                                DB = list_to_binary(DBL),\n                                if\n                                    (10 * size(DB)) < (9 * size(Bin)) ->\n                                        ?Debug(\"storing deflated version \"\n                                               \"of ~p~n\",[UT#urltype.fullpath]),\n                                        DB;\n                                    true ->\n                                        undefined\n                                end\n                        end,\n                    UT2 = UT#urltype{data = Bin, deflate = Deflated},\n                    ets:insert(E, {{url, Path}, now_secs(), UT2}),\n                    ets:insert(E, {{urlc, Path}, 1}),\n                    ets:update_counter(E, num_files, 1),\n                    ets:update_counter(E, num_bytes, cache_size(UT2)),\n                    UT2\n            end\n    end;\ncache_file(_SC, _GC, _Path, UT) ->\n    UT.\n\n\n\n%% FIXME, should not wipe entire ets table this way\ncleanup_cache(E, size) ->\n    %% remove the largest files with the least hit count  (urlc)\n    ?Debug(\"Clearing yaws internal content \"\n           \"cache, size overflow\",[]),\n    clear_ets(E);\n\ncleanup_cache(E, num) ->\n    ?Debug(\"Clearing yaws internal content \"\n           \"cache, num overflow\",[]),\n    clear_ets(E).\n\n\n\n%% Clear everything, but *not* the Yaws specs, because otherwise we\n%% would have orphan modules loaded.\nclear_ets(E) ->\n    ets:match_delete(E, {{url, '_'}, '_', '_'}),\n    ets:match_delete(E, {{urlc, '_'}, '_', '_'}),\n    ets:insert(E, {num_files, 0}),\n    ets:insert(E, {num_bytes, 0}).\n\n\n%% return #urltype record\ndo_url_type(SC, GetPath, ArgDocroot, VirtualDir) ->\n    ?Debug(\"do_url_type SC=~s~nGetPath=~p~nVirtualDir=~p~n\",\n           [?format_record(SC,sconf), GetPath,VirtualDir]),\n\n    case GetPath of\n        _ when ?sc_has_dav(SC) ->\n            {Comps, RevFile} = comp_split(GetPath),\n            {_Type, Mime} = suffix_type(SC, RevFile),\n\n            FullPath = construct_fullpath(ArgDocroot, GetPath, VirtualDir),\n\n            %%!!WARNING!!!\n            %%!TODO - review & test!\n            %%Implications of vdirs on DAV have not yet been fully\n            %% considered by author of vdir support (JMN)\n\n            #urltype{type = dav,\n                     dir = conc_path(Comps),\n                     getpath = GetPath,\n                     path = GetPath,\n                     fullpath = FullPath,\n                     mime = Mime};\n        \"/\" -> %% special case\n            case lists:keysearch(\"/\", 1, SC#sconf.appmods) of\n                {value, AppmodDef} ->\n                    %% Remove appmod for this request to avoid an infinte loop\n                    %% in case of a reentrant call\n                    put(sc, SC#sconf{appmods=lists:delete(\n                                               AppmodDef, SC#sconf.appmods\n                                              )}),\n\n                    %% AppmodDef can be either a 2-tuple or 3-tuple depending\n                    %% on whether there are exclude paths present. We want\n                    %% only the second element of the tuple in either case.\n                    Mod = element(2, AppmodDef),\n                    #urltype{type = appmod,\n                             data = {Mod, []},\n                             dir = \"\",\n                             path = \"\",\n                             fullpath = ArgDocroot};\n                _ ->\n                    maybe_return_dir(ArgDocroot, GetPath, VirtualDir)\n            end;\n        [$/, $~ |Tail] ->\n            ret_user_dir(Tail);\n        _ ->\n            FullPath = construct_fullpath(ArgDocroot, GetPath, VirtualDir),\n\n            {Comps, RevFile} = comp_split(GetPath),\n            ?Debug(\"Comps = ~p RevFile = ~p~n\",[Comps, RevFile]),\n\n            RequestSegs = string:tokens(GetPath,\"/\"),\n            case active_appmod(SC#sconf.appmods, RequestSegs) of\n                false ->\n                    ?Debug(\"FullPath = ~p~n\", [FullPath]),\n\n                    case prim_file:read_file_info(FullPath) of\n                        {ok, FI} when FI#file_info.type == regular ->\n                            {Type, Mime} = suffix_type(SC, RevFile),\n                            #urltype{type=Type,\n                                     finfo=FI,\n                                     deflate=deflate_q(?sc_has_deflate(SC),\n                                                       SC, Type, Mime),\n                                     dir = conc_path(Comps),\n                                     path = GetPath,\n                                     getpath = GetPath,\n                                     fullpath = FullPath,\n                                     mime=Mime};\n                        {ok, FI} when FI#file_info.type == directory ->\n                            case RevFile of\n                                [] ->\n                                    maybe_return_dir(ArgDocroot, GetPath,\n                                                     VirtualDir);\n                                _ ->\n                                    %%Presence of RevFile indicates dir url\n                                    %% had no trailing /\n                                    #urltype{\n                                  type = redir,\n                                  path = [GetPath, \"/\"]}\n                            end;\n                        _Err ->\n                            %% non-optimal, on purpose\n                            maybe_return_path_info(SC, Comps, RevFile,\n                                                   ArgDocroot, VirtualDir)\n                    end;\n                {ok, {Mount, Mod}} ->\n                    %% Remove appmod for this request to avoid an infinte loop\n                    %% in case of a reentrant call\n                    put(sc, SC#sconf{appmods=lists:keydelete(\n                                               Mount, 1, SC#sconf.appmods\n                                              )}),\n\n                    %%active_appmod found the most specific appmod for this\n                    %% request path\n                    %% - now we need to determine the prepath & path_info\n\n                    MountSegs = string:tokens(Mount,\"/\"),\n\n                    case Mount of\n                        [$/] ->\n                            %%'root' appmod\n                            PostSegments = lists:sublist(RequestSegs,1,\n                                                         length(RequestSegs)),\n                            Prepath = \"\";\n                        [$/|_] ->\n                            %%'anchored' appmod mount.\n                            PreSegments = lists:sublist(RequestSegs,\n                                                        length(MountSegs)-1),\n                            PostSegments = lists:sublist(RequestSegs,\n                                                         length(MountSegs)+1,\n                                                         length(RequestSegs)),\n                            Prepath = case PreSegments of\n                                          \"\" ->\n                                              \"/\";\n                                          _ ->\n                                              \"/\" ++\n                                                  yaws:join_sep(PreSegments,\"/\")\n                                                  ++ \"/\"\n                                      end;\n                        _ ->\n                            %%'floating' appmod mount.\n                            {PreSegments,PostSegments} =\n                                split_at_segment(Mount,RequestSegs,[]),\n                            Prepath = case PreSegments of\n                                          \"\" ->\n                                              \"/\";\n                                          _ ->\n                                              \"/\" ++\n                                                  yaws:join_sep(PreSegments,\"/\")\n                                                  ++ \"/\"\n                                      end\n                    end,\n                    PathI = case PostSegments of\n                                [] ->\n                                    \"\";\n                                _ ->\n                                    \"/\" ++ yaws:join_sep(PostSegments,\"/\")\n                            end,\n                    %%absence of RevFile tells us there was a trailing slash.\n                    PathInf = case RevFile of\n                                  [] ->\n                                      PathI ++ \"/\";\n                                  _ ->\n                                      PathI\n                              end,\n                    PathInfo = case PathInf of\n                                   \"\" ->\n                                       undefined;\n                                   _ ->\n                                       PathInf\n                               end,\n                    Path = case MountSegs of\n                               [] ->\n                                   %%'root' appmod\n                                   Prepath;\n                               _ ->\n                                   Prepath ++ tl(MountSegs)\n                           end,\n\n                    #urltype{\n                             type = appmod,\n                             data = {Mod, PathInfo},\n                             dir = Prepath,\n                             path = Path,\n                             fullpath = FullPath,\n                             pathinfo = PathInfo\n                            }\n\n            end\n    end.\n\n\n%% comp_split/1 - split a path around \"/\" returning final segment as\n%% reversed string.\n%% return {Comps, RevPart} where Comps is a (possibly empty) list of path\n%% components - always with trailing \"/\"\n%% revPart is the final segment in reverse and has no \"/\".\n%% e.g split( \"/test/etc/index.html\",[],[]) ->\n%%     {[\"/test/\", \"etc/\"], \"lmth.xedni\"}\n%% revPart is useful in this form for looking up the file extension's mime-type.\n%%\n%% Terminology note to devs: reserve the word 'comp' to refer to a single\n%% fragment of a path that we know has\n%% a trailing slash. If you're dealing just with the part between slashes -\n%% consider using the term 'segment' instead.\n%% e.g  \"x/\" \"/\"   are all valid 'comps'\n%% \"/x\" \"/x/y/\" \"x\" are not.\n%%\ncomp_split(Path) ->\n    do_comp_split(Path,[],[]).\n\n%%when Part /= []\ndo_comp_split([$/|Tail], Comps, Part) ->\n    NewComp = lists:reverse([$/|Part]),\n    do_comp_split(Tail,  [NewComp | Comps], []);\ndo_comp_split([H|T], Comps, Part)  ->\n    do_comp_split(T, Comps, [H|Part]);\ndo_comp_split([], Comps, Part) ->\n    {lists:reverse(Comps), Part}.\n\n\n%%active_appmod/2\n%%find longest appmod match for request. (ie 'most specific' appmod)\n%% - conceptually similar to the vdirpath scanning - but must also support\n%% 'floating appmods' i.e an appmod specified as <path , appmodname> where\n%% 'path' has no leading slash.\n%%\n%% a 'floating' appmod is not tied to a specific point in the URI structure\n%% e.g for the configuration entry <myapp , myappAppmod>\n%% the requests /docs/stuff/myapp/etc  & /otherpath/myapp   will both\n%% trigger the myappAppmod module.\n%% whereas for the configuration entry </docs/stuff/myapp , myappAppmod>\n%% the request /otherpath/myapp will not trigger the appmod.\n%%\n\nactive_appmod([], _RequestSegs) ->\n    false;\nactive_appmod(AppMods, RequestSegs) ->\n\n    %%!todo - review/test performance (e.g 'fun' calls are slower than a\n    %% call to a local func - replace?)\n\n    %%Accumulator is of form {RequestSegs, {AppmodMountPoint,Mod}}\n    Matched =\n        lists:foldl(\n          fun(Pair,Acc) ->\n                  {Mount, Mod, Excludes} = case Pair of\n                                               {X, Y} -> {X, Y, []};\n                                               {X,Y,Z} -> {X,Y,Z}\n                                           end,\n                  {ReqSegs, {LongestSoFar, _}} = Acc,\n\n                  MountSegs = string:tokens(Mount,\"/\"),\n                  case {is_excluded(ReqSegs, Excludes) ,\n                        lists:prefix(MountSegs,ReqSegs)} of\n                      {true, _} ->\n                          Acc;\n                      {false, true} ->\n                          case LongestSoFar of\n                              [$/|_] ->\n                                  %%simple comparison of string length\n                                  %% (as opposed to number of segments)\n                                  %% should be ok here.\n                                  if length(Mount) >\n                                     length(LongestSoFar) ->\n                                          {ReqSegs, {Mount, Mod}};\n                                     true ->\n                                          Acc\n                                  end;\n                              _ ->\n                                  %%existing match is 'floating' -\n                                  %% we trump it.\n\n                                  {ReqSegs, {Mount, Mod}}\n                          end;\n                      {false, false} ->\n                          case LongestSoFar of\n                              [$/|_] ->\n                                  %%There is already a match for an\n                                  %% 'anchored' (ie absolute path)\n                                  %% mount point.\n                                  %% floating appmod can't override.\n                                  Acc;\n                              _ ->\n                                  %%check for 'floating' match\n                                  case lists:member(Mount, ReqSegs) of\n                                      true ->\n                                          %%!todo - review & document.\n                                          %%latest 'floating' match wins\n                                          %% if multiple match?\n                                          %% (order in config vs position\n                                          %% in request URI ?)\n\n                                          {ReqSegs, {Mount, Mod}};\n                                      false ->\n                                          Acc\n                                  end\n                          end\n                  end\n          end, {RequestSegs, {\"\",\"\"}}, AppMods),\n\n    case Matched of\n        {_RequestSegs, {\"\",\"\"}} ->\n            %%no appmod corresponding specifically to this http_request.path\n            false;\n        {_RequestSegs, {Mount, Mod}} ->\n            {ok, {Mount, Mod}}\n    end\n        .\n\nis_excluded(_, []) ->\n    false;\nis_excluded(RequestSegs, [ExcludeSegs|T]) ->\n    case lists:prefix(ExcludeSegs, RequestSegs) of\n        true ->\n            true;\n        false ->\n            is_excluded(RequestSegs, T)\n    end.\n\n\n%%split a list of segments into 2 lists either side of element matching Seg.\n%%(no elements contain slashes)\nsplit_at_segment(_, [], _Acc) ->\n    false;\nsplit_at_segment(Seg,[Seg|Tail],Acc) ->\n    {lists:reverse(Acc),Tail};\nsplit_at_segment(Seg,[H|Tail],Acc) ->\n    split_at_segment(Seg, Tail, [H|Acc]).\n\n\n\n\n%% construct_fullpath\n%%\n%%preconditions:\n%% - DR, GetPath, VirtualDir already validated &/or normalized\n%% - VirtualDir is empty string, or a prefix of GetPath of the form \"/path/\"\n%% where path may also contain \"/\"\n%% - DocRoot is a valid physical path to a directory, with no trailing \"/\"\n%%\n%%i.e this is an inner function, so no sanity checks here.\n%%\nconstruct_fullpath(undefined,_,_) ->\n    undefined;\nconstruct_fullpath(DocRoot,GetPath,VirtualDir) ->\n    case VirtualDir of\n        [] ->\n            DocRoot ++ GetPath;\n        _ ->\n            %%trim the virtual base off the GET request path before appending\n            %% to DocRoot.\n            %%(leaving one \"/\" - therefore don't add 1 to length)\n            DocRoot ++ string:substr(GetPath,length(VirtualDir))\n    end\n        .\n\n%%preconditions:\n%% - see 'construct_fullpath'\n%%\ntry_index_file(_FullPath, _GetPath, []) ->\n    noindex;\ntry_index_file(FullPath, GetPath, [[$/|_]=Idx|Rest]) ->\n    case (GetPath =:= Idx orelse GetPath =:= Idx++\"/\") of\n        true  -> try_index_file(FullPath, GetPath, Rest);\n        false -> {redir, Idx}\n    end;\ntry_index_file(FullPath, GetPath, [Idx|Rest]) ->\n    case prim_file:read_file_info([FullPath, Idx]) of\n        {ok, FI} when FI#file_info.type == regular ->\n            {index, Idx};\n        _ ->\n            try_index_file(FullPath, GetPath, Rest)\n    end.\n\n\nmaybe_return_dir(DR, GetPath,VirtualDir) ->\n    SC = get(sc),\n    FullPath = construct_fullpath(DR, GetPath, VirtualDir),\n    case try_index_file(FullPath, GetPath, SC#sconf.index_files) of\n        {index, Idx} ->\n            do_url_type(SC, GetPath ++ Idx, DR, VirtualDir);\n        {redir, NewPath} ->\n            #urltype{type=redir, path=NewPath};\n        noindex ->\n            case file:list_dir(FullPath) of\n                {ok, List} ->\n                    #urltype{type     = directory,\n                             fullpath = FullPath,\n                             dir      = GetPath,\n                             data     = List -- [\".yaws_auth\"]};\n                _Err ->\n                    #urltype{type=error}\n            end\n    end.\n\n\n\nmaybe_return_path_info(SC, Comps, RevFile, DR, VirtualDir) ->\n\n    case path_info_split(SC, Comps, {DR, VirtualDir}) of\n        {not_a_script, error} ->\n            %%can we use urltype.data to return more info?\n            %% - logging?\n            #urltype{type=error};\n        {ok, FI, FullPath, HeadComps, File, TrailComps, Type, Mime} ->\n            %%'File' is the only comp that has been returned\n            %% without trailing \"/\"\n\n            {Type2, Mime2} =\n                case member(Type, SC#sconf.allowed_scripts) of\n                    true ->\n                        {Type, Mime};\n                    false ->\n                        %%!todo review.\n                        %%Should we really be returning the file as text/plain\n                        %% when there is pathinfo present?\n                        %%Perhaps a 403 error would be more appropriate.\n                        {regular, \"text/plain\"}\n                end,\n\n            ?Debug(\"'script-selection' FullPath= ~p~n Mime=~p~n\",\n                   [FullPath, Mime2]),\n\n            Trail = conc_path([ \"/\" ] ++ TrailComps ++\n                              [ lists:reverse(RevFile) ]),\n\n\n            #urltype{type = Type2,\n                     finfo=FI,\n                     deflate=deflate_q(?sc_has_deflate(SC),\n                                       SC, Type, Mime),\n                     dir =  conc_path(HeadComps),\n                     path = conc_path(HeadComps ++ [File]),\n                     fullpath = FullPath,\n                     pathinfo = Trail,\n                     getpath = case HeadComps of\n                                   [] -> [$/|File];\n                                   [_|_] ->\n                                       conc_path(HeadComps ++ [File])\n                               end,\n                     mime = Mime2}\n    end.\n\n\n%%scan a list of 'comps' of form \"pathsegment/\"\n%% (trailing slash always present)\n%% - looking for the rightmost dotted component that corresponds to a script\n%% file.\n\n%% By the time path_info_split is called - the fullpath has already been tested\n%%  and found not to be a file or directory\n%%\n%% Limitation: we don't support a script file without a dot.\n%%  - otherwise we'd have to hit the filesystem for too many path components\n%% to see if they exist & are an executable file.\n%%\n%% !!todo - review (potential security issue).\n%% Right-to-left scanning should stop once we reach a 'document root mount\n%% point', otherwise the Docroot that has been determined based on the full\n%%  request path becomes invalid!\n%%\npath_info_split(SC, Comps,DR_Vdir) ->\n    path_info_split(SC, lists:reverse(Comps), DR_Vdir, []).\n\npath_info_split(SC, [H|T], {DR, VirtualDir}, AccPathInfo) ->\n    [$/|RevPath] = lists:reverse(H),\n    case suffix_from_rev(RevPath) of\n        [] ->   % shortcut clause, not necessary\n            path_info_split(SC, T, {DR, VirtualDir}, [H|AccPathInfo]);\n        Suff ->\n            {Type, Mime} = mime_types:t(SC, Suff),\n            case Type of\n                regular ->\n                    %%Don't hit the filesystem to test components that\n                    %%'mime_types' indicates can't possibly be scripts\n                    path_info_split(SC, T, {DR, VirtualDir}, [H|AccPathInfo]);\n                X ->\n\n                    %%We may still be in the 'PATH_INFO' section\n                    %%Test to see if it really is a script\n\n                    TestPath = lists:flatten(lists:reverse(T)),\n                    FullPath = construct_fullpath(DR, TestPath, VirtualDir) ++\n                        string:strip(H,right,$/),\n\n                    ?Debug(\"Testing for script at: ~p~n\", [FullPath]),\n\n                    case prim_file:read_file_info(FullPath) of\n                        {ok, FI} when FI#file_info.type == regular ->\n                            {ok, FI, FullPath, lists:reverse(T),\n                             string:strip(H,right,$/), AccPathInfo, X, Mime};\n                        {ok, FI} when FI#file_info.type == directory ->\n                            %%just a case of a bad path starting at this point.\n                            {not_a_script, error};\n                        _Err ->\n                            %%just looked like a script - keep going.\n                            path_info_split(SC, T, {DR, VirtualDir},\n                                            [H|AccPathInfo])\n                    end\n            end\n    end;\npath_info_split(_SC, [], _DR_Vdir, _Acc) ->\n    {not_a_script, error}.\n\n\nsuffix_from_rev(R) ->\n    suffix_from_rev(R, []).\n\nsuffix_from_rev([$.|_], A) ->\n    A;\nsuffix_from_rev([C|T], A) ->\n    suffix_from_rev(T, [C|A]);\nsuffix_from_rev([], _A) ->\n    [].\n\n%%conc_path\n%% - single-level concatenatenation of a list of path components which\n%% already contain slashes.\n%% tests suggest it's significantly faster than lists:flatten or lists:concat\n%% & marginally faster than lists:append\n%% (for paths of 3 or more segments anyway)\n%% tested with various fairly short path lists - see src/benchmarks folder\n%%\n\n%%Original\nconc_path([]) ->\n    [];\nconc_path([H|T]) ->\n    H ++ conc_path(T).\n\n%% tail-recursive version slower for longer paths according to bench.erl\n%% (mainly because we need to do 'Acc ++ H' rather than 'H ++ Acc')\n%% Tail recursion not very useful here anyway as we're\n%% dealing with short strings.\n%%conc_path2([]) ->\n%%        [];\n%%conc_path2([H|T]) ->\n%%        cpath(T,H).\n\n%%cpath([],Acc) ->\n%%        Acc;\n%%cpath([H|[]],Acc) ->\n%%        H ++ Acc;\n%%cpath([H|T],Acc) ->\n%%        cpath(T,Acc ++ H).\n\n\n%% ret_app_mod(Path, Mod, PrePath) ->\n%%     #urltype{type = appmod,\n%%              data = {Mod, Path},\n%%              path = PrePath}.\n\n\n\n%% http://a.b.c/~user URLs\nret_user_dir(Upath)  ->\n    ?Debug(\"ret_user_dir ~p~n\", [Upath]),\n    SC = get(sc),\n    if ?sc_has_tilde_expand(SC) ->\n            case parse_user_path(SC#sconf.docroot, Upath, []) of\n                {ok, User, Path} ->\n                    %% FIXME doesn't work if passwd contains ::\n                    %% also this is unix only\n                    %% and it ain't the fastest code around.\n                    case catch yaws:user_to_home(User) of\n                        {'EXIT', _} ->\n                            #urltype{type=error};\n                        Home ->\n                            DR2 = Home ++ \"/public_html/\",\n                            SC2 = SC#sconf{\n                                    allowed_scripts =\n                                        SC#sconf.tilde_allowed_scripts,\n                                    docroot=DR2},\n                            put(sc, SC2),\n\n                            %% !todo - review interactions between Virtual\n                            %% Dirs & Home Dir paths.\n                            %% VirtualDir hardcoded empty is not\n                            %% nice behaviour -\n                            %% a rewrite mod author may reasonably expect to\n                            %% be able to have influence here.\n\n                            redir_user(do_url_type(SC2, Path, DR2,\"\"), User)\n                            %% recurse\n                    end;\n                {redir_dir, User} ->\n                    #urltype {type = redir,\n                              path = [\"/~\", User, \"/\"]}\n            end;\n       true ->\n            #urltype{type=error}\n    end.\n\n\nredir_user(UT, User) ->\n    case UT#urltype.type of\n        redir ->\n            UT#urltype{path = [\"/~\", User, UT#urltype.path]};\n        _ ->\n            UT\n    end.\n\n\n\n\nparse_user_path(_DR, [], User) ->\n    {redir_dir, reverse(User)};\nparse_user_path(_DR, [$/], User) ->\n    {ok, reverse(User), [$/]};\nparse_user_path(_DR, [$/|Tail], User) ->\n    {ok, reverse(User), [$/|Tail]};\nparse_user_path(DR, [H|T], User) ->\n    parse_user_path(DR, T, [H|User]).\n\n\ndeflate_q(true, SC, regular, Mime0) ->\n    [Mime1|_] = yaws:split_sep(Mime0, $;), %% Remove charset\n    case compressible_mime_type(Mime1, SC#sconf.deflate_options) of\n        true -> dynamic;\n        false -> undefined\n    end;\ndeflate_q(_, _, _, _) ->\n    undefined.\n\n\nsuffix_type(SC, L) ->\n    case mime_types:revt(SC, yaws:upto_char($., L)) of\n        {regular, _Ext, Mime} ->\n            {regular, Mime};\n        {X, _Ext, Mime} ->\n            case member(X, SC#sconf.allowed_scripts) of\n                true  -> {X, Mime};\n                false -> {regular, mime_types:default_type(SC)}\n            end\n    end.\n\n\ncompressible_mime_type(Mime, #deflate{mime_types=MimeTypes}) ->\n    case yaws:split_sep(Mime, $/) of\n        [Type, SubType] -> compressible_mime_type(Mime,Type,SubType,MimeTypes);\n        _               -> false\n    end.\n\ncompressible_mime_type(_, _, _, all) ->\n    true;\ncompressible_mime_type(_, _, _, []) ->\n    false;\ncompressible_mime_type(_, Type, _, [{Type, all}|_]) ->\n    true;\ncompressible_mime_type(_, Type, SubType, [{Type, SubType}|_]) ->\n    true;\ncompressible_mime_type(Mime, _, _, [Mime|_]) ->\n    true;\ncompressible_mime_type(Mime, Type, SubType, [_|Rest]) ->\n    compressible_mime_type(Mime, Type, SubType, Rest).\n\n\n\nflush(Sock, Sz, TransferEncoding) ->\n    flush(Sock, 0, Sz, TransferEncoding).\n\nflush(Sock, Pos, Sz, \"chunked\") ->\n    SC = get(sc),\n    case get_chunked_client_data(Sock, yaws:is_ssl(SC)) of\n        {partial, Bin} -> flush(Sock, Pos+size(Bin), Sz, \"chunked\");\n        _              -> Pos\n    end;\nflush(_Sock, Pos, undefined, _) ->\n    Pos;\nflush(Sock, Pos, Sz, TE) when is_list(Sz) ->\n    flush(Sock, Pos, strip_list_to_integer(Sz), TE);\nflush(Sock, Pos, Sz, _) ->\n    SC = get(sc),\n    flush(Sock, Pos, Sz, yaws:is_ssl(SC), SC#sconf.partial_post_size).\n\n\nflush(_Sock, Sz, Sz, _SSL, _PPS) ->\n    Sz;\nflush(Sock, Pos, Sz, SSL, PPS) ->\n    case yaws:do_recv(Sock, erlang:min(Sz - Pos, PPS), SSL) of\n        {ok, Bin} -> flush(Sock, Pos + size(Bin), Sz, SSL, PPS);\n        _         -> Pos\n    end.\n\n\nstrip_list_to_integer(L) ->\n    case catch list_to_integer(L) of\n        {'EXIT', _} ->\n            list_to_integer(string:strip(L, both));\n        Int ->\n            Int\n    end.\n\n\nmtime(F) ->\n    F#file_info.mtime.\n\nrunmod({ok, Mod}, GC) ->\n    runmod2(GC, [Mod | GC#gconf.runmods]);\nrunmod(_, GC) ->\n    runmod2(GC, GC#gconf.runmods).\n\nrunmod2(GC, Mods) ->\n    foreach(fun(M) ->\n                    proc_lib:spawn(?MODULE, load_and_run,\n                                   [M, ?gc_has_debug(GC)])\n            end, Mods).\n\n\n\nload_and_run(Mod, Debug) ->\n    case code:ensure_loaded(Mod) of\n        {module,Mod} when Debug == false ->\n            Mod:start();\n        {module,Mod} when Debug == true  ->\n            error_logger:info_msg(\"sync call ~p:start ~n\",[Mod]),\n            Mod:start();\n        Error ->\n            error_logger:error_msg(\"Loading '~w' failed, reason ~p~n\",\n                                   [Mod,Error])\n    end.\n\n\nsafe_ehtml_expand(X) ->\n    case (catch ehtml_expand(X)) of\n        {'EXIT', R} ->\n            {error, err_pre(R)};\n        Val ->\n            {ok, Val}\n    end.\n\nerr_pre(R) ->\n    io_lib:format(\"<pre> ~n~p~n </pre>~n\", [R]).\n\n%% mappath/3    (virtual-path to physical-path)\n%% - this returns physical path a URI would map to, taking into consideration\n%%   vdirs and assuming each path segment of the URI represents a folder\n%%   (or maybe filename at end).\n%%   ie it does not (and is not intended to) take into account 'script points'\n%%   in the path. (cgi,fcgi,php,appmod etc)\n%%   The result may not actually exists as a path.\n%%\n%% mappath/3 is analogous to the Microsoft ASP function Server.MapPath or\n%% the 'filename' array member of the result\n%% of the PHP function 'apache_lookup_uri'.\n%%\nmappath(SC, ARG, RequestPath) ->\n    {VirtualDir, DR} = vdirpath(SC, ARG, RequestPath),\n    PhysicalPath = construct_fullpath(DR, RequestPath, VirtualDir),\n    %% Resultant path might not exist - that's not the concern of the\n    %%  'mappath' function.\n    PhysicalPath.\n\n\n%% vdirpath/3\n%% find longest \"vdir\" match.\n%%  (ie a 'document-root mount-point' -> DOCUMENT_ROOT_MOUNT)\n%%\n%% e.g if we have in our .conf:\n%%    vdir = \"/app/  /path1/somewhere\"\n%%    vdir = \"/app/test/shared/ /path2/somewhere\"\n%%\n%% A request path of /app/test/doc.html  must be served from under\n%% /path1/somewhere\n%% /app/test/shared/doc.html will be served from under /path2/somewhere\n%%\n%% Also must be able to handle:\n%%   vdir = \"/somewhere/ /path3/has spaces/in path/docs\"\n%% In this case, the 1st space separates the vdir from the physical path\n%%  i.e subsequent spaces are part of the path.\n\nvdirpath(SC, ARG, RequestPath) ->\n    Opaquelist = ARG#arg.opaque,\n    %% !todo - move out of opaque.\n    %%  We don't want to scan all opaque entries each time\n    %%  - vdir directives should be pre-collated into a list somewhere.\n    %%  (own field in sconf record)\n\n\n    RequestSegs = string:tokens(RequestPath,\"/\"),\n\n    %% Accumulator is of form {RequestSegs,{VdirMountPoint,VdirPhysicalPath}}\n    Matched =\n        lists:foldl(\n          fun(ListItem,Acc) ->\n                  case ListItem of\n                      {\"vdir\",Vmap} ->\n\n                          {ReqSegs,VdirSpec} = Acc,\n\n                          [Virt |PhysParts] = string:tokens(Vmap,\" \\t\"),\n                          VirtSegs = string:tokens(Virt,\"/\"),\n                          case lists:prefix(VirtSegs,ReqSegs) of\n                              true ->\n                                  {LongestSoFar,_} = VdirSpec,\n                                  if length(Virt) > length(LongestSoFar) ->\n                                          %% reassemble (because physical\n                                          %% path may have spaces)\n                                          Phys = yaws:join_sep(PhysParts, \" \"),\n\n                                          {ReqSegs, {Virt, Phys}};\n                                     true ->\n                                          Acc\n                                  end;\n                              false ->\n                                  Acc\n                          end;\n                      _Else ->\n                          %% irrelevant member of opaque list. no change in\n                          %% accumulator\n                          Acc\n                  end\n          end, {RequestSegs,{\"\",\"\"}}, Opaquelist),\n\n\n    case Matched of\n        {_RequestSegs, {\"\",\"\"}} ->\n            %% no virtual dir corresponding to this http_request.path\n            %% NOTE - we *don't* know that the state of ARG#arg.docroot\n            %% currently reflects the main docroot\n            %% specified for the virtual server in the conf file.\n            %% This is because we may be being called from a page that is\n            %% under a vdir, and so docroot may\n            %% have been rewritten. It may also have been rewritten by an\n            %% appmod or arg_rewrite_mod.\n            %% Therefore we need to get it directly from the sconf record.\n\n            Result = {\"\",SC#sconf.docroot};\n        {_RequestSegs, {Virt,DocRoot }} ->\n            %%sanitize Virt & DocRoot so that they are correct with\n            %% regards to leading & trailing slashes\n            case string:right(Virt,1) of\n                \"/\" ->\n                    VirtualDir = Virt;\n                _ ->\n                    VirtualDir = Virt ++ \"/\"\n            end,\n            DR = string:strip(DocRoot,right,$/),\n\n            Result = {VirtualDir, DR}\n    end,\n\n    %% return {VdirURI, Physpath}  - i.e tuple representing the data\n    %% specified in conf file for the 'vdir' directive.\n    Result.\n\nclose_accepted_if_max(GS,{ok, _Socket})\n  when (GS#gs.gconf)#gconf.max_connections == nolimit ->\n    ok;\nclose_accepted_if_max(GS,{ok, Socket}) ->\n    MaxCon = (GS#gs.gconf)#gconf.max_connections,\n    NumCon = GS#gs.connections,\n    if\n        NumCon < MaxCon ->\n            ok;\n        true ->\n            S=case peername(Socket, GS#gs.ssl) of\n                  {unknown, unknown} ->\n                      \"unknown\";\n                  {IP, Port} ->\n                      io_lib:format(\"~s:~w\", [inet_parse:ntoa(IP), Port])\n              end,\n            error_logger:format(\n              \"Max connections reached - closing conn to ~s~n\",[S]),\n            if\n                GS#gs.ssl == nossl -> gen_tcp:close(Socket);\n                GS#gs.ssl == ssl   -> ssl:close(Socket)\n            end\n\n    end;\nclose_accepted_if_max(_,_) ->\n    ok.\n\ncode_change(_OldVsn, Data, _Extra) ->\n    {ok, Data}.\n\n\nfwdproxy_url(ARG) ->\n    Headers = ARG#arg.headers,\n    {abs_path, Path} = (ARG#arg.req)#http_request.path,\n\n    {Host0, Port0} = yaws:split_at(Headers#headers.host, $:),\n    {Host, Port} = case string:to_integer(Port0) of\n                       {Port1, []} ->\n                           {Host0, Port1};\n                       _ ->\n                           {Headers#headers.host, undefined}\n                   end,\n\n    #url{scheme = http,\n         host = Host,\n         port = Port,\n         path = Path}.\n\n\n\nmaybe_set_page_options() ->\n    case erase(page_options) of\n        undefined ->\n            ok;\n        Options ->\n            deepforeach(\n              fun(X) -> case X of\n                            {header, Header} ->\n                                yaws:accumulate_header(Header);\n                            {status, Code} ->\n                                yaws:outh_set_status_code(Code);\n                            _Other ->\n                                ?Debug(\"Got ~p in page option list.\", [_Other])\n                        end\n              end, Options)\n    end.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_session_server.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_session_server.erl\n%%% Author  : Claes Wikstrom <klacke@hyber.org>\n%%% Purpose : maintain state for cookie sessions\n%%% Created : 17 Sep 2002 by Claes Wikstrom <klacke@hyber.org>\n%%%----------------------------------------------------------------------\n\n-module(yaws_session_server).\n-author('klacke@hyber.org').\n\n\n-behaviour(gen_server).\n\n%% External exports\n-export([start_link/0, start/0, stop/0]).\n\n%% gen_server callbacks\n-export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2,\n         code_change/3]).\n\n-include(\"../include/yaws_api.hrl\").\n-include(\"../include/yaws.hrl\").\n\n-export([new_session/1,new_session/2,new_session/3,new_session/4,\n         cookieval_to_opaque/1,\n         print_sessions/0,\n         replace_session/2, replace_session/3,\n         delete_session/1]).\n\n%% Default ETS backend callbacks\n-export ([init_backend/1, stop_backend/0,\n          list/0, lookup/1, insert/1, delete/1,\n          traverse/1, cleanup/0]).\n\n%% Utility functions for callbacks\n-export ([has_timedout/2, report_timedout_sess/1, cookie/1]).\n\n-define(TTL, (30 * 60)).  % 30 minutes\n\n-record(ysession,\n        {cookie,       %% the cookie assigned to the session\n         to,           %% greg secs untill timeout death\n         ttl,          %% default time to live\n         starttime,    %% When calendar:local_time() did sess start\n         cleanup,      %% PID to notify of session end\n         opaque        %% any data the user supplies\n        }).\n\n-record(state,\n        {backend      %% storage engine module\n        }).\n\n\n%%%----------------------------------------------------------------------\n%%% API\n%%%----------------------------------------------------------------------\nstart_link() ->\n    Backend = get_yaws_session_server_backend(),\n    gen_server:start_link({local, yaws_session_server},\n                          yaws_session_server, Backend, []).\nstart() ->\n    Backend = get_yaws_session_server_backend(),\n    gen_server:start({local, yaws_session_server},\n                     yaws_session_server, Backend, []).\nstop() ->\n    gen_server:call(?MODULE, stop, infinity).\n\n\n%% We are bending over here in our pursuit of finding a\n%% proper ysession_server backend.\nget_yaws_session_server_backend() ->\n    #gconf{ysession_mod = DefaultBackend} = #gconf{},\n    case yaws_server:getconf() of\n        {ok, #gconf{ysession_mod = Backend}, _} -> Backend;\n        _ ->\n            case application:get_env(yaws, embedded) of\n                {ok, true} ->\n                    case application:get_env(yaws, embedded_conf) of\n                        {ok, L} when is_list(L) ->\n                            case lists:keyfind(gc, 1, L) of\n                                {_, #gconf{ysession_mod = Backend}} ->\n                                    Backend;\n                                _ ->\n                                    DefaultBackend\n                            end;\n                        _ ->\n                            DefaultBackend\n                    end;\n                _ ->\n                    DefaultBackend\n            end\n    end.\n\n\n%% will return a new cookie as a string\nnew_session(Opaque) ->\n    Call = {new_session, Opaque, ?TTL, undefined, undefined},\n    gen_server:call(?MODULE, Call, infinity).\n\nnew_session(Opaque, TTL) ->\n    Call = {new_session, Opaque, TTL, undefined, undefined},\n    gen_server:call(?MODULE, Call, infinity).\n\nnew_session(Opaque, TTL, Cleanup) ->\n    Call = {new_session, Opaque, TTL, Cleanup, undefined},\n    gen_server:call(?MODULE, Call, infinity).\n\nnew_session(Opaque, TTL, Cleanup, Cookie) ->\n    Call = {new_session, Opaque, TTL, Cleanup, Cookie},\n    gen_server:call(?MODULE, Call, infinity).\n\ncookieval_to_opaque(Cookie) ->\n    gen_server:call(?MODULE, {cookieval_to_opaque, Cookie}, infinity).\n\nprint_sessions() ->\n    gen_server:cast(?MODULE, print_sessions).\n\nreplace_session(Cookie, NewOpaque) ->\n    gen_server:call(?MODULE, {replace_session, Cookie, NewOpaque, undefined},\n                    infinity).\nreplace_session(Cookie, NewOpaque, Cleanup) ->\n    gen_server:call(?MODULE, {replace_session, Cookie, NewOpaque, Cleanup},\n                    infinity).\n\ndelete_session(CookieVal) ->\n    gen_server:call(?MODULE, {delete_session, CookieVal}, infinity).\n\n%%%----------------------------------------------------------------------\n%%% Callback functions from gen_server\n%%%----------------------------------------------------------------------\n\n%%----------------------------------------------------------------------\n%% Func: init/1\n%% Returns: {ok, State}          |\n%%          {ok, State, Timeout} |\n%%          ignore               |\n%%          {stop, Reason}\n%%----------------------------------------------------------------------\ninit(Backend) ->\n    Backend:init_backend(record_info(fields, ysession)),\n    start_long_timer(),\n    {ok, #state{backend = Backend}, to()}.\n\n%%----------------------------------------------------------------------\n%% Func: handle_call/3\n%% Returns: {reply, Reply, State}          |\n%%          {reply, Reply, State, Timeout} |\n%%          {noreply, State}               |\n%%          {noreply, State, Timeout}      |\n%%          {stop, Reason, Reply, State}   | (terminate/2 is called)\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%----------------------------------------------------------------------\n\nhandle_call({new_session, Opaque, undefined, Cleanup, Cookie}, From, State) ->\n    handle_call({new_session, Opaque, ?TTL, Cleanup, Cookie}, From, State);\n\nhandle_call({new_session, Opaque, TTL, Cleanup, undefined}, From, State) ->\n    N = bin2int(yaws_dynopts:rand_bytes(16)),\n    Cookie = atom_to_list(node()) ++ [$-|integer_to_list(N)],\n    handle_call({new_session, Opaque, TTL, Cleanup, Cookie}, From, State);\n\nhandle_call({new_session, Opaque, TTL, Cleanup, Cookie}, _From, State) ->\n    Now = gnow(),\n    TS = calendar:local_time(),\n    NS = #ysession{cookie = Cookie,\n                   starttime = TS,\n                   opaque = Opaque,\n                   to = Now + TTL,\n                   ttl = TTL,\n                   cleanup = Cleanup},\n    Backend = State#state.backend,\n    true = Backend:insert(NS),\n    {reply, Cookie, State, to()};\n\nhandle_call({cookieval_to_opaque, Cookie}, _From, State) ->\n    Backend = State#state.backend,\n    Result =\n        case Backend:lookup(Cookie) of\n            [Y] ->\n                Y2 = Y#ysession{to = gnow() + Y#ysession.ttl},\n                Backend:insert(Y2),\n                {ok, Y#ysession.opaque};\n            [] ->\n                {error, no_session}\n        end,\n    {reply, Result, State, to()};\n\nhandle_call({replace_session, Cookie, NewOpaque}, _From, State) ->\n    handle_call({replace_session, Cookie, NewOpaque, undefined}, _From, State);\nhandle_call({replace_session, Cookie, NewOpaque, Cleanup}, _From, State) ->\n    Backend = State#state.backend,\n    Result =\n        case Backend:lookup(Cookie) of\n            [Y] ->\n                Y2 = Y#ysession{to = gnow() + Y#ysession.ttl,\n                                opaque = NewOpaque,\n                                cleanup = case Cleanup of\n                                              undefined ->\n                                                  Y#ysession.cleanup;\n                                              _ ->\n                                                  Cleanup\n                                          end},\n                Backend:insert(Y2);\n            [] ->\n                error\n        end,\n    {reply, Result, State, to()};\n\nhandle_call({delete_session, CookieVal}, _From, State) ->\n    Backend = State#state.backend,\n    Result =\n        case Backend:lookup(CookieVal) of\n            [Y] ->\n                Backend:delete(CookieVal),\n                report_deleted_sess(Y);\n            [] ->\n                true\n        end,\n    {reply, Result, State, to()};\n\nhandle_call(stop, From, State) ->\n    gen_server:reply(From, ok),\n    {stop, normal, State}.\n\n%%----------------------------------------------------------------------\n%% Func: handle_cast/2\n%% Returns: {noreply, State}          |\n%%          {noreply, State, Timeout} |\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%----------------------------------------------------------------------\nhandle_cast(print_sessions, #state{backend = Backend} = State) ->\n    Ss = Backend:list(),\n    io:format(\"** ~p sessions active ~n~n\", [length(Ss)]),\n    N = gnow(),\n    lists:foreach(fun(S) ->\n                          io:format(\"Cookie   ~p ~n\", [S#ysession.cookie]),\n                          io:format(\"Start    ~p ~n\", [S#ysession.starttime]),\n                          io:format(\"TTL      ~p secs~n\", [S#ysession.to - N]),\n                          io:format(\"Opaque   ~p ~n~n~n\", [S#ysession.opaque]),\n                          ok\n                  end, Ss),\n    {noreply, State, to()};\n\nhandle_cast(_Msg, State) ->\n    {noreply, State, to()}.\n\n%%----------------------------------------------------------------------\n%% Func: handle_info/2\n%% Returns: {noreply, State}          |\n%%          {noreply, State, Timeout} |\n%%          {stop, Reason, State}            (terminate/2 is called)\n%%----------------------------------------------------------------------\nhandle_info(timeout, #state{backend = Backend} = State) ->\n    Backend:traverse(gnow()),\n    {noreply, State, to()};\n\nhandle_info(long_timeout, #state{backend = Backend} = State) ->\n    Backend:traverse(gnow()),\n    start_long_timer(),\n    {noreply, State, to()}.\n\n\n%%----------------------------------------------------------------------\n%% Func: terminate/2\n%% Purpose: Shutdown the server\n%% Returns: any (ignored by gen_server)\n%%----------------------------------------------------------------------\nterminate(_Reason, #state{backend = Backend}) ->\n    Backend:stop_backend(),\n    ok.\n\n%%----------------------------------------------------------------------\n%% Func: code_change/3\n%% Purpose: Handle upgrade\n%% Returns: new State data\n%%----------------------------------------------------------------------\ncode_change(_OldVsn, Data, _Extra) ->\n    {ok, Data}.\n\n%%%----------------------------------------------------------------------\n%%% Internal functions\n%%%----------------------------------------------------------------------\n\nbin2int(Bin) ->\n    lists:foldl(fun(N, Acc) -> Acc * 256 + N end, 0, binary_to_list(Bin)).\n\n%% timeout once every hour even if the server handles traffic all the time.\nstart_long_timer() ->\n    erlang:send_after(long_to(), self(), long_timeout).\n\nlong_to() ->\n    Default = 60 * 60 * 1000,\n    try yaws_server:getconf() of\n        {ok, #gconf{ysession_long_timeout = LongTO}, _} ->\n            LongTO;\n        _ ->\n            case application:get_env(yaws, embedded) of\n                {ok, true} ->\n                    case application:get_env(yaws, embedded_conf) of\n                        {ok, L} when is_list(L) ->\n                            case lists:keyfind(gc, 1, L) of\n                                {_, #gconf{ysession_long_timeout = LongTO}} ->\n                                    LongTO;\n                                _ ->\n                                    Default\n                            end;\n                        _ ->\n                            Default\n                    end;\n                _ ->\n                    Default\n            end\n    catch\n        _:_ ->\n            %% server not running yet; timeout quickly so we can try again\n            10 * 1000\n    end.\n\n%% timeout if the server is idle for more than 2 minutes.\nto() ->\n    Default = 2 * 60 * 1000,\n    case catch yaws_server:getconf() of\n        {ok, #gconf{ysession_idle_timeout = IdleTO}, _} ->\n            IdleTO;\n        _ ->\n            case application:get_env(yaws, embedded) of\n                {ok, true} ->\n                    case application:get_env(yaws, embedded_conf) of\n                        {ok, L} when is_list(L) ->\n                            case lists:keyfind(gc, 1, L) of\n                                {_, #gconf{ysession_idle_timeout = IdleTO}} ->\n                                    IdleTO;\n                                _ ->\n                                    Default\n                            end;\n                        _ ->\n                            Default\n                    end;\n                _ ->\n                    Default\n            end\n    end.\n\ngnow() ->\n    calendar:datetime_to_gregorian_seconds(\n      calendar:local_time()).\n\nsend_cleanup_message(Sess,Msg) ->\n    case Sess#ysession.cleanup of\n        undefined ->\n            nocleanup;\n        Pid ->\n            Pid ! Msg\n    end.\n\nreport_timedout_sess(S) ->\n    send_cleanup_message(S,{yaws_session_end,timeout,\n                            S#ysession.cookie, S#ysession.opaque}).\n\nreport_deleted_sess(S) ->\n    send_cleanup_message(S,{yaws_session_end,normal,\n                            S#ysession.cookie, S#ysession.opaque}).\n\nhas_timedout(Y, Time) ->\n    Y#ysession.to =< Time.\n\ncookie(Y) ->\n    Y#ysession.cookie.\n\n%% Backend callbacks (ETS as default)\n\ninit_backend (_) ->\n    ets:new(?MODULE, [set, named_table, public, {keypos, 2}]).\n\nstop_backend() ->\n    ok.\n\nlookup(Key) ->\n    ets:lookup(?MODULE, Key).\n\ninsert(Session) ->\n    ets:insert(?MODULE, Session).\n\nlist() ->\n    ets:tab2list(?MODULE).\n\ndelete(Key) ->\n    ets:delete(?MODULE, Key).\n\ncleanup() ->\n    ets:delete_all_objects(?MODULE).\n\ntraverse(N) ->\n    traverse(N, ets:first(?MODULE)).\n\ntraverse(_N, '$end_of_table') ->\n    ok;\ntraverse(N, Key) ->\n    case lookup(Key) of\n        [Y] ->\n            case has_timedout(Y, N) of\n                false ->\n                    traverse(N, ets:next(?MODULE, Key));\n                true ->\n                    report_timedout_sess(Y),\n                    Next = ets:next(?MODULE, Key),\n                    delete(Key),\n                    traverse(N, Next)\n            end;\n        [] ->\n            traverse(N, ets:next(?MODULE, Key))\n    end.\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_shaper.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_shaper.erl\n%%% Author  : Christopher Faulet <christopher@yakaz.com>\n%%% Purpose :\n%%% Created : 14 Dec 2010 by Christopher Faulet <christopher@yakaz.com>\n%%%----------------------------------------------------------------------\n\n-module(yaws_shaper).\n-author('christopher@yakaz.com').\n\n\n-export([behaviour_info/1]).\n\n%% API\n-export([\n         check/2,\n         update/3\n        ]).\n\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_debug.hrl\").\n\n\n%%%----------------------------------------------------------------------\n%%% API\n%%%----------------------------------------------------------------------\nbehaviour_info(callbacks) ->\n    [{check,1}, {update,3}];\nbehaviour_info(_Other) ->\n    undefined.\n\n\ncheck(#sconf{shaper=undefined}, _) ->\n    allow;\ncheck(#sconf{shaper=Mod}, IP) ->\n    case catch Mod:check(IP) of\n        allow ->\n            allow;\n        {deny, Status, Msg} ->\n            {deny, Status, Msg};\n        _ ->\n            allow\n    end.\n\nupdate(#sconf{shaper=undefined}, _, _) ->\n    ok;\nupdate(#sconf{shaper=Mod}, IP, Req) ->\n    Bytes = case Req#http_request.method of\n                'HEAD' -> 0;\n                _  ->\n                    case yaws:outh_get_contlen() of\n                        undefined ->\n                            case yaws:outh_get_act_contlen() of\n                                undefined -> 0;\n                                Actlen    -> Actlen\n                            end;\n                        I2 -> I2\n                    end\n            end,\n    catch Mod:update(IP, 1, Bytes).\n"
  },
  {
    "path": "contrib/yaws/src/yaws_soap12_lib.erl",
    "content": "%%%-------------------------------------------------------------------\n%%% Created : 29 Nov 2006 by Torbjorn Tornkvist <tobbe@tornkvist.org>\n%%% Author  : Willem de Jong (w.a.de.jong@gmail.com).\n%%% Desc.   : Common SOAP code.\n%%%-------------------------------------------------------------------\n\n%%% modified (WdJ, May 2007): deal with imports in the WSDL.\n%%% modified (WdJ, August 2007): the WSDL can contain more than 1 schema\n%%% copied from yaws_soap_lib (Kaloyan Dimitrov, February 2012):\n%% to be used for soap12 calls\n\n-module(yaws_soap12_lib).\n\n-export([initModel/1, initModel/2,\n         initModelFile/1,\n         config_file_xsd/0,\n         call/3, call/4, call/5, call/6, call/8,\n         call_attach/4, call_attach/5, call_attach/8,\n         write_hrl/2, write_hrl/3,\n         findHeader/2,\n         parseMessage/2,\n         makeFault/2,\n         is_wsdl/1, wsdl_model/1, wsdl_op_service/1,\n         wsdl_op_port/1, wsdl_op_operation/1,\n         wsdl_op_binding/1, wsdl_op_address/1,\n         wsdl_op_action/1, wsdl_operations/1,\n         get_operation/2\n        ]).\n\n\n%%% For testing...\n-export([qtest/0]).\n\n\n-include(\"../include/yaws_soap.hrl\").\n-include(\"../include/soap-envelope.hrl\").\n-include(\"../include/wsdl11soap12.hrl\").\n\n-define(HTTP_REQ_TIMEOUT, 20000).\n\n%%-define(dbg(X,Y),\n%%        error_logger:info_msg(\"*dbg ~p(~p): \" X,\n%%                              [?MODULE, ?LINE | Y])).\n-define(dbg(X,Y), true).\n\n\n-record(yaws_soap_config, {atts, xsd_path,  user_module, wsdl_file, add_files}).\n-record(xsd_file, {atts, name, prefix, import_specs}).\n-record(import_specs, {atts, namespace, prefix, location}).\n-record(namespace_spec, {namespace, prefix}).\n-record(namespace_registry, {specs = [], counter = 0}).\n\n-define(DefaultPrefix, \"p\").\n-define(CustomPrefix, \"cp\").\n\n\n%%%\n%%% Writes the header file (record definitions) for a WSDL file\n%%%\nwrite_hrl(WsdlURL, Output) when is_list(WsdlURL) ->\n    write_hrl(initModel(WsdlURL), Output);\nwrite_hrl(#wsdl{model = Model}, Output) when is_list(Output) ->\n    erlsom:write_hrl(Model, Output).\n\nwrite_hrl(WsdlURL, Output, PrefixOrOptions)\n  when is_list(WsdlURL),is_list(PrefixOrOptions) ->\n    write_hrl(initModel(WsdlURL, PrefixOrOptions), Output).\n\n\n\n%%% For testing only...\nqtest() ->\n    call(\"http://www.webservicex.net/WeatherForecast.asmx?WSDL\",\n         \"GetWeatherByPlaceName\",\n         [\"Boston\"]).\n\n%%% --------------------------------------------------------------------\n%%% Access functions\n%%% --------------------------------------------------------------------\nis_wsdl(Wsdl) when is_record(Wsdl,wsdl) -> true;\nis_wsdl(_)                           -> false.\n\nwsdl_operations(#wsdl{operations = Ops}) -> Ops.\n\nwsdl_model(#wsdl{model = Model}) -> Model.\n\nwsdl_op_service(#operation{service = Service}) -> Service.\n\nwsdl_op_port(#operation{port = Port}) -> Port.\n\nwsdl_op_operation(#operation{operation = Op}) -> Op.\n\nwsdl_op_binding(#operation{binding = Binding}) -> Binding.\n\nwsdl_op_address(#operation{address = Address}) -> Address.\n\nwsdl_op_action(#operation{action = Action}) -> Action.\n\n\n%%% --------------------------------------------------------------------\n%%% For Quick deployment\n%%% --------------------------------------------------------------------\ncall(WsdlURL, Operation, ListOfData) when is_list(WsdlURL) ->\n    Wsdl = initModel(WsdlURL, ?DefaultPrefix),\n    call(Wsdl, Operation, ListOfData);\ncall(Wsdl, Operation, ListOfData) when is_record(Wsdl, wsdl) ->\n    case get_operation(Wsdl#wsdl.operations, Operation) of\n        {ok, Op} ->\n            Msg = mk_msg(?DefaultPrefix, Operation, ListOfData),\n            call(Wsdl, Operation, Op#operation.port,\n                 Op#operation.service, [], Msg);\n        Else ->\n            Else\n    end.\n\n%%% --------------------------------------------------------------------\n%%% Takes http headers\n%%% --------------------------------------------------------------------\ncall(WsdlURL, Operation, ListOfData, http_headers, HttpHeaders)\n  when is_list(WsdlURL) ->\n    Wsdl = initModel(WsdlURL, ?DefaultPrefix),\n    call(Wsdl, Operation, ListOfData, http_headers, HttpHeaders);\ncall(Wsdl, Operation, ListOfData, http_headers, HttpHeaders)\n  when is_record(Wsdl, wsdl) ->\n    case get_operation(Wsdl#wsdl.operations, Operation) of\n        {ok, Op} ->\n            Msg = mk_msg(?DefaultPrefix, Operation, ListOfData),\n            call(Wsdl, Operation, Op#operation.port,\n                 Op#operation.service, [], Msg, http_headers, HttpHeaders);\n        Else ->\n            Else\n    end;\n\n%%% --------------------------------------------------------------------\n%%% With additional specified prefix\n%%% --------------------------------------------------------------------\ncall(WsdlURL, Operation, ListOfData, prefix, Prefix) when is_list(WsdlURL) ->\n    Wsdl = initModel(WsdlURL, Prefix),\n    call(Wsdl, Operation, ListOfData, prefix, Prefix );\ncall(Wsdl, Operation, ListOfData, prefix, Prefix) when is_record(Wsdl, wsdl) ->\n    case get_operation(Wsdl#wsdl.operations, Operation) of\n        {ok, Op} ->\n            Msg = mk_msg(Prefix, Operation, ListOfData),\n            call(Wsdl, Operation, Op#operation.port,\n                 Op#operation.service, [], Msg);\n        Else ->\n            Else\n    end.\n\n\n%%% --------------------------------------------------------------------\n%%% Takes the actual records for the Header and Body message.\n%%% --------------------------------------------------------------------\ncall(WsdlURL, Operation, Header, Msg) when is_list(WsdlURL) ->\n    Wsdl = initModel(WsdlURL, ?DefaultPrefix),\n    call(Wsdl, Operation, Header, Msg);\ncall(Wsdl, Operation, Header, Msg) when is_record(Wsdl, wsdl) ->\n    case get_operation(Wsdl#wsdl.operations, Operation) of\n        {ok, Op} ->\n            call(Wsdl, Operation, Op#operation.port, Op#operation.service,\n                 Header, Msg);\n        Else ->\n            Else\n    end.\n\n\nmk_msg(_Prefix, _Operation, ListOfData) ->\n    ListOfData.                       % rest of record data\n\nget_operation([#operation{operation = X} = Op|_], X) ->\n    {ok, Op};\nget_operation([_|T], Op)                             ->\n    get_operation(T, Op);\nget_operation([], _Op)                               ->\n    {error, \"operation not found\"}.\n\n\n%%% --------------------------------------------------------------------\n%%% Make a SOAP request (no attachments)\n%%% --------------------------------------------------------------------\ncall(Wsdl, Operation, Port, Service, Headers, Message) ->\n    call_attach(Wsdl, Operation, Port, Service, Headers, Message, [], []).\n\n%%% --------------------------------------------------------------------\n%%% Make a SOAP request (with http artifacts)\n%%% --------------------------------------------------------------------\ncall(Wsdl, Operation, Port, Service, Headers, Message,\n     http_headers, HttpHeaders) ->\n    call_attach(Wsdl, Operation, Port, Service, Headers,\n                Message, [], HttpHeaders);\n\ncall(Wsdl, Operation, Port, Service, Headers, Message,\n     http_details, HttpDetails) ->\n    call_attach(Wsdl, Operation, Port, Service, Headers,\n                Message, [], http_details, HttpDetails).\n\n\n%%% --------------------------------------------------------------------\n%%% For Quick deployment (with attachments)\n%%% --------------------------------------------------------------------\ncall_attach(WsdlURL, Operation, ListOfData, Attachments)\n  when is_list(WsdlURL) ->\n    Wsdl = initModel(WsdlURL, ?DefaultPrefix),\n    call_attach(Wsdl, Operation, ListOfData, Attachments);\ncall_attach(Wsdl, Operation, ListOfData, Attachments)\n  when is_record(Wsdl, wsdl) ->\n    case get_operation(Wsdl#wsdl.operations, Operation) of\n        {ok, Op} ->\n            Msg = mk_msg(?DefaultPrefix, Operation, ListOfData),\n            call_attach(Wsdl, Operation, Op#operation.port,\n                        Op#operation.service, [], Msg, Attachments, []);\n        Else ->\n            Else\n    end.\n\n%%% --------------------------------------------------------------------\n%%% Takes the actual records for the Header and Body message\n%%% (with attachments)\n%%% --------------------------------------------------------------------\ncall_attach(WsdlURL, Operation, Header, Msg, Attachments)\n  when is_list(WsdlURL) ->\n    Wsdl = initModel(WsdlURL, ?DefaultPrefix),\n    call_attach(Wsdl, Operation, Header, Msg, Attachments);\ncall_attach(Wsdl, Operation, Header, Msg, Attachments)\n  when is_record(Wsdl, wsdl) ->\n    case get_operation(Wsdl#wsdl.operations, Operation) of\n        {ok, Op} ->\n            call_attach(Wsdl, Operation, Op#operation.port,\n                        Op#operation.service,\n                        Header, Msg, Attachments, []);\n        Else ->\n            Else\n    end.\n\n\n%%% --------------------------------------------------------------------\n%%% Make a SOAP request (with attachments)\n%%% --------------------------------------------------------------------\ncall_attach(Wsdl, Operation, Port, Service, Headers,\n            Message, Attachments, HttpHeaders) ->\n    call_attach(Wsdl, Operation, Port, Service, Headers,\n                Message, Attachments, http_details, [{headers, HttpHeaders}]).\n\ncall_attach(#wsdl{operations = Operations, model = Model},\n            Operation, Port, Service, Headers, Message,\n            Attachments, http_details, HttpDetails) ->\n    HttpHeaders = findListValue(headers, HttpDetails),\n    HttpClientOptions = findListValue(client_options, HttpDetails),\n    %% find the operation\n    case findOperation(Operation, Port, Service, Operations) of\n        #operation{address = URL, action=Action, operation = Operation} ->\n            %% Add the Soap envelope\n            Envelope = mk_envelope(Message, Headers),\n            %% Encode the message\n            case erlsom:write(Envelope, Model) of\n                {ok, XmlMessage} ->\n\n                    {ContentType, Request} =\n                        make_request_body(XmlMessage, Attachments, Action),\n                    ?dbg(\"+++ Request = ~p~n\", [Request]),\n                    HttpRes = http_request(URL, Action, Request,\n                                           HttpClientOptions, HttpHeaders,\n                                           ContentType),\n                    ?dbg(\"+++ HttpRes = ~p~n\", [HttpRes]),\n                    case HttpRes of\n                        {ok, _Code, _ReturnHeaders, Body} ->\n                            parseMessage(Body, Model);\n                        Error ->\n                            %% in case of HTTP error: return\n                            %% {error, description}\n                            Error\n                    end;\n                {error, EncodingError} ->\n                    {error, {encoding_error, EncodingError}}\n            end;\n        false ->\n            {error, {unknown_operation, Operation}}\n    end.\n\nfindListValue(Key, KeyVals) ->\n    case lists:keyfind(Key, 1, KeyVals) of\n        {Key, List} ->\n            List;\n        false ->\n            []\n    end.\n%%%\n%%% returns {ok, Header, Body} | {error, Error}\n%%%\nparseMessage(Message, #wsdl{model = Model}) ->\n    parseMessage(Message, Model);\n%%\nparseMessage(Message, Model) ->\n    Parsed = erlsom:scan(Message, Model),\n    case Parsed of\n        {ok, #'soap:Envelope'{'Body' = #'soap:Body'{choice = Body},\n                              'Header' = undefined}, _} ->\n            {ok, undefined, Body};\n        {ok, #'soap:Envelope'{'Body' = #'soap:Body'{choice = Body},\n                              'Header' = #'soap:Header'{choice = Header}}, _} ->\n            {ok, Header, Body};\n        {error, ErrorMessage} ->\n            {error, {decoding, ErrorMessage}}\n    end.\n\n\nfindOperation(_Operation, _Port, _Service, []) ->\n    false;\nfindOperation(Operation, Port, Service,\n              [Op = #operation{operation = Operation,\n                               port = Port, service = Service} | _]) ->\n    Op;\nfindOperation(Operation, Port, Service, [#operation{} | Tail]) ->\n    findOperation(Operation, Port, Service, Tail).\n\n\nmk_envelope(M, H) when is_tuple(M) -> mk_envelope([M], H);\nmk_envelope(M, H) when is_tuple(H) -> mk_envelope(M, [H]);\n%%\nmk_envelope(Messages, []) when is_list(Messages) ->\n    #'soap:Envelope'{'Body' =  #'soap:Body'{choice = Messages}};\nmk_envelope(Messages, Headers) when is_list(Messages),is_list(Headers) ->\n    #'soap:Envelope'{'Body'   =  #'soap:Body'{choice   = Messages},\n                     'Header' =  #'soap:Header'{choice = Headers}}.\n\n%%% --------------------------------------------------------------------\n%%% Parse a WSDL file and return a 'Model'\n%%% --------------------------------------------------------------------\ninitModel(WsdlFile) ->\n    initModel(WsdlFile, ?DefaultPrefix).\n\n%% PrefixOrOptions can be a property list that contains the options\n%% for Erlsom, or a String. If it is a string, this is used as the\n%% Erlsom 'prefix' option (and the other options are left unspecified).\ninitModel(WsdlFile, PrefixOrOptions) ->\n    Options = case is_string(PrefixOrOptions) of\n        no ->\n          %% It is an option list\n          %% Add the default prefix at the end - it will only be used\n          %% if no other prefix is specified\n          PrefixOrOptions ++ [{prefix, ?DefaultPrefix}];\n        _ ->\n          %% just the prefix\n          [{prefix, PrefixOrOptions}]\n    end,\n    PrivDir = priv_dir(),\n    initModel2(WsdlFile, Options, PrivDir, undefined, undefined).\n\ninitModelFile(ConfigFile) ->\n    {ok, ConfigSchema} = erlsom:compile_xsd(config_file_xsd()),\n    %% read (parse) the config file\n    {ok, Config, _} = erlsom:scan_file(ConfigFile, ConfigSchema),\n    #yaws_soap_config{xsd_path = XsdPath,\n                      wsdl_file = Wsdl,\n                      add_files = AddFiles} = Config,\n    #xsd_file{name = WsdlFile, prefix = Prefix, import_specs = Import} = Wsdl,\n    initModel2(WsdlFile, [{prefix, Prefix}], XsdPath, Import, AddFiles).\n\npriv_dir() ->\n    yaws:get_priv_dir().\n\ninitModel2(WsdlFile, ErlsomOptions, Path, Import, AddFiles) ->\n    WsdlName = filename:join([Path, \"wsdl.xsd\"]),\n    IncludeWsdl = {\"http://schemas.xmlsoap.org/wsdl/\", \"wsdl\", WsdlName},\n    {ok, WsdlModel} = erlsom:compile_xsd_file(\n                        filename:join([Path, \"wsdl11soap12.xsd\"]),\n                        [{prefix, \"soap\"},\n                         {include_files, [IncludeWsdl]}]),\n    %% uncomment to generate the wsdl11soap12.hrl file\n    %% erlsom:write_hrl(WsdlModel, \"/home/kalski/test/wsdl11soap12.hrl\"),\n    %% add the xsd model (since xsd is also used in the wsdl)\n    WsdlModel2 = erlsom:add_xsd_model(WsdlModel),\n    Options = ErlsomOptions ++ makeOptions(Import),\n    %% parse Wsdl\n    {Model, Operations} = parseWsdls([WsdlFile], WsdlModel2,\n                                     Options, {undefined, []}),\n    %% TODO: add files as required\n    %% now compile envelope.xsd, and add Model\n    {ok, EnvelopeModel} =\n        erlsom:compile_xsd_file(\n          filename:join([Path, \"soap-envelope.xsd\"]),\n          [{prefix, \"soap\"},\n           {include_files, [{\"http://www.w3.org/XML/1998/namespace\", undefined,\n                             filename:join([Path, \"xml.xsd\"])}]}]),\n    SoapModel = erlsom:add_model(EnvelopeModel, Model),\n    %% uncomment to generate the soap-envelope.hrl file\n    %% erlsom:write_hrl(EnvelopeModel, \"/home/kalski/test/soap-envelope.hrl\"),\n    SoapModel2 = addModels(AddFiles, SoapModel),\n    #wsdl{operations = Operations, model = SoapModel2}.\n\n\n%%% --------------------------------------------------------------------\n%%% Parse a list of WSDLs and import (recursively)\n%%% Returns {Model, Operations}\n%%% --------------------------------------------------------------------\nparseWsdls(WsdlFiles, WsdlModel, Options, Acc) ->\n    parseWsdls(WsdlFiles, WsdlModel, Options, Acc, #namespace_registry{}).\n\nparseWsdls([], _WsdlModel, _Options, Acc, _NSRegistry) ->\n    Acc;\nparseWsdls([WsdlFile | Tail], WsdlModel, Options,\n           {AccModel, AccOperations}, NSRegistry) ->\n    WsdlFileNoSpaces = rmsp(WsdlFile),\n    {ok, WsdlFileContent} = get_url_file(WsdlFileNoSpaces),\n    {ok, ParsedWsdl, _} = erlsom:scan(WsdlFileContent, WsdlModel),\n    WsdlTargetNameSpace = getTargetNamespaceFromWsdl(ParsedWsdl),\n    {Prefix, PrefixlessOptions} = remove_prefix_option(Options),\n    TNSEnrichedNSRegistry = extend_namespace_registry(WsdlTargetNameSpace,\n                                                      Prefix, NSRegistry),\n    %% get the xsd elements from this model, and hand it over to erlsom_compile.\n    Xsds = getXsdsFromWsdl(ParsedWsdl),\n    %% Now we need to build a list: [{Namespace, Xsd, Prefix}, ...] for\n    %% all the Xsds in the WSDL.\n    %% This list is used when a schema includes one of the other schemas.\n    %% The AXIS java2wsdl tool generates wsdls that depend on this feature.\n    {ImportsEnrichedNSRegistry, ImportList} = makeImportList(\n                                                Xsds,\n                                                TNSEnrichedNSRegistry, []),\n    Model2 = addSchemas(Xsds, AccModel, PrefixlessOptions, ImportList),\n    Ports = getPorts(ParsedWsdl),\n    Operations = getOperations(ParsedWsdl, Ports),\n    Imports = getImports(filename:dirname(WsdlFileNoSpaces), ParsedWsdl),\n    %% use Options rather than PrefixlessOptions because imports come in\n    %% the wsdl targetNamespace\n    Model3 = addSchemaFiles(Imports, Model2, Options, []),\n    Acc2 = {Model3, Operations ++ AccOperations},\n    %% process imports (recursively, so that imports in the imported files are\n    %% processed as well).\n    %% For the moment, the namespace is ignored on operations etc.\n    %% this makes it a bit easier to deal with imported wsdl's.\n    %% TODO uncomment if imports can be WSDL\n    %%Acc3 = parseWsdls(Imports, WsdlModel, Options, Acc2,\n    %%                   ImportsEnrichedNSRegistry),\n    parseWsdls(Tail, WsdlModel, PrefixlessOptions, Acc2,\n               ImportsEnrichedNSRegistry).\n\nremove_prefix_option(Options) ->\n    case lists:keytake(prefix, 1, Options) of\n        {value, {prefix, Prefix}, NewOptions} ->\n            {Prefix, NewOptions};\n        false ->\n            {undefined, Options}\n    end.\n\n%empty registry, initializing\nextend_namespace_registry(WsdlTargetNameSpace, undefined,\n                          #namespace_registry{specs = []} = NSRegistry) ->\n    {NewCounter, NewPrefix} = create_unique_prefix(NSRegistry),\n    NSRegistry#namespace_registry{\n      specs = [#namespace_spec{namespace = WsdlTargetNameSpace,\n                               prefix = NewPrefix}], counter = NewCounter};\nextend_namespace_registry(WsdlTargetNameSpace, Prefix,\n                          #namespace_registry{specs = []} = NSRegistry) ->\n    NSRegistry#namespace_registry{\n      specs = [#namespace_spec{namespace = WsdlTargetNameSpace,\n                               prefix = Prefix}]};\nextend_namespace_registry(WsdlTargetNameSpace, _Prefix,\n                          #namespace_registry{specs = Specs} = NSRegistry) ->\n    case lists:keyfind(WsdlTargetNameSpace, #namespace_spec.namespace, Specs) of\n        #namespace_spec{} ->\n            NSRegistry;\n        false ->\n            {NewCounter, NewPrefix} = create_unique_prefix(NSRegistry),\n            NSRegistry#namespace_registry{\n              specs = [#namespace_spec{namespace = WsdlTargetNameSpace,\n                                       prefix = NewPrefix}|Specs],\n              counter = NewCounter}\n    end.\n\n\ncreate_unique_prefix(#namespace_registry{specs = Specs, counter = Counter} =\n                         NSRegistry) ->\n    NewCounter = Counter+1,\n    NewPrefix = ?CustomPrefix ++ integer_to_list(NewCounter),\n    case lists:keyfind(NewPrefix, #namespace_spec.prefix, Specs) of\n        #namespace_spec{} ->\n            create_unique_prefix(NSRegistry#namespace_registry{\n                                   counter = Counter+1});\n        false ->\n            {NewCounter, NewPrefix}\n    end.\n%%% --------------------------------------------------------------------\n%%% build a list: [{Namespace, Xsd}, ...] for all the Xsds in the WSDL.\n%%% This list is used when a schema inlcudes one of the other schemas.\n%%% The AXIS java2wsdl tool generates wsdls that depend on this feature.\nmakeImportList([], NSRegistry, Acc) ->\n    {NSRegistry, Acc};\nmakeImportList([ Xsd | Tail], NSRegistry, Acc) ->\n    XsdNS = erlsom_lib:getTargetNamespaceFromXsd(Xsd),\n    NewNSRegistry = extend_namespace_registry(XsdNS, undefined, NSRegistry),\n    #namespace_spec{prefix = Prefix} =\n        lists:keyfind(XsdNS, #namespace_spec.namespace,\n                      NewNSRegistry#namespace_registry.specs),\n    makeImportList(Tail, NewNSRegistry, [{XsdNS, Prefix, Xsd} | Acc]).\n\ngetTargetNamespaceFromWsdl(#'wsdl:tDefinitions'{targetNamespace = TNS}) ->\n    TNS.\n\n%%% --------------------------------------------------------------------\n%%% compile each of the schemas, and add it to the model.\n%%% Returns Model\n%%% (TODO: using the same prefix for all XSDS makes no sense)\n%%% --------------------------------------------------------------------\naddSchemas([], AccModel, _PrefixlessOptions, _ImportList) ->\n    AccModel;\naddSchemas([Xsd| Tail], AccModel, PrefixlessOptions, ImportList) ->\n    Model2 = case Xsd of\n                 undefined ->\n                     AccModel;\n                 _ ->\n                     {_, Prefix, _} =\n                         lists:keyfind(\n                           erlsom_lib:getTargetNamespaceFromXsd(Xsd),\n                           1, ImportList),\n                     NewOptions = [{prefix, Prefix}|PrefixlessOptions],\n                     {ok, Model} =\n                         erlsom_compile:compile_parsed_xsd(\n                           Xsd,\n                           [{include_files, ImportList} |NewOptions]),\n                     case AccModel of\n                         undefined -> Model;\n                         _ -> erlsom:add_model(AccModel, Model)\n                     end\n             end,\n    addSchemas(Tail, Model2, PrefixlessOptions, ImportList).\n\n%%% --------------------------------------------------------------------\n%%% compile each of the schema files, and add it to the model.\n%%% Returns Model\n%%% (TODO: using the same prefix for all XSD files makes no sense)\n%%% --------------------------------------------------------------------\naddSchemaFiles([], AccModel, _Options, _ImportList) ->\n    AccModel;\naddSchemaFiles([Xsd| Tail], AccModel, Options, ImportList) ->\n    {ok, Model} =\n        erlsom:compile_xsd_file(get_file_with_path(Xsd),\n                                [{include_files, ImportList} |Options]),\n    Model2 = case AccModel of\n                 undefined -> Model;\n                 _ -> erlsom:add_model(AccModel, Model)\n             end,\n    addSchemaFiles(Tail, Model2, Options, ImportList).\n\n%%% --------------------------------------------------------------------\n%%% Get a file from an URL spec.\n%%% --------------------------------------------------------------------\nget_url_file(\"http://\"++_ = URL) ->\n    case httpc:request(URL) of\n        {ok,{{_HTTP,200,_OK}, _Headers, Body}} ->\n            {ok, Body};\n        {ok,{{_HTTP,RC,Emsg}, _Headers, _Body}} ->\n            error_logger:error_msg(\"~p: http-request got: ~p~n\",\n                                   [?MODULE, {RC, Emsg}]),\n            {error, \"failed to retrieve: \"++URL};\n        {error, Reason} ->\n            error_logger:error_msg(\"~p: http-request failed: ~p~n\",\n                                   [?MODULE, Reason]),\n            {error, \"failed to retrieve: \"++URL}\n    end;\nget_url_file(\"file://\"++Fname) ->\n    {ok, Bin} = file:read_file(Fname),\n    {ok, binary_to_list(Bin)};\n%% added this, since this is what is used in many WSDLs (i.e.: just a filename).\nget_url_file(Fname) ->\n    {ok, Bin} = file:read_file(Fname),\n    {ok, binary_to_list(Bin)}.\n\n\n%%% --------------------------------------------------------------------\n%%% Make a HTTP Request\n%%% --------------------------------------------------------------------\nhttp_request(URL, Action, Request, Options, Headers, ContentType) ->\n    case code:ensure_loaded(ibrowse) of\n        {module, ibrowse} ->\n            %% If ibrowse exist in the path then let's use it...\n            ibrowse_request(URL, Action, Request, Options,\n                            Headers, ContentType);\n        _ ->\n            %% ...otherwise, let's use the OTP http client.\n            inets_request(URL, Action, Request, Options,\n                          Headers, ContentType)\n    end.\n\ninets_request(URL, Action, Request, Options, Headers, ContentType) ->\n    case Action of\n      undefined ->\n        NHeaders = Headers;\n      _ ->\n        NHeaders = [{\"SOAPAction\", Action} | Headers]\n    end,\n    NewHeaders = case proplists:get_value(\"Host\", NHeaders) of\n                     undefined ->\n                         [{\"Host\", \"localhost:8800\"}|NHeaders];\n                     _ ->\n                         NHeaders\n                 end,\n    NewOptions = [{cookies, enabled}|Options],\n    httpc:set_options(NewOptions),\n    case httpc:request(post,\n                       {URL,NewHeaders,\n                        ContentType,\n                        Request},\n                       [{timeout,?HTTP_REQ_TIMEOUT}],\n                       [{sync, true}, {full_result, true},\n                        {body_format, string}]) of\n        {ok,{{_HTTP,200,_OK},ResponseHeaders,ResponseBody}} ->\n            {ok, 200, ResponseHeaders, ResponseBody};\n        {ok,{{_HTTP,500,_Descr},ResponseHeaders,ResponseBody}} ->\n            {ok, 500, ResponseHeaders, ResponseBody};\n        {ok,{{_HTTP,ErrorCode,_Descr},ResponseHeaders,ResponseBody}} ->\n            {ok, ErrorCode, ResponseHeaders, ResponseBody};\n        Other ->\n            Other\n    end.\n\nibrowse_request(URL, Action, Request, Options, Headers, ContentType) ->\n    case start_ibrowse() of\n        ok ->\n            NewHeaders = [{\"Content-Type\", ContentType} |\n                          case Action of\n                             undefined ->\n                                  Headers;\n                              _ ->\n                                  [{\"SOAPAction\", Action} | Headers]\n                          end],\n            IbrowseF = case lists:keyfind(ibrowse_timeout, 1, Options) of\n                {_, Timeout} ->\n                    fun() ->\n                        ibrowse:send_req(URL, NewHeaders, post,\n                                         Request, Options, Timeout)\n                    end;\n                false ->\n                    fun() ->\n                        ibrowse:send_req(URL, NewHeaders, post,\n                                         Request, Options)\n                    end\n            end,\n            case IbrowseF() of\n                {ok, Status, ResponseHeaders, ResponseBody} ->\n                    {ok, list_to_integer(Status), ResponseHeaders,\n                     ResponseBody};\n                {error, Reason} ->\n                    {error, Reason}\n            end;\n        error ->\n            {error, \"could not start ibrowse\"}\n    end.\n\nstart_ibrowse() ->\n    case ibrowse:start() of\n        {ok, _} -> ok;\n        {error, {already_started, _}} -> ok;\n        _ -> error\n    end.\n\n\nrmsp(Str) -> string:strip(Str, left).\n\n\nmake_request_body(Content, [], Operation) ->\n    {\"application/soap+xml;charset=UTF-8;action=\\\"\" ++ Operation ++ \"\\\"\",\n     \"<?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?>\"++ Content};\nmake_request_body(Content, AttachedFiles, _Operation) ->\n    {\"application/dime\",\n     yaws_dime:encode(\"<?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?>\" ++ Content,\n                      AttachedFiles)}.\n\nmakeFault(FaultCode, FaultString) ->\n    try\n        \"<SOAP-ENV:Envelope xmlns:SOAP-ENV=\\\"http://schemas.xmlsoap.org/soap/envelope/\\\">\"\n            \"<SOAP-ENV:Body>\"\n            \"<SOAP-ENV:Fault>\"\n            \"<faultcode>SOAP-ENV:\" ++ FaultCode ++ \"</faultcode>\" ++\n            \"<faultstring>\" ++ FaultString ++ \"</faultstring>\" ++\n            \"</SOAP-ENV:Fault>\"\n            \"</SOAP-ENV:Body>\"\n            \"</SOAP-ENV:Envelope>\"\n    catch\n        _:_ ->\n            \"<SOAP-ENV:Envelope xmlns:SOAP-ENV=\\\"http://schemas.xmlsoap.org/soap/envelope/\\\">\"\n                \"<SOAP-ENV:Body>\"\n                \"<SOAP-ENV:Fault>\"\n                \"<faultcode>SOAP-ENV:Server</faultcode>\"\n                \"<faultstring>Server error</faultstring>\"\n                \"</SOAP-ENV:Fault>\"\n                \"</SOAP-ENV:Body>\"\n                \"</SOAP-ENV:Envelope>\"\n    end.\n\n%% record http_header is not defined??\nfindHeader(Label, Headers) ->\n    findHeader0(yaws:to_lower(Label), Headers).\n\nfindHeader0(_Label, []) ->\n    undefined;\nfindHeader0(Label, [{_,_,Hdr,_,Val}|T]) ->\n    case {Label, yaws:to_lower(Hdr)} of\n        {X,X} -> Val;\n        _     -> findHeader0(Label, T)\n    end;\nfindHeader0(_Label, undefined) ->\n    undefined.\n\n\nmakeOptions(undefined) ->\n    [];\nmakeOptions(Import) ->\n    lists:map(fun makeOption/1, Import).\n\n%% -record(import_specs, {atts, namespace, prefix, location}).\nmakeOption(#import_specs{namespace = Ns, prefix = Pf, location = Lc}) ->\n    {Ns, Pf, Lc}.\n\n\naddModels(undefined, Model) ->\n    Model;\naddModels(Import, Model) ->\n    lists:foldl(fun addModel/2, Model, Import).\n\n%% -record(xsd_file, {atts, name, prefix, import_specs}).\naddModel(undefined, Acc) ->\n    Acc;\naddModel(#xsd_file{name = XsdFile, prefix = Prefix, import_specs = Import},\n         Acc) ->\n    Options = makeOptions(Import),\n    {ok, Model2} = erlsom:add_xsd_file(XsdFile, [{prefix, Prefix}|Options],Acc),\n    Model2.\n\n%% returns [#port{}]\n%% -record(port, {service, port, binding, address}).\ngetPorts(ParsedWsdl) ->\n    Services = getTopLevelElements(ParsedWsdl, 'wsdl:tService'),\n    getPortsFromServices(Services, []).\n\ngetPortsFromServices([], Acc) ->\n    Acc;\ngetPortsFromServices([Service|Tail], Acc) ->\n    getPortsFromServices(Tail, getPortsFromService(Service) ++ Acc).\n\ngetPortsFromService(#'wsdl:tService'{name = Name, port = Ports}) ->\n    getPortsInfo(Ports, Name, []).\n\ngetPortsInfo([], _Name, Acc) ->\n    Acc;\n\ngetPortsInfo([#'wsdl:tPort'{name = Name,\n                            binding = Binding,\n                            choice =\n                                [#'soap:tAddress'{location = URL}]} | Tail],\n             ServiceName, Acc) ->\n    getPortsInfo(Tail, ServiceName, [#port{service = ServiceName,\n                                           port = Name,\n                                           binding = Binding,\n                                           address = URL}|Acc]);\n%% non-soap bindings are ignored.\ngetPortsInfo([#'wsdl:tPort'{} | Tail], ServiceName, Acc) ->\n    getPortsInfo(Tail, ServiceName, Acc).\n\n\ngetTopLevelElements(#'wsdl:tDefinitions'{choice1 = TLElements}, Type) ->\n    getTopLevelElements(TLElements, Type, []).\n\ngetTopLevelElements([], _Type, Acc) ->\n    Acc;\ngetTopLevelElements([#'wsdl:anyTopLevelOptionalElement'{choice = Tuple}| Tail],\n                    Type, Acc) ->\n    case element(1, Tuple) of\n        Type -> getTopLevelElements(Tail, Type, [Tuple|Acc]);\n        _ -> getTopLevelElements(Tail, Type, Acc)\n    end.\n\nget_file_with_path(Url) ->\n  case Url of\n    \"http://\" ++ _ ->\n      undefined;\n    \"file://\" ++ FName ->\n      FName;\n    _ ->\n      Url\n  end.\n\n\ngetImports(WsdlDirname, Definitions) ->\n    Imports = getTopLevelElements(Definitions, 'wsdl:tImport'),\n    lists:map(fun(Import) ->\n                case WsdlDirname of\n                  \"http://\" ++ _AbsDirname ->\n                    WsdlDirname ++ \"/\" ++ Import#'wsdl:tImport'.location;\n                  \"file://\" ++ _AbsDirname ->\n                    WsdlDirname ++ \"/\" ++ Import#'wsdl:tImport'.location;\n                  Fname ->\n                    filename:join(Fname, Import#'wsdl:tImport'.location)\n                end\n              end, Imports).\n\n%% returns [#operation{}]\ngetOperations(ParsedWsdl, Ports) ->\n    Bindings = getTopLevelElements(ParsedWsdl, 'wsdl:tBinding'),\n    getOperationsFromBindings(Bindings, Ports, []).\n\ngetOperationsFromBindings([], _Ports, Acc) ->\n    Acc;\ngetOperationsFromBindings([Binding|Tail], Ports, Acc) ->\n    getOperationsFromBindings(Tail, Ports,\n                              getOperationsFromBinding(Binding, Ports) ++ Acc).\n\ngetOperationsFromBinding(#'wsdl:tBinding'{name = BindingName,\n                                          type = BindingType,\n                                          choice = _Choice,\n                                          operation = Operations}, Ports) ->\n    %% TODO: get soap info from Choice\n    getOperationsFromOperations(Operations, BindingName, BindingType,\n                                Operations, Ports, []).\n\ngetOperationsFromOperation(BindingName, BindingType, Ports, Name,\n                           Action, Operations, Tail, Acc) ->\n  %% lookup Binding in Ports, and create a combined result\n            Ports2 = searchPorts(BindingName, Ports),\n            %% for each port, make an operation record\n            CombinedPorts = combinePorts(Ports2, Name, BindingName, Action),\n            getOperationsFromOperations(\n              Tail, BindingName, BindingType,\n              Operations, Ports, CombinedPorts ++ Acc).\n\ngetOperationsFromOperations([], _BindingName, _BindingType,\n                            _Operations, _Ports, Acc) ->\n    Acc;\n\ngetOperationsFromOperations([#'wsdl:tBindingOperation'{name = Name,\n                                                       choice = Choice} | Tail],\n                            BindingName, BindingType, Operations, Ports, Acc) ->\n    %% get SOAP action from Choice,\n    case Choice of\n        [#'soap:tOperation'{soapAction = Action}] ->\n            getOperationsFromOperation(BindingName, BindingType, Ports,\n                                       Name, Action, Operations, Tail, Acc);\n        _ ->\n            getOperationsFromOperation(BindingName, BindingType, Ports,\n                                       Name, undefined, Operations, Tail, Acc)\n    end.\n\ncombinePorts(Ports, Name, BindingName, Action) ->\n    combinePorts(Ports, Name, BindingName, Action, []).\n\ncombinePorts([], _Name, _BindingName, _Action, Acc) ->\n    Acc;\ncombinePorts([#port{service = Service,\n                    port = PortName,\n                    address = Address} | Tail],\n             Name, BindingName, Action, Acc) ->\n    combinePorts(Tail, Name, BindingName, Action,\n                 [#operation{service = Service,\n                             port = PortName, operation = Name,\n                             binding = BindingName,\n                             address = Address, action = Action} | Acc]).\n\nsearchPorts(BindingName, Ports) ->\n    searchPorts(BindingName, Ports, []).\n\nsearchPorts(_BindingName, [], Acc) ->\n    Acc;\nsearchPorts(BindingName, [Port | Tail], Acc) ->\n    PortBinding = erlsom_lib:localName(Port#port.binding),\n    case PortBinding of\n        BindingName ->\n            searchPorts(BindingName, Tail, [Port | Acc]);\n        _ ->\n            searchPorts(BindingName, Tail, Acc)\n    end.\n\n%% copied from yaws/json.erl\nis_string([]) -> yes;\nis_string(List) -> is_string(List, non_unicode).\n\nis_string([C|Rest], non_unicode)\n  when C >= 0, C =< 255 -> is_string(Rest, non_unicode);\nis_string([C|Rest], _) when C =< 65000 -> is_string(Rest, unicode);\nis_string([], non_unicode) -> yes;\nis_string([], unicode) -> unicode;\nis_string(_, _) -> no.\n\ngetXsdsFromWsdl(Definitions) ->\n    case getTopLevelElements(Definitions, 'wsdl:tTypes') of\n        [#'wsdl:tTypes'{choice = Xsds}] -> Xsds;\n        [] -> []\n    end.\n\nconfig_file_xsd() ->\n    \"<xs:schema xmlns:xs=\\\"http://www.w3.org/2001/XMLSchema\\\">\"\n        \"  <xs:element name=\\\"yaws_soap_config\\\">\"\n        \"     <xs:complexType>\"\n        \"       <xs:sequence>\"\n        \"         <xs:element name=\\\"xsd_path\\\" type=\\\"xs:string\\\" minOccurs=\\\"0\\\"/>\"\n        \"         <xs:element name=\\\"user_module\\\" type=\\\"xs:string\\\"/>\"\n        \"         <xs:element name=\\\"wsdl_file\\\" type=\\\"xsd_file\\\"/>\"\n        \"         <xs:element name=\\\"add_file\\\" type=\\\"xsd_file\\\" minOccurs=\\\"0\\\" maxOccurs=\\\"unbounded\\\"/>\"\n        \"       </xs:sequence>\"\n        \"     </xs:complexType>\"\n        \"  </xs:element>\"\n        \"  <xs:complexType name=\\\"xsd_file\\\">\"\n        \"    <xs:sequence>\"\n        \"      <xs:element name=\\\"import_specs\\\" type=\\\"import_specs\\\" minOccurs=\\\"0\\\" maxOccurs=\\\"unbounded\\\"/>\"\n        \"    </xs:sequence>\"\n        \"    <xs:attribute name=\\\"name\\\" type=\\\"string\\\" use=\\\"required\\\"/>\"\n        \"    <xs:attribute name=\\\"prefix\\\" type=\\\"string\\\"/>\"\n        \"  </xs:complexType>\"\n        \"  <xs:complexType name=\\\"import_specs\\\">\"\n        \"    <xs:attribute name=\\\"namespace\\\" type=\\\"string\\\" use=\\\"required\\\"/>\"\n        \"    <xs:attribute name=\\\"prefix\\\" type=\\\"string\\\"/>\"\n        \"    <xs:attribute name=\\\"location\\\" type=\\\"string\\\"/>\"\n        \"  </xs:complexType>\"\n        \"</xs:schema>\".\n\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_soap_lib.erl",
    "content": "%%%-------------------------------------------------------------------\n%%% Created : 29 Nov 2006 by Torbjorn Tornkvist <tobbe@tornkvist.org>\n%%% Author  : Willem de Jong (w.a.de.jong@gmail.com).\n%%% Desc.   : Common SOAP code.\n%%%-------------------------------------------------------------------\n\n%%% modified (WdJ, May 2007): deal with imports in the WSDL.\n%%% modified (WdJ, August 2007): the WSDL can contain more than 1 schema\n\n-module(yaws_soap_lib).\n\n-export([initModel/1, initModel/2,\n         initModelFile/1,\n         config_file_xsd/0,\n         call/3, call/4, call/5, call/6,\n         call_attach/4, call_attach/5, call_attach/7,\n         write_hrl/2, write_hrl/3,\n         findHeader/2,\n         parseMessage/2,\n         makeFault/2,\n         is_wsdl/1, wsdl_model/1, wsdl_op_service/1,\n         wsdl_op_port/1, wsdl_op_operation/1,\n         wsdl_op_binding/1, wsdl_op_address/1,\n         wsdl_op_action/1, wsdl_operations/1,\n         get_operation/2\n        ]).\n\n\n%%% For testing...\n-export([qtest/0]).\n\n\n-include(\"../include/soap.hrl\").\n\n-define(HTTP_REQ_TIMEOUT, 20000).\n\n%%-define(dbg(X,Y),\n%%        error_logger:info_msg(\"*dbg ~p(~p): \" X,\n%%                              [?MODULE, ?LINE | Y])).\n-define(dbg(X,Y), true).\n\n\n-record(yaws_soap_config, {atts, xsd_path,  user_module, wsdl_file, add_files}).\n-record(xsd_file, {atts, name, prefix, import_specs}).\n-record(import_specs, {atts, namespace, prefix, location}).\n\n-define(DefaultPrefix, \"p\").\n\n\n%%%\n%%% Writes the header file (record definitions) for a WSDL file\n%%%\nwrite_hrl(WsdlURL, Output) when is_list(WsdlURL) ->\n    write_hrl(initModel(WsdlURL), Output);\nwrite_hrl(#wsdl{model = Model}, Output) when is_list(Output) ->\n    erlsom:write_hrl(Model, Output).\n\nwrite_hrl(WsdlURL, Output, PrefixOrOptions)\n  when is_list(WsdlURL),is_list(PrefixOrOptions) ->\n    write_hrl(initModel(WsdlURL, PrefixOrOptions), Output).\n\n\n\n%%% For testing only...\nqtest() ->\n    call(\"http://www.webservicex.net/WeatherForecast.asmx?WSDL\",\n         \"GetWeatherByPlaceName\",\n         [\"Boston\"]).\n\n%%% --------------------------------------------------------------------\n%%% Access functions\n%%% --------------------------------------------------------------------\nis_wsdl(Wsdl) when is_record(Wsdl,wsdl) -> true;\nis_wsdl(_)                           -> false.\n\nwsdl_operations(#wsdl{operations = Ops}) -> Ops.\n\nwsdl_model(#wsdl{model = Model}) -> Model.\n\nwsdl_op_service(#operation{service = Service}) -> Service.\n\nwsdl_op_port(#operation{port = Port}) -> Port.\n\nwsdl_op_operation(#operation{operation = Op}) -> Op.\n\nwsdl_op_binding(#operation{binding = Binding}) -> Binding.\n\nwsdl_op_address(#operation{address = Address}) -> Address.\n\nwsdl_op_action(#operation{action = Action}) -> Action.\n\n\n%%% --------------------------------------------------------------------\n%%% For Quick deployment\n%%% --------------------------------------------------------------------\ncall(WsdlURL, Operation, ListOfData) when is_list(WsdlURL) ->\n    Wsdl = initModel(WsdlURL, ?DefaultPrefix),\n    call(Wsdl, Operation, ListOfData);\ncall(Wsdl, Operation, ListOfData) when is_record(Wsdl, wsdl) ->\n    case get_operation(Wsdl#wsdl.operations, Operation) of\n        {ok, Op} ->\n            Msg = mk_msg(?DefaultPrefix, Operation, ListOfData),\n            call(Wsdl, Operation, Op#operation.port,\n                 Op#operation.service, [], Msg);\n        Else ->\n            Else\n    end.\n\n%%% --------------------------------------------------------------------\n%%% With additional specified prefix\n%%% --------------------------------------------------------------------\ncall(WsdlURL, Operation, ListOfData, prefix, Prefix) when is_list(WsdlURL) ->\n    Wsdl = initModel(WsdlURL, Prefix),\n    call(Wsdl, Operation, ListOfData, prefix, Prefix );\ncall(Wsdl, Operation, ListOfData, prefix, Prefix) when is_record(Wsdl, wsdl) ->\n    case get_operation(Wsdl#wsdl.operations, Operation) of\n        {ok, Op} ->\n            Msg = mk_msg(Prefix, Operation, ListOfData),\n            call(Wsdl, Operation, Op#operation.port,\n                 Op#operation.service, [], Msg);\n        Else ->\n            Else\n    end.\n\n\n%%% --------------------------------------------------------------------\n%%% Takes the actual records for the Header and Body message.\n%%% --------------------------------------------------------------------\ncall(WsdlURL, Operation, Header, Msg) when is_list(WsdlURL) ->\n    Wsdl = initModel(WsdlURL, ?DefaultPrefix),\n    call(Wsdl, Operation, Header, Msg);\ncall(Wsdl, Operation, Header, Msg) when is_record(Wsdl, wsdl) ->\n    case get_operation(Wsdl#wsdl.operations, Operation) of\n        {ok, Op} ->\n            call(Wsdl, Operation, Op#operation.port, Op#operation.service,\n                 Header, Msg);\n        Else ->\n            Else\n    end.\n\n\nmk_msg(Prefix, Operation, ListOfData) ->\n    list_to_tuple([list_to_atom(Prefix++\":\"++Operation), % record name\n                   []                                    % anyAttribs\n                   | ListOfData]).                       % rest of record data\n\nget_operation([#operation{operation = X} = Op|_], X) ->\n    {ok, Op};\nget_operation([_|T], Op)                             ->\n    get_operation(T, Op);\nget_operation([], _Op)                               ->\n    {error, \"operation not found\"}.\n\n\n%%% --------------------------------------------------------------------\n%%% Make a SOAP request (no attachments)\n%%% --------------------------------------------------------------------\ncall(Wsdl, Operation, Port, Service, Headers, Message) ->\n    call_attach(Wsdl, Operation, Port, Service, Headers, Message, []).\n\n\n%%% --------------------------------------------------------------------\n%%% For Quick deployment (with attachments)\n%%% --------------------------------------------------------------------\ncall_attach(WsdlURL, Operation, ListOfData, Attachments)\n  when is_list(WsdlURL) ->\n    Wsdl = initModel(WsdlURL, ?DefaultPrefix),\n    call_attach(Wsdl, Operation, ListOfData, Attachments);\ncall_attach(Wsdl, Operation, ListOfData, Attachments)\n  when is_record(Wsdl, wsdl) ->\n    case get_operation(Wsdl#wsdl.operations, Operation) of\n        {ok, Op} ->\n            Msg = mk_msg(?DefaultPrefix, Operation, ListOfData),\n            call_attach(Wsdl, Operation, Op#operation.port,\n                        Op#operation.service, [], Msg, Attachments);\n        Else ->\n            Else\n    end.\n\n%%% --------------------------------------------------------------------\n%%% Takes the actual records for the Header and Body message\n%%% (with attachments)\n%%% --------------------------------------------------------------------\ncall_attach(WsdlURL, Operation, Header, Msg, Attachments)\n  when is_list(WsdlURL) ->\n    Wsdl = initModel(WsdlURL, ?DefaultPrefix),\n    call_attach(Wsdl, Operation, Header, Msg, Attachments);\ncall_attach(Wsdl, Operation, Header, Msg, Attachments)\n  when is_record(Wsdl, wsdl) ->\n    case get_operation(Wsdl#wsdl.operations, Operation) of\n        {ok, Op} ->\n            call_attach(Wsdl, Operation, Op#operation.port,\n                        Op#operation.service,\n                        Header, Msg, Attachments);\n        Else ->\n            Else\n    end.\n\n\n%%% --------------------------------------------------------------------\n%%% Make a SOAP request (with attachments)\n%%% --------------------------------------------------------------------\ncall_attach(#wsdl{operations = Operations, model = Model},\n            Operation, Port, Service, Headers, Message, Attachments) ->\n    %% find the operation\n    case findOperation(Operation, Port, Service, Operations) of\n        #operation{address = URL, action = SoapAction} ->\n            %% Add the Soap envelope\n            Envelope = mk_envelope(Message, Headers),\n            %% Encode the message\n            case erlsom:write(Envelope, Model) of\n                {ok, XmlMessage} ->\n\n                    {ContentType, Request} =\n                        make_request_body(XmlMessage, Attachments),\n                    HttpHeaders = [],\n                    HttpClientOptions = [],\n                    ?dbg(\"+++ Request = ~p~n\", [Request]),\n                    HttpRes = http_request(URL, SoapAction, Request,\n                                           HttpClientOptions, HttpHeaders,\n                                           ContentType),\n                    ?dbg(\"+++ HttpRes = ~p~n\", [HttpRes]),\n                    case HttpRes of\n                        {ok, _Code, _ReturnHeaders, Body} ->\n                            parseMessage(Body, Model);\n                        Error ->\n                            %% in case of HTTP error: return\n                            %% {error, description}\n                            Error\n                    end;\n                {error, EncodingError} ->\n                    {error, {encoding_error, EncodingError}}\n            end;\n        false ->\n            {error, {unknown_operation, Operation}}\n    end.\n\n%%%\n%%% returns {ok, Header, Body} | {error, Error}\n%%%\nparseMessage(Message, #wsdl{model = Model}) ->\n    parseMessage(Message, Model);\n%%\nparseMessage(Message, Model) ->\n    case erlsom:scan(Message, Model) of\n        {ok, #'soap:Envelope'{'Body' = #'soap:Body'{choice = Body},\n                              'Header' = undefined}, _} ->\n            {ok, undefined, Body};\n        {ok, #'soap:Envelope'{'Body' = #'soap:Body'{choice = Body},\n                              'Header' = #'soap:Header'{choice = Header}}, _} ->\n            {ok, Header, Body};\n        {error, ErrorMessage} ->\n            {error, {decoding, Message, ErrorMessage}}\n    end.\n\n\nfindOperation(_Operation, _Port, _Service, []) ->\n    false;\nfindOperation(Operation, Port, Service,\n              [Op = #operation{operation = Operation,\n                               port = Port, service = Service} | _]) ->\n    Op;\nfindOperation(Operation, Port, Service, [#operation{} | Tail]) ->\n    findOperation(Operation, Port, Service, Tail).\n\n\nmk_envelope(M, H) when is_tuple(M) -> mk_envelope([M], H);\nmk_envelope(M, H) when is_tuple(H) -> mk_envelope(M, [H]);\n%%\nmk_envelope(Messages, []) when is_list(Messages) ->\n    #'soap:Envelope'{'Body' =  #'soap:Body'{choice = Messages}};\nmk_envelope(Messages, Headers) when is_list(Messages),is_list(Headers) ->\n    #'soap:Envelope'{'Body'   =  #'soap:Body'{choice   = Messages},\n                     'Header' =  #'soap:Header'{choice = Headers}}.\n\n%%% --------------------------------------------------------------------\n%%% Parse a WSDL file and return a 'Model'\n%%% --------------------------------------------------------------------\ninitModel(WsdlFile) ->\n    initModel(WsdlFile, ?DefaultPrefix).\n\n%% PrefixOrOptions can be a property list that contains the options\n%% for Erlsom, or a String. If it is a string, this is used as the\n%% Erlsom 'prefix' option (and the other options are left unspecified).\ninitModel(WsdlFile, PrefixOrOptions) ->\n    Options = case is_string(PrefixOrOptions) of\n        no ->\n          %% It is an option list\n          %% Add the default prefix at the end - it will only be used\n          %% if no other prefix is specified\n          PrefixOrOptions ++ [{prefix, ?DefaultPrefix}];\n        _ ->\n          %% just the prefix\n          [{prefix, PrefixOrOptions}]\n    end,\n    PrivDir = priv_dir(),\n    initModel2(WsdlFile, Options, PrivDir, undefined, undefined).\n\ninitModelFile(ConfigFile) ->\n    {ok, ConfigSchema} = erlsom:compile_xsd(config_file_xsd()),\n    %% read (parse) the config file\n    {ok, Config, _} = erlsom:scan_file(ConfigFile, ConfigSchema),\n    #yaws_soap_config{xsd_path = XsdPath,\n                      wsdl_file = Wsdl,\n                      add_files = AddFiles} = Config,\n    #xsd_file{name = WsdlFile, prefix = Prefix, import_specs = Import} = Wsdl,\n    initModel2(WsdlFile, [{prefix, Prefix}], XsdPath, Import, AddFiles).\n\npriv_dir() ->\n    yaws:get_priv_dir().\n\ninitModel2(WsdlFile, ErlsomOptions, Path, Import, AddFiles) ->\n    WsdlName = filename:join([Path, \"wsdl.xsd\"]),\n    IncludeWsdl = {\"http://schemas.xmlsoap.org/wsdl/\", \"wsdl\", WsdlName},\n    {ok, WsdlModel} = erlsom:compile_xsd_file(\n                        filename:join([Path, \"soap.xsd\"]),\n                        [{prefix, \"soap\"},\n                         {include_files, [IncludeWsdl]}]),\n    %% add the xsd model (since xsd is also used in the wsdl)\n    WsdlModel2 = erlsom:add_xsd_model(WsdlModel),\n    Options = ErlsomOptions ++ makeOptions(Import),\n    %% parse Wsdl\n    {Model, Operations} = parseWsdls([WsdlFile], WsdlModel2,\n                                     Options, {undefined, []}),\n    %% TODO: add files as required\n    %% now compile envelope.xsd, and add Model\n    {ok, EnvelopeModel} = erlsom:compile_xsd_file(\n                            filename:join([Path, \"envelope.xsd\"]),\n                            [{prefix, \"soap\"}]),\n    SoapModel = erlsom:add_model(EnvelopeModel, Model),\n    SoapModel2 = addModels(AddFiles, SoapModel),\n    #wsdl{operations = Operations, model = SoapModel2}.\n\n\n%%% --------------------------------------------------------------------\n%%% Parse a list of WSDLs and import (recursively)\n%%% Returns {Model, Operations}\n%%% --------------------------------------------------------------------\nparseWsdls([], _WsdlModel, _Options, Acc) ->\n    Acc;\nparseWsdls([WsdlFile | Tail], WsdlModel, Options, {AccModel, AccOperations}) ->\n    {ok, WsdlFileContent} = get_url_file(rmsp(WsdlFile)),\n    {ok, ParsedWsdl, _} = erlsom:scan(WsdlFileContent, WsdlModel),\n    %% get the xsd elements from this model, and hand it over to erlsom_compile.\n    Xsds = getXsdsFromWsdl(ParsedWsdl),\n    %% Now we need to build a list: [{Namespace, Xsd, Prefix}, ...] for\n    %% all the Xsds in the WSDL.\n    %% This list is used when a schema includes one of the other schemas.\n    %% The AXIS java2wsdl tool generates wsdls that depend on this feature.\n    ImportList = makeImportList(Xsds, []),\n    Model2 = addSchemas(Xsds, AccModel, Options, ImportList),\n    Ports = getPorts(ParsedWsdl),\n    Operations = getOperations(ParsedWsdl, Ports),\n    Imports = getImports(ParsedWsdl),\n    Acc2 = {Model2, Operations ++ AccOperations},\n    %% process imports (recursively, so that imports in the imported files are\n    %% processed as well).\n    %% For the moment, the namespace is ignored on operations etc.\n    %% this makes it a bit easier to deal with imported wsdl's.\n    Acc3 = parseWsdls(Imports, WsdlModel, Options, Acc2),\n    parseWsdls(Tail, WsdlModel, Options, Acc3).\n\n%%% --------------------------------------------------------------------\n%%% build a list: [{Namespace, Xsd}, ...] for all the Xsds in the WSDL.\n%%% This list is used when a schema inlcudes one of the other schemas.\n%%% The AXIS java2wsdl tool generates wsdls that depend on this feature.\nmakeImportList([], Acc) ->\n    Acc;\nmakeImportList([ Xsd | Tail], Acc) ->\n    makeImportList(Tail, [{erlsom_lib:getTargetNamespaceFromXsd(Xsd),\n                           undefined, Xsd} | Acc]).\n\n\n%%% --------------------------------------------------------------------\n%%% compile each of the schemas, and add it to the model.\n%%% Returns Model\n%%% (TODO: using the same prefix for all XSDS makes no sense)\n%%% --------------------------------------------------------------------\naddSchemas([], AccModel, _Options, _ImportList) ->\n    AccModel;\naddSchemas([Xsd| Tail], AccModel, Options, ImportList) ->\n    Model2 = case Xsd of\n                 undefined ->\n                     AccModel;\n                 _ ->\n                     {ok, Model} =\n                         erlsom_compile:compile_parsed_xsd(\n                           Xsd,\n                           [{include_files, ImportList} |Options]),\n                     case AccModel of\n                         undefined -> Model;\n                         _ -> erlsom:add_model(AccModel, Model)\n                     end\n             end,\n    addSchemas(Tail, Model2, Options, ImportList).\n\n%%% --------------------------------------------------------------------\n%%% Get a file from an URL spec.\n%%% --------------------------------------------------------------------\nget_url_file(\"http://\"++_ = URL) ->\n    case httpc:request(URL) of\n        {ok,{{_HTTP,200,_OK}, _Headers, Body}} ->\n            {ok, Body};\n        {ok,{{_HTTP,RC,Emsg}, _Headers, _Body}} ->\n            error_logger:error_msg(\"~p: http-request got: ~p~n\",\n                                   [?MODULE, {RC, Emsg}]),\n            {error, \"failed to retrieve: \"++URL};\n        {error, Reason} ->\n            error_logger:error_msg(\"~p: http-request failed: ~p~n\",\n                                   [?MODULE, Reason]),\n            {error, \"failed to retrieve: \"++URL}\n    end;\nget_url_file(\"file://\"++Fname) ->\n    {ok, Bin} = file:read_file(Fname),\n    {ok, binary_to_list(Bin)};\n%% added this, since this is what is used in many WSDLs (i.e.: just a filename).\nget_url_file(Fname) ->\n    {ok, Bin} = file:read_file(Fname),\n    {ok, binary_to_list(Bin)}.\n\n\n%%% --------------------------------------------------------------------\n%%% Make a HTTP Request\n%%% --------------------------------------------------------------------\nhttp_request(URL, SoapAction, Request, Options, Headers, ContentType) ->\n    case code:ensure_loaded(ibrowse) of\n        {module, ibrowse} ->\n            %% If ibrowse exist in the path then let's use it...\n            ibrowse_request(URL, SoapAction, Request, Options,\n                            Headers, ContentType);\n        _ ->\n            %% ...otherwise, let's use the OTP http client.\n            inets_request(URL, SoapAction, Request, Options,\n                          Headers, ContentType)\n    end.\n\ninets_request(URL, SoapAction, Request, Options, Headers, ContentType) ->\n    NHeaders = [{\"SOAPAction\", SoapAction}|Headers],\n    NewHeaders = case proplists:get_value(\"Host\", NHeaders) of\n                     undefined ->\n                         [{\"Host\", \"localhost:8800\"}|NHeaders];\n                     _ ->\n                         NHeaders\n                 end,\n    NewOptions = [{cookies, enabled}|Options],\n    httpc:set_options(NewOptions),\n    case httpc:request(post,\n                       {URL,NewHeaders,\n                        ContentType,\n                        Request},\n                       [{timeout,?HTTP_REQ_TIMEOUT}],\n                       [{sync, true}, {full_result, true},\n                        {body_format, string}]) of\n        {ok,{{_HTTP,200,_OK},ResponseHeaders,ResponseBody}} ->\n            {ok, 200, ResponseHeaders, ResponseBody};\n        {ok,{{_HTTP,500,_Descr},ResponseHeaders,ResponseBody}} ->\n            {ok, 500, ResponseHeaders, ResponseBody};\n        {ok,{{_HTTP,ErrorCode,_Descr},ResponseHeaders,ResponseBody}} ->\n            {ok, ErrorCode, ResponseHeaders, ResponseBody};\n        Other ->\n            Other\n    end.\n\nibrowse_request(URL, SoapAction, Request, Options, Headers, ContentType) ->\n    case start_ibrowse() of\n        ok ->\n            NewHeaders = [{\"Content-Type\", ContentType},\n                          {\"SOAPAction\", SoapAction} | Headers],\n            case ibrowse:send_req(URL, NewHeaders, post, Request, Options) of\n                {ok, Status, ResponseHeaders, ResponseBody} ->\n                    {ok, list_to_integer(Status), ResponseHeaders,\n                     ResponseBody};\n                {error, Reason} ->\n                    {error, Reason}\n            end;\n        error ->\n            {error, \"could not start ibrowse\"}\n    end.\n\nstart_ibrowse() ->\n    case ibrowse:start() of\n        {ok, _} -> ok;\n        {error, {already_started, _}} -> ok;\n        _ -> error\n    end.\n\n\nrmsp(Str) -> string:strip(Str, left).\n\n\nmake_request_body(Content, []) ->\n    {\"text/xml; charset=utf-8\",\n     \"<?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?>\"++ Content};\nmake_request_body(Content, AttachedFiles) ->\n    {\"application/dime\",\n     yaws_dime:encode(\"<?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?>\" ++ Content,\n                      AttachedFiles)}.\n\nmakeFault(FaultCode, FaultString) ->\n    try\n        \"<SOAP-ENV:Envelope xmlns:SOAP-ENV=\\\"http://schemas.xmlsoap.org/soap/envelope/\\\">\"\n            \"<SOAP-ENV:Body>\"\n            \"<SOAP-ENV:Fault>\"\n            \"<faultcode>SOAP-ENV:\" ++ FaultCode ++ \"</faultcode>\" ++\n            \"<faultstring>\" ++ FaultString ++ \"</faultstring>\" ++\n            \"</SOAP-ENV:Fault>\"\n            \"</SOAP-ENV:Body>\"\n            \"</SOAP-ENV:Envelope>\"\n    catch\n        _:_ ->\n            \"<SOAP-ENV:Envelope xmlns:SOAP-ENV=\\\"http://schemas.xmlsoap.org/soap/envelope/\\\">\"\n                \"<SOAP-ENV:Body>\"\n                \"<SOAP-ENV:Fault>\"\n                \"<faultcode>SOAP-ENV:Server</faultcode>\"\n                \"<faultstring>Server error</faultstring>\"\n                \"</SOAP-ENV:Fault>\"\n                \"</SOAP-ENV:Body>\"\n                \"</SOAP-ENV:Envelope>\"\n    end.\n\n%% record http_header is not defined??\nfindHeader(Label, Headers) ->\n    findHeader0(yaws:to_lower(Label), Headers).\n\nfindHeader0(_Label, []) ->\n    undefined;\nfindHeader0(Label, [{_,_,Hdr,_,Val}|T]) ->\n    case {Label, yaws:to_lower(Hdr)} of\n        {X,X} -> Val;\n        _     -> findHeader0(Label, T)\n    end;\nfindHeader0(_Label, undefined) ->\n    undefined.\n\n\nmakeOptions(undefined) ->\n    [];\nmakeOptions(Import) ->\n    lists:map(fun makeOption/1, Import).\n\n%% -record(import_specs, {atts, namespace, prefix, location}).\nmakeOption(#import_specs{namespace = Ns, prefix = Pf, location = Lc}) ->\n    {Ns, Pf, Lc}.\n\n\naddModels(undefined, Model) ->\n    Model;\naddModels(Import, Model) ->\n    lists:foldl(fun addModel/2, Model, Import).\n\n%% -record(xsd_file, {atts, name, prefix, import_specs}).\naddModel(undefined, Acc) ->\n    Acc;\naddModel(#xsd_file{name = XsdFile, prefix = Prefix, import_specs = Import},\n         Acc) ->\n    Options = makeOptions(Import),\n    {ok, Model2} = erlsom:add_xsd_file(XsdFile, [{prefix, Prefix}|Options],Acc),\n    Model2.\n\n%% returns [#port{}]\n%% -record(port, {service, port, binding, address}).\ngetPorts(ParsedWsdl) ->\n    Services = getTopLevelElements(ParsedWsdl, 'wsdl:tService'),\n    getPortsFromServices(Services, []).\n\ngetPortsFromServices([], Acc) ->\n    Acc;\ngetPortsFromServices([Service|Tail], Acc) ->\n    getPortsFromServices(Tail, getPortsFromService(Service) ++ Acc).\n\ngetPortsFromService(#'wsdl:tService'{name = Name, port = Ports}) ->\n    getPortsInfo(Ports, Name, []).\n\ngetPortsInfo([], _Name, Acc) ->\n    Acc;\n\ngetPortsInfo([#'wsdl:tPort'{name = Name,\n                            binding = Binding,\n                            choice =\n                                [#'soap:tAddress'{location = URL}]} | Tail],\n             ServiceName, Acc) ->\n    getPortsInfo(Tail, ServiceName, [#port{service = ServiceName,\n                                           port = Name,\n                                           binding = Binding,\n                                           address = URL}|Acc]);\n%% non-soap bindings are ignored.\ngetPortsInfo([#'wsdl:tPort'{} | Tail], ServiceName, Acc) ->\n    getPortsInfo(Tail, ServiceName, Acc).\n\n\ngetTopLevelElements(#'wsdl:tDefinitions'{choice = TLElements}, Type) ->\n    getTopLevelElements(TLElements, Type, []).\n\ngetTopLevelElements([], _Type, Acc) ->\n    Acc;\ngetTopLevelElements([#'wsdl:anyTopLevelOptionalElement'{choice = Tuple}| Tail],\n                    Type, Acc) ->\n    case element(1, Tuple) of\n        Type -> getTopLevelElements(Tail, Type, [Tuple|Acc]);\n        _ -> getTopLevelElements(Tail, Type, Acc)\n    end.\n\ngetImports(Definitions) ->\n    Imports = getTopLevelElements(Definitions, 'wsdl:tImport'),\n    lists:map(fun(Import) -> Import#'wsdl:tImport'.location end, Imports).\n\n%% returns [#operation{}]\ngetOperations(ParsedWsdl, Ports) ->\n    Bindings = getTopLevelElements(ParsedWsdl, 'wsdl:tBinding'),\n    getOperationsFromBindings(Bindings, Ports, []).\n\ngetOperationsFromBindings([], _Ports, Acc) ->\n    Acc;\ngetOperationsFromBindings([Binding|Tail], Ports, Acc) ->\n    getOperationsFromBindings(Tail, Ports,\n                              getOperationsFromBinding(Binding, Ports) ++ Acc).\n\ngetOperationsFromBinding(#'wsdl:tBinding'{name = BindingName,\n                                          type = BindingType,\n                                          choice = _Choice,\n                                          operation = Operations}, Ports) ->\n    %% TODO: get soap info from Choice\n    getOperationsFromOperations(Operations, BindingName, BindingType,\n                                Operations, Ports, []).\n\ngetOperationsFromOperations([], _BindingName, _BindingType, _Operations,\n                            _Ports, Acc) ->\n    Acc;\n\ngetOperationsFromOperations([#'wsdl:tBindingOperation'{name = Name,\n                                                       choice = Choice} | Tail],\n                            BindingName, BindingType, Operations, Ports, Acc) ->\n    %% get SOAP action from Choice,\n    case Choice of\n        [#'soap:tOperation'{soapAction = Action}] ->\n            %% lookup Binding in Ports, and create a combined result\n            Ports2 = searchPorts(BindingName, Ports),\n            %% for each port, make an operation record\n            CombinedPorts = combinePorts(Ports2, Name, BindingName, Action),\n            getOperationsFromOperations(\n              Tail, BindingName, BindingType,\n              Operations, Ports, CombinedPorts ++ Acc);\n        _ ->\n            getOperationsFromOperations(Tail, BindingName, BindingType,\n                                        Operations, Ports, Acc)\n    end.\n\ncombinePorts(Ports, Name, BindingName, Action) ->\n    combinePorts(Ports, Name, BindingName, Action, []).\n\ncombinePorts([], _Name, _BindingName, _Action, Acc) ->\n    Acc;\ncombinePorts([#port{service = Service,\n                    port = PortName,\n                    address = Address} | Tail],\n             Name, BindingName, Action, Acc) ->\n    combinePorts(Tail, Name, BindingName, Action,\n                 [#operation{service = Service,\n                             port = PortName, operation = Name,\n                             binding = BindingName,\n                             address = Address, action = Action} | Acc]).\n\nsearchPorts(BindingName, Ports) ->\n    searchPorts(BindingName, Ports, []).\n\nsearchPorts(_BindingName, [], Acc) ->\n    Acc;\nsearchPorts(BindingName, [Port | Tail], Acc) ->\n    PortBinding = erlsom_lib:localName(Port#port.binding),\n    case PortBinding of\n        BindingName ->\n            searchPorts(BindingName, Tail, [Port | Acc]);\n        _ ->\n            searchPorts(BindingName, Tail, Acc)\n    end.\n\n%% copied from yaws/json.erl\nis_string([]) -> yes;\nis_string(List) -> is_string(List, non_unicode).\n\nis_string([C|Rest], non_unicode)\n  when C >= 0, C =< 255 -> is_string(Rest, non_unicode);\nis_string([C|Rest], _) when C =< 65000 -> is_string(Rest, unicode);\nis_string([], non_unicode) -> yes;\nis_string([], unicode) -> unicode;\nis_string(_, _) -> no.\n\ngetXsdsFromWsdl(Definitions) ->\n    case getTopLevelElements(Definitions, 'wsdl:tTypes') of\n        [#'wsdl:tTypes'{choice = Xsds}] -> Xsds;\n        [] -> []\n    end.\n\nconfig_file_xsd() ->\n    \"<xs:schema xmlns:xs=\\\"http://www.w3.org/2001/XMLSchema\\\">\"\n        \"  <xs:element name=\\\"yaws_soap_config\\\">\"\n        \"     <xs:complexType>\"\n        \"       <xs:sequence>\"\n        \"         <xs:element name=\\\"xsd_path\\\" type=\\\"xs:string\\\" minOccurs=\\\"0\\\"/>\"\n        \"         <xs:element name=\\\"user_module\\\" type=\\\"xs:string\\\"/>\"\n        \"         <xs:element name=\\\"wsdl_file\\\" type=\\\"xsd_file\\\"/>\"\n        \"         <xs:element name=\\\"add_file\\\" type=\\\"xsd_file\\\" minOccurs=\\\"0\\\" maxOccurs=\\\"unbounded\\\"/>\"\n        \"       </xs:sequence>\"\n        \"     </xs:complexType>\"\n        \"  </xs:element>\"\n        \"  <xs:complexType name=\\\"xsd_file\\\">\"\n        \"    <xs:sequence>\"\n        \"      <xs:element name=\\\"import_specs\\\" type=\\\"import_specs\\\" minOccurs=\\\"0\\\" maxOccurs=\\\"unbounded\\\"/>\"\n        \"    </xs:sequence>\"\n        \"    <xs:attribute name=\\\"name\\\" type=\\\"string\\\" use=\\\"required\\\"/>\"\n        \"    <xs:attribute name=\\\"prefix\\\" type=\\\"string\\\"/>\"\n        \"  </xs:complexType>\"\n        \"  <xs:complexType name=\\\"import_specs\\\">\"\n        \"    <xs:attribute name=\\\"namespace\\\" type=\\\"string\\\" use=\\\"required\\\"/>\"\n        \"    <xs:attribute name=\\\"prefix\\\" type=\\\"string\\\"/>\"\n        \"    <xs:attribute name=\\\"location\\\" type=\\\"string\\\"/>\"\n        \"  </xs:complexType>\"\n        \"</xs:schema>\".\n\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_soap_srv.erl",
    "content": "%%%-------------------------------------------------------------------\n%%% Created : 29 Nov 2006 by Torbjorn Tornkvist <tobbe@tornkvist.org>\n%%% Author  : Willem de Jong (w.a.de.jong@gmail.com).\n%%% Desc    : A SOAP server.\n%%%-------------------------------------------------------------------\n-module(yaws_soap_srv).\n\n-behaviour(gen_server).\n\n%% API\n-export([start_link/0, start_link/1,\n         setup/1, setup/2, setup/3,\n         handler/4\n        ]).\n\n%% gen_server callbacks\n-export([init/1, handle_call/3, handle_cast/2, handle_info/2,\n         terminate/2, code_change/3]).\n\n-include(\"../include/yaws_api.hrl\").\n-include(\"../include/yaws.hrl\").\n-include(\"../include/soap.hrl\").\n\n-define(SERVER, ?MODULE).\n\n%% State\n-record(s, {\n          wsdl_list = []  % list of {Id, WsdlModel} tuples, where Id == {M,F}\n         }).\n\n-define(OK_CODE, 200).\n-define(BAD_MESSAGE_CODE, 400).\n%% -define(METHOD_NOT_ALLOWED_CODE, 405).\n-define(SERVER_ERROR_CODE, 500).\n\n%%====================================================================\n%% API\n%%====================================================================\n%%--------------------------------------------------------------------\n%% Function: start_link() -> {ok,Pid} | ignore | {error,Error}\n%% Description: Starts the server\n%%--------------------------------------------------------------------\nstart_link() ->\n    start_link([]).\nstart_link(L) ->\n    %% We are dependent on erlsom\n    case code:ensure_loaded(erlsom) of\n        {error, _} ->\n            Emsg = \"could not load erlsom\",\n            error_logger:error_msg(\"~p: exiting, reason: ~s~n\",\n                                   [?MODULE, Emsg]),\n            {error, Emsg};\n        {module, erlsom} ->\n            gen_server:start_link({local, ?SERVER}, ?MODULE, L, [])\n    end.\n\n%%% To be called from yaws_rpc.erl\n%%% Return according to yaws_rpc:eval_payload/6\nhandler(Args, Id, Payload, SessionValue) ->\n    {ok, WsdlModel} = gen_server:call(?SERVER, {get_wsdl, Id}, infinity),\n    Headers = Args#arg.headers,\n    SoapAction = yaws_soap_lib:findHeader(\"SOAPAction\", Headers#headers.other),\n    Res = request(WsdlModel, Id, Payload, SessionValue, SoapAction),\n    case Res of\n        {ok, XmlDoc, ResCode, undefined} ->\n            {false, XmlDoc, ResCode};\n        {ok, XmlDoc, ResCode, SessVal} ->\n            {true, 0, SessVal, XmlDoc, ResCode};\n        {error, _, _} = Error ->\n            Error;\n        false ->\n            false    % soap notify\n    end.\n\n%% Setup a SOAP interface according to the config file.\nsetup(_ConfigFile) ->\n    tbd.\n\nsetup(Id, WsdlFile) when is_tuple(Id),size(Id)==2 ->\n    Wsdl = yaws_soap_lib:initModel(WsdlFile),\n    gen_server:call(?SERVER, {add_wsdl, Id, Wsdl}, infinity).\n\n%% PrefixOrOptions can be either a prefix (a String) or a property\n%% list. It is used to construct the options that are passed to Erlsom\n%% to compile the WSDL file. Passing a string (\"Prefix\") is equivalent\n%% to [{prefix, \"Prefix\"}].\n%% If a list of erlsom options is passed, and this does not contain\n%% the {prefix, ...} option, the yaws_soap default (\"p\") will be used.\nsetup(Id, WsdlFile, PrefixOrOptions) when is_tuple(Id),size(Id)==2 ->\n    Wsdl = yaws_soap_lib:initModel(WsdlFile, PrefixOrOptions),\n    gen_server:call(?SERVER, {add_wsdl, Id, Wsdl}, infinity).\n\n\n\n%%====================================================================\n%% gen_server callbacks\n%%====================================================================\n\n%%--------------------------------------------------------------------\n%% Function: init(Args) -> {ok, State} |\n%%                         {ok, State, Timeout} |\n%%                         ignore               |\n%%                         {stop, Reason}\n%% Description: Initiates the server\n%%--------------------------------------------------------------------\ninit(L) -> %% [ {{Mod,Handler}, WsdlFile} ]\n    WsdlList = lists:foldl( fun( SoapSrvMod, OldList) ->\n                                    setup_on_init( SoapSrvMod, OldList )\n                            end,[],L),\n    {ok, #s{wsdl_list = WsdlList}}.\n\nsetup_on_init( {Id, WsdlFile}, OldList ) when is_tuple(Id),size(Id) == 2 ->\n    Wsdl = yaws_soap_lib:initModel(WsdlFile),\n    uinsert({Id, Wsdl}, OldList);\nsetup_on_init( {Id, WsdlFile, Prefix}, OldList ) when is_tuple(Id),\n                                                      size(Id) == 2 ->\n    Wsdl = yaws_soap_lib:initModel(WsdlFile, Prefix),\n    uinsert({Id, Wsdl}, OldList).\n\n%%--------------------------------------------------------------------\n%% Function: %% handle_call(Request, From, State) -> {reply, Reply, State} |\n%%                                      {reply, Reply, State, Timeout} |\n%%                                      {noreply, State} |\n%%                                      {noreply, State, Timeout} |\n%%                                      {stop, Reason, Reply, State} |\n%%                                      {stop, Reason, State}\n%% Description: Handling call messages\n%%--------------------------------------------------------------------\nhandle_call({add_wsdl, Id, WsdlModel}, _From, State) ->\n    NewWsdlList = uinsert({Id, WsdlModel}, State#s.wsdl_list),\n    {reply, ok, State#s{wsdl_list = NewWsdlList}};\nhandle_call({get_wsdl, Id}, _From, State) ->\n    {reply, get_model(State, Id), State}.\n\n%%--------------------------------------------------------------------\n%% Function: handle_cast(Msg, State) -> {noreply, State} |\n%%                                      {noreply, State, Timeout} |\n%%                                      {stop, Reason, State}\n%% Description: Handling cast messages\n%%--------------------------------------------------------------------\nhandle_cast(_Msg, State) ->\n    {noreply, State}.\n\n%%--------------------------------------------------------------------\n%% Function: handle_info(Info, State) -> {noreply, State} |\n%%                                       {noreply, State, Timeout} |\n%%                                       {stop, Reason, State}\n%% Description: Handling all non call/cast messages\n%%--------------------------------------------------------------------\nhandle_info(_Info, State) ->\n    {noreply, State}.\n\n%%--------------------------------------------------------------------\n%% Function: terminate(Reason, State) -> void()\n%% Description: This function is called by a gen_server when it is about to\n%% terminate. It should be the opposite of Module:init/1 and do any necessary\n%% cleaning up. When it returns, the gen_server terminates with Reason.\n%% The return value is ignored.\n%%--------------------------------------------------------------------\nterminate(_Reason, _State) ->\n    ok.\n\n%%--------------------------------------------------------------------\n%% Func: code_change(OldVsn, State, Extra) -> {ok, NewState}\n%% Description: Convert process state when code is changed\n%%--------------------------------------------------------------------\ncode_change(_OldVsn, State, _Extra) ->\n    {ok, State}.\n\n%%--------------------------------------------------------------------\n%%% Internal functions\n%%--------------------------------------------------------------------\n\nrequest(Model, {M,F}, {Req, Attachments}, SessionValue, Action) ->\n    %%error_logger:info_report([?MODULE, {payload, Req}]),\n    case catch yaws_soap_lib:parseMessage(Req, Model) of\n        {ok, Header, Body} ->\n            %% call function\n            result(Model, catch apply(M, F, [Header, Body,\n                                             Action, SessionValue,\n\t\t\t\t\t     Attachments]));\n        {error, Error} ->\n            cli_error(Error);\n        OtherError ->\n            srv_error(io_lib:format(\"Error parsing message: ~p\", [OtherError]))\n    end;\nrequest(Model, {M,F}, Req, SessionValue, Action) ->\n    %%error_logger:info_report([?MODULE, {payload, Req}]),\n    Umsg = (catch erlsom_lib:toUnicode(Req)),\n    case catch yaws_soap_lib:parseMessage(Umsg, Model) of\n        {ok, Header, Body} ->\n            %% call function\n            result(Model, catch apply(M, F, [Header, Body,\n                                             Action, SessionValue]));\n        {error, Error} ->\n            cli_error(Error);\n        OtherError ->\n            srv_error(io_lib:format(\"Error parsing message: ~p\", [OtherError]))\n    end.\n\n%%% Analyse the result and produce some output\nresult(Model, {ok, ResHeader, ResBody, ResCode, SessVal}) ->\n    return(Model, ResHeader, ResBody, ResCode, SessVal, undefined);\nresult(Model, {ok, ResHeader, ResBody}) ->\n    return(Model, ResHeader, ResBody, ?OK_CODE, undefined, undefined);\nresult(Model, {ok, ResHeader, ResBody, Files}) ->\n    return(Model, ResHeader, ResBody, ?OK_CODE, undefined, Files);\nresult(_Model, {error, client, ClientMssg}) ->\n    cli_error(ClientMssg);\nresult(_Model, false) ->   % soap notify !\n    false;\nresult(_Model, Error) ->\n    srv_error(io_lib:format(\"Error processing message: ~p\", [Error])).\n\nreturn(#wsdl{model = Model}, ResHeader, ResBody, ResCode, SessVal, Files) ->\n    return(Model, ResHeader, ResBody, ResCode, SessVal, Files);\nreturn(Model, ResHeader, ResBody, ResCode, SessVal, Files)\n  when not is_list(ResBody) ->\n    return(Model, ResHeader, [ResBody], ResCode, SessVal, Files);\nreturn(Model, ResHeader, ResBody, ResCode, SessVal, Files) ->\n    %% add envelope\n    Header2 = case ResHeader of\n                  undefined -> undefined;\n                  _         -> #'soap:Header'{choice = ResHeader}\n              end,\n    Envelope = #'soap:Envelope'{'Body' =  #'soap:Body'{choice = ResBody},\n                                'Header' = Header2},\n    case catch erlsom:write(Envelope, Model) of\n        {ok, XmlDoc} ->\n\t    case Files of\n\t\tundefined ->\n\t\t    {ok, XmlDoc, ResCode, SessVal};\n\t\t_ ->\n\t\t    DIME = yaws_dime:encode(XmlDoc, Files),\n\t\t    {ok, DIME, ResCode, SessVal}\n\t    end;\n        {error, WriteError} ->\n            srv_error(f(\"Error writing XML: ~p\", [WriteError]));\n        OtherWriteError ->\n            error_logger:error_msg(\"~p(~p): OtherWriteError=~p~n\",\n                                   [?MODULE, ?LINE, OtherWriteError]),\n            srv_error(f(\"Error writing XML: ~p\", [OtherWriteError]))\n    end.\n\nf(S,A) -> lists:flatten(io_lib:format(S,A)).\n\ncli_error(Error) ->\n    error_logger:error_msg(\"~p(~p): Cli Error: ~p~n\",\n                           [?MODULE, ?LINE, Error]),\n    Fault = yaws_soap_lib:makeFault(\"Client\", \"Client error\"),\n    {error, Fault, ?BAD_MESSAGE_CODE}.\n\nsrv_error(Error) ->\n    error_logger:error_msg(\"~p(~p): Srv Error: ~p~n\",\n                           [?MODULE, ?LINE, Error]),\n    Fault = yaws_soap_lib:makeFault(\"Server\", \"Server error\"),\n    {error, Fault, ?SERVER_ERROR_CODE}.\n\n\n\n\nget_model(State, Id) ->\n    case lists:keysearch(Id, 1, State#s.wsdl_list) of\n        {value, {_, Model}} -> {ok, Model};\n        _                   -> {error, \"model not found\"}\n    end.\n\nuinsert({K,_} = E, [{K,_}|T]) -> [E|T];\nuinsert(E, [H|T])             -> [H|uinsert(E,T)];\nuinsert(E, [])                -> [E].\n"
  },
  {
    "path": "contrib/yaws/src/yaws_sse.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_sse.erl\n%%% Author  : Steve Vinoski <vinoski@ieee.org>\n%%% Purpose : Support for Server-Sent Events\n%%% Created : 31 May 2012 by Steve Vinoski <vinoski@ieee.org>\n%%%----------------------------------------------------------------------\n-module(yaws_sse).\n-author('vinoski@ieee.org').\n\n-export([headers/1,\n         event/0, event/1,\n         data/0, data/1, data/2,\n         id/0, id/1,\n         retry/0, retry/1,\n         comment/1,\n         send_events/2, send_events/3\n        ]).\n\nheaders(StreamPid) ->\n    [{status, 200},\n     {header, {\"Cache-Control\", \"no-cache\"}},\n     {header, {connection, \"close\"}},\n     {header, {transfer_encoding, erase}},\n     {streamcontent_from_pid, \"text/event-stream\", StreamPid}].\n\nevent() ->\n    <<\"event\\n\">>.\nevent(EventName) ->\n    [<<\"event:\">>, EventName, <<\"\\n\">>].\n\ndata() ->\n    <<\"data\\n\">>.\ndata(Data) ->\n    [<<\"data:\">>, Data, <<\"\\n\">>].\n%% The version below trims out all embedded newlines. If you send data\n%% containing newlines using the version above, your events will be\n%% misinterpreted at the client. If the result of trimming includes\n%% any empty strings or binaries, they are dropped and not sent.\ndata(Data0, [trim]) ->\n    Bin = iolist_to_binary(Data0),\n    Tokens = case catch binary:split(Bin, <<\"\\n\">>, [global, trim]) of\n                 {'EXIT', {undef, [{binary,split,__}|_]}} ->\n                     %% handle older releases of Erlang\n                     Lst = binary_to_list(Bin),\n                     [Tok || Tok <- string:tokens(Lst, \"\\n\")];\n                 Bins ->\n                     [B || B <- Bins, B /= <<>>]\n             end,\n    [data(Data) || Data <- Tokens].\n\nid() ->\n    <<\"id\\n\">>.\nid(Id) when is_integer(Id) ->\n    [<<\"id:\">>, integer_to_list(Id), <<\"\\n\">>];\nid(Id) ->\n    [<<\"id:\">>, Id, <<\"\\n\">>].\n\nretry() ->\n    <<\"retry\\n\">>.\nretry(ReconnectionTime) ->\n    [<<\"retry:\">>, integer_to_list(ReconnectionTime), <<\"\\n\">>].\n\ncomment(Comment) ->\n    [<<\":\">>, Comment, <<\"\\n\">>].\n\nsend_events(Socket, Events) ->\n    send_events(Socket, Events, fun yaws_api:stream_process_deliver/2).\nsend_events(Socket, Events, SendFun) ->\n    SendFun(Socket, [Events, <<\"\\n\">>]).\n"
  },
  {
    "path": "contrib/yaws/src/yaws_stats.erl",
    "content": "%%%-------------------------------------------------------------------\n%%% File    : yaws_stats.erl\n%%% Author  : Olivier Girondel <olivier@biniou.info>\n%%% Description : Statistics module for Yaws\n%%%\n%%% Created : 11 Apr 2009 by Olivier Girondel <olivier@biniou.info>\n%%%-------------------------------------------------------------------\n-module(yaws_stats).\n\n-behaviour(gen_server).\n\n-include(\"../include/yaws.hrl\").\n\n%% API\n-export([start_link/0, stop/1]).\n\n%% gen_server callbacks\n-export([init/1, handle_call/3, handle_cast/2, handle_info/2,\n         terminate/2, code_change/3]).\n\n-export([hit/0, sent/1]).\n-export([get/1]).\n\n%% statistics\n-record(stats, {\n          hits = 0,\n          sent = 0\n         }).\n\n%%====================================================================\n%% API\n%%====================================================================\n%%--------------------------------------------------------------------\n%% Function: start_link() -> {ok,Pid} | ignore | {error,Error}\n%% Description: Starts the server\n%%--------------------------------------------------------------------\nstart_link() ->\n    gen_server:start_link(?MODULE, [], []).\n\n\nstop(Pid) ->\n    gen_server:cast(Pid, {stop}).\n\n\nhit() ->\n    case get_stats() of\n        undefined ->\n            ok;\n        Pid ->\n            gen_server:cast(Pid, {hit})\n    end.\n\nsent({ok, Bytes}) ->\n    sent(Bytes);\nsent({error, _}) ->\n    ignore;\nsent(Bytes) ->\n    case get_stats() of\n        undefined ->\n            ok;\n        Pid ->\n            gen_server:cast(Pid, {sent, Bytes})\n    end.\n\n\nget(Pid) ->\n    case Pid of\n        undefined ->\n            undefined;\n        Pid ->\n            gen_server:call(Pid, {get})\n    end.\n\n\n%%====================================================================\n%% gen_server callbacks\n%%====================================================================\n\n%%--------------------------------------------------------------------\n%% Function: init(Args) -> {ok, State} |\n%%                         {ok, State, Timeout} |\n%%                         ignore               |\n%%                         {stop, Reason}\n%% Description: Initiates the server\n%%--------------------------------------------------------------------\ninit([]) ->\n    {ok, #stats{}}.\n\n%%--------------------------------------------------------------------\n%% Function: %% handle_call(Request, From, State) -> {reply, Reply, State} |\n%%                                      {reply, Reply, State, Timeout} |\n%%                                      {noreply, State} |\n%%                                      {noreply, State, Timeout} |\n%%                                      {stop, Reason, Reply, State} |\n%%                                      {stop, Reason, State}\n%% Description: Handling call messages\n%%--------------------------------------------------------------------\nhandle_call({get}, _From, State) ->\n    Reply = {State#stats.hits, State#stats.sent},\n    {reply, Reply, State};\n\n\nhandle_call(_Request, _From, State) ->\n    Reply = ok,\n    {reply, Reply, State}.\n\n%%--------------------------------------------------------------------\n%% Function: handle_cast(Msg, State) -> {noreply, State} |\n%%                                      {noreply, State, Timeout} |\n%%                                      {stop, Reason, State}\n%% Description: Handling cast messages\n%%--------------------------------------------------------------------\nhandle_cast({hit}, Stats) ->\n    NewHits = Stats#stats.hits+1,\n    {noreply, Stats#stats{hits=NewHits}};\n\n\nhandle_cast({sent, Bytes}, Stats) ->\n    NewSent = Stats#stats.sent+Bytes,\n    {noreply, Stats#stats{sent=NewSent}};\n\n\nhandle_cast({stop}, Stats) ->\n    {stop, normal, Stats};\n\n\nhandle_cast(_Msg, State) ->\n    {noreply, State}.\n\n%%--------------------------------------------------------------------\n%% Function: handle_info(Info, State) -> {noreply, State} |\n%%                                       {noreply, State, Timeout} |\n%%                                       {stop, Reason, State}\n%% Description: Handling all non call/cast messages\n%%--------------------------------------------------------------------\nhandle_info(_Info, State) ->\n    {noreply, State}.\n\n%%--------------------------------------------------------------------\n%% Function: terminate(Reason, State) -> void()\n%% Description: This function is called by a gen_server when it is about to\n%% terminate. It should be the opposite of Module:init/1 and do any necessary\n%% cleaning up. When it returns, the gen_server terminates with Reason.\n%% The return value is ignored.\n%%--------------------------------------------------------------------\nterminate(_Reason, _State) ->\n    ok.\n\n%%--------------------------------------------------------------------\n%% Func: code_change(OldVsn, State, Extra) -> {ok, NewState}\n%% Description: Convert process state when code is changed\n%%--------------------------------------------------------------------\ncode_change(_OldVsn, State, _Extra) ->\n    {ok, State}.\n\n%%--------------------------------------------------------------------\n%%% Internal functions\n%%--------------------------------------------------------------------\nget_stats() ->\n    (erlang:get(sc))#sconf.stats.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_sup.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_sup.erl\n%%% Author  : Claes Wikstrom <klacke@bluetail.com>\n%%% Purpose :\n%%% Created : 16 Jan 2002 by Claes Wikstrom <klacke@bluetail.com>\n%%%----------------------------------------------------------------------\n\n-module(yaws_sup).\n-author('klacke@bluetail.com').\n-include(\"../include/yaws.hrl\").\n\n-behaviour(supervisor).\n\n%% External exports\n-export([start_link/0]).\n\n%% supervisor callbacks\n-export([init/1]).\n-export([get_app_args/0, child_specs/0]).\n\n-import(lists, [member/2]).\n\n%%%----------------------------------------------------------------------\n%%% API\n%%%----------------------------------------------------------------------\nstart_link() ->\n    supervisor:start_link({local, ?MODULE}, ?MODULE, []).\n\n%%%----------------------------------------------------------------------\n%%% Callback functions from supervisor\n%%%----------------------------------------------------------------------\n\n%%----------------------------------------------------------------------\n%%----------------------------------------------------------------------\ninit([]) ->\n\n    ChildSpecs = child_specs(),\n\n    %% The idea behind this is if we're running in an embedded env,\n    %% typically the supervisor above us wants to control the restarts.\n    %%\n    %% If we're running standalone --heart can restart the entire node\n    %% If heart is not used, we die.\n    %% 0, 1 means that we never want supervisor restarts\n    {ok,{{one_for_all, 0, 1}, ChildSpecs}}.\n\n%%----------------------------------------------------------------------\n%%----------------------------------------------------------------------\nchild_specs() ->\n    YawsLog = {yaws_log, {yaws_log, start_link, []},\n               permanent, 5000, worker, [yaws_log]},\n\n    YawsTrace = {yaws_trace, {yaws_trace, start_link, []},\n                 permanent, 5000, worker, [yaws_trace]},\n\n    YawsServArgs = [_Env = get_app_args()],\n    YawsServ = {yaws_server, {yaws_server, start_link, YawsServArgs},\n                permanent, 120000, worker, [yaws_server]},\n\n    %% and this guy will restart auxiliary procs that can fail\n    Sup = {yaws_sup_restarts,\n           {yaws_sup_restarts, start_link, []},\n           transient, infinity, supervisor, [yaws_sup_restarts]},\n\n    [YawsLog, YawsTrace, YawsServ, Sup].\n\n%%----------------------------------------------------------------------\n%%----------------------------------------------------------------------\nget_app_args() ->\n    AS=init:get_arguments(),\n    Debug = case application:get_env(yaws, debug) of\n                undefined ->\n                    member({yaws, [\"debug\"]}, AS);\n                {ok, Val}  ->\n                    Val\n            end,\n    Trace = case application:get_env(yaws, trace) of\n                undefined ->\n                    case {member({yaws, [\"trace\", \"http\"]}, AS),\n                          member({yaws, [\"trace\", \"traffic\"]}, AS)} of\n                        {true, _} ->\n                            {true, http};\n                        {_, true} ->\n                            {true, traffic};\n                        _ ->\n                            false\n                    end;\n                {ok, http} ->\n                    {true, http};\n                {ok, traffic} ->\n                    {true, traffic};\n                _ ->\n                    false\n            end,\n    TraceOutput = case application:get_env(yaws, traceoutput) of\n                      undefined ->\n                          member({yaws, [\"traceoutput\"]}, AS);\n                      {ok, Val3}  ->\n                          Val3\n                  end,\n    Conf = case application:get_env(yaws, conf) of\n               undefined ->\n                   find_c(AS);\n               {ok, File} ->\n                   {file, File}\n           end,\n    RunMod = case application:get_env(yaws, runmod) of\n                 undefined ->\n                     find_runmod(AS);\n                 {ok,Mod} ->\n                     {ok,Mod}\n             end,\n    Embedded = case application:get_env(yaws, embedded) of\n                   undefined ->\n                       false;\n                   {ok, Emb} ->\n                       Emb\n               end,\n    Id = case application:get_env(yaws, id) of\n             undefined ->\n                 \"default\";\n             {ok, Id0} when is_atom(Id0) ->\n                 atom_to_list(Id0);\n             {ok, Id0} ->\n                 Id0\n         end,\n\n    #env{debug = Debug, trace = Trace,\n         traceoutput = TraceOutput, conf = Conf,\n         runmod = RunMod, embedded = Embedded, id = Id}.\n\n%%----------------------------------------------------------------------\n%%----------------------------------------------------------------------\nfind_c([{conf, [File]} |_]) ->\n    {file, File};\nfind_c([_|T]) ->\n    find_c(T);\nfind_c([]) ->\n    false.\n\n%%----------------------------------------------------------------------\n%%----------------------------------------------------------------------\nfind_runmod([{runmod, [Mod]} |_]) ->\n    {ok,l2a(Mod)};\nfind_runmod([_|T]) ->\n    find_runmod(T);\nfind_runmod([]) ->\n    false.\n\n%%----------------------------------------------------------------------\n%%----------------------------------------------------------------------\nl2a(L) when is_list(L) -> list_to_atom(L);\nl2a(A) when is_atom(A) -> A.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_sup_restarts.erl",
    "content": "%%% File    : yaws_sup_restarts.erl\n%%% Author  : Claes  Wikstrom <klacke@hyber.org>\n%%% Description : Procs that can be restarted\n%%% Created : 13 Jan 2009 by Claes  Wikstrom <klacke@hyber.org>\n\n-module(yaws_sup_restarts).\n\n-behaviour(supervisor).\n\n%% External exports\n-export([start_link/0]).\n\n%% supervisor callbacks\n-export([init/1]).\n\n\n%%%----------------------------------------------------------------------\n%%% API\n%%%----------------------------------------------------------------------\nstart_link() ->\n    supervisor:start_link({local, ?MODULE}, ?MODULE, []).\n\n%%%----------------------------------------------------------------------\n%%% Callback functions from supervisor\n%%%----------------------------------------------------------------------\n\n%%----------------------------------------------------------------------\n%%----------------------------------------------------------------------\ninit([]) ->\n    Sess = {yaws_session_server, {yaws_session_server, start_link, []},\n            permanent, 5000, worker, [yaws_session_server]},\n    YawsRSS = {yaws_rss,\n               {yaws_rss, start_link, []},\n               permanent, 5000, worker, [yaws_rss]},\n\n\n    YawsEventManager = {yaws_event_manager,\n                        {gen_event, start_link,[{local,yaws_event_manager}]},\n                        permanent, 5000, worker, [gen_event]},\n\n\n    %% below, ignore dialyzer warning:\n    %% \"The pattern 'false' can never match the type 'true'\"\n    SendFile = case yaws_sendfile:have_sendfile() of\n                   true ->\n                       [{yaws_sendfile,\n                         {yaws_sendfile, start_link, []},\n                         permanent, 5000, worker, [yaws_sendfile]}];\n                   false ->\n                       []\n               end,\n\n\n    {ok,{{one_for_one, 1, 60}, [Sess, YawsRSS, YawsEventManager] ++ SendFile}}.\n\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_ticker.erl",
    "content": "-module(yaws_ticker).\n-author('klacke@bluetail.com').\n\n-export([ticker/3]).\n\n%% Moved here from module yaws to ease purging of yaws.\n%% cschultz\n\nticker(Time, To, Msg) ->\n    receive\n        {'EXIT', _, _} ->\n            exit(normal)\n    after Time ->\n            To ! Msg\n    end,\n    ?MODULE:ticker(Time, To, Msg).\n"
  },
  {
    "path": "contrib/yaws/src/yaws_trace.erl",
    "content": "-module(yaws_trace).\n-author('christopher@yakaz.com').\n\n-behaviour(gen_server).\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n-include(\"yaws_debug.hrl\").\n\n%% API\n-export([\n         start_link/0,\n         setup/1,\n         set_tty_trace/1,\n         get_type/1,\n         open/0, open/1,\n         write/2, write/3,\n         disable_trace/1, enable_trace/2,\n         get_tracedir/0,\n         set_filter/1, unset_filter/0, get_filter/0,\n\n         %% Internal functions exported to be used in filters\n         get_header_value/2,\n         get_request_value/2\n        ]).\n\n%% gen_server callbacks\n-export([init/1, handle_call/3, handle_cast/2, handle_info/2,\n         terminate/2, code_change/3]).\n\n-record(state, {tracedir,\n                type      = false,\n                traces    = dict:new(),\n                tty_trace = false,\n                filter}).\n\n%%====================================================================\n%% API\n%%====================================================================\nstart_link() ->\n    gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).\n\n\nsetup(GC) ->\n    gen_server:cast(?MODULE, {setup, GC}).\n\n\nset_tty_trace(Bool) ->\n    gen_server:cast(?MODULE, {tty_trace, Bool}).\n\n\nget_type(#gconf{trace={true, Type}}) ->\n    Type;\nget_type(_) ->\n    undefined.\n\n\n\nopen() ->\n    open(self()).\nopen(Owner) ->\n    gen_server:cast(?MODULE, {open, Owner}).\n\n\nwrite(Tag, Data) ->\n    write(self(), Tag, Data).\nwrite(Pid, Tag, Data) ->\n    gen_server:cast(?MODULE, {write, Pid, Tag, Data}).\n\n\ndisable_trace(infinity) ->\n    {ok, GC, Groups} = yaws_server:getconf(),\n    case GC#gconf.trace of\n        false ->\n            error_logger:info_msg(\"Trace already disabled~n\", []);\n        {true, _} ->\n            yaws_api:setconf(GC#gconf{trace=false}, Groups),\n            error_logger:info_msg(\"Trace disabled~n\", [])\n    end;\ndisable_trace(Time) ->\n    {ok, GC, Groups} = yaws_server:getconf(),\n    case GC#gconf.trace of\n        false ->\n            error_logger:info_msg(\"Trace already disabled~n\", []);\n        {true, CurType} ->\n            yaws_api:setconf(GC#gconf{trace=false}, Groups),\n            timer:apply_after(Time, ?MODULE, enable_trace, [CurType, infinity]),\n            error_logger:info_msg(\"Trace disabled for ~w milliseconds~n\",[Time])\n    end.\n\n\nenable_trace(Type, infinity) when Type =:= http; Type =:= traffic ->\n    {ok, GC, Groups} = yaws_server:getconf(),\n    case GC#gconf.trace of\n        {true, Type} ->\n            error_logger:info_msg(\"~p trace already enabled~n\", [Type]);\n        {true, _} ->\n            yaws_api:setconf(GC#gconf{trace={true, Type}}, Groups),\n            error_logger:info_msg(\"~p trace enabled~n\", [Type]);\n        false ->\n            yaws_api:setconf(GC#gconf{trace={true, Type}}, Groups),\n            error_logger:info_msg(\"~p trace enabled~n\", [Type])\n    end;\nenable_trace(Type, Time) when Type =:= http; Type =:= traffic ->\n    {ok, GC, Groups} = yaws_server:getconf(),\n    case GC#gconf.trace of\n        {true, Type} ->\n            error_logger:info_msg(\"~p trace already enabled~n\", [Type]);\n        {true, CurType} ->\n            yaws_api:setconf(GC#gconf{trace={true, Type}}, Groups),\n            timer:apply_after(Time, ?MODULE, enable_trace, [CurType, infinity]),\n            error_logger:info_msg(\"~p trace enabled for ~w milliseconds~n\",\n                                  [Type, Time]);\n        false ->\n            yaws_api:setconf(GC#gconf{trace={true, Type}}, Groups),\n            timer:apply_after(Time, ?MODULE, disable_trace, [infinity]),\n            error_logger:info_msg(\"~p trace enabled for ~w milliseconds~n\",\n                                  [Type, Time])\n    end.\n\nget_tracedir() ->\n    gen_server:call(?MODULE, get_tracedir, infinity).\n\nset_filter(Filter) ->\n    Expr = filter2expr(Filter),\n    Fun  = lists:flatten([\n                          \"fun(Ip, Req, Headers) ->\\n\",\n                          \"  try\\n\"\n                          \"    (\", Expr, \")\\n\",\n                          \"  catch\\n\",\n                          \"    _:_ -> false\\n\"\n                          \"  end\\n\"\n                          \" end.\"\n                         ]),\n    Tokens = case erl_scan:string(Fun) of\n                 {ok, Toks, _} -> Toks;\n                 _             -> throw({error, {invalid_filter, scan_failed}})\n             end,\n    ExprList = case erl_parse:parse_exprs(Tokens) of\n                   {ok, E} -> E;\n                   _       -> throw({error, {invalid_filter, parse_failed}})\n               end,\n    try erl_eval:exprs(ExprList, []) of\n        {value, Result, _} ->\n            gen_server:call(?MODULE, {set_filter, Result}, infinity)\n    catch\n        _:_ ->\n            throw({error, {invalid_filter, eval_failed}})\n    end.\n\nunset_filter() ->\n    gen_server:call(?MODULE, unset_filter, infinity).\n\nget_filter() ->\n    gen_server:call(?MODULE, get_filter, infinity).\n\n\n%%====================================================================\n%% gen_server callbacks\n%%====================================================================\ninit([]) ->\n    process_flag(trap_exit, true),\n    {ok, #state{}}.\n\n\n%% ----\nhandle_call({set_filter, Fun}, _From, State) ->\n    {reply, ok, State#state{filter=Fun}};\nhandle_call(unset_filter, _From, State) ->\n    {reply, ok, State#state{filter=undefined}};\nhandle_call(get_filter, _From, State) ->\n    {reply, State#state.filter, State};\nhandle_call(get_tracedir, _From, State) ->\n    {reply, State#state.tracedir, State};\nhandle_call(_Request, _From, State) ->\n    Reply = ok,\n    {reply, Reply, State}.\n\n\n%% ----\nhandle_cast({setup, #gconf{trace=false}=GC}, State) ->\n    Bool = ?gc_has_tty_trace(GC),\n    {noreply, State#state{type=false, tty_trace=Bool, tracedir=undefined}};\nhandle_cast({setup, GC}, State) ->\n    {true, Type} = GC#gconf.trace,\n    Bool = ?gc_has_tty_trace(GC),\n    Dir  = filename:join(GC#gconf.logdir, subdir()),\n    case file:make_dir(Dir) of\n        ok ->\n            error_logger:info_msg(\"Trace directory ~p created~n\", [Dir]),\n            {noreply, State#state{type=Type, tty_trace=Bool, tracedir=Dir}};\n        {error, Reason} ->\n            error_logger:error_msg(\"Failed to create trace directory ~p: ~p~n\",\n                                   [Dir, file:format_error(Reason)]),\n            {noreply, State#state{type=false, tty_trace=Bool}}\n    end;\n\nhandle_cast({tty_trace, Bool}, State) ->\n    {noreply, State#state{tty_trace=Bool}};\n\nhandle_cast({open, _}, #state{type=false}=State) ->\n    {noreply, State};\nhandle_cast({open, Owner}, State) ->\n    Type = State#state.type,\n    case dict:find(Owner, State#state.traces) of\n        {ok, {Type, _Fd}} ->\n            {noreply, State};\n        {ok, {_OtherType, OldFd}} ->\n            file:close(OldFd),\n            case open_trace(State#state.tracedir, filename(Type, Owner)) of\n                {ok, NewFd} ->\n                    erlang:monitor(process, Owner),\n                    D = dict:store(Owner, {Type,NewFd}, State#state.traces),\n                    {noreply, State#state{traces=D}};\n                error ->\n                    D = dict:erase(Owner, State#state.traces),\n                    {noreply, State#state{traces=D}}\n            end;\n        error ->\n            case open_trace(State#state.tracedir, filename(Type, Owner)) of\n                {ok, NewFd} ->\n                    erlang:monitor(process, Owner),\n                    D = dict:store(Owner, {Type,NewFd}, State#state.traces),\n                    {noreply, State#state{traces=D}};\n                error ->\n                    {noreply, State}\n            end\n    end;\n\nhandle_cast({write, _, _, _}, #state{type=false}=State) ->\n    {noreply, State};\nhandle_cast({write, Pid, Tag, Data}, State) ->\n    TTYTrace = State#state.tty_trace,\n    case dict:find(Pid, State#state.traces) of\n        {ok, {_Type, Fd}} -> write_trace(Pid, Fd, Tag, Data, TTYTrace);\n        error             -> ok\n    end,\n    {noreply, State};\n\nhandle_cast(_Msg, State) ->\n    {noreply, State}.\n\n\n%% ----\nhandle_info({'DOWN', _MRef, _Type, Pid, _Info}, State) ->\n    case dict:find(Pid, State#state.traces) of\n        {ok, {_, Fd}} ->\n            file:close(Fd),\n            D = dict:erase(Pid, State#state.traces),\n            {noreply, State#state{traces=D}};\n        error ->\n            {noreply, State}\n    end;\n\nhandle_info(_Info, State) ->\n    {noreply, State}.\n\n\n%% ----\nterminate(_Reason, State) ->\n    [file:close(Fd) || {_Owner, Fd} <- dict:to_list(State#state.traces)],\n    ok.\n\n\n%% ----\ncode_change(_OldVsn, State, _Extra) ->\n    {ok, State}.\n\n\n%%====================================================================\n%% Internal functions\n%%====================================================================\nsubdir() ->\n    {{Y,M,D}, {HH,MM,SS}} = erlang:localtime(),\n    lists:flatten(\n      io_lib:format(\"trace_~4.10.0B~2.10.0B~2.10.0B_~2.10.0B~2.10.0B~2.10.0B\",\n                    [Y,M,D,HH,MM,SS])\n     ).\n\nfilename(Type, Owner) ->\n    lists:concat([\"trace.\", pid_to_list(Owner), \".\", Type]).\n\n\nopen_trace(Dir, File) ->\n    case file:open(filename:join([Dir, File]), [write, raw]) of\n        {ok, Fd} ->\n            Date = iso_8601_fmt(),\n            Str  = [\"=====\\n===== TRACE STARTED \", Date, \"\\n=====\\n\\n\"],\n            file:write(Fd, Str),\n            {ok, Fd};\n        {error, Reason} ->\n            error_logger:error_msg(\"Failed to open trace ~p: ~p~n\",\n                                   [File, file:format_error(Reason)]),\n            error\n    end.\n\n\nwrite_trace(Pid, Fd, from_server, Data, TTYTrace) ->\n    Date = iso_8601_fmt(),\n    Head = io_lib:format(\"~n[~s] ===== SRV -> CLI =====~n\", [Date]),\n    write_trace(Pid, Fd, [Head, Data], TTYTrace);\nwrite_trace(Pid, Fd, from_client, Data, TTYTrace) ->\n    Date = iso_8601_fmt(),\n    Head = io_lib:format(\"~n[~s] ===== CLI -> SRV =====~n\", [Date]),\n    write_trace(Pid, Fd, [Head, Data], TTYTrace);\nwrite_trace(Pid, Fd, _, Data, TTYTrace) ->\n    Date = iso_8601_fmt(),\n    Head = io_lib:format(\"~n[~s]~n\", [Date]),\n    write_trace(Pid, Fd, [Head, Data], TTYTrace).\n\n\nwrite_trace(Pid, Fd, Str, true) ->\n    file:write(Fd, Str),\n    io:format(\"Worker: ~p ~s~n\", [Pid, flatten([Str])]);\nwrite_trace(_Pid, Fd, Str, false) ->\n    file:write(Fd, Str).\n\n\niso_8601_fmt() ->\n    {{Year,Month,Day},{Hour,Min,Sec}} = erlang:localtime(),\n    {_,_,MicroSec} = yaws:get_time_tuple(),\n    io_lib:format(\"~4.10.0B-~2.10.0B-~2.10.0B ~2.10.0B:~2.10.0B:~2.10.0B.~w\",\n                  [Year, Month, Day, Hour, Min, Sec, (MicroSec div 1000)]).\n\n\nflatten(L) ->\n    binary_to_list(iolist_to_binary(L)).\n\n\n\n%%====================================================================\nfilter2expr({'and', Tests}) ->\n    L = [lists:flatten([\"(\", filter2expr(T), \")\"]) || T <- Tests],\n    string:join(L, \" andalso \");\nfilter2expr({'or', Tests}) ->\n    L = [lists:flatten([\"(\", filter2expr(T), \")\"]) || T <- Tests],\n    string:join(L, \" orelse \");\nfilter2expr({'and', T1, T2}) ->\n    Expr1 = lists:flatten([\"(\", filter2expr(T1), \")\"]),\n    Expr2 = lists:flatten([\"(\", filter2expr(T2), \")\"]),\n    lists:flatten([Expr1, \" andalso \", Expr2]);\nfilter2expr({'or', T1, T2}) ->\n    Expr1 = lists:flatten([\"(\", filter2expr(T1), \")\"]),\n    Expr2 = lists:flatten([\"(\", filter2expr(T2), \")\"]),\n    lists:flatten([Expr1, \" orelse \", Expr2]);\nfilter2expr({'not', T}) ->\n    Expr = lists:flatten([\"(\", filter2expr(T), \")\"]),\n    lists:flatten([\"not \", Expr]);\n\n\nfilter2expr({'equal', Elmt, Value}) ->\n    Expr = [element_to_string(Elmt), \" =:= \", value_to_string(Value)],\n    lists:flatten(Expr);\nfilter2expr({'startby', Elmt, Prefix}) ->\n    Expr = [\"lists:prefix(\", value_to_string(Prefix), \",\",\n            element_to_string(Elmt), \")\"],\n    lists:flatten(Expr);\nfilter2expr({'endwith', Elmt, Suffix}) ->\n    Expr = [\"lists:suffix(\", value_to_string(Suffix), \",\",\n            element_to_string(Elmt), \")\"],\n    lists:flatten(Expr);\nfilter2expr({'contain', Elmt, Part}) ->\n    Expr = [\"string:str(\", element_to_string(Elmt), \",\",\n            value_to_string(Part), \") =/= 0\"],\n    lists:flatten(Expr);\nfilter2expr({'match', Elmt, Re}) ->\n    Expr = [\"re:run(\", element_to_string(Elmt), \",\", value_to_string(Re),\n            \", [{capture,none}]) =:= match\"],\n    lists:flatten(Expr);\n\nfilter2expr(F) ->\n    throw({error, {unknown_filter, F}}).\n\n\n\nvalue_to_string(Value) ->\n    io_lib:format(\"~p\", [Value]).\n\nelement_to_string(ip) ->\n    \"Ip\";\nelement_to_string({header, H}) ->\n    lists:concat([\"yaws_trace:get_header_value(\", H, \", Headers)\"]);\nelement_to_string({request, R}) when R =:= method;  R =:= path; R =:= version ->\n    lists:concat([\"yaws_trace:get_request_value(\", R, \", Req)\"]);\nelement_to_string(E) ->\n    throw({error, {unknown_element, E}}).\n\n\nget_header_value(connection, Headers) ->\n    case yaws_api:headers_connection(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(accept, Headers) ->\n    case yaws_api:headers_accept(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(host, Headers) ->\n    case yaws_api:headers_host(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(if_modified_since, Headers) ->\n    case yaws_api:headers_if_modified_since(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(if_match, Headers) ->\n    case yaws_api:headers_if_match(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(if_none_match, Headers) ->\n    case yaws_api:headers_if_none_match(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(if_range, Headers) ->\n    case yaws_api:headers_if_range(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(if_unmodified_since, Headers) ->\n    case yaws_api:headers_if_unmodified_since(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(range, Headers) ->\n    case yaws_api:headers_range(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(referer, Headers) ->\n    case yaws_api:headers_referer(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(user_agent, Headers) ->\n    case yaws_api:headers_user_agent(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(accept_ranges, Headers) ->\n    case yaws_api:headers_accept_ranges(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(cookie, Headers) ->\n    lists:flatten(yaws_api:headers_cookie(Headers));\nget_header_value(keep_alive, Headers) ->\n    case yaws_api:headers_keep_alive(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(location, Headers) ->\n    case yaws_api:headers_location(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(content_length, Headers) ->\n    case yaws_api:headers_content_length(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(content_type, Headers) ->\n    case yaws_api:headers_content_length(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(content_encoding, Headers) ->\n    case yaws_api:headers_content_encoding(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(authorization, Headers) ->\n    case yaws_api:headers_authorization(Headers) of\n        undefined                    -> [];\n        {undefined, undefined, Orig} -> lists:flatten([\"::\",Orig]);\n        {User, Pass, Orig}           -> lists:flatten([User,\":\",Pass,\":\",Orig])\n    end;\nget_header_value(transfer_encoding, Headers) ->\n    case yaws_api:headers_transfer_encoding(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(x_forwarded_for, Headers) ->\n    case yaws_api:headers_x_forwarded_for(Headers) of\n        undefined -> [];\n        Value     -> Value\n    end;\nget_header_value(Name, Headers) when is_atom(Name) ->\n    Others = yaws_api:headers_other(Headers),\n    case lists:keyfind(Name, 3, Others) of\n        {http_header,_,_,_,Value} ->\n            Value;\n        false ->\n            get_header_value(atom_to_list(Name), Headers)\n    end;\nget_header_value(Name, Headers) ->\n    Others = yaws_api:headers_other(Headers),\n    case lists:keyfind(Name, 3, Others) of\n        {http_header,_,_,_,Value} ->\n            Value;\n        false ->\n            []\n    end.\n\n\nget_request_value(method, Req) ->\n    yaws_api:http_request_method(Req);\nget_request_value(path, Req) ->\n    case yaws_api:http_request_path(Req) of\n        {abs_path, Path} -> Path;\n        _                -> []\n    end;\nget_request_value(version, Req) ->\n    case yaws_api:http_request_version(Req) of\n        {Maj, Min} -> lists:concat([\"HTTP/\", Maj, \".\", Min]);\n        _          -> \"HTTP/0.9\"\n    end.\n"
  },
  {
    "path": "contrib/yaws/src/yaws_vdir.erl",
    "content": "-module(yaws_vdir).\n\n-export([arg_rewrite/1]).\n\n-include(\"../include/yaws_api.hrl\").\n\n-export([join/2]).\n\njoin(List, Sep) ->\n    lists:foldl(fun(A, \"\") -> A; (A, Acc) -> Acc ++ Sep ++ A\n                end, \"\", List).\n\n\narg_rewrite(ARG) ->\n    Req = ARG#arg.req,\n    %%io:fwrite(\"----->rewrite_mod for request: ~p\\n\",[ARG#arg.req]),\n\n    case Req#http_request.path of\n        {abs_path, RawPath} ->\n            case (catch yaws_api:url_decode_q_split(RawPath)) of\n                {'EXIT', _} ->\n                    %%broken request - ignore let yaws_server handle it.\n                    ARG2 = ARG;\n                {\"\", _QueryPart} ->\n                    ARG2 = ARG;\n                {\"/\", _QueryPart} ->\n                    %%don't allow vdir to be specified for root -\n                    %% it doesn't make sense\n                    ARG2 = ARG;\n                {DecPath, _QueryPart} ->\n                    SC = get(sc),\n\n                    %%vdirpath/3 will return the longest(ie most specific)\n                    %% 'virtual directory' match for our request\n                    %%It retrieves the vdir definitions from #arg.opaque\n                    case yaws_server:vdirpath(SC, ARG, DecPath) of\n                        {\"\",_MainDocRoot} ->\n                            %%no virtual dir corresponding to this\n                            %% http_request.path\n\n                            ARG2 = ARG;\n                        {Virt,DocRoot} ->\n\n                            %%the virtual-path of our request matches a\n                            %% vdir specification\n                            %% - rewrite ARG accordingly.\n\n                            ARG2 = ARG#arg{docroot = DocRoot,\n                                           docroot_mount = Virt}\n                    end\n            end;\n        _Else ->\n            ARG2 = ARG\n    end,\n\n    ARG2.\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_websockets.erl",
    "content": "%%%----------------------------------------------------------------------\n%%% File    : yaws_websockets.erl\n%%% Author  : Davide Marques <nesrait@gmail.com>\n%%% Purpose : support for WebSockets\n%%% Created : 18 Dec 2009 by Davide Marques <nesrait@gmail.com>\n%%% Modified: extensively revamped in 2011 by J.D. Bothma <jbothma@gmail.com>\n%%%----------------------------------------------------------------------\n\n-module(yaws_websockets).\n-author('nesrait@gmail.com').\n-author('jbothma@gmail.com').\n-behaviour(gen_server).\n\n-include(\"../include/yaws.hrl\").\n-include(\"../include/yaws_api.hrl\").\n\n-include_lib(\"kernel/include/file.hrl\").\n\n\n%% RFC 6455 section 7.4.1: Defined Status Codes\n-define(WS_STATUS_NORMAL,           1000).\n-define(WS_STATUS_PROTO_ERROR,      1002).\n-define(WS_STATUS_ABNORMAL_CLOSURE, 1006).\n-define(WS_STATUS_INVALID_PAYLOAD,  1007).\n-define(WS_STATUS_MSG_TOO_BIG,      1009).\n-define(WS_STATUS_INTERNAL_ERROR,   1011).\n\n\n-record(state, {arg,\n                opts,\n                wsstate,\n                cbinfo,\n                cbstate,\n                cbtype,\n                timeout=infinity,\n                wait_pong_frame=false,\n                close_frame_received=false,\n                close_timer,\n                reason=normal}).\n\n-record(cbinfo, {module,\n                 init_fun,\n                 terminate_fun,\n                 open_fun,\n                 msg_fun_1,\n                 msg_fun_2,\n                 info_fun}).\n\n-export([start/3, send/2, close/2]).\n-export([init/1, handle_call/3, handle_cast/2, handle_info/2,\n         terminate/2, code_change/3]).\n\n%%%----------------------------------------------------------------------\n%% API\n%%%----------------------------------------------------------------------\nstart(Arg, CallbackMod, Opts) ->\n    PrepdOpts = preprocess_opts(Opts),\n\n    %% Do some checks on request headers before starting the WebSocket process:\n    %%   o \"Connection:\" header field should contain \"websocket\"\n    %%   o \"Upgrade:\" header field should contain \"upgrade\"\n    %%   o \"Origin:\" header field should be valid\n    case check_connection(query_header(\"connection\", Arg#arg.headers)) of\n        ok ->\n            ok;\n        error ->\n            error_logger:error_msg(\"Invalid connection header\", []),\n            deliver_xxx(Arg#arg.clisock, 400),\n            exit(normal)\n    end,\n\n    case check_upgrade(query_header(\"upgrade\", Arg#arg.headers)) of\n        ok ->\n            ok;\n        error ->\n            error_logger:error_msg(\"Invalid upgrade header\", []),\n            deliver_xxx(Arg#arg.clisock, 400),\n            exit(normal)\n    end,\n    OriginOpt = get_opts(origin, PrepdOpts),\n    Origin    = get_origin_header(Arg#arg.headers),\n    case check_origin(Origin, OriginOpt) of\n        ok ->\n            ok;\n        error ->\n            error_logger:error_msg(\"Expected origin ~p but found ~p\",\n                                   [OriginOpt, Origin]),\n            deliver_xxx(Arg#arg.clisock, 403),\n            exit(normal)\n    end,\n\n    %% Start websocket process\n    OwnerPid =\n        case gen_server:start(?MODULE, [Arg, CallbackMod, PrepdOpts], []) of\n            {ok, Pid} ->\n                Pid;\n            {error, Reason1} ->\n                error_logger:error_msg(\"Failed to start websocket process: ~p\",\n                                       [Reason1]),\n                deliver_xxx(Arg#arg.clisock, 500),\n                exit(normal)\n        end,\n\n    %% Change the controller process of the client socket\n    CliSock = Arg#arg.clisock,\n    Res = case yaws_api:get_sslsocket(CliSock) of\n              {ok, SslSocket} ->\n                  ssl:setopts(SslSocket, [{packet, raw}, {active, once}]),\n                  ssl:controlling_process(SslSocket, OwnerPid);\n              undefined ->\n                  inet:setopts(CliSock, [{packet, raw}, {active, once}]),\n                  gen_tcp:controlling_process(CliSock, OwnerPid)\n          end,\n    case Res of\n        ok ->\n            gen_server:cast(OwnerPid, ok);\n        {error, Reason2} ->\n            error_logger:error_msg(\"Failed to start websocket process: ~p\",\n                                   [Reason2]),\n            gen_server:cast(OwnerPid, {stop, Reason2}),\n            deliver_xxx(Arg#arg.clisock, 500)\n    end,\n    exit(normal).\n\n\nsend(#ws_state{}=WSState, {Type, Data}) ->\n    do_send(WSState, {Type, Data});\nsend(#ws_state{}=WSState, #ws_frame{}=Frame) ->\n    do_send(WSState, Frame);\nsend(Pid, {Type, Data}) ->\n    gen_server:cast(Pid, {send, {Type, Data}});\nsend(Pid, #ws_frame{}=Frame) ->\n    gen_server:cast(Pid, {send, Frame}).\n\nclose(#ws_state{}=WSState, Reason) ->\n    do_close(WSState, Reason);\nclose(Pid, Reason) ->\n    gen_server:cast(Pid, {close, Reason}).\n\n%%%----------------------------------------------------------------------\n%% gen_server functions\n%%%----------------------------------------------------------------------\ninit([Arg, CbMod, Opts]) ->\n    case get_opts(auto_fragment_message, Opts) of\n        true ->\n            AutoFrag = {true, get_opts(auto_fragment_threshold, Opts)},\n            put(auto_fragment_message, AutoFrag);\n        false ->\n            put(auto_fragment_message, false)\n    end,\n    {CbType, FrameState, InitState} =\n        case get_opts(callback, Opts) of\n            {basic,    St} -> {basic, {none, <<>>}, St};\n            {advanced, St} -> {advanced, undefined, St};\n            _              -> {basic, {none, <<>>}, []}\n        end,\n    KeepAliveTout = case get_opts(keepalive, Opts) of\n                        true  -> get_opts(keepalive_timeout, Opts);\n                        false -> infinity\n                    end,\n    case get_callback_info(CbMod) of\n        {ok, #cbinfo{init_fun=undefined}=CbInfo} ->\n            {ok, #state{arg     = Arg,\n                        opts    = Opts,\n                        cbinfo  = CbInfo,\n                        cbtype  = CbType,\n                        cbstate = {FrameState,InitState},\n                        timeout = KeepAliveTout}, KeepAliveTout};\n        {ok, #cbinfo{init_fun=InitFun}=CbInfo} ->\n            case CbMod:InitFun([Arg, InitState]) of\n                {ok, InitState1} ->\n                    {ok, #state{arg     = Arg,\n                                opts    = Opts,\n                                cbinfo  = CbInfo,\n                                cbtype  = CbType,\n                                cbstate = {FrameState,InitState1},\n                                timeout = KeepAliveTout}, KeepAliveTout};\n                {ok, InitState1, Timeout} ->\n                    {ok, #state{arg     = Arg,\n                                opts    = Opts,\n                                cbinfo  = CbInfo,\n                                cbtype  = CbType,\n                                cbstate = {FrameState,InitState1},\n                                timeout = KeepAliveTout}, Timeout};\n                {error, Reason} ->\n                    {stop, Reason}\n            end;\n        {error, Reason} ->\n            {stop, Reason}\n    end.\n\n\n%% ----\n%% Skip all sync requests\nhandle_call(_Req, _From, State) ->\n    {reply, ok, State, State#state.timeout}.\n\n\n%% ----\n%% Send the handshake response\nhandle_cast(ok, #state{arg=Arg, cbinfo=CbInfo}=State) ->\n    CliSock = Arg#arg.clisock,\n\n    WSVersion = ws_version(Arg#arg.headers),\n    WSKey     = get_nonce_header(Arg#arg.headers),\n    if\n        element(1, WSVersion) == unsupported_version ->\n            error_logger:error_msg(\"Unsupported protocol version\", []),\n            deliver_xxx(CliSock, 400, [\"Sec-WebSocket-Version: 13, 8\"]),\n            {stop, normal, State#state{reason={error, WSVersion}}};\n\n        undefined == WSKey ->\n            error_logger:error_msg(\"No key found\", []),\n            deliver_xxx(CliSock, 400),\n            {stop, normal, State#state{reason={error, key_notfound}}};\n\n        true ->\n            Protocol = get_protocol_header(Arg#arg.headers),\n            WSState  = #ws_state{sock      = CliSock,\n                                 vsn       = WSVersion,\n                                 frag_type = none},\n            handshake(CliSock, WSKey, Protocol),\n\n            case CbInfo#cbinfo.open_fun of\n                undefined ->\n                    {noreply, State#state{wsstate=WSState},\n                     State#state.timeout};\n                OpenFun ->\n                    CbMod = CbInfo#cbinfo.module,\n                    {FrameState, CbState} = State#state.cbstate,\n                    case CbMod:OpenFun(WSState, CbState) of\n                        {ok, CbState1} ->\n                            {noreply,\n                             State#state{wsstate=WSState,\n                                         cbstate={FrameState,CbState1}},\n                             State#state.timeout};\n                        {error, Reason} ->\n                            do_close(WSState, Reason),\n                            {stop, normal, State#state{reason={error, Reason}}}\n                    end\n            end\n    end;\n\n%% ----\n%% Stop the current websocket process. Used when an error occured during process\n%% startup.\nhandle_cast({stop, Reason}, State) ->\n    {stop, normal, State#state{reason={error,Reason}}};\n\n%% Send a frame to the client\nhandle_cast({send, {Type, Data}}, #state{wsstate=WSState}=State) ->\n    do_send(WSState, {Type, Data}),\n    {noreply, State, State#state.timeout};\nhandle_cast({send, #ws_frame{}=Frame}, #state{wsstate=WSState}=State) ->\n    do_send(WSState, Frame),\n    {noreply, State, State#state.timeout};\nhandle_cast({close, Reason}, #state{wsstate=WSState}=State) ->\n    do_close(WSState, Reason),\n    {noreply, State, State#state.timeout};\n\n%% Skip all other async messages\nhandle_cast(_Msg, State) ->\n    {noreply, State, State#state.timeout}.\n\n\n%% ----\n%% Receive a TCP packet from the client\nhandle_info({tcp, Socket, FirstPacket},\n            #state{wsstate=#ws_state{sock=Socket}}=State) ->\n    handle_frames(FirstPacket, State);\nhandle_info({ssl, Socket, FirstPacket},\n            #state{wsstate=#ws_state{sock={ssl, Socket}}}=State) ->\n    handle_frames(FirstPacket, State);\n\n%% Abnormal socket closure: only if no Close frame was received or sent.\nhandle_info({tcp_closed, Socket},\n            #state{wsstate=#ws_state{sock=Socket}}=State) ->\n    if\n        State#state.close_frame_received == true orelse\n        State#state.close_timer /= undefined ->\n            {stop, normal, State};\n        true ->\n            handle_abnormal_closure(State)\n    end;\nhandle_info({ssl_closed, Socket},\n            #state{wsstate=#ws_state{sock={ssl, Socket}}}=State) ->\n    if\n        State#state.close_frame_received == true orelse\n        State#state.close_timer /= undefined ->\n            {stop, normal, State};\n        true ->\n            handle_abnormal_closure(State)\n    end;\n\n\n%% Ignore timeout if a Close frame was sent or received\nhandle_info(timeout, #state{close_frame_received=true}=State) ->\n    {noreply, State};\nhandle_info(timeout, #state{close_timer=TRef}=State) when TRef /= undefined ->\n    {noreply, State};\n\n%% Keepalive timeout: send a ping frame and wait for any reply\nhandle_info(timeout, #state{wait_pong_frame=false}=State) ->\n    GracePeriod = get_opts(keepalive_grace_period, State#state.opts),\n    do_send(State#state.wsstate, {ping, <<>>}),\n    {noreply, State#state{wait_pong_frame=true}, GracePeriod};\n\n%% Grace period timeout\nhandle_info(timeout, #state{wait_pong_frame=true}=State) ->\n    State1 = State#state{wait_pong_frame=false},\n    case get_opts(drop_on_timeout, State1#state.opts) of\n        true  -> handle_abnormal_closure(State1);\n        false -> handle_callback_info(timeout, State1)\n    end;\n\n%% Close timeout: just shutdown the gen_server\nhandle_info(close, State) ->\n    case State#state.close_timer of\n        undefined -> ok;\n        TRef      -> erlang:cancel_timer(TRef)\n    end,\n    {stop, normal, State};\n\n%% All other messages\nhandle_info(Info, State) ->\n    handle_callback_info(Info, State).\n\n\n%% ----\nterminate(_, #state{arg=Arg, cbinfo=CbInfo}=State) ->\n    case CbInfo#cbinfo.terminate_fun of\n        undefined ->\n            ok;\n        TerminateFun ->\n            CbMod = CbInfo#cbinfo.module,\n            {_, CbState} = State#state.cbstate,\n            CbMod:TerminateFun(State#state.reason, CbState)\n    end,\n    CliSock = case State#state.wsstate of\n                  undefined -> Arg#arg.clisock;\n                  WSState   -> WSState#ws_state.sock\n              end,\n    if\n        CliSock /= undefined ->\n            case yaws_api:get_sslsocket(CliSock) of\n                {ok, SslSocket} -> ssl:close(SslSocket);\n                undefined       -> gen_tcp:close(CliSock)\n            end;\n        true ->\n            ok\n    end.\n\n\n%% ----\ncode_change(_OldVsn, Data, _Extra) ->\n    {ok, Data}.\n\n\n%%%----------------------------------------------------------------------\n%% internal functions\n%%%----------------------------------------------------------------------\nget_callback_info(Mod) ->\n    case code:ensure_loaded(Mod) of\n        {module, _} ->\n            InitFun = case erlang:function_exported(Mod, init, 1) of\n                          true  -> init;\n                          false -> undefined\n                      end,\n            TerminateFun = case erlang:function_exported(Mod, terminate, 2) of\n                               true  -> terminate;\n                               false -> undefined\n                           end,\n            OpenFun = case erlang:function_exported(Mod, handle_open, 2) of\n                          true  -> handle_open;\n                          false -> undefined\n                      end,\n            MsgFun1 = case erlang:function_exported(Mod, handle_message, 1) of\n                          true  -> handle_message;\n                          false -> undefined\n                      end,\n            MsgFun2 = case erlang:function_exported(Mod, handle_message, 2) of\n                          true  -> handle_message;\n                          false -> undefined\n                      end,\n            InfoFun = case erlang:function_exported(Mod, handle_info, 2) of\n                          true  -> handle_info;\n                          false -> undefined\n                      end,\n            {ok, #cbinfo{module        = Mod,\n                         init_fun      = InitFun,\n                         terminate_fun = TerminateFun,\n                         open_fun      = OpenFun,\n                         msg_fun_1     = MsgFun1,\n                         msg_fun_2     = MsgFun2,\n                         info_fun      = InfoFun}};\n        {error, Reason} ->\n            error_logger:error_msg(\"Cannot load callback module '~p': ~p\",\n                                   [Mod, Reason]),\n            {error, Reason}\n    end.\n\n\npreprocess_opts(GivenOpts) ->\n    Fun = fun({Key, Default}, Opts) ->\n                  case lists:keyfind(Key, 1, Opts) of\n                      false -> [{Key, Default}|Opts];\n                      _     -> Opts\n                  end\n          end,\n    Defaults = [\n                {origin,                  any},\n                {callback,                basic},\n                {max_frame_size,          16 * 1024 * 1024}, %% 16 MB\n                {max_message_size,        16 * 1024 * 1024}, %% 16 MB\n                {close_if_unmasked,       false},\n                {auto_fragment_message,   false},\n                {auto_fragment_threshold, 1024 * 1024},      %%  1 MB\n                {close_timeout,           5000},             %%  5 secs\n                {keepalive,               false},\n                {keepalive_timeout,       30000},            %% 30 secs\n                {keepalive_grace_period,  2000},             %%  2 secs\n                {drop_on_timeout,         false}\n               ],\n    lists:foldl(Fun, GivenOpts, Defaults).\n\n\nget_opts(Key, Opts) ->\n    {Key, Value} = lists:keyfind(Key, 1, Opts),\n    Value.\n\ncheck_origin(_Origin, any)       -> ok;\ncheck_origin(Actual,  Actual )   -> ok;\ncheck_origin(_Actual, _Expected) -> error.\n\ncheck_connection(undefined) ->\n    error;\ncheck_connection(Connection) ->\n    Vals = yaws:split_sep(string:to_lower(Connection), $,),\n    case lists:member(\"upgrade\", Vals) of\n        true  -> ok;\n        false -> error\n    end.\n\ncheck_upgrade(undefined) ->\n    error;\ncheck_upgrade(Upgrade) ->\n    Vals = yaws:split_sep(string:to_lower(Upgrade), $,),\n    case lists:member(\"websocket\", Vals) of\n        true  -> ok;\n        false -> error\n    end.\n\n\n\nhandshake(CliSock, Key, _Protocol) ->\n    AcceptHash = hash_nonce(Key),\n    Handshake  = [\"HTTP/1.1 101 Switching Protocols\\r\\n\",\n                  \"Upgrade: websocket\\r\\n\",\n                  \"Connection: Upgrade\\r\\n\",\n                  \"Sec-WebSocket-Accept: \", AcceptHash , \"\\r\\n\",\n                  \"\\r\\n\"],\n    case yaws_api:get_sslsocket(CliSock) of\n        {ok, SslSocket} -> ssl:send(SslSocket, Handshake);\n        undefined       -> gen_tcp:send(CliSock, Handshake)\n    end.\n\n\ndo_send(#ws_state{sock=undefined}, _) ->\n    ok;\ndo_send(WSState, Messages) when is_list(Messages) ->\n    [do_send(WSState, Msg) || Msg <- Messages],\n    ok;\ndo_send(WSState, {Type, Data}) ->\n    do_send(WSState, #ws_frame{opcode=Type, payload=Data});\ndo_send(#ws_state{sock=Socket}, #ws_frame{}=Frame) ->\n    case yaws_api:get_sslsocket(Socket) of\n        {ok, SslSocket} -> [ssl:send(SslSocket, F)  || F <- frames(Frame)];\n        undefined       -> [gen_tcp:send(Socket, F) || F <- frames(Frame)]\n    end.\n\n\n%% Add this clause to be compatible with previous versions.\ndo_close(WSState, normal) ->\n    do_close(WSState, ?WS_STATUS_NORMAL);\n%% Special case for status=1006, do not send this close frame\ndo_close(_WSState, ?WS_STATUS_ABNORMAL_CLOSURE) ->\n    ok;\ndo_close(_WSState, {?WS_STATUS_ABNORMAL_CLOSURE, _}) ->\n    ok;\ndo_close(WSState, Status) when is_integer(Status) ->\n    do_send(WSState, {close, <<Status:16/big>>});\ndo_close(WSState, {Status,Reason}) when is_integer(Status), is_binary(Reason) ->\n    do_send(WSState, {close, <<Status:16/big, Reason/binary>>});\ndo_close(WSState, _) ->\n    Status = ?WS_STATUS_INTERNAL_ERROR,\n    do_send(WSState, {close, <<Status:16/big>>}).\n\n\ndeliver_xxx(CliSock, Code) ->\n    deliver_xxx(CliSock, Code, []).\n\ndeliver_xxx(CliSock, Code, Hdrs) ->\n    Reply = [\"HTTP/1.1 \",integer_to_list(Code), $\\s,\n             yaws_api:code_to_phrase(Code), \"\\r\\n\",\n             \"Connection: close\\r\\n\",\n             lists:flatmap(fun(X) -> X ++ \"\\r\\n\" end, Hdrs),\n             \"\\r\\n\"],\n    case yaws_api:get_sslsocket(CliSock) of\n        {ok, SslSocket} -> ssl:send(SslSocket, Reply);\n        undefined       -> gen_tcp:send(CliSock, Reply)\n    end.\n\n\n%% ----\nhandle_frames(FirstPacket, #state{wsstate=WSState, opts=Opts}=State) ->\n    FrameInfos = unframe_active_once(WSState, FirstPacket, Opts),\n    Result = case State#state.cbtype of\n                 basic    -> basic_messages(FrameInfos, State);\n                 advanced -> advanced_messages(FrameInfos, State)\n             end,\n    case Result of\n        {ok, State1, Timeout} ->\n            Last = lists:last(FrameInfos),\n            {noreply, State1#state{wsstate=Last#ws_frame_info.ws_state,\n                                  wait_pong_frame=false},\n             Timeout};\n        {stop, State1} ->\n            case State1#state.close_frame_received of\n                true  ->\n                    {stop, normal, State1};\n                false ->\n                    Tout = get_opts(close_timeout, State1#state.opts),\n                    TRef = erlang:send_after(Tout, self(), close),\n                    {noreply, State1#state{close_timer=TRef}}\n            end\n    end.\n\n\nhandle_callback_info(Info, #state{cbinfo=CbInfo}=State) ->\n    case CbInfo#cbinfo.info_fun of\n        undefined ->\n            {noreply, State, State#state.timeout};\n        InfoFun ->\n            {_, CbState} = State#state.cbstate,\n            CbMod = CbInfo#cbinfo.module,\n            Res   = CbMod:InfoFun(Info, CbState),\n            case handle_callback_result(Res, State) of\n                {ok, State1, Timeout} -> {noreply, State1, Timeout};\n                {stop, State1}        -> {stop, normal, State1}\n            end\n    end.\n\nhandle_abnormal_closure(#state{wsstate=WSState}=State) ->\n    %% The only way we should get here is due to an abnormal close. Section\n    %% 7.1.5 of RFC 6455 specifies 1006 as the connection close code for\n    %% abnormal closure. It's also described in section 7.4.1.\n    CloseStatus    = ?WS_STATUS_ABNORMAL_CLOSURE,\n    ClosePayload   = <<CloseStatus:16/big>>,\n    CloseWSState   = WSState#ws_state{sock=undefined,frag_type=none},\n    CloseFrameInfo = #ws_frame_info{fin         = 1,\n                                    rsv         = 0,\n                                    opcode      = close,\n                                    masked      = 0,\n                                    masking_key = <<>>,\n                                    length      = 2,\n                                    payload     = ClosePayload,\n                                    data        = ClosePayload,\n                                    ws_state    = CloseWSState},\n    Result = case State#state.cbtype of\n                 basic ->\n                     basic_messages([CloseFrameInfo],\n                                    State#state{wsstate=CloseWSState});\n                 advanced ->\n                     advanced_messages([CloseFrameInfo],\n                                       State#state{wsstate=CloseWSState})\n             end,\n    case Result of\n        {ok, State1, _} -> {stop, normal, State1};\n        {stop, State1}  -> {stop, normal, State1}\n    end.\n\n\n%% ----\nbasic_messages(FrameInfos, State) ->\n    basic_messages(FrameInfos, State, State#state.timeout).\n\nbasic_messages([], State, Tout) ->\n    {ok, State, Tout};\nbasic_messages([FrameInfo|Rest],\n               #state{cbinfo=CbInfo, cbstate={FrameState,CbState}}=State,\n               Tout) ->\n    case handle_message(FrameInfo, FrameState, State#state.opts) of\n        {continue, FrameState1} ->\n            basic_messages(Rest, State#state{cbstate={FrameState1,CbState}},\n                           Tout);\n\n        {message, Message} when State#state.close_timer /= undefined ->\n            case Message of\n                {close, _, _} ->\n                    %% The server is waiting for the Close frame. We can stop it\n                    %% now.\n                    erlang:cancel_timer(State#state.close_timer),\n                    {stop, State#state{close_frame_received=true}};\n                _ ->\n                    %% The server is waiting for the Close frame, all other\n                    %% messages are ignored.\n                    basic_messages(Rest, State, Tout)\n            end;\n\n        {message, {pong, Data}} ->\n            do_send(State#state.wsstate, {pong, Data}),\n            basic_messages(Rest, State, Tout);\n\n        {message, Message} ->\n            CbMod = CbInfo#cbinfo.module,\n            Res = case CbInfo#cbinfo.msg_fun_2 of\n                      undefined ->\n                          MsgFun1 = CbInfo#cbinfo.msg_fun_1,\n                          catch CbMod:MsgFun1(Message);\n                      MsgFun2 ->\n                          catch CbMod:MsgFun2(Message, CbState)\n                  end,\n            IsCloseFrame = case Message of\n                               {close, _, _} -> true;\n                               _             -> false\n                           end,\n            State1 = State#state{cbstate={{none, <<>>}, CbState},\n                                 close_frame_received=IsCloseFrame},\n            case handle_callback_result(Res, State1) of\n                {ok, State2, _} when IsCloseFrame == true ->\n                    %% The callback module must return a Close frame. If not, we\n                    %% force the server to close the connection\n                    {stop, State2};\n                {ok, State2, Tout1} ->\n                    basic_messages(Rest, State2, Tout1);\n                {stop, State2} ->\n                    {stop, State2}\n            end;\n\n        {error, Reason} ->\n            do_close(State#state.wsstate, Reason),\n            {stop, State#state{close_frame_received=true, reason=Reason}}\n    end.\n\n\n%% unfragmented text message\nhandle_message(#ws_frame_info{fin=1, opcode=text, data=Data}, {none, <<>>},\n               _Opts) ->\n    case unicode:characters_to_binary(Data, utf8, utf8) of\n        Data -> {message, {text, Data}};\n        _    -> {error, {?WS_STATUS_INVALID_PAYLOAD, <<\"invalid utf-8\">>}}\n    end;\n\n%% start of a fragmented text message\nhandle_message(#ws_frame_info{fin=0, opcode=text, data=Data}, {none, <<>>},\n               _Opts) ->\n    case unicode:characters_to_binary(Data, utf8, utf8) of\n        Data ->\n            {continue, {text, [Data], <<>>}};\n        {incomplete,Dec,Rest} ->\n            {continue, {text, [Dec],  Rest}};\n        _ ->\n            {error, {?WS_STATUS_INVALID_PAYLOAD, <<\"invalid utf-8\">>}}\n    end;\n\n%% non-final continuation of a fragmented text message\nhandle_message(#ws_frame_info{fin=0, data=Data, opcode=continuation},\n               {text, Dec0, Rest0}, Opts) ->\n    MaxLen = get_opts(max_message_size, Opts),\n    Len    = iolist_size(Dec0) + byte_size(Rest0) + byte_size(Data),\n    if\n        Len > MaxLen ->\n            {error, {?WS_STATUS_MSG_TOO_BIG, <<\"Message too big\">>}};\n        true ->\n            Data1 = <<Rest0/binary, Data/binary>>,\n            case unicode:characters_to_binary(Data1, utf8, utf8) of\n                Data1 ->\n                    {continue, {text, [Data1|Dec0], <<>>}};\n                {incomplete, Dec1, Rest1} ->\n                    {continue, {text, [Dec1|Dec0],  Rest1}};\n                _ ->\n                    {error, {?WS_STATUS_INVALID_PAYLOAD, <<\"invalid utf-8\">>}}\n            end\n    end;\n\n%% end of text fragmented message\nhandle_message(#ws_frame_info{fin=1, opcode=continuation, data=Data},\n               {text, Dec, Rest}, Opts) ->\n    MaxLen = get_opts(max_message_size, Opts),\n    Len    = iolist_size(Dec) + byte_size(Rest) + byte_size(Data),\n    if\n        Len > MaxLen ->\n            {error, {?WS_STATUS_MSG_TOO_BIG, <<\"Message too big\">>}};\n        true ->\n            Data1 = <<Rest/binary, Data/binary>>,\n            case unicode:characters_to_binary(Data1, utf8, utf8) of\n                Data1 ->\n                    Msg = list_to_binary(lists:reverse([Data1|Dec])),\n                    {message, {text, Msg}};\n                _ ->\n                    {error, {?WS_STATUS_INVALID_PAYLOAD, <<\"invalid utf-8\">>}}\n            end\n    end;\n\n%% unfragmented binary message\nhandle_message(#ws_frame_info{fin=1, opcode=binary, data=Data}, {none, <<>>},\n               _Opts) ->\n    {message, {binary, Data}};\n\n%% start of a fragmented binary message\nhandle_message(#ws_frame_info{fin=0, opcode=binary, data=Data}, {none, <<>>},\n               _Opts) ->\n    {continue, {binary, Data}};\n\n%% non-final continuation of a fragmented binary message\nhandle_message(#ws_frame_info{fin=0, data=Data, opcode=continuation},\n               {binary, FragAcc}, Opts) ->\n    MaxLen = get_opts(max_message_size, Opts),\n    Len    = byte_size(FragAcc) + byte_size(Data),\n    if\n        Len > MaxLen ->\n            {error, {?WS_STATUS_MSG_TOO_BIG, <<\"Message too big\">>}};\n        true ->\n            {continue, {binary, <<FragAcc/binary,Data/binary>>}}\n    end;\n\n%% end of binary fragmented message\nhandle_message(#ws_frame_info{fin=1, opcode=continuation, data=Data},\n               {binary, FragAcc}, Opts) ->\n    MaxLen = get_opts(max_message_size, Opts),\n    Len    = byte_size(FragAcc) + byte_size(Data),\n    if\n        Len > MaxLen ->\n            {error, {?WS_STATUS_MSG_TOO_BIG, <<\"Message too big\">>}};\n        true ->\n            Unfragged = <<FragAcc/binary, Data/binary>>,\n            {message, {binary, Unfragged}}\n    end;\n\nhandle_message(#ws_frame_info{opcode=ping, data=Data}, _Acc, _Opts) ->\n    {message, {pong, Data}};\n\nhandle_message(#ws_frame_info{opcode=pong}, Acc, _Opts) ->\n    %% A response to an unsolicited pong frame is not expected.\n    %% http://tools.ietf.org/html/rfc6455#section-5.5.3\n    {continue, Acc};\n\n%% According to RFC 6455 section 5.4, control messages like close MAY be\n%% injected in the middle of a fragmented message, which is why we pass FragType\n%% and FragAcc along below. Whether any clients actually do this in practice, I\n%% don't know.\nhandle_message(#ws_frame_info{opcode=close, length=Len,\n                              data=Data, ws_state=WSState},\n               _Acc, _Opts) ->\n    case Len of\n        0 ->\n            {message, {close, ?WS_STATUS_NORMAL, <<>>}};\n        1 ->\n            {error, {?WS_STATUS_PROTO_ERROR, <<\"protocol error\">>}};\n        _ ->\n            <<Status:16/big, Msg/binary>> = Data,\n            case unicode:characters_to_binary(Msg, utf8, utf8) of\n                Msg ->\n                    {message, {close, check_close_code(Status, WSState), Msg}};\n                _ ->\n                    {error, {?WS_STATUS_INVALID_PAYLOAD, <<\"invalid utf-8\">>}}\n            end\n    end;\n\nhandle_message(#ws_frame_info{}, _Acc, _Opts) ->\n    {error, {?WS_STATUS_PROTO_ERROR, <<\"protocol error\">>}};\n\nhandle_message({fail_connection, Status, Msg}, _State, _Tout) ->\n    {error, {Status, Msg}}.\n\n\n\nadvanced_messages(FrameInfos, State) ->\n    advanced_messages(FrameInfos, State, State#state.timeout).\n\nadvanced_messages([], State, Tout) ->\n    {ok, State, Tout};\nadvanced_messages([FrameInfo|Rest], #state{close_timer=TRef}=State,\n                  Tout) when TRef /= undefined ->\n    case FrameInfo of\n        #ws_frame_info{opcode=close} ->\n            %% The server is waiting for the Close frame. We can stop it now.\n            erlang:cancel_timer(TRef),\n            {stop, State#state{close_frame_received=true}};\n        _ ->\n            %% The server is waiting for the Close frame, all other messages are\n            %% ignored.\n            advanced_messages(Rest, State, Tout)\n    end;\nadvanced_messages([FrameInfo|Rest],\n                  #state{cbinfo=CbInfo, cbstate={undefined,CbState}}=State,\n                  _Tout) ->\n    CbMod   = CbInfo#cbinfo.module,\n    MsgFun2 = CbInfo#cbinfo.msg_fun_2,\n    Res     = (catch CbMod:MsgFun2(FrameInfo, CbState)),\n    IsCloseFrame = case FrameInfo of\n                       #ws_frame_info{opcode=close} -> true;\n                       _                            -> false\n                   end,\n    State1 = State#state{close_frame_received=IsCloseFrame},\n    case handle_callback_result(Res, State1) of\n        {ok, State2, _} when IsCloseFrame == true ->\n            %% The callback module must return a Close frame. If not, we force\n            %% the server to close the connection\n            {stop, State2};\n        {ok, State2, Tout1} ->\n            advanced_messages(Rest, State2, Tout1);\n        {stop, State2} ->\n            {stop, State2}\n    end.\n\n\n%% The checks for close status codes here are based on RFC 6455 and on the\n%% autobahn testsuite (http://autobahn.ws/testsuite).\ncheck_close_code(Code, WSState) ->\n    if\n        Code >= 3000 andalso Code =< 4999 ->\n            Code;\n        Code < 1000 ->\n            ?WS_STATUS_PROTO_ERROR;\n        Code == 1006 andalso WSState#ws_state.sock == undefined ->\n            Code;\n        Code >= 1004 andalso Code =< 1006 ->\n            ?WS_STATUS_PROTO_ERROR;\n        Code > 1011 ->\n            ?WS_STATUS_PROTO_ERROR;\n        true ->\n            Code\n    end.\n\n\nhandle_callback_result({reply, Messages}, State) ->\n    do_send(State#state.wsstate, Messages),\n    {ok, State, State#state.timeout};\nhandle_callback_result({reply, Messages, CbState1}, State) ->\n    do_send(State#state.wsstate, Messages),\n    {FrameState, _} = State#state.cbstate,\n    {ok, State#state{cbstate={FrameState, CbState1}}, State#state.timeout};\nhandle_callback_result({reply, Messages, CbState1, Timeout}, State) ->\n    do_send(State#state.wsstate, Messages),\n    {FrameState, _} = State#state.cbstate,\n    {ok, State#state{cbstate={FrameState, CbState1}}, Timeout};\n\nhandle_callback_result(noreply, State) ->\n    {ok, State, State#state.timeout};\nhandle_callback_result({noreply, CbState1}, State) ->\n    {FrameState, _} = State#state.cbstate,\n    {ok, State#state{cbstate={FrameState, CbState1}}, State#state.timeout};\nhandle_callback_result({noreply, CbState1, Timeout}, State) ->\n    {FrameState, _} = State#state.cbstate,\n    {ok, State#state{cbstate={FrameState, CbState1}}, Timeout};\n\nhandle_callback_result({close, Reason}, State) ->\n    do_close(State#state.wsstate, Reason),\n    case is_fatal_error(Reason) of\n        true  -> {stop, State#state{close_frame_received=true, reason=Reason}};\n        false -> {stop, State#state{reason=Reason}}\n    end;\nhandle_callback_result({close, Reason, CbState1}, State) ->\n    do_close(State#state.wsstate, Reason),\n    {FrameState, _} = State#state.cbstate,\n    State1 = State#state{cbstate={FrameState, CbState1}},\n    case is_fatal_error(Reason) of\n        true  -> {stop, State1#state{close_frame_received=true, reason=Reason}};\n        false -> {stop, State1#state{reason=Reason}}\n    end;\nhandle_callback_result({close, Reason, Messages, CbState1}, State) ->\n    do_send(State#state.wsstate, Messages),\n    do_close(State#state.wsstate, Reason),\n    {FrameState, _} = State#state.cbstate,\n    State1 = State#state{cbstate={FrameState, CbState1}},\n    case is_fatal_error(Reason) of\n        true  -> {stop, State1#state{close_frame_received=true, reason=Reason}};\n        false -> {stop, State1#state{reason=Reason}}\n    end;\n\nhandle_callback_result({'EXIT', Reason}, State) ->\n    do_close(State#state.wsstate, ?WS_STATUS_INTERNAL_ERROR),\n    {stop, State#state{close_frame_received=true, reason={error,Reason}}}.\n\n\nis_fatal_error(?WS_STATUS_PROTO_ERROR)           -> true;\nis_fatal_error({?WS_STATUS_PROTO_ERROR, _})      -> true;\nis_fatal_error(?WS_STATUS_ABNORMAL_CLOSURE)      -> true;\nis_fatal_error({?WS_STATUS_ABNORMAL_CLOSURE, _}) -> true;\nis_fatal_error(?WS_STATUS_INVALID_PAYLOAD)       -> true;\nis_fatal_error({?WS_STATUS_INVALID_PAYLOAD, _})  -> true;\nis_fatal_error(?WS_STATUS_MSG_TOO_BIG)           -> true;\nis_fatal_error({?WS_STATUS_MSG_TOO_BIG, _})      -> true;\nis_fatal_error(?WS_STATUS_INTERNAL_ERROR)        -> true;\nis_fatal_error({?WS_STATUS_INTERNAL_ERROR, _})   -> true;\nis_fatal_error(_)                                -> false.\n\n\n%% ----\nws_version(Headers) ->\n    VersionVal = query_header(\"sec-websocket-version\", Headers),\n    case VersionVal of\n        \"8\"  -> 8;\n        \"13\" -> 13;\n        V    -> {unsupported_version, V}\n    end.\n\nbuffer(Socket, Len, Buffered) ->\n    case Buffered of\n        <<_Expected:Len/binary>> -> % exactly enough\n            Buffered;\n        <<_Expected:Len/binary,_Extra/binary>> -> % more than expected\n            Buffered;\n        _ ->\n            %% not enough\n            More = do_recv(Socket, Len - byte_size(Buffered), []),\n            <<Buffered/binary, More/binary>>\n    end.\n\ndo_recv(_, 0, Acc) ->\n    list_to_binary(lists:reverse(Acc));\ndo_recv(Socket, Sz, Acc) ->\n    %% TODO: add configurable timeout to receive data\n    Res = case yaws_api:get_sslsocket(Socket) of\n              {ok, SslSocket} -> ssl:recv(SslSocket, Sz,  5000);\n              undefined       -> gen_tcp:recv(Socket, Sz, 5000)\n          end,\n    case Res of\n        {ok, Bin}       -> do_recv(Socket, Sz - byte_size(Bin), [Bin|Acc]);\n        {error, Reason} -> throw({tcp_error, Reason})\n    end.\n\n\nchecks(Unframed) ->\n    check_reserved_bits(Unframed).\n\ncheck_control_frame(Len, Opcode, Fin) ->\n    if\n        (Len > 125) and (Opcode > 7) ->\n            %% http://tools.ietf.org/html/rfc6455#section-5.5\n            {fail_connection, ?WS_STATUS_PROTO_ERROR,\n             <<\"control frame > 125 bytes\">>};\n        (Fin == 0) and (Opcode > 7) ->\n            {fail_connection, ?WS_STATUS_PROTO_ERROR,\n             <<\"control frame may not be fragmented\">>};\n        true ->\n            ok\n    end.\n\n%% no extensions are supported yet\n%% http://tools.ietf.org/html/rfc6455#section-5.2\ncheck_reserved_bits(Unframed = #ws_frame_info{rsv=0}) ->\n    check_reserved_opcode(Unframed);\ncheck_reserved_bits(#ws_frame_info{rsv=RSV}) ->\n    {fail_connection, ?WS_STATUS_PROTO_ERROR, <<\"rsv bits were \", (RSV+48),\n                              \" but should be unset\">>}.\n\n\ncheck_reserved_opcode(#ws_frame_info{opcode = undefined}) ->\n    {fail_connection, ?WS_STATUS_PROTO_ERROR, <<\"Reserved opcode\">>};\ncheck_reserved_opcode(_) ->\n    ok.\n\n\nws_frame_info(#ws_state{sock=Socket},\n              <<Fin:1, Rsv:3, Opcode:4, Masked:1, Len1:7, Rest/binary>>,\n              Opts) ->\n    case check_control_frame(Len1, Opcode, Fin) of\n        ok ->\n            case ws_frame_info_secondary(Socket, Masked, Len1, Rest, Opts) of\n                {ws_frame_info_secondary,Length,MaskingKey,Payload,Excess} ->\n                    FrameInfo = #ws_frame_info{fin=Fin,\n                                               rsv=Rsv,\n                                               opcode=opcode_to_atom(Opcode),\n                                               masked=Masked,\n                                               masking_key=MaskingKey,\n                                               length=Length,\n                                               payload=Payload},\n                    {FrameInfo, Excess};\n                Else ->\n                    Else\n            end;\n        Other ->\n            Other\n    end;\n\nws_frame_info(WSState = #ws_state{sock=Socket}, FirstPacket, Opts) ->\n    ws_frame_info(WSState, buffer(Socket, 2, FirstPacket), Opts).\n\n\nws_frame_info_secondary(Socket, Masked, Len1, Rest, Opts) ->\n    MaxLen = get_opts(max_frame_size, Opts),\n    CloseIfUnmasked = get_opts(close_if_unmasked, Opts),\n    MaskLength = case Masked of\n                     0 -> 0;\n                     1 -> 4\n                 end,\n    case Len1 of\n        126 ->\n            <<Len:16, MaskingKey:MaskLength/binary, Rest2/binary>> =\n                buffer(Socket, MaskLength + 2 , Rest);\n        127 ->\n            <<Len:64, MaskingKey:MaskLength/binary, Rest2/binary>> =\n                buffer(Socket, MaskLength + 8 , Rest);\n        Len ->\n            <<MaskingKey:MaskLength/binary, Rest2/binary>> =\n                buffer(Socket, MaskLength, Rest)\n    end,\n    if\n        CloseIfUnmasked == true andalso MaskingKey == <<>> ->\n            error_logger:error_msg(\"Unmasked frame forbidden\", []),\n            {fail_connection, ?WS_STATUS_PROTO_ERROR,\n             <<\"Unmasked frame forbidden\">>};\n        Len > MaxLen ->\n            error_logger:error_msg(\n              \"Payload length ~p longer than max allowed of ~p\",\n              [Len, MaxLen]\n             ),\n            {fail_connection, ?WS_STATUS_MSG_TOO_BIG, <<\"Frame too big\">>};\n        true ->\n            <<Payload:Len/binary, Excess/binary>> = buffer(Socket, Len, Rest2),\n            {ws_frame_info_secondary, Len, MaskingKey, Payload, Excess}\n    end.\n\nunframe_active_once(WSState, FirstPacket, Opts) ->\n    Frames = unframe(WSState, FirstPacket, Opts),\n    websocket_setopts(WSState, [{active, once}]),\n    Frames.\n\n\n%% Returns all the WebSocket frames fully or partially contained in FirstPacket,\n%% reading exactly as many more bytes from Socket as are needed to finish\n%% unframing the last frame partially included in FirstPacket, if needed.\n%%\n%% The length of this list and depth of this recursion is limited by the size of\n%% your socket receive buffer.\n%%\n%% -> { #ws_state, [#ws_frame_info,...,#ws_frame_info] }\nunframe(_WSState, <<>>, _Opts) ->\n    [];\nunframe(WSState, FirstPacket, Opts) ->\n    case unframe_one(WSState, FirstPacket, Opts) of\n        {FrameInfo = #ws_frame_info{ws_state = NewWSState}, RestBin} ->\n            %% Every new recursion uses the #ws_state from the calling\n            %% recursion.\n            [FrameInfo | unframe(NewWSState, RestBin, Opts)];\n        Fail ->\n            [Fail]\n    end.\n\n%% -> {#ws_frame_info, RestBin} | {fail_connection, Reason}\nunframe_one(WSState, FirstPacket, Opts) ->\n    case catch ws_frame_info(WSState, FirstPacket, Opts) of\n        {FrameInfo = #ws_frame_info{}, RestBin} ->\n            Unmasked = mask(FrameInfo#ws_frame_info.masking_key,\n                            FrameInfo#ws_frame_info.payload),\n            case frag_state_machine(WSState, FrameInfo) of\n                #ws_state{}=NewWSState ->\n                    Unframed = FrameInfo#ws_frame_info{data     = Unmasked,\n                                                       ws_state = NewWSState},\n                    case checks(Unframed) of\n                        ok   -> {Unframed, RestBin};\n                        Else -> Else\n                    end;\n                Else ->\n                    Else\n            end;\n        {tcp_error, Reason} ->\n            %% FIXME: close the connection ?\n            error_logger:error_msg(\"Abnormal Closure: ~p\", [Reason]),\n            CloseStatus    = ?WS_STATUS_ABNORMAL_CLOSURE,\n            ClosePayload   = <<CloseStatus:16/big>>,\n            CloseWSState   = WSState#ws_state{sock=undefined,frag_type=none},\n            {#ws_frame_info{fin         = 1,\n                            rsv         = 0,\n                            opcode      = close,\n                            masked      = 0,\n                            masking_key = 0,\n                            length      = 2,\n                            payload     = ClosePayload,\n                            data        = ClosePayload,\n                            ws_state    = CloseWSState}, <<>>};\n        Else ->\n            Else\n    end.\n\nwebsocket_setopts(#ws_state{sock=Socket}, Opts) ->\n    case yaws_api:get_sslsocket(Socket) of\n        {ok, SslSocket} -> ssl:setopts(SslSocket, Opts);\n        undefined       -> inet:setopts(Socket, Opts)\n    end.\n\nis_control_op(Op) ->\n    atom_to_opcode(Op) > 7.\n\n%% Unfragmented message\nfrag_state_machine(#ws_state{frag_type=none} = State,\n                   #ws_frame_info{fin=1}) ->\n    State;\n\n%% Beginning of fragmented text message\nfrag_state_machine(#ws_state{frag_type=none} = State,\n                   #ws_frame_info{fin=0, opcode=text}) ->\n    State#ws_state{frag_type = text};\n\n%% Beginning of fragmented binary message\nfrag_state_machine(#ws_state{frag_type=none} = State,\n                   #ws_frame_info{fin=0, opcode=binary}) ->\n    State#ws_state{frag_type = binary};\n\n%% Expecting text continuation\nfrag_state_machine(#ws_state{frag_type=text} = State,\n                   #ws_frame_info{fin=0, opcode=continuation}) ->\n    State;\n\n%% Expecting binary continuation\nfrag_state_machine(#ws_state{frag_type=binary}=State,\n                   #ws_frame_info{fin=0, opcode=continuation}) ->\n    State;\n\n%% End of fragmented text message\nfrag_state_machine(#ws_state{frag_type=text} = State,\n                   #ws_frame_info{fin=1, opcode=continuation}) ->\n    State#ws_state{frag_type = none};\n\n%% End of fragmented binary message\nfrag_state_machine(#ws_state{frag_type=binary} = State,\n                   #ws_frame_info{fin=1, opcode=continuation}) ->\n    State#ws_state{frag_type = none};\n\n\nfrag_state_machine(State, #ws_frame_info{opcode = Op}) ->\n    IsControl = is_control_op(Op),\n    if\n        IsControl == true ->\n            %% Control message never changes fragmentation state\n            State;\n        true ->\n            %% Everything else is wrong\n            {fail_connection, ?WS_STATUS_PROTO_ERROR,\n             <<\"fragmentation rules violated\">>}\n    end.\n\n\nopcode_to_atom(16#0) -> continuation;\nopcode_to_atom(16#1) -> text;\nopcode_to_atom(16#2) -> binary;\nopcode_to_atom(16#8) -> close;\nopcode_to_atom(16#9) -> ping;\nopcode_to_atom(16#A) -> pong;\nopcode_to_atom(_)    -> undefined.\n\natom_to_opcode(continuation) -> 16#0;\natom_to_opcode(text)         -> 16#1;\natom_to_opcode(binary)       -> 16#2;\natom_to_opcode(close)        -> 16#8;\natom_to_opcode(ping)         -> 16#9;\natom_to_opcode(pong)         -> 16#A.\n\n\nframes(#ws_frame{fin=true, opcode=C, payload=Payload}=Frame) when C == text;\n                                                                  C == binary ->\n    %% try auto-fragmentation only on unfragmented data frames\n    case get(auto_fragment) of\n        {true, Sz} when byte_size(Payload) > Sz ->\n            <<Chunk:Sz/binary, Rest/binary>> = Payload,\n            [frame(#ws_frame{fin=false, opcode=C, payload=Chunk}) |\n             fragment_frame(\n               #ws_frame{fin=false, opcode=continuation, payload=Rest}, Sz\n              )];\n        _ ->\n            [frame(Frame)]\n    end;\nframes(Frame) ->\n    [frame(Frame)].\n\n\n\nframe(#ws_frame{}=Frame) ->\n    Fin     = case Frame#ws_frame.fin of\n                  true  -> 1;\n                  false -> 0\n              end,\n    Rsv     = Frame#ws_frame.rsv,\n    OpCode  = atom_to_opcode(Frame#ws_frame.opcode),\n    Payload = Frame#ws_frame.payload,\n    Length  = byte_size(Payload),\n    if\n        Length < 126 ->\n            <<Fin:1, Rsv:3, OpCode:4, 0:1, Length:7, Payload/binary>>;\n        Length < 65536 ->\n            <<Fin:1, Rsv:3, OpCode:4, 0:1, 126:7, Length:16, Payload/binary>>;\n        true ->\n            <<Fin:1, Rsv:3, OpCode:4, 0:1, 127:7, Length:64, Payload/binary>>\n    end.\n\n\nfragment_frame(#ws_frame{payload=Payload}=Frame, Sz) ->\n    case Payload of\n        <<_:Sz/binary>> ->\n            [frame(Frame#ws_frame{fin=true})];\n        <<Chunk:Sz/binary, Rest/binary>> ->\n            [frame(Frame#ws_frame{payload=Chunk})|\n             fragment_frame(Frame#ws_frame{payload=Rest}, Sz)];\n        _ ->\n            [frame(Frame#ws_frame{fin=true})]\n    end.\n\n\nmask(MaskBin, Data) ->\n    list_to_binary(rmask(MaskBin, Data)).\n\n%% unmask == mask. It's XOR of the four-byte masking key.\nrmask(_,<<>>) ->\n    [<<>>];\nrmask(<<>>, Data) ->\n    [Data];\nrmask(MaskBin = <<Mask:4/integer-unit:8>>,\n      <<Data:4/integer-unit:8, Rest/binary>>) ->\n    Masked = Mask bxor Data,\n    MaskedRest = rmask(MaskBin, Rest),\n    [<<Masked:4/integer-unit:8>> | MaskedRest ];\nrmask(<<Mask:3/integer-unit:8, _Rest/binary>>, <<Data:3/integer-unit:8>>) ->\n    Masked = Mask bxor Data,\n    [<<Masked:3/integer-unit:8>>];\nrmask(<<Mask:2/integer-unit:8, _Rest/binary>>, <<Data:2/integer-unit:8>>) ->\n    Masked = Mask bxor Data,\n    [<<Masked:2/integer-unit:8>>];\nrmask(<<Mask:1/integer-unit:8, _Rest/binary>>, <<Data:1/integer-unit:8>>) ->\n    Masked = Mask bxor Data,\n    [<<Masked:1/integer-unit:8>>].\n\n\n%% Internal functions\nget_origin_header(Headers) ->\n    case query_header(\"origin\", Headers) of\n        undefined -> query_header(\"sec-websocket-origin\", Headers);\n        Origin    -> Origin\n    end.\n\nget_protocol_header(Headers) ->\n    query_header(\"sec-websocket-protocol\", Headers, \"unknown\").\n\nget_nonce_header(Headers) ->\n    query_header(\"sec-websocket-key\", Headers).\n\nquery_header(HeaderName, Headers) ->\n    query_header(HeaderName, Headers, undefined).\n\nquery_header(Header, Headers, Default) ->\n    yaws_api:get_header(Headers, Header, Default).\n\nhash_nonce(Nonce) ->\n    Salted = Nonce ++ \"258EAFA5-E914-47DA-95CA-C5AB0DC85B11\",\n    HashBin = crypto:hash(sha, Salted),\n    base64:encode_to_string(HashBin).\n"
  },
  {
    "path": "contrib/yaws/src/yaws_xmlrpc.erl",
    "content": "%% -*- coding: latin-1 -*-\n%% Copyright (C) 2003 Joakim Greben <jocke@gleipnir.com>.\n%% All rights reserved.\n%%\n%% Copyright (C) 2006 Gaspar Chilingarov <nm@web.am>\n%%                      Gurgen Tumanyan <barbarian@armkb.com>\n%% All rights reserved.\n%%\n%%\n%% Redistribution and use in source and binary forms, with or without\n%% modification, are permitted provided that the following conditions\n%% are met:\n%%\n%% 1. Redistributions of source code must retain the above copyright\n%%    notice, this list of conditions and the following disclaimer.\n%% 2. Redistributions in binary form must reproduce the above\n%%    copyright notice, this list of conditions and the following\n%%    disclaimer in the documentation and/or other materials provided\n%%    with the distribution.\n%%\n%% THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS\n%% OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED\n%% WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n%% ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY\n%% DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n%% DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE\n%% GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS\n%% INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,\n%% WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING\n%% NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS\n%% SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n\n-module(yaws_xmlrpc).\n%%-author('jocke@gleipnir.com').\n-author(\"Gaspar Chilingarov <nm@web.am>, Gurgen Tumanyan <barbarian@armkb.com>\").\n-export([handler/2]).\n-export([handler_session/2, handler_session/3]).\n\n%%-define(debug, 1).\n%%-include(\"../../yaws/src/yaws_debug.hrl\").\n\n-include(\"../include/yaws_api.hrl\").\n\n\n%%% ######################################################################\n%%% public interface\n%%%\n\n%%%\n%%% use XMLRPC handler which can automagically start sessions if we need\n%%%\nhandler_session(Args, Handler) ->\n    handler_session(Args, Handler, 'SID').\n\n%%%\n%%% allow overriding session Cookie name\n%%%\nhandler_session(Args, Handler, SID_NAME) when is_atom(SID_NAME) ->\n    handler_session(Args, Handler, atom_to_list(SID_NAME));\n\nhandler_session(Args, Handler, SID_NAME) ->\n    handler(Args, Handler, {session, SID_NAME}).\n\n%%%\n%%% xmlrpc:handler compatible call\n%%% no session support will be available\nhandler(Args, Handler) ->\n    handler(Args, Handler, simple).\n\n\n%%% ######################################################################\n%%% private functions\n%%%\n\n%%% we should be called from yaws page or module\nhandler(Args, Handler, Type) when is_record(Args, arg) ->\n    case parse_request(Args) of\n        ok ->\n            handle_payload(Args, Handler, Type);\n        {status, StatusCode} ->        %% cannot parse request\n            send(Args, StatusCode)\n    end.\n\n-define(ERROR_LOG(Reason),\n        error_logger:error_report({?MODULE, ?LINE, Reason})).\n\n%%%\n%%% check that request come in reasonable protocol version and reasonable method\n%%%\nparse_request(Args) -> %% {{{\n    case {(Args#arg.req)#http_request.method,\n          (Args#arg.req)#http_request.version} of\n        {'POST', {1,0}} ->\n            %%        ?Debug(\"HTTP Version 1.0~n\", []),\n            ok;\n        {'POST', {1,1}} ->\n            %%        ?Debug(\"HTTP Version 1.1~n\", []),\n            ok;\n        {'POST', _HTTPVersion} -> {status, 505};\n        {_Method, {1,1}} -> {status, 501};\n        _ -> {status, 400}\n    end. %% }}}\n\nhandle_payload(Args, Handler, Type) ->\n    Payload = binary_to_list(Args#arg.clidata),\n    %%    ?Debug(\"xmlrpc encoded call ~p ~n\", [Payload]),\n    case xmlrpc_decode:payload(Payload) of\n        {ok, DecodedPayload} ->\n            %%        ?Debug(\"xmlrpc decoded call ~p ~n\", [DecodedPayload]),\n            eval_payload(Args, Handler, DecodedPayload, Type);\n        {error, Reason} ->\n\t    ErrMsg = xmlrpc_http:handle_xmlprc_error(Payload, Reason),\n\t    send(Args, 400, ErrMsg, [])\n    end.\n\n%%%%%%\n%%% call handler/3 and provide session support\neval_payload(Args, {M, F}, Payload, {session, CookieName}) ->\n    {SessionValue, Cookie} =\n        case yaws_api:find_cookie_val(CookieName,\n                                      (Args#arg.headers)#headers.cookie) of\n            [] ->      % have no session started, just call handler\n                {undefined, undefined};\n            Cookie2 -> %% get old session data\n                case yaws_api:cookieval_to_opaque(Cookie2) of\n                    {ok, OP} ->\n                        yaws_api:cookieval_to_opaque(Cookie2),\n                        {OP, Cookie2};\n                    {error, _ErrMsg} -> %% cannot get corresponding session\n                        {undefined, undefined}\n                end\n        end,\n\n    case catch M:F(Args#arg.state, Payload, SessionValue) of\n        {'EXIT', Reason} ->\n            ?ERROR_LOG({M, F, {'EXIT', Reason}}),\n            send(Args, 500);\n        {error, Reason} ->\n            ?ERROR_LOG({M, F, Reason}),\n            send(Args, 500);\n        {false, ResponsePayload} ->\n            %% do not have updates in session data\n            encode_send(Args, 200, ResponsePayload, []);\n        {true, _NewTimeout, NewSessionValue, ResponsePayload} ->\n            %% be compatible with xmlrpc module\n            CO = case NewSessionValue of\n                     undefined when Cookie == undefined -> []; %% nothing to do\n                     undefined -> %% rpc handler requested session delete\n                         yaws_api:delete_cookie_session(Cookie), [];\n                     %% XXX: may be return set-cookie with empty val?\n                     _ ->  %% any other value will stored in session\n                         case SessionValue of\n                             undefined ->\n                                 %% got session data and should start\n                                 %% new session now\n                                 Cookie1 = yaws_api:new_cookie_session(\n                                             NewSessionValue),\n                                 yaws_api:setcookie(\n                                   CookieName, Cookie1, \"/\");\n                             %% return set_cookie header\n                             _ ->\n                                 yaws_api:replace_cookie_session(\n                                   Cookie, NewSessionValue),\n                                 [] %% nothing to add to yaws data\n                         end\n                 end,\n            encode_send(Args, 200, ResponsePayload, CO)\n    end;\n\n%%%\n%%% call handler/2 without session support\n%%%\neval_payload(Args, {M, F}, Payload, simple) ->\n    case catch M:F(Args#arg.state, Payload) of\n        {'EXIT', Reason} ->\n            ?ERROR_LOG({M, F, {'EXIT', Reason}}),\n            send(Args, 500);\n        {error, Reason} ->\n            ?ERROR_LOG({M, F, Reason}),\n            send(Args, 500);\n        {false, ResponsePayload} ->\n            encode_send(Args, 200, ResponsePayload, []);\n        {true, _NewTimeout, _NewState, ResponsePayload} ->\n            encode_send(Args, 200, ResponsePayload, [])\n    end.\n\n\nencode_send(Args, StatusCode, Payload, AddOn) ->\n    %%    ?Debug(\"xmlrpc decoded response ~p ~n\", [Payload]),\n    case xmlrpc_encode:payload(Payload) of\n        {ok, EncodedPayload} ->\n            %%   ?Debug(\"xmlrpc encoded response ~p ~n\", [EncodedPayload]),\n            send(Args, StatusCode, EncodedPayload, AddOn);\n        {error, Reason} ->\n            ?ERROR_LOG({xmlrpc_encode, payload, Payload, Reason}),\n            send(Args, 500)\n    end.\n\nsend(Args, StatusCode) -> send(Args, StatusCode, \"\", []).\n\nsend(Args, StatusCode, Payload, AddOnData) when not is_list(AddOnData) ->\n    send(Args, StatusCode, Payload, [AddOnData]);\n\n%%%\n%%% generate valid yaws response\nsend(_Args, StatusCode, Payload, AddOnData) ->\n    A = [\n         {status, StatusCode},\n         {content, \"text/xml\", Payload},\n         {header, {content_length, lists:flatlength(Payload) }}\n        ] ++ AddOnData,\n    A.\n\n"
  },
  {
    "path": "contrib/yaws/src/yaws_zlib.erl",
    "content": "%%% Utility functions for zlib.\n-module(yaws_zlib).\n-author('carsten@codimi.de').\n\n\n-include(\"../include/yaws.hrl\").\n\n\n-export([gzipInit/1, gzipInit/2, gzipEnd/1, gzipDeflate/4, gzip/1, gzip/2]).\n\n\ngzipInit(Z) ->\n    gzipInit(Z, #deflate{}).\n\ngzipInit(Z, DOpts) ->\n    ok = zlib:deflateInit(Z, DOpts#deflate.compression_level, deflated,\n                          DOpts#deflate.window_size, DOpts#deflate.mem_level,\n                          DOpts#deflate.strategy),\n    undefined.\n\n\ngzipEnd(Z) ->\n    zlib:deflateEnd(Z).\n\n\ngzipDeflate(Z, undefined, Bin, Flush) ->\n    Crc32 = zlib:crc32(Z),\n    Head = <<\n                                                % ID\n             16#1f, 16#8b,\n                                                % deflate\n             8:8,\n                                                % flags\n             0:8,\n                                                % mtime\n             0:32,\n                                                % xflags\n             0:8,\n                                                % OS_UNKNOWN\n                                                % Set to Unix instead?\n             255:8>>,\n    {ok, Priv, Bs} = gzipDeflate(Z, {Crc32,0}, Bin, Flush),\n    {ok, Priv, [Head | Bs]};\n\ngzipDeflate(Z, {Crc32,Size}, Bin, Flush) ->\n    Bs = zlib:deflate(Z, Bin, Flush),\n    {ok, Crc1} = crc32(Z, Crc32, Bin),\n    Size1 = Size+size(Bin),\n    Data =\n        if\n            Flush == finish ->\n                                                % Appending should not\n                                                % hurt, so let's be a\n                                                % bit more consistent\n                                                % here.\n                Bs ++ [<<Crc1:32/little, Size1:32/little>>];\n            true ->\n                Bs\n        end,\n    {ok, {Crc1, Size1}, Data}.\n\n\n%% like zlib:gzip/1, but returns an io list\ngzip(Data) ->\n    gzip(Data, #deflate{}).\n\ngzip(Data, DOpts) when is_binary(Data) ->\n    Z = zlib:open(),\n    {ok, _, D} = gzipDeflate(Z, gzipInit(Z, DOpts), Data, finish),\n    gzipEnd(Z),\n    zlib:close(Z),\n    {ok, D};\n\ngzip(Data, DOpts) ->\n    Z = zlib:open(),\n    gzip_loop(Z, gzipInit(Z, DOpts), Data, [], []).\n\ngzip_loop(Z, P, [], [], A) ->\n    {ok, _, D} = gzipDeflate(Z, P, <<>>, finish),\n    gzipEnd(Z),\n    zlib:close(Z),\n    {ok, [A|D]};\ngzip_loop(Z, P, B, C, A) when is_binary(B) ->\n    {ok, P1, D} = gzipDeflate(Z, P, B, none),\n    gzip_loop(Z, P1, C, [],\n              case D of\n                  [] -> A;\n                  _ ->\n                      case A of\n                          [] -> D;\n                          _ -> [A|D]\n                      end\n              end);\ngzip_loop(Z, P, [I|T], C, A) when is_integer(I) ->\n    gzip_loop(Z, P, list_to_binary([I|T]), C, A);\ngzip_loop(Z, P, [H], C, A) ->\n    gzip_loop(Z, P, H, C, A);\ngzip_loop(Z, P, [H|T], C, A) ->\n    gzip_loop(Z, P, H, [T|C], A);\ngzip_loop(Z, P, [], C, A) ->\n    gzip_loop(Z, P, C, [], A);\ngzip_loop(Z, P, I, C, A) when is_integer(I) ->\n    gzip_loop(Z, P, <<I>>, C, A).\n\n\n%% To work around a bug in zlib.\n\ncrc32(Z, CRC, Binary) ->\n    case port_control(Z, 17, <<CRC:32, Binary/binary>>) of\n        [2,A,B,C,D] -> {ok, (A bsl 24)+(B bsl 16)+(C bsl 8)+D}\n    end.\n\n"
  },
  {
    "path": "contrib/yaws/vsn.mk",
    "content": "YAWS_VSN=2.0.4\n"
  },
  {
    "path": "contrib/yaws-revert-broken-utf8-decoding.patch",
    "content": "This reverts commit 2e86da12b3ecc4482fd81d454cb4d70bc85269fa from the yaws\nsources that broke utf8 decoding for us. It applies to yaws 2.0.4.\n\ndiff --git a/src/json2.erl b/src/json2.erl\nindex 64a8a4f94..5f395b5dd 100644\n--- a/src/json2.erl\n+++ b/src/json2.erl\n@@ -273,8 +273,7 @@ scan_string([$\\\\,$u,U1,U2,U3,U4|Rest], A, X)\n     case erlang:list_to_integer([U1,U2,U3,U4], 16) of\n         Codepoint when Codepoint > 0 andalso\n                        (Codepoint < 16#d800 orelse Codepoint > 16#dfff) ->\n-            C = binary_to_list(unicode:characters_to_binary([Codepoint],utf8)),\n-            scan_string(Rest, lists:reverse(C)++A, X);\n+            scan_string(Rest, [Codepoint | A], X);\n         _ ->\n             Bad = [$\\\\,$u,U1,U2,U3,U4],\n             {done, {error, {bad_utf8_char, Bad}}, X}\n"
  },
  {
    "path": "contrib/yaws-setup.txt",
    "content": "1) download the new version from http://yaws.hyber.org/\n2) extract somewhere else\n3) copy ebin/, include/, src/, LICENSE, vsn.mk to <scalaris>/contrib/yaws\n4) autoreconf -fi && ./configure --disable-pam && make in the extracted yaws dir (not <scalaris>/contrib/yaws)\n5) copy src/mime_types.erl, src/yaws_appdeps.hrl, src/yaws_charset.hrl and\n   src/yaws_generated.erl to <scalaris>/contrib/yaws/src\n5a) fix absolute include path in mime_types.erl to \"yaws.hrl\"\n5b) remove src/*.hrl.in\n5c) apply yaws-revert-broken-utf8-decoding.patch in <scalaris>/contrib/yaws:\n    patch -p1 < ../yaws-revert-broken-utf8-decoding.patch\n6) in <scalaris>/contrib/yaws:\n    rm ebin/Makefile.*, include/Makefile.*, src/Makefile.*\n    echo \"/*.beam\" > ebin/.gitignore\n7) that's it, go back to <scalaris>/ and run make clean && ./configure && make\n\nYou probably have to patch yaws to make the erlang/dev dialyzer happy.\n"
  },
  {
    "path": "contrib/ycsb/.gitignore",
    "content": "/elasticsearch/target/\n"
  },
  {
    "path": "contrib/ycsb/BUILD",
    "content": "# Building YCSB\n\nTo build YCSB, run:\n\n    mvn clean package\n\n# Running YCSB\n\nOnce `mvn clean package` succeeds, you can run `ycsb` command:\n\n    ./bin/ycsb load basic workloads/workloada\n    ./bin/ycsb run basic workloads/workloada\n\n# Oracle NoSQL Database\n\nOracle NoSQL Database binding doesn't get built by default because there is no\nMaven repository for it. To build the binding:\n\n1. Download kv-ce-1.2.123.tar.gz from here:\n\n    http://www.oracle.com/technetwork/database/nosqldb/downloads/index.html\n\n2. Untar kv-ce-1.2.123.tar.gz and install kvclient-1.2.123.jar in your local\n   maven repository:\n\n    tar xfvz kv-ce-1.2.123.tar.gz\n    mvn install:install-file -Dfile=kv-1.2.123/lib/kvclient-1.2.123.jar \\\n        -DgroupId=com.oracle -DartifactId=kvclient -Dversion=1.2.123\n        -Dpackaging=jar\n\n3. Uncomment `<module>nosqldb</module>` and run `mvn clean package`.\n"
  },
  {
    "path": "contrib/ycsb/CHANGELOG",
    "content": "- gh-67 Use checkstyle (m1ch1)\n- gh-76 Implemented OrientDB client (lvca)\n- gh-88 YCSB client for Amazon DynamoDB (jananin)\n- gh-89 Patch for YCSB Cassandra Client version 1.0.6 (jananin)\n- gh-93 New ElasticSearch Database Implementation (saden1)\n- gh-97 Bug fixes in dynamodb plugin (jananin)\n\n0.1.4 - 2/22/12 \n\n- Fixes for Cassandra 0.7 client (tjake)\n- New generator FileGenerator (nono)\n- Fixes for MongoDB (nono)\n- Fixes for Voldemort (nono)\n- JDBC client (sudiptodas)\n- HotspotIntegerGenerator (sudiptodas)\t\n- Optimizing cassandra7 (joaquincasares)\n- Mysql key fix (joaquincasares)\n- Added a db plugin for Infinispan (maniksurtani)\n- gh-31 Support to stop benchmark after a maximum specified elapsed time. (sudiptodas)\n- gh-35 Cassandra0.8 support (joaquincasares)\n- gh-30 IllegalArgumentException with MongoDB (m1ch1)\n- gh-27 MongoDbClient was not working with non localhost URLs (arunxarun)\n- gh-29 Add simple sharding capabilities for JDBC driver (kibab)\n- gh-40 Merge Redis database interface layer (lehmannro)\n- gh-42 Response latencies are measured in microseconds (mikewied)\n- gh-43 Variable length fields (sears)\n- gh-44 Constant occupancy workload (sears)\n- gh-45 Modify DB API for efficient large object support (sears)\n- gh-46 Fixed typo in RedisClient (Zlender)\n- gh-49 Build fix (sears)\n- gh-50 Switch unordered key generation from FNV32 to FNV64 (sears)\n- gh-51 Improved Random Number Generation Performance and add Exponential distribution support (sears)\n- gh-52 Mongo db fix (sears)\n- gh-54 Add mapkeeper driver (m1ch1)\n- gh-57 voldemort - enable nio connector (akkumar)\n- gh-58 benchmarking with hbase 0.90.5 (akkumar)\n- gh-55 VMware vFabric GemFire (sbawaska)\n- gh-59 Cassandra 1.0.6 (akkumar)\n- gh-62 Oracle NoSQL Database Client (y-namiki)\n- gh-64 Mavenisation of YCSB with a (tar ball) distribution packager (hariprasad-k)\n- gh-65 Patch for improving the MongoDB Client (singhsiddharth)\n\n0.1.3 - 10/26/10\n\n= Voldemort binding (rsumbaly)\n= HBase client improvements (ryanobjc)\n= Fixes to Cassandra 0.7 binding (johanoskarsson, nickmbailey)\n= Added an interface for exporting the measurements and a JSON implementation. It can write to both stdout and to a file (johanoskarsson)\n-Other minor fixes (brianfrankcooper)\n\n0.1.2 - 5/12/10\n\n- MongoDB binding (ypai)\n- Cassandra 0.7 binding (johanoskarsson)\n- Simple command line interface (brianfrankcooper)\n- Faster string generation (tlipcon)\n- Avoid Bytes conversion in HBaseClient (tlipcon)\n\n0.1.1 - 4/25/10\n\n- Compiles under 1.5\n- Fixes doc and HBaseClient bugs\n\n0.1.0 - 4/23/10 \n\n- Initial open source release\n"
  },
  {
    "path": "contrib/ycsb/LICENSE.txt",
    "content": "Apache License\nVersion 2.0, January 2004\nhttp://www.apache.org/licenses/\n\nTERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n1. Definitions.\n\n\"License\" shall mean the terms and conditions for use, reproduction,\nand distribution as defined by Sections 1 through 9 of this document.\n\n\"Licensor\" shall mean the copyright owner or entity authorized by the\ncopyright owner that is granting the License.\n\n\"Legal Entity\" shall mean the union of the acting entity and all other\nentities that control, are controlled by, or are under common control\nwith that entity. For the purposes of this definition, \"control\" means\n(i) the power, direct or indirect, to cause the direction or\nmanagement of such entity, whether by contract or otherwise, or (ii)\nownership of fifty percent (50%) or more of the outstanding shares, or\n(iii) beneficial ownership of such entity.\n\n\"You\" (or \"Your\") shall mean an individual or Legal Entity exercising\npermissions granted by this License.\n\n\"Source\" form shall mean the preferred form for making modifications,\nincluding but not limited to software source code, documentation\nsource, and configuration files.\n\n\"Object\" form shall mean any form resulting from mechanical\ntransformation or translation of a Source form, including but not\nlimited to compiled object code, generated documentation, and\nconversions to other media types.\n\n\"Work\" shall mean the work of authorship, whether in Source or Object\nform, made available under the License, as indicated by a copyright\nnotice that is included in or attached to the work (an example is\nprovided in the Appendix below).\n\n\"Derivative Works\" shall mean any work, whether in Source or Object\nform, that is based on (or derived from) the Work and for which the\neditorial revisions, annotations, elaborations, or other modifications\nrepresent, as a whole, an original work of authorship. For the\npurposes of this License, Derivative Works shall not include works\nthat remain separable from, or merely link (or bind by name) to the\ninterfaces of, the Work and Derivative Works thereof.\n\n\"Contribution\" shall mean any work of authorship, including the\noriginal version of the Work and any modifications or additions to\nthat Work or Derivative Works thereof, that is intentionally submitted\nto Licensor for inclusion in the Work by the copyright owner or by an\nindividual or Legal Entity authorized to submit on behalf of the\ncopyright owner. For the purposes of this definition, \"submitted\"\nmeans any form of electronic, verbal, or written communication sent to\nthe Licensor or its representatives, including but not limited to\ncommunication on electronic mailing lists, source code control\nsystems, and issue tracking systems that are managed by, or on behalf\nof, the Licensor for the purpose of discussing and improving the Work,\nbut excluding communication that is conspicuously marked or otherwise\ndesignated in writing by the copyright owner as \"Not a Contribution.\"\n\n\"Contributor\" shall mean Licensor and any individual or Legal Entity\non behalf of whom a Contribution has been received by Licensor and\nsubsequently incorporated within the Work.\n\n2. Grant of Copyright License. Subject to the terms and conditions of\n   this License, each Contributor hereby grants to You a perpetual,\n   worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n   copyright license to reproduce, prepare Derivative Works of,\n   publicly display, publicly perform, sublicense, and distribute the\n   Work and such Derivative Works in Source or Object form.\n\n3. Grant of Patent License. Subject to the terms and conditions of\n   this License, each Contributor hereby grants to You a perpetual,\n   worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n   (except as stated in this section) patent license to make, have\n   made, use, offer to sell, sell, import, and otherwise transfer the\n   Work, where such license applies only to those patent claims\n   licensable by such Contributor that are necessarily infringed by\n   their Contribution(s) alone or by combination of their\n   Contribution(s) with the Work to which such Contribution(s) was\n   submitted. If You institute patent litigation against any entity\n   (including a cross-claim or counterclaim in a lawsuit) alleging\n   that the Work or a Contribution incorporated within the Work\n   constitutes direct or contributory patent infringement, then any\n   patent licenses granted to You under this License for that Work\n   shall terminate as of the date such litigation is filed.\n\n4. Redistribution. You may reproduce and distribute copies of the Work\n   or Derivative Works thereof in any medium, with or without\n   modifications, and in Source or Object form, provided that You meet\n   the following conditions:\n\n(a) You must give any other recipients of the Work or Derivative Works\na copy of this License; and\n\n(b) You must cause any modified files to carry prominent notices\nstating that You changed the files; and\n\n(c) You must retain, in the Source form of any Derivative Works that\nYou distribute, all copyright, patent, trademark, and attribution\nnotices from the Source form of the Work, excluding those notices that\ndo not pertain to any part of the Derivative Works; and\n\n(d) If the Work includes a \"NOTICE\" text file as part of its\ndistribution, then any Derivative Works that You distribute must\ninclude a readable copy of the attribution notices contained within\nsuch NOTICE file, excluding those notices that do not pertain to any\npart of the Derivative Works, in at least one of the following places:\nwithin a NOTICE text file distributed as part of the Derivative Works;\nwithin the Source form or documentation, if provided along with the\nDerivative Works; or, within a display generated by the Derivative\nWorks, if and wherever such third-party notices normally appear. The\ncontents of the NOTICE file are for informational purposes only and do\nnot modify the License. You may add Your own attribution notices\nwithin Derivative Works that You distribute, alongside or as an\naddendum to the NOTICE text from the Work, provided that such\nadditional attribution notices cannot be construed as modifying the\nLicense.\n\nYou may add Your own copyright statement to Your modifications and may\nprovide additional or different license terms and conditions for use,\nreproduction, or distribution of Your modifications, or for any such\nDerivative Works as a whole, provided Your use, reproduction, and\ndistribution of the Work otherwise complies with the conditions stated\nin this License.\n\n5. Submission of Contributions. Unless You explicitly state otherwise,\n   any Contribution intentionally submitted for inclusion in the Work\n   by You to the Licensor shall be under the terms and conditions of\n   this License, without any additional terms or\n   conditions. Notwithstanding the above, nothing herein shall\n   supersede or modify the terms of any separate license agreement you\n   may have executed with Licensor regarding such Contributions.\n\n6. Trademarks. This License does nr work.\n\nTo apply the Apache License to your work, attach the following\nboilerplate notice, with the fields enclosed by brackets \"[]\" replaced\nwith your own identifying information. (Don't include the brackets!)\nThe text should be enclosed in the appropriate comment syntax for the\nfile format. We also recommend that a file or class name and\ndescription of purpose be included on the same \"printed page\" as the\ncopyright notice for easier identification within third-party\narchives.\n\nCopyright [yyyy] [name of copyright owner]\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\nimplied.\n\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\n\n"
  },
  {
    "path": "contrib/ycsb/NOTICE.txt",
    "content": "=========================================================================\nNOTICE file for use with, and corresponding to Section 4 of,            \nthe Apache License, Version 2.0,                                  \nin this case for the YCSB project.                     \n=========================================================================\n\n   This product includes software developed by\n   Yahoo! Inc. (www.yahoo.com)\n   Copyright (c) 2010 Yahoo! Inc.  All rights reserved.\n"
  },
  {
    "path": "contrib/ycsb/README",
    "content": "Yahoo! Cloud System Benchmark (YCSB)\n====================================\n\nLinks\n-----\nhttp://wiki.github.com/brianfrankcooper/YCSB/\nhttp://research.yahoo.com/Web_Information_Management/YCSB\nycsb-users@yahoogroups.com\n\nGetting Started\n---------------\n\n1. Download the latest release of YCSB:\n\n    wget https://github.com/downloads/brianfrankcooper/YCSB/ycsb-0.1.4.tar.gz\n    tar xfvz ycsb-0.1.4\n    cd ycsb-0.1.4\n\n2. Set up a database to benchmark. There is a README file under each binding\n   directory.\n\n3. Run YCSB command. \n\n    bin/ycsb load basic -P workloads/workloada\n    bin/ycsb run basic -P workloads/workloada\n\n   Running the `ycsb` command without any argument will print the usage. \n   \n   See https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload\n   for a detailed documentation on how to run a workload.\n\n   See https://github.com/brianfrankcooper/YCSB/wiki/Core-Properties for \n   the list of available workload properties.\n"
  },
  {
    "path": "contrib/ycsb/bin/ycsb",
    "content": "#!/usr/bin/env python\n\nimport os\nimport sys\nimport subprocess\n\nBASE_URL = \"https://github.com/brianfrankcooper/YCSB/tree/master/\"\nCOMMANDS = {\n    \"shell\" : {\n        \"command\"     : \"\",\n        \"description\" : \"Interactive mode\",\n        \"main\"        : \"com.yahoo.ycsb.CommandLine\",\n    },\n    \"load\" : {\n        \"command\"     : \"-load\",\n        \"description\" : \"Execute the load phase\",\n        \"main\"        : \"com.yahoo.ycsb.Client\",\n    },\n    \"run\" : {\n        \"command\"     : \"-t\",\n        \"description\" : \"Execute the transaction phase\",\n        \"main\"        : \"com.yahoo.ycsb.Client\",\n    },\n}\n\nDATABASES = {\n    \"basic\"        : \"com.yahoo.ycsb.BasicDB\",\n    \"cassandra-7\"  : \"com.yahoo.ycsb.db.CassandraClient7\",\n    \"cassandra-8\"  : \"com.yahoo.ycsb.db.CassandraClient8\",\n    \"cassandra-10\" : \"com.yahoo.ycsb.db.CassandraClient10\",\n    \"dynamodb\"     : \"com.yahoo.ycsb.db.DynamoDBClient\",\n    \"elasticsearch\": \"com.yahoo.ycsb.db.ElasticSearchClient\",\n    \"gemfire\"      : \"com.yahoo.ycsb.db.GemFireClient\",\n    \"hbase\"        : \"com.yahoo.ycsb.db.HBaseClient\",\n    \"hypertable\"   : \"com.yahoo.ycsb.db.HypertableClient\",\n    \"infinispan\"   : \"com.yahoo.ycsb.db.InfinispanClient\",\n    \"jdbc\"         : \"com.yahoo.ycsb.db.JdbcDBClient\",\n    \"mapkeeper\"    : \"com.yahoo.ycsb.db.MapKeeperClient\",\n    \"mongodb\"      : \"com.yahoo.ycsb.db.MongoDbClient\",\n    \"nosqldb\"      : \"com.yahoo.ycsb.db.NoSqlDbClient\",\n    \"orientdb\"     : \"com.yahoo.ycsb.db.OrientDBClient\",\n    \"redis\"        : \"com.yahoo.ycsb.db.RedisClient\", \n    \"scalaris\"     : \"com.yahoo.ycsb.db.ScalarisClient\",\n    \"voldemort\"    : \"com.yahoo.ycsb.db.VoldemortClient\", \n}\n\nOPTIONS = {\n    \"-P file\"      : \"Specify workload file\",\n    \"-p key=value\" : \"Override workload property\",\n    \"-s\"           : \"Print status to stderr\",\n    \"-target n\"    : \"Target ops/sec (default: unthrottled)\",\n    \"-threads n\"   : \"Number of client threads (default: 1)\",\n}\n\ndef usage():\n    print \"Usage: %s command database [options]\" % sys.argv[0]\n\n    print \"\\nCommands:\"\n    for command in sorted(COMMANDS.keys()):\n        print \"    %s %s\" % (command.ljust(13), COMMANDS[command][\"description\"])\n\n    print \"\\nDatabases:\"\n    for db in sorted(DATABASES.keys()):\n        print \"    %s %s\" % (db.ljust(13), BASE_URL + db.split(\"-\")[0])\n\n    print \"\\nOptions:\"\n    for option in sorted(OPTIONS.keys()):\n        print \"    %s %s\" % (option.ljust(13), OPTIONS[option])\n\n    print \"\"\"\\nWorkload Files:\n    There are various predefined workloads under workloads/ directory.\n    See https://github.com/brianfrankcooper/YCSB/wiki/Core-Properties\n    for the list of workload properties.\"\"\"\n\n    sys.exit(1)\n\ndef find_jars(dir, database):\n    jars = []\n    for (dirpath, dirnames, filenames) in os.walk(dir):\n        if dirpath.endswith(\"conf\"):\n            jars.append(dirpath)\n        for filename in filenames:\n            if filename.endswith(\".jar\") and \\\n               (filename.startswith(\"core\") or \\\n                filename.startswith(database.split(\"-\")[0]) or \\\n                not \"binding\" in filename):\n                jars.append(os.path.join(dirpath, filename))\n    return jars\n\ndef get_ycsb_home():\n    dir = os.path.abspath(os.path.dirname(sys.argv[0]))\n    while \"CHANGELOG\" not in os.listdir(dir):\n        dir = os.path.join(dir, os.path.pardir)\n    return os.path.abspath(dir)\n\nif len(sys.argv) < 3:\n    usage()\nif sys.argv[1] not in COMMANDS:\n    print \"ERROR: Command '%s' not found\" % sys.argv[1]\n    usage()\nif sys.argv[2] not in DATABASES:\n    print \"ERROR: Database '%s' not found\" % sys.argv[2]\n    usage()\n\nycsb_home = get_ycsb_home()\ncommand = COMMANDS[sys.argv[1]][\"command\"]\ndatabase = sys.argv[2]\ndb_classname = DATABASES[database]\noptions = sys.argv[3:]\n\nycsb_command = [\"java\", \"-cp\", \":\".join(find_jars(ycsb_home, database)), \\\n                COMMANDS[sys.argv[1]][\"main\"], \"-db\", db_classname] + options\nif command:\n    ycsb_command.append(command)\nprint \" \".join(ycsb_command)\nsubprocess.call(ycsb_command)\n"
  },
  {
    "path": "contrib/ycsb/bin/ycsb.sh",
    "content": "#! /usr/bin/env bash\n\n# Set the YCSB specific environment. Adds all the required libraries to the class path.\n\n# Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n# *                                                                                                                                                                                 \n# * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n# * may not use this file except in compliance with the License. You                                                                                                                \n# * may obtain a copy of the License at                                                                                                                                             \n# *                                                                                                                                                                                 \n# * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n# *                                                                                                                                                                                 \n# * Unless required by applicable law or agreed to in writing, software                                                                                                             \n# * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n# * implied. See the License for the specific language governing                                                                                                                    \n# * permissions and limitations under the License. See accompanying                                                                                                                 \n# * LICENSE file. \n#\n\n# The Java implementation to use. This is required.\n#export JAVA_HOME=\n\n# Any JVM options to pass.\n#export YCSB_OPTS=\"-Djava.compiler=NONE\"\n\n# YCSB client heap size.\n#export YCSB_HEAP_SIZE=500\n\nthis=`dirname \"$0\"`\nthis=`cd \"$this\"; pwd`\n\nwhile [ -h \"$this\" ]; do\n  ls=`ls -ld \"$this\"`\n  link=`expr \"$ls\" : '.*-> \\(.*\\)$'`\n  if expr \"$link\" : '.*/.*' > /dev/null; then\n    this=\"$link\"\n  else\n    this=`dirname \"$this\"`/\"$link\"\n  fi\ndone\n\nbin=`dirname \"$this\"`\nscript=`basename \"$this\"`\nbin=`cd \"$bin\"; pwd`\nthis=\"$bin/$script\"\n\n# the root of the Hadoop installation\nexport YCSB_HOME=`dirname \"$this\"`\n\necho \"YCSB_HOME $YCSB_HOME\"\n\ncygwin=false\ncase \"`uname`\" in\nCYGWIN*) cygwin=true;;\nesac\n\n# if no args specified, show usage\nif [ $# = 0 ]; then\n  echo \"Usage: ycsb CLASSNAME\"\n  echo \"where CLASSNAME is the name of the class to run\"\n  echo \"The jar file for the class must be in bin, build, lib, or db/*/lib.\"\n  exit 1\nfi\n\n# get arguments\nCOMMAND=$1\nshift\n\nJAVA=\"\"\nif [ \"$JAVA_HOME\" != \"\" ]; then\n  JAVA=$JAVA_HOME/bin/java\nelse\n  echo \"JAVA_HOME must be set.\"\n  exit 1\nfi\n\nJAVA_HEAP_MAX=-Xmx500m\n# check envvars which might override default args\nif [ \"$YCSB_HEAP_SIZE\" != \"\" ]; then\n  JAVA_HEAP_MAX=\"-Xmx\"\"$YCSB_HEAP_SIZE\"\"m\"\nfi\n\n# Set the classpath.\n\nif [ \"$CLASSPATH\" != \"\" ]; then\n  CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar\nelse\n  CLASSPATH=$JAVA_HOME/lib/tools.jar\nfi\n\n# so that filenames w/ spaces are handled correctly in loops below\nIFS=\n\nfor f in $YCSB_HOME/build/*.jar; do\n  CLASSPATH=${CLASSPATH}:$f\ndone\n\nfor f in $YCSB_HOME/lib/*.jar; do\n  CLASSPATH=${CLASSPATH}:$f\ndone\n\nfor f in $YCSB_HOME/db/*; do\n  if [ -d $f ]; then\n    for j in $f/lib/*.jar; do\n      CLASSPATH=${CLASSPATH}:$j\n    done\n  fi\ndone\n\n#echo \"CLASSPATH=$CLASSPATH\"\n\n# restore ordinary behavior\nunset IFS\n\nCLASS=$COMMAND\n\n# cygwin path translation\nif $cygwin; then\n  CLASSPATH=`cygpath -p -w \"$CLASSPATH\"`\n  YCSB_HOME=`cygpath -w \"$YCSB_HOME\"`\nfi\n\n#echo \"Executing command $CLASS with options $JAVA_HEAP_MAX $YCSB_OPTS $CLASS $@\"\nexec \"$JAVA\" $JAVA_HEAP_MAX $YCSB_OPTS -classpath \"$CLASSPATH\" $CLASS \"$@\"\n"
  },
  {
    "path": "contrib/ycsb/checkstyle.xml",
    "content": "<?xml version=\"1.0\"?>\n<!DOCTYPE module PUBLIC\n    \"-//Puppy Crawl//DTD Check Configuration 1.2//EN\"\n    \"http://www.puppycrawl.com/dtds/configuration_1_2.dtd\">\n\n<!--\n\n  Checkstyle configuration for Hadoop that is based on the sun_checks.xml file\n  that is bundled with Checkstyle and includes checks for:\n\n    - the Java Language Specification at\n      http://java.sun.com/docs/books/jls/second_edition/html/index.html\n\n    - the Sun Code Conventions at http://java.sun.com/docs/codeconv/\n\n    - the Javadoc guidelines at\n      http://java.sun.com/j2se/javadoc/writingdoccomments/index.html\n\n    - the JDK Api documentation http://java.sun.com/j2se/docs/api/index.html\n\n    - some best practices\n\n  Checkstyle is very configurable. Be sure to read the documentation at\n  http://checkstyle.sf.net (or in your downloaded distribution).\n\n  Most Checks are configurable, be sure to consult the documentation.\n\n  To completely disable a check, just comment it out or delete it from the file.\n\n  Finally, it is worth reading the documentation.\n\n-->\n\n<module name=\"Checker\">\n\n    <!-- Checks that a package.html file exists for each package.     -->\n    <!-- See http://checkstyle.sf.net/config_javadoc.html#PackageHtml -->\n    <module name=\"JavadocPackage\"/>\n\n    <!-- Checks whether files end with a new line.                        -->\n    <!-- See http://checkstyle.sf.net/config_misc.html#NewlineAtEndOfFile -->\n    <!-- module name=\"NewlineAtEndOfFile\"/-->\n\n    <!-- Checks that property files contain the same keys.         -->\n    <!-- See http://checkstyle.sf.net/config_misc.html#Translation -->\n    <module name=\"Translation\"/>\n\n    <module name=\"FileLength\"/>\n    <module name=\"FileTabCharacter\"/>\n\n    <module name=\"TreeWalker\">\n\n        <!-- Checks for Javadoc comments.                     -->\n        <!-- See http://checkstyle.sf.net/config_javadoc.html -->\n        <module name=\"JavadocType\">\n          <property name=\"scope\" value=\"public\"/>\n          <property name=\"allowMissingParamTags\" value=\"true\"/>\n        </module>\n        <module name=\"JavadocStyle\"/>\n\n        <!-- Checks for Naming Conventions.                  -->\n        <!-- See http://checkstyle.sf.net/config_naming.html -->\n        <module name=\"ConstantName\"/>\n        <module name=\"LocalFinalVariableName\"/>\n        <module name=\"LocalVariableName\"/>\n        <module name=\"MemberName\"/>\n        <module name=\"MethodName\"/>\n        <module name=\"PackageName\"/>\n        <module name=\"ParameterName\"/>\n        <module name=\"StaticVariableName\"/>\n        <module name=\"TypeName\"/>\n\n\n        <!-- Checks for Headers                                -->\n        <!-- See http://checkstyle.sf.net/config_header.html   -->\n        <!-- <module name=\"Header\">                            -->\n            <!-- The follow property value demonstrates the ability     -->\n            <!-- to have access to ANT properties. In this case it uses -->\n            <!-- the ${basedir} property to allow Checkstyle to be run  -->\n            <!-- from any directory within a project. See property      -->\n            <!-- expansion,                                             -->\n            <!-- http://checkstyle.sf.net/config.html#properties        -->\n            <!-- <property                                              -->\n            <!--     name=\"headerFile\"                                  -->\n            <!--     value=\"${basedir}/java.header\"/>                   -->\n        <!-- </module> -->\n\n        <!-- Following interprets the header file as regular expressions. -->\n        <!-- <module name=\"RegexpHeader\"/>                                -->\n\n\n        <!-- Checks for imports                              -->\n        <!-- See http://checkstyle.sf.net/config_import.html -->\n        <module name=\"IllegalImport\"/> <!-- defaults to sun.* packages -->\n        <module name=\"RedundantImport\"/>\n        <module name=\"UnusedImports\"/>\n\n\n        <!-- Checks for Size Violations.                    -->\n        <!-- See http://checkstyle.sf.net/config_sizes.html -->\n        <module name=\"LineLength\"/>\n        <module name=\"MethodLength\"/>\n        <module name=\"ParameterNumber\"/>\n\n\n        <!-- Checks for whitespace                               -->\n        <!-- See http://checkstyle.sf.net/config_whitespace.html -->\n        <module name=\"EmptyForIteratorPad\"/>\n        <module name=\"MethodParamPad\"/>\n        <module name=\"NoWhitespaceAfter\"/>\n        <module name=\"NoWhitespaceBefore\"/>\n        <module name=\"ParenPad\"/>\n        <module name=\"TypecastParenPad\"/>\n        <module name=\"WhitespaceAfter\">\n\t    \t<property name=\"tokens\" value=\"COMMA, SEMI\"/>\n\t\t</module>\n\n\n        <!-- Modifier Checks                                    -->\n        <!-- See http://checkstyle.sf.net/config_modifiers.html -->\n        <module name=\"ModifierOrder\"/>\n        <module name=\"RedundantModifier\"/>\n\n\n        <!-- Checks for blocks. You know, those {}'s         -->\n        <!-- See http://checkstyle.sf.net/config_blocks.html -->\n        <module name=\"AvoidNestedBlocks\"/>\n        <module name=\"EmptyBlock\"/>\n        <module name=\"LeftCurly\"/>\n        <module name=\"NeedBraces\"/>\n        <module name=\"RightCurly\"/>\n\n\n        <!-- Checks for common coding problems               -->\n        <!-- See http://checkstyle.sf.net/config_coding.html -->\n        <!-- module name=\"AvoidInlineConditionals\"/-->\n        <module name=\"DoubleCheckedLocking\"/>\n        <module name=\"EmptyStatement\"/>\n        <module name=\"EqualsHashCode\"/>\n        <module name=\"HiddenField\">\n          <property name=\"ignoreConstructorParameter\" value=\"true\"/>\n        </module>\n        <module name=\"IllegalInstantiation\"/>\n        <module name=\"InnerAssignment\"/>\n        <module name=\"MissingSwitchDefault\"/>\n        <module name=\"RedundantThrows\"/>\n        <module name=\"SimplifyBooleanExpression\"/>\n        <module name=\"SimplifyBooleanReturn\"/>\n\n        <!-- Checks for class design                         -->\n        <!-- See http://checkstyle.sf.net/config_design.html -->\n        <module name=\"FinalClass\"/>\n        <module name=\"HideUtilityClassConstructor\"/>\n        <module name=\"InterfaceIsType\"/>\n        <module name=\"VisibilityModifier\"/>\n\n\n        <!-- Miscellaneous other checks.                   -->\n        <!-- See http://checkstyle.sf.net/config_misc.html -->\n        <module name=\"ArrayTypeStyle\"/>\n        <module name=\"Indentation\">\n            <property name=\"basicOffset\" value=\"2\" />\n            <property name=\"caseIndent\" value=\"0\" />\n        </module> \n        <module name=\"TodoComment\"/>\n        <module name=\"UpperEll\"/>\n\n    </module>\n\n</module>\n"
  },
  {
    "path": "contrib/ycsb/core/.gitignore",
    "content": "/target/\n"
  },
  {
    "path": "contrib/ycsb/core/pom.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<project xmlns=\"http://maven.apache.org/POM/4.0.0\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd\">\n  <modelVersion>4.0.0</modelVersion>\n  <parent>\n    <groupId>com.yahoo.ycsb</groupId>\n    <artifactId>root</artifactId>\n    <version>0.1.4</version>\n  </parent>\n  \n  <artifactId>core</artifactId>\n  <name>Core YCSB</name>\n  <packaging>jar</packaging>\n\n  <properties>\n     <jackson.api.version>1.9.4</jackson.api.version>\n  </properties>\n\n  <dependencies>\t\n    <dependency>\n      <groupId>org.codehaus.jackson</groupId>\n      <artifactId>jackson-mapper-asl</artifactId>\n      <version>${jackson.api.version}</version>\n    </dependency>\n    <dependency>\n      <groupId>org.codehaus.jackson</groupId>\n      <artifactId>jackson-core-asl</artifactId>\n      <version>${jackson.api.version}</version>\n    </dependency>\n    <dependency>\n      <groupId>org.testng</groupId>\n      <artifactId>testng</artifactId>\n      <version>7.7.0</version>\n      <scope>test</scope>\n    </dependency>\n  </dependencies>\t\n\n</project>\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/BasicDB.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb;\n\nimport java.util.HashMap;\nimport java.util.Properties;\nimport java.util.Set;\nimport java.util.Enumeration;\nimport java.util.Vector;\n\n\n/**\n * Basic DB that just prints out the requested operations, instead of doing them against a database.\n */\npublic class BasicDB extends DB\n{\n\tpublic static final String VERBOSE=\"basicdb.verbose\";\n\tpublic static final String VERBOSE_DEFAULT=\"true\";\n\t\n\tpublic static final String SIMULATE_DELAY=\"basicdb.simulatedelay\";\n\tpublic static final String SIMULATE_DELAY_DEFAULT=\"0\";\n\t\n\t\n\tboolean verbose;\n\tint todelay;\n\n\tpublic BasicDB()\n\t{\n\t\ttodelay=0;\n\t}\n\n\t\n\tvoid delay()\n\t{\n\t\tif (todelay>0)\n\t\t{\n\t\t\ttry\n\t\t\t{\n\t\t\t\tThread.sleep((long)Utils.random().nextInt(todelay));\n\t\t\t}\n\t\t\tcatch (InterruptedException e)\n\t\t\t{\n\t\t\t\t//do nothing\n\t\t\t}\n\t\t}\n\t}\n\n\t/**\n\t * Initialize any state for this DB.\n\t * Called once per DB instance; there is one DB instance per client thread.\n\t */\n\t@SuppressWarnings(\"unchecked\")\n\tpublic void init()\n\t{\n\t\tverbose=Boolean.parseBoolean(getProperties().getProperty(VERBOSE, VERBOSE_DEFAULT));\n\t\ttodelay=Integer.parseInt(getProperties().getProperty(SIMULATE_DELAY, SIMULATE_DELAY_DEFAULT));\n\t\t\n\t\tif (verbose)\n\t\t{\n\t\t\tSystem.out.println(\"***************** properties *****************\");\n\t\t\tProperties p=getProperties();\n\t\t\tif (p!=null)\n\t\t\t{\n\t\t\t\tfor (Enumeration e=p.propertyNames(); e.hasMoreElements(); )\n\t\t\t\t{\n\t\t\t\t\tString k=(String)e.nextElement();\n\t\t\t\t\tSystem.out.println(\"\\\"\"+k+\"\\\"=\\\"\"+p.getProperty(k)+\"\\\"\");\n\t\t\t\t}\n\t\t\t}\n\t\t\tSystem.out.println(\"**********************************************\");\n\t\t}\n\t}\n\n\t/**\n\t * Read a record from the database. Each field/value pair from the result will be stored in a HashMap.\n\t *\n\t * @param table The name of the table\n\t * @param key The record key of the record to read.\n\t * @param fields The list of fields to read, or null for all of them\n\t * @param result A HashMap of field/value pairs for the result\n\t * @return Zero on success, a non-zero error code on error\n\t */\n\tpublic int read(String table, String key, Set<String> fields, HashMap<String,ByteIterator> result)\n\t{\n\t\tdelay();\n\n\t\tif (verbose)\n\t\t{\n\t\t\tSystem.out.print(\"READ \"+table+\" \"+key+\" [ \");\n\t\t\tif (fields!=null)\n\t\t\t{\n\t\t\t\tfor (String f : fields)\n\t\t\t\t{\n\t\t\t\t\tSystem.out.print(f+\" \");\n\t\t\t\t}\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\tSystem.out.print(\"<all fields>\");\n\t\t\t}\n\n\t\t\tSystem.out.println(\"]\");\n\t\t}\n\n\t\treturn 0;\n\t}\n\t\n\t/**\n\t * Perform a range scan for a set of records in the database. Each field/value pair from the result will be stored in a HashMap.\n\t *\n\t * @param table The name of the table\n\t * @param startkey The record key of the first record to read.\n\t * @param recordcount The number of records to read\n\t * @param fields The list of fields to read, or null for all of them\n\t * @param result A Vector of HashMaps, where each HashMap is a set field/value pairs for one record\n\t * @return Zero on success, a non-zero error code on error\n\t */\n\tpublic int scan(String table, String startkey, int recordcount, Set<String> fields, Vector<HashMap<String,ByteIterator>> result)\n\t{\n\t\tdelay();\n\n\t\tif (verbose)\n\t\t{\n\t\t\tSystem.out.print(\"SCAN \"+table+\" \"+startkey+\" \"+recordcount+\" [ \");\n\t\t\tif (fields!=null)\n\t\t\t{\n\t\t\t\tfor (String f : fields)\n\t\t\t\t{\n\t\t\t\t\tSystem.out.print(f+\" \");\n\t\t\t\t}\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\tSystem.out.print(\"<all fields>\");\n\t\t\t}\n\n\t\t\tSystem.out.println(\"]\");\n\t\t}\n\n\t\treturn 0;\n\t}\n\n\t/**\n\t * Update a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n\t * record key, overwriting any existing values with the same field name.\n\t *\n\t * @param table The name of the table\n\t * @param key The record key of the record to write.\n\t * @param values A HashMap of field/value pairs to update in the record\n\t * @return Zero on success, a non-zero error code on error\n\t */\n\tpublic int update(String table, String key, HashMap<String,ByteIterator> values)\n\t{\n\t\tdelay();\n\n\t\tif (verbose)\n\t\t{\n\t\t\tSystem.out.print(\"UPDATE \"+table+\" \"+key+\" [ \");\n\t\t\tif (values!=null)\n\t\t\t{\n\t\t\t\tfor (String k : values.keySet())\n\t\t\t\t{\n\t\t\t\t\tSystem.out.print(k+\"=\"+values.get(k)+\" \");\n\t\t\t\t}\n\t\t\t}\n\t\t\tSystem.out.println(\"]\");\n\t\t}\n\n\t\treturn 0;\n\t}\n\n\t/**\n\t * Insert a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n\t * record key.\n\t *\n\t * @param table The name of the table\n\t * @param key The record key of the record to insert.\n\t * @param values A HashMap of field/value pairs to insert in the record\n\t * @return Zero on success, a non-zero error code on error\n\t */\n\tpublic int insert(String table, String key, HashMap<String,ByteIterator> values)\n\t{\n\t\tdelay();\n\n\t\tif (verbose)\n\t\t{\n\t\t\tSystem.out.print(\"INSERT \"+table+\" \"+key+\" [ \");\n\t\t\tif (values!=null)\n\t\t\t{\n\t\t\t\tfor (String k : values.keySet())\n\t\t\t\t{\n\t\t\t\t\tSystem.out.print(k+\"=\"+values.get(k)+\" \");\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tSystem.out.println(\"]\");\n\t\t}\n\n\t\treturn 0;\n\t}\n\n\n\t/**\n\t * Delete a record from the database. \n\t *\n\t * @param table The name of the table\n\t * @param key The record key of the record to delete.\n\t * @return Zero on success, a non-zero error code on error\n\t */\n\tpublic int delete(String table, String key)\n\t{\n\t\tdelay();\n\n\t\tif (verbose)\n\t\t{\n\t\t\tSystem.out.println(\"DELETE \"+table+\" \"+key);\n\t\t}\n\n\t\treturn 0;\n\t}\n\n\t/**\n\t * Short test of BasicDB\n\t */\n\t/*\n\tpublic static void main(String[] args)\n\t{\n\t\tBasicDB bdb=new BasicDB();\n\n\t\tProperties p=new Properties();\n\t\tp.setProperty(\"Sky\",\"Blue\");\n\t\tp.setProperty(\"Ocean\",\"Wet\");\n\n\t\tbdb.setProperties(p);\n\n\t\tbdb.init();\n\n\t\tHashMap<String,String> fields=new HashMap<String,String>();\n\t\tfields.put(\"A\",\"X\");\n\t\tfields.put(\"B\",\"Y\");\n\n\t\tbdb.read(\"table\",\"key\",null,null);\n\t\tbdb.insert(\"table\",\"key\",fields);\n\n\t\tfields=new HashMap<String,String>();\n\t\tfields.put(\"C\",\"Z\");\n\n\t\tbdb.update(\"table\",\"key\",fields);\n\n\t\tbdb.delete(\"table\",\"key\");\n\t}*/\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/ByteArrayByteIterator.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\npackage com.yahoo.ycsb;\n\npublic class ByteArrayByteIterator extends ByteIterator {\n\tbyte[] str;\n\tint off;\n\tfinal int len;\n\tpublic ByteArrayByteIterator(byte[] s) {\n\t\tthis.str = s;\n\t\tthis.off = 0;\n\t\tthis.len = s.length;\n\t}\n\n\tpublic ByteArrayByteIterator(byte[] s, int off, int len) {\n\t\tthis.str = s;\n\t\tthis.off = off;\n\t\tthis.len = off + len;\n\t}\n\n\t@Override\n\tpublic boolean hasNext() {\n\t\treturn off < len;\n\t}\n\n\t@Override\n\tpublic byte nextByte() {\n\t\tbyte ret = str[off];\n\t\toff++;\n\t\treturn ret;\n\t}\n\n\t@Override\n\tpublic long bytesLeft() {\n\t\treturn len - off;\n\t}\n\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/ByteIterator.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\npackage com.yahoo.ycsb;\n\nimport java.util.Iterator;\nimport java.util.ArrayList;\n/**\n * YCSB-specific buffer class.  ByteIterators are designed to support\n * efficient field generation, and to allow backend drivers that can stream\n * fields (instead of materializing them in RAM) to do so.\n * <p>\n * YCSB originially used String objects to represent field values.  This led to\n * two performance issues.\n * </p><p>\n * First, it leads to unnecessary conversions between UTF-16 and UTF-8, both\n * during field generation, and when passing data to byte-based backend\n * drivers.\n * </p><p>\n * Second, Java strings are represented internally using UTF-16, and are\n * built by appending to a growable array type (StringBuilder or\n * StringBuffer), then calling a toString() method.  This leads to a 4x memory\n * overhead as field values are being built, which prevented YCSB from\n * driving large object stores.\n * </p>\n * The StringByteIterator class contains a number of convenience methods for\n * backend drivers that convert between Map&lt;String,String&gt; and\n * Map&lt;String,ByteBuffer&gt;.\n *\n * @author sears\n */\npublic abstract class ByteIterator implements Iterator<Byte> {\n\n\t@Override\n\tpublic abstract boolean hasNext();\n\n\t@Override\n\tpublic Byte next() {\n\t\tthrow new UnsupportedOperationException();\n\t\t//return nextByte();\n\t}\n\n\tpublic abstract byte nextByte();\n        /** @return byte offset immediately after the last valid byte */\n\tpublic int nextBuf(byte[] buf, int buf_off) {\n\t\tint sz = buf_off;\n\t\twhile(sz < buf.length && hasNext()) {\n\t\t\tbuf[sz] = nextByte();\n\t\t\tsz++;\n\t\t}\n\t\treturn sz;\n\t}\n\n\tpublic abstract long bytesLeft();\n\t\n\t@Override\n\tpublic void remove() {\n\t\tthrow new UnsupportedOperationException();\n\t}\n\n\t/** Consumes remaining contents of this object, and returns them as a string. */\n\tpublic String toString() {\n\t\tStringBuilder sb = new StringBuilder();\n\t\twhile(this.hasNext()) { sb.append((char)nextByte()); }\n\t\treturn sb.toString();\n\t}\n\t/** Consumes remaining contents of this object, and returns them as a byte array. */\n\tpublic byte[] toArray() {\n\t    long left = bytesLeft();\n\t    if(left != (int)left) { throw new ArrayIndexOutOfBoundsException(\"Too much data to fit in one array!\"); }\n\t    byte[] ret = new byte[(int)left];\n\t    int off = 0;\n\t    while(off < ret.length) {\n\t\toff = nextBuf(ret, off);\n\t    }\n\t    return ret;\n\t}\n\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/Client.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb;\n\n\nimport java.io.*;\nimport java.text.DecimalFormat;\nimport java.util.*;\n\nimport com.yahoo.ycsb.measurements.Measurements;\nimport com.yahoo.ycsb.measurements.exporter.MeasurementsExporter;\nimport com.yahoo.ycsb.measurements.exporter.TextMeasurementsExporter;\n\n//import org.apache.log4j.BasicConfigurator;\n\n/**\n * A thread to periodically show the status of the experiment, to reassure you that progress is being made.\n * \n * @author cooperb\n *\n */\nclass StatusThread extends Thread\n{\n\tVector<Thread> _threads;\n\tString _label;\n\tboolean _standardstatus;\n\t\n\t/**\n\t * The interval for reporting status.\n\t */\n\tpublic static final long sleeptime=10000;\n\n\tpublic StatusThread(Vector<Thread> threads, String label, boolean standardstatus)\n\t{\n\t\t_threads=threads;\n\t\t_label=label;\n\t\t_standardstatus=standardstatus;\n\t}\n\n\t/**\n\t * Run and periodically report status.\n\t */\n\tpublic void run()\n\t{\n\t\tlong st=System.currentTimeMillis();\n\n\t\tlong lasten=st;\n\t\tlong lasttotalops=0;\n\t\t\n\t\tboolean alldone;\n\n\t\tdo \n\t\t{\n\t\t\talldone=true;\n\n\t\t\tint totalops=0;\n\n\t\t\t//terminate this thread when all the worker threads are done\n\t\t\tfor (Thread t : _threads)\n\t\t\t{\n\t\t\t\tif (t.getState()!=Thread.State.TERMINATED)\n\t\t\t\t{\n\t\t\t\t\talldone=false;\n\t\t\t\t}\n\n\t\t\t\tClientThread ct=(ClientThread)t;\n\t\t\t\ttotalops+=ct.getOpsDone();\n\t\t\t}\n\n\t\t\tlong en=System.currentTimeMillis();\n\n\t\t\tlong interval=en-st;\n\t\t\t//double throughput=1000.0*((double)totalops)/((double)interval);\n\n\t\t\tdouble curthroughput=1000.0*(((double)(totalops-lasttotalops))/((double)(en-lasten)));\n\t\t\t\n\t\t\tlasttotalops=totalops;\n\t\t\tlasten=en;\n\t\t\t\n\t\t\tDecimalFormat d = new DecimalFormat(\"#.##\");\n\t\t\t\n\t\t\tif (totalops==0)\n\t\t\t{\n\t\t\t\tSystem.err.println(_label+\" \"+(interval/1000)+\" sec: \"+totalops+\" operations; \"+Measurements.getMeasurements().getSummary());\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\tSystem.err.println(_label+\" \"+(interval/1000)+\" sec: \"+totalops+\" operations; \"+d.format(curthroughput)+\" current ops/sec; \"+Measurements.getMeasurements().getSummary());\n\t\t\t}\n\n\t\t\tif (_standardstatus)\n\t\t\t{\n\t\t\tif (totalops==0)\n\t\t\t{\n\t\t\t\tSystem.out.println(_label+\" \"+(interval/1000)+\" sec: \"+totalops+\" operations; \"+Measurements.getMeasurements().getSummary());\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\tSystem.out.println(_label+\" \"+(interval/1000)+\" sec: \"+totalops+\" operations; \"+d.format(curthroughput)+\" current ops/sec; \"+Measurements.getMeasurements().getSummary());\n\t\t\t}\n\t\t\t}\n\n\t\t\ttry\n\t\t\t{\n\t\t\t\tsleep(sleeptime);\n\t\t\t}\n\t\t\tcatch (InterruptedException e)\n\t\t\t{\n\t\t\t\t//do nothing\n\t\t\t}\n\n\t\t}\n\t\twhile (!alldone);\n\t}\n}\n\n/**\n * A thread for executing transactions or data inserts to the database.\n * \n * @author cooperb\n *\n */\nclass ClientThread extends Thread\n{\n\tDB _db;\n\tboolean _dotransactions;\n\tWorkload _workload;\n\tint _opcount;\n\tdouble _target;\n\n\tint _opsdone;\n\tint _threadid;\n\tint _threadcount;\n\tObject _workloadstate;\n\tProperties _props;\n\n\n\t/**\n\t * Constructor.\n\t * \n\t * @param db the DB implementation to use\n\t * @param dotransactions true to do transactions, false to insert data\n\t * @param workload the workload to use\n\t * @param threadid the id of this thread \n\t * @param threadcount the total number of threads \n\t * @param props the properties defining the experiment\n\t * @param opcount the number of operations (transactions or inserts) to do\n\t * @param targetperthreadperms target number of operations per thread per ms\n\t */\n\tpublic ClientThread(DB db, boolean dotransactions, Workload workload, int threadid, int threadcount, Properties props, int opcount, double targetperthreadperms)\n\t{\n\t\t//TODO: consider removing threadcount and threadid\n\t\t_db=db;\n\t\t_dotransactions=dotransactions;\n\t\t_workload=workload;\n\t\t_opcount=opcount;\n\t\t_opsdone=0;\n\t\t_target=targetperthreadperms;\n\t\t_threadid=threadid;\n\t\t_threadcount=threadcount;\n\t\t_props=props;\n\t\t//System.out.println(\"Interval = \"+interval);\n\t}\n\n\tpublic int getOpsDone()\n\t{\n\t\treturn _opsdone;\n\t}\n\n\tpublic void run()\n\t{\n\t\ttry\n\t\t{\n\t\t\t_db.init();\n\t\t}\n\t\tcatch (DBException e)\n\t\t{\n\t\t\te.printStackTrace();\n\t\t\te.printStackTrace(System.out);\n\t\t\treturn;\n\t\t}\n\n\t\ttry\n\t\t{\n\t\t\t_workloadstate=_workload.initThread(_props,_threadid,_threadcount);\n\t\t}\n\t\tcatch (WorkloadException e)\n\t\t{\n\t\t\te.printStackTrace();\n\t\t\te.printStackTrace(System.out);\n\t\t\treturn;\n\t\t}\n\n\t\t//spread the thread operations out so they don't all hit the DB at the same time\n\t\ttry\n\t\t{\n\t\t   //GH issue 4 - throws exception if _target>1 because random.nextInt argument must be >0\n\t\t   //and the sleep() doesn't make sense for granularities < 1 ms anyway\n\t\t   if ( (_target>0) && (_target<=1.0) ) \n\t\t   {\n\t\t      sleep(Utils.random().nextInt((int)(1.0/_target)));\n\t\t   }\n\t\t}\n\t\tcatch (InterruptedException e)\n\t\t{\n\t\t  // do nothing.\n\t\t}\n\t\t\n\t\ttry\n\t\t{\n\t\t\tif (_dotransactions)\n\t\t\t{\n\t\t\t\tlong st=System.currentTimeMillis();\n\n\t\t\t\twhile (((_opcount == 0) || (_opsdone < _opcount)) && !_workload.isStopRequested())\n\t\t\t\t{\n\n\t\t\t\t\tif (!_workload.doTransaction(_db,_workloadstate))\n\t\t\t\t\t{\n\t\t\t\t\t\tbreak;\n\t\t\t\t\t}\n\n\t\t\t\t\t_opsdone++;\n\n\t\t\t\t\t//throttle the operations\n\t\t\t\t\tif (_target>0)\n\t\t\t\t\t{\n\t\t\t\t\t\t//this is more accurate than other throttling approaches we have tried,\n\t\t\t\t\t\t//like sleeping for (1/target throughput)-operation latency,\n\t\t\t\t\t\t//because it smooths timing inaccuracies (from sleep() taking an int, \n\t\t\t\t\t\t//current time in millis) over many operations\n\t\t\t\t\t\twhile (System.currentTimeMillis()-st<((double)_opsdone)/_target)\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\ttry\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tsleep(1);\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tcatch (InterruptedException e)\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t  // do nothing.\n\t\t\t\t\t\t\t}\n\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\tlong st=System.currentTimeMillis();\n\n\t\t\t\twhile (((_opcount == 0) || (_opsdone < _opcount)) && !_workload.isStopRequested())\n\t\t\t\t{\n\n\t\t\t\t\tif (!_workload.doInsert(_db,_workloadstate))\n\t\t\t\t\t{\n\t\t\t\t\t\tbreak;\n\t\t\t\t\t}\n\n\t\t\t\t\t_opsdone++;\n\n\t\t\t\t\t//throttle the operations\n\t\t\t\t\tif (_target>0)\n\t\t\t\t\t{\n\t\t\t\t\t\t//this is more accurate than other throttling approaches we have tried,\n\t\t\t\t\t\t//like sleeping for (1/target throughput)-operation latency,\n\t\t\t\t\t\t//because it smooths timing inaccuracies (from sleep() taking an int, \n\t\t\t\t\t\t//current time in millis) over many operations\n\t\t\t\t\t\twhile (System.currentTimeMillis()-st<((double)_opsdone)/_target)\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\ttry \n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\tsleep(1);\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tcatch (InterruptedException e)\n\t\t\t\t\t\t\t{\n\t\t\t\t\t\t\t  // do nothing.\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tcatch (Exception e)\n\t\t{\n\t\t\te.printStackTrace();\n\t\t\te.printStackTrace(System.out);\n\t\t\tSystem.exit(0);\n\t\t}\n\n\t\ttry\n\t\t{\n\t\t\t_db.cleanup();\n\t\t}\n\t\tcatch (DBException e)\n\t\t{\n\t\t\te.printStackTrace();\n\t\t\te.printStackTrace(System.out);\n\t\t\treturn;\n\t\t}\n\t}\n}\n\n/**\n * Main class for executing YCSB.\n */\npublic class Client\n{\n\n\tpublic static final String OPERATION_COUNT_PROPERTY=\"operationcount\";\n\n\tpublic static final String RECORD_COUNT_PROPERTY=\"recordcount\";\n\n\tpublic static final String WORKLOAD_PROPERTY=\"workload\";\n\t\n\t/**\n\t * Indicates how many inserts to do, if less than recordcount. Useful for partitioning\n\t * the load among multiple servers, if the client is the bottleneck. Additionally, workloads\n\t * should support the \"insertstart\" property, which tells them which record to start at.\n\t */\n\tpublic static final String INSERT_COUNT_PROPERTY=\"insertcount\";\n\t\n\t/**\n   * The maximum amount of time (in seconds) for which the benchmark will be run.\n   */\n  public static final String MAX_EXECUTION_TIME = \"maxexecutiontime\";\n\n\tpublic static void usageMessage()\n\t{\n\t\tSystem.out.println(\"Usage: java com.yahoo.ycsb.Client [options]\");\n\t\tSystem.out.println(\"Options:\");\n\t\tSystem.out.println(\"  -threads n: execute using n threads (default: 1) - can also be specified as the \\n\" +\n\t\t\t\t\"              \\\"threadcount\\\" property using -p\");\n\t\tSystem.out.println(\"  -target n: attempt to do n operations per second (default: unlimited) - can also\\n\" +\n\t\t\t\t\"             be specified as the \\\"target\\\" property using -p\");\n\t\tSystem.out.println(\"  -load:  run the loading phase of the workload\");\n\t\tSystem.out.println(\"  -t:  run the transactions phase of the workload (default)\");\n\t\tSystem.out.println(\"  -db dbname: specify the name of the DB to use (default: com.yahoo.ycsb.BasicDB) - \\n\" +\n\t\t\t\t\"              can also be specified as the \\\"db\\\" property using -p\");\n\t\tSystem.out.println(\"  -P propertyfile: load properties from the given file. Multiple files can\");\n\t\tSystem.out.println(\"                   be specified, and will be processed in the order specified\");\n\t\tSystem.out.println(\"  -p name=value:  specify a property to be passed to the DB and workloads;\");\n\t\tSystem.out.println(\"                  multiple properties can be specified, and override any\");\n\t\tSystem.out.println(\"                  values in the propertyfile\");\n\t\tSystem.out.println(\"  -s:  show status during run (default: no status)\");\n\t\tSystem.out.println(\"  -l label:  use label for status (e.g. to label one experiment out of a whole batch)\");\n\t\tSystem.out.println(\"\");\n\t\tSystem.out.println(\"Required properties:\");\n\t\tSystem.out.println(\"  \"+WORKLOAD_PROPERTY+\": the name of the workload class to use (e.g. com.yahoo.ycsb.workloads.CoreWorkload)\");\n\t\tSystem.out.println(\"\");\n\t\tSystem.out.println(\"To run the transaction phase from multiple servers, start a separate client on each.\");\n\t\tSystem.out.println(\"To run the load phase from multiple servers, start a separate client on each; additionally,\");\n\t\tSystem.out.println(\"use the \\\"insertcount\\\" and \\\"insertstart\\\" properties to divide up the records to be inserted\");\n\t}\n\n\tpublic static boolean checkRequiredProperties(Properties props)\n\t{\n\t\tif (props.getProperty(WORKLOAD_PROPERTY)==null)\n\t\t{\n\t\t\tSystem.out.println(\"Missing property: \"+WORKLOAD_PROPERTY);\n\t\t\treturn false;\n\t\t}\n\n\t\treturn true;\n\t}\n\n\n\t/**\n\t * Exports the measurements to either sysout or a file using the exporter\n\t * loaded from conf.\n\t * @throws IOException Either failed to write to output stream or failed to close it.\n\t */\n\tprivate static void exportMeasurements(Properties props, int opcount, long runtime)\n\t\t\tthrows IOException\n\t{\n\t\tMeasurementsExporter exporter = null;\n\t\ttry\n\t\t{\n\t\t\t// if no destination file is provided the results will be written to stdout\n\t\t\tOutputStream out;\n\t\t\tString exportFile = props.getProperty(\"exportfile\");\n\t\t\tif (exportFile == null)\n\t\t\t{\n\t\t\t\tout = System.out;\n\t\t\t} else\n\t\t\t{\n\t\t\t\tout = new FileOutputStream(exportFile);\n\t\t\t}\n\n\t\t\t// if no exporter is provided the default text one will be used\n\t\t\tString exporterStr = props.getProperty(\"exporter\", \"com.yahoo.ycsb.measurements.exporter.TextMeasurementsExporter\");\n\t\t\ttry\n\t\t\t{\n\t\t\t\texporter = (MeasurementsExporter) Class.forName(exporterStr).getConstructor(OutputStream.class).newInstance(out);\n\t\t\t} catch (Exception e)\n\t\t\t{\n\t\t\t\tSystem.err.println(\"Could not find exporter \" + exporterStr\n\t\t\t\t\t\t+ \", will use default text reporter.\");\n\t\t\t\te.printStackTrace();\n\t\t\t\texporter = new TextMeasurementsExporter(out);\n\t\t\t}\n\n\t\t\texporter.write(\"OVERALL\", \"RunTime(ms)\", runtime);\n\t\t\tdouble throughput = 1000.0 * ((double) opcount) / ((double) runtime);\n\t\t\texporter.write(\"OVERALL\", \"Throughput(ops/sec)\", throughput);\n\n\t\t\tMeasurements.getMeasurements().exportMeasurements(exporter);\n\t\t} finally\n\t\t{\n\t\t\tif (exporter != null)\n\t\t\t{\n\t\t\t\texporter.close();\n\t\t\t}\n\t\t}\n\t}\n\t\n\t@SuppressWarnings(\"unchecked\")\n\tpublic static void main(String[] args)\n\t{\n\t\tString dbname;\n\t\tProperties props=new Properties();\n\t\tProperties fileprops=new Properties();\n\t\tboolean dotransactions=true;\n\t\tint threadcount=1;\n\t\tint target=0;\n\t\tboolean status=false;\n\t\tString label=\"\";\n\n\t\t//parse arguments\n\t\tint argindex=0;\n\n\t\tif (args.length==0)\n\t\t{\n\t\t\tusageMessage();\n\t\t\tSystem.exit(0);\n\t\t}\n\n\t\twhile (args[argindex].startsWith(\"-\"))\n\t\t{\n\t\t\tif (args[argindex].compareTo(\"-threads\")==0)\n\t\t\t{\n\t\t\t\targindex++;\n\t\t\t\tif (argindex>=args.length)\n\t\t\t\t{\n\t\t\t\t\tusageMessage();\n\t\t\t\t\tSystem.exit(0);\n\t\t\t\t}\n\t\t\t\tint tcount=Integer.parseInt(args[argindex]);\n\t\t\t\tprops.setProperty(\"threadcount\", tcount+\"\");\n\t\t\t\targindex++;\n\t\t\t}\n\t\t\telse if (args[argindex].compareTo(\"-target\")==0)\n\t\t\t{\n\t\t\t\targindex++;\n\t\t\t\tif (argindex>=args.length)\n\t\t\t\t{\n\t\t\t\t\tusageMessage();\n\t\t\t\t\tSystem.exit(0);\n\t\t\t\t}\n\t\t\t\tint ttarget=Integer.parseInt(args[argindex]);\n\t\t\t\tprops.setProperty(\"target\", ttarget+\"\");\n\t\t\t\targindex++;\n\t\t\t}\n\t\t\telse if (args[argindex].compareTo(\"-load\")==0)\n\t\t\t{\n\t\t\t\tdotransactions=false;\n\t\t\t\targindex++;\n\t\t\t}\n\t\t\telse if (args[argindex].compareTo(\"-t\")==0)\n\t\t\t{\n\t\t\t\tdotransactions=true;\n\t\t\t\targindex++;\n\t\t\t}\n\t\t\telse if (args[argindex].compareTo(\"-s\")==0)\n\t\t\t{\n\t\t\t\tstatus=true;\n\t\t\t\targindex++;\n\t\t\t}\n\t\t\telse if (args[argindex].compareTo(\"-db\")==0)\n\t\t\t{\n\t\t\t\targindex++;\n\t\t\t\tif (argindex>=args.length)\n\t\t\t\t{\n\t\t\t\t\tusageMessage();\n\t\t\t\t\tSystem.exit(0);\n\t\t\t\t}\n\t\t\t\tprops.setProperty(\"db\",args[argindex]);\n\t\t\t\targindex++;\n\t\t\t}\n\t\t\telse if (args[argindex].compareTo(\"-l\")==0)\n\t\t\t{\n\t\t\t\targindex++;\n\t\t\t\tif (argindex>=args.length)\n\t\t\t\t{\n\t\t\t\t\tusageMessage();\n\t\t\t\t\tSystem.exit(0);\n\t\t\t\t}\n\t\t\t\tlabel=args[argindex];\n\t\t\t\targindex++;\n\t\t\t}\n\t\t\telse if (args[argindex].compareTo(\"-P\")==0)\n\t\t\t{\n\t\t\t\targindex++;\n\t\t\t\tif (argindex>=args.length)\n\t\t\t\t{\n\t\t\t\t\tusageMessage();\n\t\t\t\t\tSystem.exit(0);\n\t\t\t\t}\n\t\t\t\tString propfile=args[argindex];\n\t\t\t\targindex++;\n\n\t\t\t\tProperties myfileprops=new Properties();\n\t\t\t\ttry\n\t\t\t\t{\n\t\t\t\t\tmyfileprops.load(new FileInputStream(propfile));\n\t\t\t\t}\n\t\t\t\tcatch (IOException e)\n\t\t\t\t{\n\t\t\t\t\tSystem.out.println(e.getMessage());\n\t\t\t\t\tSystem.exit(0);\n\t\t\t\t}\n\n\t\t\t\t//Issue #5 - remove call to stringPropertyNames to make compilable under Java 1.5\n\t\t\t\tfor (Enumeration e=myfileprops.propertyNames(); e.hasMoreElements(); )\n\t\t\t\t{\n\t\t\t\t   String prop=(String)e.nextElement();\n\t\t\t\t   \n\t\t\t\t   fileprops.setProperty(prop,myfileprops.getProperty(prop));\n\t\t\t\t}\n\n\t\t\t}\n\t\t\telse if (args[argindex].compareTo(\"-p\")==0)\n\t\t\t{\n\t\t\t\targindex++;\n\t\t\t\tif (argindex>=args.length)\n\t\t\t\t{\n\t\t\t\t\tusageMessage();\n\t\t\t\t\tSystem.exit(0);\n\t\t\t\t}\n\t\t\t\tint eq=args[argindex].indexOf('=');\n\t\t\t\tif (eq<0)\n\t\t\t\t{\n\t\t\t\t\tusageMessage();\n\t\t\t\t\tSystem.exit(0);\n\t\t\t\t}\n\n\t\t\t\tString name=args[argindex].substring(0,eq);\n\t\t\t\tString value=args[argindex].substring(eq+1);\n\t\t\t\tprops.put(name,value);\n\t\t\t\t//System.out.println(\"[\"+name+\"]=[\"+value+\"]\");\n\t\t\t\targindex++;\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\tSystem.out.println(\"Unknown option \"+args[argindex]);\n\t\t\t\tusageMessage();\n\t\t\t\tSystem.exit(0);\n\t\t\t}\n\n\t\t\tif (argindex>=args.length)\n\t\t\t{\n\t\t\t\tbreak;\n\t\t\t}\n\t\t}\n\n\t\tif (argindex!=args.length)\n\t\t{\n\t\t\tusageMessage();\n\t\t\tSystem.exit(0);\n\t\t}\n\n\t\t//set up logging\n\t\t//BasicConfigurator.configure();\n\n\t\t//overwrite file properties with properties from the command line\n\n\t\t//Issue #5 - remove call to stringPropertyNames to make compilable under Java 1.5\n\t\tfor (Enumeration e=props.propertyNames(); e.hasMoreElements(); )\n\t\t{\n\t\t   String prop=(String)e.nextElement();\n\t\t   \n\t\t   fileprops.setProperty(prop,props.getProperty(prop));\n\t\t}\n\n\t\tprops=fileprops;\n\n\t\tif (!checkRequiredProperties(props))\n\t\t{\n\t\t\tSystem.exit(0);\n\t\t}\n\t\t\n\t\tlong maxExecutionTime = Integer.parseInt(props.getProperty(MAX_EXECUTION_TIME, \"0\"));\n\n\t\t//get number of threads, target and db\n\t\tthreadcount=Integer.parseInt(props.getProperty(\"threadcount\",\"1\"));\n\t\tdbname=props.getProperty(\"db\",\"com.yahoo.ycsb.BasicDB\");\n\t\ttarget=Integer.parseInt(props.getProperty(\"target\",\"0\"));\n\t\t\n\t\t//compute the target throughput\n\t\tdouble targetperthreadperms=-1;\n\t\tif (target>0)\n\t\t{\n\t\t\tdouble targetperthread=((double)target)/((double)threadcount);\n\t\t\ttargetperthreadperms=targetperthread/1000.0;\n\t\t}\t \n\n\t\tSystem.out.println(\"YCSB Client 0.1\");\n\t\tSystem.out.print(\"Command line:\");\n\t\tfor (int i=0; i<args.length; i++)\n\t\t{\n\t\t\tSystem.out.print(\" \"+args[i]);\n\t\t}\n\t\tSystem.out.println();\n\t\tSystem.err.println(\"Loading workload...\");\n\t\t\n\t\t//show a warning message that creating the workload is taking a while\n\t\t//but only do so if it is taking longer than 2 seconds \n\t\t//(showing the message right away if the setup wasn't taking very long was confusing people)\n\t\tThread warningthread=new Thread() \n\t\t{\n\t\t\tpublic void run()\n\t\t\t{\n\t\t\t\ttry\n\t\t\t\t{\n\t\t\t\t\tsleep(2000);\n\t\t\t\t}\n\t\t\t\tcatch (InterruptedException e)\n\t\t\t\t{\n\t\t\t\t\treturn;\n\t\t\t\t}\n\t\t\t\tSystem.err.println(\" (might take a few minutes for large data sets)\");\n\t\t\t}\n\t\t};\n\n\t\twarningthread.start();\n\t\t\n\t\t//set up measurements\n\t\tMeasurements.setProperties(props);\n\t\t\n\t\t//load the workload\n\t\tClassLoader classLoader = Client.class.getClassLoader();\n\n\t\tWorkload workload=null;\n\n\t\ttry \n\t\t{\n\t\t\tClass workloadclass = classLoader.loadClass(props.getProperty(WORKLOAD_PROPERTY));\n\n\t\t\tworkload=(Workload)workloadclass.newInstance();\n\t\t}\n\t\tcatch (Exception e) \n\t\t{  \n\t\t\te.printStackTrace();\n\t\t\te.printStackTrace(System.out);\n\t\t\tSystem.exit(0);\n\t\t}\n\n\t\ttry\n\t\t{\n\t\t\tworkload.init(props);\n\t\t}\n\t\tcatch (WorkloadException e)\n\t\t{\n\t\t\te.printStackTrace();\n\t\t\te.printStackTrace(System.out);\n\t\t\tSystem.exit(0);\n\t\t}\n\t\t\n\t\twarningthread.interrupt();\n\n\t\t//run the workload\n\n\t\tSystem.err.println(\"Starting test.\");\n\n\t\tint opcount;\n\t\tif (dotransactions)\n\t\t{\n\t\t\topcount=Integer.parseInt(props.getProperty(OPERATION_COUNT_PROPERTY,\"0\"));\n\t\t}\n\t\telse\n\t\t{\n\t\t\tif (props.containsKey(INSERT_COUNT_PROPERTY))\n\t\t\t{\n\t\t\t\topcount=Integer.parseInt(props.getProperty(INSERT_COUNT_PROPERTY,\"0\"));\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\topcount=Integer.parseInt(props.getProperty(RECORD_COUNT_PROPERTY,\"0\"));\n\t\t\t}\n\t\t}\n\n\t\tVector<Thread> threads=new Vector<Thread>();\n\n\t\tfor (int threadid=0; threadid<threadcount; threadid++)\n\t\t{\n\t\t\tDB db=null;\n\t\t\ttry\n\t\t\t{\n\t\t\t\tdb=DBFactory.newDB(dbname,props);\n\t\t\t}\n\t\t\tcatch (UnknownDBException e)\n\t\t\t{\n\t\t\t\tSystem.out.println(\"Unknown DB \"+dbname);\n\t\t\t\tSystem.exit(0);\n\t\t\t}\n\n\t\t\tThread t=new ClientThread(db,dotransactions,workload,threadid,threadcount,props,opcount/threadcount,targetperthreadperms);\n\n\t\t\tthreads.add(t);\n\t\t\t//t.start();\n\t\t}\n\n\t\tStatusThread statusthread=null;\n\n\t\tif (status)\n\t\t{\n\t\t\tboolean standardstatus=false;\n\t\t\tif (props.getProperty(\"measurementtype\",\"\").compareTo(\"timeseries\")==0) \n\t\t\t{\n\t\t\t\tstandardstatus=true;\n\t\t\t}\t\n\t\t\tstatusthread=new StatusThread(threads,label,standardstatus);\n\t\t\tstatusthread.start();\n\t\t}\n\n\t\tlong st=System.currentTimeMillis();\n\n\t\tfor (Thread t : threads)\n\t\t{\n\t\t\tt.start();\n\t\t}\n\t\t\n    Thread terminator = null;\n    \n    if (maxExecutionTime > 0) {\n      terminator = new TerminatorThread(maxExecutionTime, threads, workload);\n      terminator.start();\n    }\n    \n    int opsDone = 0;\n\n\t\tfor (Thread t : threads)\n\t\t{\n\t\t\ttry\n\t\t\t{\n\t\t\t\tt.join();\n\t\t\t\topsDone += ((ClientThread)t).getOpsDone();\n\t\t\t}\n\t\t\tcatch (InterruptedException e)\n\t\t\t{\n\t\t\t}\n\t\t}\n\n\t\tlong en=System.currentTimeMillis();\n\t\t\n\t\tif (terminator != null && !terminator.isInterrupted()) {\n      terminator.interrupt();\n    }\n\n\t\tif (status)\n\t\t{\n\t\t\tstatusthread.interrupt();\n\t\t}\n\n\t\ttry\n\t\t{\n\t\t\tworkload.cleanup();\n\t\t}\n\t\tcatch (WorkloadException e)\n\t\t{\n\t\t\te.printStackTrace();\n\t\t\te.printStackTrace(System.out);\n\t\t\tSystem.exit(0);\n\t\t}\n\n\t\ttry\n\t\t{\n\t\t\texportMeasurements(props, opsDone, en - st);\n\t\t} catch (IOException e)\n\t\t{\n\t\t\tSystem.err.println(\"Could not export measurements, error: \" + e.getMessage());\n\t\t\te.printStackTrace();\n\t\t\tSystem.exit(-1);\n\t\t}\n\n\t\tSystem.exit(0);\n\t}\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/CommandLine.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb;\n\nimport java.util.Properties;\nimport java.io.FileInputStream;\nimport java.io.IOException;\nimport java.util.Enumeration;\nimport java.io.BufferedReader;\nimport java.io.InputStreamReader;\nimport java.util.HashMap;\nimport java.util.Map;\nimport java.util.Set;\nimport java.util.HashSet;\nimport java.util.Vector;\n\nimport com.yahoo.ycsb.workloads.*;\n\n/**\n * A simple command line client to a database, using the appropriate com.yahoo.ycsb.DB implementation.\n */\npublic class CommandLine\n{\n      public static final String DEFAULT_DB=\"com.yahoo.ycsb.BasicDB\";\n\n      public static void usageMessage()\n      {\n\t System.out.println(\"YCSB Command Line Client\");\n\t System.out.println(\"Usage: java com.yahoo.ycsb.CommandLine [options]\");\n\t System.out.println(\"Options:\");\n\t System.out.println(\"  -P filename: Specify a property file\");\n\t System.out.println(\"  -p name=value: Specify a property value\");\n\t System.out.println(\"  -db classname: Use a specified DB class (can also set the \\\"db\\\" property)\");\n\t System.out.println(\"  -table tablename: Use the table name instead of the default \\\"\"+CoreWorkload.TABLENAME_PROPERTY_DEFAULT+\"\\\"\");\n\t System.out.println();\n      }\n\n      public static void help()\n      {\n\t System.out.println(\"Commands:\");\n\t System.out.println(\"  read key [field1 field2 ...] - Read a record\");\n\t System.out.println(\"  scan key recordcount [field1 field2 ...] - Scan starting at key\");\n\t System.out.println(\"  insert key name1=value1 [name2=value2 ...] - Insert a new record\");\n\t System.out.println(\"  update key name1=value1 [name2=value2 ...] - Update a record\");\n\t System.out.println(\"  delete key - Delete a record\");\n\t System.out.println(\"  table [tablename] - Get or [set] the name of the table\");\n\t System.out.println(\"  quit - Quit\");\n      }\n      \n      public static void main(String[] args)\n      {\n\t int argindex=0;\n\n\t Properties props=new Properties();\n\t Properties fileprops=new Properties();\n\t String table=CoreWorkload.TABLENAME_PROPERTY_DEFAULT;\n\n\t while ( (argindex<args.length) && (args[argindex].startsWith(\"-\")) )\n\t {\n\t    if ( (args[argindex].compareTo(\"-help\")==0) ||\n\t\t (args[argindex].compareTo(\"--help\")==0) ||\n\t\t (args[argindex].compareTo(\"-?\")==0) ||\n\t\t (args[argindex].compareTo(\"--?\")==0) )\n\t    {\n\t       usageMessage();\n\t       System.exit(0);\n\t    }\n\n\t    if (args[argindex].compareTo(\"-db\")==0)\n\t    {\n\t       argindex++;\n\t       if (argindex>=args.length)\n\t       {\n\t\t  usageMessage();\n\t\t  System.exit(0);\n\t       }\n\t       props.setProperty(\"db\",args[argindex]);\n\t       argindex++;\n\t    }\n\t    else if (args[argindex].compareTo(\"-P\")==0)\n\t    {\n\t       argindex++;\n\t       if (argindex>=args.length)\n\t       {\n\t\t  usageMessage();\n\t\t  System.exit(0);\n\t       }\n\t       String propfile=args[argindex];\n\t       argindex++;\n\t       \n\t       Properties myfileprops=new Properties();\n\t       try\n\t       {\n\t\t  myfileprops.load(new FileInputStream(propfile));\n\t       }\n\t       catch (IOException e)\n\t       {\n\t\t  System.out.println(e.getMessage());\n\t\t  System.exit(0);\n\t       }\n\t       \n\t       for (Enumeration e=myfileprops.propertyNames(); e.hasMoreElements(); )\n\t       {\n\t\t  String prop=(String)e.nextElement();\n\t\t  \n\t\t  fileprops.setProperty(prop,myfileprops.getProperty(prop));\n\t       }\n\t       \n\t    }\n\t    else if (args[argindex].compareTo(\"-p\")==0)\n\t    {\n\t       argindex++;\n\t       if (argindex>=args.length)\n\t       {\n\t\t  usageMessage();\n\t\t  System.exit(0);\n\t       }\n\t       int eq=args[argindex].indexOf('=');\n\t       if (eq<0)\n\t       {\n\t\t  usageMessage();\n\t\t  System.exit(0);\n\t       }\n\t\t\t   \n\t       String name=args[argindex].substring(0,eq);\n\t       String value=args[argindex].substring(eq+1);\n\t       props.put(name,value);\n\t       //System.out.println(\"[\"+name+\"]=[\"+value+\"]\");\n\t       argindex++;\n\t    }\n\t    else if (args[argindex].compareTo(\"-table\")==0)\n\t    {\n\t       argindex++;\n\t       if (argindex>=args.length)\n\t       {\n\t\t  usageMessage();\n\t\t  System.exit(0);\n\t       }\n\t       table=args[argindex];\n\t       argindex++;\n\t    }  \n\t    else\n\t    {\n\t       System.out.println(\"Unknown option \"+args[argindex]);\n\t       usageMessage();\n\t       System.exit(0);\n\t    }\n\n\t    if (argindex>=args.length)\n\t    {\n\t       break;\n\t    }\n\t }\n\n\t if (argindex!=args.length)\n\t {\n\t    usageMessage();\n\t    System.exit(0);\n\t }\n\n\t for (Enumeration e=props.propertyNames(); e.hasMoreElements(); )\n\t {\n\t    String prop=(String)e.nextElement();\n\t    \n\t    fileprops.setProperty(prop,props.getProperty(prop));\n\t }\n\t \n\t props=fileprops;\n\n\t System.out.println(\"YCSB Command Line client\");\n\t System.out.println(\"Type \\\"help\\\" for command line help\");\n\t System.out.println(\"Start with \\\"-help\\\" for usage info\");\n\n\t //create a DB\n\t String dbname=props.getProperty(\"db\",DEFAULT_DB);\n\n\t ClassLoader classLoader = CommandLine.class.getClassLoader();\n\n\t DB db=null;\n\n\t try \n\t {\n\t    Class dbclass = classLoader.loadClass(dbname);\n\t    db=(DB)dbclass.newInstance();\n\t }\n\t catch (Exception e) \n\t {  \n\t    e.printStackTrace();\n\t    System.exit(0);\n\t }\n\t \n\t db.setProperties(props);\n\t try\n\t {\n\t    db.init();\n\t }\n\t catch (DBException e)\n\t {\n\t    e.printStackTrace();\n\t    System.exit(0);\n\t }\n\n\t System.out.println(\"Connected.\");\n\t \n\t //main loop\n\t BufferedReader br = new BufferedReader(new InputStreamReader(System.in));\n\n\t for (;;)\n\t {\n\t    //get user input\n\t    System.out.print(\"> \");\n\t    \n\t    String input=null;\n\t    \n\t    try\n\t    {\n\t       input=br.readLine();\n\t    }\n\t    catch (IOException e)\n\t    {\n\t       e.printStackTrace();\n\t       System.exit(1);\n\t    }\n\n\t    if (input.compareTo(\"\")==0) \n\t    {\n\t       continue;\n\t    }\n\n\t    if (input.compareTo(\"help\")==0) \n\t    {\n\t       help();\n\t       continue;\n\t    }\n\n\t    if (input.compareTo(\"quit\")==0)\n\t    {\n\t       break;\n\t    }\n\t    \n\t    String[] tokens=input.split(\" \");\n\t    \n\t    long st=System.currentTimeMillis();\n\t    //handle commands\n\t    if (tokens[0].compareTo(\"table\")==0)\n\t    {\n\t       if (tokens.length==1)\n\t       {\n\t\t  System.out.println(\"Using table \\\"\"+table+\"\\\"\");\n\t       }\n\t       else if (tokens.length==2)\n\t       {\n\t\t  table=tokens[1];\n\t\t  System.out.println(\"Using table \\\"\"+table+\"\\\"\");\n\t       }\n\t       else\n\t       {\n\t\t  System.out.println(\"Error: syntax is \\\"table tablename\\\"\");\n\t       }\n\t    }\n\t    else if (tokens[0].compareTo(\"read\")==0)\n\t    {\n\t       if (tokens.length==1)\n\t       {\n\t\t  System.out.println(\"Error: syntax is \\\"read keyname [field1 field2 ...]\\\"\");\n\t       }\n\t       else \n\t       {\n\t\t  Set<String> fields=null;\n\n\t\t  if (tokens.length>2)\n\t\t  {\n\t\t     fields=new HashSet<String>();\n\t\t     \n\t\t     for (int i=2; i<tokens.length; i++)\n\t\t     {\n\t\t\tfields.add(tokens[i]);\n\t\t     }\n\t\t  }\n\t\t  \n\t\t  HashMap<String,ByteIterator> result=new HashMap<String,ByteIterator>();\n\t\t  int ret=db.read(table,tokens[1],fields,result);\n\t\t  System.out.println(\"Return code: \"+ret);\n\t\t  for (Map.Entry<String,ByteIterator> ent : result.entrySet())\n\t\t  {\n\t\t     System.out.println(ent.getKey()+\"=\"+ent.getValue());\n\t\t  }\n\t       }\t\t  \n\t    }\n\t    else if (tokens[0].compareTo(\"scan\")==0)\n\t    {\n\t       if (tokens.length<3)\n\t       {\n\t\t  System.out.println(\"Error: syntax is \\\"scan keyname scanlength [field1 field2 ...]\\\"\");\n\t       }\n\t       else \n\t       {\n\t\t  Set<String> fields=null;\n\n\t\t  if (tokens.length>3)\n\t\t  {\n\t\t     fields=new HashSet<String>();\n\t\t     \n\t\t     for (int i=3; i<tokens.length; i++)\n\t\t     {\n\t\t\tfields.add(tokens[i]);\n\t\t     }\n\t\t  }\n\t\t  \n\t\t  Vector<HashMap<String,ByteIterator>> results=new Vector<HashMap<String,ByteIterator>>();\n\t\t  int ret=db.scan(table,tokens[1],Integer.parseInt(tokens[2]),fields,results);\n\t\t  System.out.println(\"Return code: \"+ret);\n\t\t  int record=0;\n\t\t  if (results.size()==0)\n\t\t  {\n\t\t     System.out.println(\"0 records\");\n\t\t  }\n\t\t  else\n\t\t  {\n\t\t     System.out.println(\"--------------------------------\");\n\t\t  }\n\t\t  for (HashMap<String,ByteIterator> result : results)\n\t\t  {\n\t\t     System.out.println(\"Record \"+(record++));\n\t\t     for (Map.Entry<String,ByteIterator> ent : result.entrySet())\n\t\t     {\n\t\t\tSystem.out.println(ent.getKey()+\"=\"+ent.getValue());\n\t\t     }\n\t\t     System.out.println(\"--------------------------------\");\n\t\t  }\n\t       }\t\t  \n\t    }\n\t    else if (tokens[0].compareTo(\"update\")==0)\n\t    {\n\t       if (tokens.length<3)\n\t       {\n\t\t  System.out.println(\"Error: syntax is \\\"update keyname name1=value1 [name2=value2 ...]\\\"\");\n\t       }\n\t       else \n\t       {\n\t\t  HashMap<String,ByteIterator> values=new HashMap<String,ByteIterator>();\n\n\t\t  for (int i=2; i<tokens.length; i++)\n\t\t  {\n\t\t     String[] nv=tokens[i].split(\"=\");\n\t\t     values.put(nv[0],new StringByteIterator(nv[1]));\n\t\t  }\n\n\t\t  int ret=db.update(table,tokens[1],values);\n\t\t  System.out.println(\"Return code: \"+ret);\n\t       }\t\t  \n\t    }\n\t    else if (tokens[0].compareTo(\"insert\")==0)\n\t    {\n\t       if (tokens.length<3)\n\t       {\n\t\t  System.out.println(\"Error: syntax is \\\"insert keyname name1=value1 [name2=value2 ...]\\\"\");\n\t       }\n\t       else \n\t       {\n\t\t  HashMap<String,ByteIterator> values=new HashMap<String,ByteIterator>();\n\n\t\t  for (int i=2; i<tokens.length; i++)\n\t\t  {\n\t\t     String[] nv=tokens[i].split(\"=\");\n\t\t     values.put(nv[0],new StringByteIterator(nv[1]));\n\t\t  }\n\n\t\t  int ret=db.insert(table,tokens[1],values);\n\t\t  System.out.println(\"Return code: \"+ret);\n\t       }\t\t  \n\t    }\n\t    else if (tokens[0].compareTo(\"delete\")==0)\n\t    {\n\t       if (tokens.length!=2)\n\t       {\n\t\t  System.out.println(\"Error: syntax is \\\"delete keyname\\\"\");\n\t       }\n\t       else \n\t       {\n\t\t  int ret=db.delete(table,tokens[1]);\n\t\t  System.out.println(\"Return code: \"+ret);\n\t       }\t\t  \n\t    }\n\t    else\n\t    {\n\t       System.out.println(\"Error: unknown command \\\"\"+tokens[0]+\"\\\"\");\n\t    }\n\t    \n\t    System.out.println((System.currentTimeMillis()-st)+\" ms\");\n\t    \n\t }\n      }\n      \n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/DB.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb;\n\nimport java.util.HashMap;\nimport java.util.Properties;\nimport java.util.Set;\nimport java.util.Vector;\n\n/**\n * A layer for accessing a database to be benchmarked. Each thread in the client\n * will be given its own instance of whatever DB class is to be used in the test.\n * This class should be constructed using a no-argument constructor, so we can\n * load it dynamically. Any argument-based initialization should be\n * done by init().\n * \n * Note that YCSB does not make any use of the return codes returned by this class.\n * Instead, it keeps a count of the return values and presents them to the user.\n * \n * The semantics of methods such as insert, update and delete vary from database\n * to database.  In particular, operations may or may not be durable once these\n * methods commit, and some systems may return 'success' regardless of whether\n * or not a tuple with a matching key existed before the call.  Rather than dictate\n * the exact semantics of these methods, we recommend you either implement them\n * to match the database's default semantics, or the semantics of your \n * target application.  For the sake of comparison between experiments we also \n * recommend you explain the semantics you chose when presenting performance results.\n */\npublic abstract class DB\n{\n\t/**\n\t * Properties for configuring this DB.\n\t */\n\tProperties _p=new Properties();\n\n\t/**\n\t * Set the properties for this DB.\n\t */\n\tpublic void setProperties(Properties p)\n\t{\n\t\t_p=p;\n\n\t}\n\n\t/**\n\t * Get the set of properties for this DB.\n\t */\n\tpublic Properties getProperties()\n\t{\n\t\treturn _p; \n\t}\n\n\t/**\n\t * Initialize any state for this DB.\n\t * Called once per DB instance; there is one DB instance per client thread.\n\t */\n\tpublic void init() throws DBException\n\t{\n\t}\n\n\t/**\n\t * Cleanup any state for this DB.\n\t * Called once per DB instance; there is one DB instance per client thread.\n\t */\n\tpublic void cleanup() throws DBException\n\t{\n\t}\n\n\t/**\n\t * Read a record from the database. Each field/value pair from the result will be stored in a HashMap.\n\t *\n\t * @param table The name of the table\n\t * @param key The record key of the record to read.\n\t * @param fields The list of fields to read, or null for all of them\n\t * @param result A HashMap of field/value pairs for the result\n\t * @return Zero on success, a non-zero error code on error or \"not found\".\n\t */\n\tpublic abstract int read(String table, String key, Set<String> fields, HashMap<String,ByteIterator> result);\n\n\t/**\n\t * Perform a range scan for a set of records in the database. Each field/value pair from the result will be stored in a HashMap.\n\t *\n\t * @param table The name of the table\n\t * @param startkey The record key of the first record to read.\n\t * @param recordcount The number of records to read\n\t * @param fields The list of fields to read, or null for all of them\n\t * @param result A Vector of HashMaps, where each HashMap is a set field/value pairs for one record\n\t * @return Zero on success, a non-zero error code on error.  See this class's description for a discussion of error codes.\n\t */\n\tpublic abstract int scan(String table, String startkey, int recordcount, Set<String> fields, Vector<HashMap<String,ByteIterator>> result);\n\t\n\t/**\n\t * Update a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n\t * record key, overwriting any existing values with the same field name.\n\t *\n\t * @param table The name of the table\n\t * @param key The record key of the record to write.\n\t * @param values A HashMap of field/value pairs to update in the record\n\t * @return Zero on success, a non-zero error code on error.  See this class's description for a discussion of error codes.\n\t */\n\tpublic abstract int update(String table, String key, HashMap<String,ByteIterator> values);\n\n\t/**\n\t * Insert a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n\t * record key.\n\t *\n\t * @param table The name of the table\n\t * @param key The record key of the record to insert.\n\t * @param values A HashMap of field/value pairs to insert in the record\n\t * @return Zero on success, a non-zero error code on error.  See this class's description for a discussion of error codes.\n\t */\n\tpublic abstract int insert(String table, String key, HashMap<String,ByteIterator> values);\n\n\t/**\n\t * Delete a record from the database. \n\t *\n\t * @param table The name of the table\n\t * @param key The record key of the record to delete.\n\t * @return Zero on success, a non-zero error code on error.  See this class's description for a discussion of error codes.\n\t */\n\tpublic abstract int delete(String table, String key);\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/DBException.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb;\n\n/**\n * Something bad happened while interacting with the database.\n */\npublic class DBException extends Exception\n{\n      /**\n\t * \n\t */\n\tprivate static final long serialVersionUID = 6646883591588721475L;\n\n\tpublic DBException(String message) \n      {\n\t super(message);\n      }\n      \n      public DBException()\n      {\n\t super();\n      }\n\n      public DBException(String message, Throwable cause)\n      {\n\t super(message,cause);\n      }\n      \n      public DBException(Throwable cause)\n      {\n\t super(cause);\n      }\n      \n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/DBFactory.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb;\n\nimport java.util.Properties;\n\n/**\n * Creates a DB layer by dynamically classloading the specified DB class.\n */\npublic class DBFactory\n{\n      @SuppressWarnings(\"unchecked\")\n\tpublic static DB newDB(String dbname, Properties properties) throws UnknownDBException\n      {\n\t ClassLoader classLoader = DBFactory.class.getClassLoader();\n\n\t DB ret=null;\n\n\t try \n\t {\n\t    Class dbclass = classLoader.loadClass(dbname);\n\t    //System.out.println(\"dbclass.getName() = \" + dbclass.getName());\n\t    \n\t    ret=(DB)dbclass.newInstance();\n\t }\n\t catch (Exception e) \n\t {  \n\t    e.printStackTrace();\n\t    return null;\n\t }\n\t \n\t ret.setProperties(properties);\n\n\t return new DBWrapper(ret);\n      }\n      \n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/DBWrapper.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb;\n\nimport java.util.HashMap;\nimport java.util.Properties;\nimport java.util.Set;\nimport java.util.Vector;\n\nimport com.yahoo.ycsb.measurements.Measurements;\n\n/**\n * Wrapper around a \"real\" DB that measures latencies and counts return codes.\n */\npublic class DBWrapper extends DB\n{\n\tDB _db;\n\tMeasurements _measurements;\n\n\tpublic DBWrapper(DB db)\n\t{\n\t\t_db=db;\n\t\t_measurements=Measurements.getMeasurements();\n\t}\n\n\t/**\n\t * Set the properties for this DB.\n\t */\n\tpublic void setProperties(Properties p)\n\t{\n\t\t_db.setProperties(p);\n\t}\n\n\t/**\n\t * Get the set of properties for this DB.\n\t */\n\tpublic Properties getProperties()\n\t{\n\t\treturn _db.getProperties();\n\t}\n\n\t/**\n\t * Initialize any state for this DB.\n\t * Called once per DB instance; there is one DB instance per client thread.\n\t */\n\tpublic void init() throws DBException\n\t{\n\t\t_db.init();\n\t}\n\n\t/**\n\t * Cleanup any state for this DB.\n\t * Called once per DB instance; there is one DB instance per client thread.\n\t */\n\tpublic void cleanup() throws DBException\n\t{\n\t\t_db.cleanup();\n\t}\n\n\t/**\n\t * Read a record from the database. Each field/value pair from the result will be stored in a HashMap.\n\t *\n\t * @param table The name of the table\n\t * @param key The record key of the record to read.\n\t * @param fields The list of fields to read, or null for all of them\n\t * @param result A HashMap of field/value pairs for the result\n\t * @return Zero on success, a non-zero error code on error\n\t */\n\tpublic int read(String table, String key, Set<String> fields, HashMap<String,ByteIterator> result)\n\t{\n\t\tlong st=System.nanoTime();\n\t\tint res=_db.read(table,key,fields,result);\n\t\tlong en=System.nanoTime();\n\t\t_measurements.measure(\"READ\",(int)((en-st)/1000));\n\t\t_measurements.reportReturnCode(\"READ\",res);\n\t\treturn res;\n\t}\n\n\t/**\n\t * Perform a range scan for a set of records in the database. Each field/value pair from the result will be stored in a HashMap.\n\t *\n\t * @param table The name of the table\n\t * @param startkey The record key of the first record to read.\n\t * @param recordcount The number of records to read\n\t * @param fields The list of fields to read, or null for all of them\n\t * @param result A Vector of HashMaps, where each HashMap is a set field/value pairs for one record\n\t * @return Zero on success, a non-zero error code on error\n\t */\n\tpublic int scan(String table, String startkey, int recordcount, Set<String> fields, Vector<HashMap<String,ByteIterator>> result)\n\t{\n\t\tlong st=System.nanoTime();\n\t\tint res=_db.scan(table,startkey,recordcount,fields,result);\n\t\tlong en=System.nanoTime();\n\t\t_measurements.measure(\"SCAN\",(int)((en-st)/1000));\n\t\t_measurements.reportReturnCode(\"SCAN\",res);\n\t\treturn res;\n\t}\n\t\n\t/**\n\t * Update a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n\t * record key, overwriting any existing values with the same field name.\n\t *\n\t * @param table The name of the table\n\t * @param key The record key of the record to write.\n\t * @param values A HashMap of field/value pairs to update in the record\n\t * @return Zero on success, a non-zero error code on error\n\t */\n\tpublic int update(String table, String key, HashMap<String,ByteIterator> values)\n\t{\n\t\tlong st=System.nanoTime();\n\t\tint res=_db.update(table,key,values);\n\t\tlong en=System.nanoTime();\n\t\t_measurements.measure(\"UPDATE\",(int)((en-st)/1000));\n\t\t_measurements.reportReturnCode(\"UPDATE\",res);\n\t\treturn res;\n\t}\n\n\t/**\n\t * Insert a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n\t * record key.\n\t *\n\t * @param table The name of the table\n\t * @param key The record key of the record to insert.\n\t * @param values A HashMap of field/value pairs to insert in the record\n\t * @return Zero on success, a non-zero error code on error\n\t */\n\tpublic int insert(String table, String key, HashMap<String,ByteIterator> values)\n\t{\n\t\tlong st=System.nanoTime();\n\t\tint res=_db.insert(table,key,values);\n\t\tlong en=System.nanoTime();\n\t\t_measurements.measure(\"INSERT\",(int)((en-st)/1000));\n\t\t_measurements.reportReturnCode(\"INSERT\",res);\n\t\treturn res;\n\t}\n\n\t/**\n\t * Delete a record from the database. \n\t *\n\t * @param table The name of the table\n\t * @param key The record key of the record to delete.\n\t * @return Zero on success, a non-zero error code on error\n\t */\n\tpublic int delete(String table, String key)\n\t{\n\t\tlong st=System.nanoTime();\n\t\tint res=_db.delete(table,key);\n\t\tlong en=System.nanoTime();\n\t\t_measurements.measure(\"DELETE\",(int)((en-st)/1000));\n\t\t_measurements.reportReturnCode(\"DELETE\",res);\n\t\treturn res;\n\t}\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/InputStreamByteIterator.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\npackage com.yahoo.ycsb;\n\nimport java.io.InputStream;\n\npublic class InputStreamByteIterator extends ByteIterator {\n\tlong len;\n\tInputStream ins;\n\tlong off;\n\t\n\tpublic InputStreamByteIterator(InputStream ins, long len) {\n\t\tthis.len = len;\n\t\tthis.ins = ins;\n\t\toff = 0;\n\t}\n\t\n\t@Override\n\tpublic boolean hasNext() {\n\t\treturn off < len;\n\t}\n\n\t@Override\n\tpublic byte nextByte() {\n\t\tint ret;\n\t\ttry {\n\t\t\tret = ins.read();\n\t\t} catch(Exception e) {\n\t\t\tthrow new IllegalStateException(e);\n\t\t}\n\t\tif(ret == -1) { throw new IllegalStateException(\"Past EOF!\"); }\n\t\toff++;\n\t\treturn (byte)ret;\n\t}\n\n\t@Override\n\tpublic long bytesLeft() {\n\t\treturn len - off;\n\t}\n\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/RandomByteIterator.java",
    "content": "/**\n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\"); you\n * may not use this file except in compliance with the License. You\n * may obtain a copy of the License at\n *                                                              \n * http://www.apache.org/licenses/LICENSE-2.0\n *                                                            \n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n * implied. See the License for the specific language governing\n * permissions and limitations under the License. See accompanying\n * LICENSE file.\n */\npackage com.yahoo.ycsb;\n\n/**\n *  A ByteIterator that generates a random sequence of bytes.\n */\npublic class RandomByteIterator extends ByteIterator {\n  private long len;\n  private long off;\n  private int bufOff;\n  private byte[] buf;\n\n  @Override\n  public boolean hasNext() {\n    return (off + bufOff) < len;\n  }\n\n  private void fillBytesImpl(byte[] buffer, int base) {\n    int bytes = Utils.random().nextInt();\n    try {\n      buffer[base+0] = (byte)(((bytes) & 31) + ' ');\n      buffer[base+1] = (byte)(((bytes >> 5) & 31) + ' ');\n      buffer[base+2] = (byte)(((bytes >> 10) & 31) + ' ');\n      buffer[base+3] = (byte)(((bytes >> 15) & 31) + ' ');\n      buffer[base+4] = (byte)(((bytes >> 20) & 31) + ' ');\n      buffer[base+5] = (byte)(((bytes >> 25) & 31) + ' ');\n    } catch (ArrayIndexOutOfBoundsException e) { /* ignore it */ }\n  }\n\n  private void fillBytes() {\n    if(bufOff ==  buf.length) {\n      fillBytesImpl(buf, 0);\n      bufOff = 0;\n      off += buf.length;\n    }\n  }\n\n  public RandomByteIterator(long len) {\n    this.len = len;\n    this.buf = new byte[6];\n    this.bufOff = buf.length;\n    fillBytes();\n    this.off = 0;\n  }\n\n  public byte nextByte() {\n    fillBytes();\n    bufOff++;\n    return buf[bufOff-1];\n  }\n\n  @Override\n  public int nextBuf(byte[] buffer, int bufferOffset) {\n    int ret;\n    if(len - off < buffer.length - bufferOffset) {\n      ret = (int)(len - off);\n    } else {\n      ret = buffer.length - bufferOffset;\n    }\n    int i;\n    for(i = 0; i < ret; i+=6) {\n      fillBytesImpl(buffer, i + bufferOffset);\n    }\n    off+=ret;\n    return ret + bufferOffset;\n  }\n\n  @Override\n  public long bytesLeft() {\n    return len - off - bufOff;\n  }\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/StringByteIterator.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb;\n\nimport java.util.Map;\nimport java.util.HashMap;\n\npublic class StringByteIterator extends ByteIterator {\n\tString str;\n\tint off;\n\n\t/**\n\t * Put all of the entries of one map into the other, converting\n\t * String values into ByteIterators.\n\t */\n\tpublic static void putAllAsByteIterators(Map<String, ByteIterator> out, Map<String, String> in) {\n\t       for(String s: in.keySet()) { out.put(s, new StringByteIterator(in.get(s))); }\n\t} \n\n\t/**\n\t * Put all of the entries of one map into the other, converting\n\t * ByteIterator values into Strings.\n\t */\n\tpublic static void putAllAsStrings(Map<String, String> out, Map<String, ByteIterator> in) {\n\t       for(String s: in.keySet()) { out.put(s, in.get(s).toString()); }\n\t} \n\n\t/**\n\t * Create a copy of a map, converting the values from Strings to\n\t * StringByteIterators.\n\t */\n\tpublic static HashMap<String, ByteIterator> getByteIteratorMap(Map<String, String> m) {\n\t\tHashMap<String, ByteIterator> ret =\n\t\t\tnew HashMap<String,ByteIterator>();\n\n\t\tfor(String s: m.keySet()) {\n\t\t\tret.put(s, new StringByteIterator(m.get(s)));\n\t\t}\n\t\treturn ret;\n\t}\n\n\t/**\n\t * Create a copy of a map, converting the values from\n\t * StringByteIterators to Strings.\n\t */\n\tpublic static HashMap<String, String> getStringMap(Map<String, ByteIterator> m) {\n\t\tHashMap<String, String> ret = new HashMap<String,String>();\n\n\t\tfor(String s: m.keySet()) {\n\t\t\tret.put(s, m.get(s).toString());;\n\t\t}\n\t\treturn ret;\n\t}\n\n\tpublic StringByteIterator(String s) {\n\t\tthis.str = s;\n\t\tthis.off = 0;\n\t}\n\t@Override\n\tpublic boolean hasNext() {\n\t\treturn off < str.length();\n\t}\n\n\t@Override\n\tpublic byte nextByte() {\n\t\tbyte ret = (byte)str.charAt(off);\n\t\toff++;\n\t\treturn ret;\n\t}\n\n\t@Override\n\tpublic long bytesLeft() {\n\t\treturn str.length() - off;\n\t}\n\n\t/**\n\t * Specialization of general purpose toString() to avoid unnecessary\n\t * copies.\n\t * <p>\n\t * Creating a new StringByteIterator, then calling toString()\n\t * yields the original String object, and does not perform any copies\n\t * or String conversion operations.\n\t * </p>\n\t */\n\t@Override\n\tpublic String toString() {\n\t\tif(off > 0) {\n\t\t\treturn super.toString();\n\t\t} else {\n\t\t\treturn str;\n\t\t}\n\t}\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/TerminatorThread.java",
    "content": "/**\n * Copyright (c) 2011 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.\n */\npackage com.yahoo.ycsb;\n\nimport java.util.Vector;\n\n/**\n * A thread that waits for the maximum specified time and then interrupts all the client\n * threads passed as the Vector at initialization of this thread.\n * \n * The maximum execution time passed is assumed to be in seconds.\n * \n * @author sudipto\n *\n */\npublic class TerminatorThread extends Thread {\n  \n  private Vector<Thread> threads;\n  private long maxExecutionTime;\n  private Workload workload;\n  private long waitTimeOutInMS;\n  \n  public TerminatorThread(long maxExecutionTime, Vector<Thread> threads, \n      Workload workload) {\n    this.maxExecutionTime = maxExecutionTime;\n    this.threads = threads;\n    this.workload = workload;\n    waitTimeOutInMS = 2000;\n    System.err.println(\"Maximum execution time specified as: \" + maxExecutionTime + \" secs\");\n  }\n  \n  public void run() {\n    try {\n      Thread.sleep(maxExecutionTime * 1000);\n    } catch (InterruptedException e) {\n      System.err.println(\"Could not wait until max specified time, TerminatorThread interrupted.\");\n      return;\n    }\n    System.err.println(\"Maximum time elapsed. Requesting stop for the workload.\");\n    workload.requestStop();\n    System.err.println(\"Stop requested for workload. Now Joining!\");\n    for (Thread t : threads) {\n      while (t.isAlive()) {\n        try {\n          t.join(waitTimeOutInMS);\n          if (t.isAlive()) {\n            System.err.println(\"Still waiting for thread \" + t.getName() + \" to complete. \" +\n                \"Workload status: \" + workload.isStopRequested());\n          }\n        } catch (InterruptedException e) {\n          // Do nothing. Don't know why I was interrupted.\n        }\n      }\n    }\n  }\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/UnknownDBException.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb;\n\n/**\n * Could not create the specified DB.\n */\npublic class UnknownDBException extends Exception\n{\n      /**\n\t * \n\t */\n\tprivate static final long serialVersionUID = 459099842269616836L;\n\n\tpublic UnknownDBException(String message) \n      {\n\t super(message);\n      }\n      \n      public UnknownDBException()\n      {\n\t super();\n      }\n\n      public UnknownDBException(String message, Throwable cause)\n      {\n\t super(message,cause);\n      }\n      \n      public UnknownDBException(Throwable cause)\n      {\n\t super(cause);\n      }\n      \n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/Utils.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb;\n\nimport java.util.Random;\n\n/**\n * Utility functions.\n */\npublic class Utils\n{\n  private static final Random rand = new Random();\n  private static final ThreadLocal<Random> rng = new ThreadLocal<Random>();\n\n  public static Random random() {\n    Random ret = rng.get();\n    if(ret == null) {\n      ret = new Random(rand.nextLong());\n      rng.set(ret);\n    }\n    return ret;\n  }\n      /**\n       * Generate a random ASCII string of a given length.\n       */\n      public static String ASCIIString(int length)\n      {\n\t int interval='~'-' '+1;\n\t\n        byte []buf = new byte[length];\n        random().nextBytes(buf);\n        for (int i = 0; i < length; i++) {\n          if (buf[i] < 0) {\n            buf[i] = (byte)((-buf[i] % interval) + ' ');\n          } else {\n            buf[i] = (byte)((buf[i] % interval) + ' ');\n          }\n        }\n        return new String(buf);\n      }\n      \n      /**\n       * Hash an integer value.\n       */\n      public static long hash(long val)\n      {\n\t return FNVhash64(val);\n      }\n\t\n      public static final int FNV_offset_basis_32=0x811c9dc5;\n      public static final int FNV_prime_32=16777619;\n      \n      /**\n       * 32 bit FNV hash. Produces more \"random\" hashes than (say) String.hashCode().\n       * \n       * @param val The value to hash.\n       * @return The hash value\n       */\n      public static int FNVhash32(int val)\n      {\n\t //from http://en.wikipedia.org/wiki/Fowler_Noll_Vo_hash\n\t int hashval = FNV_offset_basis_32;\n\t \n\t for (int i=0; i<4; i++)\n\t {\n\t    int octet=val&0x00ff;\n\t    val=val>>8;\n\t    \n\t    hashval = hashval ^ octet;\n\t    hashval = hashval * FNV_prime_32;\n\t    //hashval = hashval ^ octet;\n\t }\n\t return Math.abs(hashval);\n      }\n      \n      public static final long FNV_offset_basis_64=0xCBF29CE484222325L;\n      public static final long FNV_prime_64=1099511628211L;\n      \n      /**\n       * 64 bit FNV hash. Produces more \"random\" hashes than (say) String.hashCode().\n       * \n       * @param val The value to hash.\n       * @return The hash value\n       */\n      public static long FNVhash64(long val)\n      {\n\t //from http://en.wikipedia.org/wiki/Fowler_Noll_Vo_hash\n\t long hashval = FNV_offset_basis_64;\n\t \n\t for (int i=0; i<8; i++)\n\t {\n\t    long octet=val&0x00ff;\n\t    val=val>>8;\n\t    \n\t    hashval = hashval ^ octet;\n\t    hashval = hashval * FNV_prime_64;\n\t    //hashval = hashval ^ octet;\n\t }\n\t return Math.abs(hashval);\n      }\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/Workload.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb;\n\nimport java.util.Properties;\nimport java.util.concurrent.atomic.AtomicBoolean;\n\n/**\n * One experiment scenario. One object of this type will\n * be instantiated and shared among all client threads. This class\n * should be constructed using a no-argument constructor, so we can\n * load it dynamically. Any argument-based initialization should be\n * done by init().\n * \n * If you extend this class, you should support the \"insertstart\" property. This \n * allows the load phase to proceed from multiple clients on different machines, in case\n * the client is the bottleneck. For example, if we want to load 1 million records from\n * 2 machines, the first machine should have insertstart=0 and the second insertstart=500000. Additionally,\n * the \"insertcount\" property, which is interpreted by Client, can be used to tell each instance of the\n * client how many inserts to do. In the example above, both clients should have insertcount=500000.\n */\npublic abstract class Workload\n{\n\tpublic static final String INSERT_START_PROPERTY=\"insertstart\";\n\t\n\tpublic static final String INSERT_START_PROPERTY_DEFAULT=\"0\";\n\t\n\tprivate volatile AtomicBoolean stopRequested = new AtomicBoolean(false);\n\t\n      /**\n       * Initialize the scenario. Create any generators and other shared objects here.\n       * Called once, in the main client thread, before any operations are started.\n       */\n      public void init(Properties p) throws WorkloadException\n      {\n      }\n\n      /**\n       * Initialize any state for a particular client thread. Since the scenario object\n       * will be shared among all threads, this is the place to create any state that is specific\n       * to one thread. To be clear, this means the returned object should be created anew on each\n       * call to initThread(); do not return the same object multiple times. \n       * The returned object will be passed to invocations of doInsert() and doTransaction() \n       * for this thread. There should be no side effects from this call; all state should be encapsulated\n       * in the returned object. If you have no state to retain for this thread, return null. (But if you have\n       * no state to retain for this thread, probably you don't need to override initThread().)\n       * \n       * @return false if the workload knows it is done for this thread. Client will terminate the thread. Return true otherwise. Return true for workloads that rely on operationcount. For workloads that read traces from a file, return true when there are more to do, false when you are done.\n       */\n      public Object initThread(Properties p, int mythreadid, int threadcount) throws WorkloadException\n      {\n\t return null;\n      }\n      \n      /**\n       * Cleanup the scenario. Called once, in the main client thread, after all operations have completed.\n       */\n      public void cleanup() throws WorkloadException\n      {\n      }\n      \n      /**\n       * Do one insert operation. Because it will be called concurrently from multiple client threads, this \n       * function must be thread safe. However, avoid synchronized, or the threads will block waiting for each \n       * other, and it will be difficult to reach the target throughput. Ideally, this function would have no side\n       * effects other than DB operations and mutations on threadstate. Mutations to threadstate do not need to be\n       * synchronized, since each thread has its own threadstate instance.\n       */\n      public abstract boolean doInsert(DB db, Object threadstate);\n      \n      /**\n       * Do one transaction operation. Because it will be called concurrently from multiple client threads, this \n       * function must be thread safe. However, avoid synchronized, or the threads will block waiting for each \n       * other, and it will be difficult to reach the target throughput. Ideally, this function would have no side\n       * effects other than DB operations and mutations on threadstate. Mutations to threadstate do not need to be\n       * synchronized, since each thread has its own threadstate instance.\n       * \n       * @return false if the workload knows it is done for this thread. Client will terminate the thread. Return true otherwise. Return true for workloads that rely on operationcount. For workloads that read traces from a file, return true when there are more to do, false when you are done.\n       */\n      public abstract boolean doTransaction(DB db, Object threadstate);\n      \n      /**\n       * Allows scheduling a request to stop the workload.\n       */\n      public void requestStop() {\n        stopRequested.set(true);\n      }\n      \n      /**\n       * Check the status of the stop request flag.\n       * @return true if stop was requested, false otherwise.\n       */\n      public boolean isStopRequested() {\n        if (stopRequested.get() == true) return true;\n        else return false;\n      }\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/WorkloadException.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb;\n\n/**\n * The workload tried to do something bad.\n */\npublic class WorkloadException extends Exception\n{\n\t/**\n\t * \n\t */\n\tprivate static final long serialVersionUID = 8844396756042772132L;\n\n\tpublic WorkloadException(String message) \n\t{\n\t\tsuper(message);\n\t}\n\n\tpublic WorkloadException()\n\t{\n\t\tsuper();\n\t}\n\n\tpublic WorkloadException(String message, Throwable cause)\n\t{\n\t\tsuper(message,cause);\n\t}\n\n\tpublic WorkloadException(Throwable cause)\n\t{\n\t\tsuper(cause);\n\t}\n\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/ConstantIntegerGenerator.java",
    "content": "/**\n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.\n */\npackage com.yahoo.ycsb.generator;\n\n/**\n * A trivial integer generator that always returns the same value.\n * \n * @author sears\n *\n */\npublic class ConstantIntegerGenerator extends IntegerGenerator {\n\tprivate final int i;\n\t/**\n\t * @param i The integer that this generator will always return.\n\t */\n\tpublic ConstantIntegerGenerator(int i) {\n\t\tthis.i = i;\n\t}\n\n\t@Override\n\tpublic int nextInt() {\n\t\treturn i;\n\t}\n\n\t@Override\n\tpublic double mean() {\n\t\treturn i;\n\t}\n\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/CounterGenerator.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb.generator;\n\nimport java.util.concurrent.atomic.AtomicInteger;\n\n/**\n * Generates a sequence of integers 0, 1, ...\n */\npublic class CounterGenerator extends IntegerGenerator\n{\n\tfinal AtomicInteger counter;\n\n\t/**\n\t * Create a counter that starts at countstart\n\t */\n\tpublic CounterGenerator(int countstart)\n\t{\n\t\tcounter=new AtomicInteger(countstart);\n\t\tsetLastInt(counter.get()-1);\n\t}\n\t\n\t/**\n\t * If the generator returns numeric (integer) values, return the next value as an int. Default is to return -1, which\n\t * is appropriate for generators that do not return numeric values.\n\t */\n\tpublic int nextInt() \n\t{\n\t\tint ret = counter.getAndIncrement();\n\t\tsetLastInt(ret);\n\t\treturn ret;\n\t}\n\t@Override\n\tpublic int lastInt()\n\t{\n\t                return counter.get() - 1;\n\t}\n\t@Override\n\tpublic double mean() {\n\t\tthrow new UnsupportedOperationException(\"Can't compute mean of non-stationary distribution!\");\n\t}\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/DiscreteGenerator.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb.generator;\n\nimport java.util.Vector;\nimport java.util.Random;\n\nimport com.yahoo.ycsb.Utils;\nimport com.yahoo.ycsb.WorkloadException;\n\n/**\n * Generates a distribution by choosing from a discrete set of values.\n */\npublic class DiscreteGenerator extends Generator\n{\n\tclass Pair\n\t{\n\t\tpublic double _weight;\n\t\tpublic String _value;\n\n\t\tPair(double weight, String value)\n\t\t{\n\t\t\t_weight=weight;\n\t\t\t_value=value;\n\t\t}\n\t}\n\n\tVector<Pair> _values;\n\tString _lastvalue;\n\n\tpublic DiscreteGenerator()\n\t{\n\t\t_values=new Vector<Pair>();\n\t\t_lastvalue=null;\n\t}\n\n\t/**\n\t * Generate the next string in the distribution.\n\t */\n\tpublic String nextString()\n\t{\n\t\tdouble sum=0;\n\n\t\tfor (Pair p : _values)\n\t\t{\n\t\t\tsum+=p._weight;\n\t\t}\n\n\t\tdouble val=Utils.random().nextDouble();\n\n\t\tfor (Pair p : _values)\n\t\t{\n\t\t\tif (val<p._weight/sum)\n\t\t\t{\n\t\t\t\treturn p._value;\n\t\t\t}\n\n\t\t\tval-=p._weight/sum;\n\t\t}\n\n\t\t//should never get here.\n\t\tSystem.out.println(\"oops. should not get here.\");\n\n\t\tSystem.exit(0);\n\n\t\treturn null;\n\t}\n\n\t/**\n\t * If the generator returns numeric (integer) values, return the next value as an int. Default is to return -1, which\n\t * is appropriate for generators that do not return numeric values.\n\t * \n\t * @throws WorkloadException if this generator does not support integer values\n\t */\n\tpublic int nextInt() throws WorkloadException\n\t{\n\t\tthrow new WorkloadException(\"DiscreteGenerator does not support nextInt()\");\n\t}\n\n\t/**\n\t * Return the previous string generated by the distribution; e.g., returned from the last nextString() call. \n\t * Calling lastString() should not advance the distribution or have any side effects. If nextString() has not yet \n\t * been called, lastString() should return something reasonable.\n\t */\n\tpublic String lastString()\n\t{\n\t\tif (_lastvalue==null)\n\t\t{\n\t\t\t_lastvalue=nextString();\n\t\t}\n\t\treturn _lastvalue;\n\t}\n\n\tpublic void addValue(double weight, String value)\n\t{\n\t\t_values.add(new Pair(weight,value));\n\t}\n\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/ExponentialGenerator.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2011 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb.generator;\n\nimport java.util.Random;\n\nimport com.yahoo.ycsb.Utils;\n\n/**\n * A generator of an exponential distribution. It produces a sequence\n * of time intervals (integers) according to an exponential\n * distribution.  Smaller intervals are more frequent than larger\n * ones, and there is no bound on the length of an interval.  When you\n * construct an instance of this class, you specify a parameter gamma,\n * which corresponds to the rate at which events occur.\n * Alternatively, 1/gamma is the average length of an interval.\n */\npublic class ExponentialGenerator extends IntegerGenerator\n{\n    // What percentage of the readings should be within the most recent exponential.frac portion of the dataset?\n    public static final String EXPONENTIAL_PERCENTILE_PROPERTY=\"exponential.percentile\";\n    public static final String EXPONENTIAL_PERCENTILE_DEFAULT=\"95\";\n\n    // What fraction of the dataset should be accessed exponential.percentile of the time?\n    public static final String EXPONENTIAL_FRAC_PROPERTY = \"exponential.frac\";\n    public static final String EXPONENTIAL_FRAC_DEFAULT  = \"0.8571428571\";  // 1/7\n\n\t/**\n\t * The exponential constant to use.\n\t */\n\tdouble _gamma;\t\n\n\t/******************************* Constructors **************************************/\n\n\t/**\n\t * Create an exponential generator with a mean arrival rate of\n\t * gamma.  (And half life of 1/gamma).\n\t */\n\tpublic ExponentialGenerator(double mean)\n\t{\n\t\t_gamma = 1.0/mean;\n\t}\n\tpublic ExponentialGenerator(double percentile, double range)\n\t{\n\t\t_gamma = -Math.log(1.0-percentile/100.0) / range;  //1.0/mean;\n\t}\n\n\t/****************************************************************************************/\n\t\n\t/** \n\t * Generate the next item. this distribution will be skewed toward lower integers; e.g. 0 will\n\t * be the most popular, 1 the next most popular, etc.\n\t * @param itemcount The number of items in the distribution.\n\t * @return The next item in the sequence.\n\t */\n\t@Override\n\tpublic int nextInt()\n\t{\n\t\treturn (int)nextLong();\n\t}\n\n\t/**\n\t * Generate the next item as a long.\n\t * \n\t * @param itemcount The number of items in the distribution.\n\t * @return The next item in the sequence.\n\t */\n\tpublic long nextLong()\n\t{\n\t\treturn (long) (-Math.log(Utils.random().nextDouble()) / _gamma);\n\t}\n\n\t@Override\n\tpublic double mean() {\n\t\treturn 1.0/_gamma;\n\t}\n    public static void main(String args[]) {\n        ExponentialGenerator e = new ExponentialGenerator(90, 100);\n        int j = 0;\n        for(int i = 0; i < 1000; i++) {\n            if(e.nextInt() < 100) {\n                j++;\n            }\n        }\n        System.out.println(\"Got \" + j + \" hits.  Expect 900\");\n    }\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/FileGenerator.java",
    "content": "/**                                                                                                                                                                                \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb.generator;\n\nimport java.io.File;\nimport java.io.FileInputStream;\nimport java.io.InputStreamReader;\nimport java.io.BufferedReader;\nimport java.io.IOException;\n\n/**\n * A generator, whose sequence is the lines of a file.\n */\npublic class FileGenerator extends Generator\n{\n\tString filename;\n\tString current;\n\tBufferedReader reader;\n\n\t/**\n\t * Create a FileGenerator with the given file.\n\t * @param _filename The file to read lines from.\n\t */\n\tpublic FileGenerator(String _filename)\n\t{\n\t\ttry {\n\t\t\tfilename = _filename;\n\t\t\tFile file = new File(filename);\n\t\t\tFileInputStream in = new FileInputStream(file);\n\t\t\treader = new BufferedReader(new InputStreamReader(in));\n\t\t} catch(IOException e) {\n\t\t\tSystem.err.println(\"Exception: \" + e);\n\t\t}\n\t}\n\n\t/**\n\t * Return the next string of the sequence, ie the next line of the file.\n\t */\n\tpublic synchronized String nextString()\n\t{\n\t\ttry {\n\t\t\treturn current = reader.readLine();\n\t\t} catch(NullPointerException e) {\n\t\t\tSystem.err.println(\"NullPointerException: \" + filename + ':' + current);\n\t\t\tthrow e;\n\t\t} catch(IOException e) {\n\t\t\tSystem.err.println(\"Exception: \" + e);\n\t\t\treturn null;\n\t\t}\n\t}\n\n\t/**\n\t * Return the previous read line.\n\t */\n\tpublic String lastString()\n\t{\n\t\treturn current;\n\t}\n\n\t/**\n\t * Reopen the file to reuse values.\n\t */\n\tpublic synchronized void reloadFile()\n\t{\n\t\ttry {\n\t\t\tSystem.err.println(\"Reload \" + filename);\n\t\t\treader.close();\n\t\t\tFile file = new File(filename);\n\t\t\tFileInputStream in = new FileInputStream(file);\n\t\t\treader = new BufferedReader(new InputStreamReader(in));\n\t\t} catch(IOException e) {\n\t\t\tSystem.err.println(\"Exception: \" + e);\n\t\t}\n\t}\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/Generator.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb.generator;\n\n/**\n * An expression that generates a sequence of string values, following some distribution (Uniform, Zipfian, Sequential, etc.)\n */\npublic abstract class Generator \n{\n\t/**\n\t * Generate the next string in the distribution.\n\t */\n\tpublic abstract String nextString();\n\n\t/**\n\t * Return the previous string generated by the distribution; e.g., returned from the last nextString() call. \n\t * Calling lastString() should not advance the distribution or have any side effects. If nextString() has not yet \n\t * been called, lastString() should return something reasonable.\n\t */\n\tpublic abstract String lastString();\n}\n\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/HistogramGenerator.java",
    "content": "/**\n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.\n */\npackage com.yahoo.ycsb.generator;\nimport java.io.BufferedReader;\nimport java.io.FileReader;\nimport java.io.IOException;\nimport java.util.ArrayList;\nimport java.util.Random;\n\nimport com.yahoo.ycsb.Utils;\nimport com.yahoo.ycsb.generator.IntegerGenerator;\n\n/**\n * Generate integers according to a histogram distribution.  The histogram\n * buckets are of width one, but the values are multiplied by a block size.\n * Therefore, instead of drawing sizes uniformly at random within each\n * bucket, we always draw the largest value in the current bucket, so the value\n * drawn is always a multiple of block_size.\n * \n * The minimum value this distribution returns is block_size (not zero).\n * \n * Modified Nov 19 2010 by sears\n * \n * @author snjones\n *\n */\npublic class HistogramGenerator extends IntegerGenerator {\n\n\tlong block_size;\n\tlong[] buckets;\n\tlong area;\n\tlong weighted_area = 0;\n\tdouble mean_size = 0;\n\t\n\tpublic HistogramGenerator(String histogramfile) throws IOException {\n\tBufferedReader in = new BufferedReader(new FileReader(histogramfile));\n\tString str;\n\tString[] line;\n\t\n\tArrayList<Integer> a = new ArrayList<Integer>();\n\n\tstr = in.readLine();\n\tif(str == null) {\n\t\tthrow new IOException(\"Empty input file!\\n\");\n\t}\n\tline = str.split(\"\\t\");\n\tif(line[0].compareTo(\"BlockSize\") != 0) {\n\t\tthrow new IOException(\"First line of histogram is not the BlockSize!\\n\");\n\t}\n\tblock_size = Integer.parseInt(line[1]);\n\t\n\twhile((str = in.readLine()) != null){\n\t\t// [0] is the bucket, [1] is the value\n\t\tline = str.split(\"\\t\");\n\t\t\n\t\ta.add(Integer.parseInt(line[0]), Integer.parseInt(line[1]));\n\t}\n\tbuckets = new long[a.size()];\n\tfor(int i = 0; i < a.size(); i++) {\n\t\tbuckets[i] = a.get(i);\n\t}\n\n\tin.close();\n\tinit();\n\t}\n\n\tpublic HistogramGenerator(long[] buckets, int block_size) {\n\t\tthis.block_size = block_size;\n\t\tthis.buckets = buckets;\n\t\tinit();\n\t}\n\tprivate void init() {\n\t\tfor(int i = 0; i < buckets.length; i++) {\n\t\t\tarea += buckets[i];\n\t\t\tweighted_area = i * buckets[i];\n\t\t}\n\t\t// calculate average file size\n\t\tmean_size = ((double)block_size) * ((double)weighted_area) / (double)(area);\n\t}\n\n\t@Override\n\tpublic int nextInt() {\n\t\tint number = Utils.random().nextInt((int)area);\n\t\tint i;\n\t\t\n\t\tfor(i = 0; i < (buckets.length - 1); i++){\n\t\t\tnumber -= buckets[i];\n\t\t\tif(number <= 0){\n\t\t\t\treturn (int)((i+1)*block_size);\n\t\t\t}\n\t\t}\n\t\t\n\t\treturn (int)(i * block_size);\n\t}\n\n\t@Override\n\tpublic double mean() {\n\t\treturn mean_size;\n\t}\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/HotspotIntegerGenerator.java",
    "content": "/**\n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.\n */\npackage com.yahoo.ycsb.generator;\n\nimport java.util.Random;\n\nimport com.yahoo.ycsb.Utils;\n\n/**\n * Generate integers resembling a hotspot distribution where x% of operations\n * access y% of data items. The parameters specify the bounds for the numbers,\n * the percentage of the of the interval which comprises the hot set and\n * the percentage of operations that access the hot set. Numbers of the hot set are\n * always smaller than any number in the cold set. Elements from the hot set and\n * the cold set are chose using a uniform distribution.\n * \n * @author sudipto\n *\n */\npublic class HotspotIntegerGenerator extends IntegerGenerator {\n\n  private final int lowerBound;\n  private final int upperBound;\n  private final int hotInterval;\n  private final int coldInterval;\n  private final double hotsetFraction;\n  private final double hotOpnFraction;\n  \n  /**\n   * Create a generator for Hotspot distributions.\n   * \n   * @param lowerBound lower bound of the distribution.\n   * @param upperBound upper bound of the distribution.\n   * @param hotsetFraction percentage of data item\n   * @param hotOpnFraction percentage of operations accessing the hot set.\n   */\n  public HotspotIntegerGenerator(int lowerBound, int upperBound, \n      double hotsetFraction, double hotOpnFraction) {\n    if (hotsetFraction < 0.0 || hotsetFraction > 1.0) {\n      System.err.println(\"Hotset fraction out of range. Setting to 0.0\");\n      hotsetFraction = 0.0;\n    }\n    if (hotOpnFraction < 0.0 || hotOpnFraction > 1.0) {\n      System.err.println(\"Hot operation fraction out of range. Setting to 0.0\");\n      hotOpnFraction = 0.0;\n    }\n    if (lowerBound > upperBound) {\n      System.err.println(\"Upper bound of Hotspot generator smaller than the lower bound. \" +\n      \t\t\"Swapping the values.\");\n      int temp = lowerBound;\n      lowerBound = upperBound;\n      upperBound = temp;\n    }\n    this.lowerBound = lowerBound;\n    this.upperBound = upperBound;\n    this.hotsetFraction = hotsetFraction;\n    int interval = upperBound - lowerBound + 1;\n    this.hotInterval = (int)(interval * hotsetFraction);\n    this.coldInterval = interval - hotInterval;\n    this.hotOpnFraction = hotOpnFraction;\n  }\n  \n  @Override\n  public int nextInt() {\n    int value = 0;\n    Random random = Utils.random();\n    if (random.nextDouble() < hotOpnFraction) {\n      // Choose a value from the hot set.\n      value = lowerBound + random.nextInt(hotInterval);\n    } else {\n      // Choose a value from the cold set.\n      value = lowerBound + hotInterval + random.nextInt(coldInterval);\n    }\n    setLastInt(value);\n    return value;\n  }\n\n  /**\n   * @return the lowerBound\n   */\n  public int getLowerBound() {\n    return lowerBound;\n  }\n\n  /**\n   * @return the upperBound\n   */\n  public int getUpperBound() {\n    return upperBound;\n  }\n\n  /**\n   * @return the hotsetFraction\n   */\n  public double getHotsetFraction() {\n    return hotsetFraction;\n  }\n\n  /**\n   * @return the hotOpnFraction\n   */\n  public double getHotOpnFraction() {\n    return hotOpnFraction;\n  }\n  @Override\n  public double mean() {\n    return hotOpnFraction * (lowerBound + hotInterval/2.0)\n      + (1 - hotOpnFraction) * (lowerBound + hotInterval + coldInterval/2.0);\n  }\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/IntegerGenerator.java",
    "content": "/**                                                                                                                                                                                \r\n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \r\n * may not use this file except in compliance with the License. You                                                                                                                \r\n * may obtain a copy of the License at                                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \r\n *                                                                                                                                                                                 \r\n * Unless required by applicable law or agreed to in writing, software                                                                                                             \r\n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \r\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \r\n * implied. See the License for the specific language governing                                                                                                                    \r\n * permissions and limitations under the License. See accompanying                                                                                                                 \r\n * LICENSE file.                                                                                                                                                                   \r\n */\r\n\r\npackage com.yahoo.ycsb.generator;\r\n\r\n/**\r\n * A generator that is capable of generating ints as well as strings\r\n * \r\n * @author cooperb\r\n *\r\n */\r\npublic abstract class IntegerGenerator extends Generator \r\n{\r\n\tint lastint;\r\n\t\r\n\t/**\r\n\t * Set the last value generated. IntegerGenerator subclasses must use this call\r\n\t * to properly set the last string value, or the lastString() and lastInt() calls won't work.\r\n\t */\r\n\tprotected void setLastInt(int last)\r\n\t{\r\n\t\tlastint=last;\r\n\t}\r\n\t\r\n\t/**\r\n\t * Return the next value as an int. When overriding this method, be sure to call setLastString() properly, or the lastString() call won't work.\r\n\t */\r\n\tpublic abstract int nextInt();\r\n\t\r\n\t/**\r\n\t * Generate the next string in the distribution.\r\n\t */\r\n\tpublic String nextString()\r\n\t{\r\n\t\treturn \"\"+nextInt();\r\n\t}\r\n\t\r\n\t/**\r\n\t * Return the previous string generated by the distribution; e.g., returned from the last nextString() call. \r\n\t * Calling lastString() should not advance the distribution or have any side effects. If nextString() has not yet \r\n\t * been called, lastString() should return something reasonable.\r\n\t */\r\n\t@Override\r\n\tpublic String lastString()\r\n\t{\r\n\t\treturn \"\"+lastInt();\r\n\t}\r\n\t\r\n\t/**\r\n\t * Return the previous int generated by the distribution. This call is unique to IntegerGenerator subclasses, and assumes\r\n\t * IntegerGenerator subclasses always return ints for nextInt() (e.g. not arbitrary strings).\r\n\t */\r\n\tpublic int lastInt()\r\n\t{\r\n\t\treturn lastint;\r\n\t}\r\n\t/**\r\n\t * Return the expected value (mean) of the values this generator will return.\r\n\t */\r\n\tpublic abstract double mean();\r\n}\r\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/ScrambledZipfianGenerator.java",
    "content": "/**                                                                                                                                                                                \r\n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \r\n * may not use this file except in compliance with the License. You                                                                                                                \r\n * may obtain a copy of the License at                                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \r\n *                                                                                                                                                                                 \r\n * Unless required by applicable law or agreed to in writing, software                                                                                                             \r\n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \r\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \r\n * implied. See the License for the specific language governing                                                                                                                    \r\n * permissions and limitations under the License. See accompanying                                                                                                                 \r\n * LICENSE file.                                                                                                                                                                   \r\n */\r\n\r\npackage com.yahoo.ycsb.generator;\r\n\r\nimport com.yahoo.ycsb.Utils;\r\n\r\n/**\r\n * A generator of a zipfian distribution. It produces a sequence of items, such that some items are more popular than others, according\r\n * to a zipfian distribution. When you construct an instance of this class, you specify the number of items in the set to draw from, either\r\n * by specifying an itemcount (so that the sequence is of items from 0 to itemcount-1) or by specifying a min and a max (so that the sequence is of \r\n * items from min to max inclusive). After you construct the instance, you can change the number of items by calling nextInt(itemcount) or nextLong(itemcount).\r\n * \r\n * Unlike @ZipfianGenerator, this class scatters the \"popular\" items across the itemspace. Use this, instead of @ZipfianGenerator, if you\r\n * don't want the head of the distribution (the popular items) clustered together.\r\n */\r\npublic class ScrambledZipfianGenerator extends IntegerGenerator \r\n{\r\n\tpublic static final double ZETAN=26.46902820178302;\r\n        public static final double USED_ZIPFIAN_CONSTANT=0.99;\r\n\tpublic static final long ITEM_COUNT=10000000000L;\r\n\t\r\n\tZipfianGenerator gen;\r\n\tlong _min,_max,_itemcount;\r\n\t\r\n\t/******************************* Constructors **************************************/\r\n\r\n\t/**\r\n\t * Create a zipfian generator for the specified number of items.\r\n\t * @param _items The number of items in the distribution.\r\n\t */\r\n\tpublic ScrambledZipfianGenerator(long _items)\r\n\t{\r\n\t\tthis(0,_items-1);\r\n\t}\r\n\r\n\t/**\r\n\t * Create a zipfian generator for items between min and max.\r\n\t * @param _min The smallest integer to generate in the sequence.\r\n\t * @param _max The largest integer to generate in the sequence.\r\n\t */\r\n\tpublic ScrambledZipfianGenerator(long _min, long _max)\r\n\t{\r\n\t\tthis(_min,_max,ZipfianGenerator.ZIPFIAN_CONSTANT);\r\n\t}\r\n\r\n\t/**\r\n\t * Create a zipfian generator for the specified number of items using the specified zipfian constant.\r\n\t * \r\n\t * @param _items The number of items in the distribution.\r\n\t * @param _zipfianconstant The zipfian constant to use.\r\n\t */\r\n\t/*\r\n// not supported, as the value of zeta depends on the zipfian constant, and we have only precomputed zeta for one zipfian constant\r\n\tpublic ScrambledZipfianGenerator(long _items, double _zipfianconstant)\r\n\t{\r\n\t\tthis(0,_items-1,_zipfianconstant);\r\n\t}\r\n*/\r\n\t\r\n\t/**\r\n\t * Create a zipfian generator for items between min and max (inclusive) for the specified zipfian constant. If you \r\n\t * use a zipfian constant other than 0.99, this will take a long time to complete because we need to recompute zeta.\r\n\t * @param min The smallest integer to generate in the sequence.\r\n\t * @param max The largest integer to generate in the sequence.\r\n\t * @param _zipfianconstant The zipfian constant to use.\r\n\t */\r\n        public ScrambledZipfianGenerator(long min, long max, double _zipfianconstant)\r\n\t{\r\n\t\t_min=min;\r\n\t\t_max=max;\r\n\t\t_itemcount=_max-_min+1;\r\n\t\tif (_zipfianconstant == USED_ZIPFIAN_CONSTANT) \r\n\t\t{\r\n\t\t    gen=new ZipfianGenerator(0,ITEM_COUNT,_zipfianconstant,ZETAN);\r\n\t\t} else {\r\n\t\t    gen=new ZipfianGenerator(0,ITEM_COUNT,_zipfianconstant);\r\n\t\t}\r\n\t}\r\n\t\r\n\t/**************************************************************************************************/\r\n\t\r\n\t/**\r\n\t * Return the next int in the sequence.\r\n\t */\r\n\t@Override\r\n\tpublic int nextInt() {\r\n\t\treturn (int)nextLong();\r\n\t}\r\n\r\n\t/**\r\n\t * Return the next long in the sequence.\r\n\t */\r\n\tpublic long nextLong()\r\n\t{\r\n\t\tlong ret=gen.nextLong();\r\n\t\tret=_min+Utils.FNVhash64(ret)%_itemcount;\r\n\t\tsetLastInt((int)ret);\r\n\t\treturn ret;\r\n\t}\r\n\t\r\n\tpublic static void main(String[] args)\r\n\t{\r\n\t    double newzetan = ZipfianGenerator.zetastatic(ITEM_COUNT,ZipfianGenerator.ZIPFIAN_CONSTANT);\r\n\t    System.out.println(\"zetan: \"+newzetan);\r\n\t    System.exit(0);\r\n\r\n\t\tScrambledZipfianGenerator gen=new ScrambledZipfianGenerator(10000);\r\n\t\t\r\n\t\tfor (int i=0; i<1000000; i++)\r\n\t\t{\r\n\t\t\tSystem.out.println(\"\"+gen.nextInt());\r\n\t\t}\r\n\t}\r\n\r\n\t/**\r\n\t * since the values are scrambled (hopefully uniformly), the mean is simply the middle of the range.\r\n\t */\r\n\t@Override\r\n\tpublic double mean() {\r\n\t\treturn ((double)(((long)_min) +(long)_max))/2.0;\r\n\t}\r\n}\r\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/SkewedLatestGenerator.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb.generator;\n\n/**\n * Generate a popularity distribution of items, skewed to favor recent items significantly more than older items.\n */\npublic class SkewedLatestGenerator extends IntegerGenerator\n{\n\tCounterGenerator _basis;\n\tZipfianGenerator _zipfian;\n\n\tpublic SkewedLatestGenerator(CounterGenerator basis)\n\t{\n\t\t_basis=basis;\n\t\t_zipfian=new ZipfianGenerator(_basis.lastInt());\n\t\tnextInt();\n\t}\n\n\t/**\n\t * Generate the next string in the distribution, skewed Zipfian favoring the items most recently returned by the basis generator.\n\t */\n\tpublic int nextInt()\n\t{\n\t\tint max=_basis.lastInt();\n\t\tint nextint=max-_zipfian.nextInt(max);\n\t\tsetLastInt(nextint);\n\t\treturn nextint;\n\t}\n\n\tpublic static void main(String[] args)\n\t{\n\t\tSkewedLatestGenerator gen=new SkewedLatestGenerator(new CounterGenerator(1000));\n\t\tfor (int i=0; i<Integer.parseInt(args[0]); i++)\n\t\t{\n\t\t\tSystem.out.println(gen.nextString());\n\t\t}\n\n\t}\n\n\t@Override\n\tpublic double mean() {\n\t\tthrow new UnsupportedOperationException(\"Can't compute mean of non-stationary distribution!\");\n\t}\n\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/UniformGenerator.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb.generator;\n\nimport java.util.Vector;\n\n/**\n * An expression that generates a random integer in the specified range\n */\npublic class UniformGenerator extends Generator\n{\n\t\n\tVector<String> _values;\n\tString _laststring;\n\tUniformIntegerGenerator _gen;\n\t\n\n\t/**\n\t * Creates a generator that will return strings from the specified set uniformly randomly\n\t */\n\t@SuppressWarnings( \"unchecked\" ) \n\tpublic UniformGenerator(Vector<String> values)\n\t{\n\t\t_values=(Vector<String>)values.clone();\n\t\t_laststring=null;\n\t\t_gen=new UniformIntegerGenerator(0,values.size()-1);\n\t}\n\n\t/**\n\t * Generate the next string in the distribution.\n\t */\n\tpublic String nextString()\n\t{\n\t\t_laststring=_values.elementAt(_gen.nextInt());\n\t\treturn _laststring;\n\t}\n\t\n\t/**\n\t * Return the previous string generated by the distribution; e.g., returned from the last nextString() call. \n\t * Calling lastString() should not advance the distribution or have any side effects. If nextString() has not yet \n\t * been called, lastString() should return something reasonable.\n\t */\n\tpublic String lastString()\n\t{\n\t\tif (_laststring==null)\n\t\t{\n\t\t\tnextString();\n\t\t}\n\t\treturn _laststring;\n\t}\n}\n\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/UniformIntegerGenerator.java",
    "content": "/**                                                                                                                                                                                \r\n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \r\n * may not use this file except in compliance with the License. You                                                                                                                \r\n * may obtain a copy of the License at                                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \r\n *                                                                                                                                                                                 \r\n * Unless required by applicable law or agreed to in writing, software                                                                                                             \r\n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \r\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \r\n * implied. See the License for the specific language governing                                                                                                                    \r\n * permissions and limitations under the License. See accompanying                                                                                                                 \r\n * LICENSE file.                                                                                                                                                                   \r\n */\r\n\r\npackage com.yahoo.ycsb.generator;\r\n\r\nimport java.util.Random;\r\n\r\nimport com.yahoo.ycsb.Utils;\r\n\r\n/**\r\n * Generates integers randomly uniform from an interval.\r\n */\r\npublic class UniformIntegerGenerator extends IntegerGenerator \r\n{\r\n\tint _lb,_ub,_interval;\r\n\t\r\n\t/**\r\n\t * Creates a generator that will return integers uniformly randomly from the interval [lb,ub] inclusive (that is, lb and ub are possible values)\r\n\t *\r\n\t * @param lb the lower bound (inclusive) of generated values\r\n\t * @param ub the upper bound (inclusive) of generated values\r\n\t */\r\n\tpublic UniformIntegerGenerator(int lb, int ub)\r\n\t{\r\n\t\t_lb=lb;\r\n\t\t_ub=ub;\r\n\t\t_interval=_ub-_lb+1;\r\n\t}\r\n\t\r\n\t@Override\r\n\tpublic int nextInt() \r\n\t{\r\n\t\tint ret=Utils.random().nextInt(_interval)+_lb;\r\n\t\tsetLastInt(ret);\r\n\t\t\r\n\t\treturn ret;\r\n\t}\r\n\r\n\t@Override\r\n\tpublic double mean() {\r\n\t\treturn ((double)((long)(_lb + (long)_ub))) / 2.0;\r\n\t}\r\n}\r\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/generator/ZipfianGenerator.java",
    "content": "/**                                                                                                                                                                                \r\n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \r\n * may not use this file except in compliance with the License. You                                                                                                                \r\n * may obtain a copy of the License at                                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \r\n *                                                                                                                                                                                 \r\n * Unless required by applicable law or agreed to in writing, software                                                                                                             \r\n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \r\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \r\n * implied. See the License for the specific language governing                                                                                                                    \r\n * permissions and limitations under the License. See accompanying                                                                                                                 \r\n * LICENSE file.                                                                                                                                                                   \r\n */\r\n\r\npackage com.yahoo.ycsb.generator;\r\n\r\nimport java.util.Random;\r\n\r\nimport com.yahoo.ycsb.Utils;\r\n\r\n/**\r\n * A generator of a zipfian distribution. It produces a sequence of items, such that some items are more popular than others, according\r\n * to a zipfian distribution. When you construct an instance of this class, you specify the number of items in the set to draw from, either\r\n * by specifying an itemcount (so that the sequence is of items from 0 to itemcount-1) or by specifying a min and a max (so that the sequence is of \r\n * items from min to max inclusive). After you construct the instance, you can change the number of items by calling nextInt(itemcount) or nextLong(itemcount).\r\n * \r\n * Note that the popular items will be clustered together, e.g. item 0 is the most popular, item 1 the second most popular, and so on (or min is the most \r\n * popular, min+1 the next most popular, etc.) If you don't want this clustering, and instead want the popular items scattered throughout the \r\n * item space, then use ScrambledZipfianGenerator instead.\r\n * \r\n * Be aware: initializing this generator may take a long time if there are lots of items to choose from (e.g. over a minute\r\n * for 100 million objects). This is because certain mathematical values need to be computed to properly generate a zipfian skew, and one of those\r\n * values (zeta) is a sum sequence from 1 to n, where n is the itemcount. Note that if you increase the number of items in the set, we can compute\r\n * a new zeta incrementally, so it should be fast unless you have added millions of items. However, if you decrease the number of items, we recompute\r\n * zeta from scratch, so this can take a long time. \r\n *\r\n * The algorithm used here is from \"Quickly Generating Billion-Record Synthetic Databases\", Jim Gray et al, SIGMOD 1994.\r\n */\r\npublic class ZipfianGenerator extends IntegerGenerator\r\n{     \r\n\tpublic static final double ZIPFIAN_CONSTANT=0.99;\r\n\r\n\t/**\r\n\t * Number of items.\r\n\t */\r\n\tlong items;\r\n\t\r\n\t/**\r\n\t * Min item to generate.\r\n\t */\r\n\tlong base;\r\n\t\r\n\t/**\r\n\t * The zipfian constant to use.\r\n\t */\r\n\tdouble zipfianconstant;\r\n\t\r\n\t/**\r\n\t * Computed parameters for generating the distribution.\r\n\t */\r\n\tdouble alpha,zetan,eta,theta,zeta2theta;\r\n\t\r\n\t/**\r\n\t * The number of items used to compute zetan the last time.\r\n\t */\r\n\tlong countforzeta;\r\n\t\r\n\t/**\r\n\t * Flag to prevent problems. If you increase the number of items the zipfian generator is allowed to choose from, this code will incrementally compute a new zeta\r\n\t * value for the larger itemcount. However, if you decrease the number of items, the code computes zeta from scratch; this is expensive for large itemsets.\r\n\t * Usually this is not intentional; e.g. one thread thinks the number of items is 1001 and calls \"nextLong()\" with that item count; then another thread who thinks the \r\n\t * number of items is 1000 calls nextLong() with itemcount=1000 triggering the expensive recomputation. (It is expensive for 100 million items, not really for 1000 items.) Why\r\n\t * did the second thread think there were only 1000 items? maybe it read the item count before the first thread incremented it. So this flag allows you to say if you really do\r\n\t * want that recomputation. If true, then the code will recompute zeta if the itemcount goes down. If false, the code will assume itemcount only goes up, and never recompute. \r\n\t */\r\n\tboolean allowitemcountdecrease=false;\r\n\r\n\t/******************************* Constructors **************************************/\r\n\r\n\t/**\r\n\t * Create a zipfian generator for the specified number of items.\r\n\t * @param _items The number of items in the distribution.\r\n\t */\r\n\tpublic ZipfianGenerator(long _items)\r\n\t{\r\n\t\tthis(0,_items-1);\r\n\t}\r\n\r\n\t/**\r\n\t * Create a zipfian generator for items between min and max.\r\n\t * @param _min The smallest integer to generate in the sequence.\r\n\t * @param _max The largest integer to generate in the sequence.\r\n\t */\r\n\tpublic ZipfianGenerator(long _min, long _max)\r\n\t{\r\n\t\tthis(_min,_max,ZIPFIAN_CONSTANT);\r\n\t}\r\n\r\n\t/**\r\n\t * Create a zipfian generator for the specified number of items using the specified zipfian constant.\r\n\t * \r\n\t * @param _items The number of items in the distribution.\r\n\t * @param _zipfianconstant The zipfian constant to use.\r\n\t */\r\n\tpublic ZipfianGenerator(long _items, double _zipfianconstant)\r\n\t{\r\n\t\tthis(0,_items-1,_zipfianconstant);\r\n\t}\r\n\r\n\t/**\r\n\t * Create a zipfian generator for items between min and max (inclusive) for the specified zipfian constant.\r\n\t * @param min The smallest integer to generate in the sequence.\r\n\t * @param max The largest integer to generate in the sequence.\r\n\t * @param _zipfianconstant The zipfian constant to use.\r\n\t */\r\n\tpublic ZipfianGenerator(long min, long max, double _zipfianconstant)\r\n\t{\r\n\t\tthis(min,max,_zipfianconstant,zetastatic(max-min+1,_zipfianconstant));\r\n\t}\r\n\t\r\n\t/**\r\n\t * Create a zipfian generator for items between min and max (inclusive) for the specified zipfian constant, using the precomputed value of zeta.\r\n\t * \r\n\t * @param min The smallest integer to generate in the sequence.\r\n\t * @param max The largest integer to generate in the sequence.\r\n\t * @param _zipfianconstant The zipfian constant to use.\r\n\t * @param _zetan The precomputed zeta constant.\r\n\t */\r\n\tpublic ZipfianGenerator(long min, long max, double _zipfianconstant, double _zetan)\r\n\t{\r\n\r\n\t\titems=max-min+1;\r\n\t\tbase=min;\r\n\t\tzipfianconstant=_zipfianconstant;\r\n\r\n\t\ttheta=zipfianconstant;\r\n\r\n\t\tzeta2theta=zeta(2,theta);\r\n\r\n\t\t\r\n\t\talpha=1.0/(1.0-theta);\r\n\t\t//zetan=zeta(items,theta);\r\n\t\tzetan=_zetan;\r\n\t\tcountforzeta=items;\r\n\t\teta=(1-Math.pow(2.0/items,1-theta))/(1-zeta2theta/zetan);\r\n\t\t\r\n\t\t//System.out.println(\"XXXX 3 XXXX\");\r\n\t\tnextInt();\r\n\t\t//System.out.println(\"XXXX 4 XXXX\");\r\n\t}\r\n\t\r\n\t/**************************************************************************/\r\n\t\r\n\t/**\r\n\t * Compute the zeta constant needed for the distribution. Do this from scratch for a distribution with n items, using the \r\n\t * zipfian constant theta. Remember the value of n, so if we change the itemcount, we can recompute zeta.\r\n\t * \r\n\t * @param n The number of items to compute zeta over.\r\n\t * @param theta The zipfian constant.\r\n\t */\r\n\tdouble zeta(long n, double theta)\r\n\t{\r\n\t\tcountforzeta=n;\r\n\t\treturn zetastatic(n,theta);\r\n\t}\r\n\t\r\n\t/**\r\n\t * Compute the zeta constant needed for the distribution. Do this from scratch for a distribution with n items, using the \r\n\t * zipfian constant theta. This is a static version of the function which will not remember n.\r\n\t * @param n The number of items to compute zeta over.\r\n\t * @param theta The zipfian constant.\r\n\t */\r\n\tstatic double zetastatic(long n, double theta)\r\n\t{\r\n\t\treturn zetastatic(0,n,theta,0);\r\n\t}\r\n\t\r\n\t/**\r\n\t * Compute the zeta constant needed for the distribution. Do this incrementally for a distribution that\r\n\t * has n items now but used to have st items. Use the zipfian constant theta. Remember the new value of \r\n\t * n so that if we change the itemcount, we'll know to recompute zeta.\r\n\t * \r\n\t * @param st The number of items used to compute the last initialsum\r\n\t * @param n The number of items to compute zeta over.\r\n\t * @param theta The zipfian constant.\r\n     * @param initialsum The value of zeta we are computing incrementally from.\r\n\t */\r\n\tdouble zeta(long st, long n, double theta, double initialsum)\r\n\t{\r\n\t\tcountforzeta=n;\r\n\t\treturn zetastatic(st,n,theta,initialsum);\r\n\t}\r\n\t\r\n\t/**\r\n\t * Compute the zeta constant needed for the distribution. Do this incrementally for a distribution that\r\n\t * has n items now but used to have st items. Use the zipfian constant theta. Remember the new value of \r\n\t * n so that if we change the itemcount, we'll know to recompute zeta. \r\n\t * @param st The number of items used to compute the last initialsum\r\n\t * @param n The number of items to compute zeta over.\r\n\t * @param theta The zipfian constant.\r\n     * @param initialsum The value of zeta we are computing incrementally from.\r\n\t */\r\n\tstatic double zetastatic(long st, long n, double theta, double initialsum)\r\n\t{\r\n\t\tdouble sum=initialsum;\r\n\t\tfor (long i=st; i<n; i++)\r\n\t\t{\r\n\r\n\t\t\tsum+=1/(Math.pow(i+1,theta));\r\n\t\t}\r\n\t\t\r\n\t\t//System.out.println(\"countforzeta=\"+countforzeta);\r\n\t\t\r\n\t\treturn sum;\r\n\t}\r\n\r\n\t/****************************************************************************************/\r\n\t\r\n\t/** \r\n\t * Generate the next item. this distribution will be skewed toward lower integers; e.g. 0 will\r\n\t * be the most popular, 1 the next most popular, etc.\r\n\t * @param itemcount The number of items in the distribution.\r\n\t * @return The next item in the sequence.\r\n\t */\r\n\tpublic int nextInt(int itemcount)\r\n\t{\r\n\t\treturn (int)nextLong(itemcount);\r\n\t}\r\n\r\n\t/**\r\n\t * Generate the next item as a long.\r\n\t * \r\n\t * @param itemcount The number of items in the distribution.\r\n\t * @return The next item in the sequence.\r\n\t */\r\n\tpublic long nextLong(long itemcount)\r\n\t{\r\n\t\t//from \"Quickly Generating Billion-Record Synthetic Databases\", Jim Gray et al, SIGMOD 1994\r\n\r\n\t\tif (itemcount!=countforzeta)\r\n\t\t{\r\n\r\n\t\t\t//have to recompute zetan and eta, since they depend on itemcount\r\n\t\t\tsynchronized(this)\r\n\t\t\t{\r\n\t\t\t\tif (itemcount>countforzeta)\r\n\t\t\t\t{\r\n\t\t\t\t\t//System.err.println(\"WARNING: Incrementally recomputing Zipfian distribtion. (itemcount=\"+itemcount+\" countforzeta=\"+countforzeta+\")\");\r\n\t\t\t\t\t\r\n\t\t\t\t\t//we have added more items. can compute zetan incrementally, which is cheaper\r\n\t\t\t\t\tzetan=zeta(countforzeta,itemcount,theta,zetan);\r\n\t\t\t\t\teta=(1-Math.pow(2.0/items,1-theta))/(1-zeta2theta/zetan);\r\n\t\t\t\t}\r\n\t\t\t\telse if ( (itemcount<countforzeta) && (allowitemcountdecrease) )\r\n\t\t\t\t{\r\n\t\t\t\t\t//have to start over with zetan\r\n\t\t\t\t\t//note : for large itemsets, this is very slow. so don't do it!\r\n\r\n\t\t\t\t\t//TODO: can also have a negative incremental computation, e.g. if you decrease the number of items, then just subtract\r\n\t\t\t\t\t//the zeta sequence terms for the items that went away. This would be faster than recomputing from scratch when the number of items\r\n\t\t\t\t\t//decreases\r\n\t\t\t\t\t\r\n\t\t\t\t\tSystem.err.println(\"WARNING: Recomputing Zipfian distribtion. This is slow and should be avoided. (itemcount=\"+itemcount+\" countforzeta=\"+countforzeta+\")\");\r\n\t\t\t\t\t\r\n\t\t\t\t\tzetan=zeta(itemcount,theta);\r\n\t\t\t\t\teta=(1-Math.pow(2.0/items,1-theta))/(1-zeta2theta/zetan);\r\n\t\t\t\t}\r\n\t\t\t}\r\n\t\t}\r\n\r\n\t\tdouble u=Utils.random().nextDouble();\r\n\t\tdouble uz=u*zetan;\r\n\r\n\t\tif (uz<1.0)\r\n\t\t{\r\n\t\t\treturn 0;\r\n\t\t}\r\n\r\n\t\tif (uz<1.0+Math.pow(0.5,theta)) \r\n\t\t{\r\n\t\t\treturn 1;\r\n\t\t}\r\n\r\n\t\tlong ret=base+(long)((itemcount) * Math.pow(eta*u - eta + 1, alpha));\r\n\t\tsetLastInt((int)ret);\r\n\t\treturn ret;\r\n\t}\r\n\r\n\t/**\r\n\t * Return the next value, skewed by the Zipfian distribution. The 0th item will be the most popular, followed by the 1st, followed\r\n\t * by the 2nd, etc. (Or, if min != 0, the min-th item is the most popular, the min+1th item the next most popular, etc.) If you want the\r\n\t * popular items scattered throughout the item space, use ScrambledZipfianGenerator instead.\r\n\t */\r\n\t@Override\r\n\tpublic int nextInt() \r\n\t{\r\n\t\treturn (int)nextLong(items);\r\n\t}\r\n\r\n\t/**\r\n\t * Return the next value, skewed by the Zipfian distribution. The 0th item will be the most popular, followed by the 1st, followed\r\n\t * by the 2nd, etc. (Or, if min != 0, the min-th item is the most popular, the min+1th item the next most popular, etc.) If you want the\r\n\t * popular items scattered throughout the item space, use ScrambledZipfianGenerator instead.\r\n\t */\r\n\tpublic long nextLong()\r\n\t{\r\n\t\treturn nextLong(items);\r\n\t}\r\n\t\r\n\tpublic static void main(String[] args)\r\n\t{\r\n\t\tnew ZipfianGenerator(ScrambledZipfianGenerator.ITEM_COUNT);\r\n\t}\r\n\r\n\t/**\r\n\t * @todo Implement ZipfianGenerator.mean()\r\n\t */\r\n\t@Override\r\n\tpublic double mean() {\r\n\t\tthrow new UnsupportedOperationException(\"@todo implement ZipfianGenerator.mean()\");\r\n\t}\r\n}\r\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/measurements/Measurements.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\n\npackage com.yahoo.ycsb.measurements;\n\nimport java.io.IOException;\nimport java.util.HashMap;\nimport java.util.Properties;\n\nimport com.yahoo.ycsb.measurements.exporter.MeasurementsExporter;\n\n/**\n * Collects latency measurements, and reports them when requested.\n * \n * @author cooperb\n *\n */\npublic class Measurements\n{\n\tprivate static final String MEASUREMENT_TYPE = \"measurementtype\";\n\n\tprivate static final String MEASUREMENT_TYPE_DEFAULT = \"histogram\";\n\n\tstatic Measurements singleton=null;\n\t\n\tstatic Properties measurementproperties=null;\n\t\n\tpublic static void setProperties(Properties props)\n\t{\n\t\tmeasurementproperties=props;\n\t}\n\n      /**\n       * Return the singleton Measurements object.\n       */\n\tpublic synchronized static Measurements getMeasurements()\n\t{\n\t\tif (singleton==null)\n\t\t{\n\t\t\tsingleton=new Measurements(measurementproperties);\n\t\t}\n\t\treturn singleton;\n\t}\n\n\tHashMap<String,OneMeasurement> data;\n\tboolean histogram=true;\n\n\tprivate Properties _props;\n\t\n      /**\n       * Create a new object with the specified properties.\n       */\n\tpublic Measurements(Properties props)\n\t{\n\t\tdata=new HashMap<String,OneMeasurement>();\n\t\t\n\t\t_props=props;\n\t\t\n\t\tif (_props.getProperty(MEASUREMENT_TYPE, MEASUREMENT_TYPE_DEFAULT).compareTo(\"histogram\")==0)\n\t\t{\n\t\t\thistogram=true;\n\t\t}\n\t\telse\n\t\t{\n\t\t\thistogram=false;\n\t\t}\n\t}\n\t\n\tOneMeasurement constructOneMeasurement(String name)\n\t{\n\t\tif (histogram)\n\t\t{\n\t\t\treturn new OneMeasurementHistogram(name,_props);\n\t\t}\n\t\telse\n\t\t{\n\t\t\treturn new OneMeasurementTimeSeries(name,_props);\n\t\t}\n\t}\n\n      /**\n       * Report a single value of a single metric. E.g. for read latency, operation=\"READ\" and latency is the measured value.\n       */\n\tpublic synchronized void measure(String operation, int latency)\n\t{\n\t\tif (!data.containsKey(operation))\n\t\t{\n\t\t\tsynchronized(this)\n\t\t\t{\n\t\t\t\tif (!data.containsKey(operation))\n\t\t\t\t{\n\t\t\t\t\tdata.put(operation,constructOneMeasurement(operation));\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\ttry\n\t\t{\n\t\t\tdata.get(operation).measure(latency);\n\t\t}\n\t\tcatch (java.lang.ArrayIndexOutOfBoundsException e)\n\t\t{\n\t\t\tSystem.out.println(\"ERROR: java.lang.ArrayIndexOutOfBoundsException - ignoring and continuing\");\n\t\t\te.printStackTrace();\n\t\t\te.printStackTrace(System.out);\n\t\t}\n\t}\n\n      /**\n       * Report a return code for a single DB operaiton.\n       */\n\tpublic void reportReturnCode(String operation, int code)\n\t{\n\t\tif (!data.containsKey(operation))\n\t\t{\n\t\t\tsynchronized(this)\n\t\t\t{\n\t\t\t\tif (!data.containsKey(operation))\n\t\t\t\t{\n\t\t\t\t\tdata.put(operation,constructOneMeasurement(operation));\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tdata.get(operation).reportReturnCode(code);\n\t}\n\t\n  /**\n   * Export the current measurements to a suitable format.\n   * \n   * @param exporter Exporter representing the type of format to write to.\n   * @throws IOException Thrown if the export failed.\n   */\n  public void exportMeasurements(MeasurementsExporter exporter) throws IOException\n  {\n    for (OneMeasurement measurement : data.values())\n    {\n      measurement.exportMeasurements(exporter);\n    }\n  }\n\t\n      /**\n       * Return a one line summary of the measurements.\n       */\n\tpublic String getSummary()\n\t{\n\t\tString ret=\"\";\n\t\tfor (OneMeasurement m : data.values())\n\t\t{\n\t\t\tret+=m.getSummary()+\" \";\n\t\t}\n\t\t\n\t\treturn ret;\n\t}\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/measurements/OneMeasurement.java",
    "content": "/**                                                                                                                                                                                \r\n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \r\n * may not use this file except in compliance with the License. You                                                                                                                \r\n * may obtain a copy of the License at                                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \r\n *                                                                                                                                                                                 \r\n * Unless required by applicable law or agreed to in writing, software                                                                                                             \r\n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \r\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \r\n * implied. See the License for the specific language governing                                                                                                                    \r\n * permissions and limitations under the License. See accompanying                                                                                                                 \r\n * LICENSE file.                                                                                                                                                                   \r\n */\r\n\r\npackage com.yahoo.ycsb.measurements;\r\n\r\nimport java.io.IOException;\r\n\r\nimport com.yahoo.ycsb.measurements.exporter.MeasurementsExporter;\r\n\r\n/**\r\n * A single measured metric (such as READ LATENCY)\r\n */\r\npublic abstract class OneMeasurement {\r\n\r\n\tString _name;\r\n\t\r\n\tpublic String getName() {\r\n\t\treturn _name;\r\n\t}\r\n\r\n\t/**\r\n\t * @param _name\r\n\t */\r\n\tpublic OneMeasurement(String _name) {\r\n\t\tthis._name = _name;\r\n\t}\r\n\r\n\tpublic abstract void reportReturnCode(int code);\r\n\r\n\tpublic abstract void measure(int latency);\r\n\r\n\tpublic abstract String getSummary();\r\n\r\n  /**\r\n   * Export the current measurements to a suitable format.\r\n   * \r\n   * @param exporter Exporter representing the type of format to write to.\r\n   * @throws IOException Thrown if the export failed.\r\n   */\r\n  public abstract void exportMeasurements(MeasurementsExporter exporter) throws IOException;\r\n}\r\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/measurements/OneMeasurementHistogram.java",
    "content": "/**                                                                                                                                                                                \r\n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \r\n * may not use this file except in compliance with the License. You                                                                                                                \r\n * may obtain a copy of the License at                                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \r\n *                                                                                                                                                                                 \r\n * Unless required by applicable law or agreed to in writing, software                                                                                                             \r\n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \r\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \r\n * implied. See the License for the specific language governing                                                                                                                    \r\n * permissions and limitations under the License. See accompanying                                                                                                                 \r\n * LICENSE file.                                                                                                                                                                   \r\n */\r\n\r\npackage com.yahoo.ycsb.measurements;\r\n\r\nimport java.io.IOException;\r\nimport java.text.DecimalFormat;\r\nimport java.util.HashMap;\r\nimport java.util.Properties;\r\n\r\nimport com.yahoo.ycsb.measurements.exporter.MeasurementsExporter;\r\n\r\n\r\n/**\r\n * Take measurements and maintain a histogram of a given metric, such as READ LATENCY.\r\n * \r\n * @author cooperb\r\n *\r\n */\r\npublic class OneMeasurementHistogram extends OneMeasurement\r\n{\r\n\tpublic static final String BUCKETS=\"histogram.buckets\";\r\n\tpublic static final String BUCKETS_DEFAULT=\"1000\";\r\n\r\n\tint _buckets;\r\n\tint[] histogram;\r\n\tint histogramoverflow;\r\n\tint operations;\r\n\tlong totallatency;\r\n\t\r\n\t//keep a windowed version of these stats for printing status\r\n\tint windowoperations;\r\n\tlong windowtotallatency;\r\n\t\r\n\tint min;\r\n\tint max;\r\n\tHashMap<Integer,int[]> returncodes;\r\n\r\n\tpublic OneMeasurementHistogram(String name, Properties props)\r\n\t{\r\n\t\tsuper(name);\r\n\t\t_buckets=Integer.parseInt(props.getProperty(BUCKETS, BUCKETS_DEFAULT));\r\n\t\thistogram=new int[_buckets];\r\n\t\thistogramoverflow=0;\r\n\t\toperations=0;\r\n\t\ttotallatency=0;\r\n\t\twindowoperations=0;\r\n\t\twindowtotallatency=0;\r\n\t\tmin=-1;\r\n\t\tmax=-1;\r\n\t\treturncodes=new HashMap<Integer,int[]>();\r\n\t}\r\n\r\n\t/* (non-Javadoc)\r\n\t * @see com.yahoo.ycsb.OneMeasurement#reportReturnCode(int)\r\n\t */\r\n\tpublic synchronized void reportReturnCode(int code)\r\n\t{\r\n\t\tInteger Icode=code;\r\n\t\tif (!returncodes.containsKey(Icode))\r\n\t\t{\r\n\t\t\tint[] val=new int[1];\r\n\t\t\tval[0]=0;\r\n\t\t\treturncodes.put(Icode,val);\r\n\t\t}\r\n\t\treturncodes.get(Icode)[0]++;\r\n\t}\r\n\r\n\r\n\t/* (non-Javadoc)\r\n\t * @see com.yahoo.ycsb.OneMeasurement#measure(int)\r\n\t */\r\n\tpublic synchronized void measure(int latency)\r\n\t{\r\n\t\tif (latency/1000>=_buckets)\r\n\t\t{\r\n\t\t\thistogramoverflow++;\r\n\t\t}\r\n\t\telse\r\n\t\t{\r\n\t\t\thistogram[latency/1000]++;\r\n\t\t}\r\n\t\toperations++;\r\n\t\ttotallatency+=latency;\r\n\t\twindowoperations++;\r\n\t\twindowtotallatency+=latency;\r\n\r\n\t\tif ( (min<0) || (latency<min) )\r\n\t\t{\r\n\t\t\tmin=latency;\r\n\t\t}\r\n\r\n\t\tif ( (max<0) || (latency>max) )\r\n\t\t{\r\n\t\t\tmax=latency;\r\n\t\t}\r\n\t}\r\n\r\n\r\n  @Override\r\n  public void exportMeasurements(MeasurementsExporter exporter) throws IOException\r\n  {\r\n    exporter.write(getName(), \"Operations\", operations);\r\n    exporter.write(getName(), \"AverageLatency(us)\", (((double)totallatency)/((double)operations)));\r\n    exporter.write(getName(), \"MinLatency(us)\", min);\r\n    exporter.write(getName(), \"MaxLatency(us)\", max);\r\n    \r\n    int opcounter=0;\r\n    boolean done95th=false;\r\n    for (int i=0; i<_buckets; i++)\r\n    {\r\n      opcounter+=histogram[i];\r\n      if ( (!done95th) && (((double)opcounter)/((double)operations)>=0.95) )\r\n      {\r\n        exporter.write(getName(), \"95thPercentileLatency(ms)\", i);\r\n        done95th=true;\r\n      }\r\n      if (((double)opcounter)/((double)operations)>=0.99)\r\n      {\r\n        exporter.write(getName(), \"99thPercentileLatency(ms)\", i);\r\n        break;\r\n      }\r\n    }\r\n\r\n    for (Integer I : returncodes.keySet())\r\n    {\r\n      int[] val=returncodes.get(I);\r\n      exporter.write(getName(), \"Return=\"+I, val[0]);\r\n    }     \r\n\r\n    for (int i=0; i<_buckets; i++)\r\n    {\r\n      exporter.write(getName(), Integer.toString(i), histogram[i]);\r\n    }\r\n    exporter.write(getName(), \">\"+_buckets, histogramoverflow);\r\n  }\r\n\r\n\t@Override\r\n\tpublic String getSummary() {\r\n\t\tif (windowoperations==0)\r\n\t\t{\r\n\t\t\treturn \"\";\r\n\t\t}\r\n\t\tDecimalFormat d = new DecimalFormat(\"#.##\");\r\n\t\tdouble report=((double)windowtotallatency)/((double)windowoperations);\r\n\t\twindowtotallatency=0;\r\n\t\twindowoperations=0;\r\n\t\treturn \"[\"+getName()+\" AverageLatency(us)=\"+d.format(report)+\"]\";\r\n\t}\r\n\r\n}\r\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.java",
    "content": "/**                                                                                                                                                                                \r\n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \r\n * may not use this file except in compliance with the License. You                                                                                                                \r\n * may obtain a copy of the License at                                                                                                                                             \r\n *                                                                                                                                                                                 \r\n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \r\n *                                                                                                                                                                                 \r\n * Unless required by applicable law or agreed to in writing, software                                                                                                             \r\n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \r\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \r\n * implied. See the License for the specific language governing                                                                                                                    \r\n * permissions and limitations under the License. See accompanying                                                                                                                 \r\n * LICENSE file.                                                                                                                                                                   \r\n */\r\n\r\npackage com.yahoo.ycsb.measurements;\r\n\r\nimport java.io.IOException;\r\nimport java.text.DecimalFormat;\r\nimport java.util.HashMap;\r\nimport java.util.Properties;\r\nimport java.util.Vector;\r\n\r\nimport com.yahoo.ycsb.measurements.exporter.MeasurementsExporter;\r\n\r\nclass SeriesUnit\r\n{\r\n\t/**\r\n\t * @param time\r\n\t * @param average\r\n\t */\r\n\tpublic SeriesUnit(long time, double average) {\r\n\t\tthis.time = time;\r\n\t\tthis.average = average;\r\n\t}\r\n\tpublic long time;\r\n\tpublic double average; \r\n}\r\n\r\n/**\r\n * A time series measurement of a metric, such as READ LATENCY.\r\n */\r\npublic class OneMeasurementTimeSeries extends OneMeasurement \r\n{\r\n\t/**\r\n\t * Granularity for time series; measurements will be averaged in chunks of this granularity. Units are milliseconds.\r\n\t */\r\n\tpublic static final String GRANULARITY=\"timeseries.granularity\";\r\n\t\r\n\tpublic static final String GRANULARITY_DEFAULT=\"1000\";\r\n\t\r\n\tint _granularity;\r\n\tVector<SeriesUnit> _measurements;\r\n\t\r\n\tlong start=-1;\r\n\tlong currentunit=-1;\r\n\tint count=0;\r\n\tint sum=0;\r\n\tint operations=0;\r\n\tlong totallatency=0;\r\n\t\r\n\t//keep a windowed version of these stats for printing status\r\n\tint windowoperations=0;\r\n\tlong windowtotallatency=0;\r\n\t\r\n\tint min=-1;\r\n\tint max=-1;\r\n\r\n\tprivate HashMap<Integer, int[]> returncodes;\r\n\t\r\n\tpublic OneMeasurementTimeSeries(String name, Properties props)\r\n\t{\r\n\t\tsuper(name);\r\n\t\t_granularity=Integer.parseInt(props.getProperty(GRANULARITY,GRANULARITY_DEFAULT));\r\n\t\t_measurements=new Vector<SeriesUnit>();\r\n\t\treturncodes=new HashMap<Integer,int[]>();\r\n\t}\r\n\t\r\n\tvoid checkEndOfUnit(boolean forceend)\r\n\t{\r\n\t\tlong now=System.currentTimeMillis();\r\n\t\t\r\n\t\tif (start<0)\r\n\t\t{\r\n\t\t\tcurrentunit=0;\r\n\t\t\tstart=now;\r\n\t\t}\r\n\t\t\r\n\t\tlong unit=((now-start)/_granularity)*_granularity;\r\n\t\t\r\n\t\tif ( (unit>currentunit) || (forceend) )\r\n\t\t{\r\n\t\t\tdouble avg=((double)sum)/((double)count);\r\n\t\t\t_measurements.add(new SeriesUnit(currentunit,avg));\r\n\t\t\t\r\n\t\t\tcurrentunit=unit;\r\n\t\t\t\r\n\t\t\tcount=0;\r\n\t\t\tsum=0;\r\n\t\t}\r\n\t}\r\n\t\r\n\t@Override\r\n\tpublic void measure(int latency) \r\n\t{\r\n\t\tcheckEndOfUnit(false);\r\n\t\t\r\n\t\tcount++;\r\n\t\tsum+=latency;\r\n\t\ttotallatency+=latency;\r\n\t\toperations++;\r\n\t\twindowoperations++;\r\n\t\twindowtotallatency+=latency;\r\n\t\t\r\n\t\tif (latency>max)\r\n\t\t{\r\n\t\t\tmax=latency;\r\n\t\t}\r\n\t\t\r\n\t\tif ( (latency<min) || (min<0) )\r\n\t\t{\r\n\t\t\tmin=latency;\r\n\t\t}\r\n\t}\r\n\r\n\r\n  @Override\r\n  public void exportMeasurements(MeasurementsExporter exporter) throws IOException\r\n  {\r\n    checkEndOfUnit(true);\r\n\r\n    exporter.write(getName(), \"Operations\", operations);\r\n    exporter.write(getName(), \"AverageLatency(us)\", (((double)totallatency)/((double)operations)));\r\n    exporter.write(getName(), \"MinLatency(us)\", min);\r\n    exporter.write(getName(), \"MaxLatency(us)\", max);\r\n\r\n    //TODO: 95th and 99th percentile latency\r\n\r\n    for (Integer I : returncodes.keySet())\r\n    {\r\n      int[] val=returncodes.get(I);\r\n      exporter.write(getName(), \"Return=\"+I, val[0]);\r\n    }     \r\n\r\n    for (SeriesUnit unit : _measurements)\r\n    {\r\n      exporter.write(getName(), Long.toString(unit.time), unit.average);\r\n    }\r\n  }\r\n\t\r\n\t@Override\r\n\tpublic void reportReturnCode(int code) {\r\n\t\tInteger Icode=code;\r\n\t\tif (!returncodes.containsKey(Icode))\r\n\t\t{\r\n\t\t\tint[] val=new int[1];\r\n\t\t\tval[0]=0;\r\n\t\t\treturncodes.put(Icode,val);\r\n\t\t}\r\n\t\treturncodes.get(Icode)[0]++;\r\n\r\n\t}\r\n\r\n\t@Override\r\n\tpublic String getSummary() {\r\n\t\tif (windowoperations==0)\r\n\t\t{\r\n\t\t\treturn \"\";\r\n\t\t}\r\n\t\tDecimalFormat d = new DecimalFormat(\"#.##\");\r\n\t\tdouble report=((double)windowtotallatency)/((double)windowoperations);\r\n\t\twindowtotallatency=0;\r\n\t\twindowoperations=0;\r\n\t\treturn \"[\"+getName()+\" AverageLatency(us)=\"+d.format(report)+\"]\";\r\n\t}\r\n\r\n}\r\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/measurements/exporter/JSONMeasurementsExporter.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\npackage com.yahoo.ycsb.measurements.exporter;\n\nimport java.io.BufferedWriter;\nimport java.io.IOException;\nimport java.io.OutputStream;\nimport java.io.OutputStreamWriter;\n\nimport org.codehaus.jackson.JsonFactory;\nimport org.codehaus.jackson.JsonGenerator;\nimport org.codehaus.jackson.impl.DefaultPrettyPrinter;\n\n/**\n * Export measurements into a machine readable JSON file.\n */\npublic class JSONMeasurementsExporter implements MeasurementsExporter\n{\n\n  private JsonFactory factory = new JsonFactory();\n  private JsonGenerator g;\n\n  public JSONMeasurementsExporter(OutputStream os) throws IOException\n  {\n\n    BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(os));\n    g = factory.createJsonGenerator(bw);\n    g.setPrettyPrinter(new DefaultPrettyPrinter());\n  }\n\n  public void write(String metric, String measurement, int i) throws IOException\n  {\n    g.writeStartObject();\n    g.writeStringField(\"metric\", metric);\n    g.writeStringField(\"measurement\", measurement);\n    g.writeNumberField(\"value\", i);\n    g.writeEndObject();\n  }\n\n  public void write(String metric, String measurement, double d) throws IOException\n  {\n    g.writeStartObject();\n    g.writeStringField(\"metric\", metric);\n    g.writeStringField(\"measurement\", measurement);\n    g.writeNumberField(\"value\", d);\n    g.writeEndObject();\n  }\n\n  public void close() throws IOException\n  {\n    if (g != null)\n    {\n      g.close();\n    }\n  }\n\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/measurements/exporter/MeasurementsExporter.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\npackage com.yahoo.ycsb.measurements.exporter;\n\nimport java.io.Closeable;\nimport java.io.IOException;\n\n/**\n * Used to export the collected measurements into a useful format, for example\n * human readable text or machine readable JSON.\n */\npublic interface MeasurementsExporter extends Closeable\n{\n\n  /**\n   * Write a measurement to the exported format.\n   * \n   * @param metric Metric name, for example \"READ LATENCY\".\n   * @param measurement Measurement name, for example \"Average latency\".\n   * @param i Measurement to write.\n   * @throws IOException if writing failed\n   */\n  public void write(String metric, String measurement, int i) throws IOException;\n\n  /**\n   * Write a measurement to the exported format.\n   * \n   * @param metric Metric name, for example \"READ LATENCY\".\n   * @param measurement Measurement name, for example \"Average latency\".\n   * @param d Measurement to write.\n   * @throws IOException if writing failed\n   */\n  public void write(String metric, String measurement, double d) throws IOException;\n\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/measurements/exporter/TextMeasurementsExporter.java",
    "content": "/**                                                                                                                                                                                \n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.                                                                                                                                                                   \n */\npackage com.yahoo.ycsb.measurements.exporter;\n\nimport java.io.BufferedWriter;\nimport java.io.IOException;\nimport java.io.OutputStream;\nimport java.io.OutputStreamWriter;\n\n/**\n * Write human readable text. Tries to emulate the previous print report method.\n */\npublic class TextMeasurementsExporter implements MeasurementsExporter\n{\n\n  private BufferedWriter bw;\n\n  public TextMeasurementsExporter(OutputStream os)\n  {\n    this.bw = new BufferedWriter(new OutputStreamWriter(os));\n  }\n\n  public void write(String metric, String measurement, int i) throws IOException\n  {\n    bw.write(\"[\" + metric + \"], \" + measurement + \", \" + i);\n    bw.newLine();\n  }\n\n  public void write(String metric, String measurement, double d) throws IOException\n  {\n    bw.write(\"[\" + metric + \"], \" + measurement + \", \" + d);\n    bw.newLine();\n  }\n\n  public void close() throws IOException\n  {\n    this.bw.close();\n  }\n\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/workloads/ConstantOccupancyWorkload.java",
    "content": "/**\n * Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n *                                                                                                                                                                                 \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n * may not use this file except in compliance with the License. You                                                                                                                \n * may obtain a copy of the License at                                                                                                                                             \n *                                                                                                                                                                                 \n * http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n *                                                                                                                                                                                 \n * Unless required by applicable law or agreed to in writing, software                                                                                                             \n * distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n * implied. See the License for the specific language governing                                                                                                                    \n * permissions and limitations under the License. See accompanying                                                                                                                 \n * LICENSE file.\n */\npackage com.yahoo.ycsb.workloads;\n\nimport java.util.Properties;\n\nimport com.yahoo.ycsb.WorkloadException;\nimport com.yahoo.ycsb.Client;\nimport com.yahoo.ycsb.generator.IntegerGenerator;\n\n/**\n * A disk-fragmenting workload.\n * <p>\n * Properties to control the client:\n * </p>\n * <UL>\n * <LI><b>disksize</b>: how many bytes of storage can the disk store? (default 100,000,000)\n * <LI><b>occupancy</b>: what fraction of the available storage should be used? (default 0.9)\n * <LI><b>requestdistribution</b>: what distribution should be used to select the records to operate on - uniform, zipfian or latest (default: histogram)\n * </ul> \n *\n *\n * <p> See also:\n * Russell Sears, Catharine van Ingen.\n * <a href='https://database.cs.wisc.edu/cidr/cidr2007/papers/cidr07p34.pdf'>Fragmentation in Large Object Repositories</a>,\n * CIDR 2006. [<a href='https://database.cs.wisc.edu/cidr/cidr2007/slides/p34-sears.ppt'>Presentation</a>]\n * </p>\n *\n *\n * @author sears\n *\n */\npublic class ConstantOccupancyWorkload extends CoreWorkload {\n\tlong disksize;\n\tlong storageages;\n\tIntegerGenerator objectsizes;\n\tdouble occupancy;\n\t\n\tlong object_count;\n\t\n\tpublic static final String STORAGE_AGE_PROPERTY = \"storageages\";\n\tpublic static final long   STORAGE_AGE_PROPERTY_DEFAULT = 10;\n\t\n\tpublic static final String DISK_SIZE_PROPERTY = \"disksize\";\n\tpublic static final long   DISK_SIZE_PROPERTY_DEFAULT = 100 * 1000 * 1000;\n\t\n\tpublic static final String OCCUPANCY_PROPERTY = \"occupancy\";\n\tpublic static final double OCCUPANCY_PROPERTY_DEFAULT = 0.9;\n\t\n\t@Override\n\tpublic void init(Properties p) throws WorkloadException\n\t{\n\t\tdisksize    = Long.parseLong(    p.getProperty(DISK_SIZE_PROPERTY, DISK_SIZE_PROPERTY_DEFAULT+\"\"));\n\t\tstorageages = Long.parseLong(    p.getProperty(STORAGE_AGE_PROPERTY, STORAGE_AGE_PROPERTY_DEFAULT+\"\"));\n\t\toccupancy   = Double.parseDouble(p.getProperty(OCCUPANCY_PROPERTY, OCCUPANCY_PROPERTY_DEFAULT+\"\"));\n\t\t\n\t\tif(p.getProperty(Client.RECORD_COUNT_PROPERTY) != null ||\n\t\t   p.getProperty(Client.INSERT_COUNT_PROPERTY) != null ||\n\t\t   p.getProperty(Client.OPERATION_COUNT_PROPERTY) != null) {\n\t\t\tSystem.err.println(\"Warning: record, insert or operation count was set prior to initting ConstantOccupancyWorkload.  Overriding old values.\");\n\t\t}\n\t\tIntegerGenerator g = CoreWorkload.getFieldLengthGenerator(p);\n\t\tdouble fieldsize = g.mean();\n\t\tint fieldcount = Integer.parseInt(p.getProperty(FIELD_COUNT_PROPERTY, FIELD_COUNT_PROPERTY_DEFAULT));\n\n\t\tobject_count = (long)(occupancy * ((double)disksize / (fieldsize * (double)fieldcount)));\n                if(object_count == 0) {\n                    throw new IllegalStateException(\"Object count was zero.  Perhaps disksize is too low?\");\n                }\n\t\tp.setProperty(Client.RECORD_COUNT_PROPERTY, object_count+\"\");\n\t\tp.setProperty(Client.OPERATION_COUNT_PROPERTY, (storageages*object_count)+\"\");\n\t\tp.setProperty(Client.INSERT_COUNT_PROPERTY, object_count+\"\");\n\n\t\tsuper.init(p);\n\t}\n\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/main/java/com/yahoo/ycsb/workloads/CoreWorkload.java",
    "content": "/**\n * Copyright (c) 2010 Yahoo! Inc. All rights reserved. \n * \n * Licensed under the Apache License, Version 2.0 (the \"License\"); you\n * may not use this file except in compliance with the License. You\n * may obtain a copy of the License at\n *\n * http://www.apache.org/licenses/LICENSE-2.0 \n *\n * Unless required by applicable law or agreed to in writing, software\n * distributed under the License is distributed on an \"AS IS\" BASIS,\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n * implied. See the License for the specific language governing\n * permissions and limitations under the License. See accompanying\n * LICENSE file.\n */\n\npackage com.yahoo.ycsb.workloads;\n\nimport java.util.Properties;\nimport com.yahoo.ycsb.*;\nimport com.yahoo.ycsb.generator.CounterGenerator;\nimport com.yahoo.ycsb.generator.DiscreteGenerator;\nimport com.yahoo.ycsb.generator.ExponentialGenerator;\nimport com.yahoo.ycsb.generator.Generator;\nimport com.yahoo.ycsb.generator.ConstantIntegerGenerator;\nimport com.yahoo.ycsb.generator.HotspotIntegerGenerator;\nimport com.yahoo.ycsb.generator.HistogramGenerator;\nimport com.yahoo.ycsb.generator.IntegerGenerator;\nimport com.yahoo.ycsb.generator.ScrambledZipfianGenerator;\nimport com.yahoo.ycsb.generator.SkewedLatestGenerator;\nimport com.yahoo.ycsb.generator.UniformIntegerGenerator;\nimport com.yahoo.ycsb.generator.ZipfianGenerator;\nimport com.yahoo.ycsb.measurements.Measurements;\n\nimport java.io.IOException;\nimport java.util.HashMap;\nimport java.util.HashSet;\nimport java.util.Vector;\n\n/**\n * The core benchmark scenario. Represents a set of clients doing simple CRUD operations. The relative \n * proportion of different kinds of operations, and other properties of the workload, are controlled\n * by parameters specified at runtime.\n * \n * Properties to control the client:\n * <UL>\n * <LI><b>fieldcount</b>: the number of fields in a record (default: 10)\n * <LI><b>fieldlength</b>: the size of each field (default: 100)\n * <LI><b>readallfields</b>: should reads read all fields (true) or just one (false) (default: true)\n * <LI><b>writeallfields</b>: should updates and read/modify/writes update all fields (true) or just one (false) (default: false)\n * <LI><b>readproportion</b>: what proportion of operations should be reads (default: 0.95)\n * <LI><b>updateproportion</b>: what proportion of operations should be updates (default: 0.05)\n * <LI><b>insertproportion</b>: what proportion of operations should be inserts (default: 0)\n * <LI><b>scanproportion</b>: what proportion of operations should be scans (default: 0)\n * <LI><b>readmodifywriteproportion</b>: what proportion of operations should be read a record, modify it, write it back (default: 0)\n * <LI><b>requestdistribution</b>: what distribution should be used to select the records to operate on - uniform, zipfian, hotspot, or latest (default: uniform)\n * <LI><b>maxscanlength</b>: for scans, what is the maximum number of records to scan (default: 1000)\n * <LI><b>scanlengthdistribution</b>: for scans, what distribution should be used to choose the number of records to scan, for each scan, between 1 and maxscanlength (default: uniform)\n * <LI><b>insertorder</b>: should records be inserted in order by key (\"ordered\"), or in hashed order (\"hashed\") (default: hashed)\n * </ul> \n */\npublic class CoreWorkload extends Workload\n{\n\n\t/**\n\t * The name of the database table to run queries against.\n\t */\n\tpublic static final String TABLENAME_PROPERTY=\"table\";\n\n\t/**\n\t * The default name of the database table to run queries against.\n\t */\n\tpublic static final String TABLENAME_PROPERTY_DEFAULT=\"usertable\";\n\n\tpublic static String table;\n\n\n\t/**\n\t * The name of the property for the number of fields in a record.\n\t */\n\tpublic static final String FIELD_COUNT_PROPERTY=\"fieldcount\";\n\t\n\t/**\n\t * Default number of fields in a record.\n\t */\n\tpublic static final String FIELD_COUNT_PROPERTY_DEFAULT=\"10\";\n\n\tint fieldcount;\n\n\t/**\n\t * The name of the property for the field length distribution. Options are \"uniform\", \"zipfian\" (favoring short records), \"constant\", and \"histogram\".\n\t * \n\t * If \"uniform\", \"zipfian\" or \"constant\", the maximum field length will be that specified by the fieldlength property.  If \"histogram\", then the\n\t * histogram will be read from the filename specified in the \"fieldlengthhistogram\" property.\n\t */\n\tpublic static final String FIELD_LENGTH_DISTRIBUTION_PROPERTY=\"fieldlengthdistribution\";\n\t/**\n\t * The default field length distribution.\n\t */\n\tpublic static final String FIELD_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT = \"constant\";\n\n\t/**\n\t * The name of the property for the length of a field in bytes.\n\t */\n\tpublic static final String FIELD_LENGTH_PROPERTY=\"fieldlength\";\n\t/**\n\t * The default maximum length of a field in bytes.\n\t */\n\tpublic static final String FIELD_LENGTH_PROPERTY_DEFAULT=\"100\";\n\n\t/**\n\t * The name of a property that specifies the filename containing the field length histogram (only used if fieldlengthdistribution is \"histogram\").\n\t */\n\tpublic static final String FIELD_LENGTH_HISTOGRAM_FILE_PROPERTY = \"fieldlengthhistogram\";\n\t/**\n\t * The default filename containing a field length histogram.\n\t */\n\tpublic static final String FIELD_LENGTH_HISTOGRAM_FILE_PROPERTY_DEFAULT = \"hist.txt\";\n\n\t/**\n\t * Generator object that produces field lengths.  The value of this depends on the properties that start with \"FIELD_LENGTH_\".\n\t */\n\tIntegerGenerator fieldlengthgenerator;\n\t\n\t/**\n\t * The name of the property for deciding whether to read one field (false) or all fields (true) of a record.\n\t */\n\tpublic static final String READ_ALL_FIELDS_PROPERTY=\"readallfields\";\n\t\n\t/**\n\t * The default value for the readallfields property.\n\t */\n\tpublic static final String READ_ALL_FIELDS_PROPERTY_DEFAULT=\"true\";\n\n\tboolean readallfields;\n\n\t/**\n\t * The name of the property for deciding whether to write one field (false) or all fields (true) of a record.\n\t */\n\tpublic static final String WRITE_ALL_FIELDS_PROPERTY=\"writeallfields\";\n\t\n\t/**\n\t * The default value for the writeallfields property.\n\t */\n\tpublic static final String WRITE_ALL_FIELDS_PROPERTY_DEFAULT=\"false\";\n\n\tboolean writeallfields;\n\n\n\t/**\n\t * The name of the property for the proportion of transactions that are reads.\n\t */\n\tpublic static final String READ_PROPORTION_PROPERTY=\"readproportion\";\n\t\n\t/**\n\t * The default proportion of transactions that are reads.\t\n\t */\n\tpublic static final String READ_PROPORTION_PROPERTY_DEFAULT=\"0.95\";\n\n\t/**\n\t * The name of the property for the proportion of transactions that are updates.\n\t */\n\tpublic static final String UPDATE_PROPORTION_PROPERTY=\"updateproportion\";\n\t\n\t/**\n\t * The default proportion of transactions that are updates.\n\t */\n\tpublic static final String UPDATE_PROPORTION_PROPERTY_DEFAULT=\"0.05\";\n\n\t/**\n\t * The name of the property for the proportion of transactions that are inserts.\n\t */\n\tpublic static final String INSERT_PROPORTION_PROPERTY=\"insertproportion\";\n\t\n\t/**\n\t * The default proportion of transactions that are inserts.\n\t */\n\tpublic static final String INSERT_PROPORTION_PROPERTY_DEFAULT=\"0.0\";\n\n\t/**\n\t * The name of the property for the proportion of transactions that are scans.\n\t */\n\tpublic static final String SCAN_PROPORTION_PROPERTY=\"scanproportion\";\n\t\n\t/**\n\t * The default proportion of transactions that are scans.\n\t */\n\tpublic static final String SCAN_PROPORTION_PROPERTY_DEFAULT=\"0.0\";\n\t\n\t/**\n\t * The name of the property for the proportion of transactions that are read-modify-write.\n\t */\n\tpublic static final String READMODIFYWRITE_PROPORTION_PROPERTY=\"readmodifywriteproportion\";\n\t\n\t/**\n\t * The default proportion of transactions that are scans.\n\t */\n\tpublic static final String READMODIFYWRITE_PROPORTION_PROPERTY_DEFAULT=\"0.0\";\n\t\n\t/**\n\t * The name of the property for the the distribution of requests across the keyspace. Options are \"uniform\", \"zipfian\" and \"latest\"\n\t */\n\tpublic static final String REQUEST_DISTRIBUTION_PROPERTY=\"requestdistribution\";\n\t\n\t/**\n\t * The default distribution of requests across the keyspace\n\t */\n\tpublic static final String REQUEST_DISTRIBUTION_PROPERTY_DEFAULT=\"uniform\";\n\n\t/**\n\t * The name of the property for the max scan length (number of records)\n\t */\n\tpublic static final String MAX_SCAN_LENGTH_PROPERTY=\"maxscanlength\";\n\t\n\t/**\n\t * The default max scan length.\n\t */\n\tpublic static final String MAX_SCAN_LENGTH_PROPERTY_DEFAULT=\"1000\";\n\t\n\t/**\n\t * The name of the property for the scan length distribution. Options are \"uniform\" and \"zipfian\" (favoring short scans)\n\t */\n\tpublic static final String SCAN_LENGTH_DISTRIBUTION_PROPERTY=\"scanlengthdistribution\";\n\t\n\t/**\n\t * The default max scan length.\n\t */\n\tpublic static final String SCAN_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT=\"uniform\";\n\t\n\t/**\n\t * The name of the property for the order to insert records. Options are \"ordered\" or \"hashed\"\n\t */\n\tpublic static final String INSERT_ORDER_PROPERTY=\"insertorder\";\n\t\n\t/**\n\t * Default insert order.\n\t */\n\tpublic static final String INSERT_ORDER_PROPERTY_DEFAULT=\"hashed\";\n\t\n\t/**\n   * Percentage data items that constitute the hot set.\n   */\n  public static final String HOTSPOT_DATA_FRACTION = \"hotspotdatafraction\";\n  \n  /**\n   * Default value of the size of the hot set.\n   */\n  public static final String HOTSPOT_DATA_FRACTION_DEFAULT = \"0.2\";\n  \n  /**\n   * Percentage operations that access the hot set.\n   */\n  public static final String HOTSPOT_OPN_FRACTION = \"hotspotopnfraction\";\n  \n  /**\n   * Default value of the percentage operations accessing the hot set.\n   */\n  public static final String HOTSPOT_OPN_FRACTION_DEFAULT = \"0.8\";\n\t\n\tIntegerGenerator keysequence;\n\n\tDiscreteGenerator operationchooser;\n\n\tIntegerGenerator keychooser;\n\n\tGenerator fieldchooser;\n\n\tCounterGenerator transactioninsertkeysequence;\n\t\n\tIntegerGenerator scanlength;\n\t\n\tboolean orderedinserts;\n\n\tint recordcount;\n\t\n\tprotected static IntegerGenerator getFieldLengthGenerator(Properties p) throws WorkloadException{\n\t\tIntegerGenerator fieldlengthgenerator;\n\t\tString fieldlengthdistribution = p.getProperty(FIELD_LENGTH_DISTRIBUTION_PROPERTY, FIELD_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT);\n\t\tint fieldlength=Integer.parseInt(p.getProperty(FIELD_LENGTH_PROPERTY,FIELD_LENGTH_PROPERTY_DEFAULT));\n\t\tString fieldlengthhistogram = p.getProperty(FIELD_LENGTH_HISTOGRAM_FILE_PROPERTY, FIELD_LENGTH_HISTOGRAM_FILE_PROPERTY_DEFAULT);\n\t\tif(fieldlengthdistribution.compareTo(\"constant\") == 0) {\n\t\t\tfieldlengthgenerator = new ConstantIntegerGenerator(fieldlength);\n\t\t} else if(fieldlengthdistribution.compareTo(\"uniform\") == 0) {\n\t\t\tfieldlengthgenerator = new UniformIntegerGenerator(1, fieldlength);\n\t\t} else if(fieldlengthdistribution.compareTo(\"zipfian\") == 0) {\n\t\t\tfieldlengthgenerator = new ZipfianGenerator(1, fieldlength);\n\t\t} else if(fieldlengthdistribution.compareTo(\"histogram\") == 0) {\n\t\t\ttry {\n\t\t\t\tfieldlengthgenerator = new HistogramGenerator(fieldlengthhistogram);\n\t\t\t} catch(IOException e) {\n\t\t\t\tthrow new WorkloadException(\"Couldn't read field length histogram file: \"+fieldlengthhistogram, e);\n\t\t\t}\n\t\t} else {\n\t\t\tthrow new WorkloadException(\"Unknown field length distribution \\\"\"+fieldlengthdistribution+\"\\\"\");\n\t\t}\n\t\treturn fieldlengthgenerator;\n\t}\n\t\n\t/**\n\t * Initialize the scenario. \n\t * Called once, in the main client thread, before any operations are started.\n\t */\n\tpublic void init(Properties p) throws WorkloadException\n\t{\n\t\ttable = p.getProperty(TABLENAME_PROPERTY,TABLENAME_PROPERTY_DEFAULT);\n\t\t\n\t\tfieldcount=Integer.parseInt(p.getProperty(FIELD_COUNT_PROPERTY,FIELD_COUNT_PROPERTY_DEFAULT));\n\t\tfieldlengthgenerator = CoreWorkload.getFieldLengthGenerator(p);\n\t\t\n\t\tdouble readproportion=Double.parseDouble(p.getProperty(READ_PROPORTION_PROPERTY,READ_PROPORTION_PROPERTY_DEFAULT));\n\t\tdouble updateproportion=Double.parseDouble(p.getProperty(UPDATE_PROPORTION_PROPERTY,UPDATE_PROPORTION_PROPERTY_DEFAULT));\n\t\tdouble insertproportion=Double.parseDouble(p.getProperty(INSERT_PROPORTION_PROPERTY,INSERT_PROPORTION_PROPERTY_DEFAULT));\n\t\tdouble scanproportion=Double.parseDouble(p.getProperty(SCAN_PROPORTION_PROPERTY,SCAN_PROPORTION_PROPERTY_DEFAULT));\n\t\tdouble readmodifywriteproportion=Double.parseDouble(p.getProperty(READMODIFYWRITE_PROPORTION_PROPERTY,READMODIFYWRITE_PROPORTION_PROPERTY_DEFAULT));\n\t\trecordcount=Integer.parseInt(p.getProperty(Client.RECORD_COUNT_PROPERTY));\n\t\tString requestdistrib=p.getProperty(REQUEST_DISTRIBUTION_PROPERTY,REQUEST_DISTRIBUTION_PROPERTY_DEFAULT);\n\t\tint maxscanlength=Integer.parseInt(p.getProperty(MAX_SCAN_LENGTH_PROPERTY,MAX_SCAN_LENGTH_PROPERTY_DEFAULT));\n\t\tString scanlengthdistrib=p.getProperty(SCAN_LENGTH_DISTRIBUTION_PROPERTY,SCAN_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT);\n\t\t\n\t\tint insertstart=Integer.parseInt(p.getProperty(INSERT_START_PROPERTY,INSERT_START_PROPERTY_DEFAULT));\n\t\t\n\t\treadallfields=Boolean.parseBoolean(p.getProperty(READ_ALL_FIELDS_PROPERTY,READ_ALL_FIELDS_PROPERTY_DEFAULT));\n\t\twriteallfields=Boolean.parseBoolean(p.getProperty(WRITE_ALL_FIELDS_PROPERTY,WRITE_ALL_FIELDS_PROPERTY_DEFAULT));\n\t\t\n\t\tif (p.getProperty(INSERT_ORDER_PROPERTY,INSERT_ORDER_PROPERTY_DEFAULT).compareTo(\"hashed\")==0)\n\t\t{\n\t\t\torderedinserts=false;\n\t\t}\n\t\telse if (requestdistrib.compareTo(\"exponential\")==0)\n\t\t{\n                    double percentile = Double.parseDouble(p.getProperty(ExponentialGenerator.EXPONENTIAL_PERCENTILE_PROPERTY,\n                                                                         ExponentialGenerator.EXPONENTIAL_PERCENTILE_DEFAULT));\n                    double frac       = Double.parseDouble(p.getProperty(ExponentialGenerator.EXPONENTIAL_FRAC_PROPERTY,\n                                                                         ExponentialGenerator.EXPONENTIAL_FRAC_DEFAULT));\n                    keychooser = new ExponentialGenerator(percentile, recordcount*frac);\n\t\t}\n\t\telse\n\t\t{\n\t\t\torderedinserts=true;\n\t\t}\n\n\t\tkeysequence=new CounterGenerator(insertstart);\n\t\toperationchooser=new DiscreteGenerator();\n\t\tif (readproportion>0)\n\t\t{\n\t\t\toperationchooser.addValue(readproportion,\"READ\");\n\t\t}\n\n\t\tif (updateproportion>0)\n\t\t{\n\t\t\toperationchooser.addValue(updateproportion,\"UPDATE\");\n\t\t}\n\n\t\tif (insertproportion>0)\n\t\t{\n\t\t\toperationchooser.addValue(insertproportion,\"INSERT\");\n\t\t}\n\t\t\n\t\tif (scanproportion>0)\n\t\t{\n\t\t\toperationchooser.addValue(scanproportion,\"SCAN\");\n\t\t}\n\t\t\n\t\tif (readmodifywriteproportion>0)\n\t\t{\n\t\t\toperationchooser.addValue(readmodifywriteproportion,\"READMODIFYWRITE\");\n\t\t}\n\n\t\ttransactioninsertkeysequence=new CounterGenerator(recordcount);\n\t\tif (requestdistrib.compareTo(\"uniform\")==0)\n\t\t{\n\t\t\tkeychooser=new UniformIntegerGenerator(0,recordcount-1);\n\t\t}\n\t\telse if (requestdistrib.compareTo(\"zipfian\")==0)\n\t\t{\n\t\t\t//it does this by generating a random \"next key\" in part by taking the modulus over the number of keys\n\t\t\t//if the number of keys changes, this would shift the modulus, and we don't want that to change which keys are popular\n\t\t\t//so we'll actually construct the scrambled zipfian generator with a keyspace that is larger than exists at the beginning\n\t\t\t//of the test. that is, we'll predict the number of inserts, and tell the scrambled zipfian generator the number of existing keys\n\t\t\t//plus the number of predicted keys as the total keyspace. then, if the generator picks a key that hasn't been inserted yet, will\n\t\t\t//just ignore it and pick another key. this way, the size of the keyspace doesn't change from the perspective of the scrambled zipfian generator\n\t\t\t\n\t\t\tint opcount=Integer.parseInt(p.getProperty(Client.OPERATION_COUNT_PROPERTY));\n\t\t\tint expectednewkeys=(int)(((double)opcount)*insertproportion*2.0); //2 is fudge factor\n\t\t\t\n\t\t\tkeychooser=new ScrambledZipfianGenerator(recordcount+expectednewkeys);\n\t\t}\n\t\telse if (requestdistrib.compareTo(\"latest\")==0)\n\t\t{\n\t\t\tkeychooser=new SkewedLatestGenerator(transactioninsertkeysequence);\n\t\t}\n\t\telse if (requestdistrib.equals(\"hotspot\")) \n\t\t{\n      double hotsetfraction = Double.parseDouble(p.getProperty(\n          HOTSPOT_DATA_FRACTION, HOTSPOT_DATA_FRACTION_DEFAULT));\n      double hotopnfraction = Double.parseDouble(p.getProperty(\n          HOTSPOT_OPN_FRACTION, HOTSPOT_OPN_FRACTION_DEFAULT));\n      keychooser = new HotspotIntegerGenerator(0, recordcount - 1, \n          hotsetfraction, hotopnfraction);\n    }\n\t\telse\n\t\t{\n\t\t\tthrow new WorkloadException(\"Unknown request distribution \\\"\"+requestdistrib+\"\\\"\");\n\t\t}\n\n\t\tfieldchooser=new UniformIntegerGenerator(0,fieldcount-1);\n\t\t\n\t\tif (scanlengthdistrib.compareTo(\"uniform\")==0)\n\t\t{\n\t\t\tscanlength=new UniformIntegerGenerator(1,maxscanlength);\n\t\t}\n\t\telse if (scanlengthdistrib.compareTo(\"zipfian\")==0)\n\t\t{\n\t\t\tscanlength=new ZipfianGenerator(1,maxscanlength);\n\t\t}\n\t\telse\n\t\t{\n\t\t\tthrow new WorkloadException(\"Distribution \\\"\"+scanlengthdistrib+\"\\\" not allowed for scan length\");\n\t\t}\n\t}\n\n\tpublic String buildKeyName(long keynum) {\n \t\tif (!orderedinserts)\n \t\t{\n \t\t\tkeynum=Utils.hash(keynum);\n \t\t}\n\t\treturn \"user\"+keynum;\n\t}\n\tHashMap<String, ByteIterator> buildValues() {\n \t\tHashMap<String,ByteIterator> values=new HashMap<String,ByteIterator>();\n\n \t\tfor (int i=0; i<fieldcount; i++)\n \t\t{\n \t\t\tString fieldkey=\"field\"+i;\n \t\t\tByteIterator data= new RandomByteIterator(fieldlengthgenerator.nextInt());\n \t\t\tvalues.put(fieldkey,data);\n \t\t}\n\t\treturn values;\n\t}\n\tHashMap<String, ByteIterator> buildUpdate() {\n\t\t//update a random field\n\t\tHashMap<String, ByteIterator> values=new HashMap<String,ByteIterator>();\n\t\tString fieldname=\"field\"+fieldchooser.nextString();\n\t\tByteIterator data = new RandomByteIterator(fieldlengthgenerator.nextInt());\n\t\tvalues.put(fieldname,data);\n\t\treturn values;\n\t}\n\n\t/**\n\t * Do one insert operation. Because it will be called concurrently from multiple client threads, this \n\t * function must be thread safe. However, avoid synchronized, or the threads will block waiting for each \n\t * other, and it will be difficult to reach the target throughput. Ideally, this function would have no side\n\t * effects other than DB operations.\n\t */\n\tpublic boolean doInsert(DB db, Object threadstate)\n\t{\n\t\tint keynum=keysequence.nextInt();\n\t\tString dbkey = buildKeyName(keynum);\n\t\tHashMap<String, ByteIterator> values = buildValues();\n\t\tif (db.insert(table,dbkey,values) == 0)\n\t\t\treturn true;\n\t\telse\n\t\t\treturn false;\n\t}\n\n\t/**\n\t * Do one transaction operation. Because it will be called concurrently from multiple client threads, this \n\t * function must be thread safe. However, avoid synchronized, or the threads will block waiting for each \n\t * other, and it will be difficult to reach the target throughput. Ideally, this function would have no side\n\t * effects other than DB operations.\n\t */\n\tpublic boolean doTransaction(DB db, Object threadstate)\n\t{\n\t\tString op=operationchooser.nextString();\n\n\t\tif (op.compareTo(\"READ\")==0)\n\t\t{\n\t\t\tdoTransactionRead(db);\n\t\t}\n\t\telse if (op.compareTo(\"UPDATE\")==0)\n\t\t{\n\t\t\tdoTransactionUpdate(db);\n\t\t}\n\t\telse if (op.compareTo(\"INSERT\")==0)\n\t\t{\n\t\t\tdoTransactionInsert(db);\n\t\t}\n\t\telse if (op.compareTo(\"SCAN\")==0)\n\t\t{\n\t\t\tdoTransactionScan(db);\n\t\t}\n\t\telse\n\t\t{\n\t\t\tdoTransactionReadModifyWrite(db);\n\t\t}\n\t\t\n\t\treturn true;\n\t}\n\n    int nextKeynum() {\n        int keynum;\n        if(keychooser instanceof ExponentialGenerator) {\n            do\n                {\n                    keynum=transactioninsertkeysequence.lastInt() - keychooser.nextInt();\n                }\n            while(keynum < 0);\n        } else {\n            do\n                {\n                    keynum=keychooser.nextInt();\n                }\n            while (keynum > transactioninsertkeysequence.lastInt());\n        }\n        return keynum;\n    }\n\n\tpublic void doTransactionRead(DB db)\n\t{\n\t\t//choose a random key\n\t\tint keynum = nextKeynum();\n\t\t\n\t\tString keyname = buildKeyName(keynum);\n\t\t\n\t\tHashSet<String> fields=null;\n\n\t\tif (!readallfields)\n\t\t{\n\t\t\t//read a random field  \n\t\t\tString fieldname=\"field\"+fieldchooser.nextString();\n\n\t\t\tfields=new HashSet<String>();\n\t\t\tfields.add(fieldname);\n\t\t}\n\n\t\tdb.read(table,keyname,fields,new HashMap<String,ByteIterator>());\n\t}\n\t\n\tpublic void doTransactionReadModifyWrite(DB db)\n\t{\n\t\t//choose a random key\n\t\tint keynum = nextKeynum();\n\n\t\tString keyname = buildKeyName(keynum);\n\n\t\tHashSet<String> fields=null;\n\n\t\tif (!readallfields)\n\t\t{\n\t\t\t//read a random field  \n\t\t\tString fieldname=\"field\"+fieldchooser.nextString();\n\n\t\t\tfields=new HashSet<String>();\n\t\t\tfields.add(fieldname);\n\t\t}\n\t\t\n\t\tHashMap<String,ByteIterator> values;\n\n\t\tif (writeallfields)\n\t\t{\n\t\t   //new data for all the fields\n\t\t   values = buildValues();\n\t\t}\n\t\telse\n\t\t{\n\t\t   //update a random field\n\t\t   values = buildUpdate();\n\t\t}\n\n\t\t//do the transaction\n\t\t\n\t\tlong st=System.nanoTime();\n\n\t\tdb.read(table,keyname,fields,new HashMap<String,ByteIterator>());\n\t\t\n\t\tdb.update(table,keyname,values);\n\n\t\tlong en=System.nanoTime();\n\t\t\n\t\tMeasurements.getMeasurements().measure(\"READ-MODIFY-WRITE\", (int)((en-st)/1000));\n\t}\n\t\n\tpublic void doTransactionScan(DB db)\n\t{\n\t\t//choose a random key\n\t\tint keynum = nextKeynum();\n\n\t\tString startkeyname = buildKeyName(keynum);\n\t\t\n\t\t//choose a random scan length\n\t\tint len=scanlength.nextInt();\n\n\t\tHashSet<String> fields=null;\n\n\t\tif (!readallfields)\n\t\t{\n\t\t\t//read a random field  \n\t\t\tString fieldname=\"field\"+fieldchooser.nextString();\n\n\t\t\tfields=new HashSet<String>();\n\t\t\tfields.add(fieldname);\n\t\t}\n\n\t\tdb.scan(table,startkeyname,len,fields,new Vector<HashMap<String,ByteIterator>>());\n\t}\n\n\tpublic void doTransactionUpdate(DB db)\n\t{\n\t\t//choose a random key\n\t\tint keynum = nextKeynum();\n\n\t\tString keyname=buildKeyName(keynum);\n\n\t\tHashMap<String,ByteIterator> values;\n\n\t\tif (writeallfields)\n\t\t{\n\t\t   //new data for all the fields\n\t\t   values = buildValues();\n\t\t}\n\t\telse\n\t\t{\n\t\t   //update a random field\n\t\t   values = buildUpdate();\n\t\t}\n\n\t\tdb.update(table,keyname,values);\n\t}\n\n\tpublic void doTransactionInsert(DB db)\n\t{\n\t\t//choose the next key\n\t\tint keynum=transactioninsertkeysequence.nextInt();\n\n\t\tString dbkey = buildKeyName(keynum);\n\n\t\tHashMap<String, ByteIterator> values = buildValues();\n\t\tdb.insert(table,dbkey,values);\n\t}\n}\n"
  },
  {
    "path": "contrib/ycsb/core/src/test/java/com/yahoo/ycsb/TestByteIterator.java",
    "content": "package com.yahoo.ycsb;\n\nimport org.testng.annotations.Test;\nimport static org.testng.AssertJUnit.*;\n\npublic class TestByteIterator {\n  @Test\n  public void testRandomByteIterator() {\n    int size = 100;\n    ByteIterator itor = new RandomByteIterator(size);\n    assertTrue(itor.hasNext());\n    assertEquals(size, itor.bytesLeft());\n    assertEquals(size, itor.toString().getBytes().length);\n    assertFalse(itor.hasNext());\n    assertEquals(0, itor.bytesLeft());\n\n    itor = new RandomByteIterator(size);\n    assertEquals(size, itor.toArray().length);\n    assertFalse(itor.hasNext());\n    assertEquals(0, itor.bytesLeft());\n  }\n}\n"
  },
  {
    "path": "contrib/ycsb/distribution/pom.xml",
    "content": "<project xmlns=\"http://maven.apache.org/POM/4.0.0\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n  xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd\">\n  <modelVersion>4.0.0</modelVersion>\n\n  <parent>\n    <groupId>com.yahoo.ycsb</groupId>\n    <artifactId>root</artifactId>\n    <version>0.1.4</version>\n  </parent>\n\n  <artifactId>ycsb</artifactId>\n  <name>YCSB Release Distribution Builder</name>\n  <packaging>pom</packaging>\n\n  <description>\n    This module creates the release package of the YCSB with all DB library bindings.\n    It is only used by the build process and does not contain any real\n    code of itself.\n  </description>\n\n  <build>\n    <plugins>\n      <plugin>\n        <groupId>org.apache.maven.plugins</groupId>\n        <artifactId>maven-assembly-plugin</artifactId>\n        <version>${maven.assembly.version}</version>\n        <configuration>\n          <descriptors>\n            <descriptor>src/main/assembly/distribution.xml</descriptor>\n          </descriptors>\n          <appendAssemblyId>false</appendAssemblyId>\n        </configuration>\n        <executions>\n          <execution>\n            <phase>package</phase>\n            <goals>\n              <goal>single</goal>\n            </goals>\n          </execution>\n        </executions>\n      </plugin>\n\n    </plugins>\n  </build>\n\n</project>\n\n"
  },
  {
    "path": "contrib/ycsb/distribution/src/main/assembly/distribution.xml",
    "content": "<assembly xmlns=\"http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n  xsi:schemaLocation=\"http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd\">\n  <id>package</id>\n  <formats>\n    <format>tar.gz</format>\n  </formats>\n  <includeBaseDirectory>true</includeBaseDirectory>\n  <fileSets>\n    <fileSet>\n      <directory>..</directory>\n      <outputDirectory>.</outputDirectory>\n      <fileMode>0644</fileMode>\n      <includes>\n        <include>README</include>\n        <include>CHANGELOG</include>\n        <include>LICENSE.txt</include>\n        <include>NOTICE.txt</include>\n      </includes>\n    </fileSet>\n    <fileSet>\n      <directory>../bin</directory>\n      <outputDirectory>bin</outputDirectory>\n      <fileMode>0755</fileMode>\n    </fileSet>\n    <fileSet>\n      <directory>../workloads</directory>\n      <outputDirectory>workloads</outputDirectory>\n      <fileMode>0644</fileMode>\n    </fileSet>\n  </fileSets>\n  <moduleSets>\n    <moduleSet>\n      <useAllReactorProjects>true</useAllReactorProjects>\n      <includeSubModules>true</includeSubModules>\n      <sources>\n        <includeModuleDirectory>true</includeModuleDirectory>\n\n        <fileSets>\n          <fileSet>\n            <directory>.</directory>\n            <fileMode>0644</fileMode>\n    \t    <includes>\n              <include>README</include>\n            </includes>\n          </fileSet>\n          <fileSet>\n            <directory>src/main/conf</directory>\n            <outputDirectory>conf</outputDirectory>\n            <fileMode>0644</fileMode>\n          </fileSet>\n\t  <fileSet>\n            <outputDirectory>lib</outputDirectory>\n            <directory>target</directory>\n    \t    <includes>\n              <include>*.jar</include>\n            </includes>\n            <fileMode>0644</fileMode>\n          </fileSet>\n        </fileSets>\n      </sources>\n    </moduleSet>\n  </moduleSets>\n</assembly>\n"
  },
  {
    "path": "contrib/ycsb/doc/coreproperties.html",
    "content": "<HTML>\r\n<HEAD>\r\n<TITLE>YCSB - Core workload package properties</TITLE>\r\n</HEAD>\r\n<BODY>\r\n<H1><img src=\"images/ycsb.jpg\" width=150> Yahoo! Cloud Serving Benchmark</H1>\r\n<H3>Version 0.1.2</H3>\r\n<HR>\r\n<A HREF=\"index.html\">Home</A> - <A href=\"coreworkloads.html\">Core workloads</A> - <a href=\"tipsfaq.html\">Tips and FAQ</A>\r\n<HR>\r\n<H2>Core workload package properties</h2>\r\nThe property files used with the core workload generator can specify values for the following properties:<p>\r\n<UL>\r\n<LI><b>fieldcount</b>: the number of fields in a record (default: 10) \r\n<LI><b>fieldlength</b>: the size of each field (default: 100) \r\n<LI><b>readallfields</b>: should reads read all fields (true) or just one (false) (default: true) \r\n<LI><b>readproportion</b>: what proportion of operations should be reads (default: 0.95) \r\n<LI><b>updateproportion</b>: what proportion of operations should be updates (default: 0.05) \r\n<LI><b>insertproportion</b>: what proportion of operations should be inserts (default: 0) \r\n<LI><b>scanproportion</b>: what proportion of operations should be scans (default: 0) \r\n<LI><b>readmodifywriteproportion</b>: what proportion of operations should be read a record, modify it, write it back (default: 0) \r\n<LI><b>requestdistribution</b>: what distribution should be used to select the records to operate on - uniform, zipfian or latest (default: uniform) \r\n<LI><b>maxscanlength</b>: for scans, what is the maximum number of records to scan (default: 1000) \r\n<LI><b>scanlengthdistribution</b>: for scans, what distribution should be used to choose the number of records to scan, for each scan, between 1 and maxscanlength (default: uniform) \r\n<LI><b>insertorder</b>: should records be inserted in order by key (\"ordered\"), or in hashed order (\"hashed\") (default: hashed) \r\n</UL>\r\n<HR>\r\nYCSB - Yahoo! Research - Contact cooperb@yahoo-inc.com.\r\n</BODY>\r\n</HTML>\r\n"
  },
  {
    "path": "contrib/ycsb/doc/coreworkloads.html",
    "content": "<HTML>\r\n<HEAD>\r\n<TITLE>YCSB - Core workloads</TITLE>\r\n</HEAD>\r\n<BODY>\r\n<H1><img src=\"images/ycsb.jpg\" width=150> Yahoo! Cloud Serving Benchmark</H1>\r\n<H3>Version 0.1.2</H3>\r\n<HR>\r\n<A HREF=\"index.html\">Home</A> - <A href=\"coreworkloads.html\">Core workloads</A> - <a href=\"tipsfaq.html\">Tips and FAQ</A>\r\n<HR>\r\n<H2>Core workloads</h2>\r\nYCSB includes a set of core workloads that define a basic benchmark for cloud systems. Of course, you can define your own workloads, as described <a href=\"workload.html\">here</A>. However,\r\nthe core workloads are a useful first step, and obtaining these benchmark numbers for a variety of different systems would allow you to understand the performance\r\ntradeoffs of different systems.\r\n<P>\r\nThe core workloads consist of six different workloads:\r\n<P>\r\n<B>Workload A: Update heavy workload</B>\r\n<P>\r\nThis workload has a mix of 50/50 reads and writes. An application example is a session store recording recent actions.\r\n<P>\r\n<B>Workload B: Read mostly workload</B>\r\n<P>\r\nThis workload has a 95/5 reads/write mix. Application example: photo tagging; add a tag is an update, but most operations are to read tags.\r\n<P>\r\n<B>Workload C: Read only</B>\r\n<P>\r\nThis workload is 100% read. Application example: user profile cache, where profiles are constructed elsewhere (e.g., Hadoop).\r\n<P>\r\n<B>Workload D: Read latest workload</B>\r\n<P>\r\nIn this workload, new records are inserted, and the most recently inserted records are the most popular. Application example: user status updates; people want to read the latest.\r\n<P>\r\n<B>Workload E: Short ranges</B>\r\n<P>\r\nIn this workload, short ranges of records are queried, instead of individual records. Application example: threaded conversations, where each scan is for the posts in a given thread (assumed to be clustered by thread id).\r\n<P>\r\n<B>Workload F: Read-modify-write</B>\r\n<P>\r\nIn this workload, the client will read a record, modify it, and write back the changes. Application example: user database, where user records are read and modified by the user or to record user activity.\r\n\r\n<HR>\r\n<H2>Running the workloads</H2>\r\nAll six workloads have a data set which is similar. Workloads D and E insert records during the test run. Thus, to keep the database size consistent, we recommend the following sequence:\r\n<OL>\r\n<LI>Load the database, using workload A's parameter file (workloads/workloada) and the \"-load\" switch to the client.\r\n<LI>Run workload A (using workloads/workloada and \"-t\") for a variety of throughputs.\r\n<LI>Run workload B (using workloads/workloadb and \"-t\") for a variety of throughputs.\r\n<LI>Run workload C (using workloads/workloadc and \"-t\") for a variety of throughputs. \r\n<LI>Run workload F (using workloads/workloadf and \"-t\") for a variety of throughputs.\r\n<LI>Run workload D (using workloads/workloadd and \"-t\") for a variety of throughputs. This workload inserts records, increasing the size of the database.\r\n<LI>Delete the data in the database.\r\n<LI>Reload the database, using workload E's parameter file (workloads/workloade) and the \"-load switch to the client.\r\n<LI>Run workload E (using workloads/workloadd and \"-t\") for a variety of throughputs. This workload inserts records, increasing the size of the database.\r\n</OL>\r\n<HR>\r\nYCSB - Yahoo! Research - Contact cooperb@yahoo-inc.com.\r\n</BODY>\r\n</HTML>\r\n"
  },
  {
    "path": "contrib/ycsb/doc/dblayer.html",
    "content": "<HTML>\r\n<HEAD>\r\n<TITLE>YCSB - DB Interface Layer</TITLE>\r\n</HEAD>\r\n<BODY>\r\n<H1><img src=\"images/ycsb.jpg\" width=150> Yahoo! Cloud Serving Benchmark</H1>\r\n<H3>Version 0.1.2</H3>\r\n<HR>\r\n<A HREF=\"index.html\">Home</A> - <A href=\"coreworkloads.html\">Core workloads</A> - <a href=\"tipsfaq.html\">Tips and FAQ</A>\r\n<HR>\r\n<H2>Implementing a database interface layer - overview</H2>\r\nThe database interface layer hides the details of the specific database you are benchmarking from the YCSB Client. This\r\nallows the client to generate operations like \"read record\" or \"update record\" without having to understand \r\nthe specific API of your database. Thus, it is very easy to benchmark new database systems; once you have\r\ncreated the database interface layer, the rest of the benchmark framework runs without having to change.\r\n<P>\r\nThe database interface layer is a simple abstract class that provides read, insert, update, delete and scan operations for your\r\ndatabase. Implementing a database interface layer for your database means filling out the body of each of those methods. Once you\r\nhave compiled your layer, you can specify the name of your implemented class on the command line (or as a property) to the YCSB Client.\r\nThe YCSB Client will load your implementation dynamically when it starts. Thus, you do not need to recompile the YCSB Client itself\r\nto add or change a database interface layer.\r\n<HR>\r\n<H2>Creating a new layer step-by-step</H2>\r\n<h3>Step 1 - Extend com.yahoo.ycsb.DB</h3>\r\nThe base class of all database interface layer implementations is com.yahoo.ycsb.DB. This is an abstract class, so you need to create a new \r\nclass which extends the DB class. Your class must have a public no-argument constructor, because the instances will be constructed inside a factory\r\nwhich will use the no-argument constructor.\r\n<P>\r\nThe YCSB Client framework will create one instance of your DB class per worker thread, but there might be multiple worker threads generating the workload,\r\nso there might be multiple instances of your DB class created.\r\n\r\n<H3>Step 2 - Implement init() if necessary</h3>\r\nYou can perform any initialization of your DB object by implementing the following method\r\n<pre> \r\npublic void init() throws DBException\r\n</pre>\r\nto perform any initialization actions. The init() method will be called once per DB instance; so if there are multiple threads, each DB instance will have init()\r\ncalled separately.\r\n<P>\r\nThe init() method should be used to set up the connection to the database and do any other initialization. In particular, you can configure your database layer \r\nusing properties passed to the YCSB Client at runtime. In fact, the YCSB Client will pass to the DB interface layer \r\nall of the \r\nproperties specified in all parameter files specified when the Client starts up. Thus, you can create new properties for configuring your DB interface layer, \r\nset them in your parameter files (or on the command line), and\r\nthen retrieve them inside your implementation of the DB interface layer. \r\n<P>\r\nThese properties will be passed to the DB instance <i>after</i> the constructor, so it is important to retrieve them only in the init() method and not the\r\nconstructor. You can get the set of properties using the \r\n<pre>\r\npublic Properties getProperties()\r\n</pre>\r\nmethod which is already implemented and inherited from the DB base class.\r\n\r\n<h3>Step 3 - Implement the database query and update methods</h3>\r\n\r\nThe methods that you need to implement are:\r\n\r\n<pre>\r\n  //Read a single record\r\n  public int read(String table, String key, Set<String> fields, HashMap<String,String> result);\r\n\r\n  //Perform a range scan\r\n  public int scan(String table, String startkey, int recordcount, Set<String> fields, Vector<HashMap<String,String>> result);\r\n\t\r\n  //Update a single record\r\n  public int update(String table, String key, HashMap<String,String> values);\r\n\r\n  //Insert a single record\r\n  public int insert(String table, String key, HashMap<String,String> values);\r\n\r\n  //Delete a single record\r\n  public int delete(String table, String key);\r\n</pre>\r\nIn each case, the method takes a table name and record key. (In the case of scan, the record key is the first key in the range to scan.) For the \r\nread methods (read() and scan()) the methods additionally take a set of fields to be read, and provide a structure (HashMap or Vector of HashMaps) to store\r\nthe returned data. For the write methods (insert() and update()) the methods take HashMap which maps field names to values.\r\n<P>\r\nThe database should have the appropriate tables created before you run the benchmark. So you can assume in your implementation of the above methods\r\nthat the appropriate tables already exist, and just write code to read or write from the tables named in the \"table\" parameter.\r\n<h3>Step 4 - Compile your database interface layer</h3>\r\nYour code can be compiled separately from the compilation of the YCSB Client and framework. In particular, you can make changes to your DB class and \r\nrecompile without having to recompile the YCSB Client.\r\n<h3>Step 5 - Use it with the YCSB Client</h3>\r\nMake sure that the classes for your implementation (or a jar containing those classes) are available on your CLASSPATH, as well as any libraries/jar files used\r\nby your implementation. Now, when you run the YCSB Client, specify the \"-db\" argument on the command line and provide the fully qualified classname of your\r\nDB class. For example, to run workloada with your DB class:\r\n<pre>\r\n%  java -cp build/ycsb.jar:yourjarpath com.yahoo.ycsb.Client -t -db com.foo.YourDBClass -P workloads/workloada -P large.dat -s > transactions.dat\r\n</pre>  \r\n\r\nYou can also specify the DB interface layer using the DB property in your parameter file:\r\n<pre>\r\ndb=com.foo.YourDBClass\r\n</pre>\r\n<HR>\r\nYCSB - Yahoo! Research - Contact cooperb@yahoo-inc.com.\r\n</BODY>\r\n</HTML>\r\n"
  },
  {
    "path": "contrib/ycsb/doc/index.html",
    "content": "<html>\r\n<head>\r\n<title>YCSB - Yahoo! Cloud Serving Benchmark</title>\r\n</head>\r\n<body>\r\n<H1><img src=\"images/ycsb.jpg\" width=150> Yahoo! Cloud Serving Benchmark</H1>\r\n<H3>Version 0.1.2</H3>\r\n<hr>\r\n<A HREF=\"index.html\">Home</A> - <A href=\"coreworkloads.html\">Core workloads</A> - <a href=\"tipsfaq.html\">Tips and FAQ</A>\r\n<HR>\r\n<UL>\r\n<LI><A href=\"#overview\">Overview</A>\r\n<LI><A href=\"#download\">Download YCSB</A>\r\n<LI><A href=\"#gettingstarted\">Getting started</A>\r\n<LI><A href=\"#extending\">Extending YCSB</A>\r\n</UL>\r\n<HR>\r\n<A name=\"overview\">\r\n<H2>Overview</H2>\r\nThere are many new serving databases available, including:\r\n<ul>\r\n<LI>BigTable\r\n<ul>\r\n<LI><A HREF=\"http://hadoop.apache.org/hbase/\">HBase</A>, <A HREF=\"http://hypertable.org/\">Hypertable\r\n</ul>\r\n<LI><A HREF=\"http://www.microsoft.com/windowsazure/\">Azure</A>\r\n<LI><A HREF=\"http://incubator.apache.org/cassandra/\">Cassandra</A>\r\n<LI><A HREF=\"http://couchdb.apache.org/\">CouchDB</A>\r\n<LI><A HREF=\"http://project-voldemort.com/\">Voldemort</A>\r\n<LI><A HREF=http://wiki.github.com/cliffmoon/dynomite/dynomite-framework\">Dynomite</A>\r\n<li>...and many others\r\n</ul>\r\nIt is difficult to decide which system is right for your application, partially because the features differ between \r\nsystems, and partially because there is not an easy way to compare the performance of one system versus another.\r\n<P>\r\nThe goal of the YCSB project is to develop a framework and common set of workloads for evaluating the performance of\r\ndifferent \"key-value\" and \"cloud\" serving stores. The project comprises two things:\r\n<ul>\r\n<LI>The YCSB Client, an extensible workload generator\r\n<LI>The Core workloads, a set of workload scenarios to be executed by the generator\r\n</UL>\r\nAlthough the core workloads provide a well rounded picture of a system's performance, the Client is extensible so that\r\nyou can define new and different workloads to examine system aspects, or application scenarios, not adequately covered by\r\nthe core workload. Similarly, the Client is extensible to support benchmarking different databases. Although we include\r\nsample code for benchmarking HBase and Cassandra, it is straightforward to write a new interface layer to benchmark\r\nyour favorite database.\r\n<P>\r\nA common use of the tool is to benchmark multiple systems and compare them. For example, you can install multiple systems\r\non the same hardward configuration, and run the same workloads against each system. Then you can plot the performance \r\nof each system (for example, as latency versus throughput curves) to see when one system does better than another.\r\n<HR>\r\n<A name=\"download\">\r\n<H2>Download YCSB</H2>\r\nYCSB is available\r\nat <A HREF=\"http://wiki.github.com/brianfrankcooper/YCSB/\">http://wiki.github.com/brianfrankcooper/YCSB/</A>. \r\n<HR>\r\n<a name=\"gettingstarted\">\r\n<H2>Getting started</H2>\r\nDetailed instructions for using YCSB are available on the GitHub wiki:\r\n<A HREF=\"http://wiki.github.com/brianfrankcooper/YCSB/getting-started\">http://wiki.github.com/brianfrankcooper/YCSB/getting-started</A>.\r\n<HR>\r\n<A name=\"extending\">\r\n<H1>Extending YCSB</H1>\r\nYCSB is designed to be extensible. It is easy to add a new database interface layer to support benchmarking a new database. It is also easy to define new workloads.\r\n<ul>\r\n<li><A HREF=\"dblayer.html\">DB Interface Layer</a>\r\n<li><A HREF=\"workload.html\">Implementing new workloads</a>\r\n</UL>\r\nMore details about the entire class structure of YCSB is available here:\r\n<UL>\r\n<LI><A HREF=\"javadoc/index.html\">YCSB javadoc documentation</A>\r\n</ul>  \r\n<HR>\r\nYCSB - Yahoo! Research - Contact cooperb@yahoo-inc.com.\r\n</body>\r\n</html>\r\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/allclasses-frame.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\nAll Classes\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"stylesheet.css\" TITLE=\"Style\">\n\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\">\n<FONT size=\"+1\" CLASS=\"FrameHeadingFont\">\n<B>All Classes</B></FONT>\n<BR>\n\n<TABLE BORDER=\"0\" WIDTH=\"100%\" SUMMARY=\"\">\n<TR>\n<TD NOWRAP><FONT CLASS=\"FrameItemFont\"><A HREF=\"com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">BasicDB</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\" target=\"classFrame\">CassandraClient5</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\" target=\"classFrame\">CassandraClient6</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\" target=\"classFrame\">CassandraClient7</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">Client</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">CommandLine</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\" target=\"classFrame\">CoreWorkload</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/CounterGenerator.html\" title=\"class in com.yahoo.ycsb.generator\" target=\"classFrame\">CounterGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">DB</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">DBException</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/DBFactory.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">DBFactory</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">DBWrapper</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb.generator\" target=\"classFrame\">DiscreteGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/Generator.html\" title=\"class in com.yahoo.ycsb.generator\" target=\"classFrame\">Generator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\" target=\"classFrame\">HBaseClient</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\" target=\"classFrame\">IntegerGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/measurements/Measurements.html\" title=\"class in com.yahoo.ycsb.measurements\" target=\"classFrame\">Measurements</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\" target=\"classFrame\">MongoDbClient</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/measurements/OneMeasurement.html\" title=\"class in com.yahoo.ycsb.measurements\" target=\"classFrame\">OneMeasurement</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/measurements/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb.measurements\" target=\"classFrame\">OneMeasurementHistogram</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb.measurements\" target=\"classFrame\">OneMeasurementTimeSeries</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\" target=\"classFrame\">ScrambledZipfianGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb.generator\" target=\"classFrame\">SkewedLatestGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/UniformGenerator.html\" title=\"class in com.yahoo.ycsb.generator\" target=\"classFrame\">UniformGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/UniformIntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\" target=\"classFrame\">UniformIntegerGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">UnknownDBException</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">Utils</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">Workload</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">WorkloadException</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\" target=\"classFrame\">ZipfianGenerator</A>\n<BR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/allclasses-noframe.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\nAll Classes\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"stylesheet.css\" TITLE=\"Style\">\n\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\">\n<FONT size=\"+1\" CLASS=\"FrameHeadingFont\">\n<B>All Classes</B></FONT>\n<BR>\n\n<TABLE BORDER=\"0\" WIDTH=\"100%\" SUMMARY=\"\">\n<TR>\n<TD NOWRAP><FONT CLASS=\"FrameItemFont\"><A HREF=\"com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\">Client</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\">CommandLine</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/CounterGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">CounterGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/DBFactory.html\" title=\"class in com.yahoo.ycsb\">DBFactory</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\">DBWrapper</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">DiscreteGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/Generator.html\" title=\"class in com.yahoo.ycsb.generator\">Generator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">IntegerGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/measurements/Measurements.html\" title=\"class in com.yahoo.ycsb.measurements\">Measurements</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\">MongoDbClient</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/measurements/OneMeasurement.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurement</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/measurements/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementHistogram</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementTimeSeries</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ScrambledZipfianGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">SkewedLatestGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/UniformGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">UniformGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/UniformIntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">UniformIntegerGenerator</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\">UnknownDBException</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\">Utils</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A>\n<BR>\n<A HREF=\"com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ZipfianGenerator</A>\n<BR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/BasicDB.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:52 PDT 2010 -->\n<TITLE>\nBasicDB\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"BasicDB\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV CLASS&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/BasicDB.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"BasicDB.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass BasicDB</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.DB</A>\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.BasicDB</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>BasicDB</B><DT>extends <A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></DL>\n</PRE>\n\n<P>\nBasic DB that just prints out the requested operations, instead of doing them against a database.\n<P>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/BasicDB.html#SIMULATE_DELAY\">SIMULATE_DELAY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/BasicDB.html#SIMULATE_DELAY_DEFAULT\">SIMULATE_DELAY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/BasicDB.html#VERBOSE\">VERBOSE</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/BasicDB.html#VERBOSE_DEFAULT\">VERBOSE_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/BasicDB.html#BasicDB()\">BasicDB</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/BasicDB.html#delete(java.lang.String, java.lang.String)\">delete</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delete a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/BasicDB.html#init()\">init</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Initialize any state for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/BasicDB.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">insert</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Insert a record in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/BasicDB.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">read</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;key,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Read a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/BasicDB.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">scan</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;startkey,\n     int&nbsp;recordcount,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Perform a range scan for a set of records in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/BasicDB.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">update</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update a record in the database.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_com.yahoo.ycsb.DB\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#cleanup()\">cleanup</A>, <A HREF=\"../../../com/yahoo/ycsb/DB.html#getProperties()\">getProperties</A>, <A HREF=\"../../../com/yahoo/ycsb/DB.html#setProperties(java.util.Properties)\">setProperties</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"VERBOSE\"><!-- --></A><H3>\nVERBOSE</H3>\n<PRE>\npublic static final java.lang.String <B>VERBOSE</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.BasicDB.VERBOSE\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"VERBOSE_DEFAULT\"><!-- --></A><H3>\nVERBOSE_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>VERBOSE_DEFAULT</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.BasicDB.VERBOSE_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"SIMULATE_DELAY\"><!-- --></A><H3>\nSIMULATE_DELAY</H3>\n<PRE>\npublic static final java.lang.String <B>SIMULATE_DELAY</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.BasicDB.SIMULATE_DELAY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"SIMULATE_DELAY_DEFAULT\"><!-- --></A><H3>\nSIMULATE_DELAY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>SIMULATE_DELAY_DEFAULT</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.BasicDB.SIMULATE_DELAY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"BasicDB()\"><!-- --></A><H3>\nBasicDB</H3>\n<PRE>\npublic <B>BasicDB</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"init()\"><!-- --></A><H3>\ninit</H3>\n<PRE>\npublic void <B>init</B>()</PRE>\n<DL>\n<DD>Initialize any state for this DB.\n Called once per DB instance; there is one DB instance per client thread.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#init()\">init</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><!-- --></A><H3>\nread</H3>\n<PRE>\npublic int <B>read</B>(java.lang.String&nbsp;table,\n                java.lang.String&nbsp;key,\n                java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</PRE>\n<DL>\n<DD>Read a record from the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">read</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to read.<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A HashMap of field/value pairs for the result\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><!-- --></A><H3>\nscan</H3>\n<PRE>\npublic int <B>scan</B>(java.lang.String&nbsp;table,\n                java.lang.String&nbsp;startkey,\n                int&nbsp;recordcount,\n                java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</PRE>\n<DL>\n<DD>Perform a range scan for a set of records in the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">scan</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>startkey</CODE> - The record key of the first record to read.<DD><CODE>recordcount</CODE> - The number of records to read<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A Vector of HashMaps, where each HashMap is a set field/value pairs for one record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"update(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\nupdate</H3>\n<PRE>\npublic int <B>update</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key,\n                  java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Update a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key, overwriting any existing values with the same field name.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">update</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to write.<DD><CODE>values</CODE> - A HashMap of field/value pairs to update in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"insert(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\ninsert</H3>\n<PRE>\npublic int <B>insert</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key,\n                  java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Insert a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">insert</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to insert.<DD><CODE>values</CODE> - A HashMap of field/value pairs to insert in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"delete(java.lang.String, java.lang.String)\"><!-- --></A><H3>\ndelete</H3>\n<PRE>\npublic int <B>delete</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key)</PRE>\n<DL>\n<DD>Delete a record from the database.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#delete(java.lang.String, java.lang.String)\">delete</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to delete.\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV CLASS&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/BasicDB.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"BasicDB.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/Client.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:52 PDT 2010 -->\n<TITLE>\nClient\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"Client\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/Client.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"Client.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass Client</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.Client</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>Client</B><DT>extends java.lang.Object</DL>\n</PRE>\n\n<P>\nMain class for executing YCSB.\n<P>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Client.html#INSERT_COUNT_PROPERTY\">INSERT_COUNT_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Indicates how many inserts to do, if less than recordcount.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Client.html#OPERATION_COUNT_PROPERTY\">OPERATION_COUNT_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Client.html#RECORD_COUNT_PROPERTY\">RECORD_COUNT_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Client.html#WORKLOAD_PROPERTY\">WORKLOAD_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Client.html#Client()\">Client</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;boolean</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Client.html#checkRequiredProperties(java.util.Properties)\">checkRequiredProperties</A></B>(java.util.Properties&nbsp;props)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Client.html#main(java.lang.String[])\">main</A></B>(java.lang.String[]&nbsp;args)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Client.html#usageMessage()\">usageMessage</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"OPERATION_COUNT_PROPERTY\"><!-- --></A><H3>\nOPERATION_COUNT_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>OPERATION_COUNT_PROPERTY</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.Client.OPERATION_COUNT_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"RECORD_COUNT_PROPERTY\"><!-- --></A><H3>\nRECORD_COUNT_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>RECORD_COUNT_PROPERTY</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.Client.RECORD_COUNT_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"WORKLOAD_PROPERTY\"><!-- --></A><H3>\nWORKLOAD_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>WORKLOAD_PROPERTY</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.Client.WORKLOAD_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"INSERT_COUNT_PROPERTY\"><!-- --></A><H3>\nINSERT_COUNT_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>INSERT_COUNT_PROPERTY</B></PRE>\n<DL>\n<DD>Indicates how many inserts to do, if less than recordcount. Useful for partitioning\n the load among multiple servers, if the client is the bottleneck. Additionally, workloads\n should support the \"insertstart\" property, which tells them which record to start at.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.Client.INSERT_COUNT_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"Client()\"><!-- --></A><H3>\nClient</H3>\n<PRE>\npublic <B>Client</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"usageMessage()\"><!-- --></A><H3>\nusageMessage</H3>\n<PRE>\npublic static void <B>usageMessage</B>()</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"checkRequiredProperties(java.util.Properties)\"><!-- --></A><H3>\ncheckRequiredProperties</H3>\n<PRE>\npublic static boolean <B>checkRequiredProperties</B>(java.util.Properties&nbsp;props)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"main(java.lang.String[])\"><!-- --></A><H3>\nmain</H3>\n<PRE>\npublic static void <B>main</B>(java.lang.String[]&nbsp;args)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/Client.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"Client.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/CounterGenerator.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Fri Apr 23 10:24:25 PDT 2010 -->\n<TITLE>\nCounterGenerator\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.CounterGenerator class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"CounterGenerator\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/CounterGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"CounterGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass CounterGenerator</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.Generator</A>\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.IntegerGenerator</A>\n          <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.CounterGenerator</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>CounterGenerator</B><DT>extends <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></DL>\n</PRE>\n\n<P>\nGenerates a sequence of integers 0, 1, ...\n<P>\n\n<P>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/CounterGenerator.html#CounterGenerator(int)\">CounterGenerator</A></B>(int&nbsp;countstart)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a counter that starts at countstart</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/CounterGenerator.html#nextInt()\">nextInt</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If the generator returns numeric (integer) values, return the next value as an int.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_com.yahoo.ycsb.IntegerGenerator\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#lastInt()\">lastInt</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#lastString()\">lastString</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#nextString()\">nextString</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#setLastInt(int)\">setLastInt</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"CounterGenerator(int)\"><!-- --></A><H3>\nCounterGenerator</H3>\n<PRE>\npublic <B>CounterGenerator</B>(int&nbsp;countstart)</PRE>\n<DL>\n<DD>Create a counter that starts at countstart\n<P>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"nextInt()\"><!-- --></A><H3>\nnextInt</H3>\n<PRE>\npublic int <B>nextInt</B>()</PRE>\n<DL>\n<DD>If the generator returns numeric (integer) values, return the next value as an int. Default is to return -1, which\n is appropriate for generators that do not return numeric values.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#nextInt()\">nextInt</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/CounterGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"CounterGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/DB.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:52 PDT 2010 -->\n<TITLE>\nDB\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"DB\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/DB.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"DB.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass DB</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.DB</B>\n</PRE>\n<DL>\n<DT><B>Direct Known Subclasses:</B> <DD><A HREF=\"../../../com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A>, <A HREF=\"../../../com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>, <A HREF=\"../../../com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>, <A HREF=\"../../../com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>, <A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\">DBWrapper</A>, <A HREF=\"../../../com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>, <A HREF=\"../../../com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\">MongoDbClient</A></DD>\n</DL>\n<HR>\n<DL>\n<DT><PRE>public abstract class <B>DB</B><DT>extends java.lang.Object</DL>\n</PRE>\n\n<P>\nA layer for accessing a database to be benchmarked. Each thread in the client\n will be given its own instance of whatever DB class is to be used in the test.\n This class should be constructed using a no-argument constructor, so we can\n load it dynamically. Any argument-based initialization should be\n done by init().\n \n Note that YCSB does not make any use of the return codes returned by this class.\n Instead, it keeps a count of the return values and presents them to the user.\n \n The semantics of methods such as insert, update and delete vary from database\n to database.  In particular, operations may or may not be durable once these\n methods commit, and some systems may return 'success' regardless of whether\n or not a tuple with a matching key existed before the call.  Rather than dictate\n the exact semantics of these methods, we recommend you either implement them\n to match the database's default semantics, or the semantics of your \n target application.  For the sake of comparison between experiments we also \n recommend you explain the semantics you chose when presenting performance results.\n<P>\n\n<P>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DB.html#DB()\">DB</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DB.html#cleanup()\">cleanup</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cleanup any state for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DB.html#delete(java.lang.String, java.lang.String)\">delete</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delete a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.util.Properties</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DB.html#getProperties()\">getProperties</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Get the set of properties for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DB.html#init()\">init</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Initialize any state for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DB.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">insert</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Insert a record in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DB.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">read</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;key,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Read a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DB.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">scan</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;startkey,\n     int&nbsp;recordcount,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Perform a range scan for a set of records in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DB.html#setProperties(java.util.Properties)\">setProperties</A></B>(java.util.Properties&nbsp;p)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Set the properties for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DB.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">update</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update a record in the database.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"DB()\"><!-- --></A><H3>\nDB</H3>\n<PRE>\npublic <B>DB</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"setProperties(java.util.Properties)\"><!-- --></A><H3>\nsetProperties</H3>\n<PRE>\npublic void <B>setProperties</B>(java.util.Properties&nbsp;p)</PRE>\n<DL>\n<DD>Set the properties for this DB.\n<P>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"getProperties()\"><!-- --></A><H3>\ngetProperties</H3>\n<PRE>\npublic java.util.Properties <B>getProperties</B>()</PRE>\n<DL>\n<DD>Get the set of properties for this DB.\n<P>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"init()\"><!-- --></A><H3>\ninit</H3>\n<PRE>\npublic void <B>init</B>()\n          throws <A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Initialize any state for this DB.\n Called once per DB instance; there is one DB instance per client thread.\n<P>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"cleanup()\"><!-- --></A><H3>\ncleanup</H3>\n<PRE>\npublic void <B>cleanup</B>()\n             throws <A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Cleanup any state for this DB.\n Called once per DB instance; there is one DB instance per client thread.\n<P>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><!-- --></A><H3>\nread</H3>\n<PRE>\npublic abstract int <B>read</B>(java.lang.String&nbsp;table,\n                         java.lang.String&nbsp;key,\n                         java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                         java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</PRE>\n<DL>\n<DD>Read a record from the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to read.<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A HashMap of field/value pairs for the result\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error or \"not found\".</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><!-- --></A><H3>\nscan</H3>\n<PRE>\npublic abstract int <B>scan</B>(java.lang.String&nbsp;table,\n                         java.lang.String&nbsp;startkey,\n                         int&nbsp;recordcount,\n                         java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                         java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</PRE>\n<DL>\n<DD>Perform a range scan for a set of records in the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>startkey</CODE> - The record key of the first record to read.<DD><CODE>recordcount</CODE> - The number of records to read<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A Vector of HashMaps, where each HashMap is a set field/value pairs for one record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error.  See this class's description for a discussion of error codes.</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"update(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\nupdate</H3>\n<PRE>\npublic abstract int <B>update</B>(java.lang.String&nbsp;table,\n                           java.lang.String&nbsp;key,\n                           java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Update a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key, overwriting any existing values with the same field name.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to write.<DD><CODE>values</CODE> - A HashMap of field/value pairs to update in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error.  See this class's description for a discussion of error codes.</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"insert(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\ninsert</H3>\n<PRE>\npublic abstract int <B>insert</B>(java.lang.String&nbsp;table,\n                           java.lang.String&nbsp;key,\n                           java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Insert a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to insert.<DD><CODE>values</CODE> - A HashMap of field/value pairs to insert in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error.  See this class's description for a discussion of error codes.</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"delete(java.lang.String, java.lang.String)\"><!-- --></A><H3>\ndelete</H3>\n<PRE>\npublic abstract int <B>delete</B>(java.lang.String&nbsp;table,\n                           java.lang.String&nbsp;key)</PRE>\n<DL>\n<DD>Delete a record from the database.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to delete.\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error.  See this class's description for a discussion of error codes.</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/DB.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"DB.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/DBException.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:52 PDT 2010 -->\n<TITLE>\nDBException\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"DBException\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBFactory.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/DBException.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"DBException.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#methods_inherited_from_class_java.lang.Throwable\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;METHOD</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass DBException</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \">java.lang.Throwable\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \">java.lang.Exception\n          <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.DBException</B>\n</PRE>\n<DL>\n<DT><B>All Implemented Interfaces:</B> <DD>java.io.Serializable</DD>\n</DL>\n<HR>\n<DL>\n<DT><PRE>public class <B>DBException</B><DT>extends java.lang.Exception</DL>\n</PRE>\n\n<P>\nSomething bad happened while interacting with the database.\n<P>\n\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../serialized-form.html#com.yahoo.ycsb.DBException\">Serialized Form</A></DL>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBException.html#DBException()\">DBException</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBException.html#DBException(java.lang.String)\">DBException</A></B>(java.lang.String&nbsp;message)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBException.html#DBException(java.lang.String, java.lang.Throwable)\">DBException</A></B>(java.lang.String&nbsp;message,\n            java.lang.Throwable&nbsp;cause)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBException.html#DBException(java.lang.Throwable)\">DBException</A></B>(java.lang.Throwable&nbsp;cause)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Throwable\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Throwable</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>fillInStackTrace, getCause, getLocalizedMessage, getMessage, getStackTrace, initCause, printStackTrace, printStackTrace, printStackTrace, setStackTrace, toString</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"DBException(java.lang.String)\"><!-- --></A><H3>\nDBException</H3>\n<PRE>\npublic <B>DBException</B>(java.lang.String&nbsp;message)</PRE>\n<DL>\n</DL>\n<HR>\n\n<A NAME=\"DBException()\"><!-- --></A><H3>\nDBException</H3>\n<PRE>\npublic <B>DBException</B>()</PRE>\n<DL>\n</DL>\n<HR>\n\n<A NAME=\"DBException(java.lang.String, java.lang.Throwable)\"><!-- --></A><H3>\nDBException</H3>\n<PRE>\npublic <B>DBException</B>(java.lang.String&nbsp;message,\n                   java.lang.Throwable&nbsp;cause)</PRE>\n<DL>\n</DL>\n<HR>\n\n<A NAME=\"DBException(java.lang.Throwable)\"><!-- --></A><H3>\nDBException</H3>\n<PRE>\npublic <B>DBException</B>(java.lang.Throwable&nbsp;cause)</PRE>\n<DL>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBFactory.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/DBException.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"DBException.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#methods_inherited_from_class_java.lang.Throwable\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;METHOD</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/DBFactory.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:52 PDT 2010 -->\n<TITLE>\nDBFactory\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"DBFactory\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/DBFactory.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"DBFactory.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass DBFactory</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.DBFactory</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>DBFactory</B><DT>extends java.lang.Object</DL>\n</PRE>\n\n<P>\nCreates a DB layer by dynamically classloading the specified DB class.\n<P>\n\n<P>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBFactory.html#DBFactory()\">DBFactory</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBFactory.html#newDB(java.lang.String, java.util.Properties)\">newDB</A></B>(java.lang.String&nbsp;dbname,\n      java.util.Properties&nbsp;properties)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"DBFactory()\"><!-- --></A><H3>\nDBFactory</H3>\n<PRE>\npublic <B>DBFactory</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"newDB(java.lang.String, java.util.Properties)\"><!-- --></A><H3>\nnewDB</H3>\n<PRE>\npublic static <A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A> <B>newDB</B>(java.lang.String&nbsp;dbname,\n                       java.util.Properties&nbsp;properties)\n                throws <A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\">UnknownDBException</A></PRE>\n<DL>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\">UnknownDBException</A></CODE></DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/DBFactory.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"DBFactory.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/DBWrapper.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:52 PDT 2010 -->\n<TITLE>\nDBWrapper\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"DBWrapper\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBFactory.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/DBWrapper.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"DBWrapper.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass DBWrapper</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.DB</A>\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.DBWrapper</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>DBWrapper</B><DT>extends <A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></DL>\n</PRE>\n\n<P>\nWrapper around a \"real\" DB that measures latencies and counts return codes.\n<P>\n\n<P>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html#DBWrapper(com.yahoo.ycsb.DB)\">DBWrapper</A></B>(<A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html#cleanup()\">cleanup</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cleanup any state for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html#delete(java.lang.String, java.lang.String)\">delete</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delete a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.util.Properties</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html#getProperties()\">getProperties</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Get the set of properties for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html#init()\">init</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Initialize any state for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">insert</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Insert a record in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">read</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;key,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Read a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">scan</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;startkey,\n     int&nbsp;recordcount,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Perform a range scan for a set of records in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html#setProperties(java.util.Properties)\">setProperties</A></B>(java.util.Properties&nbsp;p)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Set the properties for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">update</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update a record in the database.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"DBWrapper(com.yahoo.ycsb.DB)\"><!-- --></A><H3>\nDBWrapper</H3>\n<PRE>\npublic <B>DBWrapper</B>(<A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db)</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"setProperties(java.util.Properties)\"><!-- --></A><H3>\nsetProperties</H3>\n<PRE>\npublic void <B>setProperties</B>(java.util.Properties&nbsp;p)</PRE>\n<DL>\n<DD>Set the properties for this DB.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#setProperties(java.util.Properties)\">setProperties</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"getProperties()\"><!-- --></A><H3>\ngetProperties</H3>\n<PRE>\npublic java.util.Properties <B>getProperties</B>()</PRE>\n<DL>\n<DD>Get the set of properties for this DB.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#getProperties()\">getProperties</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"init()\"><!-- --></A><H3>\ninit</H3>\n<PRE>\npublic void <B>init</B>()\n          throws <A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Initialize any state for this DB.\n Called once per DB instance; there is one DB instance per client thread.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#init()\">init</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"cleanup()\"><!-- --></A><H3>\ncleanup</H3>\n<PRE>\npublic void <B>cleanup</B>()\n             throws <A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Cleanup any state for this DB.\n Called once per DB instance; there is one DB instance per client thread.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#cleanup()\">cleanup</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><!-- --></A><H3>\nread</H3>\n<PRE>\npublic int <B>read</B>(java.lang.String&nbsp;table,\n                java.lang.String&nbsp;key,\n                java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</PRE>\n<DL>\n<DD>Read a record from the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">read</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to read.<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A HashMap of field/value pairs for the result\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><!-- --></A><H3>\nscan</H3>\n<PRE>\npublic int <B>scan</B>(java.lang.String&nbsp;table,\n                java.lang.String&nbsp;startkey,\n                int&nbsp;recordcount,\n                java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</PRE>\n<DL>\n<DD>Perform a range scan for a set of records in the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">scan</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>startkey</CODE> - The record key of the first record to read.<DD><CODE>recordcount</CODE> - The number of records to read<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A Vector of HashMaps, where each HashMap is a set field/value pairs for one record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"update(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\nupdate</H3>\n<PRE>\npublic int <B>update</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key,\n                  java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Update a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key, overwriting any existing values with the same field name.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">update</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to write.<DD><CODE>values</CODE> - A HashMap of field/value pairs to update in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"insert(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\ninsert</H3>\n<PRE>\npublic int <B>insert</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key,\n                  java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Insert a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">insert</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to insert.<DD><CODE>values</CODE> - A HashMap of field/value pairs to insert in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"delete(java.lang.String, java.lang.String)\"><!-- --></A><H3>\ndelete</H3>\n<PRE>\npublic int <B>delete</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key)</PRE>\n<DL>\n<DD>Delete a record from the database.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html#delete(java.lang.String, java.lang.String)\">delete</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to delete.\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBFactory.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/DBWrapper.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"DBWrapper.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/DiscreteGenerator.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Fri Apr 23 10:24:25 PDT 2010 -->\n<TITLE>\nDiscreteGenerator\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.DiscreteGenerator class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"DiscreteGenerator\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/DiscreteGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"DiscreteGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass DiscreteGenerator</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.Generator</A>\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.DiscreteGenerator</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>DiscreteGenerator</B><DT>extends <A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">Generator</A></DL>\n</PRE>\n\n<P>\nGenerates a distribution by choosing from a discrete set of values.\n<P>\n\n<P>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DiscreteGenerator.html#DiscreteGenerator()\">DiscreteGenerator</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DiscreteGenerator.html#addValue(double, java.lang.String)\">addValue</A></B>(double&nbsp;weight,\n         java.lang.String&nbsp;value)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DiscreteGenerator.html#lastString()\">lastString</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Return the previous string generated by the distribution; e.g., returned from the last nextString() call.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DiscreteGenerator.html#nextInt()\">nextInt</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;If the generator returns numeric (integer) values, return the next value as an int.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/DiscreteGenerator.html#nextString()\">nextString</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Generate the next string in the distribution.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"DiscreteGenerator()\"><!-- --></A><H3>\nDiscreteGenerator</H3>\n<PRE>\npublic <B>DiscreteGenerator</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"nextString()\"><!-- --></A><H3>\nnextString</H3>\n<PRE>\npublic java.lang.String <B>nextString</B>()</PRE>\n<DL>\n<DD>Generate the next string in the distribution.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/Generator.html#nextString()\">nextString</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">Generator</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"nextInt()\"><!-- --></A><H3>\nnextInt</H3>\n<PRE>\npublic int <B>nextInt</B>()\n            throws <A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A></PRE>\n<DL>\n<DD>If the generator returns numeric (integer) values, return the next value as an int. Default is to return -1, which\n is appropriate for generators that do not return numeric values.\n<P>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A></CODE> - if this generator does not support integer values</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"lastString()\"><!-- --></A><H3>\nlastString</H3>\n<PRE>\npublic java.lang.String <B>lastString</B>()</PRE>\n<DL>\n<DD>Return the previous string generated by the distribution; e.g., returned from the last nextString() call. \n Calling lastString() should not advance the distribution or have any side effects. If nextString() has not yet \n been called, lastString() should return something reasonable.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/Generator.html#lastString()\">lastString</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">Generator</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"addValue(double, java.lang.String)\"><!-- --></A><H3>\naddValue</H3>\n<PRE>\npublic void <B>addValue</B>(double&nbsp;weight,\n                     java.lang.String&nbsp;value)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/DiscreteGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"DiscreteGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/FindGoodAB.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Tue Apr 20 15:39:16 PDT 2010 -->\n<TITLE>\nFindGoodAB\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.FindGoodAB class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"FindGoodAB\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/FindGoodScrambleVector.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/FindGoodAB.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"FindGoodAB.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass FindGoodAB</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.FindGoodAB</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>FindGoodAB</B><DT>extends java.lang.Object</DL>\n</PRE>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/FindGoodAB.html#a\">a</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/FindGoodAB.html#b\">b</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/FindGoodAB.html#FindGoodAB()\">FindGoodAB</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/FindGoodAB.html#hash(int, int)\">hash</A></B>(int&nbsp;val,\n     int&nbsp;itemcount)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/FindGoodAB.html#main(java.lang.String[])\">main</A></B>(java.lang.String[]&nbsp;args)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/FindGoodAB.html#scramble(int)\">scramble</A></B>(int&nbsp;val)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/FindGoodAB.html#testVector(int)\">testVector</A></B>(int&nbsp;itemcount)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"a\"><!-- --></A><H3>\na</H3>\n<PRE>\npublic static int <B>a</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"b\"><!-- --></A><H3>\nb</H3>\n<PRE>\npublic static int <B>b</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"FindGoodAB()\"><!-- --></A><H3>\nFindGoodAB</H3>\n<PRE>\npublic <B>FindGoodAB</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"scramble(int)\"><!-- --></A><H3>\nscramble</H3>\n<PRE>\npublic static int <B>scramble</B>(int&nbsp;val)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"hash(int, int)\"><!-- --></A><H3>\nhash</H3>\n<PRE>\npublic static int <B>hash</B>(int&nbsp;val,\n                       int&nbsp;itemcount)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"main(java.lang.String[])\"><!-- --></A><H3>\nmain</H3>\n<PRE>\npublic static void <B>main</B>(java.lang.String[]&nbsp;args)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"testVector(int)\"><!-- --></A><H3>\ntestVector</H3>\n<PRE>\npublic static int <B>testVector</B>(int&nbsp;itemcount)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/FindGoodScrambleVector.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/FindGoodAB.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"FindGoodAB.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/FindGoodScrambleVector.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Tue Apr 20 15:39:16 PDT 2010 -->\n<TITLE>\nFindGoodScrambleVector\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.FindGoodScrambleVector class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"FindGoodScrambleVector\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/FindGoodAB.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/FindGoodScrambleVector.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"FindGoodScrambleVector.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass FindGoodScrambleVector</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.FindGoodScrambleVector</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>FindGoodScrambleVector</B><DT>extends java.lang.Object</DL>\n</PRE>\n\n<P>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/FindGoodScrambleVector.html#FindGoodScrambleVector()\">FindGoodScrambleVector</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/FindGoodScrambleVector.html#hash(int, int)\">hash</A></B>(int&nbsp;val,\n     int&nbsp;itemcount)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/FindGoodScrambleVector.html#main(java.lang.String[])\">main</A></B>(java.lang.String[]&nbsp;args)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/FindGoodScrambleVector.html#scramble(int)\">scramble</A></B>(int&nbsp;val)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int[]</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/FindGoodScrambleVector.html#scrambleArray()\">scrambleArray</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"FindGoodScrambleVector()\"><!-- --></A><H3>\nFindGoodScrambleVector</H3>\n<PRE>\npublic <B>FindGoodScrambleVector</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"scrambleArray()\"><!-- --></A><H3>\nscrambleArray</H3>\n<PRE>\npublic static int[] <B>scrambleArray</B>()</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"scramble(int)\"><!-- --></A><H3>\nscramble</H3>\n<PRE>\npublic static int <B>scramble</B>(int&nbsp;val)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"hash(int, int)\"><!-- --></A><H3>\nhash</H3>\n<PRE>\npublic static int <B>hash</B>(int&nbsp;val,\n                       int&nbsp;itemcount)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"main(java.lang.String[])\"><!-- --></A><H3>\nmain</H3>\n<PRE>\npublic static void <B>main</B>(java.lang.String[]&nbsp;args)</PRE>\n<DL>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>args</CODE> - </DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/FindGoodAB.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/FindGoodScrambleVector.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"FindGoodScrambleVector.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/Generator.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Fri Apr 23 10:24:25 PDT 2010 -->\n<TITLE>\nGenerator\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.Generator class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"Generator\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/Generator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"Generator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass Generator</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.Generator</B>\n</PRE>\n<DL>\n<DT><B>Direct Known Subclasses:</B> <DD><A HREF=\"../../../com/yahoo/ycsb/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb\">DiscreteGenerator</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A>, <A HREF=\"../../../com/yahoo/ycsb/UniformGenerator.html\" title=\"class in com.yahoo.ycsb\">UniformGenerator</A></DD>\n</DL>\n<HR>\n<DL>\n<DT><PRE>public abstract class <B>Generator</B><DT>extends java.lang.Object</DL>\n</PRE>\n\n<P>\nAn expression that generates a sequence of string values, following some distribution (Uniform, Zipfian, Sequential, etc.)\n<P>\n\n<P>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Generator.html#Generator()\">Generator</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Generator.html#lastString()\">lastString</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Return the previous string generated by the distribution; e.g., returned from the last nextString() call.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Generator.html#nextString()\">nextString</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Generate the next string in the distribution.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"Generator()\"><!-- --></A><H3>\nGenerator</H3>\n<PRE>\npublic <B>Generator</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"nextString()\"><!-- --></A><H3>\nnextString</H3>\n<PRE>\npublic abstract java.lang.String <B>nextString</B>()</PRE>\n<DL>\n<DD>Generate the next string in the distribution.\n<P>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"lastString()\"><!-- --></A><H3>\nlastString</H3>\n<PRE>\npublic abstract java.lang.String <B>lastString</B>()</PRE>\n<DL>\n<DD>Return the previous string generated by the distribution; e.g., returned from the last nextString() call. \n Calling lastString() should not advance the distribution or have any side effects. If nextString() has not yet \n been called, lastString() should return something reasonable.\n<P>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/Generator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"Generator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/IntegerGenerator.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Fri Apr 23 10:24:25 PDT 2010 -->\n<TITLE>\nIntegerGenerator\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.IntegerGenerator class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"IntegerGenerator\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Measurements.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/IntegerGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"IntegerGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass IntegerGenerator</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.Generator</A>\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.IntegerGenerator</B>\n</PRE>\n<DL>\n<DT><B>Direct Known Subclasses:</B> <DD><A HREF=\"../../../com/yahoo/ycsb/CounterGenerator.html\" title=\"class in com.yahoo.ycsb\">CounterGenerator</A>, <A HREF=\"../../../com/yahoo/ycsb/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb\">ScrambledZipfianGenerator</A>, <A HREF=\"../../../com/yahoo/ycsb/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb\">SkewedLatestGenerator</A>, <A HREF=\"../../../com/yahoo/ycsb/UniformIntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">UniformIntegerGenerator</A>, <A HREF=\"../../../com/yahoo/ycsb/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb\">ZipfianGenerator</A></DD>\n</DL>\n<HR>\n<DL>\n<DT><PRE>public abstract class <B>IntegerGenerator</B><DT>extends <A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">Generator</A></DL>\n</PRE>\n\n<P>\nA generator that is capable of generating ints as well as strings\n<P>\n\n<P>\n<DL>\n<DT><B>Author:</B></DT>\n  <DD>cooperb</DD>\n</DL>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#IntegerGenerator()\">IntegerGenerator</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#lastInt()\">lastInt</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Return the previous int generated by the distribution.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#lastString()\">lastString</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Return the previous string generated by the distribution; e.g., returned from the last nextString() call.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#nextInt()\">nextInt</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Return the next value as an int.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#nextString()\">nextString</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Generate the next string in the distribution.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#setLastInt(int)\">setLastInt</A></B>(int&nbsp;last)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Set the last value generated.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"IntegerGenerator()\"><!-- --></A><H3>\nIntegerGenerator</H3>\n<PRE>\npublic <B>IntegerGenerator</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"setLastInt(int)\"><!-- --></A><H3>\nsetLastInt</H3>\n<PRE>\npublic void <B>setLastInt</B>(int&nbsp;last)</PRE>\n<DL>\n<DD>Set the last value generated. IntegerGenerator subclasses must use this call\n to properly set the last string value, or the lastString() and lastInt() calls won't work.\n<P>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"nextInt()\"><!-- --></A><H3>\nnextInt</H3>\n<PRE>\npublic abstract int <B>nextInt</B>()</PRE>\n<DL>\n<DD>Return the next value as an int. When overriding this method, be sure to call setLastString() properly, or the lastString() call won't work.\n<P>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"nextString()\"><!-- --></A><H3>\nnextString</H3>\n<PRE>\npublic java.lang.String <B>nextString</B>()</PRE>\n<DL>\n<DD>Generate the next string in the distribution.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/Generator.html#nextString()\">nextString</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">Generator</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"lastString()\"><!-- --></A><H3>\nlastString</H3>\n<PRE>\npublic java.lang.String <B>lastString</B>()</PRE>\n<DL>\n<DD>Return the previous string generated by the distribution; e.g., returned from the last nextString() call. \n Calling lastString() should not advance the distribution or have any side effects. If nextString() has not yet \n been called, lastString() should return something reasonable.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/Generator.html#lastString()\">lastString</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">Generator</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"lastInt()\"><!-- --></A><H3>\nlastInt</H3>\n<PRE>\npublic int <B>lastInt</B>()</PRE>\n<DL>\n<DD>Return the previous int generated by the distribution. This call is unique to IntegerGenerator subclasses, and assumes\n IntegerGenerator subclasses always return ints for nextInt() (e.g. not arbitrary strings).\n<P>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Measurements.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/IntegerGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"IntegerGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/Measurements.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Fri Apr 23 10:24:25 PDT 2010 -->\n<TITLE>\nMeasurements\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.Measurements class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"Measurements\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/Measurements.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"Measurements.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass Measurements</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.Measurements</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>Measurements</B><DT>extends java.lang.Object</DL>\n</PRE>\n\n<P>\nCollects latency measurements, and reports them when requested.\n<P>\n\n<P>\n<DL>\n<DT><B>Author:</B></DT>\n  <DD>cooperb</DD>\n</DL>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Measurements.html#Measurements(java.util.Properties)\">Measurements</A></B>(java.util.Properties&nbsp;props)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Measurements.html\" title=\"class in com.yahoo.ycsb\">Measurements</A></CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Measurements.html#getMeasurements()\">getMeasurements</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Measurements.html#getSummary()\">getSummary</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Measurements.html#measure(java.lang.String, int)\">measure</A></B>(java.lang.String&nbsp;operation,\n        int&nbsp;latency)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Measurements.html#printReport(java.io.PrintStream)\">printReport</A></B>(java.io.PrintStream&nbsp;out)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Measurements.html#reportReturnCode(java.lang.String, int)\">reportReturnCode</A></B>(java.lang.String&nbsp;operation,\n                 int&nbsp;code)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Measurements.html#setProperties(java.util.Properties)\">setProperties</A></B>(java.util.Properties&nbsp;props)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"Measurements(java.util.Properties)\"><!-- --></A><H3>\nMeasurements</H3>\n<PRE>\npublic <B>Measurements</B>(java.util.Properties&nbsp;props)</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"setProperties(java.util.Properties)\"><!-- --></A><H3>\nsetProperties</H3>\n<PRE>\npublic static void <B>setProperties</B>(java.util.Properties&nbsp;props)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"getMeasurements()\"><!-- --></A><H3>\ngetMeasurements</H3>\n<PRE>\npublic static <A HREF=\"../../../com/yahoo/ycsb/Measurements.html\" title=\"class in com.yahoo.ycsb\">Measurements</A> <B>getMeasurements</B>()</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"measure(java.lang.String, int)\"><!-- --></A><H3>\nmeasure</H3>\n<PRE>\npublic void <B>measure</B>(java.lang.String&nbsp;operation,\n                    int&nbsp;latency)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"reportReturnCode(java.lang.String, int)\"><!-- --></A><H3>\nreportReturnCode</H3>\n<PRE>\npublic void <B>reportReturnCode</B>(java.lang.String&nbsp;operation,\n                             int&nbsp;code)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"printReport(java.io.PrintStream)\"><!-- --></A><H3>\nprintReport</H3>\n<PRE>\npublic void <B>printReport</B>(java.io.PrintStream&nbsp;out)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"getSummary()\"><!-- --></A><H3>\ngetSummary</H3>\n<PRE>\npublic java.lang.String <B>getSummary</B>()</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/Measurements.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"Measurements.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/OneMeasurement.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Fri Apr 23 10:24:25 PDT 2010 -->\n<TITLE>\nOneMeasurement\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.OneMeasurement class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"OneMeasurement\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Measurements.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/OneMeasurement.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"OneMeasurement.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass OneMeasurement</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.OneMeasurement</B>\n</PRE>\n<DL>\n<DT><B>Direct Known Subclasses:</B> <DD><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb\">OneMeasurementHistogram</A>, <A HREF=\"../../../com/yahoo/ycsb/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb\">OneMeasurementTimeSeries</A></DD>\n</DL>\n<HR>\n<DL>\n<DT><PRE>public abstract class <B>OneMeasurement</B><DT>extends java.lang.Object</DL>\n</PRE>\n\n<P>\nA single measured metric (e.g. READ LATENCY)\n<P>\n\n<P>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#OneMeasurement(java.lang.String)\">OneMeasurement</A></B>(java.lang.String&nbsp;_name)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#getName()\">getName</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#getSummary()\">getSummary</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#measure(int)\">measure</A></B>(int&nbsp;latency)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#printReport(java.io.PrintStream)\">printReport</A></B>(java.io.PrintStream&nbsp;out)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#reportReturnCode(int)\">reportReturnCode</A></B>(int&nbsp;code)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"OneMeasurement(java.lang.String)\"><!-- --></A><H3>\nOneMeasurement</H3>\n<PRE>\npublic <B>OneMeasurement</B>(java.lang.String&nbsp;_name)</PRE>\n<DL>\n<DL>\n<DT><B>Parameters:</B><DD><CODE>_name</CODE> - </DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"getName()\"><!-- --></A><H3>\ngetName</H3>\n<PRE>\npublic java.lang.String <B>getName</B>()</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"reportReturnCode(int)\"><!-- --></A><H3>\nreportReturnCode</H3>\n<PRE>\npublic abstract void <B>reportReturnCode</B>(int&nbsp;code)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"measure(int)\"><!-- --></A><H3>\nmeasure</H3>\n<PRE>\npublic abstract void <B>measure</B>(int&nbsp;latency)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"printReport(java.io.PrintStream)\"><!-- --></A><H3>\nprintReport</H3>\n<PRE>\npublic abstract void <B>printReport</B>(java.io.PrintStream&nbsp;out)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"getSummary()\"><!-- --></A><H3>\ngetSummary</H3>\n<PRE>\npublic abstract java.lang.String <B>getSummary</B>()</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Measurements.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/OneMeasurement.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"OneMeasurement.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/OneMeasurementHistogram.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Fri Apr 23 10:24:25 PDT 2010 -->\n<TITLE>\nOneMeasurementHistogram\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.OneMeasurementHistogram class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"OneMeasurementHistogram\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/OneMeasurementHistogram.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"OneMeasurementHistogram.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass OneMeasurementHistogram</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.OneMeasurement</A>\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.OneMeasurementHistogram</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>OneMeasurementHistogram</B><DT>extends <A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">OneMeasurement</A></DL>\n</PRE>\n\n<P>\nTake measurements and maintain a histogram of a given metric, such as READ LATENCY.\n<P>\n\n<P>\n<DL>\n<DT><B>Author:</B></DT>\n  <DD>cooperb</DD>\n</DL>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementHistogram.html#BUCKETS\">BUCKETS</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementHistogram.html#BUCKETS_DEFAULT\">BUCKETS_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementHistogram.html#OneMeasurementHistogram(java.lang.String, java.util.Properties)\">OneMeasurementHistogram</A></B>(java.lang.String&nbsp;name,\n                        java.util.Properties&nbsp;props)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementHistogram.html#getSummary()\">getSummary</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementHistogram.html#measure(int)\">measure</A></B>(int&nbsp;latency)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementHistogram.html#printReport(java.io.PrintStream)\">printReport</A></B>(java.io.PrintStream&nbsp;out)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementHistogram.html#reportReturnCode(int)\">reportReturnCode</A></B>(int&nbsp;code)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_com.yahoo.ycsb.OneMeasurement\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">OneMeasurement</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#getName()\">getName</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"BUCKETS\"><!-- --></A><H3>\nBUCKETS</H3>\n<PRE>\npublic static final java.lang.String <B>BUCKETS</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.OneMeasurementHistogram.BUCKETS\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"BUCKETS_DEFAULT\"><!-- --></A><H3>\nBUCKETS_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>BUCKETS_DEFAULT</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.OneMeasurementHistogram.BUCKETS_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"OneMeasurementHistogram(java.lang.String, java.util.Properties)\"><!-- --></A><H3>\nOneMeasurementHistogram</H3>\n<PRE>\npublic <B>OneMeasurementHistogram</B>(java.lang.String&nbsp;name,\n                               java.util.Properties&nbsp;props)</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"reportReturnCode(int)\"><!-- --></A><H3>\nreportReturnCode</H3>\n<PRE>\npublic void <B>reportReturnCode</B>(int&nbsp;code)</PRE>\n<DL>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#reportReturnCode(int)\">reportReturnCode</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">OneMeasurement</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"measure(int)\"><!-- --></A><H3>\nmeasure</H3>\n<PRE>\npublic void <B>measure</B>(int&nbsp;latency)</PRE>\n<DL>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#measure(int)\">measure</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">OneMeasurement</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"printReport(java.io.PrintStream)\"><!-- --></A><H3>\nprintReport</H3>\n<PRE>\npublic void <B>printReport</B>(java.io.PrintStream&nbsp;out)</PRE>\n<DL>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#printReport(java.io.PrintStream)\">printReport</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">OneMeasurement</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"getSummary()\"><!-- --></A><H3>\ngetSummary</H3>\n<PRE>\npublic java.lang.String <B>getSummary</B>()</PRE>\n<DL>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#getSummary()\">getSummary</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">OneMeasurement</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/OneMeasurementHistogram.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"OneMeasurementHistogram.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/OneMeasurementTimeSeries.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Fri Apr 23 10:24:25 PDT 2010 -->\n<TITLE>\nOneMeasurementTimeSeries\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.OneMeasurementTimeSeries class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"OneMeasurementTimeSeries\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/OneMeasurementTimeSeries.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"OneMeasurementTimeSeries.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass OneMeasurementTimeSeries</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.OneMeasurement</A>\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.OneMeasurementTimeSeries</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>OneMeasurementTimeSeries</B><DT>extends <A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">OneMeasurement</A></DL>\n</PRE>\n\n<P>\nA time series measurement of a metric, such as READ LATENCY.\n<P>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementTimeSeries.html#GRANULARITY\">GRANULARITY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Granularity for time series; measurements will be averaged in chunks of this granularity.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementTimeSeries.html#GRANULARITY_DEFAULT\">GRANULARITY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementTimeSeries.html#OneMeasurementTimeSeries(java.lang.String, java.util.Properties)\">OneMeasurementTimeSeries</A></B>(java.lang.String&nbsp;name,\n                         java.util.Properties&nbsp;props)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementTimeSeries.html#getSummary()\">getSummary</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementTimeSeries.html#measure(int)\">measure</A></B>(int&nbsp;latency)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementTimeSeries.html#printReport(java.io.PrintStream)\">printReport</A></B>(java.io.PrintStream&nbsp;out)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/OneMeasurementTimeSeries.html#reportReturnCode(int)\">reportReturnCode</A></B>(int&nbsp;code)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_com.yahoo.ycsb.OneMeasurement\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">OneMeasurement</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#getName()\">getName</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"GRANULARITY\"><!-- --></A><H3>\nGRANULARITY</H3>\n<PRE>\npublic static final java.lang.String <B>GRANULARITY</B></PRE>\n<DL>\n<DD>Granularity for time series; measurements will be averaged in chunks of this granularity. Units are milliseconds.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.OneMeasurementTimeSeries.GRANULARITY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"GRANULARITY_DEFAULT\"><!-- --></A><H3>\nGRANULARITY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>GRANULARITY_DEFAULT</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.OneMeasurementTimeSeries.GRANULARITY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"OneMeasurementTimeSeries(java.lang.String, java.util.Properties)\"><!-- --></A><H3>\nOneMeasurementTimeSeries</H3>\n<PRE>\npublic <B>OneMeasurementTimeSeries</B>(java.lang.String&nbsp;name,\n                                java.util.Properties&nbsp;props)</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"measure(int)\"><!-- --></A><H3>\nmeasure</H3>\n<PRE>\npublic void <B>measure</B>(int&nbsp;latency)</PRE>\n<DL>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#measure(int)\">measure</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">OneMeasurement</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"printReport(java.io.PrintStream)\"><!-- --></A><H3>\nprintReport</H3>\n<PRE>\npublic void <B>printReport</B>(java.io.PrintStream&nbsp;out)</PRE>\n<DL>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#printReport(java.io.PrintStream)\">printReport</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">OneMeasurement</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"reportReturnCode(int)\"><!-- --></A><H3>\nreportReturnCode</H3>\n<PRE>\npublic void <B>reportReturnCode</B>(int&nbsp;code)</PRE>\n<DL>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#reportReturnCode(int)\">reportReturnCode</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">OneMeasurement</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"getSummary()\"><!-- --></A><H3>\ngetSummary</H3>\n<PRE>\npublic java.lang.String <B>getSummary</B>()</PRE>\n<DL>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html#getSummary()\">getSummary</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/OneMeasurement.html\" title=\"class in com.yahoo.ycsb\">OneMeasurement</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/OneMeasurementTimeSeries.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"OneMeasurementTimeSeries.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/ScrambledZipfianGenerator.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Fri Apr 23 10:24:25 PDT 2010 -->\n<TITLE>\nScrambledZipfianGenerator\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.ScrambledZipfianGenerator class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"ScrambledZipfianGenerator\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/ScrambledZipfianGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"ScrambledZipfianGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass ScrambledZipfianGenerator</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.Generator</A>\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.IntegerGenerator</A>\n          <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.ScrambledZipfianGenerator</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>ScrambledZipfianGenerator</B><DT>extends <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></DL>\n</PRE>\n\n<P>\nA generator of a zipfian distribution. It produces a sequence of items, such that some items are more popular than others, according\n to a zipfian distribution. When you construct an instance of this class, you specify the number of items in the set to draw from, either\n by specifying an itemcount (so that the sequence is of items from 0 to itemcount-1) or by specifying a min and a max (so that the sequence is of \n items from min to max inclusive). After you construct the instance, you can change the number of items by calling nextInt(itemcount) or nextLong(itemcount).\n \n Unlike @ZipfianGenerator, this class scatters the \"popular\" items across the itemspace. Use this, instead of @ZipfianGenerator, if you\n don't want the head of the distribution (the popular items) clustered together.\n<P>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;long</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ScrambledZipfianGenerator.html#ITEM_COUNT\">ITEM_COUNT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;double</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ScrambledZipfianGenerator.html#ZETAN\">ZETAN</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ScrambledZipfianGenerator.html#ScrambledZipfianGenerator(long)\">ScrambledZipfianGenerator</A></B>(long&nbsp;_items)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a zipfian generator for the specified number of items.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ScrambledZipfianGenerator.html#ScrambledZipfianGenerator(long, long)\">ScrambledZipfianGenerator</A></B>(long&nbsp;_min,\n                          long&nbsp;_max)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a zipfian generator for items between min and max.</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ScrambledZipfianGenerator.html#main(java.lang.String[])\">main</A></B>(java.lang.String[]&nbsp;args)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ScrambledZipfianGenerator.html#nextInt()\">nextInt</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Return the next int in the sequence.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;long</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ScrambledZipfianGenerator.html#nextLong()\">nextLong</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Return the next long in the sequence.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_com.yahoo.ycsb.IntegerGenerator\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#lastInt()\">lastInt</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#lastString()\">lastString</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#nextString()\">nextString</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#setLastInt(int)\">setLastInt</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"ZETAN\"><!-- --></A><H3>\nZETAN</H3>\n<PRE>\npublic static final double <B>ZETAN</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.ScrambledZipfianGenerator.ZETAN\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"ITEM_COUNT\"><!-- --></A><H3>\nITEM_COUNT</H3>\n<PRE>\npublic static final long <B>ITEM_COUNT</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.ScrambledZipfianGenerator.ITEM_COUNT\">Constant Field Values</A></DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"ScrambledZipfianGenerator(long)\"><!-- --></A><H3>\nScrambledZipfianGenerator</H3>\n<PRE>\npublic <B>ScrambledZipfianGenerator</B>(long&nbsp;_items)</PRE>\n<DL>\n<DD>Create a zipfian generator for the specified number of items.\n<P>\n<DL>\n<DT><B>Parameters:</B><DD><CODE>_items</CODE> - The number of items in the distribution.</DL>\n</DL>\n<HR>\n\n<A NAME=\"ScrambledZipfianGenerator(long, long)\"><!-- --></A><H3>\nScrambledZipfianGenerator</H3>\n<PRE>\npublic <B>ScrambledZipfianGenerator</B>(long&nbsp;_min,\n                                 long&nbsp;_max)</PRE>\n<DL>\n<DD>Create a zipfian generator for items between min and max.\n<P>\n<DL>\n<DT><B>Parameters:</B><DD><CODE>_min</CODE> - The smallest integer to generate in the sequence.<DD><CODE>_max</CODE> - The largest integer to generate in the sequence.</DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"nextInt()\"><!-- --></A><H3>\nnextInt</H3>\n<PRE>\npublic int <B>nextInt</B>()</PRE>\n<DL>\n<DD>Return the next int in the sequence.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#nextInt()\">nextInt</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"nextLong()\"><!-- --></A><H3>\nnextLong</H3>\n<PRE>\npublic long <B>nextLong</B>()</PRE>\n<DL>\n<DD>Return the next long in the sequence.\n<P>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"main(java.lang.String[])\"><!-- --></A><H3>\nmain</H3>\n<PRE>\npublic static void <B>main</B>(java.lang.String[]&nbsp;args)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/ScrambledZipfianGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"ScrambledZipfianGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/SkewedLatestGenerator.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Fri Apr 23 10:24:25 PDT 2010 -->\n<TITLE>\nSkewedLatestGenerator\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.SkewedLatestGenerator class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"SkewedLatestGenerator\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UniformGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/SkewedLatestGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"SkewedLatestGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass SkewedLatestGenerator</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.Generator</A>\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.IntegerGenerator</A>\n          <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.SkewedLatestGenerator</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>SkewedLatestGenerator</B><DT>extends <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></DL>\n</PRE>\n\n<P>\nGenerate a popularity distribution of items, skewed to favor recent items significantly more than older items.\n<P>\n\n<P>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/SkewedLatestGenerator.html#SkewedLatestGenerator(com.yahoo.ycsb.CounterGenerator)\">SkewedLatestGenerator</A></B>(<A HREF=\"../../../com/yahoo/ycsb/CounterGenerator.html\" title=\"class in com.yahoo.ycsb\">CounterGenerator</A>&nbsp;basis)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/SkewedLatestGenerator.html#main(java.lang.String[])\">main</A></B>(java.lang.String[]&nbsp;args)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/SkewedLatestGenerator.html#nextInt()\">nextInt</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Generate the next string in the distribution, skewed Zipfian favoring the items most recently returned by the basis generator.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_com.yahoo.ycsb.IntegerGenerator\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#lastInt()\">lastInt</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#lastString()\">lastString</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#nextString()\">nextString</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#setLastInt(int)\">setLastInt</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"SkewedLatestGenerator(com.yahoo.ycsb.CounterGenerator)\"><!-- --></A><H3>\nSkewedLatestGenerator</H3>\n<PRE>\npublic <B>SkewedLatestGenerator</B>(<A HREF=\"../../../com/yahoo/ycsb/CounterGenerator.html\" title=\"class in com.yahoo.ycsb\">CounterGenerator</A>&nbsp;basis)</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"nextInt()\"><!-- --></A><H3>\nnextInt</H3>\n<PRE>\npublic int <B>nextInt</B>()</PRE>\n<DL>\n<DD>Generate the next string in the distribution, skewed Zipfian favoring the items most recently returned by the basis generator.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#nextInt()\">nextInt</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"main(java.lang.String[])\"><!-- --></A><H3>\nmain</H3>\n<PRE>\npublic static void <B>main</B>(java.lang.String[]&nbsp;args)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UniformGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/SkewedLatestGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"SkewedLatestGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/TestCollisions.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Tue Apr 20 15:39:17 PDT 2010 -->\n<TITLE>\nTestCollisions\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.TestCollisions class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"TestCollisions\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/TestExpandedZipfian.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/TestCollisions.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"TestCollisions.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass TestCollisions</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.TestCollisions</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>TestCollisions</B><DT>extends java.lang.Object</DL>\n</PRE>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/TestCollisions.html#a\">a</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/TestCollisions.html#b\">b</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/TestCollisions.html#TestCollisions()\">TestCollisions</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/TestCollisions.html#hash(int, int)\">hash</A></B>(int&nbsp;val,\n     int&nbsp;itemcount)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/TestCollisions.html#main(java.lang.String[])\">main</A></B>(java.lang.String[]&nbsp;args)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/TestCollisions.html#scramble(int)\">scramble</A></B>(int&nbsp;val)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/TestCollisions.html#testVector(int)\">testVector</A></B>(int&nbsp;itemcount)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"a\"><!-- --></A><H3>\na</H3>\n<PRE>\npublic static int <B>a</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"b\"><!-- --></A><H3>\nb</H3>\n<PRE>\npublic static int <B>b</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"TestCollisions()\"><!-- --></A><H3>\nTestCollisions</H3>\n<PRE>\npublic <B>TestCollisions</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"scramble(int)\"><!-- --></A><H3>\nscramble</H3>\n<PRE>\npublic static int <B>scramble</B>(int&nbsp;val)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"hash(int, int)\"><!-- --></A><H3>\nhash</H3>\n<PRE>\npublic static int <B>hash</B>(int&nbsp;val,\n                       int&nbsp;itemcount)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"main(java.lang.String[])\"><!-- --></A><H3>\nmain</H3>\n<PRE>\npublic static void <B>main</B>(java.lang.String[]&nbsp;args)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"testVector(int)\"><!-- --></A><H3>\ntestVector</H3>\n<PRE>\npublic static int <B>testVector</B>(int&nbsp;itemcount)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/TestExpandedZipfian.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/TestCollisions.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"TestCollisions.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/TestExpandedZipfian.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Tue Apr 20 15:39:17 PDT 2010 -->\n<TITLE>\nTestExpandedZipfian\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.TestExpandedZipfian class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"TestExpandedZipfian\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/TestCollisions.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/TestZipfian.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/TestExpandedZipfian.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"TestExpandedZipfian.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass TestExpandedZipfian</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.TestExpandedZipfian</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>TestExpandedZipfian</B><DT>extends java.lang.Object</DL>\n</PRE>\n\n<P>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/TestExpandedZipfian.html#TestExpandedZipfian()\">TestExpandedZipfian</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/TestExpandedZipfian.html#main(java.lang.String[])\">main</A></B>(java.lang.String[]&nbsp;args)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"TestExpandedZipfian()\"><!-- --></A><H3>\nTestExpandedZipfian</H3>\n<PRE>\npublic <B>TestExpandedZipfian</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"main(java.lang.String[])\"><!-- --></A><H3>\nmain</H3>\n<PRE>\npublic static void <B>main</B>(java.lang.String[]&nbsp;args)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/TestCollisions.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/TestZipfian.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/TestExpandedZipfian.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"TestExpandedZipfian.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/TestZipfian.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Tue Apr 20 15:39:17 PDT 2010 -->\n<TITLE>\nTestZipfian\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.TestZipfian class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"TestZipfian\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/TestExpandedZipfian.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UniformGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/TestZipfian.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"TestZipfian.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass TestZipfian</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.TestZipfian</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>TestZipfian</B><DT>extends java.lang.Object</DL>\n</PRE>\n\n<P>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/TestZipfian.html#TestZipfian()\">TestZipfian</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/TestZipfian.html#main(java.lang.String[])\">main</A></B>(java.lang.String[]&nbsp;args)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"TestZipfian()\"><!-- --></A><H3>\nTestZipfian</H3>\n<PRE>\npublic <B>TestZipfian</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"main(java.lang.String[])\"><!-- --></A><H3>\nmain</H3>\n<PRE>\npublic static void <B>main</B>(java.lang.String[]&nbsp;args)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/TestExpandedZipfian.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UniformGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/TestZipfian.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"TestZipfian.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/UniformGenerator.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Fri Apr 23 10:24:25 PDT 2010 -->\n<TITLE>\nUniformGenerator\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.UniformGenerator class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"UniformGenerator\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UniformIntegerGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/UniformGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"UniformGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass UniformGenerator</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.Generator</A>\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.UniformGenerator</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>UniformGenerator</B><DT>extends <A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">Generator</A></DL>\n</PRE>\n\n<P>\nAn expression that generates a random integer in the specified range\n<P>\n\n<P>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/UniformGenerator.html#UniformGenerator(java.util.Vector)\">UniformGenerator</A></B>(java.util.Vector&lt;java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Creates a generator that will return strings from the specified set uniformly randomly</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/UniformGenerator.html#lastString()\">lastString</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Return the previous string generated by the distribution; e.g., returned from the last nextString() call.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/UniformGenerator.html#nextString()\">nextString</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Generate the next string in the distribution.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"UniformGenerator(java.util.Vector)\"><!-- --></A><H3>\nUniformGenerator</H3>\n<PRE>\npublic <B>UniformGenerator</B>(java.util.Vector&lt;java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Creates a generator that will return strings from the specified set uniformly randomly\n<P>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"nextString()\"><!-- --></A><H3>\nnextString</H3>\n<PRE>\npublic java.lang.String <B>nextString</B>()</PRE>\n<DL>\n<DD>Generate the next string in the distribution.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/Generator.html#nextString()\">nextString</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">Generator</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"lastString()\"><!-- --></A><H3>\nlastString</H3>\n<PRE>\npublic java.lang.String <B>lastString</B>()</PRE>\n<DL>\n<DD>Return the previous string generated by the distribution; e.g., returned from the last nextString() call. \n Calling lastString() should not advance the distribution or have any side effects. If nextString() has not yet \n been called, lastString() should return something reasonable.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/Generator.html#lastString()\">lastString</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">Generator</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UniformIntegerGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/UniformGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"UniformGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/UniformIntegerGenerator.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Fri Apr 23 10:24:25 PDT 2010 -->\n<TITLE>\nUniformIntegerGenerator\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.UniformIntegerGenerator class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"UniformIntegerGenerator\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UniformGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/UniformIntegerGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"UniformIntegerGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass UniformIntegerGenerator</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.Generator</A>\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.IntegerGenerator</A>\n          <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.UniformIntegerGenerator</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>UniformIntegerGenerator</B><DT>extends <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></DL>\n</PRE>\n\n<P>\nGenerates integers randomly uniform from an interval.\n<P>\n\n<P>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/UniformIntegerGenerator.html#UniformIntegerGenerator(int, int)\">UniformIntegerGenerator</A></B>(int&nbsp;lb,\n                        int&nbsp;ub)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Creates a generator that will return integers uniformly randomly from the interval [lb,ub] inclusive (that is, lb and ub are possible values)</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/UniformIntegerGenerator.html#nextInt()\">nextInt</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Return the next value as an int.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_com.yahoo.ycsb.IntegerGenerator\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#lastInt()\">lastInt</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#lastString()\">lastString</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#nextString()\">nextString</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#setLastInt(int)\">setLastInt</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"UniformIntegerGenerator(int, int)\"><!-- --></A><H3>\nUniformIntegerGenerator</H3>\n<PRE>\npublic <B>UniformIntegerGenerator</B>(int&nbsp;lb,\n                               int&nbsp;ub)</PRE>\n<DL>\n<DD>Creates a generator that will return integers uniformly randomly from the interval [lb,ub] inclusive (that is, lb and ub are possible values)\n<P>\n<DL>\n<DT><B>Parameters:</B><DD><CODE>lb</CODE> - the lower bound (inclusive) of generated values<DD><CODE>ub</CODE> - the upper bound (inclusive) of generated values</DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"nextInt()\"><!-- --></A><H3>\nnextInt</H3>\n<PRE>\npublic int <B>nextInt</B>()</PRE>\n<DL>\n<DD><B>Description copied from class: <CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#nextInt()\">IntegerGenerator</A></CODE></B></DD>\n<DD>Return the next value as an int. When overriding this method, be sure to call setLastString() properly, or the lastString() call won't work.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#nextInt()\">nextInt</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UniformGenerator.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/UniformIntegerGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"UniformIntegerGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/UnknownDBException.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:52 PDT 2010 -->\n<TITLE>\nUnknownDBException\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"UnknownDBException\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/UnknownDBException.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"UnknownDBException.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#methods_inherited_from_class_java.lang.Throwable\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;METHOD</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass UnknownDBException</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \">java.lang.Throwable\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \">java.lang.Exception\n          <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.UnknownDBException</B>\n</PRE>\n<DL>\n<DT><B>All Implemented Interfaces:</B> <DD>java.io.Serializable</DD>\n</DL>\n<HR>\n<DL>\n<DT><PRE>public class <B>UnknownDBException</B><DT>extends java.lang.Exception</DL>\n</PRE>\n\n<P>\nCould not create the specified DB.\n<P>\n\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../serialized-form.html#com.yahoo.ycsb.UnknownDBException\">Serialized Form</A></DL>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html#UnknownDBException()\">UnknownDBException</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html#UnknownDBException(java.lang.String)\">UnknownDBException</A></B>(java.lang.String&nbsp;message)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html#UnknownDBException(java.lang.String, java.lang.Throwable)\">UnknownDBException</A></B>(java.lang.String&nbsp;message,\n                   java.lang.Throwable&nbsp;cause)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html#UnknownDBException(java.lang.Throwable)\">UnknownDBException</A></B>(java.lang.Throwable&nbsp;cause)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Throwable\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Throwable</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>fillInStackTrace, getCause, getLocalizedMessage, getMessage, getStackTrace, initCause, printStackTrace, printStackTrace, printStackTrace, setStackTrace, toString</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"UnknownDBException(java.lang.String)\"><!-- --></A><H3>\nUnknownDBException</H3>\n<PRE>\npublic <B>UnknownDBException</B>(java.lang.String&nbsp;message)</PRE>\n<DL>\n</DL>\n<HR>\n\n<A NAME=\"UnknownDBException()\"><!-- --></A><H3>\nUnknownDBException</H3>\n<PRE>\npublic <B>UnknownDBException</B>()</PRE>\n<DL>\n</DL>\n<HR>\n\n<A NAME=\"UnknownDBException(java.lang.String, java.lang.Throwable)\"><!-- --></A><H3>\nUnknownDBException</H3>\n<PRE>\npublic <B>UnknownDBException</B>(java.lang.String&nbsp;message,\n                          java.lang.Throwable&nbsp;cause)</PRE>\n<DL>\n</DL>\n<HR>\n\n<A NAME=\"UnknownDBException(java.lang.Throwable)\"><!-- --></A><H3>\nUnknownDBException</H3>\n<PRE>\npublic <B>UnknownDBException</B>(java.lang.Throwable&nbsp;cause)</PRE>\n<DL>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/UnknownDBException.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"UnknownDBException.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#methods_inherited_from_class_java.lang.Throwable\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;METHOD</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/Utils.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:52 PDT 2010 -->\n<TITLE>\nUtils\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"Utils\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/Utils.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"Utils.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass Utils</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.Utils</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>Utils</B><DT>extends java.lang.Object</DL>\n</PRE>\n\n<P>\nUtility functions.\n<P>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Utils.html#FNV_offset_basis_32\">FNV_offset_basis_32</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;long</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Utils.html#FNV_offset_basis_64\">FNV_offset_basis_64</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Utils.html#FNV_prime_32\">FNV_prime_32</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;long</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Utils.html#FNV_prime_64\">FNV_prime_64</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Utils.html#Utils()\">Utils</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Utils.html#ASCIIString(int)\">ASCIIString</A></B>(int&nbsp;length)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Generate a random ASCII string of a given length.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Utils.html#FNVhash32(int)\">FNVhash32</A></B>(int&nbsp;val)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;32 bit FNV hash.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;long</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Utils.html#FNVhash64(long)\">FNVhash64</A></B>(long&nbsp;val)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;64 bit FNV hash.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Utils.html#hash(int)\">hash</A></B>(int&nbsp;val)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hash an integer value.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"FNV_offset_basis_32\"><!-- --></A><H3>\nFNV_offset_basis_32</H3>\n<PRE>\npublic static final int <B>FNV_offset_basis_32</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.Utils.FNV_offset_basis_32\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"FNV_prime_32\"><!-- --></A><H3>\nFNV_prime_32</H3>\n<PRE>\npublic static final int <B>FNV_prime_32</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.Utils.FNV_prime_32\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"FNV_offset_basis_64\"><!-- --></A><H3>\nFNV_offset_basis_64</H3>\n<PRE>\npublic static final long <B>FNV_offset_basis_64</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.Utils.FNV_offset_basis_64\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"FNV_prime_64\"><!-- --></A><H3>\nFNV_prime_64</H3>\n<PRE>\npublic static final long <B>FNV_prime_64</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.Utils.FNV_prime_64\">Constant Field Values</A></DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"Utils()\"><!-- --></A><H3>\nUtils</H3>\n<PRE>\npublic <B>Utils</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"ASCIIString(int)\"><!-- --></A><H3>\nASCIIString</H3>\n<PRE>\npublic static java.lang.String <B>ASCIIString</B>(int&nbsp;length)</PRE>\n<DL>\n<DD>Generate a random ASCII string of a given length.\n<P>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"hash(int)\"><!-- --></A><H3>\nhash</H3>\n<PRE>\npublic static int <B>hash</B>(int&nbsp;val)</PRE>\n<DL>\n<DD>Hash an integer value.\n<P>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"FNVhash32(int)\"><!-- --></A><H3>\nFNVhash32</H3>\n<PRE>\npublic static int <B>FNVhash32</B>(int&nbsp;val)</PRE>\n<DL>\n<DD>32 bit FNV hash. Produces more \"random\" hashes than (say) String.hashCode().\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>val</CODE> - The value to hash.\n<DT><B>Returns:</B><DD>The hash value</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"FNVhash64(long)\"><!-- --></A><H3>\nFNVhash64</H3>\n<PRE>\npublic static long <B>FNVhash64</B>(long&nbsp;val)</PRE>\n<DL>\n<DD>64 bit FNV hash. Produces more \"random\" hashes than (say) String.hashCode().\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>val</CODE> - The value to hash.\n<DT><B>Returns:</B><DD>The hash value</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/Utils.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"Utils.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/Workload.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:52 PDT 2010 -->\n<TITLE>\nWorkload\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"Workload\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/Workload.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"Workload.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass Workload</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.Workload</B>\n</PRE>\n<DL>\n<DT><B>Direct Known Subclasses:</B> <DD><A HREF=\"../../../com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A></DD>\n</DL>\n<HR>\n<DL>\n<DT><PRE>public abstract class <B>Workload</B><DT>extends java.lang.Object</DL>\n</PRE>\n\n<P>\nOne experiment scenario. One object of this type will\n be instantiated and shared among all client threads. This class\n should be constructed using a no-argument constructor, so we can\n load it dynamically. Any argument-based initialization should be\n done by init().\n \n If you extend this class, you should support the \"insertstart\" property. This \n allows the load phase to proceed from multiple clients on different machines, in case\n the client is the bottleneck. For example, if we want to load 1 million records from\n 2 machines, the first machine should have insertstart=0 and the second insertstart=500000. Additionally,\n the \"insertcount\" property, which is interpreted by Client, can be used to tell each instance of the\n client how many inserts to do. In the example above, both clients should have insertcount=500000.\n<P>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Workload.html#INSERT_START_PROPERTY\">INSERT_START_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Workload.html#INSERT_START_PROPERTY_DEFAULT\">INSERT_START_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Workload.html#Workload()\">Workload</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Workload.html#cleanup()\">cleanup</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cleanup the scenario.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;boolean</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Workload.html#doInsert(com.yahoo.ycsb.DB, java.lang.Object)\">doInsert</A></B>(<A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db,\n         java.lang.Object&nbsp;threadstate)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Do one insert operation.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>abstract &nbsp;boolean</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Workload.html#doTransaction(com.yahoo.ycsb.DB, java.lang.Object)\">doTransaction</A></B>(<A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db,\n              java.lang.Object&nbsp;threadstate)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Do one transaction operation.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Workload.html#init(java.util.Properties)\">init</A></B>(java.util.Properties&nbsp;p)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Initialize the scenario.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.Object</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/Workload.html#initThread(java.util.Properties, int, int)\">initThread</A></B>(java.util.Properties&nbsp;p,\n           int&nbsp;mythreadid,\n           int&nbsp;threadcount)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Initialize any state for a particular client thread.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"INSERT_START_PROPERTY\"><!-- --></A><H3>\nINSERT_START_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>INSERT_START_PROPERTY</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.Workload.INSERT_START_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"INSERT_START_PROPERTY_DEFAULT\"><!-- --></A><H3>\nINSERT_START_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>INSERT_START_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.Workload.INSERT_START_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"Workload()\"><!-- --></A><H3>\nWorkload</H3>\n<PRE>\npublic <B>Workload</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"init(java.util.Properties)\"><!-- --></A><H3>\ninit</H3>\n<PRE>\npublic void <B>init</B>(java.util.Properties&nbsp;p)\n          throws <A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A></PRE>\n<DL>\n<DD>Initialize the scenario. Create any generators and other shared objects here.\n Called once, in the main client thread, before any operations are started.\n<P>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"initThread(java.util.Properties, int, int)\"><!-- --></A><H3>\ninitThread</H3>\n<PRE>\npublic java.lang.Object <B>initThread</B>(java.util.Properties&nbsp;p,\n                                   int&nbsp;mythreadid,\n                                   int&nbsp;threadcount)\n                            throws <A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A></PRE>\n<DL>\n<DD>Initialize any state for a particular client thread. Since the scenario object\n will be shared among all threads, this is the place to create any state that is specific\n to one thread. To be clear, this means the returned object should be created anew on each\n call to initThread(); do not return the same object multiple times. \n The returned object will be passed to invocations of doInsert() and doTransaction() \n for this thread. There should be no side effects from this call; all state should be encapsulated\n in the returned object. If you have no state to retain for this thread, return null. (But if you have\n no state to retain for this thread, probably you don't need to override initThread().)\n<P>\n<DD><DL>\n\n<DT><B>Returns:</B><DD>false if the workload knows it is done for this thread. Client will terminate the thread. Return true otherwise. Return true for workloads that rely on operationcount. For workloads that read traces from a file, return true when there are more to do, false when you are done.\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"cleanup()\"><!-- --></A><H3>\ncleanup</H3>\n<PRE>\npublic void <B>cleanup</B>()\n             throws <A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A></PRE>\n<DL>\n<DD>Cleanup the scenario. Called once, in the main client thread, after all operations have completed.\n<P>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"doInsert(com.yahoo.ycsb.DB, java.lang.Object)\"><!-- --></A><H3>\ndoInsert</H3>\n<PRE>\npublic abstract boolean <B>doInsert</B>(<A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db,\n                                 java.lang.Object&nbsp;threadstate)</PRE>\n<DL>\n<DD>Do one insert operation. Because it will be called concurrently from multiple client threads, this \n function must be thread safe. However, avoid synchronized, or the threads will block waiting for each \n other, and it will be difficult to reach the target throughput. Ideally, this function would have no side\n effects other than DB operations and mutations on threadstate. Mutations to threadstate do not need to be\n synchronized, since each thread has its own threadstate instance.\n<P>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"doTransaction(com.yahoo.ycsb.DB, java.lang.Object)\"><!-- --></A><H3>\ndoTransaction</H3>\n<PRE>\npublic abstract boolean <B>doTransaction</B>(<A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db,\n                                      java.lang.Object&nbsp;threadstate)</PRE>\n<DL>\n<DD>Do one transaction operation. Because it will be called concurrently from multiple client threads, this \n function must be thread safe. However, avoid synchronized, or the threads will block waiting for each \n other, and it will be difficult to reach the target throughput. Ideally, this function would have no side\n effects other than DB operations and mutations on threadstate. Mutations to threadstate do not need to be\n synchronized, since each thread has its own threadstate instance.\n<P>\n<DD><DL>\n\n<DT><B>Returns:</B><DD>false if the workload knows it is done for this thread. Client will terminate the thread. Return true otherwise. Return true for workloads that rely on operationcount. For workloads that read traces from a file, return true when there are more to do, false when you are done.</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/Workload.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"Workload.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/WorkloadException.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:52 PDT 2010 -->\n<TITLE>\nWorkloadException\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"WorkloadException\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;NEXT CLASS</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/WorkloadException.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"WorkloadException.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#methods_inherited_from_class_java.lang.Throwable\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;METHOD</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass WorkloadException</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \">java.lang.Throwable\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \">java.lang.Exception\n          <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.WorkloadException</B>\n</PRE>\n<DL>\n<DT><B>All Implemented Interfaces:</B> <DD>java.io.Serializable</DD>\n</DL>\n<HR>\n<DL>\n<DT><PRE>public class <B>WorkloadException</B><DT>extends java.lang.Exception</DL>\n</PRE>\n\n<P>\nThe workload tried to do something bad.\n<P>\n\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../serialized-form.html#com.yahoo.ycsb.WorkloadException\">Serialized Form</A></DL>\n<HR>\n\n<P>\n\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html#WorkloadException()\">WorkloadException</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html#WorkloadException(java.lang.String)\">WorkloadException</A></B>(java.lang.String&nbsp;message)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html#WorkloadException(java.lang.String, java.lang.Throwable)\">WorkloadException</A></B>(java.lang.String&nbsp;message,\n                  java.lang.Throwable&nbsp;cause)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html#WorkloadException(java.lang.Throwable)\">WorkloadException</A></B>(java.lang.Throwable&nbsp;cause)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Throwable\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Throwable</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>fillInStackTrace, getCause, getLocalizedMessage, getMessage, getStackTrace, initCause, printStackTrace, printStackTrace, printStackTrace, setStackTrace, toString</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"WorkloadException(java.lang.String)\"><!-- --></A><H3>\nWorkloadException</H3>\n<PRE>\npublic <B>WorkloadException</B>(java.lang.String&nbsp;message)</PRE>\n<DL>\n</DL>\n<HR>\n\n<A NAME=\"WorkloadException()\"><!-- --></A><H3>\nWorkloadException</H3>\n<PRE>\npublic <B>WorkloadException</B>()</PRE>\n<DL>\n</DL>\n<HR>\n\n<A NAME=\"WorkloadException(java.lang.String, java.lang.Throwable)\"><!-- --></A><H3>\nWorkloadException</H3>\n<PRE>\npublic <B>WorkloadException</B>(java.lang.String&nbsp;message,\n                         java.lang.Throwable&nbsp;cause)</PRE>\n<DL>\n</DL>\n<HR>\n\n<A NAME=\"WorkloadException(java.lang.Throwable)\"><!-- --></A><H3>\nWorkloadException</H3>\n<PRE>\npublic <B>WorkloadException</B>(java.lang.Throwable&nbsp;cause)</PRE>\n<DL>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;NEXT CLASS</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/WorkloadException.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"WorkloadException.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#methods_inherited_from_class_java.lang.Throwable\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;FIELD&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;METHOD</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/ZipfianGenerator.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Fri Apr 23 10:24:25 PDT 2010 -->\n<TITLE>\nZipfianGenerator\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.ZipfianGenerator class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"ZipfianGenerator\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;NEXT CLASS</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/ZipfianGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"ZipfianGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb</FONT>\n<BR>\nClass ZipfianGenerator</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/Generator.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.Generator</A>\n      <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.IntegerGenerator</A>\n          <IMG SRC=\"../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.ZipfianGenerator</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>ZipfianGenerator</B><DT>extends <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></DL>\n</PRE>\n\n<P>\nA generator of a zipfian distribution. It produces a sequence of items, such that some items are more popular than others, according\n to a zipfian distribution. When you construct an instance of this class, you specify the number of items in the set to draw from, either\n by specifying an itemcount (so that the sequence is of items from 0 to itemcount-1) or by specifying a min and a max (so that the sequence is of \n items from min to max inclusive). After you construct the instance, you can change the number of items by calling nextInt(itemcount) or nextLong(itemcount).\n \n Note that the popular items will be clustered together, e.g. item 0 is the most popular, item 1 the second most popular, and so on (or min is the most \n popular, min+1 the next most popular, etc.) If you don't want this clustering, and instead want the popular items scattered throughout the \n item space, then use ScrambledZipfianGenerator instead.\n \n Be aware: initializing this generator may take a long time if there are lots of items to choose from (e.g. over a minute\n for 100 million objects). This is because certain mathematical values need to be computed to properly generate a zipfian skew, and one of those\n values (zeta) is a sum sequence from 1 to n, where n is the itemcount. Note that if you increase the number of items in the set, we can compute\n a new zeta incrementally, so it should be fast unless you have added millions of items. However, if you decrease the number of items, we recompute\n zeta from scratch, so this can take a long time.\n<P>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;double</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ZipfianGenerator.html#ZIPFIAN_CONSTANT\">ZIPFIAN_CONSTANT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ZipfianGenerator.html#ZipfianGenerator(long)\">ZipfianGenerator</A></B>(long&nbsp;_items)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a zipfian generator for the specified number of items.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ZipfianGenerator.html#ZipfianGenerator(long, double)\">ZipfianGenerator</A></B>(long&nbsp;_items,\n                 double&nbsp;_zipfianconstant)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a zipfian generator for the specified number of items using the specified zipfian constant.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ZipfianGenerator.html#ZipfianGenerator(long, long)\">ZipfianGenerator</A></B>(long&nbsp;_min,\n                 long&nbsp;_max)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a zipfian generator for items between min and max.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ZipfianGenerator.html#ZipfianGenerator(long, long, double)\">ZipfianGenerator</A></B>(long&nbsp;min,\n                 long&nbsp;max,\n                 double&nbsp;_zipfianconstant)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a zipfian generator for items between min and max (inclusive) for the specified zipfian constant.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ZipfianGenerator.html#ZipfianGenerator(long, long, double, double)\">ZipfianGenerator</A></B>(long&nbsp;min,\n                 long&nbsp;max,\n                 double&nbsp;_zipfianconstant,\n                 double&nbsp;_zetan)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Create a zipfian generator for items between min and max (inclusive) for the specified zipfian constant, using the precomputed value of zeta.</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ZipfianGenerator.html#main(java.lang.String[])\">main</A></B>(java.lang.String[]&nbsp;args)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ZipfianGenerator.html#nextInt()\">nextInt</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Return the next value, skewed by the Zipfian distribution.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ZipfianGenerator.html#nextInt(int)\">nextInt</A></B>(int&nbsp;itemcount)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Generate the next item.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;long</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ZipfianGenerator.html#nextLong()\">nextLong</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Return the next value, skewed by the Zipfian distribution.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;long</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../com/yahoo/ycsb/ZipfianGenerator.html#nextLong(long)\">nextLong</A></B>(long&nbsp;itemcount)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Generate the next item as a long.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_com.yahoo.ycsb.IntegerGenerator\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#lastInt()\">lastInt</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#lastString()\">lastString</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#nextString()\">nextString</A>, <A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#setLastInt(int)\">setLastInt</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"ZIPFIAN_CONSTANT\"><!-- --></A><H3>\nZIPFIAN_CONSTANT</H3>\n<PRE>\npublic static final double <B>ZIPFIAN_CONSTANT</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../constant-values.html#com.yahoo.ycsb.ZipfianGenerator.ZIPFIAN_CONSTANT\">Constant Field Values</A></DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"ZipfianGenerator(long)\"><!-- --></A><H3>\nZipfianGenerator</H3>\n<PRE>\npublic <B>ZipfianGenerator</B>(long&nbsp;_items)</PRE>\n<DL>\n<DD>Create a zipfian generator for the specified number of items.\n<P>\n<DL>\n<DT><B>Parameters:</B><DD><CODE>_items</CODE> - The number of items in the distribution.</DL>\n</DL>\n<HR>\n\n<A NAME=\"ZipfianGenerator(long, long)\"><!-- --></A><H3>\nZipfianGenerator</H3>\n<PRE>\npublic <B>ZipfianGenerator</B>(long&nbsp;_min,\n                        long&nbsp;_max)</PRE>\n<DL>\n<DD>Create a zipfian generator for items between min and max.\n<P>\n<DL>\n<DT><B>Parameters:</B><DD><CODE>_min</CODE> - The smallest integer to generate in the sequence.<DD><CODE>_max</CODE> - The largest integer to generate in the sequence.</DL>\n</DL>\n<HR>\n\n<A NAME=\"ZipfianGenerator(long, double)\"><!-- --></A><H3>\nZipfianGenerator</H3>\n<PRE>\npublic <B>ZipfianGenerator</B>(long&nbsp;_items,\n                        double&nbsp;_zipfianconstant)</PRE>\n<DL>\n<DD>Create a zipfian generator for the specified number of items using the specified zipfian constant.\n<P>\n<DL>\n<DT><B>Parameters:</B><DD><CODE>_items</CODE> - The number of items in the distribution.<DD><CODE>_zipfianconstant</CODE> - The zipfian constant to use.</DL>\n</DL>\n<HR>\n\n<A NAME=\"ZipfianGenerator(long, long, double)\"><!-- --></A><H3>\nZipfianGenerator</H3>\n<PRE>\npublic <B>ZipfianGenerator</B>(long&nbsp;min,\n                        long&nbsp;max,\n                        double&nbsp;_zipfianconstant)</PRE>\n<DL>\n<DD>Create a zipfian generator for items between min and max (inclusive) for the specified zipfian constant.\n<P>\n<DL>\n<DT><B>Parameters:</B><DD><CODE>min</CODE> - The smallest integer to generate in the sequence.<DD><CODE>max</CODE> - The largest integer to generate in the sequence.<DD><CODE>_zipfianconstant</CODE> - The zipfian constant to use.</DL>\n</DL>\n<HR>\n\n<A NAME=\"ZipfianGenerator(long, long, double, double)\"><!-- --></A><H3>\nZipfianGenerator</H3>\n<PRE>\npublic <B>ZipfianGenerator</B>(long&nbsp;min,\n                        long&nbsp;max,\n                        double&nbsp;_zipfianconstant,\n                        double&nbsp;_zetan)</PRE>\n<DL>\n<DD>Create a zipfian generator for items between min and max (inclusive) for the specified zipfian constant, using the precomputed value of zeta.\n<P>\n<DL>\n<DT><B>Parameters:</B><DD><CODE>min</CODE> - The smallest integer to generate in the sequence.<DD><CODE>max</CODE> - The largest integer to generate in the sequence.<DD><CODE>_zipfianconstant</CODE> - The zipfian constant to use.<DD><CODE>_zetan</CODE> - The precomputed zeta constant.</DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"nextInt(int)\"><!-- --></A><H3>\nnextInt</H3>\n<PRE>\npublic int <B>nextInt</B>(int&nbsp;itemcount)</PRE>\n<DL>\n<DD>Generate the next item. this distribution will be skewed toward lower integers; e.g. 0 will\n be the most popular, 1 the next most popular, etc.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>itemcount</CODE> - The number of items in the distribution.\n<DT><B>Returns:</B><DD>The next item in the sequence.</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"nextLong(long)\"><!-- --></A><H3>\nnextLong</H3>\n<PRE>\npublic long <B>nextLong</B>(long&nbsp;itemcount)</PRE>\n<DL>\n<DD>Generate the next item as a long.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>itemcount</CODE> - The number of items in the distribution.\n<DT><B>Returns:</B><DD>The next item in the sequence.</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"nextInt()\"><!-- --></A><H3>\nnextInt</H3>\n<PRE>\npublic int <B>nextInt</B>()</PRE>\n<DL>\n<DD>Return the next value, skewed by the Zipfian distribution. The 0th item will be the most popular, followed by the 1st, followed\n by the 2nd, etc. (Or, if min != 0, the min-th item is the most popular, the min+1th item the next most popular, etc.) If you want the\n popular items scattered throughout the item space, use ScrambledZipfianGenerator instead.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html#nextInt()\">nextInt</A></CODE> in class <CODE><A HREF=\"../../../com/yahoo/ycsb/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb\">IntegerGenerator</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"nextLong()\"><!-- --></A><H3>\nnextLong</H3>\n<PRE>\npublic long <B>nextLong</B>()</PRE>\n<DL>\n<DD>Return the next value, skewed by the Zipfian distribution. The 0th item will be the most popular, followed by the 1st, followed\n by the 2nd, etc. (Or, if min != 0, the min-th item is the most popular, the min+1th item the next most popular, etc.) If you want the\n popular items scattered throughout the item space, use ScrambledZipfianGenerator instead.\n<P>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"main(java.lang.String[])\"><!-- --></A><H3>\nmain</H3>\n<PRE>\npublic static void <B>main</B>(java.lang.String[]&nbsp;args)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;NEXT CLASS</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/ZipfianGenerator.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"ZipfianGenerator.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/db/CassandraClient.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.5.0_22) on Mon Apr 19 13:46:54 PDT 2010 -->\n<TITLE>\nCassandraClient\n</TITLE>\n\n<META NAME=\"keywords\" CONTENT=\"com.yahoo.ycsb.db.CassandraClient class\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    parent.document.title=\"CassandraClient\";\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV CLASS&nbsp;\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/db/CassandraClient.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"CassandraClient.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb.db</FONT>\n<BR>\nClass CassandraClient</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.DB</A>\n      <IMG SRC=\"../../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.db.CassandraClient</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>CassandraClient</B><DT>extends <A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></DL>\n</PRE>\n\n<P>\nXXXX if we do replication, fix the consistency levels\n<P>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#CONNECTION_RETRY_PROPERTY\">CONNECTION_RETRY_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#CONNECTION_RETRY_PROPERTY_DEFAULT\">CONNECTION_RETRY_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#ConnectionRetries\">ConnectionRetries</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#Error\">Error</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#Ok\">Ok</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#OPERATION_RETRY_PROPERTY\">OPERATION_RETRY_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#OPERATION_RETRY_PROPERTY_DEFAULT\">OPERATION_RETRY_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#OperationRetries\">OperationRetries</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#CassandraClient()\">CassandraClient</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#cleanup()\">cleanup</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cleanup any state for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#delete(java.lang.String, java.lang.String)\">delete</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delete a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#init()\">init</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Initialize any state for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">insert</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Insert a record in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#main(java.lang.String[])\">main</A></B>(java.lang.String[]&nbsp;args)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">read</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;key,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Read a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">scan</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;startkey,\n     int&nbsp;recordcount,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Perform a range scan for a set of records in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">update</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update a record in the database.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_com.yahoo.ycsb.DB\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class com.yahoo.ycsb.<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#getProperties()\">getProperties</A>, <A HREF=\"../../../../com/yahoo/ycsb/DB.html#setProperties(java.util.Properties)\">setProperties</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"Ok\"><!-- --></A><H3>\nOk</H3>\n<PRE>\npublic static final int <B>Ok</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.CassandraClient.Ok\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"Error\"><!-- --></A><H3>\nError</H3>\n<PRE>\npublic static final int <B>Error</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.CassandraClient.Error\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"ConnectionRetries\"><!-- --></A><H3>\nConnectionRetries</H3>\n<PRE>\npublic int <B>ConnectionRetries</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"OperationRetries\"><!-- --></A><H3>\nOperationRetries</H3>\n<PRE>\npublic int <B>OperationRetries</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"CONNECTION_RETRY_PROPERTY\"><!-- --></A><H3>\nCONNECTION_RETRY_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>CONNECTION_RETRY_PROPERTY</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.CassandraClient.CONNECTION_RETRY_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"CONNECTION_RETRY_PROPERTY_DEFAULT\"><!-- --></A><H3>\nCONNECTION_RETRY_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>CONNECTION_RETRY_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.CassandraClient.CONNECTION_RETRY_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"OPERATION_RETRY_PROPERTY\"><!-- --></A><H3>\nOPERATION_RETRY_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>OPERATION_RETRY_PROPERTY</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.CassandraClient.OPERATION_RETRY_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"OPERATION_RETRY_PROPERTY_DEFAULT\"><!-- --></A><H3>\nOPERATION_RETRY_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>OPERATION_RETRY_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.CassandraClient.OPERATION_RETRY_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"CassandraClient()\"><!-- --></A><H3>\nCassandraClient</H3>\n<PRE>\npublic <B>CassandraClient</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"init()\"><!-- --></A><H3>\ninit</H3>\n<PRE>\npublic void <B>init</B>()\n          throws <A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Initialize any state for this DB.\n Called once per DB instance; there is one DB instance per client thread.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#init()\">init</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"cleanup()\"><!-- --></A><H3>\ncleanup</H3>\n<PRE>\npublic void <B>cleanup</B>()\n             throws <A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Cleanup any state for this DB.\n Called once per DB instance; there is one DB instance per client thread.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#cleanup()\">cleanup</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><!-- --></A><H3>\nread</H3>\n<PRE>\npublic int <B>read</B>(java.lang.String&nbsp;table,\n                java.lang.String&nbsp;key,\n                java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</PRE>\n<DL>\n<DD>Read a record from the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">read</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to read.<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A HashMap of field/value pairs for the result\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><!-- --></A><H3>\nscan</H3>\n<PRE>\npublic int <B>scan</B>(java.lang.String&nbsp;table,\n                java.lang.String&nbsp;startkey,\n                int&nbsp;recordcount,\n                java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</PRE>\n<DL>\n<DD>Perform a range scan for a set of records in the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">scan</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>startkey</CODE> - The record key of the first record to read.<DD><CODE>recordcount</CODE> - The number of records to read<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A Vector of HashMaps, where each HashMap is a set field/value pairs for one record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"update(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\nupdate</H3>\n<PRE>\npublic int <B>update</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key,\n                  java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Update a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key, overwriting any existing values with the same field name.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">update</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to write.<DD><CODE>values</CODE> - A HashMap of field/value pairs to update in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"insert(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\ninsert</H3>\n<PRE>\npublic int <B>insert</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key,\n                  java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Insert a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">insert</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to insert.<DD><CODE>values</CODE> - A HashMap of field/value pairs to insert in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"delete(java.lang.String, java.lang.String)\"><!-- --></A><H3>\ndelete</H3>\n<PRE>\npublic int <B>delete</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key)</PRE>\n<DL>\n<DD>Delete a record from the database.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#delete(java.lang.String, java.lang.String)\">delete</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to delete.\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"main(java.lang.String[])\"><!-- --></A><H3>\nmain</H3>\n<PRE>\npublic static void <B>main</B>(java.lang.String[]&nbsp;args)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV CLASS&nbsp;\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/db/CassandraClient.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"CassandraClient.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/db/HBaseClient.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\nHBaseClient\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"HBaseClient\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/db/HBaseClient.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"HBaseClient.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb.db</FONT>\n<BR>\nClass HBaseClient</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.DB</A>\n      <IMG SRC=\"../../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.db.HBaseClient</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>HBaseClient</B><DT>extends <A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></DL>\n</PRE>\n\n<P>\nHBase client for YCSB framework\n<P>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#_columnFamily\">_columnFamily</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;byte[]</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#_columnFamilyBytes\">_columnFamilyBytes</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;boolean</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#_debug\">_debug</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;HTable</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#_hTable\">_hTable</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#_table\">_table</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#HttpError\">HttpError</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#NoMatchingRecord\">NoMatchingRecord</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#Ok\">Ok</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#ServerError\">ServerError</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.Object</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#tableLock\">tableLock</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#HBaseClient()\">HBaseClient</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#cleanup()\">cleanup</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cleanup any state for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#delete(java.lang.String, java.lang.String)\">delete</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delete a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#getHTable(java.lang.String)\">getHTable</A></B>(java.lang.String&nbsp;table)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#init()\">init</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Initialize any state for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">insert</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Insert a record in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#main(java.lang.String[])\">main</A></B>(java.lang.String[]&nbsp;args)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">read</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;key,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Read a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">scan</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;startkey,\n     int&nbsp;recordcount,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Perform a range scan for a set of records in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">update</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update a record in the database.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_com.yahoo.ycsb.DB\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class com.yahoo.ycsb.<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#getProperties()\">getProperties</A>, <A HREF=\"../../../../com/yahoo/ycsb/DB.html#setProperties(java.util.Properties)\">setProperties</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"_debug\"><!-- --></A><H3>\n_debug</H3>\n<PRE>\npublic boolean <B>_debug</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"_table\"><!-- --></A><H3>\n_table</H3>\n<PRE>\npublic java.lang.String <B>_table</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"_hTable\"><!-- --></A><H3>\n_hTable</H3>\n<PRE>\npublic HTable <B>_hTable</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"_columnFamily\"><!-- --></A><H3>\n_columnFamily</H3>\n<PRE>\npublic java.lang.String <B>_columnFamily</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"_columnFamilyBytes\"><!-- --></A><H3>\n_columnFamilyBytes</H3>\n<PRE>\npublic byte[] <B>_columnFamilyBytes</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"Ok\"><!-- --></A><H3>\nOk</H3>\n<PRE>\npublic static final int <B>Ok</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.HBaseClient.Ok\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"ServerError\"><!-- --></A><H3>\nServerError</H3>\n<PRE>\npublic static final int <B>ServerError</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.HBaseClient.ServerError\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"HttpError\"><!-- --></A><H3>\nHttpError</H3>\n<PRE>\npublic static final int <B>HttpError</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.HBaseClient.HttpError\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"NoMatchingRecord\"><!-- --></A><H3>\nNoMatchingRecord</H3>\n<PRE>\npublic static final int <B>NoMatchingRecord</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.HBaseClient.NoMatchingRecord\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"tableLock\"><!-- --></A><H3>\ntableLock</H3>\n<PRE>\npublic static java.lang.Object <B>tableLock</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"HBaseClient()\"><!-- --></A><H3>\nHBaseClient</H3>\n<PRE>\npublic <B>HBaseClient</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"init()\"><!-- --></A><H3>\ninit</H3>\n<PRE>\npublic void <B>init</B>()\n          throws <A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Initialize any state for this DB.\n Called once per DB instance; there is one DB instance per client thread.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#init()\">init</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"cleanup()\"><!-- --></A><H3>\ncleanup</H3>\n<PRE>\npublic void <B>cleanup</B>()\n             throws <A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Cleanup any state for this DB.\n Called once per DB instance; there is one DB instance per client thread.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#cleanup()\">cleanup</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"getHTable(java.lang.String)\"><!-- --></A><H3>\ngetHTable</H3>\n<PRE>\npublic void <B>getHTable</B>(java.lang.String&nbsp;table)\n               throws java.io.IOException</PRE>\n<DL>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE>java.io.IOException</CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><!-- --></A><H3>\nread</H3>\n<PRE>\npublic int <B>read</B>(java.lang.String&nbsp;table,\n                java.lang.String&nbsp;key,\n                java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</PRE>\n<DL>\n<DD>Read a record from the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">read</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to read.<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A HashMap of field/value pairs for the result\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><!-- --></A><H3>\nscan</H3>\n<PRE>\npublic int <B>scan</B>(java.lang.String&nbsp;table,\n                java.lang.String&nbsp;startkey,\n                int&nbsp;recordcount,\n                java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</PRE>\n<DL>\n<DD>Perform a range scan for a set of records in the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">scan</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>startkey</CODE> - The record key of the first record to read.<DD><CODE>recordcount</CODE> - The number of records to read<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A Vector of HashMaps, where each HashMap is a set field/value pairs for one record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"update(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\nupdate</H3>\n<PRE>\npublic int <B>update</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key,\n                  java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Update a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key, overwriting any existing values with the same field name.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">update</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to write<DD><CODE>values</CODE> - A HashMap of field/value pairs to update in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"insert(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\ninsert</H3>\n<PRE>\npublic int <B>insert</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key,\n                  java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Insert a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">insert</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to insert.<DD><CODE>values</CODE> - A HashMap of field/value pairs to insert in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"delete(java.lang.String, java.lang.String)\"><!-- --></A><H3>\ndelete</H3>\n<PRE>\npublic int <B>delete</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key)</PRE>\n<DL>\n<DD>Delete a record from the database.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#delete(java.lang.String, java.lang.String)\">delete</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to delete.\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"main(java.lang.String[])\"><!-- --></A><H3>\nmain</H3>\n<PRE>\npublic static void <B>main</B>(java.lang.String[]&nbsp;args)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/db/HBaseClient.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"HBaseClient.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/db/ShardClient.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Fri Jan 22 13:42:46 PST 2010 -->\n<TITLE>\nShardClient\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-01-22\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"ShardClient\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/db/ShardClient.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"ShardClient.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb.db</FONT>\n<BR>\nClass ShardClient</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.DB</A>\n      <IMG SRC=\"../../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.db.ShardClient</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>ShardClient</B><DT>extends <A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></DL>\n</PRE>\n\n<P>\nA client for the simple sharded MySQL called shardserver.\n<P>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;boolean</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#_debug\">_debug</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;boolean</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#_donothing\">_donothing</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#HttpError\">HttpError</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#NoMatchingRecord\">NoMatchingRecord</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#NoMatchingRecordString\">NoMatchingRecordString</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#Ok\">Ok</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#ServerError\">ServerError</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;long</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#TotalGetOps\">TotalGetOps</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;long</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#TotalGetTime\">TotalGetTime</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;long</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#TotalPostOps\">TotalPostOps</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;long</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#TotalPostTime\">TotalPostTime</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#UrlPrefix\">UrlPrefix</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#ShardClient()\">ShardClient</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#cleanup()\">cleanup</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cleanup any state for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#delete(java.lang.String, java.lang.String)\">delete</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delete a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.util.Vector&lt;char[]&gt;</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#doGet(java.lang.String)\">doGet</A></B>(java.lang.String&nbsp;url)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Do a GET HTTP call.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#doPost(java.lang.String, java.lang.String)\">doPost</A></B>(java.lang.String&nbsp;url,\n       java.lang.String&nbsp;postbody)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Do a POST HTTP call.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#init()\">init</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Initialize any state for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">insert</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Insert a record in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#main(java.lang.String[])\">main</A></B>(java.lang.String[]&nbsp;args)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Simple test.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">read</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;key,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Read a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">scan</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;startkey,\n     int&nbsp;recordcount,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Perform a range scan for a set of records in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">update</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update a record in the database.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_com.yahoo.ycsb.DB\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class com.yahoo.ycsb.<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#getProperties()\">getProperties</A>, <A HREF=\"../../../../com/yahoo/ycsb/DB.html#setProperties(java.util.Properties)\">setProperties</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"UrlPrefix\"><!-- --></A><H3>\nUrlPrefix</H3>\n<PRE>\npublic static final java.lang.String <B>UrlPrefix</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.ShardClient.UrlPrefix\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"NoMatchingRecordString\"><!-- --></A><H3>\nNoMatchingRecordString</H3>\n<PRE>\npublic static final java.lang.String <B>NoMatchingRecordString</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.ShardClient.NoMatchingRecordString\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"Ok\"><!-- --></A><H3>\nOk</H3>\n<PRE>\npublic static final int <B>Ok</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.ShardClient.Ok\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"ServerError\"><!-- --></A><H3>\nServerError</H3>\n<PRE>\npublic static final int <B>ServerError</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.ShardClient.ServerError\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"HttpError\"><!-- --></A><H3>\nHttpError</H3>\n<PRE>\npublic static final int <B>HttpError</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.ShardClient.HttpError\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"NoMatchingRecord\"><!-- --></A><H3>\nNoMatchingRecord</H3>\n<PRE>\npublic static final int <B>NoMatchingRecord</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.ShardClient.NoMatchingRecord\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"_debug\"><!-- --></A><H3>\n_debug</H3>\n<PRE>\npublic boolean <B>_debug</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"_donothing\"><!-- --></A><H3>\n_donothing</H3>\n<PRE>\npublic boolean <B>_donothing</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"TotalGetTime\"><!-- --></A><H3>\nTotalGetTime</H3>\n<PRE>\npublic long <B>TotalGetTime</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"TotalGetOps\"><!-- --></A><H3>\nTotalGetOps</H3>\n<PRE>\npublic long <B>TotalGetOps</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"TotalPostTime\"><!-- --></A><H3>\nTotalPostTime</H3>\n<PRE>\npublic long <B>TotalPostTime</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n<HR>\n\n<A NAME=\"TotalPostOps\"><!-- --></A><H3>\nTotalPostOps</H3>\n<PRE>\npublic long <B>TotalPostOps</B></PRE>\n<DL>\n<DL>\n</DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"ShardClient()\"><!-- --></A><H3>\nShardClient</H3>\n<PRE>\npublic <B>ShardClient</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"init()\"><!-- --></A><H3>\ninit</H3>\n<PRE>\npublic void <B>init</B>()\n          throws <A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Initialize any state for this DB.\n Called once per DB instance; there is one DB instance per client thread.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#init()\">init</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"cleanup()\"><!-- --></A><H3>\ncleanup</H3>\n<PRE>\npublic void <B>cleanup</B>()\n             throws <A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Cleanup any state for this DB.\n Called once per DB instance; there is one DB instance per client thread.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#cleanup()\">cleanup</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"doGet(java.lang.String)\"><!-- --></A><H3>\ndoGet</H3>\n<PRE>\npublic java.util.Vector&lt;char[]&gt; <B>doGet</B>(java.lang.String&nbsp;url)\n                               throws <A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Do a GET HTTP call. Returns the results in a Vector of char[] blocks.\n<P>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"doPost(java.lang.String, java.lang.String)\"><!-- --></A><H3>\ndoPost</H3>\n<PRE>\npublic java.lang.String <B>doPost</B>(java.lang.String&nbsp;url,\n                               java.lang.String&nbsp;postbody)\n                        throws <A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Do a POST HTTP call. Returns the result as a String.\n<P>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><!-- --></A><H3>\nread</H3>\n<PRE>\npublic int <B>read</B>(java.lang.String&nbsp;table,\n                java.lang.String&nbsp;key,\n                java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</PRE>\n<DL>\n<DD>Read a record from the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">read</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to read.<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A HashMap of field/value pairs for the result\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><!-- --></A><H3>\nscan</H3>\n<PRE>\npublic int <B>scan</B>(java.lang.String&nbsp;table,\n                java.lang.String&nbsp;startkey,\n                int&nbsp;recordcount,\n                java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</PRE>\n<DL>\n<DD>Perform a range scan for a set of records in the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">scan</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>startkey</CODE> - The record key of the first record to read.<DD><CODE>recordcount</CODE> - The number of records to read<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A Vector of HashMaps, where each HashMap is a set field/value pairs for one record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"update(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\nupdate</H3>\n<PRE>\npublic int <B>update</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key,\n                  java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Update a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key, overwriting any existing values with the same field name.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">update</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to write.<DD><CODE>values</CODE> - A HashMap of field/value pairs to update in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"insert(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\ninsert</H3>\n<PRE>\npublic int <B>insert</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key,\n                  java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Insert a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">insert</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to insert.<DD><CODE>values</CODE> - A HashMap of field/value pairs to insert in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"delete(java.lang.String, java.lang.String)\"><!-- --></A><H3>\ndelete</H3>\n<PRE>\npublic int <B>delete</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key)</PRE>\n<DL>\n<DD>Delete a record from the database.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#delete(java.lang.String, java.lang.String)\">delete</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to delete.\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"main(java.lang.String[])\"><!-- --></A><H3>\nmain</H3>\n<PRE>\npublic static void <B>main</B>(java.lang.String[]&nbsp;args)</PRE>\n<DL>\n<DD>Simple test.\n<P>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>NEXT CLASS</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/db/ShardClient.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"ShardClient.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/db/SherpaClient.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Fri Jan 22 13:42:46 PST 2010 -->\n<TITLE>\nSherpaClient\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-01-22\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"SherpaClient\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;NEXT CLASS</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/db/SherpaClient.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"SherpaClient.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb.db</FONT>\n<BR>\nClass SherpaClient</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.DB</A>\n      <IMG SRC=\"../../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.db.SherpaClient</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>SherpaClient</B><DT>extends <A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></DL>\n</PRE>\n\n<P>\nThis is a Java Sherpa client to be used for the Benchmark app. Since Sherpa\n is a RESful service, this class will make use of Apache's HttpClient 4.x to\n make the web service calls.\n<P>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#ADDITIONAL_FIELD\">ADDITIONAL_FIELD</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#CODE\">CODE</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#CONTINUATION\">CONTINUATION</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#DELETE\">DELETE</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#END_HASH_KEY\">END_HASH_KEY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#END_HASH_KEY_VALUE\">END_HASH_KEY_VALUE</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#FIELDS\">FIELDS</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#FIRST_FIELD\">FIRST_FIELD</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#GET\">GET</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#HASH_SCAN\">HASH_SCAN</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#HOSTS\">HOSTS</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#HTTP\">HTTP</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#HTTP_ERROR\">HTTP_ERROR</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#JSON_ERROR\">JSON_ERROR</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#NAMESPACE\">NAMESPACE</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#OK\">OK</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#ORDERED_SCAN\">ORDERED_SCAN</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#PREDICATE\">PREDICATE</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#RECORD_LIMIT\">RECORD_LIMIT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#RECORDS\">RECORDS</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#SCAN_COMPLETED\">SCAN_COMPLETED</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#SCANTYPE\">SCANTYPE</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#SET\">SET</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#START_HASH_KEY\">START_HASH_KEY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#START_HASH_KEY_HEADER\">START_HASH_KEY_HEADER</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#START_HASH_KEY_VALUE\">START_HASH_KEY_VALUE</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#START_KEY_ADDITIONAL\">START_KEY_ADDITIONAL</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#START_KEY_FIRST\">START_KEY_FIRST</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#STATUS\">STATUS</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#URI_PATH\">URI_PATH</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#UTF8\">UTF8</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#VALUE\">VALUE</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>protected static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#YDHT\">YDHT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#SherpaClient()\">SherpaClient</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#cleanup()\">cleanup</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cleanup any state for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#createRecordBody(java.util.HashMap)\">createRecordBody</A></B>(java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Method to create the Sherpa JSON record body from an input HashMap of\n key/values.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#delete(java.lang.String, java.lang.String)\">delete</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delete a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#encodeFields(java.util.Set)\">encodeFields</A></B>(java.util.Set&lt;java.lang.String&gt;&nbsp;fields)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Method to encode a set of fields to retrieve from a Sherpa Get or Scan\n request.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#encodeString(java.lang.String)\">encodeString</A></B>(java.lang.String&nbsp;key)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Method to encode a String to UTF8.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#getHashKey(java.lang.String)\">getHashKey</A></B>(java.lang.String&nbsp;key)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Method to return the hex string representation of a 32-bit unsigned key\n value used for doing Sherpa scans.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#getResponseCode(JSONObject)\">getResponseCode</A></B>(JSONObject&nbsp;jsonObject)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Helper method to retrieve the response status code from a JSON Response\n to determine if it was successful or not.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#init()\">init</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Initialize any state for this DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">insert</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Insert a record in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#populateHashScanRecords(JSONObject, java.util.Vector)\">populateHashScanRecords</A></B>(JSONObject&nbsp;curResult,\n                        java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Method to parse and store a JSON response from a hash scan request.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#populateOrderedScanRecords(JSONObject, java.util.Vector)\">populateOrderedScanRecords</A></B>(JSONObject&nbsp;curResult,\n                           java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Method to parse and store a JSON response from an ordered scan request.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#populateReadRecord(JSONObject, java.util.HashMap)\">populateReadRecord</A></B>(JSONObject&nbsp;response,\n                   java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Method to parse and store a JSON response from reading a single Sherpa\n record into the corresponding HashMap key/value pairs.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">read</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;key,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Read a record from the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">scan</A></B>(java.lang.String&nbsp;table,\n     java.lang.String&nbsp;startkey,\n     int&nbsp;recordcount,\n     java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n     java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Perform a range scan for a set of records in the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;int</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/db/SherpaClient.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">update</A></B>(java.lang.String&nbsp;table,\n       java.lang.String&nbsp;key,\n       java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update a record in the database.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_com.yahoo.ycsb.DB\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class com.yahoo.ycsb.<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#getProperties()\">getProperties</A>, <A HREF=\"../../../../com/yahoo/ycsb/DB.html#setProperties(java.util.Properties)\">setProperties</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"GET\"><!-- --></A><H3>\nGET</H3>\n<PRE>\nprotected static final java.lang.String <B>GET</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.GET\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"SET\"><!-- --></A><H3>\nSET</H3>\n<PRE>\nprotected static final java.lang.String <B>SET</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.SET\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"DELETE\"><!-- --></A><H3>\nDELETE</H3>\n<PRE>\nprotected static final java.lang.String <B>DELETE</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.DELETE\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"HASH_SCAN\"><!-- --></A><H3>\nHASH_SCAN</H3>\n<PRE>\nprotected static final java.lang.String <B>HASH_SCAN</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.HASH_SCAN\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"ORDERED_SCAN\"><!-- --></A><H3>\nORDERED_SCAN</H3>\n<PRE>\nprotected static final java.lang.String <B>ORDERED_SCAN</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.ORDERED_SCAN\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"FIRST_FIELD\"><!-- --></A><H3>\nFIRST_FIELD</H3>\n<PRE>\nprotected static final java.lang.String <B>FIRST_FIELD</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.FIRST_FIELD\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"ADDITIONAL_FIELD\"><!-- --></A><H3>\nADDITIONAL_FIELD</H3>\n<PRE>\nprotected static final java.lang.String <B>ADDITIONAL_FIELD</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.ADDITIONAL_FIELD\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"START_KEY_FIRST\"><!-- --></A><H3>\nSTART_KEY_FIRST</H3>\n<PRE>\nprotected static final java.lang.String <B>START_KEY_FIRST</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.START_KEY_FIRST\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"START_KEY_ADDITIONAL\"><!-- --></A><H3>\nSTART_KEY_ADDITIONAL</H3>\n<PRE>\nprotected static final java.lang.String <B>START_KEY_ADDITIONAL</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.START_KEY_ADDITIONAL\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"UTF8\"><!-- --></A><H3>\nUTF8</H3>\n<PRE>\nprotected static final java.lang.String <B>UTF8</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.UTF8\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"START_HASH_KEY_HEADER\"><!-- --></A><H3>\nSTART_HASH_KEY_HEADER</H3>\n<PRE>\nprotected static final java.lang.String <B>START_HASH_KEY_HEADER</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.START_HASH_KEY_HEADER\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"CONTINUATION\"><!-- --></A><H3>\nCONTINUATION</H3>\n<PRE>\nprotected static final java.lang.String <B>CONTINUATION</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.CONTINUATION\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"START_HASH_KEY\"><!-- --></A><H3>\nSTART_HASH_KEY</H3>\n<PRE>\nprotected static final java.lang.String <B>START_HASH_KEY</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.START_HASH_KEY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"END_HASH_KEY\"><!-- --></A><H3>\nEND_HASH_KEY</H3>\n<PRE>\nprotected static final java.lang.String <B>END_HASH_KEY</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.END_HASH_KEY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"START_HASH_KEY_VALUE\"><!-- --></A><H3>\nSTART_HASH_KEY_VALUE</H3>\n<PRE>\nprotected static final java.lang.String <B>START_HASH_KEY_VALUE</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.START_HASH_KEY_VALUE\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"END_HASH_KEY_VALUE\"><!-- --></A><H3>\nEND_HASH_KEY_VALUE</H3>\n<PRE>\nprotected static final java.lang.String <B>END_HASH_KEY_VALUE</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.END_HASH_KEY_VALUE\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"RECORD_LIMIT\"><!-- --></A><H3>\nRECORD_LIMIT</H3>\n<PRE>\nprotected static final java.lang.String <B>RECORD_LIMIT</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.RECORD_LIMIT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"PREDICATE\"><!-- --></A><H3>\nPREDICATE</H3>\n<PRE>\nprotected static final java.lang.String <B>PREDICATE</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.PREDICATE\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"SCAN_COMPLETED\"><!-- --></A><H3>\nSCAN_COMPLETED</H3>\n<PRE>\nprotected static final java.lang.String <B>SCAN_COMPLETED</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.SCAN_COMPLETED\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"RECORDS\"><!-- --></A><H3>\nRECORDS</H3>\n<PRE>\nprotected static final java.lang.String <B>RECORDS</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.RECORDS\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"URI_PATH\"><!-- --></A><H3>\nURI_PATH</H3>\n<PRE>\nprotected static final java.lang.String <B>URI_PATH</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.URI_PATH\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"YDHT\"><!-- --></A><H3>\nYDHT</H3>\n<PRE>\nprotected static final java.lang.String <B>YDHT</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.YDHT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"STATUS\"><!-- --></A><H3>\nSTATUS</H3>\n<PRE>\nprotected static final java.lang.String <B>STATUS</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.STATUS\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"CODE\"><!-- --></A><H3>\nCODE</H3>\n<PRE>\nprotected static final java.lang.String <B>CODE</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.CODE\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"FIELDS\"><!-- --></A><H3>\nFIELDS</H3>\n<PRE>\nprotected static final java.lang.String <B>FIELDS</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.FIELDS\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"VALUE\"><!-- --></A><H3>\nVALUE</H3>\n<PRE>\nprotected static final java.lang.String <B>VALUE</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.VALUE\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"OK\"><!-- --></A><H3>\nOK</H3>\n<PRE>\nprotected static final int <B>OK</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.OK\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"JSON_ERROR\"><!-- --></A><H3>\nJSON_ERROR</H3>\n<PRE>\nprotected static final int <B>JSON_ERROR</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.JSON_ERROR\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"HTTP_ERROR\"><!-- --></A><H3>\nHTTP_ERROR</H3>\n<PRE>\nprotected static final int <B>HTTP_ERROR</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.HTTP_ERROR\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"HOSTS\"><!-- --></A><H3>\nHOSTS</H3>\n<PRE>\nprotected static final java.lang.String <B>HOSTS</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.HOSTS\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"HTTP\"><!-- --></A><H3>\nHTTP</H3>\n<PRE>\nprotected static final java.lang.String <B>HTTP</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.HTTP\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"NAMESPACE\"><!-- --></A><H3>\nNAMESPACE</H3>\n<PRE>\nprotected static final java.lang.String <B>NAMESPACE</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.NAMESPACE\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"SCANTYPE\"><!-- --></A><H3>\nSCANTYPE</H3>\n<PRE>\nprotected static final java.lang.String <B>SCANTYPE</B></PRE>\n<DL>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.db.SherpaClient.SCANTYPE\">Constant Field Values</A></DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"SherpaClient()\"><!-- --></A><H3>\nSherpaClient</H3>\n<PRE>\npublic <B>SherpaClient</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"init()\"><!-- --></A><H3>\ninit</H3>\n<PRE>\npublic void <B>init</B>()\n          throws <A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Initialize any state for this DB. Called once per DB instance; there is\n one DB instance per client thread.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#init()\">init</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"cleanup()\"><!-- --></A><H3>\ncleanup</H3>\n<PRE>\npublic void <B>cleanup</B>()\n             throws <A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></PRE>\n<DL>\n<DD>Cleanup any state for this DB. Called once per DB instance; there is one\n DB instance per client thread.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#cleanup()\">cleanup</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"encodeString(java.lang.String)\"><!-- --></A><H3>\nencodeString</H3>\n<PRE>\npublic static java.lang.String <B>encodeString</B>(java.lang.String&nbsp;key)</PRE>\n<DL>\n<DD>Method to encode a String to UTF8. All Sherpa table/record/field and\n record body data must be UTF8 since access is done via HTTP REST calls.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>key</CODE> - Key string we want to encode to UTF8\n<DT><B>Returns:</B><DD>UTF8 encoding of the input string</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"encodeFields(java.util.Set)\"><!-- --></A><H3>\nencodeFields</H3>\n<PRE>\npublic static java.lang.String <B>encodeFields</B>(java.util.Set&lt;java.lang.String&gt;&nbsp;fields)</PRE>\n<DL>\n<DD>Method to encode a set of fields to retrieve from a Sherpa Get or Scan\n request.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>fields</CODE> - Set of String fields to retrieve from a Sherpa Get or Scan\n            request\n<DT><B>Returns:</B><DD>String encoding of the set of fields to retrieve from a Sherpa\n         request. This has the format of ?field=<field1>&field=<field2>...</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"getHashKey(java.lang.String)\"><!-- --></A><H3>\ngetHashKey</H3>\n<PRE>\npublic static java.lang.String <B>getHashKey</B>(java.lang.String&nbsp;key)</PRE>\n<DL>\n<DD>Method to return the hex string representation of a 32-bit unsigned key\n value used for doing Sherpa scans. This needs to match how Sherpa in the\n backend is hashing the keys to determine which tablet it is stored in.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>key</CODE> - Key string to be hashed\n<DT><B>Returns:</B><DD>String hex representation of the hashing of the input key</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"getResponseCode(JSONObject)\"><!-- --></A><H3>\ngetResponseCode</H3>\n<PRE>\npublic static int <B>getResponseCode</B>(JSONObject&nbsp;jsonObject)</PRE>\n<DL>\n<DD>Helper method to retrieve the response status code from a JSON Response\n to determine if it was successful or not.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>jsonObject</CODE> - JSONObject response from Sherpa\n<DT><B>Returns:</B><DD>int Response status code indicating if the request was successful\n         or not</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"createRecordBody(java.util.HashMap)\"><!-- --></A><H3>\ncreateRecordBody</H3>\n<PRE>\npublic static java.lang.String <B>createRecordBody</B>(java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD>Method to create the Sherpa JSON record body from an input HashMap of\n key/values.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>values</CODE> - HashMap of key/values to be stored in a Sherpa record\n<DT><B>Returns:</B><DD>Sherpa JSON record representation of the input key/values HashMap</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"populateReadRecord(JSONObject, java.util.HashMap)\"><!-- --></A><H3>\npopulateReadRecord</H3>\n<PRE>\npublic static void <B>populateReadRecord</B>(JSONObject&nbsp;response,\n                                      java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)\n                               throws JSONException</PRE>\n<DL>\n<DD>Method to parse and store a JSON response from reading a single Sherpa\n record into the corresponding HashMap key/value pairs.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>response</CODE> - JSONObject response from reading a single Sherpa record<DD><CODE>result</CODE> - HashMap key/value pairs for the record's field/values\n<DT><B>Throws:</B>\n<DD><CODE>JSONException</CODE> - If there is an error parsing the JSON response</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"populateHashScanRecords(JSONObject, java.util.Vector)\"><!-- --></A><H3>\npopulateHashScanRecords</H3>\n<PRE>\npublic static void <B>populateHashScanRecords</B>(JSONObject&nbsp;curResult,\n                                           java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)\n                                    throws JSONException</PRE>\n<DL>\n<DD>Method to parse and store a JSON response from a hash scan request. This\n consists of multiple records each of which will be stored in the input\n result Vector as a HashMap of key/value pairs.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>curResult</CODE> - JSONObject response from a Sherpa hash scan request<DD><CODE>result</CODE> - Vector of Sherpa records each stored as a HashMap of\n            key/values for the record's field/values\n<DT><B>Throws:</B>\n<DD><CODE>JSONException</CODE> - If there is an error parsing the JSON response</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"populateOrderedScanRecords(JSONObject, java.util.Vector)\"><!-- --></A><H3>\npopulateOrderedScanRecords</H3>\n<PRE>\npublic static void <B>populateOrderedScanRecords</B>(JSONObject&nbsp;curResult,\n                                              java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)\n                                       throws JSONException</PRE>\n<DL>\n<DD>Method to parse and store a JSON response from an ordered scan request.\n This consists of multiple records each of which will be stored in the\n input result Vector as a HashMap of key/value pairs.\n<P>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>curResult</CODE> - JSONObject response from a Sherpa ordered scan request<DD><CODE>result</CODE> - Vector of Sherpa records each stored as a HashMap of\n            key/values for the record's field/values\n<DT><B>Throws:</B>\n<DD><CODE>JSONException</CODE> - If there is an error parsing the JSON response</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"delete(java.lang.String, java.lang.String)\"><!-- --></A><H3>\ndelete</H3>\n<PRE>\npublic int <B>delete</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key)</PRE>\n<DL>\n<DD><B>Description copied from class: <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#delete(java.lang.String, java.lang.String)\">DB</A></CODE></B></DD>\n<DD>Delete a record from the database.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#delete(java.lang.String, java.lang.String)\">delete</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to delete.\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"insert(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\ninsert</H3>\n<PRE>\npublic int <B>insert</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key,\n                  java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD><B>Description copied from class: <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">DB</A></CODE></B></DD>\n<DD>Insert a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\">insert</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to insert.<DD><CODE>values</CODE> - A HashMap of field/value pairs to insert in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><!-- --></A><H3>\nread</H3>\n<PRE>\npublic int <B>read</B>(java.lang.String&nbsp;table,\n                java.lang.String&nbsp;key,\n                java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;result)</PRE>\n<DL>\n<DD><B>Description copied from class: <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">DB</A></CODE></B></DD>\n<DD>Read a record from the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\">read</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to read.<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A HashMap of field/value pairs for the result\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><!-- --></A><H3>\nscan</H3>\n<PRE>\npublic int <B>scan</B>(java.lang.String&nbsp;table,\n                java.lang.String&nbsp;startkey,\n                int&nbsp;recordcount,\n                java.util.Set&lt;java.lang.String&gt;&nbsp;fields,\n                java.util.Vector&lt;java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&gt;&nbsp;result)</PRE>\n<DL>\n<DD><B>Description copied from class: <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">DB</A></CODE></B></DD>\n<DD>Perform a range scan for a set of records in the database. Each field/value pair from the result will be stored in a HashMap.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\">scan</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>startkey</CODE> - The record key of the first record to read.<DD><CODE>recordcount</CODE> - The number of records to read<DD><CODE>fields</CODE> - The list of fields to read, or null for all of them<DD><CODE>result</CODE> - A Vector of HashMaps, where each HashMap is a set field/value pairs for one record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"update(java.lang.String, java.lang.String, java.util.HashMap)\"><!-- --></A><H3>\nupdate</H3>\n<PRE>\npublic int <B>update</B>(java.lang.String&nbsp;table,\n                  java.lang.String&nbsp;key,\n                  java.util.HashMap&lt;java.lang.String,java.lang.String&gt;&nbsp;values)</PRE>\n<DL>\n<DD><B>Description copied from class: <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">DB</A></CODE></B></DD>\n<DD>Update a record in the database. Any field/value pairs in the specified values HashMap will be written into the record with the specified\n record key, overwriting any existing values with the same field name.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html#update(java.lang.String, java.lang.String, java.util.HashMap)\">update</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></CODE></DL>\n</DD>\n<DD><DL>\n<DT><B>Parameters:</B><DD><CODE>table</CODE> - The name of the table<DD><CODE>key</CODE> - The record key of the record to write.<DD><CODE>values</CODE> - A HashMap of field/value pairs to update in the record\n<DT><B>Returns:</B><DD>Zero on success, a non-zero error code on error</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/db/ShardClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>PREV CLASS</B></A>&nbsp;\n&nbsp;NEXT CLASS</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/db/SherpaClient.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"SherpaClient.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/db/package-frame.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\ncom.yahoo.ycsb.db\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../../stylesheet.css\" TITLE=\"Style\">\n\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\">\n<FONT size=\"+1\" CLASS=\"FrameTitleFont\">\n<A HREF=\"../../../../com/yahoo/ycsb/db/package-summary.html\" target=\"classFrame\">com.yahoo.ycsb.db</A></FONT>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" SUMMARY=\"\">\n<TR>\n<TD NOWRAP><FONT size=\"+1\" CLASS=\"FrameHeadingFont\">\nClasses</FONT>&nbsp;\n<FONT CLASS=\"FrameItemFont\">\n<BR>\n<A HREF=\"CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\" target=\"classFrame\">CassandraClient5</A>\n<BR>\n<A HREF=\"CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\" target=\"classFrame\">CassandraClient6</A>\n<BR>\n<A HREF=\"CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\" target=\"classFrame\">CassandraClient7</A>\n<BR>\n<A HREF=\"HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\" target=\"classFrame\">HBaseClient</A>\n<BR>\n<A HREF=\"MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\" target=\"classFrame\">MongoDbClient</A></FONT></TD>\n</TR>\n</TABLE>\n\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/db/package-summary.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\ncom.yahoo.ycsb.db\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"com.yahoo.ycsb.db\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Package</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/package-summary.html\"><B>PREV PACKAGE</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/generator/package-summary.html\"><B>NEXT PACKAGE</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/db/package-summary.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"package-summary.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<H2>\nPackage com.yahoo.ycsb.db\n</H2>\n\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Class Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A></B></TD>\n<TD>Cassandra 0.5 client for YCSB framework</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A></B></TD>\n<TD>Cassandra 0.6 client for YCSB framework</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A></B></TD>\n<TD>Cassandra 0.7 client for YCSB framework</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A></B></TD>\n<TD>HBase client for YCSB framework</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../../com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\">MongoDbClient</A></B></TD>\n<TD>MongoDB client for YCSB framework.</TD>\n</TR>\n</TABLE>\n&nbsp;\n\n<P>\n<DL>\n</DL>\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Package</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/package-summary.html\"><B>PREV PACKAGE</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/generator/package-summary.html\"><B>NEXT PACKAGE</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/db/package-summary.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"package-summary.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/db/package-tree.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\ncom.yahoo.ycsb.db Class Hierarchy\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"com.yahoo.ycsb.db Class Hierarchy\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Tree</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/package-tree.html\"><B>PREV</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/generator/package-tree.html\"><B>NEXT</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/db/package-tree.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"package-tree.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<CENTER>\n<H2>\nHierarchy For Package com.yahoo.ycsb.db\n</H2>\n</CENTER>\n<DL>\n<DT><B>Package Hierarchies:</B><DD><A HREF=\"../../../../overview-tree.html\">All Packages</A></DL>\n<HR>\n<H2>\nClass Hierarchy\n</H2>\n<UL>\n<LI TYPE=\"circle\">java.lang.Object<UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\"><B>DB</B></A><UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.db.<A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\"><B>CassandraClient5</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.db.<A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\"><B>CassandraClient6</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.db.<A HREF=\"../../../../com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\"><B>CassandraClient7</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.db.<A HREF=\"../../../../com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>HBaseClient</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.db.<A HREF=\"../../../../com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>MongoDbClient</B></A></UL>\n</UL>\n</UL>\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Tree</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/package-tree.html\"><B>PREV</B></A>&nbsp;\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/generator/package-tree.html\"><B>NEXT</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/db/package-tree.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"package-tree.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/package-frame.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\ncom.yahoo.ycsb\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\">\n<FONT size=\"+1\" CLASS=\"FrameTitleFont\">\n<A HREF=\"../../../com/yahoo/ycsb/package-summary.html\" target=\"classFrame\">com.yahoo.ycsb</A></FONT>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" SUMMARY=\"\">\n<TR>\n<TD NOWRAP><FONT size=\"+1\" CLASS=\"FrameHeadingFont\">\nClasses</FONT>&nbsp;\n<FONT CLASS=\"FrameItemFont\">\n<BR>\n<A HREF=\"BasicDB.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">BasicDB</A>\n<BR>\n<A HREF=\"Client.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">Client</A>\n<BR>\n<A HREF=\"CommandLine.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">CommandLine</A>\n<BR>\n<A HREF=\"DB.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">DB</A>\n<BR>\n<A HREF=\"DBFactory.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">DBFactory</A>\n<BR>\n<A HREF=\"DBWrapper.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">DBWrapper</A>\n<BR>\n<A HREF=\"Utils.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">Utils</A>\n<BR>\n<A HREF=\"Workload.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">Workload</A></FONT></TD>\n</TR>\n</TABLE>\n\n\n<TABLE BORDER=\"0\" WIDTH=\"100%\" SUMMARY=\"\">\n<TR>\n<TD NOWRAP><FONT size=\"+1\" CLASS=\"FrameHeadingFont\">\nExceptions</FONT>&nbsp;\n<FONT CLASS=\"FrameItemFont\">\n<BR>\n<A HREF=\"DBException.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">DBException</A>\n<BR>\n<A HREF=\"UnknownDBException.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">UnknownDBException</A>\n<BR>\n<A HREF=\"WorkloadException.html\" title=\"class in com.yahoo.ycsb\" target=\"classFrame\">WorkloadException</A></FONT></TD>\n</TR>\n</TABLE>\n\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/package-summary.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\ncom.yahoo.ycsb\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"com.yahoo.ycsb\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Package</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV PACKAGE&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/db/package-summary.html\"><B>NEXT PACKAGE</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/package-summary.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"package-summary.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<H2>\nPackage com.yahoo.ycsb\n</H2>\n\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Class Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A></B></TD>\n<TD>Basic DB that just prints out the requested operations, instead of doing them against a database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\">Client</A></B></TD>\n<TD>Main class for executing YCSB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\">CommandLine</A></B></TD>\n<TD>A simple command line client to a database, using the appropriate com.yahoo.ycsb.DB implementation.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A></B></TD>\n<TD>A layer for accessing a database to be benchmarked.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../com/yahoo/ycsb/DBFactory.html\" title=\"class in com.yahoo.ycsb\">DBFactory</A></B></TD>\n<TD>Creates a DB layer by dynamically classloading the specified DB class.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\">DBWrapper</A></B></TD>\n<TD>Wrapper around a \"real\" DB that measures latencies and counts return codes.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\">Utils</A></B></TD>\n<TD>Utility functions.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A></B></TD>\n<TD>One experiment scenario.</TD>\n</TR>\n</TABLE>\n&nbsp;\n\n<P>\n\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Exception Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A></B></TD>\n<TD>Something bad happened while interacting with the database.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\">UnknownDBException</A></B></TD>\n<TD>Could not create the specified DB.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A></B></TD>\n<TD>The workload tried to do something bad.</TD>\n</TR>\n</TABLE>\n&nbsp;\n\n<P>\n<DL>\n</DL>\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Package</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV PACKAGE&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/db/package-summary.html\"><B>NEXT PACKAGE</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/package-summary.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"package-summary.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/package-tree.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\ncom.yahoo.ycsb Class Hierarchy\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"com.yahoo.ycsb Class Hierarchy\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Tree</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/db/package-tree.html\"><B>NEXT</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/package-tree.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"package-tree.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<CENTER>\n<H2>\nHierarchy For Package com.yahoo.ycsb\n</H2>\n</CENTER>\n<DL>\n<DT><B>Package Hierarchies:</B><DD><A HREF=\"../../../overview-tree.html\">All Packages</A></DL>\n<HR>\n<H2>\nClass Hierarchy\n</H2>\n<UL>\n<LI TYPE=\"circle\">java.lang.Object<UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\"><B>Client</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\"><B>CommandLine</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\"><B>DB</B></A><UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\"><B>BasicDB</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\"><B>DBWrapper</B></A></UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/DBFactory.html\" title=\"class in com.yahoo.ycsb\"><B>DBFactory</B></A><LI TYPE=\"circle\">java.lang.Throwable (implements java.io.Serializable)\n<UL>\n<LI TYPE=\"circle\">java.lang.Exception<UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\"><B>DBException</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\"><B>UnknownDBException</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\"><B>WorkloadException</B></A></UL>\n</UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\"><B>Utils</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\"><B>Workload</B></A></UL>\n</UL>\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Tree</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;<A HREF=\"../../../com/yahoo/ycsb/db/package-tree.html\"><B>NEXT</B></A></FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../index.html?com/yahoo/ycsb/package-tree.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"package-tree.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/workloads/CoreWorkload.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\nCoreWorkload\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"CoreWorkload\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV CLASS&nbsp;\n&nbsp;NEXT CLASS</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/workloads/CoreWorkload.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"CoreWorkload.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<!-- ======== START OF CLASS DATA ======== -->\n<H2>\n<FONT SIZE=\"-1\">\ncom.yahoo.ycsb.workloads</FONT>\n<BR>\nClass CoreWorkload</H2>\n<PRE>\njava.lang.Object\n  <IMG SRC=\"../../../../resources/inherit.gif\" ALT=\"extended by \"><A HREF=\"../../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.Workload</A>\n      <IMG SRC=\"../../../../resources/inherit.gif\" ALT=\"extended by \"><B>com.yahoo.ycsb.workloads.CoreWorkload</B>\n</PRE>\n<HR>\n<DL>\n<DT><PRE>public class <B>CoreWorkload</B><DT>extends <A HREF=\"../../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A></DL>\n</PRE>\n\n<P>\nThe core benchmark scenario. Represents a set of clients doing simple CRUD operations. The relative \n proportion of different kinds of operations, and other properties of the workload, are controlled\n by parameters specified at runtime.\n \n Properties to control the client:\n <UL>\n <LI><b>fieldcount</b>: the number of fields in a record (default: 10)\n <LI><b>fieldlength</b>: the size of each field (default: 100)\n <LI><b>readallfields</b>: should reads read all fields (true) or just one (false) (default: true)\n <LI><b>readproportion</b>: what proportion of operations should be reads (default: 0.95)\n <LI><b>updateproportion</b>: what proportion of operations should be updates (default: 0.05)\n <LI><b>insertproportion</b>: what proportion of operations should be inserts (default: 0)\n <LI><b>scanproportion</b>: what proportion of operations should be scans (default: 0)\n <LI><b>readmodifywriteproportion</b>: what proportion of operations should be read a record, modify it, write it back (default: 0)\n <LI><b>requestdistribution</b>: what distribution should be used to select the records to operate on - uniform, zipfian or latest (default: uniform)\n <LI><b>maxscanlength</b>: for scans, what is the maximum number of records to scan (default: 1000)\n <LI><b>scanlengthdistribution</b>: for scans, what distribution should be used to choose the number of records to scan, for each scan, between 1 and maxscanlength (default: uniform)\n <LI><b>insertorder</b>: should records be inserted in order by key (\"ordered\"), or in hashed order (\"hashed\") (default: hashed)\n </ul>\n<P>\n\n<P>\n<HR>\n\n<P>\n<!-- =========== FIELD SUMMARY =========== -->\n\n<A NAME=\"field_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Field Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#FIELD_COUNT_PROPERTY\">FIELD_COUNT_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the property for the number of fields in a record.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#FIELD_COUNT_PROPERTY_DEFAULT\">FIELD_COUNT_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default number of fields in a record.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#FIELD_LENGTH_PROPERTY\">FIELD_LENGTH_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the property for the length of a field in bytes.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#FIELD_LENGTH_PROPERTY_DEFAULT\">FIELD_LENGTH_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The default length of a field in bytes.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#INSERT_ORDER_PROPERTY\">INSERT_ORDER_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the property for the order to insert records.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#INSERT_ORDER_PROPERTY_DEFAULT\">INSERT_ORDER_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default insert order.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#INSERT_PROPORTION_PROPERTY\">INSERT_PROPORTION_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the property for the proportion of transactions that are inserts.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#INSERT_PROPORTION_PROPERTY_DEFAULT\">INSERT_PROPORTION_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The default proportion of transactions that are inserts.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#MAX_SCAN_LENGTH_PROPERTY\">MAX_SCAN_LENGTH_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the property for the max scan length (number of records)</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#MAX_SCAN_LENGTH_PROPERTY_DEFAULT\">MAX_SCAN_LENGTH_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The default max scan length.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#READ_ALL_FIELDS_PROPERTY\">READ_ALL_FIELDS_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the property for deciding whether to read one field (false) or all fields (true) of a record.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#READ_ALL_FIELDS_PROPERTY_DEFAULT\">READ_ALL_FIELDS_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The default value for the readallfields property.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#READ_PROPORTION_PROPERTY\">READ_PROPORTION_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the property for the proportion of transactions that are reads.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#READ_PROPORTION_PROPERTY_DEFAULT\">READ_PROPORTION_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The default proportion of transactions that are reads.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#READMODIFYWRITE_PROPORTION_PROPERTY\">READMODIFYWRITE_PROPORTION_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the property for the proportion of transactions that are read-modify-write.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#READMODIFYWRITE_PROPORTION_PROPERTY_DEFAULT\">READMODIFYWRITE_PROPORTION_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The default proportion of transactions that are scans.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#REQUEST_DISTRIBUTION_PROPERTY\">REQUEST_DISTRIBUTION_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the property for the the distribution of requests across the keyspace.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#REQUEST_DISTRIBUTION_PROPERTY_DEFAULT\">REQUEST_DISTRIBUTION_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The default distribution of requests across the keyspace</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#SCAN_LENGTH_DISTRIBUTION_PROPERTY\">SCAN_LENGTH_DISTRIBUTION_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the property for the scan length distribution.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#SCAN_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT\">SCAN_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The default max scan length.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#SCAN_PROPORTION_PROPERTY\">SCAN_PROPORTION_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the property for the proportion of transactions that are scans.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#SCAN_PROPORTION_PROPERTY_DEFAULT\">SCAN_PROPORTION_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The default proportion of transactions that are scans.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#TABLENAME\">TABLENAME</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the database table to run queries against.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#UPDATE_PROPORTION_PROPERTY\">UPDATE_PROPORTION_PROPERTY</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The name of the property for the proportion of transactions that are updates.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#UPDATE_PROPORTION_PROPERTY_DEFAULT\">UPDATE_PROPORTION_PROPERTY_DEFAULT</A></B></CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The default proportion of transactions that are updates.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"fields_inherited_from_class_com.yahoo.ycsb.Workload\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Fields inherited from class com.yahoo.ycsb.<A HREF=\"../../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../../com/yahoo/ycsb/Workload.html#INSERT_START_PROPERTY\">INSERT_START_PROPERTY</A>, <A HREF=\"../../../../com/yahoo/ycsb/Workload.html#INSERT_START_PROPERTY_DEFAULT\">INSERT_START_PROPERTY_DEFAULT</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ======== CONSTRUCTOR SUMMARY ======== -->\n\n<A NAME=\"constructor_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Constructor Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#CoreWorkload()\">CoreWorkload</A></B>()</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n</TABLE>\n&nbsp;\n<!-- ========== METHOD SUMMARY =========== -->\n\n<A NAME=\"method_summary\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Method Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;boolean</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#doInsert(com.yahoo.ycsb.DB, java.lang.Object)\">doInsert</A></B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db,\n         java.lang.Object&nbsp;threadstate)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Do one insert operation.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;boolean</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#doTransaction(com.yahoo.ycsb.DB, java.lang.Object)\">doTransaction</A></B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db,\n              java.lang.Object&nbsp;threadstate)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Do one transaction operation.</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#doTransactionInsert(com.yahoo.ycsb.DB)\">doTransactionInsert</A></B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#doTransactionRead(com.yahoo.ycsb.DB)\">doTransactionRead</A></B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#doTransactionReadModifyWrite(com.yahoo.ycsb.DB)\">doTransactionReadModifyWrite</A></B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#doTransactionScan(com.yahoo.ycsb.DB)\">doTransactionScan</A></B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#doTransactionUpdate(com.yahoo.ycsb.DB)\">doTransactionUpdate</A></B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD ALIGN=\"right\" VALIGN=\"top\" WIDTH=\"1%\"><FONT SIZE=\"-1\">\n<CODE>&nbsp;void</CODE></FONT></TD>\n<TD><CODE><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html#init(java.util.Properties)\">init</A></B>(java.util.Properties&nbsp;p)</CODE>\n\n<BR>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Initialize the scenario.</TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_com.yahoo.ycsb.Workload\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class com.yahoo.ycsb.<A HREF=\"../../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A></B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE><A HREF=\"../../../../com/yahoo/ycsb/Workload.html#cleanup()\">cleanup</A>, <A HREF=\"../../../../com/yahoo/ycsb/Workload.html#initThread(java.util.Properties, int, int)\">initThread</A></CODE></TD>\n</TR>\n</TABLE>\n&nbsp;<A NAME=\"methods_inherited_from_class_java.lang.Object\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\"><B>Methods inherited from class java.lang.Object</B></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD>\n</TR>\n</TABLE>\n&nbsp;\n<P>\n\n<!-- ============ FIELD DETAIL =========== -->\n\n<A NAME=\"field_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Field Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"TABLENAME\"><!-- --></A><H3>\nTABLENAME</H3>\n<PRE>\npublic static final java.lang.String <B>TABLENAME</B></PRE>\n<DL>\n<DD>The name of the database table to run queries against.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.TABLENAME\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"FIELD_COUNT_PROPERTY\"><!-- --></A><H3>\nFIELD_COUNT_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>FIELD_COUNT_PROPERTY</B></PRE>\n<DL>\n<DD>The name of the property for the number of fields in a record.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.FIELD_COUNT_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"FIELD_COUNT_PROPERTY_DEFAULT\"><!-- --></A><H3>\nFIELD_COUNT_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>FIELD_COUNT_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DD>Default number of fields in a record.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"FIELD_LENGTH_PROPERTY\"><!-- --></A><H3>\nFIELD_LENGTH_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>FIELD_LENGTH_PROPERTY</B></PRE>\n<DL>\n<DD>The name of the property for the length of a field in bytes.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.FIELD_LENGTH_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"FIELD_LENGTH_PROPERTY_DEFAULT\"><!-- --></A><H3>\nFIELD_LENGTH_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>FIELD_LENGTH_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DD>The default length of a field in bytes.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.FIELD_LENGTH_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"READ_ALL_FIELDS_PROPERTY\"><!-- --></A><H3>\nREAD_ALL_FIELDS_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>READ_ALL_FIELDS_PROPERTY</B></PRE>\n<DL>\n<DD>The name of the property for deciding whether to read one field (false) or all fields (true) of a record.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.READ_ALL_FIELDS_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"READ_ALL_FIELDS_PROPERTY_DEFAULT\"><!-- --></A><H3>\nREAD_ALL_FIELDS_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>READ_ALL_FIELDS_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DD>The default value for the readallfields property.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.READ_ALL_FIELDS_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"READ_PROPORTION_PROPERTY\"><!-- --></A><H3>\nREAD_PROPORTION_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>READ_PROPORTION_PROPERTY</B></PRE>\n<DL>\n<DD>The name of the property for the proportion of transactions that are reads.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.READ_PROPORTION_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"READ_PROPORTION_PROPERTY_DEFAULT\"><!-- --></A><H3>\nREAD_PROPORTION_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>READ_PROPORTION_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DD>The default proportion of transactions that are reads.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.READ_PROPORTION_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"UPDATE_PROPORTION_PROPERTY\"><!-- --></A><H3>\nUPDATE_PROPORTION_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>UPDATE_PROPORTION_PROPERTY</B></PRE>\n<DL>\n<DD>The name of the property for the proportion of transactions that are updates.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.UPDATE_PROPORTION_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"UPDATE_PROPORTION_PROPERTY_DEFAULT\"><!-- --></A><H3>\nUPDATE_PROPORTION_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>UPDATE_PROPORTION_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DD>The default proportion of transactions that are updates.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.UPDATE_PROPORTION_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"INSERT_PROPORTION_PROPERTY\"><!-- --></A><H3>\nINSERT_PROPORTION_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>INSERT_PROPORTION_PROPERTY</B></PRE>\n<DL>\n<DD>The name of the property for the proportion of transactions that are inserts.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.INSERT_PROPORTION_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"INSERT_PROPORTION_PROPERTY_DEFAULT\"><!-- --></A><H3>\nINSERT_PROPORTION_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>INSERT_PROPORTION_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DD>The default proportion of transactions that are inserts.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.INSERT_PROPORTION_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"SCAN_PROPORTION_PROPERTY\"><!-- --></A><H3>\nSCAN_PROPORTION_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>SCAN_PROPORTION_PROPERTY</B></PRE>\n<DL>\n<DD>The name of the property for the proportion of transactions that are scans.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.SCAN_PROPORTION_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"SCAN_PROPORTION_PROPERTY_DEFAULT\"><!-- --></A><H3>\nSCAN_PROPORTION_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>SCAN_PROPORTION_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DD>The default proportion of transactions that are scans.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.SCAN_PROPORTION_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"READMODIFYWRITE_PROPORTION_PROPERTY\"><!-- --></A><H3>\nREADMODIFYWRITE_PROPORTION_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>READMODIFYWRITE_PROPORTION_PROPERTY</B></PRE>\n<DL>\n<DD>The name of the property for the proportion of transactions that are read-modify-write.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.READMODIFYWRITE_PROPORTION_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"READMODIFYWRITE_PROPORTION_PROPERTY_DEFAULT\"><!-- --></A><H3>\nREADMODIFYWRITE_PROPORTION_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>READMODIFYWRITE_PROPORTION_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DD>The default proportion of transactions that are scans.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.READMODIFYWRITE_PROPORTION_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"REQUEST_DISTRIBUTION_PROPERTY\"><!-- --></A><H3>\nREQUEST_DISTRIBUTION_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>REQUEST_DISTRIBUTION_PROPERTY</B></PRE>\n<DL>\n<DD>The name of the property for the the distribution of requests across the keyspace. Options are \"uniform\", \"zipfian\" and \"latest\"\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.REQUEST_DISTRIBUTION_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"REQUEST_DISTRIBUTION_PROPERTY_DEFAULT\"><!-- --></A><H3>\nREQUEST_DISTRIBUTION_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>REQUEST_DISTRIBUTION_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DD>The default distribution of requests across the keyspace\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.REQUEST_DISTRIBUTION_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"MAX_SCAN_LENGTH_PROPERTY\"><!-- --></A><H3>\nMAX_SCAN_LENGTH_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>MAX_SCAN_LENGTH_PROPERTY</B></PRE>\n<DL>\n<DD>The name of the property for the max scan length (number of records)\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.MAX_SCAN_LENGTH_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"MAX_SCAN_LENGTH_PROPERTY_DEFAULT\"><!-- --></A><H3>\nMAX_SCAN_LENGTH_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>MAX_SCAN_LENGTH_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DD>The default max scan length.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.MAX_SCAN_LENGTH_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"SCAN_LENGTH_DISTRIBUTION_PROPERTY\"><!-- --></A><H3>\nSCAN_LENGTH_DISTRIBUTION_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>SCAN_LENGTH_DISTRIBUTION_PROPERTY</B></PRE>\n<DL>\n<DD>The name of the property for the scan length distribution. Options are \"uniform\" and \"zipfian\" (favoring short scans)\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.SCAN_LENGTH_DISTRIBUTION_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"SCAN_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT\"><!-- --></A><H3>\nSCAN_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>SCAN_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DD>The default max scan length.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.SCAN_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"INSERT_ORDER_PROPERTY\"><!-- --></A><H3>\nINSERT_ORDER_PROPERTY</H3>\n<PRE>\npublic static final java.lang.String <B>INSERT_ORDER_PROPERTY</B></PRE>\n<DL>\n<DD>The name of the property for the order to insert records. Options are \"ordered\" or \"hashed\"\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.INSERT_ORDER_PROPERTY\">Constant Field Values</A></DL>\n</DL>\n<HR>\n\n<A NAME=\"INSERT_ORDER_PROPERTY_DEFAULT\"><!-- --></A><H3>\nINSERT_ORDER_PROPERTY_DEFAULT</H3>\n<PRE>\npublic static final java.lang.String <B>INSERT_ORDER_PROPERTY_DEFAULT</B></PRE>\n<DL>\n<DD>Default insert order.\n<P>\n<DL>\n<DT><B>See Also:</B><DD><A HREF=\"../../../../constant-values.html#com.yahoo.ycsb.workloads.CoreWorkload.INSERT_ORDER_PROPERTY_DEFAULT\">Constant Field Values</A></DL>\n</DL>\n\n<!-- ========= CONSTRUCTOR DETAIL ======== -->\n\n<A NAME=\"constructor_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Constructor Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"CoreWorkload()\"><!-- --></A><H3>\nCoreWorkload</H3>\n<PRE>\npublic <B>CoreWorkload</B>()</PRE>\n<DL>\n</DL>\n\n<!-- ============ METHOD DETAIL ========== -->\n\n<A NAME=\"method_detail\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"1\"><FONT SIZE=\"+2\">\n<B>Method Detail</B></FONT></TH>\n</TR>\n</TABLE>\n\n<A NAME=\"init(java.util.Properties)\"><!-- --></A><H3>\ninit</H3>\n<PRE>\npublic void <B>init</B>(java.util.Properties&nbsp;p)\n          throws <A HREF=\"../../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A></PRE>\n<DL>\n<DD>Initialize the scenario. \n Called once, in the main client thread, before any operations are started.\n<P>\n<DD><DL>\n<DT><B>Overrides:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/Workload.html#init(java.util.Properties)\">init</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A></CODE></DL>\n</DD>\n<DD><DL>\n\n<DT><B>Throws:</B>\n<DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A></CODE></DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"doInsert(com.yahoo.ycsb.DB, java.lang.Object)\"><!-- --></A><H3>\ndoInsert</H3>\n<PRE>\npublic boolean <B>doInsert</B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db,\n                        java.lang.Object&nbsp;threadstate)</PRE>\n<DL>\n<DD>Do one insert operation. Because it will be called concurrently from multiple client threads, this \n function must be thread safe. However, avoid synchronized, or the threads will block waiting for each \n other, and it will be difficult to reach the target throughput. Ideally, this function would have no side\n effects other than DB operations.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/Workload.html#doInsert(com.yahoo.ycsb.DB, java.lang.Object)\">doInsert</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A></CODE></DL>\n</DD>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"doTransaction(com.yahoo.ycsb.DB, java.lang.Object)\"><!-- --></A><H3>\ndoTransaction</H3>\n<PRE>\npublic boolean <B>doTransaction</B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db,\n                             java.lang.Object&nbsp;threadstate)</PRE>\n<DL>\n<DD>Do one transaction operation. Because it will be called concurrently from multiple client threads, this \n function must be thread safe. However, avoid synchronized, or the threads will block waiting for each \n other, and it will be difficult to reach the target throughput. Ideally, this function would have no side\n effects other than DB operations.\n<P>\n<DD><DL>\n<DT><B>Specified by:</B><DD><CODE><A HREF=\"../../../../com/yahoo/ycsb/Workload.html#doTransaction(com.yahoo.ycsb.DB, java.lang.Object)\">doTransaction</A></CODE> in class <CODE><A HREF=\"../../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A></CODE></DL>\n</DD>\n<DD><DL>\n\n<DT><B>Returns:</B><DD>false if the workload knows it is done for this thread. Client will terminate the thread. Return true otherwise. Return true for workloads that rely on operationcount. For workloads that read traces from a file, return true when there are more to do, false when you are done.</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"doTransactionRead(com.yahoo.ycsb.DB)\"><!-- --></A><H3>\ndoTransactionRead</H3>\n<PRE>\npublic void <B>doTransactionRead</B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"doTransactionReadModifyWrite(com.yahoo.ycsb.DB)\"><!-- --></A><H3>\ndoTransactionReadModifyWrite</H3>\n<PRE>\npublic void <B>doTransactionReadModifyWrite</B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"doTransactionScan(com.yahoo.ycsb.DB)\"><!-- --></A><H3>\ndoTransactionScan</H3>\n<PRE>\npublic void <B>doTransactionScan</B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"doTransactionUpdate(com.yahoo.ycsb.DB)\"><!-- --></A><H3>\ndoTransactionUpdate</H3>\n<PRE>\npublic void <B>doTransactionUpdate</B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<HR>\n\n<A NAME=\"doTransactionInsert(com.yahoo.ycsb.DB)\"><!-- --></A><H3>\ndoTransactionInsert</H3>\n<PRE>\npublic void <B>doTransactionInsert</B>(<A HREF=\"../../../../com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>&nbsp;db)</PRE>\n<DL>\n<DD><DL>\n</DL>\n</DD>\n</DL>\n<!-- ========= END OF CLASS DATA ========= -->\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Class</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV CLASS&nbsp;\n&nbsp;NEXT CLASS</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/workloads/CoreWorkload.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"CoreWorkload.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n<TR>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\n  SUMMARY:&nbsp;NESTED&nbsp;|&nbsp;<A HREF=\"#field_summary\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_summary\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_summary\">METHOD</A></FONT></TD>\n<TD VALIGN=\"top\" CLASS=\"NavBarCell3\"><FONT SIZE=\"-2\">\nDETAIL:&nbsp;<A HREF=\"#field_detail\">FIELD</A>&nbsp;|&nbsp;<A HREF=\"#constructor_detail\">CONSTR</A>&nbsp;|&nbsp;<A HREF=\"#method_detail\">METHOD</A></FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/workloads/package-frame.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\ncom.yahoo.ycsb.workloads\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../../stylesheet.css\" TITLE=\"Style\">\n\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\">\n<FONT size=\"+1\" CLASS=\"FrameTitleFont\">\n<A HREF=\"../../../../com/yahoo/ycsb/workloads/package-summary.html\" target=\"classFrame\">com.yahoo.ycsb.workloads</A></FONT>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" SUMMARY=\"\">\n<TR>\n<TD NOWRAP><FONT size=\"+1\" CLASS=\"FrameHeadingFont\">\nClasses</FONT>&nbsp;\n<FONT CLASS=\"FrameItemFont\">\n<BR>\n<A HREF=\"CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\" target=\"classFrame\">CoreWorkload</A></FONT></TD>\n</TR>\n</TABLE>\n\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/workloads/package-summary.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\ncom.yahoo.ycsb.workloads\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"com.yahoo.ycsb.workloads\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Package</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/measurements/package-summary.html\"><B>PREV PACKAGE</B></A>&nbsp;\n&nbsp;NEXT PACKAGE</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/workloads/package-summary.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"package-summary.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<H2>\nPackage com.yahoo.ycsb.workloads\n</H2>\n\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Class Summary</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"15%\"><B><A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A></B></TD>\n<TD>The core benchmark scenario.</TD>\n</TR>\n</TABLE>\n&nbsp;\n\n<P>\n<DL>\n</DL>\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Package</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/measurements/package-summary.html\"><B>PREV PACKAGE</B></A>&nbsp;\n&nbsp;NEXT PACKAGE</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/workloads/package-summary.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"package-summary.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/com/yahoo/ycsb/workloads/package-tree.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\ncom.yahoo.ycsb.workloads Class Hierarchy\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"../../../../stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"com.yahoo.ycsb.workloads Class Hierarchy\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Tree</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/measurements/package-tree.html\"><B>PREV</B></A>&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/workloads/package-tree.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"package-tree.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<CENTER>\n<H2>\nHierarchy For Package com.yahoo.ycsb.workloads\n</H2>\n</CENTER>\n<DL>\n<DT><B>Package Hierarchies:</B><DD><A HREF=\"../../../../overview-tree.html\">All Packages</A></DL>\n<HR>\n<H2>\nClass Hierarchy\n</H2>\n<UL>\n<LI TYPE=\"circle\">java.lang.Object<UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"../../../../com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\"><B>Workload</B></A><UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.workloads.<A HREF=\"../../../../com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\"><B>CoreWorkload</B></A></UL>\n</UL>\n</UL>\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"package-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Package</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Tree</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"../../../../help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;<A HREF=\"../../../../com/yahoo/ycsb/measurements/package-tree.html\"><B>PREV</B></A>&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"../../../../index.html?com/yahoo/ycsb/workloads/package-tree.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"package-tree.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"../../../../allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/constant-values.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\nConstant Field Values\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"Constant Field Values\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"index.html?constant-values.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"constant-values.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<CENTER>\n<H1>\nConstant Field Values</H1>\n</CENTER>\n<HR SIZE=\"4\" NOSHADE>\n<B>Contents</B><UL>\n<LI><A HREF=\"#com.yahoo\">com.yahoo.*</A>\n</UL>\n\n<A NAME=\"com.yahoo\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\"><FONT SIZE=\"+2\">\ncom.yahoo.*</FONT></TH>\n</TR>\n</TABLE>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.BasicDB.SIMULATE_DELAY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/BasicDB.html#SIMULATE_DELAY\">SIMULATE_DELAY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"basicdb.simulatedelay\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.BasicDB.SIMULATE_DELAY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/BasicDB.html#SIMULATE_DELAY_DEFAULT\">SIMULATE_DELAY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"0\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.BasicDB.VERBOSE\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/BasicDB.html#VERBOSE\">VERBOSE</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"basicdb.verbose\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.BasicDB.VERBOSE_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/BasicDB.html#VERBOSE_DEFAULT\">VERBOSE_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"true\"</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\">Client</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.Client.INSERT_COUNT_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/Client.html#INSERT_COUNT_PROPERTY\">INSERT_COUNT_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"insertcount\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.Client.OPERATION_COUNT_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/Client.html#OPERATION_COUNT_PROPERTY\">OPERATION_COUNT_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"operationcount\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.Client.RECORD_COUNT_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/Client.html#RECORD_COUNT_PROPERTY\">RECORD_COUNT_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"recordcount\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.Client.WORKLOAD_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/Client.html#WORKLOAD_PROPERTY\">WORKLOAD_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"workload\"</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\">CommandLine</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.CommandLine.DEFAULT_DB\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/CommandLine.html#DEFAULT_DB\">DEFAULT_DB</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"com.yahoo.ycsb.BasicDB\"</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\">Utils</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.Utils.FNV_offset_basis_32\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;int</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/Utils.html#FNV_offset_basis_32\">FNV_offset_basis_32</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>-2128831035</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.Utils.FNV_offset_basis_64\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;long</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/Utils.html#FNV_offset_basis_64\">FNV_offset_basis_64</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>-3750763034362895579L</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.Utils.FNV_prime_32\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;int</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/Utils.html#FNV_prime_32\">FNV_prime_32</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>16777619</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.Utils.FNV_prime_64\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;long</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/Utils.html#FNV_prime_64\">FNV_prime_64</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>1099511628211L</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.Workload.INSERT_START_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/Workload.html#INSERT_START_PROPERTY\">INSERT_START_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"insertstart\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.Workload.INSERT_START_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/Workload.html#INSERT_START_PROPERTY_DEFAULT\">INSERT_START_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"0\"</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.db.<A HREF=\"com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient5.CONNECTION_RETRY_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient5.html#CONNECTION_RETRY_PROPERTY\">CONNECTION_RETRY_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"cassandra.connectionretries\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient5.CONNECTION_RETRY_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient5.html#CONNECTION_RETRY_PROPERTY_DEFAULT\">CONNECTION_RETRY_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"300\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient5.Error\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;int</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient5.html#Error\">Error</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>-1</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient5.Ok\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;int</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient5.html#Ok\">Ok</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>0</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient5.OPERATION_RETRY_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient5.html#OPERATION_RETRY_PROPERTY\">OPERATION_RETRY_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"cassandra.operationretries\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient5.OPERATION_RETRY_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient5.html#OPERATION_RETRY_PROPERTY_DEFAULT\">OPERATION_RETRY_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"300\"</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.db.<A HREF=\"com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient6.CONNECTION_RETRY_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient6.html#CONNECTION_RETRY_PROPERTY\">CONNECTION_RETRY_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"cassandra.connectionretries\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient6.CONNECTION_RETRY_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient6.html#CONNECTION_RETRY_PROPERTY_DEFAULT\">CONNECTION_RETRY_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"300\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient6.Error\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;int</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient6.html#Error\">Error</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>-1</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient6.Ok\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;int</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient6.html#Ok\">Ok</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>0</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient6.OPERATION_RETRY_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient6.html#OPERATION_RETRY_PROPERTY\">OPERATION_RETRY_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"cassandra.operationretries\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient6.OPERATION_RETRY_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient6.html#OPERATION_RETRY_PROPERTY_DEFAULT\">OPERATION_RETRY_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"300\"</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.db.<A HREF=\"com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient7.CONNECTION_RETRY_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient7.html#CONNECTION_RETRY_PROPERTY\">CONNECTION_RETRY_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"cassandra.connectionretries\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient7.CONNECTION_RETRY_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient7.html#CONNECTION_RETRY_PROPERTY_DEFAULT\">CONNECTION_RETRY_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"300\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient7.Error\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;int</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient7.html#Error\">Error</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>-1</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient7.Ok\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;int</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient7.html#Ok\">Ok</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>0</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient7.OPERATION_RETRY_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient7.html#OPERATION_RETRY_PROPERTY\">OPERATION_RETRY_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"cassandra.operationretries\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.CassandraClient7.OPERATION_RETRY_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/CassandraClient7.html#OPERATION_RETRY_PROPERTY_DEFAULT\">OPERATION_RETRY_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"300\"</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.db.<A HREF=\"com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.HBaseClient.HttpError\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;int</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/HBaseClient.html#HttpError\">HttpError</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>-2</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.HBaseClient.NoMatchingRecord\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;int</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/HBaseClient.html#NoMatchingRecord\">NoMatchingRecord</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>-3</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.HBaseClient.Ok\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;int</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/HBaseClient.html#Ok\">Ok</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>0</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.db.HBaseClient.ServerError\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;int</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/db/HBaseClient.html#ServerError\">ServerError</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>-1</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.generator.<A HREF=\"com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ScrambledZipfianGenerator</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.generator.ScrambledZipfianGenerator.ITEM_COUNT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;long</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html#ITEM_COUNT\">ITEM_COUNT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>10000000000L</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.generator.ScrambledZipfianGenerator.ZETAN\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;double</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html#ZETAN\">ZETAN</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>52.93805640344461</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.generator.<A HREF=\"com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ZipfianGenerator</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.generator.ZipfianGenerator.ZIPFIAN_CONSTANT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;double</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/generator/ZipfianGenerator.html#ZIPFIAN_CONSTANT\">ZIPFIAN_CONSTANT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>0.99</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.measurements.<A HREF=\"com/yahoo/ycsb/measurements/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementHistogram</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.measurements.OneMeasurementHistogram.BUCKETS\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/measurements/OneMeasurementHistogram.html#BUCKETS\">BUCKETS</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"histogram.buckets\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.measurements.OneMeasurementHistogram.BUCKETS_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/measurements/OneMeasurementHistogram.html#BUCKETS_DEFAULT\">BUCKETS_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"1000\"</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.measurements.<A HREF=\"com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementTimeSeries</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.measurements.OneMeasurementTimeSeries.GRANULARITY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html#GRANULARITY\">GRANULARITY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"timeseries.granularity\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.measurements.OneMeasurementTimeSeries.GRANULARITY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html#GRANULARITY_DEFAULT\">GRANULARITY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"1000\"</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n\n<TABLE BORDER=\"1\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#EEEEFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"3\">com.yahoo.ycsb.workloads.<A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.FIELD_COUNT_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#FIELD_COUNT_PROPERTY\">FIELD_COUNT_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"fieldcount\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.FIELD_COUNT_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#FIELD_COUNT_PROPERTY_DEFAULT\">FIELD_COUNT_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"10\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.FIELD_LENGTH_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#FIELD_LENGTH_PROPERTY\">FIELD_LENGTH_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"fieldlength\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.FIELD_LENGTH_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#FIELD_LENGTH_PROPERTY_DEFAULT\">FIELD_LENGTH_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"100\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.INSERT_ORDER_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#INSERT_ORDER_PROPERTY\">INSERT_ORDER_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"insertorder\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.INSERT_ORDER_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#INSERT_ORDER_PROPERTY_DEFAULT\">INSERT_ORDER_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"hashed\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.INSERT_PROPORTION_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#INSERT_PROPORTION_PROPERTY\">INSERT_PROPORTION_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"insertproportion\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.INSERT_PROPORTION_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#INSERT_PROPORTION_PROPERTY_DEFAULT\">INSERT_PROPORTION_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"0.0\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.MAX_SCAN_LENGTH_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#MAX_SCAN_LENGTH_PROPERTY\">MAX_SCAN_LENGTH_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"maxscanlength\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.MAX_SCAN_LENGTH_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#MAX_SCAN_LENGTH_PROPERTY_DEFAULT\">MAX_SCAN_LENGTH_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"1000\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.READ_ALL_FIELDS_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#READ_ALL_FIELDS_PROPERTY\">READ_ALL_FIELDS_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"readallfields\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.READ_ALL_FIELDS_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#READ_ALL_FIELDS_PROPERTY_DEFAULT\">READ_ALL_FIELDS_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"true\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.READ_PROPORTION_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#READ_PROPORTION_PROPERTY\">READ_PROPORTION_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"readproportion\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.READ_PROPORTION_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#READ_PROPORTION_PROPERTY_DEFAULT\">READ_PROPORTION_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"0.95\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.READMODIFYWRITE_PROPORTION_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#READMODIFYWRITE_PROPORTION_PROPERTY\">READMODIFYWRITE_PROPORTION_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"readmodifywriteproportion\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.READMODIFYWRITE_PROPORTION_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#READMODIFYWRITE_PROPORTION_PROPERTY_DEFAULT\">READMODIFYWRITE_PROPORTION_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"0.0\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.REQUEST_DISTRIBUTION_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#REQUEST_DISTRIBUTION_PROPERTY\">REQUEST_DISTRIBUTION_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"requestdistribution\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.REQUEST_DISTRIBUTION_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#REQUEST_DISTRIBUTION_PROPERTY_DEFAULT\">REQUEST_DISTRIBUTION_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"uniform\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.SCAN_LENGTH_DISTRIBUTION_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#SCAN_LENGTH_DISTRIBUTION_PROPERTY\">SCAN_LENGTH_DISTRIBUTION_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"scanlengthdistribution\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.SCAN_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#SCAN_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT\">SCAN_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"uniform\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.SCAN_PROPORTION_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#SCAN_PROPORTION_PROPERTY\">SCAN_PROPORTION_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"scanproportion\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.SCAN_PROPORTION_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#SCAN_PROPORTION_PROPERTY_DEFAULT\">SCAN_PROPORTION_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"0.0\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.TABLENAME\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#TABLENAME\">TABLENAME</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"usertable\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.UPDATE_PROPORTION_PROPERTY\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#UPDATE_PROPORTION_PROPERTY\">UPDATE_PROPORTION_PROPERTY</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"updateproportion\"</CODE></TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<A NAME=\"com.yahoo.ycsb.workloads.CoreWorkload.UPDATE_PROPORTION_PROPERTY_DEFAULT\"><!-- --></A><TD ALIGN=\"right\"><FONT SIZE=\"-1\">\n<CODE>public&nbsp;static&nbsp;final&nbsp;java.lang.String</CODE></FONT></TD>\n<TD ALIGN=\"left\"><CODE><A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html#UPDATE_PROPORTION_PROPERTY_DEFAULT\">UPDATE_PROPORTION_PROPERTY_DEFAULT</A></CODE></TD>\n<TD ALIGN=\"right\"><CODE>\"0.05\"</CODE></TD>\n</TR>\n</FONT></TD>\n</TR>\n</TABLE>\n\n<P>\n\n<P>\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"index.html?constant-values.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"constant-values.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/deprecated-list.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\nDeprecated List\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"Deprecated List\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Deprecated</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"index.html?deprecated-list.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"deprecated-list.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<CENTER>\n<H2>\n<B>Deprecated API</B></H2>\n</CENTER>\n<HR SIZE=\"4\" NOSHADE>\n<B>Contents</B><UL>\n</UL>\n\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Deprecated</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"index.html?deprecated-list.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"deprecated-list.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/help-doc.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\nAPI Help\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"API Help\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Help</B></FONT>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"index.html?help-doc.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"help-doc.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<CENTER>\n<H1>\nHow This API Document Is Organized</H1>\n</CENTER>\nThis API (Application Programming Interface) document has pages corresponding to the items in the navigation bar, described as follows.<H3>\nOverview</H3>\n<BLOCKQUOTE>\n\n<P>\nThe <A HREF=\"overview-summary.html\">Overview</A> page is the front page of this API document and provides a list of all packages with a summary for each.  This page can also contain an overall description of the set of packages.</BLOCKQUOTE>\n<H3>\nPackage</H3>\n<BLOCKQUOTE>\n\n<P>\nEach package has a page that contains a list of its classes and interfaces, with a summary for each. This page can contain four categories:<UL>\n<LI>Interfaces (italic)<LI>Classes<LI>Enums<LI>Exceptions<LI>Errors<LI>Annotation Types</UL>\n</BLOCKQUOTE>\n<H3>\nClass/Interface</H3>\n<BLOCKQUOTE>\n\n<P>\nEach class, interface, nested class and nested interface has its own separate page. Each of these pages has three sections consisting of a class/interface description, summary tables, and detailed member descriptions:<UL>\n<LI>Class inheritance diagram<LI>Direct Subclasses<LI>All Known Subinterfaces<LI>All Known Implementing Classes<LI>Class/interface declaration<LI>Class/interface description\n<P>\n<LI>Nested Class Summary<LI>Field Summary<LI>Constructor Summary<LI>Method Summary\n<P>\n<LI>Field Detail<LI>Constructor Detail<LI>Method Detail</UL>\nEach summary entry contains the first sentence from the detailed description for that item. The summary entries are alphabetical, while the detailed descriptions are in the order they appear in the source code. This preserves the logical groupings established by the programmer.</BLOCKQUOTE>\n</BLOCKQUOTE>\n<H3>\nAnnotation Type</H3>\n<BLOCKQUOTE>\n\n<P>\nEach annotation type has its own separate page with the following sections:<UL>\n<LI>Annotation Type declaration<LI>Annotation Type description<LI>Required Element Summary<LI>Optional Element Summary<LI>Element Detail</UL>\n</BLOCKQUOTE>\n</BLOCKQUOTE>\n<H3>\nEnum</H3>\n<BLOCKQUOTE>\n\n<P>\nEach enum has its own separate page with the following sections:<UL>\n<LI>Enum declaration<LI>Enum description<LI>Enum Constant Summary<LI>Enum Constant Detail</UL>\n</BLOCKQUOTE>\n<H3>\nTree (Class Hierarchy)</H3>\n<BLOCKQUOTE>\nThere is a <A HREF=\"overview-tree.html\">Class Hierarchy</A> page for all packages, plus a hierarchy for each package. Each hierarchy page contains a list of classes and a list of interfaces. The classes are organized by inheritance structure starting with <code>java.lang.Object</code>. The interfaces do not inherit from <code>java.lang.Object</code>.<UL>\n<LI>When viewing the Overview page, clicking on \"Tree\" displays the hierarchy for all packages.<LI>When viewing a particular package, class or interface page, clicking \"Tree\" displays the hierarchy for only that package.</UL>\n</BLOCKQUOTE>\n<H3>\nDeprecated API</H3>\n<BLOCKQUOTE>\nThe <A HREF=\"deprecated-list.html\">Deprecated API</A> page lists all of the API that have been deprecated. A deprecated API is not recommended for use, generally due to improvements, and a replacement API is usually given. Deprecated APIs may be removed in future implementations.</BLOCKQUOTE>\n<H3>\nIndex</H3>\n<BLOCKQUOTE>\nThe <A HREF=\"index-all.html\">Index</A> contains an alphabetic list of all classes, interfaces, constructors, methods, and fields.</BLOCKQUOTE>\n<H3>\nPrev/Next</H3>\nThese links take you to the next or previous class, interface, package, or related page.<H3>\nFrames/No Frames</H3>\nThese links show and hide the HTML frames.  All pages are available with or without frames.\n<P>\n<H3>\nSerialized Form</H3>\nEach serializable or externalizable class has a description of its serialization fields and methods. This information is of interest to re-implementors, not to developers using the API. While there is no link in the navigation bar, you can get to this information by going to any serialized class and clicking \"Serialized Form\" in the \"See also\" section of the class description.\n<P>\n<H3>\nConstant Field Values</H3>\nThe <a href=\"constant-values.html\">Constant Field Values</a> page lists the static final fields and their values.\n<P>\n<FONT SIZE=\"-1\">\n<EM>\nThis help file applies to API documentation generated using the standard doclet.</EM>\n</FONT>\n<BR>\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Help</B></FONT>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"index.html?help-doc.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"help-doc.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/index-all.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\nIndex\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"./stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"Index\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"./overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"./overview-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"./deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Index</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"./help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"./index.html?index-all.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"index-all.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"./allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"./allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<A HREF=\"#_A_\">A</A> <A HREF=\"#_B_\">B</A> <A HREF=\"#_C_\">C</A> <A HREF=\"#_D_\">D</A> <A HREF=\"#_E_\">E</A> <A HREF=\"#_F_\">F</A> <A HREF=\"#_G_\">G</A> <A HREF=\"#_H_\">H</A> <A HREF=\"#_I_\">I</A> <A HREF=\"#_L_\">L</A> <A HREF=\"#_M_\">M</A> <A HREF=\"#_N_\">N</A> <A HREF=\"#_O_\">O</A> <A HREF=\"#_P_\">P</A> <A HREF=\"#_R_\">R</A> <A HREF=\"#_S_\">S</A> <A HREF=\"#_T_\">T</A> <A HREF=\"#_U_\">U</A> <A HREF=\"#_V_\">V</A> <A HREF=\"#_W_\">W</A> <A HREF=\"#_Z_\">Z</A> <A HREF=\"#___\">_</A> <HR>\n<A NAME=\"_A_\"><!-- --></A><H2>\n<B>A</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/generator/DiscreteGenerator.html#addValue(double, java.lang.String)\"><B>addValue(double, String)</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">DiscreteGenerator</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/Utils.html#ASCIIString(int)\"><B>ASCIIString(int)</B></A> - \nStatic method in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\">Utils</A>\n<DD>Generate a random ASCII string of a given length.\n</DL>\n<HR>\n<A NAME=\"_B_\"><!-- --></A><H2>\n<B>B</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\"><B>BasicDB</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/package-summary.html\">com.yahoo.ycsb</A><DD>Basic DB that just prints out the requested operations, instead of doing them against a database.<DT><A HREF=\"./com/yahoo/ycsb/BasicDB.html#BasicDB()\"><B>BasicDB()</B></A> - \nConstructor for class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html#BUCKETS\"><B>BUCKETS</B></A> - \nStatic variable in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementHistogram</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html#BUCKETS_DEFAULT\"><B>BUCKETS_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementHistogram</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_C_\"><!-- --></A><H2>\n<B>C</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\"><B>CassandraClient5</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/db/package-summary.html\">com.yahoo.ycsb.db</A><DD>Cassandra 0.5 client for YCSB framework<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#CassandraClient5()\"><B>CassandraClient5()</B></A> - \nConstructor for class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\"><B>CassandraClient6</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/db/package-summary.html\">com.yahoo.ycsb.db</A><DD>Cassandra 0.6 client for YCSB framework<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#CassandraClient6()\"><B>CassandraClient6()</B></A> - \nConstructor for class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\"><B>CassandraClient7</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/db/package-summary.html\">com.yahoo.ycsb.db</A><DD>Cassandra 0.7 client for YCSB framework<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#CassandraClient7()\"><B>CassandraClient7()</B></A> - \nConstructor for class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/Client.html#checkRequiredProperties(java.util.Properties)\"><B>checkRequiredProperties(Properties)</B></A> - \nStatic method in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\">Client</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#cleanup()\"><B>cleanup()</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>Cleanup any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#cleanup()\"><B>cleanup()</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>Cleanup any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#cleanup()\"><B>cleanup()</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>Cleanup any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/DB.html#cleanup()\"><B>cleanup()</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>\n<DD>Cleanup any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#cleanup()\"><B>cleanup()</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>Cleanup any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/DBWrapper.html#cleanup()\"><B>cleanup()</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\">DBWrapper</A>\n<DD>Cleanup any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/Workload.html#cleanup()\"><B>cleanup()</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A>\n<DD>Cleanup the scenario.\n<DT><A HREF=\"./com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\"><B>Client</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/package-summary.html\">com.yahoo.ycsb</A><DD>Main class for executing YCSB.<DT><A HREF=\"./com/yahoo/ycsb/Client.html#Client()\"><B>Client()</B></A> - \nConstructor for class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\">Client</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/package-summary.html\"><B>com.yahoo.ycsb</B></A> - package com.yahoo.ycsb<DD>&nbsp;<DT><A HREF=\"./com/yahoo/ycsb/db/package-summary.html\"><B>com.yahoo.ycsb.db</B></A> - package com.yahoo.ycsb.db<DD>&nbsp;<DT><A HREF=\"./com/yahoo/ycsb/generator/package-summary.html\"><B>com.yahoo.ycsb.generator</B></A> - package com.yahoo.ycsb.generator<DD>&nbsp;<DT><A HREF=\"./com/yahoo/ycsb/measurements/package-summary.html\"><B>com.yahoo.ycsb.measurements</B></A> - package com.yahoo.ycsb.measurements<DD>&nbsp;<DT><A HREF=\"./com/yahoo/ycsb/workloads/package-summary.html\"><B>com.yahoo.ycsb.workloads</B></A> - package com.yahoo.ycsb.workloads<DD>&nbsp;<DT><A HREF=\"./com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\"><B>CommandLine</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/package-summary.html\">com.yahoo.ycsb</A><DD>A simple command line client to a database, using the appropriate com.yahoo.ycsb.DB implementation.<DT><A HREF=\"./com/yahoo/ycsb/CommandLine.html#CommandLine()\"><B>CommandLine()</B></A> - \nConstructor for class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\">CommandLine</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#CONNECTION_RETRY_PROPERTY\"><B>CONNECTION_RETRY_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#CONNECTION_RETRY_PROPERTY\"><B>CONNECTION_RETRY_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#CONNECTION_RETRY_PROPERTY\"><B>CONNECTION_RETRY_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#CONNECTION_RETRY_PROPERTY_DEFAULT\"><B>CONNECTION_RETRY_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#CONNECTION_RETRY_PROPERTY_DEFAULT\"><B>CONNECTION_RETRY_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#CONNECTION_RETRY_PROPERTY_DEFAULT\"><B>CONNECTION_RETRY_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#ConnectionRetries\"><B>ConnectionRetries</B></A> - \nVariable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#ConnectionRetries\"><B>ConnectionRetries</B></A> - \nVariable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#ConnectionRetries\"><B>ConnectionRetries</B></A> - \nVariable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\"><B>CoreWorkload</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/workloads/package-summary.html\">com.yahoo.ycsb.workloads</A><DD>The core benchmark scenario.<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#CoreWorkload()\"><B>CoreWorkload()</B></A> - \nConstructor for class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/generator/CounterGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>CounterGenerator</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/generator/package-summary.html\">com.yahoo.ycsb.generator</A><DD>Generates a sequence of integers 0, 1, ...<DT><A HREF=\"./com/yahoo/ycsb/generator/CounterGenerator.html#CounterGenerator(int)\"><B>CounterGenerator(int)</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/CounterGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">CounterGenerator</A>\n<DD>Create a counter that starts at countstart\n</DL>\n<HR>\n<A NAME=\"_D_\"><!-- --></A><H2>\n<B>D</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\"><B>DB</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/package-summary.html\">com.yahoo.ycsb</A><DD>A layer for accessing a database to be benchmarked.<DT><A HREF=\"./com/yahoo/ycsb/DB.html#DB()\"><B>DB()</B></A> - \nConstructor for class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\"><B>DBException</B></A> - Exception in <A HREF=\"./com/yahoo/ycsb/package-summary.html\">com.yahoo.ycsb</A><DD>Something bad happened while interacting with the database.<DT><A HREF=\"./com/yahoo/ycsb/DBException.html#DBException(java.lang.String)\"><B>DBException(String)</B></A> - \nConstructor for exception com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/DBException.html#DBException()\"><B>DBException()</B></A> - \nConstructor for exception com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/DBException.html#DBException(java.lang.String, java.lang.Throwable)\"><B>DBException(String, Throwable)</B></A> - \nConstructor for exception com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/DBException.html#DBException(java.lang.Throwable)\"><B>DBException(Throwable)</B></A> - \nConstructor for exception com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">DBException</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/DBFactory.html\" title=\"class in com.yahoo.ycsb\"><B>DBFactory</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/package-summary.html\">com.yahoo.ycsb</A><DD>Creates a DB layer by dynamically classloading the specified DB class.<DT><A HREF=\"./com/yahoo/ycsb/DBFactory.html#DBFactory()\"><B>DBFactory()</B></A> - \nConstructor for class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBFactory.html\" title=\"class in com.yahoo.ycsb\">DBFactory</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\"><B>DBWrapper</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/package-summary.html\">com.yahoo.ycsb</A><DD>Wrapper around a \"real\" DB that measures latencies and counts return codes.<DT><A HREF=\"./com/yahoo/ycsb/DBWrapper.html#DBWrapper(com.yahoo.ycsb.DB)\"><B>DBWrapper(DB)</B></A> - \nConstructor for class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\">DBWrapper</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/CommandLine.html#DEFAULT_DB\"><B>DEFAULT_DB</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\">CommandLine</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/BasicDB.html#delete(java.lang.String, java.lang.String)\"><B>delete(String, String)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A>\n<DD>Delete a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#delete(java.lang.String, java.lang.String)\"><B>delete(String, String)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>Delete a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#delete(java.lang.String, java.lang.String)\"><B>delete(String, String)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>Delete a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#delete(java.lang.String, java.lang.String)\"><B>delete(String, String)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>Delete a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/DB.html#delete(java.lang.String, java.lang.String)\"><B>delete(String, String)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>\n<DD>Delete a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#delete(java.lang.String, java.lang.String)\"><B>delete(String, String)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>Delete a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html#delete(java.lang.String, java.lang.String)\"><B>delete(String, String)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\">MongoDbClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/DBWrapper.html#delete(java.lang.String, java.lang.String)\"><B>delete(String, String)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\">DBWrapper</A>\n<DD>Delete a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>DiscreteGenerator</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/generator/package-summary.html\">com.yahoo.ycsb.generator</A><DD>Generates a distribution by choosing from a discrete set of values.<DT><A HREF=\"./com/yahoo/ycsb/generator/DiscreteGenerator.html#DiscreteGenerator()\"><B>DiscreteGenerator()</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">DiscreteGenerator</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/Workload.html#doInsert(com.yahoo.ycsb.DB, java.lang.Object)\"><B>doInsert(DB, Object)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A>\n<DD>Do one insert operation.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#doInsert(com.yahoo.ycsb.DB, java.lang.Object)\"><B>doInsert(DB, Object)</B></A> - \nMethod in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>Do one insert operation.\n<DT><A HREF=\"./com/yahoo/ycsb/Workload.html#doTransaction(com.yahoo.ycsb.DB, java.lang.Object)\"><B>doTransaction(DB, Object)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A>\n<DD>Do one transaction operation.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#doTransaction(com.yahoo.ycsb.DB, java.lang.Object)\"><B>doTransaction(DB, Object)</B></A> - \nMethod in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>Do one transaction operation.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#doTransactionInsert(com.yahoo.ycsb.DB)\"><B>doTransactionInsert(DB)</B></A> - \nMethod in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#doTransactionRead(com.yahoo.ycsb.DB)\"><B>doTransactionRead(DB)</B></A> - \nMethod in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#doTransactionReadModifyWrite(com.yahoo.ycsb.DB)\"><B>doTransactionReadModifyWrite(DB)</B></A> - \nMethod in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#doTransactionScan(com.yahoo.ycsb.DB)\"><B>doTransactionScan(DB)</B></A> - \nMethod in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#doTransactionUpdate(com.yahoo.ycsb.DB)\"><B>doTransactionUpdate(DB)</B></A> - \nMethod in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_E_\"><!-- --></A><H2>\n<B>E</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#Error\"><B>Error</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#Error\"><B>Error</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#Error\"><B>Error</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_F_\"><!-- --></A><H2>\n<B>F</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#FIELD_COUNT_PROPERTY\"><B>FIELD_COUNT_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The name of the property for the number of fields in a record.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#FIELD_COUNT_PROPERTY_DEFAULT\"><B>FIELD_COUNT_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>Default number of fields in a record.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#FIELD_LENGTH_PROPERTY\"><B>FIELD_LENGTH_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The name of the property for the length of a field in bytes.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#FIELD_LENGTH_PROPERTY_DEFAULT\"><B>FIELD_LENGTH_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The default length of a field in bytes.\n<DT><A HREF=\"./com/yahoo/ycsb/Utils.html#FNV_offset_basis_32\"><B>FNV_offset_basis_32</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\">Utils</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/Utils.html#FNV_offset_basis_64\"><B>FNV_offset_basis_64</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\">Utils</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/Utils.html#FNV_prime_32\"><B>FNV_prime_32</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\">Utils</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/Utils.html#FNV_prime_64\"><B>FNV_prime_64</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\">Utils</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/Utils.html#FNVhash32(int)\"><B>FNVhash32(int)</B></A> - \nStatic method in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\">Utils</A>\n<DD>32 bit FNV hash.\n<DT><A HREF=\"./com/yahoo/ycsb/Utils.html#FNVhash64(long)\"><B>FNVhash64(long)</B></A> - \nStatic method in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\">Utils</A>\n<DD>64 bit FNV hash.\n</DL>\n<HR>\n<A NAME=\"_G_\"><!-- --></A><H2>\n<B>G</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/generator/Generator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>Generator</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/generator/package-summary.html\">com.yahoo.ycsb.generator</A><DD>An expression that generates a sequence of string values, following some distribution (Uniform, Zipfian, Sequential, etc.)<DT><A HREF=\"./com/yahoo/ycsb/generator/Generator.html#Generator()\"><B>Generator()</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/Generator.html\" title=\"class in com.yahoo.ycsb.generator\">Generator</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#getHTable(java.lang.String)\"><B>getHTable(String)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html#getMeasurements()\"><B>getMeasurements()</B></A> - \nStatic method in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html\" title=\"class in com.yahoo.ycsb.measurements\">Measurements</A>\n<DD>Return the singleton Measurements object.\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurement.html#getName()\"><B>getName()</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurement.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurement</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/DB.html#getProperties()\"><B>getProperties()</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>\n<DD>Get the set of properties for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/DBWrapper.html#getProperties()\"><B>getProperties()</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\">DBWrapper</A>\n<DD>Get the set of properties for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html#getSummary()\"><B>getSummary()</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html\" title=\"class in com.yahoo.ycsb.measurements\">Measurements</A>\n<DD>Return a one line summary of the measurements.\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurement.html#getSummary()\"><B>getSummary()</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurement.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurement</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html#getSummary()\"><B>getSummary()</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementHistogram</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html#getSummary()\"><B>getSummary()</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementTimeSeries</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html#GRANULARITY\"><B>GRANULARITY</B></A> - \nStatic variable in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementTimeSeries</A>\n<DD>Granularity for time series; measurements will be averaged in chunks of this granularity.\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html#GRANULARITY_DEFAULT\"><B>GRANULARITY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementTimeSeries</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_H_\"><!-- --></A><H2>\n<B>H</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/Utils.html#hash(int)\"><B>hash(int)</B></A> - \nStatic method in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\">Utils</A>\n<DD>Hash an integer value.\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>HBaseClient</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/db/package-summary.html\">com.yahoo.ycsb.db</A><DD>HBase client for YCSB framework<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#HBaseClient()\"><B>HBaseClient()</B></A> - \nConstructor for class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/CommandLine.html#help()\"><B>help()</B></A> - \nStatic method in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\">CommandLine</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#HttpError\"><B>HttpError</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_I_\"><!-- --></A><H2>\n<B>I</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/BasicDB.html#init()\"><B>init()</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A>\n<DD>Initialize any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#init()\"><B>init()</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>Initialize any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#init()\"><B>init()</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>Initialize any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#init()\"><B>init()</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>Initialize any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#init()\"><B>init()</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>Initialize any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/DB.html#init()\"><B>init()</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>\n<DD>Initialize any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html#init()\"><B>init()</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\">MongoDbClient</A>\n<DD>Initialize any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/DBWrapper.html#init()\"><B>init()</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\">DBWrapper</A>\n<DD>Initialize any state for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/Workload.html#init(java.util.Properties)\"><B>init(Properties)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A>\n<DD>Initialize the scenario.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#init(java.util.Properties)\"><B>init(Properties)</B></A> - \nMethod in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>Initialize the scenario.\n<DT><A HREF=\"./com/yahoo/ycsb/Workload.html#initThread(java.util.Properties, int, int)\"><B>initThread(Properties, int, int)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A>\n<DD>Initialize any state for a particular client thread.\n<DT><A HREF=\"./com/yahoo/ycsb/BasicDB.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\"><B>insert(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A>\n<DD>Insert a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\"><B>insert(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>Insert a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\"><B>insert(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>Insert a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\"><B>insert(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>Insert a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\"><B>insert(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>Insert a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/DB.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\"><B>insert(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>\n<DD>Insert a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\"><B>insert(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\">MongoDbClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/DBWrapper.html#insert(java.lang.String, java.lang.String, java.util.HashMap)\"><B>insert(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\">DBWrapper</A>\n<DD>Insert a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/Client.html#INSERT_COUNT_PROPERTY\"><B>INSERT_COUNT_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\">Client</A>\n<DD>Indicates how many inserts to do, if less than recordcount.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#INSERT_ORDER_PROPERTY\"><B>INSERT_ORDER_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The name of the property for the order to insert records.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#INSERT_ORDER_PROPERTY_DEFAULT\"><B>INSERT_ORDER_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>Default insert order.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#INSERT_PROPORTION_PROPERTY\"><B>INSERT_PROPORTION_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The name of the property for the proportion of transactions that are inserts.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#INSERT_PROPORTION_PROPERTY_DEFAULT\"><B>INSERT_PROPORTION_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The default proportion of transactions that are inserts.\n<DT><A HREF=\"./com/yahoo/ycsb/Workload.html#INSERT_START_PROPERTY\"><B>INSERT_START_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/Workload.html#INSERT_START_PROPERTY_DEFAULT\"><B>INSERT_START_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/generator/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>IntegerGenerator</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/generator/package-summary.html\">com.yahoo.ycsb.generator</A><DD>A generator that is capable of generating ints as well as strings<DT><A HREF=\"./com/yahoo/ycsb/generator/IntegerGenerator.html#IntegerGenerator()\"><B>IntegerGenerator()</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">IntegerGenerator</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html#ITEM_COUNT\"><B>ITEM_COUNT</B></A> - \nStatic variable in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ScrambledZipfianGenerator</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_L_\"><!-- --></A><H2>\n<B>L</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/generator/IntegerGenerator.html#lastInt()\"><B>lastInt()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">IntegerGenerator</A>\n<DD>Return the previous int generated by the distribution.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/DiscreteGenerator.html#lastString()\"><B>lastString()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">DiscreteGenerator</A>\n<DD>Return the previous string generated by the distribution; e.g., returned from the last nextString() call.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/Generator.html#lastString()\"><B>lastString()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/Generator.html\" title=\"class in com.yahoo.ycsb.generator\">Generator</A>\n<DD>Return the previous string generated by the distribution; e.g., returned from the last nextString() call.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/IntegerGenerator.html#lastString()\"><B>lastString()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">IntegerGenerator</A>\n<DD>Return the previous string generated by the distribution; e.g., returned from the last nextString() call.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/UniformGenerator.html#lastString()\"><B>lastString()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/UniformGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">UniformGenerator</A>\n<DD>Return the previous string generated by the distribution; e.g., returned from the last nextString() call.\n</DL>\n<HR>\n<A NAME=\"_M_\"><!-- --></A><H2>\n<B>M</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/Client.html#main(java.lang.String[])\"><B>main(String[])</B></A> - \nStatic method in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\">Client</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/CommandLine.html#main(java.lang.String[])\"><B>main(String[])</B></A> - \nStatic method in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\">CommandLine</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#main(java.lang.String[])\"><B>main(String[])</B></A> - \nStatic method in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#main(java.lang.String[])\"><B>main(String[])</B></A> - \nStatic method in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#main(java.lang.String[])\"><B>main(String[])</B></A> - \nStatic method in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#main(java.lang.String[])\"><B>main(String[])</B></A> - \nStatic method in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html#main(java.lang.String[])\"><B>main(String[])</B></A> - \nStatic method in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ScrambledZipfianGenerator</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/generator/SkewedLatestGenerator.html#main(java.lang.String[])\"><B>main(String[])</B></A> - \nStatic method in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">SkewedLatestGenerator</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html#main(java.lang.String[])\"><B>main(String[])</B></A> - \nStatic method in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ZipfianGenerator</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#MAX_SCAN_LENGTH_PROPERTY\"><B>MAX_SCAN_LENGTH_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The name of the property for the max scan length (number of records)\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#MAX_SCAN_LENGTH_PROPERTY_DEFAULT\"><B>MAX_SCAN_LENGTH_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The default max scan length.\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html#measure(java.lang.String, int)\"><B>measure(String, int)</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html\" title=\"class in com.yahoo.ycsb.measurements\">Measurements</A>\n<DD>Report a single value of a single metric.\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurement.html#measure(int)\"><B>measure(int)</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurement.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurement</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html#measure(int)\"><B>measure(int)</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementHistogram</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html#measure(int)\"><B>measure(int)</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementTimeSeries</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html\" title=\"class in com.yahoo.ycsb.measurements\"><B>Measurements</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/measurements/package-summary.html\">com.yahoo.ycsb.measurements</A><DD>Collects latency measurements, and reports them when requested.<DT><A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html#Measurements(java.util.Properties)\"><B>Measurements(Properties)</B></A> - \nConstructor for class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html\" title=\"class in com.yahoo.ycsb.measurements\">Measurements</A>\n<DD>Create a new object with the specified properties.\n<DT><A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>MongoDbClient</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/db/package-summary.html\">com.yahoo.ycsb.db</A><DD>MongoDB client for YCSB framework.<DT><A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html#MongoDbClient()\"><B>MongoDbClient()</B></A> - \nConstructor for class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\">MongoDbClient</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_N_\"><!-- --></A><H2>\n<B>N</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/DBFactory.html#newDB(java.lang.String, java.util.Properties)\"><B>newDB(String, Properties)</B></A> - \nStatic method in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBFactory.html\" title=\"class in com.yahoo.ycsb\">DBFactory</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/generator/CounterGenerator.html#nextInt()\"><B>nextInt()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/CounterGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">CounterGenerator</A>\n<DD>If the generator returns numeric (integer) values, return the next value as an int.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/DiscreteGenerator.html#nextInt()\"><B>nextInt()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">DiscreteGenerator</A>\n<DD>If the generator returns numeric (integer) values, return the next value as an int.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/IntegerGenerator.html#nextInt()\"><B>nextInt()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">IntegerGenerator</A>\n<DD>Return the next value as an int.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html#nextInt()\"><B>nextInt()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ScrambledZipfianGenerator</A>\n<DD>Return the next int in the sequence.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/SkewedLatestGenerator.html#nextInt()\"><B>nextInt()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">SkewedLatestGenerator</A>\n<DD>Generate the next string in the distribution, skewed Zipfian favoring the items most recently returned by the basis generator.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/UniformIntegerGenerator.html#nextInt()\"><B>nextInt()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/UniformIntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">UniformIntegerGenerator</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html#nextInt(int)\"><B>nextInt(int)</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ZipfianGenerator</A>\n<DD>Generate the next item.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html#nextInt()\"><B>nextInt()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ZipfianGenerator</A>\n<DD>Return the next value, skewed by the Zipfian distribution.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html#nextLong()\"><B>nextLong()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ScrambledZipfianGenerator</A>\n<DD>Return the next long in the sequence.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html#nextLong(long)\"><B>nextLong(long)</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ZipfianGenerator</A>\n<DD>Generate the next item as a long.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html#nextLong()\"><B>nextLong()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ZipfianGenerator</A>\n<DD>Return the next value, skewed by the Zipfian distribution.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/DiscreteGenerator.html#nextString()\"><B>nextString()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">DiscreteGenerator</A>\n<DD>Generate the next string in the distribution.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/Generator.html#nextString()\"><B>nextString()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/Generator.html\" title=\"class in com.yahoo.ycsb.generator\">Generator</A>\n<DD>Generate the next string in the distribution.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/IntegerGenerator.html#nextString()\"><B>nextString()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">IntegerGenerator</A>\n<DD>Generate the next string in the distribution.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/UniformGenerator.html#nextString()\"><B>nextString()</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/UniformGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">UniformGenerator</A>\n<DD>Generate the next string in the distribution.\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#NoMatchingRecord\"><B>NoMatchingRecord</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_O_\"><!-- --></A><H2>\n<B>O</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#Ok\"><B>Ok</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#Ok\"><B>Ok</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#Ok\"><B>Ok</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#Ok\"><B>Ok</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurement.html\" title=\"class in com.yahoo.ycsb.measurements\"><B>OneMeasurement</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/measurements/package-summary.html\">com.yahoo.ycsb.measurements</A><DD>A single measured metric (such as READ LATENCY)<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurement.html#OneMeasurement(java.lang.String)\"><B>OneMeasurement(String)</B></A> - \nConstructor for class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurement.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurement</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb.measurements\"><B>OneMeasurementHistogram</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/measurements/package-summary.html\">com.yahoo.ycsb.measurements</A><DD>Take measurements and maintain a histogram of a given metric, such as READ LATENCY.<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html#OneMeasurementHistogram(java.lang.String, java.util.Properties)\"><B>OneMeasurementHistogram(String, Properties)</B></A> - \nConstructor for class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementHistogram</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb.measurements\"><B>OneMeasurementTimeSeries</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/measurements/package-summary.html\">com.yahoo.ycsb.measurements</A><DD>A time series measurement of a metric, such as READ LATENCY.<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html#OneMeasurementTimeSeries(java.lang.String, java.util.Properties)\"><B>OneMeasurementTimeSeries(String, Properties)</B></A> - \nConstructor for class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementTimeSeries</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/Client.html#OPERATION_COUNT_PROPERTY\"><B>OPERATION_COUNT_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\">Client</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#OPERATION_RETRY_PROPERTY\"><B>OPERATION_RETRY_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#OPERATION_RETRY_PROPERTY\"><B>OPERATION_RETRY_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#OPERATION_RETRY_PROPERTY\"><B>OPERATION_RETRY_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#OPERATION_RETRY_PROPERTY_DEFAULT\"><B>OPERATION_RETRY_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#OPERATION_RETRY_PROPERTY_DEFAULT\"><B>OPERATION_RETRY_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#OPERATION_RETRY_PROPERTY_DEFAULT\"><B>OPERATION_RETRY_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#OperationRetries\"><B>OperationRetries</B></A> - \nVariable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#OperationRetries\"><B>OperationRetries</B></A> - \nVariable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#OperationRetries\"><B>OperationRetries</B></A> - \nVariable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_P_\"><!-- --></A><H2>\n<B>P</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html#printReport(java.io.PrintStream)\"><B>printReport(PrintStream)</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html\" title=\"class in com.yahoo.ycsb.measurements\">Measurements</A>\n<DD>Print the full report to the listed PrintStream.\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurement.html#printReport(java.io.PrintStream)\"><B>printReport(PrintStream)</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurement.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurement</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html#printReport(java.io.PrintStream)\"><B>printReport(PrintStream)</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementHistogram</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html#printReport(java.io.PrintStream)\"><B>printReport(PrintStream)</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementTimeSeries</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_R_\"><!-- --></A><H2>\n<B>R</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/BasicDB.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><B>read(String, String, Set&lt;String&gt;, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A>\n<DD>Read a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><B>read(String, String, Set&lt;String&gt;, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>Read a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><B>read(String, String, Set&lt;String&gt;, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>Read a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><B>read(String, String, Set&lt;String&gt;, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>Read a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><B>read(String, String, Set&lt;String&gt;, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>Read a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><B>read(String, String, Set&lt;String&gt;, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\">MongoDbClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/DB.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><B>read(String, String, Set&lt;String&gt;, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>\n<DD>Read a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/DBWrapper.html#read(java.lang.String, java.lang.String, java.util.Set, java.util.HashMap)\"><B>read(String, String, Set&lt;String&gt;, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\">DBWrapper</A>\n<DD>Read a record from the database.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#READ_ALL_FIELDS_PROPERTY\"><B>READ_ALL_FIELDS_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The name of the property for deciding whether to read one field (false) or all fields (true) of a record.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#READ_ALL_FIELDS_PROPERTY_DEFAULT\"><B>READ_ALL_FIELDS_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The default value for the readallfields property.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#READ_PROPORTION_PROPERTY\"><B>READ_PROPORTION_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The name of the property for the proportion of transactions that are reads.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#READ_PROPORTION_PROPERTY_DEFAULT\"><B>READ_PROPORTION_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The default proportion of transactions that are reads.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#READMODIFYWRITE_PROPORTION_PROPERTY\"><B>READMODIFYWRITE_PROPORTION_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The name of the property for the proportion of transactions that are read-modify-write.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#READMODIFYWRITE_PROPORTION_PROPERTY_DEFAULT\"><B>READMODIFYWRITE_PROPORTION_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The default proportion of transactions that are scans.\n<DT><A HREF=\"./com/yahoo/ycsb/Client.html#RECORD_COUNT_PROPERTY\"><B>RECORD_COUNT_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\">Client</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html#reportReturnCode(java.lang.String, int)\"><B>reportReturnCode(String, int)</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html\" title=\"class in com.yahoo.ycsb.measurements\">Measurements</A>\n<DD>Report a return code for a single DB operaiton.\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurement.html#reportReturnCode(int)\"><B>reportReturnCode(int)</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurement.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurement</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html#reportReturnCode(int)\"><B>reportReturnCode(int)</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementHistogram</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html#reportReturnCode(int)\"><B>reportReturnCode(int)</B></A> - \nMethod in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb.measurements\">OneMeasurementTimeSeries</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#REQUEST_DISTRIBUTION_PROPERTY\"><B>REQUEST_DISTRIBUTION_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The name of the property for the the distribution of requests across the keyspace.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#REQUEST_DISTRIBUTION_PROPERTY_DEFAULT\"><B>REQUEST_DISTRIBUTION_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The default distribution of requests across the keyspace\n</DL>\n<HR>\n<A NAME=\"_S_\"><!-- --></A><H2>\n<B>S</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/BasicDB.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><B>scan(String, String, int, Set&lt;String&gt;, Vector&lt;HashMap&lt;String, String&gt;&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A>\n<DD>Perform a range scan for a set of records in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><B>scan(String, String, int, Set&lt;String&gt;, Vector&lt;HashMap&lt;String, String&gt;&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>Perform a range scan for a set of records in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><B>scan(String, String, int, Set&lt;String&gt;, Vector&lt;HashMap&lt;String, String&gt;&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>Perform a range scan for a set of records in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><B>scan(String, String, int, Set&lt;String&gt;, Vector&lt;HashMap&lt;String, String&gt;&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>Perform a range scan for a set of records in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><B>scan(String, String, int, Set&lt;String&gt;, Vector&lt;HashMap&lt;String, String&gt;&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>Perform a range scan for a set of records in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><B>scan(String, String, int, Set&lt;String&gt;, Vector&lt;HashMap&lt;String, String&gt;&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\">MongoDbClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/DB.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><B>scan(String, String, int, Set&lt;String&gt;, Vector&lt;HashMap&lt;String, String&gt;&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>\n<DD>Perform a range scan for a set of records in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/DBWrapper.html#scan(java.lang.String, java.lang.String, int, java.util.Set, java.util.Vector)\"><B>scan(String, String, int, Set&lt;String&gt;, Vector&lt;HashMap&lt;String, String&gt;&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\">DBWrapper</A>\n<DD>Perform a range scan for a set of records in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#SCAN_LENGTH_DISTRIBUTION_PROPERTY\"><B>SCAN_LENGTH_DISTRIBUTION_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The name of the property for the scan length distribution.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#SCAN_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT\"><B>SCAN_LENGTH_DISTRIBUTION_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The default max scan length.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#SCAN_PROPORTION_PROPERTY\"><B>SCAN_PROPORTION_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The name of the property for the proportion of transactions that are scans.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#SCAN_PROPORTION_PROPERTY_DEFAULT\"><B>SCAN_PROPORTION_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The default proportion of transactions that are scans.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>ScrambledZipfianGenerator</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/generator/package-summary.html\">com.yahoo.ycsb.generator</A><DD>A generator of a zipfian distribution.<DT><A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html#ScrambledZipfianGenerator(long)\"><B>ScrambledZipfianGenerator(long)</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ScrambledZipfianGenerator</A>\n<DD>Create a zipfian generator for the specified number of items.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html#ScrambledZipfianGenerator(long, long)\"><B>ScrambledZipfianGenerator(long, long)</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ScrambledZipfianGenerator</A>\n<DD>Create a zipfian generator for items between min and max.\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#ServerError\"><B>ServerError</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/generator/IntegerGenerator.html#setLastInt(int)\"><B>setLastInt(int)</B></A> - \nMethod in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">IntegerGenerator</A>\n<DD>Set the last value generated.\n<DT><A HREF=\"./com/yahoo/ycsb/DB.html#setProperties(java.util.Properties)\"><B>setProperties(Properties)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>\n<DD>Set the properties for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/DBWrapper.html#setProperties(java.util.Properties)\"><B>setProperties(Properties)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\">DBWrapper</A>\n<DD>Set the properties for this DB.\n<DT><A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html#setProperties(java.util.Properties)\"><B>setProperties(Properties)</B></A> - \nStatic method in class com.yahoo.ycsb.measurements.<A HREF=\"./com/yahoo/ycsb/measurements/Measurements.html\" title=\"class in com.yahoo.ycsb.measurements\">Measurements</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/BasicDB.html#SIMULATE_DELAY\"><B>SIMULATE_DELAY</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/BasicDB.html#SIMULATE_DELAY_DEFAULT\"><B>SIMULATE_DELAY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/generator/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>SkewedLatestGenerator</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/generator/package-summary.html\">com.yahoo.ycsb.generator</A><DD>Generate a popularity distribution of items, skewed to favor recent items significantly more than older items.<DT><A HREF=\"./com/yahoo/ycsb/generator/SkewedLatestGenerator.html#SkewedLatestGenerator(com.yahoo.ycsb.generator.CounterGenerator)\"><B>SkewedLatestGenerator(CounterGenerator)</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">SkewedLatestGenerator</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_T_\"><!-- --></A><H2>\n<B>T</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#tableLock\"><B>tableLock</B></A> - \nStatic variable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#TABLENAME\"><B>TABLENAME</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The name of the database table to run queries against.\n</DL>\n<HR>\n<A NAME=\"_U_\"><!-- --></A><H2>\n<B>U</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/generator/UniformGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>UniformGenerator</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/generator/package-summary.html\">com.yahoo.ycsb.generator</A><DD>An expression that generates a random integer in the specified range<DT><A HREF=\"./com/yahoo/ycsb/generator/UniformGenerator.html#UniformGenerator(java.util.Vector)\"><B>UniformGenerator(Vector&lt;String&gt;)</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/UniformGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">UniformGenerator</A>\n<DD>Creates a generator that will return strings from the specified set uniformly randomly\n<DT><A HREF=\"./com/yahoo/ycsb/generator/UniformIntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>UniformIntegerGenerator</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/generator/package-summary.html\">com.yahoo.ycsb.generator</A><DD>Generates integers randomly uniform from an interval.<DT><A HREF=\"./com/yahoo/ycsb/generator/UniformIntegerGenerator.html#UniformIntegerGenerator(int, int)\"><B>UniformIntegerGenerator(int, int)</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/UniformIntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">UniformIntegerGenerator</A>\n<DD>Creates a generator that will return integers uniformly randomly from the interval [lb,ub] inclusive (that is, lb and ub are possible values)\n<DT><A HREF=\"./com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\"><B>UnknownDBException</B></A> - Exception in <A HREF=\"./com/yahoo/ycsb/package-summary.html\">com.yahoo.ycsb</A><DD>Could not create the specified DB.<DT><A HREF=\"./com/yahoo/ycsb/UnknownDBException.html#UnknownDBException(java.lang.String)\"><B>UnknownDBException(String)</B></A> - \nConstructor for exception com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\">UnknownDBException</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/UnknownDBException.html#UnknownDBException()\"><B>UnknownDBException()</B></A> - \nConstructor for exception com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\">UnknownDBException</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/UnknownDBException.html#UnknownDBException(java.lang.String, java.lang.Throwable)\"><B>UnknownDBException(String, Throwable)</B></A> - \nConstructor for exception com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\">UnknownDBException</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/UnknownDBException.html#UnknownDBException(java.lang.Throwable)\"><B>UnknownDBException(Throwable)</B></A> - \nConstructor for exception com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\">UnknownDBException</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/BasicDB.html#update(java.lang.String, java.lang.String, java.util.HashMap)\"><B>update(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A>\n<DD>Update a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html#update(java.lang.String, java.lang.String, java.util.HashMap)\"><B>update(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient5</A>\n<DD>Update a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html#update(java.lang.String, java.lang.String, java.util.HashMap)\"><B>update(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient6</A>\n<DD>Update a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html#update(java.lang.String, java.lang.String, java.util.HashMap)\"><B>update(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\">CassandraClient7</A>\n<DD>Update a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#update(java.lang.String, java.lang.String, java.util.HashMap)\"><B>update(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>Update a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html#update(java.lang.String, java.lang.String, java.util.HashMap)\"><B>update(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\">MongoDbClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/DB.html#update(java.lang.String, java.lang.String, java.util.HashMap)\"><B>update(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\">DB</A>\n<DD>Update a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/DBWrapper.html#update(java.lang.String, java.lang.String, java.util.HashMap)\"><B>update(String, String, HashMap&lt;String, String&gt;)</B></A> - \nMethod in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\">DBWrapper</A>\n<DD>Update a record in the database.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#UPDATE_PROPORTION_PROPERTY\"><B>UPDATE_PROPORTION_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The name of the property for the proportion of transactions that are updates.\n<DT><A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html#UPDATE_PROPORTION_PROPERTY_DEFAULT\"><B>UPDATE_PROPORTION_PROPERTY_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.workloads.<A HREF=\"./com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\">CoreWorkload</A>\n<DD>The default proportion of transactions that are updates.\n<DT><A HREF=\"./com/yahoo/ycsb/Client.html#usageMessage()\"><B>usageMessage()</B></A> - \nStatic method in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\">Client</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/CommandLine.html#usageMessage()\"><B>usageMessage()</B></A> - \nStatic method in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\">CommandLine</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\"><B>Utils</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/package-summary.html\">com.yahoo.ycsb</A><DD>Utility functions.<DT><A HREF=\"./com/yahoo/ycsb/Utils.html#Utils()\"><B>Utils()</B></A> - \nConstructor for class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\">Utils</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_V_\"><!-- --></A><H2>\n<B>V</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/BasicDB.html#VERBOSE\"><B>VERBOSE</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/BasicDB.html#VERBOSE_DEFAULT\"><B>VERBOSE_DEFAULT</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\">BasicDB</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_W_\"><!-- --></A><H2>\n<B>W</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\"><B>Workload</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/package-summary.html\">com.yahoo.ycsb</A><DD>One experiment scenario.<DT><A HREF=\"./com/yahoo/ycsb/Workload.html#Workload()\"><B>Workload()</B></A> - \nConstructor for class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\">Workload</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/Client.html#WORKLOAD_PROPERTY\"><B>WORKLOAD_PROPERTY</B></A> - \nStatic variable in class com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\">Client</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\"><B>WorkloadException</B></A> - Exception in <A HREF=\"./com/yahoo/ycsb/package-summary.html\">com.yahoo.ycsb</A><DD>The workload tried to do something bad.<DT><A HREF=\"./com/yahoo/ycsb/WorkloadException.html#WorkloadException(java.lang.String)\"><B>WorkloadException(String)</B></A> - \nConstructor for exception com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/WorkloadException.html#WorkloadException()\"><B>WorkloadException()</B></A> - \nConstructor for exception com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/WorkloadException.html#WorkloadException(java.lang.String, java.lang.Throwable)\"><B>WorkloadException(String, Throwable)</B></A> - \nConstructor for exception com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/WorkloadException.html#WorkloadException(java.lang.Throwable)\"><B>WorkloadException(Throwable)</B></A> - \nConstructor for exception com.yahoo.ycsb.<A HREF=\"./com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">WorkloadException</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A NAME=\"_Z_\"><!-- --></A><H2>\n<B>Z</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html#ZETAN\"><B>ZETAN</B></A> - \nStatic variable in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ScrambledZipfianGenerator</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html#ZIPFIAN_CONSTANT\"><B>ZIPFIAN_CONSTANT</B></A> - \nStatic variable in class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ZipfianGenerator</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>ZipfianGenerator</B></A> - Class in <A HREF=\"./com/yahoo/ycsb/generator/package-summary.html\">com.yahoo.ycsb.generator</A><DD>A generator of a zipfian distribution.<DT><A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html#ZipfianGenerator(long)\"><B>ZipfianGenerator(long)</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ZipfianGenerator</A>\n<DD>Create a zipfian generator for the specified number of items.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html#ZipfianGenerator(long, long)\"><B>ZipfianGenerator(long, long)</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ZipfianGenerator</A>\n<DD>Create a zipfian generator for items between min and max.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html#ZipfianGenerator(long, double)\"><B>ZipfianGenerator(long, double)</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ZipfianGenerator</A>\n<DD>Create a zipfian generator for the specified number of items using the specified zipfian constant.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html#ZipfianGenerator(long, long, double)\"><B>ZipfianGenerator(long, long, double)</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ZipfianGenerator</A>\n<DD>Create a zipfian generator for items between min and max (inclusive) for the specified zipfian constant.\n<DT><A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html#ZipfianGenerator(long, long, double, double)\"><B>ZipfianGenerator(long, long, double, double)</B></A> - \nConstructor for class com.yahoo.ycsb.generator.<A HREF=\"./com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\">ZipfianGenerator</A>\n<DD>Create a zipfian generator for items between min and max (inclusive) for the specified zipfian constant, using the precomputed value of zeta.\n</DL>\n<HR>\n<A NAME=\"___\"><!-- --></A><H2>\n<B>_</B></H2>\n<DL>\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#_columnFamily\"><B>_columnFamily</B></A> - \nVariable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#_columnFamilyBytes\"><B>_columnFamilyBytes</B></A> - \nVariable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#_debug\"><B>_debug</B></A> - \nVariable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#_hTable\"><B>_hTable</B></A> - \nVariable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>&nbsp;\n<DT><A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html#_table\"><B>_table</B></A> - \nVariable in class com.yahoo.ycsb.db.<A HREF=\"./com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\">HBaseClient</A>\n<DD>&nbsp;\n</DL>\n<HR>\n<A HREF=\"#_A_\">A</A> <A HREF=\"#_B_\">B</A> <A HREF=\"#_C_\">C</A> <A HREF=\"#_D_\">D</A> <A HREF=\"#_E_\">E</A> <A HREF=\"#_F_\">F</A> <A HREF=\"#_G_\">G</A> <A HREF=\"#_H_\">H</A> <A HREF=\"#_I_\">I</A> <A HREF=\"#_L_\">L</A> <A HREF=\"#_M_\">M</A> <A HREF=\"#_N_\">N</A> <A HREF=\"#_O_\">O</A> <A HREF=\"#_P_\">P</A> <A HREF=\"#_R_\">R</A> <A HREF=\"#_S_\">S</A> <A HREF=\"#_T_\">T</A> <A HREF=\"#_U_\">U</A> <A HREF=\"#_V_\">V</A> <A HREF=\"#_W_\">W</A> <A HREF=\"#_Z_\">Z</A> <A HREF=\"#___\">_</A> \n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"./overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"./overview-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"./deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Index</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"./help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"./index.html?index-all.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"index-all.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"./allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"./allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/index.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Frameset//EN\" \"http://www.w3.org/TR/html4/frameset.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc on Wed May 12 14:51:53 PDT 2010-->\n<TITLE>\nGenerated Documentation (Untitled)\n</TITLE>\n<SCRIPT type=\"text/javascript\">\n    targetPage = \"\" + window.location.search;\n    if (targetPage != \"\" && targetPage != \"undefined\")\n        targetPage = targetPage.substring(1);\n    if (targetPage.indexOf(\":\") != -1)\n        targetPage = \"undefined\";\n    function loadFrames() {\n        if (targetPage != \"\" && targetPage != \"undefined\")\n             top.classFrame.location = top.targetPage;\n    }\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n</HEAD>\n<FRAMESET cols=\"20%,80%\" title=\"\" onLoad=\"top.loadFrames()\">\n<FRAMESET rows=\"30%,70%\" title=\"\" onLoad=\"top.loadFrames()\">\n<FRAME src=\"overview-frame.html\" name=\"packageListFrame\" title=\"All Packages\">\n<FRAME src=\"allclasses-frame.html\" name=\"packageFrame\" title=\"All classes and interfaces (except non-static nested types)\">\n</FRAMESET>\n<FRAME src=\"overview-summary.html\" name=\"classFrame\" title=\"Package, class and interface descriptions\" scrolling=\"yes\">\n<NOFRAMES>\n<H2>\nFrame Alert</H2>\n\n<P>\nThis document is designed to be viewed using the frames feature. If you see this message, you are using a non-frame-capable web client.\n<BR>\nLink to<A HREF=\"overview-summary.html\">Non-frame version.</A>\n</NOFRAMES>\n</FRAMESET>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/overview-frame.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\nOverview List\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"stylesheet.css\" TITLE=\"Style\">\n\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\">\n\n<TABLE BORDER=\"0\" WIDTH=\"100%\" SUMMARY=\"\">\n<TR>\n<TH ALIGN=\"left\" NOWRAP><FONT size=\"+1\" CLASS=\"FrameTitleFont\">\n<B></B></FONT></TH>\n</TR>\n</TABLE>\n\n<TABLE BORDER=\"0\" WIDTH=\"100%\" SUMMARY=\"\">\n<TR>\n<TD NOWRAP><FONT CLASS=\"FrameItemFont\"><A HREF=\"allclasses-frame.html\" target=\"packageFrame\">All Classes</A></FONT>\n<P>\n<FONT size=\"+1\" CLASS=\"FrameHeadingFont\">\nPackages</FONT>\n<BR>\n<FONT CLASS=\"FrameItemFont\"><A HREF=\"com/yahoo/ycsb/package-frame.html\" target=\"packageFrame\">com.yahoo.ycsb</A></FONT>\n<BR>\n<FONT CLASS=\"FrameItemFont\"><A HREF=\"com/yahoo/ycsb/db/package-frame.html\" target=\"packageFrame\">com.yahoo.ycsb.db</A></FONT>\n<BR>\n<FONT CLASS=\"FrameItemFont\"><A HREF=\"com/yahoo/ycsb/generator/package-frame.html\" target=\"packageFrame\">com.yahoo.ycsb.generator</A></FONT>\n<BR>\n<FONT CLASS=\"FrameItemFont\"><A HREF=\"com/yahoo/ycsb/measurements/package-frame.html\" target=\"packageFrame\">com.yahoo.ycsb.measurements</A></FONT>\n<BR>\n<FONT CLASS=\"FrameItemFont\"><A HREF=\"com/yahoo/ycsb/workloads/package-frame.html\" target=\"packageFrame\">com.yahoo.ycsb.workloads</A></FONT>\n<BR>\n</TD>\n</TR>\n</TABLE>\n\n<P>\n&nbsp;\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/overview-summary.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\nOverview\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"Overview\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Overview</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"index.html?overview-summary.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"overview-summary.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Packages</B></FONT></TH>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"20%\"><B><A HREF=\"com/yahoo/ycsb/package-summary.html\">com.yahoo.ycsb</A></B></TD>\n<TD>&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"20%\"><B><A HREF=\"com/yahoo/ycsb/db/package-summary.html\">com.yahoo.ycsb.db</A></B></TD>\n<TD>&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"20%\"><B><A HREF=\"com/yahoo/ycsb/generator/package-summary.html\">com.yahoo.ycsb.generator</A></B></TD>\n<TD>&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"20%\"><B><A HREF=\"com/yahoo/ycsb/measurements/package-summary.html\">com.yahoo.ycsb.measurements</A></B></TD>\n<TD>&nbsp;</TD>\n</TR>\n<TR BGCOLOR=\"white\" CLASS=\"TableRowColor\">\n<TD WIDTH=\"20%\"><B><A HREF=\"com/yahoo/ycsb/workloads/package-summary.html\">com.yahoo.ycsb.workloads</A></B></TD>\n<TD>&nbsp;</TD>\n</TR>\n</TABLE>\n\n<P>\n&nbsp;<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Overview</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"index.html?overview-summary.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"overview-summary.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/overview-tree.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\nClass Hierarchy\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"Class Hierarchy\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Tree</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"index.html?overview-tree.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"overview-tree.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<CENTER>\n<H2>\nHierarchy For All Packages</H2>\n</CENTER>\n<DL>\n<DT><B>Package Hierarchies:</B><DD><A HREF=\"com/yahoo/ycsb/package-tree.html\">com.yahoo.ycsb</A>, <A HREF=\"com/yahoo/ycsb/db/package-tree.html\">com.yahoo.ycsb.db</A>, <A HREF=\"com/yahoo/ycsb/generator/package-tree.html\">com.yahoo.ycsb.generator</A>, <A HREF=\"com/yahoo/ycsb/measurements/package-tree.html\">com.yahoo.ycsb.measurements</A>, <A HREF=\"com/yahoo/ycsb/workloads/package-tree.html\">com.yahoo.ycsb.workloads</A></DL>\n<HR>\n<H2>\nClass Hierarchy\n</H2>\n<UL>\n<LI TYPE=\"circle\">java.lang.Object<UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/Client.html\" title=\"class in com.yahoo.ycsb\"><B>Client</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/CommandLine.html\" title=\"class in com.yahoo.ycsb\"><B>CommandLine</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/DB.html\" title=\"class in com.yahoo.ycsb\"><B>DB</B></A><UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/BasicDB.html\" title=\"class in com.yahoo.ycsb\"><B>BasicDB</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.db.<A HREF=\"com/yahoo/ycsb/db/CassandraClient5.html\" title=\"class in com.yahoo.ycsb.db\"><B>CassandraClient5</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.db.<A HREF=\"com/yahoo/ycsb/db/CassandraClient6.html\" title=\"class in com.yahoo.ycsb.db\"><B>CassandraClient6</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.db.<A HREF=\"com/yahoo/ycsb/db/CassandraClient7.html\" title=\"class in com.yahoo.ycsb.db\"><B>CassandraClient7</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/DBWrapper.html\" title=\"class in com.yahoo.ycsb\"><B>DBWrapper</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.db.<A HREF=\"com/yahoo/ycsb/db/HBaseClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>HBaseClient</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.db.<A HREF=\"com/yahoo/ycsb/db/MongoDbClient.html\" title=\"class in com.yahoo.ycsb.db\"><B>MongoDbClient</B></A></UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/DBFactory.html\" title=\"class in com.yahoo.ycsb\"><B>DBFactory</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.generator.<A HREF=\"com/yahoo/ycsb/generator/Generator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>Generator</B></A><UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.generator.<A HREF=\"com/yahoo/ycsb/generator/DiscreteGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>DiscreteGenerator</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.generator.<A HREF=\"com/yahoo/ycsb/generator/IntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>IntegerGenerator</B></A><UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.generator.<A HREF=\"com/yahoo/ycsb/generator/CounterGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>CounterGenerator</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.generator.<A HREF=\"com/yahoo/ycsb/generator/ScrambledZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>ScrambledZipfianGenerator</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.generator.<A HREF=\"com/yahoo/ycsb/generator/SkewedLatestGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>SkewedLatestGenerator</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.generator.<A HREF=\"com/yahoo/ycsb/generator/UniformIntegerGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>UniformIntegerGenerator</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.generator.<A HREF=\"com/yahoo/ycsb/generator/ZipfianGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>ZipfianGenerator</B></A></UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.generator.<A HREF=\"com/yahoo/ycsb/generator/UniformGenerator.html\" title=\"class in com.yahoo.ycsb.generator\"><B>UniformGenerator</B></A></UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.measurements.<A HREF=\"com/yahoo/ycsb/measurements/Measurements.html\" title=\"class in com.yahoo.ycsb.measurements\"><B>Measurements</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.measurements.<A HREF=\"com/yahoo/ycsb/measurements/OneMeasurement.html\" title=\"class in com.yahoo.ycsb.measurements\"><B>OneMeasurement</B></A><UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.measurements.<A HREF=\"com/yahoo/ycsb/measurements/OneMeasurementHistogram.html\" title=\"class in com.yahoo.ycsb.measurements\"><B>OneMeasurementHistogram</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.measurements.<A HREF=\"com/yahoo/ycsb/measurements/OneMeasurementTimeSeries.html\" title=\"class in com.yahoo.ycsb.measurements\"><B>OneMeasurementTimeSeries</B></A></UL>\n<LI TYPE=\"circle\">java.lang.Throwable (implements java.io.Serializable)\n<UL>\n<LI TYPE=\"circle\">java.lang.Exception<UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\"><B>DBException</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\"><B>UnknownDBException</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\"><B>WorkloadException</B></A></UL>\n</UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/Utils.html\" title=\"class in com.yahoo.ycsb\"><B>Utils</B></A><LI TYPE=\"circle\">com.yahoo.ycsb.<A HREF=\"com/yahoo/ycsb/Workload.html\" title=\"class in com.yahoo.ycsb\"><B>Workload</B></A><UL>\n<LI TYPE=\"circle\">com.yahoo.ycsb.workloads.<A HREF=\"com/yahoo/ycsb/workloads/CoreWorkload.html\" title=\"class in com.yahoo.ycsb.workloads\"><B>CoreWorkload</B></A></UL>\n</UL>\n</UL>\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#FFFFFF\" CLASS=\"NavBarCell1Rev\"> &nbsp;<FONT CLASS=\"NavBarFont1Rev\"><B>Tree</B></FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"index.html?overview-tree.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"overview-tree.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/package-list",
    "content": "com.yahoo.ycsb\ncom.yahoo.ycsb.db\ncom.yahoo.ycsb.generator\ncom.yahoo.ycsb.measurements\ncom.yahoo.ycsb.workloads\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/serialized-form.html",
    "content": "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n<!--NewPage-->\n<HTML>\n<HEAD>\n<!-- Generated by javadoc (build 1.6.0_03) on Wed May 12 14:51:53 PDT 2010 -->\n<TITLE>\nSerialized Form\n</TITLE>\n\n<META NAME=\"date\" CONTENT=\"2010-05-12\">\n\n<LINK REL =\"stylesheet\" TYPE=\"text/css\" HREF=\"stylesheet.css\" TITLE=\"Style\">\n\n<SCRIPT type=\"text/javascript\">\nfunction windowTitle()\n{\n    if (location.href.indexOf('is-external=true') == -1) {\n        parent.document.title=\"Serialized Form\";\n    }\n}\n</SCRIPT>\n<NOSCRIPT>\n</NOSCRIPT>\n\n</HEAD>\n\n<BODY BGCOLOR=\"white\" onload=\"windowTitle();\">\n<HR>\n\n\n<!-- ========= START OF TOP NAVBAR ======= -->\n<A NAME=\"navbar_top\"><!-- --></A>\n<A HREF=\"#skip-navbar_top\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_top_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"index.html?serialized-form.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"serialized-form.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_top\"></A>\n<!-- ========= END OF TOP NAVBAR ========= -->\n\n<HR>\n<CENTER>\n<H1>\nSerialized Form</H1>\n</CENTER>\n<HR SIZE=\"4\" NOSHADE>\n\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"center\"><FONT SIZE=\"+2\">\n<B>Package</B> <B>com.yahoo.ycsb</B></FONT></TH>\n</TR>\n</TABLE>\n\n<P>\n<A NAME=\"com.yahoo.ycsb.DBException\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Class <A HREF=\"com/yahoo/ycsb/DBException.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.DBException</A> extends java.lang.Exception implements Serializable</B></FONT></TH>\n</TR>\n</TABLE>\n\n<P>\n<B>serialVersionUID:&nbsp;</B>6646883591588721475L\n\n<P>\n\n<P>\n<A NAME=\"com.yahoo.ycsb.UnknownDBException\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Class <A HREF=\"com/yahoo/ycsb/UnknownDBException.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.UnknownDBException</A> extends java.lang.Exception implements Serializable</B></FONT></TH>\n</TR>\n</TABLE>\n\n<P>\n<B>serialVersionUID:&nbsp;</B>459099842269616836L\n\n<P>\n\n<P>\n<A NAME=\"com.yahoo.ycsb.WorkloadException\"><!-- --></A>\n<TABLE BORDER=\"1\" WIDTH=\"100%\" CELLPADDING=\"3\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR BGCOLOR=\"#CCCCFF\" CLASS=\"TableSubHeadingColor\">\n<TH ALIGN=\"left\" COLSPAN=\"2\"><FONT SIZE=\"+2\">\n<B>Class <A HREF=\"com/yahoo/ycsb/WorkloadException.html\" title=\"class in com.yahoo.ycsb\">com.yahoo.ycsb.WorkloadException</A> extends java.lang.Exception implements Serializable</B></FONT></TH>\n</TR>\n</TABLE>\n\n<P>\n<B>serialVersionUID:&nbsp;</B>8844396756042772132L\n\n<P>\n\n<P>\n<HR>\n\n\n<!-- ======= START OF BOTTOM NAVBAR ====== -->\n<A NAME=\"navbar_bottom\"><!-- --></A>\n<A HREF=\"#skip-navbar_bottom\" title=\"Skip navigation links\"></A>\n<TABLE BORDER=\"0\" WIDTH=\"100%\" CELLPADDING=\"1\" CELLSPACING=\"0\" SUMMARY=\"\">\n<TR>\n<TD COLSPAN=2 BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">\n<A NAME=\"navbar_bottom_firstrow\"><!-- --></A>\n<TABLE BORDER=\"0\" CELLPADDING=\"0\" CELLSPACING=\"3\" SUMMARY=\"\">\n  <TR ALIGN=\"center\" VALIGN=\"top\">\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-summary.html\"><FONT CLASS=\"NavBarFont1\"><B>Overview</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Package</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <FONT CLASS=\"NavBarFont1\">Class</FONT>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"overview-tree.html\"><FONT CLASS=\"NavBarFont1\"><B>Tree</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"deprecated-list.html\"><FONT CLASS=\"NavBarFont1\"><B>Deprecated</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"index-all.html\"><FONT CLASS=\"NavBarFont1\"><B>Index</B></FONT></A>&nbsp;</TD>\n  <TD BGCOLOR=\"#EEEEFF\" CLASS=\"NavBarCell1\">    <A HREF=\"help-doc.html\"><FONT CLASS=\"NavBarFont1\"><B>Help</B></FONT></A>&nbsp;</TD>\n  </TR>\n</TABLE>\n</TD>\n<TD ALIGN=\"right\" VALIGN=\"top\" ROWSPAN=3><EM>\n</EM>\n</TD>\n</TR>\n\n<TR>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n&nbsp;PREV&nbsp;\n&nbsp;NEXT</FONT></TD>\n<TD BGCOLOR=\"white\" CLASS=\"NavBarCell2\"><FONT SIZE=\"-2\">\n  <A HREF=\"index.html?serialized-form.html\" target=\"_top\"><B>FRAMES</B></A>  &nbsp;\n&nbsp;<A HREF=\"serialized-form.html\" target=\"_top\"><B>NO FRAMES</B></A>  &nbsp;\n&nbsp;<SCRIPT type=\"text/javascript\">\n  <!--\n  if(window==top) {\n    document.writeln('<A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>');\n  }\n  //-->\n</SCRIPT>\n<NOSCRIPT>\n  <A HREF=\"allclasses-noframe.html\"><B>All Classes</B></A>\n</NOSCRIPT>\n\n\n</FONT></TD>\n</TR>\n</TABLE>\n<A NAME=\"skip-navbar_bottom\"></A>\n<!-- ======== END OF BOTTOM NAVBAR ======= -->\n\n<HR>\n\n</BODY>\n</HTML>\n"
  },
  {
    "path": "contrib/ycsb/doc/javadoc/stylesheet.css",
    "content": "/* Javadoc style sheet */\n\n/* Define colors, fonts and other style attributes here to override the defaults */\n\n/* Page background color */\nbody { background-color: #FFFFFF; color:#000000 }\n\n/* Headings */\nh1 { font-size: 145% }\n\n/* Table colors */\n.TableHeadingColor     { background: #CCCCFF; color:#000000 } /* Dark mauve */\n.TableSubHeadingColor  { background: #EEEEFF; color:#000000 } /* Light mauve */\n.TableRowColor         { background: #FFFFFF; color:#000000 } /* White */\n\n/* Font used in left-hand frame lists */\n.FrameTitleFont   { font-size: 100%; font-family: Helvetica, Arial, sans-serif; color:#000000 }\n.FrameHeadingFont { font-size:  90%; font-family: Helvetica, Arial, sans-serif; color:#000000 }\n.FrameItemFont    { font-size:  90%; font-family: Helvetica, Arial, sans-serif; color:#000000 }\n\n/* Navigation bar fonts and colors */\n.NavBarCell1    { background-color:#EEEEFF; color:#000000} /* Light mauve */\n.NavBarCell1Rev { background-color:#00008B; color:#FFFFFF} /* Dark Blue */\n.NavBarFont1    { font-family: Arial, Helvetica, sans-serif; color:#000000;color:#000000;}\n.NavBarFont1Rev { font-family: Arial, Helvetica, sans-serif; color:#FFFFFF;color:#FFFFFF;}\n\n.NavBarCell2    { font-family: Arial, Helvetica, sans-serif; background-color:#FFFFFF; color:#000000}\n.NavBarCell3    { font-family: Arial, Helvetica, sans-serif; background-color:#FFFFFF; color:#000000}\n\n"
  },
  {
    "path": "contrib/ycsb/doc/parallelclients.html",
    "content": "<HTML>\r\n<HEAD>\r\n<TITLE>YCSB - Parallel clients</TITLE>\r\n</HEAD>\r\n<BODY>\r\n<H1><img src=\"images/ycsb.jpg\" width=150> Yahoo! Cloud Serving Benchmark</H1>\r\n<H3>Version 0.1.2</H3>\r\n<HR>\r\n<A HREF=\"index.html\">Home</A> - <A href=\"coreworkloads.html\">Core workloads</A> - <a href=\"tipsfaq.html\">Tips and FAQ</A>\r\n<HR>\r\n<H2>Running multiple clients in parallel</h2>\r\nIt is straightforward to run the transaction phase of the workload from multiple servers - just start up clients on different servers, each running the same workload. Each client will\r\nproduce performance statistics when it is done, and you'll have to aggregate these individual files into a single set of results.\r\n<P>\r\nIn some cases it makes sense to load the database using multiple servers. In this case, you will want to partition the records to be loaded among the clients. Normally, YCSB just loads\r\nall of the records (as defined by the recordcount property). However, if you want to partition the load you need to additionally specify two other properties for each client:\r\n<UL>\r\n<LI><b>insertstart</b>: The index of the record to start at.\r\n<LI><b>insertcount</b>: The number of records to insert.\r\n</UL>\r\nThese properties can be specified in a property file or on the command line using the -p option.\r\n<P>\r\nFor example, imagine you want to load 100 million records (so recordcount=100000000). Imagine you want to load with four clients. For the first client:\r\n<pre>\r\ninsertstart=0\r\ninsertcount=25000000\r\n</pre>\r\nFor the second client:\r\n<pre>\r\ninsertstart=25000000\r\ninsertcount=25000000\r\n</pre>\r\nFor the third client:\r\n<pre>\r\ninsertstart=50000000\r\ninsertcount=25000000\r\n</pre>\r\nAnd for the fourth client:\r\n<pre>\r\ninsertstart=75000000\r\ninsertcount=25000000\r\n</pre>\r\n<HR>\r\nYCSB - Yahoo! Research - Contact cooperb@yahoo-inc.com.\r\n</body>\r\n</html>\r\n"
  },
  {
    "path": "contrib/ycsb/doc/tipsfaq.html",
    "content": "<HTML>\r\n<HEAD>\r\n<TITLE>YCSB - Tips and FAQ</TITLE>\r\n</HEAD>\r\n<BODY>\r\n<H1><img src=\"images/ycsb.jpg\" width=150> Yahoo! Cloud Serving Benchmark</H1>\r\n<H3>Version 0.1.2</H3>\r\n<HR>\r\n<A HREF=\"index.html\">Home</A> - <A href=\"coreworkloads.html\">Core workloads</A> - <a href=\"tipsfaq.html\">Tips and FAQ</A>\r\n<HR>\r\n<H2>Tips</h2>\r\n<B>Tip 1 - Carefully adjust the number of threads</B>\r\n<P>\r\nThe number of threads determines how much workload you can generate against the database. Imagine that you are trying to run a test with 10,000 operations per second, \r\nbut you are only achieving 8,000 operations per second. Is this because the database can't keep up with the load? Not necessarily. Imagine that you are running with 100\r\nclient threads (e.g. \"-threads 100\") and each operation is taking 12 milliseconds on average. Each thread will only be able to generate 83 operations per second, because each\r\nthread operates sequentially. Over 100 threads, your client will only generate 8300 operations per second, even if the database can support more. Increasing the number of threads\r\nensures there are enough parallel clients hitting the database so that the database, not the client, is the bottleneck.\r\n<P>\r\nTo calculate the number of threads needed, you should have some idea of the expected latency. For example, at 10,000 operations per second, we might expect the database\r\nto have a latency of 10-30 milliseconds on average. So you to generate 10,000 operations per second, you will need (Ops per sec / (1000 / avg latency in ms) ), or (10000/(1000/30))=300 threads.\r\nIn fact, to be conservative, you might consider having 400 threads. Although this is a lot of threads, each thread will spend most of its time waiting for the database to respond,\r\nso the context switching overhead will be low. \r\n<P>\r\nExperiment with increasing the number of threads, especially if you find you are not reaching your target throughput. Eventually, of course, you will saturate the database\r\nand there will be no way to increase the number of threads to get more throughput (in fact, increasing the number of client threads may make things worse) but you need to have \r\nenough threads to ensure it is the database, not the client, that is the bottleneck.\r\n<HR>\r\nYCSB - Yahoo! Research - Contact cooperb@yahoo-inc.com.\r\n</BODY>\r\n</HTML>\r\n"
  },
  {
    "path": "contrib/ycsb/doc/workload.html",
    "content": "<HTML>\r\n<HEAD>\r\n<TITLE>YCSB - Implementing new workloads</TITLE>\r\n</HEAD>\r\n<BODY>\r\n<H1><img src=\"images/ycsb.jpg\" width=150> Yahoo! Cloud Serving Benchmark</H1>\r\n<H3>Version 0.1.2</H3>\r\n<HR>\r\n<A HREF=\"index.html\">Home</A> - <A href=\"coreworkloads.html\">Core workloads</A> - <a href=\"tipsfaq.html\">Tips and FAQ</A>\r\n<HR>\r\n<H2>Implementing new workloads - overview</h2>\r\nA workload represents the load that a given application will put on the database system. For benchmarking purposes, we must define\r\nworkloads that are relatively simple compared to real applications, so that we can better reason about the benchmarking results\r\nwe get. However, a workload should be detailed enough so that once we measure the database's performance, we know what kinds of applications\r\nmight experience similar performance.\r\n<p>\r\nIn the context of YCSB, a workload defines both a <b>data set</b>, which is a set of records to be loaded into the database, and a <b>transaction set</b>,\r\nwhich are the set of read and write operations against the database. Creating the transactions requires understanding the structure of the records, which\r\nis why both the data and the transactions must be defined in the workload.\r\n<P>\r\nFor a complete benchmark, multiple important (but distinct) workloads might be grouped together into a <i>workload package</I>. The CoreWorkload\r\npackage included with the YCSB client is an example of such a collection of workloads. \r\n<P>\r\nTypically a workload consists of two files:\r\n<UL>\r\n<LI>A java class which contains the code to create data records and generate transactions against them\r\n<LI>A parameter file which tunes the specifics of the workload\r\n</UL>\r\nFor example, a workload class file might generate some combination of read and update operations against the database. The parameter\r\nfile might specify whether the mix of reads and updates is 50/50, 80/20, etc.\r\n<P>\r\nThere are two ways to create a new workload or package of workloads.\r\n<P>\r\n<h3>Option 1: new parameter files</h3>\r\n<P>\r\nThe core workloads included with YCSB are defined by a set of parameter files (workloada, workloadb, etc.) You can create your own parameter file with new values\r\nfor the read/write mix, request distribution, etc. For example, the workloada file has the following contents:\r\n\r\n<pre>\r\nworkload=com.yahoo.ycsb.workloads.CoreWorkload\r\n\r\nreadallfields=false\r\n\r\nreadproportion=0.5\r\nupdateproportion=0.5\r\nscanproportion=0\r\ninsertproportion=0\r\n\r\nrequestdistribution=zipfian\r\n</pre>\r\n\r\nCreating a new file that changes any of these values will produce a new workload with different characteristics. The set of properties that can be specified is <a href=\"coreproperties.html\">here</a>.\r\n<P>\r\n<h3>Option 2: new java class</h3>\r\n<P>\r\nThe workload java class will be created by the YCSB Client at runtime, and will use an instance of the <a href=\"dblayer.html\">DB interface layer</A>\r\nto generate the actual operations against the database. Thus, the java class only needs to decide (based on settings in the parameter file) what records\r\nto create for the data set, and what reads, updates etc. to generate for the transaction phase. The YCSB Client will take care of creating the workload java class,\r\npassing it to a worker thread for executing, deciding how many records to create or how many operations to execute, and measuring the resulting \r\nperformance.\r\n<P>\r\nIf the CoreWorkload (or some other existing package) does not have the ability to generate the workload you desire, you can create a new workload java class.\r\nThis is done using the following steps:\r\n<H3>Step 1. Extend <a href=\"javadoc/com/yahoo/ycsb/Workload.html\">com.yahoo.ycsb.Workload</A></H3>\r\nThe base class of all workload classes is com.yahoo.ycsb.Workload. This is an abstract class, so you create a new workload that extends this base class. Your\r\nclass must have a public no-argument constructor, because the workload will be created in a factory using the no-argument constructor. The YCSB Client will\r\ncreate one Workload object for each worker thread, so if you run the Client with multiple threads, multiple workload objects will be created.\r\n<H3>Step 2. Write code to initialize your workload class</H3>\r\nThe parameter fill will be passed to the workload object after the constructor has been called, so if you are using any parameter properties, you must\r\nuse them to initialize your workload using either the init() or initThread() methods. \r\n<UL>\r\n<LI>init() - called once for all workload instances. Used to initialize any objects shared by all threads.\r\n<LI>initThread() - called once per workload instance in the context of the worker thread. Used to initialize any objects specific to a single Workload instance \r\nand single worker thread.\r\n</UL>\r\nIn either case, you can access the parameter properties using the Properties object passed in to both methods. These properties will include all properties defined\r\nin any property file passed to the YCSB Client or defined on the client command line.\r\n<H3>Step 3. Write any cleanup code</H3>\r\nThe cleanup() method is called once for all workload instances, after the workload has completed.\r\n<H3>Step 4. Define the records to be inserted</H3>\r\nThe YCSB Client will call the doInsert() method once for each record to be inserted into the database. So you should implement this method\r\nto create and insert a single record. The DB object you can use to perform the insert will be passed to the doInsert() method.\r\n<H3>Step 5. Define the transactions</H3>\r\nThe YCSB Client will call the doTransaction() method once for every transaction that is to be executed. So you should implement this method to execute\r\na single transaction, using the DB object passed in to access the database. Your implementation of this method can choose between different types of \r\ntransactions, and can make multiple calls to the DB interface layer. However, each invocation of the method should be a logical transaction. In particular, when you run the client,\r\nyou'll specify the number of operations to execute; if you request 1000 operations then doTransaction() will be executed 1000 times.\r\n<P>\r\nNote that you do not have to do any throttling of your transactions (or record insertions) to achieve the target throughput. The YCSB Client will do the throttling\r\nfor you.\r\n<P>\r\nNote also that it is allowable to insert records inside the doTransaction() method. You might do this if you wish the database to grow during the workload. In this case,\r\nthe initial dataset will be constructed using calls to the doInsert() method, while additional records would be inserted using calls to the doTransaction() method.\r\n<h3>Step 6 - Measure latency, if necessary</h3>\r\nThe YCSB client will automatically measure the latency and throughput of database operations, even for workloads that you define. However, the client will only measure\r\nthe latency of individual calls to the database, not of more complex transactions. Consider for example a workload that reads a record, modifies it, and writes\r\nthe changes back to the database. The YCSB client will automatically measure the latency of the read operation to the database; and separately will automatically measure the \r\nlatency of the update operation. However, if you would like to measure the latency of the entire read-modify-write transaction, you will need to add an additional timing step to your\r\ncode.\r\n<P>\r\nMeasurements are gathered using the Measurements.measure() call. There is a singleton instance of Measurements, which can be obtained using the \r\nMeasurements.getMeasurements() static method. For each metric you are measuring, you need to assign a string tag; this tag will label the resulting\r\naverage, min, max, histogram etc. measurements output by the tool at the end of the workload. For example, consider the following code:\r\n\r\n<pre>\r\nlong st=System.currentTimeMillis();\r\ndb.read(TABLENAME,keyname,fields,new HashMap<String,String>());\r\ndb.update(TABLENAME,keyname,values);\r\nlong en=System.currentTimeMillis();\r\nMeasurements.getMeasurements().measure(\"READ-MODIFY-WRITE\", (int)(en-st));\r\n</pre>\r\n\r\nIn this code, the calls to System.currentTimeMillis() are used to time the read and write transaction. Then, the call to measure() reports the latency to the \r\nmeasurement component. \r\n<p>\r\nUsing this pattern, your custom measurements will be gathered and aggregated using the same mechanism that is used to gather measurements for individual READ, UPDATE etc. operations.\r\n\r\n<h3>Step 7 - Use it with the YCSB Client</h3>\r\nMake sure that the classes for your implementation (or a jar containing those classes) are available on your CLASSPATH, as well as any libraries/jar files used\r\nby your implementation. Now, when you run the YCSB Client, specify the \"workload\" property to provide the fully qualified classname of your\r\nDB class. For example:\r\n\r\n<pre>\r\nworkload=com.foo.YourWorkloadClass\r\n</pre>\r\n<HR>\r\nYCSB - Yahoo! Research - Contact cooperb@yahoo-inc.com.\r\n</body>\r\n</html>\r\n"
  },
  {
    "path": "contrib/ycsb/pom.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<project xmlns=\"http://maven.apache.org/POM/4.0.0\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n  xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd\">\n  <modelVersion>4.0.0</modelVersion>\n\n  <groupId>com.yahoo.ycsb</groupId>\n  <artifactId>root</artifactId>\n  <version>0.1.4</version>\n  <packaging>pom</packaging>\n\n  <name>YCSB Root</name>\n\n  <description>\n    This is the top level project that builds, packages the core and all the DB bindings for YCSB infrastructure.\n  </description>\n  <dependencies>\n    <!-- voldemort -->\n    <dependency>\n      <groupId>checkstyle</groupId>\n      <artifactId>checkstyle</artifactId>\n      <version>5.0</version>\n    </dependency>\n    <dependency>\n      <groupId>org.jdom</groupId>\n      <artifactId>jdom</artifactId>\n      <version>1.1</version>\n    </dependency>\n    <dependency>\n      <groupId>com.google.collections</groupId>\n      <artifactId>google-collections</artifactId>\n      <version>1.0</version>\n    </dependency>\n    <!--\n    Nail down slf4j version to 1.6 so that it defaults to no-op logger.\n    http://www.slf4j.org/codes.html#StaticLoggerBinder\n    -->\n    <dependency>\n      <groupId>org.slf4j</groupId>\n      <artifactId>slf4j-api</artifactId>\n      <version>1.6.4</version>\n    </dependency>\n  </dependencies>\n \n  <!-- Properties Management -->\n  <properties>\n    <maven.assembly.version>2.2.1</maven.assembly.version>\n    <hbase.version>0.92.1</hbase.version>\n    <cassandra.version>0.7.0</cassandra.version>\n    <infinispan.version>7.1.0.CR1</infinispan.version>\n    <openjpa.jdbc.version>2.1.1</openjpa.jdbc.version>\n    <mapkeeper.version>1.0</mapkeeper.version>\n    <mongodb.version>2.8.0</mongodb.version>\n    <orientdb.version>1.0.1</orientdb.version>\n    <redis.version>2.0.0</redis.version>\n    <voldemort.version>0.81</voldemort.version>\n    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>\n    <thrift.version>0.8.0</thrift.version>\n    <hypertable.version>0.9.5.6</hypertable.version>\n    <scalaris.version>0.6.0</scalaris.version>\n  </properties>\n\n  <modules>\n    <!--module>build-tools</module-->\n    <module>core</module>\n    <module>scalaris</module>\n    <module>distribution</module>\n  </modules>\n\n  <build>\n  \t<pluginManagement>\n\t    <plugins>\n\t      <plugin>\n\t        <groupId>org.apache.maven.plugins</groupId>\n\t        <artifactId>maven-compiler-plugin</artifactId>\n\t        <version>2.3.2</version>\n\t        <configuration>\n\t          <source>1.6</source>\n\t          <target>1.6</target>\n\t        </configuration>\n\t      </plugin>\n\t      <plugin>\n\t        <groupId>org.apache.maven.plugins</groupId>\n\t        <artifactId>maven-checkstyle-plugin</artifactId>\n\t        <version>2.6</version>\n\t        <configuration>\n\t          <consoleOutput>true</consoleOutput>\n\t          <configLocation>checkstyle.xml</configLocation>\n\t        </configuration>\n\t        <executions>\n\t          <execution>\n\t          <id>validate</id>\n\t            <phase>validate</phase>\n\t            <goals>\n\t              <goal>checkstyle</goal>\n\t            </goals>\n\t          </execution>\n\t        </executions>\n\t      </plugin>\n\t    </plugins>\n\t</pluginManagement>\n  </build>\n</project>\n"
  },
  {
    "path": "contrib/ycsb/scalaris/.gitignore",
    "content": "/target/\n"
  },
  {
    "path": "contrib/ycsb/scalaris/README.md",
    "content": "Scalaris YCSB Client\n====================\n\nThe Scalaris client is not yet part of the official YCSB bundle but a fork of\nYCSB with the necessary implementation is available on GitHub.\n\nCompile\n-------\n\nCheck out the fork of YCSB with branch scalaris:\n\n\tgit clone git://github.com/mxm/YCSB.git -b scalaris\n  \nBuild with Maven:\n\n\tcd YCSB\n\tmvn package\n\nRun\n---\n\n\t./bin/ycsb load scalaris -P workloads/workloada\n\t./bin/ycsb run scalaris -P workloads/workloada\n\nTroubleshooting\n---------------\n\nIf you're experiencing problems, first make sure Scalaris is running. You can pass configuration parameters to YCBS. For instance, if your first Scalaris node is node1@localhost, the configuration looks as follows:\n\n\t./bin/ycsb load scalaris -P workloads/workloada -p scalaris.node=\"node1@localhost\"\n\nThe following parameters with their default values are available:\n\n\tscalaris.node = \"node1@localhost\"\n\tscalaris.name = \"YCSB\"\n\tscalaris.cookie = \"chocolate chip cookie\"\n"
  },
  {
    "path": "contrib/ycsb/scalaris/pom.xml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<project xmlns=\"http://maven.apache.org/POM/4.0.0\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd\">\n  <modelVersion>4.0.0</modelVersion>\n  <parent>\n     <groupId>com.yahoo.ycsb</groupId>\n     <artifactId>root</artifactId>\n     <version>0.1.4</version>\n  </parent>\n\n  <artifactId>scalaris-binding</artifactId>\n  <name>Scalaris DB Binding</name>\n  <packaging>jar</packaging>\n\n  <repositories>\n      <repository>\n          <id>scalaris-repo</id>\n          <url>https://scalaris-team.github.io/scalaris/maven</url>\n      </repository>\n  </repositories>\n\n  <dependencies>\n    <dependency>\n       <groupId>de.zib.scalaris</groupId>\n       <artifactId>java-api</artifactId>\n       <version>[0.6.0,)</version>\n     </dependency>\n     <dependency>\n       <groupId>org.erlang.otp</groupId>\n       <artifactId>jinterface</artifactId>\n       <version>[1.5.6-custom,)</version>\n     </dependency>\n     <dependency>\n       <groupId>com.yahoo.ycsb</groupId>\n       <artifactId>core</artifactId>\n       <version>[0.1.4, )</version>\n     </dependency>\n  </dependencies>\n\n\n  <build>\n    <plugins>\n     <plugin>\n        <groupId>org.apache.maven.plugins</groupId>\n        <artifactId>maven-assembly-plugin</artifactId>\n        <version>${maven.assembly.version}</version>\n        <configuration>\n          <descriptorRefs>\n            <descriptorRef>jar-with-dependencies</descriptorRef>\n          </descriptorRefs>\n          <appendAssemblyId>false</appendAssemblyId>\n        </configuration>\n        <executions>\n          <execution>\n            <phase>package</phase>\n            <goals>\n              <goal>single</goal>\n            </goals>\n          </execution>\n        </executions>\n      </plugin>\n     </plugins>\n  </build>\n</project>\n"
  },
  {
    "path": "contrib/ycsb/scalaris/src/main/java/com/yahoo/ycsb/db/ScalarisClient.java",
    "content": "/**\n * Scalaris client for YCSB\n *\n */\n\npackage com.yahoo.ycsb.db;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\nimport com.yahoo.ycsb.DB;\nimport com.yahoo.ycsb.DBException;\nimport com.yahoo.ycsb.ByteIterator;\nimport com.yahoo.ycsb.StringByteIterator;\n\nimport de.zib.scalaris.Connection;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.ConnectionFactory;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.ReplicatedDHT;\nimport de.zib.scalaris.TransactionSingleOp;\n\nimport java.util.Map;\nimport java.util.HashMap;\nimport java.util.Map.Entry;\nimport java.util.Properties;\nimport java.util.Set;\nimport java.util.Vector;\n\n\npublic class ScalarisClient extends DB {\n\n\tpublic static final String HOST_PROPERTY = \"scalaris.node\";\n\tpublic static final String CLIENT_NAME_PROPERTY = \"scalaris.name\";\n\tpublic static final String COOKIE_PROPERTY = \"scalaris.cookie\";\n\n\tpublic static final String TABLE_SEPERATOR = \":\";\n\n\tprivate Connection conn;\n\tprivate TransactionSingleOp ts;\n\tprivate ReplicatedDHT rdht;\n\n\tpublic void init() {\n\t\ttry {\n\t\t\tProperties props = this.getProperties();\n\t\t\tif (props.isEmpty()) {\n\t\t\t\tprops.setProperty(HOST_PROPERTY, \"node1@localhost\");\n\t\t\t\tprops.setProperty(CLIENT_NAME_PROPERTY, \"YCSB\");\n\t\t\t\tprops.setProperty(COOKIE_PROPERTY, \"chocolate chip cookie\");\n\t\t\t}\n\t\t\tConnectionFactory fac = new ConnectionFactory(props);\n\t\t\tconn = fac.createConnection();\n\t\t\tts = new TransactionSingleOp(conn);\n\t\t\trdht = new ReplicatedDHT(conn);\n\t\t} catch (ConnectionException e) {\n\t\t\te.printStackTrace();\n\t\t}\n\t}\n\n\tpublic void cleanup() throws DBException {\n\t\tts.closeConnection();\n\t}\n\n\t@Override\n\tpublic int read(String table, String key, Set<String> fields,\n\t\t\tHashMap<String, ByteIterator> result) {\n\n\t\ttry {\n\t\t\t// get the value\n\t\t\tMap<String, Object> dbValues = ts.read(\n\t\t\t\t\ttable + TABLE_SEPERATOR + key).jsonValue();\n\t\t\t// jsonValue() returns Map<String, Object> but we need\n\t\t\t// Map<String, ByteIterator>\n\t\t\tfor (Entry<String, Object> entry : dbValues.entrySet()) {\n\t\t\t\tresult.put(entry.getKey(), new StringByteIterator(\n\t\t\t\t\t\t(String) entry.getValue()));\n\t\t\t}\n\t\t} catch (NotFoundException e) {\n\t\t} catch (OtpErlangException e) {\n\t\t\te.printStackTrace();\n\t\t\treturn 1;\n\t\t}\n\t\treturn 0;\n\t}\n\n\t@Override\n\tpublic int insert(String table, String key,\n\t\t\tHashMap<String, ByteIterator> values) {\n\n\t\t// Convert <String, ByteIterator> to <String, Object>\n\t\tHashMap<String, String> values2 = new HashMap<String, String>(\n\t\t\t\tvalues.size());\n\t\tfor (Entry<String, ByteIterator> entry : values.entrySet()) {\n\t\t\tvalues2.put(entry.getKey(), entry.getValue().toString());\n\t\t}\n\n\t\ttry {\n\t\t\tts.write(table + TABLE_SEPERATOR + key, values2);\n\t\t} catch (OtpErlangException e) {\n\t\t\te.printStackTrace();\n\t\t\treturn 1;\n\t\t}\n\t\treturn 0;\n\t}\n\n\t@Override\n\tpublic int delete(String table, String key) {\n\n\t\ttry {\n\t\t\t// Erlang delete is not save, so we check if the key\n\t\t\t// was deleted on all four replicas\n\t\t\tif(rdht.delete(table + TABLE_SEPERATOR + key).ok != 4)\n\t\t\t\treturn 1;\t\t\n\t\t} catch (OtpErlangException e) {\n\t\t\te.printStackTrace();\n\t\t\treturn 1;\n\t\t}\n\t\treturn 0;\n\t}\n\n\t@Override\n\t/*\n\t * We have to read the old values, compare them with the new ones which\n\t * leads to replacing or adding of fields\n\t */\n\tpublic int update(String table, String key,\n\t\t\tHashMap<String, ByteIterator> values) {\n\t\t// Convert <String, ByteIterator> to <String, String>\n\t\tHashMap<String, String> values2 = new HashMap<String, String>(\n\t\t\t\tvalues.size());\n\t\tfor (Entry<String, ByteIterator> entry : values.entrySet()) {\n\t\t\tvalues2.put(entry.getKey(), entry.getValue().toString());\n\t\t}\n\n\t\t// jsonValue() returns Map<String, Object> but we need\n\t\t// Map<String, ByteIterator>\n\t\tHashMap<String, String> dbValues2 = new HashMap<String, String>();\n\t\ttry {\n\t\t\tHashMap<String, Object> dbValues = (HashMap<String, Object>) ts\n\t\t\t\t\t.read(table + TABLE_SEPERATOR + key).jsonValue();\n\n\t\t\tfor (Entry<String, Object> entry : dbValues.entrySet()) {\n\t\t\t\tdbValues2.put(entry.getKey(), (String) entry.getValue());\n\t\t\t}\n\t\t} catch (NotFoundException e) {\n\t\t} catch (OtpErlangException e) {\n\t\t\te.printStackTrace();\n\t\t}\n\n\t\t// update or add new fields and values\n\t\tdbValues2.putAll(values2);\n\n\t\ttry {\n\t\t\tts.write(table + TABLE_SEPERATOR + key, dbValues2);\n\t\t} catch (OtpErlangException e) {\n\t\t\te.printStackTrace();\n\t\t\treturn 1;\n\t\t}\n\t\treturn 0;\n\t}\n\n\t@Override\n\tpublic int scan(String table, String startkey, int recordcount,\n\t\t\tSet<String> fields, Vector<HashMap<String, ByteIterator>> result) {\n\t\t/* Scalaris doesn't support scan semantics (sorry) */\n\t\treturn 1;\n\t}\n}\n"
  },
  {
    "path": "contrib/ycsb/workloads/workloada",
    "content": "# Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n#                                                                                                                                                                                 \n# Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n# may not use this file except in compliance with the License. You                                                                                                                \n# may obtain a copy of the License at                                                                                                                                             \n#                                                                                                                                                                                 \n# http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n#                                                                                                                                                                                 \n# Unless required by applicable law or agreed to in writing, software                                                                                                             \n# distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n# implied. See the License for the specific language governing                                                                                                                    \n# permissions and limitations under the License. See accompanying                                                                                                                 \n# LICENSE file.                                                                                                                                                                   \n\n\n# Yahoo! Cloud System Benchmark\n# Workload A: Update heavy workload\n#   Application example: Session store recording recent actions\n#                        \n#   Read/update ratio: 50/50\n#   Default data size: 1 KB records (10 fields, 100 bytes each, plus key)\n#   Request distribution: zipfian\n\nrecordcount=1000\noperationcount=1000\nworkload=com.yahoo.ycsb.workloads.CoreWorkload\n\nreadallfields=true\n\nreadproportion=0.5\nupdateproportion=0.5\nscanproportion=0\ninsertproportion=0\n\nrequestdistribution=zipfian\n\n"
  },
  {
    "path": "contrib/ycsb/workloads/workloadb",
    "content": "# Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n#                                                                                                                                                                                 \n# Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n# may not use this file except in compliance with the License. You                                                                                                                \n# may obtain a copy of the License at                                                                                                                                             \n#                                                                                                                                                                                 \n# http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n#                                                                                                                                                                                 \n# Unless required by applicable law or agreed to in writing, software                                                                                                             \n# distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n# implied. See the License for the specific language governing                                                                                                                    \n# permissions and limitations under the License. See accompanying                                                                                                                 \n# LICENSE file.                                                                                                                                                                   \n\n# Yahoo! Cloud System Benchmark\n# Workload B: Read mostly workload\n#   Application example: photo tagging; add a tag is an update, but most operations are to read tags\n#                        \n#   Read/update ratio: 95/5\n#   Default data size: 1 KB records (10 fields, 100 bytes each, plus key)\n#   Request distribution: zipfian\n\nrecordcount=1000\noperationcount=1000\nworkload=com.yahoo.ycsb.workloads.CoreWorkload\n\nreadallfields=true\n\nreadproportion=0.95\nupdateproportion=0.05\nscanproportion=0\ninsertproportion=0\n\nrequestdistribution=zipfian\n\n"
  },
  {
    "path": "contrib/ycsb/workloads/workloadc",
    "content": "# Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n#                                                                                                                                                                                 \n# Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n# may not use this file except in compliance with the License. You                                                                                                                \n# may obtain a copy of the License at                                                                                                                                             \n#                                                                                                                                                                                 \n# http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n#                                                                                                                                                                                 \n# Unless required by applicable law or agreed to in writing, software                                                                                                             \n# distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n# implied. See the License for the specific language governing                                                                                                                    \n# permissions and limitations under the License. See accompanying                                                                                                                 \n# LICENSE file.                                                                                                                                                                   \n\n# Yahoo! Cloud System Benchmark\n# Workload C: Read only\n#   Application example: user profile cache, where profiles are constructed elsewhere (e.g., Hadoop)\n#                        \n#   Read/update ratio: 100/0\n#   Default data size: 1 KB records (10 fields, 100 bytes each, plus key)\n#   Request distribution: zipfian\n\nrecordcount=1000\noperationcount=1000\nworkload=com.yahoo.ycsb.workloads.CoreWorkload\n\nreadallfields=true\n\nreadproportion=1\nupdateproportion=0\nscanproportion=0\ninsertproportion=0\n\nrequestdistribution=zipfian\n\n\n\n"
  },
  {
    "path": "contrib/ycsb/workloads/workloadd",
    "content": "# Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \r\n#                                                                                                                                                                                 \r\n# Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \r\n# may not use this file except in compliance with the License. You                                                                                                                \r\n# may obtain a copy of the License at                                                                                                                                             \r\n#                                                                                                                                                                                 \r\n# http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \r\n#                                                                                                                                                                                 \r\n# Unless required by applicable law or agreed to in writing, software                                                                                                             \r\n# distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \r\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \r\n# implied. See the License for the specific language governing                                                                                                                    \r\n# permissions and limitations under the License. See accompanying                                                                                                                 \r\n# LICENSE file.                                                                                                                                                                   \r\n\r\n# Yahoo! Cloud System Benchmark\r\n# Workload D: Read latest workload\r\n#   Application example: user status updates; people want to read the latest\r\n#                        \r\n#   Read/update/insert ratio: 95/0/5\r\n#   Default data size: 1 KB records (10 fields, 100 bytes each, plus key)\r\n#   Request distribution: latest\r\n\r\n# The insert order for this is hashed, not ordered. The \"latest\" items may be \r\n# scattered around the keyspace if they are keyed by userid.timestamp. A workload\r\n# which orders items purely by time, and demands the latest, is very different than \r\n# workload here (which we believe is more typical of how people build systems.)\r\n\r\nrecordcount=1000\r\noperationcount=1000\r\nworkload=com.yahoo.ycsb.workloads.CoreWorkload\r\n\r\nreadallfields=true\r\n\r\nreadproportion=0.95\r\nupdateproportion=0\r\nscanproportion=0\r\ninsertproportion=0.05\r\n\r\nrequestdistribution=latest\r\n\r\n"
  },
  {
    "path": "contrib/ycsb/workloads/workloade",
    "content": "# Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \n#                                                                                                                                                                                 \n# Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \n# may not use this file except in compliance with the License. You                                                                                                                \n# may obtain a copy of the License at                                                                                                                                             \n#                                                                                                                                                                                 \n# http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \n#                                                                                                                                                                                 \n# Unless required by applicable law or agreed to in writing, software                                                                                                             \n# distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \n# implied. See the License for the specific language governing                                                                                                                    \n# permissions and limitations under the License. See accompanying                                                                                                                 \n# LICENSE file.                                                                                                                                                                   \n\n# Yahoo! Cloud System Benchmark\n# Workload E: Short ranges\n#   Application example: threaded conversations, where each scan is for the posts in a given thread (assumed to be clustered by thread id)\n#                        \n#   Scan/insert ratio: 95/5\n#   Default data size: 1 KB records (10 fields, 100 bytes each, plus key)\n#   Request distribution: zipfian\n\n# The insert order is hashed, not ordered. Although the scans are ordered, it does not necessarily\n# follow that the data is inserted in order. For example, posts for thread 342 may not be inserted contiguously, but\n# instead interspersed with posts from lots of other threads. The way the YCSB client works is that it will pick a start\n# key, and then request a number of records; this works fine even for hashed insertion.\n\nrecordcount=1000\noperationcount=1000\nworkload=com.yahoo.ycsb.workloads.CoreWorkload\n\nreadallfields=true\n\nreadproportion=0\nupdateproportion=0\nscanproportion=0.95\ninsertproportion=0.05\n\nrequestdistribution=zipfian\n\nmaxscanlength=100\n\nscanlengthdistribution=uniform\n\n\n"
  },
  {
    "path": "contrib/ycsb/workloads/workloadf",
    "content": "# Copyright (c) 2010 Yahoo! Inc. All rights reserved.                                                                                                                             \r\n#                                                                                                                                                                                 \r\n# Licensed under the Apache License, Version 2.0 (the \"License\"); you                                                                                                             \r\n# may not use this file except in compliance with the License. You                                                                                                                \r\n# may obtain a copy of the License at                                                                                                                                             \r\n#                                                                                                                                                                                 \r\n# http://www.apache.org/licenses/LICENSE-2.0                                                                                                                                      \r\n#                                                                                                                                                                                 \r\n# Unless required by applicable law or agreed to in writing, software                                                                                                             \r\n# distributed under the License is distributed on an \"AS IS\" BASIS,                                                                                                               \r\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or                                                                                                                 \r\n# implied. See the License for the specific language governing                                                                                                                    \r\n# permissions and limitations under the License. See accompanying                                                                                                                 \r\n# LICENSE file.                                                                                                                                                                   \r\n\r\n# Yahoo! Cloud System Benchmark\r\n# Workload F: Read-modify-write workload\r\n#   Application example: user database, where user records are read and modified by the user or to record user activity.\r\n#                        \r\n#   Read/read-modify-write ratio: 50/50\r\n#   Default data size: 1 KB records (10 fields, 100 bytes each, plus key)\r\n#   Request distribution: zipfian\r\n\r\nrecordcount=1000\r\noperationcount=1000\r\nworkload=com.yahoo.ycsb.workloads.CoreWorkload\r\n\r\nreadallfields=true\r\n\r\nreadproportion=0.5\r\nupdateproportion=0\r\nscanproportion=0\r\ninsertproportion=0\r\nreadmodifywriteproportion=0.5\r\n\r\nrequestdistribution=zipfian\r\n\r\n"
  },
  {
    "path": "cpp-api/.clang-format",
    "content": "BasedOnStyle: LLVM\n"
  },
  {
    "path": "cpp-api/.gitignore",
    "content": "libscalaris.a\nMakefile\n*.d\n*.o\ntests\n*.o.json\n"
  },
  {
    "path": "cpp-api/Makefile.in",
    "content": "SHELL=@CXXSHELL@\n\nCXX=@CXX@\nCXXFLAGS=@CXXFLAGS@ -Wall @BOOST_CPPFLAGS@ @OPENSSL_INCLUDES@ -Iinclude\nLDFLAGS=@LDFLAGS@ @BOOST_LDFLAGS@ @BOOST_SYSTEM_LIB@ @BOOST_REGEX_LIB@ @BOOST_ASIO_LIB@ @BOOST_UNIT_TEST_FRAMEWORK_LIB@ @OPENSSL_LDFLAGS@ @OPENSSL_LIBS@ -lpthread\n\nifeq ($(VERBOSE),1)\n  VC=\n  NVC=@@TRUE@ ||\nelse\n  VC=@\n  NVC=@\nendif\n\nSRC_CPP=$(filter-out src/cli.cpp, $(wildcard src/*.cpp))\nSRC_TEST_SSL=$(shell ls test/ssl-*.cpp)\nSRC_TEST_NOSSL=$(shell ls test/*.cpp | grep -v ssl)\n\nOBJS=$(SRC_CPP:.cpp=.o)\nOBJS_TEST_SSL=$(SRC_TEST_SSL:.cpp=.o)\nOBJS_TEST_NOSSL=$(SRC_TEST_NOSSL:.cpp=.o)\n\nall: libscalaris.a scalaris tests ssl-tests\n\ncpp-test : tests\n\t$(VC)./tests --report_level=detailed\n\ntests: $(OBJS_TEST_NOSSL) libscalaris.a\n\t$(NVC)echo -e \"\\e[0;33mCreating tests\\033[39m\"\n\t$(VC)$(CXX) -o $@ $(OBJS_TEST_NOSSL) libscalaris.a $(LDFLAGS)\n\nssl-tests: $(OBJS_TEST_SSL) libscalaris.a\n\t$(NVC)echo -e \"\\e[0;33mCreating ssl-tests\\033[39m\"\n\t$(VC)$(CXX) -o $@ $(OBJS_TEST_SSL) libscalaris.a $(LDFLAGS)\n\nlibscalaris.a: $(OBJS)\n\t$(NVC)echo -e \"\\e[0;33mCreating archive\" $@ \"\\033[39m\"\n\t$(VC)@ARCHIVECMD@ $@ $(OBJS)\n\n.cpp.o: Makefile\n\t$(NVC)echo -e \"\\e[0;32mCompiling\" $< \"\\033[39m\"\n\t$(VC)$(CXX) -MD $(CXXFLAGS) @JSONFLAGS@ -c $< -o $@\n\nscalaris: libscalaris.a src/cli.o\n\t$(NVC)echo -e \"\\e[0;31mCreating scalaris\\033[39m\"\n\t$(VC)$(CXX) -o $@ src/cli.o libscalaris.a $(LDFLAGS)$  @BOOST_PROGRAM_OPTIONS_LIB@\n\nclean:\n\t@-rm -rf $(shell find . -type f -name '*.o')\n\t@-rm -rf $(shell find . -type f -name '*.o.json')\n\t@-rm -rf $(shell find . -type f -name '*.d')\n\t@-rm -f ./tests ./ssl-tests ./scalaris ./libscalaris.a\n\n# note, double $ because of escaping in Makefiles\ncompile_commands.json: $(SRC_CPP) $(SRC_TEST_CPP) Makefile\n\t$(NVC)echo -e \"\\e[0;32mCreating Compilation Database\" $< \"\\033[39m\"\n\t$(VC)sed -e '1s/^/[\\n/' -e '$$ s/,$$/\\n]/' $(SRC_CPP:.cpp=.o.json) $(SRC_TEST_CPP:.cpp=.o.json) > compile_commands.json\n\n-include $(OBJS:.o=.d)\n-include $(OBJS_TEST_NOSSL:.o=.d)\n-include $(OBJS_TEST_SSL:.o=.d)\n-include src/cli.d\n\n.PHONY: all cpp-test clean beautify tidy\n\nbeautify:\n\t$(VC)@CLANGFORMAT@ -style=file -i src/*.[hc]pp test/*.[hc]pp\n\ntidy:\n\t$(VC)@CLANGTIDY@ src/cli.cpp $(filter-out src/jsoncpp.cpp, $(SRC_CPP)) $(TEST_SRC_CPP) -- $(CXXFLAGS)\n"
  },
  {
    "path": "cpp-api/include/connection.hpp",
    "content": "// Copyright 2015-2018 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#pragma once\n\n#include \"converter.hpp\"\n#include \"exceptions.hpp\"\n#include \"json/json.h\"\n\n#include <array>\n#include <iostream>\n#include <stdexcept>\n#include <string>\n\nnamespace scalaris {\n\n  /// represents a connection to Scalaris to execute JSON-RPC requests\n  class Connection {\n  protected:\n    bool closed = false;\n    std::string hostname;\n    std::string link; // the path/link of the URL for JSON-RPC\n    unsigned port;\n\n  protected:\n    /**\n     * creates a connection instance\n     * @param _hostname the host name of the Scalaris instance\n     * @param _link the URL for JSON-RPC\n     * @param _port the TCP port of the Scalaris instance\n     */\n    Connection(const std::string& _hostname,\n               const std::string& _link = \"jsonrpc.yaws\",\n               unsigned _port = 8000);\n\n  public:\n\n    virtual ~Connection() = default;\n\n    /**\n     * performs a JSON-RPC request\n     * @param methodname the name of the function to call\n     * @param args the list of arguments of the function call\n     */\n    template <typename... Args>\n    Json::Value rpc(const std::string& methodname, Args... args) {\n      std::array<Json::Value, sizeof...(args)> arg_list = {\n          {Converter<Args>::to_value(args)...}};\n\n      Json::Value params = Json::arrayValue;\n      for (size_t i = 0; i < sizeof...(args); i++)\n        params.append(arg_list[i]);\n      return exec_call(methodname, params);\n    }\n\n    std::string getHostname() const { return hostname; }\n    std::string getLink() const { return link; }\n    virtual unsigned getPort() const { return port; }\n\n    virtual bool needsConnect() const = 0;\n    virtual void connect() = 0;\n\n    virtual std::string toString() const = 0;\n\n  protected:\n    virtual Json::Value exec_call(const std::string& methodname,\n                                  Json::Value params,\n                                  bool reconnect = true) = 0;\n  };\n} // namespace scalaris\n"
  },
  {
    "path": "cpp-api/include/converter.hpp",
    "content": "// Copyright 2016-2018 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#ifndef SCALARIS_CONVERTER_HPP\n#define SCALARIS_CONVERTER_HPP\n\n#include <string>\n\n#include \"req_list.hpp\"\n#include \"json/json.h\"\n\nnamespace scalaris {\n\n  /// converts C++ values to JSON values\n  /* for compound types it recursively walks over the member objects */\n  template<typename T>\n  class Converter {};\n\n  /// \\brief converts integers to JSON integers\n  template<>\n  class Converter<int> {\n  public:\n    /**\n     * converts an integer to a JSON value\n     * @param arg the integer value\n     */\n    static Json::Value to_value(int arg) { return Json::Value(arg); }\n  };\n\n  /// \\brief converts unsigned integers to JSON integers\n  template<>\n  class Converter<unsigned int> {\n  public:\n    /**\n     * converts an integer to a JSON value\n     * @param arg the integer value\n     */\n    static Json::Value to_value(unsigned int arg) { return Json::Value(arg); }\n  };\n\n  /// converts std::string to JSON strings\n  template<>\n  class Converter<std::string> {\n  public:\n    /**\n     * converts a std::string to a JSON value\n     * @param arg the std::string value\n     */\n    static Json::Value to_value(std::string arg) { return Json::Value(arg); }\n  };\n\n  /// converts RequestLists to JSON lists\n  template<>\n  class Converter<RequestList> {\n  public:\n    /**\n     * converts a request list to a JSON value\n     * @param arg the request list value\n     */\n    static Json::Value to_value(RequestList arg) { return arg.get_json_value(); }\n  };\n\n  /// converts std::pair<T,U> to JSON structs\n  template<typename T, typename U>\n  class Converter<std::pair<T,U> > {\n  public:\n    /**\n     * converts a std::pair to a JSON value\n     * @param arg the std::pair value\n     */\n    static Json::Value to_value(std::pair<T,U> arg) {\n      Json::Value result;\n      result[\"first\"] = Converter<T>::to_value(std::get<0>(arg));\n      result[\"second\"] = Converter<T>::to_value(std::get<1>(arg));\n      return result;\n    }\n  };\n\n  /// converts std::vector<T> to JSON arrays\n  template<typename T>\n  class Converter<std::vector<T> > {\n  public:\n    /**\n     * converts a std::vector to a JSON value\n     * @param arg the std::vector value\n     */\n    static Json::Value to_value(std::vector<T> arg) {\n      Json::Value result;\n\n      for(T& t: arg)\n        result.append(Converter<T>::to_value(t));\n\n      return result;\n    }\n  };\n\n  /// converts Json::Values to JSON values\n  template<>\n  class Converter<Json::Value> {\n  public:\n    /**\n     * converts a request list to a JSON value\n     * @param arg the request list value\n     */\n    static Json::Value to_value(Json::Value arg) { return arg; }\n  };\n\n}\n\n#endif\n"
  },
  {
    "path": "cpp-api/include/exceptions.hpp",
    "content": "// Copyright 2015-2018 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#ifndef SCALARIS_EXCEPTIONS_HPP\n#define SCALARIS_EXCEPTIONS_HPP\n\n#include <stdexcept>\n#include <string>\n\nnamespace scalaris {\n\n  /// Exception to report when a key was not found\n  class NotFoundError : public std::exception {\n  };\n\n  /// Exception to report when a rpc-call is not supported\n  class NotSupportedError : public std::exception {\n  };\n\n  /// Exception to report mal-formed JSON objects\n  class MalFormedJsonError : public std::exception {\n  };\n\n  /// Exception to report errors when connecting to Scalaris\n  class ConnectionError : public std::exception {\n    std::string msg;\n  public:\n    /**\n     * Creates a ConnectionError instance\n     * @param _msg the detailed error message\n     */\n    ConnectionError(const std::string& _msg) : msg(_msg) {}\n\n    /**\n     * returns a detailed error description\n     */\n    virtual const char *what() const noexcept override { return msg.c_str(); }\n  };\n\n  /// Exception to report transaction failures\n  class CommitFailedError : public std::exception {\n    std::string msg;\n  public:\n    /**\n     * Creates a CommitFailedError instance\n     * @param _msg the detailed error message\n     */\n    CommitFailedError(const std::string& _msg) : msg(_msg) {}\n\n    /**\n     * returns a detailed error description\n     */\n    virtual const char *what() const noexcept override { return msg.c_str(); }\n  };\n\n  /// Exception to report read errors\n  class ReadFailedError : public std::exception {\n    std::string msg;\n  public:\n    /**\n     * Creates a ReadFailedError instance\n     * @param _msg the detailed error message\n     */\n    ReadFailedError(const std::string& _msg) : msg(_msg) {}\n\n    /**\n     * returns a detailed error description\n     */\n    virtual const char *what() const noexcept override { return msg.c_str(); }\n  };\n\n  /// Exception to report write errors\n  class WriteFailedError : public std::exception {\n    std::string msg;\n  public:\n    /**\n     * Creates a WriteFailedError instance\n     * @param _msg the detailed error message\n     */\n    WriteFailedError(const std::string& _msg) : msg(_msg) {}\n\n    /**\n     * returns a detailed error description\n     */\n    virtual const char *what() const noexcept override { return msg.c_str(); }\n  };\n\n}\n\n#endif\n"
  },
  {
    "path": "cpp-api/include/json/json-forwards.h",
    "content": "/// Json-cpp amalgated forward header (http://jsoncpp.sourceforge.net/).\n/// It is intended to be used with #include \"json/json-forwards.h\"\n/// This header provides forward declaration for all JsonCpp types.\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: LICENSE\n// //////////////////////////////////////////////////////////////////////\n\n/*\nThe JsonCpp library's source code, including accompanying documentation, \ntests and demonstration applications, are licensed under the following\nconditions...\n\nThe author (Baptiste Lepilleur) explicitly disclaims copyright in all \njurisdictions which recognize such a disclaimer. In such jurisdictions, \nthis software is released into the Public Domain.\n\nIn jurisdictions which do not recognize Public Domain property (e.g. Germany as of\n2010), this software is Copyright (c) 2007-2010 by Baptiste Lepilleur, and is\nreleased under the terms of the MIT License (see below).\n\nIn jurisdictions which recognize Public Domain property, the user of this \nsoftware may choose to accept it either as 1) Public Domain, 2) under the \nconditions of the MIT License (see below), or 3) under the terms of dual \nPublic Domain/MIT License conditions described here, as they choose.\n\nThe MIT License is about as close to Public Domain as a license can get, and is\ndescribed in clear, concise terms at:\n\n   http://en.wikipedia.org/wiki/MIT_License\n   \nThe full text of the MIT License follows:\n\n========================================================================\nCopyright (c) 2007-2010 Baptiste Lepilleur\n\nPermission is hereby granted, free of charge, to any person\nobtaining a copy of this software and associated documentation\nfiles (the \"Software\"), to deal in the Software without\nrestriction, including without limitation the rights to use, copy,\nmodify, merge, publish, distribute, sublicense, and/or sell copies\nof the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be\nincluded in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\nNONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\nBE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\nACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\nCONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n========================================================================\n(END LICENSE TEXT)\n\nThe MIT license is compatible with both the GPL and commercial\nsoftware, affording one all of the rights of Public Domain with the\nminor nuisance of being required to keep the above copyright notice\nand license text in the source code. Note also that by accepting the\nPublic Domain \"license\" you can re-license your copy using whatever\nlicense you like.\n\n*/\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: LICENSE\n// //////////////////////////////////////////////////////////////////////\n\n\n\n\n\n#ifndef JSON_FORWARD_AMALGATED_H_INCLUDED\n# define JSON_FORWARD_AMALGATED_H_INCLUDED\n/// If defined, indicates that the source file is amalgated\n/// to prevent private header inclusion.\n#define JSON_IS_AMALGAMATION\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: include/json/config.h\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2007-2010 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n#ifndef JSON_CONFIG_H_INCLUDED\n#define JSON_CONFIG_H_INCLUDED\n\n/// If defined, indicates that json library is embedded in CppTL library.\n//# define JSON_IN_CPPTL 1\n\n/// If defined, indicates that json may leverage CppTL library\n//#  define JSON_USE_CPPTL 1\n/// If defined, indicates that cpptl vector based map should be used instead of\n/// std::map\n/// as Value container.\n//#  define JSON_USE_CPPTL_SMALLMAP 1\n\n// If non-zero, the library uses exceptions to report bad input instead of C\n// assertion macros. The default is to use exceptions.\n#ifndef JSON_USE_EXCEPTION\n#define JSON_USE_EXCEPTION 1\n#endif\n\n/// If defined, indicates that the source file is amalgated\n/// to prevent private header inclusion.\n/// Remarks: it is automatically defined in the generated amalgated header.\n// #define JSON_IS_AMALGAMATION\n\n#ifdef JSON_IN_CPPTL\n#include <cpptl/config.h>\n#ifndef JSON_USE_CPPTL\n#define JSON_USE_CPPTL 1\n#endif\n#endif\n\n#ifdef JSON_IN_CPPTL\n#define JSON_API CPPTL_API\n#elif defined(JSON_DLL_BUILD)\n#if defined(_MSC_VER)\n#define JSON_API __declspec(dllexport)\n#define JSONCPP_DISABLE_DLL_INTERFACE_WARNING\n#endif // if defined(_MSC_VER)\n#elif defined(JSON_DLL)\n#if defined(_MSC_VER)\n#define JSON_API __declspec(dllimport)\n#define JSONCPP_DISABLE_DLL_INTERFACE_WARNING\n#endif // if defined(_MSC_VER)\n#endif // ifdef JSON_IN_CPPTL\n#if !defined(JSON_API)\n#define JSON_API\n#endif\n\n// If JSON_NO_INT64 is defined, then Json only support C++ \"int\" type for\n// integer\n// Storages, and 64 bits integer support is disabled.\n// #define JSON_NO_INT64 1\n\n#if defined(_MSC_VER) // MSVC\n#  if _MSC_VER <= 1200 // MSVC 6\n    // Microsoft Visual Studio 6 only support conversion from __int64 to double\n    // (no conversion from unsigned __int64).\n#    define JSON_USE_INT64_DOUBLE_CONVERSION 1\n    // Disable warning 4786 for VS6 caused by STL (identifier was truncated to '255'\n    // characters in the debug information)\n    // All projects I've ever seen with VS6 were using this globally (not bothering\n    // with pragma push/pop).\n#    pragma warning(disable : 4786)\n#  endif // MSVC 6\n\n#  if _MSC_VER >= 1500 // MSVC 2008\n    /// Indicates that the following function is deprecated.\n#    define JSONCPP_DEPRECATED(message) __declspec(deprecated(message))\n#  endif\n\n#endif // defined(_MSC_VER)\n\n\n#ifndef JSON_HAS_RVALUE_REFERENCES\n\n#if defined(_MSC_VER) && _MSC_VER >= 1600 // MSVC >= 2010\n#define JSON_HAS_RVALUE_REFERENCES 1\n#endif // MSVC >= 2010\n\n#ifdef __clang__\n#if __has_feature(cxx_rvalue_references)\n#define JSON_HAS_RVALUE_REFERENCES 1\n#endif  // has_feature\n\n#elif defined __GNUC__ // not clang (gcc comes later since clang emulates gcc)\n#if defined(__GXX_EXPERIMENTAL_CXX0X__) || (__cplusplus >= 201103L)\n#define JSON_HAS_RVALUE_REFERENCES 1\n#endif  // GXX_EXPERIMENTAL\n\n#endif // __clang__ || __GNUC__\n\n#endif // not defined JSON_HAS_RVALUE_REFERENCES\n\n#ifndef JSON_HAS_RVALUE_REFERENCES\n#define JSON_HAS_RVALUE_REFERENCES 0\n#endif\n\n#ifdef __clang__\n#elif defined __GNUC__ // not clang (gcc comes later since clang emulates gcc)\n#  if (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 5))\n#    define JSONCPP_DEPRECATED(message)  __attribute__ ((deprecated(message)))\n#  elif (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 1))\n#    define JSONCPP_DEPRECATED(message)  __attribute__((__deprecated__))\n#  endif  // GNUC version\n#endif // __clang__ || __GNUC__\n\n#if !defined(JSONCPP_DEPRECATED)\n#define JSONCPP_DEPRECATED(message)\n#endif // if !defined(JSONCPP_DEPRECATED)\n\nnamespace Json {\ntypedef int Int;\ntypedef unsigned int UInt;\n#if defined(JSON_NO_INT64)\ntypedef int LargestInt;\ntypedef unsigned int LargestUInt;\n#undef JSON_HAS_INT64\n#else                 // if defined(JSON_NO_INT64)\n// For Microsoft Visual use specific types as long long is not supported\n#if defined(_MSC_VER) // Microsoft Visual Studio\ntypedef __int64 Int64;\ntypedef unsigned __int64 UInt64;\n#else                 // if defined(_MSC_VER) // Other platforms, use long long\ntypedef long long int Int64;\ntypedef unsigned long long int UInt64;\n#endif // if defined(_MSC_VER)\ntypedef Int64 LargestInt;\ntypedef UInt64 LargestUInt;\n#define JSON_HAS_INT64\n#endif // if defined(JSON_NO_INT64)\n} // end namespace Json\n\n#endif // JSON_CONFIG_H_INCLUDED\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: include/json/config.h\n// //////////////////////////////////////////////////////////////////////\n\n\n\n\n\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: include/json/forwards.h\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2007-2010 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n#ifndef JSON_FORWARDS_H_INCLUDED\n#define JSON_FORWARDS_H_INCLUDED\n\n#if !defined(JSON_IS_AMALGAMATION)\n#include \"config.h\"\n#endif // if !defined(JSON_IS_AMALGAMATION)\n\nnamespace Json {\n\n// writer.h\nclass FastWriter;\nclass StyledWriter;\n\n// reader.h\nclass Reader;\n\n// features.h\nclass Features;\n\n// value.h\ntypedef unsigned int ArrayIndex;\nclass StaticString;\nclass Path;\nclass PathArgument;\nclass Value;\nclass ValueIteratorBase;\nclass ValueIterator;\nclass ValueConstIterator;\n\n} // namespace Json\n\n#endif // JSON_FORWARDS_H_INCLUDED\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: include/json/forwards.h\n// //////////////////////////////////////////////////////////////////////\n\n\n\n\n\n#endif //ifndef JSON_FORWARD_AMALGATED_H_INCLUDED\n"
  },
  {
    "path": "cpp-api/include/json/json.h",
    "content": "/// Json-cpp amalgated header (http://jsoncpp.sourceforge.net/).\n/// It is intended to be used with #include \"json/json.h\"\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: LICENSE\n// //////////////////////////////////////////////////////////////////////\n\n/*\nThe JsonCpp library's source code, including accompanying documentation, \ntests and demonstration applications, are licensed under the following\nconditions...\n\nThe author (Baptiste Lepilleur) explicitly disclaims copyright in all \njurisdictions which recognize such a disclaimer. In such jurisdictions, \nthis software is released into the Public Domain.\n\nIn jurisdictions which do not recognize Public Domain property (e.g. Germany as of\n2010), this software is Copyright (c) 2007-2010 by Baptiste Lepilleur, and is\nreleased under the terms of the MIT License (see below).\n\nIn jurisdictions which recognize Public Domain property, the user of this \nsoftware may choose to accept it either as 1) Public Domain, 2) under the \nconditions of the MIT License (see below), or 3) under the terms of dual \nPublic Domain/MIT License conditions described here, as they choose.\n\nThe MIT License is about as close to Public Domain as a license can get, and is\ndescribed in clear, concise terms at:\n\n   http://en.wikipedia.org/wiki/MIT_License\n   \nThe full text of the MIT License follows:\n\n========================================================================\nCopyright (c) 2007-2010 Baptiste Lepilleur\n\nPermission is hereby granted, free of charge, to any person\nobtaining a copy of this software and associated documentation\nfiles (the \"Software\"), to deal in the Software without\nrestriction, including without limitation the rights to use, copy,\nmodify, merge, publish, distribute, sublicense, and/or sell copies\nof the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be\nincluded in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\nNONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\nBE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\nACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\nCONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n========================================================================\n(END LICENSE TEXT)\n\nThe MIT license is compatible with both the GPL and commercial\nsoftware, affording one all of the rights of Public Domain with the\nminor nuisance of being required to keep the above copyright notice\nand license text in the source code. Note also that by accepting the\nPublic Domain \"license\" you can re-license your copy using whatever\nlicense you like.\n\n*/\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: LICENSE\n// //////////////////////////////////////////////////////////////////////\n\n\n\n\n\n#ifndef JSON_AMALGATED_H_INCLUDED\n# define JSON_AMALGATED_H_INCLUDED\n/// If defined, indicates that the source file is amalgated\n/// to prevent private header inclusion.\n#define JSON_IS_AMALGAMATION\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: include/json/version.h\n// //////////////////////////////////////////////////////////////////////\n\n// DO NOT EDIT. This file (and \"version\") is generated by CMake.\n// Run CMake configure step to update it.\n#ifndef JSON_VERSION_H_INCLUDED\n# define JSON_VERSION_H_INCLUDED\n\n# define JSONCPP_VERSION_STRING \"1.6.5\"\n# define JSONCPP_VERSION_MAJOR 1\n# define JSONCPP_VERSION_MINOR 6\n# define JSONCPP_VERSION_PATCH 5\n# define JSONCPP_VERSION_QUALIFIER\n# define JSONCPP_VERSION_HEXA ((JSONCPP_VERSION_MAJOR << 24) | (JSONCPP_VERSION_MINOR << 16) | (JSONCPP_VERSION_PATCH << 8))\n\n#endif // JSON_VERSION_H_INCLUDED\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: include/json/version.h\n// //////////////////////////////////////////////////////////////////////\n\n\n\n\n\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: include/json/config.h\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2007-2010 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n#ifndef JSON_CONFIG_H_INCLUDED\n#define JSON_CONFIG_H_INCLUDED\n\n/// If defined, indicates that json library is embedded in CppTL library.\n//# define JSON_IN_CPPTL 1\n\n/// If defined, indicates that json may leverage CppTL library\n//#  define JSON_USE_CPPTL 1\n/// If defined, indicates that cpptl vector based map should be used instead of\n/// std::map\n/// as Value container.\n//#  define JSON_USE_CPPTL_SMALLMAP 1\n\n// If non-zero, the library uses exceptions to report bad input instead of C\n// assertion macros. The default is to use exceptions.\n#ifndef JSON_USE_EXCEPTION\n#define JSON_USE_EXCEPTION 1\n#endif\n\n/// If defined, indicates that the source file is amalgated\n/// to prevent private header inclusion.\n/// Remarks: it is automatically defined in the generated amalgated header.\n// #define JSON_IS_AMALGAMATION\n\n#ifdef JSON_IN_CPPTL\n#include <cpptl/config.h>\n#ifndef JSON_USE_CPPTL\n#define JSON_USE_CPPTL 1\n#endif\n#endif\n\n#ifdef JSON_IN_CPPTL\n#define JSON_API CPPTL_API\n#elif defined(JSON_DLL_BUILD)\n#if defined(_MSC_VER)\n#define JSON_API __declspec(dllexport)\n#define JSONCPP_DISABLE_DLL_INTERFACE_WARNING\n#endif // if defined(_MSC_VER)\n#elif defined(JSON_DLL)\n#if defined(_MSC_VER)\n#define JSON_API __declspec(dllimport)\n#define JSONCPP_DISABLE_DLL_INTERFACE_WARNING\n#endif // if defined(_MSC_VER)\n#endif // ifdef JSON_IN_CPPTL\n#if !defined(JSON_API)\n#define JSON_API\n#endif\n\n// If JSON_NO_INT64 is defined, then Json only support C++ \"int\" type for\n// integer\n// Storages, and 64 bits integer support is disabled.\n// #define JSON_NO_INT64 1\n\n#if defined(_MSC_VER) // MSVC\n#  if _MSC_VER <= 1200 // MSVC 6\n    // Microsoft Visual Studio 6 only support conversion from __int64 to double\n    // (no conversion from unsigned __int64).\n#    define JSON_USE_INT64_DOUBLE_CONVERSION 1\n    // Disable warning 4786 for VS6 caused by STL (identifier was truncated to '255'\n    // characters in the debug information)\n    // All projects I've ever seen with VS6 were using this globally (not bothering\n    // with pragma push/pop).\n#    pragma warning(disable : 4786)\n#  endif // MSVC 6\n\n#  if _MSC_VER >= 1500 // MSVC 2008\n    /// Indicates that the following function is deprecated.\n#    define JSONCPP_DEPRECATED(message) __declspec(deprecated(message))\n#  endif\n\n#endif // defined(_MSC_VER)\n\n\n#ifndef JSON_HAS_RVALUE_REFERENCES\n\n#if defined(_MSC_VER) && _MSC_VER >= 1600 // MSVC >= 2010\n#define JSON_HAS_RVALUE_REFERENCES 1\n#endif // MSVC >= 2010\n\n#ifdef __clang__\n#if __has_feature(cxx_rvalue_references)\n#define JSON_HAS_RVALUE_REFERENCES 1\n#endif  // has_feature\n\n#elif defined __GNUC__ // not clang (gcc comes later since clang emulates gcc)\n#if defined(__GXX_EXPERIMENTAL_CXX0X__) || (__cplusplus >= 201103L)\n#define JSON_HAS_RVALUE_REFERENCES 1\n#endif  // GXX_EXPERIMENTAL\n\n#endif // __clang__ || __GNUC__\n\n#endif // not defined JSON_HAS_RVALUE_REFERENCES\n\n#ifndef JSON_HAS_RVALUE_REFERENCES\n#define JSON_HAS_RVALUE_REFERENCES 0\n#endif\n\n#ifdef __clang__\n#elif defined __GNUC__ // not clang (gcc comes later since clang emulates gcc)\n#  if (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 5))\n#    define JSONCPP_DEPRECATED(message)  __attribute__ ((deprecated(message)))\n#  elif (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 1))\n#    define JSONCPP_DEPRECATED(message)  __attribute__((__deprecated__))\n#  endif  // GNUC version\n#endif // __clang__ || __GNUC__\n\n#if !defined(JSONCPP_DEPRECATED)\n#define JSONCPP_DEPRECATED(message)\n#endif // if !defined(JSONCPP_DEPRECATED)\n\nnamespace Json {\ntypedef int Int;\ntypedef unsigned int UInt;\n#if defined(JSON_NO_INT64)\ntypedef int LargestInt;\ntypedef unsigned int LargestUInt;\n#undef JSON_HAS_INT64\n#else                 // if defined(JSON_NO_INT64)\n// For Microsoft Visual use specific types as long long is not supported\n#if defined(_MSC_VER) // Microsoft Visual Studio\ntypedef __int64 Int64;\ntypedef unsigned __int64 UInt64;\n#else                 // if defined(_MSC_VER) // Other platforms, use long long\ntypedef long long int Int64;\ntypedef unsigned long long int UInt64;\n#endif // if defined(_MSC_VER)\ntypedef Int64 LargestInt;\ntypedef UInt64 LargestUInt;\n#define JSON_HAS_INT64\n#endif // if defined(JSON_NO_INT64)\n} // end namespace Json\n\n#endif // JSON_CONFIG_H_INCLUDED\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: include/json/config.h\n// //////////////////////////////////////////////////////////////////////\n\n\n\n\n\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: include/json/forwards.h\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2007-2010 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n#ifndef JSON_FORWARDS_H_INCLUDED\n#define JSON_FORWARDS_H_INCLUDED\n\n#if !defined(JSON_IS_AMALGAMATION)\n#include \"config.h\"\n#endif // if !defined(JSON_IS_AMALGAMATION)\n\nnamespace Json {\n\n// writer.h\nclass FastWriter;\nclass StyledWriter;\n\n// reader.h\nclass Reader;\n\n// features.h\nclass Features;\n\n// value.h\ntypedef unsigned int ArrayIndex;\nclass StaticString;\nclass Path;\nclass PathArgument;\nclass Value;\nclass ValueIteratorBase;\nclass ValueIterator;\nclass ValueConstIterator;\n\n} // namespace Json\n\n#endif // JSON_FORWARDS_H_INCLUDED\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: include/json/forwards.h\n// //////////////////////////////////////////////////////////////////////\n\n\n\n\n\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: include/json/features.h\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2007-2010 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n#ifndef CPPTL_JSON_FEATURES_H_INCLUDED\n#define CPPTL_JSON_FEATURES_H_INCLUDED\n\n#if !defined(JSON_IS_AMALGAMATION)\n#include \"forwards.h\"\n#endif // if !defined(JSON_IS_AMALGAMATION)\n\nnamespace Json {\n\n/** \\brief Configuration passed to reader and writer.\n * This configuration object can be used to force the Reader or Writer\n * to behave in a standard conforming way.\n */\nclass JSON_API Features {\npublic:\n  /** \\brief A configuration that allows all features and assumes all strings\n   * are UTF-8.\n   * - C & C++ comments are allowed\n   * - Root object can be any JSON value\n   * - Assumes Value strings are encoded in UTF-8\n   */\n  static Features all();\n\n  /** \\brief A configuration that is strictly compatible with the JSON\n   * specification.\n   * - Comments are forbidden.\n   * - Root object must be either an array or an object value.\n   * - Assumes Value strings are encoded in UTF-8\n   */\n  static Features strictMode();\n\n  /** \\brief Initialize the configuration like JsonConfig::allFeatures;\n   */\n  Features();\n\n  /// \\c true if comments are allowed. Default: \\c true.\n  bool allowComments_;\n\n  /// \\c true if root must be either an array or an object value. Default: \\c\n  /// false.\n  bool strictRoot_;\n\n  /// \\c true if dropped null placeholders are allowed. Default: \\c false.\n  bool allowDroppedNullPlaceholders_;\n\n  /// \\c true if numeric object key are allowed. Default: \\c false.\n  bool allowNumericKeys_;\n};\n\n} // namespace Json\n\n#endif // CPPTL_JSON_FEATURES_H_INCLUDED\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: include/json/features.h\n// //////////////////////////////////////////////////////////////////////\n\n\n\n\n\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: include/json/value.h\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2007-2010 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n#ifndef CPPTL_JSON_H_INCLUDED\n#define CPPTL_JSON_H_INCLUDED\n\n#if !defined(JSON_IS_AMALGAMATION)\n#include \"forwards.h\"\n#endif // if !defined(JSON_IS_AMALGAMATION)\n#include <string>\n#include <vector>\n#include <exception>\n\n#ifndef JSON_USE_CPPTL_SMALLMAP\n#include <map>\n#else\n#include <cpptl/smallmap.h>\n#endif\n#ifdef JSON_USE_CPPTL\n#include <cpptl/forwards.h>\n#endif\n\n// Disable warning C4251: <data member>: <type> needs to have dll-interface to\n// be used by...\n#if defined(JSONCPP_DISABLE_DLL_INTERFACE_WARNING)\n#pragma warning(push)\n#pragma warning(disable : 4251)\n#endif // if defined(JSONCPP_DISABLE_DLL_INTERFACE_WARNING)\n\n/** \\brief JSON (JavaScript Object Notation).\n */\nnamespace Json {\n\n/** Base class for all exceptions we throw.\n *\n * We use nothing but these internally. Of course, STL can throw others.\n */\nclass JSON_API Exception : public std::exception {\npublic:\n  Exception(std::string const& msg);\n  ~Exception() throw() override;\n  char const* what() const throw() override;\nprotected:\n  std::string msg_;\n};\n\n/** Exceptions which the user cannot easily avoid.\n *\n * E.g. out-of-memory (when we use malloc), stack-overflow, malicious input\n * \n * \\remark derived from Json::Exception\n */\nclass JSON_API RuntimeError : public Exception {\npublic:\n  RuntimeError(std::string const& msg);\n};\n\n/** Exceptions thrown by JSON_ASSERT/JSON_FAIL macros.\n *\n * These are precondition-violations (user bugs) and internal errors (our bugs).\n * \n * \\remark derived from Json::Exception\n */\nclass JSON_API LogicError : public Exception {\npublic:\n  LogicError(std::string const& msg);\n};\n\n/// used internally\nvoid throwRuntimeError(std::string const& msg);\n/// used internally\nvoid throwLogicError(std::string const& msg);\n\n/** \\brief Type of the value held by a Value object.\n */\nenum ValueType {\n  nullValue = 0, ///< 'null' value\n  intValue,      ///< signed integer value\n  uintValue,     ///< unsigned integer value\n  realValue,     ///< double value\n  stringValue,   ///< UTF-8 string value\n  booleanValue,  ///< bool value\n  arrayValue,    ///< array value (ordered list)\n  objectValue    ///< object value (collection of name/value pairs).\n};\n\nenum CommentPlacement {\n  commentBefore = 0,      ///< a comment placed on the line before a value\n  commentAfterOnSameLine, ///< a comment just after a value on the same line\n  commentAfter, ///< a comment on the line after a value (only make sense for\n  /// root value)\n  numberOfCommentPlacement\n};\n\n//# ifdef JSON_USE_CPPTL\n//   typedef CppTL::AnyEnumerator<const char *> EnumMemberNames;\n//   typedef CppTL::AnyEnumerator<const Value &> EnumValues;\n//# endif\n\n/** \\brief Lightweight wrapper to tag static string.\n *\n * Value constructor and objectValue member assignement takes advantage of the\n * StaticString and avoid the cost of string duplication when storing the\n * string or the member name.\n *\n * Example of usage:\n * \\code\n * Json::Value aValue( StaticString(\"some text\") );\n * Json::Value object;\n * static const StaticString code(\"code\");\n * object[code] = 1234;\n * \\endcode\n */\nclass JSON_API StaticString {\npublic:\n  explicit StaticString(const char* czstring) : c_str_(czstring) {}\n\n  operator const char*() const { return c_str_; }\n\n  const char* c_str() const { return c_str_; }\n\nprivate:\n  const char* c_str_;\n};\n\n/** \\brief Represents a <a HREF=\"http://www.json.org\">JSON</a> value.\n *\n * This class is a discriminated union wrapper that can represents a:\n * - signed integer [range: Value::minInt - Value::maxInt]\n * - unsigned integer (range: 0 - Value::maxUInt)\n * - double\n * - UTF-8 string\n * - boolean\n * - 'null'\n * - an ordered list of Value\n * - collection of name/value pairs (javascript object)\n *\n * The type of the held value is represented by a #ValueType and\n * can be obtained using type().\n *\n * Values of an #objectValue or #arrayValue can be accessed using operator[]()\n * methods.\n * Non-const methods will automatically create the a #nullValue element\n * if it does not exist.\n * The sequence of an #arrayValue will be automatically resized and initialized\n * with #nullValue. resize() can be used to enlarge or truncate an #arrayValue.\n *\n * The get() methods can be used to obtain default value in the case the\n * required element does not exist.\n *\n * It is possible to iterate over the list of a #objectValue values using\n * the getMemberNames() method.\n *\n * \\note #Value string-length fit in size_t, but keys must be < 2^30.\n * (The reason is an implementation detail.) A #CharReader will raise an\n * exception if a bound is exceeded to avoid security holes in your app,\n * but the Value API does *not* check bounds. That is the responsibility\n * of the caller.\n */\nclass JSON_API Value {\n  friend class ValueIteratorBase;\npublic:\n  typedef std::vector<std::string> Members;\n  typedef ValueIterator iterator;\n  typedef ValueConstIterator const_iterator;\n  typedef Json::UInt UInt;\n  typedef Json::Int Int;\n#if defined(JSON_HAS_INT64)\n  typedef Json::UInt64 UInt64;\n  typedef Json::Int64 Int64;\n#endif // defined(JSON_HAS_INT64)\n  typedef Json::LargestInt LargestInt;\n  typedef Json::LargestUInt LargestUInt;\n  typedef Json::ArrayIndex ArrayIndex;\n\n  static const Value& null;  ///< We regret this reference to a global instance; prefer the simpler Value().\n  static const Value& nullRef;  ///< just a kludge for binary-compatibility; same as null\n  /// Minimum signed integer value that can be stored in a Json::Value.\n  static const LargestInt minLargestInt;\n  /// Maximum signed integer value that can be stored in a Json::Value.\n  static const LargestInt maxLargestInt;\n  /// Maximum unsigned integer value that can be stored in a Json::Value.\n  static const LargestUInt maxLargestUInt;\n\n  /// Minimum signed int value that can be stored in a Json::Value.\n  static const Int minInt;\n  /// Maximum signed int value that can be stored in a Json::Value.\n  static const Int maxInt;\n  /// Maximum unsigned int value that can be stored in a Json::Value.\n  static const UInt maxUInt;\n\n#if defined(JSON_HAS_INT64)\n  /// Minimum signed 64 bits int value that can be stored in a Json::Value.\n  static const Int64 minInt64;\n  /// Maximum signed 64 bits int value that can be stored in a Json::Value.\n  static const Int64 maxInt64;\n  /// Maximum unsigned 64 bits int value that can be stored in a Json::Value.\n  static const UInt64 maxUInt64;\n#endif // defined(JSON_HAS_INT64)\n\nprivate:\n#ifndef JSONCPP_DOC_EXCLUDE_IMPLEMENTATION\n  class CZString {\n  public:\n    enum DuplicationPolicy {\n      noDuplication = 0,\n      duplicate,\n      duplicateOnCopy\n    };\n    CZString(ArrayIndex index);\n    CZString(char const* str, unsigned length, DuplicationPolicy allocate);\n    CZString(CZString const& other);\n#if JSON_HAS_RVALUE_REFERENCES\n    CZString(CZString&& other);\n#endif\n    ~CZString();\n    CZString& operator=(CZString other);\n    bool operator<(CZString const& other) const;\n    bool operator==(CZString const& other) const;\n    ArrayIndex index() const;\n    //const char* c_str() const; ///< \\deprecated\n    char const* data() const;\n    unsigned length() const;\n    bool isStaticString() const;\n\n  private:\n    void swap(CZString& other);\n\n    struct StringStorage {\n      unsigned policy_: 2;\n      unsigned length_: 30; // 1GB max\n    };\n\n    char const* cstr_;  // actually, a prefixed string, unless policy is noDup\n    union {\n      ArrayIndex index_;\n      StringStorage storage_;\n    };\n  };\n\npublic:\n#ifndef JSON_USE_CPPTL_SMALLMAP\n  typedef std::map<CZString, Value> ObjectValues;\n#else\n  typedef CppTL::SmallMap<CZString, Value> ObjectValues;\n#endif // ifndef JSON_USE_CPPTL_SMALLMAP\n#endif // ifndef JSONCPP_DOC_EXCLUDE_IMPLEMENTATION\n\npublic:\n  /** \\brief Create a default Value of the given type.\n\n    This is a very useful constructor.\n    To create an empty array, pass arrayValue.\n    To create an empty object, pass objectValue.\n    Another Value can then be set to this one by assignment.\nThis is useful since clear() and resize() will not alter types.\n\n    Examples:\n\\code\nJson::Value null_value; // null\nJson::Value arr_value(Json::arrayValue); // []\nJson::Value obj_value(Json::objectValue); // {}\n\\endcode\n  */\n  Value(ValueType type = nullValue);\n  Value(Int value);\n  Value(UInt value);\n#if defined(JSON_HAS_INT64)\n  Value(Int64 value);\n  Value(UInt64 value);\n#endif // if defined(JSON_HAS_INT64)\n  Value(double value);\n  Value(const char* value); ///< Copy til first 0. (NULL causes to seg-fault.)\n  Value(const char* begin, const char* end); ///< Copy all, incl zeroes.\n  /** \\brief Constructs a value from a static string.\n\n   * Like other value string constructor but do not duplicate the string for\n   * internal storage. The given string must remain alive after the call to this\n   * constructor.\n   * \\note This works only for null-terminated strings. (We cannot change the\n   *   size of this class, so we have nowhere to store the length,\n   *   which might be computed later for various operations.)\n   *\n   * Example of usage:\n   * \\code\n   * static StaticString foo(\"some text\");\n   * Json::Value aValue(foo);\n   * \\endcode\n   */\n  Value(const StaticString& value);\n  Value(const std::string& value); ///< Copy data() til size(). Embedded zeroes too.\n#ifdef JSON_USE_CPPTL\n  Value(const CppTL::ConstString& value);\n#endif\n  Value(bool value);\n  /// Deep copy.\n  Value(const Value& other);\n#if JSON_HAS_RVALUE_REFERENCES\n  /// Move constructor\n  Value(Value&& other);\n#endif\n  ~Value();\n\n  /// Deep copy, then swap(other).\n  /// \\note Over-write existing comments. To preserve comments, use #swapPayload().\n  Value& operator=(Value other);\n  /// Swap everything.\n  void swap(Value& other);\n  /// Swap values but leave comments and source offsets in place.\n  void swapPayload(Value& other);\n\n  ValueType type() const;\n\n  /// Compare payload only, not comments etc.\n  bool operator<(const Value& other) const;\n  bool operator<=(const Value& other) const;\n  bool operator>=(const Value& other) const;\n  bool operator>(const Value& other) const;\n  bool operator==(const Value& other) const;\n  bool operator!=(const Value& other) const;\n  int compare(const Value& other) const;\n\n  const char* asCString() const; ///< Embedded zeroes could cause you trouble!\n  std::string asString() const; ///< Embedded zeroes are possible.\n  /** Get raw char* of string-value.\n   *  \\return false if !string. (Seg-fault if str or end are NULL.)\n   */\n  bool getString(\n      char const** begin, char const** end) const;\n#ifdef JSON_USE_CPPTL\n  CppTL::ConstString asConstString() const;\n#endif\n  Int asInt() const;\n  UInt asUInt() const;\n#if defined(JSON_HAS_INT64)\n  Int64 asInt64() const;\n  UInt64 asUInt64() const;\n#endif // if defined(JSON_HAS_INT64)\n  LargestInt asLargestInt() const;\n  LargestUInt asLargestUInt() const;\n  float asFloat() const;\n  double asDouble() const;\n  bool asBool() const;\n\n  bool isNull() const;\n  bool isBool() const;\n  bool isInt() const;\n  bool isInt64() const;\n  bool isUInt() const;\n  bool isUInt64() const;\n  bool isIntegral() const;\n  bool isDouble() const;\n  bool isNumeric() const;\n  bool isString() const;\n  bool isArray() const;\n  bool isObject() const;\n\n  bool isConvertibleTo(ValueType other) const;\n\n  /// Number of values in array or object\n  ArrayIndex size() const;\n\n  /// \\brief Return true if empty array, empty object, or null;\n  /// otherwise, false.\n  bool empty() const;\n\n  /// Return isNull()\n  bool operator!() const;\n\n  /// Remove all object members and array elements.\n  /// \\pre type() is arrayValue, objectValue, or nullValue\n  /// \\post type() is unchanged\n  void clear();\n\n  /// Resize the array to size elements.\n  /// New elements are initialized to null.\n  /// May only be called on nullValue or arrayValue.\n  /// \\pre type() is arrayValue or nullValue\n  /// \\post type() is arrayValue\n  void resize(ArrayIndex size);\n\n  /// Access an array element (zero based index ).\n  /// If the array contains less than index element, then null value are\n  /// inserted\n  /// in the array so that its size is index+1.\n  /// (You may need to say 'value[0u]' to get your compiler to distinguish\n  ///  this from the operator[] which takes a string.)\n  Value& operator[](ArrayIndex index);\n\n  /// Access an array element (zero based index ).\n  /// If the array contains less than index element, then null value are\n  /// inserted\n  /// in the array so that its size is index+1.\n  /// (You may need to say 'value[0u]' to get your compiler to distinguish\n  ///  this from the operator[] which takes a string.)\n  Value& operator[](int index);\n\n  /// Access an array element (zero based index )\n  /// (You may need to say 'value[0u]' to get your compiler to distinguish\n  ///  this from the operator[] which takes a string.)\n  const Value& operator[](ArrayIndex index) const;\n\n  /// Access an array element (zero based index )\n  /// (You may need to say 'value[0u]' to get your compiler to distinguish\n  ///  this from the operator[] which takes a string.)\n  const Value& operator[](int index) const;\n\n  /// If the array contains at least index+1 elements, returns the element\n  /// value,\n  /// otherwise returns defaultValue.\n  Value get(ArrayIndex index, const Value& defaultValue) const;\n  /// Return true if index < size().\n  bool isValidIndex(ArrayIndex index) const;\n  /// \\brief Append value to array at the end.\n  ///\n  /// Equivalent to jsonvalue[jsonvalue.size()] = value;\n  Value& append(const Value& value);\n\n  /// Access an object value by name, create a null member if it does not exist.\n  /// \\note Because of our implementation, keys are limited to 2^30 -1 chars.\n  ///  Exceeding that will cause an exception.\n  Value& operator[](const char* key);\n  /// Access an object value by name, returns null if there is no member with\n  /// that name.\n  const Value& operator[](const char* key) const;\n  /// Access an object value by name, create a null member if it does not exist.\n  /// \\param key may contain embedded nulls.\n  Value& operator[](const std::string& key);\n  /// Access an object value by name, returns null if there is no member with\n  /// that name.\n  /// \\param key may contain embedded nulls.\n  const Value& operator[](const std::string& key) const;\n  /** \\brief Access an object value by name, create a null member if it does not\n   exist.\n\n   * If the object has no entry for that name, then the member name used to store\n   * the new entry is not duplicated.\n   * Example of use:\n   * \\code\n   * Json::Value object;\n   * static const StaticString code(\"code\");\n   * object[code] = 1234;\n   * \\endcode\n   */\n  Value& operator[](const StaticString& key);\n#ifdef JSON_USE_CPPTL\n  /// Access an object value by name, create a null member if it does not exist.\n  Value& operator[](const CppTL::ConstString& key);\n  /// Access an object value by name, returns null if there is no member with\n  /// that name.\n  const Value& operator[](const CppTL::ConstString& key) const;\n#endif\n  /// Return the member named key if it exist, defaultValue otherwise.\n  /// \\note deep copy\n  Value get(const char* key, const Value& defaultValue) const;\n  /// Return the member named key if it exist, defaultValue otherwise.\n  /// \\note deep copy\n  /// \\note key may contain embedded nulls.\n  Value get(const char* begin, const char* end, const Value& defaultValue) const;\n  /// Return the member named key if it exist, defaultValue otherwise.\n  /// \\note deep copy\n  /// \\param key may contain embedded nulls.\n  Value get(const std::string& key, const Value& defaultValue) const;\n#ifdef JSON_USE_CPPTL\n  /// Return the member named key if it exist, defaultValue otherwise.\n  /// \\note deep copy\n  Value get(const CppTL::ConstString& key, const Value& defaultValue) const;\n#endif\n  /// Most general and efficient version of isMember()const, get()const,\n  /// and operator[]const\n  /// \\note As stated elsewhere, behavior is undefined if (end-begin) >= 2^30\n  Value const* find(char const* begin, char const* end) const;\n  /// Most general and efficient version of object-mutators.\n  /// \\note As stated elsewhere, behavior is undefined if (end-begin) >= 2^30\n  /// \\return non-zero, but JSON_ASSERT if this is neither object nor nullValue.\n  Value const* demand(char const* begin, char const* end);\n  /// \\brief Remove and return the named member.\n  ///\n  /// Do nothing if it did not exist.\n  /// \\return the removed Value, or null.\n  /// \\pre type() is objectValue or nullValue\n  /// \\post type() is unchanged\n  /// \\deprecated\n  Value removeMember(const char* key);\n  /// Same as removeMember(const char*)\n  /// \\param key may contain embedded nulls.\n  /// \\deprecated\n  Value removeMember(const std::string& key);\n  /// Same as removeMember(const char* begin, const char* end, Value* removed),\n  /// but 'key' is null-terminated.\n  bool removeMember(const char* key, Value* removed);\n  /** \\brief Remove the named map member.\n\n      Update 'removed' iff removed.\n      \\param key may contain embedded nulls.\n      \\return true iff removed (no exceptions)\n  */\n  bool removeMember(std::string const& key, Value* removed);\n  /// Same as removeMember(std::string const& key, Value* removed)\n  bool removeMember(const char* begin, const char* end, Value* removed);\n  /** \\brief Remove the indexed array element.\n\n      O(n) expensive operations.\n      Update 'removed' iff removed.\n      \\return true iff removed (no exceptions)\n  */\n  bool removeIndex(ArrayIndex i, Value* removed);\n\n  /// Return true if the object has a member named key.\n  /// \\note 'key' must be null-terminated.\n  bool isMember(const char* key) const;\n  /// Return true if the object has a member named key.\n  /// \\param key may contain embedded nulls.\n  bool isMember(const std::string& key) const;\n  /// Same as isMember(std::string const& key)const\n  bool isMember(const char* begin, const char* end) const;\n#ifdef JSON_USE_CPPTL\n  /// Return true if the object has a member named key.\n  bool isMember(const CppTL::ConstString& key) const;\n#endif\n\n  /// \\brief Return a list of the member names.\n  ///\n  /// If null, return an empty list.\n  /// \\pre type() is objectValue or nullValue\n  /// \\post if type() was nullValue, it remains nullValue\n  Members getMemberNames() const;\n\n  //# ifdef JSON_USE_CPPTL\n  //      EnumMemberNames enumMemberNames() const;\n  //      EnumValues enumValues() const;\n  //# endif\n\n  /// \\deprecated Always pass len.\n  JSONCPP_DEPRECATED(\"Use setComment(std::string const&) instead.\")\n  void setComment(const char* comment, CommentPlacement placement);\n  /// Comments must be //... or /* ... */\n  void setComment(const char* comment, size_t len, CommentPlacement placement);\n  /// Comments must be //... or /* ... */\n  void setComment(const std::string& comment, CommentPlacement placement);\n  bool hasComment(CommentPlacement placement) const;\n  /// Include delimiters and embedded newlines.\n  std::string getComment(CommentPlacement placement) const;\n\n  std::string toStyledString() const;\n\n  const_iterator begin() const;\n  const_iterator end() const;\n\n  iterator begin();\n  iterator end();\n\n  // Accessors for the [start, limit) range of bytes within the JSON text from\n  // which this value was parsed, if any.\n  void setOffsetStart(size_t start);\n  void setOffsetLimit(size_t limit);\n  size_t getOffsetStart() const;\n  size_t getOffsetLimit() const;\n\nprivate:\n  void initBasic(ValueType type, bool allocated = false);\n\n  Value& resolveReference(const char* key);\n  Value& resolveReference(const char* key, const char* end);\n\n  struct CommentInfo {\n    CommentInfo();\n    ~CommentInfo();\n\n    void setComment(const char* text, size_t len);\n\n    char* comment_;\n  };\n\n  // struct MemberNamesTransform\n  //{\n  //   typedef const char *result_type;\n  //   const char *operator()( const CZString &name ) const\n  //   {\n  //      return name.c_str();\n  //   }\n  //};\n\n  union ValueHolder {\n    LargestInt int_;\n    LargestUInt uint_;\n    double real_;\n    bool bool_;\n    char* string_;  // actually ptr to unsigned, followed by str, unless !allocated_\n    ObjectValues* map_;\n  } value_;\n  ValueType type_ : 8;\n  unsigned int allocated_ : 1; // Notes: if declared as bool, bitfield is useless.\n                               // If not allocated_, string_ must be null-terminated.\n  CommentInfo* comments_;\n\n  // [start, limit) byte offsets in the source JSON text from which this Value\n  // was extracted.\n  size_t start_;\n  size_t limit_;\n};\n\n/** \\brief Experimental and untested: represents an element of the \"path\" to\n * access a node.\n */\nclass JSON_API PathArgument {\npublic:\n  friend class Path;\n\n  PathArgument();\n  PathArgument(ArrayIndex index);\n  PathArgument(const char* key);\n  PathArgument(const std::string& key);\n\nprivate:\n  enum Kind {\n    kindNone = 0,\n    kindIndex,\n    kindKey\n  };\n  std::string key_;\n  ArrayIndex index_;\n  Kind kind_;\n};\n\n/** \\brief Experimental and untested: represents a \"path\" to access a node.\n *\n * Syntax:\n * - \".\" => root node\n * - \".[n]\" => elements at index 'n' of root node (an array value)\n * - \".name\" => member named 'name' of root node (an object value)\n * - \".name1.name2.name3\"\n * - \".[0][1][2].name1[3]\"\n * - \".%\" => member name is provided as parameter\n * - \".[%]\" => index is provied as parameter\n */\nclass JSON_API Path {\npublic:\n  Path(const std::string& path,\n       const PathArgument& a1 = PathArgument(),\n       const PathArgument& a2 = PathArgument(),\n       const PathArgument& a3 = PathArgument(),\n       const PathArgument& a4 = PathArgument(),\n       const PathArgument& a5 = PathArgument());\n\n  const Value& resolve(const Value& root) const;\n  Value resolve(const Value& root, const Value& defaultValue) const;\n  /// Creates the \"path\" to access the specified node and returns a reference on\n  /// the node.\n  Value& make(Value& root) const;\n\nprivate:\n  typedef std::vector<const PathArgument*> InArgs;\n  typedef std::vector<PathArgument> Args;\n\n  void makePath(const std::string& path, const InArgs& in);\n  void addPathInArg(const std::string& path,\n                    const InArgs& in,\n                    InArgs::const_iterator& itInArg,\n                    PathArgument::Kind kind);\n  void invalidPath(const std::string& path, int location);\n\n  Args args_;\n};\n\n/** \\brief base class for Value iterators.\n *\n */\nclass JSON_API ValueIteratorBase {\npublic:\n  typedef std::bidirectional_iterator_tag iterator_category;\n  typedef unsigned int size_t;\n  typedef int difference_type;\n  typedef ValueIteratorBase SelfType;\n\n  bool operator==(const SelfType& other) const { return isEqual(other); }\n\n  bool operator!=(const SelfType& other) const { return !isEqual(other); }\n\n  difference_type operator-(const SelfType& other) const {\n    return other.computeDistance(*this);\n  }\n\n  /// Return either the index or the member name of the referenced value as a\n  /// Value.\n  Value key() const;\n\n  /// Return the index of the referenced Value, or -1 if it is not an arrayValue.\n  UInt index() const;\n\n  /// Return the member name of the referenced Value, or \"\" if it is not an\n  /// objectValue.\n  /// \\note Avoid `c_str()` on result, as embedded zeroes are possible.\n  std::string name() const;\n\n  /// Return the member name of the referenced Value. \"\" if it is not an\n  /// objectValue.\n  /// \\deprecated This cannot be used for UTF-8 strings, since there can be embedded nulls.\n  JSONCPP_DEPRECATED(\"Use `key = name();` instead.\")\n  char const* memberName() const;\n  /// Return the member name of the referenced Value, or NULL if it is not an\n  /// objectValue.\n  /// \\note Better version than memberName(). Allows embedded nulls.\n  char const* memberName(char const** end) const;\n\nprotected:\n  Value& deref() const;\n\n  void increment();\n\n  void decrement();\n\n  difference_type computeDistance(const SelfType& other) const;\n\n  bool isEqual(const SelfType& other) const;\n\n  void copy(const SelfType& other);\n\nprivate:\n  Value::ObjectValues::iterator current_;\n  // Indicates that iterator is for a null value.\n  bool isNull_;\n\npublic:\n  // For some reason, BORLAND needs these at the end, rather\n  // than earlier. No idea why.\n  ValueIteratorBase();\n  explicit ValueIteratorBase(const Value::ObjectValues::iterator& current);\n};\n\n/** \\brief const iterator for object and array value.\n *\n */\nclass JSON_API ValueConstIterator : public ValueIteratorBase {\n  friend class Value;\n\npublic:\n  typedef const Value value_type;\n  //typedef unsigned int size_t;\n  //typedef int difference_type;\n  typedef const Value& reference;\n  typedef const Value* pointer;\n  typedef ValueConstIterator SelfType;\n\n  ValueConstIterator();\n  ValueConstIterator(ValueIterator const& other);\n\nprivate:\n/*! \\internal Use by Value to create an iterator.\n */\n  explicit ValueConstIterator(const Value::ObjectValues::iterator& current);\npublic:\n  SelfType& operator=(const ValueIteratorBase& other);\n\n  SelfType operator++(int) {\n    SelfType temp(*this);\n    ++*this;\n    return temp;\n  }\n\n  SelfType operator--(int) {\n    SelfType temp(*this);\n    --*this;\n    return temp;\n  }\n\n  SelfType& operator--() {\n    decrement();\n    return *this;\n  }\n\n  SelfType& operator++() {\n    increment();\n    return *this;\n  }\n\n  reference operator*() const { return deref(); }\n\n  pointer operator->() const { return &deref(); }\n};\n\n/** \\brief Iterator for object and array value.\n */\nclass JSON_API ValueIterator : public ValueIteratorBase {\n  friend class Value;\n\npublic:\n  typedef Value value_type;\n  typedef unsigned int size_t;\n  typedef int difference_type;\n  typedef Value& reference;\n  typedef Value* pointer;\n  typedef ValueIterator SelfType;\n\n  ValueIterator();\n  explicit ValueIterator(const ValueConstIterator& other);\n  ValueIterator(const ValueIterator& other);\n\nprivate:\n/*! \\internal Use by Value to create an iterator.\n */\n  explicit ValueIterator(const Value::ObjectValues::iterator& current);\npublic:\n  SelfType& operator=(const SelfType& other);\n\n  SelfType operator++(int) {\n    SelfType temp(*this);\n    ++*this;\n    return temp;\n  }\n\n  SelfType operator--(int) {\n    SelfType temp(*this);\n    --*this;\n    return temp;\n  }\n\n  SelfType& operator--() {\n    decrement();\n    return *this;\n  }\n\n  SelfType& operator++() {\n    increment();\n    return *this;\n  }\n\n  reference operator*() const { return deref(); }\n\n  pointer operator->() const { return &deref(); }\n};\n\n} // namespace Json\n\n\nnamespace std {\n/// Specialize std::swap() for Json::Value.\ntemplate<>\ninline void swap(Json::Value& a, Json::Value& b) { a.swap(b); }\n}\n\n\n#if defined(JSONCPP_DISABLE_DLL_INTERFACE_WARNING)\n#pragma warning(pop)\n#endif // if defined(JSONCPP_DISABLE_DLL_INTERFACE_WARNING)\n\n#endif // CPPTL_JSON_H_INCLUDED\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: include/json/value.h\n// //////////////////////////////////////////////////////////////////////\n\n\n\n\n\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: include/json/reader.h\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2007-2010 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n#ifndef CPPTL_JSON_READER_H_INCLUDED\n#define CPPTL_JSON_READER_H_INCLUDED\n\n#if !defined(JSON_IS_AMALGAMATION)\n#include \"features.h\"\n#include \"value.h\"\n#endif // if !defined(JSON_IS_AMALGAMATION)\n#include <deque>\n#include <iosfwd>\n#include <stack>\n#include <string>\n#include <istream>\n\n// Disable warning C4251: <data member>: <type> needs to have dll-interface to\n// be used by...\n#if defined(JSONCPP_DISABLE_DLL_INTERFACE_WARNING)\n#pragma warning(push)\n#pragma warning(disable : 4251)\n#endif // if defined(JSONCPP_DISABLE_DLL_INTERFACE_WARNING)\n\nnamespace Json {\n\n/** \\brief Unserialize a <a HREF=\"http://www.json.org\">JSON</a> document into a\n *Value.\n *\n * \\deprecated Use CharReader and CharReaderBuilder.\n */\nclass JSON_API Reader {\npublic:\n  typedef char Char;\n  typedef const Char* Location;\n\n  /** \\brief An error tagged with where in the JSON text it was encountered.\n   *\n   * The offsets give the [start, limit) range of bytes within the text. Note\n   * that this is bytes, not codepoints.\n   *\n   */\n  struct StructuredError {\n    size_t offset_start;\n    size_t offset_limit;\n    std::string message;\n  };\n\n  /** \\brief Constructs a Reader allowing all features\n   * for parsing.\n   */\n  Reader();\n\n  /** \\brief Constructs a Reader allowing the specified feature set\n   * for parsing.\n   */\n  Reader(const Features& features);\n\n  /** \\brief Read a Value from a <a HREF=\"http://www.json.org\">JSON</a>\n   * document.\n   * \\param document UTF-8 encoded string containing the document to read.\n   * \\param root [out] Contains the root value of the document if it was\n   *             successfully parsed.\n   * \\param collectComments \\c true to collect comment and allow writing them\n   * back during\n   *                        serialization, \\c false to discard comments.\n   *                        This parameter is ignored if\n   * Features::allowComments_\n   *                        is \\c false.\n   * \\return \\c true if the document was successfully parsed, \\c false if an\n   * error occurred.\n   */\n  bool\n  parse(const std::string& document, Value& root, bool collectComments = true);\n\n  /** \\brief Read a Value from a <a HREF=\"http://www.json.org\">JSON</a>\n   document.\n   * \\param beginDoc Pointer on the beginning of the UTF-8 encoded string of the\n   document to read.\n   * \\param endDoc Pointer on the end of the UTF-8 encoded string of the\n   document to read.\n   *               Must be >= beginDoc.\n   * \\param root [out] Contains the root value of the document if it was\n   *             successfully parsed.\n   * \\param collectComments \\c true to collect comment and allow writing them\n   back during\n   *                        serialization, \\c false to discard comments.\n   *                        This parameter is ignored if\n   Features::allowComments_\n   *                        is \\c false.\n   * \\return \\c true if the document was successfully parsed, \\c false if an\n   error occurred.\n   */\n  bool parse(const char* beginDoc,\n             const char* endDoc,\n             Value& root,\n             bool collectComments = true);\n\n  /// \\brief Parse from input stream.\n  /// \\see Json::operator>>(std::istream&, Json::Value&).\n  bool parse(std::istream& is, Value& root, bool collectComments = true);\n\n  /** \\brief Returns a user friendly string that list errors in the parsed\n   * document.\n   * \\return Formatted error message with the list of errors with their location\n   * in\n   *         the parsed document. An empty string is returned if no error\n   * occurred\n   *         during parsing.\n   * \\deprecated Use getFormattedErrorMessages() instead (typo fix).\n   */\n  JSONCPP_DEPRECATED(\"Use getFormattedErrorMessages() instead.\")\n  std::string getFormatedErrorMessages() const;\n\n  /** \\brief Returns a user friendly string that list errors in the parsed\n   * document.\n   * \\return Formatted error message with the list of errors with their location\n   * in\n   *         the parsed document. An empty string is returned if no error\n   * occurred\n   *         during parsing.\n   */\n  std::string getFormattedErrorMessages() const;\n\n  /** \\brief Returns a vector of structured erros encounted while parsing.\n   * \\return A (possibly empty) vector of StructuredError objects. Currently\n   *         only one error can be returned, but the caller should tolerate\n   * multiple\n   *         errors.  This can occur if the parser recovers from a non-fatal\n   *         parse error and then encounters additional errors.\n   */\n  std::vector<StructuredError> getStructuredErrors() const;\n\n  /** \\brief Add a semantic error message.\n   * \\param value JSON Value location associated with the error\n   * \\param message The error message.\n   * \\return \\c true if the error was successfully added, \\c false if the\n   * Value offset exceeds the document size.\n   */\n  bool pushError(const Value& value, const std::string& message);\n\n  /** \\brief Add a semantic error message with extra context.\n   * \\param value JSON Value location associated with the error\n   * \\param message The error message.\n   * \\param extra Additional JSON Value location to contextualize the error\n   * \\return \\c true if the error was successfully added, \\c false if either\n   * Value offset exceeds the document size.\n   */\n  bool pushError(const Value& value, const std::string& message, const Value& extra);\n\n  /** \\brief Return whether there are any errors.\n   * \\return \\c true if there are no errors to report \\c false if\n   * errors have occurred.\n   */\n  bool good() const;\n\nprivate:\n  enum TokenType {\n    tokenEndOfStream = 0,\n    tokenObjectBegin,\n    tokenObjectEnd,\n    tokenArrayBegin,\n    tokenArrayEnd,\n    tokenString,\n    tokenNumber,\n    tokenTrue,\n    tokenFalse,\n    tokenNull,\n    tokenArraySeparator,\n    tokenMemberSeparator,\n    tokenComment,\n    tokenError\n  };\n\n  class Token {\n  public:\n    TokenType type_;\n    Location start_;\n    Location end_;\n  };\n\n  class ErrorInfo {\n  public:\n    Token token_;\n    std::string message_;\n    Location extra_;\n  };\n\n  typedef std::deque<ErrorInfo> Errors;\n\n  bool readToken(Token& token);\n  void skipSpaces();\n  bool match(Location pattern, int patternLength);\n  bool readComment();\n  bool readCStyleComment();\n  bool readCppStyleComment();\n  bool readString();\n  void readNumber();\n  bool readValue();\n  bool readObject(Token& token);\n  bool readArray(Token& token);\n  bool decodeNumber(Token& token);\n  bool decodeNumber(Token& token, Value& decoded);\n  bool decodeString(Token& token);\n  bool decodeString(Token& token, std::string& decoded);\n  bool decodeDouble(Token& token);\n  bool decodeDouble(Token& token, Value& decoded);\n  bool decodeUnicodeCodePoint(Token& token,\n                              Location& current,\n                              Location end,\n                              unsigned int& unicode);\n  bool decodeUnicodeEscapeSequence(Token& token,\n                                   Location& current,\n                                   Location end,\n                                   unsigned int& unicode);\n  bool addError(const std::string& message, Token& token, Location extra = 0);\n  bool recoverFromError(TokenType skipUntilToken);\n  bool addErrorAndRecover(const std::string& message,\n                          Token& token,\n                          TokenType skipUntilToken);\n  void skipUntilSpace();\n  Value& currentValue();\n  Char getNextChar();\n  void\n  getLocationLineAndColumn(Location location, int& line, int& column) const;\n  std::string getLocationLineAndColumn(Location location) const;\n  void addComment(Location begin, Location end, CommentPlacement placement);\n  void skipCommentTokens(Token& token);\n\n  typedef std::stack<Value*> Nodes;\n  Nodes nodes_;\n  Errors errors_;\n  std::string document_;\n  Location begin_;\n  Location end_;\n  Location current_;\n  Location lastValueEnd_;\n  Value* lastValue_;\n  std::string commentsBefore_;\n  Features features_;\n  bool collectComments_;\n};  // Reader\n\n/** Interface for reading JSON from a char array.\n */\nclass JSON_API CharReader {\npublic:\n  virtual ~CharReader() {}\n  /** \\brief Read a Value from a <a HREF=\"http://www.json.org\">JSON</a>\n   document.\n   * The document must be a UTF-8 encoded string containing the document to read.\n   *\n   * \\param beginDoc Pointer on the beginning of the UTF-8 encoded string of the\n   document to read.\n   * \\param endDoc Pointer on the end of the UTF-8 encoded string of the\n   document to read.\n   *        Must be >= beginDoc.\n   * \\param root [out] Contains the root value of the document if it was\n   *             successfully parsed.\n   * \\param errs [out] Formatted error messages (if not NULL)\n   *        a user friendly string that lists errors in the parsed\n   * document.\n   * \\return \\c true if the document was successfully parsed, \\c false if an\n   error occurred.\n   */\n  virtual bool parse(\n      char const* beginDoc, char const* endDoc,\n      Value* root, std::string* errs) = 0;\n\n  class JSON_API Factory {\n  public:\n    virtual ~Factory() {}\n    /** \\brief Allocate a CharReader via operator new().\n     * \\throw std::exception if something goes wrong (e.g. invalid settings)\n     */\n    virtual CharReader* newCharReader() const = 0;\n  };  // Factory\n};  // CharReader\n\n/** \\brief Build a CharReader implementation.\n\nUsage:\n\\code\n  using namespace Json;\n  CharReaderBuilder builder;\n  builder[\"collectComments\"] = false;\n  Value value;\n  std::string errs;\n  bool ok = parseFromStream(builder, std::cin, &value, &errs);\n\\endcode\n*/\nclass JSON_API CharReaderBuilder : public CharReader::Factory {\npublic:\n  // Note: We use a Json::Value so that we can add data-members to this class\n  // without a major version bump.\n  /** Configuration of this builder.\n    These are case-sensitive.\n    Available settings (case-sensitive):\n    - `\"collectComments\": false or true`\n      - true to collect comment and allow writing them\n        back during serialization, false to discard comments.\n        This parameter is ignored if allowComments is false.\n    - `\"allowComments\": false or true`\n      - true if comments are allowed.\n    - `\"strictRoot\": false or true`\n      - true if root must be either an array or an object value\n    - `\"allowDroppedNullPlaceholders\": false or true`\n      - true if dropped null placeholders are allowed. (See StreamWriterBuilder.)\n    - `\"allowNumericKeys\": false or true`\n      - true if numeric object keys are allowed.\n    - `\"allowSingleQuotes\": false or true`\n      - true if '' are allowed for strings (both keys and values)\n    - `\"stackLimit\": integer`\n      - Exceeding stackLimit (recursive depth of `readValue()`) will\n        cause an exception.\n      - This is a security issue (seg-faults caused by deeply nested JSON),\n        so the default is low.\n    - `\"failIfExtra\": false or true`\n      - If true, `parse()` returns false when extra non-whitespace trails\n        the JSON value in the input string.\n    - `\"rejectDupKeys\": false or true`\n      - If true, `parse()` returns false when a key is duplicated within an object.\n    - `\"allowSpecialFloats\": false or true`\n      - If true, special float values (NaNs and infinities) are allowed \n        and their values are lossfree restorable.\n\n    You can examine 'settings_` yourself\n    to see the defaults. You can also write and read them just like any\n    JSON Value.\n    \\sa setDefaults()\n    */\n  Json::Value settings_;\n\n  CharReaderBuilder();\n  ~CharReaderBuilder() override;\n\n  CharReader* newCharReader() const override;\n\n  /** \\return true if 'settings' are legal and consistent;\n   *   otherwise, indicate bad settings via 'invalid'.\n   */\n  bool validate(Json::Value* invalid) const;\n\n  /** A simple way to update a specific setting.\n   */\n  Value& operator[](std::string key);\n\n  /** Called by ctor, but you can use this to reset settings_.\n   * \\pre 'settings' != NULL (but Json::null is fine)\n   * \\remark Defaults:\n   * \\snippet src/lib_json/json_reader.cpp CharReaderBuilderDefaults\n   */\n  static void setDefaults(Json::Value* settings);\n  /** Same as old Features::strictMode().\n   * \\pre 'settings' != NULL (but Json::null is fine)\n   * \\remark Defaults:\n   * \\snippet src/lib_json/json_reader.cpp CharReaderBuilderStrictMode\n   */\n  static void strictMode(Json::Value* settings);\n};\n\n/** Consume entire stream and use its begin/end.\n  * Someday we might have a real StreamReader, but for now this\n  * is convenient.\n  */\nbool JSON_API parseFromStream(\n    CharReader::Factory const&,\n    std::istream&,\n    Value* root, std::string* errs);\n\n/** \\brief Read from 'sin' into 'root'.\n\n Always keep comments from the input JSON.\n\n This can be used to read a file into a particular sub-object.\n For example:\n \\code\n Json::Value root;\n cin >> root[\"dir\"][\"file\"];\n cout << root;\n \\endcode\n Result:\n \\verbatim\n {\n \"dir\": {\n     \"file\": {\n     // The input stream JSON would be nested here.\n     }\n }\n }\n \\endverbatim\n \\throw std::exception on parse error.\n \\see Json::operator<<()\n*/\nJSON_API std::istream& operator>>(std::istream&, Value&);\n\n} // namespace Json\n\n#if defined(JSONCPP_DISABLE_DLL_INTERFACE_WARNING)\n#pragma warning(pop)\n#endif // if defined(JSONCPP_DISABLE_DLL_INTERFACE_WARNING)\n\n#endif // CPPTL_JSON_READER_H_INCLUDED\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: include/json/reader.h\n// //////////////////////////////////////////////////////////////////////\n\n\n\n\n\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: include/json/writer.h\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2007-2010 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n#ifndef JSON_WRITER_H_INCLUDED\n#define JSON_WRITER_H_INCLUDED\n\n#if !defined(JSON_IS_AMALGAMATION)\n#include \"value.h\"\n#endif // if !defined(JSON_IS_AMALGAMATION)\n#include <vector>\n#include <string>\n#include <ostream>\n\n// Disable warning C4251: <data member>: <type> needs to have dll-interface to\n// be used by...\n#if defined(JSONCPP_DISABLE_DLL_INTERFACE_WARNING)\n#pragma warning(push)\n#pragma warning(disable : 4251)\n#endif // if defined(JSONCPP_DISABLE_DLL_INTERFACE_WARNING)\n\nnamespace Json {\n\nclass Value;\n\n/**\n\nUsage:\n\\code\n  using namespace Json;\n  void writeToStdout(StreamWriter::Factory const& factory, Value const& value) {\n    std::unique_ptr<StreamWriter> const writer(\n      factory.newStreamWriter());\n    writer->write(value, &std::cout);\n    std::cout << std::endl;  // add lf and flush\n  }\n\\endcode\n*/\nclass JSON_API StreamWriter {\nprotected:\n  std::ostream* sout_;  // not owned; will not delete\npublic:\n  StreamWriter();\n  virtual ~StreamWriter();\n  /** Write Value into document as configured in sub-class.\n      Do not take ownership of sout, but maintain a reference during function.\n      \\pre sout != NULL\n      \\return zero on success (For now, we always return zero, so check the stream instead.)\n      \\throw std::exception possibly, depending on configuration\n   */\n  virtual int write(Value const& root, std::ostream* sout) = 0;\n\n  /** \\brief A simple abstract factory.\n   */\n  class JSON_API Factory {\n  public:\n    virtual ~Factory();\n    /** \\brief Allocate a CharReader via operator new().\n     * \\throw std::exception if something goes wrong (e.g. invalid settings)\n     */\n    virtual StreamWriter* newStreamWriter() const = 0;\n  };  // Factory\n};  // StreamWriter\n\n/** \\brief Write into stringstream, then return string, for convenience.\n * A StreamWriter will be created from the factory, used, and then deleted.\n */\nstd::string JSON_API writeString(StreamWriter::Factory const& factory, Value const& root);\n\n\n/** \\brief Build a StreamWriter implementation.\n\nUsage:\n\\code\n  using namespace Json;\n  Value value = ...;\n  StreamWriterBuilder builder;\n  builder[\"commentStyle\"] = \"None\";\n  builder[\"indentation\"] = \"   \";  // or whatever you like\n  std::unique_ptr<Json::StreamWriter> writer(\n      builder.newStreamWriter());\n  writer->write(value, &std::cout);\n  std::cout << std::endl;  // add lf and flush\n\\endcode\n*/\nclass JSON_API StreamWriterBuilder : public StreamWriter::Factory {\npublic:\n  // Note: We use a Json::Value so that we can add data-members to this class\n  // without a major version bump.\n  /** Configuration of this builder.\n    Available settings (case-sensitive):\n    - \"commentStyle\": \"None\" or \"All\"\n    - \"indentation\":  \"<anything>\"\n    - \"enableYAMLCompatibility\": false or true\n      - slightly change the whitespace around colons\n    - \"dropNullPlaceholders\": false or true\n      - Drop the \"null\" string from the writer's output for nullValues.\n        Strictly speaking, this is not valid JSON. But when the output is being\n        fed to a browser's Javascript, it makes for smaller output and the\n        browser can handle the output just fine.\n    - \"useSpecialFloats\": false or true\n      - If true, outputs non-finite floating point values in the following way:\n        NaN values as \"NaN\", positive infinity as \"Infinity\", and negative infinity\n        as \"-Infinity\".\n\n    You can examine 'settings_` yourself\n    to see the defaults. You can also write and read them just like any\n    JSON Value.\n    \\sa setDefaults()\n    */\n  Json::Value settings_;\n\n  StreamWriterBuilder();\n  ~StreamWriterBuilder() override;\n\n  /**\n   * \\throw std::exception if something goes wrong (e.g. invalid settings)\n   */\n  StreamWriter* newStreamWriter() const override;\n\n  /** \\return true if 'settings' are legal and consistent;\n   *   otherwise, indicate bad settings via 'invalid'.\n   */\n  bool validate(Json::Value* invalid) const;\n  /** A simple way to update a specific setting.\n   */\n  Value& operator[](std::string key);\n\n  /** Called by ctor, but you can use this to reset settings_.\n   * \\pre 'settings' != NULL (but Json::null is fine)\n   * \\remark Defaults:\n   * \\snippet src/lib_json/json_writer.cpp StreamWriterBuilderDefaults\n   */\n  static void setDefaults(Json::Value* settings);\n};\n\n/** \\brief Abstract class for writers.\n * \\deprecated Use StreamWriter. (And really, this is an implementation detail.)\n */\nclass JSON_API Writer {\npublic:\n  virtual ~Writer();\n\n  virtual std::string write(const Value& root) = 0;\n};\n\n/** \\brief Outputs a Value in <a HREF=\"http://www.json.org\">JSON</a> format\n *without formatting (not human friendly).\n *\n * The JSON document is written in a single line. It is not intended for 'human'\n *consumption,\n * but may be usefull to support feature such as RPC where bandwith is limited.\n * \\sa Reader, Value\n * \\deprecated Use StreamWriterBuilder.\n */\nclass JSON_API FastWriter : public Writer {\n\npublic:\n  FastWriter();\n  ~FastWriter() override {}\n\n  void enableYAMLCompatibility();\n\n  /** \\brief Drop the \"null\" string from the writer's output for nullValues.\n   * Strictly speaking, this is not valid JSON. But when the output is being\n   * fed to a browser's Javascript, it makes for smaller output and the\n   * browser can handle the output just fine.\n   */\n  void dropNullPlaceholders();\n\n  void omitEndingLineFeed();\n\npublic: // overridden from Writer\n  std::string write(const Value& root) override;\n\nprivate:\n  void writeValue(const Value& value);\n\n  std::string document_;\n  bool yamlCompatiblityEnabled_;\n  bool dropNullPlaceholders_;\n  bool omitEndingLineFeed_;\n};\n\n/** \\brief Writes a Value in <a HREF=\"http://www.json.org\">JSON</a> format in a\n *human friendly way.\n *\n * The rules for line break and indent are as follow:\n * - Object value:\n *     - if empty then print {} without indent and line break\n *     - if not empty the print '{', line break & indent, print one value per\n *line\n *       and then unindent and line break and print '}'.\n * - Array value:\n *     - if empty then print [] without indent and line break\n *     - if the array contains no object value, empty array or some other value\n *types,\n *       and all the values fit on one lines, then print the array on a single\n *line.\n *     - otherwise, it the values do not fit on one line, or the array contains\n *       object or non empty array, then print one value per line.\n *\n * If the Value have comments then they are outputed according to their\n *#CommentPlacement.\n *\n * \\sa Reader, Value, Value::setComment()\n * \\deprecated Use StreamWriterBuilder.\n */\nclass JSON_API StyledWriter : public Writer {\npublic:\n  StyledWriter();\n  ~StyledWriter() override {}\n\npublic: // overridden from Writer\n  /** \\brief Serialize a Value in <a HREF=\"http://www.json.org\">JSON</a> format.\n   * \\param root Value to serialize.\n   * \\return String containing the JSON document that represents the root value.\n   */\n  std::string write(const Value& root) override;\n\nprivate:\n  void writeValue(const Value& value);\n  void writeArrayValue(const Value& value);\n  bool isMultineArray(const Value& value);\n  void pushValue(const std::string& value);\n  void writeIndent();\n  void writeWithIndent(const std::string& value);\n  void indent();\n  void unindent();\n  void writeCommentBeforeValue(const Value& root);\n  void writeCommentAfterValueOnSameLine(const Value& root);\n  bool hasCommentForValue(const Value& value);\n  static std::string normalizeEOL(const std::string& text);\n\n  typedef std::vector<std::string> ChildValues;\n\n  ChildValues childValues_;\n  std::string document_;\n  std::string indentString_;\n  int rightMargin_;\n  int indentSize_;\n  bool addChildValues_;\n};\n\n/** \\brief Writes a Value in <a HREF=\"http://www.json.org\">JSON</a> format in a\n human friendly way,\n     to a stream rather than to a string.\n *\n * The rules for line break and indent are as follow:\n * - Object value:\n *     - if empty then print {} without indent and line break\n *     - if not empty the print '{', line break & indent, print one value per\n line\n *       and then unindent and line break and print '}'.\n * - Array value:\n *     - if empty then print [] without indent and line break\n *     - if the array contains no object value, empty array or some other value\n types,\n *       and all the values fit on one lines, then print the array on a single\n line.\n *     - otherwise, it the values do not fit on one line, or the array contains\n *       object or non empty array, then print one value per line.\n *\n * If the Value have comments then they are outputed according to their\n #CommentPlacement.\n *\n * \\param indentation Each level will be indented by this amount extra.\n * \\sa Reader, Value, Value::setComment()\n * \\deprecated Use StreamWriterBuilder.\n */\nclass JSON_API StyledStreamWriter {\npublic:\n  StyledStreamWriter(std::string indentation = \"\\t\");\n  ~StyledStreamWriter() {}\n\npublic:\n  /** \\brief Serialize a Value in <a HREF=\"http://www.json.org\">JSON</a> format.\n   * \\param out Stream to write to. (Can be ostringstream, e.g.)\n   * \\param root Value to serialize.\n   * \\note There is no point in deriving from Writer, since write() should not\n   * return a value.\n   */\n  void write(std::ostream& out, const Value& root);\n\nprivate:\n  void writeValue(const Value& value);\n  void writeArrayValue(const Value& value);\n  bool isMultineArray(const Value& value);\n  void pushValue(const std::string& value);\n  void writeIndent();\n  void writeWithIndent(const std::string& value);\n  void indent();\n  void unindent();\n  void writeCommentBeforeValue(const Value& root);\n  void writeCommentAfterValueOnSameLine(const Value& root);\n  bool hasCommentForValue(const Value& value);\n  static std::string normalizeEOL(const std::string& text);\n\n  typedef std::vector<std::string> ChildValues;\n\n  ChildValues childValues_;\n  std::ostream* document_;\n  std::string indentString_;\n  int rightMargin_;\n  std::string indentation_;\n  bool addChildValues_ : 1;\n  bool indented_ : 1;\n};\n\n#if defined(JSON_HAS_INT64)\nstd::string JSON_API valueToString(Int value);\nstd::string JSON_API valueToString(UInt value);\n#endif // if defined(JSON_HAS_INT64)\nstd::string JSON_API valueToString(LargestInt value);\nstd::string JSON_API valueToString(LargestUInt value);\nstd::string JSON_API valueToString(double value);\nstd::string JSON_API valueToString(bool value);\nstd::string JSON_API valueToQuotedString(const char* value);\n\n/// \\brief Output using the StyledStreamWriter.\n/// \\see Json::operator>>()\nJSON_API std::ostream& operator<<(std::ostream&, const Value& root);\n\n} // namespace Json\n\n#if defined(JSONCPP_DISABLE_DLL_INTERFACE_WARNING)\n#pragma warning(pop)\n#endif // if defined(JSONCPP_DISABLE_DLL_INTERFACE_WARNING)\n\n#endif // JSON_WRITER_H_INCLUDED\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: include/json/writer.h\n// //////////////////////////////////////////////////////////////////////\n\n\n\n\n\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: include/json/assertions.h\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2007-2010 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n#ifndef CPPTL_JSON_ASSERTIONS_H_INCLUDED\n#define CPPTL_JSON_ASSERTIONS_H_INCLUDED\n\n#include <stdlib.h>\n#include <sstream>\n\n#if !defined(JSON_IS_AMALGAMATION)\n#include \"config.h\"\n#endif // if !defined(JSON_IS_AMALGAMATION)\n\n/** It should not be possible for a maliciously designed file to\n *  cause an abort() or seg-fault, so these macros are used only\n *  for pre-condition violations and internal logic errors.\n */\n#if JSON_USE_EXCEPTION\n\n// @todo <= add detail about condition in exception\n# define JSON_ASSERT(condition)                                                \\\n  {if (!(condition)) {Json::throwLogicError( \"assert json failed\" );}}\n\n# define JSON_FAIL_MESSAGE(message)                                            \\\n  {                                                                            \\\n    std::ostringstream oss; oss << message;                                    \\\n    Json::throwLogicError(oss.str());                                          \\\n    abort();                                                                   \\\n  }\n\n#else // JSON_USE_EXCEPTION\n\n# define JSON_ASSERT(condition) assert(condition)\n\n// The call to assert() will show the failure message in debug builds. In\n// release builds we abort, for a core-dump or debugger.\n# define JSON_FAIL_MESSAGE(message)                                            \\\n  {                                                                            \\\n    std::ostringstream oss; oss << message;                                    \\\n    assert(false && oss.str().c_str());                                        \\\n    abort();                                                                   \\\n  }\n\n\n#endif\n\n#define JSON_ASSERT_MESSAGE(condition, message)                                \\\n  if (!(condition)) {                                                          \\\n    JSON_FAIL_MESSAGE(message);                                                \\\n  }\n\n#endif // CPPTL_JSON_ASSERTIONS_H_INCLUDED\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: include/json/assertions.h\n// //////////////////////////////////////////////////////////////////////\n\n\n\n\n\n#endif //ifndef JSON_AMALGATED_H_INCLUDED\n"
  },
  {
    "path": "cpp-api/include/module.modulemap",
    "content": "module scalaris_Connection {\n       module Connection      { header \"connection.hpp\"  }\n}\n\nmodule scalaris_Converter {\n       module Converter      { header \"converter.hpp\"  }\n}\n\nmodule scalaris_Exceptions {\n       module Exceptions      { header \"exceptions.hpp\"  }\n}\n\nmodule scalaris_Rbr {\n       module Rbr      { header \"rbr.hpp\"  }\n}\n\nmodule scalaris_ReqList {\n       module ReqList      { header \"req-list.hpp\"  }\n}\n\nmodule scalaris_RoutingTable {\n       module RoutingTable      { header \"routing-table.hpp\"  }\n}\n\nmodule scalaris_SSLConnection {\n       module SSLConnection      { header \"ssl-connection.hpp\"  }\n}\n\nmodule scalaris_TCPConnection {\n       module TCPConnection      { header \"tcp-connection.hpp\"  }\n}\n\nmodule scalaris_TransactionSingleOp {\n       module TransactionSingleOp      { header \"transaction-single-op.hpp\"  }\n}\n\n"
  },
  {
    "path": "cpp-api/include/rbr.hpp",
    "content": "// Copyright 2018-2019 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#pragma once\n\n#include <iostream>\n#include <memory>\n#include <stdexcept>\n#include <string>\n\n#include \"connection.hpp\"\n#include \"exceptions.hpp\"\n#include \"json/json.h\"\n\nnamespace scalaris {\n  /// \\brief Executes a single read or write.\n  ///\n  /// It uses kv_on_cseq to perform reads and writes.\n  class Rbr {\n    Connection& c;\n\n  public:\n    /**\n     * Creates a <code>Rbr</code> instance\n     * @param _c the Connection object\n     */\n    Rbr(Connection& _c) : c(_c) {}\n\n    std::string JsontoString(Json::Value v) {\n      std::stringstream sstream;\n      Json::StreamWriterBuilder builder;\n      std::unique_ptr<Json::StreamWriter> writer(builder.newStreamWriter());\n      writer->write(v, &sstream);\n      std::string result = sstream.str();\n      return result;\n    }\n    /**\n     * Reads a key-value pair\n     * @param key the key to lookup in Scalaris\n     */\n    Json::Value read(const std::string key) {\n      Json::Value val = c.rpc(\"rbr_read\", key);\n\n      // std::cout << \"read: \" << Json::StyledWriter().write(val) << std::endl;\n\n      if (!val.isObject())\n        throw MalFormedJsonError();\n      Json::Value status = val[\"status\"];\n\n      if (!status.isString())\n        throw MalFormedJsonError();\n\n      std::string status_str = status.asString();\n      if (status_str.compare(\"ok\") != 0)\n        throw ReadFailedError(val[\"reason\"].asString());\n\n      Json::Value value = val[\"value\"];\n      if (!value.isObject())\n        throw MalFormedJsonError();\n\n      Json::Value value_type = value[\"type\"];\n      if (!value_type.isString())\n        throw MalFormedJsonError();\n      Json::Value value_type_str = value_type.asString();\n      if (value_type_str.compare(\"as_is\") != 0)\n        throw ReadFailedError(\"unsupported value type\");\n\n      Json::Value value_value = value[\"value\"];\n\n      return value_value;\n    }\n\n    /**\n     * Writes a key-value pair\n     * @param key the key to update in Scalaris\n     * @param value the value to store under <code>key</code>\n     */\n    void write(const std::string key, const Json::Value& value) {\n      Json::Value val = c.rpc(\"rbr_write\", key, value);\n\n      // std::cout << \"write: \" << Json::StyledWriter().write(val) << std::endl;\n\n      if (!val.isObject())\n        throw MalFormedJsonError();\n      Json::Value status = val[\"status\"];\n      if (!status.isString())\n        throw MalFormedJsonError();\n      std::string status_str = status.asString();\n      if (status_str.compare(\"ok\") != 0)\n        throw WriteFailedError(val[\"reason\"].asString());\n    }\n  };\n} // namespace scalaris\n"
  },
  {
    "path": "cpp-api/include/req_list.hpp",
    "content": "// Copyright 2015-2018 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#ifndef SCALARIS_REQUESTLIST_HPP\n#define SCALARIS_REQUESTLIST_HPP\n\n#include <iostream>\n#include <string>\n#include <stdexcept>\n\n#include <sys/time.h> // clang modules\n\n#include \"json/json.h\"\n\nnamespace scalaris {\n  class RequestList {\n    Json::Value reqlist;\n    bool _is_commit = false;\n  public:\n    RequestList() : reqlist(Json::arrayValue) {}\n\n    void add_read(const std::string key) {\n      Json::Value read_request = { Json::objectValue };\n      read_request[\"read\"] = key;\n      reqlist.append(read_request);\n    }\n\n    void add_write(const std::string key, const std::string value) {\n      Json::Value write_op = { Json::objectValue };\n      write_op[key] = as_is(value);\n      Json::Value write_request = { Json::objectValue };\n      write_request[\"write\"] = write_op;\n      reqlist.append(write_request);\n\n    }\n    void add_commit() {\n      Json::Value commit_request = { Json::objectValue };\n      commit_request[\"commit\"] = \"\";\n      reqlist.append(commit_request);\n      _is_commit = true;\n    }\n\n    bool is_commit() {\n      return _is_commit;\n    }\n\n    bool is_empty() {\n      return _is_commit;\n    }\n\n    int size() {\n      return reqlist.size();\n    }\n\n    Json::Value get_json_value() const { return reqlist; }\n\n  private:\n    Json::Value as_is(const std::string& val) {\n      Json::Value result = { Json::objectValue };\n      result[\"type\"] = \"as_is\";\n      result[\"value\"] = val;\n\n      return result;\n    }\n  };\n}\n\n#endif\n"
  },
  {
    "path": "cpp-api/include/routing_table.hpp",
    "content": "// Copyright 2015-2017 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#ifndef SCALARIS_ROUTINGTABLE_HPP\n#define SCALARIS_ROUTINGTABLE_HPP\n\n#include <stdexcept>\n\n#include \"connection.hpp\"\n\nnamespace scalaris {\n\n  /// gives access to information about routing tables\n  class RoutingTable {\n    Connection& c;\n  public:\n    /**\n     * Creates a RoutingTable instance\n     * @param _c the connection object used for Scalaris access\n     */\n     RoutingTable(Connection& _c);\n\n    /// retrieves the replication factor of the Scalaris instance\n    int get_replication_factor();\n  };\n}\n\n#endif\n"
  },
  {
    "path": "cpp-api/include/ssl-connection.hpp",
    "content": "// Copyright 2015-2019 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#pragma once\n\n#include <array>\n#include <iostream>\n#include <stdexcept>\n#include <string>\n\n#include \"converter.hpp\"\n#include \"exceptions.hpp\"\n#include \"json/json.h\"\n#include <boost/asio/io_service.hpp>\n#include <boost/asio/ip/tcp.hpp>\n#include <boost/asio/ssl.hpp>\n\n#include \"connection.hpp\"\n\nnamespace scalaris {\n\n  /// represents a SSL connection to Scalaris to execute JSON-RPC requests\n  class SSLConnection final : public Connection {\n    boost::asio::io_service ioservice;\n    boost::asio::ssl::context ctx;\n    boost::asio::ssl::stream<boost::asio::ip::tcp::socket> socket;\n    std::string password = {\"\"};\n\n    bool hasToConnect = true;\n\n  public:\n    /**\n     * creates a connection instance\n     * @param _hostname the host name of the Scalaris instance\n     * @param _location the path location for JSON-RPC\n     * @param port the TCP port of the Scalaris instance\n     */\n    SSLConnection(std::string _hostname, std::string _link = \"jsonrpc.yaws\");\n\n    ~SSLConnection();\n\n    bool needsConnect() const { return hasToConnect; };\n\n    /// checks whether the TCP connection is alive\n    bool isOpen() const;\n\n    /// closes the TCP connection\n    void close();\n\n    /// returns the server port of the TCP connection\n    unsigned getPort() const;\n\n    /// connects to the specified server\n    /// it can also be used, if the connection failed\n    void connect();\n\n    void set_verify_file(const std::string& file);\n    void set_certificate_file(const std::string& file);\n    void set_private_key(const std::string& file);\n    void set_rsa_private_key(const std::string& file);\n    void set_password(const std::string& file);\n\n    std::string toString() const {\n      std::stringstream s;\n      s << \"https://\" << hostname << \":\" << port << \"/\" << link;\n      return s.str();\n    };\n\n  private:\n    virtual Json::Value exec_call(const std::string& methodname,\n                                  Json::Value params, bool reconnect = true);\n    Json::Value process_result(const Json::Value& value);\n\n    bool verify_callback(bool preverified,\n                         boost::asio::ssl::verify_context& ctx);\n\n    std::string\n    password_callback(std::size_t max_length,\n                      boost::asio::ssl::context::password_purpose purpose);\n  };\n} // namespace scalaris\n"
  },
  {
    "path": "cpp-api/include/tcp-connection.hpp",
    "content": "// Copyright 2017-2019 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#pragma once\n\n#include \"connection.hpp\"\n#include \"converter.hpp\"\n#include \"exceptions.hpp\"\n#include \"json/json.h\"\n\n#include <sys/time.h> // evil hack\n\n#include <boost/asio/io_service.hpp>\n#include <boost/asio/ip/tcp.hpp>\n\n#include <array>\n#include <iostream>\n#include <stdexcept>\n#include <string>\n\nnamespace scalaris {\n\n  /// represents a TCP connection to Scalaris to execute JSON-RPC requests\n  class TCPConnection final : public Connection {\n    boost::asio::io_service ioservice;\n    boost::asio::ip::tcp::socket socket;\n\n    bool hasToConnect = true;\n\n  public:\n\n    /**\n     * creates a connection instance\n     * @param _hostname the host name of the Scalaris instance\n     * @param _link the pathURL for JSON-RPC\n     * @param _port the TCP port of the Scalaris instance\n     */\n    TCPConnection(const std::string& _hostname,\n                  const std::string& _link = \"jsonrpc.yaws\",\n                  unsigned _port = 8000);\n\n    ~TCPConnection();\n\n    bool needsConnect() const { return hasToConnect; };\n\n    /// checks whether the TCP connection is alive\n    bool isOpen() const;\n\n    /// closes the TCP connection\n    void close();\n\n    /// returns the server port of the TCP connection\n    unsigned getPort() const;\n\n    /// connects to the specified server\n    /// it can also be used, if the connection failed\n    void connect();\n\n    std::string toString() const {\n      std::stringstream s;\n      s << \"http://\" << hostname << \":\" << port << \"/\" << link;\n      return s.str();\n    };\n\n  private:\n    virtual Json::Value exec_call(const std::string& methodname,\n                                  Json::Value params,\n                                  bool reconnect = true);\n    Json::Value process_result(const Json::Value& value);\n  };\n} // namespace scalaris\n"
  },
  {
    "path": "cpp-api/include/transaction_single_op.hpp",
    "content": "// Copyright 2015-2017 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#ifndef SCALARIS_TRANSACTIONSINGLEOP_HPP\n#define SCALARIS_TRANSACTIONSINGLEOP_HPP\n\n#include <iostream>\n#include <string>\n#include <stdexcept>\n\n#include <boost/asio.hpp>\n#include \"exceptions.hpp\"\n#include \"json/json.h\"\n#include \"req_list.hpp\"\n\n#include \"connection.hpp\"\n\nnamespace scalaris {\n  /// \\brief Executes a single read or write.\n  ///\n  /// Transactions instead execute a sequence of reads and writes.\n  class TransactionSingleOp {\n    Connection& c;\n  public:\n    /**\n     * Creates a <code>TransactionSingleOp</code> instance\n     * @param _c the Connection object\n     */\n    TransactionSingleOp(Connection& _c) : c(_c) {}\n\n    /**\n     * Reads a key-value pair\n     * @param key the key to lookup in Scalaris\n     */\n    std::string read(const std::string key) {\n      RequestList reqlist;\n      reqlist.add_read(key);\n      reqlist.add_commit();\n\n      Json::Value val = c.rpc(\"req_list\", reqlist);\n\n      if(!val.isObject())\n        throw MalFormedJsonError();\n      Json::Value val_results = val[\"results\"];\n      if(!val_results.isArray())\n        throw MalFormedJsonError();\n      Json::Value val_tlog = val[\"tlog\"];\n      if(!val_tlog.isString())\n        throw MalFormedJsonError();\n\n      // 1. success of read\n      {\n        Json::Value tmp = val_results[0];\n        if(!tmp.isObject())\n          throw MalFormedJsonError();\n        Json::Value status = tmp[\"status\"];\n        if(!status.isString())\n          throw MalFormedJsonError();\n        std::string status_str = status.asString();\n        if(status_str.compare(\"ok\") != 0)\n            throw ReadFailedError(tmp[\"reason\"].asString());\n      }\n\n      // 2. success of commit\n      {\n        Json::Value tmp = val_results[0];\n        if(!tmp.isObject())\n          throw MalFormedJsonError();\n        Json::Value status = tmp[\"status\"];\n        if(!status.isString())\n          throw MalFormedJsonError();\n        std::string status_str = status.asString();\n        if(status_str.compare(\"ok\") != 0)\n            throw CommitFailedError(tmp[\"reason\"].asString());\n      }\n\n        Json::Value value = val_results[0][\"value\"];\n        if(!value.isObject())\n          throw MalFormedJsonError();\n        Json::Value type = value[\"type\"];\n        if(!type.isString())\n          throw MalFormedJsonError();\n        std::string type_str = type.asString();\n        if(type_str.compare(\"as_is\") != 0)\n          throw NotSupportedError();\n        //type is as_is\n        Json::Value the_value = value[\"value\"];\n        if(!the_value.isString())\n          throw MalFormedJsonError();\n        return the_value.asString();\n    }\n\n    /**\n     * Writes a key-value pair\n     * @param key the key to update in Scalaris\n     * @param value the value to store under <code>key</code>\n     */\n    void write(const std::string key, const std::string value) {\n      RequestList reqlist;\n      reqlist.add_write(key, value);\n      reqlist.add_commit();\n      Json::Value val = c.rpc(\"req_list\", reqlist);\n\n      if(!val.isObject())\n        throw MalFormedJsonError();\n      Json::Value val_results = val[\"results\"];\n      if(!val_results.isArray())\n        throw MalFormedJsonError();\n      Json::Value val_tlog = val[\"tlog\"];\n      if(!val_tlog.isString())\n        throw MalFormedJsonError();\n\n      // 1. success of write\n      {\n        Json::Value tmp = val_results[0];\n        if(!tmp.isObject())\n          throw MalFormedJsonError();\n        Json::Value status = tmp[\"status\"];\n        if(!status.isString())\n          throw MalFormedJsonError();\n        std::string status_str = status.asString();\n        if(status_str.compare(\"ok\") != 0)\n            throw WriteFailedError(tmp[\"reason\"].asString());\n      }\n\n      // 2. success of commit\n      {\n        Json::Value tmp = val_results[0];\n        if(!tmp.isObject())\n          throw MalFormedJsonError();\n        Json::Value status = tmp[\"status\"];\n        if(!status.isString())\n          throw MalFormedJsonError();\n        std::string status_str = status.asString();\n        if(status_str.compare(\"ok\") != 0)\n            throw CommitFailedError(tmp[\"reason\"].asString());\n      }\n    }\n  };\n}\n\n#endif\n"
  },
  {
    "path": "cpp-api/src/cli.cpp",
    "content": "#include <chrono>\n#include <iostream>\n\n#include <boost/program_options.hpp>\n\n#include \"rbr.hpp\"\n#include \"tcp-connection.hpp\"\n#include \"transaction_single_op.hpp\"\n\nusing namespace std;\nusing namespace scalaris;\nnamespace po = boost::program_options;\n\nconst int N = 100;\n\nnamespace std {\n  istream& operator>>(istream& in, std::pair<std::string, std::string>& ss) {\n    string s;\n    in >> s;\n    const size_t sep = s.find(',');\n    if (sep == string::npos) {\n      ss.first = s;\n      ss.second = string();\n    } else {\n      ss.first = s.substr(0, sep);\n      ss.second = s.substr(sep + 1);\n    }\n    return in;\n  }\n} // namespace std\n\ntemplate <typename F> void exec_call(F& f) {\n  try {\n    TCPConnection c{\"localhost\"};\n    TransactionSingleOp op = {c};\n    f(op);\n  } catch (std::runtime_error& e) {\n    cout << \"std::runtime_error: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (ConnectionError& e) {\n    cout << \"ConnectionError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (MalFormedJsonError& e) {\n    cout << \"MalFormedJsonError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (ReadFailedError& e) {\n    cout << \"ReadFailedError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (NotFoundError& e) {\n    cout << \"NotFoundError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (WriteFailedError& e) {\n    cout << \"WriteFailedError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (CommitFailedError& e) {\n    cout << \"CommitFailedError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (NotSupportedError& e) {\n    cout << \"NotSupportedError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  }\n}\n\n// TODO merge exec_call copies\n\ntemplate <typename F> void exec_call2(F& f) {\n  try {\n    TCPConnection c{\"localhost\"};\n    Rbr op = {c};\n    f(op);\n  } catch (std::runtime_error& e) {\n    cout << \"std::runtime_error: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (ConnectionError& e) {\n    cout << \"ConnectionError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (MalFormedJsonError& e) {\n    cout << \"MalFormedJsonError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (ReadFailedError& e) {\n    cout << \"ReadFailedError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (NotFoundError& e) {\n    cout << \"NotFoundError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (WriteFailedError& e) {\n    cout << \"WriteFailedError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (CommitFailedError& e) {\n    cout << \"CommitFailedError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  } catch (NotSupportedError& e) {\n    cout << \"NotSupportedError: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  }\n}\n\nint main(int argc, char** argv) {\n  // Declare the supported options.\n  po::options_description desc(\"Allowed options\");\n  desc.add_options()\n      // clang-format off\n    (\"help\", \"produce help message\")\n    (\"read\", po::value<std::string>(), \"read key\")\n    (\"write\", po::value<pair<string, string>>(),\"write key,value\")\n    (\"rbr-read\", po::value<std::string>(), \"rbr-read key\")\n    (\"rbr-write\", po::value<pair<string, string>>(),\"rbr-write key,value\")\n    (\"bench\", \"bench\");\n  // clang-format on\n\n  po::variables_map vm;\n  po::store(po::parse_command_line(argc, argv, desc), vm);\n  po::notify(vm);\n\n  try {\n    if (vm.count(\"help\")) {\n      cout << desc << \"\\n\";\n      return 0;\n    } else if (vm.count(\"read\")) {\n      try {\n        string key = vm[\"read\"].as<string>();\n        auto p = [key](TransactionSingleOp& op) {\n          std::string value = op.read(key);\n          cout << value << endl;\n        };\n        exec_call(p);\n      } catch (const boost::exception_detail::clone_impl<\n               boost::exception_detail::error_info_injector<\n                   boost::bad_any_cast>>& e) {\n        cout << \"could not convert read parameter to a string\" << endl;\n        exit(EXIT_FAILURE);\n      }\n    } else if (vm.count(\"write\")) {\n      try {\n        pair<string, string> kv = vm[\"write\"].as<pair<string, string>>();\n        string key = get<0>(kv);\n        string value = get<1>(kv);\n        auto p = [key, value](TransactionSingleOp& op) {\n          op.write(key, value);\n        };\n        exec_call(p);\n      } catch (const boost::exception_detail::clone_impl<\n               boost::exception_detail::error_info_injector<\n                   boost::bad_any_cast>>& e) {\n        cout << \"could not convert write parameter to a pair of strings\"\n             << endl;\n        exit(EXIT_FAILURE);\n      }\n    } else if (vm.count(\"rbr-read\")) {\n      try {\n        string key = vm[\"rbr-read\"].as<string>();\n        auto p = [key](Rbr& r) {\n          std::string value = r.read(key).toStyledString();\n          cout << value << endl;\n        };\n        exec_call2(p);\n      } catch (const boost::exception_detail::clone_impl<\n               boost::exception_detail::error_info_injector<\n                   boost::bad_any_cast>>& e) {\n        cout << \"could not convert read parameter to a string\" << endl;\n        exit(EXIT_FAILURE);\n      }\n    } else if (vm.count(\"rbr-write\")) {\n      try {\n        pair<string, string> kv = vm[\"rbr-write\"].as<pair<string, string>>();\n        string key = get<0>(kv);\n        string value = get<1>(kv);\n        auto p = [key, value](Rbr& r) { r.write(key, value); };\n        exec_call2(p);\n      } catch (const boost::exception_detail::clone_impl<\n               boost::exception_detail::error_info_injector<\n                   boost::bad_any_cast>>& e) {\n        cout << \"could not convert write parameter to a pair of strings\"\n             << endl;\n        exit(EXIT_FAILURE);\n      }\n    } else if (vm.count(\"bench\")) {\n      try {\n        string key = \"bench-key\";\n        string value = \"bench-value\";\n        vector<double> latencies;\n        latencies.resize(N);\n        for (int i = 0; i < N; i++) {\n          auto start = std::chrono::high_resolution_clock::now();\n          {\n            auto p = [key, value](TransactionSingleOp& op) {\n              op.write(key, value);\n            };\n            exec_call(p);\n          }\n          auto stop = std::chrono::high_resolution_clock::now();\n          const std::chrono::duration<double> diff = stop - start;\n          const double duration = diff.count();\n          latencies[i] = duration;\n        }\n        sort(latencies.begin(), latencies.end());\n        const double latency = latencies[N / 2] * 1000;\n        printf(\"executed %d key-value pairs updates using the transaction \"\n               \"mechanism\\n\",\n               N);\n        printf(\"median latency=%fms\\n\", latency);\n      } catch (const boost::exception_detail::clone_impl<\n               boost::exception_detail::error_info_injector<\n                   boost::bad_any_cast>>& e) {\n        cout << \"could not convert write parameter to a pair of strings\"\n             << endl;\n        exit(EXIT_FAILURE);\n      }\n    } else {\n      cout << desc << \"\\n\";\n      return 0;\n    }\n  } catch (const Json::LogicError& e) {\n    cout << \"Json error: \" << e.what() << endl;\n    exit(EXIT_FAILURE);\n  }\n\n  return 0;\n}\n"
  },
  {
    "path": "cpp-api/src/connection.cpp",
    "content": "// Copyright 2015-2018 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#include \"connection.hpp\"\n\nnamespace scalaris {\n\n  Connection::Connection(const std::string& _hostname, const std::string& _link,\n                         unsigned _port)\n      : hostname(_hostname), link(_link), port(_port) {}\n\n} // namespace scalaris\n"
  },
  {
    "path": "cpp-api/src/jsoncpp.cpp",
    "content": "/// Json-cpp amalgated source (http://jsoncpp.sourceforge.net/).\n/// It is intended to be used with #include \"json/json.h\"\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: LICENSE\n// //////////////////////////////////////////////////////////////////////\n\n/*\nThe JsonCpp library's source code, including accompanying documentation,\ntests and demonstration applications, are licensed under the following\nconditions...\n\nThe author (Baptiste Lepilleur) explicitly disclaims copyright in all\njurisdictions which recognize such a disclaimer. In such jurisdictions,\nthis software is released into the Public Domain.\n\nIn jurisdictions which do not recognize Public Domain property (e.g. Germany as\nof 2010), this software is Copyright (c) 2007-2010 by Baptiste Lepilleur, and is\nreleased under the terms of the MIT License (see below).\n\nIn jurisdictions which recognize Public Domain property, the user of this\nsoftware may choose to accept it either as 1) Public Domain, 2) under the\nconditions of the MIT License (see below), or 3) under the terms of dual\nPublic Domain/MIT License conditions described here, as they choose.\n\nThe MIT License is about as close to Public Domain as a license can get, and is\ndescribed in clear, concise terms at:\n\n   http://en.wikipedia.org/wiki/MIT_License\n\nThe full text of the MIT License follows:\n\n========================================================================\nCopyright (c) 2007-2010 Baptiste Lepilleur\n\nPermission is hereby granted, free of charge, to any person\nobtaining a copy of this software and associated documentation\nfiles (the \"Software\"), to deal in the Software without\nrestriction, including without limitation the rights to use, copy,\nmodify, merge, publish, distribute, sublicense, and/or sell copies\nof the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be\nincluded in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\nNONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS\nBE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN\nACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\nCONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n========================================================================\n(END LICENSE TEXT)\n\nThe MIT license is compatible with both the GPL and commercial\nsoftware, affording one all of the rights of Public Domain with the\nminor nuisance of being required to keep the above copyright notice\nand license text in the source code. Note also that by accepting the\nPublic Domain \"license\" you can re-license your copy using whatever\nlicense you like.\n\n*/\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: LICENSE\n// //////////////////////////////////////////////////////////////////////\n\n#include \"json/json.h\"\n\n#ifndef JSON_IS_AMALGAMATION\n#error \"Compile with -I PATH_TO_JSON_DIRECTORY\"\n#endif\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: src/lib_json/json_tool.h\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2007-2010 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n#ifndef LIB_JSONCPP_JSON_TOOL_H_INCLUDED\n#define LIB_JSONCPP_JSON_TOOL_H_INCLUDED\n\n/* This header provides common string manipulation support, such as UTF-8,\n * portable conversion from/to string...\n *\n * It is an internal header that must not be exposed.\n */\n\nnamespace Json {\n\n  /// Converts a unicode code-point to UTF-8.\n  static inline std::string codePointToUTF8(unsigned int cp) {\n    std::string result;\n\n    // based on description from http://en.wikipedia.org/wiki/UTF-8\n\n    if (cp <= 0x7f) {\n      result.resize(1);\n      result[0] = static_cast<char>(cp);\n    } else if (cp <= 0x7FF) {\n      result.resize(2);\n      result[1] = static_cast<char>(0x80 | (0x3f & cp));\n      result[0] = static_cast<char>(0xC0 | (0x1f & (cp >> 6)));\n    } else if (cp <= 0xFFFF) {\n      result.resize(3);\n      result[2] = static_cast<char>(0x80 | (0x3f & cp));\n      result[1] = static_cast<char>(0x80 | (0x3f & (cp >> 6)));\n      result[0] = static_cast<char>(0xE0 | (0xf & (cp >> 12)));\n    } else if (cp <= 0x10FFFF) {\n      result.resize(4);\n      result[3] = static_cast<char>(0x80 | (0x3f & cp));\n      result[2] = static_cast<char>(0x80 | (0x3f & (cp >> 6)));\n      result[1] = static_cast<char>(0x80 | (0x3f & (cp >> 12)));\n      result[0] = static_cast<char>(0xF0 | (0x7 & (cp >> 18)));\n    }\n\n    return result;\n  }\n\n  /// Returns true if ch is a control character (in range [1,31]).\n  static inline bool isControlCharacter(char ch) {\n    return ch > 0 && ch <= 0x1F;\n  }\n\n  enum {\n    /// Constant that specify the size of the buffer that must be passed to\n    /// uintToString.\n    uintToStringBufferSize = 3 * sizeof(LargestUInt) + 1\n  };\n\n  // Defines a char buffer for use with uintToString().\n  typedef char UIntToStringBuffer[uintToStringBufferSize];\n\n  /** Converts an unsigned integer to string.\n   * @param value Unsigned interger to convert to string\n   * @param current Input/Output string buffer.\n   *        Must have at least uintToStringBufferSize chars free.\n   */\n  static inline void uintToString(LargestUInt value, char*& current) {\n    *--current = 0;\n    do {\n      *--current =\n          static_cast<signed char>(value % 10U + static_cast<unsigned>('0'));\n      value /= 10;\n    } while (value != 0);\n  }\n\n  /** Change ',' to '.' everywhere in buffer.\n   *\n   * We had a sophisticated way, but it did not work in WinCE.\n   * @see https://github.com/open-source-parsers/jsoncpp/pull/9\n   */\n  static inline void fixNumericLocale(char* begin, char* end) {\n    while (begin < end) {\n      if (*begin == ',') {\n        *begin = '.';\n      }\n      ++begin;\n    }\n  }\n\n} // namespace Json\n\n#endif // LIB_JSONCPP_JSON_TOOL_H_INCLUDED\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: src/lib_json/json_tool.h\n// //////////////////////////////////////////////////////////////////////\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: src/lib_json/json_reader.cpp\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2007-2011 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n#if !defined(JSON_IS_AMALGAMATION)\n#include \"json_tool.h\"\n#include <json/assertions.h>\n#include <json/reader.h>\n#include <json/value.h>\n#endif // if !defined(JSON_IS_AMALGAMATION)\n#include <cassert>\n#include <cstdio>\n#include <cstring>\n#include <istream>\n#include <limits>\n#include <memory>\n#include <set>\n#include <sstream>\n#include <utility>\n\n#if defined(_MSC_VER)\n#if !defined(WINCE) && defined(__STDC_SECURE_LIB__) &&                         \\\n    _MSC_VER >= 1500 // VC++ 9.0 and above\n#define snprintf sprintf_s\n#elif _MSC_VER >= 1900 // VC++ 14.0 and above\n#define snprintf std::snprintf\n#else\n#define snprintf _snprintf\n#endif\n#elif defined(__ANDROID__) || defined(__QNXNTO__)\n#define snprintf snprintf\n#elif __cplusplus >= 201103L\n#define snprintf std::snprintf\n#endif\n\n#if defined(__QNXNTO__)\n#define sscanf std::sscanf\n#endif\n\n#if defined(_MSC_VER) && _MSC_VER >= 1400 // VC++ 8.0\n// Disable warning about strdup being deprecated.\n#pragma warning(disable : 4996)\n#endif\n\nstatic int const stackLimit_g = 1000;\nstatic int stackDepth_g = 0; // see readValue()\n\nnamespace Json {\n\n#if __cplusplus >= 201103L || (defined(_CPPLIB_VER) && _CPPLIB_VER >= 520)\n  typedef std::unique_ptr<CharReader> CharReaderPtr;\n#else\n  typedef std::auto_ptr<CharReader> CharReaderPtr;\n#endif\n\n  // Implementation of class Features\n  // ////////////////////////////////\n\n  Features::Features()\n      : allowComments_(true), strictRoot_(false),\n        allowDroppedNullPlaceholders_(false), allowNumericKeys_(false) {}\n\n  Features Features::all() { return Features(); }\n\n  Features Features::strictMode() {\n    Features features;\n    features.allowComments_ = false;\n    features.strictRoot_ = true;\n    features.allowDroppedNullPlaceholders_ = false;\n    features.allowNumericKeys_ = false;\n    return features;\n  }\n\n  // Implementation of class Reader\n  // ////////////////////////////////\n\n  static bool containsNewLine(Reader::Location begin, Reader::Location end) {\n    for (; begin < end; ++begin)\n      if (*begin == '\\n' || *begin == '\\r')\n        return true;\n    return false;\n  }\n\n  // Class Reader\n  // //////////////////////////////////////////////////////////////////\n\n  Reader::Reader()\n      : errors_(), document_(), begin_(), end_(), current_(), lastValueEnd_(),\n        lastValue_(), commentsBefore_(), features_(Features::all()),\n        collectComments_() {}\n\n  Reader::Reader(const Features& features)\n      : errors_(), document_(), begin_(), end_(), current_(), lastValueEnd_(),\n        lastValue_(), commentsBefore_(), features_(features),\n        collectComments_() {}\n\n  bool Reader::parse(const std::string& document, Value& root,\n                     bool collectComments) {\n    document_ = document;\n    const char* begin = document_.c_str();\n    const char* end = begin + document_.length();\n    return parse(begin, end, root, collectComments);\n  }\n\n  bool Reader::parse(std::istream& sin, Value& root, bool collectComments) {\n    // std::istream_iterator<char> begin(sin);\n    // std::istream_iterator<char> end;\n    // Those would allow streamed input from a file, if parse() were a\n    // template function.\n\n    // Since std::string is reference-counted, this at least does not\n    // create an extra copy.\n    std::string doc;\n    std::getline(sin, doc, (char)EOF);\n    return parse(doc, root, collectComments);\n  }\n\n  bool Reader::parse(const char* beginDoc, const char* endDoc, Value& root,\n                     bool collectComments) {\n    if (!features_.allowComments_) {\n      collectComments = false;\n    }\n\n    begin_ = beginDoc;\n    end_ = endDoc;\n    collectComments_ = collectComments;\n    current_ = begin_;\n    lastValueEnd_ = 0;\n    lastValue_ = 0;\n    commentsBefore_ = \"\";\n    errors_.clear();\n    while (!nodes_.empty())\n      nodes_.pop();\n    nodes_.push(&root);\n\n    stackDepth_g = 0; // Yes, this is bad coding, but options are limited.\n    bool successful = readValue();\n    Token token;\n    skipCommentTokens(token);\n    if (collectComments_ && !commentsBefore_.empty())\n      root.setComment(commentsBefore_, commentAfter);\n    if (features_.strictRoot_) {\n      if (!root.isArray() && !root.isObject()) {\n        // Set error location to start of doc, ideally should be first token\n        // found in doc\n        token.type_ = tokenError;\n        token.start_ = beginDoc;\n        token.end_ = endDoc;\n        addError(\n            \"A valid JSON document must be either an array or an object value.\",\n            token);\n        return false;\n      }\n    }\n    return successful;\n  }\n\n  bool Reader::readValue() {\n    // This is a non-reentrant way to support a stackLimit. Terrible!\n    // But this deprecated class has a security problem: Bad input can\n    // cause a seg-fault. This seems like a fair, binary-compatible way\n    // to prevent the problem.\n    if (stackDepth_g >= stackLimit_g)\n      throwRuntimeError(\"Exceeded stackLimit in readValue().\");\n    ++stackDepth_g;\n\n    Token token;\n    skipCommentTokens(token);\n    bool successful = true;\n\n    if (collectComments_ && !commentsBefore_.empty()) {\n      currentValue().setComment(commentsBefore_, commentBefore);\n      commentsBefore_ = \"\";\n    }\n\n    switch (token.type_) {\n    case tokenObjectBegin:\n      successful = readObject(token);\n      currentValue().setOffsetLimit(current_ - begin_);\n      break;\n    case tokenArrayBegin:\n      successful = readArray(token);\n      currentValue().setOffsetLimit(current_ - begin_);\n      break;\n    case tokenNumber:\n      successful = decodeNumber(token);\n      break;\n    case tokenString:\n      successful = decodeString(token);\n      break;\n    case tokenTrue: {\n      Value v(true);\n      currentValue().swapPayload(v);\n      currentValue().setOffsetStart(token.start_ - begin_);\n      currentValue().setOffsetLimit(token.end_ - begin_);\n    } break;\n    case tokenFalse: {\n      Value v(false);\n      currentValue().swapPayload(v);\n      currentValue().setOffsetStart(token.start_ - begin_);\n      currentValue().setOffsetLimit(token.end_ - begin_);\n    } break;\n    case tokenNull: {\n      Value v;\n      currentValue().swapPayload(v);\n      currentValue().setOffsetStart(token.start_ - begin_);\n      currentValue().setOffsetLimit(token.end_ - begin_);\n    } break;\n    case tokenArraySeparator:\n    case tokenObjectEnd:\n    case tokenArrayEnd:\n      if (features_.allowDroppedNullPlaceholders_) {\n        // \"Un-read\" the current token and mark the current value as a null\n        // token.\n        current_--;\n        Value v;\n        currentValue().swapPayload(v);\n        currentValue().setOffsetStart(current_ - begin_ - 1);\n        currentValue().setOffsetLimit(current_ - begin_);\n        break;\n      } // Else, fall through...\n    default:\n      currentValue().setOffsetStart(token.start_ - begin_);\n      currentValue().setOffsetLimit(token.end_ - begin_);\n      return addError(\"Syntax error: value, object or array expected.\", token);\n    }\n\n    if (collectComments_) {\n      lastValueEnd_ = current_;\n      lastValue_ = &currentValue();\n    }\n\n    --stackDepth_g;\n    return successful;\n  }\n\n  void Reader::skipCommentTokens(Token& token) {\n    if (features_.allowComments_) {\n      do {\n        readToken(token);\n      } while (token.type_ == tokenComment);\n    } else {\n      readToken(token);\n    }\n  }\n\n  bool Reader::readToken(Token& token) {\n    skipSpaces();\n    token.start_ = current_;\n    Char c = getNextChar();\n    bool ok = true;\n    switch (c) {\n    case '{':\n      token.type_ = tokenObjectBegin;\n      break;\n    case '}':\n      token.type_ = tokenObjectEnd;\n      break;\n    case '[':\n      token.type_ = tokenArrayBegin;\n      break;\n    case ']':\n      token.type_ = tokenArrayEnd;\n      break;\n    case '\"':\n      token.type_ = tokenString;\n      ok = readString();\n      break;\n    case '/':\n      token.type_ = tokenComment;\n      ok = readComment();\n      break;\n    case '0':\n    case '1':\n    case '2':\n    case '3':\n    case '4':\n    case '5':\n    case '6':\n    case '7':\n    case '8':\n    case '9':\n    case '-':\n      token.type_ = tokenNumber;\n      readNumber();\n      break;\n    case 't':\n      token.type_ = tokenTrue;\n      ok = match(\"rue\", 3);\n      break;\n    case 'f':\n      token.type_ = tokenFalse;\n      ok = match(\"alse\", 4);\n      break;\n    case 'n':\n      token.type_ = tokenNull;\n      ok = match(\"ull\", 3);\n      break;\n    case ',':\n      token.type_ = tokenArraySeparator;\n      break;\n    case ':':\n      token.type_ = tokenMemberSeparator;\n      break;\n    case 0:\n      token.type_ = tokenEndOfStream;\n      break;\n    default:\n      ok = false;\n      break;\n    }\n    if (!ok)\n      token.type_ = tokenError;\n    token.end_ = current_;\n    return true;\n  }\n\n  void Reader::skipSpaces() {\n    while (current_ != end_) {\n      Char c = *current_;\n      if (c == ' ' || c == '\\t' || c == '\\r' || c == '\\n')\n        ++current_;\n      else\n        break;\n    }\n  }\n\n  bool Reader::match(Location pattern, int patternLength) {\n    if (end_ - current_ < patternLength)\n      return false;\n    int index = patternLength;\n    while (index--)\n      if (current_[index] != pattern[index])\n        return false;\n    current_ += patternLength;\n    return true;\n  }\n\n  bool Reader::readComment() {\n    Location commentBegin = current_ - 1;\n    Char c = getNextChar();\n    bool successful = false;\n    if (c == '*')\n      successful = readCStyleComment();\n    else if (c == '/')\n      successful = readCppStyleComment();\n    if (!successful)\n      return false;\n\n    if (collectComments_) {\n      CommentPlacement placement = commentBefore;\n      if (lastValueEnd_ && !containsNewLine(lastValueEnd_, commentBegin)) {\n        if (c != '*' || !containsNewLine(commentBegin, current_))\n          placement = commentAfterOnSameLine;\n      }\n\n      addComment(commentBegin, current_, placement);\n    }\n    return true;\n  }\n\n  static std::string normalizeEOL(Reader::Location begin,\n                                  Reader::Location end) {\n    std::string normalized;\n    normalized.reserve(end - begin);\n    Reader::Location current = begin;\n    while (current != end) {\n      char c = *current++;\n      if (c == '\\r') {\n        if (current != end && *current == '\\n')\n          // convert dos EOL\n          ++current;\n        // convert Mac EOL\n        normalized += '\\n';\n      } else {\n        normalized += c;\n      }\n    }\n    return normalized;\n  }\n\n  void Reader::addComment(Location begin, Location end,\n                          CommentPlacement placement) {\n    assert(collectComments_);\n    const std::string& normalized = normalizeEOL(begin, end);\n    if (placement == commentAfterOnSameLine) {\n      assert(lastValue_ != 0);\n      lastValue_->setComment(normalized, placement);\n    } else {\n      commentsBefore_ += normalized;\n    }\n  }\n\n  bool Reader::readCStyleComment() {\n    while (current_ != end_) {\n      Char c = getNextChar();\n      if (c == '*' && *current_ == '/')\n        break;\n    }\n    return getNextChar() == '/';\n  }\n\n  bool Reader::readCppStyleComment() {\n    while (current_ != end_) {\n      Char c = getNextChar();\n      if (c == '\\n')\n        break;\n      if (c == '\\r') {\n        // Consume DOS EOL. It will be normalized in addComment.\n        if (current_ != end_ && *current_ == '\\n')\n          getNextChar();\n        // Break on Moc OS 9 EOL.\n        break;\n      }\n    }\n    return true;\n  }\n\n  void Reader::readNumber() {\n    const char* p = current_;\n    char c = '0'; // stopgap for already consumed character\n    // integral part\n    while (c >= '0' && c <= '9')\n      c = (current_ = p) < end_ ? *p++ : 0;\n    // fractional part\n    if (c == '.') {\n      c = (current_ = p) < end_ ? *p++ : 0;\n      while (c >= '0' && c <= '9')\n        c = (current_ = p) < end_ ? *p++ : 0;\n    }\n    // exponential part\n    if (c == 'e' || c == 'E') {\n      c = (current_ = p) < end_ ? *p++ : 0;\n      if (c == '+' || c == '-')\n        c = (current_ = p) < end_ ? *p++ : 0;\n      while (c >= '0' && c <= '9')\n        c = (current_ = p) < end_ ? *p++ : 0;\n    }\n  }\n\n  bool Reader::readString() {\n    Char c = 0;\n    while (current_ != end_) {\n      c = getNextChar();\n      if (c == '\\\\')\n        getNextChar();\n      else if (c == '\"')\n        break;\n    }\n    return c == '\"';\n  }\n\n  bool Reader::readObject(Token& tokenStart) {\n    Token tokenName;\n    std::string name;\n    Value init(objectValue);\n    currentValue().swapPayload(init);\n    currentValue().setOffsetStart(tokenStart.start_ - begin_);\n    while (readToken(tokenName)) {\n      bool initialTokenOk = true;\n      while (tokenName.type_ == tokenComment && initialTokenOk)\n        initialTokenOk = readToken(tokenName);\n      if (!initialTokenOk)\n        break;\n      if (tokenName.type_ == tokenObjectEnd && name.empty()) // empty object\n        return true;\n      name = \"\";\n      if (tokenName.type_ == tokenString) {\n        if (!decodeString(tokenName, name))\n          return recoverFromError(tokenObjectEnd);\n      } else if (tokenName.type_ == tokenNumber &&\n                 features_.allowNumericKeys_) {\n        Value numberName;\n        if (!decodeNumber(tokenName, numberName))\n          return recoverFromError(tokenObjectEnd);\n        name = numberName.asString();\n      } else {\n        break;\n      }\n\n      Token colon;\n      if (!readToken(colon) || colon.type_ != tokenMemberSeparator) {\n        return addErrorAndRecover(\"Missing ':' after object member name\", colon,\n                                  tokenObjectEnd);\n      }\n      Value& value = currentValue()[name];\n      nodes_.push(&value);\n      bool ok = readValue();\n      nodes_.pop();\n      if (!ok) // error already set\n        return recoverFromError(tokenObjectEnd);\n\n      Token comma;\n      if (!readToken(comma) ||\n          (comma.type_ != tokenObjectEnd &&\n           comma.type_ != tokenArraySeparator && comma.type_ != tokenComment)) {\n        return addErrorAndRecover(\"Missing ',' or '}' in object declaration\",\n                                  comma, tokenObjectEnd);\n      }\n      bool finalizeTokenOk = true;\n      while (comma.type_ == tokenComment && finalizeTokenOk)\n        finalizeTokenOk = readToken(comma);\n      if (comma.type_ == tokenObjectEnd)\n        return true;\n    }\n    return addErrorAndRecover(\"Missing '}' or object member name\", tokenName,\n                              tokenObjectEnd);\n  }\n\n  bool Reader::readArray(Token& tokenStart) {\n    Value init(arrayValue);\n    currentValue().swapPayload(init);\n    currentValue().setOffsetStart(tokenStart.start_ - begin_);\n    skipSpaces();\n    if (*current_ == ']') // empty array\n    {\n      Token endArray;\n      readToken(endArray);\n      return true;\n    }\n    int index = 0;\n    for (;;) {\n      Value& value = currentValue()[index++];\n      nodes_.push(&value);\n      bool ok = readValue();\n      nodes_.pop();\n      if (!ok) // error already set\n        return recoverFromError(tokenArrayEnd);\n\n      Token token;\n      // Accept Comment after last item in the array.\n      ok = readToken(token);\n      while (token.type_ == tokenComment && ok) {\n        ok = readToken(token);\n      }\n      bool badTokenType =\n          (token.type_ != tokenArraySeparator && token.type_ != tokenArrayEnd);\n      if (!ok || badTokenType) {\n        return addErrorAndRecover(\"Missing ',' or ']' in array declaration\",\n                                  token, tokenArrayEnd);\n      }\n      if (token.type_ == tokenArrayEnd)\n        break;\n    }\n    return true;\n  }\n\n  bool Reader::decodeNumber(Token& token) {\n    Value decoded;\n    if (!decodeNumber(token, decoded))\n      return false;\n    currentValue().swapPayload(decoded);\n    currentValue().setOffsetStart(token.start_ - begin_);\n    currentValue().setOffsetLimit(token.end_ - begin_);\n    return true;\n  }\n\n  bool Reader::decodeNumber(Token& token, Value& decoded) {\n    // Attempts to parse the number as an integer. If the number is\n    // larger than the maximum supported value of an integer then\n    // we decode the number as a double.\n    Location current = token.start_;\n    bool isNegative = *current == '-';\n    if (isNegative)\n      ++current;\n    // TODO: Help the compiler do the div and mod at compile time or get rid of\n    // them.\n    Value::LargestUInt maxIntegerValue =\n        isNegative ? Value::LargestUInt(Value::maxLargestInt) + 1\n                   : Value::maxLargestUInt;\n    Value::LargestUInt threshold = maxIntegerValue / 10;\n    Value::LargestUInt value = 0;\n    while (current < token.end_) {\n      Char c = *current++;\n      if (c < '0' || c > '9')\n        return decodeDouble(token, decoded);\n      Value::UInt digit(c - '0');\n      if (value >= threshold) {\n        // We've hit or exceeded the max value divided by 10 (rounded down). If\n        // a) we've only just touched the limit, b) this is the last digit, and\n        // c) it's small enough to fit in that rounding delta, we're okay.\n        // Otherwise treat this number as a double to avoid overflow.\n        if (value > threshold || current != token.end_ ||\n            digit > maxIntegerValue % 10) {\n          return decodeDouble(token, decoded);\n        }\n      }\n      value = value * 10 + digit;\n    }\n    if (isNegative && value == maxIntegerValue)\n      decoded = Value::minLargestInt;\n    else if (isNegative)\n      decoded = -Value::LargestInt(value);\n    else if (value <= Value::LargestUInt(Value::maxInt))\n      decoded = Value::LargestInt(value);\n    else\n      decoded = value;\n    return true;\n  }\n\n  bool Reader::decodeDouble(Token& token) {\n    Value decoded;\n    if (!decodeDouble(token, decoded))\n      return false;\n    currentValue().swapPayload(decoded);\n    currentValue().setOffsetStart(token.start_ - begin_);\n    currentValue().setOffsetLimit(token.end_ - begin_);\n    return true;\n  }\n\n  bool Reader::decodeDouble(Token& token, Value& decoded) {\n    double value = 0;\n    std::string buffer(token.start_, token.end_);\n    std::istringstream is(buffer);\n    if (!(is >> value))\n      return addError(\"'\" + std::string(token.start_, token.end_) +\n                          \"' is not a number.\",\n                      token);\n    decoded = value;\n    return true;\n  }\n\n  bool Reader::decodeString(Token& token) {\n    std::string decoded_string;\n    if (!decodeString(token, decoded_string))\n      return false;\n    Value decoded(decoded_string);\n    currentValue().swapPayload(decoded);\n    currentValue().setOffsetStart(token.start_ - begin_);\n    currentValue().setOffsetLimit(token.end_ - begin_);\n    return true;\n  }\n\n  bool Reader::decodeString(Token& token, std::string& decoded) {\n    decoded.reserve(token.end_ - token.start_ - 2);\n    Location current = token.start_ + 1; // skip '\"'\n    Location end = token.end_ - 1;       // do not include '\"'\n    while (current != end) {\n      Char c = *current++;\n      if (c == '\"')\n        break;\n      else if (c == '\\\\') {\n        if (current == end)\n          return addError(\"Empty escape sequence in string\", token, current);\n        Char escape = *current++;\n        switch (escape) {\n        case '\"':\n          decoded += '\"';\n          break;\n        case '/':\n          decoded += '/';\n          break;\n        case '\\\\':\n          decoded += '\\\\';\n          break;\n        case 'b':\n          decoded += '\\b';\n          break;\n        case 'f':\n          decoded += '\\f';\n          break;\n        case 'n':\n          decoded += '\\n';\n          break;\n        case 'r':\n          decoded += '\\r';\n          break;\n        case 't':\n          decoded += '\\t';\n          break;\n        case 'u': {\n          unsigned int unicode;\n          if (!decodeUnicodeCodePoint(token, current, end, unicode))\n            return false;\n          decoded += codePointToUTF8(unicode);\n        } break;\n        default:\n          return addError(\"Bad escape sequence in string\", token, current);\n        }\n      } else {\n        decoded += c;\n      }\n    }\n    return true;\n  }\n\n  bool Reader::decodeUnicodeCodePoint(Token& token, Location& current,\n                                      Location end, unsigned int& unicode) {\n\n    if (!decodeUnicodeEscapeSequence(token, current, end, unicode))\n      return false;\n    if (unicode >= 0xD800 && unicode <= 0xDBFF) {\n      // surrogate pairs\n      if (end - current < 6)\n        return addError(\"additional six characters expected to parse unicode \"\n                        \"surrogate pair.\",\n                        token, current);\n      unsigned int surrogatePair;\n      if (*(current++) == '\\\\' && *(current++) == 'u') {\n        if (decodeUnicodeEscapeSequence(token, current, end, surrogatePair)) {\n          unicode =\n              0x10000 + ((unicode & 0x3FF) << 10) + (surrogatePair & 0x3FF);\n        } else\n          return false;\n      } else\n        return addError(\n            \"expecting another \\\\u token to begin the second half of \"\n            \"a unicode surrogate pair\",\n            token, current);\n    }\n    return true;\n  }\n\n  bool Reader::decodeUnicodeEscapeSequence(Token& token, Location& current,\n                                           Location end,\n                                           unsigned int& unicode) {\n    if (end - current < 4)\n      return addError(\n          \"Bad unicode escape sequence in string: four digits expected.\", token,\n          current);\n    unicode = 0;\n    for (int index = 0; index < 4; ++index) {\n      Char c = *current++;\n      unicode *= 16;\n      if (c >= '0' && c <= '9')\n        unicode += c - '0';\n      else if (c >= 'a' && c <= 'f')\n        unicode += c - 'a' + 10;\n      else if (c >= 'A' && c <= 'F')\n        unicode += c - 'A' + 10;\n      else\n        return addError(\"Bad unicode escape sequence in string: hexadecimal \"\n                        \"digit expected.\",\n                        token, current);\n    }\n    return true;\n  }\n\n  bool Reader::addError(const std::string& message, Token& token,\n                        Location extra) {\n    ErrorInfo info;\n    info.token_ = token;\n    info.message_ = message;\n    info.extra_ = extra;\n    errors_.push_back(info);\n    return false;\n  }\n\n  bool Reader::recoverFromError(TokenType skipUntilToken) {\n    int errorCount = int(errors_.size());\n    Token skip;\n    for (;;) {\n      if (!readToken(skip))\n        errors_.resize(errorCount); // discard errors caused by recovery\n      if (skip.type_ == skipUntilToken || skip.type_ == tokenEndOfStream)\n        break;\n    }\n    errors_.resize(errorCount);\n    return false;\n  }\n\n  bool Reader::addErrorAndRecover(const std::string& message, Token& token,\n                                  TokenType skipUntilToken) {\n    addError(message, token);\n    return recoverFromError(skipUntilToken);\n  }\n\n  Value& Reader::currentValue() { return *(nodes_.top()); }\n\n  Reader::Char Reader::getNextChar() {\n    if (current_ == end_)\n      return 0;\n    return *current_++;\n  }\n\n  void Reader::getLocationLineAndColumn(Location location, int& line,\n                                        int& column) const {\n    Location current = begin_;\n    Location lastLineStart = current;\n    line = 0;\n    while (current < location && current != end_) {\n      Char c = *current++;\n      if (c == '\\r') {\n        if (*current == '\\n')\n          ++current;\n        lastLineStart = current;\n        ++line;\n      } else if (c == '\\n') {\n        lastLineStart = current;\n        ++line;\n      }\n    }\n    // column & line start at 1\n    column = int(location - lastLineStart) + 1;\n    ++line;\n  }\n\n  std::string Reader::getLocationLineAndColumn(Location location) const {\n    int line, column;\n    getLocationLineAndColumn(location, line, column);\n    char buffer[18 + 16 + 16 + 1];\n    snprintf(buffer, sizeof(buffer), \"Line %d, Column %d\", line, column);\n    return buffer;\n  }\n\n  // Deprecated. Preserved for backward compatibility\n  std::string Reader::getFormatedErrorMessages() const {\n    return getFormattedErrorMessages();\n  }\n\n  std::string Reader::getFormattedErrorMessages() const {\n    std::string formattedMessage;\n    for (Errors::const_iterator itError = errors_.begin();\n         itError != errors_.end(); ++itError) {\n      const ErrorInfo& error = *itError;\n      formattedMessage +=\n          \"* \" + getLocationLineAndColumn(error.token_.start_) + \"\\n\";\n      formattedMessage += \"  \" + error.message_ + \"\\n\";\n      if (error.extra_)\n        formattedMessage +=\n            \"See \" + getLocationLineAndColumn(error.extra_) + \" for detail.\\n\";\n    }\n    return formattedMessage;\n  }\n\n  std::vector<Reader::StructuredError> Reader::getStructuredErrors() const {\n    std::vector<Reader::StructuredError> allErrors;\n    for (Errors::const_iterator itError = errors_.begin();\n         itError != errors_.end(); ++itError) {\n      const ErrorInfo& error = *itError;\n      Reader::StructuredError structured;\n      structured.offset_start = error.token_.start_ - begin_;\n      structured.offset_limit = error.token_.end_ - begin_;\n      structured.message = error.message_;\n      allErrors.push_back(structured);\n    }\n    return allErrors;\n  }\n\n  bool Reader::pushError(const Value& value, const std::string& message) {\n    size_t length = end_ - begin_;\n    if (value.getOffsetStart() > length || value.getOffsetLimit() > length)\n      return false;\n    Token token;\n    token.type_ = tokenError;\n    token.start_ = begin_ + value.getOffsetStart();\n    token.end_ = end_ + value.getOffsetLimit();\n    ErrorInfo info;\n    info.token_ = token;\n    info.message_ = message;\n    info.extra_ = 0;\n    errors_.push_back(info);\n    return true;\n  }\n\n  bool Reader::pushError(const Value& value, const std::string& message,\n                         const Value& extra) {\n    size_t length = end_ - begin_;\n    if (value.getOffsetStart() > length || value.getOffsetLimit() > length ||\n        extra.getOffsetLimit() > length)\n      return false;\n    Token token;\n    token.type_ = tokenError;\n    token.start_ = begin_ + value.getOffsetStart();\n    token.end_ = begin_ + value.getOffsetLimit();\n    ErrorInfo info;\n    info.token_ = token;\n    info.message_ = message;\n    info.extra_ = begin_ + extra.getOffsetStart();\n    errors_.push_back(info);\n    return true;\n  }\n\n  bool Reader::good() const { return !errors_.size(); }\n\n  // exact copy of Features\n  class OurFeatures {\n  public:\n    static OurFeatures all();\n    bool allowComments_;\n    bool strictRoot_;\n    bool allowDroppedNullPlaceholders_;\n    bool allowNumericKeys_;\n    bool allowSingleQuotes_;\n    bool failIfExtra_;\n    bool rejectDupKeys_;\n    bool allowSpecialFloats_;\n    int stackLimit_;\n  }; // OurFeatures\n\n  // exact copy of Implementation of class Features\n  // ////////////////////////////////\n\n  OurFeatures OurFeatures::all() { return OurFeatures(); }\n\n  // Implementation of class Reader\n  // ////////////////////////////////\n\n  // exact copy of Reader, renamed to OurReader\n  class OurReader {\n  public:\n    typedef char Char;\n    typedef const Char* Location;\n    struct StructuredError {\n      size_t offset_start;\n      size_t offset_limit;\n      std::string message;\n    };\n\n    OurReader(OurFeatures const& features);\n    bool parse(const char* beginDoc, const char* endDoc, Value& root,\n               bool collectComments = true);\n    std::string getFormattedErrorMessages() const;\n    std::vector<StructuredError> getStructuredErrors() const;\n    bool pushError(const Value& value, const std::string& message);\n    bool pushError(const Value& value, const std::string& message,\n                   const Value& extra);\n    bool good() const;\n\n  private:\n    OurReader(OurReader const&);      // no impl\n    void operator=(OurReader const&); // no impl\n\n    enum TokenType {\n      tokenEndOfStream = 0,\n      tokenObjectBegin,\n      tokenObjectEnd,\n      tokenArrayBegin,\n      tokenArrayEnd,\n      tokenString,\n      tokenNumber,\n      tokenTrue,\n      tokenFalse,\n      tokenNull,\n      tokenNaN,\n      tokenPosInf,\n      tokenNegInf,\n      tokenArraySeparator,\n      tokenMemberSeparator,\n      tokenComment,\n      tokenError\n    };\n\n    class Token {\n    public:\n      TokenType type_;\n      Location start_;\n      Location end_;\n    };\n\n    class ErrorInfo {\n    public:\n      Token token_;\n      std::string message_;\n      Location extra_;\n    };\n\n    typedef std::deque<ErrorInfo> Errors;\n\n    bool readToken(Token& token);\n    void skipSpaces();\n    bool match(Location pattern, int patternLength);\n    bool readComment();\n    bool readCStyleComment();\n    bool readCppStyleComment();\n    bool readString();\n    bool readStringSingleQuote();\n    bool readNumber(bool checkInf);\n    bool readValue();\n    bool readObject(Token& token);\n    bool readArray(Token& token);\n    bool decodeNumber(Token& token);\n    bool decodeNumber(Token& token, Value& decoded);\n    bool decodeString(Token& token);\n    bool decodeString(Token& token, std::string& decoded);\n    bool decodeDouble(Token& token);\n    bool decodeDouble(Token& token, Value& decoded);\n    bool decodeUnicodeCodePoint(Token& token, Location& current, Location end,\n                                unsigned int& unicode);\n    bool decodeUnicodeEscapeSequence(Token& token, Location& current,\n                                     Location end, unsigned int& unicode);\n    bool addError(const std::string& message, Token& token, Location extra = 0);\n    bool recoverFromError(TokenType skipUntilToken);\n    bool addErrorAndRecover(const std::string& message, Token& token,\n                            TokenType skipUntilToken);\n    void skipUntilSpace();\n    Value& currentValue();\n    Char getNextChar();\n    void getLocationLineAndColumn(Location location, int& line,\n                                  int& column) const;\n    std::string getLocationLineAndColumn(Location location) const;\n    void addComment(Location begin, Location end, CommentPlacement placement);\n    void skipCommentTokens(Token& token);\n\n    typedef std::stack<Value*> Nodes;\n    Nodes nodes_;\n    Errors errors_;\n    std::string document_;\n    Location begin_;\n    Location end_;\n    Location current_;\n    Location lastValueEnd_;\n    Value* lastValue_;\n    std::string commentsBefore_;\n    int stackDepth_;\n\n    OurFeatures const features_;\n    bool collectComments_;\n  }; // OurReader\n\n  // complete copy of Read impl, for OurReader\n\n  OurReader::OurReader(OurFeatures const& features)\n      : errors_(), document_(), begin_(), end_(), current_(), lastValueEnd_(),\n        lastValue_(), commentsBefore_(), stackDepth_(0), features_(features),\n        collectComments_() {}\n\n  bool OurReader::parse(const char* beginDoc, const char* endDoc, Value& root,\n                        bool collectComments) {\n    if (!features_.allowComments_) {\n      collectComments = false;\n    }\n\n    begin_ = beginDoc;\n    end_ = endDoc;\n    collectComments_ = collectComments;\n    current_ = begin_;\n    lastValueEnd_ = 0;\n    lastValue_ = 0;\n    commentsBefore_ = \"\";\n    errors_.clear();\n    while (!nodes_.empty())\n      nodes_.pop();\n    nodes_.push(&root);\n\n    stackDepth_ = 0;\n    bool successful = readValue();\n    Token token;\n    skipCommentTokens(token);\n    if (features_.failIfExtra_) {\n      if (token.type_ != tokenError && token.type_ != tokenEndOfStream) {\n        addError(\"Extra non-whitespace after JSON value.\", token);\n        return false;\n      }\n    }\n    if (collectComments_ && !commentsBefore_.empty())\n      root.setComment(commentsBefore_, commentAfter);\n    if (features_.strictRoot_) {\n      if (!root.isArray() && !root.isObject()) {\n        // Set error location to start of doc, ideally should be first token\n        // found in doc\n        token.type_ = tokenError;\n        token.start_ = beginDoc;\n        token.end_ = endDoc;\n        addError(\n            \"A valid JSON document must be either an array or an object value.\",\n            token);\n        return false;\n      }\n    }\n    return successful;\n  }\n\n  bool OurReader::readValue() {\n    if (stackDepth_ >= features_.stackLimit_)\n      throwRuntimeError(\"Exceeded stackLimit in readValue().\");\n    ++stackDepth_;\n    Token token;\n    skipCommentTokens(token);\n    bool successful = true;\n\n    if (collectComments_ && !commentsBefore_.empty()) {\n      currentValue().setComment(commentsBefore_, commentBefore);\n      commentsBefore_ = \"\";\n    }\n\n    switch (token.type_) {\n    case tokenObjectBegin:\n      successful = readObject(token);\n      currentValue().setOffsetLimit(current_ - begin_);\n      break;\n    case tokenArrayBegin:\n      successful = readArray(token);\n      currentValue().setOffsetLimit(current_ - begin_);\n      break;\n    case tokenNumber:\n      successful = decodeNumber(token);\n      break;\n    case tokenString:\n      successful = decodeString(token);\n      break;\n    case tokenTrue: {\n      Value v(true);\n      currentValue().swapPayload(v);\n      currentValue().setOffsetStart(token.start_ - begin_);\n      currentValue().setOffsetLimit(token.end_ - begin_);\n    } break;\n    case tokenFalse: {\n      Value v(false);\n      currentValue().swapPayload(v);\n      currentValue().setOffsetStart(token.start_ - begin_);\n      currentValue().setOffsetLimit(token.end_ - begin_);\n    } break;\n    case tokenNull: {\n      Value v;\n      currentValue().swapPayload(v);\n      currentValue().setOffsetStart(token.start_ - begin_);\n      currentValue().setOffsetLimit(token.end_ - begin_);\n    } break;\n    case tokenNaN: {\n      Value v(std::numeric_limits<double>::quiet_NaN());\n      currentValue().swapPayload(v);\n      currentValue().setOffsetStart(token.start_ - begin_);\n      currentValue().setOffsetLimit(token.end_ - begin_);\n    } break;\n    case tokenPosInf: {\n      Value v(std::numeric_limits<double>::infinity());\n      currentValue().swapPayload(v);\n      currentValue().setOffsetStart(token.start_ - begin_);\n      currentValue().setOffsetLimit(token.end_ - begin_);\n    } break;\n    case tokenNegInf: {\n      Value v(-std::numeric_limits<double>::infinity());\n      currentValue().swapPayload(v);\n      currentValue().setOffsetStart(token.start_ - begin_);\n      currentValue().setOffsetLimit(token.end_ - begin_);\n    } break;\n    case tokenArraySeparator:\n    case tokenObjectEnd:\n    case tokenArrayEnd:\n      if (features_.allowDroppedNullPlaceholders_) {\n        // \"Un-read\" the current token and mark the current value as a null\n        // token.\n        current_--;\n        Value v;\n        currentValue().swapPayload(v);\n        currentValue().setOffsetStart(current_ - begin_ - 1);\n        currentValue().setOffsetLimit(current_ - begin_);\n        break;\n      } // else, fall through ...\n    default:\n      currentValue().setOffsetStart(token.start_ - begin_);\n      currentValue().setOffsetLimit(token.end_ - begin_);\n      return addError(\"Syntax error: value, object or array expected.\", token);\n    }\n\n    if (collectComments_) {\n      lastValueEnd_ = current_;\n      lastValue_ = &currentValue();\n    }\n\n    --stackDepth_;\n    return successful;\n  }\n\n  void OurReader::skipCommentTokens(Token& token) {\n    if (features_.allowComments_) {\n      do {\n        readToken(token);\n      } while (token.type_ == tokenComment);\n    } else {\n      readToken(token);\n    }\n  }\n\n  bool OurReader::readToken(Token& token) {\n    skipSpaces();\n    token.start_ = current_;\n    Char c = getNextChar();\n    bool ok = true;\n    switch (c) {\n    case '{':\n      token.type_ = tokenObjectBegin;\n      break;\n    case '}':\n      token.type_ = tokenObjectEnd;\n      break;\n    case '[':\n      token.type_ = tokenArrayBegin;\n      break;\n    case ']':\n      token.type_ = tokenArrayEnd;\n      break;\n    case '\"':\n      token.type_ = tokenString;\n      ok = readString();\n      break;\n    case '\\'':\n      if (features_.allowSingleQuotes_) {\n        token.type_ = tokenString;\n        ok = readStringSingleQuote();\n        break;\n      } // else continue\n    case '/':\n      token.type_ = tokenComment;\n      ok = readComment();\n      break;\n    case '0':\n    case '1':\n    case '2':\n    case '3':\n    case '4':\n    case '5':\n    case '6':\n    case '7':\n    case '8':\n    case '9':\n      token.type_ = tokenNumber;\n      readNumber(false);\n      break;\n    case '-':\n      if (readNumber(true)) {\n        token.type_ = tokenNumber;\n      } else {\n        token.type_ = tokenNegInf;\n        ok = features_.allowSpecialFloats_ && match(\"nfinity\", 7);\n      }\n      break;\n    case 't':\n      token.type_ = tokenTrue;\n      ok = match(\"rue\", 3);\n      break;\n    case 'f':\n      token.type_ = tokenFalse;\n      ok = match(\"alse\", 4);\n      break;\n    case 'n':\n      token.type_ = tokenNull;\n      ok = match(\"ull\", 3);\n      break;\n    case 'N':\n      if (features_.allowSpecialFloats_) {\n        token.type_ = tokenNaN;\n        ok = match(\"aN\", 2);\n      } else {\n        ok = false;\n      }\n      break;\n    case 'I':\n      if (features_.allowSpecialFloats_) {\n        token.type_ = tokenPosInf;\n        ok = match(\"nfinity\", 7);\n      } else {\n        ok = false;\n      }\n      break;\n    case ',':\n      token.type_ = tokenArraySeparator;\n      break;\n    case ':':\n      token.type_ = tokenMemberSeparator;\n      break;\n    case 0:\n      token.type_ = tokenEndOfStream;\n      break;\n    default:\n      ok = false;\n      break;\n    }\n    if (!ok)\n      token.type_ = tokenError;\n    token.end_ = current_;\n    return true;\n  }\n\n  void OurReader::skipSpaces() {\n    while (current_ != end_) {\n      Char c = *current_;\n      if (c == ' ' || c == '\\t' || c == '\\r' || c == '\\n')\n        ++current_;\n      else\n        break;\n    }\n  }\n\n  bool OurReader::match(Location pattern, int patternLength) {\n    if (end_ - current_ < patternLength)\n      return false;\n    int index = patternLength;\n    while (index--)\n      if (current_[index] != pattern[index])\n        return false;\n    current_ += patternLength;\n    return true;\n  }\n\n  bool OurReader::readComment() {\n    Location commentBegin = current_ - 1;\n    Char c = getNextChar();\n    bool successful = false;\n    if (c == '*')\n      successful = readCStyleComment();\n    else if (c == '/')\n      successful = readCppStyleComment();\n    if (!successful)\n      return false;\n\n    if (collectComments_) {\n      CommentPlacement placement = commentBefore;\n      if (lastValueEnd_ && !containsNewLine(lastValueEnd_, commentBegin)) {\n        if (c != '*' || !containsNewLine(commentBegin, current_))\n          placement = commentAfterOnSameLine;\n      }\n\n      addComment(commentBegin, current_, placement);\n    }\n    return true;\n  }\n\n  void OurReader::addComment(Location begin, Location end,\n                             CommentPlacement placement) {\n    assert(collectComments_);\n    const std::string& normalized = normalizeEOL(begin, end);\n    if (placement == commentAfterOnSameLine) {\n      assert(lastValue_ != 0);\n      lastValue_->setComment(normalized, placement);\n    } else {\n      commentsBefore_ += normalized;\n    }\n  }\n\n  bool OurReader::readCStyleComment() {\n    while (current_ != end_) {\n      Char c = getNextChar();\n      if (c == '*' && *current_ == '/')\n        break;\n    }\n    return getNextChar() == '/';\n  }\n\n  bool OurReader::readCppStyleComment() {\n    while (current_ != end_) {\n      Char c = getNextChar();\n      if (c == '\\n')\n        break;\n      if (c == '\\r') {\n        // Consume DOS EOL. It will be normalized in addComment.\n        if (current_ != end_ && *current_ == '\\n')\n          getNextChar();\n        // Break on Moc OS 9 EOL.\n        break;\n      }\n    }\n    return true;\n  }\n\n  bool OurReader::readNumber(bool checkInf) {\n    const char* p = current_;\n    if (checkInf && p != end_ && *p == 'I') {\n      current_ = ++p;\n      return false;\n    }\n    char c = '0'; // stopgap for already consumed character\n    // integral part\n    while (c >= '0' && c <= '9')\n      c = (current_ = p) < end_ ? *p++ : 0;\n    // fractional part\n    if (c == '.') {\n      c = (current_ = p) < end_ ? *p++ : 0;\n      while (c >= '0' && c <= '9')\n        c = (current_ = p) < end_ ? *p++ : 0;\n    }\n    // exponential part\n    if (c == 'e' || c == 'E') {\n      c = (current_ = p) < end_ ? *p++ : 0;\n      if (c == '+' || c == '-')\n        c = (current_ = p) < end_ ? *p++ : 0;\n      while (c >= '0' && c <= '9')\n        c = (current_ = p) < end_ ? *p++ : 0;\n    }\n    return true;\n  }\n  bool OurReader::readString() {\n    Char c = 0;\n    while (current_ != end_) {\n      c = getNextChar();\n      if (c == '\\\\')\n        getNextChar();\n      else if (c == '\"')\n        break;\n    }\n    return c == '\"';\n  }\n\n  bool OurReader::readStringSingleQuote() {\n    Char c = 0;\n    while (current_ != end_) {\n      c = getNextChar();\n      if (c == '\\\\')\n        getNextChar();\n      else if (c == '\\'')\n        break;\n    }\n    return c == '\\'';\n  }\n\n  bool OurReader::readObject(Token& tokenStart) {\n    Token tokenName;\n    std::string name;\n    Value init(objectValue);\n    currentValue().swapPayload(init);\n    currentValue().setOffsetStart(tokenStart.start_ - begin_);\n    while (readToken(tokenName)) {\n      bool initialTokenOk = true;\n      while (tokenName.type_ == tokenComment && initialTokenOk)\n        initialTokenOk = readToken(tokenName);\n      if (!initialTokenOk)\n        break;\n      if (tokenName.type_ == tokenObjectEnd && name.empty()) // empty object\n        return true;\n      name = \"\";\n      if (tokenName.type_ == tokenString) {\n        if (!decodeString(tokenName, name))\n          return recoverFromError(tokenObjectEnd);\n      } else if (tokenName.type_ == tokenNumber &&\n                 features_.allowNumericKeys_) {\n        Value numberName;\n        if (!decodeNumber(tokenName, numberName))\n          return recoverFromError(tokenObjectEnd);\n        name = numberName.asString();\n      } else {\n        break;\n      }\n\n      Token colon;\n      if (!readToken(colon) || colon.type_ != tokenMemberSeparator) {\n        return addErrorAndRecover(\"Missing ':' after object member name\", colon,\n                                  tokenObjectEnd);\n      }\n      if (name.length() >= (1U << 30))\n        throwRuntimeError(\"keylength >= 2^30\");\n      if (features_.rejectDupKeys_ && currentValue().isMember(name)) {\n        std::string msg = \"Duplicate key: '\" + name + \"'\";\n        return addErrorAndRecover(msg, tokenName, tokenObjectEnd);\n      }\n      Value& value = currentValue()[name];\n      nodes_.push(&value);\n      bool ok = readValue();\n      nodes_.pop();\n      if (!ok) // error already set\n        return recoverFromError(tokenObjectEnd);\n\n      Token comma;\n      if (!readToken(comma) ||\n          (comma.type_ != tokenObjectEnd &&\n           comma.type_ != tokenArraySeparator && comma.type_ != tokenComment)) {\n        return addErrorAndRecover(\"Missing ',' or '}' in object declaration\",\n                                  comma, tokenObjectEnd);\n      }\n      bool finalizeTokenOk = true;\n      while (comma.type_ == tokenComment && finalizeTokenOk)\n        finalizeTokenOk = readToken(comma);\n      if (comma.type_ == tokenObjectEnd)\n        return true;\n    }\n    return addErrorAndRecover(\"Missing '}' or object member name\", tokenName,\n                              tokenObjectEnd);\n  }\n\n  bool OurReader::readArray(Token& tokenStart) {\n    Value init(arrayValue);\n    currentValue().swapPayload(init);\n    currentValue().setOffsetStart(tokenStart.start_ - begin_);\n    skipSpaces();\n    if (*current_ == ']') // empty array\n    {\n      Token endArray;\n      readToken(endArray);\n      return true;\n    }\n    int index = 0;\n    for (;;) {\n      Value& value = currentValue()[index++];\n      nodes_.push(&value);\n      bool ok = readValue();\n      nodes_.pop();\n      if (!ok) // error already set\n        return recoverFromError(tokenArrayEnd);\n\n      Token token;\n      // Accept Comment after last item in the array.\n      ok = readToken(token);\n      while (token.type_ == tokenComment && ok) {\n        ok = readToken(token);\n      }\n      bool badTokenType =\n          (token.type_ != tokenArraySeparator && token.type_ != tokenArrayEnd);\n      if (!ok || badTokenType) {\n        return addErrorAndRecover(\"Missing ',' or ']' in array declaration\",\n                                  token, tokenArrayEnd);\n      }\n      if (token.type_ == tokenArrayEnd)\n        break;\n    }\n    return true;\n  }\n\n  bool OurReader::decodeNumber(Token& token) {\n    Value decoded;\n    if (!decodeNumber(token, decoded))\n      return false;\n    currentValue().swapPayload(decoded);\n    currentValue().setOffsetStart(token.start_ - begin_);\n    currentValue().setOffsetLimit(token.end_ - begin_);\n    return true;\n  }\n\n  bool OurReader::decodeNumber(Token& token, Value& decoded) {\n    // Attempts to parse the number as an integer. If the number is\n    // larger than the maximum supported value of an integer then\n    // we decode the number as a double.\n    Location current = token.start_;\n    bool isNegative = *current == '-';\n    if (isNegative)\n      ++current;\n    // TODO: Help the compiler do the div and mod at compile time or get rid of\n    // them.\n    Value::LargestUInt maxIntegerValue =\n        isNegative ? Value::LargestUInt(-Value::minLargestInt)\n                   : Value::maxLargestUInt;\n    Value::LargestUInt threshold = maxIntegerValue / 10;\n    Value::LargestUInt value = 0;\n    while (current < token.end_) {\n      Char c = *current++;\n      if (c < '0' || c > '9')\n        return decodeDouble(token, decoded);\n      Value::UInt digit(c - '0');\n      if (value >= threshold) {\n        // We've hit or exceeded the max value divided by 10 (rounded down). If\n        // a) we've only just touched the limit, b) this is the last digit, and\n        // c) it's small enough to fit in that rounding delta, we're okay.\n        // Otherwise treat this number as a double to avoid overflow.\n        if (value > threshold || current != token.end_ ||\n            digit > maxIntegerValue % 10) {\n          return decodeDouble(token, decoded);\n        }\n      }\n      value = value * 10 + digit;\n    }\n    if (isNegative)\n      decoded = -Value::LargestInt(value);\n    else if (value <= Value::LargestUInt(Value::maxInt))\n      decoded = Value::LargestInt(value);\n    else\n      decoded = value;\n    return true;\n  }\n\n  bool OurReader::decodeDouble(Token& token) {\n    Value decoded;\n    if (!decodeDouble(token, decoded))\n      return false;\n    currentValue().swapPayload(decoded);\n    currentValue().setOffsetStart(token.start_ - begin_);\n    currentValue().setOffsetLimit(token.end_ - begin_);\n    return true;\n  }\n\n  bool OurReader::decodeDouble(Token& token, Value& decoded) {\n    double value = 0;\n    const int bufferSize = 32;\n    int count;\n    int length = int(token.end_ - token.start_);\n\n    // Sanity check to avoid buffer overflow exploits.\n    if (length < 0) {\n      return addError(\"Unable to parse token length\", token);\n    }\n\n    // Avoid using a string constant for the format control string given to\n    // sscanf, as this can cause hard to debug crashes on OS X. See here for\n    // more info:\n    //\n    //     http://developer.apple.com/library/mac/#DOCUMENTATION/DeveloperTools/gcc-4.0.1/gcc/Incompatibilities.html\n    char format[] = \"%lf\";\n\n    if (length <= bufferSize) {\n      Char buffer[bufferSize + 1];\n      memcpy(buffer, token.start_, length);\n      buffer[length] = 0;\n      count = sscanf(buffer, format, &value);\n    } else {\n      std::string buffer(token.start_, token.end_);\n      count = sscanf(buffer.c_str(), format, &value);\n    }\n\n    if (count != 1)\n      return addError(\"'\" + std::string(token.start_, token.end_) +\n                          \"' is not a number.\",\n                      token);\n    decoded = value;\n    return true;\n  }\n\n  bool OurReader::decodeString(Token& token) {\n    std::string decoded_string;\n    if (!decodeString(token, decoded_string))\n      return false;\n    Value decoded(decoded_string);\n    currentValue().swapPayload(decoded);\n    currentValue().setOffsetStart(token.start_ - begin_);\n    currentValue().setOffsetLimit(token.end_ - begin_);\n    return true;\n  }\n\n  bool OurReader::decodeString(Token& token, std::string& decoded) {\n    decoded.reserve(token.end_ - token.start_ - 2);\n    Location current = token.start_ + 1; // skip '\"'\n    Location end = token.end_ - 1;       // do not include '\"'\n    while (current != end) {\n      Char c = *current++;\n      if (c == '\"')\n        break;\n      else if (c == '\\\\') {\n        if (current == end)\n          return addError(\"Empty escape sequence in string\", token, current);\n        Char escape = *current++;\n        switch (escape) {\n        case '\"':\n          decoded += '\"';\n          break;\n        case '/':\n          decoded += '/';\n          break;\n        case '\\\\':\n          decoded += '\\\\';\n          break;\n        case 'b':\n          decoded += '\\b';\n          break;\n        case 'f':\n          decoded += '\\f';\n          break;\n        case 'n':\n          decoded += '\\n';\n          break;\n        case 'r':\n          decoded += '\\r';\n          break;\n        case 't':\n          decoded += '\\t';\n          break;\n        case 'u': {\n          unsigned int unicode;\n          if (!decodeUnicodeCodePoint(token, current, end, unicode))\n            return false;\n          decoded += codePointToUTF8(unicode);\n        } break;\n        default:\n          return addError(\"Bad escape sequence in string\", token, current);\n        }\n      } else {\n        decoded += c;\n      }\n    }\n    return true;\n  }\n\n  bool OurReader::decodeUnicodeCodePoint(Token& token, Location& current,\n                                         Location end, unsigned int& unicode) {\n\n    if (!decodeUnicodeEscapeSequence(token, current, end, unicode))\n      return false;\n    if (unicode >= 0xD800 && unicode <= 0xDBFF) {\n      // surrogate pairs\n      if (end - current < 6)\n        return addError(\"additional six characters expected to parse unicode \"\n                        \"surrogate pair.\",\n                        token, current);\n      unsigned int surrogatePair;\n      if (*(current++) == '\\\\' && *(current++) == 'u') {\n        if (decodeUnicodeEscapeSequence(token, current, end, surrogatePair)) {\n          unicode =\n              0x10000 + ((unicode & 0x3FF) << 10) + (surrogatePair & 0x3FF);\n        } else\n          return false;\n      } else\n        return addError(\n            \"expecting another \\\\u token to begin the second half of \"\n            \"a unicode surrogate pair\",\n            token, current);\n    }\n    return true;\n  }\n\n  bool OurReader::decodeUnicodeEscapeSequence(Token& token, Location& current,\n                                              Location end,\n                                              unsigned int& unicode) {\n    if (end - current < 4)\n      return addError(\n          \"Bad unicode escape sequence in string: four digits expected.\", token,\n          current);\n    unicode = 0;\n    for (int index = 0; index < 4; ++index) {\n      Char c = *current++;\n      unicode *= 16;\n      if (c >= '0' && c <= '9')\n        unicode += c - '0';\n      else if (c >= 'a' && c <= 'f')\n        unicode += c - 'a' + 10;\n      else if (c >= 'A' && c <= 'F')\n        unicode += c - 'A' + 10;\n      else\n        return addError(\"Bad unicode escape sequence in string: hexadecimal \"\n                        \"digit expected.\",\n                        token, current);\n    }\n    return true;\n  }\n\n  bool OurReader::addError(const std::string& message, Token& token,\n                           Location extra) {\n    ErrorInfo info;\n    info.token_ = token;\n    info.message_ = message;\n    info.extra_ = extra;\n    errors_.push_back(info);\n    return false;\n  }\n\n  bool OurReader::recoverFromError(TokenType skipUntilToken) {\n    int errorCount = int(errors_.size());\n    Token skip;\n    for (;;) {\n      if (!readToken(skip))\n        errors_.resize(errorCount); // discard errors caused by recovery\n      if (skip.type_ == skipUntilToken || skip.type_ == tokenEndOfStream)\n        break;\n    }\n    errors_.resize(errorCount);\n    return false;\n  }\n\n  bool OurReader::addErrorAndRecover(const std::string& message, Token& token,\n                                     TokenType skipUntilToken) {\n    addError(message, token);\n    return recoverFromError(skipUntilToken);\n  }\n\n  Value& OurReader::currentValue() { return *(nodes_.top()); }\n\n  OurReader::Char OurReader::getNextChar() {\n    if (current_ == end_)\n      return 0;\n    return *current_++;\n  }\n\n  void OurReader::getLocationLineAndColumn(Location location, int& line,\n                                           int& column) const {\n    Location current = begin_;\n    Location lastLineStart = current;\n    line = 0;\n    while (current < location && current != end_) {\n      Char c = *current++;\n      if (c == '\\r') {\n        if (*current == '\\n')\n          ++current;\n        lastLineStart = current;\n        ++line;\n      } else if (c == '\\n') {\n        lastLineStart = current;\n        ++line;\n      }\n    }\n    // column & line start at 1\n    column = int(location - lastLineStart) + 1;\n    ++line;\n  }\n\n  std::string OurReader::getLocationLineAndColumn(Location location) const {\n    int line, column;\n    getLocationLineAndColumn(location, line, column);\n    char buffer[18 + 16 + 16 + 1];\n    snprintf(buffer, sizeof(buffer), \"Line %d, Column %d\", line, column);\n    return buffer;\n  }\n\n  std::string OurReader::getFormattedErrorMessages() const {\n    std::string formattedMessage;\n    for (Errors::const_iterator itError = errors_.begin();\n         itError != errors_.end(); ++itError) {\n      const ErrorInfo& error = *itError;\n      formattedMessage +=\n          \"* \" + getLocationLineAndColumn(error.token_.start_) + \"\\n\";\n      formattedMessage += \"  \" + error.message_ + \"\\n\";\n      if (error.extra_)\n        formattedMessage +=\n            \"See \" + getLocationLineAndColumn(error.extra_) + \" for detail.\\n\";\n    }\n    return formattedMessage;\n  }\n\n  std::vector<OurReader::StructuredError>\n  OurReader::getStructuredErrors() const {\n    std::vector<OurReader::StructuredError> allErrors;\n    for (Errors::const_iterator itError = errors_.begin();\n         itError != errors_.end(); ++itError) {\n      const ErrorInfo& error = *itError;\n      OurReader::StructuredError structured;\n      structured.offset_start = error.token_.start_ - begin_;\n      structured.offset_limit = error.token_.end_ - begin_;\n      structured.message = error.message_;\n      allErrors.push_back(structured);\n    }\n    return allErrors;\n  }\n\n  bool OurReader::pushError(const Value& value, const std::string& message) {\n    size_t length = end_ - begin_;\n    if (value.getOffsetStart() > length || value.getOffsetLimit() > length)\n      return false;\n    Token token;\n    token.type_ = tokenError;\n    token.start_ = begin_ + value.getOffsetStart();\n    token.end_ = end_ + value.getOffsetLimit();\n    ErrorInfo info;\n    info.token_ = token;\n    info.message_ = message;\n    info.extra_ = 0;\n    errors_.push_back(info);\n    return true;\n  }\n\n  bool OurReader::pushError(const Value& value, const std::string& message,\n                            const Value& extra) {\n    size_t length = end_ - begin_;\n    if (value.getOffsetStart() > length || value.getOffsetLimit() > length ||\n        extra.getOffsetLimit() > length)\n      return false;\n    Token token;\n    token.type_ = tokenError;\n    token.start_ = begin_ + value.getOffsetStart();\n    token.end_ = begin_ + value.getOffsetLimit();\n    ErrorInfo info;\n    info.token_ = token;\n    info.message_ = message;\n    info.extra_ = begin_ + extra.getOffsetStart();\n    errors_.push_back(info);\n    return true;\n  }\n\n  bool OurReader::good() const { return !errors_.size(); }\n\n  class OurCharReader : public CharReader {\n    bool const collectComments_;\n    OurReader reader_;\n\n  public:\n    OurCharReader(bool collectComments, OurFeatures const& features)\n        : collectComments_(collectComments), reader_(features) {}\n    bool parse(char const* beginDoc, char const* endDoc, Value* root,\n               std::string* errs) override {\n      bool ok = reader_.parse(beginDoc, endDoc, *root, collectComments_);\n      if (errs) {\n        *errs = reader_.getFormattedErrorMessages();\n      }\n      return ok;\n    }\n  };\n\n  CharReaderBuilder::CharReaderBuilder() { setDefaults(&settings_); }\n  CharReaderBuilder::~CharReaderBuilder() {}\n  CharReader* CharReaderBuilder::newCharReader() const {\n    bool collectComments = settings_[\"collectComments\"].asBool();\n    OurFeatures features = OurFeatures::all();\n    features.allowComments_ = settings_[\"allowComments\"].asBool();\n    features.strictRoot_ = settings_[\"strictRoot\"].asBool();\n    features.allowDroppedNullPlaceholders_ =\n        settings_[\"allowDroppedNullPlaceholders\"].asBool();\n    features.allowNumericKeys_ = settings_[\"allowNumericKeys\"].asBool();\n    features.allowSingleQuotes_ = settings_[\"allowSingleQuotes\"].asBool();\n    features.stackLimit_ = settings_[\"stackLimit\"].asInt();\n    features.failIfExtra_ = settings_[\"failIfExtra\"].asBool();\n    features.rejectDupKeys_ = settings_[\"rejectDupKeys\"].asBool();\n    features.allowSpecialFloats_ = settings_[\"allowSpecialFloats\"].asBool();\n    return new OurCharReader(collectComments, features);\n  }\n  static void getValidReaderKeys(std::set<std::string>* valid_keys) {\n    valid_keys->clear();\n    valid_keys->insert(\"collectComments\");\n    valid_keys->insert(\"allowComments\");\n    valid_keys->insert(\"strictRoot\");\n    valid_keys->insert(\"allowDroppedNullPlaceholders\");\n    valid_keys->insert(\"allowNumericKeys\");\n    valid_keys->insert(\"allowSingleQuotes\");\n    valid_keys->insert(\"stackLimit\");\n    valid_keys->insert(\"failIfExtra\");\n    valid_keys->insert(\"rejectDupKeys\");\n    valid_keys->insert(\"allowSpecialFloats\");\n  }\n  bool CharReaderBuilder::validate(Json::Value* invalid) const {\n    Json::Value my_invalid;\n    if (!invalid)\n      invalid = &my_invalid; // so we do not need to test for NULL\n    Json::Value& inv = *invalid;\n    std::set<std::string> valid_keys;\n    getValidReaderKeys(&valid_keys);\n    Value::Members keys = settings_.getMemberNames();\n    size_t n = keys.size();\n    for (size_t i = 0; i < n; ++i) {\n      std::string const& key = keys[i];\n      if (valid_keys.find(key) == valid_keys.end()) {\n        inv[key] = settings_[key];\n      }\n    }\n    return 0u == inv.size();\n  }\n  Value& CharReaderBuilder::operator[](std::string key) {\n    return settings_[key];\n  }\n  // static\n  void CharReaderBuilder::strictMode(Json::Value* settings) {\n    //! [CharReaderBuilderStrictMode]\n    (*settings)[\"allowComments\"] = false;\n    (*settings)[\"strictRoot\"] = true;\n    (*settings)[\"allowDroppedNullPlaceholders\"] = false;\n    (*settings)[\"allowNumericKeys\"] = false;\n    (*settings)[\"allowSingleQuotes\"] = false;\n    (*settings)[\"stackLimit\"] = 1000;\n    (*settings)[\"failIfExtra\"] = true;\n    (*settings)[\"rejectDupKeys\"] = true;\n    (*settings)[\"allowSpecialFloats\"] = false;\n    //! [CharReaderBuilderStrictMode]\n  }\n  // static\n  void CharReaderBuilder::setDefaults(Json::Value* settings) {\n    //! [CharReaderBuilderDefaults]\n    (*settings)[\"collectComments\"] = true;\n    (*settings)[\"allowComments\"] = true;\n    (*settings)[\"strictRoot\"] = false;\n    (*settings)[\"allowDroppedNullPlaceholders\"] = false;\n    (*settings)[\"allowNumericKeys\"] = false;\n    (*settings)[\"allowSingleQuotes\"] = false;\n    (*settings)[\"stackLimit\"] = 1000;\n    (*settings)[\"failIfExtra\"] = false;\n    (*settings)[\"rejectDupKeys\"] = false;\n    (*settings)[\"allowSpecialFloats\"] = false;\n    //! [CharReaderBuilderDefaults]\n  }\n\n  //////////////////////////////////\n  // global functions\n\n  bool parseFromStream(CharReader::Factory const& fact, std::istream& sin,\n                       Value* root, std::string* errs) {\n    std::ostringstream ssin;\n    ssin << sin.rdbuf();\n    std::string doc = ssin.str();\n    char const* begin = doc.data();\n    char const* end = begin + doc.size();\n    // Note that we do not actually need a null-terminator.\n    CharReaderPtr const reader(fact.newCharReader());\n    return reader->parse(begin, end, root, errs);\n  }\n\n  std::istream& operator>>(std::istream& sin, Value& root) {\n    CharReaderBuilder b;\n    std::string errs;\n    bool ok = parseFromStream(b, sin, &root, &errs);\n    if (!ok) {\n      fprintf(stderr, \"Error from reader: %s\", errs.c_str());\n\n      throwRuntimeError(\"reader error\");\n    }\n    return sin;\n  }\n\n} // namespace Json\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: src/lib_json/json_reader.cpp\n// //////////////////////////////////////////////////////////////////////\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: src/lib_json/json_valueiterator.inl\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2007-2010 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n// included by json_value.cpp\n\nnamespace Json {\n\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // class ValueIteratorBase\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n\n  ValueIteratorBase::ValueIteratorBase() : current_(), isNull_(true) {}\n\n  ValueIteratorBase::ValueIteratorBase(\n      const Value::ObjectValues::iterator& current)\n      : current_(current), isNull_(false) {}\n\n  Value& ValueIteratorBase::deref() const { return current_->second; }\n\n  void ValueIteratorBase::increment() { ++current_; }\n\n  void ValueIteratorBase::decrement() { --current_; }\n\n  ValueIteratorBase::difference_type\n  ValueIteratorBase::computeDistance(const SelfType& other) const {\n#ifdef JSON_USE_CPPTL_SMALLMAP\n    return other.current_ - current_;\n#else\n    // Iterator for null value are initialized using the default\n    // constructor, which initialize current_ to the default\n    // std::map::iterator. As begin() and end() are two instance\n    // of the default std::map::iterator, they can not be compared.\n    // To allow this, we handle this comparison specifically.\n    if (isNull_ && other.isNull_) {\n      return 0;\n    }\n\n    // Usage of std::distance is not portable (does not compile with Sun Studio\n    // 12 RogueWave STL, which is the one used by default). Using a portable\n    // hand-made version for non random iterator instead:\n    //   return difference_type( std::distance( current_, other.current_ ) );\n    difference_type myDistance = 0;\n    for (Value::ObjectValues::iterator it = current_; it != other.current_;\n         ++it) {\n      ++myDistance;\n    }\n    return myDistance;\n#endif\n  }\n\n  bool ValueIteratorBase::isEqual(const SelfType& other) const {\n    if (isNull_) {\n      return other.isNull_;\n    }\n    return current_ == other.current_;\n  }\n\n  void ValueIteratorBase::copy(const SelfType& other) {\n    current_ = other.current_;\n    isNull_ = other.isNull_;\n  }\n\n  Value ValueIteratorBase::key() const {\n    const Value::CZString czstring = (*current_).first;\n    if (czstring.data()) {\n      if (czstring.isStaticString())\n        return Value(StaticString(czstring.data()));\n      return Value(czstring.data(), czstring.data() + czstring.length());\n    }\n    return Value(czstring.index());\n  }\n\n  UInt ValueIteratorBase::index() const {\n    const Value::CZString czstring = (*current_).first;\n    if (!czstring.data())\n      return czstring.index();\n    return Value::UInt(-1);\n  }\n\n  std::string ValueIteratorBase::name() const {\n    char const* keey;\n    char const* end;\n    keey = memberName(&end);\n    if (!keey)\n      return std::string();\n    return std::string(keey, end);\n  }\n\n  char const* ValueIteratorBase::memberName() const {\n    const char* cname = (*current_).first.data();\n    return cname ? cname : \"\";\n  }\n\n  char const* ValueIteratorBase::memberName(char const** end) const {\n    const char* cname = (*current_).first.data();\n    if (!cname) {\n      *end = NULL;\n      return NULL;\n    }\n    *end = cname + (*current_).first.length();\n    return cname;\n  }\n\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // class ValueConstIterator\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n\n  ValueConstIterator::ValueConstIterator() {}\n\n  ValueConstIterator::ValueConstIterator(\n      const Value::ObjectValues::iterator& current)\n      : ValueIteratorBase(current) {}\n\n  ValueConstIterator::ValueConstIterator(ValueIterator const& other)\n      : ValueIteratorBase(other) {}\n\n  ValueConstIterator& ValueConstIterator::\n  operator=(const ValueIteratorBase& other) {\n    copy(other);\n    return *this;\n  }\n\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // class ValueIterator\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n\n  ValueIterator::ValueIterator() {}\n\n  ValueIterator::ValueIterator(const Value::ObjectValues::iterator& current)\n      : ValueIteratorBase(current) {}\n\n  ValueIterator::ValueIterator(const ValueConstIterator& other)\n      : ValueIteratorBase(other) {\n    throwRuntimeError(\"ConstIterator to Iterator should never be allowed.\");\n  }\n\n  ValueIterator::ValueIterator(const ValueIterator& other)\n      : ValueIteratorBase(other) {}\n\n  ValueIterator& ValueIterator::operator=(const SelfType& other) {\n    copy(other);\n    return *this;\n  }\n\n} // namespace Json\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: src/lib_json/json_valueiterator.inl\n// //////////////////////////////////////////////////////////////////////\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: src/lib_json/json_value.cpp\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2011 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n#if !defined(JSON_IS_AMALGAMATION)\n#include <json/assertions.h>\n#include <json/value.h>\n#include <json/writer.h>\n#endif // if !defined(JSON_IS_AMALGAMATION)\n#include <cassert>\n#include <cstring>\n#include <math.h>\n#include <sstream>\n#include <utility>\n#ifdef JSON_USE_CPPTL\n#include <cpptl/conststring.h>\n#endif\n#include <algorithm> // min()\n#include <cstddef>   // size_t\n\n#define JSON_ASSERT_UNREACHABLE assert(false)\n\nnamespace Json {\n\n// This is a walkaround to avoid the static initialization of Value::null.\n// kNull must be word-aligned to avoid crashing on ARM.  We use an alignment of\n// 8 (instead of 4) as a bit of future-proofing.\n#if defined(__ARMEL__)\n#define ALIGNAS(byte_alignment) __attribute__((aligned(byte_alignment)))\n#else\n#define ALIGNAS(byte_alignment)\n#endif\n  static const unsigned char ALIGNAS(8) kNull[sizeof(Value)] = {0};\n  const unsigned char& kNullRef = kNull[0];\n  const Value& Value::null = reinterpret_cast<const Value&>(kNullRef);\n  const Value& Value::nullRef = null;\n\n  const Int Value::minInt = Int(~(UInt(-1) / 2));\n  const Int Value::maxInt = Int(UInt(-1) / 2);\n  const UInt Value::maxUInt = UInt(-1);\n#if defined(JSON_HAS_INT64)\n  const Int64 Value::minInt64 = Int64(~(UInt64(-1) / 2));\n  const Int64 Value::maxInt64 = Int64(UInt64(-1) / 2);\n  const UInt64 Value::maxUInt64 = UInt64(-1);\n  // The constant is hard-coded because some compiler have trouble\n  // converting Value::maxUInt64 to a double correctly (AIX/xlC).\n  // Assumes that UInt64 is a 64 bits integer.\n  static const double maxUInt64AsDouble = 18446744073709551615.0;\n#endif // defined(JSON_HAS_INT64)\n  const LargestInt Value::minLargestInt = LargestInt(~(LargestUInt(-1) / 2));\n  const LargestInt Value::maxLargestInt = LargestInt(LargestUInt(-1) / 2);\n  const LargestUInt Value::maxLargestUInt = LargestUInt(-1);\n\n#if !defined(JSON_USE_INT64_DOUBLE_CONVERSION)\n  template <typename T, typename U>\n  static inline bool InRange(double d, T min, U max) {\n    return d >= min && d <= max;\n  }\n#else  // if !defined(JSON_USE_INT64_DOUBLE_CONVERSION)\n  static inline double integerToDouble(Json::UInt64 value) {\n    return static_cast<double>(Int64(value / 2)) * 2.0 + Int64(value & 1);\n  }\n\n  template <typename T> static inline double integerToDouble(T value) {\n    return static_cast<double>(value);\n  }\n\n  template <typename T, typename U>\n  static inline bool InRange(double d, T min, U max) {\n    return d >= integerToDouble(min) && d <= integerToDouble(max);\n  }\n#endif // if !defined(JSON_USE_INT64_DOUBLE_CONVERSION)\n\n  /** Duplicates the specified string value.\n   * @param value Pointer to the string to duplicate. Must be zero-terminated if\n   *              length is \"unknown\".\n   * @param length Length of the value. if equals to unknown, then it will be\n   *               computed using strlen(value).\n   * @return Pointer on the duplicate instance of string.\n   */\n  static inline char* duplicateStringValue(const char* value, size_t length) {\n    // Avoid an integer overflow in the call to malloc below by limiting length\n    // to a sane value.\n    if (length >= (size_t)Value::maxInt)\n      length = Value::maxInt - 1;\n\n    char* newString = static_cast<char*>(malloc(length + 1));\n    if (newString == NULL) {\n      throwRuntimeError(\"in Json::Value::duplicateStringValue(): \"\n                        \"Failed to allocate string value buffer\");\n    }\n    memcpy(newString, value, length);\n    newString[length] = 0;\n    return newString;\n  }\n\n  /* Record the length as a prefix.\n   */\n  static inline char* duplicateAndPrefixStringValue(const char* value,\n                                                    unsigned int length) {\n    // Avoid an integer overflow in the call to malloc below by limiting length\n    // to a sane value.\n    JSON_ASSERT_MESSAGE(length <=\n                            (unsigned)Value::maxInt - sizeof(unsigned) - 1U,\n                        \"in Json::Value::duplicateAndPrefixStringValue(): \"\n                        \"length too big for prefixing\");\n    unsigned actualLength =\n        length + static_cast<unsigned>(sizeof(unsigned)) + 1U;\n    char* newString = static_cast<char*>(malloc(actualLength));\n    if (newString == 0) {\n      throwRuntimeError(\"in Json::Value::duplicateAndPrefixStringValue(): \"\n                        \"Failed to allocate string value buffer\");\n    }\n    *reinterpret_cast<unsigned*>(newString) = length;\n    memcpy(newString + sizeof(unsigned), value, length);\n    newString[actualLength - 1U] =\n        0; // to avoid buffer over-run accidents by users later\n    return newString;\n  }\n  inline static void decodePrefixedString(bool isPrefixed, char const* prefixed,\n                                          unsigned* length,\n                                          char const** value) {\n    if (!isPrefixed) {\n      *length = static_cast<unsigned>(strlen(prefixed));\n      *value = prefixed;\n    } else {\n      *length = *reinterpret_cast<unsigned const*>(prefixed);\n      *value = prefixed + sizeof(unsigned);\n    }\n  }\n  /** Free the string duplicated by\n   * duplicateStringValue()/duplicateAndPrefixStringValue().\n   */\n  static inline void releaseStringValue(char* value) { free(value); }\n\n} // namespace Json\n\n// //////////////////////////////////////////////////////////////////\n// //////////////////////////////////////////////////////////////////\n// //////////////////////////////////////////////////////////////////\n// ValueInternals...\n// //////////////////////////////////////////////////////////////////\n// //////////////////////////////////////////////////////////////////\n// //////////////////////////////////////////////////////////////////\n#if !defined(JSON_IS_AMALGAMATION)\n\n#include \"json_valueiterator.inl\"\n#endif // if !defined(JSON_IS_AMALGAMATION)\n\nnamespace Json {\n\n  Exception::Exception(std::string const& msg) : msg_(msg) {}\n  Exception::~Exception() throw() {}\n  char const* Exception::what() const throw() { return msg_.c_str(); }\n  RuntimeError::RuntimeError(std::string const& msg) : Exception(msg) {}\n  LogicError::LogicError(std::string const& msg) : Exception(msg) {}\n  void throwRuntimeError(std::string const& msg) { throw RuntimeError(msg); }\n  void throwLogicError(std::string const& msg) { throw LogicError(msg); }\n\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // class Value::CommentInfo\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n\n  Value::CommentInfo::CommentInfo() : comment_(0) {}\n\n  Value::CommentInfo::~CommentInfo() {\n    if (comment_)\n      releaseStringValue(comment_);\n  }\n\n  void Value::CommentInfo::setComment(const char* text, size_t len) {\n    if (comment_) {\n      releaseStringValue(comment_);\n      comment_ = 0;\n    }\n    JSON_ASSERT(text != 0);\n    JSON_ASSERT_MESSAGE(\n        text[0] == '\\0' || text[0] == '/',\n        \"in Json::Value::setComment(): Comments must start with /\");\n    // It seems that /**/ style comments are acceptable as well.\n    comment_ = duplicateStringValue(text, len);\n  }\n\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // class Value::CZString\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n\n  // Notes: policy_ indicates if the string was allocated when\n  // a string is stored.\n\n  Value::CZString::CZString(ArrayIndex aindex) : cstr_(0), index_(aindex) {}\n\n  Value::CZString::CZString(char const* str, unsigned ulength,\n                            DuplicationPolicy allocate)\n      : cstr_(str) {\n    // allocate != duplicate\n    storage_.policy_ = allocate & 0x3;\n    storage_.length_ = ulength & 0x3FFFFFFF;\n  }\n\n  Value::CZString::CZString(const CZString& other)\n      : cstr_(other.storage_.policy_ != noDuplication && other.cstr_ != 0\n                  ? duplicateStringValue(other.cstr_, other.storage_.length_)\n                  : other.cstr_) {\n    storage_.policy_ =\n        (other.cstr_ ? (static_cast<DuplicationPolicy>(\n                            other.storage_.policy_) == noDuplication\n                            ? noDuplication\n                            : duplicate)\n                     : static_cast<DuplicationPolicy>(other.storage_.policy_));\n    storage_.length_ = other.storage_.length_;\n  }\n\n#if JSON_HAS_RVALUE_REFERENCES\n  Value::CZString::CZString(CZString&& other)\n      : cstr_(other.cstr_), index_(other.index_) {\n    other.cstr_ = nullptr;\n  }\n#endif\n\n  Value::CZString::~CZString() {\n    if (cstr_ && storage_.policy_ == duplicate)\n      releaseStringValue(const_cast<char*>(cstr_));\n  }\n\n  void Value::CZString::swap(CZString& other) {\n    std::swap(cstr_, other.cstr_);\n    std::swap(index_, other.index_);\n  }\n\n  Value::CZString& Value::CZString::operator=(CZString other) {\n    swap(other);\n    return *this;\n  }\n\n  bool Value::CZString::operator<(const CZString& other) const {\n    if (!cstr_)\n      return index_ < other.index_;\n    // return strcmp(cstr_, other.cstr_) < 0;\n    // Assume both are strings.\n    unsigned this_len = this->storage_.length_;\n    unsigned other_len = other.storage_.length_;\n    unsigned min_len = std::min(this_len, other_len);\n    int comp = memcmp(this->cstr_, other.cstr_, min_len);\n    if (comp < 0)\n      return true;\n    if (comp > 0)\n      return false;\n    return (this_len < other_len);\n  }\n\n  bool Value::CZString::operator==(const CZString& other) const {\n    if (!cstr_)\n      return index_ == other.index_;\n    // return strcmp(cstr_, other.cstr_) == 0;\n    // Assume both are strings.\n    unsigned this_len = this->storage_.length_;\n    unsigned other_len = other.storage_.length_;\n    if (this_len != other_len)\n      return false;\n    int comp = memcmp(this->cstr_, other.cstr_, this_len);\n    return comp == 0;\n  }\n\n  ArrayIndex Value::CZString::index() const { return index_; }\n\n  // const char* Value::CZString::c_str() const { return cstr_; }\n  const char* Value::CZString::data() const { return cstr_; }\n  unsigned Value::CZString::length() const { return storage_.length_; }\n  bool Value::CZString::isStaticString() const {\n    return storage_.policy_ == noDuplication;\n  }\n\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // class Value::Value\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n  // //////////////////////////////////////////////////////////////////\n\n  /*! \\internal Default constructor initialization must be equivalent to:\n   * memset( this, 0, sizeof(Value) )\n   * This optimization is used in ValueInternalMap fast allocator.\n   */\n  Value::Value(ValueType vtype) {\n    initBasic(vtype);\n    switch (vtype) {\n    case nullValue:\n      break;\n    case intValue:\n    case uintValue:\n      value_.int_ = 0;\n      break;\n    case realValue:\n      value_.real_ = 0.0;\n      break;\n    case stringValue:\n      value_.string_ = 0;\n      break;\n    case arrayValue:\n    case objectValue:\n      value_.map_ = new ObjectValues();\n      break;\n    case booleanValue:\n      value_.bool_ = false;\n      break;\n    default:\n      JSON_ASSERT_UNREACHABLE;\n    }\n  }\n\n  Value::Value(Int value) {\n    initBasic(intValue);\n    value_.int_ = value;\n  }\n\n  Value::Value(UInt value) {\n    initBasic(uintValue);\n    value_.uint_ = value;\n  }\n#if defined(JSON_HAS_INT64)\n  Value::Value(Int64 value) {\n    initBasic(intValue);\n    value_.int_ = value;\n  }\n  Value::Value(UInt64 value) {\n    initBasic(uintValue);\n    value_.uint_ = value;\n  }\n#endif // defined(JSON_HAS_INT64)\n\n  Value::Value(double value) {\n    initBasic(realValue);\n    value_.real_ = value;\n  }\n\n  Value::Value(const char* value) {\n    initBasic(stringValue, true);\n    value_.string_ = duplicateAndPrefixStringValue(\n        value, static_cast<unsigned>(strlen(value)));\n  }\n\n  Value::Value(const char* beginValue, const char* endValue) {\n    initBasic(stringValue, true);\n    value_.string_ = duplicateAndPrefixStringValue(\n        beginValue, static_cast<unsigned>(endValue - beginValue));\n  }\n\n  Value::Value(const std::string& value) {\n    initBasic(stringValue, true);\n    value_.string_ = duplicateAndPrefixStringValue(\n        value.data(), static_cast<unsigned>(value.length()));\n  }\n\n  Value::Value(const StaticString& value) {\n    initBasic(stringValue);\n    value_.string_ = const_cast<char*>(value.c_str());\n  }\n\n#ifdef JSON_USE_CPPTL\n  Value::Value(const CppTL::ConstString& value) {\n    initBasic(stringValue, true);\n    value_.string_ = duplicateAndPrefixStringValue(\n        value, static_cast<unsigned>(value.length()));\n  }\n#endif\n\n  Value::Value(bool value) {\n    initBasic(booleanValue);\n    value_.bool_ = value;\n  }\n\n  Value::Value(Value const& other)\n      : type_(other.type_), allocated_(false), comments_(0),\n        start_(other.start_), limit_(other.limit_) {\n    switch (type_) {\n    case nullValue:\n    case intValue:\n    case uintValue:\n    case realValue:\n    case booleanValue:\n      value_ = other.value_;\n      break;\n    case stringValue:\n      if (other.value_.string_ && other.allocated_) {\n        unsigned len;\n        char const* str;\n        decodePrefixedString(other.allocated_, other.value_.string_, &len,\n                             &str);\n        value_.string_ = duplicateAndPrefixStringValue(str, len);\n        allocated_ = true;\n      } else {\n        value_.string_ = other.value_.string_;\n        allocated_ = false;\n      }\n      break;\n    case arrayValue:\n    case objectValue:\n      value_.map_ = new ObjectValues(*other.value_.map_);\n      break;\n    default:\n      JSON_ASSERT_UNREACHABLE;\n    }\n    if (other.comments_) {\n      comments_ = new CommentInfo[numberOfCommentPlacement];\n      for (int comment = 0; comment < numberOfCommentPlacement; ++comment) {\n        const CommentInfo& otherComment = other.comments_[comment];\n        if (otherComment.comment_)\n          comments_[comment].setComment(otherComment.comment_,\n                                        strlen(otherComment.comment_));\n      }\n    }\n  }\n\n#if JSON_HAS_RVALUE_REFERENCES\n  // Move constructor\n  Value::Value(Value&& other) {\n    initBasic(nullValue);\n    swap(other);\n  }\n#endif\n\n  Value::~Value() {\n    switch (type_) {\n    case nullValue:\n    case intValue:\n    case uintValue:\n    case realValue:\n    case booleanValue:\n      break;\n    case stringValue:\n      if (allocated_)\n        releaseStringValue(value_.string_);\n      break;\n    case arrayValue:\n    case objectValue:\n      delete value_.map_;\n      break;\n    default:\n      JSON_ASSERT_UNREACHABLE;\n    }\n\n    if (comments_)\n      delete[] comments_;\n  }\n\n  Value& Value::operator=(Value other) {\n    swap(other);\n    return *this;\n  }\n\n  void Value::swapPayload(Value& other) {\n    ValueType temp = type_;\n    type_ = other.type_;\n    other.type_ = temp;\n    std::swap(value_, other.value_);\n    int temp2 = allocated_;\n    allocated_ = other.allocated_;\n    other.allocated_ = temp2 & 0x1;\n  }\n\n  void Value::swap(Value& other) {\n    swapPayload(other);\n    std::swap(comments_, other.comments_);\n    std::swap(start_, other.start_);\n    std::swap(limit_, other.limit_);\n  }\n\n  ValueType Value::type() const { return type_; }\n\n  int Value::compare(const Value& other) const {\n    if (*this < other)\n      return -1;\n    if (*this > other)\n      return 1;\n    return 0;\n  }\n\n  bool Value::operator<(const Value& other) const {\n    int typeDelta = type_ - other.type_;\n    if (typeDelta)\n      return typeDelta < 0 ? true : false;\n    switch (type_) {\n    case nullValue:\n      return false;\n    case intValue:\n      return value_.int_ < other.value_.int_;\n    case uintValue:\n      return value_.uint_ < other.value_.uint_;\n    case realValue:\n      return value_.real_ < other.value_.real_;\n    case booleanValue:\n      return value_.bool_ < other.value_.bool_;\n    case stringValue: {\n      if ((value_.string_ == 0) || (other.value_.string_ == 0)) {\n        if (other.value_.string_)\n          return true;\n        else\n          return false;\n      }\n      unsigned this_len;\n      unsigned other_len;\n      char const* this_str;\n      char const* other_str;\n      decodePrefixedString(this->allocated_, this->value_.string_, &this_len,\n                           &this_str);\n      decodePrefixedString(other.allocated_, other.value_.string_, &other_len,\n                           &other_str);\n      unsigned min_len = std::min(this_len, other_len);\n      int comp = memcmp(this_str, other_str, min_len);\n      if (comp < 0)\n        return true;\n      if (comp > 0)\n        return false;\n      return (this_len < other_len);\n    }\n    case arrayValue:\n    case objectValue: {\n      int delta = int(value_.map_->size() - other.value_.map_->size());\n      if (delta)\n        return delta < 0;\n      return (*value_.map_) < (*other.value_.map_);\n    }\n    default:\n      JSON_ASSERT_UNREACHABLE;\n    }\n    return false; // unreachable\n  }\n\n  bool Value::operator<=(const Value& other) const { return !(other < *this); }\n\n  bool Value::operator>=(const Value& other) const { return !(*this < other); }\n\n  bool Value::operator>(const Value& other) const { return other < *this; }\n\n  bool Value::operator==(const Value& other) const {\n    // if ( type_ != other.type_ )\n    // GCC 2.95.3 says:\n    // attempt to take address of bit-field structure member\n    // `Json::Value::type_' Beats me, but a temp solves the problem.\n    int temp = other.type_;\n    if (type_ != temp)\n      return false;\n    switch (type_) {\n    case nullValue:\n      return true;\n    case intValue:\n      return value_.int_ == other.value_.int_;\n    case uintValue:\n      return value_.uint_ == other.value_.uint_;\n    case realValue:\n      return value_.real_ == other.value_.real_;\n    case booleanValue:\n      return value_.bool_ == other.value_.bool_;\n    case stringValue: {\n      if ((value_.string_ == 0) || (other.value_.string_ == 0)) {\n        return (value_.string_ == other.value_.string_);\n      }\n      unsigned this_len;\n      unsigned other_len;\n      char const* this_str;\n      char const* other_str;\n      decodePrefixedString(this->allocated_, this->value_.string_, &this_len,\n                           &this_str);\n      decodePrefixedString(other.allocated_, other.value_.string_, &other_len,\n                           &other_str);\n      if (this_len != other_len)\n        return false;\n      int comp = memcmp(this_str, other_str, this_len);\n      return comp == 0;\n    }\n    case arrayValue:\n    case objectValue:\n      return value_.map_->size() == other.value_.map_->size() &&\n             (*value_.map_) == (*other.value_.map_);\n    default:\n      JSON_ASSERT_UNREACHABLE;\n    }\n    return false; // unreachable\n  }\n\n  bool Value::operator!=(const Value& other) const { return !(*this == other); }\n\n  const char* Value::asCString() const {\n    JSON_ASSERT_MESSAGE(type_ == stringValue,\n                        \"in Json::Value::asCString(): requires stringValue\");\n    if (value_.string_ == 0)\n      return 0;\n    unsigned this_len;\n    char const* this_str;\n    decodePrefixedString(this->allocated_, this->value_.string_, &this_len,\n                         &this_str);\n    return this_str;\n  }\n\n  bool Value::getString(char const** str, char const** cend) const {\n    if (type_ != stringValue)\n      return false;\n    if (value_.string_ == 0)\n      return false;\n    unsigned length;\n    decodePrefixedString(this->allocated_, this->value_.string_, &length, str);\n    *cend = *str + length;\n    return true;\n  }\n\n  std::string Value::asString() const {\n    switch (type_) {\n    case nullValue:\n      return \"\";\n    case stringValue: {\n      if (value_.string_ == 0)\n        return \"\";\n      unsigned this_len;\n      char const* this_str;\n      decodePrefixedString(this->allocated_, this->value_.string_, &this_len,\n                           &this_str);\n      return std::string(this_str, this_len);\n    }\n    case booleanValue:\n      return value_.bool_ ? \"true\" : \"false\";\n    case intValue:\n      return valueToString(value_.int_);\n    case uintValue:\n      return valueToString(value_.uint_);\n    case realValue:\n      return valueToString(value_.real_);\n    default:\n      JSON_FAIL_MESSAGE(\"Type is not convertible to string\");\n    }\n  }\n\n#ifdef JSON_USE_CPPTL\n  CppTL::ConstString Value::asConstString() const {\n    unsigned len;\n    char const* str;\n    decodePrefixedString(allocated_, value_.string_, &len, &str);\n    return CppTL::ConstString(str, len);\n  }\n#endif\n\n  Value::Int Value::asInt() const {\n    switch (type_) {\n    case intValue:\n      JSON_ASSERT_MESSAGE(isInt(), \"LargestInt out of Int range\");\n      return Int(value_.int_);\n    case uintValue:\n      JSON_ASSERT_MESSAGE(isInt(), \"LargestUInt out of Int range\");\n      return Int(value_.uint_);\n    case realValue:\n      JSON_ASSERT_MESSAGE(InRange(value_.real_, minInt, maxInt),\n                          \"double out of Int range\");\n      return Int(value_.real_);\n    case nullValue:\n      return 0;\n    case booleanValue:\n      return value_.bool_ ? 1 : 0;\n    default:\n      break;\n    }\n    JSON_FAIL_MESSAGE(\"Value is not convertible to Int.\");\n  }\n\n  Value::UInt Value::asUInt() const {\n    switch (type_) {\n    case intValue:\n      JSON_ASSERT_MESSAGE(isUInt(), \"LargestInt out of UInt range\");\n      return UInt(value_.int_);\n    case uintValue:\n      JSON_ASSERT_MESSAGE(isUInt(), \"LargestUInt out of UInt range\");\n      return UInt(value_.uint_);\n    case realValue:\n      JSON_ASSERT_MESSAGE(InRange(value_.real_, 0, maxUInt),\n                          \"double out of UInt range\");\n      return UInt(value_.real_);\n    case nullValue:\n      return 0;\n    case booleanValue:\n      return value_.bool_ ? 1 : 0;\n    default:\n      break;\n    }\n    JSON_FAIL_MESSAGE(\"Value is not convertible to UInt.\");\n  }\n\n#if defined(JSON_HAS_INT64)\n\n  Value::Int64 Value::asInt64() const {\n    switch (type_) {\n    case intValue:\n      return Int64(value_.int_);\n    case uintValue:\n      JSON_ASSERT_MESSAGE(isInt64(), \"LargestUInt out of Int64 range\");\n      return Int64(value_.uint_);\n    case realValue:\n      JSON_ASSERT_MESSAGE(InRange(value_.real_, minInt64, maxInt64),\n                          \"double out of Int64 range\");\n      return Int64(value_.real_);\n    case nullValue:\n      return 0;\n    case booleanValue:\n      return value_.bool_ ? 1 : 0;\n    default:\n      break;\n    }\n    JSON_FAIL_MESSAGE(\"Value is not convertible to Int64.\");\n  }\n\n  Value::UInt64 Value::asUInt64() const {\n    switch (type_) {\n    case intValue:\n      JSON_ASSERT_MESSAGE(isUInt64(), \"LargestInt out of UInt64 range\");\n      return UInt64(value_.int_);\n    case uintValue:\n      return UInt64(value_.uint_);\n    case realValue:\n      JSON_ASSERT_MESSAGE(InRange(value_.real_, 0, maxUInt64),\n                          \"double out of UInt64 range\");\n      return UInt64(value_.real_);\n    case nullValue:\n      return 0;\n    case booleanValue:\n      return value_.bool_ ? 1 : 0;\n    default:\n      break;\n    }\n    JSON_FAIL_MESSAGE(\"Value is not convertible to UInt64.\");\n  }\n#endif // if defined(JSON_HAS_INT64)\n\n  LargestInt Value::asLargestInt() const {\n#if defined(JSON_NO_INT64)\n    return asInt();\n#else\n    return asInt64();\n#endif\n  }\n\n  LargestUInt Value::asLargestUInt() const {\n#if defined(JSON_NO_INT64)\n    return asUInt();\n#else\n    return asUInt64();\n#endif\n  }\n\n  double Value::asDouble() const {\n    switch (type_) {\n    case intValue:\n      return static_cast<double>(value_.int_);\n    case uintValue:\n#if !defined(JSON_USE_INT64_DOUBLE_CONVERSION)\n      return static_cast<double>(value_.uint_);\n#else  // if !defined(JSON_USE_INT64_DOUBLE_CONVERSION)\n      return integerToDouble(value_.uint_);\n#endif // if !defined(JSON_USE_INT64_DOUBLE_CONVERSION)\n    case realValue:\n      return value_.real_;\n    case nullValue:\n      return 0.0;\n    case booleanValue:\n      return value_.bool_ ? 1.0 : 0.0;\n    default:\n      break;\n    }\n    JSON_FAIL_MESSAGE(\"Value is not convertible to double.\");\n  }\n\n  float Value::asFloat() const {\n    switch (type_) {\n    case intValue:\n      return static_cast<float>(value_.int_);\n    case uintValue:\n#if !defined(JSON_USE_INT64_DOUBLE_CONVERSION)\n      return static_cast<float>(value_.uint_);\n#else  // if !defined(JSON_USE_INT64_DOUBLE_CONVERSION)\n      return integerToDouble(value_.uint_);\n#endif // if !defined(JSON_USE_INT64_DOUBLE_CONVERSION)\n    case realValue:\n      return static_cast<float>(value_.real_);\n    case nullValue:\n      return 0.0;\n    case booleanValue:\n      return value_.bool_ ? 1.0f : 0.0f;\n    default:\n      break;\n    }\n    JSON_FAIL_MESSAGE(\"Value is not convertible to float.\");\n  }\n\n  bool Value::asBool() const {\n    switch (type_) {\n    case booleanValue:\n      return value_.bool_;\n    case nullValue:\n      return false;\n    case intValue:\n      return value_.int_ ? true : false;\n    case uintValue:\n      return value_.uint_ ? true : false;\n    case realValue:\n      // This is kind of strange. Not recommended.\n      return (value_.real_ != 0.0) ? true : false;\n    default:\n      break;\n    }\n    JSON_FAIL_MESSAGE(\"Value is not convertible to bool.\");\n  }\n\n  bool Value::isConvertibleTo(ValueType other) const {\n    switch (other) {\n    case nullValue:\n      return (isNumeric() && asDouble() == 0.0) ||\n             (type_ == booleanValue && value_.bool_ == false) ||\n             (type_ == stringValue && asString() == \"\") ||\n             (type_ == arrayValue && value_.map_->size() == 0) ||\n             (type_ == objectValue && value_.map_->size() == 0) ||\n             type_ == nullValue;\n    case intValue:\n      return isInt() ||\n             (type_ == realValue && InRange(value_.real_, minInt, maxInt)) ||\n             type_ == booleanValue || type_ == nullValue;\n    case uintValue:\n      return isUInt() ||\n             (type_ == realValue && InRange(value_.real_, 0, maxUInt)) ||\n             type_ == booleanValue || type_ == nullValue;\n    case realValue:\n      return isNumeric() || type_ == booleanValue || type_ == nullValue;\n    case booleanValue:\n      return isNumeric() || type_ == booleanValue || type_ == nullValue;\n    case stringValue:\n      return isNumeric() || type_ == booleanValue || type_ == stringValue ||\n             type_ == nullValue;\n    case arrayValue:\n      return type_ == arrayValue || type_ == nullValue;\n    case objectValue:\n      return type_ == objectValue || type_ == nullValue;\n    }\n    JSON_ASSERT_UNREACHABLE;\n    return false;\n  }\n\n  /// Number of values in array or object\n  ArrayIndex Value::size() const {\n    switch (type_) {\n    case nullValue:\n    case intValue:\n    case uintValue:\n    case realValue:\n    case booleanValue:\n    case stringValue:\n      return 0;\n    case arrayValue: // size of the array is highest index + 1\n      if (!value_.map_->empty()) {\n        ObjectValues::const_iterator itLast = value_.map_->end();\n        --itLast;\n        return (*itLast).first.index() + 1;\n      }\n      return 0;\n    case objectValue:\n      return ArrayIndex(value_.map_->size());\n    }\n    JSON_ASSERT_UNREACHABLE;\n    return 0; // unreachable;\n  }\n\n  bool Value::empty() const {\n    if (isNull() || isArray() || isObject())\n      return size() == 0u;\n    else\n      return false;\n  }\n\n  bool Value::operator!() const { return isNull(); }\n\n  void Value::clear() {\n    JSON_ASSERT_MESSAGE(type_ == nullValue || type_ == arrayValue ||\n                            type_ == objectValue,\n                        \"in Json::Value::clear(): requires complex value\");\n    start_ = 0;\n    limit_ = 0;\n    switch (type_) {\n    case arrayValue:\n    case objectValue:\n      value_.map_->clear();\n      break;\n    default:\n      break;\n    }\n  }\n\n  void Value::resize(ArrayIndex newSize) {\n    JSON_ASSERT_MESSAGE(type_ == nullValue || type_ == arrayValue,\n                        \"in Json::Value::resize(): requires arrayValue\");\n    if (type_ == nullValue)\n      *this = Value(arrayValue);\n    ArrayIndex oldSize = size();\n    if (newSize == 0)\n      clear();\n    else if (newSize > oldSize)\n      (*this)[newSize - 1];\n    else {\n      for (ArrayIndex index = newSize; index < oldSize; ++index) {\n        value_.map_->erase(index);\n      }\n      assert(size() == newSize);\n    }\n  }\n\n  Value& Value::operator[](ArrayIndex index) {\n    JSON_ASSERT_MESSAGE(\n        type_ == nullValue || type_ == arrayValue,\n        \"in Json::Value::operator[](ArrayIndex): requires arrayValue\");\n    if (type_ == nullValue)\n      *this = Value(arrayValue);\n    CZString key(index);\n    ObjectValues::iterator it = value_.map_->lower_bound(key);\n    if (it != value_.map_->end() && (*it).first == key)\n      return (*it).second;\n\n    ObjectValues::value_type defaultValue(key, nullRef);\n    it = value_.map_->insert(it, defaultValue);\n    return (*it).second;\n  }\n\n  Value& Value::operator[](int index) {\n    JSON_ASSERT_MESSAGE(\n        index >= 0,\n        \"in Json::Value::operator[](int index): index cannot be negative\");\n    return (*this)[ArrayIndex(index)];\n  }\n\n  const Value& Value::operator[](ArrayIndex index) const {\n    JSON_ASSERT_MESSAGE(\n        type_ == nullValue || type_ == arrayValue,\n        \"in Json::Value::operator[](ArrayIndex)const: requires arrayValue\");\n    if (type_ == nullValue)\n      return nullRef;\n    CZString key(index);\n    ObjectValues::const_iterator it = value_.map_->find(key);\n    if (it == value_.map_->end())\n      return nullRef;\n    return (*it).second;\n  }\n\n  const Value& Value::operator[](int index) const {\n    JSON_ASSERT_MESSAGE(index >= 0, \"in Json::Value::operator[](int index) \"\n                                    \"const: index cannot be negative\");\n    return (*this)[ArrayIndex(index)];\n  }\n\n  void Value::initBasic(ValueType vtype, bool allocated) {\n    type_ = vtype;\n    allocated_ = allocated;\n    comments_ = 0;\n    start_ = 0;\n    limit_ = 0;\n  }\n\n  // Access an object value by name, create a null member if it does not exist.\n  // @pre Type of '*this' is object or null.\n  // @param key is null-terminated.\n  Value& Value::resolveReference(const char* key) {\n    JSON_ASSERT_MESSAGE(\n        type_ == nullValue || type_ == objectValue,\n        \"in Json::Value::resolveReference(): requires objectValue\");\n    if (type_ == nullValue)\n      *this = Value(objectValue);\n    CZString actualKey(key, static_cast<unsigned>(strlen(key)),\n                       CZString::noDuplication); // NOTE!\n    ObjectValues::iterator it = value_.map_->lower_bound(actualKey);\n    if (it != value_.map_->end() && (*it).first == actualKey)\n      return (*it).second;\n\n    ObjectValues::value_type defaultValue(actualKey, nullRef);\n    it = value_.map_->insert(it, defaultValue);\n    Value& value = (*it).second;\n    return value;\n  }\n\n  // @param key is not null-terminated.\n  Value& Value::resolveReference(char const* key, char const* cend) {\n    JSON_ASSERT_MESSAGE(\n        type_ == nullValue || type_ == objectValue,\n        \"in Json::Value::resolveReference(key, end): requires objectValue\");\n    if (type_ == nullValue)\n      *this = Value(objectValue);\n    CZString actualKey(key, static_cast<unsigned>(cend - key),\n                       CZString::duplicateOnCopy);\n    ObjectValues::iterator it = value_.map_->lower_bound(actualKey);\n    if (it != value_.map_->end() && (*it).first == actualKey)\n      return (*it).second;\n\n    ObjectValues::value_type defaultValue(actualKey, nullRef);\n    it = value_.map_->insert(it, defaultValue);\n    Value& value = (*it).second;\n    return value;\n  }\n\n  Value Value::get(ArrayIndex index, const Value& defaultValue) const {\n    const Value* value = &((*this)[index]);\n    return value == &nullRef ? defaultValue : *value;\n  }\n\n  bool Value::isValidIndex(ArrayIndex index) const { return index < size(); }\n\n  Value const* Value::find(char const* key, char const* cend) const {\n    JSON_ASSERT_MESSAGE(type_ == nullValue || type_ == objectValue,\n                        \"in Json::Value::find(key, end, found): requires \"\n                        \"objectValue or nullValue\");\n    if (type_ == nullValue)\n      return NULL;\n    CZString actualKey(key, static_cast<unsigned>(cend - key),\n                       CZString::noDuplication);\n    ObjectValues::const_iterator it = value_.map_->find(actualKey);\n    if (it == value_.map_->end())\n      return NULL;\n    return &(*it).second;\n  }\n  const Value& Value::operator[](const char* key) const {\n    Value const* found = find(key, key + strlen(key));\n    if (!found)\n      return nullRef;\n    return *found;\n  }\n  Value const& Value::operator[](std::string const& key) const {\n    Value const* found = find(key.data(), key.data() + key.length());\n    if (!found)\n      return nullRef;\n    return *found;\n  }\n\n  Value& Value::operator[](const char* key) {\n    return resolveReference(key, key + strlen(key));\n  }\n\n  Value& Value::operator[](const std::string& key) {\n    return resolveReference(key.data(), key.data() + key.length());\n  }\n\n  Value& Value::operator[](const StaticString& key) {\n    return resolveReference(key.c_str());\n  }\n\n#ifdef JSON_USE_CPPTL\n  Value& Value::operator[](const CppTL::ConstString& key) {\n    return resolveReference(key.c_str(), key.end_c_str());\n  }\n  Value const& Value::operator[](CppTL::ConstString const& key) const {\n    Value const* found = find(key.c_str(), key.end_c_str());\n    if (!found)\n      return nullRef;\n    return *found;\n  }\n#endif\n\n  Value& Value::append(const Value& value) { return (*this)[size()] = value; }\n\n  Value Value::get(char const* key, char const* cend,\n                   Value const& defaultValue) const {\n    Value const* found = find(key, cend);\n    return !found ? defaultValue : *found;\n  }\n  Value Value::get(char const* key, Value const& defaultValue) const {\n    return get(key, key + strlen(key), defaultValue);\n  }\n  Value Value::get(std::string const& key, Value const& defaultValue) const {\n    return get(key.data(), key.data() + key.length(), defaultValue);\n  }\n\n  bool Value::removeMember(const char* key, const char* cend, Value* removed) {\n    if (type_ != objectValue) {\n      return false;\n    }\n    CZString actualKey(key, static_cast<unsigned>(cend - key),\n                       CZString::noDuplication);\n    ObjectValues::iterator it = value_.map_->find(actualKey);\n    if (it == value_.map_->end())\n      return false;\n    *removed = it->second;\n    value_.map_->erase(it);\n    return true;\n  }\n  bool Value::removeMember(const char* key, Value* removed) {\n    return removeMember(key, key + strlen(key), removed);\n  }\n  bool Value::removeMember(std::string const& key, Value* removed) {\n    return removeMember(key.data(), key.data() + key.length(), removed);\n  }\n  Value Value::removeMember(const char* key) {\n    JSON_ASSERT_MESSAGE(type_ == nullValue || type_ == objectValue,\n                        \"in Json::Value::removeMember(): requires objectValue\");\n    if (type_ == nullValue)\n      return nullRef;\n\n    Value removed; // null\n    removeMember(key, key + strlen(key), &removed);\n    return removed; // still null if removeMember() did nothing\n  }\n  Value Value::removeMember(const std::string& key) {\n    return removeMember(key.c_str());\n  }\n\n  bool Value::removeIndex(ArrayIndex index, Value* removed) {\n    if (type_ != arrayValue) {\n      return false;\n    }\n    CZString key(index);\n    ObjectValues::iterator it = value_.map_->find(key);\n    if (it == value_.map_->end()) {\n      return false;\n    }\n    *removed = it->second;\n    ArrayIndex oldSize = size();\n    // shift left all items left, into the place of the \"removed\"\n    for (ArrayIndex i = index; i < (oldSize - 1); ++i) {\n      CZString keey(i);\n      (*value_.map_)[keey] = (*this)[i + 1];\n    }\n    // erase the last one (\"leftover\")\n    CZString keyLast(oldSize - 1);\n    ObjectValues::iterator itLast = value_.map_->find(keyLast);\n    value_.map_->erase(itLast);\n    return true;\n  }\n\n#ifdef JSON_USE_CPPTL\n  Value Value::get(const CppTL::ConstString& key,\n                   const Value& defaultValue) const {\n    return get(key.c_str(), key.end_c_str(), defaultValue);\n  }\n#endif\n\n  bool Value::isMember(char const* key, char const* cend) const {\n    Value const* value = find(key, cend);\n    return NULL != value;\n  }\n  bool Value::isMember(char const* key) const {\n    return isMember(key, key + strlen(key));\n  }\n  bool Value::isMember(std::string const& key) const {\n    return isMember(key.data(), key.data() + key.length());\n  }\n\n#ifdef JSON_USE_CPPTL\n  bool Value::isMember(const CppTL::ConstString& key) const {\n    return isMember(key.c_str(), key.end_c_str());\n  }\n#endif\n\n  Value::Members Value::getMemberNames() const {\n    JSON_ASSERT_MESSAGE(\n        type_ == nullValue || type_ == objectValue,\n        \"in Json::Value::getMemberNames(), value must be objectValue\");\n    if (type_ == nullValue)\n      return Value::Members();\n    Members members;\n    members.reserve(value_.map_->size());\n    ObjectValues::const_iterator it = value_.map_->begin();\n    ObjectValues::const_iterator itEnd = value_.map_->end();\n    for (; it != itEnd; ++it) {\n      members.push_back(std::string((*it).first.data(), (*it).first.length()));\n    }\n    return members;\n  }\n  //\n  //# ifdef JSON_USE_CPPTL\n  // EnumMemberNames\n  // Value::enumMemberNames() const\n  //{\n  //   if ( type_ == objectValue )\n  //   {\n  //      return CppTL::Enum::any(  CppTL::Enum::transform(\n  //         CppTL::Enum::keys( *(value_.map_), CppTL::Type<const CZString &>()\n  //         ), MemberNamesTransform() ) );\n  //   }\n  //   return EnumMemberNames();\n  //}\n  //\n  //\n  // EnumValues\n  // Value::enumValues() const\n  //{\n  //   if ( type_ == objectValue  ||  type_ == arrayValue )\n  //      return CppTL::Enum::anyValues( *(value_.map_),\n  //                                     CppTL::Type<const Value &>() );\n  //   return EnumValues();\n  //}\n  //\n  //# endif\n\n  static bool IsIntegral(double d) {\n    double integral_part;\n    return modf(d, &integral_part) == 0.0;\n  }\n\n  bool Value::isNull() const { return type_ == nullValue; }\n\n  bool Value::isBool() const { return type_ == booleanValue; }\n\n  bool Value::isInt() const {\n    switch (type_) {\n    case intValue:\n      return value_.int_ >= minInt && value_.int_ <= maxInt;\n    case uintValue:\n      return value_.uint_ <= UInt(maxInt);\n    case realValue:\n      return value_.real_ >= minInt && value_.real_ <= maxInt &&\n             IsIntegral(value_.real_);\n    default:\n      break;\n    }\n    return false;\n  }\n\n  bool Value::isUInt() const {\n    switch (type_) {\n    case intValue:\n      return value_.int_ >= 0 &&\n             LargestUInt(value_.int_) <= LargestUInt(maxUInt);\n    case uintValue:\n      return value_.uint_ <= maxUInt;\n    case realValue:\n      return value_.real_ >= 0 && value_.real_ <= maxUInt &&\n             IsIntegral(value_.real_);\n    default:\n      break;\n    }\n    return false;\n  }\n\n  bool Value::isInt64() const {\n#if defined(JSON_HAS_INT64)\n    switch (type_) {\n    case intValue:\n      return true;\n    case uintValue:\n      return value_.uint_ <= UInt64(maxInt64);\n    case realValue:\n      // Note that maxInt64 (= 2^63 - 1) is not exactly representable as a\n      // double, so double(maxInt64) will be rounded up to 2^63. Therefore we\n      // require the value to be strictly less than the limit.\n      return value_.real_ >= double(minInt64) &&\n             value_.real_ < double(maxInt64) && IsIntegral(value_.real_);\n    default:\n      break;\n    }\n#endif // JSON_HAS_INT64\n    return false;\n  }\n\n  bool Value::isUInt64() const {\n#if defined(JSON_HAS_INT64)\n    switch (type_) {\n    case intValue:\n      return value_.int_ >= 0;\n    case uintValue:\n      return true;\n    case realValue:\n      // Note that maxUInt64 (= 2^64 - 1) is not exactly representable as a\n      // double, so double(maxUInt64) will be rounded up to 2^64. Therefore we\n      // require the value to be strictly less than the limit.\n      return value_.real_ >= 0 && value_.real_ < maxUInt64AsDouble &&\n             IsIntegral(value_.real_);\n    default:\n      break;\n    }\n#endif // JSON_HAS_INT64\n    return false;\n  }\n\n  bool Value::isIntegral() const {\n#if defined(JSON_HAS_INT64)\n    return isInt64() || isUInt64();\n#else\n    return isInt() || isUInt();\n#endif\n  }\n\n  bool Value::isDouble() const { return type_ == realValue || isIntegral(); }\n\n  bool Value::isNumeric() const { return isIntegral() || isDouble(); }\n\n  bool Value::isString() const { return type_ == stringValue; }\n\n  bool Value::isArray() const { return type_ == arrayValue; }\n\n  bool Value::isObject() const { return type_ == objectValue; }\n\n  void Value::setComment(const char* comment, size_t len,\n                         CommentPlacement placement) {\n    if (!comments_)\n      comments_ = new CommentInfo[numberOfCommentPlacement];\n    if ((len > 0) && (comment[len - 1] == '\\n')) {\n      // Always discard trailing newline, to aid indentation.\n      len -= 1;\n    }\n    comments_[placement].setComment(comment, len);\n  }\n\n  void Value::setComment(const char* comment, CommentPlacement placement) {\n    setComment(comment, strlen(comment), placement);\n  }\n\n  void Value::setComment(const std::string& comment,\n                         CommentPlacement placement) {\n    setComment(comment.c_str(), comment.length(), placement);\n  }\n\n  bool Value::hasComment(CommentPlacement placement) const {\n    return comments_ != 0 && comments_[placement].comment_ != 0;\n  }\n\n  std::string Value::getComment(CommentPlacement placement) const {\n    if (hasComment(placement))\n      return comments_[placement].comment_;\n    return \"\";\n  }\n\n  void Value::setOffsetStart(size_t start) { start_ = start; }\n\n  void Value::setOffsetLimit(size_t limit) { limit_ = limit; }\n\n  size_t Value::getOffsetStart() const { return start_; }\n\n  size_t Value::getOffsetLimit() const { return limit_; }\n\n  std::string Value::toStyledString() const {\n    StyledWriter writer;\n    return writer.write(*this);\n  }\n\n  Value::const_iterator Value::begin() const {\n    switch (type_) {\n    case arrayValue:\n    case objectValue:\n      if (value_.map_)\n        return const_iterator(value_.map_->begin());\n      break;\n    default:\n      break;\n    }\n    return const_iterator();\n  }\n\n  Value::const_iterator Value::end() const {\n    switch (type_) {\n    case arrayValue:\n    case objectValue:\n      if (value_.map_)\n        return const_iterator(value_.map_->end());\n      break;\n    default:\n      break;\n    }\n    return const_iterator();\n  }\n\n  Value::iterator Value::begin() {\n    switch (type_) {\n    case arrayValue:\n    case objectValue:\n      if (value_.map_)\n        return iterator(value_.map_->begin());\n      break;\n    default:\n      break;\n    }\n    return iterator();\n  }\n\n  Value::iterator Value::end() {\n    switch (type_) {\n    case arrayValue:\n    case objectValue:\n      if (value_.map_)\n        return iterator(value_.map_->end());\n      break;\n    default:\n      break;\n    }\n    return iterator();\n  }\n\n  // class PathArgument\n  // //////////////////////////////////////////////////////////////////\n\n  PathArgument::PathArgument() : key_(), index_(), kind_(kindNone) {}\n\n  PathArgument::PathArgument(ArrayIndex index)\n      : key_(), index_(index), kind_(kindIndex) {}\n\n  PathArgument::PathArgument(const char* key)\n      : key_(key), index_(), kind_(kindKey) {}\n\n  PathArgument::PathArgument(const std::string& key)\n      : key_(key.c_str()), index_(), kind_(kindKey) {}\n\n  // class Path\n  // //////////////////////////////////////////////////////////////////\n\n  Path::Path(const std::string& path, const PathArgument& a1,\n             const PathArgument& a2, const PathArgument& a3,\n             const PathArgument& a4, const PathArgument& a5) {\n    InArgs in;\n    in.push_back(&a1);\n    in.push_back(&a2);\n    in.push_back(&a3);\n    in.push_back(&a4);\n    in.push_back(&a5);\n    makePath(path, in);\n  }\n\n  void Path::makePath(const std::string& path, const InArgs& in) {\n    const char* current = path.c_str();\n    const char* end = current + path.length();\n    InArgs::const_iterator itInArg = in.begin();\n    while (current != end) {\n      if (*current == '[') {\n        ++current;\n        if (*current == '%')\n          addPathInArg(path, in, itInArg, PathArgument::kindIndex);\n        else {\n          ArrayIndex index = 0;\n          for (; current != end && *current >= '0' && *current <= '9';\n               ++current)\n            index = index * 10 + ArrayIndex(*current - '0');\n          args_.push_back(index);\n        }\n        if (current == end || *current++ != ']')\n          invalidPath(path, int(current - path.c_str()));\n      } else if (*current == '%') {\n        addPathInArg(path, in, itInArg, PathArgument::kindKey);\n        ++current;\n      } else if (*current == '.') {\n        ++current;\n      } else {\n        const char* beginName = current;\n        while (current != end && !strchr(\"[.\", *current))\n          ++current;\n        args_.push_back(std::string(beginName, current));\n      }\n    }\n  }\n\n  void Path::addPathInArg(const std::string& /*path*/, const InArgs& in,\n                          InArgs::const_iterator& itInArg,\n                          PathArgument::Kind kind) {\n    if (itInArg == in.end()) {\n      // Error: missing argument %d\n    } else if ((*itInArg)->kind_ != kind) {\n      // Error: bad argument type\n    } else {\n      args_.push_back(**itInArg);\n    }\n  }\n\n  void Path::invalidPath(const std::string& /*path*/, int /*location*/) {\n    // Error: invalid path.\n  }\n\n  const Value& Path::resolve(const Value& root) const {\n    const Value* node = &root;\n    for (Args::const_iterator it = args_.begin(); it != args_.end(); ++it) {\n      const PathArgument& arg = *it;\n      if (arg.kind_ == PathArgument::kindIndex) {\n        if (!node->isArray() || !node->isValidIndex(arg.index_)) {\n          // Error: unable to resolve path (array value expected at position...\n        }\n        node = &((*node)[arg.index_]);\n      } else if (arg.kind_ == PathArgument::kindKey) {\n        if (!node->isObject()) {\n          // Error: unable to resolve path (object value expected at\n          // position...)\n        }\n        node = &((*node)[arg.key_]);\n        if (node == &Value::nullRef) {\n          // Error: unable to resolve path (object has no member named '' at\n          // position...)\n        }\n      }\n    }\n    return *node;\n  }\n\n  Value Path::resolve(const Value& root, const Value& defaultValue) const {\n    const Value* node = &root;\n    for (Args::const_iterator it = args_.begin(); it != args_.end(); ++it) {\n      const PathArgument& arg = *it;\n      if (arg.kind_ == PathArgument::kindIndex) {\n        if (!node->isArray() || !node->isValidIndex(arg.index_))\n          return defaultValue;\n        node = &((*node)[arg.index_]);\n      } else if (arg.kind_ == PathArgument::kindKey) {\n        if (!node->isObject())\n          return defaultValue;\n        node = &((*node)[arg.key_]);\n        if (node == &Value::nullRef)\n          return defaultValue;\n      }\n    }\n    return *node;\n  }\n\n  Value& Path::make(Value& root) const {\n    Value* node = &root;\n    for (Args::const_iterator it = args_.begin(); it != args_.end(); ++it) {\n      const PathArgument& arg = *it;\n      if (arg.kind_ == PathArgument::kindIndex) {\n        if (!node->isArray()) {\n          // Error: node is not an array at position ...\n        }\n        node = &((*node)[arg.index_]);\n      } else if (arg.kind_ == PathArgument::kindKey) {\n        if (!node->isObject()) {\n          // Error: node is not an object at position...\n        }\n        node = &((*node)[arg.key_]);\n      }\n    }\n    return *node;\n  }\n\n} // namespace Json\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: src/lib_json/json_value.cpp\n// //////////////////////////////////////////////////////////////////////\n\n// //////////////////////////////////////////////////////////////////////\n// Beginning of content of file: src/lib_json/json_writer.cpp\n// //////////////////////////////////////////////////////////////////////\n\n// Copyright 2011 Baptiste Lepilleur\n// Distributed under MIT license, or public domain if desired and\n// recognized in your jurisdiction.\n// See file LICENSE for detail or copy at http://jsoncpp.sourceforge.net/LICENSE\n\n#if !defined(JSON_IS_AMALGAMATION)\n#include \"json_tool.h\"\n#include <json/writer.h>\n#endif // if !defined(JSON_IS_AMALGAMATION)\n#include <cassert>\n#include <cstdio>\n#include <cstring>\n#include <iomanip>\n#include <memory>\n#include <set>\n#include <sstream>\n#include <utility>\n\n#if defined(_MSC_VER) && _MSC_VER >= 1200 &&                                   \\\n    _MSC_VER < 1800 // Between VC++ 6.0 and VC++ 11.0\n#include <float.h>\n#define isfinite _finite\n#elif defined(__sun) && defined(__SVR4) // Solaris\n#if !defined(isfinite)\n#include <ieeefp.h>\n#define isfinite finite\n#endif\n#elif defined(_AIX)\n#if !defined(isfinite)\n#include <math.h>\n#define isfinite finite\n#endif\n#elif defined(__hpux)\n#if !defined(isfinite)\n#if defined(__ia64) && !defined(finite)\n#define isfinite(x)                                                            \\\n  ((sizeof(x) == sizeof(float) ? _Isfinitef(x) : _IsFinite(x)))\n#else\n#include <math.h>\n#define isfinite finite\n#endif\n#endif\n#else\n#include <cmath>\n#if !(defined(__QNXNTO__)) // QNX already defines isfinite\n#define isfinite std::isfinite\n#endif\n#endif\n\n#if defined(_MSC_VER)\n#if !defined(WINCE) && defined(__STDC_SECURE_LIB__) &&                         \\\n    _MSC_VER >= 1500 // VC++ 9.0 and above\n#define snprintf sprintf_s\n#elif _MSC_VER >= 1900 // VC++ 14.0 and above\n#define snprintf std::snprintf\n#else\n#define snprintf _snprintf\n#endif\n#elif defined(__ANDROID__) || defined(__QNXNTO__)\n#define snprintf snprintf\n#elif __cplusplus >= 201103L\n#define snprintf std::snprintf\n#endif\n\n#if defined(__BORLANDC__)\n#include <float.h>\n#define isfinite _finite\n#define snprintf _snprintf\n#endif\n\n#if defined(_MSC_VER) && _MSC_VER >= 1400 // VC++ 8.0\n// Disable warning about strdup being deprecated.\n#pragma warning(disable : 4996)\n#endif\n\nnamespace Json {\n\n#if __cplusplus >= 201103L || (defined(_CPPLIB_VER) && _CPPLIB_VER >= 520)\n  typedef std::unique_ptr<StreamWriter> StreamWriterPtr;\n#else\n  typedef std::auto_ptr<StreamWriter> StreamWriterPtr;\n#endif\n\n  static bool containsControlCharacter(const char* str) {\n    while (*str) {\n      if (isControlCharacter(*(str++)))\n        return true;\n    }\n    return false;\n  }\n\n  static bool containsControlCharacter0(const char* str, unsigned len) {\n    char const* end = str + len;\n    while (end != str) {\n      if (isControlCharacter(*str) || 0 == *str)\n        return true;\n      ++str;\n    }\n    return false;\n  }\n\n  std::string valueToString(LargestInt value) {\n    UIntToStringBuffer buffer;\n    char* current = buffer + sizeof(buffer);\n    if (value == Value::minLargestInt) {\n      uintToString(LargestUInt(Value::maxLargestInt) + 1, current);\n      *--current = '-';\n    } else if (value < 0) {\n      uintToString(LargestUInt(-value), current);\n      *--current = '-';\n    } else {\n      uintToString(LargestUInt(value), current);\n    }\n    assert(current >= buffer);\n    return current;\n  }\n\n  std::string valueToString(LargestUInt value) {\n    UIntToStringBuffer buffer;\n    char* current = buffer + sizeof(buffer);\n    uintToString(value, current);\n    assert(current >= buffer);\n    return current;\n  }\n\n#if defined(JSON_HAS_INT64)\n\n  std::string valueToString(Int value) {\n    return valueToString(LargestInt(value));\n  }\n\n  std::string valueToString(UInt value) {\n    return valueToString(LargestUInt(value));\n  }\n\n#endif // # if defined(JSON_HAS_INT64)\n\n  std::string valueToString(double value, bool useSpecialFloats,\n                            unsigned int precision) {\n    // Allocate a buffer that is more than large enough to store the 16 digits\n    // of precision requested below.\n    char buffer[32];\n    int len = -1;\n\n    char formatString[6];\n    sprintf(formatString, \"%%.%dg\", precision);\n\n    // Print into the buffer. We need not request the alternative representation\n    // that always has a decimal point because JSON doesn't distingish the\n    // concepts of reals and integers.\n    if (isfinite(value)) {\n      len = snprintf(buffer, sizeof(buffer), formatString, value);\n    } else {\n      // IEEE standard states that NaN values will not compare to themselves\n      if (value != value) {\n        len =\n            snprintf(buffer, sizeof(buffer), useSpecialFloats ? \"NaN\" : \"null\");\n      } else if (value < 0) {\n        len = snprintf(buffer, sizeof(buffer),\n                       useSpecialFloats ? \"-Infinity\" : \"-1e+9999\");\n      } else {\n        len = snprintf(buffer, sizeof(buffer),\n                       useSpecialFloats ? \"Infinity\" : \"1e+9999\");\n      }\n      // For those, we do not need to call fixNumLoc, but it is fast.\n    }\n    assert(len >= 0);\n    fixNumericLocale(buffer, buffer + len);\n    return buffer;\n  }\n\n  std::string valueToString(double value) {\n    return valueToString(value, false, 17);\n  }\n\n  std::string valueToString(bool value) { return value ? \"true\" : \"false\"; }\n\n  std::string valueToQuotedString(const char* value) {\n    if (value == NULL)\n      return \"\";\n    // Not sure how to handle unicode...\n    if (strpbrk(value, \"\\\"\\\\\\b\\f\\n\\r\\t\") == NULL &&\n        !containsControlCharacter(value))\n      return std::string(\"\\\"\") + value + \"\\\"\";\n    // We have to walk value and escape any special characters.\n    // Appending to std::string is not efficient, but this should be rare.\n    // (Note: forward slashes are *not* rare, but I am not escaping them.)\n    std::string::size_type maxsize =\n        strlen(value) * 2 + 3; // allescaped+quotes+NULL\n    std::string result;\n    result.reserve(maxsize); // to avoid lots of mallocs\n    result += \"\\\"\";\n    for (const char* c = value; *c != 0; ++c) {\n      switch (*c) {\n      case '\\\"':\n        result += \"\\\\\\\"\";\n        break;\n      case '\\\\':\n        result += \"\\\\\\\\\";\n        break;\n      case '\\b':\n        result += \"\\\\b\";\n        break;\n      case '\\f':\n        result += \"\\\\f\";\n        break;\n      case '\\n':\n        result += \"\\\\n\";\n        break;\n      case '\\r':\n        result += \"\\\\r\";\n        break;\n      case '\\t':\n        result += \"\\\\t\";\n        break;\n      // case '/':\n      // Even though \\/ is considered a legal escape in JSON, a bare\n      // slash is also legal, so I see no reason to escape it.\n      // (I hope I am not misunderstanding something.\n      // blep notes: actually escaping \\/ may be useful in javascript to avoid\n      // </ sequence. Should add a flag to allow this compatibility mode and\n      // prevent this sequence from occurring.\n      default:\n        if (isControlCharacter(*c)) {\n          std::ostringstream oss;\n          oss << \"\\\\u\" << std::hex << std::uppercase << std::setfill('0')\n              << std::setw(4) << static_cast<int>(*c);\n          result += oss.str();\n        } else {\n          result += *c;\n        }\n        break;\n      }\n    }\n    result += \"\\\"\";\n    return result;\n  }\n\n  // https://github.com/upcaste/upcaste/blob/master/src/upcore/src/cstring/strnpbrk.cpp\n  static char const* strnpbrk(char const* s, char const* accept, size_t n) {\n    assert((s || !n) && accept);\n\n    char const* const end = s + n;\n    for (char const* cur = s; cur < end; ++cur) {\n      int const c = *cur;\n      for (char const* a = accept; *a; ++a) {\n        if (*a == c) {\n          return cur;\n        }\n      }\n    }\n    return NULL;\n  }\n  static std::string valueToQuotedStringN(const char* value, unsigned length) {\n    if (value == NULL)\n      return \"\";\n    // Not sure how to handle unicode...\n    if (strnpbrk(value, \"\\\"\\\\\\b\\f\\n\\r\\t\", length) == NULL &&\n        !containsControlCharacter0(value, length))\n      return std::string(\"\\\"\") + value + \"\\\"\";\n    // We have to walk value and escape any special characters.\n    // Appending to std::string is not efficient, but this should be rare.\n    // (Note: forward slashes are *not* rare, but I am not escaping them.)\n    std::string::size_type maxsize = length * 2 + 3; // allescaped+quotes+NULL\n    std::string result;\n    result.reserve(maxsize); // to avoid lots of mallocs\n    result += \"\\\"\";\n    char const* end = value + length;\n    for (const char* c = value; c != end; ++c) {\n      switch (*c) {\n      case '\\\"':\n        result += \"\\\\\\\"\";\n        break;\n      case '\\\\':\n        result += \"\\\\\\\\\";\n        break;\n      case '\\b':\n        result += \"\\\\b\";\n        break;\n      case '\\f':\n        result += \"\\\\f\";\n        break;\n      case '\\n':\n        result += \"\\\\n\";\n        break;\n      case '\\r':\n        result += \"\\\\r\";\n        break;\n      case '\\t':\n        result += \"\\\\t\";\n        break;\n      // case '/':\n      // Even though \\/ is considered a legal escape in JSON, a bare\n      // slash is also legal, so I see no reason to escape it.\n      // (I hope I am not misunderstanding something.)\n      // blep notes: actually escaping \\/ may be useful in javascript to avoid\n      // </ sequence. Should add a flag to allow this compatibility mode and\n      // prevent this sequence from occurring.\n      default:\n        if ((isControlCharacter(*c)) || (*c == 0)) {\n          std::ostringstream oss;\n          oss << \"\\\\u\" << std::hex << std::uppercase << std::setfill('0')\n              << std::setw(4) << static_cast<int>(*c);\n          result += oss.str();\n        } else {\n          result += *c;\n        }\n        break;\n      }\n    }\n    result += \"\\\"\";\n    return result;\n  }\n\n  // Class Writer\n  // //////////////////////////////////////////////////////////////////\n  Writer::~Writer() {}\n\n  // Class FastWriter\n  // //////////////////////////////////////////////////////////////////\n\n  FastWriter::FastWriter()\n      : yamlCompatiblityEnabled_(false), dropNullPlaceholders_(false),\n        omitEndingLineFeed_(false) {}\n\n  void FastWriter::enableYAMLCompatibility() {\n    yamlCompatiblityEnabled_ = true;\n  }\n\n  void FastWriter::dropNullPlaceholders() { dropNullPlaceholders_ = true; }\n\n  void FastWriter::omitEndingLineFeed() { omitEndingLineFeed_ = true; }\n\n  std::string FastWriter::write(const Value& root) {\n    document_ = \"\";\n    writeValue(root);\n    if (!omitEndingLineFeed_)\n      document_ += \"\\n\";\n    return document_;\n  }\n\n  void FastWriter::writeValue(const Value& value) {\n    switch (value.type()) {\n    case nullValue:\n      if (!dropNullPlaceholders_)\n        document_ += \"null\";\n      break;\n    case intValue:\n      document_ += valueToString(value.asLargestInt());\n      break;\n    case uintValue:\n      document_ += valueToString(value.asLargestUInt());\n      break;\n    case realValue:\n      document_ += valueToString(value.asDouble());\n      break;\n    case stringValue: {\n      // Is NULL possible for value.string_?\n      char const* str;\n      char const* end;\n      bool ok = value.getString(&str, &end);\n      if (ok)\n        document_ +=\n            valueToQuotedStringN(str, static_cast<unsigned>(end - str));\n      break;\n    }\n    case booleanValue:\n      document_ += valueToString(value.asBool());\n      break;\n    case arrayValue: {\n      document_ += '[';\n      int size = value.size();\n      for (int index = 0; index < size; ++index) {\n        if (index > 0)\n          document_ += ',';\n        writeValue(value[index]);\n      }\n      document_ += ']';\n    } break;\n    case objectValue: {\n      Value::Members members(value.getMemberNames());\n      document_ += '{';\n      for (Value::Members::iterator it = members.begin(); it != members.end();\n           ++it) {\n        const std::string& name = *it;\n        if (it != members.begin())\n          document_ += ',';\n        document_ += valueToQuotedStringN(name.data(),\n                                          static_cast<unsigned>(name.length()));\n        document_ += yamlCompatiblityEnabled_ ? \": \" : \":\";\n        writeValue(value[name]);\n      }\n      document_ += '}';\n    } break;\n    }\n  }\n\n  // Class StyledWriter\n  // //////////////////////////////////////////////////////////////////\n\n  StyledWriter::StyledWriter()\n      : rightMargin_(74), indentSize_(3), addChildValues_() {}\n\n  std::string StyledWriter::write(const Value& root) {\n    document_ = \"\";\n    addChildValues_ = false;\n    indentString_ = \"\";\n    writeCommentBeforeValue(root);\n    writeValue(root);\n    writeCommentAfterValueOnSameLine(root);\n    document_ += \"\\n\";\n    return document_;\n  }\n\n  void StyledWriter::writeValue(const Value& value) {\n    switch (value.type()) {\n    case nullValue:\n      pushValue(\"null\");\n      break;\n    case intValue:\n      pushValue(valueToString(value.asLargestInt()));\n      break;\n    case uintValue:\n      pushValue(valueToString(value.asLargestUInt()));\n      break;\n    case realValue:\n      pushValue(valueToString(value.asDouble()));\n      break;\n    case stringValue: {\n      // Is NULL possible for value.string_?\n      char const* str;\n      char const* end;\n      bool ok = value.getString(&str, &end);\n      if (ok)\n        pushValue(valueToQuotedStringN(str, static_cast<unsigned>(end - str)));\n      else\n        pushValue(\"\");\n      break;\n    }\n    case booleanValue:\n      pushValue(valueToString(value.asBool()));\n      break;\n    case arrayValue:\n      writeArrayValue(value);\n      break;\n    case objectValue: {\n      Value::Members members(value.getMemberNames());\n      if (members.empty())\n        pushValue(\"{}\");\n      else {\n        writeWithIndent(\"{\");\n        indent();\n        Value::Members::iterator it = members.begin();\n        for (;;) {\n          const std::string& name = *it;\n          const Value& childValue = value[name];\n          writeCommentBeforeValue(childValue);\n          writeWithIndent(valueToQuotedString(name.c_str()));\n          document_ += \" : \";\n          writeValue(childValue);\n          if (++it == members.end()) {\n            writeCommentAfterValueOnSameLine(childValue);\n            break;\n          }\n          document_ += ',';\n          writeCommentAfterValueOnSameLine(childValue);\n        }\n        unindent();\n        writeWithIndent(\"}\");\n      }\n    } break;\n    }\n  }\n\n  void StyledWriter::writeArrayValue(const Value& value) {\n    unsigned size = value.size();\n    if (size == 0)\n      pushValue(\"[]\");\n    else {\n      bool isArrayMultiLine = isMultineArray(value);\n      if (isArrayMultiLine) {\n        writeWithIndent(\"[\");\n        indent();\n        bool hasChildValue = !childValues_.empty();\n        unsigned index = 0;\n        for (;;) {\n          const Value& childValue = value[index];\n          writeCommentBeforeValue(childValue);\n          if (hasChildValue)\n            writeWithIndent(childValues_[index]);\n          else {\n            writeIndent();\n            writeValue(childValue);\n          }\n          if (++index == size) {\n            writeCommentAfterValueOnSameLine(childValue);\n            break;\n          }\n          document_ += ',';\n          writeCommentAfterValueOnSameLine(childValue);\n        }\n        unindent();\n        writeWithIndent(\"]\");\n      } else // output on a single line\n      {\n        assert(childValues_.size() == size);\n        document_ += \"[ \";\n        for (unsigned index = 0; index < size; ++index) {\n          if (index > 0)\n            document_ += \", \";\n          document_ += childValues_[index];\n        }\n        document_ += \" ]\";\n      }\n    }\n  }\n\n  bool StyledWriter::isMultineArray(const Value& value) {\n    int size = value.size();\n    bool isMultiLine = size * 3 >= rightMargin_;\n    childValues_.clear();\n    for (int index = 0; index < size && !isMultiLine; ++index) {\n      const Value& childValue = value[index];\n      isMultiLine = ((childValue.isArray() || childValue.isObject()) &&\n                     childValue.size() > 0);\n    }\n    if (!isMultiLine) // check if line length > max line length\n    {\n      childValues_.reserve(size);\n      addChildValues_ = true;\n      int lineLength = 4 + (size - 1) * 2; // '[ ' + ', '*n + ' ]'\n      for (int index = 0; index < size; ++index) {\n        if (hasCommentForValue(value[index])) {\n          isMultiLine = true;\n        }\n        writeValue(value[index]);\n        lineLength += int(childValues_[index].length());\n      }\n      addChildValues_ = false;\n      isMultiLine = isMultiLine || lineLength >= rightMargin_;\n    }\n    return isMultiLine;\n  }\n\n  void StyledWriter::pushValue(const std::string& value) {\n    if (addChildValues_)\n      childValues_.push_back(value);\n    else\n      document_ += value;\n  }\n\n  void StyledWriter::writeIndent() {\n    if (!document_.empty()) {\n      char last = document_[document_.length() - 1];\n      if (last == ' ') // already indented\n        return;\n      if (last != '\\n') // Comments may add new-line\n        document_ += '\\n';\n    }\n    document_ += indentString_;\n  }\n\n  void StyledWriter::writeWithIndent(const std::string& value) {\n    writeIndent();\n    document_ += value;\n  }\n\n  void StyledWriter::indent() {\n    indentString_ += std::string(indentSize_, ' ');\n  }\n\n  void StyledWriter::unindent() {\n    assert(int(indentString_.size()) >= indentSize_);\n    indentString_.resize(indentString_.size() - indentSize_);\n  }\n\n  void StyledWriter::writeCommentBeforeValue(const Value& root) {\n    if (!root.hasComment(commentBefore))\n      return;\n\n    document_ += \"\\n\";\n    writeIndent();\n    const std::string& comment = root.getComment(commentBefore);\n    std::string::const_iterator iter = comment.begin();\n    while (iter != comment.end()) {\n      document_ += *iter;\n      if (*iter == '\\n' && (iter != comment.end() && *(iter + 1) == '/'))\n        writeIndent();\n      ++iter;\n    }\n\n    // Comments are stripped of trailing newlines, so add one here\n    document_ += \"\\n\";\n  }\n\n  void StyledWriter::writeCommentAfterValueOnSameLine(const Value& root) {\n    if (root.hasComment(commentAfterOnSameLine))\n      document_ += \" \" + root.getComment(commentAfterOnSameLine);\n\n    if (root.hasComment(commentAfter)) {\n      document_ += \"\\n\";\n      document_ += root.getComment(commentAfter);\n      document_ += \"\\n\";\n    }\n  }\n\n  bool StyledWriter::hasCommentForValue(const Value& value) {\n    return value.hasComment(commentBefore) ||\n           value.hasComment(commentAfterOnSameLine) ||\n           value.hasComment(commentAfter);\n  }\n\n  // Class StyledStreamWriter\n  // //////////////////////////////////////////////////////////////////\n\n  StyledStreamWriter::StyledStreamWriter(std::string indentation)\n      : document_(NULL), rightMargin_(74), indentation_(indentation),\n        addChildValues_(), indented_() {}\n\n  void StyledStreamWriter::write(std::ostream& out, const Value& root) {\n    document_ = &out;\n    addChildValues_ = false;\n    indentString_ = \"\";\n    indented_ = true;\n    writeCommentBeforeValue(root);\n    if (!indented_)\n      writeIndent();\n    indented_ = true;\n    writeValue(root);\n    writeCommentAfterValueOnSameLine(root);\n    *document_ << \"\\n\";\n    document_ = NULL; // Forget the stream, for safety.\n  }\n\n  void StyledStreamWriter::writeValue(const Value& value) {\n    switch (value.type()) {\n    case nullValue:\n      pushValue(\"null\");\n      break;\n    case intValue:\n      pushValue(valueToString(value.asLargestInt()));\n      break;\n    case uintValue:\n      pushValue(valueToString(value.asLargestUInt()));\n      break;\n    case realValue:\n      pushValue(valueToString(value.asDouble()));\n      break;\n    case stringValue: {\n      // Is NULL possible for value.string_?\n      char const* str;\n      char const* end;\n      bool ok = value.getString(&str, &end);\n      if (ok)\n        pushValue(valueToQuotedStringN(str, static_cast<unsigned>(end - str)));\n      else\n        pushValue(\"\");\n      break;\n    }\n    case booleanValue:\n      pushValue(valueToString(value.asBool()));\n      break;\n    case arrayValue:\n      writeArrayValue(value);\n      break;\n    case objectValue: {\n      Value::Members members(value.getMemberNames());\n      if (members.empty())\n        pushValue(\"{}\");\n      else {\n        writeWithIndent(\"{\");\n        indent();\n        Value::Members::iterator it = members.begin();\n        for (;;) {\n          const std::string& name = *it;\n          const Value& childValue = value[name];\n          writeCommentBeforeValue(childValue);\n          writeWithIndent(valueToQuotedString(name.c_str()));\n          *document_ << \" : \";\n          writeValue(childValue);\n          if (++it == members.end()) {\n            writeCommentAfterValueOnSameLine(childValue);\n            break;\n          }\n          *document_ << \",\";\n          writeCommentAfterValueOnSameLine(childValue);\n        }\n        unindent();\n        writeWithIndent(\"}\");\n      }\n    } break;\n    }\n  }\n\n  void StyledStreamWriter::writeArrayValue(const Value& value) {\n    unsigned size = value.size();\n    if (size == 0)\n      pushValue(\"[]\");\n    else {\n      bool isArrayMultiLine = isMultineArray(value);\n      if (isArrayMultiLine) {\n        writeWithIndent(\"[\");\n        indent();\n        bool hasChildValue = !childValues_.empty();\n        unsigned index = 0;\n        for (;;) {\n          const Value& childValue = value[index];\n          writeCommentBeforeValue(childValue);\n          if (hasChildValue)\n            writeWithIndent(childValues_[index]);\n          else {\n            if (!indented_)\n              writeIndent();\n            indented_ = true;\n            writeValue(childValue);\n            indented_ = false;\n          }\n          if (++index == size) {\n            writeCommentAfterValueOnSameLine(childValue);\n            break;\n          }\n          *document_ << \",\";\n          writeCommentAfterValueOnSameLine(childValue);\n        }\n        unindent();\n        writeWithIndent(\"]\");\n      } else // output on a single line\n      {\n        assert(childValues_.size() == size);\n        *document_ << \"[ \";\n        for (unsigned index = 0; index < size; ++index) {\n          if (index > 0)\n            *document_ << \", \";\n          *document_ << childValues_[index];\n        }\n        *document_ << \" ]\";\n      }\n    }\n  }\n\n  bool StyledStreamWriter::isMultineArray(const Value& value) {\n    int size = value.size();\n    bool isMultiLine = size * 3 >= rightMargin_;\n    childValues_.clear();\n    for (int index = 0; index < size && !isMultiLine; ++index) {\n      const Value& childValue = value[index];\n      isMultiLine = ((childValue.isArray() || childValue.isObject()) &&\n                     childValue.size() > 0);\n    }\n    if (!isMultiLine) // check if line length > max line length\n    {\n      childValues_.reserve(size);\n      addChildValues_ = true;\n      int lineLength = 4 + (size - 1) * 2; // '[ ' + ', '*n + ' ]'\n      for (int index = 0; index < size; ++index) {\n        if (hasCommentForValue(value[index])) {\n          isMultiLine = true;\n        }\n        writeValue(value[index]);\n        lineLength += int(childValues_[index].length());\n      }\n      addChildValues_ = false;\n      isMultiLine = isMultiLine || lineLength >= rightMargin_;\n    }\n    return isMultiLine;\n  }\n\n  void StyledStreamWriter::pushValue(const std::string& value) {\n    if (addChildValues_)\n      childValues_.push_back(value);\n    else\n      *document_ << value;\n  }\n\n  void StyledStreamWriter::writeIndent() {\n    // blep intended this to look at the so-far-written string\n    // to determine whether we are already indented, but\n    // with a stream we cannot do that. So we rely on some saved state.\n    // The caller checks indented_.\n    *document_ << '\\n' << indentString_;\n  }\n\n  void StyledStreamWriter::writeWithIndent(const std::string& value) {\n    if (!indented_)\n      writeIndent();\n    *document_ << value;\n    indented_ = false;\n  }\n\n  void StyledStreamWriter::indent() { indentString_ += indentation_; }\n\n  void StyledStreamWriter::unindent() {\n    assert(indentString_.size() >= indentation_.size());\n    indentString_.resize(indentString_.size() - indentation_.size());\n  }\n\n  void StyledStreamWriter::writeCommentBeforeValue(const Value& root) {\n    if (!root.hasComment(commentBefore))\n      return;\n\n    if (!indented_)\n      writeIndent();\n    const std::string& comment = root.getComment(commentBefore);\n    std::string::const_iterator iter = comment.begin();\n    while (iter != comment.end()) {\n      *document_ << *iter;\n      if (*iter == '\\n' && (iter != comment.end() && *(iter + 1) == '/'))\n        // writeIndent();  // would include newline\n        *document_ << indentString_;\n      ++iter;\n    }\n    indented_ = false;\n  }\n\n  void StyledStreamWriter::writeCommentAfterValueOnSameLine(const Value& root) {\n    if (root.hasComment(commentAfterOnSameLine))\n      *document_ << ' ' << root.getComment(commentAfterOnSameLine);\n\n    if (root.hasComment(commentAfter)) {\n      writeIndent();\n      *document_ << root.getComment(commentAfter);\n    }\n    indented_ = false;\n  }\n\n  bool StyledStreamWriter::hasCommentForValue(const Value& value) {\n    return value.hasComment(commentBefore) ||\n           value.hasComment(commentAfterOnSameLine) ||\n           value.hasComment(commentAfter);\n  }\n\n  //////////////////////////\n  // BuiltStyledStreamWriter\n\n  /// Scoped enums are not available until C++11.\n  struct CommentStyle {\n    /// Decide whether to write comments.\n    enum Enum {\n      None, ///< Drop all comments.\n      Most, ///< Recover odd behavior of previous versions (not implemented\n            ///< yet).\n      All   ///< Keep all comments.\n    };\n  };\n\n  struct BuiltStyledStreamWriter : public StreamWriter {\n    BuiltStyledStreamWriter(std::string const& indentation,\n                            CommentStyle::Enum cs,\n                            std::string const& colonSymbol,\n                            std::string const& nullSymbol,\n                            std::string const& endingLineFeedSymbol,\n                            bool useSpecialFloats, unsigned int precision);\n    int write(Value const& root, std::ostream* sout) override;\n\n  private:\n    void writeValue(Value const& value);\n    void writeArrayValue(Value const& value);\n    bool isMultineArray(Value const& value);\n    void pushValue(std::string const& value);\n    void writeIndent();\n    void writeWithIndent(std::string const& value);\n    void indent();\n    void unindent();\n    void writeCommentBeforeValue(Value const& root);\n    void writeCommentAfterValueOnSameLine(Value const& root);\n    static bool hasCommentForValue(const Value& value);\n\n    typedef std::vector<std::string> ChildValues;\n\n    ChildValues childValues_;\n    std::string indentString_;\n    int rightMargin_;\n    std::string indentation_;\n    CommentStyle::Enum cs_;\n    std::string colonSymbol_;\n    std::string nullSymbol_;\n    std::string endingLineFeedSymbol_;\n    bool addChildValues_ : 1;\n    bool indented_ : 1;\n    bool useSpecialFloats_ : 1;\n    unsigned int precision_;\n  };\n  BuiltStyledStreamWriter::BuiltStyledStreamWriter(\n      std::string const& indentation, CommentStyle::Enum cs,\n      std::string const& colonSymbol, std::string const& nullSymbol,\n      std::string const& endingLineFeedSymbol, bool useSpecialFloats,\n      unsigned int precision)\n      : rightMargin_(74), indentation_(indentation), cs_(cs),\n        colonSymbol_(colonSymbol), nullSymbol_(nullSymbol),\n        endingLineFeedSymbol_(endingLineFeedSymbol), addChildValues_(false),\n        indented_(false), useSpecialFloats_(useSpecialFloats),\n        precision_(precision) {}\n  int BuiltStyledStreamWriter::write(Value const& root, std::ostream* sout) {\n    sout_ = sout;\n    addChildValues_ = false;\n    indented_ = true;\n    indentString_ = \"\";\n    writeCommentBeforeValue(root);\n    if (!indented_)\n      writeIndent();\n    indented_ = true;\n    writeValue(root);\n    writeCommentAfterValueOnSameLine(root);\n    *sout_ << endingLineFeedSymbol_;\n    sout_ = NULL;\n    return 0;\n  }\n  void BuiltStyledStreamWriter::writeValue(Value const& value) {\n    switch (value.type()) {\n    case nullValue:\n      pushValue(nullSymbol_);\n      break;\n    case intValue:\n      pushValue(valueToString(value.asLargestInt()));\n      break;\n    case uintValue:\n      pushValue(valueToString(value.asLargestUInt()));\n      break;\n    case realValue:\n      pushValue(valueToString(value.asDouble(), useSpecialFloats_, precision_));\n      break;\n    case stringValue: {\n      // Is NULL is possible for value.string_?\n      char const* str;\n      char const* end;\n      bool ok = value.getString(&str, &end);\n      if (ok)\n        pushValue(valueToQuotedStringN(str, static_cast<unsigned>(end - str)));\n      else\n        pushValue(\"\");\n      break;\n    }\n    case booleanValue:\n      pushValue(valueToString(value.asBool()));\n      break;\n    case arrayValue:\n      writeArrayValue(value);\n      break;\n    case objectValue: {\n      Value::Members members(value.getMemberNames());\n      if (members.empty())\n        pushValue(\"{}\");\n      else {\n        writeWithIndent(\"{\");\n        indent();\n        Value::Members::iterator it = members.begin();\n        for (;;) {\n          std::string const& name = *it;\n          Value const& childValue = value[name];\n          writeCommentBeforeValue(childValue);\n          writeWithIndent(valueToQuotedStringN(\n              name.data(), static_cast<unsigned>(name.length())));\n          *sout_ << colonSymbol_;\n          writeValue(childValue);\n          if (++it == members.end()) {\n            writeCommentAfterValueOnSameLine(childValue);\n            break;\n          }\n          *sout_ << \",\";\n          writeCommentAfterValueOnSameLine(childValue);\n        }\n        unindent();\n        writeWithIndent(\"}\");\n      }\n    } break;\n    }\n  }\n\n  void BuiltStyledStreamWriter::writeArrayValue(Value const& value) {\n    unsigned size = value.size();\n    if (size == 0)\n      pushValue(\"[]\");\n    else {\n      bool isMultiLine = (cs_ == CommentStyle::All) || isMultineArray(value);\n      if (isMultiLine) {\n        writeWithIndent(\"[\");\n        indent();\n        bool hasChildValue = !childValues_.empty();\n        unsigned index = 0;\n        for (;;) {\n          Value const& childValue = value[index];\n          writeCommentBeforeValue(childValue);\n          if (hasChildValue)\n            writeWithIndent(childValues_[index]);\n          else {\n            if (!indented_)\n              writeIndent();\n            indented_ = true;\n            writeValue(childValue);\n            indented_ = false;\n          }\n          if (++index == size) {\n            writeCommentAfterValueOnSameLine(childValue);\n            break;\n          }\n          *sout_ << \",\";\n          writeCommentAfterValueOnSameLine(childValue);\n        }\n        unindent();\n        writeWithIndent(\"]\");\n      } else // output on a single line\n      {\n        assert(childValues_.size() == size);\n        *sout_ << \"[\";\n        if (!indentation_.empty())\n          *sout_ << \" \";\n        for (unsigned index = 0; index < size; ++index) {\n          if (index > 0)\n            *sout_ << \", \";\n          *sout_ << childValues_[index];\n        }\n        if (!indentation_.empty())\n          *sout_ << \" \";\n        *sout_ << \"]\";\n      }\n    }\n  }\n\n  bool BuiltStyledStreamWriter::isMultineArray(Value const& value) {\n    int size = value.size();\n    bool isMultiLine = size * 3 >= rightMargin_;\n    childValues_.clear();\n    for (int index = 0; index < size && !isMultiLine; ++index) {\n      Value const& childValue = value[index];\n      isMultiLine = ((childValue.isArray() || childValue.isObject()) &&\n                     childValue.size() > 0);\n    }\n    if (!isMultiLine) // check if line length > max line length\n    {\n      childValues_.reserve(size);\n      addChildValues_ = true;\n      int lineLength = 4 + (size - 1) * 2; // '[ ' + ', '*n + ' ]'\n      for (int index = 0; index < size; ++index) {\n        if (hasCommentForValue(value[index])) {\n          isMultiLine = true;\n        }\n        writeValue(value[index]);\n        lineLength += int(childValues_[index].length());\n      }\n      addChildValues_ = false;\n      isMultiLine = isMultiLine || lineLength >= rightMargin_;\n    }\n    return isMultiLine;\n  }\n\n  void BuiltStyledStreamWriter::pushValue(std::string const& value) {\n    if (addChildValues_)\n      childValues_.push_back(value);\n    else\n      *sout_ << value;\n  }\n\n  void BuiltStyledStreamWriter::writeIndent() {\n    // blep intended this to look at the so-far-written string\n    // to determine whether we are already indented, but\n    // with a stream we cannot do that. So we rely on some saved state.\n    // The caller checks indented_.\n\n    if (!indentation_.empty()) {\n      // In this case, drop newlines too.\n      *sout_ << '\\n' << indentString_;\n    }\n  }\n\n  void BuiltStyledStreamWriter::writeWithIndent(std::string const& value) {\n    if (!indented_)\n      writeIndent();\n    *sout_ << value;\n    indented_ = false;\n  }\n\n  void BuiltStyledStreamWriter::indent() { indentString_ += indentation_; }\n\n  void BuiltStyledStreamWriter::unindent() {\n    assert(indentString_.size() >= indentation_.size());\n    indentString_.resize(indentString_.size() - indentation_.size());\n  }\n\n  void BuiltStyledStreamWriter::writeCommentBeforeValue(Value const& root) {\n    if (cs_ == CommentStyle::None)\n      return;\n    if (!root.hasComment(commentBefore))\n      return;\n\n    if (!indented_)\n      writeIndent();\n    const std::string& comment = root.getComment(commentBefore);\n    std::string::const_iterator iter = comment.begin();\n    while (iter != comment.end()) {\n      *sout_ << *iter;\n      if (*iter == '\\n' && (iter != comment.end() && *(iter + 1) == '/'))\n        // writeIndent();  // would write extra newline\n        *sout_ << indentString_;\n      ++iter;\n    }\n    indented_ = false;\n  }\n\n  void\n  BuiltStyledStreamWriter::writeCommentAfterValueOnSameLine(Value const& root) {\n    if (cs_ == CommentStyle::None)\n      return;\n    if (root.hasComment(commentAfterOnSameLine))\n      *sout_ << \" \" + root.getComment(commentAfterOnSameLine);\n\n    if (root.hasComment(commentAfter)) {\n      writeIndent();\n      *sout_ << root.getComment(commentAfter);\n    }\n  }\n\n  // static\n  bool BuiltStyledStreamWriter::hasCommentForValue(const Value& value) {\n    return value.hasComment(commentBefore) ||\n           value.hasComment(commentAfterOnSameLine) ||\n           value.hasComment(commentAfter);\n  }\n\n  ///////////////\n  // StreamWriter\n\n  StreamWriter::StreamWriter() : sout_(NULL) {}\n  StreamWriter::~StreamWriter() {}\n  StreamWriter::Factory::~Factory() {}\n  StreamWriterBuilder::StreamWriterBuilder() { setDefaults(&settings_); }\n  StreamWriterBuilder::~StreamWriterBuilder() {}\n  StreamWriter* StreamWriterBuilder::newStreamWriter() const {\n    std::string indentation = settings_[\"indentation\"].asString();\n    std::string cs_str = settings_[\"commentStyle\"].asString();\n    bool eyc = settings_[\"enableYAMLCompatibility\"].asBool();\n    bool dnp = settings_[\"dropNullPlaceholders\"].asBool();\n    bool usf = settings_[\"useSpecialFloats\"].asBool();\n    unsigned int pre = settings_[\"precision\"].asUInt();\n    CommentStyle::Enum cs = CommentStyle::All;\n    if (cs_str == \"All\") {\n      cs = CommentStyle::All;\n    } else if (cs_str == \"None\") {\n      cs = CommentStyle::None;\n    } else {\n      throwRuntimeError(\"commentStyle must be 'All' or 'None'\");\n    }\n    std::string colonSymbol = \" : \";\n    if (eyc) {\n      colonSymbol = \": \";\n    } else if (indentation.empty()) {\n      colonSymbol = \":\";\n    }\n    std::string nullSymbol = \"null\";\n    if (dnp) {\n      nullSymbol = \"\";\n    }\n    if (pre > 17)\n      pre = 17;\n    std::string endingLineFeedSymbol = \"\";\n    return new BuiltStyledStreamWriter(indentation, cs, colonSymbol, nullSymbol,\n                                       endingLineFeedSymbol, usf, pre);\n  }\n  static void getValidWriterKeys(std::set<std::string>* valid_keys) {\n    valid_keys->clear();\n    valid_keys->insert(\"indentation\");\n    valid_keys->insert(\"commentStyle\");\n    valid_keys->insert(\"enableYAMLCompatibility\");\n    valid_keys->insert(\"dropNullPlaceholders\");\n    valid_keys->insert(\"useSpecialFloats\");\n    valid_keys->insert(\"precision\");\n  }\n  bool StreamWriterBuilder::validate(Json::Value* invalid) const {\n    Json::Value my_invalid;\n    if (!invalid)\n      invalid = &my_invalid; // so we do not need to test for NULL\n    Json::Value& inv = *invalid;\n    std::set<std::string> valid_keys;\n    getValidWriterKeys(&valid_keys);\n    Value::Members keys = settings_.getMemberNames();\n    size_t n = keys.size();\n    for (size_t i = 0; i < n; ++i) {\n      std::string const& key = keys[i];\n      if (valid_keys.find(key) == valid_keys.end()) {\n        inv[key] = settings_[key];\n      }\n    }\n    return 0u == inv.size();\n  }\n  Value& StreamWriterBuilder::operator[](std::string key) {\n    return settings_[key];\n  }\n  // static\n  void StreamWriterBuilder::setDefaults(Json::Value* settings) {\n    //! [StreamWriterBuilderDefaults]\n    (*settings)[\"commentStyle\"] = \"All\";\n    (*settings)[\"indentation\"] = \"\\t\";\n    (*settings)[\"enableYAMLCompatibility\"] = false;\n    (*settings)[\"dropNullPlaceholders\"] = false;\n    (*settings)[\"useSpecialFloats\"] = false;\n    (*settings)[\"precision\"] = 17;\n    //! [StreamWriterBuilderDefaults]\n  }\n\n  std::string writeString(StreamWriter::Factory const& builder,\n                          Value const& root) {\n    std::ostringstream sout;\n    StreamWriterPtr const writer(builder.newStreamWriter());\n    writer->write(root, &sout);\n    return sout.str();\n  }\n\n  std::ostream& operator<<(std::ostream& sout, Value const& root) {\n    StreamWriterBuilder builder;\n    StreamWriterPtr const writer(builder.newStreamWriter());\n    writer->write(root, &sout);\n    return sout;\n  }\n\n} // namespace Json\n\n// //////////////////////////////////////////////////////////////////////\n// End of content of file: src/lib_json/json_writer.cpp\n// //////////////////////////////////////////////////////////////////////\n"
  },
  {
    "path": "cpp-api/src/routing_table.cpp",
    "content": "// Copyright 2015-2017 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#include \"routing_table.hpp\"\n\nnamespace scalaris {\n\n  RoutingTable::RoutingTable(Connection& _c) : c(_c) {}\n\n  int RoutingTable::get_replication_factor() {\n    Json::Value result = c.rpc(\"get_replication_factor\");\n\n    if (!result.isObject()) {\n      std::stringstream error;\n      error << result.toStyledString() << \" is no object\";\n      throw std::runtime_error(error.str());\n    }\n    Json::Value result_status = result[\"status\"];\n    if (!result_status.isString() or\n        (result_status.asString().compare(\"ok\") != 0)) {\n      std::stringstream error;\n      error << result.toStyledString() << \": status != ok\";\n      throw std::runtime_error(error.str());\n    }\n    Json::Value result_value = result[\"value\"];\n    if (!result_value.isIntegral()) {\n      std::stringstream error;\n      error << result.toStyledString() << \" has numerical value field\";\n      throw std::runtime_error(error.str());\n    }\n    return result_value.asInt();\n  }\n\n} // namespace scalaris\n"
  },
  {
    "path": "cpp-api/src/ssl-connection.cpp",
    "content": "// Copyright 2015-2019 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#include \"ssl-connection.hpp\"\n\n#include <boost/algorithm/string.hpp>\n#include <boost/asio.hpp>\n#include <boost/asio/ssl.hpp>\n\nnamespace scalaris {\n\n  SSLConnection::SSLConnection(std::string hostname, std::string link)\n      : Connection(hostname, link),\n        ctx(boost::asio::ssl::context_base::method::sslv23),\n        socket(ioservice, ctx) {\n    try {\n      connect();\n    } catch (scalaris::ConnectionError) {\n    }\n  }\n\n  void SSLConnection::connect() {\n    using boost::asio::ip::tcp;\n\n    socket.set_verify_mode(boost::asio::ssl::verify_peer);\n    socket.set_verify_callback(\n        [this](bool preverified, boost::asio::ssl::verify_context& ctx) {\n          return this->verify_callback(preverified, ctx);\n        });\n\n    // ctx.load_verify_file(\"ca.pem\");\n    ctx.set_password_callback(\n        [this](std::size_t max_length,\n               boost::asio::ssl::context::password_purpose purpose) {\n          return this->password_callback(max_length, purpose);\n        });\n\n    // Determine the location of the server.\n    tcp::resolver resolver(ioservice);\n    tcp::resolver::query query(hostname, std::to_string(getPort()));\n    tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);\n\n    // Try each endpoint until we successfully establish a connection.\n    boost::system::error_code ec;\n    boost::asio::connect(socket.lowest_layer(), endpoint_iterator, ec);\n    hasToConnect = false;\n    if (ec) {\n      std::cout << __FILE__ << \":\" << __LINE__ << \" \" << ec.message()\n                << std::endl;\n      hasToConnect = true;\n      throw ConnectionError(ec.message());\n    }\n\n    socket.handshake(boost::asio::ssl::stream_base::client, ec);\n    if (ec) {\n      std::cout << __FILE__ << \":\" << __LINE__ << \" \" << ec.message()\n                << std::endl;\n      hasToConnect = true;\n      throw ConnectionError(ec.message());\n    }\n  }\n\n  SSLConnection::~SSLConnection() {\n    if (!closed) {\n      socket.lowest_layer().close();\n    }\n  }\n\n  void SSLConnection::set_verify_file(const std::string& file) {\n    // Load a certification authority file for performing verification.\n    ctx.load_verify_file(file); //\"ca.pem\"\n  }\n\n  void SSLConnection::set_certificate_file(const std::string& file) {\n    //  Use a certificate from a file.\n    ctx.use_certificate_file(file,\n                             boost::asio::ssl::context_base::file_format::pem);\n  }\n  void SSLConnection::set_private_key(const std::string& file) {\n    // Use a private key from a file.\n    ctx.use_private_key_file(file,\n                             boost::asio::ssl::context_base::file_format::pem);\n  }\n\n  void SSLConnection::set_rsa_private_key(const std::string& file) {\n    // Use an RSA private key from a file.\n    ctx.use_rsa_private_key_file(\n        file, boost::asio::ssl::context_base::file_format::pem);\n  }\n\n  void SSLConnection::set_password(const std::string& pw) { password = pw; }\n\n  bool SSLConnection::isOpen() const {\n    return socket.lowest_layer().is_open();\n  };\n\n  void SSLConnection::close() {\n    socket.lowest_layer().close();\n    closed = true;\n  };\n\n  unsigned SSLConnection::getPort() const {\n    char* port = getenv(\"SCALARIS_UNITTEST_YAWS_PORT\");\n    if (port == NULL)\n      return 8000;\n    else\n      return atoi(port);\n  }\n\n  bool SSLConnection::verify_callback(bool preverified,\n                                      boost::asio::ssl::verify_context& ctx) {\n    // @todo\n    return true;\n  }\n\n  std::string SSLConnection::password_callback(\n      std::size_t max_length,\n      boost::asio::ssl::context::password_purpose purpose) {\n    // @todo\n    // for_reading or for_writing\n    return password;\n  }\n\n  Json::Value SSLConnection::exec_call(const std::string& methodname,\n                                       Json::Value params, bool reconnect) {\n    // FIXME add reconnect\n    Json::Value call;\n    call[\"method\"] = methodname;\n    call[\"params\"] = params;\n    call[\"id\"] = 0;\n\n    Json::FastWriter w;\n    std::string json_call_str = w.write(call);\n    boost::trim_right(json_call_str);\n\n    boost::asio::streambuf request;\n    std::ostream request_stream(&request);\n    request_stream << \"POST /\" << link << \" HTTP/1.1\\r\\n\";\n    request_stream << \"Host: \" << hostname << \"\\r\\n\";\n    request_stream << \"Content-Type: application/json-rpc\\r\\n\";\n    request_stream << \"Content-Length: \" << json_call_str.size() << \"\\r\\n\";\n    request_stream << \"Connection: keep-alive\\r\\n\\r\\n\";\n    request_stream << json_call_str << \"\\r\\n\\r\\n\";\n\n    // Send the request.\n    boost::system::error_code ec;\n    boost::asio::write(socket, request, ec);\n    if (ec) {\n      std::cout << __FILE__ << \":\" << __LINE__ << \" \" << ec.message()\n                << std::endl;\n      hasToConnect = true;\n      throw ConnectionError(ec.message());\n    }\n\n    boost::asio::streambuf response;\n    boost::asio::read_until(socket, response, \"\\r\\n\");\n    // Check that response is OK.\n    std::istream response_stream(&response);\n    std::string http_version;\n    response_stream >> http_version;\n    unsigned int status_code;\n    response_stream >> status_code;\n    std::string status_message;\n    std::getline(response_stream, status_message);\n    if (!response_stream || http_version.substr(0, 5) != \"HTTP/\") {\n      std::cout << \"Invalid response\\n\";\n      hasToConnect = true;\n      throw ConnectionError(\"Invalid response\");\n    }\n    if (status_code != 200) {\n      std::cout << \"Response returned with status code \" << status_code << \"\\n\";\n      std::stringstream error;\n      error << \"Response returned with status code \" << status_code;\n      hasToConnect = true;\n      throw ConnectionError(error.str());\n    }\n\n    // Read the response headers, which are terminated by a blank line.\n    boost::asio::read_until(socket, response, \"\\r\\n\\r\\n\");\n    // Process the response headers.\n    std::string header;\n    std::stringstream header_stream;\n    while (std::getline(response_stream, header) && header != \"\\r\")\n      header_stream << header << \"\\n\";\n    header_stream << \"\\n\";\n\n    std::stringstream json_result;\n    // Write whatever content we already have to output.\n    if (response.size() > 0)\n      json_result << &response;\n\n    Json::CharReaderBuilder reader_builder;\n    reader_builder[\"collectComments\"] = false;\n    Json::Value value;\n    std::string errs;\n    bool ok = Json::parseFromStream(reader_builder, json_result, &value, &errs);\n    if (!ok) {\n      hasToConnect = true;\n      throw ConnectionError(errs);\n    }\n\n    return process_result(value);\n  }\n\n  Json::Value SSLConnection::process_result(const Json::Value& value) {\n    // @todo\n    if (value.isObject()) {\n      Json::Value id = value[\"id\"];\n      if (!id.isIntegral() or id.asInt() != 0) {\n        std::stringstream error;\n        error << value.toStyledString() << \" is no id=0\";\n        hasToConnect = true;\n        throw ConnectionError(error.str());\n      }\n      Json::Value jsonrpc = value[\"jsonrpc\"];\n      if (!jsonrpc.isString()) {\n        std::stringstream error;\n        error << value.toStyledString() << \" has no string member: jsonrpc\";\n        hasToConnect = true;\n        throw ConnectionError(error.str());\n      }\n      Json::Value result = value[\"result\"];\n      if (!result.isObject()) {\n        std::stringstream error;\n        error << value.toStyledString() << \" has no object member: result\";\n        hasToConnect = true;\n        throw ConnectionError(error.str());\n      }\n      return result;\n    } else {\n      std::stringstream error;\n      error << value.toStyledString() << \" is no json object\";\n      hasToConnect = true;\n      throw ConnectionError(error.str());\n    }\n  }\n\n} // namespace scalaris\n"
  },
  {
    "path": "cpp-api/src/tcp-connection.cpp",
    "content": "// Copyright 2015-2019 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#include \"tcp-connection.hpp\"\n\n#include <boost/algorithm/string.hpp>\n\n#include <boost/asio.hpp>\n\nnamespace scalaris {\n\n  TCPConnection::TCPConnection(const std::string& _hostname,\n                               const std::string& _link, unsigned _port)\n      : Connection(_hostname, _link, _port), socket(ioservice) {\n\n    try {\n      connect();\n    } catch (scalaris::ConnectionError) {\n    }\n  }\n\n  void TCPConnection::connect() {\n    using boost::asio::ip::tcp;\n\n    // Determine the location of the server.\n    tcp::resolver resolver(ioservice);\n    tcp::resolver::query query(hostname, std::to_string(getPort()));\n    tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);\n\n    // create a new socket in case of reconnect\n    socket = boost::asio::ip::tcp::socket(ioservice);\n\n    // Try each endpoint until we successfully establish a connection.\n    boost::system::error_code ec;\n    boost::asio::connect(socket, endpoint_iterator, ec);\n    hasToConnect = false;\n    if (ec) {\n      hasToConnect = true;\n      throw ConnectionError(ec.message());\n    }\n\n    boost::asio::socket_base::keep_alive option(true);\n    socket.set_option(option);\n  }\n\n  TCPConnection::~TCPConnection() {\n    if (!closed) {\n      boost::system::error_code ec;\n      socket.close(ec);\n    }\n  }\n\n  bool TCPConnection::isOpen() const { return socket.is_open(); };\n\n  void TCPConnection::close() {\n    socket.close();\n    closed = true;\n  };\n\n  unsigned TCPConnection::getPort() const {\n    char* EnvPort = getenv(\"SCALARIS_UNITTEST_YAWS_PORT\");\n    if (EnvPort == NULL)\n      return port;\n    else\n      return atoi(EnvPort);\n  }\n\n  Json::Value TCPConnection::exec_call(const std::string& methodname,\n                                       Json::Value params, bool reconnect) {\n\n    // FIXME: reconnect when the connection is closed\n    Json::Value call;\n    call[\"method\"] = methodname;\n    call[\"params\"] = params;\n    call[\"id\"] = 0;\n\n    Json::FastWriter w;\n    std::string json_call_str = w.write(call);\n    boost::trim_right(json_call_str);\n\n    boost::asio::streambuf request;\n    std::ostream request_stream(&request);\n    request_stream << \"POST /\" << link << \" HTTP/1.1\\r\\n\";\n    request_stream << \"Host: \" << hostname << \"\\r\\n\";\n    request_stream << \"Content-Type: application/json-rpc\\r\\n\";\n    request_stream << \"Content-Length: \" << json_call_str.size() << \"\\r\\n\";\n    request_stream << \"Connection: keep-alive\\r\\n\\r\\n\";\n    request_stream << json_call_str << \"\\r\\n\\r\\n\";\n\n    // Send the request.\n    boost::system::error_code ec;\n    boost::asio::write(socket, request, ec);\n    if (ec) {\n      hasToConnect = true;\n      throw ConnectionError(ec.message());\n    }\n\n    try {\n      boost::asio::streambuf response;\n      boost::asio::read_until(socket, response, \"\\r\\n\");\n      // Check that response is OK.\n      std::istream response_stream(&response);\n      std::string http_version;\n      response_stream >> http_version;\n      unsigned int status_code;\n      response_stream >> status_code;\n      std::string status_message;\n      std::getline(response_stream, status_message);\n      if (!response_stream || http_version.substr(0, 5) != \"HTTP/\") {\n        std::cout << \"Invalid response\\n\";\n        hasToConnect = true;\n        throw ConnectionError(\"Invalid response\");\n      }\n      if (status_code != 200) {\n        std::cout << \"Response returned with status code \" << status_code\n                  << \"\\n\";\n        std::stringstream error;\n        error << \"Response returned with status code \" << status_code;\n        hasToConnect = true;\n        throw ConnectionError(error.str());\n      }\n\n      // Read the response headers, which are terminated by a blank line.\n      boost::asio::read_until(socket, response, \"\\r\\n\\r\\n\");\n      // Process the response headers.\n      std::string header;\n      std::stringstream header_stream;\n      while (std::getline(response_stream, header) && header != \"\\r\")\n        header_stream << header << \"\\n\";\n      header_stream << \"\\n\";\n\n      std::stringstream json_result;\n      boost::asio::read_until(socket, response, \"\\n\");\n      // Write whatever content we already have to output.\n      if (response.size() > 0)\n        json_result << &response;\n\n      Json::CharReaderBuilder reader_builder;\n      reader_builder[\"collectComments\"] = false;\n      Json::Value value;\n      std::string errs;\n      bool ok =\n          Json::parseFromStream(reader_builder, json_result, &value, &errs);\n      if (!ok) {\n        hasToConnect = true;\n        throw ConnectionError(errs);\n      }\n      return process_result(value);\n    } catch (const std::exception& e) {\n      if (reconnect) {\n        connect();\n        return exec_call(methodname, params, false);\n      } else {\n        hasToConnect = true;\n        throw ConnectionError(e.what());\n      }\n    }\n  }\n\n  Json::Value TCPConnection::process_result(const Json::Value& value) {\n    if (value.isObject()) {\n      Json::Value id = value[\"id\"];\n      if (!id.isIntegral() or id.asInt() != 0) {\n        std::stringstream error;\n        error << value.toStyledString() << \" is no id=0\";\n        hasToConnect = true;\n        throw ConnectionError(error.str());\n      }\n      Json::Value jsonrpc = value[\"jsonrpc\"];\n      if (!jsonrpc.isString()) {\n        std::stringstream error;\n        error << value.toStyledString() << \" has no string member: jsonrpc\";\n        hasToConnect = true;\n        throw ConnectionError(error.str());\n      }\n      Json::Value result = value[\"result\"];\n      if (!result.isObject()) {\n        std::stringstream error;\n        error << value.toStyledString() << \" has no object member: result\";\n        hasToConnect = true;\n        throw ConnectionError(error.str());\n      }\n      return result;\n    } else {\n      std::stringstream error;\n      error << value.toStyledString() << \" is no json object\";\n      hasToConnect = true;\n      throw ConnectionError(error.str());\n    }\n  }\n\n} // namespace scalaris\n"
  },
  {
    "path": "cpp-api/test/connection.cpp",
    "content": "// Copyright 2015-2017 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#include \"routing_table.hpp\"\n#include \"tcp-connection.hpp\"\n\n#define BOOST_TEST_MAIN\n#define BOOST_TEST_DYN_LINK\n\n#include <boost/filesystem/fstream.hpp>\n#include <boost/test/unit_test.hpp>\n#include <boost/test/unit_test_log.hpp>\n\n#include <iostream>\n\nusing namespace boost::unit_test;\nusing namespace std;\nusing namespace scalaris;\n\nBOOST_AUTO_TEST_SUITE(MasterSuite)\n\nBOOST_AUTO_TEST_CASE(create_connection) {\n  TCPConnection c = {\"localhost\"};\n\n  BOOST_CHECK(c.isOpen());\n}\n\nBOOST_AUTO_TEST_CASE(close_connection) {\n  TCPConnection c = {\"localhost\"};\n  BOOST_CHECK(c.isOpen());\n  c.close();\n  BOOST_CHECK(!c.isOpen());\n}\n\nBOOST_AUTO_TEST_SUITE_END()\n"
  },
  {
    "path": "cpp-api/test/keep_alive.cpp",
    "content": "// Copyright 2015-2017 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#include \"routing_table.hpp\"\n#include \"tcp-connection.hpp\"\n#include \"json/json.h\"\n\n#define BOOST_TEST_DYN_LINK\n\n#include <boost/filesystem/fstream.hpp>\n#include <boost/test/test_tools.hpp>\n#include <boost/test/unit_test.hpp>\n#include <boost/test/unit_test_log.hpp>\n\n#include <iostream>\n\nusing namespace boost::unit_test;\nusing namespace std;\nusing namespace scalaris;\n\nBOOST_AUTO_TEST_SUITE(ConnectionKeepAliveSuite)\n\nBOOST_AUTO_TEST_CASE(keep_alive) {\n  TCPConnection c = {\"localhost\"};\n  RoutingTable rt = {c};\n\n  for (int i = 0; i < 10; i++) {\n    int r = rt.get_replication_factor();\n    BOOST_CHECK(r > 0);\n  }\n}\n\nBOOST_AUTO_TEST_SUITE_END()\n"
  },
  {
    "path": "cpp-api/test/rbr.cpp",
    "content": "// Copyright 2018 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#include \"rbr.hpp\"\n#include \"tcp-connection.hpp\"\n#include \"json/json.h\"\n\n#define BOOST_TEST_DYN_LINK\n\n#include <boost/filesystem/fstream.hpp>\n#include <boost/test/test_tools.hpp>\n#include <boost/test/unit_test.hpp>\n#include <boost/test/unit_test_log.hpp>\n\n#include <iostream>\n\nusing namespace boost::unit_test;\nusing namespace std;\nusing namespace scalaris;\n\nBOOST_AUTO_TEST_SUITE(RbrSuite)\n\nBOOST_AUTO_TEST_CASE(read_unknown_key) {\n  TCPConnection c = {\"localhost\"};\n  Rbr r = {c};\n\n  BOOST_CHECK_THROW(std::string val = r.read(\"_no_such_key\").toStyledString(), ReadFailedError);\n}\n\nBOOST_AUTO_TEST_CASE(write) {\n  TCPConnection c = {\"localhost\"};\n  Rbr r = {c};\n\n  r.write(\"bar\", Json::Value(\"foo\"));\n}\n\nBOOST_AUTO_TEST_CASE(write_read) {\n  {\n    TCPConnection c = {\"localhost\"};\n    Rbr r = {c};\n\n    r.write(\"write_read\", Json::Value(\"foo\"));\n  }\n  {\n    TCPConnection c = {\"localhost\"};\n    Rbr r = {c};\n\n    Json::Value res = r.read(\"write_read\");\n    BOOST_CHECK(res.compare(Json::Value(\"foo\")) == 0);\n  }\n}\n\nBOOST_AUTO_TEST_SUITE_END()\n"
  },
  {
    "path": "cpp-api/test/routing_table.cpp",
    "content": "// Copyright 2015-2017 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#include \"routing_table.hpp\"\n#include \"connection.hpp\"\n#include \"tcp-connection.hpp\"\n#include \"json/json.h\"\n\n#define BOOST_TEST_DYN_LINK\n\n#include <boost/filesystem/fstream.hpp>\n#include <boost/test/test_tools.hpp>\n#include <boost/test/unit_test.hpp>\n#include <boost/test/unit_test_log.hpp>\n\n#include <iostream>\n\nusing namespace boost::unit_test;\nusing namespace std;\nusing namespace scalaris;\n\nBOOST_AUTO_TEST_SUITE(RoutingTableSuite)\n\nBOOST_AUTO_TEST_CASE(get_replication_factor) {\n  TCPConnection c = {\"localhost\"};\n  RoutingTable rt = {c};\n  int r = rt.get_replication_factor();\n  BOOST_CHECK(r > 0);\n}\n\nBOOST_AUTO_TEST_SUITE_END()\n"
  },
  {
    "path": "cpp-api/test/ssl-connection.cpp",
    "content": "// Copyright 2015-2017 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#include \"ssl-connection.hpp\"\n#include \"routing_table.hpp\"\n\n#define BOOST_TEST_MAIN\n#define BOOST_TEST_DYN_LINK\n\n#include <boost/filesystem/fstream.hpp>\n#include <boost/test/unit_test.hpp>\n#include <boost/test/unit_test_log.hpp>\n\n#include <iostream>\n\nusing namespace boost::unit_test;\nusing namespace std;\nusing namespace scalaris;\n\nBOOST_AUTO_TEST_SUITE(MasterSuite)\n\nBOOST_AUTO_TEST_CASE(create_connection) {\n  SSLConnection c = {\"localhost\"};\n\n  c.set_verify_file(\"../ca-chain.cert.pem\");\n  c.set_certificate_file(\"../host2.cert.pem\");\n  c.set_private_key(\"../host2.key.pem\");\n  //c.set_rsa_private_key(const std::string& file);\n  c.set_password(\"secretpassword\");\n\n  BOOST_CHECK(c.isOpen());\n}\n\n//BOOST_AUTO_TEST_CASE(close_connection) {\n//  SSLConnection c = {\"localhost\"};\n//  BOOST_CHECK(c.isOpen());\n//  c.close();\n//  BOOST_CHECK(!c.isOpen());\n//}\n\nBOOST_AUTO_TEST_SUITE_END()\n"
  },
  {
    "path": "cpp-api/test/ssl-routing_table.cpp",
    "content": "// Copyright 2015-2017 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#include \"connection.hpp\"\n#include \"routing_table.hpp\"\n#include \"ssl-connection.hpp\"\n#include \"json/json.h\"\n\n#define BOOST_TEST_DYN_LINK\n\n#include <boost/filesystem/fstream.hpp>\n#include <boost/test/test_tools.hpp>\n#include <boost/test/unit_test.hpp>\n#include <boost/test/unit_test_log.hpp>\n\n#include <iostream>\n\nusing namespace boost::unit_test;\nusing namespace std;\nusing namespace scalaris;\n\nBOOST_AUTO_TEST_SUITE(RoutingTableSuite)\n\n//BOOST_AUTO_TEST_CASE(get_replication_factor) {\n//  printf(\"get_replication_factor\\n\");\n//  SSLConnection c = {\"localhost\"};\n//  RoutingTable rt = {c};\n//  int r = rt.get_replication_factor();\n//  BOOST_CHECK(r > 0);\n//}\n\nBOOST_AUTO_TEST_SUITE_END()\n"
  },
  {
    "path": "cpp-api/test/ssl-transaction_single_op.cpp",
    "content": "// Copyright 2015-2017 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#include \"ssl-connection.hpp\"\n#include \"transaction_single_op.hpp\"\n#include \"json/json.h\"\n\n#define BOOST_TEST_DYN_LINK\n\n#include <boost/filesystem/fstream.hpp>\n#include <boost/test/test_tools.hpp>\n#include <boost/test/unit_test.hpp>\n#include <boost/test/unit_test_log.hpp>\n\n#include <iostream>\n\nusing namespace boost::unit_test;\nusing namespace std;\nusing namespace scalaris;\n\nBOOST_AUTO_TEST_SUITE(TransactionSingleOpSuite)\n\n//BOOST_AUTO_TEST_CASE(read_unknown_key) {\n//  SSLConnection c = {\"localhost\"};\n//  TransactionSingleOp op = {c};\n//\n//  BOOST_CHECK_THROW(std::string val = op.read(\"_no_such_key\"), ReadFailedError);\n//}\n//\n//BOOST_AUTO_TEST_CASE(write) {\n//  SSLConnection c = {\"localhost\"};\n//  TransactionSingleOp op = {c};\n//\n//  op.write(\"bar\", \"foo\");\n//}\n//\n//BOOST_AUTO_TEST_CASE(write_read) {\n//  {\n//    SSLConnection c = {\"localhost\"};\n//    TransactionSingleOp op = {c};\n//\n//    op.write(\"write_read\", \"foo\");\n//  }\n//  {\n//    SSLConnection c = {\"localhost\"};\n//    TransactionSingleOp op = {c};\n//\n//    std::string res = op.read(\"write_read\");\n//    BOOST_CHECK(res.compare(\"foo\") == 0);\n//  }\n//}\n\nBOOST_AUTO_TEST_SUITE_END()\n"
  },
  {
    "path": "cpp-api/test/transaction_single_op.cpp",
    "content": "// Copyright 2015-2017 Zuse Institute Berlin\n//\n//    Licensed under the Apache License, Version 2.0 (the \"License\");\n//    you may not use this file except in compliance with the License.\n//    You may obtain a copy of the License at\n//\n//        http://www.apache.org/licenses/LICENSE-2.0\n//\n//    Unless required by applicable law or agreed to in writing, software\n//    distributed under the License is distributed on an \"AS IS\" BASIS,\n//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n//    See the License for the specific language governing permissions and\n//    limitations under the License.\n\n#include \"transaction_single_op.hpp\"\n#include \"tcp-connection.hpp\"\n#include \"json/json.h\"\n\n#define BOOST_TEST_DYN_LINK\n\n#include <boost/filesystem/fstream.hpp>\n#include <boost/test/test_tools.hpp>\n#include <boost/test/unit_test.hpp>\n#include <boost/test/unit_test_log.hpp>\n\n#include <iostream>\n\nusing namespace boost::unit_test;\nusing namespace std;\nusing namespace scalaris;\n\nBOOST_AUTO_TEST_SUITE(TransactionSingleOpSuite)\n\nBOOST_AUTO_TEST_CASE(read_unknown_key) {\n  TCPConnection c = {\"localhost\"};\n  TransactionSingleOp op = {c};\n\n  BOOST_CHECK_THROW(std::string val = op.read(\"_no_such_key\"), ReadFailedError);\n}\n\nBOOST_AUTO_TEST_CASE(write) {\n  TCPConnection c = {\"localhost\"};\n  TransactionSingleOp op = {c};\n\n  op.write(\"bar\", \"foo\");\n}\n\nBOOST_AUTO_TEST_CASE(write_read) {\n  {\n    TCPConnection c = {\"localhost\"};\n    TransactionSingleOp op = {c};\n\n    op.write(\"write_read\", \"foo\");\n  }\n  {\n    TCPConnection c = {\"localhost\"};\n    TransactionSingleOp op = {c};\n\n    std::string res = op.read(\"write_read\");\n    BOOST_CHECK(res.compare(\"foo\") == 0);\n  }\n}\n\nBOOST_AUTO_TEST_SUITE_END()\n"
  },
  {
    "path": "data/.gitignore",
    "content": "/*\n"
  },
  {
    "path": "doc/.gitignore",
    "content": "/*.html\n/erlang.png\n/transstore\n/pubsub\n/edoc-info\n/comm_layer\n/stylesheet.css\n/cyclon\n/simulation\n/paxos\n/transactions\n/grouped_node\n/Thumbs.db\n"
  },
  {
    "path": "doc/overview.edoc",
    "content": "@copyright 2007-2011 Zuse Institute Berlin\n@version $Id$\n@title Scalaris\n\n@doc\n== Introduction ==\n\nScalaris is a distributed key-value store with support for\ntransactions.\n\n"
  },
  {
    "path": "docroot/.gitignore",
    "content": "/Thumbs.db\n"
  },
  {
    "path": "docroot/api/autoscale.yaws",
    "content": "<erl module=jsonrpc_api_autoscale_mod>\n-compile(export_all).\n-include(\"yaws_handler.hrl\").\n\nforward_to_handler(Operation, Params) ->\n  api_json_autoscale:handler(Operation, Params).\n\n</erl>"
  },
  {
    "path": "docroot/api/dht_raw.yaws",
    "content": "<erl module=jsonrpc_api_dht_raw_mod>\n-compile(export_all).\n-include(\"yaws_handler.hrl\").\n\nforward_to_handler(Operation, Params) ->\n  api_json_dht_raw:handler(Operation, Params).\n\n</erl>\n"
  },
  {
    "path": "docroot/api/monitor.yaws",
    "content": "<erl module=jsonrpc_api_monitor_mod>\n-compile(export_all).\n-include(\"yaws_handler.hrl\").\n\nforward_to_handler(Operation, Params) ->\n  api_json_monitor:handler(Operation, Params).\n\n</erl>\n"
  },
  {
    "path": "docroot/api/rbr.yaws",
    "content": "<erl module=jsonrpc_api_rbr_mod>\n-compile(export_all).\n-include(\"yaws_handler.hrl\").\n\nforward_to_handler(Operation, Params) ->\n  api_json_rbr:handler(Operation, Params).\n\n</erl>\n"
  },
  {
    "path": "docroot/api/rdht.yaws",
    "content": "<erl module=jsonrpc_api_rdht_mod>\n-compile(export_all).\n-include(\"yaws_handler.hrl\").\n\nforward_to_handler(Operation, Params) ->\n  api_json_rdht:handler(Operation, Params).\n\n</erl>\n"
  },
  {
    "path": "docroot/api/rt.yaws",
    "content": "<erl module=jsonrpc_api_rt_mod>\n-compile(export_all).\n-include(\"yaws_handler.hrl\").\n\nforward_to_handler(Operation, Params) ->\n  api_json_rt:handler(Operation, Params).\n\n</erl>\n"
  },
  {
    "path": "docroot/api/tx.yaws",
    "content": "<erl module=jsonrpc_api_tx_mod>\n-compile(export_all).\n-include(\"yaws_handler.hrl\").\n\nforward_to_handler(Operation, Params) ->\n  api_json_tx:handler(Operation, Params).\n\n</erl>\n"
  },
  {
    "path": "docroot/api/vm.yaws",
    "content": "<erl module=jsonrpc_api_monitor_mod>\n-compile(export_all).\n-include(\"yaws_handler.hrl\").\n\nforward_to_handler(Operation, Params) ->\n  api_json_vm:handler(Operation, Params).\n\n</erl>\n"
  },
  {
    "path": "docroot/api/yaws_handler.hrl",
    "content": "%  @copyright 2012-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Common functions for yaws handlers in docroot/api.\n%% @end\n%% @version $Id$\n\nout(A) ->\n    Peer = if is_tuple(A#arg.clisock) andalso\n              element(1, A#arg.clisock) =:= ssl ->\n                   {ssl, SSLSocket} = A#arg.clisock,\n                   ssl:peername(SSLSocket);\n              true ->\n                   inet:peername(A#arg.clisock)\n           end,\n    {ok, {IP, _}} = Peer,\n    A2 = A#arg{state = [{ip, IP}]},\n    case A2#arg.clidata of\n        {partial,_} -> send(A2, 413); % send error code 413 (Request Entity Too Large)\n        _ -> yaws_rpc:handler_session(A2, {?MODULE, handler})\n    end.\n\nhandler([{ip, _IP}] = _State, {call, Operation, {array, Params}}, Session) ->\n    {true, 0, Session, {response, forward_to_handler(Operation, Params)}}.\n\nsend(Args, StatusCode) -> send(Args, StatusCode, \"\").\n\nsend(_Args, StatusCode, Payload) ->\n    [{status, StatusCode},\n     {content, \"application/json\", Payload},\n     {header, {content_length, lists:flatlength(Payload) }}].\n"
  },
  {
    "path": "docroot/bench.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n    <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\">\n  <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n  <![endif]-->\n  </head>\n  <body>\n\n    <div class=\"menubar\">\n      <div class=\"nothighlighted\">\n\t    <h2>Scalaris</h2>\n\t  </div>\n      <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"index.yaws\">Home</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"ring.yaws\">Ring</a>\n      </div>\n\n<erl>\nout(Arg) ->\n    case whereis(mgmt_server) of\n        undefined -> {html, \"\"};\n        _ ->\n            {html,\n\"      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"indexed-ring.yaws\\\">Indexed Ring</a>\n      </div>\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"vivaldi.yaws\\\">Vivaldi Map</a>\n      </div>\" ++ \n    case config:read(dc_clustering_enable) of\n        true -> \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"dc.yaws\\\">Datacenter Clusters Map</a>\n      </div>\n\";\n        _ -> \"\"\n    end ++ \"\n      <div class=\\\"highlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"bench.yaws\\\">Benchmarks</a>\n      </div>\n\"}\n    end.\n</erl>\n\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"gossip.yaws\">Gossip values</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_client.yaws\">Client Monitor</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_ring.yaws\">Ring Monitor</a>\n      </div>\n\t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"documentation.yaws\">Docu</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"debug.yaws\">Debug</a>\n      </div>\n\t  <div class=\"nothighlightedlast\">\n\t    <a class=\"menua\" href=\"logger.yaws\">Message Stats</a>\n\t  </div>\n    </div><div class=\"middle\">\n      <div class=\"middleleft\">\n\n<h2>Benchmarks</h2>\n\n<form method=\"post\" action=\"/bench.yaws\">\n\n<table width=\"400\">\n<tr>\n<td>Threads per VM:</td><td><input name=\"tpm\" size=\"2\" value=\"10\" /></td>\n</tr>\n<tr>\n<td>Iterations per Thread:</td><td><input name=\"iter\" size=\"5\" value=\"1000\" /></td>\n</tr>\n<tr>\n<td>Benchmark:</td><td>\n<select name=\"bench\" size=\"3\">\n <option value=\"increment\" selected=\"selected\">Increment</option>\n <option value=\"quorum_read\">Quorum Read</option>\n <option value=\"read_read\">Read-Read</option>\n</select>\n</td>\n</tr>\n</table>\n\n<p><input type=\"submit\" value=\"Run\"/> | <a href=\"/bench.yaws\">Reset</a></p>\n</form>\n\n<erl>\nout(A) ->\n    IsPost = webhelpers:isPost(A),\n    if\n   \t  IsPost ->\n        case {postvar(A, \"tpm\"),postvar(A, \"iter\")} of\n          {{ok, TpmStr},{ok, IterStr}} ->\n            try {erlang:list_to_integer(TpmStr), erlang:list_to_integer(IterStr)} of\n              {Tpm, Iter} ->\n                Result = case postvar(A,\"bench\") of\n                    {ok, \"increment\"} ->\n                      io_lib:format(\"~p\", [bench:increment_o(Tpm, Iter, [])]);\n                    {ok, \"quorum_read\"} ->\n                      io_lib:format(\"~p\", [bench:quorum_read_o(Tpm, Iter, [])]);\n                    {ok, \"read_read\"} ->\n                      io_lib:format(\"~p\", [bench:read_read_o(Tpm, Iter, [])]);\n                    _ ->\n                      \"<div style='color: red;'>Please select a benchmark</div>\"\n                  end,\n                {ehtml, {pre, [], Result}}\n            catch error:badarg ->\n              {ehtml, {pre, [], \"<div style='color: red;'>Integer numbers required for 'Threads per VM' and 'Iterations'</div>\"}}\n            end;\n          _ ->\n            {ehtml, {pre, [], \"<div style='color: red;'>Missing required input: Need a 'Threads per VM' and 'Iterations'</div>\"}}\n        end;\n    true ->\n      {ehtml,{pre, [], \"\"}}\n    end.\n</erl>\n\n\n\n<h2>Args</h2>\n<erl>\nout(A) ->\n    IsPost = webhelpers:isPost(A),\n    case IsPost of\n      true ->\n        {ehtml, {pre, [], io_lib:format(\"GET:~p~nPOST:~p\", [yaws_api:parse_query(A), yaws_api:parse_post(A)])}};\n      false ->\n        {ehtml, {pre, [], io_lib:format(\"GET:~p\", [yaws_api:parse_query(A)])}}\n    end.\n</erl>\n\n<hr />\n<p>Last update:\n<erl>\n out(Arg) ->\n\t{Date, {Hour, Minute, Second}} = calendar:local_time(),\n    {html, io_lib:format(\"~2..0B:~2..0B:~2..0B\", [Hour, Minute, Second])}.\n</erl></p>\n</div>\n<br class=\"br_class\">&nbsp;\n</div>\n<div class=\"bottom\">\n</div>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/dc.js",
    "content": "var DC = {\n    previous_point: null\n    , get_map: function(map) {\n        $.get(map, function(data) {\n            $(\"div#loading > p\").text(\"Received data, setting up chart..\");\n            data = JSON.parse(data);\n            $(\"#zoom_out\").prop(\"disabled\", true);\n\n            // flot styling and data conversion\n            var dc = 0\n                , hosts = {}\n                , enable_legend = false\n                ;\n\n            var nodes = $.map(data.nodes, function(d) {\n                // hostname not seen before\n                if (hosts[d.info.host] === undefined) {\n                    hosts[d.info.host] = {};\n                }\n\n                // VM on hostname not seen before\n                // The VM is determined by its unique color\n                if(hosts[d.info.host][d.color] === undefined) {\n                    dc += 1;\n                    hosts[d.info.host][d.color] = true;\n                    enable_legend = true;\n                } else {\n                    enable_legend = false;\n                }\n                options = {\n                    color: d.color\n                    , data: [d.info.coords]\n                    , info: d.info\n                };\n                if (enable_legend) {\n                    options.label = {\n                        dc: \"dc \" + hosts[d.info.host]\n                        , name: d.info.name\n                        , host: d.info.host\n                    };\n                }\n                return options;\n            });\n\n            if (data.centroids){\n                $.each(data.centroids, function(_index, c){\n                    var centroid = {\n                        color: \"black\"\n                        , data: [c.coords]\n                        , label: {host: \"centroids\"}\n                        , info: {\n                            host: \"centroid\"\n                            , radius: c.radius\n                        }\n                    };\n                    nodes.push(centroid);\n                });\n\n            }\n\n            // print remaining data\n            var stats = $(\"#stats\").empty(),\n                append = [];\n            $.each(data, function(id){\n                if (id !== \"nodes\" && id !== \"centroids\") {\n                    append.push(\"<li>\"+ id + \": \" + data[id] + \"</li>\");\n                }\n            });\n\n            if (append.length > 0) {\n                $(\"#stats\").append(\"<h2>Additional Data</h2>\")\n                    .append(\"<ul></ul>\");\n                $(\"#stats ul\").append(append.join(\"\\n\"));\n            }\n\n            $(\"div#loading > p\").text(\"\");\n\n            var plot_options = {\n                series: {\n                    points: {show:true}\n                    , hoverable: true\n                    , clickable: true\n                }\n                , grid: {\n                    hoverable: true\n                    , clickable: true\n                }\n                , legend: {\n                    show: true\n                    , labelFormatter: function(label, series) {\n                        return label.host;\n                    }\n                    , container: $(\"#legend\")\n                }\n                , selection: {\n                    mode: \"xy\"\n                }\n            };\n\n            // finally create the plot\n            function redraw(options) {\n                var additional_sets,\n                    ranges;\n\n                if (options){\n                    additional_sets = options.additional_sets;\n                    ranges = options.ranges;\n                }\n                \n                var data = nodes.slice(); // copy array\n                if (additional_sets) {\n                    data.unshift(additional_sets);\n                }\n\n                if (ranges) {\n                    $.extend(plot_options,\n                        {\n                            xaxis: {min: ranges.xaxis.from, max: ranges.xaxis.to}\n                            , yaxis: {min: ranges.yaxis.from, max: ranges.yaxis.to}\n                        }\n                    );\n                    $(\"#zoom_out\").prop(\"disabled\", false);\n                }\n\n                $.plot($(\"#graph\"), data, plot_options);\n            }\n\n            redraw();\n\n            /* Eventhandler for clicks and selections */\n\n            $(\"#zoom_out\").click(function() {\n               $(this).prop(\"disabled\", true);\n               delete plot_options.xaxis;\n               delete plot_options.yaxis;\n               redraw();\n            });\n\n            $(\"#graph\").bind(\"plotselected\", function (event, ranges) {\n                redraw({ranges: ranges});\n            });\n\n            var previous_point;\n            $(\"#graph\").bind(\"plothover\", function(event, pos, item) {\n                if (item) {\n                    if (previousPoint !== item.dataIndex) {\n                        previousPoint = item.dataIndex;\n\n                        $(\"#tooltip\").remove();\n                        var x = item.datapoint[0].toFixed(2),\n                            y = item.datapoint[1].toFixed(2);\n\n                        var label;\n\n                        if (item.series.info) {\n                            if (item.series.info.name !== undefined) {\n                                label = item.series.info.name + \"@\" + item.series.info.host;\n                            } else {\n                                // for centroids\n                                label = DC.formatCentroidLabel(item.series.info.host\n                                                               , item.series.info.radius);\n                            }\n                            DC.showTooltip(item.pageX, item.pageY, label);\n                        }\n                    }\n                } else {\n                    $(\"#tooltip\").remove();\n                    previousPoint = null;            \n                }\n            });\n\n            var previousClickedPoint = null;\n            $(\"#graph\").bind(\"plotclick\", function(event, pos, item){\n                if(item) {\n                    if (previousClickedPoint !== item) {\n                        if (previousClickedPoint) {\n                            var x = parseFloat(item.datapoint[0].toFixed(2)),\n                                y = parseFloat(item.datapoint[1].toFixed(2));\n                            var prev_x = parseFloat(previousClickedPoint.item.datapoint[0].toFixed(2)),\n                                prev_y = parseFloat(previousClickedPoint.item.datapoint[1].toFixed(2));\n\n                            /* connect points */\n                            /* calculate distance */\n                            var distance = Math.sqrt(Math.pow(x-prev_x,2) + Math.pow(y-prev_y,2));\n\n                            line_between_nodes = {\n                                data: [[x,y],[prev_x, prev_y]]\n                                , lines: {show: true}\n                                , points: {\n                                    show: false\n                                    , radius: 0\n                                }\n                            };\n                            redraw({additional_sets: line_between_nodes});\n\n                            $(\"#distanceLine\").remove();\n                            DC.showTooltip((pos.pageX+previousClickedPoint.pos.pageX)/2,\n                                (pos.pageY+previousClickedPoint.pos.pageY)/2, distance,\n                                \"distanceLine\"\n                                );\n                        }\n                        previousClickedPoint = {\n                            item: item\n                            , pos: pos\n                        };\n                    }\n                } else {\n                    previousClickedPoint = null;\n                    $(\"#distanceLine\").remove();\n                    redraw();\n                }\n            });\n        });\n    }\n    , setup: function() {\n        var loc = location.href.split(\"/\")\n        .pop()\n        .split(\".\")[0]\n        ;\n\n        // set up graph\n        if (loc === \"vivaldi\" || loc === \"dc\") {\n            DC.get_map(loc + \"Map.yaws\");\n        }\n    }\n    , formatCentroidLabel: function(host, radius){\n        return \"<p><b>\" + host + \"</b><br/>radius: \" + radius + \"</p>\";\n    }\n    , showTooltip: function (x, y, contents, id) {\n        if (!id){\n            id = \"tooltip\";\n        }\n        $('<div id=\"' + id + '\">' + contents + '</div>').css({\n            position: 'absolute',\n            display: 'none',\n            top: y + 5,\n            left: x + 5,\n            border: '1px solid #fdd',\n            padding: '2px',\n            'background-color': '#fee',\n            opacity: 0.80\n        }).appendTo(\"body\").fadeIn(200);\n    }\n};\n\n$(function(){\n    DC.setup();\n});\n"
  },
  {
    "path": "docroot/dc.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n    <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\">\n    <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n    <![endif]-->\n\n    <!--[if lte IE 8]>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/excanvas.min.js\"></script>\n    <![endif]-->\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.pie.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.colorhelpers.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.navigate.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.errorbars.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.selection.min.js\"></script>\n  </head>\n  <body>\n\n    <div class=\"menubar\">\n      <div class=\"nothighlighted\">\n\t    <h2>Scalaris</h2>\n\t  </div>\n      <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"index.yaws\">Home</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"ring.yaws\">Ring</a>\n      </div>\n\n<erl>\nout(Arg) ->\n    case whereis(mgmt_server) of\n        undefined -> {html, \"\"};\n        _ ->\n            {html,\n\"      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"indexed-ring.yaws\\\">Indexed Ring</a>\n      </div>\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"vivaldi.yaws\\\">Vivaldi Map</a>\n      </div>\n      <div class=\\\"highlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"dc.yaws\\\">Datacenter Clusters Map</a>\n      </div>\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"bench.yaws\\\">Benchmarks</a>\n      </div>\n\"}\n    end.\n</erl>\n\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"gossip.yaws\">Gossip values</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_client.yaws\">Client Monitor</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_ring.yaws\">Ring Monitor</a>\n      </div>\n\t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"documentation.yaws\">Docu</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"debug.yaws\">Debug</a>\n      </div>\n\t  <div class=\"nothighlightedlast\">\n\t    <a class=\"menua\" href=\"logger.yaws\">Message Stats</a>\n\t  </div>\n    </div><div class=\"middle\">\n      <div class=\"middleleft\">\n<h2>Datacenter Clustering Map</h2>\n\n<erl>\nout(Arg) ->\n    case config:read(dc_clustering_enable) of\n        true -> {html, \"\n<div id='plot'>\n    <div id='loading'><p>Loading map...</p></div>\n    <div id='graph' style='width:800px;height:400px;float:left;'></div>\n    <div id='legend' style='float:left'></div>\n</div>\n<div class='clear'></div>\n<button id='zoom_out'>Zoom out</button>\n<br />\n<div id='stats'></div>\n<script src='dc.js'></script>\n\"};\n        _ -> {ehtml, {p, [], \"DC Clustering disabled. Enable clustering in <tt>scalaris.local.cfg</tt> (change to: <tt>{dc_clustering_enable, true}.</tt>).\"}}\n    end.\n</erl>\n\n<hr />\n<p>Last update:\n<erl>\n out(Arg) ->\n    {Date, {Hour, Minute, Second}} = calendar:local_time(),\n    {html, io_lib:format(\"~2..0B:~2..0B:~2..0B\", [Hour, Minute, Second])}.\n</erl></p>\n</div>\n<br class=\"br_class\">&nbsp;\n</div>\n<div class=\"bottom\">\n</div>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/dcMap.yaws",
    "content": "<erl>\n% vim: set ft=erlang:\nout(_A) ->\n    {Nodes, Centroids, Epoch, Radius} = webhelpers:getDCClustersAndNodes(),\n\n    NodesJSON = webhelpers:format_nodes(Nodes),\n    CentroidsJSON = webhelpers:format_centroids(Centroids),\n\n    %% this is actually json, but we\\\"ll trick yaws. YAWS doesn\\\"t know json as a return\n    % atom here.\n    {html, io_lib:format(\"{\n        \\\"nodes\\\":~s,\n        \\\"centroids\\\":~s,\n        \\\"epoch\\\":~p,\n        \\\"cluster_radius\\\":~p\n    }\",[NodesJSON, CentroidsJSON, Epoch, Radius])}\n.\n</erl>\n"
  },
  {
    "path": "docroot/debug.css",
    "content": ".folder .x-tree-node-icon{\n\t\tbackground:transparent url(folder.gif);\n\t}\n\t.x-tree-node-expanded .x-tree-node-icon{\n\t\tbackground:transparent url(folder-open.gif);\n\t}\n\n#tree {\n  float: left;\n  margin: 5px;\n  width: 225px;\n  height: 800px;\n}\n\n#table {\n  float: left;\n  margin: 5px;\n  width: 675px;\n  height: 800px;\n}\n"
  },
  {
    "path": "docroot/debug.js",
    "content": "function updateTable(group, id, tablePanel){\r\n    tablePanel.reconfigure(createStore(group, id), \r\n\t\t\t   new Ext.grid.ColumnModel([\r\n\t\t{header: 'Key', width: 120, sortable: false, dataIndex: 'key'},\r\n\t\t    {header: 'Value', width: 555, sortable: false, dataIndex: 'value'}\r\n\t\t    ]));\r\n    tablePanel.render();\r\n}\r\n\r\nfunction createStore(group, id){\r\n    return new Ext.data.JsonStore({\r\n\t    autoLoad: true,\r\n\t\turl: 'processinfo.yaws?group=' + group + '&id=' + id,\r\n\t\troot: 'pairs',\r\n\t\tfields: ['key', 'value']\r\n\t\t});\r\n}\r\n\r\nExt.onReady(function() {\r\n\t// Note: For the purposes of following along with the tutorial, all \r\n\t// new code should be placed inside this method.\r\n\r\n\t      var processTreeLoader = new Ext.tree.TreeLoader({\r\n\t\tbaseParams: {requestAction: 'processTree'},\r\n\t\t    dataUrl: 'processtree.yaws'\r\n\t\t});\r\n\r\n\t      var rootNode = new Ext.tree.TreeNode();\r\n\t      var treePanel = new Ext.tree.TreePanel({\r\n                      el: 'tree',\r\n\t\t      title : 'Process Tree',\r\n\t\t      loader: processTreeLoader,\r\n\t\t      animate: true,\r\n\t\t      autoScroll: true,\r\n\t\t      containerScroll: true\r\n\t\t});\r\n\r\n\t      var processTreeRoot = new Ext.tree.AsyncTreeNode({\r\n\t\t      text : 'Erlang VM',\r\n\t\t      draggable: false,\r\n\t\t      id : 'processTreeRoot',\r\n\t\t      expanded: true\r\n\t\t});\r\n\r\n\t      treePanel.setRootNode(processTreeRoot);\r\n \r\nvar myReader = new Ext.data.ArrayReader({}, [\r\n{name: 'key'},\r\n{name: 'value'}\r\n\t\t\t\t\t     ]);\r\n\r\nvar myJSONStore = new Ext.data.JsonStore({\r\n\tautoLoad: true,\r\n\turl: 'processinfo.yaws?group=&id=',\r\n\troot: 'pairs',\r\n\tfields: ['key', 'value']\r\n    });\r\n\r\n\r\nvar myJSONStore = createStore(\"\", \"\");\r\n\r\nvar tablePanel = new Ext.grid.GridPanel({\r\n        el: 'table',\r\n\tstore: myJSONStore,\r\n\tcolumns: [\r\n{header: 'Key', width: 120, sortable: false, dataIndex: 'key'},\r\n{header: 'Value', width: 555, sortable: false, dataIndex: 'value'}\r\n\t\t  ],\r\n\tviewConfig: {\r\n\t    forceFit: true\r\n\t},\r\n\twidth: 675,\r\n\ttitle: 'Process Properties',\r\n\tautoExpandColumn: 'Value',\r\n\tframe: true\r\n    });\r\n\r\ntreePanel.addListener( \"click\", function(node, e){\r\n\tif(node.leaf){\r\n\t    updateTable(node.parentNode.id, node.text, tablePanel);\r\n\t}\r\n    });\r\n\r\n\r\ntreePanel.render();\r\ntablePanel.render();\r\n\r\n//    viewport = new Ext.Viewport({\r\n//           layout:'border',\r\n//           items:[treePanel,tablePanel]});\r\n});\r\n"
  },
  {
    "path": "docroot/debug.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n    <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n  <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n  <![endif]-->\n\n    <!-- Include Ext and app-specific scripts: -->\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"ext-all.css\">\n    <script type=\"text/javascript\" src=\"ext-base.js\"></script>\n    <script type=\"text/javascript\" src=\"ext-all.js\"></script>\n    <script type=\"text/javascript\" src=\"debug.js\"></script>\n    \n    <!-- Include Ext stylesheets here: -->\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"debug.css\">\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\"> \n\n\n  </head>\n\n\n  <body>\n\n    <div class=\"menubar\">\n      <div class=\"nothighlighted\">\n\t    <h2>Scalaris</h2>\n\t  </div>\n      <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"index.yaws\">Home</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"ring.yaws\">Ring</a>\n      </div>\n      \n<erl>\nout(Arg) ->\n    case whereis(mgmt_server) of\n        undefined -> {html, \"\"};\n        _ ->\n            {html,\n\"      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"indexed-ring.yaws\\\">Indexed Ring</a>\n      </div>\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"vivaldi.yaws\\\">Vivaldi Map</a>\n      </div>\" ++ \n    case config:read(dc_clustering_enable) of\n        true -> \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"dc.yaws\\\">Datacenter Clusters Map</a>\n      </div>\n\";\n        _ -> \"\"\n    end ++ \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"bench.yaws\\\">Benchmarks</a>\n      </div>\n\"}\n    end.\n</erl>\n\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"gossip.yaws\">Gossip values</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_client.yaws\">Client Monitor</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_ring.yaws\">Ring Monitor</a>\n      </div>\n\t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"documentation.yaws\">Docu</a>\n\t  </div>\n      <div class=\"highlighted\">\n        <a class=\"menua\" href=\"debug.yaws\">Debug</a>\n      </div>\n      <div class=\"nothighlightedlast\">\n        <a class=\"menua\" href=\"logger.yaws\">Message Stats</a>\n      </div>\n    </div><div class=\"middle\">\n      <div class=\"middleleft\">\n<h2>Scalaris Debug Page</h2>\n\n      <div id=\"content\">\n        <div id=\"tree\">\n        </div>\n        <div id=\"table\">\n        </div>\n      </div>\n       \n      </div>\n      <br class=\"br_class\">&nbsp;\n    </div>\n    <div class=\"bottom\">\n    </div>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/documentation.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n    <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\">\n  <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n  <![endif]-->\n  </head>\n  <body>\n\n    <div class=\"menubar\">\n      <div class=\"nothighlighted\">\n\t    <h2>Scalaris</h2>\n\t  </div>\n      <div class=\"nothighlighted\">\n\t    <a  class=\"menua\" href=\"index.yaws\">Home</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"ring.yaws\">Ring</a>\n      </div>\n      \n<erl>\nout(Arg) ->\n    case whereis(mgmt_server) of\n        undefined -> {html, \"\"};\n        _ ->\n            {html,\n\"      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"indexed-ring.yaws\\\">Indexed Ring</a>\n      </div>\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"vivaldi.yaws\\\">Vivaldi Map</a>\n      </div>\" ++ \n    case config:read(dc_clustering_enable) of\n        true -> \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"dc.yaws\\\">Datacenter Clusters Map</a>\n      </div>\n\";\n        _ -> \"\"\n    end ++ \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"bench.yaws\\\">Benchmarks</a>\n      </div>\n\"}\n    end.\n</erl>\n\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"gossip.yaws\">Gossip values</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_client.yaws\">Client Monitor</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_ring.yaws\">Ring Monitor</a>\n      </div>\n\t  <div class=\"highlighted\">\n\t    <a  class=\"menua\" href=\"documentation.yaws\">Docu</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"debug.yaws\">Debug</a>\n      </div>\n      <div class=\"nothighlightedlast\">\n        <a class=\"menua\" href=\"logger.yaws\">Message Stats</a>\n      </div>\n    </div><div class=\"middle\">\n      <div class=\"middleleft\">\n<h2>Documentation</h2>\n\n<p>\nDocumentation extracted from Erlang sources <a name=\"doc\" href=\"doc\" target=\"_blank\" title=\"Documentation extracted from Erlang sources (in new window)\">(open in new window)</a>:\n</p>\n<iframe src=\"doc\" width=\"920\" height=\"600\"></iframe>\n\n<hr />\n<p>Last update:\n<erl>\n out(Arg) ->\n    {Date, {Hour, Minute, Second}} = calendar:local_time(),\n    {html, io_lib:format(\"~2..0B:~2..0B:~2..0B\", [Hour, Minute, Second])}.\n</erl></p>\n</div>\n<br class=\"br_class\">&nbsp;\n</div>\n<div class=\"bottom\">\n</div>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/ext-all.css",
    "content": "/*\r\n * Ext JS Library 2.0.2\r\n * Copyright(c) 2006-2008, Ext JS, LLC.\r\n * licensing@extjs.com\r\n * \r\n * http://extjs.com/license\r\n */\r\n\r\nhtml,body,div,dl,dt,dd,ul,ol,li,h1,h2,h3,h4,h5,h6,pre,form,fieldset,input,p,blockquote,th,td{margin:0;padding:0;}\nimg,body,html{border:0;}\naddress,caption,cite,code,dfn,em,strong,th,var{font-style:normal;font-weight:normal;}\nol,ul{list-style:none;}\ncaption,th{text-align:left;}\nh1,h2,h3,h4,h5,h6{font-size:100%;}\nq:before,q:after{content:'';}\n\n.ext-el-mask{z-index:20000;position:absolute;top:0;left:0;-moz-opacity:0.5;opacity:.50;filter:alpha(opacity=50);background-color:#CCC;width:100%;height:100%;zoom:1;}\n.ext-el-mask-msg{z-index:20001;position:absolute;top:0;left:0;border:1px solid #6593cf;background:#c3daf9 url(../images/default/box/tb-blue.gif) repeat-x 0 -16px;padding:2px;}\n.ext-el-mask-msg div{padding:5px 10px 5px 10px;background:#eee;border:1px solid #a3bad9;color:#222;font:normal 11px tahoma,arial,helvetica,sans-serif;cursor:wait;}\n.ext-shim{position:absolute;visibility:hidden;left:0;top:0;overflow:hidden;}\n.ext-ie .ext-shim{filter:alpha(opacity=0);}\n.ext-ie6 .ext-shim{margin-left:5px;margin-top:3px;}\n.x-mask-loading div{padding:5px 10px 5px 25px;background:#fbfbfb url( '../images/default/grid/loading.gif' ) no-repeat 5px 5px;line-height:16px;}\n.x-hidden,.x-hide-offsets{position:absolute;left:-10000px;top:-10000px;visibility:hidden;}\n.x-hide-display{display:none!important;}\n.x-hide-visibility{visibility:hidden!important;}\n.x-masked{overflow:hidden!important;}\n.x-masked select,.x-masked object,.x-masked embed{visibility:hidden;}\n.x-layer{visibility:hidden;}\n.x-unselectable,.x-unselectable *{-moz-user-select:none;-khtml-user-select:none;}\n.x-repaint{zoom:1;background-color:transparent;-moz-outline:none;}\n.x-item-disabled{color:gray;cursor:default;opacity:.6;-moz-opacity:.6;filter:alpha(opacity=60);}\n.x-item-disabled *{color:gray!important;cursor:default!important;}\n.x-splitbar-proxy{position:absolute;visibility:hidden;z-index:20001;background:#aaa;zoom:1;line-height:1px;font-size:1px;overflow:hidden;}\n.x-splitbar-h,.x-splitbar-proxy-h{cursor:e-resize;cursor:col-resize;}\n.x-splitbar-v,.x-splitbar-proxy-v{cursor:s-resize;cursor:row-resize;}\n.x-color-palette{width:150px;height:92px;cursor:pointer;}\n.x-color-palette a{border:1px solid #fff;float:left;padding:2px;text-decoration:none;-moz-outline:0 none;outline:0 none;cursor:pointer;}\n.x-color-palette a:hover,.x-color-palette a.x-color-palette-sel{border:1px solid #8BB8F3;background:#deecfd;}\n.x-color-palette em{display:block;border:1px solid #ACA899;}\n.x-color-palette em span{cursor:pointer;display:block;height:10px;line-height:10px;width:10px;}\n.x-ie-shadow{display:none;position:absolute;overflow:hidden;left:0;top:0;background:#777;zoom:1;}\n.x-shadow{display:none;position:absolute;overflow:hidden;left:0;top:0;}\n.x-shadow *{overflow:hidden;}\n.x-shadow *{padding:0;border:0;margin:0;clear:none;zoom:1;}\n.x-shadow .xstc,.x-shadow .xsbc{height:6px;float:left;}\n.x-shadow .xstl,.x-shadow .xstr,.x-shadow .xsbl,.x-shadow .xsbr{width:6px;height:6px;float:left;}\n.x-shadow .xsc{width:100%;}\n.x-shadow .xsml,.x-shadow .xsmr{width:6px;float:left;height:100%;}\n.x-shadow .xsmc{float:left;height:100%;background:transparent url( ../images/default/shadow-c.png );}\n.x-shadow .xst,.x-shadow .xsb{height:6px;overflow:hidden;width:100%;}\n.x-shadow .xsml{background:transparent url( ../images/default/shadow-lr.png ) repeat-y 0 0;}\n.x-shadow .xsmr{background:transparent url( ../images/default/shadow-lr.png ) repeat-y -6px 0;}\n.x-shadow .xstl{background:transparent url( ../images/default/shadow.png ) no-repeat 0 0;}\n.x-shadow .xstc{background:transparent url( ../images/default/shadow.png ) repeat-x 0 -30px;}\n.x-shadow .xstr{background:transparent url( ../images/default/shadow.png ) repeat-x 0 -18px;}\n.x-shadow .xsbl{background:transparent url( ../images/default/shadow.png ) no-repeat 0 -12px;}\n.x-shadow .xsbc{background:transparent url( ../images/default/shadow.png ) repeat-x 0 -36px;}\n.x-shadow .xsbr{background:transparent url( ../images/default/shadow.png ) repeat-x 0 -6px;}\n.loading-indicator{font-size:11px;background-image:url(../images/default/grid/loading.gif);background-repeat:no-repeat;background-position:left;padding-left:20px;line-height:16px;margin:3px;}\n.x-text-resize{position:absolute;left:-1000px;top:-1000px;visibility:hidden;zoom:1;}\n.x-drag-overlay{width:100%;height:100%;display:none;position:absolute;left:0;top:0;background-image:url(../images/default/s.gif);z-index:20000;}\n.x-clear{clear:both;height:0;overflow:hidden;line-height:0;font-size:0;}\n.x-spotlight{z-index:8999;position:absolute;top:0;left:0;-moz-opacity:0.5;opacity:.50;filter:alpha(opacity=50);background-color:#CCC;width:0;height:0;zoom:1;}\n\n.x-tab-panel{overflow:hidden;}\n.x-tab-panel-header,.x-tab-panel-footer{background:#deecfd;border:1px solid #8db2e3;overflow:hidden;zoom:1;}\n.x-tab-panel-header{border:1px solid #8db2e3;padding-bottom:2px;}\n.x-tab-panel-footer{border:1px solid #8db2e3;padding-top:2px;}\n.x-tab-strip-wrap{width:100%;overflow:hidden;position:relative;zoom:1;}\nul.x-tab-strip{display:block;width:5000px;zoom:1;}\nul.x-tab-strip-top{padding-top:1px;background:url(../images/default/tabs/tab-strip-bg.gif) #cedff5 repeat-x bottom;border-bottom:1px solid #8db2e3;}\nul.x-tab-strip-bottom{padding-bottom:1px;background:url(../images/default/tabs/tab-strip-btm-bg.gif) #cedff5 repeat-x top;border-top:1px solid #8db2e3;border-bottom:0 none;}\n.x-tab-panel-header-plain .x-tab-strip-top{background:transparent!important;padding-top:0!important;}\n.x-tab-panel-header-plain{background:transparent!important;border-width:0!important;padding-bottom:0!important;}\n.x-tab-panel-header-plain .x-tab-strip-spacer{border:1px solid #8db2e3;border-top:0 none;height:2px;background:#deecfd;font-size:1px;line-height:1px;}\n.ext-border-box .x-tab-panel-header-plain .x-tab-strip-spacer{height:3px;}\nul.x-tab-strip li{float:left;margin-left:2px;}\nul.x-tab-strip li.x-tab-edge{float:left;margin:0!important;padding:0!important;border:0 none!important;font-size:1px!important;line-height:1px!important;overflow:hidden;zoom:1;background:transparent!important;width:1px;}\n.x-tab-strip a,.x-tab-strip span,.x-tab-strip em{display:block;}\n.x-tab-strip a{text-decoration:none!important;-moz-outline:none;outline:none;cursor:pointer;}\n.x-tab-strip-inner{overflow:hidden;text-overflow:ellipsis;}\n.x-tab-strip span.x-tab-strip-text{font:normal 11px tahoma,arial,helvetica;color:#416aa3;white-space:nowrap;cursor:pointer;padding:4px 0;}\n.x-tab-strip .x-tab-with-icon .x-tab-right{padding-left:6px;}\n.x-tab-strip .x-tab-with-icon span.x-tab-strip-text{padding-left:20px;background-position:0 3px;background-repeat:no-repeat;}\n.x-tab-strip-over span.x-tab-strip-text{color:#15428b;}\n.x-tab-strip-active{cursor:default;}\n.x-tab-strip-active span.x-tab-strip-text{cursor:default;color:#15428b;font-weight:bold;}\n.x-tab-strip-disabled .x-tabs-text{cursor:default;color:#aaa;}\n.x-tab-panel-body{overflow:hidden;}\n.x-tab-panel-bwrap{overflow:hidden;}\n.ext-ie .x-tab-strip .x-tab-right{position:relative;}\n.x-tab-strip-top .x-tab-strip-active .x-tab-right{margin-bottom:-1px;}\n.x-tab-strip-top .x-tab-strip-active .x-tab-right span.x-tab-strip-text{padding-bottom:5px;}\n.x-tab-strip-bottom .x-tab-strip-active .x-tab-right{margin-top:-1px;}\n.x-tab-strip-bottom .x-tab-strip-active .x-tab-right span.x-tab-strip-text{padding-top:5px;}\n.x-tab-strip-top .x-tab-right{background:transparent url(../images/default/tabs/tabs-sprite.gif) no-repeat 0 -51px;padding-left:10px;}\n.x-tab-strip-top .x-tab-left{background:transparent url(../images/default/tabs/tabs-sprite.gif) no-repeat right -351px;padding-right:10px;}\n.x-tab-strip-top .x-tab-strip-inner{background:transparent url(../images/default/tabs/tabs-sprite.gif) repeat-x 0 -201px;}\n.x-tab-strip-top .x-tab-strip-over .x-tab-right{background-position:0 -101px;}\n.x-tab-strip-top .x-tab-strip-over .x-tab-left{background-position:right -401px;}\n.x-tab-strip-top .x-tab-strip-over .x-tab-strip-inner{background-position:0 -251px;}\n.x-tab-strip-top .x-tab-strip-active .x-tab-right{background-position:0 0;}\n.x-tab-strip-top .x-tab-strip-active .x-tab-left{background-position:right -301px;}\n.x-tab-strip-top .x-tab-strip-active .x-tab-strip-inner{background-position:0 -151px;}\n.x-tab-strip-bottom .x-tab-right{background:url(../images/default/tabs/tab-btm-inactive-right-bg.gif) no-repeat bottom right;}\n.x-tab-strip-bottom .x-tab-left{background:url(../images/default/tabs/tab-btm-inactive-left-bg.gif) no-repeat bottom left;}\n.x-tab-strip-bottom .x-tab-strip-active .x-tab-right{background:url(../images/default/tabs/tab-btm-right-bg.gif) no-repeat bottom left;}\n.x-tab-strip-bottom .x-tab-strip-active .x-tab-left{background:url(../images/default/tabs/tab-btm-left-bg.gif) no-repeat bottom right;}\n.x-tab-strip-bottom .x-tab-left{padding:0 10px;}\n.x-tab-strip-bottom .x-tab-right{padding:0;}\n.x-tab-strip .x-tab-strip-close{display:none;}\n.x-tab-strip-closable{position:relative;}\n.x-tab-strip-closable .x-tab-left{padding-right:19px;}\n.x-tab-strip .x-tab-strip-closable a.x-tab-strip-close{background-image:url(../images/default/tabs/tab-close.gif);opacity:.6;-moz-opacity:.6;background-repeat:no-repeat;display:block;width:11px;height:11px;position:absolute;top:3px;right:3px;cursor:pointer;z-index:2;}\n.x-tab-strip .x-tab-strip-active a.x-tab-strip-close{opacity:.8;-moz-opacity:.8;}\n.x-tab-strip .x-tab-strip-closable a.x-tab-strip-close:hover{background-image:url(../images/default/tabs/tab-close.gif);opacity:1;-moz-opacity:1;}\n.x-tab-panel-body{border:1px solid #8db2e3;background:#fff;}\n.x-tab-panel-body-top{border-top:0 none;}\n.x-tab-panel-body-bottom{border-bottom:0 none;}\n.x-tab-scroller-left{background:transparent url(../images/default/tabs/scroll-left.gif) no-repeat -18px 0;border-bottom:1px solid #8db2e3;width:18px;position:absolute;left:0;top:0;z-index:10;cursor:pointer;}\n.x-tab-scroller-left-over{background-position:0 0;}\n.x-tab-scroller-left-disabled{background-position:-18px 0;opacity:.5;-moz-opacity:.5;filter:alpha(opacity=50);cursor:default;}\n.x-tab-scroller-right{background:transparent url(../images/default/tabs/scroll-right.gif) no-repeat 0 0;border-bottom:1px solid #8db2e3;width:18px;position:absolute;right:0;top:0;z-index:10;cursor:pointer;}\n.x-tab-scroller-right-over{background-position:-18px 0;}\n.x-tab-scroller-right-disabled{background-position:0 0;opacity:.5;-moz-opacity:.5;filter:alpha(opacity=50);cursor:default;}\n.x-tab-scrolling .x-tab-strip-wrap{margin-left:18px;margin-right:18px;}\n.x-tab-scrolling{position:relative;}\n.x-tab-panel-bbar .x-toolbar{border:1px solid #99bbe8;border-top:0 none;overflow:hidden;padding:2px;}\n.x-tab-panel-tbar .x-toolbar{border:1px solid #99bbe8;border-top:0 none;overflow:hidden;padding:2px;}\n.x-border-layout-ct .x-tab-panel{background:white;}\n\n .x-form-field{margin:0;font:normal 12px tahoma,arial,helvetica,sans-serif;}\n.x-form-text,textarea.x-form-field{padding:1px 3px;background:#fff url(../images/default/form/text-bg.gif) repeat-x 0 0;border:1px solid #B5B8C8;}\ntextarea.x-form-field{padding:2px 3px;}\n.x-form-text{height:22px;line-height:18px;vertical-align:middle;}\n.ext-ie .x-form-text{margin:-1px 0;height:22px;line-height:18px;}\n.ext-ie textarea.x-form-field{margin:-1px 0;}\n.ext-strict .x-form-text{height:18px;}\n.ext-safari .x-form-text{height:20px;padding:0 3px;}\n.ext-safari.ext-mac textarea.x-form-field{margin-bottom:-2px;}\n.ext-gecko .x-form-text{padding-top:2px;padding-bottom:0;}\ntextarea{resize:none;}\n.x-form-select-one{height:20px;line-height:18px;vertical-align:middle;background-color:#fff;border:1px solid #B5B8C8;}\n.x-form-field-wrap{position:relative;zoom:1;white-space:nowrap;}\n.x-editor .x-form-check-wrap{background:#fff;}\n.x-form-field-wrap .x-form-trigger{width:17px;height:21px;border:0;background:transparent url(../images/default/form/trigger.gif) no-repeat 0 0;cursor:pointer;border-bottom:1px solid #B5B8C8;position:absolute;top:0;}\n.ext-safari .x-form-field-wrap .x-form-trigger{height:21px;}\n.x-form-field-wrap .x-form-date-trigger{background-image:url(../images/default/form/date-trigger.gif);cursor:pointer;}\n.x-form-field-wrap .x-form-clear-trigger{background-image:url(../images/default/form/clear-trigger.gif);cursor:pointer;}\n.x-form-field-wrap .x-form-search-trigger{background-image:url(../images/default/form/search-trigger.gif);cursor:pointer;}\n.ext-safari .x-form-field-wrap .x-form-trigger{right:0;}\n.x-form-field-wrap .x-form-twin-triggers .x-form-trigger{position:static;top:auto;vertical-align:top;}\n.x-form-field-wrap .x-form-trigger-over{background-position:-17px 0;}\n.x-form-field-wrap .x-form-trigger-click{background-position:-34px 0;}\n.x-trigger-wrap-focus .x-form-trigger{background-position:-51px 0;}\n.x-trigger-wrap-focus .x-form-trigger-over{background-position:-68px 0;}\n.x-trigger-wrap-focus .x-form-trigger-click{background-position:-85px 0;}\n.x-trigger-wrap-focus .x-form-trigger{border-bottom:1px solid #7eadd9;}\n.x-item-disabled .x-form-trigger-over{background-position:0 0!important;border-bottom:1px solid #B5B8C8;}\n.x-item-disabled .x-form-trigger-click{background-position:0 0!important;border-bottom:1px solid #B5B8C8;}\n.x-form-focus,textarea.x-form-focus{border:1px solid #7eadd9;}\n.x-form-invalid,textarea.x-form-invalid{background:#fff url(../images/default/grid/invalid_line.gif) repeat-x bottom;border:1px solid #dd7870;}\n.ext-safari .x-form-invalid{background-color:#fee;border:1px solid #ff7870;}\n.x-editor{visibility:hidden;padding:0;margin:0;}\n.x-form-check-wrap{line-height:18px;}\n.ext-ie .x-form-check-wrap input{width:15px;height:15px;}\n.x-editor .x-form-check-wrap{padding:3px;}\n.x-editor .x-form-checkbox{height:13px;}\n.x-form-grow-sizer{font:normal 12px tahoma,arial,helvetica,sans-serif;left:-10000px;padding:8px 3px;position:absolute;visibility:hidden;top:-10000px;white-space:pre-wrap;white-space:-moz-pre-wrap;white-space:-pre-wrap;white-space:-o-pre-wrap;word-wrap:break-word;zoom:1;}\n.x-form-grow-sizer p{margin:0!important;border:0 none!important;padding:0!important;}\n.x-form-item{font:normal 12px tahoma,arial,helvetica,sans-serif;display:block;margin-bottom:4px;}\n.x-form-item label{display:block;float:left;width:100px;padding:3px;padding-left:0;clear:left;z-index:2;position:relative;}\n.x-form-element{padding-left:105px;position:relative;}\n.x-form-invalid-msg{color:#e00;padding:2px;padding-left:18px;font:normal 11px tahoma,arial,helvetica,sans-serif;background:transparent url(../images/default/shared/warning.gif) no-repeat 0 2px;line-height:16px;width:200px;}\n.x-form-label-right label{text-align:right;}\n.x-form-label-top .x-form-item label{width:auto;float:none;clear:none;display:inline;margin-bottom:4px;position:static;}\n.x-form-label-top .x-form-element{padding-left:0;padding-top:4px;}\n.x-form-label-top .x-form-item{padding-bottom:4px;}\n.x-form-empty-field{color:gray;}\n.x-small-editor .x-form-field{font:normal 11px arial,tahoma,helvetica,sans-serif;}\n.x-small-editor .x-form-text{height:20px;line-height:16px;vertical-align:middle;}\n.ext-ie .x-small-editor .x-form-text{margin-top:-1px!important;margin-bottom:-1px!important;height:20px!important;line-height:16px!important;}\n.ext-strict .x-small-editor .x-form-text{height:16px!important;}\n.ext-safari .x-small-editor .x-form-field{font:normal 12px arial,tahoma,helvetica,sans-serif;}\n.ext-ie .x-small-editor .x-form-text{height:20px;line-height:16px;}\n.ext-border-box .x-small-editor .x-form-text{height:20px;}\n.x-small-editor .x-form-select-one{height:20px;line-height:16px;vertical-align:middle;}\n.x-small-editor .x-form-num-field{text-align:right;}\n.x-small-editor .x-form-field-wrap .x-form-trigger{height:19px;}\n.x-form-clear{clear:both;height:0;overflow:hidden;line-height:0;font-size:0;}\n.x-form-clear-left{clear:left;height:0;overflow:hidden;line-height:0;font-size:0;}\n.x-form-cb-label{width:'auto'!important;float:none!important;clear:none!important;display:inline!important;margin-left:4px;}\n.x-form-column{float:left;padding:0;margin:0;width:48%;overflow:hidden;zoom:1;}\n.x-form .x-form-btns-ct .x-btn{float:right;clear:none;}\n.x-form .x-form-btns-ct .x-form-btns td{border:0;padding:0;}\n.x-form .x-form-btns-ct .x-form-btns-right table{float:right;clear:none;}\n.x-form .x-form-btns-ct .x-form-btns-left table{float:left;clear:none;}\n.x-form .x-form-btns-ct .x-form-btns-center{text-align:center;}\n.x-form .x-form-btns-ct .x-form-btns-center table{margin:0 auto;}\n.x-form .x-form-btns-ct table td.x-form-btn-td{padding:3px;}\n.x-form .x-form-btns-ct .x-btn-focus .x-btn-left{background-position:0 -147px;}\n.x-form .x-form-btns-ct .x-btn-focus .x-btn-right{background-position:0 -168px;}\n.x-form .x-form-btns-ct .x-btn-focus .x-btn-center{background-position:0 -189px;}\n.x-form .x-form-btns-ct .x-btn-click .x-btn-center{background-position:0 -126px;}\n.x-form .x-form-btns-ct .x-btn-click .x-btn-right{background-position:0 -84px;}\n.x-form .x-form-btns-ct .x-btn-click .x-btn-left{background-position:0 -63px;}\n.x-form-invalid-icon{width:16px;height:18px;visibility:hidden;position:absolute;left:0;top:0;display:block;background:transparent url(../images/default/form/exclamation.gif) no-repeat 0 2px;}\n.x-fieldset{border:1px solid #B5B8C8;padding:10px;margin-bottom:10px;}\n.x-fieldset legend{font:bold 11px tahoma,arial,helvetica,sans-serif;color:#15428b;}\n.ext-ie .x-fieldset legend{margin-bottom:10px;}\n.ext-ie .x-fieldset{padding-top:0;padding-bottom:10px;}\n.x-fieldset legend .x-tool-toggle{margin-right:3px;margin-left:0;float:left!important;}\n.x-fieldset legend input{margin-right:3px;float:left!important;height:13px;width:13px;}\nfieldset.x-panel-collapsed{padding-bottom:0!important;border-width:1px 0 0 0!important;}\nfieldset.x-panel-collapsed .x-fieldset-bwrap{visibility:hidden;position:absolute;left:-1000px;top:-1000px;}\n.ext-ie .x-fieldset-bwrap{zoom:1;}\n.ext-ie td .x-form-text{position:relative;top:-1px;}\n.x-fieldset-noborder{border:0 none transparent;}\n.x-fieldset-noborder legend{margin-left:-3px;}\n.ext-ie .x-fieldset-noborder legend{position:relative;margin-bottom:23px;}\n.ext-ie .x-fieldset-noborder legend span{position:absolute;left:-5px;}\n.ext-gecko .x-window-body .x-form-item{-moz-outline:none;overflow:auto;}\n.ext-gecko .x-form-item{-moz-outline:none;}\n.x-hide-label label.x-form-item-label{display:none;}\n.x-hide-label .x-form-element{padding-left:0!important;}\n.x-fieldset{overflow:hidden;}\n.x-fieldset-bwrap{overflow:hidden;zoom:1;}\n.x-fieldset-body{overflow:hidden;}\n\n.x-btn{font:normal 11px tahoma,verdana,helvetica;cursor:pointer;white-space:nowrap;}\n.x-btn button{border:0 none;background:transparent;font:normal 11px tahoma,verdana,helvetica;padding-left:3px;padding-right:3px;cursor:pointer;margin:0;overflow:visible;width:auto;-moz-outline:0 none;outline:0 none;}\n* html .ext-ie .x-btn button{width:1px;}\n.ext-gecko .x-btn button{padding-left:0;padding-right:0;}\n.ext-ie .x-btn button{padding-top:2px;}\n.x-btn-icon .x-btn-center .x-btn-text{background-position:center;background-repeat:no-repeat;height:16px;width:16px;cursor:pointer;white-space:nowrap;padding:0;}\n.x-btn-icon .x-btn-center{padding:1px;}\n.x-btn em{font-style:normal;font-weight:normal;}\n.x-btn-text-icon .x-btn-center .x-btn-text{background-position:0 2px;background-repeat:no-repeat;padding-left:18px;padding-top:3px;padding-bottom:2px;padding-right:0;}\n.x-btn-left,.x-btn-right{font-size:1px;line-height:1px;}\n.x-btn-left{width:3px;height:21px;background:url(../images/default/button/btn-sprite.gif) no-repeat 0 0;}\n.x-btn-right{width:3px;height:21px;background:url(../images/default/button/btn-sprite.gif) no-repeat 0 -21px;}\n.x-btn-left i,.x-btn-right i{display:block;width:3px;overflow:hidden;font-size:1px;line-height:1px;}\n.x-btn-center{background:url(../images/default/button/btn-sprite.gif) repeat-x 0 -42px;vertical-align:middle;text-align:center;padding:0 5px;cursor:pointer;white-space:nowrap;}\n.x-btn-over .x-btn-left{background-position:0 -63px;}\n.x-btn-over .x-btn-right{background-position:0 -84px;}\n.x-btn-over .x-btn-center{background-position:0 -105px;}\n.x-btn-click .x-btn-center,.x-btn-menu-active .x-btn-center{background-position:0 -126px;}\n.x-btn-disabled *{color:gray!important;cursor:default!important;}\n.x-btn-menu-text-wrap .x-btn-center{padding:0 3px;}\n.ext-gecko .x-btn-menu-text-wrap .x-btn-center{padding:0 1px;}\n.x-btn-menu-arrow-wrap .x-btn-center{padding:0;}\n.x-btn-menu-arrow-wrap .x-btn-center button{width:12px!important;height:21px;padding:0!important;display:block;background:transparent url(../images/default/button/btn-arrow.gif) no-repeat left 3px;}\n.x-btn-with-menu .x-btn-center{padding-right:2px!important;}\n.x-btn-with-menu .x-btn-center em{display:block;background:transparent url(../images/default/toolbar/btn-arrow.gif) no-repeat right 0;padding-right:10px;}\n.x-btn-text-icon .x-btn-with-menu .x-btn-center em{display:block;background:transparent url(../images/default/toolbar/btn-arrow.gif) no-repeat right 3px;padding-right:10px;}\n.x-btn-pressed .x-btn-left{background:url(../images/default/button/btn-sprite.gif) no-repeat 0 -63px;}\n.x-btn-pressed .x-btn-right{background:url(../images/default/button/btn-sprite.gif) no-repeat 0 -84px;}\n.x-btn-pressed .x-btn-center{background:url(../images/default/button/btn-sprite.gif) repeat-x 0 -126px;}\n\n.x-toolbar{border-color:#a9bfd3;border-style:solid;border-width:0 0 1px 0;display:block;padding:2px;background:#d0def0 url(../images/default/toolbar/bg.gif) repeat-x top left;position:relative;zoom:1;}\n.x-toolbar .x-item-disabled .x-btn-icon{opacity:.35;-moz-opacity:.35;filter:alpha(opacity=35);}\n.x-toolbar td{vertical-align:middle;}\n.mso .x-toolbar,.x-grid-mso .x-toolbar{border:0 none;background:url(../images/default/grid/mso-hd.gif);}\n.x-toolbar td,.x-toolbar span,.x-toolbar input,.x-toolbar div,.x-toolbar select,.x-toolbar label{white-space:nowrap;font:normal 11px tahoma,arial,helvetica,sans-serif;}\n.x-toolbar .x-item-disabled{color:gray;cursor:default;opacity:.6;-moz-opacity:.6;filter:alpha(opacity=60);}\n.x-toolbar .x-item-disabled *{color:gray;cursor:default;}\n.x-toolbar .x-btn-left{background:none;}\n.x-toolbar .x-btn-right{background:none;}\n.x-toolbar .x-btn-center{background:none;padding:0;}\n.x-toolbar .x-btn-menu-text-wrap .x-btn-center button{padding-right:2px;}\n.ext-gecko .x-toolbar .x-btn-menu-text-wrap .x-btn-center button{padding-right:0;}\n.x-toolbar .x-btn-menu-arrow-wrap .x-btn-center button{padding:0 2px;}\n.x-toolbar .x-btn-menu-arrow-wrap .x-btn-center button{width:12px;background:transparent url(../images/default/toolbar/btn-arrow.gif) no-repeat 0 3px;}\n.x-toolbar .x-btn-text-icon .x-btn-menu-arrow-wrap .x-btn-center button{width:12px;background:transparent url(../images/default/toolbar/btn-arrow.gif) no-repeat 0 3px;}\n.x-toolbar .x-btn-over .x-btn-menu-arrow-wrap .x-btn-center button{background-position:0 -47px;}\n.x-toolbar .x-btn-over .x-btn-left{background:url(../images/default/toolbar/tb-btn-sprite.gif) no-repeat 0 0;}\n.x-toolbar .x-btn-over .x-btn-right{background:url(../images/default/toolbar/tb-btn-sprite.gif) no-repeat 0 -21px;}\n.x-toolbar .x-btn-over .x-btn-center{background:url(../images/default/toolbar/tb-btn-sprite.gif) repeat-x 0 -42px;}\n.x-toolbar .x-btn-click .x-btn-left,.x-toolbar .x-btn-pressed .x-btn-left,.x-toolbar .x-btn-menu-active .x-btn-left{background:url(../images/default/toolbar/tb-btn-sprite.gif) no-repeat 0 -63px;}\n.x-toolbar .x-btn-click .x-btn-right,.x-toolbar .x-btn-pressed .x-btn-right,.x-toolbar .x-btn-menu-active .x-btn-right{background:url(../images/default/toolbar/tb-btn-sprite.gif) no-repeat 0 -84px;}\n.x-toolbar .x-btn-click .x-btn-center,.x-toolbar .x-btn-pressed .x-btn-center,.x-toolbar .x-btn-menu-active .x-btn-center{background:url(../images/default/toolbar/tb-btn-sprite.gif) repeat-x 0 -105px;}\n.x-toolbar .x-btn-with-menu .x-btn-center em{padding-right:8px;}\n.x-toolbar .ytb-text{padding:2px;}\n.x-toolbar .ytb-sep{background-image:url(../images/default/grid/grid-blue-split.gif);background-position:center;background-repeat:no-repeat;display:block;font-size:1px;height:16px;width:4px;overflow:hidden;cursor:default;margin:0 2px 0;border:0;}\n.x-toolbar .ytb-spacer{width:2px;}\n.x-tbar-page-number{width:24px;height:14px;}\n.x-tbar-page-first{background-image:url(../images/default/grid/page-first.gif)!important;}\n.x-tbar-loading{background-image:url(../images/default/grid/done.gif)!important;}\n.x-tbar-page-last{background-image:url(../images/default/grid/page-last.gif)!important;}\n.x-tbar-page-next{background-image:url(../images/default/grid/page-next.gif)!important;}\n.x-tbar-page-prev{background-image:url(../images/default/grid/page-prev.gif)!important;}\n.x-item-disabled .x-tbar-loading{background-image:url(../images/default/grid/loading.gif)!important;}\n.x-item-disabled .x-tbar-page-first{background-image:url(../images/default/grid/page-first-disabled.gif)!important;}\n.x-item-disabled .x-tbar-page-last{background-image:url(../images/default/grid/page-last-disabled.gif)!important;}\n.x-item-disabled .x-tbar-page-next{background-image:url(../images/default/grid/page-next-disabled.gif)!important;}\n.x-item-disabled .x-tbar-page-prev{background-image:url(../images/default/grid/page-prev-disabled.gif)!important;}\n.x-paging-info{position:absolute;top:5px;right:8px;color:#444;}\n\n.x-resizable-handle{position:absolute;z-index:100;font-size:1px;line-height:6px;overflow:hidden;background:white;filter:alpha(opacity=0);opacity:0;zoom:1;}\n.x-resizable-handle-east{width:6px;cursor:e-resize;right:0;top:0;height:100%;}\n.ext-ie .x-resizable-handle-east{margin-right:-1px;}\n.x-resizable-handle-south{width:100%;cursor:s-resize;left:0;bottom:0;height:6px;}\n.ext-ie .x-resizable-handle-south{margin-bottom:-1px;}\n.x-resizable-handle-west{width:6px;cursor:w-resize;left:0;top:0;height:100%;}\n.x-resizable-handle-north{width:100%;cursor:n-resize;left:0;top:0;height:6px;}\n.x-resizable-handle-southeast{width:6px;cursor:se-resize;right:0;bottom:0;height:6px;z-index:101;}\n.x-resizable-handle-northwest{width:6px;cursor:nw-resize;left:0;top:0;height:6px;z-index:101;}\n.x-resizable-handle-northeast{width:6px;cursor:ne-resize;right:0;top:0;height:6px;z-index:101;}\n.x-resizable-handle-southwest{width:6px;cursor:sw-resize;left:0;bottom:0;height:6px;z-index:101;}\n.x-resizable-over .x-resizable-handle,.x-resizable-pinned .x-resizable-handle{filter:alpha(opacity=100);opacity:1;}\n.x-resizable-over .x-resizable-handle-east,.x-resizable-pinned .x-resizable-handle-east{background:url(../images/default/sizer/e-handle.gif);background-position:left;}\n.x-resizable-over .x-resizable-handle-west,.x-resizable-pinned .x-resizable-handle-west{background:url(../images/default/sizer/e-handle.gif);background-position:left;}\n.x-resizable-over .x-resizable-handle-south,.x-resizable-pinned .x-resizable-handle-south{background:url(../images/default/sizer/s-handle.gif);background-position:top;}\n.x-resizable-over .x-resizable-handle-north,.x-resizable-pinned .x-resizable-handle-north{background:url(../images/default/sizer/s-handle.gif);background-position:top;}\n.x-resizable-over .x-resizable-handle-southeast,.x-resizable-pinned .x-resizable-handle-southeast{background:url(../images/default/sizer/se-handle.gif);background-position:top left;}\n.x-resizable-over .x-resizable-handle-northwest,.x-resizable-pinned .x-resizable-handle-northwest{background:url(../images/default/sizer/nw-handle.gif);background-position:bottom right;}\n.x-resizable-over .x-resizable-handle-northeast,.x-resizable-pinned .x-resizable-handle-northeast{background:url(../images/default/sizer/ne-handle.gif);background-position:bottom left;}\n.x-resizable-over .x-resizable-handle-southwest,.x-resizable-pinned .x-resizable-handle-southwest{background:url(../images/default/sizer/sw-handle.gif);background-position:top right;}\n.x-resizable-proxy{border:1px dashed #3b5a82;position:absolute;overflow:hidden;display:none;left:0;top:0;z-index:50000;}\n.x-resizable-overlay{width:100%;height:100%;display:none;position:absolute;left:0;top:0;background:white;z-index:200000;-moz-opacity:0;opacity:0;filter:alpha(opacity=0);}\n\n .x-grid3{position:relative;overflow:hidden;background-color:#fff;}\n.x-grid-panel .x-panel-body{overflow:hidden!important;}\n.x-grid-panel .x-panel-mc .x-panel-body{border:1px solid #99bbe8;}\n.ext-ie .x-grid3 table,.ext-safari .x-grid3 table{table-layout:fixed;}\n.x-grid3-viewport{overflow:hidden;}\n.x-grid3-hd-row td,.x-grid3-row td,.x-grid3-summary-row td{font:normal 11px arial,tahoma,helvetica,sans-serif;-moz-outline:none;-moz-user-focus:normal;}\n.x-grid3-row td,.x-grid3-summary-row td{line-height:13px;vertical-align:top;padding-left:1px;padding-right:1px;-moz-user-select:none;}\n.x-grid3-hd-row td{line-height:15px;vertical-align:middle;border-left:1px solid #eee;border-right:1px solid #d0d0d0;}\n.x-grid3-hd-row .x-grid3-marker-hd{padding:3px;}\n.x-grid3-row .x-grid3-marker{padding:3px;}\n.x-grid3-cell-inner,.x-grid3-hd-inner{overflow:hidden;-o-text-overflow:ellipsis;text-overflow:ellipsis;padding:3px 3px 3px 5px;white-space:nowrap;}\n.x-grid3-hd-inner{position:relative;cursor:inherit;padding:4px 3px 4px 5px;}\n.x-grid3-row-body{white-space:normal;}\n.x-grid3-body-cell{-moz-outline:0 none;outline:0 none;}\n.ext-ie .x-grid3-cell-inner,.ext-ie .x-grid3-hd-inner{width:100%;}\n.ext-strict .x-grid3-cell-inner,.ext-strict .x-grid3-hd-inner{width:auto;}\n.x-grid-row-loading{background:#fff url(../images/default/shared/loading-balls.gif) no-repeat center center;}\n.x-grid-page{overflow:hidden;}\n.x-grid3-row{cursor:default;border:1px solid #ededed;border-top-color:#fff;width:100%;}\n.x-grid3-row-alt{background-color:#fafafa;}\n.x-grid3-row-over{border:1px solid #ddd;background:#efefef url(../images/default/grid/row-over.gif) repeat-x left top;}\n.x-grid3-resize-proxy{width:1px;left:0;background-color:#777;cursor:e-resize;cursor:col-resize;position:absolute;top:0;height:100px;overflow:hidden;visibility:hidden;border:0 none;z-index:7;}\n.x-grid3-resize-marker{width:1px;left:0;background-color:#777;position:absolute;top:0;height:100px;overflow:hidden;visibility:hidden;border:0 none;z-index:7;}\n.x-grid3-focus{position:absolute;top:0;-moz-outline:0 none;outline:0 none;-moz-user-select:normal;-khtml-user-select:normal;}\n.x-grid3-header{background:#f9f9f9 url(../images/default/grid/grid3-hrow.gif) repeat-x 0 bottom;cursor:default;zoom:1;padding:1px 0 0 0;}\n.x-grid3-header-pop{border-left:1px solid #d0d0d0;float:right;clear:none;}\n.x-grid3-header-pop-inner{border-left:1px solid #eee;width:14px;height:19px;background:transparent url(../images/default/grid/hd-pop.gif) no-repeat center center;}\n.ext-ie .x-grid3-header-pop-inner{width:15px;}\n.ext-strict .x-grid3-header-pop-inner{width:14px;}\n.x-grid3-header-inner{overflow:hidden;zoom:1;float:left;}\n.x-grid3-header-offset{padding-left:1px;width:10000px;}\ntd.x-grid3-hd-over,td.sort-desc,td.sort-asc,td.x-grid3-hd-menu-open{border-left:1px solid #aaccf6;border-right:1px solid #aaccf6;}\ntd.x-grid3-hd-over .x-grid3-hd-inner,td.sort-desc .x-grid3-hd-inner,td.sort-asc .x-grid3-hd-inner,td.x-grid3-hd-menu-open .x-grid3-hd-inner{background:#ebf3fd url(../images/default/grid/grid3-hrow-over.gif) repeat-x left bottom;}\n.x-grid3-sort-icon{background-repeat:no-repeat;display:none;height:4px;width:13px;margin-left:3px;vertical-align:middle;}\n.sort-asc .x-grid3-sort-icon{background-image:url(../images/default/grid/sort_asc.gif);display:inline;}\n.sort-desc .x-grid3-sort-icon{background-image:url(../images/default/grid/sort_desc.gif);display:inline;}\n.ext-strict .ext-ie .x-grid3-header-inner{position:relative;}\n.ext-strict .ext-ie6 .x-grid3-hd{position:relative;}\n.ext-strict .ext-ie6 .x-grid3-hd-inner{position:static;}\n.x-grid3-body{zoom:1;}\n.x-grid3-scroller{overflow:auto;zoom:1;position:relative;}\n.x-grid3-cell-text,.x-grid3-hd-text{display:block;padding:3px 5px 3px 5px;-moz-user-select:none;-khtml-user-select:none;color:black;}\n.x-grid3-split{background-image:url(../images/default/grid/grid-split.gif);background-position:center;background-repeat:no-repeat;cursor:e-resize;cursor:col-resize;display:block;font-size:1px;height:16px;overflow:hidden;position:absolute;top:2px;width:6px;z-index:3;}\n.x-grid3-hd-text{color:#15428b;}\n.x-dd-drag-proxy .x-grid3-hd-inner{background:#ebf3fd url(../images/default/grid/grid3-hrow-over.gif) repeat-x left bottom;width:120px;padding:3px;border:1px solid #aaccf6;overflow:hidden;}\n.col-move-top,.col-move-bottom{width:9px;height:9px;position:absolute;top:0;line-height:1px;font-size:1px;overflow:hidden;visibility:hidden;z-index:20000;}\n.col-move-top{background:transparent url(../images/default/grid/col-move-top.gif) no-repeat left top;}\n.col-move-bottom{background:transparent url(../images/default/grid/col-move-bottom.gif) no-repeat left top;}\n.x-grid3-row-selected{background:#DFE8F6!important;border:1px dotted #a3bae9;}\n.x-grid3-cell-selected{background-color:#B8CFEE!important;color:black;}\n.x-grid3-cell-selected span{color:black!important;}\n.x-grid3-cell-selected .x-grid3-cell-text{color:black;}\n.x-grid3-locked td.x-grid3-row-marker,.x-grid3-locked .x-grid3-row-selected td.x-grid3-row-marker{background:#ebeadb url(../images/default/grid/grid-hrow.gif) repeat-x 0 bottom!important;vertical-align:middle!important;color:black;padding:0;border-top:1px solid white;border-bottom:none!important;border-right:1px solid #6fa0df!important;text-align:center;}\n.x-grid3-locked td.x-grid3-row-marker div,.x-grid3-locked .x-grid3-row-selected td.x-grid3-row-marker div{padding:0 4px;color:#15428b!important;text-align:center;}\n.x-grid3-dirty-cell{background:transparent url(../images/default/grid/dirty.gif) no-repeat 0 0;}\n.x-grid3-topbar,.x-grid3-bottombar{font:normal 11px arial,tahoma,helvetica,sans-serif;overflow:hidden;display:none;zoom:1;position:relative;}\n.x-grid3-topbar .x-toolbar{border-right:0 none;}\n.x-grid3-bottombar .x-toolbar{border-right:0 none;border-bottom:0 none;border-top:1px solid #a9bfd3;}\n.x-props-grid .x-grid3-cell{padding:1px;}\n.x-props-grid .x-grid3-td-name .x-grid3-cell-inner{background:transparent url(../images/default/grid/grid3-special-col-bg.gif) repeat-y -16px!important;padding-left:12px;color:black!important;}\n.x-props-grid .x-grid3-body .x-grid3-td-name{padding:1px;padding-right:0;background:white!important;border:0 none;border-right:1px solid #eee;}\n.xg-hmenu-sort-asc .x-menu-item-icon{background-image:url(../images/default/grid/hmenu-asc.gif);}\n.xg-hmenu-sort-desc .x-menu-item-icon{background-image:url(../images/default/grid/hmenu-desc.gif);}\n.xg-hmenu-lock .x-menu-item-icon{background-image:url(../images/default/grid/hmenu-lock.gif);}\n.xg-hmenu-unlock .x-menu-item-icon{background-image:url(../images/default/grid/hmenu-unlock.gif);}\n.x-grid3-col-dd{border:0 none;padding:0;background:transparent;}\n.x-dd-drag-ghost .x-grid3-dd-wrap{padding:1px 3px 3px 1px;}\n.x-grid3-hd{-moz-user-select:none;}\n.x-grid3-hd-btn{display:none;position:absolute;width:14px;background:#c3daf9 url(../images/default/grid/grid3-hd-btn.gif) no-repeat left center;right:0;top:0;z-index:2;cursor:pointer;}\n.x-grid3-hd-over .x-grid3-hd-btn,.x-grid3-hd-menu-open .x-grid3-hd-btn{display:block;}\na.x-grid3-hd-btn:hover{background-position:-14px center;}\n.x-grid3-body .x-grid3-td-expander{background:transparent url(../images/default/grid/grid3-special-col-bg.gif) repeat-y right;}\n.x-grid3-body .x-grid3-td-expander .x-grid3-cell-inner{padding:0!important;height:100%;}\n.x-grid3-row-expander{width:100%;height:18px;background-position:4px 2px;background-repeat:no-repeat;background-color:transparent;background-image:url(../images/default/grid/row-expand-sprite.gif);}\n.x-grid3-row-collapsed .x-grid3-row-expander{background-position:4px 2px;}\n.x-grid3-row-expanded .x-grid3-row-expander{background-position:-21px 2px;}\n.x-grid3-row-collapsed .x-grid3-row-body{display:none!important;}\n.x-grid3-row-expanded .x-grid3-row-body{display:block!important;}\n.x-grid3-body .x-grid3-td-checker{background:transparent url(../images/default/grid/grid3-special-col-bg.gif) repeat-y right;}\n.x-grid3-body .x-grid3-td-checker .x-grid3-cell-inner,.x-grid3-header .x-grid3-td-checker .x-grid3-hd-inner{padding:0!important;height:100%;}\n.x-grid3-row-checker,.x-grid3-hd-checker{width:100%;height:18px;background-position:2px 2px;background-repeat:no-repeat;background-color:transparent;background-image:url(../images/default/grid/row-check-sprite.gif);}\n.x-grid3-row .x-grid3-row-checker{background-position:2px 2px;}\n.x-grid3-row-selected .x-grid3-row-checker,.x-grid3-hd-checker-on .x-grid3-hd-checker{background-position:-23px 2px;}\n.x-grid3-hd-checker{background-position:2px 3px;}\n.x-grid3-hd-checker-on .x-grid3-hd-checker{background-position:-23px 3px;}\n.x-grid3-body .x-grid3-td-numberer{background:transparent url(../images/default/grid/grid3-special-col-bg.gif) repeat-y right;}\n.x-grid3-body .x-grid3-td-numberer .x-grid3-cell-inner{padding:3px 5px 0 0!important;text-align:right;color:#444;}\n.x-grid3-body .x-grid3-row-selected .x-grid3-td-numberer,.x-grid3-body .x-grid3-row-selected .x-grid3-td-checker,.x-grid3-body .x-grid3-row-selected .x-grid3-td-expander{background:transparent url(../images/default/grid/grid3-special-col-sel-bg.gif) repeat-y right;}\n.x-grid3-body .x-grid3-check-col-td .x-grid3-cell-inner{padding:1px 0 0 0!important;}\n.x-grid3-check-col{width:100%;height:16px;background-position:center center;background-repeat:no-repeat;background-color:transparent;background-image:url(../images/default/menu/unchecked.gif);}\n.x-grid3-check-col-on{width:100%;height:16px;background-position:center center;background-repeat:no-repeat;background-color:transparent;background-image:url(../images/default/menu/checked.gif);}\n.x-grid-group,.x-grid-group-body,.x-grid-group-hd{zoom:1;}\n.x-grid-group-hd{border-bottom:2px solid #99bbe8;cursor:pointer;padding-top:6px;}\n.x-grid-group-hd div{background:transparent url(../images/default/grid/group-expand-sprite.gif) no-repeat 3px -47px;padding:4px 4px 4px 17px;color:#3764a0;font:bold 11px tahoma,arial,helvetica,sans-serif;}\n.x-grid-group-collapsed .x-grid-group-hd div{background-position:3px 3px;}\n.x-grid-group-collapsed .x-grid-group-body{display:none;}\n.x-group-by-icon{background-image:url(../images/default/grid/group-by.gif);}\n.x-cols-icon{background-image:url(../images/default/grid/columns.gif);}\n.x-show-groups-icon{background-image:url(../images/default/grid/group-by.gif);}\n.ext-ie .x-grid3 .x-editor .x-form-text{position:relative;top:-1px;}\n.ext-ie .x-props-grid .x-editor .x-form-text{position:static;top:0;}\n.x-grid-empty{padding:10px;color:gray;font:normal 11px tahoma,arial,helvetica,sans-serif;}\n.ext-ie7 .x-grid-panel .x-panel-bbar{position:relative;}\n\n.x-dd-drag-proxy{position:absolute;left:0;top:0;visibility:hidden;z-index:15000;}\n.x-dd-drag-ghost{color:black;font:normal 11px arial,helvetica,sans-serif;-moz-opacity:0.85;opacity:.85;filter:alpha(opacity=85);border-top:1px solid #ddd;border-left:1px solid #ddd;border-right:1px solid #bbb;border-bottom:1px solid #bbb;padding:3px;padding-left:20px;background-color:white;white-space:nowrap;}\n.x-dd-drag-repair .x-dd-drag-ghost{-moz-opacity:0.4;opacity:.4;filter:alpha(opacity=40);border:0 none;padding:0;background-color:transparent;}\n.x-dd-drag-repair .x-dd-drop-icon{visibility:hidden;}\n.x-dd-drop-icon{position:absolute;top:3px;left:3px;display:block;width:16px;height:16px;background-color:transparent;background-position:center;background-repeat:no-repeat;z-index:1;}\n.x-dd-drop-nodrop .x-dd-drop-icon{background-image:url(../images/default/dd/drop-no.gif);}\n.x-dd-drop-ok .x-dd-drop-icon{background-image:url(../images/default/dd/drop-yes.gif);}\n.x-dd-drop-ok-add .x-dd-drop-icon{background-image:url(../images/default/dd/drop-add.gif);}\n.x-view-selector{position:absolute;left:0;top:0;width:0;background:#c3daf9;border:1px dotted #39b;opacity:.5;-moz-opacity:.5;filter:alpha(opacity=50);zoom:1;}\n\n.x-tree .x-panel-body{background-color:#fff;}\n.ext-strict .ext-ie .x-tree .x-panel-bwrap{position:relative;overflow:hidden;}\n.x-tree-icon,.x-tree-ec-icon,.x-tree-elbow-line,.x-tree-elbow,.x-tree-elbow-end,.x-tree-elbow-plus,.x-tree-elbow-minus,.x-tree-elbow-end-plus,.x-tree-elbow-end-minus{border:0 none;height:18px;margin:0;padding:0;vertical-align:top;width:16px;background-repeat:no-repeat;}\n.x-tree-node-collapsed .x-tree-node-icon,.x-tree-node-expanded .x-tree-node-icon,.x-tree-node-leaf .x-tree-node-icon{border:0 none;height:18px;margin:0;padding:0;vertical-align:top;width:16px;background-position:center;background-repeat:no-repeat;}\n.ext-ie .x-tree-node-indent img,.ext-ie .x-tree-node-icon,.ext-ie .x-tree-ec-icon{vertical-align:middle!important;}\n.x-tree-node-expanded .x-tree-node-icon{background-image:url(../images/default/tree/folder-open.gif);}\n.x-tree-node-leaf .x-tree-node-icon{background-image:url(../images/default/tree/leaf.gif);}\n.x-tree-node-collapsed .x-tree-node-icon{background-image:url(../images/default/tree/folder.gif);}\n.ext-ie input.x-tree-node-cb{width:15px;height:15px;}\ninput.x-tree-node-cb{margin-left:1px;}\n.ext-ie input.x-tree-node-cb{margin-left:0;}\n.x-tree-noicon .x-tree-node-icon{width:0;height:0;}\n.x-tree-node-loading .x-tree-node-icon{background-image:url(../images/default/tree/loading.gif)!important;}\n.x-tree-node-loading a span{font-style:italic;color:#444;}\n.ext-ie .x-tree-node-el input{width:15px;height:15px;}\n.x-tree-lines .x-tree-elbow{background-image:url(../images/default/tree/elbow.gif);}\n.x-tree-lines .x-tree-elbow-plus{background-image:url(../images/default/tree/elbow-plus.gif);}\n.x-tree-lines .x-tree-elbow-minus{background-image:url(../images/default/tree/elbow-minus.gif);}\n.x-tree-lines .x-tree-elbow-end{background-image:url(../images/default/tree/elbow-end.gif);}\n.x-tree-lines .x-tree-elbow-end-plus{background-image:url(../images/default/tree/elbow-end-plus.gif);}\n.x-tree-lines .x-tree-elbow-end-minus{background-image:url(../images/default/tree/elbow-end-minus.gif);}\n.x-tree-lines .x-tree-elbow-line{background-image:url(../images/default/tree/elbow-line.gif);}\n.x-tree-no-lines .x-tree-elbow{background:transparent;}\n.x-tree-no-lines .x-tree-elbow-plus{background-image:url(../images/default/tree/elbow-plus-nl.gif);}\n.x-tree-no-lines .x-tree-elbow-minus{background-image:url(../images/default/tree/elbow-minus-nl.gif);}\n.x-tree-no-lines .x-tree-elbow-end{background:transparent;}\n.x-tree-no-lines .x-tree-elbow-end-plus{background-image:url(../images/default/tree/elbow-end-plus-nl.gif);}\n.x-tree-no-lines .x-tree-elbow-end-minus{background-image:url(../images/default/tree/elbow-end-minus-nl.gif);}\n.x-tree-no-lines .x-tree-elbow-line{background:transparent;}\n.x-tree-arrows .x-tree-elbow{background:transparent;}\n.x-tree-arrows .x-tree-elbow-plus{background:transparent url(../images/default/tree/arrows.gif) no-repeat 0 0;}\n.x-tree-arrows .x-tree-elbow-minus{background:transparent url(../images/default/tree/arrows.gif) no-repeat -16px 0;}\n.x-tree-arrows .x-tree-elbow-end{background:transparent;}\n.x-tree-arrows .x-tree-elbow-end-plus{background:transparent url(../images/default/tree/arrows.gif) no-repeat 0 0;}\n.x-tree-arrows .x-tree-elbow-end-minus{background:transparent url(../images/default/tree/arrows.gif) no-repeat -16px 0;}\n.x-tree-arrows .x-tree-elbow-line{background:transparent;}\n.x-tree-arrows .x-tree-ec-over .x-tree-elbow-plus{background-position:-32px 0;}\n.x-tree-arrows .x-tree-ec-over .x-tree-elbow-minus{background-position:-48px 0;}\n.x-tree-arrows .x-tree-ec-over .x-tree-elbow-end-plus{background-position:-32px 0;}\n.x-tree-arrows .x-tree-ec-over .x-tree-elbow-end-minus{background-position:-48px 0;}\n.x-tree-elbow-plus,.x-tree-elbow-minus,.x-tree-elbow-end-plus,.x-tree-elbow-end-minus{cursor:pointer;}\n.ext-ie ul.x-tree-node-ct{font-size:0;line-height:0;zoom:1;}\n.x-tree-node{color:black;font:normal 11px arial,tahoma,helvetica,sans-serif;white-space:nowrap;}\n.x-tree-node-el{line-height:18px;cursor:pointer;}\n.x-tree-node a,.x-dd-drag-ghost a{text-decoration:none;color:black;-khtml-user-select:none;-moz-user-select:none;-kthml-user-focus:normal;-moz-user-focus:normal;-moz-outline:0 none;outline:0 none;}\n.x-tree-node a span,.x-dd-drag-ghost a span{text-decoration:none;color:black;padding:1px 3px 1px 2px;}\n.x-tree-node .x-tree-node-disabled a span{color:gray!important;}\n.x-tree-node .x-tree-node-disabled .x-tree-node-icon{-moz-opacity:0.5;opacity:.5;filter:alpha(opacity=50);}\n.x-tree-node .x-tree-node-inline-icon{background:transparent;}\n.x-tree-node a:hover,.x-dd-drag-ghost a:hover{text-decoration:none;}\n.x-tree-node div.x-tree-drag-insert-below{border-bottom:1px dotted #36c;}\n.x-tree-node div.x-tree-drag-insert-above{border-top:1px dotted #36c;}\n.x-tree-dd-underline .x-tree-node div.x-tree-drag-insert-below{border-bottom:0 none;}\n.x-tree-dd-underline .x-tree-node div.x-tree-drag-insert-above{border-top:0 none;}\n.x-tree-dd-underline .x-tree-node div.x-tree-drag-insert-below a{border-bottom:2px solid #36c;}\n.x-tree-dd-underline .x-tree-node div.x-tree-drag-insert-above a{border-top:2px solid #36c;}\n.x-tree-node .x-tree-drag-append a span{background:#ddd;border:1px dotted gray;}\n.x-tree-node .x-tree-node-over{background-color:#eee;}\n.x-tree-node .x-tree-selected{background-color:#d9e8fb;}\n.x-dd-drag-ghost .x-tree-node-indent,.x-dd-drag-ghost .x-tree-ec-icon{display:none!important;}\n.x-tree-drop-ok-append .x-dd-drop-icon{background-image:url(../images/default/tree/drop-add.gif);}\n.x-tree-drop-ok-above .x-dd-drop-icon{background-image:url(../images/default/tree/drop-over.gif);}\n.x-tree-drop-ok-below .x-dd-drop-icon{background-image:url(../images/default/tree/drop-under.gif);}\n.x-tree-drop-ok-between .x-dd-drop-icon{background-image:url(../images/default/tree/drop-between.gif);}\n\n.x-date-picker{border:1px solid #1b376c;border-top:0 none;background:#fff;position:relative;}\n.x-date-picker a{-moz-outline:0 none;outline:0 none;}\n.x-date-inner,.x-date-inner td,.x-date-inner th{border-collapse:separate;}\n.x-date-middle,.x-date-left,.x-date-right{background:url(../images/default/shared/hd-sprite.gif) repeat-x 0 -83px;color:#FFF;font:bold 11px \"sans serif\",tahoma,verdana,helvetica;overflow:hidden;}\n.x-date-middle .x-btn-left,.x-date-middle .x-btn-center,.x-date-middle .x-btn-right{background:transparent!important;vertical-align:middle;}\n.x-date-middle .x-btn .x-btn-text{color:#fff;}\n.x-date-middle .x-btn-with-menu .x-btn-center em{background:transparent url(../images/default/toolbar/btn-arrow-light.gif) no-repeat right 0;}\n.x-date-right,.x-date-left{width:18px;}\n.x-date-right{text-align:right;}\n.x-date-middle{padding-top:2px;padding-bottom:2px;}\n.x-date-right a,.x-date-left a{display:block;width:16px;height:16px;background-position:center;background-repeat:no-repeat;cursor:pointer;-moz-opacity:0.6;opacity:.6;filter:alpha(opacity=60);}\n.x-date-right a:hover,.x-date-left a:hover{-moz-opacity:1;opacity:1;filter:alpha(opacity=100);}\n.x-date-right a{background-image:url(../images/default/shared/right-btn.gif);margin-right:2px;text-decoration:none!important;}\n.x-date-left a{background-image:url(../images/default/shared/left-btn.gif);margin-left:2px;text-decoration:none!important;}\ntable.x-date-inner{width:100%;table-layout:fixed;}\n.x-date-inner th{width:25px;}\n.x-date-inner th{background:#dfecfb url(../images/default/shared/glass-bg.gif) repeat-x left top;text-align:right!important;border-bottom:1px solid #a3bad9;font:normal 10px arial,helvetica,tahoma,sans-serif;color:#233d6d;cursor:default;padding:0;border-collapse:separate;}\n.x-date-inner th span{display:block;padding:2px;padding-right:7px;}\n.x-date-inner td{border:1px solid #fff;text-align:right;padding:0;}\n.x-date-inner a{padding:2px 5px;display:block;font:normal 11px arial,helvetica,tahoma,sans-serif;text-decoration:none;color:black;text-align:right;zoom:1;}\n.x-date-inner .x-date-active{cursor:pointer;color:black;}\n.x-date-inner .x-date-selected a{background:#dfecfb url(../images/default/shared/glass-bg.gif) repeat-x left top;border:1px solid #8db2e3;padding:1px 4px;}\n.x-date-inner .x-date-today a{border:1px solid darkred;padding:1px 4px;}\n.x-date-inner .x-date-selected span{font-weight:bold;}\n.x-date-inner .x-date-prevday a,.x-date-inner .x-date-nextday a{color:#aaa;text-decoration:none!important;}\n.x-date-bottom{padding:4px;border-top:1px solid #a3bad9;background:#dfecfb url(../images/default/shared/glass-bg.gif) repeat-x left top;}\n.x-date-inner a:hover,.x-date-inner .x-date-disabled a:hover{text-decoration:none!important;color:black;background:#ddecfe;}\n.x-date-inner .x-date-disabled a{cursor:default;background:#eee;color:#bbb;}\n.x-date-mmenu{background:#eee!important;}\n.x-date-mmenu .x-menu-item{font-size:10px;padding:1px 24px 1px 4px;white-space:nowrap;color:#000;}\n.x-date-mmenu .x-menu-item .x-menu-item-icon{width:10px;height:10px;margin-right:5px;background-position:center -4px!important;}\n.x-date-mp{position:absolute;left:0;top:0;background:white;display:none;}\n.x-date-mp td{padding:2px;font:normal 11px arial,helvetica,tahoma,sans-serif;}\ntd.x-date-mp-month,td.x-date-mp-year,td.x-date-mp-ybtn{border:0 none;text-align:center;vertical-align:middle;width:25%;}\n.x-date-mp-ok{margin-right:3px;}\n.x-date-mp-btns button{text-decoration:none;text-align:center;text-decoration:none!important;background:#083772;color:white;border:1px solid;border-color:#36c #005 #005 #36c;padding:1px 3px 1px;font:normal 11px arial,helvetica,tahoma,sans-serif;cursor:pointer;}\n.x-date-mp-btns{background:#dfecfb url(../images/default/shared/glass-bg.gif) repeat-x left top;}\n.x-date-mp-btns td{border-top:1px solid #c5d2df;text-align:center;}\ntd.x-date-mp-month a,td.x-date-mp-year a{display:block;padding:2px 4px;text-decoration:none;text-align:center;color:#15428b;}\ntd.x-date-mp-month a:hover,td.x-date-mp-year a:hover{color:#15428b;text-decoration:none;cursor:pointer;background:#ddecfe;}\ntd.x-date-mp-sel a{padding:1px 3px;background:#dfecfb url(../images/default/shared/glass-bg.gif) repeat-x left top;border:1px solid #8db2e3;}\n.x-date-mp-ybtn a{overflow:hidden;width:15px;height:15px;cursor:pointer;background:transparent url(../images/default/panel/tool-sprites.gif) no-repeat;display:block;margin:0 auto;}\n.x-date-mp-ybtn a.x-date-mp-next{background-position:0 -120px;}\n.x-date-mp-ybtn a.x-date-mp-next:hover{background-position:-15px -120px;}\n.x-date-mp-ybtn a.x-date-mp-prev{background-position:0 -105px;}\n.x-date-mp-ybtn a.x-date-mp-prev:hover{background-position:-15px -105px;}\n.x-date-mp-ybtn{text-align:center;}\ntd.x-date-mp-sep{border-right:1px solid #c5d2df;}\n\n.x-tip{position:absolute;top:0;left:0;visibility:hidden;z-index:20000;border:0 none;}\n.x-tip .x-tip-close{background-image:url(../images/default/qtip/close.gif);height:15px;float:right;width:15px;margin:0 0 2px 2px;cursor:pointer;display:none;}\n.x-tip .x-tip-tc{background:transparent url(../images/default/qtip/tip-sprite.gif) no-repeat 0 -62px;padding-top:3px;overflow:hidden;zoom:1;}\n.x-tip .x-tip-tl{background:transparent url(../images/default/qtip/tip-sprite.gif) no-repeat 0 0;padding-left:6px;overflow:hidden;zoom:1;}\n.x-tip .x-tip-tr{background:transparent url(../images/default/qtip/tip-sprite.gif) no-repeat right 0;padding-right:6px;overflow:hidden;zoom:1;}\n.x-tip .x-tip-bc{background:transparent url(../images/default/qtip/tip-sprite.gif) no-repeat 0 -121px;height:3px;overflow:hidden;}\n.x-tip .x-tip-bl{background:transparent url(../images/default/qtip/tip-sprite.gif) no-repeat 0 -59px;padding-left:6px;zoom:1;}\n.x-tip .x-tip-br{background:transparent url(../images/default/qtip/tip-sprite.gif) no-repeat right -59px;padding-right:6px;zoom:1;}\n.x-tip .x-tip-mc{border:0 none;font:normal 11px tahoma,arial,helvetica,sans-serif;}\n.x-tip .x-tip-ml{background:#fff url(../images/default/qtip/tip-sprite.gif) no-repeat 0 -124px;padding-left:6px;zoom:1;}\n.x-tip .x-tip-mr{background:transparent url(../images/default/qtip/tip-sprite.gif) no-repeat right -124px;padding-right:6px;zoom:1;}\n.ext-ie .x-tip .x-tip-header,.ext-ie .x-tip .x-tip-tc{font-size:0;line-height:0;}\n.x-tip .x-tip-header-text{font:bold 11px tahoma,arial,helvetica,sans-serif;padding:0;margin:0 0 2px 0;color:#444;}\n.x-tip .x-tip-body{font:normal 11px tahoma,arial,helvetica,sans-serif;margin:0!important;line-height:14px;color:#444;padding:0;}\n.x-tip .x-tip-body .loading-indicator{margin:0;}\n.x-tip-draggable .x-tip-header,.x-tip-draggable .x-tip-header-text{cursor:move;}\n.x-form-invalid-tip .x-tip-tc{background:url(../images/default/form/error-tip-corners.gif) repeat-x 0 -12px;padding-top:6px;}\n.x-form-invalid-tip .x-tip-tl{background-image:url(../images/default/form/error-tip-corners.gif);}\n.x-form-invalid-tip .x-tip-tr{background-image:url(../images/default/form/error-tip-corners.gif);}\n.x-form-invalid-tip .x-tip-bc{background:url(../images/default/form/error-tip-corners.gif) repeat-x 0 -18px;height:6px;}\n.x-form-invalid-tip .x-tip-bl{background:url(../images/default/form/error-tip-corners.gif) no-repeat 0 -6px;}\n.x-form-invalid-tip .x-tip-br{background:url(../images/default/form/error-tip-corners.gif) no-repeat right -6px;}\n.x-form-invalid-tip .x-tip-ml{background-image:url(../images/default/form/error-tip-corners.gif);}\n.x-form-invalid-tip .x-tip-mr{background-image:url(../images/default/form/error-tip-corners.gif);}\n.x-form-invalid-tip .x-tip-body{padding:2px;}\n.x-form-invalid-tip .x-tip-body{padding-left:24px;background:transparent url(../images/default/form/exclamation.gif) no-repeat 2px 2px;}\n\n.x-menu{border:1px solid #718bb7;z-index:15000;zoom:1;background:#f0f0f0 url(../images/default/menu/menu.gif) repeat-y;padding:2px;}\n.x-menu a{text-decoration:none!important;}\n.ext-ie .x-menu{zoom:1;overflow:hidden;}\n.x-menu-list{background:transparent;border:0 none;}\n.x-menu li{line-height:100%;}\n.x-menu li.x-menu-sep-li{font-size:1px;line-height:1px;}\n.x-menu-list-item{font:normal 11px tahoma,arial,sans-serif;white-space:nowrap;-moz-user-select:none;-khtml-user-select:none;display:block;padding:1px;}\n.x-menu-item-arrow{background:transparent url(../images/default/menu/menu-parent.gif) no-repeat right;}\n.x-menu-sep{display:block;font-size:1px;line-height:1px;margin:2px 3px;background-color:#e0e0e0;border-bottom:1px solid #fff;overflow:hidden;}\n.x-menu-focus{position:absolute;left:0;top:-5px;width:0;height:0;line-height:1px;}\n.x-menu a.x-menu-item{display:block;line-height:16px;padding:3px 21px 3px 3px;white-space:nowrap;text-decoration:none;color:#222;-moz-outline:0 none;outline:0 none;cursor:pointer;}\n.x-menu-item-active{background:#ebf3fd url(../images/default/menu/item-over.gif) repeat-x left bottom;border:1px solid #aaccf6;padding:0;}\n.x-menu-item-active a.x-menu-item{color:#233d6d;}\n.x-menu-item-icon{border:0 none;height:16px;padding:0;vertical-align:top;width:16px;margin:0 8px 0 0;background-position:center;}\n.x-menu-check-item .x-menu-item-icon{background:transparent url(../images/default/menu/unchecked.gif) no-repeat center;}\n.x-menu-item-checked .x-menu-item-icon{background-image:url(../images/default/menu/checked.gif);}\n.x-menu-group-item .x-menu-item-icon{background:transparent;}\n.x-menu-item-checked .x-menu-group-item .x-menu-item-icon{background:transparent url(../images/default/menu/group-checked.gif) no-repeat center;}\n.x-menu-plain{background:#fff!important;}\n.x-menu-date-item{padding:0;}\n.x-menu .x-color-palette,.x-menu .x-date-picker{margin-left:26px;margin-right:4px;}\n.x-menu .x-date-picker{border:1px solid #a3bad9;margin-top:2px;margin-bottom:2px;}\n.x-menu-plain .x-color-palette,.x-menu-plain .x-date-picker{margin:0;border:0 none;}\n.x-date-menu{padding:0!important;}\n.x-cycle-menu .x-menu-item-checked{border:1px dotted #a3bae9!important;background:#DFE8F6;padding:0;}\n\n .x-box-tl{background:transparent url(../images/default/box/corners.gif) no-repeat 0 0;zoom:1;}\n.x-box-tc{height:8px;background:transparent url(../images/default/box/tb.gif) repeat-x 0 0;overflow:hidden;}\n.x-box-tr{background:transparent url(../images/default/box/corners.gif) no-repeat right -8px;}\n.x-box-ml{background:transparent url(../images/default/box/l.gif) repeat-y 0;padding-left:4px;overflow:hidden;zoom:1;}\n.x-box-mc{background:#eee url(../images/default/box/tb.gif) repeat-x 0 -16px;padding:4px 10px;font-family:\"Myriad Pro\",\"Myriad Web\",\"Tahoma\",\"Helvetica\",\"Arial\",sans-serif;color:#393939;font-size:12px;}\n.x-box-mc h3{font-size:14px;font-weight:bold;margin:0 0 4px 0;zoom:1;}\n.x-box-mr{background:transparent url(../images/default/box/r.gif) repeat-y right;padding-right:4px;overflow:hidden;}\n.x-box-bl{background:transparent url(../images/default/box/corners.gif) no-repeat 0 -16px;zoom:1;}\n.x-box-bc{background:transparent url(../images/default/box/tb.gif) repeat-x 0 -8px;height:8px;overflow:hidden;}\n.x-box-br{background:transparent url(../images/default/box/corners.gif) no-repeat right -24px;}\n.x-box-tl,.x-box-bl{padding-left:8px;overflow:hidden;}\n.x-box-tr,.x-box-br{padding-right:8px;overflow:hidden;}\n.x-box-blue .x-box-bl,.x-box-blue .x-box-br,.x-box-blue .x-box-tl,.x-box-blue .x-box-tr{background-image:url(../images/default/box/corners-blue.gif);}\n.x-box-blue .x-box-bc,.x-box-blue .x-box-mc,.x-box-blue .x-box-tc{background-image:url(../images/default/box/tb-blue.gif);}\n.x-box-blue .x-box-mc{background-color:#c3daf9;}\n.x-box-blue .x-box-mc h3{color:#17385b;}\n.x-box-blue .x-box-ml{background-image:url(../images/default/box/l-blue.gif);}\n.x-box-blue .x-box-mr{background-image:url(../images/default/box/r-blue.gif);}\n\n#x-debug-browser .x-tree .x-tree-node a span{color:#222297;font-size:11px;padding-top:2px;font-family:\"monotype\",\"courier new\",sans-serif;line-height:18px;}\n#x-debug-browser .x-tree a i{color:#FF4545;font-style:normal;}\n#x-debug-browser .x-tree a em{color:#999;}\n#x-debug-browser .x-tree .x-tree-node .x-tree-selected a span{background:#c3daf9;}\n#x-debug-browser .x-tool-toggle{background-position:0 -75px;}\n#x-debug-browser .x-tool-toggle-over{background-position:-15px -75px;}\n#x-debug-browser.x-panel-collapsed .x-tool-toggle{background-position:0 -60px;}\n#x-debug-browser.x-panel-collapsed .x-tool-toggle-over{background-position:-15px -60px;}\n\n.x-combo-list{border:1px solid #98c0f4;background:#ddecfe;zoom:1;overflow:hidden;}\n.x-combo-list-inner{overflow:auto;background:white;position:relative;zoom:1;overflow-x:hidden;}\n.x-combo-list-hd{font:bold 11px tahoma,arial,helvetica,sans-serif;color:#15428b;background-image:url(../images/default/layout/panel-title-light-bg.gif);border-bottom:1px solid #98c0f4;padding:3px;}\n.x-resizable-pinned .x-combo-list-inner{border-bottom:1px solid #98c0f4;}\n.x-combo-list-item{font:normal 12px tahoma,arial,helvetica,sans-serif;padding:2px;border:1px solid #fff;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;}\n.x-combo-list .x-combo-selected{border:1px dotted #a3bae9!important;background:#DFE8F6;cursor:pointer;}\n.x-combo-noedit{cursor:pointer;}\n.x-combo-list .x-toolbar{border-top:1px solid #98c0f4;border-bottom:0 none;}\n.x-combo-list-small .x-combo-list-item{font:normal 11px tahoma,arial,helvetica,sans-serif;}\n\n.x-panel{border-style:solid;border-color:#99bbe8;border-width:0;}\n.x-panel-header{overflow:hidden;zoom:1;color:#15428b;font:bold 11px tahoma,arial,verdana,sans-serif;padding:5px 3px 4px 5px;border:1px solid #99bbe8;line-height:15px;background:transparent url(../images/default/panel/white-top-bottom.gif) repeat-x 0 -1px;}\n.x-panel-body{border:1px solid #99bbe8;border-top:0 none;overflow:hidden;background:white;position:relative;}\n.x-panel-bbar .x-toolbar{border:1px solid #99bbe8;border-top:0 none;overflow:hidden;padding:2px;}\n.x-panel-tbar .x-toolbar{border:1px solid #99bbe8;border-top:0 none;overflow:hidden;padding:2px;}\n.x-panel-tbar-noheader .x-toolbar,.x-panel-mc .x-panel-tbar .x-toolbar{border-top:1px solid #99bbe8;border-bottom:0 none;}\n.x-panel-body-noheader,.x-panel-mc .x-panel-body{border-top:1px solid #99bbe8;}\n.x-panel-header{overflow:hidden;zoom:1;}\n.x-panel-tl .x-panel-header{color:#15428b;font:bold 11px tahoma,arial,verdana,sans-serif;padding:5px 0 4px 0;border:0 none;background:transparent;}\n.x-panel-tl .x-panel-icon,.x-window-tl .x-panel-icon{padding-left:20px!important;background-repeat:no-repeat;background-position:0 4px;zoom:1;}\n.x-panel-inline-icon{width:16px;height:16px;background-repeat:no-repeat;background-position:0 0;vertical-align:middle;margin-right:4px;margin-top:-1px;margin-bottom:-1px;}\n.x-panel-tc{background:transparent url(../images/default/panel/top-bottom.gif) repeat-x 0 0;overflow:hidden;}\n.ext-strict .ext-ie7 .x-panel-tc{overflow:visible;}\n.x-panel-tl{background:transparent url(../images/default/panel/corners-sprite.gif) no-repeat 0 0;padding-left:6px;zoom:1;border-bottom:1px solid #99bbe8;}\n.x-panel-tr{background:transparent url(../images/default/panel/corners-sprite.gif) no-repeat right 0;zoom:1;padding-right:6px;}\n.x-panel-bc{background:transparent url(../images/default/panel/top-bottom.gif) repeat-x 0 bottom;zoom:1;}\n.x-panel-bc .x-panel-footer{zoom:1;}\n.x-panel-bl{background:transparent url(../images/default/panel/corners-sprite.gif) no-repeat 0 bottom;padding-left:6px;zoom:1;}\n.x-panel-br{background:transparent url(../images/default/panel/corners-sprite.gif) no-repeat right bottom;padding-right:6px;zoom:1;}\n.x-panel-mc{border:0 none;padding:0;margin:0;font:normal 11px tahoma,arial,helvetica,sans-serif;padding-top:6px;background:#dfe8f6;}\n.x-panel-mc .x-panel-body{background:transparent;border:0 none;}\n.x-panel-ml{background:#fff url(../images/default/panel/left-right.gif) repeat-y 0 0;padding-left:6px;zoom:1;}\n.x-panel-mr{background:transparent url(../images/default/panel/left-right.gif) repeat-y right 0;padding-right:6px;zoom:1;}\n.x-panel-bc .x-panel-footer{padding-bottom:6px;}\n.x-panel-nofooter .x-panel-bc{height:6px;font-size:0;line-height:0;}\n.x-panel-bwrap{overflow:hidden;zoom:1;}\n.x-panel-body{overflow:hidden;zoom:1;}\n.x-panel-collapsed .x-resizable-handle{display:none;}\n.ext-gecko .x-panel-animated div{overflow:hidden!important;}\n.x-plain-body{overflow:hidden;}\n.x-plain-bbar .x-toolbar{overflow:hidden;padding:2px;}\n.x-plain-tbar .x-toolbar{overflow:hidden;padding:2px;}\n.x-plain-bwrap{overflow:hidden;zoom:1;}\n.x-plain{overflow:hidden;}\n.x-tool{overflow:hidden;width:15px;height:15px;float:right;cursor:pointer;background:transparent url(../images/default/panel/tool-sprites.gif) no-repeat;margin-left:2px;}\n.x-tool-toggle{background-position:0 -60px;}\n.x-tool-toggle-over{background-position:-15px -60px;}\n.x-panel-collapsed .x-tool-toggle{background-position:0 -75px;}\n.x-panel-collapsed .x-tool-toggle-over{background-position:-15px -75px;}\n.x-tool-close{background-position:0 -0;}\n.x-tool-close-over{background-position:-15px 0;}\n.x-tool-minimize{background-position:0 -15px;}\n.x-tool-minimize-over{background-position:-15px -15px;}\n.x-tool-maximize{background-position:0 -30px;}\n.x-tool-maximize-over{background-position:-15px -30px;}\n.x-tool-restore{background-position:0 -45px;}\n.x-tool-restore-over{background-position:-15px -45px;}\n.x-tool-gear{background-position:0 -90px;}\n.x-tool-gear-over{background-position:-15px -90px;}\n.x-tool-pin{background-position:0 -135px;}\n.x-tool-pin-over{background-position:-15px -135px;}\n.x-tool-unpin{background-position:0 -150px;}\n.x-tool-unpin-over{background-position:-15px -150px;}\n.x-tool-right{background-position:0 -165px;}\n.x-tool-right-over{background-position:-15px -165px;}\n.x-tool-left{background-position:0 -180px;}\n.x-tool-left-over{background-position:-15px -180px;}\n.x-tool-up{background-position:0 -210px;}\n.x-tool-up-over{background-position:-15px -210px;}\n.x-tool-down{background-position:0 -195px;}\n.x-tool-down-over{background-position:-15px -195px;}\n.x-tool-refresh{background-position:0 -225px;}\n.x-tool-refresh-over{background-position:-15px -225px;}\n.x-tool-minus{background-position:0 -255px;}\n.x-tool-minus-over{background-position:-15px -255px;}\n.x-tool-plus{background-position:0 -240px;}\n.x-tool-plus-over{background-position:-15px -240px;}\n.x-tool-search{background-position:0 -270px;}\n.x-tool-search-over{background-position:-15px -270px;}\n.x-tool-save{background-position:0 -285px;}\n.x-tool-save-over{background-position:-15px -285px;}\n.x-tool-help{background-position:0 -300px;}\n.x-tool-help-over{background-position:-15px -300px;}\n.x-tool-print{background-position:0 -315px;}\n.x-tool-print-over{background-position:-15px -315px;}\n.x-panel-ghost{background:#cbddf3;z-index:12000;overflow:hidden;position:absolute;left:0;top:0;opacity:.65;-moz-opacity:.65;filter:alpha(opacity=65);}\n.x-panel-ghost ul{margin:0;padding:0;overflow:hidden;font-size:0;line-height:0;border:1px solid #99bbe8;border-top:0 none;display:block;}\n.x-panel-ghost *{cursor:move!important;}\n.x-panel-dd-spacer{border:2px dashed #99bbe8;}\n.x-panel-btns-ct{padding:5px;}\n.x-panel-btns-ct .x-btn{float:right;clear:none;}\n.x-panel-btns-ct .x-panel-btns td{border:0;padding:0;}\n.x-panel-btns-ct .x-panel-btns-right table{float:right;clear:none;}\n.x-panel-btns-ct .x-panel-btns-left table{float:left;clear:none;}\n.x-panel-btns-ct .x-panel-btns-center{text-align:center;}\n.x-panel-btns-ct .x-panel-btns-center table{margin:0 auto;}\n.x-panel-btns-ct table td.x-panel-btn-td{padding:3px;}\n.x-panel-btns-ct .x-btn-focus .x-btn-left{background-position:0 -147px;}\n.x-panel-btns-ct .x-btn-focus .x-btn-right{background-position:0 -168px;}\n.x-panel-btns-ct .x-btn-focus .x-btn-center{background-position:0 -189px;}\n.x-panel-btns-ct .x-btn-over .x-btn-left{background-position:0 -63px;}\n.x-panel-btns-ct .x-btn-over .x-btn-right{background-position:0 -84px;}\n.x-panel-btns-ct .x-btn-over .x-btn-center{background-position:0 -105px;}\n.x-panel-btns-ct .x-btn-click .x-btn-center{background-position:0 -126px;}\n.x-panel-btns-ct .x-btn-click .x-btn-right{background-position:0 -84px;}\n.x-panel-btns-ct .x-btn-click .x-btn-left{background-position:0 -63px;}\n\n.x-window{zoom:1;}\n.x-window .x-resizable-handle{opacity:0;-moz-opacity:0;filter:alpha(opacity=0);}\n.x-window-proxy{background:#C7DFFC;border:1px solid #99bbe8;z-index:12000;overflow:hidden;position:absolute;left:0;top:0;display:none;opacity:.5;-moz-opacity:.5;filter:alpha(opacity=50);}\n.x-window-header{overflow:hidden;zoom:1;}\n.x-window-bwrap{z-index:1;position:relative;zoom:1;}\n.x-window-tl .x-window-header{color:#15428b;font:bold 11px tahoma,arial,verdana,sans-serif;padding:5px 0 4px 0;}\n.x-window-header-text{cursor:pointer;}\n.x-window-tc{background:transparent url(../images/default/window/top-bottom.png) repeat-x 0 0;overflow:hidden;zoom:1;}\n.x-window-tl{background:transparent url(../images/default/window/left-corners.png) no-repeat 0 0;padding-left:6px;zoom:1;z-index:1;position:relative;}\n.x-window-tr{background:transparent url(../images/default/window/right-corners.png) no-repeat right 0;padding-right:6px;}\n.x-window-bc{background:transparent url(../images/default/window/top-bottom.png) repeat-x 0 bottom;zoom:1;}\n.x-window-bc .x-window-footer{padding-bottom:6px;zoom:1;font-size:0;line-height:0;}\n.x-window-bl{background:transparent url(../images/default/window/left-corners.png) no-repeat 0 bottom;padding-left:6px;zoom:1;}\n.x-window-br{background:transparent url(../images/default/window/right-corners.png) no-repeat right bottom;padding-right:6px;zoom:1;}\n.x-window-mc{border:1px solid #99bbe8;padding:0;margin:0;font:normal 11px tahoma,arial,helvetica,sans-serif;background:#dfe8f6;}\n.x-window-ml{background:transparent url(../images/default/window/left-right.png) repeat-y 0 0;padding-left:6px;zoom:1;}\n.x-window-mr{background:transparent url(../images/default/window/left-right.png) repeat-y right 0;padding-right:6px;zoom:1;}\n.x-panel-nofooter .x-window-bc{height:6px;}\n.x-window-body{overflow:hidden;}\n.x-window-bwrap{overflow:hidden;}\n.x-window-maximized .x-window-bl,.x-window-maximized .x-window-br,.x-window-maximized .x-window-ml,.x-window-maximized .x-window-mr,.x-window-maximized .x-window-tl,.x-window-maximized .x-window-tr{padding:0;}\n.x-window-maximized .x-window-footer{padding-bottom:0;}\n.x-window-maximized .x-window-tc{padding-left:3px;padding-right:3px;background-color:white;}\n.x-window-maximized .x-window-mc{border-left:0 none;border-right:0 none;}\n.x-window-tbar .x-toolbar,.x-window-bbar .x-toolbar{border-left:0 none;border-right:0 none;}\n.x-window-bbar .x-toolbar{border-top:1px solid #99bbe8;border-bottom:0 none;}\n.x-window-draggable,.x-window-draggable .x-window-header-text{cursor:move;}\n.x-window-maximized .x-window-draggable,.x-window-maximized .x-window-draggable .x-window-header-text{cursor:default;}\n.x-window-body{background:transparent;}\n.x-panel-ghost .x-window-tl{border-bottom:1px solid #99bbe8;}\n.x-panel-collapsed .x-window-tl{border-bottom:1px solid #84a0c4;}\n.x-window-maximized-ct{overflow:hidden;}\n.x-window-maximized .x-resizable-handle{display:none;}\n.x-window-sizing-ghost ul{border:0 none!important;}\n.x-dlg-focus{-moz-outline:0 none;outline:0 none;width:0;height:0;overflow:hidden;position:absolute;top:0;left:0;}\n.x-dlg-mask{z-index:10000;display:none;position:absolute;top:0;left:0;-moz-opacity:0.5;opacity:.50;filter:alpha(opacity=50);background-color:#CCC;}\nbody.ext-ie6.x-body-masked select{visibility:hidden;}\nbody.ext-ie6.x-body-masked .x-window select{visibility:visible;}\n.x-window-plain .x-window-mc{background:#CAD9EC;border-right:1px solid #DFE8F6;border-bottom:1px solid #DFE8F6;border-top:1px solid #a3bae9;border-left:1px solid #a3bae9;}\n.x-window-plain .x-window-body{border-left:1px solid #DFE8F6;border-top:1px solid #DFE8F6;border-bottom:1px solid #a3bae9;border-right:1px solid #a3bae9;background:transparent!important;}\nbody.x-body-masked .x-window-plain .x-window-mc{background:#C7D6E9;}\n\n.x-html-editor-wrap{border:1px solid #a9bfd3;background:white;}\n.x-html-editor-tb .x-btn-text{background:transparent url(../images/default/editor/tb-sprite.gif) no-repeat;}\n.x-html-editor-tb .x-edit-bold .x-btn-text{background-position:0 0;}\n.x-html-editor-tb .x-edit-italic .x-btn-text{background-position:-16px 0;}\n.x-html-editor-tb .x-edit-underline .x-btn-text{background-position:-32px 0;}\n.x-html-editor-tb .x-edit-forecolor .x-btn-text{background-position:-160px 0;}\n.x-html-editor-tb .x-edit-backcolor .x-btn-text{background-position:-176px 0;}\n.x-html-editor-tb .x-edit-justifyleft .x-btn-text{background-position:-112px 0;}\n.x-html-editor-tb .x-edit-justifycenter .x-btn-text{background-position:-128px 0;}\n.x-html-editor-tb .x-edit-justifyright .x-btn-text{background-position:-144px 0;}\n.x-html-editor-tb .x-edit-insertorderedlist .x-btn-text{background-position:-80px 0;}\n.x-html-editor-tb .x-edit-insertunorderedlist .x-btn-text{background-position:-96px 0;}\n.x-html-editor-tb .x-edit-increasefontsize .x-btn-text{background-position:-48px 0;}\n.x-html-editor-tb .x-edit-decreasefontsize .x-btn-text{background-position:-64px 0;}\n.x-html-editor-tb .x-edit-sourceedit .x-btn-text{background-position:-192px 0;}\n.x-html-editor-tb .x-edit-createlink .x-btn-text{background-position:-208px 0;}\n.x-html-editor-tip .x-tip-bd .x-tip-bd-inner{padding:5px;padding-bottom:1px;}\n.x-html-editor-tb .x-toolbar{position:static!important;}\n\n.x-panel-noborder .x-panel-body-noborder{border-width:0;}\n.x-panel-noborder .x-panel-header-noborder{border-width:0;border-bottom:1px solid #99bbe8;}\n.x-panel-noborder .x-panel-tbar-noborder .x-toolbar{border-width:0;border-bottom:1px solid #99bbe8;}\n.x-panel-noborder .x-panel-bbar-noborder .x-toolbar{border-width:0;border-top:1px solid #99bbe8;}\n.x-window-noborder .x-window-mc{border-width:0;}\n.x-window-plain .x-window-body-noborder{border-width:0;}\n.x-tab-panel-noborder .x-tab-panel-body-noborder{border-width:0;}\n.x-tab-panel-noborder .x-tab-panel-header-noborder{border-top-width:0;border-left-width:0;border-right-width:0;}\n.x-tab-panel-noborder .x-tab-panel-footer-noborder{border-bottom-width:0;border-left-width:0;border-right-width:0;}\n.x-tab-panel-bbar-noborder .x-toolbar{border-width:0;border-top:1px solid #99bbe8;}\n.x-tab-panel-tbar-noborder .x-toolbar{border-width:0;border-bottom:1px solid #99bbe8;}\n\n.x-border-layout-ct{background:#dfe8f6;}\n.x-border-panel{position:absolute;left:0;top:0;}\n.x-tool-collapse-south{background-position:0 -195px;}\n.x-tool-collapse-south-over{background-position:-15px -195px;}\n.x-tool-collapse-north{background-position:0 -210px;}\n.x-tool-collapse-north-over{background-position:-15px -210px;}\n.x-tool-collapse-west{background-position:0 -180px;}\n.x-tool-collapse-west-over{background-position:-15px -180px;}\n.x-tool-collapse-east{background-position:0 -165px;}\n.x-tool-collapse-east-over{background-position:-15px -165px;}\n.x-tool-expand-south{background-position:0 -210px;}\n.x-tool-expand-south-over{background-position:-15px -210px;}\n.x-tool-expand-north{background-position:0 -195px;}\n.x-tool-expand-north-over{background-position:-15px -195px;}\n.x-tool-expand-west{background-position:0 -165px;}\n.x-tool-expand-west-over{background-position:-15px -165px;}\n.x-tool-expand-east{background-position:0 -180px;}\n.x-tool-expand-east-over{background-position:-15px -180px;}\n.x-tool-expand-north,.x-tool-expand-south{float:right;margin:3px;}\n.x-tool-expand-east,.x-tool-expand-west{float:none;margin:3px auto;}\n.x-accordion-hd .x-tool-toggle{background-position:0 -255px;}\n.x-accordion-hd .x-tool-toggle-over{background-position:-15px -255px;}\n.x-panel-collapsed .x-accordion-hd .x-tool-toggle{background-position:0 -240px;}\n.x-panel-collapsed .x-accordion-hd .x-tool-toggle-over{background-position:-15px -240px;}\n.x-accordion-hd{color:#222;padding-top:4px;padding-bottom:3px;border-top:0 none;font-weight:normal;background:transparent url(../images/default/panel/light-hd.gif) repeat-x 0 -9px;}\n.x-layout-collapsed{position:absolute;left:-10000px;top:-10000px;visibility:hidden;background-color:#d2e0f2;width:20px;height:20px;overflow:hidden;border:1px solid #98c0f4;z-index:20;}\n.ext-border-box .x-layout-collapsed{width:22px;height:22px;}\n.x-layout-collapsed-over{cursor:pointer;background-color:#d9e8fb;}\n.x-layout-collapsed-west .x-layout-collapsed-tools,.x-layout-collapsed-east .x-layout-collapsed-tools{position:absolute;top:0;left:0;width:20px;height:20px;}\n.x-layout-split{position:absolute;height:5px;width:5px;line-height:1px;font-size:1px;z-index:3;background-color:transparent;}\n.x-layout-split-h{background-image:url(../images/default/s.gif);background-position:left;}\n.x-layout-split-v{background-image:url(../images/default/s.gif);background-position:top;}\n.x-column-layout-ct{overflow:hidden;zoom:1;}\n.x-column{float:left;padding:0;margin:0;overflow:hidden;zoom:1;}\n.x-layout-mini{position:absolute;top:0;left:0;display:block;width:5px;height:35px;cursor:pointer;opacity:.5;-moz-opacity:.5;filter:alpha(opacity=50);}\n.x-layout-mini-over,.x-layout-collapsed-over .x-layout-mini{opacity:1;-moz-opacity:1;filter:none;}\n.x-layout-split-west .x-layout-mini{top:48%;background-image:url(../images/default/layout/mini-left.gif);}\n.x-layout-split-east .x-layout-mini{top:48%;background-image:url(../images/default/layout/mini-right.gif);}\n.x-layout-split-north .x-layout-mini{left:48%;height:5px;width:35px;background-image:url(../images/default/layout/mini-top.gif);}\n.x-layout-split-south .x-layout-mini{left:48%;height:5px;width:35px;background-image:url(../images/default/layout/mini-bottom.gif);}\n.x-layout-cmini-west .x-layout-mini{top:48%;background-image:url(../images/default/layout/mini-right.gif);}\n.x-layout-cmini-east .x-layout-mini{top:48%;background-image:url(../images/default/layout/mini-left.gif);}\n.x-layout-cmini-north .x-layout-mini{left:48%;height:5px;width:35px;background-image:url(../images/default/layout/mini-bottom.gif);}\n.x-layout-cmini-south .x-layout-mini{left:48%;height:5px;width:35px;background-image:url(../images/default/layout/mini-top.gif);}\n.x-layout-cmini-west,.x-layout-cmini-east{border:0 none;width:5px!important;padding:0;background:transparent;}\n.x-layout-cmini-north,.x-layout-cmini-south{border:0 none;height:5px!important;padding:0;background:transparent;}\n.x-viewport,.x-viewport body{margin:0;padding:0;border:0 none;overflow:hidden;height:100%;}\n.x-abs-layout-item{position:absolute;left:0;top:0;}\n.ext-ie input.x-abs-layout-item,.ext-ie textarea.x-abs-layout-item{margin:0;}\n\n.x-progress-wrap{border:1px solid #6593cf;overflow:hidden;}\n.x-progress-inner{height:18px;background:#e0e8f3 url(../images/default/qtip/bg.gif) repeat-x;position:relative;}\n.x-progress-bar{height:18px;float:left;width:0;background:#9CBFEE url( ../images/default/progress/progress-bg.gif ) repeat-x left center;border-top:1px solid #D1E4FD;border-bottom:1px solid #7FA9E4;border-right:1px solid #7FA9E4;}\n.x-progress-text{font-size:11px;font-weight:bold;color:#fff;padding:1px 5px;overflow:hidden;position:absolute;left:0;text-align:center;}\n.x-progress-text-back{color:#396095;line-height:16px;}\n.ext-ie .x-progress-text-back{line-height:15px;}\n\n.x-window-dlg .x-window-body{border:0 none!important;padding:5px 10px;overflow:hidden!important;}\n.x-window-dlg .x-window-mc{border:0 none!important;}\n.x-window-dlg .ext-mb-text,.x-window-dlg .x-window-header-text{font-size:12px;}\n.x-window-dlg .ext-mb-input{margin-top:4px;width:95%;}\n.x-window-dlg .ext-mb-textarea{margin-top:4px;font:normal 12px tahoma,arial,helvetica,sans-serif;}\n.x-window-dlg .x-progress-wrap{margin-top:4px;}\n.ext-ie .x-window-dlg .x-progress-wrap{margin-top:6px;}\n.x-window-dlg .x-msg-box-wait{background:transparent url(../images/default/grid/loading.gif) no-repeat left;display:block;width:300px;padding-left:18px;line-height:18px;}\n.x-window-dlg .ext-mb-icon{float:left;width:47px;height:32px;}\n.ext-ie .x-window-dlg .ext-mb-icon{width:44px;}\n.x-window-dlg .ext-mb-info{background:transparent url(../images/default/window/icon-info.gif) no-repeat top left;}\n.x-window-dlg .ext-mb-warning{background:transparent url(../images/default/window/icon-warning.gif) no-repeat top left;}\n.x-window-dlg .ext-mb-question{background:transparent url(../images/default/window/icon-question.gif) no-repeat top left;}\n.x-window-dlg .ext-mb-error{background:transparent url(../images/default/window/icon-error.gif) no-repeat top left;}\n\n"
  },
  {
    "path": "docroot/ext-all.js",
    "content": "/*\r\n * Ext JS Library 2.0.2\r\n * Copyright(c) 2006-2008, Ext JS, LLC.\r\n * licensing@extjs.com\r\n * \r\n * http://extjs.com/license\r\n */\r\n\r\nExt.DomHelper=function(){var L=null;var F=/^(?:br|frame|hr|img|input|link|meta|range|spacer|wbr|area|param|col)$/i;var B=/^table|tbody|tr|td$/i;var A=function(T){if(typeof T==\"string\"){return T}var O=\"\";if(Ext.isArray(T)){for(var R=0,P=T.length;R<P;R++){O+=A(T[R])}return O}if(!T.tag){T.tag=\"div\"}O+=\"<\"+T.tag;for(var N in T){if(N==\"tag\"||N==\"children\"||N==\"cn\"||N==\"html\"||typeof T[N]==\"function\"){continue}if(N==\"style\"){var S=T[\"style\"];if(typeof S==\"function\"){S=S.call()}if(typeof S==\"string\"){O+=\" style=\\\"\"+S+\"\\\"\"}else{if(typeof S==\"object\"){O+=\" style=\\\"\";for(var Q in S){if(typeof S[Q]!=\"function\"){O+=Q+\":\"+S[Q]+\";\"}}O+=\"\\\"\"}}}else{if(N==\"cls\"){O+=\" class=\\\"\"+T[\"cls\"]+\"\\\"\"}else{if(N==\"htmlFor\"){O+=\" for=\\\"\"+T[\"htmlFor\"]+\"\\\"\"}else{O+=\" \"+N+\"=\\\"\"+T[N]+\"\\\"\"}}}}if(F.test(T.tag)){O+=\"/>\"}else{O+=\">\";var U=T.children||T.cn;if(U){O+=A(U)}else{if(T.html){O+=T.html}}O+=\"</\"+T.tag+\">\"}return O};var M=function(T,O){var S;if(Ext.isArray(T)){S=document.createDocumentFragment();for(var R=0,P=T.length;R<P;R++){M(T[R],S)}}else{if(typeof T==\"string)\"){S=document.createTextNode(T)}else{S=document.createElement(T.tag||\"div\");var Q=!!S.setAttribute;for(var N in T){if(N==\"tag\"||N==\"children\"||N==\"cn\"||N==\"html\"||N==\"style\"||typeof T[N]==\"function\"){continue}if(N==\"cls\"){S.className=T[\"cls\"]}else{if(Q){S.setAttribute(N,T[N])}else{S[N]=T[N]}}}Ext.DomHelper.applyStyles(S,T.style);var U=T.children||T.cn;if(U){M(U,S)}else{if(T.html){S.innerHTML=T.html}}}}if(O){O.appendChild(S)}return S};var I=function(S,Q,P,R){L.innerHTML=[Q,P,R].join(\"\");var N=-1,O=L;while(++N<S){O=O.firstChild}return O};var J=\"<table>\",E=\"</table>\",C=J+\"<tbody>\",K=\"</tbody>\"+E,H=C+\"<tr>\",D=\"</tr>\"+K;var G=function(N,O,Q,P){if(!L){L=document.createElement(\"div\")}var R;var S=null;if(N==\"td\"){if(O==\"afterbegin\"||O==\"beforeend\"){return }if(O==\"beforebegin\"){S=Q;Q=Q.parentNode}else{S=Q.nextSibling;Q=Q.parentNode}R=I(4,H,P,D)}else{if(N==\"tr\"){if(O==\"beforebegin\"){S=Q;Q=Q.parentNode;R=I(3,C,P,K)}else{if(O==\"afterend\"){S=Q.nextSibling;Q=Q.parentNode;R=I(3,C,P,K)}else{if(O==\"afterbegin\"){S=Q.firstChild}R=I(4,H,P,D)}}}else{if(N==\"tbody\"){if(O==\"beforebegin\"){S=Q;Q=Q.parentNode;R=I(2,J,P,E)}else{if(O==\"afterend\"){S=Q.nextSibling;Q=Q.parentNode;R=I(2,J,P,E)}else{if(O==\"afterbegin\"){S=Q.firstChild}R=I(3,C,P,K)}}}else{if(O==\"beforebegin\"||O==\"afterend\"){return }if(O==\"afterbegin\"){S=Q.firstChild}R=I(2,J,P,E)}}}Q.insertBefore(R,S);return R};return{useDom:false,markup:function(N){return A(N)},applyStyles:function(P,Q){if(Q){P=Ext.fly(P);if(typeof Q==\"string\"){var O=/\\s?([a-z\\-]*)\\:\\s?([^;]*);?/gi;var R;while((R=O.exec(Q))!=null){P.setStyle(R[1],R[2])}}else{if(typeof Q==\"object\"){for(var N in Q){P.setStyle(N,Q[N])}}else{if(typeof Q==\"function\"){Ext.DomHelper.applyStyles(P,Q.call())}}}}},insertHtml:function(P,R,Q){P=P.toLowerCase();if(R.insertAdjacentHTML){if(B.test(R.tagName)){var O;if(O=G(R.tagName.toLowerCase(),P,R,Q)){return O}}switch(P){case\"beforebegin\":R.insertAdjacentHTML(\"BeforeBegin\",Q);return R.previousSibling;case\"afterbegin\":R.insertAdjacentHTML(\"AfterBegin\",Q);return R.firstChild;case\"beforeend\":R.insertAdjacentHTML(\"BeforeEnd\",Q);return R.lastChild;case\"afterend\":R.insertAdjacentHTML(\"AfterEnd\",Q);return R.nextSibling}throw\"Illegal insertion point -> \\\"\"+P+\"\\\"\"}var N=R.ownerDocument.createRange();var S;switch(P){case\"beforebegin\":N.setStartBefore(R);S=N.createContextualFragment(Q);R.parentNode.insertBefore(S,R);return R.previousSibling;case\"afterbegin\":if(R.firstChild){N.setStartBefore(R.firstChild);S=N.createContextualFragment(Q);R.insertBefore(S,R.firstChild);return R.firstChild}else{R.innerHTML=Q;return R.firstChild}case\"beforeend\":if(R.lastChild){N.setStartAfter(R.lastChild);S=N.createContextualFragment(Q);R.appendChild(S);return R.lastChild}else{R.innerHTML=Q;return R.lastChild}case\"afterend\":N.setStartAfter(R);S=N.createContextualFragment(Q);R.parentNode.insertBefore(S,R.nextSibling);return R.nextSibling}throw\"Illegal insertion point -> \\\"\"+P+\"\\\"\"},insertBefore:function(N,P,O){return this.doInsert(N,P,O,\"beforeBegin\")},insertAfter:function(N,P,O){return this.doInsert(N,P,O,\"afterEnd\",\"nextSibling\")},insertFirst:function(N,P,O){return this.doInsert(N,P,O,\"afterBegin\",\"firstChild\")},doInsert:function(Q,S,R,T,P){Q=Ext.getDom(Q);var O;if(this.useDom){O=M(S,null);(P===\"firstChild\"?Q:Q.parentNode).insertBefore(O,P?Q[P]:Q)}else{var N=A(S);O=this.insertHtml(T,Q,N)}return R?Ext.get(O,true):O},append:function(P,R,Q){P=Ext.getDom(P);var O;if(this.useDom){O=M(R,null);P.appendChild(O)}else{var N=A(R);O=this.insertHtml(\"beforeEnd\",P,N)}return Q?Ext.get(O,true):O},overwrite:function(N,P,O){N=Ext.getDom(N);N.innerHTML=A(P);return O?Ext.get(N.firstChild,true):N.firstChild},createTemplate:function(O){var N=A(O);return new Ext.Template(N)}}}();\nExt.Template=function(E){var B=arguments;if(Ext.isArray(E)){E=E.join(\"\")}else{if(B.length>1){var C=[];for(var D=0,A=B.length;D<A;D++){if(typeof B[D]==\"object\"){Ext.apply(this,B[D])}else{C[C.length]=B[D]}}E=C.join(\"\")}}this.html=E;if(this.compiled){this.compile()}};Ext.Template.prototype={applyTemplate:function(B){if(this.compiled){return this.compiled(B)}var A=this.disableFormats!==true;var E=Ext.util.Format,C=this;var D=function(G,I,L,H){if(L&&A){if(L.substr(0,5)==\"this.\"){return C.call(L.substr(5),B[I],B)}else{if(H){var K=/^\\s*['\"](.*)[\"']\\s*$/;H=H.split(\",\");for(var J=0,F=H.length;J<F;J++){H[J]=H[J].replace(K,\"$1\")}H=[B[I]].concat(H)}else{H=[B[I]]}return E[L].apply(E,H)}}else{return B[I]!==undefined?B[I]:\"\"}};return this.html.replace(this.re,D)},set:function(A,B){this.html=A;this.compiled=null;if(B){this.compile()}return this},disableFormats:false,re:/\\{([\\w-]+)(?:\\:([\\w\\.]*)(?:\\((.*?)?\\))?)?\\}/g,compile:function(){var fm=Ext.util.Format;var useF=this.disableFormats!==true;var sep=Ext.isGecko?\"+\":\",\";var fn=function(m,name,format,args){if(format&&useF){args=args?\",\"+args:\"\";if(format.substr(0,5)!=\"this.\"){format=\"fm.\"+format+\"(\"}else{format=\"this.call(\\\"\"+format.substr(5)+\"\\\", \";args=\", values\"}}else{args=\"\";format=\"(values['\"+name+\"'] == undefined ? '' : \"}return\"'\"+sep+format+\"values['\"+name+\"']\"+args+\")\"+sep+\"'\"};var body;if(Ext.isGecko){body=\"this.compiled = function(values){ return '\"+this.html.replace(/\\\\/g,\"\\\\\\\\\").replace(/(\\r\\n|\\n)/g,\"\\\\n\").replace(/'/g,\"\\\\'\").replace(this.re,fn)+\"';};\"}else{body=[\"this.compiled = function(values){ return ['\"];body.push(this.html.replace(/\\\\/g,\"\\\\\\\\\").replace(/(\\r\\n|\\n)/g,\"\\\\n\").replace(/'/g,\"\\\\'\").replace(this.re,fn));body.push(\"'].join('');};\");body=body.join(\"\")}eval(body);return this},call:function(C,B,A){return this[C](B,A)},insertFirst:function(B,A,C){return this.doInsert(\"afterBegin\",B,A,C)},insertBefore:function(B,A,C){return this.doInsert(\"beforeBegin\",B,A,C)},insertAfter:function(B,A,C){return this.doInsert(\"afterEnd\",B,A,C)},append:function(B,A,C){return this.doInsert(\"beforeEnd\",B,A,C)},doInsert:function(C,E,B,A){E=Ext.getDom(E);var D=Ext.DomHelper.insertHtml(C,E,this.applyTemplate(B));return A?Ext.get(D,true):D},overwrite:function(B,A,C){B=Ext.getDom(B);B.innerHTML=this.applyTemplate(A);return C?Ext.get(B.firstChild,true):B.firstChild}};Ext.Template.prototype.apply=Ext.Template.prototype.applyTemplate;Ext.DomHelper.Template=Ext.Template;Ext.Template.from=function(B,A){B=Ext.getDom(B);return new Ext.Template(B.value||B.innerHTML,A||\"\")};\nExt.DomQuery=function(){var cache={},simpleCache={},valueCache={};var nonSpace=/\\S/;var trimRe=/^\\s+|\\s+$/g;var tplRe=/\\{(\\d+)\\}/g;var modeRe=/^(\\s?[\\/>+~]\\s?|\\s|$)/;var tagTokenRe=/^(#)?([\\w-\\*]+)/;var nthRe=/(\\d*)n\\+?(\\d*)/,nthRe2=/\\D/;function child(p,index){var i=0;var n=p.firstChild;while(n){if(n.nodeType==1){if(++i==index){return n}}n=n.nextSibling}return null}function next(n){while((n=n.nextSibling)&&n.nodeType!=1){}return n}function prev(n){while((n=n.previousSibling)&&n.nodeType!=1){}return n}function children(d){var n=d.firstChild,ni=-1;while(n){var nx=n.nextSibling;if(n.nodeType==3&&!nonSpace.test(n.nodeValue)){d.removeChild(n)}else{n.nodeIndex=++ni}n=nx}return this}function byClassName(c,a,v){if(!v){return c}var r=[],ri=-1,cn;for(var i=0,ci;ci=c[i];i++){if((\" \"+ci.className+\" \").indexOf(v)!=-1){r[++ri]=ci}}return r}function attrValue(n,attr){if(!n.tagName&&typeof n.length!=\"undefined\"){n=n[0]}if(!n){return null}if(attr==\"for\"){return n.htmlFor}if(attr==\"class\"||attr==\"className\"){return n.className}return n.getAttribute(attr)||n[attr]}function getNodes(ns,mode,tagName){var result=[],ri=-1,cs;if(!ns){return result}tagName=tagName||\"*\";if(typeof ns.getElementsByTagName!=\"undefined\"){ns=[ns]}if(!mode){for(var i=0,ni;ni=ns[i];i++){cs=ni.getElementsByTagName(tagName);for(var j=0,ci;ci=cs[j];j++){result[++ri]=ci}}}else{if(mode==\"/\"||mode==\">\"){var utag=tagName.toUpperCase();for(var i=0,ni,cn;ni=ns[i];i++){cn=ni.children||ni.childNodes;for(var j=0,cj;cj=cn[j];j++){if(cj.nodeName==utag||cj.nodeName==tagName||tagName==\"*\"){result[++ri]=cj}}}}else{if(mode==\"+\"){var utag=tagName.toUpperCase();for(var i=0,n;n=ns[i];i++){while((n=n.nextSibling)&&n.nodeType!=1){}if(n&&(n.nodeName==utag||n.nodeName==tagName||tagName==\"*\")){result[++ri]=n}}}else{if(mode==\"~\"){for(var i=0,n;n=ns[i];i++){while((n=n.nextSibling)&&(n.nodeType!=1||(tagName==\"*\"||n.tagName.toLowerCase()!=tagName))){}if(n){result[++ri]=n}}}}}}return result}function concat(a,b){if(b.slice){return a.concat(b)}for(var i=0,l=b.length;i<l;i++){a[a.length]=b[i]}return a}function byTag(cs,tagName){if(cs.tagName||cs==document){cs=[cs]}if(!tagName){return cs}var r=[],ri=-1;tagName=tagName.toLowerCase();for(var i=0,ci;ci=cs[i];i++){if(ci.nodeType==1&&ci.tagName.toLowerCase()==tagName){r[++ri]=ci}}return r}function byId(cs,attr,id){if(cs.tagName||cs==document){cs=[cs]}if(!id){return cs}var r=[],ri=-1;for(var i=0,ci;ci=cs[i];i++){if(ci&&ci.id==id){r[++ri]=ci;return r}}return r}function byAttribute(cs,attr,value,op,custom){var r=[],ri=-1,st=custom==\"{\";var f=Ext.DomQuery.operators[op];for(var i=0,ci;ci=cs[i];i++){var a;if(st){a=Ext.DomQuery.getStyle(ci,attr)}else{if(attr==\"class\"||attr==\"className\"){a=ci.className}else{if(attr==\"for\"){a=ci.htmlFor}else{if(attr==\"href\"){a=ci.getAttribute(\"href\",2)}else{a=ci.getAttribute(attr)}}}}if((f&&f(a,value))||(!f&&a)){r[++ri]=ci}}return r}function byPseudo(cs,name,value){return Ext.DomQuery.pseudos[name](cs,value)}var isIE=window.ActiveXObject?true:false;eval(\"var batch = 30803;\");var key=30803;function nodupIEXml(cs){var d=++key;cs[0].setAttribute(\"_nodup\",d);var r=[cs[0]];for(var i=1,len=cs.length;i<len;i++){var c=cs[i];if(!c.getAttribute(\"_nodup\")!=d){c.setAttribute(\"_nodup\",d);r[r.length]=c}}for(var i=0,len=cs.length;i<len;i++){cs[i].removeAttribute(\"_nodup\")}return r}function nodup(cs){if(!cs){return[]}var len=cs.length,c,i,r=cs,cj,ri=-1;if(!len||typeof cs.nodeType!=\"undefined\"||len==1){return cs}if(isIE&&typeof cs[0].selectSingleNode!=\"undefined\"){return nodupIEXml(cs)}var d=++key;cs[0]._nodup=d;for(i=1;c=cs[i];i++){if(c._nodup!=d){c._nodup=d}else{r=[];for(var j=0;j<i;j++){r[++ri]=cs[j]}for(j=i+1;cj=cs[j];j++){if(cj._nodup!=d){cj._nodup=d;r[++ri]=cj}}return r}}return r}function quickDiffIEXml(c1,c2){var d=++key;for(var i=0,len=c1.length;i<len;i++){c1[i].setAttribute(\"_qdiff\",d)}var r=[];for(var i=0,len=c2.length;i<len;i++){if(c2[i].getAttribute(\"_qdiff\")!=d){r[r.length]=c2[i]}}for(var i=0,len=c1.length;i<len;i++){c1[i].removeAttribute(\"_qdiff\")}return r}function quickDiff(c1,c2){var len1=c1.length;if(!len1){return c2}if(isIE&&c1[0].selectSingleNode){return quickDiffIEXml(c1,c2)}var d=++key;for(var i=0;i<len1;i++){c1[i]._qdiff=d}var r=[];for(var i=0,len=c2.length;i<len;i++){if(c2[i]._qdiff!=d){r[r.length]=c2[i]}}return r}function quickId(ns,mode,root,id){if(ns==root){var d=root.ownerDocument||root;return d.getElementById(id)}ns=getNodes(ns,mode,\"*\");return byId(ns,null,id)}return{getStyle:function(el,name){return Ext.fly(el).getStyle(name)},compile:function(path,type){type=type||\"select\";var fn=[\"var f = function(root){\\n var mode; ++batch; var n = root || document;\\n\"];var q=path,mode,lq;var tk=Ext.DomQuery.matchers;var tklen=tk.length;var mm;var lmode=q.match(modeRe);if(lmode&&lmode[1]){fn[fn.length]=\"mode=\\\"\"+lmode[1].replace(trimRe,\"\")+\"\\\";\";q=q.replace(lmode[1],\"\")}while(path.substr(0,1)==\"/\"){path=path.substr(1)}while(q&&lq!=q){lq=q;var tm=q.match(tagTokenRe);if(type==\"select\"){if(tm){if(tm[1]==\"#\"){fn[fn.length]=\"n = quickId(n, mode, root, \\\"\"+tm[2]+\"\\\");\"}else{fn[fn.length]=\"n = getNodes(n, mode, \\\"\"+tm[2]+\"\\\");\"}q=q.replace(tm[0],\"\")}else{if(q.substr(0,1)!=\"@\"){fn[fn.length]=\"n = getNodes(n, mode, \\\"*\\\");\"}}}else{if(tm){if(tm[1]==\"#\"){fn[fn.length]=\"n = byId(n, null, \\\"\"+tm[2]+\"\\\");\"}else{fn[fn.length]=\"n = byTag(n, \\\"\"+tm[2]+\"\\\");\"}q=q.replace(tm[0],\"\")}}while(!(mm=q.match(modeRe))){var matched=false;for(var j=0;j<tklen;j++){var t=tk[j];var m=q.match(t.re);if(m){fn[fn.length]=t.select.replace(tplRe,function(x,i){return m[i]});q=q.replace(m[0],\"\");matched=true;break}}if(!matched){throw\"Error parsing selector, parsing failed at \\\"\"+q+\"\\\"\"}}if(mm[1]){fn[fn.length]=\"mode=\\\"\"+mm[1].replace(trimRe,\"\")+\"\\\";\";q=q.replace(mm[1],\"\")}}fn[fn.length]=\"return nodup(n);\\n}\";eval(fn.join(\"\"));return f},select:function(path,root,type){if(!root||root==document){root=document}if(typeof root==\"string\"){root=document.getElementById(root)}var paths=path.split(\",\");var results=[];for(var i=0,len=paths.length;i<len;i++){var p=paths[i].replace(trimRe,\"\");if(!cache[p]){cache[p]=Ext.DomQuery.compile(p);if(!cache[p]){throw p+\" is not a valid selector\"}}var result=cache[p](root);if(result&&result!=document){results=results.concat(result)}}if(paths.length>1){return nodup(results)}return results},selectNode:function(path,root){return Ext.DomQuery.select(path,root)[0]},selectValue:function(path,root,defaultValue){path=path.replace(trimRe,\"\");if(!valueCache[path]){valueCache[path]=Ext.DomQuery.compile(path,\"select\")}var n=valueCache[path](root);n=n[0]?n[0]:n;var v=(n&&n.firstChild?n.firstChild.nodeValue:null);return((v===null||v===undefined||v===\"\")?defaultValue:v)},selectNumber:function(path,root,defaultValue){var v=Ext.DomQuery.selectValue(path,root,defaultValue||0);return parseFloat(v)},is:function(el,ss){if(typeof el==\"string\"){el=document.getElementById(el)}var isArray=Ext.isArray(el);var result=Ext.DomQuery.filter(isArray?el:[el],ss);return isArray?(result.length==el.length):(result.length>0)},filter:function(els,ss,nonMatches){ss=ss.replace(trimRe,\"\");if(!simpleCache[ss]){simpleCache[ss]=Ext.DomQuery.compile(ss,\"simple\")}var result=simpleCache[ss](els);return nonMatches?quickDiff(result,els):result},matchers:[{re:/^\\.([\\w-]+)/,select:\"n = byClassName(n, null, \\\" {1} \\\");\"},{re:/^\\:([\\w-]+)(?:\\(((?:[^\\s>\\/]*|.*?))\\))?/,select:\"n = byPseudo(n, \\\"{1}\\\", \\\"{2}\\\");\"},{re:/^(?:([\\[\\{])(?:@)?([\\w-]+)\\s?(?:(=|.=)\\s?['\"]?(.*?)[\"']?)?[\\]\\}])/,select:\"n = byAttribute(n, \\\"{2}\\\", \\\"{4}\\\", \\\"{3}\\\", \\\"{1}\\\");\"},{re:/^#([\\w-]+)/,select:\"n = byId(n, null, \\\"{1}\\\");\"},{re:/^@([\\w-]+)/,select:\"return {firstChild:{nodeValue:attrValue(n, \\\"{1}\\\")}};\"}],operators:{\"=\":function(a,v){return a==v},\"!=\":function(a,v){return a!=v},\"^=\":function(a,v){return a&&a.substr(0,v.length)==v},\"$=\":function(a,v){return a&&a.substr(a.length-v.length)==v},\"*=\":function(a,v){return a&&a.indexOf(v)!==-1},\"%=\":function(a,v){return(a%v)==0},\"|=\":function(a,v){return a&&(a==v||a.substr(0,v.length+1)==v+\"-\")},\"~=\":function(a,v){return a&&(\" \"+a+\" \").indexOf(\" \"+v+\" \")!=-1}},pseudos:{\"first-child\":function(c){var r=[],ri=-1,n;for(var i=0,ci;ci=n=c[i];i++){while((n=n.previousSibling)&&n.nodeType!=1){}if(!n){r[++ri]=ci}}return r},\"last-child\":function(c){var r=[],ri=-1,n;for(var i=0,ci;ci=n=c[i];i++){while((n=n.nextSibling)&&n.nodeType!=1){}if(!n){r[++ri]=ci}}return r},\"nth-child\":function(c,a){var r=[],ri=-1;var m=nthRe.exec(a==\"even\"&&\"2n\"||a==\"odd\"&&\"2n+1\"||!nthRe2.test(a)&&\"n+\"+a||a);var f=(m[1]||1)-0,l=m[2]-0;for(var i=0,n;n=c[i];i++){var pn=n.parentNode;if(batch!=pn._batch){var j=0;for(var cn=pn.firstChild;cn;cn=cn.nextSibling){if(cn.nodeType==1){cn.nodeIndex=++j}}pn._batch=batch}if(f==1){if(l==0||n.nodeIndex==l){r[++ri]=n}}else{if((n.nodeIndex+l)%f==0){r[++ri]=n}}}return r},\"only-child\":function(c){var r=[],ri=-1;for(var i=0,ci;ci=c[i];i++){if(!prev(ci)&&!next(ci)){r[++ri]=ci}}return r},\"empty\":function(c){var r=[],ri=-1;for(var i=0,ci;ci=c[i];i++){var cns=ci.childNodes,j=0,cn,empty=true;while(cn=cns[j]){++j;if(cn.nodeType==1||cn.nodeType==3){empty=false;break}}if(empty){r[++ri]=ci}}return r},\"contains\":function(c,v){var r=[],ri=-1;for(var i=0,ci;ci=c[i];i++){if((ci.textContent||ci.innerText||\"\").indexOf(v)!=-1){r[++ri]=ci}}return r},\"nodeValue\":function(c,v){var r=[],ri=-1;for(var i=0,ci;ci=c[i];i++){if(ci.firstChild&&ci.firstChild.nodeValue==v){r[++ri]=ci}}return r},\"checked\":function(c){var r=[],ri=-1;for(var i=0,ci;ci=c[i];i++){if(ci.checked==true){r[++ri]=ci}}return r},\"not\":function(c,ss){return Ext.DomQuery.filter(c,ss,true)},\"any\":function(c,selectors){var ss=selectors.split(\"|\");var r=[],ri=-1,s;for(var i=0,ci;ci=c[i];i++){for(var j=0;s=ss[j];j++){if(Ext.DomQuery.is(ci,s)){r[++ri]=ci;break}}}return r},\"odd\":function(c){return this[\"nth-child\"](c,\"odd\")},\"even\":function(c){return this[\"nth-child\"](c,\"even\")},\"nth\":function(c,a){return c[a-1]||[]},\"first\":function(c){return c[0]||[]},\"last\":function(c){return c[c.length-1]||[]},\"has\":function(c,ss){var s=Ext.DomQuery.select;var r=[],ri=-1;for(var i=0,ci;ci=c[i];i++){if(s(ss,ci).length>0){r[++ri]=ci}}return r},\"next\":function(c,ss){var is=Ext.DomQuery.is;var r=[],ri=-1;for(var i=0,ci;ci=c[i];i++){var n=next(ci);if(n&&is(n,ss)){r[++ri]=ci}}return r},\"prev\":function(c,ss){var is=Ext.DomQuery.is;var r=[],ri=-1;for(var i=0,ci;ci=c[i];i++){var n=prev(ci);if(n&&is(n,ss)){r[++ri]=ci}}return r}}}}();Ext.query=Ext.DomQuery.select;\nExt.util.Observable=function(){if(this.listeners){this.on(this.listeners);delete this.listeners}};Ext.util.Observable.prototype={fireEvent:function(){if(this.eventsSuspended!==true){var A=this.events[arguments[0].toLowerCase()];if(typeof A==\"object\"){return A.fire.apply(A,Array.prototype.slice.call(arguments,1))}}return true},filterOptRe:/^(?:scope|delay|buffer|single)$/,addListener:function(A,C,B,F){if(typeof A==\"object\"){F=A;for(var E in F){if(this.filterOptRe.test(E)){continue}if(typeof F[E]==\"function\"){this.addListener(E,F[E],F.scope,F)}else{this.addListener(E,F[E].fn,F[E].scope,F[E])}}return }F=(!F||typeof F==\"boolean\")?{}:F;A=A.toLowerCase();var D=this.events[A]||true;if(typeof D==\"boolean\"){D=new Ext.util.Event(this,A);this.events[A]=D}D.addListener(C,B,F)},removeListener:function(A,C,B){var D=this.events[A.toLowerCase()];if(typeof D==\"object\"){D.removeListener(C,B)}},purgeListeners:function(){for(var A in this.events){if(typeof this.events[A]==\"object\"){this.events[A].clearListeners()}}},relayEvents:function(F,D){var E=function(G){return function(){return this.fireEvent.apply(this,Ext.combine(G,Array.prototype.slice.call(arguments,0)))}};for(var C=0,A=D.length;C<A;C++){var B=D[C];if(!this.events[B]){this.events[B]=true}F.on(B,E(B),this)}},addEvents:function(D){if(!this.events){this.events={}}if(typeof D==\"string\"){for(var C=0,A=arguments,B;B=A[C];C++){if(!this.events[A[C]]){D[A[C]]=true}}}else{Ext.applyIf(this.events,D)}},hasListener:function(A){var B=this.events[A];return typeof B==\"object\"&&B.listeners.length>0},suspendEvents:function(){this.eventsSuspended=true},resumeEvents:function(){this.eventsSuspended=false},getMethodEvent:function(G){if(!this.methodEvents){this.methodEvents={}}var F=this.methodEvents[G];if(!F){F={};this.methodEvents[G]=F;F.originalFn=this[G];F.methodName=G;F.before=[];F.after=[];var C,B,D;var E=this;var A=function(J,I,H){if((B=J.apply(I||E,H))!==undefined){if(typeof B===\"object\"){if(B.returnValue!==undefined){C=B.returnValue}else{C=B}if(B.cancel===true){D=true}}else{if(B===false){D=true}else{C=B}}}};this[G]=function(){C=B=undefined;D=false;var I=Array.prototype.slice.call(arguments,0);for(var J=0,H=F.before.length;J<H;J++){A(F.before[J].fn,F.before[J].scope,I);if(D){return C}}if((B=F.originalFn.apply(E,I))!==undefined){C=B}for(var J=0,H=F.after.length;J<H;J++){A(F.after[J].fn,F.after[J].scope,I);if(D){return C}}return C}}return F},beforeMethod:function(D,B,A){var C=this.getMethodEvent(D);C.before.push({fn:B,scope:A})},afterMethod:function(D,B,A){var C=this.getMethodEvent(D);C.after.push({fn:B,scope:A})},removeMethodListener:function(F,D,C){var E=this.getMethodEvent(F);for(var B=0,A=E.before.length;B<A;B++){if(E.before[B].fn==D&&E.before[B].scope==C){E.before.splice(B,1);return }}for(var B=0,A=E.after.length;B<A;B++){if(E.after[B].fn==D&&E.after[B].scope==C){E.after.splice(B,1);return }}}};Ext.util.Observable.prototype.on=Ext.util.Observable.prototype.addListener;Ext.util.Observable.prototype.un=Ext.util.Observable.prototype.removeListener;Ext.util.Observable.capture=function(C,B,A){C.fireEvent=C.fireEvent.createInterceptor(B,A)};Ext.util.Observable.releaseCapture=function(A){A.fireEvent=Ext.util.Observable.prototype.fireEvent};(function(){var B=function(F,G,E){var D=new Ext.util.DelayedTask();return function(){D.delay(G.buffer,F,E,Array.prototype.slice.call(arguments,0))}};var C=function(F,G,E,D){return function(){G.removeListener(E,D);return F.apply(D,arguments)}};var A=function(E,F,D){return function(){var G=Array.prototype.slice.call(arguments,0);setTimeout(function(){E.apply(D,G)},F.delay||10)}};Ext.util.Event=function(E,D){this.name=D;this.obj=E;this.listeners=[]};Ext.util.Event.prototype={addListener:function(G,F,E){F=F||this.obj;if(!this.isListening(G,F)){var D=this.createListener(G,F,E);if(!this.firing){this.listeners.push(D)}else{this.listeners=this.listeners.slice(0);this.listeners.push(D)}}},createListener:function(G,F,H){H=H||{};F=F||this.obj;var D={fn:G,scope:F,options:H};var E=G;if(H.delay){E=A(E,H,F)}if(H.single){E=C(E,this,G,F)}if(H.buffer){E=B(E,H,F)}D.fireFn=E;return D},findListener:function(I,H){H=H||this.obj;var F=this.listeners;for(var G=0,D=F.length;G<D;G++){var E=F[G];if(E.fn==I&&E.scope==H){return G}}return -1},isListening:function(E,D){return this.findListener(E,D)!=-1},removeListener:function(F,E){var D;if((D=this.findListener(F,E))!=-1){if(!this.firing){this.listeners.splice(D,1)}else{this.listeners=this.listeners.slice(0);this.listeners.splice(D,1)}return true}return false},clearListeners:function(){this.listeners=[]},fire:function(){var F=this.listeners,I,D=F.length;if(D>0){this.firing=true;var G=Array.prototype.slice.call(arguments,0);for(var H=0;H<D;H++){var E=F[H];if(E.fireFn.apply(E.scope||this.obj||window,arguments)===false){this.firing=false;return false}}this.firing=false}return true}}})();\nExt.EventManager=function(){var T,M,I=false;var K,S,C,O;var L=Ext.lib.Event;var N=Ext.lib.Dom;var B=function(){if(!I){I=true;Ext.isReady=true;if(M){clearInterval(M)}if(Ext.isGecko||Ext.isOpera){document.removeEventListener(\"DOMContentLoaded\",B,false)}if(Ext.isIE){var D=document.getElementById(\"ie-deferred-loader\");if(D){D.onreadystatechange=null;D.parentNode.removeChild(D)}}if(T){T.fire();T.clearListeners()}}};var A=function(){T=new Ext.util.Event();if(Ext.isGecko||Ext.isOpera){document.addEventListener(\"DOMContentLoaded\",B,false)}else{if(Ext.isIE){document.write(\"<s\"+\"cript id=\\\"ie-deferred-loader\\\" defer=\\\"defer\\\" src=\\\"/\"+\"/:\\\"></s\"+\"cript>\");var D=document.getElementById(\"ie-deferred-loader\");D.onreadystatechange=function(){if(this.readyState==\"complete\"){B()}}}else{if(Ext.isSafari){M=setInterval(function(){var E=document.readyState;if(E==\"complete\"){B()}},10)}}}L.on(window,\"load\",B)};var R=function(E,U){var D=new Ext.util.DelayedTask(E);return function(V){V=new Ext.EventObjectImpl(V);D.delay(U.buffer,E,null,[V])}};var P=function(V,U,D,E){return function(W){Ext.EventManager.removeListener(U,D,E);V(W)}};var F=function(D,E){return function(U){U=new Ext.EventObjectImpl(U);setTimeout(function(){D(U)},E.delay||10)}};var J=function(U,E,D,Y,X){var Z=(!D||typeof D==\"boolean\")?{}:D;Y=Y||Z.fn;X=X||Z.scope;var W=Ext.getDom(U);if(!W){throw\"Error listening for \\\"\"+E+\"\\\". Element \\\"\"+U+\"\\\" doesn't exist.\"}var V=function(b){b=Ext.EventObject.setEvent(b);var a;if(Z.delegate){a=b.getTarget(Z.delegate,W);if(!a){return }}else{a=b.target}if(Z.stopEvent===true){b.stopEvent()}if(Z.preventDefault===true){b.preventDefault()}if(Z.stopPropagation===true){b.stopPropagation()}if(Z.normalized===false){b=b.browserEvent}Y.call(X||W,b,a,Z)};if(Z.delay){V=F(V,Z)}if(Z.single){V=P(V,W,E,Y)}if(Z.buffer){V=R(V,Z)}Y._handlers=Y._handlers||[];Y._handlers.push([Ext.id(W),E,V]);L.on(W,E,V);if(E==\"mousewheel\"&&W.addEventListener){W.addEventListener(\"DOMMouseScroll\",V,false);L.on(window,\"unload\",function(){W.removeEventListener(\"DOMMouseScroll\",V,false)})}if(E==\"mousedown\"&&W==document){Ext.EventManager.stoppedMouseDownEvent.addListener(V)}return V};var G=function(E,U,Z){var D=Ext.id(E),a=Z._handlers,X=Z;if(a){for(var V=0,Y=a.length;V<Y;V++){var W=a[V];if(W[0]==D&&W[1]==U){X=W[2];a.splice(V,1);break}}}L.un(E,U,X);E=Ext.getDom(E);if(U==\"mousewheel\"&&E.addEventListener){E.removeEventListener(\"DOMMouseScroll\",X,false)}if(U==\"mousedown\"&&E==document){Ext.EventManager.stoppedMouseDownEvent.removeListener(X)}};var H=/^(?:scope|delay|buffer|single|stopEvent|preventDefault|stopPropagation|normalized|args|delegate)$/;var Q={addListener:function(U,D,W,V,E){if(typeof D==\"object\"){var Y=D;for(var X in Y){if(H.test(X)){continue}if(typeof Y[X]==\"function\"){J(U,X,Y,Y[X],Y.scope)}else{J(U,X,Y[X])}}return }return J(U,D,E,W,V)},removeListener:function(E,D,U){return G(E,D,U)},onDocumentReady:function(U,E,D){if(I){T.addListener(U,E,D);T.fire();T.clearListeners();return }if(!T){A()}T.addListener(U,E,D)},onWindowResize:function(U,E,D){if(!K){K=new Ext.util.Event();S=new Ext.util.DelayedTask(function(){K.fire(N.getViewWidth(),N.getViewHeight())});L.on(window,\"resize\",this.fireWindowResize,this)}K.addListener(U,E,D)},fireWindowResize:function(){if(K){if((Ext.isIE||Ext.isAir)&&S){S.delay(50)}else{K.fire(N.getViewWidth(),N.getViewHeight())}}},onTextResize:function(V,U,D){if(!C){C=new Ext.util.Event();var E=new Ext.Element(document.createElement(\"div\"));E.dom.className=\"x-text-resize\";E.dom.innerHTML=\"X\";E.appendTo(document.body);O=E.dom.offsetHeight;setInterval(function(){if(E.dom.offsetHeight!=O){C.fire(O,O=E.dom.offsetHeight)}},this.textResizeInterval)}C.addListener(V,U,D)},removeResizeListener:function(E,D){if(K){K.removeListener(E,D)}},fireResize:function(){if(K){K.fire(N.getViewWidth(),N.getViewHeight())}},ieDeferSrc:false,textResizeInterval:50};Q.on=Q.addListener;Q.un=Q.removeListener;Q.stoppedMouseDownEvent=new Ext.util.Event();return Q}();Ext.onReady=Ext.EventManager.onDocumentReady;Ext.onReady(function(){var B=Ext.getBody();if(!B){return }var A=[Ext.isIE?\"ext-ie \"+(Ext.isIE6?\"ext-ie6\":\"ext-ie7\"):Ext.isGecko?\"ext-gecko\":Ext.isOpera?\"ext-opera\":Ext.isSafari?\"ext-safari\":\"\"];if(Ext.isMac){A.push(\"ext-mac\")}if(Ext.isLinux){A.push(\"ext-linux\")}if(Ext.isBorderBox){A.push(\"ext-border-box\")}if(Ext.isStrict){var C=B.dom.parentNode;if(C){C.className+=\" ext-strict\"}}B.addClass(A.join(\" \"))});Ext.EventObject=function(){var B=Ext.lib.Event;var A={63234:37,63235:39,63232:38,63233:40,63276:33,63277:34,63272:46,63273:36,63275:35};var C=Ext.isIE?{1:0,4:1,2:2}:(Ext.isSafari?{1:0,2:1,3:2}:{0:0,1:1,2:2});Ext.EventObjectImpl=function(D){if(D){this.setEvent(D.browserEvent||D)}};Ext.EventObjectImpl.prototype={browserEvent:null,button:-1,shiftKey:false,ctrlKey:false,altKey:false,BACKSPACE:8,TAB:9,RETURN:13,ENTER:13,SHIFT:16,CONTROL:17,ESC:27,SPACE:32,PAGEUP:33,PAGEDOWN:34,END:35,HOME:36,LEFT:37,UP:38,RIGHT:39,DOWN:40,DELETE:46,F5:116,setEvent:function(D){if(D==this||(D&&D.browserEvent)){return D}this.browserEvent=D;if(D){this.button=D.button?C[D.button]:(D.which?D.which-1:-1);if(D.type==\"click\"&&this.button==-1){this.button=0}this.type=D.type;this.shiftKey=D.shiftKey;this.ctrlKey=D.ctrlKey||D.metaKey;this.altKey=D.altKey;this.keyCode=D.keyCode;this.charCode=D.charCode;this.target=B.getTarget(D);this.xy=B.getXY(D)}else{this.button=-1;this.shiftKey=false;this.ctrlKey=false;this.altKey=false;this.keyCode=0;this.charCode=0;this.target=null;this.xy=[0,0]}return this},stopEvent:function(){if(this.browserEvent){if(this.browserEvent.type==\"mousedown\"){Ext.EventManager.stoppedMouseDownEvent.fire(this)}B.stopEvent(this.browserEvent)}},preventDefault:function(){if(this.browserEvent){B.preventDefault(this.browserEvent)}},isNavKeyPress:function(){var D=this.keyCode;D=Ext.isSafari?(A[D]||D):D;return(D>=33&&D<=40)||D==this.RETURN||D==this.TAB||D==this.ESC},isSpecialKey:function(){var D=this.keyCode;return(this.type==\"keypress\"&&this.ctrlKey)||D==9||D==13||D==40||D==27||(D==16)||(D==17)||(D>=18&&D<=20)||(D>=33&&D<=35)||(D>=36&&D<=39)||(D>=44&&D<=45)},stopPropagation:function(){if(this.browserEvent){if(this.browserEvent.type==\"mousedown\"){Ext.EventManager.stoppedMouseDownEvent.fire(this)}B.stopPropagation(this.browserEvent)}},getCharCode:function(){return this.charCode||this.keyCode},getKey:function(){var D=this.keyCode||this.charCode;return Ext.isSafari?(A[D]||D):D},getPageX:function(){return this.xy[0]},getPageY:function(){return this.xy[1]},getTime:function(){if(this.browserEvent){return B.getTime(this.browserEvent)}return null},getXY:function(){return this.xy},getTarget:function(E,G,D){var F=Ext.get(this.target);return E?F.findParent(E,G,D):(D?F:this.target)},getRelatedTarget:function(){if(this.browserEvent){return B.getRelatedTarget(this.browserEvent)}return null},getWheelDelta:function(){var D=this.browserEvent;var E=0;if(D.wheelDelta){E=D.wheelDelta/120}else{if(D.detail){E=-D.detail/3}}return E},hasModifier:function(){return((this.ctrlKey||this.altKey)||this.shiftKey)?true:false},within:function(E,F){var D=this[F?\"getRelatedTarget\":\"getTarget\"]();return D&&Ext.fly(E).contains(D)},getPoint:function(){return new Ext.lib.Point(this.xy[0],this.xy[1])}};return new Ext.EventObjectImpl()}();\n(function(){var D=Ext.lib.Dom;var E=Ext.lib.Event;var A=Ext.lib.Anim;var propCache={};var camelRe=/(-[a-z])/gi;var camelFn=function(m,a){return a.charAt(1).toUpperCase()};var view=document.defaultView;Ext.Element=function(element,forceNew){var dom=typeof element==\"string\"?document.getElementById(element):element;if(!dom){return null}var id=dom.id;if(forceNew!==true&&id&&Ext.Element.cache[id]){return Ext.Element.cache[id]}this.dom=dom;this.id=id||Ext.id(dom)};var El=Ext.Element;El.prototype={originalDisplay:\"\",visibilityMode:1,defaultUnit:\"px\",setVisibilityMode:function(visMode){this.visibilityMode=visMode;return this},enableDisplayMode:function(display){this.setVisibilityMode(El.DISPLAY);if(typeof display!=\"undefined\"){this.originalDisplay=display}return this},findParent:function(simpleSelector,maxDepth,returnEl){var p=this.dom,b=document.body,depth=0,dq=Ext.DomQuery,stopEl;maxDepth=maxDepth||50;if(typeof maxDepth!=\"number\"){stopEl=Ext.getDom(maxDepth);maxDepth=10}while(p&&p.nodeType==1&&depth<maxDepth&&p!=b&&p!=stopEl){if(dq.is(p,simpleSelector)){return returnEl?Ext.get(p):p}depth++;p=p.parentNode}return null},findParentNode:function(simpleSelector,maxDepth,returnEl){var p=Ext.fly(this.dom.parentNode,\"_internal\");return p?p.findParent(simpleSelector,maxDepth,returnEl):null},up:function(simpleSelector,maxDepth){return this.findParentNode(simpleSelector,maxDepth,true)},is:function(simpleSelector){return Ext.DomQuery.is(this.dom,simpleSelector)},animate:function(args,duration,onComplete,easing,animType){this.anim(args,{duration:duration,callback:onComplete,easing:easing},animType);return this},anim:function(args,opt,animType,defaultDur,defaultEase,cb){animType=animType||\"run\";opt=opt||{};var anim=Ext.lib.Anim[animType](this.dom,args,(opt.duration||defaultDur)||0.35,(opt.easing||defaultEase)||\"easeOut\",function(){Ext.callback(cb,this);Ext.callback(opt.callback,opt.scope||this,[this,opt])},this);opt.anim=anim;return anim},preanim:function(a,i){return !a[i]?false:(typeof a[i]==\"object\"?a[i]:{duration:a[i+1],callback:a[i+2],easing:a[i+3]})},clean:function(forceReclean){if(this.isCleaned&&forceReclean!==true){return this}var ns=/\\S/;var d=this.dom,n=d.firstChild,ni=-1;while(n){var nx=n.nextSibling;if(n.nodeType==3&&!ns.test(n.nodeValue)){d.removeChild(n)}else{n.nodeIndex=++ni}n=nx}this.isCleaned=true;return this},scrollIntoView:function(container,hscroll){var c=Ext.getDom(container)||Ext.getBody().dom;var el=this.dom;var o=this.getOffsetsTo(c),l=o[0]+c.scrollLeft,t=o[1]+c.scrollTop,b=t+el.offsetHeight,r=l+el.offsetWidth;var ch=c.clientHeight;var ct=parseInt(c.scrollTop,10);var cl=parseInt(c.scrollLeft,10);var cb=ct+ch;var cr=cl+c.clientWidth;if(el.offsetHeight>ch||t<ct){c.scrollTop=t}else{if(b>cb){c.scrollTop=b-ch}}c.scrollTop=c.scrollTop;if(hscroll!==false){if(el.offsetWidth>c.clientWidth||l<cl){c.scrollLeft=l}else{if(r>cr){c.scrollLeft=r-c.clientWidth}}c.scrollLeft=c.scrollLeft}return this},scrollChildIntoView:function(child,hscroll){Ext.fly(child,\"_scrollChildIntoView\").scrollIntoView(this,hscroll)},autoHeight:function(animate,duration,onComplete,easing){var oldHeight=this.getHeight();this.clip();this.setHeight(1);setTimeout(function(){var height=parseInt(this.dom.scrollHeight,10);if(!animate){this.setHeight(height);this.unclip();if(typeof onComplete==\"function\"){onComplete()}}else{this.setHeight(oldHeight);this.setHeight(height,animate,duration,function(){this.unclip();if(typeof onComplete==\"function\"){onComplete()}}.createDelegate(this),easing)}}.createDelegate(this),0);return this},contains:function(el){if(!el){return false}return D.isAncestor(this.dom,el.dom?el.dom:el)},isVisible:function(deep){var vis=!(this.getStyle(\"visibility\")==\"hidden\"||this.getStyle(\"display\")==\"none\");if(deep!==true||!vis){return vis}var p=this.dom.parentNode;while(p&&p.tagName.toLowerCase()!=\"body\"){if(!Ext.fly(p,\"_isVisible\").isVisible()){return false}p=p.parentNode}return true},select:function(selector,unique){return El.select(selector,unique,this.dom)},query:function(selector,unique){return Ext.DomQuery.select(selector,this.dom)},child:function(selector,returnDom){var n=Ext.DomQuery.selectNode(selector,this.dom);return returnDom?n:Ext.get(n)},down:function(selector,returnDom){var n=Ext.DomQuery.selectNode(\" > \"+selector,this.dom);return returnDom?n:Ext.get(n)},initDD:function(group,config,overrides){var dd=new Ext.dd.DD(Ext.id(this.dom),group,config);return Ext.apply(dd,overrides)},initDDProxy:function(group,config,overrides){var dd=new Ext.dd.DDProxy(Ext.id(this.dom),group,config);return Ext.apply(dd,overrides)},initDDTarget:function(group,config,overrides){var dd=new Ext.dd.DDTarget(Ext.id(this.dom),group,config);return Ext.apply(dd,overrides)},setVisible:function(visible,animate){if(!animate||!A){if(this.visibilityMode==El.DISPLAY){this.setDisplayed(visible)}else{this.fixDisplay();this.dom.style.visibility=visible?\"visible\":\"hidden\"}}else{var dom=this.dom;var visMode=this.visibilityMode;if(visible){this.setOpacity(0.01);this.setVisible(true)}this.anim({opacity:{to:(visible?1:0)}},this.preanim(arguments,1),null,0.35,\"easeIn\",function(){if(!visible){if(visMode==El.DISPLAY){dom.style.display=\"none\"}else{dom.style.visibility=\"hidden\"}Ext.get(dom).setOpacity(1)}})}return this},isDisplayed:function(){return this.getStyle(\"display\")!=\"none\"},toggle:function(animate){this.setVisible(!this.isVisible(),this.preanim(arguments,0));return this},setDisplayed:function(value){if(typeof value==\"boolean\"){value=value?this.originalDisplay:\"none\"}this.setStyle(\"display\",value);return this},focus:function(){try{this.dom.focus()}catch(e){}return this},blur:function(){try{this.dom.blur()}catch(e){}return this},addClass:function(className){if(Ext.isArray(className)){for(var i=0,len=className.length;i<len;i++){this.addClass(className[i])}}else{if(className&&!this.hasClass(className)){this.dom.className=this.dom.className+\" \"+className}}return this},radioClass:function(className){var siblings=this.dom.parentNode.childNodes;for(var i=0;i<siblings.length;i++){var s=siblings[i];if(s.nodeType==1){Ext.get(s).removeClass(className)}}this.addClass(className);return this},removeClass:function(className){if(!className||!this.dom.className){return this}if(Ext.isArray(className)){for(var i=0,len=className.length;i<len;i++){this.removeClass(className[i])}}else{if(this.hasClass(className)){var re=this.classReCache[className];if(!re){re=new RegExp(\"(?:^|\\\\s+)\"+className+\"(?:\\\\s+|$)\",\"g\");this.classReCache[className]=re}this.dom.className=this.dom.className.replace(re,\" \")}}return this},classReCache:{},toggleClass:function(className){if(this.hasClass(className)){this.removeClass(className)}else{this.addClass(className)}return this},hasClass:function(className){return className&&(\" \"+this.dom.className+\" \").indexOf(\" \"+className+\" \")!=-1},replaceClass:function(oldClassName,newClassName){this.removeClass(oldClassName);this.addClass(newClassName);return this},getStyles:function(){var a=arguments,len=a.length,r={};for(var i=0;i<len;i++){r[a[i]]=this.getStyle(a[i])}return r},getStyle:function(){return view&&view.getComputedStyle?function(prop){var el=this.dom,v,cs,camel;if(prop==\"float\"){prop=\"cssFloat\"}if(v=el.style[prop]){return v}if(cs=view.getComputedStyle(el,\"\")){if(!(camel=propCache[prop])){camel=propCache[prop]=prop.replace(camelRe,camelFn)}return cs[camel]}return null}:function(prop){var el=this.dom,v,cs,camel;if(prop==\"opacity\"){if(typeof el.style.filter==\"string\"){var m=el.style.filter.match(/alpha\\(opacity=(.*)\\)/i);if(m){var fv=parseFloat(m[1]);if(!isNaN(fv)){return fv?fv/100:0}}}return 1}else{if(prop==\"float\"){prop=\"styleFloat\"}}if(!(camel=propCache[prop])){camel=propCache[prop]=prop.replace(camelRe,camelFn)}if(v=el.style[camel]){return v}if(cs=el.currentStyle){return cs[camel]}return null}}(),setStyle:function(prop,value){if(typeof prop==\"string\"){var camel;if(!(camel=propCache[prop])){camel=propCache[prop]=prop.replace(camelRe,camelFn)}if(camel==\"opacity\"){this.setOpacity(value)}else{this.dom.style[camel]=value}}else{for(var style in prop){if(typeof prop[style]!=\"function\"){this.setStyle(style,prop[style])}}}return this},applyStyles:function(style){Ext.DomHelper.applyStyles(this.dom,style);return this},getX:function(){return D.getX(this.dom)},getY:function(){return D.getY(this.dom)},getXY:function(){return D.getXY(this.dom)},getOffsetsTo:function(el){var o=this.getXY();var e=Ext.fly(el,\"_internal\").getXY();return[o[0]-e[0],o[1]-e[1]]},setX:function(x,animate){if(!animate||!A){D.setX(this.dom,x)}else{this.setXY([x,this.getY()],this.preanim(arguments,1))}return this},setY:function(y,animate){if(!animate||!A){D.setY(this.dom,y)}else{this.setXY([this.getX(),y],this.preanim(arguments,1))}return this},setLeft:function(left){this.setStyle(\"left\",this.addUnits(left));return this},setTop:function(top){this.setStyle(\"top\",this.addUnits(top));return this},setRight:function(right){this.setStyle(\"right\",this.addUnits(right));return this},setBottom:function(bottom){this.setStyle(\"bottom\",this.addUnits(bottom));return this},setXY:function(pos,animate){if(!animate||!A){D.setXY(this.dom,pos)}else{this.anim({points:{to:pos}},this.preanim(arguments,1),\"motion\")}return this},setLocation:function(x,y,animate){this.setXY([x,y],this.preanim(arguments,2));return this},moveTo:function(x,y,animate){this.setXY([x,y],this.preanim(arguments,2));return this},getRegion:function(){return D.getRegion(this.dom)},getHeight:function(contentHeight){var h=this.dom.offsetHeight||0;h=contentHeight!==true?h:h-this.getBorderWidth(\"tb\")-this.getPadding(\"tb\");return h<0?0:h},getWidth:function(contentWidth){var w=this.dom.offsetWidth||0;w=contentWidth!==true?w:w-this.getBorderWidth(\"lr\")-this.getPadding(\"lr\");return w<0?0:w},getComputedHeight:function(){var h=Math.max(this.dom.offsetHeight,this.dom.clientHeight);if(!h){h=parseInt(this.getStyle(\"height\"),10)||0;if(!this.isBorderBox()){h+=this.getFrameWidth(\"tb\")}}return h},getComputedWidth:function(){var w=Math.max(this.dom.offsetWidth,this.dom.clientWidth);if(!w){w=parseInt(this.getStyle(\"width\"),10)||0;if(!this.isBorderBox()){w+=this.getFrameWidth(\"lr\")}}return w},getSize:function(contentSize){return{width:this.getWidth(contentSize),height:this.getHeight(contentSize)}},getStyleSize:function(){var w,h,d=this.dom,s=d.style;if(s.width&&s.width!=\"auto\"){w=parseInt(s.width,10);if(Ext.isBorderBox){w-=this.getFrameWidth(\"lr\")}}if(s.height&&s.height!=\"auto\"){h=parseInt(s.height,10);if(Ext.isBorderBox){h-=this.getFrameWidth(\"tb\")}}return{width:w||this.getWidth(true),height:h||this.getHeight(true)}},getViewSize:function(){var d=this.dom,doc=document,aw=0,ah=0;if(d==doc||d==doc.body){return{width:D.getViewWidth(),height:D.getViewHeight()}}else{return{width:d.clientWidth,height:d.clientHeight}}},getValue:function(asNumber){return asNumber?parseInt(this.dom.value,10):this.dom.value},adjustWidth:function(width){if(typeof width==\"number\"){if(this.autoBoxAdjust&&!this.isBorderBox()){width-=(this.getBorderWidth(\"lr\")+this.getPadding(\"lr\"))}if(width<0){width=0}}return width},adjustHeight:function(height){if(typeof height==\"number\"){if(this.autoBoxAdjust&&!this.isBorderBox()){height-=(this.getBorderWidth(\"tb\")+this.getPadding(\"tb\"))}if(height<0){height=0}}return height},setWidth:function(width,animate){width=this.adjustWidth(width);if(!animate||!A){this.dom.style.width=this.addUnits(width)}else{this.anim({width:{to:width}},this.preanim(arguments,1))}return this},setHeight:function(height,animate){height=this.adjustHeight(height);if(!animate||!A){this.dom.style.height=this.addUnits(height)}else{this.anim({height:{to:height}},this.preanim(arguments,1))}return this},setSize:function(width,height,animate){if(typeof width==\"object\"){height=width.height;width=width.width}width=this.adjustWidth(width);height=this.adjustHeight(height);if(!animate||!A){this.dom.style.width=this.addUnits(width);this.dom.style.height=this.addUnits(height)}else{this.anim({width:{to:width},height:{to:height}},this.preanim(arguments,2))}return this},setBounds:function(x,y,width,height,animate){if(!animate||!A){this.setSize(width,height);this.setLocation(x,y)}else{width=this.adjustWidth(width);height=this.adjustHeight(height);this.anim({points:{to:[x,y]},width:{to:width},height:{to:height}},this.preanim(arguments,4),\"motion\")}return this},setRegion:function(region,animate){this.setBounds(region.left,region.top,region.right-region.left,region.bottom-region.top,this.preanim(arguments,1));return this},addListener:function(eventName,fn,scope,options){Ext.EventManager.on(this.dom,eventName,fn,scope||this,options)},removeListener:function(eventName,fn){Ext.EventManager.removeListener(this.dom,eventName,fn);return this},removeAllListeners:function(){E.purgeElement(this.dom);return this},relayEvent:function(eventName,observable){this.on(eventName,function(e){observable.fireEvent(eventName,e)})},setOpacity:function(opacity,animate){if(!animate||!A){var s=this.dom.style;if(Ext.isIE){s.zoom=1;s.filter=(s.filter||\"\").replace(/alpha\\([^\\)]*\\)/gi,\"\")+(opacity==1?\"\":\" alpha(opacity=\"+opacity*100+\")\")}else{s.opacity=opacity}}else{this.anim({opacity:{to:opacity}},this.preanim(arguments,1),null,0.35,\"easeIn\")}return this},getLeft:function(local){if(!local){return this.getX()}else{return parseInt(this.getStyle(\"left\"),10)||0}},getRight:function(local){if(!local){return this.getX()+this.getWidth()}else{return(this.getLeft(true)+this.getWidth())||0}},getTop:function(local){if(!local){return this.getY()}else{return parseInt(this.getStyle(\"top\"),10)||0}},getBottom:function(local){if(!local){return this.getY()+this.getHeight()}else{return(this.getTop(true)+this.getHeight())||0}},position:function(pos,zIndex,x,y){if(!pos){if(this.getStyle(\"position\")==\"static\"){this.setStyle(\"position\",\"relative\")}}else{this.setStyle(\"position\",pos)}if(zIndex){this.setStyle(\"z-index\",zIndex)}if(x!==undefined&&y!==undefined){this.setXY([x,y])}else{if(x!==undefined){this.setX(x)}else{if(y!==undefined){this.setY(y)}}}},clearPositioning:function(value){value=value||\"\";this.setStyle({\"left\":value,\"right\":value,\"top\":value,\"bottom\":value,\"z-index\":\"\",\"position\":\"static\"});return this},getPositioning:function(){var l=this.getStyle(\"left\");var t=this.getStyle(\"top\");return{\"position\":this.getStyle(\"position\"),\"left\":l,\"right\":l?\"\":this.getStyle(\"right\"),\"top\":t,\"bottom\":t?\"\":this.getStyle(\"bottom\"),\"z-index\":this.getStyle(\"z-index\")}},getBorderWidth:function(side){return this.addStyles(side,El.borders)},getPadding:function(side){return this.addStyles(side,El.paddings)},setPositioning:function(pc){this.applyStyles(pc);if(pc.right==\"auto\"){this.dom.style.right=\"\"}if(pc.bottom==\"auto\"){this.dom.style.bottom=\"\"}return this},fixDisplay:function(){if(this.getStyle(\"display\")==\"none\"){this.setStyle(\"visibility\",\"hidden\");this.setStyle(\"display\",this.originalDisplay);if(this.getStyle(\"display\")==\"none\"){this.setStyle(\"display\",\"block\")}}},setOverflow:function(v){if(v==\"auto\"&&Ext.isMac&&Ext.isGecko){this.dom.style.overflow=\"hidden\";(function(){this.dom.style.overflow=\"auto\"}).defer(1,this)}else{this.dom.style.overflow=v}},setLeftTop:function(left,top){this.dom.style.left=this.addUnits(left);this.dom.style.top=this.addUnits(top);return this},move:function(direction,distance,animate){var xy=this.getXY();direction=direction.toLowerCase();switch(direction){case\"l\":case\"left\":this.moveTo(xy[0]-distance,xy[1],this.preanim(arguments,2));break;case\"r\":case\"right\":this.moveTo(xy[0]+distance,xy[1],this.preanim(arguments,2));break;case\"t\":case\"top\":case\"up\":this.moveTo(xy[0],xy[1]-distance,this.preanim(arguments,2));break;case\"b\":case\"bottom\":case\"down\":this.moveTo(xy[0],xy[1]+distance,this.preanim(arguments,2));break}return this},clip:function(){if(!this.isClipped){this.isClipped=true;this.originalClip={\"o\":this.getStyle(\"overflow\"),\"x\":this.getStyle(\"overflow-x\"),\"y\":this.getStyle(\"overflow-y\")};this.setStyle(\"overflow\",\"hidden\");this.setStyle(\"overflow-x\",\"hidden\");this.setStyle(\"overflow-y\",\"hidden\")}return this},unclip:function(){if(this.isClipped){this.isClipped=false;var o=this.originalClip;if(o.o){this.setStyle(\"overflow\",o.o)}if(o.x){this.setStyle(\"overflow-x\",o.x)}if(o.y){this.setStyle(\"overflow-y\",o.y)}}return this},getAnchorXY:function(anchor,local,s){var w,h,vp=false;if(!s){var d=this.dom;if(d==document.body||d==document){vp=true;w=D.getViewWidth();h=D.getViewHeight()}else{w=this.getWidth();h=this.getHeight()}}else{w=s.width;h=s.height}var x=0,y=0,r=Math.round;switch((anchor||\"tl\").toLowerCase()){case\"c\":x=r(w*0.5);y=r(h*0.5);break;case\"t\":x=r(w*0.5);y=0;break;case\"l\":x=0;y=r(h*0.5);break;case\"r\":x=w;y=r(h*0.5);break;case\"b\":x=r(w*0.5);y=h;break;case\"tl\":x=0;y=0;break;case\"bl\":x=0;y=h;break;case\"br\":x=w;y=h;break;case\"tr\":x=w;y=0;break}if(local===true){return[x,y]}if(vp){var sc=this.getScroll();return[x+sc.left,y+sc.top]}var o=this.getXY();return[x+o[0],y+o[1]]},getAlignToXY:function(el,p,o){el=Ext.get(el);if(!el||!el.dom){throw\"Element.alignToXY with an element that doesn't exist\"}var d=this.dom;var c=false;var p1=\"\",p2=\"\";o=o||[0,0];if(!p){p=\"tl-bl\"}else{if(p==\"?\"){p=\"tl-bl?\"}else{if(p.indexOf(\"-\")==-1){p=\"tl-\"+p}}}p=p.toLowerCase();var m=p.match(/^([a-z]+)-([a-z]+)(\\?)?$/);if(!m){throw\"Element.alignTo with an invalid alignment \"+p}p1=m[1];p2=m[2];c=!!m[3];var a1=this.getAnchorXY(p1,true);var a2=el.getAnchorXY(p2,false);var x=a2[0]-a1[0]+o[0];var y=a2[1]-a1[1]+o[1];if(c){var w=this.getWidth(),h=this.getHeight(),r=el.getRegion();var dw=D.getViewWidth()-5,dh=D.getViewHeight()-5;var p1y=p1.charAt(0),p1x=p1.charAt(p1.length-1);var p2y=p2.charAt(0),p2x=p2.charAt(p2.length-1);var swapY=((p1y==\"t\"&&p2y==\"b\")||(p1y==\"b\"&&p2y==\"t\"));var swapX=((p1x==\"r\"&&p2x==\"l\")||(p1x==\"l\"&&p2x==\"r\"));var doc=document;var scrollX=(doc.documentElement.scrollLeft||doc.body.scrollLeft||0)+5;var scrollY=(doc.documentElement.scrollTop||doc.body.scrollTop||0)+5;if((x+w)>dw+scrollX){x=swapX?r.left-w:dw+scrollX-w}if(x<scrollX){x=swapX?r.right:scrollX}if((y+h)>dh+scrollY){y=swapY?r.top-h:dh+scrollY-h}if(y<scrollY){y=swapY?r.bottom:scrollY}}return[x,y]},getConstrainToXY:function(){var os={top:0,left:0,bottom:0,right:0};return function(el,local,offsets,proposedXY){el=Ext.get(el);offsets=offsets?Ext.applyIf(offsets,os):os;var vw,vh,vx=0,vy=0;if(el.dom==document.body||el.dom==document){vw=Ext.lib.Dom.getViewWidth();vh=Ext.lib.Dom.getViewHeight()}else{vw=el.dom.clientWidth;vh=el.dom.clientHeight;if(!local){var vxy=el.getXY();vx=vxy[0];vy=vxy[1]}}var s=el.getScroll();vx+=offsets.left+s.left;vy+=offsets.top+s.top;vw-=offsets.right;vh-=offsets.bottom;var vr=vx+vw;var vb=vy+vh;var xy=proposedXY||(!local?this.getXY():[this.getLeft(true),this.getTop(true)]);var x=xy[0],y=xy[1];var w=this.dom.offsetWidth,h=this.dom.offsetHeight;var moved=false;if((x+w)>vr){x=vr-w;moved=true}if((y+h)>vb){y=vb-h;moved=true}if(x<vx){x=vx;moved=true}if(y<vy){y=vy;moved=true}return moved?[x,y]:false}}(),adjustForConstraints:function(xy,parent,offsets){return this.getConstrainToXY(parent||document,false,offsets,xy)||xy},alignTo:function(element,position,offsets,animate){var xy=this.getAlignToXY(element,position,offsets);this.setXY(xy,this.preanim(arguments,3));return this},anchorTo:function(el,alignment,offsets,animate,monitorScroll,callback){var action=function(){this.alignTo(el,alignment,offsets,animate);Ext.callback(callback,this)};Ext.EventManager.onWindowResize(action,this);var tm=typeof monitorScroll;if(tm!=\"undefined\"){Ext.EventManager.on(window,\"scroll\",action,this,{buffer:tm==\"number\"?monitorScroll:50})}action.call(this);return this},clearOpacity:function(){if(window.ActiveXObject){if(typeof this.dom.style.filter==\"string\"&&(/alpha/i).test(this.dom.style.filter)){this.dom.style.filter=\"\"}}else{this.dom.style.opacity=\"\";this.dom.style[\"-moz-opacity\"]=\"\";this.dom.style[\"-khtml-opacity\"]=\"\"}return this},hide:function(animate){this.setVisible(false,this.preanim(arguments,0));return this},show:function(animate){this.setVisible(true,this.preanim(arguments,0));return this},addUnits:function(size){return Ext.Element.addUnits(size,this.defaultUnit)},update:function(html,loadScripts,callback){if(typeof html==\"undefined\"){html=\"\"}if(loadScripts!==true){this.dom.innerHTML=html;if(typeof callback==\"function\"){callback()}return this}var id=Ext.id();var dom=this.dom;html+=\"<span id=\\\"\"+id+\"\\\"></span>\";E.onAvailable(id,function(){var hd=document.getElementsByTagName(\"head\")[0];var re=/(?:<script([^>]*)?>)((\\n|\\r|.)*?)(?:<\\/script>)/ig;var srcRe=/\\ssrc=([\\'\\\"])(.*?)\\1/i;var typeRe=/\\stype=([\\'\\\"])(.*?)\\1/i;var match;while(match=re.exec(html)){var attrs=match[1];var srcMatch=attrs?attrs.match(srcRe):false;if(srcMatch&&srcMatch[2]){var s=document.createElement(\"script\");s.src=srcMatch[2];var typeMatch=attrs.match(typeRe);if(typeMatch&&typeMatch[2]){s.type=typeMatch[2]}hd.appendChild(s)}else{if(match[2]&&match[2].length>0){if(window.execScript){window.execScript(match[2])}else{window.eval(match[2])}}}}var el=document.getElementById(id);if(el){Ext.removeNode(el)}if(typeof callback==\"function\"){callback()}});dom.innerHTML=html.replace(/(?:<script.*?>)((\\n|\\r|.)*?)(?:<\\/script>)/ig,\"\");return this},load:function(){var um=this.getUpdater();um.update.apply(um,arguments);return this},getUpdater:function(){if(!this.updateManager){this.updateManager=new Ext.Updater(this)}return this.updateManager},unselectable:function(){this.dom.unselectable=\"on\";this.swallowEvent(\"selectstart\",true);this.applyStyles(\"-moz-user-select:none;-khtml-user-select:none;\");this.addClass(\"x-unselectable\");return this},getCenterXY:function(){return this.getAlignToXY(document,\"c-c\")},center:function(centerIn){this.alignTo(centerIn||document,\"c-c\");return this},isBorderBox:function(){return noBoxAdjust[this.dom.tagName.toLowerCase()]||Ext.isBorderBox},getBox:function(contentBox,local){var xy;if(!local){xy=this.getXY()}else{var left=parseInt(this.getStyle(\"left\"),10)||0;var top=parseInt(this.getStyle(\"top\"),10)||0;xy=[left,top]}var el=this.dom,w=el.offsetWidth,h=el.offsetHeight,bx;if(!contentBox){bx={x:xy[0],y:xy[1],0:xy[0],1:xy[1],width:w,height:h}}else{var l=this.getBorderWidth(\"l\")+this.getPadding(\"l\");var r=this.getBorderWidth(\"r\")+this.getPadding(\"r\");var t=this.getBorderWidth(\"t\")+this.getPadding(\"t\");var b=this.getBorderWidth(\"b\")+this.getPadding(\"b\");bx={x:xy[0]+l,y:xy[1]+t,0:xy[0]+l,1:xy[1]+t,width:w-(l+r),height:h-(t+b)}}bx.right=bx.x+bx.width;bx.bottom=bx.y+bx.height;return bx},getFrameWidth:function(sides,onlyContentBox){return onlyContentBox&&Ext.isBorderBox?0:(this.getPadding(sides)+this.getBorderWidth(sides))},setBox:function(box,adjust,animate){var w=box.width,h=box.height;if((adjust&&!this.autoBoxAdjust)&&!this.isBorderBox()){w-=(this.getBorderWidth(\"lr\")+this.getPadding(\"lr\"));h-=(this.getBorderWidth(\"tb\")+this.getPadding(\"tb\"))}this.setBounds(box.x,box.y,w,h,this.preanim(arguments,2));return this},repaint:function(){var dom=this.dom;this.addClass(\"x-repaint\");setTimeout(function(){Ext.get(dom).removeClass(\"x-repaint\")},1);return this},getMargins:function(side){if(!side){return{top:parseInt(this.getStyle(\"margin-top\"),10)||0,left:parseInt(this.getStyle(\"margin-left\"),10)||0,bottom:parseInt(this.getStyle(\"margin-bottom\"),10)||0,right:parseInt(this.getStyle(\"margin-right\"),10)||0}}else{return this.addStyles(side,El.margins)}},addStyles:function(sides,styles){var val=0,v,w;for(var i=0,len=sides.length;i<len;i++){v=this.getStyle(styles[sides.charAt(i)]);if(v){w=parseInt(v,10);if(w){val+=(w>=0?w:-1*w)}}}return val},createProxy:function(config,renderTo,matchBox){config=typeof config==\"object\"?config:{tag:\"div\",cls:config};var proxy;if(renderTo){proxy=Ext.DomHelper.append(renderTo,config,true)}else{proxy=Ext.DomHelper.insertBefore(this.dom,config,true)}if(matchBox){proxy.setBox(this.getBox())}return proxy},mask:function(msg,msgCls){if(this.getStyle(\"position\")==\"static\"){this.setStyle(\"position\",\"relative\")}if(this._maskMsg){this._maskMsg.remove()}if(this._mask){this._mask.remove()}this._mask=Ext.DomHelper.append(this.dom,{cls:\"ext-el-mask\"},true);this.addClass(\"x-masked\");this._mask.setDisplayed(true);if(typeof msg==\"string\"){this._maskMsg=Ext.DomHelper.append(this.dom,{cls:\"ext-el-mask-msg\",cn:{tag:\"div\"}},true);var mm=this._maskMsg;mm.dom.className=msgCls?\"ext-el-mask-msg \"+msgCls:\"ext-el-mask-msg\";mm.dom.firstChild.innerHTML=msg;mm.setDisplayed(true);mm.center(this)}if(Ext.isIE&&!(Ext.isIE7&&Ext.isStrict)&&this.getStyle(\"height\")==\"auto\"){this._mask.setSize(this.dom.clientWidth,this.getHeight())}return this._mask},unmask:function(){if(this._mask){if(this._maskMsg){this._maskMsg.remove();delete this._maskMsg}this._mask.remove();delete this._mask}this.removeClass(\"x-masked\")},isMasked:function(){return this._mask&&this._mask.isVisible()},createShim:function(){var el=document.createElement(\"iframe\");el.frameBorder=\"no\";el.className=\"ext-shim\";if(Ext.isIE&&Ext.isSecure){el.src=Ext.SSL_SECURE_URL}var shim=Ext.get(this.dom.parentNode.insertBefore(el,this.dom));shim.autoBoxAdjust=false;return shim},remove:function(){Ext.removeNode(this.dom);delete El.cache[this.dom.id]},hover:function(overFn,outFn,scope){var preOverFn=function(e){if(!e.within(this,true)){overFn.apply(scope||this,arguments)}};var preOutFn=function(e){if(!e.within(this,true)){outFn.apply(scope||this,arguments)}};this.on(\"mouseover\",preOverFn,this.dom);this.on(\"mouseout\",preOutFn,this.dom);return this},addClassOnOver:function(className,preventFlicker){this.hover(function(){Ext.fly(this,\"_internal\").addClass(className)},function(){Ext.fly(this,\"_internal\").removeClass(className)});return this},addClassOnFocus:function(className){this.on(\"focus\",function(){Ext.fly(this,\"_internal\").addClass(className)},this.dom);this.on(\"blur\",function(){Ext.fly(this,\"_internal\").removeClass(className)},this.dom);return this},addClassOnClick:function(className){var dom=this.dom;this.on(\"mousedown\",function(){Ext.fly(dom,\"_internal\").addClass(className);var d=Ext.getDoc();var fn=function(){Ext.fly(dom,\"_internal\").removeClass(className);d.removeListener(\"mouseup\",fn)};d.on(\"mouseup\",fn)});return this},swallowEvent:function(eventName,preventDefault){var fn=function(e){e.stopPropagation();if(preventDefault){e.preventDefault()}};if(Ext.isArray(eventName)){for(var i=0,len=eventName.length;i<len;i++){this.on(eventName[i],fn)}return this}this.on(eventName,fn);return this},parent:function(selector,returnDom){return this.matchNode(\"parentNode\",\"parentNode\",selector,returnDom)},next:function(selector,returnDom){return this.matchNode(\"nextSibling\",\"nextSibling\",selector,returnDom)},prev:function(selector,returnDom){return this.matchNode(\"previousSibling\",\"previousSibling\",selector,returnDom)},first:function(selector,returnDom){return this.matchNode(\"nextSibling\",\"firstChild\",selector,returnDom)},last:function(selector,returnDom){return this.matchNode(\"previousSibling\",\"lastChild\",selector,returnDom)},matchNode:function(dir,start,selector,returnDom){var n=this.dom[start];while(n){if(n.nodeType==1&&(!selector||Ext.DomQuery.is(n,selector))){return !returnDom?Ext.get(n):n}n=n[dir]}return null},appendChild:function(el){el=Ext.get(el);el.appendTo(this);return this},createChild:function(config,insertBefore,returnDom){config=config||{tag:\"div\"};if(insertBefore){return Ext.DomHelper.insertBefore(insertBefore,config,returnDom!==true)}return Ext.DomHelper[!this.dom.firstChild?\"overwrite\":\"append\"](this.dom,config,returnDom!==true)},appendTo:function(el){el=Ext.getDom(el);el.appendChild(this.dom);return this},insertBefore:function(el){el=Ext.getDom(el);el.parentNode.insertBefore(this.dom,el);return this},insertAfter:function(el){el=Ext.getDom(el);el.parentNode.insertBefore(this.dom,el.nextSibling);return this},insertFirst:function(el,returnDom){el=el||{};if(typeof el==\"object\"&&!el.nodeType&&!el.dom){return this.createChild(el,this.dom.firstChild,returnDom)}else{el=Ext.getDom(el);this.dom.insertBefore(el,this.dom.firstChild);return !returnDom?Ext.get(el):el}},insertSibling:function(el,where,returnDom){var rt;if(Ext.isArray(el)){for(var i=0,len=el.length;i<len;i++){rt=this.insertSibling(el[i],where,returnDom)}return rt}where=where?where.toLowerCase():\"before\";el=el||{};var refNode=where==\"before\"?this.dom:this.dom.nextSibling;if(typeof el==\"object\"&&!el.nodeType&&!el.dom){if(where==\"after\"&&!this.dom.nextSibling){rt=Ext.DomHelper.append(this.dom.parentNode,el,!returnDom)}else{rt=Ext.DomHelper[where==\"after\"?\"insertAfter\":\"insertBefore\"](this.dom,el,!returnDom)}}else{rt=this.dom.parentNode.insertBefore(Ext.getDom(el),refNode);if(!returnDom){rt=Ext.get(rt)}}return rt},wrap:function(config,returnDom){if(!config){config={tag:\"div\"}}var newEl=Ext.DomHelper.insertBefore(this.dom,config,!returnDom);newEl.dom?newEl.dom.appendChild(this.dom):newEl.appendChild(this.dom);return newEl},replace:function(el){el=Ext.get(el);this.insertBefore(el);el.remove();return this},replaceWith:function(el){if(typeof el==\"object\"&&!el.nodeType&&!el.dom){el=this.insertSibling(el,\"before\")}else{el=Ext.getDom(el);this.dom.parentNode.insertBefore(el,this.dom)}El.uncache(this.id);this.dom.parentNode.removeChild(this.dom);this.dom=el;this.id=Ext.id(el);El.cache[this.id]=this;return this},insertHtml:function(where,html,returnEl){var el=Ext.DomHelper.insertHtml(where,this.dom,html);return returnEl?Ext.get(el):el},set:function(o,useSet){var el=this.dom;useSet=typeof useSet==\"undefined\"?(el.setAttribute?true:false):useSet;for(var attr in o){if(attr==\"style\"||typeof o[attr]==\"function\"){continue}if(attr==\"cls\"){el.className=o[\"cls\"]}else{if(o.hasOwnProperty(attr)){if(useSet){el.setAttribute(attr,o[attr])}else{el[attr]=o[attr]}}}}if(o.style){Ext.DomHelper.applyStyles(el,o.style)}return this},addKeyListener:function(key,fn,scope){var config;if(typeof key!=\"object\"||Ext.isArray(key)){config={key:key,fn:fn,scope:scope}}else{config={key:key.key,shift:key.shift,ctrl:key.ctrl,alt:key.alt,fn:fn,scope:scope}}return new Ext.KeyMap(this,config)},addKeyMap:function(config){return new Ext.KeyMap(this,config)},isScrollable:function(){var dom=this.dom;return dom.scrollHeight>dom.clientHeight||dom.scrollWidth>dom.clientWidth},scrollTo:function(side,value,animate){var prop=side.toLowerCase()==\"left\"?\"scrollLeft\":\"scrollTop\";if(!animate||!A){this.dom[prop]=value}else{var to=prop==\"scrollLeft\"?[value,this.dom.scrollTop]:[this.dom.scrollLeft,value];this.anim({scroll:{\"to\":to}},this.preanim(arguments,2),\"scroll\")}return this},scroll:function(direction,distance,animate){if(!this.isScrollable()){return }var el=this.dom;var l=el.scrollLeft,t=el.scrollTop;var w=el.scrollWidth,h=el.scrollHeight;var cw=el.clientWidth,ch=el.clientHeight;direction=direction.toLowerCase();var scrolled=false;var a=this.preanim(arguments,2);switch(direction){case\"l\":case\"left\":if(w-l>cw){var v=Math.min(l+distance,w-cw);this.scrollTo(\"left\",v,a);scrolled=true}break;case\"r\":case\"right\":if(l>0){var v=Math.max(l-distance,0);this.scrollTo(\"left\",v,a);scrolled=true}break;case\"t\":case\"top\":case\"up\":if(t>0){var v=Math.max(t-distance,0);this.scrollTo(\"top\",v,a);scrolled=true}break;case\"b\":case\"bottom\":case\"down\":if(h-t>ch){var v=Math.min(t+distance,h-ch);this.scrollTo(\"top\",v,a);scrolled=true}break}return scrolled},translatePoints:function(x,y){if(typeof x==\"object\"||Ext.isArray(x)){y=x[1];x=x[0]}var p=this.getStyle(\"position\");var o=this.getXY();var l=parseInt(this.getStyle(\"left\"),10);var t=parseInt(this.getStyle(\"top\"),10);if(isNaN(l)){l=(p==\"relative\")?0:this.dom.offsetLeft}if(isNaN(t)){t=(p==\"relative\")?0:this.dom.offsetTop}return{left:(x-o[0]+l),top:(y-o[1]+t)}},getScroll:function(){var d=this.dom,doc=document;if(d==doc||d==doc.body){var l,t;if(Ext.isIE&&Ext.isStrict){l=doc.documentElement.scrollLeft||(doc.body.scrollLeft||0);t=doc.documentElement.scrollTop||(doc.body.scrollTop||0)}else{l=window.pageXOffset||(doc.body.scrollLeft||0);t=window.pageYOffset||(doc.body.scrollTop||0)}return{left:l,top:t}}else{return{left:d.scrollLeft,top:d.scrollTop}}},getColor:function(attr,defaultValue,prefix){var v=this.getStyle(attr);if(!v||v==\"transparent\"||v==\"inherit\"){return defaultValue}var color=typeof prefix==\"undefined\"?\"#\":prefix;if(v.substr(0,4)==\"rgb(\"){var rvs=v.slice(4,v.length-1).split(\",\");for(var i=0;i<3;i++){var h=parseInt(rvs[i]);var s=h.toString(16);if(h<16){s=\"0\"+s}color+=s}}else{if(v.substr(0,1)==\"#\"){if(v.length==4){for(var i=1;i<4;i++){var c=v.charAt(i);color+=c+c}}else{if(v.length==7){color+=v.substr(1)}}}}return(color.length>5?color.toLowerCase():defaultValue)},boxWrap:function(cls){cls=cls||\"x-box\";var el=Ext.get(this.insertHtml(\"beforeBegin\",String.format(\"<div class=\\\"{0}\\\">\"+El.boxMarkup+\"</div>\",cls)));el.child(\".\"+cls+\"-mc\").dom.appendChild(this.dom);return el},getAttributeNS:Ext.isIE?function(ns,name){var d=this.dom;var type=typeof d[ns+\":\"+name];if(type!=\"undefined\"&&type!=\"unknown\"){return d[ns+\":\"+name]}return d[name]}:function(ns,name){var d=this.dom;return d.getAttributeNS(ns,name)||d.getAttribute(ns+\":\"+name)||d.getAttribute(name)||d[name]},getTextWidth:function(text,min,max){return(Ext.util.TextMetrics.measure(this.dom,Ext.value(text,this.dom.innerHTML,true)).width).constrain(min||0,max||1000000)}};var ep=El.prototype;ep.on=ep.addListener;ep.mon=ep.addListener;ep.getUpdateManager=ep.getUpdater;ep.un=ep.removeListener;ep.autoBoxAdjust=true;El.unitPattern=/\\d+(px|em|%|en|ex|pt|in|cm|mm|pc)$/i;El.addUnits=function(v,defaultUnit){if(v===\"\"||v==\"auto\"){return v}if(v===undefined){return\"\"}if(typeof v==\"number\"||!El.unitPattern.test(v)){return v+(defaultUnit||\"px\")}return v};El.boxMarkup=\"<div class=\\\"{0}-tl\\\"><div class=\\\"{0}-tr\\\"><div class=\\\"{0}-tc\\\"></div></div></div><div class=\\\"{0}-ml\\\"><div class=\\\"{0}-mr\\\"><div class=\\\"{0}-mc\\\"></div></div></div><div class=\\\"{0}-bl\\\"><div class=\\\"{0}-br\\\"><div class=\\\"{0}-bc\\\"></div></div></div>\";El.VISIBILITY=1;El.DISPLAY=2;El.borders={l:\"border-left-width\",r:\"border-right-width\",t:\"border-top-width\",b:\"border-bottom-width\"};El.paddings={l:\"padding-left\",r:\"padding-right\",t:\"padding-top\",b:\"padding-bottom\"};El.margins={l:\"margin-left\",r:\"margin-right\",t:\"margin-top\",b:\"margin-bottom\"};El.cache={};var docEl;El.get=function(el){var ex,elm,id;if(!el){return null}if(typeof el==\"string\"){if(!(elm=document.getElementById(el))){return null}if(ex=El.cache[el]){ex.dom=elm}else{ex=El.cache[el]=new El(elm)}return ex}else{if(el.tagName){if(!(id=el.id)){id=Ext.id(el)}if(ex=El.cache[id]){ex.dom=el}else{ex=El.cache[id]=new El(el)}return ex}else{if(el instanceof El){if(el!=docEl){el.dom=document.getElementById(el.id)||el.dom;El.cache[el.id]=el}return el}else{if(el.isComposite){return el}else{if(Ext.isArray(el)){return El.select(el)}else{if(el==document){if(!docEl){var f=function(){};f.prototype=El.prototype;docEl=new f();docEl.dom=document}return docEl}}}}}}return null};El.uncache=function(el){for(var i=0,a=arguments,len=a.length;i<len;i++){if(a[i]){delete El.cache[a[i].id||a[i]]}}};El.garbageCollect=function(){if(!Ext.enableGarbageCollector){clearInterval(El.collectorThread);return }for(var eid in El.cache){var el=El.cache[eid],d=el.dom;if(!d||!d.parentNode||(!d.offsetParent&&!document.getElementById(eid))){delete El.cache[eid];if(d&&Ext.enableListenerCollection){E.purgeElement(d)}}}};El.collectorThreadId=setInterval(El.garbageCollect,30000);var flyFn=function(){};flyFn.prototype=El.prototype;var _cls=new flyFn();El.Flyweight=function(dom){this.dom=dom};El.Flyweight.prototype=_cls;El.Flyweight.prototype.isFlyweight=true;El._flyweights={};El.fly=function(el,named){named=named||\"_global\";el=Ext.getDom(el);if(!el){return null}if(!El._flyweights[named]){El._flyweights[named]=new El.Flyweight()}El._flyweights[named].dom=el;return El._flyweights[named]};Ext.get=El.get;Ext.fly=El.fly;var noBoxAdjust=Ext.isStrict?{select:1}:{input:1,select:1,textarea:1};if(Ext.isIE||Ext.isGecko){noBoxAdjust[\"button\"]=1}Ext.EventManager.on(window,\"unload\",function(){delete El.cache;delete El._flyweights})})();\nExt.enableFx=true;Ext.Fx={slideIn:function(A,C){var B=this.getFxEl();C=C||{};B.queueFx(C,function(){A=A||\"t\";this.fixDisplay();var D=this.getFxRestore();var I=this.getBox();this.setSize(I);var F=this.fxWrap(D.pos,C,\"hidden\");var K=this.dom.style;K.visibility=\"visible\";K.position=\"absolute\";var E=function(){B.fxUnwrap(F,D.pos,C);K.width=D.width;K.height=D.height;B.afterFx(C)};var J,L={to:[I.x,I.y]},H={to:I.width},G={to:I.height};switch(A.toLowerCase()){case\"t\":F.setSize(I.width,0);K.left=K.bottom=\"0\";J={height:G};break;case\"l\":F.setSize(0,I.height);K.right=K.top=\"0\";J={width:H};break;case\"r\":F.setSize(0,I.height);F.setX(I.right);K.left=K.top=\"0\";J={width:H,points:L};break;case\"b\":F.setSize(I.width,0);F.setY(I.bottom);K.left=K.top=\"0\";J={height:G,points:L};break;case\"tl\":F.setSize(0,0);K.right=K.bottom=\"0\";J={width:H,height:G};break;case\"bl\":F.setSize(0,0);F.setY(I.y+I.height);K.right=K.top=\"0\";J={width:H,height:G,points:L};break;case\"br\":F.setSize(0,0);F.setXY([I.right,I.bottom]);K.left=K.top=\"0\";J={width:H,height:G,points:L};break;case\"tr\":F.setSize(0,0);F.setX(I.x+I.width);K.left=K.bottom=\"0\";J={width:H,height:G,points:L};break}this.dom.style.visibility=\"visible\";F.show();arguments.callee.anim=F.fxanim(J,C,\"motion\",0.5,\"easeOut\",E)});return this},slideOut:function(A,C){var B=this.getFxEl();C=C||{};B.queueFx(C,function(){A=A||\"t\";var I=this.getFxRestore();var D=this.getBox();this.setSize(D);var G=this.fxWrap(I.pos,C,\"visible\");var F=this.dom.style;F.visibility=\"visible\";F.position=\"absolute\";G.setSize(D);var J=function(){if(C.useDisplay){B.setDisplayed(false)}else{B.hide()}B.fxUnwrap(G,I.pos,C);F.width=I.width;F.height=I.height;B.afterFx(C)};var E,H={to:0};switch(A.toLowerCase()){case\"t\":F.left=F.bottom=\"0\";E={height:H};break;case\"l\":F.right=F.top=\"0\";E={width:H};break;case\"r\":F.left=F.top=\"0\";E={width:H,points:{to:[D.right,D.y]}};break;case\"b\":F.left=F.top=\"0\";E={height:H,points:{to:[D.x,D.bottom]}};break;case\"tl\":F.right=F.bottom=\"0\";E={width:H,height:H};break;case\"bl\":F.right=F.top=\"0\";E={width:H,height:H,points:{to:[D.x,D.bottom]}};break;case\"br\":F.left=F.top=\"0\";E={width:H,height:H,points:{to:[D.x+D.width,D.bottom]}};break;case\"tr\":F.left=F.bottom=\"0\";E={width:H,height:H,points:{to:[D.right,D.y]}};break}arguments.callee.anim=G.fxanim(E,C,\"motion\",0.5,\"easeOut\",J)});return this},puff:function(B){var A=this.getFxEl();B=B||{};A.queueFx(B,function(){this.clearOpacity();this.show();var F=this.getFxRestore();var D=this.dom.style;var G=function(){if(B.useDisplay){A.setDisplayed(false)}else{A.hide()}A.clearOpacity();A.setPositioning(F.pos);D.width=F.width;D.height=F.height;D.fontSize=\"\";A.afterFx(B)};var E=this.getWidth();var C=this.getHeight();arguments.callee.anim=this.fxanim({width:{to:this.adjustWidth(E*2)},height:{to:this.adjustHeight(C*2)},points:{by:[-(E*0.5),-(C*0.5)]},opacity:{to:0},fontSize:{to:200,unit:\"%\"}},B,\"motion\",0.5,\"easeOut\",G)});return this},switchOff:function(B){var A=this.getFxEl();B=B||{};A.queueFx(B,function(){this.clearOpacity();this.clip();var D=this.getFxRestore();var C=this.dom.style;var E=function(){if(B.useDisplay){A.setDisplayed(false)}else{A.hide()}A.clearOpacity();A.setPositioning(D.pos);C.width=D.width;C.height=D.height;A.afterFx(B)};this.fxanim({opacity:{to:0.3}},null,null,0.1,null,function(){this.clearOpacity();(function(){this.fxanim({height:{to:1},points:{by:[0,this.getHeight()*0.5]}},B,\"motion\",0.3,\"easeIn\",E)}).defer(100,this)})});return this},highlight:function(A,C){var B=this.getFxEl();C=C||{};B.queueFx(C,function(){A=A||\"ffff9c\";var D=C.attr||\"backgroundColor\";this.clearOpacity();this.show();var G=this.getColor(D);var H=this.dom.style[D];var F=(C.endColor||G)||\"ffffff\";var I=function(){B.dom.style[D]=H;B.afterFx(C)};var E={};E[D]={from:A,to:F};arguments.callee.anim=this.fxanim(E,C,\"color\",1,\"easeIn\",I)});return this},frame:function(A,C,D){var B=this.getFxEl();D=D||{};B.queueFx(D,function(){A=A||\"#C3DAF9\";if(A.length==6){A=\"#\"+A}C=C||1;var G=D.duration||1;this.show();var E=this.getBox();var F=function(){var H=Ext.getBody().createChild({style:{visbility:\"hidden\",position:\"absolute\",\"z-index\":\"35000\",border:\"0px solid \"+A}});var I=Ext.isBorderBox?2:1;H.animate({top:{from:E.y,to:E.y-20},left:{from:E.x,to:E.x-20},borderWidth:{from:0,to:10},opacity:{from:1,to:0},height:{from:E.height,to:(E.height+(20*I))},width:{from:E.width,to:(E.width+(20*I))}},G,function(){H.remove();if(--C>0){F()}else{B.afterFx(D)}})};F.call(this)});return this},pause:function(C){var A=this.getFxEl();var B={};A.queueFx(B,function(){setTimeout(function(){A.afterFx(B)},C*1000)});return this},fadeIn:function(B){var A=this.getFxEl();B=B||{};A.queueFx(B,function(){this.setOpacity(0);this.fixDisplay();this.dom.style.visibility=\"visible\";var C=B.endOpacity||1;arguments.callee.anim=this.fxanim({opacity:{to:C}},B,null,0.5,\"easeOut\",function(){if(C==1){this.clearOpacity()}A.afterFx(B)})});return this},fadeOut:function(B){var A=this.getFxEl();B=B||{};A.queueFx(B,function(){arguments.callee.anim=this.fxanim({opacity:{to:B.endOpacity||0}},B,null,0.5,\"easeOut\",function(){if(this.visibilityMode==Ext.Element.DISPLAY||B.useDisplay){this.dom.style.display=\"none\"}else{this.dom.style.visibility=\"hidden\"}this.clearOpacity();A.afterFx(B)})});return this},scale:function(A,B,C){this.shift(Ext.apply({},C,{width:A,height:B}));return this},shift:function(B){var A=this.getFxEl();B=B||{};A.queueFx(B,function(){var E={},D=B.width,F=B.height,C=B.x,H=B.y,G=B.opacity;if(D!==undefined){E.width={to:this.adjustWidth(D)}}if(F!==undefined){E.height={to:this.adjustHeight(F)}}if(C!==undefined||H!==undefined){E.points={to:[C!==undefined?C:this.getX(),H!==undefined?H:this.getY()]}}if(G!==undefined){E.opacity={to:G}}if(B.xy!==undefined){E.points={to:B.xy}}arguments.callee.anim=this.fxanim(E,B,\"motion\",0.35,\"easeOut\",function(){A.afterFx(B)})});return this},ghost:function(A,C){var B=this.getFxEl();C=C||{};B.queueFx(C,function(){A=A||\"b\";var H=this.getFxRestore();var E=this.getWidth(),G=this.getHeight();var F=this.dom.style;var J=function(){if(C.useDisplay){B.setDisplayed(false)}else{B.hide()}B.clearOpacity();B.setPositioning(H.pos);F.width=H.width;F.height=H.height;B.afterFx(C)};var D={opacity:{to:0},points:{}},I=D.points;switch(A.toLowerCase()){case\"t\":I.by=[0,-G];break;case\"l\":I.by=[-E,0];break;case\"r\":I.by=[E,0];break;case\"b\":I.by=[0,G];break;case\"tl\":I.by=[-E,-G];break;case\"bl\":I.by=[-E,G];break;case\"br\":I.by=[E,G];break;case\"tr\":I.by=[E,-G];break}arguments.callee.anim=this.fxanim(D,C,\"motion\",0.5,\"easeOut\",J)});return this},syncFx:function(){this.fxDefaults=Ext.apply(this.fxDefaults||{},{block:false,concurrent:true,stopFx:false});return this},sequenceFx:function(){this.fxDefaults=Ext.apply(this.fxDefaults||{},{block:false,concurrent:false,stopFx:false});return this},nextFx:function(){var A=this.fxQueue[0];if(A){A.call(this)}},hasActiveFx:function(){return this.fxQueue&&this.fxQueue[0]},stopFx:function(){if(this.hasActiveFx()){var A=this.fxQueue[0];if(A&&A.anim&&A.anim.isAnimated()){this.fxQueue=[A];A.anim.stop(true)}}return this},beforeFx:function(A){if(this.hasActiveFx()&&!A.concurrent){if(A.stopFx){this.stopFx();return true}return false}return true},hasFxBlock:function(){var A=this.fxQueue;return A&&A[0]&&A[0].block},queueFx:function(C,A){if(!this.fxQueue){this.fxQueue=[]}if(!this.hasFxBlock()){Ext.applyIf(C,this.fxDefaults);if(!C.concurrent){var B=this.beforeFx(C);A.block=C.block;this.fxQueue.push(A);if(B){this.nextFx()}}else{A.call(this)}}return this},fxWrap:function(F,D,C){var B;if(!D.wrap||!(B=Ext.get(D.wrap))){var A;if(D.fixPosition){A=this.getXY()}var E=document.createElement(\"div\");E.style.visibility=C;B=Ext.get(this.dom.parentNode.insertBefore(E,this.dom));B.setPositioning(F);if(B.getStyle(\"position\")==\"static\"){B.position(\"relative\")}this.clearPositioning(\"auto\");B.clip();B.dom.appendChild(this.dom);if(A){B.setXY(A)}}return B},fxUnwrap:function(A,C,B){this.clearPositioning();this.setPositioning(C);if(!B.wrap){A.dom.parentNode.insertBefore(this.dom,A.dom);A.remove()}},getFxRestore:function(){var A=this.dom.style;return{pos:this.getPositioning(),width:A.width,height:A.height}},afterFx:function(A){if(A.afterStyle){this.applyStyles(A.afterStyle)}if(A.afterCls){this.addClass(A.afterCls)}if(A.remove===true){this.remove()}Ext.callback(A.callback,A.scope,[this]);if(!A.concurrent){this.fxQueue.shift();this.nextFx()}},getFxEl:function(){return Ext.get(this.dom)},fxanim:function(D,E,B,F,C,A){B=B||\"run\";E=E||{};var G=Ext.lib.Anim[B](this.dom,D,(E.duration||F)||0.35,(E.easing||C)||\"easeOut\",function(){Ext.callback(A,this)},this);E.anim=G;return G}};Ext.Fx.resize=Ext.Fx.scale;Ext.apply(Ext.Element.prototype,Ext.Fx);\nExt.CompositeElement=function(A){this.elements=[];this.addElements(A)};Ext.CompositeElement.prototype={isComposite:true,addElements:function(E){if(!E){return this}if(typeof E==\"string\"){E=Ext.Element.selectorFunction(E)}var D=this.elements;var B=D.length-1;for(var C=0,A=E.length;C<A;C++){D[++B]=Ext.get(E[C])}return this},fill:function(A){this.elements=[];this.add(A);return this},filter:function(A){var B=[];this.each(function(C){if(C.is(A)){B[B.length]=C.dom}});this.fill(B);return this},invoke:function(E,B){var D=this.elements;for(var C=0,A=D.length;C<A;C++){Ext.Element.prototype[E].apply(D[C],B)}return this},add:function(A){if(typeof A==\"string\"){this.addElements(Ext.Element.selectorFunction(A))}else{if(A.length!==undefined){this.addElements(A)}else{this.addElements([A])}}return this},each:function(E,D){var C=this.elements;for(var B=0,A=C.length;B<A;B++){if(E.call(D||C[B],C[B],this,B)===false){break}}return this},item:function(A){return this.elements[A]||null},first:function(){return this.item(0)},last:function(){return this.item(this.elements.length-1)},getCount:function(){return this.elements.length},contains:function(A){return this.indexOf(A)!==-1},indexOf:function(A){return this.elements.indexOf(Ext.get(A))},removeElement:function(D,F){if(Ext.isArray(D)){for(var C=0,A=D.length;C<A;C++){this.removeElement(D[C])}return this}var B=typeof D==\"number\"?D:this.indexOf(D);if(B!==-1&&this.elements[B]){if(F){var E=this.elements[B];if(E.dom){E.remove()}else{Ext.removeNode(E)}}this.elements.splice(B,1)}return this},replaceElement:function(D,C,A){var B=typeof D==\"number\"?D:this.indexOf(D);if(B!==-1){if(A){this.elements[B].replaceWith(C)}else{this.elements.splice(B,1,Ext.get(C))}}return this},clear:function(){this.elements=[]}};(function(){Ext.CompositeElement.createCall=function(B,C){if(!B[C]){B[C]=function(){return this.invoke(C,arguments)}}};for(var A in Ext.Element.prototype){if(typeof Ext.Element.prototype[A]==\"function\"){Ext.CompositeElement.createCall(Ext.CompositeElement.prototype,A)}}})();Ext.CompositeElementLite=function(A){Ext.CompositeElementLite.superclass.constructor.call(this,A);this.el=new Ext.Element.Flyweight()};Ext.extend(Ext.CompositeElementLite,Ext.CompositeElement,{addElements:function(E){if(E){if(Ext.isArray(E)){this.elements=this.elements.concat(E)}else{var D=this.elements;var B=D.length-1;for(var C=0,A=E.length;C<A;C++){D[++B]=E[C]}}}return this},invoke:function(F,B){var D=this.elements;var E=this.el;for(var C=0,A=D.length;C<A;C++){E.dom=D[C];Ext.Element.prototype[F].apply(E,B)}return this},item:function(A){if(!this.elements[A]){return null}this.el.dom=this.elements[A];return this.el},addListener:function(B,G,F,E){var D=this.elements;for(var C=0,A=D.length;C<A;C++){Ext.EventManager.on(D[C],B,G,F||D[C],E)}return this},each:function(F,E){var C=this.elements;var D=this.el;for(var B=0,A=C.length;B<A;B++){D.dom=C[B];if(F.call(E||D,D,this,B)===false){break}}return this},indexOf:function(A){return this.elements.indexOf(Ext.getDom(A))},replaceElement:function(D,C,A){var B=typeof D==\"number\"?D:this.indexOf(D);if(B!==-1){C=Ext.getDom(C);if(A){var E=this.elements[B];E.parentNode.insertBefore(C,E);Ext.removeNode(E)}this.elements.splice(B,1,C)}return this}});Ext.CompositeElementLite.prototype.on=Ext.CompositeElementLite.prototype.addListener;if(Ext.DomQuery){Ext.Element.selectorFunction=Ext.DomQuery.select}Ext.Element.select=function(A,D,B){var C;if(typeof A==\"string\"){C=Ext.Element.selectorFunction(A,B)}else{if(A.length!==undefined){C=A}else{throw\"Invalid selector\"}}if(D===true){return new Ext.CompositeElement(C)}else{return new Ext.CompositeElementLite(C)}};Ext.select=Ext.Element.select;\nExt.data.Connection=function(A){Ext.apply(this,A);this.addEvents(\"beforerequest\",\"requestcomplete\",\"requestexception\");Ext.data.Connection.superclass.constructor.call(this)};Ext.extend(Ext.data.Connection,Ext.util.Observable,{timeout:30000,autoAbort:false,disableCaching:true,request:function(E){if(this.fireEvent(\"beforerequest\",this,E)!==false){var C=E.params;if(typeof C==\"function\"){C=C.call(E.scope||window,E)}if(typeof C==\"object\"){C=Ext.urlEncode(C)}if(this.extraParams){var G=Ext.urlEncode(this.extraParams);C=C?(C+\"&\"+G):G}var B=E.url||this.url;if(typeof B==\"function\"){B=B.call(E.scope||window,E)}if(E.form){var D=Ext.getDom(E.form);B=B||D.action;var I=D.getAttribute(\"enctype\");if(E.isUpload||(I&&I.toLowerCase()==\"multipart/form-data\")){return this.doFormUpload(E,C,B)}var H=Ext.lib.Ajax.serializeForm(D);C=C?(C+\"&\"+H):H}var J=E.headers;if(this.defaultHeaders){J=Ext.apply(J||{},this.defaultHeaders);if(!E.headers){E.headers=J}}var F={success:this.handleResponse,failure:this.handleFailure,scope:this,argument:{options:E},timeout:E.timeout||this.timeout};var A=E.method||this.method||(C?\"POST\":\"GET\");if(A==\"GET\"&&(this.disableCaching&&E.disableCaching!==false)||E.disableCaching===true){B+=(B.indexOf(\"?\")!=-1?\"&\":\"?\")+\"_dc=\"+(new Date().getTime())}if(typeof E.autoAbort==\"boolean\"){if(E.autoAbort){this.abort()}}else{if(this.autoAbort!==false){this.abort()}}if((A==\"GET\"&&C)||E.xmlData||E.jsonData){B+=(B.indexOf(\"?\")!=-1?\"&\":\"?\")+C;C=\"\"}this.transId=Ext.lib.Ajax.request(A,B,F,C,E);return this.transId}else{Ext.callback(E.callback,E.scope,[E,null,null]);return null}},isLoading:function(A){if(A){return Ext.lib.Ajax.isCallInProgress(A)}else{return this.transId?true:false}},abort:function(A){if(A||this.isLoading()){Ext.lib.Ajax.abort(A||this.transId)}},handleResponse:function(A){this.transId=false;var B=A.argument.options;A.argument=B?B.argument:null;this.fireEvent(\"requestcomplete\",this,A,B);Ext.callback(B.success,B.scope,[A,B]);Ext.callback(B.callback,B.scope,[B,true,A])},handleFailure:function(A,C){this.transId=false;var B=A.argument.options;A.argument=B?B.argument:null;this.fireEvent(\"requestexception\",this,A,B,C);Ext.callback(B.failure,B.scope,[A,B]);Ext.callback(B.callback,B.scope,[B,false,A])},doFormUpload:function(E,A,B){var C=Ext.id();var F=document.createElement(\"iframe\");F.id=C;F.name=C;F.className=\"x-hidden\";if(Ext.isIE){F.src=Ext.SSL_SECURE_URL}document.body.appendChild(F);if(Ext.isIE){document.frames[C].name=C}var D=Ext.getDom(E.form);D.target=C;D.method=\"POST\";D.enctype=D.encoding=\"multipart/form-data\";if(B){D.action=B}var L,J;if(A){L=[];A=Ext.urlDecode(A,false);for(var H in A){if(A.hasOwnProperty(H)){J=document.createElement(\"input\");J.type=\"hidden\";J.name=H;J.value=A[H];D.appendChild(J);L.push(J)}}}function G(){var M={responseText:\"\",responseXML:null};M.argument=E?E.argument:null;try{var O;if(Ext.isIE){O=F.contentWindow.document}else{O=(F.contentDocument||window.frames[C].document)}if(O&&O.body){M.responseText=O.body.innerHTML}if(O&&O.XMLDocument){M.responseXML=O.XMLDocument}else{M.responseXML=O}}catch(N){}Ext.EventManager.removeListener(F,\"load\",G,this);this.fireEvent(\"requestcomplete\",this,M,E);Ext.callback(E.success,E.scope,[M,E]);Ext.callback(E.callback,E.scope,[E,true,M]);setTimeout(function(){Ext.removeNode(F)},100)}Ext.EventManager.on(F,\"load\",G,this);D.submit();if(L){for(var I=0,K=L.length;I<K;I++){Ext.removeNode(L[I])}}}});Ext.Ajax=new Ext.data.Connection({autoAbort:false,serializeForm:function(A){return Ext.lib.Ajax.serializeForm(A)}});\nExt.Updater=function(B,A){B=Ext.get(B);if(!A&&B.updateManager){return B.updateManager}this.el=B;this.defaultUrl=null;this.addEvents(\"beforeupdate\",\"update\",\"failure\");var C=Ext.Updater.defaults;this.sslBlankUrl=C.sslBlankUrl;this.disableCaching=C.disableCaching;this.indicatorText=C.indicatorText;this.showLoadIndicator=C.showLoadIndicator;this.timeout=C.timeout;this.loadScripts=C.loadScripts;this.transaction=null;this.autoRefreshProcId=null;this.refreshDelegate=this.refresh.createDelegate(this);this.updateDelegate=this.update.createDelegate(this);this.formUpdateDelegate=this.formUpdate.createDelegate(this);if(!this.renderer){this.renderer=new Ext.Updater.BasicRenderer()}Ext.Updater.superclass.constructor.call(this)};Ext.extend(Ext.Updater,Ext.util.Observable,{getEl:function(){return this.el},update:function(B,F,H,D){if(this.fireEvent(\"beforeupdate\",this.el,B,F)!==false){var G=this.method,A,C;if(typeof B==\"object\"){A=B;B=A.url;F=F||A.params;H=H||A.callback;D=D||A.discardUrl;C=A.scope;if(typeof A.method!=\"undefined\"){G=A.method}if(typeof A.nocache!=\"undefined\"){this.disableCaching=A.nocache}if(typeof A.text!=\"undefined\"){this.indicatorText=\"<div class=\\\"loading-indicator\\\">\"+A.text+\"</div>\"}if(typeof A.scripts!=\"undefined\"){this.loadScripts=A.scripts}if(typeof A.timeout!=\"undefined\"){this.timeout=A.timeout}}this.showLoading();if(!D){this.defaultUrl=B}if(typeof B==\"function\"){B=B.call(this)}G=G||(F?\"POST\":\"GET\");if(G==\"GET\"){B=this.prepareUrl(B)}var E=Ext.apply(A||{},{url:B,params:(typeof F==\"function\"&&C)?F.createDelegate(C):F,success:this.processSuccess,failure:this.processFailure,scope:this,callback:undefined,timeout:(this.timeout*1000),argument:{\"options\":A,\"url\":B,\"form\":null,\"callback\":H,\"scope\":C||window,\"params\":F}});this.transaction=Ext.Ajax.request(E)}},formUpdate:function(C,A,B,D){if(this.fireEvent(\"beforeupdate\",this.el,C,A)!==false){if(typeof A==\"function\"){A=A.call(this)}C=Ext.getDom(C);this.transaction=Ext.Ajax.request({form:C,url:A,success:this.processSuccess,failure:this.processFailure,scope:this,timeout:(this.timeout*1000),argument:{\"url\":A,\"form\":C,\"callback\":D,\"reset\":B}});this.showLoading.defer(1,this)}},refresh:function(A){if(this.defaultUrl==null){return }this.update(this.defaultUrl,null,A,true)},startAutoRefresh:function(B,C,D,E,A){if(A){this.update(C||this.defaultUrl,D,E,true)}if(this.autoRefreshProcId){clearInterval(this.autoRefreshProcId)}this.autoRefreshProcId=setInterval(this.update.createDelegate(this,[C||this.defaultUrl,D,E,true]),B*1000)},stopAutoRefresh:function(){if(this.autoRefreshProcId){clearInterval(this.autoRefreshProcId);delete this.autoRefreshProcId}},isAutoRefreshing:function(){return this.autoRefreshProcId?true:false},showLoading:function(){if(this.showLoadIndicator){this.el.update(this.indicatorText)}},prepareUrl:function(B){if(this.disableCaching){var A=\"_dc=\"+(new Date().getTime());if(B.indexOf(\"?\")!==-1){B+=\"&\"+A}else{B+=\"?\"+A}}return B},processSuccess:function(A){this.transaction=null;if(A.argument.form&&A.argument.reset){try{A.argument.form.reset()}catch(B){}}if(this.loadScripts){this.renderer.render(this.el,A,this,this.updateComplete.createDelegate(this,[A]))}else{this.renderer.render(this.el,A,this);this.updateComplete(A)}},updateComplete:function(A){this.fireEvent(\"update\",this.el,A);if(typeof A.argument.callback==\"function\"){A.argument.callback.call(A.argument.scope,this.el,true,A,A.argument.options)}},processFailure:function(A){this.transaction=null;this.fireEvent(\"failure\",this.el,A);if(typeof A.argument.callback==\"function\"){A.argument.callback.call(A.argument.scope,this.el,false,A,A.argument.options)}},setRenderer:function(A){this.renderer=A},getRenderer:function(){return this.renderer},setDefaultUrl:function(A){this.defaultUrl=A},abort:function(){if(this.transaction){Ext.Ajax.abort(this.transaction)}},isUpdating:function(){if(this.transaction){return Ext.Ajax.isLoading(this.transaction)}return false}});Ext.Updater.defaults={timeout:30,loadScripts:false,sslBlankUrl:(Ext.SSL_SECURE_URL||\"javascript:false\"),disableCaching:false,showLoadIndicator:true,indicatorText:\"<div class=\\\"loading-indicator\\\">Loading...</div>\"};Ext.Updater.updateElement=function(D,C,E,B){var A=Ext.get(D).getUpdater();Ext.apply(A,B);A.update(C,E,B?B.callback:null)};Ext.Updater.update=Ext.Updater.updateElement;Ext.Updater.BasicRenderer=function(){};Ext.Updater.BasicRenderer.prototype={render:function(C,A,B,D){C.update(A.responseText,B.loadScripts,D)}};Ext.UpdateManager=Ext.Updater;\nDate.parseFunctions={count:0};Date.parseRegexes=[];Date.formatFunctions={count:0};Date.prototype.dateFormat=function(B){if(Date.formatFunctions[B]==null){Date.createNewFormat(B)}var A=Date.formatFunctions[B];return this[A]()};Date.prototype.format=Date.prototype.dateFormat;Date.createNewFormat=function(format){var funcName=\"format\"+Date.formatFunctions.count++;Date.formatFunctions[format]=funcName;var code=\"Date.prototype.\"+funcName+\" = function(){return \";var special=false;var ch=\"\";for(var i=0;i<format.length;++i){ch=format.charAt(i);if(!special&&ch==\"\\\\\"){special=true}else{if(special){special=false;code+=\"'\"+String.escape(ch)+\"' + \"}else{code+=Date.getFormatCode(ch)}}}eval(code.substring(0,code.length-3)+\";}\")};Date.getFormatCode=function(D){switch(D){case\"d\":return\"String.leftPad(this.getDate(), 2, '0') + \";case\"D\":return\"Date.getShortDayName(this.getDay()) + \";case\"j\":return\"this.getDate() + \";case\"l\":return\"Date.dayNames[this.getDay()] + \";case\"N\":return\"(this.getDay() ? this.getDay() : 7) + \";case\"S\":return\"this.getSuffix() + \";case\"w\":return\"this.getDay() + \";case\"z\":return\"this.getDayOfYear() + \";case\"W\":return\"String.leftPad(this.getWeekOfYear(), 2, '0') + \";case\"F\":return\"Date.monthNames[this.getMonth()] + \";case\"m\":return\"String.leftPad(this.getMonth() + 1, 2, '0') + \";case\"M\":return\"Date.getShortMonthName(this.getMonth()) + \";case\"n\":return\"(this.getMonth() + 1) + \";case\"t\":return\"this.getDaysInMonth() + \";case\"L\":return\"(this.isLeapYear() ? 1 : 0) + \";case\"o\":return\"(this.getFullYear() + (this.getWeekOfYear() == 1 && this.getMonth() > 0 ? +1 : (this.getWeekOfYear() >= 52 && this.getMonth() < 11 ? -1 : 0))) + \";case\"Y\":return\"this.getFullYear() + \";case\"y\":return\"('' + this.getFullYear()).substring(2, 4) + \";case\"a\":return\"(this.getHours() < 12 ? 'am' : 'pm') + \";case\"A\":return\"(this.getHours() < 12 ? 'AM' : 'PM') + \";case\"g\":return\"((this.getHours() % 12) ? this.getHours() % 12 : 12) + \";case\"G\":return\"this.getHours() + \";case\"h\":return\"String.leftPad((this.getHours() % 12) ? this.getHours() % 12 : 12, 2, '0') + \";case\"H\":return\"String.leftPad(this.getHours(), 2, '0') + \";case\"i\":return\"String.leftPad(this.getMinutes(), 2, '0') + \";case\"s\":return\"String.leftPad(this.getSeconds(), 2, '0') + \";case\"u\":return\"String.leftPad(this.getMilliseconds(), 3, '0') + \";case\"O\":return\"this.getGMTOffset() + \";case\"P\":return\"this.getGMTOffset(true) + \";case\"T\":return\"this.getTimezone() + \";case\"Z\":return\"(this.getTimezoneOffset() * -60) + \";case\"c\":for(var F=Date.getFormatCode,G=\"Y-m-dTH:i:sP\",C=\"\",B=0,A=G.length;B<A;++B){var E=G.charAt(B);C+=E==\"T\"?\"'T' + \":F(E)}return C;case\"U\":return\"Math.round(this.getTime() / 1000) + \";default:return\"'\"+String.escape(D)+\"' + \"}};Date.parseDate=function(A,C){if(Date.parseFunctions[C]==null){Date.createParser(C)}var B=Date.parseFunctions[C];return Date[B](A)};Date.createParser=function(format){var funcName=\"parse\"+Date.parseFunctions.count++;var regexNum=Date.parseRegexes.length;var currentGroup=1;Date.parseFunctions[format]=funcName;var code=\"Date.\"+funcName+\" = function(input){\\n\"+\"var y = -1, m = -1, d = -1, h = -1, i = -1, s = -1, ms = -1, o, z, u, v;\\n\"+\"input = String(input);var d = new Date();\\n\"+\"y = d.getFullYear();\\n\"+\"m = d.getMonth();\\n\"+\"d = d.getDate();\\n\"+\"var results = input.match(Date.parseRegexes[\"+regexNum+\"]);\\n\"+\"if (results && results.length > 0) {\";var regex=\"\";var special=false;var ch=\"\";for(var i=0;i<format.length;++i){ch=format.charAt(i);if(!special&&ch==\"\\\\\"){special=true}else{if(special){special=false;regex+=String.escape(ch)}else{var obj=Date.formatCodeToRegex(ch,currentGroup);currentGroup+=obj.g;regex+=obj.s;if(obj.g&&obj.c){code+=obj.c}}}}code+=\"if (u)\\n\"+\"{v = new Date(u * 1000);}\"+\"else if (y >= 0 && m >= 0 && d > 0 && h >= 0 && i >= 0 && s >= 0 && ms >= 0)\\n\"+\"{v = new Date(y, m, d, h, i, s, ms);}\\n\"+\"else if (y >= 0 && m >= 0 && d > 0 && h >= 0 && i >= 0 && s >= 0)\\n\"+\"{v = new Date(y, m, d, h, i, s);}\\n\"+\"else if (y >= 0 && m >= 0 && d > 0 && h >= 0 && i >= 0)\\n\"+\"{v = new Date(y, m, d, h, i);}\\n\"+\"else if (y >= 0 && m >= 0 && d > 0 && h >= 0)\\n\"+\"{v = new Date(y, m, d, h);}\\n\"+\"else if (y >= 0 && m >= 0 && d > 0)\\n\"+\"{v = new Date(y, m, d);}\\n\"+\"else if (y >= 0 && m >= 0)\\n\"+\"{v = new Date(y, m);}\\n\"+\"else if (y >= 0)\\n\"+\"{v = new Date(y);}\\n\"+\"}return (v && (z || o))?\\n\"+\"    (z ? v.add(Date.SECOND, (v.getTimezoneOffset() * 60) + (z*1)) :\\n\"+\"        v.add(Date.HOUR, (v.getGMTOffset() / 100) + (o / -100))) : v\\n\"+\";}\";Date.parseRegexes[regexNum]=new RegExp(\"^\"+regex+\"$\",\"i\");eval(code)};Date.formatCodeToRegex=function(G,F){switch(G){case\"d\":return{g:1,c:\"d = parseInt(results[\"+F+\"], 10);\\n\",s:\"(\\\\d{2})\"};case\"D\":for(var C=[],E=0;E<7;C.push(Date.getShortDayName(E)),++E){}return{g:0,c:null,s:\"(?:\"+C.join(\"|\")+\")\"};case\"j\":return{g:1,c:\"d = parseInt(results[\"+F+\"], 10);\\n\",s:\"(\\\\d{1,2})\"};case\"l\":return{g:0,c:null,s:\"(?:\"+Date.dayNames.join(\"|\")+\")\"};case\"N\":return{g:0,c:null,s:\"[1-7]\"};case\"S\":return{g:0,c:null,s:\"(?:st|nd|rd|th)\"};case\"w\":return{g:0,c:null,s:\"[0-6]\"};case\"z\":return{g:0,c:null,s:\"(?:\\\\d{1,3}\"};case\"W\":return{g:0,c:null,s:\"(?:\\\\d{2})\"};case\"F\":return{g:1,c:\"m = parseInt(Date.getMonthNumber(results[\"+F+\"]), 10);\\n\",s:\"(\"+Date.monthNames.join(\"|\")+\")\"};case\"m\":return{g:1,c:\"m = parseInt(results[\"+F+\"], 10) - 1;\\n\",s:\"(\\\\d{2})\"};case\"M\":for(var C=[],E=0;E<12;C.push(Date.getShortMonthName(E)),++E){}return{g:1,c:\"m = parseInt(Date.getMonthNumber(results[\"+F+\"]), 10);\\n\",s:\"(\"+C.join(\"|\")+\")\"};case\"n\":return{g:1,c:\"m = parseInt(results[\"+F+\"], 10) - 1;\\n\",s:\"(\\\\d{1,2})\"};case\"t\":return{g:0,c:null,s:\"(?:\\\\d{2})\"};case\"L\":return{g:0,c:null,s:\"(?:1|0)\"};case\"o\":case\"Y\":return{g:1,c:\"y = parseInt(results[\"+F+\"], 10);\\n\",s:\"(\\\\d{4})\"};case\"y\":return{g:1,c:\"var ty = parseInt(results[\"+F+\"], 10);\\n\"+\"y = ty > Date.y2kYear ? 1900 + ty : 2000 + ty;\\n\",s:\"(\\\\d{1,2})\"};case\"a\":return{g:1,c:\"if (results[\"+F+\"] == 'am') {\\n\"+\"if (h == 12) { h = 0; }\\n\"+\"} else { if (h < 12) { h += 12; }}\",s:\"(am|pm)\"};case\"A\":return{g:1,c:\"if (results[\"+F+\"] == 'AM') {\\n\"+\"if (h == 12) { h = 0; }\\n\"+\"} else { if (h < 12) { h += 12; }}\",s:\"(AM|PM)\"};case\"g\":case\"G\":return{g:1,c:\"h = parseInt(results[\"+F+\"], 10);\\n\",s:\"(\\\\d{1,2})\"};case\"h\":case\"H\":return{g:1,c:\"h = parseInt(results[\"+F+\"], 10);\\n\",s:\"(\\\\d{2})\"};case\"i\":return{g:1,c:\"i = parseInt(results[\"+F+\"], 10);\\n\",s:\"(\\\\d{2})\"};case\"s\":return{g:1,c:\"s = parseInt(results[\"+F+\"], 10);\\n\",s:\"(\\\\d{2})\"};case\"u\":return{g:1,c:\"ms = parseInt(results[\"+F+\"], 10);\\n\",s:\"(\\\\d{3})\"};case\"O\":return{g:1,c:[\"o = results[\",F,\"];\\n\",\"var sn = o.substring(0,1);\\n\",\"var hr = o.substring(1,3)*1 + Math.floor(o.substring(3,5) / 60);\\n\",\"var mn = o.substring(3,5) % 60;\\n\",\"o = ((-12 <= (hr*60 + mn)/60) && ((hr*60 + mn)/60 <= 14))?\\n\",\"    (sn + String.leftPad(hr, 2, '0') + String.leftPad(mn, 2, '0')) : null;\\n\"].join(\"\"),s:\"([+-]\\\\d{4})\"};case\"P\":return{g:1,c:[\"o = results[\",F,\"];\\n\",\"var sn = o.substring(0,1);\\n\",\"var hr = o.substring(1,3)*1 + Math.floor(o.substring(4,6) / 60);\\n\",\"var mn = o.substring(4,6) % 60;\\n\",\"o = ((-12 <= (hr*60 + mn)/60) && ((hr*60 + mn)/60 <= 14))?\\n\",\"    (sn + String.leftPad(hr, 2, '0') + String.leftPad(mn, 2, '0')) : null;\\n\"].join(\"\"),s:\"([+-]\\\\d{2}:\\\\d{2})\"};case\"T\":return{g:0,c:null,s:\"[A-Z]{1,4}\"};case\"Z\":return{g:1,c:\"z = results[\"+F+\"] * 1;\\n\"+\"z = (-43200 <= z && z <= 50400)? z : null;\\n\",s:\"([+-]?\\\\d{1,5})\"};case\"c\":var H=Date.formatCodeToRegex,D=[];var A=[H(\"Y\",1),H(\"m\",2),H(\"d\",3),H(\"h\",4),H(\"i\",5),H(\"s\",6),H(\"P\",7)];for(var E=0,B=A.length;E<B;++E){D.push(A[E].c)}return{g:1,c:D.join(\"\"),s:A[0].s+\"-\"+A[1].s+\"-\"+A[2].s+\"T\"+A[3].s+\":\"+A[4].s+\":\"+A[5].s+A[6].s};case\"U\":return{g:1,c:\"u = parseInt(results[\"+F+\"], 10);\\n\",s:\"(-?\\\\d+)\"};default:return{g:0,c:null,s:Ext.escapeRe(G)}}};Date.prototype.getTimezone=function(){return this.toString().replace(/^.* (?:\\((.*)\\)|([A-Z]{1,4})(?:[\\-+][0-9]{4})?(?: -?\\d+)?)$/,\"$1$2\").replace(/[^A-Z]/g,\"\")};Date.prototype.getGMTOffset=function(A){return(this.getTimezoneOffset()>0?\"-\":\"+\")+String.leftPad(Math.abs(Math.floor(this.getTimezoneOffset()/60)),2,\"0\")+(A?\":\":\"\")+String.leftPad(this.getTimezoneOffset()%60,2,\"0\")};Date.prototype.getDayOfYear=function(){var A=0;Date.daysInMonth[1]=this.isLeapYear()?29:28;for(var B=0;B<this.getMonth();++B){A+=Date.daysInMonth[B]}return A+this.getDate()-1};Date.prototype.getWeekOfYear=function(){var B=86400000;var C=7*B;var D=Date.UTC(this.getFullYear(),this.getMonth(),this.getDate()+3)/B;var A=Math.floor(D/7);var E=new Date(A*C).getUTCFullYear();return A-Math.floor(Date.UTC(E,0,7)/C)+1};Date.prototype.isLeapYear=function(){var A=this.getFullYear();return !!((A&3)==0&&(A%100||(A%400==0&&A)))};Date.prototype.getFirstDayOfMonth=function(){var A=(this.getDay()-(this.getDate()-1))%7;return(A<0)?(A+7):A};Date.prototype.getLastDayOfMonth=function(){var A=(this.getDay()+(Date.daysInMonth[this.getMonth()]-this.getDate()))%7;return(A<0)?(A+7):A};Date.prototype.getFirstDateOfMonth=function(){return new Date(this.getFullYear(),this.getMonth(),1)};Date.prototype.getLastDateOfMonth=function(){return new Date(this.getFullYear(),this.getMonth(),this.getDaysInMonth())};Date.prototype.getDaysInMonth=function(){Date.daysInMonth[1]=this.isLeapYear()?29:28;return Date.daysInMonth[this.getMonth()]};Date.prototype.getSuffix=function(){switch(this.getDate()){case 1:case 21:case 31:return\"st\";case 2:case 22:return\"nd\";case 3:case 23:return\"rd\";default:return\"th\"}};Date.daysInMonth=[31,28,31,30,31,30,31,31,30,31,30,31];Date.monthNames=[\"January\",\"February\",\"March\",\"April\",\"May\",\"June\",\"July\",\"August\",\"September\",\"October\",\"November\",\"December\"];Date.getShortMonthName=function(A){return Date.monthNames[A].substring(0,3)};Date.dayNames=[\"Sunday\",\"Monday\",\"Tuesday\",\"Wednesday\",\"Thursday\",\"Friday\",\"Saturday\"];Date.getShortDayName=function(A){return Date.dayNames[A].substring(0,3)};Date.y2kYear=50;Date.monthNumbers={Jan:0,Feb:1,Mar:2,Apr:3,May:4,Jun:5,Jul:6,Aug:7,Sep:8,Oct:9,Nov:10,Dec:11};Date.getMonthNumber=function(A){return Date.monthNumbers[A.substring(0,1).toUpperCase()+A.substring(1,3).toLowerCase()]};Date.prototype.clone=function(){return new Date(this.getTime())};Date.prototype.clearTime=function(A){if(A){return this.clone().clearTime()}this.setHours(0);this.setMinutes(0);this.setSeconds(0);this.setMilliseconds(0);return this};if(Ext.isSafari){Date.brokenSetMonth=Date.prototype.setMonth;Date.prototype.setMonth=function(A){if(A<=-1){var D=Math.ceil(-A);var C=Math.ceil(D/12);var B=(D%12)?12-D%12:0;this.setFullYear(this.getFullYear()-C);return Date.brokenSetMonth.call(this,B)}else{return Date.brokenSetMonth.apply(this,arguments)}}}Date.MILLI=\"ms\";Date.SECOND=\"s\";Date.MINUTE=\"mi\";Date.HOUR=\"h\";Date.DAY=\"d\";Date.MONTH=\"mo\";Date.YEAR=\"y\";Date.prototype.add=function(B,C){var D=this.clone();if(!B||C===0){return D}switch(B.toLowerCase()){case Date.MILLI:D.setMilliseconds(this.getMilliseconds()+C);break;case Date.SECOND:D.setSeconds(this.getSeconds()+C);break;case Date.MINUTE:D.setMinutes(this.getMinutes()+C);break;case Date.HOUR:D.setHours(this.getHours()+C);break;case Date.DAY:D.setDate(this.getDate()+C);break;case Date.MONTH:var A=this.getDate();if(A>28){A=Math.min(A,this.getFirstDateOfMonth().add(\"mo\",C).getLastDateOfMonth().getDate())}D.setDate(A);D.setMonth(this.getMonth()+C);break;case Date.YEAR:D.setFullYear(this.getFullYear()+C);break}return D};Date.prototype.between=function(C,A){var B=this.getTime();return C.getTime()<=B&&B<=A.getTime()};\nExt.util.DelayedTask=function(E,D,A){var G=null,F,B;var C=function(){var H=new Date().getTime();if(H-B>=F){clearInterval(G);G=null;E.apply(D,A||[])}};this.delay=function(I,K,J,H){if(G&&I!=F){this.cancel()}F=I;B=new Date().getTime();E=K||E;D=J||D;A=H||A;if(!G){G=setInterval(C,F)}};this.cancel=function(){if(G){clearInterval(G);G=null}}};\nExt.util.TaskRunner=function(E){E=E||10;var F=[],A=[];var B=0;var G=false;var D=function(){G=false;clearInterval(B);B=0};var H=function(){if(!G){G=true;B=setInterval(I,E)}};var C=function(J){A.push(J);if(J.onStop){J.onStop.apply(J.scope||J)}};var I=function(){if(A.length>0){for(var O=0,K=A.length;O<K;O++){F.remove(A[O])}A=[];if(F.length<1){D();return }}var M=new Date().getTime();for(var O=0,K=F.length;O<K;++O){var N=F[O];var J=M-N.taskRunTime;if(N.interval<=J){var L=N.run.apply(N.scope||N,N.args||[++N.taskRunCount]);N.taskRunTime=M;if(L===false||N.taskRunCount===N.repeat){C(N);return }}if(N.duration&&N.duration<=(M-N.taskStartTime)){C(N)}}};this.start=function(J){F.push(J);J.taskStartTime=new Date().getTime();J.taskRunTime=0;J.taskRunCount=0;H();return J};this.stop=function(J){C(J);return J};this.stopAll=function(){D();for(var K=0,J=F.length;K<J;K++){if(F[K].onStop){F[K].onStop()}}F=[];A=[]}};Ext.TaskMgr=new Ext.util.TaskRunner();\nExt.util.MixedCollection=function(B,A){this.items=[];this.map={};this.keys=[];this.length=0;this.addEvents(\"clear\",\"add\",\"replace\",\"remove\",\"sort\");this.allowFunctions=B===true;if(A){this.getKey=A}Ext.util.MixedCollection.superclass.constructor.call(this)};Ext.extend(Ext.util.MixedCollection,Ext.util.Observable,{allowFunctions:false,add:function(B,C){if(arguments.length==1){C=arguments[0];B=this.getKey(C)}if(typeof B==\"undefined\"||B===null){this.length++;this.items.push(C);this.keys.push(null)}else{var A=this.map[B];if(A){return this.replace(B,C)}this.length++;this.items.push(C);this.map[B]=C;this.keys.push(B)}this.fireEvent(\"add\",this.length-1,C,B);return C},getKey:function(A){return A.id},replace:function(C,D){if(arguments.length==1){D=arguments[0];C=this.getKey(D)}var A=this.item(C);if(typeof C==\"undefined\"||C===null||typeof A==\"undefined\"){return this.add(C,D)}var B=this.indexOfKey(C);this.items[B]=D;this.map[C]=D;this.fireEvent(\"replace\",C,A,D);return D},addAll:function(E){if(arguments.length>1||Ext.isArray(E)){var B=arguments.length>1?arguments:E;for(var D=0,A=B.length;D<A;D++){this.add(B[D])}}else{for(var C in E){if(this.allowFunctions||typeof E[C]!=\"function\"){this.add(C,E[C])}}}},each:function(E,D){var B=[].concat(this.items);for(var C=0,A=B.length;C<A;C++){if(E.call(D||B[C],B[C],C,A)===false){break}}},eachKey:function(D,C){for(var B=0,A=this.keys.length;B<A;B++){D.call(C||window,this.keys[B],this.items[B],B,A)}},find:function(D,C){for(var B=0,A=this.items.length;B<A;B++){if(D.call(C||window,this.items[B],this.keys[B])){return this.items[B]}}return null},insert:function(A,B,C){if(arguments.length==2){C=arguments[1];B=this.getKey(C)}if(A>=this.length){return this.add(B,C)}this.length++;this.items.splice(A,0,C);if(typeof B!=\"undefined\"&&B!=null){this.map[B]=C}this.keys.splice(A,0,B);this.fireEvent(\"add\",A,C,B);return C},remove:function(A){return this.removeAt(this.indexOf(A))},removeAt:function(A){if(A<this.length&&A>=0){this.length--;var C=this.items[A];this.items.splice(A,1);var B=this.keys[A];if(typeof B!=\"undefined\"){delete this.map[B]}this.keys.splice(A,1);this.fireEvent(\"remove\",C,B);return C}return false},removeKey:function(A){return this.removeAt(this.indexOfKey(A))},getCount:function(){return this.length},indexOf:function(A){return this.items.indexOf(A)},indexOfKey:function(A){return this.keys.indexOf(A)},item:function(A){var B=typeof this.map[A]!=\"undefined\"?this.map[A]:this.items[A];return typeof B!=\"function\"||this.allowFunctions?B:null},itemAt:function(A){return this.items[A]},key:function(A){return this.map[A]},contains:function(A){return this.indexOf(A)!=-1},containsKey:function(A){return typeof this.map[A]!=\"undefined\"},clear:function(){this.length=0;this.items=[];this.keys=[];this.map={};this.fireEvent(\"clear\")},first:function(){return this.items[0]},last:function(){return this.items[this.length-1]},_sort:function(I,A,H){var C=String(A).toUpperCase()==\"DESC\"?-1:1;H=H||function(K,J){return K-J};var G=[],B=this.keys,F=this.items;for(var D=0,E=F.length;D<E;D++){G[G.length]={key:B[D],value:F[D],index:D}}G.sort(function(K,J){var L=H(K[I],J[I])*C;if(L==0){L=(K.index<J.index?-1:1)}return L});for(var D=0,E=G.length;D<E;D++){F[D]=G[D].value;B[D]=G[D].key}this.fireEvent(\"sort\",this)},sort:function(A,B){this._sort(\"value\",A,B)},keySort:function(A,B){this._sort(\"key\",A,B||function(D,C){return String(D).toUpperCase()-String(C).toUpperCase()})},getRange:function(E,A){var B=this.items;if(B.length<1){return[]}E=E||0;A=Math.min(typeof A==\"undefined\"?this.length-1:A,this.length-1);var D=[];if(E<=A){for(var C=E;C<=A;C++){D[D.length]=B[C]}}else{for(var C=E;C>=A;C--){D[D.length]=B[C]}}return D},filter:function(C,B,D,A){if(Ext.isEmpty(B,false)){return this.clone()}B=this.createValueMatcher(B,D,A);return this.filterBy(function(E){return E&&B.test(E[C])})},filterBy:function(F,E){var G=new Ext.util.MixedCollection();G.getKey=this.getKey;var B=this.keys,D=this.items;for(var C=0,A=D.length;C<A;C++){if(F.call(E||this,D[C],B[C])){G.add(B[C],D[C])}}return G},findIndex:function(C,B,E,D,A){if(Ext.isEmpty(B,false)){return -1}B=this.createValueMatcher(B,D,A);return this.findIndexBy(function(F){return F&&B.test(F[C])},null,E)},findIndexBy:function(F,E,G){var B=this.keys,D=this.items;for(var C=(G||0),A=D.length;C<A;C++){if(F.call(E||this,D[C],B[C])){return C}}if(typeof G==\"number\"&&G>0){for(var C=0;C<G;C++){if(F.call(E||this,D[C],B[C])){return C}}}return -1},createValueMatcher:function(B,C,A){if(!B.exec){B=String(B);B=new RegExp((C===true?\"\":\"^\")+Ext.escapeRe(B),A?\"\":\"i\")}return B},clone:function(){var E=new Ext.util.MixedCollection();var B=this.keys,D=this.items;for(var C=0,A=D.length;C<A;C++){E.add(B[C],D[C])}E.getKey=this.getKey;return E}});Ext.util.MixedCollection.prototype.get=Ext.util.MixedCollection.prototype.item;\nExt.util.JSON=new (function(){var useHasOwn={}.hasOwnProperty?true:false;var pad=function(n){return n<10?\"0\"+n:n};var m={\"\\b\":\"\\\\b\",\"\\t\":\"\\\\t\",\"\\n\":\"\\\\n\",\"\\f\":\"\\\\f\",\"\\r\":\"\\\\r\",\"\\\"\":\"\\\\\\\"\",\"\\\\\":\"\\\\\\\\\"};var encodeString=function(s){if(/[\"\\\\\\x00-\\x1f]/.test(s)){return\"\\\"\"+s.replace(/([\\x00-\\x1f\\\\\"])/g,function(a,b){var c=m[b];if(c){return c}c=b.charCodeAt();return\"\\\\u00\"+Math.floor(c/16).toString(16)+(c%16).toString(16)})+\"\\\"\"}return\"\\\"\"+s+\"\\\"\"};var encodeArray=function(o){var a=[\"[\"],b,i,l=o.length,v;for(i=0;i<l;i+=1){v=o[i];switch(typeof v){case\"undefined\":case\"function\":case\"unknown\":break;default:if(b){a.push(\",\")}a.push(v===null?\"null\":Ext.util.JSON.encode(v));b=true}}a.push(\"]\");return a.join(\"\")};var encodeDate=function(o){return\"\\\"\"+o.getFullYear()+\"-\"+pad(o.getMonth()+1)+\"-\"+pad(o.getDate())+\"T\"+pad(o.getHours())+\":\"+pad(o.getMinutes())+\":\"+pad(o.getSeconds())+\"\\\"\"};this.encode=function(o){if(typeof o==\"undefined\"||o===null){return\"null\"}else{if(Ext.isArray(o)){return encodeArray(o)}else{if(Ext.isDate(o)){return encodeDate(o)}else{if(typeof o==\"string\"){return encodeString(o)}else{if(typeof o==\"number\"){return isFinite(o)?String(o):\"null\"}else{if(typeof o==\"boolean\"){return String(o)}else{var a=[\"{\"],b,i,v;for(i in o){if(!useHasOwn||o.hasOwnProperty(i)){v=o[i];switch(typeof v){case\"undefined\":case\"function\":case\"unknown\":break;default:if(b){a.push(\",\")}a.push(this.encode(i),\":\",v===null?\"null\":this.encode(v));b=true}}}a.push(\"}\");return a.join(\"\")}}}}}}};this.decode=function(json){return eval(\"(\"+json+\")\")}})();Ext.encode=Ext.util.JSON.encode;Ext.decode=Ext.util.JSON.decode;\nExt.util.Format=function(){var trimRe=/^\\s+|\\s+$/g;return{ellipsis:function(value,len){if(value&&value.length>len){return value.substr(0,len-3)+\"...\"}return value},undef:function(value){return value!==undefined?value:\"\"},defaultValue:function(value,defaultValue){return value!==undefined&&value!==\"\"?value:defaultValue},htmlEncode:function(value){return !value?value:String(value).replace(/&/g,\"&amp;\").replace(/>/g,\"&gt;\").replace(/</g,\"&lt;\").replace(/\"/g,\"&quot;\")},htmlDecode:function(value){return !value?value:String(value).replace(/&amp;/g,\"&\").replace(/&gt;/g,\">\").replace(/&lt;/g,\"<\").replace(/&quot;/g,\"\\\"\")},trim:function(value){return String(value).replace(trimRe,\"\")},substr:function(value,start,length){return String(value).substr(start,length)},lowercase:function(value){return String(value).toLowerCase()},uppercase:function(value){return String(value).toUpperCase()},capitalize:function(value){return !value?value:value.charAt(0).toUpperCase()+value.substr(1).toLowerCase()},call:function(value,fn){if(arguments.length>2){var args=Array.prototype.slice.call(arguments,2);args.unshift(value);return eval(fn).apply(window,args)}else{return eval(fn).call(window,value)}},usMoney:function(v){v=(Math.round((v-0)*100))/100;v=(v==Math.floor(v))?v+\".00\":((v*10==Math.floor(v*10))?v+\"0\":v);v=String(v);var ps=v.split(\".\");var whole=ps[0];var sub=ps[1]?\".\"+ps[1]:\".00\";var r=/(\\d+)(\\d{3})/;while(r.test(whole)){whole=whole.replace(r,\"$1\"+\",\"+\"$2\")}v=whole+sub;if(v.charAt(0)==\"-\"){return\"-$\"+v.substr(1)}return\"$\"+v},date:function(v,format){if(!v){return\"\"}if(!Ext.isDate(v)){v=new Date(Date.parse(v))}return v.dateFormat(format||\"m/d/Y\")},dateRenderer:function(format){return function(v){return Ext.util.Format.date(v,format)}},stripTagsRE:/<\\/?[^>]+>/gi,stripTags:function(v){return !v?v:String(v).replace(this.stripTagsRE,\"\")},stripScriptsRe:/(?:<script.*?>)((\\n|\\r|.)*?)(?:<\\/script>)/ig,stripScripts:function(v){return !v?v:String(v).replace(this.stripScriptsRe,\"\")},fileSize:function(size){if(size<1024){return size+\" bytes\"}else{if(size<1048576){return(Math.round(((size*10)/1024))/10)+\" KB\"}else{return(Math.round(((size*10)/1048576))/10)+\" MB\"}}},math:function(){var fns={};return function(v,a){if(!fns[a]){fns[a]=new Function(\"v\",\"return v \"+a+\";\")}return fns[a](v)}}()}}();\nExt.XTemplate=function(){Ext.XTemplate.superclass.constructor.apply(this,arguments);var P=this.html;P=[\"<tpl>\",P,\"</tpl>\"].join(\"\");var O=/<tpl\\b[^>]*>((?:(?=([^<]+))\\2|<(?!tpl\\b[^>]*>))*?)<\\/tpl>/;var N=/^<tpl\\b[^>]*?for=\"(.*?)\"/;var L=/^<tpl\\b[^>]*?if=\"(.*?)\"/;var J=/^<tpl\\b[^>]*?exec=\"(.*?)\"/;var C,B=0;var G=[];while(C=P.match(O)){var M=C[0].match(N);var K=C[0].match(L);var I=C[0].match(J);var E=null,H=null,D=null;var A=M&&M[1]?M[1]:\"\";if(K){E=K&&K[1]?K[1]:null;if(E){H=new Function(\"values\",\"parent\",\"xindex\",\"xcount\",\"with(values){ return \"+(Ext.util.Format.htmlDecode(E))+\"; }\")}}if(I){E=I&&I[1]?I[1]:null;if(E){D=new Function(\"values\",\"parent\",\"xindex\",\"xcount\",\"with(values){ \"+(Ext.util.Format.htmlDecode(E))+\"; }\")}}if(A){switch(A){case\".\":A=new Function(\"values\",\"parent\",\"with(values){ return values; }\");break;case\"..\":A=new Function(\"values\",\"parent\",\"with(values){ return parent; }\");break;default:A=new Function(\"values\",\"parent\",\"with(values){ return \"+A+\"; }\")}}G.push({id:B,target:A,exec:D,test:H,body:C[1]||\"\"});P=P.replace(C[0],\"{xtpl\"+B+\"}\");++B}for(var F=G.length-1;F>=0;--F){this.compileTpl(G[F])}this.master=G[G.length-1];this.tpls=G};Ext.extend(Ext.XTemplate,Ext.Template,{re:/\\{([\\w-\\.\\#]+)(?:\\:([\\w\\.]*)(?:\\((.*?)?\\))?)?(\\s?[\\+\\-\\*\\\\]\\s?[\\d\\.\\+\\-\\*\\\\\\(\\)]+)?\\}/g,codeRe:/\\{\\[((?:\\\\\\]|.|\\n)*?)\\]\\}/g,applySubTemplate:function(A,H,G,D,C){var J=this.tpls[A];if(J.test&&!J.test.call(this,H,G,D,C)){return\"\"}if(J.exec&&J.exec.call(this,H,G,D,C)){return\"\"}var I=J.target?J.target.call(this,H,G):H;G=J.target?H:G;if(J.target&&Ext.isArray(I)){var B=[];for(var E=0,F=I.length;E<F;E++){B[B.length]=J.compiled.call(this,I[E],G,E+1,F)}return B.join(\"\")}return J.compiled.call(this,I,G,D,C)},compileTpl:function(tpl){var fm=Ext.util.Format;var useF=this.disableFormats!==true;var sep=Ext.isGecko?\"+\":\",\";var fn=function(m,name,format,args,math){if(name.substr(0,4)==\"xtpl\"){return\"'\"+sep+\"this.applySubTemplate(\"+name.substr(4)+\", values, parent, xindex, xcount)\"+sep+\"'\"}var v;if(name===\".\"){v=\"values\"}else{if(name===\"#\"){v=\"xindex\"}else{if(name.indexOf(\".\")!=-1){v=name}else{v=\"values['\"+name+\"']\"}}}if(math){v=\"(\"+v+math+\")\"}if(format&&useF){args=args?\",\"+args:\"\";if(format.substr(0,5)!=\"this.\"){format=\"fm.\"+format+\"(\"}else{format=\"this.call(\\\"\"+format.substr(5)+\"\\\", \";args=\", values\"}}else{args=\"\";format=\"(\"+v+\" === undefined ? '' : \"}return\"'\"+sep+format+v+args+\")\"+sep+\"'\"};var codeFn=function(m,code){return\"'\"+sep+\"(\"+code+\")\"+sep+\"'\"};var body;if(Ext.isGecko){body=\"tpl.compiled = function(values, parent, xindex, xcount){ return '\"+tpl.body.replace(/(\\r\\n|\\n)/g,\"\\\\n\").replace(/'/g,\"\\\\'\").replace(this.re,fn).replace(this.codeRe,codeFn)+\"';};\"}else{body=[\"tpl.compiled = function(values, parent, xindex, xcount){ return ['\"];body.push(tpl.body.replace(/(\\r\\n|\\n)/g,\"\\\\n\").replace(/'/g,\"\\\\'\").replace(this.re,fn).replace(this.codeRe,codeFn));body.push(\"'].join('');};\");body=body.join(\"\")}eval(body);return this},apply:function(A){return this.master.compiled.call(this,A,{},1,1)},applyTemplate:function(A){return this.master.compiled.call(this,A,{},1,1)},compile:function(){return this}});Ext.XTemplate.from=function(A){A=Ext.getDom(A);return new Ext.XTemplate(A.value||A.innerHTML)};\nExt.util.CSS=function(){var D=null;var C=document;var B=/(-[a-z])/gi;var A=function(E,F){return F.charAt(1).toUpperCase()};return{createStyleSheet:function(G,J){var F;var E=C.getElementsByTagName(\"head\")[0];var I=C.createElement(\"style\");I.setAttribute(\"type\",\"text/css\");if(J){I.setAttribute(\"id\",J)}if(Ext.isIE){E.appendChild(I);F=I.styleSheet;F.cssText=G}else{try{I.appendChild(C.createTextNode(G))}catch(H){I.cssText=G}E.appendChild(I);F=I.styleSheet?I.styleSheet:(I.sheet||C.styleSheets[C.styleSheets.length-1])}this.cacheStyleSheet(F);return F},removeStyleSheet:function(F){var E=C.getElementById(F);if(E){E.parentNode.removeChild(E)}},swapStyleSheet:function(G,E){this.removeStyleSheet(G);var F=C.createElement(\"link\");F.setAttribute(\"rel\",\"stylesheet\");F.setAttribute(\"type\",\"text/css\");F.setAttribute(\"id\",G);F.setAttribute(\"href\",E);C.getElementsByTagName(\"head\")[0].appendChild(F)},refreshCache:function(){return this.getRules(true)},cacheStyleSheet:function(F){if(!D){D={}}try{var H=F.cssRules||F.rules;for(var E=H.length-1;E>=0;--E){D[H[E].selectorText]=H[E]}}catch(G){}},getRules:function(F){if(D==null||F){D={};var H=C.styleSheets;for(var G=0,E=H.length;G<E;G++){try{this.cacheStyleSheet(H[G])}catch(I){}}}return D},getRule:function(E,G){var F=this.getRules(G);if(!Ext.isArray(E)){return F[E]}for(var H=0;H<E.length;H++){if(F[E[H]]){return F[E[H]]}}return null},updateRule:function(E,H,G){if(!Ext.isArray(E)){var I=this.getRule(E);if(I){I.style[H.replace(B,A)]=G;return true}}else{for(var F=0;F<E.length;F++){if(this.updateRule(E[F],H,G)){return true}}}return false}}}();\nExt.util.ClickRepeater=function(B,A){this.el=Ext.get(B);this.el.unselectable();Ext.apply(this,A);this.addEvents(\"mousedown\",\"click\",\"mouseup\");this.el.on(\"mousedown\",this.handleMouseDown,this);if(this.preventDefault||this.stopDefault){this.el.on(\"click\",function(C){if(this.preventDefault){C.preventDefault()}if(this.stopDefault){C.stopEvent()}},this)}if(this.handler){this.on(\"click\",this.handler,this.scope||this)}Ext.util.ClickRepeater.superclass.constructor.call(this)};Ext.extend(Ext.util.ClickRepeater,Ext.util.Observable,{interval:20,delay:250,preventDefault:true,stopDefault:false,timer:0,handleMouseDown:function(){clearTimeout(this.timer);this.el.blur();if(this.pressClass){this.el.addClass(this.pressClass)}this.mousedownTime=new Date();Ext.getDoc().on(\"mouseup\",this.handleMouseUp,this);this.el.on(\"mouseout\",this.handleMouseOut,this);this.fireEvent(\"mousedown\",this);this.fireEvent(\"click\",this);if(this.accelerate){this.delay=400}this.timer=this.click.defer(this.delay||this.interval,this)},click:function(){this.fireEvent(\"click\",this);this.timer=this.click.defer(this.accelerate?this.easeOutExpo(this.mousedownTime.getElapsed(),400,-390,12000):this.interval,this)},easeOutExpo:function(B,A,D,C){return(B==C)?A+D:D*(-Math.pow(2,-10*B/C)+1)+A},handleMouseOut:function(){clearTimeout(this.timer);if(this.pressClass){this.el.removeClass(this.pressClass)}this.el.on(\"mouseover\",this.handleMouseReturn,this)},handleMouseReturn:function(){this.el.un(\"mouseover\",this.handleMouseReturn);if(this.pressClass){this.el.addClass(this.pressClass)}this.click()},handleMouseUp:function(){clearTimeout(this.timer);this.el.un(\"mouseover\",this.handleMouseReturn);this.el.un(\"mouseout\",this.handleMouseOut);Ext.getDoc().un(\"mouseup\",this.handleMouseUp);this.el.removeClass(this.pressClass);this.fireEvent(\"mouseup\",this)}});\nExt.KeyNav=function(B,A){this.el=Ext.get(B);Ext.apply(this,A);if(!this.disabled){this.disabled=true;this.enable()}};Ext.KeyNav.prototype={disabled:false,defaultEventAction:\"stopEvent\",forceKeyDown:false,prepareEvent:function(C){var A=C.getKey();var B=this.keyToHandler[A];if(Ext.isSafari&&B&&A>=37&&A<=40){C.stopEvent()}},relay:function(C){var A=C.getKey();var B=this.keyToHandler[A];if(B&&this[B]){if(this.doRelay(C,this[B],B)!==true){C[this.defaultEventAction]()}}},doRelay:function(C,B,A){return B.call(this.scope||this,C)},enter:false,left:false,right:false,up:false,down:false,tab:false,esc:false,pageUp:false,pageDown:false,del:false,home:false,end:false,keyToHandler:{37:\"left\",39:\"right\",38:\"up\",40:\"down\",33:\"pageUp\",34:\"pageDown\",46:\"del\",36:\"home\",35:\"end\",13:\"enter\",27:\"esc\",9:\"tab\"},enable:function(){if(this.disabled){if(this.forceKeyDown||Ext.isIE||Ext.isAir){this.el.on(\"keydown\",this.relay,this)}else{this.el.on(\"keydown\",this.prepareEvent,this);this.el.on(\"keypress\",this.relay,this)}this.disabled=false}},disable:function(){if(!this.disabled){if(this.forceKeyDown||Ext.isIE||Ext.isAir){this.el.un(\"keydown\",this.relay)}else{this.el.un(\"keydown\",this.prepareEvent);this.el.un(\"keypress\",this.relay)}this.disabled=true}}};\nExt.KeyMap=function(C,B,A){this.el=Ext.get(C);this.eventName=A||\"keydown\";this.bindings=[];if(B){this.addBinding(B)}this.enable()};Ext.KeyMap.prototype={stopEvent:false,addBinding:function(D){if(Ext.isArray(D)){for(var F=0,H=D.length;F<H;F++){this.addBinding(D[F])}return }var N=D.key,C=D.shift,A=D.ctrl,G=D.alt,J=D.fn||D.handler,M=D.scope;if(typeof N==\"string\"){var K=[];var I=N.toUpperCase();for(var E=0,H=I.length;E<H;E++){K.push(I.charCodeAt(E))}N=K}var B=Ext.isArray(N);var L=function(R){if((!C||R.shiftKey)&&(!A||R.ctrlKey)&&(!G||R.altKey)){var P=R.getKey();if(B){for(var Q=0,O=N.length;Q<O;Q++){if(N[Q]==P){if(this.stopEvent){R.stopEvent()}J.call(M||window,P,R);return }}}else{if(P==N){if(this.stopEvent){R.stopEvent()}J.call(M||window,P,R)}}}};this.bindings.push(L)},on:function(B,D,C){var G,A,E,F;if(typeof B==\"object\"&&!Ext.isArray(B)){G=B.key;A=B.shift;E=B.ctrl;F=B.alt}else{G=B}this.addBinding({key:G,shift:A,ctrl:E,alt:F,fn:D,scope:C})},handleKeyDown:function(D){if(this.enabled){var B=this.bindings;for(var C=0,A=B.length;C<A;C++){B[C].call(this,D)}}},isEnabled:function(){return this.enabled},enable:function(){if(!this.enabled){this.el.on(this.eventName,this.handleKeyDown,this);this.enabled=true}},disable:function(){if(this.enabled){this.el.removeListener(this.eventName,this.handleKeyDown,this);this.enabled=false}}};\nExt.util.TextMetrics=function(){var A;return{measure:function(B,C,D){if(!A){A=Ext.util.TextMetrics.Instance(B,D)}A.bind(B);A.setFixedWidth(D||\"auto\");return A.getSize(C)},createInstance:function(B,C){return Ext.util.TextMetrics.Instance(B,C)}}}();Ext.util.TextMetrics.Instance=function(B,D){var C=new Ext.Element(document.createElement(\"div\"));document.body.appendChild(C.dom);C.position(\"absolute\");C.setLeftTop(-1000,-1000);C.hide();if(D){C.setWidth(D)}var A={getSize:function(F){C.update(F);var E=C.getSize();C.update(\"\");return E},bind:function(E){C.setStyle(Ext.fly(E).getStyles(\"font-size\",\"font-style\",\"font-weight\",\"font-family\",\"line-height\"))},setFixedWidth:function(E){C.setWidth(E)},getWidth:function(E){C.dom.style.width=\"auto\";return this.getSize(E).width},getHeight:function(E){return this.getSize(E).height}};A.bind(B);return A};Ext.Element.measureText=Ext.util.TextMetrics.measure;\n(function(){var A=Ext.EventManager;var B=Ext.lib.Dom;Ext.dd.DragDrop=function(E,C,D){if(E){this.init(E,C,D)}};Ext.dd.DragDrop.prototype={id:null,config:null,dragElId:null,handleElId:null,invalidHandleTypes:null,invalidHandleIds:null,invalidHandleClasses:null,startPageX:0,startPageY:0,groups:null,locked:false,lock:function(){this.locked=true},unlock:function(){this.locked=false},isTarget:true,padding:null,_domRef:null,__ygDragDrop:true,constrainX:false,constrainY:false,minX:0,maxX:0,minY:0,maxY:0,maintainOffset:false,xTicks:null,yTicks:null,primaryButtonOnly:true,available:false,hasOuterHandles:false,b4StartDrag:function(C,D){},startDrag:function(C,D){},b4Drag:function(C){},onDrag:function(C){},onDragEnter:function(C,D){},b4DragOver:function(C){},onDragOver:function(C,D){},b4DragOut:function(C){},onDragOut:function(C,D){},b4DragDrop:function(C){},onDragDrop:function(C,D){},onInvalidDrop:function(C){},b4EndDrag:function(C){},endDrag:function(C){},b4MouseDown:function(C){},onMouseDown:function(C){},onMouseUp:function(C){},onAvailable:function(){},defaultPadding:{left:0,right:0,top:0,bottom:0},constrainTo:function(H,F,M){if(typeof F==\"number\"){F={left:F,right:F,top:F,bottom:F}}F=F||this.defaultPadding;var J=Ext.get(this.getEl()).getBox();var C=Ext.get(H);var L=C.getScroll();var I,D=C.dom;if(D==document.body){I={x:L.left,y:L.top,width:Ext.lib.Dom.getViewWidth(),height:Ext.lib.Dom.getViewHeight()}}else{var K=C.getXY();I={x:K[0]+L.left,y:K[1]+L.top,width:D.clientWidth,height:D.clientHeight}}var G=J.y-I.y;var E=J.x-I.x;this.resetConstraints();this.setXConstraint(E-(F.left||0),I.width-E-J.width-(F.right||0),this.xTickSize);this.setYConstraint(G-(F.top||0),I.height-G-J.height-(F.bottom||0),this.yTickSize)},getEl:function(){if(!this._domRef){this._domRef=Ext.getDom(this.id)}return this._domRef},getDragEl:function(){return Ext.getDom(this.dragElId)},init:function(E,C,D){this.initTarget(E,C,D);A.on(this.id,\"mousedown\",this.handleMouseDown,this)},initTarget:function(E,C,D){this.config=D||{};this.DDM=Ext.dd.DDM;this.groups={};if(typeof E!==\"string\"){E=Ext.id(E)}this.id=E;this.addToGroup((C)?C:\"default\");this.handleElId=E;this.setDragElId(E);this.invalidHandleTypes={A:\"A\"};this.invalidHandleIds={};this.invalidHandleClasses=[];this.applyConfig();this.handleOnAvailable()},applyConfig:function(){this.padding=this.config.padding||[0,0,0,0];this.isTarget=(this.config.isTarget!==false);this.maintainOffset=(this.config.maintainOffset);this.primaryButtonOnly=(this.config.primaryButtonOnly!==false)},handleOnAvailable:function(){this.available=true;this.resetConstraints();this.onAvailable()},setPadding:function(E,C,F,D){if(!C&&0!==C){this.padding=[E,E,E,E]}else{if(!F&&0!==F){this.padding=[E,C,E,C]}else{this.padding=[E,C,F,D]}}},setInitPosition:function(F,E){var G=this.getEl();if(!this.DDM.verifyEl(G)){return }var D=F||0;var C=E||0;var H=B.getXY(G);this.initPageX=H[0]-D;this.initPageY=H[1]-C;this.lastPageX=H[0];this.lastPageY=H[1];this.setStartPosition(H)},setStartPosition:function(D){var C=D||B.getXY(this.getEl());this.deltaSetXY=null;this.startPageX=C[0];this.startPageY=C[1]},addToGroup:function(C){this.groups[C]=true;this.DDM.regDragDrop(this,C)},removeFromGroup:function(C){if(this.groups[C]){delete this.groups[C]}this.DDM.removeDDFromGroup(this,C)},setDragElId:function(C){this.dragElId=C},setHandleElId:function(C){if(typeof C!==\"string\"){C=Ext.id(C)}this.handleElId=C;this.DDM.regHandle(this.id,C)},setOuterHandleElId:function(C){if(typeof C!==\"string\"){C=Ext.id(C)}A.on(C,\"mousedown\",this.handleMouseDown,this);this.setHandleElId(C);this.hasOuterHandles=true},unreg:function(){A.un(this.id,\"mousedown\",this.handleMouseDown);this._domRef=null;this.DDM._remove(this)},destroy:function(){this.unreg()},isLocked:function(){return(this.DDM.isLocked()||this.locked)},handleMouseDown:function(E,D){if(this.primaryButtonOnly&&E.button!=0){return }if(this.isLocked()){return }this.DDM.refreshCache(this.groups);var C=new Ext.lib.Point(Ext.lib.Event.getPageX(E),Ext.lib.Event.getPageY(E));if(!this.hasOuterHandles&&!this.DDM.isOverTarget(C,this)){}else{if(this.clickValidator(E)){this.setStartPosition();this.b4MouseDown(E);this.onMouseDown(E);this.DDM.handleMouseDown(E,this);this.DDM.stopEvent(E)}else{}}},clickValidator:function(D){var C=D.getTarget();return(this.isValidHandleChild(C)&&(this.id==this.handleElId||this.DDM.handleWasClicked(C,this.id)))},addInvalidHandleType:function(C){var D=C.toUpperCase();this.invalidHandleTypes[D]=D},addInvalidHandleId:function(C){if(typeof C!==\"string\"){C=Ext.id(C)}this.invalidHandleIds[C]=C},addInvalidHandleClass:function(C){this.invalidHandleClasses.push(C)},removeInvalidHandleType:function(C){var D=C.toUpperCase();delete this.invalidHandleTypes[D]},removeInvalidHandleId:function(C){if(typeof C!==\"string\"){C=Ext.id(C)}delete this.invalidHandleIds[C]},removeInvalidHandleClass:function(D){for(var E=0,C=this.invalidHandleClasses.length;E<C;++E){if(this.invalidHandleClasses[E]==D){delete this.invalidHandleClasses[E]}}},isValidHandleChild:function(F){var E=true;var H;try{H=F.nodeName.toUpperCase()}catch(G){H=F.nodeName}E=E&&!this.invalidHandleTypes[H];E=E&&!this.invalidHandleIds[F.id];for(var D=0,C=this.invalidHandleClasses.length;E&&D<C;++D){E=!B.hasClass(F,this.invalidHandleClasses[D])}return E},setXTicks:function(F,C){this.xTicks=[];this.xTickSize=C;var E={};for(var D=this.initPageX;D>=this.minX;D=D-C){if(!E[D]){this.xTicks[this.xTicks.length]=D;E[D]=true}}for(D=this.initPageX;D<=this.maxX;D=D+C){if(!E[D]){this.xTicks[this.xTicks.length]=D;E[D]=true}}this.xTicks.sort(this.DDM.numericSort)},setYTicks:function(F,C){this.yTicks=[];this.yTickSize=C;var E={};for(var D=this.initPageY;D>=this.minY;D=D-C){if(!E[D]){this.yTicks[this.yTicks.length]=D;E[D]=true}}for(D=this.initPageY;D<=this.maxY;D=D+C){if(!E[D]){this.yTicks[this.yTicks.length]=D;E[D]=true}}this.yTicks.sort(this.DDM.numericSort)},setXConstraint:function(E,D,C){this.leftConstraint=E;this.rightConstraint=D;this.minX=this.initPageX-E;this.maxX=this.initPageX+D;if(C){this.setXTicks(this.initPageX,C)}this.constrainX=true},clearConstraints:function(){this.constrainX=false;this.constrainY=false;this.clearTicks()},clearTicks:function(){this.xTicks=null;this.yTicks=null;this.xTickSize=0;this.yTickSize=0},setYConstraint:function(C,E,D){this.topConstraint=C;this.bottomConstraint=E;this.minY=this.initPageY-C;this.maxY=this.initPageY+E;if(D){this.setYTicks(this.initPageY,D)}this.constrainY=true},resetConstraints:function(){if(this.initPageX||this.initPageX===0){var D=(this.maintainOffset)?this.lastPageX-this.initPageX:0;var C=(this.maintainOffset)?this.lastPageY-this.initPageY:0;this.setInitPosition(D,C)}else{this.setInitPosition()}if(this.constrainX){this.setXConstraint(this.leftConstraint,this.rightConstraint,this.xTickSize)}if(this.constrainY){this.setYConstraint(this.topConstraint,this.bottomConstraint,this.yTickSize)}},getTick:function(I,F){if(!F){return I}else{if(F[0]>=I){return F[0]}else{for(var D=0,C=F.length;D<C;++D){var E=D+1;if(F[E]&&F[E]>=I){var H=I-F[D];var G=F[E]-I;return(G>H)?F[D]:F[E]}}return F[F.length-1]}}},toString:function(){return(\"DragDrop \"+this.id)}}})();if(!Ext.dd.DragDropMgr){Ext.dd.DragDropMgr=function(){var A=Ext.EventManager;return{ids:{},handleIds:{},dragCurrent:null,dragOvers:{},deltaX:0,deltaY:0,preventDefault:true,stopPropagation:true,initalized:false,locked:false,init:function(){this.initialized=true},POINT:0,INTERSECT:1,mode:0,_execOnAll:function(D,C){for(var E in this.ids){for(var B in this.ids[E]){var F=this.ids[E][B];if(!this.isTypeOfDD(F)){continue}F[D].apply(F,C)}}},_onLoad:function(){this.init();A.on(document,\"mouseup\",this.handleMouseUp,this,true);A.on(document,\"mousemove\",this.handleMouseMove,this,true);A.on(window,\"unload\",this._onUnload,this,true);A.on(window,\"resize\",this._onResize,this,true)},_onResize:function(B){this._execOnAll(\"resetConstraints\",[])},lock:function(){this.locked=true},unlock:function(){this.locked=false},isLocked:function(){return this.locked},locationCache:{},useCache:true,clickPixelThresh:3,clickTimeThresh:350,dragThreshMet:false,clickTimeout:null,startX:0,startY:0,regDragDrop:function(C,B){if(!this.initialized){this.init()}if(!this.ids[B]){this.ids[B]={}}this.ids[B][C.id]=C},removeDDFromGroup:function(D,B){if(!this.ids[B]){this.ids[B]={}}var C=this.ids[B];if(C&&C[D.id]){delete C[D.id]}},_remove:function(C){for(var B in C.groups){if(B&&this.ids[B][C.id]){delete this.ids[B][C.id]}}delete this.handleIds[C.id]},regHandle:function(C,B){if(!this.handleIds[C]){this.handleIds[C]={}}this.handleIds[C][B]=B},isDragDrop:function(B){return(this.getDDById(B))?true:false},getRelated:function(F,C){var E=[];for(var D in F.groups){for(j in this.ids[D]){var B=this.ids[D][j];if(!this.isTypeOfDD(B)){continue}if(!C||B.isTarget){E[E.length]=B}}}return E},isLegalTarget:function(F,E){var C=this.getRelated(F,true);for(var D=0,B=C.length;D<B;++D){if(C[D].id==E.id){return true}}return false},isTypeOfDD:function(B){return(B&&B.__ygDragDrop)},isHandle:function(C,B){return(this.handleIds[C]&&this.handleIds[C][B])},getDDById:function(C){for(var B in this.ids){if(this.ids[B][C]){return this.ids[B][C]}}return null},handleMouseDown:function(D,C){if(Ext.QuickTips){Ext.QuickTips.disable()}this.currentTarget=D.getTarget();this.dragCurrent=C;var B=C.getEl();this.startX=D.getPageX();this.startY=D.getPageY();this.deltaX=this.startX-B.offsetLeft;this.deltaY=this.startY-B.offsetTop;this.dragThreshMet=false;this.clickTimeout=setTimeout(function(){var E=Ext.dd.DDM;E.startDrag(E.startX,E.startY)},this.clickTimeThresh)},startDrag:function(B,C){clearTimeout(this.clickTimeout);if(this.dragCurrent){this.dragCurrent.b4StartDrag(B,C);this.dragCurrent.startDrag(B,C)}this.dragThreshMet=true},handleMouseUp:function(B){if(Ext.QuickTips){Ext.QuickTips.enable()}if(!this.dragCurrent){return }clearTimeout(this.clickTimeout);if(this.dragThreshMet){this.fireEvents(B,true)}else{}this.stopDrag(B);this.stopEvent(B)},stopEvent:function(B){if(this.stopPropagation){B.stopPropagation()}if(this.preventDefault){B.preventDefault()}},stopDrag:function(B){if(this.dragCurrent){if(this.dragThreshMet){this.dragCurrent.b4EndDrag(B);this.dragCurrent.endDrag(B)}this.dragCurrent.onMouseUp(B)}this.dragCurrent=null;this.dragOvers={}},handleMouseMove:function(D){if(!this.dragCurrent){return true}if(Ext.isIE&&(D.button!==0&&D.button!==1&&D.button!==2)){this.stopEvent(D);return this.handleMouseUp(D)}if(!this.dragThreshMet){var C=Math.abs(this.startX-D.getPageX());var B=Math.abs(this.startY-D.getPageY());if(C>this.clickPixelThresh||B>this.clickPixelThresh){this.startDrag(this.startX,this.startY)}}if(this.dragThreshMet){this.dragCurrent.b4Drag(D);this.dragCurrent.onDrag(D);if(!this.dragCurrent.moveOnly){this.fireEvents(D,false)}}this.stopEvent(D);return true},fireEvents:function(K,L){var N=this.dragCurrent;if(!N||N.isLocked()){return }var O=K.getPoint();var B=[];var E=[];var I=[];var G=[];var D=[];for(var F in this.dragOvers){var C=this.dragOvers[F];if(!this.isTypeOfDD(C)){continue}if(!this.isOverTarget(O,C,this.mode)){E.push(C)}B[F]=true;delete this.dragOvers[F]}for(var M in N.groups){if(\"string\"!=typeof M){continue}for(F in this.ids[M]){var H=this.ids[M][F];if(!this.isTypeOfDD(H)){continue}if(H.isTarget&&!H.isLocked()&&H!=N){if(this.isOverTarget(O,H,this.mode)){if(L){G.push(H)}else{if(!B[H.id]){D.push(H)}else{I.push(H)}this.dragOvers[H.id]=H}}}}}if(this.mode){if(E.length){N.b4DragOut(K,E);N.onDragOut(K,E)}if(D.length){N.onDragEnter(K,D)}if(I.length){N.b4DragOver(K,I);N.onDragOver(K,I)}if(G.length){N.b4DragDrop(K,G);N.onDragDrop(K,G)}}else{var J=0;for(F=0,J=E.length;F<J;++F){N.b4DragOut(K,E[F].id);N.onDragOut(K,E[F].id)}for(F=0,J=D.length;F<J;++F){N.onDragEnter(K,D[F].id)}for(F=0,J=I.length;F<J;++F){N.b4DragOver(K,I[F].id);N.onDragOver(K,I[F].id)}for(F=0,J=G.length;F<J;++F){N.b4DragDrop(K,G[F].id);N.onDragDrop(K,G[F].id)}}if(L&&!G.length){N.onInvalidDrop(K)}},getBestMatch:function(D){var F=null;var C=D.length;if(C==1){F=D[0]}else{for(var E=0;E<C;++E){var B=D[E];if(B.cursorIsOver){F=B;break}else{if(!F||F.overlap.getArea()<B.overlap.getArea()){F=B}}}}return F},refreshCache:function(C){for(var B in C){if(\"string\"!=typeof B){continue}for(var D in this.ids[B]){var E=this.ids[B][D];if(this.isTypeOfDD(E)){var F=this.getLocation(E);if(F){this.locationCache[E.id]=F}else{delete this.locationCache[E.id]}}}}},verifyEl:function(C){if(C){var B;if(Ext.isIE){try{B=C.offsetParent}catch(D){}}else{B=C.offsetParent}if(B){return true}}return false},getLocation:function(G){if(!this.isTypeOfDD(G)){return null}var E=G.getEl(),J,D,C,L,K,M,B,I,F;try{J=Ext.lib.Dom.getXY(E)}catch(H){}if(!J){return null}D=J[0];C=D+E.offsetWidth;L=J[1];K=L+E.offsetHeight;M=L-G.padding[0];B=C+G.padding[1];I=K+G.padding[2];F=D-G.padding[3];return new Ext.lib.Region(M,B,I,F)},isOverTarget:function(J,B,D){var F=this.locationCache[B.id];if(!F||!this.useCache){F=this.getLocation(B);this.locationCache[B.id]=F}if(!F){return false}B.cursorIsOver=F.contains(J);var I=this.dragCurrent;if(!I||!I.getTargetCoord||(!D&&!I.constrainX&&!I.constrainY)){return B.cursorIsOver}B.overlap=null;var G=I.getTargetCoord(J.x,J.y);var C=I.getDragEl();var E=new Ext.lib.Region(G.y,G.x+C.offsetWidth,G.y+C.offsetHeight,G.x);var H=E.intersect(F);if(H){B.overlap=H;return(D)?true:B.cursorIsOver}else{return false}},_onUnload:function(C,B){Ext.dd.DragDropMgr.unregAll()},unregAll:function(){if(this.dragCurrent){this.stopDrag();this.dragCurrent=null}this._execOnAll(\"unreg\",[]);for(var B in this.elementCache){delete this.elementCache[B]}this.elementCache={};this.ids={}},elementCache:{},getElWrapper:function(C){var B=this.elementCache[C];if(!B||!B.el){B=this.elementCache[C]=new this.ElementWrapper(Ext.getDom(C))}return B},getElement:function(B){return Ext.getDom(B)},getCss:function(C){var B=Ext.getDom(C);return(B)?B.style:null},ElementWrapper:function(B){this.el=B||null;this.id=this.el&&B.id;this.css=this.el&&B.style},getPosX:function(B){return Ext.lib.Dom.getX(B)},getPosY:function(B){return Ext.lib.Dom.getY(B)},swapNode:function(D,B){if(D.swapNode){D.swapNode(B)}else{var E=B.parentNode;var C=B.nextSibling;if(C==D){E.insertBefore(D,B)}else{if(B==D.nextSibling){E.insertBefore(B,D)}else{D.parentNode.replaceChild(B,D);E.insertBefore(D,C)}}}},getScroll:function(){var D,B,E=document.documentElement,C=document.body;if(E&&(E.scrollTop||E.scrollLeft)){D=E.scrollTop;B=E.scrollLeft}else{if(C){D=C.scrollTop;B=C.scrollLeft}else{}}return{top:D,left:B}},getStyle:function(C,B){return Ext.fly(C).getStyle(B)},getScrollTop:function(){return this.getScroll().top},getScrollLeft:function(){return this.getScroll().left},moveToEl:function(B,D){var C=Ext.lib.Dom.getXY(D);Ext.lib.Dom.setXY(B,C)},numericSort:function(C,B){return(C-B)},_timeoutCount:0,_addListeners:function(){var B=Ext.dd.DDM;if(Ext.lib.Event&&document){B._onLoad()}else{if(B._timeoutCount>2000){}else{setTimeout(B._addListeners,10);if(document&&document.body){B._timeoutCount+=1}}}},handleWasClicked:function(B,D){if(this.isHandle(D,B.id)){return true}else{var C=B.parentNode;while(C){if(this.isHandle(D,C.id)){return true}else{C=C.parentNode}}}return false}}}();Ext.dd.DDM=Ext.dd.DragDropMgr;Ext.dd.DDM._addListeners()}Ext.dd.DD=function(C,A,B){if(C){this.init(C,A,B)}};Ext.extend(Ext.dd.DD,Ext.dd.DragDrop,{scroll:true,autoOffset:function(C,B){var A=C-this.startPageX;var D=B-this.startPageY;this.setDelta(A,D)},setDelta:function(B,A){this.deltaX=B;this.deltaY=A},setDragElPos:function(C,B){var A=this.getDragEl();this.alignElWithMouse(A,C,B)},alignElWithMouse:function(C,G,F){var E=this.getTargetCoord(G,F);var B=C.dom?C:Ext.fly(C,\"_dd\");if(!this.deltaSetXY){var H=[E.x,E.y];B.setXY(H);var D=B.getLeft(true);var A=B.getTop(true);this.deltaSetXY=[D-E.x,A-E.y]}else{B.setLeftTop(E.x+this.deltaSetXY[0],E.y+this.deltaSetXY[1])}this.cachePosition(E.x,E.y);this.autoScroll(E.x,E.y,C.offsetHeight,C.offsetWidth);return E},cachePosition:function(B,A){if(B){this.lastPageX=B;this.lastPageY=A}else{var C=Ext.lib.Dom.getXY(this.getEl());this.lastPageX=C[0];this.lastPageY=C[1]}},autoScroll:function(J,I,E,K){if(this.scroll){var L=Ext.lib.Dom.getViewHeight();var B=Ext.lib.Dom.getViewWidth();var N=this.DDM.getScrollTop();var D=this.DDM.getScrollLeft();var H=E+I;var M=K+J;var G=(L+N-I-this.deltaY);var F=(B+D-J-this.deltaX);var C=40;var A=(document.all)?80:30;if(H>L&&G<C){window.scrollTo(D,N+A)}if(I<N&&N>0&&I-N<C){window.scrollTo(D,N-A)}if(M>B&&F<C){window.scrollTo(D+A,N)}if(J<D&&D>0&&J-D<C){window.scrollTo(D-A,N)}}},getTargetCoord:function(C,B){var A=C-this.deltaX;var D=B-this.deltaY;if(this.constrainX){if(A<this.minX){A=this.minX}if(A>this.maxX){A=this.maxX}}if(this.constrainY){if(D<this.minY){D=this.minY}if(D>this.maxY){D=this.maxY}}A=this.getTick(A,this.xTicks);D=this.getTick(D,this.yTicks);return{x:A,y:D}},applyConfig:function(){Ext.dd.DD.superclass.applyConfig.call(this);this.scroll=(this.config.scroll!==false)},b4MouseDown:function(A){this.autoOffset(A.getPageX(),A.getPageY())},b4Drag:function(A){this.setDragElPos(A.getPageX(),A.getPageY())},toString:function(){return(\"DD \"+this.id)}});Ext.dd.DDProxy=function(C,A,B){if(C){this.init(C,A,B);this.initFrame()}};Ext.dd.DDProxy.dragElId=\"ygddfdiv\";Ext.extend(Ext.dd.DDProxy,Ext.dd.DD,{resizeFrame:true,centerFrame:false,createFrame:function(){var B=this;var A=document.body;if(!A||!A.firstChild){setTimeout(function(){B.createFrame()},50);return }var D=this.getDragEl();if(!D){D=document.createElement(\"div\");D.id=this.dragElId;var C=D.style;C.position=\"absolute\";C.visibility=\"hidden\";C.cursor=\"move\";C.border=\"2px solid #aaa\";C.zIndex=999;A.insertBefore(D,A.firstChild)}},initFrame:function(){this.createFrame()},applyConfig:function(){Ext.dd.DDProxy.superclass.applyConfig.call(this);this.resizeFrame=(this.config.resizeFrame!==false);this.centerFrame=(this.config.centerFrame);this.setDragElId(this.config.dragElId||Ext.dd.DDProxy.dragElId)},showFrame:function(E,D){var C=this.getEl();var A=this.getDragEl();var B=A.style;this._resizeProxy();if(this.centerFrame){this.setDelta(Math.round(parseInt(B.width,10)/2),Math.round(parseInt(B.height,10)/2))}this.setDragElPos(E,D);Ext.fly(A).show()},_resizeProxy:function(){if(this.resizeFrame){var A=this.getEl();Ext.fly(this.getDragEl()).setSize(A.offsetWidth,A.offsetHeight)}},b4MouseDown:function(B){var A=B.getPageX();var C=B.getPageY();this.autoOffset(A,C);this.setDragElPos(A,C)},b4StartDrag:function(A,B){this.showFrame(A,B)},b4EndDrag:function(A){Ext.fly(this.getDragEl()).hide()},endDrag:function(C){var B=this.getEl();var A=this.getDragEl();A.style.visibility=\"\";this.beforeMove();B.style.visibility=\"hidden\";Ext.dd.DDM.moveToEl(B,A);A.style.visibility=\"hidden\";B.style.visibility=\"\";this.afterDrag()},beforeMove:function(){},afterDrag:function(){},toString:function(){return(\"DDProxy \"+this.id)}});Ext.dd.DDTarget=function(C,A,B){if(C){this.initTarget(C,A,B)}};Ext.extend(Ext.dd.DDTarget,Ext.dd.DragDrop,{toString:function(){return(\"DDTarget \"+this.id)}});\nExt.dd.DragTracker=function(A){Ext.apply(this,A);this.addEvents(\"mousedown\",\"mouseup\",\"mousemove\",\"dragstart\",\"dragend\",\"drag\");this.dragRegion=new Ext.lib.Region(0,0,0,0);if(this.el){this.initEl(this.el)}};Ext.extend(Ext.dd.DragTracker,Ext.util.Observable,{active:false,tolerance:5,autoStart:false,initEl:function(A){this.el=Ext.get(A);A.on(\"mousedown\",this.onMouseDown,this,this.delegate?{delegate:this.delegate}:undefined)},destroy:function(){this.el.un(\"mousedown\",this.onMouseDown,this)},onMouseDown:function(C,B){if(this.fireEvent(\"mousedown\",this,C)!==false&&this.onBeforeStart(C)!==false){this.startXY=this.lastXY=C.getXY();this.dragTarget=this.delegate?B:this.el.dom;C.preventDefault();var A=Ext.getDoc();A.on(\"mouseup\",this.onMouseUp,this);A.on(\"mousemove\",this.onMouseMove,this);A.on(\"selectstart\",this.stopSelect,this);if(this.autoStart){this.timer=this.triggerStart.defer(this.autoStart===true?1000:this.autoStart,this)}}},onMouseMove:function(D,C){D.preventDefault();var B=D.getXY(),A=this.startXY;this.lastXY=B;if(!this.active){if(Math.abs(A[0]-B[0])>this.tolerance||Math.abs(A[1]-B[1])>this.tolerance){this.triggerStart()}else{return }}this.fireEvent(\"mousemove\",this,D);this.onDrag(D);this.fireEvent(\"drag\",this,D)},onMouseUp:function(B){var A=Ext.getDoc();A.un(\"mousemove\",this.onMouseMove,this);A.un(\"mouseup\",this.onMouseUp,this);A.un(\"selectstart\",this.stopSelect,this);B.preventDefault();this.clearStart();this.active=false;delete this.elRegion;this.fireEvent(\"mouseup\",this,B);this.onEnd(B);this.fireEvent(\"dragend\",this,B)},triggerStart:function(A){this.clearStart();this.active=true;this.onStart(this.startXY);this.fireEvent(\"dragstart\",this,this.startXY)},clearStart:function(){if(this.timer){clearTimeout(this.timer);delete this.timer}},stopSelect:function(A){A.stopEvent();return false},onBeforeStart:function(A){},onStart:function(A){},onDrag:function(A){},onEnd:function(A){},getDragTarget:function(){return this.dragTarget},getDragCt:function(){return this.el},getXY:function(A){return A?this.constrainModes[A].call(this,this.lastXY):this.lastXY},getOffset:function(C){var B=this.getXY(C);var A=this.startXY;return[A[0]-B[0],A[1]-B[1]]},constrainModes:{\"point\":function(B){if(!this.elRegion){this.elRegion=this.getDragCt().getRegion()}var A=this.dragRegion;A.left=B[0];A.top=B[1];A.right=B[0];A.bottom=B[1];A.constrainTo(this.elRegion);return[A.left,A.top]}}});\nExt.dd.ScrollManager=function(){var C=Ext.dd.DragDropMgr;var E={};var B=null;var H={};var G=function(K){B=null;A()};var I=function(){if(C.dragCurrent){C.refreshCache(C.dragCurrent.groups)}};var D=function(){if(C.dragCurrent){var K=Ext.dd.ScrollManager;var L=H.el.ddScrollConfig?H.el.ddScrollConfig.increment:K.increment;if(!K.animate){if(H.el.scroll(H.dir,L)){I()}}else{H.el.scroll(H.dir,L,true,K.animDuration,I)}}};var A=function(){if(H.id){clearInterval(H.id)}H.id=0;H.el=null;H.dir=\"\"};var F=function(L,K){A();H.el=L;H.dir=K;H.id=setInterval(D,Ext.dd.ScrollManager.frequency)};var J=function(N,P){if(P||!C.dragCurrent){return }var Q=Ext.dd.ScrollManager;if(!B||B!=C.dragCurrent){B=C.dragCurrent;Q.refreshCache()}var R=Ext.lib.Event.getXY(N);var S=new Ext.lib.Point(R[0],R[1]);for(var L in E){var M=E[L],K=M._region;var O=M.ddScrollConfig?M.ddScrollConfig:Q;if(K&&K.contains(S)&&M.isScrollable()){if(K.bottom-S.y<=O.vthresh){if(H.el!=M){F(M,\"down\")}return }else{if(K.right-S.x<=O.hthresh){if(H.el!=M){F(M,\"left\")}return }else{if(S.y-K.top<=O.vthresh){if(H.el!=M){F(M,\"up\")}return }else{if(S.x-K.left<=O.hthresh){if(H.el!=M){F(M,\"right\")}return }}}}}}A()};C.fireEvents=C.fireEvents.createSequence(J,C);C.stopDrag=C.stopDrag.createSequence(G,C);return{register:function(M){if(Ext.isArray(M)){for(var L=0,K=M.length;L<K;L++){this.register(M[L])}}else{M=Ext.get(M);E[M.id]=M}},unregister:function(M){if(Ext.isArray(M)){for(var L=0,K=M.length;L<K;L++){this.unregister(M[L])}}else{M=Ext.get(M);delete E[M.id]}},vthresh:25,hthresh:25,increment:100,frequency:500,animate:true,animDuration:0.4,refreshCache:function(){for(var K in E){if(typeof E[K]==\"object\"){E[K]._region=E[K].getRegion()}}}}}();\nExt.dd.Registry=function(){var D={};var B={};var A=0;var C=function(F,E){if(typeof F==\"string\"){return F}var G=F.id;if(!G&&E!==false){G=\"extdd-\"+(++A);F.id=G}return G};return{register:function(H,I){I=I||{};if(typeof H==\"string\"){H=document.getElementById(H)}I.ddel=H;D[C(H)]=I;if(I.isHandle!==false){B[I.ddel.id]=I}if(I.handles){var G=I.handles;for(var F=0,E=G.length;F<E;F++){B[C(G[F])]=I}}},unregister:function(H){var J=C(H,false);var I=D[J];if(I){delete D[J];if(I.handles){var G=I.handles;for(var F=0,E=G.length;F<E;F++){delete B[C(G[F],false)]}}}},getHandle:function(E){if(typeof E!=\"string\"){E=E.id}return B[E]},getHandleFromEvent:function(F){var E=Ext.lib.Event.getTarget(F);return E?B[E.id]:null},getTarget:function(E){if(typeof E!=\"string\"){E=E.id}return D[E]},getTargetFromEvent:function(F){var E=Ext.lib.Event.getTarget(F);return E?D[E.id]||B[E.id]:null}}}();\nExt.dd.StatusProxy=function(A){Ext.apply(this,A);this.id=this.id||Ext.id();this.el=new Ext.Layer({dh:{id:this.id,tag:\"div\",cls:\"x-dd-drag-proxy \"+this.dropNotAllowed,children:[{tag:\"div\",cls:\"x-dd-drop-icon\"},{tag:\"div\",cls:\"x-dd-drag-ghost\"}]},shadow:!A||A.shadow!==false});this.ghost=Ext.get(this.el.dom.childNodes[1]);this.dropStatus=this.dropNotAllowed};Ext.dd.StatusProxy.prototype={dropAllowed:\"x-dd-drop-ok\",dropNotAllowed:\"x-dd-drop-nodrop\",setStatus:function(A){A=A||this.dropNotAllowed;if(this.dropStatus!=A){this.el.replaceClass(this.dropStatus,A);this.dropStatus=A}},reset:function(A){this.el.dom.className=\"x-dd-drag-proxy \"+this.dropNotAllowed;this.dropStatus=this.dropNotAllowed;if(A){this.ghost.update(\"\")}},update:function(A){if(typeof A==\"string\"){this.ghost.update(A)}else{this.ghost.update(\"\");A.style.margin=\"0\";this.ghost.dom.appendChild(A)}},getEl:function(){return this.el},getGhost:function(){return this.ghost},hide:function(A){this.el.hide();if(A){this.reset(true)}},stop:function(){if(this.anim&&this.anim.isAnimated&&this.anim.isAnimated()){this.anim.stop()}},show:function(){this.el.show()},sync:function(){this.el.sync()},repair:function(B,C,A){this.callback=C;this.scope=A;if(B&&this.animRepair!==false){this.el.addClass(\"x-dd-drag-repair\");this.el.hideUnders(true);this.anim=this.el.shift({duration:this.repairDuration||0.5,easing:\"easeOut\",xy:B,stopFx:true,callback:this.afterRepair,scope:this})}else{this.afterRepair()}},afterRepair:function(){this.hide(true);if(typeof this.callback==\"function\"){this.callback.call(this.scope||this)}this.callback=null;this.scope=null}};\nExt.dd.DragSource=function(B,A){this.el=Ext.get(B);if(!this.dragData){this.dragData={}}Ext.apply(this,A);if(!this.proxy){this.proxy=new Ext.dd.StatusProxy()}Ext.dd.DragSource.superclass.constructor.call(this,this.el.dom,this.ddGroup||this.group,{dragElId:this.proxy.id,resizeFrame:false,isTarget:false,scroll:this.scroll===true});this.dragging=false};Ext.extend(Ext.dd.DragSource,Ext.dd.DDProxy,{dropAllowed:\"x-dd-drop-ok\",dropNotAllowed:\"x-dd-drop-nodrop\",getDragData:function(A){return this.dragData},onDragEnter:function(C,D){var B=Ext.dd.DragDropMgr.getDDById(D);this.cachedTarget=B;if(this.beforeDragEnter(B,C,D)!==false){if(B.isNotifyTarget){var A=B.notifyEnter(this,C,this.dragData);this.proxy.setStatus(A)}else{this.proxy.setStatus(this.dropAllowed)}if(this.afterDragEnter){this.afterDragEnter(B,C,D)}}},beforeDragEnter:function(B,A,C){return true},alignElWithMouse:function(){Ext.dd.DragSource.superclass.alignElWithMouse.apply(this,arguments);this.proxy.sync()},onDragOver:function(C,D){var B=this.cachedTarget||Ext.dd.DragDropMgr.getDDById(D);if(this.beforeDragOver(B,C,D)!==false){if(B.isNotifyTarget){var A=B.notifyOver(this,C,this.dragData);this.proxy.setStatus(A)}if(this.afterDragOver){this.afterDragOver(B,C,D)}}},beforeDragOver:function(B,A,C){return true},onDragOut:function(B,C){var A=this.cachedTarget||Ext.dd.DragDropMgr.getDDById(C);if(this.beforeDragOut(A,B,C)!==false){if(A.isNotifyTarget){A.notifyOut(this,B,this.dragData)}this.proxy.reset();if(this.afterDragOut){this.afterDragOut(A,B,C)}}this.cachedTarget=null},beforeDragOut:function(B,A,C){return true},onDragDrop:function(B,C){var A=this.cachedTarget||Ext.dd.DragDropMgr.getDDById(C);if(this.beforeDragDrop(A,B,C)!==false){if(A.isNotifyTarget){if(A.notifyDrop(this,B,this.dragData)){this.onValidDrop(A,B,C)}else{this.onInvalidDrop(A,B,C)}}else{this.onValidDrop(A,B,C)}if(this.afterDragDrop){this.afterDragDrop(A,B,C)}}delete this.cachedTarget},beforeDragDrop:function(B,A,C){return true},onValidDrop:function(B,A,C){this.hideProxy();if(this.afterValidDrop){this.afterValidDrop(B,A,C)}},getRepairXY:function(B,A){return this.el.getXY()},onInvalidDrop:function(B,A,C){this.beforeInvalidDrop(B,A,C);if(this.cachedTarget){if(this.cachedTarget.isNotifyTarget){this.cachedTarget.notifyOut(this,A,this.dragData)}this.cacheTarget=null}this.proxy.repair(this.getRepairXY(A,this.dragData),this.afterRepair,this);if(this.afterInvalidDrop){this.afterInvalidDrop(A,C)}},afterRepair:function(){if(Ext.enableFx){this.el.highlight(this.hlColor||\"c3daf9\")}this.dragging=false},beforeInvalidDrop:function(B,A,C){return true},handleMouseDown:function(B){if(this.dragging){return }var A=this.getDragData(B);if(A&&this.onBeforeDrag(A,B)!==false){this.dragData=A;this.proxy.stop();Ext.dd.DragSource.superclass.handleMouseDown.apply(this,arguments)}},onBeforeDrag:function(A,B){return true},onStartDrag:Ext.emptyFn,startDrag:function(A,B){this.proxy.reset();this.dragging=true;this.proxy.update(\"\");this.onInitDrag(A,B);this.proxy.show()},onInitDrag:function(A,C){var B=this.el.dom.cloneNode(true);B.id=Ext.id();this.proxy.update(B);this.onStartDrag(A,C);return true},getProxy:function(){return this.proxy},hideProxy:function(){this.proxy.hide();this.proxy.reset(true);this.dragging=false},triggerCacheRefresh:function(){Ext.dd.DDM.refreshCache(this.groups)},b4EndDrag:function(A){},endDrag:function(A){this.onEndDrag(this.dragData,A)},onEndDrag:function(A,B){},autoOffset:function(A,B){this.setDelta(-12,-20)}});\nExt.dd.DropTarget=function(B,A){this.el=Ext.get(B);Ext.apply(this,A);if(this.containerScroll){Ext.dd.ScrollManager.register(this.el)}Ext.dd.DropTarget.superclass.constructor.call(this,this.el.dom,this.ddGroup||this.group,{isTarget:true})};Ext.extend(Ext.dd.DropTarget,Ext.dd.DDTarget,{dropAllowed:\"x-dd-drop-ok\",dropNotAllowed:\"x-dd-drop-nodrop\",isTarget:true,isNotifyTarget:true,notifyEnter:function(A,C,B){if(this.overClass){this.el.addClass(this.overClass)}return this.dropAllowed},notifyOver:function(A,C,B){return this.dropAllowed},notifyOut:function(A,C,B){if(this.overClass){this.el.removeClass(this.overClass)}},notifyDrop:function(A,C,B){return false}});\nExt.dd.DragZone=function(B,A){Ext.dd.DragZone.superclass.constructor.call(this,B,A);if(this.containerScroll){Ext.dd.ScrollManager.register(this.el)}};Ext.extend(Ext.dd.DragZone,Ext.dd.DragSource,{getDragData:function(A){return Ext.dd.Registry.getHandleFromEvent(A)},onInitDrag:function(A,B){this.proxy.update(this.dragData.ddel.cloneNode(true));this.onStartDrag(A,B);return true},afterRepair:function(){if(Ext.enableFx){Ext.Element.fly(this.dragData.ddel).highlight(this.hlColor||\"c3daf9\")}this.dragging=false},getRepairXY:function(A){return Ext.Element.fly(this.dragData.ddel).getXY()}});\nExt.dd.DropZone=function(B,A){Ext.dd.DropZone.superclass.constructor.call(this,B,A)};Ext.extend(Ext.dd.DropZone,Ext.dd.DropTarget,{getTargetFromEvent:function(A){return Ext.dd.Registry.getTargetFromEvent(A)},onNodeEnter:function(D,A,C,B){},onNodeOver:function(D,A,C,B){return this.dropAllowed},onNodeOut:function(D,A,C,B){},onNodeDrop:function(D,A,C,B){return false},onContainerOver:function(A,C,B){return this.dropNotAllowed},onContainerDrop:function(A,C,B){return false},notifyEnter:function(A,C,B){return this.dropNotAllowed},notifyOver:function(A,C,B){var D=this.getTargetFromEvent(C);if(!D){if(this.lastOverNode){this.onNodeOut(this.lastOverNode,A,C,B);this.lastOverNode=null}return this.onContainerOver(A,C,B)}if(this.lastOverNode!=D){if(this.lastOverNode){this.onNodeOut(this.lastOverNode,A,C,B)}this.onNodeEnter(D,A,C,B);this.lastOverNode=D}return this.onNodeOver(D,A,C,B)},notifyOut:function(A,C,B){if(this.lastOverNode){this.onNodeOut(this.lastOverNode,A,C,B);this.lastOverNode=null}},notifyDrop:function(A,C,B){if(this.lastOverNode){this.onNodeOut(this.lastOverNode,A,C,B);this.lastOverNode=null}var D=this.getTargetFromEvent(C);return D?this.onNodeDrop(D,A,C,B):this.onContainerDrop(A,C,B)},triggerCacheRefresh:function(){Ext.dd.DDM.refreshCache(this.groups)}});\nExt.data.SortTypes={none:function(A){return A},stripTagsRE:/<\\/?[^>]+>/gi,asText:function(A){return String(A).replace(this.stripTagsRE,\"\")},asUCText:function(A){return String(A).toUpperCase().replace(this.stripTagsRE,\"\")},asUCString:function(A){return String(A).toUpperCase()},asDate:function(A){if(!A){return 0}if(Ext.isDate(A)){return A.getTime()}return Date.parse(String(A))},asFloat:function(A){var B=parseFloat(String(A).replace(/,/g,\"\"));if(isNaN(B)){B=0}return B},asInt:function(A){var B=parseInt(String(A).replace(/,/g,\"\"));if(isNaN(B)){B=0}return B}};\nExt.data.Record=function(A,B){this.id=(B||B===0)?B:++Ext.data.Record.AUTO_ID;this.data=A};Ext.data.Record.create=function(E){var C=Ext.extend(Ext.data.Record,{});var D=C.prototype;D.fields=new Ext.util.MixedCollection(false,function(F){return F.name});for(var B=0,A=E.length;B<A;B++){D.fields.add(new Ext.data.Field(E[B]))}C.getField=function(F){return D.fields.get(F)};return C};Ext.data.Record.AUTO_ID=1000;Ext.data.Record.EDIT=\"edit\";Ext.data.Record.REJECT=\"reject\";Ext.data.Record.COMMIT=\"commit\";Ext.data.Record.prototype={dirty:false,editing:false,error:null,modified:null,join:function(A){this.store=A},set:function(A,B){if(String(this.data[A])==String(B)){return }this.dirty=true;if(!this.modified){this.modified={}}if(typeof this.modified[A]==\"undefined\"){this.modified[A]=this.data[A]}this.data[A]=B;if(!this.editing&&this.store){this.store.afterEdit(this)}},get:function(A){return this.data[A]},beginEdit:function(){this.editing=true;this.modified={}},cancelEdit:function(){this.editing=false;delete this.modified},endEdit:function(){this.editing=false;if(this.dirty&&this.store){this.store.afterEdit(this)}},reject:function(B){var A=this.modified;for(var C in A){if(typeof A[C]!=\"function\"){this.data[C]=A[C]}}this.dirty=false;delete this.modified;this.editing=false;if(this.store&&B!==true){this.store.afterReject(this)}},commit:function(A){this.dirty=false;delete this.modified;this.editing=false;if(this.store&&A!==true){this.store.afterCommit(this)}},getChanges:function(){var A=this.modified,B={};for(var C in A){if(A.hasOwnProperty(C)){B[C]=this.data[C]}}return B},hasError:function(){return this.error!=null},clearError:function(){this.error=null},copy:function(A){return new this.constructor(Ext.apply({},this.data),A||this.id)},isModified:function(A){return this.modified&&this.modified.hasOwnProperty(A)}};\nExt.StoreMgr=Ext.apply(new Ext.util.MixedCollection(),{register:function(){for(var A=0,B;B=arguments[A];A++){this.add(B)}},unregister:function(){for(var A=0,B;B=arguments[A];A++){this.remove(this.lookup(B))}},lookup:function(A){return typeof A==\"object\"?A:this.get(A)},getKey:function(A){return A.storeId||A.id}});\nExt.data.Store=function(A){this.data=new Ext.util.MixedCollection(false);this.data.getKey=function(B){return B.id};this.baseParams={};this.paramNames={\"start\":\"start\",\"limit\":\"limit\",\"sort\":\"sort\",\"dir\":\"dir\"};if(A&&A.data){this.inlineData=A.data;delete A.data}Ext.apply(this,A);if(this.url&&!this.proxy){this.proxy=new Ext.data.HttpProxy({url:this.url})}if(this.reader){if(!this.recordType){this.recordType=this.reader.recordType}if(this.reader.onMetaChange){this.reader.onMetaChange=this.onMetaChange.createDelegate(this)}}if(this.recordType){this.fields=this.recordType.prototype.fields}this.modified=[];this.addEvents(\"datachanged\",\"metachange\",\"add\",\"remove\",\"update\",\"clear\",\"beforeload\",\"load\",\"loadexception\");if(this.proxy){this.relayEvents(this.proxy,[\"loadexception\"])}this.sortToggle={};if(this.sortInfo){this.setDefaultSort(this.sortInfo.field,this.sortInfo.direction)}Ext.data.Store.superclass.constructor.call(this);if(this.storeId||this.id){Ext.StoreMgr.register(this)}if(this.inlineData){this.loadData(this.inlineData);delete this.inlineData}else{if(this.autoLoad){this.load.defer(10,this,[typeof this.autoLoad==\"object\"?this.autoLoad:undefined])}}};Ext.extend(Ext.data.Store,Ext.util.Observable,{remoteSort:false,pruneModifiedRecords:false,lastOptions:null,destroy:function(){if(this.id){Ext.StoreMgr.unregister(this)}this.data=null;this.purgeListeners()},add:function(B){B=[].concat(B);if(B.length<1){return }for(var D=0,A=B.length;D<A;D++){B[D].join(this)}var C=this.data.length;this.data.addAll(B);if(this.snapshot){this.snapshot.addAll(B)}this.fireEvent(\"add\",this,B,C)},addSorted:function(A){var B=this.findInsertIndex(A);this.insert(B,A)},remove:function(A){var B=this.data.indexOf(A);this.data.removeAt(B);if(this.pruneModifiedRecords){this.modified.remove(A)}if(this.snapshot){this.snapshot.remove(A)}this.fireEvent(\"remove\",this,A,B)},removeAll:function(){this.data.clear();if(this.snapshot){this.snapshot.clear()}if(this.pruneModifiedRecords){this.modified=[]}this.fireEvent(\"clear\",this)},insert:function(C,B){B=[].concat(B);for(var D=0,A=B.length;D<A;D++){this.data.insert(C,B[D]);B[D].join(this)}this.fireEvent(\"add\",this,B,C)},indexOf:function(A){return this.data.indexOf(A)},indexOfId:function(A){return this.data.indexOfKey(A)},getById:function(A){return this.data.key(A)},getAt:function(A){return this.data.itemAt(A)},getRange:function(B,A){return this.data.getRange(B,A)},storeOptions:function(A){A=Ext.apply({},A);delete A.callback;delete A.scope;this.lastOptions=A},load:function(B){B=B||{};if(this.fireEvent(\"beforeload\",this,B)!==false){this.storeOptions(B);var C=Ext.apply(B.params||{},this.baseParams);if(this.sortInfo&&this.remoteSort){var A=this.paramNames;C[A[\"sort\"]]=this.sortInfo.field;C[A[\"dir\"]]=this.sortInfo.direction}this.proxy.load(C,this.reader,this.loadRecords,this,B);return true}else{return false}},reload:function(A){this.load(Ext.applyIf(A||{},this.lastOptions))},loadRecords:function(G,B,F){if(!G||F===false){if(F!==false){this.fireEvent(\"load\",this,[],B)}if(B.callback){B.callback.call(B.scope||this,[],B,false)}return }var E=G.records,D=G.totalRecords||E.length;if(!B||B.add!==true){if(this.pruneModifiedRecords){this.modified=[]}for(var C=0,A=E.length;C<A;C++){E[C].join(this)}if(this.snapshot){this.data=this.snapshot;delete this.snapshot}this.data.clear();this.data.addAll(E);this.totalLength=D;this.applySort();this.fireEvent(\"datachanged\",this)}else{this.totalLength=Math.max(D,this.data.length+E.length);this.add(E)}this.fireEvent(\"load\",this,E,B);if(B.callback){B.callback.call(B.scope||this,E,B,true)}},loadData:function(C,A){var B=this.reader.readRecords(C);this.loadRecords(B,{add:A},true)},getCount:function(){return this.data.length||0},getTotalCount:function(){return this.totalLength||0},getSortState:function(){return this.sortInfo},applySort:function(){if(this.sortInfo&&!this.remoteSort){var A=this.sortInfo,B=A.field;this.sortData(B,A.direction)}},sortData:function(C,D){D=D||\"ASC\";var A=this.fields.get(C).sortType;var B=function(F,E){var H=A(F.data[C]),G=A(E.data[C]);return H>G?1:(H<G?-1:0)};this.data.sort(D,B);if(this.snapshot&&this.snapshot!=this.data){this.snapshot.sort(D,B)}},setDefaultSort:function(B,A){A=A?A.toUpperCase():\"ASC\";this.sortInfo={field:B,direction:A};this.sortToggle[B]=A},sort:function(E,C){var D=this.fields.get(E);if(!D){return false}if(!C){if(this.sortInfo&&this.sortInfo.field==D.name){C=(this.sortToggle[D.name]||\"ASC\").toggle(\"ASC\",\"DESC\")}else{C=D.sortDir}}var B=(this.sortToggle)?this.sortToggle[D.name]:null;var A=(this.sortInfo)?this.sortInfo:null;this.sortToggle[D.name]=C;this.sortInfo={field:D.name,direction:C};if(!this.remoteSort){this.applySort();this.fireEvent(\"datachanged\",this)}else{if(!this.load(this.lastOptions)){if(B){this.sortToggle[D.name]=B}if(A){this.sortInfo=A}}}},each:function(B,A){this.data.each(B,A)},getModifiedRecords:function(){return this.modified},createFilterFn:function(C,B,D,A){if(Ext.isEmpty(B,false)){return false}B=this.data.createValueMatcher(B,D,A);return function(E){return B.test(E.data[C])}},sum:function(E,F,A){var C=this.data.items,B=0;F=F||0;A=(A||A===0)?A:C.length-1;for(var D=F;D<=A;D++){B+=(C[D].data[E]||0)}return B},filter:function(D,C,E,A){var B=this.createFilterFn(D,C,E,A);return B?this.filterBy(B):this.clearFilter()},filterBy:function(B,A){this.snapshot=this.snapshot||this.data;this.data=this.queryBy(B,A||this);this.fireEvent(\"datachanged\",this)},query:function(D,C,E,A){var B=this.createFilterFn(D,C,E,A);return B?this.queryBy(B):this.data.clone()},queryBy:function(B,A){var C=this.snapshot||this.data;return C.filterBy(B,A||this)},find:function(D,C,F,E,A){var B=this.createFilterFn(D,C,E,A);return B?this.data.findIndexBy(B,null,F):-1},findBy:function(B,A,C){return this.data.findIndexBy(B,A,C)},collect:function(G,H,B){var F=(B===true&&this.snapshot)?this.snapshot.items:this.data.items;var I,J,A=[],C={};for(var D=0,E=F.length;D<E;D++){I=F[D].data[G];J=String(I);if((H||!Ext.isEmpty(I))&&!C[J]){C[J]=true;A[A.length]=I}}return A},clearFilter:function(A){if(this.isFiltered()){this.data=this.snapshot;delete this.snapshot;if(A!==true){this.fireEvent(\"datachanged\",this)}}},isFiltered:function(){return this.snapshot&&this.snapshot!=this.data},afterEdit:function(A){if(this.modified.indexOf(A)==-1){this.modified.push(A)}this.fireEvent(\"update\",this,A,Ext.data.Record.EDIT)},afterReject:function(A){this.modified.remove(A);this.fireEvent(\"update\",this,A,Ext.data.Record.REJECT)},afterCommit:function(A){this.modified.remove(A);this.fireEvent(\"update\",this,A,Ext.data.Record.COMMIT)},commitChanges:function(){var B=this.modified.slice(0);this.modified=[];for(var C=0,A=B.length;C<A;C++){B[C].commit()}},rejectChanges:function(){var B=this.modified.slice(0);this.modified=[];for(var C=0,A=B.length;C<A;C++){B[C].reject()}},onMetaChange:function(B,A,C){this.recordType=A;this.fields=A.prototype.fields;delete this.snapshot;this.sortInfo=B.sortInfo;this.modified=[];this.fireEvent(\"metachange\",this,this.reader.meta)},findInsertIndex:function(A){this.suspendEvents();var C=this.data.clone();this.data.add(A);this.applySort();var B=this.data.indexOf(A);this.data=C;this.resumeEvents();return B}});\nExt.data.SimpleStore=function(A){Ext.data.SimpleStore.superclass.constructor.call(this,Ext.apply(A,{reader:new Ext.data.ArrayReader({id:A.id},Ext.data.Record.create(A.fields))}))};Ext.extend(Ext.data.SimpleStore,Ext.data.Store,{loadData:function(E,B){if(this.expandData===true){var D=[];for(var C=0,A=E.length;C<A;C++){D[D.length]=[E[C]]}E=D}Ext.data.SimpleStore.superclass.loadData.call(this,E,B)}});\nExt.data.JsonStore=function(A){Ext.data.JsonStore.superclass.constructor.call(this,Ext.apply(A,{proxy:!A.data?new Ext.data.HttpProxy({url:A.url}):undefined,reader:new Ext.data.JsonReader(A,A.fields)}))};Ext.extend(Ext.data.JsonStore,Ext.data.Store);\nExt.data.Field=function(D){if(typeof D==\"string\"){D={name:D}}Ext.apply(this,D);if(!this.type){this.type=\"auto\"}var C=Ext.data.SortTypes;if(typeof this.sortType==\"string\"){this.sortType=C[this.sortType]}if(!this.sortType){switch(this.type){case\"string\":this.sortType=C.asUCString;break;case\"date\":this.sortType=C.asDate;break;default:this.sortType=C.none}}var E=/[\\$,%]/g;if(!this.convert){var B,A=this.dateFormat;switch(this.type){case\"\":case\"auto\":case undefined:B=function(F){return F};break;case\"string\":B=function(F){return(F===undefined||F===null)?\"\":String(F)};break;case\"int\":B=function(F){return F!==undefined&&F!==null&&F!==\"\"?parseInt(String(F).replace(E,\"\"),10):\"\"};break;case\"float\":B=function(F){return F!==undefined&&F!==null&&F!==\"\"?parseFloat(String(F).replace(E,\"\"),10):\"\"};break;case\"bool\":case\"boolean\":B=function(F){return F===true||F===\"true\"||F==1};break;case\"date\":B=function(G){if(!G){return\"\"}if(Ext.isDate(G)){return G}if(A){if(A==\"timestamp\"){return new Date(G*1000)}if(A==\"time\"){return new Date(parseInt(G,10))}return Date.parseDate(G,A)}var F=Date.parse(G);return F?new Date(F):null};break}this.convert=B}};Ext.data.Field.prototype={dateFormat:null,defaultValue:\"\",mapping:null,sortType:null,sortDir:\"ASC\"};\nExt.data.DataReader=function(A,B){this.meta=A;this.recordType=Ext.isArray(B)?Ext.data.Record.create(B):B};Ext.data.DataReader.prototype={};\nExt.data.DataProxy=function(){this.addEvents(\"beforeload\",\"load\",\"loadexception\");Ext.data.DataProxy.superclass.constructor.call(this)};Ext.extend(Ext.data.DataProxy,Ext.util.Observable);\nExt.data.MemoryProxy=function(A){Ext.data.MemoryProxy.superclass.constructor.call(this);this.data=A};Ext.extend(Ext.data.MemoryProxy,Ext.data.DataProxy,{load:function(F,C,G,D,B){F=F||{};var A;try{A=C.readRecords(this.data)}catch(E){this.fireEvent(\"loadexception\",this,B,null,E);G.call(D,null,B,false);return }G.call(D,A,B,true)},update:function(B,A){}});\nExt.data.HttpProxy=function(A){Ext.data.HttpProxy.superclass.constructor.call(this);this.conn=A;this.useAjax=!A||!A.events};Ext.extend(Ext.data.HttpProxy,Ext.data.DataProxy,{getConnection:function(){return this.useAjax?Ext.Ajax:this.conn},load:function(E,B,F,C,A){if(this.fireEvent(\"beforeload\",this,E)!==false){var D={params:E||{},request:{callback:F,scope:C,arg:A},reader:B,callback:this.loadResponse,scope:this};if(this.useAjax){Ext.applyIf(D,this.conn);if(this.activeRequest){Ext.Ajax.abort(this.activeRequest)}this.activeRequest=Ext.Ajax.request(D)}else{this.conn.request(D)}}else{F.call(C||this,null,A,false)}},loadResponse:function(E,D,B){delete this.activeRequest;if(!D){this.fireEvent(\"loadexception\",this,E,B);E.request.callback.call(E.request.scope,null,E.request.arg,false);return }var A;try{A=E.reader.read(B)}catch(C){this.fireEvent(\"loadexception\",this,E,B,C);E.request.callback.call(E.request.scope,null,E.request.arg,false);return }this.fireEvent(\"load\",this,E,E.request.arg);E.request.callback.call(E.request.scope,A,E.request.arg,true)},update:function(A){},updateResponse:function(A){}});\nExt.data.ScriptTagProxy=function(A){Ext.data.ScriptTagProxy.superclass.constructor.call(this);Ext.apply(this,A);this.head=document.getElementsByTagName(\"head\")[0]};Ext.data.ScriptTagProxy.TRANS_ID=1000;Ext.extend(Ext.data.ScriptTagProxy,Ext.data.DataProxy,{timeout:30000,callbackParam:\"callback\",nocache:true,load:function(E,F,H,I,J){if(this.fireEvent(\"beforeload\",this,E)!==false){var C=Ext.urlEncode(Ext.apply(E,this.extraParams));var B=this.url;B+=(B.indexOf(\"?\")!=-1?\"&\":\"?\")+C;if(this.nocache){B+=\"&_dc=\"+(new Date().getTime())}var A=++Ext.data.ScriptTagProxy.TRANS_ID;var K={id:A,cb:\"stcCallback\"+A,scriptId:\"stcScript\"+A,params:E,arg:J,url:B,callback:H,scope:I,reader:F};var D=this;window[K.cb]=function(L){D.handleResponse(L,K)};B+=String.format(\"&{0}={1}\",this.callbackParam,K.cb);if(this.autoAbort!==false){this.abort()}K.timeoutId=this.handleFailure.defer(this.timeout,this,[K]);var G=document.createElement(\"script\");G.setAttribute(\"src\",B);G.setAttribute(\"type\",\"text/javascript\");G.setAttribute(\"id\",K.scriptId);this.head.appendChild(G);this.trans=K}else{H.call(I||this,null,J,false)}},isLoading:function(){return this.trans?true:false},abort:function(){if(this.isLoading()){this.destroyTrans(this.trans)}},destroyTrans:function(B,A){this.head.removeChild(document.getElementById(B.scriptId));clearTimeout(B.timeoutId);if(A){window[B.cb]=undefined;try{delete window[B.cb]}catch(C){}}else{window[B.cb]=function(){window[B.cb]=undefined;try{delete window[B.cb]}catch(D){}}}},handleResponse:function(D,B){this.trans=false;this.destroyTrans(B,true);var A;try{A=B.reader.readRecords(D)}catch(C){this.fireEvent(\"loadexception\",this,D,B.arg,C);B.callback.call(B.scope||window,null,B.arg,false);return }this.fireEvent(\"load\",this,D,B.arg);B.callback.call(B.scope||window,A,B.arg,true)},handleFailure:function(A){this.trans=false;this.destroyTrans(A,false);this.fireEvent(\"loadexception\",this,null,A.arg);A.callback.call(A.scope||window,null,A.arg,false)}});\nExt.data.JsonReader=function(A,B){A=A||{};Ext.data.JsonReader.superclass.constructor.call(this,A,B||A.fields)};Ext.extend(Ext.data.JsonReader,Ext.data.DataReader,{read:function(response){var json=response.responseText;var o=eval(\"(\"+json+\")\");if(!o){throw {message:\"JsonReader.read: Json object not found\"}}if(o.metaData){delete this.ef;this.meta=o.metaData;this.recordType=Ext.data.Record.create(o.metaData.fields);this.onMetaChange(this.meta,this.recordType,o)}return this.readRecords(o)},onMetaChange:function(A,C,B){},simpleAccess:function(B,A){return B[A]},getJsonAccessor:function(){var A=/[\\[\\.]/;return function(C){try{return(A.test(C))?new Function(\"obj\",\"return obj.\"+C):function(D){return D[C]}}catch(B){}return Ext.emptyFn}}(),readRecords:function(K){this.jsonData=K;var H=this.meta,A=this.recordType,R=A.prototype.fields,F=R.items,E=R.length;if(!this.ef){if(H.totalProperty){this.getTotal=this.getJsonAccessor(H.totalProperty)}if(H.successProperty){this.getSuccess=this.getJsonAccessor(H.successProperty)}this.getRoot=H.root?this.getJsonAccessor(H.root):function(U){return U};if(H.id){var Q=this.getJsonAccessor(H.id);this.getId=function(V){var U=Q(V);return(U===undefined||U===\"\")?null:U}}else{this.getId=function(){return null}}this.ef=[];for(var O=0;O<E;O++){R=F[O];var T=(R.mapping!==undefined&&R.mapping!==null)?R.mapping:R.name;this.ef[O]=this.getJsonAccessor(T)}}var M=this.getRoot(K),S=M.length,I=S,D=true;if(H.totalProperty){var G=parseInt(this.getTotal(K),10);if(!isNaN(G)){I=G}}if(H.successProperty){var G=this.getSuccess(K);if(G===false||G===\"false\"){D=false}}var P=[];for(var O=0;O<S;O++){var L=M[O];var B={};var J=this.getId(L);for(var N=0;N<E;N++){R=F[N];var G=this.ef[N](L);B[R.name]=R.convert((G!==undefined)?G:R.defaultValue,L)}var C=new A(B,J);C.json=L;P[O]=C}return{success:D,records:P,totalRecords:I}}});\nExt.data.XmlReader=function(A,B){A=A||{};Ext.data.XmlReader.superclass.constructor.call(this,A,B||A.fields)};Ext.extend(Ext.data.XmlReader,Ext.data.DataReader,{read:function(A){var B=A.responseXML;if(!B){throw {message:\"XmlReader.read: XML Document not available\"}}return this.readRecords(B)},readRecords:function(T){this.xmlData=T;var N=T.documentElement||T;var I=Ext.DomQuery;var B=this.recordType,L=B.prototype.fields;var D=this.meta.id;var G=0,E=true;if(this.meta.totalRecords){G=I.selectNumber(this.meta.totalRecords,N,0)}if(this.meta.success){var K=I.selectValue(this.meta.success,N,true);E=K!==false&&K!==\"false\"}var Q=[];var U=I.select(this.meta.record,N);for(var P=0,R=U.length;P<R;P++){var M=U[P];var A={};var J=D?I.selectValue(D,M):undefined;for(var O=0,H=L.length;O<H;O++){var S=L.items[O];var F=I.selectValue(S.mapping||S.name,M,S.defaultValue);F=S.convert(F,M);A[S.name]=F}var C=new B(A,J);C.node=M;Q[Q.length]=C}return{success:E,records:Q,totalRecords:G||Q.length}}});\nExt.data.ArrayReader=Ext.extend(Ext.data.JsonReader,{readRecords:function(C){var B=this.meta?this.meta.id:null;var G=this.recordType,K=G.prototype.fields;var E=[];var M=C;for(var I=0;I<M.length;I++){var D=M[I];var O={};var A=((B||B===0)&&D[B]!==undefined&&D[B]!==\"\"?D[B]:null);for(var H=0,P=K.length;H<P;H++){var L=K.items[H];var F=L.mapping!==undefined&&L.mapping!==null?L.mapping:H;var N=D[F]!==undefined?D[F]:L.defaultValue;N=L.convert(N,D);O[L.name]=N}var J=new G(O,A);J.json=D;E[E.length]=J}return{records:E,totalRecords:E.length}}});\nExt.data.Tree=function(A){this.nodeHash={};this.root=null;if(A){this.setRootNode(A)}this.addEvents(\"append\",\"remove\",\"move\",\"insert\",\"beforeappend\",\"beforeremove\",\"beforemove\",\"beforeinsert\");Ext.data.Tree.superclass.constructor.call(this)};Ext.extend(Ext.data.Tree,Ext.util.Observable,{pathSeparator:\"/\",proxyNodeEvent:function(){return this.fireEvent.apply(this,arguments)},getRootNode:function(){return this.root},setRootNode:function(A){this.root=A;A.ownerTree=this;A.isRoot=true;this.registerNode(A);return A},getNodeById:function(A){return this.nodeHash[A]},registerNode:function(A){this.nodeHash[A.id]=A},unregisterNode:function(A){delete this.nodeHash[A.id]},toString:function(){return\"[Tree\"+(this.id?\" \"+this.id:\"\")+\"]\"}});Ext.data.Node=function(A){this.attributes=A||{};this.leaf=this.attributes.leaf;this.id=this.attributes.id;if(!this.id){this.id=Ext.id(null,\"ynode-\");this.attributes.id=this.id}this.childNodes=[];if(!this.childNodes.indexOf){this.childNodes.indexOf=function(D){for(var C=0,B=this.length;C<B;C++){if(this[C]==D){return C}}return -1}}this.parentNode=null;this.firstChild=null;this.lastChild=null;this.previousSibling=null;this.nextSibling=null;this.addEvents({\"append\":true,\"remove\":true,\"move\":true,\"insert\":true,\"beforeappend\":true,\"beforeremove\":true,\"beforemove\":true,\"beforeinsert\":true});this.listeners=this.attributes.listeners;Ext.data.Node.superclass.constructor.call(this)};Ext.extend(Ext.data.Node,Ext.util.Observable,{fireEvent:function(B){if(Ext.data.Node.superclass.fireEvent.apply(this,arguments)===false){return false}var A=this.getOwnerTree();if(A){if(A.proxyNodeEvent.apply(A,arguments)===false){return false}}return true},isLeaf:function(){return this.leaf===true},setFirstChild:function(A){this.firstChild=A},setLastChild:function(A){this.lastChild=A},isLast:function(){return(!this.parentNode?true:this.parentNode.lastChild==this)},isFirst:function(){return(!this.parentNode?true:this.parentNode.firstChild==this)},hasChildNodes:function(){return !this.isLeaf()&&this.childNodes.length>0},appendChild:function(E){var F=false;if(Ext.isArray(E)){F=E}else{if(arguments.length>1){F=arguments}}if(F){for(var D=0,A=F.length;D<A;D++){this.appendChild(F[D])}}else{if(this.fireEvent(\"beforeappend\",this.ownerTree,this,E)===false){return false}var B=this.childNodes.length;var C=E.parentNode;if(C){if(E.fireEvent(\"beforemove\",E.getOwnerTree(),E,C,this,B)===false){return false}C.removeChild(E)}B=this.childNodes.length;if(B==0){this.setFirstChild(E)}this.childNodes.push(E);E.parentNode=this;var G=this.childNodes[B-1];if(G){E.previousSibling=G;G.nextSibling=E}else{E.previousSibling=null}E.nextSibling=null;this.setLastChild(E);E.setOwnerTree(this.getOwnerTree());this.fireEvent(\"append\",this.ownerTree,this,E,B);if(C){E.fireEvent(\"move\",this.ownerTree,E,C,this,B)}return E}},removeChild:function(B){var A=this.childNodes.indexOf(B);if(A==-1){return false}if(this.fireEvent(\"beforeremove\",this.ownerTree,this,B)===false){return false}this.childNodes.splice(A,1);if(B.previousSibling){B.previousSibling.nextSibling=B.nextSibling}if(B.nextSibling){B.nextSibling.previousSibling=B.previousSibling}if(this.firstChild==B){this.setFirstChild(B.nextSibling)}if(this.lastChild==B){this.setLastChild(B.previousSibling)}B.setOwnerTree(null);B.parentNode=null;B.previousSibling=null;B.nextSibling=null;this.fireEvent(\"remove\",this.ownerTree,this,B);return B},insertBefore:function(D,A){if(!A){return this.appendChild(D)}if(D==A){return false}if(this.fireEvent(\"beforeinsert\",this.ownerTree,this,D,A)===false){return false}var B=this.childNodes.indexOf(A);var C=D.parentNode;var E=B;if(C==this&&this.childNodes.indexOf(D)<B){E--}if(C){if(D.fireEvent(\"beforemove\",D.getOwnerTree(),D,C,this,B,A)===false){return false}C.removeChild(D)}if(E==0){this.setFirstChild(D)}this.childNodes.splice(E,0,D);D.parentNode=this;var F=this.childNodes[E-1];if(F){D.previousSibling=F;F.nextSibling=D}else{D.previousSibling=null}D.nextSibling=A;A.previousSibling=D;D.setOwnerTree(this.getOwnerTree());this.fireEvent(\"insert\",this.ownerTree,this,D,A);if(C){D.fireEvent(\"move\",this.ownerTree,D,C,this,E,A)}return D},remove:function(){this.parentNode.removeChild(this);return this},item:function(A){return this.childNodes[A]},replaceChild:function(A,B){this.insertBefore(A,B);this.removeChild(B);return B},indexOf:function(A){return this.childNodes.indexOf(A)},getOwnerTree:function(){if(!this.ownerTree){var A=this;while(A){if(A.ownerTree){this.ownerTree=A.ownerTree;break}A=A.parentNode}}return this.ownerTree},getDepth:function(){var B=0;var A=this;while(A.parentNode){++B;A=A.parentNode}return B},setOwnerTree:function(B){if(B!=this.ownerTree){if(this.ownerTree){this.ownerTree.unregisterNode(this)}this.ownerTree=B;var D=this.childNodes;for(var C=0,A=D.length;C<A;C++){D[C].setOwnerTree(B)}if(B){B.registerNode(this)}}},getPath:function(B){B=B||\"id\";var D=this.parentNode;var A=[this.attributes[B]];while(D){A.unshift(D.attributes[B]);D=D.parentNode}var C=this.getOwnerTree().pathSeparator;return C+A.join(C)},bubble:function(C,B,A){var D=this;while(D){if(C.apply(B||D,A||[D])===false){break}D=D.parentNode}},cascade:function(F,E,B){if(F.apply(E||this,B||[this])!==false){var D=this.childNodes;for(var C=0,A=D.length;C<A;C++){D[C].cascade(F,E,B)}}},eachChild:function(F,E,B){var D=this.childNodes;for(var C=0,A=D.length;C<A;C++){if(F.apply(E||this,B||[D[C]])===false){break}}},findChild:function(D,E){var C=this.childNodes;for(var B=0,A=C.length;B<A;B++){if(C[B].attributes[D]==E){return C[B]}}return null},findChildBy:function(E,D){var C=this.childNodes;for(var B=0,A=C.length;B<A;B++){if(E.call(D||C[B],C[B])===true){return C[B]}}return null},sort:function(E,D){var C=this.childNodes;var A=C.length;if(A>0){var F=D?function(){E.apply(D,arguments)}:E;C.sort(F);for(var B=0;B<A;B++){var G=C[B];G.previousSibling=C[B-1];G.nextSibling=C[B+1];if(B==0){this.setFirstChild(G)}if(B==A-1){this.setLastChild(G)}}}},contains:function(A){return A.isAncestor(this)},isAncestor:function(A){var B=this.parentNode;while(B){if(B==A){return true}B=B.parentNode}return false},toString:function(){return\"[Node\"+(this.id?\" \"+this.id:\"\")+\"]\"}});\nExt.data.GroupingStore=Ext.extend(Ext.data.Store,{remoteGroup:false,groupOnSort:false,clearGrouping:function(){this.groupField=false;if(this.remoteGroup){if(this.baseParams){delete this.baseParams.groupBy}this.reload()}else{this.applySort();this.fireEvent(\"datachanged\",this)}},groupBy:function(C,B){if(this.groupField==C&&!B){return }this.groupField=C;if(this.remoteGroup){if(!this.baseParams){this.baseParams={}}this.baseParams[\"groupBy\"]=C}if(this.groupOnSort){this.sort(C);return }if(this.remoteGroup){this.reload()}else{var A=this.sortInfo||{};if(A.field!=C){this.applySort()}else{this.sortData(C)}this.fireEvent(\"datachanged\",this)}},applySort:function(){Ext.data.GroupingStore.superclass.applySort.call(this);if(!this.groupOnSort&&!this.remoteGroup){var A=this.getGroupState();if(A&&A!=this.sortInfo.field){this.sortData(this.groupField)}}},applyGrouping:function(A){if(this.groupField!==false){this.groupBy(this.groupField,true);return true}else{if(A===true){this.fireEvent(\"datachanged\",this)}return false}},getGroupState:function(){return this.groupOnSort&&this.groupField!==false?(this.sortInfo?this.sortInfo.field:undefined):this.groupField}});\nExt.ComponentMgr=function(){var B=new Ext.util.MixedCollection();var A={};return{register:function(C){B.add(C)},unregister:function(C){B.remove(C)},get:function(C){return B.get(C)},onAvailable:function(E,D,C){B.on(\"add\",function(F,G){if(G.id==E){D.call(C||G,G);B.un(\"add\",D,C)}})},all:B,registerType:function(D,C){A[D]=C;C.xtype=D},create:function(C,D){return new A[C.xtype||D](C)}}}();Ext.reg=Ext.ComponentMgr.registerType;\nExt.Component=function(B){B=B||{};if(B.initialConfig){if(B.isAction){this.baseAction=B}B=B.initialConfig}else{if(B.tagName||B.dom||typeof B==\"string\"){B={applyTo:B,id:B.id||B}}}this.initialConfig=B;Ext.apply(this,B);this.addEvents(\"disable\",\"enable\",\"beforeshow\",\"show\",\"beforehide\",\"hide\",\"beforerender\",\"render\",\"beforedestroy\",\"destroy\",\"beforestaterestore\",\"staterestore\",\"beforestatesave\",\"statesave\");this.getId();Ext.ComponentMgr.register(this);Ext.Component.superclass.constructor.call(this);if(this.baseAction){this.baseAction.addComponent(this)}this.initComponent();if(this.plugins){if(Ext.isArray(this.plugins)){for(var C=0,A=this.plugins.length;C<A;C++){this.plugins[C].init(this)}}else{this.plugins.init(this)}}if(this.stateful!==false){this.initState(B)}if(this.applyTo){this.applyToMarkup(this.applyTo);delete this.applyTo}else{if(this.renderTo){this.render(this.renderTo);delete this.renderTo}}};Ext.Component.AUTO_ID=1000;Ext.extend(Ext.Component,Ext.util.Observable,{disabledClass:\"x-item-disabled\",allowDomMove:true,autoShow:false,hideMode:\"display\",hideParent:false,hidden:false,disabled:false,rendered:false,ctype:\"Ext.Component\",actionMode:\"el\",getActionEl:function(){return this[this.actionMode]},initComponent:Ext.emptyFn,render:function(B,A){if(!this.rendered&&this.fireEvent(\"beforerender\",this)!==false){if(!B&&this.el){this.el=Ext.get(this.el);B=this.el.dom.parentNode;this.allowDomMove=false}this.container=Ext.get(B);if(this.ctCls){this.container.addClass(this.ctCls)}this.rendered=true;if(A!==undefined){if(typeof A==\"number\"){A=this.container.dom.childNodes[A]}else{A=Ext.getDom(A)}}this.onRender(this.container,A||null);if(this.autoShow){this.el.removeClass([\"x-hidden\",\"x-hide-\"+this.hideMode])}if(this.cls){this.el.addClass(this.cls);delete this.cls}if(this.style){this.el.applyStyles(this.style);delete this.style}this.fireEvent(\"render\",this);this.afterRender(this.container);if(this.hidden){this.hide()}if(this.disabled){this.disable()}this.initStateEvents()}return this},initState:function(A){if(Ext.state.Manager){var B=Ext.state.Manager.get(this.stateId||this.id);if(B){if(this.fireEvent(\"beforestaterestore\",this,B)!==false){this.applyState(B);this.fireEvent(\"staterestore\",this,B)}}}},initStateEvents:function(){if(this.stateEvents){for(var A=0,B;B=this.stateEvents[A];A++){this.on(B,this.saveState,this,{delay:100})}}},applyState:function(B,A){if(B){Ext.apply(this,B)}},getState:function(){return null},saveState:function(){if(Ext.state.Manager){var A=this.getState();if(this.fireEvent(\"beforestatesave\",this,A)!==false){Ext.state.Manager.set(this.stateId||this.id,A);this.fireEvent(\"statesave\",this,A)}}},applyToMarkup:function(A){this.allowDomMove=false;this.el=Ext.get(A);this.render(this.el.dom.parentNode)},addClass:function(A){if(this.el){this.el.addClass(A)}else{this.cls=this.cls?this.cls+\" \"+A:A}},removeClass:function(A){if(this.el){this.el.removeClass(A)}else{if(this.cls){this.cls=this.cls.split(\" \").remove(A).join(\" \")}}},onRender:function(B,A){if(this.autoEl){if(typeof this.autoEl==\"string\"){this.el=document.createElement(this.autoEl)}else{var C=document.createElement(\"div\");Ext.DomHelper.overwrite(C,this.autoEl);this.el=C.firstChild}if(!this.el.id){this.el.id=this.getId()}}if(this.el){this.el=Ext.get(this.el);if(this.allowDomMove!==false){B.dom.insertBefore(this.el.dom,A)}}},getAutoCreate:function(){var A=typeof this.autoCreate==\"object\"?this.autoCreate:Ext.apply({},this.defaultAutoCreate);if(this.id&&!A.id){A.id=this.id}return A},afterRender:Ext.emptyFn,destroy:function(){if(this.fireEvent(\"beforedestroy\",this)!==false){this.beforeDestroy();if(this.rendered){this.el.removeAllListeners();this.el.remove();if(this.actionMode==\"container\"){this.container.remove()}}this.onDestroy();Ext.ComponentMgr.unregister(this);this.fireEvent(\"destroy\",this);this.purgeListeners()}},beforeDestroy:Ext.emptyFn,onDestroy:Ext.emptyFn,getEl:function(){return this.el},getId:function(){return this.id||(this.id=\"ext-comp-\"+(++Ext.Component.AUTO_ID))},getItemId:function(){return this.itemId||this.getId()},focus:function(B,A){if(A){this.focus.defer(typeof A==\"number\"?A:10,this,[B,false]);return }if(this.rendered){this.el.focus();if(B===true){this.el.dom.select()}}return this},blur:function(){if(this.rendered){this.el.blur()}return this},disable:function(){if(this.rendered){this.onDisable()}this.disabled=true;this.fireEvent(\"disable\",this);return this},onDisable:function(){this.getActionEl().addClass(this.disabledClass);this.el.dom.disabled=true},enable:function(){if(this.rendered){this.onEnable()}this.disabled=false;this.fireEvent(\"enable\",this);return this},onEnable:function(){this.getActionEl().removeClass(this.disabledClass);this.el.dom.disabled=false},setDisabled:function(A){this[A?\"disable\":\"enable\"]()},show:function(){if(this.fireEvent(\"beforeshow\",this)!==false){this.hidden=false;if(this.autoRender){this.render(typeof this.autoRender==\"boolean\"?Ext.getBody():this.autoRender)}if(this.rendered){this.onShow()}this.fireEvent(\"show\",this)}return this},onShow:function(){if(this.hideParent){this.container.removeClass(\"x-hide-\"+this.hideMode)}else{this.getActionEl().removeClass(\"x-hide-\"+this.hideMode)}},hide:function(){if(this.fireEvent(\"beforehide\",this)!==false){this.hidden=true;if(this.rendered){this.onHide()}this.fireEvent(\"hide\",this)}return this},onHide:function(){if(this.hideParent){this.container.addClass(\"x-hide-\"+this.hideMode)}else{this.getActionEl().addClass(\"x-hide-\"+this.hideMode)}},setVisible:function(A){if(A){this.show()}else{this.hide()}return this},isVisible:function(){return this.rendered&&this.getActionEl().isVisible()},cloneConfig:function(B){B=B||{};var C=B.id||Ext.id();var A=Ext.applyIf(B,this.initialConfig);A.id=C;return new this.constructor(A)},getXType:function(){return this.constructor.xtype},isXType:function(B,A){return !A?(\"/\"+this.getXTypes()+\"/\").indexOf(\"/\"+B+\"/\")!=-1:this.constructor.xtype==B},getXTypes:function(){var A=this.constructor;if(!A.xtypes){var C=[],B=this;while(B&&B.constructor.xtype){C.unshift(B.constructor.xtype);B=B.constructor.superclass}A.xtypeChain=C;A.xtypes=C.join(\"/\")}return A.xtypes},findParentBy:function(A){for(var B=this.ownerCt;(B!=null)&&!A(B,this);B=B.ownerCt){}return B||null},findParentByType:function(A){return typeof A==\"function\"?this.findParentBy(function(B){return B.constructor===A}):this.findParentBy(function(B){return B.constructor.xtype===A})}});Ext.reg(\"component\",Ext.Component);\nExt.Action=function(A){this.initialConfig=A;this.items=[]};Ext.Action.prototype={isAction:true,setText:function(A){this.initialConfig.text=A;this.callEach(\"setText\",[A])},getText:function(){return this.initialConfig.text},setIconClass:function(A){this.initialConfig.iconCls=A;this.callEach(\"setIconClass\",[A])},getIconClass:function(){return this.initialConfig.iconCls},setDisabled:function(A){this.initialConfig.disabled=A;this.callEach(\"setDisabled\",[A])},enable:function(){this.setDisabled(false)},disable:function(){this.setDisabled(true)},isDisabled:function(){return this.initialConfig.disabled},setHidden:function(A){this.initialConfig.hidden=A;this.callEach(\"setVisible\",[!A])},show:function(){this.setHidden(false)},hide:function(){this.setHidden(true)},isHidden:function(){return this.initialConfig.hidden},setHandler:function(B,A){this.initialConfig.handler=B;this.initialConfig.scope=A;this.callEach(\"setHandler\",[B,A])},each:function(B,A){Ext.each(this.items,B,A)},callEach:function(E,B){var D=this.items;for(var C=0,A=D.length;C<A;C++){D[C][E].apply(D[C],B)}},addComponent:function(A){this.items.push(A);A.on(\"destroy\",this.removeComponent,this)},removeComponent:function(A){this.items.remove(A)},execute:function(){this.initialConfig.handler.apply(this.initialConfig.scope||window,arguments)}};\n(function(){Ext.Layer=function(D,C){D=D||{};var E=Ext.DomHelper;var G=D.parentEl,F=G?Ext.getDom(G):document.body;if(C){this.dom=Ext.getDom(C)}if(!this.dom){var H=D.dh||{tag:\"div\",cls:\"x-layer\"};this.dom=E.append(F,H)}if(D.cls){this.addClass(D.cls)}this.constrain=D.constrain!==false;this.visibilityMode=Ext.Element.VISIBILITY;if(D.id){this.id=this.dom.id=D.id}else{this.id=Ext.id(this.dom)}this.zindex=D.zindex||this.getZIndex();this.position(\"absolute\",this.zindex);if(D.shadow){this.shadowOffset=D.shadowOffset||4;this.shadow=new Ext.Shadow({offset:this.shadowOffset,mode:D.shadow})}else{this.shadowOffset=0}this.useShim=D.shim!==false&&Ext.useShims;this.useDisplay=D.useDisplay;this.hide()};var A=Ext.Element.prototype;var B=[];Ext.extend(Ext.Layer,Ext.Element,{getZIndex:function(){return this.zindex||parseInt(this.getStyle(\"z-index\"),10)||11000},getShim:function(){if(!this.useShim){return null}if(this.shim){return this.shim}var D=B.shift();if(!D){D=this.createShim();D.enableDisplayMode(\"block\");D.dom.style.display=\"none\";D.dom.style.visibility=\"visible\"}var C=this.dom.parentNode;if(D.dom.parentNode!=C){C.insertBefore(D.dom,this.dom)}D.setStyle(\"z-index\",this.getZIndex()-2);this.shim=D;return D},hideShim:function(){if(this.shim){this.shim.setDisplayed(false);B.push(this.shim);delete this.shim}},disableShadow:function(){if(this.shadow){this.shadowDisabled=true;this.shadow.hide();this.lastShadowOffset=this.shadowOffset;this.shadowOffset=0}},enableShadow:function(C){if(this.shadow){this.shadowDisabled=false;this.shadowOffset=this.lastShadowOffset;delete this.lastShadowOffset;if(C){this.sync(true)}}},sync:function(C){var I=this.shadow;if(!this.updating&&this.isVisible()&&(I||this.useShim)){var F=this.getShim();var H=this.getWidth(),E=this.getHeight();var D=this.getLeft(true),J=this.getTop(true);if(I&&!this.shadowDisabled){if(C&&!I.isVisible()){I.show(this)}else{I.realign(D,J,H,E)}if(F){if(C){F.show()}var G=I.adjusts,K=F.dom.style;K.left=(Math.min(D,D+G.l))+\"px\";K.top=(Math.min(J,J+G.t))+\"px\";K.width=(H+G.w)+\"px\";K.height=(E+G.h)+\"px\"}}else{if(F){if(C){F.show()}F.setSize(H,E);F.setLeftTop(D,J)}}}},destroy:function(){this.hideShim();if(this.shadow){this.shadow.hide()}this.removeAllListeners();Ext.removeNode(this.dom);Ext.Element.uncache(this.id)},remove:function(){this.destroy()},beginUpdate:function(){this.updating=true},endUpdate:function(){this.updating=false;this.sync(true)},hideUnders:function(C){if(this.shadow){this.shadow.hide()}this.hideShim()},constrainXY:function(){if(this.constrain){var G=Ext.lib.Dom.getViewWidth(),C=Ext.lib.Dom.getViewHeight();var L=Ext.getDoc().getScroll();var K=this.getXY();var H=K[0],F=K[1];var I=this.dom.offsetWidth+this.shadowOffset,D=this.dom.offsetHeight+this.shadowOffset;var E=false;if((H+I)>G+L.left){H=G-I-this.shadowOffset;E=true}if((F+D)>C+L.top){F=C-D-this.shadowOffset;E=true}if(H<L.left){H=L.left;E=true}if(F<L.top){F=L.top;E=true}if(E){if(this.avoidY){var J=this.avoidY;if(F<=J&&(F+D)>=J){F=J-D-5}}K=[H,F];this.storeXY(K);A.setXY.call(this,K);this.sync()}}},isVisible:function(){return this.visible},showAction:function(){this.visible=true;if(this.useDisplay===true){this.setDisplayed(\"\")}else{if(this.lastXY){A.setXY.call(this,this.lastXY)}else{if(this.lastLT){A.setLeftTop.call(this,this.lastLT[0],this.lastLT[1])}}}},hideAction:function(){this.visible=false;if(this.useDisplay===true){this.setDisplayed(false)}else{this.setLeftTop(-10000,-10000)}},setVisible:function(E,D,G,H,F){if(E){this.showAction()}if(D&&E){var C=function(){this.sync(true);if(H){H()}}.createDelegate(this);A.setVisible.call(this,true,true,G,C,F)}else{if(!E){this.hideUnders(true)}var C=H;if(D){C=function(){this.hideAction();if(H){H()}}.createDelegate(this)}A.setVisible.call(this,E,D,G,C,F);if(E){this.sync(true)}else{if(!D){this.hideAction()}}}},storeXY:function(C){delete this.lastLT;this.lastXY=C},storeLeftTop:function(D,C){delete this.lastXY;this.lastLT=[D,C]},beforeFx:function(){this.beforeAction();return Ext.Layer.superclass.beforeFx.apply(this,arguments)},afterFx:function(){Ext.Layer.superclass.afterFx.apply(this,arguments);this.sync(this.isVisible())},beforeAction:function(){if(!this.updating&&this.shadow){this.shadow.hide()}},setLeft:function(C){this.storeLeftTop(C,this.getTop(true));A.setLeft.apply(this,arguments);this.sync()},setTop:function(C){this.storeLeftTop(this.getLeft(true),C);A.setTop.apply(this,arguments);this.sync()},setLeftTop:function(D,C){this.storeLeftTop(D,C);A.setLeftTop.apply(this,arguments);this.sync()},setXY:function(F,D,G,H,E){this.fixDisplay();this.beforeAction();this.storeXY(F);var C=this.createCB(H);A.setXY.call(this,F,D,G,C,E);if(!D){C()}},createCB:function(D){var C=this;return function(){C.constrainXY();C.sync(true);if(D){D()}}},setX:function(C,D,F,G,E){this.setXY([C,this.getY()],D,F,G,E)},setY:function(G,C,E,F,D){this.setXY([this.getX(),G],C,E,F,D)},setSize:function(E,F,D,H,I,G){this.beforeAction();var C=this.createCB(I);A.setSize.call(this,E,F,D,H,C,G);if(!D){C()}},setWidth:function(E,D,G,H,F){this.beforeAction();var C=this.createCB(H);A.setWidth.call(this,E,D,G,C,F);if(!D){C()}},setHeight:function(E,D,G,H,F){this.beforeAction();var C=this.createCB(H);A.setHeight.call(this,E,D,G,C,F);if(!D){C()}},setBounds:function(J,H,K,D,I,F,G,E){this.beforeAction();var C=this.createCB(G);if(!I){this.storeXY([J,H]);A.setXY.call(this,[J,H]);A.setSize.call(this,K,D,I,F,C,E);C()}else{A.setBounds.call(this,J,H,K,D,I,F,C,E)}return this},setZIndex:function(C){this.zindex=C;this.setStyle(\"z-index\",C+2);if(this.shadow){this.shadow.setZIndex(C+1)}if(this.shim){this.shim.setStyle(\"z-index\",C)}}})})();\nExt.Shadow=function(C){Ext.apply(this,C);if(typeof this.mode!=\"string\"){this.mode=this.defaultMode}var D=this.offset,B={h:0};var A=Math.floor(this.offset/2);switch(this.mode.toLowerCase()){case\"drop\":B.w=0;B.l=B.t=D;B.t-=1;if(Ext.isIE){B.l-=this.offset+A;B.t-=this.offset+A;B.w-=A;B.h-=A;B.t+=1}break;case\"sides\":B.w=(D*2);B.l=-D;B.t=D-1;if(Ext.isIE){B.l-=(this.offset-A);B.t-=this.offset+A;B.l+=1;B.w-=(this.offset-A)*2;B.w-=A+1;B.h-=1}break;case\"frame\":B.w=B.h=(D*2);B.l=B.t=-D;B.t+=1;B.h-=2;if(Ext.isIE){B.l-=(this.offset-A);B.t-=(this.offset-A);B.l+=1;B.w-=(this.offset+A+1);B.h-=(this.offset+A);B.h+=1}break}this.adjusts=B};Ext.Shadow.prototype={offset:4,defaultMode:\"drop\",show:function(A){A=Ext.get(A);if(!this.el){this.el=Ext.Shadow.Pool.pull();if(this.el.dom.nextSibling!=A.dom){this.el.insertBefore(A)}}this.el.setStyle(\"z-index\",this.zIndex||parseInt(A.getStyle(\"z-index\"),10)-1);if(Ext.isIE){this.el.dom.style.filter=\"progid:DXImageTransform.Microsoft.alpha(opacity=50) progid:DXImageTransform.Microsoft.Blur(pixelradius=\"+(this.offset)+\")\"}this.realign(A.getLeft(true),A.getTop(true),A.getWidth(),A.getHeight());this.el.dom.style.display=\"block\"},isVisible:function(){return this.el?true:false},realign:function(A,M,L,D){if(!this.el){return }var I=this.adjusts,G=this.el.dom,N=G.style;var E=0;N.left=(A+I.l)+\"px\";N.top=(M+I.t)+\"px\";var K=(L+I.w),C=(D+I.h),F=K+\"px\",J=C+\"px\";if(N.width!=F||N.height!=J){N.width=F;N.height=J;if(!Ext.isIE){var H=G.childNodes;var B=Math.max(0,(K-12))+\"px\";H[0].childNodes[1].style.width=B;H[1].childNodes[1].style.width=B;H[2].childNodes[1].style.width=B;H[1].style.height=Math.max(0,(C-12))+\"px\"}}},hide:function(){if(this.el){this.el.dom.style.display=\"none\";Ext.Shadow.Pool.push(this.el);delete this.el}},setZIndex:function(A){this.zIndex=A;if(this.el){this.el.setStyle(\"z-index\",A)}}};Ext.Shadow.Pool=function(){var B=[];var A=Ext.isIE?\"<div class=\\\"x-ie-shadow\\\"></div>\":\"<div class=\\\"x-shadow\\\"><div class=\\\"xst\\\"><div class=\\\"xstl\\\"></div><div class=\\\"xstc\\\"></div><div class=\\\"xstr\\\"></div></div><div class=\\\"xsc\\\"><div class=\\\"xsml\\\"></div><div class=\\\"xsmc\\\"></div><div class=\\\"xsmr\\\"></div></div><div class=\\\"xsb\\\"><div class=\\\"xsbl\\\"></div><div class=\\\"xsbc\\\"></div><div class=\\\"xsbr\\\"></div></div></div>\";return{pull:function(){var C=B.shift();if(!C){C=Ext.get(Ext.DomHelper.insertHtml(\"beforeBegin\",document.body.firstChild,A));C.autoBoxAdjust=false}return C},push:function(C){B.push(C)}}}();\nExt.BoxComponent=Ext.extend(Ext.Component,{initComponent:function(){Ext.BoxComponent.superclass.initComponent.call(this);this.addEvents(\"resize\",\"move\")},boxReady:false,deferHeight:false,setSize:function(B,D){if(typeof B==\"object\"){D=B.height;B=B.width}if(!this.boxReady){this.width=B;this.height=D;return this}if(this.lastSize&&this.lastSize.width==B&&this.lastSize.height==D){return this}this.lastSize={width:B,height:D};var C=this.adjustSize(B,D);var F=C.width,A=C.height;if(F!==undefined||A!==undefined){var E=this.getResizeEl();if(!this.deferHeight&&F!==undefined&&A!==undefined){E.setSize(F,A)}else{if(!this.deferHeight&&A!==undefined){E.setHeight(A)}else{if(F!==undefined){E.setWidth(F)}}}this.onResize(F,A,B,D);this.fireEvent(\"resize\",this,F,A,B,D)}return this},setWidth:function(A){return this.setSize(A)},setHeight:function(A){return this.setSize(undefined,A)},getSize:function(){return this.el.getSize()},getPosition:function(A){if(A===true){return[this.el.getLeft(true),this.el.getTop(true)]}return this.xy||this.el.getXY()},getBox:function(A){var B=this.el.getSize();if(A===true){B.x=this.el.getLeft(true);B.y=this.el.getTop(true)}else{var C=this.xy||this.el.getXY();B.x=C[0];B.y=C[1]}return B},updateBox:function(A){this.setSize(A.width,A.height);this.setPagePosition(A.x,A.y);return this},getResizeEl:function(){return this.resizeEl||this.el},getPositionEl:function(){return this.positionEl||this.el},setPosition:function(A,F){if(A&&typeof A[1]==\"number\"){F=A[1];A=A[0]}this.x=A;this.y=F;if(!this.boxReady){return this}var B=this.adjustPosition(A,F);var E=B.x,D=B.y;var C=this.getPositionEl();if(E!==undefined||D!==undefined){if(E!==undefined&&D!==undefined){C.setLeftTop(E,D)}else{if(E!==undefined){C.setLeft(E)}else{if(D!==undefined){C.setTop(D)}}}this.onPosition(E,D);this.fireEvent(\"move\",this,E,D)}return this},setPagePosition:function(A,C){if(A&&typeof A[1]==\"number\"){C=A[1];A=A[0]}this.pageX=A;this.pageY=C;if(!this.boxReady){return }if(A===undefined||C===undefined){return }var B=this.el.translatePoints(A,C);this.setPosition(B.left,B.top);return this},onRender:function(B,A){Ext.BoxComponent.superclass.onRender.call(this,B,A);if(this.resizeEl){this.resizeEl=Ext.get(this.resizeEl)}if(this.positionEl){this.positionEl=Ext.get(this.positionEl)}},afterRender:function(){Ext.BoxComponent.superclass.afterRender.call(this);this.boxReady=true;this.setSize(this.width,this.height);if(this.x||this.y){this.setPosition(this.x,this.y)}else{if(this.pageX||this.pageY){this.setPagePosition(this.pageX,this.pageY)}}},syncSize:function(){delete this.lastSize;this.setSize(this.autoWidth?undefined:this.el.getWidth(),this.autoHeight?undefined:this.el.getHeight());return this},onResize:function(D,B,A,C){},onPosition:function(A,B){},adjustSize:function(A,B){if(this.autoWidth){A=\"auto\"}if(this.autoHeight){B=\"auto\"}return{width:A,height:B}},adjustPosition:function(A,B){return{x:A,y:B}}});Ext.reg(\"box\",Ext.BoxComponent);\nExt.SplitBar=function(C,E,B,D,A){this.el=Ext.get(C,true);this.el.dom.unselectable=\"on\";this.resizingEl=Ext.get(E,true);this.orientation=B||Ext.SplitBar.HORIZONTAL;this.minSize=0;this.maxSize=2000;this.animate=false;this.useShim=false;this.shim=null;if(!A){this.proxy=Ext.SplitBar.createProxy(this.orientation)}else{this.proxy=Ext.get(A).dom}this.dd=new Ext.dd.DDProxy(this.el.dom.id,\"XSplitBars\",{dragElId:this.proxy.id});this.dd.b4StartDrag=this.onStartProxyDrag.createDelegate(this);this.dd.endDrag=this.onEndProxyDrag.createDelegate(this);this.dragSpecs={};this.adapter=new Ext.SplitBar.BasicLayoutAdapter();this.adapter.init(this);if(this.orientation==Ext.SplitBar.HORIZONTAL){this.placement=D||(this.el.getX()>this.resizingEl.getX()?Ext.SplitBar.LEFT:Ext.SplitBar.RIGHT);this.el.addClass(\"x-splitbar-h\")}else{this.placement=D||(this.el.getY()>this.resizingEl.getY()?Ext.SplitBar.TOP:Ext.SplitBar.BOTTOM);this.el.addClass(\"x-splitbar-v\")}this.addEvents(\"resize\",\"moved\",\"beforeresize\",\"beforeapply\");Ext.SplitBar.superclass.constructor.call(this)};Ext.extend(Ext.SplitBar,Ext.util.Observable,{onStartProxyDrag:function(A,E){this.fireEvent(\"beforeresize\",this);this.overlay=Ext.DomHelper.append(document.body,{cls:\"x-drag-overlay\",html:\"&#160;\"},true);this.overlay.unselectable();this.overlay.setSize(Ext.lib.Dom.getViewWidth(true),Ext.lib.Dom.getViewHeight(true));this.overlay.show();Ext.get(this.proxy).setDisplayed(\"block\");var C=this.adapter.getElementSize(this);this.activeMinSize=this.getMinimumSize();this.activeMaxSize=this.getMaximumSize();var D=C-this.activeMinSize;var B=Math.max(this.activeMaxSize-C,0);if(this.orientation==Ext.SplitBar.HORIZONTAL){this.dd.resetConstraints();this.dd.setXConstraint(this.placement==Ext.SplitBar.LEFT?D:B,this.placement==Ext.SplitBar.LEFT?B:D);this.dd.setYConstraint(0,0)}else{this.dd.resetConstraints();this.dd.setXConstraint(0,0);this.dd.setYConstraint(this.placement==Ext.SplitBar.TOP?D:B,this.placement==Ext.SplitBar.TOP?B:D)}this.dragSpecs.startSize=C;this.dragSpecs.startPoint=[A,E];Ext.dd.DDProxy.prototype.b4StartDrag.call(this.dd,A,E)},onEndProxyDrag:function(C){Ext.get(this.proxy).setDisplayed(false);var B=Ext.lib.Event.getXY(C);if(this.overlay){this.overlay.remove();delete this.overlay}var A;if(this.orientation==Ext.SplitBar.HORIZONTAL){A=this.dragSpecs.startSize+(this.placement==Ext.SplitBar.LEFT?B[0]-this.dragSpecs.startPoint[0]:this.dragSpecs.startPoint[0]-B[0])}else{A=this.dragSpecs.startSize+(this.placement==Ext.SplitBar.TOP?B[1]-this.dragSpecs.startPoint[1]:this.dragSpecs.startPoint[1]-B[1])}A=Math.min(Math.max(A,this.activeMinSize),this.activeMaxSize);if(A!=this.dragSpecs.startSize){if(this.fireEvent(\"beforeapply\",this,A)!==false){this.adapter.setElementSize(this,A);this.fireEvent(\"moved\",this,A);this.fireEvent(\"resize\",this,A)}}},getAdapter:function(){return this.adapter},setAdapter:function(A){this.adapter=A;this.adapter.init(this)},getMinimumSize:function(){return this.minSize},setMinimumSize:function(A){this.minSize=A},getMaximumSize:function(){return this.maxSize},setMaximumSize:function(A){this.maxSize=A},setCurrentSize:function(B){var A=this.animate;this.animate=false;this.adapter.setElementSize(this,B);this.animate=A},destroy:function(A){if(this.shim){this.shim.remove()}this.dd.unreg();Ext.removeNode(this.proxy);if(A){this.el.remove()}}});Ext.SplitBar.createProxy=function(B){var C=new Ext.Element(document.createElement(\"div\"));C.unselectable();var A=\"x-splitbar-proxy\";C.addClass(A+\" \"+(B==Ext.SplitBar.HORIZONTAL?A+\"-h\":A+\"-v\"));document.body.appendChild(C.dom);return C.dom};Ext.SplitBar.BasicLayoutAdapter=function(){};Ext.SplitBar.BasicLayoutAdapter.prototype={init:function(A){},getElementSize:function(A){if(A.orientation==Ext.SplitBar.HORIZONTAL){return A.resizingEl.getWidth()}else{return A.resizingEl.getHeight()}},setElementSize:function(B,A,C){if(B.orientation==Ext.SplitBar.HORIZONTAL){if(!B.animate){B.resizingEl.setWidth(A);if(C){C(B,A)}}else{B.resizingEl.setWidth(A,true,0.1,C,\"easeOut\")}}else{if(!B.animate){B.resizingEl.setHeight(A);if(C){C(B,A)}}else{B.resizingEl.setHeight(A,true,0.1,C,\"easeOut\")}}}};Ext.SplitBar.AbsoluteLayoutAdapter=function(A){this.basic=new Ext.SplitBar.BasicLayoutAdapter();this.container=Ext.get(A)};Ext.SplitBar.AbsoluteLayoutAdapter.prototype={init:function(A){this.basic.init(A)},getElementSize:function(A){return this.basic.getElementSize(A)},setElementSize:function(B,A,C){this.basic.setElementSize(B,A,this.moveSplitter.createDelegate(this,[B]))},moveSplitter:function(A){var B=Ext.SplitBar;switch(A.placement){case B.LEFT:A.el.setX(A.resizingEl.getRight());break;case B.RIGHT:A.el.setStyle(\"right\",(this.container.getWidth()-A.resizingEl.getLeft())+\"px\");break;case B.TOP:A.el.setY(A.resizingEl.getBottom());break;case B.BOTTOM:A.el.setY(A.resizingEl.getTop()-A.el.getHeight());break}}};Ext.SplitBar.VERTICAL=1;Ext.SplitBar.HORIZONTAL=2;Ext.SplitBar.LEFT=1;Ext.SplitBar.RIGHT=2;Ext.SplitBar.TOP=3;Ext.SplitBar.BOTTOM=4;\nExt.Container=Ext.extend(Ext.BoxComponent,{autoDestroy:true,defaultType:\"panel\",initComponent:function(){Ext.Container.superclass.initComponent.call(this);this.addEvents(\"afterlayout\",\"beforeadd\",\"beforeremove\",\"add\",\"remove\");var A=this.items;if(A){delete this.items;if(Ext.isArray(A)){this.add.apply(this,A)}else{this.add(A)}}},initItems:function(){if(!this.items){this.items=new Ext.util.MixedCollection(false,this.getComponentId);this.getLayout()}},setLayout:function(A){if(this.layout&&this.layout!=A){this.layout.setContainer(null)}this.initItems();this.layout=A;A.setContainer(this)},render:function(){Ext.Container.superclass.render.apply(this,arguments);if(this.layout){if(typeof this.layout==\"string\"){this.layout=new Ext.Container.LAYOUTS[this.layout.toLowerCase()](this.layoutConfig)}this.setLayout(this.layout);if(this.activeItem!==undefined){var A=this.activeItem;delete this.activeItem;this.layout.setActiveItem(A);return }}if(!this.ownerCt){this.doLayout()}if(this.monitorResize===true){Ext.EventManager.onWindowResize(this.doLayout,this,[false])}},getLayoutTarget:function(){return this.el},getComponentId:function(A){return A.itemId||A.id},add:function(C){if(!this.items){this.initItems()}var B=arguments,A=B.length;if(A>1){for(var D=0;D<A;D++){this.add(B[D])}return }var F=this.lookupComponent(this.applyDefaults(C));var E=this.items.length;if(this.fireEvent(\"beforeadd\",this,F,E)!==false&&this.onBeforeAdd(F)!==false){this.items.add(F);F.ownerCt=this;this.fireEvent(\"add\",this,F,E)}return F},insert:function(D,C){if(!this.items){this.initItems()}var B=arguments,A=B.length;if(A>2){for(var E=A-1;E>=1;--E){this.insert(D,B[E])}return }var F=this.lookupComponent(this.applyDefaults(C));if(F.ownerCt==this&&this.items.indexOf(F)<D){--D}if(this.fireEvent(\"beforeadd\",this,F,D)!==false&&this.onBeforeAdd(F)!==false){this.items.insert(D,F);F.ownerCt=this;this.fireEvent(\"add\",this,F,D)}return F},applyDefaults:function(A){if(this.defaults){if(typeof A==\"string\"){A=Ext.ComponentMgr.get(A);Ext.apply(A,this.defaults)}else{if(!A.events){Ext.applyIf(A,this.defaults)}else{Ext.apply(A,this.defaults)}}}return A},onBeforeAdd:function(A){if(A.ownerCt){A.ownerCt.remove(A,false)}if(this.hideBorders===true){A.border=(A.border===true)}},remove:function(A,B){var C=this.getComponent(A);if(C&&this.fireEvent(\"beforeremove\",this,C)!==false){this.items.remove(C);delete C.ownerCt;if(B===true||(B!==false&&this.autoDestroy)){C.destroy()}if(this.layout&&this.layout.activeItem==C){delete this.layout.activeItem}this.fireEvent(\"remove\",this,C)}return C},getComponent:function(A){if(typeof A==\"object\"){return A}return this.items.get(A)},lookupComponent:function(A){if(typeof A==\"string\"){return Ext.ComponentMgr.get(A)}else{if(!A.events){return this.createComponent(A)}}return A},createComponent:function(A){return Ext.ComponentMgr.create(A,this.defaultType)},doLayout:function(D){if(this.rendered&&this.layout){this.layout.layout()}if(D!==false&&this.items){var C=this.items.items;for(var B=0,A=C.length;B<A;B++){var E=C[B];if(E.doLayout){E.doLayout()}}}},getLayout:function(){if(!this.layout){var A=new Ext.layout.ContainerLayout(this.layoutConfig);this.setLayout(A)}return this.layout},onDestroy:function(){if(this.items){var C=this.items.items;for(var B=0,A=C.length;B<A;B++){Ext.destroy(C[B])}}if(this.monitorResize){Ext.EventManager.removeResizeListener(this.doLayout,this)}Ext.Container.superclass.onDestroy.call(this)},bubble:function(C,B,A){var D=this;while(D){if(C.apply(B||D,A||[D])===false){break}D=D.ownerCt}},cascade:function(F,E,B){if(F.apply(E||this,B||[this])!==false){if(this.items){var D=this.items.items;for(var C=0,A=D.length;C<A;C++){if(D[C].cascade){D[C].cascade(F,E,B)}else{F.apply(E||this,B||[D[C]])}}}}},findById:function(C){var A,B=this;this.cascade(function(D){if(B!=D&&D.id===C){A=D;return false}});return A||null},findByType:function(A){return typeof A==\"function\"?this.findBy(function(B){return B.constructor===A}):this.findBy(function(B){return B.constructor.xtype===A})},find:function(B,A){return this.findBy(function(C){return C[B]===A})},findBy:function(D,C){var A=[],B=this;this.cascade(function(E){if(B!=E&&D.call(C||E,E,B)===true){A.push(E)}});return A}});Ext.Container.LAYOUTS={};Ext.reg(\"container\",Ext.Container);\nExt.layout.ContainerLayout=function(A){Ext.apply(this,A)};Ext.layout.ContainerLayout.prototype={monitorResize:false,activeItem:null,layout:function(){var A=this.container.getLayoutTarget();this.onLayout(this.container,A);this.container.fireEvent(\"afterlayout\",this.container,this)},onLayout:function(A,B){this.renderAll(A,B)},isValidParent:function(C,B){var A=C.getPositionEl?C.getPositionEl():C.getEl();return A.dom.parentNode==B.dom},renderAll:function(D,E){var B=D.items.items;for(var C=0,A=B.length;C<A;C++){var F=B[C];if(F&&(!F.rendered||!this.isValidParent(F,E))){this.renderItem(F,C,E)}}},renderItem:function(D,A,C){if(D&&!D.rendered){D.render(C,A);if(this.extraCls){var B=D.getPositionEl?D.getPositionEl():D;B.addClass(this.extraCls)}if(this.renderHidden&&D!=this.activeItem){D.hide()}}else{if(D&&!this.isValidParent(D,C)){if(this.extraCls){D.addClass(this.extraCls)}if(typeof A==\"number\"){A=C.dom.childNodes[A]}C.dom.insertBefore(D.getEl().dom,A||null);if(this.renderHidden&&D!=this.activeItem){D.hide()}}}},onResize:function(){if(this.container.collapsed){return }var A=this.container.bufferResize;if(A){if(!this.resizeTask){this.resizeTask=new Ext.util.DelayedTask(this.layout,this);this.resizeBuffer=typeof A==\"number\"?A:100}this.resizeTask.delay(this.resizeBuffer)}else{this.layout()}},setContainer:function(A){if(this.monitorResize&&A!=this.container){if(this.container){this.container.un(\"resize\",this.onResize,this)}if(A){A.on(\"resize\",this.onResize,this)}}this.container=A},parseMargins:function(B){var C=B.split(\" \");var A=C.length;if(A==1){C[1]=C[0];C[2]=C[0];C[3]=C[0]}if(A==2){C[2]=C[0];C[3]=C[1]}return{top:parseInt(C[0],10)||0,right:parseInt(C[1],10)||0,bottom:parseInt(C[2],10)||0,left:parseInt(C[3],10)||0}}};Ext.Container.LAYOUTS[\"auto\"]=Ext.layout.ContainerLayout;\nExt.layout.FitLayout=Ext.extend(Ext.layout.ContainerLayout,{monitorResize:true,onLayout:function(A,B){Ext.layout.FitLayout.superclass.onLayout.call(this,A,B);if(!this.container.collapsed){this.setItemSize(this.activeItem||A.items.itemAt(0),B.getStyleSize())}},setItemSize:function(B,A){if(B&&A.height>0){B.setSize(A)}}});Ext.Container.LAYOUTS[\"fit\"]=Ext.layout.FitLayout;\nExt.layout.CardLayout=Ext.extend(Ext.layout.FitLayout,{deferredRender:false,renderHidden:true,setActiveItem:function(A){A=this.container.getComponent(A);if(this.activeItem!=A){if(this.activeItem){this.activeItem.hide()}this.activeItem=A;A.show();this.layout()}},renderAll:function(A,B){if(this.deferredRender){this.renderItem(this.activeItem,undefined,B)}else{Ext.layout.CardLayout.superclass.renderAll.call(this,A,B)}}});Ext.Container.LAYOUTS[\"card\"]=Ext.layout.CardLayout;\nExt.layout.AnchorLayout=Ext.extend(Ext.layout.ContainerLayout,{monitorResize:true,getAnchorViewSize:function(A,B){return B.dom==document.body?B.getViewSize():B.getStyleSize()},onLayout:function(F,I){Ext.layout.AnchorLayout.superclass.onLayout.call(this,F,I);var O=this.getAnchorViewSize(F,I);var M=O.width,E=O.height;if(M<20||E<20){return }var B,K;if(F.anchorSize){if(typeof F.anchorSize==\"number\"){B=F.anchorSize}else{B=F.anchorSize.width;K=F.anchorSize.height}}else{B=F.initialConfig.width;K=F.initialConfig.height}var H=F.items.items,G=H.length,D,J,L,C,A;for(D=0;D<G;D++){J=H[D];if(J.anchor){L=J.anchorSpec;if(!L){var N=J.anchor.split(\" \");J.anchorSpec=L={right:this.parseAnchor(N[0],J.initialConfig.width,B),bottom:this.parseAnchor(N[1],J.initialConfig.height,K)}}C=L.right?this.adjustWidthAnchor(L.right(M),J):undefined;A=L.bottom?this.adjustHeightAnchor(L.bottom(E),J):undefined;if(C||A){J.setSize(C||undefined,A||undefined)}}}},parseAnchor:function(B,F,A){if(B&&B!=\"none\"){var D;if(/^(r|right|b|bottom)$/i.test(B)){var E=A-F;return function(G){if(G!==D){D=G;return G-E}}}else{if(B.indexOf(\"%\")!=-1){var C=parseFloat(B.replace(\"%\",\"\"))*0.01;return function(G){if(G!==D){D=G;return Math.floor(G*C)}}}else{B=parseInt(B,10);if(!isNaN(B)){return function(G){if(G!==D){D=G;return G+B}}}}}}return false},adjustWidthAnchor:function(B,A){return B},adjustHeightAnchor:function(B,A){return B}});Ext.Container.LAYOUTS[\"anchor\"]=Ext.layout.AnchorLayout;\nExt.layout.ColumnLayout=Ext.extend(Ext.layout.ContainerLayout,{monitorResize:true,extraCls:\"x-column\",scrollOffset:0,isValidParent:function(B,A){return B.getEl().dom.parentNode==this.innerCt.dom},onLayout:function(C,F){var D=C.items.items,E=D.length,G,A;if(!this.innerCt){F.addClass(\"x-column-layout-ct\");this.innerCt=F.createChild({cls:\"x-column-inner\"});this.innerCt.createChild({cls:\"x-clear\"})}this.renderAll(C,this.innerCt);var J=F.getViewSize();if(J.width<1&&J.height<1){return }var H=J.width-F.getPadding(\"lr\")-this.scrollOffset,B=J.height-F.getPadding(\"tb\"),I=H;this.innerCt.setWidth(H);for(A=0;A<E;A++){G=D[A];if(!G.columnWidth){I-=(G.getSize().width+G.getEl().getMargins(\"lr\"))}}I=I<0?0:I;for(A=0;A<E;A++){G=D[A];if(G.columnWidth){G.setSize(Math.floor(G.columnWidth*I)-G.getEl().getMargins(\"lr\"))}}}});Ext.Container.LAYOUTS[\"column\"]=Ext.layout.ColumnLayout;\nExt.layout.BorderLayout=Ext.extend(Ext.layout.ContainerLayout,{monitorResize:true,rendered:false,onLayout:function(B,X){var C;if(!this.rendered){X.position();X.addClass(\"x-border-layout-ct\");var M=B.items.items;C=[];for(var Q=0,R=M.length;Q<R;Q++){var U=M[Q];var F=U.region;if(U.collapsed){C.push(U)}U.collapsed=false;if(!U.rendered){U.cls=U.cls?U.cls+\" x-border-panel\":\"x-border-panel\";U.render(X,Q)}this[F]=F!=\"center\"&&U.split?new Ext.layout.BorderLayout.SplitRegion(this,U.initialConfig,F):new Ext.layout.BorderLayout.Region(this,U.initialConfig,F);this[F].render(X,U)}this.rendered=true}var L=X.getViewSize();if(L.width<20||L.height<20){if(C){this.restoreCollapsed=C}return }else{if(this.restoreCollapsed){C=this.restoreCollapsed;delete this.restoreCollapsed}}var J=L.width,S=L.height;var I=J,P=S,G=0,H=0;var N=this.north,K=this.south,E=this.west,T=this.east,U=this.center;if(!U){throw\"No center region defined in BorderLayout \"+B.id}if(N&&N.isVisible()){var W=N.getSize();var O=N.getMargins();W.width=J-(O.left+O.right);W.x=O.left;W.y=O.top;G=W.height+W.y+O.bottom;P-=G;N.applyLayout(W)}if(K&&K.isVisible()){var W=K.getSize();var O=K.getMargins();W.width=J-(O.left+O.right);W.x=O.left;var V=(W.height+O.top+O.bottom);W.y=S-V+O.top;P-=V;K.applyLayout(W)}if(E&&E.isVisible()){var W=E.getSize();var O=E.getMargins();W.height=P-(O.top+O.bottom);W.x=O.left;W.y=G+O.top;var A=(W.width+O.left+O.right);H+=A;I-=A;E.applyLayout(W)}if(T&&T.isVisible()){var W=T.getSize();var O=T.getMargins();W.height=P-(O.top+O.bottom);var A=(W.width+O.left+O.right);W.x=J-A+O.left;W.y=G+O.top;I-=A;T.applyLayout(W)}var O=U.getMargins();var D={x:H+O.left,y:G+O.top,width:I-(O.left+O.right),height:P-(O.top+O.bottom)};U.applyLayout(D);if(C){for(var Q=0,R=C.length;Q<R;Q++){C[Q].collapse(false)}}if(Ext.isIE&&Ext.isStrict){X.repaint()}}});Ext.layout.BorderLayout.Region=function(B,A,C){Ext.apply(this,A);this.layout=B;this.position=C;this.state={};if(typeof this.margins==\"string\"){this.margins=this.layout.parseMargins(this.margins)}this.margins=Ext.applyIf(this.margins||{},this.defaultMargins);if(this.collapsible){if(typeof this.cmargins==\"string\"){this.cmargins=this.layout.parseMargins(this.cmargins)}if(this.collapseMode==\"mini\"&&!this.cmargins){this.cmargins={left:0,top:0,right:0,bottom:0}}else{this.cmargins=Ext.applyIf(this.cmargins||{},C==\"north\"||C==\"south\"?this.defaultNSCMargins:this.defaultEWCMargins)}}};Ext.layout.BorderLayout.Region.prototype={collapsible:false,split:false,floatable:true,minWidth:50,minHeight:50,defaultMargins:{left:0,top:0,right:0,bottom:0},defaultNSCMargins:{left:5,top:5,right:5,bottom:5},defaultEWCMargins:{left:5,top:0,right:5,bottom:0},isCollapsed:false,render:function(B,C){this.panel=C;C.el.enableDisplayMode();this.targetEl=B;this.el=C.el;var A=C.getState,D=this.position;C.getState=function(){return Ext.apply(A.call(C)||{},this.state)}.createDelegate(this);if(D!=\"center\"){C.allowQueuedExpand=false;C.on({beforecollapse:this.beforeCollapse,collapse:this.onCollapse,beforeexpand:this.beforeExpand,expand:this.onExpand,hide:this.onHide,show:this.onShow,scope:this});if(this.collapsible){C.collapseEl=\"el\";C.slideAnchor=this.getSlideAnchor()}if(C.tools&&C.tools.toggle){C.tools.toggle.addClass(\"x-tool-collapse-\"+D);C.tools.toggle.addClassOnOver(\"x-tool-collapse-\"+D+\"-over\")}}},getCollapsedEl:function(){if(!this.collapsedEl){if(!this.toolTemplate){var B=new Ext.Template(\"<div class=\\\"x-tool x-tool-{id}\\\">&#160;</div>\");B.disableFormats=true;B.compile();Ext.layout.BorderLayout.Region.prototype.toolTemplate=B}this.collapsedEl=this.targetEl.createChild({cls:\"x-layout-collapsed x-layout-collapsed-\"+this.position,id:this.panel.id+\"-xcollapsed\"});this.collapsedEl.enableDisplayMode(\"block\");if(this.collapseMode==\"mini\"){this.collapsedEl.addClass(\"x-layout-cmini-\"+this.position);this.miniCollapsedEl=this.collapsedEl.createChild({cls:\"x-layout-mini x-layout-mini-\"+this.position,html:\"&#160;\"});this.miniCollapsedEl.addClassOnOver(\"x-layout-mini-over\");this.collapsedEl.addClassOnOver(\"x-layout-collapsed-over\");this.collapsedEl.on(\"click\",this.onExpandClick,this,{stopEvent:true})}else{var A=this.toolTemplate.append(this.collapsedEl.dom,{id:\"expand-\"+this.position},true);A.addClassOnOver(\"x-tool-expand-\"+this.position+\"-over\");A.on(\"click\",this.onExpandClick,this,{stopEvent:true});if(this.floatable!==false){this.collapsedEl.addClassOnOver(\"x-layout-collapsed-over\");this.collapsedEl.on(\"click\",this.collapseClick,this)}}}return this.collapsedEl},onExpandClick:function(A){if(this.isSlid){this.afterSlideIn();this.panel.expand(false)}else{this.panel.expand()}},onCollapseClick:function(A){this.panel.collapse()},beforeCollapse:function(B,A){this.lastAnim=A;if(this.splitEl){this.splitEl.hide()}this.getCollapsedEl().show();this.panel.el.setStyle(\"z-index\",100);this.isCollapsed=true;this.layout.layout()},onCollapse:function(A){this.panel.el.setStyle(\"z-index\",1);if(this.lastAnim===false||this.panel.animCollapse===false){this.getCollapsedEl().dom.style.visibility=\"visible\"}else{this.getCollapsedEl().slideIn(this.panel.slideAnchor,{duration:0.2})}this.state.collapsed=true;this.panel.saveState()},beforeExpand:function(A){var B=this.getCollapsedEl();this.el.show();if(this.position==\"east\"||this.position==\"west\"){this.panel.setSize(undefined,B.getHeight())}else{this.panel.setSize(B.getWidth(),undefined)}B.hide();B.dom.style.visibility=\"hidden\";this.panel.el.setStyle(\"z-index\",100)},onExpand:function(){this.isCollapsed=false;if(this.splitEl){this.splitEl.show()}this.layout.layout();this.panel.el.setStyle(\"z-index\",1);this.state.collapsed=false;this.panel.saveState()},collapseClick:function(A){if(this.isSlid){A.stopPropagation();this.slideIn()}else{A.stopPropagation();this.slideOut()}},onHide:function(){if(this.isCollapsed){this.getCollapsedEl().hide()}else{if(this.splitEl){this.splitEl.hide()}}},onShow:function(){if(this.isCollapsed){this.getCollapsedEl().show()}else{if(this.splitEl){this.splitEl.show()}}},isVisible:function(){return !this.panel.hidden},getMargins:function(){return this.isCollapsed&&this.cmargins?this.cmargins:this.margins},getSize:function(){return this.isCollapsed?this.getCollapsedEl().getSize():this.panel.getSize()},setPanel:function(A){this.panel=A},getMinWidth:function(){return this.minWidth},getMinHeight:function(){return this.minHeight},applyLayoutCollapsed:function(A){var B=this.getCollapsedEl();B.setLeftTop(A.x,A.y);B.setSize(A.width,A.height)},applyLayout:function(A){if(this.isCollapsed){this.applyLayoutCollapsed(A)}else{this.panel.setPosition(A.x,A.y);this.panel.setSize(A.width,A.height)}},beforeSlide:function(){this.panel.beforeEffect()},afterSlide:function(){this.panel.afterEffect()},initAutoHide:function(){if(this.autoHide!==false){if(!this.autoHideHd){var A=new Ext.util.DelayedTask(this.slideIn,this);this.autoHideHd={\"mouseout\":function(B){if(!B.within(this.el,true)){A.delay(500)}},\"mouseover\":function(B){A.cancel()},scope:this}}this.el.on(this.autoHideHd)}},clearAutoHide:function(){if(this.autoHide!==false){this.el.un(\"mouseout\",this.autoHideHd.mouseout);this.el.un(\"mouseover\",this.autoHideHd.mouseover)}},clearMonitor:function(){Ext.getDoc().un(\"click\",this.slideInIf,this)},slideOut:function(){if(this.isSlid||this.el.hasActiveFx()){return }this.isSlid=true;var A=this.panel.tools;if(A&&A.toggle){A.toggle.hide()}this.el.show();if(this.position==\"east\"||this.position==\"west\"){this.panel.setSize(undefined,this.collapsedEl.getHeight())}else{this.panel.setSize(this.collapsedEl.getWidth(),undefined)}this.restoreLT=[this.el.dom.style.left,this.el.dom.style.top];this.el.alignTo(this.collapsedEl,this.getCollapseAnchor());this.el.setStyle(\"z-index\",102);if(this.animFloat!==false){this.beforeSlide();this.el.slideIn(this.getSlideAnchor(),{callback:function(){this.afterSlide();this.initAutoHide();Ext.getDoc().on(\"click\",this.slideInIf,this)},scope:this,block:true})}else{this.initAutoHide();Ext.getDoc().on(\"click\",this.slideInIf,this)}},afterSlideIn:function(){this.clearAutoHide();this.isSlid=false;this.clearMonitor();this.el.setStyle(\"z-index\",\"\");this.el.dom.style.left=this.restoreLT[0];this.el.dom.style.top=this.restoreLT[1];var A=this.panel.tools;if(A&&A.toggle){A.toggle.show()}},slideIn:function(A){if(!this.isSlid||this.el.hasActiveFx()){Ext.callback(A);return }this.isSlid=false;if(this.animFloat!==false){this.beforeSlide();this.el.slideOut(this.getSlideAnchor(),{callback:function(){this.el.hide();this.afterSlide();this.afterSlideIn();Ext.callback(A)},scope:this,block:true})}else{this.el.hide();this.afterSlideIn()}},slideInIf:function(A){if(!A.within(this.el)){this.slideIn()}},anchors:{\"west\":\"left\",\"east\":\"right\",\"north\":\"top\",\"south\":\"bottom\"},sanchors:{\"west\":\"l\",\"east\":\"r\",\"north\":\"t\",\"south\":\"b\"},canchors:{\"west\":\"tl-tr\",\"east\":\"tr-tl\",\"north\":\"tl-bl\",\"south\":\"bl-tl\"},getAnchor:function(){return this.anchors[this.position]},getCollapseAnchor:function(){return this.canchors[this.position]},getSlideAnchor:function(){return this.sanchors[this.position]},getAlignAdj:function(){var A=this.cmargins;switch(this.position){case\"west\":return[0,0];break;case\"east\":return[0,0];break;case\"north\":return[0,0];break;case\"south\":return[0,0];break}},getExpandAdj:function(){var B=this.collapsedEl,A=this.cmargins;switch(this.position){case\"west\":return[-(A.right+B.getWidth()+A.left),0];break;case\"east\":return[A.right+B.getWidth()+A.left,0];break;case\"north\":return[0,-(A.top+A.bottom+B.getHeight())];break;case\"south\":return[0,A.top+A.bottom+B.getHeight()];break}}};Ext.layout.BorderLayout.SplitRegion=function(B,A,C){Ext.layout.BorderLayout.SplitRegion.superclass.constructor.call(this,B,A,C);this.applyLayout=this.applyFns[C]};Ext.extend(Ext.layout.BorderLayout.SplitRegion,Ext.layout.BorderLayout.Region,{splitTip:\"Drag to resize.\",collapsibleSplitTip:\"Drag to resize. Double click to hide.\",useSplitTips:false,splitSettings:{north:{orientation:Ext.SplitBar.VERTICAL,placement:Ext.SplitBar.TOP,maxFn:\"getVMaxSize\",minProp:\"minHeight\",maxProp:\"maxHeight\"},south:{orientation:Ext.SplitBar.VERTICAL,placement:Ext.SplitBar.BOTTOM,maxFn:\"getVMaxSize\",minProp:\"minHeight\",maxProp:\"maxHeight\"},east:{orientation:Ext.SplitBar.HORIZONTAL,placement:Ext.SplitBar.RIGHT,maxFn:\"getHMaxSize\",minProp:\"minWidth\",maxProp:\"maxWidth\"},west:{orientation:Ext.SplitBar.HORIZONTAL,placement:Ext.SplitBar.LEFT,maxFn:\"getHMaxSize\",minProp:\"minWidth\",maxProp:\"maxWidth\"}},applyFns:{west:function(C){if(this.isCollapsed){return this.applyLayoutCollapsed(C)}var D=this.splitEl.dom,B=D.style;this.panel.setPosition(C.x,C.y);var A=D.offsetWidth;B.left=(C.x+C.width-A)+\"px\";B.top=(C.y)+\"px\";B.height=Math.max(0,C.height)+\"px\";this.panel.setSize(C.width-A,C.height)},east:function(C){if(this.isCollapsed){return this.applyLayoutCollapsed(C)}var D=this.splitEl.dom,B=D.style;var A=D.offsetWidth;this.panel.setPosition(C.x+A,C.y);B.left=(C.x)+\"px\";B.top=(C.y)+\"px\";B.height=Math.max(0,C.height)+\"px\";this.panel.setSize(C.width-A,C.height)},north:function(C){if(this.isCollapsed){return this.applyLayoutCollapsed(C)}var D=this.splitEl.dom,B=D.style;var A=D.offsetHeight;this.panel.setPosition(C.x,C.y);B.left=(C.x)+\"px\";B.top=(C.y+C.height-A)+\"px\";B.width=Math.max(0,C.width)+\"px\";this.panel.setSize(C.width,C.height-A)},south:function(C){if(this.isCollapsed){return this.applyLayoutCollapsed(C)}var D=this.splitEl.dom,B=D.style;var A=D.offsetHeight;this.panel.setPosition(C.x,C.y+A);B.left=(C.x)+\"px\";B.top=(C.y)+\"px\";B.width=Math.max(0,C.width)+\"px\";this.panel.setSize(C.width,C.height-A)}},render:function(A,C){Ext.layout.BorderLayout.SplitRegion.superclass.render.call(this,A,C);var D=this.position;this.splitEl=A.createChild({cls:\"x-layout-split x-layout-split-\"+D,html:\"&#160;\",id:this.panel.id+\"-xsplit\"});if(this.collapseMode==\"mini\"){this.miniSplitEl=this.splitEl.createChild({cls:\"x-layout-mini x-layout-mini-\"+D,html:\"&#160;\"});this.miniSplitEl.addClassOnOver(\"x-layout-mini-over\");this.miniSplitEl.on(\"click\",this.onCollapseClick,this,{stopEvent:true})}var B=this.splitSettings[D];this.split=new Ext.SplitBar(this.splitEl.dom,C.el,B.orientation);this.split.placement=B.placement;this.split.getMaximumSize=this[B.maxFn].createDelegate(this);this.split.minSize=this.minSize||this[B.minProp];this.split.on(\"beforeapply\",this.onSplitMove,this);this.split.useShim=this.useShim===true;this.maxSize=this.maxSize||this[B.maxProp];if(C.hidden){this.splitEl.hide()}if(this.useSplitTips){this.splitEl.dom.title=this.collapsible?this.collapsibleSplitTip:this.splitTip}if(this.collapsible){this.splitEl.on(\"dblclick\",this.onCollapseClick,this)}},getSize:function(){if(this.isCollapsed){return this.collapsedEl.getSize()}var A=this.panel.getSize();if(this.position==\"north\"||this.position==\"south\"){A.height+=this.splitEl.dom.offsetHeight}else{A.width+=this.splitEl.dom.offsetWidth}return A},getHMaxSize:function(){var B=this.maxSize||10000;var A=this.layout.center;return Math.min(B,(this.el.getWidth()+A.el.getWidth())-A.getMinWidth())},getVMaxSize:function(){var B=this.maxSize||10000;var A=this.layout.center;return Math.min(B,(this.el.getHeight()+A.el.getHeight())-A.getMinHeight())},onSplitMove:function(B,A){var C=this.panel.getSize();this.lastSplitSize=A;if(this.position==\"north\"||this.position==\"south\"){this.panel.setSize(C.width,A);this.state.height=A}else{this.panel.setSize(A,C.height);this.state.width=A}this.layout.layout();this.panel.saveState();return false},getSplitBar:function(){return this.split}});Ext.Container.LAYOUTS[\"border\"]=Ext.layout.BorderLayout;\nExt.layout.FormLayout=Ext.extend(Ext.layout.AnchorLayout,{labelSeparator:\":\",getAnchorViewSize:function(A,B){return A.body.getStyleSize()},setContainer:function(B){Ext.layout.FormLayout.superclass.setContainer.call(this,B);if(B.labelAlign){B.addClass(\"x-form-label-\"+B.labelAlign)}if(B.hideLabels){this.labelStyle=\"display:none\";this.elementStyle=\"padding-left:0;\";this.labelAdjust=0}else{this.labelSeparator=B.labelSeparator||this.labelSeparator;B.labelWidth=B.labelWidth||100;if(typeof B.labelWidth==\"number\"){var C=(typeof B.labelPad==\"number\"?B.labelPad:5);this.labelAdjust=B.labelWidth+C;this.labelStyle=\"width:\"+B.labelWidth+\"px;\";this.elementStyle=\"padding-left:\"+(B.labelWidth+C)+\"px\"}if(B.labelAlign==\"top\"){this.labelStyle=\"width:auto;\";this.labelAdjust=0;this.elementStyle=\"padding-left:0;\"}}if(!this.fieldTpl){var A=new Ext.Template(\"<div class=\\\"x-form-item {5}\\\" tabIndex=\\\"-1\\\">\",\"<label for=\\\"{0}\\\" style=\\\"{2}\\\" class=\\\"x-form-item-label\\\">{1}{4}</label>\",\"<div class=\\\"x-form-element\\\" id=\\\"x-form-el-{0}\\\" style=\\\"{3}\\\">\",\"</div><div class=\\\"{6}\\\"></div>\",\"</div>\");A.disableFormats=true;A.compile();Ext.layout.FormLayout.prototype.fieldTpl=A}},renderItem:function(D,A,C){if(D&&!D.rendered&&D.isFormField&&D.inputType!=\"hidden\"){var B=[D.id,D.fieldLabel,D.labelStyle||this.labelStyle||\"\",this.elementStyle||\"\",typeof D.labelSeparator==\"undefined\"?this.labelSeparator:D.labelSeparator,(D.itemCls||this.container.itemCls||\"\")+(D.hideLabel?\" x-hide-label\":\"\"),D.clearCls||\"x-form-clear-left\"];if(typeof A==\"number\"){A=C.dom.childNodes[A]||null}if(A){this.fieldTpl.insertBefore(A,B)}else{this.fieldTpl.append(C,B)}D.render(\"x-form-el-\"+D.id)}else{Ext.layout.FormLayout.superclass.renderItem.apply(this,arguments)}},adjustWidthAnchor:function(B,A){return B-(A.isFormField?(A.hideLabel?0:this.labelAdjust):0)},isValidParent:function(B,A){return true}});Ext.Container.LAYOUTS[\"form\"]=Ext.layout.FormLayout;\nExt.layout.Accordion=Ext.extend(Ext.layout.FitLayout,{fill:true,autoWidth:true,titleCollapse:true,hideCollapseTool:false,collapseFirst:false,animate:false,sequence:false,activeOnTop:false,renderItem:function(A){if(this.animate===false){A.animCollapse=false}A.collapsible=true;if(this.autoWidth){A.autoWidth=true}if(this.titleCollapse){A.titleCollapse=true}if(this.hideCollapseTool){A.hideCollapseTool=true}if(this.collapseFirst!==undefined){A.collapseFirst=this.collapseFirst}if(!this.activeItem&&!A.collapsed){this.activeItem=A}else{if(this.activeItem){A.collapsed=true}}Ext.layout.Accordion.superclass.renderItem.apply(this,arguments);A.header.addClass(\"x-accordion-hd\");A.on(\"beforeexpand\",this.beforeExpand,this)},beforeExpand:function(C,B){var A=this.activeItem;if(A){if(this.sequence){delete this.activeItem;A.collapse({callback:function(){C.expand(B||true)},scope:this});return false}else{A.collapse(this.animate)}}this.activeItem=C;if(this.activeOnTop){C.el.dom.parentNode.insertBefore(C.el.dom,C.el.dom.parentNode.firstChild)}this.layout()},setItemSize:function(F,E){if(this.fill&&F){var B=this.container.items.items;var D=0;for(var C=0,A=B.length;C<A;C++){var G=B[C];if(G!=F){D+=(G.getSize().height-G.bwrap.getHeight())}}E.height-=D;F.setSize(E)}}});Ext.Container.LAYOUTS[\"accordion\"]=Ext.layout.Accordion;\nExt.layout.TableLayout=Ext.extend(Ext.layout.ContainerLayout,{monitorResize:false,setContainer:function(A){Ext.layout.TableLayout.superclass.setContainer.call(this,A);this.currentRow=0;this.currentColumn=0;this.cells=[]},onLayout:function(C,E){var D=C.items.items,A=D.length,F,B;if(!this.table){E.addClass(\"x-table-layout-ct\");this.table=E.createChild({tag:\"table\",cls:\"x-table-layout\",cellspacing:0,cn:{tag:\"tbody\"}},null,true);this.renderAll(C,E)}},getRow:function(A){var B=this.table.tBodies[0].childNodes[A];if(!B){B=document.createElement(\"tr\");this.table.tBodies[0].appendChild(B)}return B},getNextCell:function(H){var A=this.getNextNonSpan(this.currentColumn,this.currentRow);var E=this.currentColumn=A[0],D=this.currentRow=A[1];for(var G=D;G<D+(H.rowspan||1);G++){if(!this.cells[G]){this.cells[G]=[]}for(var C=E;C<E+(H.colspan||1);C++){this.cells[G][C]=true}}var F=document.createElement(\"td\");if(H.cellId){F.id=H.cellId}var B=\"x-table-layout-cell\";if(H.cellCls){B+=\" \"+H.cellCls}F.className=B;if(H.colspan){F.colSpan=H.colspan}if(H.rowspan){F.rowSpan=H.rowspan}this.getRow(D).appendChild(F);return F},getNextNonSpan:function(A,C){var B=this.columns;while((B&&A>=B)||(this.cells[C]&&this.cells[C][A])){if(B&&A>=B){C++;A=0}else{A++}}return[A,C]},renderItem:function(C,A,B){if(C&&!C.rendered){C.render(this.getNextCell(C))}},isValidParent:function(B,A){return true}});Ext.Container.LAYOUTS[\"table\"]=Ext.layout.TableLayout;\nExt.layout.AbsoluteLayout=Ext.extend(Ext.layout.AnchorLayout,{extraCls:\"x-abs-layout-item\",isForm:false,setContainer:function(A){Ext.layout.AbsoluteLayout.superclass.setContainer.call(this,A);if(A.isXType(\"form\")){this.isForm=true}},onLayout:function(A,B){if(this.isForm){A.body.position()}else{B.position()}Ext.layout.AbsoluteLayout.superclass.onLayout.call(this,A,B)},getAnchorViewSize:function(A,B){return this.isForm?A.body.getStyleSize():Ext.layout.AbsoluteLayout.superclass.getAnchorViewSize.call(this,A,B)},isValidParent:function(B,A){return this.isForm?true:Ext.layout.AbsoluteLayout.superclass.isValidParent.call(this,B,A)},adjustWidthAnchor:function(B,A){return B?B-A.getPosition(true)[0]:B},adjustHeightAnchor:function(B,A){return B?B-A.getPosition(true)[1]:B}});Ext.Container.LAYOUTS[\"absolute\"]=Ext.layout.AbsoluteLayout;\nExt.Viewport=Ext.extend(Ext.Container,{initComponent:function(){Ext.Viewport.superclass.initComponent.call(this);document.getElementsByTagName(\"html\")[0].className+=\" x-viewport\";this.el=Ext.getBody();this.el.setHeight=Ext.emptyFn;this.el.setWidth=Ext.emptyFn;this.el.setSize=Ext.emptyFn;this.el.dom.scroll=\"no\";this.allowDomMove=false;this.autoWidth=true;this.autoHeight=true;Ext.EventManager.onWindowResize(this.fireResize,this);this.renderTo=this.el},fireResize:function(A,B){this.fireEvent(\"resize\",this,A,B,A,B)}});Ext.reg(\"viewport\",Ext.Viewport);\nExt.Panel=Ext.extend(Ext.Container,{baseCls:\"x-panel\",collapsedCls:\"x-panel-collapsed\",maskDisabled:true,animCollapse:Ext.enableFx,headerAsText:true,buttonAlign:\"right\",collapsed:false,collapseFirst:true,minButtonWidth:75,elements:\"body\",toolTarget:\"header\",collapseEl:\"bwrap\",slideAnchor:\"t\",deferHeight:true,expandDefaults:{duration:0.25},collapseDefaults:{duration:0.25},initComponent:function(){Ext.Panel.superclass.initComponent.call(this);this.addEvents(\"bodyresize\",\"titlechange\",\"collapse\",\"expand\",\"beforecollapse\",\"beforeexpand\",\"beforeclose\",\"close\",\"activate\",\"deactivate\");if(this.tbar){this.elements+=\",tbar\";if(typeof this.tbar==\"object\"){this.topToolbar=this.tbar}delete this.tbar}if(this.bbar){this.elements+=\",bbar\";if(typeof this.bbar==\"object\"){this.bottomToolbar=this.bbar}delete this.bbar}if(this.header===true){this.elements+=\",header\";delete this.header}else{if(this.title&&this.header!==false){this.elements+=\",header\"}}if(this.footer===true){this.elements+=\",footer\";delete this.footer}if(this.buttons){var C=this.buttons;this.buttons=[];for(var B=0,A=C.length;B<A;B++){if(C[B].render){this.buttons.push(C[B])}else{this.addButton(C[B])}}}if(this.autoLoad){this.on(\"render\",this.doAutoLoad,this,{delay:10})}},createElement:function(A,C){if(this[A]){C.appendChild(this[A].dom);return }if(A===\"bwrap\"||this.elements.indexOf(A)!=-1){if(this[A+\"Cfg\"]){this[A]=Ext.fly(C).createChild(this[A+\"Cfg\"])}else{var B=document.createElement(\"div\");B.className=this[A+\"Cls\"];this[A]=Ext.get(C.appendChild(B))}}},onRender:function(H,G){Ext.Panel.superclass.onRender.call(this,H,G);this.createClasses();if(this.el){this.el.addClass(this.baseCls);this.header=this.el.down(\".\"+this.headerCls);this.bwrap=this.el.down(\".\"+this.bwrapCls);var M=this.bwrap?this.bwrap:this.el;this.tbar=M.down(\".\"+this.tbarCls);this.body=M.down(\".\"+this.bodyCls);this.bbar=M.down(\".\"+this.bbarCls);this.footer=M.down(\".\"+this.footerCls);this.fromMarkup=true}else{this.el=H.createChild({id:this.id,cls:this.baseCls},G)}var A=this.el,K=A.dom;if(this.cls){this.el.addClass(this.cls)}if(this.buttons){this.elements+=\",footer\"}if(this.frame){A.insertHtml(\"afterBegin\",String.format(Ext.Element.boxMarkup,this.baseCls));this.createElement(\"header\",K.firstChild.firstChild.firstChild);this.createElement(\"bwrap\",K);var O=this.bwrap.dom;var E=K.childNodes[1],B=K.childNodes[2];O.appendChild(E);O.appendChild(B);var P=O.firstChild.firstChild.firstChild;this.createElement(\"tbar\",P);this.createElement(\"body\",P);this.createElement(\"bbar\",P);this.createElement(\"footer\",O.lastChild.firstChild.firstChild);if(!this.footer){this.bwrap.dom.lastChild.className+=\" x-panel-nofooter\"}}else{this.createElement(\"header\",K);this.createElement(\"bwrap\",K);var O=this.bwrap.dom;this.createElement(\"tbar\",O);this.createElement(\"body\",O);this.createElement(\"bbar\",O);this.createElement(\"footer\",O);if(!this.header){this.body.addClass(this.bodyCls+\"-noheader\");if(this.tbar){this.tbar.addClass(this.tbarCls+\"-noheader\")}}}if(this.border===false){this.el.addClass(this.baseCls+\"-noborder\");this.body.addClass(this.bodyCls+\"-noborder\");if(this.header){this.header.addClass(this.headerCls+\"-noborder\")}if(this.footer){this.footer.addClass(this.footerCls+\"-noborder\")}if(this.tbar){this.tbar.addClass(this.tbarCls+\"-noborder\")}if(this.bbar){this.bbar.addClass(this.bbarCls+\"-noborder\")}}if(this.bodyBorder===false){this.body.addClass(this.bodyCls+\"-noborder\")}if(this.bodyStyle){this.body.applyStyles(this.bodyStyle)}this.bwrap.enableDisplayMode(\"block\");if(this.header){this.header.unselectable();if(this.headerAsText){this.header.dom.innerHTML=\"<span class=\\\"\"+this.headerTextCls+\"\\\">\"+this.header.dom.innerHTML+\"</span>\";if(this.iconCls){this.setIconClass(this.iconCls)}}}if(this.floating){this.makeFloating(this.floating)}if(this.collapsible){this.tools=this.tools?this.tools.slice(0):[];if(!this.hideCollapseTool){this.tools[this.collapseFirst?\"unshift\":\"push\"]({id:\"toggle\",handler:this.toggleCollapse,scope:this})}if(this.titleCollapse&&this.header){this.header.on(\"click\",this.toggleCollapse,this);this.header.setStyle(\"cursor\",\"pointer\")}}if(this.tools){var J=this.tools;this.tools={};this.addTool.apply(this,J)}else{this.tools={}}if(this.buttons&&this.buttons.length>0){var D=this.footer.createChild({cls:\"x-panel-btns-ct\",cn:{cls:\"x-panel-btns x-panel-btns-\"+this.buttonAlign,html:\"<table cellspacing=\\\"0\\\"><tbody><tr></tr></tbody></table><div class=\\\"x-clear\\\"></div>\"}},null,true);var L=D.getElementsByTagName(\"tr\")[0];for(var F=0,I=this.buttons.length;F<I;F++){var N=this.buttons[F];var C=document.createElement(\"td\");C.className=\"x-panel-btn-td\";N.render(L.appendChild(C))}}if(this.tbar&&this.topToolbar){if(Ext.isArray(this.topToolbar)){this.topToolbar=new Ext.Toolbar(this.topToolbar)}this.topToolbar.render(this.tbar)}if(this.bbar&&this.bottomToolbar){if(Ext.isArray(this.bottomToolbar)){this.bottomToolbar=new Ext.Toolbar(this.bottomToolbar)}this.bottomToolbar.render(this.bbar)}},setIconClass:function(B){var A=this.iconCls;this.iconCls=B;if(this.rendered&&this.header){if(this.frame){this.header.addClass(\"x-panel-icon\");this.header.replaceClass(A,this.iconCls)}else{var D=this.header.dom;var C=D.firstChild&&String(D.firstChild.tagName).toLowerCase()==\"img\"?D.firstChild:null;if(C){Ext.fly(C).replaceClass(A,this.iconCls)}else{Ext.DomHelper.insertBefore(D.firstChild,{tag:\"img\",src:Ext.BLANK_IMAGE_URL,cls:\"x-panel-inline-icon \"+this.iconCls})}}}},makeFloating:function(A){this.floating=true;this.el=new Ext.Layer(typeof A==\"object\"?A:{shadow:this.shadow!==undefined?this.shadow:\"sides\",shadowOffset:this.shadowOffset,constrain:false,shim:this.shim===false?false:undefined},this.el)},getTopToolbar:function(){return this.topToolbar},getBottomToolbar:function(){return this.bottomToolbar},addButton:function(A,D,C){var E={handler:D,scope:C,minWidth:this.minButtonWidth,hideParent:true};if(typeof A==\"string\"){E.text=A}else{Ext.apply(E,A)}var B=new Ext.Button(E);B.ownerCt=this;if(!this.buttons){this.buttons=[]}this.buttons.push(B);return B},addTool:function(){if(!this[this.toolTarget]){return }if(!this.toolTemplate){var F=new Ext.Template(\"<div class=\\\"x-tool x-tool-{id}\\\">&#160;</div>\");F.disableFormats=true;F.compile();Ext.Panel.prototype.toolTemplate=F}for(var E=0,C=arguments,B=C.length;E<B;E++){var A=C[E],G=\"x-tool-\"+A.id+\"-over\";var D=this.toolTemplate.insertFirst(this[this.toolTarget],A,true);this.tools[A.id]=D;D.enableDisplayMode(\"block\");D.on(\"click\",this.createToolHandler(D,A,G,this));if(A.on){D.on(A.on)}if(A.hidden){D.hide()}if(A.qtip){if(typeof A.qtip==\"object\"){Ext.QuickTips.register(Ext.apply({target:D.id},A.qtip))}else{D.dom.qtip=A.qtip}}D.addClassOnOver(G)}},onShow:function(){if(this.floating){return this.el.show()}Ext.Panel.superclass.onShow.call(this)},onHide:function(){if(this.floating){return this.el.hide()}Ext.Panel.superclass.onHide.call(this)},createToolHandler:function(C,A,D,B){return function(E){C.removeClass(D);E.stopEvent();if(A.handler){A.handler.call(A.scope||C,E,C,B)}}},afterRender:function(){if(this.fromMarkup&&this.height===undefined&&!this.autoHeight){this.height=this.el.getHeight()}if(this.floating&&!this.hidden&&!this.initHidden){this.el.show()}if(this.title){this.setTitle(this.title)}this.setAutoScroll();if(this.html){this.body.update(typeof this.html==\"object\"?Ext.DomHelper.markup(this.html):this.html);delete this.html}if(this.contentEl){var A=Ext.getDom(this.contentEl);Ext.fly(A).removeClass([\"x-hidden\",\"x-hide-display\"]);this.body.dom.appendChild(A)}if(this.collapsed){this.collapsed=false;this.collapse(false)}Ext.Panel.superclass.afterRender.call(this);this.initEvents()},setAutoScroll:function(){if(this.rendered&&this.autoScroll){this.body.setOverflow(\"auto\")}},getKeyMap:function(){if(!this.keyMap){this.keyMap=new Ext.KeyMap(this.el,this.keys)}return this.keyMap},initEvents:function(){if(this.keys){this.getKeyMap()}if(this.draggable){this.initDraggable()}},initDraggable:function(){this.dd=new Ext.Panel.DD(this,typeof this.draggable==\"boolean\"?null:this.draggable)},beforeEffect:function(){if(this.floating){this.el.beforeAction()}this.el.addClass(\"x-panel-animated\")},afterEffect:function(){this.syncShadow();this.el.removeClass(\"x-panel-animated\")},createEffect:function(B,A,C){var D={scope:C,block:true};if(B===true){D.callback=A;return D}else{if(!B.callback){D.callback=A}else{D.callback=function(){A.call(C);Ext.callback(B.callback,B.scope)}}}return Ext.applyIf(D,B)},collapse:function(B){if(this.collapsed||this.el.hasFxBlock()||this.fireEvent(\"beforecollapse\",this,B)===false){return }var A=B===true||(B!==false&&this.animCollapse);this.beforeEffect();this.onCollapse(A,B);return this},onCollapse:function(A,B){if(A){this[this.collapseEl].slideOut(this.slideAnchor,Ext.apply(this.createEffect(B||true,this.afterCollapse,this),this.collapseDefaults))}else{this[this.collapseEl].hide();this.afterCollapse()}},afterCollapse:function(){this.collapsed=true;this.el.addClass(this.collapsedCls);this.afterEffect();this.fireEvent(\"collapse\",this)},expand:function(B){if(!this.collapsed||this.el.hasFxBlock()||this.fireEvent(\"beforeexpand\",this,B)===false){return }var A=B===true||(B!==false&&this.animCollapse);this.el.removeClass(this.collapsedCls);this.beforeEffect();this.onExpand(A,B);return this},onExpand:function(A,B){if(A){this[this.collapseEl].slideIn(this.slideAnchor,Ext.apply(this.createEffect(B||true,this.afterExpand,this),this.expandDefaults))}else{this[this.collapseEl].show();this.afterExpand()}},afterExpand:function(){this.collapsed=false;this.afterEffect();this.fireEvent(\"expand\",this)},toggleCollapse:function(A){this[this.collapsed?\"expand\":\"collapse\"](A);return this},onDisable:function(){if(this.rendered&&this.maskDisabled){this.el.mask()}Ext.Panel.superclass.onDisable.call(this)},onEnable:function(){if(this.rendered&&this.maskDisabled){this.el.unmask()}Ext.Panel.superclass.onEnable.call(this)},onResize:function(A,B){if(A!==undefined||B!==undefined){if(!this.collapsed){if(typeof A==\"number\"){this.body.setWidth(this.adjustBodyWidth(A-this.getFrameWidth()))}else{if(A==\"auto\"){this.body.setWidth(A)}}if(typeof B==\"number\"){this.body.setHeight(this.adjustBodyHeight(B-this.getFrameHeight()))}else{if(B==\"auto\"){this.body.setHeight(B)}}}else{this.queuedBodySize={width:A,height:B};if(!this.queuedExpand&&this.allowQueuedExpand!==false){this.queuedExpand=true;this.on(\"expand\",function(){delete this.queuedExpand;this.onResize(this.queuedBodySize.width,this.queuedBodySize.height);this.doLayout()},this,{single:true})}}this.fireEvent(\"bodyresize\",this,A,B)}this.syncShadow()},adjustBodyHeight:function(A){return A},adjustBodyWidth:function(A){return A},onPosition:function(){this.syncShadow()},onDestroy:function(){if(this.tools){for(var B in this.tools){Ext.destroy(this.tools[B])}}if(this.buttons){for(var A in this.buttons){Ext.destroy(this.buttons[A])}}Ext.destroy(this.topToolbar,this.bottomToolbar);Ext.Panel.superclass.onDestroy.call(this)},getFrameWidth:function(){var B=this.el.getFrameWidth(\"lr\");if(this.frame){var A=this.bwrap.dom.firstChild;B+=(Ext.fly(A).getFrameWidth(\"l\")+Ext.fly(A.firstChild).getFrameWidth(\"r\"));var C=this.bwrap.dom.firstChild.firstChild.firstChild;B+=Ext.fly(C).getFrameWidth(\"lr\")}return B},getFrameHeight:function(){var A=this.el.getFrameWidth(\"tb\");A+=(this.tbar?this.tbar.getHeight():0)+(this.bbar?this.bbar.getHeight():0);if(this.frame){var C=this.el.dom.firstChild;var D=this.bwrap.dom.lastChild;A+=(C.offsetHeight+D.offsetHeight);var B=this.bwrap.dom.firstChild.firstChild.firstChild;A+=Ext.fly(B).getFrameWidth(\"tb\")}else{A+=(this.header?this.header.getHeight():0)+(this.footer?this.footer.getHeight():0)}return A},getInnerWidth:function(){return this.getSize().width-this.getFrameWidth()},getInnerHeight:function(){return this.getSize().height-this.getFrameHeight()},syncShadow:function(){if(this.floating){this.el.sync(true)}},getLayoutTarget:function(){return this.body},setTitle:function(B,A){this.title=B;if(this.header&&this.headerAsText){this.header.child(\"span\").update(B)}if(A){this.setIconClass(A)}this.fireEvent(\"titlechange\",this,B);return this},getUpdater:function(){return this.body.getUpdater()},load:function(){var A=this.body.getUpdater();A.update.apply(A,arguments);return this},beforeDestroy:function(){Ext.Element.uncache(this.header,this.tbar,this.bbar,this.footer,this.body)},createClasses:function(){this.headerCls=this.baseCls+\"-header\";this.headerTextCls=this.baseCls+\"-header-text\";this.bwrapCls=this.baseCls+\"-bwrap\";this.tbarCls=this.baseCls+\"-tbar\";this.bodyCls=this.baseCls+\"-body\";this.bbarCls=this.baseCls+\"-bbar\";this.footerCls=this.baseCls+\"-footer\"},createGhost:function(A,E,B){var D=document.createElement(\"div\");D.className=\"x-panel-ghost \"+(A?A:\"\");if(this.header){D.appendChild(this.el.dom.firstChild.cloneNode(true))}Ext.fly(D.appendChild(document.createElement(\"ul\"))).setHeight(this.bwrap.getHeight());D.style.width=this.el.dom.offsetWidth+\"px\";if(!B){this.container.dom.appendChild(D)}else{Ext.getDom(B).appendChild(D)}if(E!==false&&this.el.useShim!==false){var C=new Ext.Layer({shadow:false,useDisplay:true,constrain:false},D);C.show();return C}else{return new Ext.Element(D)}},doAutoLoad:function(){this.body.load(typeof this.autoLoad==\"object\"?this.autoLoad:{url:this.autoLoad})}});Ext.reg(\"panel\",Ext.Panel);\nExt.Window=Ext.extend(Ext.Panel,{baseCls:\"x-window\",resizable:true,draggable:true,closable:true,constrain:false,constrainHeader:false,plain:false,minimizable:false,maximizable:false,minHeight:100,minWidth:200,expandOnShow:true,closeAction:\"close\",collapsible:false,initHidden:true,monitorResize:true,elements:\"header,body\",frame:true,floating:true,initComponent:function(){Ext.Window.superclass.initComponent.call(this);this.addEvents(\"resize\",\"maximize\",\"minimize\",\"restore\")},getState:function(){return Ext.apply(Ext.Window.superclass.getState.call(this)||{},this.getBox())},onRender:function(B,A){Ext.Window.superclass.onRender.call(this,B,A);if(this.plain){this.el.addClass(\"x-window-plain\")}this.focusEl=this.el.createChild({tag:\"a\",href:\"#\",cls:\"x-dlg-focus\",tabIndex:\"-1\",html:\"&#160;\"});this.focusEl.swallowEvent(\"click\",true);this.proxy=this.el.createProxy(\"x-window-proxy\");this.proxy.enableDisplayMode(\"block\");if(this.modal){this.mask=this.container.createChild({cls:\"ext-el-mask\"},this.el.dom);this.mask.enableDisplayMode(\"block\");this.mask.hide()}},initEvents:function(){Ext.Window.superclass.initEvents.call(this);if(this.animateTarget){this.setAnimateTarget(this.animateTarget)}if(this.resizable){this.resizer=new Ext.Resizable(this.el,{minWidth:this.minWidth,minHeight:this.minHeight,handles:this.resizeHandles||\"all\",pinned:true,resizeElement:this.resizerAction});this.resizer.window=this;this.resizer.on(\"beforeresize\",this.beforeResize,this)}if(this.draggable){this.header.addClass(\"x-window-draggable\")}this.initTools();this.el.on(\"mousedown\",this.toFront,this);this.manager=this.manager||Ext.WindowMgr;this.manager.register(this);this.hidden=true;if(this.maximized){this.maximized=false;this.maximize()}if(this.closable){var A=this.getKeyMap();A.on(27,this.onEsc,this);A.disable()}},initDraggable:function(){this.dd=new Ext.Window.DD(this)},onEsc:function(){this[this.closeAction]()},beforeDestroy:function(){Ext.destroy(this.resizer,this.dd,this.proxy,this.mask);Ext.Window.superclass.beforeDestroy.call(this)},onDestroy:function(){if(this.manager){this.manager.unregister(this)}Ext.Window.superclass.onDestroy.call(this)},initTools:function(){if(this.minimizable){this.addTool({id:\"minimize\",handler:this.minimize.createDelegate(this,[])})}if(this.maximizable){this.addTool({id:\"maximize\",handler:this.maximize.createDelegate(this,[])});this.addTool({id:\"restore\",handler:this.restore.createDelegate(this,[]),hidden:true});this.header.on(\"dblclick\",this.toggleMaximize,this)}if(this.closable){this.addTool({id:\"close\",handler:this[this.closeAction].createDelegate(this,[])})}},resizerAction:function(){var A=this.proxy.getBox();this.proxy.hide();this.window.handleResize(A);return A},beforeResize:function(){this.resizer.minHeight=Math.max(this.minHeight,this.getFrameHeight()+40);this.resizer.minWidth=Math.max(this.minWidth,this.getFrameWidth()+40);this.resizeBox=this.el.getBox()},updateHandles:function(){if(Ext.isIE&&this.resizer){this.resizer.syncHandleHeight();this.el.repaint()}},handleResize:function(B){var A=this.resizeBox;if(A.x!=B.x||A.y!=B.y){this.updateBox(B)}else{this.setSize(B)}this.focus();this.updateHandles();this.saveState();this.fireEvent(\"resize\",this,B.width,B.height)},focus:function(){var C=this.focusEl,A=this.defaultButton,B=typeof A;if(B!=\"undefined\"){if(B==\"number\"){C=this.buttons[A]}else{if(B==\"string\"){C=Ext.getCmp(A)}else{C=A}}}C.focus.defer(10,C)},setAnimateTarget:function(A){A=Ext.get(A);this.animateTarget=A},beforeShow:function(){delete this.el.lastXY;delete this.el.lastLT;if(this.x===undefined||this.y===undefined){var A=this.el.getAlignToXY(this.container,\"c-c\");var B=this.el.translatePoints(A[0],A[1]);this.x=this.x===undefined?B.left:this.x;this.y=this.y===undefined?B.top:this.y}this.el.setLeftTop(this.x,this.y);if(this.expandOnShow){this.expand(false)}if(this.modal){Ext.getBody().addClass(\"x-body-masked\");this.mask.setSize(Ext.lib.Dom.getViewWidth(true),Ext.lib.Dom.getViewHeight(true));this.mask.show()}},show:function(C,A,B){if(!this.rendered){this.render(Ext.getBody())}if(this.hidden===false){this.toFront();return }if(this.fireEvent(\"beforeshow\",this)===false){return }if(A){this.on(\"show\",A,B,{single:true})}this.hidden=false;if(C!==undefined){this.setAnimateTarget(C)}this.beforeShow();if(this.animateTarget){this.animShow()}else{this.afterShow()}},afterShow:function(){this.proxy.hide();this.el.setStyle(\"display\",\"block\");this.el.show();if(this.maximized){this.fitContainer()}if(Ext.isMac&&Ext.isGecko){this.cascade(this.setAutoScroll)}if(this.monitorResize||this.modal||this.constrain||this.constrainHeader){Ext.EventManager.onWindowResize(this.onWindowResize,this)}this.doConstrain();if(this.layout){this.doLayout()}if(this.keyMap){this.keyMap.enable()}this.toFront();this.updateHandles();this.fireEvent(\"show\",this)},animShow:function(){this.proxy.show();this.proxy.setBox(this.animateTarget.getBox());this.proxy.setOpacity(0);var A=this.getBox(false);A.callback=this.afterShow;A.scope=this;A.duration=0.25;A.easing=\"easeNone\";A.opacity=0.5;A.block=true;this.el.setStyle(\"display\",\"none\");this.proxy.shift(A)},hide:function(C,A,B){if(this.hidden||this.fireEvent(\"beforehide\",this)===false){return }if(A){this.on(\"hide\",A,B,{single:true})}this.hidden=true;if(C!==undefined){this.setAnimateTarget(C)}if(this.animateTarget){this.animHide()}else{this.el.hide();this.afterHide()}},afterHide:function(){this.proxy.hide();if(this.monitorResize||this.modal||this.constrain||this.constrainHeader){Ext.EventManager.removeResizeListener(this.onWindowResize,this)}if(this.modal){this.mask.hide();Ext.getBody().removeClass(\"x-body-masked\")}if(this.keyMap){this.keyMap.disable()}this.fireEvent(\"hide\",this)},animHide:function(){this.proxy.setOpacity(0.5);this.proxy.show();var B=this.getBox(false);this.proxy.setBox(B);this.el.hide();var A=this.animateTarget.getBox();A.callback=this.afterHide;A.scope=this;A.duration=0.25;A.easing=\"easeNone\";A.block=true;A.opacity=0;this.proxy.shift(A)},onWindowResize:function(){if(this.maximized){this.fitContainer()}if(this.modal){this.mask.setSize(\"100%\",\"100%\");var A=this.mask.dom.offsetHeight;this.mask.setSize(Ext.lib.Dom.getViewWidth(true),Ext.lib.Dom.getViewHeight(true))}this.doConstrain()},doConstrain:function(){if(this.constrain||this.constrainHeader){var B;if(this.constrain){B={right:this.el.shadowOffset,left:this.el.shadowOffset,bottom:this.el.shadowOffset}}else{var A=this.getSize();B={right:-(A.width-100),bottom:-(A.height-25)}}var C=this.el.getConstrainToXY(this.container,true,B);if(C){this.setPosition(C[0],C[1])}}},ghost:function(A){var C=this.createGhost(A);var B=this.getBox(true);C.setLeftTop(B.x,B.y);C.setWidth(B.width);this.el.hide();this.activeGhost=C;return C},unghost:function(B,A){if(B!==false){this.el.show();this.focus();if(Ext.isMac&&Ext.isGecko){this.cascade(this.setAutoScroll)}}if(A!==false){this.setPosition(this.activeGhost.getLeft(true),this.activeGhost.getTop(true))}this.activeGhost.hide();this.activeGhost.remove();delete this.activeGhost},minimize:function(){this.fireEvent(\"minimize\",this)},close:function(){if(this.fireEvent(\"beforeclose\",this)!==false){this.hide(null,function(){this.fireEvent(\"close\",this);this.destroy()},this)}},maximize:function(){if(!this.maximized){this.expand(false);this.restoreSize=this.getSize();this.restorePos=this.getPosition(true);this.tools.maximize.hide();this.tools.restore.show();this.maximized=true;this.el.disableShadow();if(this.dd){this.dd.lock()}if(this.collapsible){this.tools.toggle.hide()}this.el.addClass(\"x-window-maximized\");this.container.addClass(\"x-window-maximized-ct\");this.setPosition(0,0);this.fitContainer();this.fireEvent(\"maximize\",this)}},restore:function(){if(this.maximized){this.el.removeClass(\"x-window-maximized\");this.tools.restore.hide();this.tools.maximize.show();this.setPosition(this.restorePos[0],this.restorePos[1]);this.setSize(this.restoreSize.width,this.restoreSize.height);delete this.restorePos;delete this.restoreSize;this.maximized=false;this.el.enableShadow(true);if(this.dd){this.dd.unlock()}if(this.collapsible){this.tools.toggle.show()}this.container.removeClass(\"x-window-maximized-ct\");this.doConstrain();this.fireEvent(\"restore\",this)}},toggleMaximize:function(){this[this.maximized?\"restore\":\"maximize\"]()},fitContainer:function(){var A=this.container.getViewSize();this.setSize(A.width,A.height)},setZIndex:function(A){if(this.modal){this.mask.setStyle(\"z-index\",A)}this.el.setZIndex(++A);A+=5;if(this.resizer){this.resizer.proxy.setStyle(\"z-index\",++A)}this.lastZIndex=A},alignTo:function(B,A,C){var D=this.el.getAlignToXY(B,A,C);this.setPagePosition(D[0],D[1]);return this},anchorTo:function(C,G,D,B,F){var E=function(){this.alignTo(C,G,D)};Ext.EventManager.onWindowResize(E,this);var A=typeof B;if(A!=\"undefined\"){Ext.EventManager.on(window,\"scroll\",E,this,{buffer:A==\"number\"?B:50})}E.call(this);this[F]=E;return this},toFront:function(){if(this.manager.bringToFront(this)){this.focus()}return this},setActive:function(A){if(A){if(!this.maximized){this.el.enableShadow(true)}this.fireEvent(\"activate\",this)}else{this.el.disableShadow();this.fireEvent(\"deactivate\",this)}},toBack:function(){this.manager.sendToBack(this);return this},center:function(){var A=this.el.getAlignToXY(this.container,\"c-c\");this.setPagePosition(A[0],A[1]);return this}});Ext.reg(\"window\",Ext.Window);Ext.Window.DD=function(A){this.win=A;Ext.Window.DD.superclass.constructor.call(this,A.el.id,\"WindowDD-\"+A.id);this.setHandleElId(A.header.id);this.scroll=false};Ext.extend(Ext.Window.DD,Ext.dd.DD,{moveOnly:true,headerOffsets:[100,25],startDrag:function(){var A=this.win;this.proxy=A.ghost();if(A.constrain!==false){var C=A.el.shadowOffset;this.constrainTo(A.container,{right:C,left:C,bottom:C})}else{if(A.constrainHeader!==false){var B=this.proxy.getSize();this.constrainTo(A.container,{right:-(B.width-this.headerOffsets[0]),bottom:-(B.height-this.headerOffsets[1])})}}},b4Drag:Ext.emptyFn,onDrag:function(A){this.alignElWithMouse(this.proxy,A.getPageX(),A.getPageY())},endDrag:function(A){this.win.unghost();this.win.saveState()}});\nExt.WindowGroup=function(){var F={};var D=[];var E=null;var C=function(I,H){return(!I._lastAccess||I._lastAccess<H._lastAccess)?-1:1};var G=function(){var J=D,H=J.length;if(H>0){J.sort(C);var I=J[0].manager.zseed;for(var K=0;K<H;K++){var L=J[K];if(L&&!L.hidden){L.setZIndex(I+(K*10))}}}A()};var B=function(H){if(H!=E){if(E){E.setActive(false)}E=H;if(H){H.setActive(true)}}};var A=function(){for(var H=D.length-1;H>=0;--H){if(!D[H].hidden){B(D[H]);return }}B(null)};return{zseed:9000,register:function(H){F[H.id]=H;D.push(H);H.on(\"hide\",A)},unregister:function(H){delete F[H.id];H.un(\"hide\",A);D.remove(H)},get:function(H){return typeof H==\"object\"?H:F[H]},bringToFront:function(H){H=this.get(H);if(H!=E){H._lastAccess=new Date().getTime();G();return true}return false},sendToBack:function(H){H=this.get(H);H._lastAccess=-(new Date().getTime());G();return H},hideAll:function(){for(var H in F){if(F[H]&&typeof F[H]!=\"function\"&&F[H].isVisible()){F[H].hide()}}},getActive:function(){return E},getBy:function(J,I){var K=[];for(var H=D.length-1;H>=0;--H){var L=D[H];if(J.call(I||L,L)!==false){K.push(L)}}return K},each:function(I,H){for(var J in F){if(F[J]&&typeof F[J]!=\"function\"){if(I.call(H||F[J],F[J])===false){return }}}}}};Ext.WindowMgr=new Ext.WindowGroup();\nExt.dd.PanelProxy=function(A,B){this.panel=A;this.id=this.panel.id+\"-ddproxy\";Ext.apply(this,B)};Ext.dd.PanelProxy.prototype={insertProxy:true,setStatus:Ext.emptyFn,reset:Ext.emptyFn,update:Ext.emptyFn,stop:Ext.emptyFn,sync:Ext.emptyFn,getEl:function(){return this.ghost},getGhost:function(){return this.ghost},getProxy:function(){return this.proxy},hide:function(){if(this.ghost){if(this.proxy){this.proxy.remove();delete this.proxy}this.panel.el.dom.style.display=\"\";this.ghost.remove();delete this.ghost}},show:function(){if(!this.ghost){this.ghost=this.panel.createGhost(undefined,undefined,Ext.getBody());this.ghost.setXY(this.panel.el.getXY());if(this.insertProxy){this.proxy=this.panel.el.insertSibling({cls:\"x-panel-dd-spacer\"});this.proxy.setSize(this.panel.getSize())}this.panel.el.dom.style.display=\"none\"}},repair:function(B,C,A){this.hide();if(typeof C==\"function\"){C.call(A||this)}},moveProxy:function(A,B){if(this.proxy){A.insertBefore(this.proxy.dom,B)}}};Ext.Panel.DD=function(B,A){this.panel=B;this.dragData={panel:B};this.proxy=new Ext.dd.PanelProxy(B,A);Ext.Panel.DD.superclass.constructor.call(this,B.el,A);this.setHandleElId(B.header.id);B.header.setStyle(\"cursor\",\"move\");this.scroll=false};Ext.extend(Ext.Panel.DD,Ext.dd.DragSource,{showFrame:Ext.emptyFn,startDrag:Ext.emptyFn,b4StartDrag:function(A,B){this.proxy.show()},b4MouseDown:function(B){var A=B.getPageX();var C=B.getPageY();this.autoOffset(A,C)},onInitDrag:function(A,B){this.onStartDrag(A,B);return true},createFrame:Ext.emptyFn,getDragEl:function(A){return this.proxy.ghost.dom},endDrag:function(A){this.proxy.hide();this.panel.saveState()},autoOffset:function(A,B){A-=this.startPageX;B-=this.startPageY;this.setDelta(A,B)}});\nExt.state.Provider=function(){this.addEvents(\"statechange\");this.state={};Ext.state.Provider.superclass.constructor.call(this)};Ext.extend(Ext.state.Provider,Ext.util.Observable,{get:function(B,A){return typeof this.state[B]==\"undefined\"?A:this.state[B]},clear:function(A){delete this.state[A];this.fireEvent(\"statechange\",this,A,null)},set:function(A,B){this.state[A]=B;this.fireEvent(\"statechange\",this,A,B)},decodeValue:function(A){var J=/^(a|n|d|b|s|o)\\:(.*)$/;var C=J.exec(unescape(A));if(!C||!C[1]){return }var F=C[1];var H=C[2];switch(F){case\"n\":return parseFloat(H);case\"d\":return new Date(Date.parse(H));case\"b\":return(H==\"1\");case\"a\":var G=[];var I=H.split(\"^\");for(var B=0,D=I.length;B<D;B++){G.push(this.decodeValue(I[B]))}return G;case\"o\":var G={};var I=H.split(\"^\");for(var B=0,D=I.length;B<D;B++){var E=I[B].split(\"=\");G[E[0]]=this.decodeValue(E[1])}return G;default:return H}},encodeValue:function(C){var B;if(typeof C==\"number\"){B=\"n:\"+C}else{if(typeof C==\"boolean\"){B=\"b:\"+(C?\"1\":\"0\")}else{if(Ext.isDate(C)){B=\"d:\"+C.toGMTString()}else{if(Ext.isArray(C)){var F=\"\";for(var E=0,A=C.length;E<A;E++){F+=this.encodeValue(C[E]);if(E!=A-1){F+=\"^\"}}B=\"a:\"+F}else{if(typeof C==\"object\"){var F=\"\";for(var D in C){if(typeof C[D]!=\"function\"&&C[D]!==undefined){F+=D+\"=\"+this.encodeValue(C[D])+\"^\"}}B=\"o:\"+F.substring(0,F.length-1)}else{B=\"s:\"+C}}}}}return escape(B)}});\nExt.state.Manager=function(){var A=new Ext.state.Provider();return{setProvider:function(B){A=B},get:function(C,B){return A.get(C,B)},set:function(B,C){A.set(B,C)},clear:function(B){A.clear(B)},getProvider:function(){return A}}}();\nExt.state.CookieProvider=function(A){Ext.state.CookieProvider.superclass.constructor.call(this);this.path=\"/\";this.expires=new Date(new Date().getTime()+(1000*60*60*24*7));this.domain=null;this.secure=false;Ext.apply(this,A);this.state=this.readCookies()};Ext.extend(Ext.state.CookieProvider,Ext.state.Provider,{set:function(A,B){if(typeof B==\"undefined\"||B===null){this.clear(A);return }this.setCookie(A,B);Ext.state.CookieProvider.superclass.set.call(this,A,B)},clear:function(A){this.clearCookie(A);Ext.state.CookieProvider.superclass.clear.call(this,A)},readCookies:function(){var C={};var F=document.cookie+\";\";var B=/\\s?(.*?)=(.*?);/g;var E;while((E=B.exec(F))!=null){var A=E[1];var D=E[2];if(A&&A.substring(0,3)==\"ys-\"){C[A.substr(3)]=this.decodeValue(D)}}return C},setCookie:function(A,B){document.cookie=\"ys-\"+A+\"=\"+this.encodeValue(B)+((this.expires==null)?\"\":(\"; expires=\"+this.expires.toGMTString()))+((this.path==null)?\"\":(\"; path=\"+this.path))+((this.domain==null)?\"\":(\"; domain=\"+this.domain))+((this.secure==true)?\"; secure\":\"\")},clearCookie:function(A){document.cookie=\"ys-\"+A+\"=null; expires=Thu, 01-Jan-70 00:00:01 GMT\"+((this.path==null)?\"\":(\"; path=\"+this.path))+((this.domain==null)?\"\":(\"; domain=\"+this.domain))+((this.secure==true)?\"; secure\":\"\")}});\nExt.DataView=Ext.extend(Ext.BoxComponent,{selectedClass:\"x-view-selected\",emptyText:\"\",last:false,initComponent:function(){Ext.DataView.superclass.initComponent.call(this);if(typeof this.tpl==\"string\"){this.tpl=new Ext.XTemplate(this.tpl)}this.addEvents(\"beforeclick\",\"click\",\"containerclick\",\"dblclick\",\"contextmenu\",\"selectionchange\",\"beforeselect\");this.all=new Ext.CompositeElementLite();this.selected=new Ext.CompositeElementLite()},onRender:function(){if(!this.el){this.el=document.createElement(\"div\")}Ext.DataView.superclass.onRender.apply(this,arguments)},afterRender:function(){Ext.DataView.superclass.afterRender.call(this);this.el.on({\"click\":this.onClick,\"dblclick\":this.onDblClick,\"contextmenu\":this.onContextMenu,scope:this});if(this.overClass){this.el.on({\"mouseover\":this.onMouseOver,\"mouseout\":this.onMouseOut,scope:this})}if(this.store){this.setStore(this.store,true)}},refresh:function(){this.clearSelections(false,true);this.el.update(\"\");var B=[];var A=this.store.getRange();if(A.length<1){this.el.update(this.emptyText);this.all.clear();return }this.tpl.overwrite(this.el,this.collectData(A,0));this.all.fill(Ext.query(this.itemSelector,this.el.dom));this.updateIndexes(0)},prepareData:function(A){return A},collectData:function(B,E){var D=[];for(var C=0,A=B.length;C<A;C++){D[D.length]=this.prepareData(B[C].data,E+C,B[C])}return D},bufferRender:function(A){var B=document.createElement(\"div\");this.tpl.overwrite(B,this.collectData(A));return Ext.query(this.itemSelector,B)},onUpdate:function(F,A){var B=this.store.indexOf(A);var E=this.isSelected(B);var C=this.all.elements[B];var D=this.bufferRender([A],B)[0];this.all.replaceElement(B,D,true);if(E){this.selected.replaceElement(C,D);this.all.item(B).addClass(this.selectedClass)}this.updateIndexes(B,B)},onAdd:function(D,B,C){if(this.all.getCount()==0){this.refresh();return }var A=this.bufferRender(B,C),E;if(C<this.all.getCount()){E=this.all.item(C).insertSibling(A,\"before\",true);this.all.elements.splice(C,0,E)}else{E=this.all.last().insertSibling(A,\"after\",true);this.all.elements.push(E)}this.updateIndexes(C)},onRemove:function(C,A,B){this.deselect(B);this.all.removeElement(B,true);this.updateIndexes(B)},refreshNode:function(A){this.onUpdate(this.store,this.store.getAt(A))},updateIndexes:function(D,C){var B=this.all.elements;D=D||0;C=C||((C===0)?0:(B.length-1));for(var A=D;A<=C;A++){B[A].viewIndex=A}},setStore:function(A,B){if(!B&&this.store){this.store.un(\"beforeload\",this.onBeforeLoad,this);this.store.un(\"datachanged\",this.refresh,this);this.store.un(\"add\",this.onAdd,this);this.store.un(\"remove\",this.onRemove,this);this.store.un(\"update\",this.onUpdate,this);this.store.un(\"clear\",this.refresh,this)}if(A){A=Ext.StoreMgr.lookup(A);A.on(\"beforeload\",this.onBeforeLoad,this);A.on(\"datachanged\",this.refresh,this);A.on(\"add\",this.onAdd,this);A.on(\"remove\",this.onRemove,this);A.on(\"update\",this.onUpdate,this);A.on(\"clear\",this.refresh,this)}this.store=A;if(A){this.refresh()}},findItemFromChild:function(A){return Ext.fly(A).findParent(this.itemSelector,this.el)},onClick:function(C){var B=C.getTarget(this.itemSelector,this.el);if(B){var A=this.indexOf(B);if(this.onItemClick(B,A,C)!==false){this.fireEvent(\"click\",this,A,B,C)}}else{if(this.fireEvent(\"containerclick\",this,C)!==false){this.clearSelections()}}},onContextMenu:function(B){var A=B.getTarget(this.itemSelector,this.el);if(A){this.fireEvent(\"contextmenu\",this,this.indexOf(A),A,B)}},onDblClick:function(B){var A=B.getTarget(this.itemSelector,this.el);if(A){this.fireEvent(\"dblclick\",this,this.indexOf(A),A,B)}},onMouseOver:function(B){var A=B.getTarget(this.itemSelector,this.el);if(A&&A!==this.lastItem){this.lastItem=A;Ext.fly(A).addClass(this.overClass)}},onMouseOut:function(A){if(this.lastItem){if(!A.within(this.lastItem,true)){Ext.fly(this.lastItem).removeClass(this.overClass);delete this.lastItem}}},onItemClick:function(B,A,C){if(this.fireEvent(\"beforeclick\",this,A,B,C)===false){return false}if(this.multiSelect){this.doMultiSelection(B,A,C);C.preventDefault()}else{if(this.singleSelect){this.doSingleSelection(B,A,C);C.preventDefault()}}return true},doSingleSelection:function(B,A,C){if(C.ctrlKey&&this.isSelected(A)){this.deselect(A)}else{this.select(A,false)}},doMultiSelection:function(C,A,D){if(D.shiftKey&&this.last!==false){var B=this.last;this.selectRange(B,A,D.ctrlKey);this.last=B}else{if((D.ctrlKey||this.simpleSelect)&&this.isSelected(A)){this.deselect(A)}else{this.select(A,D.ctrlKey||D.shiftKey||this.simpleSelect)}}},getSelectionCount:function(){return this.selected.getCount()},getSelectedNodes:function(){return this.selected.elements},getSelectedIndexes:function(){var B=[],D=this.selected.elements;for(var C=0,A=D.length;C<A;C++){B.push(D[C].viewIndex)}return B},getSelectedRecords:function(){var D=[],C=this.selected.elements;for(var B=0,A=C.length;B<A;B++){D[D.length]=this.store.getAt(C[B].viewIndex)}return D},getRecords:function(B){var E=[],D=B;for(var C=0,A=D.length;C<A;C++){E[E.length]=this.store.getAt(D[C].viewIndex)}return E},getRecord:function(A){return this.store.getAt(A.viewIndex)},clearSelections:function(A,B){if(this.multiSelect||this.singleSelect){if(!B){this.selected.removeClass(this.selectedClass)}this.selected.clear();this.last=false;if(!A){this.fireEvent(\"selectionchange\",this,this.selected.elements)}}},isSelected:function(A){return this.selected.contains(this.getNode(A))},deselect:function(A){if(this.isSelected(A)){var A=this.getNode(A);this.selected.removeElement(A);if(this.last==A.viewIndex){this.last=false}Ext.fly(A).removeClass(this.selectedClass);this.fireEvent(\"selectionchange\",this,this.selected.elements)}},select:function(D,F,B){if(Ext.isArray(D)){if(!F){this.clearSelections(true)}for(var C=0,A=D.length;C<A;C++){this.select(D[C],true,true)}}else{var E=this.getNode(D);if(!F){this.clearSelections(true)}if(E&&!this.isSelected(E)){if(this.fireEvent(\"beforeselect\",this,E,this.selected.elements)!==false){Ext.fly(E).addClass(this.selectedClass);this.selected.add(E);this.last=E.viewIndex;if(!B){this.fireEvent(\"selectionchange\",this,this.selected.elements)}}}}},selectRange:function(C,A,B){if(!B){this.clearSelections(true)}this.select(this.getNodes(C,A),true)},getNode:function(A){if(typeof A==\"string\"){return document.getElementById(A)}else{if(typeof A==\"number\"){return this.all.elements[A]}}return A},getNodes:function(E,A){var D=this.all.elements;E=E||0;A=typeof A==\"undefined\"?D.length-1:A;var B=[],C;if(E<=A){for(C=E;C<=A;C++){B.push(D[C])}}else{for(C=E;C>=A;C--){B.push(D[C])}}return B},indexOf:function(A){A=this.getNode(A);if(typeof A.viewIndex==\"number\"){return A.viewIndex}return this.all.indexOf(A)},onBeforeLoad:function(){if(this.loadingText){this.clearSelections(false,true);this.el.update(\"<div class=\\\"loading-indicator\\\">\"+this.loadingText+\"</div>\");this.all.clear()}}});Ext.reg(\"dataview\",Ext.DataView);\nExt.ColorPalette=function(A){Ext.ColorPalette.superclass.constructor.call(this,A);this.addEvents(\"select\");if(this.handler){this.on(\"select\",this.handler,this.scope,true)}};Ext.extend(Ext.ColorPalette,Ext.Component,{itemCls:\"x-color-palette\",value:null,clickEvent:\"click\",ctype:\"Ext.ColorPalette\",allowReselect:false,colors:[\"000000\",\"993300\",\"333300\",\"003300\",\"003366\",\"000080\",\"333399\",\"333333\",\"800000\",\"FF6600\",\"808000\",\"008000\",\"008080\",\"0000FF\",\"666699\",\"808080\",\"FF0000\",\"FF9900\",\"99CC00\",\"339966\",\"33CCCC\",\"3366FF\",\"800080\",\"969696\",\"FF00FF\",\"FFCC00\",\"FFFF00\",\"00FF00\",\"00FFFF\",\"00CCFF\",\"993366\",\"C0C0C0\",\"FF99CC\",\"FFCC99\",\"FFFF99\",\"CCFFCC\",\"CCFFFF\",\"99CCFF\",\"CC99FF\",\"FFFFFF\"],onRender:function(B,A){var C=this.tpl||new Ext.XTemplate(\"<tpl for=\\\".\\\"><a href=\\\"#\\\" class=\\\"color-{.}\\\" hidefocus=\\\"on\\\"><em><span style=\\\"background:#{.}\\\" unselectable=\\\"on\\\">&#160;</span></em></a></tpl>\");var D=document.createElement(\"div\");D.className=this.itemCls;C.overwrite(D,this.colors);B.dom.insertBefore(D,A);this.el=Ext.get(D);this.el.on(this.clickEvent,this.handleClick,this,{delegate:\"a\"});if(this.clickEvent!=\"click\"){this.el.on(\"click\",Ext.emptyFn,this,{delegate:\"a\",preventDefault:true})}},afterRender:function(){Ext.ColorPalette.superclass.afterRender.call(this);if(this.value){var A=this.value;this.value=null;this.select(A)}},handleClick:function(B,A){B.preventDefault();if(!this.disabled){var C=A.className.match(/(?:^|\\s)color-(.{6})(?:\\s|$)/)[1];this.select(C.toUpperCase())}},select:function(A){A=A.replace(\"#\",\"\");if(A!=this.value||this.allowReselect){var B=this.el;if(this.value){B.child(\"a.color-\"+this.value).removeClass(\"x-color-palette-sel\")}B.child(\"a.color-\"+A).addClass(\"x-color-palette-sel\");this.value=A;this.fireEvent(\"select\",this,A)}}});Ext.reg(\"colorpalette\",Ext.ColorPalette);\nExt.DatePicker=Ext.extend(Ext.Component,{todayText:\"Today\",okText:\"&#160;OK&#160;\",cancelText:\"Cancel\",todayTip:\"{0} (Spacebar)\",minDate:null,maxDate:null,minText:\"This date is before the minimum date\",maxText:\"This date is after the maximum date\",format:\"m/d/y\",disabledDays:null,disabledDaysText:\"\",disabledDatesRE:null,disabledDatesText:\"\",constrainToViewport:true,monthNames:Date.monthNames,dayNames:Date.dayNames,nextText:\"Next Month (Control+Right)\",prevText:\"Previous Month (Control+Left)\",monthYearText:\"Choose a month (Control+Up/Down to move years)\",startDay:0,initComponent:function(){Ext.DatePicker.superclass.initComponent.call(this);this.value=this.value?this.value.clearTime():new Date().clearTime();this.addEvents(\"select\");if(this.handler){this.on(\"select\",this.handler,this.scope||this)}this.initDisabledDays()},initDisabledDays:function(){if(!this.disabledDatesRE&&this.disabledDates){var A=this.disabledDates;var C=\"(?:\";for(var B=0;B<A.length;B++){C+=A[B];if(B!=A.length-1){C+=\"|\"}}this.disabledDatesRE=new RegExp(C+\")\")}},setValue:function(B){var A=this.value;this.value=B.clearTime(true);if(this.el){this.update(this.value)}},getValue:function(){return this.value},focus:function(){if(this.el){this.update(this.activeDate)}},onRender:function(A,F){var C=[\"<table cellspacing=\\\"0\\\">\",\"<tr><td class=\\\"x-date-left\\\"><a href=\\\"#\\\" title=\\\"\",this.prevText,\"\\\">&#160;</a></td><td class=\\\"x-date-middle\\\" align=\\\"center\\\"></td><td class=\\\"x-date-right\\\"><a href=\\\"#\\\" title=\\\"\",this.nextText,\"\\\">&#160;</a></td></tr>\",\"<tr><td colspan=\\\"3\\\"><table class=\\\"x-date-inner\\\" cellspacing=\\\"0\\\"><thead><tr>\"];var E=this.dayNames;for(var D=0;D<7;D++){var G=this.startDay+D;if(G>6){G=G-7}C.push(\"<th><span>\",E[G].substr(0,1),\"</span></th>\")}C[C.length]=\"</tr></thead><tbody><tr>\";for(var D=0;D<42;D++){if(D%7==0&&D!=0){C[C.length]=\"</tr><tr>\"}C[C.length]=\"<td><a href=\\\"#\\\" hidefocus=\\\"on\\\" class=\\\"x-date-date\\\" tabIndex=\\\"1\\\"><em><span></span></em></a></td>\"}C[C.length]=\"</tr></tbody></table></td></tr><tr><td colspan=\\\"3\\\" class=\\\"x-date-bottom\\\" align=\\\"center\\\"></td></tr></table><div class=\\\"x-date-mp\\\"></div>\";var B=document.createElement(\"div\");B.className=\"x-date-picker\";B.innerHTML=C.join(\"\");A.dom.insertBefore(B,F);this.el=Ext.get(B);this.eventEl=Ext.get(B.firstChild);new Ext.util.ClickRepeater(this.el.child(\"td.x-date-left a\"),{handler:this.showPrevMonth,scope:this,preventDefault:true,stopDefault:true});new Ext.util.ClickRepeater(this.el.child(\"td.x-date-right a\"),{handler:this.showNextMonth,scope:this,preventDefault:true,stopDefault:true});this.eventEl.on(\"mousewheel\",this.handleMouseWheel,this);this.monthPicker=this.el.down(\"div.x-date-mp\");this.monthPicker.enableDisplayMode(\"block\");var I=new Ext.KeyNav(this.eventEl,{\"left\":function(J){J.ctrlKey?this.showPrevMonth():this.update(this.activeDate.add(\"d\",-1))},\"right\":function(J){J.ctrlKey?this.showNextMonth():this.update(this.activeDate.add(\"d\",1))},\"up\":function(J){J.ctrlKey?this.showNextYear():this.update(this.activeDate.add(\"d\",-7))},\"down\":function(J){J.ctrlKey?this.showPrevYear():this.update(this.activeDate.add(\"d\",7))},\"pageUp\":function(J){this.showNextMonth()},\"pageDown\":function(J){this.showPrevMonth()},\"enter\":function(J){J.stopPropagation();return true},scope:this});this.eventEl.on(\"click\",this.handleDateClick,this,{delegate:\"a.x-date-date\"});this.eventEl.addKeyListener(Ext.EventObject.SPACE,this.selectToday,this);this.el.unselectable();this.cells=this.el.select(\"table.x-date-inner tbody td\");this.textNodes=this.el.query(\"table.x-date-inner tbody span\");this.mbtn=new Ext.Button({text:\"&#160;\",tooltip:this.monthYearText,renderTo:this.el.child(\"td.x-date-middle\",true)});this.mbtn.on(\"click\",this.showMonthPicker,this);this.mbtn.el.child(this.mbtn.menuClassTarget).addClass(\"x-btn-with-menu\");var H=(new Date()).dateFormat(this.format);this.todayBtn=new Ext.Button({renderTo:this.el.child(\"td.x-date-bottom\",true),text:String.format(this.todayText,H),tooltip:String.format(this.todayTip,H),handler:this.selectToday,scope:this});if(Ext.isIE){this.el.repaint()}this.update(this.value)},createMonthPicker:function(){if(!this.monthPicker.dom.firstChild){var A=[\"<table border=\\\"0\\\" cellspacing=\\\"0\\\">\"];for(var B=0;B<6;B++){A.push(\"<tr><td class=\\\"x-date-mp-month\\\"><a href=\\\"#\\\">\",this.monthNames[B].substr(0,3),\"</a></td>\",\"<td class=\\\"x-date-mp-month x-date-mp-sep\\\"><a href=\\\"#\\\">\",this.monthNames[B+6].substr(0,3),\"</a></td>\",B==0?\"<td class=\\\"x-date-mp-ybtn\\\" align=\\\"center\\\"><a class=\\\"x-date-mp-prev\\\"></a></td><td class=\\\"x-date-mp-ybtn\\\" align=\\\"center\\\"><a class=\\\"x-date-mp-next\\\"></a></td></tr>\":\"<td class=\\\"x-date-mp-year\\\"><a href=\\\"#\\\"></a></td><td class=\\\"x-date-mp-year\\\"><a href=\\\"#\\\"></a></td></tr>\")}A.push(\"<tr class=\\\"x-date-mp-btns\\\"><td colspan=\\\"4\\\"><button type=\\\"button\\\" class=\\\"x-date-mp-ok\\\">\",this.okText,\"</button><button type=\\\"button\\\" class=\\\"x-date-mp-cancel\\\">\",this.cancelText,\"</button></td></tr>\",\"</table>\");this.monthPicker.update(A.join(\"\"));this.monthPicker.on(\"click\",this.onMonthClick,this);this.monthPicker.on(\"dblclick\",this.onMonthDblClick,this);this.mpMonths=this.monthPicker.select(\"td.x-date-mp-month\");this.mpYears=this.monthPicker.select(\"td.x-date-mp-year\");this.mpMonths.each(function(C,D,E){E+=1;if((E%2)==0){C.dom.xmonth=5+Math.round(E*0.5)}else{C.dom.xmonth=Math.round((E-1)*0.5)}})}},showMonthPicker:function(){this.createMonthPicker();var A=this.el.getSize();this.monthPicker.setSize(A);this.monthPicker.child(\"table\").setSize(A);this.mpSelMonth=(this.activeDate||this.value).getMonth();this.updateMPMonth(this.mpSelMonth);this.mpSelYear=(this.activeDate||this.value).getFullYear();this.updateMPYear(this.mpSelYear);this.monthPicker.slideIn(\"t\",{duration:0.2})},updateMPYear:function(E){this.mpyear=E;var C=this.mpYears.elements;for(var B=1;B<=10;B++){var D=C[B-1],A;if((B%2)==0){A=E+Math.round(B*0.5);D.firstChild.innerHTML=A;D.xyear=A}else{A=E-(5-Math.round(B*0.5));D.firstChild.innerHTML=A;D.xyear=A}this.mpYears.item(B-1)[A==this.mpSelYear?\"addClass\":\"removeClass\"](\"x-date-mp-sel\")}},updateMPMonth:function(A){this.mpMonths.each(function(B,C,D){B[B.dom.xmonth==A?\"addClass\":\"removeClass\"](\"x-date-mp-sel\")})},selectMPMonth:function(A){},onMonthClick:function(D,B){D.stopEvent();var C=new Ext.Element(B),A;if(C.is(\"button.x-date-mp-cancel\")){this.hideMonthPicker()}else{if(C.is(\"button.x-date-mp-ok\")){this.update(new Date(this.mpSelYear,this.mpSelMonth,(this.activeDate||this.value).getDate()));this.hideMonthPicker()}else{if(A=C.up(\"td.x-date-mp-month\",2)){this.mpMonths.removeClass(\"x-date-mp-sel\");A.addClass(\"x-date-mp-sel\");this.mpSelMonth=A.dom.xmonth}else{if(A=C.up(\"td.x-date-mp-year\",2)){this.mpYears.removeClass(\"x-date-mp-sel\");A.addClass(\"x-date-mp-sel\");this.mpSelYear=A.dom.xyear}else{if(C.is(\"a.x-date-mp-prev\")){this.updateMPYear(this.mpyear-10)}else{if(C.is(\"a.x-date-mp-next\")){this.updateMPYear(this.mpyear+10)}}}}}}},onMonthDblClick:function(D,B){D.stopEvent();var C=new Ext.Element(B),A;if(A=C.up(\"td.x-date-mp-month\",2)){this.update(new Date(this.mpSelYear,A.dom.xmonth,(this.activeDate||this.value).getDate()));this.hideMonthPicker()}else{if(A=C.up(\"td.x-date-mp-year\",2)){this.update(new Date(A.dom.xyear,this.mpSelMonth,(this.activeDate||this.value).getDate()));this.hideMonthPicker()}}},hideMonthPicker:function(A){if(this.monthPicker){if(A===true){this.monthPicker.hide()}else{this.monthPicker.slideOut(\"t\",{duration:0.2})}}},showPrevMonth:function(A){this.update(this.activeDate.add(\"mo\",-1))},showNextMonth:function(A){this.update(this.activeDate.add(\"mo\",1))},showPrevYear:function(){this.update(this.activeDate.add(\"y\",-1))},showNextYear:function(){this.update(this.activeDate.add(\"y\",1))},handleMouseWheel:function(A){var B=A.getWheelDelta();if(B>0){this.showPrevMonth();A.stopEvent()}else{if(B<0){this.showNextMonth();A.stopEvent()}}},handleDateClick:function(B,A){B.stopEvent();if(A.dateValue&&!Ext.fly(A.parentNode).hasClass(\"x-date-disabled\")){this.setValue(new Date(A.dateValue));this.fireEvent(\"select\",this,this.value)}},selectToday:function(){this.setValue(new Date().clearTime());this.fireEvent(\"select\",this,this.value)},update:function(W){var A=this.activeDate;this.activeDate=W;if(A&&this.el){var I=W.getTime();if(A.getMonth()==W.getMonth()&&A.getFullYear()==W.getFullYear()){this.cells.removeClass(\"x-date-selected\");this.cells.each(function(a){if(a.dom.firstChild.dateValue==I){a.addClass(\"x-date-selected\");setTimeout(function(){try{a.dom.firstChild.focus()}catch(b){}},50);return false}});return }}var F=W.getDaysInMonth();var J=W.getFirstDateOfMonth();var C=J.getDay()-this.startDay;if(C<=this.startDay){C+=7}var S=W.add(\"mo\",-1);var D=S.getDaysInMonth()-C;var B=this.cells.elements;var K=this.textNodes;F+=C;var P=86400000;var U=(new Date(S.getFullYear(),S.getMonth(),D)).clearTime();var T=new Date().clearTime().getTime();var N=W.clearTime().getTime();var M=this.minDate?this.minDate.clearTime():Number.NEGATIVE_INFINITY;var Q=this.maxDate?this.maxDate.clearTime():Number.POSITIVE_INFINITY;var X=this.disabledDatesRE;var L=this.disabledDatesText;var Z=this.disabledDays?this.disabledDays.join(\"\"):false;var V=this.disabledDaysText;var R=this.format;var G=function(d,a){a.title=\"\";var b=U.getTime();a.firstChild.dateValue=b;if(b==T){a.className+=\" x-date-today\";a.title=d.todayText}if(b==N){a.className+=\" x-date-selected\";setTimeout(function(){try{a.firstChild.focus()}catch(f){}},50)}if(b<M){a.className=\" x-date-disabled\";a.title=d.minText;return }if(b>Q){a.className=\" x-date-disabled\";a.title=d.maxText;return }if(Z){if(Z.indexOf(U.getDay())!=-1){a.title=V;a.className=\" x-date-disabled\"}}if(X&&R){var c=U.dateFormat(R);if(X.test(c)){a.title=L.replace(\"%0\",c);a.className=\" x-date-disabled\"}}};var O=0;for(;O<C;O++){K[O].innerHTML=(++D);U.setDate(U.getDate()+1);B[O].className=\"x-date-prevday\";G(this,B[O])}for(;O<F;O++){intDay=O-C+1;K[O].innerHTML=(intDay);U.setDate(U.getDate()+1);B[O].className=\"x-date-active\";G(this,B[O])}var Y=0;for(;O<42;O++){K[O].innerHTML=(++Y);U.setDate(U.getDate()+1);B[O].className=\"x-date-nextday\";G(this,B[O])}this.mbtn.setText(this.monthNames[W.getMonth()]+\" \"+W.getFullYear());if(!this.internalRender){var E=this.el.dom.firstChild;var H=E.offsetWidth;this.el.setWidth(H+this.el.getBorderWidth(\"lr\"));Ext.fly(E).setWidth(H);this.internalRender=true;if(Ext.isOpera&&!this.secondPass){E.rows[0].cells[1].style.width=(H-(E.rows[0].cells[0].offsetWidth+E.rows[0].cells[2].offsetWidth))+\"px\";this.secondPass=true;this.update.defer(10,this,[W])}}},beforeDestroy:function(){this.mbtn.destroy();this.todayBtn.destroy()}});Ext.reg(\"datepicker\",Ext.DatePicker);\nExt.TabPanel=Ext.extend(Ext.Panel,{monitorResize:true,deferredRender:true,tabWidth:120,minTabWidth:30,resizeTabs:false,enableTabScroll:false,scrollIncrement:0,scrollRepeatInterval:400,scrollDuration:0.35,animScroll:true,tabPosition:\"top\",baseCls:\"x-tab-panel\",autoTabs:false,autoTabSelector:\"div.x-tab\",activeTab:null,tabMargin:2,plain:false,wheelIncrement:20,idDelimiter:\"__\",itemCls:\"x-tab-item\",elements:\"body\",headerAsText:false,frame:false,hideBorders:true,initComponent:function(){this.frame=false;Ext.TabPanel.superclass.initComponent.call(this);this.addEvents(\"beforetabchange\",\"tabchange\",\"contextmenu\");this.setLayout(new Ext.layout.CardLayout({deferredRender:this.deferredRender}));if(this.tabPosition==\"top\"){this.elements+=\",header\";this.stripTarget=\"header\"}else{this.elements+=\",footer\";this.stripTarget=\"footer\"}if(!this.stack){this.stack=Ext.TabPanel.AccessStack()}this.initItems()},render:function(){Ext.TabPanel.superclass.render.apply(this,arguments);if(this.activeTab!==undefined){var A=this.activeTab;delete this.activeTab;this.setActiveTab(A)}},onRender:function(C,A){Ext.TabPanel.superclass.onRender.call(this,C,A);if(this.plain){var E=this.tabPosition==\"top\"?\"header\":\"footer\";this[E].addClass(\"x-tab-panel-\"+E+\"-plain\")}var B=this[this.stripTarget];this.stripWrap=B.createChild({cls:\"x-tab-strip-wrap\",cn:{tag:\"ul\",cls:\"x-tab-strip x-tab-strip-\"+this.tabPosition}});this.stripSpacer=B.createChild({cls:\"x-tab-strip-spacer\"});this.strip=new Ext.Element(this.stripWrap.dom.firstChild);this.edge=this.strip.createChild({tag:\"li\",cls:\"x-tab-edge\"});this.strip.createChild({cls:\"x-clear\"});this.body.addClass(\"x-tab-panel-body-\"+this.tabPosition);if(!this.itemTpl){var D=new Ext.Template(\"<li class=\\\"{cls}\\\" id=\\\"{id}\\\"><a class=\\\"x-tab-strip-close\\\" onclick=\\\"return false;\\\"></a>\",\"<a class=\\\"x-tab-right\\\" href=\\\"#\\\" onclick=\\\"return false;\\\"><em class=\\\"x-tab-left\\\">\",\"<span class=\\\"x-tab-strip-inner\\\"><span class=\\\"x-tab-strip-text {iconCls}\\\">{text}</span></span>\",\"</em></a></li>\");D.disableFormats=true;D.compile();Ext.TabPanel.prototype.itemTpl=D}this.items.each(this.initTab,this)},afterRender:function(){Ext.TabPanel.superclass.afterRender.call(this);if(this.autoTabs){this.readTabs(false)}},initEvents:function(){Ext.TabPanel.superclass.initEvents.call(this);this.on(\"add\",this.onAdd,this);this.on(\"remove\",this.onRemove,this);this.strip.on(\"mousedown\",this.onStripMouseDown,this);this.strip.on(\"click\",this.onStripClick,this);this.strip.on(\"contextmenu\",this.onStripContextMenu,this);if(this.enableTabScroll){this.strip.on(\"mousewheel\",this.onWheel,this)}},findTargets:function(C){var B=null;var A=C.getTarget(\"li\",this.strip);if(A){B=this.getComponent(A.id.split(this.idDelimiter)[1]);if(B.disabled){return{close:null,item:null,el:null}}}return{close:C.getTarget(\".x-tab-strip-close\",this.strip),item:B,el:A}},onStripMouseDown:function(B){B.preventDefault();if(B.button!=0){return }var A=this.findTargets(B);if(A.close){this.remove(A.item);return }if(A.item&&A.item!=this.activeTab){this.setActiveTab(A.item)}},onStripClick:function(B){var A=this.findTargets(B);if(!A.close&&A.item&&A.item!=this.activeTab){this.setActiveTab(A.item)}},onStripContextMenu:function(B){B.preventDefault();var A=this.findTargets(B);if(A.item){this.fireEvent(\"contextmenu\",this,A.item,B)}},readTabs:function(D){if(D===true){this.items.each(function(G){this.remove(G)},this)}var C=this.el.query(this.autoTabSelector);for(var B=0,A=C.length;B<A;B++){var E=C[B];var F=E.getAttribute(\"title\");E.removeAttribute(\"title\");this.add({title:F,el:E})}},initTab:function(D,B){var E=this.strip.dom.childNodes[B];var A=D.closable?\"x-tab-strip-closable\":\"\";if(D.disabled){A+=\" x-item-disabled\"}if(D.iconCls){A+=\" x-tab-with-icon\"}if(D.tabCls){A+=\" \"+D.tabCls}var F={id:this.id+this.idDelimiter+D.getItemId(),text:D.title,cls:A,iconCls:D.iconCls||\"\"};var C=E?this.itemTpl.insertBefore(E,F):this.itemTpl.append(this.strip,F);Ext.fly(C).addClassOnOver(\"x-tab-strip-over\");if(D.tabTip){Ext.fly(C).child(\"span.x-tab-strip-text\",true).qtip=D.tabTip}D.on(\"disable\",this.onItemDisabled,this);D.on(\"enable\",this.onItemEnabled,this);D.on(\"titlechange\",this.onItemTitleChanged,this);D.on(\"beforeshow\",this.onBeforeShowItem,this)},onAdd:function(C,B,A){this.initTab(B,A);if(this.items.getCount()==1){this.syncSize()}this.delegateUpdates()},onBeforeAdd:function(B){var A=B.events?(this.items.containsKey(B.getItemId())?B:null):this.items.get(B);if(A){this.setActiveTab(B);return false}Ext.TabPanel.superclass.onBeforeAdd.apply(this,arguments);var C=B.elements;B.elements=C?C.replace(\",header\",\"\"):C;B.border=(B.border===true)},onRemove:function(C,B){Ext.removeNode(this.getTabEl(B));this.stack.remove(B);if(B==this.activeTab){var A=this.stack.next();if(A){this.setActiveTab(A)}else{this.setActiveTab(0)}}this.delegateUpdates()},onBeforeShowItem:function(A){if(A!=this.activeTab){this.setActiveTab(A);return false}},onItemDisabled:function(B){var A=this.getTabEl(B);if(A){Ext.fly(A).addClass(\"x-item-disabled\")}this.stack.remove(B)},onItemEnabled:function(B){var A=this.getTabEl(B);if(A){Ext.fly(A).removeClass(\"x-item-disabled\")}},onItemTitleChanged:function(B){var A=this.getTabEl(B);if(A){Ext.fly(A).child(\"span.x-tab-strip-text\",true).innerHTML=B.title}},getTabEl:function(A){var B=(typeof A===\"number\")?this.items.items[A].getItemId():A.getItemId();return document.getElementById(this.id+this.idDelimiter+B)},onResize:function(){Ext.TabPanel.superclass.onResize.apply(this,arguments);this.delegateUpdates()},beginUpdate:function(){this.suspendUpdates=true},endUpdate:function(){this.suspendUpdates=false;this.delegateUpdates()},hideTabStripItem:function(B){B=this.getComponent(B);var A=this.getTabEl(B);if(A){A.style.display=\"none\";this.delegateUpdates()}},unhideTabStripItem:function(B){B=this.getComponent(B);var A=this.getTabEl(B);if(A){A.style.display=\"\";this.delegateUpdates()}},delegateUpdates:function(){if(this.suspendUpdates){return }if(this.resizeTabs&&this.rendered){this.autoSizeTabs()}if(this.enableTabScroll&&this.rendered){this.autoScrollTabs()}},autoSizeTabs:function(){var G=this.items.length;var B=this.tabPosition!=\"bottom\"?\"header\":\"footer\";var C=this[B].dom.offsetWidth;var A=this[B].dom.clientWidth;if(!this.resizeTabs||G<1||!A){return }var I=Math.max(Math.min(Math.floor((A-4)/G)-this.tabMargin,this.tabWidth),this.minTabWidth);this.lastTabWidth=I;var K=this.stripWrap.dom.getElementsByTagName(\"li\");for(var E=0,H=K.length-1;E<H;E++){var J=K[E];var L=J.childNodes[1].firstChild.firstChild;var F=J.offsetWidth;var D=L.offsetWidth;L.style.width=(I-(F-D))+\"px\"}},adjustBodyWidth:function(A){if(this.header){this.header.setWidth(A)}if(this.footer){this.footer.setWidth(A)}return A},setActiveTab:function(C){C=this.getComponent(C);if(!C||this.fireEvent(\"beforetabchange\",this,C,this.activeTab)===false){return }if(!this.rendered){this.activeTab=C;return }if(this.activeTab!=C){if(this.activeTab){var A=this.getTabEl(this.activeTab);if(A){Ext.fly(A).removeClass(\"x-tab-strip-active\")}this.activeTab.fireEvent(\"deactivate\",this.activeTab)}var B=this.getTabEl(C);Ext.fly(B).addClass(\"x-tab-strip-active\");this.activeTab=C;this.stack.add(C);this.layout.setActiveItem(C);if(this.layoutOnTabChange&&C.doLayout){C.doLayout()}if(this.scrolling){this.scrollToTab(C,this.animScroll)}C.fireEvent(\"activate\",C);this.fireEvent(\"tabchange\",this,C)}},getActiveTab:function(){return this.activeTab||null},getItem:function(A){return this.getComponent(A)},autoScrollTabs:function(){var G=this.items.length;var D=this.header.dom.offsetWidth;var C=this.header.dom.clientWidth;var F=this.stripWrap;var E=F.dom;var B=E.offsetWidth;var H=this.getScrollPos();var A=this.edge.getOffsetsTo(this.stripWrap)[0]+H;if(!this.enableTabScroll||G<1||B<20){return }if(A<=C){E.scrollLeft=0;F.setWidth(C);if(this.scrolling){this.scrolling=false;this.header.removeClass(\"x-tab-scrolling\");this.scrollLeft.hide();this.scrollRight.hide();if(Ext.isAir){E.style.marginLeft=\"\";E.style.marginRight=\"\"}}}else{if(!this.scrolling){this.header.addClass(\"x-tab-scrolling\");if(Ext.isAir){E.style.marginLeft=\"18px\";E.style.marginRight=\"18px\"}}C-=F.getMargins(\"lr\");F.setWidth(C>20?C:20);if(!this.scrolling){if(!this.scrollLeft){this.createScrollers()}else{this.scrollLeft.show();this.scrollRight.show()}}this.scrolling=true;if(H>(A-C)){E.scrollLeft=A-C}else{this.scrollToTab(this.activeTab,false)}this.updateScrollButtons()}},createScrollers:function(){var C=this.stripWrap.dom.offsetHeight;var A=this.header.insertFirst({cls:\"x-tab-scroller-left\"});A.setHeight(C);A.addClassOnOver(\"x-tab-scroller-left-over\");this.leftRepeater=new Ext.util.ClickRepeater(A,{interval:this.scrollRepeatInterval,handler:this.onScrollLeft,scope:this});this.scrollLeft=A;var B=this.header.insertFirst({cls:\"x-tab-scroller-right\"});B.setHeight(C);B.addClassOnOver(\"x-tab-scroller-right-over\");this.rightRepeater=new Ext.util.ClickRepeater(B,{interval:this.scrollRepeatInterval,handler:this.onScrollRight,scope:this});this.scrollRight=B},getScrollWidth:function(){return this.edge.getOffsetsTo(this.stripWrap)[0]+this.getScrollPos()},getScrollPos:function(){return parseInt(this.stripWrap.dom.scrollLeft,10)||0},getScrollArea:function(){return parseInt(this.stripWrap.dom.clientWidth,10)||0},getScrollAnim:function(){return{duration:this.scrollDuration,callback:this.updateScrollButtons,scope:this}},getScrollIncrement:function(){return this.scrollIncrement||(this.resizeTabs?this.lastTabWidth+2:100)},scrollToTab:function(E,A){if(!E){return }var C=this.getTabEl(E);var G=this.getScrollPos(),D=this.getScrollArea();var F=Ext.fly(C).getOffsetsTo(this.stripWrap)[0]+G;var B=F+C.offsetWidth;if(F<G){this.scrollTo(F,A)}else{if(B>(G+D)){this.scrollTo(B-D,A)}}},scrollTo:function(B,A){this.stripWrap.scrollTo(\"left\",B,A?this.getScrollAnim():false);if(!A){this.updateScrollButtons()}},onWheel:function(D){var E=D.getWheelDelta()*this.wheelIncrement*-1;D.stopEvent();var F=this.getScrollPos();var C=F+E;var A=this.getScrollWidth()-this.getScrollArea();var B=Math.max(0,Math.min(A,C));if(B!=F){this.scrollTo(B,false)}},onScrollRight:function(){var A=this.getScrollWidth()-this.getScrollArea();var C=this.getScrollPos();var B=Math.min(A,C+this.getScrollIncrement());if(B!=C){this.scrollTo(B,this.animScroll)}},onScrollLeft:function(){var B=this.getScrollPos();var A=Math.max(0,B-this.getScrollIncrement());if(A!=B){this.scrollTo(A,this.animScroll)}},updateScrollButtons:function(){var A=this.getScrollPos();this.scrollLeft[A==0?\"addClass\":\"removeClass\"](\"x-tab-scroller-left-disabled\");this.scrollRight[A>=(this.getScrollWidth()-this.getScrollArea())?\"addClass\":\"removeClass\"](\"x-tab-scroller-right-disabled\")}});Ext.reg(\"tabpanel\",Ext.TabPanel);Ext.TabPanel.prototype.activate=Ext.TabPanel.prototype.setActiveTab;Ext.TabPanel.AccessStack=function(){var A=[];return{add:function(B){A.push(B);if(A.length>10){A.shift()}},remove:function(E){var D=[];for(var C=0,B=A.length;C<B;C++){if(A[C]!=E){D.push(A[C])}}A=D},next:function(){return A.pop()}}};\nExt.Button=Ext.extend(Ext.Component,{hidden:false,disabled:false,pressed:false,enableToggle:false,menuAlign:\"tl-bl?\",type:\"button\",menuClassTarget:\"tr\",clickEvent:\"click\",handleMouseEvents:true,tooltipType:\"qtip\",buttonSelector:\"button:first\",initComponent:function(){Ext.Button.superclass.initComponent.call(this);this.addEvents(\"click\",\"toggle\",\"mouseover\",\"mouseout\",\"menushow\",\"menuhide\",\"menutriggerover\",\"menutriggerout\");if(this.menu){this.menu=Ext.menu.MenuMgr.get(this.menu)}if(typeof this.toggleGroup===\"string\"){this.enableToggle=true}},onRender:function(C,A){if(!this.template){if(!Ext.Button.buttonTemplate){Ext.Button.buttonTemplate=new Ext.Template(\"<table border=\\\"0\\\" cellpadding=\\\"0\\\" cellspacing=\\\"0\\\" class=\\\"x-btn-wrap\\\"><tbody><tr>\",\"<td class=\\\"x-btn-left\\\"><i>&#160;</i></td><td class=\\\"x-btn-center\\\"><em unselectable=\\\"on\\\"><button class=\\\"x-btn-text\\\" type=\\\"{1}\\\">{0}</button></em></td><td class=\\\"x-btn-right\\\"><i>&#160;</i></td>\",\"</tr></tbody></table>\")}this.template=Ext.Button.buttonTemplate}var B,E=[this.text||\"&#160;\",this.type];if(A){B=this.template.insertBefore(A,E,true)}else{B=this.template.append(C,E,true)}var D=B.child(this.buttonSelector);D.on(\"focus\",this.onFocus,this);D.on(\"blur\",this.onBlur,this);this.initButtonEl(B,D);if(this.menu){this.el.child(this.menuClassTarget).addClass(\"x-btn-with-menu\")}Ext.ButtonToggleMgr.register(this)},initButtonEl:function(B,C){this.el=B;B.addClass(\"x-btn\");if(this.icon){C.setStyle(\"background-image\",\"url(\"+this.icon+\")\")}if(this.iconCls){C.addClass(this.iconCls);if(!this.cls){B.addClass(this.text?\"x-btn-text-icon\":\"x-btn-icon\")}}if(this.tabIndex!==undefined){C.dom.tabIndex=this.tabIndex}if(this.tooltip){if(typeof this.tooltip==\"object\"){Ext.QuickTips.register(Ext.apply({target:C.id},this.tooltip))}else{C.dom[this.tooltipType]=this.tooltip}}if(this.pressed){this.el.addClass(\"x-btn-pressed\")}if(this.handleMouseEvents){B.on(\"mouseover\",this.onMouseOver,this);B.on(\"mousedown\",this.onMouseDown,this)}if(this.menu){this.menu.on(\"show\",this.onMenuShow,this);this.menu.on(\"hide\",this.onMenuHide,this)}if(this.id){this.el.dom.id=this.el.id=this.id}if(this.repeat){var A=new Ext.util.ClickRepeater(B,typeof this.repeat==\"object\"?this.repeat:{});A.on(\"click\",this.onClick,this)}B.on(this.clickEvent,this.onClick,this)},afterRender:function(){Ext.Button.superclass.afterRender.call(this);if(Ext.isIE6){this.autoWidth.defer(1,this)}else{this.autoWidth()}},setIconClass:function(A){if(this.el){this.el.child(this.buttonSelector).replaceClass(this.iconCls,A)}this.iconCls=A},beforeDestroy:function(){if(this.rendered){var A=this.el.child(this.buttonSelector);if(A){A.removeAllListeners()}}if(this.menu){Ext.destroy(this.menu)}},onDestroy:function(){if(this.rendered){Ext.ButtonToggleMgr.unregister(this)}},autoWidth:function(){if(this.el){this.el.setWidth(\"auto\");if(Ext.isIE7&&Ext.isStrict){var A=this.el.child(this.buttonSelector);if(A&&A.getWidth()>20){A.clip();A.setWidth(Ext.util.TextMetrics.measure(A,this.text).width+A.getFrameWidth(\"lr\"))}}if(this.minWidth){if(this.el.getWidth()<this.minWidth){this.el.setWidth(this.minWidth)}}}},setHandler:function(B,A){this.handler=B;this.scope=A},setText:function(A){this.text=A;if(this.el){this.el.child(\"td.x-btn-center \"+this.buttonSelector).update(A)}this.autoWidth()},getText:function(){return this.text},toggle:function(A){A=A===undefined?!this.pressed:A;if(A!=this.pressed){if(A){this.el.addClass(\"x-btn-pressed\");this.pressed=true;this.fireEvent(\"toggle\",this,true)}else{this.el.removeClass(\"x-btn-pressed\");this.pressed=false;this.fireEvent(\"toggle\",this,false)}if(this.toggleHandler){this.toggleHandler.call(this.scope||this,this,A)}}},focus:function(){this.el.child(this.buttonSelector).focus()},onDisable:function(){if(this.el){if(!Ext.isIE6||!this.text){this.el.addClass(this.disabledClass)}this.el.dom.disabled=true}this.disabled=true},onEnable:function(){if(this.el){if(!Ext.isIE6||!this.text){this.el.removeClass(this.disabledClass)}this.el.dom.disabled=false}this.disabled=false},showMenu:function(){if(this.menu){this.menu.show(this.el,this.menuAlign)}return this},hideMenu:function(){if(this.menu){this.menu.hide()}return this},hasVisibleMenu:function(){return this.menu&&this.menu.isVisible()},onClick:function(A){if(A){A.preventDefault()}if(A.button!=0){return }if(!this.disabled){if(this.enableToggle&&(this.allowDepress!==false||!this.pressed)){this.toggle()}if(this.menu&&!this.menu.isVisible()&&!this.ignoreNextClick){this.showMenu()}this.fireEvent(\"click\",this,A);if(this.handler){this.handler.call(this.scope||this,this,A)}}},isMenuTriggerOver:function(B,A){return this.menu&&!A},isMenuTriggerOut:function(B,A){return this.menu&&!A},onMouseOver:function(B){if(!this.disabled){var A=B.within(this.el,true);if(!A){this.el.addClass(\"x-btn-over\");Ext.getDoc().on(\"mouseover\",this.monitorMouseOver,this);this.fireEvent(\"mouseover\",this,B)}if(this.isMenuTriggerOver(B,A)){this.fireEvent(\"menutriggerover\",this,this.menu,B)}}},monitorMouseOver:function(A){if(A.target!=this.el.dom&&!A.within(this.el)){Ext.getDoc().un(\"mouseover\",this.monitorMouseOver,this);this.onMouseOut(A)}},onMouseOut:function(B){var A=B.within(this.el)&&B.target!=this.el.dom;this.el.removeClass(\"x-btn-over\");this.fireEvent(\"mouseout\",this,B);if(this.isMenuTriggerOut(B,A)){this.fireEvent(\"menutriggerout\",this,this.menu,B)}},onFocus:function(A){if(!this.disabled){this.el.addClass(\"x-btn-focus\")}},onBlur:function(A){this.el.removeClass(\"x-btn-focus\")},getClickEl:function(B,A){return this.el},onMouseDown:function(A){if(!this.disabled&&A.button==0){this.getClickEl(A).addClass(\"x-btn-click\");Ext.getDoc().on(\"mouseup\",this.onMouseUp,this)}},onMouseUp:function(A){if(A.button==0){this.getClickEl(A,true).removeClass(\"x-btn-click\");Ext.getDoc().un(\"mouseup\",this.onMouseUp,this)}},onMenuShow:function(A){this.ignoreNextClick=0;this.el.addClass(\"x-btn-menu-active\");this.fireEvent(\"menushow\",this,this.menu)},onMenuHide:function(A){this.el.removeClass(\"x-btn-menu-active\");this.ignoreNextClick=this.restoreClick.defer(250,this);this.fireEvent(\"menuhide\",this,this.menu)},restoreClick:function(){this.ignoreNextClick=0}});Ext.reg(\"button\",Ext.Button);Ext.ButtonToggleMgr=function(){var A={};function B(E,G){if(G){var F=A[E.toggleGroup];for(var D=0,C=F.length;D<C;D++){if(F[D]!=E){F[D].toggle(false)}}}}return{register:function(C){if(!C.toggleGroup){return }var D=A[C.toggleGroup];if(!D){D=A[C.toggleGroup]=[]}D.push(C);C.on(\"toggle\",B)},unregister:function(C){if(!C.toggleGroup){return }var D=A[C.toggleGroup];if(D){D.remove(C);C.un(\"toggle\",B)}}}}();\nExt.SplitButton=Ext.extend(Ext.Button,{arrowSelector:\"button:last\",initComponent:function(){Ext.SplitButton.superclass.initComponent.call(this);this.addEvents(\"arrowclick\")},onRender:function(D,A){var B=new Ext.Template(\"<table cellspacing=\\\"0\\\" class=\\\"x-btn-menu-wrap x-btn\\\"><tr><td>\",\"<table cellspacing=\\\"0\\\" class=\\\"x-btn-wrap x-btn-menu-text-wrap\\\"><tbody>\",\"<tr><td class=\\\"x-btn-left\\\"><i>&#160;</i></td><td class=\\\"x-btn-center\\\"><button class=\\\"x-btn-text\\\" type=\\\"{1}\\\">{0}</button></td></tr>\",\"</tbody></table></td><td>\",\"<table cellspacing=\\\"0\\\" class=\\\"x-btn-wrap x-btn-menu-arrow-wrap\\\"><tbody>\",\"<tr><td class=\\\"x-btn-center\\\"><button class=\\\"x-btn-menu-arrow-el\\\" type=\\\"button\\\">&#160;</button></td><td class=\\\"x-btn-right\\\"><i>&#160;</i></td></tr>\",\"</tbody></table></td></tr></table>\");var C,F=[this.text||\"&#160;\",this.type];if(A){C=B.insertBefore(A,F,true)}else{C=B.append(D,F,true)}var E=C.child(this.buttonSelector);this.initButtonEl(C,E);this.arrowBtnTable=C.child(\"table:last\");if(this.arrowTooltip){C.child(this.arrowSelector).dom[this.tooltipType]=this.arrowTooltip}},autoWidth:function(){if(this.el){var C=this.el.child(\"table:first\");var B=this.el.child(\"table:last\");this.el.setWidth(\"auto\");C.setWidth(\"auto\");if(Ext.isIE7&&Ext.isStrict){var A=this.el.child(this.buttonSelector);if(A&&A.getWidth()>20){A.clip();A.setWidth(Ext.util.TextMetrics.measure(A,this.text).width+A.getFrameWidth(\"lr\"))}}if(this.minWidth){if((C.getWidth()+B.getWidth())<this.minWidth){C.setWidth(this.minWidth-B.getWidth())}}this.el.setWidth(C.getWidth()+B.getWidth())}},setArrowHandler:function(B,A){this.arrowHandler=B;this.scope=A},onClick:function(A){A.preventDefault();if(!this.disabled){if(A.getTarget(\".x-btn-menu-arrow-wrap\")){if(this.menu&&!this.menu.isVisible()&&!this.ignoreNextClick){this.showMenu()}this.fireEvent(\"arrowclick\",this,A);if(this.arrowHandler){this.arrowHandler.call(this.scope||this,this,A)}}else{if(this.enableToggle){this.toggle()}this.fireEvent(\"click\",this,A);if(this.handler){this.handler.call(this.scope||this,this,A)}}}},getClickEl:function(B,A){if(!A){return(this.lastClickEl=B.getTarget(\"table\",10,true))}return this.lastClickEl},onDisable:function(){if(this.el){if(!Ext.isIE6){this.el.addClass(\"x-item-disabled\")}this.el.child(this.buttonSelector).dom.disabled=true;this.el.child(this.arrowSelector).dom.disabled=true}this.disabled=true},onEnable:function(){if(this.el){if(!Ext.isIE6){this.el.removeClass(\"x-item-disabled\")}this.el.child(this.buttonSelector).dom.disabled=false;this.el.child(this.arrowSelector).dom.disabled=false}this.disabled=false},isMenuTriggerOver:function(A){return this.menu&&A.within(this.arrowBtnTable)&&!A.within(this.arrowBtnTable,true)},isMenuTriggerOut:function(B,A){return this.menu&&!B.within(this.arrowBtnTable)},onDestroy:function(){Ext.destroy(this.arrowBtnTable);Ext.SplitButton.superclass.onDestroy.call(this)}});Ext.MenuButton=Ext.SplitButton;Ext.reg(\"splitbutton\",Ext.SplitButton);\nExt.CycleButton=Ext.extend(Ext.SplitButton,{getItemText:function(A){if(A&&this.showText===true){var B=\"\";if(this.prependText){B+=this.prependText}B+=A.text;return B}return undefined},setActiveItem:function(C,A){if(typeof C!=\"object\"){C=this.menu.items.get(C)}if(C){if(!this.rendered){this.text=this.getItemText(C);this.iconCls=C.iconCls}else{var B=this.getItemText(C);if(B){this.setText(B)}this.setIconClass(C.iconCls)}this.activeItem=C;if(!C.checked){C.setChecked(true,true)}if(this.forceIcon){this.setIconClass(this.forceIcon)}if(!A){this.fireEvent(\"change\",this,C)}}},getActiveItem:function(){return this.activeItem},initComponent:function(){this.addEvents(\"change\");if(this.changeHandler){this.on(\"change\",this.changeHandler,this.scope||this);delete this.changeHandler}this.itemCount=this.items.length;this.menu={cls:\"x-cycle-menu\",items:[]};var D;for(var B=0,A=this.itemCount;B<A;B++){var C=this.items[B];C.group=C.group||this.id;C.itemIndex=B;C.checkHandler=this.checkHandler;C.scope=this;C.checked=C.checked||false;this.menu.items.push(C);if(C.checked){D=C}}this.setActiveItem(D,true);Ext.CycleButton.superclass.initComponent.call(this);this.on(\"click\",this.toggleSelected,this)},checkHandler:function(A,B){if(B){this.setActiveItem(A)}},toggleSelected:function(){this.menu.render();var C,A;for(var B=1;B<this.itemCount;B++){C=(this.activeItem.itemIndex+B)%this.itemCount;A=this.menu.items.itemAt(C);if(!A.disabled){A.setChecked(true);break}}}});Ext.reg(\"cycle\",Ext.CycleButton);\nExt.Toolbar=function(A){if(Ext.isArray(A)){A={buttons:A}}Ext.Toolbar.superclass.constructor.call(this,A)};(function(){var A=Ext.Toolbar;Ext.extend(A,Ext.BoxComponent,{trackMenus:true,initComponent:function(){A.superclass.initComponent.call(this);if(this.items){this.buttons=this.items}this.items=new Ext.util.MixedCollection(false,function(B){return B.itemId||B.id||Ext.id()})},autoCreate:{cls:\"x-toolbar x-small-editor\",html:\"<table cellspacing=\\\"0\\\"><tr></tr></table>\"},onRender:function(C,B){this.el=C.createChild(Ext.apply({id:this.id},this.autoCreate),B);this.tr=this.el.child(\"tr\",true)},afterRender:function(){A.superclass.afterRender.call(this);if(this.buttons){this.add.apply(this,this.buttons);delete this.buttons}},add:function(){var C=arguments,B=C.length;for(var D=0;D<B;D++){var E=C[D];if(E.isFormField){this.addField(E)}else{if(E.render){this.addItem(E)}else{if(typeof E==\"string\"){if(E==\"separator\"||E==\"-\"){this.addSeparator()}else{if(E==\" \"){this.addSpacer()}else{if(E==\"->\"){this.addFill()}else{this.addText(E)}}}}else{if(E.tagName){this.addElement(E)}else{if(typeof E==\"object\"){if(E.xtype){this.addField(Ext.ComponentMgr.create(E,\"button\"))}else{this.addButton(E)}}}}}}}},addSeparator:function(){return this.addItem(new A.Separator())},addSpacer:function(){return this.addItem(new A.Spacer())},addFill:function(){return this.addItem(new A.Fill())},addElement:function(B){return this.addItem(new A.Item(B))},addItem:function(B){var C=this.nextBlock();this.initMenuTracking(B);B.render(C);this.items.add(B);return B},addButton:function(D){if(Ext.isArray(D)){var F=[];for(var E=0,C=D.length;E<C;E++){F.push(this.addButton(D[E]))}return F}var B=D;if(!(D instanceof A.Button)){B=D.split?new A.SplitButton(D):new A.Button(D)}var G=this.nextBlock();this.initMenuTracking(B);B.render(G);this.items.add(B);return B},initMenuTracking:function(B){if(this.trackMenus&&B.menu){B.on({\"menutriggerover\":this.onButtonTriggerOver,\"menushow\":this.onButtonMenuShow,\"menuhide\":this.onButtonMenuHide,scope:this})}},addText:function(B){return this.addItem(new A.TextItem(B))},insertButton:function(C,F){if(Ext.isArray(F)){var E=[];for(var D=0,B=F.length;D<B;D++){E.push(this.insertButton(C+D,F[D]))}return E}if(!(F instanceof A.Button)){F=new A.Button(F)}var G=document.createElement(\"td\");this.tr.insertBefore(G,this.tr.childNodes[C]);this.initMenuTracking(F);F.render(G);this.items.insert(C,F);return F},addDom:function(C,B){var E=this.nextBlock();Ext.DomHelper.overwrite(E,C);var D=new A.Item(E.firstChild);D.render(E);this.items.add(D);return D},addField:function(C){var D=this.nextBlock();C.render(D);var B=new A.Item(D.firstChild);B.render(D);this.items.add(B);return B},nextBlock:function(){var B=document.createElement(\"td\");this.tr.appendChild(B);return B},onDestroy:function(){Ext.Toolbar.superclass.onDestroy.call(this);if(this.rendered){if(this.items){Ext.destroy.apply(Ext,this.items.items)}Ext.Element.uncache(this.tr)}},onDisable:function(){this.items.each(function(B){if(B.disable){B.disable()}})},onEnable:function(){this.items.each(function(B){if(B.enable){B.enable()}})},onButtonTriggerOver:function(B){if(this.activeMenuBtn&&this.activeMenuBtn!=B){this.activeMenuBtn.hideMenu();B.showMenu();this.activeMenuBtn=B}},onButtonMenuShow:function(B){this.activeMenuBtn=B},onButtonMenuHide:function(B){delete this.activeMenuBtn}});Ext.reg(\"toolbar\",Ext.Toolbar);A.Item=function(B){this.el=Ext.getDom(B);this.id=Ext.id(this.el);this.hidden=false};A.Item.prototype={getEl:function(){return this.el},render:function(B){this.td=B;B.appendChild(this.el)},destroy:function(){if(this.td&&this.td.parentNode){this.td.parentNode.removeChild(this.td)}},show:function(){this.hidden=false;this.td.style.display=\"\"},hide:function(){this.hidden=true;this.td.style.display=\"none\"},setVisible:function(B){if(B){this.show()}else{this.hide()}},focus:function(){Ext.fly(this.el).focus()},disable:function(){Ext.fly(this.td).addClass(\"x-item-disabled\");this.disabled=true;this.el.disabled=true},enable:function(){Ext.fly(this.td).removeClass(\"x-item-disabled\");this.disabled=false;this.el.disabled=false}};Ext.reg(\"tbitem\",A.Item);A.Separator=function(){var B=document.createElement(\"span\");B.className=\"ytb-sep\";A.Separator.superclass.constructor.call(this,B)};Ext.extend(A.Separator,A.Item,{enable:Ext.emptyFn,disable:Ext.emptyFn,focus:Ext.emptyFn});Ext.reg(\"tbseparator\",A.Separator);A.Spacer=function(){var B=document.createElement(\"div\");B.className=\"ytb-spacer\";A.Spacer.superclass.constructor.call(this,B)};Ext.extend(A.Spacer,A.Item,{enable:Ext.emptyFn,disable:Ext.emptyFn,focus:Ext.emptyFn});Ext.reg(\"tbspacer\",A.Spacer);A.Fill=Ext.extend(A.Spacer,{render:function(B){B.style.width=\"100%\";A.Fill.superclass.render.call(this,B)}});Ext.reg(\"tbfill\",A.Fill);A.TextItem=function(B){var C=document.createElement(\"span\");C.className=\"ytb-text\";C.innerHTML=B.text?B.text:B;A.TextItem.superclass.constructor.call(this,C)};Ext.extend(A.TextItem,A.Item,{enable:Ext.emptyFn,disable:Ext.emptyFn,focus:Ext.emptyFn});Ext.reg(\"tbtext\",A.TextItem);A.Button=Ext.extend(Ext.Button,{hideParent:true,onDestroy:function(){A.Button.superclass.onDestroy.call(this);if(this.container){this.container.remove()}}});Ext.reg(\"tbbutton\",A.Button);A.SplitButton=Ext.extend(Ext.SplitButton,{hideParent:true,onDestroy:function(){A.SplitButton.superclass.onDestroy.call(this);if(this.container){this.container.remove()}}});Ext.reg(\"tbsplit\",A.SplitButton);A.MenuButton=A.SplitButton})();\nExt.PagingToolbar=Ext.extend(Ext.Toolbar,{pageSize:20,displayMsg:\"Displaying {0} - {1} of {2}\",emptyMsg:\"No data to display\",beforePageText:\"Page\",afterPageText:\"of {0}\",firstText:\"First Page\",prevText:\"Previous Page\",nextText:\"Next Page\",lastText:\"Last Page\",refreshText:\"Refresh\",paramNames:{start:\"start\",limit:\"limit\"},initComponent:function(){Ext.PagingToolbar.superclass.initComponent.call(this);this.cursor=0;this.bind(this.store)},onRender:function(B,A){Ext.PagingToolbar.superclass.onRender.call(this,B,A);this.first=this.addButton({tooltip:this.firstText,iconCls:\"x-tbar-page-first\",disabled:true,handler:this.onClick.createDelegate(this,[\"first\"])});this.prev=this.addButton({tooltip:this.prevText,iconCls:\"x-tbar-page-prev\",disabled:true,handler:this.onClick.createDelegate(this,[\"prev\"])});this.addSeparator();this.add(this.beforePageText);this.field=Ext.get(this.addDom({tag:\"input\",type:\"text\",size:\"3\",value:\"1\",cls:\"x-tbar-page-number\"}).el);this.field.on(\"keydown\",this.onPagingKeydown,this);this.field.on(\"focus\",function(){this.dom.select()});this.afterTextEl=this.addText(String.format(this.afterPageText,1));this.field.setHeight(18);this.addSeparator();this.next=this.addButton({tooltip:this.nextText,iconCls:\"x-tbar-page-next\",disabled:true,handler:this.onClick.createDelegate(this,[\"next\"])});this.last=this.addButton({tooltip:this.lastText,iconCls:\"x-tbar-page-last\",disabled:true,handler:this.onClick.createDelegate(this,[\"last\"])});this.addSeparator();this.loading=this.addButton({tooltip:this.refreshText,iconCls:\"x-tbar-loading\",handler:this.onClick.createDelegate(this,[\"refresh\"])});if(this.displayInfo){this.displayEl=Ext.fly(this.el.dom).createChild({cls:\"x-paging-info\"})}if(this.dsLoaded){this.onLoad.apply(this,this.dsLoaded)}},updateInfo:function(){if(this.displayEl){var A=this.store.getCount();var B=A==0?this.emptyMsg:String.format(this.displayMsg,this.cursor+1,this.cursor+A,this.store.getTotalCount());this.displayEl.update(B)}},onLoad:function(A,C,F){if(!this.rendered){this.dsLoaded=[A,C,F];return }this.cursor=F.params?F.params[this.paramNames.start]:0;var E=this.getPageData(),B=E.activePage,D=E.pages;this.afterTextEl.el.innerHTML=String.format(this.afterPageText,E.pages);this.field.dom.value=B;this.first.setDisabled(B==1);this.prev.setDisabled(B==1);this.next.setDisabled(B==D);this.last.setDisabled(B==D);this.loading.enable();this.updateInfo()},getPageData:function(){var A=this.store.getTotalCount();return{total:A,activePage:Math.ceil((this.cursor+this.pageSize)/this.pageSize),pages:A<this.pageSize?1:Math.ceil(A/this.pageSize)}},onLoadError:function(){if(!this.rendered){return }this.loading.enable()},readPage:function(C){var A=this.field.dom.value,B;if(!A||isNaN(B=parseInt(A,10))){this.field.dom.value=C.activePage;return false}return B},onPagingKeydown:function(D){var B=D.getKey(),E=this.getPageData(),C;if(B==D.RETURN){D.stopEvent();if(C=this.readPage(E)){C=Math.min(Math.max(1,C),E.pages)-1;this.doLoad(C*this.pageSize)}}else{if(B==D.HOME||B==D.END){D.stopEvent();C=B==D.HOME?1:E.pages;this.field.dom.value=C}else{if(B==D.UP||B==D.PAGEUP||B==D.DOWN||B==D.PAGEDOWN){D.stopEvent();if(C=this.readPage(E)){var A=D.shiftKey?10:1;if(B==D.DOWN||B==D.PAGEDOWN){A*=-1}C+=A;if(C>=1&C<=E.pages){this.field.dom.value=C}}}}}},beforeLoad:function(){if(this.rendered&&this.loading){this.loading.disable()}},doLoad:function(C){var B={},A=this.paramNames;B[A.start]=C;B[A.limit]=this.pageSize;this.store.load({params:B})},onClick:function(E){var B=this.store;switch(E){case\"first\":this.doLoad(0);break;case\"prev\":this.doLoad(Math.max(0,this.cursor-this.pageSize));break;case\"next\":this.doLoad(this.cursor+this.pageSize);break;case\"last\":var D=B.getTotalCount();var A=D%this.pageSize;var C=A?(D-A):D-this.pageSize;this.doLoad(C);break;case\"refresh\":this.doLoad(this.cursor);break}},unbind:function(A){A=Ext.StoreMgr.lookup(A);A.un(\"beforeload\",this.beforeLoad,this);A.un(\"load\",this.onLoad,this);A.un(\"loadexception\",this.onLoadError,this);this.store=undefined},bind:function(A){A=Ext.StoreMgr.lookup(A);A.on(\"beforeload\",this.beforeLoad,this);A.on(\"load\",this.onLoad,this);A.on(\"loadexception\",this.onLoadError,this);this.store=A}});Ext.reg(\"paging\",Ext.PagingToolbar);\nExt.Resizable=function(D,E){this.el=Ext.get(D);if(E&&E.wrap){E.resizeChild=this.el;this.el=this.el.wrap(typeof E.wrap==\"object\"?E.wrap:{cls:\"xresizable-wrap\"});this.el.id=this.el.dom.id=E.resizeChild.id+\"-rzwrap\";this.el.setStyle(\"overflow\",\"hidden\");this.el.setPositioning(E.resizeChild.getPositioning());E.resizeChild.clearPositioning();if(!E.width||!E.height){var F=E.resizeChild.getSize();this.el.setSize(F.width,F.height)}if(E.pinned&&!E.adjustments){E.adjustments=\"auto\"}}this.proxy=this.el.createProxy({tag:\"div\",cls:\"x-resizable-proxy\",id:this.el.id+\"-rzproxy\"});this.proxy.unselectable();this.proxy.enableDisplayMode(\"block\");Ext.apply(this,E);if(this.pinned){this.disableTrackOver=true;this.el.addClass(\"x-resizable-pinned\")}var I=this.el.getStyle(\"position\");if(I!=\"absolute\"&&I!=\"fixed\"){this.el.setStyle(\"position\",\"relative\")}if(!this.handles){this.handles=\"s,e,se\";if(this.multiDirectional){this.handles+=\",n,w\"}}if(this.handles==\"all\"){this.handles=\"n s e w ne nw se sw\"}var M=this.handles.split(/\\s*?[,;]\\s*?| /);var C=Ext.Resizable.positions;for(var H=0,J=M.length;H<J;H++){if(M[H]&&C[M[H]]){var L=C[M[H]];this[L]=new Ext.Resizable.Handle(this,L,this.disableTrackOver,this.transparent)}}this.corner=this.southeast;if(this.handles.indexOf(\"n\")!=-1||this.handles.indexOf(\"w\")!=-1){this.updateBox=true}this.activeHandle=null;if(this.resizeChild){if(typeof this.resizeChild==\"boolean\"){this.resizeChild=Ext.get(this.el.dom.firstChild,true)}else{this.resizeChild=Ext.get(this.resizeChild,true)}}if(this.adjustments==\"auto\"){var B=this.resizeChild;var K=this.west,G=this.east,A=this.north,M=this.south;if(B&&(K||A)){B.position(\"relative\");B.setLeft(K?K.el.getWidth():0);B.setTop(A?A.el.getHeight():0)}this.adjustments=[(G?-G.el.getWidth():0)+(K?-K.el.getWidth():0),(A?-A.el.getHeight():0)+(M?-M.el.getHeight():0)-1]}if(this.draggable){this.dd=this.dynamic?this.el.initDD(null):this.el.initDDProxy(null,{dragElId:this.proxy.id});this.dd.setHandleElId(this.resizeChild?this.resizeChild.id:this.el.id)}this.addEvents(\"beforeresize\",\"resize\");if(this.width!==null&&this.height!==null){this.resizeTo(this.width,this.height)}else{this.updateChildSize()}if(Ext.isIE){this.el.dom.style.zoom=1}Ext.Resizable.superclass.constructor.call(this)};Ext.extend(Ext.Resizable,Ext.util.Observable,{resizeChild:false,adjustments:[0,0],minWidth:5,minHeight:5,maxWidth:10000,maxHeight:10000,enabled:true,animate:false,duration:0.35,dynamic:false,handles:false,multiDirectional:false,disableTrackOver:false,easing:\"easeOutStrong\",widthIncrement:0,heightIncrement:0,pinned:false,width:null,height:null,preserveRatio:false,transparent:false,minX:0,minY:0,draggable:false,resizeTo:function(B,A){this.el.setSize(B,A);this.updateChildSize();this.fireEvent(\"resize\",this,B,A,null)},startSizing:function(C,B){this.fireEvent(\"beforeresize\",this,C);if(this.enabled){if(!this.overlay){this.overlay=this.el.createProxy({tag:\"div\",cls:\"x-resizable-overlay\",html:\"&#160;\"},Ext.getBody());this.overlay.unselectable();this.overlay.enableDisplayMode(\"block\");this.overlay.on(\"mousemove\",this.onMouseMove,this);this.overlay.on(\"mouseup\",this.onMouseUp,this)}this.overlay.setStyle(\"cursor\",B.el.getStyle(\"cursor\"));this.resizing=true;this.startBox=this.el.getBox();this.startPoint=C.getXY();this.offsets=[(this.startBox.x+this.startBox.width)-this.startPoint[0],(this.startBox.y+this.startBox.height)-this.startPoint[1]];this.overlay.setSize(Ext.lib.Dom.getViewWidth(true),Ext.lib.Dom.getViewHeight(true));this.overlay.show();if(this.constrainTo){var A=Ext.get(this.constrainTo);this.resizeRegion=A.getRegion().adjust(A.getFrameWidth(\"t\"),A.getFrameWidth(\"l\"),-A.getFrameWidth(\"b\"),-A.getFrameWidth(\"r\"))}this.proxy.setStyle(\"visibility\",\"hidden\");this.proxy.show();this.proxy.setBox(this.startBox);if(!this.dynamic){this.proxy.setStyle(\"visibility\",\"visible\")}}},onMouseDown:function(A,B){if(this.enabled){B.stopEvent();this.activeHandle=A;this.startSizing(B,A)}},onMouseUp:function(B){var A=this.resizeElement();this.resizing=false;this.handleOut();this.overlay.hide();this.proxy.hide();this.fireEvent(\"resize\",this,A.width,A.height,B)},updateChildSize:function(){if(this.resizeChild){var C=this.el;var D=this.resizeChild;var B=this.adjustments;if(C.dom.offsetWidth){var A=C.getSize(true);D.setSize(A.width+B[0],A.height+B[1])}if(Ext.isIE){setTimeout(function(){if(C.dom.offsetWidth){var E=C.getSize(true);D.setSize(E.width+B[0],E.height+B[1])}},10)}}},snap:function(C,E,B){if(!E||!C){return C}var D=C;var A=C%E;if(A>0){if(A>(E/2)){D=C+(E-A)}else{D=C-A}}return Math.max(B,D)},resizeElement:function(){var A=this.proxy.getBox();if(this.updateBox){this.el.setBox(A,false,this.animate,this.duration,null,this.easing)}else{this.el.setSize(A.width,A.height,this.animate,this.duration,null,this.easing)}this.updateChildSize();if(!this.dynamic){this.proxy.hide()}return A},constrain:function(B,C,A,D){if(B-C<A){C=B-A}else{if(B-C>D){C=D-B}}return C},onMouseMove:function(S){if(this.enabled){try{if(this.resizeRegion&&!this.resizeRegion.contains(S.getPoint())){return }var Q=this.curSize||this.startBox;var I=this.startBox.x,H=this.startBox.y;var C=I,B=H;var J=Q.width,R=Q.height;var D=J,L=R;var K=this.minWidth,T=this.minHeight;var P=this.maxWidth,W=this.maxHeight;var F=this.widthIncrement;var A=this.heightIncrement;var U=S.getXY();var O=-(this.startPoint[0]-Math.max(this.minX,U[0]));var M=-(this.startPoint[1]-Math.max(this.minY,U[1]));var G=this.activeHandle.position;switch(G){case\"east\":J+=O;J=Math.min(Math.max(K,J),P);break;case\"south\":R+=M;R=Math.min(Math.max(T,R),W);break;case\"southeast\":J+=O;R+=M;J=Math.min(Math.max(K,J),P);R=Math.min(Math.max(T,R),W);break;case\"north\":M=this.constrain(R,M,T,W);H+=M;R-=M;break;case\"west\":O=this.constrain(J,O,K,P);I+=O;J-=O;break;case\"northeast\":J+=O;J=Math.min(Math.max(K,J),P);M=this.constrain(R,M,T,W);H+=M;R-=M;break;case\"northwest\":O=this.constrain(J,O,K,P);M=this.constrain(R,M,T,W);H+=M;R-=M;I+=O;J-=O;break;case\"southwest\":O=this.constrain(J,O,K,P);R+=M;R=Math.min(Math.max(T,R),W);I+=O;J-=O;break}var N=this.snap(J,F,K);var V=this.snap(R,A,T);if(N!=J||V!=R){switch(G){case\"northeast\":H-=V-R;break;case\"north\":H-=V-R;break;case\"southwest\":I-=N-J;break;case\"west\":I-=N-J;break;case\"northwest\":I-=N-J;H-=V-R;break}J=N;R=V}if(this.preserveRatio){switch(G){case\"southeast\":case\"east\":R=L*(J/D);R=Math.min(Math.max(T,R),W);J=D*(R/L);break;case\"south\":J=D*(R/L);J=Math.min(Math.max(K,J),P);R=L*(J/D);break;case\"northeast\":J=D*(R/L);J=Math.min(Math.max(K,J),P);R=L*(J/D);break;case\"north\":var X=J;J=D*(R/L);J=Math.min(Math.max(K,J),P);R=L*(J/D);I+=(X-J)/2;break;case\"southwest\":R=L*(J/D);R=Math.min(Math.max(T,R),W);var X=J;J=D*(R/L);I+=X-J;break;case\"west\":var E=R;R=L*(J/D);R=Math.min(Math.max(T,R),W);H+=(E-R)/2;var X=J;J=D*(R/L);I+=X-J;break;case\"northwest\":var X=J;var E=R;R=L*(J/D);R=Math.min(Math.max(T,R),W);J=D*(R/L);H+=E-R;I+=X-J;break}}this.proxy.setBounds(I,H,J,R);if(this.dynamic){this.resizeElement()}}catch(S){}}},handleOver:function(){if(this.enabled){this.el.addClass(\"x-resizable-over\")}},handleOut:function(){if(!this.resizing){this.el.removeClass(\"x-resizable-over\")}},getEl:function(){return this.el},getResizeChild:function(){return this.resizeChild},destroy:function(C){this.proxy.remove();if(this.overlay){this.overlay.removeAllListeners();this.overlay.remove()}var D=Ext.Resizable.positions;for(var A in D){if(typeof D[A]!=\"function\"&&this[D[A]]){var B=this[D[A]];B.el.removeAllListeners();B.el.remove()}}if(C){this.el.update(\"\");this.el.remove()}},syncHandleHeight:function(){var A=this.el.getHeight(true);if(this.west){this.west.el.setHeight(A)}if(this.east){this.east.el.setHeight(A)}}});Ext.Resizable.positions={n:\"north\",s:\"south\",e:\"east\",w:\"west\",se:\"southeast\",sw:\"southwest\",nw:\"northwest\",ne:\"northeast\"};Ext.Resizable.Handle=function(C,E,B,D){if(!this.tpl){var A=Ext.DomHelper.createTemplate({tag:\"div\",cls:\"x-resizable-handle x-resizable-handle-{0}\"});A.compile();Ext.Resizable.Handle.prototype.tpl=A}this.position=E;this.rz=C;this.el=this.tpl.append(C.el.dom,[this.position],true);this.el.unselectable();if(D){this.el.setOpacity(0)}this.el.on(\"mousedown\",this.onMouseDown,this);if(!B){this.el.on(\"mouseover\",this.onMouseOver,this);this.el.on(\"mouseout\",this.onMouseOut,this)}};Ext.Resizable.Handle.prototype={afterResize:function(A){},onMouseDown:function(A){this.rz.onMouseDown(this,A)},onMouseOver:function(A){this.rz.handleOver(this,A)},onMouseOut:function(A){this.rz.handleOut(this,A)}};\nExt.Editor=function(B,A){this.field=B;Ext.Editor.superclass.constructor.call(this,A)};Ext.extend(Ext.Editor,Ext.Component,{value:\"\",alignment:\"c-c?\",shadow:\"frame\",constrain:false,swallowKeys:true,completeOnEnter:false,cancelOnEsc:false,updateEl:false,initComponent:function(){Ext.Editor.superclass.initComponent.call(this);this.addEvents(\"beforestartedit\",\"startedit\",\"beforecomplete\",\"complete\",\"specialkey\")},onRender:function(B,A){this.el=new Ext.Layer({shadow:this.shadow,cls:\"x-editor\",parentEl:B,shim:this.shim,shadowOffset:4,id:this.id,constrain:this.constrain});this.el.setStyle(\"overflow\",Ext.isGecko?\"auto\":\"hidden\");if(this.field.msgTarget!=\"title\"){this.field.msgTarget=\"qtip\"}this.field.inEditor=true;this.field.render(this.el);if(Ext.isGecko){this.field.el.dom.setAttribute(\"autocomplete\",\"off\")}this.field.on(\"specialkey\",this.onSpecialKey,this);if(this.swallowKeys){this.field.el.swallowEvent([\"keydown\",\"keypress\"])}this.field.show();this.field.on(\"blur\",this.onBlur,this);if(this.field.grow){this.field.on(\"autosize\",this.el.sync,this.el,{delay:1})}},onSpecialKey:function(B,A){if(this.completeOnEnter&&A.getKey()==A.ENTER){A.stopEvent();this.completeEdit()}else{if(this.cancelOnEsc&&A.getKey()==A.ESC){this.cancelEdit()}else{this.fireEvent(\"specialkey\",B,A)}}},startEdit:function(B,C){if(this.editing){this.completeEdit()}this.boundEl=Ext.get(B);var A=C!==undefined?C:this.boundEl.dom.innerHTML;if(!this.rendered){this.render(this.parentEl||document.body)}if(this.fireEvent(\"beforestartedit\",this,this.boundEl,A)===false){return }this.startValue=A;this.field.setValue(A);this.doAutoSize();this.el.alignTo(this.boundEl,this.alignment);this.editing=true;this.show()},doAutoSize:function(){if(this.autoSize){var A=this.boundEl.getSize();switch(this.autoSize){case\"width\":this.setSize(A.width,\"\");break;case\"height\":this.setSize(\"\",A.height);break;default:this.setSize(A.width,A.height)}}},setSize:function(A,B){delete this.field.lastSize;this.field.setSize(A,B);if(this.el){this.el.sync()}},realign:function(){this.el.alignTo(this.boundEl,this.alignment)},completeEdit:function(A){if(!this.editing){return }var B=this.getValue();if(this.revertInvalid!==false&&!this.field.isValid()){B=this.startValue;this.cancelEdit(true)}if(String(B)===String(this.startValue)&&this.ignoreNoChange){this.editing=false;this.hide();return }if(this.fireEvent(\"beforecomplete\",this,B,this.startValue)!==false){this.editing=false;if(this.updateEl&&this.boundEl){this.boundEl.update(B)}if(A!==true){this.hide()}this.fireEvent(\"complete\",this,B,this.startValue)}},onShow:function(){this.el.show();if(this.hideEl!==false){this.boundEl.hide()}this.field.show();if(Ext.isIE&&!this.fixIEFocus){this.fixIEFocus=true;this.deferredFocus.defer(50,this)}else{this.field.focus()}this.fireEvent(\"startedit\",this.boundEl,this.startValue)},deferredFocus:function(){if(this.editing){this.field.focus()}},cancelEdit:function(A){if(this.editing){this.setValue(this.startValue);if(A!==true){this.hide()}}},onBlur:function(){if(this.allowBlur!==true&&this.editing){this.completeEdit()}},onHide:function(){if(this.editing){this.completeEdit();return }this.field.blur();if(this.field.collapse){this.field.collapse()}this.el.hide();if(this.hideEl!==false){this.boundEl.show()}},setValue:function(A){this.field.setValue(A)},getValue:function(){return this.field.getValue()},beforeDestroy:function(){this.field.destroy();this.field=null}});Ext.reg(\"editor\",Ext.Editor);\nExt.MessageBox=function(){var R,B,N,Q;var G,J,P,A,K,M,H,F;var O,S,L,C=\"\";var D=function(U){R.hide();Ext.callback(B.fn,B.scope||window,[U,S.dom.value],1)};var T=function(){if(B&&B.cls){R.el.removeClass(B.cls)}K.reset()};var E=function(W,U,V){if(B&&B.closable!==false){R.hide()}if(V){V.stopEvent()}};var I=function(U){var W=0;if(!U){O[\"ok\"].hide();O[\"cancel\"].hide();O[\"yes\"].hide();O[\"no\"].hide();return W}R.footer.dom.style.display=\"\";for(var V in O){if(typeof O[V]!=\"function\"){if(U[V]){O[V].show();O[V].setText(typeof U[V]==\"string\"?U[V]:Ext.MessageBox.buttonText[V]);W+=O[V].el.getWidth()+15}else{O[V].hide()}}}return W};return{getDialog:function(U){if(!R){R=new Ext.Window({autoCreate:true,title:U,resizable:false,constrain:true,constrainHeader:true,minimizable:false,maximizable:false,stateful:false,modal:true,shim:true,buttonAlign:\"center\",width:400,height:100,minHeight:80,plain:true,footer:true,closable:true,close:function(){if(B&&B.buttons&&B.buttons.no&&!B.buttons.cancel){D(\"no\")}else{D(\"cancel\")}}});O={};var V=this.buttonText;O[\"ok\"]=R.addButton(V[\"ok\"],D.createCallback(\"ok\"));O[\"yes\"]=R.addButton(V[\"yes\"],D.createCallback(\"yes\"));O[\"no\"]=R.addButton(V[\"no\"],D.createCallback(\"no\"));O[\"cancel\"]=R.addButton(V[\"cancel\"],D.createCallback(\"cancel\"));O[\"ok\"].hideMode=O[\"yes\"].hideMode=O[\"no\"].hideMode=O[\"cancel\"].hideMode=\"offsets\";R.render(document.body);R.getEl().addClass(\"x-window-dlg\");N=R.mask;G=R.body.createChild({html:\"<div class=\\\"ext-mb-icon\\\"></div><div class=\\\"ext-mb-content\\\"><span class=\\\"ext-mb-text\\\"></span><br /><input type=\\\"text\\\" class=\\\"ext-mb-input\\\" /><textarea class=\\\"ext-mb-textarea\\\"></textarea></div>\"});H=Ext.get(G.dom.firstChild);var W=G.dom.childNodes[1];J=Ext.get(W.firstChild);P=Ext.get(W.childNodes[2]);P.enableDisplayMode();P.addKeyListener([10,13],function(){if(R.isVisible()&&B&&B.buttons){if(B.buttons.ok){D(\"ok\")}else{if(B.buttons.yes){D(\"yes\")}}}});A=Ext.get(W.childNodes[3]);A.enableDisplayMode();K=new Ext.ProgressBar({renderTo:G});G.createChild({cls:\"x-clear\"})}return R},updateText:function(X){if(!R.isVisible()&&!B.width){R.setSize(this.maxWidth,100)}J.update(X||\"&#160;\");var V=C!=\"\"?(H.getWidth()+H.getMargins(\"lr\")):0;var Z=J.getWidth()+J.getMargins(\"lr\");var W=R.getFrameWidth(\"lr\");var Y=R.body.getFrameWidth(\"lr\");if(Ext.isIE&&V>0){V+=3}var U=Math.max(Math.min(B.width||V+Z+W+Y,this.maxWidth),Math.max(B.minWidth||this.minWidth,L||0));if(B.prompt===true){S.setWidth(U-V-W-Y)}if(B.progress===true||B.wait===true){K.setSize(U-V-W-Y)}R.setSize(U,\"auto\").center();return this},updateProgress:function(V,U,W){K.updateProgress(V,U);if(W){this.updateText(W)}return this},isVisible:function(){return R&&R.isVisible()},hide:function(){if(this.isVisible()){R.hide();T()}return this},show:function(X){if(this.isVisible()){this.hide()}B=X;var Y=this.getDialog(B.title||\"&#160;\");Y.setTitle(B.title||\"&#160;\");var U=(B.closable!==false&&B.progress!==true&&B.wait!==true);Y.tools.close.setDisplayed(U);S=P;B.prompt=B.prompt||(B.multiline?true:false);if(B.prompt){if(B.multiline){P.hide();A.show();A.setHeight(typeof B.multiline==\"number\"?B.multiline:this.defaultTextHeight);S=A}else{P.show();A.hide()}}else{P.hide();A.hide()}S.dom.value=B.value||\"\";if(B.prompt){Y.focusEl=S}else{var W=B.buttons;var V=null;if(W&&W.ok){V=O[\"ok\"]}else{if(W&&W.yes){V=O[\"yes\"]}}if(V){Y.focusEl=V}}this.setIcon(B.icon);L=I(B.buttons);K.setVisible(B.progress===true||B.wait===true);this.updateProgress(0,B.progressText);this.updateText(B.msg);if(B.cls){Y.el.addClass(B.cls)}Y.proxyDrag=B.proxyDrag===true;Y.modal=B.modal!==false;Y.mask=B.modal!==false?N:false;if(!Y.isVisible()){document.body.appendChild(R.el.dom);Y.setAnimateTarget(B.animEl);Y.show(B.animEl)}Y.on(\"show\",function(){if(U===true){Y.keyMap.enable()}else{Y.keyMap.disable()}},this,{single:true});if(B.wait===true){K.wait(B.waitConfig)}return this},setIcon:function(U){if(U&&U!=\"\"){H.removeClass(\"x-hidden\");H.replaceClass(C,U);C=U}else{H.replaceClass(C,\"x-hidden\");C=\"\"}return this},progress:function(W,V,U){this.show({title:W,msg:V,buttons:false,progress:true,closable:false,minWidth:this.minProgressWidth,progressText:U});return this},wait:function(W,V,U){this.show({title:V,msg:W,buttons:false,closable:false,wait:true,modal:true,minWidth:this.minProgressWidth,waitConfig:U});return this},alert:function(X,W,V,U){this.show({title:X,msg:W,buttons:this.OK,fn:V,scope:U});return this},confirm:function(X,W,V,U){this.show({title:X,msg:W,buttons:this.YESNO,fn:V,scope:U,icon:this.QUESTION});return this},prompt:function(Y,X,W,V,U){this.show({title:Y,msg:X,buttons:this.OKCANCEL,fn:W,minWidth:250,scope:V,prompt:true,multiline:U});return this},OK:{ok:true},CANCEL:{cancel:true},OKCANCEL:{ok:true,cancel:true},YESNO:{yes:true,no:true},YESNOCANCEL:{yes:true,no:true,cancel:true},INFO:\"ext-mb-info\",WARNING:\"ext-mb-warning\",QUESTION:\"ext-mb-question\",ERROR:\"ext-mb-error\",defaultTextHeight:75,maxWidth:600,minWidth:100,minProgressWidth:250,buttonText:{ok:\"OK\",cancel:\"Cancel\",yes:\"Yes\",no:\"No\"}}}();Ext.Msg=Ext.MessageBox;\nExt.Tip=Ext.extend(Ext.Panel,{minWidth:40,maxWidth:300,shadow:\"sides\",defaultAlign:\"tl-bl?\",autoRender:true,quickShowInterval:250,frame:true,hidden:true,baseCls:\"x-tip\",floating:{shadow:true,shim:true,useDisplay:true,constrain:false},autoHeight:true,initComponent:function(){Ext.Tip.superclass.initComponent.call(this);if(this.closable&&!this.title){this.elements+=\",header\"}},afterRender:function(){Ext.Tip.superclass.afterRender.call(this);if(this.closable){this.addTool({id:\"close\",handler:this.hide,scope:this})}},showAt:function(A){Ext.Tip.superclass.show.call(this);if(this.measureWidth!==false&&(!this.initialConfig||typeof this.initialConfig.width!=\"number\")){var B=this.body.getTextWidth();if(this.title){B=Math.max(B,this.header.child(\"span\").getTextWidth(this.title))}B+=this.getFrameWidth()+(this.closable?20:0)+this.body.getPadding(\"lr\");this.setWidth(B.constrain(this.minWidth,this.maxWidth))}if(this.constrainPosition){A=this.el.adjustForConstraints(A)}this.setPagePosition(A[0],A[1])},showBy:function(A,B){if(!this.rendered){this.render(Ext.getBody())}this.showAt(this.el.getAlignToXY(A,B||this.defaultAlign))},initDraggable:function(){this.dd=new Ext.Tip.DD(this,typeof this.draggable==\"boolean\"?null:this.draggable);this.header.addClass(\"x-tip-draggable\")}});Ext.Tip.DD=function(B,A){Ext.apply(this,A);this.tip=B;Ext.Tip.DD.superclass.constructor.call(this,B.el.id,\"WindowDD-\"+B.id);this.setHandleElId(B.header.id);this.scroll=false};Ext.extend(Ext.Tip.DD,Ext.dd.DD,{moveOnly:true,scroll:false,headerOffsets:[100,25],startDrag:function(){this.tip.el.disableShadow()},endDrag:function(A){this.tip.el.enableShadow(true)}});\nExt.ToolTip=Ext.extend(Ext.Tip,{showDelay:500,hideDelay:200,dismissDelay:5000,mouseOffset:[15,18],trackMouse:false,constrainPosition:true,initComponent:function(){Ext.ToolTip.superclass.initComponent.call(this);this.lastActive=new Date();this.initTarget()},initTarget:function(){if(this.target){this.target=Ext.get(this.target);this.target.on(\"mouseover\",this.onTargetOver,this);this.target.on(\"mouseout\",this.onTargetOut,this);this.target.on(\"mousemove\",this.onMouseMove,this)}},onMouseMove:function(A){this.targetXY=A.getXY();if(!this.hidden&&this.trackMouse){this.setPagePosition(this.getTargetXY())}},getTargetXY:function(){return[this.targetXY[0]+this.mouseOffset[0],this.targetXY[1]+this.mouseOffset[1]]},onTargetOver:function(A){if(this.disabled||A.within(this.target.dom,true)){return }this.clearTimer(\"hide\");this.targetXY=A.getXY();this.delayShow()},delayShow:function(){if(this.hidden&&!this.showTimer){if(this.lastActive.getElapsed()<this.quickShowInterval){this.show()}else{this.showTimer=this.show.defer(this.showDelay,this)}}else{if(!this.hidden&&this.autoHide!==false){this.show()}}},onTargetOut:function(A){if(this.disabled||A.within(this.target.dom,true)){return }this.clearTimer(\"show\");if(this.autoHide!==false){this.delayHide()}},delayHide:function(){if(!this.hidden&&!this.hideTimer){this.hideTimer=this.hide.defer(this.hideDelay,this)}},hide:function(){this.clearTimer(\"dismiss\");this.lastActive=new Date();Ext.ToolTip.superclass.hide.call(this)},show:function(){this.showAt(this.getTargetXY())},showAt:function(A){this.lastActive=new Date();this.clearTimers();Ext.ToolTip.superclass.showAt.call(this,A);if(this.dismissDelay&&this.autoHide!==false){this.dismissTimer=this.hide.defer(this.dismissDelay,this)}},clearTimer:function(A){A=A+\"Timer\";clearTimeout(this[A]);delete this[A]},clearTimers:function(){this.clearTimer(\"show\");this.clearTimer(\"dismiss\");this.clearTimer(\"hide\")},onShow:function(){Ext.ToolTip.superclass.onShow.call(this);Ext.getDoc().on(\"mousedown\",this.onDocMouseDown,this)},onHide:function(){Ext.ToolTip.superclass.onHide.call(this);Ext.getDoc().un(\"mousedown\",this.onDocMouseDown,this)},onDocMouseDown:function(A){if(this.autoHide!==false&&!A.within(this.el.dom)){this.disable();this.enable.defer(100,this)}},onDisable:function(){this.clearTimers();this.hide()},adjustPosition:function(A,D){var C=this.targetXY[1],B=this.getSize().height;if(this.constrainPosition&&D<=C&&(D+B)>=C){D=C-B-5}return{x:A,y:D}},onDestroy:function(){Ext.ToolTip.superclass.onDestroy.call(this);if(this.target){this.target.un(\"mouseover\",this.onTargetOver,this);this.target.un(\"mouseout\",this.onTargetOut,this);this.target.un(\"mousemove\",this.onMouseMove,this)}}});\nExt.QuickTip=Ext.extend(Ext.ToolTip,{interceptTitles:false,tagConfig:{namespace:\"ext\",attribute:\"qtip\",width:\"qwidth\",target:\"target\",title:\"qtitle\",hide:\"hide\",cls:\"qclass\",align:\"qalign\"},initComponent:function(){this.target=this.target||Ext.getDoc();this.targets=this.targets||{};Ext.QuickTip.superclass.initComponent.call(this)},register:function(D){var F=Ext.isArray(D)?D:arguments;for(var E=0,A=F.length;E<A;E++){var H=F[E];var G=H.target;if(G){if(Ext.isArray(G)){for(var C=0,B=G.length;C<B;C++){this.targets[Ext.id(G[C])]=H}}else{this.targets[Ext.id(G)]=H}}}},unregister:function(A){delete this.targets[Ext.id(A)]},onTargetOver:function(G){if(this.disabled){return }this.targetXY=G.getXY();var C=G.getTarget();if(!C||C.nodeType!==1||C==document||C==document.body){return }if(this.activeTarget&&C==this.activeTarget.el){this.clearTimer(\"hide\");this.show();return }if(C&&this.targets[C.id]){this.activeTarget=this.targets[C.id];this.activeTarget.el=C;this.delayShow();return }var E,F=Ext.fly(C),B=this.tagConfig;var D=B.namespace;if(this.interceptTitles&&C.title){E=C.title;C.qtip=E;C.removeAttribute(\"title\");G.preventDefault()}else{E=C.qtip||F.getAttributeNS(D,B.attribute)}if(E){var A=F.getAttributeNS(D,B.hide);this.activeTarget={el:C,text:E,width:F.getAttributeNS(D,B.width),autoHide:A!=\"user\"&&A!==\"false\",title:F.getAttributeNS(D,B.title),cls:F.getAttributeNS(D,B.cls),align:F.getAttributeNS(D,B.align)};this.delayShow()}},onTargetOut:function(A){this.clearTimer(\"show\");if(this.autoHide!==false){this.delayHide()}},showAt:function(B){var A=this.activeTarget;if(A){if(!this.rendered){this.render(Ext.getBody());this.activeTarget=A}if(A.width){this.setWidth(A.width);this.body.setWidth(this.adjustBodyWidth(A.width-this.getFrameWidth()));this.measureWidth=false}else{this.measureWidth=true}this.setTitle(A.title||\"\");this.body.update(A.text);this.autoHide=A.autoHide;this.dismissDelay=A.dismissDelay||this.dismissDelay;if(this.lastCls){this.el.removeClass(this.lastCls);delete this.lastCls}if(A.cls){this.el.addClass(A.cls);this.lastCls=A.cls}if(A.align){B=this.el.getAlignToXY(A.el,A.align);this.constrainPosition=false}else{this.constrainPosition=true}}Ext.QuickTip.superclass.showAt.call(this,B)},hide:function(){delete this.activeTarget;Ext.QuickTip.superclass.hide.call(this)}});\nExt.QuickTips=function(){var B,A=[];return{init:function(){if(!B){B=new Ext.QuickTip({elements:\"header,body\"})}},enable:function(){if(B){A.pop();if(A.length<1){B.enable()}}},disable:function(){if(B){B.disable()}A.push(1)},isEnabled:function(){return B&&!B.disabled},getQuickTip:function(){return B},register:function(){B.register.apply(B,arguments)},unregister:function(){B.unregister.apply(B,arguments)},tips:function(){B.register.apply(B,arguments)}}}();\nExt.tree.TreePanel=Ext.extend(Ext.Panel,{rootVisible:true,animate:Ext.enableFx,lines:true,enableDD:false,hlDrop:Ext.enableFx,pathSeparator:\"/\",initComponent:function(){Ext.tree.TreePanel.superclass.initComponent.call(this);if(!this.eventModel){this.eventModel=new Ext.tree.TreeEventModel(this)}this.nodeHash={};if(this.root){this.setRootNode(this.root)}this.addEvents(\"append\",\"remove\",\"movenode\",\"insert\",\"beforeappend\",\"beforeremove\",\"beforemovenode\",\"beforeinsert\",\"beforeload\",\"load\",\"textchange\",\"beforeexpandnode\",\"beforecollapsenode\",\"expandnode\",\"disabledchange\",\"collapsenode\",\"beforeclick\",\"click\",\"checkchange\",\"dblclick\",\"contextmenu\",\"beforechildrenrendered\",\"startdrag\",\"enddrag\",\"dragdrop\",\"beforenodedrop\",\"nodedrop\",\"nodedragover\");if(this.singleExpand){this.on(\"beforeexpandnode\",this.restrictExpand,this)}},proxyNodeEvent:function(C,B,A,G,F,E,D){if(C==\"collapse\"||C==\"expand\"||C==\"beforecollapse\"||C==\"beforeexpand\"||C==\"move\"||C==\"beforemove\"){C=C+\"node\"}return this.fireEvent(C,B,A,G,F,E,D)},getRootNode:function(){return this.root},setRootNode:function(B){this.root=B;B.ownerTree=this;B.isRoot=true;this.registerNode(B);if(!this.rootVisible){var A=B.attributes.uiProvider;B.ui=A?new A(B):new Ext.tree.RootTreeNodeUI(B)}return B},getNodeById:function(A){return this.nodeHash[A]},registerNode:function(A){this.nodeHash[A.id]=A},unregisterNode:function(A){delete this.nodeHash[A.id]},toString:function(){return\"[Tree\"+(this.id?\" \"+this.id:\"\")+\"]\"},restrictExpand:function(A){var B=A.parentNode;if(B){if(B.expandedChild&&B.expandedChild.parentNode==B){B.expandedChild.collapse()}B.expandedChild=A}},getChecked:function(A,B){B=B||this.root;var C=[];var D=function(){if(this.attributes.checked){C.push(!A?this:(A==\"id\"?this.id:this.attributes[A]))}};B.cascade(D);return C},getEl:function(){return this.el},getLoader:function(){return this.loader},expandAll:function(){this.root.expand(true)},collapseAll:function(){this.root.collapse(true)},getSelectionModel:function(){if(!this.selModel){this.selModel=new Ext.tree.DefaultSelectionModel()}return this.selModel},expandPath:function(F,A,G){A=A||\"id\";var D=F.split(this.pathSeparator);var C=this.root;if(C.attributes[A]!=D[1]){if(G){G(false,null)}return }var B=1;var E=function(){if(++B==D.length){if(G){G(true,C)}return }var H=C.findChild(A,D[B]);if(!H){if(G){G(false,C)}return }C=H;H.expand(false,false,E)};C.expand(false,false,E)},selectPath:function(E,A,F){A=A||\"id\";var C=E.split(this.pathSeparator);var B=C.pop();if(C.length>0){var D=function(H,G){if(H&&G){var I=G.findChild(A,B);if(I){I.select();if(F){F(true,I)}}else{if(F){F(false,I)}}}else{if(F){F(false,I)}}};this.expandPath(C.join(this.pathSeparator),A,D)}else{this.root.select();if(F){F(true,this.root)}}},getTreeEl:function(){return this.body},onRender:function(B,A){Ext.tree.TreePanel.superclass.onRender.call(this,B,A);this.el.addClass(\"x-tree\");this.innerCt=this.body.createChild({tag:\"ul\",cls:\"x-tree-root-ct \"+(this.useArrows?\"x-tree-arrows\":this.lines?\"x-tree-lines\":\"x-tree-no-lines\")})},initEvents:function(){Ext.tree.TreePanel.superclass.initEvents.call(this);if(this.containerScroll){Ext.dd.ScrollManager.register(this.body)}if((this.enableDD||this.enableDrop)&&!this.dropZone){this.dropZone=new Ext.tree.TreeDropZone(this,this.dropConfig||{ddGroup:this.ddGroup||\"TreeDD\",appendOnly:this.ddAppendOnly===true})}if((this.enableDD||this.enableDrag)&&!this.dragZone){this.dragZone=new Ext.tree.TreeDragZone(this,this.dragConfig||{ddGroup:this.ddGroup||\"TreeDD\",scroll:this.ddScroll})}this.getSelectionModel().init(this)},afterRender:function(){Ext.tree.TreePanel.superclass.afterRender.call(this);this.root.render();if(!this.rootVisible){this.root.renderChildren()}},onDestroy:function(){if(this.rendered){this.body.removeAllListeners();Ext.dd.ScrollManager.unregister(this.body);if(this.dropZone){this.dropZone.unreg()}if(this.dragZone){this.dragZone.unreg()}}this.root.destroy();this.nodeHash=null;Ext.tree.TreePanel.superclass.onDestroy.call(this)}});Ext.reg(\"treepanel\",Ext.tree.TreePanel);\nExt.tree.TreeEventModel=function(A){this.tree=A;this.tree.on(\"render\",this.initEvents,this)};Ext.tree.TreeEventModel.prototype={initEvents:function(){var A=this.tree.getTreeEl();A.on(\"click\",this.delegateClick,this);if(this.tree.trackMouseOver!==false){A.on(\"mouseover\",this.delegateOver,this);A.on(\"mouseout\",this.delegateOut,this)}A.on(\"dblclick\",this.delegateDblClick,this);A.on(\"contextmenu\",this.delegateContextMenu,this)},getNode:function(B){var A;if(A=B.getTarget(\".x-tree-node-el\",10)){var C=Ext.fly(A,\"_treeEvents\").getAttributeNS(\"ext\",\"tree-node-id\");if(C){return this.tree.getNodeById(C)}}return null},getNodeTarget:function(B){var A=B.getTarget(\".x-tree-node-icon\",1);if(!A){A=B.getTarget(\".x-tree-node-el\",6)}return A},delegateOut:function(B,A){if(!this.beforeEvent(B)){return }if(B.getTarget(\".x-tree-ec-icon\",1)){var C=this.getNode(B);this.onIconOut(B,C);if(C==this.lastEcOver){delete this.lastEcOver}}if((A=this.getNodeTarget(B))&&!B.within(A,true)){this.onNodeOut(B,this.getNode(B))}},delegateOver:function(B,A){if(!this.beforeEvent(B)){return }if(this.lastEcOver){this.onIconOut(B,this.lastEcOver);delete this.lastEcOver}if(B.getTarget(\".x-tree-ec-icon\",1)){this.lastEcOver=this.getNode(B);this.onIconOver(B,this.lastEcOver)}if(A=this.getNodeTarget(B)){this.onNodeOver(B,this.getNode(B))}},delegateClick:function(B,A){if(!this.beforeEvent(B)){return }if(B.getTarget(\"input[type=checkbox]\",1)){this.onCheckboxClick(B,this.getNode(B))}else{if(B.getTarget(\".x-tree-ec-icon\",1)){this.onIconClick(B,this.getNode(B))}else{if(this.getNodeTarget(B)){this.onNodeClick(B,this.getNode(B))}}}},delegateDblClick:function(B,A){if(this.beforeEvent(B)&&this.getNodeTarget(B)){this.onNodeDblClick(B,this.getNode(B))}},delegateContextMenu:function(B,A){if(this.beforeEvent(B)&&this.getNodeTarget(B)){this.onNodeContextMenu(B,this.getNode(B))}},onNodeClick:function(B,A){A.ui.onClick(B)},onNodeOver:function(B,A){A.ui.onOver(B)},onNodeOut:function(B,A){A.ui.onOut(B)},onIconOver:function(B,A){A.ui.addClass(\"x-tree-ec-over\")},onIconOut:function(B,A){A.ui.removeClass(\"x-tree-ec-over\")},onIconClick:function(B,A){A.ui.ecClick(B)},onCheckboxClick:function(B,A){A.ui.onCheckChange(B)},onNodeDblClick:function(B,A){A.ui.onDblClick(B)},onNodeContextMenu:function(B,A){A.ui.onContextMenu(B)},beforeEvent:function(A){if(this.disabled){A.stopEvent();return false}return true},disable:function(){this.disabled=true},enable:function(){this.disabled=false}};\nExt.tree.DefaultSelectionModel=function(A){this.selNode=null;this.addEvents(\"selectionchange\",\"beforeselect\");Ext.apply(this,A);Ext.tree.DefaultSelectionModel.superclass.constructor.call(this)};Ext.extend(Ext.tree.DefaultSelectionModel,Ext.util.Observable,{init:function(A){this.tree=A;A.getTreeEl().on(\"keydown\",this.onKeyDown,this);A.on(\"click\",this.onNodeClick,this)},onNodeClick:function(A,B){this.select(A)},select:function(B){var A=this.selNode;if(A!=B&&this.fireEvent(\"beforeselect\",this,B,A)!==false){if(A){A.ui.onSelectedChange(false)}this.selNode=B;B.ui.onSelectedChange(true);this.fireEvent(\"selectionchange\",this,B,A)}return B},unselect:function(A){if(this.selNode==A){this.clearSelections()}},clearSelections:function(){var A=this.selNode;if(A){A.ui.onSelectedChange(false);this.selNode=null;this.fireEvent(\"selectionchange\",this,null)}return A},getSelectedNode:function(){return this.selNode},isSelected:function(A){return this.selNode==A},selectPrevious:function(){var A=this.selNode||this.lastSelNode;if(!A){return null}var C=A.previousSibling;if(C){if(!C.isExpanded()||C.childNodes.length<1){return this.select(C)}else{var B=C.lastChild;while(B&&B.isExpanded()&&B.childNodes.length>0){B=B.lastChild}return this.select(B)}}else{if(A.parentNode&&(this.tree.rootVisible||!A.parentNode.isRoot)){return this.select(A.parentNode)}}return null},selectNext:function(){var B=this.selNode||this.lastSelNode;if(!B){return null}if(B.firstChild&&B.isExpanded()){return this.select(B.firstChild)}else{if(B.nextSibling){return this.select(B.nextSibling)}else{if(B.parentNode){var A=null;B.parentNode.bubble(function(){if(this.nextSibling){A=this.getOwnerTree().selModel.select(this.nextSibling);return false}});return A}}}return null},onKeyDown:function(C){var B=this.selNode||this.lastSelNode;var D=this;if(!B){return }var A=C.getKey();switch(A){case C.DOWN:C.stopEvent();this.selectNext();break;case C.UP:C.stopEvent();this.selectPrevious();break;case C.RIGHT:C.preventDefault();if(B.hasChildNodes()){if(!B.isExpanded()){B.expand()}else{if(B.firstChild){this.select(B.firstChild,C)}}}break;case C.LEFT:C.preventDefault();if(B.hasChildNodes()&&B.isExpanded()){B.collapse()}else{if(B.parentNode&&(this.tree.rootVisible||B.parentNode!=this.tree.getRootNode())){this.select(B.parentNode,C)}}break}}});Ext.tree.MultiSelectionModel=function(A){this.selNodes=[];this.selMap={};this.addEvents(\"selectionchange\");Ext.apply(this,A);Ext.tree.MultiSelectionModel.superclass.constructor.call(this)};Ext.extend(Ext.tree.MultiSelectionModel,Ext.util.Observable,{init:function(A){this.tree=A;A.getTreeEl().on(\"keydown\",this.onKeyDown,this);A.on(\"click\",this.onNodeClick,this)},onNodeClick:function(A,B){this.select(A,B,B.ctrlKey)},select:function(A,C,B){if(B!==true){this.clearSelections(true)}if(this.isSelected(A)){this.lastSelNode=A;return A}this.selNodes.push(A);this.selMap[A.id]=A;this.lastSelNode=A;A.ui.onSelectedChange(true);this.fireEvent(\"selectionchange\",this,this.selNodes);return A},unselect:function(B){if(this.selMap[B.id]){B.ui.onSelectedChange(false);var C=this.selNodes;var A=C.indexOf(B);if(A!=-1){this.selNodes.splice(A,1)}delete this.selMap[B.id];this.fireEvent(\"selectionchange\",this,this.selNodes)}},clearSelections:function(B){var D=this.selNodes;if(D.length>0){for(var C=0,A=D.length;C<A;C++){D[C].ui.onSelectedChange(false)}this.selNodes=[];this.selMap={};if(B!==true){this.fireEvent(\"selectionchange\",this,this.selNodes)}}},isSelected:function(A){return this.selMap[A.id]?true:false},getSelectedNodes:function(){return this.selNodes},onKeyDown:Ext.tree.DefaultSelectionModel.prototype.onKeyDown,selectNext:Ext.tree.DefaultSelectionModel.prototype.selectNext,selectPrevious:Ext.tree.DefaultSelectionModel.prototype.selectPrevious});\nExt.tree.TreeNode=function(A){A=A||{};if(typeof A==\"string\"){A={text:A}}this.childrenRendered=false;this.rendered=false;Ext.tree.TreeNode.superclass.constructor.call(this,A);this.expanded=A.expanded===true;this.isTarget=A.isTarget!==false;this.draggable=A.draggable!==false&&A.allowDrag!==false;this.allowChildren=A.allowChildren!==false&&A.allowDrop!==false;this.text=A.text;this.disabled=A.disabled===true;this.addEvents(\"textchange\",\"beforeexpand\",\"beforecollapse\",\"expand\",\"disabledchange\",\"collapse\",\"beforeclick\",\"click\",\"checkchange\",\"dblclick\",\"contextmenu\",\"beforechildrenrendered\");var B=this.attributes.uiProvider||this.defaultUI||Ext.tree.TreeNodeUI;this.ui=new B(this)};Ext.extend(Ext.tree.TreeNode,Ext.data.Node,{preventHScroll:true,isExpanded:function(){return this.expanded},getUI:function(){return this.ui},setFirstChild:function(A){var B=this.firstChild;Ext.tree.TreeNode.superclass.setFirstChild.call(this,A);if(this.childrenRendered&&B&&A!=B){B.renderIndent(true,true)}if(this.rendered){this.renderIndent(true,true)}},setLastChild:function(B){var A=this.lastChild;Ext.tree.TreeNode.superclass.setLastChild.call(this,B);if(this.childrenRendered&&A&&B!=A){A.renderIndent(true,true)}if(this.rendered){this.renderIndent(true,true)}},appendChild:function(){var A=Ext.tree.TreeNode.superclass.appendChild.apply(this,arguments);if(A&&this.childrenRendered){A.render()}this.ui.updateExpandIcon();return A},removeChild:function(A){this.ownerTree.getSelectionModel().unselect(A);Ext.tree.TreeNode.superclass.removeChild.apply(this,arguments);if(this.childrenRendered){A.ui.remove()}if(this.childNodes.length<1){this.collapse(false,false)}else{this.ui.updateExpandIcon()}if(!this.firstChild&&!this.isHiddenRoot()){this.childrenRendered=false}return A},insertBefore:function(C,A){var B=Ext.tree.TreeNode.superclass.insertBefore.apply(this,arguments);if(B&&A&&this.childrenRendered){C.render()}this.ui.updateExpandIcon();return B},setText:function(B){var A=this.text;this.text=B;this.attributes.text=B;if(this.rendered){this.ui.onTextChange(this,B,A)}this.fireEvent(\"textchange\",this,B,A)},select:function(){this.getOwnerTree().getSelectionModel().select(this)},unselect:function(){this.getOwnerTree().getSelectionModel().unselect(this)},isSelected:function(){return this.getOwnerTree().getSelectionModel().isSelected(this)},expand:function(A,B,C){if(!this.expanded){if(this.fireEvent(\"beforeexpand\",this,A,B)===false){return }if(!this.childrenRendered){this.renderChildren()}this.expanded=true;if(!this.isHiddenRoot()&&(this.getOwnerTree().animate&&B!==false)||B){this.ui.animExpand(function(){this.fireEvent(\"expand\",this);if(typeof C==\"function\"){C(this)}if(A===true){this.expandChildNodes(true)}}.createDelegate(this));return }else{this.ui.expand();this.fireEvent(\"expand\",this);if(typeof C==\"function\"){C(this)}}}else{if(typeof C==\"function\"){C(this)}}if(A===true){this.expandChildNodes(true)}},isHiddenRoot:function(){return this.isRoot&&!this.getOwnerTree().rootVisible},collapse:function(B,E){if(this.expanded&&!this.isHiddenRoot()){if(this.fireEvent(\"beforecollapse\",this,B,E)===false){return }this.expanded=false;if((this.getOwnerTree().animate&&E!==false)||E){this.ui.animCollapse(function(){this.fireEvent(\"collapse\",this);if(B===true){this.collapseChildNodes(true)}}.createDelegate(this));return }else{this.ui.collapse();this.fireEvent(\"collapse\",this)}}if(B===true){var D=this.childNodes;for(var C=0,A=D.length;C<A;C++){D[C].collapse(true,false)}}},delayedExpand:function(A){if(!this.expandProcId){this.expandProcId=this.expand.defer(A,this)}},cancelExpand:function(){if(this.expandProcId){clearTimeout(this.expandProcId)}this.expandProcId=false},toggle:function(){if(this.expanded){this.collapse()}else{this.expand()}},ensureVisible:function(B){var A=this.getOwnerTree();A.expandPath(this.parentNode.getPath(),false,function(){var C=A.getNodeById(this.id);A.getTreeEl().scrollChildIntoView(C.ui.anchor);Ext.callback(B)}.createDelegate(this))},expandChildNodes:function(B){var D=this.childNodes;for(var C=0,A=D.length;C<A;C++){D[C].expand(B)}},collapseChildNodes:function(B){var D=this.childNodes;for(var C=0,A=D.length;C<A;C++){D[C].collapse(B)}},disable:function(){this.disabled=true;this.unselect();if(this.rendered&&this.ui.onDisableChange){this.ui.onDisableChange(this,true)}this.fireEvent(\"disabledchange\",this,true)},enable:function(){this.disabled=false;if(this.rendered&&this.ui.onDisableChange){this.ui.onDisableChange(this,false)}this.fireEvent(\"disabledchange\",this,false)},renderChildren:function(B){if(B!==false){this.fireEvent(\"beforechildrenrendered\",this)}var D=this.childNodes;for(var C=0,A=D.length;C<A;C++){D[C].render(true)}this.childrenRendered=true},sort:function(E,D){Ext.tree.TreeNode.superclass.sort.apply(this,arguments);if(this.childrenRendered){var C=this.childNodes;for(var B=0,A=C.length;B<A;B++){C[B].render(true)}}},render:function(A){this.ui.render(A);if(!this.rendered){this.getOwnerTree().registerNode(this);this.rendered=true;if(this.expanded){this.expanded=false;this.expand(false,false)}}},renderIndent:function(B,E){if(E){this.ui.childIndent=null}this.ui.renderIndent();if(B===true&&this.childrenRendered){var D=this.childNodes;for(var C=0,A=D.length;C<A;C++){D[C].renderIndent(true,E)}}},beginUpdate:function(){this.childrenRendered=false},endUpdate:function(){if(this.expanded){this.renderChildren()}},destroy:function(){for(var B=0,A=this.childNodes.length;B<A;B++){this.childNodes[B].destroy()}this.childNodes=null;if(this.ui.destroy){this.ui.destroy()}}});\nExt.tree.AsyncTreeNode=function(A){this.loaded=false;this.loading=false;Ext.tree.AsyncTreeNode.superclass.constructor.apply(this,arguments);this.addEvents(\"beforeload\",\"load\")};Ext.extend(Ext.tree.AsyncTreeNode,Ext.tree.TreeNode,{expand:function(B,D,F){if(this.loading){var E;var C=function(){if(!this.loading){clearInterval(E);this.expand(B,D,F)}}.createDelegate(this);E=setInterval(C,200);return }if(!this.loaded){if(this.fireEvent(\"beforeload\",this)===false){return }this.loading=true;this.ui.beforeLoad(this);var A=this.loader||this.attributes.loader||this.getOwnerTree().getLoader();if(A){A.load(this,this.loadComplete.createDelegate(this,[B,D,F]));return }}Ext.tree.AsyncTreeNode.superclass.expand.call(this,B,D,F)},isLoading:function(){return this.loading},loadComplete:function(A,B,C){this.loading=false;this.loaded=true;this.ui.afterLoad(this);this.fireEvent(\"load\",this);this.expand(A,B,C)},isLoaded:function(){return this.loaded},hasChildNodes:function(){if(!this.isLeaf()&&!this.loaded){return true}else{return Ext.tree.AsyncTreeNode.superclass.hasChildNodes.call(this)}},reload:function(A){this.collapse(false,false);while(this.firstChild){this.removeChild(this.firstChild)}this.childrenRendered=false;this.loaded=false;if(this.isHiddenRoot()){this.expanded=false}this.expand(false,false,A)}});\nExt.tree.TreeNodeUI=function(A){this.node=A;this.rendered=false;this.animating=false;this.wasLeaf=true;this.ecc=\"x-tree-ec-icon x-tree-elbow\";this.emptyIcon=Ext.BLANK_IMAGE_URL};Ext.tree.TreeNodeUI.prototype={removeChild:function(A){if(this.rendered){this.ctNode.removeChild(A.ui.getEl())}},beforeLoad:function(){this.addClass(\"x-tree-node-loading\")},afterLoad:function(){this.removeClass(\"x-tree-node-loading\")},onTextChange:function(B,C,A){if(this.rendered){this.textNode.innerHTML=C}},onDisableChange:function(A,B){this.disabled=B;if(this.checkbox){this.checkbox.disabled=B}if(B){this.addClass(\"x-tree-node-disabled\")}else{this.removeClass(\"x-tree-node-disabled\")}},onSelectedChange:function(A){if(A){this.focus();this.addClass(\"x-tree-selected\")}else{this.removeClass(\"x-tree-selected\")}},onMove:function(A,G,E,F,D,B){this.childIndent=null;if(this.rendered){var H=F.ui.getContainer();if(!H){this.holder=document.createElement(\"div\");this.holder.appendChild(this.wrap);return }var C=B?B.ui.getEl():null;if(C){H.insertBefore(this.wrap,C)}else{H.appendChild(this.wrap)}this.node.renderIndent(true)}},addClass:function(A){if(this.elNode){Ext.fly(this.elNode).addClass(A)}},removeClass:function(A){if(this.elNode){Ext.fly(this.elNode).removeClass(A)}},remove:function(){if(this.rendered){this.holder=document.createElement(\"div\");this.holder.appendChild(this.wrap)}},fireEvent:function(){return this.node.fireEvent.apply(this.node,arguments)},initEvents:function(){this.node.on(\"move\",this.onMove,this);if(this.node.disabled){this.addClass(\"x-tree-node-disabled\");if(this.checkbox){this.checkbox.disabled=true}}if(this.node.hidden){this.hide()}var B=this.node.getOwnerTree();var A=B.enableDD||B.enableDrag||B.enableDrop;if(A&&(!this.node.isRoot||B.rootVisible)){Ext.dd.Registry.register(this.elNode,{node:this.node,handles:this.getDDHandles(),isHandle:false})}},getDDHandles:function(){return[this.iconNode,this.textNode,this.elNode]},hide:function(){this.node.hidden=true;if(this.wrap){this.wrap.style.display=\"none\"}},show:function(){this.node.hidden=false;if(this.wrap){this.wrap.style.display=\"\"}},onContextMenu:function(A){if(this.node.hasListener(\"contextmenu\")||this.node.getOwnerTree().hasListener(\"contextmenu\")){A.preventDefault();this.focus();this.fireEvent(\"contextmenu\",this.node,A)}},onClick:function(B){if(this.dropping){B.stopEvent();return }if(this.fireEvent(\"beforeclick\",this.node,B)!==false){var A=B.getTarget(\"a\");if(!this.disabled&&this.node.attributes.href&&A){this.fireEvent(\"click\",this.node,B);return }else{if(A&&B.ctrlKey){B.stopEvent()}}B.preventDefault();if(this.disabled){return }if(this.node.attributes.singleClickExpand&&!this.animating&&this.node.hasChildNodes()){this.node.toggle()}this.fireEvent(\"click\",this.node,B)}else{B.stopEvent()}},onDblClick:function(A){A.preventDefault();if(this.disabled){return }if(this.checkbox){this.toggleCheck()}if(!this.animating&&this.node.hasChildNodes()){this.node.toggle()}this.fireEvent(\"dblclick\",this.node,A)},onOver:function(A){this.addClass(\"x-tree-node-over\")},onOut:function(A){this.removeClass(\"x-tree-node-over\")},onCheckChange:function(){var A=this.checkbox.checked;this.node.attributes.checked=A;this.fireEvent(\"checkchange\",this.node,A)},ecClick:function(A){if(!this.animating&&(this.node.hasChildNodes()||this.node.attributes.expandable)){this.node.toggle()}},startDrop:function(){this.dropping=true},endDrop:function(){setTimeout(function(){this.dropping=false}.createDelegate(this),50)},expand:function(){this.updateExpandIcon();this.ctNode.style.display=\"\"},focus:function(){if(!this.node.preventHScroll){try{this.anchor.focus()}catch(C){}}else{if(!Ext.isIE){try{var B=this.node.getOwnerTree().getTreeEl().dom;var A=B.scrollLeft;this.anchor.focus();B.scrollLeft=A}catch(C){}}}},toggleCheck:function(B){var A=this.checkbox;if(A){A.checked=(B===undefined?!A.checked:B)}},blur:function(){try{this.anchor.blur()}catch(A){}},animExpand:function(B){var A=Ext.get(this.ctNode);A.stopFx();if(!this.node.hasChildNodes()){this.updateExpandIcon();this.ctNode.style.display=\"\";Ext.callback(B);return }this.animating=true;this.updateExpandIcon();A.slideIn(\"t\",{callback:function(){this.animating=false;Ext.callback(B)},scope:this,duration:this.node.ownerTree.duration||0.25})},highlight:function(){var A=this.node.getOwnerTree();Ext.fly(this.wrap).highlight(A.hlColor||\"C3DAF9\",{endColor:A.hlBaseColor})},collapse:function(){this.updateExpandIcon();this.ctNode.style.display=\"none\"},animCollapse:function(B){var A=Ext.get(this.ctNode);A.enableDisplayMode(\"block\");A.stopFx();this.animating=true;this.updateExpandIcon();A.slideOut(\"t\",{callback:function(){this.animating=false;Ext.callback(B)},scope:this,duration:this.node.ownerTree.duration||0.25})},getContainer:function(){return this.ctNode},getEl:function(){return this.wrap},appendDDGhost:function(A){A.appendChild(this.elNode.cloneNode(true))},getDDRepairXY:function(){return Ext.lib.Dom.getXY(this.iconNode)},onRender:function(){this.render()},render:function(B){var D=this.node,A=D.attributes;var C=D.parentNode?D.parentNode.ui.getContainer():D.ownerTree.innerCt.dom;if(!this.rendered){this.rendered=true;this.renderElements(D,A,C,B);if(A.qtip){if(this.textNode.setAttributeNS){this.textNode.setAttributeNS(\"ext\",\"qtip\",A.qtip);if(A.qtipTitle){this.textNode.setAttributeNS(\"ext\",\"qtitle\",A.qtipTitle)}}else{this.textNode.setAttribute(\"ext:qtip\",A.qtip);if(A.qtipTitle){this.textNode.setAttribute(\"ext:qtitle\",A.qtipTitle)}}}else{if(A.qtipCfg){A.qtipCfg.target=Ext.id(this.textNode);Ext.QuickTips.register(A.qtipCfg)}}this.initEvents();if(!this.node.expanded){this.updateExpandIcon(true)}}else{if(B===true){C.appendChild(this.wrap)}}},renderElements:function(D,I,H,J){this.indentMarkup=D.parentNode?D.parentNode.ui.getChildIndent():\"\";var E=typeof I.checked==\"boolean\";var B=I.href?I.href:Ext.isGecko?\"\":\"#\";var C=[\"<li class=\\\"x-tree-node\\\"><div ext:tree-node-id=\\\"\",D.id,\"\\\" class=\\\"x-tree-node-el x-tree-node-leaf x-unselectable \",I.cls,\"\\\" unselectable=\\\"on\\\">\",\"<span class=\\\"x-tree-node-indent\\\">\",this.indentMarkup,\"</span>\",\"<img src=\\\"\",this.emptyIcon,\"\\\" class=\\\"x-tree-ec-icon x-tree-elbow\\\" />\",\"<img src=\\\"\",I.icon||this.emptyIcon,\"\\\" class=\\\"x-tree-node-icon\",(I.icon?\" x-tree-node-inline-icon\":\"\"),(I.iconCls?\" \"+I.iconCls:\"\"),\"\\\" unselectable=\\\"on\\\" />\",E?(\"<input class=\\\"x-tree-node-cb\\\" type=\\\"checkbox\\\" \"+(I.checked?\"checked=\\\"checked\\\" />\":\"/>\")):\"\",\"<a hidefocus=\\\"on\\\" class=\\\"x-tree-node-anchor\\\" href=\\\"\",B,\"\\\" tabIndex=\\\"1\\\" \",I.hrefTarget?\" target=\\\"\"+I.hrefTarget+\"\\\"\":\"\",\"><span unselectable=\\\"on\\\">\",D.text,\"</span></a></div>\",\"<ul class=\\\"x-tree-node-ct\\\" style=\\\"display:none;\\\"></ul>\",\"</li>\"].join(\"\");var A;if(J!==true&&D.nextSibling&&(A=D.nextSibling.ui.getEl())){this.wrap=Ext.DomHelper.insertHtml(\"beforeBegin\",A,C)}else{this.wrap=Ext.DomHelper.insertHtml(\"beforeEnd\",H,C)}this.elNode=this.wrap.childNodes[0];this.ctNode=this.wrap.childNodes[1];var G=this.elNode.childNodes;this.indentNode=G[0];this.ecNode=G[1];this.iconNode=G[2];var F=3;if(E){this.checkbox=G[3];F++}this.anchor=G[F];this.textNode=G[F].firstChild},getAnchor:function(){return this.anchor},getTextEl:function(){return this.textNode},getIconEl:function(){return this.iconNode},isChecked:function(){return this.checkbox?this.checkbox.checked:false},updateExpandIcon:function(){if(this.rendered){var F=this.node,D,C;var A=F.isLast()?\"x-tree-elbow-end\":\"x-tree-elbow\";var E=F.hasChildNodes();if(E||F.attributes.expandable){if(F.expanded){A+=\"-minus\";D=\"x-tree-node-collapsed\";C=\"x-tree-node-expanded\"}else{A+=\"-plus\";D=\"x-tree-node-expanded\";C=\"x-tree-node-collapsed\"}if(this.wasLeaf){this.removeClass(\"x-tree-node-leaf\");this.wasLeaf=false}if(this.c1!=D||this.c2!=C){Ext.fly(this.elNode).replaceClass(D,C);this.c1=D;this.c2=C}}else{if(!this.wasLeaf){Ext.fly(this.elNode).replaceClass(\"x-tree-node-expanded\",\"x-tree-node-leaf\");delete this.c1;delete this.c2;this.wasLeaf=true}}var B=\"x-tree-ec-icon \"+A;if(this.ecc!=B){this.ecNode.className=B;this.ecc=B}}},getChildIndent:function(){if(!this.childIndent){var A=[];var B=this.node;while(B){if(!B.isRoot||(B.isRoot&&B.ownerTree.rootVisible)){if(!B.isLast()){A.unshift(\"<img src=\\\"\"+this.emptyIcon+\"\\\" class=\\\"x-tree-elbow-line\\\" />\")}else{A.unshift(\"<img src=\\\"\"+this.emptyIcon+\"\\\" class=\\\"x-tree-icon\\\" />\")}}B=B.parentNode}this.childIndent=A.join(\"\")}return this.childIndent},renderIndent:function(){if(this.rendered){var A=\"\";var B=this.node.parentNode;if(B){A=B.ui.getChildIndent()}if(this.indentMarkup!=A){this.indentNode.innerHTML=A;this.indentMarkup=A}this.updateExpandIcon()}},destroy:function(){if(this.elNode){Ext.dd.Registry.unregister(this.elNode.id)}delete this.elNode;delete this.ctNode;delete this.indentNode;delete this.ecNode;delete this.iconNode;delete this.checkbox;delete this.anchor;delete this.textNode;Ext.removeNode(this.ctNode)}};Ext.tree.RootTreeNodeUI=Ext.extend(Ext.tree.TreeNodeUI,{render:function(){if(!this.rendered){var A=this.node.ownerTree.innerCt.dom;this.node.expanded=true;A.innerHTML=\"<div class=\\\"x-tree-root-node\\\"></div>\";this.wrap=this.ctNode=A.firstChild}},collapse:Ext.emptyFn,expand:Ext.emptyFn});\nExt.tree.TreeLoader=function(A){this.baseParams={};this.requestMethod=\"POST\";Ext.apply(this,A);this.addEvents(\"beforeload\",\"load\",\"loadexception\");Ext.tree.TreeLoader.superclass.constructor.call(this)};Ext.extend(Ext.tree.TreeLoader,Ext.util.Observable,{uiProviders:{},clearOnLoad:true,load:function(A,B){if(this.clearOnLoad){while(A.firstChild){A.removeChild(A.firstChild)}}if(this.doPreload(A)){if(typeof B==\"function\"){B()}}else{if(this.dataUrl||this.url){this.requestData(A,B)}}},doPreload:function(D){if(D.attributes.children){if(D.childNodes.length<1){var C=D.attributes.children;D.beginUpdate();for(var B=0,A=C.length;B<A;B++){var E=D.appendChild(this.createNode(C[B]));if(this.preloadChildren){this.doPreload(E)}}D.endUpdate()}return true}else{return false}},getParams:function(D){var A=[],C=this.baseParams;for(var B in C){if(typeof C[B]!=\"function\"){A.push(encodeURIComponent(B),\"=\",encodeURIComponent(C[B]),\"&\")}}A.push(\"node=\",encodeURIComponent(D.id));return A.join(\"\")},requestData:function(A,B){if(this.fireEvent(\"beforeload\",this,A,B)!==false){this.transId=Ext.Ajax.request({method:this.requestMethod,url:this.dataUrl||this.url,success:this.handleResponse,failure:this.handleFailure,scope:this,argument:{callback:B,node:A},params:this.getParams(A)})}else{if(typeof B==\"function\"){B()}}},isLoading:function(){return this.transId?true:false},abort:function(){if(this.isLoading()){Ext.Ajax.abort(this.transId)}},createNode:function(attr){if(this.baseAttrs){Ext.applyIf(attr,this.baseAttrs)}if(this.applyLoader!==false){attr.loader=this}if(typeof attr.uiProvider==\"string\"){attr.uiProvider=this.uiProviders[attr.uiProvider]||eval(attr.uiProvider)}return(attr.leaf?new Ext.tree.TreeNode(attr):new Ext.tree.AsyncTreeNode(attr))},processResponse:function(response,node,callback){var json=response.responseText;try{var o=eval(\"(\"+json+\")\");node.beginUpdate();for(var i=0,len=o.length;i<len;i++){var n=this.createNode(o[i]);if(n){node.appendChild(n)}}node.endUpdate();if(typeof callback==\"function\"){callback(this,node)}}catch(e){this.handleFailure(response)}},handleResponse:function(B){this.transId=false;var A=B.argument;this.processResponse(B,A.node,A.callback);this.fireEvent(\"load\",this,A.node,B)},handleFailure:function(B){this.transId=false;var A=B.argument;this.fireEvent(\"loadexception\",this,A.node,B);if(typeof A.callback==\"function\"){A.callback(this,A.node)}}});\nExt.tree.TreeFilter=function(A,B){this.tree=A;this.filtered={};Ext.apply(this,B)};Ext.tree.TreeFilter.prototype={clearBlank:false,reverse:false,autoClear:false,remove:false,filter:function(D,A,B){A=A||\"text\";var C;if(typeof D==\"string\"){var E=D.length;if(E==0&&this.clearBlank){this.clear();return }D=D.toLowerCase();C=function(F){return F.attributes[A].substr(0,E).toLowerCase()==D}}else{if(D.exec){C=function(F){return D.test(F.attributes[A])}}else{throw\"Illegal filter type, must be string or regex\"}}this.filterBy(C,null,B)},filterBy:function(D,C,B){B=B||this.tree.root;if(this.autoClear){this.clear()}var A=this.filtered,H=this.reverse;var E=function(J){if(J==B){return true}if(A[J.id]){return false}var I=D.call(C||J,J);if(!I||H){A[J.id]=J;J.ui.hide();return false}return true};B.cascade(E);if(this.remove){for(var G in A){if(typeof G!=\"function\"){var F=A[G];if(F&&F.parentNode){F.parentNode.removeChild(F)}}}}},clear:function(){var B=this.tree;var A=this.filtered;for(var D in A){if(typeof D!=\"function\"){var C=A[D];if(C){C.ui.show()}}}this.filtered={}}};\nExt.tree.TreeSorter=function(B,C){Ext.apply(this,C);B.on(\"beforechildrenrendered\",this.doSort,this);B.on(\"append\",this.updateSort,this);B.on(\"insert\",this.updateSort,this);B.on(\"textchange\",this.updateSortParent,this);var E=this.dir&&this.dir.toLowerCase()==\"desc\";var F=this.property||\"text\";var G=this.sortType;var A=this.folderSort;var D=this.caseSensitive===true;var H=this.leafAttr||\"leaf\";this.sortFn=function(J,I){if(A){if(J.attributes[H]&&!I.attributes[H]){return 1}if(!J.attributes[H]&&I.attributes[H]){return -1}}var L=G?G(J):(D?J.attributes[F]:J.attributes[F].toUpperCase());var K=G?G(I):(D?I.attributes[F]:I.attributes[F].toUpperCase());if(L<K){return E?+1:-1}else{if(L>K){return E?-1:+1}else{return 0}}}};Ext.tree.TreeSorter.prototype={doSort:function(A){A.sort(this.sortFn)},compareNodes:function(B,A){return(B.text.toUpperCase()>A.text.toUpperCase()?1:-1)},updateSort:function(A,B){if(B.childrenRendered){this.doSort.defer(1,this,[B])}},updateSortParent:function(A){var B=A.parentNode;if(B&&B.childrenRendered){this.doSort.defer(1,this,[B])}}};\nif(Ext.dd.DropZone){Ext.tree.TreeDropZone=function(A,B){this.allowParentInsert=false;this.allowContainerDrop=false;this.appendOnly=false;Ext.tree.TreeDropZone.superclass.constructor.call(this,A.innerCt,B);this.tree=A;this.dragOverData={};this.lastInsertClass=\"x-tree-no-status\"};Ext.extend(Ext.tree.TreeDropZone,Ext.dd.DropZone,{ddGroup:\"TreeDD\",expandDelay:1000,expandNode:function(A){if(A.hasChildNodes()&&!A.isExpanded()){A.expand(false,null,this.triggerCacheRefresh.createDelegate(this))}},queueExpand:function(A){this.expandProcId=this.expandNode.defer(this.expandDelay,this,[A])},cancelExpand:function(){if(this.expandProcId){clearTimeout(this.expandProcId);this.expandProcId=false}},isValidDropPoint:function(A,I,G,D,C){if(!A||!C){return false}var E=A.node;var F=C.node;if(!(E&&E.isTarget&&I)){return false}if(I==\"append\"&&E.allowChildren===false){return false}if((I==\"above\"||I==\"below\")&&(E.parentNode&&E.parentNode.allowChildren===false)){return false}if(F&&(E==F||F.contains(E))){return false}var B=this.dragOverData;B.tree=this.tree;B.target=E;B.data=C;B.point=I;B.source=G;B.rawEvent=D;B.dropNode=F;B.cancel=false;var H=this.tree.fireEvent(\"nodedragover\",B);return B.cancel===false&&H!==false},getDropPoint:function(E,D,I){var J=D.node;if(J.isRoot){return J.allowChildren!==false?\"append\":false}var B=D.ddel;var K=Ext.lib.Dom.getY(B),G=K+B.offsetHeight;var F=Ext.lib.Event.getPageY(E);var H=J.allowChildren===false||J.isLeaf();if(this.appendOnly||J.parentNode.allowChildren===false){return H?false:\"append\"}var C=false;if(!this.allowParentInsert){C=J.hasChildNodes()&&J.isExpanded()}var A=(G-K)/(H?2:3);if(F>=K&&F<(K+A)){return\"above\"}else{if(!C&&(H||F>=G-A&&F<=G)){return\"below\"}else{return\"append\"}}},onNodeEnter:function(D,A,C,B){this.cancelExpand()},onNodeOver:function(B,G,F,E){var I=this.getDropPoint(F,B,G);var C=B.node;if(!this.expandProcId&&I==\"append\"&&C.hasChildNodes()&&!B.node.isExpanded()){this.queueExpand(C)}else{if(I!=\"append\"){this.cancelExpand()}}var D=this.dropNotAllowed;if(this.isValidDropPoint(B,I,G,F,E)){if(I){var A=B.ddel;var H;if(I==\"above\"){D=B.node.isFirst()?\"x-tree-drop-ok-above\":\"x-tree-drop-ok-between\";H=\"x-tree-drag-insert-above\"}else{if(I==\"below\"){D=B.node.isLast()?\"x-tree-drop-ok-below\":\"x-tree-drop-ok-between\";H=\"x-tree-drag-insert-below\"}else{D=\"x-tree-drop-ok-append\";H=\"x-tree-drag-append\"}}if(this.lastInsertClass!=H){Ext.fly(A).replaceClass(this.lastInsertClass,H);this.lastInsertClass=H}}}return D},onNodeOut:function(D,A,C,B){this.cancelExpand();this.removeDropIndicators(D)},onNodeDrop:function(C,I,E,D){var H=this.getDropPoint(E,C,I);var F=C.node;F.ui.startDrop();if(!this.isValidDropPoint(C,H,I,E,D)){F.ui.endDrop();return false}var G=D.node||(I.getTreeNode?I.getTreeNode(D,F,H,E):null);var B={tree:this.tree,target:F,data:D,point:H,source:I,rawEvent:E,dropNode:G,cancel:!G,dropStatus:false};var A=this.tree.fireEvent(\"beforenodedrop\",B);if(A===false||B.cancel===true||!B.dropNode){F.ui.endDrop();return B.dropStatus}F=B.target;if(H==\"append\"&&!F.isExpanded()){F.expand(false,null,function(){this.completeDrop(B)}.createDelegate(this))}else{this.completeDrop(B)}return true},completeDrop:function(G){var D=G.dropNode,E=G.point,C=G.target;if(!Ext.isArray(D)){D=[D]}var F;for(var B=0,A=D.length;B<A;B++){F=D[B];if(E==\"above\"){C.parentNode.insertBefore(F,C)}else{if(E==\"below\"){C.parentNode.insertBefore(F,C.nextSibling)}else{C.appendChild(F)}}}F.ui.focus();if(this.tree.hlDrop){F.ui.highlight()}C.ui.endDrop();this.tree.fireEvent(\"nodedrop\",G)},afterNodeMoved:function(A,C,E,D,B){if(this.tree.hlDrop){B.ui.focus();B.ui.highlight()}this.tree.fireEvent(\"nodedrop\",this.tree,D,C,A,E)},getTree:function(){return this.tree},removeDropIndicators:function(B){if(B&&B.ddel){var A=B.ddel;Ext.fly(A).removeClass([\"x-tree-drag-insert-above\",\"x-tree-drag-insert-below\",\"x-tree-drag-append\"]);this.lastInsertClass=\"_noclass\"}},beforeDragDrop:function(B,A,C){this.cancelExpand();return true},afterRepair:function(A){if(A&&Ext.enableFx){A.node.ui.highlight()}this.hideProxy()}})};\nif(Ext.dd.DragZone){Ext.tree.TreeDragZone=function(A,B){Ext.tree.TreeDragZone.superclass.constructor.call(this,A.getTreeEl(),B);this.tree=A};Ext.extend(Ext.tree.TreeDragZone,Ext.dd.DragZone,{ddGroup:\"TreeDD\",onBeforeDrag:function(A,B){var C=A.node;return C&&C.draggable&&!C.disabled},onInitDrag:function(B){var A=this.dragData;this.tree.getSelectionModel().select(A.node);this.tree.eventModel.disable();this.proxy.update(\"\");A.node.ui.appendDDGhost(this.proxy.ghost.dom);this.tree.fireEvent(\"startdrag\",this.tree,A.node,B)},getRepairXY:function(B,A){return A.node.ui.getDDRepairXY()},onEndDrag:function(A,B){this.tree.eventModel.enable.defer(100,this.tree.eventModel);this.tree.fireEvent(\"enddrag\",this.tree,A.node,B)},onValidDrop:function(A,B,C){this.tree.fireEvent(\"dragdrop\",this.tree,this.dragData.node,A,B);this.hideProxy()},beforeInvalidDrop:function(A,C){var B=this.tree.getSelectionModel();B.clearSelections();B.select(this.dragData.node)}})};\nExt.tree.TreeEditor=function(A,B){B=B||{};var C=B.events?B:new Ext.form.TextField(B);Ext.tree.TreeEditor.superclass.constructor.call(this,C);this.tree=A;if(!A.rendered){A.on(\"render\",this.initEditor,this)}else{this.initEditor(A)}};Ext.extend(Ext.tree.TreeEditor,Ext.Editor,{alignment:\"l-l\",autoSize:false,hideEl:false,cls:\"x-small-editor x-tree-editor\",shim:false,shadow:\"frame\",maxWidth:250,editDelay:350,initEditor:function(A){A.on(\"beforeclick\",this.beforeNodeClick,this);A.on(\"dblclick\",this.onNodeDblClick,this);this.on(\"complete\",this.updateNode,this);this.on(\"beforestartedit\",this.fitToTree,this);this.on(\"startedit\",this.bindScroll,this,{delay:10});this.on(\"specialkey\",this.onSpecialKey,this)},fitToTree:function(B,C){var E=this.tree.getTreeEl().dom,D=C.dom;if(E.scrollLeft>D.offsetLeft){E.scrollLeft=D.offsetLeft}var A=Math.min(this.maxWidth,(E.clientWidth>20?E.clientWidth:E.offsetWidth)-Math.max(0,D.offsetLeft-E.scrollLeft)-5);this.setSize(A,\"\")},triggerEdit:function(A,B){this.completeEdit();if(A.attributes.editable!==false){this.editNode=A;this.autoEditTimer=this.startEdit.defer(this.editDelay,this,[A.ui.textNode,A.text]);return false}},bindScroll:function(){this.tree.getTreeEl().on(\"scroll\",this.cancelEdit,this)},beforeNodeClick:function(A,B){clearTimeout(this.autoEditTimer);if(this.tree.getSelectionModel().isSelected(A)){B.stopEvent();return this.triggerEdit(A)}},onNodeDblClick:function(A,B){clearTimeout(this.autoEditTimer)},updateNode:function(A,B){this.tree.getTreeEl().un(\"scroll\",this.cancelEdit,this);this.editNode.setText(B)},onHide:function(){Ext.tree.TreeEditor.superclass.onHide.call(this);if(this.editNode){this.editNode.ui.focus.defer(50,this.editNode.ui)}},onSpecialKey:function(C,B){var A=B.getKey();if(A==B.ESC){B.stopEvent();this.cancelEdit()}else{if(A==B.ENTER&&!B.hasModifier()){B.stopEvent();this.completeEdit()}}}});\nExt.menu.Menu=function(A){if(Ext.isArray(A)){A={items:A}}Ext.apply(this,A);this.id=this.id||Ext.id();this.addEvents(\"beforeshow\",\"beforehide\",\"show\",\"hide\",\"click\",\"mouseover\",\"mouseout\",\"itemclick\");Ext.menu.MenuMgr.register(this);Ext.menu.Menu.superclass.constructor.call(this);var B=this.items;this.items=new Ext.util.MixedCollection();if(B){this.add.apply(this,B)}};Ext.extend(Ext.menu.Menu,Ext.util.Observable,{minWidth:120,shadow:\"sides\",subMenuAlign:\"tl-tr?\",defaultAlign:\"tl-bl?\",allowOtherMenus:false,hidden:true,createEl:function(){return new Ext.Layer({cls:\"x-menu\",shadow:this.shadow,constrain:false,parentEl:this.parentEl||document.body,zindex:15000})},render:function(){if(this.el){return }var B=this.el=this.createEl();if(!this.keyNav){this.keyNav=new Ext.menu.MenuNav(this)}if(this.plain){B.addClass(\"x-menu-plain\")}if(this.cls){B.addClass(this.cls)}this.focusEl=B.createChild({tag:\"a\",cls:\"x-menu-focus\",href:\"#\",onclick:\"return false;\",tabIndex:\"-1\"});var A=B.createChild({tag:\"ul\",cls:\"x-menu-list\"});A.on(\"click\",this.onClick,this);A.on(\"mouseover\",this.onMouseOver,this);A.on(\"mouseout\",this.onMouseOut,this);this.items.each(function(D){var C=document.createElement(\"li\");C.className=\"x-menu-list-item\";A.dom.appendChild(C);D.render(C,this)},this);this.ul=A;this.autoWidth()},autoWidth:function(){var D=this.el,C=this.ul;if(!D){return }var A=this.width;if(A){D.setWidth(A)}else{if(Ext.isIE){D.setWidth(this.minWidth);var B=D.dom.offsetWidth;D.setWidth(C.getWidth()+D.getFrameWidth(\"lr\"))}}},delayAutoWidth:function(){if(this.el){if(!this.awTask){this.awTask=new Ext.util.DelayedTask(this.autoWidth,this)}this.awTask.delay(20)}},findTargetItem:function(B){var A=B.getTarget(\".x-menu-list-item\",this.ul,true);if(A&&A.menuItemId){return this.items.get(A.menuItemId)}},onClick:function(B){var A;if(A=this.findTargetItem(B)){A.onClick(B);this.fireEvent(\"click\",this,A,B)}},setActiveItem:function(A,B){if(A!=this.activeItem){if(this.activeItem){this.activeItem.deactivate()}this.activeItem=A;A.activate(B)}else{if(B){A.expandMenu()}}},tryActivate:function(F,E){var B=this.items;for(var C=F,A=B.length;C>=0&&C<A;C+=E){var D=B.get(C);if(!D.disabled&&D.canActivate){this.setActiveItem(D,false);return D}}return false},onMouseOver:function(B){var A;if(A=this.findTargetItem(B)){if(A.canActivate&&!A.disabled){this.setActiveItem(A,true)}}this.fireEvent(\"mouseover\",this,B,A)},onMouseOut:function(B){var A;if(A=this.findTargetItem(B)){if(A==this.activeItem&&A.shouldDeactivate(B)){this.activeItem.deactivate();delete this.activeItem}}this.fireEvent(\"mouseout\",this,B,A)},isVisible:function(){return this.el&&!this.hidden},show:function(B,C,A){this.parentMenu=A;if(!this.el){this.render()}this.fireEvent(\"beforeshow\",this);this.showAt(this.el.getAlignToXY(B,C||this.defaultAlign),A,false)},showAt:function(C,B,A){this.parentMenu=B;if(!this.el){this.render()}if(A!==false){this.fireEvent(\"beforeshow\",this);C=this.el.adjustForConstraints(C)}this.el.setXY(C);this.el.show();this.hidden=false;this.focus();this.fireEvent(\"show\",this)},focus:function(){if(!this.hidden){this.doFocus.defer(50,this)}},doFocus:function(){if(!this.hidden){this.focusEl.focus()}},hide:function(A){if(this.el&&this.isVisible()){this.fireEvent(\"beforehide\",this);if(this.activeItem){this.activeItem.deactivate();this.activeItem=null}this.el.hide();this.hidden=true;this.fireEvent(\"hide\",this)}if(A===true&&this.parentMenu){this.parentMenu.hide(true)}},add:function(){var B=arguments,A=B.length,E;for(var C=0;C<A;C++){var D=B[C];if(D.render){E=this.addItem(D)}else{if(typeof D==\"string\"){if(D==\"separator\"||D==\"-\"){E=this.addSeparator()}else{E=this.addText(D)}}else{if(D.tagName||D.el){E=this.addElement(D)}else{if(typeof D==\"object\"){Ext.applyIf(D,this.defaults);E=this.addMenuItem(D)}}}}}return E},getEl:function(){if(!this.el){this.render()}return this.el},addSeparator:function(){return this.addItem(new Ext.menu.Separator())},addElement:function(A){return this.addItem(new Ext.menu.BaseItem(A))},addItem:function(B){this.items.add(B);if(this.ul){var A=document.createElement(\"li\");A.className=\"x-menu-list-item\";this.ul.dom.appendChild(A);B.render(A,this);this.delayAutoWidth()}return B},addMenuItem:function(A){if(!(A instanceof Ext.menu.Item)){if(typeof A.checked==\"boolean\"){A=new Ext.menu.CheckItem(A)}else{A=new Ext.menu.Item(A)}}return this.addItem(A)},addText:function(A){return this.addItem(new Ext.menu.TextItem(A))},insert:function(B,C){this.items.insert(B,C);if(this.ul){var A=document.createElement(\"li\");A.className=\"x-menu-list-item\";this.ul.dom.insertBefore(A,this.ul.dom.childNodes[B]);C.render(A,this);this.delayAutoWidth()}return C},remove:function(A){this.items.removeKey(A.id);A.destroy()},removeAll:function(){var A;while(A=this.items.first()){this.remove(A)}},destroy:function(){this.beforeDestroy();Ext.menu.MenuMgr.unregister(this);if(this.keyNav){this.keyNav.disable()}this.removeAll();if(this.ul){this.ul.removeAllListeners()}if(this.el){this.el.destroy()}},beforeDestroy:Ext.emptyFn});Ext.menu.MenuNav=function(A){Ext.menu.MenuNav.superclass.constructor.call(this,A.el);this.scope=this.menu=A};Ext.extend(Ext.menu.MenuNav,Ext.KeyNav,{doRelay:function(C,B){var A=C.getKey();if(!this.menu.activeItem&&C.isNavKeyPress()&&A!=C.SPACE&&A!=C.RETURN){this.menu.tryActivate(0,1);return false}return B.call(this.scope||this,C,this.menu)},up:function(B,A){if(!A.tryActivate(A.items.indexOf(A.activeItem)-1,-1)){A.tryActivate(A.items.length-1,-1)}},down:function(B,A){if(!A.tryActivate(A.items.indexOf(A.activeItem)+1,1)){A.tryActivate(0,1)}},right:function(B,A){if(A.activeItem){A.activeItem.expandMenu(true)}},left:function(B,A){A.hide();if(A.parentMenu&&A.parentMenu.activeItem){A.parentMenu.activeItem.activate()}},enter:function(B,A){if(A.activeItem){B.stopPropagation();A.activeItem.onClick(B);A.fireEvent(\"click\",this,A.activeItem);return true}}});\nExt.menu.MenuMgr=function(){var F,D,C={},A=false,K=new Date();function M(){F={};D=new Ext.util.MixedCollection();Ext.getDoc().addKeyListener(27,function(){if(D.length>0){H()}})}function H(){if(D&&D.length>0){var N=D.clone();N.each(function(O){O.hide()})}}function E(N){D.remove(N);if(D.length<1){Ext.getDoc().un(\"mousedown\",L);A=false}}function J(N){var O=D.last();K=new Date();D.add(N);if(!A){Ext.getDoc().on(\"mousedown\",L);A=true}if(N.parentMenu){N.getEl().setZIndex(parseInt(N.parentMenu.getEl().getStyle(\"z-index\"),10)+3);N.parentMenu.activeChild=N}else{if(O&&O.isVisible()){N.getEl().setZIndex(parseInt(O.getEl().getStyle(\"z-index\"),10)+3)}}}function B(N){if(N.activeChild){N.activeChild.hide()}if(N.autoHideTimer){clearTimeout(N.autoHideTimer);delete N.autoHideTimer}}function G(N){var O=N.parentMenu;if(!O&&!N.allowOtherMenus){H()}else{if(O&&O.activeChild){O.activeChild.hide()}}}function L(N){if(K.getElapsed()>50&&D.length>0&&!N.getTarget(\".x-menu\")){H()}}function I(O,R){if(R){var Q=C[O.group];for(var P=0,N=Q.length;P<N;P++){if(Q[P]!=O){Q[P].setChecked(false)}}}}return{hideAll:function(){H()},register:function(O){if(!F){M()}F[O.id]=O;O.on(\"beforehide\",B);O.on(\"hide\",E);O.on(\"beforeshow\",G);O.on(\"show\",J);var N=O.group;if(N&&O.events[\"checkchange\"]){if(!C[N]){C[N]=[]}C[N].push(O);O.on(\"checkchange\",onCheck)}},get:function(N){if(typeof N==\"string\"){if(!F){return null}return F[N]}else{if(N.events){return N}else{if(typeof N.length==\"number\"){return new Ext.menu.Menu({items:N})}else{return new Ext.menu.Menu(N)}}}},unregister:function(O){delete F[O.id];O.un(\"beforehide\",B);O.un(\"hide\",E);O.un(\"beforeshow\",G);O.un(\"show\",J);var N=O.group;if(N&&O.events[\"checkchange\"]){C[N].remove(O);O.un(\"checkchange\",onCheck)}},registerCheckable:function(N){var O=N.group;if(O){if(!C[O]){C[O]=[]}C[O].push(N);N.on(\"beforecheckchange\",I)}},unregisterCheckable:function(N){var O=N.group;if(O){C[O].remove(N);N.un(\"beforecheckchange\",I)}},getCheckedItem:function(P){var Q=C[P];if(Q){for(var O=0,N=Q.length;O<N;O++){if(Q[O].checked){return Q[O]}}}return null},setCheckedItem:function(P,R){var Q=C[P];if(Q){for(var O=0,N=Q.length;O<N;O++){if(Q[O].id==R){Q[O].setChecked(true)}}}return null}}}();\nExt.menu.BaseItem=function(A){Ext.menu.BaseItem.superclass.constructor.call(this,A);this.addEvents(\"click\",\"activate\",\"deactivate\");if(this.handler){this.on(\"click\",this.handler,this.scope)}};Ext.extend(Ext.menu.BaseItem,Ext.Component,{canActivate:false,activeClass:\"x-menu-item-active\",hideOnClick:true,hideDelay:100,ctype:\"Ext.menu.BaseItem\",actionMode:\"container\",render:function(A,B){this.parentMenu=B;Ext.menu.BaseItem.superclass.render.call(this,A);this.container.menuItemId=this.id},onRender:function(B,A){this.el=Ext.get(this.el);B.dom.appendChild(this.el.dom)},setHandler:function(B,A){if(this.handler){this.un(\"click\",this.handler,this.scope)}this.on(\"click\",this.handler=B,this.scope=A)},onClick:function(A){if(!this.disabled&&this.fireEvent(\"click\",this,A)!==false&&this.parentMenu.fireEvent(\"itemclick\",this,A)!==false){this.handleClick(A)}else{A.stopEvent()}},activate:function(){if(this.disabled){return false}var A=this.container;A.addClass(this.activeClass);this.region=A.getRegion().adjust(2,2,-2,-2);this.fireEvent(\"activate\",this);return true},deactivate:function(){this.container.removeClass(this.activeClass);this.fireEvent(\"deactivate\",this)},shouldDeactivate:function(A){return !this.region||!this.region.contains(A.getPoint())},handleClick:function(A){if(this.hideOnClick){this.parentMenu.hide.defer(this.hideDelay,this.parentMenu,[true])}},expandMenu:function(A){},hideMenu:function(){}});\nExt.menu.TextItem=function(A){this.text=A;Ext.menu.TextItem.superclass.constructor.call(this)};Ext.extend(Ext.menu.TextItem,Ext.menu.BaseItem,{hideOnClick:false,itemCls:\"x-menu-text\",onRender:function(){var A=document.createElement(\"span\");A.className=this.itemCls;A.innerHTML=this.text;this.el=A;Ext.menu.TextItem.superclass.onRender.apply(this,arguments)}});\nExt.menu.Separator=function(A){Ext.menu.Separator.superclass.constructor.call(this,A)};Ext.extend(Ext.menu.Separator,Ext.menu.BaseItem,{itemCls:\"x-menu-sep\",hideOnClick:false,onRender:function(A){var B=document.createElement(\"span\");B.className=this.itemCls;B.innerHTML=\"&#160;\";this.el=B;A.addClass(\"x-menu-sep-li\");Ext.menu.Separator.superclass.onRender.apply(this,arguments)}});\nExt.menu.Item=function(A){Ext.menu.Item.superclass.constructor.call(this,A);if(this.menu){this.menu=Ext.menu.MenuMgr.get(this.menu)}};Ext.extend(Ext.menu.Item,Ext.menu.BaseItem,{itemCls:\"x-menu-item\",canActivate:true,showDelay:200,hideDelay:200,ctype:\"Ext.menu.Item\",onRender:function(B,A){var C=document.createElement(\"a\");C.hideFocus=true;C.unselectable=\"on\";C.href=this.href||\"#\";if(this.hrefTarget){C.target=this.hrefTarget}C.className=this.itemCls+(this.menu?\" x-menu-item-arrow\":\"\")+(this.cls?\" \"+this.cls:\"\");C.innerHTML=String.format(\"<img src=\\\"{0}\\\" class=\\\"x-menu-item-icon {2}\\\" />{1}\",this.icon||Ext.BLANK_IMAGE_URL,this.itemText||this.text,this.iconCls||\"\");this.el=C;Ext.menu.Item.superclass.onRender.call(this,B,A)},setText:function(A){this.text=A;if(this.rendered){this.el.update(String.format(\"<img src=\\\"{0}\\\" class=\\\"x-menu-item-icon {2}\\\">{1}\",this.icon||Ext.BLANK_IMAGE_URL,this.text,this.iconCls||\"\"));this.parentMenu.autoWidth()}},setIconClass:function(A){var B=this.iconCls;this.iconCls=A;if(this.rendered){this.el.child(\"img.x-menu-item-icon\").replaceClass(B,this.iconCls)}},handleClick:function(A){if(!this.href){A.stopEvent()}Ext.menu.Item.superclass.handleClick.apply(this,arguments)},activate:function(A){if(Ext.menu.Item.superclass.activate.apply(this,arguments)){this.focus();if(A){this.expandMenu()}}return true},shouldDeactivate:function(A){if(Ext.menu.Item.superclass.shouldDeactivate.call(this,A)){if(this.menu&&this.menu.isVisible()){return !this.menu.getEl().getRegion().contains(A.getPoint())}return true}return false},deactivate:function(){Ext.menu.Item.superclass.deactivate.apply(this,arguments);this.hideMenu()},expandMenu:function(A){if(!this.disabled&&this.menu){clearTimeout(this.hideTimer);delete this.hideTimer;if(!this.menu.isVisible()&&!this.showTimer){this.showTimer=this.deferExpand.defer(this.showDelay,this,[A])}else{if(this.menu.isVisible()&&A){this.menu.tryActivate(0,1)}}}},deferExpand:function(A){delete this.showTimer;this.menu.show(this.container,this.parentMenu.subMenuAlign||\"tl-tr?\",this.parentMenu);if(A){this.menu.tryActivate(0,1)}},hideMenu:function(){clearTimeout(this.showTimer);delete this.showTimer;if(!this.hideTimer&&this.menu&&this.menu.isVisible()){this.hideTimer=this.deferHide.defer(this.hideDelay,this)}},deferHide:function(){delete this.hideTimer;this.menu.hide()}});\nExt.menu.CheckItem=function(A){Ext.menu.CheckItem.superclass.constructor.call(this,A);this.addEvents(\"beforecheckchange\",\"checkchange\");if(this.checkHandler){this.on(\"checkchange\",this.checkHandler,this.scope)}Ext.menu.MenuMgr.registerCheckable(this)};Ext.extend(Ext.menu.CheckItem,Ext.menu.Item,{itemCls:\"x-menu-item x-menu-check-item\",groupClass:\"x-menu-group-item\",checked:false,ctype:\"Ext.menu.CheckItem\",onRender:function(A){Ext.menu.CheckItem.superclass.onRender.apply(this,arguments);if(this.group){this.el.addClass(this.groupClass)}if(this.checked){this.checked=false;this.setChecked(true,true)}},destroy:function(){Ext.menu.MenuMgr.unregisterCheckable(this);Ext.menu.CheckItem.superclass.destroy.apply(this,arguments)},setChecked:function(B,A){if(this.checked!=B&&this.fireEvent(\"beforecheckchange\",this,B)!==false){if(this.container){this.container[B?\"addClass\":\"removeClass\"](\"x-menu-item-checked\")}this.checked=B;if(A!==true){this.fireEvent(\"checkchange\",this,B)}}},handleClick:function(A){if(!this.disabled&&!(this.checked&&this.group)){this.setChecked(!this.checked)}Ext.menu.CheckItem.superclass.handleClick.apply(this,arguments)}});\nExt.menu.Adapter=function(B,A){Ext.menu.Adapter.superclass.constructor.call(this,A);this.component=B};Ext.extend(Ext.menu.Adapter,Ext.menu.BaseItem,{canActivate:true,onRender:function(B,A){this.component.render(B);this.el=this.component.getEl()},activate:function(){if(this.disabled){return false}this.component.focus();this.fireEvent(\"activate\",this);return true},deactivate:function(){this.fireEvent(\"deactivate\",this)},disable:function(){this.component.disable();Ext.menu.Adapter.superclass.disable.call(this)},enable:function(){this.component.enable();Ext.menu.Adapter.superclass.enable.call(this)}});\nExt.menu.DateItem=function(A){Ext.menu.DateItem.superclass.constructor.call(this,new Ext.DatePicker(A),A);this.picker=this.component;this.addEvents(\"select\");this.picker.on(\"render\",function(B){B.getEl().swallowEvent(\"click\");B.container.addClass(\"x-menu-date-item\")});this.picker.on(\"select\",this.onSelect,this)};Ext.extend(Ext.menu.DateItem,Ext.menu.Adapter,{onSelect:function(B,A){this.fireEvent(\"select\",this,A,B);Ext.menu.DateItem.superclass.handleClick.call(this)}});\nExt.menu.ColorItem=function(A){Ext.menu.ColorItem.superclass.constructor.call(this,new Ext.ColorPalette(A),A);this.palette=this.component;this.relayEvents(this.palette,[\"select\"]);if(this.selectHandler){this.on(\"select\",this.selectHandler,this.scope)}};Ext.extend(Ext.menu.ColorItem,Ext.menu.Adapter);\nExt.menu.DateMenu=function(A){Ext.menu.DateMenu.superclass.constructor.call(this,A);this.plain=true;var B=new Ext.menu.DateItem(A);this.add(B);this.picker=B.picker;this.relayEvents(B,[\"select\"]);this.on(\"beforeshow\",function(){if(this.picker){this.picker.hideMonthPicker(true)}},this)};Ext.extend(Ext.menu.DateMenu,Ext.menu.Menu,{cls:\"x-date-menu\",beforeDestroy:function(){this.picker.destroy()}});\nExt.menu.ColorMenu=function(A){Ext.menu.ColorMenu.superclass.constructor.call(this,A);this.plain=true;var B=new Ext.menu.ColorItem(A);this.add(B);this.palette=B.palette;this.relayEvents(B,[\"select\"])};Ext.extend(Ext.menu.ColorMenu,Ext.menu.Menu);\nExt.form.Field=Ext.extend(Ext.BoxComponent,{invalidClass:\"x-form-invalid\",invalidText:\"The value in this field is invalid\",focusClass:\"x-form-focus\",validationEvent:\"keyup\",validateOnBlur:true,validationDelay:250,defaultAutoCreate:{tag:\"input\",type:\"text\",size:\"20\",autocomplete:\"off\"},fieldClass:\"x-form-field\",msgTarget:\"qtip\",msgFx:\"normal\",readOnly:false,disabled:false,isFormField:true,hasFocus:false,initComponent:function(){Ext.form.Field.superclass.initComponent.call(this);this.addEvents(\"focus\",\"blur\",\"specialkey\",\"change\",\"invalid\",\"valid\")},getName:function(){return this.rendered&&this.el.dom.name?this.el.dom.name:(this.hiddenName||\"\")},onRender:function(C,A){Ext.form.Field.superclass.onRender.call(this,C,A);if(!this.el){var B=this.getAutoCreate();if(!B.name){B.name=this.name||this.id}if(this.inputType){B.type=this.inputType}this.el=C.createChild(B,A)}var D=this.el.dom.type;if(D){if(D==\"password\"){D=\"text\"}this.el.addClass(\"x-form-\"+D)}if(this.readOnly){this.el.dom.readOnly=true}if(this.tabIndex!==undefined){this.el.dom.setAttribute(\"tabIndex\",this.tabIndex)}this.el.addClass([this.fieldClass,this.cls]);this.initValue()},initValue:function(){if(this.value!==undefined){this.setValue(this.value)}else{if(this.el.dom.value.length>0){this.setValue(this.el.dom.value)}}},isDirty:function(){if(this.disabled){return false}return String(this.getValue())!==String(this.originalValue)},afterRender:function(){Ext.form.Field.superclass.afterRender.call(this);this.initEvents()},fireKey:function(A){if(A.isSpecialKey()){this.fireEvent(\"specialkey\",this,A)}},reset:function(){this.setValue(this.originalValue);this.clearInvalid()},initEvents:function(){this.el.on(Ext.isIE?\"keydown\":\"keypress\",this.fireKey,this);this.el.on(\"focus\",this.onFocus,this);this.el.on(\"blur\",this.onBlur,this);this.originalValue=this.getValue()},onFocus:function(){if(!Ext.isOpera&&this.focusClass){this.el.addClass(this.focusClass)}if(!this.hasFocus){this.hasFocus=true;this.startValue=this.getValue();this.fireEvent(\"focus\",this)}},beforeBlur:Ext.emptyFn,onBlur:function(){this.beforeBlur();if(!Ext.isOpera&&this.focusClass){this.el.removeClass(this.focusClass)}this.hasFocus=false;if(this.validationEvent!==false&&this.validateOnBlur&&this.validationEvent!=\"blur\"){this.validate()}var A=this.getValue();if(String(A)!==String(this.startValue)){this.fireEvent(\"change\",this,A,this.startValue)}this.fireEvent(\"blur\",this)},isValid:function(A){if(this.disabled){return true}var C=this.preventMark;this.preventMark=A===true;var B=this.validateValue(this.processValue(this.getRawValue()));this.preventMark=C;return B},validate:function(){if(this.disabled||this.validateValue(this.processValue(this.getRawValue()))){this.clearInvalid();return true}return false},processValue:function(A){return A},validateValue:function(A){return true},markInvalid:function(C){if(!this.rendered||this.preventMark){return }this.el.addClass(this.invalidClass);C=C||this.invalidText;switch(this.msgTarget){case\"qtip\":this.el.dom.qtip=C;this.el.dom.qclass=\"x-form-invalid-tip\";if(Ext.QuickTips){Ext.QuickTips.enable()}break;case\"title\":this.el.dom.title=C;break;case\"under\":if(!this.errorEl){var B=this.el.findParent(\".x-form-element\",5,true);this.errorEl=B.createChild({cls:\"x-form-invalid-msg\"});this.errorEl.setWidth(B.getWidth(true)-20)}this.errorEl.update(C);Ext.form.Field.msgFx[this.msgFx].show(this.errorEl,this);break;case\"side\":if(!this.errorIcon){var B=this.el.findParent(\".x-form-element\",5,true);this.errorIcon=B.createChild({cls:\"x-form-invalid-icon\"})}this.alignErrorIcon();this.errorIcon.dom.qtip=C;this.errorIcon.dom.qclass=\"x-form-invalid-tip\";this.errorIcon.show();this.on(\"resize\",this.alignErrorIcon,this);break;default:var A=Ext.getDom(this.msgTarget);A.innerHTML=C;A.style.display=this.msgDisplay;break}this.fireEvent(\"invalid\",this,C)},alignErrorIcon:function(){this.errorIcon.alignTo(this.el,\"tl-tr\",[2,0])},clearInvalid:function(){if(!this.rendered||this.preventMark){return }this.el.removeClass(this.invalidClass);switch(this.msgTarget){case\"qtip\":this.el.dom.qtip=\"\";break;case\"title\":this.el.dom.title=\"\";break;case\"under\":if(this.errorEl){Ext.form.Field.msgFx[this.msgFx].hide(this.errorEl,this)}break;case\"side\":if(this.errorIcon){this.errorIcon.dom.qtip=\"\";this.errorIcon.hide();this.un(\"resize\",this.alignErrorIcon,this)}break;default:var A=Ext.getDom(this.msgTarget);A.innerHTML=\"\";A.style.display=\"none\";break}this.fireEvent(\"valid\",this)},getRawValue:function(){var A=this.rendered?this.el.getValue():Ext.value(this.value,\"\");if(A===this.emptyText){A=\"\"}return A},getValue:function(){if(!this.rendered){return this.value}var A=this.el.getValue();if(A===this.emptyText||A===undefined){A=\"\"}return A},setRawValue:function(A){return this.el.dom.value=(A===null||A===undefined?\"\":A)},setValue:function(A){this.value=A;if(this.rendered){this.el.dom.value=(A===null||A===undefined?\"\":A);this.validate()}},adjustSize:function(A,C){var B=Ext.form.Field.superclass.adjustSize.call(this,A,C);B.width=this.adjustWidth(this.el.dom.tagName,B.width);return B},adjustWidth:function(A,B){A=A.toLowerCase();if(typeof B==\"number\"&&!Ext.isSafari){if(Ext.isIE&&(A==\"input\"||A==\"textarea\")){if(A==\"input\"&&!Ext.isStrict){return this.inEditor?B:B-3}if(A==\"input\"&&Ext.isStrict){return B-(Ext.isIE6?4:1)}if(A=\"textarea\"&&Ext.isStrict){return B-2}}else{if(Ext.isOpera&&Ext.isStrict){if(A==\"input\"){return B+2}if(A=\"textarea\"){return B-2}}}}return B}});Ext.form.Field.msgFx={normal:{show:function(A,B){A.setDisplayed(\"block\")},hide:function(A,B){A.setDisplayed(false).update(\"\")}},slide:{show:function(A,B){A.slideIn(\"t\",{stopFx:true})},hide:function(A,B){A.slideOut(\"t\",{stopFx:true,useDisplay:true})}},slideRight:{show:function(A,B){A.fixDisplay();A.alignTo(B.el,\"tl-tr\");A.slideIn(\"l\",{stopFx:true})},hide:function(A,B){A.slideOut(\"l\",{stopFx:true,useDisplay:true})}}};Ext.reg(\"field\",Ext.form.Field);\nExt.form.TextField=Ext.extend(Ext.form.Field,{grow:false,growMin:30,growMax:800,vtype:null,maskRe:null,disableKeyFilter:false,allowBlank:true,minLength:0,maxLength:Number.MAX_VALUE,minLengthText:\"The minimum length for this field is {0}\",maxLengthText:\"The maximum length for this field is {0}\",selectOnFocus:false,blankText:\"This field is required\",validator:null,regex:null,regexText:\"\",emptyText:null,emptyClass:\"x-form-empty-field\",initComponent:function(){Ext.form.TextField.superclass.initComponent.call(this);this.addEvents(\"autosize\")},initEvents:function(){Ext.form.TextField.superclass.initEvents.call(this);if(this.validationEvent==\"keyup\"){this.validationTask=new Ext.util.DelayedTask(this.validate,this);this.el.on(\"keyup\",this.filterValidation,this)}else{if(this.validationEvent!==false){this.el.on(this.validationEvent,this.validate,this,{buffer:this.validationDelay})}}if(this.selectOnFocus||this.emptyText){this.on(\"focus\",this.preFocus,this);if(this.emptyText){this.on(\"blur\",this.postBlur,this);this.applyEmptyText()}}if(this.maskRe||(this.vtype&&this.disableKeyFilter!==true&&(this.maskRe=Ext.form.VTypes[this.vtype+\"Mask\"]))){this.el.on(\"keypress\",this.filterKeys,this)}if(this.grow){this.el.on(\"keyup\",this.onKeyUp,this,{buffer:50});this.el.on(\"click\",this.autoSize,this)}},processValue:function(A){if(this.stripCharsRe){var B=A.replace(this.stripCharsRe,\"\");if(B!==A){this.setRawValue(B);return B}}return A},filterValidation:function(A){if(!A.isNavKeyPress()){this.validationTask.delay(this.validationDelay)}},onKeyUp:function(A){if(!A.isNavKeyPress()){this.autoSize()}},reset:function(){Ext.form.TextField.superclass.reset.call(this);this.applyEmptyText()},applyEmptyText:function(){if(this.rendered&&this.emptyText&&this.getRawValue().length<1){this.setRawValue(this.emptyText);this.el.addClass(this.emptyClass)}},preFocus:function(){if(this.emptyText){if(this.el.dom.value==this.emptyText){this.setRawValue(\"\")}this.el.removeClass(this.emptyClass)}if(this.selectOnFocus){this.el.dom.select()}},postBlur:function(){this.applyEmptyText()},filterKeys:function(B){var A=B.getKey();if(!Ext.isIE&&(B.isNavKeyPress()||A==B.BACKSPACE||(A==B.DELETE&&B.button==-1))){return }var D=B.getCharCode(),C=String.fromCharCode(D);if(Ext.isIE&&(B.isSpecialKey()||!C)){return }if(!this.maskRe.test(C)){B.stopEvent()}},setValue:function(A){if(this.emptyText&&this.el&&A!==undefined&&A!==null&&A!==\"\"){this.el.removeClass(this.emptyClass)}Ext.form.TextField.superclass.setValue.apply(this,arguments);this.applyEmptyText();this.autoSize()},validateValue:function(A){if(A.length<1||A===this.emptyText){if(this.allowBlank){this.clearInvalid();return true}else{this.markInvalid(this.blankText);return false}}if(A.length<this.minLength){this.markInvalid(String.format(this.minLengthText,this.minLength));return false}if(A.length>this.maxLength){this.markInvalid(String.format(this.maxLengthText,this.maxLength));return false}if(this.vtype){var C=Ext.form.VTypes;if(!C[this.vtype](A,this)){this.markInvalid(this.vtypeText||C[this.vtype+\"Text\"]);return false}}if(typeof this.validator==\"function\"){var B=this.validator(A);if(B!==true){this.markInvalid(B);return false}}if(this.regex&&!this.regex.test(A)){this.markInvalid(this.regexText);return false}return true},selectText:function(E,A){var C=this.getRawValue();if(C.length>0){E=E===undefined?0:E;A=A===undefined?C.length:A;var D=this.el.dom;if(D.setSelectionRange){D.setSelectionRange(E,A)}else{if(D.createTextRange){var B=D.createTextRange();B.moveStart(\"character\",E);B.moveEnd(\"character\",A-C.length);B.select()}}}},autoSize:function(){if(!this.grow||!this.rendered){return }if(!this.metrics){this.metrics=Ext.util.TextMetrics.createInstance(this.el)}var C=this.el;var B=C.dom.value;var D=document.createElement(\"div\");D.appendChild(document.createTextNode(B));B=D.innerHTML;D=null;B+=\"&#160;\";var A=Math.min(this.growMax,Math.max(this.metrics.getWidth(B)+10,this.growMin));this.el.setWidth(A);this.fireEvent(\"autosize\",this,A)}});Ext.reg(\"textfield\",Ext.form.TextField);\nExt.form.TriggerField=Ext.extend(Ext.form.TextField,{defaultAutoCreate:{tag:\"input\",type:\"text\",size:\"16\",autocomplete:\"off\"},hideTrigger:false,autoSize:Ext.emptyFn,monitorTab:true,deferHeight:true,mimicing:false,onResize:function(A,B){Ext.form.TriggerField.superclass.onResize.call(this,A,B);if(typeof A==\"number\"){this.el.setWidth(this.adjustWidth(\"input\",A-this.trigger.getWidth()))}this.wrap.setWidth(this.el.getWidth()+this.trigger.getWidth())},adjustSize:Ext.BoxComponent.prototype.adjustSize,getResizeEl:function(){return this.wrap},getPositionEl:function(){return this.wrap},alignErrorIcon:function(){this.errorIcon.alignTo(this.wrap,\"tl-tr\",[2,0])},onRender:function(B,A){Ext.form.TriggerField.superclass.onRender.call(this,B,A);this.wrap=this.el.wrap({cls:\"x-form-field-wrap\"});this.trigger=this.wrap.createChild(this.triggerConfig||{tag:\"img\",src:Ext.BLANK_IMAGE_URL,cls:\"x-form-trigger \"+this.triggerClass});if(this.hideTrigger){this.trigger.setDisplayed(false)}this.initTrigger();if(!this.width){this.wrap.setWidth(this.el.getWidth()+this.trigger.getWidth())}},initTrigger:function(){this.trigger.on(\"click\",this.onTriggerClick,this,{preventDefault:true});this.trigger.addClassOnOver(\"x-form-trigger-over\");this.trigger.addClassOnClick(\"x-form-trigger-click\")},onDestroy:function(){if(this.trigger){this.trigger.removeAllListeners();this.trigger.remove()}if(this.wrap){this.wrap.remove()}Ext.form.TriggerField.superclass.onDestroy.call(this)},onFocus:function(){Ext.form.TriggerField.superclass.onFocus.call(this);if(!this.mimicing){this.wrap.addClass(\"x-trigger-wrap-focus\");this.mimicing=true;Ext.get(Ext.isIE?document.body:document).on(\"mousedown\",this.mimicBlur,this,{delay:10});if(this.monitorTab){this.el.on(\"keydown\",this.checkTab,this)}}},checkTab:function(A){if(A.getKey()==A.TAB){this.triggerBlur()}},onBlur:function(){},mimicBlur:function(A){if(!this.wrap.contains(A.target)&&this.validateBlur(A)){this.triggerBlur()}},triggerBlur:function(){this.mimicing=false;Ext.get(Ext.isIE?document.body:document).un(\"mousedown\",this.mimicBlur);if(this.monitorTab){this.el.un(\"keydown\",this.checkTab,this)}this.beforeBlur();this.wrap.removeClass(\"x-trigger-wrap-focus\");Ext.form.TriggerField.superclass.onBlur.call(this)},beforeBlur:Ext.emptyFn,validateBlur:function(A){return true},onDisable:function(){Ext.form.TriggerField.superclass.onDisable.call(this);if(this.wrap){this.wrap.addClass(\"x-item-disabled\")}},onEnable:function(){Ext.form.TriggerField.superclass.onEnable.call(this);if(this.wrap){this.wrap.removeClass(\"x-item-disabled\")}},onShow:function(){if(this.wrap){this.wrap.dom.style.display=\"\";this.wrap.dom.style.visibility=\"visible\"}},onHide:function(){this.wrap.dom.style.display=\"none\"},onTriggerClick:Ext.emptyFn});Ext.form.TwinTriggerField=Ext.extend(Ext.form.TriggerField,{initComponent:function(){Ext.form.TwinTriggerField.superclass.initComponent.call(this);this.triggerConfig={tag:\"span\",cls:\"x-form-twin-triggers\",cn:[{tag:\"img\",src:Ext.BLANK_IMAGE_URL,cls:\"x-form-trigger \"+this.trigger1Class},{tag:\"img\",src:Ext.BLANK_IMAGE_URL,cls:\"x-form-trigger \"+this.trigger2Class}]}},getTrigger:function(A){return this.triggers[A]},initTrigger:function(){var A=this.trigger.select(\".x-form-trigger\",true);this.wrap.setStyle(\"overflow\",\"hidden\");var B=this;A.each(function(D,F,C){D.hide=function(){var G=B.wrap.getWidth();this.dom.style.display=\"none\";B.el.setWidth(G-B.trigger.getWidth())};D.show=function(){var G=B.wrap.getWidth();this.dom.style.display=\"\";B.el.setWidth(G-B.trigger.getWidth())};var E=\"Trigger\"+(C+1);if(this[\"hide\"+E]){D.dom.style.display=\"none\"}D.on(\"click\",this[\"on\"+E+\"Click\"],this,{preventDefault:true});D.addClassOnOver(\"x-form-trigger-over\");D.addClassOnClick(\"x-form-trigger-click\")},this);this.triggers=A.elements},onTrigger1Click:Ext.emptyFn,onTrigger2Click:Ext.emptyFn});Ext.reg(\"trigger\",Ext.form.TriggerField);\nExt.form.TextArea=Ext.extend(Ext.form.TextField,{growMin:60,growMax:1000,growAppend:\"&#160;\\n&#160;\",growPad:0,enterIsSpecial:false,preventScrollbars:false,onRender:function(B,A){if(!this.el){this.defaultAutoCreate={tag:\"textarea\",style:\"width:100px;height:60px;\",autocomplete:\"off\"}}Ext.form.TextArea.superclass.onRender.call(this,B,A);if(this.grow){this.textSizeEl=Ext.DomHelper.append(document.body,{tag:\"pre\",cls:\"x-form-grow-sizer\"});if(this.preventScrollbars){this.el.setStyle(\"overflow\",\"hidden\")}this.el.setHeight(this.growMin)}},onDestroy:function(){if(this.textSizeEl){Ext.removeNode(this.textSizeEl)}Ext.form.TextArea.superclass.onDestroy.call(this)},fireKey:function(A){if(A.isSpecialKey()&&(this.enterIsSpecial||(A.getKey()!=A.ENTER||A.hasModifier()))){this.fireEvent(\"specialkey\",this,A)}},onKeyUp:function(A){if(!A.isNavKeyPress()||A.getKey()==A.ENTER){this.autoSize()}},autoSize:function(){if(!this.grow||!this.textSizeEl){return }var C=this.el;var A=C.dom.value;var D=this.textSizeEl;D.innerHTML=\"\";D.appendChild(document.createTextNode(A));A=D.innerHTML;Ext.fly(D).setWidth(this.el.getWidth());if(A.length<1){A=\"&#160;&#160;\"}else{if(Ext.isIE){A=A.replace(/\\n/g,\"<p>&#160;</p>\")}A+=this.growAppend}D.innerHTML=A;var B=Math.min(this.growMax,Math.max(D.offsetHeight,this.growMin)+this.growPad);if(B!=this.lastHeight){this.lastHeight=B;this.el.setHeight(B);this.fireEvent(\"autosize\",this,B)}}});Ext.reg(\"textarea\",Ext.form.TextArea);\nExt.form.NumberField=Ext.extend(Ext.form.TextField,{fieldClass:\"x-form-field x-form-num-field\",allowDecimals:true,decimalSeparator:\".\",decimalPrecision:2,allowNegative:true,minValue:Number.NEGATIVE_INFINITY,maxValue:Number.MAX_VALUE,minText:\"The minimum value for this field is {0}\",maxText:\"The maximum value for this field is {0}\",nanText:\"{0} is not a valid number\",baseChars:\"0123456789\",initEvents:function(){Ext.form.NumberField.superclass.initEvents.call(this);var B=this.baseChars+\"\";if(this.allowDecimals){B+=this.decimalSeparator}if(this.allowNegative){B+=\"-\"}this.stripCharsRe=new RegExp(\"[^\"+B+\"]\",\"gi\");var A=function(D){var C=D.getKey();if(!Ext.isIE&&(D.isSpecialKey()||C==D.BACKSPACE||C==D.DELETE)){return }var E=D.getCharCode();if(B.indexOf(String.fromCharCode(E))===-1){D.stopEvent()}};this.el.on(\"keypress\",A,this)},validateValue:function(B){if(!Ext.form.NumberField.superclass.validateValue.call(this,B)){return false}if(B.length<1){return true}B=String(B).replace(this.decimalSeparator,\".\");if(isNaN(B)){this.markInvalid(String.format(this.nanText,B));return false}var A=this.parseValue(B);if(A<this.minValue){this.markInvalid(String.format(this.minText,this.minValue));return false}if(A>this.maxValue){this.markInvalid(String.format(this.maxText,this.maxValue));return false}return true},getValue:function(){return this.fixPrecision(this.parseValue(Ext.form.NumberField.superclass.getValue.call(this)))},setValue:function(A){A=parseFloat(A);A=isNaN(A)?\"\":String(A).replace(\".\",this.decimalSeparator);Ext.form.NumberField.superclass.setValue.call(this,A)},parseValue:function(A){A=parseFloat(String(A).replace(this.decimalSeparator,\".\"));return isNaN(A)?\"\":A},fixPrecision:function(B){var A=isNaN(B);if(!this.allowDecimals||this.decimalPrecision==-1||A||!B){return A?\"\":B}return parseFloat(parseFloat(B).toFixed(this.decimalPrecision))},beforeBlur:function(){var A=this.parseValue(this.getRawValue());if(A){this.setValue(this.fixPrecision(A))}}});Ext.reg(\"numberfield\",Ext.form.NumberField);\nExt.form.DateField=Ext.extend(Ext.form.TriggerField,{format:\"m/d/y\",altFormats:\"m/d/Y|n/j/Y|n/j/y|m/j/y|n/d/y|m/j/Y|n/d/Y|m-d-y|m-d-Y|m/d|m-d|md|mdy|mdY|d|Y-m-d\",disabledDays:null,disabledDaysText:\"Disabled\",disabledDates:null,disabledDatesText:\"Disabled\",minValue:null,maxValue:null,minText:\"The date in this field must be equal to or after {0}\",maxText:\"The date in this field must be equal to or before {0}\",invalidText:\"{0} is not a valid date - it must be in the format {1}\",triggerClass:\"x-form-date-trigger\",defaultAutoCreate:{tag:\"input\",type:\"text\",size:\"10\",autocomplete:\"off\"},initComponent:function(){Ext.form.DateField.superclass.initComponent.call(this);if(typeof this.minValue==\"string\"){this.minValue=this.parseDate(this.minValue)}if(typeof this.maxValue==\"string\"){this.maxValue=this.parseDate(this.maxValue)}this.ddMatch=null;if(this.disabledDates){var A=this.disabledDates;var C=\"(?:\";for(var B=0;B<A.length;B++){C+=A[B];if(B!=A.length-1){C+=\"|\"}}this.ddMatch=new RegExp(C+\")\")}},validateValue:function(E){E=this.formatDate(E);if(!Ext.form.DateField.superclass.validateValue.call(this,E)){return false}if(E.length<1){return true}var C=E;E=this.parseDate(E);if(!E){this.markInvalid(String.format(this.invalidText,C,this.format));return false}var F=E.getTime();if(this.minValue&&F<this.minValue.getTime()){this.markInvalid(String.format(this.minText,this.formatDate(this.minValue)));return false}if(this.maxValue&&F>this.maxValue.getTime()){this.markInvalid(String.format(this.maxText,this.formatDate(this.maxValue)));return false}if(this.disabledDays){var A=E.getDay();for(var B=0;B<this.disabledDays.length;B++){if(A===this.disabledDays[B]){this.markInvalid(this.disabledDaysText);return false}}}var D=this.formatDate(E);if(this.ddMatch&&this.ddMatch.test(D)){this.markInvalid(String.format(this.disabledDatesText,D));return false}return true},validateBlur:function(){return !this.menu||!this.menu.isVisible()},getValue:function(){return this.parseDate(Ext.form.DateField.superclass.getValue.call(this))||\"\"},setValue:function(A){Ext.form.DateField.superclass.setValue.call(this,this.formatDate(this.parseDate(A)))},parseDate:function(D){if(!D||Ext.isDate(D)){return D}var B=Date.parseDate(D,this.format);if(!B&&this.altFormats){if(!this.altFormatsArray){this.altFormatsArray=this.altFormats.split(\"|\")}for(var C=0,A=this.altFormatsArray.length;C<A&&!B;C++){B=Date.parseDate(D,this.altFormatsArray[C])}}return B},onDestroy:function(){if(this.menu){this.menu.destroy()}if(this.wrap){this.wrap.remove()}Ext.form.DateField.superclass.onDestroy.call(this)},formatDate:function(A){return Ext.isDate(A)?A.dateFormat(this.format):A},menuListeners:{select:function(A,B){this.setValue(B)},show:function(){this.onFocus()},hide:function(){this.focus.defer(10,this);var A=this.menuListeners;this.menu.un(\"select\",A.select,this);this.menu.un(\"show\",A.show,this);this.menu.un(\"hide\",A.hide,this)}},onTriggerClick:function(){if(this.disabled){return }if(this.menu==null){this.menu=new Ext.menu.DateMenu()}Ext.apply(this.menu.picker,{minDate:this.minValue,maxDate:this.maxValue,disabledDatesRE:this.ddMatch,disabledDatesText:this.disabledDatesText,disabledDays:this.disabledDays,disabledDaysText:this.disabledDaysText,format:this.format,minText:String.format(this.minText,this.formatDate(this.minValue)),maxText:String.format(this.maxText,this.formatDate(this.maxValue))});this.menu.on(Ext.apply({},this.menuListeners,{scope:this}));this.menu.picker.setValue(this.getValue()||new Date());this.menu.show(this.el,\"tl-bl?\")},beforeBlur:function(){var A=this.parseDate(this.getRawValue());if(A){this.setValue(A)}}});Ext.reg(\"datefield\",Ext.form.DateField);\nExt.form.ComboBox=Ext.extend(Ext.form.TriggerField,{defaultAutoCreate:{tag:\"input\",type:\"text\",size:\"24\",autocomplete:\"off\"},listClass:\"\",selectedClass:\"x-combo-selected\",triggerClass:\"x-form-arrow-trigger\",shadow:\"sides\",listAlign:\"tl-bl?\",maxHeight:300,minHeight:90,triggerAction:\"query\",minChars:4,typeAhead:false,queryDelay:500,pageSize:0,selectOnFocus:false,queryParam:\"query\",loadingText:\"Loading...\",resizable:false,handleHeight:8,editable:true,allQuery:\"\",mode:\"remote\",minListWidth:70,forceSelection:false,typeAheadDelay:250,lazyInit:true,initComponent:function(){Ext.form.ComboBox.superclass.initComponent.call(this);this.addEvents(\"expand\",\"collapse\",\"beforeselect\",\"select\",\"beforequery\");if(this.transform){this.allowDomMove=false;var C=Ext.getDom(this.transform);if(!this.hiddenName){this.hiddenName=C.name}if(!this.store){this.mode=\"local\";var G=[],D=C.options;for(var B=0,A=D.length;B<A;B++){var F=D[B];var E=(Ext.isIE?F.getAttributeNode(\"value\").specified:F.hasAttribute(\"value\"))?F.value:F.text;if(F.selected){this.value=E}G.push([E,F.text])}this.store=new Ext.data.SimpleStore({\"id\":0,fields:[\"value\",\"text\"],data:G});this.valueField=\"value\";this.displayField=\"text\"}C.name=Ext.id();if(!this.lazyRender){this.target=true;this.el=Ext.DomHelper.insertBefore(C,this.autoCreate||this.defaultAutoCreate);Ext.removeNode(C);this.render(this.el.parentNode)}else{Ext.removeNode(C)}}this.selectedIndex=-1;if(this.mode==\"local\"){if(this.initialConfig.queryDelay===undefined){this.queryDelay=10}if(this.initialConfig.minChars===undefined){this.minChars=0}}},onRender:function(B,A){Ext.form.ComboBox.superclass.onRender.call(this,B,A);if(this.hiddenName){this.hiddenField=this.el.insertSibling({tag:\"input\",type:\"hidden\",name:this.hiddenName,id:(this.hiddenId||this.hiddenName)},\"before\",true);this.hiddenField.value=this.hiddenValue!==undefined?this.hiddenValue:this.value!==undefined?this.value:\"\";this.el.dom.removeAttribute(\"name\")}if(Ext.isGecko){this.el.dom.setAttribute(\"autocomplete\",\"off\")}if(!this.lazyInit){this.initList()}else{this.on(\"focus\",this.initList,this,{single:true})}if(!this.editable){this.editable=true;this.setEditable(false)}},initList:function(){if(!this.list){var A=\"x-combo-list\";this.list=new Ext.Layer({shadow:this.shadow,cls:[A,this.listClass].join(\" \"),constrain:false});var B=this.listWidth||Math.max(this.wrap.getWidth(),this.minListWidth);this.list.setWidth(B);this.list.swallowEvent(\"mousewheel\");this.assetHeight=0;if(this.title){this.header=this.list.createChild({cls:A+\"-hd\",html:this.title});this.assetHeight+=this.header.getHeight()}this.innerList=this.list.createChild({cls:A+\"-inner\"});this.innerList.on(\"mouseover\",this.onViewOver,this);this.innerList.on(\"mousemove\",this.onViewMove,this);this.innerList.setWidth(B-this.list.getFrameWidth(\"lr\"));if(this.pageSize){this.footer=this.list.createChild({cls:A+\"-ft\"});this.pageTb=new Ext.PagingToolbar({store:this.store,pageSize:this.pageSize,renderTo:this.footer});this.assetHeight+=this.footer.getHeight()}if(!this.tpl){this.tpl=\"<tpl for=\\\".\\\"><div class=\\\"\"+A+\"-item\\\">{\"+this.displayField+\"}</div></tpl>\"}this.view=new Ext.DataView({applyTo:this.innerList,tpl:this.tpl,singleSelect:true,selectedClass:this.selectedClass,itemSelector:this.itemSelector||\".\"+A+\"-item\"});this.view.on(\"click\",this.onViewClick,this);this.bindStore(this.store,true);if(this.resizable){this.resizer=new Ext.Resizable(this.list,{pinned:true,handles:\"se\"});this.resizer.on(\"resize\",function(E,C,D){this.maxHeight=D-this.handleHeight-this.list.getFrameWidth(\"tb\")-this.assetHeight;this.listWidth=C;this.innerList.setWidth(C-this.list.getFrameWidth(\"lr\"));this.restrictHeight()},this);this[this.pageSize?\"footer\":\"innerList\"].setStyle(\"margin-bottom\",this.handleHeight+\"px\")}}},bindStore:function(A,B){if(this.store&&!B){this.store.un(\"beforeload\",this.onBeforeLoad,this);this.store.un(\"load\",this.onLoad,this);this.store.un(\"loadexception\",this.collapse,this);if(!A){this.store=null;if(this.view){this.view.setStore(null)}}}if(A){this.store=Ext.StoreMgr.lookup(A);this.store.on(\"beforeload\",this.onBeforeLoad,this);this.store.on(\"load\",this.onLoad,this);this.store.on(\"loadexception\",this.collapse,this);if(this.view){this.view.setStore(A)}}},initEvents:function(){Ext.form.ComboBox.superclass.initEvents.call(this);this.keyNav=new Ext.KeyNav(this.el,{\"up\":function(A){this.inKeyMode=true;this.selectPrev()},\"down\":function(A){if(!this.isExpanded()){this.onTriggerClick()}else{this.inKeyMode=true;this.selectNext()}},\"enter\":function(A){this.onViewClick();this.delayedCheck=true;this.unsetDelayCheck.defer(10,this)},\"esc\":function(A){this.collapse()},\"tab\":function(A){this.onViewClick(false);return true},scope:this,doRelay:function(C,B,A){if(A==\"down\"||this.scope.isExpanded()){return Ext.KeyNav.prototype.doRelay.apply(this,arguments)}return true},forceKeyDown:true});this.queryDelay=Math.max(this.queryDelay||10,this.mode==\"local\"?10:250);this.dqTask=new Ext.util.DelayedTask(this.initQuery,this);if(this.typeAhead){this.taTask=new Ext.util.DelayedTask(this.onTypeAhead,this)}if(this.editable!==false){this.el.on(\"keyup\",this.onKeyUp,this)}if(this.forceSelection){this.on(\"blur\",this.doForce,this)}},onDestroy:function(){if(this.view){this.view.el.removeAllListeners();this.view.el.remove();this.view.purgeListeners()}if(this.list){this.list.destroy()}this.bindStore(null);Ext.form.ComboBox.superclass.onDestroy.call(this)},unsetDelayCheck:function(){delete this.delayedCheck},fireKey:function(A){if(A.isNavKeyPress()&&!this.isExpanded()&&!this.delayedCheck){this.fireEvent(\"specialkey\",this,A)}},onResize:function(A,B){Ext.form.ComboBox.superclass.onResize.apply(this,arguments);if(this.list&&this.listWidth===undefined){var C=Math.max(A,this.minListWidth);this.list.setWidth(C);this.innerList.setWidth(C-this.list.getFrameWidth(\"lr\"))}},onEnable:function(){Ext.form.ComboBox.superclass.onEnable.apply(this,arguments);if(this.hiddenField){this.hiddenField.disabled=false}},onDisable:function(){Ext.form.ComboBox.superclass.onDisable.apply(this,arguments);if(this.hiddenField){this.hiddenField.disabled=true}},setEditable:function(A){if(A==this.editable){return }this.editable=A;if(!A){this.el.dom.setAttribute(\"readOnly\",true);this.el.on(\"mousedown\",this.onTriggerClick,this);this.el.addClass(\"x-combo-noedit\")}else{this.el.dom.setAttribute(\"readOnly\",false);this.el.un(\"mousedown\",this.onTriggerClick,this);this.el.removeClass(\"x-combo-noedit\")}},onBeforeLoad:function(){if(!this.hasFocus){return }this.innerList.update(this.loadingText?\"<div class=\\\"loading-indicator\\\">\"+this.loadingText+\"</div>\":\"\");this.restrictHeight();this.selectedIndex=-1},onLoad:function(){if(!this.hasFocus){return }if(this.store.getCount()>0){this.expand();this.restrictHeight();if(this.lastQuery==this.allQuery){if(this.editable){this.el.dom.select()}if(!this.selectByValue(this.value,true)){this.select(0,true)}}else{this.selectNext();if(this.typeAhead&&this.lastKey!=Ext.EventObject.BACKSPACE&&this.lastKey!=Ext.EventObject.DELETE){this.taTask.delay(this.typeAheadDelay)}}}else{this.onEmptyResults()}},onTypeAhead:function(){if(this.store.getCount()>0){var B=this.store.getAt(0);var C=B.data[this.displayField];var A=C.length;var D=this.getRawValue().length;if(D!=A){this.setRawValue(C);this.selectText(D,C.length)}}},onSelect:function(A,B){if(this.fireEvent(\"beforeselect\",this,A,B)!==false){this.setValue(A.data[this.valueField||this.displayField]);this.collapse();this.fireEvent(\"select\",this,A,B)}},getValue:function(){if(this.valueField){return typeof this.value!=\"undefined\"?this.value:\"\"}else{return Ext.form.ComboBox.superclass.getValue.call(this)}},clearValue:function(){if(this.hiddenField){this.hiddenField.value=\"\"}this.setRawValue(\"\");this.lastSelectionText=\"\";this.applyEmptyText();this.value=\"\"},setValue:function(A){var C=A;if(this.valueField){var B=this.findRecord(this.valueField,A);if(B){C=B.data[this.displayField]}else{if(this.valueNotFoundText!==undefined){C=this.valueNotFoundText}}}this.lastSelectionText=C;if(this.hiddenField){this.hiddenField.value=A}Ext.form.ComboBox.superclass.setValue.call(this,C);this.value=A},findRecord:function(C,B){var A;if(this.store.getCount()>0){this.store.each(function(D){if(D.data[C]==B){A=D;return false}})}return A},onViewMove:function(B,A){this.inKeyMode=false},onViewOver:function(D,B){if(this.inKeyMode){return }var C=this.view.findItemFromChild(B);if(C){var A=this.view.indexOf(C);this.select(A,false)}},onViewClick:function(B){var A=this.view.getSelectedIndexes()[0];var C=this.store.getAt(A);if(C){this.onSelect(C,A)}if(B!==false){this.el.focus()}},restrictHeight:function(){this.innerList.dom.style.height=\"\";var B=this.innerList.dom;var E=this.list.getFrameWidth(\"tb\")+(this.resizable?this.handleHeight:0)+this.assetHeight;var C=Math.max(B.clientHeight,B.offsetHeight,B.scrollHeight);var A=this.getPosition()[1]-Ext.getBody().getScroll().top;var F=Ext.lib.Dom.getViewHeight()-A-this.getSize().height;var D=Math.max(A,F,this.minHeight||0)-this.list.shadow.offset-E-2;C=Math.min(C,D,this.maxHeight);this.innerList.setHeight(C);this.list.beginUpdate();this.list.setHeight(C+E);this.list.alignTo(this.el,this.listAlign);this.list.endUpdate()},onEmptyResults:function(){this.collapse()},isExpanded:function(){return this.list&&this.list.isVisible()},selectByValue:function(A,C){if(A!==undefined&&A!==null){var B=this.findRecord(this.valueField||this.displayField,A);if(B){this.select(this.store.indexOf(B),C);return true}}return false},select:function(A,C){this.selectedIndex=A;this.view.select(A);if(C!==false){var B=this.view.getNode(A);if(B){this.innerList.scrollChildIntoView(B,false)}}},selectNext:function(){var A=this.store.getCount();if(A>0){if(this.selectedIndex==-1){this.select(0)}else{if(this.selectedIndex<A-1){this.select(this.selectedIndex+1)}}}},selectPrev:function(){var A=this.store.getCount();if(A>0){if(this.selectedIndex==-1){this.select(0)}else{if(this.selectedIndex!=0){this.select(this.selectedIndex-1)}}}},onKeyUp:function(A){if(this.editable!==false&&!A.isSpecialKey()){this.lastKey=A.getKey();this.dqTask.delay(this.queryDelay)}},validateBlur:function(){return !this.list||!this.list.isVisible()},initQuery:function(){this.doQuery(this.getRawValue())},doForce:function(){if(this.el.dom.value.length>0){this.el.dom.value=this.lastSelectionText===undefined?\"\":this.lastSelectionText;this.applyEmptyText()}},doQuery:function(C,B){if(C===undefined||C===null){C=\"\"}var A={query:C,forceAll:B,combo:this,cancel:false};if(this.fireEvent(\"beforequery\",A)===false||A.cancel){return false}C=A.query;B=A.forceAll;if(B===true||(C.length>=this.minChars)){if(this.lastQuery!==C){this.lastQuery=C;if(this.mode==\"local\"){this.selectedIndex=-1;if(B){this.store.clearFilter()}else{this.store.filter(this.displayField,C)}this.onLoad()}else{this.store.baseParams[this.queryParam]=C;this.store.load({params:this.getParams(C)});this.expand()}}else{this.selectedIndex=-1;this.onLoad()}}},getParams:function(A){var B={};if(this.pageSize){B.start=0;B.limit=this.pageSize}return B},collapse:function(){if(!this.isExpanded()){return }this.list.hide();Ext.getDoc().un(\"mousewheel\",this.collapseIf,this);Ext.getDoc().un(\"mousedown\",this.collapseIf,this);this.fireEvent(\"collapse\",this)},collapseIf:function(A){if(!A.within(this.wrap)&&!A.within(this.list)){this.collapse()}},expand:function(){if(this.isExpanded()||!this.hasFocus){return }this.list.alignTo(this.wrap,this.listAlign);this.list.show();this.innerList.setOverflow(\"auto\");Ext.getDoc().on(\"mousewheel\",this.collapseIf,this);Ext.getDoc().on(\"mousedown\",this.collapseIf,this);this.fireEvent(\"expand\",this)},onTriggerClick:function(){if(this.disabled){return }if(this.isExpanded()){this.collapse();this.el.focus()}else{this.onFocus({});if(this.triggerAction==\"all\"){this.doQuery(this.allQuery,true)}else{this.doQuery(this.getRawValue())}this.el.focus()}}});Ext.reg(\"combo\",Ext.form.ComboBox);\nExt.form.Checkbox=Ext.extend(Ext.form.Field,{focusClass:undefined,fieldClass:\"x-form-field\",checked:false,defaultAutoCreate:{tag:\"input\",type:\"checkbox\",autocomplete:\"off\"},initComponent:function(){Ext.form.Checkbox.superclass.initComponent.call(this);this.addEvents(\"check\")},onResize:function(){Ext.form.Checkbox.superclass.onResize.apply(this,arguments);if(!this.boxLabel){this.el.alignTo(this.wrap,\"c-c\")}},initEvents:function(){Ext.form.Checkbox.superclass.initEvents.call(this);this.el.on(\"click\",this.onClick,this);this.el.on(\"change\",this.onClick,this)},getResizeEl:function(){return this.wrap},getPositionEl:function(){return this.wrap},markInvalid:Ext.emptyFn,clearInvalid:Ext.emptyFn,onRender:function(B,A){Ext.form.Checkbox.superclass.onRender.call(this,B,A);if(this.inputValue!==undefined){this.el.dom.value=this.inputValue}this.wrap=this.el.wrap({cls:\"x-form-check-wrap\"});if(this.boxLabel){this.wrap.createChild({tag:\"label\",htmlFor:this.el.id,cls:\"x-form-cb-label\",html:this.boxLabel})}if(this.checked){this.setValue(true)}else{this.checked=this.el.dom.checked}},onDestroy:function(){if(this.wrap){this.wrap.remove()}Ext.form.Checkbox.superclass.onDestroy.call(this)},initValue:Ext.emptyFn,getValue:function(){if(this.rendered){return this.el.dom.checked}return false},onClick:function(){if(this.el.dom.checked!=this.checked){this.setValue(this.el.dom.checked)}},setValue:function(A){this.checked=(A===true||A===\"true\"||A==\"1\"||String(A).toLowerCase()==\"on\");if(this.el&&this.el.dom){this.el.dom.checked=this.checked;this.el.dom.defaultChecked=this.checked}this.fireEvent(\"check\",this,this.checked)}});Ext.reg(\"checkbox\",Ext.form.Checkbox);\nExt.form.Radio=Ext.extend(Ext.form.Checkbox,{inputType:\"radio\",markInvalid:Ext.emptyFn,clearInvalid:Ext.emptyFn,getGroupValue:function(){var A=this.el.up(\"form\")||Ext.getBody();var B=A.child(\"input[name=\"+this.el.dom.name+\"]:checked\",true);return B?B.value:null},onClick:function(){if(this.el.dom.checked!=this.checked){var B=this.el.up(\"form\")||Ext.getBody();var A=B.select(\"input[name=\"+this.el.dom.name+\"]\");A.each(function(C){if(C.dom.id==this.id){this.setValue(true)}else{Ext.getCmp(C.dom.id).setValue(false)}},this)}},setValue:function(A){if(typeof A==\"boolean\"){Ext.form.Radio.superclass.setValue.call(this,A)}else{var B=this.el.up(\"form\").child(\"input[name=\"+this.el.dom.name+\"][value=\"+A+\"]\",true);if(B){B.checked=true}}}});Ext.reg(\"radio\",Ext.form.Radio);\nExt.form.Hidden=Ext.extend(Ext.form.Field,{inputType:\"hidden\",onRender:function(){Ext.form.Hidden.superclass.onRender.apply(this,arguments)},initEvents:function(){this.originalValue=this.getValue()},setSize:Ext.emptyFn,setWidth:Ext.emptyFn,setHeight:Ext.emptyFn,setPosition:Ext.emptyFn,setPagePosition:Ext.emptyFn,markInvalid:Ext.emptyFn,clearInvalid:Ext.emptyFn});Ext.reg(\"hidden\",Ext.form.Hidden);\nExt.form.BasicForm=function(B,A){Ext.apply(this,A);this.items=new Ext.util.MixedCollection(false,function(C){return C.id||(C.id=Ext.id())});this.addEvents(\"beforeaction\",\"actionfailed\",\"actioncomplete\");if(B){this.initEl(B)}Ext.form.BasicForm.superclass.constructor.call(this)};Ext.extend(Ext.form.BasicForm,Ext.util.Observable,{timeout:30,activeAction:null,trackResetOnLoad:false,initEl:function(A){this.el=Ext.get(A);this.id=this.el.id||Ext.id();if(!this.standardSubmit){this.el.on(\"submit\",this.onSubmit,this)}this.el.addClass(\"x-form\")},getEl:function(){return this.el},onSubmit:function(A){A.stopEvent()},destroy:function(){this.items.each(function(A){Ext.destroy(A)});if(this.el){this.el.removeAllListeners();this.el.remove()}this.purgeListeners()},isValid:function(){var A=true;this.items.each(function(B){if(!B.validate()){A=false}});return A},isDirty:function(){var A=false;this.items.each(function(B){if(B.isDirty()){A=true;return false}});return A},doAction:function(B,A){if(typeof B==\"string\"){B=new Ext.form.Action.ACTION_TYPES[B](this,A)}if(this.fireEvent(\"beforeaction\",this,B)!==false){this.beforeAction(B);B.run.defer(100,B)}return this},submit:function(B){if(this.standardSubmit){var A=this.isValid();if(A){this.el.dom.submit()}return A}this.doAction(\"submit\",B);return this},load:function(A){this.doAction(\"load\",A);return this},updateRecord:function(B){B.beginEdit();var A=B.fields;A.each(function(C){var D=this.findField(C.name);if(D){B.set(C.name,D.getValue())}},this);B.endEdit();return this},loadRecord:function(A){this.setValues(A.data);return this},beforeAction:function(A){var B=A.options;if(B.waitMsg){if(this.waitMsgTarget===true){this.el.mask(B.waitMsg,\"x-mask-loading\")}else{if(this.waitMsgTarget){this.waitMsgTarget=Ext.get(this.waitMsgTarget);this.waitMsgTarget.mask(B.waitMsg,\"x-mask-loading\")}else{Ext.MessageBox.wait(B.waitMsg,B.waitTitle||this.waitTitle||\"Please Wait...\")}}}},afterAction:function(A,C){this.activeAction=null;var B=A.options;if(B.waitMsg){if(this.waitMsgTarget===true){this.el.unmask()}else{if(this.waitMsgTarget){this.waitMsgTarget.unmask()}else{Ext.MessageBox.updateProgress(1);Ext.MessageBox.hide()}}}if(C){if(B.reset){this.reset()}Ext.callback(B.success,B.scope,[this,A]);this.fireEvent(\"actioncomplete\",this,A)}else{Ext.callback(B.failure,B.scope,[this,A]);this.fireEvent(\"actionfailed\",this,A)}},findField:function(B){var A=this.items.get(B);if(!A){this.items.each(function(C){if(C.isFormField&&(C.dataIndex==B||C.id==B||C.getName()==B)){A=C;return false}})}return A||null},markInvalid:function(G){if(Ext.isArray(G)){for(var C=0,A=G.length;C<A;C++){var B=G[C];var D=this.findField(B.id);if(D){D.markInvalid(B.msg)}}}else{var E,F;for(F in G){if(typeof G[F]!=\"function\"&&(E=this.findField(F))){E.markInvalid(G[F])}}}return this},setValues:function(C){if(Ext.isArray(C)){for(var D=0,A=C.length;D<A;D++){var B=C[D];var E=this.findField(B.id);if(E){E.setValue(B.value);if(this.trackResetOnLoad){E.originalValue=E.getValue()}}}}else{var F,G;for(G in C){if(typeof C[G]!=\"function\"&&(F=this.findField(G))){F.setValue(C[G]);if(this.trackResetOnLoad){F.originalValue=F.getValue()}}}}return this},getValues:function(B){var A=Ext.lib.Ajax.serializeForm(this.el.dom);if(B===true){return A}return Ext.urlDecode(A)},clearInvalid:function(){this.items.each(function(A){A.clearInvalid()});return this},reset:function(){this.items.each(function(A){A.reset()});return this},add:function(){this.items.addAll(Array.prototype.slice.call(arguments,0));return this},remove:function(A){this.items.remove(A);return this},render:function(){this.items.each(function(A){if(A.isFormField&&!A.rendered&&document.getElementById(A.id)){A.applyToMarkup(A.id)}});return this},applyToFields:function(A){this.items.each(function(B){Ext.apply(B,A)});return this},applyIfToFields:function(A){this.items.each(function(B){Ext.applyIf(B,A)});return this}});Ext.BasicForm=Ext.form.BasicForm;\nExt.FormPanel=Ext.extend(Ext.Panel,{buttonAlign:\"center\",minButtonWidth:75,labelAlign:\"left\",monitorValid:false,monitorPoll:200,layout:\"form\",initComponent:function(){this.form=this.createForm();Ext.FormPanel.superclass.initComponent.call(this);this.addEvents(\"clientvalidation\");this.relayEvents(this.form,[\"beforeaction\",\"actionfailed\",\"actioncomplete\"])},createForm:function(){delete this.initialConfig.listeners;return new Ext.form.BasicForm(null,this.initialConfig)},initFields:function(){var C=this.form;var A=this;var B=function(D){if(D.doLayout&&D!=A){Ext.applyIf(D,{labelAlign:D.ownerCt.labelAlign,labelWidth:D.ownerCt.labelWidth,itemCls:D.ownerCt.itemCls});if(D.items){D.items.each(B)}}else{if(D.isFormField){C.add(D)}}};this.items.each(B)},getLayoutTarget:function(){return this.form.el},getForm:function(){return this.form},onRender:function(B,A){this.initFields();Ext.FormPanel.superclass.onRender.call(this,B,A);var C={tag:\"form\",method:this.method||\"POST\",id:this.formId||Ext.id()};if(this.fileUpload){C.enctype=\"multipart/form-data\"}this.form.initEl(this.body.createChild(C))},beforeDestroy:function(){Ext.FormPanel.superclass.beforeDestroy.call(this);Ext.destroy(this.form)},initEvents:function(){Ext.FormPanel.superclass.initEvents.call(this);this.items.on(\"remove\",this.onRemove,this);this.items.on(\"add\",this.onAdd,this);if(this.monitorValid){this.startMonitoring()}},onAdd:function(A,B){if(B.isFormField){this.form.add(B)}},onRemove:function(A){if(A.isFormField){Ext.destroy(A.container.up(\".x-form-item\"));this.form.remove(A)}},startMonitoring:function(){if(!this.bound){this.bound=true;Ext.TaskMgr.start({run:this.bindHandler,interval:this.monitorPoll||200,scope:this})}},stopMonitoring:function(){this.bound=false},load:function(){this.form.load.apply(this.form,arguments)},onDisable:function(){Ext.FormPanel.superclass.onDisable.call(this);if(this.form){this.form.items.each(function(){this.disable()})}},onEnable:function(){Ext.FormPanel.superclass.onEnable.call(this);if(this.form){this.form.items.each(function(){this.enable()})}},bindHandler:function(){if(!this.bound){return false}var D=true;this.form.items.each(function(E){if(!E.isValid(true)){D=false;return false}});if(this.buttons){for(var C=0,A=this.buttons.length;C<A;C++){var B=this.buttons[C];if(B.formBind===true&&B.disabled===D){B.setDisabled(!D)}}}this.fireEvent(\"clientvalidation\",this,D)}});Ext.reg(\"form\",Ext.FormPanel);Ext.form.FormPanel=Ext.FormPanel;\nExt.form.FieldSet=Ext.extend(Ext.Panel,{baseCls:\"x-fieldset\",layout:\"form\",onRender:function(B,A){if(!this.el){this.el=document.createElement(\"fieldset\");this.el.id=this.id;if(this.title||this.header||this.checkboxToggle){this.el.appendChild(document.createElement(\"legend\")).className=\"x-fieldset-header\"}}Ext.form.FieldSet.superclass.onRender.call(this,B,A);if(this.checkboxToggle){var C=typeof this.checkboxToggle==\"object\"?this.checkboxToggle:{tag:\"input\",type:\"checkbox\",name:this.checkboxName||this.id+\"-checkbox\"};this.checkbox=this.header.insertFirst(C);this.checkbox.dom.checked=!this.collapsed;this.checkbox.on(\"click\",this.onCheckClick,this)}},onCollapse:function(A,B){if(this.checkbox){this.checkbox.dom.checked=false}this.afterCollapse()},onExpand:function(A,B){if(this.checkbox){this.checkbox.dom.checked=true}this.afterExpand()},onCheckClick:function(){this[this.checkbox.dom.checked?\"expand\":\"collapse\"]()}});Ext.reg(\"fieldset\",Ext.form.FieldSet);\nExt.form.HtmlEditor=Ext.extend(Ext.form.Field,{enableFormat:true,enableFontSize:true,enableColors:true,enableAlignments:true,enableLists:true,enableSourceEdit:true,enableLinks:true,enableFont:true,createLinkText:\"Please enter the URL for the link:\",defaultLinkValue:\"http:/\"+\"/\",fontFamilies:[\"Arial\",\"Courier New\",\"Tahoma\",\"Times New Roman\",\"Verdana\"],defaultFont:\"tahoma\",validationEvent:false,deferHeight:true,initialized:false,activated:false,sourceEditMode:false,onFocus:Ext.emptyFn,iframePad:3,hideMode:\"offsets\",defaultAutoCreate:{tag:\"textarea\",style:\"width:500px;height:300px;\",autocomplete:\"off\"},initComponent:function(){this.addEvents(\"initialize\",\"activate\",\"beforesync\",\"beforepush\",\"sync\",\"push\",\"editmodechange\")},createFontOptions:function(){var D=[],B=this.fontFamilies,C,F;for(var E=0,A=B.length;E<A;E++){C=B[E];F=C.toLowerCase();D.push(\"<option value=\\\"\",F,\"\\\" style=\\\"font-family:\",C,\";\\\"\",(this.defaultFont==F?\" selected=\\\"true\\\">\":\">\"),C,\"</option>\")}return D.join(\"\")},createToolbar:function(C){function B(F,D,E){return{itemId:F,cls:\"x-btn-icon x-edit-\"+F,enableToggle:D!==false,scope:C,handler:E||C.relayBtnCmd,clickEvent:\"mousedown\",tooltip:C.buttonTips[F]||undefined,tabIndex:-1}}var A=new Ext.Toolbar({renderTo:this.wrap.dom.firstChild});A.el.on(\"click\",function(D){D.preventDefault()});if(this.enableFont&&!Ext.isSafari){this.fontSelect=A.el.createChild({tag:\"select\",cls:\"x-font-select\",html:this.createFontOptions()});this.fontSelect.on(\"change\",function(){var D=this.fontSelect.dom.value;this.relayCmd(\"fontname\",D);this.deferFocus()},this);A.add(this.fontSelect.dom,\"-\")}if(this.enableFormat){A.add(B(\"bold\"),B(\"italic\"),B(\"underline\"))}if(this.enableFontSize){A.add(\"-\",B(\"increasefontsize\",false,this.adjustFont),B(\"decreasefontsize\",false,this.adjustFont))}if(this.enableColors){A.add(\"-\",{itemId:\"forecolor\",cls:\"x-btn-icon x-edit-forecolor\",clickEvent:\"mousedown\",tooltip:C.buttonTips[\"forecolor\"]||undefined,tabIndex:-1,menu:new Ext.menu.ColorMenu({allowReselect:true,focus:Ext.emptyFn,value:\"000000\",plain:true,selectHandler:function(E,D){this.execCmd(\"forecolor\",Ext.isSafari||Ext.isIE?\"#\"+D:D);this.deferFocus()},scope:this,clickEvent:\"mousedown\"})},{itemId:\"backcolor\",cls:\"x-btn-icon x-edit-backcolor\",clickEvent:\"mousedown\",tooltip:C.buttonTips[\"backcolor\"]||undefined,tabIndex:-1,menu:new Ext.menu.ColorMenu({focus:Ext.emptyFn,value:\"FFFFFF\",plain:true,allowReselect:true,selectHandler:function(E,D){if(Ext.isGecko){this.execCmd(\"useCSS\",false);this.execCmd(\"hilitecolor\",D);this.execCmd(\"useCSS\",true);this.deferFocus()}else{this.execCmd(Ext.isOpera?\"hilitecolor\":\"backcolor\",Ext.isSafari||Ext.isIE?\"#\"+D:D);this.deferFocus()}},scope:this,clickEvent:\"mousedown\"})})}if(this.enableAlignments){A.add(\"-\",B(\"justifyleft\"),B(\"justifycenter\"),B(\"justifyright\"))}if(!Ext.isSafari){if(this.enableLinks){A.add(\"-\",B(\"createlink\",false,this.createLink))}if(this.enableLists){A.add(\"-\",B(\"insertorderedlist\"),B(\"insertunorderedlist\"))}if(this.enableSourceEdit){A.add(\"-\",B(\"sourceedit\",true,function(D){this.toggleSourceEdit(D.pressed)}))}}this.tb=A},getDocMarkup:function(){return\"<html><head><style type=\\\"text/css\\\">body{border:0;margin:0;padding:3px;height:98%;cursor:text;}</style></head><body></body></html>\"},getEditorBody:function(){return this.doc.body||this.doc.documentElement},onRender:function(C,A){Ext.form.HtmlEditor.superclass.onRender.call(this,C,A);this.el.dom.style.border=\"0 none\";this.el.dom.setAttribute(\"tabIndex\",-1);this.el.addClass(\"x-hidden\");if(Ext.isIE){this.el.applyStyles(\"margin-top:-1px;margin-bottom:-1px;\")}this.wrap=this.el.wrap({cls:\"x-html-editor-wrap\",cn:{cls:\"x-html-editor-tb\"}});this.createToolbar(this);this.tb.items.each(function(E){if(E.itemId!=\"sourceedit\"){E.disable()}});var D=document.createElement(\"iframe\");D.name=Ext.id();D.frameBorder=\"no\";D.src=(Ext.SSL_SECURE_URL||\"javascript:false\");this.wrap.dom.appendChild(D);this.iframe=D;if(Ext.isIE){D.contentWindow.document.designMode=\"on\";this.doc=D.contentWindow.document;this.win=D.contentWindow}else{this.doc=(D.contentDocument||window.frames[D.name].document);this.win=window.frames[D.name];this.doc.designMode=\"on\"}this.doc.open();this.doc.write(this.getDocMarkup());this.doc.close();var B={run:function(){if(this.doc.body||this.doc.readyState==\"complete\"){Ext.TaskMgr.stop(B);this.doc.designMode=\"on\";this.initEditor.defer(10,this)}},interval:10,duration:10000,scope:this};Ext.TaskMgr.start(B);if(!this.width){this.setSize(this.el.getSize())}},onResize:function(B,C){Ext.form.HtmlEditor.superclass.onResize.apply(this,arguments);if(this.el&&this.iframe){if(typeof B==\"number\"){var D=B-this.wrap.getFrameWidth(\"lr\");this.el.setWidth(this.adjustWidth(\"textarea\",D));this.iframe.style.width=D+\"px\"}if(typeof C==\"number\"){var A=C-this.wrap.getFrameWidth(\"tb\")-this.tb.el.getHeight();this.el.setHeight(this.adjustWidth(\"textarea\",A));this.iframe.style.height=A+\"px\";if(this.doc){this.getEditorBody().style.height=(A-(this.iframePad*2))+\"px\"}}}},toggleSourceEdit:function(A){if(A===undefined){A=!this.sourceEditMode}this.sourceEditMode=A===true;var C=this.tb.items.get(\"sourceedit\");if(C.pressed!==this.sourceEditMode){C.toggle(this.sourceEditMode);return }if(this.sourceEditMode){this.tb.items.each(function(D){if(D.itemId!=\"sourceedit\"){D.disable()}});this.syncValue();this.iframe.className=\"x-hidden\";this.el.removeClass(\"x-hidden\");this.el.dom.removeAttribute(\"tabIndex\");this.el.focus()}else{if(this.initialized){this.tb.items.each(function(D){D.enable()})}this.pushValue();this.iframe.className=\"\";this.el.addClass(\"x-hidden\");this.el.dom.setAttribute(\"tabIndex\",-1);this.deferFocus()}var B=this.lastSize;if(B){delete this.lastSize;this.setSize(B)}this.fireEvent(\"editmodechange\",this,this.sourceEditMode)},createLink:function(){var A=prompt(this.createLinkText,this.defaultLinkValue);if(A&&A!=\"http:/\"+\"/\"){this.relayCmd(\"createlink\",A)}},adjustSize:Ext.BoxComponent.prototype.adjustSize,getResizeEl:function(){return this.wrap},getPositionEl:function(){return this.wrap},initEvents:function(){this.originalValue=this.getValue()},markInvalid:Ext.emptyFn,clearInvalid:Ext.emptyFn,setValue:function(A){Ext.form.HtmlEditor.superclass.setValue.call(this,A);this.pushValue()},cleanHtml:function(A){A=String(A);if(A.length>5){if(Ext.isSafari){A=A.replace(/\\sclass=\"(?:Apple-style-span|khtml-block-placeholder)\"/gi,\"\")}}if(A==\"&nbsp;\"){A=\"\"}return A},syncValue:function(){if(this.initialized){var D=this.getEditorBody();var C=D.innerHTML;if(Ext.isSafari){var B=D.getAttribute(\"style\");var A=B.match(/text-align:(.*?);/i);if(A&&A[1]){C=\"<div style=\\\"\"+A[0]+\"\\\">\"+C+\"</div>\"}}C=this.cleanHtml(C);if(this.fireEvent(\"beforesync\",this,C)!==false){this.el.dom.value=C;this.fireEvent(\"sync\",this,C)}}},pushValue:function(){if(this.initialized){var A=this.el.dom.value;if(!this.activated&&A.length<1){A=\"&nbsp;\"}if(this.fireEvent(\"beforepush\",this,A)!==false){this.getEditorBody().innerHTML=A;this.fireEvent(\"push\",this,A)}}},deferFocus:function(){this.focus.defer(10,this)},focus:function(){if(this.win&&!this.sourceEditMode){this.win.focus()}else{this.el.focus()}},initEditor:function(){var B=this.getEditorBody();var A=this.el.getStyles(\"font-size\",\"font-family\",\"background-image\",\"background-repeat\");A[\"background-attachment\"]=\"fixed\";B.bgProperties=\"fixed\";Ext.DomHelper.applyStyles(B,A);Ext.EventManager.on(this.doc,{\"mousedown\":this.onEditorEvent,\"dblclick\":this.onEditorEvent,\"click\":this.onEditorEvent,\"keyup\":this.onEditorEvent,buffer:100,scope:this});if(Ext.isGecko){Ext.EventManager.on(this.doc,\"keypress\",this.applyCommand,this)}if(Ext.isIE||Ext.isSafari||Ext.isOpera){Ext.EventManager.on(this.doc,\"keydown\",this.fixKeys,this)}this.initialized=true;this.fireEvent(\"initialize\",this);this.pushValue()},onDestroy:function(){if(this.rendered){this.tb.items.each(function(A){if(A.menu){A.menu.removeAll();if(A.menu.el){A.menu.el.destroy()}}A.destroy()});this.wrap.dom.innerHTML=\"\";this.wrap.remove()}},onFirstFocus:function(){this.activated=true;this.tb.items.each(function(D){D.enable()});if(Ext.isGecko){this.win.focus();var A=this.win.getSelection();if(!A.focusNode||A.focusNode.nodeType!=3){var B=A.getRangeAt(0);B.selectNodeContents(this.getEditorBody());B.collapse(true);this.deferFocus()}try{this.execCmd(\"useCSS\",true);this.execCmd(\"styleWithCSS\",false)}catch(C){}}this.fireEvent(\"activate\",this)},adjustFont:function(B){var C=B.itemId==\"increasefontsize\"?1:-1;var A=parseInt(this.doc.queryCommandValue(\"FontSize\")||2,10);if(Ext.isSafari3||Ext.isAir){if(A<=10){A=1+C}else{if(A<=13){A=2+C}else{if(A<=16){A=3+C}else{if(A<=18){A=4+C}else{if(A<=24){A=5+C}else{A=6+C}}}}}A=A.constrain(1,6)}else{if(Ext.isSafari){C*=2}A=Math.max(1,A+C)+(Ext.isSafari?\"px\":0)}this.execCmd(\"FontSize\",A)},onEditorEvent:function(A){this.updateToolbar()},updateToolbar:function(){if(!this.activated){this.onFirstFocus();return }var B=this.tb.items.map,C=this.doc;if(this.enableFont&&!Ext.isSafari){var A=(this.doc.queryCommandValue(\"FontName\")||this.defaultFont).toLowerCase();if(A!=this.fontSelect.dom.value){this.fontSelect.dom.value=A}}if(this.enableFormat){B.bold.toggle(C.queryCommandState(\"bold\"));B.italic.toggle(C.queryCommandState(\"italic\"));B.underline.toggle(C.queryCommandState(\"underline\"))}if(this.enableAlignments){B.justifyleft.toggle(C.queryCommandState(\"justifyleft\"));B.justifycenter.toggle(C.queryCommandState(\"justifycenter\"));B.justifyright.toggle(C.queryCommandState(\"justifyright\"))}if(!Ext.isSafari&&this.enableLists){B.insertorderedlist.toggle(C.queryCommandState(\"insertorderedlist\"));B.insertunorderedlist.toggle(C.queryCommandState(\"insertunorderedlist\"))}Ext.menu.MenuMgr.hideAll();this.syncValue()},relayBtnCmd:function(A){this.relayCmd(A.itemId)},relayCmd:function(B,A){this.win.focus();this.execCmd(B,A);this.updateToolbar();this.deferFocus()},execCmd:function(B,A){this.doc.execCommand(B,false,A===undefined?null:A);this.syncValue()},applyCommand:function(B){if(B.ctrlKey){var C=B.getCharCode(),A;if(C>0){C=String.fromCharCode(C);switch(C){case\"b\":A=\"bold\";break;case\"i\":A=\"italic\";break;case\"u\":A=\"underline\";break}if(A){this.win.focus();this.execCmd(A);this.deferFocus();B.preventDefault()}}}},insertAtCursor:function(B){if(!this.activated){return }if(Ext.isIE){this.win.focus();var A=this.doc.selection.createRange();if(A){A.collapse(true);A.pasteHTML(B);this.syncValue();this.deferFocus()}}else{if(Ext.isGecko||Ext.isOpera){this.win.focus();this.execCmd(\"InsertHTML\",B);this.deferFocus()}else{if(Ext.isSafari){this.execCmd(\"InsertText\",B);this.deferFocus()}}}},fixKeys:function(){if(Ext.isIE){return function(D){var A=D.getKey(),B;if(A==D.TAB){D.stopEvent();B=this.doc.selection.createRange();if(B){B.collapse(true);B.pasteHTML(\"&nbsp;&nbsp;&nbsp;&nbsp;\");this.deferFocus()}}else{if(A==D.ENTER){B=this.doc.selection.createRange();if(B){var C=B.parentElement();if(!C||C.tagName.toLowerCase()!=\"li\"){D.stopEvent();B.pasteHTML(\"<br />\");B.collapse(false);B.select()}}}}}}else{if(Ext.isOpera){return function(B){var A=B.getKey();if(A==B.TAB){B.stopEvent();this.win.focus();this.execCmd(\"InsertHTML\",\"&nbsp;&nbsp;&nbsp;&nbsp;\");this.deferFocus()}}}else{if(Ext.isSafari){return function(B){var A=B.getKey();if(A==B.TAB){B.stopEvent();this.execCmd(\"InsertText\",\"\\t\");this.deferFocus()}}}}}}(),getToolbar:function(){return this.tb},buttonTips:{bold:{title:\"Bold (Ctrl+B)\",text:\"Make the selected text bold.\",cls:\"x-html-editor-tip\"},italic:{title:\"Italic (Ctrl+I)\",text:\"Make the selected text italic.\",cls:\"x-html-editor-tip\"},underline:{title:\"Underline (Ctrl+U)\",text:\"Underline the selected text.\",cls:\"x-html-editor-tip\"},increasefontsize:{title:\"Grow Text\",text:\"Increase the font size.\",cls:\"x-html-editor-tip\"},decreasefontsize:{title:\"Shrink Text\",text:\"Decrease the font size.\",cls:\"x-html-editor-tip\"},backcolor:{title:\"Text Highlight Color\",text:\"Change the background color of the selected text.\",cls:\"x-html-editor-tip\"},forecolor:{title:\"Font Color\",text:\"Change the color of the selected text.\",cls:\"x-html-editor-tip\"},justifyleft:{title:\"Align Text Left\",text:\"Align text to the left.\",cls:\"x-html-editor-tip\"},justifycenter:{title:\"Center Text\",text:\"Center text in the editor.\",cls:\"x-html-editor-tip\"},justifyright:{title:\"Align Text Right\",text:\"Align text to the right.\",cls:\"x-html-editor-tip\"},insertunorderedlist:{title:\"Bullet List\",text:\"Start a bulleted list.\",cls:\"x-html-editor-tip\"},insertorderedlist:{title:\"Numbered List\",text:\"Start a numbered list.\",cls:\"x-html-editor-tip\"},createlink:{title:\"Hyperlink\",text:\"Make the selected text a hyperlink.\",cls:\"x-html-editor-tip\"},sourceedit:{title:\"Source Edit\",text:\"Switch to source editing mode.\",cls:\"x-html-editor-tip\"}}});Ext.reg(\"htmleditor\",Ext.form.HtmlEditor);\nExt.form.TimeField=Ext.extend(Ext.form.ComboBox,{minValue:null,maxValue:null,minText:\"The time in this field must be equal to or after {0}\",maxText:\"The time in this field must be equal to or before {0}\",invalidText:\"{0} is not a valid time\",format:\"g:i A\",altFormats:\"g:ia|g:iA|g:i a|g:i A|h:i|g:i|H:i|ga|ha|gA|h a|g a|g A|gi|hi|gia|hia|g|H\",increment:15,mode:\"local\",triggerAction:\"all\",typeAhead:false,initComponent:function(){Ext.form.TimeField.superclass.initComponent.call(this);if(typeof this.minValue==\"string\"){this.minValue=this.parseDate(this.minValue)}if(typeof this.maxValue==\"string\"){this.maxValue=this.parseDate(this.maxValue)}if(!this.store){var B=this.parseDate(this.minValue);if(!B){B=new Date().clearTime()}var A=this.parseDate(this.maxValue);if(!A){A=new Date().clearTime().add(\"mi\",(24*60)-1)}var C=[];while(B<=A){C.push([B.dateFormat(this.format)]);B=B.add(\"mi\",this.increment)}this.store=new Ext.data.SimpleStore({fields:[\"text\"],data:C});this.displayField=\"text\"}},getValue:function(){var A=Ext.form.TimeField.superclass.getValue.call(this);return this.formatDate(this.parseDate(A))||\"\"},setValue:function(A){Ext.form.TimeField.superclass.setValue.call(this,this.formatDate(this.parseDate(A)))},validateValue:Ext.form.DateField.prototype.validateValue,parseDate:Ext.form.DateField.prototype.parseDate,formatDate:Ext.form.DateField.prototype.formatDate,beforeBlur:function(){var A=this.parseDate(this.getRawValue());if(A){this.setValue(A.dateFormat(this.format))}}});Ext.reg(\"timefield\",Ext.form.TimeField);\nExt.form.Label=Ext.extend(Ext.BoxComponent,{onRender:function(B,A){if(!this.el){this.el=document.createElement(\"label\");this.el.innerHTML=this.text?Ext.util.Format.htmlEncode(this.text):(this.html||\"\");if(this.forId){this.el.setAttribute(\"htmlFor\",this.forId)}}Ext.form.Label.superclass.onRender.call(this,B,A)}});Ext.reg(\"label\",Ext.form.Label);\nExt.form.Action=function(B,A){this.form=B;this.options=A||{}};Ext.form.Action.CLIENT_INVALID=\"client\";Ext.form.Action.SERVER_INVALID=\"server\";Ext.form.Action.CONNECT_FAILURE=\"connect\";Ext.form.Action.LOAD_FAILURE=\"load\";Ext.form.Action.prototype={type:\"default\",run:function(A){},success:function(A){},handleResponse:function(A){},failure:function(A){this.response=A;this.failureType=Ext.form.Action.CONNECT_FAILURE;this.form.afterAction(this,false)},processResponse:function(A){this.response=A;if(!A.responseText){return true}this.result=this.handleResponse(A);return this.result},getUrl:function(C){var A=this.options.url||this.form.url||this.form.el.dom.action;if(C){var B=this.getParams();if(B){A+=(A.indexOf(\"?\")!=-1?\"&\":\"?\")+B}}return A},getMethod:function(){return(this.options.method||this.form.method||this.form.el.dom.method||\"POST\").toUpperCase()},getParams:function(){var A=this.form.baseParams;var B=this.options.params;if(B){if(typeof B==\"object\"){B=Ext.urlEncode(Ext.applyIf(B,A))}else{if(typeof B==\"string\"&&A){B+=\"&\"+Ext.urlEncode(A)}}}else{if(A){B=Ext.urlEncode(A)}}return B},createCallback:function(A){var A=A||{};return{success:this.success,failure:this.failure,scope:this,timeout:(A.timeout*1000)||(this.form.timeout*1000),upload:this.form.fileUpload?this.success:undefined}}};Ext.form.Action.Submit=function(B,A){Ext.form.Action.Submit.superclass.constructor.call(this,B,A)};Ext.extend(Ext.form.Action.Submit,Ext.form.Action,{type:\"submit\",run:function(){var B=this.options;var C=this.getMethod();var A=C==\"POST\";if(B.clientValidation===false||this.form.isValid()){Ext.Ajax.request(Ext.apply(this.createCallback(B),{form:this.form.el.dom,url:this.getUrl(!A),method:C,params:A?this.getParams():null,isUpload:this.form.fileUpload}))}else{if(B.clientValidation!==false){this.failureType=Ext.form.Action.CLIENT_INVALID;this.form.afterAction(this,false)}}},success:function(B){var A=this.processResponse(B);if(A===true||A.success){this.form.afterAction(this,true);return }if(A.errors){this.form.markInvalid(A.errors);this.failureType=Ext.form.Action.SERVER_INVALID}this.form.afterAction(this,false)},handleResponse:function(C){if(this.form.errorReader){var B=this.form.errorReader.read(C);var F=[];if(B.records){for(var D=0,A=B.records.length;D<A;D++){var E=B.records[D];F[D]=E.data}}if(F.length<1){F=null}return{success:B.success,errors:F}}return Ext.decode(C.responseText)}});Ext.form.Action.Load=function(B,A){Ext.form.Action.Load.superclass.constructor.call(this,B,A);this.reader=this.form.reader};Ext.extend(Ext.form.Action.Load,Ext.form.Action,{type:\"load\",run:function(){Ext.Ajax.request(Ext.apply(this.createCallback(this.options),{method:this.getMethod(),url:this.getUrl(false),params:this.getParams()}))},success:function(B){var A=this.processResponse(B);if(A===true||!A.success||!A.data){this.failureType=Ext.form.Action.LOAD_FAILURE;this.form.afterAction(this,false);return }this.form.clearInvalid();this.form.setValues(A.data);this.form.afterAction(this,true)},handleResponse:function(B){if(this.form.reader){var A=this.form.reader.read(B);var C=A.records&&A.records[0]?A.records[0].data:null;return{success:A.success,data:C}}return Ext.decode(B.responseText)}});Ext.form.Action.ACTION_TYPES={\"load\":Ext.form.Action.Load,\"submit\":Ext.form.Action.Submit};\nExt.form.VTypes=function(){var C=/^[a-zA-Z_]+$/;var D=/^[a-zA-Z0-9_]+$/;var B=/^([\\w]+)(.[\\w]+)*@([\\w-]+\\.){1,5}([A-Za-z]){2,4}$/;var A=/(((https?)|(ftp)):\\/\\/([\\-\\w]+\\.)+\\w{2,3}(\\/[%\\-\\w]+(\\.\\w{2,})?)*(([\\w\\-\\.\\?\\\\\\/+@&#;`~=%!]*)(\\.\\w{2,})?)*\\/?)/i;return{\"email\":function(E){return B.test(E)},\"emailText\":\"This field should be an e-mail address in the format \\\"user@domain.com\\\"\",\"emailMask\":/[a-z0-9_\\.\\-@]/i,\"url\":function(E){return A.test(E)},\"urlText\":\"This field should be a URL in the format \\\"http:/\"+\"/www.domain.com\\\"\",\"alpha\":function(E){return C.test(E)},\"alphaText\":\"This field should only contain letters and _\",\"alphaMask\":/[a-z_]/i,\"alphanum\":function(E){return D.test(E)},\"alphanumText\":\"This field should only contain letters, numbers and _\",\"alphanumMask\":/[a-z0-9_]/i}}();\nExt.grid.GridPanel=Ext.extend(Ext.Panel,{ddText:\"{0} selected row{1}\",minColumnWidth:25,trackMouseOver:true,enableDragDrop:false,enableColumnMove:true,enableColumnHide:true,enableHdMenu:true,stripeRows:false,autoExpandColumn:false,autoExpandMin:50,autoExpandMax:1000,view:null,loadMask:false,rendered:false,viewReady:false,stateEvents:[\"columnmove\",\"columnresize\",\"sortchange\"],initComponent:function(){Ext.grid.GridPanel.superclass.initComponent.call(this);this.autoScroll=false;this.autoWidth=false;if(Ext.isArray(this.columns)){this.colModel=new Ext.grid.ColumnModel(this.columns);delete this.columns}if(this.ds){this.store=this.ds;delete this.ds}if(this.cm){this.colModel=this.cm;delete this.cm}if(this.sm){this.selModel=this.sm;delete this.sm}this.store=Ext.StoreMgr.lookup(this.store);this.addEvents(\"click\",\"dblclick\",\"contextmenu\",\"mousedown\",\"mouseup\",\"mouseover\",\"mouseout\",\"keypress\",\"keydown\",\"cellmousedown\",\"rowmousedown\",\"headermousedown\",\"cellclick\",\"celldblclick\",\"rowclick\",\"rowdblclick\",\"headerclick\",\"headerdblclick\",\"rowcontextmenu\",\"cellcontextmenu\",\"headercontextmenu\",\"bodyscroll\",\"columnresize\",\"columnmove\",\"sortchange\")},onRender:function(C,A){Ext.grid.GridPanel.superclass.onRender.apply(this,arguments);var D=this.body;this.el.addClass(\"x-grid-panel\");var B=this.getView();B.init(this);D.on(\"mousedown\",this.onMouseDown,this);D.on(\"click\",this.onClick,this);D.on(\"dblclick\",this.onDblClick,this);D.on(\"contextmenu\",this.onContextMenu,this);D.on(\"keydown\",this.onKeyDown,this);this.relayEvents(D,[\"mousedown\",\"mouseup\",\"mouseover\",\"mouseout\",\"keypress\"]);this.getSelectionModel().init(this);this.view.render()},initEvents:function(){Ext.grid.GridPanel.superclass.initEvents.call(this);if(this.loadMask){this.loadMask=new Ext.LoadMask(this.bwrap,Ext.apply({store:this.store},this.loadMask))}},initStateEvents:function(){Ext.grid.GridPanel.superclass.initStateEvents.call(this);this.colModel.on(\"hiddenchange\",this.saveState,this,{delay:100})},applyState:function(F){var B=this.colModel;var E=F.columns;if(E){for(var C=0,A=E.length;C<A;C++){var D=E[C];var H=B.getColumnById(D.id);if(H){H.hidden=D.hidden;H.width=D.width;var G=B.getIndexById(D.id);if(G!=C){B.moveColumn(G,C)}}}}if(F.sort){this.store[this.store.remoteSort?\"setDefaultSort\":\"sort\"](F.sort.field,F.sort.direction)}},getState:function(){var C={columns:[]};for(var B=0,D;D=this.colModel.config[B];B++){C.columns[B]={id:D.id,width:D.width};if(D.hidden){C.columns[B].hidden=true}}var A=this.store.getSortState();if(A){C.sort=A}return C},afterRender:function(){Ext.grid.GridPanel.superclass.afterRender.call(this);this.view.layout();this.viewReady=true},reconfigure:function(A,B){if(this.loadMask){this.loadMask.destroy();this.loadMask=new Ext.LoadMask(this.bwrap,Ext.apply({store:A},this.initialConfig.loadMask))}this.view.bind(A,B);this.store=A;this.colModel=B;if(this.rendered){this.view.refresh(true)}},onKeyDown:function(A){this.fireEvent(\"keydown\",A)},onDestroy:function(){if(this.rendered){if(this.loadMask){this.loadMask.destroy()}var A=this.body;A.removeAllListeners();this.view.destroy();A.update(\"\")}this.colModel.purgeListeners();Ext.grid.GridPanel.superclass.onDestroy.call(this)},processEvent:function(C,E){this.fireEvent(C,E);var D=E.getTarget();var B=this.view;var G=B.findHeaderIndex(D);if(G!==false){this.fireEvent(\"header\"+C,this,G,E)}else{var F=B.findRowIndex(D);var A=B.findCellIndex(D);if(F!==false){this.fireEvent(\"row\"+C,this,F,E);if(A!==false){this.fireEvent(\"cell\"+C,this,F,A,E)}}}},onClick:function(A){this.processEvent(\"click\",A)},onMouseDown:function(A){this.processEvent(\"mousedown\",A)},onContextMenu:function(B,A){this.processEvent(\"contextmenu\",B)},onDblClick:function(A){this.processEvent(\"dblclick\",A)},walkCells:function(J,C,B,E,I){var H=this.colModel,F=H.getColumnCount();var A=this.store,G=A.getCount(),D=true;if(B<0){if(C<0){J--;D=false}while(J>=0){if(!D){C=F-1}D=false;while(C>=0){if(E.call(I||this,J,C,H)===true){return[J,C]}C--}J--}}else{if(C>=F){J++;D=false}while(J<G){if(!D){C=0}D=false;while(C<F){if(E.call(I||this,J,C,H)===true){return[J,C]}C++}J++}}return null},getSelections:function(){return this.selModel.getSelections()},onResize:function(){Ext.grid.GridPanel.superclass.onResize.apply(this,arguments);if(this.viewReady){this.view.layout()}},getGridEl:function(){return this.body},stopEditing:function(){},getSelectionModel:function(){if(!this.selModel){this.selModel=new Ext.grid.RowSelectionModel(this.disableSelection?{selectRow:Ext.emptyFn}:null)}return this.selModel},getStore:function(){return this.store},getColumnModel:function(){return this.colModel},getView:function(){if(!this.view){this.view=new Ext.grid.GridView(this.viewConfig)}return this.view},getDragDropText:function(){var A=this.selModel.getCount();return String.format(this.ddText,A,A==1?\"\":\"s\")}});Ext.reg(\"grid\",Ext.grid.GridPanel);\nExt.grid.GridView=function(A){Ext.apply(this,A);this.addEvents(\"beforerowremoved\",\"beforerowsinserted\",\"beforerefresh\",\"rowremoved\",\"rowsinserted\",\"rowupdated\",\"refresh\");Ext.grid.GridView.superclass.constructor.call(this)};Ext.extend(Ext.grid.GridView,Ext.util.Observable,{scrollOffset:19,autoFill:false,forceFit:false,sortClasses:[\"sort-asc\",\"sort-desc\"],sortAscText:\"Sort Ascending\",sortDescText:\"Sort Descending\",columnsText:\"Columns\",borderWidth:2,initTemplates:function(){var C=this.templates||{};if(!C.master){C.master=new Ext.Template(\"<div class=\\\"x-grid3\\\" hidefocus=\\\"true\\\">\",\"<div class=\\\"x-grid3-viewport\\\">\",\"<div class=\\\"x-grid3-header\\\"><div class=\\\"x-grid3-header-inner\\\"><div class=\\\"x-grid3-header-offset\\\">{header}</div></div><div class=\\\"x-clear\\\"></div></div>\",\"<div class=\\\"x-grid3-scroller\\\"><div class=\\\"x-grid3-body\\\">{body}</div><a href=\\\"#\\\" class=\\\"x-grid3-focus\\\" tabIndex=\\\"-1\\\"></a></div>\",\"</div>\",\"<div class=\\\"x-grid3-resize-marker\\\">&#160;</div>\",\"<div class=\\\"x-grid3-resize-proxy\\\">&#160;</div>\",\"</div>\")}if(!C.header){C.header=new Ext.Template(\"<table border=\\\"0\\\" cellspacing=\\\"0\\\" cellpadding=\\\"0\\\" style=\\\"{tstyle}\\\">\",\"<thead><tr class=\\\"x-grid3-hd-row\\\">{cells}</tr></thead>\",\"</table>\")}if(!C.hcell){C.hcell=new Ext.Template(\"<td class=\\\"x-grid3-hd x-grid3-cell x-grid3-td-{id}\\\" style=\\\"{style}\\\"><div {tooltip} {attr} class=\\\"x-grid3-hd-inner x-grid3-hd-{id}\\\" unselectable=\\\"on\\\" style=\\\"{istyle}\\\">\",this.grid.enableHdMenu?\"<a class=\\\"x-grid3-hd-btn\\\" href=\\\"#\\\"></a>\":\"\",\"{value}<img class=\\\"x-grid3-sort-icon\\\" src=\\\"\",Ext.BLANK_IMAGE_URL,\"\\\" />\",\"</div></td>\")}if(!C.body){C.body=new Ext.Template(\"{rows}\")}if(!C.row){C.row=new Ext.Template(\"<div class=\\\"x-grid3-row {alt}\\\" style=\\\"{tstyle}\\\"><table class=\\\"x-grid3-row-table\\\" border=\\\"0\\\" cellspacing=\\\"0\\\" cellpadding=\\\"0\\\" style=\\\"{tstyle}\\\">\",\"<tbody><tr>{cells}</tr>\",(this.enableRowBody?\"<tr class=\\\"x-grid3-row-body-tr\\\" style=\\\"{bodyStyle}\\\"><td colspan=\\\"{cols}\\\" class=\\\"x-grid3-body-cell\\\" tabIndex=\\\"0\\\" hidefocus=\\\"on\\\"><div class=\\\"x-grid3-row-body\\\">{body}</div></td></tr>\":\"\"),\"</tbody></table></div>\")}if(!C.cell){C.cell=new Ext.Template(\"<td class=\\\"x-grid3-col x-grid3-cell x-grid3-td-{id} {css}\\\" style=\\\"{style}\\\" tabIndex=\\\"0\\\" {cellAttr}>\",\"<div class=\\\"x-grid3-cell-inner x-grid3-col-{id}\\\" unselectable=\\\"on\\\" {attr}>{value}</div>\",\"</td>\")}for(var A in C){var B=C[A];if(B&&typeof B.compile==\"function\"&&!B.compiled){B.disableFormats=true;B.compile()}}this.templates=C;this.tdClass=\"x-grid3-cell\";this.cellSelector=\"td.x-grid3-cell\";this.hdCls=\"x-grid3-hd\";this.rowSelector=\"div.x-grid3-row\";this.colRe=new RegExp(\"x-grid3-td-([^\\\\s]+)\",\"\")},fly:function(A){if(!this._flyweight){this._flyweight=new Ext.Element.Flyweight(document.body)}this._flyweight.dom=A;return this._flyweight},getEditorParent:function(A){return this.scroller.dom},initElements:function(){var C=Ext.Element;var B=this.grid.getGridEl().dom.firstChild;var A=B.childNodes;this.el=new C(B);this.mainWrap=new C(A[0]);this.mainHd=new C(this.mainWrap.dom.firstChild);if(this.grid.hideHeaders){this.mainHd.setDisplayed(false)}this.innerHd=this.mainHd.dom.firstChild;this.scroller=new C(this.mainWrap.dom.childNodes[1]);if(this.forceFit){this.scroller.setStyle(\"overflow-x\",\"hidden\")}this.mainBody=new C(this.scroller.dom.firstChild);this.focusEl=new C(this.scroller.dom.childNodes[1]);this.focusEl.swallowEvent(\"click\",true);this.resizeMarker=new C(A[1]);this.resizeProxy=new C(A[2])},getRows:function(){return this.hasRows()?this.mainBody.dom.childNodes:[]},findCell:function(A){if(!A){return false}return this.fly(A).findParent(this.cellSelector,3)},findCellIndex:function(C,B){var A=this.findCell(C);if(A&&(!B||this.fly(A).hasClass(B))){return this.getCellIndex(A)}return false},getCellIndex:function(B){if(B){var A=B.className.match(this.colRe);if(A&&A[1]){return this.cm.getIndexById(A[1])}}return false},findHeaderCell:function(B){var A=this.findCell(B);return A&&this.fly(A).hasClass(this.hdCls)?A:null},findHeaderIndex:function(A){return this.findCellIndex(A,this.hdCls)},findRow:function(A){if(!A){return false}return this.fly(A).findParent(this.rowSelector,10)},findRowIndex:function(A){var B=this.findRow(A);return B?B.rowIndex:false},getRow:function(A){return this.getRows()[A]},getCell:function(B,A){return this.getRow(B).getElementsByTagName(\"td\")[A]},getHeaderCell:function(A){return this.mainHd.dom.getElementsByTagName(\"td\")[A]},addRowClass:function(C,A){var B=this.getRow(C);if(B){this.fly(B).addClass(A)}},removeRowClass:function(C,A){var B=this.getRow(C);if(B){this.fly(B).removeClass(A)}},removeRow:function(A){Ext.removeNode(this.getRow(A))},removeRows:function(C,A){var B=this.mainBody.dom;for(var D=C;D<=A;D++){Ext.removeNode(B.childNodes[C])}},getScrollState:function(){var A=this.scroller.dom;return{left:A.scrollLeft,top:A.scrollTop}},restoreScroll:function(A){var B=this.scroller.dom;B.scrollLeft=A.left;B.scrollTop=A.top},scrollToTop:function(){this.scroller.dom.scrollTop=0;this.scroller.dom.scrollLeft=0},syncScroll:function(){this.syncHeaderScroll();var A=this.scroller.dom;this.grid.fireEvent(\"bodyscroll\",A.scrollLeft,A.scrollTop)},syncHeaderScroll:function(){var A=this.scroller.dom;this.innerHd.scrollLeft=A.scrollLeft;this.innerHd.scrollLeft=A.scrollLeft},updateSortIcon:function(B,A){var D=this.sortClasses;var C=this.mainHd.select(\"td\").removeClass(D);C.item(B).addClass(D[A==\"DESC\"?1:0])},updateAllColumnWidths:function(){var D=this.getTotalWidth();var H=this.cm.getColumnCount();var F=[];for(var B=0;B<H;B++){F[B]=this.getColumnWidth(B)}this.innerHd.firstChild.firstChild.style.width=D;for(var B=0;B<H;B++){var C=this.getHeaderCell(B);C.style.width=F[B]}var G=this.getRows();for(var B=0,E=G.length;B<E;B++){G[B].style.width=D;G[B].firstChild.style.width=D;var I=G[B].firstChild.rows[0];for(var A=0;A<H;A++){I.childNodes[A].style.width=F[A]}}this.onAllColumnWidthsUpdated(F,D)},updateColumnWidth:function(D,G){var B=this.getColumnWidth(D);var C=this.getTotalWidth();this.innerHd.firstChild.firstChild.style.width=C;var H=this.getHeaderCell(D);H.style.width=B;var F=this.getRows();for(var E=0,A=F.length;E<A;E++){F[E].style.width=C;F[E].firstChild.style.width=C;F[E].firstChild.rows[0].childNodes[D].style.width=B}this.onColumnWidthUpdated(D,B,C)},updateColumnHidden:function(C,F){var B=this.getTotalWidth();this.innerHd.firstChild.firstChild.style.width=B;var H=F?\"none\":\"\";var G=this.getHeaderCell(C);G.style.display=H;var E=this.getRows();for(var D=0,A=E.length;D<A;D++){E[D].style.width=B;E[D].firstChild.style.width=B;E[D].firstChild.rows[0].childNodes[C].style.display=H}this.onColumnHiddenUpdated(C,F,B);delete this.lastViewWidth;this.layout()},doRender:function(E,G,M,A,L,Q){var B=this.templates,D=B.cell,F=B.row,H=L-1;var C=\"width:\"+this.getTotalWidth()+\";\";var T=[],N,U,O={},I={tstyle:C},K;for(var P=0,S=G.length;P<S;P++){K=G[P];N=[];var J=(P+A);for(var R=0;R<L;R++){U=E[R];O.id=U.id;O.css=R==0?\"x-grid3-cell-first \":(R==H?\"x-grid3-cell-last \":\"\");O.attr=O.cellAttr=\"\";O.value=U.renderer(K.data[U.name],O,K,J,R,M);O.style=U.style;if(O.value==undefined||O.value===\"\"){O.value=\"&#160;\"}if(K.dirty&&typeof K.modified[U.name]!==\"undefined\"){O.css+=\" x-grid3-dirty-cell\"}N[N.length]=D.apply(O)}var V=[];if(Q&&((J+1)%2==0)){V[0]=\"x-grid3-row-alt\"}if(K.dirty){V[1]=\" x-grid3-dirty-row\"}I.cols=L;if(this.getRowClass){V[2]=this.getRowClass(K,J,I,M)}I.alt=V.join(\" \");I.cells=N.join(\"\");T[T.length]=F.apply(I)}return T.join(\"\")},processRows:function(E,D){if(this.ds.getCount()<1){return }D=D||!this.grid.stripeRows;E=E||0;var I=this.getRows();var F=\" x-grid3-row-alt \";for(var B=E,C=I.length;B<C;B++){var H=I[B];H.rowIndex=B;if(!D){var A=((B+1)%2==0);var G=(\" \"+H.className+\" \").indexOf(F)!=-1;if(A==G){continue}if(A){H.className+=\" x-grid3-row-alt\"}else{H.className=H.className.replace(\"x-grid3-row-alt\",\"\")}}}},renderUI:function(){var E=this.renderHeaders();var B=this.templates.body.apply({rows:\"\"});var C=this.templates.master.apply({body:B,header:E});var D=this.grid;D.getGridEl().dom.innerHTML=C;this.initElements();this.mainBody.dom.innerHTML=this.renderRows();this.processRows(0,true);Ext.fly(this.innerHd).on(\"click\",this.handleHdDown,this);this.mainHd.on(\"mouseover\",this.handleHdOver,this);this.mainHd.on(\"mouseout\",this.handleHdOut,this);this.mainHd.on(\"mousemove\",this.handleHdMove,this);this.scroller.on(\"scroll\",this.syncScroll,this);if(D.enableColumnResize!==false){this.splitone=new Ext.grid.GridView.SplitDragZone(D,this.mainHd.dom)}if(D.enableColumnMove){this.columnDrag=new Ext.grid.GridView.ColumnDragZone(D,this.innerHd);this.columnDrop=new Ext.grid.HeaderDropZone(D,this.mainHd.dom)}if(D.enableHdMenu!==false){if(D.enableColumnHide!==false){this.colMenu=new Ext.menu.Menu({id:D.id+\"-hcols-menu\"});this.colMenu.on(\"beforeshow\",this.beforeColMenuShow,this);this.colMenu.on(\"itemclick\",this.handleHdMenuClick,this)}this.hmenu=new Ext.menu.Menu({id:D.id+\"-hctx\"});this.hmenu.add({id:\"asc\",text:this.sortAscText,cls:\"xg-hmenu-sort-asc\"},{id:\"desc\",text:this.sortDescText,cls:\"xg-hmenu-sort-desc\"});if(D.enableColumnHide!==false){this.hmenu.add(\"-\",{id:\"columns\",text:this.columnsText,menu:this.colMenu,iconCls:\"x-cols-icon\"})}this.hmenu.on(\"itemclick\",this.handleHdMenuClick,this)}if(D.enableDragDrop||D.enableDrag){var A=new Ext.grid.GridDragZone(D,{ddGroup:D.ddGroup||\"GridDD\"})}this.updateHeaderSortState()},layout:function(){if(!this.mainBody){return }var E=this.grid;var G=E.getGridEl(),I=this.cm,B=E.autoExpandColumn,A=this;var C=G.getSize(true);var H=C.width;if(H<20||C.height<20){return }if(E.autoHeight){this.scroller.dom.style.overflow=\"visible\"}else{this.el.setSize(C.width,C.height);var F=this.mainHd.getHeight();var D=C.height-(F);this.scroller.setSize(H,D);if(this.innerHd){this.innerHd.style.width=(H)+\"px\"}}if(this.forceFit){if(this.lastViewWidth!=H){this.fitColumns(false,false);this.lastViewWidth=H}}else{this.autoExpand();this.syncHeaderScroll()}this.onLayout(H,D)},onLayout:function(A,B){},onColumnWidthUpdated:function(C,A,B){},onAllColumnWidthsUpdated:function(A,B){},onColumnHiddenUpdated:function(B,C,A){},updateColumnText:function(A,B){},afterMove:function(A){},init:function(A){this.grid=A;this.initTemplates();this.initData(A.store,A.colModel);this.initUI(A)},getColumnId:function(A){return this.cm.getColumnId(A)},renderHeaders:function(){var C=this.cm,F=this.templates;var E=F.hcell;var B=[],H=[],G={};for(var D=0,A=C.getColumnCount();D<A;D++){G.id=C.getColumnId(D);G.value=C.getColumnHeader(D)||\"\";G.style=this.getColumnStyle(D,true);G.tooltip=this.getColumnTooltip(D);if(C.config[D].align==\"right\"){G.istyle=\"padding-right:16px\"}else{delete G.istyle}B[B.length]=E.apply(G)}return F.header.apply({cells:B.join(\"\"),tstyle:\"width:\"+this.getTotalWidth()+\";\"})},getColumnTooltip:function(A){var B=this.cm.getColumnTooltip(A);if(B){if(Ext.QuickTips.isEnabled()){return\"ext:qtip=\\\"\"+B+\"\\\"\"}else{return\"title=\\\"\"+B+\"\\\"\"}}return\"\"},beforeUpdate:function(){this.grid.stopEditing(true)},updateHeaders:function(){this.innerHd.firstChild.innerHTML=this.renderHeaders()},focusRow:function(A){this.focusCell(A,0,false)},focusCell:function(D,A,C){var B=this.ensureVisible(D,A,C);this.focusEl.setXY(B);if(Ext.isGecko){this.focusEl.focus()}else{this.focusEl.focus.defer(1,this.focusEl)}},ensureVisible:function(P,E,D){if(typeof P!=\"number\"){P=P.rowIndex}if(!this.ds){return }if(P<0||P>=this.ds.getCount()){return }E=(E!==undefined?E:0);var I=this.getRow(P),F;if(!(D===false&&E===0)){while(this.cm.isHidden(E)){E++}F=this.getCell(P,E)}if(!I){return }var L=this.scroller.dom;var O=0;var C=I,M=this.el.dom;while(C&&C!=M){O+=C.offsetTop;C=C.offsetParent}O-=this.mainHd.dom.offsetHeight;var N=O+I.offsetHeight;var A=L.clientHeight;var M=parseInt(L.scrollTop,10);var K=M+A;if(O<M){L.scrollTop=O}else{if(N>K){L.scrollTop=N-A}}if(D!==false){var J=parseInt(F.offsetLeft,10);var H=J+F.offsetWidth;var G=parseInt(L.scrollLeft,10);var B=G+L.clientWidth;if(J<G){L.scrollLeft=J}else{if(H>B){L.scrollLeft=H-L.clientWidth}}}return F?Ext.fly(F).getXY():[L.scrollLeft,Ext.fly(I).getY()]},insertRows:function(A,F,C,E){if(!E&&F===0&&C==A.getCount()-1){this.refresh()}else{if(!E){this.fireEvent(\"beforerowsinserted\",this,F,C)}var B=this.renderRows(F,C);var D=this.getRow(F);if(D){Ext.DomHelper.insertHtml(\"beforeBegin\",D,B)}else{Ext.DomHelper.insertHtml(\"beforeEnd\",this.mainBody.dom,B)}if(!E){this.fireEvent(\"rowsinserted\",this,F,C);this.processRows(F)}}},deleteRows:function(A,C,B){if(A.getRowCount()<1){this.refresh()}else{this.fireEvent(\"beforerowsdeleted\",this,C,B);this.removeRows(C,B);this.processRows(C);this.fireEvent(\"rowsdeleted\",this,C,B)}},getColumnStyle:function(A,C){var B=!C?(this.cm.config[A].css||\"\"):\"\";B+=\"width:\"+this.getColumnWidth(A)+\";\";if(this.cm.isHidden(A)){B+=\"display:none;\"}var D=this.cm.config[A].align;if(D){B+=\"text-align:\"+D+\";\"}return B},getColumnWidth:function(B){var A=this.cm.getColumnWidth(B);if(typeof A==\"number\"){return(Ext.isBorderBox?A:(A-this.borderWidth>0?A-this.borderWidth:0))+\"px\"}return A},getTotalWidth:function(){return this.cm.getTotalWidth()+\"px\"},fitColumns:function(D,G,E){var F=this.cm,S,L,O;var R=F.getTotalWidth(false);var J=this.grid.getGridEl().getWidth(true)-this.scrollOffset;if(J<20){return }var B=J-R;if(B===0){return false}var A=F.getColumnCount(true);var P=A-(typeof E==\"number\"?1:0);if(P===0){P=1;E=undefined}var K=F.getColumnCount();var I=[];var N=0;var M=0;var H;for(O=0;O<K;O++){if(!F.isHidden(O)&&!F.isFixed(O)&&O!==E){H=F.getColumnWidth(O);I.push(O);N=O;I.push(H);M+=H}}var C=(J-F.getTotalWidth())/M;while(I.length){H=I.pop();O=I.pop();F.setColumnWidth(O,Math.max(this.grid.minColumnWidth,Math.floor(H+H*C)),true)}if((R=F.getTotalWidth(false))>J){var Q=P!=A?E:N;F.setColumnWidth(Q,Math.max(1,F.getColumnWidth(Q)-(R-J)),true)}if(D!==true){this.updateAllColumnWidths()}return true},autoExpand:function(B){var G=this.grid,A=this.cm;if(!this.userResized&&G.autoExpandColumn){var D=A.getTotalWidth(false);var H=this.grid.getGridEl().getWidth(true)-this.scrollOffset;if(D!=H){var F=A.getIndexById(G.autoExpandColumn);var E=A.getColumnWidth(F);var C=Math.min(Math.max(((H-D)+E),G.autoExpandMin),G.autoExpandMax);if(C!=E){A.setColumnWidth(F,C,true);if(B!==true){this.updateColumnWidth(F,C)}}}}},getColumnData:function(){var D=[],A=this.cm,E=A.getColumnCount();for(var C=0;C<E;C++){var B=A.getDataIndex(C);D[C]={name:(typeof B==\"undefined\"?this.ds.fields.get(C).name:B),renderer:A.getRenderer(C),id:A.getColumnId(C),style:this.getColumnStyle(C)}}return D},renderRows:function(H,C){var D=this.grid,F=D.colModel,A=D.store,I=D.stripeRows;var G=F.getColumnCount();if(A.getCount()<1){return\"\"}var E=this.getColumnData();H=H||0;C=typeof C==\"undefined\"?A.getCount()-1:C;var B=A.getRange(H,C);return this.doRender(E,B,A,H,G,I)},renderBody:function(){var A=this.renderRows();return this.templates.body.apply({rows:A})},refreshRow:function(B){var D=this.ds,C;if(typeof B==\"number\"){C=B;B=D.getAt(C)}else{C=D.indexOf(B)}var A=[];this.insertRows(D,C,C,true);this.getRow(C).rowIndex=C;this.onRemove(D,B,C+1,true);this.fireEvent(\"rowupdated\",this,C,B)},refresh:function(B){this.fireEvent(\"beforerefresh\",this);this.grid.stopEditing(true);var A=this.renderBody();this.mainBody.update(A);if(B===true){this.updateHeaders();this.updateHeaderSortState()}this.processRows(0,true);this.layout();this.applyEmptyText();this.fireEvent(\"refresh\",this)},applyEmptyText:function(){if(this.emptyText&&!this.hasRows()){this.mainBody.update(\"<div class=\\\"x-grid-empty\\\">\"+this.emptyText+\"</div>\")}},updateHeaderSortState:function(){var B=this.ds.getSortState();if(!B){return }if(!this.sortState||(this.sortState.field!=B.field||this.sortState.direction!=B.direction)){this.grid.fireEvent(\"sortchange\",this.grid,B)}this.sortState=B;var C=this.cm.findColumnIndex(B.field);if(C!=-1){var A=B.direction;this.updateSortIcon(C,A)}},destroy:function(){if(this.colMenu){this.colMenu.removeAll();Ext.menu.MenuMgr.unregister(this.colMenu);this.colMenu.getEl().remove();delete this.colMenu}if(this.hmenu){this.hmenu.removeAll();Ext.menu.MenuMgr.unregister(this.hmenu);this.hmenu.getEl().remove();delete this.hmenu}if(this.grid.enableColumnMove){var C=Ext.dd.DDM.ids[\"gridHeader\"+this.grid.getGridEl().id];if(C){for(var A in C){if(!C[A].config.isTarget&&C[A].dragElId){var B=C[A].dragElId;C[A].unreg();Ext.get(B).remove()}else{if(C[A].config.isTarget){C[A].proxyTop.remove();C[A].proxyBottom.remove();C[A].unreg()}}if(Ext.dd.DDM.locationCache[A]){delete Ext.dd.DDM.locationCache[A]}}delete Ext.dd.DDM.ids[\"gridHeader\"+this.grid.getGridEl().id]}}Ext.destroy(this.resizeMarker,this.resizeProxy);this.initData(null,null);Ext.EventManager.removeResizeListener(this.onWindowResize,this)},onDenyColumnHide:function(){},render:function(){var A=this.cm;var B=A.getColumnCount();if(this.autoFill){this.fitColumns(true,true)}else{if(this.forceFit){this.fitColumns(true,false)}else{if(this.grid.autoExpandColumn){this.autoExpand(true)}}}this.renderUI()},initData:function(B,A){if(this.ds){this.ds.un(\"load\",this.onLoad,this);this.ds.un(\"datachanged\",this.onDataChange,this);this.ds.un(\"add\",this.onAdd,this);this.ds.un(\"remove\",this.onRemove,this);this.ds.un(\"update\",this.onUpdate,this);this.ds.un(\"clear\",this.onClear,this)}if(B){B.on(\"load\",this.onLoad,this);B.on(\"datachanged\",this.onDataChange,this);B.on(\"add\",this.onAdd,this);B.on(\"remove\",this.onRemove,this);B.on(\"update\",this.onUpdate,this);B.on(\"clear\",this.onClear,this)}this.ds=B;if(this.cm){this.cm.un(\"configchange\",this.onColConfigChange,this);this.cm.un(\"widthchange\",this.onColWidthChange,this);this.cm.un(\"headerchange\",this.onHeaderChange,this);this.cm.un(\"hiddenchange\",this.onHiddenChange,this);this.cm.un(\"columnmoved\",this.onColumnMove,this);this.cm.un(\"columnlockchange\",this.onColumnLock,this)}if(A){A.on(\"configchange\",this.onColConfigChange,this);A.on(\"widthchange\",this.onColWidthChange,this);A.on(\"headerchange\",this.onHeaderChange,this);A.on(\"hiddenchange\",this.onHiddenChange,this);A.on(\"columnmoved\",this.onColumnMove,this);A.on(\"columnlockchange\",this.onColumnLock,this)}this.cm=A},onDataChange:function(){this.refresh();this.updateHeaderSortState()},onClear:function(){this.refresh()},onUpdate:function(B,A){this.refreshRow(A)},onAdd:function(C,A,B){this.insertRows(C,B,B+(A.length-1))},onRemove:function(D,A,B,C){if(C!==true){this.fireEvent(\"beforerowremoved\",this,B,A)}this.removeRow(B);if(C!==true){this.processRows(B);this.applyEmptyText();this.fireEvent(\"rowremoved\",this,B,A)}},onLoad:function(){this.scrollToTop()},onColWidthChange:function(A,B,C){this.updateColumnWidth(B,C)},onHeaderChange:function(A,B,C){this.updateHeaders()},onHiddenChange:function(A,B,C){this.updateColumnHidden(B,C)},onColumnMove:function(A,D,B){this.indexMap=null;var C=this.getScrollState();this.refresh(true);this.restoreScroll(C);this.afterMove(B)},onColConfigChange:function(){delete this.lastViewWidth;this.indexMap=null;this.refresh(true)},initUI:function(A){A.on(\"headerclick\",this.onHeaderClick,this);if(A.trackMouseOver){A.on(\"mouseover\",this.onRowOver,this);A.on(\"mouseout\",this.onRowOut,this)}},initEvents:function(){},onHeaderClick:function(B,A){if(this.headersDisabled||!this.cm.isSortable(A)){return }B.stopEditing(true);B.store.sort(this.cm.getDataIndex(A))},onRowOver:function(B,A){var C;if((C=this.findRowIndex(A))!==false){this.addRowClass(C,\"x-grid3-row-over\")}},onRowOut:function(B,A){var C;if((C=this.findRowIndex(A))!==false&&C!==this.findRowIndex(B.getRelatedTarget())){this.removeRowClass(C,\"x-grid3-row-over\")}},handleWheel:function(A){A.stopPropagation()},onRowSelect:function(A){this.addRowClass(A,\"x-grid3-row-selected\")},onRowDeselect:function(A){this.removeRowClass(A,\"x-grid3-row-selected\")},onCellSelect:function(C,B){var A=this.getCell(C,B);if(A){this.fly(A).addClass(\"x-grid3-cell-selected\")}},onCellDeselect:function(C,B){var A=this.getCell(C,B);if(A){this.fly(A).removeClass(\"x-grid3-cell-selected\")}},onColumnSplitterMoved:function(C,B){this.userResized=true;var A=this.grid.colModel;A.setColumnWidth(C,B,true);if(this.forceFit){this.fitColumns(true,false,C);this.updateAllColumnWidths()}else{this.updateColumnWidth(C,B)}this.grid.fireEvent(\"columnresize\",C,B)},handleHdMenuClick:function(C){var B=this.hdCtxIndex;var A=this.cm,D=this.ds;switch(C.id){case\"asc\":D.sort(A.getDataIndex(B),\"ASC\");break;case\"desc\":D.sort(A.getDataIndex(B),\"DESC\");break;default:B=A.getIndexById(C.id.substr(4));if(B!=-1){if(C.checked&&A.getColumnsBy(this.isHideableColumn,this).length<=1){this.onDenyColumnHide();return false}A.setHidden(B,C.checked)}}return true},isHideableColumn:function(A){return !A.hidden&&!A.fixed},beforeColMenuShow:function(){var A=this.cm,C=A.getColumnCount();this.colMenu.removeAll();for(var B=0;B<C;B++){if(A.config[B].fixed!==true&&A.config[B].hideable!==false){this.colMenu.add(new Ext.menu.CheckItem({id:\"col-\"+A.getColumnId(B),text:A.getColumnHeader(B),checked:!A.isHidden(B),hideOnClick:false,disabled:A.config[B].hideable===false}))}}},handleHdDown:function(F,D){if(Ext.fly(D).hasClass(\"x-grid3-hd-btn\")){F.stopEvent();var E=this.findHeaderCell(D);Ext.fly(E).addClass(\"x-grid3-hd-menu-open\");var C=this.getCellIndex(E);this.hdCtxIndex=C;var B=this.hmenu.items,A=this.cm;B.get(\"asc\").setDisabled(!A.isSortable(C));B.get(\"desc\").setDisabled(!A.isSortable(C));this.hmenu.on(\"hide\",function(){Ext.fly(E).removeClass(\"x-grid3-hd-menu-open\")},this,{single:true});this.hmenu.show(D,\"tl-bl?\")}},handleHdOver:function(D,A){var C=this.findHeaderCell(A);if(C&&!this.headersDisabled){this.activeHd=C;this.activeHdIndex=this.getCellIndex(C);var B=this.fly(C);this.activeHdRegion=B.getRegion();if(!this.cm.isMenuDisabled(this.activeHdIndex)){B.addClass(\"x-grid3-hd-over\");this.activeHdBtn=B.child(\".x-grid3-hd-btn\");if(this.activeHdBtn){this.activeHdBtn.dom.style.height=(C.firstChild.offsetHeight-1)+\"px\"}}}},handleHdMove:function(F,D){if(this.activeHd&&!this.headersDisabled){var B=this.splitHandleWidth||5;var E=this.activeHdRegion;var A=F.getPageX();var C=this.activeHd.style;if(A-E.left<=B&&this.cm.isResizable(this.activeHdIndex-1)){C.cursor=Ext.isAir?\"move\":Ext.isSafari?\"e-resize\":\"col-resize\"}else{if(E.right-A<=(!this.activeHdBtn?B:2)&&this.cm.isResizable(this.activeHdIndex)){C.cursor=Ext.isAir?\"move\":Ext.isSafari?\"w-resize\":\"col-resize\"}else{C.cursor=\"\"}}}},handleHdOut:function(C,A){var B=this.findHeaderCell(A);if(B&&(!Ext.isIE||!C.within(B,true))){this.activeHd=null;this.fly(B).removeClass(\"x-grid3-hd-over\");B.style.cursor=\"\"}},hasRows:function(){var A=this.mainBody.dom.firstChild;return A&&A.className!=\"x-grid-empty\"},bind:function(A,B){this.initData(A,B)}});Ext.grid.GridView.SplitDragZone=function(A,B){this.grid=A;this.view=A.getView();this.marker=this.view.resizeMarker;this.proxy=this.view.resizeProxy;Ext.grid.GridView.SplitDragZone.superclass.constructor.call(this,B,\"gridSplitters\"+this.grid.getGridEl().id,{dragElId:Ext.id(this.proxy.dom),resizeFrame:false});this.scroll=false;this.hw=this.view.splitHandleWidth||5};Ext.extend(Ext.grid.GridView.SplitDragZone,Ext.dd.DDProxy,{b4StartDrag:function(A,E){this.view.headersDisabled=true;var D=this.view.mainWrap.getHeight();this.marker.setHeight(D);this.marker.show();this.marker.alignTo(this.view.getHeaderCell(this.cellIndex),\"tl-tl\",[-2,0]);this.proxy.setHeight(D);var B=this.cm.getColumnWidth(this.cellIndex);var C=Math.max(B-this.grid.minColumnWidth,0);this.resetConstraints();this.setXConstraint(C,1000);this.setYConstraint(0,0);this.minX=A-C;this.maxX=A+1000;this.startPos=A;Ext.dd.DDProxy.prototype.b4StartDrag.call(this,A,E)},handleMouseDown:function(A){var H=this.view.findHeaderCell(A.getTarget());if(H){var K=this.view.fly(H).getXY(),E=K[0],D=K[1];var I=A.getXY(),C=I[0],B=I[1];var G=H.offsetWidth,F=false;if((C-E)<=this.hw){F=-1}else{if((E+G)-C<=this.hw){F=0}}if(F!==false){this.cm=this.grid.colModel;var J=this.view.getCellIndex(H);if(F==-1){if(J+F<0){return }while(this.cm.isHidden(J+F)){--F;if(J+F<0){return }}}this.cellIndex=J+F;this.split=H.dom;if(this.cm.isResizable(this.cellIndex)&&!this.cm.isFixed(this.cellIndex)){Ext.grid.GridView.SplitDragZone.superclass.handleMouseDown.apply(this,arguments)}}else{if(this.view.columnDrag){this.view.columnDrag.callHandleMouseDown(A)}}}},endDrag:function(D){this.marker.hide();var A=this.view;var B=Math.max(this.minX,D.getPageX());var C=B-this.startPos;A.onColumnSplitterMoved(this.cellIndex,this.cm.getColumnWidth(this.cellIndex)+C);setTimeout(function(){A.headersDisabled=false},50)},autoOffset:function(){this.setDelta(0,0)}});\nExt.grid.GroupingView=Ext.extend(Ext.grid.GridView,{hideGroupedColumn:false,showGroupName:true,startCollapsed:false,enableGrouping:true,enableGroupingMenu:true,enableNoGroups:true,emptyGroupText:\"(None)\",ignoreAdd:false,groupTextTpl:\"{text}\",gidSeed:1000,initTemplates:function(){Ext.grid.GroupingView.superclass.initTemplates.call(this);this.state={};var A=this.grid.getSelectionModel();A.on(A.selectRow?\"beforerowselect\":\"beforecellselect\",this.onBeforeRowSelect,this);if(!this.startGroup){this.startGroup=new Ext.XTemplate(\"<div id=\\\"{groupId}\\\" class=\\\"x-grid-group {cls}\\\">\",\"<div id=\\\"{groupId}-hd\\\" class=\\\"x-grid-group-hd\\\" style=\\\"{style}\\\"><div>\",this.groupTextTpl,\"</div></div>\",\"<div id=\\\"{groupId}-bd\\\" class=\\\"x-grid-group-body\\\">\")}this.startGroup.compile();this.endGroup=\"</div></div>\"},findGroup:function(A){return Ext.fly(A).up(\".x-grid-group\",this.mainBody.dom)},getGroups:function(){return this.hasRows()?this.mainBody.dom.childNodes:[]},onAdd:function(){if(this.enableGrouping&&!this.ignoreAdd){var A=this.getScrollState();this.refresh();this.restoreScroll(A)}else{if(!this.enableGrouping){Ext.grid.GroupingView.superclass.onAdd.apply(this,arguments)}}},onRemove:function(E,A,B,D){Ext.grid.GroupingView.superclass.onRemove.apply(this,arguments);var C=document.getElementById(A._groupId);if(C&&C.childNodes[1].childNodes.length<1){Ext.removeNode(C)}this.applyEmptyText()},refreshRow:function(A){if(this.ds.getCount()==1){this.refresh()}else{this.isUpdating=true;Ext.grid.GroupingView.superclass.refreshRow.apply(this,arguments);this.isUpdating=false}},beforeMenuShow:function(){var C=this.getGroupField();var B=this.hmenu.items.get(\"groupBy\");if(B){B.setDisabled(this.cm.config[this.hdCtxIndex].groupable===false)}var A=this.hmenu.items.get(\"showGroups\");if(A){if(!!C){A.setDisabled(this.cm.config[this.hdCtxIndex].groupable===false)}A.setChecked(!!C)}},renderUI:function(){Ext.grid.GroupingView.superclass.renderUI.call(this);this.mainBody.on(\"mousedown\",this.interceptMouse,this);if(this.enableGroupingMenu&&this.hmenu){this.hmenu.add(\"-\",{id:\"groupBy\",text:this.groupByText,handler:this.onGroupByClick,scope:this,iconCls:\"x-group-by-icon\"});if(this.enableNoGroups){this.hmenu.add({id:\"showGroups\",text:this.showGroupsText,checked:true,checkHandler:this.onShowGroupsClick,scope:this})}this.hmenu.on(\"beforeshow\",this.beforeMenuShow,this)}},onGroupByClick:function(){this.grid.store.groupBy(this.cm.getDataIndex(this.hdCtxIndex));this.beforeMenuShow()},onShowGroupsClick:function(A,B){if(B){this.onGroupByClick()}else{this.grid.store.clearGrouping()}},toggleGroup:function(C,B){this.grid.stopEditing(true);C=Ext.getDom(C);var A=Ext.fly(C);B=B!==undefined?B:A.hasClass(\"x-grid-group-collapsed\");this.state[A.dom.id]=B;A[B?\"removeClass\":\"addClass\"](\"x-grid-group-collapsed\")},toggleAllGroups:function(C){var B=this.getGroups();for(var D=0,A=B.length;D<A;D++){this.toggleGroup(B[D],C)}},expandAllGroups:function(){this.toggleAllGroups(true)},collapseAllGroups:function(){this.toggleAllGroups(false)},interceptMouse:function(B){var A=B.getTarget(\".x-grid-group-hd\",this.mainBody);if(A){B.stopEvent();this.toggleGroup(A.parentNode)}},getGroup:function(A,D,F,G,B,E){var C=F?F(A,{},D,G,B,E):String(A);if(C===\"\"){C=this.cm.config[B].emptyGroupText||this.emptyGroupText}return C},getGroupField:function(){return this.grid.store.getGroupState()},renderRows:function(){var A=this.getGroupField();var D=!!A;if(this.hideGroupedColumn){var B=this.cm.findColumnIndex(A);if(!D&&this.lastGroupField!==undefined){this.mainBody.update(\"\");this.cm.setHidden(this.cm.findColumnIndex(this.lastGroupField),false);delete this.lastGroupField}else{if(D&&this.lastGroupField===undefined){this.lastGroupField=A;this.cm.setHidden(B,true)}else{if(D&&this.lastGroupField!==undefined&&A!==this.lastGroupField){this.mainBody.update(\"\");var C=this.cm.findColumnIndex(this.lastGroupField);this.cm.setHidden(C,false);this.lastGroupField=A;this.cm.setHidden(B,true)}}}}return Ext.grid.GroupingView.superclass.renderRows.apply(this,arguments)},doRender:function(D,G,P,A,O,R){if(G.length<1){return\"\"}var Y=this.getGroupField();var N=this.cm.findColumnIndex(Y);this.enableGrouping=!!Y;if(!this.enableGrouping||this.isUpdating){return Ext.grid.GroupingView.superclass.doRender.apply(this,arguments)}var H=\"width:\"+this.getTotalWidth()+\";\";var Q=this.grid.getGridEl().id;var F=this.cm.config[N];var B=F.groupRenderer||F.renderer;var S=this.showGroupName?(F.groupName||F.header)+\": \":\"\";var X=[],K,T,U,M;for(T=0,U=G.length;T<U;T++){var J=A+T;var L=G[T],E=L.data[Y],V=this.getGroup(E,L,B,J,N,P);if(!K||K.group!=V){M=Q+\"-gp-\"+Y+\"-\"+Ext.util.Format.htmlEncode(V);var C=typeof this.state[M]!==\"undefined\"?!this.state[M]:this.startCollapsed;var I=C?\"x-grid-group-collapsed\":\"\";K={group:V,gvalue:E,text:S+V,groupId:M,startRow:J,rs:[L],cls:I,style:H};X.push(K)}else{K.rs.push(L)}L._groupId=M}var W=[];for(T=0,U=X.length;T<U;T++){var V=X[T];this.doGroupStart(W,V,D,P,O);W[W.length]=Ext.grid.GroupingView.superclass.doRender.call(this,D,V.rs,P,V.startRow,O,R);this.doGroupEnd(W,V,D,P,O)}return W.join(\"\")},getGroupId:function(F){var D=this.grid.getGridEl().id;var C=this.getGroupField();var E=this.cm.findColumnIndex(C);var B=this.cm.config[E];var G=B.groupRenderer||B.renderer;var A=this.getGroup(F,{data:{}},G,0,E,this.ds);return D+\"-gp-\"+C+\"-\"+Ext.util.Format.htmlEncode(F)},doGroupStart:function(A,D,B,E,C){A[A.length]=this.startGroup.apply(D)},doGroupEnd:function(A,D,B,E,C){A[A.length]=this.endGroup},getRows:function(){if(!this.enableGrouping){return Ext.grid.GroupingView.superclass.getRows.call(this)}var G=[];var F,C=this.getGroups();for(var E=0,A=C.length;E<A;E++){F=C[E].childNodes[1].childNodes;for(var D=0,B=F.length;D<B;D++){G[G.length]=F[D]}}return G},updateGroupWidths:function(){if(!this.enableGrouping||!this.hasRows()){return }var C=Math.max(this.cm.getTotalWidth(),this.el.dom.offsetWidth-this.scrollOffset)+\"px\";var B=this.getGroups();for(var D=0,A=B.length;D<A;D++){B[D].firstChild.style.width=C}},onColumnWidthUpdated:function(C,A,B){this.updateGroupWidths()},onAllColumnWidthsUpdated:function(A,B){this.updateGroupWidths()},onColumnHiddenUpdated:function(B,C,A){this.updateGroupWidths()},onLayout:function(){this.updateGroupWidths()},onBeforeRowSelect:function(D,C){if(!this.enableGrouping){return }var B=this.getRow(C);if(B&&!B.offsetParent){var A=this.findGroup(B);this.toggleGroup(A,true)}},groupByText:\"Group By This Field\",showGroupsText:\"Show in Groups\"});Ext.grid.GroupingView.GROUP_ID=1000;\nExt.grid.HeaderDragZone=function(A,C,B){this.grid=A;this.view=A.getView();this.ddGroup=\"gridHeader\"+this.grid.getGridEl().id;Ext.grid.HeaderDragZone.superclass.constructor.call(this,C);if(B){this.setHandleElId(Ext.id(C));this.setOuterHandleElId(Ext.id(B))}this.scroll=false};Ext.extend(Ext.grid.HeaderDragZone,Ext.dd.DragZone,{maxDragWidth:120,getDragData:function(C){var A=Ext.lib.Event.getTarget(C);var B=this.view.findHeaderCell(A);if(B){return{ddel:B.firstChild,header:B}}return false},onInitDrag:function(A){this.view.headersDisabled=true;var B=this.dragData.ddel.cloneNode(true);B.id=Ext.id();B.style.width=Math.min(this.dragData.header.offsetWidth,this.maxDragWidth)+\"px\";this.proxy.update(B);return true},afterValidDrop:function(){var A=this.view;setTimeout(function(){A.headersDisabled=false},50)},afterInvalidDrop:function(){var A=this.view;setTimeout(function(){A.headersDisabled=false},50)}});Ext.grid.HeaderDropZone=function(A,C,B){this.grid=A;this.view=A.getView();this.proxyTop=Ext.DomHelper.append(document.body,{cls:\"col-move-top\",html:\"&#160;\"},true);this.proxyBottom=Ext.DomHelper.append(document.body,{cls:\"col-move-bottom\",html:\"&#160;\"},true);this.proxyTop.hide=this.proxyBottom.hide=function(){this.setLeftTop(-100,-100);this.setStyle(\"visibility\",\"hidden\")};this.ddGroup=\"gridHeader\"+this.grid.getGridEl().id;Ext.grid.HeaderDropZone.superclass.constructor.call(this,A.getGridEl().dom)};Ext.extend(Ext.grid.HeaderDropZone,Ext.dd.DropZone,{proxyOffsets:[-4,-9],fly:Ext.Element.fly,getTargetFromEvent:function(C){var A=Ext.lib.Event.getTarget(C);var B=this.view.findCellIndex(A);if(B!==false){return this.view.getHeaderCell(B)}},nextVisible:function(C){var B=this.view,A=this.grid.colModel;C=C.nextSibling;while(C){if(!A.isHidden(B.getCellIndex(C))){return C}C=C.nextSibling}return null},prevVisible:function(C){var B=this.view,A=this.grid.colModel;C=C.prevSibling;while(C){if(!A.isHidden(B.getCellIndex(C))){return C}C=C.prevSibling}return null},positionIndicator:function(D,B,E){var H=Ext.lib.Event.getPageX(E);var A=Ext.lib.Dom.getRegion(B.firstChild);var I,K,G=A.top+this.proxyOffsets[1];if((A.right-H)<=(A.right-A.left)/2){I=A.right+this.view.borderWidth;K=\"after\"}else{I=A.left;K=\"before\"}var F=this.view.getCellIndex(D);var J=this.view.getCellIndex(B);if(this.grid.colModel.isFixed(J)){return false}var C=this.grid.colModel.isLocked(J);if(K==\"after\"){J++}if(F<J){J--}if(F==J&&(C==this.grid.colModel.isLocked(F))){return false}I+=this.proxyOffsets[0];this.proxyTop.setLeftTop(I,G);this.proxyTop.show();if(!this.bottomOffset){this.bottomOffset=this.view.mainHd.getHeight()}this.proxyBottom.setLeftTop(I,G+this.proxyTop.dom.offsetHeight+this.bottomOffset);this.proxyBottom.show();return K},onNodeEnter:function(D,A,C,B){if(B.header!=D){this.positionIndicator(B.header,D,C)}},onNodeOver:function(E,B,D,C){var A=false;if(C.header!=E){A=this.positionIndicator(C.header,E,D)}if(!A){this.proxyTop.hide();this.proxyBottom.hide()}return A?this.dropAllowed:this.dropNotAllowed},onNodeOut:function(D,A,C,B){this.proxyTop.hide();this.proxyBottom.hide()},onNodeDrop:function(B,K,F,D){var E=D.header;if(E!=B){var I=this.grid.colModel;var H=Ext.lib.Event.getPageX(F);var A=Ext.lib.Dom.getRegion(B.firstChild);var L=(A.right-H)<=((A.right-A.left)/2)?\"after\":\"before\";var G=this.view.getCellIndex(E);var J=this.view.getCellIndex(B);var C=I.isLocked(J);if(L==\"after\"){J++}if(G<J){J--}if(G==J&&(C==I.isLocked(G))){return false}I.setLocked(G,C,true);I.moveColumn(G,J);this.grid.fireEvent(\"columnmove\",G,J);return true}return false}});Ext.grid.GridView.ColumnDragZone=function(A,B){Ext.grid.GridView.ColumnDragZone.superclass.constructor.call(this,A,B,null);this.proxy.el.addClass(\"x-grid3-col-dd\")};Ext.extend(Ext.grid.GridView.ColumnDragZone,Ext.grid.HeaderDragZone,{handleMouseDown:function(A){},callHandleMouseDown:function(A){Ext.grid.GridView.ColumnDragZone.superclass.handleMouseDown.call(this,A)}});\nExt.grid.SplitDragZone=function(A,C,B){this.grid=A;this.view=A.getView();this.proxy=this.view.resizeProxy;Ext.grid.SplitDragZone.superclass.constructor.call(this,C,\"gridSplitters\"+this.grid.getGridEl().id,{dragElId:Ext.id(this.proxy.dom),resizeFrame:false});this.setHandleElId(Ext.id(C));this.setOuterHandleElId(Ext.id(B));this.scroll=false};Ext.extend(Ext.grid.SplitDragZone,Ext.dd.DDProxy,{fly:Ext.Element.fly,b4StartDrag:function(A,D){this.view.headersDisabled=true;this.proxy.setHeight(this.view.mainWrap.getHeight());var B=this.cm.getColumnWidth(this.cellIndex);var C=Math.max(B-this.grid.minColumnWidth,0);this.resetConstraints();this.setXConstraint(C,1000);this.setYConstraint(0,0);this.minX=A-C;this.maxX=A+1000;this.startPos=A;Ext.dd.DDProxy.prototype.b4StartDrag.call(this,A,D)},handleMouseDown:function(B){ev=Ext.EventObject.setEvent(B);var A=this.fly(ev.getTarget());if(A.hasClass(\"x-grid-split\")){this.cellIndex=this.view.getCellIndex(A.dom);this.split=A.dom;this.cm=this.grid.colModel;if(this.cm.isResizable(this.cellIndex)&&!this.cm.isFixed(this.cellIndex)){Ext.grid.SplitDragZone.superclass.handleMouseDown.apply(this,arguments)}}},endDrag:function(C){this.view.headersDisabled=false;var A=Math.max(this.minX,Ext.lib.Event.getPageX(C));var B=A-this.startPos;this.view.onColumnSplitterMoved(this.cellIndex,this.cm.getColumnWidth(this.cellIndex)+B)},autoOffset:function(){this.setDelta(0,0)}});\nExt.grid.GridDragZone=function(B,A){this.view=B.getView();Ext.grid.GridDragZone.superclass.constructor.call(this,this.view.mainBody.dom,A);if(this.view.lockedBody){this.setHandleElId(Ext.id(this.view.mainBody.dom));this.setOuterHandleElId(Ext.id(this.view.lockedBody.dom))}this.scroll=false;this.grid=B;this.ddel=document.createElement(\"div\");this.ddel.className=\"x-grid-dd-wrap\"};Ext.extend(Ext.grid.GridDragZone,Ext.dd.DragZone,{ddGroup:\"GridDD\",getDragData:function(B){var A=Ext.lib.Event.getTarget(B);var D=this.view.findRowIndex(A);if(D!==false){var C=this.grid.selModel;if(!C.isSelected(D)||B.hasModifier()){C.handleMouseDown(this.grid,D,B)}return{grid:this.grid,ddel:this.ddel,rowIndex:D,selections:C.getSelections()}}return false},onInitDrag:function(B){var A=this.dragData;this.ddel.innerHTML=this.grid.getDragDropText();this.proxy.update(this.ddel)},afterRepair:function(){this.dragging=false},getRepairXY:function(B,A){return false},onEndDrag:function(A,B){},onValidDrop:function(A,B,C){this.hideProxy()},beforeInvalidDrop:function(A,B){}});\nExt.grid.ColumnModel=function(A){this.defaultWidth=100;this.defaultSortable=false;if(A.columns){Ext.apply(this,A);this.setConfig(A.columns,true)}else{this.setConfig(A,true)}this.addEvents(\"widthchange\",\"headerchange\",\"hiddenchange\",\"columnmoved\",\"columnlockchange\",\"configchange\");Ext.grid.ColumnModel.superclass.constructor.call(this)};Ext.extend(Ext.grid.ColumnModel,Ext.util.Observable,{getColumnId:function(A){return this.config[A].id},setConfig:function(C,B){if(!B){delete this.totalWidth;for(var D=0,A=this.config.length;D<A;D++){var E=this.config[D];if(E.editor){E.editor.destroy()}}}this.config=C;this.lookup={};for(var D=0,A=C.length;D<A;D++){var E=C[D];if(typeof E.renderer==\"string\"){E.renderer=Ext.util.Format[E.renderer]}if(typeof E.id==\"undefined\"){E.id=D}if(E.editor&&E.editor.isFormField){E.editor=new Ext.grid.GridEditor(E.editor)}this.lookup[E.id]=E}if(!B){this.fireEvent(\"configchange\",this)}},getColumnById:function(A){return this.lookup[A]},getIndexById:function(C){for(var B=0,A=this.config.length;B<A;B++){if(this.config[B].id==C){return B}}return -1},moveColumn:function(C,A){var B=this.config[C];this.config.splice(C,1);this.config.splice(A,0,B);this.dataMap=null;this.fireEvent(\"columnmoved\",this,C,A)},isLocked:function(A){return this.config[A].locked===true},setLocked:function(B,C,A){if(this.isLocked(B)==C){return }this.config[B].locked=C;if(!A){this.fireEvent(\"columnlockchange\",this,B,C)}},getTotalLockedWidth:function(){var A=0;for(var B=0;B<this.config.length;B++){if(this.isLocked(B)&&!this.isHidden(B)){this.totalWidth+=this.getColumnWidth(B)}}return A},getLockedCount:function(){for(var B=0,A=this.config.length;B<A;B++){if(!this.isLocked(B)){return B}}},getColumnCount:function(C){if(C===true){var D=0;for(var B=0,A=this.config.length;B<A;B++){if(!this.isHidden(B)){D++}}return D}return this.config.length},getColumnsBy:function(D,C){var E=[];for(var B=0,A=this.config.length;B<A;B++){var F=this.config[B];if(D.call(C||this,F,B)===true){E[E.length]=F}}return E},isSortable:function(A){if(typeof this.config[A].sortable==\"undefined\"){return this.defaultSortable}return this.config[A].sortable},isMenuDisabled:function(A){return !!this.config[A].menuDisabled},getRenderer:function(A){if(!this.config[A].renderer){return Ext.grid.ColumnModel.defaultRenderer}return this.config[A].renderer},setRenderer:function(A,B){this.config[A].renderer=B},getColumnWidth:function(A){return this.config[A].width||this.defaultWidth},setColumnWidth:function(B,C,A){this.config[B].width=C;this.totalWidth=null;if(!A){this.fireEvent(\"widthchange\",this,B,C)}},getTotalWidth:function(B){if(!this.totalWidth){this.totalWidth=0;for(var C=0,A=this.config.length;C<A;C++){if(B||!this.isHidden(C)){this.totalWidth+=this.getColumnWidth(C)}}}return this.totalWidth},getColumnHeader:function(A){return this.config[A].header},setColumnHeader:function(A,B){this.config[A].header=B;this.fireEvent(\"headerchange\",this,A,B)},getColumnTooltip:function(A){return this.config[A].tooltip},setColumnTooltip:function(A,B){this.config[A].tooltip=B},getDataIndex:function(A){return this.config[A].dataIndex},setDataIndex:function(A,B){this.config[A].dataIndex=B},findColumnIndex:function(C){var D=this.config;for(var B=0,A=D.length;B<A;B++){if(D[B].dataIndex==C){return B}}return -1},isCellEditable:function(A,B){return(this.config[A].editable||(typeof this.config[A].editable==\"undefined\"&&this.config[A].editor))?true:false},getCellEditor:function(A,B){return this.config[A].editor},setEditable:function(A,B){this.config[A].editable=B},isHidden:function(A){return this.config[A].hidden},isFixed:function(A){return this.config[A].fixed},isResizable:function(A){return A>=0&&this.config[A].resizable!==false&&this.config[A].fixed!==true},setHidden:function(A,B){var C=this.config[A];if(C.hidden!==B){C.hidden=B;this.totalWidth=null;this.fireEvent(\"hiddenchange\",this,A,B)}},setEditor:function(A,B){this.config[A].editor=B}});Ext.grid.ColumnModel.defaultRenderer=function(A){if(typeof A==\"string\"&&A.length<1){return\"&#160;\"}return A};Ext.grid.DefaultColumnModel=Ext.grid.ColumnModel;\nExt.grid.AbstractSelectionModel=function(){this.locked=false;Ext.grid.AbstractSelectionModel.superclass.constructor.call(this)};Ext.extend(Ext.grid.AbstractSelectionModel,Ext.util.Observable,{init:function(A){this.grid=A;this.initEvents()},lock:function(){this.locked=true},unlock:function(){this.locked=false},isLocked:function(){return this.locked}});\nExt.grid.RowSelectionModel=function(A){Ext.apply(this,A);this.selections=new Ext.util.MixedCollection(false,function(B){return B.id});this.last=false;this.lastActive=false;this.addEvents(\"selectionchange\",\"beforerowselect\",\"rowselect\",\"rowdeselect\");Ext.grid.RowSelectionModel.superclass.constructor.call(this)};Ext.extend(Ext.grid.RowSelectionModel,Ext.grid.AbstractSelectionModel,{singleSelect:false,initEvents:function(){if(!this.grid.enableDragDrop&&!this.grid.enableDrag){this.grid.on(\"rowmousedown\",this.handleMouseDown,this)}else{this.grid.on(\"rowclick\",function(B,D,C){if(C.button===0&&!C.shiftKey&&!C.ctrlKey){this.selectRow(D,false);B.view.focusRow(D)}},this)}this.rowNav=new Ext.KeyNav(this.grid.getGridEl(),{\"up\":function(C){if(!C.shiftKey){this.selectPrevious(C.shiftKey)}else{if(this.last!==false&&this.lastActive!==false){var B=this.last;this.selectRange(this.last,this.lastActive-1);this.grid.getView().focusRow(this.lastActive);if(B!==false){this.last=B}}else{this.selectFirstRow()}}},\"down\":function(C){if(!C.shiftKey){this.selectNext(C.shiftKey)}else{if(this.last!==false&&this.lastActive!==false){var B=this.last;this.selectRange(this.last,this.lastActive+1);this.grid.getView().focusRow(this.lastActive);if(B!==false){this.last=B}}else{this.selectFirstRow()}}},scope:this});var A=this.grid.view;A.on(\"refresh\",this.onRefresh,this);A.on(\"rowupdated\",this.onRowUpdated,this);A.on(\"rowremoved\",this.onRemove,this)},onRefresh:function(){var F=this.grid.store,B;var D=this.getSelections();this.clearSelections(true);for(var C=0,A=D.length;C<A;C++){var E=D[C];if((B=F.indexOfId(E.id))!=-1){this.selectRow(B,true)}}if(D.length!=this.selections.getCount()){this.fireEvent(\"selectionchange\",this)}},onRemove:function(A,B,C){if(this.selections.remove(C)!==false){this.fireEvent(\"selectionchange\",this)}},onRowUpdated:function(A,B,C){if(this.isSelected(C)){A.onRowSelect(B)}},selectRecords:function(B,E){if(!E){this.clearSelections()}var D=this.grid.store;for(var C=0,A=B.length;C<A;C++){this.selectRow(D.indexOf(B[C]),true)}},getCount:function(){return this.selections.length},selectFirstRow:function(){this.selectRow(0)},selectLastRow:function(A){this.selectRow(this.grid.store.getCount()-1,A)},selectNext:function(A){if(this.hasNext()){this.selectRow(this.last+1,A);this.grid.getView().focusRow(this.last);return true}return false},selectPrevious:function(A){if(this.hasPrevious()){this.selectRow(this.last-1,A);this.grid.getView().focusRow(this.last);return true}return false},hasNext:function(){return this.last!==false&&(this.last+1)<this.grid.store.getCount()},hasPrevious:function(){return !!this.last},getSelections:function(){return[].concat(this.selections.items)},getSelected:function(){return this.selections.itemAt(0)},each:function(E,D){var C=this.getSelections();for(var B=0,A=C.length;B<A;B++){if(E.call(D||this,C[B],B)===false){return false}}return true},clearSelections:function(A){if(this.locked){return }if(A!==true){var C=this.grid.store;var B=this.selections;B.each(function(D){this.deselectRow(C.indexOfId(D.id))},this);B.clear()}else{this.selections.clear()}this.last=false},selectAll:function(){if(this.locked){return }this.selections.clear();for(var B=0,A=this.grid.store.getCount();B<A;B++){this.selectRow(B,true)}},hasSelection:function(){return this.selections.length>0},isSelected:function(A){var B=typeof A==\"number\"?this.grid.store.getAt(A):A;return(B&&this.selections.key(B.id)?true:false)},isIdSelected:function(A){return(this.selections.key(A)?true:false)},handleMouseDown:function(D,F,E){if(E.button!==0||this.isLocked()){return }var A=this.grid.getView();if(E.shiftKey&&this.last!==false){var C=this.last;this.selectRange(C,F,E.ctrlKey);this.last=C;A.focusRow(F)}else{var B=this.isSelected(F);if(E.ctrlKey&&B){this.deselectRow(F)}else{if(!B||this.getCount()>1){this.selectRow(F,E.ctrlKey||E.shiftKey);A.focusRow(F)}}}},selectRows:function(C,D){if(!D){this.clearSelections()}for(var B=0,A=C.length;B<A;B++){this.selectRow(C[B],true)}},selectRange:function(B,A,D){if(this.locked){return }if(!D){this.clearSelections()}if(B<=A){for(var C=B;C<=A;C++){this.selectRow(C,true)}}else{for(var C=B;C>=A;C--){this.selectRow(C,true)}}},deselectRange:function(C,B,A){if(this.locked){return }for(var D=C;D<=B;D++){this.deselectRow(D,A)}},selectRow:function(B,D,A){if(this.locked||(B<0||B>=this.grid.store.getCount())){return }var C=this.grid.store.getAt(B);if(C&&this.fireEvent(\"beforerowselect\",this,B,D,C)!==false){if(!D||this.singleSelect){this.clearSelections()}this.selections.add(C);this.last=this.lastActive=B;if(!A){this.grid.getView().onRowSelect(B)}this.fireEvent(\"rowselect\",this,B,C);this.fireEvent(\"selectionchange\",this)}},deselectRow:function(B,A){if(this.locked){return }if(this.last==B){this.last=false}if(this.lastActive==B){this.lastActive=false}var C=this.grid.store.getAt(B);if(C){this.selections.remove(C);if(!A){this.grid.getView().onRowDeselect(B)}this.fireEvent(\"rowdeselect\",this,B,C);this.fireEvent(\"selectionchange\",this)}},restoreLast:function(){if(this._last){this.last=this._last}},acceptsNav:function(C,B,A){return !A.isHidden(B)&&A.isCellEditable(B,C)},onEditorKey:function(F,E){var C=E.getKey(),G,D=this.grid,B=D.activeEditor;var A=E.shiftKey;if(C==E.TAB){E.stopEvent();B.completeEdit();if(A){G=D.walkCells(B.row,B.col-1,-1,this.acceptsNav,this)}else{G=D.walkCells(B.row,B.col+1,1,this.acceptsNav,this)}}else{if(C==E.ENTER){E.stopEvent();B.completeEdit();if(this.moveEditorOnEnter!==false){if(A){G=D.walkCells(B.row-1,B.col,-1,this.acceptsNav,this)}else{G=D.walkCells(B.row+1,B.col,1,this.acceptsNav,this)}}}else{if(C==E.ESC){B.cancelEdit()}}}if(G){D.startEditing(G[0],G[1])}}});\nExt.grid.CellSelectionModel=function(A){Ext.apply(this,A);this.selection=null;this.addEvents(\"beforecellselect\",\"cellselect\",\"selectionchange\");Ext.grid.CellSelectionModel.superclass.constructor.call(this)};Ext.extend(Ext.grid.CellSelectionModel,Ext.grid.AbstractSelectionModel,{initEvents:function(){this.grid.on(\"cellmousedown\",this.handleMouseDown,this);this.grid.getGridEl().on(Ext.isIE?\"keydown\":\"keypress\",this.handleKeyDown,this);var A=this.grid.view;A.on(\"refresh\",this.onViewChange,this);A.on(\"rowupdated\",this.onRowUpdated,this);A.on(\"beforerowremoved\",this.clearSelections,this);A.on(\"beforerowsinserted\",this.clearSelections,this);if(this.grid.isEditor){this.grid.on(\"beforeedit\",this.beforeEdit,this)}},beforeEdit:function(A){this.select(A.row,A.column,false,true,A.record)},onRowUpdated:function(A,B,C){if(this.selection&&this.selection.record==C){A.onCellSelect(B,this.selection.cell[1])}},onViewChange:function(){this.clearSelections(true)},getSelectedCell:function(){return this.selection?this.selection.cell:null},clearSelections:function(B){var A=this.selection;if(A){if(B!==true){this.grid.view.onCellDeselect(A.cell[0],A.cell[1])}this.selection=null;this.fireEvent(\"selectionchange\",this,null)}},hasSelection:function(){return this.selection?true:false},handleMouseDown:function(B,D,A,C){if(C.button!==0||this.isLocked()){return }this.select(D,A)},select:function(F,C,B,E,D){if(this.fireEvent(\"beforecellselect\",this,F,C)!==false){this.clearSelections();D=D||this.grid.store.getAt(F);this.selection={record:D,cell:[F,C]};if(!B){var A=this.grid.getView();A.onCellSelect(F,C);if(E!==true){A.focusCell(F,C)}}this.fireEvent(\"cellselect\",this,F,C);this.fireEvent(\"selectionchange\",this,this.selection)}},isSelectable:function(C,B,A){return !A.isHidden(B)},handleKeyDown:function(F){if(!F.isNavKeyPress()){return }var E=this.grid,J=this.selection;if(!J){F.stopEvent();var I=E.walkCells(0,0,1,this.isSelectable,this);if(I){this.select(I[0],I[1])}return }var B=this;var H=function(M,K,L){return E.walkCells(M,K,L,B.isSelectable,B)};var C=F.getKey(),A=J.cell[0],G=J.cell[1];var D;switch(C){case F.TAB:if(F.shiftKey){D=H(A,G-1,-1)}else{D=H(A,G+1,1)}break;case F.DOWN:D=H(A+1,G,1);break;case F.UP:D=H(A-1,G,-1);break;case F.RIGHT:D=H(A,G+1,1);break;case F.LEFT:D=H(A,G-1,-1);break;case F.ENTER:if(E.isEditor&&!E.editing){E.startEditing(A,G);F.stopEvent();return }break}if(D){this.select(D[0],D[1]);F.stopEvent()}},acceptsNav:function(C,B,A){return !A.isHidden(B)&&A.isCellEditable(B,C)},onEditorKey:function(E,D){var B=D.getKey(),F,C=this.grid,A=C.activeEditor;if(B==D.TAB){if(D.shiftKey){F=C.walkCells(A.row,A.col-1,-1,this.acceptsNav,this)}else{F=C.walkCells(A.row,A.col+1,1,this.acceptsNav,this)}D.stopEvent()}else{if(B==D.ENTER){A.completeEdit();D.stopEvent()}else{if(B==D.ESC){D.stopEvent();A.cancelEdit()}}}if(F){C.startEditing(F[0],F[1])}}});\nExt.grid.EditorGridPanel=Ext.extend(Ext.grid.GridPanel,{clicksToEdit:2,isEditor:true,detectEdit:false,autoEncode:false,trackMouseOver:false,initComponent:function(){Ext.grid.EditorGridPanel.superclass.initComponent.call(this);if(!this.selModel){this.selModel=new Ext.grid.CellSelectionModel()}this.activeEditor=null;this.addEvents(\"beforeedit\",\"afteredit\",\"validateedit\")},initEvents:function(){Ext.grid.EditorGridPanel.superclass.initEvents.call(this);this.on(\"bodyscroll\",this.stopEditing,this,[true]);if(this.clicksToEdit==1){this.on(\"cellclick\",this.onCellDblClick,this)}else{if(this.clicksToEdit==\"auto\"&&this.view.mainBody){this.view.mainBody.on(\"mousedown\",this.onAutoEditClick,this)}this.on(\"celldblclick\",this.onCellDblClick,this)}this.getGridEl().addClass(\"xedit-grid\")},onCellDblClick:function(B,C,A){this.startEditing(C,A)},onAutoEditClick:function(C,B){if(C.button!==0){return }var E=this.view.findRowIndex(B);var A=this.view.findCellIndex(B);if(E!==false&&A!==false){this.stopEditing();if(this.selModel.getSelectedCell){var D=this.selModel.getSelectedCell();if(D&&D.cell[0]===E&&D.cell[1]===A){this.startEditing(E,A)}}else{if(this.selModel.isSelected(E)){this.startEditing(E,A)}}}},onEditComplete:function(B,D,A){this.editing=false;this.activeEditor=null;B.un(\"specialkey\",this.selModel.onEditorKey,this.selModel);var C=B.record;var F=this.colModel.getDataIndex(B.col);D=this.postEditValue(D,A,C,F);if(String(D)!==String(A)){var E={grid:this,record:C,field:F,originalValue:A,value:D,row:B.row,column:B.col,cancel:false};if(this.fireEvent(\"validateedit\",E)!==false&&!E.cancel){C.set(F,E.value);delete E.cancel;this.fireEvent(\"afteredit\",E)}}this.view.focusCell(B.row,B.col)},startEditing:function(F,B){this.stopEditing();if(this.colModel.isCellEditable(B,F)){this.view.ensureVisible(F,B,true);var C=this.store.getAt(F);var E=this.colModel.getDataIndex(B);var D={grid:this,record:C,field:E,value:C.data[E],row:F,column:B,cancel:false};if(this.fireEvent(\"beforeedit\",D)!==false&&!D.cancel){this.editing=true;var A=this.colModel.getCellEditor(B,F);if(!A.rendered){A.render(this.view.getEditorParent(A))}(function(){A.row=F;A.col=B;A.record=C;A.on(\"complete\",this.onEditComplete,this,{single:true});A.on(\"specialkey\",this.selModel.onEditorKey,this.selModel);this.activeEditor=A;var G=this.preEditValue(C,E);A.startEdit(this.view.getCell(F,B),G)}).defer(50,this)}}},preEditValue:function(A,B){return this.autoEncode&&typeof value==\"string\"?Ext.util.Format.htmlDecode(A.data[B]):A.data[B]},postEditValue:function(C,A,B,D){return this.autoEncode&&typeof C==\"string\"?Ext.util.Format.htmlEncode(C):C},stopEditing:function(A){if(this.activeEditor){this.activeEditor[A===true?\"cancelEdit\":\"completeEdit\"]()}this.activeEditor=null}});Ext.reg(\"editorgrid\",Ext.grid.EditorGridPanel);\nExt.grid.GridEditor=function(B,A){Ext.grid.GridEditor.superclass.constructor.call(this,B,A);B.monitorTab=false};Ext.extend(Ext.grid.GridEditor,Ext.Editor,{alignment:\"tl-tl\",autoSize:\"width\",hideEl:false,cls:\"x-small-editor x-grid-editor\",shim:false,shadow:false});\nExt.grid.PropertyRecord=Ext.data.Record.create([{name:\"name\",type:\"string\"},\"value\"]);Ext.grid.PropertyStore=function(A,B){this.grid=A;this.store=new Ext.data.Store({recordType:Ext.grid.PropertyRecord});this.store.on(\"update\",this.onUpdate,this);if(B){this.setSource(B)}Ext.grid.PropertyStore.superclass.constructor.call(this)};Ext.extend(Ext.grid.PropertyStore,Ext.util.Observable,{setSource:function(C){this.source=C;this.store.removeAll();var B=[];for(var A in C){if(this.isEditableValue(C[A])){B.push(new Ext.grid.PropertyRecord({name:A,value:C[A]},A))}}this.store.loadRecords({records:B},{},true)},onUpdate:function(E,A,D){if(D==Ext.data.Record.EDIT){var B=A.data[\"value\"];var C=A.modified[\"value\"];if(this.grid.fireEvent(\"beforepropertychange\",this.source,A.id,B,C)!==false){this.source[A.id]=B;A.commit();this.grid.fireEvent(\"propertychange\",this.source,A.id,B,C)}else{A.reject()}}},getProperty:function(A){return this.store.getAt(A)},isEditableValue:function(A){if(Ext.isDate(A)){return true}else{if(typeof A==\"object\"||typeof A==\"function\"){return false}}return true},setValue:function(B,A){this.source[B]=A;this.store.getById(B).set(\"value\",A)},getSource:function(){return this.source}});Ext.grid.PropertyColumnModel=function(C,B){this.grid=C;var D=Ext.grid;D.PropertyColumnModel.superclass.constructor.call(this,[{header:this.nameText,width:50,sortable:true,dataIndex:\"name\",id:\"name\",menuDisabled:true},{header:this.valueText,width:50,resizable:false,dataIndex:\"value\",id:\"value\",menuDisabled:true}]);this.store=B;this.bselect=Ext.DomHelper.append(document.body,{tag:\"select\",cls:\"x-grid-editor x-hide-display\",children:[{tag:\"option\",value:\"true\",html:\"true\"},{tag:\"option\",value:\"false\",html:\"false\"}]});var E=Ext.form;var A=new E.Field({el:this.bselect,bselect:this.bselect,autoShow:true,getValue:function(){return this.bselect.value==\"true\"}});this.editors={\"date\":new D.GridEditor(new E.DateField({selectOnFocus:true})),\"string\":new D.GridEditor(new E.TextField({selectOnFocus:true})),\"number\":new D.GridEditor(new E.NumberField({selectOnFocus:true,style:\"text-align:left;\"})),\"boolean\":new D.GridEditor(A)};this.renderCellDelegate=this.renderCell.createDelegate(this);this.renderPropDelegate=this.renderProp.createDelegate(this)};Ext.extend(Ext.grid.PropertyColumnModel,Ext.grid.ColumnModel,{nameText:\"Name\",valueText:\"Value\",dateFormat:\"m/j/Y\",renderDate:function(A){return A.dateFormat(this.dateFormat)},renderBool:function(A){return A?\"true\":\"false\"},isCellEditable:function(A,B){return A==1},getRenderer:function(A){return A==1?this.renderCellDelegate:this.renderPropDelegate},renderProp:function(A){return this.getPropertyName(A)},renderCell:function(A){var B=A;if(Ext.isDate(A)){B=this.renderDate(A)}else{if(typeof A==\"boolean\"){B=this.renderBool(A)}}return Ext.util.Format.htmlEncode(B)},getPropertyName:function(B){var A=this.grid.propertyNames;return A&&A[B]?A[B]:B},getCellEditor:function(A,E){var B=this.store.getProperty(E);var D=B.data[\"name\"],C=B.data[\"value\"];if(this.grid.customEditors[D]){return this.grid.customEditors[D]}if(Ext.isDate(C)){return this.editors[\"date\"]}else{if(typeof C==\"number\"){return this.editors[\"number\"]}else{if(typeof C==\"boolean\"){return this.editors[\"boolean\"]}else{return this.editors[\"string\"]}}}}});Ext.grid.PropertyGrid=Ext.extend(Ext.grid.EditorGridPanel,{enableColumnMove:false,stripeRows:false,trackMouseOver:false,clicksToEdit:1,enableHdMenu:false,viewConfig:{forceFit:true},initComponent:function(){this.customEditors=this.customEditors||{};this.lastEditRow=null;var B=new Ext.grid.PropertyStore(this);this.propStore=B;var A=new Ext.grid.PropertyColumnModel(this,B);B.store.sort(\"name\",\"ASC\");this.addEvents(\"beforepropertychange\",\"propertychange\");this.cm=A;this.ds=B.store;Ext.grid.PropertyGrid.superclass.initComponent.call(this);this.selModel.on(\"beforecellselect\",function(E,D,C){if(C===0){this.startEditing.defer(200,this,[D,1]);return false}},this)},onRender:function(){Ext.grid.PropertyGrid.superclass.onRender.apply(this,arguments);this.getGridEl().addClass(\"x-props-grid\")},afterRender:function(){Ext.grid.PropertyGrid.superclass.afterRender.apply(this,arguments);if(this.source){this.setSource(this.source)}},setSource:function(A){this.propStore.setSource(A)},getSource:function(){return this.propStore.getSource()}});\nExt.grid.RowNumberer=function(A){Ext.apply(this,A);if(this.rowspan){this.renderer=this.renderer.createDelegate(this)}};Ext.grid.RowNumberer.prototype={header:\"\",width:23,sortable:false,fixed:true,menuDisabled:true,dataIndex:\"\",id:\"numberer\",rowspan:undefined,renderer:function(B,C,A,D){if(this.rowspan){C.cellAttr=\"rowspan=\\\"\"+this.rowspan+\"\\\"\"}return D+1}};\nExt.grid.CheckboxSelectionModel=Ext.extend(Ext.grid.RowSelectionModel,{header:\"<div class=\\\"x-grid3-hd-checker\\\">&#160;</div>\",width:20,sortable:false,menuDisabled:true,fixed:true,dataIndex:\"\",id:\"checker\",initEvents:function(){Ext.grid.CheckboxSelectionModel.superclass.initEvents.call(this);this.grid.on(\"render\",function(){var A=this.grid.getView();A.mainBody.on(\"mousedown\",this.onMouseDown,this);Ext.fly(A.innerHd).on(\"mousedown\",this.onHdMouseDown,this)},this)},onMouseDown:function(C,B){if(C.button===0&&B.className==\"x-grid3-row-checker\"){C.stopEvent();var D=C.getTarget(\".x-grid3-row\");if(D){var A=D.rowIndex;if(this.isSelected(A)){this.deselectRow(A)}else{this.selectRow(A,true)}}}},onHdMouseDown:function(C,A){if(A.className==\"x-grid3-hd-checker\"){C.stopEvent();var B=Ext.fly(A.parentNode);var D=B.hasClass(\"x-grid3-hd-checker-on\");if(D){B.removeClass(\"x-grid3-hd-checker-on\");this.clearSelections()}else{B.addClass(\"x-grid3-hd-checker-on\");this.selectAll()}}},renderer:function(B,C,A){return\"<div class=\\\"x-grid3-row-checker\\\">&#160;</div>\"}});\nExt.LoadMask=function(C,B){this.el=Ext.get(C);Ext.apply(this,B);if(this.store){this.store.on(\"beforeload\",this.onBeforeLoad,this);this.store.on(\"load\",this.onLoad,this);this.store.on(\"loadexception\",this.onLoad,this);this.removeMask=Ext.value(this.removeMask,false)}else{var A=this.el.getUpdater();A.showLoadIndicator=false;A.on(\"beforeupdate\",this.onBeforeLoad,this);A.on(\"update\",this.onLoad,this);A.on(\"failure\",this.onLoad,this);this.removeMask=Ext.value(this.removeMask,true)}};Ext.LoadMask.prototype={msg:\"Loading...\",msgCls:\"x-mask-loading\",disabled:false,disable:function(){this.disabled=true},enable:function(){this.disabled=false},onLoad:function(){this.el.unmask(this.removeMask)},onBeforeLoad:function(){if(!this.disabled){this.el.mask(this.msg,this.msgCls)}},show:function(){this.onBeforeLoad()},hide:function(){this.onLoad()},destroy:function(){if(this.store){this.store.un(\"beforeload\",this.onBeforeLoad,this);this.store.un(\"load\",this.onLoad,this);this.store.un(\"loadexception\",this.onLoad,this)}else{var A=this.el.getUpdater();A.un(\"beforeupdate\",this.onBeforeLoad,this);A.un(\"update\",this.onLoad,this);A.un(\"failure\",this.onLoad,this)}}};\nExt.ProgressBar=Ext.extend(Ext.BoxComponent,{baseCls:\"x-progress\",waitTimer:null,initComponent:function(){Ext.ProgressBar.superclass.initComponent.call(this);this.addEvents(\"update\")},onRender:function(D,A){Ext.ProgressBar.superclass.onRender.call(this,D,A);var C=new Ext.Template(\"<div class=\\\"{cls}-wrap\\\">\",\"<div class=\\\"{cls}-inner\\\">\",\"<div class=\\\"{cls}-bar\\\">\",\"<div class=\\\"{cls}-text\\\">\",\"<div>&#160;</div>\",\"</div>\",\"</div>\",\"<div class=\\\"{cls}-text {cls}-text-back\\\">\",\"<div>&#160;</div>\",\"</div>\",\"</div>\",\"</div>\");if(A){this.el=C.insertBefore(A,{cls:this.baseCls},true)}else{this.el=C.append(D,{cls:this.baseCls},true)}if(this.id){this.el.dom.id=this.id}var B=this.el.dom.firstChild;this.progressBar=Ext.get(B.firstChild);if(this.textEl){this.textEl=Ext.get(this.textEl);delete this.textTopEl}else{this.textTopEl=Ext.get(this.progressBar.dom.firstChild);var E=Ext.get(B.childNodes[1]);this.textTopEl.setStyle(\"z-index\",99).addClass(\"x-hidden\");this.textEl=new Ext.CompositeElement([this.textTopEl.dom.firstChild,E.dom.firstChild]);this.textEl.setWidth(B.offsetWidth)}if(this.value){this.updateProgress(this.value,this.text)}else{this.updateText(this.text)}this.setSize(this.width||\"auto\",\"auto\");this.progressBar.setHeight(B.offsetHeight)},updateProgress:function(B,C){this.value=B||0;if(C){this.updateText(C)}var A=Math.floor(B*this.el.dom.firstChild.offsetWidth);this.progressBar.setWidth(A);if(this.textTopEl){this.textTopEl.removeClass(\"x-hidden\").setWidth(A)}this.fireEvent(\"update\",this,B,C);return this},wait:function(B){if(!this.waitTimer){var A=this;B=B||{};this.waitTimer=Ext.TaskMgr.start({run:function(C){var D=B.increment||10;this.updateProgress(((((C+D)%D)+1)*(100/D))*0.01)},interval:B.interval||1000,duration:B.duration,onStop:function(){if(B.fn){B.fn.apply(B.scope||this)}this.reset()},scope:A})}return this},isWaiting:function(){return this.waitTimer!=null},updateText:function(A){this.text=A||\"&#160;\";this.textEl.update(this.text);return this},setSize:function(A,C){Ext.ProgressBar.superclass.setSize.call(this,A,C);if(this.textTopEl){var B=this.el.dom.firstChild;this.textEl.setSize(B.offsetWidth,B.offsetHeight)}return this},reset:function(A){this.updateProgress(0);if(this.textTopEl){this.textTopEl.addClass(\"x-hidden\")}if(this.waitTimer){this.waitTimer.onStop=null;Ext.TaskMgr.stop(this.waitTimer);this.waitTimer=null}if(A===true){this.hide()}return this}});Ext.reg(\"progress\",Ext.ProgressBar);\n"
  },
  {
    "path": "docroot/ext-base.js",
    "content": "/*\r\n * Ext JS Library 2.0.2\r\n * Copyright(c) 2006-2008, Ext JS, LLC.\r\n * licensing@extjs.com\r\n * \r\n * http://extjs.com/license\r\n */\r\n\r\nExt={version:\"2.0.2\"};window[\"undefined\"]=window[\"undefined\"];Ext.apply=function(C,D,B){if(B){Ext.apply(C,B)}if(C&&D&&typeof D==\"object\"){for(var A in D){C[A]=D[A]}}return C};(function(){var idSeed=0;var ua=navigator.userAgent.toLowerCase();var isStrict=document.compatMode==\"CSS1Compat\",isOpera=ua.indexOf(\"opera\")>-1,isSafari=(/webkit|khtml/).test(ua),isSafari3=isSafari&&ua.indexOf(\"webkit/5\")!=-1,isIE=!isOpera&&ua.indexOf(\"msie\")>-1,isIE7=!isOpera&&ua.indexOf(\"msie 7\")>-1,isGecko=!isSafari&&ua.indexOf(\"gecko\")>-1,isBorderBox=isIE&&!isStrict,isWindows=(ua.indexOf(\"windows\")!=-1||ua.indexOf(\"win32\")!=-1),isMac=(ua.indexOf(\"macintosh\")!=-1||ua.indexOf(\"mac os x\")!=-1),isAir=(ua.indexOf(\"adobeair\")!=-1),isLinux=(ua.indexOf(\"linux\")!=-1),isSecure=window.location.href.toLowerCase().indexOf(\"https\")===0;if(isIE&&!isIE7){try{document.execCommand(\"BackgroundImageCache\",false,true)}catch(e){}}Ext.apply(Ext,{isStrict:isStrict,isSecure:isSecure,isReady:false,enableGarbageCollector:true,enableListenerCollection:false,SSL_SECURE_URL:\"javascript:false\",BLANK_IMAGE_URL:\"http:/\"+\"/extjs.com/s.gif\",emptyFn:function(){},applyIf:function(o,c){if(o&&c){for(var p in c){if(typeof o[p]==\"undefined\"){o[p]=c[p]}}}return o},addBehaviors:function(o){if(!Ext.isReady){Ext.onReady(function(){Ext.addBehaviors(o)});return }var cache={};for(var b in o){var parts=b.split(\"@\");if(parts[1]){var s=parts[0];if(!cache[s]){cache[s]=Ext.select(s)}cache[s].on(parts[1],o[b])}}cache=null},id:function(el,prefix){prefix=prefix||\"ext-gen\";el=Ext.getDom(el);var id=prefix+(++idSeed);return el?(el.id?el.id:(el.id=id)):id},extend:function(){var io=function(o){for(var m in o){this[m]=o[m]}};var oc=Object.prototype.constructor;return function(sb,sp,overrides){if(typeof sp==\"object\"){overrides=sp;sp=sb;sb=overrides.constructor!=oc?overrides.constructor:function(){sp.apply(this,arguments)}}var F=function(){},sbp,spp=sp.prototype;F.prototype=spp;sbp=sb.prototype=new F();sbp.constructor=sb;sb.superclass=spp;if(spp.constructor==oc){spp.constructor=sp}sb.override=function(o){Ext.override(sb,o)};sbp.override=io;Ext.override(sb,overrides);sb.extend=function(o){Ext.extend(sb,o)};return sb}}(),override:function(origclass,overrides){if(overrides){var p=origclass.prototype;for(var method in overrides){p[method]=overrides[method]}}},namespace:function(){var a=arguments,o=null,i,j,d,rt;for(i=0;i<a.length;++i){d=a[i].split(\".\");rt=d[0];eval(\"if (typeof \"+rt+\" == \\\"undefined\\\"){\"+rt+\" = {};} o = \"+rt+\";\");for(j=1;j<d.length;++j){o[d[j]]=o[d[j]]||{};o=o[d[j]]}}},urlEncode:function(o){if(!o){return\"\"}var buf=[];for(var key in o){var ov=o[key],k=encodeURIComponent(key);var type=typeof ov;if(type==\"undefined\"){buf.push(k,\"=&\")}else{if(type!=\"function\"&&type!=\"object\"){buf.push(k,\"=\",encodeURIComponent(ov),\"&\")}else{if(Ext.isArray(ov)){if(ov.length){for(var i=0,len=ov.length;i<len;i++){buf.push(k,\"=\",encodeURIComponent(ov[i]===undefined?\"\":ov[i]),\"&\")}}else{buf.push(k,\"=&\")}}}}}buf.pop();return buf.join(\"\")},urlDecode:function(string,overwrite){if(!string||!string.length){return{}}var obj={};var pairs=string.split(\"&\");var pair,name,value;for(var i=0,len=pairs.length;i<len;i++){pair=pairs[i].split(\"=\");name=decodeURIComponent(pair[0]);value=decodeURIComponent(pair[1]);if(overwrite!==true){if(typeof obj[name]==\"undefined\"){obj[name]=value}else{if(typeof obj[name]==\"string\"){obj[name]=[obj[name]];obj[name].push(value)}else{obj[name].push(value)}}}else{obj[name]=value}}return obj},each:function(array,fn,scope){if(typeof array.length==\"undefined\"||typeof array==\"string\"){array=[array]}for(var i=0,len=array.length;i<len;i++){if(fn.call(scope||array[i],array[i],i,array)===false){return i}}},combine:function(){var as=arguments,l=as.length,r=[];for(var i=0;i<l;i++){var a=as[i];if(Ext.isArray(a)){r=r.concat(a)}else{if(a.length!==undefined&&!a.substr){r=r.concat(Array.prototype.slice.call(a,0))}else{r.push(a)}}}return r},escapeRe:function(s){return s.replace(/([.*+?^${}()|[\\]\\/\\\\])/g,\"\\\\$1\")},callback:function(cb,scope,args,delay){if(typeof cb==\"function\"){if(delay){cb.defer(delay,scope,args||[])}else{cb.apply(scope,args||[])}}},getDom:function(el){if(!el||!document){return null}return el.dom?el.dom:(typeof el==\"string\"?document.getElementById(el):el)},getDoc:function(){return Ext.get(document)},getBody:function(){return Ext.get(document.body||document.documentElement)},getCmp:function(id){return Ext.ComponentMgr.get(id)},num:function(v,defaultValue){if(typeof v!=\"number\"){return defaultValue}return v},destroy:function(){for(var i=0,a=arguments,len=a.length;i<len;i++){var as=a[i];if(as){if(typeof as.destroy==\"function\"){as.destroy()}else{if(as.dom){as.removeAllListeners();as.remove()}}}}},removeNode:isIE?function(){var d;return function(n){if(n&&n.tagName!=\"BODY\"){d=d||document.createElement(\"div\");d.appendChild(n);d.innerHTML=\"\"}}}():function(n){if(n&&n.parentNode&&n.tagName!=\"BODY\"){n.parentNode.removeChild(n)}},type:function(o){if(o===undefined||o===null){return false}if(o.htmlElement){return\"element\"}var t=typeof o;if(t==\"object\"&&o.nodeName){switch(o.nodeType){case 1:return\"element\";case 3:return(/\\S/).test(o.nodeValue)?\"textnode\":\"whitespace\"}}if(t==\"object\"||t==\"function\"){switch(o.constructor){case Array:return\"array\";case RegExp:return\"regexp\"}if(typeof o.length==\"number\"&&typeof o.item==\"function\"){return\"nodelist\"}}return t},isEmpty:function(v,allowBlank){return v===null||v===undefined||(!allowBlank?v===\"\":false)},value:function(v,defaultValue,allowBlank){return Ext.isEmpty(v,allowBlank)?defaultValue:v},isArray:function(v){return v&&typeof v.pop==\"function\"},isDate:function(v){return v&&typeof v.getFullYear==\"function\"},isOpera:isOpera,isSafari:isSafari,isSafari3:isSafari3,isSafari2:isSafari&&!isSafari3,isIE:isIE,isIE6:isIE&&!isIE7,isIE7:isIE7,isGecko:isGecko,isBorderBox:isBorderBox,isLinux:isLinux,isWindows:isWindows,isMac:isMac,isAir:isAir,useShims:((isIE&&!isIE7)||(isGecko&&isMac))});Ext.ns=Ext.namespace})();Ext.ns(\"Ext\",\"Ext.util\",\"Ext.grid\",\"Ext.dd\",\"Ext.tree\",\"Ext.data\",\"Ext.form\",\"Ext.menu\",\"Ext.state\",\"Ext.lib\",\"Ext.layout\",\"Ext.app\",\"Ext.ux\");Ext.apply(Function.prototype,{createCallback:function(){var A=arguments;var B=this;return function(){return B.apply(window,A)}},createDelegate:function(C,B,A){var D=this;return function(){var F=B||arguments;if(A===true){F=Array.prototype.slice.call(arguments,0);F=F.concat(B)}else{if(typeof A==\"number\"){F=Array.prototype.slice.call(arguments,0);var E=[A,0].concat(B);Array.prototype.splice.apply(F,E)}}return D.apply(C||window,F)}},defer:function(C,E,B,A){var D=this.createDelegate(E,B,A);if(C){return setTimeout(D,C)}D();return 0},createSequence:function(B,A){if(typeof B!=\"function\"){return this}var C=this;return function(){var D=C.apply(this||window,arguments);B.apply(A||this||window,arguments);return D}},createInterceptor:function(B,A){if(typeof B!=\"function\"){return this}var C=this;return function(){B.target=this;B.method=C;if(B.apply(A||this||window,arguments)===false){return }return C.apply(this||window,arguments)}}});Ext.applyIf(String,{escape:function(A){return A.replace(/('|\\\\)/g,\"\\\\$1\")},leftPad:function(D,B,C){var A=new String(D);if(!C){C=\" \"}while(A.length<B){A=C+A}return A.toString()},format:function(B){var A=Array.prototype.slice.call(arguments,1);return B.replace(/\\{(\\d+)\\}/g,function(C,D){return A[D]})}});String.prototype.toggle=function(B,A){return this==B?A:B};String.prototype.trim=function(){var A=/^\\s+|\\s+$/g;return function(){return this.replace(A,\"\")}}();Ext.applyIf(Number.prototype,{constrain:function(B,A){return Math.min(Math.max(this,B),A)}});Ext.applyIf(Array.prototype,{indexOf:function(C){for(var B=0,A=this.length;B<A;B++){if(this[B]==C){return B}}return -1},remove:function(B){var A=this.indexOf(B);if(A!=-1){this.splice(A,1)}return this}});Date.prototype.getElapsed=function(A){return Math.abs((A||new Date()).getTime()-this.getTime())};\n(function(){var B;Ext.lib.Dom={getViewWidth:function(E){return E?this.getDocumentWidth():this.getViewportWidth()},getViewHeight:function(E){return E?this.getDocumentHeight():this.getViewportHeight()},getDocumentHeight:function(){var E=(document.compatMode!=\"CSS1Compat\")?document.body.scrollHeight:document.documentElement.scrollHeight;return Math.max(E,this.getViewportHeight())},getDocumentWidth:function(){var E=(document.compatMode!=\"CSS1Compat\")?document.body.scrollWidth:document.documentElement.scrollWidth;return Math.max(E,this.getViewportWidth())},getViewportHeight:function(){if(Ext.isIE){return Ext.isStrict?document.documentElement.clientHeight:document.body.clientHeight}else{return self.innerHeight}},getViewportWidth:function(){if(Ext.isIE){return Ext.isStrict?document.documentElement.clientWidth:document.body.clientWidth}else{return self.innerWidth}},isAncestor:function(F,G){F=Ext.getDom(F);G=Ext.getDom(G);if(!F||!G){return false}if(F.contains&&!Ext.isSafari){return F.contains(G)}else{if(F.compareDocumentPosition){return !!(F.compareDocumentPosition(G)&16)}else{var E=G.parentNode;while(E){if(E==F){return true}else{if(!E.tagName||E.tagName.toUpperCase()==\"HTML\"){return false}}E=E.parentNode}return false}}},getRegion:function(E){return Ext.lib.Region.getRegion(E)},getY:function(E){return this.getXY(E)[1]},getX:function(E){return this.getXY(E)[0]},getXY:function(G){var F,K,M,N,J=(document.body||document.documentElement);G=Ext.getDom(G);if(G==J){return[0,0]}if(G.getBoundingClientRect){M=G.getBoundingClientRect();N=C(document).getScroll();return[M.left+N.left,M.top+N.top]}var O=0,L=0;F=G;var E=C(G).getStyle(\"position\")==\"absolute\";while(F){O+=F.offsetLeft;L+=F.offsetTop;if(!E&&C(F).getStyle(\"position\")==\"absolute\"){E=true}if(Ext.isGecko){K=C(F);var P=parseInt(K.getStyle(\"borderTopWidth\"),10)||0;var H=parseInt(K.getStyle(\"borderLeftWidth\"),10)||0;O+=H;L+=P;if(F!=G&&K.getStyle(\"overflow\")!=\"visible\"){O+=H;L+=P}}F=F.offsetParent}if(Ext.isSafari&&E){O-=J.offsetLeft;L-=J.offsetTop}if(Ext.isGecko&&!E){var I=C(J);O+=parseInt(I.getStyle(\"borderLeftWidth\"),10)||0;L+=parseInt(I.getStyle(\"borderTopWidth\"),10)||0}F=G.parentNode;while(F&&F!=J){if(!Ext.isOpera||(F.tagName!=\"TR\"&&C(F).getStyle(\"display\")!=\"inline\")){O-=F.scrollLeft;L-=F.scrollTop}F=F.parentNode}return[O,L]},setXY:function(E,F){E=Ext.fly(E,\"_setXY\");E.position();var G=E.translatePoints(F);if(F[0]!==false){E.dom.style.left=G.left+\"px\"}if(F[1]!==false){E.dom.style.top=G.top+\"px\"}},setX:function(F,E){this.setXY(F,[E,false])},setY:function(E,F){this.setXY(E,[false,F])}};Ext.lib.Event=function(){var F=false;var G=[];var K=[];var I=0;var H=[];var E=0;var J=null;return{POLL_RETRYS:200,POLL_INTERVAL:20,EL:0,TYPE:1,FN:2,WFN:3,OBJ:3,ADJ_SCOPE:4,_interval:null,startInterval:function(){if(!this._interval){var L=this;var M=function(){L._tryPreloadAttach()};this._interval=setInterval(M,this.POLL_INTERVAL)}},onAvailable:function(N,L,O,M){H.push({id:N,fn:L,obj:O,override:M,checkReady:false});I=this.POLL_RETRYS;this.startInterval()},addListener:function(Q,M,P){Q=Ext.getDom(Q);if(!Q||!P){return false}if(\"unload\"==M){K[K.length]=[Q,M,P];return true}var O=function(R){return typeof Ext!=\"undefined\"?P(Ext.lib.Event.getEvent(R)):false};var L=[Q,M,P,O];var N=G.length;G[N]=L;this.doAdd(Q,M,O,false);return true},removeListener:function(S,O,R){var Q,N;S=Ext.getDom(S);if(!R){return this.purgeElement(S,false,O)}if(\"unload\"==O){for(Q=0,N=K.length;Q<N;Q++){var M=K[Q];if(M&&M[0]==S&&M[1]==O&&M[2]==R){K.splice(Q,1);return true}}return false}var L=null;var P=arguments[3];if(\"undefined\"==typeof P){P=this._getCacheIndex(S,O,R)}if(P>=0){L=G[P]}if(!S||!L){return false}this.doRemove(S,O,L[this.WFN],false);delete G[P][this.WFN];delete G[P][this.FN];G.splice(P,1);return true},getTarget:function(N,M){N=N.browserEvent||N;var L=N.target||N.srcElement;return this.resolveTextNode(L)},resolveTextNode:function(L){if(Ext.isSafari&&L&&3==L.nodeType){return L.parentNode}else{return L}},getPageX:function(M){M=M.browserEvent||M;var L=M.pageX;if(!L&&0!==L){L=M.clientX||0;if(Ext.isIE){L+=this.getScroll()[1]}}return L},getPageY:function(L){L=L.browserEvent||L;var M=L.pageY;if(!M&&0!==M){M=L.clientY||0;if(Ext.isIE){M+=this.getScroll()[0]}}return M},getXY:function(L){L=L.browserEvent||L;return[this.getPageX(L),this.getPageY(L)]},getRelatedTarget:function(M){M=M.browserEvent||M;var L=M.relatedTarget;if(!L){if(M.type==\"mouseout\"){L=M.toElement}else{if(M.type==\"mouseover\"){L=M.fromElement}}}return this.resolveTextNode(L)},getTime:function(N){N=N.browserEvent||N;if(!N.time){var M=new Date().getTime();try{N.time=M}catch(L){this.lastError=L;return M}}return N.time},stopEvent:function(L){this.stopPropagation(L);this.preventDefault(L)},stopPropagation:function(L){L=L.browserEvent||L;if(L.stopPropagation){L.stopPropagation()}else{L.cancelBubble=true}},preventDefault:function(L){L=L.browserEvent||L;if(L.preventDefault){L.preventDefault()}else{L.returnValue=false}},getEvent:function(M){var L=M||window.event;if(!L){var N=this.getEvent.caller;while(N){L=N.arguments[0];if(L&&Event==L.constructor){break}N=N.caller}}return L},getCharCode:function(L){L=L.browserEvent||L;return L.charCode||L.keyCode||0},_getCacheIndex:function(Q,N,P){for(var O=0,M=G.length;O<M;++O){var L=G[O];if(L&&L[this.FN]==P&&L[this.EL]==Q&&L[this.TYPE]==N){return O}}return -1},elCache:{},getEl:function(L){return document.getElementById(L)},clearCache:function(){},_load:function(M){F=true;var L=Ext.lib.Event;if(Ext.isIE){L.doRemove(window,\"load\",L._load)}},_tryPreloadAttach:function(){if(this.locked){return false}this.locked=true;var R=!F;if(!R){R=(I>0)}var Q=[];for(var M=0,L=H.length;M<L;++M){var P=H[M];if(P){var O=this.getEl(P.id);if(O){if(!P.checkReady||F||O.nextSibling||(document&&document.body)){var N=O;if(P.override){if(P.override===true){N=P.obj}else{N=P.override}}P.fn.call(N,P.obj);H[M]=null}}else{Q.push(P)}}}I=(Q.length===0)?0:I-1;if(R){this.startInterval()}else{clearInterval(this._interval);this._interval=null}this.locked=false;return true},purgeElement:function(P,Q,N){var R=this.getListeners(P,N);if(R){for(var O=0,L=R.length;O<L;++O){var M=R[O];this.removeListener(P,M.type,M.fn)}}if(Q&&P&&P.childNodes){for(O=0,L=P.childNodes.length;O<L;++O){this.purgeElement(P.childNodes[O],Q,N)}}},getListeners:function(M,R){var P=[],L;if(!R){L=[G,K]}else{if(R==\"unload\"){L=[K]}else{L=[G]}}for(var O=0;O<L.length;++O){var T=L[O];if(T&&T.length>0){for(var Q=0,S=T.length;Q<S;++Q){var N=T[Q];if(N&&N[this.EL]===M&&(!R||R===N[this.TYPE])){P.push({type:N[this.TYPE],fn:N[this.FN],obj:N[this.OBJ],adjust:N[this.ADJ_SCOPE],index:Q})}}}}return(P.length)?P:null},_unload:function(S){var R=Ext.lib.Event,P,O,M,L,N;for(P=0,L=K.length;P<L;++P){M=K[P];if(M){var Q=window;if(M[R.ADJ_SCOPE]){if(M[R.ADJ_SCOPE]===true){Q=M[R.OBJ]}else{Q=M[R.ADJ_SCOPE]}}M[R.FN].call(Q,R.getEvent(S),M[R.OBJ]);K[P]=null;M=null;Q=null}}K=null;if(G&&G.length>0){O=G.length;while(O){N=O-1;M=G[N];if(M){R.removeListener(M[R.EL],M[R.TYPE],M[R.FN],N)}O=O-1}M=null;R.clearCache()}R.doRemove(window,\"unload\",R._unload)},getScroll:function(){var L=document.documentElement,M=document.body;if(L&&(L.scrollTop||L.scrollLeft)){return[L.scrollTop,L.scrollLeft]}else{if(M){return[M.scrollTop,M.scrollLeft]}else{return[0,0]}}},doAdd:function(){if(window.addEventListener){return function(O,M,N,L){O.addEventListener(M,N,(L))}}else{if(window.attachEvent){return function(O,M,N,L){O.attachEvent(\"on\"+M,N)}}else{return function(){}}}}(),doRemove:function(){if(window.removeEventListener){return function(O,M,N,L){O.removeEventListener(M,N,(L))}}else{if(window.detachEvent){return function(N,L,M){N.detachEvent(\"on\"+L,M)}}else{return function(){}}}}()}}();var D=Ext.lib.Event;D.on=D.addListener;D.un=D.removeListener;if(document&&document.body){D._load()}else{D.doAdd(window,\"load\",D._load)}D.doAdd(window,\"unload\",D._unload);D._tryPreloadAttach();Ext.lib.Ajax={request:function(K,I,E,J,F){if(F){var G=F.headers;if(G){for(var H in G){if(G.hasOwnProperty(H)){this.initHeader(H,G[H],false)}}}if(F.xmlData){this.initHeader(\"Content-Type\",\"text/xml\",false);K=\"POST\";J=F.xmlData}else{if(F.jsonData){this.initHeader(\"Content-Type\",\"text/javascript\",false);K=\"POST\";J=typeof F.jsonData==\"object\"?Ext.encode(F.jsonData):F.jsonData}}}return this.asyncRequest(K,I,E,J)},serializeForm:function(F){if(typeof F==\"string\"){F=(document.getElementById(F)||document.forms[F])}var G,E,H,J,K=\"\",M=false;for(var L=0;L<F.elements.length;L++){G=F.elements[L];J=F.elements[L].disabled;E=F.elements[L].name;H=F.elements[L].value;if(!J&&E){switch(G.type){case\"select-one\":case\"select-multiple\":for(var I=0;I<G.options.length;I++){if(G.options[I].selected){if(Ext.isIE){K+=encodeURIComponent(E)+\"=\"+encodeURIComponent(G.options[I].attributes[\"value\"].specified?G.options[I].value:G.options[I].text)+\"&\"}else{K+=encodeURIComponent(E)+\"=\"+encodeURIComponent(G.options[I].hasAttribute(\"value\")?G.options[I].value:G.options[I].text)+\"&\"}}}break;case\"radio\":case\"checkbox\":if(G.checked){K+=encodeURIComponent(E)+\"=\"+encodeURIComponent(H)+\"&\"}break;case\"file\":case undefined:case\"reset\":case\"button\":break;case\"submit\":if(M==false){K+=encodeURIComponent(E)+\"=\"+encodeURIComponent(H)+\"&\";M=true}break;default:K+=encodeURIComponent(E)+\"=\"+encodeURIComponent(H)+\"&\";break}}}K=K.substr(0,K.length-1);return K},headers:{},hasHeaders:false,useDefaultHeader:true,defaultPostHeader:\"application/x-www-form-urlencoded\",useDefaultXhrHeader:true,defaultXhrHeader:\"XMLHttpRequest\",hasDefaultHeaders:true,defaultHeaders:{},poll:{},timeout:{},pollInterval:50,transactionId:0,setProgId:function(E){this.activeX.unshift(E)},setDefaultPostHeader:function(E){this.useDefaultHeader=E},setDefaultXhrHeader:function(E){this.useDefaultXhrHeader=E},setPollingInterval:function(E){if(typeof E==\"number\"&&isFinite(E)){this.pollInterval=E}},createXhrObject:function(I){var H,E;try{E=new XMLHttpRequest();H={conn:E,tId:I}}catch(G){for(var F=0;F<this.activeX.length;++F){try{E=new ActiveXObject(this.activeX[F]);H={conn:E,tId:I};break}catch(G){}}}finally{return H}},getConnectionObject:function(){var F;var G=this.transactionId;try{F=this.createXhrObject(G);if(F){this.transactionId++}}catch(E){}finally{return F}},asyncRequest:function(I,F,H,E){var G=this.getConnectionObject();if(!G){return null}else{G.conn.open(I,F,true);if(this.useDefaultXhrHeader){if(!this.defaultHeaders[\"X-Requested-With\"]){this.initHeader(\"X-Requested-With\",this.defaultXhrHeader,true)}}if(E&&this.useDefaultHeader){this.initHeader(\"Content-Type\",this.defaultPostHeader)}if(this.hasDefaultHeaders||this.hasHeaders){this.setHeader(G)}this.handleReadyState(G,H);G.conn.send(E||null);return G}},handleReadyState:function(F,G){var E=this;if(G&&G.timeout){this.timeout[F.tId]=window.setTimeout(function(){E.abort(F,G,true)},G.timeout)}this.poll[F.tId]=window.setInterval(function(){if(F.conn&&F.conn.readyState==4){window.clearInterval(E.poll[F.tId]);delete E.poll[F.tId];if(G&&G.timeout){window.clearTimeout(E.timeout[F.tId]);delete E.timeout[F.tId]}E.handleTransactionResponse(F,G)}},this.pollInterval)},handleTransactionResponse:function(I,J,E){if(!J){this.releaseObject(I);return }var G,F;try{if(I.conn.status!==undefined&&I.conn.status!=0){G=I.conn.status}else{G=13030}}catch(H){G=13030}if(G>=200&&G<300){F=this.createResponseObject(I,J.argument);if(J.success){if(!J.scope){J.success(F)}else{J.success.apply(J.scope,[F])}}}else{switch(G){case 12002:case 12029:case 12030:case 12031:case 12152:case 13030:F=this.createExceptionObject(I.tId,J.argument,(E?E:false));if(J.failure){if(!J.scope){J.failure(F)}else{J.failure.apply(J.scope,[F])}}break;default:F=this.createResponseObject(I,J.argument);if(J.failure){if(!J.scope){J.failure(F)}else{J.failure.apply(J.scope,[F])}}}}this.releaseObject(I);F=null},createResponseObject:function(E,K){var H={};var M={};try{var G=E.conn.getAllResponseHeaders();var J=G.split(\"\\n\");for(var I=0;I<J.length;I++){var F=J[I].indexOf(\":\");if(F!=-1){M[J[I].substring(0,F)]=J[I].substring(F+2)}}}catch(L){}H.tId=E.tId;H.status=E.conn.status;H.statusText=E.conn.statusText;H.getResponseHeader=M;H.getAllResponseHeaders=G;H.responseText=E.conn.responseText;H.responseXML=E.conn.responseXML;if(typeof K!==undefined){H.argument=K}return H},createExceptionObject:function(L,H,E){var J=0;var K=\"communication failure\";var G=-1;var F=\"transaction aborted\";var I={};I.tId=L;if(E){I.status=G;I.statusText=F}else{I.status=J;I.statusText=K}if(H){I.argument=H}return I},initHeader:function(E,H,G){var F=(G)?this.defaultHeaders:this.headers;if(F[E]===undefined){F[E]=H}else{F[E]=H+\",\"+F[E]}if(G){this.hasDefaultHeaders=true}else{this.hasHeaders=true}},setHeader:function(E){if(this.hasDefaultHeaders){for(var F in this.defaultHeaders){if(this.defaultHeaders.hasOwnProperty(F)){E.conn.setRequestHeader(F,this.defaultHeaders[F])}}}if(this.hasHeaders){for(var F in this.headers){if(this.headers.hasOwnProperty(F)){E.conn.setRequestHeader(F,this.headers[F])}}this.headers={};this.hasHeaders=false}},resetDefaultHeaders:function(){delete this.defaultHeaders;this.defaultHeaders={};this.hasDefaultHeaders=false},abort:function(F,G,E){if(this.isCallInProgress(F)){F.conn.abort();window.clearInterval(this.poll[F.tId]);delete this.poll[F.tId];if(E){delete this.timeout[F.tId]}this.handleTransactionResponse(F,G,true);return true}else{return false}},isCallInProgress:function(E){if(E.conn){return E.conn.readyState!=4&&E.conn.readyState!=0}else{return false}},releaseObject:function(E){E.conn=null;E=null},activeX:[\"MSXML2.XMLHTTP.3.0\",\"MSXML2.XMLHTTP\",\"Microsoft.XMLHTTP\"]};Ext.lib.Region=function(G,H,E,F){this.top=G;this[1]=G;this.right=H;this.bottom=E;this.left=F;this[0]=F};Ext.lib.Region.prototype={contains:function(E){return(E.left>=this.left&&E.right<=this.right&&E.top>=this.top&&E.bottom<=this.bottom)},getArea:function(){return((this.bottom-this.top)*(this.right-this.left))},intersect:function(I){var G=Math.max(this.top,I.top);var H=Math.min(this.right,I.right);var E=Math.min(this.bottom,I.bottom);var F=Math.max(this.left,I.left);if(E>=G&&H>=F){return new Ext.lib.Region(G,H,E,F)}else{return null}},union:function(I){var G=Math.min(this.top,I.top);var H=Math.max(this.right,I.right);var E=Math.max(this.bottom,I.bottom);var F=Math.min(this.left,I.left);return new Ext.lib.Region(G,H,E,F)},constrainTo:function(E){this.top=this.top.constrain(E.top,E.bottom);this.bottom=this.bottom.constrain(E.top,E.bottom);this.left=this.left.constrain(E.left,E.right);this.right=this.right.constrain(E.left,E.right);return this},adjust:function(G,F,E,H){this.top+=G;this.left+=F;this.right+=H;this.bottom+=E;return this}};Ext.lib.Region.getRegion=function(H){var J=Ext.lib.Dom.getXY(H);var G=J[1];var I=J[0]+H.offsetWidth;var E=J[1]+H.offsetHeight;var F=J[0];return new Ext.lib.Region(G,I,E,F)};Ext.lib.Point=function(E,F){if(Ext.isArray(E)){F=E[1];E=E[0]}this.x=this.right=this.left=this[0]=E;this.y=this.top=this.bottom=this[1]=F};Ext.lib.Point.prototype=new Ext.lib.Region();Ext.lib.Anim={scroll:function(H,F,I,J,E,G){return this.run(H,F,I,J,E,G,Ext.lib.Scroll)},motion:function(H,F,I,J,E,G){return this.run(H,F,I,J,E,G,Ext.lib.Motion)},color:function(H,F,I,J,E,G){return this.run(H,F,I,J,E,G,Ext.lib.ColorAnim)},run:function(I,F,K,L,E,H,G){G=G||Ext.lib.AnimBase;if(typeof L==\"string\"){L=Ext.lib.Easing[L]}var J=new G(I,F,K,L);J.animateX(function(){Ext.callback(E,H)});return J}};function C(E){if(!B){B=new Ext.Element.Flyweight()}B.dom=E;return B}if(Ext.isIE){function A(){var E=Function.prototype;delete E.createSequence;delete E.defer;delete E.createDelegate;delete E.createCallback;delete E.createInterceptor;window.detachEvent(\"onunload\",A)}window.attachEvent(\"onunload\",A)}Ext.lib.AnimBase=function(F,E,G,H){if(F){this.init(F,E,G,H)}};Ext.lib.AnimBase.prototype={toString:function(){var E=this.getEl();var F=E.id||E.tagName;return(\"Anim \"+F)},patterns:{noNegatives:/width|height|opacity|padding/i,offsetAttribute:/^((width|height)|(top|left))$/,defaultUnit:/width|height|top$|bottom$|left$|right$/i,offsetUnit:/\\d+(em|%|en|ex|pt|in|cm|mm|pc)$/i},doMethod:function(E,G,F){return this.method(this.currentFrame,G,F-G,this.totalFrames)},setAttribute:function(E,G,F){if(this.patterns.noNegatives.test(E)){G=(G>0)?G:0}Ext.fly(this.getEl(),\"_anim\").setStyle(E,G+F)},getAttribute:function(E){var G=this.getEl();var I=C(G).getStyle(E);if(I!==\"auto\"&&!this.patterns.offsetUnit.test(I)){return parseFloat(I)}var F=this.patterns.offsetAttribute.exec(E)||[];var J=!!(F[3]);var H=!!(F[2]);if(H||(C(G).getStyle(\"position\")==\"absolute\"&&J)){I=G[\"offset\"+F[0].charAt(0).toUpperCase()+F[0].substr(1)]}else{I=0}return I},getDefaultUnit:function(E){if(this.patterns.defaultUnit.test(E)){return\"px\"}return\"\"},animateX:function(G,E){var F=function(){this.onComplete.removeListener(F);if(typeof G==\"function\"){G.call(E||this,this)}};this.onComplete.addListener(F,this);this.animate()},setRuntimeAttribute:function(F){var K;var G;var H=this.attributes;this.runtimeAttributes[F]={};var J=function(L){return(typeof L!==\"undefined\")};if(!J(H[F][\"to\"])&&!J(H[F][\"by\"])){return false}K=(J(H[F][\"from\"]))?H[F][\"from\"]:this.getAttribute(F);if(J(H[F][\"to\"])){G=H[F][\"to\"]}else{if(J(H[F][\"by\"])){if(K.constructor==Array){G=[];for(var I=0,E=K.length;I<E;++I){G[I]=K[I]+H[F][\"by\"][I]}}else{G=K+H[F][\"by\"]}}}this.runtimeAttributes[F].start=K;this.runtimeAttributes[F].end=G;this.runtimeAttributes[F].unit=(J(H[F].unit))?H[F][\"unit\"]:this.getDefaultUnit(F)},init:function(G,L,K,E){var F=false;var H=null;var J=0;G=Ext.getDom(G);this.attributes=L||{};this.duration=K||1;this.method=E||Ext.lib.Easing.easeNone;this.useSeconds=true;this.currentFrame=0;this.totalFrames=Ext.lib.AnimMgr.fps;this.getEl=function(){return G};this.isAnimated=function(){return F};this.getStartTime=function(){return H};this.runtimeAttributes={};this.animate=function(){if(this.isAnimated()){return false}this.currentFrame=0;this.totalFrames=(this.useSeconds)?Math.ceil(Ext.lib.AnimMgr.fps*this.duration):this.duration;Ext.lib.AnimMgr.registerElement(this)};this.stop=function(O){if(O){this.currentFrame=this.totalFrames;this._onTween.fire()}Ext.lib.AnimMgr.stop(this)};var N=function(){this.onStart.fire();this.runtimeAttributes={};for(var O in this.attributes){this.setRuntimeAttribute(O)}F=true;J=0;H=new Date()};var M=function(){var Q={duration:new Date()-this.getStartTime(),currentFrame:this.currentFrame};Q.toString=function(){return(\"duration: \"+Q.duration+\", currentFrame: \"+Q.currentFrame)};this.onTween.fire(Q);var P=this.runtimeAttributes;for(var O in P){this.setAttribute(O,this.doMethod(O,P[O].start,P[O].end),P[O].unit)}J+=1};var I=function(){var O=(new Date()-H)/1000;var P={duration:O,frames:J,fps:J/O};P.toString=function(){return(\"duration: \"+P.duration+\", frames: \"+P.frames+\", fps: \"+P.fps)};F=false;J=0;this.onComplete.fire(P)};this._onStart=new Ext.util.Event(this);this.onStart=new Ext.util.Event(this);this.onTween=new Ext.util.Event(this);this._onTween=new Ext.util.Event(this);this.onComplete=new Ext.util.Event(this);this._onComplete=new Ext.util.Event(this);this._onStart.addListener(N);this._onTween.addListener(M);this._onComplete.addListener(I)}};Ext.lib.AnimMgr=new function(){var G=null;var F=[];var E=0;this.fps=1000;this.delay=1;this.registerElement=function(J){F[F.length]=J;E+=1;J._onStart.fire();this.start()};this.unRegister=function(K,J){K._onComplete.fire();J=J||I(K);if(J!=-1){F.splice(J,1)}E-=1;if(E<=0){this.stop()}};this.start=function(){if(G===null){G=setInterval(this.run,this.delay)}};this.stop=function(L){if(!L){clearInterval(G);for(var K=0,J=F.length;K<J;++K){if(F[0].isAnimated()){this.unRegister(F[0],0)}}F=[];G=null;E=0}else{this.unRegister(L)}};this.run=function(){for(var L=0,J=F.length;L<J;++L){var K=F[L];if(!K||!K.isAnimated()){continue}if(K.currentFrame<K.totalFrames||K.totalFrames===null){K.currentFrame+=1;if(K.useSeconds){H(K)}K._onTween.fire()}else{Ext.lib.AnimMgr.stop(K,L)}}};var I=function(L){for(var K=0,J=F.length;K<J;++K){if(F[K]==L){return K}}return -1};var H=function(K){var N=K.totalFrames;var M=K.currentFrame;var L=(K.currentFrame*K.duration*1000/K.totalFrames);var J=(new Date()-K.getStartTime());var O=0;if(J<K.duration*1000){O=Math.round((J/L-1)*K.currentFrame)}else{O=N-(M+1)}if(O>0&&isFinite(O)){if(K.currentFrame+O>=N){O=N-(M+1)}K.currentFrame+=O}}};Ext.lib.Bezier=new function(){this.getPosition=function(I,H){var J=I.length;var G=[];for(var F=0;F<J;++F){G[F]=[I[F][0],I[F][1]]}for(var E=1;E<J;++E){for(F=0;F<J-E;++F){G[F][0]=(1-H)*G[F][0]+H*G[parseInt(F+1,10)][0];G[F][1]=(1-H)*G[F][1]+H*G[parseInt(F+1,10)][1]}}return[G[0][0],G[0][1]]}};(function(){Ext.lib.ColorAnim=function(I,H,J,K){Ext.lib.ColorAnim.superclass.constructor.call(this,I,H,J,K)};Ext.extend(Ext.lib.ColorAnim,Ext.lib.AnimBase);var F=Ext.lib;var G=F.ColorAnim.superclass;var E=F.ColorAnim.prototype;E.toString=function(){var H=this.getEl();var I=H.id||H.tagName;return(\"ColorAnim \"+I)};E.patterns.color=/color$/i;E.patterns.rgb=/^rgb\\(([0-9]+)\\s*,\\s*([0-9]+)\\s*,\\s*([0-9]+)\\)$/i;E.patterns.hex=/^#?([0-9A-F]{2})([0-9A-F]{2})([0-9A-F]{2})$/i;E.patterns.hex3=/^#?([0-9A-F]{1})([0-9A-F]{1})([0-9A-F]{1})$/i;E.patterns.transparent=/^transparent|rgba\\(0, 0, 0, 0\\)$/;E.parseColor=function(H){if(H.length==3){return H}var I=this.patterns.hex.exec(H);if(I&&I.length==4){return[parseInt(I[1],16),parseInt(I[2],16),parseInt(I[3],16)]}I=this.patterns.rgb.exec(H);if(I&&I.length==4){return[parseInt(I[1],10),parseInt(I[2],10),parseInt(I[3],10)]}I=this.patterns.hex3.exec(H);if(I&&I.length==4){return[parseInt(I[1]+I[1],16),parseInt(I[2]+I[2],16),parseInt(I[3]+I[3],16)]}return null};E.getAttribute=function(H){var J=this.getEl();if(this.patterns.color.test(H)){var K=C(J).getStyle(H);if(this.patterns.transparent.test(K)){var I=J.parentNode;K=C(I).getStyle(H);while(I&&this.patterns.transparent.test(K)){I=I.parentNode;K=C(I).getStyle(H);if(I.tagName.toUpperCase()==\"HTML\"){K=\"#fff\"}}}}else{K=G.getAttribute.call(this,H)}return K};E.doMethod=function(I,M,J){var L;if(this.patterns.color.test(I)){L=[];for(var K=0,H=M.length;K<H;++K){L[K]=G.doMethod.call(this,I,M[K],J[K])}L=\"rgb(\"+Math.floor(L[0])+\",\"+Math.floor(L[1])+\",\"+Math.floor(L[2])+\")\"}else{L=G.doMethod.call(this,I,M,J)}return L};E.setRuntimeAttribute=function(I){G.setRuntimeAttribute.call(this,I);if(this.patterns.color.test(I)){var K=this.attributes;var M=this.parseColor(this.runtimeAttributes[I].start);var J=this.parseColor(this.runtimeAttributes[I].end);if(typeof K[I][\"to\"]===\"undefined\"&&typeof K[I][\"by\"]!==\"undefined\"){J=this.parseColor(K[I].by);for(var L=0,H=M.length;L<H;++L){J[L]=M[L]+J[L]}}this.runtimeAttributes[I].start=M;this.runtimeAttributes[I].end=J}}})();Ext.lib.Easing={easeNone:function(F,E,H,G){return H*F/G+E},easeIn:function(F,E,H,G){return H*(F/=G)*F+E},easeOut:function(F,E,H,G){return -H*(F/=G)*(F-2)+E},easeBoth:function(F,E,H,G){if((F/=G/2)<1){return H/2*F*F+E}return -H/2*((--F)*(F-2)-1)+E},easeInStrong:function(F,E,H,G){return H*(F/=G)*F*F*F+E},easeOutStrong:function(F,E,H,G){return -H*((F=F/G-1)*F*F*F-1)+E},easeBothStrong:function(F,E,H,G){if((F/=G/2)<1){return H/2*F*F*F*F+E}return -H/2*((F-=2)*F*F*F-2)+E},elasticIn:function(G,E,K,J,F,I){if(G==0){return E}if((G/=J)==1){return E+K}if(!I){I=J*0.3}if(!F||F<Math.abs(K)){F=K;var H=I/4}else{var H=I/(2*Math.PI)*Math.asin(K/F)}return -(F*Math.pow(2,10*(G-=1))*Math.sin((G*J-H)*(2*Math.PI)/I))+E},elasticOut:function(G,E,K,J,F,I){if(G==0){return E}if((G/=J)==1){return E+K}if(!I){I=J*0.3}if(!F||F<Math.abs(K)){F=K;var H=I/4}else{var H=I/(2*Math.PI)*Math.asin(K/F)}return F*Math.pow(2,-10*G)*Math.sin((G*J-H)*(2*Math.PI)/I)+K+E},elasticBoth:function(G,E,K,J,F,I){if(G==0){return E}if((G/=J/2)==2){return E+K}if(!I){I=J*(0.3*1.5)}if(!F||F<Math.abs(K)){F=K;var H=I/4}else{var H=I/(2*Math.PI)*Math.asin(K/F)}if(G<1){return -0.5*(F*Math.pow(2,10*(G-=1))*Math.sin((G*J-H)*(2*Math.PI)/I))+E}return F*Math.pow(2,-10*(G-=1))*Math.sin((G*J-H)*(2*Math.PI)/I)*0.5+K+E},backIn:function(F,E,I,H,G){if(typeof G==\"undefined\"){G=1.70158}return I*(F/=H)*F*((G+1)*F-G)+E},backOut:function(F,E,I,H,G){if(typeof G==\"undefined\"){G=1.70158}return I*((F=F/H-1)*F*((G+1)*F+G)+1)+E},backBoth:function(F,E,I,H,G){if(typeof G==\"undefined\"){G=1.70158}if((F/=H/2)<1){return I/2*(F*F*(((G*=(1.525))+1)*F-G))+E}return I/2*((F-=2)*F*(((G*=(1.525))+1)*F+G)+2)+E},bounceIn:function(F,E,H,G){return H-Ext.lib.Easing.bounceOut(G-F,0,H,G)+E},bounceOut:function(F,E,H,G){if((F/=G)<(1/2.75)){return H*(7.5625*F*F)+E}else{if(F<(2/2.75)){return H*(7.5625*(F-=(1.5/2.75))*F+0.75)+E}else{if(F<(2.5/2.75)){return H*(7.5625*(F-=(2.25/2.75))*F+0.9375)+E}}}return H*(7.5625*(F-=(2.625/2.75))*F+0.984375)+E},bounceBoth:function(F,E,H,G){if(F<G/2){return Ext.lib.Easing.bounceIn(F*2,0,H,G)*0.5+E}return Ext.lib.Easing.bounceOut(F*2-G,0,H,G)*0.5+H*0.5+E}};(function(){Ext.lib.Motion=function(K,J,L,M){if(K){Ext.lib.Motion.superclass.constructor.call(this,K,J,L,M)}};Ext.extend(Ext.lib.Motion,Ext.lib.ColorAnim);var H=Ext.lib;var I=H.Motion.superclass;var F=H.Motion.prototype;F.toString=function(){var J=this.getEl();var K=J.id||J.tagName;return(\"Motion \"+K)};F.patterns.points=/^points$/i;F.setAttribute=function(J,L,K){if(this.patterns.points.test(J)){K=K||\"px\";I.setAttribute.call(this,\"left\",L[0],K);I.setAttribute.call(this,\"top\",L[1],K)}else{I.setAttribute.call(this,J,L,K)}};F.getAttribute=function(J){if(this.patterns.points.test(J)){var K=[I.getAttribute.call(this,\"left\"),I.getAttribute.call(this,\"top\")]}else{K=I.getAttribute.call(this,J)}return K};F.doMethod=function(J,N,K){var M=null;if(this.patterns.points.test(J)){var L=this.method(this.currentFrame,0,100,this.totalFrames)/100;M=H.Bezier.getPosition(this.runtimeAttributes[J],L)}else{M=I.doMethod.call(this,J,N,K)}return M};F.setRuntimeAttribute=function(S){if(this.patterns.points.test(S)){var K=this.getEl();var M=this.attributes;var J;var O=M[\"points\"][\"control\"]||[];var L;var P,R;if(O.length>0&&!Ext.isArray(O[0])){O=[O]}else{var N=[];for(P=0,R=O.length;P<R;++P){N[P]=O[P]}O=N}Ext.fly(K).position();if(G(M[\"points\"][\"from\"])){Ext.lib.Dom.setXY(K,M[\"points\"][\"from\"])}else{Ext.lib.Dom.setXY(K,Ext.lib.Dom.getXY(K))}J=this.getAttribute(\"points\");if(G(M[\"points\"][\"to\"])){L=E.call(this,M[\"points\"][\"to\"],J);var Q=Ext.lib.Dom.getXY(this.getEl());for(P=0,R=O.length;P<R;++P){O[P]=E.call(this,O[P],J)}}else{if(G(M[\"points\"][\"by\"])){L=[J[0]+M[\"points\"][\"by\"][0],J[1]+M[\"points\"][\"by\"][1]];for(P=0,R=O.length;P<R;++P){O[P]=[J[0]+O[P][0],J[1]+O[P][1]]}}}this.runtimeAttributes[S]=[J];if(O.length>0){this.runtimeAttributes[S]=this.runtimeAttributes[S].concat(O)}this.runtimeAttributes[S][this.runtimeAttributes[S].length]=L}else{I.setRuntimeAttribute.call(this,S)}};var E=function(J,L){var K=Ext.lib.Dom.getXY(this.getEl());J=[J[0]-K[0]+L[0],J[1]-K[1]+L[1]];return J};var G=function(J){return(typeof J!==\"undefined\")}})();(function(){Ext.lib.Scroll=function(I,H,J,K){if(I){Ext.lib.Scroll.superclass.constructor.call(this,I,H,J,K)}};Ext.extend(Ext.lib.Scroll,Ext.lib.ColorAnim);var F=Ext.lib;var G=F.Scroll.superclass;var E=F.Scroll.prototype;E.toString=function(){var H=this.getEl();var I=H.id||H.tagName;return(\"Scroll \"+I)};E.doMethod=function(H,K,I){var J=null;if(H==\"scroll\"){J=[this.method(this.currentFrame,K[0],I[0]-K[0],this.totalFrames),this.method(this.currentFrame,K[1],I[1]-K[1],this.totalFrames)]}else{J=G.doMethod.call(this,H,K,I)}return J};E.getAttribute=function(H){var J=null;var I=this.getEl();if(H==\"scroll\"){J=[I.scrollLeft,I.scrollTop]}else{J=G.getAttribute.call(this,H)}return J};E.setAttribute=function(H,K,J){var I=this.getEl();if(H==\"scroll\"){I.scrollLeft=K[0];I.scrollTop=K[1]}else{G.setAttribute.call(this,H,K,J)}}})()})();\n"
  },
  {
    "path": "docroot/flot/API.txt",
    "content": "Flot Reference\n--------------\n\nConsider a call to the plot function:\n\n   var plot = $.plot(placeholder, data, options)\n\nThe placeholder is a jQuery object or DOM element or jQuery expression\nthat the plot will be put into. This placeholder needs to have its\nwidth and height set as explained in the README (go read that now if\nyou haven't, it's short). The plot will modify some properties of the\nplaceholder so it's recommended you simply pass in a div that you\ndon't use for anything else. Make sure you check any fancy styling\nyou apply to the div, e.g. background images have been reported to be a\nproblem on IE 7.\n\nThe format of the data is documented below, as is the available\noptions. The plot object returned from the call has some methods you\ncan call. These are documented separately below.\n\nNote that in general Flot gives no guarantees if you change any of the\nobjects you pass in to the plot function or get out of it since\nthey're not necessarily deep-copied.\n\n\nData Format\n-----------\n\nThe data is an array of data series:\n\n  [ series1, series2, ... ]\n\nA series can either be raw data or an object with properties. The raw\ndata format is an array of points:\n\n  [ [x1, y1], [x2, y2], ... ]\n\nE.g.\n\n  [ [1, 3], [2, 14.01], [3.5, 3.14] ]\n\nNote that to simplify the internal logic in Flot both the x and y\nvalues must be numbers (even if specifying time series, see below for\nhow to do this). This is a common problem because you might retrieve\ndata from the database and serialize them directly to JSON without\nnoticing the wrong type. If you're getting mysterious errors, double\ncheck that you're inputting numbers and not strings.\n\nIf a null is specified as a point or if one of the coordinates is null\nor couldn't be converted to a number, the point is ignored when\ndrawing. As a special case, a null value for lines is interpreted as a\nline segment end, i.e. the points before and after the null value are\nnot connected.\n\nLines and points take two coordinates. For filled lines and bars, you\ncan specify a third coordinate which is the bottom of the filled\narea/bar (defaults to 0).\n\nThe format of a single series object is as follows:\n\n  {\n    color: color or number\n    data: rawdata\n    label: string\n    lines: specific lines options\n    bars: specific bars options\n    points: specific points options\n    xaxis: number\n    yaxis: number\n    clickable: boolean\n    hoverable: boolean\n    shadowSize: number\n  }\n\nYou don't have to specify any of them except the data, the rest are\noptions that will get default values. Typically you'd only specify\nlabel and data, like this:\n\n  {\n    label: \"y = 3\",\n    data: [[0, 3], [10, 3]]\n  }\n\nThe label is used for the legend, if you don't specify one, the series\nwill not show up in the legend.\n\nIf you don't specify color, the series will get a color from the\nauto-generated colors. The color is either a CSS color specification\n(like \"rgb(255, 100, 123)\") or an integer that specifies which of\nauto-generated colors to select, e.g. 0 will get color no. 0, etc.\n\nThe latter is mostly useful if you let the user add and remove series,\nin which case you can hard-code the color index to prevent the colors\nfrom jumping around between the series.\n\nThe \"xaxis\" and \"yaxis\" options specify which axis to use. The axes\nare numbered from 1 (default), so { yaxis: 2} means that the series\nshould be plotted against the second y axis.\n\n\"clickable\" and \"hoverable\" can be set to false to disable\ninteractivity for specific series if interactivity is turned on in\nthe plot, see below.\n\nThe rest of the options are all documented below as they are the same\nas the default options passed in via the options parameter in the plot\ncommmand. When you specify them for a specific data series, they will\noverride the default options for the plot for that data series.\n\nHere's a complete example of a simple data specification:\n\n  [ { label: \"Foo\", data: [ [10, 1], [17, -14], [30, 5] ] },\n    { label: \"Bar\", data: [ [11, 13], [19, 11], [30, -7] ] } ]\n\n\nPlot Options\n------------\n\nAll options are completely optional. They are documented individually\nbelow, to change them you just specify them in an object, e.g.\n\n  var options = {\n    series: {\n      lines: { show: true },\n      points: { show: true }\n    }\n  };\n\n  $.plot(placeholder, data, options);\n\n\nCustomizing the legend\n======================\n\n  legend: {\n    show: boolean\n    labelFormatter: null or (fn: string, series object -> string)\n    labelBoxBorderColor: color\n    noColumns: number\n    position: \"ne\" or \"nw\" or \"se\" or \"sw\"\n    margin: number of pixels or [x margin, y margin]\n    backgroundColor: null or color\n    backgroundOpacity: number between 0 and 1\n    container: null or jQuery object/DOM element/jQuery expression\n  }\n\nThe legend is generated as a table with the data series labels and\nsmall label boxes with the color of the series. If you want to format\nthe labels in some way, e.g. make them to links, you can pass in a\nfunction for \"labelFormatter\". Here's an example that makes them\nclickable:\n\n  labelFormatter: function(label, series) {\n    // series is the series object for the label\n    return '<a href=\"#' + label + '\">' + label + '</a>';\n  }\n\n\"noColumns\" is the number of columns to divide the legend table into.\n\"position\" specifies the overall placement of the legend within the\nplot (top-right, top-left, etc.) and margin the distance to the plot\nedge (this can be either a number or an array of two numbers like [x,\ny]). \"backgroundColor\" and \"backgroundOpacity\" specifies the\nbackground. The default is a partly transparent auto-detected\nbackground.\n\nIf you want the legend to appear somewhere else in the DOM, you can\nspecify \"container\" as a jQuery object/expression to put the legend\ntable into. The \"position\" and \"margin\" etc. options will then be\nignored. Note that Flot will overwrite the contents of the container.\n\n\nCustomizing the axes\n====================\n\n  xaxis, yaxis: {\n    show: null or true/false\n    position: \"bottom\" or \"top\" or \"left\" or \"right\"\n    mode: null or \"time\"\n\n    color: null or color spec\n    tickColor: null or color spec\n    \n    min: null or number\n    max: null or number\n    autoscaleMargin: null or number\n    \n    transform: null or fn: number -> number\n    inverseTransform: null or fn: number -> number\n    \n    ticks: null or number or ticks array or (fn: range -> ticks array)\n    tickSize: number or array\n    minTickSize: number or array\n    tickFormatter: (fn: number, object -> string) or string\n    tickDecimals: null or number\n\n    labelWidth: null or number\n    labelHeight: null or number\n    reserveSpace: null or true\n    \n    tickLength: null or number\n\n    alignTicksWithAxis: null or number\n  }\n\nAll axes have the same kind of options. The following describes how to\nconfigure one axis, see below for what to do if you've got more than\none x axis or y axis.\n\nIf you don't set the \"show\" option (i.e. it is null), visibility is\nauto-detected, i.e. the axis will show up if there's data associated\nwith it. You can override this by setting the \"show\" option to true or\nfalse.\n\nThe \"position\" option specifies where the axis is placed, bottom or\ntop for x axes, left or right for y axes. The \"mode\" option determines\nhow the data is interpreted, the default of null means as decimal\nnumbers. Use \"time\" for time series data, see the time series data\nsection.\n\nThe \"color\" option determines the color of the labels and ticks for\nthe axis (default is the grid color). For more fine-grained control\nyou can also set the color of the ticks separately with \"tickColor\"\n(otherwise it's autogenerated as the base color with some\ntransparency).\n\nThe options \"min\"/\"max\" are the precise minimum/maximum value on the\nscale. If you don't specify either of them, a value will automatically\nbe chosen based on the minimum/maximum data values. Note that Flot\nalways examines all the data values you feed to it, even if a\nrestriction on another axis may make some of them invisible (this\nmakes interactive use more stable).\n\nThe \"autoscaleMargin\" is a bit esoteric: it's the fraction of margin\nthat the scaling algorithm will add to avoid that the outermost points\nends up on the grid border. Note that this margin is only applied when\na min or max value is not explicitly set. If a margin is specified,\nthe plot will furthermore extend the axis end-point to the nearest\nwhole tick. The default value is \"null\" for the x axes and 0.02 for y\naxes which seems appropriate for most cases.\n\n\"transform\" and \"inverseTransform\" are callbacks you can put in to\nchange the way the data is drawn. You can design a function to\ncompress or expand certain parts of the axis non-linearly, e.g.\nsuppress weekends or compress far away points with a logarithm or some\nother means. When Flot draws the plot, each value is first put through\nthe transform function. Here's an example, the x axis can be turned\ninto a natural logarithm axis with the following code:\n\n  xaxis: {\n    transform: function (v) { return Math.log(v); },\n    inverseTransform: function (v) { return Math.exp(v); }\n  }\n\nSimilarly, for reversing the y axis so the values appear in inverse\norder:\n  \n  yaxis: {\n    transform: function (v) { return -v; },\n    inverseTransform: function (v) { return -v; }\n  }\n\nNote that for finding extrema, Flot assumes that the transform\nfunction does not reorder values (it should be monotone).\n\nThe inverseTransform is simply the inverse of the transform function\n(so v == inverseTransform(transform(v)) for all relevant v). It is\nrequired for converting from canvas coordinates to data coordinates,\ne.g. for a mouse interaction where a certain pixel is clicked. If you\ndon't use any interactive features of Flot, you may not need it.\n\n\nThe rest of the options deal with the ticks.\n\nIf you don't specify any ticks, a tick generator algorithm will make\nsome for you. The algorithm has two passes. It first estimates how\nmany ticks would be reasonable and uses this number to compute a nice\nround tick interval size. Then it generates the ticks.\n\nYou can specify how many ticks the algorithm aims for by setting\n\"ticks\" to a number. The algorithm always tries to generate reasonably\nround tick values so even if you ask for three ticks, you might get\nfive if that fits better with the rounding. If you don't want any\nticks at all, set \"ticks\" to 0 or an empty array.\n\nAnother option is to skip the rounding part and directly set the tick\ninterval size with \"tickSize\". If you set it to 2, you'll get ticks at\n2, 4, 6, etc. Alternatively, you can specify that you just don't want\nticks at a size less than a specific tick size with \"minTickSize\".\nNote that for time series, the format is an array like [2, \"month\"],\nsee the next section.\n\nIf you want to completely override the tick algorithm, you can specify\nan array for \"ticks\", either like this:\n\n  ticks: [0, 1.2, 2.4]\n\nOr like this where the labels are also customized:\n\n  ticks: [[0, \"zero\"], [1.2, \"one mark\"], [2.4, \"two marks\"]]\n\nYou can mix the two if you like.\n  \nFor extra flexibility you can specify a function as the \"ticks\"\nparameter. The function will be called with an object with the axis\nmin and max and should return a ticks array. Here's a simplistic tick\ngenerator that spits out intervals of pi, suitable for use on the x\naxis for trigonometric functions:\n\n  function piTickGenerator(axis) {\n    var res = [], i = Math.floor(axis.min / Math.PI);\n    do {\n      var v = i * Math.PI;\n      res.push([v, i + \"\\u03c0\"]);\n      ++i;\n    } while (v < axis.max);\n    \n    return res;\n  }\n\nYou can control how the ticks look like with \"tickDecimals\", the\nnumber of decimals to display (default is auto-detected).\n\nAlternatively, for ultimate control over how ticks are formatted you can\nprovide a function to \"tickFormatter\". The function is passed two\nparameters, the tick value and an axis object with information, and\nshould return a string. The default formatter looks like this:\n\n  function formatter(val, axis) {\n    return val.toFixed(axis.tickDecimals);\n  }\n\nThe axis object has \"min\" and \"max\" with the range of the axis,\n\"tickDecimals\" with the number of decimals to round the value to and\n\"tickSize\" with the size of the interval between ticks as calculated\nby the automatic axis scaling algorithm (or specified by you). Here's\nan example of a custom formatter:\n\n  function suffixFormatter(val, axis) {\n    if (val > 1000000)\n      return (val / 1000000).toFixed(axis.tickDecimals) + \" MB\";\n    else if (val > 1000)\n      return (val / 1000).toFixed(axis.tickDecimals) + \" kB\";\n    else\n      return val.toFixed(axis.tickDecimals) + \" B\";\n  }\n\n\"labelWidth\" and \"labelHeight\" specifies a fixed size of the tick\nlabels in pixels. They're useful in case you need to align several\nplots. \"reserveSpace\" means that even if an axis isn't shown, Flot\nshould reserve space for it - it is useful in combination with\nlabelWidth and labelHeight for aligning multi-axis charts.\n\n\"tickLength\" is the length of the tick lines in pixels. By default, the\ninnermost axes will have ticks that extend all across the plot, while\nany extra axes use small ticks. A value of null means use the default,\nwhile a number means small ticks of that length - set it to 0 to hide\nthe lines completely.\n\nIf you set \"alignTicksWithAxis\" to the number of another axis, e.g.\nalignTicksWithAxis: 1, Flot will ensure that the autogenerated ticks\nof this axis are aligned with the ticks of the other axis. This may\nimprove the looks, e.g. if you have one y axis to the left and one to\nthe right, because the grid lines will then match the ticks in both\nends. The trade-off is that the forced ticks won't necessarily be at\nnatural places.\n\n\nMultiple axes\n=============\n\nIf you need more than one x axis or y axis, you need to specify for\neach data series which axis they are to use, as described under the\nformat of the data series, e.g. { data: [...], yaxis: 2 } specifies\nthat a series should be plotted against the second y axis.\n\nTo actually configure that axis, you can't use the xaxis/yaxis options\ndirectly - instead there are two arrays in the options:\n\n   xaxes: []\n   yaxes: []\n\nHere's an example of configuring a single x axis and two y axes (we\ncan leave options of the first y axis empty as the defaults are fine):\n\n  {\n    xaxes: [ { position: \"top\" } ],\n    yaxes: [ { }, { position: \"right\", min: 20 } ]\n  }\n\nThe arrays get their default values from the xaxis/yaxis settings, so\nsay you want to have all y axes start at zero, you can simply specify\nyaxis: { min: 0 } instead of adding a min parameter to all the axes.\n\nGenerally, the various interfaces in Flot dealing with data points\neither accept an xaxis/yaxis parameter to specify which axis number to\nuse (starting from 1), or lets you specify the coordinate directly as\nx2/x3/... or x2axis/x3axis/... instead of \"x\" or \"xaxis\".\n\n  \nTime series data\n================\n\nTime series are a bit more difficult than scalar data because\ncalendars don't follow a simple base 10 system. For many cases, Flot\nabstracts most of this away, but it can still be a bit difficult to\nget the data into Flot. So we'll first discuss the data format.\n\nThe time series support in Flot is based on Javascript timestamps,\ni.e. everywhere a time value is expected or handed over, a Javascript\ntimestamp number is used. This is a number, not a Date object. A\nJavascript timestamp is the number of milliseconds since January 1,\n1970 00:00:00 UTC. This is almost the same as Unix timestamps, except it's\nin milliseconds, so remember to multiply by 1000!\n\nYou can see a timestamp like this\n\n  alert((new Date()).getTime())\n\nNormally you want the timestamps to be displayed according to a\ncertain time zone, usually the time zone in which the data has been\nproduced. However, Flot always displays timestamps according to UTC.\nIt has to as the only alternative with core Javascript is to interpret\nthe timestamps according to the time zone that the visitor is in,\nwhich means that the ticks will shift unpredictably with the time zone\nand daylight savings of each visitor.\n\nSo given that there's no good support for custom time zones in\nJavascript, you'll have to take care of this server-side.\n\nThe easiest way to think about it is to pretend that the data\nproduction time zone is UTC, even if it isn't. So if you have a\ndatapoint at 2002-02-20 08:00, you can generate a timestamp for eight\no'clock UTC even if it really happened eight o'clock UTC+0200.\n\nIn PHP you can get an appropriate timestamp with\n'strtotime(\"2002-02-20 UTC\") * 1000', in Python with\n'calendar.timegm(datetime_object.timetuple()) * 1000', in .NET with\nsomething like:\n\n  public static int GetJavascriptTimestamp(System.DateTime input)\n  {\n    System.TimeSpan span = new System.TimeSpan(System.DateTime.Parse(\"1/1/1970\").Ticks);\n    System.DateTime time = input.Subtract(span);\n    return (long)(time.Ticks / 10000);\n  }\n\nJavascript also has some support for parsing date strings, so it is\npossible to generate the timestamps manually client-side.\n\nIf you've already got the real UTC timestamp, it's too late to use the\npretend trick described above. But you can fix up the timestamps by\nadding the time zone offset, e.g. for UTC+0200 you would add 2 hours\nto the UTC timestamp you got. Then it'll look right on the plot. Most\nprogramming environments have some means of getting the timezone\noffset for a specific date (note that you need to get the offset for\neach individual timestamp to account for daylight savings).\n\nOnce you've gotten the timestamps into the data and specified \"time\"\nas the axis mode, Flot will automatically generate relevant ticks and\nformat them. As always, you can tweak the ticks via the \"ticks\" option\n- just remember that the values should be timestamps (numbers), not\nDate objects.\n\nTick generation and formatting can also be controlled separately\nthrough the following axis options:\n\n  minTickSize: array\n  timeformat: null or format string\n  monthNames: null or array of size 12 of strings\n  twelveHourClock: boolean\n\nHere \"timeformat\" is a format string to use. You might use it like\nthis:\n\n  xaxis: {\n    mode: \"time\"\n    timeformat: \"%y/%m/%d\"\n  }\n  \nThis will result in tick labels like \"2000/12/24\". The following\nspecifiers are supported\n\n  %h: hours\n  %H: hours (left-padded with a zero)\n  %M: minutes (left-padded with a zero)\n  %S: seconds (left-padded with a zero)\n  %d: day of month (1-31), use %0d for zero-padding\n  %m: month (1-12), use %0m for zero-padding\n  %y: year (four digits)\n  %b: month name (customizable)\n  %p: am/pm, additionally switches %h/%H to 12 hour instead of 24\n  %P: AM/PM (uppercase version of %p)\n\nInserting a zero like %0m or %0d means that the specifier will be\nleft-padded with a zero if it's only single-digit. So %y-%0m-%0d\nresults in unambigious ISO timestamps like 2007-05-10 (for May 10th).\n\nYou can customize the month names with the \"monthNames\" option. For\ninstance, for Danish you might specify:\n\n  monthNames: [\"jan\", \"feb\", \"mar\", \"apr\", \"maj\", \"jun\", \"jul\", \"aug\", \"sep\", \"okt\", \"nov\", \"dec\"]\n\nIf you set \"twelveHourClock\" to true, the autogenerated timestamps\nwill use 12 hour AM/PM timestamps instead of 24 hour.\n  \nThe format string and month names are used by a very simple built-in\nformat function that takes a date object, a format string (and\noptionally an array of month names) and returns the formatted string.\nIf needed, you can access it as $.plot.formatDate(date, formatstring,\nmonthNames) or even replace it with another more advanced function\nfrom a date library if you're feeling adventurous.\n\nIf everything else fails, you can control the formatting by specifying\na custom tick formatter function as usual. Here's a simple example\nwhich will format December 24 as 24/12:\n\n  tickFormatter: function (val, axis) {\n    var d = new Date(val);\n    return d.getUTCDate() + \"/\" + (d.getUTCMonth() + 1);\n  }\n\nNote that for the time mode \"tickSize\" and \"minTickSize\" are a bit\nspecial in that they are arrays on the form \"[value, unit]\" where unit\nis one of \"second\", \"minute\", \"hour\", \"day\", \"month\" and \"year\". So\nyou can specify\n\n  minTickSize: [1, \"month\"]\n\nto get a tick interval size of at least 1 month and correspondingly,\nif axis.tickSize is [2, \"day\"] in the tick formatter, the ticks have\nbeen produced with two days in-between.\n\n\n\nCustomizing the data series\n===========================\n\n  series: {\n    lines, points, bars: {\n      show: boolean\n      lineWidth: number\n      fill: boolean or number\n      fillColor: null or color/gradient\n    }\n\n    points: {\n      radius: number\n      symbol: \"circle\" or function\n    }\n\n    bars: {\n      barWidth: number\n      align: \"left\" or \"center\"\n      horizontal: boolean\n    }\n\n    lines: {\n      steps: boolean\n    }\n\n    shadowSize: number\n  }\n  \n  colors: [ color1, color2, ... ]\n\nThe options inside \"series: {}\" are copied to each of the series. So\nyou can specify that all series should have bars by putting it in the\nglobal options, or override it for individual series by specifying\nbars in a particular the series object in the array of data.\n  \nThe most important options are \"lines\", \"points\" and \"bars\" that\nspecify whether and how lines, points and bars should be shown for\neach data series. In case you don't specify anything at all, Flot will\ndefault to showing lines (you can turn this off with\nlines: { show: false }). You can specify the various types\nindependently of each other, and Flot will happily draw each of them\nin turn (this is probably only useful for lines and points), e.g.\n\n  var options = {\n    series: {\n      lines: { show: true, fill: true, fillColor: \"rgba(255, 255, 255, 0.8)\" },\n      points: { show: true, fill: false }\n    }\n  };\n\n\"lineWidth\" is the thickness of the line or outline in pixels. You can\nset it to 0 to prevent a line or outline from being drawn; this will\nalso hide the shadow.\n\n\"fill\" is whether the shape should be filled. For lines, this produces\narea graphs. You can use \"fillColor\" to specify the color of the fill.\nIf \"fillColor\" evaluates to false (default for everything except\npoints which are filled with white), the fill color is auto-set to the\ncolor of the data series. You can adjust the opacity of the fill by\nsetting fill to a number between 0 (fully transparent) and 1 (fully\nopaque).\n\nFor bars, fillColor can be a gradient, see the gradient documentation\nbelow. \"barWidth\" is the width of the bars in units of the x axis (or\nthe y axis if \"horizontal\" is true), contrary to most other measures\nthat are specified in pixels. For instance, for time series the unit\nis milliseconds so 24 * 60 * 60 * 1000 produces bars with the width of\na day. \"align\" specifies whether a bar should be left-aligned\n(default) or centered on top of the value it represents. When\n\"horizontal\" is on, the bars are drawn horizontally, i.e. from the y\naxis instead of the x axis; note that the bar end points are still\ndefined in the same way so you'll probably want to swap the\ncoordinates if you've been plotting vertical bars first.\n\nFor lines, \"steps\" specifies whether two adjacent data points are\nconnected with a straight (possibly diagonal) line or with first a\nhorizontal and then a vertical line. Note that this transforms the\ndata by adding extra points.\n\nFor points, you can specify the radius and the symbol. The only\nbuilt-in symbol type is circles, for other types you can use a plugin\nor define them yourself by specifying a callback:\n\n  function cross(ctx, x, y, radius, shadow) {\n      var size = radius * Math.sqrt(Math.PI) / 2;\n      ctx.moveTo(x - size, y - size);\n      ctx.lineTo(x + size, y + size);\n      ctx.moveTo(x - size, y + size);\n      ctx.lineTo(x + size, y - size);\n  }\n\nThe parameters are the drawing context, x and y coordinates of the\ncenter of the point, a radius which corresponds to what the circle\nwould have used and whether the call is to draw a shadow (due to\nlimited canvas support, shadows are currently faked through extra\ndraws). It's good practice to ensure that the area covered by the\nsymbol is the same as for the circle with the given radius, this\nensures that all symbols have approximately the same visual weight.\n\n\"shadowSize\" is the default size of shadows in pixels. Set it to 0 to\nremove shadows.\n\nThe \"colors\" array specifies a default color theme to get colors for\nthe data series from. You can specify as many colors as you like, like\nthis:\n\n  colors: [\"#d18b2c\", \"#dba255\", \"#919733\"]\n\nIf there are more data series than colors, Flot will try to generate\nextra colors by lightening and darkening colors in the theme.\n\n\nCustomizing the grid\n====================\n\n  grid: {\n    show: boolean\n    aboveData: boolean\n    color: color\n    backgroundColor: color/gradient or null\n    labelMargin: number\n    axisMargin: number\n    markings: array of markings or (fn: axes -> array of markings)\n    borderWidth: number\n    borderColor: color or null\n    minBorderMargin: number or null\n    clickable: boolean\n    hoverable: boolean\n    autoHighlight: boolean\n    mouseActiveRadius: number\n  }\n\nThe grid is the thing with the axes and a number of ticks. Many of the\nthings in the grid are configured under the individual axes, but not\nall. \"color\" is the color of the grid itself whereas \"backgroundColor\"\nspecifies the background color inside the grid area, here null means\nthat the background is transparent. You can also set a gradient, see\nthe gradient documentation below.\n\nYou can turn off the whole grid including tick labels by setting\n\"show\" to false. \"aboveData\" determines whether the grid is drawn\nabove the data or below (below is default).\n\n\"labelMargin\" is the space in pixels between tick labels and axis\nline, and \"axisMargin\" is the space in pixels between axes when there\nare two next to each other. Note that you can style the tick labels\nwith CSS, e.g. to change the color. They have class \"tickLabel\".\n\n\"borderWidth\" is the width of the border around the plot. Set it to 0\nto disable the border. You can also set \"borderColor\" if you want the\nborder to have a different color than the grid lines.\n\"minBorderMargin\" controls the default minimum margin around the\nborder - it's used to make sure that points aren't accidentally\nclipped by the canvas edge so by default the value is computed from\nthe point radius.\n\n\"markings\" is used to draw simple lines and rectangular areas in the\nbackground of the plot. You can either specify an array of ranges on\nthe form { xaxis: { from, to }, yaxis: { from, to } } (with multiple\naxes, you can specify coordinates for other axes instead, e.g. as\nx2axis/x3axis/...) or with a function that returns such an array given\nthe axes for the plot in an object as the first parameter.\n\nYou can set the color of markings by specifying \"color\" in the ranges\nobject. Here's an example array:\n\n  markings: [ { xaxis: { from: 0, to: 2 }, yaxis: { from: 10, to: 10 }, color: \"#bb0000\" }, ... ]\n\nIf you leave out one of the values, that value is assumed to go to the\nborder of the plot. So for example if you only specify { xaxis: {\nfrom: 0, to: 2 } } it means an area that extends from the top to the\nbottom of the plot in the x range 0-2.\n\nA line is drawn if from and to are the same, e.g.\n\n  markings: [ { yaxis: { from: 1, to: 1 } }, ... ]\n\nwould draw a line parallel to the x axis at y = 1. You can control the\nline width with \"lineWidth\" in the range object.\n\nAn example function that makes vertical stripes might look like this:\n\n  markings: function (axes) {\n    var markings = [];\n    for (var x = Math.floor(axes.xaxis.min); x < axes.xaxis.max; x += 2)\n      markings.push({ xaxis: { from: x, to: x + 1 } });\n    return markings;\n  }\n\n\nIf you set \"clickable\" to true, the plot will listen for click events\non the plot area and fire a \"plotclick\" event on the placeholder with\na position and a nearby data item object as parameters. The coordinates\nare available both in the unit of the axes (not in pixels) and in\nglobal screen coordinates.\n\nLikewise, if you set \"hoverable\" to true, the plot will listen for\nmouse move events on the plot area and fire a \"plothover\" event with\nthe same parameters as the \"plotclick\" event. If \"autoHighlight\" is\ntrue (the default), nearby data items are highlighted automatically.\nIf needed, you can disable highlighting and control it yourself with\nthe highlight/unhighlight plot methods described elsewhere.\n\nYou can use \"plotclick\" and \"plothover\" events like this:\n\n    $.plot($(\"#placeholder\"), [ d ], { grid: { clickable: true } });\n\n    $(\"#placeholder\").bind(\"plotclick\", function (event, pos, item) {\n        alert(\"You clicked at \" + pos.x + \", \" + pos.y);\n        // axis coordinates for other axes, if present, are in pos.x2, pos.x3, ...\n        // if you need global screen coordinates, they are pos.pageX, pos.pageY\n\n        if (item) {\n          highlight(item.series, item.datapoint);\n          alert(\"You clicked a point!\");\n        }\n    });\n\nThe item object in this example is either null or a nearby object on the form:\n\n  item: {\n      datapoint: the point, e.g. [0, 2]\n      dataIndex: the index of the point in the data array\n      series: the series object\n      seriesIndex: the index of the series\n      pageX, pageY: the global screen coordinates of the point\n  }\n\nFor instance, if you have specified the data like this \n\n    $.plot($(\"#placeholder\"), [ { label: \"Foo\", data: [[0, 10], [7, 3]] } ], ...);\n\nand the mouse is near the point (7, 3), \"datapoint\" is [7, 3],\n\"dataIndex\" will be 1, \"series\" is a normalized series object with\namong other things the \"Foo\" label in series.label and the color in\nseries.color, and \"seriesIndex\" is 0. Note that plugins and options\nthat transform the data can shift the indexes from what you specified\nin the original data array.\n\nIf you use the above events to update some other information and want\nto clear out that info in case the mouse goes away, you'll probably\nalso need to listen to \"mouseout\" events on the placeholder div.\n\n\"mouseActiveRadius\" specifies how far the mouse can be from an item\nand still activate it. If there are two or more points within this\nradius, Flot chooses the closest item. For bars, the top-most bar\n(from the latest specified data series) is chosen.\n\nIf you want to disable interactivity for a specific data series, you\ncan set \"hoverable\" and \"clickable\" to false in the options for that\nseries, like this { data: [...], label: \"Foo\", clickable: false }.\n\n\nSpecifying gradients\n====================\n\nA gradient is specified like this:\n\n  { colors: [ color1, color2, ... ] }\n\nFor instance, you might specify a background on the grid going from\nblack to gray like this:\n\n  grid: {\n    backgroundColor: { colors: [\"#000\", \"#999\"] }\n  }\n\nFor the series you can specify the gradient as an object that\nspecifies the scaling of the brightness and the opacity of the series\ncolor, e.g.\n\n  { colors: [{ opacity: 0.8 }, { brightness: 0.6, opacity: 0.8 } ] }\n\nwhere the first color simply has its alpha scaled, whereas the second\nis also darkened. For instance, for bars the following makes the bars\ngradually disappear, without outline:\n\n  bars: {\n      show: true,\n      lineWidth: 0,\n      fill: true,\n      fillColor: { colors: [ { opacity: 0.8 }, { opacity: 0.1 } ] }\n  }\n  \nFlot currently only supports vertical gradients drawn from top to\nbottom because that's what works with IE.\n\n\nPlot Methods\n------------\n\nThe Plot object returned from the plot function has some methods you\ncan call:\n\n  - highlight(series, datapoint)\n\n    Highlight a specific datapoint in the data series. You can either\n    specify the actual objects, e.g. if you got them from a\n    \"plotclick\" event, or you can specify the indices, e.g.\n    highlight(1, 3) to highlight the fourth point in the second series\n    (remember, zero-based indexing).\n\n  \n  - unhighlight(series, datapoint) or unhighlight()\n\n    Remove the highlighting of the point, same parameters as\n    highlight.\n\n    If you call unhighlight with no parameters, e.g. as\n    plot.unhighlight(), all current highlights are removed.\n\n\n  - setData(data)\n\n    You can use this to reset the data used. Note that axis scaling,\n    ticks, legend etc. will not be recomputed (use setupGrid() to do\n    that). You'll probably want to call draw() afterwards.\n\n    You can use this function to speed up redrawing a small plot if\n    you know that the axes won't change. Put in the new data with\n    setData(newdata), call draw(), and you're good to go. Note that\n    for large datasets, almost all the time is consumed in draw()\n    plotting the data so in this case don't bother.\n\n    \n  - setupGrid()\n\n    Recalculate and set axis scaling, ticks, legend etc.\n\n    Note that because of the drawing model of the canvas, this\n    function will immediately redraw (actually reinsert in the DOM)\n    the labels and the legend, but not the actual tick lines because\n    they're drawn on the canvas. You need to call draw() to get the\n    canvas redrawn.\n    \n  - draw()\n\n    Redraws the plot canvas.\n\n  - triggerRedrawOverlay()\n\n    Schedules an update of an overlay canvas used for drawing\n    interactive things like a selection and point highlights. This\n    is mostly useful for writing plugins. The redraw doesn't happen\n    immediately, instead a timer is set to catch multiple successive\n    redraws (e.g. from a mousemove). You can get to the overlay by\n    setting up a drawOverlay hook.\n\n  - width()/height()\n\n    Gets the width and height of the plotting area inside the grid.\n    This is smaller than the canvas or placeholder dimensions as some\n    extra space is needed (e.g. for labels).\n\n  - offset()\n\n    Returns the offset of the plotting area inside the grid relative\n    to the document, useful for instance for calculating mouse\n    positions (event.pageX/Y minus this offset is the pixel position\n    inside the plot).\n\n  - pointOffset({ x: xpos, y: ypos })\n\n    Returns the calculated offset of the data point at (x, y) in data\n    space within the placeholder div. If you are working with multiple axes, you\n    can specify the x and y axis references, e.g. \n\n      o = pointOffset({ x: xpos, y: ypos, xaxis: 2, yaxis: 3 })\n      // o.left and o.top now contains the offset within the div\n\n  - resize()\n\n    Tells Flot to resize the drawing canvas to the size of the\n    placeholder. You need to run setupGrid() and draw() afterwards as\n    canvas resizing is a destructive operation. This is used\n    internally by the resize plugin.\n\n  - shutdown()\n\n    Cleans up any event handlers Flot has currently registered. This\n    is used internally.\n\n\nThere are also some members that let you peek inside the internal\nworkings of Flot which is useful in some cases. Note that if you change\nsomething in the objects returned, you're changing the objects used by\nFlot to keep track of its state, so be careful.\n\n  - getData()\n\n    Returns an array of the data series currently used in normalized\n    form with missing settings filled in according to the global\n    options. So for instance to find out what color Flot has assigned\n    to the data series, you could do this:\n\n      var series = plot.getData();\n      for (var i = 0; i < series.length; ++i)\n        alert(series[i].color);\n\n    A notable other interesting field besides color is datapoints\n    which has a field \"points\" with the normalized data points in a\n    flat array (the field \"pointsize\" is the increment in the flat\n    array to get to the next point so for a dataset consisting only of\n    (x,y) pairs it would be 2).\n\n  - getAxes()\n\n    Gets an object with the axes. The axes are returned as the\n    attributes of the object, so for instance getAxes().xaxis is the\n    x axis.\n\n    Various things are stuffed inside an axis object, e.g. you could\n    use getAxes().xaxis.ticks to find out what the ticks are for the\n    xaxis. Two other useful attributes are p2c and c2p, functions for\n    transforming from data point space to the canvas plot space and\n    back. Both returns values that are offset with the plot offset.\n    Check the Flot source code for the complete set of attributes (or\n    output an axis with console.log() and inspect it).\n\n    With multiple axes, the extra axes are returned as x2axis, x3axis,\n    etc., e.g. getAxes().y2axis is the second y axis. You can check\n    y2axis.used to see whether the axis is associated with any data\n    points and y2axis.show to see if it is currently shown. \n \n  - getPlaceholder()\n\n    Returns placeholder that the plot was put into. This can be useful\n    for plugins for adding DOM elements or firing events.\n\n  - getCanvas()\n\n    Returns the canvas used for drawing in case you need to hack on it\n    yourself. You'll probably need to get the plot offset too.\n  \n  - getPlotOffset()\n\n    Gets the offset that the grid has within the canvas as an object\n    with distances from the canvas edges as \"left\", \"right\", \"top\",\n    \"bottom\". I.e., if you draw a circle on the canvas with the center\n    placed at (left, top), its center will be at the top-most, left\n    corner of the grid.\n\n  - getOptions()\n\n    Gets the options for the plot, normalized, with default values\n    filled in. You get a reference to actual values used by Flot, so\n    if you modify the values in here, Flot will use the new values.\n    If you change something, you probably have to call draw() or\n    setupGrid() or triggerRedrawOverlay() to see the change.\n    \n\nHooks\n=====\n\nIn addition to the public methods, the Plot object also has some hooks\nthat can be used to modify the plotting process. You can install a\ncallback function at various points in the process, the function then\ngets access to the internal data structures in Flot.\n\nHere's an overview of the phases Flot goes through:\n\n  1. Plugin initialization, parsing options\n  \n  2. Constructing the canvases used for drawing\n\n  3. Set data: parsing data specification, calculating colors,\n     copying raw data points into internal format,\n     normalizing them, finding max/min for axis auto-scaling\n\n  4. Grid setup: calculating axis spacing, ticks, inserting tick\n     labels, the legend\n\n  5. Draw: drawing the grid, drawing each of the series in turn\n\n  6. Setting up event handling for interactive features\n\n  7. Responding to events, if any\n\n  8. Shutdown: this mostly happens in case a plot is overwritten \n\nEach hook is simply a function which is put in the appropriate array.\nYou can add them through the \"hooks\" option, and they are also available\nafter the plot is constructed as the \"hooks\" attribute on the returned\nplot object, e.g.\n\n  // define a simple draw hook\n  function hellohook(plot, canvascontext) { alert(\"hello!\"); };\n\n  // pass it in, in an array since we might want to specify several\n  var plot = $.plot(placeholder, data, { hooks: { draw: [hellohook] } });\n\n  // we can now find it again in plot.hooks.draw[0] unless a plugin\n  // has added other hooks\n\nThe available hooks are described below. All hook callbacks get the\nplot object as first parameter. You can find some examples of defined\nhooks in the plugins bundled with Flot.\n\n - processOptions  [phase 1]\n\n   function(plot, options)\n   \n   Called after Flot has parsed and merged options. Useful in the\n   instance where customizations beyond simple merging of default\n   values is needed. A plugin might use it to detect that it has been\n   enabled and then turn on or off other options.\n\n \n - processRawData  [phase 3]\n\n   function(plot, series, data, datapoints)\n \n   Called before Flot copies and normalizes the raw data for the given\n   series. If the function fills in datapoints.points with normalized\n   points and sets datapoints.pointsize to the size of the points,\n   Flot will skip the copying/normalization step for this series.\n   \n   In any case, you might be interested in setting datapoints.format,\n   an array of objects for specifying how a point is normalized and\n   how it interferes with axis scaling.\n\n   The default format array for points is something along the lines of:\n\n     [\n       { x: true, number: true, required: true },\n       { y: true, number: true, required: true }\n     ]\n\n   The first object means that for the first coordinate it should be\n   taken into account when scaling the x axis, that it must be a\n   number, and that it is required - so if it is null or cannot be\n   converted to a number, the whole point will be zeroed out with\n   nulls. Beyond these you can also specify \"defaultValue\", a value to\n   use if the coordinate is null. This is for instance handy for bars\n   where one can omit the third coordinate (the bottom of the bar)\n   which then defaults to 0.\n\n\n - processDatapoints  [phase 3]\n\n   function(plot, series, datapoints)\n \n   Called after normalization of the given series but before finding\n   min/max of the data points. This hook is useful for implementing data\n   transformations. \"datapoints\" contains the normalized data points in\n   a flat array as datapoints.points with the size of a single point\n   given in datapoints.pointsize. Here's a simple transform that\n   multiplies all y coordinates by 2:\n\n     function multiply(plot, series, datapoints) {\n         var points = datapoints.points, ps = datapoints.pointsize;\n         for (var i = 0; i < points.length; i += ps)\n             points[i + 1] *= 2;\n     }\n\n   Note that you must leave datapoints in a good condition as Flot\n   doesn't check it or do any normalization on it afterwards.\n\n\n - drawSeries  [phase 5]\n\n   function(plot, canvascontext, series)\n\n   Hook for custom drawing of a single series. Called just before the\n   standard drawing routine has been called in the loop that draws\n   each series.\n   \n \n - draw  [phase 5]\n\n   function(plot, canvascontext)\n \n   Hook for drawing on the canvas. Called after the grid is drawn\n   (unless it's disabled or grid.aboveData is set) and the series have\n   been plotted (in case any points, lines or bars have been turned\n   on). For examples of how to draw things, look at the source code.\n   \n \n - bindEvents  [phase 6]\n\n   function(plot, eventHolder)\n\n   Called after Flot has setup its event handlers. Should set any\n   necessary event handlers on eventHolder, a jQuery object with the\n   canvas, e.g.\n\n     function (plot, eventHolder) {\n         eventHolder.mousedown(function (e) {\n             alert(\"You pressed the mouse at \" + e.pageX + \" \" + e.pageY);\n         });\n     }\n\n   Interesting events include click, mousemove, mouseup/down. You can\n   use all jQuery events. Usually, the event handlers will update the\n   state by drawing something (add a drawOverlay hook and call\n   triggerRedrawOverlay) or firing an externally visible event for\n   user code. See the crosshair plugin for an example.\n     \n   Currently, eventHolder actually contains both the static canvas\n   used for the plot itself and the overlay canvas used for\n   interactive features because some versions of IE get the stacking\n   order wrong. The hook only gets one event, though (either for the\n   overlay or for the static canvas).\n\n   Note that custom plot events generated by Flot are not generated on\n   eventHolder, but on the div placeholder supplied as the first\n   argument to the plot call. You can get that with\n   plot.getPlaceholder() - that's probably also the one you should use\n   if you need to fire a custom event.\n\n\n - drawOverlay  [phase 7]\n\n   function (plot, canvascontext)\n\n   The drawOverlay hook is used for interactive things that need a\n   canvas to draw on. The model currently used by Flot works the way\n   that an extra overlay canvas is positioned on top of the static\n   canvas. This overlay is cleared and then completely redrawn\n   whenever something interesting happens. This hook is called when\n   the overlay canvas is to be redrawn.\n\n   \"canvascontext\" is the 2D context of the overlay canvas. You can\n   use this to draw things. You'll most likely need some of the\n   metrics computed by Flot, e.g. plot.width()/plot.height(). See the\n   crosshair plugin for an example.\n\n\n - shutdown  [phase 8]\n\n   function (plot, eventHolder)\n\n   Run when plot.shutdown() is called, which usually only happens in\n   case a plot is overwritten by a new plot. If you're writing a\n   plugin that adds extra DOM elements or event handlers, you should\n   add a callback to clean up after you. Take a look at the section in\n   PLUGINS.txt for more info.\n\n   \nPlugins\n-------\n\nPlugins extend the functionality of Flot. To use a plugin, simply\ninclude its Javascript file after Flot in the HTML page.\n\nIf you're worried about download size/latency, you can concatenate all\nthe plugins you use, and Flot itself for that matter, into one big file\n(make sure you get the order right), then optionally run it through a\nJavascript minifier such as YUI Compressor.\n\nHere's a brief explanation of how the plugin plumbings work:\n\nEach plugin registers itself in the global array $.plot.plugins. When\nyou make a new plot object with $.plot, Flot goes through this array\ncalling the \"init\" function of each plugin and merging default options\nfrom the \"option\" attribute of the plugin. The init function gets a\nreference to the plot object created and uses this to register hooks\nand add new public methods if needed.\n\nSee the PLUGINS.txt file for details on how to write a plugin. As the\nabove description hints, it's actually pretty easy.\n\n\nVersion number\n--------------\n\nThe version number of Flot is available in $.plot.version.\n"
  },
  {
    "path": "docroot/flot/LICENSE.txt",
    "content": "Copyright (c) 2007-2009 IOLA and Ole Laursen\n\nPermission is hereby granted, free of charge, to any person\nobtaining a copy of this software and associated documentation\nfiles (the \"Software\"), to deal in the Software without\nrestriction, including without limitation the rights to use,\ncopy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the\nSoftware is furnished to do so, subject to the following\nconditions:\n\nThe above copyright notice and this permission notice shall be\nincluded in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES\nOF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\nNONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT\nHOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,\nWHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\nFROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR\nOTHER DEALINGS IN THE SOFTWARE.\n"
  },
  {
    "path": "docroot/flot/VERSION",
    "content": "0.7\n"
  },
  {
    "path": "docroot/gossip.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n    <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\">\n  <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n  <![endif]-->\n  </head>\n  <body>\n\n    <div class=\"menubar\">\n      <div class=\"nothighlighted\">\n\t    <h2>Scalaris</h2>\n\t  </div>\n   \t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"index.yaws\">Home</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"ring.yaws\">Ring</a>\n      </div>\n      \n<erl>\nout(Arg) ->\n    case whereis(mgmt_server) of\n        undefined -> {html, \"\"};\n        _ ->\n            {html,\n\"      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"indexed-ring.yaws\\\">Indexed Ring</a>\n      </div>\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"vivaldi.yaws\\\">Vivaldi Map</a>\n      </div>\" ++ \n    case config:read(dc_clustering_enable) of\n        true -> \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"dc.yaws\\\">Datacenter Clusters Map</a>\n      </div>\n\";\n        _ -> \"\"\n    end ++ \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"bench.yaws\\\">Benchmarks</a>\n      </div>\n\"}\n    end.\n</erl>\n\n      <div class=\"highlighted\">\n        <a class=\"menua\" href=\"gossip.yaws\">Gossip values</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_client.yaws\">Client Monitor</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_ring.yaws\">Ring Monitor</a>\n      </div>\n\t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"documentation.yaws\">Docu</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"debug.yaws\">Debug</a>\n      </div>\n      <div class=\"nothighlightedlast\">\n        <a class=\"menua\" href=\"logger.yaws\">Message Stats</a>\n      </div>\n    </div><div class=\"middle\">\n      <div class=\"middleleft\">\n<h2>Gossip values</h2>\n\nNodes in this VM: \n<erl> \n out(Arg) ->\n    {html, integer_to_list(length(pid_groups:find_all(dht_node)))}.\n</erl>\n\n<p>\n<erl>\nout(A) ->\n    {ehtml, webhelpers:getGossipRendered()}.\n</erl>\n</p>\n\n<hr />\n<p>Last update:\n<erl>\n out(Arg) ->\n    {Date, {Hour, Minute, Second}} = calendar:local_time(),\n    {html, io_lib:format(\"~2..0B:~2..0B:~2..0B\", [Hour, Minute, Second])}.\n</erl></p>\n</div>\n<br class=\"br_class\">&nbsp;\n</div>\n<div class=\"bottom\">\n</div>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/graphs/.gitignore",
    "content": "/Thumbs.db\n"
  },
  {
    "path": "docroot/icons/.gitignore",
    "content": "/Thumbs.db\n"
  },
  {
    "path": "docroot/ie55-.css",
    "content": "div.nothighlighted {\t\t\n  width: 120px;\n}\n\ndiv.nothighlightedlast {\t\t\n  width: 120px;\n}\n\ndiv.highlighted {\t\t\n  width: 120px;\n}\n\ndiv.highlightedlast {\t\t\n  width: 120px;\n}\n\ndiv.subnothighlighted {\t\t\n  width: 100px;\n}\ndiv.subnothighlightedlast {\t\t\n  width: 100px;\n}\n\ndiv.subhighlighted {\t\t\n  width: 100px;\n}\ndiv.subhighlightedlast {\t\t\n  width: 100px;\n}\n"
  },
  {
    "path": "docroot/images/default/dd/.gitignore",
    "content": "/Thumbs.db\n"
  },
  {
    "path": "docroot/images/default/grid/.gitignore",
    "content": "/Thumbs.db\n"
  },
  {
    "path": "docroot/images/default/panel/.gitignore",
    "content": "/Thumbs.db\n"
  },
  {
    "path": "docroot/images/default/tree/.gitignore",
    "content": "/Thumbs.db\n"
  },
  {
    "path": "docroot/index.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n\t <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\">\n  <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n  <![endif]-->\n  </head>\n  <body>\n\n    <div class=\"menubar\">\n      <div class=\"nothighlighted\">\n\t    <h2>Scalaris</h2>\n\t  </div>\n      <div class=\"highlighted\">\n\t    <a class=\"menua\" href=\"index.yaws\">Home</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"ring.yaws\">Ring</a>\n      </div>\n\n<erl>\nout(Arg) ->\n    case whereis(mgmt_server) of\n        undefined -> {html, \"\"};\n        _ ->\n            {html,\n\"      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"indexed-ring.yaws\\\">Indexed Ring</a>\n      </div>\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"vivaldi.yaws\\\">Vivaldi Map</a>\n      </div>\" ++ \n    case config:read(dc_clustering_enable) of\n        true -> \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"dc.yaws\\\">Datacenter Clusters Map</a>\n      </div>\n\";\n        _ -> \"\"\n    end ++ \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"bench.yaws\\\">Benchmarks</a>\n      </div>\n\"}\n    end.\n</erl>\n\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"gossip.yaws\">Gossip values</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_client.yaws\">Client Monitor</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_ring.yaws\">Ring Monitor</a>\n      </div>\n\t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"documentation.yaws\">Docu</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"debug.yaws\">Debug</a>\n      </div>\n\t  <div class=\"nothighlightedlast\">\n\t    <a class=\"menua\" href=\"logger.yaws\">Message Stats</a>\n\t  </div>\n    </div><div class=\"middle\">\n      <div class=\"middleleft\">\n      \n<erl>\nout(Arg) ->\n    case whereis(mgmt_server) of\n        undefined -> {html, \"<h2>Scalaris Node Info Page</h2>\"};\n        _ ->\n            mgmt_server:number_of_nodes(),\n            NrNodes =\n                receive {get_list_length_response, Res} -> integer_to_list(Res)\n                after 6000 -> \"n/a\"\n                end,\n            {html, io_lib:format(\"\n<h2>Scalaris Management Server Info Page</h2>\n<p>Number of nodes: ~s</p>\", [NrNodes])}\n    end.\n</erl>\n\n<h3>Simple Storage </h3>\n<h4>Add Key Value</h4>\n\n<form method=\"post\" action=\"/index.yaws\">\n\n<table width=\"400\">\n<tr>\n<td>Key</td><td><input name=\"key\" size=\"42\" /></td>\n</tr>\n<tr>\n<td>Value</td><td><input name=\"value\" size=\"42\" /></td>\n</tr>\n</table>\n\n<input type=hidden name=\"type\" value=\"addkey\">\n<p><input type=\"submit\" value=\"Add\"/> | <a href=\"/index.yaws\">Reset</a></p>\n</form>\n\n<p>\n<erl>\nout(A) ->\n    IsPost = webhelpers:isPost(A),\n    if\n      \tIsPost ->\n               {ok, Type} = postvar(A,\"type\"),\n               if \n                 Type =:= \"addkey\" -> \n                    case {postvar(A, \"key\"),postvar(A, \"value\")} of\n\t\t      {{ok, Key},{ok, Value}} ->\n                         {Time, Result} = webhelpers:set_key(Key, Value),\n\t\t    \t {ehtml, {pre, [], io_lib:format('~w ms: ~p', [Time / 1000, Result])}};\n                      _ -> \n                         {ehtml, {pre, [], \"<div style='color: red;'>Missing required input: Need a Key and Value</div>\"}}\n                    end;\n                 true ->\n                    {ehtml,{pre, [], \"\"}}\n               end;\n \ttrue ->\n      \t\t{ehtml,{pre, [], \"\"}}\n    end.\n</erl>\n</p>\n\n\n<h4>Search</h4>\n\n<form method=\"post\" action=\"/index.yaws\">\n\n<p>Key <input name=\"key\" size=\"42\" /></p>\n\n<input type=hidden name=\"type\" value=\"querykey\">\n\n<p><input type=\"submit\" value=\"Lookup\"/> | <a href=\"/index.yaws\">Reset</a></p>\n</form>\n\n\n<p>\n<erl>\nout(A) ->\n    IsPost = webhelpers:isPost(A),\n    if\n      \tIsPost ->\n              {ok, Type} = postvar(A,\"type\"),\n              if \n                Type =:= \"querykey\" ->\n                  case postvar(A, \"key\") of\n                   {ok, Key} ->\n                      {Time, Result} = webhelpers:lookup(Key),\n\t\t      {ehtml, {pre, [], io_lib:format('~w ms: ~p', [Time / 1000, Result])}};\n                    _ ->\n                      {ehtml, {pre, [], \"\"}}\n\t\t  end;\n                true -> \n                      {ehtml, {pre, [], \"\"}}\n              end;\n\ttrue ->\n      \t\t{ehtml, \n       \t\t\t{pre, [], \n        \t\t\"\"}}\n    end.\n</erl>\n</p>\n\n<h4>Delete</h4>\n\n<form method=\"post\" action=\"/index.yaws\">\n<table width=\"400\">\n<tr>\n<td colspan=\"2\">WARNING: This can lead to inconsistent data (e.g. deleted items\ncan re-appear). Also when re-creating an item the version before the\ndelete can re-appear.</td>\n</tr>\n<tr>\n<td>Key</td><td><input name=\"key\" size=\"42\" /></td>\n</tr>\n<tr>\n<td>Timeout</td><td><input name=\"timeout\" size=\"42\" value=2000 /></td>\n</tr>\n</table>\n\n<input type=hidden name=\"type\" value=\"deletekey\">\n\n<p><input type=\"submit\" value=\"Delete\"/> | <a href=\"/index.yaws\">Reset</a></p>\n</form>\n\n<p>\n<erl>\nout(A) ->\n    IsPost = webhelpers:isPost(A),\n    if\n      IsPost ->\n        {ok, Type} = postvar(A,\"type\"),\n        if \n          Type =:= \"deletekey\" ->\n            case {postvar(A, \"key\"),postvar(A, \"timeout\")} of\n              {{ok, Key},{ok, TimeoutStr}} ->\n                Timeout = try list_to_integer(TimeoutStr)\n                          catch _:badarg -> 2000\n                          end,\n                {Time, Result} = webhelpers:delete_key(Key, Timeout),\n                {ehtml, {pre, [], io_lib:format('~w ms: ~p', [Time / 1000, Result])}};\n              _ ->\n                {ehtml, {pre, [], \"\"}}\n            end;\n          true -> \n            {ehtml, {pre, [], \"\"}}\n        end;\n      true ->\n        {ehtml, {pre, [], \"\"}}\n    end.\n</erl>\n</p>\n\n<h3>KVonCseq Storage </h3>\n<h4>Add Key Value</h4>\n\n<form method=\"post\" action=\"/index.yaws\">\n\n<table width=\"400\">\n<tr>\n<td>Key</td><td><input name=\"key\" size=\"42\" /></td>\n</tr>\n<tr>\n<td>Value</td><td><input name=\"value\" size=\"42\" /></td>\n</tr>\n</table>\n\n<input type=hidden name=\"type\" value=\"addkey\">\n<p><input type=\"submit\" value=\"Add\"/> | <a href=\"/index.yaws\">Reset</a></p>\n</form>\n\n<p>\n<erl>\nout(A) ->\n    IsPost = webhelpers:isPost(A),\n    if\n      \tIsPost ->\n               {ok, Type} = postvar(A,\"type\"),\n               if \n                 Type =:= \"addkey\" -> \n                    case {postvar(A, \"key\"),postvar(A, \"value\")} of\n\t\t      {{ok, Key},{ok, Value}} ->\n                         {Result} = webhelpers:set_key_kvoncseq(Key, Value),\n\t\t    \t {ehtml, {pre, [], io_lib:format('~p', [Result])}};\n                      _ -> \n                         {ehtml, {pre, [], \"<div style='color: red;'>Missing required input: Need a Key and Value</div>\"}}\n                    end;\n                 true ->\n                    {ehtml,{pre, [], \"\"}}\n               end;\n \ttrue ->\n      \t\t{ehtml,{pre, [], \"\"}}\n    end.\n</erl>\n</p>\n\n\n<h4>Search</h4>\n\n<form method=\"post\" action=\"/index.yaws\">\n\n<p>Key <input name=\"key\" size=\"42\" /></p>\n\n<input type=hidden name=\"type\" value=\"querykey\">\n\n<p><input type=\"submit\" value=\"Lookup\"/> | <a href=\"/index.yaws\">Reset</a></p>\n</form>\n\n\n<p>\n<erl>\nout(A) ->\n    IsPost = webhelpers:isPost(A),\n    if\n      \tIsPost ->\n              {ok, Type} = postvar(A,\"type\"),\n              if \n                Type =:= \"querykey\" ->\n                  case postvar(A, \"key\") of\n                   {ok, Key} ->\n                      Result = webhelpers:lookup_kvoncseq(Key),\n\t\t      {ehtml, {pre, [], io_lib:format('~p', [Result])}};\n                    _ ->\n                      {ehtml, {pre, [], \"\"}}\n\t\t  end;\n                true -> \n                      {ehtml, {pre, [], \"\"}}\n              end;\n\ttrue ->\n      \t\t{ehtml, \n       \t\t\t{pre, [], \n        \t\t\"\"}}\n    end.\n</erl>\n</p>\n\n<h3>Args</h3>\n<p>\n<erl>\nout(A) ->\n    IsPost = webhelpers:isPost(A),\n    case IsPost of\n      true ->\n        {ehtml, {pre, [], io_lib:format(\"GET:~p~nPOST:~p\", [yaws_api:parse_query(A), yaws_api:parse_post(A)])}};\n      false ->\n        {ehtml, {pre, [], io_lib:format(\"GET:~p\", [yaws_api:parse_query(A)])}}\n    end.\n</erl>\n</p>\n\n<hr />\n<p>Last update:\n<erl>\n out(Arg) ->\n    {Date, {Hour, Minute, Second}} = calendar:local_time(),\n    {html, io_lib:format(\"~2..0B:~2..0B:~2..0B\", [Hour, Minute, Second])}.\n</erl></p>\n</div>\n<br class=\"br_class\">&nbsp;\n</div>\n<div class=\"bottom\">\n</div>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/indexed-ring.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n    <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\">\n  <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n  <![endif]-->\n  <!--[if lte IE 8]>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/excanvas.min.js\"></script>\n  <![endif]-->\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.pie.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.colorhelpers.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"plot_ring.js\"></script>\n  </head>\n  <body>\n\n    <div class=\"menubar\">\n      <div class=\"nothighlighted\">\n\t    <h2>Scalaris</h2>\n\t  </div>\n      <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"index.yaws\">Home</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"ring.yaws\">Ring</a>\n      </div>\n      \n<erl>\nout(Arg) ->\n    case whereis(mgmt_server) of\n        undefined -> {html, \"\"};\n        _ ->\n            {html,\n\"      <div class=\\\"highlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"indexed-ring.yaws\\\">Indexed Ring</a>\n      </div>\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"vivaldi.yaws\\\">Vivaldi Map</a>\n      </div>\" ++ \n    case config:read(dc_clustering_enable) of\n        true -> \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"dc.yaws\\\">Datacenter Clusters Map</a>\n      </div>\n\";\n        _ -> \"\"\n    end ++ \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"bench.yaws\\\">Benchmarks</a>\n      </div>\n\"}\n    end.\n</erl>\n\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"gossip.yaws\">Gossip values</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_client.yaws\">Client Monitor</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_ring.yaws\">Ring Monitor</a>\n      </div>\n\t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"documentation.yaws\">Docu</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"debug.yaws\">Debug</a>\n      </div>\n      <div class=\"nothighlightedlast\">\n        <a class=\"menua\" href=\"logger.yaws\">Message Stats</a>\n      </div>\n    </div><div class=\"middle\">\n      <div class=\"middleleft\">\n<h2>Indexed Ring</h2>\n\nNumber of nodes: \n<erl> \n out(Arg) ->\n    mgmt_server:number_of_nodes(),\n    receive\n        {get_list_length_response,Res} ->\n            ok\n    end,\n    {html, integer_to_list(Res)}.\n</erl>\n<p>\n<erl>\nout(A) ->\n    {ehtml, webhelpers:getRingChart()}.\n</erl>\n</p>\n\n<p>\n<erl>\nout(A) ->\n    {ehtml, webhelpers:getIndexedRingRendered()}.\n</erl>\n</p>\n\n<hr />\n<p>Last update:\n<erl>\n out(Arg) ->\n    {Date, {Hour, Minute, Second}} = calendar:local_time(),\n    {html, io_lib:format(\"~2..0B:~2..0B:~2..0B\", [Hour, Minute, Second])}.\n</erl></p>\n</div>\n<br class=\"br_class\">&nbsp;\n</div>\n<div class=\"bottom\">\n</div>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/jsonrpc.yaws",
    "content": "<erl module=jsonrpc_mod>\n-compile(export_all).\n-include(\"yaws_handler.hrl\").\n\nforward_to_handler(Operation, Params) ->\n  api_json:handler(Operation, Params).\n\n</erl>\n"
  },
  {
    "path": "docroot/local.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n    <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\">\n  <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n  <![endif]-->\n  </head>\n  <body>\n\n<fieldset style=\"border: 1px black solid\">\n\n<legend style=\"border: 1px black solid; margin-left: 1em; padding: 0.2em 0.8em \">Local Nodes</legend>\n\n<erl>\nmake_local_link(Node) -> io_lib:format(\"node.yaws?group=~s\", [Node]).\n\nmake_a(Node) -> {a, [{href, make_local_link(Node)}], Node}.\nout(A) ->\n    Groups = pid_groups:groups(),\n    Nodes = [Group || Group <- Groups, lists:prefix(\"dht_node_\", Group)],\n    Rows = lists:map(fun (Node) -> {tr, [], [{td, [], make_a(Node)}]} end, Nodes),\n    {ehtml, {table, [], Rows}}.\n</erl>\n</fieldset>\n\n<fieldset style=\"border: 1px black solid\">\n<legend style=\"border: 1px black solid; margin-left: 1em; padding: 0.2em 0.8em \">Failure Detector</legend>\n\n</fieldset>\n\n<hr />\n<p>Last update:\n<erl>\n out(Arg) ->\n    {Date, {Hour, Minute, Second}} = calendar:local_time(),\n    {html, io_lib:format(\"~2..0B:~2..0B:~2..0B\", [Hour, Minute, Second])}.\n</erl></p>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/logger.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n    <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\">\n  <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n  <![endif]-->\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"tablesort/jquery.tablesorter.min.js\"></script>\n    <link rel=\"stylesheet\" href=\"tablesort/blue/style.css\" type=\"text/css\"/>\n    <script language=\"javascript\" type=\"text/javascript\">\n        $(document).ready(function() {\n            $(\"table\").tablesorter();\n        });\n    </script>\n  </head>\n  <body>\n\n    <div class=\"menubar\">\n      <div class=\"nothighlighted\">\n        <h2>Scalaris</h2>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"index.yaws\">Home</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"ring.yaws\">Ring</a>\n      </div>\n\n<erl>\nout(Arg) ->\n    case whereis(mgmt_server) of\n        undefined -> {html, \"\"};\n        _ ->\n            {html,\n\"      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"indexed-ring.yaws\\\">Indexed Ring</a>\n      </div>\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"vivaldi.yaws\\\">Vivaldi Map</a>\n      </div>\" ++\n    case config:read(dc_clustering_enable) of\n        true -> \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"dc.yaws\\\">Datacenter Clusters Map</a>\n      </div>\n\";\n        _ -> \"\"\n    end ++ \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"bench.yaws\\\">Benchmarks</a>\n      </div>\n\"}\n    end.\n</erl>\n\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"gossip.yaws\">Gossip values</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_client.yaws\">Client Monitor</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_ring.yaws\">Ring Monitor</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"documentation.yaws\">Docu</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"debug.yaws\">Debug</a>\n      </div>\n      <div class=\"highlightedlast\">\n        <a class=\"menua\" href=\"logger.yaws\">Message Stats</a>\n      </div>\n    </div><div class=\"middle\">\n      <div class=\"middleleft\">\n<h2>Message Statistics</h2>\n\n<erl>\nmake_footer(Start, TotalBytes, TotalCount) ->\n    [{tr, [],\n       [\n         {td, [], {p, [], \"Total\"}},\n         {td, [], io_lib:format(\"~15.10B\", [TotalBytes])},\n         {td, [], io_lib:format(\"~15.10B\", [TotalCount])},\n         {td, [], io_lib:format(\"~15.3f\", [TotalBytes / erlang:max(1, timer:now_diff(os:timestamp(), Start)) * 1000000.0])},\n         {td, [], io_lib:format(\"~15.3f\", [TotalCount / erlang:max(1, timer:now_diff(os:timestamp(), Start)) * 1000000.0])},\n         {td, [], io_lib:format(\"~15.3f\", [TotalBytes / TotalCount])},\n         {td, [], io_lib:format(\"~15.3f\", [100.0])},\n         {td, [], io_lib:format(\"~15.3f\", [100.0])}\n       ] }\n].\n\nmake_table([], _, _, _, _) ->\n    [];\nmake_table([{Tag, _} | Rest], Map, Start, TotalBytes, TotalCount) ->\n    {Size, Count} = gb_trees:get(Tag, Map),\n    RealTag = case Tag of\n                {Channel, MsgTag} -> {Channel, util:extint2atom(MsgTag)};\n                MsgTag -> util:extint2atom(MsgTag)\n              end,\n    [{tr, [],\n       [\n         {td, [], {p, [], io_lib:format(\"~p\", [RealTag])}},\n         {td, [], io_lib:format(\"~15.10B\", [Size])},\n         {td, [], io_lib:format(\"~15.10B\", [Count])},\n         {td, [], io_lib:format(\"~15.3f\", [Size / erlang:max(1, timer:now_diff(os:timestamp(), Start)) * 1000000.0])},\n         {td, [], io_lib:format(\"~15.3f\", [Count / erlang:max(1, timer:now_diff(os:timestamp(), Start)) * 1000000.0])},\n         {td, [], io_lib:format(\"~15.3f\", [Size / Count])},\n         {td, [], io_lib:format(\"~15.3f\", [Size / TotalBytes * 100.0])},\n         {td, [], io_lib:format(\"~15.3f\", [Count / TotalCount * 100.0])}\n       ] }\n       | make_table(Rest, Map, Start, TotalBytes, TotalCount)].\n\nget_totals(none, TotalBytes, TotalCount) ->\n    {TotalBytes, TotalCount};\nget_totals({_Tag, {Size, Count}, Iterator}, TotalBytes, TotalCount) ->\n    get_totals(gb_trees:next(Iterator), Size + TotalBytes, Count + TotalCount).\n\nget_tags(none) ->\n    [];\nget_tags({Tag, {Size, _Count}, Iterator}) ->\n    [{Tag, Size} | get_tags(gb_trees:next(Iterator))].\n\ntree_log_to_table(Map, Start, Heading) ->\n    case gb_trees:size(Map) of\n      0 ->\n        [{h3, [], Heading},\n         {p, [], \"No messages logged. Enable logging in <tt>include/scalaris.hrl</tt> (<tt>LOG_MESSAGE</tt> and/or <tt>LOG_MESSAGE_SOCK</tt> macro).\"}];\n      _ ->\n        SortedTagList = lists:reverse(lists:keysort(2, get_tags(gb_trees:next(gb_trees:iterator(Map))))),\n        {TotalBytes, TotalCount} = get_totals(gb_trees:next(gb_trees:iterator(Map)), 0, 0),\n        TableHead = {thead, [], [{tr, [],\n          [\n            {th, [], \"Message Tag\"},\n            {th, [], \"Total Size\"},\n            {th, [], \"Total Count\"},\n            {th, [], \"Average Byte/s\"},\n            {th, [], \"Average 1/s\"},\n            {th, [], \"Average Msg Size\"},\n            {th, [], \"Percent of Bytes\"},\n            {th, [], \"Percent of Count\"}\n            ]}\n          ]},\n        TableBody = {tbody, [], [make_table(SortedTagList, Map, Start, TotalBytes, TotalCount)]},\n        TableFooter = {tfoot, [], [make_footer(Start, TotalBytes, TotalCount)]},\n        [{h3, [], Heading},\n         {table, [{class, 'tablesorter'}], [TableHead, TableBody, TableFooter]}]\n    end.\n\nout(Arg) ->\n    {Received, Sent, Start} = comm_logger:dump(),\n    {ehtml, [tree_log_to_table(Received, Start, \"Received Messages\") ++ tree_log_to_table(Sent, Start, \"Sent Messages\")]}.\n</erl>\n\n<hr />\n<p>Last update:\n<erl>\n out(Arg) ->\n    {Date, {Hour, Minute, Second}} = calendar:local_time(),\n    {html, io_lib:format(\"~2..0B:~2..0B:~2..0B\", [Hour, Minute, Second])}.\n</erl></p>\n</div>\n<br class=\"br_class\">&nbsp;\n</div>\n<div class=\"bottom\">\n</div>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/main.css",
    "content": "body {\t\n  margin-bottom: 1em;\n  color: #333333;\n  background-color: white;\n  font-size: 14px;\t\n  font-family: arial, helvetica, sans-serif;\n  line-height: 1.4em;\n  /* Center in IE5.5 */\n  /*text-align: center;*/\n  padding: 25px; \n}\n\nimg { \n  border: none;\n} \n\nh1 { \n  font-size: 2em;\n  letter-spacing: 1px;\n  font-family: verdana, helvetica, sans-serif;\n}\n\nh2 { \n  margin-top: 10px;\n  font-size: 18px;\n  letter-spacing: 1px;\n  color: #008080;\n  font-family: tahoma, helvetica, sans-serif;\n }\n\nh2.name { \n  margin-top: 25px;\n  letter-spacing: 1px;\n  color: #344f73;\n  font-family: tahoma, helvetica, sans-serif;\n}\n\nh3 {   \n padding: 0px;\n  letter-spacing: 1px;\n font-size: 15px;\n margin-top: 20px;\n margin-bottom: 8px; \n  color: #008080;\n  font-family: tahoma, helvetica, sans-serif;\n}\n\np { \n margin-top: 6px;\n margin-bottom: 6px; }\n\ndiv.logo { \n  float: left;\n  height: 85px;\n}\n\n.header {  \n  height: 100px;\n} \n\n.left{ \n  height: 86px;\n  float:left;\n  text-align: left;\n  vertical-align: textbottom;\n}\n\n.right{ \n  margin-top: 0px;\n  margin-left: 150px;\n  text-align: right;\n}\n\n.bottom { \n  text-align: right;\n  color: #003838;\n  clear: both;\n  text-align: right;\n  min-width: 120px;\n  padding:10px;\n}\n\na {\t\t\n  color: #008080;\n  text-decoration: underline;\n  font-weight: 100;\n}\n\na:hover {\t\n  color: #008080;\n  text-decoration: underline;\n}\n\n.menua { \n  color: #759e7e;\n  color: #008080;\n  text-decoration: none;\n  font-weight: 900;\n }\n\na.menu:hover {\t\n  color: #008080;\n  text-decoration: underline;\n}\n\n.pieLabel a {  \n  color: #FFFFFF;\n  text-decoration: underline;\n}\n\ndiv.submenu {\t\t\n  margin-left: 10px;\n}\n\ndiv.green {\n  color: #008080;\n  float: left;\n}\n\n.middle { \n  float:none;\n  margin: 0px;\n  margin-left: 124px; \n  margin-top: 1px;\n  color: #003838;\n  border: 1px solid #008080;\n  background-color: #d5dede;\n  padding:10px;\n}\n\n.middleleft { \n  float: left;\n  margin-top: -1px;\n  color: #003838;\n  background-color: #d5dede;\n  padding:10px;\n  padding-top: 5px;\n}\n\n.middleright { \n  color: #003838;\n  background-color: #d5dede;\n  padding:10px;\n  padding-top: 0px;\n  text-align: right;\n  float:right;\n}\n\ndiv.specmenu {\n  float:none;\n  margin-top: 10px;\n}\n\n.menubar { \n  font-family: tahoma, helvetica, sans-serif;\n  float: left;\n  margin: 0px;\n  margin-top: 0px;\n  padding: 0px;\n  padding-top: 1px;\n  text-align: left;\n}\n\ndiv.nothighlighted {\t\t\n  float: top;\n  font-size: 13px;\n  letter-spacing: 1px;\n  margin: 0px;\n  border: 0px;\n  padding: 2px;\n  padding-top: 6px;\n  padding-bottom: 6px;\n  margin-bottom: -1px;\n  border: 1px solid #008080;\n  width: 113px;\n  padding-left: 8px;\n}\n\ndiv.nothighlightedlast {\t\t\n  float: top;\n  font-size: 13px;\n  letter-spacing: 1px;\n  margin: 0px;\n  padding: 2px;\n  padding-top: 6px;\n  padding-bottom: 6px;\n  border: 1px solid #008080;\n  width: 113px;\n  padding-left: 8px;\n}\n\ndiv.hsubnothighlighted {\t\t\n  text-align: center;\n  font-size: 13px;\n  float: left;\n  padding: 0px;\n  margin-left: -1px;\n  border: 1px solid #008080;\n  border-top: 0px;\n  width: 92px;\n  padding-bottom: 1px;\n}\n\n\ndiv.hsubhighlighted {\t\t\n  text-align: center;\n  font-size: 13px;\n  float: left;\n  padding: 0px;\n  margin-left: -1px;\n  border: 1px solid #008080;\n  border-top: 0px;\n  padding-bottom: 1px;\n  width: 92px;\n  background-color: #c5cece; \n  color: #d5dede;\n}\n\ndiv.hvsubnothighlighted {\t\t\n  text-align: left;\n  float: top;\n  padding: 0px;\n  padding-top: 1px;\n  padding-bottom: 0px;\n  margin-bottom: -1px;\n  margin-left: 0px;\n  border: 0px solid #008080;\n  width: 92px;\n}\n\ndiv.hvhsubnothighlighted {\t\t\n  text-align: left;\n  float: left;\n  padding: 0px;\n  padding-top: 1px;\n  padding-bottom: 0px;\n  margin-bottom: -1px;\n  margin-left: 0px;\n  border: 0px solid #008080;\n  width: 31px;\n}\n\ndiv.hvhsubhighlighted {\t\t\n  text-align: left;\n  float: left;\n  padding: 0px;\n  padding-top: 1px;\n  padding-bottom: 0px;\n  margin-bottom: -1px;\n  margin-left: 0px;\n  border: 0px solid #008080;\n  background-color: #c5cece; \n  width: 31px;\n}\n\ndiv.hvsubhighlighted {\t\t\n  text-align: left;\n  float: top;\n  padding: 0px;\n  padding-top: 1px;\n  padding-bottom: 0px;\n  margin-bottom: -1px;\n  margin-left: 0px;\n  border: 0px solid #008080;\n  width: 92px;\n  background-color: #c5cece; \n  color: #d5dede;\n}\n\ndiv.subnothighlighted {\t\t\n  float: top;\n  padding: 1px;\n  padding-top: 3px;\n  padding-bottom: 3px;\n  margin-bottom: -1px;\n  margin-left: 20px;\n  font-size: 12px;\n  letter-spacing: 1px;\n  border: 1px solid #008080;\n  width: 97px;\n  padding-left: 5px;\n}\n\ndiv.highlighted {\t\t\n  float: top;\n  font-size: 13px;\n  letter-spacing: 1px;\n  margin: 0px;\n  padding: 2px;\n  padding-top: 6px;\n  padding-bottom: 6px;\n  width: 114px;\n  padding-left: 8px;\n  background-color: #d5dede;\n  color: #008080;\n  text-decoration: none;\n  font-weight: 900;\n  border: 1px solid #008080;\n  margin-bottom: -1px;\n  border-right: 0px solid #d5dede;\n}\n\ndiv.highlightedlast {\t\t\n  float: top;\n  font-size: 13px;\n  letter-spacing: 1px;\n  margin: 0px;\n  padding: 2px;\n  padding-top: 6px;\n  padding-bottom: 6px;\n  width: 114px;\n  padding-left: 8px;\n  background-color: #d5dede;\n  color: #008080;\n  text-decoration: none;\n  font-weight: 900;\n  border: 1px solid #008080;\n  border-right: 0px solid #d5dede;\n}\n\ndiv.subhighlighted {\t\t\n  float: top;\n  padding: 1px;\n  padding-top: 3px;\n  padding-bottom: 3px;\n  padding-left: 5px;\n  background-color: #d5dede;\n  color: #008080;\n  text-decoration: none;\n  font-weight: 900;\n  font-size: 12px;\n  letter-spacing: 1px;\n  width: 98px;\n  border: 1px solid #008080;\n  margin-bottom: -1px;\n  margin-left: 20px;\n  border-right: 0px solid #d5dede;\n}\n\nhr.bottomhr { \n  background-color: #99cc00;\n  height: 3px;\n  border: none;\n}\n\nbr.br_class {\t\t\n  clear: both;\n}\n\nul {\n  list-style-type: square;\n}\n\ncircle.node {\n    fill: #39c;\n}\n\n.axis path,\n.axis line {\n    fill: none;\n    stroke: black;\n}\n\n.axis text {\n    font-family: sans-serif;\n    font-size: 11px;\n}\n\n#stats ul {\n    display:inline;\n    list-style-type:none;\n}\n\n#stats ul li {\n    float:left;\n    list-style-position:inside;\n}\n\n.clear {\n    clear: both;\n}\n"
  },
  {
    "path": "docroot/monitor_client.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n    <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\">\n  <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n  <![endif]-->\n  <!--[if lte IE 8]>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/excanvas.min.js\"></script>\n  <![endif]-->\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.pie.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.colorhelpers.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.navigate.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.errorbars.min.js\"></script>\n  </head>\n  <body>\n\n    <div class=\"menubar\">\n      <div class=\"nothighlighted\">\n\t    <h2>Scalaris</h2>\n\t  </div>\n   \t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"index.yaws\">Home</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"ring.yaws\">Ring</a>\n      </div>\n      \n<erl>\nout(Arg) ->\n    case whereis(mgmt_server) of\n        undefined -> {html, \"\"};\n        _ ->\n            {html,\n\"      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"indexed-ring.yaws\\\">Indexed Ring</a>\n      </div>\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"vivaldi.yaws\\\">Vivaldi Map</a>\n      </div>\" ++ \n    case config:read(dc_clustering_enable) of\n        true -> \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"dc.yaws\\\">Datacenter Clusters Map</a>\n      </div>\n\";\n        _ -> \"\"\n    end ++ \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"bench.yaws\\\">Benchmarks</a>\n      </div>\n\"}\n    end.\n</erl>\n\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"gossip.yaws\">Gossip values</a>\n      </div>\n      <div class=\"highlighted\">\n        <a class=\"menua\" href=\"monitor_client.yaws\">Client Monitor</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_ring.yaws\">Ring Monitor</a>\n      </div>\n\t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"documentation.yaws\">Docu</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"debug.yaws\">Debug</a>\n      </div>\n      <div class=\"nothighlightedlast\">\n        <a class=\"menua\" href=\"logger.yaws\">Message Stats</a>\n      </div>\n    </div><div class=\"middle\">\n      <div class=\"middleleft\">\n<h2>Client Monitor</h2>\n\n<erl>\nout(A) ->\n    {ehtml, webhelpers:getMonitorClientData()}.\n</erl>\n\n<script type=\"text/javascript\">\nfunction showTooltip(x, y, contents) {\n $('<div id=\"tooltip\">' + contents + '</div>').css( {\n  position: 'absolute',\n  display: 'none',\n  top: y + 5,\n  left: x + 5,\n  border: '1px solid #fdd',\n  padding: '2px',\n  'background-color': '#fee',\n  opacity: 0.80\n }).appendTo(\"body\").fadeIn(200);\n}\n$(function() {\n function noneFormatter(v, axis) {\n  return \"&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\" + v.toFixed(axis.tickDecimals);\n }\n function msFormatter(v, axis) {\n  return v.toFixed(axis.tickDecimals) + \" ms\";\n }\n function MbFormatter(v, axis) {\n  return v.toFixed(axis.tickDecimals) + \" MiB\";\n }\n function bFormatter(v, axis) {\n  return v.toFixed(axis.tickDecimals) + \" B\";\n }\n var series = {\n  lines: { show: true, lineWidth: 0.5 },\n  points: { show: true, radius: 1 }\n };\n var xaxes = [ { mode: 'time' } ];\n var legend = {\n  show: true,\n  position: 'sw',\n  noColumns: 6,\n  margin: [0, -50]\n };\n var grid = { hoverable: true };\n var asym_yerrbars = { \n  errorbars: \"y\", \n  yerr: {show: false, asymmetric: true, upperCap: \"-\", lowerCap: \"-\"}\n };\n\n $.plot($(\"#reqlist_cnt\"), [ {data: count_per_s_data, label: \"Count / s\"} ], {\n  series: series,\n  xaxes: xaxes,\n  yaxes: [ { min: 0, tickFormatter: noneFormatter } ],\n  legend: legend,\n  grid: grid\n });\n $(\"#reqlist_cnt\").bind(\"plothover\", plotHoverMinMax);\n\n $.plot($(\"#reqlist_lat\"),\n        [ {data: avg_min_max_ms_data, label: \"Avg\", points: asym_yerrbars},\n          {data: stddev_ms_data, label: \"Stddev\"}\n        ], {\n  series: series,\n  xaxes: xaxes,\n  yaxes: [ { min: 0, tickFormatter: msFormatter } ],\n  legend: legend,\n  grid: grid\n });\n $(\"#reqlist_lat\").bind(\"plothover\", plotHoverMinMax);\n\n $.plot($(\"#memory\"),\n\t\t[ {data: mem_total_data, label: \"mem_total\"},\n\t\t  {data: mem_processes_data, label: \"mem_processes\"},\n          {data: mem_system_data, label: \"mem_system\"},\n          {data: mem_atom_data, label: \"mem_atoms\"},\n          {data: mem_binary_data, label: \"mem_binaries\"},\n          {data: mem_ets_ms_data, label: \"mem_ets_tables\"}\n\t\t], {\n  series: series,\n  xaxes: xaxes,\n  yaxes: [ { min: 0, tickFormatter: MbFormatter } ],\n  legend: legend,\n  grid: grid\n });\n $(\"#memory\").bind(\"plothover\", plotHover);\n\n $.plot($(\"#io\"),\n        [ //{data: io_rcv_count_data, label: \"Received packets\", yaxis: 1},\n          {data: io_rcv_bytes_data, label: \"Received bytes\", yaxis: 2},\n          //{data: io_send_count_data, label: \"Sent packets\", yaxis: 1},\n          {data: io_send_bytes_data, label: \"Sent bytes\", yaxis: 2}\n        ], {\n  series: series,\n  xaxes: xaxes,\n  yaxes: [\n    { min: 0, tickFormatter: noneFormatter, position: \"left\", alignTicksWithAxis: 1 },\n    { min: 0, tickFormatter: bFormatter, position: \"left\", alignTicksWithAxis: 1 }\n   ],\n  legend: legend,\n  grid: grid\n });\n $(\"#io\").bind(\"plothover\", plotHover);\n\n var previousPoint = null;\n function plotHover(event, pos, obj) {\n  if (!obj) {\n   $(\"#tooltip\").remove();\n   previousPoint = null;\n   return;\n  }\n  if (previousPoint != obj.datapoint) {\n   previousPoint = obj.datapoint;\n   $(\"#tooltip\").remove();\n   var x = obj.datapoint[0].toFixed(2),\n       y = obj.datapoint[1].toFixed(2);\n   var xDate = new Date();\n   xDate.setTime(x)\n   showTooltip(obj.pageX, obj.pageY,\n               xDate.toUTCString() + \": <br />\" + obj.series.label + \" = \" + y);\n  }\n }\n function plotHoverMinMax(event, pos, obj) {\n  if (!obj) {\n   $(\"#tooltip\").remove();\n   previousPoint = null;\n   return;\n  }\n  if (previousPoint != obj.datapoint) {\n   previousPoint = obj.datapoint;\n   $(\"#tooltip\").remove();\n   var x = obj.datapoint[0].toFixed(2),\n       y = obj.datapoint[1].toFixed(2),\n       ymin = (obj.datapoint[1] - obj.datapoint[2]).toFixed(2),\n       ymax = (obj.datapoint[1] + obj.datapoint[3]).toFixed(2);\n   var xDate = new Date();\n   xDate.setTime(x)\n   showTooltip(obj.pageX, obj.pageY,\n               xDate.toUTCString() + \": <br />\" + obj.series.label + \" = \" + y + \", min: \" + ymin + \", max: \" + ymax);\n  }\n }\n});\n</script>\n<p>Count (api_tx:req_list/2)</p>\n<div id=\"reqlist_cnt\" style=\"width: 800px; height: 300px; margin-bottom:50px\"></div>\n<p>Latency (api_tx:req_list/2)</p>\n<div id=\"reqlist_lat\" style=\"width: 800px; height: 300px; margin-bottom:50px\"></div>\n<hr />\n<p>Memory</p>\n<div id=\"memory\" style=\"width: 800px; height: 300px; margin-bottom:50px\"></div>\n<p>I/O - Traffic (10s aggregates)</p>\n<div id=\"io\" style=\"width: 800px; height: 300px; margin-bottom:50px\"></div>\n\n<hr />\n<p>Last update:\n<erl>\n out(Arg) ->\n    {Date, {Hour, Minute, Second}} = calendar:local_time(),\n    {html, io_lib:format(\"~2..0B:~2..0B:~2..0B\", [Hour, Minute, Second])}.\n</erl></p>\n</div>\n<br class=\"br_class\">&nbsp;\n</div>\n<div class=\"bottom\">\n</div>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/monitor_ring.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n    <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\">\n  <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n  <![endif]-->\n  <!--[if lte IE 8]>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/excanvas.min.js\"></script>\n  <![endif]-->\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.pie.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.colorhelpers.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.navigate.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.errorbars.min.js\"></script>\n  </head>\n  <body>\n\n    <div class=\"menubar\">\n      <div class=\"nothighlighted\">\n\t    <h2>Scalaris</h2>\n\t  </div>\n   \t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"index.yaws\">Home</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"ring.yaws\">Ring</a>\n      </div>\n      \n<erl>\nout(Arg) ->\n    case whereis(mgmt_server) of\n        undefined -> {html, \"\"};\n        _ ->\n            {html,\n\"      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"indexed-ring.yaws\\\">Indexed Ring</a>\n      </div>\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"vivaldi.yaws\\\">Vivaldi Map</a>\n      </div>\" ++ \n    case config:read(dc_clustering_enable) of\n        true -> \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"dc.yaws\\\">Datacenter Clusters Map</a>\n      </div>\n\";\n        _ -> \"\"\n    end ++ \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"bench.yaws\\\">Benchmarks</a>\n      </div>\n\"}\n    end.\n</erl>\n\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"gossip.yaws\">Gossip values</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_client.yaws\">Client Monitor</a>\n      </div>\n      <div class=\"highlighted\">\n        <a class=\"menua\" href=\"monitor_ring.yaws\">Ring Monitor</a>\n      </div>\n\t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"documentation.yaws\">Docu</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"debug.yaws\">Debug</a>\n      </div>\n      <div class=\"nothighlightedlast\">\n        <a class=\"menua\" href=\"logger.yaws\">Message Stats</a>\n      </div>\n    </div><div class=\"middle\">\n      <div class=\"middleleft\">\n<h2>Ring Monitor</h2>\n\n<erl>\nout(A) ->\n    {ehtml, webhelpers:getMonitorRingData()}.\n</erl>\n\n<script type=\"text/javascript\">\nfunction showTooltip(x, y, contents) {\n $('<div id=\"tooltip\">' + contents + '</div>').css( {\n  position: 'absolute',\n  display: 'none',\n  top: y + 5,\n  left: x + 5,\n  border: '1px solid #fdd',\n  padding: '2px',\n  'background-color': '#fee',\n  opacity: 0.80\n }).appendTo(\"body\").fadeIn(200);\n}\n$(function() {\n function msFormatter(v, axis) {\n  return v.toFixed(axis.tickDecimals) +\"ms\";\n }\n var series = {\n  lines: { show: true, lineWidth: 0.5 },\n  points: { show: true, radius: 1 }\n };\n var xaxes = [ { mode: 'time' } ];\n var legend = {\n  show: true,\n  position: 'sw',\n  noColumns: 6,\n  margin: [0, -50]\n };\n var grid = { hoverable: true };\n var asym_yerrbars = { \n  errorbars: \"y\", \n  yerr: {show: false, asymmetric: true, upperCap: \"-\", lowerCap: \"-\"}\n };\n $.plot($(\"#rr_lat\"),\n        [ {data: rr_avg_min_max_ms_data, label: \"Avg (ms)\", points: asym_yerrbars},\n          {data: rr_stddev_ms_data, label: \"Stddev (ms)\"}\n        ], {\n  series: series,\n  xaxes: xaxes,\n  yaxes: [ { min: 0, tickFormatter: msFormatter } ],\n  legend: legend,\n  grid: grid\n });\n $(\"#rr_lat\").bind(\"plothover\", plotHover);\n $.plot($(\"#lh_count\"),\n        [ {data: lh_avg_min_max_count_data, label: \"Avg\", points: asym_yerrbars},\n          {data: lh_stddev_count_data, label: \"Stddev\"}\n        ], {\n  series: series,\n  xaxes: xaxes,\n  yaxes: [ { min: 0 } ],\n  legend: legend,\n  grid: grid\n });\n $(\"#lh_count\").bind(\"plothover\", plotHover);\n $.plot($(\"#tx_count\"), [ {data: tx_count_per_s_data, label: \"Count / s\"} ], {\n  series: series,\n  xaxes: xaxes,\n  yaxes: [ { min: 0 } ],\n  legend: legend,\n  grid: grid\n });\n $(\"#tx_count\").bind(\"plothover\", plotHover);\n var asym_yerrbars = { \n  errorbars: \"y\", \n  yerr: {show: false, asymmetric: true, upperCap: \"-\", lowerCap: \"-\"}\n };\n $.plot($(\"#tx_lat\"),\n        [ {data: tx_avg_min_max_ms_data, label: \"Avg (ms)\", points: asym_yerrbars},\n          {data: tx_stddev_ms_data, label: \"Stddev (ms)\"}\n        ], {\n  series: series,\n  xaxes: xaxes,\n  yaxes: [ { min: 0, tickFormatter: msFormatter } ],\n  legend: legend,\n  grid: grid\n });\n $(\"#tx_lat\").bind(\"plothover\", plotHover);\n var previousPoint = null;\n function plotHover(event, pos, obj) {\n  if (!obj) {\n   $(\"#tooltip\").remove();\n   previousPoint = null;\n   return;\n  }\n  if (previousPoint != obj.datapoint) {\n   previousPoint = obj.datapoint;\n   $(\"#tooltip\").remove();\n   var x = obj.datapoint[0].toFixed(2),\n       y = obj.datapoint[1].toFixed(2),\n       ymin = (obj.datapoint[1] - obj.datapoint[2]).toFixed(2),\n       ymax = (obj.datapoint[1] + obj.datapoint[3]).toFixed(2);\n   var xDate = new Date();\n   xDate.setTime(x)\n   showTooltip(obj.pageX, obj.pageY,\n               xDate.toUTCString() + \": <br />\" + obj.series.label + \" = \" + y + \", min: \" + ymin + \", max: \" + ymax);\n  }\n }\n});\n</script>\n<p>Count (api_tx:req_list/2)</p>\n<div id=\"tx_count\" style=\"width: 800px; height: 300px; margin-bottom:50px\"></div>\n<p>Latency (api_tx:req_list/2)</p>\n<div id=\"tx_lat\" style=\"width: 800px; height: 300px; margin-bottom:50px\"></div>\n<p>Latency read-read-commit (monitor_perf:run_bench/2)</p>\n<div id=\"rr_lat\" style=\"width: 800px; height: 300px; margin-bottom:50px\"></div>\n<p>Lookup Hop Count</p>\n<div id=\"lh_count\" style=\"width: 800px; height: 300px; margin-bottom:50px\"></div>\n\n<hr />\n<p>Last update:\n<erl>\n out(Arg) ->\n    {Date, {Hour, Minute, Second}} = calendar:local_time(),\n    {html, io_lib:format(\"~2..0B:~2..0B:~2..0B\", [Hour, Minute, Second])}.\n</erl></p>\n</div>\n<br class=\"br_class\">&nbsp;\n</div>\n<div class=\"bottom\">\n</div>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/node.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n    <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\">\n  <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n  <![endif]-->\n  </head>\n  <body>\n\n<erl>\nget_yaws_info_for_node(Pid) ->\n  comm:send(Pid, {get_yaws_info, comm:this()}),\n  receive\n    {get_yaws_info_response, IP, Port, Group} ->\n      {IP, Port, Group}\n  after 1000 -> timeout end.\n\nget_yaws_url_for_node(Pid) ->\n  case get_yaws_info_for_node(Pid) of\n    {IP, Port, Group} ->\n      {I0, I1, I2, I3} = IP,\n      io_lib:format(\"http://~w.~w.~w.~w:~w/node.yaws?group=~s\", [I0, I1, I2, I3, Port, Group]);\n    timeout ->\n      \"unknown\"\n  end.\n\nget_node_details(Group) ->\n  Pid = pid_groups:pid_of(Group, dht_node),\n  comm:send_local(Pid, {get_node_details, comm:this()}),\n  receive\n    {get_node_details_response, Details} ->\n      Details\n  after\n    1000 ->\n      timeout\n  end.\n\nrender_node(Node) ->\n  Id = node:id(Node),\n  Pid = node:pidX(Node),\n  URL = get_yaws_url_for_node(Pid),\n  {a, [{href, URL}], {p, [], io_lib:format(\"~.16B\", [Id])}}.\n\nget_ring_info(NodeDetails) ->\n  {ehtml, {fieldset, [{style, \"border: 1px black solid\"}],\n  [{legend, [{style, \"border: 1px black solid; margin-left: 1em; padding: 0.2em 0.8em\"}], \"Chord Ring\"},\n   {table, [], [\n{tr, [], [{th, [], \"Predecessor\"}, {th, [], \"Self\"}, {th, [], \"Successor\"}]},\n{tr, [], [{td, [], render_node(node_details:get(NodeDetails, pred))},\n          {td, [], render_node(node_details:get(NodeDetails, node))},\n          {td, [], render_node(node_details:get(NodeDetails, succ))}]}\n]}]}}.\n\nout(A) ->\n  case lists:keyfind(\"group\", 1, yaws_api:parse_query(A)) of\n    {\"group\", Group} ->\n      NodeDetails = get_node_details(Group),\n      get_ring_info(NodeDetails);\n    false ->\n      {ehtml, {pre, [], \"no node provided\"}}\n  end.\n</erl>\n\n<fieldset style=\"border: 1px black solid\">\n\n<legend style=\"border: 1px black solid; margin-left: 1em; padding: 0.2em 0.8em \">Node</legend>\n\n<erl>\nget_web_debug_info(Group) ->\n  Pid = pid_groups:pid_of(Group, dht_node),\n  util:debug_info(Pid).\n\nout(A) ->\n  case lists:keyfind(\"group\", 1, yaws_api:parse_query(A)) of\n    {\"group\", Group} ->\n      Info = get_web_debug_info(Group),\n      {ehtml, {pre, [], io_lib:format(\"~p\", [Info])}};\n    false ->\n      {ehtml, {pre, [], \"no node provided\"}}\n  end.\n</erl>\n</fieldset>\n\n<fieldset style=\"border: 1px black solid\">\n<legend style=\"border: 1px black solid; margin-left: 1em; padding: 0.2em 0.8em \">Routing Table</legend>\n<erl>\nget_state(Group, Which) ->\n  Pid = pid_groups:pid_of(Group, dht_node),\n  comm:send_local(Pid, {get_state, comm:this(), Which}),\n  receive\n    {get_state_response, State} ->\n      State\n  after\n    1000 ->\n      timeout\n  end.\n\nget_yaws_info_for_node(Pid) ->\n  comm:send(Pid, {get_yaws_info, comm:this()}),\n  receive\n    {get_yaws_info_response, IP, Port, Group} ->\n      {IP, Port, Group}\n  after 1000 -> timeout end.\n\nget_yaws_url_for_node(Pid) ->\n  case get_yaws_info_for_node(Pid) of\n    {IP, Port, Group} ->\n      {I0, I1, I2, I3} = IP,\n      io_lib:format(\"http://~w.~w.~w.~w:~w/node.yaws?group=~s\", [I0, I1, I2, I3, Port, Group]);\n    timeout ->\n      \"unknown\"\n  end.\n\nrender_node(Node) ->\n  Id = node:id(Node),\n  Pid = node:pidX(Node),\n  URL = get_yaws_url_for_node(Pid),\n  {a, [{href, URL}], {p, [], io_lib:format(\"~.16B\", [Id])}}.\n\nmake_table(RT) ->\n   {Rows, _} = lists:foldl(fun({_, Node}, {Rows, Idx}) ->\n     {\n       [{tr, [], [{td, [], io_lib:format(\"~p\", [Idx])},\n                 {td, [], render_node(Node)}]} | Rows],\n       Idx + 1\n     } end, {[], 0}, RT),\n   Head = {tr, [], [{th, [], \"Index\"}, {th, [], \"Node\"}]},\n   {table, [], [Head | Rows]}.\n\nout(A) ->\n  case lists:keyfind(\"group\", 1, yaws_api:parse_query(A)) of\n    {\"group\", Group} ->\n      Info = gb_trees:to_list(get_state(Group, rt)),\n      Table = make_table(Info),\n      {ehtml, Table};\n%{pre, [], io_lib:format(\"~p\", [Info])}\n    false ->\n      {ehtml, {pre, [], \"no node provided\"}}\n  end.\n</erl>\n</fieldset>\n\n\n<h3>Args</h3>\n<p>\n<erl>\nout(A) ->\n    IsPost = webhelpers:isPost(A),\n    case IsPost of\n      true ->\n        {ehtml, {pre, [], io_lib:format(\"GET:~p~nPOST:~p\", [yaws_api:parse_query(A), yaws_api:parse_post(A)])}};\n      false ->\n        {ehtml, {pre, [], io_lib:format(\"GET:~p\", [yaws_api:parse_query(A)])}}\n    end.\n</erl>\n</p>\n\n<hr />\n<p>Last update:\n<erl>\n out(Arg) ->\n    {Date, {Hour, Minute, Second}} = calendar:local_time(),\n    {html, io_lib:format(\"~2..0B:~2..0B:~2..0B\", [Hour, Minute, Second])}.\n</erl></p>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/plot_ring.js",
    "content": "function plot_ring(ring, colors) {\n $.plot($(\"#ring\"), ring, {\n  series: {\n   pie: {\n    show: true,\n    radius: 0.9,\n    innerRadius: 0.4,\n    label: {\n     show: true,\n     radius: 1.0,\n     formatter: function(label, series) {\n      return '<div style=\"font-size:8pt;text-align:center;padding:2px;color:white;\">'+label+'</div>';\n     },\n     background: { opacity: 0.5, color: '#000000' },\n     threshold: 0.03\n    }\n   }\n  },\n  legend: { show: false },\n  grid: { hoverable: true, clickable: true },\n  colors: colors\n });\n $(\"#ring\").bind(\"plothover\", pieHover);\n $(\"#ring\").bind(\"plotclick\", pieClick);\n}\n\nfunction pieHover(event, pos, obj) {\n if (!obj)\n  return;\n percent = parseFloat(obj.series.percent).toFixed(2);\n $(\"#ringhover\").html('<span style=\"font-weight: bold\">'+obj.series.label+' ('+percent+'%)</span>');\n}\n\nfunction pieClick(event, pos, obj) {\n if (!obj)\n  return;\n percent = parseFloat(obj.series.percent).toFixed(2);\n alert(''+obj.series.label+': '+percent+'%');\n}\n"
  },
  {
    "path": "docroot/processinfo.yaws",
    "content": "<erl>\nout(A) ->\n    {_, Group} = lists:keyfind(\"group\", 1, yaws_api:parse_query(A)),\n    {_, Id} = lists:keyfind(\"id\", 1, yaws_api:parse_query(A)),\n    if Group =:= undefined -> \n\t       {ehtml, {pre, [], \"\"}}; \n\t   Id =:= undefined ->\n      \t   {ehtml, {pre, [], \"\"}};\n\t   true ->\n\t       {html, json2:encode(pid_groups:get_web_debug_info(Group, Id))}\n    end.\n</erl>\n"
  },
  {
    "path": "docroot/processtree.yaws",
    "content": "<erl>\nout(A) ->\n    IsPost = webhelpers:isPost(A),\n    if\n        IsPost ->\n            case postvar(A, \"node\") of\n                {ok, \"processTreeRoot\"} ->\n                    {html, json2:encode(pid_groups:groups_as_json())};\n                {ok, Node} ->\n                    {html, json2:encode(pid_groups:members_by_name_as_json(Node))};\n                _ ->\n                    {ehtml, {pre, [], \"\"}}\n            end;\n        true ->\n              {ehtml, {pre, [], \"\"}}\n    end.\n</erl>\n"
  },
  {
    "path": "docroot/ring.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n    <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\">\n  <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n  <![endif]-->\n  <!--[if lte IE 8]>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/excanvas.min.js\"></script>\n  <![endif]-->\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.pie.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.colorhelpers.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"plot_ring.js\"></script>\n  </head>\n  <body>\n\n    <div class=\"menubar\">\n      <div class=\"nothighlighted\">\n\t    <h2>Scalaris</h2>\n\t  </div>\n   \t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"index.yaws\">Home</a>\n\t  </div>\n      <div class=\"highlighted\">\n        <a class=\"menua\" href=\"ring.yaws\">Ring</a>\n      </div>\n      \n<erl>\nout(Arg) ->\n    case whereis(mgmt_server) of\n        undefined -> {html, \"\"};\n        _ ->\n            {html,\n\"      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"indexed-ring.yaws\\\">Indexed Ring</a>\n      </div>\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"vivaldi.yaws\\\">Vivaldi Map</a>\n      </div>\" ++ \n    case config:read(dc_clustering_enable) of\n        true -> \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"dc.yaws\\\">Datacenter Clusters Map</a>\n      </div>\n\";\n        _ -> \"\"\n    end ++ \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"bench.yaws\\\">Benchmarks</a>\n      </div>\n\"}\n    end.\n</erl>\n\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"gossip.yaws\">Gossip values</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_client.yaws\">Client Monitor</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_ring.yaws\">Ring Monitor</a>\n      </div>\n\t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"documentation.yaws\">Docu</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"debug.yaws\">Debug</a>\n      </div>\n      <div class=\"nothighlightedlast\">\n        <a class=\"menua\" href=\"logger.yaws\">Message Stats</a>\n      </div>\n    </div><div class=\"middle\">\n      <div class=\"middleleft\">\n<h2>Ring</h2>\n\n<erl> \n out(Arg) ->\n    case whereis(mgmt_server) of\n        undefined ->\n            {html, \"\"};\n        _ ->\n            mgmt_server:number_of_nodes(),\n            receive {get_list_length_response, Res} -> ok end,\n            {html, \"Number of nodes: \" ++ integer_to_list(Res)}\n    end.\n</erl>\n\n<p>\n<erl>\nout(A) ->\n    {ehtml, webhelpers:getRingChart()}.\n</erl>\n</p>\n\n<p>\n<erl>\nout(A) ->\n    {ehtml, webhelpers:getRingRendered()}.\n</erl>\n</p>\n\n<hr />\n<p>Last update:\n<erl>\n out(Arg) ->\n    {Date, {Hour, Minute, Second}} = calendar:local_time(),\n    {html, io_lib:format(\"~2..0B:~2..0B:~2..0B\", [Hour, Minute, Second])}.\n</erl></p>\n</div>\n<br class=\"br_class\">&nbsp;\n</div>\n<div class=\"bottom\">\n</div>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/tablesort/blue/style.css",
    "content": "/* tables */\ntable.tablesorter {\n\tfont-family:arial;\n\tbackground-color: #CDCDCD;\n\tmargin:10px 0pt 15px;\n\tfont-size: 8pt;\n\twidth: 100%;\n\ttext-align: left;\n}\ntable.tablesorter thead tr th, table.tablesorter tfoot tr td {\n\tbackground-color: #e6EEEE;\n\tborder: 1px solid #FFF;\n\tfont-size: 8pt;\n\tpadding: 4px;\n}\ntable.tablesorter thead tr .header {\n\tbackground-image: url(bg.gif);\n\tbackground-repeat: no-repeat;\n\tbackground-position: center right;\n\tcursor: pointer;\n}\ntable.tablesorter tbody td {\n\tcolor: #3D3D3D;\n\tpadding: 4px;\n\tbackground-color: #FFF;\n\tvertical-align: top;\n}\ntable.tablesorter tbody tr.odd td {\n\tbackground-color:#F0F0F6;\n}\ntable.tablesorter thead tr .headerSortUp {\n\tbackground-image: url(asc.gif);\n}\ntable.tablesorter thead tr .headerSortDown {\n\tbackground-image: url(desc.gif);\n}\ntable.tablesorter thead tr .headerSortDown, table.tablesorter thead tr .headerSortUp {\nbackground-color: #8dbdd8;\n}\n\n/* Scalaris adjustments */\n\ntable.tablesorter thead {\n    text-align: left;\n}\n\ntable.tablesorter tbody, tfoot {\n    text-align: right;\n}\n\ntable.tablesorter tfoot {\n    font-weight:bold;\n}\n\ntable.tablesorter p {\n    font-weight:bold;\n    text-align:left;\n}\n\ntable.tablesorter .header {\n  height:50px;\n  width: 120px;\n}\n"
  },
  {
    "path": "docroot/vivaldi.yaws",
    "content": "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n  \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\n  <head>\n    <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"/>\n    <meta http-equiv=\"pragma\" content=\"no-cache\">\n    <meta http-equiv=\"cache-control\" content=\"no-cache\">\n    <meta http-equiv=\"expires\" content=\"-1\">\n    <link rel='shortcut icon' type='image/x-icon' href='/favicon.ico' />\n<erl>\nout(Arg) ->\n    {header, {cache_control, \"no-cache\"}}.\n</erl>\n    <title>Scalaris Management Interface</title>\n    <link rel=\"stylesheet\" type=\"text/css\" href=\"main.css\">\n    <!--[if lte IE 5.5999]>\n    <style type=\"text/css\">@import url(ie55-.css);</style>\n    <![endif]-->\n\n    <!--[if lte IE 8]>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/excanvas.min.js\"></script>\n    <![endif]-->\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.pie.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.colorhelpers.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.navigate.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.errorbars.min.js\"></script>\n    <script language=\"javascript\" type=\"text/javascript\" src=\"flot/jquery.flot.selection.min.js\"></script>\n  </head>\n  <body>\n\n    <div class=\"menubar\">\n      <div class=\"nothighlighted\">\n\t    <h2>Scalaris</h2>\n\t  </div>\n      <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"index.yaws\">Home</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"ring.yaws\">Ring</a>\n      </div>\n      \n<erl>\nout(Arg) ->\n    case whereis(mgmt_server) of\n        undefined -> {html, \"\"};\n        _ ->\n            {html,\n\"      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"indexed-ring.yaws\\\">Indexed Ring</a>\n      </div>\n      <div class=\\\"highlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"vivaldi.yaws\\\">Vivaldi Map</a>\n      </div>\" ++ \n    case config:read(dc_clustering_enable) of\n        true -> \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"dc.yaws\\\">Datacenter Clusters Map</a>\n      </div>\n\";\n        _ -> \"\"\n    end ++ \"\n      <div class=\\\"nothighlighted\\\">\n        <a class=\\\"menua\\\" href=\\\"bench.yaws\\\">Benchmarks</a>\n      </div>\n\"}\n    end.\n</erl>\n\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"gossip.yaws\">Gossip values</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_client.yaws\">Client Monitor</a>\n      </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"monitor_ring.yaws\">Ring Monitor</a>\n      </div>\n\t  <div class=\"nothighlighted\">\n\t    <a class=\"menua\" href=\"documentation.yaws\">Docu</a>\n\t  </div>\n      <div class=\"nothighlighted\">\n        <a class=\"menua\" href=\"debug.yaws\">Debug</a>\n      </div>\n\t  <div class=\"nothighlightedlast\">\n\t    <a class=\"menua\" href=\"logger.yaws\">Message Stats</a>\n\t  </div>\n    </div><div class=\"middle\">\n      <div class=\"middleleft\">\n<h2>Vivaldi Map</h2>\n\n<div id='plot'>\n    <div id=\"loading\"><p>Loading map...</p></div>\n    <div id='graph' style='width:800px;height:400px;float:left;'></div>\n    <div id='legend' style='float:left'></div>\n</div>\n<div class=\"clear\"></div>\n<button id='zoom_out'>Zoom out</button>\n<br />\n<script src=\"dc.js\"></script>\n\n<hr />\n<p>Last update:\n<erl>\n out(Arg) ->\n    {Date, {Hour, Minute, Second}} = calendar:local_time(),\n    {html, io_lib:format(\"~2..0B:~2..0B:~2..0B\", [Hour, Minute, Second])}.\n</erl></p>\n</div>\n<br class=\"br_class\">&nbsp;\n</div>\n<div class=\"bottom\">\n</div>\n</body>\n</html>\n"
  },
  {
    "path": "docroot/vivaldiMap.yaws",
    "content": "<erl>\nout(Arg) ->\n    Nodes = webhelpers:getVivaldiMap(),\n    NodesJSON = webhelpers:format_nodes(Nodes),\n\n    {html, io_lib:format(\"{\n        \\\"nodes\\\":~s\n    }\",[NodesJSON])}\n    .\n</erl>\n"
  },
  {
    "path": "include/.gitignore",
    "content": "/autoconf.hrl\n/rt.hrl\n"
  },
  {
    "path": "include/atom_ext.hrl",
    "content": "%  @copyright 2012 Zuse Institute Berlin\n%  @end\n%\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n%%-------------------------------------------------------------------\n%% File    message_tags.hrl\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Defines integers for some message tags on the hot path to reduce\n%%         overhead (atoms are send as strings!).\n%% @end\n%%-------------------------------------------------------------------\n%% @version $Id$\n\n%% TAKE EXTRA CARE THAT ALL VALUES ARE UNIQUE! %%\n\n-ifdef(enable_debug).\n-define(int_or_atom(Int, Atom), Atom).\n-else.\n-define(int_or_atom(Int, Atom), Int).\n-endif.\n\n%% lookup\n-define(lookup_aux_atom, lookup_aux).\n-define(lookup_aux, ?int_or_atom(01, ?lookup_aux_atom)).\n\n-define(lookup_fin_atom, lookup_fin).\n-define(lookup_fin, ?int_or_atom(02, ?lookup_fin_atom)).\n\n%% comm\n-define(send_to_group_member_atom, send_to_group_member).\n-define(send_to_group_member, ?int_or_atom(11, ?send_to_group_member_atom)).\n\n-define(deliver_atom, deliver).\n-define(deliver, ?int_or_atom(12, ?deliver_atom)).\n\n-define(unpack_msg_bundle_atom, unpack_msg_bundle).\n-define(unpack_msg_bundle, ?int_or_atom(13, ?unpack_msg_bundle_atom)).\n\n-define(quiet_atom, quiet).\n-define(quiet, ?int_or_atom(14, ?quiet_atom)).\n\n-define(send_to_registered_proc_atom, send_to_registered_proc).\n-define(send_to_registered_proc, ?int_or_atom(15, ?send_to_registered_proc_atom)).\n\n%% dht_node\n-define(get_key_with_id_reply_atom, get_key_with_id_reply).\n-define(get_key_with_id_reply, ?int_or_atom(21, ?get_key_with_id_reply_atom)).\n\n-define(get_key_atom, get_key).\n-define(get_key, ?int_or_atom(22, ?get_key_atom)).\n\n-define(read_op_atom, read_op).\n-define(read_op, ?int_or_atom(23, ?read_op_atom)).\n\n-define(read_op_with_id_reply_atom, read_op_with_id_reply).\n-define(read_op_with_id_reply, ?int_or_atom(24, ?read_op_with_id_reply_atom)).\n\n%% paxos\n-define(proposer_accept_atom, proposer_accept).\n-define(proposer_accept, ?int_or_atom(41, ?proposer_accept_atom)).\n\n-define(acceptor_accept_atom, acceptor_accept).\n-define(acceptor_accept, ?int_or_atom(42, ?acceptor_accept_atom)).\n\n-define(paxos_id_atom, paxos_id).\n-define(paxos_id, ?int_or_atom(43, ?paxos_id_atom)).\n\n-define(proposer_initialize_atom, proposer_initialize).\n-define(proposer_initialize, ?int_or_atom(44, ?proposer_initialize_atom)).\n\n-define(proposer_deleteids_atom, proposer_deleteids).\n-define(proposer_deleteids, ?int_or_atom(45, ?proposer_deleteids_atom)).\n\n%% transactions\n-define(register_TP_atom, register_TP).\n-define(register_TP, ?int_or_atom(61, ?register_TP_atom)).\n\n-define(tx_tm_rtm_init_RTM_atom, tx_tm_rtm_init_RTM).\n-define(tx_tm_rtm_init_RTM, ?int_or_atom(62, ?tx_tm_rtm_init_RTM_atom)).\n\n-define(tp_do_commit_abort_atom, tp_do_commit_abort).\n-define(tp_do_commit_abort, ?int_or_atom(63, ?tp_do_commit_abort_atom)).\n\n-define(tx_tm_rtm_delete_atom, tx_tm_rtm_delete).\n-define(tx_tm_rtm_delete, ?int_or_atom(64, ?tx_tm_rtm_delete_atom)).\n\n-define(tp_committed_atom, tp_committed).\n-define(tp_committed, ?int_or_atom(65, ?tp_committed_atom)).\n\n-define(tx_state_atom, tx_state).\n-define(tx_state, ?int_or_atom(66, ?tx_state_atom)).\n\n-define(tx_id_atom, tx_id).\n-define(tx_id, ?int_or_atom(67, ?tx_id_atom)).\n\n-define(tx_item_id_atom, tx_item_id).\n-define(tx_item_id, ?int_or_atom(68, ?tx_item_id_atom)).\n\n-define(tx_item_state_atom, tx_item_state).\n-define(tx_item_state, ?int_or_atom(69, ?tx_item_state_atom)).\n\n-define(commit_client_id_atom, commit_client_id).\n-define(commit_client_id, ?int_or_atom(70, ?commit_client_id_atom)).\n\n-define(undecided_atom, undecided).\n-define(undecided, ?int_or_atom(71, ?undecided_atom)).\n\n-define(prepared_atom, prepared).\n-define(prepared, ?int_or_atom(72, ?prepared_atom)).\n\n-define(commit_atom, commit).\n-define(commit, ?int_or_atom(73, ?commit_atom)).\n\n-define(abort_atom, abort).\n-define(abort, ?int_or_atom(74, ?abort_atom)).\n\n-define(value_atom, value).\n-define(value, ?int_or_atom(75, ?value_atom)).\n\n-define(read_atom, read).\n-define(read, ?int_or_atom(76, ?read_atom)).\n\n-define(write_atom, write).\n-define(write, ?int_or_atom(77, ?write_atom)).\n\n-define(value_dropped_atom, value_dropped).\n-define(value_dropped, ?int_or_atom(78, ?value_dropped_atom)).\n\n-define(init_TP_atom, init_TP).\n-define(init_TP, ?int_or_atom(79, ?init_TP_atom)).\n\n-define(tp_do_commit_abort_fwd_atom, tp_do_commit_abort_fwd).\n-define(tp_do_commit_abort_fwd, ?int_or_atom(80, ?tp_do_commit_abort_fwd_atom)).\n\n-define(random_from_list_atom, random_from_list).\n-define(random_from_list, ?int_or_atom(81, ?random_from_list_atom)).\n\n-define(partial_value_atom, partial_value).\n-define(partial_value, ?int_or_atom(82, ?partial_value_atom)).\n\n-define(sublist_atom, sublist).\n-define(sublist, ?int_or_atom(83, ?sublist_atom)).\n\n-define(ok_atom, ok).\n-define(ok, ?int_or_atom(84, ?ok_atom)).\n\n-define(fail_atom, fail).\n-define(fail, ?int_or_atom(85, ?fail_atom)).\n\n-define(not_found_atom, not_found).\n-define(not_found, ?int_or_atom(86, ?not_found_atom)).\n\n-define(empty_list_atom, empty_list).\n-define(empty_list, ?int_or_atom(87, ?empty_list_atom)).\n\n-define(not_a_list_atom, not_a_list).\n-define(not_a_list, ?int_or_atom(88, ?not_a_list_atom)).\n\n%% rrepair\n-define(check_nodes_atom, check_nodes).\n-define(check_nodes, ?int_or_atom(101, ?check_nodes_atom)).\n\n-define(check_nodes_response_atom, check_nodes_response).\n-define(check_nodes_response, ?int_or_atom(102, ?check_nodes_response_atom)).\n\n-define(key_upd_atom, key_upd).\n-define(key_upd, ?int_or_atom(103, ?key_upd_atom)).\n\n% used in the past - do not re-use this integer for now!\n%% -define(interval_upd_atom, interval_upd).\n%% -define(interval_upd, ?int_or_atom(104, ?interval_upd_atom)).\n"
  },
  {
    "path": "include/client_types.hrl",
    "content": "%  @copyright 2010-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Type definitions for the client, e.g. api_*.\n%% @end\n%% @version $Id$\n\n%% client_key() is actually of type unicode:chardata().\n%% But we cannot generate that type in our tester as it contains\n%% maybe_improper_list() without further specifying its element type.\n%% See ?RT:hash_key/1\n%% -type client_key() :: unicode:chardata().\n\n% invalid characters (from https://en.wikipedia.org/wiki/Unicode_plane):\n% * The High Surrogates (U+D800 - U+DBFF) and Low Surrogate (U+DC00 - U+DFFF)\n%   codes are reserved for encoding non-BMP characters in UTF-16 by using a\n%   pair of 16-bit codes\n% * The non-characters U+FFFE and U+FFFF can not be encoded into UTF8 by\n%   Erlang versions older than R15.\n-type unicode_char() :: 0..16#D7FF | 16#E000..16#FFFD | 16#10000..16#10ffff.\n-type client_key() :: [unicode_char()].\n-type client_value() :: any().\n-type client_version() :: non_neg_integer().\n"
  },
  {
    "path": "include/gen_component.hrl",
    "content": "%  @copyright 2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc change the first function in processes from gen_component:start/5\n%%      to ?MODULE:start_gen_component/5\n%% @end\n\n-export([start_gen_component/5]).\n-spec start_gen_component(module(), gen_component:handler(), term(),\n                          [gen_component:option()], pid()) ->\n                                 no_return() | ok.\nstart_gen_component(Module, Handler, Args, Options, Self) ->\n    gen_component:start(Module, Handler, Args, Options, Self).\n\n"
  },
  {
    "path": "include/lookup.hrl",
    "content": "%  @copyright 2007-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jens Fischer <jensvfischer@gmail.com>\n%% @doc    Header for macros used in lookups\n%% @end\n%% @version $Id$\n\n-author('jensvfischer@gmail.com').\n-vsn('$Id$').\n\n-ifdef(enable_debug).\n% add source information to debug routing damaged messages\n-define(HOPS_TO_DATA(Hops), {comm:this(), Hops}).\n-define(HOPS_FROM_DATA(Data), element(2, Data)).\n-type data() :: {Source::comm:mypid(), Hops::non_neg_integer()}.\n-else.\n-define(HOPS_TO_DATA(Hops), Hops).\n-define(HOPS_FROM_DATA(Data), Data).\n-type data() :: Hops::non_neg_integer().\n-endif.\n\n\n\n"
  },
  {
    "path": "include/record_helpers.hrl",
    "content": "%  @copyright 2010-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Helper macros for record definitions\n%% @end\n%% @version $Id$\n\n% thanks to Vincent de Phily for this macro to specify required fields in\n% record where no hard-coded value makes sense at the time of the type\n% definition:\n-define(required(Record, Field),\n        throw({record_field_required, Record, Field, ?MODULE})).\n"
  },
  {
    "path": "include/rt.hrl.in",
    "content": "-include(\"@RT@.hrl\").\n"
  },
  {
    "path": "include/rt_chord.hrl",
    "content": "%%Standard Chord routingtable\n\n-define(RT, rt_chord).\n% first valid key:\n-define(MINUS_INFINITY, 0).\n-define(MINUS_INFINITY_TYPE, 0).\n% first invalid key:\n-define(PLUS_INFINITY, 16#100000000000000000000000000000000).\n-define(PLUS_INFINITY_TYPE, 16#100000000000000000000000000000000).\n"
  },
  {
    "path": "include/rt_frt.hrl",
    "content": "%% Flexible Routing Tables\n-define(RT, rt_frt).\n% first valid key:\n-define(MINUS_INFINITY, 0).\n-define(MINUS_INFINITY_TYPE, 0).\n% first invalid key:\n-define(PLUS_INFINITY, 16#100000000000000000000000000000000).\n-define(PLUS_INFINITY_TYPE, 16#100000000000000000000000000000000).\n"
  },
  {
    "path": "include/rt_gfrt.hrl",
    "content": "%% Grouped Flexible Routing Table\n-define(GFRT, true).\n-define(RT, rt_frt).\n\n% first valid key:\n-define(MINUS_INFINITY, 0).\n-define(MINUS_INFINITY_TYPE, 0).\n% first invalid key:\n-define(PLUS_INFINITY, 16#100000000000000000000000000000000).\n-define(PLUS_INFINITY_TYPE, 16#100000000000000000000000000000000).\n"
  },
  {
    "path": "include/rt_simple.hrl",
    "content": "%%simple routingtable\n\n-define(RT, rt_simple).\n% first valid key:\n-define(MINUS_INFINITY, 0).\n-define(MINUS_INFINITY_TYPE, 0).\n% first invalid key:\n-define(PLUS_INFINITY, 16#100000000000000000000000000000000).\n-define(PLUS_INFINITY_TYPE, 16#100000000000000000000000000000000).\n"
  },
  {
    "path": "include/scalaris.hrl",
    "content": "%  @copyright 2008-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Globally included file (sets the modules being used in cases where\n%%         multiple choices exist)\n%% @end\n%% @version $Id$\n\n%% Version of Scalaris\n-define(SCALARIS_VERSION, \"0.9.0+git\").\n\n%% The RT macro is defined by the --with-rt configure parameter and determines\n%% which kind of routingtable is used.\n-include(\"rt.hrl\").\n\n% special parameters to be passed to ets:new/2 (not only used on db_ets!)\n% note: keep access level at least at protected for dht_node_db_cache to work!\n% note: named_table is not allowed any more and may result in exceptions due to non-unique table names\n-define(DB_ETS_ADDITIONAL_OPS, [protected]).\n%% userdevguide-end scalaris:db\n\n%% when defined, use new tx protocol (which is not yet complete)\n%% can be enabled with ./configure --enable-txnew\n-ifdef(enable_txnew).\n-define(TXNEW, true).\n-endif.\n\n%% userdevguide-begin scalaris:rm\n%%Standard chord ring maintenance\n%-define(RM, rm_chord).\n\n%% ring maintenance by T-man\n-define(RM, rm_tman).\n\n%% ring maintenance by T-man-Sharp\n%-define(RM, rm_tmansharp).\n%% userdevguide-end scalaris:rm\n\n% enable logging of message statistics at the granularity of socket messages\n-define(LOG_MESSAGE_SOCK(SNDRCV, MESSAGE, SIZE, CHANNEL), ok).\n%% -define(LOG_MESSAGE_SOCK(SNDRCV, BINARYMESSAGE, SIZE, CHANNEL), comm_logger:log(SNDRCV, {CHANNEL, socket_bin}, SIZE)).\n\n% enable logging of message statistics at the granularity of erlang messages\n-define(LOG_MESSAGE(SNDRCV, MESSAGE, CHANNEL), ok).\n%% -define(LOG_MESSAGE(SNDRCV, MESSAGE, CHANNEL), comm_logger:log(SNDRCV, {CHANNEL, comm:get_msg_tag(MESSAGE)}, erlang:byte_size(erlang:term_to_binary(MESSAGE, [{minor_version, 1}])))).\n\n\n% Back-end of the pdb module\n-define(PDB_ERL, true). % erlang put and get (process dictionary)\n%-define(PDB_ETS, true). % may have performance issues with msg_delay\n\n%%%%%%%%%%%%%%%%%%%%%%\n\n-define(implies(A, B), ((not (A)) orelse (B))).\n\n% for debugging:\n-ifdef(have_ctline_support).\n% allows the retrieval of the current function and line number a process is in\n% (process dictionary, key test_server_loc)\n-compile({parse_transform, ct_line}).\n-endif.\n\n-define(ASSERT(X), true = X).\n-define(ASSERT2(X, Y), case X of\n                           true -> ok;\n                           _ -> util:do_throw(Y)\n                       end).\n%% debug mode (enable with ./configure --enable-debug)\n-ifdef(enable_debug).\n-define(DBG_ASSERT(X), ?ASSERT(X)).\n-define(DBG_ASSERT2(X, Y), ?ASSERT2(X, Y)).\n% enable native register for all processes in gen_component\n% useful for debug (etop, appmon), but lets memory usage grow over time\n-define(DEBUG_REGISTER(PROCESS, PID), catch(erlang:register(PROCESS, PID))).\n-else.\n-define(DBG_ASSERT(X), ok).\n-define(DBG_ASSERT2(X, Y), ok).\n-define(DEBUG_REGISTER(PROCESS, PID), ok).\n-endif.\n\n-ifdef(have_new_stacktrace).\n    %% >= erlang 21\n-define(CATCH_CLAUSE_WITH_STACKTRACE(Level, Reason, Stacktrace),\n        Level:Reason:Stacktrace -> ok,).\n-else.\n-define(CATCH_CLAUSE_WITH_STACKTRACE(Level, Reason, Stacktrace),\n        Level:Reason -> Stacktrace = erlang:get_stacktrace(),).\n-endif.\n\n% disable compression (the overhead is too high, at least for GbE)\n-define(COMM_COMPRESS_MSG(DeliverMsg, State),\n        term_to_binary(DeliverMsg, [{minor_version, 1}])\n       ).\n% do not compress big messages (assumes that those messages contain already-compressed values)\n%% -define(COMM_COMPRESS_MSG(DeliverMsg, State),\n%%         DeliverMsgSize = erlang:byte_size(erlang:term_to_binary(DeliverMsg, [{minor_version, 1}])),\n%%         MsgQueueLen = erlang:max(1, msg_queue_len(State)),\n%%         CompressionLvl = if (DeliverMsgSize / MsgQueueLen) > 1024 -> 0;\n%%                             true -> 2\n%%                          end,\n%%         term_to_binary(DeliverMsg, [{compressed, CompressionLvl}, {minor_version, 1}])\n%%        ).\n-define(COMM_DECOMPRESS_MSG(DeliverMsg, State), binary_to_term(DeliverMsg)).\n\n% using snappy-erlang-nif:\n%% -define(COMM_COMPRESS_MSG(DeliverMsg, State),\n%%         {ok, SnappyMsg} = snappy:compress(term_to_binary(DeliverMsg, [{minor_version, 1}])),\n%%         SnappyMsg\n%%        ).\n%% -define(COMM_DECOMPRESS_MSG(DeliverMsg, State),\n%%         {ok, SnappyMsg} = snappy:decompress(DeliverMsg),\n%%         binary_to_term(SnappyMsg)\n%%        ).\n\n-define(SCALARIS_RECV_PRODUCTION(X,Y),\n        {'$gen_component', trace_mpath,\n         _ScalPState, _ScalFrom, _ScalTo, X = _ScalMsg} ->\n               trace_mpath:log_recv(_ScalPState, _ScalFrom, _ScalTo, _ScalMsg),\n               case erlang:get(trace_mpath) of\n                   undefined ->\n                       trace_mpath:start(_ScalPState),\n                       ok;\n                   _ ->\n                       trace_mpath:log_info(_ScalPState, _ScalTo, {scalaris_recv})\n               end,\n               Y;\n            X -> Y\n        ).\n\n-define(SCALARIS_RECV_DEBUG(X,Y),\n        {'$gen_component', trace_mpath,\n         _ScalPState, _ScalFrom, _ScalTo, X = _ScalMsg} ->\n               trace_mpath:log_recv(_ScalPState, _ScalFrom, _ScalTo, _ScalMsg),\n               case erlang:get(trace_mpath) of\n                   undefined ->\n                       trace_mpath:start(_ScalPState),\n                       ok;\n                   _ ->\n                       trace_mpath:log_info(_ScalPState, _ScalTo, {scalaris_recv, \"SCALARIS_RECV at client process (pid, module, line)~n\", _ScalTo, ?MODULE, ?LINE})\n               end,\n               case gen_component:is_gen_component(self())\n                   andalso ({current_function, {pid_groups, add, 3}}\n                            =/=  process_info(self(), current_function)) of\n                   false -> ok;\n                   true ->\n                       log:log(\"?SCALARIS_RECV not allowed inside a gen_component ~p\",\n                               [process_info(self(), current_function)]),\n                       erlang:error(scalaris_recv_in_gen_component)\n               end,\n               Y;\n            X = _ScalMsg->\n               ?ASSERT2(not trace_mpath:infected(), {noninfected_message_in_infected_process, _ScalMsg}),\n               case gen_component:is_gen_component(self())\n                   andalso ({current_function, {pid_groups, add, 3}}\n                            =/=  process_info(self(), current_function)) of\n                   false -> ok;\n                   true ->\n                       log:log(\"?SCALARIS_RECV not allowed inside a gen_component ~p\",\n                               [process_info(self(), current_function)]),\n                       erlang:error(scalaris_recv_in_gen_component)\n               end,\n               Y\n        ).\n\n-ifdef(enable_debug).\n-define(SCALARIS_RECV(X,Y), ?SCALARIS_RECV_DEBUG(X, Y)).\n-else.\n-define(SCALARIS_RECV(X,Y), ?SCALARIS_RECV_PRODUCTION(X, Y)).\n-endif.\n\n-define(IIF(C, A, B), case C of\n                          true -> A;\n                          _ -> B\n                      end).\n\n-ifdef(with_crypto_hash).\n-define(CRYPTO_MD5(V), crypto:hash(md5, V)).\n-define(CRYPTO_SHA(V), crypto:hash(sha, V)).\n-else.\n-define(CRYPTO_MD5(V), crypto:md5(V)).\n-define(CRYPTO_SHA(V), crypto:sha(V)).\n-endif.\n\n-define(TRACE_MPATH_SAFE(Expression),\n        begin\n            %% in case proto_sched or trace_mpath infected this msg\n            trace_mpath:clear_infection(),\n            Expression,\n            %% restore infection if infected\n            trace_mpath:restore_infection()\n        end).\n\n-include(\"types.hrl\").\n-include(\"atom_ext.hrl\").\n"
  },
  {
    "path": "include/types.hrl",
    "content": "%  @copyright 2010-2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Type definitions of non-existing, opaque or custom types which can\n%%         be used independently from the erlang version.\n%% @end\n%% @version $Id$\n\n% add type defs if required...\n\n%% more restrictive (and correct) definition of the timestamp type:\n-type erlang_timestamp() :: {MegaSecs::non_neg_integer(),\n                             Secs::0..999999,\n                             MicroSecs::0..999999}.\n"
  },
  {
    "path": "java-api/.classpath",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<classpath>\n\t<classpathentry kind=\"src\" path=\"src\"/>\n\t<classpathentry kind=\"src\" path=\"test\"/>\n\t<classpathentry kind=\"con\" path=\"org.eclipse.jdt.launching.JRE_CONTAINER\"/>\n\t<classpathentry kind=\"lib\" path=\"lib/jakarta-commons-cli-1.2.jar\" sourcepath=\"lib/jakarta-commons-cli-1.2-sources.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/jetty-util-8.0.4.v20111024.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/annotations-api.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/bootstrap.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/catalina-ant.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/catalina-ha.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/catalina-tribes.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/catalina.jar\" sourcepath=\"/home/nico/downloads/apache-tomcat-7.0.28-src/java\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/commons-daemon.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/el-api.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/jasper-el.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/jasper.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/jsp-api.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/servlet-api.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/tomcat-api.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/tomcat-coyote.jar\" sourcepath=\"/home/nico/downloads/apache-tomcat-7.0.28-src/java\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/tomcat-dbcp.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/tomcat-i18n-es.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/tomcat-i18n-fr.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/tomcat-i18n-ja.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/tomcat-jdbc.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/tomcat-juli.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/tomcat-util.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/commons-codec-1.6.jar\" sourcepath=\"/home/nico/.m2/repository/commons-codec/commons-codec/1.6/commons-codec-1.6-sources.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/ecj-4.2.2.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/junit-4.11.jar\" sourcepath=\"/home/nico/.m2/repository/junit/junit/4.11/junit-4.11-sources.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"test-lib/hamcrest-core-1.3.jar\"/>\n\t<classpathentry kind=\"lib\" path=\"lib/OtpErlang-1.6.1.jar\"/>\n\t<classpathentry kind=\"output\" path=\"bin\"/>\n</classpath>\n"
  },
  {
    "path": "java-api/.gitignore",
    "content": "/doc\n/doc-devel\n/doc-doxygen\n/scalaris\n/scalaris-examples.jar\n/chordsharp4j-examples.jar\n/Scalaris.properties\n/chordsharp4j.jar\n/ChordSharpConnection.properties\n/classes\n/scalaris.jar\n/scalaris-lib.jar\n/chordsharp4j-lib.jar\n/bin\n/chordsharp4j.zip\n/chordsharp4j.tar.bz2\n/scalaris.tar.bz2\n/chordsharp4j.tar.gz\n/scalaris.properties\n/scalaris.zip\n/scalaris.tar.gz\n/.ant-targets-build.xml\n/junit*.properties\n/junitvmwatcher*.properties\n/scalaris-java.conf\n"
  },
  {
    "path": "java-api/.project",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<projectDescription>\n\t<name>Scalaris-JavaAPI</name>\n\t<comment></comment>\n\t<projects>\n\t</projects>\n\t<buildSpec>\n\t\t<buildCommand>\n\t\t\t<name>org.eclipse.jdt.core.javabuilder</name>\n\t\t\t<arguments>\n\t\t\t</arguments>\n\t\t</buildCommand>\n\t</buildSpec>\n\t<natures>\n\t\t<nature>org.eclipse.jdt.core.javanature</nature>\n\t</natures>\n\t<filteredResources>\n\t\t<filter>\n\t\t\t<id>1398446030089</id>\n\t\t\t<name></name>\n\t\t\t<type>10</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-doc</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1398446030096</id>\n\t\t\t<name></name>\n\t\t\t<type>10</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-classes</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t\t<filter>\n\t\t\t<id>1398446030107</id>\n\t\t\t<name></name>\n\t\t\t<type>10</type>\n\t\t\t<matcher>\n\t\t\t\t<id>org.eclipse.ui.ide.multiFilter</id>\n\t\t\t\t<arguments>1.0-name-matches-true-false-target</arguments>\n\t\t\t</matcher>\n\t\t</filter>\n\t</filteredResources>\n</projectDescription>\n"
  },
  {
    "path": "java-api/.settings/org.eclipse.jdt.core.prefs",
    "content": "eclipse.preferences.version=1\norg.eclipse.jdt.core.compiler.codegen.inlineJsrBytecode=enabled\norg.eclipse.jdt.core.compiler.codegen.methodParameters=do not generate\norg.eclipse.jdt.core.compiler.codegen.targetPlatform=1.6\norg.eclipse.jdt.core.compiler.codegen.unusedLocal=preserve\norg.eclipse.jdt.core.compiler.compliance=1.6\norg.eclipse.jdt.core.compiler.debug.lineNumber=generate\norg.eclipse.jdt.core.compiler.debug.localVariable=generate\norg.eclipse.jdt.core.compiler.debug.sourceFile=generate\norg.eclipse.jdt.core.compiler.problem.assertIdentifier=error\norg.eclipse.jdt.core.compiler.problem.enumIdentifier=error\norg.eclipse.jdt.core.compiler.source=1.6\n"
  },
  {
    "path": "java-api/.settings/org.eclipse.jdt.ui.prefs",
    "content": "#Wed Apr 20 10:46:14 CEST 2011\neclipse.preferences.version=1\neditor_save_participant_org.eclipse.jdt.ui.postsavelistener.cleanup=true\nsp_cleanup.add_default_serial_version_id=true\nsp_cleanup.add_generated_serial_version_id=false\nsp_cleanup.add_missing_annotations=true\nsp_cleanup.add_missing_deprecated_annotations=true\nsp_cleanup.add_missing_methods=false\nsp_cleanup.add_missing_nls_tags=false\nsp_cleanup.add_missing_override_annotations=true\nsp_cleanup.add_missing_override_annotations_interface_methods=true\nsp_cleanup.add_serial_version_id=false\nsp_cleanup.always_use_blocks=true\nsp_cleanup.always_use_parentheses_in_expressions=true\nsp_cleanup.always_use_this_for_non_static_field_access=false\nsp_cleanup.always_use_this_for_non_static_method_access=false\nsp_cleanup.convert_to_enhanced_for_loop=true\nsp_cleanup.correct_indentation=false\nsp_cleanup.format_source_code=false\nsp_cleanup.format_source_code_changes_only=false\nsp_cleanup.make_local_variable_final=true\nsp_cleanup.make_parameters_final=true\nsp_cleanup.make_private_fields_final=true\nsp_cleanup.make_type_abstract_if_missing_method=false\nsp_cleanup.make_variable_declarations_final=true\nsp_cleanup.never_use_blocks=false\nsp_cleanup.never_use_parentheses_in_expressions=false\nsp_cleanup.on_save_use_additional_actions=true\nsp_cleanup.organize_imports=false\nsp_cleanup.qualify_static_field_accesses_with_declaring_class=false\nsp_cleanup.qualify_static_member_accesses_through_instances_with_declaring_class=true\nsp_cleanup.qualify_static_member_accesses_through_subtypes_with_declaring_class=true\nsp_cleanup.qualify_static_member_accesses_with_declaring_class=false\nsp_cleanup.qualify_static_method_accesses_with_declaring_class=false\nsp_cleanup.remove_private_constructors=true\nsp_cleanup.remove_trailing_whitespaces=true\nsp_cleanup.remove_trailing_whitespaces_all=true\nsp_cleanup.remove_trailing_whitespaces_ignore_empty=false\nsp_cleanup.remove_unnecessary_casts=true\nsp_cleanup.remove_unnecessary_nls_tags=false\nsp_cleanup.remove_unused_imports=true\nsp_cleanup.remove_unused_local_variables=false\nsp_cleanup.remove_unused_private_fields=true\nsp_cleanup.remove_unused_private_members=false\nsp_cleanup.remove_unused_private_methods=true\nsp_cleanup.remove_unused_private_types=true\nsp_cleanup.sort_members=false\nsp_cleanup.sort_members_all=false\nsp_cleanup.use_blocks=true\nsp_cleanup.use_blocks_only_for_return_and_throw=false\nsp_cleanup.use_parentheses_in_expressions=true\nsp_cleanup.use_this_for_non_static_field_access=false\nsp_cleanup.use_this_for_non_static_field_access_only_if_necessary=true\nsp_cleanup.use_this_for_non_static_method_access=false\nsp_cleanup.use_this_for_non_static_method_access_only_if_necessary=true\n"
  },
  {
    "path": "java-api/Doxyfile",
    "content": "# Doxyfile 1.5.7.1\n\n# This file describes the settings to be used by the documentation system\n# doxygen (www.doxygen.org) for a project\n#\n# All text after a hash (#) is considered a comment and will be ignored\n# The format is:\n#       TAG = value [value, ...]\n# For lists items can also be appended using:\n#       TAG += value [value, ...]\n# Values that contain spaces should be placed between quotes (\" \")\n\n#---------------------------------------------------------------------------\n# Project related configuration options\n#---------------------------------------------------------------------------\n\n# This tag specifies the encoding used for all characters in the config file \n# that follow. The default is UTF-8 which is also the encoding used for all \n# text before the first occurrence of this tag. Doxygen uses libiconv (or the \n# iconv built into libc) for the transcoding. See \n# http://www.gnu.org/software/libiconv for the list of possible encodings.\n\nDOXYFILE_ENCODING      = UTF-8\n\n# The PROJECT_NAME tag is a single word (or a sequence of words surrounded \n# by quotes) that should identify the project.\n\nPROJECT_NAME           = \"Scalaris Java Interface\"\n\n# The PROJECT_NUMBER tag can be used to enter a project or revision number. \n# This could be handy for archiving the generated documentation or \n# if some version control system is used.\n\nPROJECT_NUMBER         = 0.0.1\n\n# The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute) \n# base path where the generated documentation will be put. \n# If a relative path is entered, it will be relative to the location \n# where doxygen was started. If left blank the current directory will be used.\n\nOUTPUT_DIRECTORY       = doc-doxygen\n\n# If the CREATE_SUBDIRS tag is set to YES, then doxygen will create \n# 4096 sub-directories (in 2 levels) under the output directory of each output \n# format and will distribute the generated files over these directories. \n# Enabling this option can be useful when feeding doxygen a huge amount of \n# source files, where putting all generated files in the same directory would \n# otherwise cause performance problems for the file system.\n\nCREATE_SUBDIRS         = NO\n\n# The OUTPUT_LANGUAGE tag is used to specify the language in which all \n# documentation generated by doxygen is written. Doxygen will use this \n# information to generate all constant output in the proper language. \n# The default language is English, other supported languages are: \n# Afrikaans, Arabic, Brazilian, Catalan, Chinese, Chinese-Traditional, \n# Croatian, Czech, Danish, Dutch, Farsi, Finnish, French, German, Greek, \n# Hungarian, Italian, Japanese, Japanese-en (Japanese with English messages), \n# Korean, Korean-en, Lithuanian, Norwegian, Macedonian, Persian, Polish, \n# Portuguese, Romanian, Russian, Serbian, Serbian-Cyrilic, Slovak, Slovene, \n# Spanish, Swedish, and Ukrainian.\n\nOUTPUT_LANGUAGE        = English\n\n# If the BRIEF_MEMBER_DESC tag is set to YES (the default) Doxygen will \n# include brief member descriptions after the members that are listed in \n# the file and class documentation (similar to JavaDoc). \n# Set to NO to disable this.\n\nBRIEF_MEMBER_DESC      = NO\n\n# If the REPEAT_BRIEF tag is set to YES (the default) Doxygen will prepend \n# the brief description of a member or function before the detailed description. \n# Note: if both HIDE_UNDOC_MEMBERS and BRIEF_MEMBER_DESC are set to NO, the \n# brief descriptions will be completely suppressed.\n\nREPEAT_BRIEF           = YES\n\n# This tag implements a quasi-intelligent brief description abbreviator \n# that is used to form the text in various listings. Each string \n# in this list, if found as the leading text of the brief description, will be \n# stripped from the text and the result after processing the whole list, is \n# used as the annotated text. Otherwise, the brief description is used as-is. \n# If left blank, the following values are used (\"$name\" is automatically \n# replaced with the name of the entity): \"The $name class\" \"The $name widget\" \n# \"The $name file\" \"is\" \"provides\" \"specifies\" \"contains\" \n# \"represents\" \"a\" \"an\" \"the\"\n\nABBREVIATE_BRIEF       = \n\n# If the ALWAYS_DETAILED_SEC and REPEAT_BRIEF tags are both set to YES then \n# Doxygen will generate a detailed section even if there is only a brief \n# description.\n\nALWAYS_DETAILED_SEC    = NO\n\n# If the INLINE_INHERITED_MEMB tag is set to YES, doxygen will show all \n# inherited members of a class in the documentation of that class as if those \n# members were ordinary class members. Constructors, destructors and assignment \n# operators of the base classes will not be shown.\n\nINLINE_INHERITED_MEMB  = NO\n\n# If the FULL_PATH_NAMES tag is set to YES then Doxygen will prepend the full \n# path before files name in the file list and in the header files. If set \n# to NO the shortest path that makes the file name unique will be used.\n\nFULL_PATH_NAMES        = YES\n\n# If the FULL_PATH_NAMES tag is set to YES then the STRIP_FROM_PATH tag \n# can be used to strip a user-defined part of the path. Stripping is \n# only done if one of the specified strings matches the left-hand part of \n# the path. The tag can be used to show relative paths in the file list. \n# If left blank the directory from which doxygen is run is used as the \n# path to strip.\n\nSTRIP_FROM_PATH        = \n\n# The STRIP_FROM_INC_PATH tag can be used to strip a user-defined part of \n# the path mentioned in the documentation of a class, which tells \n# the reader which header file to include in order to use a class. \n# If left blank only the name of the header file containing the class \n# definition is used. Otherwise one should specify the include paths that \n# are normally passed to the compiler using the -I flag.\n\nSTRIP_FROM_INC_PATH    = \n\n# If the SHORT_NAMES tag is set to YES, doxygen will generate much shorter \n# (but less readable) file names. This can be useful is your file systems \n# doesn't support long names like on DOS, Mac, or CD-ROM.\n\nSHORT_NAMES            = NO\n\n# If the JAVADOC_AUTOBRIEF tag is set to YES then Doxygen \n# will interpret the first line (until the first dot) of a JavaDoc-style \n# comment as the brief description. If set to NO, the JavaDoc \n# comments will behave just like regular Qt-style comments \n# (thus requiring an explicit @brief command for a brief description.)\n\nJAVADOC_AUTOBRIEF      = NO\n\n# If the QT_AUTOBRIEF tag is set to YES then Doxygen will \n# interpret the first line (until the first dot) of a Qt-style \n# comment as the brief description. If set to NO, the comments \n# will behave just like regular Qt-style comments (thus requiring \n# an explicit \\brief command for a brief description.)\n\nQT_AUTOBRIEF           = NO\n\n# The MULTILINE_CPP_IS_BRIEF tag can be set to YES to make Doxygen \n# treat a multi-line C++ special comment block (i.e. a block of //! or /// \n# comments) as a brief description. This used to be the default behaviour. \n# The new default is to treat a multi-line C++ comment block as a detailed \n# description. Set this tag to YES if you prefer the old behaviour instead.\n\nMULTILINE_CPP_IS_BRIEF = NO\n\n# If the INHERIT_DOCS tag is set to YES (the default) then an undocumented \n# member inherits the documentation from any documented member that it \n# re-implements.\n\nINHERIT_DOCS           = YES\n\n# If the SEPARATE_MEMBER_PAGES tag is set to YES, then doxygen will produce \n# a new page for each member. If set to NO, the documentation of a member will \n# be part of the file/class/namespace that contains it.\n\nSEPARATE_MEMBER_PAGES  = NO\n\n# The TAB_SIZE tag can be used to set the number of spaces in a tab. \n# Doxygen uses this value to replace tabs by spaces in code fragments.\n\nTAB_SIZE               = 8\n\n# This tag can be used to specify a number of aliases that acts \n# as commands in the documentation. An alias has the form \"name=value\". \n# For example adding \"sideeffect=\\par Side Effects:\\n\" will allow you to \n# put the command \\sideeffect (or @sideeffect) in the documentation, which \n# will result in a user-defined paragraph with heading \"Side Effects:\". \n# You can put \\n's in the value part of an alias to insert newlines.\n\nALIASES                = \n\n# Set the OPTIMIZE_OUTPUT_FOR_C tag to YES if your project consists of C \n# sources only. Doxygen will then generate output that is more tailored for C. \n# For instance, some of the names that are used will be different. The list \n# of all members will be omitted, etc.\n\nOPTIMIZE_OUTPUT_FOR_C  = NO\n\n# Set the OPTIMIZE_OUTPUT_JAVA tag to YES if your project consists of Java \n# sources only. Doxygen will then generate output that is more tailored for \n# Java. For instance, namespaces will be presented as packages, qualified \n# scopes will look different, etc.\n\nOPTIMIZE_OUTPUT_JAVA   = YES\n\n# Set the OPTIMIZE_FOR_FORTRAN tag to YES if your project consists of Fortran \n# sources only. Doxygen will then generate output that is more tailored for \n# Fortran.\n\nOPTIMIZE_FOR_FORTRAN   = NO\n\n# Set the OPTIMIZE_OUTPUT_VHDL tag to YES if your project consists of VHDL \n# sources. Doxygen will then generate output that is tailored for \n# VHDL.\n\nOPTIMIZE_OUTPUT_VHDL   = NO\n\n# If you use STL classes (i.e. std::string, std::vector, etc.) but do not want \n# to include (a tag file for) the STL sources as input, then you should \n# set this tag to YES in order to let doxygen match functions declarations and \n# definitions whose arguments contain STL classes (e.g. func(std::string); v.s. \n# func(std::string) {}). This also make the inheritance and collaboration \n# diagrams that involve STL classes more complete and accurate.\n\nBUILTIN_STL_SUPPORT    = NO\n\n# If you use Microsoft's C++/CLI language, you should set this option to YES to\n# enable parsing support.\n\nCPP_CLI_SUPPORT        = NO\n\n# Set the SIP_SUPPORT tag to YES if your project consists of sip sources only. \n# Doxygen will parse them like normal C++ but will assume all classes use public \n# instead of private inheritance when no explicit protection keyword is present.\n\nSIP_SUPPORT            = NO\n\n# For Microsoft's IDL there are propget and propput attributes to indicate getter \n# and setter methods for a property. Setting this option to YES (the default) \n# will make doxygen to replace the get and set methods by a property in the \n# documentation. This will only work if the methods are indeed getting or \n# setting a simple type. If this is not the case, or you want to show the \n# methods anyway, you should set this option to NO.\n\nIDL_PROPERTY_SUPPORT   = YES\n\n# If member grouping is used in the documentation and the DISTRIBUTE_GROUP_DOC \n# tag is set to YES, then doxygen will reuse the documentation of the first \n# member in the group (if any) for the other members of the group. By default \n# all members of a group must be documented explicitly.\n\nDISTRIBUTE_GROUP_DOC   = NO\n\n# Set the SUBGROUPING tag to YES (the default) to allow class member groups of \n# the same type (for instance a group of public functions) to be put as a \n# subgroup of that type (e.g. under the Public Functions section). Set it to \n# NO to prevent subgrouping. Alternatively, this can be done per class using \n# the \\nosubgrouping command.\n\nSUBGROUPING            = YES\n\n# When TYPEDEF_HIDES_STRUCT is enabled, a typedef of a struct, union, or enum \n# is documented as struct, union, or enum with the name of the typedef. So \n# typedef struct TypeS {} TypeT, will appear in the documentation as a struct \n# with name TypeT. When disabled the typedef will appear as a member of a file, \n# namespace, or class. And the struct will be named TypeS. This can typically \n# be useful for C code in case the coding convention dictates that all compound \n# types are typedef'ed and only the typedef is referenced, never the tag name.\n\nTYPEDEF_HIDES_STRUCT   = NO\n\n# The SYMBOL_CACHE_SIZE determines the size of the internal cache use to \n# determine which symbols to keep in memory and which to flush to disk.\n# When the cache is full, less often used symbols will be written to disk.\n# For small to medium size projects (<1000 input files) the default value is \n# probably good enough. For larger projects a too small cache size can cause \n# doxygen to be busy swapping symbols to and from disk most of the time \n# causing a significant performance penality. \n# If the system has enough physical memory increasing the cache will improve the \n# performance by keeping more symbols in memory. Note that the value works on \n# a logarithmic scale so increasing the size by one will rougly double the \n# memory usage. The cache size is given by this formula: \n# 2^(16+SYMBOL_CACHE_SIZE). The valid range is 0..9, the default is 0, \n# corresponding to a cache size of 2^16 = 65536 symbols\n\nSYMBOL_CACHE_SIZE      = 0\n\n#---------------------------------------------------------------------------\n# Build related configuration options\n#---------------------------------------------------------------------------\n\n# If the EXTRACT_ALL tag is set to YES doxygen will assume all entities in \n# documentation are documented, even if no documentation was available. \n# Private class members and static file members will be hidden unless \n# the EXTRACT_PRIVATE and EXTRACT_STATIC tags are set to YES\n\nEXTRACT_ALL            = NO\n\n# If the EXTRACT_PRIVATE tag is set to YES all private members of a class \n# will be included in the documentation.\n\nEXTRACT_PRIVATE        = NO\n\n# If the EXTRACT_STATIC tag is set to YES all static members of a file \n# will be included in the documentation.\n\nEXTRACT_STATIC         = YES\n\n# If the EXTRACT_LOCAL_CLASSES tag is set to YES classes (and structs) \n# defined locally in source files will be included in the documentation. \n# If set to NO only classes defined in header files are included.\n\nEXTRACT_LOCAL_CLASSES  = YES\n\n# This flag is only useful for Objective-C code. When set to YES local \n# methods, which are defined in the implementation section but not in \n# the interface are included in the documentation. \n# If set to NO (the default) only methods in the interface are included.\n\nEXTRACT_LOCAL_METHODS  = NO\n\n# If this flag is set to YES, the members of anonymous namespaces will be \n# extracted and appear in the documentation as a namespace called \n# 'anonymous_namespace{file}', where file will be replaced with the base \n# name of the file that contains the anonymous namespace. By default \n# anonymous namespace are hidden.\n\nEXTRACT_ANON_NSPACES   = NO\n\n# If the HIDE_UNDOC_MEMBERS tag is set to YES, Doxygen will hide all \n# undocumented members of documented classes, files or namespaces. \n# If set to NO (the default) these members will be included in the \n# various overviews, but no documentation section is generated. \n# This option has no effect if EXTRACT_ALL is enabled.\n\nHIDE_UNDOC_MEMBERS     = NO\n\n# If the HIDE_UNDOC_CLASSES tag is set to YES, Doxygen will hide all \n# undocumented classes that are normally visible in the class hierarchy. \n# If set to NO (the default) these classes will be included in the various \n# overviews. This option has no effect if EXTRACT_ALL is enabled.\n\nHIDE_UNDOC_CLASSES     = NO\n\n# If the HIDE_FRIEND_COMPOUNDS tag is set to YES, Doxygen will hide all \n# friend (class|struct|union) declarations. \n# If set to NO (the default) these declarations will be included in the \n# documentation.\n\nHIDE_FRIEND_COMPOUNDS  = NO\n\n# If the HIDE_IN_BODY_DOCS tag is set to YES, Doxygen will hide any \n# documentation blocks found inside the body of a function. \n# If set to NO (the default) these blocks will be appended to the \n# function's detailed documentation block.\n\nHIDE_IN_BODY_DOCS      = NO\n\n# The INTERNAL_DOCS tag determines if documentation \n# that is typed after a \\internal command is included. If the tag is set \n# to NO (the default) then the documentation will be excluded. \n# Set it to YES to include the internal documentation.\n\nINTERNAL_DOCS          = NO\n\n# If the CASE_SENSE_NAMES tag is set to NO then Doxygen will only generate \n# file names in lower-case letters. If set to YES upper-case letters are also \n# allowed. This is useful if you have classes or files whose names only differ \n# in case and if your file system supports case sensitive file names. Windows \n# and Mac users are advised to set this option to NO.\n\nCASE_SENSE_NAMES       = YES\n\n# If the HIDE_SCOPE_NAMES tag is set to NO (the default) then Doxygen \n# will show members with their full class and namespace scopes in the \n# documentation. If set to YES the scope will be hidden.\n\nHIDE_SCOPE_NAMES       = NO\n\n# If the SHOW_INCLUDE_FILES tag is set to YES (the default) then Doxygen \n# will put a list of the files that are included by a file in the documentation \n# of that file.\n\nSHOW_INCLUDE_FILES     = YES\n\n# If the INLINE_INFO tag is set to YES (the default) then a tag [inline] \n# is inserted in the documentation for inline members.\n\nINLINE_INFO            = YES\n\n# If the SORT_MEMBER_DOCS tag is set to YES (the default) then doxygen \n# will sort the (detailed) documentation of file and class members \n# alphabetically by member name. If set to NO the members will appear in \n# declaration order.\n\nSORT_MEMBER_DOCS       = YES\n\n# If the SORT_BRIEF_DOCS tag is set to YES then doxygen will sort the \n# brief documentation of file, namespace and class members alphabetically \n# by member name. If set to NO (the default) the members will appear in \n# declaration order.\n\nSORT_BRIEF_DOCS        = NO\n\n# If the SORT_GROUP_NAMES tag is set to YES then doxygen will sort the \n# hierarchy of group names into alphabetical order. If set to NO (the default) \n# the group names will appear in their defined order.\n\nSORT_GROUP_NAMES       = NO\n\n# If the SORT_BY_SCOPE_NAME tag is set to YES, the class list will be \n# sorted by fully-qualified names, including namespaces. If set to \n# NO (the default), the class list will be sorted only by class name, \n# not including the namespace part. \n# Note: This option is not very useful if HIDE_SCOPE_NAMES is set to YES.\n# Note: This option applies only to the class list, not to the \n# alphabetical list.\n\nSORT_BY_SCOPE_NAME     = NO\n\n# The GENERATE_TODOLIST tag can be used to enable (YES) or \n# disable (NO) the todo list. This list is created by putting \\todo \n# commands in the documentation.\n\nGENERATE_TODOLIST      = YES\n\n# The GENERATE_TESTLIST tag can be used to enable (YES) or \n# disable (NO) the test list. This list is created by putting \\test \n# commands in the documentation.\n\nGENERATE_TESTLIST      = YES\n\n# The GENERATE_BUGLIST tag can be used to enable (YES) or \n# disable (NO) the bug list. This list is created by putting \\bug \n# commands in the documentation.\n\nGENERATE_BUGLIST       = YES\n\n# The GENERATE_DEPRECATEDLIST tag can be used to enable (YES) or \n# disable (NO) the deprecated list. This list is created by putting \n# \\deprecated commands in the documentation.\n\nGENERATE_DEPRECATEDLIST= YES\n\n# The ENABLED_SECTIONS tag can be used to enable conditional \n# documentation sections, marked by \\if sectionname ... \\endif.\n\nENABLED_SECTIONS       = \n\n# The MAX_INITIALIZER_LINES tag determines the maximum number of lines \n# the initial value of a variable or define consists of for it to appear in \n# the documentation. If the initializer consists of more lines than specified \n# here it will be hidden. Use a value of 0 to hide initializers completely. \n# The appearance of the initializer of individual variables and defines in the \n# documentation can be controlled using \\showinitializer or \\hideinitializer \n# command in the documentation regardless of this setting.\n\nMAX_INITIALIZER_LINES  = 30\n\n# Set the SHOW_USED_FILES tag to NO to disable the list of files generated \n# at the bottom of the documentation of classes and structs. If set to YES the \n# list will mention the files that were used to generate the documentation.\n\nSHOW_USED_FILES        = YES\n\n# If the sources in your project are distributed over multiple directories \n# then setting the SHOW_DIRECTORIES tag to YES will show the directory hierarchy \n# in the documentation. The default is NO.\n\nSHOW_DIRECTORIES       = NO\n\n# Set the SHOW_FILES tag to NO to disable the generation of the Files page.\n# This will remove the Files entry from the Quick Index and from the \n# Folder Tree View (if specified). The default is YES.\n\nSHOW_FILES             = YES\n\n# Set the SHOW_NAMESPACES tag to NO to disable the generation of the \n# Namespaces page.  This will remove the Namespaces entry from the Quick Index\n# and from the Folder Tree View (if specified). The default is YES.\n\nSHOW_NAMESPACES        = YES\n\n# The FILE_VERSION_FILTER tag can be used to specify a program or script that \n# doxygen should invoke to get the current version for each file (typically from \n# the version control system). Doxygen will invoke the program by executing (via \n# popen()) the command <command> <input-file>, where <command> is the value of \n# the FILE_VERSION_FILTER tag, and <input-file> is the name of an input file \n# provided by doxygen. Whatever the program writes to standard output \n# is used as the file version. See the manual for examples.\n\nFILE_VERSION_FILTER    = \n\n# The LAYOUT_FILE tag can be used to specify a layout file which will be parsed by \n# doxygen. The layout file controls the global structure of the generated output files \n# in an output format independent way. The create the layout file that represents \n# doxygen's defaults, run doxygen with the -l option. You can optionally specify a \n# file name after the option, if omitted DoxygenLayout.xml will be used as the name \n# of the layout file.\n\nLAYOUT_FILE            = \n\n#---------------------------------------------------------------------------\n# configuration options related to warning and progress messages\n#---------------------------------------------------------------------------\n\n# The QUIET tag can be used to turn on/off the messages that are generated \n# by doxygen. Possible values are YES and NO. If left blank NO is used.\n\nQUIET                  = NO\n\n# The WARNINGS tag can be used to turn on/off the warning messages that are \n# generated by doxygen. Possible values are YES and NO. If left blank \n# NO is used.\n\nWARNINGS               = YES\n\n# If WARN_IF_UNDOCUMENTED is set to YES, then doxygen will generate warnings \n# for undocumented members. If EXTRACT_ALL is set to YES then this flag will \n# automatically be disabled.\n\nWARN_IF_UNDOCUMENTED   = YES\n\n# If WARN_IF_DOC_ERROR is set to YES, doxygen will generate warnings for \n# potential errors in the documentation, such as not documenting some \n# parameters in a documented function, or documenting parameters that \n# don't exist or using markup commands wrongly.\n\nWARN_IF_DOC_ERROR      = YES\n\n# This WARN_NO_PARAMDOC option can be abled to get warnings for \n# functions that are documented, but have no documentation for their parameters \n# or return value. If set to NO (the default) doxygen will only warn about \n# wrong or incomplete parameter documentation, but not about the absence of \n# documentation.\n\nWARN_NO_PARAMDOC       = NO\n\n# The WARN_FORMAT tag determines the format of the warning messages that \n# doxygen can produce. The string should contain the $file, $line, and $text \n# tags, which will be replaced by the file and line number from which the \n# warning originated and the warning text. Optionally the format may contain \n# $version, which will be replaced by the version of the file (if it could \n# be obtained via FILE_VERSION_FILTER)\n\nWARN_FORMAT            = \"$file:$line: $text\"\n\n# The WARN_LOGFILE tag can be used to specify a file to which warning \n# and error messages should be written. If left blank the output is written \n# to stderr.\n\nWARN_LOGFILE           = \n\n#---------------------------------------------------------------------------\n# configuration options related to the input files\n#---------------------------------------------------------------------------\n\n# The INPUT tag can be used to specify the files and/or directories that contain \n# documented source files. You may enter file names like \"myfile.cpp\" or \n# directories like \"/usr/src/myproject\". Separate the files or directories \n# with spaces.\n\nINPUT                  = src/de/zib/scalaris/Scalaris.java \\\n\t\t       src/de/zib/scalaris/Transaction.java \\\n\t\t       src/de/zib/scalaris/ConnectionFactory.java \\\n\t\t       src/de/zib/scalaris/DeleteResult.java \\\n\t\t       src/de/zib/scalaris/Main.java \\\n\t\t       src/de/zib/scalaris/package-info.java\n\n# This tag can be used to specify the character encoding of the source files \n# that doxygen parses. Internally doxygen uses the UTF-8 encoding, which is \n# also the default input encoding. Doxygen uses libiconv (or the iconv built \n# into libc) for the transcoding. See http://www.gnu.org/software/libiconv for \n# the list of possible encodings.\n\nINPUT_ENCODING         = UTF-8\n\n# If the value of the INPUT tag contains directories, you can use the \n# FILE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp \n# and *.h) to filter out the source-files in the directories. If left \n# blank the following patterns are tested: \n# *.c *.cc *.cxx *.cpp *.c++ *.java *.ii *.ixx *.ipp *.i++ *.inl *.h *.hh *.hxx \n# *.hpp *.h++ *.idl *.odl *.cs *.php *.php3 *.inc *.m *.mm *.py *.f90\n\nFILE_PATTERNS          = \n\n# The RECURSIVE tag can be used to turn specify whether or not subdirectories \n# should be searched for input files as well. Possible values are YES and NO. \n# If left blank NO is used.\n\nRECURSIVE              = NO\n\n# The EXCLUDE tag can be used to specify files and/or directories that should \n# excluded from the INPUT source files. This way you can easily exclude a \n# subdirectory from a directory tree whose root is specified with the INPUT tag.\n\nEXCLUDE                = \n\n# The EXCLUDE_SYMLINKS tag can be used select whether or not files or \n# directories that are symbolic links (a Unix filesystem feature) are excluded \n# from the input.\n\nEXCLUDE_SYMLINKS       = NO\n\n# If the value of the INPUT tag contains directories, you can use the \n# EXCLUDE_PATTERNS tag to specify one or more wildcard patterns to exclude \n# certain files from those directories. Note that the wildcards are matched \n# against the file with absolute path, so to exclude all test directories \n# for example use the pattern */test/*\n\nEXCLUDE_PATTERNS       = \n\n# The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names \n# (namespaces, classes, functions, etc.) that should be excluded from the \n# output. The symbol name can be a fully qualified name, a word, or if the \n# wildcard * is used, a substring. Examples: ANamespace, AClass, \n# AClass::ANamespace, ANamespace::*Test\n\nEXCLUDE_SYMBOLS        = \n\n# The EXAMPLE_PATH tag can be used to specify one or more files or \n# directories that contain example code fragments that are included (see \n# the \\include command).\n\nEXAMPLE_PATH           = \n\n# If the value of the EXAMPLE_PATH tag contains directories, you can use the \n# EXAMPLE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp \n# and *.h) to filter out the source-files in the directories. If left \n# blank all files are included.\n\nEXAMPLE_PATTERNS       = \n\n# If the EXAMPLE_RECURSIVE tag is set to YES then subdirectories will be \n# searched for input files to be used with the \\include or \\dontinclude \n# commands irrespective of the value of the RECURSIVE tag. \n# Possible values are YES and NO. If left blank NO is used.\n\nEXAMPLE_RECURSIVE      = NO\n\n# The IMAGE_PATH tag can be used to specify one or more files or \n# directories that contain image that are included in the documentation (see \n# the \\image command).\n\nIMAGE_PATH             = \n\n# The INPUT_FILTER tag can be used to specify a program that doxygen should \n# invoke to filter for each input file. Doxygen will invoke the filter program \n# by executing (via popen()) the command <filter> <input-file>, where <filter> \n# is the value of the INPUT_FILTER tag, and <input-file> is the name of an \n# input file. Doxygen will then use the output that the filter program writes \n# to standard output.  If FILTER_PATTERNS is specified, this tag will be \n# ignored.\n\nINPUT_FILTER           = \n\n# The FILTER_PATTERNS tag can be used to specify filters on a per file pattern \n# basis.  Doxygen will compare the file name with each pattern and apply the \n# filter if there is a match.  The filters are a list of the form: \n# pattern=filter (like *.cpp=my_cpp_filter). See INPUT_FILTER for further \n# info on how filters are used. If FILTER_PATTERNS is empty, INPUT_FILTER \n# is applied to all files.\n\nFILTER_PATTERNS        = \n\n# If the FILTER_SOURCE_FILES tag is set to YES, the input filter (if set using \n# INPUT_FILTER) will be used to filter the input files when producing source \n# files to browse (i.e. when SOURCE_BROWSER is set to YES).\n\nFILTER_SOURCE_FILES    = NO\n\n#---------------------------------------------------------------------------\n# configuration options related to source browsing\n#---------------------------------------------------------------------------\n\n# If the SOURCE_BROWSER tag is set to YES then a list of source files will \n# be generated. Documented entities will be cross-referenced with these sources. \n# Note: To get rid of all source code in the generated output, make sure also \n# VERBATIM_HEADERS is set to NO.\n\nSOURCE_BROWSER         = NO\n\n# Setting the INLINE_SOURCES tag to YES will include the body \n# of functions and classes directly in the documentation.\n\nINLINE_SOURCES         = NO\n\n# Setting the STRIP_CODE_COMMENTS tag to YES (the default) will instruct \n# doxygen to hide any special comment blocks from generated source code \n# fragments. Normal C and C++ comments will always remain visible.\n\nSTRIP_CODE_COMMENTS    = YES\n\n# If the REFERENCED_BY_RELATION tag is set to YES \n# then for each documented function all documented \n# functions referencing it will be listed.\n\nREFERENCED_BY_RELATION = NO\n\n# If the REFERENCES_RELATION tag is set to YES \n# then for each documented function all documented entities \n# called/used by that function will be listed.\n\nREFERENCES_RELATION    = NO\n\n# If the REFERENCES_LINK_SOURCE tag is set to YES (the default)\n# and SOURCE_BROWSER tag is set to YES, then the hyperlinks from\n# functions in REFERENCES_RELATION and REFERENCED_BY_RELATION lists will\n# link to the source code.  Otherwise they will link to the documentstion.\n\nREFERENCES_LINK_SOURCE = YES\n\n# If the USE_HTAGS tag is set to YES then the references to source code \n# will point to the HTML generated by the htags(1) tool instead of doxygen \n# built-in source browser. The htags tool is part of GNU's global source \n# tagging system (see http://www.gnu.org/software/global/global.html). You \n# will need version 4.8.6 or higher.\n\nUSE_HTAGS              = NO\n\n# If the VERBATIM_HEADERS tag is set to YES (the default) then Doxygen \n# will generate a verbatim copy of the header file for each class for \n# which an include is specified. Set to NO to disable this.\n\nVERBATIM_HEADERS       = YES\n\n#---------------------------------------------------------------------------\n# configuration options related to the alphabetical class index\n#---------------------------------------------------------------------------\n\n# If the ALPHABETICAL_INDEX tag is set to YES, an alphabetical index \n# of all compounds will be generated. Enable this if the project \n# contains a lot of classes, structs, unions or interfaces.\n\nALPHABETICAL_INDEX     = NO\n\n# If the alphabetical index is enabled (see ALPHABETICAL_INDEX) then \n# the COLS_IN_ALPHA_INDEX tag can be used to specify the number of columns \n# in which this list will be split (can be a number in the range [1..20])\n\nCOLS_IN_ALPHA_INDEX    = 5\n\n# In case all classes in a project start with a common prefix, all \n# classes will be put under the same header in the alphabetical index. \n# The IGNORE_PREFIX tag can be used to specify one or more prefixes that \n# should be ignored while generating the index headers.\n\nIGNORE_PREFIX          = \n\n#---------------------------------------------------------------------------\n# configuration options related to the HTML output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_HTML tag is set to YES (the default) Doxygen will \n# generate HTML output.\n\nGENERATE_HTML          = YES\n\n# The HTML_OUTPUT tag is used to specify where the HTML docs will be put. \n# If a relative path is entered the value of OUTPUT_DIRECTORY will be \n# put in front of it. If left blank `html' will be used as the default path.\n\nHTML_OUTPUT            = html\n\n# The HTML_FILE_EXTENSION tag can be used to specify the file extension for \n# each generated HTML page (for example: .htm,.php,.asp). If it is left blank \n# doxygen will generate files with .html extension.\n\nHTML_FILE_EXTENSION    = .html\n\n# The HTML_HEADER tag can be used to specify a personal HTML header for \n# each generated HTML page. If it is left blank doxygen will generate a \n# standard header.\n\nHTML_HEADER            = \n\n# The HTML_FOOTER tag can be used to specify a personal HTML footer for \n# each generated HTML page. If it is left blank doxygen will generate a \n# standard footer.\n\nHTML_FOOTER            = \n\n# The HTML_STYLESHEET tag can be used to specify a user-defined cascading \n# style sheet that is used by each HTML page. It can be used to \n# fine-tune the look of the HTML output. If the tag is left blank doxygen \n# will generate a default style sheet. Note that doxygen will try to copy \n# the style sheet file to the HTML output directory, so don't put your own \n# stylesheet in the HTML output directory as well, or it will be erased!\n\nHTML_STYLESHEET        = \n\n# If the HTML_ALIGN_MEMBERS tag is set to YES, the members of classes, \n# files or namespaces will be aligned in HTML using tables. If set to \n# NO a bullet list will be used.\n\nHTML_ALIGN_MEMBERS     = YES\n\n# If the HTML_DYNAMIC_SECTIONS tag is set to YES then the generated HTML \n# documentation will contain sections that can be hidden and shown after the \n# page has loaded. For this to work a browser that supports \n# JavaScript and DHTML is required (for instance Mozilla 1.0+, Firefox \n# Netscape 6.0+, Internet explorer 5.0+, Konqueror, or Safari).\n\nHTML_DYNAMIC_SECTIONS  = NO\n\n# If the GENERATE_DOCSET tag is set to YES, additional index files \n# will be generated that can be used as input for Apple's Xcode 3 \n# integrated development environment, introduced with OSX 10.5 (Leopard). \n# To create a documentation set, doxygen will generate a Makefile in the \n# HTML output directory. Running make will produce the docset in that \n# directory and running \"make install\" will install the docset in \n# ~/Library/Developer/Shared/Documentation/DocSets so that Xcode will find \n# it at startup. \n# See http://developer.apple.com/tools/creatingdocsetswithdoxygen.html for more information.\n\nGENERATE_DOCSET        = NO\n\n# When GENERATE_DOCSET tag is set to YES, this tag determines the name of the \n# feed. A documentation feed provides an umbrella under which multiple \n# documentation sets from a single provider (such as a company or product suite) \n# can be grouped.\n\nDOCSET_FEEDNAME        = \"Doxygen generated docs\"\n\n# When GENERATE_DOCSET tag is set to YES, this tag specifies a string that \n# should uniquely identify the documentation set bundle. This should be a \n# reverse domain-name style string, e.g. com.mycompany.MyDocSet. Doxygen \n# will append .docset to the name.\n\nDOCSET_BUNDLE_ID       = org.doxygen.Project\n\n# If the GENERATE_HTMLHELP tag is set to YES, additional index files \n# will be generated that can be used as input for tools like the \n# Microsoft HTML help workshop to generate a compiled HTML help file (.chm) \n# of the generated HTML documentation.\n\nGENERATE_HTMLHELP      = NO\n\n# If the GENERATE_HTMLHELP tag is set to YES, the CHM_FILE tag can \n# be used to specify the file name of the resulting .chm file. You \n# can add a path in front of the file if the result should not be \n# written to the html output directory.\n\nCHM_FILE               = \n\n# If the GENERATE_HTMLHELP tag is set to YES, the HHC_LOCATION tag can \n# be used to specify the location (absolute path including file name) of \n# the HTML help compiler (hhc.exe). If non-empty doxygen will try to run \n# the HTML help compiler on the generated index.hhp.\n\nHHC_LOCATION           = \n\n# If the GENERATE_HTMLHELP tag is set to YES, the GENERATE_CHI flag \n# controls if a separate .chi index file is generated (YES) or that \n# it should be included in the master .chm file (NO).\n\nGENERATE_CHI           = NO\n\n# If the GENERATE_HTMLHELP tag is set to YES, the CHM_INDEX_ENCODING\n# is used to encode HtmlHelp index (hhk), content (hhc) and project file\n# content.\n\nCHM_INDEX_ENCODING     = \n\n# If the GENERATE_HTMLHELP tag is set to YES, the BINARY_TOC flag \n# controls whether a binary table of contents is generated (YES) or a \n# normal table of contents (NO) in the .chm file.\n\nBINARY_TOC             = NO\n\n# The TOC_EXPAND flag can be set to YES to add extra items for group members \n# to the contents of the HTML help documentation and to the tree view.\n\nTOC_EXPAND             = NO\n\n# If the GENERATE_QHP tag is set to YES and both QHP_NAMESPACE and QHP_VIRTUAL_FOLDER \n# are set, an additional index file will be generated that can be used as input for \n# Qt's qhelpgenerator to generate a Qt Compressed Help (.qch) of the generated \n# HTML documentation.\n\nGENERATE_QHP           = NO\n\n# If the QHG_LOCATION tag is specified, the QCH_FILE tag can \n# be used to specify the file name of the resulting .qch file. \n# The path specified is relative to the HTML output folder.\n\nQCH_FILE               = \n\n# The QHP_NAMESPACE tag specifies the namespace to use when generating \n# Qt Help Project output. For more information please see \n# <a href=\"http://doc.trolltech.com/qthelpproject.html#namespace\">Qt Help Project / Namespace</a>.\n\nQHP_NAMESPACE          = org.doxygen.Project\n\n# The QHP_VIRTUAL_FOLDER tag specifies the namespace to use when generating \n# Qt Help Project output. For more information please see \n# <a href=\"http://doc.trolltech.com/qthelpproject.html#virtual-folders\">Qt Help Project / Virtual Folders</a>.\n\nQHP_VIRTUAL_FOLDER     = doc\n\n# If the GENERATE_QHP tag is set to YES, the QHG_LOCATION tag can \n# be used to specify the location of Qt's qhelpgenerator. \n# If non-empty doxygen will try to run qhelpgenerator on the generated \n# .qhp file .\n\nQHG_LOCATION           = \n\n# The DISABLE_INDEX tag can be used to turn on/off the condensed index at \n# top of each HTML page. The value NO (the default) enables the index and \n# the value YES disables it.\n\nDISABLE_INDEX          = NO\n\n# This tag can be used to set the number of enum values (range [1..20]) \n# that doxygen will group on one line in the generated HTML documentation.\n\nENUM_VALUES_PER_LINE   = 4\n\n# The GENERATE_TREEVIEW tag is used to specify whether a tree-like index\n# structure should be generated to display hierarchical information.\n# If the tag value is set to FRAME, a side panel will be generated\n# containing a tree-like index structure (just like the one that \n# is generated for HTML Help). For this to work a browser that supports \n# JavaScript, DHTML, CSS and frames is required (for instance Mozilla 1.0+, \n# Netscape 6.0+, Internet explorer 5.0+, or Konqueror). Windows users are \n# probably better off using the HTML help feature. Other possible values \n# for this tag are: HIERARCHIES, which will generate the Groups, Directories,\n# and Class Hierarchy pages using a tree view instead of an ordered list;\n# ALL, which combines the behavior of FRAME and HIERARCHIES; and NONE, which\n# disables this behavior completely. For backwards compatibility with previous\n# releases of Doxygen, the values YES and NO are equivalent to FRAME and NONE\n# respectively.\n\nGENERATE_TREEVIEW      = NONE\n\n# If the treeview is enabled (see GENERATE_TREEVIEW) then this tag can be \n# used to set the initial width (in pixels) of the frame in which the tree \n# is shown.\n\nTREEVIEW_WIDTH         = 250\n\n# Use this tag to change the font size of Latex formulas included \n# as images in the HTML documentation. The default is 10. Note that \n# when you change the font size after a successful doxygen run you need \n# to manually remove any form_*.png images from the HTML output directory \n# to force them to be regenerated.\n\nFORMULA_FONTSIZE       = 10\n\n#---------------------------------------------------------------------------\n# configuration options related to the LaTeX output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_LATEX tag is set to YES (the default) Doxygen will \n# generate Latex output.\n\nGENERATE_LATEX         = YES\n\n# The LATEX_OUTPUT tag is used to specify where the LaTeX docs will be put. \n# If a relative path is entered the value of OUTPUT_DIRECTORY will be \n# put in front of it. If left blank `latex' will be used as the default path.\n\nLATEX_OUTPUT           = latex\n\n# The LATEX_CMD_NAME tag can be used to specify the LaTeX command name to be \n# invoked. If left blank `latex' will be used as the default command name.\n\nLATEX_CMD_NAME         = latex\n\n# The MAKEINDEX_CMD_NAME tag can be used to specify the command name to \n# generate index for LaTeX. If left blank `makeindex' will be used as the \n# default command name.\n\nMAKEINDEX_CMD_NAME     = makeindex\n\n# If the COMPACT_LATEX tag is set to YES Doxygen generates more compact \n# LaTeX documents. This may be useful for small projects and may help to \n# save some trees in general.\n\nCOMPACT_LATEX          = YES\n\n# The PAPER_TYPE tag can be used to set the paper type that is used \n# by the printer. Possible values are: a4, a4wide, letter, legal and \n# executive. If left blank a4wide will be used.\n\nPAPER_TYPE             = a4wide\n\n# The EXTRA_PACKAGES tag can be to specify one or more names of LaTeX \n# packages that should be included in the LaTeX output.\n\nEXTRA_PACKAGES         = \n\n# The LATEX_HEADER tag can be used to specify a personal LaTeX header for \n# the generated latex document. The header should contain everything until \n# the first chapter. If it is left blank doxygen will generate a \n# standard header. Notice: only use this tag if you know what you are doing!\n\nLATEX_HEADER           = \n\n# If the PDF_HYPERLINKS tag is set to YES, the LaTeX that is generated \n# is prepared for conversion to pdf (using ps2pdf). The pdf file will \n# contain links (just like the HTML output) instead of page references \n# This makes the output suitable for online browsing using a pdf viewer.\n\nPDF_HYPERLINKS         = YES\n\n# If the USE_PDFLATEX tag is set to YES, pdflatex will be used instead of \n# plain latex in the generated Makefile. Set this option to YES to get a \n# higher quality PDF documentation.\n\nUSE_PDFLATEX           = YES\n\n# If the LATEX_BATCHMODE tag is set to YES, doxygen will add the \\\\batchmode. \n# command to the generated LaTeX files. This will instruct LaTeX to keep \n# running if errors occur, instead of asking the user for help. \n# This option is also used when generating formulas in HTML.\n\nLATEX_BATCHMODE        = NO\n\n# If LATEX_HIDE_INDICES is set to YES then doxygen will not \n# include the index chapters (such as File Index, Compound Index, etc.) \n# in the output.\n\nLATEX_HIDE_INDICES     = NO\n\n#---------------------------------------------------------------------------\n# configuration options related to the RTF output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_RTF tag is set to YES Doxygen will generate RTF output \n# The RTF output is optimized for Word 97 and may not look very pretty with \n# other RTF readers or editors.\n\nGENERATE_RTF           = NO\n\n# The RTF_OUTPUT tag is used to specify where the RTF docs will be put. \n# If a relative path is entered the value of OUTPUT_DIRECTORY will be \n# put in front of it. If left blank `rtf' will be used as the default path.\n\nRTF_OUTPUT             = rtf\n\n# If the COMPACT_RTF tag is set to YES Doxygen generates more compact \n# RTF documents. This may be useful for small projects and may help to \n# save some trees in general.\n\nCOMPACT_RTF            = NO\n\n# If the RTF_HYPERLINKS tag is set to YES, the RTF that is generated \n# will contain hyperlink fields. The RTF file will \n# contain links (just like the HTML output) instead of page references. \n# This makes the output suitable for online browsing using WORD or other \n# programs which support those fields. \n# Note: wordpad (write) and others do not support links.\n\nRTF_HYPERLINKS         = NO\n\n# Load stylesheet definitions from file. Syntax is similar to doxygen's \n# config file, i.e. a series of assignments. You only have to provide \n# replacements, missing definitions are set to their default value.\n\nRTF_STYLESHEET_FILE    = \n\n# Set optional variables used in the generation of an rtf document. \n# Syntax is similar to doxygen's config file.\n\nRTF_EXTENSIONS_FILE    = \n\n#---------------------------------------------------------------------------\n# configuration options related to the man page output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_MAN tag is set to YES (the default) Doxygen will \n# generate man pages\n\nGENERATE_MAN           = NO\n\n# The MAN_OUTPUT tag is used to specify where the man pages will be put. \n# If a relative path is entered the value of OUTPUT_DIRECTORY will be \n# put in front of it. If left blank `man' will be used as the default path.\n\nMAN_OUTPUT             = man\n\n# The MAN_EXTENSION tag determines the extension that is added to \n# the generated man pages (default is the subroutine's section .3)\n\nMAN_EXTENSION          = .3\n\n# If the MAN_LINKS tag is set to YES and Doxygen generates man output, \n# then it will generate one additional man file for each entity \n# documented in the real man page(s). These additional files \n# only source the real man page, but without them the man command \n# would be unable to find the correct page. The default is NO.\n\nMAN_LINKS              = NO\n\n#---------------------------------------------------------------------------\n# configuration options related to the XML output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_XML tag is set to YES Doxygen will \n# generate an XML file that captures the structure of \n# the code including all documentation.\n\nGENERATE_XML           = NO\n\n# The XML_OUTPUT tag is used to specify where the XML pages will be put. \n# If a relative path is entered the value of OUTPUT_DIRECTORY will be \n# put in front of it. If left blank `xml' will be used as the default path.\n\nXML_OUTPUT             = xml\n\n# The XML_SCHEMA tag can be used to specify an XML schema, \n# which can be used by a validating XML parser to check the \n# syntax of the XML files.\n\nXML_SCHEMA             = \n\n# The XML_DTD tag can be used to specify an XML DTD, \n# which can be used by a validating XML parser to check the \n# syntax of the XML files.\n\nXML_DTD                = \n\n# If the XML_PROGRAMLISTING tag is set to YES Doxygen will \n# dump the program listings (including syntax highlighting \n# and cross-referencing information) to the XML output. Note that \n# enabling this will significantly increase the size of the XML output.\n\nXML_PROGRAMLISTING     = YES\n\n#---------------------------------------------------------------------------\n# configuration options for the AutoGen Definitions output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_AUTOGEN_DEF tag is set to YES Doxygen will \n# generate an AutoGen Definitions (see autogen.sf.net) file \n# that captures the structure of the code including all \n# documentation. Note that this feature is still experimental \n# and incomplete at the moment.\n\nGENERATE_AUTOGEN_DEF   = NO\n\n#---------------------------------------------------------------------------\n# configuration options related to the Perl module output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_PERLMOD tag is set to YES Doxygen will \n# generate a Perl module file that captures the structure of \n# the code including all documentation. Note that this \n# feature is still experimental and incomplete at the \n# moment.\n\nGENERATE_PERLMOD       = NO\n\n# If the PERLMOD_LATEX tag is set to YES Doxygen will generate \n# the necessary Makefile rules, Perl scripts and LaTeX code to be able \n# to generate PDF and DVI output from the Perl module output.\n\nPERLMOD_LATEX          = NO\n\n# If the PERLMOD_PRETTY tag is set to YES the Perl module output will be \n# nicely formatted so it can be parsed by a human reader.  This is useful \n# if you want to understand what is going on.  On the other hand, if this \n# tag is set to NO the size of the Perl module output will be much smaller \n# and Perl will parse it just the same.\n\nPERLMOD_PRETTY         = YES\n\n# The names of the make variables in the generated doxyrules.make file \n# are prefixed with the string contained in PERLMOD_MAKEVAR_PREFIX. \n# This is useful so different doxyrules.make files included by the same \n# Makefile don't overwrite each other's variables.\n\nPERLMOD_MAKEVAR_PREFIX = \n\n#---------------------------------------------------------------------------\n# Configuration options related to the preprocessor   \n#---------------------------------------------------------------------------\n\n# If the ENABLE_PREPROCESSING tag is set to YES (the default) Doxygen will \n# evaluate all C-preprocessor directives found in the sources and include \n# files.\n\nENABLE_PREPROCESSING   = YES\n\n# If the MACRO_EXPANSION tag is set to YES Doxygen will expand all macro \n# names in the source code. If set to NO (the default) only conditional \n# compilation will be performed. Macro expansion can be done in a controlled \n# way by setting EXPAND_ONLY_PREDEF to YES.\n\nMACRO_EXPANSION        = NO\n\n# If the EXPAND_ONLY_PREDEF and MACRO_EXPANSION tags are both set to YES \n# then the macro expansion is limited to the macros specified with the \n# PREDEFINED and EXPAND_AS_DEFINED tags.\n\nEXPAND_ONLY_PREDEF     = NO\n\n# If the SEARCH_INCLUDES tag is set to YES (the default) the includes files \n# in the INCLUDE_PATH (see below) will be search if a #include is found.\n\nSEARCH_INCLUDES        = YES\n\n# The INCLUDE_PATH tag can be used to specify one or more directories that \n# contain include files that are not input files but should be processed by \n# the preprocessor.\n\nINCLUDE_PATH           = \n\n# You can use the INCLUDE_FILE_PATTERNS tag to specify one or more wildcard \n# patterns (like *.h and *.hpp) to filter out the header-files in the \n# directories. If left blank, the patterns specified with FILE_PATTERNS will \n# be used.\n\nINCLUDE_FILE_PATTERNS  = \n\n# The PREDEFINED tag can be used to specify one or more macro names that \n# are defined before the preprocessor is started (similar to the -D option of \n# gcc). The argument of the tag is a list of macros of the form: name \n# or name=definition (no spaces). If the definition and the = are \n# omitted =1 is assumed. To prevent a macro definition from being \n# undefined via #undef or recursively expanded use the := operator \n# instead of the = operator.\n\nPREDEFINED             = \n\n# If the MACRO_EXPANSION and EXPAND_ONLY_PREDEF tags are set to YES then \n# this tag can be used to specify a list of macro names that should be expanded. \n# The macro definition that is found in the sources will be used. \n# Use the PREDEFINED tag if you want to use a different macro definition.\n\nEXPAND_AS_DEFINED      = \n\n# If the SKIP_FUNCTION_MACROS tag is set to YES (the default) then \n# doxygen's preprocessor will remove all function-like macros that are alone \n# on a line, have an all uppercase name, and do not end with a semicolon. Such \n# function macros are typically used for boiler-plate code, and will confuse \n# the parser if not removed.\n\nSKIP_FUNCTION_MACROS   = YES\n\n#---------------------------------------------------------------------------\n# Configuration::additions related to external references   \n#---------------------------------------------------------------------------\n\n# The TAGFILES option can be used to specify one or more tagfiles. \n# Optionally an initial location of the external documentation \n# can be added for each tagfile. The format of a tag file without \n# this location is as follows: \n#   TAGFILES = file1 file2 ... \n# Adding location for the tag files is done as follows: \n#   TAGFILES = file1=loc1 \"file2 = loc2\" ... \n# where \"loc1\" and \"loc2\" can be relative or absolute paths or \n# URLs. If a location is present for each tag, the installdox tool \n# does not have to be run to correct the links.\n# Note that each tag file must have a unique name\n# (where the name does NOT include the path)\n# If a tag file is not located in the directory in which doxygen \n# is run, you must also specify the path to the tagfile here.\n\nTAGFILES               = \n\n# When a file name is specified after GENERATE_TAGFILE, doxygen will create \n# a tag file that is based on the input files it reads.\n\nGENERATE_TAGFILE       = \n\n# If the ALLEXTERNALS tag is set to YES all external classes will be listed \n# in the class index. If set to NO only the inherited external classes \n# will be listed.\n\nALLEXTERNALS           = NO\n\n# If the EXTERNAL_GROUPS tag is set to YES all external groups will be listed \n# in the modules index. If set to NO, only the current project's groups will \n# be listed.\n\nEXTERNAL_GROUPS        = YES\n\n# The PERL_PATH should be the absolute path and name of the perl script \n# interpreter (i.e. the result of `which perl').\n\nPERL_PATH              = /usr/bin/perl\n\n#---------------------------------------------------------------------------\n# Configuration options related to the dot tool   \n#---------------------------------------------------------------------------\n\n# If the CLASS_DIAGRAMS tag is set to YES (the default) Doxygen will \n# generate a inheritance diagram (in HTML, RTF and LaTeX) for classes with base \n# or super classes. Setting the tag to NO turns the diagrams off. Note that \n# this option is superseded by the HAVE_DOT option below. This is only a \n# fallback. It is recommended to install and use dot, since it yields more \n# powerful graphs.\n\nCLASS_DIAGRAMS         = YES\n\n# You can define message sequence charts within doxygen comments using the \\msc \n# command. Doxygen will then run the mscgen tool (see \n# http://www.mcternan.me.uk/mscgen/) to produce the chart and insert it in the \n# documentation. The MSCGEN_PATH tag allows you to specify the directory where \n# the mscgen tool resides. If left empty the tool is assumed to be found in the \n# default search path.\n\nMSCGEN_PATH            = \n\n# If set to YES, the inheritance and collaboration graphs will hide \n# inheritance and usage relations if the target is undocumented \n# or is not a class.\n\nHIDE_UNDOC_RELATIONS   = YES\n\n# If you set the HAVE_DOT tag to YES then doxygen will assume the dot tool is \n# available from the path. This tool is part of Graphviz, a graph visualization \n# toolkit from AT&T and Lucent Bell Labs. The other options in this section \n# have no effect if this option is set to NO (the default)\n\nHAVE_DOT               = NO\n\n# By default doxygen will write a font called FreeSans.ttf to the output \n# directory and reference it in all dot files that doxygen generates. This \n# font does not include all possible unicode characters however, so when you need \n# these (or just want a differently looking font) you can specify the font name \n# using DOT_FONTNAME. You need need to make sure dot is able to find the font, \n# which can be done by putting it in a standard location or by setting the \n# DOTFONTPATH environment variable or by setting DOT_FONTPATH to the directory \n# containing the font.\n\nDOT_FONTNAME           = FreeSans\n\n# The DOT_FONTSIZE tag can be used to set the size of the font of dot graphs. \n# The default size is 10pt.\n\nDOT_FONTSIZE           = 10\n\n# By default doxygen will tell dot to use the output directory to look for the \n# FreeSans.ttf font (which doxygen will put there itself). If you specify a \n# different font using DOT_FONTNAME you can set the path where dot \n# can find it using this tag.\n\nDOT_FONTPATH           = \n\n# If the CLASS_GRAPH and HAVE_DOT tags are set to YES then doxygen \n# will generate a graph for each documented class showing the direct and \n# indirect inheritance relations. Setting this tag to YES will force the \n# the CLASS_DIAGRAMS tag to NO.\n\nCLASS_GRAPH            = YES\n\n# If the COLLABORATION_GRAPH and HAVE_DOT tags are set to YES then doxygen \n# will generate a graph for each documented class showing the direct and \n# indirect implementation dependencies (inheritance, containment, and \n# class references variables) of the class with other documented classes.\n\nCOLLABORATION_GRAPH    = YES\n\n# If the GROUP_GRAPHS and HAVE_DOT tags are set to YES then doxygen \n# will generate a graph for groups, showing the direct groups dependencies\n\nGROUP_GRAPHS           = YES\n\n# If the UML_LOOK tag is set to YES doxygen will generate inheritance and \n# collaboration diagrams in a style similar to the OMG's Unified Modeling \n# Language.\n\nUML_LOOK               = NO\n\n# If set to YES, the inheritance and collaboration graphs will show the \n# relations between templates and their instances.\n\nTEMPLATE_RELATIONS     = NO\n\n# If the ENABLE_PREPROCESSING, SEARCH_INCLUDES, INCLUDE_GRAPH, and HAVE_DOT \n# tags are set to YES then doxygen will generate a graph for each documented \n# file showing the direct and indirect include dependencies of the file with \n# other documented files.\n\nINCLUDE_GRAPH          = YES\n\n# If the ENABLE_PREPROCESSING, SEARCH_INCLUDES, INCLUDED_BY_GRAPH, and \n# HAVE_DOT tags are set to YES then doxygen will generate a graph for each \n# documented header file showing the documented files that directly or \n# indirectly include this file.\n\nINCLUDED_BY_GRAPH      = YES\n\n# If the CALL_GRAPH and HAVE_DOT options are set to YES then \n# doxygen will generate a call dependency graph for every global function \n# or class method. Note that enabling this option will significantly increase \n# the time of a run. So in most cases it will be better to enable call graphs \n# for selected functions only using the \\callgraph command.\n\nCALL_GRAPH             = NO\n\n# If the CALLER_GRAPH and HAVE_DOT tags are set to YES then \n# doxygen will generate a caller dependency graph for every global function \n# or class method. Note that enabling this option will significantly increase \n# the time of a run. So in most cases it will be better to enable caller \n# graphs for selected functions only using the \\callergraph command.\n\nCALLER_GRAPH           = NO\n\n# If the GRAPHICAL_HIERARCHY and HAVE_DOT tags are set to YES then doxygen \n# will graphical hierarchy of all classes instead of a textual one.\n\nGRAPHICAL_HIERARCHY    = YES\n\n# If the DIRECTORY_GRAPH, SHOW_DIRECTORIES and HAVE_DOT tags are set to YES \n# then doxygen will show the dependencies a directory has on other directories \n# in a graphical way. The dependency relations are determined by the #include\n# relations between the files in the directories.\n\nDIRECTORY_GRAPH        = YES\n\n# The DOT_IMAGE_FORMAT tag can be used to set the image format of the images \n# generated by dot. Possible values are png, jpg, or gif\n# If left blank png will be used.\n\nDOT_IMAGE_FORMAT       = png\n\n# The tag DOT_PATH can be used to specify the path where the dot tool can be \n# found. If left blank, it is assumed the dot tool can be found in the path.\n\nDOT_PATH               = \n\n# The DOTFILE_DIRS tag can be used to specify one or more directories that \n# contain dot files that are included in the documentation (see the \n# \\dotfile command).\n\nDOTFILE_DIRS           = \n\n# The DOT_GRAPH_MAX_NODES tag can be used to set the maximum number of \n# nodes that will be shown in the graph. If the number of nodes in a graph \n# becomes larger than this value, doxygen will truncate the graph, which is \n# visualized by representing a node as a red box. Note that doxygen if the \n# number of direct children of the root node in a graph is already larger than \n# DOT_GRAPH_MAX_NODES then the graph will not be shown at all. Also note \n# that the size of a graph can be further restricted by MAX_DOT_GRAPH_DEPTH.\n\nDOT_GRAPH_MAX_NODES    = 50\n\n# The MAX_DOT_GRAPH_DEPTH tag can be used to set the maximum depth of the \n# graphs generated by dot. A depth value of 3 means that only nodes reachable \n# from the root by following a path via at most 3 edges will be shown. Nodes \n# that lay further from the root node will be omitted. Note that setting this \n# option to 1 or 2 may greatly reduce the computation time needed for large \n# code bases. Also note that the size of a graph can be further restricted by \n# DOT_GRAPH_MAX_NODES. Using a depth of 0 means no depth restriction.\n\nMAX_DOT_GRAPH_DEPTH    = 0\n\n# Set the DOT_TRANSPARENT tag to YES to generate images with a transparent \n# background. This is disabled by default, because dot on Windows does not \n# seem to support this out of the box. Warning: Depending on the platform used, \n# enabling this option may lead to badly anti-aliased labels on the edges of \n# a graph (i.e. they become hard to read).\n\nDOT_TRANSPARENT        = NO\n\n# Set the DOT_MULTI_TARGETS tag to YES allow dot to generate multiple output \n# files in one run (i.e. multiple -o and -T options on the command line). This \n# makes dot run faster, but since only newer versions of dot (>1.8.10) \n# support this, this feature is disabled by default.\n\nDOT_MULTI_TARGETS      = NO\n\n# If the GENERATE_LEGEND tag is set to YES (the default) Doxygen will \n# generate a legend page explaining the meaning of the various boxes and \n# arrows in the dot generated graphs.\n\nGENERATE_LEGEND        = YES\n\n# If the DOT_CLEANUP tag is set to YES (the default) Doxygen will \n# remove the intermediate dot files that are used to generate \n# the various graphs.\n\nDOT_CLEANUP            = YES\n\n#---------------------------------------------------------------------------\n# Configuration::additions related to the search engine   \n#---------------------------------------------------------------------------\n\n# The SEARCHENGINE tag specifies whether or not a search engine should be \n# used. If set to NO the values of all tags below this one will be ignored.\n\nSEARCHENGINE           = NO\n"
  },
  {
    "path": "java-api/README",
    "content": "# Copyright 2007-2008 Konrad-Zuse-Zentrum für Informationstechnik Berlin\n#\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#\n#      http://www.apache.org/licenses/LICENSE-2.0\n#\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nThis project can be build using the ant build tool.\n\nThe following targets are available:\n - jar:\t\t\tcreates chordsharp4j.jar, chordsharp4j-lib.jar and chordsharp4j-examples.jar\n - doc:\t\t\tgenerates the API-docs (public entities only) in ./doc\n - devel-doc:\tgenerates the API-docs (public and private) in ./doc-devel\n - dist:\t\tgenerates archives containing the sources, the jars and the documentation:\n \t\t\t\tDistributedLogger.zip, DistributedLogger.tar.gz, DistributedLogger.tar.bz2\n - clean:\t\tremoves all generated files\n - test:\t\truns unit tests\n\nFor more information regarding the implementation of the Chord# Java Interface\nhave a look into the API-docs. The general structure is described in the\ndocumentation of the package \"de.zib.chordsharp\".\n\n./src\tcontains the application's sources\n./test\tcontains the sources of the junit test cases\n"
  },
  {
    "path": "java-api/build.xml",
    "content": "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<project name=\"Scalaris Java Interface\" default=\"jar\" basedir=\".\">\n\n  <!--\n    Setting the following environment variables influence the build:\n    SCALARIS_JAVADOC_JINTERFACE_URL - url of the javadoc of JInterface\n    SCALARIS_JAVADOC_JINTERFACE_OFFLINE (\"true\"|\"false\") - link offline using cached package-list file\n    SCALARIS_JAVADOC_JAVASE6_URL - url of the javadoc of JavaSE 6\n    SCALARIS_JAVADOC_JAVASE6_OFFLINE (\"true\"|\"false\") - link offline using cached package-list file\n    SCALARIS_JAVADOC_APACHECOMMONSCLI_URL - url of the javadoc of Apache commons cli\n    SCALARIS_JAVADOC_APACHECOMMONSCLI_OFFLINE (\"true\"|\"false\") - link offline using cached package-list file\n    SCALARIS_JAVADOC_OFFLINE (\"true\"|\"false\") - link offline using cached package-list files (sets all unset offline properties mentioned above)\n    \n    defaults:\n    SCALARIS_JAVADOC_JINTERFACE_URL = http://www.erlang.org/doc/apps/jinterface/java/\n    SCALARIS_JAVADOC_JAVASE6_URL = http://java.sun.com/javase/6/docs/api/\n    SCALARIS_JAVADOC_APACHECOMMONSCLI_URL = http://commons.apache.org/cli/api-release/\n    SCALARIS_JAVADOC_OFFLINE = true\n-->\n\n  <!-- ########## Build configuration (begin) ########## -->\n  <!-- filename without file extension -->\n  <property name=\"scalaris.dist.filename\" value=\"scalaris\" />\n  <!-- directory where the classes are compiled to -->\n  <property name=\"scalaris.classes.dirname\" value=\"classes\" />\n  <!-- version of the OtpErlang jar file to bundle and compile against (OtpErlang-<version>.jar) -->\n  <property name=\"opterlang.version\" value=\"1.6.1\" />\n  <!-- version of the jakarta-commons-cli jar file to bundle and compile against (jakarta-commons-cli-<version>.jar) -->\n  <property name=\"jakarta.commons.cli.version\" value=\"1.2\" />\n  <!-- debug level to use during compilation -->\n  <property name=\"debuglevel\" value=\"source,lines,vars\" />\n  <!-- ########## Build configuration (end) ########## -->\n\n  <!-- ########## Run configuration (begin) ########## -->\n  <condition property=\"scalaris.node\" value=\"\">\n    <not>\n      <isset property=\"scalaris.node\" />\n    </not>\n  </condition>\n  <condition property=\"scalaris.cookie\" value=\"\">\n    <not>\n      <isset property=\"scalaris.cookie\" />\n    </not>\n  </condition>\n  <condition property=\"scalaris.client.name\" value=\"\">\n    <not>\n      <isset property=\"scalaris.client.name\" />\n    </not>\n  </condition>\n  <condition property=\"scalaris.client.appendUUID\" value=\"\">\n    <not>\n      <isset property=\"scalaris.client.appendUUID\" />\n    </not>\n  </condition>\n  <!-- ########## Run configuration (end) ########## -->\n\n  <property environment=\"env\" />\n  <!-- set paths to documentations to link to -->\n  <condition property=\"javadoc.jinterface.url\" value=\"${env.SCALARIS_JAVADOC_JINTERFACE_URL}\" else=\"http://www.erlang.org/doc/apps/jinterface/java/\">\n    <isset property=\"env.SCALARIS_JAVADOC_JINTERFACE_URL\" />\n  </condition>\n  <condition property=\"javadoc.javase6.url\" value=\"${env.SCALARIS_JAVADOC_JAVASE6_URL}\" else=\"http://java.sun.com/javase/6/docs/api/\">\n    <isset property=\"env.SCALARIS_JAVADOC_JAVASE6_URL\" />\n  </condition>\n  <condition property=\"javadoc.commons-apache-cli.url\" value=\"${env.SCALARIS_JAVADOC_APACHECOMMONSCLI_URL}\" else=\"http://commons.apache.org/cli/api-release/\">\n    <isset property=\"env.SCALARIS_JAVADOC_APACHECOMMONSCLI_URL\" />\n  </condition>\n\n  <!-- set whether those documentations should be linked offline with cached package-list files -->\n  <condition property=\"javadoc.offline\" value=\"true\" else=\"false\">\n    <or>\n      <and>\n        <isset property=\"env.SCALARIS_JAVADOC_OFFLINE\" />\n        <istrue value=\"${env.SCALARIS_JAVADOC_OFFLINE}\" />\n      </and>\n      <not>\n        <isset property=\"env.SCALARIS_JAVADOC_OFFLINE\" />\n      </not>\n    </or>\n  </condition>\n  <condition property=\"javadoc.jinterface.offline\" value=\"true\" else=\"false\">\n    <or>\n      <and>\n        <isset property=\"env.SCALARIS_JAVADOC_JINTERFACE_OFFLINE\" />\n        <istrue value=\"${env.SCALARIS_JAVADOC_JINTERFACE_OFFLINE}\" />\n      </and>\n      <and>\n        <not>\n          <isset property=\"env.SCALARIS_JAVADOC_JINTERFACE_OFFLINE\" />\n        </not>\n        <istrue value=\"${javadoc.offline}\" />\n      </and>\n    </or>\n  </condition>\n  <condition property=\"javadoc.javase6.offline\" value=\"true\" else=\"false\">\n    <or>\n      <and>\n        <isset property=\"env.SCALARIS_JAVADOC_JAVASE6_OFFLINE\" />\n        <istrue value=\"${env.SCALARIS_JAVADOC_JAVASE6_OFFLINE}\" />\n      </and>\n      <and>\n        <not>\n          <isset property=\"env.SCALARIS_JAVADOC_JAVASE6_OFFLINE\" />\n        </not>\n        <istrue value=\"${javadoc.offline}\" />\n      </and>\n    </or>\n  </condition>\n  <condition property=\"javadoc.commons-apache-cli.offline\" value=\"true\" else=\"false\">\n    <or>\n      <and>\n        <isset property=\"env.SCALARIS_JAVADOC_APACHECOMMONSCLI_OFFLINE\" />\n        <istrue value=\"${env.SCALARIS_JAVADOC_APACHECOMMONSCLI_OFFLINE}\" />\n      </and>\n      <and>\n        <not>\n          <isset property=\"env.SCALARIS_JAVADOC_APACHECOMMONSCLI_OFFLINE\" />\n        </not>\n        <istrue value=\"${javadoc.offline}\" />\n      </and>\n    </or>\n  </condition>\n\n  <!-- set paths to package-list files of linked documentation -->\n  <condition property=\"javadoc.jinterface.pkglist\" value=\"${javadoc.jinterface.url}\" else=\"doc-contrib/jinterface\">\n    <isfalse value=\"${javadoc.jinterface.offline}\" />\n  </condition>\n  <condition property=\"javadoc.javase6.pkglist\" value=\"${javadoc.javase6.url}\" else=\"doc-contrib/javase6\">\n    <isfalse value=\"${javadoc.javase6.offline}\" />\n  </condition>\n  <condition property=\"javadoc.commons-apache-cli.pkglist\" value=\"${javadoc.commons-apache-cli.url}\" else=\"doc-contrib/apache_commons_cli\">\n    <isfalse value=\"${javadoc.commons-apache-cli.offline}\" />\n  </condition>\n\n  <!--\n    <echo message=\"javadoc.offline is set to = ${javadoc.offline}\"/>\n    <echo message=\"javadoc.jinterface.url is set to = ${javadoc.jinterface.url}\"/>\n    <echo message=\"javadoc.jinterface.offline is set to = ${javadoc.jinterface.offline}\"/>\n    <echo message=\"javadoc.jinterface.pkglist is set to = ${javadoc.jinterface.pkglist}\"/>\n    <echo message=\"javadoc.javase6.url is set to = ${javadoc.javase6.url}\"/>\n    <echo message=\"javadoc.javase6.offline is set to = ${javadoc.javase6.offline}\"/>\n    <echo message=\"javadoc.javase6.pkglist is set to = ${javadoc.javase6.pkglist}\"/>\n    <echo message=\"javadoc.commons-apache-cli.url is set to = ${javadoc.commons-apache-cli.url}\"/>\n    <echo message=\"javadoc.commons-apache-cli.offline is set to = ${javadoc.commons-apache-cli.offline}\"/>\n    <echo message=\"javadoc.commons-apache-cli.pkglist is set to = ${javadoc.commons-apache-cli.pkglist}\"/>\n-->\n\n  <path id=\"scalaris.classpath\">\n    <pathelement path='lib/OtpErlang-${opterlang.version}.jar' />\n    <pathelement path='lib/jakarta-commons-cli-${jakarta.commons.cli.version}.jar' />\n  </path>\n  <path id=\"scalaris.test.classpath\">\n    <path refid=\"scalaris.classpath\" />\n    <pathelement path=\"${scalaris.classes.dirname}\" />\n    <path>\n      <fileset dir=\"test-lib\" includes=\"*.jar\"/>\n    </path>\n  </path>\n\n  <!-- ########## Public targets (begin) ########## -->\n  <target name='compile' depends=\"scalaris.compile\" description=\"Compiles all source files for the library and the default Main class.\" />\n\n  <target name=\"jar\" depends=\"scalaris.jar\" description=\"Creates jar files for use on command line or in other projects.\" />\n\n  <target name='clean' depends=\"dist.clean, jar.clean, test.clean, compile.clean, doc.clean, devel-doc.clean\" description=\"Deletes all generated files.\">\n    <delete dir=\"${scalaris.classes.dirname}\" />\n  </target>\n\n  <target name=\"dist\" depends=\"scalaris.dist\" description=\"Creates archives with source, docs and jar files.\" />\n\n  <target name=\"test\" depends=\"tools.test, scalaris.test\" description=\"Runs all unit tests in non-graphical mode.\" />\n\n  <target name=\"examples\" depends=\"jar, scalaris.examples.jar\" />\n  <!-- ########## Public targets (end) ########## -->\n\n  <!-- ########## Common targets (begin) ########## -->\n  <target name=\"compile.clean\">\n    <delete includeemptydirs=\"true\" failonerror=\"false\">\n      <fileset dir='${scalaris.classes.dirname}' includes='**/*.class' />\n    </delete>\n  </target>\n  <target name=\"jar.clean\" depends=\"scalaris.jar.clean\" />\n  <target name=\"dist.clean\" depends=\"scalaris.dist.clean\" />\n  <target name=\"test.clean\">\n    <delete includeemptydirs=\"true\" failonerror=\"false\">\n      <fileset dir='${scalaris.classes.dirname}' includes='**/*Test.class, **/*.properties' />\n    </delete>\n  </target>\n\n  <target name='doc'>\n    <mkdir dir='doc' />\n    <javadoc encoding=\"UTF-8\" docencoding=\"UTF-8\" charset=\"UTF-8\" sourcepath=\"src\" destdir=\"doc\" classpathref=\"scalaris.classpath\" access=\"public\" author=\"true\" doctitle=\"Scalaris Java API\" nodeprecated=\"false\" nodeprecatedlist=\"false\" noindex=\"false\" nonavbar=\"false\" notree=\"false\" packagenames=\"de.zib.scalaris.*\" source=\"1.6\" splitindex=\"true\" use=\"true\" version=\"true\">\n\n      <link href=\"${javadoc.jinterface.url}\" offline=\"${javadoc.jinterface.offline}\" packagelistLoc=\"${javadoc.jinterface.pkglist}\" />\n      <link href=\"${javadoc.javase6.url}\" offline=\"${javadoc.javase6.offline}\" packagelistLoc=\"${javadoc.javase6.pkglist}\" />\n      <link href=\"${javadoc.commons-apache-cli.url}\" offline=\"${javadoc.commons-apache-cli.offline}\" packagelistLoc=\"${javadoc.commons-apache-cli.pkglist}\" />\n    </javadoc>\n  </target>\n\n  <target name='doc-doxygen'>\n    <mkdir dir='doc-doxygen' />\n    <doxygen>\n      <property name=\"INPUT\" value=\"src\" />\n      <property name=\"RECURSIVE\" value=\"src\" />\n      <property name=\"GENERATE_LAXTEX\" value=\"true\" />\n      <property name=\"PROJECT_NAME\" value=\"Scalaris Java API\" />\n    </doxygen>\n  </target>\n\n  <target name=\"doc.clean\">\n    <delete dir=\"doc\" />\n  </target>\n\n  <target name='devel-doc'>\n    <mkdir dir='doc-devel' />\n    <javadoc encoding=\"UTF-8\" docencoding=\"UTF-8\" charset=\"UTF-8\" sourcepath=\"src\" destdir=\"doc-devel\" classpathref=\"scalaris.classpath\" access=\"private\" author=\"true\" doctitle=\"Scalaris Java API\" nodeprecated=\"false\" nodeprecatedlist=\"false\" noindex=\"false\" nonavbar=\"false\" notree=\"false\" packagenames=\"de.zib.*\" source=\"1.6\" splitindex=\"true\" use=\"true\" version=\"true\">\n      <link href=\"${javadoc.jinterface.url}\" offline=\"${javadoc.jinterface.offline}\" packagelistLoc=\"${javadoc.jinterface.pkglist}\" />\n      <link href=\"${javadoc.javase6.url}\" offline=\"${javadoc.javase6.offline}\" packagelistLoc=\"${javadoc.javase6.pkglist}\" />\n      <link href=\"${javadoc.commons-apache-cli.url}\" offline=\"${javadoc.commons-apache-cli.offline}\" packagelistLoc=\"${javadoc.commons-apache-cli.pkglist}\" />\n    </javadoc>\n  </target>\n\n  <target name=\"devel-doc.clean\">\n    <delete dir=\"doc-devel\" />\n  </target>\n  <!-- ########## Common targets (end) ########## -->\n\n  <!-- ########## Tools targets (begin) ########## -->\n  <target name='tools.compile'>\n    <mkdir dir='${scalaris.classes.dirname}' />\n    <javac encoding=\"UTF-8\" srcdir='src' destdir='${scalaris.classes.dirname}' includes=\"de/zib/tools/*.java\" debug=\"true\" debuglevel=\"${debuglevel}\" includeAntRuntime=\"false\" />\n  </target>\n\n  <target name=\"tools.test.compile\" depends=\"tools.compile\">\n    <mkdir dir=\"${scalaris.classes.dirname}\" />\n    <javac encoding=\"UTF-8\" srcdir='test' destdir='${scalaris.classes.dirname}' classpathref=\"scalaris.test.classpath\" includes=\"de/zib/tools/*.java\" debug=\"true\" debuglevel=\"${debuglevel}\" includeAntRuntime=\"false\" />\n    <copy includeemptydirs=\"false\" todir=\"${scalaris.classes.dirname}\">\n      <fileset dir=\"test\" includes=\"de/zib/tools/*.properties\" />\n    </copy>\n  </target>\n\n  <target name=\"tools.test\" depends=\"tools.test.compile\" description=\"Run Chord# unit tests in non-graphical mode\">\n    <junit dir=\"${scalaris.classes.dirname}\" printsummary=\"true\" haltonfailure=\"true\" fork=\"yes\">\n      <formatter type=\"brief\" usefile=\"false\" />\n      <classpath refid=\"scalaris.test.classpath\" />\n      <batchtest>\n        <fileset dir=\"${scalaris.classes.dirname}\" includes=\"de/zib/tools**/*Test.class\" />\n      </batchtest>\n    </junit>\n  </target>\n  <!-- ########## Tools targets (end) ########## -->\n\n  <!-- ########## Scalaris targets (begin) ########## -->\n  <target name='scalaris.compile' depends=\"tools.compile\">\n    <mkdir dir='${scalaris.classes.dirname}' />\n    <javac encoding=\"UTF-8\" srcdir='src' destdir='${scalaris.classes.dirname}' classpathref=\"scalaris.classpath\" includes=\"de/zib/scalaris/*.java de/zib/scalaris/executor/*.java de/zib/scalaris/jmx/*.java de/zib/scalaris/operations/*.java\" debug=\"true\" debuglevel=\"${debuglevel}\" includeAntRuntime=\"false\" />\n  </target>\n  <target name='scalaris.examples.compile' depends=\"scalaris.compile\">\n    <javac encoding=\"UTF-8\" srcdir='src' destdir='${scalaris.classes.dirname}' classpathref=\"scalaris.classpath\" includes=\"de/zib/scalaris/examples/*.java\" debug=\"true\" debuglevel=\"${debuglevel}\" includeAntRuntime=\"false\" />\n  </target>\n\n  <!-- package library and examples separately -->\n  <target name='scalaris.jar' depends=\"scalaris.compile\">\n    <!--\n            copy the connection settings file to the folder where the jar files\n            resist so that the main program can access it\n        -->\n    <copy file=\"src/scalaris.properties\" todir=\".\" />\n    <jar destfile=\"${scalaris.dist.filename}.jar\" basedir=\"${scalaris.classes.dirname}\" includes=\"de/zib/scalaris/*.class de/zib/scalaris/executor/*.class de/zib/scalaris/operations/*.class de/zib/scalaris/jmx/*.class de/zib/tools/*.class\" excludes=\"**/examples** **/*Test.class **/*Test$*.class\">\n      <manifest>\n        <attribute name=\"Built-By\" value=\"${user.name}\" />\n        <attribute name=\"Bundle-Vendor\" value=\"Zuse Institute Berlin\" />\n        <attribute name=\"Bundle-Name\" value=\"Scalaris Java Interface\" />\n        <attribute name=\"Bundle-Version\" value=\"2.0.0\" />\n        <attribute name=\"Export-Package\" value=\"de.zib.scalaris.*\" />\n        <attribute name=\"Main-Class\" value=\"de.zib.scalaris.Main\" />\n        <!--\n                    <attribute name=\"Class-Path\" value=\".\n                    lib/jakarta-commons-cli-${jakarta.commons.cli.version}.jar lib/OtpErlang-${opterlang.version}.jar\" />\n                -->\n      </manifest>\n    </jar>\n  </target>\n  <target name='scalaris.examples.jar' depends=\"scalaris.examples.compile\">\n    <jar destfile=\"${scalaris.dist.filename}-examples.jar\" basedir=\"${scalaris.classes.dirname}\" includes=\"de/zib/scalaris/examples/*.class\" excludes=\"**/*Test.class **/*Test$*.class\">\n      <manifest>\n        <attribute name=\"Built-By\" value=\"${user.name}\" />\n        <attribute name=\"Bundle-Vendor\" value=\"Zuse Institute Berlin\" />\n        <attribute name=\"Bundle-Name\" value=\"Scalaris Java Interface\" />\n        <attribute name=\"Bundle-Version\" value=\"2.0.0\" />\n        <attribute name=\"Export-Package\" value=\"de.zib.scalaris.*\" />\n        <attribute name=\"Class-Path\" value=\". ${scalaris.dist.filename}.jar lib/jakarta-commons-cli-${jakarta.commons.cli.version}.jar lib/OtpErlang-${opterlang.version}.jar\" />\n      </manifest>\n    </jar>\n  </target>\n\n  <target name=\"scalaris.jar.clean\">\n    <delete file=\"scalaris.properties\" />\n    <delete file=\"${scalaris.dist.filename}.jar\" />\n    <delete file=\"${scalaris.dist.filename}-examples.jar\" />\n  </target>\n\n  <target name=\"scalaris.dist\" depends=\"scalaris.jar, scalaris.examples.jar, doc\">\n    <fileset dir=\"src\" id=\"src\">\n      <include name=\"**/*\" />\n    </fileset>\n    <fileset dir=\"test\" id=\"src-test\">\n      <include name=\"**/*\" />\n    </fileset>\n    <fileset dir=\"lib\" id=\"lib\">\n      <include name=\"**/*.jar\" />\n    </fileset>\n    <fileset dir=\"test-lib\" id=\"test-lib\">\n      <include name=\"**/*.jar\" />\n    </fileset>\n    <fileset dir=\"doc\" id=\"doc\">\n      <include name=\"**/*\" />\n    </fileset>\n    <fileset dir=\".\" id=\"files\">\n      <include name=\"build.xml\" />\n      <include name=\"scalaris.properties\" />\n      <include name=\"${scalaris.dist.filename}.jar\" />\n      <include name=\"${scalaris.dist.filename}-examples.jar\" />\n      <include name=\"LICENSE\" />\n      <include name=\"README\" />\n    </fileset>\n\n    <zip destfile=\"${scalaris.dist.filename}.zip\">\n      <zipfileset refid=\"src\" prefix=\"${scalaris.dist.filename}/src\" />\n      <zipfileset refid=\"src-test\" prefix=\"${scalaris.dist.filename}/test\" />\n      <zipfileset refid=\"lib\" prefix=\"${scalaris.dist.filename}/lib\" />\n      <zipfileset refid=\"test-lib\" prefix=\"${scalaris.dist.filename}/test-lib\" />\n      <zipfileset refid=\"doc\" prefix=\"${scalaris.dist.filename}/doc\" />\n      <zipfileset refid=\"files\" prefix=\"${scalaris.dist.filename}/\" />\n    </zip>\n\n    <tar destfile=\"${scalaris.dist.filename}.tar\">\n      <tarfileset refid=\"src\" prefix=\"${scalaris.dist.filename}/src\" />\n      <tarfileset refid=\"src-test\" prefix=\"${scalaris.dist.filename}/test\" />\n      <tarfileset refid=\"doc\" prefix=\"${scalaris.dist.filename}/doc\" />\n      <tarfileset refid=\"lib\" prefix=\"${scalaris.dist.filename}/lib\" />\n      <tarfileset refid=\"test-lib\" prefix=\"${scalaris.dist.filename}/test-lib\" />\n      <tarfileset refid=\"files\" prefix=\"${scalaris.dist.filename}/\" />\n    </tar>\n\n    <gzip src=\"${scalaris.dist.filename}.tar\" destfile=\"${scalaris.dist.filename}.tar.gz\" />\n    <bzip2 src=\"${scalaris.dist.filename}.tar\" destfile=\"${scalaris.dist.filename}.tar.bz2\" />\n\n    <delete file=\"${scalaris.dist.filename}.tar\" />\n  </target>\n\n  <target name=\"scalaris.dist.clean\">\n    <delete file=\"${scalaris.dist.filename}.zip\" />\n    <delete file=\"${scalaris.dist.filename}.tar.gz\" />\n    <delete file=\"${scalaris.dist.filename}.tar.bz2\" />\n  </target>\n\n  <target name=\"scalaris.test.compile\" depends=\"scalaris.compile\">\n    <mkdir dir=\"${scalaris.classes.dirname}\" />\n    <javac encoding=\"UTF-8\" srcdir='test' destdir='${scalaris.classes.dirname}' classpathref=\"scalaris.test.classpath\" includes=\"de/zib/scalaris/*.java\" debug=\"true\" debuglevel=\"${debuglevel}\" includeAntRuntime=\"false\" />\n    <copy includeemptydirs=\"false\" todir=\"${scalaris.classes.dirname}\">\n      <fileset dir=\"test\" includes=\"**/*.properties\" />\n    </copy>\n  </target>\n\n  <target name=\"scalaris.test\" depends=\"scalaris.test.compile\" description=\"Run Scalaris unit tests in non-graphical mode\">\n    <junit dir=\"${scalaris.classes.dirname}\" printsummary=\"true\" haltonfailure=\"true\" fork=\"yes\">\n      <sysproperty key=\"scalaris.node\" value=\"${scalaris.node}\" />\n      <sysproperty key=\"scalaris.cookie\" value=\"${scalaris.cookie}\" />\n      <sysproperty key=\"scalaris.client.name\" value=\"${scalaris.client.name}\" />\n      <sysproperty key=\"scalaris.client.appendUUID\" value=\"${scalaris.client.appendUUID}\" />\n      <formatter type=\"brief\" usefile=\"false\" />\n      <classpath refid=\"scalaris.test.classpath\" />\n      <batchtest>\n        <fileset dir=\"${scalaris.classes.dirname}\" includes=\"de/zib/scalaris**/*Test.class\" excludes=\"de/zib/scalaris**/InterOpTest.class\" />\n      </batchtest>\n    </junit>\n  </target>\n  <!-- ########## Scalaris targets (end) ########## -->\n</project>\n"
  },
  {
    "path": "java-api/doc-contrib/apache_commons_cli/package-list",
    "content": "org.apache.commons.cli\n"
  },
  {
    "path": "java-api/doc-contrib/javase6/package-list",
    "content": "java.applet\njava.awt\njava.awt.color\njava.awt.datatransfer\njava.awt.dnd\njava.awt.event\njava.awt.font\njava.awt.geom\njava.awt.im\njava.awt.im.spi\njava.awt.image\njava.awt.image.renderable\njava.awt.print\njava.beans\njava.beans.beancontext\njava.io\njava.lang\njava.lang.annotation\njava.lang.instrument\njava.lang.management\njava.lang.ref\njava.lang.reflect\njava.math\njava.net\njava.nio\njava.nio.channels\njava.nio.channels.spi\njava.nio.charset\njava.nio.charset.spi\njava.rmi\njava.rmi.activation\njava.rmi.dgc\njava.rmi.registry\njava.rmi.server\njava.security\njava.security.acl\njava.security.cert\njava.security.interfaces\njava.security.spec\njava.sql\njava.text\njava.text.spi\njava.util\njava.util.concurrent\njava.util.concurrent.atomic\njava.util.concurrent.locks\njava.util.jar\njava.util.logging\njava.util.prefs\njava.util.regex\njava.util.spi\njava.util.zip\njavax.accessibility\njavax.activation\njavax.activity\njavax.annotation\njavax.annotation.processing\njavax.crypto\njavax.crypto.interfaces\njavax.crypto.spec\njavax.imageio\njavax.imageio.event\njavax.imageio.metadata\njavax.imageio.plugins.bmp\njavax.imageio.plugins.jpeg\njavax.imageio.spi\njavax.imageio.stream\njavax.jws\njavax.jws.soap\njavax.lang.model\njavax.lang.model.element\njavax.lang.model.type\njavax.lang.model.util\njavax.management\njavax.management.loading\njavax.management.modelmbean\njavax.management.monitor\njavax.management.openmbean\njavax.management.relation\njavax.management.remote\njavax.management.remote.rmi\njavax.management.timer\njavax.naming\njavax.naming.directory\njavax.naming.event\njavax.naming.ldap\njavax.naming.spi\njavax.net\njavax.net.ssl\njavax.print\njavax.print.attribute\njavax.print.attribute.standard\njavax.print.event\njavax.rmi\njavax.rmi.CORBA\njavax.rmi.ssl\njavax.script\njavax.security.auth\njavax.security.auth.callback\njavax.security.auth.kerberos\njavax.security.auth.login\njavax.security.auth.spi\njavax.security.auth.x500\njavax.security.cert\njavax.security.sasl\njavax.sound.midi\njavax.sound.midi.spi\njavax.sound.sampled\njavax.sound.sampled.spi\njavax.sql\njavax.sql.rowset\njavax.sql.rowset.serial\njavax.sql.rowset.spi\njavax.swing\njavax.swing.border\njavax.swing.colorchooser\njavax.swing.event\njavax.swing.filechooser\njavax.swing.plaf\njavax.swing.plaf.basic\njavax.swing.plaf.metal\njavax.swing.plaf.multi\njavax.swing.plaf.synth\njavax.swing.table\njavax.swing.text\njavax.swing.text.html\njavax.swing.text.html.parser\njavax.swing.text.rtf\njavax.swing.tree\njavax.swing.undo\njavax.tools\njavax.transaction\njavax.transaction.xa\njavax.xml\njavax.xml.bind\njavax.xml.bind.annotation\njavax.xml.bind.annotation.adapters\njavax.xml.bind.attachment\njavax.xml.bind.helpers\njavax.xml.bind.util\njavax.xml.crypto\njavax.xml.crypto.dom\njavax.xml.crypto.dsig\njavax.xml.crypto.dsig.dom\njavax.xml.crypto.dsig.keyinfo\njavax.xml.crypto.dsig.spec\njavax.xml.datatype\njavax.xml.namespace\njavax.xml.parsers\njavax.xml.soap\njavax.xml.stream\njavax.xml.stream.events\njavax.xml.stream.util\njavax.xml.transform\njavax.xml.transform.dom\njavax.xml.transform.sax\njavax.xml.transform.stax\njavax.xml.transform.stream\njavax.xml.validation\njavax.xml.ws\njavax.xml.ws.handler\njavax.xml.ws.handler.soap\njavax.xml.ws.http\njavax.xml.ws.soap\njavax.xml.ws.spi\njavax.xml.ws.wsaddressing\njavax.xml.xpath\norg.ietf.jgss\norg.omg.CORBA\norg.omg.CORBA.DynAnyPackage\norg.omg.CORBA.ORBPackage\norg.omg.CORBA.TypeCodePackage\norg.omg.CORBA.portable\norg.omg.CORBA_2_3\norg.omg.CORBA_2_3.portable\norg.omg.CosNaming\norg.omg.CosNaming.NamingContextExtPackage\norg.omg.CosNaming.NamingContextPackage\norg.omg.Dynamic\norg.omg.DynamicAny\norg.omg.DynamicAny.DynAnyFactoryPackage\norg.omg.DynamicAny.DynAnyPackage\norg.omg.IOP\norg.omg.IOP.CodecFactoryPackage\norg.omg.IOP.CodecPackage\norg.omg.Messaging\norg.omg.PortableInterceptor\norg.omg.PortableInterceptor.ORBInitInfoPackage\norg.omg.PortableServer\norg.omg.PortableServer.CurrentPackage\norg.omg.PortableServer.POAManagerPackage\norg.omg.PortableServer.POAPackage\norg.omg.PortableServer.ServantLocatorPackage\norg.omg.PortableServer.portable\norg.omg.SendingContext\norg.omg.stub.java.rmi\norg.w3c.dom\norg.w3c.dom.bootstrap\norg.w3c.dom.events\norg.w3c.dom.ls\norg.xml.sax\norg.xml.sax.ext\norg.xml.sax.helpers\n"
  },
  {
    "path": "java-api/doc-contrib/jinterface/package-list",
    "content": "com.ericsson.otp.erlang\n"
  },
  {
    "path": "java-api/lib/.gitignore",
    "content": "/jakarta-commons-cli-*-javadoc.jar\n/jakarta-commons-cli-*-sources.jar\n"
  },
  {
    "path": "java-api/maven_git_hook.sh",
    "content": "#!/bin/bash\n# Deploy hook functions to glue maven and svn together\n# This should be called by the command \"maven deploy\".\n\n# scalaris maven repo\nurl=\"git@github.com:scalaris-team/scalaris.git\"\n# maven repo checkout folder\nfolder=\"../.maven\"\n\ncheckout () {\n    # check out if maven folder doesn't exist\n    # otherwise update\n    if [ ! -d ${folder} ]; then\n        echo \"checkout ${url} -> ${folder} ...\"\n        git clone --branch gh-pages --single-branch \"${url}\" \"${folder}\"\n        result=$?\n    else\n        echo \"update ${url} -> ${folder} ...\"\n        cd \"${folder}\"\n        git pull\n        result=$?\n        cd - >/dev/null\n    fi\n\n    if [ ${result} -eq 0 ]; then\n        echo \"Maven repository has been updated locally.\"\n    else\n        echo \"Maven repository couldn't be updated.\"\n        exit 1\n    fi\n}\n\ncommit () {\n    # put latest erlang jinterface jar into the repository\n    file=$(ls lib/OtpErlang-*)\n    version=$(basename $file .jar | cut -d \"-\" -f2,3)\n    mvn deploy:deploy-file  -Dfile=\"$file\" \\\n        -Dversion=\"$version\" -DgroupId=\"org.erlang.otp\" -DartifactId=\"jinterface\" \\\n        -Dpackaging=\"jar\" -DrepositoryId=\"scalaris\" -Durl=\"file:$folder/maven\"\n\n    # update the remote maven repository\n    echo -n \"Do you want to update the remote maven repository? [y/N] \"\n    read -e answer\n    if [[ ${answer} == \"y\" ]]; then\n        cd \"${folder}\"\n        git add maven\n        git commit\n        git push\n        cd - >/dev/null\n    fi\n}\n\n\nif [[ $1 == \"checkout\" ]]; then\n    checkout\nelif [[ $1 == \"commit\" ]]; then\n    commit\nelse\n    echo \"Missing an argument.\"\n    exit 1\nfi\n"
  },
  {
    "path": "java-api/pom.xml",
    "content": "<project xmlns=\"http://maven.apache.org/POM/4.0.0\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n         xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd\">\n    <modelVersion>4.0.0</modelVersion>\n\n    <groupId>de.zib.scalaris</groupId>\n    <artifactId>java-api</artifactId>\n    <version>0.9.1-SNAPSHOT</version>\n    <packaging>jar</packaging>\n\n    <name>Scalaris Java API</name>\n    <url>http://scalaris.googlecode.com</url>\n\n    <dependencies>\n        <dependency>\n            <groupId>org.erlang.otp</groupId>\n            <artifactId>jinterface</artifactId>\n            <version>[1.5.6-custom,)</version>\n        </dependency>\n        <dependency>\n            <groupId>commons-cli</groupId>\n            <artifactId>commons-cli</artifactId>\n            <version>1.2</version>\n        </dependency>\n    </dependencies>\n\n    <repositories>\n        <repository>\n            <id>scalaris-repo</id>\n            <url>https://scalaris-team.github.io/scalaris/maven</url>\n        </repository>\n    </repositories>\n\n    <distributionManagement>\n        <repository>\n            <id>scalaris</id>\n            <url>file:../.maven/maven</url>\n        </repository>\n    </distributionManagement>\n\n    <build>\n        <sourceDirectory>src</sourceDirectory>\n        <plugins>\n            <plugin>\n                <!--- Use JDK 1.6 -->\n                <artifactId>maven-compiler-plugin</artifactId>\n                <version>3.1</version>\n                <configuration>\n                    <source>1.6</source>\n                    <target>1.6</target>\n                </configuration>\n            </plugin>\n            <plugin>\n                <groupId>org.apache.maven.plugins</groupId>\n                <artifactId>maven-source-plugin</artifactId>\n                <version>2.2.1</version>\n                <executions>\n                    <execution>\n                        <id>attach-sources</id>\n                        <goals>\n                            <goal>jar</goal>\n                        </goals>\n                    </execution>\n                </executions>\n            </plugin>\n            <plugin>\n                <artifactId>exec-maven-plugin</artifactId>\n                <groupId>org.codehaus.mojo</groupId>\n                <version>1.4.0</version>\n                <executions>\n                    <execution>\n                        <id>Check out maven repository from git</id>\n                        <!-- Execute before deploy phase -->\n                        <!-- http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html -->\n                        <phase>install</phase>\n                        <goals>\n                            <goal>exec</goal>\n                        </goals>\n                        <configuration>\n                            <skip>${skipGit}</skip>\n                            <executable>maven_git_hook.sh</executable>\n                            <arguments>\n                                <argument>checkout</argument>\n                            </arguments>\n                        </configuration>\n                    </execution>\n                    <execution>\n                        <id>Update the scalaris maven repository</id>\n                        <phase>deploy</phase>\n                        <goals>\n                            <goal>exec</goal>\n                        </goals>\n                        <configuration>\n                            <skip>${skipGit}</skip>\n                            <executable>maven_git_hook.sh</executable>\n                            <arguments>\n                                <argument>commit</argument>\n                            </arguments>\n                        </configuration>\n                    </execution>\n                </executions>\n            </plugin>\n        </plugins>\n    </build>\n\n</project>\n"
  },
  {
    "path": "java-api/scalaris-java.conf.in",
    "content": "#  Copyright 2007-2011 Zuse Institute Berlin\n#\n#   Licensed under the Apache License, Version 2.0 (the \"License\");\n#   you may not use this file except in compliance with the License.\n#   You may obtain a copy of the License at\n#\n#       http://www.apache.org/licenses/LICENSE-2.0\n#\n#   Unless required by applicable law or agreed to in writing, software\n#   distributed under the License is distributed on an \"AS IS\" BASIS,\n#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#   See the License for the specific language governing permissions and\n#   limitations under the License.\n\n# sample configuration file for the scalaris java client start script\n# save file to @sysconfdir@/scalaris/scalaris-java.conf or ~/.scalaris/scalaris-java.conf\n\n# 1. rpm_mode\n#\n#  Specifies:\n#\n#      If in rpm mode system tools will be used to build the classpath and\n#      set up the java virtual machine.\n#\n#  Type of value:\n#\n#      boolean\n#\n#  Default value:\n#\n#      false\n#\n#  Effect if unset:\n#\n#      Start script builds the classpath based on the libraries provided in\n#      SCALARIS_JAVA_HOME.\n#\nrpm_mode=true\n\n# 2. SCALARIS_JAVA_HOME\n#\n#  Specifies:\n#\n#      The directory of the scalaris.jar file to use.\n#      SCALARIS_JAVA_HOME/lib provides some needed libraries.\n#\n#  Type of value:\n#\n#      Path name\n#\n#  Default value:\n#\n#      unset\n#\n#  Effect if unset:\n#\n#      Start script will set the value to the directory it is located in.\n#\nSCALARIS_JAVA_HOME=\"@datarootdir@/java/scalaris\"\n\n# 3. OPT_JAR_LIST\n#\n#  Specifies:\n#\n#      Additional libraries to add to the classpath.\n#\n#  Type of value:\n#\n#      list of names of jar files (without extension)\n#\n#  Default value:\n#\n#      empty\n#\n#  Effect if unset:\n#\n#      Start script will set the value to the directory it is located in.\n#\nOPT_JAR_LIST=\n\n# 4. LOCALCLASSPATH\n#\n#  Specifies:\n#\n#      Classpath to prepend to the path the start script sets up.\n#      (only used if not in rpm_mode)\n#\n#  Type of value:\n#\n#      java classpath\n#\n#  Default value:\n#\n#      empty\n#\n#  Notes:\n#      The following classpath will be set up by the start script if not in rpm_mode:\n#      $SCALARIS_JAVA_HOME/scalaris.jar:$SCALARIS_JAVA_HOME/lib/jakarta-commons-cli-1.1.jar:$SCALARIS_JAVA_HOME/lib/OtpErlang-1.4.jar\nLOCALCLASSPATH=\n\n# 4. SCALARIS_JAVA_CONFIG\n#\n#  Specifies:\n#\n#      Name of the config file to load (contains properties such as the node\n#      name to connect to).\n#\n#  Type of value:\n#\n#      file name (including path)\n#\n#  Default value:\n#\n#      \"scalaris.properties\"\n#      (set by the java classes)\n#\n#  Effect if unset:\n#\n#      The de.zib.scalaris.ConnectionFactory class will try to find the\n#      default file at the same location where the jar file is located.\n#      If no file exists, default values are used - see \n#      de.zib.scalaris.ConnectionFactory::ConnectionFactory() for more details.\n#\nSCALARIS_JAVA_CONFIG=\"@sysconfdir@/scalaris/scalaris.properties\"\n"
  },
  {
    "path": "java-api/scalaris-java.conf.sample",
    "content": "#  Copyright 2007-2008 Konrad-Zuse-Zentrum für Informationstechnik Berlin\n#\n#   Licensed under the Apache License, Version 2.0 (the \"License\");\n#   you may not use this file except in compliance with the License.\n#   You may obtain a copy of the License at\n#\n#       http://www.apache.org/licenses/LICENSE-2.0\n#\n#   Unless required by applicable law or agreed to in writing, software\n#   distributed under the License is distributed on an \"AS IS\" BASIS,\n#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#   See the License for the specific language governing permissions and\n#   limitations under the License.\n\n# sample configuration file for the scalaris java client start script\n# save file to /etc/scalaris/scalaris-java.conf or ~/.scalaris/scalaris-java.conf\n\n# 1. rpm_mode\n#\n#  Specifies:\n#\n#      If in rpm mode system tools will be used to build the classpath and\n#      set up the java virtual machine.\n#\n#  Type of value:\n#\n#      boolean\n#\n#  Default value:\n#\n#      false\n#\n#  Effect if unset:\n#\n#      Start script builds the classpath based on the libraries provided in\n#      SCALARIS_JAVA_HOME.\n#\nrpm_mode=true\n\n# 2. SCALARIS_JAVA_HOME\n#\n#  Specifies:\n#\n#      The directory of the scalaris.jar file to use.\n#      SCALARIS_JAVA_HOME/lib provides some needed libraries.\n#\n#  Type of value:\n#\n#      Path name\n#\n#  Default value:\n#\n#      unset\n#\n#  Effect if unset:\n#\n#      Start script will set the value to the directory it is located in.\n#\nSCALARIS_JAVA_HOME=\"/usr/share/java/scalaris\"\n\n# 3. OPT_JAR_LIST\n#\n#  Specifies:\n#\n#      Additional libraries to add to the classpath.\n#\n#  Type of value:\n#\n#      list of names of jar files (without extension)\n#\n#  Default value:\n#\n#      empty\n#\n#  Effect if unset:\n#\n#      Start script will set the value to the directory it is located in.\n#\nOPT_JAR_LIST=\n\n# 4. LOCALCLASSPATH\n#\n#  Specifies:\n#\n#      Classpath to prepend to the path the start script sets up.\n#      (only used if not in rpm_mode)\n#\n#  Type of value:\n#\n#      java classpath\n#\n#  Default value:\n#\n#      empty\n#\n#  Notes:\n#      The following classpath will be set up by the start script if not in rpm_mode:\n#      $SCALARIS_JAVA_HOME/scalaris.jar:$SCALARIS_JAVA_HOME/lib/jakarta-commons-cli-1.1.jar:$SCALARIS_JAVA_HOME/lib/OtpErlang-1.4.jar\nLOCALCLASSPATH=\n\n# 4. SCALARIS_JAVA_CONFIG\n#\n#  Specifies:\n#\n#      Name of the config file to load (contains properties such as the node\n#      name to connect to).\n#\n#  Type of value:\n#\n#      file name (including path)\n#\n#  Default value:\n#\n#      \"scalaris.properties\"\n#      (set by the java classes)\n#\n#  Effect if unset:\n#\n#      The de.zib.scalaris.ConnectionFactory class will try to find the\n#      default file at the same location where the jar file is located.\n#      If no file exists, default values are used - see \n#      de.zib.scalaris.ConnectionFactory::ConnectionFactory() for more details.\n#\nSCALARIS_JAVA_CONFIG=\"/etc/scalaris/scalaris.properties\"\n"
  },
  {
    "path": "java-api/scalaris.in",
    "content": "#!/bin/bash\n\n#  Copyright 2007-2011 Zuse Institute Berlin\n#\n#   Licensed under the Apache License, Version 2.0 (the \"License\");\n#   you may not use this file except in compliance with the License.\n#   You may obtain a copy of the License at\n#\n#       http://www.apache.org/licenses/LICENSE-2.0\n#\n#   Unless required by applicable law or agreed to in writing, software\n#   distributed under the License is distributed on an \"AS IS\" BASIS,\n#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#   See the License for the specific language governing permissions and\n#   limitations under the License.\n\n# inspired by the start script of ant\n\nOTP_VERSION=1.6.1\nJAKARTA_COMMONS_VERSION=1.2\nERL=@ERL@\n\nprefix=@prefix@\nexec_prefix=@exec_prefix@\ndatarootdir=@datarootdir@\n\n# Extract launch and scalaris arguments, (see details below).\nscalaris_exec_args=\nno_config=false\nscalaris_exec_debug=false\nshow_help=false\nno_erl=false\njmvopts=\"\"\nERLANG_HOSTNAME=${ERLANG_HOSTNAME-\"\"}\n\nuntil [ -z \"$1\" ]; do\n  case $1 in\n    \"--noconfig\")\n      no_config=true\n      shift;;\n    \"--noerl\")\n      no_erl=true\n      shift;;\n    \"--execdebug\")\n      scalaris_exec_debug=true\n      shift;;\n    \"-h\" | \"--help\" | \"-help\")\n      show_help=true\n      scalaris_exec_args=\"$scalaris_exec_args --help\"\n      shift;;\n    \"--jvmopts\")\n      shift\n      jmvopts=\"$jmvopts $1\"\n      shift;;\n    *)\n      scalaris_exec_args=\"$scalaris_exec_args \\\"$1\\\"\"\n      shift;;\n  esac\ndone\n\n# activate no_config if called in the source tree:\nABSPATH=\"$(cd \"${0%/*}\" 2>/dev/null; echo \"$PWD\"/\"${0##*/}\")\"\nDIRNAME=`dirname $ABSPATH`\n# is this a source checkout or an (rpm/deb/manual) installation?\nif [ \"$DIRNAME\" != \"@bindir@\" -a \"$DIRNAME\" != \"/bin\" ]; then\n  no_config=true\nfi\n\nget_cookie() {\n  if [ -z \"$SCALARIS_JAPI_COOKIE\" ]; then\n    echo -n \"$SCALARIS_JAPI_COOKIE\"\n    return\n  else\n    if $no_config ; then\n      echo -n \"\"\n    else\n      # system config\n      . @sysconfdir@/scalaris/scalarisctl.conf\n\n      # load user scalaris configuration (overrides system config)\n      if [ -f \"$HOME/.scalaris/scalarisctl.conf\" ] ; then\n        . $HOME/.scalaris/scalarisctl.conf\n      fi\n\n      echo -n \"${cookie:-\"chocolate chip cookie\"}\"\n    fi\n  fi\n}\n\n# Source default scalaris configuration?\nif $no_config ; then\n  rpm_mode=false\nelse\n  # load system-wide scalaris configuration (ONLY if SCALARIS_JAVA_HOME has NOT been set)\n  # otherwise assume that the configuration has already been set\n  if [ -z \"$SCALARIS_JAVA_HOME\" -o \"$SCALARIS_JAVA_HOME\" = \"@datarootdir@/java/scalaris\" ]; then\n      if [ -f \"@sysconfdir@/scalaris/scalaris-java.conf\" ] ; then\n          . @sysconfdir@/scalaris/scalaris-java.conf\n      fi\n  fi\n\n  # load user scalaris configuration\n  if [ -f \"$HOME/.scalaris/scalaris-java.conf\" ] ; then\n    . $HOME/.scalaris/scalaris-java.conf\n  fi\n\n  # provide default configuration values\n  if [ -z \"$rpm_mode\" ] ; then\n    rpm_mode=false\n  fi\nfi\n\n# Setup Java environment in rpm mode\nif $rpm_mode ; then\n  if [ -f @JAVAFUNCTIONS@ ] ; then\n    . @JAVAFUNCTIONS@\n    set_jvm\n    set_javacmd\n  fi\nfi\n\nget_hostname() {\n    # Uses the OS hostname from \"hostname -f\" for the Erlang long node name.\n    if [ -z \"$ERLANG_HOSTNAME\" ] ; then\n        ERLANG_HOSTNAME=\"`get_sys_hostname`\"\n    fi\n    # Alternative using the hostname that erlang chooses if set up to start with a long node\n    # name and falls back to \"hostname -f\" if this is not possible:\n    # NOTE: In this case, the Java-API (if used as a library) is not always able\n    #       to guess the host name that Erlang expects and connections may fail.\n    #       Therefore, the convenience node spec \"<node>@localhost\" may not be used.\n#     if [ -z \"$ERLANG_HOSTNAME\" ] ; then\n#         ERLANG_HOSTNAME=\"`get_erlang_hostname`\"\n#         if [ -z \"$ERLANG_HOSTNAME\" ] ; then\n#             # could not start erlang with long node name (no FQDN?)\n#             ERLANG_HOSTNAME=\"`get_sys_hostname`\"\n#         fi\n#     fi\n}\n\nget_erlang_hostname() {\n    MY_HOSTNAME_ERL=\"`$ERL -name foo$$ -noinput -eval 'N=erlang:atom_to_list(node()),io:format(\"~s\", [string:sub_string(N, string:chr(N, $@)+1)]), halt(0).' 2>/dev/null`\"\n    if [ $? -ne 0 ] ; then\n        MY_HOSTNAME_ERL=\n    fi\n    echo \"$MY_HOSTNAME_ERL\"\n}\n\nget_sys_hostname() {\n    hostname -f\n}\n\n# Get local host name from erlang\nif ! $no_erl; then\n  get_hostname\nfi\n\n# OS specific support.  $var _must_ be set to either true or false.\ncygwin=false;\ndarwin=false;\ncase \"`uname`\" in\n  CYGWIN*) cygwin=true ;;\n  Darwin*) darwin=true\n           if [ -z \"$JAVA_HOME\" ] ; then\n             JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home\n           fi\n           ;;\nesac\n\nif [ -z \"$SCALARIS_JAVA_HOME\" -o ! -d \"$SCALARIS_JAVA_HOME\" ] ; then\n  ## resolve links - $0 may be a link to scalaris's home\n  PRG=\"$0\"\n  progname=`basename \"$0\"`\n\n  # need this for relative symlinks\n  while [ -h \"$PRG\" ] ; do\n    ls=`ls -ld \"$PRG\"`\n    link=`expr \"$ls\" : '.*-> \\(.*\\)$'`\n    if expr \"$link\" : '/.*' > /dev/null; then\n    PRG=\"$link\"\n    else\n    PRG=`dirname \"$PRG\"`\"/$link\"\n    fi\n  done\n\n  SCALARIS_JAVA_HOME=`dirname \"$PRG\"`\n\n  # make it fully qualified\n  SCALARIS_JAVA_HOME=`cd \"$SCALARIS_JAVA_HOME\" && pwd`\nfi\n\n# For Cygwin, ensure paths are in UNIX format before anything is touched\nif $cygwin ; then\n  [ -n \"$SCALARIS_JAVA_HOME\" ] &&\n    SCALARIS_JAVA_HOME=`cygpath --unix \"$SCALARIS_JAVA_HOME\"`\n  [ -n \"$JAVA_HOME\" ] &&\n    JAVA_HOME=`cygpath --unix \"$JAVA_HOME\"`\nfi\n\nif [ -z \"$JAVACMD\" ] ; then\n  if [ -n \"$JAVA_HOME\"  ] ; then\n    # IBM's JDK on AIX uses strange locations for the executables\n    if [ -x \"$JAVA_HOME/jre/sh/java\" ] ; then\n      JAVACMD=\"$JAVA_HOME/jre/sh/java\"\n    elif [ -x \"$JAVA_HOME/jre/bin/java\" ] ; then\n      JAVACMD=\"$JAVA_HOME/jre/bin/java\"\n    else\n      JAVACMD=\"$JAVA_HOME/bin/java\"\n    fi\n  else\n    JAVACMD=`which java 2> /dev/null `\n    if [ -z \"$JAVACMD\" ] ; then\n        JAVACMD=java\n    fi\n  fi\nfi\n\nif [ ! -x \"$JAVACMD\" ] ; then\n  echo \"Error: JAVA_HOME is not defined correctly.\"\n  echo \"  We cannot execute $JAVACMD\"\n  exit 1\nfi\n\n# include our Scalaris jar files (we know where they are and do not need to\n# rely on tools like build-classpath which may include the wrong library,\n# i.e. not the installed one)\nif [ -z \"$LOCALCLASSPATH\" ] ; then\n  LOCALCLASSPATH=$SCALARIS_JAVA_HOME/scalaris.jar:$SCALARIS_JAVA_HOME/lib/jakarta-commons-cli-$JAKARTA_COMMONS_VERSION.jar:$SCALARIS_JAVA_HOME/lib/OtpErlang-$OTP_VERSION.jar\nelse\n  LOCALCLASSPATH=$SCALARIS_JAVA_HOME/scalaris.jar:$SCALARIS_JAVA_HOME/lib/jakarta-commons-cli-$JAKARTA_COMMONS_VERSION.jar:$SCALARIS_JAVA_HOME/lib/OtpErlang-$OTP_VERSION.jar:$LOCALCLASSPATH\nfi\n\n# Build local classpath manually in non-rpm mode or\n# use the Jpackage helper in rpm mode with basic and default jars\n# specified in the scalaris-java.conf configuration. A user should\n# request optional jars and their dependencies via the OPT_JAR_LIST\n# variable\nif $rpm_mode && [ -x \"@BUILDCLASSPATH@\" ] ; then\n  # If the user requested to try to add some other jars to the classpath\n  if [ -n \"$OPT_JAR_LIST\" ] ; then\n    _OPTCLASSPATH=\"$(@BUILDCLASSPATH@ $OPT_JAR_LIST 2> /dev/null)\"\n    if [ -n \"$_OPTCLASSPATH\" ] ; then \n      LOCALCLASSPATH=\"$LOCALCLASSPATH:$_OPTCLASSPATH\"\n    fi\n  fi\n\n  # Explicitly add javac path to classpath, assume JAVA_HOME set\n  # properly in rpm mode\n  if [ -f \"$JAVA_HOME/lib/tools.jar\" ] ; then\n    LOCALCLASSPATH=\"$LOCALCLASSPATH:$JAVA_HOME/lib/tools.jar\"\n  fi\n  if [ -f \"$JAVA_HOME/lib/classes.zip\" ] ; then\n    LOCALCLASSPATH=\"$LOCALCLASSPATH:$JAVA_HOME/lib/classes.zip\"\n  fi\n\n  # if CLASSPATH_OVERRIDE env var is set, LOCALCLASSPATH will be\n  # user CLASSPATH first and scalaris-found jars after.\n  # In that case, the user CLASSPATH will override scalaris-found jars\n  #\n  # if CLASSPATH_OVERRIDE is not set, we'll have the normal behaviour\n  # with scalaris-found jars first and user CLASSPATH after\n  if [ -n \"$CLASSPATH\" ] ; then\n    # merge local and specified classpath \n    if [ -z \"$LOCALCLASSPATH\" ] ; then \n      LOCALCLASSPATH=\"$CLASSPATH\"\n    elif [ -n \"$CLASSPATH_OVERRIDE\" ] ; then\n      LOCALCLASSPATH=\"$CLASSPATH:$LOCALCLASSPATH\"\n    else\n      LOCALCLASSPATH=\"$LOCALCLASSPATH:$CLASSPATH\"\n    fi\n\n    # remove class path from launcher -cp option\n    CLASSPATH=\"\"\n  fi\nfi\n\n# For Cygwin, switch paths to appropriate format before running java\n# For PATHs convert to unix format first, then to windows format to ensure\n# both formats are supported. Probably this will fail on directories with ;\n# in the name in the path. Let's assume that paths containing ; are more\n# rare than windows style paths on cygwin.\nif $cygwin; then\n  if [ \"$OS\" = \"Windows_NT\" ] && cygpath -m .>/dev/null 2>/dev/null ; then\n    format=mixed\n  else\n    format=windows\n  fi\n  SCALARIS_JAVA_HOME=`cygpath --$format \"$SCALARIS_JAVA_HOME\"`\n  JAVA_HOME=`cygpath --$format \"$JAVA_HOME\"`\n  LCP_TEMP=`cygpath --path --unix \"$LOCALCLASSPATH\"`\n  LOCALCLASSPATH=`cygpath --path --$format \"$LCP_TEMP\"`\n  if [ -n \"$CLASSPATH\" ] ; then\n    CP_TEMP=`cygpath --path --unix \"$CLASSPATH\"`\n    CLASSPATH=`cygpath --path --$format \"$CP_TEMP\"`\n  fi\n  CYGHOME=`cygpath --$format \"$HOME\"`\nfi\n\n# Show script help if requested\nif $show_help ; then\n  echo $0 '[script options] [options]'\n  echo 'Script Options:'\n  echo '  --help, -h             print this message and scalaris help'\n  echo '  --noconfig             suppress sourcing of config files in $HOME/.scalaris/'\n  echo '                         and @sysconfdir@/scalaris/'\n  echo '  --execdebug            print scalaris exec line generated by this'\n  echo '                         launch script'\n  echo '  --noerl                do not ask erlang for its (local) host name'\n  echo '  '\nfi\n# add a second backslash to variables terminated by a backslash under cygwin\nif $cygwin; then\n  case \"$SCALARIS_JAVA_HOME\" in\n    *\\\\ )\n    SCALARIS_JAVA_HOME=\"$SCALARIS_JAVA_HOME\\\\\"\n    ;;\n  esac\n  case \"$CYGHOME\" in\n    *\\\\ )\n    CYGHOME=\"$CYGHOME\\\\\"\n    ;;\n  esac\n  case \"$LOCALCLASSPATH\" in\n    *\\\\ )\n    LOCALCLASSPATH=\"$LOCALCLASSPATH\\\\\"\n    ;;\n  esac\n  case \"$CLASSPATH\" in\n    *\\\\ )\n    CLASSPATH=\"$CLASSPATH\\\\\"\n    ;;\n  esac\nfi\n\nscalaris_java_config=\nif [ -n \"$SCALARIS_JAVA_CONFIG\" ]; then\n  scalaris_java_config=\"-Dscalaris.java.config=\\\"$SCALARIS_JAVA_CONFIG\\\"\"\nfi\n\n# Execute scalaris using eval/exec to preserve spaces in paths,\n# java options, and scalaris args\nscalaris_exec_command=\"exec \\\"$JAVACMD\\\" $SCALARIS_OPTS\\\n  -classpath \\\"$LOCALCLASSPATH\\\"\\\n  $jmvopts\\\n  -Dscalaris.java.home=\\\"$SCALARIS_JAVA_HOME\\\"\\\n  -Dscalaris.erlang.nodename=\\\"$ERLANG_HOSTNAME\\\"\\\n  -Dscalaris.cookie=\\\"`get_cookie`\\\"\\\n  -Dscalaris.node=\\\"$SCALARIS_JAPI_NODE\\\"\\\n  $scalaris_java_config de.zib.scalaris.Main $SCALARIS_ARGS $scalaris_exec_args\"\nif $scalaris_exec_debug ; then\n    echo $scalaris_exec_command\nfi\neval $scalaris_exec_command\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/AbortException.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.ArrayList;\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\n/**\n * Exception that is thrown if a the commit of a transaction on a\n * scalaris ring fails.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.12\n * @since 2.5\n */\npublic class AbortException extends OtpErlangException {\n    /**\n     * class version for serialisation\n     */\n    private static final long serialVersionUID = 1L;\n\n    /**\n     * List of keys responsible for the abort.\n     */\n    private final List<String> failedKeys;\n\n    /**\n     * Creates the exception with no message.\n     *\n     * @deprecated abort failures always come with a list of responsible keys,\n     *             use {@link #AbortException(List)} instead\n     */\n    @Deprecated\n    public AbortException() {\n        this.failedKeys = new ArrayList<String>(0);\n    }\n\n    /**\n     * Creates the exception with the given message.\n     *\n     * @param msg\n     *            message of the exception\n     *\n     * @deprecated abort failures always come with a list of responsible keys,\n     *             use {@link #AbortException(String, List)} instead\n     */\n    @Deprecated\n    public AbortException(final String msg) {\n        super(msg);\n        this.failedKeys = new ArrayList<String>(0);\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e the exception to \"re-throw\"\n     *\n     * @deprecated abort failures always come with a list of responsible keys,\n     *             use {@link #AbortException(Throwable, List)} instead\n     */\n    @Deprecated\n    public AbortException(final Throwable e) {\n        super(e.getMessage());\n        setStackTrace(e.getStackTrace());\n        this.failedKeys = new ArrayList<String>(0);\n    }\n\n    /**\n     * Creates an exception including the message of the given erlang object.\n     *\n     * @param responsibleKeys\n     *            list of keys responsible for the abort\n     *\n     * @since 3.12\n     */\n    public AbortException(final List<String> responsibleKeys) {\n        super(\"Erlang message: {fail, abort, \" + responsibleKeys.toString() + \"}\");\n        this.failedKeys = responsibleKeys;\n    }\n\n    /**\n     * Creates the exception with the given message.\n     *\n     * @param msg\n     *            message of the exception\n     * @param responsibleKeys\n     *            list of keys responsible for the abort\n     *\n     * @since 3.12\n     */\n    public AbortException(final String msg, final List<String> responsibleKeys) {\n        super(msg);\n        this.failedKeys = new ArrayList<String>(0);\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e\n     *            the exception to \"re-throw\"\n     * @param responsibleKeys\n     *            list of keys responsible for the abort\n     *\n     * @since 3.12\n     */\n    public AbortException(final Throwable e, final List<String> responsibleKeys) {\n        super(e.getMessage() + \",\\n  Erlang message: {fail, abort, \" + responsibleKeys.toString() + \"}\");\n        setStackTrace(e.getStackTrace());\n        this.failedKeys = responsibleKeys;\n    }\n\n    /**\n     * @return the responsibleKeys\n     */\n    public List<String> getFailedKeys() {\n        return failedKeys;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/AbstractTransaction.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpErlangDouble;\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangLong;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\n\nimport de.zib.scalaris.operations.AddDelOnListOp;\nimport de.zib.scalaris.operations.AddOnNrOp;\nimport de.zib.scalaris.operations.Operation;\nimport de.zib.scalaris.operations.ReadOp;\nimport de.zib.scalaris.operations.TestAndSetOp;\nimport de.zib.scalaris.operations.WriteOp;\n\n/**\n * Generic base class for {@link Transaction} and {@link TransactionSingleOp}.\n *\n * @param <ReqL> {@link RequestList} type\n * @param <ResL> {@link ResultList} type\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.14\n */\npublic abstract class AbstractTransaction<ReqL extends RequestList, ResL extends ResultList> {\n\n    /**\n     * Connection to a Scalaris node.\n     */\n    protected final Connection connection;\n\n    /**\n     * Whether to compress the transfer of values.\n     *\n     * @since 3.14\n     */\n    protected boolean compressed = true;\n\n    /**\n     * Constructor, uses the default connection returned by\n     * {@link ConnectionFactory#createConnection()}.\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     */\n    public AbstractTransaction() throws ConnectionException {\n        super();\n        connection = ConnectionFactory.getInstance().createConnection();\n    }\n\n    /**\n     * Constructor, uses the given connection to an erlang node.\n     *\n     * @param conn\n     *            connection to use for the transaction\n     */\n    public AbstractTransaction(final Connection conn) {\n        connection = conn;\n    }\n\n    abstract protected ReqL newReqList();\n\n    /**\n     * Executes the given operation.\n     *\n     * @param op\n     *            the operation to execute\n     *\n     * @return results list containing a single result of the given operation\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws AbortException\n     *             if a commit failed (if there was one)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @see #req_list(RequestList)\n     *\n     * @since 3.18\n     */\n    protected ResL req_list(final Operation op) throws ConnectionException,\n            AbortException, UnknownException {\n        final ReqL reqList = newReqList();\n        reqList.addOp(op);\n        return req_list(reqList);\n    }\n\n    /**\n     * Executes all requests in <code>req</code>.\n     *\n     * <p>\n     * The transaction's log is reset if a commit in the request list was\n     * successful, otherwise it still retains in the transaction which must be\n     * successfully committed, aborted or reset in order to be (re-)used for\n     * another request.\n     * </p>\n     *\n     * @param req\n     *            the requests to issue\n     *\n     * @return results of all requests in the same order as they appear in\n     *         <code>req</code>\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws AbortException\n     *             if a commit failed (if there was one)\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    abstract public ResL req_list(final ReqL req) throws ConnectionException,\n            AbortException, UnknownException;\n\n    /**\n     * Selects the module to use depending in the {@link #compressed} property.\n     *\n     * @return a module name\n     */\n    protected String module() {\n        return compressed ? \"api_txc\" : \"api_tx\";\n    }\n\n    /**\n     * Gets the value stored under the given <code>key</code>.\n     *\n     * @param key\n     *            the key to look up\n     *\n     * @return the value stored under the given <code>key</code>\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws NotFoundException\n     *             if the requested key does not exist\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 2.9\n     */\n    public ErlangValue read(final OtpErlangString key)\n            throws ConnectionException, NotFoundException, UnknownException {\n        try {\n            final ResL result = req_list(new ReadOp(key));\n            return result.processReadAt(0);\n        } catch (final AbortException e) {\n            // should not occur (we did not commit anything)\n            throw new UnknownException(e);\n        }\n    }\n\n    /**\n     * Gets the value stored under the given <code>key</code>.\n     *\n     * @param key\n     *            the key to look up\n     *\n     * @return the value stored under the given <code>key</code>\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws NotFoundException\n     *             if the requested key does not exist\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @see #read(OtpErlangString)\n     * @since 2.9\n     */\n    public ErlangValue read(final String key) throws ConnectionException,\n            NotFoundException, UnknownException {\n        return read(new OtpErlangString(key));\n    }\n\n    /**\n     * Stores the given <code>key</code>/<code>value</code> pair.\n     *\n     * @param key\n     *            the key to store the value for\n     * @param value\n     *            the value to store\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws AbortException\n     *             if the commit failed (if there was one)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 2.9\n     */\n    public void write(final OtpErlangString key, final OtpErlangObject value)\n            throws ConnectionException, AbortException, UnknownException {\n        final ResL result = req_list(new WriteOp(key, value));\n        result.processWriteAt(0);\n    }\n\n    /**\n     * Stores the given <code>key</code>/<code>value</code> pair.\n     *\n     * @param <T>\n     *            the type of the <tt>value</tt>\n     * @param key\n     *            the key to store the value for\n     * @param value\n     *            the value to store\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws AbortException\n     *             if the commit failed (if there was one)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @see #write(OtpErlangString, OtpErlangObject)\n     * @since 2.9\n     */\n    public <T> void write(final String key, final T value)\n            throws ConnectionException, AbortException, UnknownException {\n        write(new OtpErlangString(key), ErlangValue.convertToErlang(value));\n    }\n\n    /**\n     * Changes the list stored at the given key, i.e. first adds all items in\n     * <tt>toAdd</tt> then removes all items in <tt>toRemove</tt>. Assumes en\n     * empty list if no value exists at <tt>key</tt>.\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            a list of values to add to a list\n     * @param toRemove\n     *            a list of values to remove from a list\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws NotAListException\n     *             if the previously stored value was no list\n     * @throws AbortException\n     *             if the commit failed (if there was one)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.9\n     */\n    public void addDelOnList(final OtpErlangString key,\n            final OtpErlangList toAdd, final OtpErlangList toRemove)\n            throws ConnectionException, NotAListException, AbortException,\n            UnknownException {\n        final ResL result = req_list(new AddDelOnListOp(key, toAdd, toRemove));\n        result.processAddDelOnListAt(0);\n    }\n\n    /**\n     * Changes the list stored at the given key, i.e. first adds all items in\n     * <tt>toAdd</tt> then removes all items in <tt>toRemove</tt>. Assumes en\n     * empty list if no value exists at <tt>key</tt>.\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            a list of values to add to a list\n     * @param toRemove\n     *            a list of values to remove from a list\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws NotAListException\n     *             if the previously stored value was no list\n     * @throws AbortException\n     *             if the commit failed (if there was one)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @see #addDelOnList(OtpErlangString, OtpErlangList, OtpErlangList)\n     * @since 3.9\n     */\n    public <T> void addDelOnList(final String key, final List<T> toAdd,\n            final List<T> toRemove) throws ConnectionException,\n            NotAListException, AbortException, UnknownException {\n        OtpErlangList toAddErl;\n        OtpErlangList toRemoveErl;\n        try {\n            toAddErl = (OtpErlangList) ErlangValue.convertToErlang(toAdd);\n            toRemoveErl = (OtpErlangList) ErlangValue.convertToErlang(toRemove);\n        } catch (final ClassCastException e) {\n            // one of the parameters was no list\n            // note: a ClassCastException inside ErlangValue.convertToErlang is\n            // converted to an UnknownException\n            throw new NotAListException(e);\n        }\n        addDelOnList(new OtpErlangString(key), toAddErl, toRemoveErl);\n    }\n\n    /**\n     * Changes the number stored at the given key, i.e. adds some value. Assumes\n     * <tt>0</tt> if no value exists at <tt>key</tt>.\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            the number to add to the number stored at key (may also be\n     *            negative)\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws NotANumberException\n     *             if the previously stored value was no number\n     * @throws AbortException\n     *             if the commit failed (if there was one)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @see #addOnNr_(AddOnNrOp)\n     * @since 3.9\n     */\n    public void addOnNr(final OtpErlangString key, final OtpErlangLong toAdd)\n            throws ConnectionException, NotANumberException, AbortException,\n            UnknownException {\n        addOnNr_(new AddOnNrOp(key, toAdd));\n    }\n\n    /**\n     * Changes the number stored at the given key, i.e. adds some value. Assumes\n     * <tt>0</tt> if no value exists at <tt>key</tt>.\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            the number to add to the number stored at key (may also be\n     *            negative)\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws NotANumberException\n     *             if the previously stored value was no number\n     * @throws AbortException\n     *             if the commit failed (if there was one)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @see #addOnNr_(AddOnNrOp)\n     * @since 3.9\n     */\n    public void addOnNr(final OtpErlangString key, final OtpErlangDouble toAdd)\n            throws ConnectionException, NotANumberException, AbortException,\n            UnknownException {\n        addOnNr_(new AddOnNrOp(key, toAdd));\n    }\n\n    /**\n     * Changes the number stored at the given key, i.e. adds some value. Assumes\n     * <tt>0</tt> if no value exists at <tt>key</tt>.\n     *\n     * @param op\n     *            the add_on_nr operation\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws NotANumberException\n     *             if the previously stored value was no number\n     * @throws AbortException\n     *             if the commit failed (if there was one)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @see #addOnNr(OtpErlangString, OtpErlangLong)\n     * @see #addOnNr(OtpErlangString, OtpErlangDouble)\n     * @since 3.9\n     */\n    protected void addOnNr_(final AddOnNrOp op) throws ConnectionException,\n            NotANumberException, AbortException, UnknownException {\n        final ResL result = req_list(op);\n        result.processAddOnNrAt(0);\n    }\n\n    /**\n     * Changes the number stored at the given key, i.e. adds some value. Assumes\n     * <tt>0</tt> if no value exists at <tt>key</tt>.\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            the number to add to the number stored at key (may also be\n     *            negative)\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws NotANumberException\n     *             if the previously stored value was no number\n     * @throws AbortException\n     *             if the commit failed (if there was one)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @see #addOnNr(OtpErlangString, OtpErlangLong)\n     * @see #addOnNr(OtpErlangString, OtpErlangDouble)\n     * @since 3.9\n     */\n    public <T> void addOnNr(final String key, final T toAdd)\n            throws ConnectionException, NotANumberException, AbortException,\n            UnknownException {\n        final OtpErlangObject toAddErl = ErlangValue.convertToErlang(toAdd);\n        if (toAddErl instanceof OtpErlangLong) {\n            addOnNr(new OtpErlangString(key), (OtpErlangLong) toAddErl);\n        } else if (toAddErl instanceof OtpErlangDouble) {\n            addOnNr(new OtpErlangString(key), (OtpErlangDouble) toAddErl);\n        } else {\n            throw new NotANumberException(toAddErl);\n        }\n    }\n\n    /**\n     * Stores the given <tt>key</tt>/<tt>new_value</tt> pair if the old value at\n     * <tt>key</tt> is <tt>old_value</tt> (atomic test_and_set).\n     *\n     * @param key\n     *            the key to store the value for\n     * @param oldValue\n     *            the old value to check\n     * @param newValue\n     *            the value to store\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws NotFoundException\n     *             if the requested key does not exist\n     * @throws KeyChangedException\n     *             if the key did not match <tt>old_value</tt>\n     * @throws AbortException\n     *             if the commit failed (if there was one)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.9\n     */\n    public void testAndSet(final OtpErlangString key,\n            final OtpErlangObject oldValue, final OtpErlangObject newValue)\n            throws ConnectionException, NotFoundException, KeyChangedException,\n            AbortException, UnknownException {\n        final ResL result = req_list(new TestAndSetOp(key, oldValue, newValue));\n        result.processTestAndSetAt(0);\n    }\n\n    /**\n     * Stores the given <tt>key</tt>/<tt>new_value</tt> pair if the old value at\n     * <tt>key</tt> is <tt>old_value</tt> (atomic test_and_set).\n     *\n     * @param key\n     *            the key to store the value for\n     * @param oldValue\n     *            the old value to check\n     * @param newValue\n     *            the value to store\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws NotFoundException\n     *             if the requested key does not exist\n     * @throws KeyChangedException\n     *             if the key did not match <tt>old_value</tt>\n     * @throws AbortException\n     *             if the commit failed (if there was one)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @see #testAndSet(OtpErlangString, OtpErlangObject, OtpErlangObject)\n     * @since 3.9\n     */\n    public <OldT, NewT> void testAndSet(final String key, final OldT oldValue,\n            final NewT newValue) throws ConnectionException, NotFoundException,\n            KeyChangedException, AbortException, UnknownException {\n        testAndSet(new OtpErlangString(key),\n                ErlangValue.convertToErlang(oldValue),\n                ErlangValue.convertToErlang(newValue));\n    }\n\n    /**\n     * Closes the transaction's connection to a scalaris node.\n     *\n     * Note: Subsequent calls to the other methods will throw\n     * {@link ConnectionException}s!\n     */\n    public void closeConnection() {\n        connection.close();\n    }\n\n    /**\n     * Checks whether the transfer of values is compressed or not.\n     *\n     * @return <tt>true</tt> if compressed, otherwise <tt>false</tt>\n     *\n     * @since 3.14\n     */\n    public boolean isCompressed() {\n        return compressed;\n    }\n\n    /**\n     * Sets whether to compress the transfer of values or not.\n     *\n     * @param compressed\n     *            <tt>true</tt> if compressed, otherwise <tt>false</tt>\n     *\n     * @since 3.14\n     */\n    public void setCompressed(final boolean compressed) {\n        this.compressed = compressed;\n    }\n\n}"
  },
  {
    "path": "java-api/src/de/zib/scalaris/Benchmark.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.lang.reflect.Array;\nimport java.lang.reflect.InvocationTargetException;\nimport java.nio.ByteBuffer;\nimport java.nio.charset.CharacterCodingException;\nimport java.nio.charset.Charset;\nimport java.nio.charset.CharsetDecoder;\nimport java.nio.charset.CodingErrorAction;\nimport java.util.ArrayList;\nimport java.util.Arrays;\nimport java.util.List;\nimport java.util.Random;\nimport java.util.Set;\nimport java.util.concurrent.TimeUnit;\n\nimport com.ericsson.otp.erlang.OtpErlangBinary;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\n\nimport de.zib.scalaris.operations.AddDelOnListOp;\nimport de.zib.scalaris.operations.AddOnNrOp;\nimport de.zib.scalaris.operations.ReadOp;\nimport de.zib.scalaris.operations.WriteOp;\n\n/**\n * Provides methods to run benchmarks and print the results.\n *\n * Also provides some default benchmarks.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.6\n * @since 2.0\n */\npublic class Benchmark {\n    /**\n     * The size of a single data item that is send to scalaris.\n     */\n    protected static final int BENCH_DATA_SIZE = 1000;\n    /**\n     * The time when the (whole) benchmark suite was started.\n     *\n     * This is used to create different erlang keys for each run.\n     */\n    protected static final long benchTime = System.currentTimeMillis();\n    /**\n     * Cut 5% off of both ends of the result list.\n     */\n    protected static final int percentToRemove = 5;\n    /**\n     * Number of test runs (accumulates results over all test runs).\n     */\n    protected static final int testRuns = 1;\n\n    /**\n     * UTF-8 charset object.\n     *\n     * StandardCharsets.UTF_8 is only available for Java >= 7\n     */\n    private static final Charset UTF_8 = Charset.forName(\"UTF-8\");\n\n    /**\n     * Default minimal benchmark.\n     *\n     * Tests some strategies for writing key/value pairs to scalaris:\n     * <ol>\n     * <li>writing {@link OtpErlangBinary} objects (random data, size =\n     * {@link #BENCH_DATA_SIZE})</li>\n     * <li>writing {@link String} objects (random data, size =\n     * {@link #BENCH_DATA_SIZE})</li>\n     * </ol>\n     * each with the given number of consecutive operations and parallel\n     * threads per Scalaris node,\n     * <ul>\n     * <li>first using a new {@link Transaction} or {@link TransactionSingleOp}\n     * for each test,</li>\n     * <li>then using a new {@link Transaction} or {@link TransactionSingleOp}\n     * but re-using a single {@link Connection},</li>\n     * <li>and finally re-using a single {@link Transaction} or\n     * {@link TransactionSingleOp} object.</li>\n     * </ul>\n     *\n     * @param operations\n     *            the number of test runs to execute\n     * @param threadsPerNode\n     *            number of threads to spawn for each existing Scalaris node\n     * @param benchmarks\n     *            the benchmarks to run (1-18 or -1 for all benchmarks)\n     */\n    public static void minibench(final int operations, final int threadsPerNode, final Set<Integer> benchmarks) {\n        final ConnectionFactory cf = ConnectionFactory.getInstance();\n        final List<PeerNode> nodes = cf.getNodes();\n        final int parallelRuns = nodes.size() * threadsPerNode;\n        // set a connection policy that goes through the available nodes in a round-robin fashion:\n        cf.setConnectionPolicy(new RoundRobinConnectionPolicy(nodes));\n        System.out.println(\"Number of available nodes: \" + nodes.size());\n        System.out.println(\"-> Using \" + parallelRuns + \" parallel instances per test run...\");\n        System.out.flush();\n        long[][] results;\n        String[] columns;\n        String[] rows;\n        @SuppressWarnings(\"rawtypes\")\n        Class[] testTypes;\n        String[] testTypesStr;\n        @SuppressWarnings(\"rawtypes\")\n        Class[] testBench;\n        String testGroup;\n\n        System.out.println(\"Benchmark of de.zib.scalaris.TransactionSingleOp:\");\n        System.out.flush();\n        results = getResultArray(3, 2);\n        testTypes = new Class[] {OtpErlangBinary.class, String.class};\n        testTypesStr = new String[] {\"OEB\", \"S\"};\n        columns = new String[] {\n                \"TransactionSingleOp.write(OtpErlangString, OtpErlangBinary)\",\n                \"TransactionSingleOp.write(String, String)\" };\n        testBench = new Class[] {TransSingleOpBench1.class, TransSingleOpBench2.class, TransSingleOpBench3.class};\n        rows = new String[] {\n                \"separate connection\",\n                \"re-use connection\",\n                \"re-use object\" };\n        testGroup = \"transsinglebench\";\n        runBenchAndPrintResults(benchmarks, results, columns, rows, testTypes,\n                testTypesStr, testBench, testGroup, 1, operations, parallelRuns);\n\n        System.out.println(\"-----\");\n        System.out.println(\"Benchmark of de.zib.scalaris.Transaction:\");\n        System.out.flush();\n        results = getResultArray(3, 2);\n        testTypes = new Class[] {OtpErlangBinary.class, String.class};\n        testTypesStr = new String[] {\"OEB\", \"S\"};\n        columns = new String[] {\n                \"Transaction.write(OtpErlangString, OtpErlangBinary)\",\n                \"Transaction.write(String, String)\" };\n        testBench = new Class[] {TransBench1.class, TransBench2.class, TransBench3.class};\n        rows = new String[] {\n                \"separate connection\",\n                \"re-use connection\",\n                \"re-use object\" };\n        testGroup = \"transbench\";\n        runBenchAndPrintResults(benchmarks, results, columns, rows, testTypes,\n                testTypesStr, testBench, testGroup, 1, operations, parallelRuns);\n\n        System.out.println(\"-----\");\n        System.out.println(\"Benchmark incrementing an integer key (read+write):\");\n        System.out.flush();\n        results = getResultArray(3, 1);\n        testTypes = new Class[] {null};\n        testTypesStr = new String[] {\"null\"};\n        columns = new String[] {\n                \"Transaction.addOnNr(String, Integer)\" };\n        testBench = new Class[] {TransIncrementBench1.class, TransIncrementBench2.class, TransIncrementBench3.class};\n        rows = new String[] {\n                \"separate connection\",\n                \"re-use connection\",\n                \"re-use object\" };\n        testGroup = \"transbench_inc\";\n        runBenchAndPrintResults(benchmarks, results, columns, rows, testTypes,\n                testTypesStr, testBench, testGroup, 7, operations, parallelRuns);\n\n        System.out.println(\"-----\");\n        System.out.println(\"Benchmark read 5 + write 5:\");\n        System.out.flush();\n        results = getResultArray(3, 2);\n        testTypes = new Class[] {OtpErlangBinary.class, String.class};\n        testTypesStr = new String[] {\"OEB\", \"S\"};\n        columns = new String[] {\n                \"Transaction.read(OtpErlangString) + Transaction.write(OtpErlangString, OtpErlangBinary)\",\n                \"Transaction.read(String) + Transaction.write(String, String)\" };\n        testBench = new Class[] {TransRead5Write5Bench1.class, TransRead5Write5Bench2.class, TransRead5Write5Bench3.class};\n        rows = new String[] {\n                \"separate connection\",\n                \"re-use connection\",\n                \"re-use object\" };\n        testGroup = \"transbench_r5w5\";\n        runBenchAndPrintResults(benchmarks, results, columns, rows, testTypes,\n                testTypesStr, testBench, testGroup, 10, operations,\n                parallelRuns);\n\n        System.out.println(\"-----\");\n        System.out.println(\"Benchmark appending to a String list (read+write):\");\n        System.out.flush();\n        results = getResultArray(3, 1);\n        testTypes = new Class[] {String.class};\n        testTypesStr = new String[] {\"S\"};\n        columns = new String[] {\n                \"Transaction.addDelOnList(String, StringList, [])\" };\n        testBench = new Class[] {TransAppendToListBench1.class, TransAppendToListBench2.class, TransAppendToListBench3.class};\n        rows = new String[] {\n                \"separate connection\",\n                \"re-use connection\",\n                \"re-use object\" };\n        testGroup = \"transbench_append\";\n        runBenchAndPrintResults(benchmarks, results, columns, rows, testTypes,\n                testTypesStr, testBench, testGroup, 16, operations,\n                parallelRuns);\n    }\n\n    /**\n     * Performs a benchmark writing objects using a new TransactionSingleOp\n     * object for each test.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     *\n     * @param <T>\n     *            type to bench on\n     */\n    protected static final class TransSingleOpBench1<T> extends BenchRunnable<T> {\n        public TransSingleOpBench1(final String key, final T value, final int operations) {\n            super(key, value, operations);\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            final TransactionSingleOp transaction = new TransactionSingleOp();\n            if (value instanceof OtpErlangObject) {\n                transaction.write(new OtpErlangString(key + '_' + j), (OtpErlangObject) value);\n            } else {\n                transaction.write(key + '_' + j, value);\n            }\n            transaction.closeConnection();\n        }\n    }\n\n    /**\n     * Performs a benchmark writing objects using a new TransactionSingleOp but\n     * re-using a single connection for each test.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     *\n     * @param <T> type to bench on\n     */\n    protected static final class TransSingleOpBench2<T> extends BenchRunnable2<T> {\n        public TransSingleOpBench2(final String key, final T value, final int operations) {\n            super(key, value, operations);\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            final TransactionSingleOp transaction = new TransactionSingleOp(connection);\n            if (value instanceof OtpErlangObject) {\n                transaction.write(new OtpErlangString(key + '_' + j), (OtpErlangObject) value);\n            } else {\n                transaction.write(key + '_' + j, value);\n            }\n        }\n    }\n\n    /**\n     * Performs a benchmark writing objects using a single TransactionSingleOp\n     * object for all tests.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     *\n     * @param <T> type to bench on\n     */\n    protected static final class TransSingleOpBench3<T> extends BenchRunnable<T> {\n        TransactionSingleOp transaction;\n\n        public TransSingleOpBench3(final String key, final T value, final int operations) {\n            super(key, value, operations);\n        }\n\n        @Override\n        protected void init() throws Exception {\n            transaction = new TransactionSingleOp();\n        }\n\n        @Override\n        protected void cleanup() throws Exception {\n            transaction.closeConnection();\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            if (value instanceof OtpErlangObject) {\n                transaction.write(new OtpErlangString(key + '_' + j), (OtpErlangObject) value);\n            } else {\n                transaction.write(key + '_' + j, value);\n            }\n        }\n    }\n\n    /**\n     * Performs a benchmark writing objects using a new Transaction for each test.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     *\n     * @param <T> type to bench on\n     */\n    protected static final class TransBench1<T> extends BenchRunnable<T> {\n        public TransBench1(final String key, final T value, final int operations) {\n            super(key, value, operations);\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            final Transaction transaction = new Transaction();\n            if (value instanceof OtpErlangObject) {\n                transaction.write(new OtpErlangString(key + '_' + j), (OtpErlangObject) value);\n            } else {\n                transaction.write(key + '_' + j, value);\n            }\n            transaction.commit();\n            transaction.closeConnection();\n        }\n    }\n\n    /**\n     * Performs a benchmark writing objects using a new Transaction but re-using a\n     * single connection for each test.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     *\n     * @param <T> type to bench on\n     */\n    protected static final class TransBench2<T> extends BenchRunnable2<T> {\n        public TransBench2(final String key, final T value, final int operations) {\n            super(key, value, operations);\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            final Transaction transaction = new Transaction(connection);\n            if (value instanceof OtpErlangObject) {\n                transaction.write(new OtpErlangString(key + '_' + j), (OtpErlangObject) value);\n            } else {\n                transaction.write(key + '_' + j, value);\n            }\n            transaction.commit();\n        }\n    }\n\n    /**\n     * Performs a benchmark writing objects using a single Transaction object\n     * for all tests.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     *\n     * @param <T> type to bench on\n     */\n    protected static final class TransBench3<T> extends BenchRunnable<T> {\n        Transaction transaction;\n\n        public TransBench3(final String key, final T value, final int operations) {\n            super(key, value, operations);\n        }\n\n        @Override\n        protected void init() throws Exception {\n            transaction = new Transaction();\n        }\n\n        @Override\n        protected void cleanup() throws Exception {\n            transaction.closeConnection();\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            if (value instanceof OtpErlangObject) {\n                transaction.write(new OtpErlangString(key + '_' + j), (OtpErlangObject) value);\n            } else {\n                transaction.write(key + '_' + j, value);\n            }\n            transaction.commit();\n        }\n    }\n\n    /**\n     * Performs a benchmark writing integer numbers on a single key and\n     * increasing them.\n     *\n     * Provides convenience methods for the full increment benchmark\n     * implementations.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     */\n    protected static abstract class TransIncrementBench extends BenchRunnable<Object> {\n        public TransIncrementBench(final String key, final Object value, final int operations) {\n            super(key, value, operations);\n        }\n\n        @Override\n        protected void pre_init(final Connection conn) throws Exception {\n            final Transaction tx_init = new Transaction(conn);\n            tx_init.write(key, 0);\n            tx_init.commit();\n        }\n\n        protected void operation(final Transaction tx, final int j) throws Exception {\n            final Transaction.RequestList reqs = new Transaction.RequestList();\n            reqs.addOp(new AddOnNrOp(key, 1)).addCommit();\n//            final int value_old = tx.read(key).intValue();\n//            reqs.addWrite(key, value_old + 1).addCommit();\n            final Transaction.ResultList results = tx.req_list(reqs);\n//            results.processWriteAt(0);\n            results.processAddOnNrAt(0);\n        }\n    }\n\n    /**\n     * Performs a benchmark writing integer numbers on a single key and\n     * increasing them using a new Transaction for each test.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     */\n    protected static final class TransIncrementBench1 extends TransIncrementBench {\n        public TransIncrementBench1(final String key, final Object value, final int operations) {\n            super(key, value, operations);\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            final Transaction transaction = new Transaction();\n            operation(transaction, j);\n            transaction.closeConnection();\n        }\n    }\n\n    /**\n     * Performs a benchmark writing integer numbers on a single key and\n     * increasing them using a new Transaction but re-using a single connection\n     * for each test.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     */\n    protected static final class TransIncrementBench2 extends TransIncrementBench {\n        protected Connection connection;\n\n        public TransIncrementBench2(final String key, final Object value, final int operations) {\n            super(key, value, operations);\n        }\n\n        @Override\n        protected void init() throws Exception {\n            connection = ConnectionFactory.getInstance().createConnection();\n        }\n\n        @Override\n        protected void cleanup() throws Exception {\n            connection.close();\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            final Transaction transaction = new Transaction(connection);\n            operation(transaction, j);\n        }\n    }\n\n    /**\n     * Performs a benchmark writing objects using a single Transaction object\n     * for all tests.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     */\n    protected static final class TransIncrementBench3 extends TransIncrementBench {\n        Transaction transaction;\n\n        public TransIncrementBench3(final String key, final Object value, final int operations) {\n            super(key, value, operations);\n        }\n\n        @Override\n        protected void init() throws Exception {\n            transaction = new Transaction();\n        }\n\n        @Override\n        protected void cleanup() throws Exception {\n            transaction.closeConnection();\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            operation(transaction, j);\n        }\n    }\n\n    /**\n     * Performs a benchmark reading X values and overwriting them afterwards\n     * inside a transaction. Provides convenience methods for the full read-x,\n     * write-x benchmark implementations.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     *\n     * @param <T> type to bench on\n     */\n    protected static abstract class TransReadXWriteXBench<T> extends BenchRunnable<T> {\n        private final String[] keys;\n        private final T[] valueWrite;\n\n        @SuppressWarnings(\"unchecked\")\n        public TransReadXWriteXBench(final String key, final T value, final int nrKeys, final int operations)\n                throws IllegalArgumentException, SecurityException,\n                InstantiationException, IllegalAccessException,\n                InvocationTargetException, NoSuchMethodException {\n            super(key, value, operations);\n            keys = new String[nrKeys];\n            valueWrite = (T[]) Array.newInstance(value.getClass(), nrKeys);\n            for (int i = 0; i < keys.length; ++i) {\n                keys[i] = key + \"_\" + i;\n                valueWrite[i] = (T) getRandom(BENCH_DATA_SIZE, value.getClass());\n            }\n        }\n\n        @SuppressWarnings(\"unchecked\")\n        @Override\n        protected void pre_init(final Connection conn) throws Exception {\n            final T[] valueInit = (T[]) Array.newInstance(value.getClass(), keys.length);\n            for (int i = 0; i < keys.length; ++i) {\n                valueInit[i] = (T) getRandom(BENCH_DATA_SIZE, value.getClass());\n            }\n            final Transaction tx_init = new Transaction(conn);\n            final Transaction.RequestList reqs = new Transaction.RequestList();\n            for (int i = 0; i < keys.length; ++i) {\n                reqs.addOp(new WriteOp(keys[i], valueInit[i]));\n            }\n            reqs.addCommit();\n            final Transaction.ResultList results = tx_init.req_list(reqs);\n            for (int i = 0; i < keys.length; ++i) {\n                results.processWriteAt(i);\n            }\n        }\n\n        protected void operation(final Transaction tx, final int j) throws Exception {\n            Transaction.RequestList reqs;\n            Transaction.ResultList results;\n            reqs = new Transaction.RequestList();\n            // read old values into the transaction\n            for (final String key : keys) {\n                if (value instanceof OtpErlangObject) {\n                    reqs.addOp(new ReadOp(new OtpErlangString(key)));\n                } else {\n                    reqs.addOp(new ReadOp(key));\n                }\n            }\n            reqs.addCommit();\n            results = tx.req_list(reqs);\n            // check results:\n            for (int i = 0; i < keys.length; ++i) {\n                if (value instanceof OtpErlangObject) {\n                    results.processReadAt(i); // no need to extract the value itself\n                } else {\n                    results.processReadAt(i).stringValue();\n                }\n            }\n\n            // write new values...\n            reqs = new Transaction.RequestList();\n            for (final String key : keys) {\n                final T value = valueWrite[j % valueWrite.length];\n                if (value instanceof OtpErlangObject) {\n                    reqs.addOp(new WriteOp(new OtpErlangString(key), (OtpErlangObject) value));\n                } else {\n                    reqs.addOp(new WriteOp(key, value));\n                }\n            }\n            reqs.addCommit();\n            results = tx.req_list(reqs);\n            for (int i = 0; i < keys.length; ++i) {\n                results.processWriteAt(i);\n            }\n        }\n    }\n\n    /**\n     * Performs a benchmark reading 5 values and overwriting them afterwards\n     * inside a transaction using a new Transaction for each test.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     *\n     * @param <T> type to bench on\n     */\n    protected static final class TransRead5Write5Bench1<T> extends TransReadXWriteXBench<T> {\n        public TransRead5Write5Bench1(final String key, final T value, final int operations)\n                throws IllegalArgumentException, SecurityException,\n                InstantiationException, IllegalAccessException,\n                InvocationTargetException, NoSuchMethodException {\n            super(key, value, 5, operations);\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            final Transaction tx = new Transaction();\n            operation(tx, j);\n            tx.closeConnection();\n        }\n    }\n\n    /**\n     * Performs a benchmark reading 5 values and overwriting them afterwards\n     * inside a transaction using a new Transaction but re-using a single\n     * connection for each test.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     *\n     * @param <T> type to bench on\n     */\n    protected static final class TransRead5Write5Bench2<T> extends TransReadXWriteXBench<T> {\n        protected Connection connection;\n\n        public TransRead5Write5Bench2(final String key, final T value, final int operations)\n                throws IllegalArgumentException, SecurityException,\n                InstantiationException, IllegalAccessException,\n                InvocationTargetException, NoSuchMethodException {\n            super(key, value, 5, operations);\n        }\n\n        @Override\n        protected void init() throws Exception {\n            connection = ConnectionFactory.getInstance().createConnection();\n        }\n\n        @Override\n        protected void cleanup() throws Exception {\n            connection.close();\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            final Transaction tx = new Transaction(connection);\n            operation(tx, j);\n        }\n    }\n\n    /**\n     * Performs a benchmark reading 5 values and overwriting them afterwards\n     * inside a transaction using a single Transaction object for all tests.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     *\n     * @param <T> type to bench on\n     */\n    protected static final class TransRead5Write5Bench3<T> extends TransReadXWriteXBench<T> {\n        Transaction transaction;\n\n        public TransRead5Write5Bench3(final String key, final T value, final int operations)\n                throws IllegalArgumentException, SecurityException,\n                InstantiationException, IllegalAccessException,\n                InvocationTargetException, NoSuchMethodException {\n            super(key, value, 5, operations);\n        }\n\n        @Override\n        protected void init() throws Exception {\n            transaction = new Transaction();\n        }\n\n        @Override\n        protected void cleanup() throws Exception {\n            transaction.closeConnection();\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            operation(transaction, j);\n        }\n    }\n\n    /**\n     * Performs a benchmark adding values to a list inside a transaction.\n     *\n     * Provides convenience methods for the full append-to-list benchmark\n     * implementations.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     */\n    protected static abstract class TransAppendToListBench extends BenchRunnable<String> {\n        private final List<String> valueInit;\n\n        public TransAppendToListBench(final String key, final String value, final int nrKeys, final int operations)\n                throws IllegalArgumentException, SecurityException,\n                InstantiationException, IllegalAccessException,\n                InvocationTargetException, NoSuchMethodException {\n            super(key, value, operations);\n            valueInit = new ArrayList<String>(nrKeys);\n            for (int i = 0; i < nrKeys; ++i) {\n                valueInit.add(getRandom(BENCH_DATA_SIZE, String.class));\n            }\n        }\n\n        @Override\n        protected void pre_init(final Connection conn, final int j) throws Exception {\n            final Transaction tx_init = new Transaction(conn);\n            final Transaction.RequestList reqs = new Transaction.RequestList();\n            reqs.addOp(new WriteOp(key + '_' + j, valueInit)).addCommit();\n            final Transaction.ResultList results = tx_init.req_list(reqs);\n            results.processWriteAt(0);\n        }\n\n        protected void operation(final Transaction tx, final int j) throws Exception {\n            final Transaction.RequestList reqs = new Transaction.RequestList();\n            reqs.addOp(new AddDelOnListOp(key + '_' + j, Arrays.asList(value), new ArrayList<String>(0)));\n            reqs.addCommit();\n//            // read old list into the transaction\n//            final List<String> list = tx.read(key + '_' + j).stringListValue();\n//\n//            // write new list ...\n//            list.add(value);\n//            reqs.addWrite(key + '_' + j, list).addCommit();\n            final Transaction.ResultList results = tx.req_list(reqs);\n//            results.processWriteAt(0);\n            results.processAddDelOnListAt(0);\n        }\n    }\n\n    /**\n     * Performs a benchmark adding values to a list inside a transaction using a\n     * new Transaction for each test.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     */\n    protected static final class TransAppendToListBench1 extends TransAppendToListBench {\n        public TransAppendToListBench1(final String key, final String value, final int operations)\n                throws IllegalArgumentException, SecurityException,\n                InstantiationException, IllegalAccessException,\n                InvocationTargetException, NoSuchMethodException {\n            super(key, value, 5, operations);\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            final Transaction tx = new Transaction();\n            operation(tx, j);\n            tx.closeConnection();\n        }\n    }\n\n    /**\n     * Performs a benchmark adding values to a list inside a transaction using a\n     * new Transaction but re-using a single connection for each test.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     */\n    protected static final class TransAppendToListBench2 extends TransAppendToListBench {\n        protected Connection connection;\n\n        public TransAppendToListBench2(final String key, final String value, final int operations)\n                throws IllegalArgumentException, SecurityException,\n                InstantiationException, IllegalAccessException,\n                InvocationTargetException, NoSuchMethodException {\n            super(key, value, 5, operations);\n        }\n\n        @Override\n        protected void init() throws Exception {\n            connection = ConnectionFactory.getInstance().createConnection();\n        }\n\n        @Override\n        protected void cleanup() throws Exception {\n            connection.close();\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            final Transaction tx = new Transaction(connection);\n            operation(tx, j);\n        }\n    }\n\n    /**\n     * Performs a benchmark adding values to a list inside a transaction using a\n     * single Transaction object for all tests.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     */\n    protected static final class TransAppendToListBench3 extends TransAppendToListBench {\n        Transaction transaction;\n\n        public TransAppendToListBench3(final String key, final String value, final int operations)\n                throws IllegalArgumentException, SecurityException,\n                InstantiationException, IllegalAccessException,\n                InvocationTargetException, NoSuchMethodException {\n            super(key, value, 5, operations);\n        }\n\n        @Override\n        protected void init() throws Exception {\n            transaction = new Transaction();\n        }\n\n        @Override\n        protected void cleanup() throws Exception {\n            transaction.closeConnection();\n        }\n\n        @Override\n        protected void operation(final int j) throws Exception {\n            operation(transaction, j);\n        }\n    }\n\n    /**\n     * Abstract base class of a test run that is to be run in a thread.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @since 3.6\n     *\n     * @param <T> type of the value to write\n     */\n    protected abstract static class BenchRunnable<T> extends Thread {\n        /**\n         * Tells the thread to stop.\n         */\n        public boolean stop = false;\n\n        /**\n         * The time at the start of a single benchmark.\n         */\n        private long timeAtStart = 0;\n        /**\n         * The speed of the benchmark in operations/s.\n         */\n        private long speed = -1;\n\n        /**\n         * The key to operate on.\n         */\n        protected final String key;\n        /**\n         * The value to use.\n         */\n        protected final T value;\n\n        /**\n         * Number of operations to run.\n         */\n        protected int operations;\n\n        /**\n         * Creates a new runnable.\n         *\n         * @param key\n         *            the key to operate on\n         * @param value\n         *            the value to use\n         * @param operations\n         *            number of operations to run\n         */\n        public BenchRunnable(final String key, final T value, final int operations) {\n            this.key = key;\n            this.value = value;\n            this.operations = operations;\n        }\n\n        /**\n         * Call this method when a benchmark is started.\n         *\n         * Sets the time the benchmark was started.\n         */\n        final private void testBegin() {\n            timeAtStart = System.currentTimeMillis();\n        }\n\n        /**\n         * Call this method when a benchmark is finished.\n         *\n         * Calculates the time the benchmark took and the number of transactions\n         * performed during this time.\n         */\n        final private long testEnd(final int testRuns) {\n            final long timeTaken = System.currentTimeMillis() - timeAtStart;\n            final long speed = (testRuns * 1000) / timeTaken;\n            return speed;\n        }\n\n        /**\n         * Will be called before the benchmark starts.\n         *\n         * @param conn\n         *            connection to use\n         *\n         * @throws Exception\n         */\n        protected void pre_init(final Connection conn) throws Exception {\n        }\n\n        /**\n         * Will be called before the benchmark starts with all possible\n         * variations of \"j\" in the {@link #operation(int)} call.\n         *\n         * @param conn\n         *            connection to use\n         * @param j\n         *            the index {@link #operation(int)} will be called with\n         *\n         * @throws Exception\n         */\n        protected void pre_init(final Connection conn, final int j) throws Exception {\n        }\n\n        /**\n         * Will be called at the start of the benchmark.\n         *\n         * @throws Exception\n         */\n        protected void init() throws Exception {\n        }\n\n        /**\n         * Will be called before the end of the benchmark.\n         *\n         * @throws Exception\n         */\n        protected void cleanup() throws Exception {\n        }\n\n        /**\n         * The operation to execute during the benchmark.\n         *\n         * @param j\n         *            transaction number\n         *\n         * @throws Exception\n         */\n        abstract protected void operation(int j) throws Exception;\n\n        @Override\n        final public void run() {\n            Thread.currentThread().setName(\"BenchRunnable-\" + key);\n            Connection conn = null;\n            try {\n                conn = ConnectionFactory.getInstance().createConnection();\n                for (int retry = 0; (retry < 3) && !stop; ++retry) {\n                    try {\n                        pre_init(conn);\n                        for (int j = 0; j < operations; ++j) {\n                            pre_init(conn, j);\n                        }\n                        testBegin();\n                        init();\n                        for (int j = 0; j < operations; ++j) {\n                            operation(j);\n                        }\n                        cleanup();\n                        this.speed = testEnd(operations);\n                        break;\n                    } catch (final Exception e) {\n                        // e.printStackTrace();\n                    }\n                }\n            } catch (final Exception e) {\n                // e.printStackTrace();\n            } finally {\n                if (conn != null) {\n                    conn.close();\n                }\n            }\n        }\n\n        /**\n         * @return the speed\n         */\n        public long getSpeed() {\n            return speed;\n        }\n    }\n\n    protected static abstract class BenchRunnable2<T> extends BenchRunnable<T> {\n        protected Connection connection;\n\n        protected BenchRunnable2(final String key, final T value, final int operations) {\n            super(key, value, operations);\n        }\n\n        @Override\n        protected void init() throws Exception {\n            connection = ConnectionFactory.getInstance().createConnection();\n        }\n\n        @Override\n        protected void cleanup() throws Exception {\n            connection.close();\n        }\n    }\n\n    /**\n     * Returns a pre-initialized results array with values <tt>-1</tt>.\n     *\n     * @param rows\n     *            number of rows to create\n     * @param columns\n     *            number of columns to create\n     *\n     * @return the 2d results array\n     */\n    protected static long[][] getResultArray(final int rows, final int columns) {\n        final long[][] results = new long[rows][columns];\n        for (int i = 0; i < rows; ++i) {\n            for (int j = 0; j < columns; ++j) {\n                results[i][j] = -1;\n            }\n        }\n        return results;\n    }\n\n    /**\n     * Creates an object T from <tt>size</tt> random bytes. Uses either a\n     * constructor that expects a <tt>byte[]</tt> or a {@link String} parameter\n     * (in which case the UTF-8 encoding is used for the bytes).\n     *\n     * @param <T>\n     *            the type of the object to create\n     * @param size\n     *            the number of (random) bytes to create\n     * @param c\n     *            the class of the object to create (needed due to type erasure)\n     *\n     * @return the created object\n     *\n     * @throws IllegalAccessException\n     *             - if this Constructor object enforces Java language access\n     *             control and the underlying constructor is inaccessible.\n     * @throws IllegalArgumentException\n     *             - if the number of actual and formal parameters differ; if an\n     *             unwrapping conversion for primitive arguments fails; or if,\n     *             after possible unwrapping, a parameter value cannot be\n     *             converted to the corresponding formal parameter type by a\n     *             method invocation conversion; if this constructor pertains to\n     *             an enum type.\n     * @throws InstantiationException\n     *             - if the class that declares the underlying constructor\n     *             represents an abstract class.\n     * @throws InvocationTargetException\n     *             - if the underlying constructor throws an exception.\n     * @throws ExceptionInInitializerError\n     *             - if the initialization provoked by this method fails.\n     * @throws NoSuchMethodException\n     *             - if a matching method is not found.\n     * @throws SecurityException\n     *             - If a security manager, s, is present and any of the\n     *             following conditions is met: invocation of\n     *             s.checkMemberAccess(this, Member.PUBLIC) denies access to the\n     *             constructor the caller's class loader is not the same as or\n     *             an ancestor of the class loader for the current class and\n     *             invocation of s.checkPackageAccess() denies access to the\n     *             package of this class\n     */\n    public static <T> T getRandom(final int size, final Class<T> c)\n            throws IllegalArgumentException, SecurityException,\n            InstantiationException, IllegalAccessException,\n            InvocationTargetException, NoSuchMethodException {\n        return getRandom(size, c, new Random());\n    }\n\n    /**\n     * Creates an object T from <tt>size</tt> random bytes. Uses either a\n     * constructor that expects a <tt>byte[]</tt> or a {@link String} parameter\n     * (in which case the UTF-8 encoding is used for the bytes).\n     *\n     * @param <T>\n     *            the type of the object to create\n     * @param size\n     *            the number of (random) bytes to create\n     * @param c\n     *            the class of the object to create (needed due to type erasure)\n     * @param r\n     *            the random number generator to use\n     *\n     * @return the created object\n     *\n     * @throws IllegalAccessException\n     *             - if this Constructor object enforces Java language access\n     *             control and the underlying constructor is inaccessible.\n     * @throws IllegalArgumentException\n     *             - if the number of actual and formal parameters differ; if an\n     *             unwrapping conversion for primitive arguments fails; or if,\n     *             after possible unwrapping, a parameter value cannot be\n     *             converted to the corresponding formal parameter type by a\n     *             method invocation conversion; if this constructor pertains to\n     *             an enum type.\n     * @throws InstantiationException\n     *             - if the class that declares the underlying constructor\n     *             represents an abstract class.\n     * @throws InvocationTargetException\n     *             - if the underlying constructor throws an exception.\n     * @throws ExceptionInInitializerError\n     *             - if the initialization provoked by this method fails.\n     * @throws NoSuchMethodException\n     *             - if a matching method is not found.\n     * @throws SecurityException\n     *             - If a security manager, s, is present and any of the\n     *             following conditions is met: invocation of\n     *             s.checkMemberAccess(this, Member.PUBLIC) denies access to the\n     *             constructor the caller's class loader is not the same as or\n     *             an ancestor of the class loader for the current class and\n     *             invocation of s.checkPackageAccess() denies access to the\n     *             package of this class\n     */\n    public static <T> T getRandom(final int size, final Class<T> c, final Random r)\n            throws IllegalArgumentException, SecurityException,\n            InstantiationException, IllegalAccessException,\n            InvocationTargetException, NoSuchMethodException {\n        if (c == null) {\n            return null;\n        }\n        if (Integer.class.isAssignableFrom(c)) {\n            final int par = r.nextInt();\n            return c.getConstructor(int.class).newInstance(par);\n        }\n\n        final byte[] data = new byte[size];\n        r.nextBytes(data);\n\n        try {\n            if (!String.class.isAssignableFrom(c)) {\n                return c.getConstructor(byte[].class).newInstance(data);\n            }\n        } catch (final NoSuchMethodException e) {\n            // use String constructor below\n        }\n\n        final CharsetDecoder decoder = UTF_8.newDecoder();\n        decoder.onMalformedInput(CodingErrorAction.REPLACE);\n        decoder.onUnmappableCharacter(CodingErrorAction.REPLACE);\n        String par;\n        try {\n            par = decoder.decode(ByteBuffer.wrap(data)).toString();\n            return c.getConstructor(String.class).newInstance(par);\n        } catch (final CharacterCodingException e) {\n            throw new IllegalArgumentException(e);\n        }\n    }\n\n    /**\n     * Integrates the workers' results into the result array.\n     *\n     * @param <T>\n     *            type of the value\n     * @param results\n     *            results array with operations/s\n     * @param i\n     *            current number of the test run\n     * @param worker\n     *            array of worker threads\n     * @param failed\n     *            number of previously failed threads\n     *\n     * @return (new) number of failed threads\n     */\n    private static <T> int integrateResults(final long[] results, final int i,\n            final BenchRunnable<T>[] worker, int failed) {\n        for (final BenchRunnable<T> benchThread : worker) {\n            if (failed >= 3) {\n                benchThread.stop = true;\n                try {\n                    benchThread.join();\n                } catch (final InterruptedException e) {\n                }\n            } else {\n                long speed;\n                try {\n                    benchThread.join();\n                    speed = benchThread.getSpeed();\n                } catch (final InterruptedException e) {\n                    speed = -1;\n                }\n                if (speed < 0) {\n                    ++failed;\n                } else {\n                    results[i] += speed;\n                }\n            }\n        }\n        return failed;\n    }\n\n    /**\n     * Calculates the average number of transactions per second from the results\n     * of executing 10 transactions per test run. Will remove the top and bottom\n     * {@link #percentToRemove} percent of the sorted results array.\n     *\n     * @param results\n     *            the average number of transactions per second using 10\n     *            transactions\n     *\n     * @return the average number of transactions per second\n     */\n    protected static long getAvgSpeed(final long[] results) {\n        Arrays.sort(results);\n        final int toRemove = (results.length * percentToRemove) / 100;\n        long avgSpeed = 0;\n        for (int i = toRemove; i < (results.length - toRemove); ++i) {\n            avgSpeed += results[i];\n        }\n        avgSpeed /= results.length - (2 * toRemove);\n        return avgSpeed;\n    }\n\n    @SuppressWarnings(\"unchecked\")\n    protected static void runBenchAndPrintResults(\n            final Set<Integer> benchmarks, final long[][] results,\n            final String[] columns, final String[] rows,\n            @SuppressWarnings(\"rawtypes\") final Class[] testTypes,\n            final String[] testTypesStr,\n            @SuppressWarnings(\"rawtypes\") final Class[] testBench,\n            final String testGroup, final int firstBenchId,\n            final int operations, final int parallelRuns) {\n        for (int test = 0; test < (results.length * results[0].length); ++test) {\n            try {\n                final int i = test % results.length;\n                final int j = test / results.length;\n                if (benchmarks.contains(test + firstBenchId)) {\n                    results[i][j] = runBench(operations,\n                            getRandom(BENCH_DATA_SIZE, testTypes[j]),\n                            testGroup + \"_\" + testTypesStr[j] + \"_\" + (i + 1),\n                            testBench[i], parallelRuns);\n                    TimeUnit.SECONDS.sleep(1);\n                } else {\n                    results[i][j] = -2;\n                }\n            } catch (final Exception e) {\n                 e.printStackTrace();\n            }\n        }\n        printResults(columns, rows, results, operations, parallelRuns);\n    }\n\n    /**\n     * Runs the given benchmark.\n     *\n     * @param <T>\n     *            type of the value to use\n     * @param operations\n     *            the number of operations to execute\n     * @param value\n     *            the value to use\n     * @param name\n     *            the name of the benchmark (will be used as part of the key and\n     *            must therefore be unique)\n     * @param clazz\n     *            bench runnable class\n     * @param parallelRuns\n     *            number of test runs (accumulates results over all test runs)\n     *\n     * @return the number of achieved transactions per second\n     * @throws NoSuchMethodException\n     * @throws InvocationTargetException\n     * @throws IllegalAccessException\n     * @throws InstantiationException\n     * @throws SecurityException\n     * @throws IllegalArgumentException\n     */\n    @SuppressWarnings(\"unchecked\")\n    protected static <T> long runBench(\n            final int operations,\n            final T value,\n            final String name,\n            @SuppressWarnings(\"rawtypes\") final Class<? extends BenchRunnable> clazz,\n            final int parallelRuns) throws IllegalArgumentException,\n            SecurityException, InstantiationException, IllegalAccessException,\n            InvocationTargetException, NoSuchMethodException {\n        final String key = benchTime + name;\n        final long[] results = new long[testRuns];\n        Arrays.fill(results, -1);\n\n        for (int i = 0; i < testRuns; ++i) {\n            final BenchRunnable<T> worker[] = new BenchRunnable[parallelRuns];\n            for (int thread = 0; thread < parallelRuns; ++thread) {\n                final Class<? extends Object> valueClazz = (value == null) ? Object.class : value.getClass();\n                try {\n                    worker[thread] = clazz.getConstructor(String.class, valueClazz, int.class)\n                            .newInstance(key + '_' + i + '_' + thread, value, operations);\n                } catch (final NoSuchMethodException e) {\n                    worker[thread] = clazz.getConstructor(String.class, Object.class, int.class)\n                            .newInstance(key + '_' + i + '_' + thread, value, operations);\n                }\n                worker[thread].start();\n            }\n            int failed = 0;\n            failed = integrateResults(results, i, worker, failed);\n            if (failed >= 3) {\n                return -1;\n            }\n        }\n\n        return getAvgSpeed(results);\n    }\n\n    /**\n     * Prints a result table.\n     *\n     * @param columns\n     *            names of the columns\n     * @param rows\n     *            names of the rows (max 25 chars to protect the layout)\n     * @param results\n     *            result array\n     * @param operations\n     *            number of operations to execute\n     * @param parallelRuns\n     *            number of parallel threads\n     *\n     * @param results\n     *            the results to print (results[i][j]: i = row, j = column)\n     */\n    protected static void printResults(final String[] columns, final String[] rows,\n            final long[][] results, final int operations, final int parallelRuns) {\n        System.out.println(\"Concurrent threads: \" + parallelRuns + \", each using \" + operations + \" transactions\");\n        System.out\n                .println(\"                         \\tspeed (transactions / second)\");\n        final String firstColumn = \"                         \";\n        System.out.print(firstColumn);\n        for (int i = 0; i < columns.length; ++i) {\n            System.out.print(\"\\t(\" + (i + 1) + \")\");\n        }\n        System.out.println();\n\n        for (int i = 0; i < rows.length; ++i) {\n            System.out.print(rows[i]\n                    + firstColumn.substring(0,\n                            firstColumn.length() - rows[i].length() - 1));\n            for (int j = 0; j < columns.length; j++) {\n                if (results[i][j] == -2) {\n                    System.out.print(\"\\tn/a\");\n                } else if (results[i][j] == -1) {\n                    System.out.print(\"\\tfailed\");\n                } else {\n                    System.out.print(\"\\t\" + results[i][j]);\n                }\n            }\n            System.out.println();\n        }\n\n        for (int i = 0; i < columns.length; i++) {\n            System.out.println(\"(\" + (i + 1) + \") \" + columns[i]);\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/CommonErlangObjects.java",
    "content": "/**\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.io.IOException;\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpErlangAtom;\nimport com.ericsson.otp.erlang.OtpErlangBinary;\nimport com.ericsson.otp.erlang.OtpErlangBoolean;\nimport com.ericsson.otp.erlang.OtpErlangDecodeException;\nimport com.ericsson.otp.erlang.OtpErlangDouble;\nimport com.ericsson.otp.erlang.OtpErlangLong;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\nimport com.ericsson.otp.erlang.OtpExternal;\nimport com.ericsson.otp.erlang.OtpInputStream;\nimport com.ericsson.otp.erlang.OtpOutputStream;\n\n/**\n * Contains some often used objects as static objects as static members in\n * order to avoid re-creating them each time they are needed.\n *\n * @author Nico Kruber, kruber@zib.de\n *\n * @version 3.14\n * @since 2.5\n */\n@SuppressWarnings(\"javadoc\")\npublic final class CommonErlangObjects {\n    public static final OtpErlangAtom readAtom = new OtpErlangAtom(\"read\");\n    public static final OtpErlangAtom writeAtom = new OtpErlangAtom(\"write\");\n    public static final OtpErlangAtom addDelOnListAtom = new OtpErlangAtom(\"add_del_on_list\");\n    public static final OtpErlangAtom testAndSetAtom = new OtpErlangAtom(\"test_and_set\");\n    public static final OtpErlangAtom addOnNrAtom = new OtpErlangAtom(\"add_on_nr\");\n    public static final OtpErlangAtom okAtom = new OtpErlangAtom(\"ok\");\n    public static final OtpErlangAtom failAtom = new OtpErlangAtom(\"fail\");\n    public static final OtpErlangAtom abortAtom = new OtpErlangAtom(\"abort\");\n    public static final OtpErlangAtom timeoutAtom = new OtpErlangAtom(\"timeout\");\n    public static final OtpErlangAtom notFoundAtom = new OtpErlangAtom(\"not_found\");\n    public static final OtpErlangAtom keyChangedAtom = new OtpErlangAtom(\"key_changed\");\n    public static final OtpErlangAtom notAListAtom = new OtpErlangAtom(\"not_a_list\");\n    public static final OtpErlangAtom notANumberAtom = new OtpErlangAtom(\"not_a_number\");\n    public static final OtpErlangAtom emptyListAtom = new OtpErlangAtom(\"empty_list\");\n    public static final OtpErlangTuple okTupleAtom = new OtpErlangTuple(okAtom);\n    public static final OtpErlangTuple commitTupleAtom = new OtpErlangTuple(new OtpErlangAtom(\"commit\"));\n    public static final OtpErlangAtom sublistAtom = new OtpErlangAtom(\"sublist\");\n    public static final OtpErlangAtom randomFromListAtom = new OtpErlangAtom(\"random_from_list\");\n\n    // JSON\n    public static final OtpErlangAtom structAtom = new OtpErlangAtom(\"struct\");\n    public static final OtpErlangAtom arrayAtom = new OtpErlangAtom(\"array\");\n    public static final OtpErlangAtom trueAtom = new OtpErlangAtom(\"true\");\n    public static final OtpErlangAtom falseAtom = new OtpErlangAtom(\"false\");\n    public static final OtpErlangAtom nullAtom = new OtpErlangAtom(\"null\");\n\n    /**\n     * Encoded the given erlang object to a binary the same way as\n     * <tt>rdht_tx:encode_value/1</tt>.\n     *\n     * @param value\n     *            the decoded value\n     *\n     * @return the encoded value\n     */\n    public static OtpErlangObject encode(final OtpErlangObject value) {\n        if (value instanceof OtpErlangAtom) {\n            return value;\n        } else if (value instanceof OtpErlangBoolean) {\n            return value;\n        } else if (value instanceof OtpErlangLong) {\n            return value;\n        } else if (value instanceof OtpErlangDouble) {\n            return value;\n        } else if (value instanceof OtpErlangBinary) {\n            final OtpOutputStream oos = new OtpOutputStream();\n            oos.write1(OtpExternal.versionTag);\n            oos.write_any(value);\n            final byte[] encoded = oos.toByteArray();\n            final OtpErlangBinary result = new OtpErlangBinary(encoded);\n            try {\n                oos.close();\n            } catch (final IOException e) {\n            }\n            return result;\n        } else {\n            final OtpOutputStream oos = new OtpOutputStream();\n            oos.write1(OtpExternal.versionTag);\n            oos.write_compressed(value);\n            final byte[] encoded = oos.toByteArray();\n            final OtpErlangBinary result = new OtpErlangBinary(encoded);\n            try {\n                oos.close();\n            } catch (final IOException e) {\n            }\n            return result;\n        }\n    }\n\n    /**\n     * Decodes the given Erlang object from a binary to the according\n     * {@link OtpErlangObject} the same way as <tt>rdht_tx:decode_value/1</tt>.\n     *\n     * @param value\n     *            the encoded value\n     *\n     * @return the decoded value\n     *\n     * @throws OtpErlangDecodeException\n     *             if decoding fails\n     */\n    public static OtpErlangObject decode(final OtpErlangObject value)\n            throws OtpErlangDecodeException {\n        if (value instanceof OtpErlangBinary) {\n            final OtpErlangBinary valueBin = (OtpErlangBinary) value;\n            final OtpInputStream ois = new OtpInputStream(valueBin.binaryValue());\n            try {\n                return ois.read_any();\n            } finally {\n                try {\n                    ois.close();\n                } catch (final IOException e) {\n                }\n            }\n        } else {\n            return value;\n        }\n    }\n\n    /**\n     * Processes the <tt>received_raw</tt> term from erlang and if it is a\n     * <tt>{fail, abort, KeyList}</tt>, issues an {@link AbortException}.\n     *\n     * NOTE: this method should not be called manually by an application and may\n     * change without prior notice!\n     *\n     * @param received_raw\n     *            the object to process\n     * @param compressed\n     *            whether the transfer of values is compressed or not\n     *\n     * @throws AbortException\n     *             if the commit of the write failed\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.8\n     */\n    public static final void checkResult_failAbort(final OtpErlangObject received_raw,\n            final boolean compressed) throws AbortException, UnknownException {\n        try {\n            final OtpErlangTuple received = (OtpErlangTuple) received_raw;\n            checkResult_failAbort(received, compressed);\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Processes the <tt>received_raw</tt> term from erlang and if it is a\n     * <tt>{fail, abort, KeyList}</tt>, issues an {@link AbortException}.\n     *\n     * NOTE: this method should not be called manually by an application and may\n     * change without prior notice!\n     *\n     * @param received\n     *            the object to process\n     * @param compressed\n     *            whether the transfer of values is compressed or not\n     *\n     * @throws AbortException\n     *             if the commit of the write failed\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.12\n     */\n    public static final void checkResult_failAbort(final OtpErlangTuple received,\n            final boolean compressed) throws AbortException, UnknownException {\n        try {\n            if (received.elementAt(0).equals(CommonErlangObjects.failAtom)\n                    && (received.arity() == 3)\n                    && received.elementAt(1).equals(CommonErlangObjects.abortAtom)) {\n                final List<String> responsibleKeys = new ErlangValue(\n                        received.elementAt(2)).stringListValue();\n                throw new AbortException(responsibleKeys);\n            }\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, received);\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/Connection.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.io.IOException;\nimport java.net.UnknownHostException;\n\nimport com.ericsson.otp.erlang.OtpAuthException;\nimport com.ericsson.otp.erlang.OtpConnection;\nimport com.ericsson.otp.erlang.OtpErlangExit;\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpSelf;\n\n/**\n * Wraps an {@link OtpConnection} and allows automatic re-connects using a\n * {@link ConnectionPolicy} object.\n *\n * @author Nico Kruber, kruber@zib.de\n *\n * @version 3.14\n * @since 2.3\n */\npublic class Connection {\n    /**\n     * The connection this object wraps.\n     */\n    OtpConnection connection;\n    /**\n     * The local node used for the connection to a remote node.\n     */\n    OtpSelf self;\n    /**\n     * The remote node connected to.\n     */\n    PeerNode remote;\n    /**\n     * The connection policy object that sets how and whether to automatically\n     * reconnect on failures.\n     */\n    ConnectionPolicy connectionPolicy;\n\n    /**\n     * Creates a new connection using the given nodes and a default connection\n     * policy.\n     *\n     * Provided for convenience.\n     *\n     * @param self\n     *            the local node\n     * @param remote\n     *            the remote node to connect to\n     *\n     * @throws UnknownHostException\n     *             if the remote host could not be found\n     * @throws IOException\n     *             if it was not possible to connect to the remote node\n     * @throws OtpAuthException\n     *             if the connection was refused by the remote node\n     */\n    public Connection(final OtpSelf self, final PeerNode remote) throws UnknownHostException,\n            IOException, OtpAuthException {\n        super();\n        this.self = self;\n        this.connectionPolicy = new DefaultConnectionPolicy(remote);\n        this.remote = connectionPolicy.selectNode();\n\n        connect();\n    }\n\n    /**\n     * Creates a new connection between the a <tt>self</tt> node and one of the\n     * <tt>remoteNodes</tt>, selected by the <tt>connectionPolicy</tt>.\n     *\n     * @param self\n     *            the local node\n     * @param connectionPolicy\n     *            the connection policy to use\n     *\n     * @throws UnknownHostException\n     *             if the remote host could not be found\n     * @throws IOException\n     *             if it was not possible to connect to the remote node\n     * @throws OtpAuthException\n     *             if the connection was refused by the remote node\n     */\n    public Connection(final OtpSelf self, final ConnectionPolicy connectionPolicy) throws UnknownHostException,\n            IOException, OtpAuthException {\n        super();\n        this.self = self;\n        this.remote = connectionPolicy.selectNode();\n        this.connectionPolicy = connectionPolicy;\n\n        connect();\n    }\n\n    /**\n     * Tries connecting to the current {@link #remote} node. If this fails, it\n     * will try re-connecting to a node the {@link #connectionPolicy} chooses as\n     * long as this does not throw an exception. The {@link #remote} node will\n     * be set to the node the connection has been established with (or the last\n     * tried node).\n     *\n     * @throws UnknownHostException\n     *             if the remote host could not be found\n     * @throws IOException\n     *             if it was not possible to connect to the remote node\n     * @throws OtpAuthException\n     *             if the connection was refused by the remote node\n     */\n    private void connect() throws UnknownHostException,\n    IOException, OtpAuthException {\n        boolean success = false;\n        int retry = 0;\n        while(!success) {\n            try {\n                connection = self.connect(remote.getNode());\n                connectionPolicy.nodeConnectSuccess(remote);\n                success = true;\n            } catch (final UnknownHostException e) {\n                connectionPolicy.nodeFailed(remote);\n                remote = connectionPolicy.selectNode(++retry, remote, e);\n            } catch (final OtpAuthException e) {\n                connectionPolicy.nodeFailed(remote);\n                remote = connectionPolicy.selectNode(++retry, remote, e);\n            } catch (final IOException e) {\n                connectionPolicy.nodeFailed(remote);\n                remote = connectionPolicy.selectNode(++retry, remote, e);\n            }\n        }\n    }\n\n    private void reconnect() throws UnknownHostException, IOException,\n            OtpAuthException {\n        close();\n        connect();\n    }\n\n    /**\n     * Sends the given RPC and waits for a result.\n     *\n     * @param mod\n     *            the module of the function to call\n     * @param fun\n     *            the function to call\n     * @param args\n     *            the function's arguments\n     *\n     * @return the result of the call\n     *\n     * @throws ConnectionException\n     *             if the connection is not active, a communication error\n     *             occurs, an exit signal is received from a process on the\n     *             peer node or the remote node sends a message containing an\n     *             invalid cookie\n     */\n    public OtpErlangObject doRPC(final String mod, final String fun, final OtpErlangList args)\n            throws ConnectionException {\n        try {\n            boolean success = false;\n            final boolean isConnected = connection.isConnected();\n            while(!success) {\n                try {\n                    connection.sendRPC(mod, fun, args);\n                    final OtpErlangObject result = connection.receiveRPC();\n                    // result may be null but this should not happen and is an error anyway!\n                    if (result != null) {\n                        success = true;\n                        return result;\n                    }\n                } catch (final OtpErlangExit e) {\n                    connectionPolicy.nodeFailed(remote);\n                    // first re-try (connection was the first contact)\n                    remote = connectionPolicy.selectNode(1, remote, e);\n                    // reconnect (and then re-try the operation) if no exception was thrown:\n                    reconnect();\n                } catch (final OtpAuthException e) {\n                    connectionPolicy.nodeFailed(remote);\n                    // first re-try (connection was the first contact)\n                    remote = connectionPolicy.selectNode(1, remote, e);\n                    // reconnect (and then re-try the operation) if no exception was thrown:\n                    reconnect();\n                } catch (final IOException e) {\n                    // don't count RPC requests on closed connections as a failing node:\n                    if (isConnected) {\n                        connectionPolicy.nodeFailed(remote);\n                    }\n                    // first re-try (connection was the first contact)\n                    remote = connectionPolicy.selectNode(1, remote, e);\n                    // reconnect (and then re-try the operation) if no exception was thrown:\n                    reconnect();\n                }\n            }\n            // this should not happen as there is only one way out of the while\n            // without throwing an exception\n            throw new InternalError();\n        } catch (final OtpErlangExit e) {\n            // e.printStackTrace();\n            throw new ConnectionException(e);\n        } catch (final OtpAuthException e) {\n            // e.printStackTrace();\n            throw new ConnectionException(e);\n        } catch (final IOException e) {\n            // e.printStackTrace();\n            throw new ConnectionException(e);\n        }\n    }\n\n    /**\n     * Sends the given RPC and waits for a result.\n     *\n     * Provided for convenience.\n     *\n     * @param mod\n     *            the module of the function to call\n     * @param fun\n     *            the function to call\n     * @param args\n     *            the function's arguments\n     *\n     * @return the result of the call\n     *\n     * @throws ConnectionException\n     *             if the connection is not active, a communication error\n     *             occurs, an exit signal is received from a process on the\n     *             peer node or the remote node sends a message containing an\n     *             invalid cookie\n     */\n    public OtpErlangObject doRPC(final String mod, final String fun, final OtpErlangObject[] args)\n            throws ConnectionException {\n        return doRPC(mod, fun, new OtpErlangList(args));\n    }\n\n    /**\n     * Sends the given RPC and returns immediately.\n     *\n     * @param mod\n     *            the module of the function to call\n     * @param fun\n     *            the function to call\n     * @param args\n     *            the function's arguments\n     *\n     * @throws ConnectionException\n     *             if the connection is not active, a communication error\n     *             occurs, an exit signal is received from a process on the\n     *             peer node or the remote node sends a message containing an\n     *             invalid cookie\n     */\n    public void sendRPC(final String mod, final String fun, final OtpErlangList args)\n            throws ConnectionException {\n        try {\n            boolean success = false;\n            while(!success) {\n                try {\n                    connection.sendRPC(mod, fun, args);\n                    success = true;\n                    return;\n                } catch (final IOException e) {\n                    connectionPolicy.nodeFailed(remote);\n                    // first re-try (connection was the first contact)\n                    remote = connectionPolicy.selectNode(1, remote, e);\n                    // reconnect (and then re-try the operation) if no exception was thrown:\n                    reconnect();\n                }\n            }\n            // this should not happen as there is only one way out of the while\n            // without throwing an exception\n            throw new InternalError();\n        } catch (final OtpAuthException e) {\n            // e.printStackTrace();\n            throw new ConnectionException(e);\n        } catch (final IOException e) {\n            // e.printStackTrace();\n            throw new ConnectionException(e);\n        }\n    }\n\n    /**\n     * Sends the given RPC and returns immediately.\n     *\n     * Provided for convenience.\n     *\n     * @param mod\n     *            the module of the function to call\n     * @param fun\n     *            the function to call\n     * @param args\n     *            the function's arguments\n     *\n     * @throws ConnectionException\n     *             if the connection is not active, a communication error\n     *             occurs, an exit signal is received from a process on the\n     *             peer node or the remote node sends a message containing an\n     *             invalid cookie\n     */\n    public void sendRPC(final String mod, final String fun, final OtpErlangObject[] args)\n            throws ConnectionException {\n        sendRPC(mod, fun, new OtpErlangList(args));\n    }\n\n    /**\n     * Closes the connection to the remote node.\n     */\n    public void close() {\n        connection.close();\n    }\n\n    /**\n     * Gets the local node used for the connection.\n     *\n     * @return the local node (self)\n     */\n    public OtpSelf getSelf() {\n        return self;\n    }\n\n    /**\n     * Gets the remote node connected to.\n     *\n     * @return the remote node\n     */\n    public PeerNode getRemote() {\n        return remote;\n    }\n\n    /**\n     * Gets the encapsulated OTP connection object.\n     *\n     * @return the connection object\n     */\n    public OtpConnection getConnection() {\n        return connection;\n    }\n\n    /**\n     * Closes the connection when the object is destroyed.\n     */\n    @Override\n    protected void finalize() throws Throwable {\n        close();\n        super.finalize();\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/ConnectionException.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\nimport com.ericsson.otp.erlang.OtpErlangObject;\n\n/**\n * Exception that is thrown if an operation on a scalaris ring fails\n * because the connection is not active, a communication error occurred, an\n * exit signal was received or the remote node sent a message containing an\n * invalid cookie.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.2\n * @since 2.0\n */\npublic class ConnectionException extends OtpErlangException {\n    /**\n     * class version for serialisation\n     */\n    private static final long serialVersionUID = 1L;\n\n    /**\n     * Creates the exception with no message.\n     */\n    public ConnectionException() {\n    }\n\n    /**\n     * Creates the exception with the given message.\n     *\n     * @param msg\n     *            message of the exception\n     */\n    public ConnectionException(final String msg) {\n        super(msg);\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e the exception to \"re-throw\"\n     */\n    public ConnectionException(final Throwable e) {\n        super(e.getMessage());\n        setStackTrace(e.getStackTrace());\n    }\n\n    /**\n     * Creates an exception including the message of the given erlang object.\n     *\n     * @param erlValue\n     *            the erlang message to include\n     *\n     * @since 2.2\n     */\n    public ConnectionException(final OtpErlangObject erlValue) {\n        super(\"Erlang message: \" + erlValue.toString());\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e\n     *            the exception to \"re-throw\"\n     * @param erlValue\n     *            the string representation of this erlang value is included\n     *            into the message\n     *\n     * @since 2.2\n     */\n    public ConnectionException(final Throwable e, final OtpErlangObject erlValue) {\n        super(e.getMessage() + \",\\n  Erlang message: \" + erlValue.toString());\n        setStackTrace(e.getStackTrace());\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/ConnectionFactory.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.io.PrintStream;\nimport java.net.InetAddress;\nimport java.net.UnknownHostException;\nimport java.util.ArrayList;\nimport java.util.Collections;\nimport java.util.List;\nimport java.util.Properties;\nimport java.util.concurrent.atomic.AtomicLong;\n\nimport com.ericsson.otp.erlang.OtpSelf;\n\nimport de.zib.tools.PropertyLoader;\n\n/**\n * Provides means to create connections to scalaris nodes.\n *\n * This class uses a singleton-alike pattern providing a global (static)\n * instance through its {@link #getInstance()} method but also allowing for\n * object construction which might be useful when using multiple threads each\n * creating its own connections.\n *\n * The location of the default configuration file used by\n * {@link #ConnectionFactory()} can be overridden by specifying the <tt>\n * scalaris.java.config</tt> system property - otherwise the class tries to load\n * <tt>scalaris.properties</tt>.\n *\n * A specific property can also be overridden specifying a (non-empty) system\n * property with its name.\n *\n * A user-defined {@link Properties} object can also be used by creating objects\n * with {@link #ConnectionFactory(Properties)} or setting the new values with\n * {@link #setProperties(Properties)} but must provide the following values\n * (default values as shown)\n * <ul>\n * <li><tt>scalaris.node = \"node1@localhost\"</tt></li>\n * <li><tt>scalaris.cookie = \"chocolate chip cookie\"</tt></li>\n * <li><tt>scalaris.client.name = \"java_client\"</tt></li>\n * <li><tt>scalaris.client.appendUUID = \"true\"</tt></li>\n * </ul>\n *\n * Note: {@code scalaris.node} can be a whitespace, ',' or ';' separated list of\n * available nodes. See {@link DefaultConnectionPolicy} about how this list is\n * used when connections are setup or when existing connections fail.\n * Since Java and Erlang both need to known the same node name in order to\n * connect, any \"@localhost\" in such a name is translated using\n * {@link #fixLocalhostName(String)} to a real node name. The best method of\n * changing this name is to use the name Erlang uses - it can be provided by\n * setting the <tt>scalaris.erlang.nodename</tt> system property. If this is\n * not set or empty, we will try to look-up the name ourselves (see\n * {@link #getLocalhostName()}.\n *\n * The {@link #connectionPolicy} set will be passed on to created connections.\n * Changing it via {@link #setConnectionPolicy(ConnectionPolicy)} will not\n * change previously created connections - they keep the old policy. By\n * default, {@link DefaultConnectionPolicy} is used.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.20\n * @since 2.0\n */\npublic class ConnectionFactory {\n    /**\n     * Default name of the configuration file.\n     */\n    private static final String defaultConfigFile = \"scalaris.properties\";\n\n    private static final String hostname = findLocalhostName();\n\n    /**\n     * The name of the node to connect to.\n     */\n    private final List<PeerNode> nodes = Collections.synchronizedList(new ArrayList<PeerNode>());\n    /**\n     * The cookie name to use for connections.\n     */\n    private String cookie;\n    /**\n     * The name of the (Java) client to use when establishing a connection with\n     * erlang.\n     */\n    private String clientName;\n    /**\n     * Specifies whether to append a pseudo UUID to client names or not.\n     */\n    private boolean clientNameAppendUUID;\n    /**\n     * Pseudo UUID - the number of this counter is added to client names when\n     * creating a connection if clientNameAppendUUID is set.\n     *\n     * <p>\n     * TODO: In some cases, an atomic value is not necessary, e.g. when the\n     * factory is used by only one thread, then a normal long value will\n     * suffice. Possible optimisation: use {@link AtomicLong} for singleton\n     * instance and {@link Long} for the other instances!\n     * </p>\n     */\n    private final AtomicLong clientNameUUID = new AtomicLong(0);\n\n    /**\n     * Stores which config file was actually read into the object's properties.\n     */\n    private String configFileUsed;\n\n    /**\n     * Static instance of a connection factory.\n     */\n    private static ConnectionFactory instance = new ConnectionFactory();\n\n    /**\n     * Contains the default node selection policy a copy of which will be set\n     * for each created connection.\n     *\n     * @since 2.3\n     */\n    private ConnectionPolicy connectionPolicy = new DefaultConnectionPolicy(nodes);\n\n    /**\n     * Returns the static instance of a connection factory.\n     *\n     * @return a connection factory\n     */\n    public static ConnectionFactory getInstance() {\n        return instance;\n    }\n\n    /**\n     * Constructor, sets the parameters to use for connections according to\n     * values given in a <tt>scalaris.properties</tt> file and falls back to\n     * default values if values don't exist.\n     *\n     * By default the config file is assumed to be in the same directory as\n     * the classes. Specify the <tt>scalaris.java.config</tt> system property\n     * to set a different location.\n     *\n     * Default values are:\n     * <ul>\n     * <li><tt>scalaris.node = \"node1@localhost\"</tt></li>\n     * <li><tt>scalaris.cookie = \"chocolate chip cookie\"</tt></li>\n     * <li><tt>scalaris.client.name = \"java_client\"</tt></li>\n     * <li><tt>scalaris.client.appendUUID = \"true\"</tt></li>\n     * </ul>\n     *\n     * These properties can be overridden by specifying (non-empty) system\n     * properties with their names.\n     */\n    public ConnectionFactory() {\n        final Properties properties = new Properties();\n        String configFile = System.getProperty(\"scalaris.java.config\");\n        if ((configFile == null) || (configFile.length() == 0)) {\n            configFile = defaultConfigFile;\n        }\n//        System.out.println(\"loading config file: \" + configFile);\n        PropertyLoader.loadProperties(properties, configFile, true, false,\n                new String[] {\"scalaris.node\", \"scalaris.cookie\", \"scalaris.client.name\", \"scalaris.client.appendUUID\"});\n        setProperties(properties);\n    }\n\n    /**\n     * Constructor, sets the parameters to use for connections according to\n     * values given in the given {@link Properties} object and falls back to\n     * default values if values don't exist.\n     *\n     * The {@link Properties} object should provide the following values\n     * (default values as shown):\n     * <ul>\n     * <li><tt>scalaris.node = \"node1@localhost\"</tt></li>\n     * <li><tt>scalaris.cookie = \"chocolate chip cookie\"</tt></li>\n     * <li><tt>scalaris.client.name = \"java_client\"</tt></li>\n     * <li><tt>scalaris.client.appendUUID = \"true\"</tt></li>\n     * </ul>\n     *\n     * @param properties\n     */\n    public ConnectionFactory(final Properties properties) {\n        setProperties(properties);\n    }\n\n    /**\n     * Sets the object's members used for creating connections to erlang to\n     * values provided by the given {@link Properties} object.\n     *\n     * The {@link Properties} object should provide the following values\n     * (default values as shown):\n     * <ul>\n     * <li><tt>scalaris.node = \"node1@localhost\"</tt></li>\n     * <li><tt>scalaris.cookie = \"chocolate chip cookie\"</tt></li>\n     * <li><tt>scalaris.client.name = \"java_client\"</tt></li>\n     * <li><tt>scalaris.client.appendUUID = \"true\"</tt></li>\n     * </ul>\n     *\n     * NOTE: Existing connections are not changed!\n     *\n     * @param properties\n     *            the object to get the connection parameters from\n     */\n    public void setProperties(final Properties properties) {\n        final String[] nodesTemp = properties.getProperty(\"scalaris.node\", \"node1@localhost\").split(\"[\\\\s,;]\");\n        nodes.clear();\n\n        for (final String element : nodesTemp) {\n            addNode(fixLocalhostName(element));\n        }\n        cookie = properties.getProperty(\"scalaris.cookie\", \"chocolate chip cookie\");\n        clientName = properties.getProperty(\"scalaris.client.name\", \"java_client\");\n        if (properties.getProperty(\"scalaris.client.appendUUID\", \"true\").equals(\"true\")) {\n            clientNameAppendUUID = true;\n        } else {\n            clientNameAppendUUID = false;\n        }\n        configFileUsed = properties.getProperty(\"PropertyLoader.loadedfile\", \"\");\n\n        //System.out.println(\"node: \" + node);\n    }\n\n    /**\n     * Creates a connection to a scalaris erlang node specified by the given\n     * parameters. Uses the given client name.\n     *\n     * If <tt>clientNameAppendUUID</tt> is specified a pseudo UUID is appended\n     * to the given name. BEWARE that scalaris nodes accept only one connection\n     * per client name!\n     *\n     * @param clientName\n     *            the name that identifies the java client\n     * @param clientNameAppendUUID\n     *            override the object's setting for\n     *            {@link #clientNameAppendUUID}\n     * @param connectionPolicy\n     *            override the connection policy that will be used for the new\n     *            connection\n     *\n     * @return the created connection\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     *\n     * @since 2.3\n     */\n    public Connection createConnection(String clientName,\n            final boolean clientNameAppendUUID, final ConnectionPolicy connectionPolicy)\n            throws ConnectionException {\n        if (clientNameAppendUUID) {\n            clientName = clientName + \"_\" + clientNameUUID.getAndIncrement();\n        }\n        try {\n            final OtpSelf self = new OtpSelf(clientName + \"@\" + getLocalhostName(), cookie);\n            return new Connection(self, connectionPolicy);\n        } catch (final Exception e) {\n//                 e.printStackTrace();\n            throw new ConnectionException(e);\n        }\n    }\n\n    /**\n     * Creates a connection to a scalaris erlang node specified by the given\n     * parameters. Uses the given client name.\n     *\n     * If <tt>clientNameAppendUUID</tt> is specified a pseudo UUID is appended to\n     * the given name. BEWARE that scalaris nodes accept only one connection per\n     * client name!\n     *\n     * @param clientName\n     *            the name that identifies the java client\n     * @param clientNameAppendUUID\n     *            override the object's setting for\n     *            {@link #clientNameAppendUUID}\n     *\n     * @return the created connection\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     */\n    public Connection createConnection(final String clientName, final boolean clientNameAppendUUID)\n            throws ConnectionException {\n        return createConnection(clientName, clientNameAppendUUID, connectionPolicy);\n    }\n\n    /**\n     * Creates a connection to a scalaris erlang node specified by the given\n     * parameters. Uses the given client name.\n     *\n     * If {@link #clientNameAppendUUID} has been set, a pseudo UUID is appended\n     * to the given name. BEWARE that scalaris nodes accept only one connection\n     * per client name!\n     *\n     * @param clientName\n     *            the name that identifies the java client\n     *\n     * @return the created connection\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     */\n    public Connection createConnection(final String clientName)\n            throws ConnectionException {\n        return createConnection(clientName, clientNameAppendUUID);\n    }\n\n    /**\n     * Creates a connection to a scalaris erlang node specified by the given\n     * parameters. Uses the given connection policy.\n     *\n     * If {@link #clientNameAppendUUID} has been set, a pseudo UUID is appended\n     * to the given name. BEWARE that scalaris nodes accept only one connection\n     * per client name!\n     *\n     * @param connectionPolicy\n     *            override the connection policy that will be used for the new\n     *            connection\n     *\n     * @return the created connection\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     *\n     * @since 3.10\n     */\n    public Connection createConnection(final ConnectionPolicy connectionPolicy)\n            throws ConnectionException {\n        return createConnection(clientName, clientNameAppendUUID, connectionPolicy);\n    }\n\n    /**\n     * Creates a connection to a scalaris erlang node specified by the given\n     * parameters.\n     *\n     * @return the created connection\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     */\n    public Connection createConnection() throws ConnectionException {\n        return createConnection(clientName);\n    }\n\n    /**\n     * Replaces <tt>localhost</tt> in the node's name to the machine's real host\n     * name.\n     *\n     * Due to a \"feature\" of OtpErlang >= 1.4.1 erlang nodes are now only\n     * reachable by their official host name - so <tt>...@localhost</tt> does\n     * not work anymore if there is a real host name.\n     *\n     * @param node\n     *            the name of the node to use\n     *\n     * @return the node's official host name as returned by\n     *         {@link InetAddress#getHostName()}\n     */\n    public static String fixLocalhostName(final String node) {\n        if (node.endsWith(\"@localhost\")) {\n            return node.replaceAll(\"@localhost$\", \"@\" + getLocalhostName());\n        } else if (node.indexOf('@') == -1) {\n            return node + \"@\" + getLocalhostName();\n        }\n        return node;\n    }\n\n    /**\n     * Returns the name of the local host as determined by\n     * {@link #findLocalhostName()}.\n     *\n     * @return the local host' name or <tt>localhost</tt> if no IP address was\n     *         found\n     */\n    public static final String getLocalhostName() {\n        return hostname;\n    }\n\n    /**\n     * Returns the name of the local host (or at least what Java thinks it is).\n     * If the system property <tt>scalaris.erlang.nodename</tt> is set, this\n     * will be returned, otherwise we look-up our hostname with\n     * {@link InetAddress#getHostName()}.\n     *\n     * @return the local host' name or <tt>localhost</tt> if no IP address was\n     *         found\n     */\n    private static final String findLocalhostName() {\n        String hostname = \"localhost\";\n\n        final String erlangNodeName = System.getProperty(\"scalaris.erlang.nodename\");\n        if ((erlangNodeName == null) || (erlangNodeName.length() == 0)) {\n            try {\n                hostname = InetAddress.getLocalHost().getCanonicalHostName();\n            } catch (final UnknownHostException e) {\n            }\n        } else {\n            hostname = erlangNodeName;\n        }\n\n        return hostname;\n    }\n\n    /**\n     * Prints the object's properties to System.out.\n     */\n    public void printProperties() {\n        printProperties(System.out);\n    }\n\n    /**\n     * Prints the object's properties to the given output stream.\n     *\n     * @param out  the output stream to write to\n     */\n    public void printProperties(final PrintStream out) {\n        out.println(\"ConnectionFactory properties:\");\n        out.println(\"  config file                = \" + configFileUsed);\n        out.println(\"  scalaris.node              = \" + nodes.toString());\n        out.println(\"  scalaris.cookie            = \" + cookie);\n        out.println(\"  scalaris.client.name       = \" + clientName);\n        out.println(\"  scalaris.client.appendUUID = \" + clientNameAppendUUID);\n    }\n\n    /**\n     * Gets a copy of the list of nodes available for connections.\n     *\n     * The {@link PeerNode} elements of the list will not be copied!\n     *\n     * @return a set of nodes\n     *\n     * @since 2.3\n     */\n    public List<PeerNode> getNodes() {\n        return new ArrayList<PeerNode>(nodes);\n    }\n\n    /**\n     * Returns the name of the first node available for connection.\n     *\n     * WARNING: If {@link #nodes} contains multiple nodes, only the first of\n     * them will be returned. This is not necessarily the one the class will set\n     * up connections with!\n     *\n     * @return the name of the first node or an empty string if the list is\n     *         empty\n     *\n     * @deprecated This method returns the names of all nodes available for\n     *             connection as a String (as previously when only one node was\n     *             used), use the new {@link #getNodes()} method instead.\n     */\n    @Deprecated\n    public String getNode() {\n        if (nodes.size() > 0) {\n            return nodes.get(0).getNode().node();\n        } else {\n            return \"\";\n        }\n    }\n\n    /**\n     * Sets the name of the node to connect to.\n     *\n     * The list of nodes available for connection will be cleared at first, the\n     * node will then be added.\n     *\n     * @param node\n     *            the node to set (will not be mangled by\n     *            {@link #fixLocalhostName(String)}!)\n     *\n     * @see ConnectionFactory#addNode(String)\n     */\n    public void setNode(final String node) {\n        this.nodes.clear();\n        connectionPolicy.availableNodesReset();\n        addNode(node);\n    }\n\n    /**\n     * Sets the name of the node to connect to.\n     *\n     * The list of nodes available for connection will be cleared at first, the\n     * node will then be added.\n     *\n     * @param node\n     *            the node to set (will not be mangled by\n     *            {@link #fixLocalhostName(String)}!)\n     *\n     * @see ConnectionFactory#addNode(PeerNode)\n     *\n     * @since 3.16\n     */\n    public void setNode(final PeerNode node) {\n        this.nodes.clear();\n        connectionPolicy.availableNodesReset();\n        addNode(node);\n    }\n\n    /**\n     * Adds a node to the set of nodes available for connections.\n     *\n     * @param node\n     *            the node to add (will not be mangled by\n     *            {@link #fixLocalhostName(String)}!)\n     *\n     * @see ConnectionFactory#setNode(String)\n     *\n     * @since 2.3\n     */\n    public void addNode(final String node) {\n        final PeerNode p = new PeerNode(node);\n        addNode(p);\n    }\n\n    /**\n     * Adds a node to the set of nodes available for connections.\n     *\n     * @param node\n     *            the node to add\n     *\n     * @see ConnectionFactory#setNode(String)\n     *\n     * @since 3.16\n     */\n    public void addNode(final PeerNode node) {\n        this.nodes.add(node);\n        connectionPolicy.availableNodeAdded(node);\n    }\n\n    /**\n     * Removes a node from the set of nodes available for connections.\n     *\n     * @param node\n     *            the node to remove\n     *\n     * @see ConnectionFactory#getNodes()\n     *\n     * @since 2.3\n     */\n    public void removeNode(final PeerNode node) {\n        this.nodes.remove(node);\n        connectionPolicy.availableNodeRemoved(node);\n    }\n\n    /**\n     * (Re-)evaluates all currently set nodes by trying to establish a\n     * connection to each of them.\n     *\n     * Note: This resets all {@link PeerNode} statistics collected before.\n     *\n     * @since 3.20\n     */\n    public void testAllNodes() {\n        connectionPolicy.availableNodesReset();\n        for (final PeerNode node: nodes) {\n            node.resetFailureCount();\n            try {\n                createConnection(new FixedNodeConnectionPolicy(node)).close();\n            } catch (final ConnectionException e) {\n            }\n            connectionPolicy.availableNodeAdded(node);\n        }\n    }\n\n    /**\n     * Returns the cookie name to use for connections.\n     *\n     * @return the cookie\n     */\n    public String getCookie() {\n        return cookie;\n    }\n\n    /**\n     * Sets the cookie name to use for connections.\n     *\n     * @param cookie\n     *            the cookie to set\n     */\n    public void setCookie(final String cookie) {\n        this.cookie = cookie;\n    }\n\n    /**\n     * Returns the name of the (Java) client to use when establishing a\n     * connection with erlang.\n     *\n     * @return the clientName\n     */\n    public String getClientName() {\n        return clientName;\n    }\n\n    /**\n     * Sets the name of the (Java) client to use when establishing a connection\n     * with erlang.\n     *\n     * @param clientName\n     *            the clientName to set\n     */\n    public void setClientName(final String clientName) {\n        this.clientName = clientName;\n    }\n\n    /**\n     * Returns whether an UUID is appended to client names or not.\n     *\n     * @return <tt>true</tt> if an UUID is appended, <tt>false</tt> otherwise\n     */\n    public boolean isClientNameAppendUUID() {\n        return clientNameAppendUUID;\n    }\n\n    /**\n     * Sets whether to append an UUID to client names or not.\n     *\n     * @param clientNameAppendUUID\n     *            <tt>true</tt> if an UUID is appended, <tt>false</tt> otherwise\n     */\n    public void setClientNameAppendUUID(final boolean clientNameAppendUUID) {\n        this.clientNameAppendUUID = clientNameAppendUUID;\n    }\n\n    /**\n     * Sets the connection policy to use for new connections.\n     *\n     * Warning: this will effectively disconnect the previous connection policy\n     * from any updates to the list of available nodes (they will still use the\n     * same list but will not be able to react on changes). Prefer not to change\n     * the default policy after connections have been created.\n     *\n     * @param connectionPolicy\n     *            the connection policy to set\n     *\n     * @since 2.3\n     */\n    public void setConnectionPolicy(final ConnectionPolicy connectionPolicy) {\n        this.connectionPolicy = connectionPolicy;\n    }\n\n    /**\n     * Gets the current connection policy.\n     *\n     * @return the currently used connection policy\n     *\n     * @since 2.3\n     */\n    public ConnectionPolicy getConnectionPolicy() {\n        return connectionPolicy;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/ConnectionPolicy.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.ArrayList;\nimport java.util.List;\n\n/**\n * Defines a policy on how to select a node to connect to from a set of\n * possible nodes and whether to automatically re-connect.\n *\n * @author Nico Kruber, kruber@zib.de\n *\n * @see ConnectionFactory\n *\n * @version 2.3\n * @since 2.3\n */\npublic abstract class ConnectionPolicy {\n    /**\n     * A reference to the list of available nodes\n     */\n    protected List<PeerNode> availableRemoteNodes;\n\n    /**\n     * Creates a connection policy with one available node to connect to.\n     *\n     * Provided for convenience.\n     *\n     * @param remoteNode the node available for connections\n     */\n    public ConnectionPolicy(final PeerNode remoteNode) {\n        availableRemoteNodes = new ArrayList<PeerNode>(1);\n        availableRemoteNodes.add(remoteNode);\n    }\n\n    /**\n     * Creates a connection policy with the given set of nodes available for\n     * connections.\n     *\n     * @param availableRemoteNodes available nodes to connect to\n     */\n    public ConnectionPolicy(final List<PeerNode> availableRemoteNodes) {\n        this.availableRemoteNodes = availableRemoteNodes;\n    }\n\n    /**\n     * Signals the connection policy that the given node has been added to the\n     * list of available nodes.\n     *\n     * @param newNode the new node\n     */\n    public void availableNodeAdded(final PeerNode newNode) {\n    }\n\n    /**\n     * Signals the connection policy that the given node has been removed from\n     * the list of available nodes.\n     *\n     * @param removedNode the removed node\n     */\n    public void availableNodeRemoved(final PeerNode removedNode) {\n    }\n\n    /**\n     * Signals the connection policy that the list of available nodes has been\n     * reset (cleared).\n     */\n    public void availableNodesReset() {\n    }\n\n    /**\n     * Acts upon a failure of the given node.\n     *\n     * Sets the node's last failed connect time stamp.\n     *\n     * @param node the failed node\n     */\n    public void nodeFailed(final PeerNode node) {\n        synchronized (node) {\n            node.setLastFailedConnect();\n        }\n    }\n\n    /**\n     * Acts upon a failure reset of the given node.\n     *\n     * Resets the node's last failure state.\n     *\n     * @param node the node\n     */\n    public void nodeFailReset(final PeerNode node) {\n        synchronized (node) {\n            node.resetFailureCount();\n        }\n    }\n\n    /**\n     * Acts upon a successful connect attempt of the given node.\n     *\n     * Sets the node's last successful connect time stamp and resets its failure\n     * statistics.\n     *\n     * @param node the node\n     */\n    public void nodeConnectSuccess(final PeerNode node) {\n        synchronized (node) {\n            node.resetFailureCount();\n            node.setLastConnectSuccess();\n        }\n    }\n\n    /**\n     * Selects the node to connect with when establishing a connection (no\n     * failed node, no exception that has already been thrown).\n     *\n     * Provided for convenience.\n     *\n     * @return the node to use for the connection\n     *\n     * @throws UnsupportedOperationException\n     *             is thrown if the operation can not be performed, e.g. the\n     *             list is empty\n     *\n     * @see ConnectionFactory\n     */\n    public PeerNode selectNode() throws UnsupportedOperationException {\n        try {\n            return selectNode(0, null, null);\n        } catch (final UnsupportedOperationException e) {\n            throw e;\n        } catch (final Exception e) {\n            // this should not happen - selectNode should be able to copy with\n            // this situation without throwing another exception than\n            // UnsupportedOperationException\n            throw new InternalError();\n        }\n    }\n\n    /**\n     * Selects the node to (re-)connect with.\n     *\n     * If no re-connection is desired, throw an exception!\n     *\n     * @param <E>\n     *            the type of the exception that came from the failed connection\n     *            and may be re-thrown\n     *\n     * @param retry\n     *            the n'th retry (initial connect = 0, 1st reconnect = 1,...)\n     * @param failedNode\n     *            the node from the previous connection attempt or {@code null}\n     * @param e\n     *            the exception that came back from the previous connection\n     *            attempt or {@code null}\n     *\n     * @return the new node to connect with\n     *\n     * @throws E\n     *             if thrown, automatic re-connection attempts will stop\n     * @throws UnsupportedOperationException\n     *             is thrown if the operation can not be performed, e.g. the\n     *             list is empty\n     *\n     * @see Connection#connect()\n     * @see Connection#doRPC(String, String,\n     *      com.ericsson.otp.erlang.OtpErlangList)\n     * @see Connection#doRPC(String, String,\n     *      com.ericsson.otp.erlang.OtpErlangObject[])\n     */\n    public abstract <E extends Exception> PeerNode selectNode(int retry,\n            PeerNode failedNode, E e) throws E, UnsupportedOperationException;\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/ConnectionPool.java",
    "content": "/**\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.Collection;\nimport java.util.Iterator;\nimport java.util.LinkedList;\nimport java.util.Set;\n\n/**\n * Implements a simple (thread-safe) connection pool for Scalaris connections.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.7\n * @since 3.7\n */\npublic class ConnectionPool {\n    /**\n     * Connection factory used for creating connections.\n     */\n    protected ConnectionFactory cFactory;\n    /**\n     * Maximum number of connections in this pool (including checked out and\n     * available connections).\n     */\n    protected int maxConnections;\n    /**\n     * All available connections not checked out yet.\n     */\n    protected LinkedList<Connection> availableConns;\n    /**\n     * Number of checked out connections.\n     */\n    protected int checkedOut = 0;\n\n    /**\n     * Creates a new connection pool.\n     *\n     * @param cFactory\n     *            the connection factory to use to create new connections\n     * @param maxConnections\n     *            the maximum number of connections (<tt>0</tt> for no limit)\n     */\n    public ConnectionPool(final ConnectionFactory cFactory,\n            final int maxConnections) {\n        this.cFactory = cFactory;\n        this.maxConnections = maxConnections;\n        availableConns = new LinkedList<Connection>();\n    }\n\n    /**\n     * Gets a connection from the pool. Creates a new connection if necessary.\n     * Returns <tt>null</tt> if the maximum number of connections has already\n     * been hit.\n     *\n     * @return a connection to Scalaris or <tt>null</tt> if the maximum number\n     *         of connections has been hit\n     *\n     * @throws ConnectionException\n     *             if creating the connection fails\n     */\n    public synchronized Connection getConnection() throws ConnectionException {\n        Connection conn = null;\n        // use first available connection (if any):\n        if (!availableConns.isEmpty()) {\n            conn = availableConns.remove();\n            ++checkedOut;\n        } else if ((maxConnections == 0) || (checkedOut < maxConnections)) {\n            conn = cFactory.createConnection();\n            ++checkedOut;\n        }\n        return conn;\n    }\n\n    /**\n     * Tries to get a valid connection from the pool waiting at most\n     * <tt>timeout</tt> milliseconds. Creates a new connection if necessary and\n     * the maximum number of connections has not been hit yet. If the timeout is\n     * hit and no connection is available, <tt>null</tt> is returned.\n     *\n     * @param timeout\n     *            number of milliseconds to wait at most for a valid connection\n     *            to appear (<tt>0</tt> to wait forever)\n     *\n     * @return a connection to Scalaris or <tt>null</tt> if the timeout has been\n     *         hit\n     *\n     * @throws ConnectionException\n     *             if creating the connection fails\n     */\n    public Connection getConnection(final long timeout) throws ConnectionException {\n        final long timeAtStart = System.currentTimeMillis();\n        Connection conn;\n        while ((conn = getConnection()) == null) {\n            try {\n                synchronized (this) {\n                    wait(timeout);\n                }\n            } catch (final InterruptedException e) {\n            }\n            final long timeAtEnd = System.currentTimeMillis();\n            if ((timeAtEnd - timeAtStart) >= timeout) {\n                return null; // timeout\n            }\n        }\n        return conn;\n    }\n\n    /**\n     * Puts the given connection back into the pool.\n     *\n     * @param conn\n     *            the connection to release\n     */\n    public synchronized void releaseConnection(final Connection conn) {\n        availableConns.add(conn);\n        --checkedOut;\n        // need to notify all waiting threads so they do not exceed their timeouts\n        notifyAll();\n    }\n\n    /**\n     * Closes all available pooled connections.\n     *\n     * NOTE: This does not include any checked out connections!\n     */\n    public synchronized void closeAll() {\n        for (final Connection conn : availableConns) {\n            conn.close();\n        }\n        availableConns.clear();\n    }\n\n    /**\n     * Closes all available pooled connections to any node not in the given\n     * collection.\n     *\n     * NOTE: This does not include any checked out connections!\n     *\n     * @param remainingNodes\n     *            a set of nodes to which connections should remain (fast access\n     *            to {@link Collection#contains(Object)} is preferable, e.g. use\n     *            {@link Set})\n     */\n    public synchronized void closeAllBut(\n            final Collection<PeerNode> remainingNodes) {\n        for (final Iterator<Connection> iterator = availableConns.iterator();\n                iterator.hasNext();) {\n            final Connection conn = iterator.next();\n            if (!remainingNodes.contains(conn.getRemote())) {\n                conn.close();\n                iterator.remove();\n            }\n\n        }\n    }\n\n    /* (non-Javadoc)\n     * @see java.lang.Object#finalize()\n     */\n    @Override\n    protected void finalize() throws Throwable {\n        closeAll();\n        super.finalize();\n    }\n\n    /**\n     * Gets the connection factory used by the pool.\n     *\n     * @return the connection factory\n     */\n    public ConnectionFactory getConnectionFactory() {\n        return cFactory;\n    }\n\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/DefaultConnectionPolicy.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.io.IOException;\nimport java.util.ArrayList;\nimport java.util.List;\nimport java.util.Random;\nimport java.util.SortedSet;\nimport java.util.TreeSet;\n\nimport com.ericsson.otp.erlang.OtpAuthException;\n\n/**\n * Implements a {@link ConnectionPolicy} by choosing nodes randomly.\n *\n * Sorts nodes into two lists:\n * <ul>\n * <li>good nodes</li>\n * <li>bad nodes</li>\n * </ul>\n * Nodes that failed during a connection or in an attempt to connect will be\n * transferred to the bad nodes list {@link #badNodes}. When a connection\n * attempt to a node is successful, it will be removed from this list and put\n * into the good nodes list {@link #goodNodes} and the node's connection failure\n * statistics will be reset using {@link PeerNode#resetFailureCount()}.\n *\n * Whenever a node is being selected for a new connection (or reconnect), it\n * will select one randomly from the {@link #goodNodes} list. If this list is\n * empty, it will select the least recently failed node from {@link #badNodes}.\n * At most {@link #maxRetries} retries are attempted per operation (see\n * {@link Connection#connect()},\n * {@link Connection#doRPC(String, String, com.ericsson.otp.erlang.OtpErlangList)}\n * and\n * {@link Connection#doRPC(String, String, com.ericsson.otp.erlang.OtpErlangObject[])}\n * ) - the number of the current attempt will not be cached in this class. Set\n * the maximal number of retries using {@link #setMaxRetries(int)}.\n *\n * Attention: All member's functions are synchronised as there can be a single\n * connection policy object used by many threads and the access to the\n * {@link #goodNodes} and {@link #badNodes} members are logically linked\n * together and operations on both need to be performed atomically. Additionally\n * access to {@link PeerNode} objects are synchronised on themselves. It is\n * therefore important not to use any of this classes methods in blocks that\n * synchronise on any node object. Otherwise deadlocks might occur!!\n *\n * @author Nico Kruber, kruber@zib.de\n *\n * @version 2.3\n * @since 2.3\n */\npublic class DefaultConnectionPolicy extends ConnectionPolicy {\n\n    // we could use synchronised lists and sets as provided by\n    // Collections.synchronizedList and Collections.synchronizedSortedSet\n    // but those two depend on each other and we thus need synchronised methods\n    // which make synchronisations here obsolete\n    /**\n     * A list of good nodes (nodes which recently successfully connected).\n     */\n    protected List<PeerNode> goodNodes = new ArrayList<PeerNode>();\n    /**\n     * Bad nodes (nodes which recently failed) in the order of their failed\n     * date.\n     */\n    protected SortedSet<PeerNode> badNodes = new TreeSet<PeerNode>(new LeastRecentlyFailedNodesComparator());\n\n    /**\n     * Random number generator for selecting random nodes in the\n     * {@link #goodNodes} list.\n     */\n    private final Random random = new Random();\n\n    /**\n     * The maximal number of connection retries.\n     */\n    private int maxRetries = 3;\n\n    /**\n     * Creates a new connection policy working with the given remote node.\n     *\n     * Provided for convenience.\n     *\n     * Attention: This method also synchronises on the node.\n     *\n     * @param remoteNode the (only) available remote node\n     */\n    public DefaultConnectionPolicy(final PeerNode remoteNode) {\n        super(remoteNode);\n        availableNodeAdded(remoteNode);\n    }\n\n    /**\n     * Creates a new connection policy with the given remote nodes.\n     *\n     * Attention: This method synchronises on {@code availableRemoteNodes}.\n     *\n     * Any time this list is changed, the according methods in this class should\n     * be called, i.e. {@link #availableNodeAdded(PeerNode)},\n     * {@link #availableNodeRemoved(PeerNode)}, {@link #availableNodesReset()}\n     * to update the good and bad nodes lists.\n     *\n     * @param availableRemoteNodes\n     *            the remote nodes available for connections\n     */\n    public DefaultConnectionPolicy(final List<PeerNode> availableRemoteNodes) {\n        super(availableRemoteNodes);\n        synchronized (availableRemoteNodes) {\n            for (final PeerNode remoteNode : availableRemoteNodes) {\n                availableNodeAdded(remoteNode);\n            }\n        }\n    }\n\n    /**\n     * Adds the given node to the {@link #goodNodes} list if it has no failures,\n     * otherwise it will be added to {@link #badNodes}.\n     *\n     * Attention: This method also synchronises on the node.\n     *\n     * @param newNode the new node\n     */\n    @Override\n    public synchronized void availableNodeAdded(final PeerNode newNode) {\n        synchronized (newNode) {\n            if (newNode.getFailureCount() == 0) {\n                goodNodes.add(newNode);\n            } else {\n                badNodes.add(newNode);\n            }\n        }\n    }\n\n    /**\n     * Removes the node from the {@link #goodNodes} and {@link #badNodes} lists.\n     *\n     * @param removedNode the removed node\n     */\n    @Override\n    public synchronized void availableNodeRemoved(final PeerNode removedNode) {\n        goodNodes.remove(removedNode);\n        badNodes.remove(removedNode);\n    }\n\n    /**\n     * Resets the {@link #goodNodes} and {@link #badNodes} members as the list\n     * of available nodes has been reset.\n     */\n    @Override\n    public synchronized void availableNodesReset() {\n        goodNodes.clear();\n        badNodes.clear();\n    }\n\n    /**\n     * Sets the given node's last failed connect time stamp and moves it to the\n     * {@link #badNodes} list.\n     *\n     * Attention: This method also synchronises on the node.\n     *\n     * @param node the failed node\n     */\n    @Override\n    public synchronized void nodeFailed(final PeerNode node) {\n        synchronized (node) {\n            // remove the node from the badNodes if it is in there (will be\n            // reinserted at a new point)\n            badNodes.remove(node);\n            // update fail time before adding the node to the SortedSet!\n            node.setLastFailedConnect();\n            if (node.getFailureCount() == 1) {\n                // a node that has not failed before must be in goodNodes\n                // -> move it to badNodes\n                goodNodes.remove(node);\n            }\n            badNodes.add(node);\n        }\n    }\n\n    /**\n     * Acts upon a failure reset of the given node.\n     *\n     * Resets the node's last failure state.\n     *\n     * @param node the node\n     */\n    @Override\n    public void nodeFailReset(final PeerNode node) {\n        synchronized (node) {\n            if (node.getFailureCount() > 0) {\n                // a previously failed node must be in badNodes\n                // -> move it back to goodNodes\n                badNodes.remove(node);\n                node.resetFailureCount();\n                goodNodes.add(node);\n            }\n        }\n    }\n\n    /**\n     * Sets the node's last successful connect time stamp, resets its failure\n     * statistics and moves it to the {@link #goodNodes} list.\n     *\n     * Attention: This method also synchronises on the node.\n     *\n     * @param node the node\n     */\n    @Override\n    public synchronized void nodeConnectSuccess(final PeerNode node) {\n        synchronized (node) {\n            node.setLastConnectSuccess();\n            if (node.getFailureCount() > 0) {\n                // a previously failed node must be in badNodes\n                // -> move it back to goodNodes\n                badNodes.remove(node);\n                node.resetFailureCount();\n                goodNodes.add(node);\n            }\n        }\n    }\n\n    /**\n     * Returns a random node from the list of good nodes.\n     * Assumes {@link #goodNodes} to have at least one element.\n     *\n     * @return a random good node\n     */\n    protected synchronized PeerNode getGoodNode() {\n        if (goodNodes.size() == 1) {\n            return goodNodes.get(0);\n        } else {\n            return goodNodes.get(random.nextInt(goodNodes.size()));\n        }\n    }\n\n    /**\n     * Selects the node to (re-)connect with until the maximal number of\n     * {@link #maxRetries} has been reached.\n     *\n     * Throws an exception if {@code retry > maxRetries} and thus stops further\n     * node connection attempts. Otherwise chooses a random good node or (if\n     * there are no good nodes) the least recently failed bad node.\n     *\n     * @param <E>\n     *            the type of the exception that came from the failed connection\n     *            and may be re-thrown\n     *\n     * @param retry\n     *            the n'th retry (initial connect = 0, 1st reconnect = 1,...)\n     * @param failedNode\n     *            the node from the previous connection attempt or {@code null}\n     * @param e\n     *            the exception that came back from the previous connection\n     *            attempt or {@code null}\n     *\n     * @return the new node to connect with\n     *\n     * @throws E\n     *             if thrown, automatic re-connection attempts will stop\n     * @throws UnsupportedOperationException\n     *             is thrown if the operation can not be performed, e.g. the\n     *             list is empty\n     *\n     * @see Connection#connect()\n     * @see Connection#doRPC(String, String,\n     *      com.ericsson.otp.erlang.OtpErlangList)\n     * @see Connection#doRPC(String, String,\n     *      com.ericsson.otp.erlang.OtpErlangObject[])\n     */\n    @Override\n    @SuppressWarnings(\"unchecked\")\n    public synchronized <E extends Exception> PeerNode selectNode(final int retry,\n            final PeerNode failedNode, final E e) throws E {\n        assert maxRetries >= 0;\n        if (retry <= maxRetries) {\n            if ((goodNodes.size() + badNodes.size()) < 1) {\n                throw new UnsupportedOperationException(\n                        \"Can not choose a node from an empty list.\");\n            } else if (goodNodes.size() > 0) {\n                return getGoodNode();\n            } else {\n                return badNodes.first();\n            }\n        } else {\n            final String newMessage = e.getMessage() + \", bad nodes: \" + badNodes.toString() + \", good nodes: \" + goodNodes.toString() + \", retries: \" + (retry - 1);\n            if (e instanceof OtpAuthException) {\n                final OtpAuthException e1 = new OtpAuthException(newMessage);\n                e1.setStackTrace(e.getStackTrace());\n                throw (E) e1;\n            } else if (e instanceof IOException) {\n                final IOException e1 = new IOException(newMessage);\n                e1.setStackTrace(e.getStackTrace());\n                throw (E) e1;\n            } else {\n                throw e;\n            }\n        }\n    }\n\n\n    /**\n     * Sets the maximal number of automatic connection retries.\n     *\n     * @return the maxRetries\n     */\n    public int getMaxRetries() {\n        return maxRetries;\n    }\n\n    /**\n     * Gets the maximal number of automatic connection retries.\n     *\n     * @param maxRetries the maxRetries to set (>= 0)\n     */\n    public void setMaxRetries(final int maxRetries) {\n        if (maxRetries < 0) {\n            throw new IllegalArgumentException(\"maxRetries must be >= 0\");\n        }\n        this.maxRetries = maxRetries;\n    }\n\n    /**\n     * Gets a copy of the list of good nodes (contains references to the\n     * {@link PeerNode} objects).\n     *\n     * @return the list of good nodes\n     */\n    public synchronized List<PeerNode> getGoodNodes() {\n        return new ArrayList<PeerNode>(goodNodes);\n    }\n\n    /**\n     * Gets a copy of the list of good nodes (contains references to the\n     * {@link PeerNode} objects).\n     *\n     * @return the list of good nodes\n     */\n    public synchronized List<PeerNode> getBadNodes() {\n        final ArrayList<PeerNode> result = new ArrayList<PeerNode>(badNodes.size());\n        for (final PeerNode p : badNodes) {\n            result.add(p);\n        }\n        return result;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/DeleteResult.java",
    "content": "/**\n *\n */\npackage de.zib.scalaris;\n\nimport com.ericsson.otp.erlang.OtpErlangAtom;\nimport com.ericsson.otp.erlang.OtpErlangList;\n\n/**\n * Stores the result of a delete operation.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.19\n * @since 2.2\n *\n * @see ReplicatedDHT#delete(String)\n */\npublic class DeleteResult {\n    /**\n     * Number of successfully deleted replicas.\n     */\n    public int ok = 0;\n    /**\n     * Skipped replicas because locks were set.\n     */\n    public int locks_set = 0;\n    /**\n     * Skipped replicas because they did not exist.\n     */\n    public int undef = 0;\n\n    /**\n     * Creates a delete state object by converting the result list returned from\n     * erlang.\n     *\n     * @param list\n     *            the list to convert\n     * @throws UnknownException\n     *             is thrown if an unknown reason was encountered\n     */\n    public DeleteResult(final OtpErlangList list) throws UnknownException {\n        if (list != null) {\n            for (int i = 0; i < list.arity(); ++i) {\n                final OtpErlangAtom element = (OtpErlangAtom) list.elementAt(i);\n                if (element.equals(new OtpErlangAtom(\"ok\"))) {\n                    ++ok;\n                } else if (element.equals(new OtpErlangAtom(\"locks_set\"))) {\n                    ++locks_set;\n                } else if (element.equals(new OtpErlangAtom(\"undef\"))) {\n                    ++undef;\n                } else {\n                    throw new UnknownException(\"Unknow reason: \"\n                            + element.atomValue() + \" in \" + list.toString());\n                }\n            }\n        }\n    }\n\n    /**\n     * Checks whether the delete operation has successfully deleted all replicas\n     * (replicas which did not exist are counted as successfully deleted as\n     * well). If not, the delete needs to be executed again (see\n     * {@link ReplicatedDHT#delete(com.ericsson.otp.erlang.OtpErlangString, int)}.\n     *\n     * @param conn\n     *            a connection to Scalaris to find out the current replication\n     *            degree\n     * @return whether all replicas were deleted or not\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     *\n     * @since 3.19\n     */\n    public boolean hasDeletedAll(final Connection conn) throws ConnectionException {\n        final RoutingTable rt = new RoutingTable(conn);\n        final int r = rt.getReplicationFactor();\n\n        return (ok + undef) == r;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/EmptyListException.java",
    "content": "/**\n *  Copyright 2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\nimport com.ericsson.otp.erlang.OtpErlangObject;\n\n/**\n * Exception that is thrown if a read of a random list element on a scalaris\n * ring fails because the participating values are empty lists.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.18\n */\npublic class EmptyListException extends OtpErlangException {\n    /**\n     * class version for serialisation\n     */\n    private static final long serialVersionUID = 1L;\n\n    /**\n     * Creates the exception with no message.\n     */\n    public EmptyListException() {\n    }\n\n    /**\n     * Creates the exception with the given message.\n     *\n     * @param msg\n     *            message of the exception\n     */\n    public EmptyListException(final String msg) {\n        super(msg);\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e the exception to \"re-throw\"\n     */\n    public EmptyListException(final Throwable e) {\n        super(e.getMessage());\n        setStackTrace(e.getStackTrace());\n    }\n\n    /**\n     * Creates an exception including the message of the given erlang object.\n     *\n     * @param erlValue\n     *            the erlang message to include\n     */\n    public EmptyListException(final OtpErlangObject erlValue) {\n        super(\"Erlang message: \" + erlValue.toString());\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e\n     *            the exception to \"re-throw\"\n     * @param erlValue\n     *            the string representation of this erlang value is included\n     *            into the message\n     */\n    public EmptyListException(final Throwable e, final OtpErlangObject erlValue) {\n        super(e.getMessage() + \",\\n  Erlang message: \" + erlValue.toString());\n        setStackTrace(e.getStackTrace());\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/ErlangValue.java",
    "content": "/**\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.math.BigInteger;\nimport java.util.ArrayList;\nimport java.util.Collection;\nimport java.util.List;\nimport java.util.Map;\n\nimport com.ericsson.otp.erlang.OtpErlangAtom;\nimport com.ericsson.otp.erlang.OtpErlangBinary;\nimport com.ericsson.otp.erlang.OtpErlangBoolean;\nimport com.ericsson.otp.erlang.OtpErlangDouble;\nimport com.ericsson.otp.erlang.OtpErlangException;\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangLong;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangRangeException;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\n/**\n * Encapsulates a result from a read operation on scalaris.\n * See {@link #ErlangValue(Object)} for a list of compatible types.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.19\n * @since 3.0\n */\npublic class ErlangValue implements Comparable<ErlangValue> {\n    /**\n     * The (internal representation of the) wrapped erlang value.\n     */\n    private final OtpErlangObject value;\n\n    /**\n     * Creates a new object wrapping the given erlang value.\n     *\n     * @param value\n     *            a value from erlang\n     */\n    public ErlangValue(final OtpErlangObject value) {\n        this.value = value;\n    }\n\n    /**\n     * Creates a new object from a given set of Java types. The following types\n     * are supported:\n     *\n     * <h2>native types</h2>\n     * <ul>\n     * <li>{@link Boolean} - {@link OtpErlangBoolean}</li>\n     * <li>{@link Long} - {@link OtpErlangLong}</li>\n     * <li>{@link Integer} - {@link OtpErlangLong}</li>\n     * <li>{@link BigInteger} - {@link OtpErlangLong}</li>\n     * <li>{@link Double} - {@link OtpErlangDouble}</li>\n     * <li>{@link String} - {@link OtpErlangString}</li>\n     * <li><tt>byte[]</tt> - {@link OtpErlangBinary}</li>\n     * </ul>\n     *\n     * <h2>composite types</h2>\n     * <ul>\n     * <li>{@link List}&lt;Object&gt; with one of the native types except\n     * <tt>byte[]</tt> or another (supported) list/map - {@link OtpErlangList}</li>\n     * <li>{@link Collection}&lt;Object&gt; same as {@link List}&lt;Object&gt;\n     * (internally represented as a list)</li>\n     * <li>{@link Map}&lt;String, Object&gt; representing a JSON object -\n     * {@link OtpErlangTuple}</li>\n     * </ul>\n     *\n     * <h2>custom types</h2>\n     * <ul>\n     * <li>{@link OtpErlangObject} - an arbitrary erlang value</li>\n     * <li>{@link ErlangValue}</li>\n     * </ul>\n     *\n     * @param <T>\n     *            the type of the value\n     * @param value\n     *            the value to convert to an erlang type\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    public <T> ErlangValue(final T value) throws ClassCastException {\n        this.value = convertToErlang(value);\n    }\n\n    /**\n     * Converts a (supported) Java type to an {@link OtpErlangObject}.\n     *\n     * @param <T>\n     *            the type of the value\n     * @param value\n     *            the value to convert to an erlang type\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    public static <T> OtpErlangObject convertToErlang(final T value)\n            throws ClassCastException {\n        if (value instanceof Boolean) {\n            return new OtpErlangBoolean((Boolean) value);\n        } else if (value instanceof Integer) {\n            return new OtpErlangLong((Integer) value);\n        } else if (value instanceof Long) {\n            return new OtpErlangLong((Long) value);\n        } else if (value instanceof BigInteger) {\n            return new OtpErlangLong((BigInteger) value);\n        } else if (value instanceof Double) {\n            return new OtpErlangDouble((Double) value);\n        } else if (value instanceof String) {\n            return new OtpErlangString((String) value);\n        } else if (value instanceof byte[]) {\n            return new OtpErlangBinary((byte[]) value);\n        } else if (value instanceof Collection<?>) {\n            // support collection types (represented internally as a list)\n            final Collection<?> list = (Collection<?>) value;\n            final int listSize = list.size();\n            final OtpErlangObject[] erlValue = new OtpErlangObject[listSize];\n            int i = 0;\n            // TODO: optimise for specific types, e.g. lists?\n            for (final Object iter : list) {\n                erlValue[i] = convertToErlang(iter);\n                ++i;\n            }\n            return new OtpErlangList(erlValue);\n        } else if (value instanceof Map<?, ?>) {\n            // map to JSON object notation of Scalaris\n            @SuppressWarnings(\"unchecked\")\n            final\n            Map<String, Object> map = (Map<String, Object>) value;\n            final ErlangValueJSONToMap json_converter = new ErlangValueJSONToMap();\n            return json_converter.toScalarisJSON(map);\n        } else if (value instanceof ErlangValue) {\n            return ((ErlangValue) value).value();\n        } else if (value instanceof OtpErlangObject) {\n            return (OtpErlangObject) value;\n        } else {\n            // map to JSON object notation of Scalaris\n            @SuppressWarnings(\"unchecked\")\n            final\n            ErlangValueJSONToBean<T> json_converter = new ErlangValueJSONToBean<T>((Class<T>) value.getClass());\n            return json_converter.toScalarisJSON(value);\n//            throw new ClassCastException(\"Unsupported type (value: \" + value.toString() + \")\");\n        }\n    }\n\n    /**\n     * Returns the Java int value of the wrapped erlang value.\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported or the value is too big\n     *\n     * @since 3.3\n     */\n    public boolean boolValue() throws ClassCastException {\n        if (value.equals(CommonErlangObjects.falseAtom)) {\n            return false;\n        } else if (value.equals(CommonErlangObjects.trueAtom)) {\n            return true;\n        } else {\n            throw new ClassCastException(\"No boolean.\");\n        }\n    }\n\n    /**\n     * Returns the Java int value of the wrapped erlang value.\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported or the value is too big\n     */\n    public int intValue() throws ClassCastException {\n        try {\n            return ((OtpErlangLong) value).intValue();\n        } catch (final OtpErlangRangeException e) {\n            throw new ClassCastException(\"Cannot cast to int - value is too big (use longValue() or bigIntValue() instead).\");\n        }\n    }\n\n    /**\n     * Returns the Java long value of the wrapped erlang value.\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported or the value is too big\n     */\n    public long longValue() throws ClassCastException {\n        final OtpErlangLong longValue = (OtpErlangLong) value;\n        if (longValue.isLong()) {\n            return longValue.longValue();\n        } else {\n            throw new ClassCastException(\"Cannot cast to long - value is too big (use bigIntValue() instead).\");\n        }\n    }\n\n    /**\n     * Returns the Java BigInteger value of the wrapped erlang value.\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    public BigInteger bigIntValue() throws ClassCastException {\n        return ((OtpErlangLong) value).bigIntegerValue();\n    }\n\n    /**\n     * Returns the Java double value of the wrapped erlang value.\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    public double doubleValue() throws ClassCastException {\n        return ((OtpErlangDouble) value).doubleValue();\n    }\n\n    /**\n     * Converts an {@link OtpErlangObject} to a {@link String} taking special\n     * care of lists which have not be converted to strings automatically using\n     * the OTP library.\n     *\n     * @param value\n     *            the value to convert\n     *\n     * @return the value as a String\n     *\n     * @throws ClassCastException\n     *             if the conversion fails\n     */\n    static String otpObjectToString(final OtpErlangObject value)\n            throws ClassCastException {\n        // need special handling if OTP returned an empty list\n        if (value instanceof OtpErlangList) {\n            try {\n                return ((OtpErlangList) value).stringValue();\n            } catch (final OtpErlangException e) {\n                throw new ClassCastException(\"com.ericsson.otp.erlang.OtpErlangList cannot be cast to com.ericsson.otp.erlang.OtpErlangString: \" + e.getMessage());\n            }\n        } else if (value instanceof OtpErlangAtom) {\n            return ((OtpErlangAtom) value).atomValue();\n        } else {\n            return ((OtpErlangString) value).stringValue();\n        }\n    }\n\n    /**\n     * Converts an {@link OtpErlangObject} to a {@link OtpErlangString} taking\n     * special care of lists which have not be converted to strings\n     * automatically using the OTP library.\n     *\n     * @param value\n     *            the value to convert\n     *\n     * @return the value as a String\n     *\n     * @throws ClassCastException\n     *             if the conversion fails\n     */\n    static OtpErlangString otpObjectToOtpString(final OtpErlangObject value)\n            throws ClassCastException {\n        // need special handling if OTP returned an empty list\n        if (value instanceof OtpErlangList) {\n            try {\n                return new OtpErlangString((OtpErlangList) value);\n            } catch (final OtpErlangException e) {\n                throw new ClassCastException(\"com.ericsson.otp.erlang.OtpErlangList cannot be cast to com.ericsson.otp.erlang.OtpErlangString: \" + e.getMessage());\n            }\n        } else if (value instanceof OtpErlangAtom) {\n            return new OtpErlangString(((OtpErlangAtom) value).atomValue());\n        } else {\n            return ((OtpErlangString) value);\n        }\n    }\n\n    /**\n     * Returns the Java {@link String} value of the wrapped erlang value.\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    public String stringValue() throws ClassCastException {\n        return otpObjectToString(value);\n    }\n\n    /**\n     * Returns the Java byte[] value of the wrapped erlang value.\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    public byte[] binaryValue() throws ClassCastException {\n        return ((OtpErlangBinary) value).binaryValue();\n    }\n\n    /**\n     * Returns a JSON object (as {@link Map}&lt;String, Object&gt;) of the\n     * wrapped erlang value.\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    public Map<String, Object> jsonValue() throws ClassCastException {\n        /*\n         * object(): {struct, [{key::string() | atom(), value()}]}\n         * array():  {array, [value()]}\n         * value():  number(), string(), object(), array(), 'true', 'false', 'null'\n         *\n         * first term must be an object!\n         */\n        final OtpErlangTuple value_tpl = (OtpErlangTuple) value;\n        if ((value_tpl.arity() == 2)\n                && value_tpl.elementAt(0).equals(CommonErlangObjects.structAtom)) {\n            final ErlangValueJSONToMap json_converter = new ErlangValueJSONToMap();\n            return json_converter.toJava((OtpErlangList) value_tpl.elementAt(1));\n        } else {\n            throw new ClassCastException(\"wrong tuple arity\");\n        }\n    }\n\n    /**\n     * Returns a JSON object (as an instance of the given class) of the wrapped\n     * erlang value.\n     *\n     * @param <T>\n     *            the type of the object to create\n     *\n     * @param c\n     *            the class of the created object\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    public <T> T jsonValue(final Class<T> c) throws ClassCastException {\n        /*\n         * object(): {struct, [{key::string() | atom(), value()}]}\n         * array():  {array, [value()]}\n         * value():  number(), string(), object(), array(), 'true', 'false', 'null'\n         *\n         * first term must be an object!\n         */\n        final OtpErlangTuple value_tpl = (OtpErlangTuple) value;\n        if ((value_tpl.arity() == 2)\n                && value_tpl.elementAt(0).equals(CommonErlangObjects.structAtom)) {\n            final ErlangValueJSONToBean<T> json_converter = new ErlangValueJSONToBean<T>(c);\n            return json_converter.toJava((OtpErlangList) value_tpl.elementAt(1));\n        } else {\n            throw new ClassCastException(\"wrong tuple arity\");\n        }\n    }\n\n    /**\n     * Converts an {@link OtpErlangObject} to a {@link OtpErlangList} taking\n     * special care if the OTP library converted a list to an\n     * {@link OtpErlangString}.\n     *\n     * @param value\n     *            the value to convert\n     *\n     * @return the value as a OtpErlangList\n     *\n     * @throws ClassCastException\n     *             if the conversion fails\n     */\n    public static OtpErlangList otpObjectToOtpList(final OtpErlangObject value)\n            throws ClassCastException {\n        // need special handling if OTP thought that the value is a string\n        if (value instanceof OtpErlangString) {\n            final OtpErlangString value_string = (OtpErlangString) value;\n            return new OtpErlangList(value_string.stringValue());\n        } else {\n            return (OtpErlangList) value;\n        }\n    }\n\n    /**\n     * Converts list elements to a desired type.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     *\n     * @param <T>\n     *            the type to convert to\n     *\n     * @since 3.5\n     */\n    public static interface ListElementConverter<T> {\n        /**\n         * Conversion function.\n         *\n         * @param i\n         *            the index in the list\n         * @param v\n         *            the value to convert\n         *\n         * @return the value to convert to\n         */\n        public abstract T convert(int i, ErlangValue v);\n    }\n\n    /**\n     * Converts list elements to {@link String}s.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     *\n     * @since 3.19\n     */\n    public static class StringListElementConverter implements\n            ListElementConverter<String> {\n        public String convert(final int i, final ErlangValue v) { return v.stringValue(); }\n    }\n\n    /**\n     * Returns a list of mixed Java values of the wrapped erlang value.\n     *\n     * @param <T>\n     *            type of the elements in the list\n     * @param converter\n     *            object that converts the list value to the desired type\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    public <T> List<T> listValue(final ListElementConverter<T> converter) throws ClassCastException {\n        final OtpErlangList list = otpObjectToOtpList(value);\n        final ArrayList<T> result = new ArrayList<T>(list.arity());\n        for (int i = 0; i < list.arity(); ++i) {\n            result.add(converter.convert(i, new ErlangValue(list.elementAt(i))));\n        }\n        return result;\n    }\n\n    /**\n     * Returns a {@link Collection} of mixed Java values of the wrapped erlang\n     * value (internally represented as a list in Erlang).\n     *\n     * @param <T>\n     *            type of the elements in the list\n     * @param clazz\n     *            class of the result type\n     * @param converter\n     *            object that converts the list value to the desired type\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     *\n     * @since 3.19\n     */\n    public <T> Collection<T> listCollectionValue(\n            final Class<? extends Collection<T>> clazz,\n            final ListElementConverter<T> converter) throws ClassCastException {\n        final OtpErlangList list = otpObjectToOtpList(value);\n        Collection<T> result;\n        try {\n            result = clazz.newInstance();\n            for (int i = 0; i < list.arity(); ++i) {\n                result.add(converter.convert(i, new ErlangValue(list.elementAt(i))));\n            }\n            return result;\n        } catch (final Exception e) {\n            throw new ClassCastException(\"Error creating collection: \" + e.getMessage());\n        }\n    }\n\n    /**\n     * Returns a list of mixed Java values (wrapped in {@link ErlangValue}\n     * objects) of the wrapped erlang value.\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    public List<ErlangValue> listValue() throws ClassCastException {\n        return listValue(new ListElementConverter<ErlangValue>() {\n            public ErlangValue convert(final int i, final ErlangValue v) { return v; }\n        });\n    }\n\n    /**\n     * Returns a {@link Collection} of mixed Java values (wrapped in\n     * {@link ErlangValue} objects) of the wrapped erlang value (internally\n     * represented as a list in Erlang).\n     *\n     * @param clazz\n     *            class of the result type\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     *\n     * @since 3.19\n     */\n    public Collection<ErlangValue> listCollectionValue(\n            final Class<? extends Collection<ErlangValue>> clazz)\n            throws ClassCastException {\n        return listCollectionValue(clazz, new ListElementConverter<ErlangValue>() {\n            public ErlangValue convert(final int i, final ErlangValue v) { return v; }\n        });\n    }\n\n    /**\n     * Returns a list of {@link Long} values of the wrapped erlang value.\n     * Provided for convenience.\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     *\n     * @see #listValue(ListElementConverter)\n     */\n    public List<Long> longListValue() throws ClassCastException {\n        return listValue(new ListElementConverter<Long>() {\n            public Long convert(final int i, final ErlangValue v) { return v.longValue(); }\n        });\n    }\n\n    /**\n     * Returns a list of {@link Double} values of the wrapped erlang value.\n     * Provided for convenience.\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     *\n     * @see #listValue(ListElementConverter)\n     */\n    public List<Double> doubleListValue() throws ClassCastException {\n        return listValue(new ListElementConverter<Double>() {\n            public Double convert(final int i, final ErlangValue v) { return v.doubleValue(); }\n        });\n    }\n\n    /**\n     * Returns a list of {@link String} values of the wrapped erlang value.\n     * Provided for convenience.\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     *\n     * @see #listValue(ListElementConverter)\n     */\n    public List<String> stringListValue() throws ClassCastException {\n        return listValue(new StringListElementConverter());\n    }\n\n    /**\n     * Returns a list of <tt>byte[]</tt> values of the wrapped erlang value.\n     * Provided for convenience.\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     *\n     * @see #listValue(ListElementConverter)\n     */\n    public List<byte[]> binaryListValue() throws ClassCastException {\n        return listValue(new ListElementConverter<byte[]>() {\n            public byte[] convert(final int i, final ErlangValue v) { return v.binaryValue(); }\n        });\n    }\n\n    /**\n     * Returns a list of JSON objects (as an instance of the given class) of the\n     * wrapped erlang value. Provided for convenience.\n     *\n     * @param <T>\n     *            the type of the object to create as a list element\n     *\n     * @param c\n     *            the class of the created object\n     *\n     * @return the converted value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     *\n     * @see #listValue(ListElementConverter)\n     * @since 3.5\n     */\n    public <T> List<T> jsonListValue(final Class<T> c) throws ClassCastException {\n        return listValue(new ListElementConverter<T>() {\n            public T convert(final int i, final ErlangValue v) { return v.jsonValue(c); }\n        });\n    }\n\n    /**\n     * Gets the original erlang value.\n     *\n     * @return the value as reported by erlang\n     */\n    public OtpErlangObject value() {\n        return value;\n    }\n\n    @Override\n    public boolean equals(final Object obj) {\n        if (this == obj) {\n            return true;\n        }\n        if (!(obj instanceof ErlangValue)) {\n            return false;\n        }\n\n        final ErlangValue erlValue = (ErlangValue) obj;\n        return value.equals(erlValue.value);\n    }\n\n    @Override\n    public int hashCode() {\n        return value.hashCode();\n    }\n\n    @Override\n    public String toString() {\n        return value.toString();\n    }\n\n    /**\n     * Compares two erlang values by their string representation (expensive!).\n     *\n     * @param o\n     *            another erlang value\n     */\n    public int compareTo(final ErlangValue o) {\n        return value.toString().compareTo(o.toString());\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/ErlangValueJSONBase.java",
    "content": "/**\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.math.BigInteger;\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpErlangAtom;\nimport com.ericsson.otp.erlang.OtpErlangDouble;\nimport com.ericsson.otp.erlang.OtpErlangInt;\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangLong;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangRangeException;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\n/**\n * Base class for Scalaris-JSON to Java and Java to Scalaris-JSON converters.\n *\n * Provides common methods for sub-classes.\n *\n * @author Nico Kruber, kruber@zib.de\n */\nabstract class ErlangValueJSONBase {\n    /**\n     * Converts a Java Map to a JSON object as expected by Scalaris.\n     *\n     * @param value\n     *            a Java Map with String-keys and supported JSON types as values\n     *\n     * @return a JSON object representing the value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    protected abstract OtpErlangTuple convertJavaToScalarisJSON_object(\n            Object value) throws ClassCastException;\n\n    /**\n     * Converts a Java object to a JSON value as expected by Scalaris.\n     *\n     * @param value\n     *            a Java object supported by JSON\n     *\n     * @return a JSON object representing the value\n     *\n     * @throws ClassCastException\n     *                if thrown if a conversion is not possible, i.e. the type\n     *                is not supported\n     */\n    protected OtpErlangObject convertJavaToScalarisJSON_value(final Object value)\n            throws ClassCastException {\n        if (value == null) {\n            return CommonErlangObjects.nullAtom;\n        } else if (value instanceof Integer) {\n            return new OtpErlangInt((Integer) value);\n        } else if (value instanceof Long) {\n            return new OtpErlangLong((Long) value);\n        } else if (value instanceof BigInteger) {\n            return new OtpErlangLong((BigInteger) value);\n        } else if (value instanceof Double) {\n            return new OtpErlangDouble((Double) value);\n        } else if (value instanceof String) {\n            return new OtpErlangString((String) value);\n        } else if (value instanceof List<?>) {\n            @SuppressWarnings(\"unchecked\")\n            final\n            List<Object> list = (List<Object>) value;\n            return convertJavaToScalarisJSON_array(list);\n        } else if (value.equals(true)) {\n            return CommonErlangObjects.trueAtom;\n        } else if (value.equals(false)) {\n            return CommonErlangObjects.falseAtom;\n        } else {\n            return convertJavaToScalarisJSON_object(value);\n        }\n//            throw new ClassCastException(\"Unsupported JSON type (value: \" + value.toString() + \")\");\n    }\n\n    /**\n     * Converts a list of Java Objects to a JSON array value as expected by\n     * Scalaris.\n     *\n     * @param value\n     *            a list of supported JSON values\n     *\n     * @return a JSON object representing the value\n     *\n     * @throws ClassCastException\n     *                if thrown if a conversion is not possible, i.e. the type\n     *                is not supported\n     */\n    protected OtpErlangTuple convertJavaToScalarisJSON_array(\n            final List<Object> value) throws ClassCastException {\n        final OtpErlangObject[] resultList = new OtpErlangObject[value.size()];\n        int i = 0;\n        for (final Object iter : value) {\n            resultList[i] = convertJavaToScalarisJSON_value(iter);\n            ++i;\n        }\n        final OtpErlangTuple resultTpl = new OtpErlangTuple(new OtpErlangObject[] {\n                CommonErlangObjects.arrayAtom, new OtpErlangList(resultList) });\n        return resultTpl;\n    }\n\n    /**\n     * Converts an unknown JSON value to a Java object (no composite types!).\n     *\n     * @param value\n     *            a non-composite JSON value as stored by Scalaris' JSON API,\n     *            i.e. string, number, true, false, null\n     *\n     * @return a Java object representing the value\n     *\n     * @throws ClassCastException\n     *                if thrown if a conversion is not possible, i.e. the type\n     *                is not supported\n     */\n    protected Object convertScalarisJSONtoJava_value_simple(final OtpErlangObject value)\n            throws ClassCastException {\n        if (value instanceof OtpErlangLong) {\n            final OtpErlangLong value_int = (OtpErlangLong) value;\n            try {\n                return value_int.intValue();\n            } catch (final OtpErlangRangeException e) {\n                if (value_int.isLong()) {\n                    return value_int.longValue();\n                } else {\n                    return value_int.bigIntegerValue();\n                }\n            }\n        } else if (value instanceof OtpErlangDouble) {\n            return ((OtpErlangDouble) value).doubleValue();\n        } else if (value instanceof OtpErlangString) {\n            return ((OtpErlangString) value).stringValue();\n        } else if (value instanceof OtpErlangList) {\n            try {\n                return ErlangValue.otpObjectToString(value);\n            } catch (final ClassCastException e) {\n                throw new ClassCastException(\"Unsupported JSON type (value: \" + value.toString() + \")\");\n            }\n        } else if (value.equals(CommonErlangObjects.trueAtom)) {\n            return true;\n        } else if (value.equals(CommonErlangObjects.falseAtom)) {\n            return false;\n        } else if (value.equals(CommonErlangObjects.nullAtom)) {\n            return null;\n        } else {\n            throw new ClassCastException(\"Unsupported JSON type (value: \" + value.toString() + \")\");\n        }\n    }\n\n    /**\n     * Converts the given Scalaris erlang object (as part of an object's key) to\n     * a string.\n     *\n     * @param key_erl\n     *            the erlang object containing the key\n     *\n     * @return the key\n     *\n     * @throws ClassCastException\n     *             if the key is not a valid key object\n     */\n    protected String convertScalarisJSONtoJava_key(\n            final OtpErlangObject key_erl) throws ClassCastException {\n        if (key_erl instanceof OtpErlangAtom) {\n            return ((OtpErlangAtom) key_erl).atomValue();\n        } else {\n            try {\n                return ErlangValue.otpObjectToString(key_erl);\n            } catch (final ClassCastException e) {\n                throw new ClassCastException(\"Unsupported JSON key (\"\n                        + key_erl.toString() + \"): \" + e.getMessage());\n            }\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/ErlangValueJSONInterface.java",
    "content": "/**\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\n/**\n * Interface for Scalaris-JSON to Java and Java to Scalaris-JSON converters.\n *\n * @param <T> the class to convert to/from\n *\n * @author Nico Kruber, kruber@zib.de\n */\ninterface ErlangValueJSONInterface<T> {\n    /**\n     * Converts a Java Map to a JSON object as expected by Scalaris.\n     *\n     * @param value\n     *            a Java Map with String-keys and supported JSON types as values\n     *\n     * @return a JSON object representing the value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    public abstract OtpErlangTuple toScalarisJSON(T value)\n            throws ClassCastException;\n\n    /**\n     * Converts a JSON object value (a list of key/value pairs) to a Java Map.\n     *\n     * @param value\n     *            a list of key/value pairs with JSON values and string keys as\n     *            stored by Scalaris' JSON API\n     *\n     * @return a Java object representing the value\n     *\n     * @throws ClassCastException\n     *                if thrown if a conversion is not possible, i.e. the type\n     *                is not supported\n     */\n    public abstract T toJava(OtpErlangList value) throws ClassCastException;\n\n}"
  },
  {
    "path": "java-api/src/de/zib/scalaris/ErlangValueJSONToBean.java",
    "content": "/**\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.lang.reflect.InvocationTargetException;\nimport java.lang.reflect.Method;\nimport java.lang.reflect.ParameterizedType;\nimport java.lang.reflect.Type;\nimport java.math.BigInteger;\nimport java.util.ArrayList;\nimport java.util.Arrays;\nimport java.util.LinkedHashMap;\nimport java.util.LinkedList;\nimport java.util.List;\nimport java.util.Map;\nimport java.util.regex.Pattern;\n\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\n/**\n * Converts Scalaris-JSON to Beans with setters and getters for each JSON key\n * and such Beans to Scalaris-JSON.\n *\n * Supported member types of the Bean:\n * <ul>\n * <li>boolean, {@link Boolean}</li>\n * <li>int, {@link Integer}</li>\n * <li>long, {@link Long}</li>\n * <li>{@link BigInteger}</li>\n * <li>double, {@link Double}</li>\n * <li>{@link String}</li>\n * <li>{@link List} of primitive types (any other element will be converted to\n * {@link Map}&lt;String, Object&gt;)</li>\n * <li>{@link Map} with string keys and primitive-typed values (any other value\n * will be converted to {@link Map}&lt;String, Object&gt;)</li>\n * <li>such a Bean\n * </ul>\n *\n * Note: regarding lists and maps: it is not possible to get type information\n * from their elements (type erasure).\n *\n * Setter methods must be of the form setKey(xxx), getter methods of the form\n * getKey() or isKey().\n *\n * @param <T>\n *            the Bean to convert to/from\n *\n * @author Nico Kruber, kruber@zib.de\n */\nclass ErlangValueJSONToBean<T> extends ErlangValueJSONBase implements ErlangValueJSONInterface<T> {\n    /**\n     * The class to convert the object to.\n     */\n    private final Class<T> c;\n\n    /**\n     * Creates a new object converting to the given class.\n     *\n     * @param c\n     *            the class to convert JSON to\n     */\n    public ErlangValueJSONToBean(final Class<T> c) {\n        this.c = c;\n    }\n\n    /**\n     * Gets an {@link ErlangValueJSONToBean} instance using the given\n     * {@link Class}.\n     *\n     * @param <U>\n     *            type of the {@link Class}\n     *\n     * @param c\n     *            the class\n     *\n     * @return an {@link ErlangValueJSONToBean}<U>\n     */\n    public static <U> ErlangValueJSONToBean<U> getInstance(final Class<U> c) {\n        return new ErlangValueJSONToBean<U>(c);\n    }\n\n    /**\n     * Gets an {@link ErlangValueJSONToBean} instance using the given\n     * {@link Type}.\n     *\n     * @param t\n     *            the type\n     *\n     * @return an {@link ErlangValueJSONToBean}<U>\n     */\n    public static ErlangValueJSONToBean<?> getInstance(final Type t) {\n        final ErlangValueJSONToBean<?> json_converter = getInstance(getRawType(t));\n        return json_converter;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.ErlangValueJSON#toScalarisJSON(T)\n     */\n    @Override\n    public OtpErlangTuple toScalarisJSON(final T value) throws ClassCastException {\n        return convertJavaToScalarisJSON_object2(value);\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.ErlangValueJSON#toJava(com.ericsson.otp.erlang.OtpErlangList)\n     */\n    @Override\n    @SuppressWarnings(\"unchecked\")\n    public T toJava(final OtpErlangList value) throws ClassCastException {\n        return (T) convertScalarisJSONtoJava_object2(value, null);\n    }\n\n    /**\n     * Capitalize the first letter of the given string.\n     *\n     * @param key\n     *            the string\n     *\n     * @return a string with the first character being upper case\n     */\n    private static String capFirst(final String key) {\n        if (key.length() > 0) {\n            final String keyCap = key.substring(0, 1).toUpperCase() + key.substring(1);\n            return keyCap;\n        }\n        return \"\";\n    }\n\n    /**\n     * De-capitalize the first letter of the given string.\n     *\n     * @param key\n     *            the string\n     *\n     * @return a string with the first character being lower case\n     */\n    private static String decapFirst(final String key) {\n        if (key.length() > 0) {\n            final String keyCap = key.substring(0, 1).toLowerCase() + key.substring(1);\n            return keyCap;\n        }\n        return \"\";\n    }\n\n    /**\n     * Gets the class of the raw type of the given type.\n     *\n     * @param type the type object\n     *\n     * @return the class behind the type\n     */\n    private static Class<?> getRawType(final Type type) {\n        if (type instanceof ParameterizedType) {\n            return (Class<?>) ((ParameterizedType) type).getRawType();\n        } else {\n            return (Class<?>) type;\n        }\n    }\n\n    /**\n     * Uses introspection to get the setter method for the given key of class\n     * {@link #c}.\n     * Setter methods must be of the form setKey(xxx),\n     * getter methods of the form getKey() or isKey().\n     *\n     * @param key\n     *            the key to get the setter for\n     *\n     * @return the setter method\n     *\n     * @throws ClassCastException\n     *             if there is no public setter method for <tt>key</tt>\n     */\n    private Method getSetterFor(final String key, final Type type) {\n        final String keyCap1st = capFirst(key);\n        final String setMethod = \"set\" + keyCap1st;\n        final Class<?> class_ = getRawType(type);\n        try {\n            return c.getMethod(setMethod, class_);\n        } catch (final Exception e) {\n            throw new ClassCastException(\"no setter \" + setMethod + \"(\"\n                    + class_.getSimpleName() + \"): \" + e.getMessage());\n        }\n    }\n\n    /**\n     * Uses introspection to get the type of the given key of class {@link #c}.\n     * Assumes there is a getter of the form getKey() or isKey().\n     *\n     * @param key\n     *            the key to get the type for\n     *\n     * @return the {@link Class} of the type.\n     *\n     * @throws ClassCastException\n     *             if there is no public getter method for <tt>key</tt>\n     */\n    private Type getTypeOf(final String key) throws ClassCastException {\n        final String keyCap1st = capFirst(key);\n        try {\n            try {\n                return c.getMethod(\"get\" + keyCap1st).getGenericReturnType();\n            } catch (final NoSuchMethodException e) {\n                return c.getMethod(\"is\" + keyCap1st).getGenericReturnType();\n            }\n        } catch (final Exception e) {\n            throw new ClassCastException(\"no getter [get|is]\" + keyCap1st + \": \" + e.getMessage());\n        }\n    }\n\n    private final Pattern getMatcher = java.util.regex.Pattern.compile(\"^get|is\");\n\n    /**\n     * Converts a Java Map to a JSON object as expected by Scalaris.\n     *\n     * @param value_\n     *            a Java Map with String-keys and supported JSON types as values\n     *\n     * @return a JSON object representing the value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    @Override\n    protected OtpErlangTuple convertJavaToScalarisJSON_object(\n            final Object value) throws ClassCastException {\n        final ErlangValueJSONToBean<?> json_converter = getInstance(value.getClass());\n        return json_converter.convertJavaToScalarisJSON_object2(value);\n    }\n\n    /**\n     * Converts a Java Map to a JSON object as expected by Scalaris.\n     *\n     * @param value_\n     *            a Java Map with String-keys and supported JSON types as values\n     *\n     * @return a JSON object representing the value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    protected OtpErlangTuple convertJavaToScalarisJSON_object2(\n            final Object value_) throws ClassCastException {\n        try {\n            if (value_ instanceof Map<?, ?>) {\n                @SuppressWarnings(\"unchecked\")\n                final\n                Map<String, Object> value = (Map<String, Object>) value_;\n                final OtpErlangTuple[] resultList = new OtpErlangTuple[value.size()];\n                int i = 0;\n\n                for (final Map.Entry<String, Object> entry : value.entrySet()) {\n                    resultList[i] = new OtpErlangTuple(new OtpErlangObject[] {\n                            new OtpErlangString(entry.getKey()),\n                            convertJavaToScalarisJSON_value(entry.getValue()) });\n                    ++i;\n                }\n                final OtpErlangTuple resultTpl = new OtpErlangTuple(new OtpErlangObject[] {\n                        CommonErlangObjects.structAtom, new OtpErlangList(resultList) });\n                return resultTpl;\n            } else {\n                @SuppressWarnings(\"unchecked\")\n                final\n                T value = (T) value_;\n\n                // get all getters:\n                final Method[] methods = c.getDeclaredMethods();\n                final List<OtpErlangObject> resultList = new LinkedList<OtpErlangObject>();\n\n                for (final Method method : methods) {\n                    final String methodName = method.getName();\n                    if (getMatcher.matcher(methodName).lookingAt()) {\n                        final String key_j = decapFirst(getMatcher.matcher(methodName).replaceFirst(\"\"));\n                        try {\n                            final OtpErlangObject value_j = convertJavaToScalarisJSON_value(\n                                    method.invoke(value));\n                            resultList.add(\n                                    new OtpErlangTuple(new OtpErlangObject[] {\n                                            new OtpErlangString(key_j),\n                                            value_j }));\n                        } catch (final IllegalArgumentException e) {\n                            e.printStackTrace();\n                            throw new ClassCastException(\n                                    \"cannot access getter \" + methodName\n                                            + \"() of class \" + c.getSimpleName()\n                                            + \": \" + e.getMessage());\n                        } catch (final IllegalAccessException e) {\n                            throw new ClassCastException(\n                                    \"cannot access getter \" + methodName\n                                            + \"() of class \" + c.getSimpleName()\n                                            + \": \" + e.getMessage());\n                        } catch (final InvocationTargetException e) {\n                            throw new ClassCastException(\n                                    \"cannot access getter \" + methodName\n                                            + \"() of class \" + c.getSimpleName()\n                                            + \": \" + e.getMessage());\n                        }\n                    }\n                }\n                final OtpErlangTuple resultTpl = new OtpErlangTuple(new OtpErlangObject[] {\n                        CommonErlangObjects.structAtom,\n                                new OtpErlangList(resultList\n                                        .toArray(new OtpErlangObject[resultList\n                                                .size()])) });\n                return resultTpl;\n            }\n        } catch (final ClassCastException e) {\n            e.printStackTrace();\n            throw new ClassCastException(\"Unsupported JSON type (value: \" + value_ + \")\");\n        }\n    }\n\n    /**\n     * Converts a JSON object value (a list of key/value pairs) to a Java Map.\n     *\n     * @param value\n     *            a list of key/value pairs with JSON values and string keys as\n     *            stored by Scalaris' JSON API\n     * @param type_\n     *            the (supposed) type of the returned value\n     *            (hint for container types, may be <tt>null</tt>)\n     *\n     * @return a Java object representing the value\n     *\n     * @throws ClassCastException\n     *                if thrown if a conversion is not possible, i.e. the type\n     *                is not supported\n     */\n    protected Object convertScalarisJSONtoJava_object2(\n            final OtpErlangList value, final Type type_) throws ClassCastException {\n        if (c.equals(Map.class) || Arrays.asList(c.getInterfaces()).contains(Map.class)) {\n            // target type is a map:\n            Type elementType = Object.class;\n\n            // we might have some more details about its value's type:\n            if ((type_ != null) && (type_ instanceof ParameterizedType)) {\n                final ParameterizedType type = (ParameterizedType) type_;\n                final Type[] typeArguments = type.getActualTypeArguments();\n                if (typeArguments.length == 2) {\n                    elementType = typeArguments[1];\n                }\n            }\n\n            final Map<String, Object> result = new LinkedHashMap<String, Object>(\n                    value.arity());\n            for (final OtpErlangObject iter : value) {\n                final OtpErlangTuple iter_tpl = (OtpErlangTuple) iter;\n                if (iter_tpl.arity() == 2) {\n                    final String key = convertScalarisJSONtoJava_key(\n                            iter_tpl.elementAt(0));\n                    result.put(key,\n                            convertScalarisJSONtoJava_value2(\n                                    iter_tpl.elementAt(1), elementType));\n                } else {\n                    throw new ClassCastException(\n                            \"Unsupported JSON type (value: \" + value.toString()\n                                    + \")\");\n                }\n            }\n            return result;\n        } else {\n            // target type is a bean:\n            T result;\n            try {\n                result = c.getConstructor().newInstance();\n            } catch (final Exception e) {\n                throw new ClassCastException(\n                        \"Cannot store value to JSON object (value: \"\n                                + value.toString() + \"): \" + e.getMessage());\n            }\n            for (final OtpErlangObject iter : value) {\n                final OtpErlangTuple iter_tpl = (OtpErlangTuple) iter;\n                if (iter_tpl.arity() == 2) {\n                    try {\n                        final OtpErlangObject key_erl = iter_tpl.elementAt(0);\n                        final String key = ErlangValue.otpObjectToString(key_erl);\n                        final Type elementType = getTypeOf(key);\n                        final Method setter = getSetterFor(key, elementType);\n                        final Object myValue = convertScalarisJSONtoJava_value2(\n                                iter_tpl.elementAt(1), elementType);\n                        setter.invoke(result, myValue);\n                    } catch (final Exception e) {\n                        throw new ClassCastException(\n                                \"Cannot store value to JSON object (value: \"\n                                        + value.toString() + \"): \"\n                                        + e.getMessage());\n                    }\n                } else {\n                    throw new ClassCastException(\n                            \"Unsupported JSON type (value: \" + value.toString()\n                                    + \")\");\n                }\n            }\n            return result;\n        }\n    }\n\n    /**\n     * Converts a JSON array value (a list of values) to a Java List.\n     *\n     * @param value\n     *            a list of JSON values as stored by Scalaris' JSON API\n     * @param type_\n     *            the (complete) type of the list, including parameters\n     *\n     * @return a Java object representing the value\n     *\n     * @throws ClassCastException\n     *                if thrown if a conversion is not possible, i.e. the type\n     *                is not supported\n     */\n    protected List<Object> convertScalarisJSONtoJava_array2(final OtpErlangList value, final Type type_)\n            throws ClassCastException {\n        final List<Object> result = new ArrayList<Object>(value.arity());\n        for (final OtpErlangObject iter : value) {\n            if (type_ instanceof ParameterizedType) {\n                final ParameterizedType type = (ParameterizedType) type_;\n                final Type[] typeArguments = type.getActualTypeArguments();\n                if (typeArguments.length == 1) {\n                    result.add(convertScalarisJSONtoJava_value2(iter, typeArguments[0]));\n                    continue;\n                }\n            } else {\n                // note: list type could be Object if encapsulated in another type\n                result.add(convertScalarisJSONtoJava_value2(iter, Object.class));\n            }\n        }\n        return result;\n    }\n\n    /**\n     * Converts an unknown JSON value to a Java object.\n     *\n     * @param value\n     *            a JSON value as stored by Scalaris' JSON API\n     * @param type\n     *            the (supposed) type of the returned value\n     *            (hint for container types)\n     *\n     * @return a Java object representing the value\n     *\n     * @throws ClassCastException\n     *                if thrown if a conversion is not possible, i.e. the type\n     *                is not supported\n     */\n    protected Object convertScalarisJSONtoJava_value2(final OtpErlangObject value, final Type type)\n            throws ClassCastException {\n        if (value instanceof OtpErlangTuple) {\n            final OtpErlangTuple value_tpl = (OtpErlangTuple) value;\n            if (value_tpl.arity() == 2) {\n                final OtpErlangObject tag = value_tpl.elementAt(0);\n                if (tag.equals(CommonErlangObjects.structAtom)) {\n                    // converting an object\n                    final OtpErlangList value_obj = (OtpErlangList) value_tpl.elementAt(1);\n                    final Type type1 = type.equals(Object.class) ? Map.class : type;\n                    final ErlangValueJSONToBean<?> json_converter = getInstance(type1);\n                    return json_converter.convertScalarisJSONtoJava_object2(value_obj, type1);\n                } else if (tag.equals(CommonErlangObjects.arrayAtom)) {\n                    // converting a list\n                    final OtpErlangList value_list = ErlangValue.otpObjectToOtpList(value_tpl.elementAt(1));\n                    return convertScalarisJSONtoJava_array2(value_list, type);\n                } else {\n                    throw new ClassCastException(\"unknown JSON tag\");\n                }\n            } else {\n                throw new ClassCastException(\"wrong tuple arity\");\n            }\n        } else {\n            return super.convertScalarisJSONtoJava_value_simple(value);\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/ErlangValueJSONToMap.java",
    "content": "/**\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.ArrayList;\nimport java.util.LinkedHashMap;\nimport java.util.List;\nimport java.util.Map;\n\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\n/**\n * Converts Scalaris-JSON to {@link Map}&lt;String, Object&gt; and such maps to\n * Scalaris-JSON.\n *\n * @author Nico Kruber, kruber@zib.de\n */\nclass ErlangValueJSONToMap extends ErlangValueJSONBase\n        implements ErlangValueJSONInterface<Map<String, Object>> {\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.ErlangValueJSON#toScalarisJSON(T)\n     */\n    @Override\n    public OtpErlangTuple toScalarisJSON(final Map<String, Object> value)\n            throws ClassCastException {\n        return convertJavaToScalarisJSON_object(value);\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.ErlangValueJSON#toJava(com.ericsson.otp.erlang.OtpErlangList)\n     */\n    @Override\n    @SuppressWarnings(\"unchecked\")\n    public Map<String, Object> toJava(final OtpErlangList value)\n            throws ClassCastException {\n        return (Map<String, Object>) convertScalarisJSONtoJava_object(value);\n    }\n\n    /**\n     * Converts a Java Map to a JSON object as expected by Scalaris.\n     *\n     * @param value_\n     *            a Java Map with String-keys and supported JSON types as values\n     *\n     * @return a JSON object representing the value\n     *\n     * @throws ClassCastException\n     *             if thrown if a conversion is not possible, i.e. the type is\n     *             not supported\n     */\n    @Override\n    protected OtpErlangTuple convertJavaToScalarisJSON_object(final Object value_)\n            throws ClassCastException {\n        try {\n            @SuppressWarnings(\"unchecked\")\n            final\n            Map<String, Object> value = (Map<String, Object>) value_;\n            final OtpErlangTuple[] resultList = new OtpErlangTuple[value.size()];\n            int i = 0;\n\n            for (final Map.Entry<String, Object> entry : value.entrySet()) {\n                resultList[i] = new OtpErlangTuple(new OtpErlangObject[] {\n                        new OtpErlangString(entry.getKey()),\n                        convertJavaToScalarisJSON_value(entry.getValue()) });\n                ++i;\n            }\n            final OtpErlangTuple resultTpl = new OtpErlangTuple(new OtpErlangObject[] {\n                    CommonErlangObjects.structAtom, new OtpErlangList(resultList) });\n            return resultTpl;\n        } catch (final ClassCastException e) {\n            throw new ClassCastException(\"Unsupported JSON type (value: \" + value_.toString() + \")\");\n        }\n    }\n\n    /**\n     * Converts a JSON object value (a list of key/value pairs) to a Java Map.\n     *\n     * @param value\n     *            a list of key/value pairs with JSON values and string keys as\n     *            stored by Scalaris' JSON API\n     *\n     * @return a Java object representing the value\n     *\n     * @throws ClassCastException\n     *                if thrown if a conversion is not possible, i.e. the type\n     *                is not supported\n     */\n    protected Object convertScalarisJSONtoJava_object(final OtpErlangList value)\n            throws ClassCastException {\n        final Map<String, Object> result = new LinkedHashMap<String, Object>(\n                value.arity());\n        for (final OtpErlangObject iter : value) {\n            final OtpErlangTuple iter_tpl = (OtpErlangTuple) iter;\n            if (iter_tpl.arity() == 2) {\n                final String key = convertScalarisJSONtoJava_key(iter_tpl.elementAt(0));\n                result.put(key,\n                        convertScalarisJSONtoJava_value(iter_tpl.elementAt(1)));\n            } else {\n                throw new ClassCastException(\"Unsupported JSON type (value: \"\n                        + value.toString() + \")\");\n            }\n        }\n        return result;\n    }\n\n    /**\n     * Converts an unknown JSON value to a Java object.\n     *\n     * @param value\n     *            a JSON value as stored by Scalaris' JSON API\n     *\n     * @return a Java object representing the value\n     *\n     * @throws ClassCastException\n     *                if thrown if a conversion is not possible, i.e. the type\n     *                is not supported\n     */\n    protected Object convertScalarisJSONtoJava_value(final OtpErlangObject value)\n            throws ClassCastException {\n        if (value instanceof OtpErlangTuple) {\n            final OtpErlangTuple value_tpl = (OtpErlangTuple) value;\n            if (value_tpl.arity() == 2) {\n                final OtpErlangObject tag = value_tpl.elementAt(0);\n                if (tag.equals(CommonErlangObjects.structAtom)) {\n                    return convertScalarisJSONtoJava_object((OtpErlangList) value_tpl\n                            .elementAt(1));\n                } else if (tag.equals(CommonErlangObjects.arrayAtom)) {\n                    return convertScalarisJSONtoJava_array(\n                            ErlangValue.otpObjectToOtpList(value_tpl.elementAt(1)));\n                } else {\n                    throw new ClassCastException(\"unknown JSON tag\");\n                }\n            } else {\n                throw new ClassCastException(\"wrong tuple arity\");\n            }\n        } else {\n            return super.convertScalarisJSONtoJava_value_simple(value);\n        }\n    }\n\n    /**\n     * Converts a JSON array value (a list of values) to a Java List.\n     *\n     * @param value\n     *            a list of JSON values as stored by Scalaris' JSON API\n     *\n     * @return a Java object representing the value\n     *\n     * @throws ClassCastException\n     *                if thrown if a conversion is not possible, i.e. the type\n     *                is not supported\n     */\n    protected List<Object> convertScalarisJSONtoJava_array(\n            final OtpErlangList value) throws ClassCastException {\n        final List<Object> result = new ArrayList<Object>(value.arity());\n        for (final OtpErlangObject iter : value) {\n            result.add(convertScalarisJSONtoJava_value(iter));\n        }\n        return result;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/FixedNodeConnectionPolicy.java",
    "content": "package de.zib.scalaris;\n\n/**\n * Implements a {@link ConnectionPolicy} which only supports a single node\n * and does not issue automatic re-tries.\n *\n * @author Nico Kruber, kruber@zib.de\n *\n * @version 3.10\n * @since 3.10\n *\n * @see DefaultConnectionPolicy\n */\npublic class FixedNodeConnectionPolicy extends ConnectionPolicy {\n    /**\n     * Creates a new connection policy working with the given remote node.\n     *\n     * @param remoteNode the (only) available remote node\n     */\n    public FixedNodeConnectionPolicy(final PeerNode remoteNode) {\n        super(remoteNode);\n    }\n    /**\n     * Creates a new connection policy working with the given remote node.\n     *\n     * @param remoteNode the (only) available remote node\n     */\n    public FixedNodeConnectionPolicy(final String remoteNode) {\n        this(new PeerNode(remoteNode));\n    }\n\n    @Override\n    public <E extends Exception> PeerNode selectNode(final int retry,\n            final PeerNode failedNode, final E e) throws E, UnsupportedOperationException {\n        // no re-try, i.e. re-throw any previous exception:\n        if (e != null) {\n            throw e;\n        }\n        return availableRemoteNodes.get(0);\n    }\n}"
  },
  {
    "path": "java-api/src/de/zib/scalaris/KeyChangedException.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\n/**\n * Exception that is thrown if a test_and_set operation on a scalaris ring\n * fails because the old value did not match the expected value.\n *\n * Contains the old value stored in scalaris.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.7\n * @since 2.7\n */\npublic class KeyChangedException extends OtpErlangException {\n    /**\n     * class version for serialisation\n     */\n    private static final long serialVersionUID = 1L;\n\n    /**\n     * The value stored in scalaris.\n     */\n    private final ErlangValue oldValue;\n\n    /**\n     * Creates the exception with the given old value.\n     *\n     * @param old_value\n     *            the old value stored in scalaris\n     */\n    public KeyChangedException(final ErlangValue old_value) {\n        super();\n        this.oldValue = old_value;\n    }\n\n    /**\n     * Creates the exception with the given old value taking the message of the\n     * given throwable.\n     *\n     * @param e\n     *            the exception to \"re-throw\"\n     * @param old_value\n     *            the old value stored in scalaris\n     */\n    public KeyChangedException(final Throwable e, final ErlangValue old_value) {\n        super(e.getMessage());\n        this.oldValue = old_value;\n        setStackTrace(e.getStackTrace());\n    }\n\n    /**\n     * Returns the (old) value stored in scalaris.\n     *\n     * @return the value\n     */\n    public ErlangValue getOldValue() {\n        return oldValue;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/LeastFailedNodesComparator.java",
    "content": "/**\n *  Copyright 2007-2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.Comparator;\nimport java.util.Date;\n\n/**\n * Defines the order of the {@link PeerNode} objects with least recently\n * failed nodes being first.\n *\n * @author Nico Kruber, kruber@zib.de\n *\n * @version 3.16\n * @since 3.16\n */\nclass LeastFailedNodesComparator implements Comparator<PeerNode>, java.io.Serializable {\n    /**\n     * ID for serialisation purposes.\n     */\n    private static final long serialVersionUID = 1L;\n\n    /**\n     * Defines a order of nodes with failed connections (least recently failed\n     * nodes first).\n     *\n     * Returns a negative integer, zero, or a positive integer if the first\n     * argument is less than, equal to, or greater than the second.\n     *\n     * Warning: This method is unsynchronised! Special care needs to be taken\n     * using it on collections of nodes, i.e. the node list in\n     * {@link DefaultConnectionPolicy}!\n     *\n     * @param o1\n     *            the first node\n     * @param o2\n     *            the second node\n     *\n     * @return a negative integer, zero, or a positive integer if the first\n     *         argument is less than, equal to, or greater than the second\n     */\n    public int compare(final PeerNode o1, final PeerNode o2) {\n        return LeastFailedNodesComparator.compareS(o1, o2);\n    }\n\n    /**\n     * Defines a order of nodes with failed connections (least recently failed\n     * nodes first).\n     *\n     * Returns a negative integer, zero, or a positive integer if the first\n     * argument is less than, equal to, or greater than the second.\n     *\n     * Warning: This method is unsynchronised! Special care needs to be taken\n     * using it on collections of nodes, i.e. the node list in\n     * {@link DefaultConnectionPolicy}!\n     *\n     * @param o1\n     *            the first node\n     * @param o2\n     *            the second node\n     *\n     * @return a negative integer, zero, or a positive integer if the first\n     *         argument is less than, equal to, or greater than the second\n     */\n    public static int compareS(final PeerNode o1, final PeerNode o2) {\n        if (o1 == o2) {\n            return 0;\n        }\n\n        final Integer o1FailureCount = o1.getFailureCount();\n        final Integer o2FailureCount = o2.getFailureCount();\n        final int compByFailureCount = o1FailureCount.compareTo(o2FailureCount);\n\n        if (compByFailureCount != 0) {\n            return compByFailureCount;\n        }\n\n        // two different nodes have the same fail counts\n        // -> make order dependent on their last successful connection date\n        final Date d1 = o1.getLastConnectSuccess();\n        final Date d2 = o2.getLastConnectSuccess();\n        final Long o1Time = ((d1 == null) ? 0 : d1.getTime());\n        final Long o2Time = ((d2 == null) ? 0 : d2.getTime());\n        // most recently connection success first:\n        final int compByTime = o2Time.compareTo(o1Time);\n\n        if (compByTime != 0) {\n            return compByTime;\n        }\n\n        // -> make order dependent on their hash code:\n        final int h1 = o1.hashCode();\n        final int h2 = o2.hashCode();\n        if (h1 < h2) {\n            return -1;\n        } else if (h1 > h2){\n            return 1;\n        } else {\n            // two different nodes have equal fail dates and hash codes\n            // -> compare their names (last resort)\n            final int compByName = o1.getNode().node().compareTo(o2.getNode().node());\n            if (compByName != 0) {\n                return compByName;\n            } else {\n                throw new RuntimeException(\"Cannot compare \" + o1 + \" with \"\n                        + o2 + \".\");\n            }\n        }\n    }\n}"
  },
  {
    "path": "java-api/src/de/zib/scalaris/LeastRecentlyFailedNodesComparator.java",
    "content": "/**\n *  Copyright 2007-2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.Comparator;\nimport java.util.Date;\n\n/**\n * Defines the order of the {@link PeerNode} objects with least recently\n * failed nodes being first.\n *\n * @author Nico Kruber, kruber@zib.de\n *\n * @version 2.3\n * @since 2.3\n */\nclass LeastRecentlyFailedNodesComparator implements Comparator<PeerNode>, java.io.Serializable {\n    /**\n     * ID for serialisation purposes.\n     */\n    private static final long serialVersionUID = 1L;\n\n    /**\n     * Defines a order of nodes with failed connections (least recently failed\n     * nodes first).\n     *\n     * Returns a negative integer, zero, or a positive integer if the first\n     * argument is less than, equal to, or greater than the second.\n     *\n     * Warning: This method is unsynchronised! Special care needs to be taken\n     * using it on collections of nodes, i.e. the node list in\n     * {@link DefaultConnectionPolicy}!\n     *\n     * @param o1\n     *            the first node\n     * @param o2\n     *            the second node\n     *\n     * @return a negative integer, zero, or a positive integer if the first\n     *         argument is less than, equal to, or greater than the second\n     */\n    public int compare(final PeerNode o1, final PeerNode o2) {\n        if (o1 == o2) {\n            return 0;\n        }\n\n        final Date d1 = o1.getLastFailedConnect();\n        final Date d2 = o2.getLastFailedConnect();\n        final Long o1Time = ((d1 == null) ? 0 : d1.getTime());\n        final Long o2Time = ((d2 == null) ? 0 : d2.getTime());\n        final int compByTime = o1Time.compareTo(o2Time);\n\n        if (compByTime != 0) {\n            return compByTime;\n        }\n\n        return LeastFailedNodesComparator.compareS(o1, o2);\n    }\n}"
  },
  {
    "path": "java-api/src/de/zib/scalaris/Main.java",
    "content": "/*\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.lang.management.ManagementFactory;\nimport java.text.DecimalFormat;\nimport java.util.Comparator;\nimport java.util.HashSet;\nimport javax.management.InstanceAlreadyExistsException;\nimport javax.management.MBeanRegistrationException;\nimport javax.management.MBeanServer;\nimport javax.management.MalformedObjectNameException;\nimport javax.management.NotCompliantMBeanException;\nimport javax.management.ObjectName;\n\nimport org.apache.commons.cli.CommandLine;\nimport org.apache.commons.cli.CommandLineParser;\nimport org.apache.commons.cli.GnuParser;\nimport org.apache.commons.cli.HelpFormatter;\nimport org.apache.commons.cli.Option;\nimport org.apache.commons.cli.OptionGroup;\nimport org.apache.commons.cli.Options;\nimport org.apache.commons.cli.ParseException;\n\nimport de.zib.scalaris.Monitor.GetNodeInfoResult;\nimport de.zib.scalaris.Monitor.GetNodePerformanceResult;\nimport de.zib.scalaris.Monitor.GetServiceInfoResult;\nimport de.zib.scalaris.Monitor.GetServicePerformanceResult;\n\n/**\n * Class to test basic functionality of the package and to use scalaris\n * from command line.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.0\n * @since 2.0\n */\npublic class Main {\n    /**\n     * Queries the command line options for an action to perform.\n     *\n     * <pre>\n     * <code>\n     * > java -jar scalaris.jar --help\n     * usage: scalaris [Options]\n     *  -h,--help                                   print this message\n     *  -v,--verbose                                print verbose information,\n     *                                              e.g. the properties read\n     *  -lh,--localhost                             gets the local host's name as\n     *                                              known to Java (for debugging\n     *                                              purposes)\n     *  -b,--minibench <[ops]> <[tpn]> <[benchs]>   run selected mini\n     *                                              benchmark(s) [1|...|18|all]\n     *                                              (default: all benchmarks, 500\n     *                                              operations, 10 threads per\n     *                                              Scalaris node)\n     *  -m,--monitor <node>                         print monitoring information\n     *  -r,--read <key>                             read an item\n     *  -w,--write <key> <value>                    write an item\n     *     --test-and-set <key> <old> <new>         atomic test and set, i.e.\n     *                                              write <key> to <new> if the\n     *                                              current value is <old>\n     *  -d,--delete <key> <[timeout]>               delete an item (default\n     *                                              timeout: 2000ms)\n     *                                              WARNING: This function can\n     *                                              lead to inconsistent data\n     *                                              (e.g. deleted items can\n     *                                              re-appear). Also when\n     *                                              re-creating an item the\n     *                                              version before the delete can\n     *                                              re-appear.\n     *  -jmx,--jmxservice <node>                    starts a service exposing\n     *                                              Scalaris monitoring values\n     *                                              via JMX\n     * </code>\n     * </pre>\n     *\n     * In order to override node and cookie to use for a connection, specify\n     * the <tt>scalaris.node</tt> or <tt>scalaris.cookie</tt> system properties.\n     * Their values will be used instead of the values defined in the config\n     * file!\n     *\n     * @param args\n     *            command line arguments\n     */\n    public static void main(final String[] args) {\n        boolean verbose = false;\n        final CommandLineParser parser = new GnuParser();\n        CommandLine line = null;\n        final Options options = getOptions();\n        try {\n            line = parser.parse(options, args);\n        } catch (final ParseException e) {\n            printException(\"Parsing failed\", e, false);\n            return; // will not be reached since printException exits\n        }\n\n        if (line.hasOption(\"verbose\")) {\n            verbose = true;\n            ConnectionFactory.getInstance().printProperties();\n        }\n\n        if (line.hasOption(\"minibench\")) {\n            final String[] optionValues = line.getOptionValues(\"minibench\");\n            int nrOperations = 500;\n            int threadsPerNode = 10;\n            final HashSet<Integer> benchmarks = new HashSet<Integer>(18);\n            boolean all = true;\n            if (optionValues != null) {\n                checkArguments(optionValues, 0, options, \"b\");\n                if (optionValues.length >= 1) {\n                    nrOperations = Integer.parseInt(optionValues[0]);\n                }\n                if (optionValues.length >= 2) {\n                    threadsPerNode = Integer.parseInt(optionValues[1]);\n                }\n                if (optionValues.length >= 3) {\n                    all = false;\n                    for (int i = 2; i < Math.min(20, optionValues.length); ++i) {\n                        final String benchmarks_str = optionValues[i];\n                        if (benchmarks_str.equals(\"all\")) {\n                            all = true;\n                        } else {\n                            benchmarks.add(Integer.parseInt(benchmarks_str));\n                        }\n                    }\n                }\n            }\n            if (all) {\n                for (int i = 1; i <= 18; ++i) {\n                    benchmarks.add(i);\n                }\n            }\n            Benchmark.minibench(nrOperations, threadsPerNode, benchmarks);\n        } else if (line.hasOption(\"r\")) { // read\n            final String key = line.getOptionValue(\"read\");\n            checkArguments(key, options, \"r\");\n            try {\n                final TransactionSingleOp sc = new TransactionSingleOp();\n                final String value = sc.read(key).value().toString();\n                System.out.println(\"read(\" + key + \") == \" + value);\n            } catch (final ConnectionException e) {\n                printException(\"read failed with connection error\", e, verbose);\n            } catch (final NotFoundException e) {\n                printException(\"read failed with not found\", e, verbose);\n            } catch (final UnknownException e) {\n                printException(\"read failed with unknown\", e, verbose);\n            }\n        } else if (line.hasOption(\"w\")) { // write\n            final String[] optionValues = line.getOptionValues(\"write\");\n            checkArguments(optionValues, 2, options, \"w\");\n            final String key = optionValues[0];\n            final String value = optionValues[1];\n            try {\n                final TransactionSingleOp sc = new TransactionSingleOp();\n                sc.write(key, value);\n                System.out.println(\"write(\" + key + \", \" + value + \"): ok\");\n            } catch (final ConnectionException e) {\n                printException(\"write failed with connection error\", e, verbose);\n            } catch (final AbortException e) {\n                printException(\"write failed with abort\", e, verbose);\n            } catch (final UnknownException e) {\n                printException(\"write failed with unknown\", e, verbose);\n            }\n        } else if (line.hasOption(\"test-and-set\")) { // test_and_set\n            final String[] optionValues = line.getOptionValues(\"test-and-set\");\n            checkArguments(optionValues, 3, options, \"test-and-set\");\n            final String key = optionValues[0];\n            final String oldValue = optionValues[1];\n            final String newValue = optionValues[2];\n            try {\n                final TransactionSingleOp sc = new TransactionSingleOp();\n                sc.testAndSet(key, oldValue, newValue);\n                System.out.println(\"testAndSet(\" + key + \", \" + oldValue + \", \" + newValue + \"): ok\");\n            } catch (final ConnectionException e) {\n                printException(\"testAndSet failed with connection error\", e, verbose);\n            } catch (final AbortException e) {\n                printException(\"testAndSet failed with abort\", e, verbose);\n            } catch (final UnknownException e) {\n                printException(\"testAndSet failed with unknown\", e, verbose);\n            } catch (final NotFoundException e) {\n                printException(\"testAndSet failed with not found\", e, verbose);\n            } catch (final KeyChangedException e) {\n                printException(\"testAndSet failed with key changed (current value: \" + e.getOldValue().toString() + \")\", e, verbose);\n            }\n        } else if (line.hasOption(\"d\")) { // delete\n            final String[] optionValues = line.getOptionValues(\"delete\");\n            checkArguments(optionValues, 1, options, \"d\");\n            final String key = optionValues[0];\n            int timeout = 2000;\n            if (optionValues.length >= 2) {\n                try {\n                    timeout = Integer.parseInt(optionValues[1]);\n                } catch (final Exception e) {\n                    printException(\"Parsing failed\", new ParseException(\n                            \"wrong type for timeout parameter of option d\"\n                                    + \" (parameters: <\"\n                                    + options.getOption(\"d\").getArgName()\n                                    + \">)\"), verbose);\n                }\n            }\n            try {\n                final ReplicatedDHT sc = new ReplicatedDHT();\n                final DeleteResult deleteResult = sc.delete(key, timeout);\n                System.out.println(\"delete(\" + key + \", \" + timeout + \"): \"\n                        + deleteResult.ok + \" ok, \"\n                        + deleteResult.locks_set + \" locks_set, \"\n                        + deleteResult.undef + \" undef\");\n            } catch (final ConnectionException e) {\n                printException(\"delete failed with connection error\", e, verbose);\n            } catch (final TimeoutException e) {\n                printException(\"delete failed with timeout\", e, verbose);\n            } catch (final UnknownException e) {\n                printException(\"delete failed with unknown error\", e, verbose);\n            }\n        } else if (line.hasOption(\"lh\")) { // get local host name\n            System.out.println(ConnectionFactory.getLocalhostName());\n        } else if (line.hasOption(\"monitor\")) { // print monitoring data\n            final String node = line.getOptionValue(\"monitor\");\n            checkArguments(node, options, \"monitor\");\n            try {\n                final Monitor monitor = new Monitor(node);\n                final GetNodeInfoResult nodeInfo = monitor.getNodeInfo();\n                final GetNodePerformanceResult nodePerf = monitor.getNodePerformance();\n                final GetServiceInfoResult srvInfo = monitor.getServiceInfo();\n                final GetServicePerformanceResult srvPerf = monitor.getServicePerformance();\n\n                final Double nodePerfCurLatAvg = Monitor.getCurrentPerfValue(nodePerf.latencyAvg);\n                final Double nodePerfCurLatStddev = Monitor.getCurrentPerfValue(nodePerf.latencyStddev);\n                final Double srvPerfCurLatAvg = Monitor.getCurrentPerfValue(srvPerf.latencyAvg);\n                final Double srcPerfCurLatStddev = Monitor.getCurrentPerfValue(srvPerf.latencyStddev);\n\n                final DecimalFormat df = new DecimalFormat(\"0.##\");\n                System.out.println(\"== Node Info ==\");\n                System.out.println(\"Scalaris version: \" + nodeInfo.scalarisVersion);\n                System.out.println(\"Erlang   version: \" + nodeInfo.erlangVersion);\n                System.out.println(\"# of DHT nodes  : \" + nodeInfo.dhtNodes);\n                System.out.println(\"== Service Info (from mgmt_server) ==\");\n                System.out.println(\"Total # of nodes: \" + srvInfo.nodes);\n                System.out.println(\"Total load      : \" + srvInfo.totalLoad);\n                System.out.println(\"== Node Performance ==\");\n                System.out.println(\"Current latency : \" + (nodePerfCurLatAvg == null ? \"n/a\" : df.format(nodePerfCurLatAvg)));\n                System.out.println(\"Current stddev  : \" + (nodePerfCurLatStddev == null ? \"n/a\" : df.format(nodePerfCurLatStddev)));\n                System.out.println(\"== Service Performance ==\");\n                System.out.println(\"Current latency : \" + (srvPerfCurLatAvg == null ? \"n/a\" : df.format(srvPerfCurLatAvg)));\n                System.out.println(\"Current stddev  : \" + (srcPerfCurLatStddev == null ? \"n/a\" : df.format(srcPerfCurLatStddev)));\n            } catch (final ConnectionException e) {\n                printException(\"monitor failed with connection error\", e, verbose);\n            } catch (final UnknownException e) {\n                printException(\"monitor failed with unknown error\", e, verbose);\n            }\n        } else if (line.hasOption(\"jmx\")) { // start JMX monitoring service\n            final String node = line.getOptionValue(\"jmx\");\n            checkArguments(node, options, \"jmx\");\n            startJmxService(node, verbose);\n        } else {\n            // print help if no other option was given\n//        if (line.hasOption(\"help\")) {\n            final HelpFormatter formatter = new HelpFormatter();\n            formatter.setOptionComparator(new Comparator<Option>() {\n                private int optionToInt(final Option option) {\n                    if (option.getLongOpt().equals(\"help\")) {\n                        return 1;\n                    } else if (option.getLongOpt().equals(\"verbose\")) {\n                        return 2;\n                    } else if (option.getLongOpt().equals(\"localhost\")) {\n                        return 3;\n                    } else if (option.getLongOpt().equals(\"minibench\")) {\n                        return 4;\n                    } else if (option.getLongOpt().equals(\"monitor\")) {\n                        return 5;\n                    } else if (option.getLongOpt().equals(\"read\")) {\n                        return 6;\n                    } else if (option.getLongOpt().equals(\"write\")) {\n                        return 7;\n                    } else if (option.getLongOpt().equals(\"test-and-set\")) {\n                        return 8;\n                    } else if (option.getLongOpt().equals(\"add-del-on-list\")) {\n                        return 9;\n                    } else if (option.getLongOpt().equals(\"add-on-nr\")) {\n                        return 10;\n                    } else if (option.getLongOpt().equals(\"delete\")) {\n                        return 11;\n                    } else if (option.getLongOpt().equals(\"jmxservice\")) {\n                        return 12;\n                    } else {\n                        return 13;\n                    }\n                }\n\n                public int compare(final Option arg0, final Option arg1) {\n                    final int arg0_i = optionToInt(arg0);\n                    final int arg1_i = optionToInt(arg1);\n                    if (arg0_i < arg1_i) {\n                        return -1;\n                    } else if (arg0_i == arg1_i) {\n                        return 0;\n                    } else {\n                        return 1;\n                    }\n                }\n            });\n            formatter.printHelp(\"scalaris [Options]\", getOptions());\n            if (!line.hasOption(\"help\")) {\n                System.exit(1);\n            }\n        }\n    }\n\n    /**\n     * Registers some MBeans to monitor Scalaris via JMX and then waits forever\n     * until interrupted.\n     *\n     * @param node\n     *            the node name of the Erlang VM to connect to\n     * @param verbose\n     *            whether verbose information should be printed in case of\n     *            connection failures\n     */\n    private static void startJmxService(final String node, final boolean verbose) {\n        try {\n            final MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();\n            final ObjectName nodeMonitorName = new ObjectName(\"de.zib.scalaris:type=MonitorNode\");\n            final de.zib.scalaris.jmx.MonitorNode nodeMonitorMbean = new de.zib.scalaris.jmx.MonitorNode(node);\n            final ObjectName serviceMonitorName = new ObjectName(\"de.zib.scalaris:type=MonitorService\");\n            final de.zib.scalaris.jmx.MonitorService serviceMonitorMbean = new de.zib.scalaris.jmx.MonitorService(node);\n            mbs.registerMBean(nodeMonitorMbean, nodeMonitorName);\n            mbs.registerMBean(serviceMonitorMbean, serviceMonitorName);\n            System.out.println(\"Waiting forever...\");\n            Thread.sleep(Long.MAX_VALUE);\n        } catch (final InterruptedException e) {\n            System.out.println(\"stopped service\");\n        } catch (final MalformedObjectNameException e) {\n            throw new RuntimeException(e);\n        } catch (final NullPointerException e) {\n            throw new RuntimeException(e);\n        } catch (final ConnectionException e) {\n            printException(\"JMX service failed with connection error\", e, verbose);\n        } catch (final InstanceAlreadyExistsException e) {\n            throw new RuntimeException(e);\n        } catch (final MBeanRegistrationException e) {\n            throw new RuntimeException(e);\n        } catch (final NotCompliantMBeanException e) {\n            throw new RuntimeException(e);\n        }\n    }\n\n    /**\n     * Creates the options the command line should understand.\n     *\n     * @return the options the program understands\n     */\n    private static Options getOptions() {\n        final Options options = new Options();\n        final OptionGroup group = new OptionGroup();\n\n        /* Note: arguments are set to be optional since we implement argument\n         * checks on our own (commons.cli is not flexible enough and only\n         * checks for the existence of a first argument)\n         */\n\n        options.addOption(new Option(\"h\", \"help\", false, \"print this message\"));\n\n        options.addOption(new Option(\"v\", \"verbose\", false, \"print verbose information, e.g. the properties read\"));\n\n        final Option read = new Option(\"r\", \"read\", true, \"read an item\");\n        read.setArgName(\"key\");\n        read.setArgs(1);\n        read.setOptionalArg(true);\n        group.addOption(read);\n\n        final Option write = new Option(\"w\", \"write\", true, \"write an item\");\n        write.setArgName(\"key> <value\");\n        write.setArgs(2);\n        write.setOptionalArg(true);\n        group.addOption(write);\n\n        final Option test_and_set = new Option(null, \"test-and-set\", true,\n                \"atomic test and set, i.e. write <key> to \" +\n                \"<new> if the current value is <old>\");\n        test_and_set.setArgName(\"key> <old> <new\");\n        test_and_set.setArgs(3);\n        test_and_set.setOptionalArg(true);\n        group.addOption(test_and_set);\n\n        final Option delete = new Option(\"d\", \"delete\", true,\n                \"delete an item (default timeout: 2000ms)\\n\" +\n                \"WARNING: This function can lead to inconsistent data (e.g. \" +\n                \"deleted items can re-appear). Also when re-creating an item \" +\n                \"the version before the delete can re-appear.\");\n        delete.setArgName(\"key> <[timeout]\");\n        delete.setArgs(2);\n        delete.setOptionalArg(true);\n        group.addOption(delete);\n\n        final Option bench = new Option(\"b\", \"minibench\", true, \"run selected mini benchmark(s) [1|...|18|all] (default: all benchmarks, 500 operations, 10 threads per Scalaris node)\");\n        bench.setArgName(\"[ops]> <[tpn]> <[benchs]\");\n        bench.setArgs(20);\n        bench.setOptionalArg(true);\n        group.addOption(bench);\n\n        final Option monitor = new Option(\"m\", \"monitor\", true, \"print monitoring information\");\n        monitor.setArgName(\"node\");\n        monitor.setArgs(1);\n        monitor.setOptionalArg(true);\n        group.addOption(monitor);\n\n        final Option jmx = new Option(\"jmx\", \"jmxservice\", true, \"starts a service exposing Scalaris monitoring values via JMX\");\n        jmx.setArgName(\"node\");\n        jmx.setArgs(1);\n        jmx.setOptionalArg(true);\n        group.addOption(jmx);\n\n        options.addOptionGroup(group);\n\n        options.addOption(new Option(\"lh\", \"localhost\", false, \"gets the local host's name as known to Java (for debugging purposes)\"));\n\n\n        return options;\n    }\n\n    /**\n     * Prints the given exception with the given description and terminates the\n     * JVM.\n     *\n     * @param description  will be prepended to the error message\n     * @param e            the exception to print\n     * @param verbose      specifies whether to include the stack trace or not\n     */\n    final static void printException(final String description, final ParseException e, final boolean verbose) {\n        printException(description, e, verbose, 1);\n    }\n\n    /**\n     * Prints the given exception with the given description and terminates the\n     * JVM.\n     *\n     * @param description  will be prepended to the error message\n     * @param e            the exception to print\n     * @param verbose      specifies whether to include the stack trace or not\n     */\n    final static void printException(final String description, final ConnectionException e, final boolean verbose) {\n        printException(description, e, verbose, 2);\n    }\n\n    /**\n     * Prints the given exception with the given description and terminates the\n     * JVM.\n     *\n     * @param description  will be prepended to the error message\n     * @param e            the exception to print\n     * @param verbose      specifies whether to include the stack trace or not\n     */\n    final static void printException(final String description, final TimeoutException e, final boolean verbose) {\n        printException(description, e, verbose, 3);\n    }\n\n    /**\n     * Prints the given exception with the given description and terminates the\n     * JVM.\n     *\n     * @param description  will be prepended to the error message\n     * @param e            the exception to print\n     * @param verbose      specifies whether to include the stack trace or not\n     */\n    final static void printException(final String description, final NotFoundException e, final boolean verbose) {\n        printException(description, e, verbose, 4);\n    }\n\n    /**\n     * Prints the given exception with the given description and terminates the\n     * JVM.\n     *\n     * @param description  will be prepended to the error message\n     * @param e            the exception to print\n     * @param verbose      specifies whether to include the stack trace or not\n     */\n    final static void printException(final String description, final UnknownException e, final boolean verbose) {\n        printException(description, e, verbose, 5);\n    }\n\n    /**\n     * Prints the given exception with the given description and terminates the\n     * JVM.\n     *\n     * @param description  will be prepended to the error message\n     * @param e            the exception to print\n     * @param verbose      specifies whether to include the stack trace or not\n     */\n    final static void printException(final String description, final AbortException e, final boolean verbose) {\n        printException(description, e, verbose, 7);\n    }\n\n    /**\n     * Prints the given exception with the given description and terminates the\n     * JVM.\n     *\n     * @param description  will be prepended to the error message\n     * @param e            the exception to print\n     * @param verbose      specifies whether to include the stack trace or not\n     */\n    final static void printException(final String description, final KeyChangedException e, final boolean verbose) {\n        printException(description, e, verbose, 8);\n    }\n\n    /**\n     * Prints the given exception with the given description and terminates the\n     * JVM.\n     *\n     * Note: exit status 6 was used by the NodeNotFoundException which is not\n     * needed anymore - do not re-use this exit status!\n     *\n     * @param description  will be prepended to the error message\n     * @param e            the exception to print\n     * @param verbose      specifies whether to include the stack trace or not\n     * @param exitStatus   the status code the JVM exits with\n     */\n    final static void printException(final String description, final Exception e, final boolean verbose, final int exitStatus) {\n        System.err.print(description + \": \");\n        if (verbose) {\n            System.err.println();\n            e.printStackTrace();\n        } else {\n            System.err.println(e.getMessage());\n        }\n        System.exit(exitStatus);\n    }\n\n    /**\n     * Checks that the given option value as returned from e.g.\n     * {@link CommandLine#getOptionValue(String)} does exist and prints an error\n     * message if not.\n     *\n     * @param optionValue    the value to check\n     * @param options        the available command line options\n     * @param currentOption  the short name of the current option being parsed\n     */\n    final static void checkArguments(final String optionValue,\n            final Options options, final String currentOption) {\n        if (optionValue == null) {\n            printException(\"Parsing failed\", new ParseException(\n                    \"missing parameter for option \" + currentOption\n                            + \" (required: <\"\n                            + options.getOption(currentOption).getArgName()\n                            + \">)\"), false);\n        }\n    }\n\n    /**\n     * Checks that the given option values as returned from e.g.\n     * {@link CommandLine#getOptionValues(String)} do exist and contain\n     * enough parameters. Prints an error message if not.\n     *\n     * @param optionValues   the values to check\n     * @param required       the number of required parameters\n     * @param options        the available command line options\n     * @param currentOption  the short name of the current option being parsed\n     */\n    final static void checkArguments(final String[] optionValues,\n            final int required, final Options options, final String currentOption) {\n        if ((optionValues == null) || (optionValues.length < required)) {\n            printException(\"Parsing failed\", new ParseException(\n                    \"missing parameter for option \" + currentOption\n                            + \" (required: <\"\n                            + options.getOption(currentOption).getArgName()\n                            + \">)\"), false);\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/Monitor.java",
    "content": "/*\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.LinkedHashMap;\nimport java.util.Map;\nimport java.util.Map.Entry;\nimport java.util.Set;\n\nimport com.ericsson.otp.erlang.OtpErlangAtom;\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\nimport de.zib.scalaris.ErlangValue.ListElementConverter;\n\n/**\n * Provides methods to monitor a specific Scalaris (Erlang) VM.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.11\n * @since 3.11\n */\npublic class Monitor {\n    /**\n     * Connection to a Scalaris node.\n     */\n    private final Connection connection;\n\n    /**\n     * Creates a connection to the erlang VM of the given Scalaris node. Uses\n     * the connection policy of the global connection factory.\n     *\n     * @param node\n     *            Scalaris node to connect with\n     * @throws ConnectionException\n     *             if the connection fails or the connection policy is not\n     *             cloneable\n     */\n    public Monitor(final String node) throws ConnectionException {\n        final ConnectionFactory cf = ConnectionFactory.getInstance();\n        final String fixedNode = ConnectionFactory.fixLocalhostName(node);\n        connection = cf.createConnection(new FixedNodeConnectionPolicy(fixedNode));\n    }\n\n    /**\n     * Plain old data object for results of {@link Monitor#getNodeInfo()}.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.11\n     * @since 3.11\n     */\n    public static class GetNodeInfoResult {\n        /**\n         * Scalaris version string.\n         */\n        public final String scalarisVersion;\n        /**\n         * Erlang version string.\n         */\n        public final String erlangVersion;\n        /**\n         * Number of DHT nodes in the node.\n         */\n        public final int dhtNodes;\n\n        /**\n         * @param scalarisVersion\n         *            Scalaris version string\n         * @param erlangVersion\n         *            Erlang version string\n         * @param dhtNodes\n         *            number of DHT nodes in the node\n         */\n        public GetNodeInfoResult(final String scalarisVersion, final String erlangVersion,\n                final int dhtNodes) {\n            super();\n            this.scalarisVersion = scalarisVersion;\n            this.erlangVersion = erlangVersion;\n            this.dhtNodes = dhtNodes;\n        }\n    }\n\n    /**\n     * Gets some information about the VM and Scalaris.\n     *\n     * @return VM information\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public GetNodeInfoResult getNodeInfo()\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_monitor\", \"get_node_info\",\n                    new OtpErlangObject[] {});\n        try {\n            final OtpErlangList received = (OtpErlangList) received_raw;\n            final Map<String, OtpErlangObject> result = new LinkedHashMap<String, OtpErlangObject>(\n                    received.arity());\n            for (final OtpErlangObject iter : received) {\n                final OtpErlangTuple iter_tpl = (OtpErlangTuple) iter;\n                if (iter_tpl.arity() == 2) {\n                    final String key = ((OtpErlangAtom) (iter_tpl.elementAt(0))).atomValue();\n                    result.put(key, iter_tpl.elementAt(1));\n                } else {\n                    throw new UnknownException(received_raw);\n                }\n            }\n            final String scalarisVersion = new ErlangValue(result.get(\"scalaris_version\")).stringValue();\n            final String erlangVersion = new ErlangValue(result.get(\"erlang_version\")).stringValue();\n            final int dhtNodes = new ErlangValue(result.get(\"dht_nodes\")).intValue();\n            return new GetNodeInfoResult(scalarisVersion, erlangVersion, dhtNodes);\n        } catch (final ClassCastException e) {\n            throw new UnknownException(e, received_raw);\n        } catch (final NullPointerException e) {\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Plain old data object for results of {@link Monitor#getNodePerformance()}.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.11\n     * @since 3.11\n     */\n    public static class GetNodePerformanceResult {\n        /**\n         * Average latency of transactional operations at different points in\n         * time.\n         */\n        public final Map<Long /*time*/, Double> latencyAvg;\n        /**\n         * Standard deviation of the latency of transactional operations at\n         * different points in time.\n         */\n        public final Map<Long /*time*/, Double> latencyStddev;\n\n        /**\n         * @param latencyAvg\n         *            average latency of transactional operations\n         * @param latencyStddev\n         *            standard deviation of the latency of transactional\n         *            operations\n         */\n        public GetNodePerformanceResult(final Map<Long, Double> latencyAvg,\n                final Map<Long, Double> latencyStddev) {\n            super();\n            this.latencyAvg = latencyAvg;\n            this.latencyStddev = latencyStddev;\n        }\n    }\n\n    /**\n     * Gets some information about the VM and Scalaris.\n     *\n     * @return VM information\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public GetNodePerformanceResult getNodePerformance()\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_monitor\", \"get_node_performance\",\n                    new OtpErlangObject[] {});\n        try {\n            final OtpErlangList received = (OtpErlangList) received_raw;\n            final Map<String, OtpErlangObject> result = new LinkedHashMap<String, OtpErlangObject>(\n                    received.arity());\n            for (final OtpErlangObject iter : received) {\n                final OtpErlangTuple iter_tpl = (OtpErlangTuple) iter;\n                if (iter_tpl.arity() == 2) {\n                    final String key = ((OtpErlangAtom) (iter_tpl.elementAt(0))).atomValue();\n                    result.put(key, iter_tpl.elementAt(1));\n                } else {\n                    throw new UnknownException(received_raw);\n                }\n            }\n            final Map<Long, Double> latencyAvg = tupleListToLDMap(result.get(\"latency_avg\"));\n            final Map<Long, Double> latencyStddev = tupleListToLDMap(result.get(\"latency_stddev\"));\n            return new GetNodePerformanceResult(latencyAvg, latencyStddev);\n        } catch (final ClassCastException e) {\n            throw new UnknownException(e, received_raw);\n        } catch (final NullPointerException e) {\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Plain old data object for results of {@link Monitor#getServiceInfo()}.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.11\n     * @since 3.11\n     */\n    public static class GetServiceInfoResult {\n        /**\n         * Total load accumulated among all Scalaris nodes in the ring.\n         */\n        public final Long totalLoad;\n        /**\n         * Number of Scalaris nodes in the ring.\n         */\n        public final Long nodes;\n\n        /**\n         * @param totalLoad\n         *            total load accumulated among all Scalaris nodes in the\n         *            ring\n         * @param nodes\n         *            number of Scalaris nodes in the ring\n         */\n        public GetServiceInfoResult(final Long totalLoad, final Long nodes) {\n            super();\n            this.totalLoad = totalLoad;\n            this.nodes = nodes;\n        }\n    }\n\n    /**\n     * Gets some information about the Scalaris ring.\n     *\n     * @return Scalaris ring information\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public GetServiceInfoResult getServiceInfo()\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_monitor\", \"get_service_info\",\n                    new OtpErlangObject[] {});\n        try {\n            final OtpErlangList received = (OtpErlangList) received_raw;\n            final Map<String, OtpErlangObject> result = new LinkedHashMap<String, OtpErlangObject>(\n                    received.arity());\n            for (final OtpErlangObject iter : received) {\n                final OtpErlangTuple iter_tpl = (OtpErlangTuple) iter;\n                if (iter_tpl.arity() == 2) {\n                    final String key = ((OtpErlangAtom) (iter_tpl.elementAt(0))).atomValue();\n                    result.put(key, iter_tpl.elementAt(1));\n                } else {\n                    throw new UnknownException(received_raw);\n                }\n            }\n            final Long totalLoad = new ErlangValue(result.get(\"total_load\")).longValue();\n            final Long nodes = new ErlangValue(result.get(\"nodes\")).longValue();\n            return new GetServiceInfoResult(totalLoad, nodes);\n        } catch (final ClassCastException e) {\n            throw new UnknownException(e, received_raw);\n        } catch (final NullPointerException e) {\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Plain old data object for results of {@link Monitor#getServicePerformance()}.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.11\n     * @since 3.11\n     */\n    public static class GetServicePerformanceResult {\n        /**\n         * Average latency of transactional operations at different points in\n         * time.\n         */\n        public final Map<Long /*time*/, Double> latencyAvg;\n        /**\n         * Standard deviation of the latency of transactional operations at\n         * different points in time.\n         */\n        public final Map<Long /*time*/, Double> latencyStddev;\n\n        /**\n         * @param latencyAvg\n         *            average latency of transactional operations\n         * @param latencyStddev\n         *            standard deviation of the latency of transactional\n         *            operations\n         */\n        public GetServicePerformanceResult(final Map<Long, Double> latencyAvg,\n                final Map<Long, Double> latencyStddev) {\n            super();\n            this.latencyAvg = latencyAvg;\n            this.latencyStddev = latencyStddev;\n        }\n    }\n\n    /**\n     * Gets some information about the Scalaris ring.\n     *\n     * @return Scalaris ring information\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public GetServicePerformanceResult getServicePerformance()\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_monitor\", \"get_service_performance\",\n                    new OtpErlangObject[] {});\n        try {\n            final OtpErlangList received = (OtpErlangList) received_raw;\n            final Map<String, OtpErlangObject> result = new LinkedHashMap<String, OtpErlangObject>(\n                    received.arity());\n            for (final OtpErlangObject iter : received) {\n                final OtpErlangTuple iter_tpl = (OtpErlangTuple) iter;\n                if (iter_tpl.arity() == 2) {\n                    final String key = ((OtpErlangAtom) (iter_tpl.elementAt(0))).atomValue();\n                    result.put(key, iter_tpl.elementAt(1));\n                } else {\n                    throw new UnknownException(received_raw);\n                }\n            }\n            final Map<Long, Double> latencyAvg = tupleListToLDMap(result.get(\"latency_avg\"));\n            final Map<Long, Double> latencyStddev = tupleListToLDMap(result.get(\"latency_stddev\"));\n            return new GetServicePerformanceResult(latencyAvg, latencyStddev);\n        } catch (final ClassCastException e) {\n            throw new UnknownException(e, received_raw);\n        } catch (final NullPointerException e) {\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Converts a list of 2-tuples into a map using the first entry of the tuple\n     * as the key and the second as the value.\n     *\n     * @param object\n     *            the Erlang object to convert\n     * @param keyConv\n     *            the converter to use for the key component\n     * @param valConv\n     *            the converter to use for the value component\n     *\n     * @return the converted map\n     *\n     * @throws ClassCastException\n     *             if the conversion fails\n     */\n    private static <K, V> Map<K, V> tupleListToMap(final OtpErlangObject object,\n            final ListElementConverter<K> keyConv,\n            final ListElementConverter<V> valConv) throws ClassCastException {\n        final OtpErlangList list = ErlangValue.otpObjectToOtpList(object);\n        final LinkedHashMap<K, V> result = new LinkedHashMap<K, V>(list.arity());\n        for (int i = 0; i < list.arity(); ++i) {\n            final OtpErlangTuple element = (OtpErlangTuple) list.elementAt(i);\n            if (element.arity() != 2) {\n                throw new ClassCastException(\"wrong tuple arity\");\n            }\n            final K key = keyConv.convert(i, new ErlangValue(element.elementAt(0)));\n            final V value = valConv.convert(i, new ErlangValue(element.elementAt(1)));\n            result.put(key, value);\n        }\n        return result;\n    }\n\n    /**\n     * Converts a list of 2-tuples with {@link Long} keys and {@link Double} to\n     * a map.\n     *\n     * @param object\n     *            the Erlang object to convert\n     *\n     * @return the converted map\n     *\n     * @throws ClassCastException\n     *             if the conversion fails\n     *\n     * @see #tupleListToMap(OtpErlangObject, ListElementConverter, ListElementConverter)\n     */\n    private static Map<Long, Double> tupleListToLDMap(\n            final OtpErlangObject object) throws ClassCastException {\n        return tupleListToMap(object, new ListElementConverter<Long>() {\n            public Long convert(final int i, final ErlangValue v) {\n                return v.longValue();\n            }\n        }, new ListElementConverter<Double>() {\n            public Double convert(final int i, final ErlangValue v) {\n                return v.doubleValue();\n            }\n        });\n    }\n\n    /**\n     * Closes the transaction's connection to a scalaris node.\n     *\n     * Note: Subsequent calls to the other methods will throw\n     * {@link ConnectionException}s!\n     */\n    public void closeConnection() {\n        connection.close();\n    }\n\n    /**\n     * Extracts the current performance value of a timestamp-to-double map like\n     * in the members of {@link GetNodePerformanceResult} or\n     * {@link GetServicePerformanceResult}.\n     *\n     * @param map\n     *            the map to extract from\n     *\n     * @return the latest reported performance or <tt>null</tt> if there is none\n     */\n    public static Double getCurrentPerfValue(final Map<Long, Double> map) {\n        final Set<Entry<Long, Double>> entrySet = map.entrySet();\n        if (entrySet.isEmpty()) {\n            return null;\n        } else {\n            return entrySet.iterator().next().getValue();\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/NodeDiscovery.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.Collections;\nimport java.util.HashMap;\nimport java.util.HashSet;\nimport java.util.Iterator;\nimport java.util.List;\nimport java.util.concurrent.Executors;\nimport java.util.concurrent.ScheduledExecutorService;\nimport java.util.concurrent.TimeUnit;\n\n/**\n * Provides a node discovery service.\n *\n * When started with {@link #startWithFixedDelay(long)} or\n * {@link #startWithFixedDelay(long, long, TimeUnit)}, periodically connects to\n * a Scalaris node and gets information about other Scalaris nodes. These will\n * then be added to the given {@link ConnectionFactory} where old nodes with\n * connection failures will be removed in favour of newly discovered nodes.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.16\n * @since 3.16\n */\npublic class NodeDiscovery implements Runnable {\n    /**\n     * Handler that is invoked whenever a new node is found by the\n     * {@link NodeDiscovery} thread.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     */\n    public static interface NewNodeHandler {\n        /**\n         * Called if a new node is found.\n         *\n         * @param node\n         *            the new node's name\n         */\n        public void newNodeFound(String node);\n    }\n\n    /**\n     * {@link ConnectionFactory} to work with.\n     */\n    protected final ConnectionFactory cf;\n\n    /**\n     * If not null, connections currently in the pool to removed nodes will be\n     * closed.\n     */\n    protected final ConnectionPool cPool;\n\n    /**\n     * Maximum number of nodes that should remain in the\n     * {@link ConnectionFactory} {@link #cf}.\n     */\n    protected int maxNodes = 10;\n\n    /**\n     * Minimum time in seconds since the last successful connection for a node\n     * to be removed in favour of newly discovered nodes.\n     */\n    protected int minAgeToRemove = 60;\n\n    /**\n     * Constructor\n     *\n     * @param cf\n     *            the {@link ConnectionFactory} to add nodes to / remove nodes\n     *            from\n     */\n    public NodeDiscovery(final ConnectionFactory cf) {\n        this.cf = cf;\n        this.cPool = null;\n    }\n\n    /**\n     * Constructor\n     *\n     * @param cPool\n     *            the {@link ConnectionPool} to interact with\n     */\n    public NodeDiscovery(final ConnectionPool cPool) {\n        this.cf = cPool.getConnectionFactory();\n        this.cPool = cPool;\n    }\n\n    /**\n     * Starts the node discovery service at the given fixed delay.\n     *\n     * @param delay\n     *            the delay between the termination of one execution and the\n     *            commencement of the next\n     */\n    public void startWithFixedDelay(final long delay) {\n        startWithFixedDelay(0, delay, TimeUnit.SECONDS);\n    }\n\n    /**\n     * Starts the node discovery service at the given fixed delay.\n     *\n     * @param initialDelay\n     *            the time to delay first execution\n     * @param delay\n     *            the delay between the termination of one execution and the\n     *            commencement of the next\n     * @param unit\n     *            the time unit of the initialDelay and delay parameters\n     */\n    public void startWithFixedDelay(final long initialDelay,\n            final long delay,\n            final TimeUnit unit) {\n        final ScheduledExecutorService ses = Executors.newScheduledThreadPool(1);\n        ses.scheduleWithFixedDelay(this, initialDelay, delay, unit);\n    }\n\n    /**\n     * Executed a single call to a known Scalaris node asking for other known\n     * nodes. These nodes will then be added to the {@link ConnectionFactory}\n     * {@link #cf}.\n     */\n    public void run() {\n        try {\n            final ScalarisVM vm = new ScalarisVM(cf.getConnectionPolicy().selectNode());\n            final List<String> otherVms = vm.getOtherVMs(maxNodes);\n            final List<PeerNode> existingNodes = cf.getNodes();\n\n            // get a mapping of node names to PeerNode objects for faster access\n            final HashMap<String, PeerNode> existingNodesMap = new HashMap<String, PeerNode>(existingNodes.size());\n            for (final PeerNode node : existingNodes) {\n                existingNodesMap.put(node.toString(), node);\n            }\n            // remove nodes already known\n            for (final Iterator<String> iterator = otherVms.iterator(); iterator.hasNext();) {\n                final String otherVm = iterator.next();\n                if (existingNodesMap.containsKey(otherVm)) {\n                    iterator.remove();\n                }\n            }\n\n            if (otherVms.isEmpty()) {\n                // no new nodes found...\n                return;\n            }\n\n            final int remainingNodes = removeFailedNodes(existingNodes, otherVms);\n            addNewNodes(existingNodes, otherVms, remainingNodes);\n            if (cPool != null) {\n                cPool.closeAllBut(new HashSet<PeerNode>(existingNodes));\n            }\n        } catch (final ConnectionException e) {\n            e.printStackTrace();\n        }\n    }\n\n    /**\n     * Removes nodes with failed connection attempts, without any previous\n     * connections, or with connections longer than minAgeToRemove seconds ago\n     * from the {@link ConnectionFactory} {@link #cf} to make room for newly\n     * discovered nodes.\n     *\n     * @param existingNodes\n     *            existing Erlang VMs\n     * @param otherVms\n     *            newly discovered Erlang VMs not already in\n     *            <tt>existingNodes</tt> (non-empty)\n     *\n     * @return number of remaining nodes in the {@link ConnectionFactory}\n     *         {@link #cf}\n     */\n    protected int removeFailedNodes(final List<PeerNode> existingNodes,\n            final List<String> otherVms) {\n        // sort existing nodes, least failed connection attempts first\n        Collections.sort(existingNodes, new LeastFailedNodesComparator());\n        int failedNodesToRemove;\n        final int existingNodesCount = existingNodes.size();\n        if (otherVms.size() > maxNodes) {\n            failedNodesToRemove = existingNodesCount;\n        } else if ((existingNodesCount + otherVms.size()) <= maxNodes) {\n            failedNodesToRemove = 0;\n        } else {\n            failedNodesToRemove = (existingNodesCount + otherVms.size()) - maxNodes;\n        }\n        assert failedNodesToRemove >= 0;\n        assert failedNodesToRemove <= existingNodesCount;\n\n        int lastNodeIdx = existingNodesCount - 1;\n        for (; lastNodeIdx >= (existingNodesCount - failedNodesToRemove); --lastNodeIdx) {\n            final PeerNode node = existingNodes.get(lastNodeIdx);\n            if (  // failed connection?\n                       (node.getFailureCount() > 0)\n                  // never connected?\n                    || (node.getLastConnectSuccess() == null)\n                  // last connection longer than minAgeToRemove seconds ago?\n                    || (node.getLastConnectSuccess().getTime() < (System\n                            .currentTimeMillis() - (minAgeToRemove * 1000)))) {\n                cf.removeNode(node);\n                existingNodes.remove(lastNodeIdx);\n            } else {\n                break;\n            }\n        }\n        return lastNodeIdx + 1;\n    }\n\n    /**\n     * Adds newly discovered nodes to the {@link ConnectionFactory} {@link #cf}.\n     *\n     * @param existingNodes\n     *            existing Erlang VMs\n     * @param otherVms\n     *            newly discovered Erlang VMs not already in\n     *            <tt>existingNodes</tt> (non-empty)\n     * @param remainingNodes\n     *            number of remaining nodes in the {@link ConnectionFactory}\n     *            {@link #cf}\n     */\n    protected void addNewNodes(final List<PeerNode> existingNodes,final List<String> otherVms,\n            final int remainingNodes) {\n        // then add new nodes (not more than maxNodes number of available nodes!)\n        for (int i = 0; (i < (maxNodes - remainingNodes)) && (i < otherVms.size()); ++i) {\n            final PeerNode p = new PeerNode(otherVms.get(i));\n            cf.addNode(p);\n            existingNodes.add(p);\n        }\n        // TODO: replace some more remaining nodes with newly discovered ones?\n        // e.g. randomly select some to be replaced?\n    }\n\n    /**\n     * Gets the maximum number of nodes that should remain in the\n     * {@link ConnectionFactory} {@link #cf}.\n     *\n     * @return the maxNodes member\n     */\n    public final int getMaxNodes() {\n        return maxNodes;\n    }\n\n    /**\n     * Sets the maximum number of nodes that should remain in the\n     * {@link ConnectionFactory} {@link #cf}.\n     *\n     * @param maxNodes\n     *            the maxNodes to set\n     */\n    public final void setMaxNodes(final int maxNodes) {\n        this.maxNodes = maxNodes;\n    }\n\n    /**\n     * Gets the minimum time in seconds since the last successful connection for\n     * a node to be removed in favour of newly discovered nodes.\n     *\n     * @return the minAgeToRemove member\n     */\n    public final int getMinAgeToRemove() {\n        return minAgeToRemove;\n    }\n\n    /**\n     * Sets the minimum time in seconds since the last successful connection for\n     * a node to be removed in favour of newly discovered nodes.\n     *\n     * @param minAgeToRemove\n     *            the minAgeToRemove to set\n     */\n    public final void setMinAgeToRemove(final int minAgeToRemove) {\n        this.minAgeToRemove = minAgeToRemove;\n    }\n\n    /**\n     * Gets the {@link ConnectionFactory} to work with.\n     *\n     * @return the connection factory\n     */\n    public final ConnectionFactory getCf() {\n        return cf;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/NotAListException.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\nimport com.ericsson.otp.erlang.OtpErlangObject;\n\n/**\n * Exception that is thrown if a add_del_on_list operation on a scalaris ring\n * fails because the participating values are not lists.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.8\n * @since 3.8\n */\npublic class NotAListException extends OtpErlangException {\n    /**\n     * class version for serialisation\n     */\n    private static final long serialVersionUID = 1L;\n\n    /**\n     * Creates the exception with no message.\n     */\n    public NotAListException() {\n    }\n\n    /**\n     * Creates the exception with the given message.\n     *\n     * @param msg\n     *            message of the exception\n     */\n    public NotAListException(final String msg) {\n        super(msg);\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e the exception to \"re-throw\"\n     */\n    public NotAListException(final Throwable e) {\n        super(e.getMessage());\n        setStackTrace(e.getStackTrace());\n    }\n\n    /**\n     * Creates an exception including the message of the given erlang object.\n     *\n     * @param erlValue\n     *            the erlang message to include\n     */\n    public NotAListException(final OtpErlangObject erlValue) {\n        super(\"Erlang message: \" + erlValue.toString());\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e\n     *            the exception to \"re-throw\"\n     * @param erlValue\n     *            the string representation of this erlang value is included\n     *            into the message\n     */\n    public NotAListException(final Throwable e, final OtpErlangObject erlValue) {\n        super(e.getMessage() + \",\\n  Erlang message: \" + erlValue.toString());\n        setStackTrace(e.getStackTrace());\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/NotANumberException.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\nimport com.ericsson.otp.erlang.OtpErlangObject;\n\n/**\n * Exception that is thrown if a add_del_on_list operation on a scalaris ring\n * fails because the participating values are not numbers.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.8\n * @since 3.8\n */\npublic class NotANumberException extends OtpErlangException {\n    /**\n     * class version for serialisation\n     */\n    private static final long serialVersionUID = 1L;\n\n    /**\n     * Creates the exception with no message.\n     */\n    public NotANumberException() {\n    }\n\n    /**\n     * Creates the exception with the given message.\n     *\n     * @param msg\n     *            message of the exception\n     */\n    public NotANumberException(final String msg) {\n        super(msg);\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e the exception to \"re-throw\"\n     */\n    public NotANumberException(final Throwable e) {\n        super(e.getMessage());\n        setStackTrace(e.getStackTrace());\n    }\n\n    /**\n     * Creates an exception including the message of the given erlang object.\n     *\n     * @param erlValue\n     *            the erlang message to include\n     */\n    public NotANumberException(final OtpErlangObject erlValue) {\n        super(\"Erlang message: \" + erlValue.toString());\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e\n     *            the exception to \"re-throw\"\n     * @param erlValue\n     *            the string representation of this erlang value is included\n     *            into the message\n     */\n    public NotANumberException(final Throwable e, final OtpErlangObject erlValue) {\n        super(e.getMessage() + \",\\n  Erlang message: \" + erlValue.toString());\n        setStackTrace(e.getStackTrace());\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/NotFoundException.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\nimport com.ericsson.otp.erlang.OtpErlangObject;\n\n/**\n * Exception that is thrown if a read operation on a scalaris ring fails\n * because the key did not exist before.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.2\n * @since 2.0\n */\npublic class NotFoundException extends OtpErlangException {\n    /**\n     * class version for serialisation\n     */\n    private static final long serialVersionUID = 1L;\n\n    /**\n     * Creates the exception with no message.\n     */\n    public NotFoundException() {\n    }\n\n    /**\n     * Creates the exception with the given message.\n     *\n     * @param msg\n     *            message of the exception\n     */\n    public NotFoundException(final String msg) {\n        super(msg);\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e the exception to \"re-throw\"\n     */\n    public NotFoundException(final Throwable e) {\n        super(e.getMessage());\n        setStackTrace(e.getStackTrace());\n    }\n\n    /**\n     * Creates an exception including the message of the given erlang object.\n     *\n     * @param erlValue\n     *            the erlang message to include\n     *\n     * @since 2.2\n     */\n    public NotFoundException(final OtpErlangObject erlValue) {\n        super(\"Erlang message: \" + erlValue.toString());\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e\n     *            the exception to \"re-throw\"\n     * @param erlValue\n     *            the string representation of this erlang value is included\n     *            into the message\n     *\n     * @since 2.2\n     */\n    public NotFoundException(final Throwable e, final OtpErlangObject erlValue) {\n        super(e.getMessage() + \",\\n  Erlang message: \" + erlValue.toString());\n        setStackTrace(e.getStackTrace());\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/PeerNode.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.Date;\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpPeer;\n\n/**\n * Wrapper class to the {@link OtpPeer} class, adding some additional\n * information.\n *\n * @author Nico Kruber, kruber@zib.de\n */\npublic class PeerNode {\n    /**\n     * The node this object wraps.\n     */\n    private final OtpPeer node;\n    /**\n     * Date of the last failed connection.\n     */\n    private Date lastFailedConnection = null;\n\n    /**\n     * Total number of connection failures.\n     */\n    private int failureCount = 0;\n\n    /**\n     * Date of the last successful connection attempt.\n     */\n    private Date lastConnectSuccess = null;\n\n    /**\n     * Creates a new object using the given node.\n     *\n     * @param node\n     *            the node to wrap\n     */\n    public PeerNode(final OtpPeer node) {\n        super();\n        this.node = node;\n    }\n\n    /**\n     * Creates a new object using the given node.\n     *\n     * Provided for convenience.\n     *\n     * @param node\n     *            the name of the node to wrap\n     */\n    public PeerNode(final String node) {\n        super();\n        this.node = new OtpPeer(node);\n    }\n\n    /**\n     * Gets the OTP node that is being wrapped.\n     *\n     * @return the node\n     */\n    public OtpPeer getNode() {\n        return node;\n    }\n\n    /**\n     * Sets the last failed connection (attempt or broken connection) with the\n     * current date and time.\n     *\n     * Note: Only call this from a connection policy since it might set up\n     * additional data structures based on this time.\n     */\n    synchronized void setLastFailedConnect() {\n        lastFailedConnection = new Date();\n        ++failureCount;\n    }\n\n    /**\n     * Gets the date of the last failed connection.\n     *\n     * @return the date of the last connection failure (or {@code null})\n     */\n    synchronized public Date getLastFailedConnect() {\n        return lastFailedConnection;\n    }\n\n    /**\n     * Gets the number of failed connections.\n     *\n     * This is faster than getting a list of failed connections and calling\n     * {@link List#size()} since this does not require a conversion.\n     *\n     * @return the number of failed connections (dates and times)\n     */\n    synchronized public int getFailureCount() {\n        return failureCount;\n    }\n\n    /**\n     * Resets the failed connections statistics.\n     *\n     * Note: Only call this from a connection policy since it might set up\n     * additional data structures based on this time.\n     */\n    synchronized void resetFailureCount() {\n        failureCount = 0;\n        lastFailedConnection = null;\n    }\n\n    /**\n     * Gets the date of the last successful connection.\n     *\n     * @return the last connection success\n     */\n    synchronized public Date getLastConnectSuccess() {\n        return lastConnectSuccess;\n    }\n\n    /**\n     * Adds a connection success with the current date.\n     *\n     * Note: Only call this from a connection policy since it might set up\n     * additional data structures based on this time.\n     */\n    synchronized void setLastConnectSuccess() {\n        this.lastConnectSuccess = new Date();\n    }\n\n    /**\n     * Returns a string representation of this node.\n     *\n     * @return the name of the node\n     */\n    @Override\n    public String toString() {\n        return node.toString();\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/ReplicatedDHT.java",
    "content": "/*\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport com.ericsson.otp.erlang.OtpErlangAtom;\nimport com.ericsson.otp.erlang.OtpErlangInt;\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangLong;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangRangeException;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\n/**\n * Provides methods to delete all replicas of the given key\n * (from <code>api_rdht.erl</code>).\n *\n * <p>\n * Instances of this class can be generated using a given connection to a\n * scalaris node using {@link #ReplicatedDHT(Connection)} or without a\n * connection ({@link #ReplicatedDHT()}) in which case a new connection is\n * created using {@link ConnectionFactory#createConnection()}.\n * </p>\n *\n * <h3>Deleting values</h3>\n * <pre>\n * <code style=\"white-space:pre;\">\n *   String key;\n *   int timeout;\n *   DeleteResult result;\n *\n *   TransactionSingleOp sc = new ReplicatedDHT();\n *   sc.delete(key);                    // {@link #delete(String)}\n *   sc.delete(key, timeout);           // {@link #delete(String, int)}\n *   result = sc.getLastDeleteResult(); // {@link #getLastDeleteResult()}\n * </code>\n * </pre>\n *\n * <h3>Connection errors</h3>\n *\n * Errors when setting up connections or trying to send/receive RPCs will be\n * handed to the {@link ConnectionPolicy} that has been set when the connection\n * was created. By default, {@link ConnectionFactory} uses\n * {@link DefaultConnectionPolicy} which implements automatic connection-retries\n * by classifying nodes as good or bad depending on their previous state. The\n * number of automatic retries is adjustable (default: 3).\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.19\n * @since 2.6\n */\npublic class ReplicatedDHT {\n    /**\n     * Stores the result list returned by erlang during a delete operation.\n     *\n     * @see #delete(String)\n     */\n    private DeleteResult lastDeleteResult = null;\n\n    /**\n     * Connection to a TransactionSingleOp node.\n     */\n    private final Connection connection;\n\n    /**\n     * Constructor, uses the default connection returned by\n     * {@link ConnectionFactory#createConnection()}.\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     */\n    public ReplicatedDHT() throws ConnectionException {\n        connection = ConnectionFactory.getInstance().createConnection();\n    }\n\n    /**\n     * Constructor, uses the given connection to an erlang node.\n     *\n     * @param conn\n     *            connection to use for the transaction\n     */\n    public ReplicatedDHT(final Connection conn) {\n        connection = conn;\n    }\n\n    // /////////////////////////////\n    // delete methods\n    // /////////////////////////////\n\n    /**\n     * Tries to delete all replicas of the given <tt>key</tt> in 2000ms.\n     *\n     * WARNING: This function can lead to inconsistent data (e.g. deleted items\n     * can re-appear). Also when re-creating an item the version before the\n     * delete can re-appear.\n     *\n     * @param key\n     *            the key to delete\n     *\n     * @return a delete result object\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws TimeoutException\n     *             if a timeout occurred while trying to delete the value\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.19\n     *\n     * @see #delete(OtpErlangString, int)\n     */\n    public DeleteResult delete(final OtpErlangString key) throws ConnectionException,\n    TimeoutException, UnknownException {\n        return delete(key, 2000);\n    }\n\n    /**\n     * Tries to delete all replicas of the given <tt>key</tt>.\n     *\n     * WARNING: This function can lead to inconsistent data (e.g. deleted items\n     * can re-appear). Also when re-creating an item the version before the\n     * delete can re-appear.\n     *\n     * @param key\n     *            the key to delete\n     * @param timeout\n     *            the time (in milliseconds) to wait for results\n     *\n     * @return a delete result object\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws TimeoutException\n     *             if a timeout occurred while trying to delete the value\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.19\n     */\n    public DeleteResult delete(final OtpErlangString key, final int timeout) throws ConnectionException,\n    TimeoutException, UnknownException {\n        lastDeleteResult = null;\n        final OtpErlangObject received_raw = connection.doRPC(\"api_rdht\", \"delete\",\n                new OtpErlangObject[] { key, new OtpErlangInt(timeout) });\n        try {\n            final OtpErlangTuple received = (OtpErlangTuple) received_raw;\n            final OtpErlangAtom state = (OtpErlangAtom) received.elementAt(0);\n\n            /*\n             * possible return values:\n             *  - {ok, ResultsOk::pos_integer(), ResultList::[ok | undef]}\n             *  - {fail, timeout, ResultsOk::pos_integer(), ResultList::[ok | undef]}\n             */\n            if (state.equals(CommonErlangObjects.okAtom) && (received.arity() == 3)) {\n                lastDeleteResult = new DeleteResult((OtpErlangList) received.elementAt(2));\n                final int succeeded = ((OtpErlangLong) received.elementAt(1)).intValue();\n                // plausibility check:\n                if (succeeded != lastDeleteResult.ok) {\n                    throw new UnknownException(received_raw);\n                }\n                return lastDeleteResult;\n            } else if (state.equals(CommonErlangObjects.failAtom) && (received.arity() == 4)) {\n                final OtpErlangObject reason = received.elementAt(1);\n                if (reason.equals(CommonErlangObjects.timeoutAtom)) {\n                    lastDeleteResult = new DeleteResult((OtpErlangList) received.elementAt(3));\n                    final int succeeded = ((OtpErlangLong) received.elementAt(2)).intValue();\n                    // plausibility check:\n                    if (succeeded != lastDeleteResult.ok) {\n                        throw new UnknownException(received_raw);\n                    }\n                    throw new TimeoutException(received_raw);\n                }\n            }\n            throw new UnknownException(received_raw);\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, received_raw);\n        } catch (final OtpErlangRangeException e) {\n            // there should not this many replicates that do not fit into an integer!\n            // e.printStackTrace();\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Tries to delete all replicas of the given <tt>key</tt> in 2000ms.\n     *\n     * @param key\n     *            the key to delete\n     *\n     * @return a delete result object\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws TimeoutException\n     *             if a timeout occurred while trying to delete the value\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.19\n     *\n     * @see #delete(String, int)\n     */\n    public DeleteResult delete(final String key) throws ConnectionException,\n    TimeoutException, UnknownException {\n        return delete(key, 2000);\n    }\n\n    /**\n     * Tries to delete all replicas of the given <tt>key</tt>.\n     *\n     * WARNING: This function can lead to inconsistent data (e.g. deleted items\n     * can re-appear). Also when re-creating an item the version before the\n     * delete can re-appear.\n     *\n     * @param key\n     *            the key to delete\n     * @param timeout\n     *            the time (in milliseconds) to wait for results\n     *\n     * @return a delete result object\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws TimeoutException\n     *             if a timeout occurred while trying to delete the value\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.19\n     *\n     * @see #delete(OtpErlangString, int)\n     */\n    public DeleteResult delete(final String key, final int timeout) throws ConnectionException,\n    TimeoutException, UnknownException {\n        return delete(new OtpErlangString(key), timeout);\n    }\n\n    /**\n     * Returns the result of the last call to {@link #delete(String)}.\n     *\n     * @return the delete result\n     *\n     * @throws UnknownException\n     *             is thrown if an unknown reason was encountered\n     *\n     * @see #delete(String)\n     */\n    public DeleteResult getLastDeleteResult() {\n        return lastDeleteResult;\n    }\n\n    /**\n     * Closes the transaction's connection to a scalaris node.\n     *\n     * Note: Subsequent calls to the other methods will throw\n     * {@link ConnectionException}s!\n     */\n    public void closeConnection() {\n        connection.close();\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/RequestList.java",
    "content": "/**\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.math.BigInteger;\nimport java.util.ArrayList;\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpErlangDouble;\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangLong;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\n\nimport de.zib.scalaris.operations.AddDelOnListOp;\nimport de.zib.scalaris.operations.AddOnNrOp;\nimport de.zib.scalaris.operations.CommitOp;\nimport de.zib.scalaris.operations.Operation;\nimport de.zib.scalaris.operations.ReadOp;\nimport de.zib.scalaris.operations.TestAndSetOp;\nimport de.zib.scalaris.operations.WriteOp;\n\n/**\n * Generic request list.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.5\n */\npublic abstract class RequestList {\n    protected final List<Operation> requests = new ArrayList<Operation>(5);\n    private CommitOp commitOp = null;\n\n    /**\n     * Default constructor.\n     */\n    protected RequestList() {\n    }\n\n    /**\n     * Creates a new request list with the given operation.\n     *\n     * Provided for convenience.\n     *\n     * @since 3.18\n     */\n    protected RequestList(final Operation op) {\n        addOp(op);\n    }\n\n    /**\n     * Copy constructor.\n     *\n     * @param other the request list to copy from\n     */\n    protected RequestList(final RequestList other) {\n        addAll_(other);\n    }\n\n    /**\n     * Adds a generic operation to the list of requests.\n     *\n     * @param op\n     *            the operation to add\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     */\n    public RequestList addOp(final Operation op) throws UnsupportedOperationException {\n        if (isCommit()) {\n            throw new UnsupportedOperationException(\n                    \"No further request supported after a commit!\");\n        }\n        requests.add(op);\n        return this;\n    }\n\n    /**\n     * Adds a read operation to the list of requests.\n     *\n     * @param key\n     *            the key to read\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     */\n    @Deprecated\n    public RequestList addRead(final OtpErlangString key)\n            throws UnsupportedOperationException {\n        return addOp(new ReadOp(key));\n    }\n\n    /**\n     * Adds a read operation to the list of requests.\n     *\n     * @param key\n     *            the key to read\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     */\n    @Deprecated\n    public RequestList addRead(final String key)\n            throws UnsupportedOperationException {\n        return addOp(new ReadOp(key));\n    }\n\n    /**\n     * Adds a write operation to the list of requests.\n     *\n     * @param key\n     *            the key to write the value to\n     * @param value\n     *            the value to write\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     */\n    @Deprecated\n    public RequestList addWrite(final OtpErlangString key, final OtpErlangObject value)\n            throws UnsupportedOperationException {\n        return addOp(new WriteOp(key, value));\n    }\n\n    /**\n     * Adds a write operation to the list of requests.\n     *\n     * @param <T>\n     *            type of the value to write\n     * @param key\n     *            the key to write the value to\n     * @param value\n     *            the value to write\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     */\n    @Deprecated\n    public <T> RequestList addWrite(final String key, final T value)\n            throws UnsupportedOperationException {\n        return addOp(new WriteOp(key, value));\n    }\n\n    /**\n     * Adds a add_del_on_list operation to the list of requests.\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            a list of values to add to a list\n     * @param toRemove\n     *            a list of values to remove from a list\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     *\n     * @since 3.9\n     */\n    @Deprecated\n    public RequestList addAddDelOnList(final OtpErlangString key, final OtpErlangList toAdd, final OtpErlangList toRemove)\n            throws UnsupportedOperationException {\n        return addOp(new AddDelOnListOp(key, toAdd, toRemove));\n    }\n\n    /**\n     * Adds a add_del_on_list operation to the list of requests.\n     *\n     * @param <T>\n     *            type of the value to write\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            a list of values to add to a list\n     * @param toRemove\n     *            a list of values to remove from a list\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     *\n     * @since 3.9\n     */\n    @Deprecated\n    public <T> RequestList addAddDelOnList(final String key, final List<T> toAdd, final List<T> toRemove)\n            throws UnsupportedOperationException {\n        return addOp(new AddDelOnListOp(key, toAdd, toRemove));\n    }\n\n    /**\n     * Adds an add_on_nr operation to the list of requests.\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            the number to add to the number stored at key\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     *\n     * @since 3.9\n     */\n    @Deprecated\n    public RequestList addAddOnNr(final OtpErlangString key, final OtpErlangLong toAdd)\n            throws UnsupportedOperationException {\n        return addOp(new AddOnNrOp(key, toAdd));\n    }\n\n    /**\n     * Adds an add_on_nr operation to the list of requests.\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            the number to add to the number stored at key\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     *\n     * @since 3.9\n     */\n    @Deprecated\n    public RequestList addAddOnNr(final OtpErlangString key, final OtpErlangDouble toAdd)\n            throws UnsupportedOperationException {\n        return addOp(new AddOnNrOp(key, toAdd));\n    }\n\n    /**\n     * Adds an add_on_nr operation to the list of requests.\n     *\n     * @param <T>\n     *            type of the value to write; WARNING: the actual supported\n     *            types only include {@link Integer}, {@link Long},\n     *            {@link BigInteger} and {@link Double} - see\n     *            {@link ErlangValue#convertToErlang(Object)}.\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            the number to add to the number stored at key\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     *\n     * @since 3.9\n     */\n    @Deprecated\n    public <T extends Number> RequestList addAddOnNr(final String key, final T toAdd)\n            throws UnsupportedOperationException {\n        return addOp(new AddOnNrOp(key, toAdd));\n    }\n\n    /**\n     * Adds an add_on_nr operation to the list of requests.\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            the number to add to the number stored at key\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     *\n     * @since 3.9\n     */\n    @Deprecated\n    public RequestList addAddOnNr(final String key, final Double toAdd)\n            throws UnsupportedOperationException {\n        return addOp(new AddOnNrOp(key, toAdd));\n    }\n\n    /**\n     * Adds a test_and_set operation to the list of requests (<tt>newValue</tt>\n     * is only written if the currently stored value is <tt>oldValue</tt>).\n     *\n     * @param key\n     *            the key to write the value to\n     * @param oldValue\n     *            the old value to verify\n     * @param newValue\n     *            the new value to write of oldValue is correct\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     *\n     * @since 3.8\n     */\n    @Deprecated\n    public RequestList addTestAndSet(final OtpErlangString key, final OtpErlangObject oldValue, final OtpErlangObject newValue)\n            throws UnsupportedOperationException {\n        return addOp(new TestAndSetOp(key, oldValue, newValue));\n    }\n\n    /**\n     * Adds a test_and_set operation to the list of requests (<tt>newValue</tt>\n     * is only written if the currently stored value is <tt>oldValue</tt>).\n     *\n     * @param <OldT>\n     *            the type of the stored (old) value. See {@link ErlangValue}\n     *            for a list of supported types.\n     * @param <NewT>\n     *            the type of the (new) value to store. See {@link ErlangValue}\n     *            for a list of supported types.\n     * @param key\n     *            the key to write the value to\n     * @param oldValue\n     *            the old value to verify\n     * @param newValue\n     *            the new value to write of oldValue is correct\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     *\n     * @since 3.8\n     */\n    @Deprecated\n    public <OldT, NewT> RequestList addTestAndSet(final String key, final OldT oldValue, final NewT newValue)\n            throws UnsupportedOperationException {\n        return addOp(new TestAndSetOp(key, oldValue, newValue));\n    }\n\n    /**\n     * Adds a commit operation to the list of requests.\n     *\n     * @return this {@link RequestList} object\n     *\n     * @throws UnsupportedOperationException\n     *             if the operation is unsupported, e.g. there may only be one\n     *             \"commit\" in a request list and no request after that\n     */\n    public RequestList addCommit() throws UnsupportedOperationException {\n        final CommitOp op = new CommitOp();\n        addOp(op);\n        this.commitOp = op;\n        return this;\n    }\n\n    /**\n     * Gets the whole request list as erlang terms as required by\n     * <code>api_tx:req_list/2</code>\n     * Note: this parses through the requests to create the erlang objects.\n     *\n     * @param compressed\n     *            whether the value part in the term should be encoded, i.e.\n     *            compressed into an Erlang binary, or not\n     *\n     * @return an erlang list of requests\n     */\n    OtpErlangList getErlangReqList(final boolean compressed) {\n        final OtpErlangObject[] result = new OtpErlangObject[requests.size()];\n        int i = 0;\n        for (final Operation op : requests) {\n            result[i++] = op.getErlang(compressed);\n        }\n        return new OtpErlangList(result);\n    }\n\n    /**\n     * Returns whether the transactions contains a commit or not.\n     *\n     * @return <tt>true</tt> if the operation contains a commit,\n     *         <tt>false</tt> otherwise\n     */\n    public boolean isCommit() {\n        return commitOp != null;\n    }\n\n    /**\n     * Returns the commit operation (if present).\n     *\n     * @return the commit operation or <tt>null</tt> if there is none\n     */\n    public CommitOp getCommit() {\n        return commitOp;\n    }\n\n    /**\n     * Checks whether the request list is empty.\n     *\n     * @return <tt>true</tt> is empty, <tt>false</tt> otherwise\n     */\n    public boolean isEmpty() {\n        return requests.isEmpty();\n    }\n\n    /**\n     * Gets the number of requests in the list.\n     *\n     * @return number of requests\n     */\n    public int size() {\n        return requests.size();\n    }\n\n    /**\n     * Adds all requests of the other request list to the end of this list.\n     *\n     * Use in implementation in sub-classes with according types as different\n     * request lists may not be compatible with each other.\n     *\n     * @param other another request list\n     *\n     * @return this {@link RequestList} object\n     */\n    protected RequestList addAll_(final RequestList other) {\n        requests.addAll(other.requests);\n        return this;\n    }\n\n    /**\n     * Gets all operations of the request list.\n     *\n     * @return the requests\n     *\n     * @since 3.14\n     */\n    public List<Operation> getRequests() {\n        return requests;\n    }\n\n    @Override\n    public String toString() {\n        return requests.toString();\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/ResultList.java",
    "content": "/**\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangObject;\n\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.operations.Operation;\nimport de.zib.scalaris.operations.ReadOp;\n\n/**\n * Generic result list.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.8\n * @since 3.5\n */\npublic abstract class ResultList {\n    final protected List<Operation> operations;\n\n    /**\n     * Default constructor.\n     *\n     * @param results\n     *            the raw results list as returned by scalaris\n     * @param compressed\n     *            whether the value part in the term is encoded, i.e. compressed\n     *            into an Erlang binary, or not\n     * @param requests\n     *            request list which created this result list\n     *\n     * @throws UnknownException\n     *             if the result list size does not match the request list size\n     */\n    protected ResultList(final OtpErlangList results, final boolean compressed,\n            final RequestList requests) throws UnknownException {\n        if (results.arity() != requests.size()) {\n            throw new UnknownException(\"Result list size different from request list size!\");\n        }\n        this.operations = requests.getRequests();\n\n        // assign the results to their appropriate operations\n        for (int i = 0; i < results.arity(); ++i) {\n            final OtpErlangObject result = results.elementAt(i);\n            this.operations.get(i).setResult(result, compressed);\n        }\n    }\n\n    /**\n     * Gets the number of results in the list.\n     *\n     * @return total number of results\n     */\n    public int size() {\n        return operations.size();\n    }\n\n    /**\n     * Returns the operation at the specified position for e.g. further result\n     * processing.\n     *\n     * @param index\n     *            index of the operation/result to return\n     *\n     * @return the operation at the specified position\n     *\n     * @throws IndexOutOfBoundsException\n     *             if the index is out of range (\n     *             <tt>index &lt; 0 || index &gt;= size()</tt>)\n     *\n     * @since 3.18\n     */\n    public Operation get(final int index) {\n        return operations.get(index);\n    }\n\n    /**\n     * Processes the result at the given position which originated from a read\n     * request and returns the value that has been read.\n     *\n     * @param pos\n     *            the position in the result list (starting at 0)\n     *\n     * @return the stored value\n     *\n     * @throws NotFoundException\n     *             if the requested key does not exist\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.13\n     */\n    public ErlangValue processReadAt(final int pos) throws NotFoundException,\n            UnknownException {\n        return ((ReadOp) get(pos)).processResult();\n    }\n\n    /**\n     * Processes the result at the given position which originated from\n     * a write request.\n     *\n     * @param pos\n     *            the position in the result list (starting at 0)\n     *\n     * @throws AbortException\n     *             if the commit of the write failed (if there was a commit)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.13\n     */\n    public abstract void processWriteAt(final int pos) throws AbortException,\n            UnknownException;\n\n    /**\n     * Processes the result at the given position which originated from\n     * a add_del_on_list request.\n     *\n     * @param pos\n     *            the position in the result list (starting at 0)\n     *\n     * @throws NotAListException\n     *             if the previously stored value was no list\n     * @throws AbortException\n     *             if the commit of the write failed (if there was a commit)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.13\n     */\n    public abstract void processAddDelOnListAt(final int pos)\n            throws NotAListException, AbortException, UnknownException;\n\n    /**\n     * Processes the result at the given position which originated from\n     * an add_on_nr request.\n     *\n     * @param pos\n     *            the position in the result list (starting at 0)\n     *\n     * @throws NotANumberException\n     *             if the previously stored value was not a number\n     * @throws AbortException\n     *             if the commit of the write failed (if there was a commit)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.13\n     */\n    public abstract void processAddOnNrAt(final int pos)\n            throws NotANumberException, AbortException, UnknownException;\n\n    /**\n     * Processes the result at the given position which originated from\n     * a test_and_set request.\n     *\n     * @param pos\n     *            the position in the result list (starting at 0)\n     *\n     * @throws NotFoundException\n     *             if the requested key does not exist\n     * @throws KeyChangedException\n     *             if the key did not match <tt>old_value</tt>\n     * @throws AbortException\n     *             if the commit of the write failed (if there was a commit)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.13\n     */\n    public abstract void processTestAndSetAt(final int pos)\n            throws NotFoundException, KeyChangedException, AbortException,\n            UnknownException;\n\n    @Override\n    public String toString() {\n        int i = 1;\n        final StringBuilder sb = new StringBuilder();\n        sb.append('[');\n        for (final Operation op : operations) {\n            sb.append(op.getResult().toString());\n            if (i != operations.size()) {\n                sb.append(\", \");\n            }\n            ++i;\n        }\n        sb.append(']');\n        return sb.toString();\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/RoundRobinConnectionPolicy.java",
    "content": "/**\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.List;\n\n/**\n * Implements a {@link ConnectionPolicy} by choosing nodes round-robin.\n *\n * This implementation is based on {@link DefaultConnectionPolicy} and thus\n * also differentiates good and bad nodes.\n *\n * @author Nico Kruber, kruber@zib.de\n *\n * @version 3.5\n * @since 3.5\n *\n * @see DefaultConnectionPolicy\n */\npublic class RoundRobinConnectionPolicy extends DefaultConnectionPolicy {\n    int nextNode = 0;\n\n    /**\n     * Creates a new connection policy working with the given remote node.\n     *\n     * Provided for convenience.\n     *\n     * Attention: This method also synchronises on the node.\n     *\n     * @param remoteNode the (only) available remote node\n     */\n    public RoundRobinConnectionPolicy(final PeerNode remoteNode) {\n        super(remoteNode);\n    }\n\n    /**\n     * Creates a new connection policy with the given remote nodes.\n     *\n     * Attention: This method synchronises on {@code availableRemoteNodes}.\n     *\n     * Any time this list is changed, the according methods in this class should\n     * be called, i.e. {@link #availableNodeAdded(PeerNode)},\n     * {@link #availableNodeRemoved(PeerNode)}, {@link #availableNodesReset()}\n     * to update the good and bad nodes lists.\n     *\n     * @param availableRemoteNodes\n     *            the remote nodes available for connections\n     */\n    public RoundRobinConnectionPolicy(final List<PeerNode> availableRemoteNodes) {\n        super(availableRemoteNodes);\n    }\n\n    /**\n     * Selects a good node in a round-robin fashion.\n     */\n    @Override\n    protected synchronized PeerNode getGoodNode() {\n        if (goodNodes.size() == 1) {\n            return goodNodes.get(0);\n        } else {\n            nextNode %= goodNodes.size();\n            return goodNodes.get(nextNode++);\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/RoutingTable.java",
    "content": "/*\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport com.ericsson.otp.erlang.OtpErlangLong;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangRangeException;\n\n/**\n * Scalaris interface to basic routing table information.\n *\n * @author Thorsten Schuett, schuett@zib.de\n * @version 3.20\n * @since 3.20\n */\npublic class RoutingTable {\n    /**\n     * Connection to a Scalaris node.\n     */\n    private final Connection connection;\n\n    /**\n     * Constructor, uses the default connection returned by\n     * {@link ConnectionFactory#createConnection()}.\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     */\n    public RoutingTable() throws ConnectionException {\n        connection = ConnectionFactory.getInstance().createConnection();\n    }\n\n    /**\n     * Constructor, uses the given connection to an erlang node.\n     *\n     * @param conn\n     *            connection to use for the Scalaris access\n     */\n    public RoutingTable(final Connection conn) {\n        connection = conn;\n    }\n\n    /**\n     * Returns the replication factor used by the current routing table\n     * implementation.\n     *\n     * @return the current replication factor\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public int getReplicationFactor() throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_rt\", \"get_replication_factor\",\n                new OtpErlangObject[] { });\n        try {\n            final OtpErlangLong received = (OtpErlangLong) received_raw;\n            return received.intValue();\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, received_raw);\n        } catch (final OtpErlangRangeException e) {\n            // there should not this many replicates that do not fit into an integer!\n            // e.printStackTrace();\n            throw new UnknownException(e, received_raw);\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/Scalaris.java",
    "content": "/*\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.ArrayList;\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpErlangAtom;\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\n/**\n * Provides methods to interact with some Scalaris (Erlang) VM.\n *\n * <p>\n * Instances of this class can be generated using a given connection to a\n * scalaris node ({@link #Scalaris(Connection)}) or without a\n * connection ({@link #Scalaris()}) in which case a new connection\n * is created using {@link ConnectionFactory#createConnection()}.\n * </p>\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.10\n * @since 3.10\n */\npublic class Scalaris {\n    /**\n     * Connection to a Scalaris node.\n     */\n    private final Connection connection;\n\n    /**\n     * Constructor, uses the default connection returned by\n     * {@link ConnectionFactory#createConnection()}.\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     */\n    public Scalaris() throws ConnectionException {\n        connection = ConnectionFactory.getInstance().createConnection();\n    }\n\n    /**\n     * Constructor, uses the given connection to an erlang node.\n     *\n     * @param conn\n     *            connection to use for the transaction\n     */\n    public Scalaris(final Connection conn) {\n        connection = conn;\n    }\n\n    /**\n     * Retrieves random nodes from Scalaris for use by\n     * {@link ConnectionFactory#addNode(String)} or {@link ScalarisVM}.\n     * \n     * @param max\n     *            maximum number of nodes to return (> 0)\n     * \n     * @return a list of nodes (if an empty node list was returned from\n     *         Scalaris, the connection's node will be returned here)\n     * \n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public List<String> getRandomNodes(int max)\n            throws ConnectionException, UnknownException {\n        if (max <= 0) {\n            throw new IllegalArgumentException(\"max must be an integer > 0\");\n        }\n        final OtpErlangObject received_raw = connection.doRPC(\"api_vm\", \"get_other_vms\",\n                    new OtpErlangObject[] { ErlangValue.convertToErlang(max) });\n        try {\n            final OtpErlangList list = ErlangValue.otpObjectToOtpList(received_raw);\n            final ArrayList<String> result = new ArrayList<String>(list.arity());\n            for (int i = 0; i < list.arity(); ++i) {\n                OtpErlangTuple connTuple = ((OtpErlangTuple) list.elementAt(i));\n                if (connTuple.arity() != 4) {\n                    throw new UnknownException(received_raw);\n                }\n                OtpErlangAtom name_otp = (OtpErlangAtom) connTuple.elementAt(i);\n                result.add(name_otp.atomValue());\n            }\n            if (result.isEmpty()) {\n                result.add(connection.getRemote().getNode().toString());\n            }\n            return result;\n        } catch (final ClassCastException e) {\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Closes the transaction's connection to a scalaris node.\n     *\n     * Note: Subsequent calls to the other methods will throw\n     * {@link ConnectionException}s!\n     */\n    public void closeConnection() {\n        connection.close();\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/ScalarisVM.java",
    "content": "/*\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.net.Inet4Address;\nimport java.net.UnknownHostException;\nimport java.util.ArrayList;\nimport java.util.LinkedHashMap;\nimport java.util.List;\nimport java.util.Map;\n\nimport com.ericsson.otp.erlang.OtpErlangAtom;\nimport com.ericsson.otp.erlang.OtpErlangInt;\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\n/**\n * Provides methods to interact with a specific Scalaris (Erlang) VM.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.10\n * @since 3.6\n */\npublic class ScalarisVM {\n    /**\n     * Connection to a Scalaris node.\n     */\n    private final Connection connection;\n\n    /**\n     * Creates a connection to the erlang VM of the given Scalaris node. Uses\n     * the connection policy of the global connection factory.\n     *\n     * @param node\n     *            Scalaris node to connect with\n     * @throws ConnectionException\n     *             if the connection fails or the connection policy is not\n     *             cloneable\n     */\n    public ScalarisVM(final PeerNode node) throws ConnectionException {\n        final ConnectionFactory cf = ConnectionFactory.getInstance();\n        connection = cf.createConnection(new FixedNodeConnectionPolicy(node));\n    }\n\n    /**\n     * Creates a connection to the erlang VM of the given Scalaris node. Uses\n     * the connection policy of the global connection factory.\n     *\n     * @param node\n     *            Scalaris node to connect with\n     * @throws ConnectionException\n     *             if the connection fails or the connection policy is not\n     *             cloneable\n     */\n    public ScalarisVM(final String node) throws ConnectionException {\n        final ConnectionFactory cf = ConnectionFactory.getInstance();\n        connection = cf.createConnection(new FixedNodeConnectionPolicy(node));\n    }\n\n    /**\n     * Gets the version of the Scalaris VM of the current connection.\n     *\n     * @return Scalaris version string\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public String getVersion()\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_vm\", \"get_version\",\n                    new OtpErlangObject[] {});\n        try {\n            return new ErlangValue(received_raw).stringValue();\n        } catch (final ClassCastException e) {\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Plain old data object for results of {@link ScalarisVM#getInfo()}.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.6\n     * @since 3.6\n     */\n    public static class GetInfoResult {\n        /**\n         * Scalaris version string.\n         */\n        public final String scalarisVersion;\n        /**\n         * Erlang version string.\n         */\n        public final String erlangVersion;\n        /**\n         * Total amount of memory currently allocated.\n         */\n        public final int memTotal;\n        /**\n         * Uptime of the Erlang VM.\n         */\n        public final int uptime;\n        /**\n         * Erlang node name.\n         */\n        public final String erlangNode;\n        /**\n         * IP address to reach the Scalaris node inside the Erlang VM.\n         */\n        public final Inet4Address ip;\n        /**\n         * Port to reach the Scalaris node inside the Erlang VM.\n         */\n        public final int port;\n        /**\n         * Yaws port to reach the JSON API and web debug interface.\n         */\n        public final int yawsPort;\n\n        protected GetInfoResult(final String scalarisVersion, final String erlangVersion,\n                final int memTotal, final int uptime, final String erlangNode, final Inet4Address ip,\n                final int port, final int yawsPort) {\n            super();\n            this.scalarisVersion = scalarisVersion;\n            this.erlangVersion = erlangVersion;\n            this.memTotal = memTotal;\n            this.uptime = uptime;\n            this.erlangNode = erlangNode;\n            this.ip = ip;\n            this.port = port;\n            this.yawsPort = yawsPort;\n        }\n    }\n\n    /**\n     * Gets some information about the VM and Scalaris.\n     *\n     * @return VM information\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public GetInfoResult getInfo()\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_vm\", \"get_info\",\n                    new OtpErlangObject[] {});\n        try {\n            final OtpErlangList received = (OtpErlangList) received_raw;\n            final Map<String, OtpErlangObject> result = new LinkedHashMap<String, OtpErlangObject>(\n                    received.arity());\n            for (final OtpErlangObject iter : received) {\n                final OtpErlangTuple iter_tpl = (OtpErlangTuple) iter;\n                if (iter_tpl.arity() == 2) {\n                    final String key = ((OtpErlangAtom) (iter_tpl.elementAt(0))).atomValue();\n                    result.put(key, iter_tpl.elementAt(1));\n                } else {\n                    throw new UnknownException(received_raw);\n                }\n            }\n            final String scalarisVersion = new ErlangValue(result.get(\"scalaris_version\")).stringValue();\n            final String erlangVersion = new ErlangValue(result.get(\"erlang_version\")).stringValue();\n            final int memTotal = new ErlangValue(result.get(\"mem_total\")).intValue();\n            final int uptime = new ErlangValue(result.get(\"uptime\")).intValue();\n            final String erlangNode = new ErlangValue(result.get(\"erlang_node\")).stringValue();\n            final OtpErlangTuple erlIP = (OtpErlangTuple) result.get(\"ip\");\n            final Inet4Address ip = (Inet4Address) Inet4Address.getByName(\n                    erlIP.elementAt(0) + \".\" + erlIP.elementAt(1) + \".\" + erlIP.elementAt(2) + \".\" + erlIP.elementAt(3));\n            final int port = new ErlangValue(result.get(\"port\")).intValue();\n            final int yawsPort = new ErlangValue(result.get(\"yaws_port\")).intValue();\n            return new GetInfoResult(scalarisVersion, erlangVersion, memTotal, uptime, erlangNode, ip, port, yawsPort);\n        } catch (final ClassCastException e) {\n            throw new UnknownException(e, received_raw);\n        } catch (final NullPointerException e) {\n            throw new UnknownException(e, received_raw);\n        } catch (final UnknownHostException e) {\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Gets the number of nodes in the Scalaris VM of the current connection.\n     *\n     * @return number of nodes\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public int getNumberOfNodes()\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_vm\", \"number_of_nodes\",\n                    new OtpErlangObject[] {});\n        try {\n            return new ErlangValue(received_raw).intValue();\n        } catch (final ClassCastException e) {\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Gets the names of the nodes in the Scalaris VM of the current connection.\n     *\n     * @return the names of the nodes (arbitrary type!)\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public List<ErlangValue> getNodes()\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_vm\", \"get_nodes\",\n                    new OtpErlangObject[] {});\n        try {\n            return new ErlangValue(received_raw).listValue();\n        } catch (final ClassCastException e) {\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Plain old data object for results of {@link ScalarisVM#addNodes(int)}.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.6\n     * @since 3.6\n     */\n    public static class AddNodesResult {\n        /**\n         * Names of successfully added nodes.\n         */\n        public final List<ErlangValue> successful;\n        /**\n         * Error string for nodes that could not be started (empty if all nodes\n         * have been started successfully).\n         */\n        public final String errors;\n\n        protected AddNodesResult(final List<ErlangValue> successful, final String errors) {\n            this.successful = successful;\n            this.errors = errors;\n        }\n    }\n\n    /**\n     * Adds the given number of nodes to the Scalaris VM of the current connection.\n     *\n     * @param number\n     *            number of nodes to add\n     *\n     * @return result of the operation\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public AddNodesResult addNodes(final int number)\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_vm\", \"add_nodes\",\n                    new OtpErlangObject[] { new OtpErlangInt(number) });\n        try {\n            final OtpErlangTuple received = (OtpErlangTuple) received_raw;\n            final List<ErlangValue> successful = new ErlangValue(received.elementAt(0)).listValue();\n            final OtpErlangList errors = ErlangValue.otpObjectToOtpList(received.elementAt(1));\n            String error_str;\n            if (errors.arity() == 0) {\n                error_str = \"\";\n            } else {\n                error_str = errors.toString();\n            }\n            return new AddNodesResult(successful, error_str);\n        } catch (final ClassCastException e) {\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Shuts down the given node (graceful leave) inside the Scalaris VM of the\n     * current connection.\n     *\n     * @param name\n     *            the name of a node\n     *\n     * @return <tt>true</tt> if the node was shut down,\n     *         <tt>false</tt> if the node was not found\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public boolean shutdownNode(final ErlangValue name)\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_vm\", \"shutdown_node\",\n                    new OtpErlangObject[] { name.value() });\n        if (received_raw.equals(CommonErlangObjects.okAtom)) {\n            return true;\n        } else if (received_raw.equals(CommonErlangObjects.notFoundAtom)) {\n            return false;\n        }\n        throw new UnknownException(received_raw);\n    }\n\n    /**\n     * Kills the given node inside the Scalaris VM of the current connection.\n     *\n     * @param name\n     *            the name of a node\n     *\n     * @return <tt>true</tt> if the node was shut down,\n     *         <tt>false</tt> if the node was not found\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public boolean killNode(final ErlangValue name)\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_vm\", \"kill_node\",\n                    new OtpErlangObject[] { name.value() });\n        if (received_raw.equals(CommonErlangObjects.okAtom)) {\n            return true;\n        } else if (received_raw.equals(CommonErlangObjects.notFoundAtom)) {\n            return false;\n        }\n        throw new UnknownException(received_raw);\n    }\n\n    /**\n     * Shuts down the given number of nodes (graceful leave) inside the\n     * Scalaris VM of the current connection.\n     *\n     * @param number\n     *            number of nodes to shut down\n     *\n     * @return result of the operation\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public List<ErlangValue> shutdownNodes(final int number)\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_vm\", \"shutdown_nodes\",\n                    new OtpErlangObject[] { new OtpErlangInt(number) });\n        return makeDeleteResult(received_raw);\n    }\n\n    /**\n     * Kills the given number of nodes inside the Scalaris VM of the current\n     * connection.\n     *\n     * @param number\n     *            number of nodes to kill\n     *\n     * @return result of the operation\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public List<ErlangValue> killNodes(final int number)\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_vm\", \"kill_nodes\",\n                    new OtpErlangObject[] { new OtpErlangInt(number) });\n        return makeDeleteResult(received_raw);\n    }\n\n    /**\n     * Transforms the given result from a \"delete nodes\"-operation into a\n     * {@link DeleteResult}.\n     *\n     * @param received_raw\n     *            raw erlang result\n     *\n     * @return {@link DeleteResult} object\n     *\n     * @throws UnknownException\n     *             if an error occurs during transformation\n     */\n    private final List<ErlangValue> makeDeleteResult(\n            final OtpErlangObject received_raw) throws UnknownException {\n        try {\n            return new ErlangValue(received_raw).listValue();\n        } catch (final ClassCastException e) {\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Plain old data object for results of\n     * {@link ScalarisVM#shutdownNodes(int)}, {@link ScalarisVM#shutdownNodesByName(List)},\n     * {@link ScalarisVM#killNodes(int)} and {@link ScalarisVM#killNodes(List)}.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.6\n     * @since 3.6\n     */\n    public static class DeleteNodesByNameResult {\n        /**\n         * Names of successfully deleted nodes.\n         */\n        public final List<ErlangValue> successful;\n        /**\n         * Nodes which do not exist (anymore) in the VM.\n         */\n        public final List<ErlangValue> notFound;\n\n        protected DeleteNodesByNameResult(final List<ErlangValue> successful,\n                final List<ErlangValue> notFound) {\n            this.successful = successful;\n            this.notFound = notFound;\n        }\n    }\n\n    /**\n     * Shuts down the given nodes (graceful leave) inside the Scalaris VM of the\n     * current connection.\n     *\n     * @param names\n     *            names of the nodes to shut down\n     *\n     * @return result of the operation\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public DeleteNodesByNameResult shutdownNodesByName(final List<ErlangValue> names)\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_vm\", \"shutdown_nodes_by_name\",\n                    new OtpErlangObject[] { ErlangValue.convertToErlang(names) });\n        return makeDeleteByNameResult(received_raw);\n    }\n\n    /**\n     * Kills the given nodes inside the Scalaris VM of the current connection.\n     *\n     * @param names\n     *            names of the nodes to kill\n     *\n     * @return result of the operation\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public DeleteNodesByNameResult killNodes(final List<ErlangValue> names)\n            throws ConnectionException, UnknownException {\n        final OtpErlangObject received_raw = connection.doRPC(\"api_vm\", \"kill_nodes_by_name\",\n                    new OtpErlangObject[] { ErlangValue.convertToErlang(names) });\n        return makeDeleteByNameResult(received_raw);\n    }\n\n    /**\n     * Transforms the given result from a \"delete nodes by name\"-operation into\n     * a {@link DeleteNodesByNameResult}.\n     *\n     * @param received_raw\n     *            raw erlang result\n     *\n     * @return {@link DeleteResult} object\n     *\n     * @throws UnknownException\n     *             if an error occurs during transformation\n     */\n    private final DeleteNodesByNameResult makeDeleteByNameResult(\n            final OtpErlangObject received_raw) throws UnknownException {\n        try {\n            final OtpErlangTuple received = (OtpErlangTuple) received_raw;\n            final List<ErlangValue> successful = new ErlangValue(received.elementAt(0)).listValue();\n            final List<ErlangValue> not_found = new ErlangValue(received.elementAt(1)).listValue();\n            return new DeleteNodesByNameResult(successful, not_found);\n        } catch (final ClassCastException e) {\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Retrieves additional nodes from the Scalaris VM of the current\n     * connection for use by {@link ConnectionFactory#addNode(String)}.\n     *\n     * @param max\n     *            maximum number of nodes to return (> 0)\n     *\n     * @return a list of nodes (may be empty!)\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public List<String> getOtherVMs(final int max)\n            throws ConnectionException, UnknownException {\n        if (max <= 0) {\n            throw new IllegalArgumentException(\"max must be an integer > 0\");\n        }\n        final OtpErlangObject received_raw = connection.doRPC(\"api_vm\", \"get_other_vms\",\n                    new OtpErlangObject[] { ErlangValue.convertToErlang(max) });\n        try {\n            final OtpErlangList list = ErlangValue.otpObjectToOtpList(received_raw);\n            final ArrayList<String> result = new ArrayList<String>(list.arity());\n            for (int i = 0; i < list.arity(); ++i) {\n                final OtpErlangTuple connTuple = ((OtpErlangTuple) list.elementAt(i));\n                if (connTuple.arity() != 4) {\n                    throw new UnknownException(received_raw);\n                }\n                final OtpErlangAtom name_otp = (OtpErlangAtom) connTuple.elementAt(0);\n                result.add(name_otp.atomValue());\n            }\n            return result;\n        } catch (final ClassCastException e) {\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Tells the Scalaris VM of the current connection to shut down gracefully.\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public void shutdownVM() throws ConnectionException {\n        connection.sendRPC(\"api_vm\", \"shutdown_vm\", new OtpErlangObject[] {});\n    }\n\n    /**\n     * Kills the Scalaris VM of the current connection.\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public void killVM() throws ConnectionException {\n        connection.sendRPC(\"api_vm\", \"kill_vm\", new OtpErlangObject[] {});\n    }\n\n    /**\n     * Closes the transaction's connection to a scalaris node.\n     *\n     * Note: Subsequent calls to the other methods will throw\n     * {@link ConnectionException}s!\n     */\n    public void closeConnection() {\n        connection.close();\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/TimeoutException.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\nimport com.ericsson.otp.erlang.OtpErlangObject;\n\n/**\n * Exception that is thrown if a read or write operation on a scalaris ring\n * fails due to a timeout.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.2\n * @since 2.0\n */\npublic class TimeoutException extends OtpErlangException {\n    /**\n     * class version for serialisation\n     */\n    private static final long serialVersionUID = 1L;\n\n    /**\n     * Creates the exception with no message.\n     */\n    public TimeoutException() {\n    }\n\n    /**\n     * Creates the exception with the given message.\n     *\n     * @param msg\n     *            message of the exception\n     */\n    public TimeoutException(final String msg) {\n        super(msg);\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e the exception to \"re-throw\"\n     */\n    public TimeoutException(final Throwable e) {\n        super(e.getMessage());\n        setStackTrace(e.getStackTrace());\n    }\n\n    /**\n     * Creates an exception including the message of the given erlang object.\n     *\n     * @param erlValue\n     *            the erlang message to include\n     *\n     * @since 2.2\n     */\n    public TimeoutException(final OtpErlangObject erlValue) {\n        super(\"Erlang message: \" + erlValue.toString());\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e\n     *            the exception to \"re-throw\"\n     * @param erlValue\n     *            the string representation of this erlang value is included\n     *            into the message\n     *\n     * @since 2.2\n     */\n    public TimeoutException(final Throwable e, final OtpErlangObject erlValue) {\n        super(e.getMessage() + \",\\n  Erlang message: \" + erlValue.toString());\n        setStackTrace(e.getStackTrace());\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/Transaction.java",
    "content": "/*\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.util.HashSet;\nimport java.util.LinkedHashMap;\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpErlangDouble;\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangLong;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\nimport de.zib.scalaris.operations.AddDelOnListOp;\nimport de.zib.scalaris.operations.AddOnNrOp;\nimport de.zib.scalaris.operations.CommitOp;\nimport de.zib.scalaris.operations.Operation;\nimport de.zib.scalaris.operations.TestAndSetOp;\nimport de.zib.scalaris.operations.TransactionOperation;\nimport de.zib.scalaris.operations.WriteOp;\n\n/**\n * Provides means to realise a transaction with the scalaris ring using Java.\n *\n * <p>\n * Instances of this class can be generated using a given connection to a\n * scalaris node using {@link #Transaction(Connection)} or without a\n * connection ({@link #Transaction()}) in which case a new connection is\n * created using {@link ConnectionFactory#createConnection()}.\n * </p>\n *\n * <p>\n * There are two paradigms for reading and writing values:\n * <ul>\n *  <li> using arbitrary erlang objects extending OtpErlangObject:\n *       {@link #read(OtpErlangString)},\n *       {@link #write(OtpErlangString, OtpErlangObject)}\n *  <li> using (supported) Java objects:\n *       {@link #read(String)}, {@link #write(String, Object)}\n *       <p>These types can be accessed from any Scalaris API and translate to\n *       each language's native types, e.g. String and OtpErlangString.\n *       A list of supported types can be found in the {@link ErlangValue}\n *       class which will perform the conversion.</p>\n *       <p>Additional (custom) types can be used by providing a class that\n *       extends the {@link ErlangValue} class.\n *       The user can specify custom behaviour but the correct\n *       handling of these values is at the user's hand.</p>\n *       <p>An example using erlang objects to improve performance for\n *       inserting strings is provided by\n *       {@link de.zib.scalaris.examples.ErlangValueFastString} and can be\n *       tested by {@link de.zib.scalaris.examples.FastStringBenchmark}.</p>\n * </ul>\n * </p>\n *\n * <h3>Example:</h3>\n * <pre>\n * <code style=\"white-space:pre;\">\n *   OtpErlangString otpKey;\n *   OtpErlangString otpValue;\n *   OtpErlangObject otpResult;\n *\n *   String key;\n *   String value;\n *   String result;\n *\n *   Transaction t1 = new Transaction();  // {@link #Transaction()}\n *\n *   t1.write(key, value);                // {@link #write(String, Object)}\n *   t1.write(otpKey, otpValue);          // {@link #write(OtpErlangString, OtpErlangObject)}\n *\n *   result = t1.read(key).stringValue(); //{@link #read(String)}\n *   otpResult = t1.read(otpKey).value(); //{@link #read(OtpErlangString)}\n *\n *   transaction.commit(); // {@link #commit()}\n * </code>\n * </pre>\n *\n * <p>\n * For more examples, have a look at\n * {@link de.zib.scalaris.examples.TransactionReadExample},\n * {@link de.zib.scalaris.examples.TransactionWriteExample} and\n * {@link de.zib.scalaris.examples.TransactionReadWriteExample}.\n * </p>\n *\n * <h3>Connection errors</h3>\n *\n * Errors when setting up connections or trying to send/receive RPCs will be\n * handed to the {@link ConnectionPolicy} that has been set when the connection\n * was created. By default, {@link ConnectionFactory} uses\n * {@link DefaultConnectionPolicy} which implements automatic connection-retries\n * by classifying nodes as good or bad depending on their previous state. The\n * number of automatic retries is adjustable (default: 3).\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 2.0\n */\npublic class Transaction extends\n        AbstractTransaction<de.zib.scalaris.Transaction.RequestList, de.zib.scalaris.Transaction.ResultList> {\n    /**\n     * Erlang transaction log.\n     */\n    protected final Translog transLog = getTranslogImpl();\n\n    /**\n     * Gets the {@link Translog} implementation to use for {@link #transLog}.\n     * Will be executed only once during construction of the object!\n     *\n     * Re-define in sub-classes to use a different implementation.\n     *\n     * @return a translog object\n     */\n    protected Translog getTranslogImpl() {\n        return new FilteringTransLog();\n    }\n\n    /**\n     * Constructor, uses the default connection returned by\n     * {@link ConnectionFactory#createConnection()}.\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     */\n    public Transaction() throws ConnectionException {\n        super();\n    }\n\n    /**\n     * Constructor, uses the given connection to an erlang node.\n     *\n     * @param conn\n     *            connection to use for the transaction\n     */\n    public Transaction(final Connection conn) {\n        super(conn);\n    }\n\n    /**\n     * Encapsulates requests that can be used for transactions in\n     * {@link Transaction#req_list(RequestList)}.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.18\n     * @since 3.4\n     */\n    public static class RequestList extends de.zib.scalaris.RequestList {\n        /**\n         * Default constructor.\n         */\n        public RequestList() {\n            super();\n        }\n\n        /**\n         * Creates a new request list with the given operation.\n         *\n         * Provided for convenience.\n         *\n         * @since 3.18\n         */\n        protected RequestList(final TransactionOperation op) {\n            super(op);\n        }\n\n        /**\n         * Copy constructor.\n         *\n         * @param other the request list to copy from\n         */\n        public RequestList(final RequestList other) {\n            super(other);\n        }\n\n        /* (non-Javadoc)\n         * @see de.zib.scalaris.RequestList#addOp(de.zib.scalaris.operations.Operation)\n         */\n        @Override\n        public RequestList addOp(final Operation op)\n                throws UnsupportedOperationException {\n            if (!(op instanceof TransactionOperation)) {\n                throw new UnsupportedOperationException();\n            }\n            return (RequestList) super.addOp(op);\n        }\n\n        /**\n         * Adds all requests of the other request list to the end of this list.\n         *\n         * @param other another request list\n         *\n         * @return this {@link RequestList} object\n         */\n        public RequestList addAll(final RequestList other) {\n            return (RequestList) super.addAll_(other);\n        }\n    }\n\n    /**\n     * Encapsulates a list of results as returned by\n     * {@link Transaction#req_list(RequestList)}.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.5\n     * @since 3.4\n     */\n    public static class ResultList extends de.zib.scalaris.ResultList {\n        /**\n         * Default constructor.\n         *\n         * @param results\n         *            the raw results list as returned by scalaris\n         * @param compressed\n         *            whether the value part in the term is encoded, i.e.\n         *            compressed into an Erlang binary, or not\n         * @param requests\n         *            request list which created this result list\n         */\n        ResultList(final OtpErlangList results, final boolean compressed,\n                final RequestList requests) {\n            super(results, compressed, requests);\n        }\n\n        /**\n         * Processes the result at the given position which originated from\n         * a write request.\n         *\n         * @param pos\n         *            the position in the result list (starting at 0)\n         *\n         * @throws UnknownException\n         *             if any other error occurs\n         */\n        @Override\n        public void processWriteAt(final int pos) throws UnknownException {\n            ((WriteOp) get(pos)).processResult();\n        }\n\n        /**\n         * Processes the result at the given position which originated from\n         * a add_del_on_list request.\n         *\n         * @param pos\n         *            the position in the result list (starting at 0)\n         *\n         * @throws NotAListException\n         *             if the previously stored value was no list\n         * @throws UnknownException\n         *             if any other error occurs\n         *\n         * @since 3.9\n         */\n        @Override\n        public void processAddDelOnListAt(final int pos)\n                throws NotAListException, UnknownException {\n            ((AddDelOnListOp) get(pos)).processResult();\n        }\n\n        /**\n         * Processes the result at the given position which originated from\n         * an add_on_nr request.\n         *\n         * @param pos\n         *            the position in the result list (starting at 0)\n         *\n         * @throws NotANumberException\n         *             if the previously stored value was not a number\n         * @throws UnknownException\n         *             if any other error occurs\n         *\n         * @since 3.9\n         */\n        @Override\n        public void processAddOnNrAt(final int pos) throws NotANumberException,\n                UnknownException {\n            ((AddOnNrOp) get(pos)).processResult();\n        }\n\n        /**\n         * Processes the result at the given position which originated from\n         * an add_on_nr request.\n         *\n         * @param pos\n         *            the position in the result list (starting at 0)\n         *\n         * @throws NotFoundException\n         *             if the requested key does not exist\n         * @throws KeyChangedException\n         *             if the key did not match <tt>old_value</tt>\n         * @throws UnknownException\n         *             if any other error occurs\n         *\n         * @since 3.8\n         */\n        @Override\n        public void processTestAndSetAt(final int pos)\n                throws NotFoundException, KeyChangedException, UnknownException {\n            ((TestAndSetOp) get(pos)).processResult();\n        }\n\n        /**\n         * Processes the result at the given position which originated from\n         * a commit request.\n         *\n         * Note: it is not necessary to call this method manually! A commit at\n         * the end of a request list will be evaluated automatically!\n         *\n         * @param pos\n         *            the position in the result list (starting at 0)\n         *\n         * @throws AbortException\n         *             if the commit failed\n         * @throws UnknownException\n         *             if any other error occurs\n         */\n        public void processCommitAt(final int pos) throws AbortException,\n                UnknownException {\n            ((CommitOp) get(pos)).processResult();\n        }\n    }\n\n    /**\n     * TransLog abstraction layer, tightly coupled with <tt>tx_tlog</tt>.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.17\n     * @since 3.17\n     */\n    protected static interface Translog {\n        /**\n         * Merges a new tlog object (from Scalaris) with this translog by adding\n         * new entries and replacing existing ones.\n         *\n         * @param newTLog\n         *            new tlog entries from Scalaris\n         *\n         * @return this object\n         */\n        public abstract Translog merge(final OtpErlangObject newTLog);\n\n        /**\n         * Checks whether the translog is empty.\n         *\n         * @return <tt>true</tt> if empty\n         */\n        public abstract boolean isEmpty();\n\n        /**\n         * Clears the translog.\n         */\n        public abstract void reset();\n\n        /**\n         * Creates a minimal tlog only containing entries for the keys in the\n         * given request list.\n         *\n         * @param req\n         *            the request list\n         *\n         * @return minimal tlog\n         */\n        public abstract OtpErlangObject filter(final RequestList req);\n    }\n\n    /**\n     * TransLog abstraction layer sending the whole tlog for each request,\n     * using the Scalaris-provided tlog as is.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.17\n     * @since 3.17\n     */\n    protected static class FullTransLog implements Translog {\n        protected OtpErlangObject tlog = null;\n\n        /* (non-Javadoc)\n         * @see de.zib.scalaris.Translog#merge(com.ericsson.otp.erlang.OtpErlangObject)\n         */\n        @Override\n        public Translog merge(final OtpErlangObject newTLog) {\n            this.tlog = newTLog;\n            return this;\n        }\n\n        /* (non-Javadoc)\n         * @see de.zib.scalaris.Translog#isEmpty()\n         */\n        @Override\n        public boolean isEmpty() {\n            return this.tlog == null;\n        }\n\n        /* (non-Javadoc)\n         * @see de.zib.scalaris.Translog#reset()\n         */\n        @Override\n        public void reset() {\n            this.tlog = null;\n        }\n\n        /* (non-Javadoc)\n         * @see de.zib.scalaris.Translog#filter(de.zib.scalaris.Transaction.RequestList)\n         */\n        @Override\n        public OtpErlangObject filter(final RequestList req) {\n            return this.tlog;\n        }\n    }\n\n    /**\n     * TransLog abstraction layer only sending the subset of the tlog that is\n     * actually required for a request, tightly coupled with <tt>tx_tlog</tt>.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.17\n     * @since 3.17\n     */\n    protected static class FilteringTransLog implements Translog {\n        protected LinkedHashMap<OtpErlangString, OtpErlangTuple> entries = new LinkedHashMap<OtpErlangString, OtpErlangTuple>();\n\n        /* (non-Javadoc)\n         * @see de.zib.scalaris.Translog#merge(com.ericsson.otp.erlang.OtpErlangObject)\n         */\n        @Override\n        public Translog merge(final OtpErlangObject newTLog) {\n            try {\n                final OtpErlangList newTLogL = (OtpErlangList) newTLog;\n                for (int i = 0; i < newTLogL.arity(); ++i) {\n                    final OtpErlangTuple entry = (OtpErlangTuple) newTLogL.elementAt(i);\n                    final OtpErlangString key = ErlangValue.otpObjectToOtpString(entry.elementAt(1));\n                    entries.put(key, entry);\n                }\n            } catch (final ClassCastException e) {\n                throw new UnknownException(newTLog);\n            }\n            return this;\n        }\n\n        /* (non-Javadoc)\n         * @see de.zib.scalaris.Translog#isEmpty()\n         */\n        @Override\n        public boolean isEmpty() {\n            return entries.isEmpty();\n        }\n\n        /* (non-Javadoc)\n         * @see de.zib.scalaris.Translog#reset()\n         */\n        @Override\n        public void reset() {\n            entries.clear();\n        }\n\n        /* (non-Javadoc)\n         * @see de.zib.scalaris.Translog#filter(de.zib.scalaris.Transaction.RequestList)\n         */\n        @Override\n        public OtpErlangObject filter(final RequestList req) {\n            OtpErlangList result;\n            if (req.isCommit()) {\n                result = new OtpErlangList(entries.values()\n                        .toArray(new OtpErlangTuple[entries.size()]));\n            } else {\n                final HashSet<OtpErlangTuple> resultJ = new HashSet<OtpErlangTuple>(\n                        req.size());\n                for (final Operation op : req.getRequests()) {\n                    final OtpErlangTuple entry = entries.get(op.getKey());\n                    if (entry != null) {\n                        resultJ.add(entry);\n                    }\n                }\n                result = new OtpErlangList(\n                        resultJ.toArray(new OtpErlangTuple[resultJ.size()]));\n            }\n            return result;\n        }\n    }\n\n    /**\n     * Executes the given operation.\n     *\n     * @param op\n     *            the operation to execute\n     *\n     * @return results list containing a single result of the given operation\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws AbortException\n     *             if a commit failed (if there was one)\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @see #req_list(RequestList)\n     *\n     * @since 3.18\n     */\n    public ResultList req_list(final TransactionOperation op)\n            throws ConnectionException, AbortException, UnknownException {\n        return super.req_list(op);\n    }\n\n    /**\n     * Executes all requests in <code>req</code>.\n     *\n     * <p>\n     * The transaction's log is reset if a commit in the request list was\n     * successful, otherwise it still retains in the transaction which must be\n     * successfully committed, aborted or reset in order to be (re-)used for\n     * another request.\n     * </p>\n     *\n     * @param req\n     *            the requests to issue\n     *\n     * @return results of all requests in the same order as they appear in\n     *         <code>req</code>\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws AbortException\n     *             if the commit failed\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    @Override\n    public ResultList req_list(final RequestList req)\n            throws ConnectionException, AbortException, UnknownException {\n        if (req.isEmpty()) {\n            return new ResultList(new OtpErlangList(), compressed, req);\n        }\n        OtpErlangObject received_raw = null;\n        final OtpErlangList erlangReqList = req.getErlangReqList(compressed);\n        if (transLog.isEmpty()) {\n            received_raw = connection.doRPC(module(), \"req_list\",\n                    new OtpErlangObject[] { erlangReqList });\n        } else {\n            received_raw = connection.doRPC(module(), \"req_list\",\n                    new OtpErlangObject[] { transLog.filter(req), erlangReqList });\n        }\n        try {\n            /*\n             * possible return values:\n             *  {tx_tlog:tlog(), [{ok} | {ok, Value} | {fail, abort | timeout | not_found}]}\n             */\n            final OtpErlangTuple received = (OtpErlangTuple) received_raw;\n            transLog.merge(received.elementAt(0));\n            if (received.arity() == 2) {\n                final ResultList result = new ResultList((OtpErlangList) received.elementAt(1), compressed, req);\n                if (req.isCommit()) {\n                    req.getCommit().processResult();\n                    // transaction was successful: reset transaction log\n                    transLog.reset();\n                }\n                return result;\n            }\n            throw new UnknownException(received_raw);\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    /**\n     * Commits the current transaction.\n     *\n     * <p>\n     * The transaction's log is reset if the commit was successful, otherwise it\n     * still retains in the transaction which must be successfully committed,\n     * aborted or reset in order to be (re-)used for another request.\n     * </p>\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws AbortException\n     *             if the commit failed\n     * @throws UnknownException\n     *             If the commit fails or the returned value from erlang is of\n     *             an unknown type/structure, this exception is thrown. Neither\n     *             the transaction log nor the local operations buffer is\n     *             emptied, so that the commit can be tried again.\n     *\n     * @see #abort()\n     */\n    public void commit() throws ConnectionException, AbortException, UnknownException {\n        req_list((RequestList) new RequestList().addCommit());\n    }\n\n    /**\n     * Cancels the current transaction.\n     *\n     * <p>\n     * For a transaction to be cancelled, only the {@link #transLog} needs to be\n     * reset. Nothing else needs to be done since the data was not modified\n     * until the transaction was committed.\n     * </p>\n     *\n     * @see #commit()\n     */\n    public void abort() {\n        transLog.reset();\n    }\n\n    @Override\n    protected RequestList newReqList() {\n        return new RequestList();\n    }\n\n    // some overrides (we do not commit transactions in these methods and thus\n    // do not throw an AbortException):\n\n    @Override\n    public void write(final OtpErlangString key, final OtpErlangObject value)\n            throws ConnectionException, UnknownException {\n        try {\n            super.write(key, value);\n        } catch (final AbortException e) {\n            // should not occur (we did not commit anything)\n            throw new UnknownException(e);\n        }\n    }\n\n    @Override\n    public <T> void write(final String key, final T value)\n            throws ConnectionException, UnknownException {\n        try {\n            super.write(key, value);\n        } catch (final AbortException e) {\n            // should not occur (we did not commit anything)\n            throw new UnknownException(e);\n        }\n    }\n\n    @Override\n    public void addDelOnList(final OtpErlangString key,\n            final OtpErlangList toAdd, final OtpErlangList toRemove)\n            throws ConnectionException, NotAListException, UnknownException {\n        try {\n            super.addDelOnList(key, toAdd, toRemove);\n        } catch (final AbortException e) {\n            // should not occur (we did not commit anything)\n            throw new UnknownException(e);\n        }\n    }\n\n    @Override\n    public <T> void addDelOnList(final String key, final List<T> toAdd,\n            final List<T> toRemove) throws ConnectionException,\n            NotAListException, UnknownException {\n        try {\n            super.addDelOnList(key, toAdd, toRemove);\n        } catch (final AbortException e) {\n            // should not occur (we did not commit anything)\n            throw new UnknownException(e);\n        }\n    }\n\n    @Override\n    public void addOnNr(final OtpErlangString key, final OtpErlangLong toAdd)\n            throws ConnectionException, NotANumberException, UnknownException {\n        try {\n            super.addOnNr(key, toAdd);\n        } catch (final AbortException e) {\n            // should not occur (we did not commit anything)\n            throw new UnknownException(e);\n        }\n    }\n\n    @Override\n    public void addOnNr(final OtpErlangString key, final OtpErlangDouble toAdd)\n            throws ConnectionException, NotANumberException, UnknownException {\n        try {\n            super.addOnNr(key, toAdd);\n        } catch (final AbortException e) {\n            // should not occur (we did not commit anything)\n            throw new UnknownException(e);\n        }\n    }\n\n    @Override\n    public <T> void addOnNr(final String key, final T toAdd)\n            throws ConnectionException, NotANumberException, UnknownException {\n        try {\n            super.addOnNr(key, toAdd);\n        } catch (final AbortException e) {\n            // should not occur (we did not commit anything)\n            throw new UnknownException(e);\n        }\n    }\n\n    @Override\n    public void testAndSet(final OtpErlangString key,\n            final OtpErlangObject oldValue, final OtpErlangObject newValue)\n            throws ConnectionException, NotFoundException, KeyChangedException,\n            UnknownException {\n        try {\n            super.testAndSet(key, oldValue, newValue);\n        } catch (final AbortException e) {\n            // should not occur (we did not commit anything)\n            throw new UnknownException(e);\n        }\n    }\n\n    @Override\n    public <OldT, NewT> void testAndSet(final String key, final OldT oldValue,\n            final NewT newValue) throws ConnectionException, NotFoundException,\n            KeyChangedException, UnknownException {\n        try {\n            super.testAndSet(key, oldValue, newValue);\n        } catch (final AbortException e) {\n            // should not occur (we did not commit anything)\n            throw new UnknownException(e);\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/TransactionSingleOp.java",
    "content": "/*\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\n\nimport de.zib.scalaris.operations.AddDelOnListOp;\nimport de.zib.scalaris.operations.AddOnNrOp;\nimport de.zib.scalaris.operations.Operation;\nimport de.zib.scalaris.operations.TestAndSetOp;\nimport de.zib.scalaris.operations.TransactionSingleOpOperation;\nimport de.zib.scalaris.operations.WriteOp;\n\n/**\n * Provides methods to read and write key/value pairs to/from a scalaris ring.\n *\n * <p>\n * Each operation is a single transaction. If you are looking for more\n * transactions, use the {@link Transaction} class instead.\n * </p>\n *\n * <p>\n * Instances of this class can be generated using a given connection to a\n * scalaris node ({@link #TransactionSingleOp(Connection)}) or without a\n * connection ({@link #TransactionSingleOp()}) in which case a new connection\n * is created using {@link ConnectionFactory#createConnection()}.\n * </p>\n *\n * <p>\n * There are two paradigms for reading and writing values:\n * <ul>\n *  <li> using arbitrary erlang objects extending OtpErlangObject:\n *       {@link #read(OtpErlangString)},\n *       {@link #write(OtpErlangString, OtpErlangObject)}\n *  <li> using (supported) Java objects:\n *       {@link #read(String)}, {@link #write(String, Object)}\n *       <p>These types can be accessed from any Scalaris API and translate to\n *       each language's native types, e.g. String and OtpErlangString.\n *       A list of supported types can be found in the {@link ErlangValue}\n *       class which will perform the conversion.</p>\n *       <p>Additional (custom) types can be used by providing a class that\n *       extends the {@link ErlangValue} class.\n *       The user can specify custom behaviour but the correct\n *       handling of these values is at the user's hand.</p>\n *       <p>An example using erlang objects to improve performance for\n *       inserting strings is provided by\n *       {@link de.zib.scalaris.examples.ErlangValueFastString} and can be\n *       tested by {@link de.zib.scalaris.examples.FastStringBenchmark}.</p>\n * </ul>\n * </p>\n *\n * <h3>Reading values</h3>\n * <pre>\n * <code style=\"white-space:pre;\">\n *   String key;\n *   OtpErlangString otpKey;\n *\n *   TransactionSingleOp sc = new TransactionSingleOp();\n *   String value             = sc.read(key).stringValue(); // {@link #read(String)}\n *   OtpErlangObject optValue = sc.read(otpKey).value();    // {@link #read(OtpErlangString)}\n * </code>\n * </pre>\n *\n * <p>For the full example, see\n * {@link de.zib.scalaris.examples.TransactionSingleOpReadExample}</p>\n *\n * <h3>Writing values</h3>\n * <pre>\n * <code style=\"white-space:pre;\">\n *   String key;\n *   String value;\n *   OtpErlangString otpKey;\n *   OtpErlangString otpValue;\n *\n *   TransactionSingleOp sc = new TransactionSingleOp();\n *   sc.write(key, value);             // {@link #write(String, Object)}\n *   sc.writeObject(otpKey, otpValue); // {@link #write(OtpErlangString, OtpErlangObject)}\n * </code>\n * </pre>\n *\n * <p>For the full example, see\n * {@link de.zib.scalaris.examples.TransactionSingleOpWriteExample}</p>\n *\n * <h3>Connection errors</h3>\n *\n * Errors when setting up connections or trying to send/receive RPCs will be\n * handed to the {@link ConnectionPolicy} that has been set when the connection\n * was created. By default, {@link ConnectionFactory} uses\n * {@link DefaultConnectionPolicy} which implements automatic connection\n * retries by classifying nodes as good or bad depending on their previous\n * state. The number of automatic retries is adjustable (default: 3).\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 2.0\n */\npublic class TransactionSingleOp extends\n        AbstractTransaction<de.zib.scalaris.TransactionSingleOp.RequestList, de.zib.scalaris.TransactionSingleOp.ResultList> {\n\n    /**\n     * Constructor, uses the default connection returned by\n     * {@link ConnectionFactory#createConnection()}.\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     */\n    public TransactionSingleOp() throws ConnectionException {\n        super();\n    }\n\n    /**\n     * Constructor, uses the given connection to an erlang node.\n     *\n     * @param conn\n     *            connection to use for the transaction\n     */\n    public TransactionSingleOp(final Connection conn) {\n        super(conn);\n    }\n\n    /**\n     * Encapsulates requests that can be used for transactions in\n     * {@link TransactionSingleOp#req_list(RequestList)}.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.18\n     * @since 3.5\n     */\n    public static class RequestList extends de.zib.scalaris.RequestList {\n        /**\n         * Default constructor.\n         */\n        public RequestList() {\n            super();\n        }\n\n        /**\n         * Creates a new request list with the given operation.\n         *\n         * Provided for convenience.\n         *\n         * @since 3.18\n         */\n        protected RequestList(final TransactionSingleOpOperation op) {\n            super(op);\n        }\n\n        /**\n         * Copy constructor.\n         *\n         * @param other the request list to copy from\n         */\n        public RequestList(final RequestList other) {\n            super(other);\n        }\n\n        /* (non-Javadoc)\n         * @see de.zib.scalaris.RequestList#addOp(de.zib.scalaris.operations.Operation)\n         */\n        @Override\n        public RequestList addOp(final Operation op)\n                throws UnsupportedOperationException {\n            if (!(op instanceof TransactionSingleOpOperation)) {\n                throw new UnsupportedOperationException();\n            }\n            return (RequestList) super.addOp(op);\n        }\n\n        /**\n         * Throws an {@link UnsupportedOperationException} as a commit is not\n         * supported here.\n         *\n         * @return this {@link RequestList} object\n         *\n         * @throws UnsupportedOperationException\n         *             always thrown in this class\n         */\n        @Override\n        public RequestList addCommit() {\n            throw new UnsupportedOperationException();\n        }\n\n        /**\n         * Adds all requests of the other request list to the end of this list.\n         *\n         * @param other another request list\n         *\n         * @return this {@link RequestList} object\n         */\n        public RequestList addAll(final RequestList other) {\n            return (RequestList) super.addAll_(other);\n        }\n    }\n\n    /**\n     * Encapsulates a list of results as returned by\n     * {@link TransactionSingleOp#req_list(RequestList)}.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.8\n     * @since 3.5\n     */\n    public static class ResultList extends de.zib.scalaris.ResultList {\n        /**\n         * Default constructor.\n         *\n         * @param results\n         *            the raw results list as returned by scalaris\n         * @param compressed\n         *            whether the value part in the term is encoded, i.e.\n         *            compressed into an Erlang binary, or not\n         * @param requests\n         *            request list which created this result list\n         */\n        ResultList(final OtpErlangList results, final boolean compressed,\n                final RequestList requests) {\n            super(results, compressed, requests);\n        }\n\n        /**\n         * Processes the result at the given position which originated from\n         * a write request.\n         *\n         * @param pos\n         *            the position in the result list (starting at 0)\n         *\n         * @throws AbortException\n         *             if the commit of the write failed\n         * @throws UnknownException\n         *             if any other error occurs\n         */\n        @Override\n        public void processWriteAt(final int pos) throws AbortException,\n                UnknownException {\n            ((WriteOp) get(pos)).processResultSingle();\n        }\n\n        /**\n         * Processes the result at the given position which originated from\n         * a add_del_on_list request.\n         *\n         * @param pos\n         *            the position in the result list (starting at 0)\n         *\n         * @throws NotAListException\n         *             if the previously stored value was no list\n         * @throws AbortException\n         *             if the commit of the write failed\n         * @throws UnknownException\n         *             if any other error occurs\n         *\n         * @since 3.9\n         */\n        @Override\n        public void processAddDelOnListAt(final int pos)\n                throws NotAListException, AbortException, UnknownException {\n            ((AddDelOnListOp) get(pos)).processResultSingle();\n        }\n\n        /**\n         * Processes the result at the given position which originated from\n         * an add_on_nr request.\n         *\n         * @param pos\n         *            the position in the result list (starting at 0)\n         *\n         * @throws NotANumberException\n         *             if the previously stored value was not a number\n         * @throws AbortException\n         *             if the commit of the write failed\n         * @throws UnknownException\n         *             if any other error occurs\n         *\n         * @since 3.9\n         */\n        @Override\n        public void processAddOnNrAt(final int pos) throws NotANumberException,\n                AbortException, UnknownException {\n            ((AddOnNrOp) get(pos)).processResultSingle();\n        }\n\n        /**\n         * Processes the result at the given position which originated from\n         * a test_and_set request.\n         *\n         * @param pos\n         *            the position in the result list (starting at 0)\n         *\n         * @throws NotFoundException\n         *             if the requested key does not exist\n         * @throws KeyChangedException\n         *             if the key did not match <tt>old_value</tt>\n         * @throws AbortException\n         *             if the commit of the write failed\n         * @throws UnknownException\n         *             if any other error occurs\n         *\n         * @since 3.8\n         */\n        @Override\n        public void processTestAndSetAt(final int pos)\n                throws NotFoundException, KeyChangedException, AbortException,\n                UnknownException {\n            ((TestAndSetOp) get(pos)).processResultSingle();\n        }\n    }\n\n    /**\n     * Executes the given operation.\n     *\n     * @param op\n     *            the operation to execute\n     *\n     * @return results list containing a single result of the given operation\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @see #req_list(RequestList)\n     *\n     * @since 3.18\n     */\n    public ResultList req_list(final TransactionSingleOpOperation op)\n            throws ConnectionException, UnknownException {\n        try {\n            return super.req_list(op);\n        } catch (final AbortException e) {\n            // should not occur!\n            throw new UnknownException(e);\n        }\n    }\n\n    /**\n     * Executes all requests in <code>req</code> and commits each one of them in\n     * a single transaction.\n     *\n     * NOTE: The execution order of multiple requests on the same key is\n     * undefined!\n     *\n     * @param req\n     *            the requests to issue\n     *\n     * @return results of all requests in the same order as they appear in\n     *         <code>req</code>\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    @Override\n    public ResultList req_list(final RequestList req)\n            throws ConnectionException, UnknownException {\n        if (req.isEmpty()) {\n            return new ResultList(new OtpErlangList(), compressed, req);\n        }\n        final OtpErlangObject received_raw = connection.doRPC(module(), \"req_list_commit_each\",\n                    new OtpErlangObject[] { req.getErlangReqList(compressed) });\n        try {\n            /*\n             * possible return values:\n             *  [api_tx:result()]\n             */\n            return new ResultList((OtpErlangList) received_raw, compressed, req);\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, received_raw);\n        }\n    }\n\n    @Override\n    protected RequestList newReqList() {\n        return new RequestList();\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/UnknownException.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport com.ericsson.otp.erlang.OtpErlangObject;\n\n/**\n * Generic exception that is thrown during operations on a scalaris ring, e.g.\n * if an unknown result has been returned.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.2\n * @since 2.0\n */\npublic class UnknownException extends RuntimeException {\n    /**\n     * class version for serialisation\n     */\n    private static final long serialVersionUID = 1L;\n\n    /**\n     * Creates the exception with no message.\n     */\n    public UnknownException() {\n    }\n\n    /**\n     * Creates the exception with the given message.\n     *\n     * @param msg\n     *            message of the exception\n     */\n    public UnknownException(final String msg) {\n        super(msg);\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e\n     *            the exception to \"re-throw\"\n     */\n    public UnknownException(final Throwable e) {\n        super(e.getMessage());\n        setStackTrace(e.getStackTrace());\n    }\n\n    /**\n     * Creates an exception including the message of the given erlang object.\n     *\n     * @param erlValue\n     *            the erlang message to include\n     *\n     * @since 2.2\n     */\n    public UnknownException(final OtpErlangObject erlValue) {\n        super(\"Erlang message: \" + erlValue.toString());\n    }\n\n    /**\n     * Creates an exception taking the message of the given throwable.\n     *\n     * @param e\n     *            the exception to \"re-throw\"\n     * @param erlValue\n     *            the string representation of this erlang value is included\n     *            into the message\n     *\n     * @since 2.2\n     */\n    public UnknownException(final Throwable e, final OtpErlangObject erlValue) {\n        super(e.getMessage() + \",\\n  Erlang message: \" + erlValue.toString());\n        setStackTrace(e.getStackTrace());\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/examples/ErlangValueBitString.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples;\n\nimport java.nio.charset.Charset;\nimport com.ericsson.otp.erlang.OtpErlangBitstr;\nimport com.ericsson.otp.erlang.OtpErlangObject;\n\nimport de.zib.scalaris.ErlangValue;\n\n/**\n * Implements a faster {@link String} storage mechanism using erlang bitstrings.\n *\n * <p>\n * Run a benchmark of the different String implementations with\n * <code>java -cp scalaris-examples.jar de.zib.scalaris.examples.FastStringBenchmark</code>\n * </p>\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.9\n * @since 2.9\n */\npublic class ErlangValueBitString extends ErlangValue {\n    /**\n     * UTF-8 charset object.\n     *\n     * StandardCharsets.UTF_8 is only available for Java >= 7\n     */\n    private static final Charset UTF_8 = Charset.forName(\"UTF-8\");\n\n    /**\n     * Creates an object with the given (Java) value.\n     *\n     * @param value\n     *            the value to use\n     */\n    public ErlangValueBitString(final String value) {\n        super(new OtpErlangBitstr(value.getBytes(UTF_8)));\n    }\n\n    /**\n     * Creates an object with the given (erlang) value.\n     *\n     * @param otpValue\n     *            the value to use\n     */\n    public ErlangValueBitString(final OtpErlangObject otpValue) {\n        super(otpValue);\n    }\n\n    /**\n     * Creates an object with the given (erlang) value.\n     * Provided for convenience.\n     *\n     * @param value\n     *            the value to use\n     *\n     * @see ErlangValue\n     */\n    public ErlangValueBitString(final ErlangValue value) {\n        super(value.value());\n    }\n\n    /**\n     * Converts the stored erlang value created by this object to a Java\n     * {@link String}.\n     *\n     * @throws ClassCastException\n     *             if the conversion fails\n     */\n    @Override\n    public String stringValue() {\n        return new String(((OtpErlangBitstr) value()).binaryValue(), UTF_8);\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/examples/ErlangValueFastString.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples;\n\nimport java.nio.charset.Charset;\nimport com.ericsson.otp.erlang.OtpErlangAtom;\nimport com.ericsson.otp.erlang.OtpErlangBinary;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\nimport de.zib.scalaris.ErlangValue;\n\n/**\n * Implements a faster {@link String} storage mechanism if only Java access to\n * scalaris is used.\n *\n * <p>\n * Uses {@link OtpErlangBinary} objects that store the array of bytes a\n * {@link String} consists of and writes this binary to scalaris wrapped into a\n * {@link OtpErlangTuple}. Trying to read strings will convert a returned\n * {@link OtpErlangBinary} to a {@link String} or return the value of a\n * {@link OtpErlangString} result.\n * </p>\n *\n * <p>\n * Run a benchmark of the different String implementations with\n * <code>java -cp scalaris-examples.jar de.zib.scalaris.examples.FastStringBenchmark</code>\n * </p>\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.9\n * @since 2.9\n */\npublic class ErlangValueFastString extends ErlangValue {\n    /**\n     * UTF-8 charset object.\n     *\n     * StandardCharsets.UTF_8 is only available for Java >= 7\n     */\n    private static final Charset UTF_8 = Charset.forName(\"UTF-8\");\n\n    /**\n     * Identifies the tuple of the stored erlang object.\n     */\n    private static OtpErlangAtom identifier = new OtpErlangAtom(\"$fs\");\n\n    /**\n     * Creates an object with the given (Java) value.\n     *\n     * @param value\n     *            the value to use\n     */\n    public ErlangValueFastString(final String value) {\n        super(new OtpErlangTuple(new OtpErlangObject[] {\n                identifier,\n                new OtpErlangBinary(value.getBytes(UTF_8)) }));\n    }\n\n    /**\n     * Creates an object with the given (erlang) value.\n     *\n     * @param otpValue\n     *            the value to use\n     */\n    public ErlangValueFastString(final OtpErlangObject otpValue) {\n        super(otpValue);\n    }\n\n    /**\n     * Creates an object with the given (erlang) value.\n     * Provided for convenience.\n     *\n     * @param value\n     *            the value to use\n     *\n     * @see ErlangValue\n     */\n    public ErlangValueFastString(final ErlangValue value) {\n        super(value.value());\n    }\n\n    /**\n     * Converts the stored erlang value created by this object to a Java\n     * {@link String}.\n     *\n     * @throws ClassCastException\n     *             if the conversion fails\n     */\n    @Override\n    public String stringValue() {\n        final OtpErlangTuple otpTuple = (OtpErlangTuple) value();\n        if (otpTuple.elementAt(0).equals(identifier)) {\n            return new String(\n                    ((OtpErlangBinary) otpTuple.elementAt(0)).binaryValue(),\n                    UTF_8);\n        }\n\n        throw new ClassCastException(\"Unexpected result type: \"\n                + value().getClass());\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/examples/FastStringBenchmark.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples;\n\nimport java.util.HashSet;\nimport java.util.List;\nimport java.util.Set;\n\nimport com.ericsson.otp.erlang.OtpErlangBinary;\nimport com.ericsson.otp.erlang.OtpErlangString;\n\nimport de.zib.scalaris.Benchmark;\nimport de.zib.scalaris.ConnectionFactory;\nimport de.zib.scalaris.PeerNode;\nimport de.zib.scalaris.RoundRobinConnectionPolicy;\n\n/**\n * Mini benchmark of the {@link de.zib.scalaris.Transaction} and\n * {@link de.zib.scalaris.TransactionSingleOp} class using custom objects\n * provided by {@link ErlangValueFastString} and {@link ErlangValueBitString}.\n *\n * <p>\n * Run the benchmark with\n * <code>java -cp scalaris-examples.jar de.zib.scalaris.examples.FastStringBenchmark</code>\n * </p>\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.0\n * @since 2.0\n */\npublic class FastStringBenchmark extends Benchmark {\n    /**\n     * Runs a mini benchmark of the {@link de.zib.scalaris.Transaction} and\n     * {@link de.zib.scalaris.TransactionSingleOp} class using custom objects\n     * provided by {@link ErlangValueFastString} and\n     * {@link ErlangValueBitString}. Accepts the same parameters as the\n     * {@link de.zib.scalaris.Main#main(String[])} method's benchmark parameter.\n     *\n     * @param args\n     *            command line arguments\n     *\n     * @see de.zib.scalaris.Main#main(String[])\n     */\n    public static void main(final String[] args) {\n        int operations = 500;\n        int threadsPerNode = 10;\n        final HashSet<Integer> benchmarks = new HashSet<Integer>(10);\n        boolean all = false;\n        if ((args == null) || (args.length == 0)) {\n            all = true;\n        } else if (args.length == 1) {\n            operations = Integer.parseInt(args[0]);\n            all = true;\n        } else if (args.length == 2) {\n            operations = Integer.parseInt(args[0]);\n            threadsPerNode = Integer.parseInt(args[1]);\n            all = true;\n        } else if (args.length >= 3) {\n            operations = Integer.parseInt(args[0]);\n            threadsPerNode = Integer.parseInt(args[1]);\n            for (int i = 2; i < Math.min(12, args.length); ++i) {\n                final String benchmarks_str = args[i];\n                if (benchmarks_str.equals(\"all\")) {\n                    all = true;\n                } else {\n                    benchmarks.add(Integer.parseInt(benchmarks_str));\n                }\n            }\n        }\n        if (all) {\n            for (int i = 1; i <= 9; ++i) {\n                benchmarks.add(i);\n            }\n        }\n        minibench(operations, threadsPerNode, benchmarks);\n    }\n\n    /**\n     * Runs the benchmark.\n     *\n     * Tests some strategies for writing key/value pairs to scalaris:\n     * <ol>\n     *  <li>writing {@link OtpErlangBinary} objects (random data, size = {@link #BENCH_DATA_SIZE})</li>\n     *  <li>writing {@link OtpErlangString} objects (random data, size = {@link #BENCH_DATA_SIZE})</li>\n     *  <li>writing {@link String} objects (random data, size = {@link #BENCH_DATA_SIZE})</li>\n     *  <li>writing {@link String} objects by converting them to {@link OtpErlangBinary}s\n     *      (random data, size = {@link #BENCH_DATA_SIZE})</li>\n     * </ol>\n     * each with the given number of consecutive operations and parallel\n     * threads per Scalaris node,\n     * <ul>\n     *  <li>first using a new {@link de.zib.scalaris.Transaction} for each test,</li>\n     *  <li>then using a new {@link de.zib.scalaris.Transaction} but re-using a single {@link de.zib.scalaris.Connection},</li>\n     *  <li>and finally re-using a single {@link de.zib.scalaris.Transaction} object.</li>\n     * </ul>\n     *\n     * @param operations\n     *            the number of test runs to execute\n     * @param threadsPerNode\n     *            number of threads to spawn for each existing Scalaris node\n     * @param benchmarks\n     *            the benchmarks to run (1-9 or -1 for all benchmarks)\n     */\n    public static void minibench(final int operations, final int threadsPerNode, final Set<Integer> benchmarks) {\n        final ConnectionFactory cf = ConnectionFactory.getInstance();\n        final List<PeerNode> nodes = cf.getNodes();\n        final int parallelRuns = nodes.size();\n        // set a connection policy that goes through the available nodes in a round-robin fashion:\n        cf.setConnectionPolicy(new RoundRobinConnectionPolicy(nodes));\n        System.out.println(\"Number of available nodes: \" + nodes.size());\n        System.out.println(\"-> Using \" + parallelRuns + \" parallel instances per test run...\");\n        long[][] results;\n        String[] columns;\n        String[] rows;\n        @SuppressWarnings(\"rawtypes\")\n        Class[] testTypes;\n        String[] testTypesStr;\n        @SuppressWarnings(\"rawtypes\")\n        Class[] testBench;\n        String testGroup;\n\n        System.out.println(\"Benchmark of de.zib.scalaris.TransactionSingleOp:\");\n        results = getResultArray(3, 3);\n        testTypes = new Class[] {String.class, ErlangValueBitString.class, ErlangValueFastString.class};\n        testTypesStr = new String[] {\"S\", \"EVBS\", \"EVFS\"};\n        columns = new String[] {\n                \"TransactionSingleOp.write(String, String)\",\n                \"TransactionSingleOp.write(String, ErlangValueBitString)\",\n                \"TransactionSingleOp.write(String, ErlangValueFastString)\" };\n        testBench = new Class[] {Benchmark.TransSingleOpBench1.class, Benchmark.TransSingleOpBench2.class, Benchmark.TransSingleOpBench3.class};\n        rows = new String[] {\n                \"separate connection\",\n                \"re-use connection\",\n                \"re-use object\" };\n        testGroup = \"transsinglebench\";\n        runBenchAndPrintResults(benchmarks, results, columns, rows, testTypes,\n                testTypesStr, testBench, testGroup, 1, operations, parallelRuns);\n\n        System.out.println(\"-----\");\n        System.out.println(\"Benchmark of de.zib.scalaris.Transaction:\");\n        results = getResultArray(3, 3);\n        testTypes = new Class[] {String.class, ErlangValueBitString.class, ErlangValueFastString.class};\n        testTypesStr = new String[] {\"S\", \"EVBS\", \"EVFS\"};\n        columns = new String[] {\n                \"Transaction.write(String, String)\",\n                \"Transaction.write(String, ErlangValueBitString)\",\n                \"Transaction.write(String, ErlangValueFastString)\" };\n        testBench = new Class[] {Benchmark.TransBench1.class, Benchmark.TransBench2.class, Benchmark.TransBench3.class};\n        rows = new String[] {\n                \"separate connection\",\n                \"re-use connection\",\n                \"re-use object\" };\n        testGroup = \"transsinglebench\";\n        runBenchAndPrintResults(benchmarks, results, columns, rows, testTypes,\n                testTypesStr, testBench, testGroup, 1, operations, parallelRuns);\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/examples/TransactionReadExample.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples;\n\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangString;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.Transaction;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Provides an example for using the <code>read</code> method of the\n * {@link Transaction} class.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.0\n * @since 2.0\n */\npublic class TransactionReadExample {\n    /**\n     * Reads all keys given on the command line (given as \"key1 key2 ...\") with\n     * the {@link Transaction#read(String)} and\n     * {@link Transaction#read(OtpErlangString)} methods in a single\n     * transaction.<br />\n     * If no keys are given, the default keys <code>\"key1\"</code>, <code>\"key2\"</code>,\n     * <code>\"key3\"</code> are used.\n     *\n     * @param args\n     *            command line arguments (optional keys to look up)\n     */\n    public static void main(final String[] args) {\n        String[] keys;\n        String value;\n\n        if (args.length == 0) {\n            keys = new String[] { \"key1\", \"key2\", \"key3\" };\n        } else {\n            keys = args;\n        }\n\n        final OtpErlangString[] otpKeys_temp = new OtpErlangString[keys.length];\n        for (int i = 0; i < keys.length; ++i) {\n            otpKeys_temp[i] = new OtpErlangString(keys[i]);\n        }\n        final OtpErlangList otpKeys = (new OtpErlangList(otpKeys_temp));\n        OtpErlangString otpValue;\n\n        System.out.println(\"Reading values with the class `Transaction`:\");\n\n        System.out.print(\"    Initialising Transaction object... \");\n        try {\n            final Transaction transaction = new Transaction();\n            System.out.println(\"done\");\n\n            System.out.print(\"    Starting transaction... \");\n            System.out.println(\"done\");\n\n            System.out\n                    .println(\"    `OtpErlangObject readObject(OtpErlangString)`...\");\n            for (int i = 0; i < otpKeys.arity(); ++i) {\n                final OtpErlangString otpKey = (OtpErlangString) otpKeys.elementAt(i);\n                try {\n                    otpValue = (OtpErlangString) transaction.read(otpKey).value();\n                    System.out.println(\"      read(\" + otpKey.stringValue()\n                            + \") == \" + otpValue.stringValue());\n                } catch (final ConnectionException e) {\n                    System.out.println(\"      read(\" + otpKey.stringValue()\n                            + \") failed: \" + e.getMessage());\n                } catch (final UnknownException e) {\n                    System.out.println(\"      read(\" + otpKey.stringValue()\n                            + \") failed with unknown: \" + e.getMessage());\n                } catch (final NotFoundException e) {\n                    System.out.println(\"      read(\" + otpKey.stringValue()\n                            + \") failed with not found: \" + e.getMessage());\n                } catch (final ClassCastException e) {\n                    System.out.println(\"      read(\" + otpKey.stringValue()\n                            + \") failed with unknown return type: \" + e.getMessage());\n                }\n            }\n\n            System.out.println(\"    `String read(String)`...\");\n            for (final String key : keys) {\n                try {\n                    value = transaction.read(key).stringValue();\n                    System.out.println(\"      read(\" + key + \") == \" + value);\n                } catch (final ConnectionException e) {\n                    System.out.println(\"      read(\" + key + \") failed: \"\n                            + e.getMessage());\n                } catch (final UnknownException e) {\n                    System.out.println(\"      read(\" + key\n                            + \") failed with unknown: \" + e.getMessage());\n                } catch (final NotFoundException e) {\n                    System.out.println(\"      read(\" + key\n                            + \") failed with not found: \" + e.getMessage());\n                }\n            }\n\n            System.out.print(\"    Committing transaction... \");\n            transaction.commit();\n            System.out.println(\"done\");\n        } catch (final ConnectionException e) {\n            System.out.println(\"failed: \" + e.getMessage());\n            return;\n        } catch (final AbortException e) {\n            System.out.println(\"failed: \" + e.getMessage());\n            return;\n        } catch (final UnknownException e) {\n            System.out.println(\"failed: \" + e.getMessage());\n            return;\n        }\n    }\n\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/examples/TransactionReadWriteExample.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples;\n\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.Transaction;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Provides an example for using the <code>read</code> and <code>write</code> methods of\n * the {@link Transaction} class together in one transaction.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.0\n * @since 2.0\n */\npublic class TransactionReadWriteExample {\n    /**\n     * Implements the following transactions each with the Java type methods and\n     * the erlang (OTP) type methods. The keys and values are given on the\n     * command line as \"key1 value1 key2 value2 key3\".<br />\n     * If no values are given, the default values\n     * <code>(key1, value1) = (\"key1\", \"value1\")</code>,\n     * <code>(key2, value2) = (\"key2\", \"value2\")</code> and <code>key3 = \"key3\"</code> are\n     * used.\n     *\n     * <h3>Transaction 1:</h3>\n     * <code style=\"white-space:pre;\">\n     *   write(key1, value1);\n     *   write(key2, value2);\n     *\n     *   result1 = read(key1);\n     *   result2 = read(key2);\n     *   result3 = read(key3);\n     *\n     *   write(key3, result1 + result2);\n     *\n     *   result3 = read(key3);\n     *\n     *   commit();\n     * </code>\n     *\n     * <h3>Transaction 2:</h3>\n     * <code style=\"white-space:pre;\">\n     *   write(key1, value1);\n     *   commit();\n     *\n     *   write(key1, \"WRONG value\");\n     *   read(key1);\n     *   abort();\n     *\n     *   read(key1);\n     *   commit();\n     * </code>\n     *\n     * @param args\n     *            command line arguments with the structure \"key1 value1 key2\n     *            value2 key3\"\n     * @see #Transaction1(String, String, String, String, String)\n     * @see #Transaction2(String, String, String, String, String)\n     */\n    public static void main(final String[] args) {\n        String key1, key2, key3;\n        String value1, value2;\n\n        if (args.length != 5) {\n            key1 = \"key1\";\n            key2 = \"key2\";\n            key3 = \"key3\";\n            value1 = \"value1\";\n            value2 = \"value2\";\n        } else {\n            key1 = args[0];\n            key2 = args[2];\n            key3 = args[4];\n            value1 = args[1];\n            value2 = args[3];\n        }\n\n        Transaction1(key1, value1, key2, value2, key3);\n        Transaction2(key1, value1, key2, value2, key3);\n    }\n\n    /**\n     * Implements the following transaction with the Java type methods and the\n     * erlang (OTP) type methods:\n     *\n     * <p>\n     * <code style=\"white-space:pre;\">\n     *   write(key1, value1);\n     *   write(key2, value2);\n     *\n     *   result1 = read(key1);\n     *   result2 = read(key2);\n     *   result3 = read(key3);\n     *\n     *   write(key3, result1 + result2);\n     *\n     *   result3 = read(key3);\n     *\n     *   commit();\n     * </code>\n     * </p>\n     *\n     * @param key1\n     *            key1 used the way described above\n     * @param value1\n     *            value1 used the way described above\n     * @param key2\n     *            key2 used the way described above\n     * @param value2\n     *            value2 used the way described above\n     * @param key3\n     *            key3 used the way described above\n     */\n    private static void Transaction1(final String key1, final String value1, final String key2,\n            final String value2, final String key3) {\n        String result1 = \"\";\n        String result2 = \"\";\n        @SuppressWarnings(\"unused\")\n        String result3 = \"\";\n        System.out.println(\"Transaction 1 using class `Transaction`:\");\n\n        System.out.print(\"    Initialising Transaction object... \");\n        try {\n            final Transaction transaction = new Transaction();\n            System.out.println(\"done\");\n\n            try {\n                System.out.print(\"    Starting transaction... \");\n                System.out.println(\"done\");\n\n                otpWrite(transaction, key1, value1);\n                otpWrite(transaction, key2, value2);\n\n                result1 = otpRead(transaction, key1);\n                result2 = otpRead(transaction, key2);\n                try {\n                    result3 = otpRead(transaction, key3);\n                } catch (final NotFoundException e) {\n//                    System.out.println(\"    caught not_found for \" + key3);\n                }\n\n                otpWrite(transaction, key3, result1 + result2);\n\n                result3 = otpRead(transaction, key3);\n\n                System.out.print(\"    Committing transaction... \");\n                transaction.commit();\n                System.out.println(\"done\");\n\n                System.out.println();\n                System.out.println(\"    --------------------\");\n                System.out.println();\n\n                System.out.print(\"    Starting transaction... \");\n                System.out.println(\"done\");\n\n                write(transaction, key1, value1);\n                write(transaction, key2, value2);\n\n                result1 = read(transaction, key1);\n                result2 = read(transaction, key2);\n                try {\n                    result3 = read(transaction, key3);\n                } catch (final NotFoundException e) {\n//                  System.out.println(\"    caught not_found for \" + key3);\n                }\n\n                write(transaction, key3, result1 + result2);\n\n                result3 = read(transaction, key3);\n\n                System.out.print(\"    Committing transaction... \");\n                transaction.commit();\n                System.out.println(\"done\");\n\n            } catch (final NotFoundException e) {\n                // read/write operation\n                transaction.abort();\n                System.out\n                        .println(\"    Transaction aborted because elements were not found although they should exist: \"\n                                + e.getMessage());\n            } catch (final AbortException e) {\n                System.out.println(\"    Transaction aborted during commit: \"\n                        + e.getMessage());\n            } catch (final UnknownException e) {\n                // any operation\n                System.out.println(\"failed: \" + e.getMessage());\n            }\n\n        } catch (final ConnectionException e) {\n            System.out.println(\"failed: \" + e.getMessage());\n        }\n        System.out.println(\"End Transaction 1\");\n    }\n\n    /**\n     * Implements the following transaction with the Java type methods and the\n     * erlang (OTP) type methods:\n     *\n     * <p>\n     * <code style=\"white-space:pre;\">\n     *   write(key1, value1);\n     *   commit();\n     *\n     *   write(key1, \"WRONG value\");\n     *   read(key1);\n     *   abort();\n     *\n     *   read(key1);\n     *   commit();\n     * </code>\n     * </p>\n     *\n     * @param key1\n     *            key1 used the way described above\n     * @param value1\n     *            value1 used the way described above\n     * @param key2\n     *            key2 used the way described above\n     * @param value2\n     *            value2 used the way described above\n     * @param key3\n     *            key3 used the way described above\n     */\n    private static void Transaction2(final String key1, final String value1, final String key2,\n            final String value2, final String key3) {\n        System.out.println(\"Transaction 2 using class `Transaction`:\");\n\n        System.out.print(\"    Initialising Transaction object... \");\n        try {\n            final Transaction transaction = new Transaction();\n            System.out.println(\"done\");\n\n            try {\n                System.out.print(\"    Starting transaction... \");\n                System.out.println(\"done\");\n\n                otpWrite(transaction, key1, value1);\n\n                System.out.print(\"    Committing transaction... \");\n                transaction.commit();\n                System.out.println(\"done\");\n\n                System.out.print(\"    Starting transaction... \");\n                System.out.println(\"done\");\n\n                otpWrite(transaction, key1, \"WRONG value\");\n                otpRead(transaction, key1);\n\n                System.out.print(\"    Aborting transaction... \");\n                transaction.abort();\n                System.out.println(\"done\");\n\n                System.out.print(\"    Starting transaction... \");\n                System.out.println(\"done\");\n\n                otpRead(transaction, key1);\n\n                System.out.print(\"    Committing transaction... \");\n                transaction.commit();\n                System.out.println(\"done\");\n\n                System.out.println();\n                System.out.println(\"    --------------------\");\n                System.out.println();\n\n                System.out.print(\"    Starting transaction... \");\n                System.out.println(\"done\");\n\n                write(transaction, key1, value1);\n\n                System.out.print(\"    Committing transaction... \");\n                transaction.commit();\n                System.out.println(\"done\");\n\n                System.out.print(\"    Starting transaction... \");\n                System.out.println(\"done\");\n\n                write(transaction, key1, \"WRONG value\");\n                read(transaction, key1);\n\n                System.out.print(\"    Aborting transaction... \");\n                transaction.abort();\n                System.out.println(\"done\");\n\n                System.out.print(\"    Starting transaction... \");\n                System.out.println(\"done\");\n\n                read(transaction, key1);\n\n                System.out.print(\"    Committing transaction... \");\n                transaction.commit();\n                System.out.println(\"done\");\n\n            } catch (final NotFoundException e) {\n                // read/write operation\n                transaction.abort();\n                System.out\n                        .println(\"    Transaction aborted because elements were not found although they should exist: \"\n                                + e.getMessage());\n            } catch (final AbortException e) {\n                System.out.println(\"    Transaction aborted during commit: \"\n                        + e.getMessage());\n            } catch (final UnknownException e) {\n                // any operation\n                System.out.println(\"failed: \" + e.getMessage());\n            }\n\n        } catch (final ConnectionException e) {\n            System.out.println(\"failed: \" + e.getMessage());\n        }\n        System.out.println(\"End Transaction 2\");\n    }\n\n    /**\n     * Writes the given <code>key</code> and <code>value</code> with the\n     * {@link Transaction#write(OtpErlangString, OtpErlangObject)} method on the\n     * given <code>transaction</code> object and generates output according to the\n     * result.\n     *\n     * @param transaction\n     *            the transaction object to operate on\n     * @param key\n     *            the key to store the value under\n     * @param value\n     *            the value to store\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    private static void otpWrite(final Transaction transaction,\n            final String key, final String value) throws ConnectionException,\n            UnknownException {\n        System.out.println(\"    `write(OtpErlangString, OtpErlangString)`...\");\n        final OtpErlangString otpKey = new OtpErlangString(key);\n        final OtpErlangString otpValue = new OtpErlangString(value);\n        try {\n            transaction.write(otpKey, otpValue);\n            System.out.println(\"      write(\" + otpKey.stringValue() + \", \"\n                    + otpValue.stringValue() + \") succeeded\");\n        } catch (final ConnectionException e) {\n            System.out.println(\"      write(\" + otpKey.stringValue() + \", \"\n                    + otpValue.stringValue() + \") failed: \" + e.getMessage());\n            throw new ConnectionException(e);\n        } catch (final UnknownException e) {\n            System.out.println(\"      write(\" + otpKey.stringValue() + \", \"\n                    + otpValue.stringValue() + \") failed with unknown: \"\n                    + e.getMessage());\n            throw new UnknownException(e);\n        }\n    }\n\n    /**\n     * Writes the given <code>key</code> and <code>value</code> with the\n     * {@link Transaction#write(String, Object)} method on the given\n     * <code>transaction</code> object and generates output according to the result.\n     *\n     * @param transaction\n     *            the transaction object to operate on\n     * @param key\n     *            the key to store the value under\n     * @param value\n     *            the value to store\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    private static void write(final Transaction transaction, final String key,\n            final String value) throws ConnectionException, UnknownException {\n        System.out.println(\"    `write(String, String)`...\");\n        try {\n            transaction.write(key, value);\n            System.out.println(\"      write(\" + key + \", \" + value\n                    + \") succeeded\");\n        } catch (final ConnectionException e) {\n            System.out.println(\"      write(\" + key + \", \" + value\n                    + \") failed: \" + e.getMessage());\n            throw new ConnectionException(e);\n        } catch (final UnknownException e) {\n            System.out.println(\"      write(\" + key + \", \" + value\n                    + \") failed with unknown: \" + e.getMessage());\n            throw new UnknownException(e);\n        }\n    }\n\n    /**\n     * Reads the given <code>key</code> with the\n     * {@link Transaction#read(OtpErlangString)} method on the given\n     * <code>transaction</code> object and generates output according to the result.\n     *\n     * @param transaction\n     *            the transaction object to operate on\n     * @param key\n     *            the key to store the value under\n     * @return the value read from scalaris\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws NotFoundException\n     *             if the requested key does not exist\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    private static String otpRead(final Transaction transaction,\n            final String key) throws ConnectionException, UnknownException,\n            NotFoundException {\n        System.out.println(\"    `OtpErlangString read(OtpErlangString)`...\");\n        final OtpErlangString otpKey = new OtpErlangString(key);\n        OtpErlangString otpValue;\n        try {\n            otpValue = (OtpErlangString) transaction.read(otpKey).value();\n            System.out.println(\"      read(\" + otpKey.stringValue() + \") == \"\n                    + otpValue.stringValue());\n            return otpValue.stringValue();\n        } catch (final ConnectionException e) {\n            System.out.println(\"      read(\" + otpKey.stringValue()\n                    + \") failed: \" + e.getMessage());\n            throw new ConnectionException(e);\n        } catch (final UnknownException e) {\n            System.out.println(\"      read(\" + otpKey.stringValue()\n                    + \") failed with unknown: \" + e.getMessage());\n            throw new UnknownException(e);\n        } catch (final NotFoundException e) {\n            System.out.println(\"      read(\" + otpKey.stringValue()\n                    + \") failed with not found: \" + e.getMessage());\n            throw new NotFoundException(e);\n        } catch (final ClassCastException e) {\n            System.out.println(\"      read(\" + otpKey.stringValue()\n                    + \") failed with unknown return type: \" + e.getMessage());\n            throw new UnknownException(e);\n        }\n    }\n\n    /**\n     * Reads the given <code>key</code> with the {@link Transaction#read(String)}\n     * method on the given <code>transaction</code> object and generates output\n     * according to the result.\n     *\n     * @param transaction\n     *            the transaction object to operate on\n     * @param key\n     *            the key to store the value under\n     * @return the value read from scalaris\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws NotFoundException\n     *             if the requested key does not exist\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    private static String read(final Transaction transaction, final String key)\n            throws ConnectionException, UnknownException, NotFoundException {\n        System.out.println(\"    `String read(String)`...\");\n        String value;\n        try {\n            value = transaction.read(key).stringValue();\n            System.out.println(\"      read(\" + key + \") == \" + value);\n            return value;\n        } catch (final ConnectionException e) {\n            System.out.println(\"      read(\" + key + \") failed: \"\n                    + e.getMessage());\n            throw new ConnectionException(e);\n        } catch (final UnknownException e) {\n            System.out.println(\"      read(\" + key + \") failed with unknown: \"\n                    + e.getMessage());\n            throw new UnknownException(e);\n        } catch (final NotFoundException e) {\n            System.out.println(\"      read(\" + key\n                    + \") failed with not found: \" + e.getMessage());\n            throw new NotFoundException(e);\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/examples/TransactionSingleOpReadExample.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples;\n\nimport com.ericsson.otp.erlang.OtpErlangString;\n\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.TransactionSingleOp;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Provides an example for using the <tt>read</tt> methods of the\n * {@link TransactionSingleOp} class.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.0\n * @since 2.0\n */\npublic class TransactionSingleOpReadExample {\n    /**\n     * Reads a key given on the command line with the <tt>read</tt> methods of\n     * {@link TransactionSingleOp}.<br />\n     * If no key is given, the default key <tt>\"key\"</tt> is used.\n     *\n     * @param args\n     *            command line arguments (first argument can be an optional key\n     *            to look up)\n     */\n    public static void main(final String[] args) {\n        String key;\n        String value;\n\n        if (args.length != 1) {\n            key = \"key\";\n        } else {\n            key = args[0];\n        }\n\n        final OtpErlangString otpKey = new OtpErlangString(key);\n        OtpErlangString otpValue;\n\n        System.out.println(\"Reading values with the class `TransactionSingleOp`:\");\n\n        try {\n            System.out.println(\"  creating object...\");\n            final TransactionSingleOp sc = new TransactionSingleOp();\n            System.out\n                    .println(\"    `OtpErlangObject readObject(OtpErlangString)`...\");\n            otpValue = (OtpErlangString) sc.read(otpKey).value();\n            System.out.println(\"      read(\" + otpKey.stringValue() + \") == \"\n                    + otpValue.stringValue());\n        } catch (final ConnectionException e) {\n            System.out.println(\"      read(\" + otpKey.stringValue()\n                    + \") failed: \" + e.getMessage());\n        } catch (final NotFoundException e) {\n            System.out.println(\"      read(\" + otpKey.stringValue()\n                    + \") failed with not found: \" + e.getMessage());\n        } catch (final ClassCastException e) {\n            System.out.println(\"      read(\" + otpKey.stringValue()\n                    + \") failed with unexpected return type: \" + e.getMessage());\n        } catch (final UnknownException e) {\n            System.out.println(\"      read(\" + otpKey.stringValue()\n                    + \") failed with unknown: \" + e.getMessage());\n        }\n\n        try {\n            System.out.println(\"  creating object...\");\n            final TransactionSingleOp sc = new TransactionSingleOp();\n            System.out.println(\"    `String read(String)`...\");\n            value = sc.read(key).stringValue();\n            System.out.println(\"      read(\" + key + \") == \" + value);\n        } catch (final ConnectionException e) {\n            System.out.println(\"      read(\" + key + \") failed: \"\n                    + e.getMessage());\n        } catch (final ClassCastException e) {\n            System.out.println(\"      read(\" + key + \") failed with unexpected return type: \"\n                    + e.getMessage());\n        } catch (final NotFoundException e) {\n            System.out.println(\"      read(\" + key + \") failed with not found: \"\n                    + e.getMessage());\n        } catch (final UnknownException e) {\n            System.out.println(\"      read(\" + key + \") failed with unknown: \"\n                    + e.getMessage());\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/examples/TransactionSingleOpWriteExample.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples;\n\nimport com.ericsson.otp.erlang.OtpErlangString;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.TransactionSingleOp;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Provides an example for using the <tt>write</tt> methods of the\n * {@link TransactionSingleOp} class.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.0\n * @since 2.0\n */\npublic class TransactionSingleOpWriteExample {\n    /**\n     * Writes a key/value pair given on the command line with the <tt>write</tt>\n     * methods of {@link TransactionSingleOp}.<br />\n     * If no value or key is given, the default key <tt>\"key\"</tt> and the\n     * default value <tt>\"value\"</tt> is used.\n     *\n     * @param args\n     *            command line arguments (first argument can be an optional key\n     *            and the second an optional value)\n     */\n    public static void main(final String[] args) {\n        String key;\n        String value;\n\n        if (args.length == 0) {\n            key = \"key\";\n            value = \"value\";\n        } else if (args.length == 1) {\n            key = args[0];\n            value = \"value\";\n        } else {\n            key = args[0];\n            value = args[1];\n        }\n\n        final OtpErlangString otpKey = new OtpErlangString(key);\n        final OtpErlangString otpValue = new OtpErlangString(value);\n\n        System.out.println(\"Writing values with the class `TransactionSingleOp`:\");\n\n        try {\n            System.out.println(\"  creating object...\");\n            final TransactionSingleOp sc = new TransactionSingleOp();\n            System.out\n                    .println(\"    `void writeObject(OtpErlangString, OtpErlangObject)`...\");\n            sc.write(otpKey, otpValue);\n            System.out.println(\"      write(\" + otpKey.stringValue() + \", \"\n                    + otpValue.stringValue() + \") succeeded\");\n        } catch (final ConnectionException e) {\n            System.out.println(\"      write(\" + otpKey.stringValue() + \", \"\n                    + otpValue.stringValue() + \") failed: \" + e.getMessage());\n        } catch (final AbortException e) {\n            System.out.println(\"      write(\" + otpKey.stringValue() + \", \"\n                    + otpValue.stringValue() + \") failed with abort: \"\n                    + e.getMessage());\n        } catch (final UnknownException e) {\n            System.out.println(\"      write(\" + otpKey.stringValue() + \", \"\n                    + otpValue.stringValue() + \") failed with unknown: \"\n                    + e.getMessage());\n        }\n\n        try {\n            System.out.println(\"  creating object...\");\n            final TransactionSingleOp sc = new TransactionSingleOp();\n            System.out.println(\"    `void write(String, String)`...\");\n            sc.write(key, value);\n            System.out.println(\"      write(\" + key + \", \" + value\n                    + \") succeeded\");\n        } catch (final ConnectionException e) {\n            System.out.println(\"      write(\" + key + \", \" + value\n                    + \") failed: \" + e.getMessage());\n        } catch (final AbortException e) {\n            System.out.println(\"      write(\" + otpKey.stringValue() + \", \"\n                    + otpValue.stringValue() + \") failed with abort: \"\n                    + e.getMessage());\n        } catch (final UnknownException e) {\n            System.out.println(\"      write(\" + key + \", \" + value\n                    + \") failed with unknown: \" + e.getMessage());\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/examples/TransactionWriteExample.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.examples;\n\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.Transaction;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Provides an example for using the <code>write</code> method of the\n * {@link Transaction} class.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.0\n * @since 2.0\n */\npublic class TransactionWriteExample {\n    /**\n     * Writes all key/value pairs given on the command line (given as \"key1\n     * value1 key2 value2 ...\") with the\n     * {@link Transaction#write(String, Object)} and\n     * {@link Transaction#write(OtpErlangString, OtpErlangObject)} methods\n     * in a single transaction.<br />\n     * If no key/value pair is given, the default pairs <code>(key1, value1)</code>,\n     * <code>(key2, value2)</code> and <code>(key3, value3)</code> are used.\n     *\n     * @param args\n     *            command line arguments (optional key/value pairs to store)\n     */\n    public static void main(final String[] args) {\n        String[] keys;\n        String[] values;\n\n        if ((args.length < 2) || ((args.length % 2) != 0)) {\n            keys = new String[] { \"key1\", \"key2\", \"key3\" };\n            values = new String[] { \"value1\", \"value2\", \"value3\" };\n        } else {\n            keys = new String[args.length / 2];\n            values = new String[args.length / 2];\n            for (int i = 0; i < args.length;) {\n                keys[i] = args[i++];\n                values[i] = args[i++];\n            }\n        }\n\n        final OtpErlangString[] otpKeys_temp = new OtpErlangString[keys.length];\n        for (int i = 0; i < keys.length; ++i) {\n            otpKeys_temp[i] = new OtpErlangString(keys[i]);\n        }\n        final OtpErlangList otpKeys = (new OtpErlangList(otpKeys_temp));\n\n        final OtpErlangString[] otpValues_temp = new OtpErlangString[values.length];\n        for (int i = 0; i < values.length; ++i) {\n            otpValues_temp[i] = new OtpErlangString(values[i]);\n        }\n        final OtpErlangList otpValues = (new OtpErlangList(otpValues_temp));\n\n        System.out.println(\"Writing values with the class `Transaction`:\");\n\n        System.out.print(\"    Initialising Transaction object... \");\n        try {\n            final Transaction transaction = new Transaction();\n            System.out.println(\"done\");\n\n            System.out.print(\"    Starting transaction... \");\n            System.out.println(\"done\");\n\n            System.out\n                    .println(\"    `writeObject(OtpErlangString, OtpErlangObject)`...\");\n            for (int i = 0; i < otpKeys.arity(); ++i) {\n                final OtpErlangString otpKey = (OtpErlangString) otpKeys.elementAt(i);\n                final OtpErlangString otpValue = (OtpErlangString) otpValues\n                        .elementAt(i);\n                try {\n                    transaction.write(otpKey, otpValue);\n                    System.out.println(\"      write(\" + otpKey.stringValue()\n                            + \", \" + otpValue.stringValue() + \") succeeded\");\n                } catch (final ConnectionException e) {\n                    System.out.println(\"      write(\" + otpKey.stringValue()\n                            + \", \" + otpValue.stringValue() + \") failed: \"\n                            + e.getMessage());\n                } catch (final UnknownException e) {\n                    System.out.println(\"      write(\" + otpKey.stringValue()\n                            + \", \" + otpValue.stringValue()\n                            + \") failed with unknown: \" + e.getMessage());\n                }\n            }\n\n            System.out.println(\"    `write(String, String)`...\");\n            for (int i = 0; i < keys.length; ++i) {\n                final String key = keys[i];\n                final String value = values[i];\n                try {\n                    transaction.write(key, value);\n                    System.out.println(\"      write(\" + key + \", \" + value\n                            + \") succeeded\");\n                } catch (final ConnectionException e) {\n                    System.out.println(\"      write(\" + key + \", \" + value\n                            + \") failed: \" + e.getMessage());\n                } catch (final UnknownException e) {\n                    System.out.println(\"      write(\" + key + \", \" + value\n                            + \") failed with unknown: \" + e.getMessage());\n                }\n            }\n\n            System.out.print(\"    Committing transaction... \");\n            transaction.commit();\n            System.out.println(\"done\");\n        } catch (final ConnectionException e) {\n            System.out.println(\"failed: \" + e.getMessage());\n            return;\n        } catch (final AbortException e) {\n            System.out.println(\"failed: \" + e.getMessage());\n            return;\n        } catch (final UnknownException e) {\n            System.out.println(\"failed: \" + e.getMessage());\n            return;\n        }\n    }\n\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/examples/package-info.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\n/**\n * This package contains examples how to use the classes of the\n * {@link de.zib.scalaris} package.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.9\n * @since 2.0\n */\npackage de.zib.scalaris.examples;\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/executor/ScalarisChangeListOp1.java",
    "content": "package de.zib.scalaris.executor;\n\nimport java.util.ArrayList;\nimport java.util.LinkedList;\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.ResultList;\nimport de.zib.scalaris.UnknownException;\nimport de.zib.scalaris.operations.ReadOp;\nimport de.zib.scalaris.operations.WriteOp;\n\n/**\n * Implements a list change operation using the read and write operations of\n * Scalaris. Supports an (optional) list counter key which is updated\n * accordingly.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.18\n */\npublic abstract class ScalarisChangeListOp1 implements ScalarisOp {\n    /**\n     * Key used to store the list.\n     */\n    protected final String key;\n    /**\n     * Key used to store the list counter.\n     */\n    protected final String countKey;\n    /**\n     * Sub-classes need to set this variable in {@link #changeList(List)},\n     * otherwise the list is not written back.\n     */\n    protected boolean listChanged = false;\n    /**\n     * Sub-classes need to set this variable in {@link #changeList(List)},\n     * otherwise the list is not written back.\n     */\n    protected boolean listCountChanged = false;\n\n    /**\n     * Creates a new list change operation.\n     *\n     * @param key       the key to change the list at\n     * @param countKey  the key for the counter of the entries in the list\n     *                  (may be <tt>null</tt>)\n     */\n    public ScalarisChangeListOp1(final String key, final String countKey) {\n        this.key = key;\n        this.countKey = countKey;\n    }\n\n    public int workPhases() {\n        return 2;\n    }\n\n    public final int doPhase(final int phase, final int firstOp,\n            final ResultList results, final RequestList requests)\n            throws OtpErlangException, UnknownException,\n            IllegalArgumentException {\n        switch (phase) {\n            case 0: return prepareRead(requests);\n            case 1: return prepareWrite(firstOp, results, requests);\n            case 2: return checkWrite(firstOp, results);\n            default:\n                throw new IllegalArgumentException(\"No phase \" + phase);\n        }\n    }\n\n    /**\n     * Adds a read operation for the list to the request list.\n     *\n     * @param requests the request list\n     *\n     * @return <tt>0</tt> (no operation processed since no results are used)\n     */\n    protected int prepareRead(final RequestList requests) {\n        requests.addOp(new ReadOp(key));\n        return 0;\n    }\n\n    /**\n     * Verifies the read operation, changes the list and adds write operations\n     * to the request list: one for the list, one for the counter (if present).\n     *\n     * @param firstOp   the first operation to process inside the result list\n     * @param results   the result list\n     * @param requests  the request list\n     *\n     * @return <tt>1</tt> operation processed (the read)\n     */\n    protected int prepareWrite(final int firstOp, final ResultList results,\n            final RequestList requests) throws OtpErlangException,\n            UnknownException {\n        assert results != null;\n        List<ErlangValue> pageList;\n        try {\n            pageList = results.processReadAt(firstOp).listValue();\n        } catch (final NotFoundException e) {\n            // this is ok\n            pageList = new LinkedList<ErlangValue>();\n        }\n        pageList = changeList(pageList);\n        if (listChanged) {\n            requests.addOp(new WriteOp(key, pageList));\n            if ((countKey != null) && listCountChanged) {\n                requests.addOp(new WriteOp(countKey, pageList.size()));\n            }\n        }\n        return 1;\n    }\n\n    /**\n     * Changes the given page list.\n     *\n     * @param pageList\n     *            the page list to change\n     *\n     * @return the new list (may be the same object as <tt>pageList</tt>)\n     */\n    protected abstract List<ErlangValue> changeList(List<ErlangValue> pageList);\n\n    /**\n     * Verifies the write operations.\n     *\n     * @param firstOp   the first operation to process inside the result list\n     * @param results   the result list\n     *\n     * @return number of processed operations (<tt>1</tt> or <tt>2</tt>)\n     */\n    protected int checkWrite(final int firstOp, final ResultList results)\n            throws OtpErlangException, UnknownException {\n        int checkedOps = 0;\n\n        if (listChanged) {\n            assert results != null;\n            results.processWriteAt(firstOp + checkedOps);\n            ++checkedOps;\n            if ((countKey != null) && listCountChanged) {\n                results.processWriteAt(firstOp + checkedOps);\n                ++checkedOps;\n            }\n        }\n        return checkedOps;\n    }\n\n    /**\n     * Converts a list of <tt>T</tt> to a list of {@link ErlangValue} objects.\n     *\n     * @param list\n     *            the list to convert\n     *\n     * @throws ClassCastException\n     *             if the conversion fails\n     */\n    protected static <T> List<ErlangValue> toErlangValueList(final List<T> list)\n            throws ClassCastException {\n        final List<ErlangValue> result = new ArrayList<ErlangValue>(list.size());\n        for (final T t : list) {\n            result.add(new ErlangValue(t));\n        }\n        return result;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/executor/ScalarisChangeListOp2.java",
    "content": "package de.zib.scalaris.executor;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.ResultList;\nimport de.zib.scalaris.UnknownException;\nimport de.zib.scalaris.operations.AddOnNrOp;\n\n/**\n * Implements a list change operation using the append operation of Scalaris.\n * Supports an (optional) list counter key which is updated accordingly.\n *\n * Sub-classes need to override {@link #changeList(RequestList)} to perform the\n * changes and issue a <em>single</em> {@link AddOnNrOp} operation and\n * (optionally) a second {@link AddOnNrOp} for a list counter key!\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.18\n */\npublic abstract class ScalarisChangeListOp2 implements ScalarisOp {\n    /**\n     * Key used to store the list.\n     */\n    protected final String key;\n    /**\n     * Key used to store the list counter.\n     */\n    protected final String countKey;\n\n    /**\n     * Creates a new list change operation.\n     *\n     * @param key       the key to change the list at\n     * @param countKey  the key for the counter of the entries in the list\n     *                  (may be <tt>null</tt>)\n     */\n    public ScalarisChangeListOp2(final String key, final String countKey) {\n        this.key = key;\n        this.countKey = countKey;\n    }\n\n    public int workPhases() {\n        return 1;\n    }\n\n    public final int doPhase(final int phase, final int firstOp,\n            final ResultList results, final RequestList requests)\n            throws OtpErlangException, UnknownException,\n            IllegalArgumentException {\n        switch (phase) {\n            case 0: return changeList(requests);\n            case 1: return checkChange(firstOp, results);\n            default:\n                throw new IllegalArgumentException(\"No phase \" + phase);\n        }\n    }\n\n    /**\n     * Changes the given page list and its counter (if present).\n     *\n     * Sub-classes overriding this method need to perform the changes and issue\n     * a <em>single</em> {@link AddOnNrOp} operation and (optionally) a second\n     * {@link AddOnNrOp} for a list counter key!\n     *\n     * @param requests\n     *            the request list\n     *\n     * @return number of processed operations (should be <tt>0</tt>)\n     */\n    protected abstract int changeList(final RequestList requests);\n\n    /**\n     * Verifies the list change operation.\n     *\n     * @param firstOp   the first operation to process inside the result list\n     * @param results   the result list\n     *\n     * @return number of processed operations (<tt>1</tt> or <tt>2</tt>)\n     */\n    protected int checkChange(final int firstOp, final ResultList results)\n            throws OtpErlangException, UnknownException {\n        assert results != null;\n        int checkedOps = 0;\n        results.processAddDelOnListAt(firstOp + checkedOps);\n        ++checkedOps;\n        if (countKey != null) {\n            results.processAddOnNrAt(firstOp + checkedOps);\n            ++checkedOps;\n        }\n        return checkedOps;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/executor/ScalarisIncrementOp1.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.executor;\n\nimport java.math.BigInteger;\nimport java.security.InvalidParameterException;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.ResultList;\nimport de.zib.scalaris.UnknownException;\nimport de.zib.scalaris.operations.ReadOp;\nimport de.zib.scalaris.operations.WriteOp;\n\n/**\n * Implements an increment operation using the read and write operations of\n * Scalaris.\n *\n * @param <T> the type of the value to write\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.13\n * @since 3.13\n */\npublic class ScalarisIncrementOp1<T extends Number> implements ScalarisOp {\n    final protected String key;\n    final protected BigInteger value;\n\n    /**\n     * Creates a write operation.\n     *\n     * @param key\n     *            the key to write to\n     * @param value\n     *            the value to write\n     */\n    public ScalarisIncrementOp1(final String key, final T value) {\n        this.key = key;\n        if (value instanceof Integer) {\n            this.value = BigInteger.valueOf((Integer) value);\n        } else if (value instanceof Long) {\n            this.value = BigInteger.valueOf((Long) value);\n        } else if (value instanceof BigInteger) {\n            this.value = (BigInteger) value;\n        } else {\n            throw new InvalidParameterException(\"Type not supported for increment: \" + value.getClass().toString());\n        }\n    }\n\n    public int workPhases() {\n        return 2;\n    }\n\n    public final int doPhase(final int phase, final int firstOp, final ResultList results,\n            final RequestList requests) throws OtpErlangException, UnknownException,\n            IllegalArgumentException {\n        switch (phase) {\n        case 0: return prepareRead(requests);\n        case 1: return prepareWrite(firstOp, results, requests);\n        case 2: return checkWrite(firstOp, results);\n        default:\n            throw new IllegalArgumentException(\"No phase \" + phase);\n        }\n    }\n\n    /**\n     * Adds a read operation for the current value to the request list.\n     *\n     * @param requests the request list\n     *\n     * @return <tt>0</tt> (no operation processed since no results are used)\n     */\n    protected int prepareRead(final RequestList requests) {\n        requests.addOp(new ReadOp(key));\n        return 0;\n    }\n\n    /**\n     * Verifies the read operation, increments the number and adds a write\n     * operation to the request list.\n     *\n     * @param firstOp\n     *            the first operation to process inside the result list\n     * @param results\n     *            the result list\n     * @param requests\n     *            the request list\n     *\n     * @return <tt>1</tt> operation processed (the read)\n     */\n    protected int prepareWrite(final int firstOp, final ResultList results,\n            final RequestList requests) throws OtpErlangException,\n            UnknownException {\n        assert results != null;\n        BigInteger newValue;\n        try {\n            newValue = results.processReadAt(firstOp).bigIntValue().add(value);\n        } catch (final NotFoundException e) {\n            // this is ok\n            newValue = value;\n        }\n        requests.addOp(new WriteOp(key, newValue));\n        return 1;\n    }\n\n    /**\n     * Verifies the write operation.\n     *\n     * @param firstOp   the first operation to process inside the result list\n     * @param results   the result list\n     *\n     * @return <tt>1</tt> operation processed (the write)\n     */\n    protected int checkWrite(final int firstOp, final ResultList results)\n            throws OtpErlangException, UnknownException {\n        assert results != null;\n        results.processWriteAt(firstOp);\n        return 1;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.ScalarisOp#toString()\n     */\n    @Override\n    public String toString() {\n        return \"Scalaris.increment(\" + key + \", \" + value + \")\";\n    }\n\n    /**\n     * Gets the key to write to.\n     *\n     * @return the key\n     */\n    public String getKey() {\n        return key;\n    }\n\n    /**\n     * Gets the value to increment by.\n     *\n     * @return the value\n     */\n    public BigInteger getIncValue() {\n        return value;\n    }\n\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/executor/ScalarisIncrementOp2.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.executor;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.ResultList;\nimport de.zib.scalaris.UnknownException;\nimport de.zib.scalaris.operations.AddOnNrOp;\n\n/**\n * Implements an increment operation using the increment operation of\n * Scalaris.\n *\n * @param <T> the type of the value to write\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.13\n * @since 3.13\n */\npublic class ScalarisIncrementOp2<T extends Number> implements ScalarisOp {\n    final protected String key;\n    final protected T value;\n\n    /**\n     * Creates a write operation.\n     *\n     * @param key\n     *            the key to write to\n     * @param value\n     *            the value to write\n     */\n    public ScalarisIncrementOp2(final String key, final T value) {\n        this.key = key;\n        this.value = value;\n    }\n\n    public int workPhases() {\n        return 1;\n    }\n\n    public final int doPhase(final int phase, final int firstOp, final ResultList results,\n            final RequestList requests) throws OtpErlangException, UnknownException,\n            IllegalArgumentException {\n        switch (phase) {\n        case 0: return prepareIncrement(requests);\n        case 1: return checkIncrement(firstOp, results);\n        default:\n            throw new IllegalArgumentException(\"No phase \" + phase);\n        }\n    }\n\n    /**\n     * Adds the increment operation to the request list.\n     *\n     * @param requests  the request list\n     *\n     * @return <tt>0</tt> (no operation processed)\n     */\n    protected int prepareIncrement(final RequestList requests) throws OtpErlangException,\n            UnknownException {\n        requests.addOp(new AddOnNrOp(key, value));\n        return 0;\n    }\n\n    /**\n     * Verifies the increment operation.\n     *\n     * @param firstOp   the first operation to process inside the result list\n     * @param results   the result list\n     *\n     * @return <tt>1</tt> operation processed (the write)\n     */\n    protected int checkIncrement(final int firstOp, final ResultList results)\n            throws OtpErlangException, UnknownException {\n        assert results != null;\n        results.processAddOnNrAt(firstOp);\n        return 1;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.ScalarisOp#toString()\n     */\n    @Override\n    public String toString() {\n        return \"Scalaris.increment(\" + key + \", \" + value + \")\";\n    }\n\n    /**\n     * Gets the key to write to.\n     *\n     * @return the key\n     */\n    public String getKey() {\n        return key;\n    }\n\n    /**\n     * Gets the value to increment by.\n     *\n     * @return the value\n     */\n    public T getIncValue() {\n        return value;\n    }\n\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/executor/ScalarisListAppendRemoveOp1.java",
    "content": "package de.zib.scalaris.executor;\n\nimport java.util.List;\n\nimport de.zib.scalaris.ErlangValue;\n\n/**\n * Implements a list append and remove operation using the read and write\n * operations of Scalaris. Supports an (optional) list counter key which is\n * updated accordingly.\n *\n * First adds a set of elements, then removes another set of elements.\n *\n * @param <T>\n *            the type of objects in the list\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.18\n */\npublic class ScalarisListAppendRemoveOp1<T> extends ScalarisChangeListOp1 {\n    /**\n     * Elements to add to the stored list.\n     */\n    final protected List<ErlangValue> toAdd;\n    /**\n     * Elements to remove from the stored list (after adding {@link #toAdd}).\n     */\n    final protected List<ErlangValue> toRemove;\n\n    /**\n     * Creates a new append+remove operation.\n     *\n     * @param key       the key to append/remove the values to/from\n     * @param toAdd     the values to add\n     * @param toRemove  the values to remove\n     * @param countKey  the key for the counter of the entries in the list\n     *                  (may be <tt>null</tt>)\n     */\n    public ScalarisListAppendRemoveOp1(final String key, final List<T> toAdd,\n            final List<T> toRemove, final String countKey) {\n        super(key, countKey);\n        this.toAdd = toErlangValueList(toAdd);\n        this.toRemove = toErlangValueList(toRemove);\n    }\n\n    /**\n     * Adds {@link #toAdd} to the given page list.\n     *\n     * @param pageList\n     *            the original page list (will be changed in-place)\n     *\n     * @return the changed list (a reference to the changed <tt>pageList</tt>)\n     */\n    @Override\n    protected List<ErlangValue> changeList(final List<ErlangValue> pageList) {\n        final int oldCount = pageList.size();\n        int count = oldCount;\n        pageList.addAll(toAdd);\n        if (pageList.size() != count) {\n            count = pageList.size();\n            listChanged = true;\n        }\n        pageList.removeAll(toRemove);\n        if (pageList.size() != count) {\n            count = pageList.size();\n            listChanged = true;\n        }\n        if (count != oldCount) {\n            listCountChanged = true;\n        }\n        return pageList;\n    }\n\n    @Override\n    public String toString() {\n        return \"Scalaris.append_remove(\" + key + \", \" + toAdd.toString() + \", \"\n                + toRemove.toString() + \", \" + countKey + \")\";\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/executor/ScalarisListAppendRemoveOp2.java",
    "content": "package de.zib.scalaris.executor;\n\nimport java.util.List;\n\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.operations.AddDelOnListOp;\nimport de.zib.scalaris.operations.AddOnNrOp;\n\n/**\n * Implements a list append operation using the append operation of Scalaris.\n * Supports an (optional) list counter key which is updated accordingly.\n *\n * For a correct counter in the <tt>countKey</tt>, this class assumes that every\n * element from the <tt>toRemove</tt> list existed in the list (at least after\n * adding the elements from <tt>toAdd</tt>).\n *\n * @param <T>\n *            the type of objects in the list\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.18\n */\npublic class ScalarisListAppendRemoveOp2<T> extends ScalarisChangeListOp2 {\n    /**\n     * Elements to add to the stored list.\n     */\n    final protected List<T> toAdd;\n    /**\n     * Elements to remove from the stored list (after adding {@link #toAdd}).\n     */\n    final protected List<T> toRemove;\n\n    /**\n     * Creates a new append+remove operation.\n     *\n     * @param key       the key to append/remove the values to/from\n     * @param toAdd     the values to add\n     * @param toRemove  the values to remove\n     * @param countKey  the key for the counter of the entries in the list\n     *                  (may be <tt>null</tt>)\n     */\n    public ScalarisListAppendRemoveOp2(final String key, final List<T> toAdd,\n            final List<T> toRemove, final String countKey) {\n        super(key, countKey);\n        this.toAdd = toAdd;\n        this.toRemove = toRemove;\n    }\n\n    /**\n     * Adds {@link #toAdd} to the list at {@link ScalarisListOp2#key},\n     * removes {@link #toRemove} and updates the list counter (if present).\n     *\n     * @param requests\n     *            the request list\n     *\n     * @return <tt>0</tt> (number of processed operations)\n     */\n    @Override\n    protected int changeList(final RequestList requests) {\n        requests.addOp(new AddDelOnListOp(key, toAdd, toRemove));\n        if (countKey != null) {\n            requests.addOp(new AddOnNrOp(countKey, toAdd.size() - toRemove.size()));\n        }\n        return 0;\n    }\n\n    @Override\n    public String toString() {\n        return \"Scalaris.append_remove(\" + key + \", \" + toAdd.toString() + \", \"\n                + toRemove.toString() + \", \" + countKey + \")\";\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/executor/ScalarisOp.java",
    "content": "package de.zib.scalaris.executor;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.ResultList;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Interface for arbitrary Scalaris operations.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.13\n * @since 3.13\n */\npublic interface ScalarisOp {\n\n    /**\n     * Gets the number of work phases needed by this operation (not including\n     * the final result verification phase).\n     *\n     * @return number of required phases\n     */\n    public abstract int workPhases();\n\n    /**\n     * Executes the given phase.\n     *\n     * @param phase\n     *            the number of the current phase\n     * @param firstOp\n     *            the current operation's index in the result list\n     * @param results\n     *            the results from the previous operations\n     *            (may be <tt>null</tt> if there was none)\n     * @param requests\n     *            the requests for the next operations\n     *            (may be <tt>null</tt> if there are none, i.e. in the\n     *            verification phase)\n     *\n     * @return the number of processed operations from the results list\n     *\n     * @throws OtpErlangException\n     *             if an error occured verifying a result from previous\n     *             operations\n     * @throws UnknownException\n     *             if an error occured verifying a result from previous\n     *             operations\n     * @throws IllegalArgumentException\n     *             if the given work phase is not supported\n     *\n     * @see #workPhases()\n     */\n    public abstract int doPhase(int phase, int firstOp, ResultList results,\n            RequestList requests) throws OtpErlangException, UnknownException,\n            IllegalArgumentException;\n\n    /* (non-Javadoc)\n     * @see java.lang.Object#toString()\n     */\n    public abstract String toString();\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/executor/ScalarisOpExecutor.java",
    "content": "package de.zib.scalaris.executor;\n\nimport java.util.ArrayList;\nimport com.ericsson.otp.erlang.OtpErlangException;\n\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.ResultList;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Executes multiple {@link ScalarisOp} operations in multiple phases only\n * sending requests to Scalaris once per work phase.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.13\n * @since 3.13\n */\npublic abstract class ScalarisOpExecutor {\n\n    /**\n     * All operations to be executed.\n     */\n    protected final ArrayList<ScalarisOp> ops = new ArrayList<ScalarisOp>();\n    /**\n     * The highest work phase id.\n     */\n    protected int workPhases = 0;\n\n\n    /**\n     * Adds the given operation to be executed.\n     *\n     * @param op\n     *            the operation to add\n     */\n    public void addOp(final ScalarisOp op) {\n        if (op.workPhases() > workPhases) {\n            workPhases = op.workPhases();\n        }\n        ops.add(op);\n    }\n\n    /**\n     * Re-sets the executor as if created from scratch.\n     */\n    public void reset() {\n        ops.clear();\n        workPhases = 0;\n    }\n\n    /**\n     * Executes all operations previously added with {@link #addOp(ScalarisOp)}.\n     *\n     * @throws OtpErlangException\n     *             if an error occurred verifying a result from previous\n     *             operations\n     * @throws UnknownException\n     *             if an error occurred verifying a result from previous\n     *             operations\n     */\n    public void run() throws OtpErlangException, UnknownException {\n        ResultList results = null;\n        for (int phase = 0; phase <= workPhases; ++phase) {\n            final RequestList requests = newRequestList();\n            int curOp = 0;\n            for (final ScalarisOp op : ops) {\n                // translate the global phase into an operation-specific phase,\n                // execute operations as late as possible\n                final int opPhase = phase - (workPhases - op.workPhases());\n                if ((opPhase >= 0) && (opPhase <= op.workPhases())) {\n                    curOp += op.doPhase(opPhase, curOp, results, requests);\n                }\n            }\n            endWorkPhase(phase, requests);\n            if (phase != workPhases) {\n                results = executeRequests(requests);\n            }\n        }\n    }\n\n    /**\n     * This method is called at the end of each work phase and allows\n     * implementing sub-classes to add additional operations.\n     *\n     * @param phase\n     *            the current work phase\n     * @param requests\n     *            the requests\n     */\n    protected void endWorkPhase(final int phase, final RequestList requests) {\n    }\n\n    /**\n     * Creates a new request list.\n     *\n     * @return an empty request list\n     */\n    protected abstract RequestList newRequestList();\n\n    /**\n     * Executes the given requests.\n     *\n     * @param requests\n     *            a request list to execute\n     *\n     * @return the results from executing the requests\n     *\n     * @throws OtpErlangException\n     *             if an error occurred verifying a result from previous\n     *             operations\n     * @throws UnknownException\n     *             if an error occurred verifying a result from previous\n     *             operations\n     */\n    protected abstract ResultList executeRequests(RequestList requests)\n            throws OtpErlangException, UnknownException;\n\n    /**\n     * @return the workPhases\n     */\n    public int getWorkPhases() {\n        return workPhases;\n    }\n\n    /**\n     * Gets the current list of operations. This is backed by the operations in\n     * this executor - if it is reset, the list will be empty.\n     *\n     * Create a copy of this list if it should be retained.\n     *\n     * @return the ops the current list of operations\n     */\n    public ArrayList<ScalarisOp> getOps() {\n        return ops;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/executor/ScalarisReadOp.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.executor;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.ResultList;\nimport de.zib.scalaris.UnknownException;\nimport de.zib.scalaris.operations.ReadOp;\n\n/**\n * Implements a read operation (tolerates \"not found\" and in this case contains\n * <tt>null</tt>).\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.13\n * @since 3.13\n */\npublic class ScalarisReadOp implements ScalarisOp {\n    final protected String key;\n    protected ErlangValue value = null;\n\n    /**\n     * Creates a read operation.\n     *\n     * @param key\n     *            the key to read from\n     */\n    public ScalarisReadOp(final String key) {\n        this.key = key;\n    }\n\n    public int workPhases() {\n        return 1;\n    }\n\n    public final int doPhase(final int phase, final int firstOp, final ResultList results,\n            final RequestList requests) throws OtpErlangException, UnknownException,\n            IllegalArgumentException {\n        switch (phase) {\n        case 0: return prepareRead(requests);\n        case 1: return checkRead(firstOp, results);\n        default:\n            throw new IllegalArgumentException(\"No phase \" + phase);\n        }\n    }\n\n    /**\n     * Adds the read operation to the request list.\n     *\n     * @param requests  the request list\n     *\n     * @return <tt>0</tt> (no operation processed)\n     */\n    protected int prepareRead(final RequestList requests) throws OtpErlangException,\n            UnknownException {\n        requests.addOp(new ReadOp(key));\n        return 0;\n    }\n\n    /**\n     * Verifies the read operation and stores the value.\n     *\n     * @param firstOp   the first operation to process inside the result list\n     * @param results   the result list\n     *\n     * @return <tt>1</tt> operation processed (the read)\n     */\n    protected int checkRead(final int firstOp, final ResultList results)\n            throws OtpErlangException, UnknownException {\n        try {\n            value = results.processReadAt(firstOp);\n        } catch (final NotFoundException e) {\n        }\n        return 1;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.ScalarisOp#toString()\n     */\n    @Override\n    public String toString() {\n        return \"Scalaris.read(\" + key + \", \" + value + \")\";\n    }\n\n    /**\n     * Gets the key to read from.\n     *\n     * @return the key\n     */\n    public String getKey() {\n        return key;\n    }\n\n    /**\n     * Gets the value from the read.\n     *\n     * @return the value that has been read (may be <tt>null</tt>\n     */\n    public ErlangValue getValue() {\n        return value;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/executor/ScalarisSingleOpExecutor.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.executor;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.TransactionSingleOp;\nimport de.zib.scalaris.TransactionSingleOp.ResultList;\nimport de.zib.scalaris.UnknownException;\n\n\n/**\n * Executes multiple {@link ScalarisOp} operations in multiple phases only\n * sending requests to Scalaris once per work phase. Uses\n * {@link TransactionSingleOp}.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.13\n * @since 3.13\n */\npublic class ScalarisSingleOpExecutor extends ScalarisOpExecutor {\n    protected final TransactionSingleOp scalaris_single;\n\n    /**\n     * Creates a new executor.\n     *\n     * @param scalaris_single\n     *            the Scalaris connection to use\n     */\n    public ScalarisSingleOpExecutor(final TransactionSingleOp scalaris_single) {\n        this.scalaris_single = scalaris_single;\n        reset();\n    }\n\n    @Override\n    protected ResultList executeRequests(final RequestList requests)\n            throws ConnectionException, AbortException, UnknownException {\n        return scalaris_single.req_list((TransactionSingleOp.RequestList) requests);\n    }\n\n    @Override\n    protected RequestList newRequestList() {\n        return new TransactionSingleOp.RequestList();\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/executor/ScalarisTxOpExecutor.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.executor;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.Transaction;\nimport de.zib.scalaris.Transaction.ResultList;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Executes multiple {@link ScalarisOp} operations in multiple phases only\n * sending requests to Scalaris once per work phase. Uses {@link Transaction}.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.13\n * @since 3.13\n */\npublic class ScalarisTxOpExecutor extends ScalarisOpExecutor {\n    protected final Transaction scalaris_tx;\n    /**\n     * Whether to commit the requests in the last work phase.\n     */\n    protected boolean commitLast;\n\n    /**\n     * Creates a new executor.\n     *\n     * @param scalaris_tx\n     *            the Scalaris connection to use\n     */\n    public ScalarisTxOpExecutor(final Transaction scalaris_tx) {\n        this.scalaris_tx = scalaris_tx;\n        reset();\n    }\n\n    /**\n     * Adds a commit to the requests in the last work phase.\n     *\n     * @param phase\n     *            the current work phase\n     * @param requests\n     *            the requests\n     */\n    @Override\n    protected void endWorkPhase(final int phase, final RequestList requests) {\n        if ((phase == (workPhases - 1)) && commitLast) {\n            requests.addCommit();\n        }\n    }\n\n    @Override\n    protected ResultList executeRequests(final RequestList requests)\n            throws ConnectionException, AbortException, UnknownException {\n        return scalaris_tx.req_list((Transaction.RequestList) requests);\n    }\n\n    @Override\n    protected RequestList newRequestList() {\n        return new Transaction.RequestList();\n    }\n\n    /**\n     * @param commitLast the commitLast to set\n     */\n    public void setCommitLast(final boolean commitLast) {\n        this.commitLast = commitLast;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.executor.ScalarisOpExecutor#reset()\n     */\n    @Override\n    public void reset() {\n        super.reset();\n        commitLast = false;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/executor/ScalarisWriteOp.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.executor;\n\nimport com.ericsson.otp.erlang.OtpErlangException;\n\nimport de.zib.scalaris.RequestList;\nimport de.zib.scalaris.ResultList;\nimport de.zib.scalaris.UnknownException;\nimport de.zib.scalaris.operations.WriteOp;\n\n/**\n * Implements a write operation.\n *\n * @param <T> the type of the value to write\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.13\n * @since 3.13\n */\npublic class ScalarisWriteOp<T> implements ScalarisOp {\n    final protected String key;\n    final protected T value;\n\n    /**\n     * Creates a write operation.\n     *\n     * @param key\n     *            the key to write to\n     * @param value\n     *            the value to write\n     */\n    public ScalarisWriteOp(final String key, final T value) {\n        this.key = key;\n        this.value = value;\n    }\n\n    public int workPhases() {\n        return 1;\n    }\n\n    public final int doPhase(final int phase, final int firstOp, final ResultList results,\n            final RequestList requests) throws OtpErlangException, UnknownException,\n            IllegalArgumentException {\n        switch (phase) {\n        case 0: return prepareWrite(requests);\n        case 1: return checkWrite(firstOp, results);\n        default:\n            throw new IllegalArgumentException(\"No phase \" + phase);\n        }\n    }\n\n    /**\n     * Adds the write operation to the request list.\n     *\n     * @param requests  the request list\n     *\n     * @return <tt>0</tt> (no operation processed)\n     */\n    protected int prepareWrite(final RequestList requests) throws OtpErlangException,\n            UnknownException {\n        requests.addOp(new WriteOp(key, value));\n        return 0;\n    }\n\n    /**\n     * Verifies the write operations.\n     *\n     * @param firstOp   the first operation to process inside the result list\n     * @param results   the result list\n     *\n     * @return <tt>1</tt> operation processed (the write)\n     */\n    protected int checkWrite(final int firstOp, final ResultList results)\n            throws OtpErlangException, UnknownException {\n        results.processWriteAt(firstOp);\n        return 1;\n    }\n\n    /* (non-Javadoc)\n     * @see de.zib.scalaris.examples.wikipedia.ScalarisOp#toString()\n     */\n    @Override\n    public String toString() {\n        return \"Scalaris.write(\" + key + \", \" + value + \")\";\n    }\n\n    /**\n     * Gets the key to write to.\n     *\n     * @return the key\n     */\n    public String getKey() {\n        return key;\n    }\n\n    /**\n     * Gets the value to write.\n     *\n     * @return the value that will (or has been) be written\n     */\n    public T getValue() {\n        return value;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/jmx/MonitorNode.java",
    "content": "package de.zib.scalaris.jmx;\n\nimport java.util.Map;\n\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.Monitor;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Provides methods to monitor a specific Scalaris (Erlang) VM via JMX.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.15\n * @since 3.15\n */\npublic class MonitorNode implements MonitorNodeMBean {\n    protected final de.zib.scalaris.Monitor monitor;\n\n    /**\n     * Creates a connection to the erlang VM of the given Scalaris node. Uses\n     * the connection policy of the global connection factory.\n     *\n     * @param node\n     *            Scalaris node to connect with\n     * @throws ConnectionException\n     *             if the connection fails or the connection policy is not\n     *             cloneable\n     */\n    public MonitorNode(final String node) throws ConnectionException {\n        this.monitor = new de.zib.scalaris.Monitor(node);\n    }\n\n    /* (non-Javadoc)\n     * @see jmx.MonitorNodeMBean#getScalarisVersion()\n     */\n    public String getScalarisVersion() throws ConnectionException, UnknownException {\n        return monitor.getNodeInfo().scalarisVersion;\n    }\n    /* (non-Javadoc)\n     * @see jmx.MonitorNodeMBean#getErlangVersion()\n     */\n    public String getErlangVersion() throws ConnectionException, UnknownException {\n        return monitor.getNodeInfo().erlangVersion;\n    }\n    /* (non-Javadoc)\n     * @see jmx.MonitorNodeMBean#getDhtNodes()\n     */\n    public int getDhtNodes() throws ConnectionException, UnknownException {\n        return monitor.getNodeInfo().dhtNodes;\n    }\n    /* (non-Javadoc)\n     * @see jmx.MonitorNodeMBean#getLatencyAvg()\n     */\n    public Map<Long, Double> getLatencyAvg() throws ConnectionException, UnknownException {\n        return monitor.getNodePerformance().latencyAvg;\n    }\n    /* (non-Javadoc)\n     * @see jmx.MonitorNodeMBean#getLatencyStddev()\n     */\n    public Map<Long, Double> getLatencyStddev() throws ConnectionException, UnknownException {\n        return monitor.getNodePerformance().latencyStddev;\n    }\n    /* (non-Javadoc)\n     * @see jmx.MonitorNodeMBean#getCurLatencyAvg()\n     */\n    public Double getCurLatencyAvg() throws ConnectionException, UnknownException {\n        return Monitor.getCurrentPerfValue(monitor.getNodePerformance().latencyAvg);\n    }\n    /* (non-Javadoc)\n     * @see jmx.MonitorNodeMBean#getCurLatencyStddev()\n     */\n    public Double getCurLatencyStddev() throws ConnectionException, UnknownException {\n        return Monitor.getCurrentPerfValue(monitor.getNodePerformance().latencyStddev);\n    }\n\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/jmx/MonitorNodeMBean.java",
    "content": "package de.zib.scalaris.jmx;\n\nimport java.util.Map;\n\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Provides methods to monitor a specific Scalaris (Erlang) VM via JMX.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.15\n * @since 3.15\n */\npublic interface MonitorNodeMBean {\n\n    /**\n     * Gets the version of Scalaris running in the VM the monitor is connected\n     * to.\n     *\n     * @return version string\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public abstract String getScalarisVersion() throws ConnectionException, UnknownException;\n\n    /**\n     * Gets the version of Erlang running in the VM the monitor is connected\n     * to.\n     *\n     * @return version string\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public abstract String getErlangVersion() throws ConnectionException, UnknownException;\n\n    /**\n     * Gets the number of Scalaris nodes running in the VM the monitor is\n     * connected to.\n     *\n     * @return number of DHT nodes\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public abstract int getDhtNodes() throws ConnectionException, UnknownException;\n\n    /**\n     * Gets average latency values collected in the VM the monitor is connected\n     * to.\n     *\n     * @return map of timestamps to average latencies\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public abstract Map<Long, Double> getLatencyAvg() throws ConnectionException, UnknownException;\n\n    /**\n     * Gets the standard deviation of the latency values collected in the VM the\n     * monitor is connected to.\n     *\n     * @return map of timestamps to latency (standard) deviation\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public abstract Map<Long, Double> getLatencyStddev() throws ConnectionException, UnknownException;\n\n    /**\n     * Gets the current, i.e. latest, average latency collected in the VM the\n     * monitor is connected to.\n     *\n     * @return latest average latency (or <tt>null</tt> if there is none)\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public abstract Double getCurLatencyAvg() throws ConnectionException, UnknownException;\n\n    /**\n     * Gets the current, i.e. latest, standard deviation of the latency\n     * collected in the VM the monitor is connected to.\n     *\n     * @return latest latency (standard) deviation (or <tt>null</tt> if there is\n     *         none)\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public abstract Double getCurLatencyStddev() throws ConnectionException, UnknownException;\n\n}"
  },
  {
    "path": "java-api/src/de/zib/scalaris/jmx/MonitorService.java",
    "content": "package de.zib.scalaris.jmx;\n\nimport java.util.Map;\n\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.Monitor;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Provides methods to monitor a specific Scalaris (Erlang) VM via JMX.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.15\n * @since 3.15\n */\npublic class MonitorService implements MonitorServiceMBean {\n    protected final de.zib.scalaris.Monitor monitor;\n\n    /**\n     * Creates a connection to the erlang VM of the given Scalaris node. Uses\n     * the connection policy of the global connection factory.\n     *\n     * @param node\n     *            Scalaris node to connect with\n     * @throws ConnectionException\n     *             if the connection fails or the connection policy is not\n     *             cloneable\n     */\n    public MonitorService(final String node) throws ConnectionException {\n        this.monitor = new de.zib.scalaris.Monitor(node);\n    }\n    /* (non-Javadoc)\n     * @see jmx.MonitorServiceMBean#getTotalLoad()\n     */\n    public Long getTotalLoad() throws ConnectionException, UnknownException {\n        return monitor.getServiceInfo().totalLoad;\n    }\n    /* (non-Javadoc)\n     * @see jmx.MonitorServiceMBean#getNodes()\n     */\n    public Long getNodes() throws ConnectionException, UnknownException {\n        return monitor.getServiceInfo().nodes;\n    }\n    /* (non-Javadoc)\n     * @see jmx.MonitorServiceMBean#getLatencyAvg()\n     */\n    public Map<Long, Double> getLatencyAvg() throws ConnectionException, UnknownException {\n        return monitor.getServicePerformance().latencyAvg;\n    }\n    /* (non-Javadoc)\n     * @see jmx.MonitorServiceMBean#getLatencyStddev()\n     */\n    public Map<Long, Double> getLatencyStddev() throws ConnectionException, UnknownException {\n        return monitor.getServicePerformance().latencyStddev;\n    }\n    /* (non-Javadoc)\n     * @see jmx.MonitorServiceMBean#getCurLatencyAvg()\n     */\n    public Double getCurLatencyAvg() throws ConnectionException, UnknownException {\n        return Monitor.getCurrentPerfValue(monitor.getServicePerformance().latencyAvg);\n    }\n    /* (non-Javadoc)\n     * @see jmx.MonitorServiceMBean#getCurLatencyStddev()\n     */\n    public Double getCurLatencyStddev() throws ConnectionException, UnknownException {\n        return Monitor.getCurrentPerfValue(monitor.getServicePerformance().latencyStddev);\n    }\n\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/jmx/MonitorServiceMBean.java",
    "content": "package de.zib.scalaris.jmx;\n\nimport java.util.Map;\n\nimport de.zib.scalaris.ConnectionException;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Provides methods to monitor a whole Scalaris ring via JMX.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.15\n * @since 3.15\n */\npublic interface MonitorServiceMBean {\n    /**\n     * Gets the total load of the whole Scalaris ring from the VM the monitor is\n     * connected to.\n     *\n     * @return number of DHT nodes\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public abstract Long getTotalLoad() throws ConnectionException, UnknownException;\n\n    /**\n     * Gets the number of Scalaris nodes of the whole Scalaris ring from the VM\n     * the monitor is connected to.\n     *\n     * @return number of DHT nodes\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public abstract Long getNodes() throws ConnectionException, UnknownException;\n\n    /**\n     * Gets average latency values of the whole Scalaris ring from the VM the\n     * monitor is connected to.\n     *\n     * @return map of timestamps to average latencies\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public abstract Map<Long, Double> getLatencyAvg() throws ConnectionException, UnknownException;\n\n    /**\n     * Gets the standard deviation of the latency values of the whole Scalaris\n     * ring from the VM the monitor is connected to.\n     *\n     * @return map of timestamps to latency (standard) deviation\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public abstract Map<Long, Double> getLatencyStddev() throws ConnectionException, UnknownException;\n\n    /**\n     * Gets the current, i.e. latest, average latency of the whole Scalaris ring\n     * from the VM the monitor is connected to.\n     *\n     * @return latest average latency (or <tt>null</tt> if there is none)\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public abstract Double getCurLatencyAvg() throws ConnectionException, UnknownException;\n\n    /**\n     * Gets the current, i.e. latest, standard deviation of the latency of the\n     * whole Scalaris ring from the VM the monitor is connected to.\n     *\n     * @return latest latency (standard) deviation (or <tt>null</tt> if there is\n     *         none)\n     *\n     * @throws ConnectionException\n     *             if the connection is not active or a communication error\n     *             occurs or an exit signal was received or the remote node\n     *             sends a message containing an invalid cookie\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public abstract Double getCurLatencyStddev() throws ConnectionException, UnknownException;\n\n}"
  },
  {
    "path": "java-api/src/de/zib/scalaris/operations/AddDelOnListOp.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.operations;\n\nimport java.util.List;\n\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.CommonErlangObjects;\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.NotAListException;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Operation appending to / removing from a list.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.14\n * @since 3.14\n */\npublic class AddDelOnListOp implements TransactionOperation, TransactionSingleOpOperation {\n    final protected OtpErlangString key;\n    final protected OtpErlangObject toAdd;\n    final protected OtpErlangObject toRemove;\n    protected OtpErlangObject resultRaw = null;\n    protected boolean resultCompressed = false;\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            a list of values to add to a list\n     * @param toRemove\n     *            a list of values to remove from a list\n     */\n    public AddDelOnListOp(final OtpErlangString key, final OtpErlangList toAdd, final OtpErlangList toRemove) {\n        this.key = key;\n        this.toAdd = toAdd;\n        this.toRemove = toRemove;\n    }\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            a list of values to add to a list\n     * @param toRemove\n     *            a list of values to remove from a list\n     */\n    public <T> AddDelOnListOp(final String key, final List<T> toAdd, final List<T> toRemove) {\n        this.key = new OtpErlangString(key);\n        this.toAdd = (toAdd == null) ? new OtpErlangList() : (OtpErlangList) ErlangValue.convertToErlang(toAdd);\n        this.toRemove = (toRemove == null) ? new OtpErlangList() : (OtpErlangList) ErlangValue.convertToErlang(toRemove);\n    }\n\n    public OtpErlangObject getErlang(final boolean compressed) {\n        return new OtpErlangTuple(new OtpErlangObject[] {\n                CommonErlangObjects.addDelOnListAtom, key,\n                compressed ? CommonErlangObjects.encode(toAdd) : toAdd,\n                compressed ? CommonErlangObjects.encode(toRemove) : toRemove });\n    }\n\n    public OtpErlangString getKey() {\n        return key;\n    }\n\n    public void setResult(final OtpErlangObject resultRaw, final boolean compressed) {\n        this.resultRaw = resultRaw;\n        this.resultCompressed = compressed;\n    }\n\n    public OtpErlangObject getResult() {\n        return this.resultRaw;\n    }\n\n    public boolean getResultCompressed() {\n        return this.resultCompressed;\n    }\n\n    public Object processResult() throws UnknownException,\n            NotAListException {\n        /*\n         * possible return values:\n         *  {ok} | {fail, not_a_list}.\n         */\n        try {\n            final OtpErlangTuple received = (OtpErlangTuple) resultRaw;\n            if (received.equals(CommonErlangObjects.okTupleAtom)) {\n                return null;\n            } else if (received.elementAt(0).equals(CommonErlangObjects.failAtom) && (received.arity() == 2)) {\n                final OtpErlangObject reason = received.elementAt(1);\n                if (reason.equals(CommonErlangObjects.notAListAtom)) {\n                    throw new NotAListException(resultRaw);\n                }\n            }\n            throw new UnknownException(resultRaw);\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, resultRaw);\n        }\n    }\n\n    public Object processResultSingle() throws AbortException,\n            NotAListException, UnknownException {\n        CommonErlangObjects.checkResult_failAbort(resultRaw, resultCompressed);\n        return processResult();\n    }\n\n    @Override\n    public String toString() {\n        return \"add_del_on_list(\" + key + \", \" + toAdd + \", \" + toRemove + \")\";\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/operations/AddOnNrOp.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.operations;\n\nimport com.ericsson.otp.erlang.OtpErlangDouble;\nimport com.ericsson.otp.erlang.OtpErlangLong;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.CommonErlangObjects;\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.NotANumberException;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Operation incrementing a numeric value.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.14\n * @since 3.14\n */\npublic class AddOnNrOp implements TransactionOperation, TransactionSingleOpOperation {\n    final protected OtpErlangString key;\n    final protected OtpErlangObject toAdd;\n    protected OtpErlangObject resultRaw = null;\n    protected boolean resultCompressed = false;\n\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            the number to add to the number stored at key\n     */\n    public AddOnNrOp(final OtpErlangString key, final OtpErlangLong toAdd) {\n        this.key = key;\n        this.toAdd = toAdd;\n    }\n\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            the number to add to the number stored at key\n     */\n    public AddOnNrOp(final OtpErlangString key, final OtpErlangDouble toAdd) {\n        this.key = key;\n        this.toAdd = toAdd;\n    }\n\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            the number to add to the number stored at key\n     */\n    public <T extends Number> AddOnNrOp(final String key, final T toAdd) {\n        this.key = new OtpErlangString(key);\n        this.toAdd = ErlangValue.convertToErlang(toAdd);\n    }\n\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to write the value to\n     * @param toAdd\n     *            the number to add to the number stored at key\n     */\n    public AddOnNrOp(final String key, final Double toAdd) {\n        this.key = new OtpErlangString(key);\n        this.toAdd = ErlangValue.convertToErlang(toAdd);\n    }\n\n    public OtpErlangObject getErlang(final boolean compressed) {\n        return new OtpErlangTuple(new OtpErlangObject[] {\n                CommonErlangObjects.addOnNrAtom, key,\n                compressed ? CommonErlangObjects.encode(toAdd) : toAdd });\n    }\n\n    public OtpErlangString getKey() {\n        return key;\n    }\n\n    public void setResult(final OtpErlangObject resultRaw, final boolean compressed) {\n        this.resultRaw = resultRaw;\n        this.resultCompressed = compressed;\n    }\n\n    public OtpErlangObject getResult() {\n        return this.resultRaw;\n    }\n\n    public boolean getResultCompressed() {\n        return this.resultCompressed;\n    }\n\n    public Object processResult() throws UnknownException,\n            NotANumberException {\n        /*\n         * possible return values:\n         *  {ok} | {fail, not_a_number}.\n         */\n        try {\n            final OtpErlangTuple received = (OtpErlangTuple) resultRaw;\n            if (received.equals(CommonErlangObjects.okTupleAtom)) {\n                return null;\n            } else if (received.elementAt(0).equals(CommonErlangObjects.failAtom) && (received.arity() == 2)) {\n                final OtpErlangObject reason = received.elementAt(1);\n                if (reason.equals(CommonErlangObjects.notANumberAtom)) {\n                    throw new NotANumberException(resultRaw);\n                }\n            }\n            throw new UnknownException(resultRaw);\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, resultRaw);\n        }\n    }\n\n    public Object processResultSingle() throws AbortException,\n            NotANumberException, UnknownException {\n        CommonErlangObjects.checkResult_failAbort(resultRaw, resultCompressed);\n        return processResult();\n    }\n\n    @Override\n    public String toString() {\n        return \"add_on_nr(\" + key + \", \" + toAdd + \")\";\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/operations/CommitOp.java",
    "content": "package de.zib.scalaris.operations;\n\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.CommonErlangObjects;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * An operation committing a transaction.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.18\n */\npublic class CommitOp implements TransactionOperation {\n    protected OtpErlangObject resultRaw = null;\n    protected boolean resultCompressed = false;\n    /**\n     * Constructor\n     */\n    public CommitOp() {\n    }\n\n    public OtpErlangObject getErlang(final boolean compressed) {\n        return CommonErlangObjects.commitTupleAtom;\n    }\n\n    public OtpErlangString getKey() {\n        return null;\n    }\n\n    public void setResult(final OtpErlangObject resultRaw, final boolean compressed) {\n        this.resultRaw = resultRaw;\n        this.resultCompressed = compressed;\n    }\n\n    public OtpErlangObject getResult() {\n        return this.resultRaw;\n    }\n\n    public boolean getResultCompressed() {\n        return this.resultCompressed;\n    }\n\n    public Object processResult() throws AbortException, UnknownException {\n        processResult_commit(resultRaw, resultCompressed);\n        return null;\n    }\n\n    @Override\n    public String toString() {\n        return \"commit()\";\n    }\n\n    /**\n     * Processes the <tt>received_raw</tt> term from erlang interpreting it as a\n     * result from a commit operation.\n     *\n     * NOTE: this method should not be called manually by an application and may\n     * change without prior notice!\n     *\n     * @param received_raw\n     *            the object to process\n     * @param compressed\n     *            whether the transfer of values is compressed or not\n     *\n     * @throws AbortException\n     *             if the commit of the commit failed\n     * @throws UnknownException\n     *             if any other error occurs\n     */\n    public static final void processResult_commit(final OtpErlangObject received_raw,\n            final boolean compressed) throws AbortException,\n            UnknownException {\n        /*\n         * possible return values:\n         *  {ok} | {fail, abort, KeyList}\n         */\n        try {\n            final OtpErlangTuple received = (OtpErlangTuple) received_raw;\n            if (received.equals(CommonErlangObjects.okTupleAtom)) {\n                return;\n            } else {\n                CommonErlangObjects.checkResult_failAbort(received, compressed);\n            }\n            throw new UnknownException(received_raw);\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, received_raw);\n        }\n    }\n}"
  },
  {
    "path": "java-api/src/de/zib/scalaris/operations/Operation.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.operations;\n\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.EmptyListException;\nimport de.zib.scalaris.KeyChangedException;\nimport de.zib.scalaris.NotAListException;\nimport de.zib.scalaris.NotANumberException;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Generic interface for operations which can be added to a request list.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.14\n */\npublic interface Operation {\n    /**\n     * Gets the erlang representation of the operation.\n     *\n     * @param compressed\n     *            whether the value part in the term should be encoded, i.e.\n     *            compressed into an Erlang binary, or not\n     *\n     * @return erlang representation for api_tx:req_list\n     */\n    abstract public OtpErlangObject getErlang(final boolean compressed);\n     /**\n      * Gets the key the operation is working on (if available)\n      *\n      * @return the key or <tt>null</tt>\n      */\n    abstract public OtpErlangString getKey();\n\n    /**\n     * Sets the raw erlang result value. It can be processed using\n     * {@link #processResult()}.\n     *\n     * @param resultRaw\n     *            the result\n     * @param compressed\n     *            whether the value inside the result is compressed or not\n     *\n     * @since 3.18\n     */\n    public abstract void setResult(final OtpErlangObject resultRaw,\n            final boolean compressed);\n\n    /**\n     * Gets the (raw Erlang) result set via\n     * {@link #setResult(OtpErlangObject, boolean)}.\n     *\n     * @return the result object or <tt>null</tt> if not set\n     *\n     * @since 3.18\n     */\n    public abstract OtpErlangObject getResult();\n\n    /**\n     * Determines if the result set via\n     * {@link #setResult(OtpErlangObject, boolean)} is compressed or not.\n     *\n     * @return <tt>true</tt> if compressed, <tt>false</tt> otherwise, undefined\n     *         if no result was set\n     *\n     * @since 3.18\n     */\n    public abstract boolean getResultCompressed();\n\n    /**\n     * Processes the result set by {@link #setResult(OtpErlangObject, boolean)}.\n     *\n     * Note: the created value is not cached!\n     *\n     * @return a (potentially) read value (may be <tt>null</tt>)\n     *\n     * @throws NotFoundException\n     *             if the requested key does not exist\n     * @throws KeyChangedException\n     *             if the key did not match <tt>old_value</tt>\n     * @throws NotANumberException\n     *             if the previously stored value was not a number\n     * @throws NotAListException\n     *             if the previously stored value was no list\n     * @throws EmptyListException\n     *             if the stored value is an empty list but the op requires a\n     *             non-empty list\n     * @throws AbortException\n     *             if a commit failed\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.18\n     */\n    public abstract Object processResult() throws NotFoundException,\n            KeyChangedException, NotANumberException, NotAListException,\n            AbortException, EmptyListException, UnknownException;\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/operations/PartialReadOp.java",
    "content": "/**\n *  Copyright 2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.operations;\n\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\n\n/**\n * Operation reading a partial value.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.18\n */\npublic abstract class PartialReadOp implements TransactionOperation,\n        TransactionSingleOpOperation {\n    final protected OtpErlangString key;\n    protected OtpErlangObject resultRaw = null;\n    protected boolean resultCompressed = false;\n\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to read\n     */\n    public PartialReadOp(final OtpErlangString key) {\n        this.key = key;\n    }\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to read\n     */\n    public PartialReadOp(final String key) {\n        this.key = new OtpErlangString(key);\n    }\n\n    public OtpErlangString getKey() {\n        return key;\n    }\n\n    public void setResult(final OtpErlangObject resultRaw, final boolean compressed) {\n        this.resultRaw = resultRaw;\n        this.resultCompressed = compressed;\n    }\n\n    public OtpErlangObject getResult() {\n        return this.resultRaw;\n    }\n\n    public boolean getResultCompressed() {\n        return this.resultCompressed;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/operations/ReadOp.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.operations;\n\nimport com.ericsson.otp.erlang.OtpErlangDecodeException;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.CommonErlangObjects;\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.KeyChangedException;\nimport de.zib.scalaris.NotAListException;\nimport de.zib.scalaris.NotANumberException;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Operation reading a value.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.14\n * @since 3.14\n */\npublic class ReadOp implements TransactionOperation, TransactionSingleOpOperation {\n    final protected OtpErlangString key;\n    protected OtpErlangObject resultRaw = null;\n    protected boolean resultCompressed = false;\n\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to read\n     */\n    public ReadOp(final OtpErlangString key) {\n        this.key = key;\n    }\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to read\n     */\n    public ReadOp(final String key) {\n        this.key = new OtpErlangString(key);\n    }\n\n    public OtpErlangObject getErlang(final boolean compressed) {\n        return new OtpErlangTuple(new OtpErlangObject[] {\n                CommonErlangObjects.readAtom, key });\n    }\n\n    public OtpErlangString getKey() {\n        return key;\n    }\n\n    public void setResult(final OtpErlangObject resultRaw, final boolean compressed) {\n        this.resultRaw = resultRaw;\n        this.resultCompressed = compressed;\n    }\n\n    public OtpErlangObject getResult() {\n        return this.resultRaw;\n    }\n\n    public boolean getResultCompressed() {\n        return this.resultCompressed;\n    }\n\n    public ErlangValue processResult() throws NotFoundException,\n            UnknownException {\n        /*\n         * possible return values:\n         *  {ok, Value} | {fail, not_found}\n         */\n        try {\n            final OtpErlangTuple received = (OtpErlangTuple) resultRaw;\n            final OtpErlangObject state = received.elementAt(0);\n            if (received.arity() != 2) {\n                throw new UnknownException(resultRaw);\n            }\n            if (state.equals(CommonErlangObjects.okAtom)) {\n                OtpErlangObject result = received.elementAt(1);\n                if (resultCompressed) {\n                    result = CommonErlangObjects.decode(result);\n                }\n                return new ErlangValue(result);\n            } else if (state.equals(CommonErlangObjects.failAtom)) {\n                final OtpErlangObject reason = received.elementAt(1);\n                if (reason.equals(CommonErlangObjects.notFoundAtom)) {\n                    throw new NotFoundException(resultRaw);\n                }\n            }\n            throw new UnknownException(resultRaw);\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, resultRaw);\n        } catch (final OtpErlangDecodeException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, resultRaw);\n        }\n    }\n\n    public ErlangValue processResultSingle() throws NotFoundException,\n            KeyChangedException, NotANumberException, NotAListException,\n            AbortException, UnknownException {\n        return processResult();\n    }\n\n    @Override\n    public String toString() {\n        return \"read(\" + key + \")\";\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/operations/ReadRandomFromListOp.java",
    "content": "/**\n *  Copyright 2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.operations;\n\nimport com.ericsson.otp.erlang.OtpErlangDecodeException;\nimport com.ericsson.otp.erlang.OtpErlangLong;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangRangeException;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\nimport de.zib.scalaris.CommonErlangObjects;\nimport de.zib.scalaris.EmptyListException;\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.NotAListException;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Operation reading a random entry from a (non-empty) list value.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.18\n */\npublic class ReadRandomFromListOp extends PartialReadOp {\n    /**\n     * Result type of random_from_list operations.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.18\n     * @since 3.18\n     */\n    public static class Result {\n        /**\n         * A random value from the stored list.\n         */\n        public final ErlangValue randomElement;\n        /**\n         * The length of the whole list stored in Scalaris.\n         */\n        public final int listLength;\n\n        protected Result(final OtpErlangObject result0, final boolean compressed)\n                throws OtpErlangDecodeException, UnknownException, ClassCastException {\n            // {RandomValue, ListLength}\n            OtpErlangTuple result;\n            if (compressed) {\n                result = (OtpErlangTuple) CommonErlangObjects.decode(result0);\n            } else {\n                result = (OtpErlangTuple) result0;\n            }\n\n            if (result.arity() != 2) {\n                throw new UnknownException(result);\n            }\n            randomElement = new ErlangValue(result.elementAt(0));\n            final OtpErlangLong listLengthOtp = (OtpErlangLong) result.elementAt(1);\n            try {\n                listLength = listLengthOtp.intValue();\n            } catch (final OtpErlangRangeException e) {\n                throw new UnknownException(\"Unsupported list length (\"\n                        + listLengthOtp + \")\");\n            }\n        }\n\n        /* (non-Javadoc)\n         * @see java.lang.Object#toString()\n         */\n        @Override\n        public String toString() {\n            return \"{randomElement: \" + randomElement.toString()\n                    + \", listLength: \" + listLength + \"}\";\n        }\n    }\n\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to read\n     */\n    public ReadRandomFromListOp(final OtpErlangString key) {\n        super(key);\n    }\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to read\n     */\n    public ReadRandomFromListOp(final String key) {\n        super(key);\n    }\n\n    public OtpErlangObject getErlang(final boolean compressed) {\n        return new OtpErlangTuple(new OtpErlangObject[] {\n                CommonErlangObjects.readAtom, key,\n                CommonErlangObjects.randomFromListAtom });\n    }\n\n    public Result processResult() throws NotFoundException, EmptyListException,\n            NotAListException, UnknownException {\n        /*\n         * possible return values:\n         *  {ok, {RandomValue, ListLength}} | {fail, not_found | empty_list | not_a_list}\n         */\n        try {\n            final OtpErlangTuple received = (OtpErlangTuple) resultRaw;\n            final OtpErlangObject state = received.elementAt(0);\n            if (received.arity() != 2) {\n                throw new UnknownException(resultRaw);\n            }\n            if (state.equals(CommonErlangObjects.okAtom)) {\n                return new Result(received.elementAt(1), resultCompressed);\n            } else if (state.equals(CommonErlangObjects.failAtom)) {\n                final OtpErlangObject reason = received.elementAt(1);\n                if (reason.equals(CommonErlangObjects.notFoundAtom)) {\n                    throw new NotFoundException(resultRaw);\n                } else if (reason.equals(CommonErlangObjects.emptyListAtom)) {\n                    throw new EmptyListException(resultRaw);\n                } else if (reason.equals(CommonErlangObjects.notAListAtom)) {\n                    throw new NotAListException(resultRaw);\n                }\n            }\n            throw new UnknownException(resultRaw);\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, resultRaw);\n        } catch (final OtpErlangDecodeException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, resultRaw);\n        }\n    }\n\n    public Result processResultSingle() throws NotFoundException,\n            EmptyListException, NotAListException, UnknownException {\n        return processResult();\n    }\n\n    @Override\n    public String toString() {\n        return \"readRandomFromList(\" + key + \")\";\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/operations/ReadSublistOp.java",
    "content": "/**\n *  Copyright 2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.operations;\n\nimport java.math.BigInteger;\n\nimport com.ericsson.otp.erlang.OtpErlangDecodeException;\nimport com.ericsson.otp.erlang.OtpErlangInt;\nimport com.ericsson.otp.erlang.OtpErlangLong;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangRangeException;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\nimport de.zib.scalaris.CommonErlangObjects;\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.NotAListException;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Operation reading a sublist from a value.\n *\n * Extracts a sublist of length <tt>Length</tt> starting at <tt>Start</tt>.\n * <ul>\n * <li>If Start is negative, we count from the end, e.g. <tt>-1</tt> is the last\n * element, <tt>-2</tt> the second last.</li>\n * <li>If Length is negative, the sublist is created in reversed direction, e.g.\n * <tt>sublist([a,b,c], -1, -2)</tt> gets <tt>[c, b]</tt>.</li>\n * <li>If Start is less than -ListLength and Length is non-negative, it will be\n * set to <tt>1</tt>. If Length is negative in this case, an empty sublist will\n * be returned.</li>\n * <li>If Start is greater than ListLength and Length is non-negative, an empty\n * sublist will be returned. If Length is negative in this case, it will be set\n * to <tt>ListLength</tt>.</li>\n * <li>Note: sublists never wrap between start and end, i.e.\n * <tt>sublist([a,b,c], 1,\n * -2)</tt> gets <tt>[]</tt>!</li>\n * <li>Examples:\n * <ul>\n * <li>first 10: <tt>sublist(L, 1, 10)</tt> | <tt>sublist(L, 10, -10)</tt>\n * (reverse order)</li>\n * <li>last 10 : <tt>sublist(L, -10, 10)</tt> | <tt>sublist(L, -1, -10)</tt>\n * (reverse order)</li>\n * </ul>\n * </li>\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.18\n */\npublic class ReadSublistOp extends PartialReadOp {\n    /**\n     * Result type of sublist operations.\n     *\n     * @author Nico Kruber, kruber@zib.de\n     * @version 3.18\n     * @since 3.18\n     */\n    public static class Result {\n        /**\n         * The retrieved sublist.\n         */\n        public final ErlangValue subList;\n        /**\n         * The length of the whole list stored in Scalaris.\n         */\n        public final int listLength;\n\n        protected Result(final OtpErlangObject result0, final boolean compressed)\n                throws OtpErlangDecodeException, UnknownException {\n            // {SubList, ListLength}\n            OtpErlangTuple result;\n            if (compressed) {\n                result = (OtpErlangTuple) CommonErlangObjects.decode(result0);\n            } else {\n                result = (OtpErlangTuple) result0;\n            }\n\n            if (result.arity() != 2) {\n                throw new UnknownException(result);\n            }\n            subList = new ErlangValue(ErlangValue.otpObjectToOtpList(result.elementAt(0)));\n            final OtpErlangLong listLengthOtp = (OtpErlangLong) result.elementAt(1);\n            try {\n                listLength = listLengthOtp.intValue();\n            } catch (final OtpErlangRangeException e) {\n                throw new UnknownException(\"Unsupported list length (\"\n                        + listLengthOtp + \")\");\n            }\n        }\n\n        /* (non-Javadoc)\n         * @see java.lang.Object#toString()\n         */\n        @Override\n        public String toString() {\n            return \"{subList: \" + subList.toString() + \", listLength: \"\n                    + listLength + \"}\";\n        }\n    }\n\n    final protected OtpErlangInt start;\n    final protected OtpErlangInt length;\n\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to read\n     * @param start\n     *            the start of the sublist (may be negative - see\n     *            {@link ReadSublistOp})\n     * @param length\n     *            the length of the sublist (may be negative - see\n     *            {@link ReadSublistOp})\n     */\n    public ReadSublistOp(final OtpErlangString key, final OtpErlangInt start, final OtpErlangInt length) {\n        super(key);\n        assert(!start.bigIntegerValue().equals(BigInteger.ZERO));\n        this.start = start;\n        this.length = length;\n    }\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to read\n     * @param start\n     *            the start of the sublist (may be negative - see\n     *            {@link ReadSublistOp})\n     * @param length\n     *            the length of the sublist (may be negative - see\n     *            {@link ReadSublistOp})\n     */\n    public ReadSublistOp(final String key, final int start, final int length) {\n        super(key);\n        assert(start != 0);\n        this.start = new OtpErlangInt(start);\n        this.length = new OtpErlangInt(length);\n    }\n\n    public OtpErlangObject getErlang(final boolean compressed) {\n        return new OtpErlangTuple(new OtpErlangObject[] {\n                CommonErlangObjects.readAtom, key,\n                new OtpErlangTuple(new OtpErlangObject[] {\n                        CommonErlangObjects.sublistAtom, start, length }) });\n    }\n\n    public Result processResult() throws NotFoundException, NotAListException,\n            UnknownException {\n        /*\n         * possible return values:\n         *  {ok, {SubList, ListLength}} | {fail, not_found | not_a_list}\n         */\n        try {\n            final OtpErlangTuple received = (OtpErlangTuple) resultRaw;\n            final OtpErlangObject state = received.elementAt(0);\n            if (received.arity() != 2) {\n                throw new UnknownException(resultRaw);\n            }\n            if (state.equals(CommonErlangObjects.okAtom)) {\n                return new Result(received.elementAt(1), resultCompressed);\n            } else if (state.equals(CommonErlangObjects.failAtom)) {\n                final OtpErlangObject reason = received.elementAt(1);\n                if (reason.equals(CommonErlangObjects.notFoundAtom)) {\n                    throw new NotFoundException(resultRaw);\n                } else if (reason.equals(CommonErlangObjects.notAListAtom)) {\n                    throw new NotAListException(resultRaw);\n                }\n            }\n            throw new UnknownException(resultRaw);\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, resultRaw);\n        } catch (final OtpErlangDecodeException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, resultRaw);\n        }\n    }\n\n    public Result processResultSingle() throws NotFoundException,\n            NotAListException, UnknownException {\n        return processResult();\n    }\n\n    @Override\n    public String toString() {\n        return \"readSublist(\" + key + \",\" + start + \",\" + length + \")\";\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/operations/TestAndSetOp.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.operations;\n\nimport com.ericsson.otp.erlang.OtpErlangDecodeException;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.CommonErlangObjects;\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.KeyChangedException;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * Atomic test-and-set operation, i.e. {@link #newValue} is only written if the\n * currently stored value is {@link #oldValue}.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.14\n * @since 3.14\n */\npublic class TestAndSetOp implements TransactionOperation, TransactionSingleOpOperation {\n    final protected OtpErlangString key;\n    final protected OtpErlangObject oldValue;\n    final protected OtpErlangObject newValue;\n    protected OtpErlangObject resultRaw = null;\n    protected boolean resultCompressed = false;\n\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to write the value to\n     * @param oldValue\n     *            the old value to verify\n     * @param newValue\n     *            the new value to write of oldValue is correct\n     */\n    public TestAndSetOp(final OtpErlangString key, final OtpErlangObject oldValue, final OtpErlangObject newValue) {\n        this.key = key;\n        this.oldValue = oldValue;\n        this.newValue = newValue;\n    }\n\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to write the value to\n     * @param oldValue\n     *            the old value to verify\n     * @param newValue\n     *            the new value to write of oldValue is correct\n     */\n    public <OldT, NewT> TestAndSetOp(final String key, final OldT oldValue, final NewT newValue) {\n        this.key = new OtpErlangString(key);\n        this.oldValue = ErlangValue.convertToErlang(oldValue);\n        this.newValue = ErlangValue.convertToErlang(newValue);\n    }\n\n    public OtpErlangObject getErlang(final boolean compressed) {\n        return new OtpErlangTuple(new OtpErlangObject[] {\n                CommonErlangObjects.testAndSetAtom, key,\n                compressed ? CommonErlangObjects.encode(oldValue) : oldValue,\n                compressed ? CommonErlangObjects.encode(newValue) : newValue });\n    }\n\n    public OtpErlangString getKey() {\n        return key;\n    }\n\n    public void setResult(final OtpErlangObject resultRaw, final boolean compressed) {\n        this.resultRaw = resultRaw;\n        this.resultCompressed = compressed;\n    }\n\n    public OtpErlangObject getResult() {\n        return this.resultRaw;\n    }\n\n    public boolean getResultCompressed() {\n        return this.resultCompressed;\n    }\n\n    public Object processResult() throws NotFoundException,\n            KeyChangedException, UnknownException {\n        /*\n         * possible return values:\n         *  {ok} | {fail, not_found | {key_changed, RealOldValue}\n         */\n        try {\n            final OtpErlangTuple received = (OtpErlangTuple) resultRaw;\n            if (received.equals(CommonErlangObjects.okTupleAtom)) {\n                return null;\n            } else if (received.elementAt(0).equals(CommonErlangObjects.failAtom) && (received.arity() == 2)) {\n                final OtpErlangObject reason = received.elementAt(1);\n                if (reason.equals(CommonErlangObjects.notFoundAtom)) {\n                    throw new NotFoundException(resultRaw);\n                } else {\n                    final OtpErlangTuple reason_tpl = (OtpErlangTuple) reason;\n                    if (reason_tpl.elementAt(0).equals(\n                            CommonErlangObjects.keyChangedAtom)\n                            && (reason_tpl.arity() == 2)) {\n                        OtpErlangObject result = reason_tpl.elementAt(1);\n                        if (resultCompressed) {\n                            result = CommonErlangObjects.decode(result);\n                        }\n                        throw new KeyChangedException(new ErlangValue(result));\n                    }\n                }\n            }\n            throw new UnknownException(resultRaw);\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, resultRaw);\n        } catch (final OtpErlangDecodeException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, resultRaw);\n        }\n    }\n\n    public Object processResultSingle() throws AbortException,\n            NotFoundException, KeyChangedException, UnknownException {\n        CommonErlangObjects.checkResult_failAbort(resultRaw, resultCompressed);\n        return processResult();\n    }\n\n    @Override\n    public String toString() {\n        return \"test_and_set(\" + key + \", \" + oldValue + \", \" + newValue + \")\";\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/operations/TransactionOperation.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.operations;\n\nimport de.zib.scalaris.Transaction;\n\n/**\n * An operation suitable for use in {@link Transaction}.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.14\n * @since 3.14\n */\npublic interface TransactionOperation extends Operation {\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/operations/TransactionSingleOpOperation.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.operations;\n\nimport com.ericsson.otp.erlang.OtpErlangObject;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.EmptyListException;\nimport de.zib.scalaris.KeyChangedException;\nimport de.zib.scalaris.NotAListException;\nimport de.zib.scalaris.NotANumberException;\nimport de.zib.scalaris.NotFoundException;\nimport de.zib.scalaris.TransactionSingleOp;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * An operation suitable for use in {@link TransactionSingleOp}.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.14\n */\npublic interface TransactionSingleOpOperation extends Operation {\n\n    /**\n     * Processes the result set by {@link #setResult(OtpErlangObject, boolean)}\n     * assuming that operation was committed.\n     *\n     * In contrast to {@link #processResult()} operations like {@link WriteOp}\n     * will throw a proper {@link AbortException} for their commit part instead\n     * of an {@link UnknownException}.\n     *\n     * Note: the created value is not cached!\n     *\n     * @return a (potentially) read value (may be <tt>null</tt>)\n     *\n     * @throws NotFoundException\n     *             if the requested key does not exist\n     * @throws KeyChangedException\n     *             if the key did not match <tt>old_value</tt>\n     * @throws NotANumberException\n     *             if the previously stored value was not a number\n     * @throws NotAListException\n     *             if the previously stored value was no list\n     * @throws EmptyListException\n     *             if the stored value is an empty list but the op requires a\n     *             non-empty list\n     * @throws AbortException\n     *             if a commit failed\n     * @throws UnknownException\n     *             if any other error occurs\n     *\n     * @since 3.18\n     */\n    public abstract Object processResultSingle() throws NotFoundException,\n            KeyChangedException, NotANumberException, NotAListException,\n            AbortException, EmptyListException, UnknownException;\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/operations/WriteOp.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris.operations;\n\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\nimport de.zib.scalaris.AbortException;\nimport de.zib.scalaris.CommonErlangObjects;\nimport de.zib.scalaris.ErlangValue;\nimport de.zib.scalaris.UnknownException;\n\n/**\n * An operation writing a value.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.14\n * @since 3.14\n */\npublic class WriteOp implements TransactionOperation, TransactionSingleOpOperation {\n    final protected OtpErlangString key;\n    final protected OtpErlangObject value;\n    protected OtpErlangObject resultRaw = null;\n    protected boolean resultCompressed = false;\n\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to write the value to\n     * @param value\n     *            the value to write\n     */\n    public WriteOp(final OtpErlangString key, final OtpErlangObject value) {\n        this.key = key;\n        this.value = value;\n    }\n    /**\n     * Constructor\n     *\n     * @param key\n     *            the key to write the value to\n     * @param value\n     *            the value to write\n     */\n    public <T> WriteOp(final String key, final T value) {\n        this.key = new OtpErlangString(key);\n        this.value = ErlangValue.convertToErlang(value);\n    }\n\n    public OtpErlangObject getErlang(final boolean compressed) {\n        return new OtpErlangTuple(new OtpErlangObject[] {\n                CommonErlangObjects.writeAtom, key,\n                compressed ? CommonErlangObjects.encode(value) : value });\n    }\n\n    public OtpErlangString getKey() {\n        return key;\n    }\n\n    public void setResult(final OtpErlangObject resultRaw, final boolean compressed) {\n        this.resultRaw = resultRaw;\n        this.resultCompressed = compressed;\n    }\n\n    public OtpErlangObject getResult() {\n        return this.resultRaw;\n    }\n\n    public boolean getResultCompressed() {\n        return this.resultCompressed;\n    }\n\n    public Object processResult() throws UnknownException {\n        /*\n         * possible return values:\n         *  {ok}\n         */\n        try {\n            final OtpErlangTuple received = (OtpErlangTuple) resultRaw;\n            if (received.equals(CommonErlangObjects.okTupleAtom)) {\n                return null;\n            }\n            throw new UnknownException(resultRaw);\n        } catch (final ClassCastException e) {\n            // e.printStackTrace();\n            throw new UnknownException(e, resultRaw);\n        }\n    }\n\n    public Object processResultSingle() throws AbortException, UnknownException {\n        CommonErlangObjects.checkResult_failAbort(resultRaw, resultCompressed);\n        return processResult();\n    }\n\n    @Override\n    public String toString() {\n        return \"write(\" + key + \", \" + value + \")\";\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/scalaris/package-info.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\n/**\n * This package contains means to communicate with the erlang scalaris ring from Java.\n *\n * <h3>The TransactionSingleOp class</h3>\n * <p>\n * The {@link de.zib.scalaris.TransactionSingleOp} class provides methods for\n * reading and writing values, with both,\n * erlang objects ({@link com.ericsson.otp.erlang.OtpErlangObject}) and\n * Java objects like {@link java.lang.String}.\n * </p>\n *\n * <h4>Example:</h4>\n * <pre>\n * <code style=\"white-space:pre;\">\n *   try {\n *     TransactionSingleOp sc = new TransactionSingleOp();\n *     String value = sc.read(\"key\").stringValue();\n *   } catch (ConnectionException e) {\n *     System.err.println(\"read failed: \" + e.getMessage());\n *   } catch (NotFoundException e) {\n *     System.err.println(\"read failed with not found: \" + e.getMessage());\n *   } catch (ClassCastException e) {\n *     System.err.println(\"read failed with unexpected return type: \" + e.getMessage());\n *   } catch (UnknownException e) {\n *     System.err.println(\"read failed with unknown: \" + e.getMessage());\n *   }\n * </code>\n * </pre>\n *\n * <p>See the {@link de.zib.scalaris.TransactionSingleOp} class documentation\n * for more details.</p>\n *\n * <h3>The Transaction class</h3>\n * <p>\n * The {@link de.zib.scalaris.Transaction} class provides means to realise a\n * scalaris transaction from Java. There are methods to read and write values\n * with both erlang objects ({@link com.ericsson.otp.erlang.OtpErlangObject})\n * and Java objects like {@link java.lang.String}. The transaction can then be\n * committed or aborted.\n * </p>\n *\n * <h4>Example:</h4>\n * <pre>\n * <code style=\"white-space:pre;\">\n *   try {\n *     Transaction transaction = new Transaction();\n *     String value = transaction.read(\"key\").stringValue();\n *     transaction.write(\"key\", \"value\");\n *     transaction.commit();\n *   } catch (ConnectionException e) {\n *     System.err.println(\"read failed: \" + e.getMessage());\n *   } catch (NotFoundException e) {\n *     System.err.println(\"read failed with not found: \" + e.getMessage());\n *   } catch (UnknownException e) {\n *     System.err.println(\"read failed with unknown: \" + e.getMessage());\n *   }\n * </code>\n * </pre>\n *\n * <p>See the {@link de.zib.scalaris.Transaction} class documentation for more\n * details.</p>\n *\n * <h3>The ReplicatedDHT class</h3>\n * <p>\n * The {@link de.zib.scalaris.ReplicatedDHT} class provides methods for\n * working inconsistently on replicated key/value pairs, e.g. to delete\n * replicas. It supports both, erlang strings\n * ({@link com.ericsson.otp.erlang.OtpErlangString}) and Java strings\n * ({@link java.lang.String}).\n * </p>\n *\n * <h4>Example:</h4>\n * <pre>\n * <code style=\"white-space:pre;\">\n *   try {\n *     ReplicatedDHT sc = new ReplicatedDHT();\n *     long deleted = sc.delete(\"key\");\n *     DeleteResult delRes = sc.getLastDeleteResult();\n *   } catch (ConnectionException e) {\n *     System.err.println(\"delete failed: \" + e.getMessage());\n *   } catch (TimeoutException e) {\n *     System.err.println(\"delete failed with timeout: \" + e.getMessage());\n *   } catch (NodeNotFoundException e) {\n *     System.err.println(\"delete failed with node not found: \" + e.getMessage());\n *   } catch (UnknownException e) {\n *     System.err.println(\"delete failed with unknown: \" + e.getMessage());\n *   }\n * </code>\n * </pre>\n *\n * <p>See the {@link de.zib.scalaris.ReplicatedDHT} class documentation for\n * more details.</p>\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.9\n * @since 2.0\n */\npackage de.zib.scalaris;\n"
  },
  {
    "path": "java-api/src/de/zib/tools/CircularByteArrayOutputStream.java",
    "content": "/**\n *  Copyright 2011-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.tools;\n\nimport java.io.OutputStream;\nimport java.io.UnsupportedEncodingException;\nimport java.util.Arrays;\n\n/**\n * Circular {@link OutputStream} that limits the number of bytes.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.18\n */\npublic class CircularByteArrayOutputStream extends OutputStream {\n    /**\n     * The buffer where data is stored.\n     */\n    protected byte buf[];\n\n    protected int pos = 0;\n    boolean filled = false;\n\n    /**\n     * Creates a new byte array output stream, with a buffer capacity of\n     * the specified size, in bytes.\n     *\n     * @param   size   the initial size.\n     * @exception  IllegalArgumentException if size is negative.\n     */\n    public CircularByteArrayOutputStream(final int size) {\n        if (size < 0) {\n            throw new IllegalArgumentException(\"Negative initial size: \"\n                    + size);\n        }\n        buf = new byte[size];\n    }\n\n    @Override\n    public synchronized void write(final int b) {\n        if (pos == buf.length) {\n            filled = true;\n            pos = 0;\n        }\n        buf[pos++] = (byte) b;\n    }\n\n    /**\n     * Clears all contents of the buffer.\n     */\n    public synchronized void clear() {\n        pos = 0;\n        filled = false;\n    }\n\n    /**\n     * Creates a newly allocated byte array. Its size is the current\n     * size of this output stream and the valid contents of the buffer\n     * have been copied into it.\n     *\n     * @return  the current contents of this output stream, as a byte array.\n     * @see     java.io.ByteArrayOutputStream#size()\n     */\n    public synchronized byte toByteArray()[] {\n        if (!filled) {\n            return Arrays.copyOf(buf, pos);\n        }\n        final byte[] ret = new byte[buf.length];\n        System.arraycopy(buf, pos, ret, 0, buf.length - pos);\n        System.arraycopy(buf, 0, ret, buf.length - pos, pos);\n        return ret;\n    }\n\n\n    /**\n     * Converts the buffer's contents into a string decoding bytes using the\n     * platform's default character set. The length of the new <tt>String</tt>\n     * is a function of the character set, and hence may not be equal to the\n     * size of the buffer.\n     *\n     * <p> This method always replaces malformed-input and unmappable-character\n     * sequences with the default replacement string for the platform's\n     * default character set. The {@linkplain java.nio.charset.CharsetDecoder}\n     * class should be used when more control over the decoding process is\n     * required.\n     *\n     * @return String decoded from the buffer's contents.\n     */\n    @Override\n    public synchronized String toString() {\n        return new String(toByteArray());\n    }\n\n    /**\n     * Converts the buffer's contents into a string by decoding the bytes using\n     * the specified {@link java.nio.charset.Charset charsetName}. The length of\n     * the new <tt>String</tt> is a function of the charset, and hence may not be\n     * equal to the length of the byte array.\n     *\n     * <p> This method always replaces malformed-input and unmappable-character\n     * sequences with this charset's default replacement string. The {@link\n     * java.nio.charset.CharsetDecoder} class should be used when more control\n     * over the decoding process is required.\n     *\n     * @param  charsetName  the name of a supported\n     *          {@linkplain java.nio.charset.Charset </code>charset<code>}\n     * @return String decoded from the buffer's contents.\n     * @exception  UnsupportedEncodingException\n     *             If the named charset is not supported\n     */\n    public synchronized String toString(final String charsetName)\n            throws UnsupportedEncodingException {\n        return new String(toByteArray(), charsetName);\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/tools/LinkedMultiHashMap.java",
    "content": "/**\n *  Copyright 2012-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.tools;\n\nimport java.util.LinkedHashMap;\nimport java.util.List;\nimport java.util.Map;\n\n/**\n * Provides a multi-map, i.e. a map associating multiple values with a single\n * key. Wraps the {@link LinkedHashMap} class.\n *\n * @param <K>\n *            key type\n * @param <V>\n *            value type\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.18\n */\npublic class LinkedMultiHashMap<K, V> extends MultiMap<LinkedHashMap<K, List<V>>, K, V> {\n    /**\n     * Constructs an empty map.\n     */\n    public LinkedMultiHashMap() {\n        super(LinkedHashMap.class);\n    }\n\n    /**\n     * Constructs an empty map.\n     *\n     * @param initialCapacity\n     *            the initial capacity of the hash table\n     */\n    public LinkedMultiHashMap(final int initialCapacity) {\n        super(LinkedHashMap.class, initialCapacity);\n    }\n\n    /**\n     * Creates a map containing all values from another map.\n     *\n     * @param other\n     *            the map to copy the values from\n     *\n     * @see #putAll(Map)\n     */\n    public LinkedMultiHashMap(final Map<? extends K, ? extends List<V>> other) {\n        super(LinkedHashMap.class, other);\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/tools/MultiHashMap.java",
    "content": "/**\n *  Copyright 2012-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.tools;\n\nimport java.util.HashMap;\nimport java.util.List;\nimport java.util.Map;\n\n/**\n * Provides a multi-map, i.e. a map associating multiple values with a single\n * key. Wraps the {@link HashMap} class.\n *\n * @param <K>\n *            key type\n * @param <V>\n *            value type\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.18\n */\npublic class MultiHashMap<K, V> extends MultiMap<HashMap<K, List<V>>, K, V> {\n    /**\n     * Constructs an empty map.\n     */\n    public MultiHashMap() {\n        super(HashMap.class);\n    }\n\n    /**\n     * Constructs an empty map.\n     *\n     * @param initialCapacity\n     *            the initial capacity of the hash table\n     */\n    public MultiHashMap(final int initialCapacity) {\n        super(HashMap.class, initialCapacity);\n    }\n\n    /**\n     * Creates a map containing all values from another map.\n     *\n     * @param other\n     *            the map to copy the values from\n     *\n     * @see #putAll(Map)\n     */\n    public MultiHashMap(final Map<? extends K, ? extends List<V>> other) {\n        super(HashMap.class, other);\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/tools/MultiMap.java",
    "content": "/**\n *  Copyright 2012-2013 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.tools;\n\nimport java.util.ArrayList;\nimport java.util.Collection;\nimport java.util.List;\nimport java.util.Map;\nimport java.util.Set;\n\n/**\n * Provides a multi-map, i.e. a map associating multiple values with a single\n * key. Wraps the an arbitrary {@link Map}-class.\n *\n * @param <T>\n *            wrapped map type\n * @param <K>\n *            key type\n * @param <V>\n *            value type\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.18\n * @since 3.18\n */\npublic class MultiMap<T extends Map<K, List<V>>, K, V> implements Map<K, List<V>> {\n\n    protected final T data;\n\n    /**\n     * Constructs an empty map.\n     *\n     * @param clazz\n     *            the {@link Map}-class to use for the internal representation\n     */\n    @SuppressWarnings({ \"unchecked\", \"rawtypes\" })\n    public <U extends Map> MultiMap(final Class<U> clazz) {\n        super();\n        try {\n            data = (T) clazz.getConstructor().newInstance();\n        } catch (final Exception e) {\n            throw new RuntimeException(e);\n        }\n    }\n\n    /**\n     * Constructs an empty map.\n     *\n     * @param clazz\n     *            the {@link Map}-class to use for the internal representation\n     * @param initialCapacity\n     *            the initial capacity of the hash table\n     */\n    @SuppressWarnings({ \"unchecked\", \"rawtypes\" })\n    public <U extends Map> MultiMap(final Class<U> clazz, final int initialCapacity) {\n        super();\n        try {\n            data = (T) clazz.getConstructor(int.class).newInstance(initialCapacity);\n        } catch (final Exception e) {\n            throw new RuntimeException(e);\n        }\n    }\n\n    /**\n     * Creates a map containing all values from another map.\n     *\n     * @param clazz\n     *            the {@link Map}-class to use for the internal representation\n     * @param other\n     *            the map to copy the values from\n     *\n     * @see #putAll(Map)\n     */\n    @SuppressWarnings({ \"unchecked\", \"rawtypes\" })\n    public <U extends Map> MultiMap(final Class<U> clazz, final Map<? extends K, ? extends List<V>> other) {\n        super();\n        try {\n            data = (T) clazz.getConstructor().newInstance();\n        } catch (final Exception e) {\n            throw new RuntimeException(e);\n        }\n        putAll(other);\n    }\n\n    /**\n     * {@inheritDoc}\n     */\n    public void clear() {\n        data.clear();\n    }\n\n    /**\n     * {@inheritDoc}\n     */\n    public boolean containsKey(final Object key) {\n        return data.containsKey(key);\n    }\n\n    /**\n     * {@inheritDoc}\n     */\n    public boolean containsValue(final Object value) {\n        for (final Entry<? extends K, ? extends List<V>> stat : data.entrySet()) {\n            if (stat.getValue().contains(value)) {\n                return true;\n            }\n        }\n        return false;\n    }\n\n    /**\n     * {@inheritDoc}\n     */\n    public Set<java.util.Map.Entry<K, List<V>>> entrySet() {\n        return data.entrySet();\n    }\n\n    /**\n     * {@inheritDoc}\n     */\n    public List<V> get(final Object key) {\n        return data.get(key);\n    }\n\n    /**\n     * {@inheritDoc}\n     */\n    public boolean isEmpty() {\n        return data.isEmpty();\n    }\n\n    /**\n     * {@inheritDoc}\n     */\n    public Set<K> keySet() {\n        return data.keySet();\n    }\n\n    /**\n     * Convenience method for inserting a single value into the map.\n     *\n     * @param key\n     *            the key to insert at\n     * @param value\n     *            the value to insert\n     *\n     * @return the previous value associated with key, or <tt>null</tt> if there\n     *         was no mapping for key. (A <tt>null</tt> return can also indicate\n     *         that the map previously associated <tt>null</tt> with key, if the\n     *         implementation supports <tt>null</tt> values.)\n     */\n    public List<V> put1(final K key, final V value) {\n        final ArrayList<V> list = new ArrayList<V>();\n        list.add(value);\n        return put(key, list);\n    }\n\n    /**\n     * {@inheritDoc}\n     */\n    public List<V> put(final K key, final List<V> value) {\n        final List<V> l = data.get(key);\n        if (l == null) {\n            return data.put(key, value);\n        } else {\n            l.addAll(value);\n            return data.put(key, l);\n        }\n    }\n\n    /**\n     * {@inheritDoc}\n     */\n    public void putAll(final Map<? extends K, ? extends List<V>> m) {\n        for (final Entry<? extends K, ? extends List<V>> value : m.entrySet()) {\n            put(value.getKey(), value.getValue());\n        }\n    }\n\n    /**\n     * {@inheritDoc}\n     */\n    public List<V> remove(final Object key) {\n        return data.remove(key);\n    }\n\n    /**\n     * {@inheritDoc}\n     */\n    public int size() {\n        return data.size();\n    }\n\n    /**\n     * {@inheritDoc}\n     */\n    public Collection<List<V>> values() {\n        return data.values();\n    }\n\n    @Override\n    public String toString() {\n        return data.toString();\n    }\n\n}"
  },
  {
    "path": "java-api/src/de/zib/tools/PropertyLoader.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.tools;\n\nimport java.io.FileInputStream;\nimport java.io.FileNotFoundException;\nimport java.io.IOException;\nimport java.net.URL;\nimport java.net.URLDecoder;\nimport java.util.Arrays;\nimport java.util.HashSet;\nimport java.util.Properties;\n\n/**\n * Provides methods to load property files with default look up mechanisms.\n *\n * <h3>Example:</h3>\n * <code style=\"white-space:pre;\">\n *   Properties properties = new Properties();\n *   PropertyLoader.loadProperties(properties, \"PropertiesFile.properties\"); // {@link #loadProperties(java.util.Properties, String)}\n * </code>\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 1.0\n */\npublic class PropertyLoader {\n    /**\n     * Tries to locate the file given by {@code filename} and loads it into the\n     * given properties parameter.\n     *\n     * @param properties\n     *            the {@link Properties} object to load the file into\n     * @param filename\n     *            the filename of the file containing the properties\n     *\n     * @return indicates whether the properties have been successfully loaded\n     */\n    public static boolean loadProperties(final Properties properties, final String filename) {\n        return loadProperties(properties, filename, false, false, new String[0]);\n    }\n\n    /**\n     * Tries to locate the file given by {@code filename} and loads it into the\n     * given properties parameter.\n     *\n     * @param properties\n     *            the {@link Properties} object to load the file into\n     * @param filename\n     *            the filename of the file containing the properties\n     * @param envNonemptyOverwrite\n     *            allows properties to be overwritten by non-empty system\n     *            properties\n     * @param envEmptyOverwrite\n     *            allows properties to be overwritten by empty system properties\n     * @param addPropNames\n     *            names of additional properties which can be fetched from\n     *            system properties\n     *\n     * @return indicates whether the properties have been successfully loaded\n     */\n    public static boolean loadProperties(final Properties properties,\n            final String filename, final boolean envNonemptyOverwrite,\n            final boolean envEmptyOverwrite, final String[] addPropNames) {\n        FileInputStream fis = null;\n        try {\n            final ClassLoader classLoader = PropertyLoader.class.getClassLoader();\n            if (classLoader != null) {\n                final URL url = classLoader.getResource(filename);\n                if (url != null) {\n                    final String path = URLDecoder.decode(url.getFile(), \"UTF-8\");\n                    fis = new FileInputStream(path);\n                    properties.load(fis);\n                    properties.setProperty(\"PropertyLoader.loadedfile\", path);\n                    fis.close();\n                    return true;\n                }\n            }\n            // try default path if the file was not found\n            fis = new FileInputStream(filename);\n            properties.load(fis);\n            properties.setProperty(\"PropertyLoader.loadedfile\", filename);\n            fis.close();\n            return true;\n        } catch (final FileNotFoundException e) {\n            // TODO add logging\n            // e.printStackTrace();\n        } catch (final IOException e) {\n            // TODO add logging\n            // e.printStackTrace();\n        } finally {\n            if (fis != null) {\n                try {\n                    fis.close();\n                } catch (final IOException e) {\n                }\n            }\n            if (envEmptyOverwrite || envNonemptyOverwrite) {\n                final HashSet<Object> keys = new HashSet<Object>(properties.keySet());\n                keys.addAll(Arrays.asList(addPropNames));\n                for (final Object key: keys) {\n                    final String propName = (String) key;\n                    if (!propName.equals(\"PropertyLoader.loadedfile\")) {\n                        final String prop = System.getProperty(propName);\n                        if ((prop != null)) {\n                            if ((envNonemptyOverwrite && !prop.isEmpty()) ||\n                                    (envEmptyOverwrite && prop.isEmpty())) {\n                                properties.setProperty(propName, prop);\n                            }\n                        }\n                    }\n                }\n            }\n        }\n        return false;\n    }\n}\n"
  },
  {
    "path": "java-api/src/de/zib/tools/package-info.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\n/**\n * This package contains some generic tools useful for most implementations.\n *\n * <h3>The PropertyLoader class</h3>\n * <p>\n * The {@link de.zib.tools.PropertyLoader} class provides methods to load\n * property files with look-up mechanisms to find a file given only by its name.\n * </p>\n *\n * <h4>Example:</h4>\n * <code style=\"white-space:pre;\">\n *   Properties properties = new Properties();\n *   PropertyLoader.loadProperties(properties, \"PropertiesFile.properties\");\n * </code>\n *\n * <p>See the {@link de.zib.tools.PropertyLoader} class documentation for more\n * details.</p>\n *\n * <h3>MultiMap classes</h3>\n *\n * Provides a multi-map, i.e. a map associating multiple values with a single\n * key.\n *\n * <p>See the {@link de.zib.tools.MultiMap} class documentation for more\n * details.</p>\n */\npackage de.zib.tools;\n"
  },
  {
    "path": "java-api/src/scalaris.properties",
    "content": "# the name of the scalaris node to establish the connection to\nscalaris.node=node@localhost,node1@localhost,node2@localhost,node3@localhost\n\n# the cookie the scalaris node uses for connections\nscalaris.cookie=chocolate chip cookie\n\n# the name of the (Java) client to use when establishing a connection with erlang\nscalaris.client.name=java_client\n\n# specifies whether to append an UUID to client names or not\nscalaris.client.appendUUID=true\n"
  },
  {
    "path": "java-api/test/de/zib/scalaris/ConnectionTest.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport static org.junit.Assert.*;\n\nimport java.io.IOException;\nimport java.util.Date;\nimport java.util.concurrent.TimeUnit;\n\nimport org.junit.Test;\n\nimport com.ericsson.otp.erlang.OtpAuthException;\nimport com.ericsson.otp.erlang.OtpErlangExit;\nimport com.ericsson.otp.erlang.OtpErlangInt;\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangLong;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangRangeException;\nimport com.ericsson.otp.erlang.OtpSelf;\n\n/**\n * Test cases for the {@link Connection} class.\n *\n * @author Nico Kruber, kruber@zib.de\n *\n * @version 2.3\n * @since 2.3\n */\npublic class ConnectionTest {\n\n    final static String scalarisNode;\n\n    static {\n        // determine good/bad nodes:\n        final ConnectionFactory cf = ConnectionFactory.getInstance();\n        cf.testAllNodes();\n        // set not to automatically try reconnects (auto-retries prevent ConnectionException tests from working):\n        final DefaultConnectionPolicy cp = ((DefaultConnectionPolicy) cf.getConnectionPolicy());\n        cp.setMaxRetries(0);\n        scalarisNode = cp.selectNode().toString();\n    }\n\n    /**\n     * Test method for {@link Connection#Connection(OtpSelf, PeerNode)}.\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     * @throws IOException\n     *             if the connection is not active or a communication error\n     *             occurs\n     * @throws OtpErlangExit\n     *             if an exit signal is received from a process on the peer node\n     * @throws OtpAuthException\n     *             if the remote node sends a message containing an invalid\n     *             cookie\n     * @throws InterruptedException if the sleep is interrupted\n     */\n    @Test\n    public final void testConnectionOtpSelfPeerNode() throws ConnectionException,\n            OtpErlangExit, OtpAuthException, IOException, InterruptedException {\n        final OtpSelf self = new OtpSelf(\"testConnection@\" + ConnectionFactory.getLocalhostName(), ConnectionFactory\n                .getInstance().getCookie());\n        final PeerNode remote = new PeerNode(scalarisNode);\n        final Date d0 = new Date();\n        TimeUnit.MILLISECONDS.sleep(10);\n        final Connection c = new Connection(self, remote);\n\n        assertEquals(self, c.getSelf());\n        assertEquals(remote, c.getRemote());\n        assertTrue(c.getConnection().isConnected());\n        assertNotNull(remote.getLastConnectSuccess());\n        assertTrue(d0.getTime() < remote.getLastConnectSuccess().getTime());\n        assertEquals(0, remote.getFailureCount());\n        assertNull(remote.getLastFailedConnect());\n\n        c.close();\n    }\n\n    /**\n     * Test method for {@link Connection#Connection(OtpSelf, ConnectionPolicy)}.\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     * @throws IOException\n     *             if the connection is not active or a communication error\n     *             occurs\n     * @throws OtpErlangExit\n     *             if an exit signal is received from a process on the peer node\n     * @throws OtpAuthException\n     *             if the remote node sends a message containing an invalid\n     *             cookie\n     * @throws InterruptedException if the sleep is interrupted\n     */\n    @Test\n    public final void testConnectionOtpSelfConnectionPolicy() throws ConnectionException,\n            OtpErlangExit, OtpAuthException, IOException, InterruptedException {\n        final OtpSelf self = new OtpSelf(\"testConnection@\" + ConnectionFactory.getLocalhostName(), ConnectionFactory\n                .getInstance().getCookie());\n        final PeerNode remote = new PeerNode(scalarisNode);\n        final Date d0 = new Date();\n        TimeUnit.MILLISECONDS.sleep(10);\n        final Connection c = new Connection(self, new DefaultConnectionPolicy(remote));\n\n        assertEquals(self, c.getSelf());\n        assertEquals(remote, c.getRemote());\n        assertTrue(c.getConnection().isConnected());\n        assertNotNull(remote.getLastConnectSuccess());\n        assertTrue(d0.getTime() < remote.getLastConnectSuccess().getTime());\n        assertEquals(0, remote.getFailureCount());\n        assertNull(remote.getLastFailedConnect());\n\n        c.close();\n    }\n\n    /**\n     * Test method for\n     * {@link Connection#Connection(OtpSelf, PeerNode)}.\n     *\n     * Tries several kinds of connections that fail and evaluates the statistics\n     * of the {@link PeerNode} object.\n     *\n     * @throws IOException\n     *             if the connection is not active or a communication error\n     *             occurs (should not happen)\n     * @throws InterruptedException if the sleep is interrupted\n     */\n    @Test\n    public final void testFailedConnection() throws IOException, InterruptedException {\n        OtpSelf self;\n        PeerNode remote;\n        Connection c;\n        DefaultConnectionPolicy connectionPolicy;\n        final Date d0 = new Date();\n        Date d1, d2;\n        TimeUnit.MILLISECONDS.sleep(10);\n\n        // wrong cookie:\n        self = new OtpSelf(\"testFailedConnection@\" + ConnectionFactory.getLocalhostName(),\n                ConnectionFactory.getInstance().getCookie() + \"someWrongCookieValue\");\n        remote = new PeerNode(scalarisNode);\n        connectionPolicy = new DefaultConnectionPolicy(remote);\n        connectionPolicy.setMaxRetries(0);\n        try {\n            c = new Connection(self, connectionPolicy);\n            // this should have failed!\n            fail();\n            c.close();\n        } catch (final Exception e) {\n        }\n        assertEquals(1, remote.getFailureCount());\n        d1 = remote.getLastFailedConnect();\n        assertNotNull(d1);\n        assertTrue(d0.getTime() < d1.getTime());\n        TimeUnit.MILLISECONDS.sleep(10);\n        try {\n            c = new Connection(self, connectionPolicy);\n            // this should have failed!\n            fail();\n            c.close();\n        } catch (final Exception e) {\n        }\n        assertEquals(2, remote.getFailureCount());\n        d2 = remote.getLastFailedConnect();\n        assertNotNull(d2);\n        assertTrue(d0.getTime() < d2.getTime());\n        assertTrue(d1.getTime() < d2.getTime());\n        TimeUnit.MILLISECONDS.sleep(10);\n\n        // unknown host name:\n        self = new OtpSelf(\"testFailedConnection@\" + ConnectionFactory.getLocalhostName(),\n                ConnectionFactory.getInstance().getCookie());\n        remote = new PeerNode(scalarisNode + \"noneExistingHost\");\n        connectionPolicy = new DefaultConnectionPolicy(remote);\n        connectionPolicy.setMaxRetries(0);\n        try {\n            c = new Connection(self, connectionPolicy);\n            // this should have failed!\n            fail();\n            c.close();\n        } catch (final Exception e) {\n        }\n        assertEquals(1, remote.getFailureCount());\n        d1 = remote.getLastFailedConnect();\n        assertNotNull(d1);\n        assertTrue(d0.getTime() < d1.getTime());\n        TimeUnit.MILLISECONDS.sleep(10);\n        try {\n            c = new Connection(self, connectionPolicy);\n            // this should have failed!\n            fail();\n            c.close();\n        } catch (final Exception e) {\n        }\n        assertEquals(2, remote.getFailureCount());\n        d2 = remote.getLastFailedConnect();\n        assertNotNull(d2);\n        assertTrue(d0.getTime() < d2.getTime());\n        assertTrue(d1.getTime() < d2.getTime());\n        TimeUnit.MILLISECONDS.sleep(10);\n\n        // non-existing node name:\n        self = new OtpSelf(\"testFailedConnection@\" + ConnectionFactory.getLocalhostName(),\n                ConnectionFactory.getInstance().getCookie());\n        remote = new PeerNode(\"noneExistingNode\" + scalarisNode);\n        connectionPolicy = new DefaultConnectionPolicy(remote);\n        connectionPolicy.setMaxRetries(0);\n        try {\n            c = new Connection(self, connectionPolicy);\n            // this should have failed!\n            fail();\n            c.close();\n        } catch (final Exception e) {\n        }\n        assertEquals(1, remote.getFailureCount());\n        d1 = remote.getLastFailedConnect();\n        assertNotNull(d1);\n        assertTrue(d0.getTime() < d1.getTime());\n        TimeUnit.MILLISECONDS.sleep(10);\n        try {\n            c = new Connection(self, connectionPolicy);\n            // this should have failed!\n            fail();\n            c.close();\n        } catch (final Exception e) {\n        }\n        assertEquals(2, remote.getFailureCount());\n        d2 = remote.getLastFailedConnect();\n        assertNotNull(d2);\n        assertTrue(d0.getTime() < d2.getTime());\n        assertTrue(d1.getTime() < d2.getTime());\n        TimeUnit.MILLISECONDS.sleep(10);\n    }\n\n    /**\n     * Test method for\n     * {@link Connection#doRPC(String, String, OtpErlangList)}.\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     * @throws IOException\n     *             if the connection is not active or a communication error\n     *             occurs\n     * @throws OtpErlangExit\n     *             if an exit signal is received from a process on the peer node\n     * @throws OtpAuthException\n     *             if the remote node sends a message containing an invalid\n     *             cookie\n     * @throws OtpErlangRangeException\n     *             if the value is too large to be represented as an int\n     */\n    @Test\n    public final void testDoRPCStringStringOtpErlangList()\n            throws ConnectionException, OtpErlangExit, OtpAuthException,\n            IOException, OtpErlangRangeException {\n        final OtpSelf self = new OtpSelf(\"testDoRPCStringStringOtpErlangList@\" + ConnectionFactory.getLocalhostName(),\n                ConnectionFactory.getInstance().getCookie());\n        final PeerNode remote = new PeerNode(scalarisNode);\n        final Connection c = new Connection(self, remote);\n\n        final OtpErlangObject raw_result = c.doRPC(\"lists\", \"sum\", new OtpErlangList(\n                new OtpErlangList(new OtpErlangObject[] { new OtpErlangInt(1),\n                        new OtpErlangInt(2), new OtpErlangInt(3) })));\n        final OtpErlangLong result = (OtpErlangLong) raw_result;\n\n        assertEquals(6, result.intValue());\n        assertNotNull(remote.getLastConnectSuccess());\n        assertEquals(0, remote.getFailureCount());\n        assertNull(remote.getLastFailedConnect());\n\n        c.close();\n    }\n\n    /**\n     * Test method for\n     * {@link Connection#doRPC(String, String, OtpErlangList)}.\n     *\n     * Closes the connection before doing the RPC which thus fails. Evaluates\n     * the statistics of the {@link PeerNode} object.\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     * @throws IOException\n     *             if the connection is not active or a communication error\n     *             occurs\n     * @throws OtpErlangExit\n     *             if an exit signal is received from a process on the peer node\n     * @throws OtpAuthException\n     *             if the remote node sends a message containing an invalid\n     *             cookie\n     * @throws OtpErlangRangeException\n     *             if the value is too large to be represented as an int\n     * @throws InterruptedException if the sleep is interrupted\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testDoRPCStringStringOtpErlangList_fail()\n            throws ConnectionException, OtpErlangExit, OtpAuthException,\n            IOException, OtpErlangRangeException, InterruptedException {\n        final OtpSelf self = new OtpSelf(\"testDoRPCStringStringOtpErlangList@\" + ConnectionFactory.getLocalhostName(),\n                ConnectionFactory.getInstance().getCookie());\n        final PeerNode remote = new PeerNode(scalarisNode);\n        final DefaultConnectionPolicy connectionPolicy = new DefaultConnectionPolicy(remote);\n        connectionPolicy.setMaxRetries(0);\n        TimeUnit.MILLISECONDS.sleep(10);\n        final Connection c = new Connection(self, connectionPolicy);\n\n        c.close();\n\n        try {\n            c.doRPC(\"lists\", \"sum\",\n                    new OtpErlangList(new OtpErlangList(new OtpErlangObject[] {\n                            new OtpErlangInt(1), new OtpErlangInt(2),\n                            new OtpErlangInt(3) })));\n            c.close();\n            // this should have failed!\n            fail();\n        } catch (final ConnectionException e) {\n            assertEquals(0, remote.getFailureCount());\n            assertNull(remote.getLastFailedConnect());\n            throw e;\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link Connection#doRPC(String, String, OtpErlangObject[])}\n     * .\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     * @throws IOException\n     *             if the connection is not active or a communication error\n     *             occurs\n     * @throws OtpErlangExit\n     *             if an exit signal is received from a process on the peer node\n     * @throws OtpAuthException\n     *             if the remote node sends a message containing an invalid\n     *             cookie\n     * @throws OtpErlangRangeException\n     *             if the value is too large to be represented as an int\n     */\n    @Test\n    public final void testDoRPCStringStringOtpErlangObjectArray()\n            throws ConnectionException, OtpErlangExit, OtpAuthException,\n            IOException, OtpErlangRangeException {\n        final OtpSelf self = new OtpSelf(\"testDoRPCStringStringOtpErlangObjectArray@\" + ConnectionFactory.getLocalhostName(),\n                ConnectionFactory.getInstance().getCookie());\n        final PeerNode remote = new PeerNode(scalarisNode);\n        final Connection c = new Connection(self, remote);\n\n        final OtpErlangObject raw_result = c.doRPC(\"lists\", \"sum\",\n                new OtpErlangObject[] { new OtpErlangList(\n                        new OtpErlangObject[] { new OtpErlangInt(1),\n                                new OtpErlangInt(2), new OtpErlangInt(3) }) });\n        final OtpErlangLong result = (OtpErlangLong) raw_result;\n\n        assertEquals(6, result.intValue());\n        assertNotNull(remote.getLastConnectSuccess());\n        assertEquals(0, remote.getFailureCount());\n        assertNull(remote.getLastFailedConnect());\n\n        c.close();\n    }\n\n    /**\n     * Test method for\n     * {@link Connection#doRPC(String, String, OtpErlangObject[])}.\n     *\n     * Closes the connection before doing the RPC which thus fails. Evaluates\n     * the statistics of the {@link PeerNode} object.\n     *\n     * @throws ConnectionException\n     *             if the connection fails\n     * @throws IOException\n     *             if the connection is not active or a communication error\n     *             occurs\n     * @throws OtpErlangExit\n     *             if an exit signal is received from a process on the peer node\n     * @throws OtpAuthException\n     *             if the remote node sends a message containing an invalid\n     *             cookie\n     * @throws OtpErlangRangeException\n     *             if the value is too large to be represented as an int\n     * @throws InterruptedException if the sleep is interrupted\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testDoRPCStringStringOtpErlangObjectArray_fail()\n            throws ConnectionException, OtpErlangExit, OtpAuthException,\n            IOException, OtpErlangRangeException, InterruptedException {\n        final OtpSelf self = new OtpSelf(\"testDoRPCStringStringOtpErlangList@\" + ConnectionFactory.getLocalhostName(),\n                ConnectionFactory.getInstance().getCookie());\n        final PeerNode remote = new PeerNode(scalarisNode);\n        final DefaultConnectionPolicy connectionPolicy = new DefaultConnectionPolicy(remote);\n        connectionPolicy.setMaxRetries(0);\n        TimeUnit.MILLISECONDS.sleep(10);\n        final Connection c = new Connection(self, connectionPolicy);\n\n        c.close();\n\n        try {\n            c.doRPC(\"lists\", \"sum\",\n                    new OtpErlangObject[] { new OtpErlangList(\n                            new OtpErlangObject[] { new OtpErlangInt(1),\n                                    new OtpErlangInt(2), new OtpErlangInt(3) }) });\n            c.close();\n            // this should have failed!\n            fail();\n        } catch (final ConnectionException e) {\n            assertEquals(0, remote.getFailureCount());\n            assertNull(remote.getLastFailedConnect());\n            throw e;\n        }\n    }\n\n}\n"
  },
  {
    "path": "java-api/test/de/zib/scalaris/DefaultConnectionPolicyTest.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport static org.junit.Assert.assertEquals;\nimport static org.junit.Assert.assertFalse;\nimport static org.junit.Assert.assertNotNull;\nimport static org.junit.Assert.assertTrue;\nimport static org.junit.Assert.fail;\n\nimport java.util.ArrayList;\nimport java.util.Date;\nimport java.util.List;\nimport java.util.concurrent.TimeUnit;\n\nimport org.junit.Test;\n\n/**\n * Test cases for the {@link DefaultConnectionPolicy} class.\n *\n * @author Nico Kruber, kruber@zib.de\n *\n * @version 2.3\n * @since 2.3\n */\npublic class DefaultConnectionPolicyTest {\n    /**\n     * Test method for {@link DefaultConnectionPolicy#DefaultConnectionPolicy(PeerNode)}.\n     */\n    @Test\n    public final void testDefaultConnectionPolicyPeerNode() {\n        PeerNode remote;\n        DefaultConnectionPolicy p;\n        List<PeerNode> goodNodes, badNodes;\n\n        remote = new PeerNode(\"test@localhost\");\n        p = new DefaultConnectionPolicy(remote);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertTrue(goodNodes.contains(remote));\n        assertEquals(0, badNodes.size());\n        assertEquals(1, goodNodes.size());\n\n        remote = new PeerNode(\"test@localhost\");\n        remote.setLastConnectSuccess();\n        p = new DefaultConnectionPolicy(remote);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertTrue(goodNodes.contains(remote));\n        assertEquals(0, badNodes.size());\n        assertEquals(1, goodNodes.size());\n\n        remote = new PeerNode(\"test@localhost\");\n        remote.setLastFailedConnect();\n        p = new DefaultConnectionPolicy(remote);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertTrue(badNodes.contains(remote));\n        assertEquals(0, goodNodes.size());\n        assertEquals(1, badNodes.size());\n    }\n\n    /**\n     * Test method for {@link DefaultConnectionPolicy#DefaultConnectionPolicy(List)}.\n     * @throws InterruptedException if the sleep is interrupted\n     */\n    @Test\n    public final void testDefaultConnectionPolicyListOfPeerNode() throws InterruptedException {\n        final List<PeerNode> remotes = new ArrayList<PeerNode>();\n        DefaultConnectionPolicy p;\n        PeerNode p1, p2, p3;\n        List<PeerNode> goodNodes, badNodes;\n\n        // single node list:\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        remotes.add(p1);\n        p = new DefaultConnectionPolicy(remotes);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertTrue(goodNodes.contains(p1));\n        assertEquals(0, badNodes.size());\n        assertEquals(1, goodNodes.size());\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p1.setLastConnectSuccess();\n        remotes.add(p1);\n        p = new DefaultConnectionPolicy(remotes);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertTrue(goodNodes.contains(p1));\n        assertEquals(0, badNodes.size());\n        assertEquals(1, goodNodes.size());\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p1.setLastFailedConnect();\n        remotes.add(p1);\n        p = new DefaultConnectionPolicy(remotes);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertTrue(badNodes.contains(p1));\n        assertEquals(1, badNodes.size());\n        assertEquals(0, goodNodes.size());\n\n        // more nodes:\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertTrue(goodNodes.contains(p1));\n        assertTrue(goodNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(1, badNodes.size());\n        assertEquals(2, goodNodes.size());\n\n        // different order in the list should yield to the same result:\n        remotes.clear();\n        remotes.add(p2);\n        remotes.add(p3);\n        remotes.add(p1);\n        p = new DefaultConnectionPolicy(remotes);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertTrue(goodNodes.contains(p1));\n        assertTrue(goodNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(1, badNodes.size());\n        assertEquals(2, goodNodes.size());\n\n        // try more failed nodes, also check order:\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertTrue(goodNodes.contains(p1));\n        assertTrue(badNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(2, badNodes.size());\n        assertEquals(1, goodNodes.size());\n        assertEquals(p2, badNodes.get(0));\n\n        // all failed nodes, also check order:\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p1.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        p2.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertTrue(badNodes.contains(p1));\n        assertTrue(badNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(3, badNodes.size());\n        assertEquals(0, goodNodes.size());\n        assertEquals(p1, badNodes.get(0));\n        assertEquals(p3, badNodes.get(badNodes.size() - 1));\n\n        // different order in the list should yield to the same result:\n        remotes.clear();\n        remotes.add(p2);\n        remotes.add(p3);\n        remotes.add(p1);\n        p = new DefaultConnectionPolicy(remotes);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertTrue(badNodes.contains(p1));\n        assertTrue(badNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(3, badNodes.size());\n        assertEquals(0, goodNodes.size());\n        assertEquals(p1, badNodes.get(0));\n        assertEquals(p3, badNodes.get(badNodes.size() - 1));\n\n        // all failed nodes at the same time, also check order:\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p1.setLastFailedConnect();\n        p2.setLastFailedConnect();\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertTrue(badNodes.contains(p1));\n        assertTrue(badNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(3, badNodes.size());\n        assertEquals(0, goodNodes.size());\n        // the actual order now depends on the nodes' hash codes and can't be\n        // checked here\n//        assertEquals(p1, badNodes.get(0));\n//        assertEquals(p3, badNodes.get(badNodes.size() - 1));\n    }\n\n    /**\n     * Test method for {@link DefaultConnectionPolicy#availableNodeAdded(PeerNode)}.\n     */\n    @Test\n    public final void testAvailableNodeAdded() {\n        final PeerNode remote = new PeerNode(\"test@localhost\");\n        DefaultConnectionPolicy p;\n        PeerNode p1, p2;\n\n        p = new DefaultConnectionPolicy(remote);\n        p1 = new PeerNode(\"test1@localhost\");\n        p.availableNodeAdded(p1);\n        assertTrue(p.getGoodNodes().contains(remote));\n        assertTrue(p.getGoodNodes().contains(p1));\n        assertEquals(0, p.getBadNodes().size());\n        assertEquals(2, p.getGoodNodes().size());\n\n        p = new DefaultConnectionPolicy(remote);\n        p1 = new PeerNode(\"test1@localhost\");\n        p1.setLastConnectSuccess();\n        p.availableNodeAdded(p1);\n        assertTrue(p.getGoodNodes().contains(remote));\n        assertTrue(p.getGoodNodes().contains(p1));\n        assertEquals(0, p.getBadNodes().size());\n        assertEquals(2, p.getGoodNodes().size());\n\n        p = new DefaultConnectionPolicy(remote);\n        p1 = new PeerNode(\"test1@localhost\");\n        p1.setLastFailedConnect();\n        p.availableNodeAdded(p1);\n        assertTrue(p.getGoodNodes().contains(remote));\n        assertTrue(p.getBadNodes().contains(p1));\n        assertEquals(1, p.getBadNodes().size());\n        assertEquals(1, p.getGoodNodes().size());\n\n        p = new DefaultConnectionPolicy(remote);\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p.availableNodeAdded(p1);\n        p.availableNodeAdded(p2);\n        assertTrue(p.getGoodNodes().contains(remote));\n        assertTrue(p.getGoodNodes().contains(p1));\n        assertTrue(p.getGoodNodes().contains(p2));\n        assertEquals(0, p.getBadNodes().size());\n        assertEquals(3, p.getGoodNodes().size());\n    }\n\n    /**\n     * Test method for {@link DefaultConnectionPolicy#availableNodeRemoved(PeerNode)}.\n     */\n    @Test\n    public final void testAvailableNodeRemoved() {\n        final List<PeerNode> remotes = new ArrayList<PeerNode>();\n        DefaultConnectionPolicy p;\n        PeerNode p1, p2, p3;\n        List<PeerNode> goodNodes, badNodes;\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n\n        p.availableNodeRemoved(p1);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertFalse(goodNodes.contains(p1));\n        assertTrue(goodNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(1, badNodes.size());\n        assertEquals(1, goodNodes.size());\n\n        p.availableNodeRemoved(p2);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertFalse(goodNodes.contains(p1));\n        assertFalse(goodNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(1, badNodes.size());\n        assertEquals(0, goodNodes.size());\n\n        p.availableNodeRemoved(p3);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertFalse(goodNodes.contains(p1));\n        assertFalse(goodNodes.contains(p2));\n        assertFalse(badNodes.contains(p3));\n        assertEquals(0, badNodes.size());\n        assertEquals(0, goodNodes.size());\n    }\n\n    /**\n     * Test method for {@link DefaultConnectionPolicy#availableNodesReset()}.\n     */\n    @Test\n    public final void testAvailableNodesReset() {\n        final List<PeerNode> remotes = new ArrayList<PeerNode>();\n        DefaultConnectionPolicy p;\n        PeerNode p1, p2, p3;\n        List<PeerNode> goodNodes, badNodes;\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n\n        p.availableNodesReset();\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertEquals(0, badNodes.size());\n        assertEquals(0, goodNodes.size());\n    }\n\n    /**\n     * Test method for {@link DefaultConnectionPolicy#nodeFailed(PeerNode)}.\n     * @throws InterruptedException  if the sleep is interrupted\n     */\n    @Test\n    public final void testNodeFailed() throws InterruptedException {\n        final List<PeerNode> remotes = new ArrayList<PeerNode>();\n        DefaultConnectionPolicy p;\n        PeerNode p1, p2, p3;\n        List<PeerNode> goodNodes, badNodes;\n\n        // without time:\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        p.nodeFailed(p1);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertEquals(1, p1.getFailureCount());\n        assertEquals(0, p2.getFailureCount());\n        assertEquals(1, p3.getFailureCount());\n        assertTrue(badNodes.contains(p1));\n        assertTrue(goodNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(2, badNodes.size());\n        assertEquals(1, goodNodes.size());\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        p.nodeFailed(p2);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertEquals(0, p1.getFailureCount());\n        assertEquals(1, p2.getFailureCount());\n        assertEquals(1, p3.getFailureCount());\n        assertTrue(goodNodes.contains(p1));\n        assertTrue(badNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(2, badNodes.size());\n        assertEquals(1, goodNodes.size());\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        p.nodeFailed(p3);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertEquals(0, p1.getFailureCount());\n        assertEquals(0, p2.getFailureCount());\n        assertEquals(2, p3.getFailureCount());\n        assertTrue(goodNodes.contains(p1));\n        assertTrue(goodNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(1, badNodes.size());\n        assertEquals(2, goodNodes.size());\n\n        // with time:\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        p.nodeFailed(p1);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertEquals(1, p1.getFailureCount());\n        assertEquals(0, p2.getFailureCount());\n        assertEquals(1, p3.getFailureCount());\n        assertTrue(badNodes.contains(p1));\n        assertTrue(goodNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(2, badNodes.size());\n        assertEquals(1, goodNodes.size());\n        assertEquals(p3, badNodes.get(0));\n        assertEquals(p1, badNodes.get(badNodes.size() - 1));\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        p.nodeFailed(p2);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertEquals(0, p1.getFailureCount());\n        assertEquals(1, p2.getFailureCount());\n        assertEquals(1, p3.getFailureCount());\n        assertTrue(goodNodes.contains(p1));\n        assertTrue(badNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(2, badNodes.size());\n        assertEquals(1, goodNodes.size());\n        assertEquals(p3, badNodes.get(0));\n        assertEquals(p2, badNodes.get(badNodes.size() - 1));\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        p.nodeFailed(p3);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertEquals(0, p1.getFailureCount());\n        assertEquals(0, p2.getFailureCount());\n        assertEquals(2, p3.getFailureCount());\n        assertTrue(goodNodes.contains(p1));\n        assertTrue(goodNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(1, badNodes.size());\n        assertEquals(2, goodNodes.size());\n        assertEquals(p3, badNodes.get(0));\n    }\n\n    /**\n     * Test method for {@link DefaultConnectionPolicy#nodeFailReset(PeerNode)}.\n     */\n    @Test\n    public final void testnodeFailReset() {\n        final List<PeerNode> remotes = new ArrayList<PeerNode>();\n        DefaultConnectionPolicy p;\n        PeerNode p1, p2, p3;\n        List<PeerNode> goodNodes, badNodes;\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        p.nodeFailReset(p1);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertEquals(0, p1.getFailureCount());\n        assertEquals(0, p2.getFailureCount());\n        assertEquals(1, p3.getFailureCount());\n        assertTrue(goodNodes.contains(p1));\n        assertTrue(goodNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(1, badNodes.size());\n        assertEquals(2, goodNodes.size());\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        p.nodeFailReset(p2);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertEquals(0, p1.getFailureCount());\n        assertEquals(0, p2.getFailureCount());\n        assertEquals(1, p3.getFailureCount());\n        assertTrue(goodNodes.contains(p1));\n        assertTrue(goodNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(1, badNodes.size());\n        assertEquals(2, goodNodes.size());\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        p.nodeFailReset(p3);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertEquals(0, p1.getFailureCount());\n        assertEquals(0, p2.getFailureCount());\n        assertEquals(0, p3.getFailureCount());\n        assertTrue(goodNodes.contains(p1));\n        assertTrue(goodNodes.contains(p2));\n        assertTrue(goodNodes.contains(p3));\n        assertEquals(0, badNodes.size());\n        assertEquals(3, goodNodes.size());\n    }\n\n    /**\n     * Test method for {@link DefaultConnectionPolicy#nodeConnectSuccess(PeerNode)}.\n     * @throws InterruptedException if the sleep is interrupted\n     */\n    @Test\n    public final void testNodeConnectSuccess() throws InterruptedException {\n        final List<PeerNode> remotes = new ArrayList<PeerNode>();\n        DefaultConnectionPolicy p;\n        PeerNode p1, p2, p3;\n        List<PeerNode> goodNodes, badNodes;\n        Date d0, d2;\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        d0 = new Date();\n        TimeUnit.MILLISECONDS.sleep(10);\n        p = new DefaultConnectionPolicy(remotes);\n        p.nodeConnectSuccess(p1);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertEquals(0, p1.getFailureCount());\n        assertEquals(0, p2.getFailureCount());\n        assertEquals(1, p3.getFailureCount());\n        assertTrue(goodNodes.contains(p1));\n        assertTrue(goodNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(1, badNodes.size());\n        assertEquals(2, goodNodes.size());\n        assertNotNull(p1.getLastConnectSuccess());\n        assertTrue(d0.getTime() < p1.getLastConnectSuccess().getTime());\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        d0 = new Date();\n        TimeUnit.MILLISECONDS.sleep(10);\n        d2 = p2.getLastConnectSuccess();\n        TimeUnit.MILLISECONDS.sleep(10);\n        p = new DefaultConnectionPolicy(remotes);\n        p.nodeConnectSuccess(p2);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertEquals(0, p1.getFailureCount());\n        assertEquals(0, p2.getFailureCount());\n        assertEquals(1, p3.getFailureCount());\n        assertTrue(goodNodes.contains(p1));\n        assertTrue(goodNodes.contains(p2));\n        assertTrue(badNodes.contains(p3));\n        assertEquals(1, badNodes.size());\n        assertEquals(2, goodNodes.size());\n        assertNotNull(p2.getLastConnectSuccess());\n        assertTrue(d0.getTime() < p2.getLastConnectSuccess().getTime());\n        assertTrue(d2.getTime() < p2.getLastConnectSuccess().getTime());\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        p3.setLastFailedConnect();\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        d0 = new Date();\n        TimeUnit.MILLISECONDS.sleep(10);\n        p = new DefaultConnectionPolicy(remotes);\n        p.nodeConnectSuccess(p3);\n        goodNodes = p.getGoodNodes();\n        badNodes = p.getBadNodes();\n        assertEquals(0, p1.getFailureCount());\n        assertEquals(0, p2.getFailureCount());\n        assertEquals(0, p3.getFailureCount());\n        assertTrue(goodNodes.contains(p1));\n        assertTrue(goodNodes.contains(p2));\n        assertTrue(goodNodes.contains(p3));\n        assertEquals(0, badNodes.size());\n        assertEquals(3, goodNodes.size());\n        assertNotNull(p3.getLastConnectSuccess());\n        assertTrue(d0.getTime() < p3.getLastConnectSuccess().getTime());\n    }\n\n    /**\n     * Test method for {@link DefaultConnectionPolicy#selectNode()}.\n     * @throws InterruptedException if the sleep is interrupted\n     */\n    @Test\n    public final void testSelectNode() throws InterruptedException {\n        final List<PeerNode> remotes = new ArrayList<PeerNode>();\n        DefaultConnectionPolicy p;\n        PeerNode p1, p2, p3;\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p1.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        p2.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        p3.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        p.setMaxRetries(0);\n\n        // cycling through bad nodes:\n        try {\n            assertEquals(p1, p.selectNode());\n            assertEquals(p1, p.selectNode());\n        } catch (final Exception e) {\n            fail();\n        }\n\n        try {\n            assertEquals(p1, p.selectNode());\n            p.nodeFailed(p1);\n            TimeUnit.MILLISECONDS.sleep(10);\n            assertEquals(p2, p.selectNode());\n            p.nodeFailed(p2);\n            TimeUnit.MILLISECONDS.sleep(10);\n            assertEquals(p3, p.selectNode());\n            p.nodeFailed(p3);\n            TimeUnit.MILLISECONDS.sleep(10);\n            assertEquals(p1, p.selectNode());\n        } catch (final RuntimeException e) {\n            fail();\n        }\n\n        // getting random good nodes:\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        TimeUnit.MILLISECONDS.sleep(10);\n        p3.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        p.setMaxRetries(0);\n        int p1Found = 0;\n        int p2Found = 0;\n        int p3Found = 0;\n\n        try {\n            // get 100 random nodes:\n            for (int i = 0; i < 100; ++i) {\n                final PeerNode n = p.selectNode();\n                if (n == p1) {\n                    ++p1Found;\n                }\n                if (n == p2) {\n                    ++p2Found;\n                }\n                if (n == p3) {\n                    ++p3Found;\n                }\n            }\n            assertTrue(p1Found > 0);\n            assertTrue(p2Found > 0);\n            assertTrue(p3Found == 0);\n            // this may fail but the uniform random number distribution should\n            // at result in p1 and p2 to be found 40 times:\n            if ((p1Found < 40) || (p2Found < 40)) {\n                System.err\n                        .println(\"Warning: DefaultConnectionPolicyTest::testSelectNodeIntPeerNodeE(): 100 selects, p1=\"\n                                + p1Found + \", p2=\" + p2Found);\n            }\n        } catch (final Exception e) {\n            fail();\n        }\n    }\n\n    /**\n     * Test method for {@link DefaultConnectionPolicy#selectNode(int, PeerNode, Exception)}.\n     * @throws InterruptedException if the sleep is interrupted\n     */\n    @Test\n    public final void testSelectNodeIntPeerNodeE() throws InterruptedException {\n        final List<PeerNode> remotes = new ArrayList<PeerNode>();\n        DefaultConnectionPolicy p;\n        PeerNode p1, p2, p3;\n        final PeerNode failedNode = new PeerNode(\"failedNode@localhost\");\n\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p1.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        p2.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        p3.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        p.setMaxRetries(0);\n\n        // cycling through bad nodes:\n        try {\n            assertEquals(p1, p.selectNode(0, failedNode, new Exception()));\n            assertEquals(p1, p.selectNode(0, failedNode, new Exception()));\n        } catch (final Exception e) {\n            fail();\n        }\n\n        try {\n            assertEquals(p1, p.selectNode(0, failedNode, new Exception()));\n            p.nodeFailed(p1);\n            TimeUnit.MILLISECONDS.sleep(10);\n            assertEquals(p2, p.selectNode(0, failedNode, new Exception()));\n            p.nodeFailed(p2);\n            TimeUnit.MILLISECONDS.sleep(10);\n            assertEquals(p3, p.selectNode(0, failedNode, new Exception()));\n            p.nodeFailed(p3);\n            TimeUnit.MILLISECONDS.sleep(10);\n            assertEquals(p1, p.selectNode(0, failedNode, new Exception()));\n        } catch (final Exception e) {\n            fail();\n        }\n\n        // getting random good nodes:\n        remotes.clear();\n        p1 = new PeerNode(\"test1@localhost\");\n        p2 = new PeerNode(\"test2@localhost\");\n        p3 = new PeerNode(\"test3@localhost\");\n        p2.setLastConnectSuccess();\n        TimeUnit.MILLISECONDS.sleep(10);\n        p3.setLastFailedConnect();\n        TimeUnit.MILLISECONDS.sleep(10);\n        remotes.add(p1);\n        remotes.add(p2);\n        remotes.add(p3);\n        p = new DefaultConnectionPolicy(remotes);\n        p.setMaxRetries(0);\n        int p1Found = 0;\n        int p2Found = 0;\n        int p3Found = 0;\n\n        try {\n            // get 100 random nodes:\n            for (int i = 0; i < 100; ++i) {\n                final PeerNode n = p.selectNode(0, failedNode, new Exception());\n                if (n == p1) {\n                    ++p1Found;\n                }\n                if (n == p2) {\n                    ++p2Found;\n                }\n                if (n == p3) {\n                    ++p3Found;\n                }\n            }\n            assertTrue(p1Found > 0);\n            assertTrue(p2Found > 0);\n            assertTrue(p3Found == 0);\n            // this may fail but the uniform random number distribution should\n            // at result in p1 and p2 to be found 40 times:\n            if ((p1Found < 40) || (p2Found < 40)) {\n                System.err\n                        .println(\"Warning: DefaultConnectionPolicyTest::testSelectNodeIntPeerNodeE(): 100 selects, p1=\"\n                                + p1Found + \", p2=\" + p2Found);\n            }\n        } catch (final Exception e) {\n            fail();\n        }\n    }\n\n    /**\n     * Test method for {@link DefaultConnectionPolicy#getMaxRetries()}.\n     */\n    @Test\n    public final void testGetMaxRetries() {\n        final DefaultConnectionPolicy p = new DefaultConnectionPolicy(new PeerNode(\"test@localhost\"));\n        assertEquals(3, p.getMaxRetries());\n        p.setMaxRetries(5);\n        assertEquals(5, p.getMaxRetries());\n        p.setMaxRetries(1);\n        assertEquals(1, p.getMaxRetries());\n        p.setMaxRetries(0);\n        assertEquals(0, p.getMaxRetries());\n    }\n\n    /**\n     * Test method for {@link DefaultConnectionPolicy#setMaxRetries(int)}.\n     *\n     * Tries to set 3, 5, 1 and 0 retries and checks whether\n     * {@link DefaultConnectionPolicy#selectNode(int, PeerNode, Exception)}\n     * performs this many retries.\n     */\n    @Test\n    public final void testSetMaxRetries() {\n        PeerNode remote = new PeerNode(\"test@localhost\");\n        final DefaultConnectionPolicy p = new DefaultConnectionPolicy(remote);\n        boolean exceptionThrown = false;\n\n        ///// 3 retries:\n        do {\n            p.setMaxRetries(3);\n\n            try {\n                remote = p.selectNode(0, remote, new Exception());\n            } catch (final Exception e) {\n                fail();\n            }\n\n            try {\n                remote = p.selectNode(2, remote, new Exception());\n            } catch (final Exception e) {\n                fail();\n            }\n\n            try {\n                remote = p.selectNode(3, remote, new Exception());\n            } catch (final Exception e) {\n                fail();\n            }\n\n            exceptionThrown = false;\n            try {\n                remote = p.selectNode(4, remote, new Exception());\n            } catch (final Exception e) {\n                exceptionThrown = true;\n            }\n            assertTrue(exceptionThrown);\n\n            exceptionThrown = false;\n            try {\n                remote = p.selectNode(5, remote, new Exception());\n            } catch (final Exception e) {\n                exceptionThrown = true;\n            }\n            assertTrue(exceptionThrown);\n\n            int retry = 0;\n            try {\n                for (retry = 0; retry < 10; ++retry) {\n                    remote = p.selectNode(retry, remote, new Exception());\n    //                p.nodeFailed(remote);\n                }\n            } catch (final Exception e) {\n            }\n            assertEquals(4, retry); // the 4th retry has not been performed\n                                    // though\n        } while (false);\n\n        ///// 5 retries:\n        do {\n            p.setMaxRetries(5);\n\n            try {\n                remote = p.selectNode(0, remote, new Exception());\n            } catch (final Exception e) {\n                fail();\n            }\n\n            try {\n                remote = p.selectNode(2, remote, new Exception());\n            } catch (final Exception e) {\n                fail();\n            }\n\n            try {\n                remote = p.selectNode(5, remote, new Exception());\n            } catch (final Exception e) {\n                fail();\n            }\n\n            exceptionThrown = false;\n            try {\n                remote = p.selectNode(6, remote, new Exception());\n            } catch (final Exception e) {\n                exceptionThrown = true;\n            }\n            assertTrue(exceptionThrown);\n\n            exceptionThrown = false;\n            try {\n                remote = p.selectNode(7, remote, new Exception());\n            } catch (final Exception e) {\n                exceptionThrown = true;\n            }\n            assertTrue(exceptionThrown);\n\n            int retry = 0;\n            try {\n                for (retry = 0; retry < 10; ++retry) {\n                    remote = p.selectNode(retry, remote, new Exception());\n                    // p.nodeFailed(remote);\n                }\n            } catch (final Exception e) {\n            }\n            assertEquals(6, retry); // the 6th retry has not been performed\n                                    // though\n        } while (false);\n\n        ///// 1 retry:\n        do {\n            p.setMaxRetries(1);\n\n            try {\n                remote = p.selectNode(0, remote, new Exception());\n            } catch (final Exception e) {\n                fail();\n            }\n\n            try {\n                remote = p.selectNode(1, remote, new Exception());\n            } catch (final Exception e) {\n                fail();\n            }\n\n            exceptionThrown = false;\n            try {\n                remote = p.selectNode(2, remote, new Exception());\n            } catch (final Exception e) {\n                exceptionThrown = true;\n            }\n            assertTrue(exceptionThrown);\n\n            exceptionThrown = false;\n            try {\n                remote = p.selectNode(3, remote, new Exception());\n            } catch (final Exception e) {\n                exceptionThrown = true;\n            }\n            assertTrue(exceptionThrown);\n\n            int retry = 0;\n            try {\n                for (retry = 0; retry < 10; ++retry) {\n                    remote = p.selectNode(retry, remote, new Exception());\n                    // p.nodeFailed(remote);\n                }\n            } catch (final Exception e) {\n            }\n            assertEquals(2, retry); // the 2nd retry has not been performed\n                                    // though\n        } while (false);\n\n        ///// 0 retries:\n        do {\n            p.setMaxRetries(0);\n\n            try {\n                remote = p.selectNode(0, remote, new Exception());\n            } catch (final Exception e) {\n                fail();\n            }\n\n            exceptionThrown = false;\n            try {\n                remote = p.selectNode(1, remote, new Exception());\n            } catch (final Exception e) {\n                exceptionThrown = true;\n            }\n            assertTrue(exceptionThrown);\n\n            exceptionThrown = false;\n            try {\n                remote = p.selectNode(2, remote, new Exception());\n            } catch (final Exception e) {\n                exceptionThrown = true;\n            }\n            assertTrue(exceptionThrown);\n\n            int retry = 0;\n            try {\n                for (retry = 0; retry < 10; ++retry) {\n                    remote = p.selectNode(retry, remote, new Exception());\n                    // p.nodeFailed(remote);\n                }\n            } catch (final Exception e) {\n            }\n            assertEquals(1, retry); // the 1st retry has not been performed\n                                    // though\n        } while (false);\n    }\n\n}\n"
  },
  {
    "path": "java-api/test/de/zib/scalaris/ErlangValueTest.java",
    "content": "/**\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport static org.junit.Assert.*;\n\nimport java.math.BigInteger;\nimport java.util.ArrayList;\nimport java.util.HashMap;\nimport java.util.LinkedHashMap;\nimport java.util.List;\nimport java.util.Map;\nimport java.util.Map.Entry;\nimport java.util.Random;\nimport java.util.LinkedList;\n\nimport org.junit.Test;\n\nimport com.ericsson.otp.erlang.OtpErlangBinary;\nimport com.ericsson.otp.erlang.OtpErlangBoolean;\nimport com.ericsson.otp.erlang.OtpErlangDouble;\nimport com.ericsson.otp.erlang.OtpErlangInt;\nimport com.ericsson.otp.erlang.OtpErlangList;\nimport com.ericsson.otp.erlang.OtpErlangLong;\nimport com.ericsson.otp.erlang.OtpErlangObject;\nimport com.ericsson.otp.erlang.OtpErlangString;\nimport com.ericsson.otp.erlang.OtpErlangTuple;\n\n/**\n * Unit tests for {@link ErlangValue}.\n *\n * TODO: implement tests verifying that the expected exceptions are thrown for\n * unsupported conversions\n *\n * @author Nico Kruber, kruber@zib.de\n */\npublic class ErlangValueTest {\n    private static BigInteger getRandomBigInt(final Random random) {\n        return BigInteger.valueOf(random.nextLong()).multiply(BigInteger.valueOf(random.nextLong()));\n    }\n\n    private static byte[] getRandomBytes(final Random random, final int size) {\n        final byte[] bytes = new byte[size];\n        random.nextBytes(bytes);\n        return bytes;\n    }\n\n    private static String getRandomString(final Random random, final int length, final boolean onlyChars) {\n        if (onlyChars) {\n            return getRandomCharString(random, length);\n        } else {\n            try {\n                return Benchmark.getRandom(length, String.class, random);\n            } catch (final Exception e) {\n                throw new RuntimeException(e);\n            }\n        }\n    }\n\n    private static final char[] chars = new char[26];\n    private static final char[] digits = new char[10];\n    private static final char[] symbols = new char[36];\n\n    static {\n        for (int i = 0; i < 26; ++i) {\n            chars[i] = (char) ('a' + i);\n            symbols[i] = (char) ('a' + i);\n        }\n        for (int i = 0; i < 10; ++i) {\n            digits[i] = (char) ('0' + i);\n            symbols[i + 26] = (char) ('0' + i);\n        }\n    }\n\n    private static String getRandomCharString(final Random random, final int length) {\n        if(length > 0) {\n            final char[] result = new char[length];\n            // lets always start with a character:\n            result[0] = chars[random.nextInt(chars.length)];\n            for (int i = 1; i < result.length; ++i) {\n                result[0] = symbols[random.nextInt(symbols.length)];\n            }\n            return new String(result);\n        }\n        return \"\";\n    }\n\n    private static List<Object> getRandomList(final Random random, final int capacity) {\n        // note: we do not generate (recursive) maps -> so there won't be keys to worry about\n        return getRandomListRecursive(random, capacity, 0, false);\n    }\n\n    private static List<Object> getRandomListRecursive(final Random random, final int capacity, final int maxDepth, final boolean mapKeyOnlyChars) {\n        List<Object> currentList = null;\n        currentList = new ArrayList<Object>(capacity);\n        final int curMaxDepth = maxDepth == 0 ? 0 : random.nextInt(maxDepth);\n        final int maxType = curMaxDepth == 0 ? 6 : 8;\n        switch (random.nextInt(maxType)) {\n            case 0: // bool\n                currentList.add(random.nextBoolean());\n                break;\n            case 1: // int\n                currentList.add(random.nextInt());\n                break;\n            case 2: // long\n                currentList.add(random.nextLong());\n                break;\n            case 3: // BigInteger\n                currentList.add(getRandomBigInt(random));\n                break;\n            case 4: // double\n                currentList.add(random.nextDouble());\n                break;\n            case 5: // String\n                currentList.add(getRandomString(random, random.nextInt(10), false));\n                break;\n            case 6: // List\n                currentList.add(getRandomListRecursive(random, capacity, curMaxDepth, mapKeyOnlyChars));\n                break;\n            case 7: // Map\n                currentList.add(getRandomMapRecursive(random, capacity, curMaxDepth, mapKeyOnlyChars));\n                break;\n            default:\n                throw new RuntimeException(\"unexpected random number\");\n        }\n        return currentList;\n    }\n\n    private static Map<String, Object> getRandomMapRecursive(final Random random, final int capacity, final int maxDepth, final boolean keyOnlyChars) {\n        Map<String, Object> currentMap = null;\n        currentMap = new LinkedHashMap<String, Object>(capacity);\n        for (int i = 0; i < capacity; ++i) {\n            // key:\n            final String key = getRandomString(random, random.nextInt(10), keyOnlyChars);\n            // value:\n            final int curMaxDepth = maxDepth == 0 ? 0 : random.nextInt(maxDepth);\n            final int maxType = curMaxDepth == 0 ? 6 : 8;\n            switch (random.nextInt(maxType)) {\n                case 0: // bool\n                    currentMap.put(key, random.nextBoolean());\n                    break;\n                case 1: // int\n                    currentMap.put(key, random.nextInt());\n                    break;\n                case 2: // long\n                    currentMap.put(key, random.nextLong());\n                    break;\n                case 3: // BigInteger\n                    currentMap.put(key, getRandomBigInt(random));\n                    break;\n                case 4: // double\n                    currentMap.put(key, random.nextDouble());\n                    break;\n                case 5: // String\n                    currentMap.put(key, getRandomString(random, random.nextInt(10), false));\n                    break;\n                case 6: // List\n                    currentMap.put(key, getRandomListRecursive(random, capacity, curMaxDepth, keyOnlyChars));\n                    break;\n                case 7: // Map\n                    currentMap.put(key, getRandomMapRecursive(random, capacity, curMaxDepth, keyOnlyChars));\n                    break;\n                default:\n                    throw new RuntimeException(\"unexpected random number\");\n            }\n        }\n        return currentMap;\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#boolValue()}.\n     */\n    @Test\n    public final void testBoolValue() {\n        final ErlangValue trueVal = new ErlangValue(true);\n        final ErlangValue trueValOtp = new ErlangValue(new OtpErlangBoolean(true));\n        final ErlangValue falseVal = new ErlangValue(false);\n        final ErlangValue falseValOtp = new ErlangValue(new OtpErlangBoolean(false));\n\n        assertEquals(true, trueVal.boolValue());\n        assertEquals(true, trueValOtp.boolValue());\n        assertEquals(false, falseVal.boolValue());\n        assertEquals(false, falseValOtp.boolValue());\n        assertEquals(trueVal, trueValOtp);\n        assertEquals(trueValOtp, trueVal);\n        assertEquals(falseVal, falseValOtp);\n        assertEquals(falseValOtp, falseVal);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#intValue()}.\n     *\n     * @throws Exception if a test with a random integer failed\n     */\n    @Test\n    public final void testIntValue() throws Exception {\n        final Random random = new Random();\n        for (int i = 0; i < 10000; ++i) {\n            Integer currentInt = null;\n            try {\n                currentInt = random.nextInt();\n                testIntValue(currentInt);\n            } catch (final ClassCastException e) {\n                throw new Exception(\"testIntValue(\" + currentInt + \") failed\", e);\n            }\n        }\n    }\n\n    private final void testIntValue(final int value) {\n        final ErlangValue eVal = new ErlangValue(value);\n        final ErlangValue eValOtp = new ErlangValue(new OtpErlangInt(value));\n\n        assertEquals(value, eVal.intValue());\n        assertEquals(value, eValOtp.intValue());\n        assertEquals(eVal, eValOtp);\n        assertEquals(eValOtp, eVal);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#longValue()}.\n     *\n     * @throws Exception if a test with a random long failed\n     */\n    @Test\n    public final void testLongValue() throws Exception {\n        final Random random = new Random();\n        for (int i = 0; i < 10000; ++i) {\n            Long currentLong = null;\n            try {\n                currentLong = random.nextLong();\n                testLongValue(currentLong);\n            } catch (final ClassCastException e) {\n                throw new Exception(\"testLongValue(\" + currentLong + \") failed\", e);\n            }\n        }\n    }\n\n    private final void testLongValue(final long value) {\n        final ErlangValue eVal = new ErlangValue(value);\n        final ErlangValue eValOtp = new ErlangValue(new OtpErlangLong(value));\n\n        assertEquals(value, eVal.longValue());\n        assertEquals(value, eValOtp.longValue());\n        assertEquals(eVal, eValOtp);\n        assertEquals(eValOtp, eVal);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#bigIntValue()}.\n     *\n     * @throws Exception if a test with a random big integer failed\n     */\n    @Test\n    public final void testBigIntValue() throws Exception {\n        final Random random = new Random();\n        for (int i = 0; i < 10000; ++i) {\n            BigInteger currentBigInt = null;\n            try {\n                currentBigInt = getRandomBigInt(random);\n                testBigIntValue(currentBigInt);\n            } catch (final ClassCastException e) {\n                throw new Exception(\"testBigIntValue(\" + currentBigInt + \") failed\", e);\n            }\n        }\n    }\n\n    private final void testBigIntValue(final BigInteger value) {\n        final ErlangValue eVal = new ErlangValue(value);\n        final ErlangValue eValOtp = new ErlangValue(new OtpErlangLong(value));\n\n        assertEquals(value, eVal.bigIntValue());\n        assertEquals(value, eValOtp.bigIntValue());\n        assertEquals(eVal, eValOtp);\n        assertEquals(eValOtp, eVal);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#doubleValue()}.\n     *\n     * @throws Exception if a test with a random double failed\n     */\n    @Test\n    public final void testDoubleValue() throws Exception {\n        final Random random = new Random();\n        for (int i = 0; i < 10000; ++i) {\n            Double currentDouble = null;\n            try {\n                currentDouble = random.nextDouble();\n                testDoubleValue(currentDouble);\n            } catch (final ClassCastException e) {\n                throw new Exception(\"testDoubleValue(\" + currentDouble + \") failed\", e);\n            }\n        }\n    }\n\n    private final void testDoubleValue(final double value) {\n        final ErlangValue eVal = new ErlangValue(value);\n        final ErlangValue eValOtp = new ErlangValue(new OtpErlangDouble(value));\n\n        assertEquals(value, eVal.doubleValue(), 0.0);\n        assertEquals(value, eValOtp.doubleValue(), 0.0);\n        assertEquals(eVal, eValOtp);\n        assertEquals(eValOtp, eVal);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#stringValue()}.\n     *\n     * @throws Exception if a test with a random double failed\n     */\n    @Test\n    public final void testStringValue() throws Exception {\n        final Random random = new Random();\n        for (int i = 0; i < 10000; ++i) {\n            String currentString = null;\n            try {\n                currentString = getRandomString(random, random.nextInt(1000), false);\n                testStringValue(currentString);\n            } catch (final ClassCastException e) {\n                throw new Exception(\"testStringValue(\" + currentString + \") failed\", e);\n            }\n        }\n    }\n\n    private final void testStringValue(final String value) {\n        final ErlangValue eVal = new ErlangValue(value);\n        final ErlangValue eValOtp = new ErlangValue(new OtpErlangString(value));\n\n        assertEquals(value, eVal.stringValue());\n        assertEquals(value, eValOtp.stringValue());\n        assertEquals(eVal, eValOtp);\n        assertEquals(eValOtp, eVal);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#binaryValue()}.\n     *\n     * @throws Exception if a test with a random byte array failed\n     */\n    @Test\n    public final void testBinaryValue() throws Exception {\n        final Random random = new Random();\n        for (int i = 0; i < 5000; ++i) {\n            try {\n                testBinaryValue(getRandomBytes(random, random.nextInt(1000)));\n            } catch (final ClassCastException e) {\n                // do not print generated bytes (probably not useful)\n                throw new Exception(\"testBinaryValue(...) failed\", e);\n            }\n        }\n    }\n\n    private final void testBinaryValue(final byte[] value) {\n        final ErlangValue eVal = new ErlangValue(value);\n        final ErlangValue eValOtp = new ErlangValue(new OtpErlangBinary(value));\n\n        assertArrayEquals(value, eVal.binaryValue());\n        assertArrayEquals(value, eValOtp.binaryValue());\n        assertEquals(eVal, eValOtp);\n        assertEquals(eValOtp, eVal);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#listValue()}.\n     *\n     * @throws Exception\n     *             if a test with a random list of mixed objects (bool, int,\n     *             long, BigInteger, double, String) failed\n     */\n    @Test\n    public final void testListValue() throws Exception {\n        final Random random = new Random();\n        for (int i = 0; i < 10000; ++i) {\n            List<Object> currentList = null;\n            try {\n                final int capacity = random.nextInt(1000);\n                currentList = getRandomList(random, capacity);\n                testListValue(currentList);\n            } catch (final ClassCastException e) {\n                throw new Exception(\"testListValue(\" + currentList + \") failed\", e);\n            }\n        }\n    }\n\n    private final void testListValue(final List<Object> value) {\n        final ErlangValue eVal = new ErlangValue(value);\n        final OtpErlangObject[] valueOtp = new OtpErlangObject[value.size()];\n        int i = 0;\n        for (final Object value_i : value) {\n            OtpErlangObject valueOtp_i;\n\n            if (value_i instanceof Boolean) {\n                valueOtp_i = new OtpErlangBoolean((Boolean) value_i);\n            } else if (value_i instanceof Integer) {\n                valueOtp_i = new OtpErlangInt((Integer) value_i);\n            } else if (value_i instanceof Long) {\n                valueOtp_i = new OtpErlangLong((Long) value_i);\n            } else if (value_i instanceof BigInteger) {\n                valueOtp_i = new OtpErlangLong((BigInteger) value_i);\n            } else if (value_i instanceof Double) {\n                valueOtp_i = new OtpErlangDouble((Double) value_i);\n            } else if (value_i instanceof String) {\n                valueOtp_i = new OtpErlangString((String) value_i);\n            } else {\n                fail(\"unsupported (expected) value: \" + value_i);\n                return; // so the Java-compiler does not complain about the following assignment\n            }\n            valueOtp[i++] = valueOtp_i;\n        }\n        final ErlangValue eValOtp = new ErlangValue(new OtpErlangList(valueOtp));\n\n        compareList(value, eVal.listValue());\n        compareList(value, eValOtp.listValue());\n        assertEquals(eVal, eValOtp);\n        assertEquals(eValOtp, eVal);\n    }\n\n    private static void compareList(final List<Object> expected, final List<ErlangValue> actual) {\n        assertEquals(expected.size(), actual.size());\n        for (int j = 0; j < actual.size(); ++j) {\n            final Object expected_j = expected.get(j);\n            final ErlangValue actual_j = actual.get(j);\n            if (expected_j instanceof Boolean) {\n                assertEquals(expected_j, actual_j.boolValue());\n            } else if (expected_j instanceof Integer) {\n                assertEquals(expected_j, new Integer(actual_j.intValue()));\n            } else if (expected_j instanceof Long) {\n                assertEquals(expected_j, new Long(actual_j.longValue()));\n            } else if (expected_j instanceof BigInteger) {\n                assertEquals(expected_j, actual_j.bigIntValue());\n            } else if (expected_j instanceof Double) {\n                assertEquals(expected_j, new Double(actual_j.doubleValue()));\n            } else if (expected_j instanceof String) {\n                assertEquals(expected_j, actual_j.stringValue());\n            } else {\n                fail(\"unsupported (expected) value: \" + expected_j);\n            }\n        }\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#longListValue()}.\n     *\n     * @throws Exception if a test with a random list of longs failed\n     */\n    @Test\n    public final void testLongListValue() throws Exception {\n        final Random random = new Random();\n        for (int i = 0; i < 10000; ++i) {\n            List<Long> currentList = null;\n            try {\n                final int capacity = random.nextInt(1000);\n                currentList = new ArrayList<Long>(capacity);\n                for (int j = 0; j < capacity; ++j) {\n                    currentList.add(random.nextLong());\n                }\n                testLongListValue(currentList);\n            } catch (final ClassCastException e) {\n                throw new Exception(\"testLongListValue(\" + currentList + \") failed\", e);\n            }\n        }\n    }\n\n    private final void testLongListValue(final List<Long> value) {\n        final ErlangValue eVal = new ErlangValue(value);\n        final OtpErlangLong[] valueOtp = new OtpErlangLong[value.size()];\n        int i = 0;\n        for (final Long long2 : value) {\n            final Long long1 = long2;\n            valueOtp[i++] = new OtpErlangLong(long1);\n        }\n        final ErlangValue eValOtp = new ErlangValue(new OtpErlangList(valueOtp));\n\n        assertEquals(value, eVal.longListValue());\n        assertEquals(value, eValOtp.longListValue());\n        assertEquals(eVal, eValOtp);\n        assertEquals(eValOtp, eVal);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#doubleListValue()}.\n     *\n     * @throws Exception if a test with a random list of doubles failed\n     */\n    @Test\n    public final void testDoubleListValue() throws Exception {\n        final Random random = new Random();\n        for (int i = 0; i < 10000; ++i) {\n            List<Double> currentList = null;\n            try {\n                final int capacity = random.nextInt(1000);\n                currentList = new ArrayList<Double>(capacity);\n                for (int j = 0; j < capacity; ++j) {\n                    currentList.add(random.nextDouble());\n                }\n                testDoubleListValue(currentList);\n            } catch (final ClassCastException e) {\n                throw new Exception(\"testDoubleListValue(\" + currentList + \") failed\", e);\n            }\n        }\n    }\n\n    private final void testDoubleListValue(final List<Double> value) {\n        final ErlangValue eVal = new ErlangValue(value);\n        final OtpErlangDouble[] valueOtp = new OtpErlangDouble[value.size()];\n        int i = 0;\n        for (final Double double2 : value) {\n            final Double double1 = double2;\n            valueOtp[i++] = new OtpErlangDouble(double1);\n        }\n        final ErlangValue eValOtp = new ErlangValue(new OtpErlangList(valueOtp));\n\n        assertEquals(value, eVal.doubleListValue());\n        assertEquals(value, eValOtp.doubleListValue());\n        assertEquals(eVal, eValOtp);\n        assertEquals(eValOtp, eVal);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#stringListValue()}.\n     *\n     * @throws Exception if a test with a random list of strings failed\n     */\n    @Test\n    public final void testStringListValue() throws Exception {\n        final Random random = new Random();\n        for (int i = 0; i < 10000; ++i) {\n            List<String> currentList = null;\n            try {\n                final int capacity = random.nextInt(10);\n                currentList = new ArrayList<String>(capacity);\n                for (int j = 0; j < capacity; ++j) {\n                    currentList.add(getRandomString(random, random.nextInt(1000), false));\n                }\n                testStringListValue(currentList);\n            } catch (final ClassCastException e) {\n                throw new Exception(\"testStringListValue(\" + currentList + \") failed\", e);\n            }\n        }\n    }\n\n    private final void testStringListValue(final List<String> value) {\n        final ErlangValue eVal = new ErlangValue(value);\n        final OtpErlangString[] valueOtp = new OtpErlangString[value.size()];\n        int i = 0;\n        for (final String string : value) {\n            final String string1 = string;\n            valueOtp[i++] = new OtpErlangString(string1);\n        }\n        final ErlangValue eValOtp = new ErlangValue(new OtpErlangList(valueOtp));\n\n        assertEquals(value, eVal.stringListValue());\n        assertEquals(value, eValOtp.stringListValue());\n        assertEquals(eVal, eValOtp);\n        assertEquals(eValOtp, eVal);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#binaryListValue()}.\n     *\n     * @throws Exception if a test with a random list of binaries failed\n     */\n    @Test\n    public final void testBinaryListValue() throws Exception {\n        final Random random = new Random();\n        for (int i = 0; i < 1000; ++i) {\n            try {\n                final int capacity = random.nextInt(10);\n                final List<byte[]> currentList = new ArrayList<byte[]>(capacity);\n                for (int j = 0; j < capacity; ++j) {\n                    currentList.add(getRandomBytes(random, random.nextInt(1000)));\n                }\n                testBinaryListValue(currentList);\n            } catch (final ClassCastException e) {\n                // do not print generated bytes (probably not useful)\n                throw new Exception(\"testBinaryListValue(...) failed\", e);\n            }\n        }\n    }\n\n    private final void testBinaryListValue(final List<byte[]> value) {\n        final ErlangValue eVal = new ErlangValue(value);\n        final OtpErlangBinary[] valueOtp = new OtpErlangBinary[value.size()];\n        int i = 0;\n        for (final byte[] b : value) {\n            final byte[] binary1 = b;\n            valueOtp[i++] = new OtpErlangBinary(binary1);\n        }\n        final ErlangValue eValOtp = new ErlangValue(new OtpErlangList(valueOtp));\n\n        final List<byte[]> actual = eVal.binaryListValue();\n        final List<byte[]> actualOtp = eValOtp.binaryListValue();\n        assertEquals(value.size(), actual.size());\n        assertEquals(value.size(), actualOtp.size());\n        for (int j = 0; j < actual.size(); ++j) {\n            assertArrayEquals(value.get(j), actual.get(j));\n            assertArrayEquals(value.get(j), actualOtp.get(j));\n        }\n        assertEquals(eVal, eValOtp);\n        assertEquals(eValOtp, eVal);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#jsonValue()}.\n     *\n     * @throws Exception\n     *             if a test with a random list of mixed objects (bool, int,\n     *             long, BigInteger, double, String) failed\n     */\n    @Test\n    public final void testJsonValue() throws Exception {\n        final Random random = new Random();\n        for (int i = 0; i < 10000; ++i) {\n            Map<String, Object> currentMap = null;\n            try {\n                currentMap = getRandomMapRecursive(random, random.nextInt(10), 3, false);\n                testJsonValue(currentMap);\n            } catch (final ClassCastException e) {\n                throw new Exception(\"testJsonValue(\" + currentMap + \") failed\", e);\n            }\n        }\n    }\n\n    private final void testJsonValue(final Map<String, Object> value) {\n        final ErlangValue eVal = new ErlangValue(value);\n        final Map<String, Object> actual = eVal.jsonValue();\n        compareMap(value, actual);\n        final ErlangValue eVal2 = new ErlangValue(value);\n        assertEquals(eVal, eVal2);\n        assertEquals(eVal2, eVal);\n    }\n\n    private final void compareMap(final Map<String, Object> expected, final Map<String, Object> actual) {\n        assertEquals(expected.size(), actual.size());\n        for (final Entry<String, Object> entry : expected.entrySet()) {\n            final String expected_key_i = entry.getKey();\n            final Object expected_value_i = entry.getValue();\n\n            assertTrue(actual.containsKey(expected_key_i));\n            final Object actual_value_i = actual.get(expected_key_i);\n            assertEquals(expected_value_i, actual_value_i);\n        }\n    }\n\n    private static class JSONBeanTest1 {\n        private boolean a = true;\n        private int b = 0;\n        private long c = 0;\n        private BigInteger d = new BigInteger(\"0\");\n        private double e = 0.0;\n        private String f = \"\";\n\n        public JSONBeanTest1() {}\n\n        public boolean getA() { return a; }\n        public int getB() { return b; }\n        public long getC() { return c; }\n        public BigInteger getD() { return d; }\n        public double getE() { return e; }\n        public String getF() { return f; }\n\n        public void setA(final boolean a_) { this.a = a_; }\n        public void setB(final int b_) { this.b = b_; }\n        public void setC(final long c_) { this.c = c_; }\n        public void setD(final BigInteger d_) { this.d = d_; }\n        public void setE(final double e_) { this.e = e_; }\n        public void setF(final String f_) { this.f = f_; }\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#jsonValue(Class)}\n     * writing a {@link Map} and reading a {@link JSONBeanTest1}.\n     *\n     * @throws Exception\n     *             if a test with a random list of mixed objects (bool, int,\n     *             long, BigInteger, double, String) failed\n     */\n    @Test\n    public final void testJsonValueBean1a() throws Exception {\n        final Random random = new Random();\n\n        for (int i = 0; i < 5000; ++i) {\n            final Map<String, Object> map = new LinkedHashMap<String, Object>(6);\n            map.put(\"a\", random.nextBoolean());\n            map.put(\"b\", random.nextInt());\n            map.put(\"c\", random.nextLong());\n            map.put(\"d\", getRandomBigInt(random));\n            map.put(\"e\", random.nextDouble());\n            map.put(\"f\", getRandomString(random, random.nextInt(100), false));\n            final ErlangValue value = new ErlangValue(map);\n\n            final JSONBeanTest1 actual = value.jsonValue(JSONBeanTest1.class);\n            assertEquals(map.get(\"a\"), actual.getA());\n            assertEquals(map.get(\"b\"), actual.getB());\n            assertEquals(map.get(\"c\"), actual.getC());\n            assertEquals(map.get(\"d\"), actual.getD());\n            assertEquals(map.get(\"e\"), actual.getE());\n            assertEquals(map.get(\"f\"), actual.getF());\n            compareScalarisJSON(value, new ErlangValue(actual));\n        }\n    }\n\n    /**\n     * Compares two erlang values which represent JSON tuples.\n     *\n     * The order of the JSON objects' properties does not matter.\n     *\n     * @param expected\n     * @param actual\n     */\n    private final void compareScalarisJSON(final ErlangValue expected,\n            final ErlangValue actual) {\n        compareScalarisJSON(expected.value(), actual.value(), actual.toString());\n    }\n\n    /**\n     * Compares the two erlang object assuming they are both JSON tuples.\n     *\n     * @param expected\n     * @param actual\n     */\n    private final void compareScalarisJSON(final OtpErlangObject expected,\n            final OtpErlangObject actual, final String actualOriginal) {\n        // verify: tuples with arity 2 and \"struct\" as the first element\n        final OtpErlangTuple expectedT = (OtpErlangTuple) expected;\n        assert (expectedT.arity() == 2);\n        String msg = \"Checking '\" + actual + \"' in \" + actualOriginal;\n        assertTrue(msg, actual instanceof OtpErlangTuple);\n        final OtpErlangTuple actualT = (OtpErlangTuple) actual;\n        assertTrue(msg, actualT.arity() == 2);\n        assert (expectedT.elementAt(0).equals(new OtpErlangString(\"struct\")));\n        assertEquals(\"Checking '\" + actualT.elementAt(0) + \"' in \"\n                + actualOriginal, CommonErlangObjects.structAtom, actualT.elementAt(0));\n\n        // verify: second element in the struct-tuple is a list of properties\n        // ({Key, Value} tuples with string keys)\n        final OtpErlangList expectedPropsL = (OtpErlangList) expectedT\n                .elementAt(1);\n        msg = \"Checking '\" + actualT.elementAt(1) + \"' in \" + actualOriginal;\n        assertTrue(msg, actualT.elementAt(1) instanceof OtpErlangList);\n        final OtpErlangList actualPropsL = (OtpErlangList) actualT.elementAt(1);\n        assertEquals(msg, expectedPropsL.arity(), actualPropsL.arity());\n        final HashMap<String, OtpErlangObject> actualProperties = new HashMap<String, OtpErlangObject>(\n                actualPropsL.arity());\n        // put the actual object's properties into a hash map for quick access\n        // while on it, verify the types, too\n        for (int i = 0; i < actualPropsL.arity(); ++i) {\n            final OtpErlangObject element = actualPropsL.elementAt(i);\n            msg = \"Checking '\" + element + \"' in \" + actualOriginal;\n            assertTrue(msg, element instanceof OtpErlangTuple);\n            final OtpErlangTuple actualPropT = (OtpErlangTuple) element;\n            assertTrue(msg, actualPropT.arity() == 2);\n\n            final OtpErlangObject actualPropKey = actualPropT.elementAt(0);\n            final OtpErlangObject actualPropVal = actualPropT.elementAt(1);\n            assertTrue(\"Checking property '\" + actualPropKey + \"' in \"\n                    + actualOriginal, actualPropKey instanceof OtpErlangString);\n            actualProperties.put(\n                    ((OtpErlangString) actualPropKey).stringValue(),\n                    actualPropVal);\n        }\n\n        // verify: all properties from expected exist in actual and no more\n        for (int i = 0; i < expectedPropsL.arity(); ++i) {\n            final OtpErlangTuple expectedPropT = (OtpErlangTuple) expectedPropsL\n                    .elementAt(i);\n            assert (expectedPropT.arity() == 2);\n            final String expectedPropKey = ((OtpErlangString) expectedPropT\n                    .elementAt(0)).stringValue();\n            final OtpErlangObject expectedPropVal = expectedPropT.elementAt(1);\n            assertTrue(\"Checking property '\" + expectedPropKey + \"' in \"\n                    + actualOriginal,\n                    actualProperties.containsKey(expectedPropKey));\n            final OtpErlangObject actualPropVal = actualProperties\n                    .get(expectedPropKey);\n            if (actualPropVal instanceof OtpErlangTuple) {\n                final OtpErlangTuple expectedPropValT = (OtpErlangTuple) expectedPropVal;\n                final OtpErlangTuple actualPropValT = (OtpErlangTuple) actualPropVal;\n                if ((expectedPropValT.arity() == 2)\n                        && (actualPropValT.arity() == 2)\n                        && expectedPropValT.elementAt(0).equals(CommonErlangObjects.arrayAtom)\n                        && actualPropValT.elementAt(0).equals(CommonErlangObjects.arrayAtom)\n                        && (expectedPropValT.elementAt(1) instanceof OtpErlangList)\n                        && (actualPropValT.elementAt(1) instanceof OtpErlangList)) {\n                    // these lists must be equal including their order\n                    assertEquals(\"Checking '\" + actualPropValT.elementAt(1) + \"' in \" + actualOriginal,\n                            actualPropValT.elementAt(1), actualPropValT.elementAt(1));\n                } else {\n                    compareScalarisJSON(expectedPropVal, actualPropVal,\n                            actualOriginal);\n                }\n            } else {\n                assertEquals(\"Checking value of property '\" + expectedPropKey\n                        + \"' in \" + actualOriginal, expectedPropVal,\n                        actualPropVal);\n            }\n        }\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#jsonValue(Class)}\n     * writing a {@link JSONBeanTest1} and reading a {@link JSONBeanTest1}.\n     *\n     * @throws Exception\n     *             if a test with a random list of mixed objects (bool, int,\n     *             long, BigInteger, double, String) failed\n     */\n    @Test\n    public final void testJsonValueBean1b() throws Exception {\n        final Random random = new Random();\n\n        for (int i = 0; i < 5000; ++i) {\n            final JSONBeanTest1 bean1 = new JSONBeanTest1();\n            bean1.setA(random.nextBoolean());\n            bean1.setB(random.nextInt());\n            bean1.setC(random.nextLong());\n            bean1.setD(getRandomBigInt(random));\n            bean1.setE(random.nextDouble());\n            bean1.setF(getRandomString(random, random.nextInt(100), false));\n            final ErlangValue value = new ErlangValue(bean1);\n\n            final JSONBeanTest1 actual = value.jsonValue(JSONBeanTest1.class);\n            assertEquals(bean1.getA(), actual.getA());\n            assertEquals(bean1.getB(), actual.getB());\n            assertEquals(bean1.getC(), actual.getC());\n            assertEquals(bean1.getD(), actual.getD());\n            assertEquals(bean1.getE(), actual.getE(), 0.0);\n            assertEquals(bean1.getF(), actual.getF());\n            compareScalarisJSON(value, new ErlangValue(actual));\n        }\n    }\n\n    private static class JSONBeanTest2 {\n        private boolean a2 = true;\n        private int b2 = 0;\n        private long c2 = 0;\n        private BigInteger d2 = new BigInteger(\"0\");\n        private double e2 = 0.0;\n        private String f2 = \"\";\n        private List<Object> g2 = new ArrayList<Object>();\n        private JSONBeanTest1 h2 = new JSONBeanTest1();\n        private Map<String, Object> i2 = new LinkedHashMap<String, Object>();\n\n        public JSONBeanTest2() {}\n\n        public boolean getA2() { return a2; }\n        public int getB2() { return b2; }\n        public long getC2() { return c2; }\n        public BigInteger getD2() { return d2; }\n        public double getE2() { return e2; }\n        public String getF2() { return f2; }\n        public List<Object> getG2() { return g2; }\n        public JSONBeanTest1 getH2() { return h2; }\n        public Map<String, Object> getI2() { return i2; }\n\n        public void setA2(final boolean a_) { this.a2 = a_; }\n        public void setB2(final int b_) { this.b2 = b_; }\n        public void setC2(final long c_) { this.c2 = c_; }\n        public void setD2(final BigInteger d_) { this.d2 = d_; }\n        public void setE2(final double e_) { this.e2 = e_; }\n        public void setF2(final String f_) { this.f2 = f_; }\n        public void setG2(final List<Object> g_) { this.g2 = g_; }\n        public void setH2(final JSONBeanTest1 h_) { this.h2 = h_; }\n        public void setI2(final Map<String, Object> i_) { this.i2 = i_; }\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#jsonValue(Class)}\n     * writing a {@link Map} and reading a {@link JSONBeanTest2}.\n     *\n     * @throws Exception\n     *             if a test with a random list of mixed objects (bool, int,\n     *             long, BigInteger, double, String) failed\n     */\n    @Test\n    public final void testJsonValueBean2a() throws Exception {\n        final Random random = new Random();\n\n        for (int i = 0; i < 5000; ++i) {\n            final Map<String, Object> map = new LinkedHashMap<String, Object>(7);\n            map.put(\"a2\", random.nextBoolean());\n            map.put(\"b2\", random.nextInt());\n            map.put(\"c2\", random.nextLong());\n            map.put(\"d2\", getRandomBigInt(random));\n            map.put(\"e2\", random.nextDouble());\n            map.put(\"f2\", getRandomString(random, random.nextInt(100), false));\n            map.put(\"g2\", getRandomList(random, 100));\n            final Map<String, Object> bean1 = new LinkedHashMap<String, Object>(6);\n            bean1.put(\"a\", random.nextBoolean());\n            bean1.put(\"b\", random.nextInt());\n            bean1.put(\"c\", random.nextLong());\n            bean1.put(\"d\", getRandomBigInt(random));\n            bean1.put(\"e\", random.nextDouble());\n            bean1.put(\"f\", getRandomString(random, 10, false));\n            map.put(\"h2\", bean1);\n            final Map<String, Object> map2 = getRandomMapRecursive(random, 10, 3, true);\n            map.put(\"i2\", map2);\n            final ErlangValue value = new ErlangValue(map);\n\n            final JSONBeanTest2 actual = value.jsonValue(JSONBeanTest2.class);\n            assertEquals(map.get(\"a2\"), actual.getA2());\n            assertEquals(map.get(\"b2\"), actual.getB2());\n            assertEquals(map.get(\"c2\"), actual.getC2());\n            assertEquals(map.get(\"d2\"), actual.getD2());\n            assertEquals(map.get(\"e2\"), actual.getE2());\n            assertEquals(map.get(\"f2\"), actual.getF2());\n            assertEquals(map.get(\"g2\"), actual.getG2());\n            final JSONBeanTest1 bean1_act = actual.getH2();\n            assertEquals(bean1.get(\"a\"), bean1_act.getA());\n            assertEquals(bean1.get(\"b\"), bean1_act.getB());\n            assertEquals(bean1.get(\"c\"), bean1_act.getC());\n            assertEquals(bean1.get(\"d\"), bean1_act.getD());\n            assertEquals(bean1.get(\"e\"), bean1_act.getE());\n            assertEquals(bean1.get(\"f\"), bean1_act.getF());\n            compareMap(map2, actual.getI2());\n            compareScalarisJSON(value, new ErlangValue(actual));\n        }\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#jsonValue(Class)}\n     * writing a {@link JSONBeanTest2} and reading a {@link JSONBeanTest2}.\n     *\n     * @throws Exception\n     *             if a test with a random list of mixed objects (bool, int,\n     *             long, BigInteger, double, String) failed\n     */\n    @Test\n    public final void testJsonValueBean2b() throws Exception {\n        final Random random = new Random();\n\n        for (int i = 0; i < 5000; ++i) {\n            final JSONBeanTest2 bean2 = new JSONBeanTest2();\n            bean2.setA2(random.nextBoolean());\n            bean2.setB2(random.nextInt());\n            bean2.setC2(random.nextLong());\n            bean2.setD2(getRandomBigInt(random));\n            bean2.setE2(random.nextDouble());\n            bean2.setF2(getRandomString(random, random.nextInt(100), false));\n            bean2.setG2(getRandomList(random, 100));\n            final JSONBeanTest1 bean1 = new JSONBeanTest1();\n            bean1.setA(random.nextBoolean());\n            bean1.setB(random.nextInt());\n            bean1.setC(random.nextLong());\n            bean1.setD(getRandomBigInt(random));\n            bean1.setE(random.nextDouble());\n            bean1.setF(getRandomString(random, 10, false));\n            bean2.setH2(bean1);\n            final Map<String, Object> map2 = getRandomMapRecursive(random, 10, 3, true);\n            bean2.setI2(map2);\n            final ErlangValue value = new ErlangValue(bean2);\n\n            final JSONBeanTest2 actual = value.jsonValue(JSONBeanTest2.class);\n            assertEquals(bean2.getA2(), actual.getA2());\n            assertEquals(bean2.getB2(), actual.getB2());\n            assertEquals(bean2.getC2(), actual.getC2());\n            assertEquals(bean2.getD2(), actual.getD2());\n            assertEquals(bean2.getE2(), actual.getE2(), 0.0);\n            assertEquals(bean2.getF2(), actual.getF2());\n            assertEquals(bean2.getG2(), actual.getG2());\n            final JSONBeanTest1 bean1_act = actual.getH2();\n            assertEquals(bean1.getA(), bean1_act.getA());\n            assertEquals(bean1.getB(), bean1_act.getB());\n            assertEquals(bean1.getC(), bean1_act.getC());\n            assertEquals(bean1.getD(), bean1_act.getD());\n            assertEquals(bean1.getE(), bean1_act.getE(), 0.0);\n            assertEquals(bean1.getF(), bean1_act.getF());\n            compareMap(map2, actual.getI2());\n            compareScalarisJSON(value, new ErlangValue(actual));\n        }\n    }\n\n\n    private static class JSONBeanTest3 {\n        private List<JSONBeanTest1> a3 = new ArrayList<JSONBeanTest1>();\n        private Map<String, JSONBeanTest1> b3 = new LinkedHashMap<String, JSONBeanTest1>();\n\n        public JSONBeanTest3() {}\n\n        public List<JSONBeanTest1> getA3() { return a3; }\n        public Map<String, JSONBeanTest1> getB3() { return b3; }\n\n        public void setA3(final List<JSONBeanTest1> a_) { this.a3 = a_; }\n        public void setB3(final Map<String, JSONBeanTest1> b_) { this.b3 = b_; }\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#jsonValue(Class)}\n     * writing a {@link Map} and reading a {@link JSONBeanTest3}.\n     *\n     * @throws Exception\n     *             if a test with a random list of mixed objects (bool, int,\n     *             long, BigInteger, double, String) failed\n     */\n    @Test\n    public final void testJsonValueBean3a() throws Exception {\n        final Random random = new Random();\n\n        for (int i = 0; i < 5000; ++i) {\n            final Map<String, Object> map = new LinkedHashMap<String, Object>(7);\n            final Map<String, Object> bean1a = new LinkedHashMap<String, Object>(6);\n            bean1a.put(\"a\", random.nextBoolean());\n            bean1a.put(\"b\", random.nextInt());\n            bean1a.put(\"c\", random.nextLong());\n            bean1a.put(\"d\", getRandomBigInt(random));\n            bean1a.put(\"e\", random.nextDouble());\n            bean1a.put(\"f\", getRandomString(random, 10, false));\n            final Map<String, Object> bean1b = new LinkedHashMap<String, Object>(6);\n            bean1b.put(\"a\", random.nextBoolean());\n            bean1b.put(\"b\", random.nextInt());\n            bean1b.put(\"c\", random.nextLong());\n            bean1b.put(\"d\", getRandomBigInt(random));\n            bean1b.put(\"e\", random.nextDouble());\n            bean1b.put(\"f\", getRandomString(random, 10, false));\n            final List<Map<String, Object>> list1 = new LinkedList<Map<String,Object>>();\n            list1.add(bean1a);\n            list1.add(bean1b);\n            map.put(\"a3\", list1);\n            final Map<String, Map<String, Object>> map1 = new LinkedHashMap<String, Map<String,Object>>();\n            map1.put(\"a4\", bean1a);\n            map1.put(\"b4\", bean1b);\n            map.put(\"b3\", map1);\n            final ErlangValue value = new ErlangValue(map);\n\n            final JSONBeanTest3 actual = value.jsonValue(JSONBeanTest3.class);\n            final List<JSONBeanTest1> actual_a3 = actual.getA3();\n            assertEquals(list1.size(), actual_a3.size());\n            for (int j = 0; j < list1.size(); ++j) {\n                assertEquals(list1.get(j).get(\"a\"), actual_a3.get(j).getA());\n                assertEquals(list1.get(j).get(\"b\"), actual_a3.get(j).getB());\n                assertEquals(list1.get(j).get(\"c\"), actual_a3.get(j).getC());\n                assertEquals(list1.get(j).get(\"d\"), actual_a3.get(j).getD());\n                assertEquals(list1.get(j).get(\"e\"), actual_a3.get(j).getE());\n                assertEquals(list1.get(j).get(\"f\"), actual_a3.get(j).getF());\n            }\n            final Map<String, JSONBeanTest1> actual_b3 = actual.getB3();\n            assertEquals(map1.size(), actual_b3.size());\n            for (final Entry<String, Map<String, Object>> key : map1.entrySet()) {\n                assertEquals(key.getValue().get(\"a\"), actual_b3.get(key.getKey()).getA());\n                assertEquals(key.getValue().get(\"b\"), actual_b3.get(key.getKey()).getB());\n                assertEquals(key.getValue().get(\"c\"), actual_b3.get(key.getKey()).getC());\n                assertEquals(key.getValue().get(\"d\"), actual_b3.get(key.getKey()).getD());\n                assertEquals(key.getValue().get(\"e\"), actual_b3.get(key.getKey()).getE());\n                assertEquals(key.getValue().get(\"f\"), actual_b3.get(key.getKey()).getF());\n            }\n            compareScalarisJSON(value, new ErlangValue(actual));\n        }\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.ErlangValue#jsonValue(Class)}\n     * writing a {@link JSONBeanTest2} and reading a {@link JSONBeanTest2}.\n     *\n     * @throws Exception\n     *             if a test with a random list of mixed objects (bool, int,\n     *             long, BigInteger, double, String) failed\n     */\n    @Test\n    public final void testJsonValueBean3b() throws Exception {\n        final Random random = new Random();\n\n        for (int i = 0; i < 10000; ++i) {\n            final JSONBeanTest3 bean3 = new JSONBeanTest3();\n            final JSONBeanTest1 bean1a = new JSONBeanTest1();\n            bean1a.setA(random.nextBoolean());\n            bean1a.setB(random.nextInt());\n            bean1a.setC(random.nextLong());\n            bean1a.setD(getRandomBigInt(random));\n            bean1a.setE(random.nextDouble());\n            bean1a.setF(getRandomString(random, 10, false));\n            final JSONBeanTest1 bean1b = new JSONBeanTest1();\n            bean1b.setA(random.nextBoolean());\n            bean1b.setB(random.nextInt());\n            bean1b.setC(random.nextLong());\n            bean1b.setD(getRandomBigInt(random));\n            bean1b.setE(random.nextDouble());\n            bean1b.setF(getRandomString(random, 10, false));\n            final List<JSONBeanTest1> list1 = new LinkedList<JSONBeanTest1>();\n            list1.add(bean1a);\n            list1.add(bean1b);\n            bean3.setA3(list1);\n            final Map<String, JSONBeanTest1> map1 = new LinkedHashMap<String, JSONBeanTest1>();\n            map1.put(\"a4\", bean1a);\n            map1.put(\"b4\", bean1b);\n            bean3.setB3(map1);\n            final ErlangValue value = new ErlangValue(bean3);\n\n            final JSONBeanTest3 actual = value.jsonValue(JSONBeanTest3.class);\n            final List<JSONBeanTest1> actual_a3 = actual.getA3();\n            assertEquals(list1.size(), actual_a3.size());\n            for (int j = 0; j < list1.size(); ++j) {\n                assertEquals(list1.get(j).getA(), actual_a3.get(j).getA());\n                assertEquals(list1.get(j).getB(), actual_a3.get(j).getB());\n                assertEquals(list1.get(j).getC(), actual_a3.get(j).getC());\n                assertEquals(list1.get(j).getD(), actual_a3.get(j).getD());\n                assertEquals(list1.get(j).getE(), actual_a3.get(j).getE(), 0.0);\n                assertEquals(list1.get(j).getF(), actual_a3.get(j).getF());\n            }\n            final Map<String, JSONBeanTest1> actual_b3 = actual.getB3();\n            assertEquals(map1.size(), actual_b3.size());\n            for (final Entry<String, JSONBeanTest1> key : map1.entrySet()) {\n                assertEquals(key.getValue().getA(), actual_b3.get(key.getKey()).getA());\n                assertEquals(key.getValue().getB(), actual_b3.get(key.getKey()).getB());\n                assertEquals(key.getValue().getC(), actual_b3.get(key.getKey()).getC());\n                assertEquals(key.getValue().getD(), actual_b3.get(key.getKey()).getD());\n                assertEquals(key.getValue().getE(), actual_b3.get(key.getKey()).getE(), 0.0);\n                assertEquals(key.getValue().getF(), actual_b3.get(key.getKey()).getF());\n            }\n            compareScalarisJSON(value, new ErlangValue(actual));\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/test/de/zib/scalaris/InterOpTest.java",
    "content": "/*\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport java.io.PrintStream;\nimport java.math.BigInteger;\nimport java.util.ArrayList;\nimport java.util.LinkedHashMap;\nimport java.util.List;\nimport java.util.Map;\nimport java.util.Map.Entry;\n\nimport org.apache.commons.cli.CommandLine;\nimport org.apache.commons.cli.CommandLineParser;\nimport org.apache.commons.cli.GnuParser;\nimport org.apache.commons.cli.HelpFormatter;\nimport org.apache.commons.cli.Option;\nimport org.apache.commons.cli.OptionGroup;\nimport org.apache.commons.cli.Options;\nimport org.apache.commons.cli.ParseException;\n\n/**\n * Class to test interoperability between the different Scalaris APIs.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.1\n * @since 3.1\n */\npublic class InterOpTest {\n    private enum Mode { READ, WRITE };\n\n    /**\n     * Queries the command line options for an action to perform.\n     *\n     * <pre>\n     * <code>\n     * > java -jar scalaris.jar -help\n     * usage: scalaris [Options]\n     *  -h,--help                        print this message\n     *  -v,--verbose                     print verbose information, e.g. the\n     *                                   properties read\n     *  -lh,--localhost                  gets the local host's name as known to\n     *                                   Java (for debugging purposes)\n     *  -r,--read <basekey>              read all items with the given basekey\n     *  -w,--write <basekey>             writes items of different types to the\n     *                                   given basekey\n     * </code>\n     * </pre>\n     *\n     * In order to override node and cookie to use for a connection, specify\n     * the <tt>scalaris.node</tt> or <tt>scalaris.cookie</tt> system properties.\n     * Their values will be used instead of the values defined in the config\n     * file!\n     *\n     * @param args\n     *            command line arguments\n     */\n    public static void main(final String[] args) {\n        boolean verbose = false;\n        final CommandLineParser parser = new GnuParser();\n        CommandLine line = null;\n        final Options options = getOptions();\n        try {\n            line = parser.parse(options, args);\n        } catch (final ParseException e) {\n            Main.printException(\"Parsing failed\", e, false);\n            return; // will not be reached since printException exits\n        }\n\n        if (line.hasOption(\"verbose\")) {\n            verbose = true;\n            ConnectionFactory.getInstance().printProperties();\n        }\n\n        String basekey;\n        Mode mode;\n        if (line.hasOption(\"r\")) { // read\n            mode = Mode.READ;\n            final String[] optionValues = line.getOptionValues(\"read\");\n            Main.checkArguments(optionValues, 2, options, \"r\");\n            basekey = optionValues[0];\n            final String language = optionValues[1];\n            basekey += \"_\" + language;\n            System.out.println(\"Java-API: reading from \" + language);\n        } else if (line.hasOption(\"w\")) { // write\n            mode = Mode.WRITE;\n            basekey = line.getOptionValue(\"write\");\n            Main.checkArguments(basekey, options, \"w\");\n            basekey += \"_java\";\n            System.out.println(\"Java-API: writing values\");\n        } else if (line.hasOption(\"lh\")) { // get local host name\n            System.out.println(ConnectionFactory.getLocalhostName());\n            return;\n        } else {\n            // print help if no other option was given\n//        if (line.hasOption(\"help\")) {\n            final HelpFormatter formatter = new HelpFormatter();\n            formatter.printHelp(\"scalaris [Options]\", getOptions());\n            return;\n        }\n        try {\n            final TransactionSingleOp sc = new TransactionSingleOp();\n            int failed = 0;\n\n            failed += read_write_boolean(basekey, sc, mode);\n            failed += read_write_integer(basekey, sc, mode);\n            failed += read_write_long(basekey, sc, mode);\n            failed += read_write_biginteger(basekey, sc, mode);\n            failed += read_write_double(basekey, sc, mode);\n            failed += read_write_string(basekey, sc, mode);\n            failed += read_write_binary(basekey, sc, mode);\n            failed += read_write_list(basekey, sc, mode);\n            failed += read_write_map(basekey, sc, mode);\n            System.out.println();\n\n            sc.closeConnection();\n            if (failed > 0) {\n                if (mode == Mode.WRITE) {\n                    System.out.println(failed + \" number of writes failed.\");\n                } else {\n                    System.out.println(failed + \" number of reads failed.\");\n                }\n                System.out.println();\n                System.exit(1);\n            }\n        } catch (final ConnectionException e) {\n            Main.printException(\"failed with connection error\", e, verbose);\n        } catch (final UnknownException e) {\n            Main.printException(\"failed with unknown\", e, verbose);\n        }\n    }\n\n    private static int read_or_write(final TransactionSingleOp sc, final String key, final Object value, final Mode mode) {\n        try {\n            switch (mode) {\n                case READ:\n                    final ErlangValue actual = sc.read(key);\n                    Object jresult = null;\n                    if (value instanceof Boolean) {\n                        jresult = actual.boolValue();\n                    } else if (value instanceof Integer) {\n                        jresult = actual.intValue();\n                    } else if (value instanceof Long) {\n                        jresult = actual.longValue();\n                    } else if (value instanceof BigInteger) {\n                        jresult = actual.bigIntValue();\n                    } else if (value instanceof Double) {\n                        jresult = actual.doubleValue();\n                    } else if (value instanceof String) {\n                        jresult = actual.stringValue();\n                    } else if (value instanceof byte[]) {\n                        jresult = actual.binaryValue();\n                    } else if (value instanceof List<?>) {\n                        jresult = actual.listValue();\n                    } else if (value instanceof Map<?, ?>) {\n                        jresult = actual.jsonValue();\n                    } else {\n                        jresult = actual.value();\n                    }\n                    PrintStream out;\n                    int result;\n                    String resultStr;\n                    if (compare(jresult, value)) {\n                        out = System.out;\n                        result = 0;\n                        resultStr = \"ok\";\n                    } else {\n                        out = System.err;\n                        result = 1;\n                        resultStr = \"fail\";\n                    }\n\n                    out.println(\"read(\" + key + \")\");\n                    out.println(\"  expected: \" + valueToStr(value));\n                    out.println(\"  read raw: \" + actual.value().toString());\n                    out.println(\" read java: \" + valueToStr(jresult));\n                    out.println(resultStr);\n                    return result;\n                case WRITE:\n                    System.out.println(\"write(\" + key + \", \" + valueToStr(value) + \")\");\n                    sc.write(key, value);\n                    return 0;\n            }\n        } catch (final ConnectionException e) {\n            System.out.println(\"failed with connection error\");\n        } catch (final AbortException e) {\n            System.out.println(\"failed with abort\");\n        } catch (final NotFoundException e) {\n            System.out.println(\"failed with not_found\");\n        } catch (final UnknownException e) {\n            System.out.println(\"failed with unknown\");\n            e.printStackTrace();\n        } catch (final ClassCastException e) {\n            System.out.println(\"failed with ClassCastException\");\n            e.printStackTrace();\n        }\n        return 1;\n    }\n\n    private static boolean compare(final byte[] actual, final Object expected) {\n        try {\n            final byte[] expected_bytes = (byte[]) expected;\n            if (expected_bytes.length != actual.length) {\n                return false;\n            }\n            for (int i = 0; i < expected_bytes.length; ++i) {\n                if (expected_bytes[i] != actual[i]) {\n                    return false;\n                }\n            }\n            return true;\n        } catch (final Exception e) {\n            return false;\n        }\n    }\n\n    private static boolean compare(final List<Object> actual, final Object expected) {\n        try {\n            @SuppressWarnings(\"unchecked\")\n            final\n            List<Object> expected_list = (List<Object>) expected;\n            if (expected_list.size() != actual.size()) {\n                return false;\n            }\n            for (int i = 0; i < expected_list.size(); ++i) {\n                if (!compare(actual.get(i), expected_list.get(i))) {\n                    return false;\n                }\n            }\n            return true;\n        } catch (final Exception e) {\n            return false;\n        }\n    }\n\n    private static boolean compare(final Map<String, Object> actual, final Object expected) {\n        try {\n            @SuppressWarnings(\"unchecked\")\n            final\n            Map<String, Object> expected_map = (Map<String, Object>) expected;\n            if (expected_map.size() != actual.size()) {\n                return false;\n            }\n            for (final Entry<String, Object> value : expected_map.entrySet()) {\n                if (!compare(actual.get(value.getKey()), value.getValue())) {\n                    return false;\n                }\n            }\n            return true;\n        } catch (final Exception e) {\n            return false;\n        }\n    }\n\n    private static boolean compare(final Object actual, final Object expected) {\n        try {\n            if (expected instanceof byte[]) {\n                if (actual instanceof ErlangValue) {\n                    return compare(((ErlangValue) actual).binaryValue(), expected);\n                } else {\n                    return compare((byte[]) actual, expected);\n                }\n            } else if (expected instanceof List<?>) {\n                if (actual instanceof ErlangValue) {\n                    return compare(((ErlangValue) actual).listValue(), expected);\n                } else {\n                    @SuppressWarnings(\"unchecked\")\n                    final\n                    List<Object> actual_list = (List<Object>) actual;\n                    return compare(actual_list, expected);\n                }\n            } else if (expected instanceof Map<?, ?>) {\n                if (actual instanceof ErlangValue) {\n                    return compare(((ErlangValue) actual).jsonValue(), expected);\n                } else {\n                    @SuppressWarnings(\"unchecked\")\n                    final\n                    Map<String, Object> actual_map = (Map<String, Object>) actual;\n                    return compare(actual_map, expected);\n                }\n            } else if (expected instanceof Boolean) {\n                if (actual instanceof ErlangValue) {\n                    return expected.equals(((ErlangValue) actual).boolValue());\n                } else {\n                    return expected.equals(actual);\n                }\n            } else if (expected instanceof Integer) {\n                if (actual instanceof ErlangValue) {\n                    return expected.equals(((ErlangValue) actual).intValue());\n                } else {\n                    return expected.equals(actual);\n                }\n            } else if (expected instanceof Long) {\n                if (actual instanceof ErlangValue) {\n                    return expected.equals(((ErlangValue) actual).longValue());\n                } else {\n                    return expected.equals(actual);\n                }\n            } else if (expected instanceof BigInteger) {\n                if (actual instanceof ErlangValue) {\n                    return expected.equals(((ErlangValue) actual).bigIntValue());\n                } else {\n                    return expected.equals(actual);\n                }\n            } else if (expected instanceof Double) {\n                if (actual instanceof ErlangValue) {\n                    return expected.equals(((ErlangValue) actual).doubleValue());\n                } else {\n                    return expected.equals(actual);\n                }\n            } else if (expected instanceof String) {\n                if (actual instanceof ErlangValue) {\n                    return expected.equals(((ErlangValue) actual).stringValue());\n                } else {\n                    return expected.equals(actual);\n                }\n            } else {\n                return false;\n            }\n        } catch (final Exception e) {\n            return false;\n        }\n    }\n\n    private static String valueToStr(final Object value) {\n        final StringBuilder sb = new StringBuilder();\n        valueToStr(value, sb);\n        return sb.toString();\n    }\n\n    private static void valueToStr(final Object value, final StringBuilder sb) {\n        if (value instanceof String) {\n          sb.append('\"');\n//          sb.append(((String) value).replace(\"\\n\", \"\\\\n\"));\n          sb.append(value);\n          sb.append('\"');\n      } else if (value instanceof byte[]) {\n          final byte[] bytes = ((byte[]) value);\n          sb.append(\"<<\");\n          for (final byte b : bytes) {\n              sb.append(b);\n          }\n          sb.append(\">>\");\n      } else if (value instanceof Map<?, ?>) {\n          @SuppressWarnings(\"unchecked\")\n          final\n          Map<String, Object> map = ((Map<String, Object>) value);\n          sb.append('{');\n          for (final Map.Entry<String, Object> entry : map.entrySet()) {\n              valueToStr(entry.getKey(), sb);\n              sb.append('=');\n              valueToStr(entry.getValue(), sb);\n              sb.append(',');\n          }\n          // remove last \",\"\n          if (map.size() > 0) {\n              sb.deleteCharAt(sb.length() - 1);\n          }\n          sb.append('}');\n      } else if (value instanceof List<?>) {\n          @SuppressWarnings(\"unchecked\")\n          final\n          List<Object> list = (List<Object>) value;\n          sb.append('[');\n          for (final Object object : list) {\n              valueToStr(object, sb);\n              sb.append(',');\n          }\n          // remove last \",\"\n          if (list.size() > 0) {\n              sb.deleteCharAt(sb.length() - 1);\n          }\n          sb.append(']');\n      } else if (value instanceof ErlangValue) {\n          sb.append(((ErlangValue) value).value().toString());\n      } else {\n          sb.append(value.toString());\n      }\n    }\n\n    private static int read_write_boolean(final String basekey, final TransactionSingleOp sc, final Mode mode) {\n        int failed = 0;\n        String key;\n        Object value;\n\n        key = basekey + \"_bool_false\"; value = false;\n        failed += read_or_write(sc, key, value, mode);\n\n        key = basekey + \"_bool_true\"; value = true;\n        failed += read_or_write(sc, key, value, mode);\n\n        return failed;\n    }\n\n    private static int read_write_integer(final String basekey, final TransactionSingleOp sc, final Mode mode) {\n        int failed = 0;\n        String key;\n        Object value;\n\n        key = basekey + \"_int_0\"; value = 0;\n        failed += read_or_write(sc, key, value, mode);\n\n        key = basekey + \"_int_1\"; value = 1;\n        failed += read_or_write(sc, key, value, mode);\n\n//        lastKey = basekey + \"_int_min\"; lastValue = Integer.MIN_VALUE;\n        key = basekey + \"_int_min\"; value = -2147483648;\n        failed += read_or_write(sc, key, value, mode);\n\n//        lastKey = basekey + \"_int_max\"; lastValue = Integer.MAX_VALUE;\n        key = basekey + \"_int_max\"; value = 2147483647;\n        failed += read_or_write(sc, key, value, mode);\n\n//        lastKey = basekey + \"_int_max_div_2\"; lastValue = Integer.MAX_VALUE / 2;\n        key = basekey + \"_int_max_div_2\"; value = 2147483647 / 2;\n        failed += read_or_write(sc, key, value, mode);\n\n        return failed;\n    }\n\n    private static int read_write_long(final String basekey, final TransactionSingleOp sc, final Mode mode) {\n        int failed = 0;\n        String key;\n        Object value;\n\n        key = basekey + \"_long_0\"; value = 0l;\n        failed += read_or_write(sc, key, value, mode);\n\n        key = basekey + \"_long_1\"; value = 1l;\n        failed += read_or_write(sc, key, value, mode);\n\n//        lastKey = basekey + \"_long_min\"; lastValue = Long.MIN_VALUE;\n        key = basekey + \"_long_min\"; value = -9223372036854775808l;\n        failed += read_or_write(sc, key, value, mode);\n\n//        lastKey = basekey + \"_long_max\"; lastValue = Long.MAX_VALUE;\n        key = basekey + \"_long_max\"; value = 9223372036854775807l;\n        failed += read_or_write(sc, key, value, mode);\n\n//        lastKey = basekey + \"_long_max_div_2\"; lastValue = Long.MAX_VALUE / 2l;\n        key = basekey + \"_long_max_div_2\"; value = 9223372036854775807l / 2l;\n        failed += read_or_write(sc, key, value, mode);\n\n        return failed;\n    }\n\n    private static int read_write_biginteger(final String basekey, final TransactionSingleOp sc, final Mode mode) {\n        int failed = 0;\n        String key;\n        Object value;\n\n        key = basekey + \"_bigint_0\"; value = new BigInteger(\"0\");\n        failed += read_or_write(sc, key, value, mode);\n\n        key = basekey + \"_bigint_1\"; value = new BigInteger(\"1\");\n        failed += read_or_write(sc, key, value, mode);\n\n        key = basekey + \"_bigint_min\"; value = new BigInteger(\"-100000000000000000000\");\n        failed += read_or_write(sc, key, value, mode);\n\n        key = basekey + \"_bigint_max\"; value = new BigInteger(\"100000000000000000000\");\n        failed += read_or_write(sc, key, value, mode);\n\n        key = basekey + \"_bigint_max_div_2\"; value = new BigInteger(\"100000000000000000000\").divide(new BigInteger(\"2\"));\n        failed += read_or_write(sc, key, value, mode);\n\n        return failed;\n    }\n\n    private static int read_write_double(final String basekey, final TransactionSingleOp sc, final Mode mode) {\n        int failed = 0;\n        String key;\n        Object value;\n\n        key = basekey + \"_float_0.0\"; value = 0.0;\n        failed += read_or_write(sc, key, value, mode);\n\n        key = basekey + \"_float_1.5\"; value = 1.5;\n        failed += read_or_write(sc, key, value, mode);\n\n        key = basekey + \"_float_-1.5\"; value = -1.5;\n        failed += read_or_write(sc, key, value, mode);\n\n//        lastKey = basekey + \"_float_min\"; lastValue = Double.MIN_VALUE;\n        key = basekey + \"_float_min\"; value = 4.9E-324;\n        failed += read_or_write(sc, key, value, mode);\n\n//        lastKey = basekey + \"_float_max\"; lastValue = Double.MAX_VALUE;\n        key = basekey + \"_float_max\"; value = 1.7976931348623157E308;\n        failed += read_or_write(sc, key, value, mode);\n\n//        lastKey = basekey + \"_float_max_div_2\"; lastValue = Double.MAX_VALUE / 2.0;\n        key = basekey + \"_float_max_div_2\"; value = 1.7976931348623157E308 / 2.0;\n        failed += read_or_write(sc, key, value, mode);\n\n        // not supported by OTP:\n//        key = basekey + \"_float_neg_inf\"; value = Double.NEGATIVE_INFINITY;\n//        failed += read_or_write(sc, key, value, mode);\n//        key = basekey + \"_float_pos_inf\"; value = Double.POSITIVE_INFINITY;\n//        failed += read_or_write(sc, key, value, mode);\n//        key = basekey + \"_float_nan\"; value = Double.NaN;\n//        failed += read_or_write(sc, key, value, mode);\n\n        return failed;\n    }\n\n    private static int read_write_string(final String basekey, final TransactionSingleOp sc, final Mode mode) {\n        int failed = 0;\n        String key;\n        Object value;\n\n        key = basekey + \"_string_empty\"; value = \"\";\n        failed += read_or_write(sc, key, value, mode);\n\n        key = basekey + \"_string_foobar\"; value = \"foobar\";\n        failed += read_or_write(sc, key, value, mode);\n\n        key = basekey + \"_string_foo\\\\nbar\"; value = \"foo\\nbar\";\n        failed += read_or_write(sc, key, value, mode);\n\n        // some (arbitrary) unicode characters\n        // (please don't be offended if they actually mean something)\n        key = basekey + \"_string_unicode\"; value = \"foo\\u0180\\u01E3\\u11E5\";\n        failed += read_or_write(sc, key, value, mode);\n\n        return failed;\n    }\n\n    private static int read_write_binary(final String basekey, final TransactionSingleOp sc, final Mode mode) {\n        int failed = 0;\n        String key;\n        Object value;\n\n        key = basekey + \"_byte_empty\"; value = new byte[0];\n        failed += read_or_write(sc, key, value, mode);\n\n        key = basekey + \"_byte_0\"; value = new byte[] {0};\n        failed += read_or_write(sc, key, value, mode);\n\n        key = basekey + \"_byte_0123\"; value = new byte[] {0, 1, 2, 3};\n        failed += read_or_write(sc, key, value, mode);\n\n        return failed;\n    }\n\n    private static int read_write_list(final String basekey, final TransactionSingleOp sc, final Mode mode) {\n        int failed = 0;\n        String key;\n        Object value;\n\n        key = basekey + \"_list_empty\"; value = new ArrayList<Object>();\n        failed += read_or_write(sc, key, value, mode);\n\n        final ArrayList<Integer> list1 = new ArrayList<Integer>();\n        list1.add(0); list1.add(1); list1.add(2); list1.add(3);\n        key = basekey + \"_list_0_1_2_3\"; value = list1;\n        failed += read_or_write(sc, key, value, mode);\n\n        final ArrayList<Integer> list2 = new ArrayList<Integer>();\n        list2.add(0); list2.add(123); list2.add(456); list2.add(65000);\n        key = basekey + \"_list_0_123_456_65000\"; value = list2;\n        failed += read_or_write(sc, key, value, mode);\n\n        final ArrayList<Integer> list3 = new ArrayList<Integer>();\n        list3.add(0); list3.add(123); list3.add(456); list3.add(0x10ffff);\n        key = basekey + \"_list_0_123_456_0x10ffff\"; value = list3;\n        failed += read_or_write(sc, key, value, mode);\n\n        final ArrayList<Object> list4 = new ArrayList<Object>();\n        list4.add(0);\n        list4.add(\"foo\");\n        list4.add(1.5);\n        list4.add(false);\n        key = basekey + \"_list_0_foo_1.5_false\"; value = list4;\n        failed += read_or_write(sc, key, value, mode);\n\n        // note: we do not support binaries in lists anymore because JSON would\n        // need special handling for each list element which introduces too\n        // much overhead\n//        ArrayList<Object> list5 = new ArrayList<Object>();\n//        list5.add(0);\n//        list5.add(\"foo\");\n//        list5.add(1.5);\n//        list5.add(new byte[] {0, 1, 2, 3});\n//        key = basekey + \"_list_0_foo_1.5_<<0123>>\"; value = list5;\n//        failed += read_or_write(sc, key, value, mode);\n\n        final ArrayList<Object> list6 = new ArrayList<Object>();\n        list6.add(0);\n        list6.add(\"foo\");\n        list6.add(1.5);\n        list6.add(false);\n        list6.add(list4);\n        key = basekey + \"_list_0_foo_1.5_false_list4\"; value = list6;\n        failed += read_or_write(sc, key, value, mode);\n\n        final LinkedHashMap<String, Integer> map1 = new LinkedHashMap<String, Integer>();\n        map1.put(\"x\", 0);\n        map1.put(\"y\", 1);\n        final ArrayList<Object> list7 = new ArrayList<Object>();\n        list7.add(0);\n        list7.add(\"foo\");\n        list7.add(1.5);\n        list7.add(false);\n        list7.add(list4);\n        list7.add(map1);\n        key = basekey + \"_list_0_foo_1.5_false_list4_map_x=0_y=1\"; value = list7;\n        failed += read_or_write(sc, key, value, mode);\n\n        return failed;\n    }\n\n    // Map/JSON:\n    private static int read_write_map(final String basekey, final TransactionSingleOp sc, final Mode mode) {\n        int failed = 0;\n        String key;\n        Object value;\n\n        key = basekey + \"_map_empty\";\n        value = new LinkedHashMap<String, Object>();\n        failed += read_or_write(sc, key, value, mode);\n\n        final LinkedHashMap<String, Integer> map1 = new LinkedHashMap<String, Integer>();\n        map1.put(\"x\", 0);\n        map1.put(\"y\", 1);\n        key = basekey + \"_map_x=0_y=1\";\n        value = map1;\n        failed += read_or_write(sc, key, value, mode);\n\n        final LinkedHashMap<String, Object> map2 = new LinkedHashMap<String, Object>();\n        map2.put(\"a\", 0);\n        map2.put(\"b\", \"foo\");\n        map2.put(\"c\", 1.5);\n        map2.put(\"d\", \"foo\\nbar\");\n        final ArrayList<Integer> list1 = new ArrayList<Integer>();\n        list1.add(0);\n        list1.add(1);\n        list1.add(2);\n        list1.add(3);\n        map2.put(\"e\", list1);\n        map2.put(\"f\", map1);\n        key = basekey + \"_map_a=0_b=foo_c=1.5_d=foo<nl>bar_e=list0123_f=mapx0y1\";\n        value = map2;\n        failed += read_or_write(sc, key, value, mode);\n\n        final LinkedHashMap<String, Integer> map3 = new LinkedHashMap<String, Integer>();\n        map3.put(\"\", 0);\n        key = basekey + \"_map_=0\";\n        value = map3;\n        failed += read_or_write(sc, key, value, mode);\n\n        final LinkedHashMap<String, Object> map4 = new LinkedHashMap<String, Object>();\n        // some (arbitrary) unicode characters\n        // (please don't be offended if they actually mean something)\n        map4.put(\"x\", 0);\n        map4.put(\"y\", \"foo\\u0180\\u01E3\\u11E5\");\n        key = basekey + \"_map_x=0_y=foo\\u0180\\u01E3\\u11E5\";\n        value = map4;\n        failed += read_or_write(sc, key, value, mode);\n\n        final LinkedHashMap<String, Integer> map5 = new LinkedHashMap<String, Integer>();\n        // some (arbitrary) unicode characters\n        // (please don't be offended if they actually mean something)\n        map5.put(\"x\", 0);\n        map5.put(\"foo\\u0180\\u01E3\\u11E5\", 1);\n        key = basekey + \"_map_x=0_foo\\u0180\\u01E3\\u11E5=1\";\n        value = map5;\n        failed += read_or_write(sc, key, value, mode);\n\n        return failed;\n    }\n\n\n\n    /**\n     * Creates the options the command line should understand.\n     *\n     * @return the options the program understands\n     */\n    private static Options getOptions() {\n        final Options options = new Options();\n        final OptionGroup group = new OptionGroup();\n\n        /* Note: arguments are set to be optional since we implement argument\n         * checks on our own (commons.cli is not flexible enough and only\n         * checks for the existence of a first argument)\n         */\n\n        options.addOption(new Option(\"h\", \"help\", false, \"print this message\"));\n\n        options.addOption(new Option(\"v\", \"verbose\", false, \"print verbose information, e.g. the properties read\"));\n\n        final Option read = new Option(\"r\", \"read\", true, \"read an item\");\n        read.setArgName(\"basekey> <language\");\n        read.setArgs(2);\n        read.setOptionalArg(true);\n        group.addOption(read);\n\n        final Option write = new Option(\"w\", \"write\", true, \"write an item\");\n        write.setArgName(\"basekey\");\n        write.setArgs(1);\n        write.setOptionalArg(true);\n        group.addOption(write);\n\n        options.addOptionGroup(group);\n\n        options.addOption(new Option(\"lh\", \"localhost\", false, \"gets the local host's name as known to Java (for debugging purposes)\"));\n\n        return options;\n    }\n}\n"
  },
  {
    "path": "java-api/test/de/zib/scalaris/MonitorTest.java",
    "content": "/**\n *  Copyright 2012 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport static org.junit.Assert.*;\n\nimport java.util.Map.Entry;\n\nimport org.junit.Test;\n\nimport de.zib.scalaris.Monitor.GetNodeInfoResult;\nimport de.zib.scalaris.Monitor.GetNodePerformanceResult;\nimport de.zib.scalaris.Monitor.GetServiceInfoResult;\nimport de.zib.scalaris.Monitor.GetServicePerformanceResult;\n\n/**\n * Unit tests for the {@link Monitor} class.\n *\n * @author Nico Kruber, kruber@zib.de\n * @since 3.11\n */\npublic class MonitorTest {\n\n    final static String scalarisNode;\n\n    static {\n        // determine good/bad nodes:\n        final ConnectionFactory cf = ConnectionFactory.getInstance();\n        cf.testAllNodes();\n        // set not to automatically try reconnects (auto-retries prevent ConnectionException tests from working):\n        final DefaultConnectionPolicy cp = ((DefaultConnectionPolicy) cf.getConnectionPolicy());\n        cp.setMaxRetries(0);\n        scalarisNode = cp.selectNode().toString();\n    }\n\n    /**\n     * Test method for {@link Monitor#Monitor(String)}.\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public final void testMonitor() throws ConnectionException {\n        final Monitor conn = new Monitor(scalarisNode);\n        conn.closeConnection();\n    }\n\n    /**\n     * Test method for {@link Monitor#closeConnection()} trying to close the\n     * connection twice.\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public void testDoubleClose() throws ConnectionException {\n        final Monitor conn = new Monitor(scalarisNode);\n        conn.closeConnection();\n        conn.closeConnection();\n    }\n\n    /**\n     * Test method for {@link Monitor#getNodeInfo()} with a closed connection.\n     *\n     * @throws ConnectionException\n     */\n    @Test(expected = ConnectionException.class)\n    public final void testGetNodeInfo_NotConnected() throws ConnectionException {\n        final Monitor conn = new Monitor(scalarisNode);\n        conn.closeConnection();\n        conn.getNodeInfo();\n    }\n\n    /**\n     * Test method for {@link Monitor#getNodeInfo()}.\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public final void testGetNodeInfo1() throws ConnectionException {\n        final Monitor conn = new Monitor(scalarisNode);\n        try {\n            final GetNodeInfoResult nodeInfo = conn.getNodeInfo();\n            assertTrue(nodeInfo.dhtNodes >= 0);\n            assertTrue(!nodeInfo.scalarisVersion.isEmpty());\n            assertTrue(!nodeInfo.erlangVersion.isEmpty());\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Monitor#getNodePerformance()} with a closed\n     * connection.\n     *\n     * @throws ConnectionException\n     */\n    @Test(expected = ConnectionException.class)\n    public final void testGetNodePerformance_NotConnected()\n            throws ConnectionException {\n        final Monitor conn = new Monitor(scalarisNode);\n        conn.closeConnection();\n        conn.getNodePerformance();\n    }\n\n    /**\n     * Test method for {@link Monitor#getNodePerformance()}.\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public final void testGetNodePerformance1() throws ConnectionException {\n        final Monitor conn = new Monitor(scalarisNode);\n        try {\n            final GetNodePerformanceResult nodePerformance = conn.getNodePerformance();\n            for (final Entry<Long, Double> latencyAvg : nodePerformance.latencyAvg.entrySet()) {\n                assertTrue(latencyAvg.toString(), latencyAvg.getKey() >= 0);\n                assertTrue(latencyAvg.toString(), latencyAvg.getValue() >= 0);\n            }\n            for (final Entry<Long, Double> latencyStddev : nodePerformance.latencyStddev.entrySet()) {\n                assertTrue(latencyStddev.toString(), latencyStddev.getKey() >= 0);\n                assertTrue(latencyStddev.toString(), latencyStddev.getValue() >= 0);\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Monitor#getServiceInfo()} with a closed\n     * connection.\n     *\n     * @throws ConnectionException\n     */\n    @Test(expected = ConnectionException.class)\n    public final void testGetServiceInfo_NotConnected()\n            throws ConnectionException {\n        final Monitor conn = new Monitor(scalarisNode);\n        conn.closeConnection();\n        conn.getServiceInfo();\n    }\n\n    /**\n     * Test method for {@link Monitor#getServiceInfo()}.\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public final void testGetServiceInfo1() throws ConnectionException {\n        final Monitor conn = new Monitor(scalarisNode);\n        try {\n            final GetServiceInfoResult serviceInfo = conn.getServiceInfo();\n            assertTrue(serviceInfo.nodes >= 0);\n            assertTrue(serviceInfo.totalLoad >= 0);\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Monitor#getServicePerformance()} with a closed\n     * connection.\n     *\n     * @throws ConnectionException\n     */\n    @Test(expected = ConnectionException.class)\n    public final void testGetServicePerformance_NotConnected()\n            throws ConnectionException {\n        final Monitor conn = new Monitor(scalarisNode);\n        conn.closeConnection();\n        conn.getServicePerformance();\n    }\n\n    /**\n     * Test method for {@link Monitor#getServicePerformance()}.\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public final void testGetServicePerformance1() throws ConnectionException {\n        final Monitor conn = new Monitor(scalarisNode);\n        try {\n            final GetServicePerformanceResult nodePerformance = conn.getServicePerformance();\n            for (final Entry<Long, Double> latencyAvg : nodePerformance.latencyAvg.entrySet()) {\n                assertTrue(latencyAvg.toString(), latencyAvg.getKey() >= 0);\n                assertTrue(latencyAvg.toString(), latencyAvg.getValue() >= 0);\n            }\n            for (final Entry<Long, Double> latencyStddev : nodePerformance.latencyStddev.entrySet()) {\n                assertTrue(latencyStddev.toString(), latencyStddev.getKey() >= 0);\n                assertTrue(latencyStddev.toString(), latencyStddev.getValue() >= 0);\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/test/de/zib/scalaris/PeerNodeTest.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport static org.junit.Assert.*;\n\nimport java.util.Date;\nimport java.util.concurrent.TimeUnit;\n\nimport org.junit.Test;\n\nimport com.ericsson.otp.erlang.OtpPeer;\n\n/**\n * Test cases for the {@link PeerNode} class.\n *\n * @author Nico Kruber, kruber@zib.de\n *\n * @version 2.3\n * @since 2.3\n */\npublic class PeerNodeTest {\n    /**\n     * Test method for {@link PeerNode#PeerNode(OtpPeer)}.\n     */\n    @Test\n    public final void testPeerNodeOtpPeer() {\n        final PeerNode p = new PeerNode(new OtpPeer(\"test@localhost\"));\n        assertEquals(\"test@localhost\", p.getNode().node());\n    }\n\n    /**\n     * Test method for {@link PeerNode#PeerNode(String)}.\n     */\n    @Test\n    public final void testPeerNodeString() {\n        final PeerNode p = new PeerNode(\"test@localhost\");\n        assertEquals(\"test@localhost\", p.getNode().node());\n    }\n\n    /**\n     * Test method for {@link PeerNode#getLastFailedConnect()},\n     * {@link PeerNode#setLastFailedConnect()} and\n     * {@link PeerNode#getFailureCount()}.\n     *\n     * @throws InterruptedException\n     *             if the sleep is interrupted\n     */\n    @Test\n    public final void testGetSetLastFailedConnect() throws InterruptedException {\n        final PeerNode p = new PeerNode(\"test\");\n        final Date d0 = new Date();\n\n        assertEquals(0, p.getFailureCount());\n        assertEquals(null, p.getLastFailedConnect());\n        TimeUnit.MILLISECONDS.sleep(10);\n\n        p.setLastFailedConnect();\n        final Date d1 = p.getLastFailedConnect();\n        assertEquals(1, p.getFailureCount());\n        assertNotNull(d1);\n        assertTrue(d0.getTime() < d1.getTime());\n        TimeUnit.MILLISECONDS.sleep(10);\n\n        p.setLastFailedConnect();\n        final Date d2 = p.getLastFailedConnect();\n        assertEquals(2, p.getFailureCount());\n        assertNotNull(d2);\n        assertTrue(d0.getTime() < d2.getTime());\n        assertTrue(d1.getTime() < d2.getTime());\n    }\n\n    /**\n     * Test method for {@link PeerNode#resetFailureCount()}.\n     */\n    @Test\n    public final void testResetFailureCount() {\n        final PeerNode p = new PeerNode(\"test\");\n\n        p.resetFailureCount();\n        assertEquals(0, p.getFailureCount());\n        assertEquals(null, p.getLastFailedConnect());\n\n        p.setLastFailedConnect();\n\n        p.resetFailureCount();\n        assertEquals(0, p.getFailureCount());\n        assertEquals(null, p.getLastFailedConnect());\n\n        p.setLastFailedConnect();\n        p.setLastFailedConnect();\n\n        p.resetFailureCount();\n        assertEquals(0, p.getFailureCount());\n        assertEquals(null, p.getLastFailedConnect());\n    }\n\n    /**\n     * Test method for {@link PeerNode#getLastConnectSuccess()} and\n     * {@link PeerNode#setLastConnectSuccess()}.\n     *\n     * @throws InterruptedException\n     *             if the sleep is interrupted\n     */\n    @Test\n    public final void testGetSetLastConnectSuccess()\n            throws InterruptedException {\n        final PeerNode p = new PeerNode(\"test\");\n        final Date d0 = new Date();\n\n        assertEquals(null, p.getLastConnectSuccess());\n\n        TimeUnit.MILLISECONDS.sleep(10);\n        p.setLastConnectSuccess();\n        final Date d1 = p.getLastConnectSuccess();\n        assertNotNull(d1);\n        assertTrue(d0.getTime() < d1.getTime());\n\n        TimeUnit.MILLISECONDS.sleep(10);\n        p.setLastConnectSuccess();\n        final Date d2 = p.getLastConnectSuccess();\n        assertNotNull(d2);\n        assertTrue(d0.getTime() < d2.getTime());\n        assertTrue(d1.getTime() < d2.getTime());\n    }\n\n}\n"
  },
  {
    "path": "java-api/test/de/zib/scalaris/ReplicatedDHTTest.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport static org.junit.Assert.assertEquals;\nimport static org.junit.Assert.assertTrue;\n\nimport org.junit.Test;\n\n/**\n * Unit test for the {@link ReplicatedDHT} class.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 2.6\n * @since 2.6\n */\npublic class ReplicatedDHTTest {\n    private final static long testTime = System.currentTimeMillis();\n\n    private final static String[] testData = {\n        \"ahz2ieSh\", \"wooPhu8u\", \"quai9ooK\", \"Oquae4ee\", \"Airier1a\", \"Boh3ohv5\", \"ahD3Saog\", \"EM5ooc4i\",\n        \"Epahrai8\", \"laVahta7\", \"phoo6Ahj\", \"Igh9eepa\", \"aCh4Lah6\", \"ooT0ath5\", \"uuzau4Ie\", \"Iup6mae6\",\n//        \"xie7iSie\", \"ail8yeeP\", \"ooZ4eesi\", \"Ahn7ohph\", \"Ohy5moo6\", \"xooSh9Oo\", \"ieb6eeS7\", \"Thooqu9h\",\n//        \"eideeC9u\", \"phois3Ie\", \"EimaiJ2p\", \"sha6ahR1\", \"Pheih3za\", \"bai4eeXe\", \"rai0aB7j\", \"xahXoox6\",\n//        \"Xah4Okeg\", \"cieG8Yae\", \"Pe9Ohwoo\", \"Eehig6ph\", \"Xe7rooy6\", \"waY2iifu\", \"kemi8AhY\", \"Che7ain8\",\n//        \"ohw6seiY\", \"aegh1oBa\", \"thoh9IeG\", \"Kee0xuwu\", \"Gohng8ee\", \"thoh9Chi\", \"aa4ahQuu\", \"Iesh5uge\",\n//        \"Ahzeil8n\", \"ieyep5Oh\", \"xah3IXee\", \"Eefa5qui\", \"kai8Muuf\", \"seeCe0mu\", \"cooqua5Y\", \"Ci3ahF6z\",\n//        \"ot0xaiNu\", \"aewael8K\", \"aev3feeM\", \"Fei7ua5t\", \"aeCa6oph\", \"ag2Aelei\", \"Shah1Pho\", \"ePhieb0N\",\n//        \"Uqu7Phup\", \"ahBi8voh\", \"oon3aeQu\", \"Koopa0nu\", \"xi0quohT\", \"Oog4aiph\", \"Aip2ag5D\", \"tirai7Ae\",\n//        \"gi0yoePh\", \"uay7yeeX\", \"aeb6ahC1\", \"OoJeic2a\", \"ieViom1y\", \"di0eeLai\", \"Taec2phe\", \"ID2cheiD\",\n//        \"oi6ahR5M\", \"quaiGi8W\", \"ne1ohLuJ\", \"DeD0eeng\", \"yah8Ahng\", \"ohCee2ie\", \"ecu1aDai\", \"oJeijah4\",\n//        \"Goo9Una1\", \"Aiph3Phi\", \"Ieph0ce5\", \"ooL6cae7\", \"nai0io1H\", \"Oop2ahn8\", \"ifaxae7O\", \"NeHai1ae\",\n//        \"Ao8ooj6a\", \"hi9EiPhi\", \"aeTh9eiP\", \"ao8cheiH\", \"Yieg3sha\", \"mah7cu2D\", \"Uo5wiegi\", \"Oowei0ya\",\n//        \"efeiDee7\", \"Oliese6y\", \"eiSh1hoh\", \"Joh6hoh9\", \"zib6Ooqu\", \"eejiJie4\", \"lahZ3aeg\", \"keiRai1d\",\n//        \"Fei0aewe\", \"aeS8aboh\", \"hae3ohKe\", \"Een9ohQu\", \"AiYeeh7o\", \"Yaihah4s\", \"ood4Giez\", \"Oumai7te\",\n//        \"hae2kahY\", \"afieGh4v\", \"Ush0boo0\", \"Ekootee5\", \"Ya8iz6Ie\", \"Poh6dich\", \"Eirae4Ah\", \"pai8Eeme\",\n//        \"uNah7dae\", \"yo3hahCh\", \"teiTh7yo\", \"zoMa5Cuv\", \"ThiQu5ax\", \"eChi5caa\", \"ii9ujoiV\", \"ge7Iekui\",\n        \"sai2aiTa\", \"ohKi9rie\", \"ei2ioChu\", \"aaNgah9y\", \"ooJai1Ie\", \"shoh0oH9\", \"Ool4Ahya\", \"poh0IeYa\",\n        \"Uquoo0Il\", \"eiGh4Oop\", \"ooMa0ufe\", \"zee6Zooc\", \"ohhao4Ah\", \"Uweekek5\", \"aePoos9I\", \"eiJ9noor\",\n        \"phoong1E\", \"ianieL2h\", \"An7ohs4T\", \"Eiwoeku3\", \"sheiS3ao\", \"nei5Thiw\", \"uL5iewai\", \"ohFoh9Ae\"};\n\n    static {\n        // determine good/bad nodes:\n        final ConnectionFactory cf = ConnectionFactory.getInstance();\n        cf.testAllNodes();\n        // set not to automatically try reconnects (auto-retries prevent ConnectionException tests from working):\n        ((DefaultConnectionPolicy) cf.getConnectionPolicy()).setMaxRetries(0);\n    }\n\n    /**\n     * Test method for\n     * {@link ReplicatedDHT#ReplicatedDHT()}.\n     * @throws ConnectionException\n     */\n    @Test\n    public void testReplicatedDHT1() throws ConnectionException {\n        final ReplicatedDHT rdht = new ReplicatedDHT();\n        rdht.closeConnection();\n    }\n\n    /**\n     * Test method for\n     * {@link ReplicatedDHT#ReplicatedDHT(Connection)}.\n     * @throws ConnectionException\n     */\n    @Test\n    public void testReplicatedDHT2() throws ConnectionException {\n        final ReplicatedDHT rdht = new ReplicatedDHT(ConnectionFactory.getInstance().createConnection(\"test\"));\n        rdht.closeConnection();\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#closeConnection()} trying to\n     * close the connection twice.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     */\n    @Test\n    public void testDoubleClose() throws ConnectionException {\n        final TransactionSingleOp rdht = new TransactionSingleOp();\n        rdht.closeConnection();\n        rdht.closeConnection();\n    }\n\n    /**\n     * Tries to read the value at the given <tt>key</tt> and fails if this does\n     * not fail with a {@link NotFoundException}.\n     *\n     * @param key the key to look up\n     *\n     * @throws ConnectionException\n     * @throws TimeoutException\n     * @throws UnknownException\n     */\n    private void checkKeyDoesNotExist(final String key) throws ConnectionException,\n            TimeoutException, UnknownException {\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        try {\n            conn.read(key);\n            // a not found exception must be thrown\n            assertTrue(false);\n        } catch (final NotFoundException e) {\n            // nothing to do here\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link ReplicatedDHT#delete(String)}.\n     * Tries to delete some not existing keys.\n     *\n     * @throws UnknownException\n     * @throws TimeoutException\n     * @throws ConnectionException\n     */\n    @Test\n    public void testDelete_notExistingKey() throws ConnectionException,\n            TimeoutException, UnknownException {\n        final String key = \"_Delete_NotExistingKey\";\n        final Connection c = ConnectionFactory.getInstance().createConnection(\"test\");\n        final ReplicatedDHT rdht = new ReplicatedDHT(c);\n        final RoutingTable rt = new RoutingTable(c);\n        final int r = rt.getReplicationFactor();\n\n        try {\n            for (int i = 0; i < testData.length; ++i) {\n                final DeleteResult result0 = rdht.delete(testTime + key + i);\n                final DeleteResult result = rdht.getLastDeleteResult();\n                assertEquals(result0, result);\n                assertEquals(true, result.hasDeletedAll(c));\n                assertEquals(0, result.ok);\n                assertEquals(0, result.locks_set);\n                assertEquals(r, result.undef);\n\n                // make sure the key does not exist afterwards:\n                checkKeyDoesNotExist(testTime + key + i);\n            }\n        } finally {\n            c.close();\n        }\n    }\n\n    /**\n     * Test method for {@link ReplicatedDHT#delete(String)} and\n     * {@link TransactionSingleOp#write(String, Object)}.\n     * Inserts some values, tries to delete them afterwards and tries the\n     * delete again.\n     *\n     * @throws UnknownException\n     * @throws TimeoutException\n     * @throws ConnectionException\n     * @throws AbortException\n     */\n    @Test\n    public void testDelete1() throws ConnectionException,\n            TimeoutException, UnknownException, AbortException {\n        final String key = \"_Delete1\";\n        final Connection c = ConnectionFactory.getInstance().createConnection(\"test\");\n        final ReplicatedDHT rdht = new ReplicatedDHT(c);\n        final TransactionSingleOp sc = new TransactionSingleOp(c);\n        final RoutingTable rt = new RoutingTable(c);\n        final int r = rt.getReplicationFactor();\n\n        try {\n            for (int i = 0; i < testData.length; ++i) {\n                sc.write(testTime + key + i, testData[i]);\n            }\n\n            // now try to delete the data:\n            for (int i = 0; i < testData.length; ++i) {\n                DeleteResult result0 = rdht.delete(testTime + key + i);\n                DeleteResult result = rdht.getLastDeleteResult();\n                assertEquals(result0, result);\n                assertEquals(true, result.hasDeletedAll(c));\n                assertEquals(r, result.ok);\n                assertEquals(0, result.locks_set);\n                assertEquals(0, result.undef);\n\n                // make sure the key does not exist afterwards:\n                checkKeyDoesNotExist(testTime + key + i);\n\n                // try again (should be successful with 0 deletes)\n                result0 = rdht.delete(testTime + key + i);\n                result = rdht.getLastDeleteResult();\n                assertEquals(result0, result);\n                assertEquals(true, result.hasDeletedAll(c));\n                assertEquals(0, result.ok);\n                assertEquals(0, result.locks_set);\n                assertEquals(r, result.undef);\n\n                // make sure the key does not exist afterwards:\n                checkKeyDoesNotExist(testTime + key + i);\n            }\n        } finally {\n            c.close();\n        }\n    }\n\n    /**\n     * Test method for {@link ReplicatedDHT#delete(String)} and\n     * {@link TransactionSingleOp#write(String, Object)}.\n     * Inserts some values, tries to delete them afterwards, inserts them again\n     * and tries to delete them again (twice).\n     *\n     * @throws UnknownException\n     * @throws TimeoutException\n     * @throws ConnectionException\n     * @throws AbortException\n     */\n    @Test\n    public void testDelete2() throws ConnectionException,\n            TimeoutException, UnknownException, AbortException {\n        final String key = \"_Delete2\";\n        final Connection c = ConnectionFactory.getInstance().createConnection(\"test\");\n        final ReplicatedDHT rdht = new ReplicatedDHT(c);\n        final TransactionSingleOp sc = new TransactionSingleOp(c);\n        final RoutingTable rt = new RoutingTable(c);\n        final int r = rt.getReplicationFactor();\n\n        try {\n            for (int i = 0; i < testData.length; ++i) {\n                sc.write(testTime + key + i, testData[i]);\n            }\n\n            // now try to delete the data:\n            for (int i = 0; i < testData.length; ++i) {\n                final DeleteResult result0 = rdht.delete(testTime + key + i);\n                final DeleteResult result = rdht.getLastDeleteResult();\n                assertEquals(result0, result);\n                assertEquals(true, result.hasDeletedAll(c));\n                assertEquals(r, result.ok);\n                assertEquals(0, result.locks_set);\n                assertEquals(0, result.undef);\n\n                // make sure the key does not exist afterwards:\n                checkKeyDoesNotExist(testTime + key + i);\n            }\n\n            for (int i = 0; i < testData.length; ++i) {\n                sc.write(testTime + key + i, testData[i]);\n            }\n\n            // now try to delete the data:\n            for (int i = 0; i < testData.length; ++i) {\n                DeleteResult result0 = rdht.delete(testTime + key + i);\n                DeleteResult result = rdht.getLastDeleteResult();\n                assertEquals(result0, result);\n                assertEquals(true, result.hasDeletedAll(c));\n                assertEquals(r, result.ok);\n                assertEquals(0, result.locks_set);\n                assertEquals(0, result.undef);\n\n                // make sure the key does not exist afterwards:\n                checkKeyDoesNotExist(testTime + key + i);\n\n                // try again (should be successful with 0 deletes)\n                result0 = rdht.delete(testTime + key + i);\n                result = rdht.getLastDeleteResult();\n                assertEquals(result0, result);\n                assertEquals(true, result.hasDeletedAll(c));\n                assertEquals(0, result.ok);\n                assertEquals(0, result.locks_set);\n                assertEquals(r, result.undef);\n\n                // make sure the key does not exist afterwards:\n                checkKeyDoesNotExist(testTime + key + i);\n            }\n        } finally {\n            c.close();\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/test/de/zib/scalaris/ScalarisTest.java",
    "content": "/**\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport static org.junit.Assert.assertTrue;\n\nimport java.util.List;\n\nimport org.junit.Test;\n\n/**\n * Test class for {@link Scalaris}.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.10\n * @since 3.10\n */\npublic class ScalarisTest {\n\n    static {\n        // determine good/bad nodes:\n        final ConnectionFactory cf = ConnectionFactory.getInstance();\n        cf.testAllNodes();\n        // set not to automatically try reconnects (auto-retries prevent ConnectionException tests from working):\n        ((DefaultConnectionPolicy) cf.getConnectionPolicy()).setMaxRetries(0);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.Scalaris#Scalaris()}.\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public final void testScalaris1() throws ConnectionException {\n        final Scalaris conn = new Scalaris();\n        conn.closeConnection();\n    }\n\n    /**\n     * Test method for\n     * {@link de.zib.scalaris.Scalaris#Scalaris(de.zib.scalaris.Connection)}\n     * .\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public final void testScalaris2() throws ConnectionException {\n        final Scalaris conn = new Scalaris(ConnectionFactory.getInstance().createConnection(\"test\"));\n        conn.closeConnection();\n    }\n\n    /**\n     * Test method for {@link Scalaris#closeConnection()} trying to close the\n     * connection twice.\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public void testDoubleClose() throws ConnectionException {\n        final Scalaris conn = new Scalaris();\n        conn.closeConnection();\n        conn.closeConnection();\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.Scalaris#getRandomNodes(int)} with\n     * a closed connection.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testGetRandomNodes_NotConnected() throws ConnectionException, InterruptedException {\n        final Scalaris conn = new Scalaris();\n        conn.closeConnection();\n        conn.getRandomNodes(1);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.Scalaris#getRandomNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testGetRandomNodes1() throws ConnectionException, InterruptedException {\n        testGetRandomNodesX(1);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.Scalaris#getRandomNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testGetRandomNodes2() throws ConnectionException, InterruptedException {\n        testGetRandomNodesX(2);\n    }\n\n    /**\n     * Test method for {@link de.zib.scalaris.Scalaris#getRandomNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testGetRandomNodes3() throws ConnectionException, InterruptedException {\n        testGetRandomNodesX(3);\n    }\n\n    private final void testGetRandomNodesX(final int max) throws ConnectionException, InterruptedException {\n        final Scalaris conn = new Scalaris();\n        try {\n            final List<String> result = conn.getRandomNodes(max);\n            assertTrue(\"list empty:\" + result.toString(), !result.isEmpty());\n            assertTrue(\"list too long:\" + result.toString(), result.size() <= max);\n            for (final String node : result) {\n                final ScalarisVM conn2 = new ScalarisVM(node);\n                try {\n                    conn2.getInfo();\n                } finally {\n                    conn2.closeConnection();\n                }\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/test/de/zib/scalaris/ScalarisVMTest.java",
    "content": "/**\n *  Copyright 2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport static org.junit.Assert.*;\n\nimport java.util.Arrays;\nimport java.util.Collections;\nimport java.util.List;\n\nimport org.junit.Ignore;\nimport org.junit.Test;\n\nimport de.zib.scalaris.ScalarisVM.AddNodesResult;\nimport de.zib.scalaris.ScalarisVM.DeleteNodesByNameResult;\nimport de.zib.scalaris.ScalarisVM.GetInfoResult;\n\n/**\n * Test class for {@link ScalarisVM}.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.6\n * @since 3.6\n */\npublic class ScalarisVMTest {\n\n    protected enum DeleteAction {\n        SHUTDOWN, KILL\n    }\n\n    final static String scalarisNode;\n\n    static {\n        // determine good/bad nodes:\n        final ConnectionFactory cf = ConnectionFactory.getInstance();\n        cf.testAllNodes();\n        // set not to automatically try reconnects (auto-retries prevent ConnectionException tests from working):\n        final DefaultConnectionPolicy cp = ((DefaultConnectionPolicy) cf.getConnectionPolicy());\n        cp.setMaxRetries(0);\n        scalarisNode = cp.selectNode().toString();\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#ScalarisVM(String)} .\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public final void testScalarisVM() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#closeConnection()} trying to close the\n     * connection twice.\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public void testDoubleClose() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.closeConnection();\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#getVersion()} with a closed connection.\n     *\n     * @throws ConnectionException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testGetVersion_NotConnected() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.getVersion();\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#getVersion()}.\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public final void testGetVersion1() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        try {\n            final String version = conn.getVersion();\n            assertTrue(!version.isEmpty());\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#getInfo()} with a closed connection.\n     *\n     * @throws ConnectionException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testGetInfo_NotConnected() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.getInfo();\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#getInfo()}.\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public final void testGetInfo1() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        try {\n            final GetInfoResult info = conn.getInfo();\n            //        System.out.println(info.scalarisVersion);\n            assertTrue(\"scalaris_version (\" + info.scalarisVersion + \") != \\\"\\\"\", !info.scalarisVersion.isEmpty());\n            assertTrue(\"erlang_version (\" + info.erlangVersion + \") != \\\"\\\"\", !info.erlangVersion.isEmpty());\n            assertTrue(\"mem_total (\" + info.memTotal + \") >= 0\", info.memTotal >= 0);\n            assertTrue(\"uptime (\" + info.uptime + \") >= 0\", info.uptime >= 0);\n            assertTrue(\"erlang_node (\" + info.erlangNode + \") != \\\"\\\"\", !info.erlangNode.isEmpty());\n            assertTrue(\"0 <= port (\" + info.port + \") <= 65535\", (info.port >= 0) && (info.port <= 65535));\n            assertTrue(\"0 <= yaws_port (\" + info.yawsPort + \") <= 65535\", (info.yawsPort >= 0) && (info.yawsPort <= 65535));\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#getNumberOfNodes()} with a closed\n     * connection.\n     *\n     * @throws ConnectionException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testGetNumberOfNodes_NotConnected() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.getNumberOfNodes();\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#getNumberOfNodes()}.\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public final void testGetNumberOfNodes1() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        try {\n            final int numberOfNodes = conn.getNumberOfNodes();\n            assertTrue(numberOfNodes >= 0);\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#getNodes()} with a closed connection.\n     *\n     * @throws ConnectionException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testGetNodes_NotConnected() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.getNodes();\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#getNodes()}.\n     *\n     * @throws ConnectionException\n     */\n    @Test\n    public final void testGetNodes1() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        try {\n            final List<ErlangValue> nodes = conn.getNodes();\n            assertEquals(conn.getNumberOfNodes(), nodes.size());\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#addNodes(int)} with a closed\n     * connection.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testAddNodes_NotConnected() throws ConnectionException, InterruptedException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.addNodes(1);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#addNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testAddNodes0() throws ConnectionException, InterruptedException {\n        testAddNodesX(0);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#addNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testAddNodes1() throws ConnectionException, InterruptedException {\n        testAddNodesX(1);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#addNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testAddNodes3() throws ConnectionException, InterruptedException {\n        testAddNodesX(3);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownNodes(int)} and\n     * {@link ScalarisVM#killNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    private final void testAddNodesX(final int nodesToAdd) throws ConnectionException, InterruptedException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        try {\n            int size = conn.getNumberOfNodes();\n            final AddNodesResult addedNodes = conn.addNodes(nodesToAdd);\n            size += nodesToAdd;\n            assertEquals(nodesToAdd, addedNodes.successful.size());\n            assertEquals(addedNodes.errors.length(), 0);\n            assertEquals(size, conn.getNumberOfNodes());\n            final List<ErlangValue> nodes = conn.getNodes();\n            for (final ErlangValue name : addedNodes.successful) {\n                assertTrue(nodes.toString() + \" should contain \" + name, nodes.contains(name));\n            }\n            for (final ErlangValue name : addedNodes.successful) {\n                conn.killNode(name);\n            }\n            size -= nodesToAdd;\n            assertEquals(size, conn.getNumberOfNodes());\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownNode(ErlangValue)} with a closed\n     * connection.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testShutdownNode_NotConnected() throws ConnectionException, InterruptedException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.shutdownNode(new ErlangValue(\"test\"));\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownNode(ErlangValue)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testShutdownNode1() throws ConnectionException, InterruptedException {\n        testDeleteNode(DeleteAction.SHUTDOWN);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#killNode(ErlangValue)} with a closed\n     * connection.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testKillNode_NotConnected() throws ConnectionException, InterruptedException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.killNode(new ErlangValue(\"test\"));\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#killNode(ErlangValue)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testKillNode1() throws ConnectionException, InterruptedException {\n        testDeleteNode(DeleteAction.KILL);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownNode(ErlangValue)} and\n     * {@link ScalarisVM#killNode(ErlangValue)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    private final void testDeleteNode(final DeleteAction action) throws ConnectionException, InterruptedException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        try {\n            final int size = conn.getNumberOfNodes();\n            final ErlangValue name = conn.addNodes(1).successful.get(0);\n            assertEquals(size + 1, conn.getNumberOfNodes());\n            boolean result = false;\n            switch (action) {\n                case SHUTDOWN:\n                    result = conn.shutdownNode(name);\n                    break;\n                case KILL:\n                    result = conn.killNode(name);\n                    break;\n            }\n            assertTrue(result);\n            assertEquals(size, conn.getNumberOfNodes());\n            final List<ErlangValue> nodes = conn.getNodes();\n            assertTrue(nodes.toString() + \" should not contain \" + name, !nodes.contains(name));\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownNodes(int)} with a closed\n     * connection.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testShutdownNodes_NotConnected() throws ConnectionException, InterruptedException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.shutdownNodes(1);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testShutdownNodes0() throws ConnectionException, InterruptedException {\n        testDeleteNodesX(0, DeleteAction.SHUTDOWN);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testShutdownNodes1() throws ConnectionException, InterruptedException {\n        testDeleteNodesX(1, DeleteAction.SHUTDOWN);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testShutdownNodes3() throws ConnectionException, InterruptedException {\n        testDeleteNodesX(3, DeleteAction.SHUTDOWN);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#killNodes(int)} with a closed\n     * connection.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testKillNodes_NotConnected() throws ConnectionException, InterruptedException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.killNodes(1);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#killNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testKillNodes0() throws ConnectionException, InterruptedException {\n        testDeleteNodesX(0, DeleteAction.KILL);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#killNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testKillNodes1() throws ConnectionException, InterruptedException {\n        testDeleteNodesX(1, DeleteAction.KILL);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#killNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testKillNodes3() throws ConnectionException, InterruptedException {\n        testDeleteNodesX(3, DeleteAction.KILL);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownNodes(int)} and\n     * {@link ScalarisVM#killNodes(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    private final void testDeleteNodesX(final int nodesToRemove, final DeleteAction action) throws ConnectionException, InterruptedException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        try {\n            final int size = conn.getNumberOfNodes();\n            if (nodesToRemove >= 1) {\n                conn.addNodes(nodesToRemove);\n                assertEquals(size + nodesToRemove, conn.getNumberOfNodes());\n            }\n            List<ErlangValue> result = null;\n            switch (action) {\n                case SHUTDOWN:\n                    result = conn.shutdownNodes(nodesToRemove);\n                    break;\n                case KILL:\n                    result = conn.killNodes(nodesToRemove);\n                    break;\n                default:\n                    throw new RuntimeException();\n            }\n            assertEquals(nodesToRemove, result.size());\n            assertEquals(size, conn.getNumberOfNodes());\n            final List<ErlangValue> nodes = conn.getNodes();\n            for (final ErlangValue name : result) {\n                assertTrue(nodes.toString() + \" should not contain \" + name, !nodes.contains(name));\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownNodesByName(List)} with a\n     * closed connection.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testShutdownByName_NotConnected() throws ConnectionException, InterruptedException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.shutdownNodesByName(Arrays.asList(new ErlangValue(\"test\")));\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownNodesByName(List)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testShutdownByName0() throws ConnectionException, InterruptedException {\n        testDeleteNodesByNameX(0, DeleteAction.SHUTDOWN);\n    }\n\n    /**\n     * Test method for\n     * {@link ScalarisVM#shutdownNodesByName(List)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testShutdownByName1() throws ConnectionException, InterruptedException {\n        testDeleteNodesByNameX(1, DeleteAction.SHUTDOWN);\n    }\n\n    /**\n     * Test method for\n     * {@link ScalarisVM#shutdownNodesByName(List)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testShutdownByName3() throws ConnectionException, InterruptedException {\n        testDeleteNodesByNameX(3, DeleteAction.SHUTDOWN);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#killNodes(List)} with a closed\n     * connection.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testKillByName_NotConnected() throws ConnectionException, InterruptedException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.killNodes(Arrays.asList(new ErlangValue(\"test\")));\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#killNodes(List)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testKillByName0() throws ConnectionException, InterruptedException {\n        testDeleteNodesByNameX(0, DeleteAction.KILL);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#killNodes(List)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testKillByName1() throws ConnectionException, InterruptedException {\n        testDeleteNodesByNameX(1, DeleteAction.KILL);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#killNodes(List)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testKillByName3() throws ConnectionException, InterruptedException {\n        testDeleteNodesByNameX(3, DeleteAction.KILL);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownNodesByName(List)} and\n     * {@link ScalarisVM#killNodes(List)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    private final void testDeleteNodesByNameX(final int nodesToRemove, final DeleteAction action) throws ConnectionException, InterruptedException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        try {\n            final int size = conn.getNumberOfNodes();\n            if (nodesToRemove >= 1) {\n                conn.addNodes(nodesToRemove);\n                assertEquals(size + nodesToRemove, conn.getNumberOfNodes());\n            }\n            List<ErlangValue> nodes = conn.getNodes();\n            Collections.shuffle(nodes);\n            final List<ErlangValue> removedNodes = nodes.subList(nodes.size() - nodesToRemove, nodes.size());\n            DeleteNodesByNameResult result = null;\n            switch (action) {\n                case SHUTDOWN:\n                    result = conn.shutdownNodesByName(removedNodes);\n                    break;\n                case KILL:\n                    result = conn.killNodes(removedNodes);\n                    break;\n                default:\n                    throw new RuntimeException();\n            }\n            assertEquals(nodesToRemove, result.successful.size());\n            assertEquals(0, result.notFound.size());\n            Collections.sort(removedNodes);\n            Collections.sort(result.successful);\n            assertEquals(removedNodes, result.successful);\n            assertEquals(size, conn.getNumberOfNodes());\n            nodes = conn.getNodes();\n            for (final ErlangValue name : result.successful) {\n                assertTrue(nodes.toString() + \" should not contain \" + name, !nodes.contains(name));\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#getOtherVMs(int)} with a closed\n     * connection.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testGetOtherVMs_NotConnected() throws ConnectionException, InterruptedException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.getOtherVMs(1);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#getOtherVMs(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testGetOtherVMs1() throws ConnectionException, InterruptedException {\n        testGetOtherVMsX(1);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#getOtherVMs(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testGetOtherVMs2() throws ConnectionException, InterruptedException {\n        testGetOtherVMsX(2);\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#getOtherVMs(int)}.\n     *\n     * @throws ConnectionException\n     * @throws InterruptedException\n     */\n    @Test\n    public final void testGetOtherVMs3() throws ConnectionException, InterruptedException {\n        testGetOtherVMsX(3);\n    }\n\n    private final void testGetOtherVMsX(final int max) throws ConnectionException, InterruptedException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        try {\n            final List<String> result = conn.getOtherVMs(max);\n            assertTrue(\"list too long: \" + result.toString(), result.size() <= max);\n            for (final String node : result) {\n                final ScalarisVM conn2 = new ScalarisVM(node);\n                try {\n                    conn2.getInfo();\n                } finally {\n                    conn2.closeConnection();\n                }\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownVM()} with a closed connection.\n     *\n     * @throws ConnectionException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testShutdownVM_NotConnected() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.shutdownVM();\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#shutdownVM()}.\n     *\n     * @throws ConnectionException\n     */\n    @Ignore(\"we still need the Scalaris Erlang VM\")\n    @Test\n    public final void testShutdownVM1() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.shutdownVM();\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#killVM()} with a closed connection.\n     *\n     * @throws ConnectionException\n     */\n    @Test(expected=ConnectionException.class)\n    public final void testKillVM_NotConnected() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.closeConnection();\n        conn.killVM();\n    }\n\n    /**\n     * Test method for {@link ScalarisVM#killVM()}.\n     *\n     * @throws ConnectionException\n     */\n    @Ignore(\"we still need the Scalaris Erlang VM\")\n    @Test\n    public final void testKillVM1() throws ConnectionException {\n        final ScalarisVM conn = new ScalarisVM(scalarisNode);\n        conn.killVM();\n    }\n}\n"
  },
  {
    "path": "java-api/test/de/zib/scalaris/TransactionSingleOpTest.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport static org.junit.Assert.assertEquals;\nimport static org.junit.Assert.assertTrue;\n\nimport java.io.ByteArrayInputStream;\nimport java.io.ByteArrayOutputStream;\nimport java.io.IOException;\nimport java.util.ArrayList;\nimport java.util.Arrays;\nimport java.util.List;\nimport java.util.zip.GZIPInputStream;\nimport java.util.zip.GZIPOutputStream;\n\nimport org.apache.commons.codec.binary.Base64;\nimport org.junit.Test;\n\nimport com.ericsson.otp.erlang.OtpErlangObject;\n\nimport de.zib.scalaris.TransactionSingleOp.RequestList;\nimport de.zib.scalaris.TransactionSingleOp.ResultList;\nimport de.zib.scalaris.operations.ReadOp;\nimport de.zib.scalaris.operations.ReadRandomFromListOp;\nimport de.zib.scalaris.operations.WriteOp;\n\n/**\n * Unit test for the {@link TransactionSingleOp} class.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.2\n * @since 2.0\n */\npublic class TransactionSingleOpTest {\n    private final static long testTime = System.currentTimeMillis();\n\n    private final static String[] testData = {\n        \"ahz2ieSh\", \"wooPhu8u\", \"quai9ooK\", \"Oquae4ee\", \"Airier1a\", \"Boh3ohv5\", \"ahD3Saog\", \"EM5ooc4i\",\n        \"Epahrai8\", \"laVahta7\", \"phoo6Ahj\", \"Igh9eepa\", \"aCh4Lah6\", \"ooT0ath5\", \"uuzau4Ie\", \"Iup6mae6\",\n//        \"xie7iSie\", \"ail8yeeP\", \"ooZ4eesi\", \"Ahn7ohph\", \"Ohy5moo6\", \"xooSh9Oo\", \"ieb6eeS7\", \"Thooqu9h\",\n//        \"eideeC9u\", \"phois3Ie\", \"EimaiJ2p\", \"sha6ahR1\", \"Pheih3za\", \"bai4eeXe\", \"rai0aB7j\", \"xahXoox6\",\n//        \"Xah4Okeg\", \"cieG8Yae\", \"Pe9Ohwoo\", \"Eehig6ph\", \"Xe7rooy6\", \"waY2iifu\", \"kemi8AhY\", \"Che7ain8\",\n//        \"ohw6seiY\", \"aegh1oBa\", \"thoh9IeG\", \"Kee0xuwu\", \"Gohng8ee\", \"thoh9Chi\", \"aa4ahQuu\", \"Iesh5uge\",\n//        \"Ahzeil8n\", \"ieyep5Oh\", \"xah3IXee\", \"Eefa5qui\", \"kai8Muuf\", \"seeCe0mu\", \"cooqua5Y\", \"Ci3ahF6z\",\n//        \"ot0xaiNu\", \"aewael8K\", \"aev3feeM\", \"Fei7ua5t\", \"aeCa6oph\", \"ag2Aelei\", \"Shah1Pho\", \"ePhieb0N\",\n//        \"Uqu7Phup\", \"ahBi8voh\", \"oon3aeQu\", \"Koopa0nu\", \"xi0quohT\", \"Oog4aiph\", \"Aip2ag5D\", \"tirai7Ae\",\n//        \"gi0yoePh\", \"uay7yeeX\", \"aeb6ahC1\", \"OoJeic2a\", \"ieViom1y\", \"di0eeLai\", \"Taec2phe\", \"ID2cheiD\",\n//        \"oi6ahR5M\", \"quaiGi8W\", \"ne1ohLuJ\", \"DeD0eeng\", \"yah8Ahng\", \"ohCee2ie\", \"ecu1aDai\", \"oJeijah4\",\n//        \"Goo9Una1\", \"Aiph3Phi\", \"Ieph0ce5\", \"ooL6cae7\", \"nai0io1H\", \"Oop2ahn8\", \"ifaxae7O\", \"NeHai1ae\",\n//        \"Ao8ooj6a\", \"hi9EiPhi\", \"aeTh9eiP\", \"ao8cheiH\", \"Yieg3sha\", \"mah7cu2D\", \"Uo5wiegi\", \"Oowei0ya\",\n//        \"efeiDee7\", \"Oliese6y\", \"eiSh1hoh\", \"Joh6hoh9\", \"zib6Ooqu\", \"eejiJie4\", \"lahZ3aeg\", \"keiRai1d\",\n//        \"Fei0aewe\", \"aeS8aboh\", \"hae3ohKe\", \"Een9ohQu\", \"AiYeeh7o\", \"Yaihah4s\", \"ood4Giez\", \"Oumai7te\",\n//        \"hae2kahY\", \"afieGh4v\", \"Ush0boo0\", \"Ekootee5\", \"Ya8iz6Ie\", \"Poh6dich\", \"Eirae4Ah\", \"pai8Eeme\",\n//        \"uNah7dae\", \"yo3hahCh\", \"teiTh7yo\", \"zoMa5Cuv\", \"ThiQu5ax\", \"eChi5caa\", \"ii9ujoiV\", \"ge7Iekui\",\n        \"sai2aiTa\", \"ohKi9rie\", \"ei2ioChu\", \"aaNgah9y\", \"ooJai1Ie\", \"shoh0oH9\", \"Ool4Ahya\", \"poh0IeYa\",\n        \"Uquoo0Il\", \"eiGh4Oop\", \"ooMa0ufe\", \"zee6Zooc\", \"ohhao4Ah\", \"Uweekek5\", \"aePoos9I\", \"eiJ9noor\",\n        \"phoong1E\", \"ianieL2h\", \"An7ohs4T\", \"Eiwoeku3\", \"sheiS3ao\", \"nei5Thiw\", \"uL5iewai\", \"ohFoh9Ae\"};\n\n    static {\n        // determine good/bad nodes:\n        final ConnectionFactory cf = ConnectionFactory.getInstance();\n        cf.testAllNodes();\n        // set not to automatically try reconnects (auto-retries prevent ConnectionException tests from working):\n        ((DefaultConnectionPolicy) cf.getConnectionPolicy()).setMaxRetries(0);\n    }\n\n    /**\n     * Test method for\n     * {@link TransactionSingleOp#TransactionSingleOp()}.\n     * @throws ConnectionException\n     */\n    @Test\n    public void testTransactionSingleOp1() throws ConnectionException {\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        conn.closeConnection();\n    }\n\n    /**\n     * Test method for\n     * {@link TransactionSingleOp#TransactionSingleOp(Connection)}.\n     * @throws ConnectionException\n     */\n    @Test\n    public void testTransactionSingleOp2() throws ConnectionException {\n        final TransactionSingleOp conn = new TransactionSingleOp(ConnectionFactory.getInstance().createConnection(\"test\"));\n        conn.closeConnection();\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#closeConnection()} trying to\n     * close the connection twice.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     */\n    @Test\n    public void testDoubleClose() throws ConnectionException {\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        conn.closeConnection();\n        conn.closeConnection();\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#read(String)}.\n     *\n     * @throws NotFoundException\n     * @throws UnknownException\n     * @throws ConnectionException\n     */\n    @Test(expected=NotFoundException.class)\n    public void testRead_NotFound() throws ConnectionException,\n            UnknownException, NotFoundException {\n        final String key = \"_Read_NotFound\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        try {\n            conn.read(testTime + key);\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#read(String)} with a closed connection.\n     *\n     * @throws NotFoundException\n     * @throws UnknownException\n     * @throws ConnectionException\n     */\n    @Test(expected=ConnectionException.class)\n    public void testRead_NotConnected() throws ConnectionException,\n            UnknownException, NotFoundException {\n        final String key = \"_Read_NotConnected\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        conn.closeConnection();\n        conn.read(testTime + key);\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#write(String, String)} with a\n     * closed connection.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     */\n    @Test(expected=ConnectionException.class)\n    public void testWriteString_NotConnected() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final String key = \"_WriteString_NotConnected\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        conn.closeConnection();\n        conn.write(testTime + key, testData[0]);\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#write(String, String)} and\n     * {@link TransactionSingleOp#read(String)}. Writes strings and uses a\n     * distinct key for each value. Tries to read the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws EmptyListException\n     */\n    @Test\n    public void testWriteString1() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException, EmptyListException {\n        final String key = \"_WriteString1_\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            for (int i = 0; i < testData.length; ++i) {\n                conn.write(testTime + key + i, testData[i]);\n            }\n\n            // now try to read the data:\n            for (int i = 0; i < testData.length; ++i) {\n                final String actual = conn.read(testTime + key + i).stringValue();\n                assertEquals(testData[i], actual);\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#write(String, String)} and\n     * {@link TransactionSingleOp#read(String)}. Writes strings and uses a\n     * single key for all the values. Tries to read the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     */\n    @Test\n    public void testWriteString2() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final String key = \"_WriteString2\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            for (final String element : testData) {\n                conn.write(testTime + key, element);\n            }\n\n            // now try to read the data:\n            final String actual = conn.read(testTime + key).stringValue();\n            assertEquals(testData[testData.length - 1], actual);\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#write(String, Integer)} with a\n     * closed connection.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     */\n    @Test(expected=ConnectionException.class)\n    public void testWriteNumber_NotConnected() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final String key = \"_WriteString_NotConnected\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        conn.closeConnection();\n        conn.write(testTime + key, 0);\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#write(String, Integer)},\n     * {@link TransactionSingleOp#read(String)} and {@link ReadRandomFromListOp}\n     * . Writes integers and uses a distinct key for each value. Tries to read\n     * the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws EmptyListException\n     */\n    @Test\n    public void testWriteNumber1() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException, EmptyListException {\n        final String key = \"_WriteString1_\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            for (int i = 0; i < testData.length; ++i) {\n                conn.write(testTime + key + i, i);\n            }\n\n            // now try to read the data:\n            for (int i = 0; i < testData.length; ++i) {\n                final int actual = conn.read(testTime + key + i).intValue();\n                assertEquals(i, actual);\n\n                final ReadRandomFromListOp op = new ReadRandomFromListOp(testTime + key + i);\n                final ResultList resultList = conn.req_list(op);\n                assertEquals(op, resultList.get(0));\n                try {\n                    final ReadRandomFromListOp.Result randResult = op.processResultSingle();\n                    assertTrue(\"Trying to read random list entry from non-list should fail: \" + randResult, false);\n                } catch (final NotAListException e) {\n                    // ok\n                }\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#write(String, Integer)},\n     * {@link TransactionSingleOp#read(String)} and {@link ReadRandomFromListOp}\n     * . Writes strings and uses a single key for all the values. Tries to read\n     * the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws EmptyListException\n     */\n    @Test\n    public void testWriteNumber2() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException, EmptyListException {\n        final String key = \"_WriteString2\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            for (int i = 0; i < testData.length; ++i) {\n                conn.write(testTime + key, i);\n            }\n\n            // now try to read the data:\n            final int actual = conn.read(testTime + key).intValue();\n            assertEquals(testData.length - 1, actual);\n\n            final ReadRandomFromListOp op = new ReadRandomFromListOp(testTime + key);\n            final ResultList resultList = conn.req_list(op);\n            assertEquals(op, resultList.get(0));\n            try {\n                final ReadRandomFromListOp.Result randResult = op.processResultSingle();\n                assertTrue(\"Trying to read random list entry from non-list should fail: \" + randResult, false);\n            } catch (final NotAListException e) {\n                // ok\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#write(String, List)} with a\n     * closed connection.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     *\n     * @since 3.2\n     */\n    @Test(expected=ConnectionException.class)\n    public void testWriteList_NotConnected() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final String key = \"_WriteList_NotConnected\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        conn.closeConnection();\n        final ArrayList<String> list = new ArrayList<String>();\n        list.add(testData[0]);\n        list.add(testData[1]);\n        conn.write(testTime + key, list);\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#write(String, List)},\n     * {@link TransactionSingleOp#read(String)} and {@link ReadRandomFromListOp}\n     * . Writes lists and uses a distinct key for each value. Tries to read the\n     * data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws NotAListException\n     * @throws EmptyListException\n     *\n     * @since 3.2\n     */\n    @Test\n    public void testWriteList1() throws ConnectionException, UnknownException,\n            NotFoundException, AbortException, EmptyListException, NotAListException {\n        final String key = \"_WriteList1_\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            for (int i = 0; i < (testData.length - 1); ++i) {\n                final ArrayList<String> list = new ArrayList<String>();\n                if ((i % 2) == 0) {\n                    list.add(testData[i]);\n                    list.add(testData[i + 1]);\n                }\n                conn.write(testTime + key + i, list);\n            }\n\n            // now try to read the data:\n\n            for (int i = 0; i < (testData.length - 1); ++i) {\n                final ArrayList<String> expected = new ArrayList<String>();\n                if ((i % 2) == 0) {\n                    expected.add(testData[i]);\n                    expected.add(testData[i + 1]);\n                }\n                final List<String> actual = conn.read(testTime + key + i).stringListValue();\n                assertEquals(expected, actual);\n\n                final ReadRandomFromListOp op = new ReadRandomFromListOp(testTime + key + i);\n                final ResultList resultList = conn.req_list(op);\n                assertEquals(op, resultList.get(0));\n                try {\n                    final ReadRandomFromListOp.Result randResult = op.processResultSingle();\n                    if ((i % 2) == 0) {\n                        assertEquals(expected.size(), randResult.listLength);\n                        assertTrue(expected.toString() + \".contains(\" + randResult.randomElement + \")\",\n                                expected.contains(randResult.randomElement.stringValue()));\n                    } else {\n                        assertTrue(\"Trying to read random list entry from an empty list should fail: \" + randResult, false);\n                    }\n                } catch (final EmptyListException e) {\n                    if ((i % 2) == 0) {\n                        assertTrue(\"Got EmptyListException on non-empty list: \" + e, false);\n                    }\n                }\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#write(String, List)},\n     * {@link TransactionSingleOp#read(String)} and {@link ReadRandomFromListOp}\n     * . Writes lists and uses a single key for all the values. Tries to read\n     * the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws NotAListException\n     * @throws EmptyListException\n     *\n     * @since 3.2\n     */\n    @Test\n    public void testWriteList2() throws ConnectionException, UnknownException,\n            NotFoundException, AbortException, EmptyListException, NotAListException {\n        final String key = \"_WriteList2\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            final List<String> list = new ArrayList<String>();\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                list.clear();\n                list.add(testData[i]);\n                list.add(testData[i + 1]);\n                conn.write(testTime + key, list);\n            }\n\n            // now try to read the data:\n\n            final List<String> actual = conn.read(testTime + key).stringListValue();\n            final List<String> expected = list;\n            assertEquals(expected, actual);\n\n            final ReadRandomFromListOp op = new ReadRandomFromListOp(testTime + key);\n            final ResultList resultList = conn.req_list(op);\n            assertEquals(op, resultList.get(0));\n            final ReadRandomFromListOp.Result randResult = op.processResultSingle();\n            assertEquals(expected.size(), randResult.listLength);\n            assertTrue(expected.toString() + \".contains(\" + randResult.randomElement + \")\",\n                    expected.contains(randResult.randomElement.stringValue()));\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link TransactionSingleOp#testAndSet(String, String, List)}\n     * with a closed connection.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 3.2\n     */\n    @Test(expected=ConnectionException.class)\n    public void testTestAndSetList_NotConnected() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException,\n            KeyChangedException {\n        final String key = \"_TestAndSetList_NotConnected\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        conn.closeConnection();\n        final ArrayList<String> list = new ArrayList<String>();\n        list.add(testData[0]);\n        list.add(testData[1]);\n        conn.testAndSet(testTime + key, \"fail\", list);\n    }\n\n    /**\n     * Test method for\n     * {@link TransactionSingleOp#testAndSet(String, String, List)}.\n     * Tries test_and_set with a non-existing key.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 3.2\n     */\n    @Test(expected=NotFoundException.class)\n    public void testTestAndSetList_NotFound() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException,\n            KeyChangedException {\n        final String key = \"_TestAndSetList_NotFound\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            final ArrayList<String> list = new ArrayList<String>();\n            list.add(testData[0]);\n            list.add(testData[1]);\n            conn.testAndSet(testTime + key, \"fail\", list);\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link TransactionSingleOp#testAndSet(String, List, List)},\n     * {@link TransactionSingleOp#read(String)}\n     * and {@link TransactionSingleOp#write(String, List)}.\n     * Writes a list and tries to overwrite it using test_and_set\n     * knowing the correct old value. Tries to read the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testTestAndSetList1a() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException,\n            KeyChangedException {\n        final String key = \"_TestAndSetList1a\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            // first write all values:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final ArrayList<String> list = new ArrayList<String>();\n                list.add(testData[i]);\n                list.add(testData[i + 1]);\n                conn.write(testTime + key + i, list);\n            }\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final ArrayList<String> old_list = new ArrayList<String>();\n                old_list.add(testData[i]);\n                old_list.add(testData[i + 1]);\n                final ArrayList<String> new_list = new ArrayList<String>();\n                new_list.add(testData[i + 1]);\n                new_list.add(testData[i]);\n                conn.testAndSet(testTime + key + i, old_list, new_list);\n            }\n\n            // now try to read the data:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final List<String> actual = conn.read(testTime + key + i).stringListValue();\n                final ArrayList<String> expected = new ArrayList<String>();\n                expected.add(testData[i + 1]);\n                expected.add(testData[i]);\n                assertEquals(expected, actual);\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link TransactionSingleOp#testAndSet(String, Object, List)},\n     * {@link TransactionSingleOp#read(String)}\n     * and {@link TransactionSingleOp#write(String, List)}.\n     * Writes a list and tries to overwrite it using test_and_set\n     * knowing the wrong old value. Tries to read the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testTestAndSetList1b() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final String key = \"_TestAndSetList1b\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            // first write all values:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final ArrayList<String> list = new ArrayList<String>();\n                list.add(testData[i]);\n                list.add(testData[i + 1]);\n                conn.write(testTime + key + i, list);\n            }\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final int new_value = 1;\n                try {\n                    conn.testAndSet(testTime + key + i, \"fail\", new_value);\n                    // a key changed exception must be thrown\n                    assertTrue(false);\n                } catch (final KeyChangedException e) {\n                    final ArrayList<String> expected = new ArrayList<String>();\n                    expected.add(testData[i]);\n                    expected.add(testData[i + 1]);\n                    assertEquals(expected, e.getOldValue().stringListValue());\n                }\n            }\n\n            // now try to read the data:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final List<String> actual = conn.read(testTime + key + i).stringListValue();\n                final ArrayList<String> expected = new ArrayList<String>();\n                expected.add(testData[i]);\n                expected.add(testData[i + 1]);\n                assertEquals(expected, actual);\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link TransactionSingleOp#testAndSet(String, List, List)},\n     * {@link TransactionSingleOp#read(String)} and\n     * {@link TransactionSingleOp#write(String, List)}. Writes a list and tries\n     * to overwrite it using test_and_set knowing the correct old value and\n     * using a single key for all the values. Tries to read the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testTestAndSetList2a() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException,\n            KeyChangedException {\n        final String key = \"_TestAndSetList2a\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            // first write all values:\n            final ArrayList<String> list = new ArrayList<String>();\n            list.add(testData[0]);\n            list.add(testData[1]);\n            conn.write(testTime + key, list);\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 1; i < (testData.length - 1); ++i) {\n                final ArrayList<String> old_list = new ArrayList<String>();\n                old_list.add(testData[i - 1]);\n                old_list.add(testData[i]);\n                final ArrayList<String> new_list = new ArrayList<String>();\n                new_list.add(testData[i]);\n                new_list.add(testData[i + 1]);\n                conn.testAndSet(testTime + key, old_list, new_list);\n            }\n\n            // now try to read the data:\n            final List<String> actual = conn.read(testTime + key).stringListValue();\n            final ArrayList<String> expected = new ArrayList<String>();\n            expected.add(testData[testData.length - 2]);\n            expected.add(testData[testData.length - 1]);\n            assertEquals(expected, actual);\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link TransactionSingleOp#testAndSet(String, Object, List)},\n     * {@link TransactionSingleOp#read(String)} and\n     * {@link TransactionSingleOp#write(String, List)}.\n     * Writes a list and tries to overwrite it using test_and_set knowing the\n     * wrong old value and using a single key for all the values. Tries to read\n     * the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testTestAndSetList2b() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final String key = \"_TestAndSetList2b\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            // first write all values:\n            final ArrayList<String> list = new ArrayList<String>();\n            list.add(testData[0]);\n            list.add(testData[1]);\n            conn.write(testTime + key, list);\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 0; i < (testData.length - 1); ++i) {\n                final int new_value = 1;\n                try {\n                    conn.testAndSet(testTime + key, \"fail\", new_value);\n                    // a key changed exception must be thrown\n                    assertTrue(false);\n                } catch (final KeyChangedException e) {\n                    final ArrayList<String> expected = new ArrayList<String>();\n                    expected.add(testData[0]);\n                    expected.add(testData[1]);\n                    assertEquals(expected, e.getOldValue().stringListValue());\n                }\n            }\n\n            // now try to read the data:\n            final List<String> actual = conn.read(testTime + key).stringListValue();\n            final ArrayList<String> expected = list;\n            assertEquals(expected, actual);\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link TransactionSingleOp#testAndSet(String, String, String)}\n     * with a closed connection.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 2.7\n     */\n    @Test(expected=ConnectionException.class)\n    public void testTestAndSetString_NotConnected() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException,\n            KeyChangedException {\n        final String key = \"_TestAndSetString_NotConnected\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        conn.closeConnection();\n        conn.testAndSet(testTime + key, testData[0], testData[1]);\n    }\n\n    /**\n     * Test method for\n     * {@link TransactionSingleOp#testAndSet(String, String, String)}.\n     * Tries test_and_set with a non-existing key.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 2.7\n     */\n    @Test(expected=NotFoundException.class)\n    public void testTestAndSetString_NotFound() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException, KeyChangedException {\n        final String key = \"_TestAndSetString_NotFound\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            conn.testAndSet(testTime + key, testData[0], testData[1]);\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link TransactionSingleOp#testAndSet(String, String, String)},\n     * {@link TransactionSingleOp#read(String)}\n     * and {@link TransactionSingleOp#write(String, Object)}.\n     * Writes a string and tries to overwrite it using test_and_set\n     * knowing the correct old value. Tries to read the string afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 2.7\n     */\n    @Test\n    public void testTestAndSetString1() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException, KeyChangedException {\n        final String key = \"_TestAndSetString1\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            // first write all values:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                conn.write(testTime + key + i, testData[i]);\n            }\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                conn.testAndSet(testTime + key + i, testData[i], testData[i + 1]);\n            }\n\n            // now try to read the data:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                assertEquals(testData[i + 1], conn.read(testTime + key + i).stringValue());\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link TransactionSingleOp#testAndSet(String, String, String)},\n     * {@link TransactionSingleOp#read(String)}\n     * and {@link TransactionSingleOp#write(String, Object)}.\n     * Writes a string and tries to overwrite it using test_and_set\n     * knowing the wrong old value. Tries to read the string afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     *\n     * @since 2.7\n     */\n    @Test\n    public void testTestAndSetString2() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final String key = \"_TestAndSetString2\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            // first write all values:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                conn.write(testTime + key + i, testData[i]);\n            }\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                try {\n                    conn.testAndSet(testTime + key + i, testData[i + 1], \"fail\");\n                    // a key changed exception must be thrown\n                    assertTrue(false);\n                } catch (final KeyChangedException e) {\n                    assertEquals(testData[i], e.getOldValue().stringValue());\n                }\n            }\n\n            // now try to read the data:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                assertEquals(testData[i], conn.read(testTime + key + i).stringValue());\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#req_list(RequestList)} with an\n     * empty request list.\n     *\n     * @throws ConnectionException\n     * @throws UnknownException\n     * @throws AbortException\n     */\n    @Test\n    public void testReqList_Empty() throws ConnectionException,\n            UnknownException, AbortException {\n        final TransactionSingleOp t = new TransactionSingleOp();\n        try {\n            t.req_list(new RequestList());\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#req_list(RequestList)} with a\n     * mixed request list.\n     *\n     * @throws ConnectionException\n     * @throws UnknownException\n     * @throws AbortException\n     * @throws NotFoundException\n     */\n    @Test\n    public void testReqList1() throws ConnectionException, UnknownException,\n            AbortException, NotFoundException {\n        final String key = \"_ReqList1_\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n\n        try {\n            final RequestList readRequests = new RequestList();\n            final RequestList firstWriteRequests = new RequestList();\n            final RequestList writeRequests = new RequestList();\n            for (int i = 0; i < testData.length; ++i) {\n                if ((i % 2) == 0) {\n                    firstWriteRequests.addOp(new WriteOp(testTime + key + i, \"first_\" + testData[i]));\n                }\n                writeRequests.addOp(new WriteOp(testTime + key + i, \"second_\" + testData[i]));\n                readRequests.addOp(new ReadOp(testTime + key + i));\n            }\n\n            ResultList results = conn.req_list(firstWriteRequests);\n            // evaluate the first write results:\n            for (int i = 0; i < firstWriteRequests.size(); ++i) {\n                results.processWriteAt(i);\n            }\n\n            results = conn.req_list(readRequests);\n            assertEquals(readRequests.size(), results.size());\n            // now evaluate the read results:\n            for (int i = 0; i < readRequests.size(); ++i) {\n                if ((i % 2) == 0) {\n                    final String actual = results.processReadAt(i).stringValue();\n                    assertEquals(\"first_\" + testData[i], actual);\n                } else {\n                    try {\n                        final OtpErlangObject result = results.processReadAt(i).value();\n                        // a not found exception must be thrown\n                        assertTrue(\"Expected a NotFoundException, got: \" + result.toString(), false);\n                    } catch (final NotFoundException e) {\n                    }\n                }\n            }\n\n            results = conn.req_list(writeRequests);\n            assertEquals(writeRequests.size(), results.size());\n\n            // now evaluate the write results:\n            for (int i = 0; i < writeRequests.size(); ++i) {\n                results.processWriteAt(i);\n            }\n\n            // once again test reads - now all reads should be successful\n            results = conn.req_list(readRequests);\n            assertEquals(readRequests.size(), results.size());\n\n            // now evaluate the read results:\n            for (int i = 0; i < readRequests.size(); ++i) {\n                final String actual = results.processReadAt(i).stringValue();\n                assertEquals(\"second_\" + testData[i], actual);\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Tests how long it takes to read a large list without compression.\n     *\n     * @throws ConnectionException\n     * @throws UnknownException\n     * @throws AbortException\n     * @throws NotFoundException\n     */\n    @Test\n    public void testReadLargeList1() throws ConnectionException,\n            UnknownException, AbortException, NotFoundException {\n        testReadLargeList(false, \"_ReadLargeList1\");\n    }\n\n    /**\n     * Tests how long it takes to read a large list with compression.\n     *\n     * @throws ConnectionException\n     * @throws UnknownException\n     * @throws AbortException\n     * @throws NotFoundException\n     */\n    @Test\n    public void testReadLargeList2() throws ConnectionException,\n            UnknownException, AbortException, NotFoundException {\n        testReadLargeList(true, \"_ReadLargeList2\");\n    }\n\n    /**\n     * Tests how long it takes to read a large list with or without compression.\n     *\n     * @param compressed\n     *            whether to compress or not\n     * @param key\n     *            the key to append to the {@link #testTime}\n     *\n     * @throws ConnectionException\n     * @throws AbortException\n     * @throws UnknownException\n     * @throws ClassCastException\n     * @throws NotFoundException\n     */\n    protected void testReadLargeList(final boolean compressed, final String key)\n            throws ConnectionException, AbortException, UnknownException,\n            ClassCastException, NotFoundException {\n        final ArrayList<String> expected = new ArrayList<String>(testData.length * 100);\n        for (int i = 0; i < 100; ++i) {\n            expected.addAll(Arrays.asList(testData));\n        }\n\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        conn.setCompressed(compressed);\n        conn.write(testTime + key, expected);\n\n        try {\n            for (int i = 0; i < 500; ++i) {\n                final List<String> actual = conn.read(testTime + key).stringListValue();\n                assertEquals(expected, actual);\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Tests how long it takes to read a large string without compression.\n     *\n     * @throws ConnectionException\n     * @throws UnknownException\n     * @throws AbortException\n     * @throws NotFoundException\n     * @throws IOException\n     */\n    @Test\n    public void testReadLargeString1() throws ConnectionException,\n            UnknownException, AbortException, NotFoundException, IOException {\n        testReadLargeString(1, \"_testReadLargeString1\");\n    }\n\n    /**\n     * Tests how long it takes to read a large string with compression.\n     *\n     * @throws ConnectionException\n     * @throws UnknownException\n     * @throws AbortException\n     * @throws NotFoundException\n     * @throws IOException\n     */\n    @Test\n    public void testReadLargeString2() throws ConnectionException,\n            UnknownException, AbortException, NotFoundException, IOException {\n        testReadLargeString(2, \"_testReadLargeString2\");\n    }\n\n    /**\n     * Tests how long it takes to read a large string with manual compression,\n     * then encoding the string in base64.\n     *\n     * @throws ConnectionException\n     * @throws UnknownException\n     * @throws AbortException\n     * @throws NotFoundException\n     * @throws IOException\n     */\n    @Test\n    public void testReadLargeString3() throws ConnectionException,\n            UnknownException, AbortException, NotFoundException, IOException {\n        testReadLargeString(3, \"_testReadLargeString3\");\n    }\n\n    /**\n     * Tests how long it takes to read a large string with manual compression,\n     * then encoding the string in base64 and then using automatic compression.\n     *\n     * @throws ConnectionException\n     * @throws UnknownException\n     * @throws AbortException\n     * @throws NotFoundException\n     * @throws IOException\n     */\n    @Test\n    public void testReadLargeString4() throws ConnectionException,\n            UnknownException, AbortException, NotFoundException, IOException {\n        testReadLargeString(4, \"_testReadLargeString4\");\n    }\n\n    /**\n     * Tests how long it takes to read a large string with different compression\n     * schemes.\n     *\n     * @param compressed\n     *            how to compress\n     * @param key\n     *            the key to append to the {@link #testTime}\n     *\n     * @throws ConnectionException\n     * @throws UnknownException\n     * @throws AbortException\n     * @throws NotFoundException\n     * @throws IOException\n     */\n    protected void testReadLargeString(final int compression, final String key)\n            throws ConnectionException, UnknownException, AbortException,\n            NotFoundException, IOException {\n        final StringBuilder sb = new StringBuilder(testData.length * 8 * 100);\n        for (int i = 0; i < 100; ++i) {\n            for (final String data : testData) {\n                sb.append(data);\n            }\n        }\n        final String expected = sb.toString();\n\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        conn.setCompressed(true);\n        switch (compression) {\n            case 1:\n                conn.setCompressed(false);\n            case 2:\n                conn.write(testTime + key, expected);\n                break;\n            case 3:\n                conn.setCompressed(false);\n            case 4:\n                final ByteArrayOutputStream bos = new ByteArrayOutputStream();\n                final GZIPOutputStream gos = new GZIPOutputStream(bos);\n                gos.write(expected.getBytes(\"UTF-8\"));\n                gos.flush();\n                gos.close();\n                conn.write(testTime + key, new Base64(0).encodeToString(bos.toByteArray()));\n                break;\n            default:\n                return;\n        }\n\n        try {\n            for (int i = 0; i < 500; ++i) {\n                String actual = conn.read(testTime + key).stringValue();\n                if (compression >= 3) {\n                    final byte[] packed = new Base64(0).decode(actual);\n                    final ByteArrayOutputStream unpacked = new ByteArrayOutputStream();\n                    final ByteArrayInputStream bis = new ByteArrayInputStream(packed);\n                    final GZIPInputStream gis = new GZIPInputStream(bis);\n                    final byte[] bbuf = new byte[256];\n                    int read = 0;\n                    while ((read = gis.read(bbuf)) >= 0) {\n                        unpacked.write(bbuf, 0, read);\n                    }\n                    gis.close();\n                    actual = new String(unpacked.toString(\"UTF-8\"));\n                }\n                assertEquals(expected, actual);\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link ReadRandomFromListOp}.\n     *\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws EmptyListException\n     * @throws NotAListException\n     * @throws UnknownException\n     */\n    @Test(expected=NotFoundException.class)\n    public void testReadRandomFromList_NotFound() throws ConnectionException,\n            NotFoundException, EmptyListException, NotAListException,\n            UnknownException {\n        final String key = \"_ReadRandomFromList_NotFound\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        try {\n            final ReadRandomFromListOp op = new ReadRandomFromListOp(testTime + key);\n            final ResultList resultList = conn.req_list(op);\n            assertEquals(op, resultList.get(0));\n            op.processResultSingle();\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link TransactionSingleOp#read(String)} with a closed connection.\n     *\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws EmptyListException\n     * @throws NotAListException\n     * @throws UnknownException\n     */\n    @Test(expected=ConnectionException.class)\n    public void testReadRandomFromListOp_NotConnected()\n            throws ConnectionException, NotFoundException, EmptyListException,\n            NotAListException, UnknownException {\n        final String key = \"_ReadRandomFromListOp_NotConnected\";\n        final TransactionSingleOp conn = new TransactionSingleOp();\n        conn.closeConnection();\n        final ReadRandomFromListOp op = new ReadRandomFromListOp(testTime + key);\n        final ResultList resultList = conn.req_list(op);\n        assertEquals(op, resultList.get(0));\n        op.processResultSingle();\n    }\n}\n"
  },
  {
    "path": "java-api/test/de/zib/scalaris/TransactionTest.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.scalaris;\n\nimport static org.junit.Assert.*;\n\nimport java.util.ArrayList;\nimport java.util.Date;\nimport java.util.List;\nimport java.util.TreeSet;\n\nimport org.junit.Test;\n\nimport de.zib.scalaris.Transaction.RequestList;\nimport de.zib.scalaris.Transaction.ResultList;\nimport de.zib.scalaris.operations.ReadOp;\nimport de.zib.scalaris.operations.WriteOp;\n\n\n/**\n * Unit test for the {@link Transaction} class.\n *\n * @author Nico Kruber, kruber@zib.de\n * @version 3.17\n * @since 2.0\n */\npublic class TransactionTest {\n    private static long testTime = System.currentTimeMillis();\n\n    private final static String[] testData = {\n        \"ahz2ieSh\", \"wooPhu8u\", \"quai9ooK\", \"Oquae4ee\", \"Airier1a\", \"Boh3ohv5\", \"ahD3Saog\", \"EM5ooc4i\",\n        \"Epahrai8\", \"laVahta7\", \"phoo6Ahj\", \"Igh9eepa\", \"aCh4Lah6\", \"ooT0ath5\", \"uuzau4Ie\", \"Iup6mae6\",\n//        \"xie7iSie\", \"ail8yeeP\", \"ooZ4eesi\", \"Ahn7ohph\", \"Ohy5moo6\", \"xooSh9Oo\", \"ieb6eeS7\", \"Thooqu9h\",\n//        \"eideeC9u\", \"phois3Ie\", \"EimaiJ2p\", \"sha6ahR1\", \"Pheih3za\", \"bai4eeXe\", \"rai0aB7j\", \"xahXoox6\",\n//        \"Xah4Okeg\", \"cieG8Yae\", \"Pe9Ohwoo\", \"Eehig6ph\", \"Xe7rooy6\", \"waY2iifu\", \"kemi8AhY\", \"Che7ain8\",\n//        \"ohw6seiY\", \"aegh1oBa\", \"thoh9IeG\", \"Kee0xuwu\", \"Gohng8ee\", \"thoh9Chi\", \"aa4ahQuu\", \"Iesh5uge\",\n//        \"Ahzeil8n\", \"ieyep5Oh\", \"xah3IXee\", \"Eefa5qui\", \"kai8Muuf\", \"seeCe0mu\", \"cooqua5Y\", \"Ci3ahF6z\",\n//        \"ot0xaiNu\", \"aewael8K\", \"aev3feeM\", \"Fei7ua5t\", \"aeCa6oph\", \"ag2Aelei\", \"Shah1Pho\", \"ePhieb0N\",\n//        \"Uqu7Phup\", \"ahBi8voh\", \"oon3aeQu\", \"Koopa0nu\", \"xi0quohT\", \"Oog4aiph\", \"Aip2ag5D\", \"tirai7Ae\",\n//        \"gi0yoePh\", \"uay7yeeX\", \"aeb6ahC1\", \"OoJeic2a\", \"ieViom1y\", \"di0eeLai\", \"Taec2phe\", \"ID2cheiD\",\n//        \"oi6ahR5M\", \"quaiGi8W\", \"ne1ohLuJ\", \"DeD0eeng\", \"yah8Ahng\", \"ohCee2ie\", \"ecu1aDai\", \"oJeijah4\",\n//        \"Goo9Una1\", \"Aiph3Phi\", \"Ieph0ce5\", \"ooL6cae7\", \"nai0io1H\", \"Oop2ahn8\", \"ifaxae7O\", \"NeHai1ae\",\n//        \"Ao8ooj6a\", \"hi9EiPhi\", \"aeTh9eiP\", \"ao8cheiH\", \"Yieg3sha\", \"mah7cu2D\", \"Uo5wiegi\", \"Oowei0ya\",\n//        \"efeiDee7\", \"Oliese6y\", \"eiSh1hoh\", \"Joh6hoh9\", \"zib6Ooqu\", \"eejiJie4\", \"lahZ3aeg\", \"keiRai1d\",\n//        \"Fei0aewe\", \"aeS8aboh\", \"hae3ohKe\", \"Een9ohQu\", \"AiYeeh7o\", \"Yaihah4s\", \"ood4Giez\", \"Oumai7te\",\n//        \"hae2kahY\", \"afieGh4v\", \"Ush0boo0\", \"Ekootee5\", \"Ya8iz6Ie\", \"Poh6dich\", \"Eirae4Ah\", \"pai8Eeme\",\n//        \"uNah7dae\", \"yo3hahCh\", \"teiTh7yo\", \"zoMa5Cuv\", \"ThiQu5ax\", \"eChi5caa\", \"ii9ujoiV\", \"ge7Iekui\",\n        \"sai2aiTa\", \"ohKi9rie\", \"ei2ioChu\", \"aaNgah9y\", \"ooJai1Ie\", \"shoh0oH9\", \"Ool4Ahya\", \"poh0IeYa\",\n        \"Uquoo0Il\", \"eiGh4Oop\", \"ooMa0ufe\", \"zee6Zooc\", \"ohhao4Ah\", \"Uweekek5\", \"aePoos9I\", \"eiJ9noor\",\n        \"phoong1E\", \"ianieL2h\", \"An7ohs4T\", \"Eiwoeku3\", \"sheiS3ao\", \"nei5Thiw\", \"uL5iewai\", \"ohFoh9Ae\"};\n\n    static {\n        // determine good/bad nodes:\n        final ConnectionFactory cf = ConnectionFactory.getInstance();\n        cf.testAllNodes();\n        // set not to automatically try reconnects (auto-retries prevent ConnectionException tests from working):\n        ((DefaultConnectionPolicy) cf.getConnectionPolicy()).setMaxRetries(0);\n    }\n\n    /**\n     * Test method for {@link Transaction#Transaction()}.\n     * @throws ConnectionException\n     */\n    @Test\n    public void testTransaction1() throws ConnectionException {\n        final Transaction t = new Transaction();\n        t.closeConnection();\n    }\n\n    /**\n     * Test method for {@link Transaction#Transaction(Connection)}.\n     * @throws ConnectionException\n     */\n    @Test\n    public void testTransaction2() throws ConnectionException {\n        final Transaction t = new Transaction(ConnectionFactory.getInstance().createConnection(\"test\"));\n        t.closeConnection();\n    }\n\n    /**\n     * Test method for {@link Transaction#closeConnection()} trying to\n     * close the connection twice.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     */\n    @Test\n    public void testDoubleClose() throws ConnectionException {\n        final Transaction t = new Transaction(ConnectionFactory.getInstance().createConnection(\"test\"));\n        t.closeConnection();\n        t.closeConnection();\n    }\n\n    /**\n     * Test method for {@link Transaction#commit()} with a closed connection.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     */\n    @Test(expected=ConnectionException.class)\n    public void testCommit_NotConnected() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final Transaction t = new Transaction();\n        try {\n            t.closeConnection();\n            t.commit();\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Transaction#commit()} which commits an empty\n     * transaction.\n     *\n     * @throws ConnectionException\n     * @throws UnknownException\n     * @throws AbortException\n     */\n    @Test\n    public void testCommit_Empty() throws ConnectionException,\n            UnknownException, AbortException {\n        final Transaction t = new Transaction();\n        try {\n            t.commit();\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Transaction#abort()} with a closed connection.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     */\n    @Test\n    public void testAbort_NotConnected() throws ConnectionException,\n            UnknownException, NotFoundException {\n        final Transaction t = new Transaction();\n        t.closeConnection();\n        t.abort();\n    }\n\n    /**\n     * Test method for {@link Transaction#abort()} which aborts an empty\n     * transaction.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     */\n    @Test\n    public void testAbort_Empty() throws ConnectionException, UnknownException {\n        final Transaction t = new Transaction();\n        try {\n            t.abort();\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Transaction#read(String)}.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     */\n    @Test(expected = NotFoundException.class)\n    public void testRead_NotFound() throws ConnectionException,\n            UnknownException, NotFoundException {\n        final String key = \"_Read_NotFound\";\n        final Transaction t = new Transaction();\n        try {\n            t.read(testTime + key);\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Transaction#read(String)} with a closed\n     * connection.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     */\n    @Test(expected = ConnectionException.class)\n    public void testRead_NotConnected() throws ConnectionException,\n            UnknownException, NotFoundException {\n        final String key = \"_Read_NotConnected\";\n        final Transaction t = new Transaction();\n        try {\n            t.closeConnection();\n            t.read(testTime + key);\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Transaction#write(String, Object)} with a closed\n     * connection.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     */\n    @Test(expected = ConnectionException.class)\n    public void testWriteString_NotConnected() throws ConnectionException,\n            UnknownException, NotFoundException {\n        final String key = \"_WriteString_NotConnected\";\n        final Transaction t = new Transaction();\n        try {\n            t.closeConnection();\n            t.write(testTime + key, testData[0]);\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Transaction#read(String)} and\n     * {@link Transaction#write(String, Object)} which should show that\n     * writing a value for a key for which a previous read returned a\n     * NotFoundException is possible.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     */\n    @Test\n    public void testWriteString_NotFound() throws ConnectionException,\n            UnknownException, NotFoundException {\n        final String key = \"_WriteString_notFound\";\n        final Transaction t = new Transaction();\n        try {\n            try {\n                t.read(testTime + key);\n                assertTrue(false);\n            } catch (final NotFoundException e) {\n            }\n            t.write(testTime + key, testData[0]);\n\n            assertEquals(testData[0], t.read(testTime + key).stringValue());\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Transaction#write(String, Object)} and\n     * {@link Transaction#read(String)}. Writes strings and uses a distinct key\n     * for each value. Tries to read the data afterwards.\n     *\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws UnknownException\n     * @throws AbortException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testWriteString1() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final String key = \"_testWriteString1_\";\n        Transaction t = new Transaction();\n        try {\n            for (int i = 0; i < testData.length; ++i) {\n                t.write(testTime + key + i, testData[i]);\n            }\n\n            // now try to read the data:\n            for (int i = 0; i < testData.length; ++i) {\n                final String actual = t.read(testTime + key + i).stringValue();\n                assertEquals(testData[i], actual);\n            }\n\n            // try to read the data with another transaction (keys should not exist):\n            do {\n                final Transaction t2 = new Transaction();\n                for (int i = 0; i < testData.length; ++i) {\n                    try {\n                        t2.read(testTime + key + i).stringValue();\n                        // a key changed exception must be thrown\n                        assertTrue(false);\n                    } catch (final NotFoundException e) {\n                        t2.abort();\n                    }\n                }\n            } while (false);\n\n            // commit the transaction and try to read the data with a new one:\n            t.commit();\n            t = new Transaction();\n            for (int i = 0; i < testData.length; ++i) {\n                final String actual = t.read(testTime + key + i).stringValue();\n                assertEquals(testData[i], actual);\n            }\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Transaction#write(String, Object)} and\n     * {@link Transaction#read(String)}. Writes strings and uses a single key\n     * for all the values. Tries to read the data afterwards.\n     *\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws UnknownException\n     * @throws AbortException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testWriteString2() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final String key = \"_testWriteString2_\";\n        Transaction t = new Transaction();\n        try {\n            for (final String element : testData) {\n                t.write(testTime + key, element);\n            }\n\n            // now try to read the data:\n            final String actual1 = t.read(testTime + key).stringValue();\n            assertEquals(testData[testData.length - 1], actual1);\n\n            // try to read the data with another transaction (keys should not exist):\n            do {\n                final Transaction t2 = new Transaction();\n                try {\n                    t2.read(testTime + key).stringValue();\n                    // a key changed exception must be thrown\n                    assertTrue(false);\n                } catch (final NotFoundException e) {\n                    t2.abort();\n                }\n            } while (false);\n\n            // commit the transaction and try to read the data with a new one:\n            t.commit();\n            t = new Transaction();\n            final String actual2 = t.read(testTime + key).stringValue();\n            assertEquals(testData[testData.length - 1], actual2);\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Transaction#write(String, List)} and\n     * {@link Transaction#read(String)}. Writes lists and uses a single key for\n     * all the values. Tries to read the data afterwards.\n     *\n     * @throws NotFoundException\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws AbortException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testWriteList1() throws ConnectionException, UnknownException,\n            NotFoundException, AbortException {\n        final String key = \"_testWriteList1_\";\n        Transaction t = new Transaction();\n        try {\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final ArrayList<String> list = new ArrayList<String>();\n                list.add(testData[i]);\n                list.add(testData[i + 1]);\n                t.write(testTime + key + i, list);\n            }\n\n            // now try to read the data:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final List<String> actual = t.read(testTime + key + i).stringListValue();\n                final ArrayList<String> expected = new ArrayList<String>();\n                expected.add(testData[i]);\n                expected.add(testData[i + 1]);\n                assertEquals(expected, actual);\n            }\n\n            // try to read the data with another transaction (keys should not exist):\n            do {\n                final Transaction t2 = new Transaction();\n                for (int i = 0; i < (testData.length - 1); i += 2) {\n                    try {\n                        t2.read(testTime + key + i).stringListValue();\n                        // a key changed exception must be thrown\n                        assertTrue(false);\n                    } catch (final NotFoundException e) {\n                        t2.abort();\n                    }\n                }\n            } while (false);\n\n            // commit the transaction and try to read the data with a new one:\n            t.commit();\n            t = new Transaction();\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final ArrayList<String> expected = new ArrayList<String>();\n                expected.add(testData[i]);\n                expected.add(testData[i + 1]);\n                final List<String> actual = t.read(testTime + key + i).stringListValue();\n\n                assertEquals(expected, actual);\n            }\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Transaction#write(String, List)} and\n     * {@link Transaction#read(String)}. Writes lists and uses a single\n     * key for all the values. Tries to read the data afterwards.\n     *\n     * @throws NotFoundException\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws AbortException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testWriteList2() throws ConnectionException, UnknownException,\n            NotFoundException, AbortException {\n        final String key = \"_testWriteList1_\";\n        Transaction t = new Transaction();\n        try {\n            final ArrayList<String> list = new ArrayList<String>();\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                list.clear();\n                list.add(testData[i]);\n                list.add(testData[i + 1]);\n                t.write(testTime + key, list);\n            }\n\n            // now try to read the data:\n            final List<String> actual1 = t.read(testTime + key).stringListValue();\n            assertEquals(list, actual1);\n\n            // try to read the data with another transaction (keys should not exist):\n            do {\n                final Transaction t2 = new Transaction();\n                try {\n                    t2.read(testTime + key).stringValue();\n                    // a key changed exception must be thrown\n                    assertTrue(false);\n                } catch (final NotFoundException e) {\n                    t2.abort();\n                }\n            } while (false);\n\n            // commit the transaction and try to read the data with a new one:\n            t.commit();\n            t = new Transaction();\n            final List<String> actual2 = t.read(testTime + key).stringListValue();\n            assertEquals(list, actual2);\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link Transaction#testAndSet(String, Object, List)}\n     * with a closed connection.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 3.9\n     */\n    @Test(expected=ConnectionException.class)\n    public void testTestAndSetList_NotConnected() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException,\n            KeyChangedException {\n        final String key = \"_TestAndSetList_NotConnected\";\n        final Transaction t = new Transaction();\n        t.closeConnection();\n        final ArrayList<String> list = new ArrayList<String>();\n        list.add(testData[0]);\n        list.add(testData[1]);\n        t.testAndSet(testTime + key, \"fail\", list);\n    }\n\n    /**\n     * Test method for\n     * {@link Transaction#testAndSet(String, Object, List)}\n     * Tries test_and_set with a non-existing key.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 3.9\n     */\n    @Test(expected=NotFoundException.class)\n    public void testTestAndSetList_NotFound() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException,\n            KeyChangedException {\n        final String key = \"_TestAndSetList_NotFound\";\n        final Transaction t = new Transaction();\n\n        try {\n            final ArrayList<String> list = new ArrayList<String>();\n            list.add(testData[0]);\n            list.add(testData[1]);\n            t.testAndSet(testTime + key, \"fail\", list);\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link Transaction#testAndSet(String, List, List)},\n     * {@link Transaction#read(String)}\n     * and {@link Transaction#write(String, List)}.\n     * Writes an erlang list and tries to overwrite it using test_and_set\n     * knowing the correct old value. Tries to read the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testTestAndSetList1a() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException,\n            KeyChangedException {\n        final String key = \"_TestAndSetList1a\";\n        Transaction t = new Transaction();\n\n        try {\n            // first write all values:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final ArrayList<String> list = new ArrayList<String>();\n                list.add(testData[i]);\n                list.add(testData[i + 1]);\n                t.write(testTime + key + i, list);\n                t.commit();\n            }\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final ArrayList<String> old_list = new ArrayList<String>();\n                old_list.add(testData[i]);\n                old_list.add(testData[i + 1]);\n                final ArrayList<String> new_list = new ArrayList<String>();\n                new_list.add(testData[i + 1]);\n                new_list.add(testData[i]);\n                t.testAndSet(testTime + key + i, old_list, new_list);\n            }\n\n            // now try to read the data:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final List<String> actual = t.read(testTime + key + i).stringListValue();\n                final ArrayList<String> expected = new ArrayList<String>();\n                expected.add(testData[i + 1]);\n                expected.add(testData[i]);\n                assertEquals(expected, actual);\n            }\n\n            // try to read the data with another transaction (verify old value is read):\n            do {\n                final Transaction t2 = new Transaction();\n                for (int i = 0; i < (testData.length - 1); i += 2) {\n                    final List<String> actual = t2.read(testTime + key + i).stringListValue();\n                    final ArrayList<String> expected = new ArrayList<String>();\n                    expected.add(testData[i]);\n                    expected.add(testData[i + 1]);\n                    assertEquals(expected, actual);\n                    t2.commit();\n                }\n            } while (false);\n\n            // commit the transaction and try to read the data with a new one:\n            t.commit();\n            t = new Transaction();\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final List<String> actual = t.read(testTime + key + i).stringListValue();\n                final ArrayList<String> expected = new ArrayList<String>();\n                expected.add(testData[i + 1]);\n                expected.add(testData[i]);\n                assertEquals(expected, actual);\n                t.abort(); // do not accumulate state in tlog\n            }\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link Transaction#testAndSet(String, Object, Object)},\n     * {@link Transaction#read(String)}\n     * and {@link Transaction#write(String, List)}.\n     * Writes an erlang list and tries to overwrite it using test_and_set\n     * knowing the wrong old value. Tries to read the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testTestAndSetList1b() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final String key = \"_TestAndSetList1b\";\n        Transaction t = new Transaction();\n\n        try {\n            // first write all values:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final ArrayList<String> list = new ArrayList<String>();\n                list.add(testData[i]);\n                list.add(testData[i + 1]);\n                t.write(testTime + key + i, list);\n                t.commit();\n            }\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final int new_value = 1;\n                try {\n                    t.testAndSet(testTime + key + i, \"fail\", new_value);\n                    // a key changed exception must be thrown\n                    assertTrue(false);\n                } catch (final KeyChangedException e) {\n                    final ArrayList<String> expected = new ArrayList<String>();\n                    expected.add(testData[i]);\n                    expected.add(testData[i + 1]);\n                    assertEquals(expected, e.getOldValue().stringListValue());\n                }\n            }\n\n            // now try to read the data:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                t.read(testTime + key + i).stringListValue();\n            }\n\n            // try to read the data with another transaction (verify old value is read):\n            do {\n                final Transaction t2 = new Transaction();\n                for (int i = 0; i < (testData.length - 1); i += 2) {\n                    final List<String> actual = t2.read(testTime + key + i).stringListValue();\n                    final ArrayList<String> expected = new ArrayList<String>();\n                    expected.add(testData[i]);\n                    expected.add(testData[i + 1]);\n                    assertEquals(expected, actual);\n                    t2.commit();\n                }\n            } while (false);\n\n            // commit the transaction and try to read the data with a new one:\n            try {\n                t.commit();\n                // an AbortException must be thrown\n                assertTrue(false);\n            } catch (final AbortException e) {\n                final TreeSet<String> expFailKeys = new TreeSet<String>();\n                for (int i = 0; i < (testData.length - 1); i += 2) {\n                    expFailKeys.add(testTime + key + i);\n                }\n                assertEquals(expFailKeys, new TreeSet<String>(e.getFailedKeys()));\n                t.abort();\n            }\n            t = new Transaction();\n\n            // now try to read the data:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final List<String> actual = t.read(testTime + key + i).stringListValue();\n                final ArrayList<String> expected = new ArrayList<String>();\n                expected.add(testData[i]);\n                expected.add(testData[i + 1]);\n                assertEquals(expected, actual);\n                t.abort(); // do not accumulate state in tlog\n            }\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link Transaction#testAndSet(String, List, List)},\n     * {@link Transaction#read(String)} and\n     * {@link Transaction#write(String, List)}. Writes a list and tries\n     * to overwrite it using test_and_set knowing the correct old value and\n     * using a single key for all the values. Tries to read the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testTestAndSetList2a() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException,\n            KeyChangedException {\n        final String key = \"_TestAndSetList2a\";\n        Transaction t = new Transaction();\n\n        try {\n            // first write all values:\n            final ArrayList<String> list = new ArrayList<String>();\n            list.add(testData[0]);\n            list.add(testData[1]);\n            t.write(testTime + key, list);\n            t.commit();\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 1; i < (testData.length - 1); ++i) {\n                final ArrayList<String> old_list = new ArrayList<String>();\n                old_list.add(testData[i - 1]);\n                old_list.add(testData[i]);\n                final ArrayList<String> new_list = new ArrayList<String>();\n                new_list.add(testData[i]);\n                new_list.add(testData[i + 1]);\n                t.testAndSet(testTime + key, old_list, new_list);\n            }\n\n            final ArrayList<String> expected = new ArrayList<String>();\n            expected.add(testData[testData.length - 2]);\n            expected.add(testData[testData.length - 1]);\n\n            // now try to read the data:\n            final List<String> actual1 = t.read(testTime + key).stringListValue();\n            assertEquals(expected, actual1);\n\n            // try to read the data with another transaction (verify old value is read):\n            do {\n                final Transaction t2 = new Transaction();\n                final List<String> actual = t2.read(testTime + key).stringListValue();\n                assertEquals(list, actual);\n                t2.commit();\n            } while (false);\n\n            // commit the transaction and try to read the data with a new one:\n            t.commit();\n            t = new Transaction();\n            final List<String> actual2 = t.read(testTime + key).stringListValue();\n            assertEquals(expected, actual2);\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link Transaction#testAndSet(String, Object, Object)},\n     * {@link Transaction#read(String)} and\n     * {@link Transaction#write(String, List)}.\n     * Writes a list and tries to overwrite it using test_and_set knowing the\n     * wrong old value and using a single key for all the values. Tries to read\n     * the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testTestAndSetList2b() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final String key = \"_TestAndSetList2b\";\n        Transaction t = new Transaction();\n\n        try {\n            // first write all values:\n            final ArrayList<String> list = new ArrayList<String>();\n            list.add(testData[0]);\n            list.add(testData[1]);\n            t.write(testTime + key, list);\n            t.commit();\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 0; i < (testData.length - 1); ++i) {\n                final int new_value = 1;\n                try {\n                    t.testAndSet(testTime + key, \"fail\", new_value);\n                    // a key changed exception must be thrown\n                    assertTrue(false);\n                } catch (final KeyChangedException e) {\n                    final ArrayList<String> expected = new ArrayList<String>();\n                    expected.add(testData[0]);\n                    expected.add(testData[1]);\n                    assertEquals(expected, e.getOldValue().stringListValue());\n                }\n            }\n\n            // now try to read the data:\n            t.read(testTime + key).stringListValue();\n\n            // try to read the data with another transaction (verify old value is read):\n            do {\n                final Transaction t2 = new Transaction();\n                final List<String> actual = t2.read(testTime + key).stringListValue();\n                assertEquals(list, actual);\n                t2.commit();\n            } while (false);\n\n            // commit the transaction and try to read the data with a new one:\n            try {\n                t.commit();\n                // an AbortException must be thrown\n                assertTrue(false);\n            } catch (final AbortException e) {\n                final TreeSet<String> expFailKeys = new TreeSet<String>();\n                expFailKeys.add(testTime + key);\n                assertEquals(expFailKeys, new TreeSet<String>(e.getFailedKeys()));\n                t.abort();\n            }\n            t = new Transaction();\n            final List<String> actual2 = t.read(testTime + key).stringListValue();\n            assertEquals(list, actual2);\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link Transaction#testAndSet(String, String, String)}\n     * with a closed connection.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 3.9\n     */\n    @Test(expected=ConnectionException.class)\n    public void testTestAndSetString_NotConnected() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException,\n            KeyChangedException {\n        final String key = \"_TestAndSetString_NotConnected\";\n        final Transaction t = new Transaction();\n        t.closeConnection();\n        t.testAndSet(testTime + key, testData[0], testData[1]);\n    }\n\n    /**\n     * Test method for\n     * {@link Transaction#testAndSet(String, String, String)}.\n     * Tries test_and_set with a non-existing key.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 3.9\n     */\n    @Test(expected=NotFoundException.class)\n    public void testTestAndSetString_NotFound() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException,\n            KeyChangedException {\n        final String key = \"_TestAndSetString_NotFound\";\n        final Transaction t = new Transaction();\n\n        try {\n            t.testAndSet(testTime + key, testData[0], testData[1]);\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link Transaction#testAndSet(String, String, String)},\n     * {@link Transaction#read(String)}\n     * and {@link Transaction#write(String, Object)}.\n     * Writes a string and tries to overwrite it using test_and_set\n     * knowing the correct old value. Tries to read the string afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testTestAndSetString1a() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException,\n            KeyChangedException {\n        final String key = \"_TestAndSetString1a\";\n        Transaction t = new Transaction();\n\n        try {\n            // first write all values:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                t.write(testTime + key + i, testData[i]);\n                t.commit();\n            }\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                t.testAndSet(testTime + key + i, testData[i], testData[i + 1]);\n            }\n\n            // now try to read the data:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final String actual = t.read(testTime + key + i).stringValue();\n                final String expected = testData[i + 1];\n                assertEquals(expected, actual);\n            }\n\n            // try to read the data with another transaction (verify old value is read):\n            do {\n                final Transaction t2 = new Transaction();\n                for (int i = 0; i < (testData.length - 1); i += 2) {\n                    final String actual = t2.read(testTime + key + i).stringValue();\n                    final String expected = testData[i];\n                    assertEquals(expected, actual);\n                    t2.commit();\n                }\n            } while (false);\n\n            // commit the transaction and try to read the data with a new one:\n            t.commit();\n            t = new Transaction();\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final String actual = t.read(testTime + key + i).stringValue();\n                final String expected = testData[i + 1];\n                assertEquals(expected, actual);\n                t.abort(); // do not accumulate state in tlog\n            }\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link Transaction#testAndSet(String, String, String)},\n     * {@link Transaction#read(String)}\n     * and {@link Transaction#write(String, Object)}.\n     * Writes a string and tries to overwrite it using test_and_set\n     * knowing the wrong old value. Tries to read the string afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testTestAndSetString1b() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final String key = \"_TestAndSetString1b\";\n        Transaction t = new Transaction();\n\n        try {\n            // first write all values:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                t.write(testTime + key + i, testData[i]);\n                t.commit();\n            }\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                try {\n                    t.testAndSet(testTime + key + i, testData[i + 1], \"fail\");\n                    // a key changed exception must be thrown\n                    assertTrue(false);\n                } catch (final KeyChangedException e) {\n                    assertEquals(testData[i], e.getOldValue().stringValue());\n                }\n            }\n\n            // now try to read the data:\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                t.read(testTime + key + i).stringValue();\n            }\n\n            // try to read the data with another transaction (verify old value is read):\n            do {\n                final Transaction t2 = new Transaction();\n                for (int i = 0; i < (testData.length - 1); i += 2) {\n                    final String actual = t2.read(testTime + key + i).stringValue();\n                    final String expected = testData[i];\n                    assertEquals(expected, actual);\n                    t2.commit();\n                }\n            } while (false);\n\n            // commit the transaction and try to read the data with a new one:\n            try {\n                t.commit();\n                // an AbortException must be thrown\n                assertTrue(false);\n            } catch (final AbortException e) {\n                final TreeSet<String> expFailKeys = new TreeSet<String>();\n                for (int i = 0; i < (testData.length - 1); i += 2) {\n                    expFailKeys.add(testTime + key + i);\n                }\n                assertEquals(expFailKeys, new TreeSet<String>(e.getFailedKeys()));\n                t.abort();\n            }\n            t = new Transaction();\n            for (int i = 0; i < (testData.length - 1); i += 2) {\n                final String actual = t.read(testTime + key + i).stringValue();\n                final String expected = testData[i];\n                assertEquals(expected, actual);\n                t.abort(); // do not accumulate state in tlog\n            }\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link Transaction#testAndSet(String, String, String)},\n     * {@link Transaction#read(String)} and\n     * {@link Transaction#write(String, String)}. Writes a string and tries\n     * to overwrite it using test_and_set knowing the correct old value and\n     * using a single key for all the values. Tries to read the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     * @throws KeyChangedException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testTestAndSetString2a() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException,\n            KeyChangedException {\n        final String key = \"_TestAndSetString2a\";\n        Transaction t = new Transaction();\n\n        try {\n            // first write all values:\n            t.write(testTime + key, testData[0]);\n            t.commit();\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 1; i < (testData.length - 1); ++i) {\n                final String old_value = testData[i - 1];\n                final String new_value = testData[i];\n                t.testAndSet(testTime + key, old_value, new_value);\n            }\n\n            final String expected = testData[testData.length - 2];\n\n            // now try to read the data:\n            final String actual1 = t.read(testTime + key).stringValue();\n            assertEquals(expected, actual1);\n\n            // try to read the data with another transaction (verify old value is read):\n            do {\n                final Transaction t2 = new Transaction();\n                final String actual = t2.read(testTime + key).stringValue();\n                assertEquals(testData[0], actual);\n                t2.commit();\n            } while (false);\n\n            // commit the transaction and try to read the data with a new one:\n            t.commit();\n            t = new Transaction();\n            final String actual2 = t.read(testTime + key).stringValue();\n            assertEquals(expected, actual2);\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for\n     * {@link Transaction#testAndSet(String, Object, Object)},\n     * {@link Transaction#read(String)} and\n     * {@link Transaction#write(String, String)}.\n     * Writes a string and tries to overwrite it using test_and_set knowing the\n     * wrong old value and using a single key for all the values. Tries to read\n     * the data afterwards.\n     *\n     * @throws UnknownException\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws AbortException\n     *\n     * @since 3.9\n     */\n    @Test\n    public void testTestAndSetString2b() throws ConnectionException,\n            UnknownException, NotFoundException, AbortException {\n        final String key = \"_TestAndSetString2b\";\n        Transaction t = new Transaction();\n\n        try {\n            // first write all values:\n            final String expected = testData[0];\n            t.write(testTime + key, expected);\n            t.commit();\n\n            // now try to overwrite them using test_and_set:\n            for (int i = 0; i < (testData.length - 1); ++i) {\n                final int new_value = 1;\n                try {\n                    t.testAndSet(testTime + key, \"fail\", new_value);\n                    // a key changed exception must be thrown\n                    assertTrue(false);\n                } catch (final KeyChangedException e) {\n                    assertEquals(expected, e.getOldValue().stringValue());\n                }\n            }\n\n            // now try to read the data:\n            t.read(testTime + key).stringValue();\n\n            // try to read the data with another transaction (verify old value is read):\n            do {\n                final Transaction t2 = new Transaction();\n                final String actual = t2.read(testTime + key).stringValue();\n                assertEquals(expected, actual);\n                t2.commit();\n            } while (false);\n\n            // commit the transaction and try to read the data with a new one:\n            try {\n                t.commit();\n                // an AbortException must be thrown\n                assertTrue(false);\n            } catch (final AbortException e) {\n                final TreeSet<String> expFailKeys = new TreeSet<String>();\n                expFailKeys.add(testTime + key);\n                assertEquals(expFailKeys, new TreeSet<String>(e.getFailedKeys()));\n                t.abort();\n            }\n            t = new Transaction();\n            final String actual2 = t.read(testTime + key).stringValue();\n            assertEquals(expected, actual2);\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Transaction#req_list(RequestList)} with an empty\n     * request list.\n     *\n     * @throws ConnectionException\n     * @throws UnknownException\n     * @throws AbortException\n     */\n    @Test\n    public void testReqList_Empty() throws ConnectionException,\n            UnknownException, AbortException {\n        final Transaction t = new Transaction();\n        try {\n            t.req_list(new RequestList());\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Test method for {@link Transaction#req_list(RequestList)} with a mixed\n     * request list.\n     *\n     * @throws ConnectionException\n     * @throws UnknownException\n     * @throws AbortException\n     * @throws NotFoundException\n     */\n    @Test\n    public void testReqList1() throws ConnectionException, UnknownException,\n            AbortException, NotFoundException {\n        final String key = \"_ReqList1_\";\n        final Transaction conn = new Transaction();\n\n        try {\n            final RequestList readRequests = new RequestList();\n            final RequestList firstWriteRequests = new RequestList();\n            final RequestList writeRequests = new RequestList();\n            for (int i = 0; i < testData.length; ++i) {\n                if ((i % 2) == 0) {\n                    firstWriteRequests.addOp(new WriteOp(testTime + key + i, testData[i]));\n                }\n                writeRequests.addOp(new WriteOp(testTime + key + i, testData[i]));\n                readRequests.addOp(new ReadOp(testTime + key + i));\n            }\n\n            ResultList results = conn.req_list(firstWriteRequests);\n            // evaluate the first write results:\n            for (int i = 0; i < firstWriteRequests.size(); ++i) {\n                results.processWriteAt(i);\n            }\n\n            final RequestList requests = new RequestList(readRequests);\n            requests.addAll(writeRequests).addCommit();\n            results = conn.req_list(requests);\n            assertEquals(requests.size(), results.size());\n\n            // now evaluate the read results:\n            for (int i = 0; i < readRequests.size(); ++i) {\n                if ((i % 2) == 0) {\n                    final String actual = results.processReadAt(i).stringValue();\n                    assertEquals(testData[i], actual);\n                } else {\n                    try {\n                        results.processReadAt(i);\n                        // a not found exception must be thrown\n                        assertTrue(false);\n                    } catch (final NotFoundException e) {\n                    }\n                }\n            }\n\n            // now evaluate the write results:\n            for (int i = 0; i < writeRequests.size(); ++i) {\n                final int pos = readRequests.size() + i;\n                results.processWriteAt(pos);\n            }\n\n            // once again test reads - now all reads should be successful\n            results = conn.req_list(readRequests);\n            assertEquals(readRequests.size(), results.size());\n\n            // now evaluate the read results:\n            for (int i = 0; i < readRequests.size(); ++i) {\n                final String actual = results.processReadAt(i).stringValue();\n                assertEquals(testData[i], actual);\n            }\n        } finally {\n            conn.closeConnection();\n        }\n    }\n\n    /**\n     * Various tests.\n     *\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws UnknownException\n     * @throws AbortException\n     *\n     * @since 3.17\n     */\n    @Test\n    public void testVarious() throws ConnectionException, UnknownException,\n            NotFoundException, AbortException {\n        writeSingleTest(\"_0:Šarplaninac:page_\", testData[0]);\n    }\n\n    protected void writeSingleTest(final String key, final String data)\n            throws ConnectionException, UnknownException, ClassCastException,\n            NotFoundException, AbortException {\n        Transaction t = new Transaction();\n        try {\n            t.write(testTime + key, data);\n            // now try to read the data:\n            assertEquals(data, t.read(testTime + key).stringValue());\n            // try to read the data with another transaction (keys should not exist):\n            do {\n                final Transaction t2 = new Transaction();\n                try {\n                    t2.read(testTime + key).stringValue();\n                    // a not found exception must be thrown\n                    assertTrue(false);\n                } catch (final NotFoundException e) {\n                    t2.abort();\n                }\n            } while (false);\n\n            // commit the transaction and try to read the data with a new one:\n            t.commit();\n            t = new Transaction();\n            assertEquals(data, t.read(testTime + key).stringValue());\n        } finally {\n            t.closeConnection();\n        }\n    }\n\n    /**\n     * Tests the performance of the Transaction class with a very long list of\n     * random strings (enable manually).\n     *\n     * @throws ConnectionException\n     * @throws NotFoundException\n     * @throws ClassCastException\n     * @throws AbortException\n     * @throws UnknownException\n     */\n//    @Test\n    public void testPerformance1() throws ConnectionException,\n            NotFoundException, ClassCastException, AbortException,\n            UnknownException {\n        final String key = \"_testPerformance1_\";\n        final Transaction t = new Transaction();\n        try {\n            System.out.println((\"[\" + new Date()).toString() + \"] 1a\");\n            final ArrayList<String> list = new ArrayList<String>(100000);\n            for (int i = 0; i < 100000; ++i) {\n//                list.add(testData[i % testData.length]);\n                try {\n                    list.add(Benchmark.getRandom(20, String.class));\n                } catch (final Exception e) {\n                    e.printStackTrace();\n                }\n            }\n            System.out.println((\"[\" + new Date()).toString() + \"] 1b\");\n            final Transaction.RequestList reqs = new Transaction.RequestList();\n            reqs.addOp(new WriteOp(testTime + key, list)).addCommit();\n            System.out.println((\"[\" + new Date()).toString() + \"] 2a\");\n            t.req_list(reqs);\n            System.out.println((\"[\" + new Date()).toString() + \"] 2b\");\n\n            // commit the transaction and try to read the data with a new one:\n            final ErlangValue readVal = t.read(testTime + key);\n            System.out.println((\"[\" + new Date()).toString() + \"] 3\");\n            final List<String> actual = readVal.stringListValue();\n            System.out.println((\"[\" + new Date()).toString() + \"] 4\");\n            assertEquals(list, actual);\n//            reqs = new Transaction.RequestList();\n//            reqs.addCommit();\n            t.req_list(reqs);\n            System.out.println((\"[\" + new Date()).toString() + \"] 5\");\n        } finally {\n            t.closeConnection();\n        }\n    }\n}\n"
  },
  {
    "path": "java-api/test/de/zib/tools/PropertyLoaderTest.java",
    "content": "/**\n *  Copyright 2007-2011 Zuse Institute Berlin\n *\n *   Licensed under the Apache License, Version 2.0 (the \"License\");\n *   you may not use this file except in compliance with the License.\n *   You may obtain a copy of the License at\n *\n *       http://www.apache.org/licenses/LICENSE-2.0\n *\n *   Unless required by applicable law or agreed to in writing, software\n *   distributed under the License is distributed on an \"AS IS\" BASIS,\n *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n *   See the License for the specific language governing permissions and\n *   limitations under the License.\n */\npackage de.zib.tools;\n\nimport static org.junit.Assert.*;\n\nimport java.io.UnsupportedEncodingException;\nimport java.net.URL;\nimport java.net.URLDecoder;\nimport java.util.Properties;\n\nimport org.junit.After;\nimport org.junit.AfterClass;\nimport org.junit.Before;\nimport org.junit.BeforeClass;\nimport org.junit.Test;\n\n/**\n * Unit test for the {@link PropertyLoader} class.\n *\n * @author Nico Kruber, kruber@zib.de\n */\npublic class PropertyLoaderTest {\n\n    /**\n     * @throws java.lang.Exception\n     */\n    @BeforeClass\n    public static void setUpBeforeClass() throws Exception {\n    }\n\n    /**\n     * @throws java.lang.Exception\n     */\n    @AfterClass\n    public static void tearDownAfterClass() throws Exception {\n    }\n\n    /**\n     * @throws java.lang.Exception\n     */\n    @Before\n    public void setUp() throws Exception {\n    }\n\n    /**\n     * @throws java.lang.Exception\n     */\n    @After\n    public void tearDown() throws Exception {\n    }\n\n    /**\n     * Test method for {@link PropertyLoader#loadProperties(Properties, String)}\n     * that tries to load a properties file using an relative file name.\n     */\n    @Test\n    public final void testLoadProperties1() {\n        final Properties properties = new Properties();\n        assertTrue(PropertyLoader.loadProperties(properties, \"de/zib/tools/test.properties\"));\n        assertEquals(\"ahz2ieSh\", properties.get(\"cs.node\"));\n        assertEquals(\"wooPhu8u\", properties.get(\"cs.cookie\"));\n    }\n\n    /**\n     * Test method for {@link PropertyLoader#loadProperties(Properties, String)}\n     * that tries to load a properties file using an absolute file name.\n     *\n     * @throws UnsupportedEncodingException\n     *             if the path cannot be decoded using UTF-8\n     */\n    @Test\n    public final void testLoadProperties2() throws UnsupportedEncodingException {\n        System.out.println(\"Working Directory = \" + System.getProperty(\"user.dir\"));\n\n        final URL url = ClassLoader.getSystemResource(\"de/zib/tools/test.properties\");\n        assertNotNull(url);\n        final Properties properties = new Properties();\n        assertTrue(PropertyLoader.loadProperties(properties, URLDecoder.decode(url.getFile(), \"UTF-8\")));\n        assertEquals(\"ahz2ieSh\", properties.get(\"cs.node\"));\n        assertEquals(\"wooPhu8u\", properties.get(\"cs.cookie\"));\n    }\n\n    /**\n     * Test method for {@link PropertyLoader#loadProperties(Properties, String)}.\n     */\n    @Test\n    public final void testLoadProperties3() {\n        final Properties properties = new Properties();\n        assertFalse(PropertyLoader.loadProperties(properties, \"de/zib/tools/ahz2ieSh.wooPhu8u\"));\n    }\n\n}\n"
  },
  {
    "path": "java-api/test/de/zib/tools/test.properties",
    "content": "cs.node=ahz2ieSh\ncs.cookie=wooPhu8u\n"
  },
  {
    "path": "java-api/test/scalaris.properties",
    "content": "# the name of the scalaris node to establish the connection to\nscalaris.node=node1@localhost\n\n# the cookie the scalaris node uses for connections\nscalaris.cookie=chocolate chip cookie\n\n# the name of the (Java) client to use when establishing a connection with erlang\nscalaris.client.name=java_client\n\n# specifies whether to append an UUID to client names or not\nscalaris.client.appendUUID=true\n"
  },
  {
    "path": "log/.gitignore",
    "content": "/*\n"
  },
  {
    "path": "m4/ax_boost_asio.m4",
    "content": "# ===========================================================================\n#       http://www.gnu.org/software/autoconf-archive/ax_boost_asio.html\n# ===========================================================================\n#\n# SYNOPSIS\n#\n#   AX_BOOST_ASIO\n#\n# DESCRIPTION\n#\n#   Test for Asio library from the Boost C++ libraries. The macro requires a\n#   preceding call to AX_BOOST_BASE. Further documentation is available at\n#   <http://randspringer.de/boost/index.html>.\n#\n#   This macro calls:\n#\n#     AC_SUBST(BOOST_ASIO_LIB)\n#\n#   And sets:\n#\n#     HAVE_BOOST_ASIO\n#\n# LICENSE\n#\n#   Copyright (c) 2008 Thomas Porschberg <thomas@randspringer.de>\n#   Copyright (c) 2008 Pete Greenwell <pete@mu.org>\n#\n#   Copying and distribution of this file, with or without modification, are\n#   permitted in any medium without royalty provided the copyright notice\n#   and this notice are preserved. This file is offered as-is, without any\n#   warranty.\n\n#serial 16\n\nAC_DEFUN([AX_BOOST_ASIO],\n[\n\tAC_ARG_WITH([boost-asio],\n\tAS_HELP_STRING([--with-boost-asio@<:@=special-lib@:>@],\n                   [use the ASIO library from boost - it is possible to specify a certain library for the linker\n                        e.g. --with-boost-asio=boost_system-gcc41-mt-1_34 ]),\n        [\n        if test \"$withval\" = \"no\"; then\n\t\t\twant_boost=\"no\"\n        elif test \"$withval\" = \"yes\"; then\n            want_boost=\"yes\"\n            ax_boost_user_asio_lib=\"\"\n        else\n\t\t    want_boost=\"yes\"\n\t\tax_boost_user_asio_lib=\"$withval\"\n\t\tfi\n        ],\n        [want_boost=\"yes\"]\n\t)\n\n\tif test \"x$want_boost\" = \"xyes\"; then\n        AC_REQUIRE([AC_PROG_CC])\n\t\tCPPFLAGS_SAVED=\"$CPPFLAGS\"\n\t\tCPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n\t\texport CPPFLAGS\n\n\t\tLDFLAGS_SAVED=\"$LDFLAGS\"\n\t\tLDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n\t\texport LDFLAGS\n\n        AC_CACHE_CHECK(whether the Boost::ASIO library is available,\n\t\t\t\t\t   ax_cv_boost_asio,\n        [AC_LANG_PUSH([C++])\n\t\t AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ @%:@include <boost/asio.hpp>\n\t\t\t\t\t\t\t\t\t\t\t]],\n                                  [[\n\n                                    boost::asio::io_service io;\n                                    boost::system::error_code timer_result;\n                                    boost::asio::deadline_timer t(io);\n                                    t.cancel();\n                                    io.run_one();\n\t\t\t\t\t\t\t\t\treturn 0;\n                                   ]])],\n                             ax_cv_boost_asio=yes, ax_cv_boost_asio=no)\n         AC_LANG_POP([C++])\n\t\t])\n\t\tif test \"x$ax_cv_boost_asio\" = \"xyes\"; then\n\t\t\tAC_DEFINE(HAVE_BOOST_ASIO,,[define if the Boost::ASIO library is available])\n\t\t\tBN=boost_system\n\t\t\tBOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\\/@:>@*//'`\n            if test \"x$ax_boost_user_asio_lib\" = \"x\"; then\n\t\t\t\tfor ax_lib in `ls $BOOSTLIBDIR/libboost_system*.so* $BOOSTLIBDIR/libboost_system*.dylib* $BOOSTLIBDIR/libboost_system*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\\(boost_system.*\\)\\.so.*$;\\1;' -e 's;^lib\\(boost_system.*\\)\\.dylib.*$;\\1;' -e 's;^lib\\(boost_system.*\\)\\.a.*$;\\1;' ` ; do\n\t\t\t\t    AC_CHECK_LIB($ax_lib, main, [BOOST_ASIO_LIB=\"-l$ax_lib\" AC_SUBST(BOOST_ASIO_LIB) link_thread=\"yes\" break],\n                                 [link_thread=\"no\"])\n\t\t\t\tdone\n            else\n               for ax_lib in $ax_boost_user_asio_lib $BN-$ax_boost_user_asio_lib; do\n\t\t\t\t      AC_CHECK_LIB($ax_lib, main,\n                                   [BOOST_ASIO_LIB=\"-l$ax_lib\" AC_SUBST(BOOST_ASIO_LIB) link_asio=\"yes\" break],\n                                   [link_asio=\"no\"])\n                  done\n\n            fi\n            if test \"x$ax_lib\" = \"x\"; then\n                AC_MSG_ERROR(Could not find a version of the library!)\n            fi\n\t\t\tif test \"x$link_asio\" = \"xno\"; then\n\t\t\t\tAC_MSG_ERROR(Could not link against $ax_lib !)\n\t\t\tfi\n\t\tfi\n\n\t\tCPPFLAGS=\"$CPPFLAGS_SAVED\"\n\tLDFLAGS=\"$LDFLAGS_SAVED\"\n\tfi\n])\n"
  },
  {
    "path": "m4/ax_boost_base.m4",
    "content": "# ===========================================================================\n#       http://www.gnu.org/software/autoconf-archive/ax_boost_base.html\n# ===========================================================================\n#\n# SYNOPSIS\n#\n#   AX_BOOST_BASE([MINIMUM-VERSION], [ACTION-IF-FOUND], [ACTION-IF-NOT-FOUND])\n#\n# DESCRIPTION\n#\n#   Test for the Boost C++ libraries of a particular version (or newer)\n#\n#   If no path to the installed boost library is given the macro searchs\n#   under /usr, /usr/local, /opt and /opt/local and evaluates the\n#   $BOOST_ROOT environment variable. Further documentation is available at\n#   <http://randspringer.de/boost/index.html>.\n#\n#   This macro calls:\n#\n#     AC_SUBST(BOOST_CPPFLAGS) / AC_SUBST(BOOST_LDFLAGS)\n#\n#   And sets:\n#\n#     HAVE_BOOST\n#\n# LICENSE\n#\n#   Copyright (c) 2008 Thomas Porschberg <thomas@randspringer.de>\n#   Copyright (c) 2009 Peter Adolphs\n#\n#   Copying and distribution of this file, with or without modification, are\n#   permitted in any medium without royalty provided the copyright notice\n#   and this notice are preserved. This file is offered as-is, without any\n#   warranty.\n\n#serial 25\n\nAC_DEFUN([AX_BOOST_BASE],\n[\nAC_ARG_WITH([boost],\n  [AS_HELP_STRING([--with-boost@<:@=ARG@:>@],\n    [use Boost library from a standard location (ARG=yes),\n     from the specified location (ARG=<path>),\n     or disable it (ARG=no)\n     @<:@ARG=yes@:>@ ])],\n    [\n    if test \"$withval\" = \"no\"; then\n        want_boost=\"no\"\n    elif test \"$withval\" = \"yes\"; then\n        want_boost=\"yes\"\n        ac_boost_path=\"\"\n    else\n        want_boost=\"yes\"\n        ac_boost_path=\"$withval\"\n    fi\n    ],\n    [want_boost=\"yes\"])\n\n\nAC_ARG_WITH([boost-libdir],\n        AS_HELP_STRING([--with-boost-libdir=LIB_DIR],\n        [Force given directory for boost libraries. Note that this will override library path detection, so use this parameter only if default library detection fails and you know exactly where your boost libraries are located.]),\n        [\n        if test -d \"$withval\"\n        then\n                ac_boost_lib_path=\"$withval\"\n        else\n                AC_MSG_ERROR(--with-boost-libdir expected directory name)\n        fi\n        ],\n        [ac_boost_lib_path=\"\"]\n)\n\nif test \"x$want_boost\" = \"xyes\"; then\n    boost_lib_version_req=ifelse([$1], ,1.20.0,$1)\n    boost_lib_version_req_shorten=`expr $boost_lib_version_req : '\\([[0-9]]*\\.[[0-9]]*\\)'`\n    boost_lib_version_req_major=`expr $boost_lib_version_req : '\\([[0-9]]*\\)'`\n    boost_lib_version_req_minor=`expr $boost_lib_version_req : '[[0-9]]*\\.\\([[0-9]]*\\)'`\n    boost_lib_version_req_sub_minor=`expr $boost_lib_version_req : '[[0-9]]*\\.[[0-9]]*\\.\\([[0-9]]*\\)'`\n    if test \"x$boost_lib_version_req_sub_minor\" = \"x\" ; then\n        boost_lib_version_req_sub_minor=\"0\"\n        fi\n    WANT_BOOST_VERSION=`expr $boost_lib_version_req_major \\* 100000 \\+  $boost_lib_version_req_minor \\* 100 \\+ $boost_lib_version_req_sub_minor`\n    AC_MSG_CHECKING(for boostlib >= $boost_lib_version_req)\n    succeeded=no\n\n    dnl On 64-bit systems check for system libraries in both lib64 and lib.\n    dnl The former is specified by FHS, but e.g. Debian does not adhere to\n    dnl this (as it rises problems for generic multi-arch support).\n    dnl The last entry in the list is chosen by default when no libraries\n    dnl are found, e.g. when only header-only libraries are installed!\n    libsubdirs=\"lib\"\n    ax_arch=`uname -m`\n    case $ax_arch in\n      x86_64)\n        libsubdirs=\"lib64 libx32 lib lib64\"\n        ;;\n      ppc64|s390x|sparc64|aarch64|ppc64le)\n        libsubdirs=\"lib64 lib lib64 ppc64le\"\n        ;;\n    esac\n\n    dnl allow for real multi-arch paths e.g. /usr/lib/x86_64-linux-gnu. Give\n    dnl them priority over the other paths since, if libs are found there, they\n    dnl are almost assuredly the ones desired.\n    AC_REQUIRE([AC_CANONICAL_HOST])\n    libsubdirs=\"lib/${host_cpu}-${host_os} $libsubdirs\"\n\n    case ${host_cpu} in\n      i?86)\n        libsubdirs=\"lib/i386-${host_os} $libsubdirs\"\n        ;;\n    esac\n\n    dnl first we check the system location for boost libraries\n    dnl this location ist chosen if boost libraries are installed with the --layout=system option\n    dnl or if you install boost with RPM\n    if test \"$ac_boost_path\" != \"\"; then\n        BOOST_CPPFLAGS=\"-I$ac_boost_path/include\"\n        for ac_boost_path_tmp in $libsubdirs; do\n                if test -d \"$ac_boost_path\"/\"$ac_boost_path_tmp\" ; then\n                        BOOST_LDFLAGS=\"-L$ac_boost_path/$ac_boost_path_tmp\"\n                        break\n                fi\n        done\n    elif test \"$cross_compiling\" != yes; then\n        for ac_boost_path_tmp in /usr /usr/local /opt /opt/local ; do\n            if test -d \"$ac_boost_path_tmp/include/boost\" && test -r \"$ac_boost_path_tmp/include/boost\"; then\n                for libsubdir in $libsubdirs ; do\n                    if ls \"$ac_boost_path_tmp/$libsubdir/libboost_\"* >/dev/null 2>&1 ; then break; fi\n                done\n                BOOST_LDFLAGS=\"-L$ac_boost_path_tmp/$libsubdir\"\n                BOOST_CPPFLAGS=\"-I$ac_boost_path_tmp/include\"\n                break;\n            fi\n        done\n    fi\n\n    dnl overwrite ld flags if we have required special directory with\n    dnl --with-boost-libdir parameter\n    if test \"$ac_boost_lib_path\" != \"\"; then\n       BOOST_LDFLAGS=\"-L$ac_boost_lib_path\"\n    fi\n\n    CPPFLAGS_SAVED=\"$CPPFLAGS\"\n    CPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n    export CPPFLAGS\n\n    LDFLAGS_SAVED=\"$LDFLAGS\"\n    LDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n    export LDFLAGS\n\n    AC_REQUIRE([AC_PROG_CXX])\n    AC_LANG_PUSH(C++)\n        AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[\n    @%:@include <boost/version.hpp>\n    ]], [[\n    #if BOOST_VERSION >= $WANT_BOOST_VERSION\n    // Everything is okay\n    #else\n    #  error Boost version is too old\n    #endif\n    ]])],[\n        AC_MSG_RESULT(yes)\n    succeeded=yes\n    found_system=yes\n        ],[\n        ])\n    AC_LANG_POP([C++])\n\n\n\n    dnl if we found no boost with system layout we search for boost libraries\n    dnl built and installed without the --layout=system option or for a staged(not installed) version\n    if test \"x$succeeded\" != \"xyes\"; then\n        _version=0\n        if test \"$ac_boost_path\" != \"\"; then\n            if test -d \"$ac_boost_path\" && test -r \"$ac_boost_path\"; then\n                for i in `ls -d $ac_boost_path/include/boost-* 2>/dev/null`; do\n                    _version_tmp=`echo $i | sed \"s#$ac_boost_path##\" | sed 's/\\/include\\/boost-//' | sed 's/_/./'`\n                    V_CHECK=`expr $_version_tmp \\> $_version`\n                    if test \"$V_CHECK\" = \"1\" ; then\n                        _version=$_version_tmp\n                    fi\n                    VERSION_UNDERSCORE=`echo $_version | sed 's/\\./_/'`\n                    BOOST_CPPFLAGS=\"-I$ac_boost_path/include/boost-$VERSION_UNDERSCORE\"\n                done\n            fi\n        else\n            if test \"$cross_compiling\" != yes; then\n                for ac_boost_path in /usr /usr/local /opt /opt/local ; do\n                    if test -d \"$ac_boost_path\" && test -r \"$ac_boost_path\"; then\n                        for i in `ls -d $ac_boost_path/include/boost-* 2>/dev/null`; do\n                            _version_tmp=`echo $i | sed \"s#$ac_boost_path##\" | sed 's/\\/include\\/boost-//' | sed 's/_/./'`\n                            V_CHECK=`expr $_version_tmp \\> $_version`\n                            if test \"$V_CHECK\" = \"1\" ; then\n                                _version=$_version_tmp\n                                best_path=$ac_boost_path\n                            fi\n                        done\n                    fi\n                done\n\n                VERSION_UNDERSCORE=`echo $_version | sed 's/\\./_/'`\n                BOOST_CPPFLAGS=\"-I$best_path/include/boost-$VERSION_UNDERSCORE\"\n                if test \"$ac_boost_lib_path\" = \"\"; then\n                    for libsubdir in $libsubdirs ; do\n                        if ls \"$best_path/$libsubdir/libboost_\"* >/dev/null 2>&1 ; then break; fi\n                    done\n                    BOOST_LDFLAGS=\"-L$best_path/$libsubdir\"\n                fi\n            fi\n\n            if test \"x$BOOST_ROOT\" != \"x\"; then\n                for libsubdir in $libsubdirs ; do\n                    if ls \"$BOOST_ROOT/stage/$libsubdir/libboost_\"* >/dev/null 2>&1 ; then break; fi\n                done\n                if test -d \"$BOOST_ROOT\" && test -r \"$BOOST_ROOT\" && test -d \"$BOOST_ROOT/stage/$libsubdir\" && test -r \"$BOOST_ROOT/stage/$libsubdir\"; then\n                    version_dir=`expr //$BOOST_ROOT : '.*/\\(.*\\)'`\n                    stage_version=`echo $version_dir | sed 's/boost_//' | sed 's/_/./g'`\n                        stage_version_shorten=`expr $stage_version : '\\([[0-9]]*\\.[[0-9]]*\\)'`\n                    V_CHECK=`expr $stage_version_shorten \\>\\= $_version`\n                    if test \"$V_CHECK\" = \"1\" -a \"$ac_boost_lib_path\" = \"\" ; then\n                        AC_MSG_NOTICE(We will use a staged boost library from $BOOST_ROOT)\n                        BOOST_CPPFLAGS=\"-I$BOOST_ROOT\"\n                        BOOST_LDFLAGS=\"-L$BOOST_ROOT/stage/$libsubdir\"\n                    fi\n                fi\n            fi\n        fi\n\n        CPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n        export CPPFLAGS\n        LDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n        export LDFLAGS\n\n        AC_LANG_PUSH(C++)\n            AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[\n        @%:@include <boost/version.hpp>\n        ]], [[\n        #if BOOST_VERSION >= $WANT_BOOST_VERSION\n        // Everything is okay\n        #else\n        #  error Boost version is too old\n        #endif\n        ]])],[\n            AC_MSG_RESULT(yes)\n        succeeded=yes\n        found_system=yes\n            ],[\n            ])\n        AC_LANG_POP([C++])\n    fi\n\n    if test \"$succeeded\" != \"yes\" ; then\n        if test \"$_version\" = \"0\" ; then\n            AC_MSG_NOTICE([[We could not detect the boost libraries (version $boost_lib_version_req_shorten or higher). If you have a staged boost library (still not installed) please specify \\$BOOST_ROOT in your environment and do not give a PATH to --with-boost option.  If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.]])\n        else\n            AC_MSG_NOTICE([Your boost libraries seems to old (version $_version).])\n        fi\n        # execute ACTION-IF-NOT-FOUND (if present):\n        ifelse([$3], , :, [$3])\n    else\n        AC_SUBST(BOOST_CPPFLAGS)\n        AC_SUBST(BOOST_LDFLAGS)\n        AC_DEFINE(HAVE_BOOST,,[define if the Boost library is available])\n        # execute ACTION-IF-FOUND (if present):\n        ifelse([$2], , :, [$2])\n    fi\n\n    CPPFLAGS=\"$CPPFLAGS_SAVED\"\n    LDFLAGS=\"$LDFLAGS_SAVED\"\nfi\n\n])\n"
  },
  {
    "path": "m4/ax_boost_program_options.m4",
    "content": "# ============================================================================\n#  http://www.gnu.org/software/autoconf-archive/ax_boost_program_options.html\n# ============================================================================\n#\n# SYNOPSIS\n#\n#   AX_BOOST_PROGRAM_OPTIONS\n#\n# DESCRIPTION\n#\n#   Test for program options library from the Boost C++ libraries. The macro\n#   requires a preceding call to AX_BOOST_BASE. Further documentation is\n#   available at <http://randspringer.de/boost/index.html>.\n#\n#   This macro calls:\n#\n#     AC_SUBST(BOOST_PROGRAM_OPTIONS_LIB)\n#\n#   And sets:\n#\n#     HAVE_BOOST_PROGRAM_OPTIONS\n#\n# LICENSE\n#\n#   Copyright (c) 2009 Thomas Porschberg <thomas@randspringer.de>\n#\n#   Copying and distribution of this file, with or without modification, are\n#   permitted in any medium without royalty provided the copyright notice\n#   and this notice are preserved. This file is offered as-is, without any\n#   warranty.\n\n#serial 22\n\nAC_DEFUN([AX_BOOST_PROGRAM_OPTIONS],\n[\n\tAC_ARG_WITH([boost-program-options],\n\t\tAS_HELP_STRING([--with-boost-program-options@<:@=special-lib@:>@],\n                       [use the program options library from boost - it is possible to specify a certain library for the linker\n                        e.g. --with-boost-program-options=boost_program_options-gcc-mt-1_33_1 ]),\n        [\n        if test \"$withval\" = \"no\"; then\n\t\t\twant_boost=\"no\"\n        elif test \"$withval\" = \"yes\"; then\n            want_boost=\"yes\"\n            ax_boost_user_program_options_lib=\"\"\n        else\n\t\t    want_boost=\"yes\"\n\t\tax_boost_user_program_options_lib=\"$withval\"\n\t\tfi\n        ],\n        [want_boost=\"yes\"]\n\t)\n\n\tif test \"x$want_boost\" = \"xyes\"; then\n        AC_REQUIRE([AC_PROG_CC])\n\t    export want_boost\n\t\tCPPFLAGS_SAVED=\"$CPPFLAGS\"\n\t\tCPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n\t\texport CPPFLAGS\n\t\tLDFLAGS_SAVED=\"$LDFLAGS\"\n\t\tLDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n\t\texport LDFLAGS\n\t\tAC_CACHE_CHECK([whether the Boost::Program_Options library is available],\n\t\t\t\t\t   ax_cv_boost_program_options,\n\t\t\t\t\t   [AC_LANG_PUSH(C++)\n\t\t\t\tAC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include <boost/program_options.hpp>\n                                                          ]],\n                                  [[boost::program_options::options_description generic(\"Generic options\");\n                                   return 0;]])],\n                           ax_cv_boost_program_options=yes, ax_cv_boost_program_options=no)\n\t\t\t\t\tAC_LANG_POP([C++])\n\t\t])\n\t\tif test \"$ax_cv_boost_program_options\" = yes; then\n\t\t\t\tAC_DEFINE(HAVE_BOOST_PROGRAM_OPTIONS,,[define if the Boost::PROGRAM_OPTIONS library is available])\n                  BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\\/@:>@*//'`\n                if test \"x$ax_boost_user_program_options_lib\" = \"x\"; then\n                for libextension in `ls $BOOSTLIBDIR/libboost_program_options*.so* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\\(boost_program_options.*\\)\\.so.*$;\\1;'` `ls $BOOSTLIBDIR/libboost_program_options*.dylib* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\\(boost_program_options.*\\)\\.dylib.*$;\\1;'` `ls $BOOSTLIBDIR/libboost_program_options*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\\(boost_program_options.*\\)\\.a.*$;\\1;'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    AC_CHECK_LIB($ax_lib, exit,\n                                 [BOOST_PROGRAM_OPTIONS_LIB=\"-l$ax_lib\"; AC_SUBST(BOOST_PROGRAM_OPTIONS_LIB) link_program_options=\"yes\"; break],\n                                 [link_program_options=\"no\"])\n\t\t\t\tdone\n                if test \"x$link_program_options\" != \"xyes\"; then\n                for libextension in `ls $BOOSTLIBDIR/boost_program_options*.dll* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\\(boost_program_options.*\\)\\.dll.*$;\\1;'` `ls $BOOSTLIBDIR/boost_program_options*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\\(boost_program_options.*\\)\\.a.*$;\\1;'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    AC_CHECK_LIB($ax_lib, exit,\n                                 [BOOST_PROGRAM_OPTIONS_LIB=\"-l$ax_lib\"; AC_SUBST(BOOST_PROGRAM_OPTIONS_LIB) link_program_options=\"yes\"; break],\n                                 [link_program_options=\"no\"])\n\t\t\t\tdone\n                fi\n                else\n                  for ax_lib in $ax_boost_user_program_options_lib boost_program_options-$ax_boost_user_program_options_lib; do\n\t\t\t\t      AC_CHECK_LIB($ax_lib, main,\n                                   [BOOST_PROGRAM_OPTIONS_LIB=\"-l$ax_lib\"; AC_SUBST(BOOST_PROGRAM_OPTIONS_LIB) link_program_options=\"yes\"; break],\n                                   [link_program_options=\"no\"])\n                  done\n                fi\n            if test \"x$ax_lib\" = \"x\"; then\n                AC_MSG_ERROR(Could not find a version of the library!)\n            fi\n\t\t\t\tif test \"x$link_program_options\" != \"xyes\"; then\n\t\t\t\t\tAC_MSG_ERROR([Could not link against [$ax_lib] !])\n\t\t\t\tfi\n\t\tfi\n\t\tCPPFLAGS=\"$CPPFLAGS_SAVED\"\n\tLDFLAGS=\"$LDFLAGS_SAVED\"\n\tfi\n])\n"
  },
  {
    "path": "m4/ax_boost_regex.m4",
    "content": "# ===========================================================================\n#      http://www.gnu.org/software/autoconf-archive/ax_boost_regex.html\n# ===========================================================================\n#\n# SYNOPSIS\n#\n#   AX_BOOST_REGEX\n#\n# DESCRIPTION\n#\n#   Test for Regex library from the Boost C++ libraries. The macro requires\n#   a preceding call to AX_BOOST_BASE. Further documentation is available at\n#   <http://randspringer.de/boost/index.html>.\n#\n#   This macro calls:\n#\n#     AC_SUBST(BOOST_REGEX_LIB)\n#\n#   And sets:\n#\n#     HAVE_BOOST_REGEX\n#\n# LICENSE\n#\n#   Copyright (c) 2008 Thomas Porschberg <thomas@randspringer.de>\n#   Copyright (c) 2008 Michael Tindal\n#\n#   Copying and distribution of this file, with or without modification, are\n#   permitted in any medium without royalty provided the copyright notice\n#   and this notice are preserved. This file is offered as-is, without any\n#   warranty.\n\n#serial 22\n\nAC_DEFUN([AX_BOOST_REGEX],\n[\n\tAC_ARG_WITH([boost-regex],\n\tAS_HELP_STRING([--with-boost-regex@<:@=special-lib@:>@],\n                   [use the Regex library from boost - it is possible to specify a certain library for the linker\n                        e.g. --with-boost-regex=boost_regex-gcc-mt-d-1_33_1 ]),\n        [\n        if test \"$withval\" = \"no\"; then\n\t\t\twant_boost=\"no\"\n        elif test \"$withval\" = \"yes\"; then\n            want_boost=\"yes\"\n            ax_boost_user_regex_lib=\"\"\n        else\n\t\t    want_boost=\"yes\"\n\t\tax_boost_user_regex_lib=\"$withval\"\n\t\tfi\n        ],\n        [want_boost=\"yes\"]\n\t)\n\n\tif test \"x$want_boost\" = \"xyes\"; then\n        AC_REQUIRE([AC_PROG_CC])\n\t\tCPPFLAGS_SAVED=\"$CPPFLAGS\"\n\t\tCPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n\t\texport CPPFLAGS\n\n\t\tLDFLAGS_SAVED=\"$LDFLAGS\"\n\t\tLDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n\t\texport LDFLAGS\n\n        AC_CACHE_CHECK(whether the Boost::Regex library is available,\n\t\t\t\t\t   ax_cv_boost_regex,\n        [AC_LANG_PUSH([C++])\n\t\t\t AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include <boost/regex.hpp>\n\t\t\t\t\t\t\t\t\t\t\t\t]],\n                                   [[boost::regex r(); return 0;]])],\n                   ax_cv_boost_regex=yes, ax_cv_boost_regex=no)\n         AC_LANG_POP([C++])\n\t\t])\n\t\tif test \"x$ax_cv_boost_regex\" = \"xyes\"; then\n\t\t\tAC_DEFINE(HAVE_BOOST_REGEX,,[define if the Boost::Regex library is available])\n            BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\\/@:>@*//'`\n            if test \"x$ax_boost_user_regex_lib\" = \"x\"; then\n                for libextension in `ls $BOOSTLIBDIR/libboost_regex*.so* $BOOSTLIBDIR/libboost_regex*.dylib* $BOOSTLIBDIR/libboost_regex*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\\(boost_regex.*\\)\\.so.*$;\\1;' -e 's;^lib\\(boost_regex.*\\)\\.dylib.*;\\1;' -e 's;^lib\\(boost_regex.*\\)\\.a.*$;\\1;'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    AC_CHECK_LIB($ax_lib, exit,\n                                 [BOOST_REGEX_LIB=\"-l$ax_lib\"; AC_SUBST(BOOST_REGEX_LIB) link_regex=\"yes\"; break],\n                                 [link_regex=\"no\"])\n\t\t\t\tdone\n                if test \"x$link_regex\" != \"xyes\"; then\n                for libextension in `ls $BOOSTLIBDIR/boost_regex*.dll* $BOOSTLIBDIR/boost_regex*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\\(boost_regex.*\\)\\.dll.*$;\\1;' -e 's;^\\(boost_regex.*\\)\\.a.*$;\\1;'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    AC_CHECK_LIB($ax_lib, exit,\n                                 [BOOST_REGEX_LIB=\"-l$ax_lib\"; AC_SUBST(BOOST_REGEX_LIB) link_regex=\"yes\"; break],\n                                 [link_regex=\"no\"])\n\t\t\t\tdone\n                fi\n\n            else\n               for ax_lib in $ax_boost_user_regex_lib boost_regex-$ax_boost_user_regex_lib; do\n\t\t\t\t      AC_CHECK_LIB($ax_lib, main,\n                                   [BOOST_REGEX_LIB=\"-l$ax_lib\"; AC_SUBST(BOOST_REGEX_LIB) link_regex=\"yes\"; break],\n                                   [link_regex=\"no\"])\n               done\n            fi\n            if test \"x$ax_lib\" = \"x\"; then\n                AC_MSG_ERROR(Could not find a version of the Boost::Regex library!)\n            fi\n\t\t\tif test \"x$link_regex\" != \"xyes\"; then\n\t\t\t\tAC_MSG_ERROR(Could not link against $ax_lib !)\n\t\t\tfi\n\t\tfi\n\n\t\tCPPFLAGS=\"$CPPFLAGS_SAVED\"\n\tLDFLAGS=\"$LDFLAGS_SAVED\"\n\tfi\n])\n"
  },
  {
    "path": "m4/ax_boost_system.m4",
    "content": "# ===========================================================================\n#      http://www.gnu.org/software/autoconf-archive/ax_boost_system.html\n# ===========================================================================\n#\n# SYNOPSIS\n#\n#   AX_BOOST_SYSTEM\n#\n# DESCRIPTION\n#\n#   Test for System library from the Boost C++ libraries. The macro requires\n#   a preceding call to AX_BOOST_BASE. Further documentation is available at\n#   <http://randspringer.de/boost/index.html>.\n#\n#   This macro calls:\n#\n#     AC_SUBST(BOOST_SYSTEM_LIB)\n#\n#   And sets:\n#\n#     HAVE_BOOST_SYSTEM\n#\n# LICENSE\n#\n#   Copyright (c) 2008 Thomas Porschberg <thomas@randspringer.de>\n#   Copyright (c) 2008 Michael Tindal\n#   Copyright (c) 2008 Daniel Casimiro <dan.casimiro@gmail.com>\n#\n#   Copying and distribution of this file, with or without modification, are\n#   permitted in any medium without royalty provided the copyright notice\n#   and this notice are preserved. This file is offered as-is, without any\n#   warranty.\n\n#serial 17\n\nAC_DEFUN([AX_BOOST_SYSTEM],\n[\n\tAC_ARG_WITH([boost-system],\n\tAS_HELP_STRING([--with-boost-system@<:@=special-lib@:>@],\n                   [use the System library from boost - it is possible to specify a certain library for the linker\n                        e.g. --with-boost-system=boost_system-gcc-mt ]),\n        [\n        if test \"$withval\" = \"no\"; then\n\t\t\twant_boost=\"no\"\n        elif test \"$withval\" = \"yes\"; then\n            want_boost=\"yes\"\n            ax_boost_user_system_lib=\"\"\n        else\n\t\t    want_boost=\"yes\"\n\t\tax_boost_user_system_lib=\"$withval\"\n\t\tfi\n        ],\n        [want_boost=\"yes\"]\n\t)\n\n\tif test \"x$want_boost\" = \"xyes\"; then\n        AC_REQUIRE([AC_PROG_CC])\n        AC_REQUIRE([AC_CANONICAL_BUILD])\n\t\tCPPFLAGS_SAVED=\"$CPPFLAGS\"\n\t\tCPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n\t\texport CPPFLAGS\n\n\t\tLDFLAGS_SAVED=\"$LDFLAGS\"\n\t\tLDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n\t\texport LDFLAGS\n\n        AC_CACHE_CHECK(whether the Boost::System library is available,\n\t\t\t\t\t   ax_cv_boost_system,\n        [AC_LANG_PUSH([C++])\n\t\t\t CXXFLAGS_SAVE=$CXXFLAGS\n\n\t\t\t AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include <boost/system/error_code.hpp>]],\n                                   [[boost::system::system_category]])],\n                   ax_cv_boost_system=yes, ax_cv_boost_system=no)\n\t\t\t CXXFLAGS=$CXXFLAGS_SAVE\n             AC_LANG_POP([C++])\n\t\t])\n\t\tif test \"x$ax_cv_boost_system\" = \"xyes\"; then\n\t\t\tAC_SUBST(BOOST_CPPFLAGS)\n\n\t\t\tAC_DEFINE(HAVE_BOOST_SYSTEM,,[define if the Boost::System library is available])\n            BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\\/@:>@*//'`\n\n\t\t\tLDFLAGS_SAVE=$LDFLAGS\n            if test \"x$ax_boost_user_system_lib\" = \"x\"; then\n                for libextension in `ls -r $BOOSTLIBDIR/libboost_system* 2>/dev/null | sed 's,.*/lib,,' | sed 's,\\..*,,'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    AC_CHECK_LIB($ax_lib, exit,\n                                 [BOOST_SYSTEM_LIB=\"-l$ax_lib\"; AC_SUBST(BOOST_SYSTEM_LIB) link_system=\"yes\"; break],\n                                 [link_system=\"no\"])\n\t\t\t\tdone\n                if test \"x$link_system\" != \"xyes\"; then\n                for libextension in `ls -r $BOOSTLIBDIR/boost_system* 2>/dev/null | sed 's,.*/,,' | sed -e 's,\\..*,,'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    AC_CHECK_LIB($ax_lib, exit,\n                                 [BOOST_SYSTEM_LIB=\"-l$ax_lib\"; AC_SUBST(BOOST_SYSTEM_LIB) link_system=\"yes\"; break],\n                                 [link_system=\"no\"])\n\t\t\t\tdone\n                fi\n\n            else\n               for ax_lib in $ax_boost_user_system_lib boost_system-$ax_boost_user_system_lib; do\n\t\t\t\t      AC_CHECK_LIB($ax_lib, exit,\n                                   [BOOST_SYSTEM_LIB=\"-l$ax_lib\"; AC_SUBST(BOOST_SYSTEM_LIB) link_system=\"yes\"; break],\n                                   [link_system=\"no\"])\n                  done\n\n            fi\n            if test \"x$ax_lib\" = \"x\"; then\n                AC_MSG_ERROR(Could not find a version of the library!)\n            fi\n\t\t\tif test \"x$link_system\" = \"xno\"; then\n\t\t\t\tAC_MSG_ERROR(Could not link against $ax_lib !)\n\t\t\tfi\n\t\tfi\n\n\t\tCPPFLAGS=\"$CPPFLAGS_SAVED\"\n\tLDFLAGS=\"$LDFLAGS_SAVED\"\n\tfi\n])\n"
  },
  {
    "path": "m4/ax_boost_test_exec_monitor.m4",
    "content": "# ==============================================================================\n#  http://www.gnu.org/software/autoconf-archive/ax_boost_test_exec_monitor.html\n# ==============================================================================\n#\n# SYNOPSIS\n#\n#   AX_BOOST_TEST_EXEC_MONITOR\n#\n# DESCRIPTION\n#\n#   Test for Test_Exec_Monitor library from the Boost C++ libraries. The\n#   macro requires a preceding call to AX_BOOST_BASE. Further documentation\n#   is available at <http://randspringer.de/boost/index.html>.\n#\n#   This macro calls:\n#\n#     AC_SUBST(BOOST_TEST_EXEC_MONITOR_LIB)\n#\n#   And sets:\n#\n#     HAVE_BOOST_TEST_EXEC_MONITOR\n#\n# LICENSE\n#\n#   Copyright (c) 2008 Dodji Seketeli <dodji@seketeli.org>\n#   Copyright (c) 2008 Thomas Porschberg <thomas@randspringer.de>\n#\n#   Copying and distribution of this file, with or without modification, are\n#   permitted in any medium without royalty provided the copyright notice\n#   and this notice are preserved. This file is offered as-is, without any\n#   warranty.\n\n#serial 20\n\nAC_DEFUN([AX_BOOST_TEST_EXEC_MONITOR],\n[\n\tAC_ARG_WITH([boost-test-exec-monitor],\n\tAS_HELP_STRING([--with-boost-test-exec-monitor@<:@=special-lib@:>@],\n                   [use the Test_Exec_Monitor library from boost - it is possible to specify a certain library for the linker\n                        e.g. --with-boost-test-exec-monitor=boost_test_exec_monitor-gcc ]),\n        [\n        if test \"$withval\" = \"no\"; then\n\t\t\twant_boost=\"no\"\n        elif test \"$withval\" = \"yes\"; then\n            want_boost=\"yes\"\n            ax_boost_user_test_exec_monitor_lib=\"\"\n        else\n\t\t    want_boost=\"yes\"\n\t\tax_boost_user_test_exec_monitor_lib=\"$withval\"\n\t\tfi\n        ],\n        [want_boost=\"yes\"]\n\t)\n\n\tif test \"x$want_boost\" = \"xyes\"; then\n        AC_REQUIRE([AC_PROG_CC])\n\t\tCPPFLAGS_SAVED=\"$CPPFLAGS\"\n\t\tCPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n\t\texport CPPFLAGS\n\n\t\tLDFLAGS_SAVED=\"$LDFLAGS\"\n\t\tLDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n\t\texport LDFLAGS\n\n        AC_CACHE_CHECK(whether the Boost::Test_Exec_Monitor library is available,\n\t\t\t\t\t   ax_cv_boost_test_exec_monitor,\n        [AC_LANG_PUSH([C++])\n\t\t\t AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include <boost/test/test_tools.hpp>]],\n                                    [[int i=1 ; BOOST_REQUIRE(i==1); ; return 0;]])],\n                   ax_cv_boost_test_exec_monitor=yes, ax_cv_boost_test_exec_monitor=no)\n         AC_LANG_POP([C++])\n\t\t])\n\t\tif test \"x$ax_cv_boost_test_exec_monitor\" = \"xyes\"; then\n\t\t\tAC_DEFINE(HAVE_BOOST_TEST_EXEC_MONITOR,,[define if the Boost::Test_Exec_Monitor library is available])\n            BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\\/@:>@*//'`\n\n            if test \"x$ax_boost_user_test_exec_monitor_lib\" = \"x\"; then\n\t\t\tsaved_ldflags=\"${LDFLAGS}\"\n\n                for monitor_library in `ls $BOOSTLIBDIR/libboost_test_exec_monitor*.so* $BOOSTLIBDIR/libboost_test_exec_monitor*.dylib* $BOOSTLIBDIR/libboost_test_exec_monitor*.a* 2>/dev/null` ; do\n                    if test -r $monitor_library ; then\n                       libextension=`echo $monitor_library | sed 's,.*/,,' | sed -e 's;^lib\\(boost_test_exec_monitor.*\\)\\.so.*$;\\1;' -e 's;^lib\\(boost_test_exec_monitor.*\\)\\.dylib.*$;\\1;' -e 's;^lib\\(boost_test_exec_monitor.*\\)\\.a.*$;\\1;'`\n                       ax_lib=${libextension}\n                       link_test_exec_monitor=\"yes\"\n                    else\n                       link_test_exec_monitor=\"no\"\n                    fi\n\n\t\t\t      if test \"x$link_test_exec_monitor\" = \"xyes\"; then\n                      BOOST_TEST_EXEC_MONITOR_LIB=\"-l$ax_lib\"\n                      AC_SUBST(BOOST_TEST_EXEC_MONITOR_LIB)\n\t\t\t\t\t  break\n\t\t\t\t  fi\n                done\n                if test \"x$link_test_exec_monitor\" != \"xyes\"; then\n                for libextension in `ls $BOOSTLIBDIR/boost_test_exec_monitor*.dll* $BOOSTLIBDIR/boost_test_exec_monitor*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\\(boost_test_exec_monitor.*\\)\\.dll.*$;\\1;' -e 's;^\\(boost_test_exec_monitor.*\\)\\.a.*$;\\1;'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    AC_CHECK_LIB($ax_lib, exit,\n                                 [BOOST_TEST_EXEC_MONITOR_LIB=\"-l$ax_lib\"; AC_SUBST(BOOST_TEST_EXEC_MONITOR_LIB) link_test_exec_monitor=\"yes\"; break],\n                                 [link_test_exec_monitor=\"no\"])\n\t\t\t\tdone\n                fi\n\n            else\n                link_test_exec_monitor=\"no\"\n\t\t\tsaved_ldflags=\"${LDFLAGS}\"\n                for ax_lib in boost_test_exec_monitor-$ax_boost_user_test_exec_monitor_lib $ax_boost_user_test_exec_monitor_lib ; do\n                   if test \"x$link_test_exec_monitor\" = \"xyes\"; then\n                      break;\n                   fi\n                   for monitor_library in `ls $BOOSTLIBDIR/lib${ax_lib}.so* $BOOSTLIBDIR/lib${ax_lib}.a* 2>/dev/null` ; do\n                   if test -r $monitor_library ; then\n                       libextension=`echo $monitor_library | sed 's,.*/,,' | sed -e 's;^lib\\(boost_test_exec_monitor.*\\)\\.so.*$;\\1;' -e 's;^lib\\(boost_test_exec_monitor.*\\)\\.a*$;\\1;'`\n                       ax_lib=${libextension}\n                       link_test_exec_monitor=\"yes\"\n                    else\n                       link_test_exec_monitor=\"no\"\n                    fi\n\n\t\t\t\tif test \"x$link_test_exec_monitor\" = \"xyes\"; then\n                        BOOST_TEST_EXEC_MONITOR_LIB=\"-l$ax_lib\"\n                        AC_SUBST(BOOST_TEST_EXEC_MONITOR_LIB)\n\t\t\t\t\t    break\n\t\t\t\t    fi\n                  done\n               done\n            fi\n            if test \"x$ax_lib\" = \"x\"; then\n                AC_MSG_ERROR(Could not find a version of the library!)\n            fi\n\t\t\tif test \"x$link_test_exec_monitor\" != \"xyes\"; then\n\t\t\t\tAC_MSG_ERROR(Could not link against $ax_lib !)\n\t\t\tfi\n\t\tfi\n\n\t\tCPPFLAGS=\"$CPPFLAGS_SAVED\"\n\tLDFLAGS=\"$LDFLAGS_SAVED\"\n\tfi\n])\n"
  },
  {
    "path": "m4/ax_boost_unit_test_framework.m4",
    "content": "# ================================================================================\n#  http://www.gnu.org/software/autoconf-archive/ax_boost_unit_test_framework.html\n# ================================================================================\n#\n# SYNOPSIS\n#\n#   AX_BOOST_UNIT_TEST_FRAMEWORK\n#\n# DESCRIPTION\n#\n#   Test for Unit_Test_Framework library from the Boost C++ libraries. The\n#   macro requires a preceding call to AX_BOOST_BASE. Further documentation\n#   is available at <http://randspringer.de/boost/index.html>.\n#\n#   This macro calls:\n#\n#     AC_SUBST(BOOST_UNIT_TEST_FRAMEWORK_LIB)\n#\n#   And sets:\n#\n#     HAVE_BOOST_UNIT_TEST_FRAMEWORK\n#\n# LICENSE\n#\n#   Copyright (c) 2008 Thomas Porschberg <thomas@randspringer.de>\n#\n#   Copying and distribution of this file, with or without modification, are\n#   permitted in any medium without royalty provided the copyright notice\n#   and this notice are preserved. This file is offered as-is, without any\n#   warranty.\n\n#serial 19\n\nAC_DEFUN([AX_BOOST_UNIT_TEST_FRAMEWORK],\n[\n\tAC_ARG_WITH([boost-unit-test-framework],\n\tAS_HELP_STRING([--with-boost-unit-test-framework@<:@=special-lib@:>@],\n                   [use the Unit_Test_Framework library from boost - it is possible to specify a certain library for the linker\n                        e.g. --with-boost-unit-test-framework=boost_unit_test_framework-gcc ]),\n        [\n        if test \"$withval\" = \"no\"; then\n\t\t\twant_boost=\"no\"\n        elif test \"$withval\" = \"yes\"; then\n            want_boost=\"yes\"\n            ax_boost_user_unit_test_framework_lib=\"\"\n        else\n\t\t    want_boost=\"yes\"\n\t\tax_boost_user_unit_test_framework_lib=\"$withval\"\n\t\tfi\n        ],\n        [want_boost=\"yes\"]\n\t)\n\n\tif test \"x$want_boost\" = \"xyes\"; then\n        AC_REQUIRE([AC_PROG_CC])\n\t\tCPPFLAGS_SAVED=\"$CPPFLAGS\"\n\t\tCPPFLAGS=\"$CPPFLAGS $BOOST_CPPFLAGS\"\n\t\texport CPPFLAGS\n\n\t\tLDFLAGS_SAVED=\"$LDFLAGS\"\n\t\tLDFLAGS=\"$LDFLAGS $BOOST_LDFLAGS\"\n\t\texport LDFLAGS\n\n        AC_CACHE_CHECK(whether the Boost::Unit_Test_Framework library is available,\n\t\t\t\t\t   ax_cv_boost_unit_test_framework,\n        [AC_LANG_PUSH([C++])\n\t\t\t AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include <boost/test/unit_test.hpp>]],\n                                    [[using boost::unit_test::test_suite;\n\t\t\t\t\t\t\t test_suite* test= BOOST_TEST_SUITE( \"Unit test example 1\" ); return 0;]])],\n                   ax_cv_boost_unit_test_framework=yes, ax_cv_boost_unit_test_framework=no)\n         AC_LANG_POP([C++])\n\t\t])\n\t\tif test \"x$ax_cv_boost_unit_test_framework\" = \"xyes\"; then\n\t\t\tAC_DEFINE(HAVE_BOOST_UNIT_TEST_FRAMEWORK,,[define if the Boost::Unit_Test_Framework library is available])\n            BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\\/@:>@*//'`\n\n            if test \"x$ax_boost_user_unit_test_framework_lib\" = \"x\"; then\n\t\t\tsaved_ldflags=\"${LDFLAGS}\"\n                for monitor_library in `ls $BOOSTLIBDIR/libboost_unit_test_framework*.so* $BOOSTLIBDIR/libboost_unit_test_framework*.dylib* $BOOSTLIBDIR/libboost_unit_test_framework*.a* 2>/dev/null` ; do\n                    if test -r $monitor_library ; then\n                       libextension=`echo $monitor_library | sed 's,.*/,,' | sed -e 's;^lib\\(boost_unit_test_framework.*\\)\\.so.*$;\\1;' -e 's;^lib\\(boost_unit_test_framework.*\\)\\.dylib.*$;\\1;' -e 's;^lib\\(boost_unit_test_framework.*\\)\\.a.*$;\\1;'`\n                       ax_lib=${libextension}\n                       link_unit_test_framework=\"yes\"\n                    else\n                       link_unit_test_framework=\"no\"\n                    fi\n\n\t\t\t    if test \"x$link_unit_test_framework\" = \"xyes\"; then\n                      BOOST_UNIT_TEST_FRAMEWORK_LIB=\"-l$ax_lib\"\n                      AC_SUBST(BOOST_UNIT_TEST_FRAMEWORK_LIB)\n\t\t\t\t\t  break\n\t\t\t\t    fi\n                done\n                if test \"x$link_unit_test_framework\" != \"xyes\"; then\n                for libextension in `ls $BOOSTLIBDIR/boost_unit_test_framework*.dll* $BOOSTLIBDIR/boost_unit_test_framework*.a* 2>/dev/null  | sed 's,.*/,,' | sed -e 's;^\\(boost_unit_test_framework.*\\)\\.dll.*$;\\1;' -e 's;^\\(boost_unit_test_framework.*\\)\\.a.*$;\\1;'` ; do\n                     ax_lib=${libextension}\n\t\t\t\t    AC_CHECK_LIB($ax_lib, exit,\n                                 [BOOST_UNIT_TEST_FRAMEWORK_LIB=\"-l$ax_lib\"; AC_SUBST(BOOST_UNIT_TEST_FRAMEWORK_LIB) link_unit_test_framework=\"yes\"; break],\n                                 [link_unit_test_framework=\"no\"])\n\t\t\t\tdone\n                fi\n            else\n                link_unit_test_framework=\"no\"\n\t\t\tsaved_ldflags=\"${LDFLAGS}\"\n                for ax_lib in boost_unit_test_framework-$ax_boost_user_unit_test_framework_lib $ax_boost_user_unit_test_framework_lib ; do\n                   if test \"x$link_unit_test_framework\" = \"xyes\"; then\n                      break;\n                   fi\n                   for unittest_library in `ls $BOOSTLIBDIR/lib${ax_lib}.so* $BOOSTLIBDIR/lib${ax_lib}.a* 2>/dev/null` ; do\n                   if test -r $unittest_library ; then\n                       libextension=`echo $unittest_library | sed 's,.*/,,' | sed -e 's;^lib\\(boost_unit_test_framework.*\\)\\.so.*$;\\1;' -e 's;^lib\\(boost_unit_test_framework.*\\)\\.a*$;\\1;'`\n                       ax_lib=${libextension}\n                       link_unit_test_framework=\"yes\"\n                    else\n                       link_unit_test_framework=\"no\"\n                    fi\n\n\t\t\t\tif test \"x$link_unit_test_framework\" = \"xyes\"; then\n                        BOOST_UNIT_TEST_FRAMEWORK_LIB=\"-l$ax_lib\"\n                        AC_SUBST(BOOST_UNIT_TEST_FRAMEWORK_LIB)\n\t\t\t\t\t    break\n\t\t\t\t    fi\n                  done\n               done\n            fi\n            if test \"x$ax_lib\" = \"x\"; then\n                AC_MSG_ERROR(Could not find a version of the library!)\n            fi\n\t\t\tif test \"x$link_unit_test_framework\" != \"xyes\"; then\n\t\t\t\tAC_MSG_ERROR(Could not link against $ax_lib !)\n\t\t\tfi\n\t\tfi\n\n\t\tCPPFLAGS=\"$CPPFLAGS_SAVED\"\n\tLDFLAGS=\"$LDFLAGS_SAVED\"\n\tfi\n])\n"
  },
  {
    "path": "m4/ax_check_compile_flag.m4",
    "content": "# ===========================================================================\n#  https://www.gnu.org/software/autoconf-archive/ax_check_compile_flag.html\n# ===========================================================================\n#\n# SYNOPSIS\n#\n#   AX_CHECK_COMPILE_FLAG(FLAG, [ACTION-SUCCESS], [ACTION-FAILURE], [EXTRA-FLAGS], [INPUT])\n#\n# DESCRIPTION\n#\n#   Check whether the given FLAG works with the current language's compiler\n#   or gives an error.  (Warnings, however, are ignored)\n#\n#   ACTION-SUCCESS/ACTION-FAILURE are shell commands to execute on\n#   success/failure.\n#\n#   If EXTRA-FLAGS is defined, it is added to the current language's default\n#   flags (e.g. CFLAGS) when the check is done.  The check is thus made with\n#   the flags: \"CFLAGS EXTRA-FLAGS FLAG\".  This can for example be used to\n#   force the compiler to issue an error when a bad flag is given.\n#\n#   INPUT gives an alternative input source to AC_COMPILE_IFELSE.\n#\n#   NOTE: Implementation based on AX_CFLAGS_GCC_OPTION. Please keep this\n#   macro in sync with AX_CHECK_{PREPROC,LINK}_FLAG.\n#\n# LICENSE\n#\n#   Copyright (c) 2008 Guido U. Draheim <guidod@gmx.de>\n#   Copyright (c) 2011 Maarten Bosmans <mkbosmans@gmail.com>\n#\n#   This program is free software: you can redistribute it and/or modify it\n#   under the terms of the GNU General Public License as published by the\n#   Free Software Foundation, either version 3 of the License, or (at your\n#   option) any later version.\n#\n#   This program is distributed in the hope that it will be useful, but\n#   WITHOUT ANY WARRANTY; without even the implied warranty of\n#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General\n#   Public License for more details.\n#\n#   You should have received a copy of the GNU General Public License along\n#   with this program. If not, see <https://www.gnu.org/licenses/>.\n#\n#   As a special exception, the respective Autoconf Macro's copyright owner\n#   gives unlimited permission to copy, distribute and modify the configure\n#   scripts that are the output of Autoconf when processing the Macro. You\n#   need not follow the terms of the GNU General Public License when using\n#   or distributing such scripts, even though portions of the text of the\n#   Macro appear in them. The GNU General Public License (GPL) does govern\n#   all other use of the material that constitutes the Autoconf Macro.\n#\n#   This special exception to the GPL applies to versions of the Autoconf\n#   Macro released by the Autoconf Archive. When you make and distribute a\n#   modified version of the Autoconf Macro, you may extend this special\n#   exception to the GPL to apply to your modified version as well.\n\n#serial 5\n\nAC_DEFUN([AX_CHECK_COMPILE_FLAG],\n[AC_PREREQ(2.64)dnl for _AC_LANG_PREFIX and AS_VAR_IF\nAS_VAR_PUSHDEF([CACHEVAR],[ax_cv_check_[]_AC_LANG_ABBREV[]flags_$4_$1])dnl\nAC_CACHE_CHECK([whether _AC_LANG compiler accepts $1], CACHEVAR, [\n  ax_check_save_flags=$[]_AC_LANG_PREFIX[]FLAGS\n  _AC_LANG_PREFIX[]FLAGS=\"$[]_AC_LANG_PREFIX[]FLAGS $4 $1\"\n  AC_COMPILE_IFELSE([m4_default([$5],[AC_LANG_PROGRAM()])],\n    [AS_VAR_SET(CACHEVAR,[yes])],\n    [AS_VAR_SET(CACHEVAR,[no])])\n  _AC_LANG_PREFIX[]FLAGS=$ax_check_save_flags])\nAS_VAR_IF(CACHEVAR,yes,\n  [m4_default([$2], :)],\n  [m4_default([$3], :)])\nAS_VAR_POPDEF([CACHEVAR])dnl\n])dnl AX_CHECK_COMPILE_FLAGS\n"
  },
  {
    "path": "m4/ax_check_openssl.m4",
    "content": "# ===========================================================================\n#     https://www.gnu.org/software/autoconf-archive/ax_check_openssl.html\n# ===========================================================================\n#\n# SYNOPSIS\n#\n#   AX_CHECK_OPENSSL([action-if-found[, action-if-not-found]])\n#\n# DESCRIPTION\n#\n#   Look for OpenSSL in a number of default spots, or in a user-selected\n#   spot (via --with-openssl).  Sets\n#\n#     OPENSSL_INCLUDES to the include directives required\n#     OPENSSL_LIBS to the -l directives required\n#     OPENSSL_LDFLAGS to the -L or -R flags required\n#\n#   and calls ACTION-IF-FOUND or ACTION-IF-NOT-FOUND appropriately\n#\n#   This macro sets OPENSSL_INCLUDES such that source files should use the\n#   openssl/ directory in include directives:\n#\n#     #include <openssl/hmac.h>\n#\n# LICENSE\n#\n#   Copyright (c) 2009,2010 Zmanda Inc. <http://www.zmanda.com/>\n#   Copyright (c) 2009,2010 Dustin J. Mitchell <dustin@zmanda.com>\n#\n#   Copying and distribution of this file, with or without modification, are\n#   permitted in any medium without royalty provided the copyright notice\n#   and this notice are preserved. This file is offered as-is, without any\n#   warranty.\n\n#serial 10\n\nAU_ALIAS([CHECK_SSL], [AX_CHECK_OPENSSL])\nAC_DEFUN([AX_CHECK_OPENSSL], [\n    found=false\n    AC_ARG_WITH([openssl],\n        [AS_HELP_STRING([--with-openssl=DIR],\n            [root of the OpenSSL directory])],\n        [\n            case \"$withval\" in\n            \"\" | y | ye | yes | n | no)\n            AC_MSG_ERROR([Invalid --with-openssl value])\n              ;;\n            *) ssldirs=\"$withval\"\n              ;;\n            esac\n        ], [\n            ## # if pkg-config is installed and openssl has installed a .pc file,\n            ## # then use that information and don't search ssldirs\n            ## AC_CHECK_TOOL([PKG_CONFIG], [pkg-config])\n            ## if test x\"$PKG_CONFIG\" != x\"\"; then\n            ##     OPENSSL_LDFLAGS=`$PKG_CONFIG openssl --libs-only-L 2>/dev/null`\n            ##     if test $? = 0; then\n            ##         OPENSSL_LIBS=`$PKG_CONFIG openssl --libs-only-l 2>/dev/null`\n            ##         OPENSSL_INCLUDES=`$PKG_CONFIG openssl --cflags-only-I 2>/dev/null`\n            ##         found=true\n            ##     fi\n            ## fi\n\n            # no such luck; use some default ssldirs\n            if ! $found; then\n                ssldirs=\"/usr/local/ssl /usr/lib/ssl /usr/ssl /usr/pkg /usr/local /usr /usr/local/Cellar/openssl/*\"\n            fi\n        ]\n        )\n\n\n    # note that we #include <openssl/foo.h>, so the OpenSSL headers have to be in\n    # an 'openssl' subdirectory\n\n    if ! $found; then\n        OPENSSL_INCLUDES=\n        for ssldir in $ssldirs; do\n            AC_MSG_CHECKING([for openssl/ssl.h in $ssldir])\n            if test -f \"$ssldir/include/openssl/ssl.h\"; then\n                OPENSSL_INCLUDES=\"-I$ssldir/include\"\n                OPENSSL_LDFLAGS=\"-L$ssldir/lib\"\n                OPENSSL_LIBS=\"-lssl -lcrypto\"\n                found=true\n                AC_MSG_RESULT([yes])\n                break\n            else\n                AC_MSG_RESULT([no])\n            fi\n        done\n\n        # if the file wasn't found, well, go ahead and try the link anyway -- maybe\n        # it will just work!\n    fi\n\n    # try the preprocessor and linker with our new flags,\n    # being careful not to pollute the global LIBS, LDFLAGS, and CPPFLAGS\n\n    AC_MSG_CHECKING([whether compiling and linking against OpenSSL works])\n    echo \"Trying link with OPENSSL_LDFLAGS=$OPENSSL_LDFLAGS;\" \\\n        \"OPENSSL_LIBS=$OPENSSL_LIBS; OPENSSL_INCLUDES=$OPENSSL_INCLUDES\" >&AS_MESSAGE_LOG_FD\n\n    save_LIBS=\"$LIBS\"\n    save_LDFLAGS=\"$LDFLAGS\"\n    save_CPPFLAGS=\"$CPPFLAGS\"\n    LDFLAGS=\"$LDFLAGS $OPENSSL_LDFLAGS\"\n    LIBS=\"$OPENSSL_LIBS $LIBS\"\n    CPPFLAGS=\"$OPENSSL_INCLUDES $CPPFLAGS\"\n    AC_LINK_IFELSE(\n        [AC_LANG_PROGRAM([#include <openssl/ssl.h>], [SSL_new(NULL)])],\n        [\n            AC_MSG_RESULT([yes])\n            $1\n        ], [\n            AC_MSG_RESULT([no])\n            $2\n        ])\n    CPPFLAGS=\"$save_CPPFLAGS\"\n    LDFLAGS=\"$save_LDFLAGS\"\n    LIBS=\"$save_LIBS\"\n\n    AC_SUBST([OPENSSL_INCLUDES])\n    AC_SUBST([OPENSSL_LIBS])\n    AC_SUBST([OPENSSL_LDFLAGS])\n])\n"
  },
  {
    "path": "m4/ax_cxx_compile_stdcxx.m4",
    "content": "# ===========================================================================\n#  https://www.gnu.org/software/autoconf-archive/ax_cxx_compile_stdcxx.html\n# ===========================================================================\n#\n# SYNOPSIS\n#\n#   AX_CXX_COMPILE_STDCXX(VERSION, [ext|noext], [mandatory|optional])\n#\n# DESCRIPTION\n#\n#   Check for baseline language coverage in the compiler for the specified\n#   version of the C++ standard.  If necessary, add switches to CXX and\n#   CXXCPP to enable support.  VERSION may be '11' (for the C++11 standard)\n#   or '14' (for the C++14 standard).\n#\n#   The second argument, if specified, indicates whether you insist on an\n#   extended mode (e.g. -std=gnu++11) or a strict conformance mode (e.g.\n#   -std=c++11).  If neither is specified, you get whatever works, with\n#   preference for an extended mode.\n#\n#   The third argument, if specified 'mandatory' or if left unspecified,\n#   indicates that baseline support for the specified C++ standard is\n#   required and that the macro should error out if no mode with that\n#   support is found.  If specified 'optional', then configuration proceeds\n#   regardless, after defining HAVE_CXX${VERSION} if and only if a\n#   supporting mode is found.\n#\n# LICENSE\n#\n#   Copyright (c) 2008 Benjamin Kosnik <bkoz@redhat.com>\n#   Copyright (c) 2012 Zack Weinberg <zackw@panix.com>\n#   Copyright (c) 2013 Roy Stogner <roystgnr@ices.utexas.edu>\n#   Copyright (c) 2014, 2015 Google Inc.; contributed by Alexey Sokolov <sokolov@google.com>\n#   Copyright (c) 2015 Paul Norman <penorman@mac.com>\n#   Copyright (c) 2015 Moritz Klammler <moritz@klammler.eu>\n#   Copyright (c) 2016 Krzesimir Nowak <qdlacz@gmail.com>\n#\n#   Copying and distribution of this file, with or without modification, are\n#   permitted in any medium without royalty provided the copyright notice\n#   and this notice are preserved.  This file is offered as-is, without any\n#   warranty.\n\n#serial 7\n\ndnl  This macro is based on the code from the AX_CXX_COMPILE_STDCXX_11 macro\ndnl  (serial version number 13).\n\nAX_REQUIRE_DEFINED([AC_MSG_WARN])\nAC_DEFUN([AX_CXX_COMPILE_STDCXX], [dnl\n  m4_if([$1], [11], [ax_cxx_compile_alternatives=\"11 0x\"],\n        [$1], [14], [ax_cxx_compile_alternatives=\"14 1y\"],\n        [$1], [17], [ax_cxx_compile_alternatives=\"17 1z\"],\n        [m4_fatal([invalid first argument `$1' to AX_CXX_COMPILE_STDCXX])])dnl\n  m4_if([$2], [], [],\n        [$2], [ext], [],\n        [$2], [noext], [],\n        [m4_fatal([invalid second argument `$2' to AX_CXX_COMPILE_STDCXX])])dnl\n  m4_if([$3], [], [ax_cxx_compile_cxx$1_required=true],\n        [$3], [mandatory], [ax_cxx_compile_cxx$1_required=true],\n        [$3], [optional], [ax_cxx_compile_cxx$1_required=false],\n        [m4_fatal([invalid third argument `$3' to AX_CXX_COMPILE_STDCXX])])\n  AC_LANG_PUSH([C++])dnl\n  ac_success=no\n  AC_CACHE_CHECK(whether $CXX supports C++$1 features by default,\n  ax_cv_cxx_compile_cxx$1,\n  [AC_COMPILE_IFELSE([AC_LANG_SOURCE([_AX_CXX_COMPILE_STDCXX_testbody_$1])],\n    [ax_cv_cxx_compile_cxx$1=yes],\n    [ax_cv_cxx_compile_cxx$1=no])])\n  if test x$ax_cv_cxx_compile_cxx$1 = xyes; then\n    ac_success=yes\n  fi\n\n  m4_if([$2], [noext], [], [dnl\n  if test x$ac_success = xno; then\n    for alternative in ${ax_cxx_compile_alternatives}; do\n      switch=\"-std=gnu++${alternative}\"\n      cachevar=AS_TR_SH([ax_cv_cxx_compile_cxx$1_$switch])\n      AC_CACHE_CHECK(whether $CXX supports C++$1 features with $switch,\n                     $cachevar,\n        [ac_save_CXX=\"$CXX\"\n         CXX=\"$CXX $switch\"\n         AC_COMPILE_IFELSE([AC_LANG_SOURCE([_AX_CXX_COMPILE_STDCXX_testbody_$1])],\n          [eval $cachevar=yes],\n          [eval $cachevar=no])\n         CXX=\"$ac_save_CXX\"])\n      if eval test x\\$$cachevar = xyes; then\n        CXX=\"$CXX $switch\"\n        if test -n \"$CXXCPP\" ; then\n          CXXCPP=\"$CXXCPP $switch\"\n        fi\n        ac_success=yes\n        break\n      fi\n    done\n  fi])\n\n  m4_if([$2], [ext], [], [dnl\n  if test x$ac_success = xno; then\n    dnl HP's aCC needs +std=c++11 according to:\n    dnl http://h21007.www2.hp.com/portal/download/files/unprot/aCxx/PDF_Release_Notes/769149-001.pdf\n    dnl Cray's crayCC needs \"-h std=c++11\"\n    for alternative in ${ax_cxx_compile_alternatives}; do\n      for switch in -std=c++${alternative} +std=c++${alternative} \"-h std=c++${alternative}\"; do\n        cachevar=AS_TR_SH([ax_cv_cxx_compile_cxx$1_$switch])\n        AC_CACHE_CHECK(whether $CXX supports C++$1 features with $switch,\n                       $cachevar,\n          [ac_save_CXX=\"$CXX\"\n           CXX=\"$CXX $switch\"\n           AC_COMPILE_IFELSE([AC_LANG_SOURCE([_AX_CXX_COMPILE_STDCXX_testbody_$1])],\n            [eval $cachevar=yes],\n            [eval $cachevar=no])\n           CXX=\"$ac_save_CXX\"])\n        if eval test x\\$$cachevar = xyes; then\n          CXX=\"$CXX $switch\"\n          if test -n \"$CXXCPP\" ; then\n            CXXCPP=\"$CXXCPP $switch\"\n          fi\n          ac_success=yes\n          break\n        fi\n      done\n      if test x$ac_success = xyes; then\n        break\n      fi\n    done\n  fi])\n  AC_LANG_POP([C++])\n  if test x$ax_cxx_compile_cxx$1_required = xtrue; then\n    if test x$ac_success = xno; then\n      AC_MSG_ERROR([*** A compiler with support for C++$1 language features is required.])\n    fi\n  fi\n  if test x$ac_success = xno; then\n    HAVE_CXX$1=0\n    AC_MSG_NOTICE([No compiler with C++$1 support was found])\n  else\n    HAVE_CXX$1=1\n    AC_DEFINE(HAVE_CXX$1,1,\n              [define if the compiler supports basic C++$1 syntax])\n  fi\n  AC_SUBST(HAVE_CXX$1)\n  m4_if([$1], [17], [AC_MSG_WARN([C++17 is not yet standardized, so the checks may change in incompatible ways anytime])])\n])\n\n\ndnl  Test body for checking C++11 support\n\nm4_define([_AX_CXX_COMPILE_STDCXX_testbody_11],\n  _AX_CXX_COMPILE_STDCXX_testbody_new_in_11\n)\n\n\ndnl  Test body for checking C++14 support\n\nm4_define([_AX_CXX_COMPILE_STDCXX_testbody_14],\n  _AX_CXX_COMPILE_STDCXX_testbody_new_in_11\n  _AX_CXX_COMPILE_STDCXX_testbody_new_in_14\n)\n\nm4_define([_AX_CXX_COMPILE_STDCXX_testbody_17],\n  _AX_CXX_COMPILE_STDCXX_testbody_new_in_11\n  _AX_CXX_COMPILE_STDCXX_testbody_new_in_14\n  _AX_CXX_COMPILE_STDCXX_testbody_new_in_17\n)\n\ndnl  Tests for new features in C++11\n\nm4_define([_AX_CXX_COMPILE_STDCXX_testbody_new_in_11], [[\n\n// If the compiler admits that it is not ready for C++11, why torture it?\n// Hopefully, this will speed up the test.\n\n#ifndef __cplusplus\n\n#error \"This is not a C++ compiler\"\n\n#elif __cplusplus < 201103L\n\n#error \"This is not a C++11 compiler\"\n\n#else\n\nnamespace cxx11\n{\n\n  namespace test_static_assert\n  {\n\n    template <typename T>\n    struct check\n    {\n      static_assert(sizeof(int) <= sizeof(T), \"not big enough\");\n    };\n\n  }\n\n  namespace test_final_override\n  {\n\n    struct Base\n    {\n      virtual void f() {}\n    };\n\n    struct Derived : public Base\n    {\n      virtual void f() override {}\n    };\n\n  }\n\n  namespace test_double_right_angle_brackets\n  {\n\n    template < typename T >\n    struct check {};\n\n    typedef check<void> single_type;\n    typedef check<check<void>> double_type;\n    typedef check<check<check<void>>> triple_type;\n    typedef check<check<check<check<void>>>> quadruple_type;\n\n  }\n\n  namespace test_decltype\n  {\n\n    int\n    f()\n    {\n      int a = 1;\n      decltype(a) b = 2;\n      return a + b;\n    }\n\n  }\n\n  namespace test_type_deduction\n  {\n\n    template < typename T1, typename T2 >\n    struct is_same\n    {\n      static const bool value = false;\n    };\n\n    template < typename T >\n    struct is_same<T, T>\n    {\n      static const bool value = true;\n    };\n\n    template < typename T1, typename T2 >\n    auto\n    add(T1 a1, T2 a2) -> decltype(a1 + a2)\n    {\n      return a1 + a2;\n    }\n\n    int\n    test(const int c, volatile int v)\n    {\n      static_assert(is_same<int, decltype(0)>::value == true, \"\");\n      static_assert(is_same<int, decltype(c)>::value == false, \"\");\n      static_assert(is_same<int, decltype(v)>::value == false, \"\");\n      auto ac = c;\n      auto av = v;\n      auto sumi = ac + av + 'x';\n      auto sumf = ac + av + 1.0;\n      static_assert(is_same<int, decltype(ac)>::value == true, \"\");\n      static_assert(is_same<int, decltype(av)>::value == true, \"\");\n      static_assert(is_same<int, decltype(sumi)>::value == true, \"\");\n      static_assert(is_same<int, decltype(sumf)>::value == false, \"\");\n      static_assert(is_same<int, decltype(add(c, v))>::value == true, \"\");\n      return (sumf > 0.0) ? sumi : add(c, v);\n    }\n\n  }\n\n  namespace test_noexcept\n  {\n\n    int f() { return 0; }\n    int g() noexcept { return 0; }\n\n    static_assert(noexcept(f()) == false, \"\");\n    static_assert(noexcept(g()) == true, \"\");\n\n  }\n\n  namespace test_constexpr\n  {\n\n    template < typename CharT >\n    unsigned long constexpr\n    strlen_c_r(const CharT *const s, const unsigned long acc) noexcept\n    {\n      return *s ? strlen_c_r(s + 1, acc + 1) : acc;\n    }\n\n    template < typename CharT >\n    unsigned long constexpr\n    strlen_c(const CharT *const s) noexcept\n    {\n      return strlen_c_r(s, 0UL);\n    }\n\n    static_assert(strlen_c(\"\") == 0UL, \"\");\n    static_assert(strlen_c(\"1\") == 1UL, \"\");\n    static_assert(strlen_c(\"example\") == 7UL, \"\");\n    static_assert(strlen_c(\"another\\0example\") == 7UL, \"\");\n\n  }\n\n  namespace test_rvalue_references\n  {\n\n    template < int N >\n    struct answer\n    {\n      static constexpr int value = N;\n    };\n\n    answer<1> f(int&)       { return answer<1>(); }\n    answer<2> f(const int&) { return answer<2>(); }\n    answer<3> f(int&&)      { return answer<3>(); }\n\n    void\n    test()\n    {\n      int i = 0;\n      const int c = 0;\n      static_assert(decltype(f(i))::value == 1, \"\");\n      static_assert(decltype(f(c))::value == 2, \"\");\n      static_assert(decltype(f(0))::value == 3, \"\");\n    }\n\n  }\n\n  namespace test_uniform_initialization\n  {\n\n    struct test\n    {\n      static const int zero {};\n      static const int one {1};\n    };\n\n    static_assert(test::zero == 0, \"\");\n    static_assert(test::one == 1, \"\");\n\n  }\n\n  namespace test_lambdas\n  {\n\n    void\n    test1()\n    {\n      auto lambda1 = [](){};\n      auto lambda2 = lambda1;\n      lambda1();\n      lambda2();\n    }\n\n    int\n    test2()\n    {\n      auto a = [](int i, int j){ return i + j; }(1, 2);\n      auto b = []() -> int { return '0'; }();\n      auto c = [=](){ return a + b; }();\n      auto d = [&](){ return c; }();\n      auto e = [a, &b](int x) mutable {\n        const auto identity = [](int y){ return y; };\n        for (auto i = 0; i < a; ++i)\n          a += b--;\n        return x + identity(a + b);\n      }(0);\n      return a + b + c + d + e;\n    }\n\n    int\n    test3()\n    {\n      const auto nullary = [](){ return 0; };\n      const auto unary = [](int x){ return x; };\n      using nullary_t = decltype(nullary);\n      using unary_t = decltype(unary);\n      const auto higher1st = [](nullary_t f){ return f(); };\n      const auto higher2nd = [unary](nullary_t f1){\n        return [unary, f1](unary_t f2){ return f2(unary(f1())); };\n      };\n      return higher1st(nullary) + higher2nd(nullary)(unary);\n    }\n\n  }\n\n  namespace test_variadic_templates\n  {\n\n    template <int...>\n    struct sum;\n\n    template <int N0, int... N1toN>\n    struct sum<N0, N1toN...>\n    {\n      static constexpr auto value = N0 + sum<N1toN...>::value;\n    };\n\n    template <>\n    struct sum<>\n    {\n      static constexpr auto value = 0;\n    };\n\n    static_assert(sum<>::value == 0, \"\");\n    static_assert(sum<1>::value == 1, \"\");\n    static_assert(sum<23>::value == 23, \"\");\n    static_assert(sum<1, 2>::value == 3, \"\");\n    static_assert(sum<5, 5, 11>::value == 21, \"\");\n    static_assert(sum<2, 3, 5, 7, 11, 13>::value == 41, \"\");\n\n  }\n\n  // http://stackoverflow.com/questions/13728184/template-aliases-and-sfinae\n  // Clang 3.1 fails with headers of libstd++ 4.8.3 when using std::function\n  // because of this.\n  namespace test_template_alias_sfinae\n  {\n\n    struct foo {};\n\n    template<typename T>\n    using member = typename T::member_type;\n\n    template<typename T>\n    void func(...) {}\n\n    template<typename T>\n    void func(member<T>*) {}\n\n    void test();\n\n    void test() { func<foo>(0); }\n\n  }\n\n}  // namespace cxx11\n\n#endif  // __cplusplus >= 201103L\n\n]])\n\n\ndnl  Tests for new features in C++14\n\nm4_define([_AX_CXX_COMPILE_STDCXX_testbody_new_in_14], [[\n\n// If the compiler admits that it is not ready for C++14, why torture it?\n// Hopefully, this will speed up the test.\n\n#ifndef __cplusplus\n\n#error \"This is not a C++ compiler\"\n\n#elif __cplusplus < 201402L\n\n#error \"This is not a C++14 compiler\"\n\n#else\n\nnamespace cxx14\n{\n\n  namespace test_polymorphic_lambdas\n  {\n\n    int\n    test()\n    {\n      const auto lambda = [](auto&&... args){\n        const auto istiny = [](auto x){\n          return (sizeof(x) == 1UL) ? 1 : 0;\n        };\n        const int aretiny[] = { istiny(args)... };\n        return aretiny[0];\n      };\n      return lambda(1, 1L, 1.0f, '1');\n    }\n\n  }\n\n  namespace test_binary_literals\n  {\n\n    constexpr auto ivii = 0b0000000000101010;\n    static_assert(ivii == 42, \"wrong value\");\n\n  }\n\n  namespace test_generalized_constexpr\n  {\n\n    template < typename CharT >\n    constexpr unsigned long\n    strlen_c(const CharT *const s) noexcept\n    {\n      auto length = 0UL;\n      for (auto p = s; *p; ++p)\n        ++length;\n      return length;\n    }\n\n    static_assert(strlen_c(\"\") == 0UL, \"\");\n    static_assert(strlen_c(\"x\") == 1UL, \"\");\n    static_assert(strlen_c(\"test\") == 4UL, \"\");\n    static_assert(strlen_c(\"another\\0test\") == 7UL, \"\");\n\n  }\n\n  namespace test_lambda_init_capture\n  {\n\n    int\n    test()\n    {\n      auto x = 0;\n      const auto lambda1 = [a = x](int b){ return a + b; };\n      const auto lambda2 = [a = lambda1(x)](){ return a; };\n      return lambda2();\n    }\n\n  }\n\n  namespace test_digit_separators\n  {\n\n    constexpr auto ten_million = 100'000'000;\n    static_assert(ten_million == 100000000, \"\");\n\n  }\n\n  namespace test_return_type_deduction\n  {\n\n    auto f(int& x) { return x; }\n    decltype(auto) g(int& x) { return x; }\n\n    template < typename T1, typename T2 >\n    struct is_same\n    {\n      static constexpr auto value = false;\n    };\n\n    template < typename T >\n    struct is_same<T, T>\n    {\n      static constexpr auto value = true;\n    };\n\n    int\n    test()\n    {\n      auto x = 0;\n      static_assert(is_same<int, decltype(f(x))>::value, \"\");\n      static_assert(is_same<int&, decltype(g(x))>::value, \"\");\n      return x;\n    }\n\n  }\n\n}  // namespace cxx14\n\n#endif  // __cplusplus >= 201402L\n\n]])\n\n\ndnl  Tests for new features in C++17\n\nm4_define([_AX_CXX_COMPILE_STDCXX_testbody_new_in_17], [[\n\n// If the compiler admits that it is not ready for C++17, why torture it?\n// Hopefully, this will speed up the test.\n\n#ifndef __cplusplus\n\n#error \"This is not a C++ compiler\"\n\n#elif __cplusplus <= 201402L\n\n#error \"This is not a C++17 compiler\"\n\n#else\n\n#if defined(__clang__)\n  #define REALLY_CLANG\n#else\n  #if defined(__GNUC__)\n    #define REALLY_GCC\n  #endif\n#endif\n\n#include <initializer_list>\n#include <utility>\n#include <type_traits>\n\nnamespace cxx17\n{\n\n#if !defined(REALLY_CLANG)\n  namespace test_constexpr_lambdas\n  {\n\n    // TODO: test it with clang++ from git\n\n    constexpr int foo = [](){return 42;}();\n\n  }\n#endif // !defined(REALLY_CLANG)\n\n  namespace test::nested_namespace::definitions\n  {\n\n  }\n\n  namespace test_fold_expression\n  {\n\n    template<typename... Args>\n    int multiply(Args... args)\n    {\n      return (args * ... * 1);\n    }\n\n    template<typename... Args>\n    bool all(Args... args)\n    {\n      return (args && ...);\n    }\n\n  }\n\n  namespace test_extended_static_assert\n  {\n\n    static_assert (true);\n\n  }\n\n  namespace test_auto_brace_init_list\n  {\n\n    auto foo = {5};\n    auto bar {5};\n\n    static_assert(std::is_same<std::initializer_list<int>, decltype(foo)>::value);\n    static_assert(std::is_same<int, decltype(bar)>::value);\n  }\n\n  namespace test_typename_in_template_template_parameter\n  {\n\n    template<template<typename> typename X> struct D;\n\n  }\n\n  namespace test_fallthrough_nodiscard_maybe_unused_attributes\n  {\n\n    int f1()\n    {\n      return 42;\n    }\n\n    [[nodiscard]] int f2()\n    {\n      [[maybe_unused]] auto unused = f1();\n\n      switch (f1())\n      {\n      case 17:\n        f1();\n        [[fallthrough]];\n      case 42:\n        f1();\n      }\n      return f1();\n    }\n\n  }\n\n  namespace test_extended_aggregate_initialization\n  {\n\n    struct base1\n    {\n      int b1, b2 = 42;\n    };\n\n    struct base2\n    {\n      base2() {\n        b3 = 42;\n      }\n      int b3;\n    };\n\n    struct derived : base1, base2\n    {\n        int d;\n    };\n\n    derived d1 {{1, 2}, {}, 4};  // full initialization\n    derived d2 {{}, {}, 4};      // value-initialized bases\n\n  }\n\n  namespace test_general_range_based_for_loop\n  {\n\n    struct iter\n    {\n      int i;\n\n      int& operator* ()\n      {\n        return i;\n      }\n\n      const int& operator* () const\n      {\n        return i;\n      }\n\n      iter& operator++()\n      {\n        ++i;\n        return *this;\n      }\n    };\n\n    struct sentinel\n    {\n      int i;\n    };\n\n    bool operator== (const iter& i, const sentinel& s)\n    {\n      return i.i == s.i;\n    }\n\n    bool operator!= (const iter& i, const sentinel& s)\n    {\n      return !(i == s);\n    }\n\n    struct range\n    {\n      iter begin() const\n      {\n        return {0};\n      }\n\n      sentinel end() const\n      {\n        return {5};\n      }\n    };\n\n    void f()\n    {\n      range r {};\n\n      for (auto i : r)\n      {\n        [[maybe_unused]] auto v = i;\n      }\n    }\n\n  }\n\n  namespace test_lambda_capture_asterisk_this_by_value\n  {\n\n    struct t\n    {\n      int i;\n      int foo()\n      {\n        return [*this]()\n        {\n          return i;\n        }();\n      }\n    };\n\n  }\n\n  namespace test_enum_class_construction\n  {\n\n    enum class byte : unsigned char\n    {};\n\n    byte foo {42};\n\n  }\n\n  namespace test_constexpr_if\n  {\n\n    template <bool cond>\n    int f ()\n    {\n      if constexpr(cond)\n      {\n        return 13;\n      }\n      else\n      {\n        return 42;\n      }\n    }\n\n  }\n\n  namespace test_selection_statement_with_initializer\n  {\n\n    int f()\n    {\n      return 13;\n    }\n\n    int f2()\n    {\n      if (auto i = f(); i > 0)\n      {\n        return 3;\n      }\n\n      switch (auto i = f(); i + 4)\n      {\n      case 17:\n        return 2;\n\n      default:\n        return 1;\n      }\n    }\n\n  }\n\n#if !defined(REALLY_CLANG)\n  namespace test_template_argument_deduction_for_class_templates\n  {\n\n    // TODO: test it with clang++ from git\n\n    template <typename T1, typename T2>\n    struct pair\n    {\n      pair (T1 p1, T2 p2)\n        : m1 {p1},\n          m2 {p2}\n      {}\n\n      T1 m1;\n      T2 m2;\n    };\n\n    void f()\n    {\n      [[maybe_unused]] auto p = pair{13, 42u};\n    }\n\n  }\n#endif // !defined(REALLY_CLANG)\n\n  namespace test_non_type_auto_template_parameters\n  {\n\n    template <auto n>\n    struct B\n    {};\n\n    B<5> b1;\n    B<'a'> b2;\n\n  }\n\n#if !defined(REALLY_CLANG)\n  namespace test_structured_bindings\n  {\n\n    // TODO: test it with clang++ from git\n\n    int arr[2] = { 1, 2 };\n    std::pair<int, int> pr = { 1, 2 };\n\n    auto f1() -> int(&)[2]\n    {\n      return arr;\n    }\n\n    auto f2() -> std::pair<int, int>&\n    {\n      return pr;\n    }\n\n    struct S\n    {\n      int x1 : 2;\n      volatile double y1;\n    };\n\n    S f3()\n    {\n      return {};\n    }\n\n    auto [ x1, y1 ] = f1();\n    auto& [ xr1, yr1 ] = f1();\n    auto [ x2, y2 ] = f2();\n    auto& [ xr2, yr2 ] = f2();\n    const auto [ x3, y3 ] = f3();\n\n  }\n#endif // !defined(REALLY_CLANG)\n\n#if !defined(REALLY_CLANG)\n  namespace test_exception_spec_type_system\n  {\n\n    // TODO: test it with clang++ from git\n\n    struct Good {};\n    struct Bad {};\n\n    void g1() noexcept;\n    void g2();\n\n    template<typename T>\n    Bad\n    f(T*, T*);\n\n    template<typename T1, typename T2>\n    Good\n    f(T1*, T2*);\n\n    static_assert (std::is_same_v<Good, decltype(f(g1, g2))>);\n\n  }\n#endif // !defined(REALLY_CLANG)\n\n  namespace test_inline_variables\n  {\n\n    template<class T> void f(T)\n    {}\n\n    template<class T> inline T g(T)\n    {\n      return T{};\n    }\n\n    template<> inline void f<>(int)\n    {}\n\n    template<> int g<>(int)\n    {\n      return 5;\n    }\n\n  }\n\n}  // namespace cxx17\n\n#endif  // __cplusplus <= 201402L\n\n]])\n"
  },
  {
    "path": "m4/ax_cxx_compile_stdcxx_14.m4",
    "content": "# =============================================================================\n#  https://www.gnu.org/software/autoconf-archive/ax_cxx_compile_stdcxx_14.html\n# =============================================================================\n#\n# SYNOPSIS\n#\n#   AX_CXX_COMPILE_STDCXX_14([ext|noext], [mandatory|optional])\n#\n# DESCRIPTION\n#\n#   Check for baseline language coverage in the compiler for the C++14\n#   standard; if necessary, add switches to CXX and CXXCPP to enable\n#   support.\n#\n#   This macro is a convenience alias for calling the AX_CXX_COMPILE_STDCXX\n#   macro with the version set to C++14.  The two optional arguments are\n#   forwarded literally as the second and third argument respectively.\n#   Please see the documentation for the AX_CXX_COMPILE_STDCXX macro for\n#   more information.  If you want to use this macro, you also need to\n#   download the ax_cxx_compile_stdcxx.m4 file.\n#\n# LICENSE\n#\n#   Copyright (c) 2015 Moritz Klammler <moritz@klammler.eu>\n#\n#   Copying and distribution of this file, with or without modification, are\n#   permitted in any medium without royalty provided the copyright notice\n#   and this notice are preserved. This file is offered as-is, without any\n#   warranty.\n\n#serial 5\n\nAX_REQUIRE_DEFINED([AX_CXX_COMPILE_STDCXX])\nAC_DEFUN([AX_CXX_COMPILE_STDCXX_14], [AX_CXX_COMPILE_STDCXX([14], [$1], [$2])])\n"
  },
  {
    "path": "priv/.gitignore",
    "content": "*.so\n"
  },
  {
    "path": "python-api/.gitignore",
    "content": "/*.pyc\n/*.pyo\n/.*\n/scalaris\n"
  },
  {
    "path": "python-api/Scalaris.e4p",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<!DOCTYPE Project SYSTEM \"Project-5.1.dtd\">\n<!-- eric5 project file for project Scalaris -->\n<!-- Saved: 2012-03-26, 10:40:53 -->\n<!-- Copyright (C) 2012 Nico Kruber, kruber@zib.de -->\n<Project version=\"5.1\">\n  <Language>en</Language>\n  <Hash>32ef564fa76bfa734e558228c0a08174613aa726</Hash>\n  <ProgLanguage mixed=\"0\">Python2</ProgLanguage>\n  <ProjectType>Console</ProjectType>\n  <Description></Description>\n  <Version>0.1</Version>\n  <Author>Nico Kruber</Author>\n  <Email>kruber@zib.de</Email>\n  <Sources>\n    <Source>scalaris.py</Source>\n    <Source>scalaris_bench.py</Source>\n    <Source>scalaris_client.py</Source>\n    <Source>scalaris_interop_test.py</Source>\n    <Source>scalaris_test.py</Source>\n    <Source>scalaris_userdevguide_jsontrace.py</Source>\n  </Sources>\n  <Forms/>\n  <Translations/>\n  <Resources/>\n  <Interfaces/>\n  <Others/>\n  <MainScript>scalaris.py</MainScript>\n  <Vcs>\n    <VcsType>None</VcsType>\n  </Vcs>\n  <FiletypeAssociations>\n    <FiletypeAssociation pattern=\"*.idl\" type=\"INTERFACES\"/>\n    <FiletypeAssociation pattern=\"*.ptl\" type=\"SOURCES\"/>\n    <FiletypeAssociation pattern=\"*.py\" type=\"SOURCES\"/>\n    <FiletypeAssociation pattern=\"*.pyw\" type=\"SOURCES\"/>\n  </FiletypeAssociations>\n  <Checkers>\n    <CheckersParams>\n      <dict>\n        <key>\n          <string>PYLINT</string>\n        </key>\n        <value>\n          <dict>\n            <key>\n              <string>dialogReport</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>enableBasic</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>enableClasses</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>enableDesign</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>enableExceptions</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>enableFormat</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>enableImports</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>enableMetrics</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>enableMiscellaneous</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>enableNewstyle</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>enableRPython</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>enableSimilarities</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>enableTypecheck</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>enableVariables</string>\n            </key>\n            <value>\n              <bool>True</bool>\n            </value>\n            <key>\n              <string>htmlReport</string>\n            </key>\n            <value>\n              <bool>False</bool>\n            </value>\n            <key>\n              <string>txtReport</string>\n            </key>\n            <value>\n              <bool>False</bool>\n            </value>\n          </dict>\n        </value>\n      </dict>\n    </CheckersParams>\n  </Checkers>\n</Project>\n"
  },
  {
    "path": "python-api/doc/.gitignore",
    "content": "/html\n/pdf\n"
  },
  {
    "path": "python-api/epydoc.html.cfg",
    "content": "[epydoc] # Epydoc section marker (required by ConfigParser)\n\n# The list of objects to document.  Objects can be named using\n# dotted names, module filenames, or package directory names.\n# Alases for this option include \"objects\" and \"values\".\nmodules: scalaris.py scalaris_bench.py\n\n# The type of output that should be generated.  Should be one\n# of: html, text, latex, dvi, ps, pdf.\noutput: html\n\n# The path to the output directory.  May be relative or absolute.\ntarget: doc/html/\n\n# An integer indicating how verbose epydoc should be.  The default\n# value is 0; negative values will supress warnings and errors;\n# positive values will give more verbose output.\nverbosity: 0\n\n# A boolean value indicating that Epydoc should show a tracaback\n# in case of unexpected error. By default don't show tracebacks\ndebug: 0\n\n# If True, don't try to use colors or cursor control when doing\n# textual output. The default False assumes a rich text prompt\nsimple-term: 0\n\n\n### Generation options\n\n# The default markup language for docstrings, for modules that do\n# not define __docformat__.  Defaults to epytext.\ndocformat: epytext\n\n# Whether or not parsing should be used to examine objects.\nparse: yes\n\n# Whether or not introspection should be used to examine objects.\nintrospect: yes\n\n# Don't examine in any way the modules whose dotted name match this\n# regular expression pattern.\n#exclude\n\n# Don't perform introspection on the modules whose dotted name match this\n# regular expression pattern.\n#exclude-introspect\n\n# Don't perform parsing on the modules whose dotted name match this\n# regular expression pattern.\n#exclude-parse\n\n# The format for showing inheritance objects.\n# It should be one of: 'grouped', 'listed', 'included'.\ninheritance: grouped\n\n# Whether or not to inclue private variables.  (Even if included,\n# private variables will be hidden by default.)\nprivate: yes\n\n# Whether or not to list each module's imports.\nimports: yes\n\n# Whether or not to include syntax highlighted source code in\n# the output (HTML only).\nsourcecode: yes\n\n# Whether or not to includea a page with Epydoc log, containing\n# effective option at the time of generation and the reported logs.\ninclude-log: no\n\n\n### Output options\n\n# The documented project's name.\nname: Scalaris Python API\n\n# The CSS stylesheet for HTML output.  Can be the name of a builtin\n# stylesheet, or the name of a file.\ncss: white\n\n# The documented project's URL.\nurl: http://code.google.com/p/scalaris/\n\n# HTML code for the project link in the navigation bar.  If left\n# unspecified, the project link will be generated based on the\n# project's name and URL.\n#link: <a href=\"somewhere\">My Cool Project</a>\n\n# The \"top\" page for the documentation.  Can be a URL, the name\n# of a module or class, or one of the special names \"trees.html\",\n# \"indices.html\", or \"help.html\"\n#top: os.path\n\n# An alternative help file.  The named file should contain the\n# body of an HTML file; navigation bars will be added to it.\n#help: my_helpfile.html\n\n# Whether or not to include a frames-based table of contents.\nframes: yes\n\n# Whether each class should be listed in its own section when\n# generating LaTeX or PDF output.\nseparate-classes: no\n\n\n### API linking options\n\n# Define a new API document.  A new interpreted text role\n# will be created\n#external-api: epydoc\n\n# Use the records in this file to resolve objects in the API named NAME.\n#external-api-file: epydoc:api-objects.txt\n\n# Use this URL prefix to configure the string returned for external API.\n#external-api-root: epydoc:http://epydoc.sourceforge.net/api\n\n\n### Graph options\n\n# The list of graph types that should be automatically included\n# in the output.  Graphs are generated using the Graphviz \"dot\"\n# executable.  Graph types include: \"classtree\", \"callgraph\",\n# \"umlclass\".  Use \"all\" to include all graph types\ngraph: all\n\n# The path to the Graphviz \"dot\" executable, used to generate\n# graphs.\n#dotpath: /usr/local/bin/dot\n\n# The name of one or more pstat files (generated by the profile\n# or hotshot module).  These are used to generate call graphs.\n#pstat: profile.out\n\n# Specify the font used to generate Graphviz graphs.\n# (e.g., helvetica or times).\ngraph-font: Helvetica\n\n# Specify the font size used to generate Graphviz graphs.\ngraph-font-size: 10\n\n\n### Return value options\n\n# The condition upon which Epydoc should exit with a non-zero\n# exit status. Possible values are error, warning, docstring_warning\nfail-on: error\n"
  },
  {
    "path": "python-api/epydoc.pdf.cfg",
    "content": "[epydoc] # Epydoc section marker (required by ConfigParser)\n\n# The list of objects to document.  Objects can be named using\n# dotted names, module filenames, or package directory names.\n# Alases for this option include \"objects\" and \"values\".\nmodules: scalaris.py scalaris_bench.py\n\n# The type of output that should be generated.  Should be one\n# of: html, text, latex, dvi, ps, pdf.\noutput: pdf\n\n# The path to the output directory.  May be relative or absolute.\ntarget: doc/pdf/\n\n# An integer indicating how verbose epydoc should be.  The default\n# value is 0; negative values will supress warnings and errors;\n# positive values will give more verbose output.\nverbosity: 0\n\n# A boolean value indicating that Epydoc should show a tracaback\n# in case of unexpected error. By default don't show tracebacks\ndebug: 0\n\n# If True, don't try to use colors or cursor control when doing\n# textual output. The default False assumes a rich text prompt\nsimple-term: 0\n\n\n### Generation options\n\n# The default markup language for docstrings, for modules that do\n# not define __docformat__.  Defaults to epytext.\ndocformat: epytext\n\n# Whether or not parsing should be used to examine objects.\nparse: yes\n\n# Whether or not introspection should be used to examine objects.\nintrospect: yes\n\n# Don't examine in any way the modules whose dotted name match this\n# regular expression pattern.\n#exclude\n\n# Don't perform introspection on the modules whose dotted name match this\n# regular expression pattern.\n#exclude-introspect\n\n# Don't perform parsing on the modules whose dotted name match this\n# regular expression pattern.\n#exclude-parse\n\n# The format for showing inheritance objects.\n# It should be one of: 'grouped', 'listed', 'included'.\ninheritance: listed\n\n# Whether or not to inclue private variables.  (Even if included,\n# private variables will be hidden by default.)\nprivate: no\n\n# Whether or not to list each module's imports.\nimports: no\n\n# Whether or not to include syntax highlighted source code in\n# the output (HTML only).\nsourcecode: no\n\n# Whether or not to includea a page with Epydoc log, containing\n# effective option at the time of generation and the reported logs.\ninclude-log: no\n\n\n### Output options\n\n# The documented project's name.\nname: Scalaris Python API\n\n# The CSS stylesheet for HTML output.  Can be the name of a builtin\n# stylesheet, or the name of a file.\ncss: white\n\n# The documented project's URL.\nurl: http://code.google.com/p/scalaris/\n\n# HTML code for the project link in the navigation bar.  If left\n# unspecified, the project link will be generated based on the\n# project's name and URL.\n#link: <a href=\"somewhere\">My Cool Project</a>\n\n# The \"top\" page for the documentation.  Can be a URL, the name\n# of a module or class, or one of the special names \"trees.html\",\n# \"indices.html\", or \"help.html\"\n#top: os.path\n\n# An alternative help file.  The named file should contain the\n# body of an HTML file; navigation bars will be added to it.\n#help: my_helpfile.html\n\n# Whether or not to include a frames-based table of contents.\nframes: yes\n\n# Whether each class should be listed in its own section when\n# generating LaTeX or PDF output.\nseparate-classes: no\n\n\n### API linking options\n\n# Define a new API document.  A new interpreted text role\n# will be created\n#external-api: epydoc\n\n# Use the records in this file to resolve objects in the API named NAME.\n#external-api-file: epydoc:api-objects.txt\n\n# Use this URL prefix to configure the string returned for external API.\n#external-api-root: epydoc:http://epydoc.sourceforge.net/api\n\n\n### Graph options\n\n# The list of graph types that should be automatically included\n# in the output.  Graphs are generated using the Graphviz \"dot\"\n# executable.  Graph types include: \"classtree\", \"callgraph\",\n# \"umlclass\".  Use \"all\" to include all graph types\ngraph: all\n\n# The path to the Graphviz \"dot\" executable, used to generate\n# graphs.\n#dotpath: /usr/local/bin/dot\n\n# The name of one or more pstat files (generated by the profile\n# or hotshot module).  These are used to generate call graphs.\n#pstat: profile.out\n\n# Specify the font used to generate Graphviz graphs.\n# (e.g., helvetica or times).\ngraph-font: Helvetica\n\n# Specify the font size used to generate Graphviz graphs.\ngraph-font-size: 10\n\n\n### Return value options\n\n# The condition upon which Epydoc should exit with a non-zero\n# exit status. Possible values are error, warning, docstring_warning\nfail-on: error\n"
  },
  {
    "path": "python-api/scalaris.in",
    "content": "#!/bin/bash\n\n#  Copyright 2007-2015 Zuse Institute Berlin\n#\n#   Licensed under the Apache License, Version 2.0 (the \"License\");\n#   you may not use this file except in compliance with the License.\n#   You may obtain a copy of the License at\n#\n#       http://www.apache.org/licenses/LICENSE-2.0\n#\n#   Unless required by applicable law or agreed to in writing, software\n#   distributed under the License is distributed on an \"AS IS\" BASIS,\n#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#   See the License for the specific language governing permissions and\n#   limitations under the License.\n\n# inspired by the start script of ant\n\nprefix=@prefix@\nexec_prefix=@exec_prefix@\nSCALARISDIR=@libdir@/scalaris\nPYTHON2=\"@PYTHON2@\"\nSCALARISCLIENT_PYTHON2=\"@PYTHON2SITELIBDIR@/scalaris_client.py\"\n\nif [ ! -x \"$PYTHON2\" -o \"x$PYTHON2\" = \"x@FALSE@\" ] ; then\n  echo \"Python not found or not executable: $PYTHON2\"\n  exit 1\nfi\n\nfor arg in \"$@\"; do\n  args=\"$args \\\"$arg\\\"\"\ndone\n\n# activate no_config if called in the source tree:\nABSPATH=\"$(cd \"${0%/*}\" 2>/dev/null; echo \"$PWD\"/\"${0##*/}\")\"\nDIRNAME=`dirname $ABSPATH`\n# is this a source checkout or an (rpm/deb/manual) installation?\nif [ \"$DIRNAME\" != \"@bindir@\" -a \"$DIRNAME\" != \"/bin\" -a \"${DIRNAME##*/}\" = \"python-api\" ]; then\n  # local mode, i.e. uninstalled\n  SCALARISCLIENT_PYTHON2=\"`dirname $0`/scalaris_client.py\"\nfi\n\neval $PYTHON2 $SCALARISCLIENT_PYTHON2 $args\n"
  },
  {
    "path": "python-api/scalaris.py",
    "content": "# Copyright 2011-2014 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nimport httplib, urlparse, base64, urllib\nimport os, threading, numbers, socket\nfrom datetime import datetime, timedelta\ntry: import simplejson as json\nexcept ImportError: import json\n\nif 'SCALARIS_JSON_URL' in os.environ and os.environ['SCALARIS_JSON_URL'] != '':\n    DEFAULT_URL = os.environ['SCALARIS_JSON_URL']\nelse:\n    DEFAULT_URL = 'http://localhost:8000'\n\"\"\"default URL and port to a scalaris node\"\"\"\nDEFAULT_PATH = '/jsonrpc.yaws'\n\"\"\"path to the json rpc page\"\"\"\n\nclass JSONConnection(object):\n    \"\"\"\n    Abstracts connections to scalaris using JSON\n    \"\"\"\n\n    def __init__(self, url = DEFAULT_URL, timeout = socket.getdefaulttimeout()):\n        \"\"\"\n        Creates a JSON connection to the given URL using the given TCP timeout\n        \"\"\"\n        try:\n            uri = urlparse.urlparse(url)\n            self._conn = httplib.HTTPConnection(uri.hostname, uri.port,\n                                                timeout = timeout)\n        except Exception as instance:\n            raise ConnectionError(instance)\n\n    def callp(self, path, function, params, retry_if_bad_status = True):\n        return self.call(function, params, path = path, retry_if_bad_status = retry_if_bad_status)\n\n    def call(self, function, params, path = DEFAULT_PATH, retry_if_bad_status = True):\n        \"\"\"\n        Calls the given function with the given parameters via the JSON\n        interface of scalaris.\n        \"\"\"\n        params2 = {'jsonrpc': '2.0',\n                  'method': function,\n                  'params': params,\n                  'id': 0}\n        try:\n            data = None\n            response = None\n            # use compact JSON encoding:\n            params_json = json.dumps(params2, separators=(',',':'))\n            headers = {\"Content-type\": \"application/json; charset=utf-8\"}\n            # note: we need to quote, e.g. if a '%' is in the value string:\n            self._conn.request(\"POST\", path, urllib.quote(params_json), headers)\n            response = self._conn.getresponse()\n            #print response.status, response.reason\n            data = response.read().decode('utf-8')\n            if (response.status < 200 or response.status >= 300):\n                raise ConnectionError(data, response = response)\n            response_json = json.loads(data)\n            return response_json['result']\n        except httplib.BadStatusLine as instance:\n            #print 'HTTP STATUS:', response.status, response.reason, params_json\n            self.close()\n            if retry_if_bad_status:\n                return self.call(function, params, path = path, retry_if_bad_status = False)\n            else:\n                raise ConnectionError(data, response = response, error = instance)\n        except ConnectionError:\n            #print 'HTTP STATUS:', response.status, response.reason, params_json\n            self.close()\n            raise\n        except Exception as instance:\n            #print 'HTTP STATUS:', response.status, response.reason, params_json\n            self.close()\n            raise ConnectionError(data, response = response, error = instance)\n\n    @staticmethod\n    def encode_value(value):\n        \"\"\"\n        Encodes the value to the form required by the scalaris JSON API\n        \"\"\"\n        if isinstance(value, bytearray):\n            return {'type': 'as_bin', 'value': (base64.b64encode(bytes(value))).decode('ascii')}\n        else:\n            return {'type': 'as_is', 'value': value}\n\n    @staticmethod\n    def decode_value(value):\n        \"\"\"\n        Decodes the value from the scalaris JSON API form to a native type\n        \"\"\"\n        if ('type' not in value) or ('value' not in value):\n            raise UnknownError(value)\n        if value['type'] == 'as_bin':\n            return bytearray(base64.b64decode(value['value'].encode('ascii')))\n        else:\n            return value['value']\n\n    # result: {'status': 'ok'} or\n    #         {'status': 'fail', 'reason': 'timeout'}\n    @staticmethod\n    def check_fail_abort(result):\n        \"\"\"\n        Processes the result of some Scalaris operation and raises a\n        TimeoutError if found.\n        \"\"\"\n        if result == {'status': 'fail', 'reason': 'timeout'}:\n            raise TimeoutError(result)\n\n    # result: {'status': 'ok', 'value': xxx} or\n    #         {'status': 'fail', 'reason': 'timeout' or 'not_found'}\n    @staticmethod\n    def process_result_read(result):\n        \"\"\"\n        Processes the result of a read operation.\n        Returns the read value on success.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'status' in result and len(result) == 2:\n            if result['status'] == 'ok' and 'value' in result:\n                return JSONConnection.decode_value(result['value'])\n            elif result['status'] == 'fail' and 'reason' in result:\n                if result['reason'] == 'timeout':\n                    raise TimeoutError(result)\n                elif result['reason'] == 'not_found':\n                    raise NotFoundError(result)\n        raise UnknownError(result)\n\n    # result: {'status': 'ok'} or\n    #         {'status': 'fail', 'reason': 'timeout'}\n    @staticmethod\n    def process_result_write(result):\n        \"\"\"\n        Processes the result of a write operation.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict):\n            if result == {'status': 'ok'}:\n                return None\n            elif result == {'status': 'fail', 'reason': 'timeout'}:\n                raise TimeoutError(result)\n        raise UnknownError(result)\n\n    # result: {'status': 'ok'} or\n    #         {'status': 'fail', 'reason': 'abort', 'keys': <list>} or\n    #         {'status': 'fail', 'reason': 'timeout'}\n    @staticmethod\n    def process_result_commit(result):\n        \"\"\"\n        Processes the result of a commit operation.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'status' in result:\n            if result == {'status': 'ok'}:\n                return None\n            elif result['status'] == 'fail' and 'reason' in result:\n                if len(result) == 2 and result['reason'] == 'timeout':\n                    raise TimeoutError(result)\n                elif len(result) == 3 and result['reason'] == 'abort' and 'keys' in result:\n                    raise AbortError(result, result['keys'])\n        raise UnknownError(result)\n\n    # results: {'status': 'ok'} or\n    #          {'status': 'fail', 'reason': 'timeout' or 'not_a_list'} or\n    @staticmethod\n    def process_result_add_del_on_list(result):\n        \"\"\"\n        Processes the result of a add_del_on_list operation.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'status' in result:\n            if result == {'status': 'ok'}:\n                return None\n            elif result['status'] == 'fail' and 'reason' in result:\n                if len(result) == 2:\n                    if result['reason'] == 'timeout':\n                        raise TimeoutError(result)\n                    elif result['reason'] == 'not_a_list':\n                        raise NotAListError(result)\n        raise UnknownError(result)\n\n    # results: {'status': 'ok'} or\n    #          {'status': 'fail', 'reason': 'timeout' or 'not_a_number'} or\n    @staticmethod\n    def process_result_add_on_nr(result):\n        \"\"\"\n        Processes the result of a add_on_nr operation.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'status' in result:\n            if result == {'status': 'ok'}:\n                return None\n            elif result['status'] == 'fail' and 'reason' in result:\n                if len(result) == 2:\n                    if result['reason'] == 'timeout':\n                        raise TimeoutError(result)\n                    elif result['reason'] == 'not_a_number':\n                        raise NotANumberError(result)\n        raise UnknownError(result)\n\n    # results: {'status': 'ok'} or\n    #          {'status': 'fail', 'reason': 'timeout' or 'not_found'} or\n    #          {'status': 'fail', 'reason': 'key_changed', 'value': xxx}\n    @staticmethod\n    def process_result_test_and_set(result):\n        \"\"\"\n        Processes the result of a test_and_set operation.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'status' in result:\n            if result == {'status': 'ok'}:\n                return None\n            elif result['status'] == 'fail' and 'reason' in result:\n                if len(result) == 2:\n                    if result['reason'] == 'timeout':\n                        raise TimeoutError(result)\n                    elif result['reason'] == 'not_found':\n                        raise NotFoundError(result)\n                elif result['reason'] == 'key_changed' and 'value' in result and len(result) == 3:\n                    raise KeyChangedError(result, JSONConnection.decode_value(result['value']))\n        raise UnknownError(result)\n\n    # results: {'ok': xxx, 'results': ['ok' or 'locks_set' or 'undef']} or\n    #          {'failure': 'timeout', 'ok': xxx, 'results': ['ok' or 'locks_set' or 'undef']}\n    @staticmethod\n    def process_result_delete(result):\n        \"\"\"\n        Processes the result of a delete operation.\n        Returns the tuple\n        (<success (True | 'timeout')>, <number of deleted items>, <detailed results>) on success.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'ok' in result and 'results' in result:\n            if 'failure' not in result:\n                return (True, result['ok'], result['results'])\n            elif result['failure'] == 'timeout':\n                return ('timeout', result['ok'], result['results'])\n        raise UnknownError(result)\n\n    # results: ['ok' or 'locks_set' or 'undef']\n    @staticmethod\n    def create_delete_result(result):\n        \"\"\"\n        Creates a new DeleteResult from the given result list.\n        \"\"\"\n        ok = 0\n        locks_set = 0\n        undefined = 0\n        if isinstance(result, list):\n            for element in result:\n                if element == 'ok':\n                    ok += 1\n                elif element == 'locks_set':\n                    locks_set += 1\n                elif element == 'undef':\n                    undefined += 1\n                else:\n                    raise UnknownError('Unknown reason ' + element + 'in ' + result)\n            return DeleteResult(ok, locks_set, undefined)\n        raise UnknownError('Unknown result ' + result)\n\n    # results: {'tlog': xxx,\n    #           'results': [{'status': 'ok'} or {'status': 'ok', 'value': xxx} or\n    #                       {'status': 'fail', 'reason': 'timeout' or 'abort' or 'not_found'}]}\n    @staticmethod\n    def process_result_req_list_t(result):\n        \"\"\"\n        Processes the result of a req_list operation of the Transaction class.\n        Returns the tuple (<tlog>, <result>) on success.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if 'tlog' not in result or 'results' not in result or \\\n            not isinstance(result['results'], list):\n            raise UnknownError(result)\n        return (result['tlog'], result['results'])\n\n    # results: [{'status': 'ok'} or {'status': 'ok', 'value': xxx} or\n    #           {'status': 'fail', 'reason': 'timeout' or 'abort' or 'not_found'}]\n    @staticmethod\n    def process_result_req_list_tso(result):\n        \"\"\"\n        Processes the result of a req_list operation of the TransactionSingleOp class.\n        Returns <result> on success.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if not isinstance(result, list):\n            raise UnknownError(result)\n        return result\n\n    # results: {'status': 'ok', 'value': xxx}\n    @staticmethod\n    def process_result_vm_get_version(result):\n        \"\"\"\n        Processes the result of a api_vm/get_version operation.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'status' in result and 'value' in result:\n            if result['status'] == 'ok':\n                return result['value']\n        raise UnknownError(result)\n\n    # value: {'scalaris_version': xxx,\n    #         'erlang_version': xxx,\n    #         'mem_total': xxx,\n    #         'uptime': xxx,\n    #         'erlang_node': xxx,\n    #         'ip': xxx,\n    #         'port': xxx,\n    #         'yaws_port': xxx}\n    # results: {'status': 'ok', 'value': <value>\n    @staticmethod\n    def process_result_vm_get_info(result):\n        \"\"\"\n        Processes the result of a api_vm/get_info operation.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'status' in result and 'value' in result:\n            value = result['value']\n            if result['status'] == 'ok' and \\\n               'scalaris_version' in value and 'erlang_version' in value and \\\n               'mem_total' in value and 'uptime' in value and \\\n               'erlang_node' in value and 'ip' in value and \\\n               'port' in value and 'yaws_port' in value:\n                try:\n                    return ScalarisVM.GetInfoResult(value['scalaris_version'],\n                                                    value['erlang_version'],\n                                                    int(value['mem_total']),\n                                                    int(value['uptime']),\n                                                    value['erlang_node'],\n                                                    value['ip'],\n                                                    int(value['port']),\n                                                    int(value['yaws_port']))\n                except:\n                    pass\n        raise UnknownError(result)\n\n    # results: {'status': 'ok', 'value': xxx}\n    @staticmethod\n    def process_result_vm_get_number_of_nodes(result):\n        \"\"\"\n        Processes the result of a api_vm/number_of_nodes operation.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'status' in result and 'value' in result:\n            if result['status'] == 'ok':\n                try:\n                    return int(result['value'])\n                except:\n                    pass\n        raise UnknownError(result)\n\n    # results: {'status': 'ok', 'value': [xxx]}\n    @staticmethod\n    def process_result_vm_get_nodes(result):\n        \"\"\"\n        Processes the result of a api_vm/get_nodes operation.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'status' in result and 'value' in result:\n            if result['status'] == 'ok' and isinstance(result['value'], list):\n                return result['value']\n        raise UnknownError(result)\n\n    # results: {'status': 'ok', 'ok': [xxx], 'failed': [xxx]}\n    @staticmethod\n    def process_result_vm_add_nodes(result):\n        \"\"\"\n        Processes the result of a api_vm/add_nodes operation.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'status' in result and 'ok' in result and 'failed' in result:\n            if result['status'] == 'ok' and isinstance(result['ok'], list) and isinstance(result['failed'], list):\n                return (result['ok'], result['failed'])\n        raise UnknownError(result)\n\n    # results: {'status': 'ok' | 'not_found'}\n    @staticmethod\n    def process_result_vm_delete_node(result):\n        \"\"\"\n        Processes the result of a api_vm/shutdown_node and api_vm/kill_node operations.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if result == {'status': 'ok'}:\n            return True\n        if result == {'status': 'not_found'}:\n            return False\n        raise UnknownError(result)\n\n    # results: {'status': 'ok', 'ok': [xxx]}\n    @staticmethod\n    def process_result_vm_delete_nodes(result):\n        \"\"\"\n        Processes the result of a api_vm/shutdown_nodes and api_vm/kill_nodes operations.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'status' in result and 'ok' in result and \\\n           result['status'] == 'ok' and isinstance(result['ok'], list):\n            return result['ok']\n        raise UnknownError(result)\n\n    # results: {'status': 'ok', 'ok': [xxx], 'not_found': [xxx]}\n    @staticmethod\n    def process_result_vm_delete_nodes_by_name(result):\n        \"\"\"\n        Processes the result of a api_vm/shutdown_nodes_by_name and api_vm/kill_nodes_by_name operations.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'status' in result and 'ok' in result and 'not_found' in result:\n            if result['status'] == 'ok' and isinstance(result['ok'], list) and isinstance(result['not_found'], list):\n                return (result['ok'], result['not_found'])\n        raise UnknownError(result)\n\n    # results: {'status': 'ok'}\n    @staticmethod\n    def process_result_vm_delete_vm(result):\n        \"\"\"\n        Processes the result of a api_vm/shutdown_vm and api_vm/kill_vm operations.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if result == {'status': 'ok'}:\n            return None\n        raise UnknownError(result)\n\n    # VM: {'erlang_node': xxx,\n    #      'ip': xxx,\n    #      'port': xxx,\n    #      'yaws_port': xxx}\n    # results: {'status': 'ok', 'value': [<VM>]\n    @staticmethod\n    def process_result_vm_get_other_vms(result):\n        \"\"\"\n        Processes the result of a api_vm/get_other_vms operation.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if isinstance(result, dict) and 'status' in result and 'value' in result:\n            value = result['value']\n            if result['status'] == 'ok' and isinstance(value, list):\n                vms = []\n                try:\n                    for vm in value:\n                        if 'erlang_node' in vm and 'ip' in vm and \\\n                           'port' in vm and 'yaws_port' in vm:\n                            vms.append('http://' + vm['ip'] + ':' + str(int(vm['yaws_port'])))\n                        else:\n                            raise UnknownError(result)\n                    return vms\n                except:\n                    pass\n        raise UnknownError(result)\n\n    # results: {'status': 'ok' | 'error'}\n    @staticmethod\n    def process_result_autoscale_check_config(result):\n        if isinstance(result, dict) and 'status' in result:\n            if result['status'] == 'ok':\n                return True\n            else:\n                return False\n        raise UnknownError(result)\n\n    # <reason>: 'resp_timeout' | 'autoscale_false'\n    # results: {'status': 'ok', 'value' : <number>} or\n    #          {'status': 'error', 'reason': <reason>}\n    @staticmethod\n    def process_result_autoscale_pull_scale_req(result):\n        if isinstance(result, dict) and 'status' in result:\n            if result['status'] == 'ok' and 'value' in result and isinstance(result['value'], int):\n                return result['value']\n\n            if result['status'] == 'error' and 'reason' in result:\n                if result['reason'] == 'resp_timeout':\n                    raise TimeoutError(result)\n                elif result['reason'] == 'autoscale_false':\n                    raise ConfigError(result)\n        raise UnknownError(result)\n\n    # <reason>: 'locked' | 'resp_timeout' | 'autoscale_false'\n    # results: {'status' : 'ok'} or\n    #          {'status': 'error', 'reason': <reason>}\n    @staticmethod\n    def process_result_autoscale_lock_scale_req(result):\n        if isinstance(result, dict) and 'status' in result:\n            if result['status'] == 'ok':\n                return result['status']\n\n            if result['status'] == 'error':\n                if result['reason'] == 'locked':\n                    raise LockError(result)\n                elif result['reason'] == 'resp_timeout':\n                    raise TimeoutError(result)\n                elif result['reason'] == 'autoscale_false':\n                    raise ConfigError(result)\n\n        return UnknownError(result)\n\n    # <reason>: 'not_locked' | 'resp_timeout' | 'autoscale_false'\n    # results: {'status' : 'ok'} or\n    #          {'status': 'error', 'reason': <reason>}\n    @staticmethod\n    def process_result_autoscale_unlock_scale_req(result):\n        if isinstance(result, dict) and 'status' in result:\n            if result['status'] == 'ok':\n                return result['status']\n\n            if result['status'] == 'error':\n                if result['reason'] == 'not_locked':\n                    raise LockError(result)\n                elif result['reason'] == 'resp_timeout':\n                    raise TimeoutError(result)\n                elif result['reason'] == 'autoscale_false':\n                    raise ConfigError(result)\n\n        return UnknownError(result)\n\n    # result: 'ok'\n    @staticmethod\n    def process_result_nop(result):\n        \"\"\"\n        Processes the result of a nop operation.\n        Raises the appropriate exception if the operation failed.\n        \"\"\"\n        if result != 'ok':\n            raise UnknownError(result)\n\n    @staticmethod\n    def new_req_list_t(other = None):\n        \"\"\"\n        Returns a new ReqList object allowing multiple parallel requests for\n        the Transaction class.\n        \"\"\"\n        return _JSONReqListTransaction(other)\n\n    @staticmethod\n    def new_req_list_tso(other = None):\n        \"\"\"\n        Returns a new ReqList object allowing multiple parallel requests for\n        the TransactionSingleOp class.\n        \"\"\"\n        return _JSONReqListTransactionSingleOp(other)\n\n    def close(self):\n        self._conn.close()\n\nclass ScalarisError(Exception):\n    \"\"\"Base class for errors in the scalaris package.\"\"\"\n\nclass AbortError(ScalarisError):\n    \"\"\"\n    Exception that is thrown if a the commit of a write operation on a scalaris\n    ring fails.\n    \"\"\"\n\n    def __init__(self, raw_result, failed_keys):\n        self.raw_result = raw_result\n        self.failed_keys = failed_keys\n    def __str__(self):\n        return repr(self.raw_result)\n\nclass ConnectionError(ScalarisError):\n    \"\"\"\n    Exception that is thrown if an operation on a scalaris ring fails because\n    a connection does not exist or has been disconnected.\n    \"\"\"\n\n    def __init__(self, raw_result, response = None, error = None):\n        self.raw_result = raw_result\n        self.response = response\n        self.error = error\n    def __str__(self):\n        result_str = ''\n        if self.response is not None:\n            result_str += 'status: ' + str(self.response.status)\n            result_str += ', reason: ' + self.response.reason + '\\n'\n        if self.error is not None:\n            result_str += 'error: ' + repr(self.error) + '\\n'\n        result_str += 'data: ' + repr(self.raw_result)\n        return result_str\n\nclass KeyChangedError(ScalarisError):\n    \"\"\"\n    Exception that is thrown if a test_and_set operation on a scalaris ring\n    fails because the old value did not match the expected value.\n    \"\"\"\n\n    def __init__(self, raw_result, old_value):\n        self.raw_result = raw_result\n        self.old_value = old_value\n    def __str__(self):\n        return repr(self.raw_result) + ', old value: ' + repr(self.old_value)\n\nclass NodeNotFoundError(ScalarisError):\n    \"\"\"\n    Exception that is thrown if a delete operation on a scalaris ring fails\n    because no scalaris node was found.\n    \"\"\"\n\n    def __init__(self, raw_result):\n        self.raw_result = raw_result\n    def __str__(self):\n        return repr(self.raw_result)\n\nclass NotFoundError(ScalarisError):\n    \"\"\"\n    Exception that is thrown if a read operation on a scalaris ring fails\n    because the key did not exist before.\n    \"\"\"\n\n    def __init__(self, raw_result):\n        self.raw_result = raw_result\n    def __str__(self):\n        return repr(self.raw_result)\n\nclass NotAListError(ScalarisError):\n    \"\"\"\n    Exception that is thrown if a add_del_on_list operation on a scalaris ring\n    fails because the participating values are not lists.\n    \"\"\"\n\n    def __init__(self, raw_result):\n        self.raw_result = raw_result\n    def __str__(self):\n        return repr(self.raw_result)\n\nclass NotANumberError(ScalarisError):\n    \"\"\"\n    Exception that is thrown if a add_del_on_list operation on a scalaris ring\n    fails because the participating values are not numbers.\n    \"\"\"\n\n    def __init__(self, raw_result):\n        self.raw_result = raw_result\n    def __str__(self):\n        return repr(self.raw_result)\n\nclass TimeoutError(ScalarisError):\n    \"\"\"\n    Exception that is thrown if a read or write operation on a scalaris ring\n    fails due to a timeout.\n    \"\"\"\n\n    def __init__(self, raw_result):\n        self.raw_result = raw_result\n    def __str__(self):\n        return repr(self.raw_result)\n\nclass ConfigError(ScalarisError):\n    \"\"\"\n    Exception that is thrown if a autoscale operation fails, because it was not\n    configured correctly.\n    \"\"\"\n    def __init__(self, raw_result):\n        self.raw_result = raw_result\n    def __str__(self):\n        return repr(self.raw_result)\n\nclass LockError(ScalarisError):\n    \"\"\"\n    Exception that is thrown if a autoscale lock/unlock operation fails,\n    because of a wrong lock state, i.e. lock when is already locked or unlock\n    when not locked.\n    \"\"\"\n    def __init__(self, raw_result):\n        self.raw_result = raw_result\n    def __str__(self):\n        return repr(self.raw_result)\n\nclass UnknownError(ScalarisError):\n    \"\"\"\n    Generic exception that is thrown during operations on a scalaris ring, e.g.\n    if an unknown result has been returned.\n    \"\"\"\n\n    def __init__(self, raw_result):\n        self.raw_result = raw_result\n    def __str__(self):\n        return repr(self.raw_result)\n\nclass DeleteResult(object):\n    \"\"\"\n    Stores the result of a delete operation.\n    \"\"\"\n    def __init__(self, ok, locks_set, undefined):\n        self.ok = ok\n        self.locks_set = locks_set\n        self.undefined = undefined\n\nclass ConnectionPool(object):\n    \"\"\"\n    Implements a simple (thread-safe) connection pool for Scalaris connections.\n    \"\"\"\n\n    def __init__(self, max_connections):\n        \"\"\"\n        Create a new connection pool with the given maximum number of connections.\n        \"\"\"\n        self._max_connections = max_connections\n        self._available_conns = []\n        self._checked_out_sema = threading.BoundedSemaphore(value=max_connections)\n        self._wait_cond = threading.Condition()\n\n    def _new_connection(self):\n        \"\"\"\n        Creates a new connection for the pool. Override this to use some other\n        connection class than JSONConnection.\n        \"\"\"\n        return JSONConnection()\n\n    def _get_connection(self):\n        \"\"\"\n        Gets a connection from the pool. Creates a new connection if necessary.\n        Returns <tt>None</tt> if the maximum number of connections has already\n        been hit.\n        \"\"\"\n        conn = None\n        if self._max_connections == 0:\n            conn = self._new_connection()\n        elif self._checked_out_sema.acquire(False):\n            try:\n                conn = self._available_conns.pop(0)\n            except IndexError:\n                conn = self._new_connection()\n        return conn\n\n    def get_connection(self, timeout = None):\n        \"\"\"\n        Tries to get a valid connection from the pool waiting at most\n        the given timeout. If timeout is an integer, it will be interpreted as\n        a number of milliseconds. Alternatively, timeout can be given as a\n        datetime.timedelta. Creates a new connection if necessary\n        and the maximum number of connections has not been hit yet.\n        If the timeout is hit and no connection is available, <tt>None</tt> is\n        returned.\n        \"\"\"\n        if timeout == None:\n            return self._get_connection()\n        else:\n            if isinstance(timeout, numbers.Integral ):\n                timeout = timedelta(milliseconds=timeout)\n            start = datetime.now()\n            while True:\n                conn = self._get_connection()\n                if not conn is None:\n                    return conn\n                self._wait_cond.wait(timeout.microseconds / 1000.0)\n                end = datetime.now()\n                if end - start > timeout:\n                    return None\n\n    def release_connection(self, connection):\n        \"\"\"\n        Puts the given connection back into the pool.\n        \"\"\"\n        self._available_conns.append(connection)\n        self._checked_out_sema.release()\n        self._wait_cond.notify_all()\n\n    def close_all(self):\n        \"\"\"\n        Close all connections to scalaris.\n        \"\"\"\n        for conn in self._available_conns:\n            conn.close()\n        self._available_conns = []\n\nclass TransactionSingleOp(object):\n    \"\"\"\n    Single write or read operations on scalaris.\n    \"\"\"\n\n    def __init__(self, conn = None):\n        \"\"\"\n        Create a new object using the given connection\n        \"\"\"\n        if conn is None:\n            conn = JSONConnection()\n        self._conn = conn\n\n    def new_req_list(self, other = None):\n        \"\"\"\n        Returns a new ReqList object allowing multiple parallel requests.\n        \"\"\"\n        return self._conn.new_req_list_tso(other)\n\n    def req_list(self, reqlist):\n        \"\"\"\n        Issues multiple parallel requests to scalaris; each will be committed.\n        NOTE: The execution order of multiple requests on the same key is\n        undefined!\n        Request lists can be created using new_req_list().\n        The returned list has the following form:\n        [{'status': 'ok'} or {'status': 'ok', 'value': xxx} or\n        {'status': 'fail', 'reason': 'timeout' or 'abort' or 'not_found'}].\n        Elements of this list can be processed with process_result_read() and\n        process_result_write().\n        \"\"\"\n        result = self._conn.callp('/api/tx.yaws', 'req_list_commit_each', [reqlist.get_requests()])\n        result = self._conn.process_result_req_list_tso(result)\n        return result\n\n    def process_result_read(self, result):\n        \"\"\"\n        Processes a result element from the list returned by req_list() which\n        originated from a read operation.\n        Returns the read value on success.\n        Raises the appropriate exceptions if a failure occurred during the\n        operation.\n        Beware: lists of (small) integers may be (falsely) returned as a string -\n        use str_to_list() to convert such strings.\n        \"\"\"\n        return self._conn.process_result_read(result)\n\n    def process_result_write(self, result):\n        \"\"\"\n        Processes a result element from the list returned by req_list() which\n        originated from a write operation.\n        Raises the appropriate exceptions if a failure occurred during the\n        operation.\n        \"\"\"\n        self._conn.check_fail_abort(result)\n        return self._conn.process_result_write(result)\n\n    def process_result_add_del_on_list(self, result):\n        \"\"\"\n        Processes a result element from the list returned by req_list() which\n        originated from a add_del_on_list operation.\n        Raises the appropriate exceptions if a failure occurred during the\n        operation.\n        \"\"\"\n        self._conn.check_fail_abort(result)\n        self._conn.process_result_add_del_on_list(result)\n\n    def process_result_add_on_nr(self, result):\n        \"\"\"\n        Processes a result element from the list returned by req_list() which\n        originated from a add_on_nr operation.\n        Raises the appropriate exceptions if a failure occurred during the\n        operation.\n        \"\"\"\n        self._conn.check_fail_abort(result)\n        self._conn.process_result_add_on_nr(result)\n\n    def process_result_test_and_set(self, result):\n        \"\"\"\n        Processes a result element from the list returned by req_list() which\n        originated from a test_and_set operation.\n        Raises the appropriate exceptions if a failure occurred during the\n        operation.\n        \"\"\"\n        self._conn.check_fail_abort(result)\n        self._conn.process_result_test_and_set(result)\n\n    def read(self, key):\n        \"\"\"\n        Read the value at key.\n        Beware: lists of (small) integers may be (falsely) returned as a string -\n        use str_to_list() to convert such strings.\n        \"\"\"\n        result = self._conn.callp('/api/tx.yaws', 'read', [key])\n        return self._conn.process_result_read(result)\n\n    def write(self, key, value):\n        \"\"\"\n        Write the value to key.\n        \"\"\"\n        value = self._conn.encode_value(value)\n        result = self._conn.callp('/api/tx.yaws', 'write', [key, value])\n        self._conn.check_fail_abort(result)\n        self._conn.process_result_write(result)\n\n    def add_del_on_list(self, key, to_add, to_remove):\n        \"\"\"\n        Changes the list stored at the given key, i.e. first adds all items in\n        to_add then removes all items in to_remove.\n        Both, to_add and to_remove, must be lists.\n        Assumes en empty list if no value exists at key.\n        \"\"\"\n        result = self._conn.callp('/api/tx.yaws', 'add_del_on_list', [key, to_add, to_remove])\n        self._conn.check_fail_abort(result)\n        self._conn.process_result_add_del_on_list(result)\n\n    def add_on_nr(self, key, to_add):\n        \"\"\"\n        Changes the number stored at the given key, i.e. adds some value.\n        Assumes 0 if no value exists at key.\n        \"\"\"\n        result = self._conn.callp('/api/tx.yaws', 'add_on_nr', [key, to_add])\n        self._conn.check_fail_abort(result)\n        self._conn.process_result_add_on_nr(result)\n\n    def test_and_set(self, key, old_value, new_value):\n        \"\"\"\n        Atomic test and set, i.e. if the old value at key is old_value, then\n        write new_value.\n        \"\"\"\n        old_value = self._conn.encode_value(old_value)\n        new_value = self._conn.encode_value(new_value)\n        result = self._conn.callp('/api/tx.yaws', 'test_and_set', [key, old_value, new_value])\n        self._conn.check_fail_abort(result)\n        self._conn.process_result_test_and_set(result)\n\n    def nop(self, value):\n        \"\"\"\n        No operation (may be used for measuring the JSON overhead).\n        \"\"\"\n        value = self._conn.encode_value(value)\n        result = self._conn.callp('/api/tx.yaws', 'nop', [value])\n        self._conn.process_result_nop(result)\n\n    def close_connection(self):\n        \"\"\"\n        Close the connection to scalaris\n        (it will automatically be re-opened on the next request).\n        \"\"\"\n        self._conn.close()\n\nclass Transaction(object):\n    \"\"\"\n    Write or read operations on scalaris inside a transaction.\n    \"\"\"\n\n    def __init__(self, conn = None):\n        \"\"\"\n        Create a new object using the given connection\n        \"\"\"\n        if conn is None:\n            conn = JSONConnection()\n        self._conn = conn\n        self._tlog = None\n\n    def new_req_list(self, other = None):\n        \"\"\"\n        Returns a new ReqList object allowing multiple parallel requests.\n        \"\"\"\n        return self._conn.new_req_list_t(other)\n\n    def req_list(self, reqlist):\n        \"\"\"\n        Issues multiple parallel requests to scalaris.\n        Request lists can be created using new_req_list().\n        The returned list has the following form:\n        [{'status': 'ok'} or {'status': 'ok', 'value': xxx} or\n        {'status': 'fail', 'reason': 'timeout' or 'abort' or 'not_found'}].\n        Elements of this list can be processed with process_result_read() and\n        process_result_write().\n        A commit (at the end of the request list) will be automatically checked\n        for its success.\n        \"\"\"\n        if self._tlog is None:\n            result = self._conn.callp('/api/tx.yaws', 'req_list', [reqlist.get_requests()])\n        else:\n            result = self._conn.callp('/api/tx.yaws', 'req_list', [self._tlog, reqlist.get_requests()])\n        (tlog, result) = self._conn.process_result_req_list_t(result)\n        self._tlog = tlog\n        if reqlist.is_commit():\n            self._process_result_commit(result[-1])\n            # transaction was successful: reset transaction log\n            self._tlog = None\n        return result\n\n    def process_result_read(self, result):\n        \"\"\"\n        Processes a result element from the list returned by req_list() which\n        originated from a read operation.\n        Returns the read value on success.\n        Raises the appropriate exceptions if a failure occurred during the\n        operation.\n        Beware: lists of (small) integers may be (falsely) returned as a string -\n        use str_to_list() to convert such strings.\n        \"\"\"\n        return self._conn.process_result_read(result)\n\n    def process_result_write(self, result):\n        \"\"\"\n        Processes a result element from the list returned by req_list() which\n        originated from a write operation.\n        Raises the appropriate exceptions if a failure occurred during the\n        operation.\n        \"\"\"\n        return self._conn.process_result_write(result)\n\n    def process_result_add_del_on_list(self, result):\n        \"\"\"\n        Processes a result element from the list returned by req_list() which\n        originated from a add_del_on_list operation.\n        Raises the appropriate exceptions if a failure occurred during the\n        operation.\n        \"\"\"\n        self._conn.process_result_add_del_on_list(result)\n\n    def process_result_add_on_nr(self, result):\n        \"\"\"\n        Processes a result element from the list returned by req_list() which\n        originated from a add_on_nr operation.\n        Raises the appropriate exceptions if a failure occurred during the\n        operation.\n        \"\"\"\n        self._conn.process_result_add_on_nr(result)\n\n    def process_result_test_and_set(self, result):\n        \"\"\"\n        Processes a result element from the list returned by req_list() which\n        originated from a test_and_set operation.\n        Raises the appropriate exceptions if a failure occurred during the\n        operation.\n        \"\"\"\n        self._conn.process_result_test_and_set(result)\n\n    def _process_result_commit(self, result):\n        \"\"\"\n        Processes a result element from the list returned by req_list() which\n        originated from a commit operation.\n        Raises the appropriate exceptions if a failure occurred during the\n        operation.\n        \"\"\"\n        return self._conn.process_result_commit(result)\n\n    def commit(self):\n        \"\"\"\n        Issues a commit operation to scalaris validating the previously\n        created operations inside the transaction.\n        \"\"\"\n        result = self.req_list(self.new_req_list().add_commit())[0]\n        self._process_result_commit(result)\n        # reset tlog (minor optimization which is not done in req_list):\n        self._tlog = None\n\n    def abort(self):\n        \"\"\"\n        Aborts all previously created operations inside the transaction.\n        \"\"\"\n        self._tlog = None\n\n    def read(self, key):\n        \"\"\"\n        Issues a read operation to scalaris, adds it to the current\n        transaction and returns the result.\n        Beware: lists of (small) integers may be (falsely) returned as a string -\n        use str_to_list() to convert such strings.\n        \"\"\"\n        result = self.req_list(self.new_req_list().add_read(key))[0]\n        return self.process_result_read(result)\n\n    def write(self, key, value):\n        \"\"\"\n        Issues a write operation to scalaris and adds it to the current\n        transaction.\n        \"\"\"\n        result = self.req_list(self.new_req_list().add_write(key, value))[0]\n        self.process_result_write(result)\n\n    def add_del_on_list(self, key, to_add, to_remove):\n        \"\"\"\n        Issues a add_del_on_list operation to scalaris and adds it to the\n        current transaction.\n        Changes the list stored at the given key, i.e. first adds all items in\n        to_add then removes all items in to_remove.\n        Both, to_add and to_remove, must be lists.\n        Assumes en empty list if no value exists at key.\n        \"\"\"\n        result = self.req_list(self.new_req_list().add_add_del_on_list(key, to_add, to_remove))[0]\n        self.process_result_add_del_on_list(result)\n\n    def add_on_nr(self, key, to_add):\n        \"\"\"\n        Issues a add_on_nr operation to scalaris and adds it to the\n        current transaction.\n        Changes the number stored at the given key, i.e. adds some value.\n        Assumes 0 if no value exists at key.\n        \"\"\"\n        result = self.req_list(self.new_req_list().add_add_on_nr(key, to_add))[0]\n        self.process_result_add_on_nr(result)\n\n    def test_and_set(self, key, old_value, new_value):\n        \"\"\"\n        Issues a test_and_set operation to scalaris and adds it to the\n        current transaction.\n        Atomic test and set, i.e. if the old value at key is old_value, then\n        write new_value.\n        \"\"\"\n        result = self.req_list(self.new_req_list().add_test_and_set(key, old_value, new_value))[0]\n        self.process_result_test_and_set(result)\n\n    def nop(self, value):\n        \"\"\"\n        No operation (may be used for measuring the JSON overhead).\n        \"\"\"\n        value = self._conn.encode_value(value)\n        result = self._conn.callp('/api/tx.yaws', 'nop', [value])\n        self._conn.process_result_nop(result)\n\n    def close_connection(self):\n        \"\"\"\n        Close the connection to scalaris\n        (it will automatically be re-opened on the next request).\n        \"\"\"\n        self._conn.close()\n\nclass _JSONReqList(object):\n    \"\"\"\n    Generic request list.\n    \"\"\"\n\n    def __init__(self, other = None):\n        \"\"\"\n        Create a new object using a JSON connection.\n        \"\"\"\n        self._requests = []\n        self._is_commit = False\n        if other is not None:\n            self.extend(other)\n\n    def add_read(self, key):\n        \"\"\"\n        Adds a read operation to the request list.\n        \"\"\"\n        if (self._is_commit):\n            raise RuntimeError(\"No further request supported after a commit!\")\n        self._requests.append({'read': key})\n        return self\n\n    def add_write(self, key, value):\n        \"\"\"\n        Adds a write operation to the request list.\n        \"\"\"\n        if (self._is_commit):\n            raise RuntimeError(\"No further request supported after a commit!\")\n        self._requests.append({'write': {key: JSONConnection.encode_value(value)}})\n        return self\n\n    def add_add_del_on_list(self, key, to_add, to_remove):\n        \"\"\"\n        Adds a add_del_on_list operation to the request list.\n        \"\"\"\n        if (self._is_commit):\n            raise RuntimeError(\"No further request supported after a commit!\")\n        self._requests.append({'add_del_on_list': {'key': key, 'add': to_add, 'del': to_remove}})\n        return self\n\n    def add_add_on_nr(self, key, to_add):\n        \"\"\"\n        Adds a add_on_nr operation to the request list.\n        \"\"\"\n        if (self._is_commit):\n            raise RuntimeError(\"No further request supported after a commit!\")\n        self._requests.append({'add_on_nr': {key: to_add}})\n        return self\n\n    def add_test_and_set(self, key, old_value, new_value):\n        \"\"\"\n        Adds a test_and_set operation to the request list.\n        \"\"\"\n        if (self._is_commit):\n            raise RuntimeError(\"No further request supported after a commit!\")\n        self._requests.append({'test_and_set': {'key': key, 'old': old_value, 'new': new_value}})\n        return self\n\n    def add_commit(self):\n        \"\"\"\n        Adds a commit operation to the request list.\n        \"\"\"\n        if (self._is_commit):\n            raise RuntimeError(\"Only one commit per request list allowed!\")\n        self._requests.append({'commit': ''})\n        self._is_commit = True\n        return self\n\n    def get_requests(self):\n        \"\"\"\n        Gets the collected requests.\n        \"\"\"\n        return self._requests\n\n    def is_commit(self):\n        \"\"\"\n        Returns whether the transactions contains a commit or not.\n        \"\"\"\n        return self._is_commit\n\n    def is_empty(self):\n        \"\"\"\n        Checks whether the request list is empty.\n        \"\"\"\n        return self._requests == []\n\n    def size(self):\n        \"\"\"\n        Gets the number of requests in the list.\n        \"\"\"\n        return len(self._requests)\n\n    def extend(self, other):\n        \"\"\"\n        Adds all requests of the other request list to the end of this list.\n        \"\"\"\n        self._requests.extend(other._requests)\n        return self\n\nclass _JSONReqListTransaction(_JSONReqList):\n    \"\"\"\n    Request list for use with Transaction.req_list().\n    \"\"\"\n\n    def __init__(self, other = None):\n        _JSONReqList.__init__(self, other)\n\nclass _JSONReqListTransactionSingleOp(_JSONReqList):\n    \"\"\"\n    Request list for use with TransactionSingleOp.req_list() which does not\n    support commits.\n    \"\"\"\n\n    def __init__(self, other = None):\n        _JSONReqList.__init__(self, other)\n\n    def add_commit(self):\n        \"\"\"\n        Adds a commit operation to the request list.\n        \"\"\"\n        raise RuntimeError(\"No commit allowed in TransactionSingleOp.req_list()!\")\n\nclass ReplicatedDHT(object):\n    \"\"\"\n    Non-transactional operations on the replicated DHT of scalaris\n    \"\"\"\n\n    def __init__(self, conn = None):\n        \"\"\"\n        Create a new object using the given connection.\n        \"\"\"\n        if conn is None:\n            conn = JSONConnection()\n        self._conn = conn\n\n    # returns the number of successfully deleted items\n    # use get_last_delete_result() to get more details\n    def delete(self, key, timeout = 2000):\n        \"\"\"\n        Tries to delete the value at the given key.\n\n        WARNING: This function can lead to inconsistent data (e.g. deleted items\n        can re-appear). Also when re-creating an item the version before the\n        delete can re-appear.\n        \"\"\"\n        result = self._conn.callp('/api/rdht.yaws', 'delete', [key, timeout])\n        (success, ok, results) = self._conn.process_result_delete(result)\n        self._lastDeleteResult = results\n        if success == True:\n            return ok\n        elif success == 'timeout':\n            raise TimeoutError(result)\n        else:\n            raise UnknownError(result)\n\n    def get_last_delete_result(self):\n        \"\"\"\n        Returns the result of the last call to delete().\n\n        NOTE: This function traverses the result list returned by scalaris and\n        therefore takes some time to process. It is advised to store the returned\n        result object once generated.\n        \"\"\"\n        return self._conn.create_delete_result(self._lastDeleteResult)\n\n    def nop(self, value):\n        \"\"\"\n        No operation (may be used for measuring the JSON overhead).\n        \"\"\"\n        value = self._conn.encode_value(value)\n        result = self._conn.callp('/api/rdht.yaws', 'nop', [value])\n        self._conn.process_result_nop(result)\n\n    def close_connection(self):\n        \"\"\"\n        Close the connection to scalaris\n        (it will automatically be re-opened on the next request).\n        \"\"\"\n        self._conn.close()\n\nclass RoutingTable(object):\n    \"\"\"\n    API for using routing tables\n    \"\"\"\n\n    def __init__(self, conn = None):\n        \"\"\"\n        Create a new object using the given connection.\n        \"\"\"\n        if conn is None:\n            conn = JSONConnection()\n        self._conn = conn\n\n    def get_replication_factor(self):\n      result = self._conn.callp('/api/rt.yaws', 'get_replication_factor', [])\n      if isinstance(result, dict) and 'status' in result and len(result) == 2 and result['status'] == 'ok' and 'value' in result:\n          return result['value']\n      else:\n        raise UnknownError(result)\n\nclass ScalarisVM(object):\n    \"\"\"\n    Provides methods to interact with a specific Scalaris (Erlang) VM.\n    \"\"\"\n\n    class GetInfoResult(object):\n        def __init__(self, scalarisVersion, erlangVersion, memTotal, uptime,\n                     erlangNode, ip, port, yawsPort):\n            self.scalarisVersion = scalarisVersion\n            self.erlangVersion = erlangVersion\n            self.memTotal = memTotal\n            self.uptime = uptime\n            self.erlangNode = erlangNode\n            self.ip = ip\n            self.port = port\n            self.yawsPort = yawsPort\n\n    def __init__(self, conn = None):\n        \"\"\"\n        Create a new object using the given connection.\n        \"\"\"\n        if conn is None:\n            conn = JSONConnection()\n        self._conn = conn\n\n    def getVersion(self):\n        \"\"\"\n        Gets the version of the Scalaris VM of the current connection.\n        \"\"\"\n        result = self._conn.callp('/api/vm.yaws', 'get_version', [])\n        return self._conn.process_result_vm_get_version(result)\n\n    def getInfo(self):\n        \"\"\"\n        Gets some information about the VM and Scalaris.\n        \"\"\"\n        result = self._conn.callp('/api/vm.yaws', 'get_info', [])\n        return self._conn.process_result_vm_get_info(result)\n\n    def getNumberOfNodes(self):\n        \"\"\"\n        Gets the number of nodes in the Scalaris VM of the current connection.\n        \"\"\"\n        result = self._conn.callp('/api/vm.yaws', 'number_of_nodes', [])\n        return self._conn.process_result_vm_get_number_of_nodes(result)\n\n    def getNodes(self):\n        \"\"\"\n        Gets the names of the nodes in the Scalaris VM of the current connection.\n        \"\"\"\n        result = self._conn.callp('/api/vm.yaws', 'get_nodes', [])\n        return self._conn.process_result_vm_get_nodes(result)\n\n    def addNodes(self, number):\n        \"\"\"\n        Adds Scalaris nodes to the Scalaris VM of the current connection.\n        \"\"\"\n        result = self._conn.callp('/api/vm.yaws', 'add_nodes', [number])\n        return self._conn.process_result_vm_add_nodes(result)\n\n    def shutdownNode(self, name):\n        \"\"\"\n        Shuts down the given node (graceful leave) in the Scalaris VM of the current connection.\n        \"\"\"\n        result = self._conn.callp('/api/vm.yaws', 'shutdown_node', [name])\n        return self._conn.process_result_vm_delete_node(result)\n\n    def killNode(self, name):\n        \"\"\"\n        Kills the given node in the Scalaris VM of the current connection.\n        \"\"\"\n        result = self._conn.callp('/api/vm.yaws', 'kill_node', [name])\n        return self._conn.process_result_vm_delete_node(result)\n\n    def shutdownNodes(self, number):\n        \"\"\"\n        Shuts down the given number of nodes (graceful leave) in the Scalaris VM of the current connection.\n        \"\"\"\n        result = self._conn.callp('/api/vm.yaws', 'shutdown_nodes', [number])\n        return self._conn.process_result_vm_delete_nodes(result)\n\n    def killNodes(self, number):\n        \"\"\"\n        Kills the given number of nodes in the Scalaris VM of the current connection.\n        \"\"\"\n        result = self._conn.callp('/api/vm.yaws', 'kill_nodes', [number])\n        return self._conn.process_result_vm_delete_nodes(result)\n\n    def shutdownNodesByName(self, names):\n        \"\"\"\n        Shuts down the given nodes (graceful leave) in the Scalaris VM of the current connection.\n        \"\"\"\n        result = self._conn.callp('/api/vm.yaws', 'shutdown_nodes_by_name', [names])\n        return self._conn.process_result_vm_delete_nodes(result)\n\n    def killNodesByName(self, names):\n        \"\"\"\n        Kills the given nodes in the Scalaris VM of the current connection.\n        \"\"\"\n        result = self._conn.callp('/api/vm.yaws', 'kill_nodes_by_name', [names])\n        return self._conn.process_result_vm_delete_nodes(result)\n\n    def getOtherVMs(self, maxVMs):\n        \"\"\"\n        Retrieves additional nodes from the Scalaris VM of the current\n        connection for use as URLs in JSONConnection.\n        \"\"\"\n        if maxVMs <= 0:\n            raise ValueError(\"max must be an integer > 0\")\n        result = self._conn.callp('/api/vm.yaws', 'get_other_vms', [maxVMs])\n        return self._conn.process_result_vm_get_other_vms(result)\n\n    def shutdownVM(self):\n        \"\"\"\n        Tells the Scalaris VM of the current connection to shut down gracefully.\n        \"\"\"\n        result = self._conn.callp('/api/vm.yaws', 'shutdown_vm', [])\n        return self._conn.process_result_vm_delete_vm(result)\n\n    def killVM(self):\n        \"\"\"\n        Kills the Scalaris VM of the current connection.\n        \"\"\"\n        result = self._conn.callp('/api/vm.yaws', 'kill_vm', [])\n        return self._conn.process_result_vm_delete_vm(result)\n\n    def nop(self, value):\n        \"\"\"\n        No operation (may be used for measuring the JSON overhead).\n        \"\"\"\n        value = self._conn.encode_value(value)\n        result = self._conn.callp('/api/vm.yaws', 'nop', [value])\n        self._conn.process_result_nop(result)\n\n    def close_connection(self):\n        \"\"\"\n        Close the connection to scalaris\n        (it will automatically be re-opened on the next request).\n        \"\"\"\n        self._conn.close()\n\nclass Autoscale(object):\n    \"\"\"\n    Provides methods to interact with autoscale API.\n    \"\"\"\n\n    api = '/api/autoscale.yaws'\n\n    \"\"\"\n    Create a new object using the given connection.\n    \"\"\"\n    def __init__(self, conn = None):\n        if conn is None:\n            conn = JSONConnection()\n        self._conn = conn\n\n    def process_result_check_config(self, result):\n        return self._conn.process_result_autoscale_check_config(result)\n\n    def process_result_pull_scale_req(self, result):\n        return self._conn.process_result_autoscale_pull_scale_req(result)\n\n    def process_result_lock_scale_req(self, result):\n        return self._conn.process_result_autoscale_lock_scale_req(result)\n\n    def process_result_unlock_scale_req(self, result):\n        return self._conn.process_result_autoscale_unlock_scale_req(result)\n\n    \"\"\" API calls \"\"\"\n    def check_config(self):\n        result = self._conn.callp(Autoscale.api, 'check_config', [])\n        return self.process_result_check_config(result)\n\n    def pull_scale_req(self):\n        result = self._conn.callp(Autoscale.api, 'pull_scale_req', [])\n        return self.process_result_pull_scale_req(result)\n\n    def lock_scale_req(self):\n        result = self._conn.callp(Autoscale.api, 'lock_scale_req', [])\n        return self.process_result_lock_scale_req(result)\n\n    def unlock_scale_req(self):\n        result = self._conn.callp(Autoscale.api, 'unlock_scale_req', [])\n        return self.process_result_unlock_scale_req(result)\n\n    def close_connection(self):\n        \"\"\"\n        Close the connection to scalaris\n        (it will automatically be re-opened on the next request).\n        \"\"\"\n        self._conn.close()\n\ndef str_to_list(value):\n    \"\"\"\n    Converts a string to a list of integers.\n    If the expected value of a read operation is a list, the returned value\n    could be (mistakenly) a string if it is a list of integers.\n    \"\"\"\n    if (isinstance(value, str) or isinstance(value, unicode)):\n        chars = list(value)\n        return [ord(char) for char in chars]\n    else:\n        return value\n"
  },
  {
    "path": "python-api/scalaris_bench.py",
    "content": "# Copyright 2011-2015 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nimport scalaris\nfrom datetime import datetime\nfrom threading import Thread\nimport time, threading\nimport random, string\nimport os, sys, traceback\n\n_BENCH_DATA_SIZE = 1000\n\"\"\"The size of a single data item that is send to scalaris.\"\"\"\n_benchTime = 0\n\"\"\"This is used to create different erlang keys for each run.\"\"\"\n_PERCENT_TO_REMOVE = 5\n\"\"\"Cut 5% off of both ends of the result list.\"\"\"\n_TESTRUNS = 1;\n\"\"\"Number of test runs (accumulates results over all test runs).\"\"\"\n\nif 'SCALARIS_JSON_URLS' in os.environ and os.environ['SCALARIS_JSON_URLS'] != '':\n    DEFAULT_URLS = os.environ['SCALARIS_JSON_URLS'].split()\nelse:\n    DEFAULT_URLS = [scalaris.DEFAULT_URL]\n\ndef minibench(operations, threads_per_node, benchmarks):\n    \"\"\"\n    Default minimal benchmark.\n    \n    Tests some strategies for writing key/value pairs to scalaris:\n    1) writing binary objects (random data, size = _BENCH_DATA_SIZE)\n    2) writing string objects (random data, size = _BENCH_DATA_SIZE)\n    each with the given number of consecutive operations and parallel\n    threads per Scalaris node,\n    * first using a new Transaction or TransactionSingleOp for each test,\n    * then using a new Transaction or TransactionSingleOp but re-using a single connection,\n    * and finally re-using a single Transaction or TransactionSingleOp object.\n    \"\"\"\n    # The time when the (whole) benchmark suite was started.\n    # This is used to create different erlang keys for each run.\n    _benchTime = _getCurrentMillis()\n    \n    parallel_runs = len(DEFAULT_URLS) * threads_per_node;\n    print 'Number of available nodes: ' + str(len(DEFAULT_URLS))\n    print '-> Using ' + str(parallel_runs) + ' parallel instances per test run...'\n    sys.stdout.flush()\n\n    print 'Benchmark of scalaris.TransactionSingleOp:'\n    sys.stdout.flush()\n    test_types = ['binary', 'string']\n    test_types_str = ['B', 'S']\n    columns = ['TransactionSingleOp.write(string, bytearray)',\n               'TransactionSingleOp.write(string, string)']\n    test_bench = [TransSingleOpBench1, TransSingleOpBench2, TransSingleOpBench3]\n    rows = ['separate connection', 're-use connection', 're-use object']\n    test_group = 'transsinglebench';\n    results = _getResultArray(rows, columns)\n    _runBenchAndPrintResults(benchmarks, results, columns, rows, test_types,\n            test_types_str, test_bench, test_group, 1, operations, parallel_runs)\n\n    print '-----'\n    print 'Benchmark of scalaris.Transaction:'\n    sys.stdout.flush()\n    test_types = ['binary', 'string']\n    test_types_str = ['B', 'S']\n    columns = ['Transaction.write(string, bytearray)',\n               'Transaction.write(string, string)']\n    test_bench = [TransBench1, TransBench2, TransBench3]\n    rows = ['separate connection', 're-use connection', 're-use object']\n    test_group = 'transbench';\n    results = _getResultArray(rows, columns)\n    _runBenchAndPrintResults(benchmarks, results, columns, rows, test_types,\n            test_types_str, test_bench, test_group, 1, operations, parallel_runs)\n\n    print '-----'\n    print 'Benchmark incrementing an integer key (read+write):'\n    sys.stdout.flush()\n    test_types = ['int']\n    test_types_str = ['I']\n    columns = ['Transaction.add_add_on_nr(string, int)']\n    test_bench = [TransIncrementBench1, TransIncrementBench2, TransIncrementBench3]\n    rows = ['separate connection', 're-use connection', 're-use object']\n    test_group = 'transbench_inc';\n    results = _getResultArray(rows, columns)\n    _runBenchAndPrintResults(benchmarks, results, columns, rows, test_types,\n            test_types_str, test_bench, test_group, 7, operations, parallel_runs)\n\n    print '-----'\n    print 'Benchmark read 5 + write 5:'\n    sys.stdout.flush()\n    test_types = ['binary', 'string']\n    test_types_str = ['B', 'S']\n    columns = ['Transaction.read(string) + Transaction.write(string, binary)',\n               'Transaction.read(string) + Transaction.write(string, string)']\n    test_bench = [TransRead5Write5Bench1, TransRead5Write5Bench2, TransRead5Write5Bench3]\n    rows = ['separate connection', 're-use connection', 're-use object']\n    test_group = 'transbench_r5w5';\n    results = _getResultArray(rows, columns)\n    _runBenchAndPrintResults(benchmarks, results, columns, rows, test_types,\n            test_types_str, test_bench, test_group, 10, operations, parallel_runs)\n\n    print '-----'\n    print 'Benchmark appending to a String list (read+write):'\n    sys.stdout.flush()\n    test_types = ['string']\n    test_types_str = ['S']\n    columns = ['Transaction.add_add_del_on_list(string, [string], [])']\n    test_bench = [TransAppendToListBench1, TransAppendToListBench2, TransAppendToListBench3]\n    rows = ['separate connection', 're-use connection', 're-use object']\n    test_group = 'transbench_append';\n    results = _getResultArray(rows, columns)\n    _runBenchAndPrintResults(benchmarks, results, columns, rows, test_types,\n            test_types_str, test_bench, test_group, 16, operations, parallel_runs)\n\nclass BenchRunnable(Thread):\n    \"\"\"\n    Abstract base class of a test run that is to be run in a thread.\n    \"\"\"\n    \n    def __init__(self, key, value, operations):\n        \"\"\"\n        Create a new runnable.\n        \"\"\"\n        Thread.__init__(self)\n        self._key = key\n        self._value = value\n        self._operations = operations\n        \n        self._shouldStop = False\n        self._timeAtStart = 0\n        self._speed = -1\n\n    def _testBegin(self):\n        \"\"\"\n         Call this method when a benchmark is started.\n         Sets the time the benchmark was started.\n        \"\"\"\n        self._timeAtStart = _getCurrentMillis()\n\n    def _testEnd(self, testRuns):\n        \"\"\"\n        Call this method when a benchmark is finished.\n        Calculates the time the benchmark took and the number of transactions\n        performed during this time.\n         \"\"\"\n        timeTaken = _getCurrentMillis() - self._timeAtStart\n        speed = (testRuns * 1000) / timeTaken\n        return speed\n    \n    def pre_init(self, j = None):\n        \"\"\"\n        Will be called before the benchmark starts with all possible\n        variations of \"j\" in the operation() call.\n        \"j\" with None is the overall initialisation run at first.\n        \"\"\"\n        pass\n    \n    def init(self):\n        \"\"\"\n        Will be called at the start of the benchmark.\n        \"\"\"\n        pass\n    \n    def cleanup(self):\n        \"\"\"\n        Will be called before the end of the benchmark.\n        \"\"\"\n        pass\n    \n    def operation(self, j):\n        \"\"\"\n        The operation to execute during the benchmark.\n        \"\"\"\n        pass\n    \n    def run(self):\n        threading.currentThread().name = \"BenchRunnable-\" + self._key\n        retry = 0\n        while (retry < 3) and (not self._shouldStop):\n            try:\n                self.pre_init()\n                for j in xrange(self._operations):\n                    self.pre_init(j)\n                self._testBegin()\n                self.init()\n                for j in xrange(self._operations):\n                    self.operation(j)\n                self.cleanup()\n                self._speed = self._testEnd(self._operations)\n                break\n            except:\n                # _printException()\n                pass\n            retry += 1\n\n    def getSpeed(self):\n        return self._speed\n\n    def shouldStop(self):\n        self._shouldStop = True\n\nclass BenchRunnable2(BenchRunnable):\n    def __init__(self, key, value, operations):\n        \"\"\"\n        Create a new runnable.\n        \"\"\"\n        BenchRunnable.__init__(self, key, value, operations)\n    \n    def init(self):\n        \"\"\"\n        Will be called at the start of the benchmark.\n        \"\"\"\n        self._connection = _getConnection()\n    \n    def cleanup(self):\n        \"\"\"\n        Will be called before the end of the benchmark.\n        \"\"\"\n        self._connection.close()\n\nclass TransSingleOpBench1(BenchRunnable):\n    \"\"\"\n    Performs a benchmark writing objects using a new TransactionSingleOp object\n    for each test.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        BenchRunnable.__init__(self, key, value, operations)\n    \n    def operation(self, j):\n        tx = scalaris.TransactionSingleOp(conn = _getConnection())\n        tx.write(self._key + '_' + str(j), self._value)\n        tx.close_connection()\n\nclass TransSingleOpBench2(BenchRunnable2):\n    \"\"\"\n    Performs a benchmark writing objects using a new TransactionSingleOp but\n    re-using a single connection for each test.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        BenchRunnable2.__init__(self, key, value, operations)\n\n    def operation(self, j):\n        tx = scalaris.TransactionSingleOp(conn = self._connection)\n        tx.write(self._key + '_' + str(j), self._value)\n\nclass TransSingleOpBench3(BenchRunnable):\n    \"\"\"\n    Performs a benchmark writing objects using a single TransactionSingleOp\n    object for all tests.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        BenchRunnable.__init__(self, key, value, operations)\n\n    def init(self):\n        self._tx = scalaris.TransactionSingleOp(conn = _getConnection())\n\n    def cleanup(self):\n        self._tx.close_connection()\n\n    def operation(self, j):\n        self._tx.write(self._key + '_' + str(j), self._value)\n\nclass TransBench1(BenchRunnable):\n    \"\"\"\n    Performs a benchmark writing objects using a new Transaction for each test.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        BenchRunnable.__init__(self, key, value, operations)\n\n    def operation(self, j):\n        tx = scalaris.Transaction(conn = _getConnection())\n        tx.write(self._key + '_' + str(j), self._value)\n        tx.commit()\n        tx.close_connection()\n\nclass TransBench2(BenchRunnable2):\n    \"\"\"\n    Performs a benchmark writing objects using a new Transaction but re-using a\n    single connection for each test.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        BenchRunnable2.__init__(self, key, value, operations)\n\n    def operation(self, j):\n        tx = scalaris.Transaction(conn = self._connection)\n        tx.write(self._key + '_' + str(j), self._value)\n        tx.commit()\n\nclass TransBench3(BenchRunnable):\n    \"\"\"\n    Performs a benchmark writing objects using a single Transaction object\n    for all tests.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        BenchRunnable.__init__(self, key, value, operations)\n\n    def init(self):\n        self._tx = scalaris.Transaction(conn = _getConnection())\n\n    def cleanup(self):\n        self._tx.close_connection()\n\n    def operation(self, j):\n        self._tx.write(self._key + '_' + str(j), self._value)\n        self._tx.commit()\n\nclass TransIncrementBench(BenchRunnable):\n    \"\"\"\n    Performs a benchmark writing integer numbers on a single key and\n    increasing them.\n    Provides convenience methods for the full increment benchmark\n    implementations.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        BenchRunnable.__init__(self, key, value, operations)\n    \n    def pre_init(self, j = None):\n        tx_init = scalaris.Transaction(conn = _getConnection())\n        tx_init.write(self._key, 0)\n        tx_init.commit()\n        tx_init.close_connection()\n\n    def operation2(self, tx, j):\n        reqs = tx.new_req_list()\n        reqs.add_add_on_nr(self._key, 1).add_commit()\n        # value_old = tx.read(self._key)\n        # reqs.add_write(key, value_old + 1).add_commit()\n        results = tx.req_list(reqs)\n        # tx.process_result_write(results[0])\n        tx.process_result_add_on_nr(results[0])\n\nclass TransIncrementBench1(TransIncrementBench):\n    \"\"\"\n    Performs a benchmark writing integer numbers on a single key and\n    increasing them using a new Transaction for each test.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        TransIncrementBench.__init__(self, key, value, operations)\n\n    def operation(self, j):\n        tx = scalaris.Transaction(conn = _getConnection())\n        self.operation2(tx, j)\n        tx.close_connection()\n\nclass TransIncrementBench2(TransIncrementBench):\n    \"\"\"\n    Performs a benchmark writing integer numbers on a single key and\n    increasing them using a new Transaction but re-using a single\n    connection for each test.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        TransIncrementBench.__init__(self, key, value, operations)\n    \n    def init(self):\n        self._connection = _getConnection()\n    \n    def cleanup(self):\n        self._connection.close()\n\n    def operation(self, j):\n        tx = scalaris.Transaction(conn = self._connection)\n        self.operation2(tx, j)\n\nclass TransIncrementBench3(TransIncrementBench):\n    \"\"\"\n    Performs a benchmark writing objects using a single Transaction\n    object for all tests.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        TransIncrementBench.__init__(self, key, value, operations)\n    \n    def init(self):\n        self._tx = scalaris.Transaction(conn = _getConnection())\n    \n    def cleanup(self):\n        self._tx.close_connection()\n\n    def operation(self, j):\n        self.operation2(self._tx, j)\n\nclass TransReadXWriteXBench(BenchRunnable):\n    \"\"\"\n    Performs a benchmark reading X values and overwriting them afterwards\n    inside a transaction.\n    Provides convenience methods for the full read-x, write-x benchmark\n    implementations.\n    \"\"\"\n    def __init__(self, key, value, nr_keys, operations):\n        BenchRunnable.__init__(self, key, value, operations)\n        self._keys = []\n        self._value_write = []\n        for i in xrange(nr_keys):\n            self._keys.append(key + \"_\" + str(i))\n            self._value_write.append(_getRandom(_BENCH_DATA_SIZE, type(value).__name__))\n    \n    def pre_init(self, j = None):\n        value_init = []\n        for i in xrange(len(self._keys)):\n            value_init.append(_getRandom(_BENCH_DATA_SIZE, type(self._value).__name__))\n        \n        tx_init = scalaris.Transaction(conn = _getConnection())\n        reqs = tx_init.new_req_list()\n        for i in xrange(len(self._keys)):\n            reqs.add_write(self._keys[i], value_init[i])\n        reqs.add_commit()\n        results = tx_init.req_list(reqs)\n        for i in xrange(len(self._keys)):\n            tx_init.process_result_write(results[i])\n        tx_init.close_connection()\n\n    def operation2(self, tx, j):\n        reqs = tx.new_req_list()\n        # read old values into the transaction\n        for i in xrange(len(self._keys)):\n            reqs.add_read(self._keys[i])\n        reqs.add_commit()\n        results = tx.req_list(reqs)\n        for i in xrange(len(self._keys)):\n            tx.process_result_read(results[i])\n        \n        # write new values... \n        reqs = tx.new_req_list()\n        for i in xrange(len(self._keys)):\n            value = self._value_write[j % len(self._value_write)]\n            reqs.add_write(self._keys[i], value)\n        reqs.add_commit()\n        results = tx.req_list(reqs)\n        for i in xrange(len(self._keys)):\n            tx.process_result_write(results[i])\n\nclass TransRead5Write5Bench1(TransReadXWriteXBench):\n    \"\"\"\n    Performs a benchmark reading 5 values and overwriting them afterwards\n    inside a transaction using a new Transaction for each test.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        TransReadXWriteXBench.__init__(self, key, value, 5, operations)\n\n    def operation(self, j):\n        tx = scalaris.Transaction(conn = _getConnection())\n        self.operation2(tx, j)\n        tx.close_connection()\n\nclass TransRead5Write5Bench2(TransReadXWriteXBench):\n    \"\"\"\n    Performs a benchmark reading 5 values and overwriting them afterwards\n    inside a transaction using a new Transaction but re-using a single\n    connection for each test.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        TransReadXWriteXBench.__init__(self, key, value, 5, operations)\n    \n    def init(self):\n        self._connection = _getConnection()\n    \n    def cleanup(self):\n        self._connection.close()\n\n    def operation(self, j):\n        tx = scalaris.Transaction(conn = self._connection)\n        self.operation2(tx, j)\n\nclass TransRead5Write5Bench3(TransReadXWriteXBench):\n    \"\"\"\n    Performs a benchmark reading 5 values and overwriting them afterwards\n    inside a transaction using a single Transaction object for all tests.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        TransReadXWriteXBench.__init__(self, key, value, 5, operations)\n    \n    def init(self):\n        self._tx = scalaris.Transaction(conn = _getConnection())\n    \n    def cleanup(self):\n        self._tx.close_connection()\n\n    def operation(self, j):\n        self.operation2(self._tx, j)\n\nclass TransAppendToListBench(BenchRunnable):\n    \"\"\"\n    Performs a benchmark adding values to a list inside a transaction.\n    Provides convenience methods for the full append-to-list benchmark\n    implementations.\n    \"\"\"\n    def __init__(self, key, value, nr_keys, operations):\n        BenchRunnable.__init__(self, key, value, operations)\n        self._value_init = []\n        for _i in xrange(nr_keys):\n            self._value_init.append(_getRandom(_BENCH_DATA_SIZE, 'string'))\n    \n    def pre_init(self, j = None):\n        if j is None:\n            return\n        tx_init = scalaris.Transaction(conn = _getConnection())\n        reqs = tx_init.new_req_list()\n        reqs.add_write(self._key + '_' + str(j), self._value_init).add_commit()\n        results = tx_init.req_list(reqs)\n        tx_init.process_result_write(results[0])\n        tx_init.close_connection()\n\n    def operation2(self, tx, j):\n        reqs = tx.new_req_list()\n        reqs.add_add_del_on_list(self._key + '_' + str(j), [self._value], []).add_commit()\n        # read old list into the transaction\n        # list = scalaris.str_to_list(tx.read(self._key + '_' + str(j)))\n        # write new list ...\n        # list.append(self._value)\n        # reqs.add_write(self._key + '_' + str(j), list).add_commit())\n        \n        results = tx.req_list(reqs)\n        # tx.process_result_write(results[0])\n        tx.process_result_add_del_on_list(results[0])\n\nclass TransAppendToListBench1(TransAppendToListBench):\n    \"\"\"\n    Performs a benchmark adding values to a list inside a transaction\n    using a new Transaction for each test.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        TransAppendToListBench.__init__(self, key, value, 5, operations)\n\n    def operation(self, j):\n        tx = scalaris.Transaction(conn = _getConnection())\n        self.operation2(tx, j)\n        tx.close_connection()\n\nclass TransAppendToListBench2(TransAppendToListBench):\n    \"\"\"\n    Performs a benchmark adding values to a list inside a transaction using a\n    new Transaction but re-using a single connection for each test.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        TransAppendToListBench.__init__(self, key, value, 5, operations)\n    \n    def init(self):\n        self._connection = _getConnection()\n    \n    def cleanup(self):\n        self._connection.close()\n\n    def operation(self, j):\n        tx = scalaris.Transaction(conn = self._connection)\n        self.operation2(tx, j)\n\nclass TransAppendToListBench3(TransAppendToListBench):\n    \"\"\"\n    Performs a benchmark adding values to a list inside a transaction using a\n    single Transaction object for all tests.\n    \"\"\"\n    def __init__(self, key, value, operations):\n        TransAppendToListBench.__init__(self, key, value, 5, operations)\n    \n    def init(self):\n        self._tx = scalaris.Transaction(conn = _getConnection())\n    \n    def cleanup(self):\n        self._tx.close_connection()\n\n    def operation(self, j):\n        self.operation2(self._tx, j)\n\ndef _getCurrentMillis():\n    \"\"\"\n    Gets the number of milliseconds since epoch.\n    \"\"\"\n    now = datetime.now()\n    return int(time.mktime(now.timetuple())) * 1000 + (now.microsecond // 1000)\n\ndef _testBegin():\n    \"\"\"\n    Call this method when a benchmark is started.\n    Sets the time the benchmark was started.\n    \"\"\"\n    global _timeAtStart\n    _timeAtStart = _getCurrentMillis()\n\ndef _testEnd(testruns):\n    \"\"\"\n    Call this method when a benchmark is finished.\n    Calculates the time the benchmark took and the number of transactions\n    performed during this time.\n    Returns the number of achieved transactions per second.\n    \"\"\"\n    global _timeAtStart\n    timeTaken = _getCurrentMillis() - _timeAtStart\n    speed = (testruns * 1000) // timeTaken\n    return speed\n\ndef _getConnection():\n    url = random.choice(DEFAULT_URLS)\n    return scalaris.JSONConnection(url = url)\n\ndef _getResultArray(rows, columns):\n    \"\"\"\n    Returns a pre-initialized results array with values <tt>-1</tt>.\n    \"\"\"\n    results = {}\n    for row in rows:\n        results[row] = {}\n        for column in columns:\n            results[row][column] = -1\n    return results\n\ndef _getRandom(size, mytype):\n    \"\"\"\n    Creates an random string or binary object from <size> random characters/bytes.\n    \"\"\"\n    if mytype == 'int':\n        return random.randint(0, 2147483647)\n    elif mytype == 'string' or mytype == 'str':\n        return ''.join(random.choice(string.printable) for _x in xrange(size))\n    elif mytype == 'binary':\n        return bytearray(random.randrange(0, 256) for _x in xrange(size))\n\ndef _integrateResults(results, i, worker, failed):\n    \"\"\"\n    Integrates the workers' results into the result array.\n    \"\"\"\n    try:\n        for bench_thread in worker:\n            if failed >= 3:\n                bench_thread.shouldStop()\n                try:\n                    while(bench_thread.isAlive()): # non-blocking join so we are able to receive CTRL-C\n                        bench_thread.join(1)\n                except RuntimeError:\n                    pass\n            else:\n                try:\n                    while(bench_thread.isAlive()): # non-blocking join so we are able to receive CTRL-C\n                        bench_thread.join(1)\n                    speed = bench_thread.getSpeed()\n                except RuntimeError:\n                    speed = -1\n                \n                if speed < 0:\n                    failed += 1\n                else:\n                    results[i] += speed\n        return failed\n    except KeyboardInterrupt:\n        print 'CTRL-C received, aborting...'\n        for bench_thread in worker:\n            bench_thread.shouldStop()\n        sys.exit(1)\n    \ndef _getAvgSpeed(results):\n    \"\"\"\n    Calculates the average number of transactions per second from the results\n    of executing 10 transactions per test run. Will remove the top and bottom\n    _PERCENT_TO_REMOVE percent of the sorted results array.\n    Returns the average number of transactions per second.\n    \"\"\"\n    results.sort()\n    toRemove = int((len(results) * _PERCENT_TO_REMOVE) // 100);\n    avgSpeed = 0;\n    for i in xrange(toRemove, (len(results) - toRemove)):\n        avgSpeed += results[i]\n    \n    avgSpeed //= len(results) - 2 * toRemove\n    return avgSpeed\n\ndef _runBenchAndPrintResults(benchmarks, results, columns, rows, test_types,\n                             test_types_str, test_bench, test_group,\n                             first_bench_id, operations, parallel_runs):\n    \"\"\"\n    Runs the given benchmarks and prints a results table.\n    \"\"\"\n    # assume non-empty results dict:\n    for test in xrange(len(results) * len(results[list(results.keys())[0]])):\n        try:\n            i = test % len(results);\n            j = test // len(results);\n            if (test + first_bench_id) in benchmarks:\n                results[rows[i]][columns[j]] = _runBench(operations,\n                        _getRandom(_BENCH_DATA_SIZE, test_types[j]),\n                        test_group + \"_\" + test_types_str[j] + \"_\" + str(i + 1),\n                        test_bench[i], parallel_runs)\n                time.sleep(1)\n            else:\n                results[rows[i]][columns[j]] = -2\n        except Exception: # do not catch SystemExit\n            _printException()\n    \n    _printResults(columns, rows, results, operations, parallel_runs)\n\ndef _runBench(operations, value, name, clazz, parallel_runs):\n    \"\"\"\n    Runs the given benchmark.\n    \"\"\"\n    key = str(_benchTime) + name\n    results = [-1]*_TESTRUNS\n\n    for i in xrange(_TESTRUNS):\n        worker = []\n        for thread in xrange(parallel_runs):\n            new_worker = clazz(key + '_' + str(i) + '_' + str(thread), value, operations)\n            worker.append(new_worker) \n            new_worker.start()\n        failed = 0\n        failed = _integrateResults(results, i, worker, failed);\n        if failed >= 3:\n            return -1\n\n    return _getAvgSpeed(results)\n\ndef _printResults(columns, rows, results, operations, parallel_runs):\n    \"\"\"\n    Prints a result table.\n    \"\"\"\n    print 'Concurrent threads: ' + str(parallel_runs) + ', each using ' + str(operations) + ' transactions'\n    colLen = 25\n    emptyFirstColumn = ''.join([' ']*colLen)\n    print emptyFirstColumn + '\\tspeed (transactions / second)'\n    print emptyFirstColumn, \n    i = 1\n    for column in columns:\n        print '\\t(' + str(i) + ')', \n        i += 1\n    print ''\n    for row in rows:\n        print row + ''.join([' ']*(colLen - len(row))),\n        for column in columns:\n            value = results[row][column]\n            if (value == -2):\n                print '\\tn/a', \n            elif (value == -1):\n                print '\\tfailed', \n            else:\n                print '\\t' + str(int(value)),\n        print ''\n    \n    i = 1\n    for column in columns:\n        print '(' + str(i) + ') ' + column \n        i += 1\n    sys.stdout.flush()\n\ndef _printException():\n    mytype, message, trace = sys.exc_info()\n    print str(mytype) + str(message)\n    traceback.print_tb(trace)\n\ndef run_from_cmd(argv):\n    nr_operations = 500\n    threads_per_node = 10\n    allBenchs = False\n    if (len(argv) == 1):\n        allBenchs = True\n    elif (len(argv) == 2):\n        allBenchs = True\n        nr_operations = int(argv[1])\n    elif (len(argv) == 3):\n        allBenchs = True\n        nr_operations = int(argv[1])\n        threads_per_node = int(argv[2])\n    elif (len(argv) >= 4):\n        nr_operations = int(argv[1])\n        threads_per_node = int(argv[2])\n        benchmarks = []\n        for i in xrange(3, len(argv)):\n            if argv[i] == 'all':\n                allBenchs = True\n            else:\n                benchmarks.append(int(argv[i]))\n    if allBenchs:\n        benchmarks = xrange(1, 19, 1)\n    minibench(nr_operations, threads_per_node, benchmarks)\n\nif __name__ == \"__main__\":\n    run_from_cmd(sys.argv)\n"
  },
  {
    "path": "python-api/scalaris_client.py",
    "content": "# Copyright 2011-2015 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nfrom scalaris import TransactionSingleOp, ReplicatedDHT, RoutingTable\nfrom scalaris import ConnectionError, TimeoutError, NotFoundError, AbortError, KeyChangedError, UnknownError\nimport scalaris_bench\nimport sys\n\nif __name__ == \"__main__\":\n    if (len(sys.argv) == 3 and sys.argv[1] in [\"--read\", \"-r\"]):\n        sc = TransactionSingleOp()\n        key = sys.argv[2]\n        try:\n            value = sc.read(key)\n            print 'read(' + key + ') = ' + repr(value)\n        except ConnectionError as instance:\n            print 'read(' + key + ') failed with connection error'\n            sys.exit(1)\n        except TimeoutError as instance:\n            print 'read(' + key + ') failed with timeout'\n            sys.exit(1)\n        except NotFoundError as instance:\n            print 'read(' + key + ') failed with not_found'\n            sys.exit(1)\n        except UnknownError as instance:\n            print 'read(' + key + ') failed with unknown: ' + str(instance)\n            sys.exit(1)\n    elif (len(sys.argv) == 4 and sys.argv[1] in [\"--write\", \"-w\"]):\n        sc = TransactionSingleOp()\n        key = sys.argv[2]\n        value = sys.argv[3]\n        try:\n            sc.write(key, value)\n            print 'write(' + key + ', ' + value + '): ok'\n        except ConnectionError as instance:\n            print 'write(' + key + ', ' + value + ') failed with connection error'\n            sys.exit(1)\n        except TimeoutError as instance:\n            print 'write(' + key + ', ' + value + ') failed with timeout'\n            sys.exit(1)\n        except AbortError as instance:\n            print 'write(' + key + ', ' + value + ') failed with abort'\n            sys.exit(1)\n        except UnknownError as instance:\n            print 'write(' + key + ', ' + value + ') failed with unknown: ' + str(instance)\n            sys.exit(1)\n    elif (len(sys.argv) == 5 and sys.argv[1] == \"--test-and-set\"):\n        sc = TransactionSingleOp()\n        key = sys.argv[2]\n        old_value = sys.argv[3]\n        new_value = sys.argv[4]\n        try:\n            sc.test_and_set(key, old_value, new_value)\n            print 'test_and_set(' + key + ', ' + old_value + ', ' + new_value + '): ok'\n        except ConnectionError as instance:\n            print 'test_and_set(' + key + ', ' + old_value + ', ' + new_value + ') failed with connection error'\n            sys.exit(1)\n        except TimeoutError as instance:\n            print 'test_and_set(' + key + ', ' + old_value + ', ' + new_value + ') failed with timeout'\n            sys.exit(1)\n        except AbortError as instance:\n            print 'test_and_set(' + key + ', ' + old_value + ', ' + new_value + ') failed with abort'\n            sys.exit(1)\n        except KeyChangedError as instance:\n            print 'test_and_set(' + key + ', ' + old_value + ', ' + new_value + ') failed with key_change (current old value: ' + instance.old_value + ')'\n            sys.exit(1)\n        except UnknownError as instance:\n            print 'test_and_set(' + key + ', ' + old_value + ', ' + new_value + ') failed with unknown: ' + str(instance)\n            sys.exit(1)\n    elif (len(sys.argv) == 3 and sys.argv[1] in [\"--delete\", \"-d\"]):\n        rdht = ReplicatedDHT()\n        key = sys.argv[2]\n        if len(sys.argv) >= 4:\n            timeout = sys.argv[3]\n        else:\n            timeout = 2000\n        \n        try:\n            ok = rdht.delete(key)\n            results = rdht.get_last_delete_result()\n            print 'delete(' + key + ', ' + str(timeout) + '): ok, deleted: ' + str(ok) + ' (' + repr(results) + ')'\n        except TimeoutError as instance:\n            results = rdht.get_last_delete_result()\n            print 'delete(' + key + ', ' + str(timeout) + '): failed (timeout), deleted: ' + str(ok) + ' (' + repr(results) + ')'\n        except UnknownError as instance:\n            print 'delete(' + key + ') failed with unknown: ' + str(instance)\n            sys.exit(1)\n    elif (len(sys.argv) >= 2 and sys.argv[1] in [\"--minibench\", \"-b\"]):\n        scalaris_bench.run_from_cmd(sys.argv[:1] + sys.argv[2:])\n    elif (len(sys.argv) >= 2 and sys.argv[1] in [\"--get-replication-factor\", \"-f\"]):\n        print RoutingTable().get_replication_factor()\n    else:\n        print 'usage: ' + sys.argv[0] + ' [Options]'\n        print ' -r,--read <key>'\n        print '                            read an item'\n        print ' -w,--write <key> <value>'\n        print '                            write an item'\n        print ' --test-and-set <key> <old_value> <new_value>'\n        print '                            atomic test and set, i.e. write <key> to'\n        print '                            <new_value> if the current value is <old_value>'\n        print ' -d,--delete <key> [<timeout>]'\n        print '                            delete an item (default timeout: 2000ms)'\n        print '                            WARNING: This function can lead to inconsistent'\n        print '                            data (e.g. deleted items can re-appear).'\n        print '                            Also if an item is re-created, the version'\n        print '                            before the delete can re-appear.'\n        print ' -h,--help'\n        print '                            print this message'\n        print ' -b,--minibench [<ops> [<threads_per_node> [<benchmarks>]]]'\n        print '                            run selected mini benchmark(s)'\n        print '                            [1|...|9|all] (default: all benchmarks, 500'\n        print '                            operations each, 10 threads per Scalaris node)'\n        print ' -f, --get-replication-factor'\n        print '                            print the replication factor'\n        if len(sys.argv) == 1 or (len(sys.argv) >= 2 and not sys.argv[1] in [\"--help\", \"-h\"]):\n            sys.exit(1)\n"
  },
  {
    "path": "python-api/scalaris_interop_test.py",
    "content": "# Copyright 2011-2015 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nimport scalaris\nimport sys\n\ndef read_or_write(sc, key, value, mode):\n    try:\n        if (mode == 'read'):\n            actual = sc.read(key)\n            # if the expected value is a list, the returned value could by (mistakenly) a string if it is a list of integers\n            # -> convert such a string to a list\n            if (type(value).__name__=='list'):\n                try:\n                    actual = scalaris.str_to_list(actual)\n                except:\n                    pass # will fail the comparison anyway\n            if (actual == value):\n                out = sys.stdout\n                result = 0\n                result_str = 'ok'\n            else:\n                out = sys.stderr\n                result = 1\n                result_str = 'fail'\n\n            print >> out, 'read(' + key + ')'\n            print >> out, '  expected: ' + repr(value)\n            print >> out, '  read raw: ' + repr(actual)\n            print >> out, '   read py: ' + repr(actual)\n            print >> out, result_str\n            return result\n        elif (mode == 'write'):\n            print 'write(' + key + ', ' + repr(value) + ')'\n            sc.write(key, value)\n            return 0\n    except scalaris.ConnectionError as instance:\n        print 'failed with connection error'\n        return 1\n    except scalaris.TimeoutError as instance:\n        print 'failed with timeout'\n        return 1\n    except scalaris.AbortError as instance:\n        print 'failed with abort'\n        return 1\n    except scalaris.NotFoundError as instance:\n        print 'failed with not_found'\n        return 1\n    except scalaris.UnknownError as instance:\n        print 'failed with unknown: ' + str(instance)\n        return 1\n    except Exception as instance:\n        print 'failed with ' + str(instance)\n        return 1\n\ndef read_write_boolean(basekey, sc, mode):\n    failed = 0\n    \n    failed += read_or_write(sc, basekey + \"_bool_false\", False, mode)\n    failed += read_or_write(sc, basekey + \"_bool_true\", True, mode)\n    \n    return failed\n\ndef read_write_integer(basekey, sc, mode):\n    failed = 0\n    \n    failed += read_or_write(sc, basekey + \"_int_0\", 0, mode)\n    failed += read_or_write(sc, basekey + \"_int_1\", 1, mode)\n    failed += read_or_write(sc, basekey + \"_int_min\", -2147483648, mode)\n    failed += read_or_write(sc, basekey + \"_int_max\", 2147483647, mode)\n    failed += read_or_write(sc, basekey + \"_int_max_div_2\", 2147483647 // 2, mode)\n    \n    return failed\n\ndef read_write_long(basekey, sc, mode):\n    failed = 0\n    \n    failed += read_or_write(sc, basekey + \"_long_0\", 0l, mode)\n    failed += read_or_write(sc, basekey + \"_long_1\", 1l, mode)\n    failed += read_or_write(sc, basekey + \"_long_min\", -9223372036854775808l, mode)\n    failed += read_or_write(sc, basekey + \"_long_max\", 9223372036854775807l, mode)\n    failed += read_or_write(sc, basekey + \"_long_max_div_2\", 9223372036854775807l // 2l, mode)\n    \n    return failed\n\ndef read_write_biginteger(basekey, sc, mode):\n    failed = 0\n    \n    failed += read_or_write(sc, basekey + \"_bigint_0\", 0, mode)\n    failed += read_or_write(sc, basekey + \"_bigint_1\", 1, mode)\n    failed += read_or_write(sc, basekey + \"_bigint_min\", -100000000000000000000, mode)\n    failed += read_or_write(sc, basekey + \"_bigint_max\", 100000000000000000000, mode)\n    failed += read_or_write(sc, basekey + \"_bigint_max_div_2\", 100000000000000000000 // 2, mode)\n    \n    return failed\n\ndef read_write_double(basekey, sc, mode):\n    failed = 0\n    \n    failed += read_or_write(sc, basekey + \"_float_0.0\", 0.0, mode)\n    failed += read_or_write(sc, basekey + \"_float_1.5\", 1.5, mode)\n    failed += read_or_write(sc, basekey + \"_float_-1.5\", -1.5, mode)\n    failed += read_or_write(sc, basekey + \"_float_min\", 4.9E-324, mode)\n    failed += read_or_write(sc, basekey + \"_float_max\", 1.7976931348623157E308, mode)\n    failed += read_or_write(sc, basekey + \"_float_max_div_2\", 1.7976931348623157E308 / 2.0, mode)\n    \n    # not supported by erlang:\n    #failed += read_or_write(sc, basekey + \"_float_neg_inf\", float('-inf'), mode)\n    #failed += read_or_write(sc, basekey + \"__float_pos_inf\", float('+inf'), mode)\n    #failed += read_or_write(sc, basekey + \"_float_nan\", float('nan'), mode)\n    \n    return failed\n\ndef read_write_string(basekey, sc, mode):\n    failed = 0\n    \n    failed += read_or_write(sc, basekey + \"_string_empty\", '', mode)\n    failed += read_or_write(sc, basekey + \"_string_foobar\", 'foobar', mode)\n    failed += read_or_write(sc, basekey + \"_string_foo\\\\nbar\", 'foo\\nbar', mode)\n    failed += read_or_write(sc, basekey + \"_string_unicode\", u'foo\\u0180\\u01E3\\u11E5', mode)\n    \n    return failed\n\ndef read_write_binary(basekey, sc, mode):\n    failed = 0\n    \n    # note: binary not supported by JSON\n    failed += read_or_write(sc, basekey + \"_byte_empty\", bytearray(), mode)\n    failed += read_or_write(sc, basekey + \"_byte_0\", bytearray([0]), mode)\n    failed += read_or_write(sc, basekey + \"_byte_0123\", bytearray([0, 1, 2, 3]), mode)\n    \n    return failed\n\ndef read_write_list(basekey, sc, mode):\n    failed = 0\n    \n    failed += read_or_write(sc, basekey + \"_list_empty\", [], mode)\n    failed += read_or_write(sc, basekey + \"_list_0_1_2_3\", [0, 1, 2, 3], mode)\n    failed += read_or_write(sc, basekey + \"_list_0_123_456_65000\", [0, 123, 456, 65000], mode)\n    failed += read_or_write(sc, basekey + \"_list_0_123_456_0x10ffff\", [0, 123, 456, 0x10ffff], mode)\n    list4 = [0, 'foo', 1.5, False]\n    failed += read_or_write(sc, basekey + \"_list_0_foo_1.5_false\", list4, mode)\n    # note: binary not supported in lists\n    #failed += read_or_write(sc, basekey + \"_list_0_foo_1.5_<<0123>>\", [0, 'foo', 1.5, bytearray([0, 1, 2, 3])], mode)\n    failed += read_or_write(sc, basekey + \"_list_0_foo_1.5_false_list4\", [0, 'foo', 1.5, False, list4], mode)\n    failed += read_or_write(sc, basekey + \"_list_0_foo_1.5_false_list4_map_x=0_y=1\", [0, 'foo', 1.5, False, list4, {'x': 0, 'y': 1}], mode)\n    \n    return failed\n\ndef read_write_map(basekey, sc, mode):\n    failed = 0\n    \n    failed += read_or_write(sc, basekey + \"_map_empty\", {}, mode)\n    failed += read_or_write(sc, basekey + \"_map_x=0_y=1\", {'x': 0, 'y': 1}, mode)\n    failed += read_or_write(sc, basekey + \"_map_a=0_b=foo_c=1.5_d=foo<nl>bar_e=list0123_f=mapx0y1\",\n                            {'a': 0, 'b': 'foo', 'c': 1.5, 'd': 'foo\\nbar', 'e': [0, 1, 2, 3], 'f': {'x': 0, 'y': 1}}, mode)\n    failed += read_or_write(sc, basekey + \"_map_=0\", {'': 0}, mode)\n    failed += read_or_write(sc, basekey + u\"_map_x=0_y=foo\\u0180\\u01E3\\u11E5\", {'x': 0, 'y': u'foo\\u0180\\u01E3\\u11E5'}, mode)\n    failed += read_or_write(sc, basekey + u\"_map_x=0_foo\\u0180\\u01E3\\u11E5=1\", {'x': 0, u'foo\\u0180\\u01E3\\u11E5': 1}, mode)\n    \n    return failed\n\nif __name__ == \"__main__\":\n    if (sys.argv[1] == \"read\"):\n        basekey = sys.argv[2]\n        language = sys.argv[3]\n        basekey += '_' + language\n        mode = 'read'\n        print 'Python-JSON-API: reading from ' + language\n    elif (sys.argv[1] == \"write\"):\n        basekey = sys.argv[2]\n        basekey += '_json_python'\n        mode = 'write'\n        print 'Python-JSON-API: writing values'\n    else:\n        print 'unknown commands: ' + str(sys.argv)\n        sys.exit(1)\n    \n    sc = scalaris.TransactionSingleOp()\n    \n    failed = 0\n    failed += read_write_boolean(basekey, sc, mode)\n    failed += read_write_integer(basekey, sc, mode)\n    failed += read_write_long(basekey, sc, mode)\n    failed += read_write_biginteger(basekey, sc, mode)\n    failed += read_write_double(basekey, sc, mode)\n    failed += read_write_string(basekey, sc, mode)\n    failed += read_write_binary(basekey, sc, mode)\n    failed += read_write_list(basekey, sc, mode)\n    failed += read_write_map(basekey, sc, mode)\n    print ''\n    \n    if (failed > 0):\n        print str(failed) + ' number of ' + mode + 's failed.'\n        print ''\n        sys.exit(1)\n"
  },
  {
    "path": "python-api/scalaris_test.py",
    "content": "# Copyright 2011-2015 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nfrom scalaris import TransactionSingleOp, Transaction, ReplicatedDHT, ScalarisVM,\\\n    JSONConnection\nimport scalaris\nimport time, threading, json, socket\nfrom datetime import datetime\nfrom BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler\nfrom random import shuffle\nimport unittest\n\n_TEST_DATA = [\n             \"ahz2ieSh\", \"wooPhu8u\", \"quai9ooK\", \"Oquae4ee\", \"Airier1a\", \"Boh3ohv5\", \"ahD3Saog\", \"EM5ooc4i\", \n             \"Epahrai8\", \"laVahta7\", \"phoo6Ahj\", \"Igh9eepa\", \"aCh4Lah6\", \"ooT0ath5\", \"uuzau4Ie\", \"Iup6mae6\", \n#             \"xie7iSie\", \"ail8yeeP\", \"ooZ4eesi\", \"Ahn7ohph\", \"Ohy5moo6\", \"xooSh9Oo\", \"ieb6eeS7\", \"Thooqu9h\", \n#             \"eideeC9u\", \"phois3Ie\", \"EimaiJ2p\", \"sha6ahR1\", \"Pheih3za\", \"bai4eeXe\", \"rai0aB7j\", \"xahXoox6\", \n#             \"Xah4Okeg\", \"cieG8Yae\", \"Pe9Ohwoo\", \"Eehig6ph\", \"Xe7rooy6\", \"waY2iifu\", \"kemi8AhY\", \"Che7ain8\", \n#             \"ohw6seiY\", \"aegh1oBa\", \"thoh9IeG\", \"Kee0xuwu\", \"Gohng8ee\", \"thoh9Chi\", \"aa4ahQuu\", \"Iesh5uge\", \n#             \"Ahzeil8n\", \"ieyep5Oh\", \"xah3IXee\", \"Eefa5qui\", \"kai8Muuf\", \"seeCe0mu\", \"cooqua5Y\", \"Ci3ahF6z\", \n#             \"ot0xaiNu\", \"aewael8K\", \"aev3feeM\", \"Fei7ua5t\", \"aeCa6oph\", \"ag2Aelei\", \"Shah1Pho\", \"ePhieb0N\", \n#             \"Uqu7Phup\", \"ahBi8voh\", \"oon3aeQu\", \"Koopa0nu\", \"xi0quohT\", \"Oog4aiph\", \"Aip2ag5D\", \"tirai7Ae\", \n#             \"gi0yoePh\", \"uay7yeeX\", \"aeb6ahC1\", \"OoJeic2a\", \"ieViom1y\", \"di0eeLai\", \"Taec2phe\", \"ID2cheiD\", \n#             \"oi6ahR5M\", \"quaiGi8W\", \"ne1ohLuJ\", \"DeD0eeng\", \"yah8Ahng\", \"ohCee2ie\", \"ecu1aDai\", \"oJeijah4\", \n#             \"Goo9Una1\", \"Aiph3Phi\", \"Ieph0ce5\", \"ooL6cae7\", \"nai0io1H\", \"Oop2ahn8\", \"ifaxae7O\", \"NeHai1ae\", \n#             \"Ao8ooj6a\", \"hi9EiPhi\", \"aeTh9eiP\", \"ao8cheiH\", \"Yieg3sha\", \"mah7cu2D\", \"Uo5wiegi\", \"Oowei0ya\", \n#             \"efeiDee7\", \"Oliese6y\", \"eiSh1hoh\", \"Joh6hoh9\", \"zib6Ooqu\", \"eejiJie4\", \"lahZ3aeg\", \"keiRai1d\", \n#             \"Fei0aewe\", \"aeS8aboh\", \"hae3ohKe\", \"Een9ohQu\", \"AiYeeh7o\", \"Yaihah4s\", \"ood4Giez\", \"Oumai7te\", \n#             \"hae2kahY\", \"afieGh4v\", \"Ush0boo0\", \"Ekootee5\", \"Ya8iz6Ie\", \"Poh6dich\", \"Eirae4Ah\", \"pai8Eeme\", \n#             \"uNah7dae\", \"yo3hahCh\", \"teiTh7yo\", \"zoMa5Cuv\", \"ThiQu5ax\", \"eChi5caa\", \"ii9ujoiV\", \"ge7Iekui\",\n             \"sai2aiTa\", \"ohKi9rie\", \"ei2ioChu\", \"aaNgah9y\", \"ooJai1Ie\", \"shoh0oH9\", \"Ool4Ahya\", \"poh0IeYa\", \n             \"Uquoo0Il\", \"eiGh4Oop\", \"ooMa0ufe\", \"zee6Zooc\", \"ohhao4Ah\", \"Uweekek5\", \"aePoos9I\", \"eiJ9noor\", \n             \"phoong1E\", \"ianieL2h\", \"An7ohs4T\", \"Eiwoeku3\", \"sheiS3ao\", \"nei5Thiw\", \"uL5iewai\", \"ohFoh9Ae\"]\n\n_TOO_LARGE_REQUEST_SIZE = 1024*1024*10 # number of bytes\n\nclass TestTransactionSingleOp(unittest.TestCase):\n    def setUp(self):\n        # The time when the test suite was started.\n        now = datetime.now()\n        # This is used to create different erlang keys for each run.\n        self._testTime = int(time.mktime(now.timetuple()) * 1000 + (now.microsecond / 1000.0))\n\n    # Test method for TransactionSingleOp()\n    def testTransactionSingleOp1(self):\n        conn = TransactionSingleOp()\n        conn.close_connection()\n\n    # Test method for TransactionSingleOp(conn)\n    def testTransactionSingleOp2(self):\n        conn = TransactionSingleOp(conn = scalaris.JSONConnection(url = scalaris.DEFAULT_URL))\n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.close_connection() trying to close the connection twice.\n    def testDoubleClose(self):\n        conn = TransactionSingleOp()\n        conn.close_connection()\n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.read(key)\n    def testRead_NotFound(self):\n        key = \"_Read_NotFound\"\n        conn = TransactionSingleOp()\n        self.assertRaises(scalaris.NotFoundError, conn.read, str(self._testTime) + key)\n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.read(key) with a closed connection.\n    def testRead_NotConnected(self):\n        key = \"_Read_NotConnected\"\n        conn = TransactionSingleOp()\n        conn.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, conn.read, str(self._testTime) + key)\n        self.assertRaises(scalaris.NotFoundError, conn.read, str(self._testTime) + key)\n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.write(key, value=str()) with a closed connection.\n    def testWriteString_NotConnected(self):\n        key = \"_WriteString_NotConnected\"\n        conn = TransactionSingleOp()\n        conn.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, conn.write, str(self._testTime) + key, _TEST_DATA[0])\n        conn.write(str(self._testTime) + key, _TEST_DATA[0])\n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.write(key, value=str()) and TransactionSingleOp.read(key).\n    # Writes strings and uses a distinct key for each value. Tries to read the data afterwards.\n    def testWriteString1(self):\n        key = \"_WriteString1_\"\n        conn = TransactionSingleOp()\n        \n        for i in xrange(len(_TEST_DATA)):\n            conn.write(str(self._testTime) + key + str(i), _TEST_DATA[i])\n        \n        # now try to read the data:\n        for i in xrange(len(_TEST_DATA)):\n            actual = conn.read(str(self._testTime) + key + str(i))\n            self.assertEqual(actual, _TEST_DATA[i])\n        \n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.write(key, value=str()) and TransactionSingleOp.read(key).\n    # Writes strings and uses a single key for all the values. Tries to read the data afterwards.\n    def testWriteString2(self):\n        key = \"_WriteString2\"\n        conn = TransactionSingleOp()\n        \n        for i in xrange(len(_TEST_DATA)):\n            conn.write(str(self._testTime) + key, _TEST_DATA[i])\n        \n        # now try to read the data:\n        actual = conn.read(str(self._testTime) + key)\n        self.assertEqual(actual, _TEST_DATA[len(_TEST_DATA) - 1])\n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.write(key, value=list()) with a closed connection.\n    def testWriteList_NotConnected(self):\n        key = \"_WriteList_NotConnected\"\n        conn = TransactionSingleOp()\n        conn.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, conn.write, str(self._testTime) + key, [_TEST_DATA[0], _TEST_DATA[1]])\n        conn.write(str(self._testTime) + key, [_TEST_DATA[0], _TEST_DATA[1]])\n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.write(key, value=list()) and TransactionSingleOp.read(key).\n    # Writes strings and uses a distinct key for each value. Tries to read the data afterwards.\n    def testWriteList1(self):\n        key = \"_WriteList1_\"\n        conn = TransactionSingleOp()\n        \n        for i in xrange(0, len(_TEST_DATA) - 1, 2):\n            conn.write(str(self._testTime) + key + str(i), [_TEST_DATA[i], _TEST_DATA[i + 1]])\n        \n        # now try to read the data:\n        for i in xrange(0, len(_TEST_DATA), 2):\n            actual = conn.read(str(self._testTime) + key + str(i))\n            self.assertEqual(actual, [_TEST_DATA[i], _TEST_DATA[i + 1]])\n        \n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.write(key, value=list()) and TransactionSingleOp.read(key).\n    # Writes strings and uses a single key for all the values. Tries to read the data afterwards.\n    def testWriteList2(self):\n        key = \"_WriteList2\"\n        conn = TransactionSingleOp()\n        \n        mylist = []\n        for i in xrange(0, len(_TEST_DATA) - 1, 2):\n            mylist = [_TEST_DATA[i], _TEST_DATA[i + 1]]\n            conn.write(str(self._testTime) + key, mylist)\n        \n        # now try to read the data:\n        actual = conn.read(str(self._testTime) + key)\n        self.assertEqual(actual, mylist)\n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=str()) with a closed connection.\n    def testTestAndSetString_NotConnected(self):\n        key = \"_TestAndSetString_NotConnected\"\n        conn = TransactionSingleOp()\n        conn.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, conn.test_and_set, str(self._testTime) + key, _TEST_DATA[0], _TEST_DATA[1])\n        self.assertRaises(scalaris.NotFoundError, conn.test_and_set, str(self._testTime) + key, _TEST_DATA[0], _TEST_DATA[1])\n        conn.close_connection()\n    \n    # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=str()).\n    # Tries test_and_set with a non-existing key.\n    def testTestAndSetString_NotFound(self):\n        key = \"_TestAndSetString_NotFound\"\n        conn = TransactionSingleOp()\n        self.assertRaises(scalaris.NotFoundError, conn.test_and_set, str(self._testTime) + key, _TEST_DATA[0], _TEST_DATA[1])\n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=str()),\n    # TransactionSingleOp.read(key) and TransactionSingleOp.write(key, value=str()).\n    # Writes a string and tries to overwrite it using test_and_set\n    # knowing the correct old value. Tries to read the string afterwards.\n    def testTestAndSetString1(self):\n        key = \"_TestAndSetString1\"\n        conn = TransactionSingleOp()\n        \n        # first write all values:\n        for i in xrange(0, len(_TEST_DATA) - 1, 2):\n            conn.write(str(self._testTime) + key + str(i), _TEST_DATA[i])\n        \n        # now try to overwrite them using test_and_set:\n        for i in xrange(0, len(_TEST_DATA) - 1, 2):\n            conn.test_and_set(str(self._testTime) + key + str(i), _TEST_DATA[i], _TEST_DATA[i + 1])\n        \n        # now try to read the data:\n        for i in xrange(0, len(_TEST_DATA), 2):\n            actual = conn.read(str(self._testTime) + key + str(i))\n            self.assertEqual(actual, _TEST_DATA[i + 1])\n        \n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=str()),\n    # TransactionSingleOp.read(key) and TransactionSingleOp.write(key, value=str()).\n    # Writes a string and tries to overwrite it using test_and_set\n    # knowing the wrong old value. Tries to read the string afterwards.\n    def testTestAndSetString2(self):\n        key = \"_TestAndSetString2\"\n        conn = TransactionSingleOp()\n        \n        # first write all values:\n        for i in xrange(0, len(_TEST_DATA) - 1, 2):\n            conn.write(str(self._testTime) + key + str(i), _TEST_DATA[i])\n        \n        # now try to overwrite them using test_and_set:\n        for i in xrange(0, len(_TEST_DATA) - 1, 2):\n            try:\n                conn.test_and_set(str(self._testTime) + key + str(i), _TEST_DATA[i + 1], \"fail\")\n                self.fail('expected a KeyChangedError')\n            except scalaris.KeyChangedError as exception:\n                self.assertEqual(exception.old_value, _TEST_DATA[i])\n        \n        # now try to read the data:\n        for i in xrange(0, len(_TEST_DATA), 2):\n            actual = conn.read(str(self._testTime) + key + str(i))\n            self.assertEqual(actual, _TEST_DATA[i])\n        \n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=list()) with a closed connection.\n    def testTestAndSetList_NotConnected(self):\n        key = \"_TestAndSetList_NotConnected\"\n        conn = TransactionSingleOp()\n        conn.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, conn.test_and_set, str(self._testTime) + key, \"fail\", [_TEST_DATA[0], _TEST_DATA[1]])\n        self.assertRaises(scalaris.NotFoundError, conn.test_and_set, str(self._testTime) + key, \"fail\", [_TEST_DATA[0], _TEST_DATA[1]])\n        conn.close_connection()\n    \n    # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=list()).\n    # Tries test_and_set with a non-existing key.\n    def testTestAndSetList_NotFound(self):\n        key = \"_TestAndSetList_NotFound\"\n        conn = TransactionSingleOp()\n        self.assertRaises(scalaris.NotFoundError, conn.test_and_set, str(self._testTime) + key, \"fail\", [_TEST_DATA[0], _TEST_DATA[1]])\n        conn.close_connection()\n    \n    # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=list()),\n    # TransactionSingleOp.read(key) and TransactionSingleOp.write(key, value=list()).\n    # Writes a list and tries to overwrite it using test_and_set\n    # knowing the correct old value. Tries to read the string afterwards.\n    def testTestAndSetList1(self):\n        key = \"_TestAndSetList1\"\n        conn = TransactionSingleOp()\n        \n        # first write all values:\n        for i in xrange(0, len(_TEST_DATA) - 1, 2):\n            conn.write(str(self._testTime) + key + str(i), [_TEST_DATA[i], _TEST_DATA[i + 1]])\n        \n        # now try to overwrite them using test_and_set:\n        for i in xrange(0, len(_TEST_DATA) - 1, 2):\n            conn.test_and_set(str(self._testTime) + key + str(i), [_TEST_DATA[i], _TEST_DATA[i + 1]], [_TEST_DATA[i + 1], _TEST_DATA[i]])\n        \n        # now try to read the data:\n        for i in xrange(0, len(_TEST_DATA) - 1, 2):\n            actual = conn.read(str(self._testTime) + key + str(i))\n            self.assertEqual(actual, [_TEST_DATA[i + 1], _TEST_DATA[i]])\n        \n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=list()),\n    # TransactionSingleOp.read(key) and TransactionSingleOp.write(key, value=list()).\n    # Writes a string and tries to overwrite it using test_and_set\n    # knowing the wrong old value. Tries to read the string afterwards.\n    def testTestAndSetList2(self):\n        key = \"_TestAndSetList2\"\n        conn = TransactionSingleOp()\n        \n        # first write all values:\n        for i in xrange(0, len(_TEST_DATA) - 1, 2):\n            conn.write(str(self._testTime) + key + str(i), [_TEST_DATA[i], _TEST_DATA[i + 1]])\n        \n        # now try to overwrite them using test_and_set:\n        for i in xrange(0, len(_TEST_DATA) - 1, 2):\n            try:\n                conn.test_and_set(str(self._testTime) + key + str(i), \"fail\", 1)\n                self.fail('expected a KeyChangedError')\n            except scalaris.KeyChangedError as exception:\n                self.assertEqual(exception.old_value, [_TEST_DATA[i], _TEST_DATA[i + 1]])\n        \n        # now try to read the data:\n        for i in xrange(0, len(_TEST_DATA) - 1, 2):\n            actual = conn.read(str(self._testTime) + key + str(i))\n            self.assertEqual(actual, [_TEST_DATA[i], _TEST_DATA[i + 1]])\n        \n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.req_list(RequestList) with an\n    # empty request list.\n    def testReqList_Empty(self):\n        conn = TransactionSingleOp()\n        conn.req_list(conn.new_req_list())\n        conn.close_connection()\n\n    # Test method for TransactionSingleOp.req_list(RequestList) with a\n    # mixed request list.\n    def testReqList1(self):\n        key = \"_ReqList1_\"\n        conn = TransactionSingleOp()\n        \n        readRequests = conn.new_req_list()\n        firstWriteRequests = conn.new_req_list()\n        writeRequests = conn.new_req_list()\n        for i in xrange(0, len(_TEST_DATA)):\n            if (i % 2) == 0:\n                firstWriteRequests.add_write(str(self._testTime) + key + str(i), \"first_\" + _TEST_DATA[i])\n            writeRequests.add_write(str(self._testTime) + key + str(i), \"second_\" + _TEST_DATA[i])\n            readRequests.add_read(str(self._testTime) + key + str(i))\n        \n        results = conn.req_list(firstWriteRequests)\n        # evaluate the first write results:\n        for i in xrange(0, firstWriteRequests.size()):\n            conn.process_result_write(results[i])\n\n        results = conn.req_list(readRequests)\n        self.assertEqual(readRequests.size(), len(results))\n        # now evaluate the read results:\n        for i in xrange(0, readRequests.size()):\n            if (i % 2) == 0:\n                actual = conn.process_result_read(results[i])\n                self.assertEqual(\"first_\" + _TEST_DATA[i], actual)\n            else:\n                try:\n                    result = conn.process_result_read(results[i])\n                    # a not found exception must be thrown\n                    self.fail('expected a NotFoundError, got: ' + str(result))\n                except scalaris.NotFoundError:\n                    pass\n\n        results = conn.req_list(writeRequests)\n        self.assertEqual(writeRequests.size(), len(results))\n        # now evaluate the write results:\n        for i in xrange(0, writeRequests.size()):\n            conn.process_result_write(results[i])\n\n        # once again test reads - now all reads should be successful\n        results = conn.req_list(readRequests)\n        self.assertEqual(readRequests.size(), len(results))\n\n        # now evaluate the read results:\n        for i in xrange(0, readRequests.size()):\n            actual = conn.process_result_read(results[i])\n            self.assertEqual(\"second_\" + _TEST_DATA[i], actual)\n        \n        conn.close_connection();\n\n    # Test method for TransactionSingleOp.write(key, value=bytearray()) with a\n    # request that is too large.\n    def testReqTooLarge(self):\n        conn = TransactionSingleOp()\n        data = '0' * _TOO_LARGE_REQUEST_SIZE\n        key = \"_ReqTooLarge\"\n        try:\n            conn.write(str(self._testTime) + key, data)\n            self.fail('The write should have failed unless yaws_max_post_data was set larger than ' + str(_TOO_LARGE_REQUEST_SIZE))\n        except scalaris.ConnectionError:\n            pass\n        \n        conn.close_connection()\n\nclass TestTransaction(unittest.TestCase):\n    def setUp(self):\n        # The time when the test suite was started.\n        now = datetime.now()\n        # This is used to create different erlang keys for each run.\n        self._testTime = int(time.mktime(now.timetuple()) * 1000 + (now.microsecond / 1000.0))\n\n    # Test method for Transaction()\n    def testTransaction1(self):\n        t = Transaction()\n        t.close_connection()\n\n    # Test method for Transaction(conn)\n    def testTransaction3(self):\n        t = Transaction(conn = scalaris.JSONConnection(url = scalaris.DEFAULT_URL))\n        t.close_connection()\n\n    # Test method for Transaction.close_connection() trying to close the connection twice.\n    def testDoubleClose(self):\n        t = Transaction()\n        t.close_connection()\n        t.close_connection()\n\n    # Test method for Transaction.commit() with a closed connection.\n    def testCommit_NotConnected(self):\n        t = Transaction()\n        t.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, t.commit)\n        t.commit()\n        t.close_connection()\n\n    # Test method for Transaction.commit() which commits an empty transaction.\n    def testCommit_Empty(self):\n        t = Transaction()\n        t.commit()\n        t.close_connection()\n\n    # Test method for Transaction.abort() with a closed connection.\n    def testAbort_NotConnected(self):\n        t = Transaction()\n        t.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, t.abort)\n        t.abort()\n        t.close_connection()\n\n    # Test method for Transaction.abort() which aborts an empty transaction.\n    def testAbort_Empty(self):\n        t = Transaction()\n        t.abort()\n        t.close_connection()\n\n    # Test method for Transaction.read(key)\n    def testRead_NotFound(self):\n        key = \"_Read_NotFound\"\n        t = Transaction()\n        self.assertRaises(scalaris.NotFoundError, t.read, str(self._testTime) + key)\n        t.close_connection()\n\n    # Test method for Transaction.read(key) with a closed connection.\n    def testRead_NotConnected(self):\n        key = \"_Read_NotConnected\"\n        t = Transaction()\n        t.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, t.read, str(self._testTime) + key)\n        self.assertRaises(scalaris.NotFoundError, t.read, str(self._testTime) + key)\n        t.close_connection()\n\n    # Test method for Transaction.write(key, value=str()) with a closed connection.\n    def testWriteString_NotConnected(self):\n        key = \"_WriteString_NotConnected\"\n        t = Transaction()\n        t.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, t.write, str(self._testTime) + key, _TEST_DATA[0])\n        t.write(str(self._testTime) + key, _TEST_DATA[0])\n        t.close_connection()\n\n    # Test method for Transaction.read(key) and Transaction.write(key, value=str())\n    # which should show that writing a value for a key for which a previous read\n    # returned a NotFoundError is possible.\n    def testWriteString_NotFound(self):\n        key = \"_WriteString_notFound\"\n        t = Transaction()\n        notFound = False\n        try:\n            t.read(str(self._testTime) + key)\n        except scalaris.NotFoundError:\n            notFound = True\n        \n        self.assertTrue(notFound)\n        t.write(str(self._testTime) + key, _TEST_DATA[0])\n        self.assertEqual(t.read(str(self._testTime) + key), _TEST_DATA[0])\n        t.close_connection()\n\n    # Test method for Transaction.write(key, value=str()) and Transaction.read(key).\n    # Writes strings and uses a distinct key for each value. Tries to read the data afterwards.\n    def testWriteString(self):\n        key = \"_testWriteString1_\"\n        t = Transaction()\n        \n        for i in xrange(len(_TEST_DATA)):\n            t.write(str(self._testTime) + key + str(i), _TEST_DATA[i])\n        \n        # now try to read the data:\n        for i in xrange(len(_TEST_DATA)):\n            actual = t.read(str(self._testTime) + key + str(i))\n            self.assertEqual(actual, _TEST_DATA[i])\n        \n        # commit the transaction and try to read the data with a new one:\n        t.commit()\n        t.close_connection()\n        t = Transaction()\n        for i in xrange(len(_TEST_DATA)):\n            actual = t.read(str(self._testTime) + key + str(i))\n            self.assertEqual(actual, _TEST_DATA[i])\n        \n        t.close_connection()\n\n    # Test method for Transaction.write(key, value=list()) and Transaction.read(key).\n    # Writes a list and uses a distinct key for each value. Tries to read the data afterwards.\n    def testWriteList1(self):\n        key = \"_testWriteList1_\"\n        t = scalaris.Transaction()\n        \n        for i in xrange(0, len(_TEST_DATA) - 1, 2):\n            t.write(str(self._testTime) + key + str(i), [_TEST_DATA[i], _TEST_DATA[i + 1]])\n        \n        # now try to read the data:\n        for i in xrange(0, len(_TEST_DATA), 2):\n            actual = t.read(str(self._testTime) + key + str(i))\n            self.assertEqual(actual, [_TEST_DATA[i], _TEST_DATA[i + 1]])\n        \n        # commit the transaction and try to read the data with a new one:\n        t.commit()\n        t.close_connection()\n        t = Transaction()\n        for i in xrange(0, len(_TEST_DATA), 2):\n            actual = t.read(str(self._testTime) + key + str(i))\n            self.assertEqual(actual, [_TEST_DATA[i], _TEST_DATA[i + 1]])\n        \n        t.close_connection()\n\n    # Test method for Transaction.req_list(RequestList) with an\n    # empty request list.\n    def testReqList_Empty(self):\n        conn = Transaction()\n        conn.req_list(conn.new_req_list())\n        conn.close_connection()\n\n    # Test method for Transaction.req_list(RequestList) with a\n    # mixed request list.\n    def testReqList1(self):\n        key = \"_ReqList1_\"\n        conn = Transaction()\n        \n        readRequests = conn.new_req_list()\n        firstWriteRequests = conn.new_req_list()\n        writeRequests = conn.new_req_list()\n        for i in xrange(0, len(_TEST_DATA)):\n            if (i % 2) == 0:\n                firstWriteRequests.add_write(str(self._testTime) + key + str(i), _TEST_DATA[i])\n            writeRequests.add_write(str(self._testTime) + key + str(i), _TEST_DATA[i])\n            readRequests.add_read(str(self._testTime) + key + str(i))\n        \n        results = conn.req_list(firstWriteRequests)\n        # evaluate the first write results:\n        for i in xrange(0, firstWriteRequests.size()):\n            conn.process_result_write(results[i])\n\n        requests = conn.new_req_list(readRequests).extend(writeRequests).add_commit()\n        results = conn.req_list(requests)\n        self.assertEqual(requests.size(), len(results))\n\n        # now evaluate the read results:\n        for i in xrange(0, readRequests.size()):\n            if (i % 2) == 0:\n                actual = conn.process_result_read(results[i])\n                self.assertEqual(_TEST_DATA[i], actual)\n            else:\n                try:\n                    conn.process_result_read(results[i])\n                    # a not found exception must be thrown\n                    self.fail('expected a NotFoundError')\n                except scalaris.NotFoundError:\n                    pass\n\n        # now evaluate the write results:\n        for i in xrange(0, writeRequests.size()):\n            pos = readRequests.size() + i\n            conn.process_result_write(results[pos])\n\n        # once again test reads - now all reads should be successful\n        results = conn.req_list(readRequests)\n        self.assertEqual(readRequests.size(), len(results))\n\n        # now evaluate the read results:\n        for i in xrange(0, readRequests.size()):\n            actual = conn.process_result_read(results[i])\n            self.assertEqual(_TEST_DATA[i], actual)\n        \n        conn.close_connection();\n\n    # Test method for Transaction.write(key, value=bytearray()) with a\n    # request that is too large.\n    def testReqTooLarge(self):\n        conn = Transaction()\n        data = '0' * _TOO_LARGE_REQUEST_SIZE\n        key = \"_ReqTooLarge\"\n        try:\n            conn.write(str(self._testTime) + key, data)\n            self.fail('The write should have failed unless yaws_max_post_data was set larger than ' + str(_TOO_LARGE_REQUEST_SIZE))\n        except scalaris.ConnectionError:\n            pass\n        \n        conn.close_connection()\n\n    # Various tests.\n    def testVarious(self):\n        self._writeSingleTest(\"_0:\\u0160arplaninac:page_\", _TEST_DATA[0])\n\n    # Helper function for single write tests.\n    # Writes a strings to some key and tries to read it afterwards.\n    def _writeSingleTest(self, key, data):\n        t = Transaction()\n        \n        t.write(str(self._testTime) + key, data)\n        # now try to read the data:\n        self.assertEqual(t.read(str(self._testTime) + key), data)\n        # commit the transaction and try to read the data with a new one:\n        t.commit()\n        t.close_connection()\n        t = Transaction()\n        self.assertEqual(t.read(str(self._testTime) + key), data)\n        \n        t.close_connection()\n\nclass TestReplicatedDHT(unittest.TestCase):\n    def setUp(self):\n        # The time when the test suite was started.\n        now = datetime.now()\n        # This is used to create different erlang keys for each run.\n        self._testTime = int(time.mktime(now.timetuple()) * 1000 + (now.microsecond / 1000.0))\n\n    # Test method for ReplicatedDHT()\n    def testReplicatedDHT1(self):\n        rdht = ReplicatedDHT()\n        rdht.close_connection()\n\n    # Test method for ReplicatedDHT(conn)\n    def testReplicatedDHT2(self):\n        rdht = ReplicatedDHT(conn = scalaris.JSONConnection(url = scalaris.DEFAULT_URL))\n        rdht.close_connection()\n\n    # Test method for ReplicatedDHT.close_connection() trying to close the connection twice.\n    def testDoubleClose(self):\n        rdht = ReplicatedDHT()\n        rdht.close_connection()\n        rdht.close_connection()\n    \n    # Tries to read the value at the given key and fails if this does\n    # not fail with a NotFoundError.\n    def _checkKeyDoesNotExist(self, key):\n        conn = scalaris.TransactionSingleOp()\n        try:\n            conn.read(key)\n            self.fail('the value at ' + key + ' should not exist anymore')\n        except scalaris.NotFoundError:\n            # nothing to do here\n            pass\n        conn.close_connection()\n\n    # Test method for ReplicatedDHT.delete(key).\n    # Tries to delete some not existing keys.\n    def testDelete_notExistingKey(self):\n        key = \"_Delete_NotExistingKey\"\n        rdht = ReplicatedDHT()\n        rt = scalaris.RoutingTable()\n        r = rt.get_replication_factor()\n        \n        for i in xrange(len(_TEST_DATA)):\n            ok = rdht.delete(str(self._testTime) + key + str(i))\n            self.assertEqual(ok, 0)\n            results = rdht.get_last_delete_result()\n            self.assertEqual((results.ok, results.locks_set, results.undefined), (0, 0, r))\n            self._checkKeyDoesNotExist(str(self._testTime) + key + str(i))\n        \n        rdht.close_connection()\n\n    # Test method for ReplicatedDHT.delete(key) and TransactionSingleOp#write(key, value=str()).\n    # Inserts some values, tries to delete them afterwards and tries the delete again.\n    def testDelete1(self):\n        key = \"_Delete1\"\n        c = scalaris.JSONConnection(url = scalaris.DEFAULT_URL)\n        rdht = ReplicatedDHT(conn = c)\n        sc = scalaris.TransactionSingleOp(conn = c)\n        rt = scalaris.RoutingTable(conn = c)\n        r = rt.get_replication_factor()\n        \n        for i in xrange(len(_TEST_DATA)):\n            sc.write(str(self._testTime) + key + str(i), _TEST_DATA[i])\n        \n        # now try to delete the data:\n        for i in xrange(len(_TEST_DATA)):\n            ok = rdht.delete(str(self._testTime) + key + str(i))\n            self.assertEqual(ok, r)\n            results = rdht.get_last_delete_result()\n            self.assertEqual((results.ok, results.locks_set, results.undefined), (r, 0, 0))\n            self._checkKeyDoesNotExist(str(self._testTime) + key + str(i))\n            \n            # try again (should be successful with 0 deletes)\n            ok = rdht.delete(str(self._testTime) + key + str(i))\n            self.assertEqual(ok, 0)\n            results = rdht.get_last_delete_result()\n            self.assertEqual((results.ok, results.locks_set, results.undefined), (0, 0, r))\n            self._checkKeyDoesNotExist(str(self._testTime) + key + str(i))\n        \n        c.close()\n\n    # Test method for ReplicatedDHT.delete(key) and TransactionSingleOp#write(key, value=str()).\n    # Inserts some values, tries to delete them afterwards, inserts them again and tries to delete them again (twice).\n    def testDelete2(self):\n        key = \"_Delete2\"\n        c = scalaris.JSONConnection(url = scalaris.DEFAULT_URL)\n        rdht = ReplicatedDHT(conn = c)\n        sc = scalaris.TransactionSingleOp(conn = c)\n        rt = scalaris.RoutingTable(conn = c)\n        r = rt.get_replication_factor()\n        \n        for i in xrange(len(_TEST_DATA)):\n            sc.write(str(self._testTime) + key + str(i), _TEST_DATA[i])\n        \n        # now try to delete the data:\n        for i in xrange(len(_TEST_DATA)):\n            ok = rdht.delete(str(self._testTime) + key + str(i))\n            self.assertEqual(ok, r)\n            results = rdht.get_last_delete_result()\n            self.assertEqual((results.ok, results.locks_set, results.undefined), (r, 0, 0))\n            self._checkKeyDoesNotExist(str(self._testTime) + key + str(i))\n        \n        for i in xrange(len(_TEST_DATA)):\n            sc.write(str(self._testTime) + key + str(i), _TEST_DATA[i])\n        \n        # now try to delete the data:\n        for i in xrange(len(_TEST_DATA)):\n            ok = rdht.delete(str(self._testTime) + key + str(i))\n            self.assertEqual(ok, r)\n            results = rdht.get_last_delete_result()\n            self.assertEqual((results.ok, results.locks_set, results.undefined), (r, 0, 0))\n            self._checkKeyDoesNotExist(str(self._testTime) + key + str(i))\n            \n            # try again (should be successful with 0 deletes)\n            ok = rdht.delete(str(self._testTime) + key + str(i))\n            self.assertEqual(ok, 0)\n            results = rdht.get_last_delete_result()\n            self.assertEqual((results.ok, results.locks_set, results.undefined), (0, 0, r))\n            self._checkKeyDoesNotExist(str(self._testTime) + key + str(i))\n        \n        c.close()\n\nclass TestScalarisVM(unittest.TestCase):\n    def setUp(self):\n        # The time when the test suite was started.\n        now = datetime.now()\n        # This is used to create different erlang keys for each run.\n        self._testTime = int(time.mktime(now.timetuple()) * 1000 + (now.microsecond / 1000.0))\n\n    # Test method for ScalarisVM()\n    def testScalarisVM1(self):\n        rdht = ScalarisVM()\n        rdht.close_connection()\n\n    # Test method for ScalarisVM(conn)\n    def testScalarisVM2(self):\n        rdht = ScalarisVM(conn = scalaris.JSONConnection(url = scalaris.DEFAULT_URL))\n        rdht.close_connection()\n\n    # Test method for ScalarisVM.close_connection() trying to close the connection twice.\n    def testDoubleClose(self):\n        rdht = ScalarisVM()\n        rdht.close_connection()\n        rdht.close_connection()\n\n    def testGetVersion_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.getVersion() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.getVersion())\n        vm.getVersion()\n        vm.close_connection()\n\n    def testGetVersion1(self):\n        \"\"\"Test method for ScalarisVM.getVersion().\"\"\"\n        vm = ScalarisVM()\n        version = vm.getVersion()\n        self.assertTrue(isinstance(version, basestring), msg = version)\n        self.assertTrue(len(version) > 0)\n        vm.close_connection()\n\n    def testGetInfo_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.getInfo() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.getInfo())\n        vm.getInfo()\n        vm.close_connection()\n\n    def testGetInfo1(self):\n        \"\"\"Test method for ScalarisVM.getInfo().\"\"\"\n        vm = ScalarisVM()\n        info = vm.getInfo()\n        self.assertTrue(isinstance(info.scalarisVersion, basestring), msg = info.scalarisVersion)\n        self.assertTrue(len(info.scalarisVersion) > 0, msg = \"scalaris_version (\" + info.scalarisVersion + \") != \\\"\\\"\");\n        self.assertTrue(isinstance(info.erlangVersion, basestring), msg = info.erlangVersion)\n        self.assertTrue(len(info.erlangVersion) > 0, msg = \"erlang_version (\" + info.erlangVersion + \") != \\\"\\\"\");\n        self.assertTrue(isinstance(info.memTotal, int), msg = info.memTotal)\n        self.assertTrue(info.memTotal >= 0, msg = \"mem_total (\" + str(info.memTotal) + \") >= 0\");\n        self.assertTrue(isinstance(info.uptime, int), msg = info.uptime)\n        self.assertTrue(info.uptime >= 0, msg = \"uptime (\" + str(info.uptime) + \") >= 0\");\n        self.assertTrue(isinstance(info.erlangNode, basestring), msg = info.erlangNode)\n        self.assertTrue(len(info.erlangNode) > 0, msg = \"erlang_node (\" + info.erlangNode + \") != \\\"\\\"\");\n        self.assertTrue(isinstance(info.port, int), msg = info.port)\n        self.assertTrue(info.port >= 0 and info.port <= 65535, msg = \"0 <= port (\" + str(info.port) + \") <= 65535\");\n        self.assertTrue(isinstance(info.yawsPort, int), msg = info.yawsPort)\n        self.assertTrue(info.yawsPort >= 0 and info.yawsPort <= 65535, msg = \"0 <= yaws_port (\" + str(info.yawsPort) + \") <= 65535\");\n        vm.close_connection()\n\n    def testGetNumberOfNodes_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.getNumberOfNodes() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.getNumberOfNodes())\n        vm.getNumberOfNodes()\n        vm.close_connection()\n\n    def testGetNumberOfNodes1(self):\n        \"\"\"Test method for ScalarisVM.getVersion().\"\"\"\n        vm = ScalarisVM()\n        number = vm.getNumberOfNodes()\n        self.assertTrue(isinstance(number, int), msg = number)\n        self.assertTrue(number >= 0)\n        vm.close_connection()\n\n    def testGetNodes_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.getNodes() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.getNodes())\n        vm.getNodes()\n        vm.close_connection()\n\n    def testGetNodes1(self):\n        \"\"\"Test method for ScalarisVM.getNodes().\"\"\"\n        vm = ScalarisVM()\n        nodes = vm.getNodes()\n        self.assertTrue(isinstance(nodes, list), msg = nodes)\n        self.assertTrue(len(nodes) >= 0)\n        self.assertEqual(len(nodes), vm.getNumberOfNodes())\n        vm.close_connection()\n\n    def testAddNodes_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.addNodes() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.addNodes(0))\n        vm.addNodes(0)\n        vm.close_connection()\n\n    def testAddNodes0(self):\n        \"\"\"Test method for ScalarisVM.addNodes(0).\"\"\"\n        self._testAddNodes(0)\n\n    def testAddNodes1(self):\n        \"\"\"Test method for ScalarisVM.addNodes(1).\"\"\"\n        self._testAddNodes(1)\n\n    def testAddNodes3(self):\n        \"\"\"Test method for ScalarisVM.addNodes(3).\"\"\"\n        self._testAddNodes(3)\n\n    def _testAddNodes(self, nodesToAdd):\n        \"\"\"Test method for ScalarisVM.addNodes(nodesToAdd).\"\"\"\n        vm = ScalarisVM()\n        size = vm.getNumberOfNodes();\n        (ok, failed) = vm.addNodes(nodesToAdd)\n        size = size + nodesToAdd\n        self.assertEqual(nodesToAdd, len(ok))\n        self.assertEqual(len(failed), 0)\n        self.assertEqual(size, vm.getNumberOfNodes())\n        nodes = vm.getNodes()\n        for name in ok:\n            self.assertTrue(name in nodes, str(nodes) + \" should contain \" + name)\n        for name in ok:\n            vm.killNode(name)\n        size = size - nodesToAdd\n        self.assertEqual(size, vm.getNumberOfNodes())\n        vm.close_connection()\n\n    def testShutdownNode_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.shutdownNode() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.shutdownNode(\"test\"))\n        vm.shutdownNode(\"test\")\n        vm.close_connection()\n\n    def testShutdownNode1(self):\n        \"\"\"Test method for ScalarisVM.shutdownNode().\"\"\"\n        self._testDeleteNode('shutdown')\n\n    def testKillNode_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.killNode() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.killNode(\"test\"))\n        vm.killNode(\"test\")\n        vm.close_connection()\n\n    def testKillNode1(self):\n        \"\"\"Test method for ScalarisVM.killNode().\"\"\"\n        self._testDeleteNode('kill')\n\n    def _testDeleteNode(self, action):\n        \"\"\"Test method for ScalarisVM.shutdownNode() and ScalarisVM.killNode().\"\"\"\n        vm = ScalarisVM()\n        size = vm.getNumberOfNodes();\n        (ok, _failed) = vm.addNodes(1)\n        name = ok[0]\n        self.assertEqual(size + 1, vm.getNumberOfNodes())\n        if action == 'shutdown':\n            result = vm.shutdownNode(name)\n        elif action == 'kill':\n            result = vm.killNode(name)\n        self.assertTrue(result)\n        self.assertEqual(size, vm.getNumberOfNodes())\n        nodes = vm.getNodes()\n        self.assertTrue(not name in nodes, str(nodes) + \" should not contain \" + name)\n        vm.close_connection()\n\n    def testShutdownNodes_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.shutdownNodes() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.shutdownNodes(0))\n        vm.shutdownNodes(0)\n        vm.close_connection()\n\n    def testShutdownNodes0(self):\n        \"\"\"Test method for ScalarisVM.shutdownNodes(0).\"\"\"\n        self._testDeleteNodes(0, 'shutdown')\n\n    def testShutdownNodes1(self):\n        \"\"\"Test method for ScalarisVM.shutdownNodes(1).\"\"\"\n        self._testDeleteNodes(1, 'shutdown')\n\n    def testShutdownNodes3(self):\n        \"\"\"Test method for ScalarisVM.shutdownNodes(3).\"\"\"\n        self._testDeleteNodes(3, 'shutdown')\n\n    def testKillNodes_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.killNodes() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.killNodes(0))\n        vm.killNodes(0)\n        vm.close_connection()\n\n    def testKillNodes0(self):\n        \"\"\"Test method for ScalarisVM.killNodes(0).\"\"\"\n        self._testDeleteNodes(0, 'kill')\n\n    def testKillNodes1(self):\n        \"\"\"Test method for ScalarisVM.killNodes(1).\"\"\"\n        self._testDeleteNodes(1, 'kill')\n\n    def testKillNodes3(self):\n        \"\"\"Test method for ScalarisVM.killNodes(3).\"\"\"\n        self._testDeleteNodes(3, 'kill')\n\n    def _testDeleteNodes(self, nodesToRemove, action):\n        \"\"\"Test method for ScalarisVM.shutdownNode() and ScalarisVM.killNode().\"\"\"\n        vm = ScalarisVM()\n        size = vm.getNumberOfNodes();\n        if nodesToRemove >= 1:\n            vm.addNodes(nodesToRemove)\n            self.assertEqual(size + nodesToRemove, vm.getNumberOfNodes())\n        if action == 'shutdown':\n            result = vm.shutdownNodes(nodesToRemove)\n        elif action == 'kill':\n            result = vm.killNodes(nodesToRemove)\n        self.assertEqual(nodesToRemove, len(result))\n        self.assertEqual(size, vm.getNumberOfNodes())\n        nodes = vm.getNodes()\n        for name in result:\n            self.assertTrue(not name in nodes, str(nodes) + \" should not contain \" + name)\n        vm.close_connection()\n\n    def testShutdownNodesByName_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.shutdownNodesByName() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.shutdownNodesByName([\"test\"]))\n        vm.shutdownNodesByName([\"test\"])\n        vm.close_connection()\n\n    def testShutdownNodesByName0(self):\n        \"\"\"Test method for ScalarisVM.shutdownNodesByName(0).\"\"\"\n        self._testDeleteNodes(0, 'shutdown')\n\n    def testShutdownNodesByName1(self):\n        \"\"\"Test method for ScalarisVM.shutdownNodesByName(1).\"\"\"\n        self._testDeleteNodes(1, 'shutdown')\n\n    def testShutdownNodesByName3(self):\n        \"\"\"Test method for ScalarisVM.shutdownNodesByName(3).\"\"\"\n        self._testDeleteNodes(3, 'shutdown')\n\n    def testKillNodesByName_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.killNodesByName() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.killNodesByName([\"test\"]))\n        vm.killNodesByName([\"test\"])\n        vm.close_connection()\n\n    def testKillNodesByName0(self):\n        \"\"\"Test method for ScalarisVM.killNodesByName(0).\"\"\"\n        self._testDeleteNodes(0, 'kill')\n\n    def testKillNodesByName1(self):\n        \"\"\"Test method for ScalarisVM.killNodesByName(1).\"\"\"\n        self._testDeleteNodes(1, 'kill')\n\n    def testKillNodesByName3(self):\n        \"\"\"Test method for ScalarisVM.killNodesByName(3).\"\"\"\n        self._testDeleteNodes(3, 'shutdown')\n\n    def _testDeleteNodesByName(self, nodesToRemove, action):\n        \"\"\"Test method for ScalarisVM.shutdownNode() and ScalarisVM.killNode().\"\"\"\n        vm = ScalarisVM()\n        size = vm.getNumberOfNodes();\n        if nodesToRemove >= 1:\n            vm.addNodes(nodesToRemove)\n            self.assertEqual(size + nodesToRemove, vm.getNumberOfNodes())\n        nodes = vm.getNodes()\n        shuffle(nodes)\n        removedNodes = nodes[:nodesToRemove]\n        if action == 'shutdown':\n            (ok, not_found) = vm.shutdownNodesByName(removedNodes)\n        elif action == 'kill':\n            (ok, not_found) = vm.killNodesByName(removedNodes)\n        self.assertEqual(nodesToRemove, len(ok))\n        self.assertEqual(nodesToRemove, len(not_found))\n        list.sort(removedNodes)\n        list.sort(ok)\n        self.assertEqual(removedNodes, ok)\n        self.assertEqual(size, vm.getNumberOfNodes())\n        nodes = vm.getNodes()\n        for name in ok:\n            self.assertTrue(not name in nodes, str(nodes) + \" should not contain \" + name)\n        vm.close_connection()\n        \n    def testGetOtherVMs_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.getOtherVMs() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.getOtherVMs(0))\n        vm.getOtherVMs(1)\n        vm.close_connection()\n        \n    def testGetOtherVMs1(self):\n        \"\"\"Test method for ScalarisVM.getOtherVMs(1).\"\"\"\n        self._testGetOtherVMs(1)\n        \n    def testGetOtherVMs2(self):\n        \"\"\"Test method for ScalarisVM.getOtherVMs(2).\"\"\"\n        self._testGetOtherVMs(2)\n        \n    def testGetOtherVMs3(self):\n        \"\"\"Test method for ScalarisVM.getOtherVMs(3).\"\"\"\n        self._testGetOtherVMs(3)\n        \n    def _testGetOtherVMs(self, maxVMs):\n        \"\"\"Test method for ScalarisVM.getOtherVMs().\"\"\"\n        vm = ScalarisVM()\n        otherVMs = vm.getOtherVMs(maxVMs)\n        self.assertTrue(len(otherVMs) <= maxVMs, \"list too long: \" + str(otherVMs))\n        for otherVMUrl in otherVMs:\n            otherVM = ScalarisVM(JSONConnection(otherVMUrl))\n            otherVM.getInfo()\n            otherVM.close_connection()\n        vm.close_connection()\n        \n    # not tested because we still need the Scalaris Erlang VM:\n    def _testShutdownVM_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.shutdownVM() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.shutdownVM())\n        vm.shutdownVM()\n        vm.close_connection()\n        \n    # not tested because we still need the Scalaris Erlang VM:\n    def _testShutdownVM1(self):\n        \"\"\"Test method for ScalarisVM.shutdownVM().\"\"\"\n        vm = ScalarisVM()\n        vm.shutdownVM()\n        vm.close_connection()\n\n    # not tested because we still need the Scalaris Erlang VM:\n    def _testKillVM_NotConnected(self):\n        \"\"\"Test method for ScalarisVM.killVM() with a closed connection.\"\"\"\n        vm = ScalarisVM()\n        vm.close_connection()\n        #self.assertRaises(scalaris.ConnectionError, vm.killVM())\n        vm.killVM()\n        vm.close_connection()\n\n    # not tested because we still need the Scalaris Erlang VM:\n    def _testKillVM1(self):\n        \"\"\"Test method for ScalarisVM.killVM().\"\"\"\n        vm = ScalarisVM()\n        vm.killVM()\n        vm.close_connection()\n\nif __name__ == \"__main__\":\n    #import sys;sys.argv = ['', 'Test.testName']\n    unittest.main()\n"
  },
  {
    "path": "python-api/scalaris_userdevguide_jsontrace.py",
    "content": "# Copyright 2011-2015 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nimport json, socket\nimport scalaris\n\nclass ConnectionWithTrace(scalaris.JSONConnection):\n    def __init__(self, url = scalaris.DEFAULT_URL, timeout = socket.getdefaulttimeout()):\n        scalaris.JSONConnection.__init__(self, url = scalaris.DEFAULT_URL, timeout = timeout)\n\n    def call(self, function, params):\n        params = {'jsonrpc': '2.0',\n                  'method': function,\n                  'params': params,\n                  'id': 0}\n        # use compact JSON encoding:\n        params_json = json.dumps(params, separators=(',',':'))\n        print ''\n        print 'request:'\n        print json.dumps(params, indent=1)\n        headers = {\"Content-type\": \"application/json\"}\n        try:\n            self._conn.request(\"POST\", scalaris.DEFAULT_PATH, params_json, headers)\n            response = self._conn.getresponse()\n            if (response.status < 200 or response.status >= 300):\n                raise scalaris.ConnectionError(response)\n            data = response.read()\n        except Exception as instance:\n            raise scalaris.ConnectionError(instance)\n        print ''\n        print 'response:'\n        print json.dumps(json.loads(data), indent=1)\n        response_json = json.loads(data)\n        return response_json['result']\n\nif __name__ == \"__main__\":\n    sc1 = scalaris.Transaction(conn = ConnectionWithTrace())\n    sc1.req_list(sc1.new_req_list().add_write(\"keyA\", \"valueA\").add_write(\"keyB\", \"valueB\").add_commit())\n    sc1.close_connection()\n    sc2 = scalaris.Transaction(conn = ConnectionWithTrace())\n    sc2.req_list(sc2.new_req_list().add_read(\"keyA\").add_read(\"keyB\"))\n    sc2.req_list(sc2.new_req_list().add_write(\"keyA\", \"valueA2\").add_commit())\n"
  },
  {
    "path": "python-api/setup.py",
    "content": "#!/usr/bin/env python\n\nfrom distutils.core import setup\n\nsetup(name='scalaris',\n      version='0.9.0+git',\n      description='Scalaris python bindings',\n      author='Nico Kruber',\n      author_email='kruber@zib.de',\n      url='https://code.google.com/p/scalaris/',\n      py_modules=['scalaris', 'scalaris_bench'],\n     )\n"
  },
  {
    "path": "ruby-api/.gitignore",
    "content": "/*.gem\n/.buildpath\n/.project\n/scalaris\n"
  },
  {
    "path": "ruby-api/scalaris.gemspec",
    "content": "# coding: UTF-8\n\nGem::Specification.new do |s|\n  s.name              = \"scalaris\"\n  s.version           = \"0.0.1\"\n  s.platform          = Gem::Platform::RUBY\n  s.authors           = [\"Nico Kruber, Florian Schintke, Thorsten Schütt\"]\n  s.email             = [\"scalaris@googlegroups.com\"]\n  s.homepage          = \"https://code.google.com/p/scalaris/\"\n  s.summary           = \"Ruby bindings for Scalaris\"\n  s.description       = \"Ruby bindings for Scalaris\"\n\n  s.required_rubygems_version = \">= 1.3.6\"\n  \n  # If you have runtime dependencies, add them here\n  s.add_runtime_dependency \"json\", \">= 1.4.0\"\n  \n  # If you have development dependencies, add them here\n  # s.add_development_dependency \"another\", \"= 0.9\"\n\n  # The list of files to be contained in the gem\n  s.files         = Dir[\"scalaris.rb\", \"scalarisclient.rb\"]\n  s.bindir        = '.'\n  s.executables   = Dir[\"scalarisclient.rb\"]\n  # s.extensions    = Dir[\"ext/extconf.rb\"]\n  \n  # s.require_path = 'lib'\n\n  # For C extensions\n  # s.extensions = \"ext/extconf.rb\"\nend\n"
  },
  {
    "path": "ruby-api/scalaris.in",
    "content": "#!/bin/bash\n\n#  Copyright 2007-2015 Zuse Institute Berlin\n#\n#   Licensed under the Apache License, Version 2.0 (the \"License\");\n#   you may not use this file except in compliance with the License.\n#   You may obtain a copy of the License at\n#\n#       http://www.apache.org/licenses/LICENSE-2.0\n#\n#   Unless required by applicable law or agreed to in writing, software\n#   distributed under the License is distributed on an \"AS IS\" BASIS,\n#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#   See the License for the specific language governing permissions and\n#   limitations under the License.\n\n# inspired by the start script of ant\n\nprefix=@prefix@\nexec_prefix=@exec_prefix@\nSCALARISDIR=@libdir@/scalaris\nSCALARISCLIENT_RUBY=\"@RUBYSITELIBDIR@/scalaris_client.rb\"\n\nfor arg in \"$@\"; do\n  args=\"$args \\\"$arg\\\"\"\ndone\n\n# activate no_config if called in the source tree:\nABSPATH=\"$(cd \"${0%/*}\" 2>/dev/null; echo \"$PWD\"/\"${0##*/}\")\"\nDIRNAME=`dirname $ABSPATH`\n# is this a source checkout or an (rpm/deb/manual) installation?\nif [ \"$DIRNAME\" != \"@bindir@\" -a \"$DIRNAME\" != \"/bin\" -a \"${DIRNAME##*/}\" = \"ruby-api\" ]; then\n  # local mode, i.e. uninstalled\n  SCALARISCLIENT_RUBY=\"`dirname $0`/scalaris_client.rb\"\nfi\n\neval $SCALARISCLIENT_RUBY $args\n"
  },
  {
    "path": "ruby-api/scalaris.rb",
    "content": "#!/usr/bin/ruby -KU\n# Copyright 2008-2011 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nrequire 'rubygems'\nbegin\ngem 'json', '>=1.4.1'\nrescue LoadError\nend\nrequire 'json'\nrequire 'net/http'\nrequire 'base64'\nrequire 'open-uri'\n\nmodule InternalScalarisSimpleError\n  attr_reader :raw_result\n  def initialize(raw_result)\n    @raw_result = raw_result\n  end\n\n  def to_s\n    @raw_result\n  end\nend\n\nmodule InternalScalarisNopClose\n  # No operation (may be used for measuring the JSON overhead).\n  def nop(value)\n    value = @conn.class.encode_value(value)\n    result = @conn.call(:nop, [value])\n    @conn.class.process_result_nop(result)\n  end\n  \n  # Close the connection to Scalaris\n  # (it will automatically be re-opened on the next request).\n  def close_connection\n    @conn.close()\n  end\nend\n\n# work around floating point numbers not being printed precisely enough\nclass Float\n  # note: can not override to_json (this is not done recursively, e.g. in a Hash, before ruby 1.9)\n  alias_method :orig_t_s, :to_s\n  def to_s\n    if not finite?\n      orig_to_json(*a)\n    else\n      sprintf(\"%#.17g\", self)\n    end\n  end\nend\n\nmodule Scalaris\n  \n  # default URL and port to a scalaris node\n  if ENV.has_key?('SCALARIS_JSON_URL') and not ENV['SCALARIS_JSON_URL'].empty?\n    DEFAULT_URL = ENV['SCALARIS_JSON_URL']\n  else\n    DEFAULT_URL = 'http://localhost:8000'\n  end\n  \n  # path to the json rpc page\n  DEFAULT_PATH = '/jsonrpc.yaws'\n  \n  # Base class for errors in the scalaris package.\n  class ScalarisError < StandardError\n  end\n  \n  # Exception that is thrown if a the commit of a write operation on a Scalaris\n  # ring fails.\n  class AbortError < ScalarisError\n    attr_reader :raw_result\n    attr_reader :failed_keys\n    def initialize(raw_result, failed_keys)\n      @raw_result = raw_result\n      @failed_keys = failed_keys\n    end\n  \n    def to_s\n      @raw_result\n    end\n  end\n\n  # Exception that is thrown if an operation on a Scalaris ring fails because\n  # a connection does not exist or has been disconnected.\n  class ConnectionError < ScalarisError\n    include InternalScalarisSimpleError\n  end\n\n  # Exception that is thrown if a test_and_set operation on a Scalaris ring\n  # fails because the old value did not match the expected value.\n  class KeyChangedError < ScalarisError\n    attr_reader :raw_result\n    attr_reader :old_value\n    def initialize(raw_result, old_value)\n      @raw_result = raw_result\n      @old_value = old_value\n    end\n\n    def to_s\n      @raw_result + \", old value: \" + @old_value\n    end\n  end\n\n  # Exception that is thrown if a delete operation on a Scalaris ring fails\n  # because no Scalaris node was found.\n  class NodeNotFoundError < ScalarisError\n    include InternalScalarisSimpleError\n  end\n\n  # Exception that is thrown if a read operation on a Scalaris ring fails\n  # because the key did not exist before.\n  class NotFoundError < ScalarisError\n    include InternalScalarisSimpleError\n  end\n\n  # Exception that is thrown if a add_del_on_list operation on a scalaris ring\n  # fails because the participating values are not lists.\n  class NotAListError < ScalarisError\n    include InternalScalarisSimpleError\n  end\n\n  # Exception that is thrown if a add_del_on_list operation on a scalaris ring\n  # fails because the participating values are not numbers.\n  class NotANumberError < ScalarisError\n    include InternalScalarisSimpleError\n  end\n\n  # Exception that is thrown if a read or write operation on a Scalaris ring\n  # fails due to a timeout.\n  class TimeoutError < ScalarisError\n    include InternalScalarisSimpleError\n  end\n\n  # Generic exception that is thrown during operations on a Scalaris ring, e.g.\n  # if an unknown result has been returned.\n  class UnknownError < ScalarisError\n    include InternalScalarisSimpleError\n  end\n\n  # Stores the result of a delete operation.\n  class DeleteResult\n    attr_reader :ok\n    attr_reader :locks_set\n    attr_reader :undefined\n    def initialize(ok, locks_set, undefined)\n      @ok = ok\n      @locks_set = locks_set\n      @undefined = undefined\n    end\n  end\n  \n  # Abstracts connections to Scalaris using JSON\n  class JSONConnection\n    # Creates a JSON connection to the given URL using the given TCP timeout (or default)\n    def initialize(url = DEFAULT_URL, timeout = nil)\n      begin\n        @uri = URI.parse(url)\n        @timeout = timeout\n        start\n      rescue Exception => error\n        raise ConnectionError.new(error)\n      end\n    end\n    \n    def start\n      if @conn == nil or not @conn.started?\n        @conn = Net::HTTP.start(@uri.host, @uri.port)\n        unless @timeout.nil?\n          @conn.read_timeout = @timeout\n        end\n      end\n    end\n    private :start\n    \n    # Calls the given function with the given parameters via the JSON\n    # interface of Scalaris.\n    def call(function, params)\n      start\n      req = Net::HTTP::Post.new(DEFAULT_PATH)\n      req.add_field('Content-Type', 'application/json; charset=utf-8')\n      req.body = URI::encode({\n        :jsonrpc => :'2.0',\n        :method => function,\n        :params => params,\n        :id => 0 }.to_json({:ascii_only => true}))\n      begin\n        res = @conn.request(req)\n        if res.is_a?(Net::HTTPSuccess)\n          data = res.body\n          return JSON.parse(data)['result']\n        else\n          raise ConnectionError.new(res)\n        end\n      rescue ConnectionError => error\n        raise error\n      rescue Exception => error\n        raise ConnectionError.new(error)\n      end\n    end\n    \n    # Encodes the value to the form required by the Scalaris JSON API\n    def self.encode_value(value, binary = false)\n      if binary\n        return { :type => :as_bin, :value => Base64.encode64(value) }\n      else\n        return { :type => :as_is, :value => value }\n      end\n    end\n    \n    # Decodes the value from the Scalaris JSON API form to a native type\n    def self.decode_value(value)\n      if not (value.has_key?('type') and value.has_key?('value'))\n        raise ConnectionError.new(value)\n      end\n      if value['type'] == 'as_bin'\n        return Base64.decode64(value['value'])\n      else\n        return value['value']\n      end\n    end\n\n    # Processes the result of some Scalaris operation and raises a\n    # TimeoutError if found.\n    #\n    # result: {'status': 'ok'} or\n    #         {'status': 'fail', 'reason': 'timeout'}\n    def self.check_fail_abort(result)\n      if result == {:status => 'fail', :reason => 'timeout'}\n        raise TimeoutError.new(result)\n      end\n    end\n\n    # Processes the result of a read operation.\n    # Returns the read value on success.\n    # Raises the appropriate exception if the operation failed.\n    # \n    # result: {'status' => 'ok', 'value': xxx} or\n    #         {'status' => 'fail', 'reason' => 'timeout' or 'not_found'}\n    def self.process_result_read(result)\n      if result.is_a?(Hash) and result.has_key?('status') and result.length == 2\n        if result['status'] == 'ok' and result.has_key?('value')\n          return decode_value(result['value'])\n        elsif result['status'] == 'fail' and result.has_key?('reason')\n          if result['reason'] == 'timeout'\n            raise TimeoutError.new(result)\n          elsif result['reason'] == 'not_found'\n            raise NotFoundError.new(result)\n          end\n        end\n      end\n      raise UnknownError.new(result)\n    end\n\n    # Processes the result of a write operation.\n    # Raises the appropriate exception if the operation failed.\n    # \n    # result: {'status' => 'ok'} or\n    #         {'status' => 'fail', 'reason' => 'timeout'}\n    def self.process_result_write(result)\n      if result.is_a?(Hash)\n        if result == {'status' => 'ok'}\n          return true\n        elsif result == {'status' => 'fail', 'reason' => 'timeout'}\n          raise TimeoutError.new(result)\n        end\n      end\n      raise UnknownError.new(result)\n    end\n\n    # Processes the result of a commit operation.\n    # Raises the appropriate exception if the operation failed.\n    # \n    # result: {'status' => 'ok'} or\n    #         {'status' => 'fail', 'reason' => 'abort', 'keys' => <list>} or\n    #         {'status' => 'fail', 'reason' => 'timeout'}\n    def self.process_result_commit(result)\n      if result.is_a?(Hash) and result.has_key?('status')\n        if result == {'status' => 'ok'}\n          return true\n        elsif result['status'] == 'fail' and result.has_key?('reason')\n          if result.length == 2 and result['reason'] == 'timeout'\n            raise TimeoutError.new(result)\n          elsif result.length == 3 and result['reason'] == 'abort' and result.has_key?('keys')\n            raise AbortError.new(result, result['keys'])\n          end\n        end\n      end\n      raise UnknownError.new(result)\n    end\n\n    # Processes the result of a add_del_on_list operation.\n    # Raises the appropriate exception if the operation failed.\n    #\n    # results: {'status': 'ok'} or\n    #          {'status': 'fail', 'reason': 'timeout' or 'not_a_list'}\n    def self.process_result_add_del_on_list(result)\n      if result.is_a?(Hash) and result.has_key?('status')\n        if result == {'status' => 'ok'}\n          return nil\n        elsif result['status'] == 'fail' and result.has_key?('reason')\n          if result.length == 2\n            if result['reason'] == 'timeout'\n              raise TimeoutError.new(result)\n            elsif result['reason'] == 'not_a_list'\n              raise NotAListError.new(result)\n            end\n          end\n        end\n      end\n      raise UnknownError.new(result)\n    end\n\n    # Processes the result of a add_on_nr operation.\n    # Raises the appropriate exception if the operation failed.\n    #\n    # results: {'status': 'ok'} or\n    #          {'status': 'fail', 'reason': 'timeout' or 'not_a_number'}\n    def self.process_result_add_on_nr(result)\n      if result.is_a?(Hash) and result.has_key?('status')\n        if result == {'status' => 'ok'}\n          return nil\n        elsif result['status'] == 'fail' and result.has_key?('reason')\n          if result.length == 2\n            if result['reason'] == 'timeout'\n              raise TimeoutError.new(result)\n            elsif result['reason'] == 'not_a_number'\n              raise NotANumberError.new(result)\n            end\n          end\n        end\n      end\n      raise UnknownError.new(result)\n    end\n\n    # Processes the result of a test_and_set operation.\n    # Raises the appropriate exception if the operation failed.\n    # \n    # results: {'status' => 'ok'} or\n    #          {'status' => 'fail', 'reason' => 'timeout' or 'not_found'} or\n    #          {'status' => 'fail', 'reason' => 'key_changed', 'value': xxx}\n    def self.process_result_test_and_set(result)\n      if result.is_a?(Hash) and result.has_key?('status')\n        if result == {'status' => 'ok'}\n          return nil\n        elsif result['status'] == 'fail' and result.has_key?('reason')\n          if result.length == 2\n            if result['reason'] == 'timeout'\n              raise TimeoutError.new(result)\n            elsif result['reason'] == 'not_found'\n              raise NotFoundError.new(result)\n            end\n          elsif result['reason'] == 'key_changed' and result.has_key?('value') and result.length == 3\n            raise KeyChangedError.new(result, decode_value(result['value']))\n          end\n        end\n      end\n      raise UnknownError.new(result)\n    end\n\n    # Processes the result of a delete operation.\n    # Returns an Array of\n    # {:success => true | :timeout, :ok => <number of deleted items>, :results => <detailed results>}\n    # on success.\n    # Does not raise an exception if the operation failed unless the result\n    # is invalid!\n    # \n    # results: {'ok': xxx, 'results': ['ok' or 'locks_set' or 'undef']} or\n    #          {'failure': 'timeout', 'ok': xxx, 'results': ['ok' or 'locks_set' or 'undef']}\n    def self.process_result_delete(result)\n      if result.is_a?(Hash) and result.has_key?('ok') and result.has_key?('results')\n        if not result.has_key?('failure')\n          return {:success => true,\n            :ok => result['ok'],\n            :results => result['results']}\n        elsif result['failure'] == 'timeout'\n          return {:success => :timeout,\n            :ok => result['ok'],\n            :results => result['results']}\n        end\n      end\n      raise UnknownError.new(result)\n    end\n    \n    # Creates a new DeleteResult from the given result list.\n    # \n    # result: ['ok' or 'locks_set' or 'undef']\n    def self.create_delete_result(result)\n      ok = 0\n      locks_set = 0\n      undefined = 0\n      if result.is_a?(Array)\n        for element in result\n          if element == 'ok'\n              ok += 1\n          elsif element == 'locks_set'\n              locks_set += 1\n          elsif element == 'undef'\n              undefined += 1\n          else\n            raise UnknownError.new(:'Unknown reason ' + element + :'in ' + result)\n          end\n        end\n        return DeleteResult.new(ok, locks_set, undefined)\n      end\n      raise UnknownError.new(:'Unknown result ' + result)\n    end\n    \n    # Processes the result of a req_list operation of the Transaction class.\n    # Returns the Array (:tlog => <tlog>, :result => <result>) on success.\n    # Raises the appropriate exception if the operation failed.\n    # \n    # results: {'tlog': xxx,\n    #           'results': [{'status': 'ok'} or {'status': 'ok', 'value': xxx} or\n    #                       {'status': 'fail', 'reason': 'timeout' or 'abort' or 'not_found'}]}\n    def self.process_result_req_list_t(result)\n      if (not result.has_key?('tlog')) or (not result.has_key?('results')) or\n          (not result['results'].is_a?(Array))\n        raise UnknownError.new(result)\n      end\n      {:tlog => result['tlog'], :result => result['results']}\n    end\n\n    # Processes the result of a req_list operation of the TransactionSingleOp class.\n    # Returns <result> on success.\n    # Raises the appropriate exception if the operation failed.\n    # \n    # results: [{'status': 'ok'} or {'status': 'ok', 'value': xxx} or\n    #           {'status': 'fail', 'reason': 'timeout' or 'abort' or 'not_found'}]\n    def self.process_result_req_list_tso(result)\n      if not result.is_a?(Array)\n        raise UnknownError.new(result)\n      end\n      result\n    end\n\n    # Processes the result of a nop operation.\n    # Raises the appropriate exception if the operation failed.\n    # \n    # result: 'ok'\n    def self.process_result_nop(result)\n      if result != 'ok'\n        raise UnknownError.new(result)\n      end\n    end\n    \n    # Returns a new ReqList object allowing multiple parallel requests for\n    # the Transaction class.\n    def self.new_req_list_t(other = nil)\n      JSONReqListTransaction.new(other)\n    end\n    \n    # Returns a new ReqList object allowing multiple parallel requests for\n    # the TransactionSingleOp class.\n    def self.new_req_list_tso(other = nil)\n      JSONReqListTransactionSingleOp.new(other)\n    end\n\n    def close\n      if @conn.started?\n        @conn.finish()\n      end\n    end\n  end\n  \n  # Single write or read operations on Scalaris.\n  class TransactionSingleOp\n    # Create a new object using the given connection\n    def initialize(conn = JSONConnection.new())\n      @conn = conn\n    end\n    \n    # Returns a new ReqList object allowing multiple parallel requests.\n    def new_req_list(other = nil)\n      @conn.class.new_req_list_t(other)\n    end\n    \n    # Issues multiple parallel requests to scalaris; each will be committed.\n    # NOTE: The execution order of multiple requests on the same key is\n    # undefined!\n    # Request lists can be created using new_req_list().\n    # The returned list has the following form:\n    # [{'status': 'ok'} or {'status': 'ok', 'value': xxx} or\n    # {'status': 'fail', 'reason': 'timeout' or 'abort' or 'not_found'}].\n    # Elements of this list can be processed with process_result_read() and\n    # process_result_write().\n    def req_list(reqlist)\n      result = @conn.call(:req_list_commit_each, [reqlist.get_requests()])\n      @conn.class.process_result_req_list_tso(result)\n    end\n\n    # Processes a result element from the list returned by req_list() which\n    # originated from a read operation.\n    # Returns the read value on success.\n    # Raises the appropriate exceptions if a failure occurred during the\n    # operation.\n    # Beware: lists of (small) integers may be (falsely) returned as a string -\n    # use str_to_list() to convert such strings.\n    def process_result_read(result)\n      @conn.class.process_result_read(result)\n    end\n\n    # Processes a result element from the list returned by req_list() which\n    # originated from a write operation.\n    # Raises the appropriate exceptions if a failure occurred during the\n    # operation.\n    def process_result_write(result)\n      # note: we need to process a commit result as the write has been committed\n      @conn.class.check_fail_abort(result)\n      @conn.class.process_result_commit(result)\n    end\n    \n    # Processes a result element from the list returned by req_list() which\n    # originated from a add_del_on_list operation.\n    # Raises the appropriate exceptions if a failure occurred during the\n    # operation.\n    def process_result_add_del_on_list(result)\n      @conn.class.check_fail_abort(result)\n      @conn.class.process_result_add_del_on_list(result)\n    end\n\n    # Processes a result element from the list returned by req_list() which\n    # originated from a add_on_nr operation.\n    # Raises the appropriate exceptions if a failure occurred during the\n    # operation.\n    def process_result_add_on_nr(result)\n      @conn.class.check_fail_abort(result)\n      @conn.class.process_result_add_on_nr(result)\n    end\n\n    # Processes a result element from the list returned by req_list() which\n    # originated from a test_and_set operation.\n    # Raises the appropriate exceptions if a failure occurred during the\n    # operation.\n    def process_result_test_and_set(result)\n      @conn.class.check_fail_abort(result)\n      @conn.class.process_result_test_and_set(result)\n    end\n\n    # Read the value at key.\n    # Beware: lists of (small) integers may be (falsely) returned as a string -\n    # use str_to_list() to convert such strings.\n    def read(key)\n      result = @conn.call(:read, [key])\n      @conn.class.process_result_read(result)\n    end\n\n    # Write the value to key.\n    def write(key, value, binary = false)\n      value = @conn.class.encode_value(value, binary)\n      result = @conn.call(:write, [key, value])\n      @conn.class.check_fail_abort(result)\n      @conn.class.process_result_commit(result)\n    end\n\n    # Changes the list stored at the given key, i.e. first adds all items in\n    # to_add then removes all items in to_remove.\n    # Both, to_add and to_remove, must be lists.\n    # Assumes en empty list if no value exists at key.\n    def add_del_on_list(key, to_add, to_remove)\n      result = @conn.call(:add_del_on_list, [key, to_add, to_remove])\n      @conn.class.check_fail_abort(result)\n      @conn.class.process_result_add_del_on_list(result)\n    end\n\n    # Changes the number stored at the given key, i.e. adds some value.\n    # Assumes 0 if no value exists at key.\n    def add_on_nr(key, to_add)\n      result = @conn.call(:add_on_nr, [key, to_add])\n      @conn.class.check_fail_abort(result)\n      @conn.class.process_result_add_on_nr(result)\n    end\n\n    # Atomic test and set, i.e. if the old value at key is old_value, then\n    # write new_value.\n    def test_and_set(key, old_value, new_value)\n      result = @conn.call(:test_and_set, [key, old_value, new_value])\n      @conn.class.check_fail_abort(result)\n      @conn.class.process_result_test_and_set(result)\n    end\n\n    # Atomic test and set, i.e. if the old value at key is oldvalue, then\n    # write newvalue.\n    def test_and_set(key, oldvalue, newvalue)\n      oldvalue = @conn.class.encode_value(oldvalue)\n      newvalue = @conn.class.encode_value(newvalue)\n      result = @conn.call(:test_and_set, [key, oldvalue, newvalue])\n      @conn.class.check_fail_abort(result)\n      @conn.class.process_result_test_and_set(result)\n    end\n\n    include InternalScalarisNopClose\n  end\n\n  # Write or read operations on Scalaris inside a transaction.\n  class Transaction\n    # Create a new object using the given connection\n    def initialize(conn = JSONConnection.new())\n      @conn = conn\n      @tlog = nil\n    end\n    \n    # Returns a new ReqList object allowing multiple parallel requests.\n    def new_req_list(other = nil)\n      @conn.class.new_req_list_t(other)\n    end\n    \n    # Issues multiple parallel requests to Scalaris.\n    # Request lists can be created using new_req_list().\n    # The returned list has the following form:\n    # [{'status': 'ok'} or {'status': 'ok', 'value': xxx} or\n    # {'status': 'fail', 'reason': 'timeout' or 'abort' or 'not_found'}].\n    # The elements of this list can be processed with process_result_read(),\n    # process_result_write() and process_result_commit().\n    def req_list(reqlist)\n      if @tlog == nil\n        result = @conn.call(:req_list, [reqlist.get_requests()])\n      else\n        result = @conn.call(:req_list, [@tlog, reqlist.get_requests()])\n      end\n      result = @conn.class.process_result_req_list_t(result)\n      @tlog = result[:tlog]\n      result = result[:result]\n      if reqlist.is_commit()\n        _process_result_commit(result[-1])\n        # transaction was successful: reset transaction log\n        @tlog = nil\n      end\n      result\n    end\n    \n    # Processes a result element from the list returned by req_list() which\n    # originated from a read operation.\n    # Returns the read value on success.\n    # Raises the appropriate exceptions if a failure occurred during the\n    # operation.\n    # Beware: lists of (small) integers may be (falsely) returned as a string -\n    # use str_to_list() to convert such strings.\n    def process_result_read(result)\n      @conn.class.process_result_read(result)\n    end\n    \n    # Processes a result element from the list returned by req_list() which\n    # originated from a write operation.\n    # Raises the appropriate exceptions if a failure occurred during the\n    # operation.\n    def process_result_write(result)\n      @conn.class.process_result_write(result)\n    end\n\n    # Processes a result element from the list returned by req_list() which\n    # originated from a add_del_on_list operation.\n    # Raises the appropriate exceptions if a failure occurred during the\n    # operation.\n    def process_result_add_del_on_list(result)\n      @conn.class.process_result_add_del_on_list(result)\n    end\n\n    # Processes a result element from the list returned by req_list() which\n    # originated from a add_on_nr operation.\n    # Raises the appropriate exceptions if a failure occurred during the\n    # operation.\n    def process_result_add_on_nr(result)\n      @conn.class.process_result_add_on_nr(result)\n    end\n\n    # Processes a result element from the list returned by req_list() which\n    # originated from a test_and_set operation.\n    # Raises the appropriate exceptions if a failure occurred during the\n    # operation.\n    def process_result_test_and_set(result)\n      @conn.class.process_result_test_and_set(result)\n    end\n    \n    # Processes a result element from the list returned by req_list() which\n    # originated from a commit operation.\n    # Raises the appropriate exceptions if a failure occurred during the\n    # operation.\n    def _process_result_commit(result)\n      @conn.class.process_result_commit(result)\n    end\n\n    private :_process_result_commit\n    \n    # Issues a commit operation to Scalaris validating the previously\n    # created operations inside the transaction.\n    def commit\n      result = req_list(new_req_list().add_commit())[0]\n      _process_result_commit(result)\n      # reset tlog (minor optimization which is not done in req_list):\n      @tlog = nil\n    end\n    \n    # Aborts all previously created operations inside the transaction.\n    def abort\n      @tlog = nil\n    end\n    \n    # Issues a read operation to Scalaris, adds it to the current\n    # transaction and returns the result.\n    # Beware: lists of (small) integers may be (falsely) returned as a string -\n    # use str_to_list() to convert such strings.\n    def read(key)\n      result = req_list(new_req_list().add_read(key))[0]\n      return process_result_read(result)\n    end\n    \n    # Issues a write operation to Scalaris and adds it to the current\n    # transaction.\n    def write(key, value, binary = false)\n      result = req_list(new_req_list().add_write(key, value, binary))[0]\n      _process_result_commit(result)\n    end\n\n    # Issues a add_del_on_list operation to scalaris and adds it to the\n    # current transaction.\n    # Changes the list stored at the given key, i.e. first adds all items in\n    # to_add then removes all items in to_remove.\n    # Both, to_add and to_remove, must be lists.\n    # Assumes en empty list if no value exists at key.\n    def add_del_on_list(key, to_add, to_remove)\n      result = req_list(new_req_list().add_add_del_on_list(key, to_add, to_remove))[0]\n      process_result_add_del_on_list(result)\n    end\n\n    # Issues a add_on_nr operation to scalaris and adds it to the\n    # current transaction.\n    # Changes the number stored at the given key, i.e. adds some value.\n    # Assumes 0 if no value exists at key.\n    def add_on_nr(key, to_add)\n      result = req_list(new_req_list().add_add_on_nr(key, to_add))[0]\n      process_result_add_on_nr(result)\n    end\n\n    # Issues a test_and_set operation to scalaris and adds it to the\n    # current transaction.\n    # Atomic test and set, i.e. if the old value at key is old_value, then\n    # write new_value.\n    def test_and_set(key, old_value, new_value)\n      result = req_list(new_req_list().add_test_and_set(key, old_value, new_value))[0]\n      process_result_test_and_set(result)\n    end\n\n    include InternalScalarisNopClose\n  end\n\n  # Request list for use with Transaction.req_list()\n  class JSONReqList\n    # Create a new object using a JSON connection.\n    def initialize(other = nil)\n      @requests = []\n      @is_commit = false\n      if not other == nil\n        concat(other)\n      end\n    end\n    \n    # Adds a read operation to the request list.\n    def add_read(key)\n      if (@is_commit)\n          raise RuntimeError.new('No further request supported after a commit!')\n      end\n      @requests << {'read' => key}\n      self\n    end\n    \n    # Adds a write operation to the request list.\n    def add_write(key, value, binary = false)\n      if (@is_commit)\n          raise RuntimeError.new('No further request supported after a commit!')\n      end\n      @requests << {'write' => {key => JSONConnection.encode_value(value, binary)}}\n      self\n    end\n\n    # Adds a add_del_on_list operation to the request list.\n    def add_add_del_on_list(key, to_add, to_remove)\n      if (@is_commit)\n          raise RuntimeError.new('No further request supported after a commit!')\n      end\n      @requests << {'add_del_on_list' => {'key' => key, 'add' => to_add, 'del'=> to_remove}}\n      self\n    end\n\n    # Adds a add_on_nr operation to the request list.\n    def add_add_on_nr(key, to_add)\n      if (@is_commit)\n          raise RuntimeError.new('No further request supported after a commit!')\n      end\n      @requests << {'add_on_nr' => {key => to_add}}\n      self\n    end\n\n    # Adds a test_and_set operation to the request list.\n    def add_test_and_set(key, old_value, new_value)\n      if (@is_commit)\n          raise RuntimeError.new('No further request supported after a commit!')\n      end\n      @requests << {'test_and_set' => {'key' => key,\n          'old' => JSONConnection.encode_value(old_value, false),\n          'new' => JSONConnection.encode_value(new_value, false)}}\n      self\n    end\n\n    # Adds a commit operation to the request list.\n    def add_commit\n      if (@is_commit)\n          raise RuntimeError.new('Only one commit per request list allowed!')\n      end\n      @requests << {'commit' => ''}\n      @is_commit = true\n      self\n    end\n    \n    # Gets the collected requests.\n    def get_requests\n      @requests\n    end\n\n    # Returns whether the transactions contains a commit or not.\n    def is_commit()\n      @is_commit\n    end\n    \n    # Checks whether the request list is empty.\n    def is_empty()\n      @requests.empty?\n    end\n    \n    # Gets the number of requests in the list.\n    def size()\n      @requests.length\n    end\n\n    # Adds all requests of the other request list to the end of this list.\n    def concat(other)\n      @requests.concat(other.get_requests())\n      self\n    end\n  end\n\n  # Request list for use with Transaction.req_list().\n  class JSONReqListTransaction < JSONReqList\n    def initialize(other = nil)\n      super(other)\n    end\n  end\n  \n  # Request list for use with TransactionSingleOp.req_list() which does not\n  # support commits.\n  class JSONReqListTransactionSingleOp < JSONReqList\n    def initialize(other = nil)\n      super(other)\n    end\n    \n    # Adds a commit operation to the request list.\n    def add_commit()\n      raise RuntimeError.new('No commit allowed in TransactionSingleOp.req_list()!')\n    end\n  end\n\n  # Non-transactional operations on the replicated DHT of Scalaris\n  class ReplicatedDHT\n    # Create a new object using the given connection.\n    def initialize(conn = JSONConnection.new())\n      @conn = conn\n    end\n    \n    # Tries to delete the value at the given key.\n    # \n    # WARNING: This function can lead to inconsistent data (e.g. deleted items\n    # can re-appear). Also when re-creating an item the version before the\n    # delete can re-appear.\n    # \n    # returns the number of successfully deleted items\n    # use get_last_delete_result() to get more details\n    def delete(key, timeout = 2000)\n      result_raw = @conn.call(:delete, [key, timeout])\n      result = @conn.class.process_result_delete(result_raw)\n      @lastDeleteResult = result[:results]\n      if result[:success] == true\n        return result[:ok]\n      elsif result[:success] == :timeout\n        raise TimeoutError.new(result_raw)\n      else\n        raise UnknownError.new(result_raw)\n      end\n    end\n    \n    # Returns the result of the last call to delete().\n    # \n    # NOTE: This function traverses the result list returned by Scalaris and\n    # therefore takes some time to process. It is advised to store the returned\n    # result object once generated.\n    def get_last_delete_result\n      @conn.class.create_delete_result(@lastDeleteResult)\n    end\n\n    include InternalScalarisNopClose\n  end\n\n  # API for using routing tables\n  class RoutingTable\n    # Create a new object using the given connection.\n    def initialize(conn = JSONConnection.new())\n      @conn = conn\n    end\n\n    def get_replication_factor()\n      result_raw = @conn.call(:get_replication_factor, [])\n      if result_raw.is_a?(Hash) and result_raw.has_key?('status') and result_raw.has_key?('value')\n        return result_raw['value']\n      else\n        raise UnknownError.new(result)\n      end\n    end\n\n    include InternalScalarisNopClose\n  end\n\n  # Converts a string to a list of integers.\n  # If the expected value of a read operation is a list, the returned value\n  # could be (mistakenly) a string if it is a list of integers.\n  def str_to_list(value)\n    if value.is_a?(String)\n      return value.unpack(\"U*\")\n    else\n      return value\n    end\n  end\n  \n  module_function :str_to_list\nend\n"
  },
  {
    "path": "ruby-api/scalaris_client.rb",
    "content": "#!/usr/bin/ruby -KU\n# Copyright 2008-2011 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nrequire 'rubygems'\nbegin\ngem 'json', '>=1.4.1'\nrescue LoadError\nend\nrequire 'json'\nrequire 'optparse'\nrequire 'pp'\nbegin\n  require \"#{File.expand_path(File.dirname(__FILE__))}/scalaris\"\nrescue LoadError => e\n  raise unless e.message =~ /scalaris/\n  require \"scalaris\"\nend\n\ndef get_replication_factor()\n  (Scalaris::RoutingTable.new).get_replication_factor\nend\n\ndef write(sc, key_value_list)\n  key, value = key_value_list\n  sc.write(key, value)\nend\n\ndef test_and_set(sc, key_value_list)\n  key, old_value, new_value = key_value_list\n  sc.test_and_set(key, old_value, new_value)\nend\n\ndef add_on_nr(sc, key_value_list)\n  begin\n    key, to_add = key_value_list\n    sc.add_on_nr(key, to_add)\n  rescue Scalaris::NotANumberError\n    $stderr.print \"Error: \" + $!.to_s.to_json + \"\\n\"\n    exit\n  end\nend\n\ndef add_del_on_list(sc, key_value_list)\n  begin\n    key, to_add, to_remove = key_value_list\n    sc.add_del_on_list(key, to_add, to_remove)\n  rescue Scalaris::NotAListError\n    $stderr.print \"Error: \" + $!.to_s.to_json + \"\\n\"\n    exit\n  end\nend\n\noptions = {}\n\noptparse = OptionParser.new do |opts|\n  options[:help] = true\n\n  options[:replication_factor] = nil\n  opts.on('-f', '--get-replication-factor', 'get the replication factor' ) do |f|\n    options[:factor] = f\n    options[:help] = false\n  end\n\n  options[:read] = nil\n  opts.on('-r', '--read KEY', 'read key KEY' ) do |key|\n    options[:read] = key\n    options[:help] = false\n  end\n\n  options[:write] = nil\n  opts.on('-w', '--write KEY,VALUE', Array, 'write key KEY to VALUE' ) do |list|\n    raise OptionParser::InvalidOption.new(list) unless list.size == 2\n    options[:write] = list\n    options[:help] = false\n  end\n\n  options[:test_and_set] = nil\n  opts.on('--test-and-set KEY,OLDVALUE,NEWVALUE', Array, 'write key KEY to NEWVALUE if the current value is OLDVALUE' ) do |list|\n    raise OptionParser::InvalidOption.new(list) unless list.size == 3\n    options[:test_and_set] = list\n    options[:help] = false\n  end\n\n  options[:add_del_on_list] = nil\n  opts.on('--add-del-on-list KEY,TOADD,TOREMOVE', Array, 'add and remove elements from the value at key KEY' ) do |list|\n    raise OptionParser::InvalidOption.new(list) unless list.size == 3\n    options[:add_del_on_list] = list\n    options[:help] = false\n  end\n\n  options[:add_on_nr] = nil\n  opts.on('--add-on-nr KEY,VALUE', Array, 'add VALUE to the value at key KEY' ) do |list|\n    raise OptionParser::InvalidOption.new(list) unless list.size == 2\n    options[:add_on_nr] = list\n    options[:help] = false\n  end\n\n  opts.on_tail(\"-h\", \"--help\", \"Show this message\") do\n    puts opts\n    exit\n  end\nend\n\nbegin\n  optparse.parse!\nrescue OptionParser::ParseError\n  $stderr.print \"Error: \" + $! + \"\\n\"\n  exit\nend\n\nsc = nil\ncommunicating_options = [options[:read], options[:write], options[:test_and_set], \n                         options[:add_on_nr], options[:add_del_on_list]]\n\nif communicating_options.compact != []\n  sc = Scalaris::TransactionSingleOp.new\nend\n\npp sc.read(options[:read]) unless options[:read] == nil\npp write(sc, options[:write]) unless options[:write] == nil\npp test_and_set(sc, options[:test_and_set]) unless options[:test_and_set] == nil\npp add_on_nr(sc, options[:add_on_nr]) unless options[:add_on_nr] == nil\npp add_del_on_list(sc, options[:add_del_on_list]) unless options[:add_del_on_list] == nil\npp get_replication_factor() unless options[:factor] == nil\nputs optparse if options[:help]\n"
  },
  {
    "path": "ruby-api/scalaris_interop_test.rb",
    "content": "#!/usr/bin/ruby -KU\n# Copyright 2011 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nbegin\n  require \"#{File.expand_path(File.dirname(__FILE__))}/scalaris\"\nrescue LoadError => e\n  raise unless e.message =~ /scalaris/\n  require \"scalaris\"\nend\n\ndef read_or_write(sc, key, value, mode, binary = false)\n  begin\n    if (mode == :read)\n      actual = sc.read(key)\n      # if the expected value is a list, the returned value could by (mistakenly) a string if it is a list of integers\n      # -> convert such a string to a list\n      if value.is_a?(Array)\n        begin\n            actual = Scalaris::str_to_list(actual)\n        rescue Exception => e\n            puts 'fail'\n            return 1\n        end\n      end\n      if (actual == value)\n        out = $stdout\n        result = 0\n        result_str = 'ok'\n      else\n        out = $stderr\n        result = 1\n        result_str = 'fail'\n      end\n\n      out.puts 'read(' + key.to_s + ')'\n      out.puts '  expected: ' + value.inspect\n      out.puts '  read raw: ' + actual.inspect\n      out.puts '   read rb: ' + actual.inspect\n      out.puts result_str\n      return result\n    elsif (mode == :write)\n      puts 'write(' + key.to_s + ', ' + value.to_s + ')'\n      sc.write(key, value, binary)\n      return 0\n    end\n  rescue Scalaris::ConnectionError => instance\n    puts 'failed with connection error'\n    return 1\n  rescue Scalaris::TimeoutError => instance\n    puts 'failed with timeout'\n    return 1\n  rescue Scalaris::AbortError => instance\n    puts 'failed with abort'\n    return 1\n  rescue Scalaris::NotFoundError => instance\n    puts 'failed with not_found'\n    return 1\n  rescue Scalaris::UnknownError => instance\n    puts 'failed with unknown: ' + instance.inspect\n    return 1\n  rescue Exception => instance\n    puts 'failed with ' + instance.inspect\n    return 1\n  end\nend\n\ndef read_write_boolean(basekey, sc, mode)\n  failed = 0\n  \n  failed += read_or_write(sc, basekey + \"_bool_false\", false, mode)\n  failed += read_or_write(sc, basekey + \"_bool_true\", true, mode)\n  \n  return failed\nend\n\ndef read_write_integer(basekey, sc, mode)\n  failed = 0\n  \n  failed += read_or_write(sc, basekey + \"_int_0\", 0, mode)\n  failed += read_or_write(sc, basekey + \"_int_1\", 1, mode)\n  failed += read_or_write(sc, basekey + \"_int_min\", -2147483648, mode)\n  failed += read_or_write(sc, basekey + \"_int_max\", 2147483647, mode)\n  failed += read_or_write(sc, basekey + \"_int_max_div_2\", 2147483647 / 2, mode)\n  \n  return failed\nend\n\ndef read_write_long(basekey, sc, mode)\n  failed = 0\n  \n  failed += read_or_write(sc, basekey + \"_long_0\", 0, mode)\n  failed += read_or_write(sc, basekey + \"_long_1\", 1, mode)\n  failed += read_or_write(sc, basekey + \"_long_min\", -9223372036854775808, mode)\n  failed += read_or_write(sc, basekey + \"_long_max\", 9223372036854775807, mode)\n  failed += read_or_write(sc, basekey + \"_long_max_div_2\", 9223372036854775807 / 2, mode)\n  \n  return failed\nend\n  \ndef read_write_biginteger(basekey, sc, mode)\n  failed = 0\n  \n  failed += read_or_write(sc, basekey + \"_bigint_0\", 0, mode)\n  failed += read_or_write(sc, basekey + \"_bigint_1\", 1, mode)\n  failed += read_or_write(sc, basekey + \"_bigint_min\", -100000000000000000000, mode)\n  failed += read_or_write(sc, basekey + \"_bigint_max\", 100000000000000000000, mode)\n  failed += read_or_write(sc, basekey + \"_bigint_max_div_2\", 100000000000000000000 / 2, mode)\n  \n  return failed\nend\n\ndef read_write_double(basekey, sc, mode)\n  failed = 0\n  \n  failed += read_or_write(sc, basekey + \"_float_0.0\", 0.0, mode)\n  failed += read_or_write(sc, basekey + \"_float_1.5\", 1.5, mode)\n  failed += read_or_write(sc, basekey + \"_float_-1.5\", -1.5, mode)\n  failed += read_or_write(sc, basekey + \"_float_min\", 4.9E-324, mode)\n  failed += read_or_write(sc, basekey + \"_float_max\", 1.7976931348623157E308, mode)\n  failed += read_or_write(sc, basekey + \"_float_max_div_2\", 1.7976931348623157E308 / 2.0, mode)\n  \n  # not supported by erlang:\n  #failed += read_or_write(sc, basekey + \"_float_neg_inf\", float('-inf'), mode)\n  #failed += read_or_write(sc, basekey + \"__float_pos_inf\", float('+inf'), mode)\n  #failed += read_or_write(sc, basekey + \"_float_nan\", float('nan'), mode)\n  \n  return failed\nend\n\ndef read_write_string(basekey, sc, mode)\n  failed = 0\n  \n  failed += read_or_write(sc, basekey + \"_string_empty\", '', mode)\n  failed += read_or_write(sc, basekey + \"_string_foobar\", 'foobar', mode)\n  failed += read_or_write(sc, basekey + \"_string_foo\\\\nbar\", \"foo\\nbar\", mode)\n  failed += read_or_write(sc, basekey + \"_string_unicode\", 'foo' + [0x0180, 0x01E3, 0x11E5].pack(\"U*\"), mode)\n  \n  return failed\nend\n\ndef read_write_binary(basekey, sc, mode)\n  failed = 0\n  \n  # note: binary not supported by JSON\n  failed += read_or_write(sc, basekey + \"_byte_empty\", [].pack(\"c*\"), mode, true)\n  failed += read_or_write(sc, basekey + \"_byte_0\", [0].pack(\"c*\"), mode, true)\n  failed += read_or_write(sc, basekey + \"_byte_0123\", [0, 1, 2, 3].pack(\"c*\"), mode, true)\n  \n  return failed\nend\n\ndef read_write_list(basekey, sc, mode)\n  failed = 0\n  \n  failed += read_or_write(sc, basekey + \"_list_empty\", [], mode)\n  failed += read_or_write(sc, basekey + \"_list_0_1_2_3\", [0, 1, 2, 3], mode)\n  failed += read_or_write(sc, basekey + \"_list_0_123_456_65000\", [0, 123, 456, 65000], mode)\n  failed += read_or_write(sc, basekey + \"_list_0_123_456_0x10ffff\", [0, 123, 456, 0x10ffff], mode)\n  list4 = [0, 'foo', 1.5, false]\n  failed += read_or_write(sc, basekey + \"_list_0_foo_1.5_false\", list4, mode)\n  # note: binary not supported in lists\n  #failed += read_or_write(sc, basekey + \"_list_0_foo_1.5_<<0123>>\", [0, 'foo', 1.5, bytearray([0, 1, 2, 3])], mode)\n  failed += read_or_write(sc, basekey + \"_list_0_foo_1.5_false_list4\", [0, 'foo', 1.5, false, list4], mode)\n  failed += read_or_write(sc, basekey + \"_list_0_foo_1.5_false_list4_map_x=0_y=1\", [0, 'foo', 1.5, false, list4, {'x' => 0, 'y' => 1}], mode)\n  \n  return failed\nend\n\ndef read_write_map(basekey, sc, mode)\n  failed = 0\n  \n  failed += read_or_write(sc, basekey + \"_map_empty\", {}, mode)\n  failed += read_or_write(sc, basekey + \"_map_x=0_y=1\", {'x' => 0, 'y' => 1}, mode)\n  failed += read_or_write(sc, basekey + \"_map_a=0_b=foo_c=1.5_d=foo<nl>bar_e=list0123_f=mapx0y1\",\n    {'a' => 0, 'b' => 'foo', 'c' => 1.5, 'd' => \"foo\\nbar\", 'e' => [0, 1, 2, 3], 'f' => {'x' => 0, 'y' => 1}}, mode)\n  failed += read_or_write(sc, basekey + \"_map_=0\", {'' => 0}, mode)\n  failed += read_or_write(sc, basekey + '_map_x=0_y=foo' + [0x0180, 0x01E3, 0x11E5].pack(\"U*\"),\n    {'x' => 0, 'y' => 'foo' + [0x0180, 0x01E3, 0x11E5].pack(\"U*\")}, mode)\n  failed += read_or_write(sc, basekey + '_map_x=0_foo' + [0x0180, 0x01E3, 0x11E5].pack(\"U*\") + '=1',\n    {'x' => 0, 'foo' + [0x0180, 0x01E3, 0x11E5].pack(\"U*\") => 1}, mode)\n  \n  return failed\nend\n\nif (ARGV[0] == \"read\")\n  basekey = ARGV[1]\n  language = ARGV[2]\n  basekey += '_' + language\n  mode = :read\n  puts 'Ruby-JSON-API: reading from ' + language\nelsif (ARGV[0] == \"write\")\n  basekey = ARGV[1]\n  basekey += '_json_ruby'\n  mode = :write\n  puts 'Ruby-JSON-API: writing values'\nelse\n  puts 'unknown commands: ' + ARGV.to_s\n  exit 1\nend\n\nsc = Scalaris::TransactionSingleOp.new\n\nfailed = 0\nfailed += read_write_boolean(basekey, sc, mode)\nfailed += read_write_integer(basekey, sc, mode)\nfailed += read_write_long(basekey, sc, mode)\nfailed += read_write_biginteger(basekey, sc, mode)\nfailed += read_write_double(basekey, sc, mode)\nfailed += read_write_string(basekey, sc, mode)\nfailed += read_write_binary(basekey, sc, mode)\nfailed += read_write_list(basekey, sc, mode)\nfailed += read_write_map(basekey, sc, mode)\nputs ''\n\nif (failed > 0)\n  puts failed.to_s + ' number of ' + mode.to_s + 's failed.'\n  puts ''\n  exit 1\nend\n"
  },
  {
    "path": "ruby-api/scalaris_test.rb",
    "content": "#!/usr/bin/ruby -KU\n# Copyright 2011 Zuse Institute Berlin\n#\n#    Licensed under the Apache License, Version 2.0 (the \"License\");\n#    you may not use this file except in compliance with the License.\n#    You may obtain a copy of the License at\n#\n#        http://www.apache.org/licenses/LICENSE-2.0\n#\n#    Unless required by applicable law or agreed to in writing, software\n#    distributed under the License is distributed on an \"AS IS\" BASIS,\n#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#    See the License for the specific language governing permissions and\n#    limitations under the License.\n\nrequire \"test/unit\"\nbegin\n  require \"#{File.expand_path(File.dirname(__FILE__))}/scalaris\"\nrescue LoadError => e\n  raise unless e.message =~ /scalaris/\n  require \"scalaris\"\nend\n\n$_TEST_DATA = [\n             \"ahz2ieSh\", \"wooPhu8u\", \"quai9ooK\", \"Oquae4ee\", \"Airier1a\", \"Boh3ohv5\", \"ahD3Saog\", \"EM5ooc4i\", \n             \"Epahrai8\", \"laVahta7\", \"phoo6Ahj\", \"Igh9eepa\", \"aCh4Lah6\", \"ooT0ath5\", \"uuzau4Ie\", \"Iup6mae6\", \n#             \"xie7iSie\", \"ail8yeeP\", \"ooZ4eesi\", \"Ahn7ohph\", \"Ohy5moo6\", \"xooSh9Oo\", \"ieb6eeS7\", \"Thooqu9h\", \n#             \"eideeC9u\", \"phois3Ie\", \"EimaiJ2p\", \"sha6ahR1\", \"Pheih3za\", \"bai4eeXe\", \"rai0aB7j\", \"xahXoox6\", \n#             \"Xah4Okeg\", \"cieG8Yae\", \"Pe9Ohwoo\", \"Eehig6ph\", \"Xe7rooy6\", \"waY2iifu\", \"kemi8AhY\", \"Che7ain8\", \n#             \"ohw6seiY\", \"aegh1oBa\", \"thoh9IeG\", \"Kee0xuwu\", \"Gohng8ee\", \"thoh9Chi\", \"aa4ahQuu\", \"Iesh5uge\", \n#             \"Ahzeil8n\", \"ieyep5Oh\", \"xah3IXee\", \"Eefa5qui\", \"kai8Muuf\", \"seeCe0mu\", \"cooqua5Y\", \"Ci3ahF6z\", \n#             \"ot0xaiNu\", \"aewael8K\", \"aev3feeM\", \"Fei7ua5t\", \"aeCa6oph\", \"ag2Aelei\", \"Shah1Pho\", \"ePhieb0N\", \n#             \"Uqu7Phup\", \"ahBi8voh\", \"oon3aeQu\", \"Koopa0nu\", \"xi0quohT\", \"Oog4aiph\", \"Aip2ag5D\", \"tirai7Ae\", \n#             \"gi0yoePh\", \"uay7yeeX\", \"aeb6ahC1\", \"OoJeic2a\", \"ieViom1y\", \"di0eeLai\", \"Taec2phe\", \"ID2cheiD\", \n#             \"oi6ahR5M\", \"quaiGi8W\", \"ne1ohLuJ\", \"DeD0eeng\", \"yah8Ahng\", \"ohCee2ie\", \"ecu1aDai\", \"oJeijah4\", \n#             \"Goo9Una1\", \"Aiph3Phi\", \"Ieph0ce5\", \"ooL6cae7\", \"nai0io1H\", \"Oop2ahn8\", \"ifaxae7O\", \"NeHai1ae\", \n#             \"Ao8ooj6a\", \"hi9EiPhi\", \"aeTh9eiP\", \"ao8cheiH\", \"Yieg3sha\", \"mah7cu2D\", \"Uo5wiegi\", \"Oowei0ya\", \n#             \"efeiDee7\", \"Oliese6y\", \"eiSh1hoh\", \"Joh6hoh9\", \"zib6Ooqu\", \"eejiJie4\", \"lahZ3aeg\", \"keiRai1d\", \n#             \"Fei0aewe\", \"aeS8aboh\", \"hae3ohKe\", \"Een9ohQu\", \"AiYeeh7o\", \"Yaihah4s\", \"ood4Giez\", \"Oumai7te\", \n#             \"hae2kahY\", \"afieGh4v\", \"Ush0boo0\", \"Ekootee5\", \"Ya8iz6Ie\", \"Poh6dich\", \"Eirae4Ah\", \"pai8Eeme\", \n#             \"uNah7dae\", \"yo3hahCh\", \"teiTh7yo\", \"zoMa5Cuv\", \"ThiQu5ax\", \"eChi5caa\", \"ii9ujoiV\", \"ge7Iekui\",\n             \"sai2aiTa\", \"ohKi9rie\", \"ei2ioChu\", \"aaNgah9y\", \"ooJai1Ie\", \"shoh0oH9\", \"Ool4Ahya\", \"poh0IeYa\", \n             \"Uquoo0Il\", \"eiGh4Oop\", \"ooMa0ufe\", \"zee6Zooc\", \"ohhao4Ah\", \"Uweekek5\", \"aePoos9I\", \"eiJ9noor\", \n             \"phoong1E\", \"ianieL2h\", \"An7ohs4T\", \"Eiwoeku3\", \"sheiS3ao\", \"nei5Thiw\", \"uL5iewai\", \"ohFoh9Ae\"]\n\n$_TOO_LARGE_REQUEST_SIZE = 1024*1024*10 # number of bytes\n\nclass TestTransactionSingleOp < Test::Unit::TestCase\n  def setup\n    @testTime = (Time.now.to_f * 1000).to_i\n  end\n  \n  # Test method for TransactionSingleOp()\n  def testTransactionSingleOp1()\n    conn = Scalaris::TransactionSingleOp.new()\n    conn.close_connection()\n  end\n\n  # Test method for TransactionSingleOp(conn)\n  def testTransactionSingleOp2()\n    conn = Scalaris::TransactionSingleOp.new(conn = Scalaris::JSONConnection.new(url = Scalaris::DEFAULT_URL))\n    conn.close_connection()\n  end\n\n  # Test method for TransactionSingleOp.close_connection() trying to close the connection twice.\n  def testDoubleClose()\n    conn = Scalaris::TransactionSingleOp.new()\n    conn.close_connection()\n    conn.close_connection()\n  end\n\n  # Test method for TransactionSingleOp.read(key)\n  def testRead_NotFound()\n    key = \"_Read_NotFound\"\n    conn = Scalaris::TransactionSingleOp.new()\n    assert_raise( Scalaris::NotFoundError ) { conn.read(@testTime.to_s + key) }\n    conn.close_connection()\n  end\n\n  # Test method for TransactionSingleOp.read(key) with a closed connection.\n  def testRead_NotConnected()\n    key = \"_Read_NotConnected\"\n    conn = Scalaris::TransactionSingleOp.new()\n    conn.close_connection()\n    #assert_raise( Scalaris::ConnectionError ) { conn.read(@testTime.to_s + key) }\n    assert_raise( Scalaris::NotFoundError ) { conn.read(@testTime.to_s + key) }\n    conn.close_connection()\n  end\n\n  # Test method for TransactionSingleOp.write(key, value=str()) with a closed connection.\n  def testWriteString_NotConnected()\n    key = \"_WriteString_NotConnected\"\n    conn = Scalaris::TransactionSingleOp.new()\n    conn.close_connection()\n    #assert_raise( Scalaris::ConnectionError ) { conn.write(@testTime.to_s + key, $_TEST_DATA[0]) }\n    conn.write(@testTime.to_s + key, $_TEST_DATA[0])\n    conn.close_connection()\n  end\n\n  # Test method for TransactionSingleOp.write(key, value=str()) and TransactionSingleOp.read(key).\n  # Writes strings and uses a distinct key for each value. Tries to read the data afterwards.\n  def testWriteString1()\n    key = \"_WriteString1_\"\n    conn = Scalaris::TransactionSingleOp.new()\n\n    (0..($_TEST_DATA.length - 1)).each do |i|\n      conn.write(@testTime.to_s + key + i.to_s, $_TEST_DATA[i])\n    end\n    \n    # now try to read the data:\n    (0..($_TEST_DATA.length - 1)).each do |i|\n      actual = conn.read(@testTime.to_s + key + i.to_s)\n      assert_equal($_TEST_DATA[i], actual)\n    end\n    \n    conn.close_connection()\n  end\n\n  # Test method for TransactionSingleOp.write(key, value=str()) and TransactionSingleOp.read(key).\n  # Writes strings and uses a single key for all the values. Tries to read the data afterwards.\n  def testWriteString2()\n    key = \"_WriteString2\"\n    conn = Scalaris::TransactionSingleOp.new()\n\n    (0..($_TEST_DATA.length - 1)).each do |i|\n      conn.write(@testTime.to_s + key.to_s, $_TEST_DATA[i])\n    end\n    \n    # now try to read the data:\n    actual = conn.read(@testTime.to_s + key.to_s)\n    assert_equal($_TEST_DATA[$_TEST_DATA.length - 1], actual)\n    conn.close_connection()\n  end\n\n  # Test method for TransactionSingleOp.write(key, value=list()) with a closed connection.\n  def testWriteList_NotConnected()\n    key = \"_WriteList_NotConnected\"\n    conn = Scalaris::TransactionSingleOp.new()\n    conn.close_connection()\n    #assert_raise( Scalaris::ConnectionError ) { conn.write(@testTime.to_s + key, [$_TEST_DATA[0], $_TEST_DATA[1]]) }\n    conn.write(@testTime.to_s + key, [$_TEST_DATA[0], $_TEST_DATA[1]])\n    conn.close_connection()\n  end\n\n  # Test method for TransactionSingleOp.write(key, value=list()) and TransactionSingleOp.read(key).\n  # Writes strings and uses a distinct key for each value. Tries to read the data afterwards.\n  def testWriteList1()\n    key = \"_WriteList1_\"\n    conn = Scalaris::TransactionSingleOp.new()\n\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      conn.write(@testTime.to_s + key + i.to_s, [$_TEST_DATA[i], $_TEST_DATA[i + 1]])\n    end\n    \n    # now try to read the data:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      actual = conn.read(@testTime.to_s + key + i.to_s)\n      assert_equal([$_TEST_DATA[i], $_TEST_DATA[i + 1]], actual)\n    end\n    \n    conn.close_connection()\n  end\n\n  # Test method for TransactionSingleOp.write(key, value=list()) and TransactionSingleOp.read(key).\n  # Writes strings and uses a single key for all the values. Tries to read the data afterwards.\n  def testWriteList2()\n    key = \"_WriteList2\"\n    conn = Scalaris::TransactionSingleOp.new()\n    \n    list = []\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      list = [$_TEST_DATA[i], $_TEST_DATA[i + 1]]\n      conn.write(@testTime.to_s + key, list)\n    end\n    \n    # now try to read the data:\n    actual = conn.read(@testTime.to_s + key)\n    assert_equal(list, actual)\n    conn.close_connection()\n  end\n\n  # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=str()) with a closed connection.\n  def testTestAndSetString_NotConnected()\n    key = \"_TestAndSetString_NotConnected\"\n    conn = Scalaris::TransactionSingleOp.new()\n    conn.close_connection()\n    #assert_raise( Scalaris::ConnectionError ) { conn.test_and_set(@testTime.to_s + key, $_TEST_DATA[0], $_TEST_DATA[1]) }\n    assert_raise( Scalaris::NotFoundError ) { conn.test_and_set(@testTime.to_s + key, $_TEST_DATA[0], $_TEST_DATA[1]) }\n    conn.close_connection()\n  end\n  \n  # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=str()).\n  # Tries test_and_set with a non-existing key.\n  def testTestAndSetString_NotFound()\n    key = \"_TestAndSetString_NotFound\"\n    conn = Scalaris::TransactionSingleOp.new()\n    assert_raise( Scalaris::NotFoundError ) { conn.test_and_set(@testTime.to_s + key, $_TEST_DATA[0], $_TEST_DATA[1]) }\n    conn.close_connection()\n  end\n  \n  # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=str()),\n  # TransactionSingleOp.read(key) and TransactionSingleOp.write(key, value=str()).\n  # Writes a string and tries to overwrite it using test_and_set\n  # knowing the correct old value. Tries to read the string afterwards.\n  def testTestAndSetString1()\n    key = \"_TestAndSetString1\"\n    conn = Scalaris::TransactionSingleOp.new()\n    \n    # first write all values:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      conn.write(@testTime.to_s + key + i.to_s, $_TEST_DATA[i])\n    end\n    \n    # now try to overwrite them using test_and_set:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      conn.test_and_set(@testTime.to_s + key + i.to_s, $_TEST_DATA[i], $_TEST_DATA[i + 1])\n    end\n    \n    # now try to read the data:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      actual = conn.read(@testTime.to_s + key + i.to_s)\n      assert_equal($_TEST_DATA[i + 1], actual)\n    end\n    \n    conn.close_connection()\n  end\n  \n  # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=str()),\n  # TransactionSingleOp.read(key) and TransactionSingleOp.write(key, value=str()).\n  # Writes a string and tries to overwrite it using test_and_set\n  # knowing the wrong old value. Tries to read the string afterwards.\n  def testTestAndSetString2()\n    key = \"_TestAndSetString2\"\n    conn = Scalaris::TransactionSingleOp.new()\n    \n    # first write all values:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      conn.write(@testTime.to_s + key + i.to_s, $_TEST_DATA[i])\n    end\n    \n    # now try to overwrite them using test_and_set:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      begin\n        conn.test_and_set(@testTime.to_s + key + i.to_s, $_TEST_DATA[i + 1], \"fail\")\n        assert(false, 'expected a KeyChangedError')\n      rescue Scalaris::KeyChangedError => exception\n        assert_equal($_TEST_DATA[i], exception.old_value)\n      end\n    end\n    \n    # now try to read the data:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      actual = conn.read(@testTime.to_s + key + i.to_s)\n      assert_equal($_TEST_DATA[i], actual)\n    end\n    \n    conn.close_connection()\n  end\n  \n  # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=list()) with a closed connection.\n  def testTestAndSetList_NotConnected()\n    key = \"_TestAndSetList_NotConnected\"\n    conn = Scalaris::TransactionSingleOp.new()\n    conn.close_connection()\n    #assert_raise( Scalaris::ConnectionError ) { conn.test_and_set(@testTime.to_s + key, \"fail\", [$_TEST_DATA[0], $_TEST_DATA[1]]) }\n    assert_raise( Scalaris::NotFoundError ) { conn.test_and_set(@testTime.to_s + key, \"fail\", [$_TEST_DATA[0], $_TEST_DATA[1]]) }\n    conn.close_connection()\n  end\n  \n  # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=list()).\n  # Tries test_and_set with a non-existing key.\n  def testTestAndSetList_NotFound()\n    key = \"_TestAndSetList_NotFound\"\n    conn = Scalaris::TransactionSingleOp.new()\n    assert_raise( Scalaris::NotFoundError ) { conn.test_and_set(@testTime.to_s + key, \"fail\", [$_TEST_DATA[0], $_TEST_DATA[1]]) }\n    conn.close_connection()\n  end\n  \n  # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=list()),\n  # TransactionSingleOp.read(key) and TransactionSingleOp.write(key, value=list()).\n  # Writes a list and tries to overwrite it using test_and_set\n  # knowing the correct old value. Tries to read the string afterwards.\n  def testTestAndSetList1()\n    key = \"_TestAndSetList1\"\n    conn = Scalaris::TransactionSingleOp.new()\n    \n    # first write all values:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      conn.write(@testTime.to_s + key + i.to_s, [$_TEST_DATA[i], $_TEST_DATA[i + 1]])\n    end\n    \n    # now try to overwrite them using test_and_set:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      conn.test_and_set(@testTime.to_s + key + i.to_s, [$_TEST_DATA[i], $_TEST_DATA[i + 1]], [$_TEST_DATA[i + 1], $_TEST_DATA[i]])\n    end\n    \n    # now try to read the data:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      actual = conn.read(@testTime.to_s + key + i.to_s)\n      assert_equal([$_TEST_DATA[i + 1], $_TEST_DATA[i]], actual)\n    end\n    \n    conn.close_connection()\n  end\n  \n  # Test method for TransactionSingleOp.test_and_set(key, oldvalue=str(), newvalue=list()),\n  # TransactionSingleOp.read(key) and TransactionSingleOp.write(key, value=list()).\n  # Writes a string and tries to overwrite it using test_and_set\n  # knowing the wrong old value. Tries to read the string afterwards.\n  def testTestAndSetList2()\n    key = \"_TestAndSetList2\"\n    conn = Scalaris::TransactionSingleOp.new()\n    \n    # first write all values:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      conn.write(@testTime.to_s + key + i.to_s, [$_TEST_DATA[i], $_TEST_DATA[i + 1]])\n    end\n    \n    # now try to overwrite them using test_and_set:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      begin\n        conn.test_and_set(@testTime.to_s + key + i.to_s, \"fail\", 1)\n        assert(false, 'expected a KeyChangedError')\n      rescue Scalaris::KeyChangedError => exception\n        assert_equal([$_TEST_DATA[i], $_TEST_DATA[i + 1]], exception.old_value)\n      end\n    end\n    \n    # now try to read the data:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      actual = conn.read(@testTime.to_s + key + i.to_s)\n      assert_equal([$_TEST_DATA[i], $_TEST_DATA[i + 1]], actual)\n    end\n    \n    conn.close_connection()\n  end\n\n  # Test method for TransactionSingleOp.req_list(RequestList) with an\n  # empty request list.\n  def testReqList_Empty()\n    conn = Scalaris::TransactionSingleOp.new()\n    conn.req_list(conn.new_req_list())\n    conn.close_connection()\n  end\n  \n  # Test method for TransactionSingleOp.req_list(RequestList) with a\n  # mixed request list.\n  def testReqList1()\n    key = \"_ReqList1_\"\n    conn = Scalaris::TransactionSingleOp.new()\n    \n    readRequests = conn.new_req_list()\n    firstWriteRequests = conn.new_req_list()\n    writeRequests = conn.new_req_list()\n    (0..($_TEST_DATA.length - 1)).each do |i|\n      if (i % 2) == 0\n        firstWriteRequests.add_write(@testTime.to_s + key + i.to_s, \"first_\" + $_TEST_DATA[i])\n      end\n      writeRequests.add_write(@testTime.to_s + key + i.to_s, \"second_\" + $_TEST_DATA[i])\n      readRequests.add_read(@testTime.to_s + key + i.to_s)\n    end\n    \n    results = conn.req_list(firstWriteRequests)\n    # evaluate the first write results:\n    (0..(firstWriteRequests.size() - 1)).step(2) do |i|\n      conn.process_result_write(results[i])\n    end\n\n    results = conn.req_list(readRequests)\n    assert_equal(readRequests.size(), results.length)\n    # now evaluate the read results:\n    (0..(readRequests.size() - 1)).step(2) do |i|\n      if (i % 2) == 0\n        actual = conn.process_result_read(results[i])\n        assert_equal(\"first_\" + $_TEST_DATA[i], actual)\n      else\n        begin\n          conn.process_result_read(results[i])\n          # a not found exception must be thrown\n          assert(false, 'expected a NotFoundError')\n        rescue Scalaris::NotFoundError\n        end\n      end\n    end\n\n    results = conn.req_list(writeRequests)\n    assert_equal(writeRequests.size(), results.length)\n    # now evaluate the write results:\n    (0..(writeRequests.size() - 1)).step(2) do |i|\n      conn.process_result_write(results[i])\n    end\n\n    # once again test reads - now all reads should be successful\n    results = conn.req_list(readRequests)\n    assert_equal(readRequests.size(), results.length)\n\n    # now evaluate the read results:\n    (0..(readRequests.size() - 1)).step(2) do |i|\n      actual = conn.process_result_read(results[i])\n      assert_equal(\"second_\" + $_TEST_DATA[i], actual)\n    end\n    \n    conn.close_connection();\n  end\n\n  # Test method for TransactionSingleOp.write(key, value=bytearray()) with a\n  # request that is too large.\n  def testReqTooLarge()\n      conn = Scalaris::TransactionSingleOp.new()\n      data = \"0\" * ($_TOO_LARGE_REQUEST_SIZE)\n      key = \"_ReqTooLarge\"\n      begin\n        conn.write(@testTime.to_s + key, data)\n        assert(false, 'The write should have failed unless yaws_max_post_data was set larger than ' + $_TOO_LARGE_REQUEST_SIZE.to_s())\n      rescue Scalaris::ConnectionError\n      end\n      \n      conn.close_connection()\n  end\nend\n\nclass TestTransaction < Test::Unit::TestCase\n  def setup\n    @testTime = (Time.now.to_f * 1000).to_i\n  end\n  \n  # Test method for Transaction()\n  def testTransaction1()\n    t = Scalaris::Transaction.new()\n    t.close_connection()\n  end\n\n  # Test method for Transaction(conn)\n  def testTransaction3()\n    t = Scalaris::Transaction.new(conn = Scalaris::JSONConnection.new(url = Scalaris::DEFAULT_URL))\n    t.close_connection()\n  end\n\n  # Test method for Transaction.close_connection() trying to close the connection twice.\n  def testDoubleClose()\n    t = Scalaris::Transaction.new()\n    t.close_connection()\n    t.close_connection()\n  end\n\n  # Test method for Transaction.commit() with a closed connection.\n  def testCommit_NotConnected()\n    t = Scalaris::Transaction.new()\n    t.close_connection()\n    #assert_raise( Scalaris::ConnectionError ) { t.commit() }\n    t.commit()\n    t.close_connection()\n  end\n\n  # Test method for Transaction.commit() which commits an empty transaction.\n  def testCommit_Empty()\n    t = Scalaris::Transaction.new()\n    t.commit()\n    t.close_connection()\n  end\n\n  # Test method for Transaction.abort() with a closed connection.\n  def testAbort_NotConnected()\n    t = Scalaris::Transaction.new()\n    t.close_connection()\n    #assert_raise( Scalaris::ConnectionError ) { t.abort() }\n    t.abort()\n    t.close_connection()\n  end\n\n  # Test method for Transaction.abort() which aborts an empty transaction.\n  def testAbort_Empty()\n    t = Scalaris::Transaction.new()\n    t.abort()\n    t.close_connection()\n  end\n\n  # Test method for Transaction.read(key)\n  def testRead_NotFound()\n    key = \"_Read_NotFound\"\n    t = Scalaris::Transaction.new()\n    assert_raise( Scalaris::NotFoundError ) { t.read(@testTime.to_s + key) }\n    t.close_connection()\n  end\n\n  # Test method for Transaction.read(key) with a closed connection.\n  def testRead_NotConnected()\n    key = \"_Read_NotConnected\"\n    t = Scalaris::Transaction.new()\n    t.close_connection()\n    #assert_raise( Scalaris::ConnectionError ) { t.read(@testTime.to_s + key) }\n    assert_raise( Scalaris::NotFoundError ) { t.read(@testTime.to_s + key) }\n    t.close_connection()\n  end\n\n  # Test method for Transaction.write(key, value=str()) with a closed connection.\n  def testWriteString_NotConnected()\n    key = \"_WriteString_NotConnected\"\n    t = Scalaris::Transaction.new()\n    t.close_connection()\n    #assert_raise( Scalaris::ConnectionError ) { t.write(@testTime.to_s + key, $_TEST_DATA[0]) }\n    t.write(@testTime.to_s + key, $_TEST_DATA[0])\n    t.close_connection()\n  end\n\n  # Test method for Transaction.read(key) and Transaction.write(key, value=str())\n  # which should show that writing a value for a key for which a previous read\n  # returned a NotFoundError is possible.\n  def testWriteString_NotFound()\n    key = \"_WriteString_notFound\"\n    t = Scalaris::Transaction.new()\n    notFound = false\n    begin\n      t.read(@testTime.to_s + key)\n    rescue Scalaris::NotFoundError\n      notFound = true\n    end\n    \n    assert(notFound)\n    t.write(@testTime.to_s + key, $_TEST_DATA[0])\n    assert_equal($_TEST_DATA[0], t.read(@testTime.to_s + key))\n    t.close_connection()\n  end\n\n  # Test method for Transaction.write(key, value=str()) and Transaction.read(key).\n  # Writes strings and uses a distinct key for each value. Tries to read the data afterwards.\n  def testWriteString()\n    key = \"_testWriteString1_\"\n    t = Scalaris::Transaction.new()\n\n    (0..($_TEST_DATA.length - 1)).each do |i|\n      t.write(@testTime.to_s + key + i.to_s, $_TEST_DATA[i])\n    end\n    \n    # now try to read the data:\n    (0..($_TEST_DATA.length - 1)).each do |i|\n      actual = t.read(@testTime.to_s + key + i.to_s)\n      assert_equal($_TEST_DATA[i], actual)\n    end\n    \n    # commit the transaction and try to read the data with a new one:\n    t.commit()\n    t = Scalaris::Transaction.new()\n    (0..($_TEST_DATA.length - 1)).each do |i|\n      actual = t.read(@testTime.to_s + key + i.to_s)\n      assert_equal($_TEST_DATA[i], actual)\n    end\n    \n    t.close_connection()\n  end\n\n  # Test method for Transaction.write(key, value=list()) and Transaction.read(key).\n  # Writes a list and uses a distinct key for each value. Tries to read the data afterwards.\n  def testWriteList1()\n    key = \"_testWriteList1_\"\n    t = Scalaris::Transaction.new()\n\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      t.write(@testTime.to_s + key + i.to_s, [$_TEST_DATA[i], $_TEST_DATA[i + 1]])\n    end\n    \n    # now try to read the data:\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      actual = t.read(@testTime.to_s + key + i.to_s)\n      assert_equal([$_TEST_DATA[i], $_TEST_DATA[i + 1]], actual)\n    end\n    \n    t.close_connection()\n    \n    # commit the transaction and try to read the data with a new one:\n    t.commit()\n    t = Scalaris::Transaction.new()\n    (0..($_TEST_DATA.length - 2)).step(2) do |i|\n      actual = t.read(@testTime.to_s + key + i.to_s)\n      assert_equal([$_TEST_DATA[i], $_TEST_DATA[i + 1]], actual)\n    end\n    \n    t.close_connection()\n  end\n\n  # Test method for Transaction.req_list(RequestList) with an\n  # empty request list.\n  def testReqList_Empty()\n    conn = Scalaris::Transaction.new()\n    conn.req_list(conn.new_req_list())\n    conn.close_connection()\n  end\n\n  # Test method for Transaction.req_list(RequestList) with a\n  # mixed request list.\n  def testReqList1()\n    key = \"_ReqList1_\"\n    conn = Scalaris::Transaction.new()\n    \n    readRequests = conn.new_req_list()\n    firstWriteRequests = conn.new_req_list()\n    writeRequests = conn.new_req_list()\n    (0..($_TEST_DATA.length - 1)).each do |i|\n      if (i % 2) == 0\n        firstWriteRequests.add_write(@testTime.to_s + key + i.to_s, $_TEST_DATA[i])\n      end\n      writeRequests.add_write(@testTime.to_s + key + i.to_s, $_TEST_DATA[i])\n      readRequests.add_read(@testTime.to_s + key + i.to_s)\n    end\n    \n    results = conn.req_list(firstWriteRequests)\n    # evaluate the first write results:\n    (0..(firstWriteRequests.size() - 1)).each do |i|\n      conn.process_result_write(results[i])\n    end\n\n    requests = conn.new_req_list(readRequests).concat(writeRequests).add_commit()\n    results = conn.req_list(requests)\n    assert_equal(requests.size(), results.length)\n\n    # now evaluate the read results:\n    (0..(readRequests.size() - 1)).each do |i|\n      if (i % 2) == 0\n        actual = conn.process_result_read(results[i])\n        assert_equal($_TEST_DATA[i], actual)\n      else\n        begin\n          conn.process_result_read(results[i])\n          # a not found exception must be thrown\n          assert(false, 'expected a NotFoundError')\n        rescue Scalaris::NotFoundError\n        end\n      end\n    end\n\n    # now evaluate the write results:\n    (0..(writeRequests.size() - 1)).each do |i|\n      pos = readRequests.size() + i\n      conn.process_result_write(results[pos])\n    end\n\n    # once again test reads - now all reads should be successful\n    results = conn.req_list(readRequests)\n    assert_equal(readRequests.size(), results.length)\n\n    # now evaluate the read results:\n    (0..(readRequests.size() - 1)).each do |i|\n      actual = conn.process_result_read(results[i])\n      assert_equal($_TEST_DATA[i], actual)\n    end\n    \n    conn.close_connection();\n  end\n\n  # Test method for Transaction.write(key, value=bytearray()) with a\n  # request that is too large.\n  def testReqTooLarge()\n      conn = Scalaris::Transaction.new()\n      data = \"0\" * ($_TOO_LARGE_REQUEST_SIZE)\n      key = \"_ReqTooLarge\"\n      begin\n        conn.write(@testTime.to_s + key, data)\n        assert(false, 'The write should have failed unless yaws_max_post_data was set larger than ' + $_TOO_LARGE_REQUEST_SIZE.to_s())\n      rescue Scalaris::ConnectionError\n      end\n      \n      conn.close_connection()\n  end\n\n  # Various tests.\n  def testVarious()\n      _writeSingleTest(\"_0:\" + [0x0160].pack(\"U*\") + \"arplaninac:page_\", $_TEST_DATA[0])\n  end\n  \n  # Helper function for single write tests.\n  # Writes a strings to some key and tries to read it afterwards.\n  def _writeSingleTest(key, data)\n    t = Scalaris::Transaction.new()\n  \n    t.write(@testTime.to_s + key, data)\n    # now try to read the data:\n    assert_equal(data, t.read(@testTime.to_s + key))\n    # commit the transaction and try to read the data with a new one:\n    t.commit()\n    t = Scalaris::Transaction.new()\n    assert_equal(data, t.read(@testTime.to_s + key))\n    \n    t.close_connection()\n  end\n  private :_writeSingleTest\nend\n\nclass TestReplicatedDHT < Test::Unit::TestCase\n  def setup\n    @testTime = (Time.now.to_f * 1000).to_i\n  end\n  \n  # Test method for ReplicatedDHT()\n  def testReplicatedDHT1()\n    rdht = Scalaris::ReplicatedDHT.new()\n    rdht.close_connection()\n  end\n\n  # Test method for ReplicatedDHT(conn)\n  def testReplicatedDHT2()\n    rdht = Scalaris::ReplicatedDHT.new(conn = Scalaris::JSONConnection.new(url = Scalaris::DEFAULT_URL))\n    rdht.close_connection()\n  end\n\n  # Test method for ReplicatedDHT.close_connection() trying to close the connection twice.\n  def testDoubleClose()\n    rdht = Scalaris::ReplicatedDHT.new()\n    rdht.close_connection()\n    rdht.close_connection()\n  end\n  \n  # Tries to read the value at the given key and fails if this does\n  # not fail with a NotFoundError.\n  def _checkKeyDoesNotExist(key)\n    conn = Scalaris::TransactionSingleOp.new()\n    begin\n      conn.read(key)\n      assert(false, 'the value at ' + key + ' should not exist anymore')\n    rescue Scalaris::NotFoundError\n      # nothing to do here\n    end\n    conn.close_connection()\n  end\n\n  # Test method for ReplicatedDHT.delete(key).\n  # Tries to delete some not existing keys.\n  def testDelete_notExistingKey()\n    key = \"_Delete_NotExistingKey\"\n    rdht = Scalaris::ReplicatedDHT.new()\n    rt = Scalaris::RoutingTable.new\n    r = rt.get_replication_factor\n\n    (0..($_TEST_DATA.length - 1)).each do |i|\n      ok = rdht.delete(@testTime.to_s + key + i.to_s)\n      assert_equal(0, ok)\n      results = rdht.get_last_delete_result()\n      assert_equal(0, results.ok)\n      assert_equal(0, results.locks_set)\n      assert_equal(r, results.undefined)\n      _checkKeyDoesNotExist(@testTime.to_s + key + i.to_s)\n    end\n    \n    rdht.close_connection()\n  end\n\n  # Test method for ReplicatedDHT.delete(key) and TransactionSingleOp#write(key, value=str()).\n  # Inserts some values, tries to delete them afterwards and tries the delete again.\n  def testDelete1()\n    key = \"_Delete1\"\n    c = Scalaris::JSONConnection.new(url = Scalaris::DEFAULT_URL)\n    rdht = Scalaris::ReplicatedDHT.new(conn = c)\n    sc = Scalaris::TransactionSingleOp.new(conn = c)\n    rt = Scalaris::RoutingTable.new\n    r = rt.get_replication_factor\n    \n    (0..($_TEST_DATA.length - 1)).each do |i|\n      sc.write(@testTime.to_s + key + i.to_s, $_TEST_DATA[i])\n    end\n    \n    # now try to delete the data:\n    (0..($_TEST_DATA.length - 1)).each do |i|\n      ok = rdht.delete(@testTime.to_s + key + i.to_s)\n      assert_equal(r, ok)\n      results = rdht.get_last_delete_result()\n      assert_equal(r, results.ok)\n      assert_equal(0, results.locks_set)\n      assert_equal(0, results.undefined)\n      _checkKeyDoesNotExist(@testTime.to_s + key + i.to_s)\n      \n      # try again (should be successful with 0 deletes)\n      ok = rdht.delete(@testTime.to_s + key + i.to_s)\n      assert_equal(0, ok)\n      results = rdht.get_last_delete_result()\n      assert_equal(0, results.ok)\n      assert_equal(0, results.locks_set)\n      assert_equal(r, results.undefined)\n      _checkKeyDoesNotExist(@testTime.to_s + key + i.to_s)\n    end\n    \n    c.close()\n  end\n\n  # Test method for ReplicatedDHT.delete(key) and TransactionSingleOp#write(key, value=str()).\n  # Inserts some values, tries to delete them afterwards, inserts them again and tries to delete them again (twice).\n  def testDelete2()\n    key = \"_Delete2\"\n    c = Scalaris::JSONConnection.new(url = Scalaris::DEFAULT_URL)\n    rdht = Scalaris::ReplicatedDHT.new(conn = c)\n    sc = Scalaris::TransactionSingleOp.new(conn = c)\n    rt = Scalaris::RoutingTable.new\n    r = rt.get_replication_factor\n\n    (0..($_TEST_DATA.length - 1)).each do |i|\n      sc.write(@testTime.to_s + key + i.to_s, $_TEST_DATA[i])\n    end\n    \n    # now try to delete the data:\n    (0..($_TEST_DATA.length - 1)).each do |i|\n      ok = rdht.delete(@testTime.to_s + key + i.to_s)\n      assert_equal(r, ok)\n      results = rdht.get_last_delete_result()\n      assert_equal(r, results.ok)\n      assert_equal(0, results.locks_set)\n      assert_equal(0, results.undefined)\n      _checkKeyDoesNotExist(@testTime.to_s + key + i.to_s)\n    end\n  \n    (0..($_TEST_DATA.length - 1)).each do |i|\n      sc.write(@testTime.to_s + key + i.to_s, $_TEST_DATA[i])\n    end\n    \n    # now try to delete the data:\n    (0..($_TEST_DATA.length - 1)).each do |i|\n      ok = rdht.delete(@testTime.to_s + key + i.to_s)\n      assert_equal(r, ok)\n      results = rdht.get_last_delete_result()\n      assert_equal(r, results.ok)\n      assert_equal(0, results.locks_set)\n      assert_equal(0, results.undefined)\n      _checkKeyDoesNotExist(@testTime.to_s + key + i.to_s)\n      \n      # try again (should be successful with 0 deletes)\n      ok = rdht.delete(@testTime.to_s + key + i.to_s)\n      assert_equal(0, ok)\n      results = rdht.get_last_delete_result()\n      assert_equal(0, results.ok)\n      assert_equal(0, results.locks_set)\n      assert_equal(r, results.undefined)\n      _checkKeyDoesNotExist(@testTime.to_s + key + i.to_s)\n    end\n    \n    c.close()\n  end\nend\n"
  },
  {
    "path": "src/.gitignore",
    "content": "/.webhelpers.erl.swp\n/.self_man.erl.swp\n/.cs_api.erl.swp\n"
  },
  {
    "path": "src/admin.erl",
    "content": "% @copyright 2008-2019 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Aministrative helper functions (mostly for debugging)\n%% @version $Id$\n-module(admin).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([add_node/1, add_node_at_id/1, add_nodes/1, add_nodes_at_ids/1,\n         add_remaining_symmetric_nodes/0,\n         del_node/2, del_nodes/2, del_nodes_by_name/2,\n         get_dht_node_specs/0,\n         wait_for_stable_ring/1,\n         check_ring/0, check_ring_deep/0, nodes/0, start_link/0, start/0, get_dump/0,\n         get_dump_bw/0, diff_dump/2, print_ages/0,\n         number_of_nodes/0,\n         check_leases/0, check_leases/1]).\n\n-include(\"scalaris.hrl\").\n\n-type check_ring_deep_error() ::\n        {error,\n         'in_node:', Node::node:node_type(),\n         'predList:', Preds::nodelist:non_empty_snodelist(),\n         'succList:', Succs::nodelist:non_empty_snodelist(),\n         'nodes:', Nodes::nodelist:non_empty_snodelist(),\n         'UnknownPreds:', UnknownPreds::nodelist:snodelist(),\n         'UnknownSuccs:', UnknownSuccs::nodelist:snodelist()}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% API functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% userdevguide-begin admin:add_nodes\n% @doc add new Scalaris nodes on the local node\n-spec add_node_at_id(?RT:key()) -> pid_groups:groupname() | {error, term()}.\nadd_node_at_id(Id) ->\n    add_node([{{dht_node, id}, Id}, {skip_psv_lb}, {add_node}]).\n\n-spec add_node([tuple()]) -> pid_groups:groupname() | {error, term()}.\nadd_node(Options) ->\n    DhtNodeId = case lists:keyfind({dht_node, id}, 1, Options) of\n                    %% supervisor Id for a dht_node has to be a string\n                    {{dht_node, id}, Id} when is_integer(Id) -> integer_to_list(Id);\n                    {{dht_node, id}, Id} -> Id;\n                    _ -> randoms:getRandomString()\n                end,\n    Group = pid_groups:new(dht_node),\n    Desc = sup:supervisor_desc(\n             DhtNodeId, sup_dht_node, start_link,\n             [{Group,\n               [{my_sup_dht_node_id, DhtNodeId}, {add_node} | Options]}]),\n    Sup = erlang:whereis(main_sup),\n    case sup:start_sup_as_child([\" +\"], Sup, Desc) of\n        {ok, _Child, Group}           ->\n            DhtNodePid = pid_groups:pid_of(Group, dht_node),\n            comm:send_local(DhtNodePid, {join, start}),\n            Group;\n        {error, already_present}      -> add_node(Options); % try again, different Id\n        {error, {already_started, _}} -> add_node(Options); % try again, different Id\n        {error, _Error} = X           -> X\n    end.\n\n-spec add_nodes(non_neg_integer()) -> {[pid_groups:groupname()], [{error, term()}]}.\nadd_nodes(0) -> {[], []};\nadd_nodes(Count) ->\n    Results = [add_node([]) || _X <- lists:seq(1, Count)],\n    lists:partition(fun(E) -> case E of\n                                  {error, _} -> false;\n                                  _ -> true\n                              end end, Results).\n%% userdevguide-end admin:add_nodes\n\n-spec add_nodes_at_ids(list(?RT:key())) -> {[pid_groups:groupname()], [{error, term()}]}.\nadd_nodes_at_ids([]) -> {[], []};\nadd_nodes_at_ids(Keys) ->\n    Results = [add_node_at_id(Key) || Key <- Keys],\n    lists:partition(fun(E) -> case E of\n                                  {error, _} -> false;\n                                  _ -> true\n                              end end, Results).\n\n% @doc if you started a single firstnode.sh you can add the remaining symmetric r-1 nodes in the local VM,\n%       for example to interactively run a unittest scenario step by step.\n-spec add_remaining_symmetric_nodes() -> {[pid_groups:groupname()], [{error, term()}]}.\nadd_remaining_symmetric_nodes() ->\n    ?ASSERT2(1 =:= admin:number_of_nodes(), more_than_one_dht_node),\n    DHTNode = pid_groups:find_a(dht_node),\n    comm:send_local(DHTNode, {get_state, comm:this(), node_id}),\n    receive\n        ?SCALARIS_RECV({get_state_response, Id}, Id)\n    end,\n    Keys = ?RT:get_replica_keys(Id),\n    add_nodes_at_ids([X || X <- Keys, X =/= Id]).\n\n-spec get_dht_node_specs()\n        -> [{Id::term() | undefined, Child::pid() | undefined,\n             Type::worker | supervisor, Modules::[module()] | dynamic}].\nget_dht_node_specs() ->\n    % note: only sup_dht_node children have strings as identifiers!\n    [Spec || {Id, Pid, _Type, _} = Spec <- supervisor:which_children(main_sup),\n             is_pid(Pid), is_list(Id)].\n\n%% @doc Deletes Scalaris nodes from the current VM.\n-spec del_nodes(Count::non_neg_integer(), Graceful::boolean())\n        -> Successful::[pid_groups:groupname()].\ndel_nodes(Count, Graceful) ->\n    del_nodes(Count, Graceful, []).\n\n-spec del_nodes(Count::non_neg_integer(), Graceful::boolean(), Prev::Acc)\n        -> Acc when is_subtype(Acc, Successful::[pid_groups:groupname()]).\ndel_nodes(X, _Graceful, Prev) when X =< 0 ->\n    Prev;\ndel_nodes(Count, Graceful, Prev) ->\n    % note: specs selected now may not be available anymore when trying to\n    % delete them during concurrent executions\n    case util:random_subset(Count, get_dht_node_specs()) of\n        [] -> Prev;\n        [_|_] = Specs ->\n            {Successful, NoPartner} =\n                lists:foldr(fun(Spec = {_Id, Pid, _Type, _}, {Successful, NoPartner}) ->\n                                    Name = pid_groups:group_of(Pid),\n                                    case del_node(Spec, Graceful) of\n                                        ok -> {[Name | Successful], NoPartner};\n                                        {error, not_found} -> {Successful, NoPartner};\n                                        {error, no_partner_found} -> {Successful, [Name | NoPartner]}\n                                    end\n                            end, {[], []}, Specs),\n            Missing = Count - length(Successful) - length(NoPartner),\n            del_nodes(Missing, Graceful, Successful)\n    end.\n\n-spec del_nodes_by_name(Names::[pid_groups:groupname()], Graceful::boolean())\n        -> {Successful::[pid_groups:groupname()], NotFound::[pid_groups:groupname()]}.\ndel_nodes_by_name(Names, Graceful) ->\n    % note: specs selected now may not be available anymore when trying to\n    % delete them during concurrent executions\n    Specs = [{Spec, pid_groups:group_of(Pid)}\n             || {_Id, Pid, _Type, _} = Spec <- get_dht_node_specs(),\n                lists:member(pid_groups:group_of(Pid), Names)],\n    case Specs of\n        [] -> {[], []};\n        [_|_] when Graceful ->\n            lists:foldr(\n              fun({Spec, Name}, {Ok, NotFound}) ->\n                      case del_node(Spec, Graceful) of\n                          ok -> {[Name | Ok], NotFound};\n                          {error, not_found} -> {Ok, [Name | NotFound]};\n                          {error, no_partner_found} -> {Ok, NotFound}\n                      end\n              end, {[], []}, Specs);\n        [_|_] when not Graceful ->\n            Pids = [Pid || {{_Id, Pid, _Type, _}, _Name} <- Specs],\n            AllChildren = lists:flatmap(fun sup:sup_get_all_children/1, Pids),\n            comm:send_local(erlang:whereis(fd), {del_all_subscriptions, AllChildren}),\n            sup:sup_terminate_childs(Pids),\n            lists:foldr(\n              fun({{Id, _Pid, _Type, _}, Name}, {Ok, NotFound}) ->\n                      _ = supervisor:terminate_child(main_sup, Id),\n                      case supervisor:delete_child(main_sup, Id) of\n                          ok -> {[Name | Ok], NotFound};\n                          {error, not_found} -> {Ok, [Name | NotFound]}\n                      end\n              end, {[], []}, Specs)\n    end.\n\n%% @doc Delete a single node.\n-spec del_node({Id::term() | undefined, Child::pid() | undefined,\n                Type::worker | supervisor, Modules::[module()] | dynamic},\n               Graceful::boolean()) -> ok | {error, not_found | no_partner_found}.\ndel_node({Id, Pid, _Type, _}, Graceful) ->\n    case Graceful of\n        true ->\n            Group = pid_groups:group_of(Pid),\n            case pid_groups:pid_of(Group, dht_node) of\n                failed  -> {error, not_found};\n                DhtNode ->\n                    UId = uid:get_pids_uid(),\n                    Self = comm:reply_as(comm:this(), 3, {leave_result, UId, '_'}),\n                    comm:send_local(DhtNode, {leave, Self}),\n                    trace_mpath:thread_yield(),\n                    receive\n                        ?SCALARIS_RECV(\n                            {leave_result, UId, {move, result, leave, ok}}, %% ->\n                            ok\n                          );\n                        ?SCALARIS_RECV(\n                            {leave_result, UId, {move, result, leave, leave_no_partner_found}}, %% ->\n                            {error, no_partner_found}\n                          )\n                    end\n            end;\n        false ->\n            sup:sup_terminate_childs(Pid),\n            _ = supervisor:terminate_child(main_sup, Id),\n            supervisor:delete_child(main_sup, Id)\n    end.\n\n-spec wait_for_stable_ring(non_neg_integer()) -> ok.\nwait_for_stable_ring(NrOfNodes) ->\n    Fun = fun() ->\n                  case config:read(leases) of\n                      true ->\n                          (number_of_nodes() =:= NrOfNodes) andalso\n                              (check_leases() =:= true);\n                      _ ->\n                          (number_of_nodes() =:= NrOfNodes) andalso\n                              (check_ring() =:= ok) andalso\n                              (check_ring_deep() =:= ok)\n                  end\n          end,\n    util:wait_for(Fun).\n\n%% @doc Contact mgmt server and check that each node's successor and predecessor\n%%      are correct.\n-spec check_ring() -> {error, string()} | ok.\ncheck_ring() ->\n    Nodes = statistics:get_ring_details(),\n    case lists:foldl(fun check_ring_foldl/2, first, Nodes) of\n        {error, Reason} -> {error, Reason};\n        _ -> ok\n    end.\n\n-spec check_ring_foldl(NodeState::statistics:ring_element(),\n                       Acc::first | ?RT:key())\n        -> first | {error, Reason::string()} | {Pred::?RT:key(), PredsSucc::?RT:key()}.\ncheck_ring_foldl({ok, NodeDetails}, first) ->\n    {node:id(node_details:get(NodeDetails, node)),\n     node:id(node_details:get(NodeDetails, succ))};\ncheck_ring_foldl(_, {error, Message}) ->\n    {error, Message};\ncheck_ring_foldl({ok, NodeDetails}, {Pred, PredsSucc}) ->\n    MyPredId = node:id(node_details:get(NodeDetails, pred)),\n    MyId = node:id(node_details:get(NodeDetails, node)),\n    if\n        MyId =/= PredsSucc ->\n            {error, lists:flatten(io_lib:format(\"MyID ~p didn't match preds succ ~p\", [MyId, PredsSucc]))};\n        MyPredId =/= Pred ->\n            {error, lists:flatten(io_lib:format(\"MyPredID ~p didn't match preds id ~p\", [MyPredId, Pred]))};\n        true ->\n            {node:id(node_details:get(NodeDetails, node)),\n             node:id(node_details:get(NodeDetails, succ))}\n    end;\ncheck_ring_foldl({failed, _}, Previous) ->\n    Previous.\n\n%% @doc Contact mgmt server and check that each node's successor and\n%%      predecessor lists are correct.\n-spec check_ring_deep() -> ok | check_ring_deep_error().\ncheck_ring_deep() ->\n    case check_ring() of\n        ok ->\n            Ring = statistics:get_ring_details(),\n            Nodes = [node_details:get(Details, node) || {ok, Details} <- Ring],\n            case lists:foldl(fun check_ring_deep_foldl/2, {ok, Nodes}, Ring) of\n                {ok, _} -> ok;\n                X       -> X\n            end;\n        X -> X\n    end.\n\n-spec check_ring_deep_foldl(Element::statistics:ring_element(),\n        {ok, Nodes::[node:node_type()]} | check_ring_deep_error())\n            -> {ok, Nodes::[node:node_type()]} | check_ring_deep_error().\ncheck_ring_deep_foldl({ok, NodeDetails}, {ok, Nodes}) ->\n    PredList = node_details:get(NodeDetails, predlist),\n    SuccList = node_details:get(NodeDetails, succlist),\n    CheckIsKnownNode = fun (Node) -> not lists:member(Node, Nodes) end,\n    case {lists:filter(CheckIsKnownNode, PredList),\n          lists:filter(CheckIsKnownNode, SuccList)} of\n        {[],[]} -> {ok, Nodes};\n        {UnknownPreds, UnknownSuccs} ->\n            {error,\n             'in_node:', node_details:get(NodeDetails, node),\n             'predList:', PredList,\n             'succList:', SuccList,\n             'nodes:', Nodes,\n             'UnknownPreds:', UnknownPreds,\n             'UnknownSuccs:', UnknownSuccs}\n    end;\ncheck_ring_deep_foldl({failed, _}, Previous) ->\n    Previous;\ncheck_ring_deep_foldl(_, Previous) ->\n    Previous.\n\n-spec check_leases() -> boolean().\ncheck_leases() ->\n    lease_checker:check_leases_for_all_nodes() andalso\n        lease_checker:check_leases_for_the_ring().\n\n-spec check_leases(pos_integer()) -> boolean().\ncheck_leases(TargetSize) ->\n    lease_checker:check_leases_for_all_nodes() andalso\n        lease_checker:check_leases_for_the_ring(TargetSize).\n\n-spec number_of_nodes() -> non_neg_integer() | timeout.\nnumber_of_nodes() ->\n    mgmt_server:number_of_nodes(),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n            {get_list_length_response, X}, %% ->\n            X\n          )\n    after\n        5000 -> timeout\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%=\n% % comm_logger functions\n% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%=\n% @doc returns communications information. the comm-layer logs for\n% each message-tag how many message were sent and how large were these\n% messages in total. get_dump/0 returns a map from message-tag to\n% message-count and message-size and a timestamp when the measurement\n% was started.\n-spec get_dump() -> {Received::comm_logger:stat_tree(), Sent::comm_logger:stat_tree(), erlang_timestamp()}.\nget_dump() ->\n    Servers = util:get_proc_in_vms(admin_server),\n    _ = [comm:send(Server, {get_comm_layer_dump, comm:this()})\n         || Server <- Servers],\n    %% list({Map, StartTime})\n    Dumps = [  begin\n                   trace_mpath:thread_yield(),\n                   receive\n                       ?SCALARIS_RECV(\n                          {get_comm_layer_dump_response, Dump}, %% ->\n                          Dump\n                         )\n                       end\n               end || _ <- Servers],\n    StartTime = lists:min([Start || {_, _, Start} <- Dumps]),\n    GetKeys =\n        fun(DumpsX, ElementX) ->\n                lists:usort(\n                  lists:flatmap(fun(X) -> gb_trees:keys(element(ElementX, X)) end,\n                                DumpsX))\n        end,\n    ReceivedKeys = GetKeys(Dumps, 1),\n    SentKeys = GetKeys(Dumps, 2),\n    {lists:foldl(fun (Tag, Map) ->\n                         gb_trees:enter(Tag, get_aggregate(Tag, Dumps, 1), Map)\n                 end, gb_trees:empty(), ReceivedKeys),\n     lists:foldl(fun (Tag, Map) ->\n                         gb_trees:enter(Tag, get_aggregate(Tag, Dumps, 2), Map)\n                 end, gb_trees:empty(), SentKeys),\n     StartTime}.\n\n% @doc returns communications information. similar to get_dump/0 it\n% returns message statistics per message-tag, but it scales all values\n% by the elapsed time. so it returns messages per second and bytes per\n% second for each message-tag\n-spec get_dump_bw() -> {Received::[{atom(), float(), float()}], Sent::[{atom(), float(), float()}]}.\nget_dump_bw() ->\n    {Received, Sent, StartTime} = get_dump(),\n    RunTime = timer:now_diff(os:timestamp(), StartTime),\n    AggFun =\n        fun(Map) ->\n                [{Tag, Size / RunTime, Count / RunTime} || {Tag, {Size, Count}} <- gb_trees:to_list(Map)]\n        end,\n    {AggFun(Received), AggFun(Sent)}.\n\nget_aggregate(_Tag, [], _Element) ->\n    {0, 0};\nget_aggregate(Tag, [Dump | Rest], Element) ->\n    Map = element(Element, Dump),\n    case gb_trees:lookup(Tag, Map) of\n        none ->\n            get_aggregate(Tag, Rest, Element);\n        {value, {Size, Count}} ->\n            {AggSize, AggCount} = get_aggregate(Tag, Rest, Element),\n            {AggSize + Size, AggCount + Count}\n    end.\n\n-spec diff_dump(Before::comm_logger:stat_tree(), After::comm_logger:stat_tree())\n        -> [{Tag::atom(), Size::integer(), Count::integer()}].\ndiff_dump(BeforeDump, AfterDump) ->\n    Tags = lists:usort(gb_trees:keys(BeforeDump) ++ gb_trees:keys(AfterDump)),\n    diff(Tags, BeforeDump, AfterDump).\n\n-spec diff(Tags::[atom()], Before::comm_logger:stat_tree(), After::comm_logger:stat_tree())\n        -> [{Tag::atom(), Size::integer(), Count::integer()}].\ndiff([], _Before, _After) ->\n    [];\ndiff([Tag | Rest], Before, After) ->\n    {NewSize, NewCount} = gb_trees:get(Tag, After),\n    case gb_trees:lookup(Tag, Before) of\n        none ->\n            [{Tag, NewSize, NewCount} | diff(Rest, Before, After)];\n        {value, {Size, Count}} ->\n            [{Tag, NewSize - Size, NewCount - Count} | diff(Rest, Before, After)]\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% admin server functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec start_link() -> {ok, pid()}.\nstart_link() ->\n    {ok, spawn_link(?MODULE, start, [])}.\n\n-spec start() -> ok.\nstart() ->\n    erlang:register(admin_server, self()),\n    loop().\n\nloop() ->\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n            {halt, N}, %% ->\n            begin\n                init:stop(N),\n                receive nothing -> ok end % wait forever\n            end\n          );\n        ?SCALARIS_RECV(\n            {get_comm_layer_dump, Sender}, %% ->\n            begin\n                comm:send(Sender, {get_comm_layer_dump_response,\n                                  comm_logger:dump()}),\n                loop()\n            end\n          )\n    end.\n\n% @doc contact mgmt server and list the known ip addresses\n-spec(nodes() -> list()).\nnodes() ->\n    mgmt_server:node_list(),\n    Nodes = begin\n                trace_mpath:thread_yield(),\n                receive\n                    ?SCALARIS_RECV(\n                       {get_list_response, List}, %% ->\n                       lists:usort([IP || {IP, _, _} <- List])\n                      )\n                    end\n            end,\n    Nodes.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Debug functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec print_ages() -> ok.\nprint_ages() ->\n    mgmt_server:node_list(),\n    _ = begin\n            trace_mpath:thread_yield(),\n            receive\n                ?SCALARIS_RECV(\n                   {get_list_response, List}, %% ->\n                   [ comm:send(Node, {cb_msg, {gossip_cyclon, default}, {get_ages, self()}},\n                               [{group_member, gossip}]) || Node <- List ]\n                  )\n                end\n        end,\n    worker_loop().\n\n-spec worker_loop() -> ok.\nworker_loop() ->\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n            {cy_ages, Ages}, %% ->\n            begin\n                io:format(\"~p~n\", [Ages]),\n                worker_loop()\n            end\n          )\n        after 400 ->\n            ok\n    end.\n"
  },
  {
    "path": "src/admin_first.erl",
    "content": "%  @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc    determine if first vm in this deployment\n%% @end\n%% @version $Id$\n-module(admin_first).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n-export([is_first_vm/0]).\n\n-spec is_first_vm() -> boolean().\nis_first_vm() ->\n    config:read(start_type) =:= first\n        orelse\n    config:read(start_type) =:= first_nostart\n        orelse\n    (config:read(start_type) =:= quorum andalso has_first_quorum()).\n\nhas_first_quorum() ->\n    KnownHosts = config:read(known_hosts),\n    MyServicePerVM = comm:get(service_per_vm, comm:this()),\n    case lists:member(MyServicePerVM, KnownHosts) of\n        false ->\n            false;\n        true ->\n            PaxosId = leader_election_for_first_quorum,\n            MyProposer = comm:get('basic_services-paxos_proposer', comm:this()),\n            MyAcceptor = comm:get('basic_services-paxos_acceptor', comm:this()),\n            MyLearner = comm:get('basic_services-paxos_learner', comm:this()),\n            Acceptors = [comm:get('basic_services-paxos_acceptor', Host) || Host <- KnownHosts],\n            Learners = [comm:get('basic_services-paxos_learner', Host) || Host <- KnownHosts],\n            Proposal = {leader, MyServicePerVM},\n            MaxProposers = length(KnownHosts),\n            Majority = quorum:majority_for_accept(MaxProposers),\n            Rank = util:lists_index_of(MyServicePerVM, KnownHosts),\n            ?ASSERT(Rank =/= not_found),\n\n            acceptor:start_paxosid(MyAcceptor, PaxosId, Learners),\n            learner:start_paxosid(MyLearner, PaxosId, Majority, comm:this(), client_cookie),\n            proposer:start_paxosid(MyProposer, PaxosId,\n                                   Acceptors, Proposal, Majority, MaxProposers, Rank),\n            trace_mpath:thread_yield(),\n            receive\n                ?SCALARIS_RECV(\n                    {learner_decide, client_cookie, PaxosId, {leader, TheLeader}}, %%->\n                    TheLeader =:= MyServicePerVM\n                  )\n            end\n    end.\n"
  },
  {
    "path": "src/api_autoscale.erl",
    "content": "% @copyright 2013-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Ufuk Celebi <celebi@zib.de>\n%% @doc Simple auto-scaling service API.\n%% @end\n-module(api_autoscale).\n-author('celebi@zib.de').\n\n-include(\"scalaris.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Types\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-type autoscale_error_resp() :: {error, resp_timeout | autoscale_false}.\n\n%% Misc API\n-export([check_config/0]).\n%% Polling API\n-export([pull_scale_req/0, lock_scale_req/0, unlock_scale_req/0]).\n%% Alarm state API\n-export([toggle_alarm/1, activate_alarms/0, deactivate_alarms/0]).\n%% Autoscale server API\n-export([write_plot_data/0, reset_plot_data/0]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Misc API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% @doc Checks whether config parameters exist and are valid (requires a pull\n%%      configuration, i.e. cloud_cps as cloud_module).\n-spec check_config() -> boolean().\ncheck_config() ->\n    autoscale:check_config() andalso is_in_pull_mode().\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Polling API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Pull current scale request from autoscale leader. If the request is\n%%      not 0, further requests should be locked with lock_scale_req/0 until a\n%%      unlock_scale_req/0 call has been made. Autoscale defines a timeout,\n%%      after which the lock will be automatically freed, i.e. the caller has\n%%      $timeout seconds to satisfy the request and notify autoscale by\n%%      unlock_scale_req/0.\n-spec pull_scale_req() -> {ok, Req :: integer()} | autoscale_error_resp().\npull_scale_req() ->\n    case autoscale:check_config() andalso is_in_pull_mode() of\n        true  -> send_to_leader_wait_resp({pull_scale_req, comm:this()},\n                                          scale_req_resp, 5);\n        false -> {error, autoscale_false}\n    end.\n\n-spec lock_scale_req() -> ok | {error, locked} | autoscale_error_resp().\nlock_scale_req() ->\n    case autoscale:check_config() andalso is_in_pull_mode() of\n        true  -> send_to_leader_wait_resp({lock_scale_req, comm:this()},\n                                          scale_req_resp, 5);\n        false -> {error, autoscale_false}\n    end.\n\n-spec unlock_scale_req() -> ok | {error, not_locked} | autoscale_error_resp().\nunlock_scale_req() ->\n    case autoscale:check_config() andalso is_in_pull_mode() of\n        true  -> send_to_leader_wait_resp({unlock_scale_req, comm:this()},\n                                          scale_req_resp, 5);\n        false -> {error, autoscale_false}\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Alarm state API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% @doc Toggle state of alarm Name from active to inactive and vice versa.\n-spec toggle_alarm(Name :: atom()) -> {ok, {new_state, NewState :: active | inactive}} |\n                                      {error, unknown_alarm | tx_fail} |\n                                      autoscale_error_resp().\ntoggle_alarm(Name) ->\n    case autoscale:check_config() of\n        true  -> send_to_leader_wait_resp({toggle_alarm, Name, comm:this()},\n                                          toggle_alarm_resp, 5);\n        false -> {error, autoscale_false}\n    end.\n\n%% @doc Set all alarms to active.\n-spec activate_alarms() -> ok | {error, tx_fail} | autoscale_error_resp().\nactivate_alarms () ->\n    case autoscale:check_config() of\n        true  -> send_to_leader_wait_resp({activate_alarms, comm:this()},\n                                          activate_alarms_resp, 5);\n        false -> {error, autoscale_false}\n    end.\n\n%% @doc Set all alarms to inactive.\n-spec deactivate_alarms() -> ok | {error, tx_fail} | autoscale_error_resp().\ndeactivate_alarms () ->\n    case autoscale:check_config() of\n        true  -> send_to_leader_wait_resp({deactivate_alarms, comm:this()},\n                                          deactivate_alarms_resp, 5);\n        false -> {error, autoscale_false}\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Autoscale server API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec write_plot_data() -> ok | {error, mgmt_server_false | autoscale_server_false}.\nwrite_plot_data() ->\n    ?IIF(config:read(autoscale_server),\n        case MgmtServer = config:read(mgmt_server) of\n            failed -> {error, mgmt_server_false};\n            _      ->\n                comm:send(MgmtServer, {write_to_file}, [{group_member, autoscale_server}])\n        end,\n        {error, autoscale_server_false}).\n\n-spec reset_plot_data() -> ok | {error, mgmt_server_false | autoscale_server_false}.\nreset_plot_data() ->\n    ?IIF(config:read(autoscale_server),\n        case MgmtServer = config:read(mgmt_server) of\n            failed -> {error, mgmt_server_false};\n            _      ->\n                comm:send(MgmtServer, {reset}, [{group_member, autoscale_server}])\n        end,\n        {error, autoscale_server_false}).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Misc\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% @doc Send to autoscale leader.\n-spec send_to_leader(Msg :: comm:message()) -> ok.\nsend_to_leader(Msg) ->\n    api_dht_raw:unreliable_lookup(?RT:hash_key(\"0\"),\n                                  {?send_to_group_member, autoscale, Msg}).\n\n%% @doc Send to autoscale leader and wait for response.\n-spec send_to_leader_wait_resp(Msg :: comm:message(),\n                               RespTag :: comm:msg_tag(),\n                               Timeout :: pos_integer()) ->\n          Resp :: term() | {error, resp_timeout}.\nsend_to_leader_wait_resp(Msg, RespTag, Timeout) ->\n    send_to_leader(Msg),\n    receive\n        {RespTag, Resp} -> Resp\n    after\n        Timeout*1000    -> {error, resp_timeout}\n    end.\n\n%% @doc Check whether scale requests are pulled or pushed to cloud_module.\n-spec is_in_pull_mode() -> boolean().\nis_in_pull_mode() ->\n    config:read(autoscale_cloud_module) =:= cloud_cps.\n"
  },
  {
    "path": "src/api_dht.erl",
    "content": "%  @copyright 2007-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc API for access to the non replicated DHT items.\n%% @end\n%% @version $Id$\n-module(api_dht).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([hash_key/1]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-spec hash_key(client_key()) -> ?RT:key().\nhash_key(Key) -> ?RT:hash_key(Key).\n"
  },
  {
    "path": "src/api_dht_raw.erl",
    "content": "%% @copyright 2011-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc API for raw access to DHT based on (already) hashed keys.\n%% @version $Id$\n-module(api_dht_raw).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-export([get_replica_keys/1, range_read/2, split_ring/1]).\n\n-export([unreliable_lookup/2,\n         unreliable_get_key/1, unreliable_get_key/3]).\n\n-include(\"scalaris.hrl\").\n\n-spec get_replica_keys(?RT:key()) -> [?RT:key()].\nget_replica_keys(Key) -> ?RT:get_replica_keys(Key).\n\n%% userdevguide-begin api_dht_raw:lookup\n-spec unreliable_lookup(Key::?RT:key(), Msg::comm:message()) -> ok.\nunreliable_lookup(Key, Msg) ->\n    comm:send_local(pid_groups:find_a(routing_table), {?lookup_aux, Key, 0, Msg}).\n\n-spec unreliable_get_key(Key::?RT:key()) -> ok.\nunreliable_get_key(Key) ->\n    unreliable_lookup(Key, {?get_key, comm:this(), noid, Key}).\n\n-spec unreliable_get_key(CollectorPid::comm:mypid(),\n                         ReqId::{rdht_req_id, pos_integer()},\n                         Key::?RT:key()) -> ok.\nunreliable_get_key(CollectorPid, ReqId, Key) ->\n    unreliable_lookup(Key, {?get_key, CollectorPid, ReqId, Key}).\n%% userdevguide-end api_dht_raw:lookup\n\n%% @doc Read a range of key-value pairs between the given two keys (inclusive).\n-spec range_read(intervals:key(), intervals:key())\n                -> {ok | timeout, [db_entry:entry()]}.\nrange_read(From, To) ->\n    Interval = case From of\n                   To -> intervals:all();\n                   _  -> intervals:new('[', From, To, ']')\n               end,\n    range_read(Interval).\n\n%% @doc Read a range of key-value pairs in the given interval.\n-spec range_read(intervals:interval()) -> {ok | timeout, [db_entry:entry()]}.\nrange_read(Interval) ->\n    Id = uid:get_global_uid(),\n    bulkowner:issue_bulk_owner(Id, Interval,\n                               {bulk_read_entry, comm:this()}),\n    TimerRef = comm:send_local_after(config:read(range_read_timeout), self(),\n                                     {range_read_timeout, Id}),\n    range_read_loop(Interval, Id, intervals:empty(), [], TimerRef).\n\n-spec range_read_loop(Interval::intervals:interval(), Id::uid:global_uid(),\n        Done::intervals:interval(), Data::[[db_entry:entry()]],\n        TimerRef::reference()) -> {ok | timeout, [db_entry:entry()]}.\nrange_read_loop(Interval, Id, Done, Data, TimerRef) ->\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({range_read_timeout, Id}, %% ->\n            {timeout, lists:append(Data)});\n        ?SCALARIS_RECV(\n        {bulkowner, reply, Id, {bulk_read_entry_response, NowDone, NewData}}, %% ->\n           begin\n            Done2 = intervals:union(NowDone, Done),\n            case intervals:is_subset(Interval, Done2) of\n                false ->\n                    range_read_loop(Interval, Id, Done2, [NewData | Data], TimerRef);\n                true ->\n                    delete_and_cleanup_timer(TimerRef, Id),\n                    {ok, lists:append([NewData | Data])}\n            end\n           end)\n    end.\n\n-spec delete_and_cleanup_timer(reference(), uid:global_uid()) -> ok.\ndelete_and_cleanup_timer(TimerRef, Id) ->\n    %% cancel timeout\n    _ = erlang:cancel_timer(TimerRef),\n    %% consume potential timeout message\n    %% note: do not yield trace_mpath thread with \"after 0\"!\n    receive\n        ?SCALARIS_RECV({range_read_timeout, Id}, ok) %% -> ok\n    after 0 -> ok\n    end.\n\n-spec split_ring(pos_integer()) -> [?RT:key()].\nsplit_ring(Parts) ->\n    [?MINUS_INFINITY | ?RT:get_split_keys(?MINUS_INFINITY, ?PLUS_INFINITY, Parts)].\n"
  },
  {
    "path": "src/api_monitor.erl",
    "content": "%% @copyright 2011-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc API for monitoring individual nodes and the whole ring\n%% @version $Id$\n-module(api_monitor).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([get_node_info/0, get_node_performance/0, get_service_info/0, get_service_performance/0]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% @doc Gets some information about the current Erlang node.\n%%      Note: The list of returned tuples may grow in future versions; do not\n%%            expect that only the given values are returned!\n-spec get_node_info() -> [{scalaris_version | erlang_version, nonempty_string()} |\n                          {dht_nodes, non_neg_integer()},...].\nget_node_info() ->\n    %MyMonitor = pid_groups:pid_of(clients_group, monitor),\n    %statistics:getTimingMonitorStats(MyMonitor, Keys, tuple),\n    [{scalaris_version, ?SCALARIS_VERSION},\n     {erlang_version, erlang:system_info(otp_release)},\n     {dht_nodes, length(pid_groups:find_all(dht_node))}].\n\n-spec get_node_performance() -> list().\nget_node_performance() ->\n    Monitor = pid_groups:pid_of(basic_services, monitor),\n    {_CountD, _CountPerSD, AvgMsD, _MinMsD, _MaxMsD, StddevMsD, _HistMsD} =\n        case statistics:getTimingMonitorStats(Monitor, [{monitor_perf, 'read_read'}], tuple) of\n            []                                  -> {[], [], [], [], [], [], []};\n            [{monitor_perf, 'read_read', Data}] -> Data\n        end,\n    [{latency_avg, AvgMsD},\n     {latency_stddev, StddevMsD}].\n\n-spec get_service_info() -> list().\nget_service_info() ->\n    Ring = statistics:get_ring_details(),\n    [{total_load, statistics:get_total_load(load, Ring)},\n     {nodes, length(Ring)}].\n\n-spec get_service_performance() -> list().\nget_service_performance() ->\n    Monitor = pid_groups:pid_of(basic_services, monitor),\n    {_CountD, _CountPerSD, AvgMsD, _MinMsD, _MaxMsD, StddevMsD, _HistMsD} =\n        case statistics:getTimingMonitorStats(Monitor, [{monitor_perf, 'agg_read_read'}], tuple) of\n            []                                  -> {[], [], [], [], [], [], []};\n            [{monitor_perf, 'agg_read_read', Data}] -> Data\n        end,\n    [{latency_avg, AvgMsD},\n     {latency_stddev, StddevMsD}].\n"
  },
  {
    "path": "src/api_mr.erl",
    "content": "%  @copyright 2011-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc    API functions for the Map-Reduce system\n%%\n%%         The Map-Reduce system will snapshot the database and apply the the\n%%         supplied job to that data. Since snapshots work on the overlay level\n%%         only the hashed keys are available.\n%%         Since hashed keys are not useful in this context, the MR system will\n%%         only considers values of the following structure as inputs:\n%%         ```\n%%         {Key::string(), Value::term()}\n%%         or\n%%         {Tag::atom(), Key:string(), Value::term()}\n%%         '''\n%%         Either all 2-tuples are considered to be input or, if\n%%         `{tag, some_tag}' is found in the options list, all 3-tuples with `some_tag'\n%%         as the first element are.\n%%\n%%         A job description is a 2-tuple where the first element is a list of\n%%         phase specifications and the second element a list of options.\n%%         A simple example job could look like this:\n%%         ```\n%%            Map = fun({_Key, Line}) ->\n%%                Tokens = string:tokens(Line, \" \\n,.;:?!()\\\"'-_\"),\n%%                [{string:to_lower(X),1} || X <- Tokens]\n%%            end,\n%%            Reduce = fun(KVList) ->\n%%                    lists:map(fun({K, V}) ->\n%%                                      {K, lists:sum(V)}\n%%                                end, KVList)\n%%            end,\n%%            api_mr:start_job({[{map, erlanon, Map},\n%%                               {reduce, erlanon, Reduce}],\n%%                              []}).\n%%         '''\n%%         It considers all `{string(), string()}' as input and returns the word count of all\n%%         values.\n%%\n%% @end\n%% @version $Id$\n-module(api_mr).\n-author('fajerski@zib.de').\n-vsn('$Id$').\n\n%% -define(TRACE(X, Y), io:format(X, Y)).\n-define(TRACE(X, Y), ok).\n\n-export([start_job/1]).\n\n-include(\"scalaris.hrl\").\n\n%% @doc synchronous call to start a map reduce job.\n%%      it will return the results of the job.\n-spec start_job(mr_state:job_description()) -> [any()].\nstart_job(Job) ->\n    Id = randoms:getRandomString(),\n    api_dht_raw:unreliable_lookup(api_dht:hash_key(Id), {mr_master, init, comm:this(), Id, Job}),\n    wait_for_results([], intervals:empty(), Id).\n\n-spec wait_for_results([any()], intervals:interval(), mr_state:jobid()) -> [any()].\nwait_for_results(Data, Interval, Id) ->\n    {NewData, NewInterval} =\n        begin\n            trace_mpath:thread_yield(),\n            receive\n                ?SCALARIS_RECV({mr_results, PartData, PartInterval, Id},\n                               case PartData of\n                                   {error, Reason} ->\n                                       {[{error, Reason}], intervals:all()};\n                                   PartData ->\n                                       {[PartData | Data], intervals:union(PartInterval, Interval)}\n                               end)\n                end\n        end,\n    ?TRACE(\"mr_api: received data for job ~p: ~p~n\", [Id, hd(NewData)]),\n    case intervals:is_all(NewInterval) of\n        true ->\n            lists:append(NewData);\n        _ ->\n            wait_for_results(NewData, NewInterval, Id)\n    end.\n"
  },
  {
    "path": "src/api_rdht.erl",
    "content": "%% @copyright 2011-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc API for inconsistent access to the replicated DHT items. This\n%%      is not compatible with the api_tx functions.\n%% @version $Id$\n-module(api_rdht).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-export([get_replica_keys/1]).\n-export([delete/1, delete/2]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-type(delete_result() ::\n        {ok, ResultsOk::non_neg_integer(),\n         ResultList::[ok | locks_set | undef]}\n      | {fail, timeout, ResultsOk::non_neg_integer(),\n         ResultList::[ok | locks_set | undef]}).\n\n-spec delete(Key::client_key()) -> delete_result().\ndelete(Key) -> delete(Key, 2000).\n\n%% @doc try to delete the given key and return a list of replicas\n%%      successfully deleted.  WARNING: this function can lead to\n%%      inconsistencies for api_tx functions.\n-spec delete(client_key(), Timeout::pos_integer()) -> delete_result().\ndelete(Key, Timeout) ->\n    ClientsId = {delete_client_id, uid:get_global_uid()},\n    ReplicaKeys = ?RT:get_replica_keys(?RT:hash_key(Key)),\n    _ = [ api_dht_raw:unreliable_lookup(\n            Replica, {delete_key, comm:this(), ClientsId, Replica})\n          || Replica <- ReplicaKeys],\n    msg_delay:send_local_as_client(Timeout div 1000, self(),\n                                   {timeout, ClientsId}),\n    delete_collect_results(ReplicaKeys, ClientsId, []).\n\n%% @doc collect the response for the delete requests\n-spec delete_collect_results(ReplicaKeys::[?RT:key()],\n                             ClientsId::{delete_client_id, uid:global_uid()},\n                             Results::[ok | locks_set | undef])\n                            -> delete_result().\ndelete_collect_results([], _ClientsId, Results) ->\n    OKs = length([ok || ok <- Results]),\n    {ok, OKs, Results};\ndelete_collect_results([_|_] = ReplicaKeys, ClientsId, Results) ->\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({delete_key_response, ClientsId, Key, Result}, %% ->\n            case lists:member(Key, ReplicaKeys) of\n                true ->\n                    delete_collect_results(lists:delete(Key, ReplicaKeys),\n                                           ClientsId, [Result | Results]);\n                false ->\n                    delete_collect_results(ReplicaKeys, ClientsId, Results)\n            end);\n        ?SCALARIS_RECV({timeout, ClientsId}, %% ->\n            begin\n               OKs = length([ok || ok <- Results]),\n               {fail, timeout, OKs, Results}\n            end);\n        ?SCALARIS_RECV({delete_key_response, _, _, _}, %% ->\n            %% probably an outdated message: drop it.\n            delete_collect_results(ReplicaKeys, ClientsId, Results));\n        ?SCALARIS_RECV({timeout, _}, %% ->\n            %% probably an outdated message: drop it.\n            delete_collect_results(ReplicaKeys, ClientsId, Results))\n    end.\n\n-spec get_replica_keys(client_key()) -> [?RT:key()].\nget_replica_keys(Key) -> ?RT:get_replica_keys(?RT:hash_key(Key)).\n"
  },
  {
    "path": "src/api_ring.erl",
    "content": "%% @copyright 2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schütt <schuett@zib.de>\n%% @doc API for querying properties of rings.\n%%\n-module(api_ring).\n-author('schuett@zib.de').\n\n-export([get_ring_size/1, wait_for_ring_size/2]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-spec get_ring_size(TimeOut::integer()) -> failed | integer().\nget_ring_size(TimeOut) ->\n    Id = uid:get_global_uid(),\n    bulkowner:issue_bulk_owner(Id, intervals:all(),\n                               {deliver_ping, comm:this(), {get_ring_size, Id}}),\n    get_ring_size_internal(intervals:empty(), 0, TimeOut, Id).\n\nget_ring_size_internal(Interval, Count, TimeOut, Id) ->\n    receive\n        {pong, AnInterval, Id, {get_ring_size, Id}} ->\n            Interval2 = intervals:union(Interval, AnInterval),\n            case intervals:is_all(Interval2) of\n                true ->\n                    Count + 1;\n                false ->\n                    get_ring_size_internal(Interval2, Count + 1, TimeOut, Id)\n            end;\n        X ->\n            io:format(\"~w~n\", [X])\n    after\n        TimeOut ->\n            failed\n    end.\n\n-spec wait_for_ring_size(Size::integer(), TimeOut::integer()) -> ok.\nwait_for_ring_size(Size, TimeOut) ->\n    case get_ring_size(TimeOut) of\n        Size ->\n            ok;\n        _ ->\n            wait_for_ring_size(Size, TimeOut)\n    end.\n"
  },
  {
    "path": "src/api_rt.erl",
    "content": "%  @copyright 2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de> \n%% @doc API for using the routing table.\n%%  @end\n-module(api_rt).\n-author('schuett@zib.de').\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([get_replication_factor/0, get_evenly_spaced_keys/1, escaped_list_of_keys/1]).\n\n-spec get_replication_factor() -> pos_integer().\nget_replication_factor() ->\n    config:read(replication_factor).\n\n-spec get_evenly_spaced_keys(N::pos_integer()) -> list(?RT:key()).\nget_evenly_spaced_keys(N) ->\n    case config:read(replication_factor) of\n        N ->\n            ?RT:get_replica_keys(?MINUS_INFINITY);\n        _ ->\n            [?MINUS_INFINITY | \n             ?RT:get_split_keys(?MINUS_INFINITY, ?PLUS_INFINITY, N)]\n    end.\n\n-spec escaped_list_of_keys(list(?RT:key())) ->  string().\nescaped_list_of_keys(Keys) ->\n    tl(lists:foldl(fun (Key, S) ->\n                        case erlang:is_list(Key) of\n                            true ->\n                                S ++ \" \" ++ util:escape_quotes(Key);\n                            false ->\n                                S ++ \" \" ++ erlang:integer_to_list(Key)\n                        end\n                end, \"\", Keys)).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% helper functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n"
  },
  {
    "path": "src/api_tx.erl",
    "content": "%% @copyright 2011, 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>,\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc API for transactional access to replicated DHT items.\n%% For single request/single item operations, we provide read/1,\n%% write/2, test_and_set/3, add_on_nr/2 and add_del_on_list/3\n%% functions that directly commit.\n%%\n%% For compound transactions a transaction log has to be passed\n%% through all operations and finally has to be committed. This is\n%% supported by the functions new_tlog/0, read/2, write/3, req_list/2,\n%% and commit/1.\n%% @end\n%% @version $Id$\n-module(api_tx).\n-author('schintke@zib.de').\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-export_type([request/0, request_on_key/0,\n              request_enc/0, request_on_key_enc/0,\n              read_request/0, read_random_from_list_request/0,\n              read_sublist_request/0,\n              write_request/0,\n              add_del_on_list_request/0, add_on_nr_request/0,\n              test_and_set_request/0]).\n-export_type([result/0, read_result/0, read_random_from_list_result/0,\n              read_sublist_result/0,\n              write_result/0, commit_result/0]).\n-export_type([listop_result/0, numberop_result/0, testandset_result/0]).\n-export_type([client_key/0]).\n\n-include(\"api_tx.hrl\").\n\n%% @doc Perform several requests inside a transaction and monitors its\n%%      execution time.\n-spec req_list(tx_tlog:tlog_ext(), [request()]) -> {tx_tlog:tlog_ext(), [result()]}.\nreq_list(TLog, ReqList) ->\n    {TimeInUs, Result} = util:tc(fun rdht_tx:req_list/3, [TLog, ReqList, true]),\n    monitor:client_monitor_set_value(api_tx, 'req_list', TimeInUs / 1000),\n    Result.\n"
  },
  {
    "path": "src/api_tx.hrl",
    "content": "%% @copyright 2011, 2012, 2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>,\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc API for transactional access to replicated DHT items.\n%% For single request/single item operations, we provide read/1,\n%% write/2, and test_and_set/3 functions that directly commit.\n%%\n%% For compound transactions a transaction log has to be passed\n%% through all operations and finally has to be committed. This is\n%% supported by the functions new_tlog/0, read/2, write/3, req_list/2,\n%% and commit/1.\n%% @end\n%% @version $Id$\n\n%% Perform a chain of operations (passing a transaction log) and\n%% finally commit the transaction.\n-export([new_tlog/0, req_list/1, req_list/2,\n         read/2, write/3, add_del_on_list/4, add_on_nr/3, test_and_set/4, commit/1]).\n\n%% Perform single operation transactions.\n-export([req_list_commit_each/1,\n         read/1, write/2, add_del_on_list/3, add_on_nr/2, test_and_set/3,\n         get_system_snapshot/0]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n% Public Interface\n-type read_request() :: {read, client_key()}.\n-type read_random_from_list_request() :: {read, client_key(), random_from_list}.\n-type read_sublist_request() :: {read, client_key(), {sublist, Start::pos_integer() | neg_integer(), Len::integer()}}.\n-type write_request() :: {write, client_key(), client_value()}.\n-type write_request_enc() :: {write, client_key(), rdht_tx:encoded_value()}.\n-type add_del_on_list_request() ::\n          {add_del_on_list, client_key(),\n           client_value(),  %% abort when not ToAdd::[client_value()],\n           client_value()}.  %% abort when not ToRemove::[client_value()]}\n-type add_del_on_list_request_enc() ::\n          {add_del_on_list, client_key(),\n           rdht_tx:encoded_value(),   %% abort when not decoded to [client_value()],\n           rdht_tx:encoded_value()}.  %% abort when not decoded to [client_value()]}\n-type add_on_nr_request() ::\n          {add_on_nr, client_key(),\n           client_value()}. %% abort when not number()}\n-type add_on_nr_request_enc() ::\n          {add_on_nr, client_key(),\n           rdht_tx:encoded_value()}. %% abort when not decoded to number()}\n-type test_and_set_request() ::\n          {test_and_set, client_key(),\n           Old::client_value(), New::client_value()}.\n-type test_and_set_request_enc() ::\n          {test_and_set, client_key(),\n           Old::rdht_tx:encoded_value(), New::rdht_tx:encoded_value()}.\n\n-type request_on_key() ::\n          read_request()\n        | read_random_from_list_request()\n        | read_sublist_request()\n        | write_request()\n        | add_del_on_list_request()\n        | add_on_nr_request()\n        | test_and_set_request().\n-type request_on_key_enc() ::\n          read_request()\n        | read_random_from_list_request()\n        | read_sublist_request()\n        | write_request_enc()\n        | add_del_on_list_request_enc()\n        | add_on_nr_request_enc()\n        | test_and_set_request_enc().\n-type request() :: request_on_key() | {commit}.\n-type request_enc() :: request_on_key_enc() | {commit}.\n\n-type read_result() :: {ok, client_value()} | {fail, not_found}.\n-type read_random_from_list_result() :: {ok, RandomValue_ListLen_MaybeEncoded::client_value()} | {fail, not_found | empty_list | not_a_list}.\n-type read_sublist_result() :: {ok, SubList_ListLen_MaybeEncoded::client_value()} | {fail, not_found | not_a_list}.\n-type write_result() :: {ok}.\n-type listop_result() :: write_result() | {fail, not_a_list}.\n-type numberop_result() :: write_result() | {fail, not_a_number}.\n-type commit_result() :: {ok} | {fail, abort, [client_key()]}.\n-type testandset_result() ::\n          write_result()\n        | {fail, not_found | {key_changed, RealOldValue::client_value()}}.\n-type result() ::\n          read_result()\n        | read_random_from_list_result()\n        | read_sublist_result()\n        | write_result()\n        | listop_result()\n        | numberop_result()\n        | testandset_result()\n        | commit_result().\n\n%% @doc Get an empty transaction log to start a new transaction.\n-spec new_tlog() -> tx_tlog:tlog_ext().\nnew_tlog() -> tx_tlog:empty().\n\n%% @doc Perform several requests starting a new transaction.\n-spec req_list([request()]) -> {tx_tlog:tlog_ext(), [result()]}.\nreq_list(ReqList) ->\n    req_list(new_tlog(), ReqList).\n\n%% @doc Perform a read inside a transaction.\n-spec read(tx_tlog:tlog_ext(), client_key())\n          -> {tx_tlog:tlog_ext(), read_result()}.\nread(TLog, Key) ->\n    {NewTLog, [Result]} = req_list(TLog, [{read, Key}]),\n    {NewTLog, Result}.\n\n%% @doc Perform a write inside a transaction.\n-spec write(tx_tlog:tlog_ext(), client_key(), client_value())\n           -> {tx_tlog:tlog_ext(), write_result()}.\nwrite(TLog, Key, Value) ->\n    {NewTLog, [Result]} = req_list(TLog, [{write, Key, Value}]),\n    {NewTLog, Result}.\n\n%% @doc Perform a add_del_on_list operation inside a transaction.\n-spec add_del_on_list(tx_tlog:tlog_ext(), client_key(), ToAdd::[client_value()], ToRemove::[client_value()])\n           -> {tx_tlog:tlog_ext(), listop_result()}.\nadd_del_on_list(TLog, Key, ToAdd, ToRemove) ->\n    {NewTLog, [Result]} = req_list(TLog, [{add_del_on_list, Key, ToAdd, ToRemove}]),\n    {NewTLog, Result}.\n\n%% @doc Perform a add_del_on_list operation inside a transaction.\n-spec add_on_nr(tx_tlog:tlog_ext(), client_key(), ToAdd::number())\n           -> {tx_tlog:tlog_ext(), numberop_result()}.\nadd_on_nr(TLog, Key, ToAdd) ->\n    {NewTLog, [Result]} = req_list(TLog, [{add_on_nr, Key, ToAdd}]),\n    {NewTLog, Result}.\n\n%% @doc Atomically compare and set a value for a key inside a transaction.\n%%      If the value stored at Key is the same as OldValue, then NewValue will\n%%      be stored.\n-spec test_and_set(tx_tlog:tlog_ext(), Key::client_key(), OldValue::client_value(), NewValue::client_value())\n        -> {tx_tlog:tlog_ext(), testandset_result()}.\ntest_and_set(TLog, Key, OldValue, NewValue) ->\n    {NewTLog, [Result]} = req_list(TLog, [{test_and_set, Key, OldValue, NewValue}]),\n    {NewTLog, Result}.\n\n%% @doc Finish a transaction and materialize it atomically on the DHT.\n-spec commit(tx_tlog:tlog_ext()) -> commit_result().\ncommit(TLog) ->\n    {_NewTLog, [Result]} = req_list(TLog, [{commit}]),\n    Result.\n\n%% @doc Atomically read a single key (not as part of a transaction).\n-spec read(client_key()) -> read_result().\nread(Key) ->\n    ReqList = [{read, Key}],\n    {_TLog, [Res]} = req_list(tx_tlog:empty(), ReqList),\n    Res.\n\n%% @doc Atomically write and commit a single key (not as part of a transaction).\n-spec write(client_key(), client_value()) -> commit_result().\nwrite(Key, Value) ->\n    ReqList = [{write, Key, Value}, {commit}],\n    {_TLog, [_Res1, Res2]} = req_list(tx_tlog:empty(), ReqList),\n    Res2.\n\n%% @doc Atomically perform a add_del_on_list operation and a commit (not as part of a transaction).\n-spec add_del_on_list(client_key(), ToAdd::[client_value()], ToRemove::[client_value()])\n           -> listop_result() | {fail, abort, [client_key()]}.\nadd_del_on_list(Key, ToAdd, ToRemove) ->\n    ReqList = [{add_del_on_list, Key, ToAdd, ToRemove}, {commit}],\n    {_TLog, [Res1, Res2]} = req_list(tx_tlog:empty(), ReqList),\n    case Res1 of\n        X when erlang:is_tuple(X) andalso erlang:element(1, X) =:= fail -> X;\n        _ -> Res2\n    end.\n\n%% @doc Atomically perform a add_del_on_list operation and a commit (not as part of a transaction).\n-spec add_on_nr(client_key(), ToAdd::number())\n           -> numberop_result() | {fail, abort, [client_key()]}.\nadd_on_nr(Key, ToAdd) ->\n    ReqList = [{add_on_nr, Key, ToAdd}, {commit}],\n    {_TLog, [Res1, Res2]} = req_list(tx_tlog:empty(), ReqList),\n    case Res1 of\n        X when erlang:is_tuple(X) andalso erlang:element(1, X) =:= fail -> X;\n        _ -> Res2\n    end.\n\n%% @doc Atomically compare and set a value for a key (not as part of a transaction).\n%%      If the value stored at Key is the same as OldValue, then NewValue will\n%%      be stored.\n-spec test_and_set(Key::client_key(), OldValue::client_value(), NewValue::client_value())\n        -> testandset_result() | {fail, abort, [client_key()]}.\ntest_and_set(Key, OldValue, NewValue) ->\n    ReqList = [{test_and_set, Key, OldValue, NewValue}, {commit}],\n    {_TLog, [Res1, Res2]} = req_list(tx_tlog:empty(), ReqList),\n    case Res1 of\n        X when erlang:is_tuple(X) andalso erlang:element(1, X) =:= fail -> X;\n        _ -> Res2\n    end.\n\n-spec commit_req(request_on_key()) -> result().\ncommit_req(Request) ->\n    case Request of\n        {read, Key} -> read(Key);\n        {write, Key, Value} -> write(Key, Value);\n        {add_del_on_list, Key, ToAdd, ToRemove} -> add_del_on_list(Key, ToAdd, ToRemove);\n        {add_on_nr, Key, ToAdd} -> add_on_nr(Key, ToAdd);\n        {test_and_set, Key, Old, New} -> test_and_set(Key, Old, New);\n        {read, _Key, _Op} = Req ->\n            ReqList = [Req],\n            {_TLog, [Res]} = req_list(tx_tlog:empty(), ReqList),\n            Res;\n        _ -> {fail, abort, []}\n    end.\n\n%% @doc Perform several requests in parallel and commit each.\n%%      The execution order of multiple requests on the same key is undefined!\n-spec req_list_commit_each([request_on_key()]) -> [result()].\nreq_list_commit_each(ReqList) ->\n    util:par_map(fun commit_req/1, ReqList, 50).\n\n%% @doc Get system snapshot\n-spec get_system_snapshot() -> [tuple()] | {snapshot_failed,\n                                            intervals:interval(),\n                                            [tuple()]}.\nget_system_snapshot() ->\n    Key = ?RT:hash_key(randoms:getRandomString()),\n    api_dht_raw:unreliable_lookup(Key, {?send_to_group_member, snapshot_leader,\n                                       {init_snapshot, comm:this()}}),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({global_snapshot_done, Data}, Data);\n        ?SCALARIS_RECV({global_snapshot_done_with_errors, ErrorInterval, Data},\n                       {snapshot_failed, ErrorInterval, Data})\n    end.\n"
  },
  {
    "path": "src/api_txc.erl",
    "content": "%% @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>,\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc API for compressed transactional access to replicated DHT items.\n%% Same as api_tx but transmits compressed values in both request and result\n%% list.\n%% @end\n%% @version $Id$\n-module(api_txc).\n-author('schintke@zib.de').\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-export_type([request/0, request_on_key/0,\n              request_enc/0, request_on_key_enc/0,\n              read_request/0, read_random_from_list_request/0,\n              read_sublist_request/0,\n              write_request/0,\n              add_del_on_list_request/0, add_on_nr_request/0,\n              test_and_set_request/0]).\n-export_type([result/0, read_result/0, read_random_from_list_result/0,\n              read_sublist_result/0,\n              write_result/0, commit_result/0]).\n-export_type([listop_result/0, numberop_result/0, testandset_result/0]).\n-export_type([client_key/0]).\n\n-include(\"api_tx.hrl\").\n\n%% @doc Perform several requests inside a transaction and monitors its\n%%      execution time.\n-spec req_list(tx_tlog:tlog(), [request()]) -> {tx_tlog:tlog(), [result()]}.\nreq_list(TLog, ReqList) ->\n    {TimeInUs, Result} = util:tc(fun rdht_tx:req_list/3, [TLog, ReqList, false]),\n    monitor:client_monitor_set_value(api_tx, 'req_list', TimeInUs / 1000),\n    Result.\n"
  },
  {
    "path": "src/api_vm.erl",
    "content": "%  @copyright 2011-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Administrative functions available for external programs.\n%% @end\n%% @version $Id$\n-module(api_vm).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-export([get_version/0, get_info/0,\n         number_of_nodes/0, get_nodes/0, add_nodes/1, add_nodes_at_ids/1,\n         shutdown_node/1, shutdown_nodes/1, shutdown_nodes_by_name/1,\n         kill_node/1, kill_nodes/1, kill_nodes_by_name/1,\n         get_other_vms/1,\n         shutdown_vm/0, kill_vm/0,\n         wait_for_scalaris_to_start/0]).\n\n-include(\"scalaris.hrl\").\n\n%% @doc Gets the version of Scalaris.\n-spec get_version() -> string().\nget_version() ->\n    ?SCALARIS_VERSION.\n\n%% @doc Gets some information about the VM and Scalaris.\n-spec get_info() -> [{scalaris_version | erlang_version, string()} |\n                     {mem_total, non_neg_integer()} | {uptime, Ms::non_neg_integer()} |\n                     {erlang_node, node()} | {ip, inet:ip_address()} |\n                     {port, non_neg_integer()} | {yaws_port, non_neg_integer()}].\nget_info() ->\n    [{scalaris_version, ?SCALARIS_VERSION},\n     {erlang_version, erlang:system_info(otp_release)},\n     {mem_total, erlang:memory(total)},\n     {uptime, erlang:element(1, erlang:statistics(wall_clock))},\n     {erlang_node, node()},\n     {ip, comm:get_ip(comm:this())},\n     {port, comm:get_port(comm:this())},\n     {yaws_port, config:read(yaws_port)}\n    ].\n\n%% @doc Gets the number of Scalaris nodes inside this VM.\n-spec number_of_nodes() -> non_neg_integer().\nnumber_of_nodes() ->\n    length(get_nodes()).\n\n%% @doc Gets the names of all Scalaris nodes inside this VM.\n-spec get_nodes() -> [pid_groups:groupname()].\nget_nodes() ->\n    DhtModule = config:read(dht_node),\n    [pid_groups:group_of(Pid) || Pid <- pid_groups:find_all(dht_node),\n                                 DhtModule:is_alive(gen_component:get_state(Pid))].\n\n%% userdevguide-begin api_vm:add_nodes\n%% @doc Adds Number Scalaris nodes to this VM.\n-spec add_nodes(non_neg_integer()) -> {[pid_groups:groupname()], [{error, term()}]}.\nadd_nodes(Number) when is_integer(Number) andalso Number >= 0 ->\n    Result = {Ok, _Failed} = admin:add_nodes(Number),\n    % at least wait for the successful nodes to have joined, i.e. left the join phases\n    util:wait_for(\n      fun() ->\n              DhtModule = config:read(dht_node),\n              NotReady = [Name || Name <- Ok,\n                                  not DhtModule:is_alive(\n                                    gen_component:get_state(\n                                      pid_groups:pid_of(Name, dht_node)))],\n              [] =:= NotReady\n      end),\n    Result.\n%% userdevguide-end api_vm:add_nodes\n\n-spec add_nodes_at_ids(list(?RT:key())) -> {[pid_groups:groupname()], [{error, term()}]}.\nadd_nodes_at_ids(Keys) when is_list(Keys) ->\n    Result = {Ok, _Failed} = admin:add_nodes_at_ids(Keys),\n    % at least wait for the successful nodes to have joined, i.e. left the join phases\n    util:wait_for(\n      fun() ->\n              DhtModule = config:read(dht_node),\n              NotReady = [Name || Name <- Ok,\n                                  not DhtModule:is_alive(\n                                    gen_component:get_state(\n                                      pid_groups:pid_of(Name, dht_node)))],\n              [] =:= NotReady\n      end),\n    Result.\n\n%% @doc Wait for the given nodes to disappear.\n-spec wait_for_nodes_to_disappear(Names::[pid_groups:groupname()]) -> ok.\nwait_for_nodes_to_disappear(Names) ->\n    util:wait_for(\n      fun() ->\n              [] =:= [Name || Name <- Names,\n                              pid_groups:pid_of(Name, dht_node) =/= failed]\n      end).\n\n%% @doc Sends a graceful leave request to a given node.\n-spec shutdown_node(pid_groups:groupname()) -> ok | not_found.\nshutdown_node(Name) ->\n    case element(1, shutdown_nodes_by_name([Name])) of\n        [] -> not_found;\n        _  -> ok\n    end.\n%% @doc Sends a graceful leave request to multiple nodes.\n-spec shutdown_nodes(Count::non_neg_integer()) -> Ok::[pid_groups:groupname()].\nshutdown_nodes(Count) when is_integer(Count) andalso Count >= 0 ->\n    Ok = admin:del_nodes(Count, true),\n    wait_for_nodes_to_disappear(Ok),\n    Ok.\n-spec shutdown_nodes_by_name(Names::[pid_groups:groupname()]) -> {Ok::[pid_groups:groupname()], NotFound::[pid_groups:groupname()]}.\nshutdown_nodes_by_name(Names) when is_list(Names) ->\n    Result = {Ok, _NotFound} = admin:del_nodes_by_name(Names, true),\n    wait_for_nodes_to_disappear(Ok),\n    Result.\n\n%% @doc Kills a given node.\n-spec kill_node(pid_groups:groupname()) -> ok | not_found.\nkill_node(Name) ->\n    case element(1, kill_nodes_by_name([Name])) of\n        [] -> not_found;\n        _  -> ok\n    end.\n%% @doc Kills multiple nodes.\n-spec kill_nodes(Count::non_neg_integer()) -> Ok::[pid_groups:groupname()].\nkill_nodes(Count) when is_integer(Count) andalso Count >= 0 ->\n    %% spawn to be sure that all nodes are killed, even when killing\n    %% ourselves.\n    Pid = self(),\n    Child = spawn(\n              fun() -> %% infection is passed via {do_kill_nodes} message\n                      trace_mpath:thread_yield(),\n                      receive ?SCALARIS_RECV({do_kill_nodes}, ok) end,\n                      Ok = admin:del_nodes(Count, false),\n                      wait_for_nodes_to_disappear(Ok),\n                      comm:send_local(Pid, {kill_nodes_done, Ok}),\n                      %% pass flow control back to proto_sched\n                      trace_mpath:thread_yield()\n              end),\n    comm:send_local(Child, {do_kill_nodes}),\n    trace_mpath:thread_yield(),\n    receive ?SCALARIS_RECV({kill_nodes_done, Result}, Result) end.\n\n-spec kill_nodes_by_name(Names::[pid_groups:groupname()]) -> {Ok::[pid_groups:groupname()], NotFound::[pid_groups:groupname()]}.\nkill_nodes_by_name(Names) when is_list(Names) ->\n    %% spawn to be sure that all nodes are killed, even when killing\n    %% ourselves.\n    Pid = self(),\n    spawn(\n      fun() ->\n              Result = {Ok, _NotFound} = admin:del_nodes_by_name(Names, false),\n              wait_for_nodes_to_disappear(Ok),\n              Pid ! {kill_nodes_done, Result}\n            end),\n    trace_mpath:thread_yield(),\n    receive ?SCALARIS_RECV({kill_nodes_done, Result}, Result) end.\n\n%% @doc Gets connection info for a random subset of known nodes by the cyclon\n%%      processes of the dht_node processes in this VM.\n-spec get_other_vms(MaxVMs::pos_integer()) -> [{ErlNode::node(), Ip::inet:ip_address(), Port::non_neg_integer(), YawsPort::non_neg_integer()}].\nget_other_vms(MaxVMs) when is_integer(MaxVMs) andalso MaxVMs > 0 ->\n    DhtModule = config:read(dht_node),\n    RandomConns =\n        lists:append(\n          [begin\n               GlobalPid = comm:make_global(Pid),\n               comm:send(GlobalPid,\n                         {cb_msg, {gossip_cyclon, default}, {get_subset_rand, MaxVMs, self()}},\n                         [{group_member, gossip}]),\n               trace_mpath:thread_yield(),\n               receive\n                   ?SCALARIS_RECV({cy_cache, Cache}, %% ->\n                       [{erlang:node(comm:make_local(node:pidX(Node))),\n                         comm:get_ip(node:pidX(Node)),\n                         comm:get_port(node:pidX(Node)),\n                         node:yawsPort(Node)} || Node <- Cache])\n               end\n           end || Pid <- pid_groups:find_all(dht_node),\n                  DhtModule:is_alive_fully_joined(gen_component:get_state(Pid))]),\n    util:random_subset(MaxVMs, lists:usort(RandomConns)).\n\n%% @doc Graceful shutdown of this VM. If the last Scalaris node of this VM\n%%      cannot gracefully shut down because it doesn't know any other node to\n%%      move its data to, 'no_partner_found' will be returned.\n-spec shutdown_vm() -> ok | no_partner_found.\nshutdown_vm() ->\n    NodesToShutdown = number_of_nodes(),\n    Ok = shutdown_nodes(NodesToShutdown),\n    Rest = erlang:max(0, NodesToShutdown - length(Ok)),\n    util:wait_for(fun() -> number_of_nodes() =:= Rest end),\n    if Rest =:= 0 -> kill_vm();\n       true       -> no_partner_found\n    end.\n\n%% @doc Kills this VM.\n-spec kill_vm() -> ok.\nkill_vm() ->\n    spawn(fun kill_vm2/0),\n    ok.\n\n-spec kill_vm2() -> no_return().\nkill_vm2() ->\n    timer:sleep(1000),\n    erlang:halt().\n\n%% @doc waits until the supervisor has started and the VM has IP+port\n-spec wait_for_scalaris_to_start() -> ok.\nwait_for_scalaris_to_start() ->\n    util:wait_for(fun service_per_vm:is_scalaris_ready/0).\n"
  },
  {
    "path": "src/autoscale.erl",
    "content": "% @copyright 2013-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Ufuk Celebi <celebi@zib.de>\n%% @doc Auto-scaling service.\n%%\n%%      {autoscale, true} in scalaris.local.cfg will enable this service.\n%%\n%%      The main task of the autoscale is to manage a set of installed alarms\n%%      by calling their alarm handlers periodically. Every alarm handler can\n%%      request to add or remove VMs (see alarm_handler/2).\n%%\n%%      Alarms are read from the config as a list of alarm records:\n%%      {autoscale_alarms, [{alarm, Name (req.), Options (opt.),\n%%                                  Interval (opt.), Cooldown (opt.),\n%%                                  State (opt.)}]}.\n%%\n%%      Every provided tuple corresponds to the alarm record type:\n%%        - Name:     the name of the alarm handler (should be unique)\n%%        - Options:  options are alarm handler specific (see alarm_handler/2)\n%%        - Interval: interval between alarm checks in seconds\n%%        - Cooldown: interval after a scale request\n%%        - State:    active or inactive\n%%\n%%      Example:\n%%      {autoscale_alarms, [{alarm, rand_churn, [{lower_limit, -3}, {upper_limit, 10}], 60, 0, inactive},\n%%                          {alarm, lat_avg, [], 60, 120, inactive}]}.\n%%\n%%      In this example, the rand_churn alarm handler is called every 60\n%%      seconds and configured with the options lower_limit and upper_limit.\n%%      The lat_avg alarm handler is called every 60 seconds and falls\n%%      back to the default options (set in the alarm handler), because no\n%%      options are provided. If the lat_avg handler requested to add or remove\n%%      VMs, the handler will be called after 120 seconds in order to allow the\n%%      changes to take effect (cooldown).\n%%\n%%      VMs requests are handled by the cloud module. It can be configured with\n%%        {autoscale_cloud_module, CloudModule}.\n%%\n%%      Possible modules are: cloud_local, cloud_ssh, and cloud_cps. For both\n%%      cloud_local and cloud_ssh, scale requests are pushed by autoscale, for\n%%      cloud_cps, autoscale expects pulling of requests.\n%%\n%%      cloud_cps is a dummy and not actually implemented as a module. Instead,\n%%      the manager of the ConPaaS scalaris service needs to pull the current\n%%      request via the JSON API.\n%% @end\n%% @version $Id$\n-module(autoscale).\n-author('celebi@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"record_helpers.hrl\").\n-include(\"scalaris.hrl\").\n\n%% gen_component functions\n-export([start_link/1, init/1, on/2]).\n%% rm_loop interaction\n-export([send_my_range_req/4]).\n%% misc (check_config for api_autoscale)\n-export([check_config/0]).\n\n%% rm subscription\n-export([rm_exec/5]).\n\n-define(TRACE1(X), ?TRACE(X, [])).\n%% -define(TRACE(X,Y), io:format(\"as: \" ++ X ++ \"~n\",Y)).\n-define(TRACE(_X,_Y), ok).\n\n-define(CLOUD, (config:read(autoscale_cloud_module))).\n-define(AUTOSCALE_TX_KEY, \"d9c966df633f8b1577eacff013166db95917a7002999b6fbb\").\n-define(DEFAULT_LOCK_TIMEOUT, 60).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Types\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-type key_value_list() :: [{Key :: atom(), Value :: term()}].\n\n-record(alarm, {name     = ?required(alarm, name) :: atom(),\n                options  = []                     :: key_value_list(),\n                interval = 60                     :: pos_integer(),\n                cooldown = 90                     :: non_neg_integer(),\n                state    = inactive               :: active | inactive }).\n\n-record(scale_req, {req   = 0        :: integer(),\n                    lock  = unlocked :: locked | unlocked,\n                    mode  = push     :: push | pull}).\n\n-type alarm()     :: #alarm{}.\n-type alarms()    :: [alarm()].\n-type scale_req() :: #scale_req{}.\n-type triggers()  :: [{Name :: atom(), Trigger :: reference() | ok}].\n\n-type(api_message() ::\n          {pull_scale_req, Pid :: pid()} |\n          {unlock_scale_req, Pid :: pid()} |\n          {toggle_alarm, Name :: atom(), Pid :: pid()} |\n          {activate_alarms, Pid :: pid()} |\n          {deactivate_alarms, Pid :: pid()}).\n\n-type(message() ::\n          {get_state_response, MyRange :: intervals:interval()} |\n          {check_alarm, Name :: atom()} |\n          {push_scale_req} |\n          api_message()).\n\n-type state() :: {IsLeader :: boolean(),\n                  Alarms   :: alarms(),\n                  ScaleReq :: scale_req(),\n                  Triggers :: triggers()}.\n\n-export_type([key_value_list/0]).\n\n-include(\"gen_component.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Alarm handlers\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% @doc Check average latency measured by monitor_perf and request scaling if\n%%      average latency is out of provided bounds.\n%%      Options:\n%%        lower_limit_ms: lowest avg. lat. to tolerate before scaling down\n%%        upper_limit_ms: highest avg. lat. to tolerate before scaling up\n%%        vms_to_remove: number of vms to add, when avg. lat. &lt; lower_limit_ms\n%%        vms_to_add: number of vms to add, when avg. lat. &gt; upper_limit_ms\n%% @todo add history option (-> don't check only latest avg lat)\nalarm_handler(lat_avg, Options) ->\n    LoMs        = get_alarm_option(Options, lower_limit_ms, 1000),\n    HiMs        = get_alarm_option(Options, upper_limit_ms, 2000),\n    VmsToRemove = get_alarm_option(Options, vms_to_remove, 1),\n    VmsToAdd    = get_alarm_option(Options, vms_to_add, 1),\n\n    Monitor = pid_groups:pid_of(basic_services, monitor),\n    {_CountD, _CountPerSD, AvgMsD, _MinMsD, _MaxMsD, _StddevMsD, _HistMsD} =\n        case statistics:getTimingMonitorStats(Monitor, [{api_tx, 'agg_req_list'}], tuple) of\n            []                           -> {[], [], [], [], [], [], []};\n            [{api_tx, 'agg_req_list', Data}] -> Data\n        end,\n    [{_, LatestAvgMsD}|_] = AvgMsD,\n\n    log(lat_avg, LatestAvgMsD),\n\n    if\n        LatestAvgMsD < LoMs ->\n            -VmsToRemove;\n        LatestAvgMsD > HiMs ->\n            +VmsToAdd;\n        true ->\n            0\n    end;\n\n%% @doc Pick a number UAR and add or remove VMs by that amount.\n%%      Options:\n%%        lower_limit: max. number of vms to remove\n%%        upper_limit: max. number of vms to add\nalarm_handler(rand_churn, Options) ->\n    Lo = get_alarm_option(Options, lower_limit, -5),\n    Hi = get_alarm_option(Options, upper_limit, 5),\n    Churn = randoms:rand_uniform(Lo, Hi+1),\n\n    log(rand_churn, Churn),\n\n    Churn.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Msg loop: main functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec on(message(), state()) -> state().\n\n%% @doc Set new leader state at startup and ring changes.\non({get_state_response, MyRange}, {IsLeader, Alarms, ScaleReq, Triggers}) ->\n    IsNewLeader = intervals:in(?RT:hash_key(\"0\"), MyRange),\n\n    {NewAlarms, NewScaleReq, NewTriggers} =\n        case {IsLeader, IsNewLeader} of\n            {false, true} ->\n                {tx_merge_alarms(Alarms),\n                 ScaleReq#scale_req{lock = unlocked, req = 0},\n                 % init triggers for alarms\n                 lists:map(\n                   fun(Alarm) ->\n                           {Alarm#alarm.name,\n                            next(Alarm#alarm.name, Alarm#alarm.interval)}\n                   end, Alarms)};\n            {true, false} ->\n                {Alarms,\n                 ScaleReq#scale_req{lock = unlocked, req = 0},\n                 % cancel all triggers\n                 lists:map(fun({Name, Trigger}) -> {Name, cancel(Trigger)} end,\n                           Triggers)};\n            {_, _}        ->\n                {Alarms, ScaleReq, Triggers}\n        end,\n\n    ?TRACE(\"leadership: ~p\", [{IsLeader, IsNewLeader}]),\n    {IsNewLeader, NewAlarms, NewScaleReq, NewTriggers};\n\n%% @doc Check alarm Name for new scale request.\non({check_alarm, Name}, {IsLeader, Alarms, ScaleReq, Triggers})\n  when IsLeader andalso ScaleReq#scale_req.lock =:= unlocked  ->\n    {ok, Alarm} = get_alarm(Name, Alarms),\n\n    % log current number of vms\n    log_vms(),\n\n    % call alarm handler\n    NewReq =\n        case Alarm#alarm.state of\n            active   -> alarm_handler(Name, Alarm#alarm.options);\n            inactive -> ScaleReq#scale_req.req\n        end,\n\n    % delay for next check (cooldown if scale req /= 0)\n    NewDelay =\n        case NewReq == 0 of\n            true  -> Alarm#alarm.interval;\n            false -> erlang:max(Alarm#alarm.interval, Alarm#alarm.cooldown)\n        end,\n\n    % push request to cloud module (except cloud_cps)\n    case ScaleReq#scale_req.mode of\n        push -> comm:send_local(self(), {push_scale_req});\n        pull -> ok\n    end,\n\n    ?TRACE(\"check_alarm: ~p, ~p new vms requested, ~p s delayed\",\n           [Name, NewReq, NewDelay]),\n    {IsLeader, Alarms, ScaleReq#scale_req{req = NewReq},\n     next(Name, NewDelay, Triggers)};\n\non({check_alarm, Name}, {IsLeader, Alarms, ScaleReq, Triggers})\n  when IsLeader andalso ScaleReq#scale_req.lock =:= locked  ->\n    {ok, Alarm} = get_alarm(Name, Alarms),\n\n    ?TRACE(\"check_alarm: ~p, scale_req is locked\", [Name]),\n    {IsLeader, Alarms, ScaleReq, next(Name, Alarm#alarm.interval, Triggers)};\n\non({push_scale_req}, {IsLeader, _Alarms, ScaleReq, _Triggers})\n  when IsLeader andalso ScaleReq#scale_req.lock =:= unlocked ->\n\n    Req = ScaleReq#scale_req.req,\n    if\n        Req > 0  -> ?CLOUD:add_vms(Req);\n        Req < 0  -> ?CLOUD:remove_vms(Req*-1);\n        Req == 0 -> ok\n    end,\n\n    ?TRACE(\"push_scale_req: ~p\", [ScaleReq]),\n    {IsLeader, _Alarms, ScaleReq#scale_req{req = 0}, _Triggers};\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Msg loop: API msgs -> scale req polling API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({pull_scale_req, Pid}, State = {_IsLeader, _Alarms, ScaleReq, _Triggers}) ->\n    comm:send(Pid, {scale_req_resp, {ok, ScaleReq#scale_req.req}}),\n    ?TRACE(\"pull_scale_req,~nscale_req: ~p\", [ScaleReq]),\n    State;\n\non({lock_scale_req, Pid}, State = {_IsLeader, _Alarms, ScaleReq, _Triggers})\n  when ScaleReq#scale_req.lock =:= locked ->\n    comm:send(Pid, {scale_req_resp, {error, locked}}),\n    ?TRACE(\"lock_scale_req,~nscale_req: ~p\", [ScaleReq]),\n    State;\non({lock_scale_req, Pid}, {_IsLeader, _Alarms, ScaleReq, Triggers})\n  when ScaleReq#scale_req.lock =:= unlocked ->\n    comm:send(Pid, {scale_req_resp, ok}),\n    % start timeout @todo add timeout to cofig\n    Trigger = comm:send_local_after(get_lock_timeout()*1000, self(),\n                                    {unlock_scale_req_timeout}),\n    ?TRACE(\"lock_scale_req,~nscale_req: ~p\", [ScaleReq]),\n    {_IsLeader, _Alarms, ScaleReq#scale_req{lock = locked},\n                         Triggers ++ [{timeout, Trigger}]};\n\non({unlock_scale_req, Pid}, {_IsLeader, _Alarms, ScaleReq, Triggers})\n  when ScaleReq#scale_req.lock =:= locked ->\n    comm:send(Pid, {scale_req_resp, ok}),\n    % cancel timeout\n    {value, {timeout, Trigger}, NewTriggers} = lists:keytake(\n                                                 timeout, 1, Triggers),\n    cancel(Trigger),\n    ?TRACE(\"unlock_scale_req,~nscale_req: ~p\", [ScaleReq]),\n    {_IsLeader, _Alarms, ScaleReq#scale_req{lock = unlocked, req  = 0},\n                         NewTriggers};\non({unlock_scale_req, Pid}, State = {_IsLeader, _Alarms, ScaleReq, _Triggers})\n  when ScaleReq#scale_req.lock =:= unlocked ->\n    comm:send(Pid, {scale_req_resp, {error, not_locked}}),\n    ?TRACE(\"unlock_scale_req,~nscale_req: ~p\", [ScaleReq]),\n    State;\n\non({unlock_scale_req_timeout}, {_IsLeader, _Alarms, ScaleReq, Triggers}) ->\n    ?TRACE1(\"unlock_scale_req_timeout\"),\n    {_IsLeader, _Alarms, ScaleReq#scale_req{lock = unlocked},\n                         lists:keydelete(timeout, 1, Triggers)};\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Msg loop: API msgs -> alarm state API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({toggle_alarm, Name, Pid}, {_IsLeader, Alarms, _ScaleReq, _Triggers}) ->\n    NewAlarms =\n       case get_alarm(Name, Alarms) of\n            {error, unknown_alarm} ->\n                comm:send(Pid, {toggle_alarm_resp, {error, unknown_alarm}}),\n                Alarms;\n            {ok, Alarm} ->\n                Toggled = case Alarm#alarm.state of\n                              active   -> Alarm#alarm{state = inactive};\n                              inactive -> Alarm#alarm{state = active}\n                          end,\n                case tx_update_alarms([Toggled], [Alarm]) of\n                    ok   ->\n                        comm:send(Pid, {toggle_alarm_resp,\n                                   {ok, {new_state, Toggled#alarm.state}}}),\n                        update_alarm(Toggled, Alarms);\n                    fail ->\n                        comm:send(Pid, {toggle_alarm_resp, {error, tx_fail}}),\n                        Alarms\n                end\n        end,\n    ?TRACE(\"alarms: ~p, ~nnewalarms: ~p\", [Alarms, NewAlarms]),\n    {_IsLeader, NewAlarms, _ScaleReq, _Triggers};\n\non({activate_alarms, Pid}, {_IsLeader, Alarms, _ScaleReq, _Triggers}) ->\n    AllActive = lists:map(fun(Alarm) -> Alarm#alarm{state = active} end,\n                          Alarms),\n    NewAlarms =\n        case tx_update_alarms(AllActive, Alarms) of\n            ok ->\n                comm:send(Pid, {activate_alarms_resp, ok}),\n                AllActive;\n            fail ->\n                comm:send(Pid, {activate_alarms_resp, {error, tx_fail}}),\n                Alarms\n        end,\n    {_IsLeader, NewAlarms, _ScaleReq, _Triggers};\n\non({deactivate_alarms, Pid}, {_IsLeader, Alarms, _ScaleReq, _Triggers}) ->\n    AllInactive = lists:map(fun(Alarm) -> Alarm#alarm{state = inactive} end,\n                            Alarms),\n    NewAlarms =\n        case tx_update_alarms(AllInactive, Alarms) of\n            ok ->\n                comm:send(Pid, {deactivate_alarms_resp, ok}),\n                AllInactive;\n            fail ->\n                comm:send(Pid, {deactivate_alarms_resp, {error, tx_fail}}),\n                Alarms\n        end,\n    {_IsLeader, NewAlarms, _ScaleReq, _Triggers};\n\non({update_alarm, Name, NewOptions, Pid},\n   {_IsLeader, Alarms, _ScaleReq, _Triggers}) ->\n    NewAlarms =\n        case get_alarm(Name, Alarms) of\n            {error, unknown_alarm} ->\n                comm:send(Pid, {update_alarm_resp, {error, unknown_alarm}}),\n                Alarms;\n            {ok, Alarm} ->\n                UpdatedOptions = lists:foldl(\n                                   fun(Option = {Key, _Value}, Acc) ->\n                                           lists:keystore(Key, 1, Acc, Option)\n                                   end, Alarm#alarm.options, NewOptions),\n                UpdatedAlarm = Alarm#alarm{options=UpdatedOptions},\n                case tx_update_alarms(UpdatedAlarm, Alarm) of\n                    ok   ->\n                        comm:send(Pid, {update_alarm_resp,\n                                   {ok, {new_options, UpdatedOptions}}}),\n                        update_alarm(UpdatedAlarm, Alarms);\n                    fail ->\n                        comm:send(Pid, {update_alarm_resp, {error, tx_fail}}),\n                        Alarms\n                end\n        end,\n    {_IsLeader, NewAlarms, _ScaleReq, _Triggers};\n\non(_Msg, State) ->\n    State.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Triggers\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% @doc Check alarm Name in Delay seconds.\n-spec next(Name :: atom(), Delay :: pos_integer()) -> reference().\nnext(Name, Delay) ->\n    comm:send_local_after(Delay*1000, self(), {check_alarm, Name}).\n\n%% @doc Check alarm Name from trigger list in Delay seconds.\n-spec next(Name :: atom(), Delay :: pos_integer(), Triggers :: triggers()) ->\n          triggers().\nnext(Name, Delay, Triggers) ->\n    lists:keyreplace(Name, 1, Triggers, {Name, next(Name, Delay)}).\n\n%% @doc Cancel timer of trigger.\n-spec cancel(Trigger :: reference() | ok) -> ok.\ncancel(Trigger) ->\n    _ = erlang:cancel_timer(Trigger),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Alarm helpers\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% @doc Get alarm by name.\n-spec get_alarm(Name :: atom(), Alarms :: alarms()) -> {ok, alarm()} |\n                                                       {error, unknown_alarm}.\nget_alarm(Name, Alarms) ->\n    case Alarm = lists:keyfind(Name, 2, Alarms) of\n        false  -> {error, unknown_alarm};\n        _Found -> {ok, Alarm}\n    end.\n\n%% @doc Get value for specified option key from key-value list. Return provided\n%%      default value, if key does not exist.\n-spec get_alarm_option(Options :: key_value_list(), Key :: atom(),\n                       Default :: any()) -> any().\nget_alarm_option(Options, FieldKey, Default) ->\n    case lists:keyfind(FieldKey, 1, Options) of\n        {FieldKey, Value} -> Value;\n        false             -> Default\n    end.\n\n-spec update_alarm(UpdatedAlarm :: #alarm{}, Alarms :: alarms()) -> alarms().\nupdate_alarm(UpdatedAlarm, Alarms) ->\n    lists:keystore(UpdatedAlarm#alarm.name, 2, Alarms, UpdatedAlarm).\n\n%% @doc Log key-value-pair at autoscale_server.\n-spec log(Key :: atom(), Value :: term()) -> ok.\nlog(Key, Value) ->\n    case autoscale_server:check_config() andalso ?CLOUD =/= cloud_cps of\n        true  ->\n            autoscale_server:log(Key, Value);\n        false ->\n            ok\n    end.\n\n%% @doc Log number of VMs at autoscale_server.\n-spec log_vms() -> ok.\nlog_vms() ->\n    case autoscale_server:check_config() andalso ?CLOUD =/= cloud_cps of\n        true  ->\n            autoscale_server:log(vms, ?CLOUD:get_number_of_vms());\n        false ->\n            ok\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Misc\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% @doc Request my_range from dht_node (called by rm_loop at direct neighborhood\n%%      changes).\n-spec send_my_range_req(Pid :: pid(), Tag :: ?MODULE,\n                        _OldN :: nodelist:neighborhood(),\n                        _NewN :: nodelist:neighborhood()) -> ok.\nsend_my_range_req(Pid, ?MODULE, _OldN, _NewN) ->\n    DhtNodeOfPid = pid_groups:pid_of(pid_groups:group_of(Pid), dht_node),\n    comm:send_local(DhtNodeOfPid, {get_state, comm:make_global(Pid), my_range}).\n\n%% @doc Write alarm updates to dht. Alarms in ToAdd are added to the ring and\n%%      alarms in ToRem are removed. Usually, ToAdd should contain the new\n%%      version of the alarm and ToRem the old version (which could possibly\n%%      have been written to the dht before).\n%%\n%%      When a node becomes leader, she merges her local alarms with alarms from\n%%      the ring (alarms on ring overwrite local alarms). See tx_merge_alarms/2.\n-spec tx_update_alarms(ToAdd::alarms(), ToRem::alarms()) -> ok | fail;\n                      (ToAdd::alarm(), ToRem::alarm()) -> ok | fail.\ntx_update_alarms(ToAdd, ToRem)\n  when erlang:is_tuple(ToAdd) andalso erlang:is_tuple(ToRem) ->\n    tx_update_alarms([ToAdd], [ToRem]);\ntx_update_alarms(ToAdd, ToRem) ->\n    {TLog, _Res} = api_tx:add_del_on_list(\n                     api_tx:new_tlog(), ?AUTOSCALE_TX_KEY, ToAdd, ToRem),\n    case api_tx:commit(TLog) of\n        {ok} ->\n            ok;\n        {fail, abort, _ClientKeys} ->\n            fail\n    end.\n\n%% @doc Merge local alarms with updated alarms from dht.\n-spec tx_merge_alarms(Alarms :: alarms()) -> alarms().\ntx_merge_alarms(Alarms) ->\n    case api_tx:read(?AUTOSCALE_TX_KEY) of\n        {ok, Updates} ->\n            lists:foldl(\n              fun(Alarm, Acc) ->\n                      lists:keyreplace(Alarm#alarm.name, 2, Acc, Alarm)\n              end, Alarms, Updates);\n        {fail, not_found} ->\n            Alarms\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Startup\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec start_link(DHTNodeGroup :: pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                             [{pid_groups_join_as, DHTNodeGroup, autoscale}]).\n\n-spec init([]) -> state().\ninit([]) ->\n    % initial my_range request to dht_node\n    comm:send_local(pid_groups:get_my(dht_node),\n                    {get_state, comm:this(), my_range}),\n\n    % check my_range at direct neighborhood changes\n    rm_loop:subscribe(\n      self(), ?MODULE, fun rm_loop:subscribe_dneighbor_change_filter/3,\n      fun ?MODULE:rm_exec/5, inf),\n\n    % intial state\n    {_IsLeader = false,\n     _Alarms   = get_alarms(),\n     _ScaleReq =\n         case ?CLOUD =:= cloud_cps of\n             true  -> #scale_req{mode = pull};\n             false ->\n                 ?CLOUD:init(),\n                 #scale_req{mode = push}\n         end,\n     _Triggers = []}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Config\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec get_alarms() -> [#alarm{}].\nget_alarms() ->\n    case check_config() of\n        true  ->\n            % @todo add convenience for cfg\n            config:read(autoscale_alarms);\n        false ->\n            []\n    end.\n\n%% @doc Get timeout for pull lock (in seconds) from config.\n-spec get_lock_timeout() -> pos_integer().\nget_lock_timeout() ->\n    case config:read(autoscale_lock_timeout) of\n        Timeout when erlang:is_integer(Timeout) andalso Timeout > 0 ->\n            Timeout;\n        _ -> ?DEFAULT_LOCK_TIMEOUT\n    end.\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_list(autoscale_alarms) andalso\n    config:cfg_exists(autoscale_cloud_module) andalso\n    config:cfg_is_list(autoscale_alarms,\n                       fun(X) -> erlang:is_record(X, alarm) end,\n                       \"alarm record tuple\").\n\n-spec rm_exec(pid(), Tag::?MODULE,\n              OldNeighbors::nodelist:neighborhood(),\n              NewNeighbors::nodelist:neighborhood(),\n              Reason::rm_loop:reason()) -> ok.\nrm_exec(Pid, ?MODULE, _OldN, _NewN, _Reason) ->\n    DhtNodeOfPid = pid_groups:pid_of(pid_groups:group_of(Pid),\n                                     dht_node),\n    comm:send_local(DhtNodeOfPid,\n                    {get_state, comm:make_global(Pid), my_range}).\n"
  },
  {
    "path": "src/autoscale_server.erl",
    "content": "% @copyright 2013-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Ufuk Celebi <celebi@zib.de>\n%% @doc Server for collecting autoscale plot data from autoscale leader(s).\n%%\n%%      {autoscale_server, true} in the config will enable this service. The\n%%      autoscale processes from which data is collected assume that this\n%%      server runs in the same VM as mgmt_server (needs to be set in config).\n%%\n%%      {autoscale_server_plot_path, PATH} needs to be set in order to write\n%%      the collected data to the file system (see on({write_to_file})).\n%% @end\n%% @version $Id$\n-module(autoscale_server).\n-author('celebi@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n-export([start_link/1, init/1, on/2]).\n-export([check_config/0, log/1, log/2]).\n\n-include(\"gen_component.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% types\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-type(state() ::\n    {PlotData::dict:dict(PlotKey::atom(), {Timestamp::pos_integer(), Value::number()})} |\n    unknown_event).\n\n-type(message() ::\n    {collect, PlotKey::atom(), Timestamp::pos_integer(), Value::number()} |\n    {reset} |\n    {write_to_file}).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% startup\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec start_link(ServiceGroup::pid_groups:groupname()) -> {ok, pid()}.\nstart_link(ServiceGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, null,\n                             [{pid_groups_join_as, ServiceGroup, autoscale_server}]).\n\n-spec init(null) -> state().\ninit(_Options) ->\n    {_PlotData = dict:new()}.\n\n-spec on(message(), state()) -> state().\non({collect, PlotKey, Timestamp, Value}, {PlotData}) ->\n    {_NewPlotData = dict:append(PlotKey, {Timestamp, Value}, PlotData)};\non({reset}, {_PlotData}) ->\n    {_NewPlotData = dict:new()};\non({write_to_file}, {PlotData}) ->\n    case PlotPath = config:read(autoscale_server_plot_path) of\n        failed -> false;\n        _      ->\n            lists:foreach(\n                fun({PlotKey, Data}) ->\n                    Filename = filename:join(PlotPath, PlotKey) ++ \".dat\",\n                    {ok, File} = file:open(Filename, [write]),\n                    lists:foreach(\n                        fun({Timestamp, Value}) ->\n                            io:format(File, \"~p ~p~n\", [Timestamp, Value])\n                        end, Data),\n                    file:close(File)\n                end, dict:to_list(PlotData))\n    end,\n    {PlotData};\non(_, _) ->\n    unknown_event.\n\n-spec log(Key :: atom(), Value :: term()) -> ok.\nlog(Key, Value) ->\n    log([{Key, Value}]).\n\n-spec log(KeyValueList :: [{Key :: atom(), Value :: term()}]) -> ok.\nlog(KeyValueList) when erlang:is_list(KeyValueList)  ->\n    {Ms, S, _Us} = os:timestamp(),\n    NowMs = Ms*1000000+S,\n    MgmtServer = config:read(mgmt_server),\n\n    lists:foreach(\n      fun({Key, Value}) ->\n              comm:send(MgmtServer, {collect, Key, NowMs, Value},\n                        [{group_member, autoscale_server}])\n      end, KeyValueList),\n    ok.\n\n%% @doc Checks whether config parameters for autoscale_server exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_exists(autoscale_server_plot_path) andalso\n    config:read(mgmt_server) =/= failed.\n"
  },
  {
    "path": "src/bench.erl",
    "content": "% @copyright 2007-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc This is a small server for running benchmarks\n%% @end\n-module(bench).\n-author('schuett@zib.de').\n\n%% public interface\n-export([increment/2, increment_o/3,\n         increment_with_histo/2, increment_with_histo_o/3,\n         increment/3, increment_o/4,\n         quorum_read/2, quorum_read_o/3,\n         read_read/2, read_read_o/3,\n         load_start/1, load_stop/0,\n         print_results/2]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-type options() :: [locally | print | verbose | profile\n                    | {copies, non_neg_integer()}].\n\n%% @doc run an increment benchmark (i++) on all nodes\n-spec increment(ThreadsPerVM::pos_integer(), Iterations::pos_integer()) -> {ok, [{atom(), term()},...]}.\nincrement(ThreadsPerVM, Iterations) ->\n    increment_o(ThreadsPerVM, Iterations, [print, verbose]).\n\n%% @doc run an increment benchmark (i++) on all nodes with custom options\n-spec increment_o(ThreadsPerVM::pos_integer(), Iterations::pos_integer(), Options::options()) -> {ok, [{atom(), term()},...]}.\nincrement_o(ThreadsPerVM, Iterations, Options) ->\n    Msg = {bench, increment, ThreadsPerVM, Iterations, comm:this(), undefined},\n    {ok, Res} = manage_run(ThreadsPerVM, Iterations, Options, Msg),\n    print_results(Res, Options),\n    {ok, Res}.\n\n%% @doc run an increment benchmark (i++) on all nodes\n-spec increment_with_histo(ThreadsPerVM::pos_integer(), Iterations::pos_integer()) -> {ok, [{atom(), term()},...]}.\nincrement_with_histo(ThreadsPerVM, Iterations) ->\n    {ok, Res} = increment_with_histo_o(ThreadsPerVM, Iterations, [print, verbose]),\n    print_results(Res, [print, verbose]),\n    {ok, Res}.\n\n%% @doc run an increment benchmark (i++) on all nodes with custom options\n-spec increment_with_histo_o(ThreadsPerVM::pos_integer(), Iterations::pos_integer(), Options::options()) -> {ok, [{atom(), term()},...]}.\nincrement_with_histo_o(ThreadsPerVM, Iterations, Options) ->\n    Msg = {bench, increment_with_histo, ThreadsPerVM, Iterations, comm:this(), undefined},\n    {ok, Res} = manage_run(ThreadsPerVM, Iterations, Options, Msg),\n    print_results(Res, Options),\n    {ok, Res}.\n\n%% @doc run an increment benchmark on all nodes (with a user-specified key)\n-spec increment(ThreadsPerVM::pos_integer(), Iterations::pos_integer(),\n                Key::client_key()) -> {ok, [{atom(), term()},...]} | {failed, init_per_key}.\nincrement(ThreadsPerVM, Iterations, Key) ->\n    increment_o(ThreadsPerVM, Iterations, Key, [print, verbose]).\n\n%% @doc run an increment benchmark on all nodes (with a user-specified key) with custom options\n-spec increment_o(ThreadsPerVM::pos_integer(), Iterations::pos_integer(),\n                Key::client_key(), Options::options()) -> {ok, [{atom(), term()},...]} | {failed, init_per_key}.\nincrement_o(ThreadsPerVM, Iterations, Key, Options) ->\n    case init_key(Key, 100) of\n        failed ->\n            {failed, init_per_key};\n        _ ->\n            Msg = {bench, increment_with_key, ThreadsPerVM, Iterations, comm:this(), Key},\n            {ok, Res} = manage_run(ThreadsPerVM, Iterations, Options, Msg),\n            print_results(Res, Options),\n            {ok, Res}\n    end.\n\n%% @doc run an read benchmark on all nodes\n-spec quorum_read(ThreadsPerVM::pos_integer(), Iterations::pos_integer()) -> {ok, [{atom(), term()},...]}.\nquorum_read(ThreadsPerVM, Iterations) ->\n    quorum_read_o(ThreadsPerVM, Iterations, [print, verbose]).\n\n%% @doc run an read benchmark on all nodes with custom options\n-spec quorum_read_o(ThreadsPerVM::pos_integer(), Iterations::pos_integer(), Options::options()) -> {ok, [{atom(), term()},...]}.\nquorum_read_o(ThreadsPerVM, Iterations, Options) ->\n    Msg = {bench, quorum_read, ThreadsPerVM, Iterations, comm:this(), undefined},\n    {ok, Res} = manage_run(ThreadsPerVM, Iterations, Options, Msg),\n    print_results(Res, Options),\n    {ok, Res}.\n\n%% @doc run an read benchmark on all nodes\n-spec read_read(ThreadsPerVM::pos_integer(), Iterations::pos_integer()) -> {ok, [{atom(), term()},...]}.\nread_read(ThreadsPerVM, Iterations) ->\n    read_read_o(ThreadsPerVM, Iterations, [print, verbose]).\n\n%% @doc run an read benchmark on all nodes with custom options\n-spec read_read_o(ThreadsPerVM::pos_integer(), Iterations::pos_integer(), Options::options()) -> {ok, [{atom(), term()},...]}.\nread_read_o(ThreadsPerVM, Iterations, Options) ->\n    Msg = {bench, read_read, ThreadsPerVM, Iterations, comm:this(), undefined},\n    {ok, Res} = manage_run(ThreadsPerVM, Iterations, Options, Msg),\n    print_results(Res, Options),\n    {ok, Res}.\n\n-spec load_start(Gap::pos_integer()) -> ok.\nload_start(Gap) ->\n    ServerList = util:get_proc_in_vms(bench_server),\n    Msg = {load_start, Gap},\n    _ = [comm:send(Server, Msg) || Server <- ServerList],\n    ok.\n\n-spec load_stop() -> ok.\nload_stop() ->\n    ServerList = util:get_proc_in_vms(bench_server),\n    Msg = {load_stop},\n    _ = [comm:send(Server, Msg) || Server <- ServerList],\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% common functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% @doc spread run over the available VMs and collect results\n%% (executed in clients context)\n-spec manage_run(ThreadsPerVM::pos_integer(), Iterations::pos_integer(),\n                 Options::options(),\n                 Message::comm:message()) -> {ok, [{atom(), term()},...]}.\nmanage_run(ThreadsPerVM, Iterations, Options, Message) ->\n    Pid = self(),\n    %% function is spawned to catch superfluous reply messages after\n    %% actual workload is done.\n    Child = spawn_link(fun() ->\n                  %% to get the newly spawned process in the\n                  %% proto_sched schedule we start it working by\n                  %% sending it a message (which also performs the\n                  %% infection).\n                  trace_mpath:thread_yield(), %noop, as not infected\n                  receive\n                      ?SCALARIS_RECV({start_work},%% ->\n                                     ok)\n                      end,\n                   Msg = setelement(5, Message, comm:this()),\n                   Res = manage_run_internal(ThreadsPerVM, Iterations, Options,\n                                                    Msg),\n                   comm:send_local(Pid, Res)\n                  end),\n    comm:send_local(Child, {start_work}),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({ok, Result}, %% ->\n            {ok, Result})\n    end.\n\n-spec manage_run_internal(ThreadsPerVM::pos_integer(), Iterations::pos_integer(),\n                 Options::options(), Message::comm:message())\n                         -> {ok, [{atom(), term()},...]}.\nmanage_run_internal(ThreadsPerVM, Iterations, Options, Message) ->\n    Print = lists:member(print, Options),\n    ServerList = util:get_proc_in_vms(bench_server),\n    %% io:format(\"~p~n\", [ServerList]),\n    Before = os:timestamp(),\n    _ = [comm:send(Server, Message) || Server <- ServerList],\n    fd:subscribe(self(), ServerList),\n    ?IIF(Print, io:format(\"Collecting results... ~n\"), ok),\n    Statistics = collect(length(ServerList), [], Print),\n    fd:unsubscribe(self(), ServerList),\n    ?IIF(Print, io:format(\"stats: ~p ~n\", [Statistics]), ok),\n    {MinTP, MeanTP, MaxTP} = lists:foldl(\n                               fun (Stats, {Min, Mean, Max}) ->\n                                       {Min + Iterations / element(4, Stats) * 1000000.0 * ThreadsPerVM,\n                                        Mean + Iterations / element(3, Stats) * 1000000.0 * ThreadsPerVM,\n                                        Max + Iterations / element(2, Stats) * 1000000.0 * ThreadsPerVM}\n                     end, {0.0, 0.0, 0.0}, Statistics),\n    After = os:timestamp(),\n    RunTime = timer:now_diff(After, Before),\n    NrServers = length(ServerList),\n    WallClockTime = RunTime / 1000000.0,\n    WallClockTP = NrServers * ThreadsPerVM * Iterations / RunTime * 1000000.0,\n    WallClockLat = RunTime / 1000.0 / Iterations,\n    MinTPAll     = [Iterations / element(4, Node) * 1000000.0 || Node <- Statistics],\n    MeanTPAll    = [Iterations / element(3, Node) * 1000000.0 || Node <- Statistics],\n    MaxTPAll     = [Iterations / element(2, Node) * 1000000.0 || Node <- Statistics],\n    MinLatAll    = [element(2, Node) / Iterations / 1000.0 || Node <- Statistics],\n    AvgLatAll    = [element(3, Node) / Iterations / 1000.0 || Node <- Statistics],\n    MaxLatAll    = [element(4, Node) / Iterations / 1000.0 || Node <- Statistics],\n    AvgLat       = lists:foldl(fun (Lat, Acc) ->\n                                       Lat + Acc\n                               end, 0.0, AvgLatAll) / NrServers,\n    AvgExTimeAll = [element(3, Node) / 1000.0 || Node <- Statistics],\n    Aborts = [element(6, Node) || Node <- Statistics],\n    Result = [{servers, ServerList},\n              {threads_per_vm, ThreadsPerVM},\n              {iterations, Iterations},\n              {statistics, Statistics},\n              {wall_clock_time, WallClockTime},\n              {wall_clock_throughput, WallClockTP},\n              {wall_clock_latency, WallClockLat},\n              {min_throughput_overall, MinTP},\n              {min_throughput_each, MinTPAll},\n              {mean_throughput_overall, MeanTP},\n              {mean_throughput_each, MeanTPAll},\n              {max_throughput_overall, MaxTP},\n              {max_throughput_each, MaxTPAll},\n              {min_latency_each, MinLatAll},\n              {avg_latency_each, AvgLatAll},\n              {max_latency_each, MaxLatAll},\n              {avg_latency_overall, AvgLat},\n              {avg_exec_time, AvgExTimeAll},\n              {aborts, Aborts}\n             ],\n    {ok, Result}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% helper functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec init_key(Key::client_key(), non_neg_integer()) -> failed | client_key().\ninit_key(_Key, 0) ->\n    io:format(\"init_key failed~n\", []),\n    failed;\ninit_key(Key, Count) ->\n    case api_tx:write(Key, 0) of\n        {ok}                 -> Key;\n        {fail, abort, [Key]} -> init_key(Key, Count - 1)\n    end.\n\ncollect(0, L, _Print)     -> L;\ncollect(Length, L, Print) ->\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({done, X, WallClockTime, MeanTime, Variance, MinTime, MaxTime, Aborts}, %% ->\n         begin\n             Done = erlang:length(L),\n             ?IIF(Print, io:format(\"BS [~p/~p]: ~p @ ~p~n\",[Done+1, Done+Length, WallClockTime, X]), ok),\n             collect(Length - 1, [{WallClockTime, MinTime, MeanTime, MaxTime, Variance, Aborts} | L], Print)\n          end);\n        ?SCALARIS_RECV({fd_notify, crash, Pid, _Reason}, %% ->\n           begin\n               ?IIF(Print, io:format(\"ignoring ~p, because it crashed~n\", [Pid]), ok),\n               collect(Length - 1, L, Print)\n           end);\n        ?SCALARIS_RECV({fd_notify, _Event, _Pid, _Reason}, %% ->\n           begin\n               collect(Length, L, Print)\n           end)\n    end.\n\n\n-spec print_results(Results::[{atom(), term()},...], Options::options()) -> ok.\nprint_results(Result, Options) ->\n    Verbose = lists:member(verbose, Options),\n    Print = lists:member(print, Options),\n    [{servers, ServerList},\n     {threads_per_vm, ThreadsPerVM},\n     {iterations, Iterations},\n     {statistics, _Statistics},\n     {wall_clock_time, WallClockTime},\n     {wall_clock_throughput, WallClockTP},\n     {wall_clock_latency, WallClockLat},\n     {min_throughput_overall, MinTP},\n     {min_throughput_each, MinTPAll},\n     {mean_throughput_overall, MeanTP},\n     {mean_throughput_each, MeanTPAll},\n     {max_throughput_overall, MaxTP},\n     {max_throughput_each, MaxTPAll},\n     {min_latency_each, MinLatAll},\n     {avg_latency_each, AvgLatAll},\n     {max_latency_each, MaxLatAll},\n     {avg_latency_overall, _AvgLat},\n     {avg_exec_time, AvgExTimeAll},\n     {aborts, Aborts}\n    ]  = Result,\n    NrServers = length(ServerList),\n    case Verbose andalso Print of\n        true ->\n            io:format(\"servers: ~p threads/vm: ~p iterations: ~p~n\",\n                      [NrServers, ThreadsPerVM, Iterations]),\n            io:format(\"wall clock time        : ~p~n\", [WallClockTime]),\n            io:format(\"wall clock throughput  : ~p~n\", [WallClockTP]),\n            io:format(\"wall clock avg. latency: ~p ms~n\", [WallClockLat]),\n            io:format(\"min. throughput: ~p ~p (1/s)~n\", [MinTP, MinTPAll]),\n            io:format(\"avg. throughput: ~p ~p (1/s)~n\", [MeanTP, MeanTPAll]),\n            io:format(\"max. throughput: ~p ~p (1/s)~n\", [MaxTP, MaxTPAll]),\n            io:format(\"min. latency   : ~p (ms)~n\", [MinLatAll]),\n            io:format(\"avg. latency   : ~p (ms)~n\", [AvgLatAll]),\n            io:format(\"max. latency   : ~p (ms)~n\", [MaxLatAll]),\n            io:format(\"avg. ex. time  : ~p (ms)~n\", [AvgExTimeAll]),\n            %io:format(\"std. dev.(%)   : ~p~n\",\n            %          [[math:sqrt(element(5, Node)) / element(3, Node) * 100 || Node <- Statistics]]),\n            io:format(\"aborts         : ~p~n\", [Aborts]);\n            %io:format(\"Statistics: ~p~n\", [ Statistics ]);\n        false -> ok\n    end.\n"
  },
  {
    "path": "src/bench_fun.erl",
    "content": "%  @copyright 2008-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    i++ benchmark\n%% @end\n%% @version $Id$\n-module(bench_fun).\n\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([increment/1, increment_with_histo/1, increment_with_key/2, quorum_read/1, read_read/1]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% public API\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec increment(integer()) -> fun().\nincrement(Iterations) ->\n    fun(Parent) ->\n            Key = get_and_init_key(),\n            {Diff, Aborts} = util:tc(fun increment_iter/3, [Key, Iterations, 0]),\n            comm:send_local(Parent, {done, Diff, Aborts})\n    end.\n\n-spec increment_with_histo(integer()) -> fun().\nincrement_with_histo(Iterations) ->\n    fun(Parent) ->\n            Key = get_and_init_key(),\n            {Diff, {Aborts, H2}} =\n                util:tc(fun() ->\n                                H = histogram:create(20),\n                                increment_with_histo_iter(H, Key, Iterations, 0)\n                        end),\n            io:format(\"~p~n\", [histogram:get_data(H2)]),\n            comm:send_local(Parent, {done, Diff, Aborts})\n    end.\n\n-spec increment_with_key(integer(), client_key()) -> fun().\nincrement_with_key(Iterations, Key) ->\n    fun(Parent) ->\n            {Diff, Aborts} = util:tc(fun increment_iter/3, [Key, Iterations, 0]),\n            comm:send_local(Parent, {done, Diff, Aborts})\n    end.\n\n-spec quorum_read(integer()) -> fun().\nquorum_read(Iterations) ->\n    fun (Parent) ->\n            Key = get_and_init_key(),\n            {Diff, Aborts} = util:tc(fun read_iter/3, [Key, Iterations, 0]),\n            comm:send_local(Parent, {done, Diff, Aborts})\n    end.\n\n-spec read_read(integer()) -> fun().\nread_read(Iterations) ->\n    fun (Parent) ->\n            Key1 = get_and_init_key(),\n            Key2 = get_and_init_key(),\n            {Diff, Aborts} = util:tc(fun read_read_iter/4, [Key1, Key2, Iterations, 0]),\n            comm:send_local(Parent, {done, Diff, Aborts})\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% increment\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec inc(Key::string()) -> api_tx:commit_result() | api_tx:read_result().\ninc(Key) ->\n    {TLog1, [ReadResult]} =\n        api_tx:req_list([{read, Key}]),\n    case ReadResult of\n        {ok, Value} ->\n            {_TLog, [{ok}, CommitResult]} =\n                api_tx:req_list(TLog1, [{write, Key, Value + 1}, {commit}]),\n            CommitResult;\n        Fail -> Fail\n    end.\n\n-spec increment_iter(string(), integer(), non_neg_integer()) -> non_neg_integer().\nincrement_iter(_Key, 0, Aborts) ->\n    Aborts;\nincrement_iter(Key, Iterations, Aborts) ->\n    Result = inc(Key),\n    case Result of\n        {ok}              -> increment_iter(Key, Iterations - 1, Aborts);\n        {fail, abort, [Key]}     ->\n            timer:sleep(randoms:rand_uniform(1, erlang:max(2, 10 * Aborts + 1))),\n            increment_iter(Key, Iterations, Aborts + 1);\n        {fail, not_found} ->\n            timer:sleep(randoms:rand_uniform(1, erlang:max(2, 10 * Aborts + 1))),\n            increment_iter(Key, Iterations, Aborts + 1);\n        X ->\n            log:log(warn, \"bench_fun:increment_iter unexpected return: ~p\", [X]),\n            increment_iter(Key, Iterations, Aborts + 1)\n    end.\n\n-spec increment_with_histo_iter(histogram:histogram(), string(), integer(), non_neg_integer()) -> {non_neg_integer(), histogram:histogram()}.\nincrement_with_histo_iter(H, _Key, 0, Aborts) ->\n    {Aborts, H};\nincrement_with_histo_iter(H, Key, Iterations, Aborts) ->\n    Before = os:timestamp(),\n    Result = inc(Key),\n    After = os:timestamp(),\n    case Result of\n        {ok}              -> increment_with_histo_iter(\n                               histogram:add(timer:now_diff(After, Before) / 1000, H),\n                               Key, Iterations - 1, Aborts);\n        {fail, abort, [Key]}     ->\n            timer:sleep(randoms:rand_uniform(1, erlang:max(2, 10 * Aborts + 1))),\n            increment_with_histo_iter(H, Key, Iterations, Aborts + 1);\n        {fail, not_found} ->\n            timer:sleep(randoms:rand_uniform(1, erlang:max(2, 10 * Aborts + 1))),\n            increment_with_histo_iter(H, Key, Iterations, Aborts + 1);\n        X ->\n            log:log(warn, \"bench_fun:increment_with_histo_iter unexpected return: ~p\", [X]),\n            increment_with_histo_iter(H, Key, Iterations, Aborts + 1)\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% quorum_read\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nread_iter(_Key, 0, Aborts) ->\n    Aborts;\nread_iter(Key, Iterations, Aborts) ->\n    case api_tx:read(Key) of\n        {ok, _Value}    -> read_iter(Key, Iterations - 1, Aborts);\n        {fail, _Reason} -> read_iter(Key, Iterations, Aborts + 1)\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% read_read (reads two items in one transaction)\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nread_read_tx(Key1, Key2) ->\n    {_, Result} = api_tx:req_list([{read, Key1}, {read, Key2}, {commit}]),\n    Result.\n\nread_read_iter(_Key1, _Key2, 0, Aborts) ->\n    Aborts;\nread_read_iter(Key1, Key2, Iterations, Aborts) ->\n    case read_read_tx(Key1, Key2) of\n        [{ok, 0},{ok,0},{ok}] -> read_read_iter(Key1, Key2, Iterations - 1, Aborts);\n        _ -> read_read_iter(Key1, Key2, Iterations, Aborts + 1)\n    end.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% initialize keys\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec retries() -> pos_integer().\nretries() -> 10.\n\n-spec get_and_init_key() -> string().\nget_and_init_key() ->\n    Key = randoms:getRandomString(),\n    get_and_init_key(Key, retries(), _TriedKeys = 1).\n\n-spec get_and_init_key(Key::string(), Retries::non_neg_integer(),\n                       TriedKeys::pos_integer()) -> FinalKey::string().\nget_and_init_key(_Key, 0, TriedKeys) ->\n    NewKey = randoms:getRandomString(),\n    io:format(\"geT_and_init_key choosing new key and retrying~n\"),\n    timer:sleep(randoms:rand_uniform(1, 10000 * TriedKeys)),\n    get_and_init_key(NewKey, retries(), TriedKeys + 1);\n\nget_and_init_key(Key, Count, TriedKeys) ->\n    case api_tx:write(Key, 0) of\n        {ok} ->\n            Key;\n        {fail, abort, [Key]} ->\n            SleepTime = randoms:rand_uniform(1, TriedKeys * 2000 * 11 - Count),\n            io:format(\"geT_and_init_key 1 failed, retrying in ~p ms~n\",\n                      [SleepTime]),\n            timer:sleep(SleepTime),\n            get_and_init_key(Key, Count - 1, TriedKeys)\n    end.\n"
  },
  {
    "path": "src/bench_load.erl",
    "content": "% @copyright 2007-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Load generator\n%% @end\n%% @version $Id$\n-module(bench_load).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([start/1]).\n\n-include(\"scalaris.hrl\").\n\n-spec start(Gap::pos_integer()) -> ok.\nstart(Gap) ->\n    loop(Gap, []).\n\n-spec loop(Gap::pos_integer(), [pid()]) -> ok.\nloop(Gap, Pids) ->\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n            {load_stop}, %%->\n            begin\n                _ = [erlang:exit(Pid, kill) || Pid <- Pids],\n                ok\n            end\n          )\n    after Gap ->\n            Pid = spawn_new(),\n            loop(Gap, [Pid | Pids])\n    end.\n\nspawn_new() ->\n    io:format(\"spawn~n\", []),\n    spawn(fun() ->\n                 worker()\n          end).\n\nworker() ->\n    Key1 = randoms:getRandomString(),\n    Key2 = randoms:getRandomString(),\n    read_read_iter(Key1, Key2, 1000, 0),\n    worker().\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% read_read (reads two items in one transaction)\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nread_read_tx(Key1, Key2) ->\n    {_, Result} = api_tx:req_list([{read, Key1}, {read, Key2}, {commit}]),\n    Result.\n\nread_read_iter(_Key1, _Key2, 0, Aborts) ->\n    Aborts;\nread_read_iter(Key1, Key2, Iterations, Aborts) ->\n    case read_read_tx(Key1, Key2) of\n        [{ok, 0},{ok,0},{ok}] -> read_read_iter(Key1, Key2, Iterations - 1, Aborts);\n        _ -> read_read_iter(Key1, Key2, Iterations, Aborts + 1)\n    end.\n\n"
  },
  {
    "path": "src/bench_server.erl",
    "content": "% @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc This is a small server for running benchmarks\n%% @end\n%% @version $Id$\n-module(bench_server).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-export([start_link/1, init/1, on/2]).\n\n-include(\"scalaris.hrl\").\n\n-export([run_threads/2]).\n\n-include(\"gen_component.hrl\").\n\n-record(state,\n        {load_pid             :: pid() | ok,\n         bench_owner   = ok   :: comm:mypid() | ok,\n         bench_start   = ok   :: erlang_timestamp() | ok,\n         bench_threads = 0    :: ThreadsLeft::non_neg_integer(),\n         bench_data    = null :: {N::non_neg_integer(), Mean::float(), M2::float(),\n                                  Min::non_neg_integer(), Max::non_neg_integer(),\n                                  AggAborts::non_neg_integer()} | null\n         }).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% the server code\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec on(comm:message(), #state{}) -> #state{}.\non({load_start, Gap}, State) ->\n    Pid = spawn(fun() ->\n                        bench_load:start(Gap)\n                end),\n    State#state{load_pid=Pid};\non({load_stop}, #state{load_pid=Pid} = State) ->\n    Pid ! {load_stop},\n    State#state{load_pid=ok};\non({bench, Op, Threads, Iterations, Owner, Param}, State) ->\n    Bench = case Op of\n                increment_with_histo ->\n                    bench_fun:increment_with_histo(Iterations);\n                increment ->\n                    bench_fun:increment(Iterations);\n                increment_with_key ->\n                    bench_fun:increment_with_key(Iterations, Param);\n                quorum_read ->\n                    bench_fun:quorum_read(Iterations);\n                read_read ->\n                    bench_fun:read_read(Iterations)\n            end,\n    BenchStart = os:timestamp(),\n    run_threads(Threads, Bench),\n    State#state{bench_start = BenchStart,\n                bench_owner = Owner,\n                bench_threads = Threads};\non({done, Time, Aborts},\n   State = #state{bench_owner = BenchOwner,\n                  bench_start = BenchStart,\n                  bench_threads = BenchThreads,\n                  bench_data = BenchData})\n  when BenchOwner =/= ok andalso BenchStart =/= ok andalso BenchThreads > 0 ->\n  % see http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#On-line_algorithm\n    NewBenchData =\n        case BenchData of\n            null ->\n                {1, Time, 0.0, Time, Time, Aborts};\n            {N, Mean, M2, Min, Max, AggAborts} ->\n                Delta = Time - Mean,\n                NewMean = Mean + Delta / (N + 1),\n                {N + 1, NewMean, M2 + Delta*(Time - NewMean),\n                 erlang:min(Time, Min), erlang:max(Time, Max),\n                 AggAborts + Aborts}\n        end,\n    if BenchThreads =:= 1 ->\n           RepTime = timer:now_diff(os:timestamp(), BenchStart),\n           {NewN, RepMeanTime, NewM2, RepMinTime, RepMaxTime, RepAborts} = NewBenchData,\n           RepVariance = if NewN =:= 1 -> 0.0;\n                            true       -> NewM2 / (NewN - 1)\n                         end,\n           comm:send(BenchOwner, {done, comm_server:get_local_address_port(),\n                                  RepTime, RepMeanTime, RepVariance,\n                                  RepMinTime, RepMaxTime, RepAborts}),\n           State#state{bench_owner = ok,\n                       bench_start = ok,\n                       bench_threads = 0,\n                       bench_data = null};\n       true ->\n           State#state{bench_threads = BenchThreads - 1,\n                       bench_data = NewBenchData}\n    end.\n\n-spec init([]) -> #state{}.\ninit([]) ->\n    % load bench module\n    _X = bench:module_info(),\n    #state{load_pid=ok}.\n\n%% @doc spawns a bench_server\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(ServiceGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [], [{pid_groups_join_as, ServiceGroup, ?MODULE},\n                                                             {erlang_register, ?MODULE}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% helper functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% @doc spawns threads and collect statistics\n-spec run_threads(integer(),\n                  fun((Parent::comm:erl_local_pid()) -> any())) -> ok.\nrun_threads(Threads, Bench) ->\n    Self = self(),\n    util:for_to(1, Threads,\n       fun(_X) ->\n           Pid = spawn_link(fun()->\n                         trace_mpath:thread_yield(),\n                         receive\n                             ?SCALARIS_RECV({start_thread}, ok)\n                         end,\n                         log:log(debug, \"Bench is infected ~p\", [proto_sched:infected()]),\n                         Bench(Self),\n                         trace_mpath:thread_yield()\n                       end),\n                         log:log(debug, \"Sending to spawned Bench ~p\", [proto_sched:infected()]),\n               comm:send_local(Pid, {start_thread})\n       end),\n    ok.\n"
  },
  {
    "path": "src/bulkowner.erl",
    "content": "%  @copyright 2007-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Bulk owner operations (for now only broadcasting).\n\n%% @version $Id$\n%% @reference Ali Ghodsi, <em>Distributed k-ary System: Algorithms for Distributed Hash Tables</em>, PhD Thesis, page 129.\n-module(bulkowner).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-define(TRACE(X, Y), ok).\n%% -define(TRACE(X, Y), io:format(X, Y)).\n\n-include(\"scalaris.hrl\").\n\n% public API:\n-export([issue_bulk_owner/3, issue_send_reply/4,\n         issue_bulk_distribute/5, issue_bulk_distribute/6,\n         send_reply/5, send_reply_failed/6]).\n\n% only use inside the dht_node process:\n-export([on/2]).\n\n-export_type([bulkowner_msg/0]).\n\n-type bulkowner_req() ::\n      {bulkowner, start,\n       Id::uid:global_uid(), I::intervals:interval(), Msg::comm:message()}\n      | {bulkowner, Id::uid:global_uid(), I::intervals:interval(),\n         Msg::comm:message(), Parents::[comm:mypid(),...]}\n      | {bulkowner, deliver, Id::uid:global_uid(), Range::intervals:interval(),\n         Msg::comm:message(), Parents::[comm:mypid(),...]}\n      | {bulkowner, reply, Id::uid:global_uid(), Target::comm:mypid(),\n         Msg::comm:message(), Parents::[comm:mypid()]}\n      | {bulkowner, reply_process_all}\n      | {bulkowner, gather, Id::uid:global_uid(), Target::comm:mypid(),\n         [comm:message()], Parents::[comm:mypid()]}.\n\n-type bulkowner_send_error() ::\n        {send_error, comm:mypid(), bulkowner_req(), atom() }.\n\n-type bulkowner_msg() :: bulkowner_req() | bulkowner_send_error().\n\n%% @doc Start a bulk owner operation to send the message to all nodes in the\n%%      given interval.\n-spec issue_bulk_owner(Id::uid:global_uid(), I::intervals:interval(), Msg::comm:message()) -> ok.\nissue_bulk_owner(Id, I, Msg) ->\n    DHTNode = pid_groups:find_a(dht_node),\n    comm:send_local(DHTNode, {bulkowner, start, Id, I, Msg}).\n\n-spec issue_send_reply(Id::uid:global_uid(), Target::comm:mypid(), Msg::comm:message(),\n                       Parents::[comm:mypid()]) -> ok.\nissue_send_reply(Id, Target, Msg, Parents) ->\n    DHTNode = pid_groups:find_a(dht_node),\n    comm:send_local(DHTNode, {bulkowner, reply, Id, Target, Msg, Parents}).\n\n-spec issue_bulk_distribute(ID::uid:global_uid(), atom(), pos_integer(),\n                           comm:message(), [{?RT:key(), string(), term()}] | {ets, ets:tab()}) -> ok.\nissue_bulk_distribute(Id, Proc, Pos, Msg, Data) ->\n    issue_bulk_distribute(Id, Proc, Pos, Msg, Data, intervals:all()).\n\n-spec issue_bulk_distribute(ID::uid:global_uid(), atom(), pos_integer(),\n                           comm:message(), [{?RT:key(), string(), term()}] | {ets, ets:tab()},\n                           intervals:interval()) -> ok.\nissue_bulk_distribute(Id, Proc, Pos, Msg, Data, Interval) ->\n    DHTNode = pid_groups:find_a(dht_node),\n    comm:send_local(DHTNode, {bulkowner, start, Id, Interval, {bulk_distribute, Proc,\n                                                        Pos, Msg, Data}}).\n\n-spec send_reply(Id::uid:global_uid(), Target::comm:mypid(), Msg::comm:message() | comm:group_message(),\n                 Parents::[comm:mypid()], Shepherd::comm:erl_local_pid()) -> ok.\nsend_reply(Id, Target, {?send_to_group_member, Proc, Msg}, [], Shepherd) ->\n    comm:send(Target, {bulkowner, reply, Id, Msg}, [{shepherd, Shepherd}, {group_member, Proc}]);\nsend_reply(Id, Target, {?send_to_registered_proc, Proc, Msg}, [], Shepherd) ->\n    comm:send(Target, {bulkowner, reply, Id, Msg}, [{shepherd, Shepherd}, {registered_proc, Proc}]);\nsend_reply(Id, Target, Msg, [], Shepherd) ->\n    comm:send(Target, {bulkowner, reply, Id, Msg}, [{shepherd, Shepherd}]);\nsend_reply(Id, Target, Msg, [Parent | Rest], Shepherd) ->\n    comm:send(Parent, {bulkowner, reply, Id, Target, Msg, Rest}, [{shepherd, Shepherd}]).\n\n-spec send_reply_failed(Id::uid:global_uid(), Target::comm:mypid(), Msg::comm:message(),\n                        Parents::[comm:mypid()], Shepherd::comm:erl_local_pid(),\n                        FailedPid::comm:mypid()) -> ok.\nsend_reply_failed(_Id, Target, Msg, [], _Shepherd, Target) ->\n    log:log(warn, \"[ ~p ] cannot send bulkowner_reply with message ~p (target node not available)\",\n            [pid_groups:pid_to_name(self()), Msg]);\nsend_reply_failed(Id, Target, Msg, Parents, Shepherd, _FailedPid) ->\n    send_reply(Id, Target, Msg, Parents, Shepherd).\n\n%% @doc main routine. It spans a broadcast tree over the nodes in I\n-spec bulk_owner(State::dht_node_state:state(), Id::uid:global_uid(),\n                 I::intervals:interval(), Msg::comm:message(),\n                 Parents::[comm:mypid()]) -> ok.\nbulk_owner(State, Id, I, Msg, Parents) ->\n%%     log:pal(\"bulk_owner:~n self:~p,~n int :~p,~n rt  :~p~n\", [dht_node_state:get(State, node), I, ?RT:to_list(State)]),\n    Neighbors = dht_node_state:get(State, neighbors),\n    SuccIntI = intervals:intersection(I, nodelist:succ_range(Neighbors)),\n    case intervals:is_empty(SuccIntI) of\n        true  -> ok;\n        false ->\n            case Msg of\n                {bulk_distribute, Proc, N, Env, Data} ->\n                    SuccData = get_range_data(Data, SuccIntI),\n                    comm:send(node:pidX(nodelist:succ(Neighbors)),\n                              {bulkowner, deliver, Id, SuccIntI,\n                              {bulk_distribute, Proc, N, Env, SuccData}, Parents});\n                _ ->\n                    comm:send(node:pidX(nodelist:succ(Neighbors)),\n                              {bulkowner, deliver, Id, SuccIntI, Msg, Parents})\n            end\n    end,\n    case I =:= SuccIntI of\n        true  -> ok;\n        false ->\n            MyNode = nodelist:node(Neighbors),\n            NewParents = [node:pidX(MyNode) | Parents],\n            RTList = lists:reverse(?RT:to_list(State)),\n            RemainingInterval = intervals:minus(I, SuccIntI),\n            bulk_owner_iter(RTList, Id, RemainingInterval, Msg, node:id(MyNode),\n                            NewParents)\n    end.\n\n%% @doc Iterates through the list of (unique) nodes in the routing table and\n%%      sends them the according bulkowner messages for sub-intervals of I.\n%%      The first call should have Limit=Starting_nodeid. The method will\n%%      then go through the ReverseRTList (starting with the longest finger,\n%%      ending with the node's successor) and send each node a bulk_owner\n%%      message for the interval it is responsible for:\n%%      I \\cap (id(Node_in_reversertlist), Limit], e.g.\n%%      node Nl from the longest finger is responsible for\n%%      I \\cap (id(Nl), id(Starting_node)].\n%%      Note that the range (id(Starting_node), id(Succ_of_starting_node)]\n%%      has already been covered by bulk_owner/3.\n-spec bulk_owner_iter(ReverseRTList::[{Id::?RT:key(), Pid::comm:mypid()}],\n                      Id::uid:global_uid(),\n                      I::intervals:interval(), Msg::comm:message(),\n                      Limit::?RT:key(), Parents::[comm:mypid(),...]) -> ok.\nbulk_owner_iter([], _Id, _I, _Msg, _Limit, _Parents) ->\n    ok;\nbulk_owner_iter([{HeadId, HeadPid} | Tail], Id, I, Msg, Limit, Parents) ->\n    case intervals:is_empty(I) of\n        false ->\n            Interval_Head_Limit =\n                node:mk_interval_between_ids(HeadId, Limit),\n            Range = intervals:intersection(I, Interval_Head_Limit),\n            %%     log:pal(\"send_bulk_owner_if: ~p ~p ~n\", [I, Range]),\n            NewLimit =\n                case intervals:is_empty(Range) of\n                    false ->\n                        case Msg of\n                            {bulk_distribute, Proc, N, Env, Data} ->\n                                RangeData = get_range_data(Data, Range),\n                                comm:send(HeadPid,\n                                          {bulkowner, Id, Range,\n                                           {bulk_distribute, Proc, N, Env, RangeData}, Parents},\n                                         [{group_member, dht_node}]);\n                            _ ->\n                                comm:send(HeadPid,\n                                          {bulkowner, Id, Range, Msg, Parents},\n                                          [{group_member, dht_node}])\n                        end,\n                        HeadId;\n                    true  -> Limit\n                end,\n            RemainingInterval = intervals:minus(I, Range),\n            bulk_owner_iter(Tail, Id, RemainingInterval, Msg, NewLimit, Parents);\n        true -> ok\n    end.\n\n%% protocol implementation / called inside the dht_node on handler\n-spec on(bulkowner_msg(), dht_node_state:state()) -> dht_node_state:state().\non({bulkowner, start, Id, I, Msg}, State) ->\n    bulk_owner(State, Id, I, Msg, []),\n    State;\n\non({bulkowner, Id, I, Msg, Parents}, State) ->\n    bulk_owner(State, Id, I, Msg, Parents),\n    State;\n\non({bulkowner, deliver, Id, Range, Msg, Parents}, State) ->\n    MsgFwd = dht_node_state:get(State, msg_fwd),\n\n    F = fun({FwdInt, FwdPid}, AccI) ->\n                case intervals:is_subset(FwdInt, AccI) of\n                    true ->\n                        FwdRange = intervals:intersection(AccI, FwdInt),\n                        ?TRACE(\"~p forwarding msg to ~p~nbecause of fwdInt ~p~n\",\n                               [self(), FwdPid, FwdInt]),\n                        case Msg of\n                            %% when Msg = bulk_distribute only forward the\n                            %% FwdInt part of the data. otherwise we get duplicate\n                            %% data\n                            {bulk_distribute, Proc, N, Msg1, Data} ->\n                                RangeData = get_range_data(Data, FwdRange),\n                                comm:send(FwdPid,\n                                      {bulkowner, deliver, Id, FwdRange,\n                                      {bulk_distribute, Proc, N, Msg1, RangeData},\n                                      Parents});\n                            _ ->\n                                comm:send(FwdPid, {bulkowner, deliver, Id, FwdRange, Msg, Parents})\n                        end,\n                        intervals:minus(AccI, FwdRange);\n                    _    -> AccI\n                end\n        end,\n    % we are responsible for the intersection of the BulkOwner range and our\n    % ring maintenance range (plus DB range from ongoing slides).\n    DBRange = lists:foldl(fun({I, _SlideId}, AccI) ->\n                                  intervals:union(AccI, I)\n                          end, intervals:empty(),\n                          dht_node_state:get(State, db_range)),\n    MyDBRange = intervals:union(dht_node_state:get(State, my_range), DBRange),\n    MyRange0 = lists:foldl(F, Range, MsgFwd),\n    MyRange = intervals:intersection(MyRange0, MyDBRange),\n    NewState = case intervals:is_empty(MyRange) of\n        true -> State;\n        _ ->\n            %% check if node has left the ring via a slide\n            %% if rm_loop:has_left/1 is true this message arrived through a\n            %% msg_fwd and needs to be passed back into the system\n            case rm_loop:has_left(dht_node_state:get(State, rm_state)) of\n                false ->\n                    handle_delivery(Msg, MyRange, Id, Parents, State);\n                true ->\n                    ?TRACE(\"~p bulkowner: delivering to leaving node...\n                            forwarding msg ~p to succ ~p~n\",\n                           [self(), Msg, nodelist:succ(dht_node_state:get(State,\n                                                                          neighbors))]),\n                    Neighbors = dht_node_state:get(State, neighbors),\n                    comm:send(node:pidX(nodelist:succ(Neighbors)),\n                              {bulkowner, Id, Range, Msg, Parents}),\n                    State\n            end\n    end,\n    RestRange = intervals:minus(MyRange0, MyRange),\n    % no need to check whether non-empty - this is a no-op anyway\n    bulk_owner(State, Id, RestRange, Msg, Parents),\n    NewState;\n\non({bulkowner, reply, Id, Target, Msg, Parents}, State) ->\n    % target information should be the same\n    PrevMsgs =\n        case erlang:get({'$bulkowner_reply_msg', Id}) of\n            undefined ->\n                _ = comm:send_local_after(100, self(),\n                                          {bulkowner, reply_process_all, Id}),\n                [];\n            {Id, Target, X = [_|_], _PrevParents} ->\n                X\n        end,\n    % Parents may be different if the node is twice in the tree -> use the latest:\n    _ = erlang:put({'$bulkowner_reply_msg', Id},\n                   {Id, Target, [Msg | PrevMsgs], Parents}),\n    State;\n\non({bulkowner, reply_process_all, Id}, State) ->\n    {Id, Target, Msgs, Parents} = erlang:erase({'$bulkowner_reply_msg', Id}),\n    case Msgs of\n        [] -> ok;\n        [{bulk_read_entry_response, _HRange, _HData} | _] ->\n            %% all messages must have the same type:\n            ?DBG_ASSERT([] =:= [Msg1 || Msg1 <- Msgs,\n                                        element(1, Msg1) =/= bulk_read_entry_response]),\n            comm:send_local(self(),\n                            {bulkowner, gather, Id, Target, Msgs, Parents});\n        [{?send_to_group_member, Proc, _Msg} | _] ->\n            %% all messages must have the same type:\n            ?DBG_ASSERT([] =:= [Msg1 || Msg1 <- Msgs,\n                                        element(1, Msg1) =/= ?send_to_group_member orelse\n                                            element(2, Msg1) =/= Proc]),\n            Msgs1 = [Msg1 || {?send_to_group_member, _Proc, Msg1} <- Msgs],\n            comm:forward_to_group_member(\n              Proc, {bulkowner, gather, Id, Target, Msgs1, Parents});\n        [{?send_to_registered_proc, Proc, _Msg} | _] ->\n            %% all messages must have the same type:\n            ?DBG_ASSERT([] =:= [Msg1 || Msg1 <- Msgs,\n                                        element(1, Msg1) =/= ?send_to_registered_proc orelse\n                                            element(2, Msg1) =/= Proc]),\n            Msgs1 = [Msg1 || {?send_to_registered_proc, _Proc, Msg1} <- Msgs],\n            comm:forward_to_registered_proc(\n              Proc, {bulkowner, gather, Id, Target, Msgs1, Parents})\n    end,\n    State;\n\non({bulkowner, gather, Id, Target, [H = {bulk_read_entry_response, _HRange, _HData} | T], Parents}, State) ->\n    Msg = lists:foldl(\n            fun({bulk_read_entry_response, Range, Data},\n                {bulk_read_entry_response, AccRange, AccData}) ->\n                    {bulk_read_entry_response,\n                     intervals:union(Range, AccRange),\n                     lists:append(Data, AccData)}\n            end, H, T),\n    send_reply(Id, Target, Msg, Parents, self()),\n    State;\n\non({send_error, FailedTarget, {bulkowner, reply, Id, Target, Msg, Parents}, _Reason}, State) ->\n    bulkowner:send_reply_failed(Id, Target, Msg, Parents, self(), FailedTarget),\n    State.\n\nhandle_delivery({bulk_read_entry, Issuer}, MyRange, Id, _Parents, State) ->\n    Data = db_dht:get_entries(dht_node_state:get(State, db), MyRange),\n    ReplyMsg = {bulk_read_entry_response, MyRange, Data},\n    % for aggregation using a tree, activate this instead:\n    % issue_send_reply(Id, Issuer, ReplyMsg, Parents);\n    comm:send(Issuer, {bulkowner, reply, Id, ReplyMsg}),\n    State;\nhandle_delivery({bulk_distribute, Proc, N, Msg1, Data}, MyRange, Id, Parents, State) ->\n    ?TRACE(\"~p bulkowner: delivering {~p, ~p} locally to ~p\",\n           [self(),\n            element(1, Msg1),\n            element(2, Msg1),\n            MyRange]),\n    RangeData = get_range_data(Data, MyRange),\n    %% only deliver data in MyRange as data outside of it was\n    %% forwarded\n    case Proc of\n        dht_node ->\n            gen_component:post_op({bulk_distribute, Id, MyRange,\n                                   setelement(N, Msg1, RangeData), Parents},\n                                  State);\n        _ ->\n            comm:forward_to_group_member(\n              Proc, {bulk_distribute, Id, MyRange,\n                     setelement(N, Msg1, RangeData), Parents}),\n            State\n    end;\nhandle_delivery({?send_to_group_member, Proc, Msg1}, MyRange, Id, Parents, State) ->\n    comm:forward_to_group_member(\n      Proc, {bulkowner, deliver, Id, MyRange, Msg1, Parents}),\n    State;\nhandle_delivery({?send_to_registered_proc, Proc, Msg1}, MyRange, Id, Parents, State) ->\n    comm:forward_to_registered_proc(\n      Proc, {bulkowner, deliver, Id, MyRange, Msg1, Parents}),\n    State;\nhandle_delivery({do_snapshot, _SnapNo, _Leader} = Msg, _MyRange, _Id, _Parents, State) ->\n    gen_component:post_op(Msg, State);\nhandle_delivery(MrMsg , MyRange, _Id, _Parents, State) when mr =:= element(1, MrMsg) ->\n    gen_component:post_op(erlang:append_element(MrMsg, MyRange), State);\nhandle_delivery({deliver_ping, Pid, Msg}, _MyRange, _Id, _Parents, State) ->\n    gen_component:post_op({ping, Pid, {pong, _MyRange, _Id, Msg}}, State);\nhandle_delivery(Msg, _MyRange, _Id, _Parents, State) ->\n    log:log(warn, \"[ ~p ] cannot deliver message ~p (don't know how to handle)\",\n            [pid_groups:pid_to_name(self()), Msg]),\n    State.\n\n-spec get_range_data({ets, db_ets:db()} | [{?RT:key(), nonempty_string(), term()},...],\n                     intervals:interval()) -> [{?RT:key(), nonempty_string(), term()}].\nget_range_data({ets, ETS}, Range) ->\n    lists:foldl(fun(Interval, Acc1) ->\n                    db_ets:foldl(ETS,\n                                 fun(E, Acc) -> [db_ets:get(ETS,E) | Acc] end,\n                                 Acc1, Interval)\n                end,\n                [], intervals:get_simple_intervals(Range));\nget_range_data(Data, Range) ->\n    lists:filter(\n      fun(Entry) ->\n              intervals:in(element(1, Entry),\n                           Range)\n      end, Data).\n"
  },
  {
    "path": "src/cloud_beh.erl",
    "content": "% @copyright 2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maximilian Michels <michels@zib.de>\n%% @doc Behaviour for various operations in a cloud environment, e.g. for\n%%      scaling.\n%% @end\n%% @version $Id$\n-module(cloud_beh).\n-author('michels@zib.de').\n-vsn('$Id$').\n\n-ifndef(have_callback_support).\n-export([behaviour_info/1]).\n-endif.\n\n-ifdef(have_callback_support).\n-include(\"scalaris.hrl\").\n\n-callback init() -> failed | ok.\n-callback get_number_of_vms() -> failed | non_neg_integer().\n-callback add_vms(Count::non_neg_integer()) -> failed | ok.\n-callback remove_vms(Count::non_neg_integer()) -> failed | ok.\n\n-else.\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n    [\n     {init, 0},\n     {get_number_of_vms, 0},\n     {add_vms, 1},\n     {remove_vms, 1}\n    ];\nbehaviour_info(_Other) ->\n    undefined.\n-endif.\n"
  },
  {
    "path": "src/cloud_local.erl",
    "content": "% @copyright 2013-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maximilian Michels <michels@zib.de>\n%% @doc CLOUD LOCAL starts or stops local erlang vms based on alarms defined for\n%%      the autoscale process.\n%%      The module is used by autoscale if the following option has been set in\n%%      scalaris.local.cfg:\n%%        {autoscale_cloud_module, cloud_local}\n%%      The following options can also be set:\n%%        {cloud_local_min_vms, integer()}.\n%%        {cloud_local_max_vms, integer()}.\n%% @end\n%% @version $Id$\n-module(cloud_local).\n-author('michels@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(_X,_Y), ok).\n\n-include(\"scalaris.hrl\").\n\n-behaviour(cloud_beh).\n\n-export([init/0, get_number_of_vms/0, add_vms/1, remove_vms/1]).\n-export([check_config/0]).\n\n\n%%%%%%%%%%%%%%%%%%%%%\n%%%% Behavior methods\n%%%%%%%%%%%%%%%%%%%%%\n\n-spec init() -> ok.\ninit() ->\n    ok.\n\n-spec get_vms() -> [string()].\nget_vms() ->\n    % only include VMs started by autoscale:\n    _VMs = [Name || {Name, _EpmdPort} <- erlang:element(2, erl_epmd:names()),\n                    re:run(Name, \"autoscale_.*\") =/= nomatch].\n\n-spec get_number_of_vms() -> non_neg_integer().\nget_number_of_vms() ->\n    length(get_vms()).\n\n-spec add_vms(non_neg_integer()) -> ok.\nadd_vms(N) ->\n    BaseScalarisPort = config:read(port),\n    BaseYawsPort = config:read(yaws_port),\n    MyPidStr = [case C of\n                    $. -> $-;\n                    _ -> C\n                end || C <- erlang:pid_to_list(self()),\n                       C =/= $<, C=/= $>],\n    NewNodesBase = format(\"autoscale_~s_~B\", [MyPidStr, uid:get_pids_uid()]),\n    SpawnFun = \n        fun (X) -> \n                Port = find_free_port(BaseScalarisPort),\n                YawsPort = find_free_port(BaseYawsPort),\n                NodeName = format(\"~s_~B_~s\", [NewNodesBase, X, node()]),\n                Cmd = format(\"./../bin/scalarisctl -e -detached -s -p ~p -y ~p -n ~s start\", \n                             [Port, YawsPort, NodeName]),\n                ?TRACE(\"Executing: ~p~n\", [Cmd]),\n                 NumberVMs = get_number_of_vms(),\n                _ = exec(Cmd),\n                util:wait_for(fun() -> get_number_of_vms() =:= NumberVMs + 1 end)\n        end,\n    _ = [SpawnFun(X) || X <- lists:seq(1, N), get_number_of_vms() < config:read(cloud_local_max_vms)],\n    ok.\n\n-spec remove_vms(non_neg_integer()) -> ok.\nremove_vms(N) ->\n    VMs = get_vms(),\n    RemoveFun = \n        fun(NodeName) ->\t\t\t\t\t\t\n                Cmd = format(\"./../bin/scalarisctl -n ~s gstop\", [NodeName]),\n                NumberVMs = get_number_of_vms(),\n                ?TRACE(\"Executing: ~p~n\", [Cmd]),\n                _ = exec(Cmd),\n                util:wait_for(fun() -> get_number_of_vms() =:= NumberVMs - 1 end)\n        end,\n    _ = [RemoveFun(NodeName) || NodeName <- lists:sublist(VMs, N),\n                                get_number_of_vms() > config:read(cloud_local_min_vms)],\n    ok.\n\n%%%%%%%%%%%%%%%%%%%\n%%%% Helper methods\n%%%%%%%%%%%%%%%%%%%\n-spec find_free_port(comm_server:tcp_port() | [comm_server:tcp_port()] |\n                     {From::comm_server:tcp_port(), To::comm_server:tcp_port()})\n                    -> comm_server:tcp_port().\nfind_free_port({From, To}) ->\n    find_free_port(lists:seq(From, To));\nfind_free_port([Port | Rest]) when is_integer(Port) andalso Port >= 0 andalso Port =< 65535 ->\n    case gen_tcp:listen(Port, []) of\n        {ok, Socket} -> gen_tcp:close(Socket), \n                        Port;\n        _ when Rest =:= [] -> find_free_port([Port+1]);\n        _ -> find_free_port(Rest)\n    end;\nfind_free_port([]) ->\n    find_free_port([0]);\nfind_free_port(Port) ->\n    find_free_port([Port]).\n\n\n-spec format(string(), list()) -> string().\nformat(FormatString, Items) ->\n    lists:flatten(io_lib:format(FormatString, Items)).\n\n-spec exec(string()) -> pid().\nexec(Cmd) ->\n    _ = spawn(os, cmd, [Cmd]).\n\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(cloud_local_min_vms) and\n    config:cfg_is_integer(cloud_local_max_vms) and\n    config:cfg_is_greater_than_equal(cloud_local_min_vms, 0) and\n    config:cfg_is_greater_than(cloud_local_max_vms, config:read(cloud_local_min_vms)).\n"
  },
  {
    "path": "src/cloud_ssh.erl",
    "content": "% @copyright 2013-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maximilian Michels <michels@zib.de>\n%% @doc Cloud SSH starts or stops erlang vms on ssh hosts based on alarms\n%%      defined for the autoscale process.\n%%      The module is used by autoscale if the following option has been set in\n%%      scalaris.local.cfg:\n%%        {autoscale_cloud_module, cloud_ssh}\n%%      The following options can also be set:\n%%        {cloud_ssh_hosts, [\"host1\", \"host2\", ..., \"hostn\"]}.\n%%        {cloud_ssh_path, \"path/to/scalaris/installation/on/host\"}.\n%%      Additional services besides Scalaris may be specified:\n%%        {cloud_ssh_services, [{ServiceStartCmd, ServiceStopCmd}, {Service2StartCmd, Service2StopCmd}, ...]}.\n%% @end\n%% @version $Id$\n-module(cloud_ssh).\n-author('michels@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(_X,_Y), ok).\n\n-include(\"scalaris.hrl\").\n\n-behaviour(cloud_beh).\n\n-export([init/0, get_number_of_vms/0, add_vms/1, remove_vms/1, killall_vms/0]).\n-export([check_config/0]).\n\n-define(cloud_ssh_key, \"2a42cb863313526fca96098a95020db2a904b01157f191a9bb3200829f8596c7\").\n-define(scalaris_start, \"bin/./scalarisctl -e -detached -s -p 14915 -y 8000 -n node1 start\").\n-define(scalaris_stop, \"bin/./scalarisctl -n node1 gstop\").\n\n-type status() :: active | inactive.\n-type host() :: {string(), status()}.\n-type hostlist() :: list(host()).\n\n%%%%%%%%%%%%%%%%%%%%%\n%%%% Behavior methods\n%%%%%%%%%%%%%%%%%%%%%\n\n-spec init() -> failed | ok.\ninit() ->\n    case get_hosts() of\n        failed ->\n            Hostnames = config:read(cloud_ssh_hosts),\n            HostsWithStatus = lists:map(fun(Host) -> {Host, inactive} end, Hostnames),\n            save_hosts(HostsWithStatus);\n        _ -> ok\n    end.\n\n-spec get_number_of_vms() -> failed | non_neg_integer().\nget_number_of_vms() ->\n    case get_hosts() of\n        failed -> failed;\n        {_, List} -> get_number_of_vms(List)\n    end.\n\n-spec get_number_of_vms(hostlist()) -> failed | non_neg_integer().\nget_number_of_vms(Hosts) ->\n    lists:foldl(fun (VM, NumActive) ->\n                        case VM of\n                            {_IP, active} ->\n                                NumActive + 1;\n                            _ -> NumActive\n                        end\n                end, 0, Hosts).\n\n-spec add_vms(pos_integer()) -> failed | ok.\nadd_vms(N) ->\n    add_or_remove_vms(add, N).\n\n-spec remove_vms(pos_integer()) -> failed | ok.\nremove_vms(N) ->\n    add_or_remove_vms(remove, N).\n\n-spec add_or_remove_vms(add | remove, integer()) ->  failed | ok.\nadd_or_remove_vms(Flag, Pending) ->\n    case get_hosts() of\n    \t{TLog, Hosts} ->\n            UpdatedHosts = add_or_remove_vms(Flag, Pending, Hosts, []),\n            save_hosts(TLog, UpdatedHosts);\n        failed -> failed\n    end.\n\n-spec add_or_remove_vms(add | remove, integer(), hostlist(), hostlist()) -> hostlist().\nadd_or_remove_vms(_Flag, _Pending = 0, Hosts, UpdatedHosts) ->\n    UpdatedHosts ++ Hosts;\nadd_or_remove_vms(_Flag, _Pending, _Hosts = [], UpdatedHosts) ->\n    UpdatedHosts;\nadd_or_remove_vms(add, Pending, Hosts, UpdatedHosts) ->\n    [Host | RemainingHosts] = Hosts,\n    case Host of\n        {_, active} ->\n            add_or_remove_vms(add, Pending, RemainingHosts, UpdatedHosts ++ [Host]);\n        {Hostname, _} ->\n            scalaris_vm(start, Hostname),\n            add_or_remove_vms(add, Pending - 1, RemainingHosts, UpdatedHosts ++ [{Hostname, active}])\n    end;\nadd_or_remove_vms(remove, Pending, Hosts, UpdatedHosts) ->\n    [Host | RemainingHosts] = Hosts,\n    case Host of\n        {Hostname, active} ->\n            scalaris_vm(stop, Hostname),\n            add_or_remove_vms(remove, Pending - 1, RemainingHosts, UpdatedHosts ++ [{Hostname, inactive}]);\n        {_, _} ->\n            add_or_remove_vms(remove, Pending, RemainingHosts, UpdatedHosts ++ [Host])\n    end.\n\n\n%%%%%%%%%%%%%%%%%%%\n%%%% Helper methods\n%%%%%%%%%%%%%%%%%%%\n\n-spec killall_vms() -> failed | ok.\nkillall_vms() ->\n    case get_hosts() of\n        {_, Hosts} ->\n            lists:foreach(fun({Hostname, Status}) ->\n                                  case Status of\n                                      active ->\n                                          scalaris_vm(stop, Hostname);\n                                      inactive ->\n                                          ok\n                                  end\n                          end, Hosts),\n            ok;\n        failed -> failed\n    end.\n\n-spec scalaris_vm(start | stop, string()) -> ok.\nscalaris_vm(Action, Hostname) ->\n    Scalaris = get_scalaris_service(),\n    Services = [Scalaris | get_additional_services()],\n    lists:foreach(fun (Service) ->\n                          {StartCmd, StopCmd} = Service,\n                          ServiceCmd =\n                              case Action of\n                                  start -> StartCmd;\n                                  stop  -> StopCmd\n                              end,\n                          Cmd = format(\"ssh -n -f ~s \\\"(~s)\\\"\", [Hostname, ServiceCmd]),\n                          ?TRACE(\"Executing: ~p~n\", [Cmd]),\n                          _ = exec(Cmd)\n                  end, Services),\n    ok.\n\n-spec format(string(), list()) -> string().\nformat(FormatString, Items) ->\n    lists:flatten(io_lib:format(FormatString, Items)).\n\n-spec get_hosts() -> failed | {tx_tlog:tlog_ext(), hostlist()}.\nget_hosts() ->\n    case api_tx:read(api_tx:new_tlog(), ?cloud_ssh_key) of\n        {TLog, {ok, Hosts}} -> {TLog, Hosts};\n        _ -> failed\n    end.\n\n-spec save_hosts(hostlist()) -> failed | ok.\nsave_hosts(Hosts) ->\n    save_hosts(api_tx:new_tlog(), Hosts).\n\n-spec save_hosts(tx_tlog:tlog_ext(), hostlist()) -> failed | ok.\nsave_hosts(TLog, Hosts) ->\n    case api_tx:req_list(TLog, [{write, ?cloud_ssh_key, Hosts}, {commit}]) of\n        {[], [{ok}, {ok}]} -> ok;\n        _ -> failed\n    end.\n\n-spec get_scalaris_service() -> {string(), string()}.\nget_scalaris_service() ->\n    Path =\n        case config:read(cloud_ssh_path) of\n            failed -> \"scalaris\";\n            Arg -> Arg\n        end,\n    {format(\"~s/~s\", [Path, ?scalaris_start]), format(\"~s/~s\", [Path, ?scalaris_stop])}.\n\n-spec get_additional_services() -> [{string(),string()}].\nget_additional_services() ->\n    case config:read(cloud_ssh_services) of\n        failed -> [];\n        [T|H] -> [T|H]\n    end.\n\n-spec exec(string()) -> pid().\nexec(Cmd) ->\n    _ = spawn(os, cmd, [Cmd]).\n\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_list(cloud_ssh_hosts) and\n    config:cfg_is_string(cloud_ssh_path) and\n    config:cfg_is_list(cloud_ssh_services).\n"
  },
  {
    "path": "src/comm.erl",
    "content": "%  @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Message Sending.\n%%\n%%  Messages consist of a tuple of which the first element is the message's\n%%  tag, i.e. an atom. Process identifiers depend on local and global target\n%%  and can also wrap up an reply envelope that can be used to\n%%  rewrite reply messages.\n%%\n%%  Sending messages to so-enveloped process identifiers works\n%%  seamlessly, e.g.  a server receiving message {tag, SourcePid} can\n%%  reply with comm:send(SourcePid, {tag_response}). On the receiving\n%%  side (a client), the reply message is embedded into the envelope\n%%  tuple at the specified position. Pids\n%%  with envelopes  can be created using reply_as/3.  @end\n%%  @version $Id$\n-module(comm).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n%% Sending messages\n-export([send/2, send/3, send_local/2, send_local/3, send_local_after/3,\n         forward_to_group_member/2, forward_to_registered_proc/2]).\n\n%% Pid manipulation\n-export([make_global/1, make_local/1]).\n-export([this/0, get/2]).\n-export([reply_as/3]).\n-export([is_valid/1, is_local/1]).\n-export([get_ip/1, get_port/1]).\n\n%% Message manipulation\n-export([get_msg_tag/1]).\n-export([unpack_cookie/2, get_plain_pid/1]).\n\n%% initialization\n-export([init_and_wait_for_valid_IP/0]).\n\n-export_type([message/0, group_message/0, msg_tag/0,\n              mypid/0, mypid_plain/0,\n              erl_local_pid/0, erl_local_pid_plain/0,\n              erl_local_pid_with_reply_as/0,\n              send_options/0, send_local_options/0, channel/0]).\n\n-type msg_tag() :: atom() | byte(). %% byte() in case of compact external atoms. See include/atom_ext.hrl\n\n%% there is no variable length-tuple definition for types\n%% -> declare messages with up to 15 parameters here:\n-type envelope() ::\n        {msg_tag(), any()} |\n        {msg_tag(), any(), any()} |\n        {msg_tag(), any(), any(), any()} |\n        {msg_tag(), any(), any(), any(), any()} |\n        {msg_tag(), any(), any(), any(), any(), any()} |\n        {msg_tag(), any(), any(), any(), any(), any(), any()} |\n        {msg_tag(), any(), any(), any(), any(), any(), any(), any()} |\n        {msg_tag(), any(), any(), any(), any(), any(), any(), any(), any()} |\n        {msg_tag(), any(), any(), any(), any(), any(), any(), any(), any(), any()} |\n        {msg_tag(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any()} |\n        {msg_tag(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any()} |\n        {msg_tag(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any()} |\n        {msg_tag(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any()} |\n        {msg_tag(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any()} |\n        {msg_tag(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any()}.\n-type message() ::\n        {msg_tag()} | envelope().\n\n-type channel() :: main | prio.\n\n-type reg_name()                  :: atom().\n-type erl_local_pid_plain()       :: pid() | reg_name().\n-type mypid_plain() :: {inet:ip_address(),\n                        comm_server:tcp_port(),\n                        erl_local_pid_plain()}.\n\n-type group_message() :: {?send_to_group_member | ?send_to_registered_proc, atom(), message() | group_message()}.\n% envelopes:\n-type erl_local_pid_with_reply_as() ::\n        {erl_local_pid_plain(),         e, pos_integer(), envelope()}\n      | {erl_local_pid_with_reply_as(), e, pos_integer(), envelope()}.\n-type mypid_with_reply_as() ::\n        {mypid_plain(),         e, pos_integer(), envelope()}\n      | {mypid_with_reply_as(), e, pos_integer(), envelope()}.\n\n-type erl_local_pid() :: erl_local_pid_plain() | erl_local_pid_with_reply_as().\n-type mypid()         :: mypid_plain() | mypid_with_reply_as().\n\n%-type plain_pid() :: mypid_plain() | erl_local_pid_plain().\n\n-type send_options() :: [{shepherd, Pid::erl_local_pid()} |\n                         {group_member, Process::atom()} |\n                         {channel, channel()} | {?quiet} |\n                         {no_keep_alive}].\n\n-type send_local_options() :: [{?quiet}].\n\n-dialyzer({no_contracts, [unpack_cookie/2, get_plain_pid/1]}).\n\n%% @doc Sends a message to a process given by its pid.\n-spec send(mypid(), message() | group_message()) -> ok.\nsend(Pid, Msg) -> send(Pid, Msg, []).\n\n%% @doc Send a message to an arbitrary process with the given options.\n%%      If a shepherd is given, it will be informed when the sending fails;\n%%      with a message of the form:\n%%       {send_error, Pid, Msg, Reason}.\n%%      If a group_member is given, the message is send to an arbitrary process\n%%      of another node instructing it to forward the message to a process in\n%%      its group with the given name.\n-spec send(mypid(), message() | group_message(), send_options()) -> ok.\nsend(Pid, Msg, Options) ->\n    {RealPid, RealMsg1} = unpack_cookie(Pid, Msg),\n    {RealMsg, RealOpts} = pack_group_member(RealMsg1, Options),\n    case erlang:get(trace_mpath) of\n        undefined ->\n            comm_server:send(RealPid, RealMsg, RealOpts);\n        Logger ->\n            RealNumericPid =\n                case comm_server:is_local(RealPid) of\n                    true ->\n                        LocalPidPart = make_local(RealPid),\n                        LocalNumericPid = case is_atom(LocalPidPart) of\n                                              true -> whereis(LocalPidPart);\n                                              false -> LocalPidPart\n                                          end,\n                        make_global(LocalNumericPid);\n                    false ->\n                        % TODO: convert named process on remote node?\n                        RealPid\n                end,\n            Deliver = trace_mpath:log_send(Logger, self(),\n                                           RealNumericPid, RealMsg, global,\n                                           Options),\n            case Deliver of\n                false -> ok;\n                true ->\n                    %% send infected message to destination\n                    LogEpidemicMsg = trace_mpath:epidemic_reply_msg(\n                                       Logger, self(), RealPid, RealMsg),\n                    comm_server:send(RealPid, LogEpidemicMsg, RealOpts)\n            end\n    end,\n    ok.\n\n-ifdef(enable_debug).\n-define(SEND_LOCAL_CHECK_PID(Pid, Msg, Options),\n        if is_atom(Pid) ->\n               case whereis(Pid) of\n                   undefined ->\n                       case lists:member({?quiet}, Options) of\n                           false ->\n                               log:log(warn, \"~p (name: ~.0p) Send to ~.0p failed, \"\n                                       \"drop message ~.0p due to ~p\",\n                                       [self(), pid_groups:my_pidname(), RealPid,\n                                        RealMsg, local_target_not_alive]);\n                           _ -> ok\n                       end;\n                   _ -> ok\n               end;\n           is_pid(Pid) ->\n               case is_process_alive(Pid) andalso\n                        erlang:process_info(Pid, priority) =/= {priority, low} of\n                   true -> ok;\n                   false ->\n                       case lists:member({?quiet}, Options) of\n                           false ->\n                               log:log(warn, \"~p (name: ~.0p) Send to ~.0p failed, \"\n                                       \"drop message ~.0p due to ~p\",\n                                       [self(), pid_groups:my_pidname(), RealPid,\n                                        RealMsg, local_target_not_alive]);\n                           _ -> ok\n                       end\n               end\n        end).\n-else.\n-define(SEND_LOCAL_CHECK_PID(Pid, Msg, Options), ok).\n-endif.\n\n%% @doc Sends a message to a local process given by its local pid\n%%      (as returned by self()).\n-spec send_local(erl_local_pid(), message() | group_message()) -> ok.\nsend_local(Pid, Msg) ->\n    send_local(Pid, Msg, []).\n\n%% @doc Sends a message to a local process given by its local pid\n%%      (as returned by self()).\n-spec send_local(erl_local_pid(), message() | group_message(),\n                 send_local_options()) -> ok.\nsend_local(Pid, Msg, Options) ->\n    {RealPid, RealMsg} = unpack_cookie(Pid, Msg),\n    _ = case erlang:get(trace_mpath) of\n            undefined ->\n                ?SEND_LOCAL_CHECK_PID(RealPid, RealMsg, Options),\n                RealPid ! RealMsg;\n            Logger ->\n                RealNumericPid = case is_atom(RealPid) of\n                                     true -> whereis(RealPid);\n                                     false -> RealPid\n                                 end,\n                Deliver = trace_mpath:log_send(Logger, self(),\n                                               RealNumericPid, RealMsg, local,\n                                               Options),\n                case Deliver of\n                    false -> ok;\n                    true ->\n                        LogEpidemicMsg = trace_mpath:epidemic_reply_msg(\n                                           Logger, self(), RealPid, RealMsg),\n                        ?SEND_LOCAL_CHECK_PID(RealPid, LogEpidemicMsg, Options),\n                        RealPid ! LogEpidemicMsg\n                end\n        end,\n    ok.\n\n%% @doc Sends a message to a local process given by its local pid\n%%      (as returned by self()) after the given delay in milliseconds.\n-spec send_local_after(Delay::0..4294967295, erl_local_pid(), message() | group_message()) -> reference().\nsend_local_after(Delay, Pid, Msg) ->\n    {RealPid, RealMsg} = unpack_cookie(Pid, Msg),\n    case erlang:get(trace_mpath) of\n        undefined ->\n            erlang:send_after(Delay, RealPid, RealMsg);\n        Logger ->\n            %% TODO: put RealMsg into the delayed pool of proto_sched\n            %%       by using new a new send type 'local_after'. Have\n            %%       also to adapt trace_mpath then.\n            Deliver = trace_mpath:log_send(Logger, self(),\n                                           RealPid, RealMsg, local_after, []),\n            %% should we also deliver using the normal way?\n            %% (is it a trace (=true) or a proto_sched (=false)).\n            case Deliver of\n                false ->\n                    %% to keep the -spec we return a reference().  In\n                    %% contrast to erlang:send_after() one cannot\n                    %% cancel this message with a correspondig\n                    %% erlang:cancel_timer(). It still will be\n                    %% delivered after the corresponding attempt to\n                    %% cancel it.  We cannot solve this better, as we\n                    %% cannot track the corresponding cancel_timer().\n                    erlang:make_ref();\n                true ->\n                    LogEpidemicMsg = trace_mpath:epidemic_reply_msg(\n                                       Logger, self(), RealPid, RealMsg),\n                    erlang:send_after(Delay, RealPid, LogEpidemicMsg)\n            end\n    end.\n\n%% @doc Convert a local or global erlang pid to a global pid of type\n%%      mypid() for use in send/2.\n-spec make_global(erl_local_pid() | mypid()) -> mypid().\nmake_global({Pid, e, Nth, Cookie}) ->\n    {make_global(Pid), e, Nth, Cookie};\nmake_global(Pid) when is_pid(Pid) -> get(Pid, this());\nmake_global(Pid) when is_atom(Pid) -> get(Pid, this());\nmake_global(GlobalPid) -> GlobalPid.\n\n%% @doc Convert a global mypid() of the current node to a local erlang pid.\n-spec make_local(erl_local_pid() | mypid()) -> erl_local_pid().\nmake_local({Pid, e, Nth, Cookie}) when is_pid(Pid) orelse is_atom(Pid) ->\n    {Pid, e, Nth, Cookie};\nmake_local({Pid, e, Nth, Cookie}) ->\n    {make_local(Pid), e, Nth, Cookie};\nmake_local(Pid) when is_pid(Pid) orelse is_atom(Pid) -> Pid;\nmake_local(Pid) -> comm_server:make_local(Pid).\n\n%% @doc Returns the global pid of the current process.\n-spec this() -> mypid_plain().\nthis() -> comm_server:this().\n\n%% @doc Creates the plain PID a process with name Name would have on node Node.\n-spec get(Name::erl_local_pid_plain(), Node::mypid()) -> mypid_plain().\nget(Name, {Pid, e, _Nth, _Envelope} = _Node) ->\n    get(Name, Pid);\nget(Name, {IP, Port, _Pid} = _Node) -> {IP, Port, Name}.\n\n\n%% @doc Encapsulates the given pid (local or global) with the reply_as\n%%      request, so a send/2 to the generated target will put a reply\n%%      message at the Nth position of the given envelope.\n-spec reply_as(erl_local_pid() | mypid(), 2..16, envelope()) ->\n               mypid_with_reply_as() | erl_local_pid_with_reply_as().\nreply_as(Target, Nth, Envelope) ->\n    ?DBG_ASSERT('_' =:= element(Nth, Envelope)),\n    {Target, e, Nth, Envelope}.\n\n%% @doc Check whether the given pid is well formed.\n-spec is_valid(mypid() | any()) -> boolean().\nis_valid({Pid, e, _Nth, _Cookie}) -> is_valid(Pid);\nis_valid(Pid) -> comm_server:is_valid(Pid).\n\n%% @doc Check whether a global mypid() can be converted to a local\n%%      pid of the current node.\n-spec is_local(mypid()) -> boolean().\nis_local(Pid) ->\n    comm_server:is_local(get_plain_pid(Pid)).\n\n%% @doc Gets the IP address of the given (global) mypid().\n-spec get_ip(mypid()) -> inet:ip_address().\nget_ip(Pid) ->\n    comm_server:get_ip(get_plain_pid(Pid)).\n\n%% @doc Gets the port of the given (global) mypid().\n-spec get_port(mypid()) -> non_neg_integer().\nget_port(Pid) ->\n    comm_server:get_port(get_plain_pid(Pid)).\n\n\n%% @doc Gets the tag of a message (the first element of its tuple - should be an\n%%      atom).\n-spec get_msg_tag(message() | group_message()) -> msg_tag().\nget_msg_tag({Msg, _Cookie})\n  when is_tuple(Msg) andalso (is_atom(erlang:element(1, Msg)) orelse is_integer(erlang:element(1, Msg))) ->\n    get_msg_tag(Msg);\nget_msg_tag({?send_to_group_member, _ProcessName, Msg})\n  when is_tuple(Msg) andalso (is_atom(erlang:element(1, Msg)) orelse is_integer(erlang:element(1, Msg))) ->\n    get_msg_tag(Msg);\nget_msg_tag({?send_to_registered_proc, _ProcessName, Msg})\n  when is_tuple(Msg) andalso (is_atom(erlang:element(1, Msg)) orelse is_integer(erlang:element(1, Msg))) ->\n    get_msg_tag(Msg);\nget_msg_tag(Msg)\n  when is_tuple(Msg) andalso (is_atom(erlang:element(1, Msg)) orelse is_integer(erlang:element(1, Msg))) ->\n    erlang:element(1, Msg).\n\n-spec unpack_cookie(mypid(), message()) -> {mypid_plain(), message()};\n                   (erl_local_pid(), message()) -> {erl_local_pid_plain(), message()}.\nunpack_cookie({Pid, e, Nth, Envelope}, Msg) ->\n    unpack_cookie(Pid, setelement(Nth, Envelope, Msg));\nunpack_cookie(Pid, Msg) ->\n    {Pid, Msg}.\n\n-spec get_plain_pid(mypid()) -> mypid_plain();\n                   (erl_local_pid()) -> erl_local_pid_plain().\nget_plain_pid({Pid, e, _Nth, _Envelope}) ->\n    get_plain_pid(Pid);\nget_plain_pid(Pid) ->\n    Pid.\n\n%% @doc Creates a group member message and filter out the send options for the\n%%      comm_server process.\n-spec pack_group_member(message() | group_message(), send_options()) -> {message() | group_message(), send_options()}.\npack_group_member(Msg, [] = Opts)                      -> {Msg, Opts};\npack_group_member(Msg, [{shepherd, _Shepherd}] = Opts) -> {Msg, Opts};\npack_group_member(Msg, Opts)                           ->\n    case lists:keytake(group_member, 1, Opts) of\n        false ->\n            case lists:keytake(registered_proc, 1, Opts) of\n                false ->\n                    {Msg, Opts};\n                {value, {registered_proc, Process}, Opts2} ->\n                    {{?send_to_registered_proc, Process, Msg}, Opts2}\n            end;\n        {value, {group_member, Process}, Opts2} ->\n            {{?send_to_group_member, Process, Msg}, Opts2}\n    end.\n\n%% @doc Forwards a message to a group member by its process name.\n%%      NOTE: Does _not_ warn if the group member does not exist (should only\n%%            happen during node leave operations).\n-spec forward_to_group_member(atom(), message()) -> ok.\nforward_to_group_member(Processname, Msg) ->\n    case pid_groups:get_my(Processname) of\n        failed -> ok;\n        Pid    -> comm:send_local(Pid, Msg)\n    end.\n\n%% @doc Forwards a message to a registered process.\n%%      NOTE: Does _not_ warn if the process does not exist (should only\n%%            happen during node leave operations).\n-spec forward_to_registered_proc(atom(), message()) -> ok.\nforward_to_registered_proc(Processname, Msg) ->\n    case whereis(Processname) of\n        undefined            -> ok;\n        Pid when is_pid(Pid) -> comm:send_local(Pid, Msg)\n    end.\n\n%% @doc Initializes the comm layer by sending a message to\n%%      known_hosts. A valid IP (and comm:mypid()) for comm:this/0\n%%      will be available afterwards.  (ugly hack to get a valid\n%%      ip-address into the comm-layer)\n-spec init_and_wait_for_valid_IP() -> ok.\ninit_and_wait_for_valid_IP() ->\n    case is_valid(this()) of\n        true -> ok;\n        false ->\n            KnownHosts1 = config:read(known_hosts),\n            % maybe the list of known nodes is empty and we have a mgmt_server?\n            MgmtServer = config:read(mgmt_server),\n            KnownHosts = case is_valid(MgmtServer) of\n                             true -> [MgmtServer | KnownHosts1];\n                             _ -> KnownHosts1\n                         end,\n            % note, comm:this() may be invalid at this moment\n            _ = [send(KnownHost, {comm_says_hi}, [{group_member, service_per_vm}, {?quiet}])\n                   || KnownHost <- KnownHosts],\n            timer:sleep(100),\n            init_and_wait_for_valid_IP()\n    end.\n"
  },
  {
    "path": "src/comm_layer/comm_connection.erl",
    "content": "%% @copyright 2007-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Florian Schintke <schintke@zib.de>\n\n%% @doc creates and destroys connections and represents the endpoint\n%%      of a connection where messages are received from and send to the\n%%      network.\n%% @end\n%% @version $Id$\n-module(comm_connection).\n-author('schuett@zib.de').\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n-behaviour(gen_component).\n\n%% number of tags or messages to keep (for debugging)\n-define(NUM_KEEP, 10).\n%% keep only tag (uncomment to keep msg)\n-define(KEEP_TAG, true).\n\n-define(COMM, (config:read(comm_backend))). %% comm_layer backend\n\n-compile({inline, [dest_ip/1, dest_port/1, local_listen_port/1, channel/1,\n                   socket/1, set_socket/2,\n                   started/1,\n                   s_msg_count/1, inc_s_msg_count/2,\n                   r_msg_count/1, inc_r_msg_count/2,\n                   msg_queue/1, set_msg_queue/2,\n                   msg_queue_len/1, set_msg_queue_len/2,\n                   desired_bundle_size/1, set_desired_bundle_size/2,\n                   msgs_since_bundle_start/1, inc_msgs_since_bundle_start/1,\n                   set_msgs_since_bundle_start/2,\n                   last_stat_report/1, set_last_stat_report/2]}).\n\n-include(\"scalaris.hrl\").\n\n-ifdef(KEEP_TAG).\n    -define(GET_MSG_OR_TAG(Msg), comm:get_msg_tag(Msg)).\n    -define(PRETTY_PRINT_MSG(Tag), util:extint2atom(Tag)).\n-else.\n    -define(GET_MSG_OR_TAG(Msg), Msg).\n    -define(PRETTY_PRINT_MSG(Msg), setelement(1, Msg, util:extint2atom(element(1, Msg)))).\n-endif.\n\n-export([start_link/6, init/1, on/2]).\n\n-include(\"gen_component.hrl\").\n\n-type msg_queue() :: {MQueue::[{DestPid::pid(), Message::comm:message()}],\n                      OQueue::[comm:send_options()]}.\n-type stat_report() :: {RcvCnt::non_neg_integer(), RcvBytes::non_neg_integer(),\n                        SendCnt::non_neg_integer(), SendBytes::non_neg_integer()}.\n-type time_last_msg() :: erlang_timestamp().\n-type msg_or_tag() :: comm:message() | comm:msg_tag().\n\n-type socket() :: inet:socket() | ssl:sslsocket().\n\n-type state() ::\n    {DestIP                  :: inet:ip_address(),\n     DestPort                :: comm_server:tcp_port(),\n     LocalListenPort         :: comm_server:tcp_port(),\n     Channel                 :: comm:channel(),\n     Socket                  :: socket() | notconnected,\n     StartTime               :: erlang_timestamp(),\n     SentMsgCount            :: non_neg_integer(),\n     ReceivedMsgCount        :: non_neg_integer(),\n     MsgQueue                :: msg_queue(),\n     MsgQueueLen             :: non_neg_integer(),\n     DesiredBundleSize       :: non_neg_integer(),\n     MsgsSinceBundleStart    :: non_neg_integer(),\n     LastStatReport          :: stat_report(),\n     TimeLastMsgSent         :: time_last_msg(),\n     TimeLastMsgReceived     :: time_last_msg(),\n     SentMsgCountSession     :: non_neg_integer(),\n     ReceivedMsgCountSession :: non_neg_integer(),\n     SessionCount            :: non_neg_integer(),\n     LastMsgSent             :: [msg_or_tag()],\n     LastMsgReceived         :: [msg_or_tag()]\n    }.\n-type message() ::\n    {send, DestPid::pid(), Message::comm:message(), Options::comm:send_options()} |\n    {tcp, Socket::socket(), Data::binary()} |\n    {tcp_closed, Socket::socket()} |\n    {report_stats} |\n    {web_debug_info, Requestor::comm:erl_local_pid()}.\n\n%% be startable via supervisor, use gen_component\n\n-spec start_link(pid_groups:groupname(), DestIP::inet:ip_address(),\n                 comm_server:tcp_port(), socket() | notconnected,\n                 Channel::comm:channel(), Dir::'rcv' | 'send' | 'both') -> {ok, pid()}.\nstart_link(CommLayerGroup, {IP1, IP2, IP3, IP4} = DestIP, DestPort, Socket, Channel, Dir) ->\n    {_, LocalListenPort} = comm_server:get_local_address_port(),\n    DirStr = case Dir of\n                 'rcv'  -> \" <-  \";\n                 'send' -> \"  -> \";\n                 'both' -> \" <-> \"\n             end,\n    PidName = atom_to_list(Channel) ++ DirStr ++ integer_to_list(IP1) ++ \".\"\n        ++ integer_to_list(IP2) ++ \".\" ++ integer_to_list(IP3) ++ \".\"\n        ++ integer_to_list(IP4) ++ \":\" ++ integer_to_list(DestPort),\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2,\n                             {DestIP, DestPort, LocalListenPort, Channel, Socket},\n                             [{pid_groups_join_as, CommLayerGroup, PidName},\n                              {spawn_opts, [{fullsweep_after, 0},\n                                            {min_heap_size, 131071}]}]).\n\n%% @doc initialize: return initial state.\n-spec init({DestIP::inet:ip_address(), DestPort::comm_server:tcp_port(),\n            LocalListenPort::comm_server:tcp_port(), Channel::comm:channel(),\n            Socket::socket() | notconnected}) -> state().\ninit({DestIP, DestPort, LocalListenPort, Channel, Socket}) ->\n    msg_delay:send_trigger(10, {report_stats}),\n    msg_delay:send_trigger(10, {check_idle}),\n    state_new(DestIP, DestPort, LocalListenPort, Channel, Socket).\n\n%% @doc Forwards a message to the given PID or named process.\n%%      Logs a warning if a named process does not exist.\n-spec forward_msg(Process::pid() | atom(), Message::comm:message(), State::state()) -> ok.\nforward_msg(Process, Message, _State) ->\n    ?LOG_MESSAGE('rcv', Message, channel(_State)),\n    case is_pid(Process) of\n        true ->\n            % TODO: report error if process is not alive?\n            comm:send_local(Process, Message), ok;\n        false ->\n            case whereis(Process) of\n                undefined ->\n                    log:log(warn,\n                            \"[ CC ~p (~p) ] Cannot accept msg for unknown named\"\n                                \" process ~p: ~.0p~n\",\n                            [self(), pid_groups:my_pidname(), Process, Message]);\n                PID -> comm:send_local(PID, Message), ok\n            end\n    end.\n\n%% @doc message handler\n-spec on(message(), state()) -> state().\non({send, DestPid, Message, Options}, State) ->\n    case socket(State) of\n        notconnected ->\n            log:log(info, \"Connecting to ~.0p:~.0p\", [dest_ip(State), dest_port(State)]),\n            Socket = new_connection(dest_ip(State), dest_port(State),\n                                    local_listen_port(State),\n                                    channel(State)),\n            case Socket of\n                notconnected ->\n                    comm_server:report_send_error(Options,\n                                                  {dest_ip(State), dest_port(State), DestPid},\n                                                  Message, tcp_connect_failed),\n                    State;\n                _ ->\n                    State1 = set_socket(State, Socket),\n                    State2 = set_last_stat_report(State1, {0, 0, 0, 0}),\n                    send_or_bundle(DestPid, Message, Options, State2)\n            end;\n        _ -> send_or_bundle(DestPid, Message, Options, State)\n    end;\n\non({tcp, Socket, Data}, State) ->\n    handle_data(Socket, Data, State);\n\non({ssl, Socket, Data}, State) -> %% copy of tcp ...\n    handle_data(Socket, Data, State);\n\non({tcp_closed, Socket}, State) ->\n    log:log(info,\"[ CC ~p (~p) ] tcp closed\", [self(), pid_groups:my_pidname()]),\n    close_connection(Socket, State);\n\non({ssl_closed, Socket}, State) ->\n    log:log(info,\"[ CC ~p (~p) ] ssl closed\", [self(), pid_groups:my_pidname()]),\n    close_connection(Socket, State);\n\non({tcp_error, Socket, Reason}, State) ->\n    % example Reason: etimedout, ehostunreach\n    log:log(warn,\"[ CC ~p (~p) ] tcp error: ~p\", [self(), pid_groups:my_pidname(), Reason]),\n    %% may fail, when tcp just closed\n    _ = inet:setopts(Socket, [{active, once}]),\n    send_bundle_if_ready(State);\n\non({report_stats}, State) ->\n    %% re-trigger\n    msg_delay:send_trigger(10, {report_stats}),\n    NewState = report_stats(State),\n    send_bundle_if_ready(NewState);\n\n%% checks if the connection hasn't been used recently\non({check_idle}, State) ->\n    msg_delay:send_trigger(10, {check_idle}),\n    NewState = send_bundle_if_ready(State),\n\n    Timeout = config:read(tcp_idle_timeout),\n    TimeLastMsgSent = time_last_msg_seen(NewState),\n    case (notconnected =/= socket(NewState)) andalso\n        timer:now_diff(os:timestamp(), TimeLastMsgSent) div 1000 > Timeout of\n        true ->\n            %% we timed out\n            ?TRACE(\"Closing idle connection: ~p~n\", [NewState]),\n            %% TODO: check whether data was received on this socket?\n            %% (maybe a part of a huge message that takes longer\n            %% than the tcp_idle_timeout to receive?)\n            close_connection(socket(NewState), NewState);\n        _ ->\n            ?TRACE(\"Connection not idle~n\", []),\n            NewState\n            %% send_bundle_if_ready() is called in the\n            %% beginning of this on handler to make the\n            %% decision on tcp_idle_timeout on the newest\n            %% possible state NewState\n    end;\n\non({web_debug_info, Requestor}, State) ->\n    Now = os:timestamp(),\n    TimeLastMsgSent = time_last_msg_seen(State),\n    TimeLastMsgReceived = time_last_msg_received(State),\n    SecondsAgoSent = seconds_ago(Now, TimeLastMsgSent),\n    SecondsAgoReceived = seconds_ago(Now, TimeLastMsgReceived),\n    Runtime = timer:now_diff(Now, started(State)) / 1000000,\n    {SentPerS, ReceivedPerS} =\n        if Runtime =< 0 -> {\"n/a\", \"n/a\"};\n           true         -> {s_msg_count(State) / Runtime,\n                            r_msg_count(State) / Runtime}\n        end,\n    case socket(State) of\n        notconnected ->\n            MyAddress = MyPort = \"n/a\",\n            PeerAddress = PeerPort = \"n/a\",\n            RcvAgv = RcvCnt = RcvBytes = \"n/a\",\n            SendAgv = SendCnt = SendBytes = \"n/a\",\n            ok;\n        Socket ->\n            case sockname(Socket) of\n                {ok, {MyAddress, MyPort}} -> ok;\n                {error, _Reason1}          -> MyAddress = MyPort = \"n/a\"\n            end,\n            case peername(Socket) of\n                {ok, {PeerAddress, PeerPort}} -> ok;\n                {error, _Reason2}              -> PeerAddress = PeerPort = \"n/a\"\n            end,\n            case getstat(Socket, [recv_avg, recv_cnt, recv_oct,\n                                       send_avg, send_cnt, send_oct]) of\n                {ok, [{recv_avg, RcvAgv}, {recv_cnt, RcvCnt}, {recv_oct, RcvBytes},\n                      {send_avg, SendAgv}, {send_cnt, SendCnt}, {send_oct, SendBytes}]} -> ok;\n                {error, _Reason} ->\n                    RcvAgv = RcvCnt = RcvBytes = \"n/a\",\n                    SendAgv = SendCnt = SendBytes = \"n/a\"\n            end\n    end,\n    LastMsgsSent = [?PRETTY_PRINT_MSG(X) || X <- last_msg_sent(State)],\n    LastMsgsReceived = [?PRETTY_PRINT_MSG(X) || X <- last_msg_received(State)],\n    KeyValueList =\n        [\n         {\"status\",\n          webhelpers:safe_html_string(\"~p\", [status(State)])},\n         {\"my IP:\",\n          webhelpers:safe_html_string(\"~p\", [MyAddress])},\n         {\"my port\",\n          webhelpers:safe_html_string(\"~p\", [MyPort])},\n         {\"peer IP:\",\n          webhelpers:safe_html_string(\"~p\", [PeerAddress])},\n         {\"channel:\",\n          webhelpers:safe_html_string(\"~p\", [channel(State)])},\n         {\"peer port\",\n          webhelpers:safe_html_string(\"~p\", [PeerPort])},\n         {\"running since (s)\",\n          webhelpers:safe_html_string(\"~p\", [Runtime])},\n         {\"sent_tcp_messages\",\n          webhelpers:safe_html_string(\"~p\", [s_msg_count(State)])},\n         {\"sent_tcp_packets\",\n          webhelpers:safe_html_string(\"~p\", [SendCnt])},\n         {\"~ sent messages/s\",\n          webhelpers:safe_html_string(\"~p\", [SentPerS])},\n         {\"~ sent avg packet size\",\n          webhelpers:safe_html_string(\"~p\", [SendAgv])},\n         {\"sent total bytes\",\n          webhelpers:safe_html_string(\"~p\", [SendBytes])},\n         {\"recv_tcp_messages\",\n          webhelpers:safe_html_string(\"~p\", [r_msg_count(State)])},\n         {\"recv_tcp_packets\",\n          webhelpers:safe_html_string(\"~p\", [RcvCnt])},\n         {\"~ recv messages/s\",\n          webhelpers:safe_html_string(\"~p\", [ReceivedPerS])},\n         {\"~ recv avg packet size\",\n          webhelpers:safe_html_string(\"~p\", [RcvAgv])},\n         {\"recv total bytes\",\n          webhelpers:safe_html_string(\"~p\", [RcvBytes])},\n         {\"time last message sent\",\n          webhelpers:safe_html_string(\"~p sec ago\", [SecondsAgoSent])},\n         {\"time last message received\",\n          webhelpers:safe_html_string(\"~p sec ago\", [SecondsAgoReceived])},\n         {\"last message sent:\",\n          webhelpers:html_pre(\"~0p\", [LastMsgsSent])},\n         {\"last message received\",\n          webhelpers:html_pre(\"~0p\", [LastMsgsReceived])},\n         {\"session variables:\", \"\"},\n         {\"num sessions\",\n          webhelpers:safe_html_string(\"~p\", [session_count(State)])},\n         {\"sent_tcp_messages\",\n          webhelpers:safe_html_string(\"~p\", [s_msg_count_session(State)])},\n         {\"recv_tcp_messages\",\n          webhelpers:safe_html_string(\"~p\", [r_msg_count_session(State)])}\n        ],\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    send_bundle_if_ready(State);\n\non(UnknownMessage, State) ->\n    %% we want to count messages, so we need this default handler.\n    log:log(error,\"unknown message: ~.0p~n in Module: ~p and handler ~p~n in State ~.0p\",[UnknownMessage,?MODULE,on,State]),\n    send_bundle_if_ready(State).\n\n-spec report_stats(State::state()) -> state().\nreport_stats(State) ->\n    case socket(State) of\n        notconnected -> State;\n        Socket ->\n            case getstat(Socket, [recv_cnt, recv_oct,\n                                  send_cnt, send_oct]) of\n                {ok, [{recv_cnt, RcvCnt}, {recv_oct, RcvBytes},\n                      {send_cnt, SendCnt}, {send_oct, SendBytes}]} ->\n                    PrevStat = last_stat_report(State),\n                    NewStat = {RcvCnt, RcvBytes, SendCnt, SendBytes},\n                    case PrevStat of\n                        NewStat -> State;\n                        _ ->\n                            {PrevRcvCnt, PrevRcvBytes, PrevSendCnt, PrevSendBytes} = PrevStat,\n                            comm:send_local(\n                              comm_stats,\n                              {report_stat,\n                               overflow_aware_diff(RcvCnt, PrevRcvCnt),\n                               overflow_aware_diff(RcvBytes, PrevRcvBytes),\n                               overflow_aware_diff(SendCnt, PrevSendCnt),\n                               overflow_aware_diff(SendBytes, PrevSendBytes)}),\n                            set_last_stat_report(State, NewStat)\n                    end;\n                {error, _Reason} -> State\n            end\n    end.\n\n%% @doc Diff between A and B taking an overflow at 2^32 or 2^64 into account,\n%%      otherwise <tt>A - B</tt>. Fails if A &lt; B and B &lt;= 2^32 or 2^64.\n-spec overflow_aware_diff(number(), number()) -> number().\noverflow_aware_diff(A, B) when A >= B ->\n    A - B;\noverflow_aware_diff(A, B) when A < B andalso B < 4294967296 -> % 2^32\n    4294967296 - B + A;\noverflow_aware_diff(A, B) when A < B andalso B < 18446744073709551616 -> % 2^64\n    18446744073709551616 - B + A.\n\n% PRE: connected socket\n-spec send_or_bundle(DestPid::pid(), Message::comm:message(), Options::comm:send_options(), State::state()) -> state().\nsend_or_bundle(DestPid, Message, Options, State) ->\n    case msg_queue_len(State) of\n        0 ->\n            erlang:yield(), % give other processes a chance to enqueue more messages\n            {_, MQL} = process_info(self(), message_queue_len),\n            if MQL > 0 ->\n                   %% start message bundle for sending\n                   %% log:log(\"MQL ~p~n\", [MQL]),\n                   MaxBundle = erlang:max(200, MQL div 100),\n                   T1 = set_msg_queue(State, {[{DestPid, Message}], [Options]}),\n                   T2 = set_msg_queue_len(T1, 1),\n                   % note: need to set the bundle size equal to MQL\n                   % -> to process this 1 msg + MQL messages (see below)\n                   set_desired_bundle_size(T2, erlang:min(MQL, MaxBundle));\n               true ->\n                   send(DestPid, Message, Options, State)\n            end;\n        QL ->\n            {MsgQueue0, OptionQueue0} = msg_queue(State),\n            MQueue = [{DestPid, Message} | MsgQueue0],\n            OQueue = [Options | OptionQueue0],\n            % similar to send_bundle_if_ready/1 (keep in sync!)\n            DBS = desired_bundle_size(State),\n            MSBS = msgs_since_bundle_start(State),\n            % can check for QL instead of QL+1 here due to DBS init above\n            case (QL + MSBS) >= DBS of\n                true when DBS >= 100 -> % quick path without message_queue_len check\n                    %% log:log(\"Bundle Size: ~p~n\", [length(MQueue)]),\n                    send_msg_bundle(State, MQueue, OQueue, QL + 1);\n                true ->\n                    erlang:yield(), % give other processes a chance to enqueue more messages\n                    {_, MQL} = process_info(self(), message_queue_len),\n                    if MQL > 0 ->\n                           %% add to message bundle\n                           T1 = set_msg_queue(State, {MQueue, OQueue}),\n                           T2 = set_msg_queue_len(T1, QL + 1),\n                           set_desired_bundle_size(T2, erlang:min(100, DBS + MQL));\n                       true ->\n                           %% log:log(\"Bundle Size: ~p~n\", [length(MQueue)]),\n                           send_msg_bundle(State, MQueue, OQueue, QL + 1)\n                    end;\n                false ->\n                    %% add to message bundle\n                    T1 = set_msg_queue(State, {MQueue, OQueue}),\n                    set_msg_queue_len(T1, QL + 1)\n            end\n    end.\n\n-spec send(pid(), comm:message(), comm:send_options(), state())\n            -> state();\n          (?unpack_msg_bundle, [{pid(), comm:message()}], [comm:send_options()], state())\n            -> state().\nsend(Pid, Message, Options, State) ->\n    DeliverMsg = {?deliver, Pid, Message},\n    BinaryMessage = ?COMM_COMPRESS_MSG(DeliverMsg, State),\n    send_internal(Pid, Message, Options, BinaryMessage, State, 0, 0).\n\n-spec send_internal\n    (pid(), comm:message(), comm:send_options(), BinMsg::binary(), state(), Timeouts::non_neg_integer(), Errors::non_neg_integer())\n        -> state();\n    (?unpack_msg_bundle, [{pid(), comm:message()}], [comm:send_options()], BinMsg::binary(), state(), Timeouts::non_neg_integer(), Errors::non_neg_integer())\n        -> state().\nsend_internal(Pid, Message, Options, BinaryMessage, State, Timeouts, Errors) ->\n    Socket = socket(State),\n    case Socket of\n        notconnected ->\n            log:log(error, \"[ CC ~p (~p) ] couldn't send message (tcp connect failed)\",\n                    [self(), pid_groups:my_pidname()]),\n            set_socket(reset_msg_counters(State), notconnected);\n        Socket ->\n            ?LOG_MESSAGE_SOCK('send', Message, byte_size(BinaryMessage), channel(State)),\n            case ?COMM:send(Socket, BinaryMessage) of\n                ok ->\n                    ?TRACE(\"~.0p Sent message ~.0p~n\",\n                           [pid_groups:my_pidname(), Message]),\n                    SendMsgCount = s_msg_count_session(State),\n                    State2 = save_n_msgs(Message, fun set_last_msg_sent/2, State),\n                    %% only close in case of no_keep_alive if the\n                    %% connection was solely initiated for this send\n                    case SendMsgCount =< 1 andalso\n                        lists:member({no_keep_alive}, Options) of\n                        true -> close_connection(Socket, State2);\n                        _    -> State2\n                    end;\n                {error, closed} ->\n                    case Errors < 1 of\n                        true ->\n                            State2 = close_connection(Socket, State),\n                            State3 = set_socket(State2, reconnect(State2)),\n                            send_internal(Pid, Message, Options, BinaryMessage, State3, Timeouts, Errors + 1);\n                        _    ->\n                            Address = dest_ip(State),\n                            Port = dest_port(State),\n                            report_bundle_error(Options, {Address, Port, Pid}, Message,\n                                                socket_closed),\n                            log:log(warn,\"[ CC ~p (~p) ] sending closed connection\", [self(), pid_groups:my_pidname()]),\n                            close_connection(Socket, State)\n                    end;\n                {error, timeout} ->\n                    if  % retry 5 times\n                        Timeouts < 5 ->\n                            log:log(error,\"[ CC ~p (~p) ] couldn't send message (~.0p). retrying.\",\n                                    [self(), pid_groups:my_pidname(), timeout]),\n                            send_internal(Pid, Message, Options, BinaryMessage, State, Timeouts + 1, Errors);\n                        true ->\n                            log:log(error,\"[ CC ~p (~p) ] couldn't send message (~.0p). retried 5 times, now closing the connection.\",\n                                    [self(), pid_groups:my_pidname(), timeout]),\n                            Address = dest_ip(State),\n                            Port = dest_port(State),\n                            report_bundle_error(Options, {Address, Port, Pid}, Message,\n                                                socket_timeout),\n                            close_connection(Socket, State)\n                    end;\n                {error, Reason} ->\n                    case Errors < 1 of\n                        true ->\n                            State2 = close_connection(Socket, State),\n                            State3 = set_socket(State2, reconnect(State2)),\n                            send_internal(Pid, Message, Options, BinaryMessage, State3, Timeouts, Errors + 1);\n                        _    ->\n                            Address = dest_ip(State),\n                            Port = dest_port(State),\n                            report_bundle_error(Options, {Address, Port, Pid}, Message,\n                                                Reason),\n                            log:log(error,\"[ CC ~p (~p) ] couldn't send message (~.0p). closing connection\",\n                                    [self(), pid_groups:my_pidname(), Reason]),\n                            close_connection(Socket, State)\n                    end\n            end\n    end.\n\n-spec new_connection(inet:ip_address(), comm_server:tcp_port(),\n                     comm_server:tcp_port(), Channel::comm:channel() | unknown)\n        -> socket() | notconnected.\nnew_connection(Address, Port, MyPort, Channel) ->\n    new_connection(Address, Port, MyPort, Channel, 0).\n\n-spec new_connection(inet:ip_address(), comm_server:tcp_port(),\n                     comm_server:tcp_port(), Channel::comm:channel() | unknown,\n                     non_neg_integer())\n        -> socket() | notconnected.\nnew_connection(Address, Port, MyPort, Channel, Retries) ->\n    StrictOpts =\n        case config:read(ssl_mode) of\n            strict -> [{cacertfile, config:read(cacertfile)},\n                       {password, config:read(ssl_password)}];\n            normal -> []\n        end,\n    SSLOpts = case ?COMM of\n                  ssl -> [{certfile, config:read(certfile)},\n                          {keyfile, config:read(keyfile)},\n                          {secure_renegotiate, true}\n                         ];\n                  gen_tcp -> []\n              end,\n    case ?COMM:connect(Address, Port, [binary, {packet, 4}]\n                         ++ comm_server:tcp_options(Channel) ++ StrictOpts ++ SSLOpts,\n                         config:read(tcp_connect_timeout)) of\n        {ok, Socket} ->\n            % send end point data (the other node needs to know my listen port\n            % in order to have only a single connection to me)\n            case sockname(Socket) of\n                {ok, {MyAddress, _SocketPort}} ->\n                    case comm_server:get_local_address_port() of\n                        {undefined,_} ->\n                            comm_server:set_local_address(MyAddress, MyPort);\n                        _ -> ok\n                    end,\n                    Message = term_to_binary({endpoint, MyAddress, MyPort, Channel},\n                                             [{compressed, 2}, {minor_version, 1}]),\n                    _ = ?COMM:send(Socket, Message),\n                    Socket;\n                {error, Reason} ->\n                    % note: this should not occur since the socket was just created with 'ok'\n                    log:log(error,\"[ CC ~p (~p) ] reconnect because socket is ~.0p\",\n                            [self(), pid_groups:my_pidname(), Reason]),\n                    ?COMM:close(Socket),\n                    new_connection(Address, Port, MyPort, Channel, Retries + 1)\n            end;\n        {error, Reason} ->\n            log:log(info,\"[ CC ~p (~p) ] couldn't connect (~.0p)\",\n                    [self(), pid_groups:my_pidname(), Reason]),\n            case Retries >= 3 of\n                true -> notconnected;\n                _    -> timer:sleep(config:read(tcp_connect_timeout) * (Retries + 1)),\n                        new_connection(Address, Port, MyPort, Channel, Retries + 1)\n            end\n    end.\n\n-spec reconnect(state()) -> socket() | notconnected.\nreconnect(State) ->\n    Address = dest_ip(State),\n    Port = dest_port(State),\n    MyPort = local_listen_port(State),\n    Channel = channel(State),\n    new_connection(Address, Port, MyPort, Channel, 0).\n\n-spec close_connection(Socket::socket(), State::state()) -> state().\nclose_connection(Socket, State) ->\n    ?COMM:close(Socket),\n    case socket(State) of\n        Socket ->\n            % report stats out of the original schedule\n            % (these would otherwise be lost)\n            StateNew = report_stats(State),\n            set_socket(reset_msg_counters(StateNew), notconnected);\n        _ -> State\n    end.\n\n-spec send_bundle_if_ready(state()) -> state().\nsend_bundle_if_ready(InState) ->\n    QL = msg_queue_len(InState),\n    case QL of\n        0 -> InState;\n        _ ->\n            % similar to send_or_bundle/4 (keep in sync!)\n            State = inc_msgs_since_bundle_start(InState),\n            DBS = desired_bundle_size(State),\n            MSBS = msgs_since_bundle_start(State),\n            case (QL + MSBS) >= DBS of\n                true when DBS >= 100 -> % quick path without message_queue_len check\n                    %% log:log(\"Sending packet with ~p msgs~n\", [length(element(1, msg_queue(State)))]),\n                    {MQueue, OQueue} = msg_queue(State),\n                    send_msg_bundle(State, MQueue, OQueue, QL);\n                true ->\n                    erlang:yield(), % give other processes a chance to enqueue more messages\n                    {_, MQL} = process_info(self(), message_queue_len),\n                    if MQL > 0 ->\n                           set_desired_bundle_size(State, erlang:min(100, DBS + MQL));\n                       true ->\n                           %% log:log(\"Sending packet with ~p msgs~n\", [length(element(1, msg_queue(State)))]),\n                           {MQueue, OQueue} = msg_queue(State),\n                           send_msg_bundle(State, MQueue, OQueue, QL)\n                    end;\n                false -> State\n            end\n    end.\n\n-spec send_msg_bundle(state(),\n                      MQueue::[{DestPid::pid(), Message::comm:message()}],\n                      OQueue::[comm:send_options()], QL::pos_integer()) -> state().\nsend_msg_bundle(State, MQueue, OQueue, QL) ->\n    case socket(State) of\n        notconnected ->\n            % should not occur often - just in case a new MQueue, OQueue was given:\n            T1 = set_msg_queue(State, {MQueue, OQueue}),\n            set_msg_queue_len(T1, QL);\n        _ ->\n            T1State = send(?unpack_msg_bundle, MQueue, OQueue, State),\n            T2State = set_msg_queue(T1State, {[], []}),\n            T3State = set_msg_queue_len(T2State, 0),\n            _T4State = set_msgs_since_bundle_start(T3State,0)\n    end.\n\n-spec state_new(DestIP::inet:ip_address(), DestPort::comm_server:tcp_port(),\n                LocalListenPort::comm_server:tcp_port(), Channel::comm:channel(),\n                Socket::socket() | notconnected) -> state().\nstate_new(DestIP, DestPort, LocalListenPort, Channel, Socket) ->\n    {DestIP, DestPort, LocalListenPort, Channel, Socket,\n     _StartTime = os:timestamp(), _SentMsgCount = 0, _ReceivedMsgCount = 0,\n     _MsgQueue = {[], []}, _Len = 0,\n     _DesiredBundleSize = 0, _MsgsSinceBundleStart = 0,\n     _LastStatReport = {0, 0, 0, 0},\n     _TimeLastMsgSent = {0, 0, 0},\n     _TimeLastMsgReceived = {0, 0, 0},\n     _SentMsgCountSession = 0, _ReceivedMsgCountSession = 0,\n     _SessionCount = 0,\n     _LastMsgsSent = [], _LastMsgsReceived = []\n    }.\n\n-spec dest_ip(state()) -> inet:ip_address().\ndest_ip(State)                 -> element(1, State).\n\n-spec dest_port(state()) -> comm_server:tcp_port().\ndest_port(State)               -> element(2, State).\n\n-spec local_listen_port(state()) -> comm_server:tcp_port().\nlocal_listen_port(State)       -> element(3, State).\n\n-spec channel(state()) -> comm:channel().\nchannel(State)                 -> element(4, State).\n\n-spec socket(state()) -> socket() | notconnected.\nsocket(State)                  -> element(5, State).\n\n-spec set_socket(state(), socket() | notconnected) -> state().\nset_socket(State, Val)         -> setelement(5, State, Val).\n\n-spec started(state()) -> erlang_timestamp().\nstarted(State)                 -> element(6, State).\n\n-spec s_msg_count(state()) -> non_neg_integer().\ns_msg_count(State)             -> element(7, State).\n-spec inc_s_msg_count(state(), pos_integer()) -> state().\ninc_s_msg_count(State, N)      -> State2 = setelement(7, State, s_msg_count(State) + N),\n                                  inc_s_msg_count_session(State2, N).\n\n-spec r_msg_count(state()) -> non_neg_integer().\nr_msg_count(State)             -> element(8, State).\n-spec inc_r_msg_count(state(), pos_integer()) -> state().\ninc_r_msg_count(State, N)      -> State2 = setelement(8, State, r_msg_count(State) + N),\n                                  inc_r_msg_count_session(State2, N).\n\n-spec msg_queue(state()) -> msg_queue().\nmsg_queue(State)               -> element(9, State).\n-spec set_msg_queue(state(), msg_queue()) -> state().\nset_msg_queue(State, Val)      -> setelement(9, State, Val).\n\n-spec msg_queue_len(state()) -> non_neg_integer().\nmsg_queue_len(State)           -> element(10, State).\n-spec set_msg_queue_len(state(), non_neg_integer()) -> state().\nset_msg_queue_len(State, Val)  -> setelement(10, State, Val).\n\n-spec desired_bundle_size(state()) -> non_neg_integer().\ndesired_bundle_size(State)     -> element(11, State).\n-spec set_desired_bundle_size(state(), non_neg_integer()) -> state().\nset_desired_bundle_size(State, Val) -> setelement(11, State, Val).\n\n-spec msgs_since_bundle_start(state()) -> non_neg_integer().\nmsgs_since_bundle_start(State) -> element(12, State).\n-spec inc_msgs_since_bundle_start(state()) -> state().\ninc_msgs_since_bundle_start(State) ->\n    setelement(12, State, msgs_since_bundle_start(State) + 1).\n-spec set_msgs_since_bundle_start(state(), non_neg_integer()) -> state().\nset_msgs_since_bundle_start(State, Val) ->\n    setelement(12, State, Val).\n\n-spec last_stat_report(state()) -> stat_report().\nlast_stat_report(State)          -> element(13, State).\n-spec set_last_stat_report(state(), stat_report()) -> state().\nset_last_stat_report(State, Val) -> setelement(13, State, Val).\n\n-spec time_last_msg_seen(state()) -> erlang_timestamp().\ntime_last_msg_seen(State) -> element(14, State).\n-spec set_time_last_msg_seen(state()) -> state().\nset_time_last_msg_seen(State) -> setelement(14, State, os:timestamp()).\n\n-spec time_last_msg_received(state()) -> erlang_timestamp().\ntime_last_msg_received(State) -> element(15, State).\n-spec set_time_last_msg_received(state()) -> state().\nset_time_last_msg_received(State) -> setelement(15, State, os:timestamp()).\n\n-spec s_msg_count_session(state()) -> non_neg_integer().\ns_msg_count_session(State)         -> element(16, State).\n-spec inc_s_msg_count_session(state(), pos_integer()) -> state().\ninc_s_msg_count_session(State, N)  -> setelement(16, State, s_msg_count_session(State) + N).\n\n-spec r_msg_count_session(state()) -> non_neg_integer().\nr_msg_count_session(State)         -> element(17, State).\n-spec inc_r_msg_count_session(state(), pos_integer()) -> state().\ninc_r_msg_count_session(State, N)  -> setelement(17, State, r_msg_count_session(State) + N).\n\n-spec reset_msg_counters(state()) -> state().\nreset_msg_counters(State) -> State1 = setelement(16, State, 0),\n                             State2 = setelement(17, State1, 0),\n                             inc_session_count(State2).\n\n-spec session_count(state()) -> non_neg_integer().\nsession_count(State) -> element(18, State).\n-spec inc_session_count(state()) -> state().\ninc_session_count(State) -> setelement(18, State, session_count(State) + 1).\n\n-spec last_msg_sent(state()) -> [msg_or_tag()].\nlast_msg_sent(State) -> element(19, State).\n-spec set_last_msg_sent(state(), [msg_or_tag()]) -> state().\nset_last_msg_sent(State, MsgList) ->\n    MsgListLen = length(MsgList),\n    OldList = last_msg_sent(State),\n    NewList = case erlang:max(?NUM_KEEP - MsgListLen, 0) of\n                  0         -> MsgList;\n                  NumToKeep -> MsgList ++ lists:sublist(OldList, NumToKeep)\n              end,\n    State2 = set_time_last_msg_seen(State),\n    State3 = inc_s_msg_count(State2, MsgListLen),\n    setelement(19, State3, NewList).\n\n-spec last_msg_received(state()) -> [msg_or_tag()].\nlast_msg_received(State) -> element(20, State).\n-spec set_last_msg_received(state(), [msg_or_tag()]) -> state().\nset_last_msg_received(State, MsgList) ->\n    MsgListLen = length(MsgList),\n    OldList = last_msg_received(State),\n    NewList = case erlang:max(?NUM_KEEP - MsgListLen, 0) of\n                  0         -> MsgList;\n                  NumToKeep -> MsgList ++ lists:sublist(OldList, NumToKeep)\n              end,\n    State2 = set_time_last_msg_received(State),\n    State3 = inc_r_msg_count(State2, MsgListLen),\n    setelement(20, State3, NewList).\n\n-spec save_n_msgs(comm:message() | [{any(), comm:message()}],\n                  fun((state(), MsgList::[msg_or_tag()]) -> state()),\n                  state()) -> state().\nsave_n_msgs(Msg, SaveFun, State) when is_tuple(Msg) ->\n    save_n_msgs([{single, Msg}], SaveFun, State);\nsave_n_msgs(Msgs, SaveFun, State) when is_list(Msgs) ->\n    List = [get_msg_tag(M) || M <- lists:sublist(Msgs, ?NUM_KEEP)],\n    SaveFun(State, List).\n\n-spec get_msg_tag({any(), comm:message()}) -> msg_or_tag().\nget_msg_tag(Msg) ->\n    {_Pid, ActualMessage} = Msg,\n    ?GET_MSG_OR_TAG(ActualMessage).\n\n-spec seconds_ago(time_last_msg(), time_last_msg()) -> non_neg_integer().\nseconds_ago(Now, Time) ->\n    case Time of\n        {0,0,0} -> infinity;\n        _ -> timer:now_diff(Now, Time) div 1000000\n    end.\n\n\n-spec status(State::state()) -> notconnected | connected.\nstatus(State) ->\n     case socket(State) of\n         notconnected -> notconnected;\n         _            -> connected\n     end.\n\n-spec report_bundle_error\n        (comm:send_options(), {inet:ip_address(), comm_server:tcp_port(), pid()},\n         comm:message(), socket_closed | inet:posix()) -> ok;\n        ([comm:send_options()], {inet:ip_address(), comm_server:tcp_port(), ?unpack_msg_bundle},\n         [{pid(), comm:message()}], socket_closed | inet:posix()) -> ok.\nreport_bundle_error(Options, {Address, Port, ?unpack_msg_bundle}, Message, Reason) ->\n    zip_and_foldr(\n      fun (OptionsX, {DestPid, MessageX}) ->\n               comm_server:report_send_error(\n                 OptionsX, {Address, Port, DestPid}, MessageX, Reason)\n      end, Options, Message);\nreport_bundle_error(Options, {Address, Port, Pid}, Message, Reason) ->\n    comm_server:report_send_error(Options, {Address, Port, Pid}, Message, Reason).\n\n-spec zip_and_foldr(fun((E1, E2) -> any()), [E1], [E2]) -> ok.\nzip_and_foldr(_F, [], []) ->\n    ok;\nzip_and_foldr(F, [El1 | R1] , [El2 | R2]) ->\n    zip_and_foldr(F, R1, R2),\n    F(El1, El2).\n\nsockname(Socket) ->\n    case ?COMM of\n        ssl ->\n            ssl:sockname(Socket);\n        gen_tcp ->\n            inet:sockname(Socket)\n    end.\n\n-spec peername(Socket::socket()) -> term().\npeername(Socket) ->\n    case ?COMM of\n        ssl ->\n            ssl:peername(Socket);\n        gen_tcp ->\n            inet:peername(Socket)\n    end.\n\ngetstat(Socket, Options) ->\n    case ?COMM of\n        ssl ->\n            ssl:getstat(Socket, Options);\n        gen_tcp ->\n            inet:getstat(Socket, Options)\n    end.\n\n-spec setopts(Socket::socket(), list()) -> term().\nsetopts(Socket, Options) ->\n    case ?COMM of\n        ssl ->\n            ssl:setopts(Socket, Options);\n        gen_tcp ->\n            inet:setopts(Socket, Options)\n    end.\n\nhandle_data(Socket, Data, State) ->\n    DeliverMsg = ?COMM_DECOMPRESS_MSG(Data, State),\n    NewState =\n        case DeliverMsg of\n            {?deliver, ?unpack_msg_bundle, Message} ->\n                ?LOG_MESSAGE_SOCK('rcv', Data, byte_size(Data), channel(State)),\n                ?TRACE(\"Received message ~.0p\", [Message]),\n                lists:foldr(fun({DestPid, Msg}, _) -> forward_msg(DestPid, Msg, State) end,\n                            ok, Message),\n                %% may fail, when tcp just closed\n                _ = setopts(Socket, [{active, once}]),\n                save_n_msgs(Message, fun set_last_msg_received/2, State);\n            {?deliver, Process, Message} ->\n                ?TRACE(\"Received message ~.0p\", [Message]),\n                ?LOG_MESSAGE_SOCK('rcv', Data, byte_size(Data), channel(State)),\n                forward_msg(Process, Message, State),\n                %% may fail, when tcp just closed\n                _ = setopts(Socket, [{active, once}]),\n                save_n_msgs(Message, fun set_last_msg_received/2, State);\n            {user_close} ->\n                log:log(warn,\"[ CC ~p (~p) ] tcp user_close request\", [self(), pid_groups:my_pidname()]),\n                close_connection(Socket, State);\n            Unknown ->\n                log:log(warn,\"[ CC ~p (~p) ] unknown message ~.0p\", [self(), pid_groups:my_pidname(), Unknown]),\n                %% may fail, when tcp just closed\n                _ = setopts(Socket, [{active, once}]),\n                State\n        end,\n    New2State = set_time_last_msg_seen(NewState),\n    send_bundle_if_ready(New2State).\n"
  },
  {
    "path": "src/comm_layer/comm_logger.erl",
    "content": "% @copyright 2008-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Service for loggins messages.\n%% @version $Id$\n-module(comm_logger).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_server).\n\n-include(\"scalaris.hrl\").\n\n%% API\n-export([start_link/0]).\n\n-export([log/3, dump/0]).\n\n%% gen_server callbacks\n-export([init/1, handle_call/3, handle_cast/2, handle_info/2,\n         terminate/2, code_change/3]).\n\n-export_type([stat_tree/0]).\n\n-type stat_tree() :: gb_trees:tree(Tag::atom(), {Size::non_neg_integer(), Count::pos_integer()}).\n-record(state, {start    :: erlang_timestamp(),\n                received :: stat_tree(),\n                sent     :: stat_tree()}).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%--------------------------------------------------------------------\n%% Function: start_link() -> {ok,Pid} | ignore | {error,Error}\n%% Description: Starts the server\n%%--------------------------------------------------------------------\n-spec start_link() -> {ok,pid()} | ignore | {error, any()}.\nstart_link() ->\n    gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).\n\n%%--------------------------------------------------------------------\n%% Function: log(Tag, Size) -> ok\n%% Description: logs a message type with its size\n%%--------------------------------------------------------------------\n-spec log('send' | 'rcv', term(), non_neg_integer()) -> ok.\nlog(SendRcv, Tag, Size) ->\n    gen_server:cast(?MODULE, {log, SendRcv, Tag, Size}).\n\n%%--------------------------------------------------------------------\n%% Function: dump() -> {gb_tree:gb_trees(), {Date, Time}}\n%% Description: gets the logging state\n%%--------------------------------------------------------------------\n-spec dump() -> {Received::stat_tree(), Sent::stat_tree(), erlang_timestamp()}.\ndump() ->\n    gen_server:call(?MODULE, {dump}).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% gen_server callbacks\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%%--------------------------------------------------------------------\n%% Function: init(Args) -> {ok, State} |\n%%                         {ok, State, Timeout} |\n%%                         ignore               |\n%%                         {stop, Reason}\n%% Description: Initiates the server\n%%--------------------------------------------------------------------\n-spec init(list()) -> {ok, #state{}}.\ninit([]) ->\n    {ok, #state{start=os:timestamp(), received=gb_trees:empty(), sent=gb_trees:empty()}}.\n\n%%--------------------------------------------------------------------\n%% Function: %% handle_call(Request, From, State) -> {reply, Reply, State} |\n%%                                      {reply, Reply, State, Timeout} |\n%%                                      {noreply, State} |\n%%                                      {noreply, State, Timeout} |\n%%                                      {stop, Reason, Reply, State} |\n%%                                      {stop, Reason, State}\n%% Description: Handling call messages\n%%--------------------------------------------------------------------\n-spec handle_call({dump}, any(), #state{})\n        -> {reply, {Received::stat_tree(), Sent::stat_tree(), erlang_timestamp()}, #state{}}.\nhandle_call({dump}, _From, State) ->\n    Reply = {State#state.received, State#state.sent, State#state.start},\n    {reply, Reply, State};\nhandle_call(_Request, _From, State) ->\n    Reply = ok,\n    {reply, Reply, State}.\n\n%%--------------------------------------------------------------------\n%% Function: handle_cast(Msg, State) -> {noreply, State} |\n%%                                      {noreply, State, Timeout} |\n%%                                      {stop, Reason, State}\n%% Description: Handling cast messages\n%%--------------------------------------------------------------------\n-spec handle_cast({log, 'send' | 'rcv', atom(), non_neg_integer()}, #state{})\n        -> {noreply, #state{}}.\nhandle_cast({log, 'rcv', Tag, Size}, State) ->\n    case gb_trees:lookup(Tag, State#state.received) of\n        none ->\n            {noreply, State#state{received=gb_trees:insert(Tag, {Size, 1}, State#state.received)}};\n        {value, {OldSize, OldCount}} ->\n            {noreply, State#state{received=gb_trees:update(Tag, {Size + OldSize, OldCount + 1}, State#state.received)}}\n    end;\nhandle_cast({log, 'send', Tag, Size}, State) ->\n    case gb_trees:lookup(Tag, State#state.sent) of\n        none ->\n            {noreply, State#state{sent=gb_trees:insert(Tag, {Size, 1}, State#state.sent)}};\n        {value, {OldSize, OldCount}} ->\n            {noreply, State#state{sent=gb_trees:update(Tag, {Size + OldSize, OldCount + 1}, State#state.sent)}}\n    end;\nhandle_cast(_Msg, State) ->\n    {noreply, State}.\n\n%%--------------------------------------------------------------------\n%% Function: handle_info(Info, State) -> {noreply, State} |\n%%                                       {noreply, State, Timeout} |\n%%                                       {stop, Reason, State}\n%% Description: Handling all non call/cast messages\n%%--------------------------------------------------------------------\n-spec handle_info(any(), #state{}) -> {noreply, #state{}}.\nhandle_info(_Info, State) ->\n    {noreply, State}.\n\n%%--------------------------------------------------------------------\n%% Function: terminate(Reason, State) -> void()\n%% Description: This function is called by a gen_server when it is about to\n%% terminate. It should be the opposite of Module:init/1 and do any necessary\n%% cleaning up. When it returns, the gen_server terminates with Reason.\n%% The return value is ignored.\n%%--------------------------------------------------------------------\n-spec terminate(any(), #state{}) -> ok.\nterminate(_Reason, _State) ->\n    ok.\n\n%%--------------------------------------------------------------------\n%% Func: code_change(OldVsn, State, Extra) -> {ok, NewState}\n%% Description: Convert process state when code is changed\n%%--------------------------------------------------------------------\n-spec code_change(any(), #state{}, any()) -> {ok, #state{}}.\ncode_change(_OldVsn, State, _Extra) ->\n    {ok, State}.\n\n%%--------------------------------------------------------------------\n%%% Internal functions\n%%--------------------------------------------------------------------\n"
  },
  {
    "path": "src/comm_layer/comm_server.erl",
    "content": "% @copyright 2008-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @doc CommLayer: Management of comm_connection processes,\n%%      generic functions to send messages.  Distinguishes on runtime\n%%      whether the destination is in the same Erlang virtual machine\n%%      (use ! for sending) or on a remote site (send through comm_connection).\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Florian Schintke <schintke@zib.de>\n%% @version $Id$\n-module(comm_server).\n-author('schuett@zib.de').\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n-export_type([tcp_port/0]).\n\n-export([send/3, this/0, is_valid/1, is_local/1, make_local/1,\n         get_ip/1, get_port/1, report_send_error/4]).\n\n-export([start_link/1, init/1, on/2]).\n-export([tcp_options/1]).\n-export([unregister_connection/2, create_connection/4,\n         set_local_address/2, get_local_address_port/0]).\n\n-include(\"gen_component.hrl\").\n\n-type tcp_port() :: 0..65535.\n-type message() ::\n    {create_connection, Address::inet:ip_address(), Port::tcp_port(),\n     Socket::inet:socket() | ssl:sslsocket(), Channel::comm:channel(), Client::pid()} |\n    {send, Address::inet:ip_address(), Port::tcp_port(), Pid::pid(), Message::comm:message()} |\n    {unregister_conn, Address::inet:ip_address(), Port::tcp_port(), Client::pid()} |\n    {set_local_address, Address::inet:ip_address(), Port::tcp_port(), Client::pid()}.\n\n-type process_id() ::\n        {inet:ip_address(), tcp_port(), comm:erl_local_pid_plain()}.\n\n%% @doc send message via tcp, if target is not in same Erlang VM.\n-spec send(process_id(), comm:message(), comm:send_options()) -> ok.\nsend({{_IP1, _IP2, _IP3, _IP4} = TargetIP, TargetPort, TargetPid} = Target,\n     Message, Options) ->\n    % integrated is_local/1 and make_local/1:\n    {MyIP, MyPort} = get_local_address_port(),\n    if MyIP =:= TargetIP andalso MyPort =:= TargetPort andalso is_pid(TargetPid) ->\n           % local process identified by PID\n           case erlang:process_info(TargetPid, priority) of\n               {priority, low} -> % about to be killed\n                   report_send_error(Options, Target, Message,\n                                     local_target_not_alive);\n               undefined -> % process is not alive\n                   report_send_error(Options, Target, Message,\n                                     local_target_not_alive);\n               _ ->\n                   %% minor gap of error reporting, if PID\n                   %% dies at this moment, but better than a\n                   %% false positive reporting when first\n                   %% sending and then checking, if message\n                   %% leads to process termination (as in the\n                   %% RPCs of the Java binding)\n                   TargetPid ! Message, ok\n           end;\n       MyIP =:= TargetIP andalso MyPort =:= TargetPort andalso is_atom(TargetPid) ->\n           % named local process\n           case whereis(TargetPid) of\n               undefined ->\n                   log:log(warn,\n                           \"[ CC ] Cannot locally send msg to unknown named\"\n                               \" process ~p: ~.0p~n\", [TargetPid, Message]),\n                   report_send_error(Options, Target, Message, unknown_named_process);\n               PID ->\n                   case erlang:process_info(PID, priority) of\n                       {priority, low} -> % about to be killed\n                           report_send_error(Options, Target, Message,\n                                             local_target_not_alive);\n                       undefined ->\n                           report_send_error(Options, Target, Message, local_target_not_alive);\n                       _ ->\n                           % minor gap of error reporting as above\n                           TargetPid ! Message, ok\n                   end\n           end;\n       true ->\n           ?LOG_MESSAGE('send', Message, proplists:get_value(channel, Options, main)),\n           ?MODULE ! {send, TargetIP, TargetPort, TargetPid, Message, Options}\n    end.\n\n%% @doc returns process descriptor for the calling process\n-spec this() -> process_id().\nthis() ->\n    %% Note: We had caching enabled here, but the eshell takes over\n    %% the process dictionary to a new pid in case of failures, so we\n    %% got outdated pid info here.\n    %% case erlang:get(comm_this) of\n    %%    undefined ->\n    {LocalIP, LocalPort} = get_local_address_port(),\n    _This1 = {LocalIP, LocalPort, self()}\n    %% , case LocalIP of\n    %%     undefined -> ok;\n    %%     _         -> erlang:put(comm_this, This1)\n    %% end,\n    %% This1;\n    %%     This -> This\n    %% end\n    .\n\n-spec is_valid(process_id() | any()) -> boolean().\nis_valid({{_IP1, _IP2, _IP3, _IP4} = _IP, _Port, _Pid}) -> true;\nis_valid(_) -> false.\n\n-spec is_local(process_id()) -> boolean().\nis_local({IP, Port, _Pid}) ->\n    {MyIP, MyPort} = get_local_address_port(),\n    IP =:= MyIP andalso Port =:= MyPort.\n\n-spec make_local(process_id()) -> comm:erl_local_pid_plain().\nmake_local({_IP, _Port, Pid}) -> Pid.\n\n%% @doc Gets the IP address of the given process id.\n-spec get_ip(process_id()) -> inet:ip_address().\nget_ip({IP, _Port, _Pid}) -> IP.\n\n%% @doc Gets the port of the given process id.\n-spec get_port(process_id()) -> tcp_port().\nget_port({_IP, Port, _Pid}) -> Port.\n\n-spec report_send_error(comm:send_options(), process_id(), comm:message(), atom()) -> ok.\nreport_send_error(Options, Target, Message, Reason) ->\n    case lists:keyfind(shepherd, 1, Options) of\n        false ->\n            case lists:member({?quiet}, Options) of\n                false ->\n                    log:log(warn, \"~p (name: ~.0p) Send to ~.0p failed, drop message ~.0p due to ~p\",\n                            [self(), pid_groups:my_pidname(), Target, Message, Reason]);\n                _ -> ok\n            end,\n            ok;\n        {shepherd, ShepherdPid} ->\n            comm:send_local(ShepherdPid, {send_error, Target, Message, Reason})\n    end,\n    ok.\n\n%% be startable via supervisor, use gen_component\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(CommLayerGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2,\n                             [],\n                             [ {erlang_register, ?MODULE},\n                               {pid_groups_join_as, CommLayerGroup, ?MODULE},\n                               {spawn_opts, [{fullsweep_after, 0},\n                                             {min_heap_size, 131071}]},\n                               {wait_for_init} %% uses protected ets table\n                             ]).\n\n%% @doc initialize: return initial state.\n-spec init([]) -> null.\ninit([]) ->\n    _ = ets:new(?MODULE, [set, protected, named_table]),\n    _State = null.\n\n%% @doc message handler\n-spec on(message(), State::null) -> null.\non({create_connection, Address, Port, Socket, Channel, Client}, State) ->\n    % helper for comm_[tcp|ssl]_acceptor as we need to synchronise the creation of\n    % connections in order to prevent multiple connections to/from a single IP\n    {Channel, Dir} = case Channel of\n                         main -> {main, 'rcv'};\n                         prio -> {prio, 'both'}\n                     end,\n    ConnPid = get_connection(Address, Port, Socket, Channel, Dir),\n    Client ! {create_connection_done, ConnPid},\n    State;\n\non({send, Address, Port, Pid, Message, Options}, State) ->\n    case lists:keytake(channel, 1, Options) of\n        false -> Options1 = Options, Channel = main, Dir = 'send';\n        {value, {channel, Channel = main}, Options1} -> Dir = 'send';\n        {value, {channel, Channel = prio}, Options1} -> Dir = 'both'\n    end,\n    ConnPid = get_connection(Address, Port, notconnected, Channel, Dir),\n    ConnPid ! {send, Pid, Message, Options1},\n    State;\n\non({unregister_conn, Address, Port, Client}, State) ->\n    erlang:erase({Address, Port}),\n    Client ! {unregister_conn_done},\n    State;\n\non({set_local_address, Address, Port, Client}, State) ->\n    ets:insert(?MODULE, {local_address_port, {Address, Port}}),\n    Client ! {set_local_address_done},\n    State;\n\non({get_no_of_ch, SourcePid}, State) ->\n    Dict = get(),\n    Channels = [X || X = {{_Addr, _Port, Ch, _Dir}, _Pid} <- Dict,\n                     Ch =:= main orelse Ch =:= prio],\n    comm:send(SourcePid, {get_no_of_ch_response, comm:this(), length(Channels)}),\n    State.\n\n-spec tcp_options(Channel::comm:channel()) -> [{term(), term()}].\ntcp_options(Channel) ->\n    TcpSendTimeout = case Channel of\n                         prio -> config:read(tcp_send_timeout);\n                         main -> infinity\n                     end,\n    [{active, once},\n     {nodelay, true},\n     {keepalive, true},\n     {reuseaddr, true},\n     {send_timeout, TcpSendTimeout}].\n\n%% @doc Synchronous call to create (or get) a connection for the given Address+Port using Socket.\n-spec create_connection(Address::inet:ip_address(), Port::tcp_port(),\n                        Socket::inet:socket() | ssl:sslsocket(), Channel::comm:channel()) -> pid().\ncreate_connection(Address, Port, Socket, Channel) ->\n    ?MODULE ! {create_connection, Address, Port, Socket, Channel, self()},\n    receive {create_connection_done, ConnPid} -> ConnPid end.\n\n%% @doc Synchronous call to de-register a connection with the comm server.\n-spec unregister_connection(inet:ip_address(), tcp_port()) -> ok.\nunregister_connection(Adress, Port) ->\n    ?MODULE ! {unregister_conn, Adress, Port, self()},\n    receive {unregister_conn_done} -> ok end.\n\n-spec set_local_address(inet:ip_address() | undefined, tcp_port()) -> ok.\nset_local_address(Address, Port) ->\n    ?MODULE ! {set_local_address, Address, Port, self()},\n    receive {set_local_address_done} -> ok end.\n\n%% @doc returns the local ip address and port\n-spec(get_local_address_port() -> {inet:ip_address(), tcp_port()}\n                                      | {undefined, tcp_port()}).\nget_local_address_port() ->\n    case erlang:get(local_address_port) of\n        undefined ->\n            % ets:lookup will throw if the table does not exist yet\n            try ets:lookup(?MODULE, local_address_port) of\n                [{local_address_port, Value = {undefined, _MyPort}}] ->\n                    Value;\n                [{local_address_port, Value}] ->\n                    erlang:put(local_address_port, Value),\n                    Value;\n                [] ->\n                    {undefined, 0}\n            catch\n                error:_ -> {undefined, 0}\n            end;\n        Value -> Value\n    end.\n\n%% @doc Gets or creates a connection for the given Socket or address/port.\n%%      Only a single connection to any IP+Port combination is created.\n%%      Socket is the initial socket when a connection needs to be created.\n-spec get_connection(Address::inet:ip_address(), Port::tcp_port(),\n                     Socket::inet:socket() | notconnected,\n                     Channel::comm:channel(), Dir::'rcv' | 'send' | 'both') -> pid().\nget_connection(Address, Port, Socket, Channel, Dir) ->\n    case erlang:get({Address, Port, Channel, Dir}) of\n        undefined ->\n            %% start Erlang process responsible for the connection\n            {ok, ConnPid} = comm_connection:start_link(\n                              pid_groups:my_groupname(), Address, Port, Socket, Channel, Dir),\n            erlang:put({Address, Port, Channel, Dir}, ConnPid),\n            ok;\n        ConnPid -> ok\n    end,\n    ConnPid.\n"
  },
  {
    "path": "src/comm_layer/comm_ssl_acceptor.erl",
    "content": "% @copyright 2017-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc SSL Acceptor.\n%%\n%%      This module accepts new ssl connections and starts corresponding\n%%      comm_connection processes.\n%% @version $Id$\n-module(comm_ssl_acceptor).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([start_link/1, init/2, check_config/0]).\n\n-include(\"gen_component.hrl\").\n-include(\"scalaris.hrl\").\n\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(GroupName) ->\n    Pid = spawn_link(?MODULE, init, [self(), GroupName]),\n    receive\n        {started, Pid} -> {ok, Pid}\n    end.\n\n-spec init(pid(), pid_groups:groupname()) -> any().\ninit(Supervisor, GroupName) ->\n    Socket =\n        try\n            erlang:register(comm_layer_acceptor, self()),\n            pid_groups:join_as(GroupName, comm_acceptor),\n\n            IP = case config:read(listen_ip) of\n                     undefined -> abort;\n                     X         -> X\n                 end,\n            Port = config:read(port),\n\n            log:log(info,\"[ CC ] listening on ~p:~p\", [IP, Port]),\n            io:format(\"[ CC ] listening on ~p:~p~n\", [IP, Port]),\n            util:if_verbose(\"Listening on ~p:~p.~n\", [IP, Port]),\n\n            LS = open_listen_port(Port, IP),\n            comm_server:set_local_address(IP, Port),\n            io:format(\"this() == ~w~n\", [{IP, Port}]), %\n            LS\n        catch\n            % If init throws up, send 'started' to the supervisor but exit.\n            % The supervisor will try to restart the process as it is watching\n            % this PID.\n            ?CATCH_CLAUSE_WITH_STACKTRACE(Level, Reason, Stacktrace)\n                log:log(error,\"Error: exception ~p:~p in ~p:init/2:  ~.0p\",\n                        [Level, Reason, ?MODULE, Stacktrace]),\n                erlang:Level(Reason)\n        after\n            Supervisor ! {started, self()}\n        end,\n    server(Socket).\n\nserver(LS) ->\n    io:format(\"server(LS)~n\", []),\n    case ssl:transport_accept(LS) of\n        {ok, S} ->\n            io:format(\"ssl:transport_accept(LS)~n\", []),\n            case ssl:ssl_accept(S) of\n                {error, Reason} ->\n                    io:format(\"ssl:ssl_accept(S): ~p~n\", [Reason]),\n                    foo;\n                ok ->\n                    io:format(\"ssl:ssl_accept(S)~n\", []),\n                    %% receive first message on the channel (generated by\n                    %% comm_connection to get the listen port of the other side in order\n                    %% to use the same connection for sending messages)\n                    receive\n                        {ssl, S, Msg} ->\n                            {endpoint, Address, Port, Channel} = binary_to_term(Msg),\n                            io:format(\"~p~n\", [{endpoint, Address, Port, Channel}]),\n                            %% auto determine remote address, if not sent correctly\n                            NewAddress =\n                                if Address =:= {0,0,0,0}\n                                   orelse Address =:= {127,0,0,1} ->\n                                        case ssl:peername(S) of\n                                            {ok, {PeerAddress, _Port}} -> PeerAddress;\n                                            {error, _Why} -> Address\n                                        end;\n                                   true -> Address\n                                end,\n                            ConnPid =\n                                comm_server:create_connection(NewAddress, Port, S, Channel),\n                            %% note: need to set controlling process from here as we created the socket\n                            _ = ssl:controlling_process(S, ConnPid),\n                            _ = ssl:setopts(S, comm_server:tcp_options(Channel)),\n                            ok;\n                        X ->\n                            io:format(\"ssl:ssl_accept(S)~p~n\", [X])\n                    end\n            end;\n        Other ->\n            log:log(warn,\"[ CC ] unknown message ~p\", [Other])\n    end,\n    server(LS).\n\n-spec open_listen_port(comm_server:tcp_port(), IP::inet:ip_address()) -> inet:socket() | abort.\nopen_listen_port(Port, IP) ->\n    StrictOpts =\n        case config:read(ssl_mode) of\n            strict -> [{cacertfile, config:read(cacertfile)},\n                       {password, config:read(ssl_password)},\n                       {verify, verify_peer}];\n            normal -> []\n        end,\n    case ssl:listen(Port, [\n                           {certfile, config:read(certfile)},\n                           {keyfile, config:read(keyfile)},\n                           {secure_renegotiate, true},\n                           binary,\n                           {packet, 4},\n                           {ip, IP},\n                           {backlog, 128}\n                          ]\n                    ++ StrictOpts ++ comm_server:tcp_options(main)) of\n        {ok, ListenSocket} ->\n            log:log(info,\"[ CC ] listening on ~p:~p\", [IP, Port]),\n            ListenSocket;\n        {error, Reason} ->\n            log:log(error,\"[ CC ] can't listen on ~p: ~p\", [Port, Reason]),\n            abort\n    end.\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_in(comm_backend, [ssl, gen_tcp]) and\n    config:cfg_exists(certfile) and\n    config:cfg_exists(keyfile) and\n    config:cfg_is_port(port) and\n    config:cfg_is_ip(listen_ip, true).\n\n\n%% https://gist.github.com/mtigas/952344\n"
  },
  {
    "path": "src/comm_layer/comm_stats.erl",
    "content": "% @copyright 2012-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @doc: CommLayer: statistics gathering.\n%% @author Nico Kruber <kruber@zib.de>\n%% @version $Id$\n-module(comm_stats).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n-export([start_link/1, init/1, on/2]).\n-export([get_stats/0, get_no_of_ch/0]).\n\n-include(\"gen_component.hrl\").\n\n-type state() :: {StartTime::erlang_timestamp(), RcvCnt::non_neg_integer(),\n                  RcvBytes::non_neg_integer(), SendCnt::non_neg_integer(),\n                  SendBytes::non_neg_integer()}.\n\n-type message() ::\n    {report_stat, RcvCnt::non_neg_integer(), RcvBytes::non_neg_integer(),\n     SendCnt::non_neg_integer(), SendBytes::non_neg_integer()}.\n\n-spec get_stats() -> {RcvCnt::non_neg_integer(), RcvBytes::non_neg_integer(),\n     SendCnt::non_neg_integer(), SendBytes::non_neg_integer()}.\nget_stats() ->\n    comm:send_local(?MODULE, {get_stats, self()}),\n    receive {get_stats_response, RcvCnt, RcvBytes, SendCnt, SendBytes} ->\n                {RcvCnt, RcvBytes, SendCnt, SendBytes}\n    end.\n\n%% @doc Get number of channels for the current VM.\n%%      The answer is collected synchronously, use only for debugging.\n-spec get_no_of_ch() -> {no_of_channels, comm:mypid(), pos_integer()}.\nget_no_of_ch() ->\n    Pid = pid_groups:find_a(comm_server),\n    comm:send_local(Pid, {get_no_of_ch, comm:this()}),\n    receive\n        ?SCALARIS_RECV({get_no_of_ch_response, CommServer, NoOfCh}, %% ->\n                       {no_of_channels, CommServer, NoOfCh})\n    end.\n\n\n%% be startable via supervisor, use gen_component\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(CommLayerGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2,\n                             [],\n                             [ {erlang_register, ?MODULE},\n                               {pid_groups_join_as, CommLayerGroup, ?MODULE}\n                             ]).\n\n%% @doc initialize: return initial state.\n-spec init([]) -> state().\ninit([]) ->\n    { os:timestamp(), 0, 0, 0, 0}.\n\n%% @doc message handler\n-spec on(message(), State::state()) -> state().\non({report_stat, RcvCnt, RcvBytes, SendCnt, SendBytes},\n   {StartTime, PrevRcvCnt, PrevRcvBytes, PrevSendCnt, PrevSendBytes})\n  when RcvCnt >= 0 andalso RcvBytes >= 0 andalso\n       SendCnt >= 0 andalso SendBytes >= 0 ->\n    {StartTime,\n     PrevRcvCnt + RcvCnt, PrevRcvBytes + RcvBytes,\n     PrevSendCnt + SendCnt, PrevSendBytes + SendBytes};\n\non({get_stats, Pid},\n   State = {_StartTime, RcvCnt, RcvBytes, SendCnt, SendBytes}) ->\n    comm:send_local(Pid, {get_stats_response, RcvCnt, RcvBytes, SendCnt, SendBytes}),\n    State;\n\non({web_debug_info, Requestor}, State = {StartTime, RcvCnt, RcvBytes, SendCnt, SendBytes}) ->\n    Now = os:timestamp(),\n    Runtime = timer:now_diff(Now, StartTime) / 1000000,\n    {SendCntPerS, SendBytesPerS, RcvCntPerS, RcvBytesPerS} =\n        if Runtime =< 0 -> {\"n/a\", \"n/a\", \"n/a\", \"n/a\"};\n           true         -> {SendCnt / Runtime, SendBytes / Runtime,\n                            RcvCnt / Runtime, RcvBytes / Runtime}\n        end,\n    KeyValueList =\n        [\n         {\"sent_tcp_packets\",\n          webhelpers:safe_html_string(\"~p\", [SendCnt])},\n         {\"~ sent messages/s\",\n          webhelpers:safe_html_string(\"~p\", [SendCntPerS])},\n         {\"sent total bytes\",\n          webhelpers:safe_html_string(\"~p\", [SendBytes])},\n         {\"~ sent bytes/s\",\n          webhelpers:safe_html_string(\"~p\", [SendBytesPerS])},\n         {\"recv_tcp_packets\",\n          webhelpers:safe_html_string(\"~p\", [RcvCnt])},\n         {\"~ recv messages/s\",\n          webhelpers:safe_html_string(\"~p\", [RcvCntPerS])},\n         {\"recv total bytes\",\n          webhelpers:safe_html_string(\"~p\", [RcvBytes])},\n         {\"~ recv bytes/s\",\n          webhelpers:safe_html_string(\"~p\", [RcvBytesPerS])}\n        ],\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    State.\n"
  },
  {
    "path": "src/comm_layer/comm_tcp_acceptor.erl",
    "content": "% @copyright 2008-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Acceptor.\n%%\n%%      This module accepts new connections and starts corresponding\n%%      comm_connection processes.\n%% @version $Id$\n-module(comm_tcp_acceptor).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([start_link/1, init/2, check_config/0]).\n\n-include(\"gen_component.hrl\").\n-include(\"scalaris.hrl\").\n\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(GroupName) ->\n    Pid = spawn_link(?MODULE, init, [self(), GroupName]),\n    receive\n        {started, Pid} -> {ok, Pid}\n    end.\n\n-spec init(pid(), pid_groups:groupname()) -> any().\ninit(Supervisor, GroupName) ->\n    Socket =\n        try\n            erlang:register(comm_layer_acceptor, self()),\n            pid_groups:join_as(GroupName, comm_acceptor),\n\n            IP = case config:read(listen_ip) of\n                     undefined -> first_ip();\n                     X         -> X\n                 end,\n            Port = config:read(port),\n\n            log:log(info,\"[ CC ] listening on ~p:~p\", [IP, Port]),\n            util:if_verbose(\"Listening on ~p:~p.~n\", [IP, Port]),\n\n            LS = open_listen_port(Port, IP),\n            {ok, {_LocalAddress, LocalPort}} = inet:sockname(LS),\n            comm_server:set_local_address(undefined, LocalPort),\n            %io:format(\"this() == ~w~n\", [{LocalAddress, LocalPort}]),\n            LS\n        catch\n            % If init throws up, send 'started' to the supervisor but exit.\n            % The supervisor will try to restart the process as it is watching\n            % this PID.\n            ?CATCH_CLAUSE_WITH_STACKTRACE(Level, Reason, Stacktrace)\n                log:log(error,\"Error: exception ~p:~p in ~p:init/2:  ~.0p\",\n                        [Level, Reason, ?MODULE, Stacktrace]),\n                erlang:Level(Reason)\n        after\n            Supervisor ! {started, self()}\n        end,\n    server(Socket).\n\nserver(LS) ->\n    case gen_tcp:accept(LS) of\n        {ok, S} ->\n            case comm_server:get_local_address_port() of\n                {undefined, LocalPort} ->\n                    {ok, {MyIP, _LocalPort}} = inet:sockname(S),\n                    comm_server:set_local_address(MyIP, LocalPort);\n                _ -> ok\n            end,\n            % receive first message on the channel (generated by\n            % comm_connection to get the listen port of the other side in order\n            % to use the same connection for sending messages)\n            receive\n                {tcp, S, Msg} ->\n                    {endpoint, Address, Port, Channel} = binary_to_term(Msg),\n                    % auto determine remote address, if not sent correctly\n                    NewAddress =\n                        if Address =:= {0,0,0,0}\n                           orelse Address =:= {127,0,0,1} ->\n                                case inet:peername(S) of\n                                    {ok, {PeerAddress, _Port}} -> PeerAddress;\n                                    {error, _Why} -> Address\n                                end;\n                           true -> Address\n                        end,\n                    ConnPid =\n                        comm_server:create_connection(NewAddress, Port, S, Channel),\n                    % note: need to set controlling process from here as we created the socket\n                    _ = gen_tcp:controlling_process(S, ConnPid),\n                    _ = inet:setopts(S, comm_server:tcp_options(Channel)),\n                    ok\n            end;\n        Other ->\n            log:log(warn,\"[ CC ] unknown message ~p\", [Other])\n    end,\n    server(LS).\n\n-spec open_listen_port(comm_server:tcp_port() | [comm_server:tcp_port()] | {From::comm_server:tcp_port(), To::comm_server:tcp_port()},\n                       IP::inet:ip_address()) -> inet:socket() | abort.\nopen_listen_port({From, To}, IP) ->\n    open_listen_port(lists:seq(From, To), IP);\nopen_listen_port([Port | Rest], IP) when is_integer(Port) andalso Port >= 0 andalso Port =< 65535 ->\n    case gen_tcp:listen(Port, [binary, {packet, 4}, {ip, IP}, {backlog, 128}]\n                        ++ comm_server:tcp_options(main)) of\n        {ok, Socket} ->\n            log:log(info,\"[ CC ] listening on ~p:~p\", [IP, Port]),\n            Socket;\n        {error, Reason} ->\n            log:log(error,\"[ CC ] can't listen on ~p: ~p\", [Port, Reason]),\n            open_listen_port(Rest, IP)\n    end;\nopen_listen_port([], _) ->\n    abort;\nopen_listen_port(Port, IP) ->\n    open_listen_port([Port], IP).\n\n-include_lib(\"kernel/include/inet.hrl\").\n\nfirst_ip() ->\n    {ok, Hostname} = inet:gethostname(),\n    {ok, HostEntry} = inet:gethostbyname(Hostname),\n    erlang:hd(HostEntry#hostent.h_addr_list).\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_in(comm_backend, [ssl, gen_tcp]) and\n    config:cfg_is_port(port) and\n    config:cfg_is_ip(listen_ip, true).\n"
  },
  {
    "path": "src/config.erl",
    "content": "%  @copyright 2007-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Config file parser for scalaris.\n%% @end\n%% @version $Id$\n-module(config).\n-author('schuett@zib.de').\n-vsn('$Id$').\n-include(\"scalaris.hrl\").\n\n-export([\n         read/1, write/2,\n         init/1,\n         check_config/0,\n\n         cfg_exists/1, cfg_is_atom/1, cfg_is_bool/1, cfg_is_mypid/1,\n         cfg_is_ip/1, cfg_is_ip/2, cfg_is_port/1,\n         cfg_is_integer/1, cfg_is_float/1, cfg_is_number/1,\n         cfg_is_tuple/2, cfg_is_tuple/4, cfg_is_list/1, cfg_is_list/3, cfg_is_string/1,\n         cfg_is_in_range/3, cfg_is_greater_than/2, cfg_is_greater_than_equal/2,\n         cfg_is_less_than/2, cfg_is_less_than_equal/2, cfg_is_in/2, cfg_is_module/1,\n         cfg_test_and_error/3\n        ]).\n\n%% public functions\n\n%% @doc Reads a config parameter. If it is not found, the application's\n%%      environment is checked or failed is returned. The result will be\n%%      cached in the config.\n-spec read(Key::atom()) -> any() | failed.\nread(Key) ->\n    %% If an environment variable sets a config parameter that is present in the\n    % config, it will override the config (see populate_db/1, process_term/1).\n    % We can thus first check the ets table and fall back to the environment\n    % check afterwards.\n    case erlang:get({config,Key}) of\n        undefined ->\n            case ets:info(config_ets) of\n                undefined ->\n                    io:format(\"config not started yet (trying to read ~p)\\n\",\n                              [Key]),\n                    failed;\n                _ ->\n%%                    log:log(\"~p config read in hot path ~p\", [self(), Key]),\n                    case ets:lookup(config_ets, Key) of\n                        [{Key, Value}] ->\n                            erlang:put({config,Key}, Value),\n                            Value;\n                        [] -> Value = util:app_get_env(Key, failed),\n                              case Value of\n                                  failed -> ok;\n                                  _ -> %% case self() =:= erlang:whereis(config)\n                                       %% of\n                                       %%    true -> ets:insert(config_ets,\n                                       %%                       {Key, Value});\n                                       %%    _    ->\n                                            write(Key, Value)\n                                       %% end\n                              end,\n                              erlang:put({config,Key}, Value),\n                              Value\n                    end\n            end;\n        Value -> Value\n    end.\n\n%% @doc Writes a config parameter.\n-spec write(atom(), any()) -> ok.\nwrite(Key, Value) ->\n    %% config is not dynamic. But we update the local cache, so one\n    %% can write config in a local context (in unittests).\n    %% io:format(\"Writing ~p~n\", [{Key, Value}]),\n    erlang:put({config,Key}, Value),\n    ets:insert(config_ets, {Key, Value}),\n    ok.\n\n%@private\n-spec init(Options::[tuple()]) -> ok | no_return().\ninit(Options) ->\n    Files = [util:app_get_env(config, \"scalaris.cfg\"),\n             util:app_get_env(local_config, \"scalaris.local.cfg\")],\n    _ = ets:new(config_ets, [set, public, named_table]),\n    TheFiles = case util:app_get_env(add_config, []) of\n                   []         -> Files;\n                   ConfigFile -> lists:append(Files, [ConfigFile])\n               end,\n    _ = [ populate_db(File) || File <- TheFiles],\n\n    ConfigParameters = case lists:keyfind(config, 1, Options) of\n                           {config, ConfPars} -> ConfPars;\n                           _ -> []\n                       end,\n    AdditionalKVs = util:app_get_env(config_kvs, []),\n    %% io:format(\"~p~n\", [AdditionalKVs]),\n    _ = [write(K, V) || {K, V} <- ConfigParameters],\n    _ = [write(K, V) || {K, V} <- AdditionalKVs],\n\n    %% use different subdirectory per vmname to be independent of distr. Erlang\n    DBSubDir = case read(vmname) of\n                   failed -> novmname;\n                   X -> X\n               end,\n    _ = write(db_directory, read(db_directory) ++ \"/\"\n              ++ atom_to_list(DBSubDir) ++ \"/\"),\n    ok = filelib:ensure_dir(read(db_directory)),\n\n    try check_config() of\n        true -> ok;\n        _    -> % wait so the error output can be written:\n            init:stop(1),\n            receive nothing -> ok end\n    catch Err:Reason -> % wait so the error output can be written:\n            error_logger:error_msg(\"check_config/0 crashed with: ~.0p:~.0p~nStacktrace:~p~n\",\n                                   [Err, Reason, util:get_stacktrace()]),\n            init:stop(1),\n            receive nothing -> ok end\n    end.\n\n%@private\n-spec populate_db(File::file:name()) -> ok | fail.\npopulate_db([]) -> ok;\npopulate_db(File) ->\n    %% note: log4erl may not be available -> use error_logger instead of log\n    case file:consult(File) of\n        {ok, Terms} ->\n            _ = lists:map(fun process_term/1, Terms),\n            ok;\n        {error, enoent} ->\n            case lists:suffix(\"scalaris.local.cfg\", File) of\n                false ->\n                    error_logger:info_msg(\n                      \"Can't load config file ~p: File does not exist. \"\n                      \" Ignoring.\\n\", [File]);\n                true -> ok\n            end,\n            fail;\n        {error, Reason} ->\n            error_logger:error_msg(\"Can't load config file ~p: ~p. Exiting.\\n\",\n                                   [File, Reason]),\n            init:stop(1),\n            receive nothing -> ok end\n            %fail\n    end.\n\n-spec process_term({Key::atom(), Value::term()}) -> true.\nprocess_term({Key, Value}) ->\n    ets:insert(config_ets, {Key, util:app_get_env(Key, Value)}).\n\n%% check config methods\n\n%% @doc Checks whether config parameters of all processes exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    Checks =\n        [ case X() of\n              true -> true;\n              false ->\n                  error_logger:error_msg(\"check_config ~p failed.~n\", [X]),\n                  false\n          end || X  <- [ fun log:check_config/0,\n                         fun sup_scalaris:check_config/0,\n                         fun sup_dht_node_core:check_config/0,\n                         fun proto_sched:check_config/0,\n                         fun acceptor:check_config/0,\n                         fun learner:check_config/0,\n                         fun rdht_tx:check_config/0,\n                         fun rdht_tx_read:check_config/0,\n                         fun rdht_tx_write:check_config/0,\n                         fun ?RT:check_config/0,\n                         fun rt_loop:check_config/0,\n                         fun tx_tm_rtm:check_config/0,\n                         fun vivaldi_latency:check_config/0,\n                         fun ?RM:check_config/0,\n                         fun fd_hbs:check_config/0,\n                         fun dht_node_move:check_config/0,\n                         fun dht_node_join:check_config/0,\n                         %% note: need to check all passive load\n                         %%       balancing algorithm's parameters\n                         %%       (another node may ask us to provide\n                         %%       a candidate for any of them)\n                         fun lb_psv_simple:check_config/0,\n                         fun lb_psv_split:check_config/0,\n                         fun lb_psv_gossip:check_config/0,\n                         fun gossip:check_config/0,\n                         fun comm_tcp_acceptor:check_config/0,\n                         fun comm_ssl_acceptor:check_config/0,\n                         fun monitor:check_config/0,\n                         fun monitor_perf:check_config/0,\n                         fun rrd:check_config/0,\n                         fun rrepair:check_config/0,\n                         fun rr_recon:check_config/0,\n                         fun sup_yaws:check_config/0,\n                         fun dc_clustering:check_config/0,\n                         fun ganglia:check_config/0,\n                         fun autoscale:check_config/0,\n                         fun autoscale_server:check_config/0,\n                         fun cloud_local:check_config/0,\n                         fun cloud_ssh:check_config/0,\n                         fun lb_active:check_config/0,\n                         fun gossip_cyclon:check_config/0,\n                         fun gossip_vivaldi:check_config/0,\n                         fun db_prbr:check_config/0,\n                         fun l_on_cseq:check_config/0,\n                         fun prbr:check_config/0,\n                         fun rbrcseq:check_config/0,\n                         fun crdt:check_config/0,\n                         fun crdt_proposer:check_config/0,\n                         fun db_bitcask_merge_extension:check_config/0\n                       ]],\n    lists:foldl(fun(A,B) -> A and B end, true, Checks).\n\n-spec cfg_exists(Key::atom()) -> boolean().\ncfg_exists(Key) ->\n    case read(Key) of\n        failed ->\n            error_logger:error_msg(\"~p not defined (see scalaris.cfg and scalaris.local.cfg)~n\", [Key]),\n            false;\n        _X -> true\n    end.\n\n%% @doc Tests the config parameter stored under atom Key with function Pred and\n%%      prints an error message if not, also returns the result.\n-spec cfg_test_and_error(Key::atom(), Pred::fun((any()) -> boolean()), Msg::list()) -> boolean().\ncfg_test_and_error(Key, Pred, Msg) ->\n    case read(Key) of\n        failed ->\n            error_logger:error_msg(\"~p not defined (see scalaris.cfg and scalaris.local.cfg)~n\", [Key]),\n            false;\n        Value ->\n            case Pred(Value) of\n                true -> true;\n                _ -> error_logger:error_msg(\"~p = ~p ~s (see scalaris.cfg and scalaris.local.cfg)~n\",\n                                            [Key, Value, lists:flatten(Msg)]),\n                     false\n            end\n    end.\n\n-spec cfg_is_atom(Key::atom()) -> boolean().\ncfg_is_atom(Key) ->\n    Pred = fun erlang:is_atom/1,\n    Msg = \"is not an atom\",\n    cfg_test_and_error(Key, Pred, Msg).\n\n-spec cfg_is_module(Key::atom()) -> boolean().\ncfg_is_module(Key) ->\n    Pred = fun(Value) ->\n                   erlang:is_atom(Value) andalso\n                       code:which(Value) =/= non_existing\n           end,\n    Msg = \"is not an existing module\",\n    cfg_test_and_error(Key, Pred, Msg).\n\n-spec cfg_is_bool(Key::atom()) -> boolean().\ncfg_is_bool(Key) ->\n    Pred = fun erlang:is_boolean/1,\n    Msg = \"is not a boolean\",\n    cfg_test_and_error(Key, Pred, Msg).\n\n-spec cfg_is_mypid(Key::atom()) -> boolean().\ncfg_is_mypid(Key) ->\n    Pred = fun comm:is_valid/1,\n    Msg = \"is not a valid pid\",\n    cfg_test_and_error(Key, Pred, Msg).\n\n-spec cfg_is_ip(Key::atom()) -> boolean().\ncfg_is_ip(Key) ->\n    cfg_is_ip(Key, false).\n\n-spec cfg_is_ip(Key::atom(), AllowUnknown::boolean()) -> boolean().\ncfg_is_ip(Key, AllowUnknown) ->\n    IsIp = fun(Value) ->\n                   case Value of\n                       {IP1, IP2, IP3, IP4} ->\n                           ((IP1 >= 0) andalso (IP1 =< 255)\n                            andalso (IP2 >= 0) andalso (IP2 =< 255)\n                            andalso (IP3 >= 0) andalso (IP3 =< 255)\n                            andalso (IP4 >= 0) andalso (IP4 =< 255));\n                       unknown when AllowUnknown -> true;\n                       _X -> false\n                   end\n           end,\n    Msg = \"is not a valid IPv4 address\",\n    cfg_test_and_error(Key, IsIp, Msg).\n\n-spec cfg_is_port(Key::atom()) -> boolean().\ncfg_is_port(Key) ->\n    IsPort = fun(Value) ->\n                     case Value of\n                         X when erlang:is_integer(X) ->\n                             X >= 0 andalso X =< 65535;\n                         Y when erlang:is_list(Y) ->\n                             lists:all(fun(P) ->\n                                               erlang:is_integer(P) andalso\n                                                   P >= 0 andalso P =< 655351\n                                       end, Y);\n                         {From, To} ->\n                             erlang:is_integer(From) andalso From >= 0 andalso From =< 655351 andalso\n                                 erlang:is_integer(To) andalso To >= 0 andalso To =< 655351;\n                         _ -> false\n                     end\n             end,\n    Msg = \"is not a valid Port address\",\n    cfg_test_and_error(Key, IsPort, Msg).\n\n-spec cfg_is_integer(Key::atom()) -> boolean().\ncfg_is_integer(Key) ->\n    Pred = fun erlang:is_integer/1,\n    Msg = \"is not a valid integer\",\n    cfg_test_and_error(Key, Pred, Msg).\n\n-spec cfg_is_float(Key::atom()) -> boolean().\ncfg_is_float(Key) ->\n    Pred = fun erlang:is_float/1,\n    Msg = \"is not a valid float\",\n    cfg_test_and_error(Key, Pred, Msg).\n\n-spec cfg_is_number(Key::atom()) -> boolean().\ncfg_is_number(Key) ->\n    Pred = fun erlang:is_number/1,\n    Msg = \"is not a valid number\",\n    cfg_test_and_error(Key, Pred, Msg).\n\n-spec cfg_is_tuple(Key::atom(), TupleSize::pos_integer()) -> boolean().\ncfg_is_tuple(Key, Size) ->\n    Pred = fun(Value) ->\n                   erlang:is_tuple(Value) andalso\n                       (erlang:tuple_size(Value) =:= Size)\n           end,\n    Msg = io_lib:format(\"is not a valid tuple of size ~p\", [Size]),\n    cfg_test_and_error(Key, Pred, Msg).\n\n-spec cfg_is_tuple(Key::atom(), TupleSize::pos_integer(), Pred::fun((any()) -> boolean()), PredDescr::string()) -> boolean().\ncfg_is_tuple(Key, Size, Pred, PredDescr) ->\n    CompletePred = fun(Value) ->\n                           erlang:is_tuple(Value) andalso\n                               (erlang:tuple_size(Value) =:= Size) and\n                               Pred(Value)\n                   end,\n    Msg = io_lib:format(\"is not a valid tuple of size ~p satisfying ~p\", [Size, PredDescr]),\n    cfg_test_and_error(Key, CompletePred, Msg).\n\n-spec cfg_is_list(Key::atom()) -> boolean().\ncfg_is_list(Key) ->\n    Pred = fun erlang:is_list/1,\n    Msg = \"is not a valid list\",\n    cfg_test_and_error(Key, Pred, Msg).\n\n-spec cfg_is_list(Key::atom(), Pred::fun((any()) -> boolean()), PredDescr::string()) -> boolean().\ncfg_is_list(Key, Pred, PredDescr) ->\n    IsListWithPred = fun(Value) ->\n                             case Value of\n                                 X when erlang:is_list(X) ->\n                                     lists:all(Pred, X);\n                                 _X -> false\n                             end\n                     end,\n    Msg = io_lib:format(\"is not a valid list with elements satisfying ~p\", [PredDescr]),\n    cfg_test_and_error(Key, IsListWithPred, Msg).\n\n-spec cfg_is_string(Key::atom()) -> boolean().\ncfg_is_string(Key) ->\n    IsChar = fun(X) -> (X >= 0) andalso (X =< 255) end,\n    IsString = fun(Value) ->\n                   case Value of\n                       X when erlang:is_list(X) ->\n                           lists:all(IsChar, X);\n                       _X -> false\n                   end\n           end,\n    Msg = \"is not a (printable) string\",\n    cfg_test_and_error(Key, IsString, Msg).\n\n-spec cfg_is_in_range(Key::atom(), Min::number(), Max::number()) -> boolean().\ncfg_is_in_range(Key, Min, Max) ->\n    IsInRange = fun(Value) -> (Value >= Min) andalso (Value =< Max) end,\n    Msg = io_lib:format(\"is not between ~p and ~p (both inclusive)\",\n                        [Min, Max]),\n    cfg_test_and_error(Key, IsInRange, Msg).\n\n-spec cfg_is_greater_than(Key::atom(), Min::number() | atom()) -> boolean().\ncfg_is_greater_than(_Key, failed) -> false; %% stop endless loop\ncfg_is_greater_than(Key, Min0) when erlang:is_atom(Min0) ->\n    Min = read(Min0),\n    %% stop endless loop (do not recurse!)\n    IsGreaterThan = fun(Value) -> (Value > Min) end,\n    Msg = io_lib:format(\"is not larger than ~p\", [Min]),\n    cfg_test_and_error(Key, IsGreaterThan, Msg);\ncfg_is_greater_than(Key, Min) ->\n    IsGreaterThan = fun(Value) -> (Value > Min) end,\n    Msg = io_lib:format(\"is not larger than ~p\", [Min]),\n    cfg_test_and_error(Key, IsGreaterThan, Msg).\n\n-spec cfg_is_greater_than_equal(Key::atom(), Min::number() | atom()) -> boolean().\ncfg_is_greater_than_equal(_Key, failed) -> false; %% stop endless loop\ncfg_is_greater_than_equal(Key, Min0) when erlang:is_atom(Min0) ->\n    Min = read(Min0),\n    %% stop endless loop (do not recurse!)\n    IsGreaterThanEqual = fun(Value) -> (Value >= Min) end,\n    Msg = io_lib:format(\"is not larger than or equal to ~p\", [Min]),\n    cfg_test_and_error(Key, IsGreaterThanEqual, Msg);\ncfg_is_greater_than_equal(Key, Min) ->\n    IsGreaterThanEqual = fun(Value) -> (Value >= Min) end,\n    Msg = io_lib:format(\"is not larger than or equal to ~p\", [Min]),\n    cfg_test_and_error(Key, IsGreaterThanEqual, Msg).\n\n-spec cfg_is_less_than(Key::atom(), Max::number() | atom()) -> boolean().\ncfg_is_less_than(_Key, failed) -> false; %% stop endless loop\ncfg_is_less_than(Key, Max0) when erlang:is_atom(Max0) ->\n    Max = read(Max0),\n    %% stop endless loop (do not recurse!)\n    IsLessThan = fun(Value) -> (Value < Max) end,\n    Msg = io_lib:format(\"is not less than ~p\", [Max]),\n    cfg_test_and_error(Key, IsLessThan, Msg);\ncfg_is_less_than(Key, Max) ->\n    IsLessThan = fun(Value) -> (Value < Max) end,\n    Msg = io_lib:format(\"is not less than ~p\", [Max]),\n    cfg_test_and_error(Key, IsLessThan, Msg).\n\n-spec cfg_is_less_than_equal(Key::atom(), Max::number() | atom()) -> boolean().\ncfg_is_less_than_equal(_Key, failed) -> false; %% stop endless loop\ncfg_is_less_than_equal(Key, Max0) when erlang:is_atom(Max0) ->\n    Max = read(Max0),\n    %% stop endless loop (do not recurse!)\n    IsLessThanEqual = fun(Value) -> (Value =< Max) end,\n    Msg = io_lib:format(\"is not less than or equal to ~p\", [Max]),\n    cfg_test_and_error(Key, IsLessThanEqual, Msg);\ncfg_is_less_than_equal(Key, Max) ->\n    IsLessThanEqual = fun(Value) -> (Value =< Max) end,\n    Msg = io_lib:format(\"is not less than or equal to ~p\", [Max]),\n    cfg_test_and_error(Key, IsLessThanEqual, Msg).\n\n-spec cfg_is_in(Key::atom(), ValidValues::[any(),...]) -> boolean().\ncfg_is_in(Key, ValidValues) ->\n    IsIn = fun(Value) -> lists:any(fun(X) -> X =:= Value end,\n                                   ValidValues) end,\n    Msg = io_lib:format(\"is not allowed (valid values: ~p)\",\n                        [ValidValues]),\n    cfg_test_and_error(Key, IsIn, Msg).\n"
  },
  {
    "path": "src/configure.erl",
    "content": "% @copyright 2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Make configure options and findings available to the runtime.\n%% @version $Id:$\n-module(configure).\n-author('schintke@zib.de').\n-vsn('$Id:$').\n\n-export([is_enable_debug/0,\n         get_proplist/0,\n         show_config/0]).\n\n-spec is_enable_debug() -> boolean().\n%% return true if './configure --enable-debug' was used.\n-ifdef(enable_debug).\nis_enable_debug() -> true.\n-else.\nis_enable_debug() -> false.\n-endif.\n\n-spec get_proplist() -> list().\nget_proplist() ->\n    [{enable_debug, is_enable_debug()}].\n\n-spec show_config() -> ok.\nshow_config() ->\n    io:format(\"--enable-debug: ~p~n\", [is_enable_debug()]),\n    ok.\n\n\n\n"
  },
  {
    "path": "src/cp/l_on_cseq.erl",
    "content": "% @copyright 2012-2017 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc lease store based on rbrcseq.\n%% @end\n%% @version $Id$\n-module(l_on_cseq).\n-author('schintke@zib.de').\n-author('schuett@zib.de').\n-vsn('$Id:$ ').\n\n%-define(TRACE(X,Y), log:log(X,Y)).\n-define(TRACE(X,Y), ok).\n\n%-define(TRACE_ERROR(X,Y), log:log(X,Y)).\n-define(TRACE_ERROR(X,Y), ok).\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-define(WARN(BOOL, BOOL2, TEXT),\n        if\n            (BOOL) andalso not (BOOL2) ->\n                case util:is_unittest() of\n                    true ->\n                        ct:fail(\"loncq: \" ++ TEXT);\n                    false ->\n                        log:log(\"loncq: \" ++ TEXT)\n                end;\n            true ->\n                ok\n        end).\n\n-define(DELTA, (config:read(leases_delta))).\n%% -define(DELTA, (10)).\n\n-export([check_config/0]).\n\n-export([read/1, read/2]).\n%%-export([write/2]).\n\n-export([on/2]).\n\n-export([lease_renew/2, lease_renew/3]).\n-export([lease_handover/3]).\n-export([lease_takeover/2]).\n-export([lease_takeover_after/3]).\n-export([lease_split/4]).\n-export([lease_merge/3]).\n-export([lease_send_lease_to_node/3]).\n-export([lease_split_and_change_owner/5]).\n-export([disable_lease/2]).\n\n-export([id/1]).\n\n% for unit tests\n-export([unittest_lease_update/4, unittest_lease_update_unsafe/3]).\n-export([unittest_create_lease/1]).\n-export([unittest_create_lease_with_range/3]).\n-export([unittest_clear_lease_list/1]).\n-export([unittest_get_delta/0]).\n\n% lease accessors\n-export([is_a_lease/1, get_version/1,set_version/2,\n         get_epoch/1, set_epoch/2,\n         new_timeout/0, set_timeout/1, get_timeout/1, get_pretty_timeout/1,\n         get_id/1,\n         get_owner/1, set_owner/2,\n         get_aux/1, set_aux/2,\n         get_range/1, set_range/2,\n         split_range/1,\n         is_valid/1, has_timed_out/1]).\n\n-export([add_first_lease_to_db/2]).\n\n-export_type([lease_t/0, lease_id/0]).\n\n%% filters and checks for rbr_cseq operations\n%% consistency\n\n-type lease_id() :: ?RT:key().\n-type lease_aux() ::\n        empty\n      | {change_owner, comm:mypid()}\n      | {invalid, split, intervals:interval(), intervals:interval()}\n      | {valid,   split, intervals:interval(), intervals:interval()}\n      | {invalid, merge, intervals:interval(), intervals:interval()}\n      | {valid,   merge, intervals:interval(), intervals:interval()}.\n\n-record(lease, {\n          id      = ?required(lease, id     ) :: lease_id(),\n          epoch   = ?required(lease, epoch  ) :: non_neg_integer(),\n          owner   = ?required(lease, owner  ) :: comm:mypid_plain() | nil,\n          range   = ?required(lease, range  ) :: intervals:interval(),\n          aux     = ?required(lease, aux    ) :: lease_aux(),\n          version = ?required(lease, version) :: non_neg_integer(),\n          timeout = ?required(lease, timeout) :: erlang_timestamp()}).\n-type lease_t() :: #lease{}.\n\n-type generic_failed_reason() :: lease_does_not_exist\n                               | unexpected_id\n                               | unexpected_owner\n                               | unexpected_aux\n                               | unexpected_range\n                               | unexpected_timeout\n                               | unexpected_epoch\n                               | unexpected_version\n                               | timeout_is_not_newer_than_current_lease.\n\n-type update_failed_reason() :: lease_does_not_exist\n                              | epoch_or_version_mismatch.\n\n-type split_step1_failed_reason() :: lease_already_exists.\n\n-type content_check_t() :: fun ((Current::lease_t() | prbr_bottom,\n                                 WriteFilter::prbr:write_filter(),\n                                 Next::lease_t()) ->\n          {Result::boolean(), {Reason::generic_failed_reason(), Current::lease_t() | prbr_bottom,\n                               Next::lease_t()} | null}).\n\n-spec delta() -> pos_integer().\ndelta() -> ?DELTA.\n\n-spec unittest_get_delta() -> pos_integer().\nunittest_get_delta() ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    delta().\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% Public API\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec lease_renew(lease_t(), active | passive) -> ok.\nlease_renew(Lease, Mode) ->\n    lease_renew(pid_groups:get_my(dht_node), Lease, Mode).\n\n-spec lease_renew(comm:erl_local_pid(), lease_t(), active | passive) -> ok.\nlease_renew(Pid, Lease, Mode) ->\n    comm:send_local(Pid,\n                    {l_on_cseq, renew, Lease, Mode}),\n    ok.\n\n-spec lease_handover(lease_t(), comm:mypid(), comm:erl_local_pid()) -> ok.\nlease_handover(Lease, NewOwner, ReplyTo) ->\n    % @todo precondition: i am owner of Lease\n    comm:send_local(pid_groups:get_my(dht_node),\n                    {l_on_cseq, handover, Lease, NewOwner, ReplyTo}),\n    ok.\n\n-spec lease_takeover(lease_t(), comm:erl_local_pid()) -> ok.\nlease_takeover(Lease, ReplyTo) ->\n    % @todo precondition: Lease has timed out\n    comm:send_local(pid_groups:get_my(dht_node),\n                    {l_on_cseq, takeover, Lease, ReplyTo}),\n    ok.\n\n-spec lease_takeover_after(non_neg_integer(), lease_t(), comm:erl_local_pid()) -> ok.\nlease_takeover_after(Delay, Lease, ReplyTo) ->\n    % @todo precondition: Lease has timed eout\n    msg_delay:send_local(Delay, pid_groups:get_my(dht_node),\n                         {l_on_cseq, takeover, Lease, ReplyTo}),\n    ok.\n\n-spec lease_split(lease_t(), intervals:interval(),\n                  intervals:interval(),\n                  comm:erl_local_pid()) -> ok.\nlease_split(Lease, R1, R2, ReplyTo) ->\n    % @todo precondition: i am owner of Lease and id(R2) == id(Lease)\n    comm:send_local(pid_groups:get_my(dht_node),\n                    {l_on_cseq, split, Lease, R1, R2, ReplyTo, empty}),\n    ok.\n\n-spec lease_merge(lease_t(), lease_t(), comm:erl_local_pid()) -> ok.\nlease_merge(Lease1, Lease2, ReplyTo) ->\n    % @todo precondition: i am owner of Lease1 and Lease2\n    comm:send_local(pid_groups:get_my(dht_node),\n                    {l_on_cseq, merge, Lease1, Lease2, ReplyTo}),\n    ok.\n\n-spec lease_send_lease_to_node(Pid::comm:mypid(), Lease::lease_t(), active | passive) -> ok.\nlease_send_lease_to_node(Pid, Lease, Mode) ->\n    % @todo precondition: Pid is a dht_node\n    comm:send(Pid, {l_on_cseq, send_lease_to_node, Lease, Mode}),\n    ok.\n\n-spec lease_split_and_change_owner(lease_t(),\n                                   intervals:interval(), intervals:interval(),\n                                   comm:mypid(), comm:erl_local_pid()) -> ok.\nlease_split_and_change_owner(Lease, R1, R2, NewOwner, ReplyPid) ->\n    % @todo precondition: i am owner of Lease and id(R2) == id(Lease)\n    % @todo precondition: i am owner of Lease\n    DHTNode = pid_groups:get_my(dht_node),\n    SplitReply = comm:reply_as(DHTNode, 6,\n                               {l_on_cseq, split_and_change_owner, Lease,\n                                NewOwner, ReplyPid, '_'}),\n    comm:send_local(DHTNode,\n                    {l_on_cseq, split, Lease, R1, R2, SplitReply,\n                     {change_owner, NewOwner}}),\n    ok.\n\n-spec disable_lease(State::dht_node_state:state(), Lease::lease_t()) -> dht_node_state:state().\ndisable_lease(State, Lease) ->\n    lease_list:remove_lease_from_dht_node_state(Lease, get_id(Lease), State, passive).\n\n% for unit tests\n-spec unittest_lease_update(lease_t(), lease_t(), active | passive, pid()) -> ok | failed.\nunittest_lease_update(Old, New, Mode, DHTNode) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    comm:send_local(DHTNode,\n                    {l_on_cseq, unittest_update, Old, New, Mode, self()}),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n            {l_on_cseq, unittest_update_success, Old, New}, %% ->\n            ok);\n        ?SCALARIS_RECV(\n            {l_on_cseq, unittest_update_failed, Old, New}, %% ->\n            failed\n          )\n    end.\n\n-spec unittest_lease_update_unsafe(lease_t(), lease_t(), active | passive) -> ok | failed.\nunittest_lease_update_unsafe(Old, New, Mode) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    comm:send_local(pid_groups:get_my(dht_node),\n                    {l_on_cseq, unittest_update, Old, New, Mode, self()}),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n            {l_on_cseq, unittest_update_success, Old, New}, %% ->\n            ok);\n        ?SCALARIS_RECV(\n            {l_on_cseq, unittest_update_failed, Old, New}, %% ->\n            failed\n          )\n    end.\n\n-spec unittest_clear_lease_list(Pid::comm:erl_local_pid()) -> ok.\nunittest_clear_lease_list(Pid) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    comm:send_local(Pid,\n                    {l_on_cseq, unittest_clear_lease_list, comm:this()}),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n            {l_on_cseq, unittest_clear_lease_list_success}, %% ->\n            ok\n          )\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% gen_component\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec on(comm:message(), dht_node_state:state()) -> dht_node_state:state() | kill.\non({l_on_cseq, split_and_change_owner, _Lease, NewOwner, ReplyPid, SplitResult}, State) ->\n    case SplitResult of\n        {split, success, L2, _L1} ->\n            gen_component:post_op({l_on_cseq, handover, L2, NewOwner, ReplyPid},\n                                  State);\n        {split, fail, L1} ->\n            comm:send_local(ReplyPid, {split, fail, L1}),\n            State\n    end;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% lease renewal\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, renew, Old = #lease{owner=Owner,epoch=OldEpoch,version=OldVersion}, Mode},\n   State) ->\n    %log:pal(\"on renew ~w (~w, ~w)~n\", [Old, Mode, self()]),\n    Self = comm:this(),\n    {New, Renew} =\n        case Mode of\n            passive -> % passive lease are only renewed\n                {Old#lease{version=OldVersion+1, timeout=new_timeout()}, renew};\n            active ->\n                case get_aux(Old) of\n                    % change owner to self -> remove aux\n                    {change_owner, Self} ->\n                        {Old#lease{aux=empty,version=OldVersion+1, timeout=new_timeout()}, renew};\n                    _ ->\n                        case comm:this() of\n                            Owner ->\n                                {Old#lease{version=OldVersion+1, timeout=new_timeout()}, renew};\n                            _ ->\n                                % we are trying to recover\n                                log:log(\"trying to recover: owner=~p id=~p, self=~p\",\n                                        [Owner, get_id(Old), comm:this()]),\n                                {Old#lease{owner = comm:this(), epoch = OldEpoch+1, version=0,\n                                           aux=empty, timeout=new_timeout()}, renew_recover}\n                        end\n                end\n          end,\n    ContentCheck = generic_content_check(Old, New, Renew),\n%% @todo New passed for debugging only:\n    ReplyTo = comm:reply_as(self(), 3, {l_on_cseq, renew_reply, '_', New, Mode, Renew}),\n    update_lease(ReplyTo, ContentCheck, Old, New, State),\n    State;\n\non({l_on_cseq, renew_reply, {qwrite_done, _ReqId, Round, Value, _WriteRet}, _New, Mode, _Renew}, State) ->\n    %% log:pal(\"successful renew~n~w~n~w~n\", [Value, l_on_cseq:get_id(Value)]),\n    case lease_list:contains_lease(Value, State, Mode) of\n        true ->\n            lease_list:update_lease_in_dht_node_state(Value,\n                                                      lease_list:update_next_round(l_on_cseq:get_id(Value),\n                                                                                   Round, State),\n                                                      Mode, renew);\n        false ->\n            State\n    end;\n\non({l_on_cseq, renew_reply,\n    {qwrite_deny, _ReqId, Round, Value, {content_check_failed, {Reason, _Current, _Next}}},\n    _New, Mode, Renew}, State) ->\n    % @todo retry\n    ?TRACE_ERROR(\"renew denied: ~p~nVal: ~p~nNew: ~p~n~p~n\", [Reason, Value, _New, Mode]),\n    ?TRACE_ERROR(\"id: ~p~n\", [dht_node_state:get(State, node_id)]),\n    ?TRACE_ERROR(\"lease list: ~p~n\", [dht_node_state:get(State, lease_list)]),\n    ?TRACE_ERROR(\"timeout: ~p~n\", [calendar:now_to_local_time(get_timeout(Value))]),\n    case Reason of\n        lease_does_not_exist ->\n            case Value of %@todo is this necessary?\n                prbr_bottom ->\n                    State;\n                _ ->\n                    lease_list:remove_lease_from_dht_node_state(Value, get_id(Value), State,\n                                                                Mode)\n            end;\n        unexpected_owner   ->\n            CurrentOwner = get_owner(Value),\n            case {comm:this(), Renew} of\n                {CurrentOwner, renew_recover} ->\n                    % the owner was already changed in a recover\n                    State;\n                _ ->\n                    lease_list:remove_lease_from_dht_node_state(Value, get_id(Value), State, Mode)\n            end;\n        unexpected_aux     ->\n            case get_aux(Value) of\n                empty                  ->\n                    renew_and_update_round(Value, Round, Mode, State);\n                {invalid, split, _, _} ->\n                    renew_and_update_round(Value, Round, Mode, State);\n                {invalid, merge, _, _} ->\n                    renew_and_update_round(Value, Round, Mode, State);\n                {valid, split, _, _}   ->\n                    renew_and_update_round(Value, Round, Mode, State);\n                {valid, merge, _, _}   ->\n                    renew_and_update_round(Value, Round, Mode, State);\n                {change_owner, _Pid}  ->\n                    renew_and_update_round(Value, Round, Mode, State)\n            end;\n        unexpected_range   ->\n            renew_and_update_round(Value, Round, Mode, State);\n        unexpected_timeout ->\n            renew_and_update_round(Value, Round, Mode, State);\n        unexpected_epoch   ->\n            renew_and_update_round(Value, Round, Mode, State);\n        unexpected_version ->\n            renew_and_update_round(Value, Round, Mode, State);\n        timeout_is_not_newer_than_current_lease ->\n            renew_and_update_round(Value, Round, Mode, State)\n    end;\n\non({l_on_cseq, send_lease_to_node, Lease, Mode}, State) ->\n    % @todo do we need any checks?\n    % @todo do i need to notify rm about the new range?\n    ?TRACE(\"send_lease_to_node ~p ~p~n\", [self(), Lease]),\n    lease_list:update_lease_in_dht_node_state(Lease, State, Mode, received_lease);\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% lease update (only for unit tests)\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, unittest_update,\n    Old = #lease{id=Id, epoch=OldEpoch,version=OldVersion},\n    New, Mode, Caller}, State) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    %% io:format(\"renew ~p~n\", [Old]),\n    ContentCheck = is_valid_update(OldEpoch, OldVersion),\n    DB = rbrcseq:get_db_for_id(lease_db, Id),\n    %% @todo New passed for debugging only:\n    Self = comm:reply_as(self(), 3, {l_on_cseq, unittest_update_reply, '_',\n                                     Old, New, Mode, Caller}),\n    rbrcseq:qwrite(DB, Self, Id, ?MODULE, ContentCheck, New),\n    State;\n\non({l_on_cseq, unittest_update_reply, {qwrite_done, _ReqId, _Round, Value, _WriteRet},\n    Old, New, Mode, Caller}, State) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    %% io:format(\"successful update~n\", []),\n    comm:send_local(Caller, {l_on_cseq, unittest_update_success, Old, New}),\n    lease_list:update_lease_in_dht_node_state(Value, State, Mode, unittest);\n\non({l_on_cseq, unittest_update_reply,\n    {qwrite_deny, _ReqId, _Round, _Value, {content_check_failed, _Reason}},\n    Old, New, _Mode, Caller}, State) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    comm:send_local(Caller, {l_on_cseq, unittest_update_failed, Old, New}),\n    State;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% clear lease list (only for unit tests)\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, unittest_clear_lease_list, Pid}, State) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    comm:send(Pid, {l_on_cseq, unittest_clear_lease_list_success}),\n    dht_node_state:set_lease_list(State, lease_list:empty());\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% lease handover\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, handover, Old = #lease{epoch=OldEpoch},\n    NewOwner, ReplyTo}, State) ->\n    %log:log(\"handover with aux= ~p\", [Old]),\n    New = case get_aux(Old) of\n              empty ->\n                  Old#lease{epoch   = OldEpoch + 1,\n                            owner   = NewOwner,\n                            version = 0,\n                            timeout = new_timeout()};\n              {change_owner, NewOwner} ->\n                  Old#lease{epoch   = OldEpoch + 1,\n                            owner   = NewOwner,\n                            aux     = empty,\n                            version = 0,\n                            timeout = new_timeout()}\n          end,\n    ContentCheck = generic_content_check(Old, New, handover),\n    Self = comm:reply_as(self(), 3, {l_on_cseq, handover_reply, '_', ReplyTo,\n                                     NewOwner, New}),\n    update_lease(Self, ContentCheck, Old, New, State),\n    State;\n\n\non({l_on_cseq, handover_reply, {qwrite_done, _ReqId, _Round, Value, _WriteRet}, ReplyTo,\n    _NewOwner, _New}, State) ->\n    % @todo if success update lease in State\n    ?TRACE(\"successful handover ~p~n\", [Value]),\n    comm:send_local(ReplyTo, {handover, success, Value}),\n    lease_list:update_lease_in_dht_node_state(Value, State, passive, handover);\n\non({l_on_cseq, handover_reply, {qwrite_deny, _ReqId, _Round, Value,\n                                {content_check_failed, {Reason, _Current, _Next}}},\n    ReplyTo, NewOwner, _New}, State) ->\n    ?TRACE_ERROR(\"handover denied: ~p ~p ~p~n\", [Reason, Value, _New]),\n    case Reason of\n        lease_does_not_exist ->\n            comm:send_local(ReplyTo, {handover, failed, Value}),\n            case Value of %@todo is this necessary?\n                prbr_bottom ->\n                    State;\n                _ ->\n                    lease_list:remove_lease_from_dht_node_state(Value, get_id(Value), State, any)\n            end;\n        unexpected_owner   ->\n            comm:send_local(ReplyTo, {handover, failed, Value}),\n            lease_list:remove_lease_from_dht_node_state(Value, get_id(Value), State, any);\n        unexpected_aux     ->\n            comm:send_local(ReplyTo, {handover, failed, Value}), State;\n        unexpected_range   ->\n            comm:send_local(ReplyTo, {handover, failed, Value}), State;\n        unexpected_timeout -> lease_handover(Value, NewOwner, ReplyTo), State;\n        unexpected_epoch   -> lease_handover(Value, NewOwner, ReplyTo), State;\n        unexpected_version -> lease_handover(Value, NewOwner, ReplyTo), State;\n        timeout_is_not_newer_than_current_lease ->\n            lease_handover(Value, NewOwner, ReplyTo),\n            State\n    end;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% lease takeover\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, takeover, Old = #lease{epoch=OldEpoch},\n    ReplyTo}, State) ->\n    New = Old#lease{epoch   = OldEpoch + 1,\n                    version = 0,\n                    owner   = comm:this(),\n                    timeout = new_timeout()},\n    ContentCheck = generic_content_check(Old, New, takeover),\n    Self = comm:reply_as(self(), 4, {l_on_cseq, takeover_reply, ReplyTo, '_'}),\n    update_lease(Self, ContentCheck, Old, New, State),\n    State;\n\n\non({l_on_cseq, takeover_reply, ReplyTo,\n    {qwrite_done, _ReqId, _Round, Value, _WriteRet}}, State) ->\n    %% log:log(\"takeover success ~p~n\", [Value]),\n    comm:send_local(ReplyTo, {takeover, success, Value}),\n    lease_list:update_lease_in_dht_node_state(Value, State, passive, takeover);\n\non({l_on_cseq, takeover_reply, ReplyTo,\n    {qwrite_deny, _ReqId, _Round, Value,\n     {content_check_failed, {Reason, _Current, _Next}}}}, State) ->\n    ?TRACE(\"takeover failed ~p ~p~n\", [Value, Reason]),\n    comm:send_local(ReplyTo, {takeover, failed, Value, Reason}),\n    State;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% lease merge (step1)\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, merge, _L1 = #lease{}, _L2 = empty, _ReplyTo}, State) ->\n    ?TRACE(\"trying to merge with empty lease ?!?\", []),\n    State;\n\non({l_on_cseq, merge, L1 = #lease{epoch=OldEpoch}, L2, ReplyTo}, State) ->\n    New = L1#lease{epoch   = OldEpoch + 1,\n                   version = 0,\n                   aux     = {invalid, merge, get_range(L1), get_range(L2)},\n                   timeout = new_timeout()},\n    ContentCheck = generic_content_check(L1, New, merge_step1),\n    Self = comm:reply_as(self(), 5, {l_on_cseq, merge_reply_step1,\n                                     L2, ReplyTo, '_'}),\n    update_lease(Self, ContentCheck, L1, New, State),\n    State;\n\non({l_on_cseq, merge_reply_step1, L2, ReplyTo,\n    {qwrite_deny, _ReqId, Round, L1, {content_check_failed,\n                                      {Reason, Current, Next}}}}, State) ->\n    % @todo if success update lease in State\n    ?TRACE_ERROR(\"merge step1 failed~n~w~n~w~n~w~n~w:~w~n\", [Reason, L1, L2, _ReqId, Round]),\n    % retry?\n    case Reason of\n        lease_does_not_exist ->\n            %% cannot happen\n            comm:send_local(ReplyTo, {merge, fail, lease_does_not_exist, step1,\n                                      L1, L2, Current, Next}),\n            State;\n        unexpected_id ->\n            %% cannot happen\n            comm:send_local(ReplyTo, {merge, fail, unexpected_id, step1, L1, L2,\n                                      Current, Next}),\n            State;\n        unexpected_owner ->\n            %% give up, there was probably a concurrent merge\n            comm:send_local(ReplyTo, {merge, fail, unexpected_owner, step1, L1, L2,\n                                     Current, Next}),\n            State;\n        unexpected_aux ->\n            %% give up, there was probably a concurrent merge\n            comm:send_local(ReplyTo, {merge, fail, unexpected_aux, step1, L1, L2,\n                                      Current, Next}),\n            State;\n        unexpected_range ->\n            %% give up, there was probably a concurrent merge\n            comm:send_local(ReplyTo, {merge, fail, unexpected_range, step1, L1, L2,\n                                     Current, Next}),\n            State;\n        unexpected_timeout ->\n            %% retry\n            NextState = lease_list:update_next_round(l_on_cseq:get_id(L1),\n                                                     Round, State),\n            gen_component:post_op({l_on_cseq, merge, L1, L2, ReplyTo}, NextState);\n        %unexpected_epoch ->\n        %    % cannot happen\n        %    gen_component:post_op({l_on_cseq, merge_reply_step1, L2, ReplyTo,\n        %                           {qwrite_done, fake_reqid, fake_round, L1}},\n        %                          lease_list:update_next_round(l_on_cseq:get_id(L2), Round, State));\n        %unexpected_version ->\n        %    % cannot happen\n        %    gen_component:post_op({l_on_cseq, merge_reply_step1, L2, ReplyTo,\n        %                           {qwrite_done, fake_reqid, fake_round, L1}},\n        %                          lease_list:update_next_round(l_on_cseq:get_id(L2), Round, State));\n        timeout_is_not_newer_than_current_lease ->\n            % retry\n            NextState = lease_list:update_next_round(l_on_cseq:get_id(L1),\n                                                     Round, State),\n            gen_component:post_op({l_on_cseq, merge, L1, L2, ReplyTo}, NextState)\n    end;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% lease merge (step2)\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, merge_reply_step1, L2 = #lease{epoch=OldEpoch}, ReplyTo,\n    {qwrite_done, _ReqId, _Round, L1, _WriteRet}}, State) ->\n    % @todo if success update lease in State\n    New = L2#lease{epoch   = OldEpoch + 1,\n                   version = 0,\n                   range   = intervals:union(L1#lease.range, L2#lease.range),\n                   aux     = {valid, merge, get_range(L1), get_range(L2)},\n                   timeout = new_timeout()},\n    ContentCheck = generic_content_check(L2, New, merge_step2),\n    Self = comm:reply_as(self(), 5, {l_on_cseq, merge_reply_step2,\n                                     L1, ReplyTo, '_'}),\n    update_lease(Self, ContentCheck, L2, New, State),\n    lease_list:update_lease_in_dht_node_state(L1, State, passive,\n                                              merge_reply_step1);\n\n\n\non({l_on_cseq, merge_reply_step2, L1, ReplyTo,\n    {qwrite_deny, _ReqId, Round, L2,\n     {content_check_failed, {Reason, Current, Next}}}}, State) ->\n    % @todo if success update lease in State\n    ?TRACE_ERROR(\"merge step2 failed~n~w~n~w~n~w~n~w~n~w~n\", [Reason, L1, L2, Current, Next]),\n    case Reason of\n        lease_does_not_exist ->\n            %% cannot happen\n            comm:send_local(ReplyTo, {merge, fail, lease_does_not_exist, step2, L1, L2, Current, Next}),\n            State;\n        unexpected_id ->\n            %% cannot happen\n            comm:send_local(ReplyTo, {merge, fail, unexpected_id, step2, L1, L2, Current, Next}),\n            State;\n        unexpected_owner ->\n            %% give up, there was probably a concurrent merge\n            comm:send_local(ReplyTo, {merge, fail, unexpected_owner, step2, L1, L2, Current, Next}),\n            State;\n        unexpected_aux ->\n            %% give up, there was probably a concurrent merge\n            comm:send_local(ReplyTo, {merge, fail, unexpected_aux, step2, L1, L2, Current, Next}),\n            State;\n        unexpected_range ->\n            %% give up, there was probably a concurrent merge\n            comm:send_local(ReplyTo, {merge, fail, unexpected_range, step2, L1, L2, Current, Next}),\n            State;\n        unexpected_timeout ->\n            % retry\n            gen_component:post_op({l_on_cseq, merge_reply_step1, L2, ReplyTo,\n                                   {qwrite_done, fake_reqid, fake_round, L1, none}},\n                                  lease_list:update_next_round(l_on_cseq:get_id(L2), Round, State));\n        %unexpected_epoch ->\n        %    % cannot happen\n        %    gen_component:post_op({l_on_cseq, merge_reply_step1, L2, ReplyTo,\n        %                           {qwrite_done, fake_reqid, fake_round, L1}},\n        %                          lease_list:update_next_round(l_on_cseq:get_id(L2), Round, State));\n        %unexpected_version ->\n        %    % cannot happen\n        %    gen_component:post_op({l_on_cseq, merge_reply_step1, L2, ReplyTo,\n        %                           {qwrite_done, fake_reqid, fake_round, L1}},\n        %                          lease_list:update_next_round(l_on_cseq:get_id(L2), Round, State));\n        timeout_is_not_newer_than_current_lease ->\n            % retry\n            gen_component:post_op({l_on_cseq, merge_reply_step1, L2, ReplyTo,\n                                   {qwrite_done, fake_reqid, fake_round, L1, none}},\n                                  lease_list:update_next_round(l_on_cseq:get_id(L2), Round, State))\n    end;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% lease merge (step3)\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, merge_reply_step2, L1 = #lease{epoch=_OldEpoch}, ReplyTo,\n    {qwrite_done, _ReqId, _Round, L2, _WriteRet}}, State) ->\n    New = prbr_bottom,\n    ContentCheck = cc_delete_lease(L1, merge_step3),\n    Self = comm:reply_as(self(), 5, {l_on_cseq, merge_reply_step3,\n                                     {get_id(L1), L2}, ReplyTo, '_'}),\n    update_lease(Self, ContentCheck, L1, New, State),\n    lease_list:update_lease_in_dht_node_state(L2, State, active,\n                                              merge_reply_step2);\n\non({l_on_cseq, merge_reply_step3, {L1Id, L2}, ReplyTo,\n    {qwrite_deny, _ReqId, Round, L1, {content_check_failed,\n                                      {Reason, Current, Next}}}}, State) ->\n    % @todo if success update lease in State\n    ?TRACE_ERROR(\"merge step3 failed~n~w~n~w~n~w~n\", [Reason, L1, L2]),\n    case Reason of\n        lease_does_not_exist ->\n            %% cannot happen\n            comm:send_local(ReplyTo, {merge, fail, lease_does_not_exist, step3, L1, L2, Current, Next}),\n          State;\n        unexpected_id ->\n            %% cannot happen\n            comm:send_local(ReplyTo, {merge, fail, unexpected_id, step3, L1, L2, Current, Next}),\n            State;\n        unexpected_owner ->\n            %% give up, there was probably a concurrent merge\n            comm:send_local(ReplyTo, {merge, fail, unexpected_owner, step3, L1, L2, Current, Next}),\n            State;\n        unexpected_aux ->\n            %% give up, there was probably a concurrent merge\n            comm:send_local(ReplyTo, {merge, fail, unexpected_aux, step3, L1, L2, Current, Next}),\n            State;\n        unexpected_range ->\n            %% give up, there was probably a concurrent merge\n            comm:send_local(ReplyTo, {merge, fail, unexpected_range, step3, L1, L2, Current, Next}),\n            State;\n        unexpected_timeout ->\n            %% retry\n            NextState = lease_list:update_next_round(L1Id,\n                                                     Round, State),\n            gen_component:post_op({l_on_cseq, merge, L1, L2, ReplyTo}, NextState);\n        %unexpected_epoch ->\n        %    % cannot happen\n        %    gen_component:post_op({l_on_cseq, merge_reply_step1, L2, ReplyTo,\n        %                           {qwrite_done, fake_reqid, fake_round, L1}},\n        %                          lease_list:update_next_round(l_on_cseq:get_id(L2), Round, State));\n        %unexpected_version ->\n        %    % cannot happen\n        %    gen_component:post_op({l_on_cseq, merge_reply_step1, L2, ReplyTo,\n        %                           {qwrite_done, fake_reqid, fake_round, L1}},\n        %                          lease_list:update_next_round(l_on_cseq:get_id(L2), Round, State));\n        timeout_is_not_newer_than_current_lease ->\n            % retry\n            NextState = lease_list:update_next_round(l_on_cseq:get_id(L1),\n                                                     Round, State),\n            gen_component:post_op({l_on_cseq, merge, L1, L2, ReplyTo}, NextState)\n    end;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% lease merge (step4)\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, merge_reply_step3, {L1Id, L2 = #lease{epoch=OldEpoch}}, ReplyTo,\n    {qwrite_done, _ReqId, _Round, L1, _WriteRet}}, State) ->\n    % @todo if success update lease in State\n    ?TRACE(\"successful merge step3 ~p~n\", [L1]),\n    New = L2#lease{epoch   = OldEpoch + 1,\n                   version = 0,\n                   aux     = empty,\n                   timeout = new_timeout()},\n    ContentCheck = generic_content_check(L2, New, merge_step4),\n    Self = comm:reply_as(self(), 5, {l_on_cseq, merge_reply_step4,\n                                     {L1Id, L1}, ReplyTo, '_'}),\n    update_lease(Self, ContentCheck, L2, New, State),\n    lease_list:remove_lease_from_dht_node_state(L1, L1Id, State, passive);\n\non({l_on_cseq, merge_reply_step4, {_L1Id, L1}, ReplyTo,\n    {qwrite_done, _ReqId, Round, L2, _WriteRet}}, State) ->\n    ?TRACE(\"successful merge ~p~p~n\", [ReplyTo, L2]),\n    comm:send_local(ReplyTo, {merge, success, L2, L1}),\n    lease_list:update_lease_in_dht_node_state(L2,\n                                              lease_list:update_next_round(l_on_cseq:get_id(L2),\n                                                                           Round, State),\n                                              active,\n                                              merge_reply_step3);\n\non({l_on_cseq, merge_reply_step4, {L1Id, L1}, ReplyTo,\n    {qwrite_deny, _ReqId, Round, L2, {content_check_failed,\n                                      {Reason, Current, Next}}}}, State) ->\n    % @todo if success update lease in State\n    ?TRACE_ERROR(\"merge step4 failed~n~w~n~w~n~w~n\", [Reason, L1, L2]),\n    % retry?\n    case Reason of\n        unexpected_owner ->\n            %% give up, there was probably a concurrent merge\n            comm:send_local(ReplyTo, {merge, fail, unexpected_owner, step4, L1, L2, Current, Next}),\n            State;\n        unexpected_timeout ->\n            % retry\n            gen_component:post_op({l_on_cseq, merge_reply_step3, {L1Id, L2}, ReplyTo,\n                                   {qwrite_done, fake_reqid, fake_round, L1, none}},\n                                  lease_list:update_next_round(l_on_cseq:get_id(L2), Round, State));\n        timeout_is_not_newer_than_current_lease ->\n            % retry\n            gen_component:post_op({l_on_cseq, merge_reply_step3, {L1Id, L2}, ReplyTo,\n                                   {qwrite_done, fake_reqid, fake_round, L1, none}},\n                                  lease_list:update_next_round(l_on_cseq:get_id(L2), Round, State))\n    end;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% lease split (step1)\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, split, Lease, R1, R2, ReplyTo, PostAux}, State) ->\n    Id = id(R1),\n    ?TRACE(\"split first step: creating new lease L1(~w) (~p)~n\", [self(), Id]),\n    _Active = get_active_lease(State),\n    ?TRACE(\"going to split(~w):~n~w~n~w~n\", [self(), _Active, Lease]),\n    New = #lease{id      = id(R1),\n                 epoch   = 1,\n                 owner   = comm:this(),\n                 range   = R1,\n                 aux     = {invalid, split, R1, R2},\n                 version = 0,\n                 timeout = new_timeout()},\n    ContentCheck = is_valid_split_step1(),\n    DB = rbrcseq:get_db_for_id(lease_db, Id),\n    Self = comm:reply_as(self(), 8, {l_on_cseq, split_reply_step1, Lease, R1, R2,\n                                     ReplyTo, PostAux, '_'}),\n    %log:log(\"self in split firststep: ~w\", [Self]),\n    rbrcseq:qwrite(DB, Self, Id, ?MODULE, ContentCheck, New),\n    State;\n\non({l_on_cseq, split_reply_step1, _Lease, _R1, _R2, ReplyTo, _PostAux,\n    {qwrite_deny, _ReqId, _Round, Lease, {content_check_failed, Reason}}}, State) ->\n    ?TRACE_ERROR(\"split first step failed: ~p~n\", [Reason]),\n    case Reason of\n        lease_already_exists ->\n            comm:send_local(ReplyTo, {split, fail, Lease}),\n            State\n    end;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% lease split (step2)\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, split_reply_step1, L2=#lease{id=_Id,epoch=OldEpoch}, R1, R2,\n    ReplyTo, PostAux, {qwrite_done, _ReqId, _Round, L1, _WriteRet}}, State) ->\n    ?TRACE(\"split second step(~w): updating L2 (~p)~n\", [self(), _Id]),\n    _Active = get_active_lease(State),\n    ?TRACE(\"split second step(~w):~n~w~n~w~n~w~n\", [self(), _Active, L1, L2]),\n    New = L2#lease{\n            epoch   = OldEpoch + 1,\n            range   = R2,\n            aux     = {valid, split, R1, R2},\n            version = 0,\n            timeout = new_timeout()},\n    ContentCheck = generic_content_check(L2, New, split_reply_step1),\n    Self = comm:reply_as(self(), 8, {l_on_cseq, split_reply_step2,\n                                     L1, R1, R2, ReplyTo, PostAux, '_'}),\n    update_lease(Self, ContentCheck, L2, New, State),\n    lease_list:update_lease_in_dht_node_state(L1, State, passive,\n                                              split_reply_step1);\n\non({l_on_cseq, split_reply_step2, L1, R1, R2, ReplyTo, PostAux,\n    {qwrite_deny, _ReqId, _Round, L2, {content_check_failed,\n                                       {Reason, _Current, _Next}}}}, State) ->\n    ?TRACE_ERROR(\"split second step failed: ~p~n\", [Reason]),\n    case Reason of\n        lease_does_not_exist -> comm:send_local(ReplyTo, {split, fail, L2}), State; %@todo\n        unexpected_owner     -> comm:send_local(ReplyTo, {split, fail, L2}),\n                                lease_list:remove_lease_from_dht_node_state(L2, get_id(L2),\n                                                                            State, any); %@todo\n        unexpected_range     -> comm:send_local(ReplyTo, {split, fail, L2}), State; %@todo\n        unexpected_aux       -> comm:send_local(ReplyTo, {split, fail, L2}), State; %@todo\n        unexpected_timeout ->\n            %% retry\n            gen_component:post_op({l_on_cseq, split_reply_step1, L2, R1, R2,\n                                   ReplyTo, PostAux,\n                                   {qwrite_done, fake_reqid, fake_round, L1, none}},\n                                  State);\n        timeout_is_not_newer_than_current_lease ->\n            %% retry\n            gen_component:post_op({l_on_cseq, split_reply_step1, L2, R1, R2,\n                                   ReplyTo, PostAux,\n                                   {qwrite_done, fake_reqid, fake_round, L1, none}},\n                                  State);\n        unexpected_epoch ->\n            %% retry\n            gen_component:post_op({l_on_cseq, split_reply_step1, L2, R1, R2,\n                                   ReplyTo, PostAux,\n                                   {qwrite_done, fake_reqid, fake_round, L1, none}},\n                                  State);\n        unexpected_version ->\n            %% retry\n            gen_component:post_op({l_on_cseq, split_reply_step1, L2, R1, R2,\n                                   ReplyTo, PostAux,\n                                   {qwrite_done, fake_reqid, fake_round, L1, none}},\n                                  State)\n    end;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% lease split (step3)\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, split_reply_step2,\n    L1 = #lease{id=_Id,epoch=OldEpoch}, R1, R2, ReplyTo, PostAux,\n    {qwrite_done, _ReqId, _Round, L2, _WriteRet}}, State) ->\n    ?TRACE(\"split third step(~w): renew L1 ~p~n\", [self(), _Id]),\n    _Active = get_active_lease(State),\n    ?TRACE(\"split_reply_step2(~w):~n~w~n~w~n~w~n\", [self(), _Active, L1, L2]),\n    New = L1#lease{\n            epoch   = OldEpoch + 1,\n            aux     = PostAux,\n            version = 0,\n            timeout = new_timeout()},\n    ContentCheck = generic_content_check(L1, New, split_reply_step2),\n    Self = comm:reply_as(self(), 8, {l_on_cseq, split_reply_step3, L2, R1, R2,\n                                     ReplyTo, PostAux, '_'}),\n    update_lease(Self, ContentCheck, L1, New, State),\n    lease_list:update_lease_in_dht_node_state(L2, State, active, split_reply_step2);\n\non({l_on_cseq, split_reply_step3, L2, R1, R2, ReplyTo, PostAux,\n    {qwrite_deny, _ReqId, _Round, L1, {content_check_failed,\n                                       {Reason, _Current, _Next}}}}, State) ->\n    %% @todo\n    ?TRACE_ERROR(\"split third step failed: ~p~n\", [Reason]),\n    case Reason of\n        lease_does_not_exist -> comm:send_local(ReplyTo, {split, fail, L1}), State; %@todo\n        unexpected_owner     -> comm:send_local(ReplyTo, {split, fail, L1}),\n                                lease_list:remove_lease_from_dht_node_state(L1, get_id(L1),\n                                                                            State, any); %@todo\n        unexpected_range     -> comm:send_local(ReplyTo, {split, fail, L1}), State; %@todo\n        unexpected_aux       -> comm:send_local(ReplyTo, {split, fail, L1}), State; %@todo\n        unexpected_timeout ->\n            %% retry\n            gen_component:post_op({l_on_cseq, split_reply_step2, L1, R1, R2,\n                                   ReplyTo, PostAux,\n                                   {qwrite_done, fake_reqid, fake_round, L2, none}},\n                                  State);\n        timeout_is_not_newer_than_current_lease ->\n            %% retry\n            gen_component:post_op({l_on_cseq, split_reply_step2, L1, R1, R2,\n                                   ReplyTo, PostAux,\n                                   {qwrite_done, fake_reqid, fake_round, L2, none}},\n                                  State);\n        unexpected_epoch ->\n            %% retry\n            gen_component:post_op({l_on_cseq, split_reply_step2, L1, R1, R2,\n                                   ReplyTo, PostAux,\n                                   {qwrite_done, fake_reqid, fake_round, L2, none}},\n                                  State);\n        unexpected_version ->\n            %% retry\n            gen_component:post_op({l_on_cseq, split_reply_step2, L1, R1, R2,\n                                   ReplyTo, PostAux,\n                                   {qwrite_done, fake_reqid, fake_round, L2, none}},\n                                  State)\n    end;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% lease split (step4)\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, split_reply_step3,\n    L2 = #lease{id=_Id,epoch=OldEpoch}, R1, R2, ReplyTo, PostAux,\n    {qwrite_done, _ReqId, _Round, L1, _WriteRet}}, State) ->\n    ?TRACE(\"split fourth step: renew L2 ~p ~p ~p ~p~n\", [R1, R2, _Id, PostAux]),\n    New = L2#lease{\n            epoch   = OldEpoch + 1,\n            aux     = empty,\n            version = 0,\n            timeout = new_timeout()},\n    ContentCheck = generic_content_check(L2, New, split_reply_step3),\n    Self = comm:reply_as(self(), 8, {l_on_cseq, split_reply_step4, L1, R1, R2,\n                                     ReplyTo, PostAux, '_'}),\n    update_lease(Self, ContentCheck, L2, New, State),\n    lease_list:update_lease_in_dht_node_state(L1, State, passive, split_reply_step3);\n\non({l_on_cseq, split_reply_step4, L1, _R1, _R2, ReplyTo, _PostAux,\n    {qwrite_done, _ReqId, _Round, L2, _WriteRet}}, State) ->\n    ?TRACE(\"successful split~n\", []),\n    ?TRACE(\"successful split ~p~n\", [ReplyTo]),\n    _Active = get_active_lease(State),\n    ?TRACE(\"split_reply_step4(~w):~n~w~n~w~n~w~n\", [self(), _Active, L1, L2]),\n    comm:send_local(ReplyTo, {split, success, L1, L2}),\n    lease_list:update_lease_in_dht_node_state(L2, State, active, split_reply_step4);\n\non({l_on_cseq, split_reply_step4, L1, R1, R2, ReplyTo, PostAux,\n    {qwrite_deny, _ReqId, _Round, L2, {content_check_failed,\n                                       {Reason, _Current, _Next}}}}, State) ->\n    %% @todo\n    ?TRACE_ERROR(\"split fourth step failed: ~p~n\", [Reason]),\n    case Reason of\n        lease_does_not_exist -> comm:send_local(ReplyTo, {split, fail, L2}), State;\n        unexpected_owner     -> comm:send_local(ReplyTo, {split, fail, L2}),\n                                lease_list:remove_lease_from_dht_node_state(L2, get_id(L2),\n                                                                            State, active);\n        unexpected_range     -> comm:send_local(ReplyTo, {split, fail, L2}), State;\n        unexpected_aux       -> comm:send_local(ReplyTo, {split, fail, L2}), State;\n        unexpected_timeout ->\n            %% retry\n            gen_component:post_op({l_on_cseq, split_reply_step3, L2, R1, R2,\n                                   ReplyTo, PostAux,\n                                   {qwrite_done, fake_reqid, fake_round, L1, none}},\n                                  State);\n        timeout_is_not_newer_than_current_lease ->\n            %% retry\n            gen_component:post_op({l_on_cseq, split_reply_step3, L2, R1, R2,\n                                   ReplyTo, PostAux,\n                                   {qwrite_done, fake_reqid, fake_round, L1, none}},\n                                  State);\n        unexpected_epoch ->\n            %% retry\n            gen_component:post_op({l_on_cseq, split_reply_step3, L2, R1, R2,\n                                   ReplyTo, PostAux,\n                                   {qwrite_done, fake_reqid, fake_round, L1, none}},\n                                  State);\n        unexpected_version ->\n            %% retry\n            gen_component:post_op({l_on_cseq, split_reply_step3, L2, R1, R2,\n                                   ReplyTo, PostAux,\n                                   {qwrite_done, fake_reqid, fake_round, L1, none}},\n                                  State)\n    end;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% garbage collector results\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, garbage_collector, {merge, success, _, _}}, State) ->\n    log:pal(\"garbage collector: success~n\"),\n    State;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% renew all local leases\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({l_on_cseq, renew_leases}, State) ->\n    LeaseList = dht_node_state:get(State, lease_list),\n    ActiveLease = lease_list:get_active_lease(LeaseList),\n    PassiveLeaseList = lease_list:get_passive_leases(LeaseList),\n    % log:pal(\"renewing local leases: ~p~n\", [ActiveLease]),\n    case ActiveLease of\n        empty -> ok;\n        _ ->\n            lease_renew(self(), ActiveLease, active)\n    end,\n    _ = [lease_renew(self(), L, passive) ||\n            L <- PassiveLeaseList],\n    msg_delay:send_trigger(delta() div 2, {l_on_cseq, renew_leases}),\n    State;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% post recover operations\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\non({l_on_cseq, wait_for_recover}, State) ->\n    LeaseList = dht_node_state:get(State, lease_list),\n    ActiveLease = lease_list:get_active_lease(LeaseList),\n    case get_owner(ActiveLease) =:= comm:this() of\n        true ->\n            %% the active lease has been recovered\n            PassiveLeaseList = lease_list:get_passive_leases(LeaseList),\n            %% find the passive lease next to the active lease\n            Candidates = [ L || L <- PassiveLeaseList,\n                                intervals:is_left_of(get_range(L),\n                                                     get_range(ActiveLease))],\n            %% there should be 0 or 1 candidates\n            case Candidates of\n                [] -> ok;\n                [Passive| _] ->\n                    Me = comm:reply_as(self(), 3, {l_on_cseq, post_recover_takeover, '_'}),\n                    lease_takeover(Passive, Me)\n            end;\n        false ->\n            %% the active lease has not been recovered\n            msg_delay:send_trigger(delta() div 2, {l_on_cseq, wait_for_recover})\n    end,\n    State;\n\non({l_on_cseq, post_recover_takeover, Result}, State) ->\n    case Result of\n        {takeover, success, Value} ->\n            %% merge the new lease with the active lease\n            log:log(\"recover(~p): the takeover of ~p was a success\", [self(), Value]),\n            LeaseList = dht_node_state:get(State, lease_list),\n            ActiveLease = lease_list:get_active_lease(LeaseList),\n            case intervals:is_adjacent(l_on_cseq:get_range(Value),\n                                       l_on_cseq:get_range(ActiveLease))\n                andalso intervals:is_left_of(l_on_cseq:get_range(Value),\n                                             l_on_cseq:get_range(ActiveLease)) of\n                true ->\n                    Me = comm:reply_as(self(), 3, {l_on_cseq, post_recover_merge, '_'}),\n                    lease_merge(Value, ActiveLease, Me),\n                    State;\n                false ->\n                    %% give up ?!?\n                    State\n            end;\n        {takeover, failed, Value, Reason} ->\n            %% remove from passive list just in case\n            log:log(\"recover(~p): the takeover of ~p failed: ~p\", [self(), Value, Reason]),\n            ReplyTo = comm:reply_as(self(), 3, {l_on_cseq, post_recover_takeover, '_'}),\n            case Reason of\n                lease_does_not_exist ->\n                    lease_list:remove_lease_from_dht_node_state(Value, get_id(Value),\n                                                                State, passive);\n                unexpected_owner     ->\n                    lease_list:remove_lease_from_dht_node_state(Value, get_id(Value),\n                                                                State, passive);\n                unexpected_range     ->\n                    lease_list:remove_lease_from_dht_node_state(Value, get_id(Value),\n                                                                State, passive);\n                unexpected_aux       ->\n                    lease_list:remove_lease_from_dht_node_state(Value, get_id(Value),\n                                                                State, passive);\n                unexpected_timeout ->\n                    %% retry\n                    gen_component:post_op({l_on_cseq, takeover, Value, ReplyTo}, State);\n                timeout_is_not_newer_than_current_lease ->\n                    %% retry\n                    gen_component:post_op({l_on_cseq, takeover, Value, ReplyTo}, State);\n                unexpected_epoch ->\n                    %% retry\n                    gen_component:post_op({l_on_cseq, takeover, Value, ReplyTo}, State);\n                unexpected_version ->\n                    %% retry\n                    gen_component:post_op({l_on_cseq, takeover, Value, ReplyTo}, State)\n            end\n    end;\n\non({l_on_cseq, post_recover_merge, Result}, State) ->\n    case Result of\n        {merge, success, L2, L1} ->\n            log:log(\"recover(~p): the merge of ~p and ~p was a success\", [self(), L2, L1]),\n            %% check for more passive leases\n            gen_component:post_op({l_on_cseq, wait_for_recover}, State);\n        {merge, fail, Reason, Step, _L1, _L2, Current, Next} ->\n            log:log(\"recover(~p): the merge failed in step ~p with ~p~nL1: ~p~nL2: ~p~nCurrent: ~p~nNext: ~p~n\",\n                    [self(), Step, Reason, _L1, _L2, Current, Next]),\n            State\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% content checks\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% % @doc generic content check. used for almost all qwrite operations\n%% -spec generic_content_check_(lease_t(), lease_t(), atom()) ->\n%%                                    content_check_t(). %% content check\n%% generic_content_check_(#lease{id=OldId,owner=_OldOwner,aux = OldAux,range=OldRange,\n%%                              epoch=OldEpoch,version=OldVersion,timeout=OldTimeout} = Old,\n%%                      New, Writer) ->\n%%     fun(prbr_bottom, _WriteFilter, Next) ->\n%%             {false, {lease_does_not_exist, prbr_bottom, Next}};\n%%         (Current, _WriteFilter, Next) when Current =:= New ->\n%%             log:pal(\"re-write in CC:~n~w~n~w~n~w~n~w~n~w~n\", [Current, Next, Old, New, Writer]),\n%%             {true, null};\n%%         (#lease{id = Id0} = Current, _, Next)    when Id0 =/= OldId->\n%%             {false, {unexpected_id, Current, Next}};\n%%         %(#lease{owner = O0} = Current, _, Next)    when O0 =/= OldOwner->\n%%         %    {false, {unexpected_owner, Current, Next}};\n%%         (#lease{aux = Aux0} = Current, _, Next)    when Aux0 =/= OldAux->\n%%             {false, {unexpected_aux, Current, Next}};\n%%         (#lease{range = R0} = Current, _, Next)    when R0 =/= OldRange->\n%%             {false, {unexpected_range, Current, Next}};\n%%         (#lease{timeout = T0} = Current, _, Next)                   when T0 =/= OldTimeout->\n%%             {false, {unexpected_timeout, Current, Next}};\n%%         (#lease{epoch = E0} = Current, _, Next)                     when E0 =/= OldEpoch ->\n%%             {false, {unexpected_epoch, Current, Next}};\n%%         (#lease{version = V0} = Current, _, Next)                   when V0 =/= OldVersion->\n%%             {false, {unexpected_version, Current, Next}};\n%%         (#lease{timeout = T0} = Current, _, #lease{timeout = T1} = Next)  when not (T0 < T1)->\n%%             {false, {timeout_is_not_newer_than_current_lease, Current, Next}};\n%%         (_, _, _) ->\n%%             {true, null}\n%%     end.\n\n% @doc generic content check. used for almost all qwrite operations\n-spec generic_content_check(lease_t(), lease_t(), atom()) ->\n                                   content_check_t(). %% content check\ngeneric_content_check(#lease{id=OldId,owner=OldOwner,aux = OldAux,range=OldRange,\n                             epoch=OldEpoch,version=OldVersion,timeout=OldTimeout} = Old,\n                     #lease{id=NewId,owner=NewOwner,aux = NewAux,range=NewRange,\n                             epoch=NewEpoch,version=NewVersion,timeout=NewTimeout} = New, Writer) ->\n    fun(Current, _WriteFilter, Next) ->\n            case Current of\n                % check for prbr_bottom\n%                {prbr_bottom, _WriteFilter, _Next} ->\n                prbr_bottom ->\n                    {false, {lease_does_not_exist, prbr_bottom, Next}};\n                % check for re-write\n                New -> % Current =:= New\n                    log:pal(\"re-write in CC:~n~w~n~w~n~w~n~w~n~w~n\",\n                            [Current, Next, Old, New, Writer]),\n                    {true, null};\n                % special case for renew after crash-recovery\n                #lease{epoch = E0, owner = O0, version = V0}\n                  when E0 =:= OldEpoch andalso V0 =:= OldVersion andalso O0 =:= OldOwner\n                       andalso NewOwner =/= OldOwner\n                       andalso (E0+1) =:= NewEpoch andalso NewVersion =:= 0\n                       andalso Writer =:= renew_recover ->\n                       % after a crash the logical owner should not\n                       % have changed. however its pid will have\n                       % changed. this special case checks that epoch\n                       % and version are correctly guessed, but the\n                       % owner is wrong. in addition, we require that\n                       % this is a renew.\n                    %% log:log(\"loncq(~p): this has to be a renew after a recovery(~p)\",\n                    %%         [self(), OldId]),\n                    {true, null};\n                % check that epoch and version match with Old\n                % we only warn/fail if the remaining fields do not match\n                #lease{epoch = E0, version = V0} when  E0 =:= OldEpoch andalso V0 =:= OldVersion ->\n                    % @todo sanity check with warnings: protocol was implemented correctly\n                    EpochUpdate = (NewEpoch =:= OldEpoch + 1) andalso (NewVersion =:= 0),\n                    VersionUpdate = (NewEpoch =:= OldEpoch) andalso (NewVersion =:= OldVersion + 1),\n                    % check for correct epoch/version handling\n                    ?WARN(OldEpoch =:= NewEpoch,\n                          OldVersion + 1 =:= NewVersion,\n                         \"the version has to increase by one\"),\n                    ?WARN(OldEpoch + 1 =:= NewEpoch,\n                          0 =:= NewVersion,\n                         \"the version has to be zero after an epoch update: \"\n                             ++ atom_to_list(Writer)),\n                    % check that the id does not change\n                    ?WARN(OldId =/= NewId,\n                          false,\n                          \"the id may never change: \"\n                              ++ atom_to_list(Writer)),\n                    % check for correct behavior for the remaining fields\n                    ?WARN(OldOwner =/= NewOwner,\n                          EpochUpdate,\n                          \"the owner changed without an epoch update: \"\n                              ++ atom_to_list(Writer)),\n                    ?WARN(OldRange =/= NewRange,\n                         EpochUpdate,\n                         \"the range changed without an epoch update: \"\n                             ++ atom_to_list(Writer)),\n                    ?WARN(OldAux =/= NewAux,\n                         EpochUpdate,\n                         \"the aux changed without an epoch update: \"\n                             ++ atom_to_list(Writer)),\n                    ?WARN(OldTimeout =/= NewTimeout,\n                         VersionUpdate orelse EpochUpdate,\n                         \"the timeout changed without a version update: \"\n                             ++ atom_to_list(Writer)),\n                    ?WARN(OldTimeout =/= NewTimeout,\n                         OldTimeout < NewTimeout,\n                         \"the new timeout is not newer than the old: \"\n                             ++ atom_to_list(Writer)),\n                    {true, null};\n                % epoch and/or version did not match: give useful error message\n                #lease{id = Id0, epoch = E0, owner = O0, range = R0, aux = Aux0,\n                       version = V0, timeout = T0} ->\n                    if\n                        Id0 =/= OldId ->\n                            {false, {unexpected_id, Current, Next}};\n                        O0 =/= OldOwner ->\n                            {false, {unexpected_owner, Current, Next}};\n                        Aux0 =/= OldAux ->\n                            {false, {unexpected_aux, Current, Next}};\n                        R0 =/= OldRange ->\n                            {false, {unexpected_range, Current, Next}};\n                        T0 =/= OldTimeout ->\n                            {false, {unexpected_timeout, Current, Next}};\n                        E0 =/= OldEpoch ->\n                            {false, {unexpected_epoch, Current, Next}};\n                        V0 =/= OldVersion ->\n                            {false, {unexpected_version, Current, Next}}%;\n                        %not (T0 < NewTimeout) ->\n                        %    {false, {timeout_is_not_newer_than_current_lease, Current, Next}}\n                    end\n            end\n    end.\n\n-spec cc_delete_lease(lease_t(), atom()) ->\n                             content_check_t(). %% content check\ncc_delete_lease(#lease{id=OldId,owner=OldOwner,aux = OldAux,range=OldRange,\n                                          epoch=OldEpoch,version=OldVersion,timeout=OldTimeout} = Old,\n                                   Writer) ->\n    fun(Current, _WriteFilter, Next) ->\n            New = prbr_bottom,\n            case Current of\n                %% check for re-write\n                New -> %% Current =:= New\n                    log:pal(\"re-write in CC:~n~w~n~w~n~w~n~w~n~w~n\",\n                            [Current, Next, Old, New, Writer]),\n                    {true, null};\n                %% check that epoch and version match with Old\n                #lease{epoch = E0, version = V0} when  E0 =:= OldEpoch andalso V0 =:= OldVersion ->\n                    {true, null};\n                % epoch and/or version did not match: give useful error message\n                #lease{id = Id0, epoch = E0, owner = O0, range = R0, aux = Aux0,\n                       version = V0, timeout = T0} ->\n                    if\n                        Id0 =/= OldId ->\n                            {false, {unexpected_id, Current, Next}};\n                        O0 =/= OldOwner ->\n                            {false, {unexpected_owner, Current, Next}};\n                        Aux0 =/= OldAux ->\n                            {false, {unexpected_aux, Current, Next}};\n                        R0 =/= OldRange ->\n                            {false, {unexpected_range, Current, Next}};\n                        T0 =/= OldTimeout ->\n                            {false, {unexpected_timeout, Current, Next}};\n                        E0 =/= OldEpoch ->\n                            {false, {unexpected_epoch, Current, Next}};\n                        V0 =/= OldVersion ->\n                            {false, {unexpected_version, Current, Next}}%;\n                        %not (T0 < NewTimeout) ->\n                        %    {false, {timeout_is_not_newer_than_current_lease, Current, Next}}\n                    end\n            end\n    end.\n\n\n% @doc only for unittests\n-spec is_valid_update(non_neg_integer(), non_neg_integer()) ->\n    fun ((Current::lease_t(), WriteFilter::prbr:write_filter(), Next::lease_t()) ->\n                {Result::boolean(), update_failed_reason() | null}). %% content check\nis_valid_update(CurrentEpoch, CurrentVersion) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    fun (#lease{epoch = E0}, _, _)                     when E0 =/= CurrentEpoch ->\n            %% log:pal(\"is_valid_update: expected ~p, got ~p\", [CurrentEpoch, E0]),\n            {false, epoch_or_version_mismatch};\n        (#lease{version = V0}, _, _)                   when V0 =/= CurrentVersion->\n            %% log:pal(\"is_valid_update: expected ~p, got ~p\", [CurrentVersion, V0]),\n            {false, epoch_or_version_mismatch};\n        (_Current, _WriteFilter, _Next) ->\n            {true, null}\n    end.\n\n%@doc only content check which allows to create a new lease\n-spec is_valid_split_step1() ->\n        fun ((Current::lease_t(), WriteFilter::prbr:write_filter(), Next::lease_t()) ->\n                     {Result::boolean(), split_step1_failed_reason() | null}). %% content check\nis_valid_split_step1() ->\n    fun (Current, _WriteFilter, _Next) ->\n            case Current == prbr_bottom of\n                true ->\n                    {true, null};\n                false ->\n                    {false, lease_already_exists}\n            end\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% util\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec read(lease_id()) -> api_tx:read_result().\nread(Key) ->\n    read(Key, self()),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({qread_done, _ReqId, _Round, _OldWriteRound, Value},\n                       case Value of\n                           no_value_yet -> {fail, not_found};\n                           _ -> {ok, Value}\n                       end\n                      )\n        end.\n\n-spec read(lease_id(), comm:erl_local_pid()) -> ok.\nread(Key, Pid) ->\n    %% decide which lease db is responsible, ie. if the key is from\n    %% the first quarter of the ring, use lease_db1, if from 2nd\n    %% quarter -> use lease_db2, ...\n    DB = rbrcseq:get_db_for_id(lease_db, Key),\n    %% perform qread\n    rbrcseq:qread(DB, Pid, Key, ?MODULE).\n\n%% write(Key, Value, ContentCheck) ->\n%%     %% decide which lease db is responsible, ie. if the key is from\n%%     %% the first quarter of the ring, use lease_db1, if from 2nd\n%%     %% quarter -> use lease_db2, ...\n%%         DB = rbrcseq:get_db_for_id(lease_db, Key),\n%%     rbrcseq:qwrite(DB, self(), Key, ?MODULE, ContentCheck, Value),\n%%     trace_mpath:thread_yield(),\n%%     receive\n%%         ?SCALARIS_RECV({qwrite_done, _ReqId, _Round, _Value}, {ok} ) %%;\n%%         %% ?SCALARIS_RECV({qwrite_deny, _ReqId, _Round, _Value, Reason}, {fail, timeout} )\n%%         end.\n\n%% -spec write(lease_id(), lease_t()) -> api_tx:write_result().\n%% write(Key, Value) ->\n%%     write(Key, Value, fun l_on_cseq:is_valid_state_change/3).\n\n-spec add_first_lease_to_db(?RT:key(), dht_node_state:state()) ->\n                                  dht_node_state:state().\nadd_first_lease_to_db(Id, State) ->\n    DB = rbrcseq:get_db_for_id(lease_db, Id),\n    Lease = #lease{id      = Id, %% set to 0 in dht_node\n                   epoch   = 1,\n                   owner   = comm:this(),\n                   range   = intervals:all(),\n                   aux     = empty,\n                   version = 1,\n                   timeout = new_timeout()\n                  },\n    DBHandle = dht_node_state:get(State, DB),\n    _ = [ begin\n              Entry = prbr:new(X, Lease),\n              prbr:set_entry(Entry, DBHandle)\n          end || X <- ?RT:get_replica_keys(Id) ],\n    dht_node_state:set_lease_list(State,\n                                  lease_list:make_lease_list(Lease, [], [])).\n\n-spec unittest_create_lease(?RT:key()) -> lease_t().\nunittest_create_lease(Id) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    #lease{id      = Id,\n           epoch   = 1,\n           owner   = comm:this(),\n           range   = intervals:all(),\n           aux     = empty,\n           version = 1,\n           timeout = new_timeout()\n          }.\n\n-spec unittest_create_lease_with_range(?RT:key(), ?RT:key(), comm:mypid_plain()) -> lease_t().\nunittest_create_lease_with_range(From, To, Owner) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    Range = node:mk_interval_between_ids(From, To),\n    Id = id(Range),\n    #lease{id      = Id,\n           epoch   = 1,\n           owner   = Owner,\n           range   = Range,\n           aux     = empty,\n           version = 1,\n           timeout = new_timeout()\n          }.\n\n-spec new_timeout() -> erlang_timestamp().\nnew_timeout() ->\n    util:time_plus_s(os:timestamp(), delta()).\n\n-spec is_a_lease(term()) -> boolean().\nis_a_lease(L) ->\n    is_record(L, lease).\n\n-spec get_version(lease_t()) -> non_neg_integer().\nget_version(#lease{version=Version}) -> Version.\n\n-spec set_version(lease_t(), non_neg_integer()) -> lease_t().\nset_version(Lease, Version) -> Lease#lease{version=Version}.\n\n-spec get_epoch(lease_t()) -> non_neg_integer().\nget_epoch(#lease{epoch=Epoch}) -> Epoch.\n\n-spec set_epoch(lease_t(), non_neg_integer()) -> lease_t().\nset_epoch(Lease, Epoch) -> Lease#lease{epoch=Epoch}.\n\n-spec set_timeout(lease_t()) -> lease_t().\nset_timeout(Lease) -> Lease#lease{timeout=new_timeout()}.\n\n-spec get_timeout(lease_t()) -> erlang_timestamp().\nget_timeout(#lease{timeout=Timeout}) -> Timeout.\n\n-spec get_pretty_timeout(lease_t()) -> string().\nget_pretty_timeout(L) ->\n    format_utc_timestamp(get_timeout(L)).\n\n-spec get_id(lease_t()) -> ?RT:key().\nget_id(#lease{id=Id}) -> Id.\n\n-spec get_owner(lease_t()) -> comm:mypid_plain() | nil.\nget_owner(#lease{owner=Owner}) -> Owner.\n\n-spec set_owner(lease_t(), comm:mypid_plain() | nil) -> lease_t().\nset_owner(L, NewOwner) -> L#lease{owner=NewOwner}.\n\n-spec get_aux(lease_t()) -> lease_aux().\nget_aux(#lease{aux=Aux}) -> Aux.\n\n-spec set_aux(lease_t(), lease_aux()) -> lease_t().\nset_aux(L, Aux) -> L#lease{aux=Aux}.\n\n-spec get_range(lease_t()) -> intervals:interval().\nget_range(#lease{range=Range}) -> Range.\n\n-spec set_range(lease_t(), intervals:interval()) -> lease_t().\nset_range(L, Range) -> L#lease{range=Range}.\n\n-spec is_valid(lease_t()) -> boolean().\nis_valid(L) ->\n    case L of\n        empty ->\n            %% should not happen but it does\n            log:log(\"loncq: you are calling is_valid on an empty lease~n\"),\n            false;\n        _ ->\n            os:timestamp() <  L#lease.timeout\n    end.\n\n-spec has_timed_out(lease_t()) -> boolean().\nhas_timed_out(L) ->\n    not is_valid(L).\n\n-spec id(intervals:interval()) -> ?RT:key().\nid([all]) -> ?MINUS_INFINITY;\nid(X) ->\n    {_, _, Id, _} = intervals:get_bounds(X),\n    Id.\n\n-spec split_range(intervals:interval()) ->\n                         {ok, intervals:interval(), intervals:interval()}.\nsplit_range(Range) ->\n    {_, Low, Hi, _} = intervals:get_bounds(Range),\n    Key = ?RT:get_split_key(Low, Hi, {1,2}),\n    R1 = node:mk_interval_between_ids(Low, Key),\n    R2 = node:mk_interval_between_ids(Key, Hi),\n    {ok, R1, R2}.\n\n-spec get_active_lease(dht_node_state:state()) -> lease_list:active_lease_t().\nget_active_lease(State) ->\n    LeaseList = dht_node_state:get(State, lease_list),\n    lease_list:get_active_lease(LeaseList).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% qwrite_done messages should always be received by the current owner\n% of the message. The message handler can directly change the lease\n% list in the dht_node. The next message will than be handled under\n% the new lease list.\n%\n% If we would update the lease of another node and the qwrite_done is\n% received by us, the system assumes that the lease was updated but\n% the owner will still work according to the old version of the\n% lease. In some cases such a remote-modify is acceptable and correct.\n% E.g. if we use a remote-modify to extend the lease's range, the\n% remote node can continue to work with his old lease until the next\n% renewal. On renewal, he will notice that it fails because the lease\n% changed and additionally he will get notified of the range\n% extension.\n%\n% The goal is to limited remote-modifies to correct operations,\n% i.e. renewals, aux updates and range extensions.\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% Lease agreement\n%\n% Rule 16: The lease's id never changes.\n% Rule 17: Every content check has to perform the checks in the same order.\n%          1. Does the lease exist?\n%          2. Is the value of the owner field as expected?\n%          3. Is the value of the aux   field as expected?\n%          4. Is the value of the range field as expected?\n%          5. Are the values of the epoch and version fields as expected?\n%          6. Is the proposed timeout newer than the current one and is it in\n%             the future (for debugging only)?\n%          7. Only, now it may check for debug purposes whether the proposed\n%             changes are acceptable.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% TODO\n%\n% - fix merge protocol\n% - improve error handling for deny in renewal\n% - do i need to check for timeout_is_not_in_the_future?\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%        (_Current, _WriteFilter, Next) ->\n%            case (os:timestamp() <  Next#lease.timeout) of\n%                false ->\n%                    {false, timeout_is_not_in_the_future};\n%                true ->\n%                    {true, null}\n%            end\n\n-spec format_utc_timestamp(erlang_timestamp()) -> string().\nformat_utc_timestamp({_,_,Micro} = TS) ->\n    {{Year,Month,Day},{Hour,Minute,Second}} = calendar:now_to_local_time(TS),\n    Mstr = element(Month,{\"Jan\",\"Feb\",\"Mar\",\"Apr\",\"May\",\"Jun\",\"Jul\", \"Aug\",\"Sep\",\n                          \"Oct\",\"Nov\",\"Dec\"}),\n    lists:flatten(io_lib:format(\"~2w ~s ~4w ~2w:~2..0w:~2..0w.~6..0w\",\n                  [Day,Mstr,Year,Hour,Minute,Second,Micro])).\n\n% @doc updates lease and tries to use qwrite_fast whenever\n%      possible. almost all leases updates use this routine\n-spec update_lease(ReplyTo::comm:erl_local_pid(),\n                   ContentCheck::content_check_t(),\n                   Old::lease_t(), New::lease_t() | prbr_bottom, dht_node_state:state()) -> ok.\nupdate_lease(ReplyTo, ContentCheck, Old, New, State) ->\n    ?DBG_ASSERT(New =:= prbr_bottom orelse get_id(Old) =:= get_id(New)), % the lease id may not be changed\n    LeaseId = get_id(Old), %% New could prbr_bottom\n    DB = rbrcseq:get_db_for_id(lease_db, LeaseId),\n    case lease_list:get_next_round(LeaseId, State) of\n        failed ->\n            rbrcseq:qwrite     (DB, ReplyTo, LeaseId, ?MODULE, ContentCheck, New);\n        NextRound ->\n            rbrcseq:qwrite_fast(DB, ReplyTo, LeaseId, ?MODULE, ContentCheck, New, NextRound, Old)\n    end.\n\n% triggers renew of lease and updates known round number for the lease\n-spec renew_and_update_round(lease_t(), pr:pr(), active | passive, dht_node_state:state()) ->\n                                    dht_node_state:state().\nrenew_and_update_round(Lease, Round, _Mode, State) ->\n    %lease_renew(self(), Lease, Mode),\n    lease_list:update_next_round(l_on_cseq:get_id(Lease), Round, State).\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    case config:read(leases) of\n        true ->\n            config:cfg_is_integer(leases_delta) andalso\n                config:cfg_is_greater_than(leases_delta, 0);\n        _ ->\n            true\n    end.\n"
  },
  {
    "path": "src/cp/lease_checker.erl",
    "content": "% @copyright 2012-2017 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc check ring.\n%% @end\n%% @version $Id$\n-module(lease_checker).\n-author('schuett@zib.de').\n-vsn('$Id:$ ').\n\n-include(\"scalaris.hrl\").\n\n\n-export([check_leases_for_all_nodes/0]).\n-export([check_leases_for_the_ring/0]).\n-export([check_leases_for_the_ring/1]).\n-export([get_random_save_node/0]).\n\n-export([get_relative_range_unittest/1]).\n-export([get_dht_node_state_unittest/2]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% public api\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec check_leases_for_all_nodes() -> boolean().\ncheck_leases_for_all_nodes() ->\n    io:format(\"======= node local test ==========~n\"),\n    lists:all(fun (B) -> B end, [ check_local_leases(DHTNode) || DHTNode <- all_dht_nodes() ]).\n\n-spec check_leases_for_the_ring() -> boolean().\ncheck_leases_for_the_ring() ->\n    io:format(\"======= global test ==========~n\"),\n    lease_checker(admin:number_of_nodes()).\n\n-spec check_leases_for_the_ring(pos_integer()) -> boolean().\ncheck_leases_for_the_ring(TargetSize) ->\n    io:format(\"======= global test ==========~n\"),\n    lease_checker(TargetSize).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% check leases\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ncheck_local_leases(DHTNode) ->\n    case get_dht_node_state(DHTNode, [lease_list, my_range]) of\n        false ->\n            false;\n        {true, [{lease_list, LeaseList}, {my_range, MyRange}]} ->\n            ActiveLease = lease_list:get_active_lease(LeaseList),\n            PassiveLeases = lease_list:get_passive_leases(LeaseList),\n            ActiveInterval = case ActiveLease of\n                                 empty ->\n                                     intervals:empty();\n                                 _ ->\n                                     l_on_cseq:get_range(ActiveLease)\n                             end,\n            LocalCorrect = MyRange =:= ActiveInterval,\n            RelRange = get_relative_range(ActiveInterval),\n            io:format(\"rm =:= leases:~w~n active lease=~p~n my_range    =~p~n rel_range     =~p~n\",\n                      [LocalCorrect, ActiveInterval, MyRange, RelRange]),\n            length(PassiveLeases) == 0 andalso LocalCorrect\n    end.\n\nlease_checker(TargetSize) ->\n    LeaseLists = get_all_leases(),\n    ActiveLeases  = [lease_list:get_active_lease(LL)  || LL  <- LeaseLists],\n    PassiveLeases = lists:flatmap(fun lease_list:get_passive_leases/1, LeaseLists),\n    ActiveIntervals = [l_on_cseq:get_range(Lease) || Lease <- ActiveLeases, Lease =/= empty],\n    NormalizedActiveIntervals = intervals:union(ActiveIntervals),\n    %log:log(\"Lease-Checker: ~w ~w ~w\", [ActiveLeases, ActiveIntervals, PassiveLeases]),\n    %ct:pal(\"ActiveIntervals: ~p\", [ActiveIntervals]),\n    %ct:pal(\"PassiveLeases: ~p\", [PassiveLeases]),\n    IsAll = intervals:is_all(NormalizedActiveIntervals),\n    IsDisjoint = is_disjoint(ActiveIntervals),\n    HaveAllActiveLeases = length(ActiveLeases) == TargetSize,\n    HaveNoPassiveLeases = length(PassiveLeases) == 0,\n    HaveAllAuxEmpty = lists:all(fun(L) ->\n                                        L =/= empty andalso l_on_cseq:get_aux(L) =:= empty\n                                end, ActiveLeases),\n    % ct:pal(\"lease checker: ~w ~w ~w ~w~n~w~n~w~n\", [IsAll, IsDisjoint, HaveAllActiveLeases, HaveNoPassiveLeases,PassiveLeases, NormalizedActiveIntervals]),\n    case IsAll of\n        false ->\n            %print_all_active_leases(),\n            ok;\n        true ->\n            ok\n    end,\n    io:format(\"complete ring covered by leases: ~w~n\", [IsAll]),\n    io:format(\"all aux-fields are empty       : ~w~n\", [HaveAllAuxEmpty]),\n    io:format(\"no leases overlap              : ~w~n\", [IsDisjoint]),\n    io:format(\"each node has one active lease : ~w~n\", [HaveAllActiveLeases]),\n    io:format(\"no passive leases              : ~w~n\", [HaveNoPassiveLeases]),\n    case HaveAllAuxEmpty of\n        false ->\n            io:format(\"aux fields: ~w~n\", [[ l_on_cseq:get_aux(L) || L <- ActiveLeases,\n                                                                     L =/= empty ]]);\n        true ->\n            ok\n    end,\n    IsAll andalso\n        HaveAllAuxEmpty andalso\n        IsDisjoint andalso\n        HaveAllActiveLeases andalso % @todo enable after garbage collection is implemented\n        HaveNoPassiveLeases.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% utility functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n\n%@doc returns a random node which covers a minority of the key-space\n-spec get_random_save_node() -> comm:mypid() | failed.\nget_random_save_node() ->\n    R = config:read(replication_factor),\n    SaveFraction = quorum:minority(R) / R,\n    LeaseNodes = [{ActiveLease, Node} || Node <- all_dht_nodes(),\n                                         {true, LL} <- [get_dht_node_state(Node, lease_list)],\n                                         ActiveLease <- [lease_list:get_active_lease(LL)],\n                                            ActiveLease =/= empty],\n\n    SaveNodes = [{Range, Node} || {Lease, Node} <- LeaseNodes,\n                                   Range <- [get_relative_range(l_on_cseq:get_range(Lease))],\n                                    Range =< SaveFraction],\n\n    case SaveNodes of\n        [] ->\n            failed;\n        _ ->\n            Rand = randoms:uniform(),\n            ReturnNode = if Rand < 0.5 ->\n                                 _UnsafestSafeNode = lists:max(SaveNodes);\n                            true ->\n                                 _RandomSafeNode = util:randomelem(SaveNodes)\n                         end,\n            element(2, ReturnNode)\n    end.\n\n-spec is_disjoint([intervals:interval()]) -> boolean().\nis_disjoint([]) ->\n    true;\nis_disjoint([H | T]) ->\n    is_disjoint(H, T) andalso\n        is_disjoint(T).\n\nis_disjoint(_I, []) ->\n    true;\nis_disjoint(I, [H|T]) ->\n    intervals:is_empty(intervals:intersection(I,H))\n        andalso is_disjoint(I, T).\n\n-spec get_relative_range_unittest(intervals:interval()) -> float().\nget_relative_range_unittest(ActiveInterval) ->\n    ?ASSERT(util:is_unittest()),\n    get_relative_range(ActiveInterval).\n\n-spec get_relative_range(intervals:interval()) -> float().\nget_relative_range(ActiveInterval) ->\n    case intervals:empty() of\n        ActiveInterval ->\n            0.0 / ?RT:n();\n        _ ->\n            {_, Begin, End, _} = intervals:get_bounds(ActiveInterval),\n            ?RT:get_range(Begin, End) / ?RT:n()\n    end.\n\n-spec get_dht_node_state_unittest(comm:mypid(), atom() | list(atom())) -> term() | list(term()).\nget_dht_node_state_unittest(Pid, What) ->\n    ?ASSERT(util:is_unittest()),\n    get_dht_node_state(Pid, What).\n\n-spec get_dht_node_state(comm:mypid(), atom() | list(atom())) -> term() | list(term()).\nget_dht_node_state(Pid, What) ->\n    false = trace_mpath:infected(),\n    Cookie = {os:timestamp(), randoms:getRandomInt()},\n    This = comm:reply_as(comm:this(), 2, {get_dht_node_state_response, '_', Cookie}),\n    comm:send(Pid, {get_state, This, What}),\n    trace_mpath:thread_yield(),\n    Result =\n        receive\n            ?SCALARIS_RECV({get_dht_node_state_response, {get_state_response, Data}, Cookie},% ->\n                {true, Data})\n        after 50 ->\n                false\n        end,\n    % drain message queue\n    drain_message_queue(),\n    Result.\n\ndrain_message_queue() ->\n    false = trace_mpath:infected(),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({get_dht_state_response, _Data, _Cookie},% ->\n                       ok)\n    after 0 ->\n            ok\n    end.\n\n-spec get_all_leases() -> list(lease_list:lease_list()).\nget_all_leases() ->\n    % short for lists:filtermap/2\n    [ element(2, L) || Node <- all_dht_nodes(),\n                       (L = get_dht_node_state(Node, lease_list)) =/= false ].\n\n-spec all_dht_nodes() -> list(comm:mypid()).\nall_dht_nodes() ->\n    mgmt_server:node_list(),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({get_list_response, Nodes},% ->\n            Nodes)\n    end.\n"
  },
  {
    "path": "src/cp/lease_checker2.erl",
    "content": "% @copyright 2012-2016 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc check ring.\n%% @end\n%% @version $Id$\n-module(lease_checker2).\n-author('schuett@zib.de').\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-record(leases_state_t, {\n          last_check  = ?required(lease_state_t, last_check) :: erlang:timestamp(),\n          node_infos  = ?required(lease_state_t, node_infos) :: node_list(),\n          last_failed = ?required(lease_state_t, last_faile) :: boolean()\n         }).\n\n-record(node_info_t, {\n          lease_list  = ?required(lease_state_t, lease_list) :: lease_list:lease_list(),\n          my_range    = ?required(lease_state_t, my_range)   :: intervals:interval()\n         }).\n\n-type node_list() :: gb_trees:tree(comm:mypid(), node_info() | empty).\n-type leases_state() :: #leases_state_t{}.\n-type node_info() :: #node_info_t{}.\n\n-type option() :: {ring_size, pos_integer()} | {ring_size_range, pos_integer(), pos_integer()}.\n-type options() :: list(option()).\n\n-export_type([leases_state/0]).\n\n-export([wait_for_clean_leases/2]).\n\n-export([get_kv_db/0, get_kv_db/1]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% public api\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec wait_for_clean_leases(WaitTimeInMs::pos_integer(), Options::options()) -> ok.\nwait_for_clean_leases(WaitTimeInMs, Options) ->\n    ?ASSERT(not gen_component:is_gen_component(self())),\n    wait_for_clean_leases(WaitTimeInMs, Options, true, create_new_state()).\n\n-spec get_kv_db() -> ok.\nget_kv_db() ->\n    KVDBs = [ get_dht_node_state(Pid, kv_db) || Pid <- all_dht_nodes()],\n    Data = [prbr:tab2list(DB) || {true, DB} <- KVDBs, DB =/= false],\n    FlattenedData = lists:flatten(Data),\n    io:format(\"kv-pairs: ~p~n\", [length(FlattenedData)]),\n    Empties = [ DB || DB <- KVDBs, DB =:= false],\n    Bottoms = [Value || {_Key, Value} <- FlattenedData, Value =:= prbr_bottom],\n    io:format(\"falses: ~p~n\", [length(Empties)]),\n    io:format(\"prbr_bottoms: ~p~n\", [length(Bottoms)]),\n    %% io:format(\"data: ~p~n\", [FlattenedData]),\n    ok.\n\n-spec get_kv_db(term()) -> ok.\nget_kv_db(Pid) ->\n    io:format(\"~p~n\", [Pid]),\n    {true, DB} = get_dht_node_state(comm:make_global(Pid), kv_db),\n    io:format(\"kv-pairs: ~p~n\", [length(prbr:tab2list(DB))]),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% internal api\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec wait_for_clean_leases(WaitTimeInMs::pos_integer(), Options::options(),\n                            First::boolean(), State::leases_state()) -> ok.\nwait_for_clean_leases(WaitTimeInMs, Options, First, State) ->\n    case check_leases(State, Options, First) of\n        {true, _}  -> ok;\n        {false, NewState} ->\n            WaitID = uid:get_pids_uid(),\n            comm:send_local_after(WaitTimeInMs, self(), {continue_wait, WaitID}),\n            trace_mpath:thread_yield(),\n            receive\n                ?SCALARIS_RECV({continue_wait, WaitID},% ->\n                               wait_for_clean_leases(WaitTimeInMs, Options, false, NewState))\n            end\n    end.\n\n-spec check_leases(OldState::leases_state(), Options::options(), First::boolean()) ->\n                          {boolean(), leases_state()}.\ncheck_leases(OldState, Options, First) ->\n    LastFailed = OldState#leases_state_t.last_failed,\n    NewState = create_new_state(),\n    TargetSize = renderTargetSize(Options),\n    case {gb_trees:size(OldState#leases_state_t.node_infos),\n          gb_trees:size(NewState#leases_state_t.node_infos)} of\n        {Old, Old} ->\n            io:format(\"================= check leases (~p of ~p) ====================~n\",\n                      [Old, TargetSize]);\n        {Old, New} ->\n          io:format(\"================= check leases ((~p -> ~p) of ~p) ====================~n\",\n                   [Old, New, TargetSize])\n     end,\n    case First of\n        true -> io:format(\"begin existing nodes~n\"),\n                describe_nodes(OldState#leases_state_t.node_infos),\n                io:format(\"end existing nodes~n\");\n        false -> ok\n    end,\n    Changed =\n        case compare_node_lists(OldState#leases_state_t.node_infos,\n                                NewState#leases_state_t.node_infos) of\n            true -> false;\n            false ->\n                io:format(\"begin diff~n\"),\n                describe_lease_states_diff(OldState, NewState),\n                io:format(\"end diff~n\"),\n                true\n        end,\n    Verbose = First orelse not LastFailed orelse Changed,\n    Res = check_state(NewState, Verbose, Options),\n    io:format(\"check_state returned(verbose=~p) ~p~n\", [Verbose, Res]),\n    {Res, NewState#leases_state_t{last_failed=not Res}}.\n\n-spec check_state(State::leases_state(), Verbose::boolean(),\n                  Options::options()) -> boolean().\ncheck_state(State, Verbose, Options) ->\n    case check_leases_locally(State, Verbose) of\n        true ->\n            case check_leases_globally(State, Verbose, Options) of\n                true ->\n                    true;\n                false -> io:format(\"check_leases_globally failed~n\"),\n                         false\n            end;\n        false -> io:format(\"check_leases_locally failed~n\"),\n                 false\n    end.\n    %% check_leases_locally(State, Verbose) andalso\n    %%     check_leases_globally(State, Verbose, TargetSize).\n\n-spec check_leases_locally(leases_state(), boolean()) -> boolean().\ncheck_leases_locally(#leases_state_t{node_infos=Nodes}, Verbose) ->\n    lists:foldl(fun ({Pid, Node}, Acc) ->\n                        Acc and check_local_leases(Pid, Node, Verbose)\n                end, true, gb_trees:to_list(Nodes)).\n\n-spec check_leases_globally(leases_state(), boolean(), options()) -> boolean().\ncheck_leases_globally(State, Verbose, Options) ->\n    lease_checker(State, Verbose, Options).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% compare functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc returns true iff the lists are equal\n-spec compare_node_lists(node_list(), node_list()) -> boolean().\ncompare_node_lists(Old, New) ->\n    OldPids = ordsets:from_list(gb_trees:keys(Old)),\n    NewPids = ordsets:from_list(gb_trees:keys(New)),\n\n    case OldPids =:= NewPids of\n        false -> false;\n        true ->\n            lists:foldl(fun (Pid, Acc) ->\n                                Acc andalso compare_node_infos(gb_trees:get(Pid, Old),\n                                                               gb_trees:get(Pid, New))\n                        end, true, ordsets:to_list(NewPids))\n    end.\n\n-spec compare_node_infos(node_info() | empty, node_info() | empty) -> boolean().\ncompare_node_infos(Old, New) ->\n    case {Old, New} of\n        {empty, empty} -> true;\n        {empty, New} -> false;\n        {Old, empty} -> false;\n        {Old, New}   -> Old#node_info_t.my_range =:= New#node_info_t.my_range\n                            andalso compare_lease_lists(Old#node_info_t.lease_list,\n                                                        New#node_info_t.lease_list)\n    end.\n\n-spec compare_lease_lists(L1::lease_list:lease_list(), L2::lease_list:lease_list()) -> boolean().\ncompare_lease_lists(L1, L2) ->\n    compare_leases(lease_list:get_active_lease(L1), lease_list:get_active_lease(L2))\n        andalso compare_passive_leases(lease_list:get_passive_leases(L1),\n                                       lease_list:get_passive_leases(L2)).\n\n-spec compare_passive_leases(L1::[l_on_cseq:lease_t()], L2::[l_on_cseq:lease_t()]) -> boolean().\ncompare_passive_leases(L1, L2) ->\n    Ids1 = [l_on_cseq:get_id(L) || L <- L1],\n    Ids2 = [l_on_cseq:get_id(L) || L <- L2],\n    SetOfIds1 = ordsets:from_list(Ids1),\n    SetOfIds2 = ordsets:from_list(Ids2),\n    case SetOfIds1 =:= SetOfIds2 of\n        true ->\n            %% @todo use lists:foldl\n            lists:all(fun (Bool) -> Bool end,\n                      lists:zipwith(fun (Lease1, Lease2) -> compare_leases(Lease1, Lease2) end,\n                                    L1, L2));\n        false ->\n            false\n    end.\n\n-spec compare_leases(L1::l_on_cseq:lease_t() | empty, L2::l_on_cseq:lease_t() | empty) -> boolean().\ncompare_leases(L1, L2) ->\n    case {L1, L2} of\n        {empty, empty} -> true;\n        {empty, L2} -> false;\n        {L1, empty} -> false;\n        _ ->\n            l_on_cseq:get_id(L1) =:= l_on_cseq:get_id(L2)\n                andalso l_on_cseq:get_owner(L1) =:= l_on_cseq:get_owner(L2)\n                andalso l_on_cseq:get_range(L1) =:= l_on_cseq:get_range(L2)\n                andalso l_on_cseq:get_aux(L1) =:= l_on_cseq:get_aux(L2)\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% describe things functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec describe_nodes(node_list()) -> ok.\ndescribe_nodes(Nodes) ->\n    _ = [ describe_node(Pid, Node) || {Pid, Node} <- gb_trees:to_list(Nodes) ],\n    ok.\n\n-spec describe_list_of_leases(Leases::[l_on_cseq:lease_t()], active | passive) -> ok.\ndescribe_list_of_leases(Leases, Type) ->\n    _ = [ describe_lease(L, Type) || L <- Leases ],\n    ok.\n\n-spec describe_lease(L1::l_on_cseq:lease_t(), active | passive) -> ok.\ndescribe_lease(Lease, _Type) ->\n    %% @todo use type parameter\n    Interval = l_on_cseq:get_range(Lease),\n    RelRange = get_relative_range(Interval),\n    Owner    = l_on_cseq:get_owner(Lease),\n    Aux      = l_on_cseq:get_aux(Lease),\n\n    io:format(\"    range:~p~n   rel_range:~p~n   owner:~p~n   aux:~p~n\", [Interval, RelRange, Owner, Aux]),\n    ok.\n\n%% @todo change to /1 with | empty\n-spec describe_node(Node::comm:mypid(), node_info() | empty) -> ok.\ndescribe_node(Pid, NodeInfo) ->\n    case NodeInfo of\n        empty -> ok;\n        _ ->\n            LeaseList = NodeInfo#node_info_t.lease_list,\n            MyRange   = NodeInfo#node_info_t.my_range,\n            ActiveLease = lease_list:get_active_lease(LeaseList),\n            PassiveLeases = lease_list:get_passive_leases(LeaseList),\n            ActiveInterval = case ActiveLease of\n                                 empty ->\n                                     intervals:empty();\n                                 _ ->\n                                     l_on_cseq:get_range(ActiveLease)\n                             end,\n            RelRange = get_relative_range(ActiveInterval),\n            Aux = case ActiveLease of\n                      empty -> no_lease;\n                      _ -> l_on_cseq:get_aux(ActiveLease)\n                  end,\n            LocalCorrect = MyRange =:= ActiveInterval,\n            io:format(\"  ~p~n\", [Pid]),\n            io:format(\"    rm =:= leases -> ~w~n\", [LocalCorrect]),\n            io:format(\"      active lease=~p~n\", [ActiveInterval]),\n            io:format(\"        my_range  =~p~n\", [MyRange]),\n            io:format(\"        rel_range =~p~n\", [RelRange]),\n            io:format(\"        aux       =~p~n\", [Aux]),\n            io:format(\"      passive     =~p~n\", [PassiveLeases]),\n            ok\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% describe differences functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec describe_lease_states_diff(leases_state(), leases_state()) -> ok.\ndescribe_lease_states_diff(Old, New) ->\n    %% PRE they differ\n    describe_nodes_diff(Old#leases_state_t.node_infos, New#leases_state_t.node_infos).\n\n-spec describe_nodes_diff(node_list(), node_list()) -> ok.\ndescribe_nodes_diff(OldNodeInfos, NewNodeInfos) ->\n    %% PRE they differ\n    OldPids = ordsets:from_list(gb_trees:keys(OldNodeInfos)),\n    NewPids = ordsets:from_list(gb_trees:keys(NewNodeInfos)),\n\n    Unchanged = ordsets:intersection(OldPids, NewPids),\n    FoundPids = ordsets:subtract(NewPids, OldPids),\n    LostPids  = ordsets:subtract(OldPids, NewPids),\n    %% lost\n    _ = case ordsets:size(LostPids) of\n        0 -> ok;\n        N -> Old = ordsets:to_list(LostPids),\n             io:format(\"lost ~p nodes: ~p~n\", [N, Old]),\n             describe_nodes(gb_trees_filter(fun (Pid, _Node) ->\n                                                    ordsets:is_element(Pid, LostPids) end,\n                                            OldNodeInfos))\n        end,\n    %% found\n    _ = case ordsets:size(FoundPids) of\n        0 -> ok;\n        N2 -> New = ordsets:to_list(FoundPids),\n             io:format(\"found ~p new nodes: ~p~n\", [N2, New]),\n             describe_nodes(gb_trees_filter(fun (Pid, _Node) ->\n                                                    ordsets:is_element(Pid, FoundPids) end,\n                                            NewNodeInfos)),\n             io:format(\"end found~n\", [])\n    end,\n    _ = case ordsets:size(Unchanged) of\n        0 -> ok;\n        _ ->\n            [ describe_node_diff(Pid, gb_trees:get(Pid, OldNodeInfos),\n                                 gb_trees:get(Pid, NewNodeInfos))\n              || Pid <- ordsets:to_list(Unchanged)]\n    end,\n    ok.\n\n-spec describe_node_diff(Node::comm:mypid(), OldNodeInfo::node_info() | empty,\n                         NewNodeInfo::node_info() | empty) -> ok.\ndescribe_node_diff(Node, OldNodeInfo, NewNodeInfo) ->\n    case {OldNodeInfo, NewNodeInfo} of\n        {empty, empty}       -> ok;\n        {empty, NewNodeInfo} ->\n            io:format(\"the node ~p has changed~n\", [Node]),\n            io:format(\"node info changed from empty to~n\"),\n            describe_node(Node, NewNodeInfo);\n        {OldNodeInfo, empty} ->\n            io:format(\"the node ~p has changed~n\", [Node]),\n            io:format(\"node info changed to empty from~n\"),\n            describe_node(Node, OldNodeInfo);\n        {OldNodeInfo, NewNodeInfo} ->\n            OldLeaseList = OldNodeInfo#node_info_t.lease_list,\n            NewLeaseList = NewNodeInfo#node_info_t.lease_list,\n            LeasesDiffer = not compare_lease_lists(OldLeaseList, NewLeaseList),\n            OldRange = OldNodeInfo#node_info_t.my_range,\n            NewRange = NewNodeInfo#node_info_t.my_range,\n            RangesDiffer = OldRange =/= NewRange,\n\n            case LeasesDiffer orelse RangesDiffer of\n                true ->\n                    io:format(\"the node ~p has changed~n\", [Node]),\n                    _ = case LeasesDiffer of\n                            true ->\n                                describe_lease_list_diff(OldLeaseList, NewLeaseList);\n                            false ->\n                                ok\n                        end,\n                    _ = case RangesDiffer of\n                            true -> io:format(\"    the nodes' range changed~n      ~p~n      ~p~n\",\n                                              [OldRange, NewRange]);\n                            false -> ok\n                        end,\n                    ok;\n                false ->\n                    ok\n            end\n    end.\n\n-spec describe_lease_list_diff(lease_list:lease_list(), lease_list:lease_list()) -> ok.\ndescribe_lease_list_diff(OldLeaseList, NewLeaseList) ->\n    describe_lease_diff(lease_list:get_active_lease(OldLeaseList),\n                        lease_list:get_active_lease(NewLeaseList), active),\n    describe_list_of_leases_diff(lease_list:get_passive_leases(OldLeaseList),\n                                 lease_list:get_passive_leases(NewLeaseList), passive),\n    ok.\n\n-spec describe_lease_diff(l_on_cseq:lease_t() | empty, l_on_cseq:lease_t() | empty,\n                          active | passive) -> ok.\ndescribe_lease_diff(OldLease, NewLease, Type) ->\n    case {OldLease, NewLease} of\n        {empty, empty} -> ok;\n        {empty, NewLease} -> io:format(\"nyi3~n\");\n        {OldLease, empty} -> io:format(\"nyi4~n\");\n        {_, _} ->\n            case compare_leases(OldLease, NewLease) of\n                true -> ok;\n                false ->\n                    io:format(\"  an ~p lease has changed~n\", [Type]),\n                    case l_on_cseq:get_id(OldLease) =:= l_on_cseq:get_id(NewLease) of\n                        true -> ok;\n                        false ->\n                            io:format(\"    the id has changed~n      ~p~n      ~p~n\",\n                                      [l_on_cseq:get_id(OldLease),\n                                       l_on_cseq:get_id(NewLease)])\n                    end,\n                    case l_on_cseq:get_owner(OldLease) =:= l_on_cseq:get_owner(NewLease) of\n                        true -> ok;\n                        false ->\n                            io:format(\"    the owner has changed~n      ~p~n      ~p~n\",\n                                      [l_on_cseq:get_owner(OldLease),\n                                       l_on_cseq:get_owner(NewLease)])\n                    end,\n                    case l_on_cseq:get_range(OldLease) =:= l_on_cseq:get_range(NewLease) of\n                        true -> ok;\n                        false ->\n                            io:format(\"    the range has changed from~n    ~p~n    ->~n    ~p~n\",\n                                      [l_on_cseq:get_range(OldLease),\n                                       l_on_cseq:get_range(NewLease)])\n                    end,\n                    case l_on_cseq:get_aux(OldLease) =:= l_on_cseq:get_aux(NewLease) of\n                        true -> ok;\n                        false ->\n                            io:format(\"    the aux has changed~n    ~p~n    ->~n    ~p~n\",\n                                      [l_on_cseq:get_aux(OldLease),\n                                       l_on_cseq:get_aux(NewLease)])\n                    end\n            end\n    end,\n    ok.\n\n-spec describe_list_of_leases_diff([l_on_cseq:lease_t()], [l_on_cseq:lease_t()],\n                                   active | passive) -> ok.\ndescribe_list_of_leases_diff(OldLeases, NewLeases, Type) ->\n    OldIds = ordsets:from_list([l_on_cseq:get_id(L) || L <- OldLeases]),\n    NewIds = ordsets:from_list([l_on_cseq:get_id(L) || L <- NewLeases]),\n\n    Unchanged = ordsets:intersection(OldIds, NewIds),\n    FoundIds = ordsets:subtract(NewIds, OldIds),\n    LostIds  = ordsets:subtract(OldIds, NewIds),\n\n    %% lost\n    _ = case ordsets:size(LostIds) of\n        0 -> ok;\n        N -> io:format(\"  lost ~p passive leases: ~p~n\", [N, ordsets:to_list(LostIds)]),\n             describe_list_of_leases(lists:filter(fun (L) ->\n                                                          ordsets:is_element(l_on_cseq:get_id(L),\n                                                                             LostIds)\n                                                  end, OldLeases), Type)\n    end,\n    %% found\n    _ = case ordsets:size(FoundIds) of\n        0 -> ok;\n        N2 -> io:format(\"  found ~p new passive leases: ~p~n\", [N2, ordsets:to_list(FoundIds)]),\n              describe_list_of_leases(lists:filter(fun (L) ->\n                                                           ordsets:is_element(l_on_cseq:get_id(L),\n                                                                              FoundIds)\n                                                   end, NewLeases), Type)\n    end,\n    _ = case ordsets:size(Unchanged) of\n        0 -> ok;\n        _ -> OL = lists:sort(fun (L1, L2) -> l_on_cseq:get_id(L1) < l_on_cseq:get_id(L2) end,\n                             OldLeases),\n             NL = lists:sort(fun (L1, L2) -> l_on_cseq:get_id(L1) < l_on_cseq:get_id(L2) end,\n                             NewLeases),\n             [ describe_lease_diff(L1,L2, Type) || L1 <- OL, L2 <- NL,\n                                                   ordsets:is_element(l_on_cseq:get_id(L1),\n                                                                      Unchanged)]\n    end,\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% state handling\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec create_new_state() -> leases_state().\ncreate_new_state() ->\n    Nodes = lists:foldl(fun (DHTNode, Tree) ->\n                                Info = create_node_info(DHTNode),\n                                gb_trees:insert(DHTNode, Info, Tree)\n                        end, gb_trees:empty(), all_dht_nodes()),\n    #leases_state_t{last_check = os:timestamp(), node_infos=Nodes, last_failed=false}.\n\n-spec create_node_info(comm:mypid()) -> node_info() | empty.\ncreate_node_info(DHTNode) ->\n    case get_dht_node_state(DHTNode, [lease_list, my_range]) of\n        false ->\n            empty;\n        {true, [{lease_list, LeaseList}, {my_range, MyRange}]} ->\n            #node_info_t{lease_list = LeaseList, my_range = MyRange}\n    end.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% check leases\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec check_local_leases(comm:mypid(), node_info(), boolean()) -> boolean().\ncheck_local_leases(Pid, NodeInfo, Verbose) ->\n    case NodeInfo of\n        empty ->\n            false;\n        #node_info_t{lease_list=LeaseList, my_range=MyRange} ->\n            ActiveLease = lease_list:get_active_lease(LeaseList),\n            PassiveLeases = lease_list:get_passive_leases(LeaseList),\n            ActiveInterval = case ActiveLease of\n                                 empty ->\n                                     intervals:empty();\n                                 _ ->\n                                     l_on_cseq:get_range(ActiveLease)\n                             end,\n            LocalCorrect = MyRange =:= ActiveInterval,\n            RelRange = get_relative_range(ActiveInterval),\n            case length(PassiveLeases) == 0 andalso LocalCorrect of\n                true -> true;\n                false ->\n                    case Verbose of\n                        true ->\n                            case ActiveLease of\n                                empty ->\n                                    io:format(\"the active lease is empty~n\");\n                                _ ->\n                                    Aux = l_on_cseq:get_aux(ActiveLease),\n                                    io:format(\"  ~p~n\", [Pid]),\n                                    io:format(\"    rm =:= leases -> ~w~n\", [LocalCorrect]),\n                                    io:format(\"      active lease=~p~n\", [ActiveInterval]),\n                                    io:format(\"        my_range  =~p~n\", [MyRange]),\n                                    io:format(\"        rel_range =~p~n\", [RelRange]),\n                                    io:format(\"        aux       =~p~n\", [Aux]),\n                                    io:format(\"      passive     =~p~n\", [PassiveLeases])\n                            end;\n                        false ->\n                            ok\n                    end,\n                    false\n            end\n    end.\n\n-spec lease_checker(State::leases_state(), Verbose::boolean(),\n                    Options::options()) -> boolean().\nlease_checker(#leases_state_t{node_infos=NodeInfos}, Verbose, Options) ->\n    LeaseLists = [Node#node_info_t.lease_list || Node <- gb_trees:values(NodeInfos)],\n    ActiveLeases  = [lease_list:get_active_lease(LL)  || LL  <- LeaseLists],\n    PassiveLeases = lists:flatmap(fun lease_list:get_passive_leases/1, LeaseLists),\n    ActiveIntervals = [l_on_cseq:get_range(Lease) || Lease <- ActiveLeases, Lease =/= empty],\n    NormalizedActiveIntervals = intervals:union(ActiveIntervals),\n    %io:format(\"Lease-Checker: ~w ~w ~w\", [ActiveLeases, ActiveIntervals, PassiveLeases]),\n    %ct:pal(\"ActiveIntervals: ~p\", [ActiveIntervals]),\n    %ct:pal(\"PassiveLeases: ~p\", [PassiveLeases]),\n    IsAll = intervals:is_all(NormalizedActiveIntervals),\n    IsDisjoint = is_disjoint(ActiveIntervals),\n    HaveAllActiveLeases = checkActiveLeases(length(ActiveLeases), Options),\n    %% HaveAllActiveLeases = length(ActiveLeases) == TargetSize,\n    HaveNoPassiveLeases = length(PassiveLeases) == 0,\n    HaveAllAuxEmpty = lists:all(fun(L) ->\n                                        L =/= empty andalso l_on_cseq:get_aux(L) =:= empty\n                                end, ActiveLeases),\n    % ct:pal(\"lease checker: ~w ~w ~w ~w~n~w~n~w~n\", [IsAll, IsDisjoint, HaveAllActiveLeases, HaveNoPassiveLeases,PassiveLeases, NormalizedActiveIntervals]),\n    case IsAll of\n        false ->\n            %print_all_active_leases(),\n            ok;\n        true ->\n            ok\n    end,\n    case IsAll andalso HaveAllAuxEmpty andalso IsDisjoint andalso HaveAllActiveLeases andalso HaveNoPassiveLeases of\n        true -> ok;\n        false ->\n            case Verbose of\n                true ->\n                    io:format(\"complete ring covered by leases: ~w~n\", [IsAll]),\n                    io:format(\"all aux-fields are empty       : ~w~n\", [HaveAllAuxEmpty]),\n                    io:format(\"no leases overlap              : ~w~n\", [IsDisjoint]),\n                    io:format(\"each node has one active lease : ~w~n\", [HaveAllActiveLeases]),\n                    io:format(\"no passive leases              : ~w~n\", [HaveNoPassiveLeases]),\n                    case HaveAllAuxEmpty of\n                        false ->\n                            io:format(\"aux fields: ~w~n\", [[ l_on_cseq:get_aux(L) || L <- ActiveLeases,\n                                                                                     L =/= empty ]]);\n                        true ->\n                            ok\n                    end;\n                false -> ok\n            end\n    end,\n    IsAll andalso\n        HaveAllAuxEmpty andalso\n        IsDisjoint andalso\n        HaveAllActiveLeases andalso % @todo enable after garbage collection is implemented\n        HaveNoPassiveLeases.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% utility functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec is_disjoint([intervals:interval()]) -> boolean().\nis_disjoint([]) ->\n    true;\nis_disjoint([H | T]) ->\n    is_disjoint(H, T) andalso\n        is_disjoint(T).\n\nis_disjoint(_I, []) ->\n    true;\nis_disjoint(I, [H|T]) ->\n    intervals:is_empty(intervals:intersection(I,H))\n        andalso is_disjoint(I, T).\n\n-spec get_relative_range(intervals:interval()) -> float().\nget_relative_range(ActiveInterval) ->\n    case intervals:empty() of\n        ActiveInterval ->\n            0.0 / ?RT:n();\n        _ ->\n            {_, Begin, End, _} = intervals:get_bounds(ActiveInterval),\n            ?RT:get_range(Begin, End) / ?RT:n()\n    end.\n\n-spec get_dht_node_state(comm:mypid(), atom() | list(atom())) -> term() | list(term()).\nget_dht_node_state(Pid, What) ->\n    case proto_sched:infected() of\n        true ->\n            Cookie = {os:timestamp(), randoms:getRandomInt()},\n            This = comm:reply_as(comm:this(), 2, {get_dht_node_state_response, '_', Cookie}),\n            comm:send(Pid, {get_state, This, What}),\n            trace_mpath:thread_yield(),\n            receive\n                ?SCALARIS_RECV({get_dht_node_state_response,\n                                {get_state_response, Data}, Cookie},% ->\n                               {true, Data})\n                end;\n        false ->\n            false = trace_mpath:infected(),\n            Cookie = {os:timestamp(), randoms:getRandomInt()},\n            This = comm:reply_as(comm:this(), 2, {get_dht_node_state_response, '_', Cookie}),\n            comm:send(Pid, {get_state, This, What}),\n            Result =\n                receive\n                    ?SCALARIS_RECV({get_dht_node_state_response, {get_state_response, Data}, Cookie},% ->\n                                   {true, Data})\n                after 50 ->\n                        false\n                end,\n            %% drain message queue\n            drain_message_queue(),\n            Result\n    end.\n\ndrain_message_queue() ->\n    false = trace_mpath:infected(),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({get_dht_state_response, _Data, _Cookie},% ->\n                       ok)\n    after 0 ->\n            ok\n    end.\n\n-spec all_dht_nodes() -> list(comm:mypid()).\nall_dht_nodes() ->\n    mgmt_server:node_list(),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({get_list_response, Nodes},% ->\n            Nodes)\n    end.\n\n%% @doc keep alle elements of Tree for which F(K,V) is true\n-spec gb_trees_filter(F::fun((K, V) -> boolean()), Tree::gb_trees:tree(K,V)) -> gb_trees:tree(K,V).\ngb_trees_filter(F, Tree) ->\n    gb_trees_filter(F, gb_trees:empty(), gb_trees:iterator(Tree)).\n\n-spec gb_trees_filter(F::fun((K, V) -> boolean()), Acc::gb_trees:tree(K, V),\n                      Iter::gb_trees:iter(K,V)) -> gb_trees:tree(K, V).\ngb_trees_filter(F, Acc, Iter) ->\n    case gb_trees:next(Iter) of\n        none -> Acc;\n        {Key, Value, Iter2} ->\n            case F(Key,Value) of\n                true ->\n                    gb_trees_filter(F, gb_trees:enter(Key, Value, Acc), Iter2);\n                false ->\n                    gb_trees_filter(F, Acc, Iter2)\n            end\n    end.\n\n-spec renderTargetSize(Options::options()) -> term().\nrenderTargetSize(Options) ->\n    case lists:keyfind(ring_size, 1, Options) of\n        false ->\n            case lists:keyfind(ring_size_range, 1, Options) of\n                false ->\n                    unknown;\n                {ring_size_range, From, To} ->\n                    [From, To]\n            end;\n        {ring_size, TargetSize} ->\n            TargetSize\n    end.\n\n-spec checkActiveLeases(NumberOfActiveLeases::pos_integer(), Options::options()) -> boolean().\ncheckActiveLeases(NumberOfActiveLeases, Options) ->\n    case lists:keyfind(ring_size, 1, Options) of\n        false ->\n            case lists:keyfind(ring_size_range, 1, Options) of\n                false ->\n                    true; %% unknown\n                {ring_size_range, From, To} ->\n                    From =< NumberOfActiveLeases andalso  NumberOfActiveLeases =< To\n            end;\n        {ring_size, TargetSize} ->\n            NumberOfActiveLeases =:= TargetSize\n    end.\n"
  },
  {
    "path": "src/cp/lease_list.erl",
    "content": "% @copyright 2012-2014, 2016 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc lease list in the dht_node_state.\n%% @end\n%% @version $Id$\n-module(lease_list).\n-author('schintke@zib.de').\n-author('schuett@zib.de').\n-vsn('$Id:$ ').\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-type active_lease_t() :: l_on_cseq:lease_t() | empty.\n\n-type next_round_map() :: [{l_on_cseq:lease_id(), pr:pr()}].\n\n-record(lease_list_t, {\n          active         = ?required(lease_list_t, active )        :: active_lease_t(),\n          passive        = ?required(lease_list_t, passive)        :: [l_on_cseq:lease_t()],\n          next_round_map = ?required(lease_list_t, next_round_map) :: next_round_map()\n         }).\n\n-type lease_list() :: #lease_list_t{}.\n\n-export_type([lease_list/0]).\n-export_type([active_lease_t/0]).\n\n-export([empty/0]).\n-export([update_lease_in_dht_node_state/4]).\n-export([remove_lease_from_dht_node_state/4]).\n-export([make_lease_list/3]).\n-export([get_active_lease/1]).\n-export([get_passive_leases/1]).\n-export([get_active_range/1]).\n-export([get_next_round/2]).\n-export([contains_lease/3]).\n-export([update_next_round/3]).\n\n-spec empty() -> lease_list().\nempty() ->\n    make_lease_list(empty, [], []).\n\n-spec make_lease_list(active_lease_t(), [l_on_cseq:lease_t()], next_round_map()) ->\n                             lease_list().\nmake_lease_list(Active, Passive, NextRounds) ->\n    #lease_list_t{\n       active = Active,\n       passive   = Passive,\n       next_round_map = NextRounds\n      }.\n\n-spec get_active_lease(lease_list()) -> active_lease_t().\nget_active_lease(#lease_list_t{active=Active}) ->\n    Active.\n\n-spec get_active_range(lease_list()) -> intervals:interval().\nget_active_range(#lease_list_t{active=Active}) ->\n    case Active of\n        empty ->\n            intervals:empty();\n        _ ->\n            l_on_cseq:get_range(Active)\n    end.\n\n-spec get_passive_leases(lease_list()) -> list(l_on_cseq:lease_t()).\nget_passive_leases(#lease_list_t{passive=Passive}) ->\n    Passive.\n\n-spec get_next_round(l_on_cseq:lease_id(), dht_node_state:state()) ->\n                            pr:pr() | failed.\nget_next_round(Id, State) ->\n    #lease_list_t{next_round_map=NextRounds} = dht_node_state:get(State, lease_list),\n    case lists:keyfind(Id, 1, NextRounds) of\n        {Id, NextRound} ->\n             NextRound;\n        false ->\n            failed\n    end.\n\n-spec update_next_round(l_on_cseq:lease_id(), pr:pr(), dht_node_state:state()) ->\n                            dht_node_state:state().\nupdate_next_round(Id, NextRound, State) ->\n    LeaseList = dht_node_state:get(State, lease_list),\n    NextRounds = LeaseList#lease_list_t.next_round_map,\n    NewList = case lists:keyfind(Id, 1, NextRounds) of\n                  {Id, _NextRound} ->\n                      NewNextRounds = lists:keyreplace(Id, 1, NextRounds,\n                                                       {Id, NextRound}),\n                      LeaseList#lease_list_t{next_round_map=NewNextRounds};\n                  false ->\n                      LeaseList#lease_list_t{next_round_map=[{Id, NextRound}|NextRounds]}\n              end,\n    dht_node_state:set_lease_list(State, NewList).\n\n-spec remove_next_round(l_on_cseq:lease_id(), dht_node_state:state()) ->\n                               dht_node_state:state().\nremove_next_round(Id, State) ->\n    LeaseList = dht_node_state:get(State, lease_list),\n    NextRounds = LeaseList#lease_list_t.next_round_map,\n    NewNextRounds = lists:keydelete(Id, 1, NextRounds),\n    dht_node_state:set_lease_list(State,\n                                  LeaseList#lease_list_t{next_round_map=NewNextRounds}).\n\n-spec update_lease_in_dht_node_state(l_on_cseq:lease_t(), dht_node_state:state(),\n                                     active | passive,\n                                     renew | received_lease | unittest | handover |\n                                     takeover |\n                                     recover |\n                                     merge_reply_step1 | merge_reply_step2 |\n                                     merge_reply_step3 |\n                                     split_reply_step1 | split_reply_step2 |\n                                     split_reply_step3 | split_reply_step4) ->\n    dht_node_state:state().\nupdate_lease_in_dht_node_state(Lease, State, Mode, Reason) ->\n    LeaseList = dht_node_state:get(State, lease_list),\n    case {Mode, Reason} of\n        {_, received_lease} ->\n            MyActiveLease = lease_list:get_active_lease(LeaseList),\n            %log:log(\"received lease ~w~n~w\", [Lease, MyActiveLease]),\n            if\n                MyActiveLease =:= empty ->\n                    update_lease_in_dht_node_state(Lease, State, Mode);\n                true ->\n                    NewRange = l_on_cseq:get_range(Lease),\n                    ActiveRange = l_on_cseq:get_range(MyActiveLease),\n                    if\n                        NewRange =:= ActiveRange ->\n                            update_lease_in_dht_node_state(Lease, State, Mode);\n                        true ->\n                            IsAdjacent = intervals:is_adjacent(NewRange, ActiveRange),\n                            IsLeftOf = intervals:is_left_of(NewRange, ActiveRange),\n                            log:log(\"ranges ~w ~w\", [NewRange, ActiveRange]),\n                            log:log(\"received lease ~w ~w ~w ~w ~w\", [intervals:is_adjacent(NewRange, ActiveRange),\n                                                                      intervals:is_left_of(NewRange, ActiveRange),\n                                                                      intervals:is_right_of(NewRange, ActiveRange),\n                                                                      Lease, MyActiveLease]),\n                            if\n                                IsAdjacent andalso IsLeftOf ->\n                                    DHTNode = pid_groups:get_my(rm_leases),\n                                    Pid = comm:reply_as(DHTNode, 4,\n                                                        {merge_after_leave,\n                                                         Lease, MyActiveLease, '_'}),\n                                    l_on_cseq:lease_merge(Lease, MyActiveLease, Pid),\n                                    update_lease_in_dht_node_state(Lease, State, passive);\n                                true ->\n                                    log:log(\"shall merge non-adjacent leases\"),\n                                    ?DBG_ASSERT(false),\n                                    State\n                            end\n                    end\n            end;\n        {_, _} ->\n            update_lease_in_dht_node_state(Lease, State, Mode)\n    end.\n\n-spec contains_lease(l_on_cseq:lease_t(), dht_node_state:state(),\n                     active | passive) -> boolean().\ncontains_lease(Lease, State, Mode) ->\n    LeaseList = dht_node_state:get(State, lease_list),\n    case Mode of\n        active ->\n            case get_active_lease(LeaseList) of\n                empty ->\n                    false;\n                L ->\n                    l_on_cseq:get_id(L) =:= l_on_cseq:get_id(Lease)\n            end;\n        passive ->\n            PassiveLeases = get_passive_leases(LeaseList),\n            lists:keyfind(l_on_cseq:get_id(Lease), 2, PassiveLeases) =/= false\n    end.\n\n\n-spec update_lease_in_dht_node_state(l_on_cseq:lease_t(), dht_node_state:state(),\n                                     active | passive) ->\n    dht_node_state:state().\nupdate_lease_in_dht_node_state(Lease, State, Mode) ->\n    LeaseList = dht_node_state:get(State, lease_list),\n    case Mode of\n        active ->\n            dht_node_state:set_lease_list(State, update_active_lease(Lease, LeaseList));\n        passive ->\n            dht_node_state:set_lease_list(State, update_passive_lease(Lease, LeaseList))\n    end.\n\n\n-spec remove_active_lease_from_dht_node_state(l_on_cseq:lease_t(), l_on_cseq:lease_id(),\n                                              dht_node_state:state()) ->\n                                                     dht_node_state:state().\nremove_active_lease_from_dht_node_state(Lease, Id, State) ->\n    log:log(\"you are trying to remove an active lease via any?!? ~w\", [Lease]),\n    LeaseList = dht_node_state:get(State, lease_list),\n    Active = LeaseList#lease_list_t.active,\n    case l_on_cseq:get_id(Active) of\n        Id ->\n            lease_recover:restart_node(),\n            dht_node_state:set_lease_list(remove_next_round(Id, State),\n                                          LeaseList#lease_list_t{active=empty});\n        _ ->\n            State\n    end.\n\n-spec remove_passive_lease_from_dht_node_state(l_on_cseq:lease_t(), l_on_cseq:lease_id(),\n                                              dht_node_state:state()) ->\n                                                     dht_node_state:state().\nremove_passive_lease_from_dht_node_state(_Lease, Id, State) ->\n    LeaseList = dht_node_state:get(State, lease_list),\n    Passive = LeaseList#lease_list_t.passive,\n    NewPassive = lists:keydelete(Id, 2, Passive),\n    dht_node_state:set_lease_list(remove_next_round(Id, State),\n                                  LeaseList#lease_list_t{passive=NewPassive}).\n\n-spec remove_lease_from_dht_node_state(l_on_cseq:lease_t(), l_on_cseq:lease_id(),\n                                       dht_node_state:state(),\n                                       active | passive | any) ->\n                                              dht_node_state:state().\nremove_lease_from_dht_node_state(Lease, Id, State, Mode) ->\n    case Mode of\n        passive ->\n            remove_passive_lease_from_dht_node_state(Lease, Id, State);\n        active ->\n            log:log(\"you are trying to remove an active lease\"),\n            lease_recover:restart_node(),\n            remove_active_lease_from_dht_node_state(Lease, Id, State);\n        any ->\n            remove_passive_lease_from_dht_node_state(Lease, Id,\n              remove_active_lease_from_dht_node_state(Lease, Id, State))\n    end.\n\n-spec update_passive_lease(Lease::l_on_cseq:lease_t(), LeaseList::lease_list()) -> lease_list().\nupdate_passive_lease(Lease, LeaseList = #lease_list_t{passive=Passive}) ->\n    Id = l_on_cseq:get_id(Lease),\n    NewPassive = case lists:keyfind(Id, 2, Passive) of\n                     false ->\n                         [Lease|Passive];\n                     OldLease ->\n                         case is_newer(Lease, OldLease) of\n                             true ->\n                                 lists:keyreplace(Id, 2, Passive, Lease);\n                             false ->\n                                 Passive\n                         end\n                 end,\n    LeaseList#lease_list_t{passive=NewPassive}.\n\n-spec update_active_lease(Lease::l_on_cseq:lease_t(), LeaseList::lease_list()) -> lease_list().\nupdate_active_lease(Lease, LeaseList = #lease_list_t{active=Active}) ->\n    case Lease of\n        empty ->\n            log:log(\"you are trying to remove an active lease in update\"),\n            lease_recover:restart_node(),\n            ok;\n        _ ->\n            ok\n    end,\n    case Active of\n        empty ->\n            LeaseList#lease_list_t{active=Lease};\n        _ ->\n            case l_on_cseq:get_id(Lease) =/= l_on_cseq:get_id(Active) of\n                true ->\n                    log:log(\"bad update active lease (~w)~n~w~n~w~n\", [self(), Lease, Active]),\n                    log:log(\"~w ~w\", [l_on_cseq:get_id(Lease), l_on_cseq:get_id(Active)]);\n                false ->\n                    ok\n            end,\n            ?DBG_ASSERT(l_on_cseq:get_id(Lease) =:= l_on_cseq:get_id(Active)),\n            % @todo new lease should be newer than the old lease !!!\n            case is_newer(Lease, Active) of\n                true ->\n                    LeaseList#lease_list_t{active=Lease};\n                false ->\n                    LeaseList\n            end\n    end.\n\n-spec is_newer(New::l_on_cseq:lease_t(), Old::l_on_cseq:lease_t()) -> boolean().\nis_newer(New, Old) ->\n    NewEpoch = l_on_cseq:get_epoch(New),\n    NewVersion = l_on_cseq:get_version(New),\n    OldEpoch = l_on_cseq:get_epoch(Old),\n    OldVersion = l_on_cseq:get_version(Old),\n    ((NewEpoch =:= OldEpoch) andalso (NewVersion > OldVersion))\n        orelse\n          (NewEpoch > OldEpoch).\n"
  },
  {
    "path": "src/cp/lease_recover.erl",
    "content": "% @copyright 2007-2017 Zuse Institute Berlin\n\n%  Licensed under the Apache License, Version 2.0 (the \"License\");\n%  you may not use this file except in compliance with the License.\n%  You may obtain a copy of the License at\n%\n%      http://www.apache.org/licenses/LICENSE-2.0\n%\n%  Unless required by applicable law or agreed to in writing, software\n%  distributed under the License is distributed on an \"AS IS\" BASIS,\n%  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%  See the License for the specific language governing permissions and\n%  limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Recover leases.\n%% @end\n%% @version $$\n-module(lease_recover).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-export([recover/1]).\n-export([restart_node/0]).\n\n-spec recover(list(prbr:state())) -> lease_list:lease_list().\nrecover(LeaseDBs) ->\n    AllLeases = lists:flatmap(fun prbr:tab2list/1, LeaseDBs),\n    LocalLeases = [L || {Id, L} <- AllLeases,\n                        L =/= prbr_bottom, %% ??\n                        Id =:= l_on_cseq:get_id(L) %% is this the first replica?,\n                  ],\n    log:log(\"leases:~n~p~n\", [LocalLeases]),\n    case length(LocalLeases) of\n        0 ->\n            log:log(\"recovered with zero leases (~p)~n\", [comm:this()]),\n            restart_node(),\n            lease_list:empty();\n        1 ->\n            wait_for_leases_to_timeout(LocalLeases),\n            Lease = hd(LocalLeases),\n            %% one potentially active lease: set active lease\n            lease_list:make_lease_list(Lease, [], []);\n        _ ->\n            %% could be an ongoing split: finish operation\n            wait_for_leases_to_timeout(LocalLeases),\n            msg_delay:send_trigger(10, {l_on_cseq, wait_for_recover}),\n            get_lease_list(LocalLeases)\n    end.\n\n-spec wait_for_leases_to_timeout([l_on_cseq:lease_t()]) -> ok.\nwait_for_leases_to_timeout(LocalLeases) ->\n    MaxTimeout = lists:max([l_on_cseq:get_timeout(L) || L <- LocalLeases]),\n    WaitTime = timer:now_diff(MaxTimeout, os:timestamp()) * 1000,\n    if\n        WaitTime >= 0 ->\n            timer:sleep(WaitTime);\n        true ->\n            ok\n    end,\n    ?DBG_ASSERT(lists:all(fun l_on_cseq:has_timed_out/1, LocalLeases)),\n    ok.\n\n-spec get_lease_list([l_on_cseq:lease_t()]) ->\n                            lease_list:lease_list().\nget_lease_list(LocalLeases) ->\n    Ranges = [l_on_cseq:get_range(L) || L <- LocalLeases],\n    case intervals:is_all(intervals:union(Ranges)) of\n        true ->\n            %% any lease is ok\n            lease_list:make_lease_list(hd(LocalLeases), tl(LocalLeases), []);\n        false ->\n            %% With high probability, the leases are adjacent, because they were\n            %% created by splits. If the leases are not adjacent, the result is garbage\n            %% @todo: handle non-adjacent case\n            %% bring adjacent leases in order by sorting\n            SortedLeases = lists:reverse(\n                             lists:sort(fun (First, Second) ->\n                                                intervals:is_left_of(l_on_cseq:get_range(First),\n                                                                     l_on_cseq:get_range(Second))\n                                        end, LocalLeases)),\n            lease_list:make_lease_list(hd(SortedLeases), tl(SortedLeases), [])\n    end.\n\n-spec restart_node() -> no_return().\nrestart_node() ->\n    log:log(\"we are stopping ~p~n\", [comm:this()]),\n    Pids = admin:add_nodes(1),\n    log:log(\"we started ~p~n\", [Pids]),\n    %% async. call!\n    service_per_vm:kill_nodes_by_name([pid_groups:my_groupname()]),\n    util:sleep_for_ever().\n"
  },
  {
    "path": "src/cp/leases.erl",
    "content": "% @copyright 2012-2013 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc leases.\n%% @end\n%% @version $Id$\n-module(leases).\n-author('schintke@zib.de').\n-author('schuett@zib.de').\n-vsn('$Id:$ ').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-export([is_responsible/2]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% Public API\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec is_responsible(dht_node_state:state(), ?RT:key()) -> boolean() | maybe.\nis_responsible(State, Key) ->\n    LeaseList = dht_node_state:get(State, lease_list),\n    is_responsible_(LeaseList, Key).\n\n-spec is_responsible_(lease_list:lease_list(), ?RT:key()) -> boolean() | maybe.\nis_responsible_(LeaseList, Key) ->\n    ActiveLease = lease_list:get_active_lease(LeaseList),\n    %PassiveLease = lease_list:get_passive_leases(LeaseList),\n    case ActiveLease of\n        empty ->\n            PassiveLeases = lease_list:get_passive_leases(LeaseList),\n            is_responsible_passive_leases(PassiveLeases, Key);\n        _ ->\n            RangeMatches = intervals:in(Key, l_on_cseq:get_range(ActiveLease)),\n            IsAlive = l_on_cseq:is_valid(ActiveLease),\n            if\n                IsAlive andalso RangeMatches ->\n                    true;\n                RangeMatches ->\n                    %log:log(\"returned maybe\"),\n                    maybe;\n                true ->\n                    %log:log(\"returned false\"),\n                    PassiveLeases = lease_list:get_passive_leases(LeaseList),\n                    is_responsible_passive_leases(PassiveLeases, Key)\n            end\n    end.\n\n-spec is_responsible_passive_leases(list(l_on_cseq:lease_t()), ?RT:key()) -> boolean() | maybe.\nis_responsible_passive_leases([], _Key) ->\n    false;\nis_responsible_passive_leases([Lease|Rest], Key) ->\n    case intervals:in(Key, l_on_cseq:get_range(Lease)) of\n        true ->\n            maybe;\n        false ->\n            is_responsible_passive_leases(Rest, Key)\n    end.\n"
  },
  {
    "path": "src/cp/rm_leases.erl",
    "content": "% @copyright 2007-2018 Zuse Institute Berlin\n\n%  Licensed under the Apache License, Version 2.0 (the \"License\");\n%  you may not use this file except in compliance with the License.\n%  You may obtain a copy of the License at\n%\n%      http://www.apache.org/licenses/LICENSE-2.0\n%\n%  Unless required by applicable law or agreed to in writing, software\n%  distributed under the License is distributed on an \"AS IS\" BASIS,\n%  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%  See the License for the specific language governing permissions and\n%  limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Ring maintenance with leases.\n%% @end\n%% @version $$\n-module(rm_leases).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n%% gen_component callbacks\n-export([start_link/1, init/1, on/2]).\n\n%% rm subscriptions\n-export([rm_filter/3, rm_exec/5]).\n\n% for unit tests\n-export([get_takeovers/1]).\n\n-include(\"gen_component.hrl\").\n\n-record(state, {\n          takeovers     = ?required(state, takeovers) :: gb_trees:tree(?RT:key(),\n                                                                       l_on_cseq:lease_t())\n         }).\n\n-type state() :: #state{}.\n\n%% gen_component functions\n%% @doc Starts the ring maintenence process\n-spec start_link(pid_groups:groupname()) -> {ok, pid()} | ignore.\nstart_link(DHTNodeGroup) ->\n    case config:read(leases) of\n        true ->\n            gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                                     [{wait_for_init}, {erlang_register, ?MODULE},\n                                      {pid_groups_join_as, DHTNodeGroup, ?MODULE}]);\n        _ ->\n            ignore\n    end.\n\n%% @doc Initialises the module with an empty state.\n-spec init([]) -> state().\ninit([]) ->\n    rm_loop:subscribe(self(), ?MODULE,\n                      fun ?MODULE:rm_filter/3,\n                      fun ?MODULE:rm_exec/5, inf),\n    #state{\n       takeovers=gb_trees:empty()\n      }.\n\n-spec rm_filter(OldNeighbors::nodelist:neighborhood(),\n                NewNeighbors::nodelist:neighborhood(),\n                IsSlide::rm_loop:reason()) -> boolean().\nrm_filter(Old, New, Reason) ->\n    OldRange = nodelist:node_range(Old),\n    NewRange = nodelist:node_range(New),\n    case Reason of\n        {slide_finished, _} ->\n            false;\n        {add_subscriber} ->\n            false;\n        {node_crashed, _} ->\n            OldRange =/= NewRange;\n        {node_discovery} ->\n            OldRange =/= NewRange;\n        {graceful_leave, _Node} -> % @todo ?\n            false;\n        {update_id_failed} -> % @todo ?\n            false;\n        {unknown} -> % @todo ?\n            false\n    end.\n\n-spec rm_exec(pid(), Tag::?MODULE,\n              OldNeighbors::nodelist:neighborhood(),\n              NewNeighbors::nodelist:neighborhood(),\n              Reason::rm_loop:reason()) -> ok.\nrm_exec(Pid, _Tag, Old, New, _Reason) ->\n  comm:send_local(Pid, {rm_change, nodelist:node_range(Old),\n                        nodelist:node_range(New)}).\n\n%% @private\n-spec on(comm:message(), state()) -> state().\non({rm_change, OldRange, NewRange}, State) ->\n    ?TRACE(\"the range has changed: ~w -> ~w\", [OldRange, NewRange]),\n    ?TRACE(\"state: ~w\", [State]),\n    This = comm:reply_as(comm:this(), 4, {compare_and_fix, OldRange, NewRange, '_'}),\n    comm:send_local(pid_groups:get_my(dht_node), {get_state, This, [lease_list, my_range]}),\n    State;\n\non({compare_and_fix, OldRange, NewRange,\n    {get_state_response, [{lease_list, L}, {my_range, Range}]}}, State) ->\n    compare_and_fix_rm_with_leases(State, OldRange, NewRange, L, Range);\n\non({read_after_rm_change, _MissingRange, Result}, State) ->\n    ?TRACE(\"read_after_rm_change ~w\", [Result]),\n    case Result of\n        {qread_done, _ReqId, _Round, _OldWriteRound, prbr_bottom} ->\n            log:log(\"not so well-formed qread-response: ~p\", [Result]),\n            State;\n        {qread_done, _ReqId, _Round, _OldWriteRound, Lease} ->\n            LeaseId = l_on_cseq:get_id(Lease),\n            Pid = comm:reply_as(self(), 4, {takeover_after_rm_change, LeaseId, Lease, '_'}),\n            %% log:log(\"rm_leases(~p): going to take over ~p~n\", [self(), Lease]),\n            l_on_cseq:lease_takeover(Lease, Pid),\n            add_takeover(State, Lease);\n        _ ->\n            log:log(\"not so well-formed qread-response: ~p\", [Result]),\n            State\n    end;\n\non({takeover_after_rm_change, LeaseId, _Lease, Result}, State) ->\n    ?TRACE(\"takeover_after_rm_change ~w\", [Result]),\n    case Result of\n        {takeover, failed, L, Error} ->\n            case Error of\n                {content_check_failed,lease_is_still_valid} ->\n                    case is_current_takeover(State, L) of\n                        {value, L2} ->\n                            case l_on_cseq:get_timeout(L) =:= l_on_cseq:get_timeout(L2) of\n                                true ->\n                                    ?TRACE(\"retry ~s\", [l_on_cseq:get_pretty_timeout(L)]),\n                                    LeaseTimeout = l_on_cseq:get_timeout(L),\n                                    Pid = comm:reply_as(self(), 4, {takeover_after_rm_change, LeaseId, L, '_'}),\n                                    WaitTime = timer:now_diff(LeaseTimeout, os:timestamp()),\n                                    ?TRACE(\"retry ~s ~w\", [l_on_cseq:get_pretty_timeout(L), WaitTime]),\n                                    case WaitTime < 500*1000 of\n                                        true ->\n                                            l_on_cseq:lease_takeover(L, Pid);\n                                        false ->\n                                            PostponeBy = trunc(0.5 + WaitTime / (1000*1000)),\n                                            ?TRACE(\"delaying takeover by ~ws\", [PostponeBy]),\n                                            l_on_cseq:lease_takeover_after(PostponeBy, L, Pid)\n                                    end,\n                                    State;\n                                false ->\n                                    propose_new_neighbors(l_on_cseq:get_owner(L)),\n                                    remove_takeover(State, LeaseId, L)\n                            end;\n                        none ->\n                            propose_new_neighbors(l_on_cseq:get_owner(L)),\n                            remove_takeover(State, LeaseId, L)\n                    end;\n                _ ->\n                    case L of\n                        prbr_bottom ->\n                            ok;\n                        _ ->\n                            propose_new_neighbors(l_on_cseq:get_owner(L))\n                    end,\n                    %%log:log(\"unknown error in takeover_after_rm_change ~w\", [Error]),\n                    remove_takeover(State, LeaseId, L)\n                end;\n        {takeover, success, L2} ->\n            ?TRACE(\"takeover_after_rm_change success\", []),\n            This = comm:reply_as(comm:this(), 5, {takeover_success, get_state, L2, LeaseId, '_'}),\n            comm:send_local(pid_groups:get_my(dht_node), {get_state, This, lease_list}),\n            State\n    end;\n\non({takeover_success, get_state, L2, LeaseId, {get_state_response, LeaseList}}, State) ->\n    ActiveLease = lease_list:get_active_lease(LeaseList),\n    Pid = comm:reply_as(self(), 4, {merge_after_rm_change, L2, ActiveLease, '_'}),\n    %% log:log(\"rm_leases: merging ~p and ~p~n\", [L2, ActiveLease]),\n    l_on_cseq:lease_merge(L2, ActiveLease, Pid),\n    remove_takeover(State, LeaseId, L2);\n\non({merge_after_rm_change, _L2, _ActiveLease, Result}, State) ->\n    ?TRACE(\"merge after rm_change: ~w\", [Result]),\n    case Result of\n        {merge, success, __L2, _L1} ->\n            State;\n        {merge, fail, _Reason, _Step, __L1, __L2, _Current, _Next} ->\n            %% @todo\n            State\n    end;\n\non({merge_after_leave, _NewLease, _OldLease, _Result}, State) ->\n    ?TRACE(\"merge after finish done: ~w\", [_Result]),\n    State;\n\non({get_node_for_new_neighbor, {get_state_response, Node}}, State) ->\n    rm_loop:propose_new_neighbors([Node]),\n    State;\n\non({get_takeovers, Pid}, #state{takeovers=Takeovers} = State) ->\n    comm:send(Pid, {get_takeovers_response, Takeovers}),\n    State.\n\n-spec compare_and_fix_rm_with_leases(state(), intervals:interval(), intervals:interval(),\n                                     lease_list:lease_list(), intervals:interval())\n                                     -> state().\ncompare_and_fix_rm_with_leases(State, OldRange, NewRange, LeaseList, MyRange) ->\n    ?TRACE(\"lease list ~w\", [LeaseList]),\n    ActiveRange = lease_list:get_active_range(LeaseList),\n    case intervals:is_empty(ActiveRange) of\n        true ->\n            % we lost our active lease; we do not participate in the ring anymore\n            State;\n        false ->\n            case intervals:is_subset(OldRange, NewRange) of\n                true ->\n                    % NewRange > OldRange\n                    % my range became larger -> do takeover\n                    prepare_takeover(State, MyRange, ActiveRange);\n                false ->\n                    case intervals:is_subset(NewRange, OldRange) of\n                        true ->\n                            % OldRange > NewRange\n                            % my range became smaller -> ignore\n                            State;\n                        false ->\n                            State\n                    end\n            end\n    end.\n\n-spec prepare_takeover(state(), intervals:interval(), intervals:interval()) \n                                     -> state().\nprepare_takeover(State, MyRange, ActiveRange) ->\n    MissingRange = intervals:minus(MyRange, ActiveRange),\n    case intervals:is_non_empty(MissingRange) \n        andalso\n        intervals:is_left_of(MissingRange, ActiveRange) of\n        false ->\n            State;\n        true ->\n            %% log:log(\"missing range:~n missing=~p~n active=~p~n my_range=~p\", \n            %%         [MissingRange, ActiveRange, MyRange]),\n            %% log:log(\"missing range: ~w\", [MissingRange]),\n            LeaseId = l_on_cseq:id(MissingRange),\n            Pid = comm:reply_as(self(), 3, {read_after_rm_change, MissingRange, '_'}),\n            l_on_cseq:read(LeaseId, Pid),\n            State\n    end.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% public helper functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec get_takeovers(comm:erl_local_pid()) -> \n                           gb_trees:tree(?RT:key(), l_on_cseq:lease_t()).\nget_takeovers(RMLeasesPid) ->\n    comm:send_local(RMLeasesPid, {get_takeovers, comm:this()}),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({get_takeovers_response, TakeOvers},% ->\n            TakeOvers)\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% state management\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec add_takeover(state(), l_on_cseq:lease_t()) -> state().\nadd_takeover(#state{takeovers=Takeovers} = State, Lease) ->\n    Id = l_on_cseq:get_id(Lease),\n    case gb_trees:lookup(Id, Takeovers) of\n        {value, _Val} ->\n            % @todo ?!?\n            State;\n        none ->\n            NewTakeovers = gb_trees:insert(Id, Lease, Takeovers),\n            State#state{takeovers=NewTakeovers}\n    end.\n\n-spec remove_takeover(state(), l_on_cseq:lease_id(), l_on_cseq:lease_t()) -> state().\nremove_takeover(#state{takeovers=Takeovers} = State, Id, _Lease) ->\n    NewTakeovers = gb_trees:delete_any(Id, Takeovers),\n    State#state{takeovers=NewTakeovers}.\n\n% @doc the given lease is the one we recorded earlier\n-spec is_current_takeover(state(), l_on_cseq:lease_t()) -> {value, l_on_cseq:lease_t()} | none.\nis_current_takeover(#state{takeovers=Takeovers}, L) ->\n    Id = l_on_cseq:get_id(L),\n    gb_trees:lookup(Id, Takeovers).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% utilities\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec propose_new_neighbors(comm:mypid() | nil) -> ok.\npropose_new_neighbors(PidOrNil) ->\n    ?TRACE(\"somebody else updated this lease\", []),\n    case PidOrNil of\n        nil ->\n            ok;\n        Pid ->\n            ReplyPid = comm:reply_as(comm:this(), 2, {get_node_for_new_neighbor, '_'}),\n            comm:send(Pid, {get_state, ReplyPid, node}),\n            ok\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% todo\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n"
  },
  {
    "path": "src/crdt/crdt_acceptor.erl",
    "content": "% @copyright 2012-2018 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak\n%% @doc    Paxos register for CRDT's. Implements the role of acceptor\n%% @end\n-module(crdt_acceptor).\n-author('skrzypczak@zib.de').\n\n%-define(TRACE(X,Y), ct:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-include(\"scalaris.hrl\").\n\n-define(PDB, db_prbr).\n%%% functions for module where embedded into\n-export([on/2, init/1, close/1, close_and_delete/1]).\n-export([check_config/0]).\n-export([new/2]).\n-export([set_entry/2]).\n-export([get_entry/2]).\n-export([entry_key/1]).\n-export([entry_val/2]).\n-export([entry_set_val/2]).\n\n%% let fetch the number of DB entries\n-export([get_load/1]).\n\n%% only for unittests\n-export([tab2list_raw_unittest/1]).\n\n%% only during recover\n-export([tab2list/1]).\n\n-export_type([state/0]).\n-export_type([entry/0]).\n\n-type state() :: {?PDB:db(), [comm:mypid_plain()]}.\n\n%% so there are no longer any read denies, all reads succeed.\n-type entry() :: { any(), %% key\n                   pr:pr(), %% r_read\n                   pr:pr(), %% r_write  %%TODO: write roudns are not needed???\n                   crdt:crdt() | crdt_bottom %% value\n                 }.\n\n%% Messages to expect from this module\n-spec msg_update_reply(comm:mypid(), any(), crdt:crdt()) -> ok.\nmsg_update_reply(Client, ReqId, CVal) ->\n    comm:send(Client, {update_reply, ReqId, CVal}).\n\n-spec msg_merge_reply(comm:mypid(), any()) -> ok.\nmsg_merge_reply(Client, ReqId) ->\n    comm:send(Client, {merge_reply, ReqId, done}).\n\n-spec msg_prepare_reply(comm:mypid(), any(), pr:pr(), pr:pr(), crdt:crdt()) -> ok.\nmsg_prepare_reply(Client, ReqId, ReadRound, WriteRound, CVal) ->\n    comm:send(Client, {prepare_reply, ReqId, ReadRound, WriteRound, CVal}).\n\n-spec msg_prepare_deny(comm:mypid(), any(), pr:pr(), pr:pr()) -> ok.\nmsg_prepare_deny(Client, ReqId, TriedReadRound, RequiredReadRound) ->\n    comm:send(Client, {read_deny, ReqId, inc, TriedReadRound, RequiredReadRound}).\n\n-spec msg_vote_reply(comm:mypid(), any()) -> ok.\nmsg_vote_reply(Client, ReqId) ->\n    comm:send(Client, {vote_reply, ReqId, done}).\n\n-spec msg_vote_deny(comm:mypid(), any(), pr:pr(), pr:pr()) -> ok.\nmsg_vote_deny(Client, ReqId, TriedWriteRound, RequiredReadRound) ->\n    comm:send(Client, {read_deny, ReqId, inc, TriedWriteRound, RequiredReadRound}).\n\n-spec msg_registered_proposers(comm:mypid_plain(), [comm:mypid_plain()]) -> ok.\nmsg_registered_proposers(Proposer, ListOfProposers) ->\n  comm:send(Proposer, {registered_proposers, ListOfProposers}).\n\n%% initialize: return initial state.\n-spec init(atom() | tuple()) -> state().\ninit(DBName) -> {?PDB:new(DBName), []}.\n\n\n%% @doc Closes the given DB (it may be recoverable using open/1 depending on\n%%      the DB back-end).\n-spec close(state()) -> true.\nclose(State) -> ?PDB:close(State).\n\n%% @doc Closes the given DB and deletes all contents (this DB can thus not be\n%%      re-opened using open/1).\n-spec close_and_delete(state()) -> true.\nclose_and_delete(State) -> ?PDB:close_and_delete(State).\n\n-spec on(tuple(), state()) -> state().\non({crdt_acceptor, update, _Cons, Proposer, ReqId, Key, DataType, UpdateFuns}, State) ->\n    ?TRACE(\"crdt_acceptor:update: ~p ~p \", [Key, Proposer]),\n    TableName = tablename(State),\n    Entry = get_entry(Key, TableName),\n\n    Keys = lists:sort(replication:get_keys(Key)),\n    Tmp = lists:dropwhile(fun(E) -> E =/= Key end, Keys),\n    ThisReplicaId = length(Keys) - length(Tmp) + 1,\n\n    CVal = entry_val(Entry, DataType),\n    NewCVal =\n      lists:foldl(\n        fun(UpdateFun, TmpVal) ->\n          DataType:apply_update(UpdateFun, ThisReplicaId, TmpVal)\n        end, CVal, lists:flatten(UpdateFuns)),\n    ?ASSERT(DataType:lteq(CVal, NewCVal)),\n\n    NewEntry = entry_set_val(Entry, NewCVal),\n    _ = set_entry(NewEntry, TableName),\n    msg_update_reply(Proposer, ReqId, NewCVal),\n\n    trace_mpath:log_info(self(), {acceptor_update,\n                                  key, Key,\n                                  old_value, CVal,\n                                  new_value, NewCVal}),\n    State;\n\non({crdt_acceptor, merge, _Cons, Proposer, ReqId, Key, DataType, CValToMerge}, State) ->\n    ?TRACE(\"crdt_acceptor:merge: ~p ~p\", [Key, Proposer]),\n    TableName = tablename(State),\n    Entry = get_entry(Key, TableName),\n\n    CVal = entry_val(Entry, DataType),\n    NewCVal = DataType:merge(CVal, CValToMerge),\n    ?ASSERT(DataType:lteq(CVal, NewCVal)),\n\n    NewEntry = entry_set_val(Entry, NewCVal),\n    _ = set_entry(NewEntry, TableName),\n\n    msg_merge_reply(Proposer, ReqId),\n\n    trace_mpath:log_info(self(), {acceptor_merge,\n                                  key, Key,\n                                  old_value, CVal,\n                                  new_value, NewCVal}),\n    State;\n\n%% SC-read phase 1\non({crdt_acceptor, prepare, _Cons, Proposer, ReqId, Key, DataType, Round, CValToMerge}, State) ->\n    ?TRACE(\"crdt:prepare: ~p in round ~p~n\", [Key, ProposalRound]),\n    TableName = tablename(State),\n    Entry = get_entry(Key, TableName),\n    CVal = entry_val(Entry, DataType),\n    NewCVal = DataType:merge(CValToMerge, CVal),\n    NewEntry = entry_set_val(Entry, NewCVal),\n\n    ProposalRound =\n        case Round of\n            {inc, Id} ->\n                OldRound =  entry_r_read(NewEntry),\n                pr:new(pr:get_r(OldRound) + 1, Id);\n            _ ->\n                Round\n        end,\n\n    _ = case pr:get_r(ProposalRound) > pr:get_r(entry_r_read(NewEntry)) of\n         true ->\n            CurrentWriteRound = entry_r_write(NewEntry),\n            NewEntry2 = entry_set_r_read(Entry, ProposalRound),\n            _ = set_entry(NewEntry2, TableName),\n            msg_prepare_reply(Proposer, ReqId, ProposalRound, CurrentWriteRound,\n                              NewCVal);\n         _ ->\n            _ = set_entry(NewEntry, TableName),\n            msg_prepare_deny(Proposer, ReqId, ProposalRound, entry_r_read(NewEntry))\n    end,\n\n    trace_mpath:log_info(self(), {acceptor_read_p1,\n                                  key, Key,\n                                  round, Round,\n                                  merge_value, CValToMerge}),\n    State;\n\n%% SC-read phase 2\non({crdt_acceptor, vote, _Cons, Proposer, ReqId, Key, DataType, ProposalRound, CValToMerge}, State) ->\n    ?TRACE(\"prbr:vote for key: ~p in round ~p~n\", [Key, ProposalRound]),\n    TableName = tablename(State),\n    Entry = get_entry(Key, TableName),\n    CurrentReadRound = entry_r_read(Entry),\n    CVal = entry_val(Entry, DataType),\n    NewCVal = DataType:merge(CVal, CValToMerge),\n    ?ASSERT(DataType:lteq(CVal, NewCVal)),\n    NewEntry = entry_set_val(Entry, NewCVal),\n\n    _ = case ProposalRound =:= CurrentReadRound orelse\n             (pr:get_r(ProposalRound) > pr:get_r(CurrentReadRound)\n             andalso DataType:lteq(CVal, CValToMerge)) of\n            true ->\n                NewEntry2 = entry_set_r_read(NewEntry, ProposalRound),\n                NewEntry3 = entry_set_r_write(NewEntry2, ProposalRound),\n                _ = set_entry(NewEntry3, TableName),\n                msg_vote_reply(Proposer, ReqId);\n            false ->\n                _ = set_entry(NewEntry, TableName),\n                msg_vote_deny(Proposer, ReqId, ProposalRound, CurrentReadRound)\n        end,\n\n    trace_mpath:log_info(self(), {acceptor_read_p2,\n                                  key, Key,\n                                  current_round, CurrentReadRound,\n                                  tried_round, ProposalRound,\n                                  merge_value, CValToMerge,\n                                  full_value, NewCVal}),\n    State;\n\n%% discovery of all proposers. Needed as write-free wrapper must communicate\n%% between proposer processes.\non({crdt_acceptor, register_proposer, _Cons, Proposer, _ReqId, _Key}, State) ->\n  ProposerList = proposerlist(State),\n  NewProposerList = lists:usort([Proposer | ProposerList]),\n  NewState = set_proposerlist(State, NewProposerList),\n  [msg_registered_proposers(P, NewProposerList) || P <- NewProposerList],\n  NewState;\n\non({crdt_acceptor, get_entry, _DB, Client, Key}, State) ->\n    comm:send_local(Client, {entry, get_entry(Key, tablename(State))}),\n    State;\n\non({crdt_acceptor, tab2list_raw, DB, Client}, State) ->\n    comm:send_local(Client, {DB, tab2list_raw(tablename(State))}),\n    State.\n\n-spec tablename(state()) -> ?PDB:db().\ntablename(State) -> element(1, State).\n\n-spec proposerlist(state()) -> [comm:mypid_plain()].\nproposerlist(State) -> element(2, State).\n\n-spec set_proposerlist(state(), [comm:mypid_plain()]) -> state().\nset_proposerlist(State, PropList) -> setelement(2, State, PropList).\n\n\n-spec get_entry(any(), state()) -> entry().\nget_entry(Id, TableName) ->\n    case ?PDB:get(TableName, Id) of\n        {}    -> new(Id);\n        Entry -> Entry\n    end.\n\n-spec set_entry(entry(), state()) -> state().\nset_entry(NewEntry, TableName) ->\n    _ = ?PDB:set(TableName, NewEntry),\n    TableName.\n\n\n-spec get_load(state()) -> non_neg_integer().\nget_load(State) -> ?PDB:get_load(State).\n\n-spec tab2list(state()) -> [{any(),any()}].\ntab2list(State) ->\n    %% without prbr own data\n    Entries = tab2list_raw(State),\n    [ {entry_key(X), entry_val(X)} || X <- Entries].\n\n-spec tab2list_raw_unittest(state()) -> [entry()].\ntab2list_raw_unittest(State) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    tab2list_raw(State).\n\n-spec tab2list_raw(state()) -> [entry()].\ntab2list_raw(State) ->\n    %% with prbr own data\n    ?PDB:tab2list(State).\n\n%% operations for abstract data type entry()\n\n-spec new(any()) -> entry().\nnew(Key) ->\n    new(Key, crdt_bottom).\n\n-spec new(any(), crdt:crdt() | crdt_bottom) -> entry().\nnew(Key, Val) ->\n    {Key,\n     %% Note: atoms < pids, so this is a good default.\n     _R_Read = pr:new(0, '_'),\n     %% Note: atoms < pids, so this is a good default.\n     _R_Write = pr:new(0, '_'),\n     _Value = Val\n     }.\n\n-spec entry_key(entry()) -> any().\nentry_key(Entry) -> element(1, Entry).\n-spec entry_r_read(entry()) -> pr:pr().\nentry_r_read(Entry) -> element(2, Entry).\n-spec entry_set_r_read(entry(), pr:pr()) -> entry().\nentry_set_r_read(Entry, Round) -> setelement(2, Entry, Round).\n-spec entry_r_write(entry()) -> pr:pr().\nentry_r_write(Entry) -> element(3, Entry).\n-spec entry_set_r_write(entry(), pr:pr()) -> entry().\nentry_set_r_write(Entry, Round) -> setelement(3, Entry, Round).\n-spec entry_val(entry()) -> crdt:crdt() | crdt_bottom.\nentry_val(Entry) -> element(4, Entry).\n-spec entry_val(entry(), crdt:crdt_module()) -> crdt:crdt().\nentry_val(Entry, DataType) ->\n    case entry_val(Entry) of\n        crdt_bottom ->\n            DataType:new();\n        Any -> Any\n    end.\n-spec entry_set_val(entry(), crdt:crdt()) -> entry().\nentry_set_val(Entry, Value) -> setelement(4, Entry, Value).\n\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() -> true.\n\n"
  },
  {
    "path": "src/crdt/crdt_on_cseq.erl",
    "content": "% @copyright 2014-2018 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    API to CRDT-based state machine replication\n%% @end\n-module(crdt_on_cseq).\n-author('skrzypczak@zib.de').\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% API with linearizable semantics (strong consistency)\n-export([read/3]).\n-export([write/3]).\n\n-spec read(client_key(), crdt:crdt_module(), crdt:query_fun() | [crdt:query_fun()])\n    -> {ok, client_value() | [client_value()]}.\nread(Key, DataType, QueryFun) ->\n    Mod = crdt:proposer_module(),\n    FunName = read,\n    read_helper(Key, Mod, FunName, DataType, QueryFun).\n\n-spec read_helper(client_key(), module(), atom(), crdt:crdt_module(), crdt:query_fun()\n    | [crdt:query_fun()]) -> {ok, client_value() | [client_value()]}.\nread_helper(Key, ApiMod, ApiFun, DataType, QueryFun) ->\n    ApiMod:ApiFun(crdt_db, self(), ?RT:hash_key(Key), DataType, QueryFun),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({read_done, ReturnValues}, {ok,ReturnValues})\n    after 1000 ->\n        log:log(\"read hangs ~p~n\", [erlang:process_info(self(), messages)]),\n        receive\n            ?SCALARIS_RECV({read_done, ReturnValues},\n                            begin\n                                log:log(\"~p read was only slow at key ~p~n\",\n                                        [self(), Key]),\n                                {ok, ReturnValues}\n                            end)\n        end\n    end.\n\n\n-spec write(client_key(), crdt:crdt_module(), crdt:update_fun() | [crdt:update_fun()]) -> ok.\nwrite(Key, DataType, UpdateFun) ->\n    Mod = crdt:proposer_module(),\n    FunName = write,\n    write_helper(Key, Mod, FunName, DataType, UpdateFun).\n\n-spec write_helper(client_key(), module(), atom(), crdt:crdt_module(),\n    crdt:update_fun() | [crdt:update_fun()]) -> ok.\nwrite_helper(Key, ApiMod, ApiFun, DataType, UpdateFun) ->\n    ApiMod:ApiFun(crdt_db, self(), ?RT:hash_key(Key), DataType, UpdateFun),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({write_done}, ok)\n    after 1000 ->\n        log:log(\"~p write hangs at key ~p, ~p~n\",\n                [self(), Key, erlang:process_info(self(), messages)]),\n        receive\n            ?SCALARIS_RECV({write_done},\n                            begin\n                                log:log(\"~p write was only slow at key ~p~n\",\n                                        [self(), Key]),\n                                ok\n                            end)\n        end\n    end.\n\n"
  },
  {
    "path": "src/crdt/crdt_proposer.erl",
    "content": "% @copyright 2012-2018 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    Paxos register for CRDT's. Implements the role of proposer\n%% @end\n-module(crdt_proposer).\n-author('skrzypczak.de').\n\n-define(PDB, pdb).\n%-define(TRACE(X,Y), ct:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n\n-define(CACHED_ROUTING, (config:read(cache_dht_nodes))).\n\n-include(\"scalaris.hrl\").\n\n-behaviour(gen_component).\n\n-export_type([state/0]).\n\n-export([write/5]).\n-export([read/5]).\n\n-export([send_to_all_replicas/2]).\n\n-export([check_config/0]).\n-export([start_link/3]).\n-export([init/1, on/2]).\n\n-type state() :: { ?PDB:tableid(),\n                   dht_node_state:db_selector(),\n                   non_neg_integer(), %% period this process is in\n                   boolean(),\n                   boolean()\n                 }.\n\n-record(r_replies,  {\n                        reply_count = 0 :: non_neg_integer(),\n                        highest_replies = 0 :: non_neg_integer(),\n                        highest_seen_round :: pr:pr(),\n                        cons_value = true :: boolean(),\n                        value :: crdt:crdt()\n                    }).\n\n-record(w_replies, {reply_count = 0 :: non_neg_integer()}).\n\n-type entry() :: {any(),        %% request id\n                  comm:erl_local_pid(), %% client\n                  ?RT:key(),    %% key\n                  crdt:crdt_module(),     %% data type\n                  [crdt_function()], %% fun used to read/write crdt state\n                  replies(),     %% aggregates replies\n                  non_neg_integer() %% round trips executed\n                 }.\n\n-type replies() :: #w_replies{} | #r_replies{}.\n-type crdt_function() :: crdt:query_fun() | crdt:update_fun().\n\n-include(\"gen_component.hrl\").\n\n\n-spec start_link(pid_groups:groupname(), pid_groups:pidname(), dht_node_state:db_selector())\n                -> {ok, pid()}.\nstart_link(DHTNodeGroup, Name, DBSelector) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, DBSelector,\n                             [{pid_groups_join_as, DHTNodeGroup, Name}]).\n\n-spec init(dht_node_state:db_selector()) -> state().\ninit(DBSelector) ->\n    {?PDB:new(?MODULE, [set]), DBSelector, 0, false, false}.\n\n\n%%%% API\n\n-spec read(pid_groups:pidname(), comm:erl_local_pid(), ?RT:key(), crdt:crdt_module(),\n    [crdt:query_fun()] | crdt:query_fun()) -> ok.\nread(CSeqPidName, Client, Key, DataType, QueryFuns) ->\n    start_request(CSeqPidName, {req_start, {read, Client, Key, DataType, QueryFuns, none, 0}}).\n\n-spec write(pid_groups:pidname(), comm:erl_local_pid(), ?RT:key(), crdt:crdt_module(),\n    [crdt:update_fun()] | crdt:update_fun()) -> ok.\nwrite(CSeqPidName, Client, Key, DataType, UpdateFun) when is_function(UpdateFun) ->\n    write(CSeqPidName, Client, Key, DataType, [UpdateFun]);\n\nwrite(CSeqPidName, Client, Key, DataType, UpdateFuns) when is_list(UpdateFuns) ->\n    start_request(CSeqPidName, {req_start, {write, Client, Key, DataType, UpdateFuns}}).\n\n-spec start_request(pid_groups:pidname(), comm:message()) -> ok.\nstart_request(CSeqPidName, Msg) ->\n    Pid = pid_groups:find_a(CSeqPidName),\n    trace_mpath:log_info(self(), {start_request, request, Msg}),\n    comm:send_local(Pid, Msg).\n\n\n-spec on(comm:message(), state()) -> state().\n\n%%%%%%%% Query protocol (as query is a keyword, use read/write terminology)\non({req_start, {read, Client, Key, DataType, QueryFuns, PreviousRound, PreviousRoundTrips}}, State) ->\n    ReqId = uid:get_pids_uid(),\n    Entry = entry_new_read(ReqId, Client, Key, DataType, QueryFuns, DataType:new()),\n\n    Round = case PreviousRound of\n                none ->\n                    {inc, ReqId};\n                _ ->\n                    round_inc(PreviousRound, ReqId)\n            end,\n\n    This = comm:reply_as(comm:this(), 2, {read, '_'}),\n    Msg = {crdt_acceptor, prepare, '_', This, ReqId, key, DataType, Round, DataType:new()},\n\n    NewEntry = entry_set_round_trips(Entry, PreviousRoundTrips + 1),\n    save_entry(NewEntry, tablename(State)),\n    send_to_all_replicas(Key, Msg),\n\n    State;\n\non({read, {prepare_reply, ReqId, UsedReadRound, WriteRound, CVal}}, State) ->\n    _ = case get_entry(ReqId, tablename(State)) of\n            undefined ->\n                %% ignore replies for unknown requests (i.e. because we already\n                %% have processed them)\n                State;\n            Entry ->\n                Replies = entry_replies(Entry),\n                {Done, NewReplies} = add_read_reply(Replies, UsedReadRound, WriteRound,\n                                                    CVal, entry_datatype(Entry)),\n                NewEntry = entry_set_replies(Entry, NewReplies),\n                case Done of\n                    false ->\n                        save_entry(NewEntry, tablename(State)),\n                        State;\n                    cons_read ->\n                        %% value is already established in a quorum, skip 2. phase\n                        QueryFuns = entry_funs(NewEntry),\n                        Type = entry_datatype(NewEntry),\n                        ReturnVals = apply_queries(Type, QueryFuns, NewReplies#r_replies.value),\n\n                        trace_mpath:log_info(self(), {read_done,\n                                                      crdt_value, NewReplies#r_replies.value,\n                                                      return_value, ReturnVals}),\n                        inform_client(read_done, Entry, ReturnVals),\n                        delete_entry(Entry, tablename(State)),\n                        State;\n                    retry_read ->\n                        %% we received inconsistent read rounds... thus we have\n                        %% concurrency and must retry to get consistent rounds accepts\n                        %% entry will be deleted in called onhandler\n                        on({read, {read_deny, ReqId, fixed, UsedReadRound,\n                                round_inc(NewReplies#r_replies.highest_seen_round)}}, State);\n                    true ->\n                        delete_entry(NewEntry, tablename(State)),\n\n                        %% new entry for next step of protocol\n                        NewReqId = uid:get_pids_uid(),\n                        Type = entry_datatype(NewEntry),\n                        TEntry = entry_new_read(NewReqId,\n                                                entry_client(NewEntry),\n                                                entry_key(NewEntry),\n                                                Type,\n                                                entry_funs(NewEntry),\n                                                Type:new()),\n                        T2Entry = entry_set_round_trips(TEntry, entry_round_trips(NewEntry) + 1),\n                        %% keep the merged value we have collected in this phase, which will\n                        %% be the value returned if phase 2 succeeds\n                        NextStepEntry = entry_set_replies(T2Entry, NewReplies#r_replies{reply_count=0}),\n                        save_entry(NextStepEntry, tablename(State)),\n\n                        trace_mpath:log_info(self(), {read_phase2_start,\n                                                      round, UsedReadRound,\n                                                      value, NewReplies#r_replies.value}),\n                        This = comm:reply_as(comm:this(), 2, {read, '_'}),\n                        Msg = {crdt_acceptor, vote, '_', This, NewReqId, key,\n                               entry_datatype(NewEntry), UsedReadRound, NewReplies#r_replies.value},\n                        send_to_all_replicas(entry_key(NewEntry), Msg),\n                        State\n                end\n        end;\n\non({read, {vote_reply, ReqId, done}}, State) ->\n    _ = case get_entry(ReqId, tablename(State)) of\n            undefined ->\n                ok;\n            Entry ->\n                Replies = entry_replies(Entry),\n                {Done, NewReplies} = add_vote_reply(Replies),\n                NewEntry = entry_set_replies(Entry, NewReplies),\n\n                case Done of\n                    false -> save_entry(NewEntry, tablename(State));\n                    true ->\n                        QueryFuns = entry_funs(NewEntry),\n                        DataType = entry_datatype(NewEntry),\n                        ReturnVals = apply_queries(DataType, QueryFuns, NewReplies#r_replies.value),\n\n                        trace_mpath:log_info(self(), {read_done,\n                                                      crdt_value, NewReplies#r_replies.value,\n                                                      return_value, ReturnVals}),\n                        inform_client(read_done, Entry, ReturnVals),\n                        delete_entry(Entry, tablename(State))\n                end\n        end,\n    State;\n\non({read, {read_deny, ReqId, RetryMode, TriedRound, RequiredRound}}, State) ->\n    _ = case get_entry(ReqId, tablename(State)) of\n            undefined ->\n                %% ignore replies for unknown requests\n                ok;\n            Entry ->\n                %set_last_used_round(NextRound, tablename(State)),\n                delete_entry(Entry, tablename(State)),\n\n                NextRound = case RetryMode of\n                                inc -> none;\n                                fixed -> round_inc(RequiredRound)\n                            end,\n\n                trace_mpath:log_info(self(), {read_deny,\n                                              retry_mode, RetryMode,\n                                              round_tried, TriedRound,\n                                              round_requed, RequiredRound\n                                             }),\n                %% retry the read in a higher round...\n                %% TODO more intelligent retry mechanism?\n                RoundTrips = entry_round_trips(Entry),\n                Delay = randoms:rand_uniform(0, 10),\n                comm:send_local_after(Delay, self(),\n                                        {req_start, {read,\n                                        entry_client(Entry),\n                                        entry_key(Entry),\n                                        entry_datatype(Entry),\n                                        entry_funs(Entry),\n                                        NextRound,\n                                        RoundTrips}})\n        end,\n    State;\n\n\n%%%%% UPDATE protocol (as query is a keyword, use read/write terminology)\n\non({req_start, {write, Client, Key, DataType, UpdateFuns}}, State) ->\n    ReqId = uid:get_pids_uid(),\n    Entry = entry_new_write(ReqId, Client, Key, DataType, UpdateFuns),\n    save_entry(Entry, tablename(State)),\n\n    This = comm:reply_as(comm:this(), 2, {write, '_'}),\n    Msg = {crdt_acceptor, update, '_', This, ReqId, key, DataType, UpdateFuns},\n    send_to_local_replica(Key, Msg),\n\n    State;\n\non({write, {update_reply, ReqId, CVal}}, State) ->\n    This = comm:reply_as(comm:this(), 2, {write, '_'}),\n\n    _ = case get_entry(ReqId, tablename(State)) of\n            undefined ->\n                %% ignore replies for unknown requests (i.e. because we already\n                %% have processed them)\n                ok;\n            Entry ->\n                Msg = {crdt_acceptor, merge, '_', This, ReqId, key,\n                       entry_datatype(Entry), CVal},\n                trace_mpath:log_info(self(), {write_start,\n                                             value, CVal}),\n                NewEntry = entry_inc_round_trips(Entry),\n                save_entry(NewEntry, tablename(State)),\n                send_to_all_replicas(entry_key(NewEntry), Msg)\n        end,\n    State;\n\non({write, {merge_reply, ReqId, done}}, State) ->\n    _ = case get_entry(ReqId, tablename(State)) of\n            undefined ->\n                %% ignore replies for unknown requests\n                ok;\n            Entry ->\n                Replies = entry_replies(Entry),\n                {Done, NewReplies} = add_write_reply(Replies),\n                NewEntry = entry_set_replies(Entry, NewReplies),\n                case Done of\n                    false -> save_entry(NewEntry, tablename(State));\n                    true ->\n                        trace_mpath:log_info(self(), {write_done}),\n                        inform_client(write_done, Entry),\n                        delete_entry(Entry, tablename(State))\n                end\n        end,\n    State;\n\non({local_range_req, Key, Message, {get_state_response, LocalRange}}, State) ->\n    Keys = replication:get_keys(Key),\n\n    LocalKeys = lists:filter(fun(K) -> intervals:in(K, LocalRange) end, Keys),\n\n    K = case LocalKeys of\n        [] ->\n            ?TRACE(\"cannot send locally ~p ~p \", [Keys, LocalRange]),\n            %% the local dht node is not responsible for any replca... route to\n            %% random replica\n            Idx = randoms:rand_uniform(1, length(Keys)+1),\n            lists:nth(Idx, Keys);\n        [H|_] ->\n            %% use replica managed by local dht node\n            H\n        end,\n\n    Dest = pid_groups:find_a(routing_table),\n    LookupEnvelope = dht_node_lookup:envelope(3, setelement(6, Message, K)),\n    comm:send_local(Dest, {?lookup_aux, K, 0, LookupEnvelope}),\n    State.\n\n%%%%%% internal helper\n%% Apply all queries. Can handle nested lists of queries.\n-spec apply_queries(crdt:crdt_module(), [crdt:query_fun()] | crdt:query_fun(), crdt:crdt()) -> any() | [any()].\napply_queries(DataType, QueryFun, Crdt) when is_function(QueryFun) ->\n    DataType:apply_query(QueryFun, Crdt);\napply_queries(DataType, QueryFuns, Crdt) ->\n   [apply_queries(DataType, Fun, Crdt) || Fun <- QueryFuns].\n\n-spec send_to_local_replica(?RT:key(), tuple()) -> ok.\nsend_to_local_replica(Key, Message) ->\n    %% assert element(3, message) =:= '_'\n    %% assert element(6, message) =:= key\n    send_to_local_replica(Key, Message, ?CACHED_ROUTING).\n\n-spec send_to_local_replica(?RT:key(), tuple(), boolean()) -> ok.\nsend_to_local_replica(Key, Message, _CachedRouting=true) ->\n    dht_node_cache:cached_send_to_local_replica(Key, _KeyPos=6,\n                                                Message, _LookupEnvPos=3);\nsend_to_local_replica(Key, Message, _CachedRouting=false) ->\n    LocalDhtNode = pid_groups:get_my(dht_node),\n    This = comm:reply_as(comm:this(), 4, {local_range_req, Key, Message, '_'}),\n    comm:send_local(LocalDhtNode, {get_state, This, my_range}).\n\n-spec send_to_all_replicas(?RT:key(), tuple()) -> ok.\nsend_to_all_replicas(Key, Message) ->\n    %% assert element(3, message) =:= '_'\n    %% assert element(6, message) =:= key\n    send_to_all_replicas(Key, Message, ?CACHED_ROUTING).\n\n-spec send_to_all_replicas(?RT:key(), tuple(), boolean()) -> ok.\nsend_to_all_replicas(Key, Message, _CachedRouting=true) ->\n    _ = [begin\n            Msg = setelement(6, Message, K),\n            dht_node_cache:cached_send(K, Msg, _LookupEnvPos=3)\n         end\n        || K <- replication:get_keys(Key)],\n    ok;\n\nsend_to_all_replicas(Key, Message, _CachedRouting=false) ->\n    Dest = pid_groups:find_a(routing_table),\n    _ = [begin\n            LookupEnvelope = dht_node_lookup:envelope(3, setelement(6, Message, K)),\n            comm:send_local(Dest, {?lookup_aux, K, 0, LookupEnvelope})\n         end\n        || K <- replication:get_keys(Key)],\n    ok.\n\n-spec inform_client(write_done, entry()) -> ok.\ninform_client(write_done, Entry) ->\n    Client = entry_client(Entry),\n    case is_tuple(Client) of\n        true ->\n            % must unpack envelope\n            comm:send(entry_client(Entry), {write_done});\n        false ->\n            comm:send_local(entry_client(Entry), {write_done})\n    end.\n\n-spec inform_client(read_done, entry(), [any()]) -> ok.\ninform_client(read_done, Entry, QueryResults) ->\n    Client = entry_client(Entry),\n    case is_tuple(Client) of\n        true ->\n            % must unpack envelope\n            comm:send(entry_client(Entry), {read_done, QueryResults});\n        false ->\n            comm:send_local(entry_client(Entry), {read_done, QueryResults})\n    end.\n\n-spec add_write_reply(#w_replies{}) -> {boolean(), #w_replies{}}.\nadd_write_reply(Replies) ->\n    ReplyCount = Replies#w_replies.reply_count + 1,\n    NewReplies = Replies#w_replies{reply_count=ReplyCount},\n    Done = replication:quorum_accepted(ReplyCount),\n    {Done, NewReplies}.\n\n-spec add_read_reply(#r_replies{}, pr:pr(), pr:pr(), crdt:crdt(), crdt:crdt_module()) ->\n    {boolean() | cons_read | retry_read, #r_replies{}}.\nadd_read_reply(Replies, UsedReadRound, _WriteRound, Value, DataType) ->\n    NewReplyCount = Replies#r_replies.reply_count + 1,\n    NewMaxRound = max(Replies#r_replies.highest_seen_round, UsedReadRound),\n    HighestReplies =\n        case NewMaxRound =:= Replies#r_replies.highest_seen_round of\n            true -> Replies#r_replies.highest_replies + 1;\n            false -> 1\n        end,\n    {NewValue, IsValueCons} =\n        case DataType:eq(Value, Replies#r_replies.value) of\n            true ->\n                {Value, true andalso Replies#r_replies.cons_value};\n            false ->\n                {DataType:merge(Replies#r_replies.value, Value),\n                NewReplyCount =:= 1} % not inconsistent if this is our first reply\n        end,\n    NewReplies = Replies#r_replies{\n                   reply_count=NewReplyCount,\n                   highest_seen_round=NewMaxRound,\n                   highest_replies = HighestReplies,\n                   cons_value = IsValueCons,\n                   value=NewValue\n                  },\n\n    %% There are multiple ways to terminate the current protocol step. All\n    %% of them requiring a quorum of replies first:\n    %% (cons_read) We received consistent values in replies\n    %%  ->  This means the current value is already establish and thus no subsequent\n    %%      read request will be able to read a smaller value. Therefore, we can skip\n    %%      the 2. phase (vote) and directly deliver the result to the client\n    %% (true) We received inconsistent values and consistent read rounds:\n    %% ->   Default Paxos-like behaviour. Proceed to 2. phase to establish value\n    %%      in a quorum\n    %% (retry_read) We received inconsistent value and inconsistent reads rounds:\n    %% ->   Might happen for concurrent phase 1 read executions since we are doing\n    %%      an incremental round negotiaton mechanism. We have to retry the first phase\n    %%      with an explicit round, otherwise reads might return conflicting values\n    Done = case replication:quorum_accepted(NewReplyCount) of\n               false -> false;\n               true ->\n                   case {IsValueCons, HighestReplies =:= NewReplyCount} of\n                       {true, _} -> cons_read;\n                       {false, true} -> true;\n                       {false, false} -> retry_read\n                   end\n           end,\n\n    {Done, NewReplies}.\n\n-spec add_vote_reply(#r_replies{}) -> {boolean(), #r_replies{}}.\nadd_vote_reply(Replies) ->\n    ReplyCount = Replies#r_replies.reply_count + 1,\n    NewReplies = Replies#r_replies{reply_count=ReplyCount},\n    Done = replication:quorum_accepted(ReplyCount),\n    {Done, NewReplies}.\n\n-spec entry_new_read(any(), comm:erl_local_pid(), ?RT:key(), crdt:crdt_module(),\n        [crdt:query_fun()], crdt:crdt()) -> entry().\nentry_new_read(ReqId, Client, Key, DataType, QueryFuns, EmptyVal) ->\n    {ReqId, Client, Key, DataType, QueryFuns,\n     #r_replies{reply_count=0, highest_seen_round=pr:new(0,0), highest_replies=0,\n                value=EmptyVal, cons_value=true}, _RoundTrips=0}.\n\n-spec entry_new_write(any(), comm:erl_local_pid(), ?RT:key(), crdt:crdt_module(),\n        [crdt:update_fun()]) -> entry().\nentry_new_write(ReqId, Client, Key, DataType, UpdateFuns) ->\n    {ReqId, Client, Key, DataType, UpdateFuns, #w_replies{reply_count=0}, _RoundTrips=0}.\n\n-spec entry_reqid(entry())        -> any().\nentry_reqid(Entry)                -> element(1, Entry).\n-spec entry_client(entry())       -> comm:erl_local_pid().\nentry_client(Entry)               -> element(2, Entry).\n-spec entry_key(entry())          -> any().\nentry_key(Entry)                  -> element(3, Entry).\n-spec entry_datatype(entry())     -> crdt:crdt_module().\nentry_datatype(Entry)             -> element(4, Entry).\n-spec entry_funs(entry())         -> [crdt_function()].\nentry_funs(Entry)                 -> element(5, Entry).\n-spec entry_replies(entry())      -> replies().\nentry_replies(Entry)              -> element(6, Entry).\n-spec entry_set_replies(entry(), replies()) -> entry().\nentry_set_replies(Entry, Replies) -> setelement(6, Entry, Replies).\n-spec entry_round_trips(entry())  -> non_neg_integer().\nentry_round_trips(Entry)          -> element(7, Entry).\n-spec entry_inc_round_trips(entry()) -> entry().\nentry_inc_round_trips(Entry) -> setelement(7, Entry, element(7, Entry) + 1).\n-spec entry_set_round_trips(entry(), non_neg_integer()) -> entry().\nentry_set_round_trips(Entry, RoundTrips) -> setelement(7, Entry, RoundTrips).\n\n-spec get_entry(any(), ?PDB:tableid()) -> entry() | undefined.\nget_entry(ReqId, TableName) ->\n    ?PDB:get(ReqId, TableName).\n\n-spec save_entry(entry() , ?PDB:tableid()) -> ok.\nsave_entry(NewEntry, TableName) ->\n    ?PDB:set(NewEntry, TableName).\n\n-spec delete_entry(entry() , ?PDB:tableid()) -> ok.\ndelete_entry(Entry, TableName) ->\n    ReqId = entry_reqid(Entry),\n    ?PDB:delete(ReqId, TableName).\n\n-spec tablename(state()) -> ?PDB:tableid().\ntablename(State) -> element(1, State).\n\n-spec round_inc(pr:pr()) -> pr:pr().\nround_inc(Round) ->\n    pr:new(pr:get_r(Round)+1, pr:get_id(Round)).\n\n-spec round_inc(pr:pr(), any()) -> pr:pr().\nround_inc(Round, ID) ->\n    pr:new(pr:get_r(Round)+1, ID).\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_bool(cache_dht_nodes).\n\n"
  },
  {
    "path": "src/crdt/crdt_wait_free_wrapper.erl",
    "content": "% @copyright 2012-2020 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    An wrapper around CRDTPaxos to make it wait-free\n%% @end\n\n-module(crdt_wait_free_wrapper).\n-author('skrzypczak@zib.de').\n\n-define(PDB, pdb).\n-define(TRACE(X), ?TRACE(X, [])).\n-define(TRACE(X,Y),\n\tcase is_list(Y) of\n\t\ttrue ->\n\t\t\tct:pal(X ++ \"~n\",Y);\n\t\tfalse ->\n\t\t\tct:pal(X ++ \"~n\", [Y])\n\tend).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n-behaviour(gen_component).\n\n%% protocol API\n-export([write/5, read/5]).\n\n%% gen_componenet message handling\n-export([start_link/3]).\n-export([init/1, on/2]).\n\n\n%% protocol state\n-define(get(X, Y), get_field(?i(X), Y)).\n-define(set(X, Y, V), set_field(?i(X), Y, V)).\n-define(add(X, Client, DataType, Fun, Y), add_to_buffer(?i(X), Client, DataType, Fun, Y)).\n\n-define(i(X), #pstate.X).\n\n-record(pstate,\n\t{\n\t\t%% state needed to execute updates\n\t\tupdate_in_progress = false :: boolean(),\n\t\tbuffer_id = 0 :: non_neg_integer(),\n\t\tupdate_buffer = new_buffer() :: buffer(),\n\t\tcurr_update_buffer = new_buffer() :: buffer(),\n\n\t\t%% state needed to execute queries\n\t\tquery_id = 0 :: non_neg_integer(), %% needed to prevent sending multiple replies to client\n\t\tquery_in_progress = false :: boolean(),\n\t\tquery_buffer = new_buffer() :: buffer(),\n\t\tcurr_query_buffer = new_buffer() :: buffer(),\n\n\t\tpending_queries = [] :: waitlist(),\n\t\tupdates_waiting = [] :: waitlist()\n\t}).\n\n-type buffer() :: [buffer_element()].\n-type buffer_element() :: {{comm:mypid(), non_neg_integer()} | comm:erl_local_pid(),\n\tcrdt:crdt_module(), crdt:query_fun() | crdt:update_fun()}.\n\n-type waitlist() :: [{comm:mypid_plain(), non_neg_integer()}].\n\n%$ scalaris state\n-type state() ::\n\t{\n\t\t?PDB:tableid(),\n\t\tdht_node_state:db_selector(),\n\t\t[comm:mypid_plain()],\n\t\tcrdt_proposer:state()\t%% it this wrapper is running, crdt_proposer gen_component isn't!\n\t}.\n\n\n-include(\"gen_component.hrl\").\n\n-spec start_link(pid_groups:groupname(), pid_groups:pidname(), dht_node_state:db_selector())\n                -> {ok, pid()}.\nstart_link(DHTNodeGroup, Name, DBSelector) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, DBSelector,\n                             [{pid_groups_join_as, DHTNodeGroup, Name}]).\n\n-spec init(dht_node_state:db_selector()) -> state().\ninit(DBSelector) ->\n\tcrdt_proposer:send_to_all_replicas(?RT:hash_key(\"1\"),\n\t\t{crdt_acceptor, register_proposer, '_', comm:this(), 0, key}),\n    {?PDB:new(?MODULE, [set]), DBSelector, [comm:this()], crdt_proposer:init(DBSelector)}.\n\n\n%%%%%%%%% API\n%% Do not use query/update as query is a reserved keyword...?\n-spec read(pid_groups:pidname(), comm:erl_local_pid(), ?RT:key(),\n\tcrdt:crdt_module(), crdt:query_fun()) -> ok.\nread(CSeqPidName, Client, Key, DataType, QueryFun) ->\n\tPid = pid_groups:find_a(CSeqPidName),\n\tcomm:send_local(Pid, {new_request, {read, Client, Key, DataType, QueryFun}}).\n\n-spec write(pid_groups:pidname(), comm:erl_local_pid(), ?RT:key(),\n\tcrdt:crdt_module(), crdt:update_fun()) -> ok.\nwrite(CSeqPidName, Client, Key, DataType, UpdateFun) ->\n\tPid = pid_groups:find_a(CSeqPidName),\n\tcomm:send_local(Pid, {new_request, {write, Client, Key, DataType, UpdateFun}}).\n\n\n%%%%%%%%%% PROTOCOL IMPLEMENTATION\n-spec on(comm:message(), state()) -> state().\n\non({new_request, {read, Client, Key, DataType, QueryFun}}, State) ->\n\tPState = get_pstate(Key, State),\n\tQueryId = ?get(query_id, PState) + 1,\n\tNP1 = ?set(query_id, PState, QueryId),\n\tNP2 = add_to_pending(Client, QueryId, NP1),\n\tsave_pstate(Key, NP2, State),\n\t%% send to every proposer!\n\t%% This is needed to guarantee wait freedom, as only progress of a\n\t%% single proposer can be ensured.\n\tsend_to_all_proposers({query_internal, {comm:this(), QueryId}, Key, DataType, QueryFun}, State),\n\tState;\n\non({query_internal, FromProposer, Key, DataType, QueryFun}, State) ->\n\tPState = get_pstate(Key, State),\n\tNewPState = ?add(query_buffer, FromProposer, DataType, QueryFun, PState),\n\tsave_pstate(Key, NewPState, State),\n\tgen_component:post_op({process_query, Key}, State);\n\non({process_query, Key}, State) ->\n\tPState = get_pstate(Key, State),\n\tQueryBuf = ?get(query_buffer, PState),\n\tInProgress = ?get(query_in_progress, PState),\n\tcase InProgress orelse QueryBuf == [] of\n\t\ttrue ->\n\t\t\t%% do not process current buffer\n\t\t\tState;\n\t\tfalse ->\n\t\t\tNP1 = ?set(query_in_progress, PState, true),\n\n\t\t\t%% base protocol query execution...\n\t\t\tDataType = element(2, hd(QueryBuf)), %% assumes every op access same datatype!\n\t\t\tQueryFuns = [element(3, E) || E <- QueryBuf],\n\t\t\t\n\t\t\tNP2 = ?set(query_buffer, NP1, new_buffer()),\n\t\t\tNP3 = ?set(curr_query_buffer, NP2, QueryBuf),\n\t\t\tsave_pstate(Key, NP3, State),\n\n\t\t\tThis = comm:reply_as(comm:this(), 3, {query_buffer_done, Key, '_'}),\n\t\t\tcrdt_proposer:read(crdt_db, This, Key, DataType, QueryFuns),\n\t\t\tState\n\tend;\n\non({query_buffer_done, Key, _Result={read_done, Results}}, State) ->\n\tPState = get_pstate(Key, State),\n\tQueryBuf = ?get(curr_query_buffer, PState),\n\tnotify_waiting_progress(Key, PState),\n\tNP1 = clear_waitlist(PState),\n\tNP2 = ?set(query_in_progress, NP1, false),\n\tsave_pstate(Key, NP2, State),\n\n\t[\n\t\tcomm:send(Proposer, {query_done, Key, QueryId, Result}) ||\n\t\t{{{Proposer, QueryId}, _, _}, Result} <- lists:zip(QueryBuf, Results)\n\t],\n\tgen_component:post_op({process_query, Key}, State);\n\non({query_done, Key, QueryId, Result}, State) ->\n\tPState = get_pstate(Key, State),\n\tcase remove_pending(QueryId, PState) of\n\t\t{{Client, QueryId}, NewPState} ->\n\t\t\tinform_client(read_done, Client, Result),\n\t\t\tsave_pstate(Key, NewPState, State);\n\t\t{none, _PState} ->\n\t\t\t%% client was already notified\n\t\t\tok\n\tend,\n\tState;\n\n%%%% update protocol\non({new_request, {write, Client, Key, DataType, UpdateFun}}, State) ->\n\tPState = get_pstate(Key, State),\n\tNewPState = ?add(update_buffer, Client, DataType, UpdateFun, PState),\n\tsave_pstate(Key, NewPState, State),\n\tgen_component:post_op({process_update, Key}, State);\t\n\non({process_update, Key}, State) ->\n\tPState = get_pstate(Key, State),\n\tUpdateBuf = ?get(update_buffer, PState),\n\tInProgress = ?get(update_in_progress, PState),\n\tcase InProgress orelse UpdateBuf == [] of\n\t\ttrue ->\n\t\t\t%% do not process current buffer\n\t\t\tState;\n\t\tfalse -> \n\t\t\tNP1 = ?set(update_in_progress, PState, true),\n\t\t\tBuffId = ?get(buffer_id, NP1),\n\t\t\tsave_pstate(Key, NP1, State),\n\t\t\tcase ?get(query_in_progress, PState) of\n\t\t\t\ttrue ->\n\t\t\t\t\tsend_to_all_proposers({wait_for_progess, comm:this(), Key, BuffId}, State),\n\t\t\t\t\tState;\n\t\t\t\tfalse ->\n\t\t\t\t\tgen_component:post_op({progress_made, Key, BuffId}, State)\n\t\t\tend\n\tend;\n\non({progress_made, Key, ThisBuffId}, State) ->\n\tPState = get_pstate(Key, State),\n\tBuffId = ?get(buffer_id, PState),\n\tcase ?get(buffer_id, PState) =/= ThisBuffId of\n\t\ttrue ->\n\t\t\t%% this is an outdated progress message which we can ignore\n\t\t\tState;\n\t\tfalse ->\n\t\t\tUpdateBuf = ?get(update_buffer, PState),\n\t\t\tNP1 = ?set(curr_update_buffer, PState, UpdateBuf),\n\t\t\tNP2 = ?set(update_buffer, NP1, new_buffer()),\n\t\t\tNP3 = ?set(buffer_id, NP2,BuffId + 1),\n\t\t\t\n\t\t\t%% base protocol query execution...\n\t\t\tDataType = element(2, hd(UpdateBuf)), %% assumes every op access same datatype!\n\t\t\tUpdateFuns = [element(3, E) || E <- UpdateBuf],\n\t\t\tsave_pstate(Key, NP3, State),\n\n\t\t\tThis = comm:reply_as(comm:this(), 3, {update_buffer_done, Key, '_'}),\n\t\t\tcrdt_proposer:write(crdt_db, This, Key, DataType, UpdateFuns),\n\t\t\tState\n\tend;\n\non({update_buffer_done, Key, {write_done}}, State) ->\n\tPState = get_pstate(Key, State),\n\tCurrBuff = ?get(curr_update_buffer, PState),\n\tNP1 = ?set(update_in_progress, PState, false),\n\tinform_all_clients(write_done, CurrBuff),\n\tsave_pstate(Key, NP1, State),\n\tgen_component:post_op({process_update, Key}, State);\n\non({wait_for_progess, Proposer, Key, BuffId}, State) ->\n\tPState = get_pstate(Key, State),\n\tcase ?get(query_in_progress, PState) of\n\t\ttrue ->\n\t\t\tNP1 = add_to_waitlist(Proposer, BuffId, PState),\n\t\t\tsave_pstate(Key, NP1, State);\n\t\tfalse ->\n\t\t\t%% no query in progress -> we can immediately reply\n\t\t\tcomm:send(Proposer, {progress_made, Key, BuffId})\n\tend,\n\tState;\n\non({registered_proposers, ProposerList}, State) ->\n\tOldList = proposerlist(State),\n\tcase length(OldList) > length(ProposerList) of\n\t\ttrue ->\n\t\t\tState;\n\t\tfalse ->\n\t\t\tset_proposerlist(State, ProposerList)\n\tend;\n\n%% forward unknown messages to crdt_proposer!\non(UnknownMessage, State) ->\n\t%?TRACE(\"crdt_proposer msg ~n~p\", UnknownMessage),\n\tProposerState = proposerstate(State),\n\tNewProposerState = crdt_proposer:on(UnknownMessage, ProposerState),\n\tset_proposerstate(State, NewProposerState).\n\n%%%%%%% STATE MANAGEMENT\n-spec get_pstate(client_key(), state()) -> #pstate{}.\nget_pstate(Key, State) ->\n    case ?PDB:get(Key, tablename(State)) of\n    \tundefined ->\n    \t\t#pstate{};\n    \t{Key, PState} ->\n    \t\tPState\n    end.\n\n-spec save_pstate(client_key(), #pstate{}, state()) -> ok.\nsave_pstate(Key, PState, State) ->\n    ?PDB:set({Key, PState}, tablename(State)).\n\n-spec get_field(non_neg_integer(), #pstate{}) -> any().\nget_field(FieldIdx, PState) ->\n\telement(FieldIdx, PState).\n\n-spec set_field(non_neg_integer(), #pstate{}, any()) -> #pstate{}.\nset_field(FieldIdx, PState, NewValue) ->\n\tsetelement(FieldIdx, PState, NewValue).\n\n-spec new_buffer() -> buffer().\nnew_buffer() -> [].\n\n-spec add_to_buffer(non_neg_integer(), {comm:mypid(), non_neg_integer()} | comm:erl_local_pid(),\n\tcrdt:crdt_module(), crdt:query_fun() | crdt:update_fun(), #pstate{}) -> #pstate{}.\nadd_to_buffer(BufIdx, Client, DataType, Fun, PState) ->\n\tBufEle = {Client, DataType, Fun},\n\tNewBuf = [BufEle | element(BufIdx, PState)],\n\tsetelement(BufIdx, PState, NewBuf).\n\n-spec add_to_pending(comm:mypid_plain(), non_neg_integer(), #pstate{}) -> #pstate{}.\nadd_to_pending(Client, Id, PState) ->\n\tPending = ?get(pending_queries, PState),\n\tNewPending = [{Client, Id} | Pending],\n\t?set(pending_queries, PState, NewPending).\n\n-spec remove_pending(non_neg_integer(), #pstate{}) ->\n\t{none | {comm:mypid_plain(), non_neg_integer()}, #pstate{}}.\nremove_pending(Id, PState) -> \n\tPending = ?get(pending_queries, PState),\n\t{Removed, NewPending} = pd_helper(Id, Pending, []),\n\t{Removed, ?set(pending_queries, PState, NewPending)}.\n\n-spec pd_helper(non_neg_integer(), waitlist(), waitlist()) -> \n\t{none | {comm:mypid_plain(), non_neg_integer()}, waitlist()}.\npd_helper(_Id, [], Pending) -> {none, Pending};\npd_helper(Id, [E={_, Id} | T], Pending) -> {E, T ++ Pending};\npd_helper(Id, [H | T], Pending) -> pd_helper(Id, T, [H | Pending]).\n\n-spec add_to_waitlist(comm:mypid_plain(), non_neg_integer(), #pstate{}) -> #pstate{}.\nadd_to_waitlist(Client, Id, PState) ->\n\tWaitlist = ?get(updates_waiting, PState),\n\tNewWaitList = wl_helper(Client, Id, Waitlist),\n\t?set(updates_waiting, PState, NewWaitList).\n\n% only keep the maximal bufferID for each proposer\n-spec wl_helper(comm:mypid_plain(), non_neg_integer(), waitlist()) -> waitlist().\nwl_helper(P, B, [])\t-> [{P, B}];\nwl_helper(P, B, [{P, OB} | T]) -> [{P, max(B, OB)} | T];\nwl_helper(P, B, [H | T]) -> [H | wl_helper(P, B, T)].\n\n-spec clear_waitlist(#pstate{}) -> #pstate{}.\nclear_waitlist(PState) ->\n\t?set(updates_waiting, PState, []).\n\n-spec tablename(state()) -> ?PDB:tableid().\ntablename(State) -> element(1, State).\n\n-spec proposerlist(state()) -> [comm:mypid_plain()].\nproposerlist(State) -> element(3, State).\n-spec set_proposerlist(state(), [comm:mypid_plain()]) -> state().\nset_proposerlist(State, ProposerList) -> setelement(3, State, ProposerList).\n\n-spec proposerstate(state()) -> crdt_proposer:state().\nproposerstate(State) -> element(4, State).\n-spec set_proposerstate(state(), crdt_proposer:state()) -> state().\nset_proposerstate(State, ProposerState) -> setelement(4, State, ProposerState).\n\n%%%%% Communication helper\n-spec send_to_all_proposers(any(), state()) -> ok.\nsend_to_all_proposers(Msg, State) ->\n\tProposerList = proposerlist(State),\n\t[comm:send(Proposer, Msg) || Proposer <- ProposerList],\n\tok.\n\n-spec notify_waiting_progress(client_key(), #pstate{}) -> ok.\nnotify_waiting_progress(Key, PState) ->\n\tWaitList = ?get(updates_waiting, PState),\n\t[comm:send(Proposer, {progress_made, Key, BuffId}) ||\n\t\t{Proposer, BuffId} <- WaitList],\n\tok.\n\n\n-spec inform_all_clients(atom(), buffer()) -> ok.\ninform_all_clients(write_done, Buffer) ->\n\t[inform_client(write_done, E) || E <- Buffer],\n\tok.\n\n-spec inform_client(write_done, buffer_element()) -> ok.\ninform_client(write_done, BufEle) ->\n\tClient = element(1, BufEle),\n    case is_tuple(Client) of\n        true ->\n            % must unpack envelope\n            comm:send(Client, {write_done});\n        false ->\n            comm:send_local(Client, {write_done})\n    end.\n\n-spec inform_client(read_done, comm:erl_local_pid(), any()) -> ok.\ninform_client(read_done, Client, QueryResult) ->\n    case is_tuple(Client) of\n        true ->\n            % must unpack envelope\n            comm:send(Client, {read_done, QueryResult});\n        false ->\n            comm:send_local(Client, {read_done, QueryResult})\n    end.\n"
  },
  {
    "path": "src/crdt/gcounter_on_cseq.erl",
    "content": "% @copyright 2014-2018 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    API to increment a replicated counter using a\n%%         state-based G-Counter CRDT.\n%% @end\n-module(gcounter_on_cseq).\n-author('skrzypczak@zib.de').\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([read/1]).\n-export([read_state/1]).\n\n-export([inc/1]).\n\n%% Reads counter with strong consistency semantics\n-spec read(client_key()) -> {ok, client_value()}.\nread(Key) ->\n    crdt_on_cseq:read(Key, gcounter, fun gcounter:query_counter/1).\n\n%% Reads the full state with strong consistentcy semantics\n-spec read_state(client_key()) -> {ok, client_value()}.\nread_state(Key) ->\n    crdt_on_cseq:read(Key, gcounter, fun crdt:query_noop/1).\n\n%% Increments counter with strong consistency semantics\n-spec inc(client_key()) -> ok.\ninc(Key) ->\n    UpdateFun = fun(ReplicaId, CRDT) -> gcounter:update_add(ReplicaId, 1, CRDT) end,\n    crdt_on_cseq:write(Key, gcounter, UpdateFun).\n\n"
  },
  {
    "path": "src/crdt/gla_acceptor.erl",
    "content": "% @copyright 2012-2018 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak\n%% @doc    Acceptor implementation from \"Generalized Lattice Agreement\"\n%%  TODO:\n%%      - right now proposerlist is lazily extended\n%% @end\n-module(gla_acceptor).\n-author('skrzypczak@zib.de').\n\n%-define(TRACE(X,Y), ct:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-include(\"scalaris.hrl\").\n\n-define(PDB, db_prbr).\n\n%%% functions for module where embedded into\n-export([on/2, init/1, close/1, close_and_delete/1]).\n-export([check_config/0]).\n-export([new/2]).\n-export([set_entry/2]).\n-export([get_entry/2]).\n-export([entry_key/1]).\n-export([entry_val/1]).\n-export([entry_set_val/2]).\n\n-export_type([state/0]).\n-export_type([entry/0]).\n\n-type state() :: {?PDB:db(), [comm:mypid()]}.\n\n-type entry() :: {\n                   any(), %% key\n                   gset:crdt() | gla_bottom, %% value\n                   any()\n                 }.\n\n%% Messages to expect from this module\n-spec msg_ack_reply(comm:mypid(), ?RT:key(), pr:pr()) -> ok.\nmsg_ack_reply(Client, Key, ProposalNumber) ->\n    comm:send(Client, {ack_reply, Key, ProposalNumber}).\n\n-spec msg_learner_ack_reply(comm:mypid(), ?RT:key(), crdt:crdt_module(), pr:pr(), gset:crdt(), any()) -> ok.\nmsg_learner_ack_reply(Proposer, Key, DataType, ProposalNumber, ProposalValue, ProposerId) ->\n    comm:send(Proposer, {learner_ack_reply, Key, DataType, ProposalNumber, ProposalValue, ProposerId}).\n\n-spec msg_nack_reply(comm:mypid(), ?RT:key(), pr:pr(), crdt:crdt()) -> ok.\nmsg_nack_reply(Client, Key, ProposalNumber, ProposalValue) ->\n    comm:send(Client, {nack_reply, Key, ProposalNumber, ProposalValue}).\n\n\n%% initialize: return initial state.\n-spec init(atom() | tuple()) -> state().\ninit(DBName) -> {?PDB:new(DBName), []}.\n\n%% @doc Closes the given DB (it may be recoverable using open/1 depending on\n%%      the DB back-end).\n-spec close(state()) -> true.\nclose(_State={TableName, _}) -> ?PDB:close(TableName).\n\n%% @doc Closes the given DB and deletes all contents (this DB can thus not be\n%%      re-opened using open/1).\n-spec close_and_delete(state()) -> true.\nclose_and_delete(_State={TableName, _}) -> ?PDB:close_and_delete(TableName).\n\n%% implements acceptor action Accept and Reject\n-spec on(tuple(), state()) -> state().\non({gla_acceptor, propose, _Cons, Proposer, Key, DataType, ProposalNumber, PartialProposedValue},\n   _State={TableName, ProposerList}) ->\n    ?TRACE(\"crdt_acceptor:propose: ~p ~p ~n ~p ~p\", [Key, Proposer, ProposalNumber, PartialProposedValue]),\n    % TODO since no initial discovery is implemented, add unknown proposers to our list\n    NewProposerList = case lists:member(Proposer, ProposerList) of\n                          true -> ProposerList;\n                          false -> [Proposer | ProposerList]\n                      end,\n\n    TEntry = get_entry(Key, TableName),\n    Entry = entry_update_proposed(TEntry, Proposer, PartialProposedValue),\n    ProposedValue = entry_proposed(Entry, Proposer),\n\n    AcceptedValue = entry_val(Entry),\n\n    case gset:lteq(AcceptedValue, ProposedValue) of\n        true ->\n            %% Accept action\n            NewEntry = entry_set_val(Entry, ProposedValue),\n            _ = set_entry(NewEntry, TableName),\n\n            _ = [msg_learner_ack_reply(Learner, Key, DataType, ProposalNumber, ProposedValue, Proposer)\n                 || Learner <- NewProposerList], %% each proposer is also a learner\n            msg_ack_reply(Proposer, Key, ProposalNumber);\n        false ->\n            %% Reject action\n            MergedValue = gset:merge(AcceptedValue, ProposedValue),\n            NewEntry = entry_set_val(Entry, MergedValue),\n            _ = set_entry(NewEntry, TableName),\n            MissingValues = gset:subtract(AcceptedValue, ProposedValue),\n            msg_nack_reply(Proposer, Key, ProposalNumber, MissingValues)\n    end,\n\n    {TableName, NewProposerList}.\n\n\n-spec get_entry(any(), ?PDB:db()) -> entry().\nget_entry(Id, TableName) ->\n    case ?PDB:get(TableName, Id) of\n        {}    -> new(Id);\n        Entry -> Entry\n    end.\n\n-spec set_entry(entry(), ?PDB:db()) -> ?PDB:db().\nset_entry(NewEntry, TableName) ->\n    _ = ?PDB:set(TableName, NewEntry),\n    TableName.\n\n\n-spec new(any()) -> entry().\nnew(Key) ->\n    new(Key, gla_bottom).\n\n-spec new(any(), gset:crdt() | gla_bottom) -> entry().\nnew(Key, Val) ->\n    {Key, Val, dict:new()}.\n\n-spec entry_key(entry()) -> any().\nentry_key(Entry) -> element(1, Entry).\n\n-spec entry_val(entry()) -> gset:crdt() | gla_bottom.\nentry_val(Entry) ->\n    case element(2, Entry) of\n        gla_bottom -> gset:new();\n        Any -> Any\n    end.\n-spec entry_set_val(entry(), gset:crdt()) -> entry().\nentry_set_val(Entry, Value) -> setelement(2, Entry, Value).\n\n-spec entry_proposed(entry(), any()) -> gset:crdt().\nentry_proposed(Entry, Proposer) -> dict:fetch(Proposer, element(3, Entry)).\n-spec entry_update_proposed(entry(), any(), gset:crdt()) -> entry().\nentry_update_proposed(Entry, Proposer, PartialVal) ->\n    Dict = element(3, Entry),\n    Dict2 = dict:update(Proposer, fun(E) -> gset:merge(PartialVal, E) end, PartialVal, Dict),\n    setelement(3, Entry, Dict2).\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() -> true.\n\n"
  },
  {
    "path": "src/crdt/gla_proposer.erl",
    "content": "% @copyright 2012-2020 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    Proposer + Learner implementation from \"Generalized Lattice Agreement\"\n%% TODO: instead of sending write done messages, let them poll? only local repla sends msg %% @end\n-module(gla_proposer).\n-author('skrzypczak.de').\n\n-define(PDB, pdb).\n%-define(TRACE(X,Y), ct:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n\n-define(ROUTING_DISABLED, false).\n-define(WRITE_DONE_POLLING_PERIOD, 10). %% time in ms how often to check if writes completed\n\n-include(\"scalaris.hrl\").\n\n-behaviour(gen_component).\n\n-export([write/5]).\n-export([read/5]).\n\n-export([start_link/3]).\n-export([init/1, on/2]).\n\n-type state() :: { ?PDB:tableid(),\n                   dht_node_state:db_selector(),\n                   non_neg_integer(), %% period this process is in\n                   [any()], %% list of known proposers\n                   [{any(), ?RT:key(), any()}] %% list of writer clients waiting for reply\n                 }.\n\n-type entry() :: {?RT:key(), %% key\n                  crdt:crdt_module(), %% data type\n                  [any()], %% clients waiting for reply TODO use this or keep close to paper\n                  active | passive,\n                  non_neg_integer(), %% ack count\n                  non_neg_integer(), %% nack count\n                  non_neg_integer(), %% active proposal number\n                  gset:crdt(), %% proposed value\n                  gset:crdt(), %% buffered value\n                  gset:crdt(), %% output value\n                  gset:crdt() %% refine value\n                 }.\n\n-type learner_entry() :: {{learner, ?RT:key()},\n                           crdt:crdt_module(),\n                           gset:crdt(),\n                           [{any(), non_neg_integer()}]}.\n\n-include(\"gen_component.hrl\").\n\n\n-spec start_link(pid_groups:groupname(), pid_groups:pidname(), dht_node_state:db_selector())\n                -> {ok, pid()}.\nstart_link(DHTNodeGroup, Name, DBSelector) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, DBSelector,\n                             [{pid_groups_join_as, DHTNodeGroup, Name}]).\n\n-spec init(dht_node_state:db_selector()) -> state().\ninit(DBSelector) ->\n    {?PDB:new(?MODULE, [set]), DBSelector, 0, [], []}.\n\n\n%%%% API\n\n-spec write(pid_groups:pidname(), comm:erl_local_pid(), ?RT:key(), crdt:crdt_module(), crdt:update_fun()) -> ok.\nwrite(CSeqPidName, Client, Key, DataType, UpdateFun) ->\n    start_request(CSeqPidName, {req_start, {write, Client, Key, DataType, UpdateFun}}).\n\n-spec read(pid_groups:pidname(), comm:erl_local_pid(), ?RT:key(), crdt:crdt_module(), crdt:query_fun()) -> ok.\nread(CSeqPidName, Client, Key, DataType, QueryFun) ->\n    start_request(CSeqPidName, {req_start, {read, Client, Key, DataType, QueryFun}}).\n\n-spec start_request(pid_groups:pidname(), comm:message()) -> ok.\nstart_request(CSeqPidName, Msg) ->\n    Pid = pid_groups:find_a(CSeqPidName),\n    trace_mpath:log_info(self(), {start_request, request, Msg}),\n    comm:send_local(Pid, Msg).\n\n\n-spec on(comm:message(), state()) -> state().\n\n%%%% read\non({req_start, {read, Client, Key, DataType, QueryFun}}, State) ->\n    This = comm:reply_as(comm:this(), 6, {read_write_done, Client, Key, DataType, QueryFun, '_'}),\n    gen_component:post_op({req_start, {write, This, Key, DataType, fun crdt:update_noop/2}}, State);\n\non({read_write_done, Client, Key, DataType, QueryFun, _Msg}, State) ->\n    This = comm:reply_as(comm:this(), 5, {apply_query, Client, DataType, QueryFun, '_'}),\n    gen_component:post_op({learnt_value, Key, This}, State);\n\non({apply_query, Client, DataType, QueryFun, {LearntValue}}, State) ->\n    CrdtVal = gset:fold(fun({_CmdId, UpdateCmd}, Acc) ->\n                                %% replica ID does not matter (i.e. when using gcounter)\n                                %% as the CRDT value is not distributed, only its update commands.\n                                DataType:apply_update(UpdateCmd, 1, Acc)\n                        end,\n                        DataType:new(), LearntValue),\n    inform_client(read_done, Client, DataType:apply_query(QueryFun, CrdtVal)),\n    State;\n\n%%%%% write\n\n%% procedure ExecuteUpdate\non({req_start, {write, Client, Key, DataType, UpdateFun}}, State) ->\n    NewState =\n        case get_entry(Key, tablename(State)) of\n            undefined ->\n                NewEntry = new_entry(Key, DataType),\n                save_entry(NewEntry, tablename(State)),\n                add_proposer(State, comm:this());\n            _ -> State\n        end,\n\n    CmdId = {comm:this(), uid:get_pids_uid()}, %% unique identifier of this command\n    PropVal = gset:update_add({CmdId, UpdateFun}, gset:new()),\n\n    case waiting_clients(NewState) of\n        [] ->\n            %% we do not poll currently as the wait set was empty -> start polling\n            comm:send_local_after(?WRITE_DONE_POLLING_PERIOD, self(), {learnt_cmd_poller}),\n            ok;\n        _ ->\n            %% we are already polling\n            ok\n    end,\n    NewState2 = add_waiting_client(NewState, {CmdId, Key, Client}),\n    gen_component:post_op({receive_value, Key, DataType, PropVal},  NewState2);\n\n%% check which commands are learnt to notify the client\non({learnt_cmd_poller}, State) ->\n    %% notfiy all clients whose write was completed and remove the respective requests\n    %% from the wait set\n    NewList =\n       lists:filter(fun({CmdId, Key, Client}) ->\n                        case get_entry({learner, Key}, tablename(State)) of\n                            undefined -> true;\n                            Entry ->\n                                CmdSet = learner_learnt(Entry),\n                                case gset:exists(fun(E) -> element(1, E) =:= CmdId end, CmdSet) of\n                                    true ->\n                                        inform_client(write_done, Client),\n                                        false;\n                                    false ->\n                                        true\n                                end\n                        end\n                    end, waiting_clients(State)),\n\n    case NewList of\n        [] ->\n            %% no waiting clients... we can stop polling for now\n            ok;\n        _ ->\n            %% there are still waiting clients... continue to poll\n            comm:send_local_after(?WRITE_DONE_POLLING_PERIOD, self(), {learnt_cmd_poller}),\n            ok\n    end,\n\n    NewState = set_waiting_clients(State, NewList),\n    NewState;\n\n\n%% action ReceiveValue\non({receive_value, Key, DataType, Command}, State) ->\n    ProposerList = proposers(State),\n    [comm:send(Proposer, {internal_receive, Key, DataType, Command})\n     || Proposer <- ProposerList],\n\n    State;\n\n%% Internal Receive\non({internal_receive, Key, DataType, PropVal}, State) ->\n    Entry = get_entry(Key, DataType, tablename(State)),\n    NewBufVal = gset:merge(entry_bufval(Entry), PropVal),\n    NewEntry = entry_set_bufval(Entry, NewBufVal),\n    _ = save_entry(NewEntry, tablename(State)),\n\n    gen_component:post_op({propose, Key}, State);\n\n%% Propose\non({propose, Key}, State) ->\n    Entry = get_entry(Key, tablename(State)),\n    PropVal = entry_propval(Entry),\n    BufVal = entry_bufval(Entry),\n    NewPropVal = gset:merge(PropVal, BufVal),\n    case entry_status(Entry) =:= passive\n        andalso gset:lt(PropVal, NewPropVal) of\n        true ->\n            NewPropNum = entry_propnum(Entry) + 1,\n            NewEntry1 = entry_set_propval(Entry, NewPropVal),\n            NewEntry2 = entry_set_status(NewEntry1, active),\n            NewEntry3 = entry_set_ackcount(NewEntry2, 0),\n            NewEntry4 = entry_set_nackcount(NewEntry3, 0),\n            NewEntry5 = entry_set_propnum(NewEntry4, NewPropNum),\n            _ = save_entry(NewEntry5, tablename(State)),\n\n            DataType = entry_datatype(NewEntry5),\n            Msg =  {gla_acceptor, propose, '_', comm:this(), key, DataType, NewPropNum, BufVal},\n            send_to_all_replicas(Key, Msg),\n\n            NewEntry6 = entry_set_bufval(NewEntry5, gset:new()),\n            _ = save_entry(NewEntry6, tablename(State)),\n\n            State;\n        false ->\n            State\n    end;\n\n%% Proposer Ack\non({ack_reply, Key, ProposalNumber}, State) ->\n    Entry = get_entry(Key, tablename(State)),\n    case entry_propnum(Entry) =:= ProposalNumber of\n        true ->\n            NewAckCount = entry_ackcount(Entry) + 1,\n            NewEntry = entry_set_ackcount(Entry, NewAckCount),\n            _ = save_entry(NewEntry, tablename(State)),\n            gen_component:post_op({refine, Key}, State);\n        false -> State\n    end;\n\n%% Proposer Nack\non({nack_reply, Key, ProposalNumber, MissingValues}, State) ->\n    Entry = get_entry(Key, tablename(State)),\n    case entry_propnum(Entry) =:= ProposalNumber of\n        true ->\n            NewAckCount = entry_nackcount(Entry) + 1,\n            RefVal = gset:merge(MissingValues, entry_refval(Entry)),\n            NewEntry = entry_set_nackcount(Entry, NewAckCount),\n            NewEntry2 = entry_set_refval(NewEntry, RefVal),\n            _ = save_entry(NewEntry2, tablename(State)),\n            gen_component:post_op({refine, Key}, State);\n        false -> State\n    end;\n\n%% Proposer Refine\non({refine, Key}, State) ->\n    Entry = get_entry(Key, tablename(State)),\n    case entry_status(Entry) =:= active andalso\n         entry_nackcount(Entry) > 0 andalso\n         replication:quorum_accepted(entry_nackcount(Entry) + entry_ackcount(Entry)) of\n        true ->\n            NewPropNum = entry_propnum(Entry) + 1,\n            RefVal = entry_refval(Entry),\n            PropVal = gset:merge(entry_propval(Entry), RefVal),\n            NewEntry1 = entry_set_ackcount(Entry, 0),\n            NewEntry2 = entry_set_nackcount(NewEntry1, 0),\n            NewEntry3 = entry_set_propnum(NewEntry2, NewPropNum),\n            NewEntry4 = entry_set_propval(NewEntry3, PropVal),\n            NewEntry5 = entry_set_refval(NewEntry4, gset:new()),\n            _ = save_entry(NewEntry5, tablename(State)),\n\n            DataType = entry_datatype(Entry),\n            Msg =  {gla_acceptor, propose, '_', comm:this(), key, DataType, NewPropNum, RefVal},\n            send_to_all_replicas(Key, Msg),\n            State;\n        false ->\n            gen_component:post_op({decide, Key}, State)\n    end;\n\n%% Proposer Decide\non({decide, Key}, State) ->\n    Entry = get_entry(Key, tablename(State)),\n    case entry_status(Entry) =:= active andalso\n         replication:quorum_accepted(entry_ackcount(Entry)) of\n        true ->\n            PropVal = entry_propval(Entry),\n            NewEntry = entry_set_outval(Entry, PropVal),\n            NewEntry2 = entry_set_status(NewEntry, passive),\n            _ = save_entry(NewEntry2, tablename(State)),\n            gen_component:post_op({propose, Key}, State);\n        false ->\n            State\n    end;\n\n%% Learner ack\non({learner_ack_reply, Key, DataType, ProposalNumber, Value, ProposerId}, State) ->\n    Entry = case get_entry({learner, Key}, tablename(State)) of\n                undefined -> new_learner_entry(Key, DataType);\n                E -> E\n            end,\n\n    NewEntry = learner_add_vote(Entry, ProposerId, ProposalNumber),\n    VoteCount = learner_votes(NewEntry, ProposerId, ProposalNumber),\n    Learnt = learner_learnt(NewEntry),\n    NewEntry2 =\n        case replication:quorum_accepted(VoteCount) andalso\n            gset:lt(Learnt, Value) of\n            true ->\n                learner_set_learnt(NewEntry, Value);\n            false ->\n                NewEntry\n        end,\n    _ = save_entry(NewEntry2, tablename(State)),\n\n    %% lazily complete lists of active proposers\n    add_proposer(State, ProposerId);\n\n%% procedure LearntValue\non({learnt_value, Key, Client}, State) ->\n    Ret =\n        case get_entry({learner, Key}, tablename(State)) of\n            undefined -> gset:new();\n            Entry -> learner_learnt(Entry)\n        end,\n    comm:send(Client, {Ret}),\n    State;\n\n%% Internal to get local acceptor process\non({local_range_req, Key, Message, {get_state_response, LocalRange}}, State) ->\n    Keys = replication:get_keys(Key),\n\n    LocalKeys = lists:filter(fun(K) -> intervals:in(K, LocalRange) end, Keys),\n\n    K = case LocalKeys of\n        [] ->\n            ?TRACE(\"cannot send locally ~p ~p \", [Keys, LocalRange]),\n            %% the local dht node is not responsible for any replca... route to\n            %% random replica\n            Idx = randoms:rand_uniform(1, length(Keys)+1),\n            lists:nth(Idx, Keys);\n        [H|_] ->\n            %% use replica managed by local dht node\n            H\n        end,\n\n    Dest = pid_groups:find_a(routing_table),\n    LookupEnvelope = dht_node_lookup:envelope(3, setelement(5, Message, K)),\n    comm:send_local(Dest, {?lookup_aux, K, 0, LookupEnvelope}),\n    State.\n\n%%%%%%%%%%%%%%%% message sending helpers\n\n-spec send_to_all_replicas(?RT:key(), tuple()) -> ok.\nsend_to_all_replicas(Key, Message) ->\n    %% assert element(3, message) =:= '_'\n    %% assert element(6, message) =:= key\n    send_to_all_replicas(Key, Message, not ?ROUTING_DISABLED).\n\n-spec send_to_all_replicas(?RT:key(), tuple(), boolean()) -> ok.\nsend_to_all_replicas(Key, Message, _Routing=true) ->\n    Dest = pid_groups:find_a(routing_table),\n\n    _ = [begin\n            LookupEnvelope = dht_node_lookup:envelope(3, setelement(5, Message, K)),\n            comm:send_local(Dest, {?lookup_aux, K, 0, LookupEnvelope})\n         end\n        || K <- replication:get_keys(Key)],\n    ok.\n\n%%%%%%%%%%%%%%%%%% access of proposer entry\n-spec entry_datatype(entry())       -> crdt:crdt_module().\nentry_datatype(Entry)               -> element(2, Entry).\n\n-spec entry_status(entry())         -> passive | active.\nentry_status(Entry)                 -> element(4, Entry).\n-spec entry_ackcount(entry())       -> non_neg_integer().\nentry_ackcount(Entry)               -> element(5, Entry).\n-spec entry_nackcount(entry())      -> non_neg_integer().\nentry_nackcount(Entry)              -> element(6, Entry).\n-spec entry_propnum(entry())        -> non_neg_integer().\nentry_propnum(Entry)                -> element(7, Entry).\n-spec entry_propval(entry())        -> gset:crdt().\nentry_propval(Entry)                -> element(8, Entry).\n-spec entry_bufval(entry())        -> gset:crdt().\nentry_bufval(Entry)                -> element(9, Entry).\n-spec entry_refval(entry())        -> gset:crdt().\nentry_refval(Entry)                -> element(11, Entry).\n\n-spec entry_set_status(entry(), passive | active) -> entry().\nentry_set_status(Entry, Status) -> setelement(4, Entry, Status).\n-spec entry_set_ackcount(entry(), non_neg_integer()) -> entry().\nentry_set_ackcount(Entry, X)    -> setelement(5, Entry, X).\n-spec entry_set_nackcount(entry(), non_neg_integer()) -> entry().\nentry_set_nackcount(Entry, X)   -> setelement(6, Entry, X).\n-spec entry_set_propnum(entry(), non_neg_integer()) -> entry().\nentry_set_propnum(Entry, X)     -> setelement(7, Entry, X).\n-spec entry_set_propval(entry(), gset:crdt()) -> entry().\nentry_set_propval(Entry, X)     -> setelement(8, Entry, X).\n-spec entry_set_bufval(entry(), gset:crdt()) -> entry().\nentry_set_bufval(Entry, X)      -> setelement(9, Entry, X).\n-spec entry_set_outval(entry(), gset:crdt()) -> entry().\nentry_set_outval(Entry, X)      -> setelement(10, Entry, X).\n-spec entry_set_refval(entry(), gset:crdt()) -> entry().\nentry_set_refval(Entry, X)      -> setelement(11, Entry, X).\n\n%%%%%%%%%%%%%%%% access of learner entry\n-spec learner_learnt(learner_entry()) -> gset:crdt().\nlearner_learnt(Entry) -> element(3, Entry).\n\n-spec learner_set_learnt(learner_entry(), gset:crdt()) -> learner_entry().\nlearner_set_learnt(Entry, Value) -> setelement(3, Entry, Value).\n\n-spec learner_votes(learner_entry(), any(), non_neg_integer()) -> non_neg_integer().\nlearner_votes(Entry, Proposer, ProposalNumber) ->\n    Votes = element(4, Entry),\n    Key = {Proposer, ProposalNumber},\n    case lists:keyfind(Key, 1, Votes) of\n        false -> 0;\n        E -> element(2, E)\n    end.\n\n-spec learner_add_vote(learner_entry(), any(), non_neg_integer()) -> learner_entry().\nlearner_add_vote(Entry, Proposer, ProposalNumber) ->\n    Votes = element(4, Entry),\n    Dummy = {{Proposer, -1}, 0},\n\n    {HighestPropNum, OldCount} = lists:max([{PropNum, OldCount} ||\n                                            {{P, PropNum}, OldCount} <- [Dummy | Votes], P =:= Proposer]),\n    case HighestPropNum > ProposalNumber of\n        true -> Entry;\n        false ->\n            NewVoteCount = case ProposalNumber =:= HighestPropNum of\n                               true -> OldCount + 1;\n                               false -> 1\n                           end,\n            NewList = [E || E={{P,_},_} <- Votes, P =/= Proposer],\n            NewVotes = [{{Proposer, ProposalNumber}, NewVoteCount}| NewList],\n            setelement(4, Entry, NewVotes)\n    end.\n\n%%%%%%%%%%%%%%% creation/retrieval/save of entries\n-spec new_entry(?RT:key(), crdt:crdt_module()) -> entry().\nnew_entry(Key, DataType) ->\n    {lowest_key(Key), DataType, [], passive, 0, 0, 0, gset:new(), gset:new(), gset:new(), gset:new()}.\n\n-spec new_learner_entry(?RT:key(), crdt:crdt_module()) -> learner_entry().\nnew_learner_entry(Key, DataType) ->\n    {{learner, lowest_key(Key)}, DataType, gset:new(), []}.\n\n\n-spec get_entry(?RT:key() | {learner, ?RT:key()}, ?PDB:tableid()) -> entry() | learner_entry() | undefined.\nget_entry({learner, Key}, TableName) ->\n    ?PDB:get({learner, lowest_key(Key)}, TableName);\nget_entry(Key, TableName) ->\n    ?PDB:get(lowest_key(Key), TableName).\n\n-spec get_entry(?RT:key(), crdt:crdt_module(), ?PDB:tableid()) -> entry().\nget_entry(Key, DataType, TableName) ->\n    case ?PDB:get(lowest_key(Key), TableName) of\n        undefined -> new_entry(Key, DataType);\n        Entry -> Entry\n    end.\n\n-spec save_entry(entry() | learner_entry(), ?PDB:tableid()) -> ok.\nsave_entry(NewEntry, TableName) ->\n    ?PDB:set(NewEntry, TableName).\n\n\n%%%%%%%%%%%%%% functions to retrieve/modify state\n-spec tablename(state()) -> ?PDB:tableid().\ntablename(State) -> element(1, State).\n\n-spec proposers(state()) -> [any()].\nproposers(State) -> element(4, State).\n-spec add_proposer(state(), any()) -> state().\nadd_proposer(State, Proposer) ->\n    Known =  element(4, State),\n    case lists:member(Proposer, Known) of\n        true -> State;\n        false -> setelement(4, State, [Proposer | Known])\n    end.\n\n-spec waiting_clients(state()) -> [{any(), ?RT:key(), any()}].\nwaiting_clients(State) -> element(5, State).\n-spec set_waiting_clients(state(), [{any(), ?RT:key(), any()}]) -> state().\nset_waiting_clients(State, Clients) -> setelement(5, State, Clients).\n-spec add_waiting_client(state(), {any(), ?RT:key(), any()}) -> state().\nadd_waiting_client(State, Client) ->\n    Known =  element(5, State),\n    case lists:member(Client, Known) of\n        true -> State;\n        false -> setelement(5, State, [Client | Known])\n    end.\n\n%%%%%%%%%%%%%% misc\n-spec inform_client(write_done, comm:mypid()) -> ok.\ninform_client(write_done, Client) ->\n    case is_tuple(Client) of\n        false -> comm:send_local(Client, {write_done});\n        true -> comm:send(Client, {write_done})\n    end.\n\n-spec inform_client(read_done, comm:mypid(), any()) -> ok.\ninform_client(read_done, Client, Value) ->\n    comm:send_local(Client, {read_done, Value}).\n\n-spec lowest_key(?RT:key()) -> ?RT:key().\nlowest_key(Key) ->\n    hd(lists:sort(replication:get_keys(Key))).\n"
  },
  {
    "path": "src/crdt/pncounter_on_cseq.erl",
    "content": "% @copyright 2014-2018 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    API to increase/decrease a replicated counter using a\n%%         state-based PN-Counter CRDT .\n%% @end\n-module(pncounter_on_cseq).\n-author('skrzypczak@zib.de').\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([read/1]).\n-export([read_state/1]).\n\n-export([add/2]).\n-export([subtract/2]).\n\n%% Reads counter with strong consistency semantics\n-spec read(client_key()) -> {ok, client_value()}.\nread(Key) ->\n    crdt_on_cseq:read(Key, pncounter, fun pncounter:query_counter/1).\n\n%% Reads the full state with strong consistentcy semantics\n-spec read_state(client_key()) -> {ok, client_value()}.\nread_state(Key) ->\n    crdt_on_cseq:read(Key, pncounter, fun crdt:query_noop/1).\n\n%% Increase counter with strong consistency semantics\n-spec add(client_key(), non_neg_integer()) -> ok.\nadd(Key, ToAdd) ->\n    UpdateFun = fun(ReplicaId, CRDT) -> pncounter:update_add(ReplicaId, ToAdd, CRDT) end,\n    crdt_on_cseq:write(Key, pncounter, UpdateFun).\n\n%% Decrease counter with strong consistency semantics\n-spec subtract(client_key(), non_neg_integer()) -> ok.\nsubtract(Key, ToSubtract) ->\n    UpdateFun = fun(ReplicaId, CRDT) -> pncounter:update_subtract(ReplicaId, ToSubtract, CRDT) end,\n    crdt_on_cseq:write(Key, pncounter, UpdateFun).\n"
  },
  {
    "path": "src/crdt/types/crdt.erl",
    "content": "%  @copyright 2008-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc Some common type defs and functions for CRDTs.\n%% @end\n-module(crdt).\n-author('skrzypczak@zib.de').\n\n-export_type([crdt/0, crdt_module/0, update_fun/0, query_fun/0]).\n\n-export([update_noop/1, update_noop/2, query_noop/1]).\n-export([check_config/0, proposer_module/0, acceptor_module/0]).\n\n% for tester\n-export([tester_create_update_fun/1, tester_create_query_fun/1]).\n\n-type crdt()        :: gcounter:crdt() | pncounter:crdt() | gset:crdt().\n-type crdt_module() :: gcounter | pncounter | gset.\n\n-type update_fun()  :: fun((crdt()) -> crdt()) | fun((non_neg_integer(), crdt()) -> crdt()) |\n\t\t\t\t\t\t[fun((crdt()) -> crdt())] | [fun((non_neg_integer(), crdt()) -> crdt())].\n-type query_fun()   :: fun((crdt()) -> term()) | [fun((crdt()) -> term())].\n\n-spec update_noop(crdt()) -> crdt().\nupdate_noop(CRDT) -> CRDT.\n\n-spec update_noop(non_neg_integer(), crdt()) -> crdt().\nupdate_noop(_ReplicaId, CRDT) -> CRDT.\n\n-spec query_noop(crdt()) -> term().\nquery_noop(CRDT) -> CRDT.\n\n-spec tester_create_update_fun(0) -> update_fun().\ntester_create_update_fun(0) -> fun update_noop/1.\n\n-spec tester_create_query_fun(0) -> query_fun().\ntester_create_query_fun(0) -> fun query_noop/1.\n\n-spec rsm_implementation() -> {module(), module()}.\nrsm_implementation() ->\n    Available =\n        [{wf_crdt_paxos, {crdt_wait_free_wrapper, crdt_acceptor}},\n         {crdt_paxos, {crdt_proposer, crdt_acceptor}},\n         {zheng, {zheng_proposer, zheng_acceptor}},\n         {lattice, {gla_proposer, gla_acceptor}}],\n    element(2, lists:keyfind(config:read(crdt_rsm), 1, Available)).\n\n-spec proposer_module() -> module().\nproposer_module() -> element(1, rsm_implementation()).\n-spec acceptor_module() -> module().\nacceptor_module() -> element(2, rsm_implementation()).\n\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_atom(crdt_rsm).\n\n"
  },
  {
    "path": "src/crdt/types/crdt_beh.erl",
    "content": "%  @copyright 2008-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc State-based CRDT behaviour\n%% @end\n%%\n-module(crdt_beh).\n-author('skrzypczak@zib.de').\n\n-ifdef(have_callback_support).\n-include(\"scalaris.hrl\").\n\n-type crdt() :: term().\n\n-callback new() -> crdt().\n\n-callback apply_update(crdt:update_fun(), non_neg_integer(), crdt()) -> crdt().\n-callback apply_query(crdt:query_fun(), crdt()) -> term().\n-callback merge(crdt(), crdt()) -> crdt().\n\n-callback eq(crdt(), crdt()) -> boolean().\n-callback lteq(crdt(), crdt()) -> boolean().\n\n-else.\n\n-export([behaviour_info/1]).\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n    [\n        {new, 0},\n        {apply_update, 3}, {apply_query, 2},\n        {merge, 2},\n        {eq, 2}, {lteq, 2}\n    ];\nbehaviour_info(_Other) ->\n    undefined.\n\n-endif.\n\n"
  },
  {
    "path": "src/crdt/types/crdt_beh.hrl",
    "content": "%  @copyright 2008-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc State-based CRDT behaviour\n%% @end\n\n-author('skrzypczak@zib.de').\n\n-export_type([crdt/0]).\n\n-export([new/0, apply_update/3, apply_query/2, merge/2, eq/2, lteq/2]).\n\n-spec apply_update(crdt:update_fun(), non_neg_integer(), crdt()) -> crdt().\napply_update(U, ReplicaId, CRDT) ->\n    case erlang:fun_info(U, arity) of\n        {arity, 1} -> U(CRDT);\n        {arity, 2} -> U(ReplicaId, CRDT)\n    end.\n\n-spec apply_query(crdt:query_fun(), crdt()) -> term().\napply_query(Q, CRDT) -> Q(CRDT).\n\n"
  },
  {
    "path": "src/crdt/types/gcounter.erl",
    "content": "%  @copyright 2008-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc Implementation of a G-Counter (Grow-only Counter) state-based CRDT.\n%% @end\n%% @version $Id$\n\n-module(gcounter).\n-author('skrzypczak@zib.de').\n-vsn('Id$').\n\n-include(\"scalaris.hrl\").\n-define(R, config:read(replication_factor)).\n\n-export([update_add/3]).\n-export([query_counter/1]).\n\n-export([update_nth/3]). %% internal for usage in pncounter\n\n-behaviour(crdt_beh).\n\n-opaque crdt() :: [non_neg_integer()].\n\n-include(\"crdt_beh.hrl\").\n\n\n-spec new() -> crdt().\nnew() -> [0 || _ <- lists:seq(1, ?R)].\n\n-spec merge(crdt(), crdt()) -> crdt().\nmerge(CRDT1, CRDT2) -> lists:zipwith(fun(A, B) -> max(A,  B) end, CRDT1, CRDT2).\n\n-spec eq(crdt(), crdt()) -> boolean().\neq(CRDT1, CRDT2) -> CRDT1 =:= CRDT2.\n\n-spec lteq(crdt(), crdt()) -> boolean().\nlteq(CRDT1, CRDT2) ->\n    Lteq = lists:foldl(fun({E1, E2}, Acc) ->\n                                Acc andalso E1 =< E2\n                           end, true, lists:zip(CRDT1, CRDT2)),\n    Lteq.\n\n%%%%%%%%%%%%%%% Available update and query functions\n\n-spec update_add(non_neg_integer(), non_neg_integer(), crdt()) -> crdt().\nupdate_add(ReplicaId, ToAdd, CRDT) -> update_nth(CRDT, ReplicaId, ToAdd).\n\n-spec query_counter(crdt()) -> non_neg_integer().\nquery_counter(CRDT) -> lists:foldl(fun(E, Acc) -> E + Acc end, 0, CRDT).\n\n%%%%%%%%%%%%%%% Internal\n-spec update_nth(crdt(), non_neg_integer(), non_neg_integer()) -> crdt().\nupdate_nth(CRDT, 0, _ToAdd) -> CRDT;\nupdate_nth(CRDT, Idx, _ToAdd) when length(CRDT) < Idx -> CRDT;\nupdate_nth(CRDT, Idx, ToAdd) -> update_nth(CRDT, Idx, ToAdd, 1).\n\n-spec update_nth(crdt(), non_neg_integer(), non_neg_integer(), non_neg_integer()) -> crdt().\nupdate_nth(_CRDT=[H|T], Cur, ToAdd, Cur) -> [H + ToAdd | T];\nupdate_nth(_CRDT=[H|T], Idx, ToAdd, Cur) -> [H | update_nth(T, Idx, ToAdd, Cur+1)].\n\n\n"
  },
  {
    "path": "src/crdt/types/gset.erl",
    "content": "%  @copyright 2008-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc Implementation of a G-Set (Grow-only Set) state-based CRDT.\n%% @end\n%% @version $Id$\n\n-module(gset).\n-author('skrzypczak@zib.de').\n-vsn('Id$').\n\n-include(\"scalaris.hrl\").\n\n-export([update_add/2]).\n-export([query_lookup/2]).\n\n%% GLA utility functions\n-export([lt/2]).\n-export([exists/2]).\n-export([subtract/2]).\n-export([fold/3]).\n\n-behaviour(crdt_beh).\n-define(SET, ordsets).\n-opaque crdt() :: ?SET:ordset(term()).\n\n-include(\"crdt_beh.hrl\").\n\n-spec new() -> crdt().\nnew() -> ?SET:new().\n\n-spec merge(crdt(), crdt()) -> crdt().\nmerge(CRDT1, CRDT2) -> ?SET:union(CRDT1, CRDT2).\n\n-spec eq(crdt(), crdt()) -> boolean().\neq(CRDT1, CRDT2) ->\n    ?SET:is_subset(CRDT1, CRDT2) andalso\n    ?SET:is_subset(CRDT2, CRDT1).\n\n-spec lteq(crdt(), crdt()) -> boolean().\nlteq(CRDT1, CRDT2) -> ?SET:is_subset(CRDT1, CRDT2).\n\n%%%%%%%%%%%%%%% Available update and query functions\n\n-spec update_add(term(), crdt()) -> crdt().\nupdate_add(ToAdd, CRDT) -> ?SET:add_element(ToAdd, CRDT).\n\n-spec query_lookup(term(), crdt()) -> boolean().\nquery_lookup(Element, CRDT) -> ?SET:is_element(Element, CRDT).\n\n%%%%%%%%%%%%%%% Utility functions used in GLA implementation\n-spec lt(crdt(), crdt()) -> boolean().\nlt(CRDT1, CRDT2) -> lteq(CRDT1, CRDT2) andalso not eq(CRDT1, CRDT2).\n\n-spec exists(fun((term()) -> boolean()), crdt()) -> boolean().\nexists(PredFun, CRDT) ->\n    ?SET:fold(fun(_, true) -> true;\n                     (E, false) ->\n                         PredFun(E)\n                  end, false, CRDT).\n\n-spec fold(fun((term(), term()) -> term()), term(), crdt()) -> term().\nfold(Fun, Acc0, CRDT) ->\n    ?SET:fold(Fun, Acc0, CRDT).\n\n-spec subtract(crdt(), crdt()) -> crdt().\nsubtract(CRDT1, CRDT2) -> ?SET:subtract(CRDT1, CRDT2).\n"
  },
  {
    "path": "src/crdt/types/optorset.erl",
    "content": "%  @copyright 2008-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc Implementation of an optimized OR-Set state-based CRDT\n%% as specified by https://arxiv.org/pdf/1210.3368.pdf\n%% @end\n%% @version $Id$\n\n-module(optorset).\n-author('skrzypczak@zib.de').\n-vsn('Id$').\n\n-include(\"scalaris.hrl\").\n-define(R, config:read(replication_factor)).\n-define(SET, ordsets).\n\n-export([lteq2/2]).\n\n-export([update_add/3]).\n-export([update_remove/2]).\n\n-export([query_contains/2]).\n-export([query_elements/1]).\n\n-behaviour(crdt_beh).\n\n-type vector() :: [non_neg_integer()].\n-type element() :: {Element::term(), TimeStamp::term(), ReplicaID::term()}.\n-opaque crdt() :: {E::?SET:ordset(element()), V::vector()}.\n\n-include(\"crdt_beh.hrl\").\n\n-spec new() -> crdt().\nnew() -> {?SET:new(), [0 || _ <- lists:seq(1, ?R)]}.\n\n-spec merge(crdt(), crdt()) -> crdt().\nmerge(_CRDT1={E1, V1}, _CRDT2={E2, V2}) ->\n    %% TODO: make this more efficient?\n    M = ?SET:union(E1, E2),\n    T2 = ?SET:subtract(E1, E2),\n    M2 = ?SET:filter(fun({_E, C, I}) -> C > vector_get(I, V2) end, T2),\n    T3 = ?SET:subtract(E2, E1),\n    M3 = ?SET:filter(fun({_E, C, I}) -> C > vector_get(I, V1) end, T3),\n    U = ?SET:union([M, M2, M3]),\n\n    % find the largest C of all {E, _, I} tripples in linear time\n    D = ?SET:fold(fun({E, C, I}, DictAcc) ->\n                        dict:update({E,I}, fun(A) -> max(A, C) end, C, DictAcc)\n                   end, dict:new(), U),\n    % from dict to set again...\n    NewE =  dict:fold(fun({E, I}, C, SetAcc) ->\n                        ?SET:add_element({E, C, I}, SetAcc)\n                      end, ?SET:new(), D),\n    NewV = vector_max(V1, V2),\n\n    {NewE, NewV}.\n\n-spec eq(crdt(), crdt()) -> boolean().\neq(_CRDT1={E1, V1}, _CRDT2={E2, V2}) ->\n    vector_eq(V1, V2) andalso ?SET:is_subset(E1, E2) andalso ?SET:is_subset(E2, E1).\n\n-spec lteq(crdt(), crdt()) -> boolean().\nlteq(_CRDT1={E1, V1}, _CRDT2={E2, V2}) ->\n    case vector_lteq(V1, V2) of\n        false -> false;\n        true ->\n            %% optimized implementation compared to paper exploiting the fact\n            %% that V1 =< V2\n            Fun = fun({_, C, I}, SetAcc) ->\n                        case C =< vector_get(I, V1) of\n                            true -> ?SET:add_element({C, I}, SetAcc);\n                            false -> SetAcc\n                        end\n                  end,\n\n            X = ?SET:fold(Fun, ?SET:new(), E1),\n            X2 = ?SET:fold(Fun, ?SET:new(), E2),\n\n            %% Is the same as is_subset(D - X,D - X2) with\n            %% D = set of all possible (C,I) pairs\n            ?SET:is_subset(X2, X)\n    end.\n\n%% lteq implementation as specified in paper... used for testing purposes\n-spec lteq2(crdt(), crdt()) -> boolean().\nlteq2(_CRDT1={E1, V1}, _CRDT2={E2, V2}) ->\n    case vector_lteq(V1, V2) of\n        false -> false;\n        true ->\n            TFun = fun({_, C, I}, SetAcc) -> ?SET:add_element({C,I}, SetAcc) end,\n            X1 = ?SET:fold(TFun, ?SET:new(), E1),\n            X2 = ?SET:fold(TFun, ?SET:new(), E2),\n\n            Possible1 = [{B, A} || A <- lists:seq(1, ?R), B <- lists:seq(1, vector_get(A, V1))],\n            Possible2 = [{B, A} || A <- lists:seq(1, ?R), B <- lists:seq(1, vector_get(A, V2))],\n\n            R1 = lists:foldl(fun(E, Acc) ->\n                                     case ?SET:is_element(E, X1) of\n                                         true -> Acc;\n                                         false -> ?SET:add_element(E, Acc)\n                                     end\n                             end, ?SET:new(), Possible1),\n\n            R2 = lists:foldl(fun(E, Acc) ->\n                                     case ?SET:is_element(E, X2) of\n                                         true -> Acc;\n                                         false -> ?SET:add_element(E, Acc)\n                                     end\n                             end, ?SET:new(), Possible2),\n\n            ?SET:is_subset(R1, R2)\n    end.\n\n%%%%%%%%%%%%%%%%%%%%% Update and Query functions\n-spec update_add(ReplicaId::non_neg_integer(), Element::term(), CRDT::crdt()) -> crdt().\nupdate_add(ReplicaId, Element, _CRDT={E, V}) ->\n    {TimeStamp, V2} = vector_inc_and_get(ReplicaId, V),\n    E2 = ?SET:add_element({Element, TimeStamp, ReplicaId}, E),\n    E3 = ?SET:filter(fun({TE, TC, _TI}) ->\n                            not (TE =:= Element andalso TC < TimeStamp)\n                     end, E2),\n    {E3, V2}.\n\n-spec update_remove(Element::term(), C::crdt()) -> crdt().\nupdate_remove(Element, _CRDT={E, V}) ->\n    E2 = ?SET:filter(fun({TE, _TC, _TI}) -> Element =/= TE end, E),\n    {E2, V}.\n\n-spec query_elements(crdt()) -> ?SET:ordset(element()).\nquery_elements(_CRDT={Elements, _V}) ->\n    ?SET:fold(fun({E, _C, _I}, AccSet) -> ?SET:add_element(E, AccSet) end,\n              ?SET:new(), Elements).\n\n-spec query_contains(term(), crdt()) -> boolean().\nquery_contains(Element, _CRDT={Elements, _V}) ->\n    ?SET:fold(fun({E, _C, _I}, _Acc) when Element =:= E -> true;\n                 (_C, Acc) -> Acc\n              end, false, Elements).\n\n%%%%%%%%%%%%%%%%%%%%% Some helper funcitons for vector manipulation\n%% TODO use dict instead of list?\n-spec vector_max(vector(), vector()) -> vector().\nvector_max([], []) -> [];\nvector_max([H1 | T1], [H2 | T2]) -> [max(H1, H2) | vector_max(T1, T2)].\n\n-spec vector_get(non_neg_integer(), vector()) -> non_neg_integer().\nvector_get(Idx, V) -> lists:nth(Idx, V).\n\n-spec vector_inc_and_get(non_neg_integer(), vector()) -> {non_neg_integer(), vector()}.\nvector_inc_and_get(Idx, V) ->\n    TimeStamp = lists:nth(Idx, V) + 1,\n    V2 = lists:sublist(V, Idx - 1) ++ [TimeStamp] ++ lists:nthtail(Idx, V),\n    {TimeStamp, V2}.\n\n-spec vector_eq(vector(), vector()) -> boolean().\nvector_eq(V1, V2) -> V1 =:= V2.\n\n-spec vector_lteq(vector(), vector()) -> boolean().\nvector_lteq([], []) -> true;\nvector_lteq([H1 | _T1], [H2 | _T2]) when H1 > H2 -> false;\nvector_lteq([_H1 | T1], [_H2 | T2]) -> vector_lteq(T1, T2).\n\n\n\n"
  },
  {
    "path": "src/crdt/types/pncounter.erl",
    "content": "%  @copyright 2008-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc Implementation of a PN-Counter (Positive-Negative Counter) state-based CRDT.\n%% @end\n-module(pncounter).\n-author('skrzypczak@zib.de').\n\n-include(\"scalaris.hrl\").\n-define(R, config:read(replication_factor)).\n\n-export([update_add/3]).\n-export([update_subtract/3]).\n-export([query_counter/1]).\n\n-behaviour(crdt_beh).\n\n-opaque crdt() :: {gcounter:crdt(), gcounter:crdt()}.\n\n-include(\"crdt_beh.hrl\").\n\n\n-spec new() -> crdt().\nnew() -> {gcounter:new(), gcounter:new()}.\n\n-spec merge(crdt(), crdt()) -> crdt().\nmerge({P1, N1}, {P2, N2}) -> {gcounter:merge(P1, P2), gcounter:merge(N1, N2)}.\n\n-spec eq(crdt(), crdt()) -> boolean().\neq({P1, N1}, {P2, N2}) -> gcounter:eq(P1, P2) andalso gcounter:eq(N1, N2).\n\n-spec lteq(crdt(), crdt()) -> boolean().\nlteq({P1, N1}, {P2, N2}) -> gcounter:lteq(P1, P2) andalso gcounter:lteq(N1, N2).\n\n%%%%%%%%%%%%%%% Available update and query functions\n\n-spec update_add(non_neg_integer(), non_neg_integer(), crdt()) -> crdt().\nupdate_add(ReplicaId, ToAdd, _CRDT={P, N}) ->\n        {gcounter:update_nth(P, ReplicaId, ToAdd), N}.\n\n-spec update_subtract(non_neg_integer(), non_neg_integer(), crdt()) -> crdt().\nupdate_subtract(ReplicaId, ToSubtract, _CRDT={P, N}) ->\n        {P, gcounter:update_nth(N, ReplicaId, ToSubtract)}.\n\n-spec query_counter(crdt()) -> non_neg_integer().\nquery_counter(_CRDT={P, N}) -> gcounter:query_counter(P) - gcounter:query_counter(N).\n\n\n"
  },
  {
    "path": "src/crdt/zheng_acceptor.erl",
    "content": "% @copyright 2012-2018 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak\n%% @doc U/Q Replicated State Machine implementation of Zheng et al.\n%%      For every key, a separate instance of the protocol is managed.\n%% @end\n-module(zheng_acceptor).\n-author('skrzypczak@zib.de').\n\n%-define(TRACE(X,Y), ct:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-include(\"scalaris.hrl\").\n\n-define(PDB, db_prbr).\n\n%%% functions for module where embedded into\n-export([on/2, init/1, close/1, close_and_delete/1]).\n-export([check_config/0]).\n\n%% let fetch the number of DB entries\n-export([get_load/1]).\n\n%% only for unittests\n-export([tab2list_raw_unittest/1]).\n\n%% only during recover\n-export([tab2list/1]).\n\n-export_type([state/0]).\n\n-type state() :: {?PDB:db(), [comm:mypid_plain()]}.\n\n-define(n, config:read(replication_factor)).\n-define(f, ((?n - 1) div 2)).\n\n\n-define(get(X, State), get_field(?i(X), State)).\n-define(set(X, NewV, State), set_field(?i(X), State, NewV)).\n-define(union(X, ListOfElements, State), union_buffer(?i(X), ListOfElements, State)).\n-define(add(X, Cmd, State), add_to_buffer(?i(X), Cmd, State)).\n\n-define(rinc(X, State), inc_field(?r(X), State)).\n-define(rget(X, State), get_field(?r(X), State)).\n-define(rset(X, NewV, State), set_field(?r(X), State, NewV)).\n-define(radd(X, Buffer, State), add_to_list(?r(X), Buffer, State)).\n\n-define(i(X), #pstate.X).\n-define(r(X), #replies.X).\n\n-record(replies,\n  {\n    count = 0 :: non_neg_integer(),\n    accept_count = 0 :: non_neg_integer(),\n    decide_val = [] :: [buffer()],\n    reject_val = [] :: [buffer()]\n  }).\n\n-record(pstate,\n  {\n    cid = 0 :: non_neg_integer(), %% used to give command unique identifier\n\n    %% core protocol variables\n    seq = 0 :: non_neg_integer(),\n    max_seq = -1 :: integer(),\n\n    buff_val = new_buffer() :: buffer(),\n    %% is used to track if commands are learned to avoid creating unnecesasry noops\n    %% for queries\n    last_update_cmd_received_in_buf = null :: null | cmd(),\n    learned_val = dict:new() :: dict:dict(non_neg_integer(), buffer()),\n\n    accept_val = new_buffer() :: buffer(),\n    active = false :: boolean(),\n\n    %%% management of pending cmds to notify clients\n    to_notify_pending = [] :: [notify_cmd()], %% list before they are included in accept_val\n    wait_for_learned = dict:new() :: dict:dict(cmd_id(), [notify_cmd()]),\n\n    %% CRDT managment\n    crdt_type = null :: null | crdt:crdt_module(),\n    cmd_set = new_buffer() :: buffer(),\n    crdt = null :: null | crdt:crdt(),\n\n    %% loop variables for agreement loop\n    loop_val = new_buffer :: buffer(),\n    loop_iter = 0 :: non_neg_integer(),\n    replies = #replies{} :: #replies{} | done,\n\n    %% for waiting in prop functions\n    prop_wait_list = [] :: [tuple()]\n  }).\n\n%% TODO More sensible management of query commands... these do not have to be sent\n%% to remotely as they are noops for updates?\n-type client() :: {any(), any()}. %% reqid and pid\n-type notify_cmd() :: {client(), query_cmd() | notify_ok}.\n\n-type buffer() :: set:sets(cmd()).\n-type cmd() :: {cmd_id(), query_cmd() | update_cmd() | noop}.\n\n-type cmd_id() :: any().\n-type query_cmd() :: {qfun, crdt:query_fun() | [crdt:query_fun()]}.\n-type update_cmd() :: crdt:update_fun() | [crdt:update_fun()].\n\n\n%% initialize: return initial state.\n-spec init(atom() | tuple()) -> state().\ninit(DBName) ->\n  {?PDB:new(DBName), []}.\n\n%% @doc Closes the given DB (it may be recoverable using open/1 depending on\n%%      the DB back-end).\n-spec close(state()) -> true.\nclose(State) -> ?PDB:close(State).\n\n%% @doc Closes the given DB and deletes all contents (this DB can thus not be\n%%      re-opened using open/1).\n-spec close_and_delete(state()) -> true.\nclose_and_delete(State) -> ?PDB:close_and_delete(State).\n\n-spec on(tuple(), state()) -> state().\n%% Procedure ReceiveValue\non({zheng_acceptor, new_read, _Cons, Proposer, Key, ReqId, DataType, QueryFuns}, State) ->\n    ?TRACE(\"~p NEW QUERY COMMAND ~p ~p~n~p\", [Key, Proposer, ReqId, QueryFuns]),\n    PState = get_pstate(Key, State),\n\n    {Cid, NP1} = get_new_id(Key, PState),\n    QueryCmd = new_query_cmd(Cid, QueryFuns),\n    Client = new_client(ReqId, Proposer),\n\n    NP2 = add_to_notify(Client, QueryCmd, NP1),\n    NP3 = ?add(buff_val, QueryCmd, NP2),\n    save_pstate(Key, NP3, State),\n\n    comm:send_local(self(), {zheng_acceptor, agree, Key, DataType}),\n    State;\n\n\non({zheng_acceptor, new_write, _Cons, Proposer, Key, ReqId, DataType, UpdateFuns}, State) ->\n    ?TRACE(\"~p NEW WRITE COMMAND ~p ~p~n~p\", [Key, Proposer, ReqId, UpdateFuns]),\n    PState = get_pstate(Key, State),\n\n    {Cid, NP1} = get_new_id(Key, PState),\n    UpdateCmd = new_update_cmd(Cid, UpdateFuns),\n    Client = new_client(ReqId, Proposer),\n\n    NP2 = add_to_notify(Client, UpdateCmd, NP1),\n    NP3 = ?add(buff_val, UpdateCmd, NP2),\n    NP4 = ?set(last_update_cmd_received_in_buf, UpdateCmd, NP3),\n    save_pstate(Key, NP4, State),\n\n    comm:send_local(self(), {zheng_acceptor, agree, Key, DataType}),\n    State;\n\n\n%% The following on-handlers implement the Agree Procedure in the paper\non({zheng_acceptor, agree, Key, DataType}, State) ->\n  PState = get_pstate(Key, State),\n  NP1 = init_pstate_if_not_already(PState, DataType),\n  Buffered = ?get(buff_val, NP1),\n\n  case ?get(active, NP1) =:= false andalso\n    (not is_empty(Buffered) orelse ?get(max_seq, NP1) >= ?get(seq, NP1)) of\n    true ->\n\n      %% As per otimization of the full paper, queries do not have be included\n      %% into the set (should not filter anything if notify buffer below is empty)\n      UpdateCmds = remove_query_commans(Buffered),\n\n      %% If we included commands from clients in this buffer that needs to be\n      %% notified of the result as soon as the commands are learend, add a noop\n      %% with an ID. If this noop is learned, notify all clients.\n      ToNotifyList = ?get(to_notify_pending, NP1),\n      NP2 =\n        case ToNotifyList =:= [] of\n          true ->\n            ?union(accept_val, UpdateCmds, NP1);\n          false ->\n            %% As all newyly received commands in the UpdateCmd set are always learned together,\n            %% we can just use some existing update to listing for completion instead\n            %% of creating a noop every time. If none exists, create a noop\n            T3 =\n              case ?get(last_update_cmd_received_in_buf, NP1) of\n                null ->\n                  {Cid, T1} = get_new_id(Key, NP1),\n                  NoopCmd = new_noop(Cid),\n                  T2 = ?union(accept_val, sets:add_element(NoopCmd, UpdateCmds), T1),\n                  set_to_notify_after_cmd_learned(ToNotifyList, NoopCmd, T2);\n                Cmd ->\n                  T1 = ?union(accept_val, UpdateCmds, NP1),\n                  set_to_notify_after_cmd_learned(ToNotifyList, Cmd, T1)\n              end,\n            ?set(to_notify_pending, [], T3)\n        end,\n\n      ?TRACE(\"START AGREE ~p~n~p\", [Key, {?get(active, NP1), ?get(accept_val, NP2),\n        ?get(max_seq, NP1), ?get(seq, NP1)}]),\n\n      NP3 = ?set(active, true, NP2),\n      NP4 = ?set(buff_val, new_buffer(), NP3),\n      NP5 = ?set(last_update_cmd_received_in_buf, null, NP4),\n      save_pstate(Key, NP5, State),\n      comm:send_local(self(), {zheng_acceptor, agree_loop, Key, _Iteration=1}),\n      State;\n    false ->\n      %% GUARD is false -> nothing to do\n      State\n  end;\n\n\non({zheng_acceptor, agree_loop, Key, Iteration}, State) ->\n  PState = get_pstate(Key, State),\n  Val = ?get(accept_val, PState),\n  %?TRACE(\"~p ~p\", [sets:size(Val),Val]),\n  SeqNum = ?get(seq, PState),\n  DataType = ?get(crdt_type, PState),\n  ?TRACE(\"~p LOOP START SEQNUM ~p Iteration ~p\", [Key, SeqNum, Iteration]),\n\n  NP1 = ?set(loop_iter, Iteration, PState),\n  NP2 = ?set(loop_val, Val, NP1),\n  NP3 = ?set(replies, new_replies(), NP2),\n  save_pstate(Key, NP3, State),\n\n  %% the second key entry is necesasry as this is not replaced by the dht and is used\n  %% for identification if one acceptor process manages multiple replicas\n  NewWaiting =\n    case Iteration =:= 1 of\n      true -> ?get(wait_for_learned, NP3);\n      false -> dict:new()\n    end,\n  Message = {zheng_acceptor, prop, Key, Key, DataType, comm:this(), Val, Iteration, SeqNum,\n    NewWaiting, '_'},\n  send_to_all_replicas(Key, Message, _ConsLookup=11, _KeyReplacement=3),\n\n  State;\n\non({zheng_acceptor, agree_loop_collect, Key,\n    {ReplyType, ReplyVal, Iter, SeqNum}}, State) ->\n  PState = get_pstate(Key, State),\n  Val = ?get(loop_val, PState),\n  CurIter = ?get(loop_iter, PState),\n  CurSeqNum = ?get(seq, PState),\n  Replies = ?get(replies, PState),\n\n  case CurSeqNum =:= SeqNum andalso CurIter =:= Iter andalso Replies =/= done of\n    false ->\n      %% is outdated reply from a previous iteration / seqnumber --> ignore msg\n      State;\n    true ->\n      NR1 = ?rinc(count, Replies),\n      NR2 =\n        case ReplyType of\n          accept -> ?rinc(accept_count, NR1);\n          reject -> ?radd(reject_val, ReplyVal, NR1);\n          decide -> ?radd(decide_val, ReplyVal, NR1)\n        end,\n      NP1 = ?set(replies, NR2, PState),\n      ?TRACE(\"~p LOOP COLLECT~n~p\", [Key, {CurSeqNum, CurIter, NR2}]),\n\n      TotalReplyCount = ?rget(count, NR2),\n      case TotalReplyCount =:= ?n - ?f of\n        false ->\n          %% not enough replies yet\n          save_pstate(Key, NP1, State),\n          State;\n        true ->\n          DecidedVals = ?rget(decide_val, NR2),\n          Accepted = ?rget(accept_count, NR2),\n          RejectVals = ?rget(reject_val, NR2),\n          NP2 =\n            case {DecidedVals =/= [], Accepted > (?n div 2)} of\n              {true, _} ->\n                LearnedVal = union(DecidedVals),\n                accept_loop_end(Key, LearnedVal, NP1, State);\n              {false, true} ->\n                accept_loop_end(Key, Val, NP1, State);\n              {false, false} ->\n                AcceptVal = ?get(accept_val, NP1),\n                TP = ?set(accept_val, union([AcceptVal | RejectVals]), NP1),\n                %% check if we are doing another loop iteration\n                case CurIter < ?f + 1 of\n                  true ->\n                    ?TRACE(\"~p LOOP NEW ITERATION ~p\", [Key, CurSeqNum]),\n                    comm:send_local(self(), {zheng_acceptor, agree_loop, Key, CurIter+1}),\n                    TP;\n                  false ->\n                    accept_loop_end(Key, Val, TP, State)\n                end\n            end,\n          NP3 = ?set(replies, done, NP2),\n          save_pstate(Key, NP3, State),\n          State\n      end\n  end;\n\n%%% prop function\non({zheng_acceptor, prop, Key, SrcKey, DataType, Client, Val, Iteration, SeqNum, NewWaiting,\n  _Cons}, State) ->\n  ?TRACE(\"~p prop: receievd msg from~n~p \", [Key, {SrcKey, Iteration, SeqNum}]),\n  PState = get_pstate(Key, State),\n  Seq = ?get(seq, PState),\n\n  case SeqNum < Seq of\n    true ->\n      ?TRACE(\"~p ADD TO BUFF ~n~p\", [Key, Val]),\n      NP1 = ?union(buff_val, Val, PState),\n      %% is the other replica is slower, merge the others' wait_for_learned dict\n      %% with ours, so that all clients of the included commands potentially receive\n      %% replies faster\n      NP2 =\n        case NewWaiting =:= dict:new() of\n          true -> NP1;\n          false ->\n            NW = dict:merge(\n              fun(_K, A, _B) ->\n                %% If there is a conflict, then both dicts already included the same commands\n                A\n              end, NewWaiting, ?get(wait_for_learned, NP1)),\n            ?set(wait_for_learned, NW, NP1)\n        end,\n\n      Lv = ?get(learned_val, PState),\n      send_prop_reply(Client, SrcKey, {decide, dict:fetch(SeqNum, Lv), Iteration, SeqNum}),\n      save_pstate(Key, NP2, State);\n    false ->\n      NP1 = ?set(max_seq, max(SeqNum, Seq), PState),\n      PropWaitList = ?get(prop_wait_list, NP1),\n      %% Optimization: only keep for each client the entry wait entry with the\n      %% highest {seqnum, iteration} pair. As newer msgs with a higher pair indicate\n      %% that the previous iteration is already complete. This prevents a lacking\n      %% behind process from being overwhelmed by lots of wait-list entries, as\n      %% all entries have to be checked.\n      {Highest, FilteredList} = lists:foldl(\n        fun(E = {ESrcKey, EClient, EVal, EIter, ESeq},{AccHigh = {ASrcKey, AClient, AVal, AIter, ASeq}, L}) ->\n          case ESrcKey =:= ASrcKey of\n            true ->\n              {S, I, V, C} = max({ESeq, EIter, EVal, EClient}, {ASeq, AIter, AVal, AClient}),\n              {{ASrcKey, C, V, I, S}, L };\n            false ->\n              {AccHigh, [E | L]}\n          end\n        end, {{SrcKey, Client, Val, Iteration, SeqNum}, []}, PropWaitList),\n\n      ?TRACE(\"~p Propowaitlist ~nOld ~p~n New ~p ~n Added~p\",[Key, PropWaitList, [Highest | FilteredList],\n        {SrcKey, Client, Val, Iteration, SeqNum}]),\n\n      NP2 = ?set(prop_wait_list, [Highest | FilteredList], NP1),\n      save_pstate(Key, NP2, State),\n      on({zheng_acceptor, prop_wait, Key}, State)\n  end,\n  comm:send_local(self(), {zheng_acceptor, agree, Key, DataType}),\n  State;\n\non({zheng_acceptor, prop_wait, Key}, State) ->\n  PState = get_pstate(Key, State),\n  WaitList = ?get(prop_wait_list, PState),\n  CurrentSeqNum = ?get(seq, PState),\n\n  NewWaitList =\n    lists:filter(\n      fun({SrcKey, Client, Val, Iteration, SeqNum}) ->\n        eval_prop_wait(Key, SrcKey, Client, Val, Iteration, CurrentSeqNum, SeqNum, State)\n      end, WaitList),\n  %?TRACE(\"TEST ~p\", [length(NewWaitList)]),\n  NP1 = get_pstate(Key, State), %% TODO: eval_prop_wait is ugly and has side-effects...\n  NP2 = ?set(prop_wait_list, NewWaitList, NP1),\n\n  save_pstate(Key, NP2, State),\n  State.\n\n%%%%%%%%%%%%%%%%%%%%%\n%% Functions that execute part of the core protocol logic\n%%%%%%%%%%%%%%%%%%%%%\neval_prop_wait(Key, SrcKey, Client, Val, Iteration, CurrentSeqNum, WaitForSeqNum, State) ->\n  ?TRACE(\"~p prop: waiting ~p ~p\", [Key,CurrentSeqNum, WaitForSeqNum]),\n  case CurrentSeqNum < WaitForSeqNum of\n    true ->\n      %% wait more\n      true;\n    false ->\n      PState = get_pstate(Key, State),\n      ?TRACE(\"~p prop: wait done seqnum ~p \", [Key, ?get(seq, PState)]),\n\n      AcceptVal = ?get(accept_val, PState),\n      case {is_subset(AcceptVal, Val), CurrentSeqNum =:= WaitForSeqNum} of\n        {true, true} ->\n          send_prop_reply(Client, SrcKey, {accept, null, Iteration, WaitForSeqNum}),\n          NP1 = ?set(accept_val, Val, PState),\n          save_pstate(Key, NP1, State);\n        {false, true} ->\n          send_prop_reply(Client, SrcKey, {reject, AcceptVal, Iteration, WaitForSeqNum});\n        {_, false} ->\n          %% Should never happen\n          ?TRACE(\"SKIPPED PROP WAIT WITH SEQ NUMBER ~p!\", [WaitForSeqNum]),\n          ok\n      end,\n      false\n  end.\n\naccept_loop_end(Key, LearnedVal, PState, State) ->\n  SeqNum = ?get(seq, PState),\n  LearnedDict = ?get(learned_val, PState),\n  AcceptedVal = ?get(accept_val, PState),\n  NewLD = dict:store(SeqNum, LearnedVal, LearnedDict),\n\n  %% truncate acceted val ... line delta2 in paper\n  NewAV =\n    case dict:find(SeqNum - 1, NewLD) of\n      {ok, Val} -> subtract(AcceptedVal, Val);\n      error -> AcceptedVal\n    end,\n\n  NP1 = ?set(learned_val, NewLD, PState),\n  NP2 = ?set(accept_val, NewAV, NP1),\n  NP3 = ?set(seq, SeqNum + 1, NP2),\n  NP4 = ?set(active, false, NP3),\n\n\n  %% apply all learned updates commands to the CRDT\n  CrdtAppliedSet = ?get(cmd_set, NP4),\n  DataType = ?get(crdt_type, NP4),\n  Crdt = ?get(crdt, NP4),\n\n  % No CRDT values are ever merged, only command sets.\n  %% Thus every command is applied at every replica. If using\n  %% the correct replica id (or own) here, this would result in possibe\n  %% differenct internal CRDT states for every replica\n  %% (e.g. increment the counter would only increment each respictive slot)\n  %% TODO: Is a fixed ID a problem? If yes, we musst attach the ID at the time\n  %% the command is recieved from the first replica before distribution.\n  ReplicaId = 1,\n\n  NewLearned = subtract(LearnedVal, CrdtAppliedSet),\n  LearnedAsList = sets:to_list(NewLearned),\n  NCrdt = lists:foldl(\n                fun(Cmd, Acc) ->\n                  apply_updates(DataType, Cmd, ReplicaId, Acc)\n                end, Crdt, LearnedAsList),\n  NP5 = ?set(crdt, NCrdt, NP4),\n  NP6 = ?set(cmd_set, union(NewLearned, CrdtAppliedSet), NP5),\n\n  %% notify all clients of learned commands and apply queries\n  NP7 = notify_clients(Key, DataType, NCrdt, sets:to_list(LearnedVal), NP6),\n\n  comm:send_local(self(), {zheng_acceptor, agree, Key, DataType}),\n  save_pstate(Key, NP7, State),\n  ?TRACE(\"~p RESPOND DONE seq number ~p~n CRDT VALUE ~p\", [Key, ?get(seq, NP7) - 1, NCrdt]),\n\n  %% must be executed immediately, so that sequence number is not missed\n  %% at prop_wait, but we cannot use gen_component:post_op(?)\n  on({zheng_acceptor, prop_wait, Key}, State),\n  get_pstate(Key, State). %% retrieve state again as changes by prop_wait\n\n\n%%%%%%%%%%%%%%%%%%%%%\n%% Application of learned functions on the CRDT\n%%%%%%%%%%%%%%%%%%%%%\napply_updates(DataType, Cmd, ReplicaId, Crdt) ->\n  case is_noop(Cmd) orelse is_query(Cmd) of\n    true -> Crdt;\n    false -> apply_updates_helper(DataType, get_cmd_fun(Cmd), ReplicaId, Crdt)\n  end.\n\napply_updates_helper(DataType, Funs, ReplicaId, Crdt) when is_list(Funs) ->\n  lists:foldl(\n    fun(UpdateFun, Acc) ->\n      apply_updates_helper(DataType, UpdateFun, ReplicaId, Acc)\n    end, Crdt, lists:flatten(Funs));\napply_updates_helper(DataType, Fun, ReplicaId, Crdt) ->\n  DataType:apply_update(Fun, ReplicaId, Crdt).\n\n-spec apply_queries(crdt:crdt_module(), [crdt:query_fun()] | crdt:query_fun(),\n    crdt:crdt()) -> any() | [any()].\napply_queries(DataType, QueryFun, Crdt) when is_function(QueryFun) ->\n    DataType:apply_query(QueryFun, Crdt);\napply_queries(DataType, QueryFuns, Crdt) ->\n   [apply_queries(DataType, Fun, Crdt) || Fun <- QueryFuns].\n\n%%%%%%%%%%%%%%%%%%%%%\n%% Helpers for sending messages\n%%%%%%%%%%%%%%%%%%%%%\nsend_to_all_replicas(Key, Message, ConsLookupField, KeyFieldReplace) ->\n  Dest = pid_groups:find_a(routing_table),\n  [\n    begin\n      %% let fill in whether lookup was consistent\n      Msg = setelement(KeyFieldReplace, Message, K),\n      LookupEnvelope = dht_node_lookup:envelope(ConsLookupField, Msg),\n      comm:send_local(Dest, {?lookup_aux, K, 0, LookupEnvelope})\n    end\n  || K <- replication:get_keys(Key) ].\n\nsend_prop_reply(Client, Key, Msg) ->\n  comm:send(Client, {zheng_acceptor, agree_loop_collect, Key, Msg}).\n\nget_replica_id(Key) ->\n  Keys = lists:sort(replication:get_keys(Key)),\n  Tmp = lists:dropwhile(fun(E) -> E =/= Key end, Keys),\n  length(Keys) - length(Tmp) + 1.\n\nnotify_clients(_Key, DataType, Crdt, LearnedCmds, PState) ->\n  ListenDict = ?get(wait_for_learned, PState),\n  ?TRACE(\"~p Notify_clients ~n~p ~n~p\", [_Key, LearnedCmds, ListenDict]),\n  NewListenDict =\n    lists:foldl(\n      fun(Cmd, DictAcc) ->\n        case dict:take(get_cmd_id(Cmd), DictAcc) of\n          error ->\n            DictAcc;\n          {ToNotify, NewDict} ->\n            notify_clients_helper(_Key, DataType, ToNotify, Crdt),\n            NewDict\n        end\n      end, ListenDict, LearnedCmds),\n\n  ?set(wait_for_learned, NewListenDict, PState).\n\nnotify_clients_helper(_Key, DataType, ToNotifyList, Crdt) ->\n  [begin\n    ?TRACE(\"~p Responding to request ~p\", [_Key, _R]),\n    case Fun of\n      {qfun, F} ->\n        Result = apply_queries(DataType, F, Crdt),\n        comm:send(Client, {read_result, ReqId, Result});\n      _ ->\n        comm:send(Client, {write_result, ReqId, ok})\n    end\n  end || _R = {{ReqId, Client}, Fun} <- ToNotifyList],\n  ok.\n\n%%%%%%%%%%%%%%%%%%%%%\n%% PState management\n%%%%%%%%%%%%%%%%%%%%%\n%% Initialize the CRDT and CRDT type if it is not already.\n%% This saves of from including the CRDT module in the majority of messages.\n-spec init_pstate_if_not_already(#pstate{}, crdt:crdt_module()) -> #pstate{}.\ninit_pstate_if_not_already(PState, DataType) ->\n  case ?get(crdt, PState) =:= null of\n    true ->\n      T = ?set(crdt, DataType:new(), PState),\n      ?set(crdt_type, DataType, T);\n    false ->\n      PState\n  end.\n\n-spec get_pstate(any(), state()) -> #pstate{}.\nget_pstate(Key, State) ->\n  case ?PDB:get(tablename(State), Key) of\n    {} ->\n      #pstate{};\n    {Key, PState} ->\n      PState\n  end.\n\n-spec save_pstate(any(), #pstate{}, state()) -> ok.\nsave_pstate(Key, PState, State) ->\n  ?PDB:set(tablename(State), {Key, PState}).\n\n-spec get_new_id(any(), #pstate{}) -> {cmd_id(), #pstate{}}.\nget_new_id(Key, PState) ->\n  Cid = ?get(cid, PState) + 1,\n  NP1 = ?set(cid, Cid, PState),\n  Rid = get_replica_id(Key),\n  {{Rid, Cid}, NP1}.\n\n-spec new_replies() -> #replies{}.\nnew_replies() -> #replies{}.\n\n%% generic record access\n-spec union_buffer(non_neg_integer(), buffer(), tuple()) -> tuple().\nunion_buffer(BufIdx, BufferToMerge, PState) ->\n  NewBuf = union(BufferToMerge, element(BufIdx, PState)),\n  setelement(BufIdx, PState, NewBuf).\n\n-spec add_to_list(non_neg_integer(), any(), tuple()) -> tuple().\nadd_to_list(BufIdx, NewListEle, PState) ->\n  NewBuf = [NewListEle | element(BufIdx, PState)],\n  setelement(BufIdx, PState, NewBuf).\n\n-spec add_to_buffer(non_neg_integer(), cmd(), tuple()) -> tuple().\nadd_to_buffer(BufIdx, Cmd, PState) ->\n  NewBuf = add(Cmd, element(BufIdx, PState)),\n  setelement(BufIdx, PState, NewBuf).\n\n-spec get_field(non_neg_integer(), tuple()) -> any().\nget_field(FieldIdx, PState) ->\n  element(FieldIdx, PState).\n\n-spec inc_field(non_neg_integer(), tuple()) -> tuple().\ninc_field(FieldIdx, PState) ->\n  NewVal = element(FieldIdx, PState) + 1,\n  setelement(FieldIdx, PState, NewVal).\n\n-spec set_field(non_neg_integer(), tuple(), any()) -> tuple().\nset_field(FieldIdx, PState, NewValue) ->\n  setelement(FieldIdx, PState, NewValue).\n\n%% more specialized acess\nadd_to_notify(Client, Cmd, PState) ->\n  NotifyCmd = to_notify_cmd(Client, Cmd),\n  ToNotifyList = ?get(to_notify_pending, PState),\n  ?set(to_notify_pending, [NotifyCmd | ToNotifyList], PState).\n\nset_to_notify_after_cmd_learned(NotifyList, WaitForCmd, PState) ->\n  ListenDict = ?get(wait_for_learned, PState),\n  CmdId = get_cmd_id(WaitForCmd),\n  ?set(wait_for_learned, dict:store(CmdId, NotifyList, ListenDict), PState).\n\n%%%%%%%%%%%%%%%%%%%%%\n%% Command and Command set managment\n%%%%%%%%%%%%%%%%%%%%%\n-spec new_client(any(), any()) -> client().\nnew_client(ReqId, Client) -> {ReqId, Client}.\n\n-spec new_update_cmd(cmd_id(), crdt:update_fun()) -> cmd().\nnew_update_cmd(Id, Fun) -> {Id, Fun}.\n\n-spec new_query_cmd(cmd_id(), crdt:query_fun()) -> cmd().\nnew_query_cmd(Id, Fun) -> {Id, {qfun, Fun}}.\n\n-spec new_noop(cmd_id()) -> cmd().\nnew_noop(Id) -> {Id, noop}.\n\n%% Notification commands are used only for client notification\n%% We do not need to store the update function as the resutl is always ok.\n%% Query functions are applied upon learning. We do not need old ID because\n%% they are never merged with other buffers.\n-spec to_notify_cmd(client(), cmd()) -> notify_cmd().\nto_notify_cmd(Client, Cmd = {_ID, Fun}) ->\n  case is_query(Cmd) of\n    true ->\n      {Client, Fun};\n    false ->\n      {Client, notify_ok}\n  end.\n\n-spec get_cmd_id(cmd()) -> cmd_id().\nget_cmd_id({Id, _}) -> Id.\n\n-spec get_cmd_fun(cmd()) -> any().\nget_cmd_fun({_Id, Fun}) -> Fun.\n\n-spec is_noop(cmd()) -> boolean().\nis_noop({_Id, noop}) -> true;\nis_noop(_) -> false.\n\n-spec is_query(cmd()) -> boolean().\nis_query({_Id, {qfun, _Fun}}) -> true;\nis_query(_) -> false.\n\n-spec is_update(cmd()) -> boolean().\nis_update(Cmd) -> not is_query(Cmd).\n\n-spec add(cmd(), buffer()) -> buffer().\nadd(Element, Buff) -> sets:add_element(Element, Buff).\n\n-spec subtract(buffer(), buffer()) -> buffer().\nsubtract(A, B) -> sets:subtract(A, B).\n\n-spec is_subset(buffer(), buffer()) -> boolean().\nis_subset(A, B) -> sets:is_subset(A, B).\n\n-spec is_empty(buffer()) -> boolean().\nis_empty(A) -> sets:is_empty(A).\n\n-spec union([buffer()]) -> buffer().\nunion(ListOfBuffers) -> sets:union(ListOfBuffers).\n\n-spec union(buffer(), buffer()) -> buffer().\nunion(A, B) -> sets:union(A, B).\n\n-spec new_buffer() -> buffer().\nnew_buffer() -> sets:new().\n\n-spec remove_query_commans(buffer()) -> buffer().\nremove_query_commans(Buffer) -> sets:filter(fun is_update/1, Buffer).\n\n%%%%%%%%%%%%%%%%%%%%%\n%% Access of module State\n%%%%%%%%%%%%%%%%%%%%%\n\n-spec tablename(state()) -> ?PDB:db().\ntablename(State) -> element(1, State).\n\n-spec get_load(state()) -> non_neg_integer().\nget_load(State) -> ?PDB:get_load(State).\n\n-spec tab2list(state()) -> [{any(), #pstate{}}].\ntab2list(State) ->\n    %% without prbr own data\n    tab2list_raw(State).\n\n-spec tab2list_raw_unittest(state()) -> [{any(), #pstate{}}].\ntab2list_raw_unittest(State) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    tab2list_raw(State).\n\n-spec tab2list_raw(state()) -> [{any(), #pstate{}}].\ntab2list_raw(State) ->\n    %% with prbr own data\n    ?PDB:tab2list(State).\n\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() -> true.\n\n%%%%%%%%%%%%%%%%%%%%%\n%% Application of learned functions on the CRDT\n%%%%%%%%%%%%%%%%%%%%%\nprint_pstate(Key, PState) ->\n  print_pstate(\"\", Key, PState).\nprint_pstate(_Msg, _Key, _PState) ->\n  ?TRACE(\n    \"Msg: ~p~n\"\n    \"Key:~p~n~n\" ++\n    \"seq:~p~n\" ++\n    \"max_seq:~p~n\" ++\n    \"buff_val:~p~n\" ++\n    \"learned_val:~p~n\" ++\n    \"accept_val:~p~n\" ++\n    \"active:~p~n\" ++\n    \"cmd_set:~p~n\" ++\n    \"crdt_type:~p~n\" ++\n    \"crdt:~p~n\" ++\n    \"replies:~p~n\"\n    ,\n    [_Msg, _Key,\n    ?get(seq, _PState),\n    ?get(max_seq, _PState),\n    ?get(buff_val, _PState),\n    ?get(learned_val, _PState),\n    ?get(accept_val, _PState),\n    ?get(active, _PState),\n\n    ?get(cmd_set, _PState),\n    ?get(crdt_type, _PState),\n    ?get(crdt, _PState),\n    ?get(replies, _PState)\n    ]).\n\n"
  },
  {
    "path": "src/crdt/zheng_proposer.erl",
    "content": "% @copyright 2012-2018 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    Wrapper of Zheng et al. wrapper implementaiton\n%% @end\n-module(zheng_proposer).\n-author('skrzypczak.de').\n\n-define(PDB, pdb).\n%-define(TRACE(X,Y), ct:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n\n-define(CACHED_ROUTING, (config:read(cache_dht_nodes))).\n\n-include(\"scalaris.hrl\").\n\n-behaviour(gen_component).\n\n-export_type([state/0]).\n\n-export([write/5]).\n-export([read/5]).\n\n-export([check_config/0]).\n-export([start_link/3]).\n-export([init/1, on/2]).\n\n-type state() :: { ?PDB:tableid(),\n                   dht_node_state:db_selector(),\n                   non_neg_integer() %% reqid\n                 }.\n\n-include(\"gen_component.hrl\").\n\n\n-spec start_link(pid_groups:groupname(), pid_groups:pidname(), dht_node_state:db_selector())\n                -> {ok, pid()}.\nstart_link(DHTNodeGroup, Name, DBSelector) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, DBSelector,\n                             [{pid_groups_join_as, DHTNodeGroup, Name}]).\n\n-spec init(dht_node_state:db_selector()) -> state().\ninit(DBSelector) ->\n    {?PDB:new(?MODULE, [set]), DBSelector, 0, false, false}.\n\n\n%%%% API\n\n-spec read(pid_groups:pidname(), comm:erl_local_pid(), ?RT:key(), crdt:crdt_module(),\n    [crdt:query_fun()] | crdt:query_fun()) -> ok.\nread(CSeqPidName, Client, Key, DataType, QueryFuns) ->\n    start_request(CSeqPidName, {req_start, {read, Client, Key, DataType, QueryFuns}}).\n\n-spec write(pid_groups:pidname(), comm:erl_local_pid(), ?RT:key(), crdt:crdt_module(),\n    [crdt:update_fun()] | crdt:update_fun()) -> ok.\nwrite(CSeqPidName, Client, Key, DataType, UpdateFun) when is_function(UpdateFun) ->\n    write(CSeqPidName, Client, Key, DataType, [UpdateFun]);\n\nwrite(CSeqPidName, Client, Key, DataType, UpdateFuns) when is_list(UpdateFuns) ->\n    start_request(CSeqPidName, {req_start, {write, Client, Key, DataType, UpdateFuns}}).\n\n-spec start_request(pid_groups:pidname(), comm:message()) -> ok.\nstart_request(CSeqPidName, Msg) ->\n    Pid = pid_groups:find_a(CSeqPidName),\n    trace_mpath:log_info(self(), {start_request, request, Msg}),\n    comm:send_local(Pid, Msg).\n\n\n-spec on(comm:message(), state()) -> state().\n\n%%%%%%%% Query protocol (as query is a keyword, use read/write terminology)\non({req_start, {read, Client, Key, DataType, QueryFuns}}, State) ->\n    {ReqId, NewState} = get_next_reqid(State),\n    create_open_req_entry(ReqId, Client, tablename(State)),\n\n    This = comm:this(),\n    Msg = {zheng_acceptor, new_read, '_', This, key, ReqId, DataType, QueryFuns},\n    send_to_local_replica(Key, Msg),\n    NewState;\n\non({read_result, ReqId, Result}, State) ->\n    case get_open_req_entry(ReqId, tablename(State)) of\n        undefined -> ok; % already answered\n        {ReqId, Client} ->\n            delete_open_req_entry(ReqId, tablename(State)),\n            inform_client(read_done, Client, Result)\n    end,\n    State;\n   \n\n%%%%% UPDATE protocol (as query is a keyword, use read/write terminology)\n\non({req_start, {write, Client, Key, DataType, UpdateFuns}}, State) ->\n    {ReqId, NewState} = get_next_reqid(State),\n    create_open_req_entry(ReqId, Client, tablename(State)),\n\n    This = comm:this(),\n    Msg = {zheng_acceptor, new_write, '_', This, key, ReqId, DataType, UpdateFuns},\n    send_to_local_replica(Key, Msg),\n    NewState;\n\non({write_result, ReqId, ok}, State) ->\n    case get_open_req_entry(ReqId, tablename(State)) of\n        undefined -> ok; % already answered\n        {ReqId, Client} ->\n            delete_open_req_entry(ReqId, tablename(State)),\n            inform_client(write_done, Client)\n    end,\n    State;\n\non({local_range_req, Key, Message, {get_state_response, LocalRange}}, State) ->\n    Keys = replication:get_keys(Key),\n\n    LocalKeys = lists:filter(fun(K) -> intervals:in(K, LocalRange) end, Keys),\n\n    K = case LocalKeys of\n        [] ->\n            ?TRACE(\"cannot send locally ~p ~p \", [Keys, LocalRange]),\n            %% the local dht node is not responsible for any replca... route to\n            %% random replica\n            Idx = randoms:rand_uniform(1, length(Keys)+1),\n            lists:nth(Idx, Keys);\n        [H|_] ->\n            %% use replica managed by local dht node\n            H\n        end,\n\n    Dest = pid_groups:find_a(routing_table),\n    LookupEnvelope = dht_node_lookup:envelope(3, setelement(5, Message, K)),\n    comm:send_local(Dest, {?lookup_aux, K, 0, LookupEnvelope}),\n    State.\n\n%%%%%% state management\ntablename(State) -> element(1, State).\n\nget_next_reqid(State) ->\n    Next = element(3, State) + 1,\n    {Next, setelement(3, State, Next)}.\n\ncreate_open_req_entry(ReqId, Client, TableName) ->\n   ?PDB:set({ReqId, Client}, TableName).\n\nget_open_req_entry(ReqId, TableName) ->\n    ?PDB:get(ReqId, TableName).\n\ndelete_open_req_entry(ReqId, TableName) ->\n    ?PDB:delete(ReqId, TableName).\n\n\n%%%%%% internal helper\n\n\n-spec send_to_local_replica(?RT:key(), tuple()) -> ok.\nsend_to_local_replica(Key, Message) ->\n    %% assert element(3, message) =:= '_'\n    %% assert element(5, message) =:= key\n    send_to_local_replica(Key, Message, ?CACHED_ROUTING).\n\n-spec send_to_local_replica(?RT:key(), tuple(), boolean()) -> ok.\nsend_to_local_replica(Key, Message, _CachedRouting=true) ->\n    dht_node_cache:cached_send_to_local_replica(Key, _KeyPos=5,\n                                                Message, _LookupEnvPos=3);\nsend_to_local_replica(Key, Message, _CachedRouting=false) ->\n    LocalDhtNode = pid_groups:get_my(dht_node),\n    This = comm:reply_as(comm:this(), 4, {local_range_req, Key, Message, '_'}),\n    comm:send_local(LocalDhtNode, {get_state, This, my_range}).\n\n\n-spec inform_client(write_done, any()) -> ok.\ninform_client(write_done, Client) ->\n    case is_tuple(Client) of\n        true ->\n            % must unpack envelope\n            comm:send(Client, {write_done});\n        false ->\n            comm:send_local(Client, {write_done})\n    end.\n\n-spec inform_client(read_done, any(), [any()]) -> ok.\ninform_client(read_done, Client, QueryResults) ->\n    case is_tuple(Client) of\n        true ->\n            % must unpack envelope\n            comm:send(Client, {read_done, QueryResults});\n        false ->\n            comm:send_local(Client, {read_done, QueryResults})\n    end.\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_bool(cache_dht_nodes).\n\n"
  },
  {
    "path": "src/cyclon_cache.erl",
    "content": "%  @copyright 2008-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Christian Hennig <hennig@zib.de>\n%% @doc Cyclon node cache implementation using a list.\n%% @end\n%% @version $Id$\n-module(cyclon_cache).\n-author('hennig@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n%% API\n\n-export([new/0, new/2, size/1,\n         add_node/3, remove_node/2, trim/2,\n         get_random_subset/2, get_random_nodes/2,\n         get_nodes/1, get_ages/1,\n         inc_age/1, merge/5,\n         pop_random_node/1, pop_oldest_node/1,\n         debug_format_by_age/1]).\n\n-export_type([age/0, cache/0]).\n\n-type age() :: non_neg_integer().\n-type element() :: {node:node_type(), age()}.\n-type cache() :: [ element() ].\n\n%% @doc Creates a new and empty node cache.\n-spec new() -> cache().\nnew() ->\n    [].\n\n%% @doc Creates a new node cache with the given two nodes and ages 0.\n-spec new(node:node_type(), node:node_type()) -> cache().\nnew(Node1, Node2) ->\n    case node:same_process(Node1, Node2) of\n        true  -> [{node:newer(Node1, Node2), 0}];\n        false -> [{Node2, 0}, {Node1, 0}]\n    end.\n\n%% @doc Counts the number of Cache entries.\n-spec size(cache()) -> non_neg_integer().\nsize(Cache) ->\n    length(Cache).\n\n%% @doc Returns a random node from the (non-empty!) cache.\n-spec get_random_node(Cache::[element(),...]) -> node:node_type().\nget_random_node(Cache) ->\n    {Node, _Age} = util:randomelem(Cache),\n    Node.\n\n%% @doc Removes a random element from the (non-empty!) cache and returns the\n%%      resulting cache and the removed node.\n-spec pop_random_node([Cache::element(),...]) -> {NewCache::cache(), PoppedNode::node:node_type()}.\npop_random_node(Cache) ->\n    pop_random_node(Cache, cyclon_cache:size(Cache)).\n\n%% @doc Removes a random element from the (non-empty!) cache and returns the\n%%      resulting cache and the removed node.\n-spec pop_random_node(Cache::[element(),...], CacheSize::non_neg_integer()) -> {NewCache::cache(), PoppedNode::node:node_type()}.\npop_random_node(Cache, CacheSize) ->\n    {NewCache, {Node, _Age}} = util:pop_randomelem(Cache, CacheSize),\n    {NewCache, Node}.\n\n%% @doc Returns a random subset of N elements from the cache.\n-spec get_random_subset(N::non_neg_integer(), Cache::cache()) -> RandomSubset::cache().\nget_random_subset(N, Cache) ->\n    util:random_subset(N, Cache).\n\n%% @doc Returns a random subset of N nodes from the cache.\n-spec get_random_nodes(N::non_neg_integer(), Cache::cache()) -> Nodes::[node:node_type()].\nget_random_nodes(N, Cache) ->\n    [Node || {Node, _Age} <- util:random_subset(N, Cache)].\n\n%% @doc Finds the oldest element (randomized if multiple oldest elements) and\n%%      removes it from the cache returning the new cache and this node.\n-spec pop_oldest_node(Cache::cache()) -> {NewCache::cache(), PoppedNode::node:node_type()}.\npop_oldest_node(Cache) ->\n    {OldElements, _MaxAge} =\n        lists:foldl(\n          fun ({Node, Age}, {PrevOldElems, MaxAge}) ->\n                   if Age > MaxAge ->\n                          {[{Node, Age}], Age};\n                      Age =:= MaxAge ->\n                          {[{Node, Age} | PrevOldElems] , Age};\n                      Age < MaxAge ->\n                          {PrevOldElems, MaxAge}\n                   end\n          end,\n          {[], 0},\n          Cache),\n    NodeP = get_random_node(OldElements),\n    NewCache = remove_node(NodeP, Cache),\n    {NewCache, NodeP}.\n\n%% @doc Increases the age of every element in the cache by 1.\n-spec inc_age(Cache::cache()) -> NewCache::cache().\ninc_age(Cache) ->\n    [{Node, Age + 1} || {Node, Age} <- Cache].\n\n%% @doc Checks whether the cache contains an element with the given Node.\n-spec contains_node(Node::node:node_type(), Cache::cache()) -> Result::boolean().\ncontains_node(Node, Cache) ->\n    lists:any(fun({SomeNode, _Age}) -> node:same_process(SomeNode, Node) end, Cache).\n\n%% @doc Returns the ages of all nodes in the cache.\n-spec get_ages(Cache::cache()) -> Ages::[age()].\nget_ages(Cache) ->\n    [Age || {_Node, Age} <- Cache].\n\n%% @doc Returns all nodes in the cache (without their ages).\n-spec get_nodes(Cache::cache()) -> Nodes::[node:node_type()].\nget_nodes(Cache) ->\n    [Node || {Node, _Age} <- Cache].\n\n%% @doc Merges MyCache at node MyNode with the ReceivedCache from another node\n%%      to whom SendCache has been send. The final cache size will not extend\n%%      TargetSize.\n%%      This will discard received entries pointing at MyNode and entries\n%%      already contained in MyCache, fill up empty slots in the cache with\n%%      received entries and further replace elements in MyCache using\n%%      replace/5.\n-spec merge(MyCache::cache(), MyNode::node:node_type(), ReceivedCache::cache(), SendCache::cache(), TargetSize::pos_integer()) -> NewCache::cache().\nmerge(MyCache, MyNode, ReceivedCache, SendCache, TargetSize) ->\n    % first sort the two lists to allow transformation into 3 lists containing\n    % the received entries without the already known nodes, a list of entries\n    % from both caches (with updated IDVersions) and a list of entries from my\n    % cache without the updated entries\n    {EntriesInReceivedCacheOnly, EntriesInBoth_Updated, EntriesInMyCacheOnly} =\n        util:split_unique(ReceivedCache, MyCache,\n                           fun({N1, _}, {N2, _}) -> node:pidX(N1) =< node:pidX(N2) end,\n                           fun({N1, _}, {N2, MyAge}) -> {node:newer(N1, N2), MyAge} end),\n    MyC1 = EntriesInMyCacheOnly ++ EntriesInBoth_Updated,\n    MyC1Size = cyclon_cache:size(MyC1),\n\n    % remove eventually existing references to the node itself\n    ReceivedCache_Filtered =\n        [Elem || {Node, _Age} = Elem <- EntriesInReceivedCacheOnly,\n                 not node:same_process(Node, MyNode)],\n    SendCache_Filtered =\n        [Elem || {Node, _Age} = Elem <- SendCache,\n                 not node:same_process(Node, MyNode)],\n    % finally fill up my cache to the full size (if necessary) and start\n    % replacing entries\n    {MyC2, ReceivedCacheRest, AddedElements} =\n        fillup(MyC1, ReceivedCache_Filtered, TargetSize - MyC1Size),\n    MyC2Size = MyC1Size + AddedElements,\n    replace(MyC2, MyC2Size, ReceivedCacheRest, SendCache_Filtered, TargetSize).\n\n%% @doc Trims the cache to size TargetSize (if necessary) by deleting random\n%%      entries as long as the cache is larger than the given TargetSize.\n-spec trim(Cache::cache(), CacheSize::non_neg_integer(), TargetSize::pos_integer()) -> NewCache::cache().\ntrim(Cache, CacheSize, TargetSize) ->\n    case CacheSize =< TargetSize of\n        true ->\n            Cache;\n        false ->\n            {NewCache, _Element} = util:pop_randomelem(Cache, CacheSize),\n            trim(NewCache, CacheSize - 1, TargetSize)\n    end.\n\n%% @doc Fills up MyCache with (up to) ToAddCount entries from ReceivedCache,\n%%      returning the new cache, the rest of the ReceivedCache and the number of\n%%      actually added elements.\n-spec fillup(MyCache::cache(), ReceivedCache::cache(), ToAddCount::non_neg_integer()) -> {MyNewCache::cache(), ReceivedCacheRest::cache(), AddedElements::non_neg_integer()}.\nfillup(MyCache, ReceivedCache, ToAddCount) ->\n    fillup(MyCache, ReceivedCache, ToAddCount, 0).\n\n%% @doc Helper to fill up MyCache with (up to) ToAddCount entries from\n%%      ReceivedCache, returning the new cache, the rest of the ReceivedCache\n%%      and the number of actually added elements.\n-spec fillup(MyCache::cache(), ReceivedCache::cache(), ToAddCount::non_neg_integer(), AddedElements::non_neg_integer()) -> {MyNewCache::cache(), ReceivedCacheRest::cache(), AddedElements::non_neg_integer()}.\nfillup(MyCache, ReceivedCache, 0 = _ToAddCount, AddedElements) ->\n    {MyCache, ReceivedCache, AddedElements};\nfillup(MyCache, [], _ToAddCount, AddedElements) ->\n    {MyCache, [], AddedElements};\nfillup(MyCache, [Elem | Rest] = _ReceivedCache, ToAddCount, AddedElements) ->\n    fillup([Elem | MyCache], Rest, ToAddCount - 1, AddedElements + 1).\n\n%% @doc Updates MyCache to include all entries of ReceivedCache by firstly\n%%      replacing entries among SendCache and thirdly by replacing random\n%%      entries.\n%%      ReceivedCache must not contain the local node and must not contain any\n%%      node that MyCache already contains!\n%%      SendCache must not contain the local node!\n-spec replace(MyCache::cache(), MyCacheSize::non_neg_integer(), ReceivedCache::cache(), SendCache::cache(), TargetSize::pos_integer()) -> MyNewCache::cache().\nreplace([] = _MyCache, MyCacheSize, ReceivedCache, _SendCache, TargetSize) ->\n    % the cache size (although otherwise not needed) should still be correct:\n    0 = MyCacheSize,\n    trim(ReceivedCache, cyclon_cache:size(ReceivedCache), TargetSize);\n\nreplace(MyCache, _MyCacheSize, [], _SendCache, _TargetSize) ->\n    MyCache;\nreplace(MyCache, MyCacheSize, ReceivedCache, [] = _SendCache, TargetSize) ->\n    % trim MyCache so it has enough space for all elements of ReceivedCache\n    % and add all received elements\n    ReceivedCacheSize = cyclon_cache:size(ReceivedCache),\n    MyC1 = trim(MyCache, MyCacheSize, TargetSize - ReceivedCacheSize),\n    MyC2 = MyC1 ++ ReceivedCache,\n    MyC2;\n\nreplace(MyCache, _MyCacheSize, ReceivedCache, SendCache, TargetSize) ->\n    % filter all nodes from SendCache out of MyCache to make room for entries\n    % from ReceivedCache\n    {MyC1, SendCache_new} =\n        lists:partition(\n          fun({Node, _Age}) -> not contains_node(Node, SendCache) end,\n          MyCache),\n    MyC1Size = cyclon_cache:size(MyC1),\n    % trim MyC1 so it has enough space for all elements of ReceivedCache\n    ReceivedCacheSize = cyclon_cache:size(ReceivedCache),\n    MyC2 = trim(MyC1, MyC1Size, TargetSize - ReceivedCacheSize),\n    MyC2Size = erlang:min(MyC1Size, TargetSize - ReceivedCacheSize),\n    % add all received elements to MyC2\n    MyC3 = MyC2 ++ ReceivedCache,\n    MyC3Size = MyC2Size + ReceivedCacheSize,\n    % finally fill up MyC3 (if necessary) with elements from SendCache_new that\n    % are not in ReceivedCache and thus not in MyC3\n    case MyC3Size < TargetSize of\n        true ->\n            SendC3 = [Elem || {Node, _Age} = Elem <- SendCache_new,\n                              not contains_node(Node, ReceivedCache)],\n            {MyC4, _SendC3Rest, _AddedElements} =\n                fillup(MyC3, SendC3, TargetSize - MyC3Size),\n            MyC4;\n        false ->\n            MyC3\n    end.\n\n%% @doc Adds the given node to the cache or updates its age in the Cache if\n%%      present.\n%%      Beware: the node will be added to the cache no matter what size it\n%%      already has!\n-spec add_node(Node::node:node_type(), Age::age(), Cache::cache()) -> NewCache::cache().\nadd_node(Node, Age, Cache) ->\n    [{Node, Age} | remove_node(Node, Cache)].\n\n%% @doc Removes any element with the given Node from the Cache.\n-spec remove_node(Node::node:node_type(), Cache::cache()) -> NewCache::cache().\nremove_node(Node, Cache) ->\n    [Element || {SomeNode, _Age} = Element <- Cache, not node:same_process(SomeNode, Node)].\n\n%% @doc Trims the cache to size TargetSize (if necessary) by deleting random\n%%      entries as long as the cache is larger than the given TargetSize.\n-spec trim(Cache::cache(), TargetSize::pos_integer()) -> NewCache::cache().\ntrim(Cache, TargetSize) ->\n    trim(Cache, cyclon_cache:size(Cache), TargetSize).\n\n%% @doc Returns a list of keys (ages) and string values (nodes) for debug output\n%%      used in the web interface.\n-spec debug_format_by_age(Cache::cache()) -> KeyValueList::[{Age::string(), Node::string()}].\ndebug_format_by_age(Cache) ->\n    [{integer_to_list(Age), webhelpers:safe_html_string(\"~p\", [Node])} || {Node, Age} <- Cache].\n"
  },
  {
    "path": "src/db_backend_beh.erl",
    "content": "% @copyright 2013 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc    Behaviour for DB back-ends.\n%% @end\n%% @version $Id$\n-module(db_backend_beh).\n-author('fajerski@zib.de').\n-vsn('$Id$').\n\n-type db() :: any().\n-type key() :: term(). %% '$end_of_table' is not allowed as key() or else iterations won't work with ets!\n-type entry() :: {key(), term()}\n               | {key(), term(), term()}\n               | {key(), term(), term(), term()}\n               | {key(), term(), term(), term(), term()}\n               | {key(), term(), term(), term(), term(), term()}\n               | {key(), term(), term(), term(), term(), term(), term()}\n               | {key(), term(), term(), term(), term(), term(), term(), term()}\n               | {key(), term(), term(), term(), term(), term(), term(), term(), term()}\n               | {key(), term(), term(), term(), term(), term(), term(), term(), term(), term()}\n               | {key(), term(), term(), term(), term(), term(), term(), term(), term(), term(), term()}.\n-type left_bracket() :: '(' | '['.\n-type right_bracket() :: ')' | ']'.\n-type interval() :: {key()} | all | {left_bracket(), key(), key(), right_bracket()}.\n\n-export_type([db/0, key/0, entry/0, interval/0]).\n\n% for tester:\n-export([tester_is_valid_db_key/1, tester_create_db_key/1]).\n\n-ifdef(have_callback_support).\n-include(\"scalaris.hrl\").\n\n-callback new(DBName::nonempty_string()) -> db().\n-callback open(DBName::nonempty_string()) -> db().\n-callback close(db()) -> true.\n-callback close_and_delete(db()) -> true.\n-callback put(db(), entry()) -> db().\n-callback get(db(), key()) -> entry() | {}.\n-callback delete(db(), key()) -> db().\n\n-callback get_persisted_tables() -> [nonempty_string()].\n-callback get_name(db()) -> nonempty_string().\n-callback get_load(db()) -> non_neg_integer().\n-callback is_available() -> boolean() | [MissingModule::atom()].\n-callback supports_feature(Feature::atom()) -> boolean().\n\n-callback foldl(db(), fun((key(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\n-callback foldl(db(), fun((key(), AccIn::A) -> AccOut::A), Acc0::A, interval()) -> Acc1::A.\n-callback foldl(db(), fun((key(), AccIn::A) -> AccOut::A), Acc0::A, interval(), non_neg_integer()) -> Acc1::A.\n\n-callback foldr(db(), fun((key(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\n-callback foldr(db(), fun((key(), AccIn::A) -> AccOut::A), Acc0::A, interval()) -> Acc1::A.\n-callback foldr(db(), fun((key(), AccIn::A) -> AccOut::A), Acc0::A, interval(), non_neg_integer()) -> Acc1::A.\n\n-callback foldl_unordered(DB::db(), Fun::fun((Entry::entry(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\n\n-callback tab2list(Table_name::db()) -> [Entries::entry()].\n-else.\n\n-export([behaviour_info/1]).\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n    [\n        {new, 1}, {open, 1}, {close, 1}, {close_and_delete, 1}, {put, 2}, {get, 2}, {delete, 2},\n        {get_persisted_tables, 0}, {get_name, 1}, {get_load, 1}, \n        {supports_feature, 1}, {is_available, 0},\n        {foldl, 3}, {foldl, 4}, {foldl, 5},\n        {foldr, 3}, {foldr, 4}, {foldr, 5},\n        {foldl_unordered, 3},\n        {tab2list, 1}\n    ];\nbehaviour_info(_Other) ->\n    undefined.\n\n-endif.\n\n-spec tester_is_valid_db_key(term()) -> boolean().\ntester_is_valid_db_key('$end_of_table') -> false;\ntester_is_valid_db_key(_) -> true.\n\n-spec tester_create_db_key(term()) -> key().\ntester_create_db_key('$end_of_table') -> end_of_table;\ntester_create_db_key(K) -> K.\n"
  },
  {
    "path": "src/db_bitcask.erl",
    "content": "% @copyright 2016 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    DB back-end using bitcask.\n%%         `./configure --enable-bitcask=/home/scalaris/apps/bitcask/'\n%%         export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/Users/scalaris/apps/bitcask/priv/\n%%         resp.\n%%         export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/scalaris/apps/bitcask/priv/\n%% @end\n-module(db_bitcask).\n-author('schuett@zib.de').\n\n-include(\"scalaris.hrl\").\n\n-behaviour(db_backend_beh).\n\n-define(TRACE(_X, _Y), ok).\n%% -define(TRACE(X, Y), ct:pal(X, Y)).\n\n-define(IN(E), erlang:term_to_binary(E, [{minor_version, 1}])).\n-define(OUT(E), erlang:binary_to_term(E)).\n\n%% primitives\n-export([new/1, open/1, close/1, close_and_delete/1,\n         put/2, get/2, delete/2]).\n%% db info\n-export([get_persisted_tables/0, get_name/1, get_load/1,\n         is_available/0, supports_feature/1]).\n\n%% iteration\n-export([foldl/3, foldl/4, foldl/5]).\n-export([foldr/3, foldr/4, foldr/5]).\n-export([foldl_unordered/3]).\n-export([tab2list/1]).\n\n-type db() :: {DB::reference(), DBName::nonempty_string()}.\n-type key() :: db_backend_beh:key(). %% '$end_of_table' is not allowed as key() or else iterations won't work!\n-type entry() :: db_backend_beh:entry().\n\n-export_type([db/0]).\n\n%% @doc Creates new DB handle named DBName.\n-spec new(DBName::nonempty_string()) -> db().\nnew(DBName) ->\n    new_db(DBName, [read_write]).\n\n%% @doc Open a previously existing database.\n-spec open(DBName::nonempty_string()) -> db().\nopen(DBName) ->\n    new_db(DBName, [read_write]).\n\n-spec new_db(DBName::nonempty_string(),\n             BitcaskOptions::[read_write]) -> db().\nnew_db(DBName, BitcaskOptions) ->\n    FullDir1 = [config:read(db_directory), \"/\", atom_to_list(node())],\n    _ = case file:make_dir(FullDir1) of\n        ok -> ok;\n        {error, eexist} -> ok;\n        {error, Error0} -> erlang:exit({db_bitcask, 'cannot create dir', FullDir1, Error0})\n    end,\n    FullDir = [config:read(db_directory), \"/\", atom_to_list(node()), \"/\", DBName],\n    _ = case file:make_dir(FullDir) of\n        ok -> ok;\n        {error, eexist} -> ok;\n        {error, Error1} -> erlang:exit({db_bitcask, 'cannot create dir', FullDir, Error1})\n    end,\n    BitcaskDir = filename:absname(lists:flatten(FullDir)),\n    case bitcask:open(BitcaskDir, BitcaskOptions) of\n        {error, Error} ->\n            log:log(error, \"[ Node ~w:db_bitcask ] ~.0p\", [self(), Error]),\n            erlang:error({bitcask_failed, Error});\n        Handle ->\n            db_bitcask_merge_extension:init(Handle, BitcaskDir),\n            {Handle, DBName}\n    end.\n\n%% @doc Closes the DB named DBName\n-spec close(db()) -> true.\nclose({DB, _DBName}) ->\n    bitcask:close(DB),\n    erlang:erase(bitcask_efile_port), %% @todo close port?!?\n    true.\n\n%% @doc Closes and deletes the DB named DBName\n-spec close_and_delete(db()) -> true.\nclose_and_delete({_DB, DBName} = State) ->\n    close(State),\n    FullDir = lists:flatten([config:read(db_directory), \"/\", atom_to_list(node()), \"/\", DBName]),\n    cleanup(FullDir), true.\n\n%% @doc Gets a list of persisted tables.\n-spec get_persisted_tables() -> [nonempty_string()].\nget_persisted_tables() ->\n    FullDir = [config:read(db_directory), \"/\", atom_to_list(node())],\n    case file:list_dir(FullDir) of\n        {ok, Directories} ->\n            Directories; %% @todo filter for directories\n        {error, enoent} -> []\n    end.\n\n%% @doc Saves arbitrary tuple Entry in DB DBName and returns the new DB.\n%%      The key is expected to be the first element of Entry.\n-spec put(db(), Entry::entry()) -> db().\nput({DB, _DBName} = State, Entry) ->\n    _ = bitcask:put(DB, term_to_binary(element(1, Entry)), term_to_binary(Entry)),\n    State.\n\n%% @doc Returns the entry that corresponds to Key or {} if no such tuple exists.\n-spec get(DB::db(), Key::key()) -> entry() | {}.\nget({DB, _DBName}, Key) ->\n    case bitcask:get(DB, term_to_binary(Key)) of\n        not_found -> {};\n        {error, _Err} -> {};\n        {ok, BinaryValue} ->\n            binary_to_term(BinaryValue)\n    end.\n\n%% @doc Deletes the tuple saved under Key and returns the new DB.\n%%      If such a tuple does not exists nothing is changed.\n-spec delete(DB::db(), Key::key()) -> db().\ndelete({DB, _DBName} = State, Key) ->\n    bitcask:delete(DB, term_to_binary(Key)),\n    State.\n\n%% @doc Returns the name of the DB specified in @see new/1.\n-spec get_name(DB::db()) -> nonempty_string().\nget_name({_DB, DBName}) ->\n    DBName.\n\n%% @doc Checks for modules required for this DB backend. Returns true if no\n%%      modules are missing, or else a list of missing modules\n-spec is_available() -> boolean() | [atom()].\nis_available() ->\n    case code:which(bitcask) of\n        non_existing -> [bitcask];\n        _ -> true\n    end.\n\n%% @doc Returns true if the DB support a specific feature (e.g. recovery), false otherwise.\n-spec supports_feature(Feature::atom()) -> boolean().\nsupports_feature(recover) -> false;\nsupports_feature(_) -> false.\n\n%% @doc Returns the current load (i.e. number of stored tuples) of the DB.\n-spec get_load(DB::db()) -> non_neg_integer().\nget_load({DB, _DBName}) ->\n    {KeyCount, _L} = bitcask:status(DB),\n    KeyCount.\n\n%% @doc Equivalent to toke_drv:fold(Fun, Acc0, DB).\n%%      Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldl(State, Fun, Acc) ->\n    foldl_helper(State, Fun, Acc, all, -1).\n\n%% @equiv foldl(DB, Fun, Acc0, Interval, get_load(DB))\n%% @doc   Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Interval::db_backend_beh:interval()) -> Acc1::A.\nfoldl(State, Fun, Acc, Interval) ->\n    foldl_helper(State, Fun, Acc, Interval, -1).\n\n%% @doc foldl iterates over DB and applies Fun(Entry, AccIn) to every element\n%%      encountered in Interval. On the first call AccIn == Acc0. The iteration\n%%      stops as soon as MaxNum elements have been encountered.\n%%      Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Intervall::db_backend_beh:interval(), MaxNum::non_neg_integer()) -> Acc1::A.\nfoldl(State, Fun, Acc, Interval, MaxNum) ->\n    %% HINT\n    %% Fun can only be applied in a second pass. It could do a delete (or other\n    %% write op) and toke can not handle writes whiles folding.\n    %% Since we reversed the order while accumulating reverse it by using lists\n    %% fold but \"from the other side\"\n    foldl_helper(State, Fun, Acc, Interval, MaxNum).\n\n%% @private this helper enables us to use -1 as MaxNum. MaxNum == -1 signals that all\n%%          data is to be retrieved.\n-spec foldl_helper(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Intervall::db_backend_beh:interval(), MaxNum::integer()) -> Acc1::A.\nfoldl_helper({DB, _DBName}, Fun, Acc, Interval, MaxNum) ->\n    Keys = get_all_keys(DB, Interval, MaxNum),\n    lists:foldr(Fun, Acc, Keys).\n\n%% @doc makes a foldr over the whole dataset.\n%%      Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldr(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldr(State, Fun, Acc) ->\n    foldr_helper(State, Fun, Acc, all, -1).\n\n%% @equiv foldr(DB, Fun, Acc0, Interval, get_load(DB))\n%% @doc   Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldr(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Interval::db_backend_beh:interval()) -> Acc1::A.\nfoldr(State, Fun, Acc, Interval) ->\n    foldr_helper(State, Fun, Acc, Interval, -1).\n\n%% @doc foldr iterates over DB and applies Fun(Entry, AccIn) to every element\n%%      encountered in Interval. On the first call AccIn == Acc0. The iteration\n%%      stops as soon as MaxNum elements have been encountered.\n%%      Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldr(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Intervall::db_backend_beh:interval(), MaxNum::non_neg_integer()) -> Acc1::A.\nfoldr(State, Fun, Acc, Interval, MaxNum) ->\n    foldr_helper(State, Fun, Acc, Interval, MaxNum).\n\n%% @private this helper enables us to use -1 as MaxNum. MaxNum == -1 signals that all\n%%          data is to be retrieved.\n-spec foldr_helper(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Intervall::db_backend_beh:interval(), MaxNum::integer()) -> Acc1::A.\nfoldr_helper({DB, _DBName}, Fun, Acc, Interval, MaxNum) ->\n    %% first only retrieve keys so we don't have to load the whole db into memory\n    Keys = get_all_keys(DB, Interval, -1),\n    CutData = case MaxNum of\n                  N when N < 0 ->\n                      Keys;\n                  _ ->\n                      lists:sublist(Keys, MaxNum)\n              end,\n    %% see HINT in foldl/5\n    %% now retrieve actual data\n    lists:foldl(Fun, Acc, CutData).\n\n%% @doc Works similar to foldl/3 but uses toke_drv:fold instead of our own implementation.\n%% The order in which will be iterated over is unspecified, but using this fuction\n%% might be faster than foldl/3 if it does not matter.\n-spec foldl_unordered(DB::db(), Fun::fun((Entry::entry(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldl_unordered(State, Fun, Acc) ->\n        %TODO Use native fold\n        foldl(State, fun(K, AccIn) -> Fun(get(State, K), AccIn) end, Acc).\n\n%% @private get_all_keys/3 retrieves all keys in DB that fall into Interval but\n%%          not more than MaxNum. If MaxNum == -1 all Keys are retrieved. If\n%%          MaxNum is positive it starts from the left in term order.\n-spec get_all_keys(reference(), db_backend_beh:interval(), -1 | non_neg_integer()) ->\n    [key()].\nget_all_keys(DB, Interval, MaxNum) ->\n    Keys = case bitcask:list_keys(DB) of\n               {error, _Term} -> [];\n               BinaryKeys -> [ binary_to_term(BinaryKey) || BinaryKey <- BinaryKeys]\n           end,\n    {_, In} = lists:foldl(fun(_, {0, _} = AccIn) ->\n                                  AccIn;\n                             (Key, {Max, KeyAcc} = AccIn) ->\n                          case is_in(Interval, Key) of\n                              true ->\n                                  {Max - 1, [Key | KeyAcc]};\n                              _ ->\n                                  AccIn\n                          end\n                end, {MaxNum, []}, lists:sort(Keys)),\n    In.\n\n\nis_in({Key}, OtherKey) -> Key =:= OtherKey;\nis_in(all, _Key) -> true;\nis_in({'(', L, R, ')'}, Key) -> Key > L andalso Key < R;\nis_in({'(', L, R, ']'}, Key) -> Key > L andalso ((Key < R) orelse (Key =:= R));\nis_in({'[', L, R, ')'}, Key) -> ((Key > L) orelse (Key =:= L)) andalso Key < R;\nis_in({'[', L, R, ']'}, Key) -> ((Key > L) orelse (Key =:= L)) andalso\n                                          ((Key < R) orelse (Key =:= R)).\n%% @doc Returns a list of all objects in the table Table_name.\n-spec tab2list(Table_name::db()) -> [Entries::entry()].\ntab2list(_Table_name) ->\n    %% Not implemented yet.\n    [].\n\ncleanup(Path) ->\n    Re = \".*\",\n    Files = filelib:fold_files(Path, Re, true, fun(File, Acc) -> [File | Acc] end, []),\n    _ = [file:delete(File) || File <- Files],\n    ok.\n"
  },
  {
    "path": "src/db_bitcask_merge_extension.erl",
    "content": "% @copyright 2013-2016 Zuse Institute Berlin,\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc Triggers periodically a merge operation for a Bitcask DB.\n%% @end\n%% @version $Id$\n-module(db_bitcask_merge_extension).\n-author('skrzypczak@zib.de').\n\n-include(\"scalaris.hrl\").\n\n-define(TRACE(X), ?TRACE(X, [])).\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n\n-define(MERGE_INTERVAL, config:read(bitcask_merge_interval)).\n-define(MERGE_OFFSET, config:read(bitcask_merge_offset)).\n\n-export([check_config/0]).\n-export([init/1, init/2]).\n-export([on/2]).\n\n%% This component uses a trigger mechanism to become active from time\n%% to time. It is a dht_node_extension and therefore embedded in the\n%% dht_node process.\n\n-spec init(_Options::any()) -> ok.\ninit(_Options) ->\n    % ignore dht_node_extension init since this module needs a\n    % bitcask DB reference. db_bitcask.erl calls init/2 upon opening\n    % a bitcask store.\n    ok.\n\n%% @doc Sends a periodic trigger to itself\n-spec init(BitcaskHandle::reference(), Dir::string()) -> ok.\ninit(BitcaskHandle, Dir) ->\n    ?TRACE(\"bitcask_merge: init for DB directory ~p~n\", [Dir]),\n    send_trigger(merge_offset, BitcaskHandle, Dir, ?MERGE_OFFSET),\n    ok.\n\n-spec on(comm:message(), State::dht_node_state:state()) -> dht_node_state:state().\non({merge_offset, BitcaskHandle, Dir}, State) ->\n    %% calculate initial offset to prevent that all nodes are merging at the\n    %% same time.\n    mgmt_server:node_list(),\n    Nodes = receive\n                ?SCALARIS_RECV(\n                    {get_list_response, List},\n                    lists:usort([Pid || {_, _, Pid} <- List])\n                )\n            after 2000 ->\n                [self()]\n            end,\n\n    Slot = get_idx(self(), Nodes),\n    SlotLength = ?MERGE_INTERVAL / length(Nodes),\n    Offset = trunc(SlotLength * Slot),\n    ?TRACE(\"PIDs: ~p~n~nSlot choosen: ~p~n\", [Nodes, Slot]),\n    send_trigger(merge_trigger, BitcaskHandle, Dir, Offset),\n\n    State;\n\non({merge_trigger, BitcaskHandle, Dir}, State) ->\n    %% merges if necessary\n\n    _ = case erlang:get(BitcaskHandle) of\n        undefined ->\n            %% handle was closed, no more merge operations\n            %% possible.\n            ?TRACE(\"bitcask_merge: Bitcask handle closed for ~p~n\", [Dir]),\n            ok;\n        _ -> \n            %% bitcask:needs_merge/1 cleans up open file descriptors left\n            %% over from the last successful merge. Deleted files stay\n            %% open until then...\n            _ = case bitcask:needs_merge(BitcaskHandle) of\n                {true, Files} ->\n                    ?TRACE(\"bitcask_merge: starting merge in dir ~p~n\", [Dir]),\n                    bitcask_merge_worker:merge(Dir, [], Files);\n                false ->\n                    ok\n            end,\n            send_trigger(merge_trigger, BitcaskHandle, Dir, ?MERGE_INTERVAL)\n    end,\n    State.\n\n%% @doc Sends a trigger message to itself\n-spec send_trigger(Message::atom(), BitcaskHandle::reference(),\n    Dir::string(), Delay::integer()) -> ok.\nsend_trigger(Message, BitcaskHandle, Dir, Delay) ->\n    msg_delay:send_trigger(Delay,\n                           dht_node_extensions:wrap(?MODULE, {Message,\n                                                              BitcaskHandle,\n                                                              Dir})),\n    ok.\n\n%% @doc Helper to find the index of an element in a list.\n%% Returns length(List)+1 if List does not contain E.\n-spec get_idx(E::any(), List::any()) -> integer().\nget_idx(E, List) -> get_idx(E, List, 1).\n\n-spec get_idx(E::any(), List::any(), Idx::integer()) -> integer().\nget_idx(_, [], Idx) -> Idx;\nget_idx(E, [E | _], Idx) -> Idx;\nget_idx(E, [_ | T], Idx) -> get_idx(E, T, Idx+1).\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(bitcask_merge_interval) and\n    config:cfg_is_integer(bitcask_merge_offset).\n"
  },
  {
    "path": "src/db_dht.erl",
    "content": "% @copyright 2009-2015 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc    DB back-end for DHTs storing db_entry:entry() objects.\n%% @end\n%% @version $Id$\n-module(db_dht).\n-author('schintke@zib.de').\n-author('kruber@zib.de').\n-author('fajerski@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% -define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n%% -define(TRACE_SNAP(X, Y), ct:pal(X, Y)).\n%% -define(TRACE_SNAP(X, Y), ?TRACE(X, Y)).\n-define(TRACE_SNAP(X, Y), ok).\n%% -define(TRACE_CHUNK(X, Y), ct:pal(X, Y)).\n-define(TRACE_CHUNK(X, Y), ok).\n-define(DB, (config:read(db_backend))). %% DB backend\n-define(CKETS, ets). %% changed keys database\n\n%% whole DB management\n-export([new/1, close/1, close_and_delete/1]).\n-export([get_load/1, get_load/2]).\n-export([get_data/1, get_data/3]).\n\n%% raw whole db_entry operations\n-export([get_entry/2, set_entry/2, set_entry/4]).\n-export([delete_entry/2, delete_entry_at_key/2]).\n-export([update_entry/2]).\n\n%% access as used in tx and quorum reads (value-version-based)\n-export([read/2, write/4, delete/2]).\n\n%% bulk entry operations on intervals or filter funs for replica\n%% repair and local use (for data slide)\n-export([get_chunk/4, get_chunk/6]).\n\n%% slide: slide DB (see dht_node_state.erl)\n-export([get_split_key/5]).\n-export([split_data/2, add_data/2]).\n-export([get_entries/2, get_entries/3]).\n\n%% slide: delta recording (see dht_node_state.erl)\n-export([record_changes/2]).\n-export([stop_record_changes/1, stop_record_changes/2]).\n-export([get_changes/1, get_changes/2]).\n-export([delete_entries/2]).\n\n%% slide: slide snapshot data (see dht_node_state.erl)\n-export([snapshot_is_running/1]).\n-export([add_snapshot_data/2]).\n-export([get_snapshot_data/2]).\n\n%% subscriptions to DB changes (called locally by delta recording)\n%%-export([set_subscription/2, get_subscription/2, remove_subscription/2]).\n\n%% snapshots: business logic\n-export([init_snapshot/1]).\n-export([delete_snapshot/1]).\n-export([join_snapshot_data/1]).\n-export([snapshot_is_lockfree/1]).\n-export([get_live_lc/1, get_snap_lc/1]).\n\n%% debugging, diagnostic outputs\n-export([get_snapshot_data/1]).\n-export([set_snapshot_entry/2, get_snapshot_entry/2]).\n-export([delete_snapshot_entry/2, delete_snapshot_entry_at_key/2]).\n\n%% for unittests\n-export([check_db/1]).\n-export([copy_value_to_snapshot_table/2]).\n-export([update_entries/4]).\n\n\n-type db() :: {KeyValueDB  :: term(),\n               Subscribers :: db_ets:db(), %% for delta recording\n               SnaphotInfo :: {term() | false,\n                               LiveLockCount :: non_neg_integer(),\n                               SnapLockCount :: non_neg_integer()}}.\n-type version() :: client_version().\n-type value() :: rdht_tx:encoded_value().\n-type db_as_list() :: [db_entry:entry()].\n\n-type subscr_op_t() :: {write, db_entry:entry()} | {delete | split, ?RT:key()}.\n-type subscr_changes_fun_t() :: fun((DB::db(), Tag::any(), Operation::subscr_op_t()) -> db()).\n-type subscr_remove_fun_t() :: fun((Tag::any()) -> any()).\n-type subscr_t() :: {Tag::any(), intervals:interval(), ChangesFun::subscr_changes_fun_t(), CloseDBFun::subscr_remove_fun_t()}.\n\n\n-export_type([db/0, value/0, version/0, db_as_list/0]).\n\n%%%%%%\n%%% whole DB management\n%%%%%%\n\n%% @doc Initializes a new database.\n-spec new(nonempty_string() | atom()) -> db().\nnew(DBName) ->\n    DBNameNew = db_util:get_name(DBName),\n    SubscrName = db_util:get_subscriber_name(DBNameNew),\n    {?DB:new(DBNameNew), db_ets:new(SubscrName), {false, 0, 0}}.\n\n%% @doc Closes the given DB (it may be recoverable using open/1 depending on\n%%      the DB back-end).\n-spec close(db()) -> true.\nclose(State) ->\n    close(State, close).\n\n%% @doc Closes the given DB and deletes all contents (this DB can thus not be\n%%      re-opened using open/1).\n-spec close_and_delete(db()) -> true.\nclose_and_delete(State) ->\n    close(State, close_and_delete).\n\n%% @doc Helper for close/1 and close_and_delete/1.\n-spec close(db(), CloseFn::close | close_and_delete) -> true.\nclose({KVStore, Subscribers, {SnapDB, _LLC, _SLC}} = State, CloseFn) ->\n    _ = call_subscribers(State, close_db),\n    ?DB:CloseFn(KVStore),\n    db_ets:CloseFn(Subscribers),\n    case SnapDB of\n        false ->\n            ok;\n        ETSTable ->\n            ?DB:CloseFn(ETSTable)\n    end.\n\n%% @doc Returns the number of stored keys.\n-spec get_load(DB::db()) -> Load::non_neg_integer().\nget_load({DB, _Subscr, _SnapState}) ->\n    ?DB:get_load(DB).\n\n%% @doc Returns the number of stored keys in the given interval.\n-spec get_load(DB::db(), Interval::intervals:interval()) -> Load::integer().\nget_load(State, [all]) -> get_load(State);\nget_load(_State, []) -> 0;\nget_load({DB, _Subscr, _Snap}, Interval) ->\n    ?DB:foldl(\n            DB,\n            fun(Key, AccIn) ->\n                case intervals:in(Key, Interval) of\n                    true -> AccIn + 1;\n                    _ -> AccIn\n                end\n            end, 0).\n\n%% @doc Returns all (including empty, but not null) DB entries.\n-spec get_data(DB::db()) -> db_as_list().\nget_data(State) ->\n    get_data(State, fun(_) -> true end, fun(E) -> E end).\n\n-spec get_data(DB::db(), \n               FilterFun::fun((db_entry:entry()) -> boolean()),\n               ValueFun::fun((db_entry:entry()) -> V)) \n        -> [V].\nget_data({DB, _Subscr, _Snap}, FilterFun, ValueFun) ->\n    FoldlFun = fun(Entry, Acc) ->\n                       case FilterFun(Entry) of\n                            true -> [ValueFun(Entry) | Acc];\n                            _  -> Acc\n                       end\n               end,\n    ?DB:foldl_unordered(DB, FoldlFun, []).\n\n%%%%%%\n%%% raw whole db_entry operations\n%%%%%%\n\n%% @doc Gets an entry from the DB. If there is no entry with the given key,\n%%      an empty entry will be returned.\n-spec get_entry(db(), Key::?RT:key()) -> db_entry:entry().\nget_entry({KVStore, _Subscr, _Snap}, Key) ->\n    %% report read to process rrd and check for report to monitor\n    lb_stats:update_db_monitor(db_reads, Key),\n    case ?DB:get(KVStore, Key) of\n        {} ->\n            db_entry:new(Key);\n        Entry ->\n            Entry\n    end.\n\n-spec set_entry(db(), Entry::db_entry:entry()) -> db().\nset_entry(DB, Entry) ->\n    set_entry(DB, Entry, 1, 2).\n\n-spec set_entry(db(), Entry::db_entry:entry(), non_neg_integer(),\n                 non_neg_integer()) -> db().\nset_entry(State, Entry, TLogSnapNo, OwnSnapNo) ->\n    %% report write to process rrd and check for report to monitor\n    lb_stats:update_db_monitor(db_writes, db_entry:get_key(Entry)),\n    case db_entry:is_null(Entry) of\n        true ->\n            delete_entry(State, Entry);\n        _ ->\n            %% do lockcounting and copy-on-write logic\n            {KVStore, Subscr, Snap} = snaps(State, Entry, TLogSnapNo, OwnSnapNo),\n            %% set actual entry in DB\n            call_subscribers({?DB:put(KVStore, Entry), Subscr, Snap},\n                             {write, Entry})\n    end.\n\n%% @doc Removes all values with the given entry's key from the DB.\n-spec delete_entry(DB::db(), Entry::db_entry:entry()) -> NewDB::db().\ndelete_entry(State, Entry) ->\n    delete_entry_at_key(State, db_entry:get_key(Entry)).\n\n-spec delete_entry_at_key(DB::db(), ?RT:key()) -> NewDB::db().\ndelete_entry_at_key(State, Key) ->\n    delete_entry_at_key(State, Key, delete).\n\n%% @doc Removes all values with the given key from the DB with specified reason.\n-spec delete_entry_at_key(DB::db(), ?RT:key(), delete | split) ->  NewDB::db().\ndelete_entry_at_key({DB, Subscr, {Snap, LiveLC, SnapLC}} = State,  Key, Reason) ->\n    %% TODO count locks\n    OldEntry = get_entry(State, Key),\n    Delta = -db_entry:lockcount(OldEntry),\n    NewLiveLC = LiveLC + Delta,\n    NewSnapLC = case Snap of\n        false -> SnapLC;\n        _ -> SnapLC + Delta\n    end,\n    %% @doc removes all entries from the DB that correspond to Key\n    call_subscribers({?DB:delete(DB, Key), Subscr, {Snap, NewLiveLC, NewSnapLC}}, {Reason, Key}).\n\n%% @doc Updates an existing (!) entry in the DB.\n%% TODO: necessary?\n-spec update_entry(DB::db(), Entry::db_entry:entry()) -> NewDB::db().\nupdate_entry(DB, Entry) ->\n    set_entry(DB, Entry).\n\n\n%%%%%%\n%%% DB access as used in tx and quorum reads (value-version-based)\n%%%%%%\n\n%% @doc Reads the version and value of a key.\n-spec read(DB::db(), Key::?RT:key()) ->\n         {ok, Value::value(), Version::version()} | {ok, empty_val, -1}.\nread(DB, Key) ->\n    DBEntry = get_entry(DB, Key),\n    {ok, db_entry:get_value(DBEntry), db_entry:get_version(DBEntry)}.\n\n%% @doc Updates the value of the given key.\n-spec write(DB::db(), Key::?RT:key(), Value::value(), Version::version()) ->\n         NewDB::db().\nwrite(DB, Key, Value, Version) ->\n    DBEntry = get_entry(DB, Key),\n    case db_entry:is_null(DBEntry) of\n        true ->\n            NewEntry = db_entry:new(Key, Value, Version),\n            set_entry(DB, NewEntry);\n        _ ->\n            NewEntry = db_entry:set_value(DBEntry, Value, Version),\n            update_entry(DB, NewEntry)\n    end.\n\n%% @doc Deletes the key. Returns {DB, undef} if the key does not exist in the\n%%      DB, {DB, locks_set} if read or write locks are still set and {DB, ok}\n%%      if the operation was successfully performed.\n-spec delete(DB::db(), Key::?RT:key()) ->\n         {NewDB::db(), Status::ok | locks_set | undef}.\ndelete(DB, Key) ->\n    DBEntry = get_entry(DB, Key),\n    case db_entry:is_null(DBEntry) of\n        true ->\n            {DB, undef};\n        _ ->\n            case db_entry:is_locked(DBEntry) of\n                false ->\n                    {delete_entry(DB, DBEntry), ok};\n                _ ->\n                    {DB, locks_set}\n            end\n    end.\n\n%% @doc Returns all key-value pairs of the given DB which are in the given\n%%      interval but at most ChunkSize elements.\n%%      Assumes the ets-table is an ordered_set,\n%%      may return data from \"both ends\" of the DB-range if the interval is\n%%      \"\"wrapping around\", i.e. its begin is larger than its end.\n%%      Returns the chunk and the remaining interval for which the DB may still\n%%      have data (a subset of I).\n%%      Precond: Interval is a subset of the range of the dht_node and continuous!\n-spec get_chunk(DB::db(), StartId::?RT:key(), Interval::intervals:interval(), ChunkSize::pos_integer() | all)\n        -> {intervals:interval(), db_as_list()}.\nget_chunk(DB, StartId, Interval, ChunkSize) ->\n    get_chunk(DB, StartId, Interval, fun(_) -> true end, fun(E) -> E end, ChunkSize).\n\n-spec get_chunk(DB::db(), StartId::?RT:key(), Interval::intervals:interval(),\n                FilterFun::fun((db_entry:entry()) -> boolean()),\n                ValueFun::fun((db_entry:entry()) -> V),\n                ChunkSize::pos_integer() | all)\n        -> {intervals:interval(), [V]}.\nget_chunk(_State, _StartId, [], _FilterFun, _ValueFun, _ChunkSize) ->\n    {intervals:empty(), []};\nget_chunk(State, StartId, Interval, FilterFun, ValueFun, all) ->\n    {_Next, Chunk} =\n        get_chunk(State, StartId, Interval, FilterFun, ValueFun, get_load(State)),\n    {intervals:empty(), Chunk};\n\n\n%%%%%%\n%% bulk entry operations on intervals or filter funs for replica\n%% repair and local use (for data slide)\n%%%%%%\n\nget_chunk({DB, _Subscr, _Snap}, StartId, Interval, FilterFun, ValueFun, ChunkSize) ->\n    %% split intervals in a way so that the first simple interval of After\n    %% either contains StartId or is the closest following after StartId\n    ?TRACE_CHUNK(\"get_chunk:~nStartID: ~p~nInterval:~p~nChunksize: ~p~n\",\n                 [StartId, Interval, ChunkSize]),\n    %% rotate and split intervals so that StartId is the first element looked at\n    {Before, After} = lists:splitwith(\n            fun(all) -> false;\n               ({Key}) ->\n                    StartId > Key;\n               ({_LBr, _L, R, ']'}) ->\n                StartId > R;\n               ({_LBr, _L, R, ')'}) ->\n                StartId >= R\n            end, intervals:get_simple_intervals(Interval)),\n    ?TRACE_CHUNK(\"split: ~p~n~p~n\", [Before, After]),\n    RotatedInterval = case After of\n        [] -> Before;\n        [all] ->\n            [{'[', StartId, ?PLUS_INFINITY, ')'},\n             {'[', ?MINUS_INFINITY, StartId, ')'}];\n        [{_K} | _Rest] ->\n            After ++ Before;\n        [{LBr, L, R, RBr} | Tail] ->\n            case intervals:in(StartId, intervals:new(LBr, L, R, RBr)) of\n                true ->\n                    lists:append([[{'[', StartId, R, RBr}],\n                                  Tail, Before,\n                                  [{LBr, L, StartId, ')'}]]);\n                _ ->\n                    After ++ Before\n            end\n    end,\n    AddDataFun = fun(Key, {Acc, RemainingChunkSize}) ->\n                         Entry = ?DB:get(DB, Key),\n                         case FilterFun(Entry) of\n                             true -> {[Entry | Acc],\n                                      RemainingChunkSize - 1};\n                             _    -> {Acc, RemainingChunkSize}\n                         end\n                 end,\n    ?TRACE_CHUNK(\"get_chunk2: asking db:foldl to look in those intervals ~p~n\",\n                 [RotatedInterval]),\n    {Chunk, Remaining} = lists:foldl(\n        fun(I, {Acc, RemainingChunkSize}) ->\n            ?DB:foldl(DB, AddDataFun, {Acc, RemainingChunkSize}, I,\n                      RemainingChunkSize)\n        end, {[], ChunkSize}, RotatedInterval),\n    %% calculate the leftover interval and return\n    ?TRACE_CHUNK(\"StartId: ~p~nInterval: ~p~nChunkSize: ~p~nChunk: ~p~nRemaining: ~p~nOpen: ~p~n\",\n           [StartId, Interval, ChunkSize, Chunk, Remaining, calc_remaining_interval(StartId, Remaining, Chunk, Interval)]),\n    {calc_remaining_interval(StartId, Remaining, Chunk, Interval),\n     lists:foldl(\n            fun(E, AccIn) ->\n                [ValueFun(E) | AccIn] end,\n            [], Chunk)}.\n\n%%%%%%\n%%% slide: slide DB (see dht_node_state.erl)\n%%%%%%\n\n%% @doc Returns the key that would remove not more than TargetLoad entries\n%%      from the DB when starting at the key directly after Begin in case of\n%%      forward searches and directly at Begin in case of backward searches,\n%%      respectively.\n%%      Precond: a load larger than 0\n%%      Note: similar to get_chunk/2.\n-spec get_split_key(DB::db(), Begin::?RT:key(), End::?RT:key(), TargetLoad::pos_integer(), forward | backward)\n        -> {?RT:key(), TakenLoad::non_neg_integer()}.\nget_split_key({DB, _Subscr, _Snap}, Begin, End, TargetLoad, forward)\n        when Begin > End ->\n    %% when Begin and End wrap around do two folds\n    {Key, Taken1} = ?DB:foldl(DB,\n              fun(E, {_El, Taken}) -> {E, Taken + 1} end,\n              {End, 0},\n              {'(', Begin, ?PLUS_INFINITY, ')'},\n              TargetLoad),\n    Split = ?DB:foldl(DB,\n              fun(E, {_El, Taken}) -> {E, Taken + 1} end,\n              {Key, Taken1},\n              {'[', ?MINUS_INFINITY, End, ']'},\n              TargetLoad - Taken1),\n    normalize_split_key(Split, TargetLoad, End);\nget_split_key({DB, _Subscr, _Snap}, Begin, End, TargetLoad, forward) ->\n    Split = ?DB:foldl(DB,\n              fun(E, {_El, Taken}) -> {E, Taken + 1} end,\n              {End, 0},\n              {'(', Begin, End, ']'},\n              TargetLoad),\n    normalize_split_key(Split, TargetLoad, End);\n\nget_split_key({DB, _Subscr, _Snap}, Begin, End, TargetLoad, backward)\n        when Begin < End ->\n    %% when Begin and End wrap around do two folds\n    {Key, Taken1} = ?DB:foldr(DB,\n              fun(E, {_El, Taken}) -> {E, Taken + 1} end,\n              {End, 0},\n              {'[', ?MINUS_INFINITY, Begin, ']'},\n              TargetLoad + 1),\n    ?TRACE(\"first fold done~nnew target:~p~nstart: ~p~nend: ~p~nacc: ~p\",\n           [TargetLoad - Taken1, ?PLUS_INFINITY, End, {Key, Taken1}]),\n    Split = ?DB:foldr(DB,\n              fun(E, {_El, Taken}) -> {E, Taken + 1} end,\n              {Key, Taken1},\n              {'(', End, ?PLUS_INFINITY, ')'},\n              TargetLoad - Taken1 + 1),\n    normalize_split_key_b(Split, TargetLoad, End);\nget_split_key({DB, _Subscr, _Snap}, Begin, End, TargetLoad, backward) ->\n    Split = ?DB:foldr(DB,\n              fun(E, {_El, Taken}) -> {E, Taken + 1} end,\n              {End, 0},\n              {'(', End, Begin, ']'},\n              TargetLoad + 1),\n    normalize_split_key_b(Split, TargetLoad, End).\n\nnormalize_split_key({_Key, TakenLoad}, TargetLoad, End)\n  when TakenLoad < TargetLoad ->\n    {End, TakenLoad};\nnormalize_split_key({Key, TakenLoad}, _TargetLoad, _End) ->\n    {Key, TakenLoad}.\n\nnormalize_split_key_b({Key, TakenLoad}, TargetLoad, End)\n  when TakenLoad > TargetLoad ->\n    normalize_split_key({Key, TakenLoad - 1}, TargetLoad, End);\nnormalize_split_key_b({_Key, TakenLoad}, TargetLoad, End)\n  when TakenLoad == TargetLoad ->\n    normalize_split_key({End, TakenLoad}, TargetLoad, End);\nnormalize_split_key_b(Split, TargetLoad, End) ->\n    normalize_split_key(Split, TargetLoad, End).\n\n\n%% @doc Splits the database into a database (first element) which contains all\n%% keys in MyNewInterval and a list of the other values (second element).\n%% Note: removes all keys not in MyNewInterval from the list of changed\n%% keys!\n-spec split_data(DB::db(), MyNewInterval::intervals:interval()) -> {NewDB::db(), db_as_list()}.\nsplit_data(State = {DB, _Subscr, _SnapState}, MyNewInterval) ->\n    F = fun (Key, {StateAcc, HisList}) ->\n            DBEntry = ?DB:get(DB, Key),\n            case intervals:in(Key, MyNewInterval) of\n                true -> {StateAcc, HisList};\n                _ -> NewHisList = case db_entry:is_empty(DBEntry) of\n                                      false -> [DBEntry | HisList];\n                                      _ -> HisList\n                                  end,\n                    {delete_entry_at_key(StateAcc, Key, split), NewHisList}\n            end\n    end,\n    ?DB:foldl(DB, F, {State, []}).\n\n%% @doc Adds all db_entry objects in the Data list.\n-spec add_data(DB::db(), db_as_list()) -> NewDB::db().\nadd_data(DB, Data) ->\n    lists:foldl(fun(Entry, DBAcc) ->\n                        set_entry(DBAcc, Entry)\n                    end, DB, Data).\n\n%% @doc Gets (non-empty) db_entry objects in the given range.\n-spec get_entries(DB::db(), Range::intervals:interval()) -> db_as_list().\nget_entries(State, Interval) ->\n    {Elements, RestInterval} = intervals:get_elements(Interval),\n    case intervals:is_empty(RestInterval) of\n        true ->\n            [E || Key <- Elements, not db_entry:is_empty(E = get_entry(State, Key))];\n        _ ->\n            {_, Data} =\n                get_chunk(State, ?RT:hash_key(\"0\"), % any key will work, here!\n                           Interval,\n                           fun(DBEntry) -> not db_entry:is_empty(DBEntry) end,\n                           fun(E) -> E end, all),\n            Data\n    end.\n\n%% @doc Gets all custom objects (created by ValueFun(DBEntry)) from the DB for\n%% which FilterFun returns true.\n%% TODO only for legacy compatability; get_chunk should be used\n-spec get_entries(DB::db(),\n                  FilterFun::fun((DBEntry::db_entry:entry()) ->\n                                        boolean()),\n                  ValueFun::fun((DBEntry::db_entry:entry())\n                                -> Value)) -> [Value].\nget_entries(State, FilterFun, ValueFun) ->\n    element(2, get_chunk(State, ?RT:hash_key(\"0\"),\n                         intervals:all(), FilterFun, ValueFun, all)).\n\n\n%%%%%%\n%%% slide: delta recording (see dht_node_state.erl)\n%%%%%%\n\n%% @doc Adds the new interval to the interval to record changes for. Entries\n%%      which have (potentially) changed can then be gathered by get_changes/1.\n-spec record_changes(OldDB::db(), intervals:interval()) -> NewDB::db().\nrecord_changes(State, NewInterval) ->\n    RecChanges = get_subscription(State, record_changes),\n    ?TRACE(\"Old Subscription is ~p~n\", [RecChanges]),\n    NewSubscr =\n        case RecChanges of\n            [] -> {record_changes, NewInterval,\n                   fun subscr_delta/3, fun subscr_delta_close_table/1};\n            [{Tag, I, ChangesFun, RemSubscrFun}] ->\n                {Tag, intervals:union(I, NewInterval), ChangesFun, RemSubscrFun}\n        end,\n    ?TRACE(\"setting new subscription ~p~n\", [NewSubscr]),\n    set_subscription(State, NewSubscr).\n\n%% @doc Stops recording changes and removes all entries from the table of\n%%      changed keys.\n-spec stop_record_changes(OldDB::db()) -> NewDB::db().\nstop_record_changes(State) ->\n    remove_subscription(State, record_changes).\n\n%% @doc Stops recording changes in the given interval and removes all such\n%%      entries from the table of changed keys.\n-spec stop_record_changes(OldDB::db(), intervals:interval()) -> NewDB::db().\nstop_record_changes(State, Interval) ->\n    RecChanges = get_subscription(State, record_changes),\n    case RecChanges of\n        [] -> State;\n        [{Tag, I, ChangesFun, RemSubscrFun}] ->\n            subscr_delta_remove(State, Interval),\n            NewI = intervals:minus(I, Interval),\n            case intervals:is_empty(NewI) of\n                true -> remove_subscription(State, Tag);\n                _ -> set_subscription(State, {Tag, NewI, ChangesFun, RemSubscrFun})\n            end\n    end.\n\n%% @doc Gets all db_entry objects which have (potentially) been changed or\n%%      deleted (might return objects that have not changed but have been\n%%      touched by one of the DB setters).\n-spec get_changes(DB::db()) -> {Changed::db_as_list(), Deleted::[?RT:key()]}.\nget_changes(State) ->\n    get_changes(State, intervals:all()).\n\n%% @doc Gets all db_entry objects in the given interval which have\n%%      (potentially) been changed or deleted (might return objects that have\n%%      not changed but have been touched by one of the DB setters).\n-spec get_changes(DB::db(), intervals:interval()) -> {Changed::db_as_list(), Deleted::[?RT:key()]}.\nget_changes(State, Interval) ->\n    case erlang:get('$delta_tab') of\n        undefined -> get_changes_helper(State, [], Interval, [], []);\n        CKDB -> get_changes_helper(State, ?CKETS:tab2list(CKDB), Interval, [], [])\n    end.\n\n%% @doc Helper for get_changes/2 that adds the entry of a changed key either to\n%%      the list of changed entries or to the list of deleted entries.\n-spec get_changes_helper(State::db(), ChangedKeys::[{?RT:key()}],\n        Interval::intervals:interval(), ChangedEntries::[db_entry:entry()],\n        DeletedKeys::[?RT:key()])\n            -> {ChangedEntries::[db_entry:entry()], DeletedKeys::[?RT:key()]}.\nget_changes_helper(_State, [], _Interval, ChangedEntries, DeletedKeys) ->\n    {ChangedEntries, DeletedKeys};\nget_changes_helper(State, [{CurKey} | RestKeys], Interval, ChangedEntries, DeletedKeys) ->\n    case intervals:in(CurKey, Interval) of\n        true ->\n            Entry = get_entry(State, CurKey),\n            case db_entry:is_null(Entry) of\n                false -> ?TRACE(\"~p get_changes: ~p was changed~n\", [self(), CurKey]),\n                    get_changes_helper(State, RestKeys, Interval, [Entry | ChangedEntries], DeletedKeys);\n                _    -> ?TRACE(\"~p get_changes: ~p was deleted~n\", [self(), CurKey]),\n                    get_changes_helper(State, RestKeys, Interval, ChangedEntries, [CurKey | DeletedKeys])\n            end;\n        _ -> ?TRACE(\"~p get_changes: key ~p is not in ~p~n\", [self(), CurKey, Interval]),\n            get_changes_helper(State, RestKeys, Interval, ChangedEntries, DeletedKeys)\n    end.\n\n%% @doc Inserts/removes the key into the table of changed keys depending on the\n%%      operation (called whenever the DB is changed).\n-spec subscr_delta(State::db(), Tag::any(), Operation::subscr_op_t()) -> db().\nsubscr_delta(State, _Tag, Operation) ->\n    CKDB = subscr_delta_check_table(State),\n    ?TRACE(\"subscr_delta is called for op ~p~n\", [Operation]),\n    case Operation of\n        {write, Entry} -> ?CKETS:insert(CKDB, {db_entry:get_key(Entry)});\n        {delete, Key}  -> ?CKETS:insert(CKDB, {Key});\n        {split, Key}   -> ?CKETS:delete(CKDB, Key)\n    end,\n    State.\n\n%% @doc Cleans up, i.e. deletes, the table with changed keys (called on\n%%      subscription removal).\n-spec subscr_delta_close_table(Tag::any()) -> ok | true.\nsubscr_delta_close_table(_Tag) ->\n    case erlang:erase('$delta_tab') of\n        undefined -> ok;\n        CKDB -> ?CKETS:delete(CKDB)\n    end.\n\n%% @doc Check that the table storing changed keys exists and create it if\n%%      necessary.\n-spec subscr_delta_check_table(State::db()) -> ets:tid() | atom().\nsubscr_delta_check_table(_State) ->\n    DeltaDB = case erlang:get('$delta_tab') of\n        undefined ->\n            CKDB = ?CKETS:new(dht_node_db_ck, [ordered_set | ?DB_ETS_ADDITIONAL_OPS]),\n            erlang:put('$delta_tab', CKDB),\n            CKDB;\n        CKDB -> CKDB\n    end,\n    DeltaDB.\n\n%% @doc Removes any changed key in interval I (called when some (sub-)interval\n%%      is unsubscribed).\n-spec subscr_delta_remove(State::db(), I::intervals:interval()) -> ok.\nsubscr_delta_remove(State, Interval) ->\n    CKDB = subscr_delta_check_table(State),\n    F = fun(DBEntry, _) ->\n                Key = db_entry:get_key(DBEntry),\n                case intervals:in(Key, Interval) of\n                    true -> ?CKETS:delete(CKDB, Key);\n                    _    -> true\n                end\n        end,\n    ?CKETS:foldl(F, true, CKDB),\n    ok.\n\n%% @doc Deletes all objects in the given Range or (if a function is provided)\n%%      for which the FilterFun returns true from the DB.\n-spec delete_entries(DB::db(),\n                     RangeOrFun::intervals:interval() |\n                                 fun((DBEntry::db_entry:entry()) -> boolean()))\n        -> NewDB::db().\ndelete_entries(State = {DB, _Subscr, _SnapState}, FilterFun)\n  when is_function(FilterFun) ->\n    F = fun(Key, StateAcc) ->\n                DBEntry = ?DB:get(DB, Key),\n                case FilterFun(DBEntry) of\n                    false -> StateAcc;\n                    _     -> delete_entry(StateAcc, DBEntry)\n                end\n        end,\n    ?DB:foldl(DB, F, State);\ndelete_entries({DB, _Subscr, _SnapState} = State, Interval) ->\n    {Elements, RestInterval} = intervals:get_elements(Interval),\n    case intervals:is_empty(RestInterval) of\n        true ->\n            lists:foldl(fun(Key, State1) ->\n                                delete_entry_at_key(State1, Key)\n                        end, State, Elements);\n        _ ->\n            F = fun(Key, StateAcc) ->\n                        DBEntry = ?DB:get(DB, Key),\n                        delete_entry(StateAcc, DBEntry)\n                end,\n            SimpleI = intervals:get_simple_intervals(Interval),\n            lists:foldl(fun(I, AccIn) ->\n                                ?DB:foldl(DB, F, AccIn, I)\n                        end, State, SimpleI)\n    end.\n\n%%%%%%\n%%% end slide: delta recording (see dht_node_state.erl)\n%%%%%%\n\n\n%%%%%%\n%%% slide: slide snapshot data (see dht_node_state.erl)\n%%%%%%\n\n-spec snapshot_is_running(DB::db()) -> boolean().\nsnapshot_is_running({_DB, _Subscr, {SnapTable, _LiveLC, _SnapLC}}) ->\n    case SnapTable of\n        false -> false;\n        _     -> true\n    end.\n\n\n-spec add_snapshot_data(DB::db(), db_as_list()) -> NewDB::db().\nadd_snapshot_data(State, Entries) ->\n    lists:foldl(\n        fun(Entry, StateAcc) ->\n                set_snapshot_entry(StateAcc, Entry)\n        end, State, Entries).\n\n%% @doc Returns snapshot data as is for a specific interval\n-spec get_snapshot_data(DB::db(), intervals:interval()) -> db_as_list().\nget_snapshot_data({_DB, _Subscr, {false, _, _}}, _Interval) ->\n    [];\nget_snapshot_data({_DB, _Subscr, {SnapTable, _, _}}, [all]) ->\n    ?DB:foldl(SnapTable, fun(K, AccIn) -> [?DB:get(SnapTable, K) | AccIn] end, []);\nget_snapshot_data({_DB, _Subscr, {SnapTable, _, _}}, Interval) ->\n    %% TODO usort is only to make test suite happy since it thinks [all, all] is\n    %% a sensible interval and double entries where returned\n    lists:usort(fun({K1,_,_,_,_}, {K2,_,_,_,_}) -> K1 =< K2 end,\n                    lists:foldl(\n        fun(I, Acc) ->\n            ?DB:foldl(SnapTable,\n                      fun(Key, AccIn) ->\n                            Entry = ?DB:get(SnapTable, Key),\n                            ?TRACE(\"get_snapshot_data: adding ~p to data\",\n                                   [Entry]),\n                            [Entry | AccIn]\n                      end,\n                      Acc,\n                      I)\n                    end, [], intervals:get_simple_intervals(Interval))).\n\n\n%%%%%%\n%%% subscriptions to DB changes (called locally by delta recording)\n%%%%%%\n\n%% @doc Adds a subscription for the given interval under Tag (overwrites an\n%%     existing subscription with that tag).\n-spec set_subscription(State::db(), subscr_t()) -> db().\nset_subscription({DB, Subscr, SnapState}, Subscription) ->\n    {DB, db_ets:put(Subscr, Subscription), SnapState}.\n\n%% @doc Gets a subscription stored under Tag (empty list if there is none).\n-spec get_subscription(State::db(), Tag::any()) -> [subscr_t()].\nget_subscription({_DB, Subscr, _SnapState}, Tag) ->\n    case db_ets:get(Subscr, Tag) of\n        {} ->\n            [];\n        SubsT ->\n            [SubsT]\n    end.\n\n%% @doc Removes a subscription stored under Tag (if there is one).\n-spec remove_subscription(State::db(), Tag::any()) -> db().\nremove_subscription({DB, Subscr, SnapState}, Tag) ->\n    case db_ets:get(Subscr, Tag) of\n        {} -> ok;\n        {Tag, _I, _ChangesFun, RemSubscrFun} -> RemSubscrFun(Tag)\n    end,\n    {DB, db_ets:delete(Subscr, Tag), SnapState}.\n\n%% @doc Go through all subscriptions and perform the given operation if\n%%      matching.\n-spec call_subscribers(State::db(), Operation::close_db | subscr_op_t()) -> db().\ncall_subscribers(State = {_DB, Subscr, _SnapState}, Operation) ->\n    {NewState, _Op} = db_ets:foldl(Subscr,\n              fun call_subscribers_iter/2,\n              {State, Operation}),\n    NewState.\n\n%% @doc Iterates over all susbcribers and calls their subscribed functions.\n-spec call_subscribers_iter(subscr_t(), {State::db(), Operation::close_db |\n                                         subscr_op_t()}) -> {db(),\n                                                             Operation::close_db\n                                                             | subscr_op_t()}.\ncall_subscribers_iter(Tag, {{_DB, Subscr, _SnapState} = State, Op}) ->\n    % assume the key exists (it should since we are iterating over the table!)\n    {Tag, I, ChangesFun, RemSubscrFun} = db_ets:get(Subscr, Tag),\n    NewState =\n        case Op of\n            close_db ->\n                RemSubscrFun(Tag),\n                State;\n            Operation ->\n                Key = case Operation of\n                    {write, Entry} -> db_entry:get_key(Entry);\n                    {delete, K}  -> K;\n                    {split, K}  -> K\n                end,\n                case intervals:in(Key, I) of\n                    false ->\n                        ?TRACE(\"not calling subscribers...~p not in interval ~p~n\",\n                               [Key, I]),\n                        State;\n                    _     ->\n                        ?TRACE(\"calling subscriber for tag ~p and op ~p~n\", [Tag,\n                                                                             Operation]),\n                        ChangesFun(State, Tag, Operation)\n                end\n        end,\n    {NewState, Op}.\n\n\n\n%%%%%%\n%%% snapshots: business logic\n%%%%%%\n\n-spec init_snapshot(DB::db()) -> NewDB::db().\ninit_snapshot({DB, Subscr, {SnapTable, LiveLC, _SnapLC}}) ->\n    case SnapTable of\n        false -> ok;\n        _     -> ?DB:close(SnapTable)\n    end,\n    SnapDBName = \"db_\" ++ randoms:getRandomString() ++ \":snapshot\",\n    % copy live db lock count to new snapshot db\n    {DB, Subscr, {?DB:new(SnapDBName), LiveLC, LiveLC}}.\n\n-spec delete_snapshot(DB::db()) -> NewDB::db().\ndelete_snapshot({_DB, _Subscr, {false, _LiveLC, _SnapLC}} = State) ->\n    State;\ndelete_snapshot({DB, Subscr, {SnapTable, LiveLC, _SnapLC}}) ->\n    ?DB:close(SnapTable),\n    {DB, Subscr, {false, LiveLC, 0}}.\n\n%% @doc Join snapshot and primary db such that all tuples in the\n%%      primary db are replaced if there is a matching tuple available\n%%      in the snapshot set. The other tuples are returned as is.\n-spec join_snapshot_data(DB::db()) -> db_as_list().\njoin_snapshot_data(State) ->\n    PrimaryDB = lists:keysort(1, get_entries(State, intervals:all())),\n    SnapshotDB = lists:keysort(1, get_snapshot_data(State)),\n    join_snapshot_data_helper(SnapshotDB, PrimaryDB).\n\n-spec join_snapshot_data_helper(SnapshotDB::db_as_list(),\n                                PrimaryDB::db_as_list())\n                               -> db_as_list().\njoin_snapshot_data_helper([], Result) -> Result;\njoin_snapshot_data_helper([{Key, _, _, _, _} = Tuple | More], List2) ->\n    Newlist = lists:keyreplace(Key, 1, List2, Tuple),\n    join_snapshot_data_helper(More, Newlist).\n\n-spec snapshot_is_lockfree(DB::db()) -> boolean().\nsnapshot_is_lockfree({_DB, _Subscr, {_SnapTable, _LiveLC, SnapLC}}) ->\n    SnapLC =:= 0.\n\n-spec get_live_lc(DB::db()) -> non_neg_integer().\nget_live_lc({_DB, _Subscr, {_SnapTable, LiveLC, _SnapLC}}) ->\n    LiveLC.\n\n-spec get_snap_lc(DB::db()) -> non_neg_integer().\nget_snap_lc({_DB, _Subscr, {_SnapTable, _LiveLC, SnapLC}}) ->\n    SnapLC.\n\n%%-spec decrease_snapshot_lockcount(DB::db()) -> NewDB::db().\n%%decrease_snapshot_lockcount({DB, Subscr, {SnapTable, LiveLC, SnapLC}}) ->\n%%    {DB, Subscr, {SnapTable, LiveLC, SnapLC - 1}}.\n\n%% do all the necessary things to maintain possibly running\n%% snapshots. called in set_entry()\nsnaps({DB, Subscr, {false, LiveLC, SnapLC}} = State, Entry, _OpSnapNum,\n      _OwnSnapNo) ->\n    %% no snapshot running, just update the live lockcount\n    LC_new = db_entry:lockcount(Entry),\n    % NOTE: only retrieve OldEntry if needed\n    LC_old =\n        if LC_new =:= 0 andalso LiveLC =:= 0 andalso SnapLC =:= 0 ->\n               0; % must be 0, otherwise DB is in erroneous state!\n           true ->\n               db_entry:lockcount(get_entry(State, db_entry:get_key(Entry)))\n        end,\n    {DB, Subscr, {false, LiveLC + LC_new - LC_old, SnapLC}};\nsnaps({DB, Subscr, {SnapDB, LiveLC, SnapLC}} = State, Entry, OpSnapNum,\n      OwnSnapNo) when OpSnapNum >= OwnSnapNo ->\n    OldEntry = get_entry(State, db_entry:get_key(Entry)),\n    %% This case hints at the validate phase of a\n    %% transaction. Lockcounts should only increase and in this case\n    %% we should do copy-on-write. There is a special case to handle:\n    %% when a write transaction is aborted but we voted prepare we\n    %% need to correct the lockcounts.\n    case db_entry:lockcount(Entry) - db_entry:lockcount(OldEntry) of\n        Delta when Delta < 0 ->\n            %% check if LiveLC and SnapLC are > 1 in old db in this\n            %% case a transaction got through validation just before a\n            %% new snapshot begun on this node.  On other nodes the\n            %% snapshots was triggered before the transaction...hence\n            %% the abort.  lockcount needs to be decreased so\n            %% snapshots can advance\n            if LiveLC > 0 andalso SnapLC >= LiveLC ->\n                    %% in case the tx was validated before new\n                    %% snapshot we need to decrease the copied\n                    %% lockcount\n                    ?TRACE_SNAP(\"db:snaps ~p\n                                snapnumbers not ok but snaplocks exist\n                                ~p   ~p~n~p\",\n                                [self(), OpSnapNum, OwnSnapNo,\n                                 {DB, Subscr, {SnapDB, LiveLC + Delta, SnapLC +\n                                               Delta}}]),\n                    {DB, Subscr, {SnapDB, LiveLC + Delta, SnapLC + Delta}};\n                true ->\n                    ?TRACE_SNAP(\"db:snaps ~p\n                    snapnumbers not ok but locks are ok\n                                ~p   ~p~n~p~n~p\",\n                                [self(), OpSnapNum,\n                                 OwnSnapNo, DB, {DB, Subscr, {SnapDB, LiveLC +\n                                                              Delta, SnapLC +\n                                                              Delta}}]),\n                    {DB, Subscr, {SnapDB, LiveLC + Delta, SnapLC}}\n            end;\n        0 ->\n            %% not sure if this can happen....but it just does nothing\n            State;\n        Delta ->\n            %% new transaction validated. this one should not belong to the\n            %% snapshot so do copy-on-write\n            copy_value_to_snapshot_table({DB, Subscr, {SnapDB, LiveLC + Delta,\n                                                        SnapLC}}, db_entry:get_key(Entry))\n\n    end;\nsnaps({DB, Subscr, {SnapDB, LiveLC, SnapLC}} = State, Entry, OpSnapNum,\n      OwnSnapNo) when OpSnapNum < OwnSnapNo ->\n    OldEntry = get_entry(State, db_entry:get_key(Entry)),\n    %% this case hints at commit or abort of a transaction that belongs into the\n    %% snapshot. Since this is the finishing phase of the transaction locks\n    %% should decrease. If locks increase log a warning since thsi should not\n    %% happen,\n    case db_entry:lockcount(Entry) - db_entry:lockcount(OldEntry) of\n        Delta when Delta < 0 ->\n            SnapEntry = get_snapshot_entry(State, db_entry:get_key(Entry)),\n            case db_entry:is_null(SnapEntry) of\n                false ->\n                    ?TRACE_SNAP(\"db:snaps:~p~nkey in snapdb...reducing lockcount\",\n                                [self()]),\n                    % in this case there was an entry with this key in the snapshot table\n                    % so it might have different locks than the one in the live db.\n                    % we're applying the lock decrease on the snapshot table entry\n                    set_snapshot_entry({DB, Subscr, {SnapDB, LiveLC + Delta,\n                                                         SnapLC}}, Entry);\n                _ ->\n                    ?TRACE_SNAP(\"db:snaps ~p~nkey not in snapdb~n~p\",\n                                [self(), db_entry:get_key(Entry)]),\n                    % key was not found in snapshot table -> both dbs are in sync for this key\n                    {DB, Subscr, {SnapDB, LiveLC + Delta, SnapLC + Delta}}\n            end;\n        0 ->\n            %% not sure if this can happen....but it just does nothing\n            State;\n        _Delta ->\n            log:log(warn, \"db_common:snaps(): ~p~nlockcount increase but op has old\n                    snapnumber...should not happen~p  ~p~n~p~n~p~n~p\",\n                    [self(), OpSnapNum, OwnSnapNo, OldEntry, Entry, State]),\n            State\n    end.\n\n\n%%%%%%\n%%% debugging, diagnostic outputs\n%%%%%%\n\n%% @doc Returns snapshot data as is for whole interval\n-spec get_snapshot_data(DB::db()) -> db_as_list().\nget_snapshot_data(DB) ->\n    get_snapshot_data(DB, intervals:all()).\n\n-spec set_snapshot_entry(DB::db(), Entry::db_entry:entry()) -> NewDB::db().\nset_snapshot_entry(State = {DB, Subscr, {SnapTable, LiveLC, SnapLC}}, Entry) ->\n    case db_entry:is_null(Entry) of\n        true -> delete_snapshot_entry(State, Entry);\n        _    ->\n            % if there is a snapshot entry for this key, we base our lock calculation on that,\n            % if not, we have to consider the live db because of the copy-on-write logic\n            OldEntry = get_snapshot_entry(State, db_entry:get_key(Entry)),\n            NewSnapLC = case db_entry:is_null(OldEntry) of\n                false ->\n                    SnapLC + db_entry:lockcount(Entry) - db_entry:lockcount(OldEntry);\n                _ ->\n                    LiveEntry = get_entry(State, db_entry:get_key(Entry)),\n                    SnapLC + db_entry:lockcount(Entry) - db_entry:lockcount(LiveEntry)\n            end,\n            ?TRACE_SNAP(\"set_snapshot_entry: ~p~n~p~n~p\",\n                        [self(), NewSnapLC, Entry]),\n            {DB, Subscr, {?DB:put(SnapTable, Entry), LiveLC, NewSnapLC}}\n    end.\n\n-spec get_snapshot_entry(DB::db(), Key::?RT:key()) -> db_entry:entry().\nget_snapshot_entry({_DB, _Subscr, {SnapTable, _LiveLC, _SnapLC}}, Key) ->\n    case ?DB:get(SnapTable, Key) of\n        {} ->\n            db_entry:new(Key);\n        Entry ->\n            Entry\n    end.\n\n%% @doc Removes all values with the given entry's key from the Snapshot DB.\n-spec delete_snapshot_entry(DB::db(), Entry::db_entry:entry()) -> NewDB::db().\ndelete_snapshot_entry(State, Entry) ->\n    Key = db_entry:get_key(Entry),\n    delete_snapshot_entry_at_key(State, Key).\n\n-spec delete_snapshot_entry_at_key(DB::db(), Key::?RT:key()) -> NewDB::db().\ndelete_snapshot_entry_at_key(State = {DB, Subscr, {SnapTable, LiveLC, SnapLC}}, Key) ->\n    OldEntry = get_snapshot_entry(State, Key),\n    NewSnapLC = case db_entry:is_null(OldEntry) of\n        false ->\n            SnapLC - db_entry:lockcount(OldEntry);\n        _ ->\n            LiveEntry = get_entry(State, Key),\n            SnapLC - db_entry:lockcount(LiveEntry)\n    end,\n    ?TRACE(\"deleting key ~p\", [Key]),\n    {DB, Subscr, {?DB:delete(SnapTable, Key), LiveLC, NewSnapLC}}.\n\n\n%%%%%%\n%%% for unittests\n%%%%%%\n\n%% @doc Checks whether all entries in the DB are valid, i.e.\n%%      - no writelocks and readlocks at the same time\n%%      - no empty_val values (these should only be in the DB temporarily)\n%%      - version is greater than or equal to 0\n%%      Returns the result of the check and a list of invalid entries.\n%%      Used in unittests.\n-spec check_db(DB::db()) -> {true, []} | {false, InvalidEntries::db_as_list()}.\ncheck_db({DB, _Subscr, _Snap}) ->\n    Data = ?DB:foldl(DB, fun(K, A) -> [?DB:get(DB, K) | A] end, []),\n    ValidFun = fun(DBEntry) ->\n                       not db_entry:is_empty(DBEntry) andalso\n                           not (db_entry:get_writelock(DBEntry) =/= false andalso\n                                    db_entry:get_readlock(DBEntry) > 0) andalso\n                           db_entry:get_version(DBEntry) >= 0\n               end,\n    {_Valid, Invalid} = lists:partition(ValidFun, Data),\n    case Invalid of\n        [] -> {true, []};\n        _  -> {false, Invalid}\n    end.\n\n%% @doc Copy existing entry to snapshot table\n-spec copy_value_to_snapshot_table(DB::db(), Key::?RT:key()) -> NewDB::db().\ncopy_value_to_snapshot_table(State = {DB, Subscr, {SnapTable, LiveLC, SnapLC}}, Key) ->\n    Entry = get_entry(State, Key),\n    NewSnap = case db_entry:is_null(Entry) of\n        false   ->\n            OldSnapEntry = get_snapshot_entry(State, db_entry:get_key(Entry)),\n            ?TRACE_SNAP(\"copy_value_to_snapshot_table: ~p~nfrom ~p to ~p~n~p\",\n                        [self(), SnapLC, TmpLC, Entry]),\n            {?DB:put(SnapTable, Entry), LiveLC, SnapLC +\n             db_entry:lockcount(Entry) - db_entry:lockcount(OldSnapEntry)};\n        _ ->\n            {SnapTable, LiveLC, SnapLC}\n    end,\n    {DB, Subscr, NewSnap}.\n\n%% @doc Updates all (existing or non-existing) non-locked entries from\n%%      NewEntries for which Pred(OldEntry, NewEntry) returns true with\n%%      UpdateFun(OldEntry, NewEntry).\n-spec update_entries(DB::db(),\n                     Values::[db_entry:entry()],\n                     Pred::fun((OldEntry::db_entry:entry(),\n                                NewEntry::db_entry:entry()) -> boolean()),\n                     UpdateFun::fun((OldEntry::db_entry:entry(),\n                                     NewEntry::db_entry:entry()) ->\n                                           UpdatedEntry::db_entry:entry()))\n                    -> NewDB::db().\nupdate_entries(OldDB, NewEntries, Pred, UpdateFun) ->\n    F = fun(NewEntry, DB) ->\n                OldEntry = get_entry(DB, db_entry:get_key(NewEntry)),\n                IsNotLocked = not db_entry:is_locked(OldEntry),\n                IsUpdatable = IsNotLocked andalso Pred(OldEntry, NewEntry),\n                case db_entry:is_null(OldEntry) of\n                    true when IsUpdatable ->\n                        set_entry(DB, UpdateFun(OldEntry, NewEntry));\n                    _ when IsUpdatable ->\n                        update_entry(DB, UpdateFun(OldEntry, NewEntry));\n                    _ ->\n                        DB\n                end\n        end,\n    lists:foldl(F, OldDB, NewEntries).\n\n\n%%%%%%\n%%% locally used helper functions to long to place inside\n%%% corresponding context block\n%%%%%%\n\n-spec calc_remaining_interval(?RT:key(), non_neg_integer(), db_as_list(),\n                              intervals:interval()) -> intervals:interval().\n%% if there are less elements in Chunk that ChunkSize allows, the whole interval\n%% was covered\ncalc_remaining_interval(_StartId, Remaining, _Chunk, _Interval)\n        when Remaining > 0 -> intervals:empty();\n%% if Chunk is empty the whole Interval was covered\ncalc_remaining_interval(_StartId, _Remaining, [], _Interval) ->\n    intervals:empty();\ncalc_remaining_interval(StartId, _Remaining, Chunk, Interval) ->\n    %% the interval covered by chunk is either the biggest key left of startid\n    %% or if there are no keys left of startid simply the biggest key in chunk\n    Last = calc_last_key_rem_int(Chunk, StartId),\n    intervals:minus(Interval, intervals:new('[', StartId, Last, ']')).\n\n%% @doc Gets the largest key in Chunk left of StartId if there is one,\n%%      otherwise gets the largest key of all items.\n-spec calc_last_key_rem_int(Chunk::[db_entry:entry(),...], StartId::?RT:key()) -> ?RT:key().\ncalc_last_key_rem_int([{Key, _, _, _, _} | Rest], StartId) ->\n    calc_last_key_rem_int(Rest, StartId, Key, Key < StartId).\n\n%% @doc Helper for calc_last_key_rem_int/2.\n-spec calc_last_key_rem_int(db_as_list(), StartId::?RT:key(), Max::?RT:key(),\n                            OnlySmallerThanStart::boolean()) -> ?RT:key().\ncalc_last_key_rem_int([], _StartId, Max, _OnlySmallerThanStart) -> Max;\ncalc_last_key_rem_int([{Key, _, _, _, _} | Rest], StartId, Max, true) when Key < StartId ->\n    calc_last_key_rem_int(Rest, StartId, ?IIF(Key > Max, Key, Max), true);\ncalc_last_key_rem_int([_ | Rest], StartId, Max, true) ->\n    calc_last_key_rem_int(Rest, StartId, Max, true);\ncalc_last_key_rem_int([{Key, _, _, _, _} | Rest], StartId, _Max, false) when Key < StartId ->\n    calc_last_key_rem_int(Rest, StartId, Key, true);\ncalc_last_key_rem_int([{Key, _, _, _, _} | Rest], StartId, Max, false) ->\n    calc_last_key_rem_int(Rest, StartId, ?IIF(Key > Max, Key, Max), false).\n\n\n"
  },
  {
    "path": "src/db_entry.erl",
    "content": "%% @copyright 2010-2012 Zuse Institute Berlin\n%%            and onScale solutions GmbH\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@onscale.de>\n%% @doc Abstract datatype of a single DB entry.\n%% @version $Id$\n-module(db_entry).\n-author('schintke@onscale.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([new/1, new/3,\n         get_key/1,\n         get_value/1, set_value/3,\n         get_readlock/1, inc_readlock/1, dec_readlock/1,\n         get_writelock/1, set_writelock/2, unset_writelock/1,\n         get_version/1,\n         reset_locks/1, is_locked/1,\n         is_empty/1, is_null/1,\n         lockcount/1]).\n\n% only for unit tests:\n-export([inc_version/1, dec_version/1]).\n\n-export_type([entry/0, entry_ex/0, entry_empty/0]).\n\n% note: do not make opaque so DB implementations can rely on an entry() being a\n% tuple as defined here\n% note: WriteLock is either false or the version (>= Version) that a write\n% operation is working on (this allows proper cleanup - see rdht_tx_write)\n-type entry_ex() ::\n          {Key::?RT:key(), Value::db_dht:value(), WriteLock::false | client_version(),\n           ReadLock::non_neg_integer(), Version::client_version()}.\n-type entry_empty() ::\n          {Key::?RT:key(), empty_val | db_dht:value(), WriteLock::false | -1 | client_version(),\n           ReadLock::non_neg_integer(), Version::-1}.\n-type entry() :: entry_ex() | entry_empty().\n\n-spec new(Key::?RT:key()) -> {?RT:key(), empty_val, false, 0, -1}.\nnew(Key) -> {Key, empty_val, false, 0, -1}.\n\n-spec new(Key::?RT:key(), Value::db_dht:value(), Version::client_version()) ->\n    {Key::?RT:key(), Value::db_dht:value(), WriteLock::false,\n     ReadLock::0, Version::client_version()}.\nnew(Key, Value, Version) -> {Key, Value, false, 0, Version}.\n\n-spec get_key(DBEntry::entry()) -> ?RT:key().\nget_key(DBEntry) -> element(1, DBEntry).\n\n-spec get_value(DBEntry::entry()) -> db_dht:value().\nget_value(DBEntry) -> element(2, DBEntry).\n\n-spec set_value(DBEntry::entry(), Value::db_dht:value(), Version::client_version()) -> entry().\nset_value(DBEntry, Value, Version) ->\n    setelement(2, setelement(5, DBEntry, Version), Value).\n\n-spec get_writelock(DBEntry::entry()) -> WriteLock::false | -1 | client_version().\nget_writelock(DBEntry) -> element(3, DBEntry).\n\n-spec set_writelock(entry_ex(), false | client_version()) -> entry_ex();\n                   (entry_empty(), false | -1 | client_version()) -> entry_empty().\nset_writelock(DBEntry, WriteLock) -> setelement(3, DBEntry, WriteLock).\n\n-spec unset_writelock(DBEntry::entry()) -> entry().\nunset_writelock(DBEntry) -> set_writelock(DBEntry, false).\n\n-spec get_readlock(DBEntry::entry()) -> ReadLock::non_neg_integer().\nget_readlock(DBEntry) -> element(4, DBEntry).\n\n-spec set_readlock(DBEntry::entry(), ReadLock::non_neg_integer()) -> entry().\nset_readlock(DBEntry, ReadLock) -> setelement(4, DBEntry, ReadLock).\n\n-spec inc_readlock(DBEntry::entry()) -> entry().\ninc_readlock(DBEntry) -> set_readlock(DBEntry, get_readlock(DBEntry) + 1).\n\n-spec dec_readlock(DBEntry::entry()) -> entry().\ndec_readlock(DBEntry) ->\n    case get_readlock(DBEntry) of\n        0 -> log:log(warn, \"Decreasing empty readlock\"), DBEntry;\n        N -> set_readlock(DBEntry, N - 1)\n    end.\n\n-spec get_version(DBEntry::entry()) -> client_version() | -1.\nget_version(DBEntry) -> element(5, DBEntry).\n\n-spec inc_version(DBEntry::entry()) -> entry().\ninc_version(DBEntry) -> setelement(5, DBEntry, get_version(DBEntry) + 1).\n\n-spec dec_version(DBEntry::entry()) -> entry().\ndec_version(DBEntry) -> setelement(5, DBEntry, get_version(DBEntry) - 1).\n\n-spec reset_locks(DBEntry::entry()) ->\n    {Key::?RT:key(), Value::db_dht:value(), WriteLock::false,\n     ReadLock::0, Version::client_version()} |\n    {Key::?RT:key(), empty_val | db_dht:value(), WriteLock::false,\n     ReadLock::0, Version::-1}.\nreset_locks(DBEntry) ->\n    TmpEntry = set_readlock(DBEntry, 0),\n    set_writelock(TmpEntry, false).\n\n-spec is_locked(DBEntry::entry()) -> boolean().\nis_locked(DBEntry) ->\n    get_readlock(DBEntry) > 0 orelse get_writelock(DBEntry) =/= false.\n\n%% @doc Returns whether the item is an empty_val item with version -1.\n%%      Note: The number of read or write locks does not matter here!\n-spec is_empty(entry()) -> boolean().\nis_empty({_Key, empty_val, _WriteLock, _ReadLock, -1}) -> true;\nis_empty(_) -> false.\n\n%% @doc Returns whether the item is an empty_val item with version -1 and no\n%%      read or write locks.\n-spec is_null(entry()) -> boolean().\nis_null({_Key, empty_val, false, 0, -1}) -> true;\nis_null(_) -> false.\n\n%% @doc Returns how many read and write locks are currently set.\n-spec lockcount(db_entry:entry()) -> integer().\nlockcount(Entry) ->\n    WL = get_writelock(Entry),\n    if WL =:= false -> get_readlock(Entry);\n       true         -> get_readlock(Entry) + 1\n    end.\n"
  },
  {
    "path": "src/db_ets.erl",
    "content": "% @copyright 2013-2014 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc    DB back-end using ets.\n%%         Two keys K and L are considered equal if K == L yields true.\n%% @end\n%% @version $Id$\n-module(db_ets).\n-author('fajerski@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n-behaviour(db_backend_beh).\n\n-define(TRACE(_X, _Y), ok).\n%% -define(TRACE(X, Y), io:format(X, Y)).\n%% -define(TRACE(X, Y), ct:pal(X, Y)).\n\n%% primitives\n-export([new/1, new/2, open/1, close/1, close_and_delete/1, put/2, get/2, delete/2]).\n%% db info\n-export([get_persisted_tables/0, get_name/1, get_load/1, \n         is_available/0, supports_feature/1]).\n\n%% iteration\n-export([foldl/3, foldl/4, foldl/5]).\n-export([foldr/3, foldr/4, foldr/5]).\n-export([foldl_unordered/3]).\n-export([tab2list/1]).\n\n-type db() :: ets:tab().\n-type key() :: db_backend_beh:key(). %% '$end_of_table' is not allowed as key() or else iterations won't work!\n-type entry() :: db_backend_beh:entry().\n\n-export_type([db/0]).\n\n%% @doc Creates new DB handle named DBName.\n-spec new(DBName::nonempty_string()) -> db().\nnew(_DBName) ->\n    %% IMPORTANT: this module only works correctly when using ordered_set ets\n    %% tables. Other table types could throw bad_argument exceptions while\n    %% calling ets:next/2\n    ets:new(dht_node_db, [ordered_set | ?DB_ETS_ADDITIONAL_OPS]).\n\n%% @doc Creates new DB handle named DBName with possibility to pass Options.\n-spec new(DBName::nonempty_string(), Options::[term()] ) -> db().\nnew(_DBName, Options) ->\n    %% IMPORTANT: this module only works correctly when using ordered_set ets\n    %% tables. Other table types could throw bad_argument exceptions while\n    %% calling ets:next/2\n    ets:new(dht_node_db, [ordered_set | Options]).\n\n%% @doc Open a previously existing database. Not supported by ets.\n%%      A new database is created\n-spec open(DBName::nonempty_string()) -> no_return().\nopen(_DBName) ->\n    erlang:throw(\"open/1 not supported by ets\").\n\n%% @doc Closes and deletes the DB named DBName\n-spec close(DBName::db()) -> true.\nclose(DBName) ->\n    ets:delete(DBName).\n\n%% @doc Closes and deletes the DB named DBName\n-spec close_and_delete(DBName::db()) -> true.\nclose_and_delete(DBName) ->\n    ets:delete(DBName).\n\n%% @doc Saves arbitrary tuple Entry or list of tuples Entries\n%%      in DB DBName and returns the new DB.\n%%      The key is expected to be the first element of Entry.\n-spec put(DBName::db(), Entry::entry() | [Entries::entry()]) -> db().\nput(DBName, []) ->\n    DBName;\nput(DBName, Entry) ->\n    ?DBG_ASSERT(case is_list(Entry) of\n                    true ->\n                        lists:all(\n                          fun(E) ->\n                                  element(1, E) =/= '$end_of_table'\n                          end,\n                          Entry);\n                    false ->\n                        element(1, Entry) =/= '$end_of_table'\n                end),\n    ets:insert(DBName, Entry),\n    DBName.\n\n%% @doc Returns the entry that corresponds to Key or {} if no such tuple exists.\n-spec get(DBName::db(), Key::key()) -> entry() | {}.\nget(DBName, Key) ->\n    case ets:lookup(DBName, Key) of\n        [Entry] ->\n            Entry;\n        [] ->\n            {}\n    end.\n\n%% @doc Deletes the tuple saved under Key and returns the new DB.\n%%      If such a tuple does not exists nothing is changed.\n-spec delete(DBName::db(), Key::key()) -> db().\ndelete(DBName, Key) ->\n    ets:delete(DBName, Key),\n    DBName.\n\n%% @doc Gets a list of persisted tables (none with ets!).\n-spec get_persisted_tables() -> [nonempty_string()].\nget_persisted_tables() ->\n    [].\n\n%% @doc Returns the name of the DB specified in new/1.\n-spec get_name(DB::db()) -> nonempty_string().\nget_name(DB) ->\n    erlang:atom_to_list(ets:info(DB, name)).\n\n%% @doc Checks for modules required for this DB backend. Returns true if no \n%%      modules are missing, or else a list of missing modules\n-spec is_available() -> boolean() | [atom()].\nis_available() ->\n    case code:which(ets) of\n        non_existing -> [ets];\n        _ -> true\n    end.\n\n%% @doc Returns true if the DB support a specific feature (e.g. recovery), false otherwise.\n-spec supports_feature(Feature::atom()) -> boolean().\nsupports_feature(_Feature) -> false.\n\n%% @doc Returns the current load (i.e. number of stored tuples) of the DB.\n-spec get_load(DB::db()) -> non_neg_integer().\nget_load(DB) ->\n    ets:info(DB, size).\n\n%% @doc Is equivalent to ets:foldl(Fun, Acc0, DB).\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldl(DB, Fun, Acc) ->\n    foldl(DB, Fun, Acc, {'[', ets:first(DB), ets:last(DB), ']'}, ets:info(DB, size)).\n\n%% @doc Is equivalent to foldl(DB, Fun, Acc0, Interval, get_load(DB)).\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Interval::db_backend_beh:interval()) -> Acc1::A.\nfoldl(DB, Fun, Acc, Interval) ->\n    foldl(DB, Fun, Acc, Interval, ets:info(DB, size)).\n\n%% @doc foldl iterates over DB and applies Fun(Entry, AccIn) to every element\n%%      encountered in Interval. On the first call AccIn == Acc0. The iteration\n%%      stops as soon as MaxNum elements have been encountered.\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Intervall::db_backend_beh:interval(), MaxNum::non_neg_integer()) -> Acc1::A.\nfoldl(_DB, _Fun, Acc, _Interval, 0) -> Acc;\nfoldl(_DB, _Fun, Acc, {_, '$end_of_table', _End, _}, _MaxNum) -> Acc;\nfoldl(_DB, _Fun, Acc, {_, _Start, '$end_of_table', _}, _MaxNum) -> Acc;\nfoldl(_DB, _Fun, Acc, {_, Start, End, _}, _MaxNum) when Start > End -> Acc;\nfoldl(DB, Fun, Acc, {El}, _MaxNum) ->\n    case ets:lookup(DB, El) of\n        [] ->\n            Acc;\n        [_Entry] ->\n            Fun(El, Acc)\n    end;\nfoldl(DB, Fun, Acc, all, MaxNum) ->\n    foldl(DB, Fun, Acc, {'[', ets:first(DB), ets:last(DB), ']'},\n          MaxNum);\nfoldl(DB, Fun, Acc, {'(', Start, End, RBr}, MaxNum) ->\n    foldl(DB, Fun, Acc, {'[', ets:next(DB, Start), End, RBr}, MaxNum);\nfoldl(DB, Fun, Acc, {LBr, Start, End, ')'}, MaxNum) ->\n    foldl(DB, Fun, Acc, {LBr, Start, ets:prev(DB, End), ']'}, MaxNum);\nfoldl(DB, Fun, Acc, {'[', Start, End, ']'}, MaxNum) ->\n    case ets:lookup(DB, Start) of\n        [] ->\n            foldl(DB, Fun, Acc, {'[', ets:next(DB, Start), End, ']'},\n                       MaxNum);\n        [_Entry] ->\n            foldl_iter(DB, Fun, Acc, {'[', Start, End, ']'},\n                       MaxNum)\n    end.\n\n-spec foldl_iter(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Intervall::db_backend_beh:interval(), MaxNum::non_neg_integer()) -> Acc1::A.\nfoldl_iter(_DB, _Fun, Acc, _Interval, 0) -> Acc;\nfoldl_iter(_DB, _Fun, Acc, {_, '$end_of_table', _End, _}, _MaxNum) -> Acc;\nfoldl_iter(_DB, _Fun, Acc, {_, _Start, '$end_of_table', _}, _MaxNum) -> Acc;\nfoldl_iter(_DB, _Fun, Acc, {_, Start, End, _}, _MaxNum) when Start > End -> Acc;\nfoldl_iter(DB, Fun, Acc, {'[', Start, End, ']'}, MaxNum) ->\n    ?TRACE(\"foldl:~nstart: ~p~nend:   ~p~nmaxnum: ~p~ninterval: ~p~n\",\n           [Start, End, MaxNum, {'[', Start, End, ']'}]),\n    foldl_iter(DB, Fun, Fun(Start, Acc),\n               {'[', ets:next(DB, Start), End, ']'}, MaxNum - 1).\n\n%% @doc Is equivalent to ets:foldr(Fun, Acc0, DB).\n-spec foldr(db(), fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldr(DB, Fun, Acc) ->\n    foldr(DB, Fun, Acc, {'[', ets:first(DB), ets:last(DB), ']'}, ets:info(DB, size)).\n\n%% @doc Is equivalent to foldr(DB, Fun, Acc0, Interval, get_load(DB)).\n-spec foldr(db(), fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A, db_backend_beh:interval()) -> Acc1::A.\nfoldr(DB, Fun, Acc, Interval) ->\n    foldr(DB, Fun, Acc, Interval, ets:info(DB, size)).\n\n%% @doc Behaves like foldl/5 with the difference that it starts at the end of\n%%      Interval and iterates towards the start of Interval.\n-spec foldr(db(), fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A, db_backend_beh:interval(), non_neg_integer()) -> Acc1::A.\nfoldr(_DB, _Fun, Acc, _Interval, 0) -> Acc;\nfoldr(_DB, _Fun, Acc, {_, _End, '$end_of_table', _}, _MaxNum) -> Acc;\nfoldr(_DB, _Fun, Acc, {_, '$end_of_table', _Start, _}, _MaxNum) -> Acc;\nfoldr(_DB, _Fun, Acc, {_, End, Start, _}, _MaxNum) when Start < End -> Acc;\nfoldr(DB, Fun, Acc, {El}, _MaxNum) ->\n    case ets:lookup(DB, El) of\n        [] ->\n            Acc;\n        [_Entry] ->\n            Fun(El, Acc)\n    end;\nfoldr(DB, Fun, Acc, all, MaxNum) ->\n    foldr(DB, Fun, Acc, {'[', ets:first(DB), ets:last(DB), ']'},\n          MaxNum);\nfoldr(DB, Fun, Acc, {'(', End, Start, RBr}, MaxNum) ->\n    foldr(DB, Fun, Acc, {'[', ets:next(DB, End), Start, RBr}, MaxNum);\nfoldr(DB, Fun, Acc, {LBr, End, Start, ')'}, MaxNum) ->\n    foldr(DB, Fun, Acc, {LBr, End, ets:prev(DB, Start), ']'}, MaxNum);\nfoldr(DB, Fun, Acc, {'[', End, Start, ']'}, MaxNum) ->\n    case ets:lookup(DB, Start) of\n        [] ->\n            foldr(DB, Fun, Acc, {'[', End, ets:prev(DB, Start), ']'},\n                       MaxNum);\n        [_Entry] ->\n            foldr_iter(DB, Fun, Acc, {'[', End, Start, ']'},\n                       MaxNum)\n    end.\n\n-spec foldr_iter(db(), fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A, db_backend_beh:interval(), non_neg_integer()) -> Acc1::A.\nfoldr_iter(_DB, _Fun, Acc, _Interval, 0) -> Acc;\nfoldr_iter(_DB, _Fun, Acc, {_, _End, '$end_of_table', _}, _MaxNum) -> Acc;\nfoldr_iter(_DB, _Fun, Acc, {_, '$end_of_table', _Start, _}, _MaxNum) -> Acc;\nfoldr_iter(_DB, _Fun, Acc, {_, End, Start, _}, _MaxNum) when Start < End -> Acc;\nfoldr_iter(DB, Fun, Acc, {'[', End, Start, ']'}, MaxNum) ->\n    ?TRACE(\"foldr:~nstart: ~p~nend ~p~nmaxnum: ~p~nfound\",\n           [Start, End, MaxNum]),\n    foldr_iter(DB, Fun, Fun(Start, Acc), {'[', End, ets:prev(DB, Start), ']'}, MaxNum -\n          1).\n\n%% @doc Works similar to foldl/3 but uses ets:foldl instead of our own implementation. \n%% The order in which will be iterated over is unspecified, but using this fuction\n%% might be faster than foldl/3 if it does not matter.\n-spec foldl_unordered(DB::db(), Fun::fun((Entry::entry(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldl_unordered(DB, Fun, Acc) ->\n    ets:foldl(Fun, Acc, DB).\n\n\n%% @doc Returns a list of all objects in the table Table_name.\n-spec tab2list(Table_name::db()) -> [Entries::entry()].\ntab2list(Table_name) ->\n    ets:tab2list(Table_name).\n"
  },
  {
    "path": "src/db_hanoidb.erl",
    "content": "%% @copyright 2013-2018 Scalaris project http://scalaris.zib.de\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Pierre M.\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    DB back-end using HanoiDB.\n%%         HanoiDB is a memory-cached disk backend.\n%%         As disks are large (TB) HanoiDB can hold data much larger than RAM (GB).\n%%         As disks persist data HanoiDB can be stoped and restarted without data loss.\n%%         It is a pure Erlang implementation of Google's LevelDB disk-backed K/V store.\n%%         See http://code.google.com/p/leveldb/ for background about storage levels.\n%%         How to use scalaris with this hanoidb backend:\n%%         -download https://github.com/krestenkrab/hanoidb and compile HanoiDB\n%%         -make sure this db_hanoidb.erl file is in src/ (right with db_ets.erl)\n%%         -rerun scalaris' configure with --enable-hanoidb\n%%             ./configure --enable-hanoidb=/path/to/hanoidb\n%%         -rerun make to rebuild scalaris and run tests\n%%             ./make\n%%             ./make test\n%%         -enjoy\n%%         Two keys K and L are considered equal if they match, i.e. K =:= L\n%%         Made after v0.6.1 svn rev 5666.\n%% @end\n-module(db_hanoidb).\n\n-include(\"scalaris.hrl\").\n\n-behaviour(db_backend_beh).\n\n-define(IN(E), erlang:term_to_binary(E, [{minor_version, 1}])).\n-define(OUT(E), erlang:binary_to_term(E)).\n\n%% primitives\n-export([new/1, open/1]).\n-export([put/2, get/2, delete/2]).\n-export([close/1, close_and_delete/1]).\n\n%% db info\n-export([get_persisted_tables/0, get_name/1, get_load/1,\n         is_available/0, supports_feature/1]).\n\n%% iteration\n-export([foldl/3, foldl/4, foldl/5]).\n-export([foldr/3, foldr/4, foldr/5]).\n-export([foldl_unordered/3]).\n-export([tab2list/1]).\n\n-type db() :: {DB::pid(), FileName::nonempty_string()}.\n-type key() :: db_backend_beh:key(). %% '$end_of_table' is not allowed as key() or else iterations won't work!\n-type entry() :: db_backend_beh:entry().\n\n-export_type([db/0]).\n\n-type hanoidb_config_option() ::  {compress, none | gzip | snappy | lz4}\n                                | {page_size, pos_integer()}\n                                | {read_buffer_size, pos_integer()}\n                                | {write_buffer_size, pos_integer()}\n                                | {merge_strategy, fast | predictable }\n                                | {sync_strategy, none | sync | {seconds, pos_integer()}}\n                                | {expiry_secs, non_neg_integer()}\n                                | {spawn_opt, list()}.\n\n%% @doc Creates new DB handle named DBName.\n-spec new(DBName::nonempty_string()) -> db().\nnew(DBName) ->\n    new_db(DBName, []). % hanoidb's default options. May need tuning.\n\n%% @doc Re-opens an existing-on-disk database.\n-spec open(DBName::nonempty_string()) -> db().\nopen(DBName) ->\n    new_db(DBName, []). % hanoidb's default options. May need tuning.\n\n%% @doc Creates new DB handle named DBName with options.\n-spec new_db(DirName::string(), HanoiOptions::[hanoidb_config_option()]) -> db().\nnew_db(DBName, HanoiOptions) ->\n    BaseDir = [config:read(db_directory), \"/\", atom_to_list(node())],\n    _ = case file:make_dir(BaseDir) of\n            ok -> ok;\n            {error, eexist} -> ok;\n            {error, Error0} -> erlang:exit({?MODULE, 'cannot create dir', BaseDir, Error0})\n        end,\n\n    % HanoiDB stores not in a file but a dir store\n    FullDBDir = lists:flatten([BaseDir, \"/\", DBName]),\n    case hanoidb:open(FullDBDir, HanoiOptions) of\n        {ok, Tree} ->    {Tree, DBName};\n        ignore ->    log:log(error, \"[ Node ~w:db_hanoidb ] ~.0p\", [self(), ignore]),\n                     erlang:error({hanoidb_failed, ignore});\n        {error, Error2} -> log:log(error, \"[ Node ~w:db_hanoidb ] ~.0p\", [self(), Error2]),\n                           erlang:error({hanoidb_failed, Error2})\n    end.\n\n%% @doc Closes the DB named DBName keeping its data on disk.\n-spec close(DB::db()) -> true.\nclose({DB, _FileName}) ->\n    ok = hanoidb:close(DB),\n    true.\n    % hanoidb:stop(). Not needed.\n\n%% @doc Closes and deletes the DB named DBName\n-spec close_and_delete(DB::db()) -> true.\nclose_and_delete({_DB, DBName} = State) ->\n    close(State),\n    % A disk backend happens in some directory\n\n    DirName = [config:read(db_directory), \"/\", atom_to_list(node()), \"/\", DBName],\n\n    % Delete all DB files\n    {ok, Files} = file:list_dir(DirName),\n    lists:foreach(fun(FileName) ->\n                          FullFileName = lists:flatten([DirName, \"/\", FileName]),\n                          case file:delete(FullFileName) of\n                              ok -> ok;\n                              {error, Reason} ->\n                                  log:log(error, \"[ Node ~w:~w ] deleting ~.0p failed: ~.0p\",\n                                          [self(), ?MODULE, FileName, Reason])\n                          end\n                  end, Files),\n\n    % Delete DB dir\n    case file:del_dir(DirName) of\n        ok -> ok;\n        {error, Reason} -> log:log(error, \"[ Node ~w:db_hanoidb ] deleting ~.0p failed: ~.0p\",\n                                   [self(), DirName, Reason])\n    end.\n\n%% @doc Saves arbitrary tuple Entry in DB DBName and returns the new DB.\n%%      The key is expected to be the first element of Entry.\n-spec put(DB::db(), Entry::entry()) -> db().\nput({DB, _DBName} = State, Entry) ->\n    ok = hanoidb:put(DB, ?IN(element(1, Entry)), ?IN(Entry)    ),\n    State.\n\n%% @doc Returns the entry that corresponds to Key or {} if no such tuple exists.\n-spec get(DB::db(), Key::key()) -> entry() | {}.\nget({DB, _DBName}, Key) ->\n    case hanoidb:get(DB, ?IN(Key)) of\n        not_found    -> {};\n        {ok, Entry}    -> ?OUT(Entry)\n    end.\n\n%% @doc Deletes the tuple saved under Key and returns the new DB.\n%%      If such a tuple does not exists nothing is changed.\n-spec delete(DB::db(), Key::key()) -> db().\ndelete({DB, _FileName} = State, Key) ->\n    ok = hanoidb:delete(DB, ?IN(Key)),\n    State.\n\n%% @doc Gets a list of persisted tables.\n-spec get_persisted_tables() -> [nonempty_string()].\nget_persisted_tables() ->\n    %% TODO: implement\n    [].\n\n\n%% @doc Checks for modules required for this DB backend. Returns true if no\n%%      modules are missing, or else a list of missing modules\n-spec is_available() -> boolean() | [atom()].\nis_available() ->\n    case code:which(hanoidb) of\n        non_existing -> [hanoidb];\n        _ -> true\n    end.\n\n%% @doc Returns true if the DB support a specific feature (e.g. recovery), false otherwise.\n-spec supports_feature(Feature::atom()) -> boolean().\nsupports_feature(recover) -> true;\nsupports_feature(_) -> false.\n\n%% @doc Returns the name of the DB specified in @see new/1 and open/1.\n-spec get_name(DB::db()) -> nonempty_string().\nget_name({_DB, DBName}) ->\n    DBName.\n\n%% @doc Returns the number of stored keys.\n-spec get_load(DB::db()) -> non_neg_integer().\nget_load({DB, _DBName}) ->\n    %% TODO: not really efficient (maybe store the load in the DB?)\n    hanoidb:fold(DB, fun (_K, _V, Load) -> Load + 1 end, 0).\n\n%% @equiv hanoidb:fold_range(DB, Fun, Acc0, #key_range{from_key = <<>>, to_key = undefined})\n%% @doc Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldl(State, Fun, Acc) ->\n    %hanoidb:fold(DB, fun (K, _V, AccIn) -> Fun(?OUT(K), AccIn) end, Acc0).\n    foldl_helper(State, Fun, Acc, all, -1).\n\n%% @equiv foldl(DB, Fun, Acc0, Interval, get_load(DB))\n%% @doc   Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n            Interval::db_backend_beh:interval()) -> Acc1::A.\nfoldl(State, Fun, Acc, Interval) ->\n    %hanoidb:fold_range(DB, Fun, Acc, #key_range{from_key=K1, to_key=K2}). % TODO check it is possible\n    foldl_helper(State, Fun, Acc, Interval, -1).\n\n%% @doc foldl iterates over DB and applies Fun(Entry, AccIn) to every element\n%%      encountered in Interval. On the first call AccIn == Acc0. The iteration\n%%      stops as soon as MaxNum elements have been encountered.\n%%      Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n            Intervall::db_backend_beh:interval(), MaxNum::non_neg_integer()) -> Acc1::A.\nfoldl(State, Fun, Acc, Interval, MaxNum) ->\n    %% HINT\n    %% Fun can only be applied in a second pass. It could do a delete (or other\n    %% write op) but CAN HanoiDB handle writes whiles folding ? (TODO check YES?)\n    %% Since we reversed the order while accumulating reverse it by using lists\n    %% fold but \"from the other side\". TODO check this for HanoiDB\n    %hanoidb:fold_range(DB, Fun, Acc, #key_range{limit=N, from_key=K1, to_key=K2}) % TODO check it is possible\n    foldl_helper(State, Fun, Acc, Interval, MaxNum).\n\n%% @private this helper enables us to use -1 as MaxNum. MaxNum == -1 signals that all\n%%          data is to be retrieved.\n-spec foldl_helper(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                   Intervall::db_backend_beh:interval(), MaxNum::integer()) -> Acc1::A.\nfoldl_helper({DB, _FileName}, Fun, Acc, Interval, MaxNum) ->\n    Keys = get_all_keys(DB, Interval, MaxNum), % hopefully MaxNum caps it.\n    lists:foldr(Fun, Acc, Keys). % db:foldL calls lists:foldR\n    % TODO May be hanoidb:fold_range is less RAM intensive : no need to keep all keys in RAM at once, but continuous folding instead.\n\n%% @doc makes a foldr over the whole dataset.\n%%      Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldr(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldr(State, Fun, Acc) ->\n    foldr_helper(State, Fun, Acc, all, -1).\n\n%% @equiv foldr(DB, Fun, Acc0, Interval, get_load(DB))\n%% @doc   Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldr(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n            Interval::db_backend_beh:interval()) -> Acc1::A.\nfoldr(State, Fun, Acc, Interval) ->\n    foldr_helper(State, Fun, Acc, Interval, -1).\n\n%% @doc foldr iterates over DB and applies Fun(Entry, AccIn) to every element\n%%      encountered in Interval. On the first call AccIn == Acc0. The iteration\n%%      stops as soon as MaxNum elements have been encountered.\n%%      Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldr(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n            Intervall::db_backend_beh:interval(), MaxNum::non_neg_integer()) -> Acc1::A.\nfoldr(State, Fun, Acc, Interval, MaxNum) ->\n    foldr_helper(State, Fun, Acc, Interval, MaxNum).\n\n%% @private this helper enables us to use -1 as MaxNum. MaxNum == -1 signals that all\n%%          data is to be retrieved.\n-spec foldr_helper(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                   Intervall::db_backend_beh:interval(), MaxNum::integer()) -> Acc1::A.\nfoldr_helper({DB, _FileName}, Fun, Acc, Interval, MaxNum) ->\n    % TODO evaluate hanoidb:fold_range(DB, Fun, Acc, #key_range{limit=N, from_key=K1, to_key=K2})\n    %% first only retrieve keys so we don't have to load the whole db into memory\n    Keys = get_all_keys(DB, Interval, -1),\n    CutData = case MaxNum of\n                  N when N < 0 ->\n                      Keys;\n                  _ ->\n                      lists:sublist(Keys, MaxNum)\n              end,\n    %% see HINT in foldl/5\n    %% now retrieve actual data\n    lists:foldl(Fun, Acc, CutData).\n\n%% @doc Works similar to foldl/3 but uses hanoidb:fold instead of our own implementation.\n%% The order in which will be iterated over is unspecified, but using this fuction\n%% might be faster than foldl/3 if it does not matter.\n-spec foldl_unordered(DB::db(), Fun::fun((Entry::entry(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldl_unordered({DB, _DBName}, Fun, Acc) ->\n    hanoidb:fold(DB, fun (_K, Entry, AccIn) -> Fun(?OUT(Entry), AccIn) end, Acc).\n\n\n%% @private get_all_keys/3 retrieves all keys in DB that fall into Interval but\n%%          not more than MaxNum. If MaxNum == -1 all Keys are retrieved. If\n%%          MaxNum is positive it starts from the left in term order.\n-spec get_all_keys(pid(), db_backend_beh:interval(), -1 | non_neg_integer())\n        -> [key()].\nget_all_keys(DB, Interval, MaxNum) ->\n    % TODO evaluate converting scalaris:Intervals to hanoidb:ranges\n    % in order to leverage hanoidb:fold rather than get_all_keys+lists:fold.\n\n    Keys = hanoidb:fold(DB, fun(Key, _Entry, AccIn) -> [?OUT(Key) | AccIn] end, []),\n\n    {_, In} = lists:foldl(fun\n                             (_, {0, _} = AccIn) ->\n                                  AccIn;\n                             (Key, {Max, KeyAcc} = AccIn) ->\n                                  case is_in(Interval, Key) of\n                                      true ->\n                                          {Max - 1, [Key | KeyAcc]};\n                                      _ ->\n                                          AccIn\n                                  end\n                          end, {MaxNum, []}, lists:sort(Keys)),\n    In.\n\nis_in({Key}, OtherKey) -> Key =:= OtherKey;\nis_in(all, _Key) -> true;\nis_in({'(', L, R, ')'}, Key) -> Key > L andalso Key < R;\nis_in({'(', L, R, ']'}, Key) -> Key > L andalso ((Key < R) orelse (Key =:= R));\nis_in({'[', L, R, ')'}, Key) -> ((Key > L) orelse (Key =:= L)) andalso Key < R;\nis_in({'[', L, R, ']'}, Key) -> ((Key > L) orelse (Key =:= L)) andalso\n                                    ((Key < R) orelse (Key =:= R)).\n\n%% @doc Returns a list of all objects in the table Table_name.\n-spec tab2list(Table_name::db()) -> [Entries::entry()].\ntab2list(_Table_name) ->\n    %% Not implemented yet.\n    [].\n"
  },
  {
    "path": "src/db_mnesia.erl",
    "content": "% @copyright 2013-2019 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Tanguy Racinet\n%% @doc    DB back-end using mnesia.\n%%         Two keys K and L are considered equal if K == L yields true.\n%% @end\n-module(db_mnesia).\n-author('tanracinet@gmail.com').\n-vsn('$Id: db_ets.erl 6270 2014-03-28 14:25:52Z fajerski@zib.de $').\n\n-include(\"scalaris.hrl\").\n\n-behaviour(db_backend_beh).\n\n-define(TRACE(_X, _Y), ok).\n%% -define(TRACE(X, Y), io:format(X, Y)).\n%% -define(TRACE(X, Y), ct:pal(X, Y)).\n\n%% primitives\n-export([start/0, new/1, new/2, open/1, close/1, put/2, get/2, delete/2]).\n%% db info\n-export([get_persisted_tables/0, get_name/1, get_load/1,\n         is_available/0, supports_feature/1]).\n%% cleanup functions\n-export([mnesia_tables_of/1, delete_mnesia_tables/1, close_and_delete/1]).\n\n%% iteration\n-export([foldl/3, foldl/4, foldl/5]).\n-export([foldr/3, foldr/4, foldr/5]).\n-export([foldl_unordered/3]).\n-export([tab2list/1]).\n\n-type db() :: atom().\n-type key() :: db_backend_beh:key(). %% '$end_of_table' is not allowed as key() or else iterations won't work!\n-type entry() :: db_backend_beh:entry().\n\n-export_type([db/0]).\n\n-export([traverse_table_and_show/1]).\n\n-spec start() -> ok.\nstart() ->\n  FullDataDir = config:read(db_directory) ++ \"/\" ++ atom_to_list(node()),\n  io:format(\"FullDataDir: ~p\", [FullDataDir]),\n  application:set_env(mnesia, dir, FullDataDir),\n%  case config:read(_type) of\n  case config:read(start_type) of\n    recover ->\n      case mnesia:create_schema([node()]) of\n        ok ->\n          io:format(\"starting mnesia: no previous Schema to recover from.~n\"),\n          ok = mnesia:delete_schema([node()]),\n          erlang:halt();\n        {error, {_, {already_exists, _}}} ->\n          io:format(\"starting mnesia: recovering.~n\");\n        Msg ->\n              case util:is_unittest() of\n                  true  -> ct:pal(\"starting mnesia recover : ~w~n\", [Msg]);\n                  false -> io:format(\"starting mnesia recover : ~w~n\", [Msg])\n              end,\n              erlang:halt()\n      end;\n    _ ->\n      case mnesia:create_schema([node()]) of\n        ok -> ok;\n        Msg ->\n              VMName = case config:read(vmname) of\n                           failed -> atom_to_list(novmname);\n                           X -> atom_to_list(X)\n                       end,\n              case util:is_unittest() of\n                  true ->\n                      ct:pal(\"starting mnesia: ~p~n\", [Msg]),\n                      ct:pal(\"starting mnesia: maybe you tried to start a new node \"\n                             \"while we still found persisted data of a node with the \"\n                             \"same name. If you want to get rid of the old persisted \"\n                             \"data, delete them using ~p.~n\",\n                             [\"rm -rf data/\" ++ VMName ++ \"/\" ++ atom_to_list(node())]);\n                  false ->\n                      io:format(\"starting mnesia: ~p~n\", [Msg]),\n                      io:format(\"starting mnesia: maybe you tried to start a new node \"\n                                \"while we still found persisted data of a node with the \"\n                                \"same name. If you want to get rid of the old persisted \"\n                                \"data, delete them using ~p.~n\",\n                                [\"rm -rf data/\" ++ VMName ++ \"/\" ++ atom_to_list(node())])\n              end,\n              erlang:halt()\n      end\n  end,\n  _ = application:start(mnesia),\n  ok.\n\n%% @doc traverse table and print content\n-spec traverse_table_and_show(Table_name::nonempty_string()) -> ok.\ntraverse_table_and_show(Table_name)->\n  Iterator =  fun(Rec,_)->\n    log:log(warn, \"~p~n\",[Rec]),\n    []\n  end,\n  case mnesia:is_transaction() of\n    true -> mnesia:foldl(Iterator,[],Table_name);\n    false ->\n      Exec = fun({Fun,Tab}) -> mnesia:foldl(Fun, [],Tab) end,\n      mnesia:activity(sync_transaction,Exec,[{Iterator,Table_name}],mnesia_frag)\n  end.\n\n%% @doc Return all the tables owned by PidGroup\n%%      NOTE: only returns tables with names according to this regular expression:\n%%            <tt>^[^:]+:PidGroup(:.*)?$</tt>\n-spec mnesia_tables_of(PidGroup::pid_groups:groupname()) -> [atom()].\nmnesia_tables_of(PidGroup) ->\n  Tabs = mnesia:system_info(tables),\n  [ Tab || Tab <- Tabs,\n           element(2, db_util:parse_table_name(\n                     erlang:atom_to_list(Tab))) =:= PidGroup ].\n\n%% @doc Gets a list of persisted tables.\n-spec get_persisted_tables() -> [nonempty_string()].\nget_persisted_tables() ->\n      [atom_to_list(Table) || Table <- mnesia:system_info(tables), Table =/= schema].\n\n%% @doc Close recursivly all mnesia tables in List\n-spec delete_mnesia_tables(list()) -> ok.\ndelete_mnesia_tables(Tabs) ->\n    _ = [close(Tab) || Tab <- Tabs],\n    ok.\n\n%% @doc Creates new DB handle named DBName.\n-spec new(DBName::nonempty_string()) -> db().\nnew(DBName) ->\n    ?TRACE(\"new:~nDB_name:~p~n\",[DBName]),\n    DbAtom = list_to_atom(DBName),\n    {atomic, ok} = mnesia:create_table(DbAtom, [{disc_copies, [node()]},\n                                                {type, ordered_set}]),\n    DbAtom.\n\n%% @doc Creates new DB handle named DBName with possibility to pass Options.\n-spec new(DBName::nonempty_string(), Options::[term()] ) -> db().\nnew(DBName, Options) ->\n    ?TRACE(\"new:~nDB_name:~p~nOption~p~n\",[DBName, Options]),\n    DbAtom = list_to_atom(DBName),\n    {atomic, ok} = mnesia:create_table(DbAtom, [{disc_copies, [node()]},\n                                                {type, ordered_set} | Options]),\n    DbAtom.\n\n%% @doc Open a previously existing database assuming the database has been\n%%      restored by the start of the mnesia application.\n-spec open(DBName::nonempty_string()) -> db().\nopen(DBName) ->\n    erlang:list_to_atom(DBName).\n\n%% @doc Closes the DB named DBName\n-spec close(DB::db()) -> true.\nclose(DB) ->\n  ?TRACE(\"close:~nDB_name:~p~n\",[DB]),\n  {atomic, ok} = mnesia:delete_table(DB),\n  true.\n\n%% @doc Closes and deletes the DB named DBName\n-spec close_and_delete(DBName::db()) -> true.\nclose_and_delete(DBName) ->\n  close(DBName).\n\n%% @doc Saves arbitrary tuple Entry or list of tuples Entries\n%%      in DB DBName and returns the new DB.\n%%      The key is expected to be the first element of Entry.\n-spec put(DBName::db(), Entry::entry() | [Entries::entry()]) -> db().\nput(DBName, []) ->\n    DBName;\nput(DBName, Entry) ->\n    ?DBG_ASSERT(case is_list(Entry) of\n                    true ->\n                        lists:all(\n                          fun(E) ->\n                                  element(1, E) =/= '$end_of_table'\n                          end,\n                          Entry);\n                    false ->\n                        element(1, Entry) =/= '$end_of_table'\n                end),\n    {atomic, _} = mnesia:transaction(fun() -> mnesia:write({DBName, element(1, Entry), Entry}) end),\n    %% FUTOPT (future optimization): remove the 'catch' when we\n    %% drop support for Erlang < 17.0\n    %% greatly kills performance...\n    %% catch %% when undefined; Erlang < 17.0-rc2 does not have this function\n    %% mnesia:sync_log(),\n    DBName.\n\n%% @doc Returns the entry that corresponds to Key or {} if no such tuple exists.\n-spec get(DBName::db(), Key::key()) -> entry() | {}.\nget(DBName, Key) ->\n    case mnesia:transaction(fun() -> mnesia:read(DBName, Key) end) of\n      {atomic, [Entry]} ->\n            element(3, Entry);\n      {atomic, []} ->\n            {}\n    end.\n\n%% @doc Deletes the tuple saved under Key and returns the new DB.\n%%      If such a tuple does not exists nothing is changed.\n-spec delete(DBName::db(), Key::key()) -> db().\ndelete(DBName, Key) ->\n    {atomic, _} = mnesia:transaction(fun()-> mnesia:delete({DBName, Key}) end),\n    DBName.\n\n%% @doc Returns the name of the DB specified in new/1.\n-spec get_name(DB::db()) -> nonempty_string().\nget_name(DB) ->\n    erlang:atom_to_list(mnesia:table_info(DB, name)).\n\n%% @doc Checks for modules required for this DB backend. Returns true if no \n%%      modules are missing, or else a list of missing modules\n-spec is_available() -> boolean() | [atom()].\nis_available() ->\n    case code:which(mnesia) of\n        non_existing -> [mnesia];\n        _ -> true\n    end.\n\n%% @doc Returns true if the DB support a specific feature (e.g. recovery), false otherwise.\n-spec supports_feature(Feature::atom()) -> boolean().\nsupports_feature(recover) -> true;\nsupports_feature(_) -> false.\n\n%% @doc Returns the current load (i.e. number of stored tuples) of the DB.\n-spec get_load(DB::db()) -> non_neg_integer().\nget_load(DB) ->\n    mnesia:table_info(DB, size).\n\n%% @doc Is equivalent to ets:foldl(Fun, Acc0, DB).\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldl(DB, Fun, Acc) ->\n  ?TRACE(\"foldl/3:~n\",[]),\n  {atomic, First} = mnesia:transaction(fun()-> mnesia:first(DB) end),\n  {atomic, Last} = mnesia:transaction(fun()-> mnesia:last(DB) end),\n  foldl(DB, Fun, Acc, {'[', First, Last, ']'}, mnesia:table_info(DB, size)).\n\n%% @doc foldl/4 iterates over DB and applies Fun(Entry, AccIn) to every element\n%%      encountered in Interval. On the first call AccIn == Acc0. The iteration\n%%      only apply Fun to the elements inside the Interval.\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Interval::db_backend_beh:interval()) -> Acc1::A.\nfoldl(DB, Fun, Acc, Interval) ->\n    ?TRACE(\"foldl/4:~nstart:~n\",[]),\n    foldl(DB, Fun, Acc, Interval, mnesia:table_info(DB, size)).\n\n%% @doc foldl/5 iterates over DB and applies Fun(Entry, AccIn) to every element\n%%      encountered in Interval. On the first call AccIn == Acc0. The iteration\n%%      stops as soon as MaxNum elements have been encountered.\n-spec foldl(db(), fun((Key::key(), A) -> A), A,\n                               db_backend_beh:interval(), non_neg_integer()) -> Acc1::A.\nfoldl(_DB, _Fun, Acc, _Interval, 0) -> Acc;\nfoldl(_DB, _Fun, Acc, {_, '$end_of_table', _End, _}, _MaxNum) -> Acc;\nfoldl(_DB, _Fun, Acc, {_, _Start, '$end_of_table', _}, _MaxNum) -> Acc;\nfoldl(_DB, _Fun, Acc, {_, Start, End, _}, _MaxNum) when Start > End -> Acc;\nfoldl(DB, Fun, Acc, {El}, _MaxNum) ->\n    case mnesia:transaction(fun() -> mnesia:read(DB, El) end) of\n      {atomic, []} ->\n            Acc;\n      {atomic, [_Entry]} ->\n            Fun(El, Acc)\n    end;\nfoldl(DB, Fun, Acc, all, MaxNum) ->\n  {atomic, First} = mnesia:transaction(fun()-> mnesia:first(DB) end),\n  {atomic, Last} = mnesia:transaction(fun()-> mnesia:last(DB) end),\n  foldl(DB, Fun, Acc, {'[', First, Last, ']'}, MaxNum);\nfoldl(DB, Fun, Acc, {'(', Start, End, RBr}, MaxNum) ->\n    {atomic, Next} = mnesia:transaction(fun()-> mnesia:next(DB, Start) end),\n    foldl(DB, Fun, Acc, {'[', Next, End, RBr}, MaxNum);\nfoldl(DB, Fun, Acc, {LBr, Start, End, ')'}, MaxNum) ->\n    {atomic, Previous} = mnesia:transaction(fun()-> mnesia:prev(DB, End) end),\n    foldl(DB, Fun, Acc, {LBr, Start, Previous, ']'}, MaxNum);\nfoldl(DB, Fun, Acc, {'[', Start, End, ']'}, MaxNum) ->\n    ?TRACE(\"foldl:~nstart: ~p~nend:   ~p~nmaxnum: ~p~ninterval: ~p~n\",\n      [Start, End, MaxNum, {'[', Start, End, ']'}]),\n    case mnesia:transaction(fun()-> mnesia:read(DB, Start) end) of\n      {atomic, []} ->\n            {atomic, Next} = mnesia:transaction(fun()-> mnesia:next(DB, Start) end),\n            foldl(DB, Fun, Acc, {'[', Next, End, ']'},\n                       MaxNum);\n      {atomic, [_Entry]} ->\n            foldl_iter(DB, Fun, Acc, {'[', Start, End, ']'},\n                       MaxNum)\n    end.\n\n%% @doc foldl_iter(/5) is a recursive function applying Fun only on elements\n%%      inside the Interval. It is called by every foldl operation.\n-spec foldl_iter(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Intervall::db_backend_beh:interval(), MaxNum::non_neg_integer()) -> Acc1::A.\nfoldl_iter(_DB, _Fun, Acc, _Interval, 0) -> Acc;\nfoldl_iter(_DB, _Fun, Acc, {_, '$end_of_table', _End, _}, _MaxNum) -> Acc;\nfoldl_iter(_DB, _Fun, Acc, {_, _Start, '$end_of_table', _}, _MaxNum) -> Acc;\nfoldl_iter(_DB, _Fun, Acc, {_, Start, End, _}, _MaxNum) when Start > End -> Acc;\nfoldl_iter(DB, Fun, Acc, {'[', Start, End, ']'}, MaxNum) ->\n    ?TRACE(\"foldl_iter:~nstart: ~p~nend:   ~p~nmaxnum: ~p~ninterval: ~p~n\",\n           [Start, End, MaxNum, {'[', Start, End, ']'}]),\n    {atomic, Next} = mnesia:transaction(fun()-> mnesia:next(DB, Start) end),\n    foldl_iter(DB, Fun, Fun(Start, Acc), {'[', Next, End, ']'}, MaxNum - 1).\n\n%% @doc Is equivalent to foldr(Fun, Acc0, DB).\n-spec foldr(db(), fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldr(DB, Fun, Acc) ->\n    {atomic, First} = mnesia:transaction(fun()-> mnesia:first(DB) end),\n    {atomic, Last} = mnesia:transaction(fun()-> mnesia:last(DB) end),\n    foldr(DB, Fun, Acc, {'[', First, Last, ']'}, mnesia:table_info(DB, size)).\n\n%% @doc Is equivalent to foldr(DB, Fun, Acc0, Interval, get_load(DB)).\n-spec foldr(db(), fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A, db_backend_beh:interval()) -> Acc1::A.\nfoldr(DB, Fun, Acc, Interval) ->\n    foldr(DB, Fun, Acc, Interval, mnesia:table_info(DB, size)).\n\n%% @doc Behaves like foldl/5 with the difference that it starts at the end of\n%%      Interval and iterates towards the start of Interval.\n-spec foldr(db(), fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A, db_backend_beh:interval(), non_neg_integer()) -> Acc1::A.\nfoldr(_DB, _Fun, Acc, _Interval, 0) -> Acc;\nfoldr(_DB, _Fun, Acc, {_, _End, '$end_of_table', _}, _MaxNum) -> Acc;\nfoldr(_DB, _Fun, Acc, {_, '$end_of_table', _Start, _}, _MaxNum) -> Acc;\nfoldr(_DB, _Fun, Acc, {_, End, Start, _}, _MaxNum) when Start < End -> Acc;\nfoldr(DB, Fun, Acc, {El}, _MaxNum) ->\n    case mnesia:transaction(fun()-> mnesia:read(DB, El) end) of\n      {atomic, []} ->\n            Acc;\n      {atomic, [_Entry]} ->\n            Fun(El, Acc)\n    end;\nfoldr(DB, Fun, Acc, all, MaxNum) ->\n    {atomic, First} = mnesia:transaction(fun()-> mnesia:first(DB) end),\n    {atomic, Last} = mnesia:transaction(fun()-> mnesia:last(DB) end),\n    foldr(DB, Fun, Acc, {'[', First, Last, ']'}, MaxNum);\nfoldr(DB, Fun, Acc, {'(', End, Start, RBr}, MaxNum) ->\n    {atomic, Next} = mnesia:transaction(fun()-> mnesia:next(DB, End) end),\n    foldr(DB, Fun, Acc, {'[', Next, Start, RBr}, MaxNum);\nfoldr(DB, Fun, Acc, {LBr, End, Start, ')'}, MaxNum) ->\n    {atomic, Previous} = mnesia:transaction(fun()-> mnesia:prev(DB, Start) end),\n    foldr(DB, Fun, Acc, {LBr, End, Previous, ']'}, MaxNum);\nfoldr(DB, Fun, Acc, {'[', End, Start, ']'}, MaxNum) ->\n    case mnesia:transaction(fun()-> mnesia:read(DB, Start) end) of\n      {atomic, []} ->\n            {atomic, Previous} = mnesia:transaction(fun()-> mnesia:prev(DB, Start) end),\n            foldr(DB, Fun, Acc, {'[', End, Previous, ']'}, MaxNum);\n      {atomic, [_Entry]} ->\n            foldr_iter(DB, Fun, Acc, {'[', End, Start, ']'}, MaxNum)\n    end.\n\n-spec foldr_iter(db(), fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A, db_backend_beh:interval(), non_neg_integer()) -> Acc1::A.\nfoldr_iter(_DB, _Fun, Acc, _Interval, 0) -> Acc;\nfoldr_iter(_DB, _Fun, Acc, {_, _End, '$end_of_table', _}, _MaxNum) -> Acc;\nfoldr_iter(_DB, _Fun, Acc, {_, '$end_of_table', _Start, _}, _MaxNum) -> Acc;\nfoldr_iter(_DB, _Fun, Acc, {_, End, Start, _}, _MaxNum) when Start < End -> Acc;\nfoldr_iter(DB, Fun, Acc, {'[', End, Start, ']'}, MaxNum) ->\n    ?TRACE(\"foldr:~nstart: ~p~nend ~p~nmaxnum: ~p~nfound\",\n           [Start, End, MaxNum]),\n  {atomic, Previous} = mnesia:transaction(fun()-> mnesia:prev(DB, Start) end),\n    foldr_iter(DB, Fun, Fun(Start, Acc), {'[', End, Previous, ']'}, MaxNum - 1).\n\n%% @doc Works similar to foldl/3 but uses mnesia:foldl instead of our own implementation.\n%% The order in which will be iterated over is unspecified, but using this fuction\n%% might be faster than foldl/3 if it does not matter.\n-spec foldl_unordered(DB::db(), Fun::fun((Entry::entry(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldl_unordered(DB, Fun, Acc) ->\n    % Entry = {db, key, value}\n    FoldlFun = fun(Entry, AccIn) -> Fun(element(3, Entry), AccIn) end,\n\n    {atomic, Result}  = mnesia:transaction(fun() ->\n                                                mnesia:foldl(FoldlFun, Acc, DB)\n                                           end),\n    Result.\n\n\n-spec tab2list(Table_name::db()) -> [Entries::entry()].\ntab2list(Table_name) ->\n    Iterator =  fun({_DBName, _Key, Entry}, Acc)->\n                        [Entry | Acc]\n                end,\n    case mnesia:is_transaction() of\n        true -> mnesia:foldl(Iterator,[],Table_name);\n        false ->\n            Exec = fun({Fun,Tab}) -> mnesia:foldl(Fun, [],Tab) end,\n            mnesia:activity(sync_transaction,Exec,[{Iterator,Table_name}],mnesia_frag)\n    end.\n"
  },
  {
    "path": "src/db_prbr.erl",
    "content": "% @copyright 2013-2018 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc    DB back-end for DHT nodes storing prbr objects.\n%% @end\n%% @version $Id$\n-module(db_prbr).\n-author('schintke@zib.de').\n-author('kruber@zib.de').\n-author('fajerski@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n%% -define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n%% -define(TRACE_SNAP(X, Y), ct:pal(X, Y)).\n%% -define(TRACE_SNAP(X, Y), ?TRACE(X, Y)).\n-define(TRACE_SNAP(X, Y), ok).\n%% -define(TRACE_CHUNK(X, Y), ct:pal(X, Y)).\n-define(TRACE_CHUNK(X, Y), ok).\n\n-define(DB, (config:read(db_backend))). %% DB backend\n\n-define(CKETS, ets). %% changed keys database\n\n%% whole DB management\n-export([check_config/0]).\n-export([new/1, open/1]).\n-export([close/1, close_and_delete/1]).\n-export([get_load/1, get_load/2]).\n-export([tab2list/1]).\n-export([get_chunk2/7]).\n\n%% raw whole db entry operations\n-export([get/2]).\n-export([set/2]).\n\n-export([add_data/2]).\n-export([get_entries/2]).\n\n%% slide: delta recording (see dht_node_state.erl)\n-export([record_changes/2]).\n-export([stop_record_changes/2]).\n-export([get_changes/2]).\n-export([delete_entries/2]).\n-export([delete_entry/2]).\n\n-type db() :: {KeyValueDB  :: term(),\n               Subscribers :: db_ets:db(), %% for delta recording\n               SnaphotInfo :: {term() | false,\n                               LiveLockCount :: non_neg_integer(),\n                               SnapLockCount :: non_neg_integer()}}.\n-type version() :: non_neg_integer().\n-type value() :: rdht_tx:encoded_value().\n-type entry() :: tuple(). %% first element must be the key().\n-type db_as_list() :: [entry()].\n\n-type subscr_op_t() :: {write, entry()} | {delete | split, ?RT:key()}.\n-type subscr_changes_fun_t() :: fun((DB::db(), Tag::any(), Operation::subscr_op_t()) -> db()).\n-type subscr_remove_fun_t() :: fun((Tag::any()) -> any()).\n-type subscr_t() :: {Tag::any(), intervals:interval(), ChangesFun::subscr_changes_fun_t(), CloseDBFun::subscr_remove_fun_t()}.\n\n\n-export_type([db/0, value/0, version/0, db_as_list/0]).\n\n%%%%%%\n%%% whole DB management\n%%%%%%\n\n%% @doc Initializes a new database.\n-spec new(nonempty_string() | atom() | tuple()) -> db().\nnew(DBName) ->\n  DBNameNew = db_util:get_name(DBName),\n  SubscrName = db_util:get_subscriber_name(DBNameNew),\n  {?DB:new(DBNameNew), db_ets:new(SubscrName), {false, 0, 0}}.\n\n%% @doc Re-opens an existing database.\n-spec open(DB::nonempty_string()) -> db().\nopen(DBName) ->\n  DB = ?DB:open(DBName),\n  SubscrName = ?DB:get_name(DB) ++ \":subscribers\",\n  {DB, db_ets:new(SubscrName), {false, 0, 0}}.\n\n%% @doc Closes the given DB (it may be recoverable using open/1 depending on\n%%      the DB back-end).\n-spec close(db()) -> true.\nclose(State) ->\n    close(State, close).\n\n%% @doc Closes the given DB and deletes all contents (this DB can thus not be\n%%      re-opened using open/1).\n-spec close_and_delete(db()) -> true.\nclose_and_delete(State) ->\n    close(State, close_and_delete).\n\n%% @doc Helper for close/1 and close_and_delete/1.\n-spec close(db(), CloseFn::close | close_and_delete) -> true.\nclose({KVStore, Subscribers, {SnapDB, _LLC, _SLC}} = State, CloseFn) ->\n    _ = call_subscribers(State, close_db),\n    ?DB:CloseFn(KVStore),\n    db_ets:CloseFn(Subscribers),\n    case SnapDB of\n        false ->\n            ok;\n        ETSTable ->\n            ?DB:CloseFn(ETSTable)\n    end.\n\n%% @doc Returns the number of stored keys.\n-spec get_load(DB::db()) -> Load::integer().\nget_load({DB, _Subscr, _SnapState}) ->\n    ?DB:get_load(DB).\n\n%% @doc Returns the number of stored keys in the given interval.\n-spec get_load(DB::db(), Interval::intervals:interval()) -> Load::integer().\nget_load(State, [all]) -> get_load(State);\nget_load(_State, []) -> 0;\nget_load({DB, _Subscr, _Snap}, Interval) ->\n    ?DB:foldl(\n            DB,\n            fun(Key, AccIn) ->\n                case intervals:in(Key, Interval) of\n                    true -> AccIn + 1;\n                    _ -> AccIn\n                end\n            end, 0).\n\n-spec tab2list(Table_name::db()) -> [Entries::entry()].\ntab2list({DB, _Subscr, _Snap}) ->\n    ?DB:tab2list(DB).\n\n%%%%%%\n%%% raw whole db entry operations\n%%%%%%\n\n%% @doc Gets an entry from the DB. If there is no entry with the given key,\n%%      an empty entry will be returned.\n-spec get(db(), Key::?RT:key()) -> entry() | {}.\nget({KVStore, _Subscr, _Snap}, Key) ->\n    ?DB:get(KVStore, Key).\n\n-spec set(db(), entry()) -> db().\nset(DB, Entry) ->\n    set_entry(DB, Entry, 1, 2).\n\n-spec set_entry(db(), entry(), non_neg_integer(),\n                 non_neg_integer()) -> db().\nset_entry(State, Entry, _TLogSnapNo, _OwnSnapNo) ->\n    {KVStore, Subscr, Snap} = State,\n    %% set actual entry in DB\n    NewDB = {?DB:put(KVStore, Entry), Subscr, Snap},\n    call_subscribers(NewDB, {write, Entry}).\n\n%% @doc Removes all values with the given entry's key from the DB.\n-spec delete_entry(DB::db(), entry()) -> NewDB::db().\ndelete_entry(State, Entry) ->\n    delete_entry_at_key(State, entry_key(Entry)).\n\n-spec delete_entry_at_key(DB::db(), ?RT:key()) -> NewDB::db().\ndelete_entry_at_key(State, Key) ->\n    delete_entry_at_key(State, Key, delete).\n\n%% @doc Removes all values with the given key from the DB with specified reason.\n-spec delete_entry_at_key(DB::db(), ?RT:key(), delete | split) ->  NewDB::db().\ndelete_entry_at_key({DB, Subscr, {Snap, LiveLC, SnapLC}} = State,  Key, Reason) ->\n    %% TODO count locks\n    _OldEntry = get(State, Key),\n    %% @doc removes all entries from the DB that correspond to Key\n    call_subscribers({?DB:delete(DB, Key), Subscr, {Snap, LiveLC, SnapLC}}, {Reason, Key}).\n\n\n\n%% @doc Returns all key-value pairs of the given DB which are in the given\n%%      interval but at most ChunkSize elements.\n%%      Assumes the ets-table is an ordered_set,\n%%      may return data from \"both ends\" of the DB-range if the interval is\n%%      \"\"wrapping around\", i.e. its begin is larger than its end.\n%%      Returns the chunk and the remaining interval for which the DB may still\n%%      have data (a subset of I).\n%%      Precond: Interval is a subset of the range of the dht_node and continuous!\n-spec get_chunk(DB::db(), StartId::?RT:key(), Interval::intervals:interval(),\n                FilterFun::fun((entry()) -> boolean()),\n                ValueFun::fun((entry()) -> V),\n                ChunkSize::pos_integer() | all)\n        -> {intervals:interval(), [V]}.\nget_chunk(State, StartId, Interval, FilterFun, ValueFun, ChunkSize) ->\n    case intervals:is_empty(Interval) of\n        true -> {intervals:empty(), []};\n        false ->\n            case config:read(db_prbr_chunker) of\n                db_prbr ->\n                    get_chunk2(State, StartId, Interval,\n                               FilterFun, ValueFun, ChunkSize);\n                Module -> Module:get_chunk2(State, StartId, Interval,\n                                            FilterFun, ValueFun, ChunkSize)\n            end\n    end.\n\n%% @doc Helper for get_chunk/6.\n-spec get_chunk2(DB::db(), StartId::?RT:key(), Interval::intervals:interval(),\n                FilterFun::fun((entry()) -> boolean()),\n                ValueFun::fun((entry()) -> V),\n                ChunkSize::pos_integer() | all)\n        -> {intervals:interval(), [V]}.\nget_chunk2(State, StartId, Interval, FilterFun, ValueFun, all) ->\n    {_Next, Chunk} =\n        get_chunk2(State, StartId, Interval, FilterFun, ValueFun, get_load(State)),\n    {intervals:empty(), Chunk};\n\nget_chunk2({DB, Subscr, Snap}, StartId, Interval, FilterFun, ValueFun, ChunkSize) ->\n    AddDataFun = fun(Key, {Acc, RemainingChunkSize}) ->\n                     Entry = ?DB:get(DB, Key),\n                     case FilterFun(Entry) of\n                         true -> {[Entry | Acc],\n                                  RemainingChunkSize - 1};\n                         _    -> {Acc, RemainingChunkSize}\n                     end\n             end,\n    get_chunk2({DB, Subscr, Snap}, StartId, Interval, FilterFun, ValueFun, AddDataFun, ChunkSize).\n\n%%%%%%\n%% bulk entry operations on intervals or filter funs for replica\n%% repair and local use (for data slide)\n%%%%%%\n\n-spec get_chunk2(DB::db(), StartId::?RT:key(), Interval::intervals:interval(),\n                 FilterFun::fun((entry()) -> boolean()),\n                 ValueFun::fun((entry()) -> V),\n                 AddDataFun::fun((Key::?RT:key(), {Acc::[V], RemainingChunkSize::pos_integer()}) ->\n                                        {Acc::[V], RemainingChunkSize::pos_integer()}),\n                ChunkSize::pos_integer() | all)\n        -> {intervals:interval(), [V]}.\nget_chunk2({DB, _Subscr, _Snap}, StartId, Interval, _FilterFun, ValueFun, AddDataFun, ChunkSize) ->\n    %% split intervals in a way so that the first simple interval of After\n    %% either contains StartId or is the closest following after StartId\n    ?TRACE_CHUNK(\"get_chunk:~nStartID: ~p~nInterval:~p~nChunksize: ~p~n\",\n                 [StartId, Interval, ChunkSize]),\n    %% rotate and split intervals so that StartId is the first element looked at\n    {Before, After} = lists:splitwith(\n            fun(all) -> false;\n               ({Key}) ->\n                    StartId > Key;\n               ({_LBr, _L, R, _RBr}) ->\n                StartId > R\n            end, intervals:get_simple_intervals(Interval)),\n    RotatedInterval = case After of\n        [] -> Before;\n        [all] ->\n            [{'[', StartId, ?PLUS_INFINITY, ')'},\n             {'[', ?MINUS_INFINITY, StartId, ')'}];\n        [{_K} | _Rest] ->\n            After ++ Before;\n        [{LBr, L, R, RBr} | Tail] ->\n            case intervals:in(StartId, intervals:new(LBr, L, R, RBr)) of\n                true ->\n                    lists:append([[{'[', StartId, R, RBr}],\n                                  Tail, Before,\n                                  [{LBr, L, StartId, ')'}]]);\n                _ ->\n                    After ++ Before\n            end\n    end,\n    ?TRACE_CHUNK(\"get_chunk2: asking db:foldl to look in those intervals ~p~n\",\n                 [RotatedInterval]),\n    {Chunk, Remaining} = lists:foldl(\n        fun(I, {Acc, RemainingChunkSize}) ->\n            ?DB:foldl(DB, AddDataFun, {Acc, RemainingChunkSize}, I,\n                      RemainingChunkSize)\n        end, {[], ChunkSize}, RotatedInterval),\n    %% calculate the leftover interval and return\n    ?TRACE_CHUNK(\"StartId: ~p~nInterval: ~p~nChunkSize: ~p~nChunk: ~p~nRemaining: ~p~nOpen: ~p~n\",\n           [StartId, Interval, ChunkSize, Chunk, Remaining, calc_remaining_interval(StartId, Remaining, Chunk, Interval)]),\n    {calc_remaining_interval(StartId, Remaining, Chunk, Interval),\n     lists:foldl(\n            fun(E, AccIn) ->\n                [ValueFun(E) | AccIn] end,\n            [], Chunk)}.\n\n%%%%%%\n%%% slide: slide DB (see dht_node_state.erl)\n%%%%%%\n\n%% @doc Adds all db entry objects in the Data list.\n-spec add_data(DB::db(), db_as_list()) -> NewDB::db().\nadd_data(DB, Data) ->\n    lists:foldl(fun(Entry, DBAcc) ->\n                        case Entry of\n                            {} -> DBAcc;\n                            _ -> set(DBAcc, Entry)\n                        end\n                end, DB, Data).\n\n%% @doc Gets (non-empty) db entry objects in the given range.\n-spec get_entries(DB::db(), Range::intervals:interval()) -> db_as_list().\nget_entries(State, Interval) ->\n    {Elements, RestInterval} = intervals:get_elements(Interval),\n    case intervals:is_empty(RestInterval) of\n        true ->\n            [E || Key <- Elements, {} =/= (E = get(State, Key))];\n        _ ->\n            {_, Data} =\n                get_chunk(State, ?RT:hash_key(\"0\"), % any key will work, here!\n                           Interval,\n                           fun(DBEntry) -> {} =/= DBEntry end,\n                           fun(E) -> E end, all),\n            Data\n    end.\n\n\n%%%%%%\n%%% slide: delta recording (see dht_node_state.erl)\n%%%%%%\n\n%% @doc Adds the new interval to the interval to record changes for. Entries\n%%      which have (potentially) changed can then be gathered by get_changes/1.\n-spec record_changes(OldDB::db(), intervals:interval()) -> NewDB::db().\nrecord_changes(State, NewInterval) ->\n    RecChanges = get_subscription(State, record_changes),\n    ?TRACE(\"Old Subscription is ~p~n\", [RecChanges]),\n    NewSubscr =\n        case RecChanges of\n            [] -> {record_changes, NewInterval,\n                   fun subscr_delta/3, fun subscr_delta_close_table/1};\n            [{Tag, I, ChangesFun, RemSubscrFun}] ->\n                {Tag, intervals:union(I, NewInterval), ChangesFun, RemSubscrFun}\n        end,\n    ?TRACE(\"setting new subscription ~p~n\", [NewSubscr]),\n    set_subscription(State, NewSubscr).\n\n\n%% @doc Stops recording changes in the given interval and removes all such\n%%      entries from the table of changed keys.\n-spec stop_record_changes(OldDB::db(), intervals:interval()) -> NewDB::db().\nstop_record_changes(State, Interval) ->\n    RecChanges = get_subscription(State, record_changes),\n    case RecChanges of\n        [] -> State;\n        [{Tag, I, ChangesFun, RemSubscrFun}] ->\n            subscr_delta_remove(State, Interval),\n            NewI = intervals:minus(I, Interval),\n            case intervals:is_empty(NewI) of\n                true -> remove_subscription(State, Tag);\n                _ -> set_subscription(State, {Tag, NewI, ChangesFun, RemSubscrFun})\n            end\n    end.\n\n\n%% @doc Gets all db entries in the given interval which have\n%% (potentially) been changed or deleted (might return objects that\n%% have not changed but have been touched by one of the DB setters).\n-spec get_changes(DB::db(), intervals:interval()) -> {Changed::db_as_list(), Deleted::[?RT:key()]}.\nget_changes(State, Interval) ->\n    case erlang:get('$delta_tab') of\n        undefined -> get_changes_helper(State, [], Interval, [], []);\n        CKDB -> get_changes_helper(State, ?CKETS:tab2list(CKDB), Interval, [], [])\n    end.\n\n%% @doc Helper for get_changes/2 that adds the entry of a changed key either to\n%%      the list of changed entries or to the list of deleted entries.\n-spec get_changes_helper(State::db(), ChangedKeys::[{?RT:key()}],\n        Interval::intervals:interval(), ChangedEntries::[entry()],\n        DeletedKeys::[?RT:key()])\n            -> {ChangedEntries::[entry()], DeletedKeys::[?RT:key()]}.\nget_changes_helper(_State, [], _Interval, ChangedEntries, DeletedKeys) ->\n    {ChangedEntries, DeletedKeys};\nget_changes_helper(State, [{CurKey} | RestKeys], Interval, ChangedEntries, DeletedKeys) ->\n    case intervals:in(CurKey, Interval) of\n        true ->\n            Entry = get(State, CurKey),\n            case Entry of\n                {} ->\n                    ?TRACE(\"~p get_changes: ~p was deleted~n\", [self(), CurKey]),\n                    get_changes_helper(State, RestKeys, Interval, ChangedEntries, [CurKey | DeletedKeys]);\n                _ ->\n                    ?TRACE(\"~p get_changes: ~p was changed~n\", [self(), CurKey]),\n                    get_changes_helper(State, RestKeys, Interval, [Entry | ChangedEntries], DeletedKeys)\n            end;\n        _ -> ?TRACE(\"~p get_changes: key ~p is not in ~p~n\", [self(), CurKey, Interval]),\n             get_changes_helper(State, RestKeys, Interval, ChangedEntries, DeletedKeys)\n    end.\n\n%% @doc Inserts/removes the key into the table of changed keys depending on the\n%%      operation (called whenever the DB is changed).\n-spec subscr_delta(State::db(), Tag::any(), Operation::subscr_op_t()) -> db().\nsubscr_delta(State, _Tag, Operation) ->\n    CKDB = subscr_delta_check_table(State),\n    ?TRACE(\"subscr_delta is called for op ~p~n\", [Operation]),\n    case Operation of\n        {write, Entry} -> ?CKETS:insert(CKDB, {entry_key(Entry)});\n        {delete, Key}  -> ?CKETS:insert(CKDB, {Key});\n        {split, Key}   -> ?CKETS:delete(CKDB, Key)\n    end,\n    State.\n\n%% @doc Cleans up, i.e. deletes, the table with changed keys (called on\n%%      subscription removal).\n-spec subscr_delta_close_table(Tag::any()) -> ok | true.\nsubscr_delta_close_table(_Tag) ->\n    case erlang:erase('$delta_tab') of\n        undefined -> ok;\n        CKDB -> ?CKETS:delete(CKDB)\n    end.\n\n%% @doc Check that the table storing changed keys exists and creates it if\n%%      necessary.\n-spec subscr_delta_check_table(State::db()) -> ets:tid() | atom().\nsubscr_delta_check_table(_State) ->\n    DeltaDB = case erlang:get('$delta_tab') of\n        undefined ->\n            CKDB = ?CKETS:new(dht_node_db_ck, [ordered_set | ?DB_ETS_ADDITIONAL_OPS]),\n            erlang:put('$delta_tab', CKDB),\n            CKDB;\n        CKDB -> CKDB\n    end,\n    DeltaDB.\n\n%% @doc Removes any changed key in interval I (called when some (sub-)interval\n%%      is unsubscribed).\n-spec subscr_delta_remove(State::db(), I::intervals:interval()) -> ok.\nsubscr_delta_remove(State, Interval) ->\n    CKDB = subscr_delta_check_table(State),\n    F = fun(DBEntry, _) ->\n                Key = entry_key(DBEntry),\n                case intervals:in(Key, Interval) of\n                    true -> ?CKETS:delete(CKDB, Key);\n                    _    -> true\n                end\n        end,\n    ?CKETS:foldl(F, true, CKDB),\n    ok.\n\n%% @doc Deletes all objects in the given Range or (if a function is provided)\n%%      for which the FilterFun returns true from the DB.\n-spec delete_entries(DB::db(),\n                     RangeOrFun::intervals:interval() |\n                                 fun((entry()) -> boolean()))\n        -> NewDB::db().\ndelete_entries(State = {DB, _Subscr, _SnapState}, FilterFun)\n  when is_function(FilterFun) ->\n    F = fun(Key, StateAcc) ->\n                DBEntry = ?DB:get(DB, Key),\n                case FilterFun(DBEntry) of\n                    false -> StateAcc;\n                    _     -> delete_entry(StateAcc, DBEntry)\n                end\n        end,\n    ?DB:foldl(DB, F, State);\ndelete_entries({DB, _Subscr, _SnapState} = State, Interval) ->\n    {Elements, RestInterval} = intervals:get_elements(Interval),\n    case intervals:is_empty(RestInterval) of\n        true ->\n            lists:foldl(fun(Key, State1) ->\n                                delete_entry_at_key(State1, Key)\n                        end, State, Elements);\n        _ ->\n            F = fun(Key, StateAcc) ->\n                        Entry = ?DB:get(DB, Key),\n                        delete_entry(StateAcc, Entry)\n                end,\n            SimpleI = intervals:get_simple_intervals(Interval),\n            lists:foldl(fun(I, AccIn) ->\n                                ?DB:foldl(DB, F, AccIn, I)\n                        end, State, SimpleI)\n    end.\n\n%%%%%%\n%%% end slide: delta recording (see dht_node_state.erl)\n%%%%%%\n\n%%%%%%\n%%% subscriptions to DB changes (called locally by delta recording)\n%%%%%%\n\n%% @doc Adds a subscription for the given interval under Tag (overwrites an\n%%     existing subscription with that tag).\n-spec set_subscription(State::db(), subscr_t()) -> db().\nset_subscription({DB, Subscr, SnapState}, Subscription) ->\n    {DB, db_ets:put(Subscr, Subscription), SnapState}.\n\n%% @doc Gets a subscription stored under Tag (empty list if there is none).\n-spec get_subscription(State::db(), Tag::any()) -> [subscr_t()].\nget_subscription({_DB, Subscr, _SnapState}, Tag) ->\n    case db_ets:get(Subscr, Tag) of\n        {} ->\n            [];\n        SubsT ->\n            [SubsT]\n    end.\n\n%% @doc Removes a subscription stored under Tag (if there is one).\n-spec remove_subscription(State::db(), Tag::any()) -> db().\nremove_subscription({DB, Subscr, SnapState}, Tag) ->\n    case db_ets:get(Subscr, Tag) of\n        {} -> ok;\n        {Tag, _I, _ChangesFun, RemSubscrFun} -> RemSubscrFun(Tag)\n    end,\n    {DB, db_ets:delete(Subscr, Tag), SnapState}.\n\n%% @doc Go through all subscriptions and perform the given operation if\n%%      matching.\n-spec call_subscribers(State::db(), Operation::close_db | subscr_op_t()) -> db().\ncall_subscribers(State = {_DB, Subscr, _SnapState}, Operation) ->\n    {NewState, _Op} = db_ets:foldl(Subscr,\n              fun call_subscribers_iter/2,\n              {State, Operation}),\n    NewState.\n\n%% @doc Iterates over all susbcribers and calls their subscribed functions.\n-spec call_subscribers_iter(subscr_t(),\n                            {State::db(), Operation::close_db | subscr_op_t()})\n        -> {db(), Operation::close_db | subscr_op_t()}.\ncall_subscribers_iter(Tag, {{_DB, Subscr, _SnapState} = State, Op}) ->\n    % assume the key exists (it should since we are iterating over the table!)\n    {Tag, I, ChangesFun, RemSubscrFun} = db_ets:get(Subscr, Tag),\n    NewState =\n        case Op of\n            close_db ->\n                RemSubscrFun(Tag),\n                State;\n            Operation ->\n                Key = case Operation of\n                    {write, Entry} -> entry_key(Entry);\n                    {delete, K}  -> K;\n                    {split, K}  -> K\n                end,\n                case intervals:in(Key, I) of\n                    false ->\n                        ?TRACE(\"not calling subscribers...~p not in interval ~p~n\",\n                               [Key, I]),\n                        State;\n                    _     ->\n                        ?TRACE(\"calling subscriber for tag ~p and op ~p~n\", [Tag,\n                                                                             Operation]),\n                        ChangesFun(State, Tag, Operation)\n                end\n        end,\n    {NewState, Op}.\n\n\n\n%%%%%%\n%%% for unittests\n%%%%%%\n\n%%%%%%\n%%% locally used helper functions to long to place inside\n%%% corresponding context block\n%%%%%%\n\n-spec calc_remaining_interval(?RT:key(), non_neg_integer(), db_as_list(),\n                              intervals:interval()) -> intervals:interval().\n%% if there are less elements in Chunk that ChunkSize allows, the whole interval\n%% was covered\ncalc_remaining_interval(_StartId, Remaining, _Chunk, _Interval)\n        when Remaining > 0 -> intervals:empty();\n%% if Chunk is empty the whole Interval was covered\ncalc_remaining_interval(_StartId, _Remaining, [], _Interval) ->\n    intervals:empty();\ncalc_remaining_interval(StartId, _Remaining, Chunk, Interval) ->\n    %% the interval covered by chunk is either the biggest key left of startid\n    %% or if there are no keys left of startid simply the biggest key in chunk\n    Last = calc_last_key_rem_int(Chunk, StartId),\n    ?TRACE_CHUNK(\"left: ~p~nright: ~p~ncalc_remaining:~n~p~nNext is ~p minus ~p\",\n                 [Left, Right, Chunk, Interval, intervals:new('[', StartId, Last, ']')]),\n    intervals:minus(Interval, intervals:new('[', StartId, Last, ']')).\n\n%% @doc Gets the largest key in Chunk left of StartId if there is one,\n%%      otherwise gets the largest key of all items.\n-spec calc_last_key_rem_int(Chunk::[entry(),...], StartId::?RT:key()) -> ?RT:key().\ncalc_last_key_rem_int([X | Rest], StartId) ->\n    Key = entry_key(X),\n    calc_last_key_rem_int(Rest, StartId, Key, Key < StartId).\n\n%% @doc Helper for calc_last_key_rem_int/2.\n-spec calc_last_key_rem_int(db_as_list(), StartId::?RT:key(), Max::?RT:key(),\n                            OnlySmallerThanStart::boolean()) -> ?RT:key().\ncalc_last_key_rem_int([], _StartId, Max, _OnlySmallerThanStart) -> Max;\ncalc_last_key_rem_int([X | Rest], StartId, Max, true) ->\n    Key = entry_key(X),\n    case Key < StartId of\n        true ->\n            calc_last_key_rem_int(Rest, StartId,\n                                  ?IIF(Key > Max, Key, Max), true);\n        false ->\n            calc_last_key_rem_int(Rest, StartId, Max, true)\n    end;\ncalc_last_key_rem_int([X | Rest], StartId, Max, false) ->\n    Key = entry_key(X),\n    case Key < StartId of\n        true ->\n            calc_last_key_rem_int(Rest, StartId, Key, true);\n        false ->\n            calc_last_key_rem_int(Rest, StartId,\n                                  ?IIF(Key > Max, Key, Max), false)\n    end.\n\nentry_key(E) -> element(1, E).\n\n-spec check_config() -> boolean().\ncheck_config() ->\n    All_DBs = [db_ets, db_mnesia, db_toke, db_hanoidb, db_bitcask],\n    Current_DB = config:read(db_backend),\n\n    config:cfg_is_module(db_backend) andalso\n    config:cfg_is_in(db_backend, All_DBs) andalso\n    case Current_DB:is_available() of\n        true ->\n            true;\n        Missing ->\n            error_logger:error_msg(\"Modules ~p for selected DB backend ~p are missing.~n\",\n                                   [Missing, Current_DB]),\n            false\n    end andalso\n    case config:read(ensure_recover) of\n        true ->\n            % ensure_recovery is enabled which means we must check if\n            % leases are enabled and the chosen DB supports recovery\n            case config:read(leases) of\n                true ->\n                    true;\n                _ ->\n                    error_logger:error_msg(\n                      \"Ensure_recover enabled but option leases not true \"\n                      \"(see scalaris.cfg and scalaris.local.cfg).~n\"\n                      \" Leases must be defined and set to true.~n\"),\n                    false\n            end and\n            case Current_DB:supports_feature(recover) of\n                true ->\n                   true;\n                _ ->\n                    % get DBs which have recovery and are installed\n                    Rec_DBs = [DB || DB <- All_DBs,\n                                     DB:supports_feature(recover),\n                                     DB:is_available()],\n                    error_logger:error_msg(\n                      \"Ensure_recover enabled but selected DB backend (~p) does not \"\n                      \"support recovery.~n Try one of the following: ~p \"\n                      \"(see scalaris.cfg and scalaris.local.cfg).~n\",\n                      [Current_DB, Rec_DBs]),\n                    false\n            end;\n        _ ->\n            % ensure recovery is not enabled and thus no checks are needed\n            true\n    end.\n"
  },
  {
    "path": "src/db_toke.erl",
    "content": "% @copyright 2013-2014 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc    DB back-end using tokyo cabinet via toke.\n%%         Two keys K and L are considered equal if they match, i.e. K =:= L\n%%\n%%         To use this backend you have to install\n%%         Tokyo Cabinet [http://fallabs.com/tokyocabinet] and\n%%         Toke [http://hg.opensource.lshift.net/toke/] (a simple Erlang\n%%         wrapper for Tokyo Cabinets hash API tchdb*).\n%%         For development Tokyo Cabinet V1.4.48-1 and the at-the-time\n%%         latest commit to the toke-repository (f178e55bb6b5) was used.\n%%\n%%         After building and installing Tokyo Cabinet and toke\n%%         rerun configure with --enable-toke. configure assumes that you\n%%         installed toke in your erlang's lib directory, i.e.\n%%         `<erlang_dir>/lib/toke' or `<erlang_dir>/lib/toke-<version>'. If you\n%%         used a different directory, e.g. /home/scalaris/apps/toke, you have\n%%         to provide the path to configure:\n%%\n%%         `./configure --enable-toke=/home/scalaris/apps/toke/'\n%%\n%%         Rerun make and you can use db_toke.\n%% @end\n%% @version $Id$\n-module(db_toke).\n-author('fajerski@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n-behaviour(db_backend_beh).\n\n-define(TRACE(_X, _Y), ok).\n%% -define(TRACE(X, Y), ct:pal(X, Y)).\n\n-define(IN(E), erlang:term_to_binary(E, [{minor_version, 1}])).\n-define(OUT(E), erlang:binary_to_term(E)).\n\n%% primitives\n-export([new/1, open/1, close/1, close_and_delete/1,\n         put/2, get/2, delete/2]).\n%% db info\n-export([get_persisted_tables/0, get_name/1, get_load/1, \n         is_available/0, supports_feature/1]).\n\n%% iteration\n-export([foldl/3, foldl/4, foldl/5]).\n-export([foldr/3, foldr/4, foldr/5]).\n-export([foldl_unordered/3]).\n-export([tab2list/1]).\n\n-type db() :: {DB::pid(), DBName::nonempty_string()}.\n-type key() :: db_backend_beh:key(). %% '$end_of_table' is not allowed as key() or else iterations won't work!\n-type entry() :: db_backend_beh:entry().\n\n-export_type([db/0]).\n\n%% @doc Creates new DB handle named DBName.\n-spec new(DBName::nonempty_string()) -> db().\nnew(DBName) ->\n    new_db(DBName, [read, write, create, truncate]).\n\n%% @doc Open a previously existing database.\n-spec open(DBName::nonempty_string()) -> db().\nopen(DBName) ->\n    new_db(DBName, [read, write]).\n\n-spec new_db(DBName::nonempty_string(),\n             TokeOptions::[read | write | create | truncate | no_lock |\n                           lock_no_block | sync_on_transaction]) -> db().\nnew_db(DBName, TokeOptions) ->\n    FullDir = [config:read(db_directory), \"/\", atom_to_list(node())],\n    _ = case file:make_dir(FullDir) of\n        ok -> ok;\n        {error, eexist} -> ok;\n        {error, Error0} -> erlang:exit({?MODULE, 'cannot create dir', FullDir, Error0})\n    end,\n    FileBaseName = util:make_filename(io_lib:format(\"~s.tch\", [DBName])),\n    FullFileName = lists:flatten([FullDir, \"/\", FileBaseName]),\n    DB = case toke_drv:start_link() of\n        {ok, Pid} -> Pid;\n        ignore ->\n            log:log(error, \"[ Node ~w:~w ] process start returned\n                    'ignore'\", [self(), ?MODULE]),\n            erlang:error({toke_failed, drv_start_ignore});\n        {error, Error} ->\n            log:log(error, \"[ Node ~w:~w ] ~.0p\", [self(), ?MODULE, Error]),\n            erlang:error({toke_failed, Error})\n    end,\n    case toke_drv:new(DB) of\n        ok ->\n            case toke_drv:open(DB, FullFileName, TokeOptions) of\n                ok -> {DB, DBName};\n                Error2 ->\n                    log:log(error, \"[ Node ~w:~w ] ~.0p\", [self(), ?MODULE, Error2]),\n                    erlang:error({toke_failed, Error2})\n            end;\n        Error1 ->\n            log:log(error, \"[ Node ~w:~w ] ~.0p\", [self(), ?MODULE, Error1]),\n            erlang:error({toke_failed, Error1})\n    end.\n\n%% @doc Closes the DB named DBName\n-spec close(DB::db()) -> true.\nclose({DB, _DBName}) ->\n    toke_drv:close(DB),\n    toke_drv:delete(DB),\n    toke_drv:stop(DB).\n\n%% @doc Closes and deletes the DB named DBName\n-spec close_and_delete(DB::db()) -> true.\nclose_and_delete({_DB, DBName} = State) ->\n    close(State),\n    FileName = [config:read(db_directory), \"/\", atom_to_list(node()), \"/\",\n                DBName, \".tch\"],\n    case file:delete(FileName) of\n        ok -> ok;\n        {error, Reason} ->\n            log:log(error, \"[ Node ~w:~w ] deleting ~.0p failed: ~.0p\",\n                    [self(), ?MODULE, FileName, Reason])\n    end.\n\n%% @doc Gets a list of persisted tables.\n-spec get_persisted_tables() -> [nonempty_string()].\nget_persisted_tables() ->\n    FullDir = [config:read(db_directory), \"/\", atom_to_list(node())],\n    case file:list_dir(FullDir) of\n        {ok, Files} ->\n            [lists:sublist(File, length(File) - 4)\n            || File <- Files, lists:suffix(\".tch\", File)];\n        {error, enoent} -> []\n    end.\n\n\n%% @doc Saves arbitrary tuple Entry in DB DBName and returns the new DB.\n%%      The key is expected to be the first element of Entry.\n-spec put(DB::db(), Entry::entry()) -> db().\nput({DB, _DBName} = State, Entry) ->\n    toke_drv:insert(DB, ?IN(element(1, Entry)), ?IN(Entry)),\n    State.\n\n%% @doc Returns the entry that corresponds to Key or {} if no such tuple exists.\n-spec get(DB::db(), Key::key()) -> entry() | {}.\nget({DB, _DBName}, Key) ->\n    case toke_drv:get(DB, ?IN(Key)) of\n        not_found -> {};\n        Entry -> ?OUT(Entry)\n    end.\n\n%% @doc Deletes the tuple saved under Key and returns the new DB.\n%%      If such a tuple does not exists nothing is changed.\n-spec delete(DB::db(), Key::key()) -> db().\ndelete({DB, _DBName} = State, Key) ->\n    toke_drv:delete(DB, ?IN(Key)),\n    State.\n\n%% @doc Returns the name of the DB specified in @see new/1.\n-spec get_name(DB::db()) -> nonempty_string().\nget_name({_DB, DBName}) ->\n    DBName.\n\n%% @doc Checks for modules required for this DB backend. Returns true if no \n%%      modules are missing, or else a list of missing modules\n-spec is_available() -> boolean() | [atom()].\nis_available() ->\n    case code:which(toke_drv) of\n        non_existing -> [toke_drv];\n        _ -> true\n    end.\n\n%% @doc Returns true if the DB support a specific feature (e.g. recovery), false otherwise.\n-spec supports_feature(Feature::atom()) -> boolean().\nsupports_feature(recover) -> true;\nsupports_feature(_) -> false.\n\n%% @doc Returns the current load (i.e. number of stored tuples) of the DB.\n-spec get_load(DB::db()) -> non_neg_integer().\nget_load({DB, _DBName}) ->\n    %% TODO: not really efficient (maybe store the load in the DB?)\n    toke_drv:fold(fun (_K, _V, Load) -> Load + 1 end, 0, DB).\n\n%% @doc Equivalent to toke_drv:fold(Fun, Acc0, DB).\n%%      Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldl(State, Fun, Acc) ->\n    foldl_helper(State, Fun, Acc, all, -1).\n\n%% @equiv foldl(DB, Fun, Acc0, Interval, get_load(DB))\n%% @doc   Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Interval::db_backend_beh:interval()) -> Acc1::A.\nfoldl(State, Fun, Acc, Interval) ->\n    foldl_helper(State, Fun, Acc, Interval, -1).\n\n%% @doc foldl iterates over DB and applies Fun(Entry, AccIn) to every element\n%%      encountered in Interval. On the first call AccIn == Acc0. The iteration\n%%      stops as soon as MaxNum elements have been encountered.\n%%      Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldl(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Intervall::db_backend_beh:interval(), MaxNum::non_neg_integer()) -> Acc1::A.\nfoldl(State, Fun, Acc, Interval, MaxNum) ->\n    %% HINT\n    %% Fun can only be applied in a second pass. It could do a delete (or other\n    %% write op) and toke can not handle writes whiles folding.\n    %% Since we reversed the order while accumulating reverse it by using lists\n    %% fold but \"from the other side\"\n    foldl_helper(State, Fun, Acc, Interval, MaxNum).\n\n%% @private this helper enables us to use -1 as MaxNum. MaxNum == -1 signals that all\n%%          data is to be retrieved.\n-spec foldl_helper(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Intervall::db_backend_beh:interval(), MaxNum::integer()) -> Acc1::A.\nfoldl_helper({DB, _DBName}, Fun, Acc, Interval, MaxNum) ->\n    Keys = get_all_keys(DB, Interval, MaxNum),\n    lists:foldr(Fun, Acc, Keys).\n\n%% @doc makes a foldr over the whole dataset.\n%%      Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldr(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldr(State, Fun, Acc) ->\n    foldr_helper(State, Fun, Acc, all, -1).\n\n%% @equiv foldr(DB, Fun, Acc0, Interval, get_load(DB))\n%% @doc   Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldr(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Interval::db_backend_beh:interval()) -> Acc1::A.\nfoldr(State, Fun, Acc, Interval) ->\n    foldr_helper(State, Fun, Acc, Interval, -1).\n\n%% @doc foldr iterates over DB and applies Fun(Entry, AccIn) to every element\n%%      encountered in Interval. On the first call AccIn == Acc0. The iteration\n%%      stops as soon as MaxNum elements have been encountered.\n%%      Returns a potentially larger-than-memory dataset. Use with care.\n-spec foldr(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Intervall::db_backend_beh:interval(), MaxNum::non_neg_integer()) -> Acc1::A.\nfoldr(State, Fun, Acc, Interval, MaxNum) ->\n    foldr_helper(State, Fun, Acc, Interval, MaxNum).\n\n%% @private this helper enables us to use -1 as MaxNum. MaxNum == -1 signals that all\n%%          data is to be retrieved.\n-spec foldr_helper(DB::db(), Fun::fun((Key::key(), AccIn::A) -> AccOut::A), Acc0::A,\n                               Intervall::db_backend_beh:interval(), MaxNum::integer()) -> Acc1::A.\nfoldr_helper({DB, _DBName}, Fun, Acc, Interval, MaxNum) ->\n    %% first only retrieve keys so we don't have to load the whole db into memory\n    Keys = get_all_keys(DB, Interval, -1),\n    CutData = case MaxNum of\n                  N when N < 0 ->\n                      Keys;\n                  _ ->\n                      lists:sublist(Keys, MaxNum)\n              end,\n    %% see HINT in foldl/5\n    %% now retrieve actual data\n    lists:foldl(Fun, Acc, CutData).\n\n%% @doc Works similar to foldl/3 but uses toke_drv:fold instead of our own implementation. \n%% The order in which will be iterated over is unspecified, but using this fuction\n%% might be faster than foldl/3 if it does not matter.\n-spec foldl_unordered(DB::db(), Fun::fun((Entry::entry(), AccIn::A) -> AccOut::A), Acc0::A) -> Acc1::A.\nfoldl_unordered(State, Fun, Acc) ->\n        %TODO Use native fold\n        foldl(State, fun(K, AccIn) -> Fun(get(State, K), AccIn) end, Acc).\n\n%% @private get_all_keys/3 retrieves all keys in DB that fall into Interval but\n%%          not more than MaxNum. If MaxNum == -1 all Keys are retrieved. If\n%%          MaxNum is positive it starts from the left in term order.\n-spec get_all_keys(pid(), db_backend_beh:interval(), -1 | non_neg_integer()) ->\n    [key()].\nget_all_keys(DB, Interval, MaxNum) ->\n    Keys = toke_drv:fold(fun(Key, _Entry, AccIn) ->\n                              [?OUT(Key) | AccIn]\n                         end, [], DB),\n    {_, In} = lists:foldl(fun(_, {0, _} = AccIn) ->\n                                  AccIn;\n                             (Key, {Max, KeyAcc} = AccIn) ->\n                          case is_in(Interval, Key) of\n                              true ->\n                                  {Max - 1, [Key | KeyAcc]};\n                              _ ->\n                                  AccIn\n                          end\n                end, {MaxNum, []}, lists:sort(Keys)),\n    In.\n\n\nis_in({Key}, OtherKey) -> Key =:= OtherKey;\nis_in(all, _Key) -> true;\nis_in({'(', L, R, ')'}, Key) -> Key > L andalso Key < R;\nis_in({'(', L, R, ']'}, Key) -> Key > L andalso ((Key < R) orelse (Key =:= R));\nis_in({'[', L, R, ')'}, Key) -> ((Key > L) orelse (Key =:= L)) andalso Key < R;\nis_in({'[', L, R, ']'}, Key) -> ((Key > L) orelse (Key =:= L)) andalso\n                                          ((Key < R) orelse (Key =:= R)).\n%% @doc Returns a list of all objects in the table Table_name.\n-spec tab2list(Table_name::db()) -> [Entries::entry()].\ntab2list(_Table_name) ->\n    %% Not implemented yet.\n    [].\n"
  },
  {
    "path": "src/db_util.erl",
    "content": "% @copyright 2015 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    DB back-end utilities.\n%% @end\n%% @version $Id$\n-module(db_util).\n-author('schuett@zib.de').\n-author('kruber@zib.de').\n\n-include(\"scalaris.hrl\").\n\n-export([get_name/1, get_subscriber_name/1,\n         get_recoverable_dbs/0, parse_table_name/1]).\n\n%% @doc Initializes a new database.\n-spec get_name(DBName::nonempty_string() | atom() | tuple()) -> nonempty_string().\nget_name(DBName) when is_atom(DBName) ->\n    get_name(erlang:atom_to_list(DBName));\nget_name(DBName = {Name, Id}) when is_tuple(DBName) ->\n    get_name(erlang:atom_to_list(Name) ++ \"-\" ++ erlang:integer_to_list(Id));\nget_name(DBName) ->\n    ?DBG_ASSERT(not lists:member($+, DBName)),\n    RandomName = randoms:getRandomString(),\n    DBName ++ \"+\" ++ pid_groups:group_to_filename(pid_groups:my_groupname())\n        ++ \"+\" ++ RandomName.\n\n-spec get_subscriber_name(DBName::nonempty_string()) -> nonempty_string().\nget_subscriber_name(DBName) ->\n    ?DBG_ASSERT(not lists:member($#, DBName)),\n    DBName ++ \"#subscribers\".\n\n-spec get_recoverable_dbs()\n        -> [{DB_type::nonempty_string(), PID_group::pid_groups:groupname(), DB_name::nonempty_string()}].\nget_recoverable_dbs() ->\n    Tables = (config:read(db_backend)):get_persisted_tables(),\n    %% io:format(\"tables list: ~w~n\", [Tables]),\n    %% creating tuples with DB_names different parts : {DB_type, PID_group, DB_name}\n    [parse_table_name(Table) || Table <- Tables].\n\n-spec parse_table_name(Table::nonempty_string() | atom())\n        -> {DB_type::nonempty_string(), PID_group::pid_groups:groupname(), Table::nonempty_string()}.\nparse_table_name(Table) when is_atom(Table) ->\n    parse_table_name(erlang:atom_to_list(Table));\nparse_table_name(Table) ->\n    {string:sub_word(Table, 1, $+),\n     pid_groups:filename_to_group(string:sub_word(Table, 2, $+)),\n     Table}.\n"
  },
  {
    "path": "src/dc_centroids.erl",
    "content": "%  @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Magnus Mueller <mamuelle@informatik.hu-berlin.de>\n%% @doc    Centroids data structure for dc_clustering.\n%% @end\n%% @reference T. Schuett, A. Reinefeld,F. Schintke, M. Hoffmann.\n%% Gossip-based Topology Inference for Efficient Overlay Mapping on Data Centers.\n%% 9th Int. Conf. on Peer-to-Peer Computing Seattle, Sept. 2009.\n%% @version $Id$\n-module(dc_centroids).\n-author('mamuelle@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-export([\n        new/2\n        , empty_centroids_list/0\n\n        %% getters\n        , get_coordinate/1\n        , get_relative_size/1\n        , get_coordinate_and_relative_size/1\n\n        %% setters\n        , set_relative_size/2\n\n        %% helpers\n        , distance/2\n    ]).\n\n-type(coordinate() :: gossip_vivaldi:network_coordinate()).\n-type(relative_size() :: float()).\n\n-record(centroid,\n    {\n        coordinate = [] :: coordinate()\n        , relative_size = 1.0 :: relative_size()\n    }).\n\n-type(centroid() :: #centroid{}).\n-type(centroids() :: [centroid()]).\n\n-export_type([centroid/0, centroids/0]).\n\n% @doc Create a new centroid.\n-spec new(coordinate(), relative_size()) -> centroid().\nnew(Coordinate, RelativeSize) ->\n    #centroid{coordinate=Coordinate,relative_size=RelativeSize}.\n\n% @doc Get centroid's coordinate\n-spec get_coordinate(Centroid :: centroid()) -> coordinate().\nget_coordinate(#centroid{coordinate=Coordinate}) -> Coordinate.\n\n% @doc Get centroid's relative size\n-spec get_relative_size(Centroid :: centroid()) -> relative_size().\nget_relative_size(#centroid{relative_size=RelativeSize}) -> RelativeSize.\n\n% @doc Get a centroid's coordinate and relative size as a tuple\n-spec get_coordinate_and_relative_size(Centroid :: centroid())\n            -> {coordinate(), relative_size()}.\nget_coordinate_and_relative_size(#centroid{coordinate=Coordinate, relative_size=RelativeSize}) ->\n    {Coordinate, RelativeSize}.\n\n% @doc Set the relative size of a centroid\n-spec set_relative_size(Centroid :: centroid(), relative_size()) -> centroid().\nset_relative_size(#centroid{coordinate=Coordinate}, RelativeSize) ->\n    #centroid{coordinate=Coordinate, relative_size=RelativeSize}.\n\n%% @doc Helper to return an empty list of centroids.\n-spec empty_centroids_list() -> centroids().\nempty_centroids_list() -> [].\n\n%% @doc Get the distance between two centroids\n-spec distance(U :: centroid(), V :: centroid()) -> float().\ndistance(U,V) ->\n    mathlib:euclideanDistance(get_coordinate(U), get_coordinate(V)).\n\n"
  },
  {
    "path": "src/dc_clustering.erl",
    "content": "%  @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Marie Hoffmann <hoffmann@zib.de>\n%% @doc    Re-register with boot nodes\n%% @end\n%% @reference T. Schuett, A. Reinefeld,F. Schintke, M. Hoffmann.\n%% Gossip-based Topology Inference for Efficient Overlay Mapping on Data Centers.\n%% 9th Int. Conf. on Peer-to-Peer Computing Seattle, Sept. 2009.\n%% @version $Id$\n-module(dc_clustering).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n-include(\"scalaris.hrl\").\n\n-export([start_link/1]).\n\n-export([init/1, on_inactive/2, on_active/2,\n         activate/0, deactivate/0,\n         check_config/0]).\n\n-include(\"gen_component.hrl\").\n\n-record(state_active,{\n        centroids = [] :: dc_centroids:centroids()\n        , local_epoch = 0 :: non_neg_integer()\n        , radius :: float()\n}).\n\n-record(state_inactive,{\n        queued_messages :: msg_queue:msg_queue()\n        , radius :: float()\n}).\n\n% state of the clustering loop\n-type(state_active() :: #state_active{}).\n-type(state_inactive() :: #state_inactive{}).\n%% -type(state() :: state_active() | state_inactive()).\n\n% accepted messages of an initialized dc_clustering process\n-type(message() ::\n    {dc_trigger} |       %% just for periodic wake up\n    {dc_reset_trigger} | %% just for periodic wake up\n    {start_clustering_shuffle} | %% start a single clustering shuffle\n    {reset_clustering} | %% get the own vivaldi coordinate and reset\n    {vivaldi_get_coordinate_response, gossip_vivaldi:network_coordinate(), gossip_vivaldi:est_error()} |\n    {cy_cache, [node:node_type()]} |\n    {clustering_shuffle, comm:mypid(), dc_centroids:centroids(), non_neg_integer()} |\n    {clustering_shuffle_reply, comm:mypid(), dc_centroids:centroids(), non_neg_integer()} |\n    {query_clustering, comm:mypid()} |\n    {query_epoch, comm:mypid()} |\n    {query_my, atom(), comm:mypid()}\n).\n\n%% @doc Sends an initialization message to the node's dc_clustering process.\n-spec activate() -> ok | ignore.\nactivate() ->\n    case config:read(dc_clustering_enable) of\n        true ->\n            Pid = pid_groups:get_my(dc_clustering),\n            comm:send_local(Pid, {activate_clustering});\n        _ ->\n            ignore\n    end.\n\n%% @doc Deactivates the clustering process.\n-spec deactivate() -> ok | ignore.\ndeactivate() ->\n    case config:read(dc_clustering_enable) of\n        true ->\n            Pid = pid_groups:get_my(dc_clustering),\n            comm:send_local(Pid, {deactivate_clustering});\n        _ ->\n            ignore\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Init\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Starts the dc_clustering process, registers it with the process\n%%      dictionary and returns its pid for use by a supervisor.\n-spec start_link(pid_groups:groupname()) -> {ok, pid()} | ignore.\nstart_link(DHTNodeGroup) ->\n    case config:read(dc_clustering_enable) of\n        true ->\n            gen_component:start_link(?MODULE, fun ?MODULE:on_inactive/2, [],\n                                     [{pid_groups_join_as, DHTNodeGroup, dc_clustering}]);\n        _ ->\n            ignore\n    end.\n\n%% @doc Initialises the module with an empty state.\n-spec init([]) -> state_inactive().\ninit([]) ->\n    %% generate trigger msgs only once and then keep them repeating\n    msg_delay:send_trigger(get_clustering_interval(), {dc_trigger}),\n    msg_delay:send_trigger(get_clustering_reset_interval(), {dc_reset_trigger}),\n    #state_inactive{\n        queued_messages = msg_queue:new()\n        , radius = config:read(dc_clustering_radius)\n    }\n    .\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Message Loop\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Message handler during start up phase (will change to on_active/2 when a\n%%      'activate_clustering' message is received).\n-spec on_inactive(message(), state_inactive()) -> state_inactive();\n                ({activate_clustering}, state_inactive())\n            -> {'$gen_component', [{on_handler, Handler::gen_component:handler()}], State::state_active()}.\non_inactive({activate_clustering}, #state_inactive{\n                                    queued_messages=QueuedMessages\n                                    ,radius=R\n                                }) ->\n    log:log(info, \"[ Clustering ~.0p ] activating...~n\", [comm:this()]),\n    comm:send_local(self(), {reset_clustering}),\n    comm:send_local(self(), {start_clustering_shuffle}),\n    msg_queue:send(QueuedMessages),\n    StateActive = #state_active{\n        centroids = dc_centroids:empty_centroids_list()\n        , radius=R\n    },\n    gen_component:change_handler(StateActive, fun ?MODULE:on_active/2);\n\non_inactive(Msg = {query_clustering, _Pid}, #state_inactive{queued_messages=QueuedMessages} = State) ->\n    State#state_inactive{queued_messages=msg_queue:add(QueuedMessages, Msg)}\n    ;\n\non_inactive({query_epoch, _Pid} = Msg, #state_inactive{queued_messages=Q} = State) ->\n    State#state_inactive{queued_messages=msg_queue:add(Q, Msg)};\n\non_inactive({web_debug_info, Requestor}, #state_inactive{queued_messages=Q} = State) ->\n    % get a list of up to 50 queued messages to display:\n    MessageListTmp = [{\"\", webhelpers:safe_html_string(\"~p\", [Message])}\n                  || Message <- lists:sublist(Q, 50)],\n    MessageList = case length(Q) > 50 of\n                      true -> lists:append(MessageListTmp, [{\"...\", \"\"}]);\n                      _    -> MessageListTmp\n                  end,\n    KeyValueList = [{\"\", \"\"}, {\"inactive clustering process\", \"\"}, {\"queued messages:\", \"\"} | MessageList],\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    State;\n\non_inactive({dc_trigger}, State) ->\n    %% keep trigger active to avoid generating new triggers when\n    %% frequently jumping between inactive and active state.\n    msg_delay:send_trigger(get_clustering_interval(), {dc_trigger}),\n    State;\n\non_inactive({dc_reset_trigger}, State) ->\n    %% keep trigger active to avoid generating new triggers when\n    %% frequently jumping between inactive and active state.\n    msg_delay:send_trigger(get_clustering_reset_interval(), {dc_reset_trigger}),\n    State;\n\non_inactive(_Msg, State) ->\n    State.\n\n%% @doc Message handler when the module is fully initialized.\n-spec on_active(Message::message(), State::state_active()) -> state_active();\n         ({deactivate_clustering}, state_active()) -> {'$gen_component', [{on_handler, Handler::gen_component:handler()}], State::state_inactive()}.\non_active({deactivate_clustering}\n          , #state_active{radius = R}) ->\n    log:log(info, \"[ Clustering ~.0p ] deactivating...~n\", [comm:this()]),\n    StateInactive = #state_inactive{\n        queued_messages = msg_queue:new()\n        , radius = R\n    },\n    gen_component:change_handler(StateInactive, fun ?MODULE:on_inactive/2);\n\n% ignore activate_clustering messages in active state\n% note: remove this if the clustering process is to be deactivated on leave (see\n% dht_node_move.erl). In the current implementation we can not distinguish\n% between the first join and a re-join but after every join, the process is\n% (re-)activated.\non_active({activate_clustering}, State) ->\n    State;\n\non_active({dc_trigger}, State) ->\n    msg_delay:send_trigger(get_clustering_interval(), {dc_trigger}),\n    gen_component:post_op({start_clustering_shuffle}, State);\n\non_active({start_clustering_shuffle}, State) ->\n    % start new clustering shuffle -> gossip communication\n    gossip_cyclon:get_subset_rand(1),\n    State;\n\non_active({dc_reset_trigger}, State) ->\n    msg_delay:send_trigger(get_clustering_reset_interval(), {dc_reset_trigger}),\n    gen_component:post_op({reset_clustering}, State);\n\n% ask vivaldi for network coordinate\non_active({reset_clustering}, State) ->\n    gossip_vivaldi:get_coordinate(),\n    State;\n\n% reset the local state and start a new epoche\non_active({vivaldi_get_coordinate_response, Coordinate, _Confidence}\n           , #state_active{local_epoch=Epoch} = State) ->\n    State#state_active{\n        local_epoch=Epoch + 1\n        , centroids=[dc_centroids:new(Coordinate, 1.0)]\n    };\n\non_active({cy_cache, []}, State)  ->\n    % ignore empty cache from cyclon\n    State;\n\n% got random node from cyclon\non_active({cy_cache, [Node] = _Cache}\n          , #state_active{centroids=C, local_epoch=E} = State) ->\n    %io:format(\"~p~n\",[_Cache]),\n    comm:send(node:pidX(Node), {clustering_shuffle, comm:this(), C, E},\n              [{group_member, dc_clustering}]),\n    State;\n\n% have been asked to shuffle\non_active({clustering_shuffle, RemoteNode, RemoteCentroids, RemoteEpoch}, State) ->\n    %io:format(\"{shuffle, ~p, ~p}~n\", [RemoteCoordinate, RemoteConfidence]),\n    handle_shuffle(State, {RemoteNode, RemoteEpoch, RemoteCentroids})\n    ;\n\n% got shuffle response\non_active({clustering_shuffle_reply, _RemoteNode, RemoteCentroids, RemoteEpoch}, State) ->\n    %io:format(\"{shuffle_reply, ~p, ~p}~n\", [RemoteCoordinate, RemoteConfidence]),\n    %vivaldi_latency:measure_latency(RemoteNode, RemoteCoordinate, RemoteConfidence),\n    handle_shuffle(State, {ignore, RemoteEpoch, RemoteCentroids})\n    ;\n\n% return my clusters\non_active({query_clustering, Pid}, #state_active{centroids=C} = State) ->\n    comm:send(Pid, {query_clustering_response, C}),\n    State\n    ;\n\non_active({query_my, Atom, Pid}, State) ->\n    case Atom of\n        local_epoch -> Msg = State#state_active.local_epoch\n            , comm:send(Pid, {query_my_response, local_epoch, Msg})\n            ;\n        radius -> Msg = State#state_active.radius\n            , comm:send(Pid, {query_my_response, radius, Msg})\n    end,\n    State\n    .\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    case config:read(dc_clustering_enable) of\n        true ->\n            config:cfg_is_integer(dc_clustering_reset_interval) andalso\n            config:cfg_is_greater_than(dc_clustering_reset_interval, 0) andalso\n            config:cfg_is_integer(dc_clustering_interval) andalso\n            config:cfg_is_greater_than(dc_clustering_interval, 0) andalso\n            config:cfg_is_float(dc_clustering_radius) andalso\n            config:cfg_is_greater_than(dc_clustering_radius, 0.0);\n        _ -> true\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Helpers\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec cluster(dc_centroids:centroids(), dc_centroids:centroids(), float()) -> dc_centroids:centroids().\ncluster([], RemoteCentroids, _Radius) -> RemoteCentroids;\ncluster(Centroids, RemoteCentroids, Radius) ->\n    NewCentroids = mathlib:aggloClustering(Centroids ++ RemoteCentroids, Radius),\n    NormalizedCentroids = lists:map(fun (Centroid) ->\n                dc_centroids:set_relative_size(Centroid,\n                    0.5 * dc_centroids:get_relative_size(Centroid))\n                end, NewCentroids),\n    NormalizedCentroids.\n\n%% @doc Gets the clustering reset interval set in scalaris.cfg.\n-spec get_clustering_reset_interval() -> pos_integer().\nget_clustering_reset_interval() ->\n    config:read(dc_clustering_reset_interval) div 1000.\n\n%% @doc Gets the clustering interval, e.g. how often to calculate clusters, set\n%%      in scalaris.cfg.\n-spec get_clustering_interval() -> pos_integer().\nget_clustering_interval() ->\n    config:read(dc_clustering_interval) div 1000.\n\n-spec handle_shuffle(state_active(),\n        {RemoteNode :: comm:mypid() | ignore, RemoteEpoch :: non_neg_integer(),\n         RemoteCentroids :: dc_centroids:centroids()}) -> state_active().\nhandle_shuffle(#state_active{\n        centroids=Centroids\n        , local_epoch=Epoch\n        , radius=Radius\n    } = State, {RemoteNode, RemoteEpoch, RemoteCentroids}) ->\n    % Check the epoch to swallow old messages and delete old state:\n    %\n    % - if the local epoch is less than the remote epoch, we didn't see a certain wave yet and\n    % have to reset the local cluster state.\n    % - if the local epoch is bigger than the remote epoch, we ignore the information we\n    % received and only send our information, as the remote node is behind the last wave\n    % we saw.\n\n    % With this, we purge \"zombie centroids\" (stale centroids far away from any real\n    % clusters) over time: clusters can not survive a wave if they seized to exist during\n    % the cut.\n\n    {NewEpoch, NewCentroids} =\n    if Epoch < RemoteEpoch -> % reset and merge remote information\n            gossip_vivaldi:get_coordinate(),\n            {RemoteEpoch, []}\n            ;\n        Epoch > RemoteEpoch -> % ignore remote information, keep talking\n            case RemoteNode of\n                ignore -> ok;\n                R -> comm:send(R, {clustering_shuffle_reply, comm:this(), Centroids, Epoch})\n            end,\n            {Epoch, Centroids};\n        true -> % Epoch == RemoteEpoch\n            case RemoteNode of\n                ignore -> ok;\n                R -> comm:send(R, {clustering_shuffle_reply, comm:this(), Centroids, Epoch})\n            end,\n            {Epoch, cluster(Centroids, RemoteCentroids, Radius)}\n    end,\n    case NewEpoch of % reset needed?\n        Epoch -> State#state_active{centroids=NewCentroids};\n        _Else ->\n            % Note: NewEpoch - 1 because we will increment it in the handler for the\n            % vivaldi reply\n            State#state_active{\n                centroids=NewCentroids\n                , local_epoch=NewEpoch - 1\n            }\n    end\n    .\n"
  },
  {
    "path": "src/debug.erl",
    "content": "% @copyright 2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc Methods for (interactive) debugging.\n%% @version $Id$\n-module(debug).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([get_round_trip/2]).\n-export([\n         dump/0, dump2/0, dump3/0, dumpX/1, dumpX/2,\n         topDumpX/1, topDumpX/3,\n         topDumpXEvery/3, topDumpXEvery/5, topDumpXEvery_helper/4\n        ]).\n-export([rr_count_old_replicas/2]).\n\n%% @doc Simple round-trip benchmark to an arbitrary gen_component.\n-spec get_round_trip(GPid::comm:mypid(), Iterations::pos_integer()) -> float().\nget_round_trip(GPid, Iterations) ->\n    Start = os:timestamp(),\n    get_round_trip_helper(GPid, Iterations),\n    End = os:timestamp(),\n    timer:now_diff(End, Start) / Iterations.\n\n%% @doc Helper for get_round_trip/2.\n-spec get_round_trip_helper(GPid::comm:mypid(), Iterations::pos_integer()) -> ok.\nget_round_trip_helper(_GPid, 0) ->\n    ok;\nget_round_trip_helper(GPid, Iterations) ->\n    comm:send(GPid, {ping, comm:this()}),\n    receive _Any -> ok end,\n    get_round_trip_helper(GPid, Iterations - 1).\n\n%% @doc Extracts a given ItemInfo from an ItemList that has been returned from\n%%      e.g. erlang:process_info/2 for the dump* methods.\n-spec dump_extract_from_list\n        ([{Item::atom(), Info::term()}], ItemInfo::memory | message_queue_len | stack_size | heap_size) -> non_neg_integer();\n        ([{Item::atom(), Info::term()}], ItemInfo::messages) -> [tuple()];\n        ([{Item::atom(), Info::term()}], ItemInfo::current_function) -> Fun::mfa();\n        ([{Item::atom(), Info::term()}], ItemInfo::dictionary) -> [{Key::term(), Value::term()}].\ndump_extract_from_list(List, Key) ->\n    element(2, lists:keyfind(Key, 1, List)).\n\n%% @doc Returns a list of all currently executed functions and the number of\n%%      instances for each of them.\n-spec dump() -> [{Fun::mfa(), FunExecCount::pos_integer()}].\ndump() ->\n    Info = [element(2, Fun) || X <- processes(),\n                               Fun <- [process_info(X, current_function)],\n                               Fun =/= undefined],\n    FunCnt = dict:to_list(lists:foldl(fun(Fun, DictIn) ->\n                                              dict:update_counter(Fun, 1, DictIn)\n                                      end, dict:new(), Info)),\n    lists:reverse(lists:keysort(2, FunCnt)).\n\n%% @doc Returns information about all processes' memory usage.\n-spec dump2() -> [{PID::pid(), [pos_integer() | mfa() | any()]}].\ndump2() ->\n    dumpX([memory, current_function, dictionary],\n          fun(K, Value) ->\n                  case K of\n                      dictionary -> util:extract_from_list_may_not_exist(Value, test_server_loc);\n                      _          -> Value\n                  end\n          end).\n\n%% @doc Returns various data about all processes.\n-spec dump3() -> [{PID::pid(), [Mem | MsgQLength | StackSize | HeapSize | Messages | Fun]}]\n        when is_subtype(Mem, non_neg_integer()),\n             is_subtype(MsgQLength, non_neg_integer()),\n             is_subtype(StackSize, non_neg_integer()),\n             is_subtype(HeapSize, non_neg_integer()),\n             is_subtype(Messages, [atom()]),\n             is_subtype(Fun, mfa()).\ndump3() ->\n    dumpX([memory, message_queue_len, stack_size, heap_size, messages, current_function],\n          fun(K, Value) ->\n                  case K of\n                      messages -> [element(1, V) || V <- Value];\n                      _        -> Value\n                  end\n          end).\n\n-spec default_dumpX_val_fun(K::atom(), Value::term()) -> term().\ndefault_dumpX_val_fun(K, Value) ->\n    case K of\n        messages -> [element(1, V) || V <- Value];\n        dictionary -> [element(1, V) || V <- Value];\n        registered_name when Value =:= [] -> undefined;\n        _        -> Value\n    end.\n\n%% @doc Returns various data about all processes.\n-spec dumpX([ItemInfo::atom(),...]) -> [tuple(),...].\ndumpX(Keys) ->\n    dumpX(Keys, fun default_dumpX_val_fun/2).\n\n%% @doc Returns various data about all processes.\n-spec dumpX([ItemInfo::atom(),...], ValueFun::fun((atom(), term()) -> term())) -> [tuple(),...].\ndumpX(Keys, ValueFun) ->\n    lists:reverse(lists:keysort(2, dumpXNoSort(Keys, ValueFun))).\n\n-spec dumpXNoSort([ItemInfo::atom(),...], ValueFun::fun((atom(), term()) -> term())) -> [tuple(),...].\ndumpXNoSort(Keys, ValueFun) ->\n    [{Pid, [ValueFun(Key, dump_extract_from_list(Data, Key)) || Key <- Keys]}\n     || Pid <- processes(),\n        undefined =/= (Data = process_info(Pid, Keys))].\n\n%% @doc Convenience wrapper to topDumpX/3.\n-spec topDumpX(Keys | Seconds | ValueFun) -> [{pid(), [Reductions | RegName | term(),...]},...]\n        when is_subtype(Keys, [ItemInfo::atom()]),\n             is_subtype(Seconds, pos_integer()),\n             is_subtype(ValueFun, fun((atom(), term()) -> term())),\n             is_subtype(Reductions, non_neg_integer()),\n             is_subtype(RegName, atom()).\ntopDumpX(Keys) when is_list(Keys) ->\n    topDumpX(Keys, fun default_dumpX_val_fun/2, 5);\ntopDumpX(Seconds) when is_integer(Seconds) ->\n    topDumpX([], fun default_dumpX_val_fun/2, Seconds);\ntopDumpX(ValueFun) when is_function(ValueFun, 2) ->\n    topDumpX([], ValueFun, 5).\n\n%% @doc Gets the number of reductions for each process within the next Seconds\n%%      and dumps some process data defined by Keys (sorted by the number of\n%%      reductions).\n-spec topDumpX(Keys, ValueFun, Seconds) -> [{pid(), [Reductions | RegName | term(),...]},...]\n        when is_subtype(Keys, [ItemInfo::atom()]),\n             is_subtype(Seconds, pos_integer()),\n             is_subtype(ValueFun, fun((atom(), term()) -> term())),\n             is_subtype(Reductions, non_neg_integer()),\n             is_subtype(RegName, atom()).\ntopDumpX(Keys, ValueFun, Seconds) when is_integer(Seconds) andalso Seconds >= 1 ->\n    Start = lists:keysort(1, dumpXNoSort([reductions, registered_name], ValueFun)),\n    timer:sleep(1000 * Seconds),\n    End = lists:keysort(1, dumpXNoSort([reductions, registered_name | Keys], ValueFun)),\n    Self = self(),\n    lists:reverse(\n      lists:keysort(\n        2, util:smerge2(Start, End,\n                   fun(T1, T2) -> element(1, T1) =< element(1, T2) end,\n                   fun(T1, T2) ->\n                           % {<0.77.0>,[250004113,comm_server,692064]}\n                           % io:format(\"T1=~.0p~nT2=~.0p~n\", [T1, T2]),\n                           {Pid, [Red1, RName]} = T1,\n                           {Pid, [Red2, RName | Rest2]} = T2,\n                           case Pid of\n                               Self -> [];\n                               _    -> [{Pid, [(Red2 - Red1), RName | Rest2]}]\n                           end\n                   end,\n                   fun(_T1) -> [] end, % omit processes not alive at the end any more\n                   fun(T2) -> [T2] end))).\n\n%% @doc Convenience wrapper to topDumpXEvery/5.\n-spec topDumpXEvery(Keys | Seconds | ValueFun, Subset::pos_integer(), StopAfter::pos_integer()) -> timer:tref()\n        when is_subtype(Keys, [ItemInfo::atom()]),\n             is_subtype(Seconds, pos_integer()),\n             is_subtype(ValueFun, fun((atom(), term()) -> term())).\ntopDumpXEvery(Keys, Subset, StopAfter) when is_list(Keys) ->\n    topDumpXEvery(Keys, fun default_dumpX_val_fun/2, 1, Subset, StopAfter);\ntopDumpXEvery(Seconds, Subset, StopAfter) when is_integer(Seconds) ->\n    topDumpXEvery([], fun default_dumpX_val_fun/2, Seconds, Subset, StopAfter);\ntopDumpXEvery(ValueFun, Subset, StopAfter) when is_function(ValueFun, 2) ->\n    topDumpXEvery([], ValueFun, 1, Subset, StopAfter).\n\n%% @doc Calls topDumpX/3 every Seconds and prints the top Subset processes with\n%%      the highest number of reductions. Stops after StopAfter seconds.\n-spec topDumpXEvery(Keys, ValueFun, Seconds, Subset::pos_integer(), StopAfter::pos_integer()) -> timer:tref()\n        when is_subtype(Keys, [ItemInfo::atom()]),\n             is_subtype(Seconds, pos_integer()),\n             is_subtype(ValueFun, fun((atom(), term()) -> term())).\ntopDumpXEvery(Keys, ValueFun, IntervalS, Subset, StopAfter)\n  when is_integer(IntervalS) andalso IntervalS >= 1 andalso\n           is_integer(Subset) andalso Subset >= 1 andalso\n           is_integer(StopAfter) andalso StopAfter >= IntervalS ->\n    {ok, TRef} = timer:apply_interval(1000 * IntervalS, ?MODULE, topDumpXEvery_helper,\n                                      [Keys, ValueFun, IntervalS, Subset]),\n    {ok, _} = timer:apply_after(1000 * StopAfter, timer, cancel, [TRef]),\n    TRef.\n\n%% @doc Helper function for topDumpXEvery/5 (export needed for timer:apply_after/4).\n-spec topDumpXEvery_helper(Keys, ValueFun, Seconds, Subset::pos_integer()) -> ok\n        when is_subtype(Keys, [ItemInfo::atom()]),\n             is_subtype(Seconds, pos_integer()),\n             is_subtype(ValueFun, fun((atom(), term()) -> term())).\ntopDumpXEvery_helper(Keys, ValueFun, Seconds, Subset) ->\n    io:format(\"-----~n~.0p~n\", [lists:sublist(topDumpX(Keys, ValueFun, Seconds), Subset)]).\n\n-spec rr_count_old_replicas(Key::?RT:key(), Interval::intervals:interval())\n        -> non_neg_integer().\nrr_count_old_replicas(Key, ReqI) ->\n    NodeDetails = rr_count_old_replicas_nd(Key),\n\n    I = intervals:intersection(ReqI, node_details:get(NodeDetails, my_range)),\n    case intervals:is_empty(I) of\n        true -> 0;\n        false ->\n            DBList = rr_count_old_replicas_data(\n                       node:pidX(node_details:get(NodeDetails, node))),\n            This = comm:this(),\n            lists:foldl(\n              fun({KeyX, VerX}, Acc) ->\n                      _ = [api_dht_raw:unreliable_lookup(K, {?read_op, This, 0, K, ?write})\n                             || K <- ?RT:get_replica_keys(KeyX), K =/= KeyX],\n                      % note: receive wrapped in anonymous functions to allow\n                      %       ?SCALARIS_RECV in multiple receive statements\n                      V1 = fun() ->\n                                   trace_mpath:thread_yield(),\n                                   receive ?SCALARIS_RECV({?read_op_with_id_reply,\n                                                           0, _, ?ok,\n                                                           ?value_dropped, V},\n                                                          V)\n                                   end end(),\n                      V2 = fun() ->\n                                   trace_mpath:thread_yield(),\n                                   receive ?SCALARIS_RECV({?read_op_with_id_reply,\n                                                           0, _, ?ok,\n                                                           ?value_dropped, V},\n                                                          V)\n                                   end end(),\n                      V3 = fun() ->\n                                   trace_mpath:thread_yield(),\n                                   receive ?SCALARIS_RECV({?read_op_with_id_reply,\n                                                           0, _, ?ok,\n                                                           ?value_dropped, V},\n                                                          V)\n                            end end(),\n                      case VerX =:= lists:max([V1, V2, V3]) of\n                          true -> Acc;\n                          false -> Acc + 1\n                      end\n              end, 0, DBList)\nend.\n\n-spec rr_count_old_replicas_nd(Key::?RT:key()) -> node_details:node_details().\nrr_count_old_replicas_nd(Key) ->\n    api_dht_raw:unreliable_lookup(Key, {get_node_details, comm:this(),\n                                        [node, my_range]}),\n    trace_mpath:thread_yield(),\n    receive ?SCALARIS_RECV({get_node_details_response, NodeDetails},% ->\n                           NodeDetails) end.\n\n-spec rr_count_old_replicas_data(Pid::comm:mypid()) -> [{?RT:key(), client_version()}].\nrr_count_old_replicas_data(Pid) ->\n    comm:send(Pid, {unittest_get_bounds_and_data, comm:this(), kv}),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n        {unittest_get_bounds_and_data_response, _Bounds, DBList, _Pred, _Succ},% ->\n        DBList)\n    end.\n"
  },
  {
    "path": "src/dht_node.erl",
    "content": "%  @copyright 2007-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    dht_node main file\n%% @end\n%% @version $Id$\n-module(dht_node).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n-include(\"lookup.hrl\").\n\n-behaviour(gen_component).\n\n-export([start_link/2, on/2, init/1]).\n\n-export([is_first/1, is_alive/1, is_alive_no_slide/1, is_alive_fully_joined/1]).\n\n-export_type([message/0]).\n\n-include(\"gen_component.hrl\").\n\n-type(database_message() ::\n      {?get_key, Source_PID::comm:mypid(), SourceId::any(), HashedKey::?RT:key()} |\n      {get_entries, Source_PID::comm:mypid(), Interval::intervals:interval()} |\n      {get_entries, Source_PID::comm:mypid(), FilterFun::fun((db_entry:entry()) -> boolean()),\n            ValFun::fun((db_entry:entry()) -> any())} |\n      {get_chunk, Source_PID::comm:mypid(), Interval::intervals:interval(), MaxChunkSize::pos_integer() | all} |\n      {get_chunk, Source_PID::comm:mypid(), Interval::intervals:interval(), FilterFun::fun((db_entry:entry()) -> boolean()),\n            ValFun::fun((db_entry:entry()) -> any()), MaxChunkSize::pos_integer() | all} |\n      {update_key_entries, Source_PID::comm:mypid(), [{HashedKey::?RT:key(), NewValue::db_dht:value(), NewVersion::client_version()}]} |\n%%      % DB subscriptions:\n%%      {db_set_subscription, SubscrTuple::db_dht:subscr_t()} |\n%%      {db_get_subscription, Tag::any(), SourcePid::comm:erl_local_pid()} |\n%%      {db_remove_subscription, Tag::any()} |\n      % direct DB manipulation:\n      {get_key_entry, Source_PID::comm:mypid(), HashedKey::?RT:key()} |\n      {set_key_entry, Source_PID::comm:mypid(), Entry::db_entry:entry()} |\n      {delete_key, Source_PID::comm:mypid(), ClientsId::{delete_client_id, uid:global_uid()}, HashedKey::?RT:key()} |\n      {add_data, Source_PID::comm:mypid(), db_dht:db_as_list()} |\n      {drop_data, Data::db_dht:db_as_list(), Sender::comm:mypid()}).\n\n-type(lookup_message() ::\n      {?lookup_aux, Key::?RT:key(), Hops::pos_integer(), Msg::comm:message()} |\n      {?lookup_fin, Key::?RT:key(), Data::dht_node_lookup:data(), Msg::comm:message()}).\n\n-type(snapshot_message() ::\n      {do_snapshot, SnapNumber::non_neg_integer(), Leader::comm:mypid()} |\n      {local_snapshot_is_done}).\n\n-type(rt_message() ::\n      {rt_update, RoutingTable::?RT:external_rt()}).\n\n-type(misc_message() ::\n      {get_yaws_info, Pid::comm:mypid()} |\n      {get_state, Pid::comm:mypid(), Which::dht_node_state:name()} |\n      {get_node_details, Pid::comm:mypid()} |\n      {get_node_details, Pid::comm:mypid(), Which::[node_details:node_details_name()]} |\n      {get_pid_group, Pid::comm:mypid()} |\n      {dump} |\n      {web_debug_info, Requestor::comm:erl_local_pid()} |\n      {get_dht_nodes_response, KnownHosts::[comm:mypid()]} |\n      {unittest_get_bounds_and_data, SourcePid::comm:mypid(), full | kv}).\n\n% accepted messages of dht_node processes\n-type message() ::\n    bulkowner:bulkowner_msg() |\n    database_message() |\n    lookup_message() |\n    dht_node_join:join_message() |\n    rt_message() |\n    dht_node_move:move_message() |\n    misc_message() |\n    snapshot_message() |\n    {zombie, Node::node:node_type()} |\n    {fd_notify, fd:event(), DeadPid::comm:mypid(), Reason::fd:reason()} |\n    {leave, SourcePid::comm:erl_local_pid() | null} |\n    {rejoin, IdVersion::non_neg_integer(), JoinOptions::[tuple()],\n      {get_move_state_response, MoveState::[tuple()]}}.\n\n%% @doc message handler\n-spec on(message(), dht_node_state:state()) -> dht_node_state:state() | kill.\n%% Join messages (see dht_node_join.erl)\n%% userdevguide-begin dht_node:join_message\non(Msg, State) when join =:= element(1, Msg) ->\n    lb_stats:set_ignore_db_requests(true),\n    NewState = dht_node_join:process_join_msg(Msg, State),\n    lb_stats:set_ignore_db_requests(false),\n    NewState;\n%% userdevguide-end dht_node:join_message\n\n% Move messages (see dht_node_move.erl)\non(Msg, State) when move =:= element(1, Msg) ->\n    lb_stats:set_ignore_db_requests(true),\n    NewState = dht_node_move:process_move_msg(Msg, State),\n    lb_stats:set_ignore_db_requests(false),\n    NewState;\n\n% Lease management messages (see l_on_cseq.erl)\non(Msg, State) when l_on_cseq =:= element(1, Msg) ->\n    l_on_cseq:on(Msg, State);\n\n% DHT node extensions (see dht_node_extensions.erl)\non(Msg, State) when extensions =:= element(1, Msg) ->\n    dht_node_extensions:on(Msg, State);\n\n% RM messages (see rm_loop.erl)\non(Msg, State) when element(1, Msg) =:= rm ->\n    RMState = dht_node_state:get(State, rm_state),\n    RMState1 = rm_loop:on(Msg, RMState),\n    dht_node_state:set_rm(State, RMState1);\n\non({leave, SourcePid}, State) ->\n    dht_node_move:make_slide_leave(State, SourcePid);\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Finger Maintenance\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({rt_update, RoutingTable}, State) ->\n    dht_node_state:set_rt(State, RoutingTable);\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Transactions (see transactions/*.erl)\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({get_rtm, Source_PID, Key, Process}, State) ->\n    case pid_groups:get_my(Process) of\n        failed ->\n            R = config:read(replication_factor),\n            case Process of\n                {tx_rtm,X} when X =< R ->\n                    %% these rtms are concurrently started by the supervisor\n                    %% we just have to wait a bit...\n                    comm:send_local(self(),\n                                    {get_rtm, Source_PID, Key, Process});\n                _ ->\n                    log:log(warn, \"[ ~.0p ] requested non-existing rtm ~.0p~n\",\n                            [comm:this(), Process])\n            end;\n        Pid ->\n            GPid = comm:make_global(Pid),\n            GPidAcc = comm:make_global(tx_tm_rtm:get_my(Process, acceptor)),\n            comm:send(Source_PID, {get_rtm_reply, Key, GPid, GPidAcc})\n    end,\n    State;\n\n%% messages handled as a transaction participant (TP)\non({?init_TP, {_Tid, _RTMs, _Accs, _TM, _RTLogEntry, _ItemId, _PaxId, SnapNo} = Params}, State) ->\n    % check if new snapshot\n    SnapState = dht_node_state:get(State,snapshot_state),\n    LocalSnapNumber = snapshot_state:get_number(SnapState),\n    case SnapNo > LocalSnapNumber of\n        true ->\n            comm:send(comm:this(), {do_snapshot, SnapNo, none});\n        false ->\n            ok\n    end,\n    tx_tp:on_init_TP(Params, State);\non({?tp_do_commit_abort, Id, Result, SnapNumber}, State) ->\n    tx_tp:on_do_commit_abort(Id, Result, SnapNumber, State);\non({?tp_do_commit_abort_fwd, TM, TMItemId, RTLogEntry, Result, OwnProposal, SnapNumber}, State) ->\n    tx_tp:on_do_commit_abort_fwd(TM, TMItemId, RTLogEntry, Result, OwnProposal, SnapNumber, State);\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Caching of dht_node responsibilties (see dht_node_cache.erl)\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({cache_interval_lookup, Key, Client, _Cons}, State) ->\n    Interval = dht_node_state:get(State, my_range),\n    comm:send(Client, {cache_interval_lookup_reply, comm:this(), Key, Interval}),\n    State;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Lookup (see api_dht_raw.erl and dht_node_lookup.erl)\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\non({?lookup_aux, Key, Hops, Msg}=FullMsg, State) ->\n    %% Forward msg to the routing_table (rt_loop).\n    %% If possible, messages should be sent directly to routing_table (rt_loop).\n    %% Only forward messages for which we aren't responsible (leases).\n    %% log:pal(\"lookup_aux_leases in dht_node\"),\n    case config:read(leases) of\n        true ->\n            %% Check lease and translate to lookup_fin or forward to rt_loop\n            %% accordingly.\n            dht_node_lookup:lookup_aux_leases(State, Key, Hops, Msg);\n        _ ->\n            %% simply forward the message to routing_table (rt_loop)\n            comm:send_local(pid_groups:get_my(routing_table), FullMsg)\n    end,\n    State;\n\non({lookup_decision, Key, Hops, Msg}, State) ->\n    %% message from rt_loop requesting a decision about a aux-fin transformation\n    %% (chord only)\n    dht_node_lookup:lookup_decision_chord(State, Key, Hops, Msg),\n    State;\n\non({?lookup_fin, Key, Data, Msg}, State) ->\n    dht_node_lookup:lookup_fin(State, Key, Data, Msg);\n\non({send_error, Target, {?lookup_aux, _, _, _} = Message, _Reason}, State) ->\n    dht_node_lookup:lookup_aux_failed(State, Target, Message);\n\non({send_error, Target, {?send_to_group_member, routing_table,\n                         {?lookup_aux, _Key, _Hops, _Msg} = Message}, _Reason}, State) ->\n    dht_node_lookup:lookup_aux_failed(State, Target, Message);\n\n%on({send_failed, {send_error, Target, {?lookup_aux, _, _, _}} = Message, _Reason}, _Pids}}, State) ->\n%    dht_node_lookup:lookup_aux_failed(State, Target, Message);\non({send_error, Target, {?lookup_fin, _, _, _} = Message, _Reason}, State) ->\n    dht_node_lookup:lookup_fin_failed(State, Target, Message);\n\n%% messages handled as a prbr\non(X, State) when is_tuple(X) andalso element(1, X) =:= prbr ->\n    %% as prbr has several use cases (may operate on different DBs) in\n    %% the dht_node, the addressed use case is given in the third\n    %% element by convention.\n    DBKind = element(3, X),\n    PRBRState = dht_node_state:get(State, DBKind),\n    NewRBRState = prbr:on(X, PRBRState),\n    dht_node_state:set_prbr_state(State, DBKind, NewRBRState);\n\non(X, State) when is_tuple(X) andalso (element(1, X) =:= crdt_acceptor\n        orelse element(1,X) =:= zheng_acceptor orelse element(1,X) =:= gla_acceptor) ->\n    Module = element(1, X),\n    CrdtState = dht_node_state:get(State, crdt_db),\n    NewCrdtState = Module:on(X, CrdtState),\n    dht_node_state:set_prbr_state(State, crdt_db, NewCrdtState);\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Database\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({?get_key, Source_PID, SourceId, HashedKey}, State) ->\n    Msg = {?get_key_with_id_reply, SourceId, HashedKey,\n           db_dht:read(dht_node_state:get(State, db), HashedKey)},\n    comm:send(Source_PID, Msg),\n    State;\n\non({?read_op, Source_PID, SourceId, HashedKey, Op}, State) ->\n    DB = dht_node_state:get(State, db),\n    {ok, Value, Version} = db_dht:read(DB, HashedKey),\n    {Ok_Fail, Val_Reason, Vers} = rdht_tx_read:extract_from_value(Value, Version, Op),\n    SnapInfo = dht_node_state:get(State, snapshot_state),\n    SnapNumber = snapshot_state:get_number(SnapInfo),\n    Msg = {?read_op_with_id_reply, SourceId, SnapNumber, Ok_Fail, Val_Reason, Vers},\n    comm:send(Source_PID, Msg),\n    State;\n\non({get_entries, Source_PID, Interval}, State) ->\n    Entries = db_dht:get_entries(dht_node_state:get(State, db), Interval),\n    comm:send_local(Source_PID, {get_entries_response, Entries}),\n    State;\n\non({get_entries, Source_PID, FilterFun, ValFun}, State) ->\n    Entries = db_dht:get_entries(dht_node_state:get(State, db), FilterFun, ValFun),\n    comm:send_local(Source_PID, {get_entries_response, Entries}),\n    State;\n\non({get_data, Source_PID}, State) ->\n    Data = db_dht:get_data(dht_node_state:get(State, db)),\n    comm:send_local(Source_PID, {get_data_response, Data}),\n    State;\n\non({get_data, Source_PID, FilterFun, ValueFun}, State) ->\n    Data = db_dht:get_data(dht_node_state:get(State, db), FilterFun, ValueFun),\n    comm:send_local(Source_PID, {get_data_response, Data}),\n    State;\n\non({get_chunk, Source_PID, Interval, MaxChunkSize}, State) ->\n    Chunk = db_dht:get_chunk(dht_node_state:get(State, db),\n                             dht_node_state:get(State, pred_id),\n                          Interval, MaxChunkSize),\n    comm:send_local(Source_PID, {get_chunk_response, Chunk}),\n    State;\n\non({get_chunk, Source_PID, Interval, FilterFun, ValueFun, MaxChunkSize}, State) ->\n    Chunk = db_dht:get_chunk(dht_node_state:get(State, db),\n                             dht_node_state:get(State, pred_id),\n                          Interval, FilterFun, ValueFun, MaxChunkSize),\n    comm:send_local(Source_PID, {get_chunk_response, Chunk}),\n    State;\n\non({update_key_entries, Source_PID, KvvList}, State) ->\n    DB = dht_node_state:get(State, db),\n    {NewDB, NewEntryList} = update_key_entries(KvvList, DB, State, []),\n    % send caller update_key_entries_ack with list of {Entry, Exists (Yes/No), Updated (Yes/No)}\n    comm:send(Source_PID, {update_key_entries_ack, NewEntryList}),\n    dht_node_state:set_db(State, NewDB);\n\n%%on({db_set_subscription, SubscrTuple}, State) ->\n%%    DB2 = db_dht:set_subscription(dht_node_state:get(State, db), SubscrTuple),\n%%    dht_node_state:set_db(State, DB2);\n%%\n%%on({db_get_subscription, Tag, SourcePid}, State) ->\n%%    Subscr = db_dht:get_subscription(dht_node_state:get(State, db), Tag),\n%%    comm:send_local(SourcePid, {db_get_subscription_response, Tag, Subscr}),\n%%    State;\n%%\n%%on({db_remove_subscription, Tag}, State) ->\n%%    DB2 = db_dht:remove_subscription(dht_node_state:get(State, db), Tag),\n%%    dht_node_state:set_db(State, DB2);\n\non({delete_key, Source_PID, ClientsId, HashedKey}, State) ->\n    {DB2, Result} = db_dht:delete(dht_node_state:get(State, db), HashedKey),\n    comm:send(Source_PID, {delete_key_response, ClientsId, HashedKey, Result}),\n    dht_node_state:set_db(State, DB2);\n\n%% for unit testing only: allow direct DB manipulation\non({get_key_entry, Source_PID, HashedKey}, State) ->\n    Entry = db_dht:get_entry(dht_node_state:get(State, db), HashedKey),\n    comm:send(Source_PID, {get_key_entry_reply, Entry}),\n    State;\n\non({set_key_entry, Source_PID, Entry}, State) ->\n    NewDB = db_dht:set_entry(dht_node_state:get(State, db), Entry),\n    comm:send(Source_PID, {set_key_entry_reply, Entry}),\n    dht_node_state:set_db(State, NewDB);\n\non({add_data, Source_PID, Data}, State) ->\n    NewDB = db_dht:add_data(dht_node_state:get(State, db), Data),\n    comm:send(Source_PID, {add_data_reply}),\n    dht_node_state:set_db(State, NewDB);\n\non({delete_keys, Source_PID, HashedKeys}, State) ->\n    DB2 = db_dht:delete_entries(dht_node_state:get(State, db), intervals:from_elements(HashedKeys)),\n    comm:send(Source_PID, {delete_keys_reply}),\n    dht_node_state:set_db(State, DB2);\n\non({drop_data, Data, Sender}, State) ->\n    comm:send(Sender, {drop_data_ack}),\n    DB = db_dht:add_data(dht_node_state:get(State, db), Data),\n    dht_node_state:set_db(State, DB);\n\non({get_split_key, DB, Begin, End, TargetLoad, Direction, Sender}, State) ->\n    comm:send_local(Sender, {get_split_key_response,\n                             db_dht:get_split_key(DB, Begin, End,\n                                                  TargetLoad, Direction)}),\n    State;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Bulk owner messages (see bulkowner.erl)\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non(Msg, State) when bulkowner =:= element(1, Msg) ->\n    bulkowner:on(Msg, State);\n\non({send_error, _FailedTarget, FailedMsg, _Reason} = Msg, State)\n  when bulkowner =:= element(1, FailedMsg) ->\n    bulkowner:on(Msg, State);\n\non({bulk_distribute, _Id, _Range, InnerMsg, _Parents} = Msg, State)\n  when mr =:= element(1, InnerMsg) ->\n    mr:on(Msg, State);\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% map reduce related messages (see mr.erl)\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non(Msg, State) when mr =:= element(1, Msg) ->\n    mr:on(Msg, State);\n\non(Msg, State) when mr_master =:= element(1, Msg) ->\n    try\n        mr_master:on(Msg, State)\n    catch\n        error:function_clause ->\n            log:log(warn, \"Received Message to non-existing master...ignoring!\"),\n            State\n    end;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% active load balancing messages (see lb_active_*.erl)\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non(Msg, State) when lb_active =:= element(1, Msg) ->\n    lb_active:handle_dht_msg(Msg, State);\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% handling of failed sends\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Misc.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\non({get_yaws_info, Pid}, State) ->\n    comm:send(Pid, {get_yaws_info_response, comm:get_ip(comm:this()), config:read(yaws_port), pid_groups:my_groupname()}),\n    State;\non({get_state, Pid, Which}, State) when is_list(Which) ->\n    comm:send(Pid, {get_state_response,\n                    [{X, dht_node_state:get(State, X)} || X <- Which]}),\n    State;\non({get_state, Pid, Which}, State) when is_atom(Which) ->\n    comm:send(Pid, {get_state_response, dht_node_state:get(State, Which)}),\n    State;\non({set_state, Pid, F}, State) when is_function(F) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    NewState = F(State),\n    comm:send(Pid, {set_state_response, NewState}),\n    NewState;\non({get_node_details, Pid}, State) ->\n    comm:send(Pid, {get_node_details_response, dht_node_state:details(State)}),\n    State;\non({get_node_details, Pid, Which}, State) ->\n    comm:send(Pid, {get_node_details_response, dht_node_state:details(State, Which)}),\n    State;\non({get_pid_group, Pid}, State) ->\n    comm:send(Pid, {get_pid_group_response, pid_groups:my_groupname()}),\n    State;\n\non({dump}, State) ->\n    dht_node_state:dump(State),\n    State;\n\non({web_debug_info, Requestor}, State) ->\n    RMState = dht_node_state:get(State, rm_state),\n    Load = dht_node_state:get(State, load),\n    % get a list of up to 50 KV pairs to display:\n    DataListTmp = [{\"\",\n                    webhelpers:safe_html_string(\"~p\", [DBEntry])}\n                  || DBEntry <- element(2, db_dht:get_chunk(dht_node_state:get(State, db),\n                                                         dht_node_state:get(State, node_id),\n                                                         intervals:all(), 50))],\n    DataList = case Load > 50 of\n                   true  -> lists:append(DataListTmp, [{\"...\", \"\"}]);\n                   false -> DataListTmp\n               end,\n    KVList1 =\n        [{\"rt_algorithm\", webhelpers:safe_html_string(\"~p\", [?RT])},\n         {\"rt_size\", dht_node_state:get(State, rt_size)},\n         {\"my_range\", webhelpers:safe_html_string(\"~p\", [intervals:get_bounds(dht_node_state:get(State, my_range))])},\n         {\"db_range\", webhelpers:safe_html_string(\"~p\", [dht_node_state:get(State, db_range)])},\n         {\"load\", webhelpers:safe_html_string(\"~p\", [Load])},\n         {\"join_time\", webhelpers:safe_html_string(\"~p UTC\", [calendar:now_to_universal_time(dht_node_state:get(State, join_time))])},\n%%          {\"db\", webhelpers:safe_html_string(\"~p\", [dht_node_state:get(State, db)])},\n%%          {\"proposer\", webhelpers:safe_html_string(\"~p\", [pid_groups:get_my(paxos_proposer)])},\n         {\"tx_tp_db\", webhelpers:safe_html_string(\"~p\", [dht_node_state:get(State, tx_tp_db)])},\n         {\"slide_pred\", webhelpers:safe_html_string(\"~p\", [dht_node_state:get(State, slide_pred)])},\n         {\"slide_succ\", webhelpers:safe_html_string(\"~p\", [dht_node_state:get(State, slide_succ)])},\n         {\"msg_fwd\", webhelpers:safe_html_string(\"~p\", [dht_node_state:get(State, msg_fwd)])}\n        ],\n    KVList2 = lists:append(KVList1, [{\"\", \"\"} | rm_loop:get_web_debug_info(RMState)]),\n    KVList3 = lists:append(KVList2, [{\"\", \"\"} , {\"data (db_entry):\", \"\"} | DataList]),\n    comm:send_local(Requestor, {web_debug_info_reply, KVList3}),\n    State;\n\non({unittest_get_bounds_and_data, SourcePid, Type}, State) ->\n    MyRange = dht_node_state:get(State, my_range),\n    MyBounds = intervals:get_bounds(MyRange),\n    DB = dht_node_state:get(State, db),\n    Data =\n        case Type of\n            kv ->\n                element(\n                  2,\n                  db_dht:get_chunk(\n                    DB, ?MINUS_INFINITY, intervals:all(),\n                    fun(_) -> true end,\n                    fun(E) -> {db_entry:get_key(E), db_entry:get_version(E)} end,\n                    all));\n            full ->\n                db_dht:get_data(DB)\n        end,\n    Pred = dht_node_state:get(State, pred),\n    Succ = dht_node_state:get(State, succ),\n    comm:send(SourcePid, {unittest_get_bounds_and_data_response, MyBounds, Data, Pred, Succ}),\n    State;\n\non({unittest_consistent_send, Pid, _X} = Msg, State) ->\n    ?ASSERT(util:is_unittest()),\n    comm:send_local(Pid, Msg),\n    State;\n\non({get_dht_nodes_response, _KnownHosts}, State) ->\n    % will ignore these messages after join\n    State;\n\non({fd_notify, Event, DeadPid, Data}, State) ->\n    % TODO: forward to further integrated modules, e.g. join?\n    RMState = dht_node_state:get(State, rm_state),\n    RMState1 = rm_loop:fd_notify(RMState, Event, DeadPid, Data),\n    dht_node_state:set_rm(State, RMState1);\n\n% dead-node-cache reported dead node to be alive again\non({zombie, Node}, State) ->\n    RMState = dht_node_state:get(State, rm_state),\n    RMState1 = rm_loop:zombie_node(RMState, Node),\n    % TODO: call other modules, e.g. join, move\n    dht_node_state:set_rm(State, RMState1);\n\non({do_snapshot, SnapNumber, Leader}, State) ->\n    snapshot:on_do_snapshot(SnapNumber, Leader, State);\n\non({local_snapshot_is_done}, State) ->\n    snapshot:on_local_snapshot_is_done(State);\n\non({ping, Pid, Msg}, State) ->\n    comm:send(Pid, Msg),\n    State;\n\non({rejoin, Id, Options, {get_move_state_response, MoveState}}, State) ->\n    % clean up RM, e.g. fd subscriptions:\n    rm_loop:cleanup(dht_node_state:get(State, rm_state)),\n    %% start new join\n    comm:send_local(self(), {join, start}),\n    JoinOptions = [{move_state, MoveState} | Options],\n    IdVersion = node:id_version(dht_node_state:get(State, node)),\n    dht_node_state:delete_for_rejoin(State), % clean up state!\n    dht_node_join:join_as_other(Id, IdVersion+1, JoinOptions).\n\n%% userdevguide-begin dht_node:start\n%% @doc joins this node in the ring and calls the main loop\n-spec init(Options::[tuple()])\n        -> dht_node_state:state() |\n           {'$gen_component', [{on_handler, Handler::gen_component:handler()}], State::dht_node_join:join_state()}.\ninit(Options) ->\n    {my_sup_dht_node_id, MySupDhtNode} = lists:keyfind(my_sup_dht_node_id, 1, Options),\n    erlang:put(my_sup_dht_node_id, MySupDhtNode),\n    % start trigger here to prevent infection when tracing e.g. node joins\n    % (otherwise the trigger would be started at the end of the join and thus\n    % be infected forever)\n    % NOTE: any trigger started here, needs an exception for queuing messages\n    %       in dht_node_join to prevent infection with msg_queue:send/1!\n    rm_loop:init_first(),\n    dht_node_move:send_trigger(),\n\n    Recover = config:read(start_type) =:= recover,\n    dht_node_extensions:init(Options),\n    case {is_first(Options), config:read(leases), Recover, is_add_nodes(Options)} of\n        {_   , true, true, false} ->\n            % we are recovering\n            dht_node_join_recover:join(Options);\n        {true, true, false, _} ->\n            msg_delay:send_trigger(1, {l_on_cseq, renew_leases}),\n            Id = l_on_cseq:id(intervals:all()),\n            TmpState = dht_node_join:join_as_first(Id, 0, Options),\n            %% we have to inject the first lease by hand, as otherwise\n            %% no routing will work.\n            l_on_cseq:add_first_lease_to_db(Id, TmpState);\n        {false, true, _, true} ->\n            msg_delay:send_trigger(1, {l_on_cseq, renew_leases}),\n            % get my ID (if set, otherwise chose a random ID):\n            Id = case lists:keyfind({dht_node, id}, 1, Options) of\n                     {{dht_node, id}, IdX} -> IdX;\n                     _ -> ?RT:get_random_node_id()\n                 end,\n            dht_node_join:join_as_other(Id, 0, Options);\n        {IsFirst, _, _, _} ->\n            % get my ID (if set, otherwise chose a random ID):\n            Id = case lists:keyfind({dht_node, id}, 1, Options) of\n                     {{dht_node, id}, IdX} -> IdX;\n                     _ -> ?RT:get_random_node_id()\n                 end,\n            case {IsFirst, modr:is_enabled()} of\n                {true, _} -> dht_node_join:join_as_first(Id, 0, Options);\n                {false, false} -> dht_node_join:join_as_other(Id, 0, Options);\n                {false, true} -> %% disable passive lb during join operation\n                    dht_node_join:join_as_other(Id, 0, [{skip_psv_lb} | Options])\n            end\n    end.\n%% userdevguide-end dht_node:start\n\n%% userdevguide-begin dht_node:start_link\n%% @doc spawns a scalaris node, called by the scalaris supervisor process\n-spec start_link(pid_groups:groupname(), [tuple()]) -> {ok, pid()}.\nstart_link(DHTNodeGroup, Options) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, Options,\n                             [{pid_groups_join_as, DHTNodeGroup, dht_node},\n                              {wait_for_init},\n                              {spawn_opts, [{fullsweep_after, 0},\n                                            {min_heap_size, 131071}]}]).\n%% userdevguide-end dht_node:start_link\n\n%% @doc Checks whether this VM is marked as first, e.g. in a unit test, and\n%%      this is the first node in this VM.\n-spec is_first([tuple()]) -> boolean().\nis_first(Options) ->\n    lists:member({first}, Options) andalso admin_first:is_first_vm().\n\n-spec is_add_nodes([tuple()]) -> boolean().\nis_add_nodes(Options) ->\n    lists:member({add_node}, Options).\n\n-spec is_alive(State::dht_node_join:join_state() | dht_node_state:state() | term()) -> boolean().\nis_alive(State) ->\n    erlang:is_tuple(State) andalso element(1, State) =:= state.\n\n-spec is_alive_no_slide(State::dht_node_join:join_state() | dht_node_state:state() | term()) -> boolean().\nis_alive_no_slide(State) ->\n    try\n        SlidePred = dht_node_state:get(State, slide_pred), % note: this also tests dht_node_state:state()\n        SlideSucc = dht_node_state:get(State, slide_succ),\n        SlidePred =:= null andalso SlideSucc =:= null\n    catch _:_ -> false\n    end.\n\n-spec is_alive_fully_joined(State::dht_node_join:join_state() | dht_node_state:state() | term()) -> boolean().\nis_alive_fully_joined(State) ->\n    try\n        SlidePred = dht_node_state:get(State, slide_pred), % note: this also tests dht_node_state:state()\n        (SlidePred =:= null orelse not slide_op:is_join(SlidePred, 'rcv'))\n    catch _:_ -> false\n    end.\n\n-spec update_key_entries(Entries::[{?RT:key(), db_dht:value(), client_version()}],\n                         DB, dht_node_state:state(), NewEntries) -> {DB, NewEntries}\n    when is_subtype(DB, db_dht:db()),\n         is_subtype(NewEntries, [{db_entry:entry(), Exists::boolean(), Done::boolean()}]).\nupdate_key_entries([], DB, _State, NewEntries) ->\n    {DB, lists:reverse(NewEntries)};\nupdate_key_entries([{Key, NewValue, NewVersion} | Entries], DB, State, NewEntries) ->\n    IsResponsible = dht_node_state:is_db_responsible(Key, State),\n    Entry = db_dht:get_entry(DB, Key),\n    Exists = not db_entry:is_null(Entry),\n    EntryVersion = db_entry:get_version(Entry),\n    WL = db_entry:get_writelock(Entry),\n    DoUpdate = Exists\n                   andalso EntryVersion =/= -1\n                   andalso EntryVersion < NewVersion\n                   andalso (WL =:= false orelse WL < NewVersion)\n                   andalso IsResponsible,\n    DoRegen = (not Exists) andalso IsResponsible,\n%%     log:pal(\"update_key_entries:~nold: ~p~nnew: ~p~nDoUpdate: ~w, DoRegen: ~w\",\n%%             [{db_entry:get_key(Entry), db_entry:get_version(Entry)},\n%%              {Key, NewVersion}, DoUpdate, DoRegen]),\n    if\n        DoUpdate ->\n            UpdEntry = db_entry:set_value(Entry, NewValue, NewVersion),\n            NewEntry = if WL < NewVersion -> db_entry:reset_locks(UpdEntry);\n                          true -> UpdEntry\n                       end,\n            NewDB = db_dht:update_entry(DB, NewEntry),\n            ok;\n        DoRegen ->\n            NewEntry = db_entry:new(Key, NewValue, NewVersion),\n            NewDB = db_dht:set_entry(DB, NewEntry),\n            ok;\n        true ->\n            NewDB = DB,\n            NewEntry = Entry,\n            ok\n    end,\n    update_key_entries(Entries, NewDB, State,\n                       [{NewEntry, Exists, DoUpdate orelse DoRegen} | NewEntries]).\n"
  },
  {
    "path": "src/dht_node_cache.erl",
    "content": "% @copyright 2012-2018 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    This gen_component can be used to cache the intervals dht_nodes\n%%         are responsible for. This can be used to send messages directyl to\n%%         the dht_nodes, circumventing the routing mechanism. Is the responsible\n%%         node not known yet, a separate requests is used to find the responsible\n%%         node which used the configured routing protocol.\n%%\n%%         Note that ring reconfigurations or node failures, interval changes etc.\n%%         are not handled by this module! This can lead to inconsistent lookups!\n%%\n%%         This component is intendet to be used in certain benchmarking settings\n%%         in which the overhead of the underlying routing mechanism is not desired.\n%% @end\n-module(dht_node_cache).\n-author('skrzypczak.de').\n\n%-define(TRACE(X,Y), ct:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n-export([start_link/1]).\n-export([init/1, on/2]).\n\n-export([cached_send/2, cached_send/3]).\n-export([cached_send_to_local_replica/3, cached_send_to_local_replica/4]).\n\n-include(\"gen_component.hrl\").\n\n-type undelivered_msg() ::  {{local, ?RT:key(), non_neg_integer()}, comm:message()} |\n                            {?RT:key(), comm:message()}.\n-type cached_dht_node() ::  {intervals:interval(), comm:mypid()}.\n-type local_dht_node() ::   {unknown, unknown, unknown} |\n                            {intervals:interval(), comm:mypid(), comm:erl_local_pid()}.\n-type state() :: {\n                    [undelivered_msg()],\n                    [cached_dht_node()],\n                    local_dht_node()\n                 }.\n\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                             [{pid_groups_join_as, DHTNodeGroup, dht_node_cache}]).\n\n-spec init([]) -> state().\ninit([]) -> {[], [], {unknown, unknown, unknown}}.\n\n\n%%%%%%%%% API %%%%%%%%%%%\n\n%% @doc Sends message Message to the dht_node responsible for key Key. Is the\n%% dht_node is not known, it is fetched along with its interval and added\n%% to the cache.\n-spec cached_send(?RT:key(), comm:message(), non_neg_integer()) -> ok.\ncached_send(Key, Message, LookupEnvPosition) ->\n    NewMessage = setelement(LookupEnvPosition, Message, false),\n    cached_send(Key, NewMessage).\n\n%% @doc Sends message Message to the dht_node responsible for key Key. Is the\n%% dht_node is not known, it is fetched along with its interval and added\n%% to the cache.\n-spec cached_send(?RT:key(), comm:message()) -> ok.\ncached_send(Key, Message) ->\n    Cache = pid_groups:find_a(dht_node_cache),\n    comm:send_local(Cache, {cached_send, Key, Message}).\n\n%% @doc Sends a single message Message to the local dht_node if it is reponsible\n%% for at least one replica based on replication:get_keys(Key). If not, the message\n%% is sent to a remote dht_node responsible for any of the replica keys.\n%% Note: If the local node is reponsible for more than one key, any of them may be used.\n-spec cached_send_to_local_replica(?RT:key(), non_neg_integer(), comm:message(),\n                                   non_neg_integer()) -> ok.\ncached_send_to_local_replica(Key, KeyPos, Message, LookupEnvPosition) ->\n    NewMessage = setelement(LookupEnvPosition, Message, false),\n    cached_send_to_local_replica(Key, KeyPos, NewMessage).\n\n%% @doc Sends a single message Message to the local dht_node if it is reponsible\n%% for at least one replica based on replication:get_keys(Key). If not, the message\n%% is sent to an other dht_node responsible for any of the replica keys.\n%% Note: If the local node is reponsible for more than one key, any of them may be used.\n-spec cached_send_to_local_replica(?RT:key(), non_neg_integer(), comm:message()) -> ok.\ncached_send_to_local_replica(Key, KeyPos, Message) ->\n    Cache = pid_groups:find_a(dht_node_cache),\n    comm:send_local(Cache, {cached_send_to_local_replica, Key, KeyPos, Message}).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec on(comm:message(), state()) -> state().\non({cached_send, Key, Msg},  State) ->\n    {_Interval, LocalDhtNodeGlobalPid, LocalDhtNodeLocalPid} = local_dht_node(State),\n    case responsible_dht_node(State, Key) of\n        unknown ->\n            NewState = add_undelivered_message(State, Key, Msg),\n\n            Dest = pid_groups:find_a(routing_table),\n            LookupEnvelope = dht_node_lookup:envelope(4, {cache_interval_lookup,\n                                                        Key, comm:this(), '_'}),\n            comm:send_local(Dest, {?lookup_aux, Key, 0, LookupEnvelope}),\n\n            NewState;\n        LocalDhtNodeGlobalPid ->\n            comm:send_local(LocalDhtNodeLocalPid, Msg),\n            State;\n        DhtNodePid ->\n            comm:send(DhtNodePid, Msg),\n            State\n    end;\n\non({cache_interval_lookup_reply, DhtNodePid, _Key, Interval}, State) ->\n    {NewState, PoppedMsgs} = pop_undelivered_messages(State, Interval),\n    [comm:send(DhtNodePid, Msg) || {_K, Msg} <- PoppedMsgs],\n\n    LocalDhtNode = pid_groups:find_a(dht_node),\n    NewState2 =\n        case DhtNodePid =:= comm:make_global(LocalDhtNode) of\n            true -> set_local_dht_node(State, DhtNodePid, LocalDhtNode, Interval);\n            false -> NewState\n        end,\n    add_dht_node(NewState2, DhtNodePid, Interval);\n\non({cached_send_to_local_replica, Key, KeyPos, Msg}, State) ->\n    {Interval, _GloalPid, LocalDhtNodeLocalPid} = local_dht_node(State),\n\n    case Interval of\n       unknown ->\n            NewState = add_undelivered_message(State, {local, Key, KeyPos}, Msg),\n\n            Dest = pid_groups:find_a(dht_node),\n            This = comm:reply_as(comm:this(), 3, {local_send, Dest, '_'}),\n            comm:send_local(Dest, {cache_interval_lookup, Key, This, _Cons=false}),\n\n            NewState;\n        _ ->\n           send_to_local_if_possible(Key, KeyPos, Msg, LocalDhtNodeLocalPid, Interval),\n           State\n    end;\n\non({local_send, LocalPid, {cache_interval_lookup_reply, GlobalPid, _Key, Interval}}, State) ->\n    NewState = set_local_dht_node(State, GlobalPid, LocalPid, Interval),\n    {NewState2, LocalMsgs} = pop_undelivered_local_messages(NewState),\n    [send_to_local_if_possible(Key, KeyPos, Msg, LocalPid, Interval) ||\n     {{local, Key, KeyPos}, Msg} <- LocalMsgs],\n\n    NewState2.\n\nsend_to_local_if_possible(Key, KeyPos, Msg, LocalDhtPid, LocalInterval) ->\n    Keys = replication:get_keys(Key),\n    LocalKeys = lists:filter(fun(K) -> intervals:in(K, LocalInterval) end, Keys),\n    case LocalKeys of\n        [] ->\n            %% sending locally is not possible as the local dht node\n            %% does not manage any replica... use random key\n            Idx = randoms:rand_uniform(1, length(Keys)+1),\n            TargetKey = lists:nth(Idx, Keys),\n            NewMsg = setelement(KeyPos, Msg, TargetKey),\n            comm:send_local(self(), {cached_send, TargetKey, NewMsg});\n        _ ->\n            TargetKey = hd(LocalKeys),\n            NewMsg = setelement(KeyPos, Msg, TargetKey),\n            comm:send_local(LocalDhtPid, NewMsg)\n    end.\n\n-spec add_undelivered_message(state(), {local, ?RT:key(), non_neg_integer()} |\n                              ?RT:key(), comm:message()) -> state().\nadd_undelivered_message(State, Key, Msg) ->\n    MsgList = element(1, State),\n    setelement(1, State, [{Key, Msg} | MsgList]).\n\n-spec pop_undelivered_messages(state(), intervals:interval()) -> {state(), [undelivered_msg()]}.\npop_undelivered_messages(State, Interval) ->\n    AllMsgs = element(1, State),\n    {Matching, Rest} = lists:partition(fun({Key, _Msg}) ->\n                                            intervals:in(Key, Interval)\n                                       end, AllMsgs),\n    {setelement(1, State, Rest), Matching}.\n\n-spec pop_undelivered_local_messages(state()) -> {state(), [undelivered_msg()]}.\npop_undelivered_local_messages(State) ->\n    AllMsgs = element(1, State),\n    {Matching, Rest} = lists:partition(fun({{local, _Key, _KeyPos}, _Msg}) -> true;\n                                          ({_Key, _Msg}) -> false\n                                       end, AllMsgs),\n    {setelement(1, State, Rest), Matching}.\n\n-spec add_dht_node(state(), comm:mypid(), intervals:interval()) -> state().\nadd_dht_node(State, DhtNodePid, Interval) ->\n    DhtList = element(2, State),\n    setelement(2, State, [{Interval, DhtNodePid} | DhtList]).\n\n-spec responsible_dht_node(state() | [cached_dht_node()], ?RT:key()) -> unknown | comm:mypid().\nresponsible_dht_node([], _Key) -> unknown;\nresponsible_dht_node([{Interval, Pid} | T], Key) ->\n    case intervals:in(Key, Interval) of\n        true -> Pid;\n        false -> responsible_dht_node(T, Key)\n    end;\nresponsible_dht_node(State, Key) ->\n    responsible_dht_node(element(2, State), Key).\n\n-spec set_local_dht_node(state(), comm:mypid(), comm:erl_local_pid(), intervals:interval()) -> state().\nset_local_dht_node(State, GlobalPid, LocalPid, Interval) ->\n    setelement(3, State, {Interval, GlobalPid, LocalPid}).\n-spec local_dht_node(state()) -> local_dht_node().\nlocal_dht_node(State) -> element(3, State).\n\n"
  },
  {
    "path": "src/dht_node_db_cache.erl",
    "content": "%  @copyright 2013-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Caches some DB values that are not relevant for consistence.\n%% @end\n%% @version $Id$\n-module(dht_node_db_cache).\n\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n-export([start_link/1]).\n-export([init/1, on/2]).\n\n-include(\"gen_component.hrl\").\n\n-type(message() ::\n    {get_split_key, DB::db_dht:db(), CurRange::intervals:interval(),\n     Begin::?RT:key(), End::?RT:key(),\n     TargetLoad::pos_integer(), Direction::forward | backward, SourcePid::comm:erl_local_pid()} |\n    {get_split_key_wrapper, SourcePid::comm:erl_local_pid(), DB::db_dht:db(),\n     CurRange::intervals:interval(), {get_split_key_response, Val::{?RT:key(),\n     TakenLoad::non_neg_integer()}}} |\n    {web_debug_info, Requestor::comm:erl_local_pid()}).\n\n-type state() :: [{DB::db_dht:db(), Range::intervals:interval(), Expires::erlang_timestamp(),\n                   Key::get_split_key, Val::{?RT:key(), TakenLoad::non_neg_integer()}}].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Startup\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Starts a db_dht cache process, registers it with the process\n%%      dictionary and returns its pid for use by a supervisor.\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                             [{pid_groups_join_as, DHTNodeGroup, ?MODULE}]).\n\n%% @doc Initialises the module with an uninitialized state.\n-spec init([]) -> state().\ninit([]) ->\n    [].\n      \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Message Loop\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec on(message(), state()) -> state().\non({get_split_key, DB, CurRange, Begin, End, TargetLoad, Direction, SourcePid}, State) ->\n    Now = util:timestamp2us(os:timestamp()),\n    case lists:keyfind(get_split_key, 4, State) of\n        {DB, CurRange, Expires, get_split_key, Val} when Now < Expires ->\n            comm:send_local(SourcePid, {get_split_key_response, Val}),\n            State;\n        _ ->\n            Sender = comm:reply_as(self(), 5,\n                                   {get_split_key_wrapper, SourcePid, DB, CurRange, '_'}),\n            comm:send_local(pid_groups:get_my(dht_node),\n                            {get_split_key, DB, Begin, End,\n                             TargetLoad, Direction, Sender}),\n            State\n    end;\n\non({get_split_key_wrapper, SourcePid, DB, CurRange, {get_split_key_response, Val}}, State) ->\n    comm:send_local(SourcePid, {get_split_key_response, Val}),\n    Now = util:timestamp2us(os:timestamp()),\n    case lists:keyfind(get_split_key, 4, State) of\n        {DB, CurRange, Expires, get_split_key, Val} when Now < Expires ->\n            State;\n        _ ->\n            CachedVal = {DB, CurRange, Now + 10 * 1000000, % 10s\n                         get_split_key, Val},\n            lists:keystore(get_split_key, 4, State, CachedVal)\n    end;\n\non({web_debug_info, Requestor}, State) ->\n    KeyValueList =\n        [{\"Cached values:\", \"\"} |\n             [{webhelpers:safe_html_string(\n                 \"~.0p (Expires: ~.0p UTC)\",\n                 [Key, calendar:now_to_universal_time(util:us2timestamp(Expires))]),\n               webhelpers:safe_html_string(\"~.0p\", [Val])}\n             || {_DB, _Range, Expires, Key, Val} <- State]],\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    State.\n"
  },
  {
    "path": "src/dht_node_extensions.erl",
    "content": "%  @copyright 2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    dht node extensions\n%% @end\n-module(dht_node_extensions).\n-author('schintke@zib.de').\n-author('schuett@zib.de').\n\n-include(\"scalaris.hrl\").\n\n-export([on/2, init/1, wrap/2]).\n\n-export_type([extension/0]).\n\n-type extension() :: atom().\n\n-spec wrap(extension(), comm:message()) -> {atom(), {extension(), comm:message()}}.\nwrap(Extension, Message) ->\n    {extensions, {Extension, Message}}.\n\n-spec on({extensions, {extension(), comm:message()}}, dht_node_state:state()) -> dht_node_state:state().\non({extensions, {Extension, Message}}, State) ->\n    Extension:on(Message, State).\n\n-spec init(any()) -> ok.\ninit(Options) ->\n    Extensions = config:read(extensions),\n    _ = [Extension:init(Options) || Extension <- Extensions],\n    ok.\n"
  },
  {
    "path": "src/dht_node_join.erl",
    "content": "%  @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    dht_node join procedure\n%% @end\n%% @version $Id$\n-module(dht_node_join).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-define(TRACE_SEND(Pid, Msg), ?TRACE(\"[ ~.0p ] to ~.0p: ~.0p)~n\", [self(), Pid, Msg])).\n-define(TRACE_JOIN1(Msg, JoinState),\n        ?TRACE(\"[ ~.0p ]~n  Msg: ~.0p~n  Phase: ~.0p~n  JoinUUID: ~.0p\"\n               \"~n  Version: ~.0p~n  Contacts: ~.0p~n  JoinIDs: ~.0p\"\n               \"~n  Candidates: ~.0p~n\",\n               [self(), Msg, get_phase(JoinState), get_join_uuid(JoinState), get_id_version(JoinState),\n                lists:sublist(lists:reverse(get_connections(JoinState)), erlang:min(5, length(get_connections(JoinState)))),\n                get_join_ids(JoinState),\n                lists:sublist(get_candidates(JoinState), erlang:min(5, length(get_candidates(JoinState))))])).\n-define(TRACE1(Msg, State),\n        ?TRACE(\"[ ~.0p ]~n  Msg: ~.0p~n  State: ~.0p)~n\", [self(), Msg, State])).\n-define(TRACE_JOIN_STATE(State), ?TRACE(\"[ ~.0p ]~n  Phase: ~.0p~n  JoinUUID: ~.0p\"\n        \"~n  Version: ~.0p~n  Contacts: ~.0p~n  JoinIDs: ~.0p\"\n        \"~n  Candidates: ~.0p~n\",\n        [self(), get_phase(State), get_join_uuid(State), get_id_version(State),\n         lists:sublist(lists:reverse(get_connections(State)), erlang:min(5, length(get_connections(State)))),\n         get_join_ids(State),\n         lists:sublist(get_candidates(State), erlang:min(5, length(get_candidates(State))))])).\n\n-define(VALID_PASSIVE_ALGORITHMS, [lb_psv_simple, lb_psv_split, lb_psv_gossip]).\n\n-export([join_as_first/3, join_as_other/3,\n         process_join_state/2, process_join_msg/2, check_config/0]).\n\n% for join_leave_SUITE:\n-export([reject_join_response/4]).\n\n-export_type([join_state/0, join_message/0, connection/0]).\n\n-include(\"scalaris.hrl\").\n\n-type connection() :: {null | pos_integer(), comm:mypid()}.\n\n-type(join_message() ::\n    % messages at the joining node:\n    {join, start} |\n    {get_dht_nodes_response, Nodes::[node:node_type()]} |\n    {join, get_number_of_samples, Samples::non_neg_integer(), Conn::connection()} |\n    {join, get_candidate_response, OrigJoinId::?RT:key(), Candidate::lb_op:lb_op(), Conn::connection()} |\n    {join, join_response, Succ::node:node_type(), Pred::node:node_type(), MoveFullId::slide_op:id(),\n     CandId::lb_op:id(), TargetId::?RT:key(), NextOp::slide_op:next_op()} |\n    {join, join_response, not_responsible | busy, CandId::lb_op:id()} |\n    {join, known_hosts_timeout, JoinUUId::pos_integer()} |\n    {join, lookup_timeout, Conn::connection(), JoinId::?RT:key(), JoinUUId::pos_integer()} |\n    {join, get_number_of_samples_timeout, Conn::connection(), JoinUUId::pos_integer()} |\n    {join, join_request_timeout, Timeouts::non_neg_integer(), CandId::lb_op:id(), JoinUUId::pos_integer()} |\n    {join, timeout, JoinUUId::pos_integer()} |\n    % messages at the existing node:\n    {join, number_of_samples_request, SourcePid::comm:mypid(), LbPsv::module(), Conn::connection()} |\n    {join, get_candidate, Source_PID::comm:mypid(), Key::?RT:key(), LbPsv::module(), Conn::connection()} |\n    {join, join_request, NewPred::node:node_type(), CandId::lb_op:id(), MaxTransportEntries::unknown | pos_integer()} |\n    {join, LbPsv::module(),\n     Msg::lb_psv_simple:custom_message() | lb_psv_split:custom_message() |\n          lb_psv_gossip:custom_message(),\n     LbPsvState::term()}\n    ).\n\n-type phase2() ::\n    {phase2,  JoinUUId::pos_integer(), Options::[tuple()], MyKeyVersion::non_neg_integer(),\n     Connections::[{null | pos_integer(), comm:mypid()}],\n     JoinIds::[?RT:key()], Candidates::[lb_op:lb_op()]}.\n-type phase2b() ::\n    {phase2b, JoinUUId::pos_integer(), Options::[tuple()], MyKeyVersion::non_neg_integer(),\n     Connections::[{null | pos_integer(), comm:mypid()},...],\n     JoinIds::[?RT:key()], Candidates::[lb_op:lb_op()]}.\n-type phase3() ::\n    {phase3,  JoinUUId::pos_integer(), Options::[tuple()], MyKeyVersion::non_neg_integer(),\n     Connections::[{null | pos_integer(), comm:mypid()}],\n     JoinIds::[?RT:key()], Candidates::[lb_op:lb_op()]}.\n-type phase4() ::\n    {phase4,  JoinUUId::pos_integer(), Options::[tuple()], MyKeyVersion::non_neg_integer(),\n     Connections::[{null | pos_integer(), comm:mypid()}],\n     JoinIds::[?RT:key()], Candidates::[lb_op:lb_op()]}.\n-type phase_2_4() :: phase2() | phase2b() | phase3() | phase4().\n\n-type join_state() ::\n    {join, {phase1,  JoinUUId::pos_integer(), Options::[tuple()], MyKeyVersion::non_neg_integer(),\n            Connections::[], JoinIds::[?RT:key()], Candidates::[]},\n     QueuedMessages::msg_queue:msg_queue()} |\n    {join, phase_2_4(), QueuedMessages::msg_queue:msg_queue()}.\n\n%% userdevguide-begin dht_node_join:join_as_first\n-spec join_as_first(Id::?RT:key(), IdVersion::non_neg_integer(), Options::[tuple()])\n        -> dht_node_state:state().\njoin_as_first(Id, IdVersion, _Options) ->\n    log:log(info, \"[ Node ~w ] joining as first: (~.0p, ~.0p)\",\n            [self(), Id, IdVersion]),\n    Me = node:new(comm:this(), Id, IdVersion),\n    % join complete, State is the first \"State\"\n    finish_join(Me, Me, Me, db_dht:new(db_dht), msg_queue:new(), []).\n%% userdevguide-end dht_node_join:join_as_first\n\n%% userdevguide-begin dht_node_join:join_as_other\n-spec join_as_other(Id::?RT:key(), IdVersion::non_neg_integer(), Options::[tuple()])\n        -> {'$gen_component', [{on_handler, Handler::gen_component:handler()}],\n            State::{join, phase2(), msg_queue:msg_queue()}}.\njoin_as_other(Id, IdVersion, Options) ->\n    log:log(info,\"[ Node ~w ] joining, trying ID: (~.0p, ~.0p)\",\n            [self(), Id, IdVersion]),\n    JoinUUID = uid:get_pids_uid(),\n    gen_component:change_handler(\n      {join, {phase1, JoinUUID, Options, IdVersion, [], [Id], []},\n       msg_queue:new()},\n      fun ?MODULE:process_join_state/2).\n%% userdevguide-end dht_node_join:join_as_other\n\n% join protocol\n%% @doc Process a DHT node's join messages during the join phase.\n-spec process_join_state(dht_node:message(), join_state())\n        -> join_state() |\n           {'$gen_component', [{on_handler, Handler::gen_component:handler()}], dht_node_state:state()}.\n% !first node\n% start join (starting via message allows tracing with trace_mpath)\nprocess_join_state({join, start} = _Msg,\n                   {join, JoinState, QueuedMessages}) ->\n    JoinUUID = element(2, JoinState),\n    get_known_nodes(JoinUUID),\n    msg_delay:send_local(get_join_timeout() div 1000, self(),\n                         {join, timeout, JoinUUID}),\n    {join, set_phase(phase2, JoinState), QueuedMessages};\n% 2. Find known hosts\n% no matter which phase, if there are no contact nodes (yet), try to get some\nprocess_join_state({join, known_hosts_timeout, JoinUUId} = _Msg,\n                   {join, JoinState, _QueuedMessages} = State)\n  when element(2, JoinState) =:= JoinUUId ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    case get_connections(JoinState) of\n        [] -> get_known_nodes(JoinUUId);\n        [_|_] -> ok\n    end,\n    State;\n\n% ignore unrelated known_hosts_timeout:\nprocess_join_state({join, known_hosts_timeout, _JoinId} = _Msg,\n                   {join, _JoinState, _QueuedMessages} = State) ->\n    ?TRACE_JOIN1(_Msg, _JoinState),\n    State;\n\n%% userdevguide-begin dht_node_join:join_other_p2\n% in phase 2 add the nodes and do lookups with them / get number of samples\nprocess_join_state({get_dht_nodes_response, Nodes} = _Msg,\n                   {join, JoinState, QueuedMessages})\n  when element(1, JoinState) =:= phase2 ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    JoinOptions = get_join_options(JoinState),\n    %% additional nodes required when firstnode jumps and he's the only known host\n    DhtNodes = Nodes ++ proplists:get_value(bootstrap_nodes, JoinOptions, []),\n    Connections = [{null, Node} || Node <- DhtNodes, Node =/= comm:this()],\n    JoinState1 = add_connections(Connections, JoinState, back),\n    NewJoinState = phase2_next_step(JoinState1, Connections),\n    ?TRACE_JOIN_STATE(NewJoinState),\n    {join, NewJoinState, QueuedMessages};\n\n% in all other phases, just add the provided nodes:\nprocess_join_state({get_dht_nodes_response, Nodes} = _Msg,\n                   {join, JoinState, QueuedMessages})\n  when element(1, JoinState) =:= phase2b orelse\n           element(1, JoinState) =:= phase3 orelse\n           element(1, JoinState) =:= phase4 ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    Connections = [{null, Node} || Node <- Nodes, Node =/= comm:this()],\n    JoinState1 = add_connections(Connections, JoinState, back),\n    ?TRACE_JOIN_STATE(JoinState1),\n    {join, JoinState1, QueuedMessages};\n%% userdevguide-end dht_node_join:join_other_p2\n\n% 2b. get the number of nodes/ids to sample\n\n% on timeout:\n%  - in phase 2b, remove failed connection, start over\n%  - in other phases, ignore this timeout (but remove the connection)\nprocess_join_state({join, get_number_of_samples_timeout, Conn, JoinUUId} = _Msg,\n                   {join, JoinState, QueuedMessages})\n  when element(2, JoinState) =:= JoinUUId ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    JoinState1 = remove_connection(Conn, JoinState),\n    NewJoinState = case get_phase(JoinState1) of\n                       phase2b -> start_over(JoinState1);\n                       _       -> JoinState1\n                   end,\n    ?TRACE_JOIN_STATE(NewJoinState),\n    {join, NewJoinState, QueuedMessages};\n\nprocess_join_state({join, get_number_of_samples_timeout, _Conn, _JoinId} = _Msg,\n                   {join, _JoinState, _QueuedMessages} = State) ->\n    ?TRACE_JOIN1(_Msg, _JoinState),\n    State;\n\n%% userdevguide-begin dht_node_join:join_other_p2b\n% note: although this message was send in phase2b, also accept message in\n% phase2, e.g. messages arriving from previous calls\nprocess_join_state({join, get_number_of_samples, Samples, Conn} = _Msg,\n                   {join, JoinState, QueuedMessages})\n  when element(1, JoinState) =:= phase2 orelse\n           element(1, JoinState) =:= phase2b ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    % prefer node that send get_number_of_samples as first contact node\n    JoinState1 = reset_connection(Conn, JoinState),\n    % (re-)issue lookups for all existing IDs and\n    % create additional samples, if required\n    NewJoinState = lookup_new_ids2(Samples, JoinState1),\n    ?TRACE_JOIN_STATE(NewJoinState),\n    {join, NewJoinState, QueuedMessages};\n\n% ignore message arriving in other phases:\nprocess_join_state({join, get_number_of_samples, _Samples, Conn} = _Msg,\n                   {join, JoinState, QueuedMessages}) ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    NewJoinState = reset_connection(Conn, JoinState),\n    ?TRACE_JOIN_STATE(NewJoinState),\n    {join, NewJoinState, QueuedMessages};\n%% userdevguide-end dht_node_join:join_other_p2b\n\n% 3. lookup all positions\nprocess_join_state({join, lookup_timeout, Conn, Id, JoinUUId} = _Msg,\n                   {join, JoinState, QueuedMessages})\n  when element(1, JoinState) =:= phase3 andalso\n           element(2, JoinState) =:= JoinUUId ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    % do not know whether the contact node is dead or the lookup takes too long\n    % -> simple solution: try with an other contact node, remove the current one\n    JoinState1 = remove_connection(Conn, JoinState),\n    NewJoinState = lookup(JoinState1, [Id]),\n    ?TRACE_JOIN_STATE(NewJoinState),\n    {join, NewJoinState, QueuedMessages};\n\n% only remove the failed connection in other phases:\nprocess_join_state({join, lookup_timeout, Conn, _Id, JoinUUId} = _Msg,\n                   {join, JoinState, QueuedMessages})\n  when element(2, JoinState) =:= JoinUUId ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    NewJoinState = remove_connection(Conn, JoinState),\n    ?TRACE_JOIN_STATE(NewJoinState),\n    {join, NewJoinState, QueuedMessages};\n\n% ignore unrelated lookup_timeout messages:\nprocess_join_state({join, lookup_timeout, _Conn, _Id, _JoinId} = _Msg,\n                   {join, _JoinState, _QueuedMessages} = State) ->\n    ?TRACE_JOIN1(_Msg, _JoinState),\n    State;\n\n%% userdevguide-begin dht_node_join:join_other_p3\nprocess_join_state({join, get_candidate_response, OrigJoinId, Candidate, Conn} = _Msg,\n                   {join, JoinState, QueuedMessages})\n  when element(1, JoinState) =:= phase3 ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    JoinState0 = reset_connection(Conn, JoinState),\n    JoinState1 = remove_join_id(OrigJoinId, JoinState0),\n    JoinState2 = integrate_candidate(Candidate, JoinState1, front),\n    NewJoinState =\n        case get_join_ids(JoinState2) of\n            [] -> % no more join ids to look up -> join with the best:\n                contact_best_candidate(JoinState2);\n            [_|_] -> % still some unprocessed join ids -> wait\n                JoinState2\n        end,\n    ?TRACE_JOIN_STATE(NewJoinState),\n    {join, NewJoinState, QueuedMessages};\n\n% In phase 2 or 2b, also add the candidate but do not continue.\n% In phase 4, add the candidate to the end of the candidates as they are sorted\n% and the join with the first has already started (use this candidate as backup\n% if the join fails). Do not start a new join.\nprocess_join_state({join, get_candidate_response, OrigJoinId, Candidate, Conn} = _Msg,\n                   {join, JoinState, QueuedMessages})\n  when element(1, JoinState) =:= phase2 orelse\n           element(1, JoinState) =:= phase2b orelse\n           element(1, JoinState) =:= phase4 ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    JoinState0 = reset_connection(Conn, JoinState),\n    JoinState1 = remove_join_id(OrigJoinId, JoinState0),\n    JoinState2 = case get_phase(JoinState1) of\n                     phase4 -> integrate_candidate(Candidate, JoinState1, back);\n                     _      -> integrate_candidate(Candidate, JoinState1, front)\n                 end,\n    ?TRACE_JOIN_STATE(JoinState2),\n    {join, JoinState2, QueuedMessages};\n%% userdevguide-end dht_node_join:join_other_p3\n\n% 4. joining my neighbor\nprocess_join_state({join, join_request_timeout, Timeouts, CandId, JoinUUId} = _Msg,\n                   {join, JoinState, QueuedMessages} = State)\n  when element(1, JoinState) =:= phase4 andalso\n           element(2, JoinState) =:= JoinUUId ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    case get_candidates(JoinState) of\n        [] -> State; % no candidates -> late timeout, ignore\n        [BestCand | _] ->\n            case lb_op:get(BestCand, id) =:= CandId of\n                false -> State; % unrelated/old message\n                _ ->\n                    NewJoinState =\n                        case Timeouts < get_join_request_timeouts() of\n                            true ->\n                                send_join_request(JoinState, Timeouts + 1);\n                            _ ->\n                                % no response from responsible node\n                                % -> select new candidate, try again\n                                log:log(warn, \"[ Node ~w ] no response on join \"\n                                              \"request for the chosen ID, \"\n                                              \"trying next candidate\", [self()]),\n                                try_next_candidate(JoinState)\n                        end,\n                    ?TRACE_JOIN_STATE(NewJoinState),\n                    {join, NewJoinState, QueuedMessages}\n            end\n    end;\n\n% ignore late or unrelated join_request_timeout message:\nprocess_join_state({join, join_request_timeout, _Timeouts, _CandId, _JoinId} = _Msg,\n                   {join, _JoinState, _QueuedMessages} = State) ->\n    ?TRACE_JOIN1(_Msg, _JoinState),\n    State;\n\n%% userdevguide-begin dht_node_join:join_other_p4\nprocess_join_state({join, join_response, Reason, CandId} = _Msg,\n                   {join, JoinState, QueuedMessages} = State)\n  when element(1, JoinState) =:= phase4 andalso\n           (Reason =:= not_responsible orelse Reason =:= busy) ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    % the node we contacted is not responsible for the selected key anymore\n    % -> try the next candidate, if the message is related to the current candidate\n    case get_candidates(JoinState) of\n        [] -> % no candidates -> should not happen in phase4!\n            log:log(error, \"[ Node ~w ] empty candidate list in join phase 4, \"\n                        \"starting over\", [self()]),\n            NewJoinState = start_over(JoinState),\n            ?TRACE_JOIN_STATE(NewJoinState),\n            {join, NewJoinState, QueuedMessages};\n        [Candidate | _Rest] ->\n            case lb_op:get(Candidate, id) =:= CandId of\n                false -> State; % unrelated/old message\n                _ ->\n                    if Reason =:= not_responsible ->\n                           log:log(info,\n                                   \"[ Node ~w ] node contacted for join is not \"\n                                       \"responsible for the selected ID (anymore), \"\n                                       \"trying next candidate\",\n                                   [self()]);\n                       Reason =:= busy ->\n                           log:log(info,\n                                   \"[ Node ~w ] node contacted for join is busy, \"\n                                       \"trying next candidate\",\n                                   [self()])\n                    end,\n                    NewJoinState = try_next_candidate(JoinState),\n                    ?TRACE_JOIN_STATE(NewJoinState),\n                    {join, NewJoinState, QueuedMessages}\n            end\n    end;\n\n% in other phases remove the candidate from the list (if it still exists):\nprocess_join_state({join, join_response, Reason, CandId} = _Msg,\n                   {join, JoinState, QueuedMessages})\n  when (Reason =:= not_responsible orelse Reason =:= busy) ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    {join, remove_candidate(CandId, JoinState), QueuedMessages};\n\n% note: accept (delayed) join_response messages in any phase\nprocess_join_state({join, join_response, Succ, Pred, MoveId, CandId, TargetId, NextOp} = _Msg,\n                   {join, JoinState, QueuedMessages} = State) ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    % only act on related messages, i.e. messages from the current candidate\n    Phase = get_phase(JoinState),\n    State1 = case get_candidates(JoinState) of\n        [] when Phase =:= phase4 ->\n            % no candidates -> should not happen in phase4!\n            log:log(error, \"[ Node ~w ] empty candidate list in join phase 4, \"\n                           \"starting over\", [self()]),\n            reject_join_response(Succ, Pred, MoveId, CandId),\n            NewJoinState = start_over(JoinState),\n            ?TRACE_JOIN_STATE(NewJoinState),\n            {join, NewJoinState, QueuedMessages};\n        [] ->\n            % in all other phases, ignore the delayed join_response if no\n            % candidates exist\n            reject_join_response(Succ, Pred, MoveId, CandId),\n            State;\n        [Candidate | _Rest] ->\n            CandidateNode = node_details:get(lb_op:get(Candidate, n1succ_new), node),\n            CandidateNodeSame = node:same_process(CandidateNode, Succ),\n            case lb_op:get(Candidate, id) =:= CandId of\n                false ->\n                    % ignore old/unrelated message\n                    log:log(warn, \"[ Node ~w ] ignoring old or unrelated \"\n                                  \"join_response message\", [self()]),\n                    reject_join_response(Succ, Pred, MoveId, CandId),\n                    State;\n                _ when not CandidateNodeSame ->\n                    % id is correct but the node is not (should never happen!)\n                    log:log(error, \"[ Node ~w ] got join_response but the node \"\n                                  \"changed, trying next candidate\", [self()]),\n                    reject_join_response(Succ, Pred, MoveId, CandId),\n                    NewJoinState = try_next_candidate(JoinState),\n                    ?TRACE_JOIN_STATE(NewJoinState),\n                    {join, NewJoinState, QueuedMessages};\n                _ ->\n                    MyId = TargetId,\n                    MyIdVersion = get_id_version(JoinState),\n                    case MyId =:= node:id(Succ) orelse MyId =:= node:id(Pred) of\n                        true ->\n                            log:log(warn, \"[ Node ~w ] chosen ID already exists, \"\n                                          \"trying next candidate\", [self()]),\n                            reject_join_response(Succ, Pred, MoveId, CandId),\n                            % note: can not keep Id, even if skip_psv_lb is set\n                            JoinState1 = remove_candidate_front(JoinState),\n                            NewJoinState = contact_best_candidate(JoinState1),\n                            ?TRACE_JOIN_STATE(NewJoinState),\n                            {join, NewJoinState, QueuedMessages};\n                        _ ->\n                            ?TRACE(\"[ ~.0p ]~n  joined MyId:~.0p, MyIdVersion:~.0p~n  \"\n                                       \"Succ: ~.0p~n  Pred: ~.0p~n\",\n                                       [self(), MyId, MyIdVersion, Succ, Pred]),\n                            Me = node:new(comm:this(), MyId, MyIdVersion),\n                            log:log(info, \"[ Node ~w ] joined between ~w and ~w\",\n                                    [self(), Pred, Succ]),\n                            rm_loop:notify_new_succ(node:pidX(Pred), Me),\n                            rm_loop:notify_new_pred(node:pidX(Succ), Me),\n\n                            JoinOptions = get_join_options(JoinState),\n\n                            finish_join_and_slide(Me, Pred, Succ, db_dht:new(db_dht),\n                                                  QueuedMessages, MoveId, NextOp, JoinOptions)\n                    end\n            end\n    end,\n    State1;\n%% userdevguide-end dht_node_join:join_other_p4\n\n% a join timeout message re-starts the complete join\nprocess_join_state({join, timeout, JoinUUId} = _Msg, {join, JoinState, QueuedMessages})\n  when (element(1, JoinState) =:= phase2 orelse\n            element(1, JoinState) =:= phase2b orelse\n            element(1, JoinState) =:= phase3 orelse\n            element(1, JoinState) =:= phase4) andalso\n           element(2, JoinState) =:= JoinUUId ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    log:log(warn, \"[ Node ~w ] join timeout hit, starting over...\", [self()]),\n    NewJoinState = start_over(JoinState),\n    ?TRACE_JOIN_STATE(NewJoinState),\n    {join, NewJoinState, QueuedMessages};\n\n% ignore unrelated join timeout message:\nprocess_join_state({join, timeout, _JoinId} = _Msg,\n                   {join, _JoinState, _QueuedMessages} = State) ->\n    ?TRACE_JOIN1(_Msg, _JoinState),\n    State;\n\nprocess_join_state({web_debug_info, Requestor} = _Msg,\n                   {join, JoinState, QueuedMessages} = State) ->\n    ?TRACE_JOIN1(_Msg, JoinState),\n    % get a list of up to 50 queued messages to display:\n    MessageListTmp = [{\"\", webhelpers:safe_html_string(\"~p\", [Message])}\n                  || Message <- lists:sublist(QueuedMessages, 50)],\n    MessageList = case length(QueuedMessages) > 50 of\n                      true -> lists:append(MessageListTmp, [{\"...\", \"\"}]);\n                      _    -> MessageListTmp\n                  end,\n    Phase = get_phase(JoinState),\n    StateInfo =\n        case lists:member(Phase, [phase2, phase2b, phase3, phase4]) of\n            true ->\n                [{\"phase\",       Phase},\n                 {\"key_vers\",    get_id_version(JoinState)},\n                 {\"connections\", webhelpers:safe_html_string(\"~p\", [get_connections(JoinState)])},\n                 {\"join_ids\",    webhelpers:safe_html_string(\"~p\", [get_join_ids(JoinState)])},\n                 {\"candidates\",  webhelpers:safe_html_string(\"~p\", [get_candidates(JoinState)])}];\n            _ -> [{\"phase\",      Phase}]\n        end,\n    KeyValueList =\n        [{\"\", \"\"}, {\"joining dht_node process\", \"\"} |\n             lists:append(StateInfo,\n                          [{\"\", \"\"},\n                           {\"queued messages:\", \"\"} | MessageList])],\n    ?TRACE_SEND(Requestor, {web_debug_info_reply, KeyValueList}),\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    State;\n\nprocess_join_state({?lookup_aux, Key, Hops, Msg} = FullMsg,\n                   {join, JoinState, _QueuedMessages} = State) ->\n    case get_connections(JoinState) of\n        [] ->\n            _ = comm:send_local_after(100, self(), FullMsg),\n            ok;\n        [{_, Pid} | _] ->\n            % integrate the list of processes for which the send previously failed:\n            Self = comm:reply_as(self(), 2, {join, '_', []}),\n            comm:send(Pid, {?lookup_aux, Key, Hops + 1, Msg}, [{shepherd, Self}])\n    end,\n    State;\nprocess_join_state({join, {send_error, Target, {?lookup_aux, Key, Hops, Msg}, _Reason},\n                    FailedPids}, {join, JoinState, _QueuedMessages} = State) ->\n    Connections = get_connections(JoinState),\n    case lists:dropwhile(fun({_, Pid}) -> lists:member(Pid, FailedPids) end, Connections) of\n        [] ->\n            _ = comm:send_local_after(100, pid_groups:get_my(routing_table), {?lookup_aux, Key, Hops + 1, Msg}),\n            ok;\n        [{_, Pid} | _] ->\n            % integrate the list of processes for which the send previously failed:\n            Self = comm:reply_as(self(), 2, {join, '_', [Target | FailedPids]}),\n            comm:send(Pid, {?lookup_aux, Key, Hops + 1, Msg}, [{shepherd, Self}])\n    end,\n    State;\nprocess_join_state({join, {send_error, Target, {get_dht_nodes, _This}, _Reason},\n                    FailedPids}, State) ->\n    KnownHosts = config:read(known_hosts),\n    NextHost = case lists:dropwhile(fun(Pid) ->\n                                            lists:member(Pid, FailedPids)\n                                    end, KnownHosts) of\n                   [] -> util:randomelem(KnownHosts);\n                   [X | _] -> X\n               end,\n    % integrate the list of processes for which the send previously failed:\n    Self = comm:reply_as(self(), 2, {join, '_', [Target | FailedPids]}),\n    ?TRACE_SEND(NextHost, {get_dht_nodes, comm:this()}),\n    comm:send(NextHost, {get_dht_nodes, comm:this()}, [{shepherd, Self}]),\n    State;\n\n% do not queue rm_trigger to prevent its infection with msg_queue:send/1\nprocess_join_state({rm, trigger} = _Msg, State) ->\n    ?TRACE_JOIN1(_Msg, element(2, State)),\n    rm_loop:send_trigger(),\n    State;\n\n% do not queue rm_trigger to prevent its infection with msg_queue:send/1\nprocess_join_state({move, check_for_timeouts} = _Msg, State) ->\n    ?TRACE_JOIN1(_Msg, element(2, State)),\n    dht_node_move:send_trigger(),\n    State;\n\n% Catch all other messages until the join procedure is complete\nprocess_join_state(Msg, {join, JoinState, QueuedMessages}) ->\n    ?TRACE_JOIN1(Msg, JoinState),\n    %log:log(info(\"[dhtnode] [~p] postponed delivery of ~p\", [self(), Msg]),\n    {join, JoinState, msg_queue:add(QueuedMessages, Msg)}.\n\n%% @doc Process requests from a joining node at a existing node:\n-spec process_join_msg(join_message(), dht_node_state:state()) -> dht_node_state:state().\nprocess_join_msg({join, start}, State) ->\n    State;\nprocess_join_msg({join, number_of_samples_request, SourcePid, LbPsv, Conn} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    call_lb_psv(LbPsv, get_number_of_samples_remote, [SourcePid, Conn]),\n    State;\n\n%% userdevguide-begin dht_node_join:get_candidate\nprocess_join_msg({join, get_candidate, Source_PID, Key, LbPsv, Conn} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    call_lb_psv(LbPsv, create_join, [State, Key, Source_PID, Conn]);\n%% userdevguide-end dht_node_join:get_candidate\n\n%% userdevguide-begin dht_node_join:join_request1\nprocess_join_msg({join, join_request, NewPred, CandId, MaxTransportEntries} = _Msg, State)\n  when (not is_atom(NewPred)) -> % avoid confusion with not_responsible message\n    ?TRACE1(_Msg, State),\n    TargetId = node:id(NewPred),\n    JoinType = {join, 'send'},\n    MyNode = dht_node_state:get(State, node),\n    Command = dht_node_move:check_setup_slide_not_found(\n                State, JoinType, MyNode, NewPred, TargetId),\n    case Command of\n        {ok, JoinType} ->\n            MoveFullId = uid:get_global_uid(),\n            State1 = dht_node_move:exec_setup_slide_not_found(\n                       Command, State, MoveFullId, NewPred, TargetId, join,\n                       MaxTransportEntries, null, nomsg, {none}, false),\n            % set up slide, now send join_response:\n            MyOldPred = dht_node_state:get(State1, pred),\n            % no need to tell the ring maintenance -> the other node will trigger an update\n            % also this is better in case the other node dies during the join\n            %%     rm_loop:notify_new_pred(comm:this(), NewPred),\n            SlideOp = dht_node_state:get(State1, slide_pred),\n            Msg = {join, join_response, MyNode, MyOldPred, MoveFullId, CandId,\n                   slide_op:get_target_id(SlideOp), slide_op:get_next_op(SlideOp)},\n            dht_node_move:send(node:pidX(NewPred), Msg, MoveFullId),\n            State1;\n        {abort, ongoing_slide, JoinType} ->\n            ?TRACE(\"[ ~.0p ]~n  rejecting join_request from ~.0p due to a running slide~n\",\n                   [self(), NewPred]),\n            ?TRACE_SEND(node:pidX(NewPred), {join, join_response, busy, CandId}),\n            comm:send(node:pidX(NewPred), {join, join_response, busy, CandId}),\n            State;\n        {abort, _Reason, JoinType} -> % all other errors:\n            ?TRACE(\"~p\", [Command]),\n            ?TRACE_SEND(node:pidX(NewPred),\n                        {join, join_response, not_responsible, CandId}),\n            comm:send(node:pidX(NewPred),\n                      {join, join_response, not_responsible, CandId}),\n            State\n    end;\n%% userdevguide-end dht_node_join:join_request1\n\n% only messages with the first element being \"join\" are processed here\n% -> see dht_node.erl\nprocess_join_msg({get_dht_nodes_response, _Nodes} = _Msg, State) ->\n    State;\nprocess_join_msg({join, get_number_of_samples, _Samples, _Conn} = _Msg, State) ->\n    State;\nprocess_join_msg({join, get_candidate_response, _OrigJoinId, _Candidate, _Conn} = _Msg, State) ->\n    State;\nprocess_join_msg({join, join_response, Succ, Pred, MoveFullId, CandId, _TargetId, _NextOp} = _Msg, State) ->\n    reject_join_response(Succ, Pred, MoveFullId, CandId),\n    State;\nprocess_join_msg({join, join_response, Reason, _CandId} = _Msg, State)\n  when (Reason =:= not_responsible orelse Reason =:= busy) ->\n    State;\nprocess_join_msg({join, lookup_timeout, _Conn, _Id, _JoinId} = _Msg, State) ->\n    State;\nprocess_join_msg({join, known_hosts_timeout, _JoinId} = _Msg, State) ->\n    State;\nprocess_join_msg({join, get_number_of_samples_timeout, _Conn, _JoinId} = _Msg, State) ->\n    State;\nprocess_join_msg({join, join_request_timeout, _Timeouts, _CandId, _JoinId} = _Msg, State) ->\n    State;\nprocess_join_msg({join, {send_error, _Target, _Msg, _Reason}, _FailedPids}, State) ->\n    State;\n% messages send by the passive load balancing module\n% -> forward to the module\nprocess_join_msg({join, LbPsv, Msg, LbPsvState} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    call_lb_psv(LbPsv, process_join_msg, [Msg, LbPsvState, State]);\nprocess_join_msg({join, timeout, _JoinId} = _Msg, State) ->\n    State.\n\n-spec call_lb_psv(LbPsv::term(), Function::atom(), Parameters::[term()]) -> term().\ncall_lb_psv(LbPsv, Function, Parameters)\n  when is_atom(Function) andalso is_list(Parameters) ->\n    Module =\n    case lists:member(LbPsv, ?VALID_PASSIVE_ALGORITHMS) of\n        true -> LbPsv;\n        _    -> MyLbPsv = config:read(join_lb_psv),\n                log:log(error, \"[ Node ~.0p ] unknown passive load balancing \"\n                               \"algorithm requested: ~.0p - using ~.0p instead\",\n                        [LbPsv, MyLbPsv]),\n                MyLbPsv\n    end,\n    erlang:apply(Module, Function, Parameters).\n\n%% @doc Contacts all nodes set in the known_hosts config parameter and request\n%%      a list of dht_node instances in their VMs.\n-spec get_known_nodes(JoinUUId::pos_integer()) -> ok.\nget_known_nodes(JoinUUId) ->\n    KnownHosts = config:read(known_hosts),\n    % contact a subset of at most 3 random known VMs\n    Self = comm:reply_as(self(), 2, {join, '_', []}),\n    _ = [begin\n            ?TRACE_SEND(Host, {get_dht_nodes, comm:this()}),\n            comm:send(Host, {get_dht_nodes, comm:this()}, [{shepherd, Self}])\n         end || Host <- util:random_subset(3, KnownHosts)],\n    % also try to get some nodes from the current erlang VM:\n    OwnServicePerVm = whereis(service_per_vm),\n    ?TRACE_SEND(OwnServicePerVm, {get_dht_nodes, comm:this()}),\n    comm:send_local(OwnServicePerVm, {get_dht_nodes, comm:this()}),\n    % timeout just in case\n    msg_delay:send_local(get_known_hosts_timeout() div 1000, self(),\n                         {join, known_hosts_timeout, JoinUUId}).\n\n-spec phase2_next_step(JoinState::phase2(), Connections::[connection()])\n        -> phase_2_4().\nphase2_next_step(JoinState, Connections) ->\n    case skip_psv_lb(JoinState) of\n        true -> % skip phase2b (use only the given ID)\n            % (re-)issue lookups for all existing IDs and make sure there is at\n            % least one id (the chosen Id may have been removed due to a\n            % collision, for example)\n            lookup_new_ids2(1, JoinState);\n        _    -> get_number_of_samples(JoinState, Connections)\n    end.\n\n%% @doc Calls get_number_of_samples/1 on the configured passive load balancing\n%%      algorithm if there is a contact node in Connections and then adds them\n%%      to the list of connections.\n-spec get_number_of_samples\n        (Phase, Connections::[]) -> Phase when is_subtype(Phase, phase_2_4());\n        (phase_2_4(), Connections::[connection(),...]) -> phase2b().\nget_number_of_samples(JoinState, []) ->\n    % do not act here - wait other responses - a known_hosts_timeout will\n    % occur if something is wrong\n    JoinState;\nget_number_of_samples(JoinState, [Conn | _]) ->\n    NewConn = new_connection(Conn),\n    LbPsv = get_lb_psv(JoinState),\n    LbPsv:get_number_of_samples(NewConn),\n    msg_delay:send_local(\n      get_number_of_samples_timeout() div 1000, self(),\n      {join, get_number_of_samples_timeout, NewConn, get_join_uuid(JoinState)}),\n    JoinState1 = update_connection(Conn, NewConn, JoinState),\n    set_phase(phase2b, JoinState1).\n\n%% @doc Select Count new (random) IDs and issue lookups.\n-spec lookup_new_ids1\n        (Count::pos_integer(), JoinState::phase2() | phase2b() | phase3())\n            -> phase2() | phase3();\n        (Count::pos_integer(), JoinState::phase4()) -> phase4().\nlookup_new_ids1(Count, JoinState) ->\n    OldIds = get_join_ids(JoinState),\n    {NewJoinIds, OnlyNewJoinIds} = create_join_ids(Count, OldIds),\n    JoinState1 = set_join_ids(NewJoinIds, JoinState),\n    % try to contact Ids not in Candidates even if they have already been contacted\n    lookup(JoinState1, OnlyNewJoinIds).\n\n%% @doc Select as many new (random) IDs as needed to create (at least)\n%%      TotalCount candidates and issue lookups for all existing as well as the\n%%      new IDs.\n%%      Note: Existing IDs are not removed.\n%%      Note: If no join IDs remain, the next best candidate will be contacted!\n-spec lookup_new_ids2(TotalCount::pos_integer(), JoinState::phase2() | phase2b())\n        -> phase_2_4().\nlookup_new_ids2(TotalCount, JoinState) ->\n    % (re-)issue lookups for all existing IDs\n    JoinState2 = lookup(JoinState),\n    OldIds = get_join_ids(JoinState2),\n    CurCount = length(OldIds) + length(get_candidates(JoinState2)),\n    NewIdsCount = erlang:max(TotalCount - CurCount, 0),\n    case NewIdsCount of\n        0 ->\n            % if there are no join IDs left, then there will be no candidate response\n            % -> contact best candidate\n            case get_join_ids(JoinState2) of\n                []    -> contact_best_candidate(JoinState2);\n                [_|_] -> JoinState2\n            end;\n        _ -> lookup_new_ids1(NewIdsCount, JoinState2)\n    end.\n\n%% @doc Creates Count new (unique) additional IDs.\n-spec create_join_ids(Count::pos_integer(), OldIds::[?RT:key()])\n        -> {AllKeys::[?RT:key(),...], OnlyNewKeys::[?RT:key(),...]}.\ncreate_join_ids(Count, OldIds) ->\n    OldIdsSet = gb_sets:from_list(OldIds),\n    NewIdsSet = create_join_ids_helper(Count + gb_sets:size(OldIdsSet), OldIdsSet),\n    OnlyNewIdsSet = gb_sets:subtract(NewIdsSet, OldIdsSet),\n    {gb_sets:to_list(NewIdsSet), gb_sets:to_list(OnlyNewIdsSet)}.\n\n%% @doc Helper for create_join_ids/2 that creates the new unique IDs.\n-spec create_join_ids_helper(TotalCount::pos_integer(), gb_sets:set(?RT:key())) -> gb_sets:set(?RT:key()).\ncreate_join_ids_helper(TotalCount, Ids) ->\n    case gb_sets:size(Ids) of\n        TotalCount -> Ids;\n        _          -> create_join_ids_helper(TotalCount, gb_sets:add(?RT:get_random_node_id(), Ids))\n    end.\n\n%% @doc Tries to do a lookup for all join IDs in JoinState by contacting the\n%%      first node among the Connections, then go to phase 3. If there is no\n%%      node to contact, try to get new contact nodes and continue in phase 2.\n%%      A node that has been contacted will be put at the end of the\n%%      Connections list.\n%%      Note: the returned join state will stay in phase 4 if already in phase 4\n-spec lookup(phase2() | phase2b() | phase3()) -> NewState::phase_2_4().\nlookup(JoinState) ->\n    case get_join_ids(JoinState) of\n        [] ->\n            ?TRACE(\"[ Node ~w ] tried to look up nodes with an empty list \"\n                   \"of IDs, trying to contact best candidate...\", [self()]),\n            contact_best_candidate(JoinState);\n        [_|_] = JoinIds -> lookup(JoinState, JoinIds)\n    end.\n\n%% @doc Like lookup/1 but only looks up the given JoinIds (if there are any).\n%%      Note: the returned join state will stay in phase 4 if already in phase 4.\n-spec lookup(phase2() | phase2b() | phase3(), JoinIds::[?RT:key(),...]) -> NewState::phase2() | phase3();\n            (phase4(), JoinIds::[?RT:key(),...]) -> NewState::phase4().\nlookup(JoinState, JoinIds = [_|_]) ->\n    Phase = get_phase(JoinState),\n    case get_connections(JoinState) of\n        [] ->\n            case Phase of\n                phase4 ->\n                    % can't do lookup here but we are already waiting for a\n                    % join_response -> for now, try to get contact nodes\n                    % (may be useful for a lookup in a later step)\n                    get_known_nodes(get_join_uuid(JoinState)),\n                    JoinState;\n                phase2 ->\n                    % do not immediately re-issue a request for known nodes\n                    % -> wait for further replies until the known_hosts_timeout hits\n                    log:log(warn, \"[ Node ~w ] empty list of contact nodes, \"\n                                \"waiting...\", [self()]),\n                    JoinState;\n                _      ->\n                    log:log(warn, \"[ Node ~w ] no further nodes to contact, \"\n                                \"trying to get new nodes...\", [self()]),\n                    get_known_nodes(get_join_uuid(JoinState)),\n                    set_phase(phase2, JoinState)\n            end;\n        [{_, Node} = Conn | _] ->\n            MyLbPsv = get_lb_psv(JoinState),\n            % we do at least contact one node -> move this connection to the\n            % end if it has been used before\n            % otherwise new connections will be added later on:\n            JoinState1 = remove_connection(Conn, JoinState),\n            JoinState2 = case element(1, Conn) of\n                             null -> JoinState1;\n                             _    -> add_connections([Conn], JoinState1, back)\n                         end,\n            NewConns =\n                [begin\n                     NewConn = new_connection(Conn),\n                     Msg = {?lookup_aux, Id, 0,\n                            {join, get_candidate, comm:this(), Id, MyLbPsv, NewConn}},\n                     ?TRACE_SEND(Node, Msg),\n                     comm:send(Node, Msg),\n                     msg_delay:send_local(\n                       get_lookup_timeout() div 1000, self(),\n                       {join, lookup_timeout, NewConn, Id, get_join_uuid(JoinState2)}),\n                     NewConn\n                 end\n                 || Id <- JoinIds],\n            JoinState3 = add_connections(NewConns, JoinState2, back),\n            case Phase of\n                phase4 -> JoinState3;\n                _      -> set_phase(phase3, JoinState3)\n            end\n    end.\n\n%% @doc Integrates the Candidate into the join state if is is valid, i.e. a\n%%      slide with an ID not equal to the node to join at.\n%%      Note: a new ID will be sampled if the candidate is invalid.\n-spec integrate_candidate\n        (Candidate::lb_op:lb_op(), JoinState::phase2() | phase2b() | phase3(), Position::front | back)\n            -> phase2() | phase2b() | phase3();\n        (Candidate::lb_op:lb_op(), JoinState::phase4(), Position::front | back)\n            -> phase4().\nintegrate_candidate(Candidate, JoinState, Position) ->\n    % the other node could suggest a no_op -> select a new ID\n    case lb_op:is_slide(Candidate) of\n        false ->\n            log:log(warn, \"[ Node ~w ] got no_op candidate, \"\n                        \"looking up a new ID...\", [self()]),\n            lookup_new_ids1(1, JoinState);\n        _ ->\n            % note: can not use an existing ID! the other node should not have\n            % replied with such an ID though...\n            NewIdCand = node_details:get(lb_op:get(Candidate, n1_new), new_key),\n            SuccIdCand = node:id(node_details:get(lb_op:get(Candidate, n1succ_new), node)),\n            case NewIdCand =:= SuccIdCand of\n                true -> % the other node should not have send such an operation!\n                    log:log(warn, \"[ Node ~w ] chosen ID already exists, trying a \"\n                                \"new ID\", [self()]),\n                    lookup_new_ids1(1, JoinState);\n                _ when Position =:= front ->\n                    add_candidate_front(Candidate, JoinState);\n                _ when Position =:= back ->\n                    add_candidate_back(Candidate, JoinState)\n            end\n    end.\n\n%% userdevguide-begin dht_node_join:contact_best_candidate\n%% @doc Contacts the best candidate among all stored candidates and sends a\n%%      join_request (Timeouts = 0).\n-spec contact_best_candidate(JoinState::phase_2_4())\n        -> phase2() | phase2b() | phase4().\ncontact_best_candidate(JoinState) ->\n    JoinState1 = sort_candidates(JoinState),\n    send_join_request(JoinState1, 0).\n%% userdevguide-end dht_node_join:contact_best_candidate\n\n%% userdevguide-begin dht_node_join:send_join_request\n%% @doc Sends a join request to the first candidate. Timeouts is the number of\n%%      join_request_timeout messages previously received.\n%%      PreCond: the id has been set to the ID to join at and has been updated\n%%               in JoinState.\n-spec send_join_request(JoinState::phase_2_4(), Timeouts::non_neg_integer())\n        -> phase2() | phase2b() | phase4().\nsend_join_request(JoinState, Timeouts) ->\n    case get_candidates(JoinState) of\n        [] -> % no candidates -> start over (can happen, e.g. when join candidates are busy):\n            start_over(JoinState);\n        [BestCand | _] ->\n            Id = node_details:get(lb_op:get(BestCand, n1_new), new_key),\n            IdVersion = get_id_version(JoinState),\n            NewSucc = node_details:get(lb_op:get(BestCand, n1succ_new), node),\n            Me = node:new(comm:this(), Id, IdVersion),\n            CandId = lb_op:get(BestCand, id),\n            MyMTE = case dht_node_move:use_incremental_slides() of\n                        true -> dht_node_move:get_max_transport_entries();\n                        false -> unknown\n                    end,\n            Msg = {join, join_request, Me, CandId, MyMTE},\n            ?TRACE_SEND(node:pidX(NewSucc), Msg),\n            comm:send(node:pidX(NewSucc), Msg),\n            msg_delay:send_local(\n              get_join_request_timeout() div 1000, self(),\n              {join, join_request_timeout, Timeouts, CandId, get_join_uuid(JoinState)}),\n            set_phase(phase4, JoinState)\n    end.\n%% userdevguide-end dht_node_join:send_join_request\n\n%% userdevguide-begin dht_node_join:start_over\n%% @doc Goes back to phase 2 or 2b depending on whether contact nodes are\n%%      available or not.\n-spec start_over(JoinState::phase_2_4()) -> phase2() | phase2b().\nstart_over(JoinState) ->\n    JoinState1 = set_new_join_uuid(JoinState),\n    msg_delay:send_local(get_join_timeout() div 1000, self(),\n                         {join, timeout, get_join_uuid(JoinState1)}),\n    case get_connections(JoinState1) of\n        [] ->\n            get_known_nodes(get_join_uuid(JoinState1)),\n            set_phase(phase2, JoinState1);\n        [_|_] = Connections ->\n            JoinState2 = set_phase(phase2, JoinState1),\n            phase2_next_step(JoinState2, Connections)\n    end.\n%% userdevguide-end dht_node_join:start_over\n\n%% @doc Removes the candidate currently at the front and tries to contact\n%%      the next candidate. Keeps the join ID from the front candidate if\n%%      skip_psv_lb is set.\n-spec try_next_candidate(JoinState::phase_2_4())\n        -> phase2() | phase2b() | phase4().\ntry_next_candidate(JoinState) ->\n    JoinState1 =\n        case skip_psv_lb(JoinState) of\n            true -> remove_candidate_front_keep_id(JoinState);\n            _    -> remove_candidate_front(JoinState)\n        end,\n    contact_best_candidate(JoinState1).\n\n%% userdevguide-begin dht_node_join:finish_join\n%% @doc Finishes the join and sends all queued messages.\n-spec finish_join(Me::node:node_type(), Pred::node:node_type(),\n                  Succ::node:node_type(), DB::db_dht:db(),\n                  QueuedMessages::msg_queue:msg_queue(),\n                  JoinOptions::[tuple()])\n        -> dht_node_state:state().\nfinish_join(Me, Pred, Succ, DB, QueuedMessages, JoinOptions) ->\n    %% get old rt loop subscribtion table (if available)\n    MoveState = proplists:get_value(move_state, JoinOptions, []),\n    OldSubscrTable = proplists:get_value(subscr_table, MoveState, null),\n    RMState = rm_loop:init(Me, Pred, Succ, OldSubscrTable),\n    Neighbors = rm_loop:get_neighbors(RMState),\n    % wait for the ring maintenance to initialize and tell us its table ID\n    rt_loop:activate(Neighbors),\n    if MoveState =:= [] ->\n           dc_clustering:activate(),\n           gossip:activate(Neighbors);\n       true -> ok\n    end,\n    dht_node_reregister:activate(),\n    msg_queue:send(QueuedMessages),\n    NewRT_ext = ?RT:empty_ext(Neighbors),\n    service_per_vm:register_dht_node(node:pidX(Me)),\n    dht_node_state:new(NewRT_ext, RMState, DB).\n\n-spec reject_join_response(Succ::node:node_type(), Pred::node:node_type(),\n                           MoveFullId::slide_op:id(), CandId::lb_op:id()) -> ok.\nreject_join_response(Succ, _Pred, MoveId, _CandId) ->\n    % similar to dht_node_move:abort_slide/9 - keep message in sync!\n    Msg = {move, slide_abort, pred, MoveId, ongoing_slide},\n    ?TRACE_SEND(node:pidX(Succ), Msg),\n    dht_node_move:send_no_slide(node:pidX(Succ), Msg, 0).\n\n%% @doc Finishes the join by setting up a slide operation to get the data from\n%%      the other node and sends all queued messages.\n-spec finish_join_and_slide(Me::node:node_type(), Pred::node:node_type(),\n                            Succ::node:node_type(), DB::db_dht:db(),\n                            QueuedMessages::msg_queue:msg_queue(),\n                            MoveId::slide_op:id(), NextOp::slide_op:next_op(),\n                            JoinOptions::[tuple()])\n        -> {'$gen_component', [{on_handler, Handler::gen_component:handler()}],\n            State::dht_node_state:state()}.\nfinish_join_and_slide(Me, Pred, Succ, DB, QueuedMessages, MoveId, NextOp, JoinOptions) ->\n    State = finish_join(Me, Pred, Succ, DB, QueuedMessages, JoinOptions),\n    {SourcePid, Tag} =\n        case lists:keyfind(jump, 1, JoinOptions) of\n            {jump, JumpTag, Pid} -> {Pid, JumpTag};\n            _ -> {null, join}\n        end,\n    State1 = dht_node_move:exec_setup_slide_not_found(\n               {ok, {join, 'rcv'}}, State, MoveId, Succ, node:id(Me), Tag,\n               unknown, SourcePid, nomsg, NextOp, false),\n    gen_component:change_handler(State1, fun dht_node:on/2).\n%% userdevguide-end dht_node_join:finish_join\n\n% getter:\n-spec get_phase(phase_2_4()) -> phase2 | phase2b | phase3 | phase4.\nget_phase(JoinState)         -> element(1, JoinState).\n-spec get_join_uuid(phase_2_4()) -> pos_integer().\nget_join_uuid(JoinState)     -> element(2, JoinState).\n-spec get_join_options(phase_2_4()) -> [tuple()].\nget_join_options(JoinState) -> element(3, JoinState).\n-spec get_id_version(phase_2_4()) -> non_neg_integer().\nget_id_version(JoinState)    -> element(4, JoinState).\n-spec get_connections(phase_2_4()) -> [connection()].\nget_connections(JoinState)   -> element(5, JoinState).\n-spec get_join_ids(phase_2_4()) -> [?RT:key()].\nget_join_ids(JoinState)      -> element(6, JoinState).\n-spec get_candidates(phase_2_4()) -> [lb_op:lb_op()].\nget_candidates(JoinState)    -> element(7, JoinState).\n\n% setter:\n-spec set_phase(phase2, phase_2_4()) -> phase2();\n               (phase2b, phase_2_4()) -> phase2b();\n               (phase3, phase_2_4()) -> phase3();\n               (phase4, phase_2_4()) -> phase4().\nset_phase(Phase, JoinState) -> setelement(1, JoinState, Phase).\n-spec set_new_join_uuid(Phase) -> Phase when is_subtype(Phase, phase_2_4()).\nset_new_join_uuid(JoinState) -> setelement(2, JoinState, uid:get_pids_uid()).\n-spec set_join_ids(JoinIds::[?RT:key()], phase_2_4()) -> phase_2_4().\nset_join_ids(JoinIds, JoinState) -> setelement(6, JoinState, JoinIds).\n-spec remove_join_id(JoinId::?RT:key(), phase_2_4()) -> phase_2_4().\nremove_join_id(JoinIdToRemove, {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, Candidates}) ->\n    {Phase, JoinUUId, Options, CurIdVersion, Connections,\n     [Id || Id <- JoinIds, Id =/= JoinIdToRemove], Candidates}.\n-spec add_candidate_front(Candidate::lb_op:lb_op(), phase_2_4()) -> phase_2_4().\nadd_candidate_front(Candidate, {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, Candidates}) ->\n    % filter previous candidates with the same ID (only use the newest one)\n    CandId = lb_op:get(Candidate, id),\n    Candidates1 = [C || C <- Candidates, lb_op:get(C, id) =/= CandId],\n    {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, [Candidate | Candidates1]}.\n-spec add_candidate_back(Candidate::lb_op:lb_op(), phase_2_4()) -> phase_2_4().\nadd_candidate_back(Candidate, {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, Candidates}) ->\n    {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, lists:append(Candidates, [Candidate])}.\n-spec sort_candidates(phase_2_4()) -> phase_2_4().\nsort_candidates({Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, Candidates} = JoinState) ->\n    LbPsv = get_lb_psv(JoinState),\n    {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, LbPsv:sort_candidates(Candidates)}.\n-spec remove_candidate(CandId::lb_op:id(), phase_2_4()) -> phase_2_4().\nremove_candidate(CandId, {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, Candidates}) ->\n    NewCandidates = [C || C <- Candidates, lb_op:get(C, id) =/= CandId],\n    {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, NewCandidates}.\n-spec remove_candidate_front(phase_2_4()) -> phase_2_4().\nremove_candidate_front({Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, []}) ->\n    {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, []};\nremove_candidate_front({Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, [_ | Candidates]}) ->\n    {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, Candidates}.\n-spec remove_candidate_front_keep_id(phase_2_4()) -> phase_2_4().\nremove_candidate_front_keep_id({Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, []}) ->\n    {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, []};\nremove_candidate_front_keep_id({Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, [Front | Candidates]}) ->\n    IdFront = node_details:get(lb_op:get(Front, n1_new), new_key),\n    {Phase, JoinUUId, Options, CurIdVersion, Connections, [IdFront | JoinIds], Candidates}.\n-spec skip_psv_lb(phase_2_4()) -> boolean().\nskip_psv_lb({_Phase, _JoinUUId, Options, _CurIdVersion, _Connections, _JoinIds, _Candidates}) ->\n    lists:member({skip_psv_lb}, Options).\n\n-spec new_connection(OldConn::{null | pos_integer(), comm:mypid()})\n        -> connection().\nnew_connection({_, Node}) ->\n    {uid:get_pids_uid(), Node}.\n\n-spec update_connection(\n        OldConn::{null | pos_integer(), comm:mypid()},\n        NewConn::{null | pos_integer(), comm:mypid()}, Phase)\n            -> Phase when is_subtype(Phase, phase_2_4()).\nupdate_connection({null, _} = OldConn, NewConn, JoinState) ->\n    JoinState1 = remove_connection(OldConn, JoinState),\n    add_connections([NewConn], JoinState1, back);\nupdate_connection(OldConn, NewConn, JoinState) ->\n    JoinState1 = remove_connection(OldConn, JoinState),\n    add_connections([OldConn, NewConn], JoinState1, back).\n\n-spec reset_connection(OldConn::{null | pos_integer(), comm:mypid()}, JoinState)\n        -> JoinState when is_subtype(JoinState, phase_2_4()).\nreset_connection({_, Node} = OldConn, JoinState) ->\n    JoinState1 = remove_connection(OldConn, JoinState),\n    add_connections([{null, Node}], JoinState1, front).\n\n-spec add_connections([connection()], JoinState, front | back) -> JoinState when is_subtype(JoinState, phase_2_4()).\nadd_connections([], JoinState, _Pos) -> JoinState;\nadd_connections([CNHead], {Phase, JoinUUId, Options, CurIdVersion, [CNHead | _] = Connections, JoinIds, Candidates}, _Pos) ->\n    {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, Candidates};\nadd_connections(Nodes = [_|_], {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, Candidates}, back) ->\n    {Phase, JoinUUId, Options, CurIdVersion, lists:append(Connections, Nodes), JoinIds, Candidates};\nadd_connections(Nodes = [_|_], {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, Candidates}, front) ->\n    {Phase, JoinUUId, Options, CurIdVersion, lists:append(Nodes, Connections), JoinIds, Candidates}.\n\n-spec remove_connection(connection(), JoinState) -> JoinState when is_subtype(JoinState, phase_2_4()).\nremove_connection(Conn, {Phase, JoinUUId, Options, CurIdVersion, Connections, JoinIds, Candidates}) ->\n    {Phase, JoinUUId, Options, CurIdVersion, [C || C <- Connections, C =/= Conn],\n     JoinIds, Candidates}.\n\n%% @doc Checks whether config parameters of the dht_node process during join\n%%      exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(join_request_timeout) and\n    config:cfg_is_greater_than_equal(join_request_timeout, 1000) and\n\n    config:cfg_is_integer(join_request_timeouts) and\n    config:cfg_is_greater_than_equal(join_request_timeouts, 1) and\n\n    config:cfg_is_integer(join_lookup_timeout) and\n    config:cfg_is_greater_than_equal(join_lookup_timeout, 1000) and\n\n    config:cfg_is_integer(join_known_hosts_timeout) and\n    config:cfg_is_greater_than_equal(join_known_hosts_timeout, 1000) and\n\n    config:cfg_is_integer(join_timeout) and\n    config:cfg_is_greater_than_equal(join_timeout, 1000) and\n\n    config:cfg_is_integer(join_get_number_of_samples_timeout) and\n    config:cfg_is_greater_than_equal(join_get_number_of_samples_timeout, 1000) and\n\n    config:cfg_is_module(join_lb_psv).\n\n%% @doc Gets the max number of ms to wait for a scalaris node to reply on a\n%%      join request (set in the config files).\n-spec get_join_request_timeout() -> pos_integer().\nget_join_request_timeout() ->\n    config:read(join_request_timeout).\n\n%% @doc Gets the max number of join_request_timeouts before a candidate is\n%%      removed and the next one is taken (set in the config files).\n-spec get_join_request_timeouts() -> pos_integer().\nget_join_request_timeouts() ->\n    config:read(join_request_timeouts).\n\n%% @doc Gets the max number of ms to wait for a key lookup response\n%%      (set in the config files).\n-spec get_lookup_timeout() -> pos_integer().\nget_lookup_timeout() ->\n    config:read(join_lookup_timeout).\n\n%% @doc Gets the max number of ms to wait for a key lookup response\n%%      (set in the config files).\n-spec get_known_hosts_timeout() -> pos_integer().\nget_known_hosts_timeout() ->\n    config:read(join_known_hosts_timeout).\n\n%% @doc Gets the max number of ms to wait for a join to be completed\n%%      (set in the config files).\n-spec get_join_timeout() -> pos_integer().\nget_join_timeout() ->\n    config:read(join_timeout).\n\n%% @doc Gets the max number of ms to wait for a join to be completed\n%%      (set in the config files).\n-spec get_number_of_samples_timeout() -> pos_integer().\nget_number_of_samples_timeout() ->\n    config:read(join_get_number_of_samples_timeout).\n\n%% @doc Gets the passive load balancing algorithm to use for the given\n%%      JoinState (set in the config files). If skip_psv_lb is set,\n%%      lb_psv_simple will be used independent from the module set in the\n%%      config.\n-spec get_lb_psv(JoinState::phase_2_4()) -> module().\nget_lb_psv(JoinState) ->\n    case skip_psv_lb(JoinState) of\n        true -> lb_psv_simple;\n        _    -> config:read(join_lb_psv)\n    end.\n"
  },
  {
    "path": "src/dht_node_join_recover.erl",
    "content": "% @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Join ring on recover.\n%% @version $Id$\n-module(dht_node_join_recover).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-export([join/1, on/2]).\n\n-spec join(Options::[tuple()]) ->\n                  {'$gen_component', [{on_handler, Handler::gen_component:handler()},...],\n                   [tuple()]}.\njoin(Options) ->\n    comm:send_local(self(), {recover_now}),\n    gen_component:change_handler(\n      Options,\n      fun ?MODULE:on/2).\n\n-spec on(tuple(), [tuple()]) -> dht_node_state:state() |\n              {'$gen_component', [{on_handler, Handler::gen_component:handler()},...], [tuple()]}.\non({recover_now}, Options) ->\n    State = recover_state(Options),\n    gen_component:change_handler(\n      State,\n      fun dht_node:on/2);\non(Msg, Options) ->\n    comm:send_local(self(), Msg),\n    Options.\n\n-spec recover_state(Options::[tuple()]) -> dht_node_state:state().\nrecover_state(Options) ->\n    %% 1. get old lease databases\n    LeaseDBs = [get_db(Options, erlang:list_to_atom(\"lease_db-\" ++ erlang:integer_to_list(I))) || I <- lists:seq(1, config:read(replication_factor))],\n    %% 2. find old leases\n    LeaseList = lease_recover:recover(LeaseDBs),\n    %% 3. create state with old mnesias\n    MyId = l_on_cseq:get_id(lease_list:get_active_lease(LeaseList)),\n    Me = node:new(comm:this(), MyId, 0),\n    Neighbors = nodelist:new_neighborhood(Me), % needed for ?RT:empty_ext/1\n    EmptyRT = ?RT:empty_ext(Neighbors), % only for rt_chord\n    RMState = rm_loop:init(Me, Me, Me, null),\n    rm_loop:send_trigger(), % speed up RM\n    KV_DB = get_db(Options, kv_db),\n    io:format(\"recovered a kv_db of size ~p~n\", [length(prbr:tab2list(KV_DB))]),\n    TXID_DBs = [get_db(Options, erlang:list_to_atom(\"tx_id-\" ++ erlang:integer_to_list(I))) || I <- lists:seq(1, config:read(replication_factor))],\n    State = dht_node_state:new_on_recover(EmptyRT, RMState,\n                                          KV_DB, TXID_DBs, LeaseDBs, LeaseList),\n    %% 3. after the leases are known, we can start ring-maintenance and routing\n    %% 4. now we can try to refresh the local leases\n    msg_delay:send_trigger(1, {l_on_cseq, renew_leases}),\n\n    %% copied from dht_node_join\n    rt_loop:activate(Neighbors),\n    dc_clustering:activate(),\n    gossip:activate(Neighbors),\n    dht_node_reregister:activate(),\n    service_per_vm:register_dht_node(node:pidX(Me)),\n    State.\n\nget_db(Options, DBName) ->\n    case lists:keyfind(DBName, 1, Options) of\n        false ->\n            log:log(\"error: Options:~p ~nDBName:~w\", [Options, DBName]),\n            log:log(\"error in dht_node_join_recover:get_db/2. The mnesia database ~w was not found in the dht_node options (~w)\", [DBName, Options]),\n            ok; % will fail in tab2list\n        {DBName, DB} ->\n            db_prbr:open(DB)\n    end.\n"
  },
  {
    "path": "src/dht_node_lookup.erl",
    "content": "%  @copyright 2007-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    dht_node lookup algorithm (interacts with the dht_node process)\n%% @end\n%% @version $Id$\n-module(dht_node_lookup).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"lookup.hrl\").\n\n-export([lookup_aux_leases/4, lookup_decision_chord/4, lookup_fin/4, lookup_aux_failed/3, lookup_fin_failed/3]).\n\n-export([envelope/2]).\n\n-export_type([data/0]).\n\n-type enveloped_message() :: {pos_integer(), f, comm:message()}.\n\n-spec envelope(pos_integer(), comm:message()) -> enveloped_message().\nenvelope(Nth, Msg) ->\n    ?DBG_ASSERT('_' =:= element(Nth, Msg)),\n    {Nth, f, Msg}.\n\n%% userdevguide-begin dht_node_lookup:routing\n%% @doc Check the lease and translate to lookup_fin or forward to rt_loop\n%%      accordingly.\n-spec lookup_aux_leases(State::dht_node_state:state(), Key::intervals:key(),\n                       Hops::non_neg_integer(), Msg::comm:message()) -> ok.\nlookup_aux_leases(State, Key, Hops, Msg) ->\n    %% When the neighborhood information from rm in rt_loop and leases disagree\n    %% over responsibility, the lookup is forwared to the successor instead of\n    %% asking rt. We can not simply forward this message to rt_loop though:\n    %% rt_loop thinks (based on the neighborhood from rm) that the node is\n    %% responsible and forwards the msg as a lookup_fin msg to its dht_node. The\n    %% dht_node decides (based on the lease, see lookup_fin_leases) that the\n    %% node is not responsible and forwards the message to the next node. When\n    %% the message arrives again at the node, the same steps repeat. As\n    %% joining/sliding needs lookup messages, the new leases are never\n    %% established and the message circle endlessly.\n    case leases:is_responsible(State, Key) of\n        false ->\n            FullMsg = {?lookup_aux, Key, Hops, Msg},\n            comm:send_local(pid_groups:get_my(routing_table), FullMsg);\n        _ ->\n            ERT = dht_node_state:get(State, rt),\n            Neighbors = dht_node_state:get(State, neighbors),\n            WrappedMsg = ?RT:wrap_message(Key, Msg, ERT, Neighbors, Hops),\n            comm:send_local(self(),\n                            {?lookup_fin, Key, ?HOPS_TO_DATA(Hops), WrappedMsg})\n    end.\n\n%% @doc Decide, whether a lookup_aux message should be translated into a lookup_fin\n%%      message\n-spec lookup_decision_chord(State::dht_node_state:state(), Key::intervals:key(),\n                       Hops::non_neg_integer(), Msg::comm:message()) -> ok.\nlookup_decision_chord(State, Key, Hops, Msg) ->\n    Neighbors = dht_node_state:get(State, neighbors),\n    Succ = node:pidX(nodelist:succ(Neighbors)),\n    % NOTE: re-evaluate that the successor is really (still) responsible:\n    case intervals:in(Key, nodelist:succ_range(Neighbors)) of\n        true ->\n            %% log:log(warn, \"[dht_node] lookup_fin on lookup_decision\"),\n            NewMsg = {?lookup_fin, Key, ?HOPS_TO_DATA(Hops + 1), Msg},\n            comm:send(Succ, NewMsg, [{shepherd, pid_groups:get_my(routing_table)}]);\n        _ ->\n            %% log:log(warn, \"[dht_node] lookup_aux on lookup_decision\"),\n            NewMsg = {?lookup_aux, Key, Hops + 1, Msg},\n            comm:send(Succ, NewMsg, [{shepherd, pid_groups:get_my(routing_table)},\n                                     {group_member, routing_table}])\n    end.\n\n%% check_local_leases(DHTNodeState, MyRange) ->\n%%     LeaseList = dht_node_state:get(DHTNodeState, lease_list),\n%%     ActiveLease = lease_list:get_active_lease(LeaseList),\n%%     PassiveLeases = lease_list:get_passive_leases(LeaseList),\n%%     ActiveInterval = case ActiveLease of\n%%                          empty ->\n%%                              intervals:empty();\n%%                          _ ->\n%%                              l_on_cseq:get_range(ActiveLease)\n%%                      end,\n%%     LocalCorrect = MyRange =:= ActiveInterval,\n%%     log:pal(\"~p rm =:= leases: ~w. lease=~w. my_range=~w\",\n%%               [self(), LocalCorrect, ActiveInterval, MyRange]).\n\n\n%% @doc Find the node responsible for Key and send him the message Msg.\n-spec lookup_fin(State::dht_node_state:state(), Key::intervals:key(),\n                 Data::data(), Msg::comm:message()) -> dht_node_state:state().\nlookup_fin(State, Key, Hops, Msg) ->\n    case config:read(leases) of\n        true ->\n            lookup_fin_leases(State, Key, Hops, Msg);\n        _ ->\n            lookup_fin_chord(State, Key, Hops, Msg)\n    end.\n\n-spec lookup_fin_chord(State::dht_node_state:state(), Key::intervals:key(),\n                       Data::data(), Msg::comm:message()) -> dht_node_state:state().\nlookup_fin_chord(State, Key, Data, Msg) ->\n    MsgFwd = dht_node_state:get(State, msg_fwd),\n    FwdList = [P || {I, P} <- MsgFwd, intervals:in(Key, I)],\n    Hops = ?HOPS_FROM_DATA(Data),\n    case FwdList of\n        []    ->\n            case dht_node_state:is_db_responsible__no_msg_fwd_check(Key, State) of\n                true ->\n                    %comm:send_local(pid_groups:get_my(dht_node_monitor),\n                    %                {lookup_hops, Hops}),\n                    %Unwrap = ?RT:unwrap_message(Msg, State),\n                    %gen_component:post_op(Unwrap, State);\n                    deliver(State, Msg, false, Hops);\n                false ->\n                    % do not warn if\n                    % a) received lookup_fin due to a msg_fwd while sliding and\n                    %    before the other node removed the message forward or\n                    % b) our pred is not be aware of our ID change yet (after\n                    %    moving data to our successor) yet\n                    SlidePred = dht_node_state:get(State, slide_pred),\n                    SlideSucc = dht_node_state:get(State, slide_succ),\n                    Neighbors = dht_node_state:get(State, neighbors),\n                    InSlideIntervalFun =\n                        fun(SlideOp) ->\n                                slide_op:is_slide(SlideOp) andalso\n                                    slide_op:get_sendORreceive(SlideOp) =:= 'send' andalso\n                                    intervals:in(Key, slide_op:get_interval(SlideOp))\n                        end,\n                    case lists:any(InSlideIntervalFun, [SlidePred, SlideSucc]) orelse\n                             intervals:in(Key, nodelist:succ_range(Neighbors)) of\n                        true -> ok;\n                        false ->\n                            DBRange = dht_node_state:get(State, db_range),\n                            DBRange2 = [begin\n                                            case intervals:is_continuous(Interval) of\n                                                true -> {intervals:get_bounds(Interval), Id};\n                                                _    -> {Interval, Id}\n                                            end\n                                        end || {Interval, Id} <- DBRange],\n                            log:log(warn,\n                                    \"[ ~.0p ] Routing is damaged (~p)!! Trying again...~n\"\n                                    \"  myrange:~p~n  db_range:~p~n  msgfwd:~p~n  Key:~p~n\"\n                                    \"  pred: ~.4p~n  node: ~.4p~n  succ: ~.4p\",\n                                    [self(), Data, intervals:get_bounds(nodelist:node_range(Neighbors)),\n                                     DBRange2, MsgFwd, Key, nodelist:pred(Neighbors),\n                                     nodelist:node(Neighbors), nodelist:succ(Neighbors)])\n                    end,\n                    lookup_decision_chord(State, Key, Hops, Msg),\n                    State\n            end;\n        [Pid] -> comm:send(Pid, {?lookup_fin, Key, ?HOPS_TO_DATA(Hops + 1), Msg}),\n                 State\n    end.\n\n-spec lookup_fin_leases(State::dht_node_state:state(), Key::intervals:key(),\n                        Data::data(), Msg::comm:message()) -> dht_node_state:state().\nlookup_fin_leases(State, Key, Data, Msg) ->\n    Hops = ?HOPS_FROM_DATA(Data),\n    case leases:is_responsible(State, Key) of\n        true ->\n            deliver(State, Msg, true, Hops);\n        maybe ->\n            deliver(State, Msg, false, Hops);\n        false ->\n            %% We are here because the neighborhood information from rm in rt_loop\n            %% and leases disagree over responsibility. One cause for this case\n            %% can be join/sliding. Our successor still has the lease for our range.\n            %% But rm already believes that we are responsible for our range. The\n            %% solution is to forward the lookup to our successor instead of asking rt.\n            %% Simply forwarding a lookup_fin msg would lead to linear routing,\n            %% therefore we use lookup_aux. See also lookup_aux().\n            %% log:log(\"lookup_fin fail: ~p\", [self()]),\n            NewMsg = {?lookup_aux, Key, Hops + 1, Msg},\n            comm:send(dht_node_state:get(State, succ_pid), NewMsg),\n            State\n    end.\n%% userdevguide-end dht_node_lookup:routing\n\n-spec lookup_aux_failed(dht_node_state:state(), Target::comm:mypid(),\n                        Msg::{?lookup_aux, Key::?RT:key(), Hops::pos_integer(), Msg::comm:message()}) -> ok.\nlookup_aux_failed(State, _Target, {?lookup_aux, Key, Hops, Msg} = _Message) ->\n    log:log(warn, \"[dht_node] lookup_aux_failed. Target: ~p. Msg: ~p.\", [_Target, _Message]),\n    _ = comm:send_local_after(100, self(), {?lookup_aux, Key, Hops + 1, Msg}),\n    State.\n\n-spec lookup_fin_failed(dht_node_state:state(), Target::comm:mypid(),\n                        Msg::{?lookup_fin, Key::?RT:key(), Data::data(), Msg::comm:message()}) -> ok.\nlookup_fin_failed(State, _Target, {?lookup_fin, Key, Data, Msg} = _Message) ->\n    log:log(warn, \"[dht_node] lookup_aux_failed. Target: ~p. Msg: ~p.\", [_Target, _Message]),\n    _ = comm:send_local_after(100, self(), {?lookup_aux, Key, ?HOPS_FROM_DATA(Data) + 1, Msg}),\n    State.\n\ndeliver(State, Msg, Consistency, Hops) ->\n    %log:log(\"lookup_fin success: ~p ~p\", [self(), Msg]),\n    case config:read(dht_node_monitor) of\n        true ->\n            comm:send_local(pid_groups:get_my(dht_node_monitor),\n                            {lookup_hops, Hops});\n        _ -> ok\n    end,\n    Unwrap = ?RT:unwrap_message(Msg, State),\n    case Unwrap of\n        {Nth, f, InnerMsg} ->\n            gen_component:post_op(erlang:setelement(Nth, InnerMsg, Consistency), State);\n        _ ->\n            gen_component:post_op(Unwrap, State)\n    end.\n"
  },
  {
    "path": "src/dht_node_monitor.erl",
    "content": "%  @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    dht_node helper process for monitoring\n%% @end\n%% @version $Id$\n-module(dht_node_monitor).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-behaviour(gen_component).\n\n-export([start_link/2, on/2, init/1]).\n\n-include(\"gen_component.hrl\").\n\n% accepted messages of dht_node_monitor processes\n-type message() :: {Key::lookup_hops, Value::pos_integer()} |\n                    {Msg::db_histogram_init, Value::?RT:key(), Range::intervals:interval()} |\n                    {Key::db_op, Value::?RT:key()}.\n-type state() :: {LookupHops::rrd:rrd(),\n                  DBOps::rrd:rrd(), %% counts the number of db operations in a fixed interval\n                  DBHistogram::rrd:rrd() | uninit %% histogram of db operations which gets resetted on pred or node id change\n                 }.\n\n%% @doc message handler\n-spec on(message(), state()) -> state().\non({lookup_hops, Hops}, {OldLookupHops, OldDBOps, OldDBHistogram}) ->\n    NewLookupHops = rrd:add_now(Hops, OldLookupHops),\n    monitor:check_report(dht_node, 'lookup_hops', OldLookupHops, NewLookupHops),\n    {NewLookupHops, OldDBOps, OldDBHistogram};\n\non({db_histogram_init, Id}, {OldLookupHops, OldDBOps, _OldDBHistogram}) ->\n    NewDBHistogram = lb_stats:init_db_histogram(Id),\n    {OldLookupHops, OldDBOps, NewDBHistogram};\n\non({db_report}, {OldLookupHops, OldDBOps, OldDBHistogram}) ->\n    case lb_active:requests_balance() of\n        true ->\n            NewDBHistogram = rrd:add_now(no_op, OldDBHistogram),\n            monitor:check_report(lb_active, db_histogram, OldDBHistogram, NewDBHistogram);\n        _ ->\n            NewDBHistogram = OldDBHistogram\n    end,\n    NewDBOps = rrd:add_now(0, OldDBOps),\n    monitor:check_report(lb_active, db_ops, OldDBOps, NewDBOps),\n    {OldLookupHops, NewDBOps, NewDBHistogram};\n\non({db_op, Key}, {OldLookupHops, OldDBOps, OldDBHistogram}) ->\n    NewDBOps = rrd:add_now(1, OldDBOps),\n    NewDBHistogram = lb_stats:update_db_histogram(Key, OldDBHistogram),\n    {OldLookupHops, NewDBOps, NewDBHistogram}.\n\n%% @doc initialisation\n-spec init(Options::[tuple()]) -> state().\ninit(_Options) ->\n    % 1m monitoring interval, only keep newest\n    LookupHops = rrd:create(60 * 1000000, 1, {timing, count}),\n    DBOps = rrd:create(config:read(lb_active_monitor_resolution) * 1000, 1, counter),\n    %% initialized by dht_node with handler db_op_init\n    DBHistogram = uninit,\n    {LookupHops, DBOps, DBHistogram}.\n\n%% @doc spawns a dht_node_monitor, called by the scalaris supervisor process\n-spec start_link(pid_groups:groupname(), [tuple()]) -> {ok, pid()}.\nstart_link(DHTNodeGroup, Options) ->\n    gen_component:start_link(\n      ?MODULE, fun ?MODULE:on/2, Options, [{pid_groups_join_as, DHTNodeGroup, dht_node_monitor}]).\n"
  },
  {
    "path": "src/dht_node_move.erl",
    "content": "%  @copyright 2010-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    dht_node move procedure\n%%         Note: assumes that the neighborhood does not change during the\n%%         handling of a message.\n%% @end\n%% @version $Id$\n-module(dht_node_move).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-define(TRACE_SEND(Pid, Msg), ?TRACE(\"[ ~.0p ] to ~.0p: ~.0p~n\", [self(), Pid, Msg])).\n-define(TRACE1(Msg, State),\n        ?TRACE(\"[ ~.0p ]~n  Msg: ~.0p~n\"\n               \"  State: pred: ~.0p~n\"\n               \"         node: ~.0p~n\"\n               \"         succ: ~.0p~n\"\n               \"   slide_pred: ~.0p~n\"\n               \"   slide_succ: ~.0p~n\"\n               \"      msg_fwd: ~.0p~n\"\n               \"     db_range: ~.0p~n\",\n               [self(), Msg, dht_node_state:get(State, pred),\n                dht_node_state:get(State, node), dht_node_state:get(State, succ),\n                dht_node_state:get(State, slide_pred), dht_node_state:get(State, slide_succ),\n                dht_node_state:get(State, msg_fwd), dht_node_state:get(State, db_range)])).\n\n-define(FD_SUBSCR_PID(MoveFullId),\n        comm:reply_as(self(), 3, {move, MoveFullId, '_'})).\n\n-export([process_move_msg/2, send_trigger/0,\n         make_slide/5,\n         make_slide_leave/2, make_jump/4,\n         check_config/0]).\n% for dht_node_join:\n-export([send/3, send_no_slide/3, notify_source_pid/2,\n         check_setup_slide_not_found/5, exec_setup_slide_not_found/11,\n         use_incremental_slides/0, get_max_transport_entries/0]).\n\n-export_type([move_message/0, next_op_msg/0, abort_reason/0,\n              result_message/0]).\n\n-type abort_reason() ::\n    ongoing_slide |\n    target_id_not_in_range |\n    % moving node and its succ or pred did not share the same knowledge about\n    % each other or the pred/succ changed during a move:\n    wrong_pred_succ_node |\n    % received slide_succ with different parameters than previously set up before sending slide_pred to succ:\n    changed_parameters |\n    % node received data but its interval is not adjacent to the range of the node: (should never occur - error in the protocoll!) or\n    % node received data_ack but the send interval is not correct anymore, i.e. at either end of our range\n    {wrong_interval, MyRange::intervals:interval(), MovingDataInterval::intervals:interval()} |\n    % node received data but previous data still exists (should never occur - error in the protocoll!)\n    existing_data |\n    target_down | % target node reported down by fd\n    scheduled_leave | % tried to continue a slide but a leave was already scheduled\n    leave_no_partner_found | % tried to do a graceful leave but no successor to move the data to\n    next_op_mismatch |\n    {protocol_error, term()}.\n\n-type result_message() :: {move, result, Tag::any(), Reason::abort_reason() | ok}.\n\n-type next_op_msg() ::\n    {none} |\n    {continue, NewSlideId::slide_op:id()} |\n    {abort, NewSlideId::slide_op:id(), abort_reason()}.\n\n-type move_message1() ::\n    {move, start_slide, pred | succ, TargetId::?RT:key(), Tag::any(), SourcePid::comm:mypid() | null} |\n    {move, start_jump, TargetId::?RT:key(), Tag::any(), SourcePid::comm:mypid() | null} |\n    {move, slide, OtherType::slide_op:type(), MoveFullId::slide_op:id(),\n     InitNode::node:node_type(), TargetNode::node:node_type(), TargetId::?RT:key(),\n     Tag::any(), NextOp::slide_op:next_op(), MaxTransportEntries::unknown | pos_integer()} |\n    {move, slide_abort, pred | succ, MoveFullId::slide_op:id(), Reason::abort_reason()} |\n    {move, data, MovingData::dht_node_state:slide_data(), MoveFullId::slide_op:id(), TargetId::?RT:key(), NextOp::slide_op:next_op()} |\n    {move, data_ack, MoveFullId::slide_op:id()} |\n    {move, delta, ChangedData::dht_node_state:slide_delta(), MoveFullId::slide_op:id()} |\n    {move, delta_ack, MoveFullId::slide_op:id(), next_op_msg()} |\n    {move, rm_db_range, MoveFullId::slide_op:id()} |\n    {move, done, MoveFullId::slide_op:id()} |\n    {move, continue, MoveFullId::slide_op:id(),\n     Operation::prepare_send_data2 | update_rcv_data2 | prepare_send_delta2 |\n         finish_delta2 | {finish_delta_ack2, NextOpMsg::next_op_msg()},\n     EmbeddedMsg::comm:message()}\n.\n\n-type move_message() ::\n    move_message1() |\n    {move, {fd_notify, fd:event(), DeadPid::comm:mypid(), Reason::fd:reason()}} |\n    {move, check_for_timeouts} |\n    {move, {send_error, Target::comm:mypid(), Message::move_message1(), Reason::atom()}, {timeouts, Timeouts::non_neg_integer()}} |\n    {move, {send_error, Target::comm:mypid(), Message::move_message1(), Reason::atom()}, MoveFullId::slide_op:id()}.\n\n-spec send_trigger() -> ok.\nsend_trigger() ->\n    msg_delay:send_trigger(get_wait_for_reply_timeout() div 4, {move, check_for_timeouts}).\n\n%% @doc Processes move messages for the dht_node and implements the node move\n%%      protocol.\n-spec process_move_msg(move_message(), dht_node_state:state()) -> dht_node_state:state().\nprocess_move_msg({move, start_slide, PredOrSucc, TargetId, Tag, SourcePid} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    make_slide(State, PredOrSucc, TargetId, Tag, SourcePid);\n\nprocess_move_msg({move, start_jump, TargetId, Tag, SourcePid} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    make_jump(State, TargetId, Tag, SourcePid);\n\n% notification from predecessor/successor that it wants to slide with our node\n% (maybe incremental)\nprocess_move_msg({move, slide, MyType, MoveFullId, OtherNode,\n                  OtherTargetNode, TargetId, Tag, NextOp, MaxTransportEntries} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    setup_slide(State, MyType, MoveFullId,\n                OtherTargetNode, OtherNode, TargetId, Tag, MaxTransportEntries,\n                null, slide, NextOp);\n\n% notification from predecessor/successor that the move is a noop, i.e. already\n% finished\nprocess_move_msg({move, done, MoveFullId} = _Msg, MyState) ->\n    ?TRACE1(_Msg, MyState),\n    WorkerFun =\n        fun(SlideOp, State) ->\n                notify_source_pid(slide_op:get_source_pid(SlideOp),\n                                  {move, result, slide_op:get_tag(SlideOp), ok}),\n                dht_node_state:set_slide(State, slide_op:get_predORsucc(SlideOp), null)\n        end,\n    safe_operation(WorkerFun, MyState, MoveFullId, [wait_for_other], done, false);\n\n% notification from pred/succ that he could not aggree on a slide with us\nprocess_move_msg({move, slide_abort, PredOrSucc, MoveFullId, Reason} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    SlideOp = case PredOrSucc of\n                  pred -> dht_node_state:get(State, slide_pred);\n                  succ -> dht_node_state:get(State, slide_succ)\n              end,\n    case slide_op:is_slide(SlideOp) andalso slide_op:get_id(SlideOp) =:= MoveFullId of\n        true ->\n            abort_slide(State, SlideOp, Reason, false, slide_abort);\n        _ ->\n            log:log(warn, \"[ dht_node_move ~.0p ] slide_abort (~.0p) received with no \"\n                          \"matching slide operation (ID: ~.0p, slide_~.0p: ~.0p)~n\",\n                    [comm:this(), PredOrSucc, MoveFullId, PredOrSucc, SlideOp]),\n            State\n    end;\n\nprocess_move_msg({move, node_leave} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    % note: can't use safe_operation/6 - there is no slide op id\n    SlideOp = dht_node_state:get(State, slide_succ),\n    % only handle node_update in wait_for_continue phase\n    case slide_op:is_slide(SlideOp) andalso slide_op:is_leave(SlideOp) andalso\n             (is_tuple(Phase = slide_op:get_phase(SlideOp)) andalso\n                  element(1, Phase) =:= wait_for_continue) andalso\n             slide_op:get_sendORreceive(SlideOp) =:= 'send' of\n        true ->\n            prepare_send_data2(State, SlideOp, {continue});\n        _ ->\n            % we should not receive node update messages unless we are waiting for them\n            % (node id updates should only be triggered by this module anyway)\n            log:log(warn, \"[ dht_node_move ~w ] received rm node leave message with no \"\n                          \"matching slide operation (slide_succ: ~w)~n\",\n                    [comm:this(), SlideOp]),\n            State % ignore unrelated node leave messages\n    end;\n\n% data from a neighbor\nprocess_move_msg({move, data = MoveMsgTag, MovingData, MoveFullId, TargetId, NextOp} = _Msg, MyState) ->\n    ?TRACE1(_Msg, MyState),\n    WorkerFun =\n        fun(SlideOp, State) ->\n                SlideOp1 = slide_op:reset_send_errors(SlideOp),\n                update_rcv_data1(State, SlideOp1, MovingData, TargetId, NextOp, MoveMsgTag)\n        end,\n    safe_operation(WorkerFun, MyState, MoveFullId, [wait_for_data, wait_for_other], MoveMsgTag, false);\n\n% acknowledgement from neighbor that its node received data for the slide op with the given id\nprocess_move_msg({move, data_ack = MoveMsgTag, MoveFullId} = _Msg, MyState) ->\n    ?TRACE1(_Msg, MyState),\n    WorkerFun =\n        fun(SlideOp, State) ->\n                SlideOp1 = slide_op:reset_send_errors(SlideOp),\n                prepare_send_delta1(State, SlideOp1, MoveMsgTag)\n        end,\n    safe_operation(WorkerFun, MyState, MoveFullId, [wait_for_data_ack], MoveMsgTag, false);\n\n% delta from neighbor\nprocess_move_msg({move, delta = MoveMsgTag, ChangedData, MoveFullId} = _Msg, MyState) ->\n    ?TRACE1(_Msg, MyState),\n    WorkerFun =\n        fun(SlideOp, State) ->\n                SlideOp1 = slide_op:reset_send_errors(SlideOp),\n                finish_delta1(State, SlideOp1, ChangedData, MoveMsgTag)\n        end,\n    safe_operation(WorkerFun, MyState, MoveFullId, [wait_for_delta], MoveMsgTag, true);\n\n% acknowledgement from neighbor that its node received delta for the slide op\n% with the given id and information about how to continue\nprocess_move_msg({move, delta_ack = MoveMsgTag, MoveFullId, NextOpMsg} = _Msg, MyState) ->\n    ?TRACE1(_Msg, MyState),\n    WorkerFun =\n        fun(SlideOp, State) ->\n                SlideOp1 = slide_op:reset_send_errors(SlideOp),\n                finish_delta_ack1(State, SlideOp1, NextOpMsg, MoveMsgTag)\n        end,\n    safe_operation(WorkerFun, MyState, MoveFullId, [wait_for_delta_ack], MoveMsgTag, true);\n\nprocess_move_msg({move, {send_error, Target, Message, _Reason}, {timeouts, Timeouts}} = _Msg, MyState) ->\n    ?TRACE1(_Msg, MyState),\n    NewTimeouts = Timeouts + 1,\n    MaxRetries = get_send_msg_retries(),\n    % TODO: keep references to target nodes? (could stop sending messages if fd reports down, otherwise send indefinitely)\n    case NewTimeouts =< MaxRetries of\n        true -> send_no_slide(Target, Message, NewTimeouts);\n        _    -> log:log(warn,\n                        \"[ dht_node_move ~.0p ] giving up to send ~.0p to ~.0p (~p unsuccessful retries)\",\n                        [comm:this(), Message, Target, NewTimeouts])\n    end,\n    MyState;\n\nprocess_move_msg({move, {send_error, Target, Message, _Reason}, MoveFullId} = _Msg, MyState) ->\n    ?TRACE1(_Msg, MyState),\n    % delay the actual re-try (it may be a crash or a temporary failure)\n    _ = comm:send_local_after(\n          config:read(move_send_msg_retry_delay), self(),\n          {move, {send_error_retry, Target, Message, _Reason}, MoveFullId}),\n    MyState;\n\nprocess_move_msg({move, {send_error_retry, Target, Message, _Reason}, MoveFullId} = _Msg, MyState) ->\n    ?TRACE1(_Msg, MyState),\n    WorkerFun =\n        fun(SlideOp, State) ->\n                NewSlideOp = slide_op:inc_send_errors(SlideOp),\n                MaxRetries = get_send_msg_retries(),\n                PredOrSucc = slide_op:get_predORsucc(SlideOp),\n                case slide_op:get_send_errors(SlideOp) of\n                    T when T =< MaxRetries -> ok;\n                    T    ->\n                        log:log(warn,\n                                \"[ dht_node_move ~.0p ] slide with ~p: ~p unsuccessful retries to send message ~.0p\",\n                                [comm:this(), PredOrSucc, T, Message])\n                end,\n                send(Target, Message, MoveFullId),\n                dht_node_state:set_slide(State, PredOrSucc, NewSlideOp)\n        end,\n    % TODO: ignore wrong neighbors in case of delta or delta_ack?!\n    safe_operation(WorkerFun, MyState, MoveFullId, all, send_error_retry, false);\n\n% no reply from the target node within get_wait_for_reply_timeout() ms\nprocess_move_msg({move, check_for_timeouts} = _Msg, MyState) ->\n    Now = os:timestamp(),\n    NewState =\n        lists:foldl(\n          fun({PredOrSucc, SlidePredOrSucc}, StateX) ->\n                  case dht_node_state:get(StateX, SlidePredOrSucc) of\n                      null ->\n                          StateX;\n                      SlideOp ->\n                          case slide_op:get_time_next_warn(SlideOp) of\n                              never ->\n                                  StateX;\n                              WTime ->\n                                  case timer:now_diff(WTime, Now) =< 0 of\n                                      false ->\n                                          StateX;\n                                      true ->\n                                          log:log(warn,\n                                                  \"[ dht_node_move ~.0p ] slide with ~p: no reply received within ~pms\",\n                                                  [comm:this(), PredOrSucc,\n                                                   timer:now_diff(Now, slide_op:get_time_last_send(SlideOp))]),\n                                          WTime1 = util:time_plus_ms(Now, get_wait_for_reply_timeout()),\n                                          SlideOp1 = slide_op:set_time_next_warn(SlideOp, WTime1),\n                                          dht_node_state:set_slide(StateX, PredOrSucc, SlideOp1)\n                                  end\n                          end\n                  end\n          end, MyState, [{pred, slide_pred}, {succ, slide_succ}]),\n    send_trigger(),\n    NewState;\n\nprocess_move_msg({move, continue, MoveFullId, Operation, EmbeddedMsg} = _Msg, MyState) ->\n    ?TRACE1(_Msg, MyState),\n    case Operation of\n        % although Operation matches the function name, verify\n        % validity here and call functions manually to avoid\n        % having to export them\n        prepare_send_data2 ->\n            WorkerFun = fun(SlideOp, State) ->\n                                prepare_send_data2(State, SlideOp, EmbeddedMsg)\n                        end,\n            Phase = wait_for_other;\n        update_rcv_data2 ->\n            WorkerFun = fun(SlideOp, State) ->\n                                update_rcv_data2(State, SlideOp, EmbeddedMsg)\n                        end,\n            Phase = wait_for_data;\n        prepare_send_delta2 ->\n            WorkerFun = fun(SlideOp, State) ->\n                                prepare_send_delta2(State, SlideOp, EmbeddedMsg)\n                        end,\n            Phase = wait_for_data_ack;\n        finish_delta2 ->\n            WorkerFun = fun(SlideOp, State) ->\n                                finish_delta2(State, SlideOp, EmbeddedMsg)\n                        end,\n            Phase = wait_for_delta;\n        {finish_delta_ack2, NextOpMsg} ->\n            WorkerFun = fun(SlideOp, State) ->\n                                finish_delta_ack2(State, SlideOp, NextOpMsg, EmbeddedMsg)\n                        end,\n            Phase = wait_for_delta_ack\n    end,\n    safe_operation(WorkerFun, MyState, MoveFullId, [{wait_for_continue, Phase}], continue,\n                   Operation =:= finish_delta2 orelse\n                       (is_tuple(Operation) andalso element(1, Operation) =:= finish_delta_ack2));\n\n% failure detector reported dead node\nprocess_move_msg({move, MoveFullId, {fd_notify, crash, _DeadPid, _Reason}} = _Msg, MyState) ->\n    ?TRACE1(_Msg, MyState),\n    WorkerFun =\n        fun(SlideOp, State) ->\n                abort_slide(State, SlideOp, target_down, false, crash)\n        end,\n    safe_operation(WorkerFun, MyState, MoveFullId, all, crashed_node, false);\nprocess_move_msg({move, _MoveFullId, {fd_notify, _Event, _DeadPid, _Reason}} = _Msg, MyState) ->\n    MyState.\n\n% misc.\n\n%% @doc Sends a move message using the dht_node as the shepherd to handle\n%%      broken connections.\n-spec send(Pid::comm:mypid(), Message::comm:message(), MoveFullId::slide_op:id()) -> ok.\nsend(Pid, Message, MoveFullId) ->\n    Shepherd = comm:reply_as(self(), 2, {move, '_', MoveFullId}),\n    ?TRACE_SEND(Pid, Message),\n    comm:send(Pid, Message, [{shepherd, Shepherd}]).\n\n%% @doc Sends a move message using the dht_node as the shepherd to handle\n%%      broken connections. This does not require a slide_op being set up.\n%%      The error message handler can count the number of timeouts using the\n%%      provided cookie.\n-spec send_no_slide(Pid::comm:mypid(), Message::comm:message(), Timeouts::non_neg_integer()) -> ok.\nsend_no_slide(Pid, Message, Timeouts) ->\n    Shepherd = comm:reply_as(self(), 2, {move, '_', {timeouts, Timeouts}}),\n    ?TRACE_SEND(Pid, Message),\n    comm:send(Pid, Message, [{shepherd, Shepherd}]).\n\n%% @doc Sends a move message using the dht_node as the shepherd to handle\n%%      broken connections. The target node is determined from the SlideOp.\n%%      A timeout counter in the SlideOp is reset and the dht_node_state is\n%%      updated with the (new) slide operation.\n-spec send2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(), Message::comm:message()) -> dht_node_state:state().\nsend2(State, SlideOp, Message) ->\n    MoveFullId = slide_op:get_id(SlideOp),\n    Target = node:pidX(slide_op:get_node(SlideOp)),\n    PredOrSucc = slide_op:get_predORsucc(SlideOp),\n    SlideOp1 = slide_op:set_time_last_send(\n                 SlideOp,\n                 util:time_plus_ms(os:timestamp(), get_wait_for_reply_timeout())),\n    send(Target, Message, MoveFullId),\n    dht_node_state:set_slide(State, PredOrSucc, SlideOp1).\n\n%% @doc Notifies the successor or predecessor about a (new) slide operation.\n%%      Assumes that the information in the slide operation is still correct\n%%      (check with safe_operation/6!).\n%%      Will also set the appropriate slide_phase and update the slide op in\n%%      the dht_node.\n-spec notify_other(SlideOp::slide_op:slide_op(), State::dht_node_state:state())\n        -> dht_node_state:state().\nnotify_other(SlideOp, State) ->\n    null = slide_op:get_phase(SlideOp), % just to check\n    Type = slide_op:get_type(SlideOp),\n    SetupAtOther = slide_op:is_setup_at_other(SlideOp),\n    SendOrReceive = slide_op:get_sendORreceive(Type),\n    UseIncrSlides = use_incremental_slides(),\n    if SendOrReceive =:= 'rcv' ->\n           IncrSlide = slide_op:is_incremental(SlideOp),\n           {MTE, NextOp} =\n               if IncrSlide     -> {unknown, slide_op:get_next_op(SlideOp)};\n                  UseIncrSlides -> {get_max_transport_entries(), {none}};\n                  true          -> {unknown, {none}}\n               end,\n           SlideOp1 = slide_op:set_phase(SlideOp, wait_for_data),\n           send2(State, SlideOp1,\n                 {move, slide, slide_op:other_type_to_my_type(Type),\n                  slide_op:get_id(SlideOp1), dht_node_state:get(State, node),\n                  slide_op:get_node(SlideOp), slide_op:get_target_id(SlideOp1),\n                  slide_op:get_tag(SlideOp1), NextOp, MTE});\n       not SetupAtOther -> % beware: overlap with 1st\n           SlideOp1 = slide_op:set_phase(SlideOp, wait_for_other),\n           case slide_op:is_join(Type, 'send') of\n               true ->\n                   % first message here is a join_request which will be answered\n                   % by code in dht_node_join! (we simply set the last_send time here)\n                   SlideOp2 = slide_op:set_time_last_send(\n                                SlideOp1,\n                                util:time_plus_ms(os:timestamp(), get_wait_for_reply_timeout())),\n                   PredOrSucc = slide_op:get_predORsucc(Type),\n                   dht_node_state:set_slide(State, PredOrSucc, SlideOp2);\n               false ->\n                   IncrSlide = slide_op:is_incremental(SlideOp1),\n                   {MTE, NextOp} =\n                       if IncrSlide     -> {unknown, slide_op:get_next_op(SlideOp1)};\n                          UseIncrSlides -> {get_max_transport_entries(), {none}};\n                          true          -> {unknown, {none}}\n                       end,\n                   send2(State, SlideOp1,\n                         {move, slide, slide_op:other_type_to_my_type(Type),\n                          slide_op:get_id(SlideOp1), dht_node_state:get(State, node),\n                          slide_op:get_node(SlideOp), slide_op:get_target_id(SlideOp1),\n                          slide_op:get_tag(SlideOp1), NextOp, MTE})\n           end\n    end.\n\n%% @doc Sets up a new slide operation with the node's successor or predecessor\n%%      after a request for a slide has been received.\n-spec setup_slide(State::dht_node_state:state(), Type::slide_op:type(),\n                  MoveFullId::slide_op:id(), MyNode::node:node_type(),\n                  TargetNode::node:node_type(), TargetId::?RT:key(),\n                  Tag::any(), MaxTransportEntries::unknown | pos_integer(),\n                  SourcePid::comm:mypid() | null,\n                  MsgTag::nomsg | slide,\n                  NextOp::slide_op:next_op())\n        -> dht_node_state:state().\nsetup_slide(State, Type, MoveFullId, MyNode, TargetNode, TargetId, Tag,\n            MaxTransportEntries, SourcePid, MsgTag, NextOp) ->\n    case get_slide(State, MoveFullId) of\n        {ok, _PredOrSucc, SlideOp} ->\n            % there could already be a slide operation because\n            % a) a second message was send after an unsuccessful send-message\n            %    on the other node\n            % -> ignore this message\n            % b) we are waiting for the second node's MaxTransportEntries\n            % -> re-create the slide (TargetId or NextOp may have changed)\n            case slide_op:get_phase(SlideOp) of\n                wait_for_other ->\n                    WorkerFun =\n                        fun(SlideOp0, State0) ->\n                                % during a join, everything was already set up\n                                % by join_request\n                                % -> don't re-create the slide!!\n                                % (the interval may be wrong since the joining\n                                % node may already be known to the rm)\n                                case slide_op:is_join(SlideOp0, 'send') of\n                                    true ->\n                                        prepare_send_data1(State0, SlideOp, slide);\n                                    false ->\n                                        TargetIdReal = \n                                            %% in case of a jump, the target id is always\n                                            %% the pred id which is set as target id for the\n                                            %% slide later on.\n                                            case slide_op:is_jump(SlideOp0, 'send') of\n                                                true ->\n                                                    slide_op:get_jump_target_id(SlideOp0);\n                                                _    ->\n                                                    TargetId\n                                            end,\n                                        recreate_existing_slide(\n                                          SlideOp0, State0, TargetIdReal,\n                                          MaxTransportEntries, MsgTag, NextOp)\n                                end\n                        end,\n                    safe_operation(WorkerFun, State, MoveFullId, [wait_for_other], slide, false);\n                _ ->\n                    State\n            end;\n        not_found ->\n            Command = check_setup_slide_not_found(\n                        State, Type, MyNode, TargetNode, TargetId),\n            exec_setup_slide_not_found(\n              Command, State, MoveFullId, TargetNode, TargetId, Tag,\n              MaxTransportEntries, SourcePid, MsgTag, NextOp, false);\n        {wrong_neighbor, _PredOrSucc, SlideOp} -> % wrong pred or succ\n            abort_slide(State, SlideOp, wrong_pred_succ_node, true, slide)\n    end.\n\n-type command() :: {abort, abort_reason(), OrigType::slide_op:type()} |\n                   {ok, NewType::slide_op:type() | move_done}.\n\n%% @doc Checks whether a new slide operation with the node's successor or\n%%      predecessor and the given parameters can be set up.\n-spec check_setup_slide_not_found(State::dht_node_state:state(),\n        Type::slide_op:type(), MyNode::node:node_type(),\n        TargetNode::node:node_type(), TargetId::?RT:key())\n        -> Command::command().\ncheck_setup_slide_not_found(State, Type, MyNode, TNode, TId) ->\n    PredOrSucc = slide_op:get_predORsucc(Type),\n    SlideSucc = dht_node_state:get(State, slide_succ),\n    DBRange = dht_node_state:get(State, db_range),\n    SlidePred = dht_node_state:get(State, slide_pred),\n    CanSlide = case PredOrSucc of\n                   pred -> can_slide_pred(SlidePred, SlideSucc, DBRange, TId, Type);\n                   succ -> can_slide_succ(SlidePred, SlideSucc, DBRange, TId, Type)\n               end,\n    % correct pred/succ info? did pred/succ know our current ID? -> compare node info\n    Neighbors = dht_node_state:get(State, neighbors),\n    NodesCorrect = MyNode =:= nodelist:node(Neighbors) andalso\n                       (TNode =:= nodelist:PredOrSucc(Neighbors) orelse Type =:= {join, 'send'}),\n    MoveDone = (PredOrSucc =:= pred andalso node:id(TNode) =:= TId\n                    andalso Type =/= {join, 'send'}\n                    andalso Type =/= {leave, 'rcv'}) orelse\n               (PredOrSucc =:= succ andalso node:id(MyNode) =:= TId),\n    HasLeft = dht_node_state:has_left(State),\n    Command =\n        if CanSlide andalso NodesCorrect andalso not MoveDone andalso not HasLeft ->\n                SendOrReceive = slide_op:get_sendORreceive(Type),\n                case slide_op:is_leave(Type) of\n                    true when SendOrReceive =:= 'send' ->\n                        % graceful leave (slide with succ, send all data)\n                        % note: may also be a jump operation!\n                        % TODO: check for running slide, abort it if possible, eventually extend it\n                        case slide_op:is_jump(Type) of\n                            true ->\n                                TIdInRange = intervals:in(TId, nodelist:node_range(Neighbors)),\n                                TIdInSuccRange =\n                                    intervals:in(TId, nodelist:succ_range(Neighbors)),\n                                if % convert jump to slide?\n                                    TIdInRange     -> {ok, {slide, succ, 'send'}};\n                                    TIdInSuccRange -> {ok, {slide, succ, 'rcv'}};\n                                    true ->\n                                        case SlideSucc =:= null andalso\n                                                 SlidePred =:= null of\n                                            true -> {ok, {jump, 'send'}};\n                                            _    -> {abort, ongoing_slide, Type}\n                                        end\n                                end;\n                            _ -> {ok, {leave, 'send'}}\n                        end;\n                    true when SendOrReceive =:= 'rcv' ->\n                        {ok, {leave, SendOrReceive}};\n                    false when SendOrReceive =:= 'rcv' -> % no leave/jump\n                        case TId =:= node:id(MyNode) of\n                            true -> {abort, target_id_not_in_range, Type};\n                            _    -> {ok, Type}\n                        end;\n                    false when SendOrReceive =:= 'send' -> % no leave/jump\n                        TIdInRange = intervals:in(TId, nodelist:node_range(Neighbors)),\n                        case not TIdInRange orelse TId =:= node:id(MyNode) of\n                            true -> {abort, target_id_not_in_range, Type};\n                            _    -> {ok, Type}\n                        end\n                end;\n            not CanSlide -> {abort, ongoing_slide, Type};\n            not NodesCorrect -> {abort, wrong_pred_succ_node, Type};\n            HasLeft -> {abort, target_down, Type};\n            true ->\n                % MoveDone, i.e. target id already reached (noop)\n                case slide_op:is_leave(Type) andalso not slide_op:is_jump(Type) of\n                    false -> {ok, move_done};\n                    true  -> {abort, leave_no_partner_found, Type}\n                end\n        end,\n    Command.\n\n%% @doc Creates a new slide operation with the node's successor or\n%%      predecessor and the given parameters according to the command created\n%%      by check_setup_slide_not_found/5.\n%%      Note: assumes that such a slide does not already exist if command is not abort.\n-spec exec_setup_slide_not_found(\n        Command::command(),\n        State::dht_node_state:state(), MoveFullId::slide_op:id(),\n        TargetNode::node:node_type(), TargetId::?RT:key(), Tag::any(),\n        OtherMaxTransportEntries::unknown | pos_integer(),\n        SourcePid::comm:mypid() | null,\n        MsgTag::nomsg | slide | delta_ack,\n        NextOp::slide_op:next_op(),\n        FdSubscribed::boolean()) -> dht_node_state:state().\nexec_setup_slide_not_found(Command, State, MoveFullId, TargetNode, TargetId, Tag,\n                           OtherMTE, SourcePid, MsgTag, NextOp, FdSubscribed) ->\n    Neighbors = dht_node_state:get(State, neighbors),\n    % note: NewType (inside the command) may be different than the initially planned type\n    case Command of\n        {abort, Reason, OrigType} ->\n            ?IIF(FdSubscribed,\n                 fd:unsubscribe(?FD_SUBSCR_PID(MoveFullId), [node:pidX(TargetNode)]),\n                 ok),\n            abort_slide(State, TargetNode, MoveFullId, null, SourcePid, Tag,\n                        OrigType, Reason, MsgTag =/= nomsg);\n        {ok, {join, 'send'} = NewType} -> % similar to {slide, pred, 'send'}\n            ?IIF(FdSubscribed, ok,\n                 fd:subscribe(?FD_SUBSCR_PID(MoveFullId), [node:pidX(TargetNode)])),\n            UseIncrSlides = use_incremental_slides(),\n            SlideOp =\n                if UseIncrSlides orelse OtherMTE =/= unknown->\n                       IncTargetKey = find_incremental_target_id(\n                                        Neighbors, State,\n                                        TargetId, NewType, OtherMTE),\n                       slide_op:new_sending_slide_join_i(\n                        MoveFullId, TargetNode, IncTargetKey, join, Neighbors);\n                   true ->\n                       slide_op:new_sending_slide_join(\n                        MoveFullId, TargetNode, join, Neighbors)\n                end,\n            % note: phase will be set by notify_other/2 and needs to remain null here\n            SlideMod = get_slide_mod(),\n            case SlideMod:prepare_join_send(State, SlideOp) of\n                {ok, State1, SlideOp1} when MsgTag =:= nomsg ->\n                    notify_other(SlideOp1, State1);\n                {ok, State1, SlideOp1} when MsgTag =:= slide ->\n                    prepare_send_data1(State1, SlideOp1, MsgTag);\n                {abort, Reason, State1, SlideOp1} ->\n                    abort_slide(State1, SlideOp1, Reason, MsgTag =/= nomsg, MsgTag)\n            end;\n        {ok, {slide, pred, 'send'} = NewType} ->\n            ?IIF(FdSubscribed, ok,\n                 fd:subscribe(?FD_SUBSCR_PID(MoveFullId), [node:pidX(TargetNode)])),\n            UseIncrSlides = use_incremental_slides(),\n            case MsgTag of\n                nomsg -> % first inform other node:\n                    SlideOp = slide_op:new_slide(\n                                MoveFullId, NewType, TargetId, Tag, SourcePid,\n                                OtherMTE, NextOp, Neighbors),\n                    notify_other(SlideOp, State);\n                Y when (Y =:= slide orelse Y =:= delta_ack) ->\n                    SlideOp =\n                        if UseIncrSlides orelse OtherMTE =/= unknown->\n                               IncTargetKey = find_incremental_target_id(\n                                                Neighbors, State,\n                                                TargetId, NewType, OtherMTE),\n                               slide_op:new_slide_i(\n                                 MoveFullId, NewType, IncTargetKey, TargetId,\n                                 Tag, SourcePid, OtherMTE, Neighbors);\n                           true ->\n                               slide_op:new_slide(\n                                 MoveFullId, NewType, TargetId, Tag, SourcePid,\n                                 OtherMTE, NextOp, Neighbors)\n                        end,\n                    % note: phase will be set by prepare_send_data1/2 and needs to remain null here\n                    prepare_send_data1(State, SlideOp, MsgTag)\n            end;\n        {ok, {join, 'rcv'}} -> % similar to {slide, succ, 'rcv'}\n            ?IIF(FdSubscribed, ok,\n                 fd:subscribe(?FD_SUBSCR_PID(MoveFullId), [node:pidX(TargetNode)])),\n            SlideOp = slide_op:new_receiving_slide_join(MoveFullId, TargetId, Tag, SourcePid, Neighbors),\n            % note: phase will be set by notify_other/2 and needs to remain null here\n            SlideOp1 = slide_op:set_setup_at_other(SlideOp), % we received a join_response before\n            SlideOp2 = slide_op:set_next_op(SlideOp1, NextOp),\n            SlideMod = get_slide_mod(),\n            case SlideMod:prepare_rcv_data(State, SlideOp2) of\n                {ok, State1, SlideOp3} ->\n                    notify_other(SlideOp3, State1);\n                {abort, Reason, State1, SlideOp3} ->\n                    abort_slide(State1, SlideOp3, Reason, true, MsgTag)\n            end;\n        {ok, {jump, 'send'} = NewType} -> % similar to {ok, {slide, succ, 'send'}}\n            ?IIF(FdSubscribed, ok,\n                 fd:subscribe(?FD_SUBSCR_PID(MoveFullId), [node:pidX(TargetNode)])),\n            UseIncrSlides = use_incremental_slides(),            \n            LeaveTargetId = node:id(nodelist:pred(Neighbors)),\n            CurTargetId =\n                if UseIncrSlides orelse OtherMTE =/= unknown->\n                       find_incremental_target_id(\n                         Neighbors, State, LeaveTargetId, NewType, OtherMTE);\n                   true -> LeaveTargetId\n                end, \n            SlideOp = slide_op:new_sending_slide_jump(\n                        MoveFullId, CurTargetId, TargetId, SourcePid, Tag, Neighbors),\n            case MsgTag of\n                nomsg ->\n                    notify_other(SlideOp, State);\n                X when (X =:= slide orelse X =:= delta_ack) ->\n                    prepare_send_data1(State, SlideOp, MsgTag)\n            end;\n        {ok, {leave, 'send'} = NewType} -> % similar to {ok, {slide, succ, 'send'}}\n            ?IIF(FdSubscribed, ok,\n                 fd:subscribe(?FD_SUBSCR_PID(MoveFullId), [node:pidX(TargetNode)])),\n            UseIncrSlides = use_incremental_slides(),\n            % the successor may have send a wrong (final) target ID - use the correct one here\n            RealTargetId = node:id(nodelist:pred(Neighbors)),\n            TargetId1 =\n                if UseIncrSlides orelse OtherMTE =/= unknown->\n                       find_incremental_target_id(\n                         Neighbors, State, RealTargetId, NewType, OtherMTE);\n                   true -> RealTargetId\n                end,\n            SlideOp = slide_op:new_sending_slide_leave(\n                         MoveFullId, TargetId1, leave, SourcePid, Neighbors),\n            case MsgTag of\n                nomsg ->\n                    notify_other(SlideOp, State);\n                X when (X =:= slide orelse X =:= delta_ack) ->\n                    prepare_send_data1(State, SlideOp, MsgTag)\n            end;\n        {ok, {slide, succ, 'send'} = NewType} ->\n            ?IIF(FdSubscribed, ok,\n                 fd:subscribe(?FD_SUBSCR_PID(MoveFullId), [node:pidX(TargetNode)])),\n            UseIncrSlides = use_incremental_slides(),\n            case MsgTag of\n                nomsg -> % first inform other node:\n                    SlideOp = slide_op:new_slide(\n                                MoveFullId, NewType, TargetId, Tag, SourcePid,\n                                OtherMTE, NextOp, Neighbors),\n                    notify_other(SlideOp, State);\n                X when (X =:= slide orelse X =:= delta_ack) ->\n                    SlideOp =\n                        if UseIncrSlides orelse OtherMTE =/= unknown->\n                               IncTargetKey = find_incremental_target_id(\n                                                Neighbors, State,\n                                                TargetId, NewType, OtherMTE),\n                               slide_op:new_slide_i(\n                                 MoveFullId, NewType, IncTargetKey, TargetId,\n                                 Tag, SourcePid, OtherMTE, Neighbors);\n                           true ->\n                               slide_op:new_slide(\n                                 MoveFullId, NewType, TargetId, Tag, SourcePid,\n                                 OtherMTE, NextOp, Neighbors)\n                        end,\n                    prepare_send_data1(State, SlideOp, MsgTag)\n            end;\n        {ok, NewType} when NewType =:= {slide, pred, 'rcv'} orelse\n                               NewType =:= {leave, 'rcv'} orelse\n                               NewType =:= {slide, succ, 'rcv'} ->\n            ?IIF(FdSubscribed, ok,\n                 fd:subscribe(?FD_SUBSCR_PID(MoveFullId), [node:pidX(TargetNode)])),\n            SlideOp = slide_op:new_slide(MoveFullId, NewType, TargetId, Tag,\n                                         SourcePid, OtherMTE, NextOp, Neighbors),\n            % note: phase will be set by notify_other/2 and needs to remain null here\n            SlideMod = get_slide_mod(),\n            case SlideMod:prepare_rcv_data(State, SlideOp) of\n                {ok, State1, SlideOp1} when MsgTag =:= nomsg ->\n                    notify_other(SlideOp1, State1);\n                {ok, State1, SlideOp1} when MsgTag =:= slide ->\n                    notify_other(slide_op:set_setup_at_other(SlideOp1), State1);\n                {abort, Reason, State1, SlideOp1} ->\n                    abort_slide(State1, SlideOp1, Reason, MsgTag =/= nomsg, MsgTag)\n            end;\n        {ok, move_done} ->\n            ?IIF(FdSubscribed,\n                 fd:unsubscribe(?FD_SUBSCR_PID(MoveFullId), [node:pidX(TargetNode)]),\n                 ok),\n            notify_source_pid(SourcePid, {move, result, Tag, ok}),\n            case MsgTag of\n                nomsg -> ok;\n                _     -> Msg = {move, done, MoveFullId},\n                         send_no_slide(node:pidX(TargetNode), Msg, 0)\n            end,\n            State\n    end.\n\n%% @doc On the sending node: looks into the DB and selects a key in the slide\n%%      op's range which involves at most OtherMaxTransportEntries DB entries\n%%      to be moved.\n-spec find_incremental_target_id(Neighbors::nodelist:neighborhood(),\n        State::dht_node_state:state(), FinalTargetId::?RT:key(), Type::slide_op:type(),\n        OtherMaxTransportEntries::unknown | pos_integer()) -> ?RT:key().\nfind_incremental_target_id(Neighbors, State, FinalTargetId, Type, OtherMTE) ->\n    ?DBG_ASSERT('send' =:= slide_op:get_sendORreceive(Type)), % just in case\n    MTE = case OtherMTE of\n              unknown -> get_max_transport_entries();\n              _       -> erlang:min(OtherMTE, get_max_transport_entries())\n          end,\n    case slide_op:get_predORsucc(Type) of\n        pred -> BeginId = node:id(nodelist:pred(Neighbors)), Dir = forward;\n        succ -> BeginId = nodelist:nodeid(Neighbors), Dir = backward\n    end,\n    % TODO: optimise here - if the remaining interval has no data, return FinalTargetId\n    case dht_node_state:get_split_key(State, BeginId, FinalTargetId, MTE, Dir) of\n        {SplitKey, MTE} -> SplitKey;\n        {_SplitKey, MTEX} when MTEX < MTE -> FinalTargetId\n    end.\n\n%% @doc Change the local node's ID to the given TargetId and progresses to the\n%%      next phase, e.g. wait_for_continue. \n%% @see slide_chord:prepare_send_data1/3\n-spec prepare_send_data1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                         MoveMsgTag::atom()) -> dht_node_state:state().\nprepare_send_data1(State, SlideOp, MoveMsgTag) ->\n    MoveFullId = slide_op:get_id(SlideOp),\n    SlideOp1 = slide_op:set_setup_at_other(SlideOp),\n    ReplyPid = comm:reply_as(self(), 5, {move, continue, MoveFullId, prepare_send_data2, '_'}),\n    SlideOp2 = slide_op:set_phase(SlideOp1, {wait_for_continue, wait_for_other}),\n    SlideMod = get_slide_mod(),\n    case SlideMod:prepare_send_data1(State, SlideOp2, ReplyPid) of\n        {ok, State1, SlideOp3} ->\n            PredOrSucc = slide_op:get_predORsucc(SlideOp3),\n            dht_node_state:set_slide(State1, PredOrSucc, SlideOp3);\n        {abort, Reason, State1, SlideOp3} ->\n            abort_slide(State1, SlideOp3, Reason, true, MoveMsgTag)\n    end.\n\n%% @doc Gets all data in the slide operation's interval from the DB and sends\n%%      it to the target node. Also sets the DB to record changes in this\n%%      interval and changes the slide operation's phase to wait_for_data_ack.\n%% @see slide_chord:prepare_send_data2/3\n-spec prepare_send_data2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                         EmbeddedMsg::comm:message()) -> dht_node_state:state().\nprepare_send_data2(State, SlideOp, EmbeddedMsg) ->\n    SlideMod = get_slide_mod(),\n    case SlideMod:prepare_send_data2(State, SlideOp, EmbeddedMsg) of\n        {ok, State1, SlideOp1} ->\n            % last part of a leave? -> transfer all DB entries!\n            % since in this case there is no other slide, we can safely use intervals:all()\n            MovingInterval =\n                case slide_op:is_leave(SlideOp1)\n                         andalso slide_op:get_next_op(SlideOp1) =:= {none} of\n                    true  -> intervals:all();\n                    false -> slide_op:get_interval(SlideOp1)\n                end,\n            {State2, MovingData} = dht_node_state:slide_get_data_start_record(\n                                          State1, MovingInterval),\n            SlideOp2 = slide_op:set_phase(SlideOp1, wait_for_data_ack),\n            Msg = {move, data, MovingData, slide_op:get_id(SlideOp2),\n                   slide_op:get_target_id(SlideOp2),\n                   slide_op:get_next_op(SlideOp2)},\n            send2(State2, SlideOp2, Msg);\n        {abort, Reason, State1, SlideOp1} ->\n            abort_slide(State1, SlideOp1, Reason, true, continue)\n    end.\n\n%% @doc Accepts data received during the given (existing!) slide operation and\n%%      writes it to the DB.\n%% @see slide_chord:update_rcv_data1/2\n-spec update_rcv_data1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                       Data::dht_node_state:slide_data(), TargetId::?RT:key(),\n                       NextOp::slide_op:next_op(), MoveMsgTag::atom())\n        -> dht_node_state:state().\nupdate_rcv_data1(State, SlideOp0, Data, TargetId, NextOp, MoveMsgTag) ->\n    SlideOp = slide_op:set_setup_at_other(SlideOp0),\n    MoveFullId = slide_op:get_id(SlideOp),\n    PredOrSucc = slide_op:get_predORsucc(SlideOp),\n    State1 = update_target_on_existing_slide(\n               SlideOp, State, TargetId, NextOp, MoveMsgTag),\n    case dht_node_state:get_slide(State1, MoveFullId) of\n        {PredOrSucc, SlideOp1} ->\n            MoveFullId = slide_op:get_id(SlideOp1),\n            ReplyPid = comm:reply_as(self(), 5, {move, continue, MoveFullId, update_rcv_data2, '_'}),\n            SlideOp2 = slide_op:set_phase(SlideOp1, {wait_for_continue, wait_for_data}),\n            State2 = dht_node_state:slide_add_data(State1, Data),\n            SlideMod = get_slide_mod(),\n            case SlideMod:update_rcv_data1(State2, SlideOp2, ReplyPid) of\n                {ok, State3, SlideOp3} ->\n                    PredOrSucc = slide_op:get_predORsucc(SlideOp3),\n                    dht_node_state:set_slide(State3, PredOrSucc, SlideOp3);\n                {abort, Reason, State3, SlideOp3} ->\n                    abort_slide(State3, SlideOp3, Reason, true, MoveMsgTag)\n            end;\n        not_found ->\n            State1 % if aborted\n    end.\n\n%% @doc Sends data_ack message and progresses to the next phase, i.e. wait_for_delta.\n%% @see slide_chord:prepare_send_delta2/3\n-spec update_rcv_data2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                       EmbeddedMsg::comm:message()) -> dht_node_state:state().\nupdate_rcv_data2(State, SlideOp, EmbeddedMsg) ->\n    SlideMod = get_slide_mod(),\n    case SlideMod:update_rcv_data2(State, SlideOp, EmbeddedMsg) of\n        {ok, State1, SlideOp1} ->\n            SlideOp2 = slide_op:set_phase(SlideOp1, wait_for_delta),\n            Msg = {move, data_ack, slide_op:get_id(SlideOp2)},\n            send2(State1, SlideOp2, Msg);\n        {abort, Reason, State1, SlideOp1} ->\n            abort_slide(State1, SlideOp1, Reason, true, continue)\n    end.\n\n%% @doc Prepares to send a delta message for the given (existing!) slide operation and\n%%      continues.\n%% @see slide_chord:prepare_send_delta1/3\n-spec prepare_send_delta1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                          MoveMsgTag::atom()) -> dht_node_state:state().\nprepare_send_delta1(State, OldSlideOp, MoveMsgTag) ->\n    MoveFullId = slide_op:get_id(OldSlideOp),\n    ReplyPid = comm:reply_as(self(), 5, {move, continue, MoveFullId, prepare_send_delta2, '_'}),\n    SlideOp1 = slide_op:set_phase(OldSlideOp, {wait_for_continue, wait_for_data_ack}),\n    SlideMod = get_slide_mod(),\n    case SlideMod:prepare_send_delta1(State, SlideOp1, ReplyPid) of\n        {ok, State1, SlideOp2} ->\n            PredOrSucc = slide_op:get_predORsucc(SlideOp2),\n            dht_node_state:set_slide(State1, PredOrSucc, SlideOp2);\n        {abort, Reason, State1, SlideOp2} ->\n            abort_slide(State1, SlideOp2, Reason, true, MoveMsgTag)\n    end.\n\n%% @doc Gets changed data in the slide operation's interval from the DB and\n%%      sends it as a delta to the target node. Also sets the DB to stop\n%%      recording changes in this interval and delete any such entries. Changes\n%%      the slide operation's phase to wait_for_delta_ack.\n%% @see slide_chord:prepare_send_delta2/3\n-spec prepare_send_delta2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                          EmbeddedMsg::comm:message()) -> dht_node_state:state().\nprepare_send_delta2(State, SlideOp, EmbeddedMsg) ->\n    SlideMod = get_slide_mod(),\n    case SlideMod:prepare_send_delta2(State, SlideOp, EmbeddedMsg) of\n        {ok, State1, SlideOp1} ->\n            % last part of a leave? -> transfer all DB entries!\n            % since in this case there is no other slide, we can safely use intervals:all()\n            SlideOpInterval =\n                case slide_op:is_leave(SlideOp1) andalso not slide_op:is_jump(SlideOp1)\n                         andalso slide_op:get_next_op(SlideOp1) =:= {none} of\n                    true  -> intervals:all();\n                    false -> slide_op:get_interval(SlideOp1)\n                end,\n            {State2, ChangedData} = dht_node_state:slide_take_delta_stop_record(\n                                      State1, SlideOpInterval),\n            % send delta (values of keys that have changed during the move)\n            SlideOp2 = slide_op:set_phase(SlideOp1, wait_for_delta_ack),\n            Msg = {move, delta, ChangedData, slide_op:get_id(SlideOp2)},\n            send2(State2, SlideOp2, Msg);\n        {abort, Reason, State1, SlideOp1} ->\n            abort_slide(State1, SlideOp1, Reason, true, continue)\n    end.\n\n%% @doc Accepts delta received during the given (existing!) slide operation and\n%%      continues.\n%% @see slide_chord:finish_delta1/4\n-spec finish_delta1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                   ChangedData::dht_node_state:slide_delta(), MoveMsgTag::atom())\n        -> dht_node_state:state().\nfinish_delta1(State, OldSlideOp, ChangedData, MoveMsgTag) ->\n    MoveFullId = slide_op:get_id(OldSlideOp),\n    ReplyPid = comm:reply_as(self(), 5, {move, continue, MoveFullId, finish_delta2, '_'}),\n    SlideOp1 = slide_op:set_phase(OldSlideOp, {wait_for_continue, wait_for_delta}),\n    State1 = dht_node_state:slide_add_delta(State, ChangedData),\n    SlideMod = get_slide_mod(),\n    case SlideMod:finish_delta1(State1, SlideOp1, ReplyPid) of\n        {ok, State2, SlideOp2} ->\n            PredOrSucc = slide_op:get_predORsucc(SlideOp2),\n            dht_node_state:set_slide(State2, PredOrSucc, SlideOp2);\n        {abort, Reason, State2, SlideOp2} ->\n            abort_slide(State2, SlideOp2, Reason, true, MoveMsgTag)\n    end.\n\n-spec send_delta_ack(SlideOp::slide_op:slide_op(), NextOp::next_op_msg()) -> ok.\nsend_delta_ack(SlideOp, NextOp) ->\n    Pid = node:pidX(slide_op:get_node(SlideOp)),\n    Msg = {move, delta_ack, slide_op:get_id(SlideOp), NextOp},\n    send_no_slide(Pid, Msg, 0).\n\n-spec finish_slide(State::dht_node_state:state(), SlideOp::slide_op:slide_op())\n        -> dht_node_state:state().\nfinish_slide(State, SlideOp) ->\n    Pid = node:pidX(slide_op:get_node(SlideOp)),\n    MoveFullId = slide_op:get_id(SlideOp),\n    fd:unsubscribe(?FD_SUBSCR_PID(MoveFullId), [Pid]),\n    case slide_op:is_jump(SlideOp) of\n        true -> ok; %% notify later when join at other node is complete\n        _ -> notify_source_pid(slide_op:get_source_pid(SlideOp),\n                               {move, result, slide_op:get_tag(SlideOp), ok})\n    end,\n    PredOrSucc = slide_op:get_predORsucc(SlideOp),\n    rm_loop:notify_slide_finished(PredOrSucc),\n    dht_node_state:set_slide(State, PredOrSucc, null).\n\n-spec finish_delta2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                    EmbeddedMsg::comm:message()) -> dht_node_state:state().\nfinish_delta2(State, SlideOp, EmbeddedMsg) ->\n    SlideMod = get_slide_mod(),\n    case SlideMod:finish_delta2(State, SlideOp, EmbeddedMsg) of\n        {ok, State1, SlideOp1} ->\n            % continue with the next planned operation:\n            case slide_op:is_incremental(SlideOp1) of\n                true ->\n                    Type = slide_op:get_type(SlideOp1),\n                    PredOrSucc = slide_op:get_predORsucc(SlideOp1),\n                    % TODO: support other types\n                    case slide_op:get_next_op(SlideOp1) of\n                        {slide, continue, NewTargetId} when Type =:= {slide, PredOrSucc, 'rcv'} orelse\n                                                                Type =:= {join, 'rcv'} ->\n                            Type1 = {slide, PredOrSucc, 'rcv'}, % converts join\n                            finish_delta2B(State1, SlideOp1, Type1, NewTargetId);\n                        {leave, continue} ->\n                            % this is not the correct target ID, but the leaving node will set it itself\n                            NewTargetId = dht_node_state:get(State, pred_id),\n                            finish_delta2B(State1, SlideOp1, Type, NewTargetId)\n                    end;\n                _ ->\n                    % note: send delta_ack and a potential new slide setup in two\n                    %       messages so new op suggestions also have a chance to be\n                    %       setup instead\n                    send_delta_ack(SlideOp1, {none}),\n                    finish_slide_and_continue_with_next_op(State1, SlideOp1)\n            end;\n        {abort, Reason, State1, SlideOp1} ->\n            % an abort at this stage is really useless (data has been fully integrated!)\n            % nevertheless at least the source can be notified... \n            abort_slide(State1, SlideOp1, Reason, true, continue)\n    end.\n\n%% @doc Finish after receiving delta and continue with the (incremental) slide.\n-spec finish_delta2B(State1::dht_node_state:state(), SlideOp1::slide_op:slide_op(),\n                     Type1::slide_op:type(), NewTargetId::?RT:key())\n        -> dht_node_state:state().\nfinish_delta2B(State, SlideOp, Type, NewTargetId) ->\n    SlideMod = get_slide_mod(),\n    PredOrSucc = slide_op:get_predORsucc(SlideOp),\n    State1 = dht_node_state:set_slide(State, PredOrSucc, null),\n    MoveFullId = slide_op:get_id(SlideOp),\n    Tag = slide_op:get_tag(SlideOp),\n    SourcePid = slide_op:get_source_pid(SlideOp),\n    % note: the node of the slide op may be outdated; if the current\n    %       PredOrSucc does not point to the same node though, abort!\n    TargetNode = dht_node_state:get(State1, PredOrSucc),\n    SlideOpNode = slide_op:get_node(SlideOp),\n    TargetNodePid = node:pidX(SlideOpNode), % the pid should be the same as TargetNode!\n    % unsubscribe old slide from fd:\n    fd:unsubscribe(?FD_SUBSCR_PID(MoveFullId), [TargetNodePid]),\n    NewMoveFullId = uid:get_global_uid(),\n    case node:same_process(TargetNode, SlideOpNode) of\n        true ->\n            MyNode = dht_node_state:get(State1, node),\n            OtherMTE = slide_op:get_other_max_entries(SlideOp),\n            Command = check_setup_slide_not_found(\n                        State1, Type, MyNode, TargetNode, NewTargetId),\n            case Command of\n                {ok, NewType} when NewType =:= {slide, PredOrSucc, 'rcv'} orelse\n                                       NewType =:= {leave, 'rcv'} ->\n                    Neighbors = dht_node_state:get(State1, neighbors),\n                    % continued slide with pred/succ, receive data\n                    % -> reserve slide_op with pred/succ\n                    % note: we are already subscribed to TargetNode but with the old MoveID\n                    fd:subscribe(?FD_SUBSCR_PID(NewMoveFullId), [TargetNodePid]),\n                    NextSlideOp =\n                        slide_op:new_slide(\n                          NewMoveFullId, NewType, NewTargetId, Tag,\n                          SourcePid, OtherMTE, {none}, Neighbors),\n                    % note: phase will be set by notify_other/2 and needs to remain null here\n                    case SlideMod:prepare_rcv_data(State1, NextSlideOp) of\n                        {ok, State3, NextSlideOp1} ->\n                            NextSlideOp2 = slide_op:set_phase(NextSlideOp1, wait_for_data),\n                            Msg = {move, delta_ack, MoveFullId, {continue, NewMoveFullId}},\n                            send2(State3, NextSlideOp2, Msg);\n                        {abort, Reason, State3, NextSlideOp1} ->\n                            % let this op finish and abort the continued one:\n                            send_delta_ack(SlideOp, {abort, NewMoveFullId, Reason}),\n                            abort_slide(State3, NextSlideOp1, Reason, false, continue)\n                    end;\n                {abort, Reason, NewType} -> % note: the type returned here is the same as Type\n                    % let this op finish and abort the continued one:\n                    send_delta_ack(SlideOp, {abort, NewMoveFullId, Reason}),\n                    abort_slide(State1, TargetNode, NewMoveFullId, null, SourcePid, Tag,\n                                NewType, Reason, false)\n            end;\n        false ->\n            % let this op finish and abort the continued one:\n            Reason = wrong_pred_succ_node,\n            send_delta_ack(SlideOp, {abort, NewMoveFullId, Reason}),\n            abort_slide(State1, SlideOpNode, NewMoveFullId, null, SourcePid, Tag,\n                        Type, Reason, false)\n    end.\n\n%% @doc Extracts the next planned operation from OldSlideOp and sets it up.\n%%      Note: Next ops from incremental slides will be silently ignored here\n%%            and must be handled elsewhere!\n-spec finish_slide_and_continue_with_next_op(State::dht_node_state:state(), OldSlideOp::slide_op:slide_op())\n        -> dht_node_state:state().\nfinish_slide_and_continue_with_next_op(State0, OldSlideOp) ->\n    State = finish_slide(State0, OldSlideOp),\n    case slide_op:get_next_op(OldSlideOp) of\n        {none} -> State;\n        % ignore incremental slide ops\n        {slide, continue, _Id} -> State;\n        {leave, continue} -> State;\n        {slide, PredOrSucc, NewTargetId, NewTag, NewSourcePid} ->\n            % continue operation with the same node previously sliding with or\n            % try setting up a slide with the other node\n            make_slide(State, PredOrSucc, NewTargetId, NewTag, NewSourcePid);\n        {jump, NewTargetId, NewTag, NewSourcePid} ->\n            make_jump(State, NewTargetId, NewTag, NewSourcePid);\n        {leave, NewSourcePid} ->\n            make_slide_leave(State, NewSourcePid)\n    end.\n\n%% @doc Accepts delta_ack received during the given (existing!) slide operation and\n%%      continues.\n%% @see slide_chord:finish_delta_ack1/4\n-spec finish_delta_ack1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                        NextOpMsg::next_op_msg(), MoveMsgTag::atom())\n        -> dht_node_state:state().\nfinish_delta_ack1(State, OldSlideOp, NextOpMsg, MoveMsgTag) ->\n    MoveFullId = slide_op:get_id(OldSlideOp),\n    ReplyPid = comm:reply_as(self(), 5, {move, continue, MoveFullId, {finish_delta_ack2, NextOpMsg}, '_'}),\n    SlideOp1 = slide_op:set_phase(OldSlideOp, {wait_for_continue, wait_for_delta_ack}),\n    SlideMod = get_slide_mod(),\n    case SlideMod:finish_delta_ack1(State, SlideOp1, ReplyPid) of\n        {ok, State1, SlideOp2} ->\n            PredOrSucc = slide_op:get_predORsucc(SlideOp2),\n            dht_node_state:set_slide(State1, PredOrSucc, SlideOp2);\n        {abort, Reason, State1, SlideOp2} ->\n            abort_slide(State1, SlideOp2, Reason, true, MoveMsgTag)\n    end.\n\n-spec finish_delta_ack2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                        NextOp::next_op_msg(), EmbeddedMsg::comm:message())\n        -> dht_node_state:state().\nfinish_delta_ack2(State, SlideOp, NextOpMsg, EmbeddedMsg) ->\n    SlideMod = get_slide_mod(),\n    case SlideMod:finish_delta_ack2(State, SlideOp, NextOpMsg, EmbeddedMsg) of\n        {ok, State1, SlideOp1, NextOpMsg1} ->\n            IsJump = slide_op:is_jump(SlideOp1),\n            NextOpMsg2 =\n                case slide_op:is_leave(SlideOp1)\n                         andalso NextOpMsg1 =:= {none} of\n                    true when not IsJump -> \n                        {finish_leave};\n                    true when IsJump ->\n                        {finish_jump};\n                    false -> \n                        NextOpMsg1\n                end,\n            finish_delta_ack2B(State1, SlideOp1, NextOpMsg2);\n        {abort, Reason, State1, SlideOp1} ->\n            abort_slide(State1, SlideOp1, Reason, true, continue)\n    end.\n\n%% Pre: SlideOp is no finished leaving slide (see finish_delta_ack2/3)\n-spec finish_delta_ack2B(\n        State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n        NextOpMsg::next_op_msg() |\n          {finish_leave} |\n          {NextOpType::slide_op:type(), NewSlideId::slide_op:id(),\n          InitNode::node:node_type(), TargetNode::node:node_type(),\n          TargetId::?RT:key(), Tag::any(), SourcePid::comm:mypid() | null})\n        -> dht_node_state:state().\nfinish_delta_ack2B(State, SlideOp, {finish_leave}) ->\n    SupDhtNode = pid_groups:get_my(sup_dht_node),\n    % note: the prepare_send_data1 callback already notified nodes subscribed to fd\n    State1 = finish_slide(State, SlideOp),\n    SupDhtNodeId = erlang:get(my_sup_dht_node_id),\n    comm:send_local(whereis(service_per_vm),\n                    {delete_node, SupDhtNode, SupDhtNodeId}),\n    % note: we will be killed soon but need to be removed from the supervisor first\n    % -> do not kill this process\n    State1;\nfinish_delta_ack2B(State, SlideOp, {finish_jump}) ->\n    NewId = slide_op:get_jump_target_id(SlideOp),\n    % note: the prepare_send_data1 callback already notified nodes subscribed to fd\n    State1 = finish_slide(State, SlideOp),\n\n    %% Rejoin at NewId but keep processes\n    SupDhtNodeId = erlang:get(my_sup_dht_node_id),\n    %% Get additional nodes for bootstrapping\n    %% If the only known host jumps, it can't bootstrap itself.\n    Neighborhood = dht_node_state:get(State, neighbors),\n    OtherNodes = tl(nodelist:to_list(Neighborhood)),\n    BootstrapNodes = [node:pidX(Node) || Node <- OtherNodes],\n    %% reply after join is complete\n    JumpTag = slide_op:get_tag(SlideOp),\n    SourcePid = slide_op:get_source_pid(SlideOp),\n    JoinOptions = [{{dht_node, id}, NewId}, {my_sup_dht_node_id, SupDhtNodeId},\n                   {jump, JumpTag, SourcePid},\n                   {bootstrap_nodes, BootstrapNodes}],\n    JoinOptions2 =\n        case config:read(lb_active_and_psv) of\n            true -> JoinOptions;\n            _    -> [{skip_psv_lb} | JoinOptions]\n        end,\n    %% Request move state of rm before rejoining\n    ReplyMsg = {rejoin, NewId, JoinOptions2, '_'},\n    ReplyPid = comm:reply_as(self(), 4, ReplyMsg),\n    comm:send_local(self(), {rm, get_move_state, ReplyPid}),\n    State1;\nfinish_delta_ack2B(State, SlideOp, {none}) ->\n    finish_slide_and_continue_with_next_op(State, SlideOp);\nfinish_delta_ack2B(State, SlideOp, {abort, NewSlideId, Reason}) ->\n    Type = slide_op:get_type(SlideOp),\n    PredOrSucc = slide_op:get_predORsucc(Type),\n    TargetNode = dht_node_state:get(State, PredOrSucc),\n    Tag = slide_op:get_tag(SlideOp),\n    SourcePid = slide_op:get_source_pid(SlideOp),\n    State2 = finish_slide_and_continue_with_next_op(State, SlideOp),\n    abort_slide(State2, TargetNode, NewSlideId, null, SourcePid, Tag,\n                Type, Reason, false);\nfinish_delta_ack2B(State, SlideOp, {continue, NewSlideId}) ->\n    MyNode = dht_node_state:get(State, node),\n    Type = slide_op:get_type(SlideOp),\n    PredOrSucc = slide_op:get_predORsucc(Type),\n    TargetNode = dht_node_state:get(State, PredOrSucc),\n    Tag = slide_op:get_tag(SlideOp),\n    SourcePid = slide_op:get_source_pid(SlideOp),\n    % TODO: support other types\n    case slide_op:get_next_op(SlideOp) of\n        {slide, continue, NewTargetId} when Type =:= {slide, PredOrSucc, 'send'} orelse\n                                                Type =:= {join, 'send'} ->\n            Type1 = {slide, PredOrSucc, 'send'}, % converts join\n            finish_delta_ack2B(\n              State, SlideOp, {Type1, NewSlideId, MyNode,\n                               TargetNode, NewTargetId, Tag, SourcePid});\n        {leave, continue} when Type =:= {leave, 'send'} ->\n            NewTargetId = dht_node_state:get(State, pred_id),\n            finish_delta_ack2B(\n              State, SlideOp, {Type, NewSlideId, MyNode,\n                               TargetNode, NewTargetId, Tag, SourcePid});\n        {leave, continue} when Type =:= {jump, 'send'} ->\n            %% keep the old jump target id\n            NewTargetId = slide_op:get_jump_target_id(SlideOp),\n            finish_delta_ack2B(\n              State, SlideOp, {Type, NewSlideId, MyNode,\n                               TargetNode, NewTargetId, Tag, SourcePid});\n        _ -> % our next op is different from the other node's next op\n            % TODO\n            abort_slide(State, SlideOp, next_op_mismatch, true, continue)\n    end;\nfinish_delta_ack2B(State, SlideOp, {MyNextOpType, NewSlideId, MyNode,\n                                   TargetNode, TargetId, Tag, SourcePid}) ->\n    fd:unsubscribe(?FD_SUBSCR_PID(slide_op:get_id(SlideOp)), [node:pidX(slide_op:get_node(SlideOp))]),\n    % Always prefer the other node's next_op over ours as it is almost\n    % set up. Unless our scheduled op is a leave operation which needs\n    % to be favoured.\n    case slide_op:get_next_op(SlideOp) of\n        {leave, NewSourcePid} when NewSourcePid =/= continue ->\n            State1 = abort_slide(State, TargetNode, NewSlideId, null, SourcePid,\n                                 Tag, MyNextOpType, scheduled_leave, true),\n            make_slide_leave(State1, NewSourcePid);\n        MyNextOp ->\n            % TODO: check if warnings are generated in all cases\n            case MyNextOp of\n                {none} -> ok;\n                {slide, continue, TargetId} -> ok;\n                {leave, continue} -> ok;\n                _ ->\n                    log:log(info, \"[ dht_node_move ~.0p ] removing \"\n                                \"scheduled next op ~.0p, \"\n                                \"got next op: ~.0p\",\n                            [comm:this(), MyNextOp, MyNextOpType])\n            end,\n            State1 = dht_node_state:set_slide(\n                       State, slide_op:get_predORsucc(SlideOp), null),\n            Command = check_setup_slide_not_found(\n                        State1, MyNextOpType, MyNode, TargetNode, TargetId),\n            exec_setup_slide_not_found(\n              Command, State1, NewSlideId, TargetNode, TargetId,\n              Tag, slide_op:get_other_max_entries(SlideOp), SourcePid,\n              delta_ack, {none}, false)\n    end.\n\n%% @doc Checks if a slide operation with the given MoveFullId exists and\n%%      executes WorkerFun if everything is ok. If the successor/predecessor\n%%      information in the slide operation is incorrect, the slide is aborted\n%%      (a message to the pred/succ is send, too). An exception is made for a\n%%      crashed_node message which would naturally result in a wrong_neighbor!\n-spec safe_operation(\n    WorkerFun::fun((SlideOp::slide_op:slide_op(), State::dht_node_state:state())\n                    -> dht_node_state:state()),\n    State::dht_node_state:state(), MoveFullId::slide_op:id(),\n    WorkPhases::[slide_op:phase(),...] | all,\n    MoveMsgTag::atom(), IgnoreWrongNeighbor::boolean()) -> dht_node_state:state().\nsafe_operation(WorkerFun, State, MoveFullId, WorkPhases, MoveMsgTag, IgnoreWrongNeighbor) ->\n    case get_slide(State, MoveFullId) of\n        {_, _PredOrSucc, SlideOp} when MoveMsgTag =:= crashed_node ->\n            WorkerFun(SlideOp, State);\n        {Status, _PredOrSucc, SlideOp} when Status =:= ok\n          orelse (IgnoreWrongNeighbor andalso Status =:= wrong_neighbor)->\n            case WorkPhases =:= all orelse lists:member(slide_op:get_phase(SlideOp), WorkPhases) of\n                true -> WorkerFun(SlideOp, State);\n                _    ->\n                    log:log(info, \"[ dht_node_move ~.0p ] unexpected message ~.0p received in phase ~.0p\",\n                            [comm:this(), MoveMsgTag, slide_op:get_phase(SlideOp)]),\n                    State\n            end;\n        not_found when MoveMsgTag =:= send_error_retry ->\n            State;\n        not_found ->\n            log:log(warn,\"[ dht_node_move ~.0p ] ~.0p received with no matching \"\n                   \"slide operation (ID: ~.0p, slide_pred: ~.0p, slide_succ: ~.0p)~n\",\n                    [comm:this(), MoveMsgTag, MoveFullId,\n                     dht_node_state:get(State, slide_pred),\n                     dht_node_state:get(State, slide_succ)]),\n            State;\n        {wrong_neighbor, PredOrSucc, SlideOp0} -> % wrong pred or succ\n            SlideOp = case MoveMsgTag of\n                          data -> slide_op:set_setup_at_other(SlideOp0);\n                          _    -> SlideOp0\n                      end,\n            case WorkPhases =:= all orelse lists:member(slide_op:get_phase(SlideOp), WorkPhases) of\n                true ->\n                    log:log(warn,\"[ dht_node_move ~.0p ] ~.0p received but ~s \"\n                           \"changed during move (ID: ~.0p, node(slide): ~.0p, new_~s: ~.0p)~n\",\n                            [comm:this(), MoveMsgTag, PredOrSucc, MoveFullId,\n                             slide_op:get_node(SlideOp),\n                             PredOrSucc, dht_node_state:get(State, PredOrSucc)]),\n                    abort_slide(State, SlideOp, wrong_pred_succ_node, true, MoveMsgTag);\n                _ -> State\n            end\n    end.\n\n%% @doc Tries to find a slide operation with the given MoveFullId and returns\n%%      it including its type (pred or succ) if successful and its pred/succ\n%%      info is correct (a wrong predecessor is tolerated if the slide\n%%      operation is a leave since the node will leave and thus we will get\n%%      a new predecessor). Otherwise returns {fail, wrong_pred} if the\n%%      predecessor info is wrong (slide with pred) and {fail, wrong_succ} if\n%%      the successor info is wrong (slide with succ). If not found,\n%%      {fail, not_found} is returned.\n-spec get_slide(State::dht_node_state:state(), MoveFullId::slide_op:id()) ->\n        {Result::ok, Type::pred | succ, SlideOp::slide_op:slide_op()} |\n        {Result::wrong_neighbor, Type::pred | succ, SlideOp::slide_op:slide_op()} |\n        not_found.\nget_slide(State, MoveFullId) ->\n    case dht_node_state:get_slide(State, MoveFullId) of\n        not_found -> not_found;\n        {PredOrSucc, SlideOp} ->\n            Node = dht_node_state:get(State, PredOrSucc),\n            NodeSlOp = slide_op:get_node(SlideOp),\n            % - allow changed pred during a leave if the new pred is not between\n            %   the leaving node and the current node!\n            % - allow outdated pred during join (as an existing node) if the new\n            %   pred is in the current range\n            case node:same_process(Node, NodeSlOp) orelse\n                     (PredOrSucc =:= pred andalso\n                          (slide_op:is_leave(SlideOp) orelse\n                               slide_op:is_join(SlideOp)) andalso\n                          intervals:in(node:id(NodeSlOp), dht_node_state:get(State, my_range))) of\n                true -> {ok,             PredOrSucc, SlideOp};\n                _    -> {wrong_neighbor, PredOrSucc, SlideOp}\n            end\n    end.\n\n%% @doc Returns whether a slide with the successor is possible for the given\n%%      target id.\n%%      1) TargetId not interfering with SlidePred?\n%%      2) no left-over DBRange (from send-to-succ) waiting for RM-update\n%%         -> needs to be integrated into my_range first to proceed (most code\n%%            only uses my_range to create intervals!)\n%% @see can_slide_pred/5\n-spec can_slide_succ(SlidePred::Slide, SlideSucc::Slide,\n                     DBRange::[{intervals:interval(), slide_op:id()}],\n                     TargetId::?RT:key(), Type::slide_op:type()) -> boolean()\n        when is_subtype(Slide, slide_op:slide_op() | null).\ncan_slide_succ(null, null, [], _TargetId, _Type) ->\n    true;\ncan_slide_succ(null, null, [_|_], _TargetId, _Type) ->\n    false;\ncan_slide_succ(SlidePred, null, [], TargetId, Type) ->\n    % TargetId not interfering with SlidePred?\n    not intervals:in(TargetId, slide_op:get_interval(SlidePred)) andalso\n        not (slide_op:is_leave(Type) andalso slide_op:is_leave(SlidePred));\ncan_slide_succ(SlidePred, null, [{_I, SlideId}], TargetId, Type) ->\n    % no left-over DBRange (from send-to-succ) waiting for RM-update\n    SlideId =:= slide_op:get_id(SlidePred) andalso\n        % TargetId not interfering with SlidePred?\n        (not intervals:in(TargetId, slide_op:get_interval(SlidePred)) andalso\n             not (slide_op:is_leave(Type) andalso slide_op:is_leave(SlidePred)));\ncan_slide_succ(_SlidePred, _SlideSucc, _DBRange, _TargetId, _Type) ->\n    false.\n\n%% @doc Returns whether a slide with the predecessor is possible for the given\n%%      target id.\n%%      1) TargetId not interfering with SlideSucc?\n%%      2) no left-over DBRange (from receive-from-pred) waiting for RM-update\n%%         -> needs to be integrated into my_range first to proceed (most code\n%%            only uses my_range to create intervals!)\n%% @see can_slide_succ/5\n-spec can_slide_pred(SlidePred::Slide, SlideSucc::Slide,\n                     DBRange::[{intervals:interval(), slide_op:id()}],\n                     TargetId::?RT:key(), Type::slide_op:type()) -> boolean()\n        when is_subtype(Slide, slide_op:slide_op() | null).\ncan_slide_pred(null, null, [], _TargetId, _Type) ->\n    true;\ncan_slide_pred(null, null, [_|_], _TargetId, _Type) ->\n    false;\ncan_slide_pred(null, SlideSucc, [], TargetId, _Type) ->\n    % TargetId not interfering with SlideSucc?\n    not intervals:in(TargetId, slide_op:get_interval(SlideSucc)) andalso\n         not slide_op:is_leave(SlideSucc);\ncan_slide_pred(null, SlideSucc, [{_I, SlideId}], TargetId, _Type) ->\n    % no left-over DBRange (from receive-from-pred) waiting for RM-update\n    SlideId =:= slide_op:get_id(SlideSucc) andalso\n        % TargetId not interfering with SlideSucc?\n        (not intervals:in(TargetId, slide_op:get_interval(SlideSucc)) andalso\n             not slide_op:is_leave(SlideSucc)\n        );\ncan_slide_pred(_SlidePred, _SlideSucc, _DBRange, _TargetId, _Type) ->\n    false.\n\n%% @doc Sends the source pid the given message if it is not 'null'.\n-spec notify_source_pid(SourcePid::comm:mypid() | null, Message::result_message()) -> ok.\nnotify_source_pid(SourcePid, Message) ->\n    case SourcePid of\n        null -> ok;\n        _ -> ?TRACE_SEND(SourcePid, Message),\n             comm:send(SourcePid, Message)\n    end.\n\n%% @doc Updates TargetId and NextOp after receiving it along with a data message.\n-spec update_target_on_existing_slide(\n        OldSlideOp::slide_op:slide_op(), State::dht_node_state:state(),\n        TargetId::?RT:key(), NextOp::slide_op:next_op(), MoveMsgTag::atom())\n        -> dht_node_state:state().\nupdate_target_on_existing_slide(OldSlideOp, State, TargetId, NextOp, MoveMsgTag) ->\n            PredOrSucc = slide_op:get_predORsucc(OldSlideOp),\n    case slide_op:get_target_id(OldSlideOp) of\n        TargetId ->\n            SlideOp1 = slide_op:set_next_op(OldSlideOp, NextOp),\n            dht_node_state:set_slide(State, PredOrSucc, SlideOp1);\n        _ ->\n            SendOrReceive = slide_op:get_sendORreceive(OldSlideOp),\n            AllowedI =\n                if PredOrSucc =:= succ andalso SendOrReceive =:= 'rcv' ->\n                       % new target ID can only be between my current ID and the old target ID!\n                       OldTargetId = slide_op:get_target_id(OldSlideOp),\n                       MyId = dht_node_state:get(State, node_id),\n                       node:mk_interval_between_ids(MyId, OldTargetId);\n                   PredOrSucc =:= succ andalso SendOrReceive =:= 'send' ->\n                       % new target ID can only be between the old target ID and my current ID!\n                       OldTargetId = slide_op:get_target_id(OldSlideOp),\n                       MyId = dht_node_state:get(State, node_id),\n                       node:mk_interval_between_ids(OldTargetId, MyId);\n                   PredOrSucc =:= pred ->\n                       % we cannot really check anything here with chord as the pred may have already changed\n                       intervals:all()\n                end,\n            case intervals:in(TargetId, AllowedI) of\n                true ->\n                    % TODO: if there is any other NextOp planned, abort that!\n                    % (currently there is no mechanism to add NextOp's other than\n                    % incremental slides, so it is ok to just remove the old one for now)\n                    SlideOp1 = slide_op:update_target_id(\n                                 OldSlideOp, TargetId, NextOp,\n                                 dht_node_state:get(State, neighbors)),\n                    \n                    MoveFullId = slide_op:get_id(SlideOp1),\n                    HasDBRange = lists:any(fun({_, Id}) -> Id =:= MoveFullId end,\n                                           dht_node_state:get(State, db_range)),\n                    State1 =\n                        if HasDBRange ->\n                               % assume we can add a DB range if it exists already\n                               dht_node_state:add_db_range(\n                                 dht_node_state:rm_db_range(State, MoveFullId),\n                                 slide_op:get_interval(SlideOp1), MoveFullId);\n                           true -> State\n                        end,\n                    dht_node_state:set_slide(State1, PredOrSucc, SlideOp1);\n                false ->\n                    log:log(warn,\"[ dht_node_move ~.0p ] new TargetId and NextOp received \"\n                                \"but not in allowed range (ID: ~.0p, node(slide): ~.0p, \"\n                                \"my_id: ~.0p, target_id: ~.0p, new_target_id: ~.0p)~n\",\n                            [comm:this(), slide_op:get_id(OldSlideOp),\n                             dht_node_state:get(State, node_id),\n                             slide_op:get_target_id(OldSlideOp), TargetId]),\n                    abort_slide(State, OldSlideOp, changed_parameters, true, MoveMsgTag)\n            end\n    end.\n\n%% @doc Re-creates a slide operation with the given (updated) parameters.\n-spec recreate_existing_slide(\n        OldSlideOp::slide_op:slide_op(), State::dht_node_state:state(),\n        TargetId::?RT:key(), OtherMaxTransportEntries::unknown | pos_integer(),\n        MsgTag::nomsg | slide | delta_ack,\n        NextOp::slide_op:next_op()) -> dht_node_state:state().\nrecreate_existing_slide(OldSlideOp, State, TargetId, OtherMTE, MsgTag, NextOp) ->\n    % TODO: if there is any other NextOp planned, abort that!\n    % (currently there is no mechanism to add NextOp's other than\n    % incremental slides, so it is ok to just remove the old one for now)\n    PredOrSucc = slide_op:get_predORsucc(OldSlideOp),\n    MoveFullId = slide_op:get_id(OldSlideOp),\n    % simply re-create the slide (TargetId or NextOp have changed)\n    State1 = dht_node_state:set_slide(State, PredOrSucc, null), % just in case\n    % note: msg_fwd are stored in the slide and do not require additional removal\n    State2 = dht_node_state:rm_db_range(State1, MoveFullId),\n    Command = {ok, slide_op:get_type(OldSlideOp)},\n    exec_setup_slide_not_found(\n      Command, State2, MoveFullId, slide_op:get_node(OldSlideOp), TargetId,\n      slide_op:get_tag(OldSlideOp), OtherMTE, slide_op:get_source_pid(OldSlideOp),\n      MsgTag, NextOp, true).\n\n%% @doc Aborts the given slide operation. Assume the SlideOp has already been\n%%      set in the dht_node and resets the according slide in its state to\n%%      null.\n%% @see abort_slide/8\n-spec abort_slide(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                  Reason::abort_reason(), NotifyNode::boolean(), MoveMsgTag::atom())\n        -> dht_node_state:state().\nabort_slide(State, SlideOp, Reason, NotifyNode, MoveMsgTag) ->\n    % write to log when aborting an already set-up slide:\n    case slide_op:is_setup_at_other(SlideOp) of\n        true ->\n            log:log(warn, \"[ dht_node_move ~.0p ] abort_slide(op: ~.0p, reason: ~.0p)~n\",\n                    [comm:this(), SlideOp, Reason]);\n        _ -> ok\n    end,\n    % potentially set up for joining nodes (slide with pred) or\n    % nodes sending data to their predecessor:\n    RMSubscrTag = {move, slide_op:get_id(SlideOp)},\n    rm_loop:unsubscribe(self(), RMSubscrTag),\n    State1 = dht_node_state:rm_db_range(State, slide_op:get_id(SlideOp)),\n    SlideMod = get_slide_mod(),\n    State2 = SlideMod:abort_slide(State1, SlideOp, Reason, MoveMsgTag),\n    % set a 'null' slide_op if there was an old one with the given ID\n    Type = slide_op:get_type(SlideOp),\n    PredOrSucc = slide_op:get_predORsucc(Type),\n    Node = slide_op:get_node(SlideOp),\n    Id = slide_op:get_id(SlideOp),\n    fd:unsubscribe(?FD_SUBSCR_PID(Id), [node:pidX(Node)]),\n    State3 = dht_node_state:set_slide(State2, PredOrSucc, null),\n    State4 = dht_node_state:slide_stop_record(State3, slide_op:get_interval(SlideOp), false),\n    abort_slide(State4, Node, Id, slide_op:get_phase(SlideOp),\n                slide_op:get_source_pid(SlideOp), slide_op:get_tag(SlideOp),\n                Type, Reason, NotifyNode).\n\n%% @doc Like abort_slide/5 but does not need a slide operation in order to\n%%      work. Note: prefer using abort_slide/5 when a slide operation is\n%%      available as this also resets all its timers!\n-spec abort_slide(State::dht_node_state:state(), Node::node:node_type(),\n                  SlideOpId::slide_op:id(), Phase::slide_op:phase(),\n                  SourcePid::comm:mypid() | null,\n                  Tag::any(), Type::slide_op:type(), Reason::abort_reason(),\n                  NotifyNode::boolean()) -> dht_node_state:state().\nabort_slide(State, Node, SlideOpId, _Phase, SourcePid, Tag, Type, Reason, NotifyNode) ->\n    PredOrSucc = slide_op:get_predORsucc(Type),\n    % abort slide on the (other) node:\n    case NotifyNode of\n        true ->\n            PredOrSuccOther = case PredOrSucc of\n                                  pred -> succ;\n                                  succ -> pred\n                              end,\n            NodePid = node:pidX(Node),\n            Msg = {move, slide_abort, PredOrSuccOther, SlideOpId, Reason},\n            send_no_slide(NodePid, Msg, 0);\n        _ -> ok\n    end,\n    % re-start a leaving slide on the leaving node if it hasn't left the ring yet:\n    case Reason =/= leave_no_partner_found andalso\n             slide_op:is_leave(Type, 'send') andalso\n             not slide_op:is_jump(Type) of\n        true -> comm:send_local(self(), {leave, SourcePid}),\n                State;\n        _    -> notify_source_pid(SourcePid, {move, result, Tag, Reason}),\n                State\n    end.\n\n%% @doc Creates a slide with the node's successor or predecessor. TargetId will\n%%      become the ID between the two nodes, i.e. either the current node or\n%%      the other node will change its ID to TargetId. SourcePid will be\n%%      notified about the result.\n-spec make_slide(State::dht_node_state:state(), pred | succ, TargetId::?RT:key(),\n        Tag::any(), SourcePid::comm:mypid() | null) -> dht_node_state:state().\nmake_slide(State, PredOrSucc, TargetId, Tag, SourcePid) ->\n    % slide with PredOrSucc possible? if so, receive or send data?\n    Neighbors = dht_node_state:get(State, neighbors),\n    SendOrReceive =\n        case PredOrSucc of\n            succ ->\n                case intervals:in(TargetId, nodelist:succ_range(Neighbors)) of\n                    true -> 'rcv';\n                    _    -> 'send'\n                end;\n            pred ->\n                case intervals:in(TargetId, nodelist:node_range(Neighbors)) of\n                    true -> 'send';\n                    _    -> 'rcv'\n                end\n        end,\n    MoveFullId = uid:get_global_uid(),\n    MyNode = nodelist:node(Neighbors),\n    TargetNode = nodelist:PredOrSucc(Neighbors),\n    setup_slide(State, {slide, PredOrSucc, SendOrReceive},\n                MoveFullId, MyNode, TargetNode, TargetId, Tag,\n                unknown, SourcePid, nomsg, {none}).\n\n%% @doc Creates a slide with the node's predecessor. The predecessor will\n%%      change its ID to TargetId, SourcePid will be notified about the result.\n-spec make_jump(State::dht_node_state:state(), TargetId::?RT:key(),\n                Tag::any(), SourcePid::comm:mypid() | null)\n    -> dht_node_state:state().\nmake_jump(State, TargetId, Tag, SourcePid) ->\n    MoveFullId = uid:get_global_uid(),\n    MyNode = dht_node_state:get(State, node),\n    TargetNode = dht_node_state:get(State, succ),\n    log:log(info, \"[ Node ~.0p ] starting jump (succ: ~.0p, TargetId: ~.0p)~n\",\n            [MyNode, TargetNode, TargetId]),\n    setup_slide(State, {jump, 'send'},\n                MoveFullId, MyNode, TargetNode, TargetId, Tag,\n                unknown, SourcePid, nomsg, {none}).\n\n%% @doc Creates a slide that will move all data to the successor and leave the\n%%      ring. Note: Will re-try (forever) to successfully start a leaving slide\n%%      if anything causes an abort!\n-spec make_slide_leave(State::dht_node_state:state(), SourcePid::comm:mypid() | null)\n        -> dht_node_state:state().\nmake_slide_leave(State, SourcePid) ->\n    MoveFullId = uid:get_global_uid(),\n    InitNode = dht_node_state:get(State, node),\n    OtherNode = dht_node_state:get(State, succ),\n    PredNode = dht_node_state:get(State, pred),\n    log:log(info, \"[ Node ~.0p ] starting leave (succ: ~.0p)~n\", [InitNode, OtherNode]),\n    setup_slide(State, {leave, 'send'}, MoveFullId, InitNode,\n                OtherNode, node:id(PredNode), leave,\n                unknown, SourcePid, nomsg, {none}).\n\n-spec get_slide_mod() -> slide_chord | slide_leases.\nget_slide_mod() ->\n    case config:read(leases) of\n        true -> slide_leases;\n        failed -> slide_chord;\n        false -> slide_chord\n    end.\n\n%% @doc Checks whether config parameters regarding dht_node moves exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(move_max_transport_entries) and\n    config:cfg_is_greater_than(move_max_transport_entries, 0) and\n\n    config:cfg_is_integer(move_wait_for_reply_timeout) and\n    config:cfg_is_greater_than(move_wait_for_reply_timeout, 0) and\n\n    config:cfg_is_integer(move_send_msg_retries) and\n    config:cfg_is_greater_than(move_send_msg_retries, 0) and\n\n    config:cfg_is_integer(move_send_msg_retry_delay) and\n    config:cfg_is_greater_than_equal(move_send_msg_retry_delay, 0) and\n\n    config:cfg_is_bool(move_use_incremental_slides).\n    \n%% @doc Gets the max number of DB entries per data move operation (set in the\n%%      config files).\n-spec get_max_transport_entries() -> pos_integer().\nget_max_transport_entries() ->\n    config:read(move_max_transport_entries).\n\n%% @doc Gets the max number of ms to wait for the other node's reply until\n%%      logging a warning (set in the config files).\n-spec get_wait_for_reply_timeout() -> pos_integer().\nget_wait_for_reply_timeout() ->\n    config:read(move_wait_for_reply_timeout).\n\n%% @doc Gets the max number of retries to send a message to the other node\n%%      until logging a warning (set in the config files).\n-spec get_send_msg_retries() -> pos_integer().\nget_send_msg_retries() ->\n    config:read(move_send_msg_retries).\n\n%% @doc Checks whether incremental slides are to be used\n%%      (set in the config files).\n-spec use_incremental_slides() -> boolean().\nuse_incremental_slides() ->\n    config:read(move_use_incremental_slides).\n"
  },
  {
    "path": "src/dht_node_reregister.erl",
    "content": "%  @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Re-register with mgmt_server nodes\n%% @end\n%% @version $Id$\n-module(dht_node_reregister).\n\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n-export([start_link/1]).\n-export([init/1, on_active/2, on_inactive/2,\n         activate/0, deactivate/0]).\n\n-include(\"gen_component.hrl\").\n\n-type(message() ::\n    {register_trigger} |\n    {register} |\n    {web_debug_info, Requestor::comm:erl_local_pid()}).\n\n-type state_active() :: ok.\n-type state_inactive() :: inactive.\n\n%% @doc Activates the re-register process. If not activated, it will\n%%      queue most messages without processing them.\n-spec activate() -> ok.\nactivate() ->\n    Pid = pid_groups:get_my(dht_node_reregister),\n    comm:send_local(Pid, {activate_reregister}).\n\n%% @doc Deactivates the re-register process.\n-spec deactivate() -> ok.\ndeactivate() ->\n    Pid = pid_groups:get_my(dht_node_reregister),\n    comm:send_local(Pid, {deactivate_reregister}).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Startup\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Starts a re-register process, registers it with the process\n%%      dictionary and returns its pid for use by a supervisor.\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on_inactive/2, [],\n                             [{pid_groups_join_as, DHTNodeGroup, ?MODULE}]).\n\n%% @doc Initialises the module with an uninitialized state.\n-spec init([]) -> state_inactive().\ninit([]) ->\n    msg_delay:send_trigger(get_base_interval(), {register_trigger}),\n    inactive.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Message Loop\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec on_inactive(message(), state_inactive()) -> state_inactive();\n                 ({activate_reregister}, state_inactive()) -> {'$gen_component', [{on_handler, Handler::gen_component:handler()}], State::state_active()}.\non_inactive({activate_reregister}, _State) ->\n    log:log(info, \"[ Reregister ~.0p ] activating...~n\", [comm:this()]),\n    comm:send_local(self(), {register}),\n    gen_component:change_handler(ok, fun ?MODULE:on_active/2);\n\non_inactive({web_debug_info, Requestor}, State) ->\n    KeyValueList = [{\"\", \"\"}, {\"inactive re-register process\", \"\"}],\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    State;\n\non_inactive({register_trigger}, State) ->\n    msg_delay:send_trigger(get_base_interval(), {register_trigger}),\n    State;\n\non_inactive(_Msg, State) ->\n    State.\n\n-spec on_active(message(), state_active()) -> state_active();\n         ({deactivate_reregister}, state_active()) -> {'$gen_component', [{on_handler, Handler::gen_component:handler()}], State::state_inactive()}.\non_active({deactivate_reregister}, _State)  ->\n    log:log(info, \"[ Reregister ~.0p ] deactivating...~n\", [comm:this()]),\n    gen_component:change_handler(inactive, fun ?MODULE:on_inactive/2);\n\non_active({register_trigger}, State) ->\n    msg_delay:send_trigger(get_base_interval(), {register_trigger}),\n    gen_component:post_op({register}, State);\n\non_active({register}, State) ->\n    comm:send_local(pid_groups:get_my(dht_node),\n                    {get_node_details, comm:this(), [node]}),\n    State;\n\non_active({get_node_details_response, NodeDetails}, State) ->\n    RegisterMessage = {register, node_details:get(NodeDetails, node)},\n    _ = case config:read(register_hosts) of\n            failed -> MgmtServer = mgmtServer(),\n                      case comm:is_valid(MgmtServer) of\n                          true -> comm:send(MgmtServer, RegisterMessage);\n                          _ -> ok\n                      end;\n            Hosts  -> [comm:send(Host, RegisterMessage) || Host <- Hosts]\n        end,\n    State;\n\non_active({web_debug_info, Requestor}, State) ->\n    KeyValueList =\n        case config:read(register_hosts) of\n            failed -> [{\"Hosts (mgmt_server):\", webhelpers:safe_html_string(\"~.0p\", [mgmtServer()])}];\n            Hosts  -> [{\"Hosts:\", \"\"} |\n                           [{\"\", webhelpers:safe_html_string(\"~.0p\", [Host])} || Host <- Hosts]]\n        end,\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    State.\n\n%% @doc Gets the interval to trigger re-registering the node set in\n%%      scalaris.cfg and converts it to seconds.\n-spec get_base_interval() -> Seconds::pos_integer().\nget_base_interval() ->\n    config:read(reregister_interval) div 1000.\n\n%% @doc pid of the mgmt server (may be invalid)\n-spec mgmtServer() -> comm:mypid() | any().\nmgmtServer() ->\n    config:read(mgmt_server).\n"
  },
  {
    "path": "src/dht_node_state.erl",
    "content": "% @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc State of a dht_node.\n%% @version $Id$\n-module(dht_node_state).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-define(TRACE_MR_SLIDE(X,Y), ?TRACE(X, Y)).\n\n-export([new/3, new_on_recover/6,\n         delete_for_rejoin/1,\n         get/2,\n         dump/1,\n         set_rt/2, set_rm/2, set_db/2, set_lease_list/2,\n         details/1, details/2]).\n%% node responsibility:\n-export([has_left/1, is_db_responsible/2, is_db_responsible__no_msg_fwd_check/2]).\n%% transactions:\n-export([set_tx_tp_db/2]).\n%% node moves:\n-export([get_slide/2, set_slide/3,\n         slide_get_data_start_record/2, slide_add_data/2,\n         slide_take_delta_stop_record/2, slide_add_delta/2,\n         slide_stop_record/3,\n         get_split_key/5,\n         add_db_range/3, rm_db_range/2]).\n%% prbr DBs and states:\n-export([set_prbr_state/3]).\n%% snapshots\n-export([set_snapshot_state/2]).\n%% map reduce\n-export([set_mr_state/3,\n         get_mr_state/2,\n         delete_mr_state/2,\n         set_mr_master_state/3,\n         get_mr_master_state/2,\n         delete_mr_master_state/2]).\n\n\n-export_type([state/0, name/0, db_selector/0, slide_data/0, slide_delta/0]).\n\n-type db_selector() :: kv_db | crdt_db | {tx_id, pos_integer()} | {lease_db, pos_integer()}.\n\n-type name() :: rt | rt_size | neighbors | succlist | succ | succ_id\n              | succ_pid | predlist | pred | pred_id | pred_pid | node\n              | node_id | my_range | db_range | succ_range | join_time\n              | db | tx_tp_db | load | slide_pred | slide_succ\n              | msg_fwd | rm_state | prbr_state.\n\n-type slide_snap() :: {snapshot_state:snapshot_state(), db_dht:db_as_list()} | {false}.\n\n-type slide_data() :: {{MovingData::db_dht:db_as_list(), slide_snap()},\n                       [{db_selector(), db_prbr:db_as_list()}]}.\n-type slide_delta() :: {{ChangedData::db_dht:db_as_list(), DeletedKeys::[?RT:key()]},\n                        [{db_selector(), {Changed::db_prbr:db_as_list(),\n                                          Deleted::[?RT:key()]}}],\n                        {MRDelta::orddict:orddict(),\n                         MRMasterDelta::orddict:orddict()}}.\n\n%% userdevguide-begin dht_node_state:state\n-record(state, {% external_rt stored here for bulkowner\n                rt         = ?required(state, rt)        :: ?RT:external_rt(),\n                rm_state   = ?required(state, rm_state)  :: rm_loop:state(),\n                join_time  = ?required(state, join_time) :: erlang_timestamp(),\n                db         = ?required(state, db)        :: db_dht:db(),\n                tx_tp_db   = ?required(state, tx_tp_db)  :: any(),\n                % slide with pred (must not overlap with 'slide with succ'!):\n                slide_pred              = null :: slide_op:slide_op() | null,\n                % slide with succ (must not overlap with 'slide with pred'!):\n                slide_succ              = null :: slide_op:slide_op() | null,\n                % additional range to respond to during a move:\n                db_range   = []   :: [{intervals:interval(), slide_op:id()}],\n                kv_db = ?required(state, kv_db) :: prbr:state(),\n                crdt_db = ?required(state, crdt_db) :: crdt_acceptor:state(),\n                txid_dbs = ?required(state, txid_dbs) :: tuple(),\n                lease_dbs = ?required(state, lease_dbs) :: tuple(),\n                lease_list = ?required(state, lease_list) :: lease_list:lease_list(),\n                snapshot_state   = null :: snapshot_state:snapshot_state() | null,\n                mr_state   = ?required(state, mr_state)  :: orddict:orddict(),\n                mr_master_state   = ?required(state, mr_master_state)  :: orddict:orddict()\n               }).\n-opaque state() :: #state{}.\n%% userdevguide-end dht_node_state:state\n\n-dialyzer({no_opaque, get/2}).\n\n-spec new(?RT:external_rt(), RMState::rm_loop:state(), db_dht:db()) -> state().\nnew(RT, RMState, DB) ->\n    TxidDBs  = [{Id, prbr:init({tx_id, Id})} || Id <- lists:seq(1, config:read(replication_factor))],\n    LeaseDBs = [{Id, prbr:init({lease_db, Id})} || Id <- lists:seq(1, config:read(replication_factor))],\n    CrdtMod = crdt:acceptor_module(),\n    #state{rt = RT,\n           rm_state = RMState,\n           join_time = os:timestamp(),\n           db = DB,\n           tx_tp_db = tx_tp:init(),\n           kv_db = prbr:init(kv_db),\n           crdt_db = CrdtMod:init(crdt_db),\n           txid_dbs = erlang:make_tuple(config:read(replication_factor), ok, TxidDBs),\n           lease_dbs = erlang:make_tuple(config:read(replication_factor), ok, LeaseDBs),\n           lease_list = lease_list:empty(),\n           snapshot_state = snapshot_state:new(),\n           mr_state = orddict:new(),\n           mr_master_state = orddict:new()\n          }.\n\n-spec new_on_recover(?RT:external_rt(), RMState::rm_loop:state(),\n                     PRBR_KV_DB::prbr:state(),\n                     TXID_DBs::list(prbr:state()),\n                     Lease_DBs::list(prbr:state()),\n                     LeaseList::lease_list:lease_list()) -> state().\nnew_on_recover(RT, RMState,\n               KV_DB,\n               TXID_DBs,\n               Lease_DBs,\n               LeaseList) ->\n    IndexedTXIDs  = lists:zip(lists:seq(1, config:read(replication_factor)), TXID_DBs),\n    IndexedLeases = lists:zip(lists:seq(1, config:read(replication_factor)), Lease_DBs),\n    CrdtMod = crdt:acceptor_module(),\n    #state{rt = RT,\n           rm_state = RMState,\n           join_time = os:timestamp(),\n           db = db_dht:new(db_dht),\n           tx_tp_db = tx_tp:init(),\n           kv_db = KV_DB,\n           crdt_db = CrdtMod:init(crdt_db), %% TODO??\n           txid_dbs  = erlang:make_tuple(config:read(replication_factor), ok, IndexedTXIDs),\n           lease_dbs = erlang:make_tuple(config:read(replication_factor), ok, IndexedLeases),\n           lease_list = LeaseList,\n           snapshot_state = snapshot_state:new(),\n           mr_state = orddict:new(),\n           mr_master_state = orddict:new()\n          }.\n\n%% @doc Clean up tables before rejoining with a new state.\n-spec delete_for_rejoin(state()) -> ok.\ndelete_for_rejoin(\n  #state{db = DB, kv_db=PRBRState,\n         txid_dbs=TXID_DBs, lease_dbs=Lease_DBs}) ->\n    % note: rm_state is transferred (ref. move_state in rm_loop)\n    % TODO: transfer snapshot state / data?!\n    % TODO: transfer MR state / data?!\n    db_dht:close_and_delete(DB),\n    prbr:close_and_delete(PRBRState),\n    [prbr:close_and_delete(element(I,TXID_DBs )) || I <- lists:seq(1,tuple_size(TXID_DBs))],\n    [prbr:close_and_delete(element(I,Lease_DBs)) || I <- lists:seq(1,tuple_size(Lease_DBs))],\n    ok.\n\n%% @doc Gets the given property from the dht_node state.\n%%      Allowed keys include:\n%%      <ul>\n%%        <li>rt = routing table,</li>\n%%        <li>rt_size = size of the routing table (provided for convenience),</li>\n%%        <li>succlist = successor list,</li>\n%%        <li>succ = successor (provided for convenience),</li>\n%%        <li>succ_id = ID of the successor (provided for convenience),</li>\n%%        <li>succ_pid = PID of the successor (provided for convenience),</li>\n%%        <li>predlist = predecessor list,</li>\n%%        <li>pred = predecessor (provided for convenience),</li>\n%%        <li>pred_id = ID of the predecessor (provided for convenience),</li>\n%%        <li>pred_pid = PID of the predecessor (provided for convenience),</li>\n%%        <li>node = the own node,</li>\n%%        <li>node_id = the ID of the own node (provided for convenience),</li>\n%%        <li>my_range = the range of the own node,</li>\n%%        <li>succ_range = the range of the successor,</li>\n%%        <li>db_range = temporarily added range (during slides),</li>\n%%        <li>msg_fwd = temporarily removed range and their current destination (during slides),</li>\n%%        <li>full_range = the real responsibility range of the own node (see is_db_responsible/2),</li>\n%%        <li>join_time = the time the node was created, i.e. joined the system,</li>\n%%        <li>db = DB storing the items,</li>\n%%        <li>tx_tp_db = transaction participant DB,</li>\n%%        <li>load = the load (items) of the own node (provided for convenience).</li>\n%%        <li>load2 = the load (see in config lb_active_load_metric) of the own node (provided for convenience).</li>\n%%        <li>load3 = the load (see in config lb_active_request_metric) of the own node (provided for convenience).</li>\n%%        <li>slide_pred = information about the node's current slide operation with its predecessor.</li>\n%%        <li>slide_succ = information about the node's current slide operation with its successor.</li>\n%%        <li>snapshot_state = snapshot algorithm state information</li>\n%%        <li>lease_list = the list of all leases</li>\n%%        <li>mr_state = a dictionary containing all map reduce states currently running on this node</li>\n%%      </ul>\n%%      Beware of race conditions since the neighborhood may have changed at\n%%      the next call.\n-spec get(state(), rt) -> ?RT:external_rt();\n         (state(), rt_size) -> non_neg_integer();\n         (state(), neighbors) -> nodelist:neighborhood();\n         (state(), succlist) -> nodelist:non_empty_snodelist();\n         (state(), succ) -> node:node_type();\n         (state(), succ_id) -> ?RT:key();\n         (state(), succ_pid) -> comm:mypid();\n         (state(), predlist) -> nodelist:non_empty_snodelist();\n         (state(), pred) -> node:node_type();\n         (state(), pred_id) -> ?RT:key();\n         (state(), pred_pid) -> comm:mypid();\n         (state(), node) -> node:node_type();\n         (state(), node_id) -> ?RT:key();\n         (state(), my_range) -> intervals:interval();\n         (state(), db_range) -> [{intervals:interval(), slide_op:id()}];\n         (state(), succ_range) -> intervals:interval();\n         (state(), full_range) -> intervals:interval();\n         (state(), join_time) -> erlang_timestamp();\n         (state(), db) -> db_dht:db();\n         (state(), tx_tp_db) -> any();\n         (state(), load) -> non_neg_integer();\n         (state(), load2) -> unknown | lb_stats:load();\n         (state(), load3) -> lb_stats:load();\n         (state(), slide_pred) -> slide_op:slide_op() | null;\n         (state(), slide_succ) -> slide_op:slide_op() | null;\n         (state(), snapshot_state) -> snapshot_state:snapshot_state() | null;\n         (state(), msg_fwd) -> [{intervals:interval(), comm:mypid()}];\n         (state(), rm_state) -> rm_loop:state();\n         (state(), kv_db) -> prbr:state();\n         (state(), crdt_db) -> crdt_acceptor:state();\n         (state(), {tx_id, pos_integer()}) -> prbr:state();\n         (state(), {lease_db, pos_integer()}) -> prbr:state();\n         (state(), lease_list) -> lease_list:lease_list().\nget(#state{rt=RT, rm_state=RMState, join_time=JoinTime,\n           db=DB, tx_tp_db=TxTpDb,\n           slide_pred=SlidePred, slide_succ=SlideSucc,\n           db_range=DBRange, kv_db=PRBRState, crdt_db=CRDTState,\n           lease_list=LeaseList, txid_dbs = TXID_DBs, lease_dbs = LeaseDBs,\n           snapshot_state=SnapState} = State, Key) ->\n    case Key of\n        rt           -> RT;\n        rt_size      -> ?RT:get_size_ext(RT);\n        neighbors    -> rm_loop:get_neighbors(RMState);\n        my_range     -> Neighbors = rm_loop:get_neighbors(RMState),\n                        nodelist:node_range(Neighbors);\n        db_range     -> DBRange;\n        full_range   -> Range1 = lists:foldl(fun({I, _SlideId}, AccI) ->\n                                                     intervals:union(AccI, I)\n                                             end, get(State, my_range),\n                                             DBRange),\n                        MsgFwd = get(State, msg_fwd),\n                        lists:foldl(fun({FwdInt, _FwdPid}, AccI) ->\n                                            intervals:minus(AccI, FwdInt)\n                                    end, Range1, MsgFwd);\n        succ_range   -> Neighbors = rm_loop:get_neighbors(RMState),\n                        nodelist:succ_range(Neighbors);\n        msg_fwd      -> MsgFwdPred = slide_op:get_msg_fwd(SlidePred),\n                        MsgFwdSucc = slide_op:get_msg_fwd(SlideSucc),\n                        if MsgFwdPred =:= [] -> MsgFwdSucc;\n                           MsgFwdSucc =:= [] -> MsgFwdPred;\n                           true -> lists:append(MsgFwdPred, MsgFwdSucc)\n                        end;\n        db           -> DB;\n        tx_tp_db     -> TxTpDb;\n        slide_pred   -> SlidePred;\n        slide_succ   -> SlideSucc;\n        rm_state     -> RMState;\n        snapshot_state -> SnapState;\n        succlist     -> nodelist:succs(rm_loop:get_neighbors(RMState));\n        succ         -> nodelist:succ(rm_loop:get_neighbors(RMState));\n        succ_id      -> node:id(nodelist:succ(rm_loop:get_neighbors(RMState)));\n        succ_pid     -> node:pidX(nodelist:succ(rm_loop:get_neighbors(RMState)));\n        predlist     -> nodelist:preds(rm_loop:get_neighbors(RMState));\n        pred         -> nodelist:pred(rm_loop:get_neighbors(RMState));\n        pred_id      -> node:id(nodelist:pred(rm_loop:get_neighbors(RMState)));\n        pred_pid     -> node:pidX(nodelist:pred(rm_loop:get_neighbors(RMState)));\n        node         -> nodelist:node(rm_loop:get_neighbors(RMState));\n        node_id      -> nodelist:nodeid(rm_loop:get_neighbors(RMState));\n        join_time    -> JoinTime;\n        load         -> db_dht:get_load(DB)\n                        %% and the prbr kv entries:\n                            + prbr:get_load(PRBRState);\n        load2        -> lb_stats:get_load_metric();\n        load3        -> lb_stats:get_request_metric();\n        kv_db        -> PRBRState;\n        crdt_db      -> CRDTState;\n        {tx_id, I}   -> element(I, TXID_DBs);\n        {lease_db, I}-> element(I, LeaseDBs);\n        lease_list   -> LeaseList\n    end.\n\n-spec set_prbr_state(state(), db_selector(), prbr:state()) -> state().\nset_prbr_state(State = #state{txid_dbs=TXID_DBs, lease_dbs = LeaseDBs},\n               WhichDB, Value) ->\n    case WhichDB of\n        kv_db -> State#state{kv_db = Value};\n        crdt_db -> State#state{crdt_db = Value};\n        {tx_id, I} -> State#state{txid_dbs = setelement(I, TXID_DBs, Value)};\n        {lease_db, I} -> State#state{lease_dbs = setelement(I, LeaseDBs, Value)}\n    end.\n\n-spec set_lease_list(state(), lease_list:lease_list()) -> state().\nset_lease_list(State, LeaseList) ->\n    State#state{lease_list = LeaseList}.\n\n%% @doc Checks whether the current node has already left the ring, i.e. the has\n%%      already changed his ID in order to leave or jump.\n-spec has_left(State::state()) -> boolean().\nhas_left(#state{rm_state=RMState}) ->\n    rm_loop:has_left(RMState).\n\n%% @doc Checks whether the node is responsible for the given key either by its\n%%      current range or for a range the node is temporarily responsible for\n%%      during a slide operation, i.e. we temporarily read/modify data a\n%%      neighbor is responsible for but hasn't yet received the data from us.\n-spec is_db_responsible(Key::intervals:key(), State::state()) -> boolean().\nis_db_responsible(Key, State) ->\n    is_db_responsible__no_msg_fwd_check(Key, State) andalso\n        lists:all(fun({Interval, _Pid}) ->\n                          not intervals:in(Key, Interval)\n                  end, get(State, msg_fwd)).\n\n%% @doc Checks whether the node is responsible for the given key either by its\n%%      current range or for a range the node is temporarily responsible for\n%%      during a slide operation, i.e. we temporarily read/modify data a\n%%      neighbor is responsible for but hasn't yet received the data from us.\n-spec is_db_responsible__no_msg_fwd_check(Key::intervals:key(), State::state()) -> boolean().\nis_db_responsible__no_msg_fwd_check(Key, #state{db_range = DBRange, rm_state = RMState}) ->\n    rm_loop:is_responsible(Key, RMState) orelse\n        lists:any(fun({Interval, _Id}) ->\n                          intervals:in(Key, Interval)\n                  end, DBRange).\n\n%% @doc Tries to find a slide operation with the given MoveFullId and returns\n%%      it including its type (pred or succ) if successful and its pred/succ\n%%      info is correct. Otherwise returns {fail, wrong_pred} if the\n%%      predecessor info is wrong (slide with pred) and {fail, wrong_succ} if\n%%      the successor info is wrong (slide with succ). If not found,\n%%      {fail, not_found} is returned.\n-spec get_slide(State::state(), MoveFullId::slide_op:id()) ->\n        {Type::pred | succ, SlideOp::slide_op:slide_op()} |\n        not_found.\nget_slide(#state{slide_pred=SlidePred, slide_succ=SlideSucc}, MoveFullId) ->\n    case slide_op:is_slide(SlidePred) andalso\n             slide_op:get_id(SlidePred) =:= MoveFullId of\n        true -> {pred, SlidePred};\n        _ ->\n            case slide_op:is_slide(SlideSucc) andalso\n                     slide_op:get_id(SlideSucc) =:= MoveFullId of\n                true -> {succ, SlideSucc};\n                _ -> not_found\n            end\n    end.\n\n-spec set_tx_tp_db(State::state(), NewTxTpDb::any()) -> state().\nset_tx_tp_db(State, DB) -> State#state{tx_tp_db = DB}.\n\n-spec set_db(State::state(), NewDB::db_dht:db()) -> state().\nset_db(State, DB) -> State#state{db = DB}.\n\n-spec set_rt(State::state(), NewRT::?RT:external_rt()) -> state().\nset_rt(State, RT) -> State#state{rt = RT}.\n\n-spec set_rm(State::state(), NewRMState::rm_loop:state()) -> state().\nset_rm(State, RMState) -> State#state{rm_state = RMState}.\n\n-spec set_slide(state(), pred | succ, slide_op:slide_op() | null) -> state().\nset_slide(State, pred, SlidePred) -> State#state{slide_pred=SlidePred};\nset_slide(State, succ, SlideSucc) -> State#state{slide_succ=SlideSucc}.\n\n-spec set_snapshot_state(State::state(),NewInfo::snapshot_state:snapshot_state()) -> state().\nset_snapshot_state(State,NewInfo) -> State#state{snapshot_state=NewInfo}.\n\n-spec get_mr_state(State::state(), mr_state:jobid()) ->\n    {ok, mr_state:state()} | error.\nget_mr_state(#state{mr_state = MRStates}, JobId) ->\n    orddict:find(JobId, MRStates).\n\n-spec set_mr_state(State::state(), nonempty_string(), mr_state:state()) -> state().\nset_mr_state(#state{mr_state = MRStates} = State, JobId, MRState) ->\n    State#state{mr_state = orddict:store(JobId, MRState, MRStates)}.\n\n-spec delete_mr_state(State::state(), nonempty_string()) -> state().\ndelete_mr_state(#state{mr_state = MRStateList} = State, JobId) ->\n    State#state{mr_state = orddict:erase(JobId, MRStateList)}.\n\n-spec get_mr_master_state(State::state(), mr_state:jobid()) -> mr_master_state:state().\nget_mr_master_state(#state{mr_master_state = MRMStates}, JobId) ->\n    orddict:fetch(JobId, MRMStates).\n\n-spec set_mr_master_state(State::state(), nonempty_string(), mr_master_state:state()) -> state().\nset_mr_master_state(#state{mr_master_state = MRMStates} = State, JobId, MRMState) ->\n    State#state{mr_master_state = orddict:store(JobId, MRMState, MRMStates)}.\n\n-spec delete_mr_master_state(State::state(), nonempty_string()) -> state().\ndelete_mr_master_state(#state{mr_master_state = MRMStateList} = State, JobId) ->\n    State#state{mr_master_state = orddict:erase(JobId, MRMStateList)}.\n\n-spec add_db_range(State::state(), Interval::intervals:interval(),\n                   SlideId::slide_op:id()) -> state().\nadd_db_range(State = #state{db_range=DBRange}, Interval, SlideId) ->\n    ?DBG_ASSERT(not intervals:is_all(Interval)),\n    ?DBG_ASSERT2(intervals:is_subset(Interval, MyRange = get(State, my_range)) orelse\n                     intervals:is_adjacent(Interval, MyRange),\n                 {new_interval, Interval, not_subset_of, MyRange}),\n    ?TRACE(\"[ ~.0p ] add_db_range: ~.0p~n\", [self(), Interval]),\n    State#state{db_range = [{Interval, SlideId} | DBRange]}.\n\n-spec rm_db_range(State::state(), SlideId::slide_op:id()) -> state().\nrm_db_range(State = #state{db_range=DBRange}, SlideId) ->\n    ?TRACE(\"[ ~.0p ] rm_db_range: ~.0p~n\", [self(), [I || {I, Id} <- DBRange, Id =:= SlideId]]),\n    State#state{db_range = [X || X = {_, Id} <- DBRange, Id =/= SlideId]}.\n\n%%% util\n-spec dump(state()) -> ok.\ndump(State) ->\n    io:format(\"dump <~s,~w> <~s,~w> <~s,~w>~n\",\n              [get(State, node_id), self(),\n               get(State, pred_id), get(State, pred_pid),\n               get(State, succ_id), get(State, succ_pid)]),\n    ok.\n\n%% @doc Gets the requested details about the current node.\n-spec details(state(), [node_details:node_details_name()]) -> node_details:node_details().\ndetails(State, Which) ->\n    ExtractValues =\n        fun(Elem, NodeDetails) ->\n                case Elem of\n                    hostname    -> node_details:set(NodeDetails, hostname, net_adm:localhost());\n                    message_log -> node_details:set(NodeDetails, message_log, ok);\n                    memory      -> node_details:set(NodeDetails, memory, erlang:memory(total));\n                    is_leaving  -> node_details:set(NodeDetails, Elem, rm_loop:has_left(get(State, rm_state)));\n                    Tag         -> node_details:set(NodeDetails, Tag, get(State, Tag))\n                end\n        end,\n    lists:foldl(ExtractValues, node_details:new(), Which).\n\n%% @doc Gets the following details about the current node:\n%%      predecessor and successor lists, the node itself, its load, hostname,\n%%      routing table size, memory usage.\n-spec details(state()) -> node_details:node_details().\ndetails(State) ->\n    Neighbors = get(State, neighbors),\n    PredList = nodelist:preds(Neighbors),\n    SuccList = nodelist:succs(Neighbors),\n    Node = nodelist:node(Neighbors),\n    Load = get(State, load),\n    Load2 = get(State, load2),\n    Load3 = get(State, load3),\n    Hostname = net_adm:localhost(),\n    RTSize = get(State, rt_size),\n    node_details:new(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, erlang:memory(total)).\n\n-spec get_prbr_selectors() -> list(db_selector()).\nget_prbr_selectors() ->\n    [kv_db | lists:flatmap(fun(I) -> [{tx_id, I}, {lease_db, I}] end,\n                                lists:seq(1, config:read(replication_factor)))].\n\n%% @doc Gets all entries to transfer (slide) in the given range and starts delta\n%%      recording on the DB for changes in this interval.\n-spec slide_get_data_start_record(state(), MovingInterval::intervals:interval())\n        -> {state(), slide_data()}.\nslide_get_data_start_record(State, MovingInterval) ->\n    %% all prbr dbs:\n    {T1State, MoveRBRData} =\n        lists:foldl(\n          fun(X, {AccState, AccData}) ->\n                  Old = get(State, X),\n                  MoveData = db_prbr:get_entries(Old, MovingInterval),\n                  New = db_prbr:record_changes(Old, MovingInterval),\n                  {set_prbr_state(AccState, X, New), [{X, MoveData} | AccData]}\n          end,\n          {State, []},\n          get_prbr_selectors()\n         ),\n\n    %% snapshot state and db\n    OldDB = get(T1State, db),\n    MovingSnapData =\n        case db_dht:snapshot_is_running(OldDB) of\n            true ->\n                {get(T1State, snapshot_state),\n                 db_dht:get_snapshot_data(OldDB, MovingInterval)};\n            false ->\n                {false}\n        end,\n\n    %% dht db\n    MovingData = db_dht:get_entries(OldDB, MovingInterval),\n    NewDB = db_dht:record_changes(OldDB, MovingInterval),\n    ?TRACE(\"~p:slide_get_data_start_record: ~p~nMovingData: ~n~p~nMovingSnapData: ~n~p~nfor\n           interval ~p~n~p~n~p\",\n           [?MODULE, comm:this(), MovingData, MovingSnapData,\n            MovingInterval, OldDB, NewDB]),\n    NewState = set_db(T1State, NewDB),\n    {NewState, {{MovingData, MovingSnapData}, MoveRBRData}}.\n\n\n%% @doc Adds data from slide_get_data_start_record/2 to the local DB.\n-spec slide_add_data(state(),slide_data()) -> state().\nslide_add_data(State, {{Data, SnapData}, PRBRData}) ->\n    T1DB = db_dht:add_data(get(State, db), Data),\n    ?TRACE(\"~p:slide_add_data: ~p~nMovingData:~n~p~nMovingSnapData:~n~p~nPRBR:~n~p\",\n           [?MODULE, comm:this(), Data, SnapData, PRBRData]),\n    T2State =\n        case SnapData of\n            {false} ->\n                set_db(State, T1DB);\n            {SnapState, SnapEntries} ->\n                T2DB = db_dht:init_snapshot(T1DB),\n                T3DB = db_dht:add_snapshot_data(T2DB, SnapEntries),\n                T1State = set_db(State, T3DB),\n                set_snapshot_state(T1State, SnapState)\n        end,\n\n    %% all prbr dbs\n    lists:foldl(\n      fun({X, XData}, AccState) ->\n              DB = get(AccState, X),\n              NewDB = db_prbr:add_data(DB, XData),\n              set_prbr_state(AccState, X, NewDB)\n      end,\n      T2State,\n      PRBRData).\n\n%% @doc Gets all DB changes in the given interval, stops recording delta infos\n%%      and removes the entries in this range from the DB.\n-spec slide_take_delta_stop_record(state(), MovingInterval::intervals:interval())\n        -> {state(), slide_delta()}.\nslide_take_delta_stop_record(State, MovingInterval) ->\n    %% all prbr dbs:\n    DeltaRBR =\n        lists:foldl(\n          fun(X, AccData) ->\n                  DB = get(State, X),\n                  Delta = db_prbr:get_changes(DB, MovingInterval),\n                  [{X, Delta} | AccData]\n          end,\n          [],\n          get_prbr_selectors()\n         ),\n\n    %% db\n    OldDB = get(State, db),\n    ChangedData = db_dht:get_changes(OldDB, MovingInterval),\n\n    {T1State, MRDelta} = mr_get_delta_states(State, MovingInterval),\n\n    NewState = slide_stop_record(T1State, MovingInterval, true),\n    ?TRACE(\"~p:slide_take_delta_stop_record: ~p~nChangedData: ~n~p~n~p\",\n           [?MODULE, comm:this(), ChangedData, get(NewState, db)]),\n    {NewState, {ChangedData, DeltaRBR, MRDelta}}.\n\n%% @doc Adds delta infos from slide_take_delta_stop_record/2 to the local DB.\n-spec slide_add_delta(state(), slide_delta()) -> state().\nslide_add_delta(State, {{ChangedData, DeletedKeys}, PRBRDelta, MRDelta}) ->\n    NewDB1 = db_dht:add_data(get(State, db), ChangedData),\n    NewDB2 = db_dht:delete_entries(NewDB1, intervals:from_elements(DeletedKeys)),\n    ?TRACE(\"~p:slide_add_delta: ~p~nChangedData: ~n~p~n~p\",\n           [?MODULE, comm:this(), {ChangedData, DeletedKeys}, NewDB2]),\n    T1State = set_db(State, NewDB2),\n\n    T2State = mr_add_delta(T1State, MRDelta),\n\n    %% all prbr dbs\n    lists:foldl(\n      fun({X, {XData, DelKeys}}, AccState) ->\n              DB = get(AccState, X),\n              TDB = db_prbr:add_data(DB, XData),\n              NewDB = db_prbr:delete_entries(\n                        TDB,\n                        intervals:from_elements(DelKeys)),\n              set_prbr_state(AccState, X, NewDB)\n      end,\n      T2State,\n      PRBRDelta).\n\n%% @doc Stops recording changes in the given interval.\n%%      Optionally, the data in this range can be deleted.\n-spec slide_stop_record(state(), MovingInterval::intervals:interval(),\n                        RemoveDataInInterval::boolean()) -> state().\nslide_stop_record(State, MovingInterval, Remove) ->\n    %% all prbr dbs:\n    T1State =\n        lists:foldl(\n          fun(X, AccState) ->\n                  DB = get(AccState, X),\n                  TDB = db_prbr:stop_record_changes(DB, MovingInterval),\n                  XDB =\n                      if Remove -> db_prbr:delete_entries(TDB, MovingInterval);\n                         true   -> TDB\n                      end,\n                  set_prbr_state(AccState, X, XDB)\n          end,\n          State,\n          get_prbr_selectors()\n         ),\n\n    NewDB1 = db_dht:stop_record_changes(get(T1State, db), MovingInterval),\n    NewDB = if Remove -> db_dht:delete_entries(NewDB1, MovingInterval);\n               true   -> NewDB1\n            end,\n    set_db(T1State, NewDB).\n\n%% @doc Returns a key so that there are no more than TargetLoad entries\n%%      between Begin and this key in the DBs.\n-spec get_split_key(state(), Begin::?RT:key(), End::?RT:key(),\n                    TargetLoad::pos_integer(), Direction::forward | backward)\n        -> {?RT:key(), TakenLoad::pos_integer()}.\nget_split_key(State, Begin, End, TargetLoad, Direction) ->\n    db_dht:get_split_key(get(State, db), Begin, End, TargetLoad, Direction).\n\n-spec mr_get_delta_states(state(), intervals:interval()) -> {state(),\n                                                             {orddict:orddict(),\n                                                              orddict:orddict()}}.\nmr_get_delta_states(State = #state{mr_state = MRStates,\n                                   mr_master_state = MasterStates},\n                    Interval) ->\n    {NewMRStates, MRDelta} = orddict:fold(\n     fun(K, MRState, {StateAcc, DeltaAcc}) ->\n             {NewState, Delta} = mr_state:get_slide_delta(MRState, Interval),\n             {orddict:store(K, NewState, StateAcc),\n              orddict:store(K, Delta, DeltaAcc)}\n     end,\n     {orddict:new(), orddict:new()},\n     MRStates),\n    ?TRACE_MR_SLIDE(\"~p fold over master states ~p~n\", [self(), MasterStates]),\n    {RemainingMasterState, MovingMasterState} =\n    orddict:fold(\n     fun(K, MasterState, {StateAcc, DeltaAcc}) ->\n             case mr_master_state:get_slide_delta(MasterState, Interval) of\n                 {true, Delta} ->\n                     %% slide\n                     {StateAcc,\n                      orddict:store(K, Delta, DeltaAcc)};\n                 {false, NewState} ->\n                     %% no slide\n                     {orddict:store(K, NewState, StateAcc),\n                      DeltaAcc}\n             end\n     end,\n     {orddict:new(), orddict:new()},\n     MasterStates),\n    ?TRACE_MR_SLIDE(\"~p delta mrstates are ~p~ndelta master states are~p~n\",\n                    [self(), {NewMRStates, MRDelta},\n                     {RemainingMasterState,\n                      MovingMasterState}]),\n    {State#state{mr_state = NewMRStates,\n                 mr_master_state = RemainingMasterState},\n     {MRDelta, MovingMasterState}}.\n\n-spec mr_add_delta(state(), {orddict:orddict(), orddict:orddict()}) -> state().\nmr_add_delta(State = #state{mr_state = MRStates,\n                            mr_master_state = MasterStates},\n             {MRDeltaStates, MasterDelta}) ->\n    ?TRACE_MR_SLIDE(\"~p adding delta state: ~p~n~p~n\", [self(), MRDeltaStates,\n                                              MasterDelta]),\n    NewMRState = orddict:fold(\n     fun(K, MRState, Acc) ->\n             New = case orddict:find(K, Acc) of\n                 {ok, ExState} ->\n                     mr_state:merge_states(ExState, MRState);\n                 error ->\n                     mr_state:init_slide_state(MRState)\n             end,\n             orddict:store(K, New, Acc)\n     end,\n     MRStates,\n     MRDeltaStates),\n    NewMasterStates =\n    orddict:fold(\n     fun(K, NewState, Acc) ->\n             case mr_master_state:get(outstanding, NewState) of\n                 snapshot ->\n                     ?TRACE_MR_SLIDE(\n                        \"master state moved while snapshoting...dispatching again\",\n                        []),\n                     mr_master:dispatch_snapshot(K);\n                 false ->\n                     %% nothing to do\n                     ok\n             end,\n             orddict:store(K, NewState, Acc)\n     end,\n     MasterStates,\n     MasterDelta),\n    ?TRACE_MR_SLIDE(\"~p delta states added~n~p~n\", [self(), NewMRState]),\n    State#state{mr_state = NewMRState,\n                mr_master_state = NewMasterStates}.\n"
  },
  {
    "path": "src/dn_cache.erl",
    "content": "%  @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Christian Hennig <hennig@zib.de>\n%% @doc    Dead node Cache\n%% @end\n%% @version $Id$\n-module(dn_cache).\n-author('hennig@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n-export([start_link/1]).\n-export([init/1, on/2]).\n\n-export([add_zombie_candidate/1, subscribe/0, unsubscribe/0]).\n\n-include(\"gen_component.hrl\").\n\n-type(message() ::\n    {trigger} |\n    {trigger_reply, {pong, PidName::pid_groups:pidname() | undefined}, node:node_type()} |\n    {add_zombie_candidate, node:node_type()} |\n    {subscribe, comm:erl_local_pid()} |\n    {unsubscribe, comm:erl_local_pid()} |\n    {send_error, Target::comm:mypid(), {ping, ThisWithCookie::comm:mypid()}, Reason::atom()} |\n    {web_debug_info, Requestor::comm:erl_local_pid()}).\n\n-type(state() :: {fix_queue:fix_queue(node:node_type()), Subscribers::gb_sets:set(comm:erl_local_pid())}).\n\n-define(SEND_OPTIONS, [{channel, prio}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Public Interface\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Adds a dht_node PID to the dead node cache for further alive-checks.\n-spec add_zombie_candidate(node:node_type()) -> ok.\nadd_zombie_candidate(Node) ->\n    comm:send_local(get_pid(), {add_zombie_candidate, Node}).\n\n-spec subscribe() -> ok.\nsubscribe() ->\n    comm:send_local(get_pid(), {subscribe, self()}).\n\n-spec unsubscribe() -> ok.\nunsubscribe() ->\n    comm:send_local(get_pid(), {unsubscribe, self()}).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Startup\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Starts a Dead Node Cache process, registers it with the process\n%%      dictionary and returns its pid for use by a supervisor.\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                             [{pid_groups_join_as, DHTNodeGroup, dn_cache}]).\n\n%% @doc Initialises the module with an empty state.\n-spec init([]) -> state().\ninit([]) ->\n    msg_delay:send_trigger(0, {trigger}),\n    {fix_queue:new(config:read(zombieDetectorSize)), gb_sets:new()}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Message Loop\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc the Token takes care, that there is only one timermessage for stabilize\n\n-spec on(message(), state()) -> state().\non({trigger}, {Queue, Subscribers}) ->\n    _ = fix_queue:map(\n          fun(X) ->\n                  Pid = case node:is_valid(X) of\n                            true -> node:pidX(X);\n                            _    -> X\n                        end,\n                  SPid = comm:reply_as(comm:this(), 2, {trigger_reply, '_', X}),\n                  comm:send(Pid, {ping, SPid},\n                            ?SEND_OPTIONS ++ [{shepherd, self()}])\n          end, Queue),\n    msg_delay:send_trigger(get_base_interval(), {trigger}),\n    {fix_queue:new(config:read(zombieDetectorSize)), Subscribers};\n\non({trigger_reply, {pong, dht_node}, Zombie},\n   {_Queue, Subscribers} = State) ->\n    log:log(warn,\"[ dn_cache ~p ] found zombie ~.0p\", [comm:this(), Zombie]),\n    report_zombie(Subscribers, Zombie),\n    State;\n\non({trigger_reply, {pong, PidName}, Zombie}, State) ->\n    log:log(warn,\"[ dn_cache ~p ] found zombie ~.0p but no dht_node process (reports as ~.0p)\",\n            [comm:this(), Zombie, PidName]),\n    State;\n\non({add_zombie_candidate, Node}, {Queue, Subscribers}) ->\n    {add_to_queue(Queue, Node), Subscribers};\n\non({subscribe, Node}, {Queue, Subscribers}) ->\n    {Queue, gb_sets:insert(Node, Subscribers)};\n\non({unsubscribe, Node}, {Queue, Subscribers}) ->\n    {Queue, gb_sets:del_element(Node, Subscribers)};\n\non({send_error, _Target, {ping, ThisWithCookie}, _Reason}, {Queue, Subscribers}) ->\n    {_This, {trigger_reply, {null}, Node}} = comm:unpack_cookie(ThisWithCookie, {null}),\n    {add_to_queue(Queue, Node), Subscribers};\n\non({web_debug_info, Requestor}, {Queue, Subscribers} = State) ->\n    KeyValueList =\n        lists:flatten(\n          [{\"max_length\", fix_queue:max_length(Queue)},\n           {\"queue length\", fix_queue:length(Queue)},\n           {\"queue (node):\", \"\"},\n           [{\"\", webhelpers:safe_html_string(\"~p\", [Node])} || Node <- queue:to_list(fix_queue:queue(Queue))],\n           {\"subscribers\", gb_sets:size(Subscribers)},\n           {\"subscribers (pid):\", \"\"},\n           [{\"\", pid_groups:pid_to_name(Pid)} || Pid <- gb_sets:to_list(Subscribers)]]),\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    State.\n\n-spec add_to_queue(Queue::fix_queue:fix_queue(node:node_type()), Node::node:node_type())\n        -> fix_queue:fix_queue(node:node_type()).\nadd_to_queue(Queue, Node) ->\n    case node:is_valid(Node) of\n        true -> fix_queue:add_unique_head(Node, Queue, fun node:same_process/2,\n                                          fun node:newer/2);\n        _    -> fix_queue:add_unique_head(Node, Queue, fun erlang:'=:='/2,\n                                          fun(_Old, New) -> New end)\n    end.\n\n-spec report_zombie(Subscribers::gb_sets:set(comm:erl_local_pid()), Zombie::node:node_type()) -> ok.\nreport_zombie(Subscribers, Zombie) ->\n    gb_sets:fold(fun(X, _) ->\n                         comm:send_local(X, {zombie, Zombie})\n                 end, ok, Subscribers),\n    ok.\n\n%% @doc Gets the pid of the dn_cache process in the same group as the calling\n%%      process.\n-spec get_pid() -> pid() | failed.\nget_pid() ->\n    pid_groups:get_my(dn_cache).\n\n%% @doc Gets the zombie detector interval set in scalaris.cfg.\n-spec get_base_interval() -> pos_integer().\nget_base_interval() ->\n    config:read(zombieDetectorInterval).\n"
  },
  {
    "path": "src/fd.erl",
    "content": "% @copyright 2007-2015 Zuse Institute Berlin\n\n%  Licensed under the Apache License, Version 2.0 (the \"License\");\n%  you may not use this file except in compliance with the License.\n%  You may obtain a copy of the License at\n%\n%      http://www.apache.org/licenses/LICENSE-2.0\n%\n%  Unless required by applicable law or agreed to in writing, software\n%  distributed under the License is distributed on an \"AS IS\" BASIS,\n%  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%  See the License for the specific language governing permissions and\n%  limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Failure detector based on Guerraoui.\n%% @end\n%% @version $Id$\n-module(fd).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n%% -define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(_X,_Y), ok).\n-behaviour(gen_component).\n-include(\"scalaris.hrl\").\n\n-export_type([reason/0, event/0]).\n\n-export([subscribe/2, subscribe_refcount/2]).\n-export([unsubscribe/2, unsubscribe_refcount/2]).\n-export([update_subscriptions/3]).\n-export([report/3]).\n%% gen_server & gen_component callbacks\n-export([start_link/1, init/1, on/2]).\n\n%% debug purposes\n-export([subscriptions/0]).\n\n-include(\"gen_component.hrl\").\n\n-type reason() :: 'DOWN' | noconnection | term().\n-type event() :: crash | jump | leave.\n-type state() :: [HBPid::pid()]. % a list of all hbs processes launched by this fd\n\n-define(SEND_OPTIONS, [{channel, prio}]).\n\n%% @doc Generates a failure detector for the calling process on the given pid.\n-spec subscribe(Subscriber::comm:erl_local_pid(), WatchedPids::[comm:mypid()]) -> ok.\nsubscribe(Subscriber, [_|_] = GlobalPids) ->\n    FD = my_fd_pid(),\n    _ = [subscribe_single(FD, Subscriber, Pid) || Pid <- GlobalPids],\n    ok;\nsubscribe(_Subscriber, []) ->\n    ok.\n\n-spec subscribe_single(FD::pid(), Subscriber::comm:erl_local_pid(),\n                       WatchedPid::comm:mypid()) -> ok.\nsubscribe_single(FD, Subscriber, GlobalPid) ->\n    comm:send_local(FD, {add_subscriber_via_fd, Subscriber, GlobalPid}).\n\n%% @doc Generates a failure detector for the calling process on the\n%%      given pid - uses reference counting to be subscribed to a pid only once.\n%%      Unsubscribe with unsubscribe_refcount/3!\n-spec subscribe_refcount(Subscriber::comm:erl_local_pid(), WatchedPids::[comm:mypid()]) -> ok.\nsubscribe_refcount(Subscriber, [_|_] = GlobalPids) ->\n    FD = my_fd_pid(),\n    _ = [subscribe_single_refcount(FD, Subscriber, Pid) || Pid <- GlobalPids],\n    ok;\nsubscribe_refcount(_Subscriber, []) ->\n    ok.\n\n-spec subscribe_single_refcount(FD::pid(), Subscriber::comm:erl_local_pid(),\n                                WatchedPid::comm:mypid()) -> ok.\nsubscribe_single_refcount(FD, Subscriber, GlobalPid) ->\n    Key = {'$fd_subscribe', GlobalPid},\n    OldCount = case erlang:get(Key) of\n                   undefined -> subscribe_single(FD, Subscriber, GlobalPid),\n                                0;\n                   X -> X\n               end,\n    erlang:put(Key, OldCount + 1),\n    ok.\n\n%% @doc Deletes the failure detector for the given pid.\n-spec unsubscribe(Subscriber::comm:erl_local_pid(), WatchedPids::[comm:mypid()]) -> ok.\nunsubscribe(Subscriber, [_|_] = GlobalPids) ->\n    FD = my_fd_pid(),\n    _ = [unsubscribe_single(FD, Subscriber, Pid) || Pid <- GlobalPids],\n    ok;\nunsubscribe(_Subscriber, []) ->\n    ok.\n\n-spec unsubscribe_single(FD::pid(), Subscriber::comm:erl_local_pid(),\n                         WatchedPid::comm:mypid()) -> ok.\nunsubscribe_single(FD, Subscriber, GlobalPid) ->\n    comm:send_local(FD, {del_subscriber_via_fd, Subscriber, GlobalPid}).\n\n%% @doc Deletes the failure detector for the given pid - uses\n%%      reference counting to be subscribed to a pid only once.\n%%      Subscribe with subscribe_refcount/3!\n-spec unsubscribe_refcount(Subscriber::comm:erl_local_pid(),\n                           WatchedPids::[comm:mypid()]) -> ok.\nunsubscribe_refcount(Subscriber, [_|_] = GlobalPids) ->\n    FD = my_fd_pid(),\n    _ = [unsubscribe_single_refcount(FD, Subscriber, Pid) || Pid <- GlobalPids],\n    ok;\nunsubscribe_refcount(_Subscriber, []) ->\n    ok.\n\n-spec unsubscribe_single_refcount(FD::pid(), Subscriber::comm:erl_local_pid(),\n                                  WatchedPid::comm:mypid()) -> ok.\nunsubscribe_single_refcount(FD, Subscriber, GlobalPid) ->\n    Key = {'$fd_subscribe', GlobalPid},\n    _ = case erlang:get(Key) of\n            undefined -> ok; % TODO: warn here?\n            1 -> %% delay the actual unsubscribe for better perf.?\n                unsubscribe_single(FD, Subscriber, GlobalPid),\n                erlang:erase(Key);\n            OldCount ->\n                erlang:put(Key, OldCount - 1)\n        end,\n    ok.\n\n%% @doc Unsubscribes Subscriber from the pids in OldPids but not in NewPids\n%%      and subscribes to the pids in NewPids but not in OldPids\n%%      (Subscribers can be pid() or an envelop as created by comm:reply_as/3).\n-spec update_subscriptions(Subscriber::comm:erl_local_pid(),\n                           OldWatchedPids::[comm:mypid()],\n                           NewWatchedPids::[comm:mypid()]) -> ok.\nupdate_subscriptions(Subscriber, OldPids, NewPids) ->\n    {OnlyOldPids, _Same, OnlyNewPids} = util:split_unique(OldPids, NewPids),\n    unsubscribe(Subscriber, OnlyOldPids),\n    subscribe(Subscriber, OnlyNewPids).\n\n%% @doc Reports the calling process' group as being shut down due to a graceful\n%%      leave operation.\n-spec report(Event::event(), LocalPids::[pid()], Data::term()) -> ok.\nreport(Event, LocalPids, Data) ->\n    comm:send_local(my_fd_pid(), {report, Event, LocalPids, Data}).\n\n%% gen_component functions\n%% @doc Starts the failure detector server\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(ServiceGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n      [%% {wait_for_init}, %% when using protected table (for debugging)\n       {erlang_register, ?MODULE},\n       {pid_groups_join_as, ServiceGroup, ?MODULE}]).\n\n%% @doc Initialises the module with an empty state.\n-spec init([]) -> state().\ninit([]) ->\n    % local heartbeat processes\n    _ = pdb:new(fd_hbs, [set]), %% for debugging ++ [protected, named_table]),\n    [].\n\n%% @private\n-spec on(comm:message(), state()) -> state().\non({hbs_finished, RemoteWatchedPid}, State) ->\n    RemFD = comm:get(fd, RemoteWatchedPid),\n    case pdb:take(RemFD, fd_hbs) of\n        undefined -> State;\n        {RemFD, HBPid} -> lists:delete(HBPid, State)\n    end;\n\non({subscribe_heartbeats, Subscriber, TargetPid}, State) ->\n    %% we establish the back-direction here, so we subscribe to the\n    %% subscriber and add the TargetPid to the local monitoring.\n    ?TRACE(\"FD: subscribe_heartbeats~n\", []),\n    {HBPid, NewState} = forward_to_hbs(State, Subscriber, {add_watching_of, TargetPid}),\n    comm:send(Subscriber, {update_remote_hbs_to, comm:make_global(HBPid)}, ?SEND_OPTIONS),\n    NewState;\n\non({pong, RemHBSSubscriber, RemoteDelay}, State) ->\n    ?TRACE(\"FD: pong, ~p~n\", [RemHBSSubscriber]),\n    element(2, forward_to_hbs(State, RemHBSSubscriber,\n                              {pong_via_fd, RemHBSSubscriber, RemoteDelay}));\n\non({add_subscriber_via_fd, Subscriber, WatchedPid}, State) ->\n    ?TRACE(\"FD: subscribe ~p to ~p~n\", [Subscriber, WatchedPid]),\n    element(2, forward_to_hbs(State, WatchedPid,\n                              {add_subscriber, Subscriber, WatchedPid}));\n\non({del_subscriber_via_fd, Subscriber, WatchedPid}, State) ->\n    ?TRACE(\"FD: unsubscribe ~p to ~p~n\", [Subscriber, WatchedPid]),\n    element(2, forward_to_hbs(State, WatchedPid,\n                              {del_subscriber, Subscriber, WatchedPid}));\n\non({add_watching_of_via_fd, Subscriber, Pid}, State) ->\n    ?TRACE(\"FD: add_watching_of ~p~n\", [Pid]),\n    element(2, forward_to_hbs(State, Subscriber, {add_watching_of, Pid}));\n\non({del_watching_of_via_fd, Subscriber, Pid}, State) ->\n    ?TRACE(\"FD: del_watching_of ~p~n\", [Pid]),\n    element(2, forward_to_hbs(State, Subscriber, {del_watching_of, Pid}));\n\non({crashed, WatchedPid, _Warn} = Msg, State) ->\n    ?TRACE(\"FD: crashed message via fd for watched pid ~p~n\", [WatchedPid]),\n    element(2, forward_to_hbs(State, WatchedPid, Msg, false));\n\n%% on({web_debug_info, _Requestor}, State) ->\n%%     ?TRACE(\"FD: web_debug_info~n\", []),\n%% TODO: reimplement for new fd.\n%%     Subscriptions = fd_db:get_subscriptions(),\n%%     % resolve (local and remote) pids to names:\n%%     S2 = [begin\n%%               case comm:is_local(TargetPid) of\n%%                   true -> {Subscriber,\n%%                            pid_groups:pid_to_name(comm:make_local(TargetPid))};\n%%                   _ ->\n%%                       comm:send(comm:get(pid_groups, TargetPid),\n%%                                 {group_and_name_of, TargetPid, comm:this()}, ?SEND_OPTIONS),\n%%                       receive\n%%                           {group_and_name_of_response, Name} ->\n%%                               {Subscriber, pid_groups:pid_to_name2(Name)}\n%%                       after 2000 -> X\n%%                       end\n%%               end\n%%           end || X = {Subscriber, TargetPid} <- Subscriptions],\n%%     KeyValueList =\n%%         [{\"subscriptions\", length(Subscriptions)},\n%%          {\"subscriptions (subscriber, target):\", \"\"} |\n%%          [{pid_groups:pid_to_name(Pid),\n%%            webhelpers:safe_html_string(\"~p\", [X]))} || {Pid, X} <- S2]],\n%%     comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n%%     State;\n\non({report, _Event, _LocalPids, _Data} = Msg, State) ->\n    ?TRACE(\"FD: report ~p for pids ~.2p with data ~.2p~n\",\n           [_Event, _LocalPids, _Data]),\n    % don't create new hbs processes!\n    _ = [comm:send_local(HBS, Msg) || HBS <- State],\n    State;\non({del_all_subscriptions, _Subscribers} = Msg, State) ->\n    _ = [comm:send_local(HBS, Msg) || HBS <- State],\n    State.\n\n%%% Internal functions\n-spec my_fd_pid() -> pid() | failed.\nmy_fd_pid() ->\n    case whereis(?MODULE) of\n        undefined ->\n            log:log(error, \"[ FD ] call of my_fd_pid undefined\"),\n            failed;\n        PID -> PID\n    end.\n\n%% @doc Forwards the given message to the registered HBS or creates a new HBS.\n-spec forward_to_hbs(state(), comm:mypid(), comm:message()) -> {HBPid::pid(), state()}.\nforward_to_hbs(State, Pid, Msg) ->\n    forward_to_hbs(State, Pid, Msg, true).\n\n%% @doc Forwards the given message to the registered hbs or either creates a\n%%      new hbs inside the fd process context (if Create is true) or\n%%      ignores the message (Create is false).\n-spec forward_to_hbs(state(), comm:mypid(), comm:message(), Create::true) -> {HBPid::pid(), state()};\n                    (state(), comm:mypid(), comm:message(), Create::false) -> {HBPid::pid() | undefined, state()}.\nforward_to_hbs(State, Pid, Msg, Create) ->\n    FDPid = comm:get(fd, Pid),\n    case pdb:get(FDPid, fd_hbs) of\n        undefined when Create ->\n            % synchronously create new hb process\n            {ok, HBSPid} = fd_hbs:start_link(pid_groups:my_groupname(), FDPid),\n            pdb:set({FDPid, HBSPid}, fd_hbs),\n            comm:send_local(HBSPid, Msg),\n            {HBSPid, [HBSPid | State]};\n        undefined ->\n            {undefined, State};\n        Entry ->\n            HBSPid = element(2, Entry),\n            comm:send_local(HBSPid, Msg),\n            {HBSPid, State}\n    end.\n\n%% @doc show subscriptions\n-spec subscriptions() -> ok.\nsubscriptions() ->\n    FD = my_fd_pid(),\n    _ = case FD of\n            failed -> [];\n            FD ->\n                TranslatePid = fun(Pid) ->\n                                       case pid_groups:group_and_name_of(Pid) of\n                                           failed       -> Pid;\n                                           GroupAndName -> GroupAndName\n                                       end\n                               end,\n                {dictionary, Dictionary} = process_info(FD, dictionary),\n                All_HBS = [ X || {{_,_,fd},{{_,_,fd},X}} <- Dictionary ],\n                io:format(\"Remote nodes watched: ~p~n\", [length(All_HBS)]),\n                [ begin\n                      io:format(\"fd_hbs: ~p~n\", [pid_groups:group_and_name_of(X)]),\n                      {dictionary, FD_HBS_Dict} = process_info(X, dictionary),\n                      [ begin\n                            PlainPid = comm:get_plain_pid(LSub),\n                            SubPid = TranslatePid(PlainPid),\n                            Watched = case comm:is_local(WPid) of\n                                          true ->\n                                              TranslatePid(comm:make_local(WPid));\n                                          false ->\n                                              WPid\n                                      end,\n                            case PlainPid of\n                                LSub -> io:format(\"  ~p -> ~p ~p~n\",\n                                                  [SubPid, Watched, Count]);\n                                _ -> io:format(\"  ~p -> ~p ~p~n    (subscribed as ~p)~n\",\n                                               [SubPid, Watched, Count, LSub])\n                            end\n                        end\n                        || {{LSub, {_,_,_} = WPid} = Key, {Key, Count}} <- FD_HBS_Dict ]\n                  end || X <- All_HBS ]\n        end,\n    ok.\n"
  },
  {
    "path": "src/fd_hbs.erl",
    "content": "% @copyright 2010-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc    Heartbeat server (HBS) for fd.erl.\n%%         Instatiated per pair of Erlang nodes/VMs.\n%%         Sends heartbeats (symmetrically), and is proxy for local /\n%%         remote subscriptions on Pid granularity. Uses\n%%         erlang:monitor/2 to watch local Pids and forwards crash\n%%         notification, when a watched Pid finishes.\n%% @end\n%% @version $Id$\n-module(fd_hbs).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-compile({inline, [state_get_rem_hbs/1, state_set_rem_hbs/2,\n                   state_get_rem_pids/1, state_set_rem_pids/2,\n                   state_get_last_pong/1, state_set_last_pong/2,\n                   state_get_crashed_after/1, state_set_crashed_after/2,\n                   state_get_table/1,\n                   state_get_monitor_tab/1%, state_set_monitor_tab/2\n                  ]}).\n\n-compile({inline, [rempid_get_rempid/1,\n                   rempid_refcount/1,\n                   rempid_get_last_modified/1, rempid_set_last_modified/2,\n                   rempid_get_pending_demonitor/1, rempid_set_pending_demonitor/2\n                  ]}).\n\n%% -define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(_X,_Y), ok).\n%-define(TRACEPONG(X,Y), io:format(X,Y)).\n-define(TRACEPONG(_X,_Y), ok).\n%%-define(TRACE_NOT_SUBSCRIBED_UNSUBSCRIBE(X,Y), log:log(warn, X, Y)).\n-define(TRACE_NOT_SUBSCRIBED_UNSUBSCRIBE(_X,_Y), ok).\n\n-behaviour(gen_component).\n-include(\"scalaris.hrl\").\n\n-export([init/1, on/2, start_link/2, check_config/0]).\n\n-include(\"gen_component.hrl\").\n\n-type(rempid() :: %% locally existing subscriptions for remote pids\n        {\n          comm:mypid(),  %% remote pid a local subscriber is subscribed to\n                         %% the other end (fd_hbs) has a monitor\n                         %% established for this\n          non_neg_integer(), %% number of local subscribers for the remote pid\n          erlang_timestamp(),   %% delay remote demonitoring:\n                         %%   time of ref count reached 0\n                         %%   all other modifications change this to {0,0,0}\n          boolean()      %% delayed demonitoring requested and still open?\n        }).\n\n-type(state() :: {\n       comm:mypid(), %% remote hbs\n       [rempid()],  %% subscribed rem. pids + ref counting\n       erlang_timestamp(),  %% local time of last pong arrival\n       erlang_timestamp(),  %% remote is crashed if no pong arrives until this\n       LocalSubscriberTab::pdb:tableid(),\n       %% locally erlang:monitored() pids for a remote hbs:\n       MonitorTab::pdb:tableid()\n     }).\n\n-define(SEND_OPTIONS, [{channel, prio}]).\n\n%% Akronyms: HBS =:= (local) heartbeat server instance\n\n%% @doc spawns a fd_hbs instance\n-spec start_link(pid_groups:groupname(), comm:mypid()) -> {ok, pid()}.\nstart_link(ServiceGroup, RemotePid) ->\n    RemoteFDPid = comm:get(fd, RemotePid),\n    Name = lists:flatten(io_lib:format(\"fd <-> ~p\", [RemoteFDPid])),\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, RemotePid,\n                             [%% {wait_for_init}, %% when using protected ets table\n                              {pid_groups_join_as, ServiceGroup, Name}]).\n\n-spec init(comm:mypid()) -> state().\ninit(RemotePid) ->\n    ?TRACE(\"fd_hbs init: RemotePid ~p~n\", [RemotePid]),\n    LocalSubscriberTab = pdb:new(?MODULE, [set]), %% debugging: ++ [protected]),\n    MonitorTab = pdb:new(?MODULE, [set]), %% debugging: ++ [protected]),\n    RemoteFDPid = comm:get(fd, RemotePid),\n    comm:send(RemoteFDPid,\n              {subscribe_heartbeats, comm:this(), RemotePid},\n              ?SEND_OPTIONS ++ [{shepherd, shepherd_new(0)}]),\n\n    %% no periodic alive check inside same vm (to succeed unittests)\n    case comm:is_local(RemotePid) of\n        false -> comm:send_local(self(), {periodic_alive_check});\n        true -> ok\n    end,\n    Now = os:timestamp(),\n    state_new(_RemoteHBS = RemoteFDPid,\n              _RemotePids = [],\n              _LastPong = Now,\n              _CrashedAfter = util:time_plus_ms(Now, delayfactor() * failureDetectorInterval()),\n              LocalSubscriberTab, MonitorTab).\n\n-spec on(comm:message(), state()) -> state() | kill.\non({add_subscriber, Subscriber, WatchedPid} = _Msg, State) ->\n    ?TRACE(\"fd_hbs add_subscriber ~.0p~n\", [_Msg]),\n    %% register subscriber locally\n    S1 = state_add_entry(State, {Subscriber, WatchedPid}),\n    %% add watched pid remotely, if not already watched\n    _S2 = state_add_watched_pid(S1, WatchedPid);\n\non({del_subscriber, Subscriber, WatchedPid} = _Msg, State) ->\n    ?TRACE(\"fd_hbs del_subscriber ~.0p~n\", [_Msg]),\n    %% unregister subscriber locally\n    {Changed, S1} = state_del_entry(State, {Subscriber, WatchedPid}),\n    %% delete watched pid remotely, if no longer needed\n    case Changed of\n        deleted -> _S2 = state_del_watched_pid(S1, WatchedPid, Subscriber);\n        unchanged -> S1\n    end;\n\non({del_all_subscriptions, Subscribers} = _Msg, State) ->\n    ?TRACE(\"fd_hbs del_all_subscriptions ~.0p~n\", [_Msg]),\n    state_del_all_subscriptions(State, Subscribers);\n\non({check_delayed_del_watching_of, WatchedPid, Time} = _Msg, State) ->\n    ?TRACE(\"fd_hbs check_delayed_del_watching_of ~.0p~n\", [_Msg]),\n    %% initiate demonitoring and delete local entry if\n    %% entry time is still unmodified since this message was triggered\n    RemPids = state_get_rem_pids(State),\n    case lists:keyfind(WatchedPid, 1, RemPids) of\n        false ->\n            %% WatchedPid may be crashed and therefore the entry was\n            %% already removed in on({crashed, ...}, ...).\n            log:log(info, \"req. to delete non watched/crashed pid ~p.~n\",\n                    [WatchedPid]),\n            State;\n        Entry ->\n            NewRemPids =\n                case rempid_get_last_modified(Entry) of\n                    Time -> %% untouched for whole wait period\n                        RemHBS = state_get_rem_hbs(State),\n                        case comm:make_local(RemHBS) of\n                            fd -> comm:send(RemHBS, {del_watching_of_via_fd, comm:this(), WatchedPid}, ?SEND_OPTIONS);\n                            _ -> comm:send(RemHBS, {del_watching_of, WatchedPid}, ?SEND_OPTIONS)\n                        end,\n                        lists:keydelete(WatchedPid, 1, RemPids);\n                    _ ->\n                        NewEntry =\n                            case rempid_refcount(Entry) of\n                                0 -> trigger_delayed_del_watching(Entry);\n                                _ -> rempid_set_pending_demonitor(Entry, false)\n                            end,\n                        lists:keyreplace(WatchedPid, 1, RemPids, NewEntry)\n                end,\n            state_set_rem_pids(State, NewRemPids)\n    end;\n\non({add_watching_of, WatchedPid} = _Msg, State) ->\n    ?TRACE(\"fd_hbs add_watching_of ~.0p~n\", [_Msg]),\n    %% request from remote fd_hbs: watch a pid locally and forward\n    %% events on it to the other side\n    state_add_monitor(State, WatchedPid);\n\non({del_watching_of, WatchedPid} = _Msg, State) ->\n    ?TRACE(\"fd_hbs del_watching_of ~.0p~n\", [_Msg]),\n    %% request from remote fd_hbs: no longer watch a pid locally\n    element(2, state_del_monitor(State, WatchedPid));\n\non({update_remote_hbs_to, Pid}, State) ->\n    ?TRACE(\"fd_hbs update_remote_hbs_to ~p~n\", [Pid]),\n    %% process Pid is remote contact for this fd_hbs. First, we\n    %% register the fd process of the remote side. When a fd_hbs is\n    %% started remotely, we get its reference via this message.\n    state_set_rem_hbs(State, Pid);\n\non({stop}, _State) ->\n    ?TRACE(\"fd_hbs stop~n\", []),\n    kill;\n\non({pong_via_fd, RemHBSSubscriber, RemoteDelay}, State) ->\n    ?TRACEPONG(\"fd_hbs pong via fd~n\", []),\n    comm:send(RemHBSSubscriber, {update_remote_hbs_to, comm:this()}, ?SEND_OPTIONS),\n    NewState = state_set_rem_hbs(State, RemHBSSubscriber),\n    on({pong, RemHBSSubscriber, RemoteDelay}, NewState);\n\non({pong, _Subscriber, RemoteDelay}, State) ->\n    ?TRACEPONG(\"Pinger pong for ~p~n\", [_Subscriber]),\n    Now = os:timestamp(),\n    LastPong = state_get_last_pong(State),\n    CrashedAfter = state_get_crashed_after(State),\n    PongDelay = abs(timer:now_diff(Now, LastPong)),\n    Delay = erlang:max(PongDelay, failureDetectorInterval()),\n    S1 = state_set_last_pong(State, Now),\n    NewCrashedAfter = lists:max(\n                        [util:time_plus_us(Now, delayfactor() * Delay),\n                         util:time_plus_ms(CrashedAfter, 1000),\n                         util:time_plus_us(Now, RemoteDelay)]),\n    %% io:format(\"Time for next pong: ~p s~n\",\n    %%           [timer:now_diff(NewCrashedAfter, Now)/1000000]),\n    state_set_crashed_after(S1, NewCrashedAfter);\n\non({periodic_alive_check}, State) ->\n    ?TRACEPONG(\"Pinger periodic_alive_check~n\", []),\n    Now = os:timestamp(),\n    CrashedAfter = state_get_crashed_after(State),\n    comm:send(\n      state_get_rem_hbs(State),\n      {pong, comm:this(),\n       timer:now_diff(\n         CrashedAfter,\n         util:time_plus_ms(state_get_last_pong(State),\n                           failureDetectorInterval()\n                           %% the following is the reduction rate\n                           %% when increased earlier\n                           + failureDetectorInterval() div 3))},\n      ?SEND_OPTIONS ++ [{shepherd, shepherd_new(0)}]),\n    NewState = case 0 < timer:now_diff(Now, CrashedAfter) of\n                   true -> report_connection_crash(State);\n                   false -> State\n    end,\n    %% trigger next timeout\n    _ = comm:send_local_after(failureDetectorInterval(),\n                              self(), {periodic_alive_check}),\n    NewState;\n\non({send_retry, {send_error, Target, Message, _Reason} = Err, Count}, State) ->\n    NextOp =\n        case Count of\n            1 -> {retry};\n            2 -> {delay, _Wait = 1};\n            3 -> {retry};\n            4 -> {delay, _Wait = 2};\n            5 -> {retry};\n            6 -> {giveup}\n        end,\n    case NextOp of\n        {retry} ->\n            comm:send(Target, Message,\n                      ?SEND_OPTIONS ++ [{shepherd, shepherd_new(Count)}]),\n            State;\n        {delay, Sec} ->\n            msg_delay:send_local(\n              Sec, self(), {send_retry, Err, Count + 1}),\n            State;\n        {giveup} ->\n            log:log(warn, \"[ FD ] Sending ~.0p failed 3 times. \"\n                    \"Report target ~.0p as crashed.~n\", [Message, Target]),\n            %% report whole node as crashed, when not reachable via tcp:\n            report_connection_crash(State)\n    end;\n\non({crashed, WatchedPid, Reason}, State) ->\n    ?TRACE(\"fd_hbs crashed ~p~n\", [WatchedPid]),\n    report_crashed_remote_pid(State, WatchedPid, Reason, warn);\n\non({fd_notify, Event, WatchedPids, Data}, State) ->\n    ?TRACE(\"fd_hbs: fd_notify ~p for pids ~.2p with data ~.2p~n\",\n           [Event, WatchedPids, Data]),\n    lists:foldl(fun(WatchedPid, StateX) ->\n                        fd_notify(StateX, Event, WatchedPid, Data)\n                end, State, WatchedPids);\n\non({report, crash, LocalPids, Reason}, State) ->\n    ?TRACE(\"fd_hbs: report crash for pids ~.2p with reason ~.2p~n\",\n           [LocalPids, Reason]),\n    % similar to 'DOWN' report below\n    {WatchedPids, State1} =\n        lists:foldl(\n          fun(LocalPid, {WatchedPidsX, StateX}) ->\n                  GlobalPid = comm:make_global(LocalPid),\n                  %% delete WatchedPid and MonRef locally (if existing)\n                  case state_del_monitor(StateX, GlobalPid) of\n                      {true, StateX1} -> {[GlobalPid | WatchedPidsX], StateX1};\n                      {false, StateX1} -> {WatchedPidsX, StateX1}\n                  end\n          end, {[], State}, LocalPids),\n    %% send crash report to remote end.\n    comm:send(state_get_rem_hbs(State),\n              {fd_notify, crash, WatchedPids, Reason},\n              ?SEND_OPTIONS),\n    State1;\n\non({report, Event, LocalPids, Data}, State) ->\n    ?TRACE(\"fd_hbs: report ~p for pids ~.2p with data ~.2p~n\",\n           [Event, LocalPids, Data]),\n    % similar to 'DOWN' report below\n    WatchedPids =\n        [GlobalPid || LocalPid <- LocalPids,\n                      state_has_monitor(State,\n                                        GlobalPid = comm:make_global(LocalPid))],\n    %% send event to remote end.\n    comm:send(state_get_rem_hbs(State),\n              {fd_notify, Event, WatchedPids, Data},\n              ?SEND_OPTIONS),\n    State;\n\non({'DOWN', _MonitorRef, process, WatchedPid, _}, State) ->\n    ?TRACE(\"fd_hbs DOWN reported ~.0p, ~.0p~n\",\n           [WatchedPid, pid_groups:group_and_name_of(WatchedPid)]),\n    %% send crash report to remote end.\n    GlobalPid = comm:make_global(WatchedPid),\n    comm:send(state_get_rem_hbs(State),\n              {crashed, GlobalPid, 'DOWN'}, ?SEND_OPTIONS),\n    %% delete WatchedPid and MonRef locally (MonRef is already\n    %% invalid, as Pid crashed)\n    _S1 = element(2, state_del_monitor(State, GlobalPid)).\n\n%% @doc Checks existence and validity of config parameters for this module.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(failure_detector_interval) and\n    config:cfg_is_greater_than(failure_detector_interval, 0).\n\n%% @doc Reports a crashed connection to local subscribers.\n%% @private\n-spec report_crashed_remote_pid(state(), WatchedPid::comm:mypid(),\n                                Reason::fd:reason(), warn | nowarn) -> state().\nreport_crashed_remote_pid(State, WatchedPid, Reason, Warn) ->\n    %% inform all local subscribers\n    Subscriptions = state_get_subscriptions(State, WatchedPid),\n    %% only there because of delayed demonitoring?\n    ?TRACE(\"~p found subs: ~p~n\", [self(), Subscriptions]),\n    RemPids = state_get_rem_pids(State),\n    case Subscriptions of\n        [] ->\n            %% Report if subcription entry exists not because of a\n            %% delayed demonitoring\n            Report = case lists:keyfind(WatchedPid, 1, RemPids) of\n                         false -> true;\n                         RemPid ->\n                             not rempid_get_pending_demonitor(RemPid)\n                     end,\n            case Report andalso (nowarn =/= Warn) of\n                true ->\n                    log:log(info, \"No one to inform on crash of ~.0p~n\",\n                            [WatchedPid]);\n                false -> ok\n            end;\n        _ -> ok\n    end,\n    This = comm:this(),\n    _ = [ begin\n              log:log(debug, \"[ FD ~p ] Sending crash to ~.0p/~.0p~n\",\n                      [This, X, pid_groups:group_and_name_of(comm:get_plain_pid(X))]),\n              comm:send_local(X, {fd_notify, crash, WatchedPid, Reason}, [{?quiet}])\n          end\n          || X <- Subscriptions ],\n    %% delete from remote_pids\n    NewRemPids = lists:keydelete(WatchedPid, 1, RemPids),\n    S1 = state_set_rem_pids(State, NewRemPids),\n    %% delete subscription entries with this pid\n    lists:foldl(fun(Sub, StAgg) ->\n                        {_, Res} = state_del_entry(StAgg, {Sub, WatchedPid}),\n                        Res\n                end,\n                S1, Subscriptions).\n\n%% @doc Reports a crashed connection to local subscribers.\n%% @private\n-spec fd_notify(state(), Event::fd:event(), Pid::comm:mypid(),\n                Data::fd:reason()) -> state().\nfd_notify(State, crash, WatchedPid, Reason) ->\n    report_crashed_remote_pid(State, WatchedPid, Reason, nowarn);\nfd_notify(State, Event, WatchedPid, Data) ->\n    %% inform all local subscribers\n    Subscriptions = state_get_subscriptions(State, WatchedPid),\n    ?TRACE(\"~p found subs: ~p~n\", [self(), Subscriptions]),\n    % note: in contrast to report_crashed_remote_pid, do not warn if no subscribers exist\n    This = comm:this(),\n    _ = [ begin\n              log:log(debug, \"[ FD ~p ] Sending crash to ~.0p/~.0p~n\",\n                      [This, X, pid_groups:group_and_name_of(comm:get_plain_pid(X))]),\n              comm:send_local(X, {fd_notify, Event, WatchedPid, Data})\n          end\n          || X <- Subscriptions ],\n    State.\n\n%% @doc Reports a crashed connection to local subscribers.\n%% @private\n-spec report_connection_crash(state()) -> kill.\nreport_connection_crash(State) ->\n    RemPids = state_get_rem_pids(State),\n    case RemPids of\n        [] ->\n            log:log(info, \"[ FD ~p ] remote host ~p (fd_hbs) is unresponsive.~n\"\n                    ++    \"          No pids to report as crashed locally.~n\"\n                    ++    \"          Finishing own fd_hbs.\",\n                    [comm:this(), state_get_rem_hbs(State)]);\n        _ -> log:log(warn, \"[ FD ~p ] reports ~.0p as crashed and finishes.\",\n                     [comm:this(), RemPids])\n    end,\n    FD = pid_groups:find_a(fd),\n    comm:send_local(FD, {hbs_finished, state_get_rem_hbs(State)}),\n    erlang:unlink(FD),\n    _ = try\n            lists:foldl(fun(X, S) -> on({crashed, X, noconnection}, S) end,\n                        State, [ rempid_get_rempid(RemPidEntry)\n                                 || RemPidEntry <- state_get_rem_pids(State)])\n        catch _:_ -> ignore_exception\n        end,\n    kill.\n\n%% @doc The interval between two failure detection runs.\n-spec failureDetectorInterval() -> pos_integer().\nfailureDetectorInterval() -> config:read(failure_detector_interval).\n\n-spec delayfactor() -> pos_integer().\ndelayfactor() -> 4.\n\n-spec trigger_delayed_del_watching(rempid()) -> rempid().\ntrigger_delayed_del_watching(RemPidEntry) ->\n    %% delayed demonitoring: remember current time self-inform on\n    %% pending demonitoring with current time.\n    %% actually delete if timestamp of the entry is still the same\n    %% after delay\n    WatchedPid = rempid_get_rempid(RemPidEntry),\n    Time = os:timestamp(),\n    msg_delay:send_local(1, self(),\n                         {check_delayed_del_watching_of, WatchedPid, Time}),\n    E1 = rempid_set_last_modified(RemPidEntry, Time),\n    rempid_set_pending_demonitor(E1, true).\n\n-spec state_new(RemoteHBS::comm:mypid(), RemotePids::[rempid()],\n                LastPong::erlang_timestamp(), CrashedAfter::erlang_timestamp(),\n                LocalSubscriberTab::pdb:tableid(),\n                MonitorTab::pdb:tableid()) -> state().\nstate_new(RemoteHBS, RemotePids, LastPong, CrashedAfter, LocalSubscriberTab, MonitorTab) ->\n    {RemoteHBS, RemotePids, LastPong, CrashedAfter, LocalSubscriberTab, MonitorTab}.\n\n-spec state_get_rem_hbs(state())    -> comm:mypid().\nstate_get_rem_hbs(State)            -> element(1, State).\n-spec state_set_rem_hbs(state(), comm:mypid()) -> state().\nstate_set_rem_hbs(State, Val)       -> setelement(1, State, Val).\n-spec state_get_rem_pids(state())   -> [rempid()].\nstate_get_rem_pids(State)           -> element(2, State).\n-spec state_set_rem_pids(state(), [rempid()]) -> state().\nstate_set_rem_pids(State, Val)      -> setelement(2, State, Val).\n-spec state_get_last_pong(state())  -> erlang_timestamp().\nstate_get_last_pong(State)          -> element(3, State).\n-spec state_set_last_pong(state(), erlang_timestamp()) -> state().\nstate_set_last_pong(State, Val)     -> setelement(3, State, Val).\n-spec state_get_crashed_after(state()) -> erlang_timestamp().\nstate_get_crashed_after(State)      -> element(4, State).\n-spec state_set_crashed_after(state(), erlang_timestamp()) -> state().\nstate_set_crashed_after(State, Val) -> setelement(4, State, Val).\n-spec state_get_table(state()) -> pdb:tableid().\nstate_get_table(State)              -> element(5, State).\n-spec state_get_monitor_tab(state())-> pdb:tableid().\nstate_get_monitor_tab(State)        -> element(6, State).\n%% -spec state_set_monitor_tab(state(), pdb:tableid()) -> state().\n%% state_set_monitor_tab(State, Val)   -> setelement(6, State, Val).\n\n-spec state_add_entry(state(), {comm:erl_local_pid(), comm:mypid()}) -> state().\nstate_add_entry(State, {_Subscriber, _WatchedPid} = X) ->\n    %% implement reference counting on subscriptions:\n    %% instead of storing in the state, we silently store in a pdb for\n    %% better performance.\n    Table = state_get_table(State),\n    case pdb:get(X, Table) of\n        undefined ->\n            pdb:set({X, 1}, Table);\n        {X, Count} ->\n            pdb:set({X, Count + 1}, Table)\n    end,\n    State.\n\n-spec state_del_entry(state(), {comm:erl_local_pid(), comm:mypid()})\n        -> {deleted | unchanged, state()}.\nstate_del_entry(State, {_Subscriber, _WatchedPid} = X) ->\n    %% implement reference counting on subscriptions:\n    %% instead of storing in the state, we silently store in a pdb for\n    %% better performance.\n    Table = state_get_table(State),\n    case pdb:get(X, Table) of\n        undefined ->\n            ?TRACE_NOT_SUBSCRIBED_UNSUBSCRIBE(\n               \"got unsubscribe for not registered subscription ~.0p, \"\n               \"Subscriber ~.0p, Watching group and name: ~.0p.~n\",\n               [{unsubscribe, _Subscriber, _WatchedPid},\n                pid_groups:group_and_name_of(comm:get_plain_pid(_Subscriber)),\n                pid_groups:group_and_name_of(comm:get_plain_pid(comm:make_local(_WatchedPid)))]),\n            {unchanged, State};\n        {X, 1} ->\n            pdb:delete(X, Table),\n            {deleted, State};\n        {X, Count} when Count > 1 ->\n            pdb:set({X, Count - 1}, Table),\n            {unchanged, State}\n    end.\n\n-spec state_get_subscriptions(state(), comm:mypid()) -> [comm:erl_local_pid()].\nstate_get_subscriptions(State, SearchedPid) ->\n    Table = state_get_table(State),\n    Entries = pdb:tab2list(Table),\n    _Res = [ Subscriber\n             || {{Subscriber, WatchedPid}, _Num} <- Entries,\n                %% pdb:tab2list may contain unrelated entries, but <- lets\n                %% only pass structurally matching entries here without an\n                %% assignment exception.\n                Subscriber =/= '$monitor_count',\n                Subscriber =/= '$monitor',\n                SearchedPid =:= WatchedPid].\n\n-spec state_del_all_subscriptions(state(), [comm:erl_local_pid()]) -> state().\nstate_del_all_subscriptions(State, SubscriberPids) ->\n    Table = state_get_table(State),\n    Set = gb_sets:from_list(SubscriberPids),\n    Entries = pdb:tab2list(Table),\n    lists:foldl(\n      fun({Key = {Subscriber, WatchedPid}, _Num}, StateX) ->\n              case gb_sets:is_member(Subscriber, Set) of\n                  true ->\n                      %% unregister subscriber locally\n                      pdb:delete(Key, Table),\n                      %% delete watched pid remotely, if no longer needed\n                      state_del_watched_pid(StateX, WatchedPid, Subscriber);\n                  false ->\n                      StateX\n              end;\n         (_, StateX) ->\n              StateX\n      end, State, Entries).\n\n-spec state_add_watched_pid(state(), comm:mypid()) -> state().\nstate_add_watched_pid(State, WatchedPid) ->\n    %% add watched pid remotely, if not already watched\n    RemPids = state_get_rem_pids(State),\n    case lists:keytake(WatchedPid, 1, RemPids) of\n        {value, Entry, RestRemPids} ->\n            state_set_rem_pids(State, [rempid_inc(Entry) | RestRemPids]);\n        false ->\n            %% add to remote site\n            RemHBS = state_get_rem_hbs(State),\n            case comm:make_local(RemHBS) of\n                fd -> comm:send(\n                        RemHBS, {add_watching_of_via_fd, comm:this(), WatchedPid},\n                        ?SEND_OPTIONS ++ [{shepherd, shepherd_new(0)}]);\n                _  -> comm:send(\n                        RemHBS, {add_watching_of, WatchedPid},\n                        ?SEND_OPTIONS ++ [{shepherd, shepherd_new(0)}])\n            end,\n            %% add to list\n            state_set_rem_pids(\n              State, [rempid_inc(rempid_new(WatchedPid)) | RemPids])\n    end.\n\n-spec state_del_watched_pid(state(), comm:mypid(), comm:erl_local_pid()) -> state().\nstate_del_watched_pid(State, WatchedPid, Subscriber) ->\n    %% del watched pid remotely, if not longer necessary\n    RemPids = state_get_rem_pids(State),\n    case lists:keytake(WatchedPid, 1, RemPids) of\n        {value, Entry, RestRemPids} ->\n            TmpEntry = rempid_dec(Entry),\n            NewEntry =\n                case {rempid_refcount(TmpEntry),\n                      rempid_get_pending_demonitor(TmpEntry)} of\n                    {0, false} -> %% dec to 0 and triggger new delayed message\n                        trigger_delayed_del_watching(TmpEntry);\n                    {0, true} -> %% dec to 0 and no new delayed message needed\n                        TmpEntry;\n                    _ -> TmpEntry\n                end,\n            state_set_rem_pids(State, [NewEntry | RestRemPids]);\n        false -> log:log(warn,\n                         \"req. from ~p (~p) to delete non watched pid ~p.~n\",\n                        [Subscriber,\n                         pid_groups:group_and_name_of(\n                           comm:get_plain_pid(Subscriber)),\n                         WatchedPid]),\n                 State\n    end.\n\n%% @doc Helper to extract the real pid from a watched (local!) pid.\n-spec get_real_pid(comm:mypid()) -> pid().\nget_real_pid(WatchedPid) ->\n    PlainPid = comm:make_local(comm:get_plain_pid(WatchedPid)),\n    if is_atom(PlainPid) ->\n           case whereis(PlainPid) of\n               undefined ->\n                   % this process is not alive anymore -> store the\n                   % registered name instead so the check in\n                   % gen_component:demonitor/2 does not hit\n                   PlainPid;\n               X when is_pid(X) ->\n                   X\n           end;\n       is_pid(PlainPid) ->\n           PlainPid\n    end.\n\n-spec state_add_monitor(state(), comm:mypid()) -> state().\nstate_add_monitor(State, WatchedPid) ->\n    Table = state_get_monitor_tab(State),\n    PlainPid = get_real_pid(WatchedPid),\n    CountKey = {'$monitor_count', PlainPid},\n    X = case pdb:get(CountKey, Table) of\n            undefined ->\n                MonRef = gen_component:monitor(PlainPid),\n                pdb:set({{'$monitor', PlainPid}, MonRef}, Table),\n                1;\n            {CountKey, I} when is_integer(I) ->\n                I + 1\n        end,\n    pdb:set({CountKey, X}, Table),\n    State.\n\n-spec state_del_monitor(state(), comm:mypid()) -> {Found::boolean(), state()}.\nstate_del_monitor(State, WatchedPid) ->\n    Table = state_get_monitor_tab(State),\n    PlainPid = get_real_pid(WatchedPid),\n    CountKey = {'$monitor_count', PlainPid},\n    Found = case pdb:take(CountKey, Table) of\n                undefined ->\n                    false;\n                {CountKey, 1} ->\n                    MonKey = {'$monitor', PlainPid},\n                    {MonKey, MonRef} = pdb:take(MonKey, Table),\n                    gen_component:demonitor(MonRef),\n                    true;\n                {CountKey, X} when is_integer(X) andalso X > 1 ->\n                    pdb:set({CountKey, X - 1}, Table),\n                    true\n            end,\n    {Found, State}.\n\n-spec state_has_monitor(state(), comm:mypid()) -> boolean().\nstate_has_monitor(State, WatchedPid) ->\n    Table = state_get_monitor_tab(State),\n    PlainPid = get_real_pid(WatchedPid),\n    MonKey = {'$monitor', PlainPid},\n    pdb:get(MonKey, Table) =/= undefined.\n\n-spec rempid_new(comm:mypid()) -> rempid().\nrempid_new(Pid) ->\n    {Pid, _RefCount = 0, _DecTo0 = {0,0,0}, _PendingDemonitor = false}.\n-spec rempid_get_rempid(rempid()) -> comm:mypid().\nrempid_get_rempid(Entry) -> element(1, Entry).\n-spec rempid_refcount(rempid()) -> non_neg_integer().\nrempid_refcount(Entry) -> element(2, Entry).\n-spec rempid_inc(rempid()) -> rempid().\nrempid_inc(Entry) ->\n    TmpEntry = setelement(2, Entry, element(2, Entry) + 1),\n    rempid_set_last_modified(TmpEntry, {0,0,0}).\n-spec rempid_dec(rempid()) -> rempid().\nrempid_dec(Entry) ->\n    TmpEntry = setelement(2, Entry, element(2, Entry) - 1),\n    rempid_set_last_modified(TmpEntry, {0,0,0}).\n-spec rempid_get_last_modified(rempid()) -> erlang_timestamp().\nrempid_get_last_modified(Entry) -> element(3, Entry).\n-spec rempid_set_last_modified(rempid(), erlang_timestamp()) -> rempid().\nrempid_set_last_modified(Entry, Time) -> setelement(3, Entry, Time).\n-spec rempid_get_pending_demonitor(rempid()) -> boolean().\nrempid_get_pending_demonitor(Entry) -> element(4, Entry).\n-spec rempid_set_pending_demonitor(rempid(), boolean()) -> rempid().\nrempid_set_pending_demonitor(Entry, Val) -> setelement(4, Entry, Val).\n\n-spec shepherd_new(non_neg_integer()) -> comm:erl_local_pid_with_reply_as().\nshepherd_new(Count) ->\n    comm:reply_as(self(), 2, {send_retry, '_', Count + 1}).\n"
  },
  {
    "path": "src/fix_queue.erl",
    "content": "%  @copyright 2009-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Christian Hennig <hennig@zib.de>\n%% @doc    Queue implementation with a fixed maximum length.\n%% @end\n%% @version $Id$\n-module(fix_queue).\n-author('hennig@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n-export_type([fix_queue/1]).\n\n-export([new/1, add/2, add_unique_head/4, map/2, remove/3,\n         length/1, max_length/1, queue/1]).\n\n-opaque(fix_queue(Element) :: {MaxLength :: pos_integer(),\n                               Length    :: non_neg_integer(),\n                               Queue     :: queue:queue(Element)}).\n\n%% @doc Creates a new fixed-size queue.\n-spec new(MaxLength::pos_integer()) -> fix_queue(any()).\nnew(MaxLength) ->\n    {MaxLength, 0, queue:new()}.\n\n%% @doc Adds an element to the given queue. \n-spec add(Element, Queue::fix_queue(Element)) -> fix_queue(Element).\nadd(Elem, {MaxLength, Length, Queue}) ->\n    {NewLength, NewQueue} =\n        case Length =:= MaxLength of\n            true -> {Length, queue:in(Elem, queue:drop(Queue))};\n            _    -> {Length + 1, queue:in(Elem, Queue)}\n        end,\n    {MaxLength, NewLength, NewQueue}.\n\n%% @doc Adds an element to the given queue. If there is already an equal\n%%      element at the \"head\" (rear) of the queue, it will be replaced by the\n%%      element selected by SelectFun.\n%%      Note that this is much cheaper than checking all elements!\n-spec add_unique_head(Element, Queue::fix_queue(Element),\n        EqFun::fun((Old::Element, New::Element) -> boolean()),\n        SelectFun::fun((Old::Element, New::Element) -> Element)) -> fix_queue(Element).\nadd_unique_head(Elem, {MaxLength, Length, Queue}, EqFun, SelectFun) ->\n    {NewL1, NewQ1} =\n        case queue:peek_r(Queue) of\n            {value, Item} ->\n                case EqFun(Item, Elem) of\n                    true -> {Length, queue:in(SelectFun(Item, Elem), queue:drop_r(Queue))};\n                    _    -> {Length + 1, queue:in(Elem, Queue)}\n                end;\n            empty -> {1, queue:in(Elem, Queue)}\n        end,\n    {NewLength, NewQueue} =\n        case NewL1 > MaxLength of\n            true -> {MaxLength, queue:drop(NewQ1)};\n            _    -> {Length, NewQ1}\n        end,\n    {MaxLength, NewLength, NewQueue}.\n\n%% @doc Maps a function to all elements of the given queue.\n-spec map(fun((Element) -> X), Queue::fix_queue(Element)) -> [X].\nmap(Fun, {_MaxLength, _Length, Queue}) ->\n    lists:map(Fun, queue:to_list(Queue)).\n\n-spec remove(Element, Queue::fix_queue(Element),\n             EqFun::fun((Element, Element) -> boolean())) -> fix_queue(Element).\nremove(Elem, {MaxLength, _Length, Queue}, EqFun) ->\n    NewQueue = queue:filter(fun(X) -> not EqFun(X, Elem) end, Queue),\n    NewLength = queue:len(NewQueue),\n    {MaxLength, NewLength, NewQueue}.\n\n-spec length(fix_queue(any())) -> non_neg_integer().\nlength({_MaxLength, Length, _Queue}) -> Length.\n\n-spec max_length(fix_queue(any())) -> pos_integer().\nmax_length({MaxLength, _Length, _Queue}) -> MaxLength.\n\n-spec queue(fix_queue(Element)) -> queue:queue(Element).\nqueue({_MaxLength, _Length, Queue}) -> Queue.\n"
  },
  {
    "path": "src/ganglia.erl",
    "content": "% @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Marie Hoffmann <hoffmann@zib.de>\n%% @doc Ganglia monitoring interface.\n%% @end\n%% @version $Id$\n-module(ganglia).\n-author('hoffmann@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n-export([start_link/1, init/1, on/2,\n         check_config/0]).\n\n-include(\"gen_component.hrl\").\n\n-define(TRACE(X,Y), ok).\n%define(TRACE(X,Y), io:format(X, Y)).\n\n-define(FD_SUBSCR_PID(AggId),\n        comm:reply_as(self(), 3, {fd, AggId, '_'})).\n\n-type load_aggregation() :: {AggId :: non_neg_integer(),\n                             Pending :: non_neg_integer(),\n                             Load :: non_neg_integer()}.\n\n-type state() :: {LastUpdated :: non_neg_integer(),\n                  LoadAggregation :: load_aggregation()}.\n\n-type message() :: {ganglia_trigger} |\n                   {ganglia_periodic} |\n                   {ganglia_dht_load_aggregation, DHTNode :: pid(), AggId :: non_neg_integer(), message()} |\n                   {ganglia_vivaldi_confidence, pid(), message()} |\n                   {fd_notify, fd:event(), PID::comm:mypid(), _Cookie::{ganglia, AggId::non_neg_integer()}, Reason::fd:reason()}.\n\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(ServiceGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                             [{pid_groups_join_as, ServiceGroup, ganglia}]).\n\n-spec init([]) -> state().\ninit([]) ->\n    UpdateInterval = config:read(ganglia_interval) div 1000,\n    msg_delay:send_trigger(UpdateInterval, {ganglia_trigger}),\n    {_LastActive = 0, {0, 0, 0}}.\n\n-spec on(message(), state()) -> state().\non({ganglia_trigger}, State) ->\n    UpdateInterval = config:read(ganglia_interval) div 1000,\n    msg_delay:send_trigger(UpdateInterval, {ganglia_trigger}),\n    StateNew = inc_agg_id(State),\n    gen_component:post_op(_Msg = {ganglia_periodic}, StateNew);\n\non({ganglia_periodic}, State) ->\n    NewState = send_dht_node_metrics(State),\n    send_general_metrics(),\n    send_message_metrics(),\n    send_rrd_metrics(),\n    send_vivaldi_errors(),\n    set_last_active(NewState);\n\n%% @doc aggregates load information from the dht nodes and\n%%     and sends it out to ganglia if all nodes have replied\non({ganglia_dht_load_aggregation, PID, AggId, Msg}, State) ->\n    ?TRACE(\"~p: ~p~n\", [AggId, Msg]),\n    % all went well, we no longer need the failure detector\n    fd:unsubscribe(?FD_SUBSCR_PID(AggId), [PID]),\n    {get_node_details_response, [{load, Load}]} = Msg,\n    CurAggId = get_agg_id(State),\n    if\n        %% check for correct aggregation id\n        AggId == CurAggId  ->\n            NewState = set_agg_load(Load, State),\n            agg_check_pending(NewState);\n        %% otherwise drop this message\n        true -> State\n    end;\n\n%% @doc handler for messages from failure detector\n%%     if a node crashes before sending out the load data\n%%     we ignore its load information\non({fd, AggId, {fd_notify, crash, _PID, _Reason}}, State) ->\n    ?TRACE(\"Node failed~n\",[]),\n    CurAggId = get_agg_id(State),\n    if\n        AggId == CurAggId ->\n            Pending = get_agg_pending(State),\n            set_agg_pending(Pending - 1, State);\n        true ->\n            State\n    end;\non({fd, _AggId, {fd_notify, _Event, _PID, jump}}, State) ->\n    State;\n\n%% @doc handler for messages from the vivaldi process\n%%     reporting its confidence\non({ganglia_vivaldi_confidence, DHTName, Msg}, State) ->\n    {vivaldi_get_coordinate_response, _, Confidence} = Msg,\n    _ = gmetric(both, lists:flatten(io_lib:format(\"vivaldi_confidence_~s\", [DHTName])), \"float\", Confidence, \"Confidence\"),\n    State.\n\n-spec send_general_metrics() -> ok.\nsend_general_metrics() ->\n    % general erlang status information\n    _ = gmetric(both, \"Erlang Processes\", \"int32\", erlang:system_info(process_count), \"Total Number\"),\n    _ = gmetric(both, \"Memory used by Erlang processes\", \"int32\", erlang:memory(processes_used), \"Bytes\"),\n    _ = gmetric(both, \"Memory used by ETS tables\", \"int32\", erlang:memory(ets), \"Bytes\"),\n    _ = gmetric(both, \"Memory used by atoms\", \"int32\", erlang:memory(atom), \"Bytes\"),\n    _ = gmetric(both, \"Memory used by binaries\", \"int32\", erlang:memory(binary), \"Bytes\"),\n    _ = gmetric(both, \"Memory used by system\", \"int32\", erlang:memory(system), \"Bytes\"),\n    ok.\n\n%%@doc aggregate the number of key-value pairs\n%%     and the amount of memory for this VM\n-spec send_dht_node_metrics(state()) -> state().\nsend_dht_node_metrics(State) ->\n    DHTNodes = pid_groups:find_all(dht_node),\n    AggId = get_agg_id(State),\n    %% Memory Usage of DHT Nodes\n    MemoryUsage = lists:sum([element(2, erlang:process_info(P, memory))\n                             || P <- DHTNodes]),\n    _ = gmetric(both, \"Memory used by dht_nodes\", \"int32\", MemoryUsage, \"Bytes\"),\n    %% Query DHT Nodes for load, let them reply to the on handler ganglia_dht_node_reply\n    %% The FD tells us if a node is down\n    lists:foreach(fun (PID) ->\n                          GlobalPID = comm:make_global(PID),\n                          %% Let the failure detector inform ganglia about crashes of this DHT node\n                          fd:subscribe(?FD_SUBSCR_PID(AggId), [GlobalPID]),\n                          Envelope = comm:reply_as(comm:this(), 4,\n                                                   {ganglia_dht_load_aggregation, GlobalPID, AggId , '_'}),\n                          comm:send_local(PID, {get_node_details, Envelope, [load]})\n                  end, DHTNodes),\n    set_agg_pending(length(DHTNodes), State).\n\n-spec send_message_metrics() -> ok.\nsend_message_metrics() ->\n    {Received, Sent, _Time} = comm_logger:dump(),\n    traverse(received, gb_trees:iterator(Received)),\n    traverse(sent, gb_trees:iterator(Sent)),\n    ok.\n\n%% @doc Sends latency and transactions/s rrd data from the monitor to Ganglia\n-spec send_rrd_metrics() -> ok.\nsend_rrd_metrics() ->\n    case pid_groups:pid_of(clients_group, monitor) of\n        failed -> ok;\n        ClientMonitor ->\n            RRDMetrics = case monitor:get_rrds(ClientMonitor, [{api_tx, 'req_list'}]) of\n                             [{_,_, undefined}] -> [];\n                             [{_, _, RRD}] ->\n                                 case rrd:dump(RRD) of\n                                     [H | _] ->\n                                         {From_, To_, Value} = H,\n                                         Diff_in_s = timer:now_diff(To_, From_) div 1000000,\n                                         {Sum, _Sum2, Count, _Min, _Max, _Hist} = Value,\n                                         AvgPerS = Count / Diff_in_s,\n                                         Avg = Sum / Count,\n                                         [{both, \"tx latency\", \"float\", Avg, \"ms\"},\n                                          {both, \"transactions/s\", \"float\", AvgPerS, \"1/s\"}];\n                                     _ -> []\n                                 end\n                         end,\n            gmetric(RRDMetrics)\n    end,\n    ok.\n\n-spec send_vivaldi_errors() -> ok.\nsend_vivaldi_errors() ->\n    DHTNodeGroups = case pid_groups:groups_with(dht_node) of\n                        failed ->\n                            [];\n                        X -> X\n                    end,\n    lists:foreach(fun (Group) ->\n                          PID = pid_groups:pid_of(Group, gossip),\n                          Envelope = comm:reply_as(comm:this(), 3,\n                                                   {ganglia_vivaldi_confidence, Group, '_'}),\n                          comm:send_local(PID, {cb_msg, {gossip_vivaldi, default}, {get_coordinate, Envelope}})\n                  end, DHTNodeGroups),\n    ok.\n\n\n%%%%%%%%%%%%\n%% Helpers %\n%%%%%%%%%%%%\n\n-spec gmetric(list()) -> ok.\ngmetric(MetricsList) ->\n    lists:foreach(fun({Slope, Metric, Type, Value, Unit}) ->\n                          gmetric(Slope, Metric, Type, Value, Unit)\n                  end, MetricsList).\n\n-spec gmetric(Slope::both | positive, Metric::string(), Type::string(), Value::number(), Unit::string()) -> string().\ngmetric(Slope, Metric, Type, Value, Unit) ->\n    Cmd = lists:flatten(io_lib:format(\"gmetric --slope ~p --name ~p --type ~p --value ~p --units ~p~n\",\n                                      [Slope, Metric, Type, Value, Unit])),\n    Res = os:cmd(Cmd),\n    ?TRACE(\"~s: ~s~n\", [Cmd, Res]),\n    Res.\n\n-spec traverse(received | sent, Iter1::term()) -> ok.\ntraverse(RcvSnd, Iter1) ->\n  case gb_trees:next(Iter1) of\n    none -> ok;\n    {Key, {Bytes, _Count}, Iter2} ->\n          _ = gmetric(positive, lists:flatten(io_lib:format(\"~s ~p\", [RcvSnd, Key])), \"int32\", Bytes, \"Bytes\"),\n          traverse(RcvSnd, Iter2)\n  end.\n\n-spec agg_check_pending(state()) -> state().\nagg_check_pending(State) ->\n    Pending = get_agg_pending(State),\n    if\n        Pending == 0 ->\n            Load = get_agg_load(State),\n            _ = gmetric(both, \"kv pairs\", \"int32\", Load, \"pairs\"),\n            State;\n        true ->\n            State\n    end.\n\n\n%% -spec get_last_active(state()) -> non_neg_integer().\n%% get_last_active(State) ->\n%%     element(1, State).\n\n-spec set_last_active(state()) -> state().\nset_last_active(State) ->\n    {_Megasecs, Secs, _MicroSecs} = os:timestamp(),\n    setelement(1, State, Secs).\n\n-spec get_load_agg(state()) -> load_aggregation().\nget_load_agg(State) ->\n    element(2, State).\n\n-spec set_load_agg(state(), load_aggregation()) -> state().\nset_load_agg(State, LoadAgg) ->\n    setelement(2, State, LoadAgg).\n\n-spec get_agg_id(state()) -> non_neg_integer().\nget_agg_id(State) ->\n    element(1, get_load_agg(State)).\n\n-spec get_agg_load(state()) -> non_neg_integer().\nget_agg_load(State) ->\n    {_AggId, _Pending, Load} = get_load_agg(State),\n    Load.\n\n-spec set_agg_load(non_neg_integer(), state()) -> state().\nset_agg_load(Load, State) ->\n    {AggId, Pending, OldLoad} = get_load_agg(State),\n    set_load_agg(State, {AggId, Pending - 1, OldLoad + Load}).\n\n-spec get_agg_pending(state()) -> non_neg_integer().\nget_agg_pending(State) ->\n    {_AggId, Pending, _Load} = get_load_agg(State),\n    Pending.\n\n-spec set_agg_pending(non_neg_integer(), state()) -> state().\nset_agg_pending(NumPending, State) ->\n    {AggId, _Pending, Load} = get_load_agg(State),\n    set_load_agg(State, {AggId, NumPending, Load}).\n\n-spec inc_agg_id(state()) -> state().\ninc_agg_id(State) ->\n    {AggId, _Pending, _Load} = get_load_agg(State),\n    set_load_agg(State, {AggId + 1, 0, 0}).\n\n%% @doc Checks whether config parameters of the ganglia process exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(ganglia_interval) and\n    config:cfg_is_greater_than(ganglia_interval, 0).\n"
  },
  {
    "path": "src/gen_component.erl",
    "content": "%% @copyright 2007-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Generic component framework. This component and its usage\n%%      are described in more detail in the 'User and Developers Guide'\n%%      which can be found in user-dev-guide/main.pdf and online at\n%%      [http://scalaris.zib.de].\n%% @end\n%% @version $Id$\n-module(gen_component).\n-vsn('$Id$').\n\n-compile({inline, [gc_mod/1, gc_hand/1, gc_set_hand/2,\n                   gc_bps/1, gc_set_bps/2,\n                   gc_bpactive/1, gc_set_bpactive/2,\n                   gc_bpqueue/1, gc_set_bpqueue/2,\n                   gc_bpstepped/1, gc_set_bpstepped/2,\n                   gc_bpstepper/1, gc_set_bpstepper/2,\n                   handle_message/3\n                  ]}).\n\n-include(\"scalaris.hrl\").\n\n%% breakpoint tracing\n%-define(TRACE_BP(X,Y), log:pal(X,Y)).\n-define(TRACE_BP(X,Y), ok).\n%% userdevguide-begin gen_component:trace_bp_steps\n%-define(TRACE_BP_STEPS(X,Y), io:format(X,Y)).     %% output on console\n%-define(TRACE_BP_STEPS(X,Y), log:pal(X,Y)).        %% output even if called by unittest\n%-define(TRACE_BP_STEPS(X,Y), io:format(user,X,Y)). %% clean output even if called by unittest\n-define(TRACE_BP_STEPS(X,Y), ok).\n%% userdevguide-end gen_component:trace_bp_steps\n\n-ifndef(have_callback_support).\n-export([behaviour_info/1]).\n-endif.\n-export([start_link/4, start/4, start/5]).\n-export([kill/1, sleep/2, is_gen_component/1, runnable/1,\n         get_state/1, get_state/2,\n         get_component_state/1, get_component_state/2,\n         change_handler/2, post_op/2]).\n-export([bp_set/3, bp_set_cond/3, bp_set_cond_async/3,\n         bp_del/2, bp_del_async/2]).\n-export([bp_step/1, bp_cont/1, bp_barrier/1]).\n-export([bp_about_to_kill/1,\n         monitor/1, demonitor/1, demonitor/2]).\n\n-export_type([handler/0, option/0]).\n-export_type([bp_name/0]). %% for unittests\n\n-type bp_name() :: atom().\n\n-type bp_msg() ::\n          {'$gen_component', bp, breakpoint, step, pid()}\n        | {'$gen_component', bp, breakpoint, cont}\n        | {'$gen_component', bp, msg_in_bp_waiting, pid()}\n        | {'$gen_component', bp, barrier}\n        | {'$gen_component', bp, bp_set_cond, fun(), bp_name(), pid() | none}\n        | {'$gen_component', bp, bp_set, comm:msg_tag(), bp_name(), pid() | none}\n        | {'$gen_component', bp, bp_del, bp_name(), pid() | none}.\n\n-type gc_msg() ::\n          bp_msg()\n        | {'$gen_component', sleep, pos_integer()}\n        | {'$gen_component', about_to_kill, pos_integer(), pid()}\n        | {'$gen_component', get_state, pid(), Cookie::pos_integer()}\n        | {'$gen_component', get_component_state, pid(), Cookie::pos_integer()}\n        | {'$gen_component', trace_mpath, trace_mpath:passed_state(),\n           From::term(), To::term(), comm:message()}.\n\n-type bp() ::\n        {bp, MsgTag :: comm:msg_tag(), bp_name()}\n      | {bp_cond, Condition :: fun(), bp_name()}\n      | {bp_cond, {module(), atom(), pos_integer()}, bp_name()}.\n\n-type user_state() :: term().\n-type handler() :: fun((comm:message(), user_state()) -> user_state()).\n-type gc_state() ::\n        { module(),\n          handler(),\n          [bp()],             %% registered breakpoints\n          boolean(),          %% bp active?\n          [bp_msg()],         %% queued bp messages\n          boolean(),          %% bp stepped?\n          pid() | unknown,    %% bp stepper\n          [comm:mypid()]      %% exception subscribers\n        }.\n\n%% define macros for the tuple positions to use them also in guards.\n-define(MOD,         1).\n-define(HAND,        2).\n%-define(OPTS,        3).\n%-define(SLOWEST,     4).\n-define(BPS,         3).\n-define(BP_ACTIVE,   4).\n-define(BP_QUEUE,    5).\n-define(BP_STEPPED,  6).\n-define(BP_STEPPER,  7).\n-define(EXC_SUBS,    8).\n\n%% userdevguide-begin gen_component:behaviour\n-ifdef(have_callback_support).\n-callback init(Args::term()) -> user_state().\n-else.\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n    [\n     {init, 1} %% initialize component\n     %% note: can use arbitrary on-handler, but by default on/2 is used:\n     %% {on, 2} %% handle a single message\n     %% on(Msg, UserState) -> NewUserState | unknown_event | kill\n    ];\nbehaviour_info(_Other) -> undefined.\n-endif.\n%% userdevguide-end gen_component:behaviour\n\n%%% API\n-spec kill(Pid::pid() | port() | atom()) -> ok.\nkill(Pid) ->        Pid ! {'$gen_component', kill}, ok.\n-spec sleep(Pid::pid() | port() | atom(), TimeInMs::integer() | infinity) -> ok.\nsleep(Pid, Time) -> Pid ! {'$gen_component', sleep, Time}, ok.\n\n-spec is_gen_component(Pid::pid()) -> boolean().\nis_gen_component(Pid) ->\n    case erlang:process_info(Pid, initial_call) of\n        undefined -> false; % process not alive\n        {initial_call, {Module, Function, _Arity}} ->\n                gen_component =:= Module orelse\n                    start_gen_component =:= Function\n    end.\n\n-spec runnable(Pid::pid()) -> boolean().\nrunnable(Pid) ->\n    {message_queue_len, MQLen} = erlang:process_info(Pid, message_queue_len),\n    MQResult =\n        case MQLen of\n            0 -> false;\n            _ ->\n                %% are there messages which are not gen_component messages?\n                {messages, Msgs} = erlang:process_info(Pid, messages),\n                lists:any(fun(X) -> element(1, X) =/= '$gen_component' end,\n                          Msgs)\n        end,\n    case MQResult of\n        false ->\n            Pid ! {'$gen_component', bp, msg_in_bp_waiting, self()},\n            receive\n                {'$gen_component', bp, msg_in_bp_waiting_response, Runnable} ->\n                    Runnable\n            end;\n        true -> true\n    end.\n\n-spec receive_state_if_alive(\n        Pid::pid(), MsgTag::get_state_response | get_component_state_response,\n        Cookie::pos_integer()) -> term() | failed.\nreceive_state_if_alive(Pid, MsgTag, Cookie) ->\n    case erlang:is_process_alive(Pid) of\n        true ->\n            receive\n                {'$gen_component', MsgTag, Cookie, State} ->\n                    State;\n                {'$gen_component', MsgTag, _OldCookie, _State} ->\n                    % remove unrelated/old incarnations of this message\n                    receive_state_if_alive(Pid, MsgTag, Cookie)\n            after 100 ->\n                receive_state_if_alive(Pid, MsgTag, Cookie)\n            end;\n        _ ->\n            receive\n                {'$gen_component', MsgTag, State} -> State\n            after 0 ->\n                failed\n            end\n    end.\n\n-spec receive_state_if_alive(\n        Pid::pid(), MsgTag::get_state_response | get_component_state_response,\n        Cookie::pos_integer(), Timeout::non_neg_integer()) -> term() | failed.\nreceive_state_if_alive(Pid, MsgTag, Cookie, Timeout) when Timeout >= 0->\n    case erlang:is_process_alive(Pid) of\n        true ->\n            receive\n                {'$gen_component', MsgTag, Cookie, State} ->\n                    State;\n                {'$gen_component', MsgTag, _OldCookie, _State} ->\n                    % remove unrelated/old incarnations of this message\n                    receive_state_if_alive(Pid, MsgTag, Cookie, Timeout)\n            after 100 ->\n                receive_state_if_alive(Pid, MsgTag, Cookie, Timeout - 100)\n            end;\n        _ ->\n            receive\n                {'$gen_component', MsgTag, Cookie, State} -> State\n            after 0 ->\n                failed\n            end\n    end;\nreceive_state_if_alive(_Pid, MsgTag, Cookie, _Timeout) ->\n    % one last try looking into the message box only\n    receive\n        {'$gen_component', MsgTag, Cookie, State} ->\n            State;\n        {'$gen_component', MsgTag, _OldCookie, _State} ->\n            % remove unrelated/old incarnations of this message\n            failed\n    after 0 ->\n        failed\n    end.\n\n-spec get_state(Pid::pid()) -> user_state() | failed.\nget_state(Pid) ->\n    Cookie = randoms:getRandomInt(),\n    Pid ! {'$gen_component', get_state, self(), Cookie},\n    receive_state_if_alive(Pid, get_state_response, Cookie).\n\n-spec get_state(Pid::pid(), Timeout::non_neg_integer()) -> user_state() | failed.\nget_state(Pid, Timeout) ->\n    Cookie = randoms:getRandomInt(),\n    Pid ! {'$gen_component', get_state, self(), Cookie},\n    receive_state_if_alive(Pid, get_state_response, Cookie, Timeout).\n\n-spec get_component_state(Pid::pid()) -> gc_state().\nget_component_state(Pid) ->\n    Cookie = randoms:getRandomInt(),\n    Pid ! {'$gen_component', get_component_state, self(), Cookie},\n    receive_state_if_alive(Pid, get_component_state_response, Cookie).\n\n-spec get_component_state(pid(), Timeout::non_neg_integer()) -> gc_state() | failed.\nget_component_state(Pid, Timeout) ->\n    Cookie = randoms:getRandomInt(),\n    Pid ! {'$gen_component', get_component_state, self(), Cookie},\n    receive_state_if_alive(Pid, get_component_state_response, Cookie, Timeout).\n\n%% @doc change the handler for handling messages\n-spec change_handler(user_state(), Handler::handler())\n        -> {'$gen_component', [{on_handler, Handler::handler()},...], user_state()}.\nchange_handler(UState, Handler) when is_function(Handler, 2) ->\n    {'$gen_component', [{on_handler, Handler}], UState}.\n\n%% @doc perform a post op, i.e. handle a message directly after another\n-spec post_op(comm:message(), user_state())\n        -> {'$gen_component', [{post_op, comm:message()},...], user_state()}.\npost_op(Msg, UState) ->\n    {'$gen_component', [{post_op, Msg}], UState}.\n\n%% requests regarding breakpoint processing\n-spec bp_set(pid(), comm:msg_tag(), bp_name()) -> ok.\nbp_set(Pid, MsgTag, BPName) ->\n    Pid ! Msg = {'$gen_component', bp, bp_set, MsgTag, BPName, self()},\n    receive Msg -> ok end,\n    ok.\n\n%% @doc Module:Function(Message, State, Params) will be evaluated to decide\n%% whether a BP is reached. Params can be used as a payload.\n-spec bp_set_cond(pid(),\n                  Cond::{module(), atom(), 2}\n                      | fun((comm:message(), State::any()) -> boolean()),\n                  bp_name()) -> ok.\nbp_set_cond(Pid, {_Module, _Function, _Params = 2} = Cond, BPName) ->\n    Pid ! Msg = {'$gen_component', bp, bp_set_cond, Cond, BPName, self()},\n    receive Msg -> ok end,\n    ok;\nbp_set_cond(Pid, Cond, BPName) when is_function(Cond, 2) ->\n    Pid ! Msg = {'$gen_component', bp, bp_set_cond, Cond, BPName, self()},\n    receive Msg -> ok end,\n    ok.\n\n-spec bp_set_cond_async(pid(),\n                        Cond::{module(), atom(), 2}\n                            | fun((comm:message(), State::any()) -> boolean()),\n                        bp_name()) -> ok.\nbp_set_cond_async(Pid, {_Module, _Function, _Params = 2} = Cond, BPName) ->\n    Pid ! {'$gen_component', bp, bp_set_cond, Cond, BPName, none},\n    ok;\nbp_set_cond_async(Pid, Cond, BPName) when is_function(Cond, 2) ->\n    Pid ! {'$gen_component', bp, bp_set_cond, Cond, BPName, none},\n    ok.\n\n-spec bp_del(pid(), bp_name()) -> ok.\nbp_del(Pid, BPName) ->\n    Pid ! Msg = {'$gen_component', bp, bp_del, BPName, self()},\n    receive Msg -> ok end,\n    ok.\n\n-spec bp_del_async(pid(), bp_name()) -> ok.\nbp_del_async(Pid, BPName) ->\n    Pid ! {'$gen_component', bp, bp_del, BPName, none},\n    ok.\n\n-spec bp_step(pid()) -> {module(), On::atom(), comm:message()}.\nbp_step(Pid) ->\n    ?TRACE_BP_STEPS(\"Do step ~p ~p~n\", [Pid, catch pid_groups:group_and_name_of(Pid)]),\n    Pid !  {'$gen_component', bp, breakpoint, step, self()},\n    receive {'$gen_component', bp, breakpoint, step_done,\n             _GCPid, Module, On, Msg} ->\n            ?TRACE_BP_STEPS(\"    Handler: ~p:~p/2~n\"\n                            \"*** Handling done.~n\",\n                            [Module, On]),\n            {Module, On, Msg}\n    end.\n\n-spec bp_cont(pid()) -> ok.\nbp_cont(Pid) ->\n    Pid !  {'$gen_component', bp, breakpoint, cont},\n    ok.\n\n%% @doc delay further breakpoint requests until a breakpoint actually occurs\n-spec bp_barrier(pid()) -> ok.\nbp_barrier(Pid) ->\n    Pid ! {'$gen_component', bp, barrier},\n    ok.\n\n%% @doc Brings the given gen_component into a state that is paused in\n%%      preparation of a graceful shutdown of all children of a supervisor.\n%%      Note: A monitor is used to safe-guard the (synchronous) creation of the\n%%      breakpoint in cases of another interfering shutdown process.\n%% @see sup:sup_terminate_childs/1\n-spec bp_about_to_kill(pid()) -> ok.\nbp_about_to_kill(Pid) ->\n    Self = self(),\n    ?DBG_ASSERT(Pid =/= Self),\n    % if infected, remove all monitors, send infected DOWN messages ourself\n    % note: need to get the monitors before we add ours below\n    MonitoredBy =\n        case trace_mpath:infected() of\n            true ->\n                {monitored_by, X} = erlang:process_info(Pid, monitored_by),\n                X;\n            _ ->\n                []\n        end,\n\n    %% about_to_kill handling\n    MonitorRef = erlang:monitor(process, Pid),\n    % suspend gen_component independently from break points being set/active\n    % (60s should be enough for the sub-sequent kill)\n    % see sup:sup_terminate_childs/1\n    Pid ! Msg = {'$gen_component', about_to_kill, 60000, self()},\n    receive\n        Msg -> erlang:demonitor(MonitorRef, [flush]), ok;\n        {'DOWN', MonitorRef, process, Pid, _Info1} -> ok\n    after 2000 ->\n            erlang:demonitor(MonitorRef, [flush]),\n            log:pal(\"Failed to pause ~p (~p) within 2s~n~.2p\",\n                    [Pid, pid_groups:group_and_name_of(Pid),\n                     erlang:process_info(Pid, [registered_name, initial_call, current_function])])\n    end,\n\n    %% continued infected 'DOWN' message handling (after pausing the processes)\n    % TODO: add infected 'EXIT' messages from links?\n    % note: the proto_sched process also sets a monitor but this should not be\n    %       the process to kill since processes do not kill themselves!\n    _ = [begin\n             ?DBG_ASSERT(PidX =/= Self),\n             % we can only handly gen_components here, other monitors cause uncontrolled side effects\n             ?DBG_ASSERT(is_gen_component(PidX)),\n             MonitorX = erlang:monitor(process, PidX),\n             PidX ! {'$gen_component', remove_monitor, Pid, Self},\n             receive\n                 {'$gen_component', monitor, MonRef} when is_reference(MonRef) ->\n                     erlang:demonitor(MonitorX, [flush]),\n                     %log:pal(\"about_to_kill: ~p monitors ~p with ref ~p\", [PidX, Pid, MonRef]),\n                     % send infected DOWN message:\n                     comm:send_local(PidX, {'DOWN', MonRef, process, Pid, about_to_kill}),\n                     ok;\n                 {'$gen_component', monitor, undefined} ->\n                     % not monitoring any more\n                     erlang:demonitor(MonitorX, [flush]),\n                     %log:pal(\"about_to_kill: ~p monitors ~p with ref ~p\", [PidX, Pid, no_ref]),\n                     ok;\n                 {'DOWN', MonitorX, process, PidX, _Info2} ->\n                     % monitoring PID is unavailable\n                     ok\n             after 2000 ->\n                 erlang:demonitor(MonitorX, [flush]),\n                 log:pal(\"Failed to remove uninfected monitor at ~p (~p) for ~p within 2s~n~.2p\",\n                         [PidX, pid_groups:group_and_name_of(PidX), Pid,\n                          erlang:process_info(PidX, [registered_name, initial_call, current_function])])\n             end\n         end || PidX <- MonitoredBy,\n                % exclude already paused pids:\n                erlang:process_info(PidX, priority) =/= {priority, low}],\n\n    ok.\n\n%% @doc Sets an erlang monitor using erlang:monitor/2 in a gen_component\n%%      process and stores info to support killing gen_components with\n%%      trace_mpath/proto_sched.\n%% @see bp_about_to_kill/1\n-spec monitor(comm:erl_local_pid_plain()) -> reference().\nmonitor(Pid) ->\n    ?DBG_ASSERT(is_gen_component(self())),\n    MonRef = erlang:monitor(process, Pid),\n    %log:pal(\"monitor: ~p monitors ~p with ref ~p\", [self(), Pid, MonRef]),\n    % we need the real pid in the process dictionary for bp_about_to_kill/1 to work\n    RealPid = if is_atom(Pid) ->\n                     case whereis(Pid) of\n                         undefined ->\n                             % this process is not alive anymore -> store the\n                             % registered name instead so the check in\n                             % demonitor/2 does not hit\n                             Pid;\n                         X when is_pid(X) ->\n                             X\n                     end;\n                 is_pid(Pid) ->\n                     Pid\n              end,\n    % only a single monitor allowed:,\n    undefined = erlang:put({'$gen_component_monitor_ref', RealPid}, MonRef),\n    undefined = erlang:put({'$gen_component_monitor_pid', MonRef}, RealPid),\n\n    % if the process did not exist yet, convert the DOWN message to an infected message:\n    case trace_mpath:infected() of\n        true ->\n            receive\n                {'DOWN', MonRef, process, _Pid, noproc = _Reason} = Msg ->\n                    comm:send_local(self(), Msg)\n            after 0 -> ok\n            end;\n        _ ->\n            ok\n    end,\n    MonRef.\n\n-spec demonitor(MonitorRef::reference()) -> true.\ndemonitor(MonitorRef) ->\n    ?MODULE:demonitor(MonitorRef, []).\n\n-spec demonitor(MonitorRef::reference(), OptionList::[flush | info]) -> boolean().\ndemonitor(MonitorRef, OptionList) ->\n    ?DBG_ASSERT(is_gen_component(self())),\n    %log:pal(\"demonitor: ~p demonitors ref ~p\", [self(), MonitorRef]),\n    Res = erlang:demonitor(MonitorRef, OptionList),\n    % previous monitor must exist:\n    Pid = erlang:erase({'$gen_component_monitor_pid', MonitorRef}),\n    ?ASSERT2(Pid =/= undefined, Pid),\n    MonitorRef = erlang:erase({'$gen_component_monitor_ref', Pid}),\n    Res.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% generic framework\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% profile\n\n-type spawn_option() ::\n            link | monitor\n          | {priority, Level::normal | high | max}\n          | {fullsweep_after, Number::non_neg_integer()}\n          | {min_heap_size, Size::non_neg_integer()}\n          | {min_bin_vheap_size, VSize::non_neg_integer()}.\n-type option() ::\n            {pid_groups_join_as, pid_groups:groupname(), pid_groups:pidname()}\n          | {pid_groups_join_as, pid_groups:groupname(), {short_lived, pid_groups:pidname()}}\n          | {erlang_register, Name::atom()}\n          | {spawn_opts, [spawn_option()]}\n          | {wait_for_init}.\n\n-spec start_link(module(), handler(), term(), [option()]) -> {ok, pid()}.\nstart_link(Module, Handler, Args, Options) ->\n    case lists:keytake(spawn_opts, 1, Options) of\n        {value, {spawn_opts, SpawnOpt}, Options1} -> ok;\n        false -> Options1 = Options,\n                 SpawnOpt = [],\n                 ok\n    end,\n    Pid = case erlang:function_exported(Module, start_gen_component, 5) of\n              true ->\n                  spawn_opt(Module, start_gen_component,\n                                  [Module, Handler, Args, Options1, self()],\n                                  [link | SpawnOpt]);\n              false ->\n                  log:log(\"[ gen_component ] the module ~p does not provide its own start/5 function (please include gen_component.hrl)\",\n                          [Module]),\n                  spawn_opt(?MODULE, start,\n                                  [Module, Handler, Args, Options1, self()],\n                                  [link | SpawnOpt])\n    end,\n    receive {started, Pid} -> {ok, Pid} end.\n\n-spec start(module(), handler(), term(), [option()]) -> {ok, pid()}.\nstart(Module, Handler, Args, Options) ->\n    Pid = spawn(?MODULE, start, [Module, Handler, Args, Options, self()]),\n    receive {started, Pid} -> {ok, Pid} end.\n\n-spec start(module(), handler(), term(), [option()], pid()) -> no_return() | ok.\nstart(Module, DefaultHandler, Args, Options, Supervisor) ->\n    % note: register globally first (disables the registered name from a\n    %       potential DEBUG_REGISTER below)\n    Registered =\n        case lists:keyfind(erlang_register, 1, Options) of\n            {erlang_register, Name} ->\n                _ = case whereis(Name) of\n                        undefined -> ok;\n                        _ -> catch(erlang:unregister(Name)) %% unittests may leave garbage\n                    end,\n                erlang:register(Name, self()),\n                true;\n            false ->\n                false\n        end,\n    case lists:keyfind(pid_groups_join_as, 1, Options) of\n        {pid_groups_join_as, GroupId, {short_lived, PidName}} ->\n            % if short-lived processes are spawned multiple times, we cannot\n            % create a new atom (for process registration) for each spawn or\n            % we will run out of atoms!\n            % -> no DEBUG_REGISTER!\n            pid_groups:join_as(GroupId, PidName),\n            log:log(info, \"[ gen_component ] ~p started ~p:~p as ~s:~p\",\n                    [Supervisor, self(), Module, GroupId, PidName]),\n            ok;\n        {pid_groups_join_as, GroupId, PidName} ->\n            pid_groups:join_as(GroupId, PidName),\n            log:log(info, \"[ gen_component ] ~p started ~p:~p as ~s:~p\",\n                    [Supervisor, self(), Module, GroupId, PidName]),\n            if Registered -> ok;\n               is_atom(PidName) ->\n                   %% we can give this process a better name for\n                   %% debugging, for example for etop.\n                   ?DEBUG_REGISTER(list_to_atom(pid_groups:group_to_string(GroupId)\n                                               ++ \"-\" ++ atom_to_list(PidName)),\n                                   self()),\n                   ok;\n               true ->\n                   ?DEBUG_REGISTER(list_to_atom(pid_groups:group_to_string(GroupId)\n                                               ++ \"-\" ++ PidName),\n                                   self()),\n                   ok\n            end;\n        false when Registered ->\n            log:log(info, \"[ gen_component ] ~p started ~p:~p\", [Supervisor, self(), Module]);\n        false ->\n            ?DEBUG_REGISTER(list_to_atom(atom_to_list(Module) ++ \"_\" ++ randoms:getRandomString()),\n                            self()),\n            log:log(info, \"[ gen_component ] ~p started ~p:~p\", [Supervisor, self(), Module])\n    end,\n    WaitForInit = lists:member({wait_for_init}, Options),\n    _ = case WaitForInit of\n            true -> ok;\n            false -> Supervisor ! {started, self()}\n        end,\n    {NewUState, GCState} =\n        try\n            T1State = gc_new(Module, DefaultHandler),\n            Handler = case Module:init(Args) of\n                          {'$gen_component', Config, UState} ->\n                              {on_handler, NewHandler} =\n                                  lists:keyfind(on_handler, 1, Config),\n                              NewHandler;\n                          UState ->\n                              DefaultHandler\n                      end,\n            T2State = case lists:keyfind(exception_subscription, 1, Options) of\n                          {exception_subscription, Subs} ->\n                              gc_set_exc_subs(T1State, Subs);\n                          _ ->\n                              T1State\n                      end,\n            {UState, gc_set_hand(T2State, Handler)}\n        catch\n            % If init throws up, send 'started' to the supervisor but exit.\n            % The supervisor will try to restart the process as it is watching\n            % this PID.\n            ?CATCH_CLAUSE_WITH_STACKTRACE(Level, Reason, Stacktrace)\n                log:log(error,\"Error: exception ~p:~p in init of ~p:  ~.0p\",\n                        [Level, Reason, Module, Stacktrace]),\n                erlang:Level(Reason)\n        after\n            case WaitForInit of\n                false -> ok;\n                true -> Supervisor ! {started, self()}, ok\n            end\n        end,\n    loop(NewUState, GCState).\n\n-spec loop(user_state(), gc_state()) -> no_return() | ok.\nloop(UState, GCState) ->\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_in_loop, self()}),\n    receive Msg ->\n%%            try\n                on(Msg, UState, GCState)\n            %% ?CATCH_CLAUSE_WITH_STACKTRACE(Level, Reason, Stacktrace)\n            %%         log:log(error,\n            %%                 \"Error: exception ~p:~p in loop of ~.0p~n \",\n            %%                 [Level, Reason, {UState, GCState}]),\n            %%         on_exception(Msg, Level, Reason, Stacktrace,\n            %%                      UState, GCState)\n            %% end\n    end.\n\n%% @doc Helper for on/3 and on_traced_msg/3 handling gen_component-provided\n%%      default messages as well as user-messages.\n-spec handle_message(comm:message() | comm:group_message(), user_state(), handler())\n        -> user_state() | kill | unknown_event |\n           {'$gen_component', Commands::[{on_handler, Handler::handler()} |\n                                         {post_op, comm:message()}], user_state()}.\nhandle_message({ping, Pid}, UState, _Handler) ->\n    %% generic ping message\n    comm:send(Pid, {pong, pid_groups:my_pidname()}, [{channel, prio}]),\n    UState;\nhandle_message({?send_to_group_member, Processname, Msg}, UState, _Handler) ->\n    %% forward a message to group member by its process name\n    %% initiated via comm:send/3 with group_member\n    comm:forward_to_group_member(Processname, Msg),\n    UState;\nhandle_message({?send_to_registered_proc, Processname, Msg}, UState, _Handler) ->\n    %% forward a message to a registered process\n    %% initiated via comm:send/3 with registered_proc\n    comm:forward_to_registered_proc(Processname, Msg),\n    UState;\nhandle_message(Msg, UState, Handler) ->\n    Handler(Msg, UState).\n\n%%%%%%%%%%%%%%%%%%%%\n%% Attention!:\n%%   we actually manage two queues here:\n%%     user requests and breakpoint requests\n%%   FIFO order has to be maintained for both (separately).\n%%   This is done by a hold back queue for breakpoint messages, while\n%%   not in a breakpoint. A selective receive of breakpoint messages,\n%%   provides further bp instructions when needed inside a bp.\n%%%%%%%%%%%%%%%%%%%%\n-spec on(comm:message() | comm:group_message(), user_state(), gc_state())\n        -> ok.\non(GCMsg, UState, GCState)\n  when is_tuple(GCMsg) andalso '$gen_component' =:= element(1, GCMsg) ->\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_in_on, self(), GCMsg}),\n    on_gc_msg(GCMsg, UState, GCState);\non(Msg, UState, GCState) ->\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_in_on, self(), Msg}),\n    T1GCState = on_bp(Msg, UState, GCState),\n    case T1GCState of\n        {drop_single, T2GCState} ->\n            ?DBG_ASSERT2(not trace_mpath:infected(), {infected_after_on, self(), Msg}),\n            loop(UState, T2GCState);\n        _ ->\n            Module  = gc_mod(T1GCState),\n            Handler = gc_hand(T1GCState),\n            try handle_message(Msg, UState, Handler) of\n                kill ->\n                    log:log(info, \"[ gen_component ] ~.0p killed (~.0p:~.0p/2):\",\n                            [self(), Module, Handler]),\n                    ok;\n                {'$gen_component', [{post_op, Msg1}], NewUState} ->\n                    on_post_op(Msg1, NewUState, T1GCState);\n                {'$gen_component', [{on_handler, NewHandler}], NewUState} ->\n                    %% This is not counted as a bp_step\n                    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_after_on, self(), Msg}),\n                    loop(NewUState, gc_set_hand(T1GCState, NewHandler));\n                {'$gen_component', Commands, _NewUState} ->\n                    %% let's fail since the Config list was either\n                    %% empty or contained an invalid entry\n                    log:log(warn, \"[ gen_component ] unknown command(s): ~.0p\",\n                            [Commands]),\n                    erlang:throw('unknown gen_component command');\n                unknown_event ->\n                    %% drop T2State, as it contains the error message\n                    on_unknown_event(Msg, UState, T1GCState),\n                    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_after_on, self(), Msg}),\n                    case gc_bpactive(T1GCState) andalso gc_bpstepped(T1GCState) of\n                        false -> loop(UState, T1GCState);\n                        true -> loop(UState, bp_step_done(Msg, T1GCState))\n                    end;\n                NewUState ->\n                    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_after_on, self(), Msg}),\n                    case gc_bpactive(T1GCState) andalso gc_bpstepped(T1GCState) of\n                        false -> loop(NewUState, T1GCState);\n                        true -> loop(NewUState, bp_step_done(Msg, T1GCState))\n                    end\n            catch ?CATCH_CLAUSE_WITH_STACKTRACE(Level, Reason, Stacktrace)\n                    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_after_on, self(), Msg}),\n                    case Stacktrace of\n                        %% erlang < R15 : {Module, Handler, [Msg, State]}\n                        %% erlang >= R15: {Module, Handler, [Msg, State], _}\n                        [T | _] when\n                              erlang:element(1, T) =:= Module andalso\n                              %% erlang:element(2, T) =:= Handler andalso\n                              erlang:element(3, T) =:= [Msg, UState] andalso\n                              Reason =:= function_clause andalso\n                              Level =:= error ->\n                            on_unknown_event(Msg, UState, T1GCState),\n                            loop(UState, T1GCState);\n                        _ ->\n                            on_exception(Msg, Level, Reason, Stacktrace,\n                                         UState, T1GCState),\n                            loop(UState, T1GCState)\n                    end\n            end\n    end.\n\n-spec on_traced_msg(comm:message(), user_state(), gc_state())\n                   -> ok.\non_traced_msg(GCMsg, UState, GCState)\n  when is_tuple(GCMsg) andalso '$gen_component' =:= element(1, GCMsg) ->\n    ?DBG_ASSERT2(trace_mpath:infected(), {not_infected_in_traced_msg, self(), GCMsg}),\n    on_gc_msg(GCMsg, UState, GCState);\non_traced_msg(Msg, UState, GCState) ->\n    ?DBG_ASSERT2(trace_mpath:infected(), {not_infected_in_traced_msg, self(), Msg}),\n    T1GCState = on_bp(Msg, UState, GCState),\n    Module  = gc_mod(T1GCState),\n    Handler = gc_hand(T1GCState),\n    try handle_message(Msg, UState, Handler) of\n        kill ->\n            log:log(info, \"[ gen_component ] ~.0p killed (~.0p:~.0p/2):\",\n                    [self(), Module, Handler]),\n            MsgTag = erlang:erase('$gen_component_trace_mpath_msg_tag'),\n            trace_mpath:log_info(self(), {gc_on_done, MsgTag}),\n            trace_mpath:stop(),\n            ok;\n        {'$gen_component', [{post_op, Msg1}], NewUState} ->\n            on_post_op(Msg1, NewUState, T1GCState);\n        {'$gen_component', [{on_handler, NewHandler}], NewUState} ->\n            %% This is not counted as a bp_step\n            MsgTag = erlang:erase('$gen_component_trace_mpath_msg_tag'),\n            trace_mpath:log_info(self(), {gc_on_done, MsgTag}),\n            trace_mpath:stop(),\n            loop(NewUState, gc_set_hand(T1GCState, NewHandler));\n        {'$gen_component', Commands, _NewUState} ->\n            %% let's fail since the Config list was either\n            %% empty or contained an invalid entry\n            log:log(warn, \"[ gen_component ] unknown command(s): ~.0p\",\n                    [Commands]),\n            erlang:throw('unknown gen_component command');\n        unknown_event ->\n            %% drop T2State, as it contains the error message\n            on_unknown_event(Msg, UState, T1GCState),\n            MsgTag = erlang:erase('$gen_component_trace_mpath_msg_tag'),\n            trace_mpath:log_info(self(), {gc_on_done, MsgTag}),\n            trace_mpath:stop(),\n            case gc_bpactive(T1GCState) andalso gc_bpstepped(T1GCState) of\n                false -> loop(UState, T1GCState);\n                true -> loop(UState, bp_step_done(Msg, T1GCState))\n            end;\n        NewUState ->\n            MsgTag = erlang:erase('$gen_component_trace_mpath_msg_tag'),\n            trace_mpath:log_info(self(), {gc_on_done, MsgTag}),\n            trace_mpath:stop(),\n            case gc_bpactive(T1GCState) andalso gc_bpstepped(T1GCState) of\n                false -> loop(NewUState, T1GCState);\n                true -> loop(NewUState, bp_step_done(Msg, T1GCState))\n            end\n    catch ?CATCH_CLAUSE_WITH_STACKTRACE(Level, Reason, Stacktrace)\n              case Stacktrace of\n                  %% erlang < R15 : {Module, Handler, [Msg, State]}\n                  %% erlang >= R15: {Module, Handler, [Msg, State], _}\n                  [T | _] when\n                    erlang:element(1, T) =:= Module andalso\n                        %% erlang:element(2, T) =:= Handler andalso\n                        erlang:element(3, T) =:= [Msg, UState] andalso\n                        Reason =:= function_clause andalso\n                        Level =:= error ->\n                      on_unknown_event(Msg, UState, T1GCState),\n                      MsgTag = erlang:erase('$gen_component_trace_mpath_msg_tag'),\n                      trace_mpath:log_info(self(), {gc_on_done, MsgTag}),\n                      trace_mpath:stop(),\n                      loop(UState, T1GCState);\n                  _ ->\n                      on_exception(Msg, Level, Reason, Stacktrace,\n                                   UState, T1GCState),\n                      MsgTag = erlang:erase('$gen_component_trace_mpath_msg_tag'),\n                      trace_mpath:log_info(self(), {gc_on_done, MsgTag}),\n                      trace_mpath:stop(),\n                      loop(UState, T1GCState)\n              end\n    end.\n\n-spec on_unknown_event(comm:message(), user_state(), gc_state())\n                      -> ok.\non_unknown_event({web_debug_info, Requestor}, UState, GCState) ->\n    comm:send_local(Requestor, {web_debug_info_reply,\n                                [{\"generic info from gen_component:\", \"\"},\n                                 {\"module\", webhelpers:safe_html_string(\"~.0p\", [gc_mod(GCState)])},\n                                 {\"handler\", webhelpers:safe_html_string(\"~.0p\", [gc_hand(GCState)])},\n                                 {\"state\", webhelpers:html_pre(\"~50p\", [UState])}]});\non_unknown_event(UnknownMessage, UState, GCState) ->\n    DbgMsg = \"~n** Unknown message:~n ~.0p~n\"\n               \"** Module:~n ~.0p~n\"\n               \"** Handler:~n ~.0p~n\"\n               \"** Pid:~n ~p ~.0p~n\"\n               \"** State:~n ~.0p~n\",\n    DbgVal = [setelement(1, UnknownMessage, util:extint2atom(element(1, UnknownMessage))),\n              gc_mod(GCState),\n              gc_hand(GCState),\n              self(), catch pid_groups:group_and_name_of(self()),\n              {UState, GCState}],\n    log_or_fail(DbgMsg, DbgVal, unknown_message).\n\non_exception(Msg, Level, Reason, Stacktrace, UState, GCState) ->\n    DbgMsg = \"~n** Exception:~n ~.0p:~.0p~n\"\n               \"** Current message:~n ~.0p~n\"\n               \"** Module:~n ~.0p~n\"\n               \"** Handler:~n ~.0p~n\"\n               \"** Pid:~n ~p ~.0p~n\"\n               \"** Source linetrace (enable in scalaris.hrl):~n ~.0p~n\"\n               \"** State:~n ~.0p~n\"\n               \"** Stacktrace:~n ~.0p~n\",\n    Mod = gc_mod(GCState),\n    DbgVal = [Level, Reason,\n              setelement(1, Msg, util:extint2atom(element(1, Msg))),\n              Mod,\n              gc_hand(GCState),\n              self(), catch pid_groups:group_and_name_of(self()),\n              erlang:get(test_server_loc),\n              {UState, GCState},\n              Stacktrace],\n    inform_exc_subs(gc_exc_subs(GCState), self(), Mod, {Level, Reason, Msg}),\n    log_or_fail(DbgMsg, DbgVal, exception_throw).\n\n-spec inform_exc_subs([comm:mypid()], pid(), atom(), term()) -> ok.\ninform_exc_subs([], _Source, _Mod, _DbgMsg) -> ok;\ninform_exc_subs([Pid | Tail], Source, Mod, DbgMsg) ->\n    comm:send(Pid, {'DOWN', exc_subs, Mod, Source, DbgMsg}),\n    inform_exc_subs(Tail, Source, Mod, DbgMsg).\n\n-spec log_or_fail(DbgMsg::string(), DbgVal::[term()],\n                  Reason::unknown_message | exception_throw) -> ok.\nlog_or_fail(DbgMsg, DbgVal, Reason) ->\n    case util:is_unittest() of\n        true ->\n            % use ct:pal here as logging may have been stopped already\n            ct:pal(DbgMsg, DbgVal),\n            catch tester_global_state:log_last_calls(),\n            ct:abort_current_testcase(Reason);\n        _ ->\n            log:log(error, DbgMsg, DbgVal)\n    end,\n    ok.\n\n-spec on_post_op(comm:message(), user_state(), gc_state())\n                -> ok.\non_post_op(Msg, UState, GCState) ->\n    case gc_bpactive(GCState) of\n        true ->\n            ?TRACE_BP_STEPS(\"~n\"\n                            \"*** Trigger post-op...~n\"\n                            \"    Process: ~p (~p)~n\"\n                            \"    Handler: ~p:~p/2~n\"\n                            \"    Message: ~.0p~n\",\n                            [self(), catch pid_groups:group_and_name_of(self()),\n                             gc_mod(GCState), gc_hand(GCState), Msg]),\n            self() ! {'$gen_component', bp, breakpoint, step,\n                      gc_bpstepper(GCState)},\n            ok;\n        false -> ok\n    end,\n    case erlang:get(trace_mpath) of\n        undefined -> on(Msg, UState, GCState);\n        Logger    -> trace_mpath:log_info(\n                       Logger, self(),\n                       {gc_post_op, trace_mpath:get_msg_tag(Msg)}),\n                     on_traced_msg(Msg, UState, GCState)\n    end.\n\n-spec on_gc_msg(gc_msg(), user_state(), gc_state()) -> ok.\non_gc_msg({'$gen_component', trace_mpath, PState, From, To, Msg}, UState, GCState) ->\n    trace_mpath:log_recv(PState, From, To, Msg),\n    erlang:put('$gen_component_trace_mpath_msg_tag', element(1, Msg)),\n    trace_mpath:start(PState),\n    on_traced_msg(Msg, UState, GCState);\non_gc_msg({'$gen_component', kill}, _UState, GCState) ->\n    log:log(info, \"[ gen_component ] ~.0p killed (~.0p:~.0p/2):\",\n            [self(), gc_mod(GCState), gc_hand(GCState)]),\n    ok;\non_gc_msg({'$gen_component', bp, msg_in_bp_waiting, Pid} = _Msg, UState, GCState) ->\n    Pid ! {'$gen_component', bp, msg_in_bp_waiting_response, false},\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_gc_msg, self(), _Msg}),\n    loop(UState, GCState);\non_gc_msg({'$gen_component', sleep, Time} = _Msg, UState, GCState) ->\n    timer:sleep(Time),\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_gc_msg, self(), _Msg}),\n    loop(UState, GCState);\non_gc_msg({'$gen_component', about_to_kill, Time, Pid} = Msg, UState, GCState) ->\n    comm:send_local(Pid, Msg),\n    process_flag(priority, low),\n    timer:sleep(Time),\n    process_flag(priority, normal),\n    log:log(warn, \"[ ~p ] about_to_kill timeout hit without being killed\", [self()]),\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_gc_msg, self(), Msg}),\n    loop(UState, GCState);\non_gc_msg({'$gen_component', remove_monitor, Pid, Source} = _Msg, UState, GCState) ->\n    MonRef = erlang:get({'$gen_component_monitor_ref', Pid}),\n    erlang:demonitor(MonRef),\n    comm:send_local(Source, {'$gen_component', monitor, MonRef}),\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_gc_msg, self(), _Msg}),\n    loop(UState, GCState);\non_gc_msg({'$gen_component', get_state, Pid, Cookie} = _Msg, UState, GCState) ->\n    comm:send_local(\n      Pid, {'$gen_component', get_state_response, Cookie, UState}),\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_gc_msg, self(), _Msg}),\n    loop(UState, GCState);\non_gc_msg({'$gen_component', get_component_state, Pid, Cookie} = _Msg, UState, GCState) ->\n    comm:send_local(\n      Pid, {'$gen_component', get_component_state_response, Cookie,\n            {gc_mod(GCState), gc_hand(GCState), GCState}}),\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_gc_msg, self(), _Msg}),\n    loop(UState, GCState);\non_gc_msg({'$gen_component', bp, bp_set_cond, Cond, BPName, Pid} = Msg, UState, GCState) ->\n    NewGCState =\n        case gc_bpqueue(GCState) of\n            [] ->\n                case Pid of none -> ok; _ -> Pid ! Msg, ok end,\n                gc_bp_set_cond(GCState, Cond, BPName);\n            _ -> gc_bp_hold_back(GCState, Msg)\n        end,\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_gc_msg, self(), Msg}),\n    loop(UState, NewGCState);\non_gc_msg({'$gen_component', bp, bp_set, MsgTag, BPName, Pid} = Msg, UState, GCState) ->\n    NewGCState =\n        case gc_bpqueue(GCState) of\n            [] ->\n                Pid ! Msg,\n                gc_bp_set(GCState, MsgTag, BPName);\n            _ -> gc_bp_hold_back(GCState, Msg)\n        end,\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_gc_msg, self(), Msg}),\n    loop(UState, NewGCState);\non_gc_msg({'$gen_component', bp, bp_del, BPName, Pid} = Msg, UState, GCState) ->\n    NewGCState =\n        case gc_bpqueue(GCState) of\n            [] ->\n                case Pid of none -> ok; _ -> Pid ! Msg, ok end,\n                gc_bp_del(GCState, BPName);\n            _ -> gc_bp_hold_back(GCState, Msg)\n        end,\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_gc_msg, self(), Msg}),\n    loop(UState, NewGCState);\non_gc_msg({'$gen_component', bp, barrier} = Msg, UState, GCState) ->\n    NewGCState = gc_bp_hold_back(GCState, Msg),\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_gc_msg, self(), Msg}),\n    loop(UState, NewGCState);\n\non_gc_msg({'$gen_component', bp, breakpoint, step, _Stepper} = Msg,\n          UState, GCState) ->\n    NewGCState = gc_bp_hold_back(GCState, Msg),\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_gc_msg, self(), Msg}),\n    loop(UState, NewGCState);\non_gc_msg({'$gen_component', bp, breakpoint, cont} = Msg,\n          UState, GCState) ->\n    NewGCState = gc_bp_hold_back(GCState, Msg),\n    ?DBG_ASSERT2(not trace_mpath:infected(), {infected_gc_msg, self(), Msg}),\n    loop(UState, NewGCState).\n\n\n-spec on_bp(comm:message(), user_state(), gc_state()) ->\n                   gc_state() | {drop_single, gc_state()}.\non_bp(_Msg, _UState, GCState)\n  when (false =:= element(?BP_ACTIVE, GCState))\n       andalso ([] =:= element(?BPS, GCState)) ->\n   GCState;\non_bp(Msg, UState, GCState) ->\n    BPActive = bp_active(Msg, UState, GCState),\n    wait_for_bp_leave(Msg, GCState, BPActive).\n\n-spec bp_active(comm:message(), user_state(), gc_state()) ->\n                       boolean() | drop_single.\nbp_active(_Msg, _UState, GCState)\n  when (false =:= element(?BP_ACTIVE, GCState))\n       andalso ([] =:= element(?BPS, GCState))->\n    false;\nbp_active(Msg, UState, GCState) ->\n    gc_bpactive(GCState)\n        orelse\n        begin\n            [ ThisBP | RemainingBPs ] = gc_bps(GCState),\n            Decision = case ThisBP of\n                           {bp, ThisTag, _BPName} ->\n                               ThisTag =:= comm:get_msg_tag(Msg);\n                           {bp_cond, Cond, _BPName} when is_function(Cond) ->\n                               Cond(Msg, UState);\n                           {bp_cond, {Module, Fun, Params}, _BPName} ->\n                               apply(Module, Fun, [Msg, UState, Params])\n                       end,\n            case Decision of\n                true ->\n                    ?TRACE_BP(\"~p Now in BP ~p via: ~p~n\", [self(), ThisBP, Msg]),\n                    Decision;\n                false ->\n                    bp_active(Msg, UState, gc_set_bps(GCState, RemainingBPs));\n                drop_single ->\n                    ?TRACE_BP(\"~p Now in BP ~p via: ~p~n\", [self(), ThisBP, Msg]),\n                    Decision\n            end\n        end.\n\n-spec wait_for_bp_leave(comm:message(), gc_state(), boolean() | drop_single) ->\n                               gc_state() | {drop_single, gc_state()}.\nwait_for_bp_leave(_Msg, State, _BP_Active = false) -> State;\nwait_for_bp_leave(_Msg, State, _BP_Active = drop_single) -> {drop_single, State};\nwait_for_bp_leave(Msg, State, _BP_Active = true) ->\n    ?TRACE_BP(\"~p In wait for bp leave~n\", [self()]),\n    {Queue, IsFromQueue} =\n        case gc_bpqueue(State) of\n            [] ->\n                %% trigger a selective receive\n                ?TRACE_BP(\"~p wait for bp op by receive...~n\", [self()]),\n                receive\n                    BPMsg when\n                          is_tuple(BPMsg),\n                          '$gen_component' =:= element(1, BPMsg),\n                          bp =:= element(2, BPMsg) ->\n                        ?TRACE_BP(\"~p got bp op by receive ~p.~n\", [self(), BPMsg]),\n                        {[BPMsg], false};\n                    GetCompStateMsg when\n                          is_tuple(GetCompStateMsg),\n                          '$gen_component' =:= element(1, GetCompStateMsg),\n                          get_component_state =:= element(2, GetCompStateMsg) ->\n                        {[GetCompStateMsg], false}\n                end;\n            BPQueue ->\n                ?TRACE_BP(\"~p process queued bp op ~p.~n\",\n                          [self(), hd(BPQueue)]),\n                {BPQueue, true}\n        end,\n    T1State = gc_set_bpqueue(State, tl(Queue)),\n    T2State = gc_set_bpactive(T1State, true),\n    ?TRACE_BP(\"~p Handle bp request in bp ~p ~n\", [self(), hd(Queue)]),\n    on_bp_req_in_bp(Msg, T2State, hd(Queue), IsFromQueue).\n\n-spec on_bp_req_in_bp(comm:message(), gc_state(), bp_msg(), boolean())\n                     -> gc_state().\non_bp_req_in_bp(_Msg, State,\n                {'$gen_component', bp, breakpoint, step, StepperPid},\n                _IsFromQueue) ->\n    ?TRACE_BP_STEPS(\"rec step req in bp ~p~n\", [self()]),\n    T1State = gc_set_bpstepped(State, true),\n    T2State = gc_set_bpstepper(T1State, StepperPid),\n    ?TRACE_BP_STEPS(\"~n\"\n                    \"*** Start handling message...~n\"\n                    \"    Process: ~p (~p)~n\"\n                    \"    Message: ~.0p~n\",\n                    [self(), catch pid_groups:group_and_name_of(self()), _Msg]),\n    T2State;\non_bp_req_in_bp(_Msg, State,\n                {'$gen_component', bp, breakpoint, cont},\n                _IsFromQueue) ->\n    T1State = gc_set_bpstepper(State, unknown),\n    T2State = gc_set_bpstepped(T1State, false),\n    gc_set_bpactive(T2State, false);\non_bp_req_in_bp(Msg, State,\n                {'$gen_component', bp, msg_in_bp_waiting, Pid},\n                _IsFromQueue) ->\n    Pid ! {'$gen_component', bp, msg_in_bp_waiting_response, true},\n    wait_for_bp_leave(Msg, State, true);\non_bp_req_in_bp(Msg, State,\n                {'$gen_component', bp, barrier}, _IsFromQueue) ->\n    %% we are in breakpoint. Consume this bp message\n    wait_for_bp_leave(Msg, State, true);\non_bp_req_in_bp(Msg, State,\n                {'$gen_component', bp, bp_set_cond, Cond, BPName, Pid} = BPMsg,\n                IsFromQueue) ->\n    NextState =\n        case gc_bpqueue(State) of\n            [_H|_T] when not IsFromQueue -> gc_bp_hold_back(State, BPMsg);\n            _ ->\n                case Pid of none -> ok; _ -> Pid ! Msg, ok end,\n                gc_bp_set_cond(State, Cond, BPName)\n        end,\n    wait_for_bp_leave(Msg, NextState, true);\non_bp_req_in_bp(Msg, State,\n                {'$gen_component', bp, bp_set, MsgTag, BPName, Pid} = BPMsg,\n                IsFromQueue) ->\n    NextState =\n        case gc_bpqueue(State) of\n            [_H|_T] when not IsFromQueue -> gc_bp_hold_back(State, BPMsg);\n            _ ->\n                Pid ! BPMsg,\n                gc_bp_set(State, MsgTag, BPName)\n        end,\n    wait_for_bp_leave(Msg, NextState, true);\non_bp_req_in_bp(Msg, State,\n                {'$gen_component', bp, bp_del, BPName, Pid}= BPMsg,\n                IsFromQueue)->\n    NextState =\n        case gc_bpqueue(State) of\n            [_H|_T] when not IsFromQueue -> gc_bp_hold_back(State, BPMsg);\n            _ ->\n                case Pid of none -> ok; _ -> Pid ! BPMsg, ok end,\n                gc_bp_del(State, BPName)\n        end,\n    wait_for_bp_leave(Msg, NextState, true);\non_bp_req_in_bp(Msg, State,\n                {'$gen_component', get_component_state, Pid, Cookie},\n                _IsFromQueue) ->\n    comm:send_local(\n      Pid, {'$gen_component', get_component_state_response, Cookie, State}),\n    wait_for_bp_leave(Msg, State, true);\non_bp_req_in_bp(Msg, State, {'$gen_component', remove_monitor, Pid, Source},\n                _IsFromQueue) ->\n    MonRef = erlang:get({'$gen_component_monitor_ref', Pid}),\n    erlang:demonitor(MonRef),\n    comm:send_local(Source, {'$gen_component', monitor, MonRef}),\n    wait_for_bp_leave(Msg, State, true);\non_bp_req_in_bp(Msg, State,\n                {'$gen_component', about_to_kill, Time, Pid} = SleepMsg,\n                _IsFromQueue) ->\n    comm:send_local(Pid, SleepMsg),\n    process_flag(priority, low),\n    timer:sleep(Time),\n    process_flag(priority, normal),\n    log:log(warn, \"[ ~p ] about_to_kill timeout hit without being killed\", [self()]),\n    wait_for_bp_leave(Msg, State, true).\n\n\n%% @doc release the bp_step function when we executed a bp_step\n-spec bp_step_done(comm:message(), gc_state()) -> gc_state().\nbp_step_done(_, State)\n  when not element(?BP_ACTIVE, State)\n       orelse not element(?BP_STEPPED, State) ->\n    State;\nbp_step_done(Msg, State) ->\n    ?TRACE_BP_STEPS(\"step done ~.0p~n\", [Msg]),\n    comm:send_local(gc_bpstepper(State),\n                    {'$gen_component', bp, breakpoint, step_done,\n                     self(), gc_mod(State), gc_hand(State), Msg}),\n    T1State = gc_set_bpstepper(State, unknown),\n    gc_set_bpstepped(T1State, false).\n\n\n-spec gc_new(module(), handler()) -> gc_state().\ngc_new(Module, Handler) ->\n    {Module, Handler,\n%     Options, _Slowest = 0.0,\n     _BPs = [], _BPActive = false,\n     _BPHoldbackQueue = [], _BPStepped = false,\n     _BPStepperPid = unknown,\n     _ExcSubs = []\n    }.\ngc_mod(State) ->                element(?MOD, State).\ngc_hand(State) ->               element(?HAND, State).\ngc_set_hand(State, Handler) ->  setelement(?HAND, State, Handler).\ngc_exc_subs(State) ->           element(?EXC_SUBS, State).\ngc_set_exc_subs(State, Subs) -> setelement(?EXC_SUBS, State, Subs).\n%% opts\n%% slowest\n-spec gc_bps(gc_state()) -> [bp()].\ngc_bps(State) ->                element(?BPS, State).\n-spec gc_set_bps(gc_state(), [bp()]) -> gc_state().\ngc_set_bps(State, Val) ->       setelement(?BPS, State, Val).\n-spec gc_bpactive(gc_state()) -> boolean().\ngc_bpactive(State) ->           element(?BP_ACTIVE, State).\n-spec gc_set_bpactive(gc_state(), boolean()) -> gc_state().\ngc_set_bpactive(State, Val) ->  setelement(?BP_ACTIVE, State, Val).\n-spec gc_bpqueue(gc_state()) -> [bp_msg()].\ngc_bpqueue(State) ->            element(?BP_QUEUE, State).\n-spec gc_set_bpqueue(gc_state(), [bp_msg()]) -> gc_state().\ngc_set_bpqueue(State, Val) ->   setelement(?BP_QUEUE, State, Val).\n-spec gc_bpstepped(gc_state()) -> boolean().\ngc_bpstepped(State) -> element(?BP_STEPPED, State).\n-spec gc_set_bpstepped(gc_state(), boolean()) -> gc_state().\ngc_set_bpstepped(State, Val) -> setelement(?BP_STEPPED, State, Val).\n-spec gc_bpstepper(gc_state()) -> pid() | 'unknown'.\ngc_bpstepper(State) ->          element(?BP_STEPPER, State).\n-spec gc_set_bpstepper(gc_state(), pid() | 'unknown') -> gc_state().\ngc_set_bpstepper(State, Val) -> setelement(?BP_STEPPER, State, Val).\n\n-spec gc_bp_hold_back(gc_state(), bp_msg()) -> gc_state().\ngc_bp_hold_back(State, Msg) ->\n    ?TRACE_BP(\"Enqueuing bp op ~p -> ~p~n\", [Msg, State]),\n    NewQueue = gc_bpqueue(State) ++ [Msg],\n    gc_set_bpqueue(State, NewQueue).\n\n-spec gc_bp_set_cond(gc_state(), fun(), bp_name()) -> gc_state().\ngc_bp_set_cond(State, Cond, BPName) ->\n    ?TRACE_BP(\"Set bp ~p with name ~p~n\", [Cond, BPName]),\n    NewBPs = [ {bp_cond, Cond, BPName} | gc_bps(State)],\n    gc_set_bps(State, NewBPs).\n\n-spec gc_bp_set(gc_state(),comm:msg_tag(),atom()) -> gc_state().\ngc_bp_set(State, MsgTag, BPName) ->\n    ?TRACE_BP(\"Set bp ~p with name ~p~n\", [MsgTag, BPName]),\n    NewBPs = [ {bp, MsgTag, BPName} | gc_bps(State)],\n    gc_set_bps(State, NewBPs).\n\n-spec gc_bp_del(gc_state(), atom()) -> gc_state().\ngc_bp_del(State, BPName) ->\n    ?TRACE_BP(\"~p Del bp ~p~n\", [self(), BPName]),\n    NewBPs = lists:keydelete(BPName, 3, gc_bps(State)),\n    gc_set_bps(State, NewBPs).\n"
  },
  {
    "path": "src/gossip.erl",
    "content": "%  @copyright 2010-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jens V. Fischer <jensvfischer@gmail.com>\n%%\n%% @doc The behaviour modul (gossip_beh.erl) of the gossiping framework.\n%%\n%%      The framework is designed to allow the implementation of gossip based\n%%      dissemination and gossip based aggregation protocols. Anti-entropy\n%%      gossiping was not considered. The communication scheme used by the\n%%      framework is push-pull gossiping as this offers the best speed of\n%%      convergence. The membership protocol used for the peer selection is\n%%      Cyclon.\n%%\n%%      The gossiping framework comprises three kinds of components:\n%%      <ol>\n%%          <li> The gossiping behaviour (interface) gossip_beh.erl. The\n%%               behaviour defines the contract that allows the callback module\n%%               to be used by the behaviour module. The behaviour defines the\n%%               contract by specifying functions the callback module has to\n%%               implement. </li>\n%%          <li> The callback modules. A callback module implements a concrete\n%%               gossiping protocol by implementing the gossip_beh.erl, i.e. by\n%%               implementing the functions specified in the gossip_beh.erl.\n%%               The callback module provides the protocol specific code.\n%%               For an example callback module see gossip_load.erl.</li>\n%%          <li> The behaviour module gossip.erl (this module). The behaviour\n%%               module provides the generic code of the gossiping  framework.\n%%               It calls the callback functions of the callback modules defined\n%%               in gossip_beh.erl.</li>\n%%      </ol>\n%%\n%%      The relation between behaviour and callback modules is modelled as a\n%%      one-to-many relation. That is to say, the behaviour module is implemented\n%%      as single process (per node) and all the callback module run in the\n%%      context of this single process. This has the advantage of reducing the\n%%      number of spawned processes and allowing for a better grouping of messages.\n%%\n%%      The framework is started as part of the startup procedure of a dht_node.\n%%      The framework maintains a list of callback modules in the CBMODULES macro\n%%      which are started together with the framework. It is also possible to\n%%      individually start and stop callback modules later.\n%%\n%%      The pattern for communication between the behaviour module and a callback\n%%      module is the following: From the behaviour module to a callback module\n%%      communication occurs as a call to a function of the callback module.\n%%      These calls have to return quickly, no long-lasting operations, especially\n%%      no receiving of messages, are allowed. Therefore, the answers to these\n%%      function calls are mainly realised as messages from the respective\n%%      callback module to the behaviour module, not as return values of the\n%%      function calls.\n%%\n%%      == Phases of a Gossiping Operation ==\n%%\n%%      === Prepare-Request Phase ===\n%%\n%%      The  prepare-request phase consists of peer and data selection. The\n%%      selection of the peer is usually managed by the framework. At the beginning\n%%      of every cycle the behaviour module requests a peer from the Cyclon\n%%      module of Scalaris, which is then used for the data exchange. The peer\n%%      selection is governed by the select_node() function: returning\n%%      false causes the behaviour module to handle the peer selection as described.\n%%      Returning true causes the behaviour module to expect a selected_peer\n%%      message with a peer to be used by for the exchange. How many peers are\n%%      contracted for data exchanges every cycle depends on the fanout() config\n%%      function.\n%%\n%%      The selection of the exchange data is dependent on the specific gossiping\n%%      task and therefore done by a callback module. It is initiated by a call\n%%      to select_data(). When called with select_data(), the respective callback\n%%      module has to initiate a selected_data message to the behaviour module,\n%%      containing the selected exchange data. Both peer and data selection are\n%%      initiated in immediate succession through periodical trigger messages,\n%%      so they can run concurrently. When both data and peer are received by\n%%      the behaviour module, a p2p_exch message with the exchange data is sent\n%%      to the peer, that is to say to the gossip behaviour module of the peer.\n%%\n%%      === Prepare-Reply Phase ===\n%%\n%%      Upon receiving a p2p_exch message, a node enters the prepare-reply\n%%      phase and is now in its passive role as responder. This phase is about\n%%      the integration of the received data and the preparation of the reply data.\n%%      Both of these tasks need to be handled by the callback module. The\n%%      behaviour module passes the received data with a call to select_reply_data(QData)\n%%      to the correspondent callback module, which merges the data with its own\n%%      local data and prepares the reply data. The reply data is sent back to\n%%      the behaviour module with a selected_reply_data message. The behaviour\n%%      module then sends the reply data as a  p2p_exch_reply message back to\n%%      the original requester.\n%%\n%%      === Integrate-Reply Phase ===\n%%\n%%      The integrate-reply phase is triggered by a p2p_exch_reply message.\n%%      Every p2p_exch_reply is the response to an earlier p2p_exch (although\n%%      not necessarily to the last p2p_exch request. The p2p_exch_reply contains\n%%      the reply data from the peer, which is passed to the correspondent\n%%      callback module with a call to integrate_data(QData). The callback module\n%%      processes the received data and signals to the behaviour module the\n%%      completion with an integrated_data message. On a conceptual level, a full\n%%      cycle is finished at this point and the behaviour module counts cycles\n%%      by counting the integrated_data messages. Due to the uncertainties\n%%      of message delays and local clock drift it should be clear however, that\n%%      this can only be an approximation. For instance, a new cycle could have\n%%      been started before the reply to the current request has been received\n%%      (phase interleaving) and, respectively, replies from the other cycle could\n%%      be \"wrongly\" counted as finishing the current cycle (cycle interleaving).\n%%\n%%      == Instantiation ==\n%%\n%%      Many of the interactions conducted by the behaviour module are specific\n%%      to a certain callback module. Therefore, all messages and function\n%%      concerning a certain callback module need to identify with which callback\n%%      module the message or call is associated. This is achieved by adding a\n%%      tuple of the module name and an instance id to all those messages and\n%%      calls. While the name would be enough to identify the module, adding the\n%%      instance id allows for multiple instantiation of the same callback module\n%%      by one behaviour module. This tuple of callback module and instance id\n%%      is also used to store information specific to a certain callback module\n%%      in the behaviour module's state.\n%%\n%%\n%%      == Messages to the Callback Modules (cb_msg) ==\n%%\n%%      Messages which shall be passed directly to a callback module need to have\n%%      the form {cb_msg, CModule, Msg}, where CBModule is of type cb_module() and\n%%      Msg is any message the respective callback module handles.\n%%\n%%      Messages in this form are unpacked by the gossip module and only the Msg\n%%      is send to the given CMModule.\n%%\n%%      If a callback module wants to receive a response from another process, it\n%%      should pack its Pid with an envelope of the form\n%%          {PidOfRequestor, e, 3, {cb_msg, CBModule, '_'}}\n%%      with a call to\n%%          EnvPid = comm:reply_as(PidOfRequestor, 3, {cb_msg, CBModule, '_'})\n%%      and use the EnvPid as SourcePid when sending the request, e.g.\n%%          comm:send(Pid, {get_dht_nodes, EnvPid}, [{?quiet}])\n%%\n%%\n%%      == Used abbreviations ==\n%%\n%%         <ul>\n%%            <li> cb: callback module (a module implementing the\n%%                     gossip_beh.erl behaviour)\n%%            </li>\n%%         </ul>\n%%\n%% @version $Id$\n-module(gossip).\n-author('jensvfischer@gmail.com').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n% interaction with gen_component\n-export([init/1, on_inactive/2, on_active/2]).\n\n%API\n-export([start_link/1, activate/1, remove_all_tombstones/0, check_config/0]).\n\n% interaction with the ring maintenance:\n-export([rm_filter_slide_msg/3, rm_send_activation_msg/5, rm_my_range_changed/3, rm_send_new_range/5]).\n\n% testing and debugging\n-export([start_gossip_task/2, stop_gossip_task/1, tester_create_cb_module_names/1]).\n\n-include(\"gen_component.hrl\").\n\n%% -define(PDB, pdb_ets). % easier debugging because state accesible from outside the process\n-define(PDB_OPTIONS, [set]).\n-define(PDB, pdb). % better performance\n\n% prevent warnings in the log\n-define(SEND_TO_GROUP_MEMBER(Pid, Process, Msg),\n        comm:send(Pid, Msg, [{group_member, Process}, {shepherd, self()}])).\n\n%% -define(SHOW, config:read(log_level)).\n-define(SHOW, debug).\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n%% -define(TRACE(X,Y), ok).\n-define(TRACE_TRIGGER(FormatString, Data), ok).\n%% -define(TRACE_TRIGGER(FormatString, Data), log:pal(FormatString, Data)).\n-define(TRACE_ROUND(FormatString, Data), ok).\n%% -define(TRACE_ROUND(FormatString, Data), log:pal(FormatString, Data)).\n\n%% list of callback modules to be activated on startup\n-define(CBMODULES, [{gossip_load, default}, {gossip_cyclon, default}, {gossip_vivaldi, default}]).\n\n% for developement, should be disabled for production\n-define(FIRST_TRIGGER_DELAY, 0). % delay in s for first trigger\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Type Definitions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-type status() :: init | uninit.\n\n-type cb_module_name() :: module().\n-type cb_module_id() :: Id :: atom() | uid:global_uid().\n-type cb_module() :: {Module::cb_module_name(), Id::cb_module_id()}.\n-type cb_status() :: unstarted | started | tombstone.\n-type exch_data() :: { ExchData :: undefined | any(), Peer :: undefined | comm:mypid()}.\n\n-type cb_fun_name() :: handle_msg | integrate_data | notify_change | round_has_converged |\n                       select_data | select_node | select_reply_data | web_debug_info | shutdown.\n\n% state record\n-record(state, {\n        cb_modules = [] :: [cb_module()] ,\n        msg_queue = msg_queue:new() :: msg_queue:msg_queue(),\n        range = intervals:all() :: intervals:interval(),\n        status = uninit  :: init | uninit,\n        trigger_add = [] :: [pos_integer()],\n        trigger_groups = [] :: [{TriggerInterval::pos_integer(), CBModules::[cb_module()]}],\n        trigger_locks = [] :: [{cb_module(), locked | free}],\n        trigger_remove = [] :: [pos_integer()],\n        cb_states = [] :: [{cb_module(), any()}],\n        cb_stati = [] :: [{cb_module(), cb_status()}],\n        cycles = [] :: [{cb_module(), non_neg_integer()}],\n        exch_datas = [] :: [{cb_module(), exch_data()}],\n        reply_peers = [] :: [{Ref::pos_integer(), Pid::comm:mypid()}],\n        rounds= [] :: [{cb_module(), non_neg_integer()}]\n}).\n\n-type state() :: #state{}.\n\n% accepted messages of gossip behaviour module\n-type send_error() :: {send_error, _Pid::comm:mypid(), Msg::message(), Reason::atom()}.\n\n-type bh_message() ::\n    {activate_gossip, Neighbors::nodelist:neighborhood()} |\n    {start_gossip_task, CBModule::cb_module(), Args::list()} |\n    {gossip_trigger, TriggerInterval::pos_integer()} |\n    {trigger_action, TriggerInterval::pos_integer()} |\n    {update_range, NewRange::intervals:interval()} |\n    {web_debug_info, SourcePid::comm:mypid()} |\n    send_error() |\n    {bulkowner, deliver, Id::uid:global_uid(), Range::intervals:interval(),\n        Msg::comm:message(), Parents::[comm:mypid(),...]} |\n    {remove_all_tombstones}.\n\n-type cb_message() ::\n    {selected_data, CBModule::cb_module(), PData::gossip_beh:exch_data()} |\n    {selected_peer, CBModule::cb_module(), CyclonMsg::{cy_cache,\n            RandomNodes::[node:node_type()]} } |\n    {p2p_exch, CBModule::cb_module(), SourcePid::comm:mypid(),\n        PData::gossip_beh:exch_data(), OtherRound::non_neg_integer()} |\n    {selected_reply_data, CBModule::cb_module(), QData::gossip_beh:exch_data(),\n        Ref::pos_integer(), Round::non_neg_integer()} |\n    {p2p_exch_reply, CBModule::cb_module(), SourcePid::comm:mypid(),\n        QData::gossip_beh:exch_data(), OtherRound::non_neg_integer()} |\n    {integrated_data, CBModule::cb_module(), current_round} |\n    {new_round, CBModule::cb_module(), NewRound::non_neg_integer()} |\n    {cb_msg, CBModule::cb_module(), Msg::comm:message()} |\n    {stop_gossip_task, CBModule::cb_module()} |\n    {no_msg}.\n\n-type message() :: bh_message() | cb_message().\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Start the process of the gossip module. <br/>\n%%      Called by sup_dht_node, calls gen_component:start_link to start the process.\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on_inactive/2, [],\n                             [{wait_for_init},\n                              {pid_groups_join_as, DHTNodeGroup, gossip}]).\n\n\n%% @doc Initialises the state of the gossip module. <br/>\n%%      Called by gen_component, results in on_inactive handler.\n-spec init([]) -> state().\ninit([]) ->\n    State = #state{},\n\n    % initialise a base trigger\n    msg_delay:send_trigger(?FIRST_TRIGGER_DELAY,  {gossip_trigger, 1}),\n    State.\n\n\n%% @doc Activate the gossip module. <br/>\n%%      Called by dht_node_join. Activates process (when only node of the system)\n%%      or subscribes to the rm to activate on slide_finished messages. <br/>\n%%      Result of the activation is to switch to the on_active handler.\n-spec activate(Neighbors::nodelist:neighborhood()) -> ok.\nactivate(Neighbors) ->\n    case nodelist:node_range(Neighbors) =:= intervals:all() of\n        true ->\n            % We're the first node covering the whole ring range.\n            % Start gossip right away because it's needed for passive\n            % load balancing when new nodes join the ring.\n            comm:send_local(pid_groups:get_my(gossip), {activate_gossip, Neighbors});\n        _    ->\n            % subscribe to ring maintenance (rm) for {slide_finished, succ} or {slide_finished, pred}\n            rm_loop:subscribe(self(), ?MODULE,\n                              fun gossip:rm_filter_slide_msg/3,\n                              fun gossip:rm_send_activation_msg/5, 1)\n    end.\n\n\n%% @doc Globally removes all tombstones from previously stopped callback modules.\n-spec remove_all_tombstones() -> ok.\nremove_all_tombstones() ->\n    Msg = {?send_to_group_member, gossip, {remove_all_tombstones}},\n    bulkowner:issue_bulk_owner(uid:get_global_uid(), intervals:all(), Msg).\n\n\n%% @doc Checks whether the received notification is a {slide_finished, succ} or\n%%      {slide_finished, pred} msg. Used as filter function for the ring maintanance.\n-spec rm_filter_slide_msg(Neighbors, Neighbors, Reason) -> boolean() when\n                          is_subtype(Neighbors, nodelist:neighborhood()),\n                          is_subtype(Reason, rm_loop:reason()).\nrm_filter_slide_msg(_OldNeighbors, _NewNeighbors, Reason) ->\n        Reason =:= {slide_finished, pred} orelse Reason =:= {slide_finished, succ}.\n\n%% @doc Sends the activation message to the behaviour module (this module)\n%%      Used to subscribe to the ring maintenance for {slide_finished, succ} or\n%%      {slide_finished, pred} msg.\n-spec rm_send_activation_msg(Subscriber, ?MODULE, Neighbours, Neighbours, Reason) -> ok when\n                             is_subtype(Subscriber, pid()),\n                             is_subtype(Neighbours, nodelist:neighborhood()),\n                             is_subtype(Reason, rm_loop:reason()).\nrm_send_activation_msg(_Pid, ?MODULE, _OldNeighbours, NewNeighbours, _Reason) ->\n    %% io:format(\"Pid: ~w. Self: ~w. PidGossip: ~w~n\", [Pid, self(), Pid2]),\n    Pid = pid_groups:get_my(gossip),\n    comm:send_local(Pid, {activate_gossip, NewNeighbours}).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Main Message Loop\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%%-------------------------- on_inactive ---------------------------%%\n\n\n%% @doc Message handler during the startup of the gossip module.\n-spec on_inactive(Msg::message(), State::state()) -> state().\non_inactive({activate_gossip, Neighbors}, State) ->\n    MyRange = nodelist:node_range(Neighbors),\n    State1 = state_set(status, init, State),\n\n    % subscribe to ring maintenance (rm)\n    rm_loop:subscribe(self(), ?MODULE,\n                      fun gossip:rm_my_range_changed/3,\n                      fun gossip:rm_send_new_range/5, inf),\n\n    % set range and notify cb modules about leader state\n    State2 = state_set(range, MyRange, State1),\n    Msg1 = case is_leader(MyRange) of\n        true -> {is_leader, MyRange};\n        false -> {no_leader, MyRange}\n    end,\n    State3 = lists:foldl(fun(CBModule, StateIn) ->\n                                 NewState = cb_notify_change(leader, Msg1, CBModule, StateIn),\n                                 NewState\n                         end, State2, state_get(cb_modules, State2)),\n\n    State4 = init_gossip_tasks(Neighbors, State3),\n    State5 = msg_queue_send(State4),\n\n    % change handler to on_active\n    gen_component:change_handler(State5, fun ?MODULE:on_active/2);\n\n\non_inactive({gossip_trigger, TriggerInterval}=_Msg, State) ->\n    ?TRACE_TRIGGER(\"[ Gossip ] on_inactive: ~w\", [_Msg]),\n    handle_trigger(TriggerInterval, State);\n\n\non_inactive({p2p_exch, _CBModule, SourcePid, _PData, _Round}=Msg, State) ->\n    comm:send(SourcePid, {send_error, comm:this(), Msg, on_inactive}),\n    State;\n\n\non_inactive({p2p_exch_reply, _CBModule, SourcePid, _QData, _Round}=Msg, State) ->\n    comm:send(SourcePid, {send_error, comm:this(), Msg, on_inactive}),\n    State;\n\n%% for debugging\non_inactive(print_state, State) ->\n    log:log(warn, \"~s\", [to_string(State)]),\n    State;\n\n\non_inactive({web_debug_info, _Requestor}=Msg, State) ->\n    msg_queue_add(Msg, State);\n\n\non_inactive({stop_gossip_task, _CBModule}=Msg, State) ->\n    msg_queue_add(Msg, State);\n\n\non_inactive({start_gossip_task, _CBModule, _Args}=Msg, State) ->\n    msg_queue_add(Msg, State);\n\n\non_inactive({remove_all_tombstones}=Msg, State) ->\n    msg_queue_add(Msg, State);\n\n\non_inactive({cb_msg, _CBModule, _msg}=Msg, State) ->\n    msg_queue_add(Msg, State);\n\n%% consume all other message (inluding: trigger messages)\non_inactive(_Msg, State) ->\n    State.\n\n\n%%--------------------------- on_active ----------------------------%%\n\n%% @doc Message handler during the normal operation of the gossip module.\n%% @end\n\n-spec on_active(Msg::message(), State::state()) -> state().\non_active({deactivate_gossip}, _State) ->\n    rm_loop:unsubscribe(self(), ?MODULE),\n    gen_component:change_handler(#state{}, fun ?MODULE:on_inactive/2);\n\n\n%% This message is received from self() from init_gossip_task or through\n%% start_gossip_task()/bulkowner\non_active({start_gossip_task, CBModule, Args}, State) ->\n    CBModules = state_get(cb_modules, State),\n    State1 = case lists:member(CBModule, CBModules) of\n        true ->\n            log:log(warn, \"[ Gossip ] Trying to start an already existing Module: ~w .\"\n                ++ \"Request will be ignored.\", [CBModule]),\n            State;\n        false -> init_gossip_task(CBModule, Args, State)\n    end,\n    State1;\n\n\n%% Trigger message starting a new cycle.\non_active({gossip_trigger, TriggerInterval}=_Msg, State) ->\n    ?TRACE_TRIGGER(\"[ Gossip ]:~w\", [_Msg]),\n    State1 = handle_trigger(TriggerInterval, State),\n    case state_get({trigger_group, TriggerInterval}, State) of\n        false ->\n            State1; %% trigger group does not exist, do nothing (e.g. during startup)\n        CBModules ->\n            [ comm:send_local(self(), {trigger_action, CBModule}) || CBModule <- CBModules ],\n            State1\n    end;\n\n\non_active({trigger_action, CBModule}=_Msg, State) ->\n    State1 = msg_queue_send(State),\n\n    case state_get({trigger_lock, CBModule}, State1) of\n        free ->\n            log:log(debug, \"[ Gossip ] Module ~w got triggered\", [CBModule]),\n            log:log(?SHOW, \"[ Gossip ] Cycle: ~w, Round: ~w\",\n                    [state_get({cycle, CBModule}, State1), state_get({round, CBModule}, State1)]),\n\n            %% set cycle status to active\n            State2 = state_set({trigger_lock, CBModule}, locked, State1),\n\n            %% reset exch_data\n            State3 = state_set({exch_data, CBModule}, {undefined, undefined}, State2),\n\n            %% request node (by the cb module or the bh module)\n            State4 = case cb_select_node(CBModule, State3) of\n                {true, NewState} -> NewState;\n                {false, NewState} -> request_random_node(CBModule), NewState\n            end,\n\n            %% request data\n            State5 = cb_select_data(CBModule, State4),\n            State5;\n        locked -> State1 % ignore trigger when within prepare-request phase\n    end;\n\n\n%% received through the rm on key range changes\non_active({update_range, NewRange}, State) ->\n    State1 = state_set(range, NewRange, State),\n    Msg = case is_leader(NewRange) of\n        true -> {is_leader, NewRange};\n        false -> {no_leader, NewRange}\n    end,\n    Fun = fun (CBModule, StateIn) ->\n                StateOut = cb_notify_change(leader, Msg, CBModule, StateIn),\n                StateOut\n          end,\n    CBModules = state_get(cb_modules, State1),\n    lists:foldl(Fun, State1, CBModules);\n\n\n%% request for debug info\non_active({web_debug_info, Requestor}, State) ->\n    CBModules = state_get(cb_modules, State),\n    Fun = fun(CBModule, {KVListIn, StateIn}) ->\n            {KVListCBModule, NewState} = cb_web_debug_info(CBModule, StateIn),\n            {KVListIn ++ [{\"\",\"\"}] ++ KVListCBModule, NewState}\n          end,\n    {KVListCBModule, State1} = lists:foldr(Fun, {[], State}, CBModules),\n    KVListAll = [{\"\",\"\"}] ++ web_debug_info(State) ++ KVListCBModule,\n    comm:send_local(Requestor, {web_debug_info_reply, KVListAll}),\n    State1;\n\n\n%% received from shepherd, from on_inactive or from rejected messages\non_active({send_error, _Pid, Msg, Reason}=ErrorMsg, State) ->\n    % unpack msg if necessary\n    MsgUnpacked = case Msg of\n        % msg from shepherd\n        {_, ?MODULE, OriginalMsg} -> OriginalMsg;\n        % other send_error msgs, e.g. from on_inactive\n        _Msg -> _Msg\n    end,\n    CBStatus = state_get({cb_status, element(2, MsgUnpacked)}, State),\n    case MsgUnpacked of\n        _ when CBStatus =:= tombstone ->\n            log:log(warn(), \"[ Gossip ] Got ~w msg for tombstoned module ~w. Reason: ~w. Original Msg: ~w\",\n                [element(1, ErrorMsg), element(2, MsgUnpacked), Reason, element(1, Msg)]),\n            State;\n        {p2p_exch, CBModule, _SourcePid, PData, Round} ->\n            log:log(warn(), \"[ Gossip ] p2p_exch from ~w (gossip) to ~w (dht_node)\" ++\n                    \" failed because of ~w\", [_SourcePid, _Pid, Reason]),\n            NewState1 = cb_notify_change(exch_failure, {p2p_exch, PData, Round}, CBModule, State),\n            NewState1;\n        {p2p_exch_reply, CBModule, _SourcePid, QData, Round} ->\n            log:log(warn(), \"[ Gossip ] p2p_exch_reply failed because of ~w\", [Reason]),\n            NewState1 = cb_notify_change(exch_failure, {p2p_exch_reply, QData, Round}, CBModule, State),\n            NewState1;\n        _ ->\n            log:log(?SHOW, \"[ Gossip ] Failed to deliever the Msg ~w because ~w\", [Msg, Reason]),\n            State\n    end;\n\n\n%% unpack bulkowner msg\non_active({bulkowner, deliver, _Id, _Range, Msg, _Parents}, State) ->\n    comm:send_local(self(), Msg),\n    State;\n\n\n%% received through remove_all_tombstones()/bulkowner\non_active({remove_all_tombstones}, State) ->\n    lists:foldl(fun(CBModule, StateIn) -> state_remove({cb_status, CBModule}, StateIn) end,\n                State, get_tombstones(State));\n\n\n%% for debugging\non_active(print_state, State) ->\n    log:log(warn, \"~s\", [to_string(State)]),\n    State;\n\n%% for debugging\non_active({get_state, SourcePid}, State) ->\n    comm:send(SourcePid, State),\n    %% log:log(warn, \"~s\", [to_string(State)]),\n    State;\n\n%% Only messages for callback modules are expected to reach this on_active clause.\n%% they have the form:\n%%   {MsgTag, CBModule, ...}\n%%   element(1, Msg) = MsgTag\n%%   element(2, Msg) = CBModule\non_active(Msg, State) ->\n    State1 = try state_get({cb_status, element(2, Msg)}, State) of\n        tombstone ->\n            log:log(warn(), \"[ Gossip ] Got ~w msg for tombstoned module ~w\",\n                [element(1, Msg), element(2, Msg)]),\n            State;\n        started ->\n            handle_msg(Msg, State);\n        false ->\n            log:log(warn, \"[ Gossip ] Unknown Callback Module: ~w\", [element(2, Msg)]),\n            State\n    catch\n        _:_ -> log:log(warn, \"[ Gossip ] Unknown msg: ~w\", [Msg]),\n               State\n    end,\n    State1.\n\n\n%% This message is received as a response to a get_subset message to the\n%% cyclon process and should contain a list of random nodes.\n-spec handle_msg(Msg::cb_message(), State::state()) -> state().\n% re-request node if node list is empty\nhandle_msg({selected_peer, CBModule, _Msg={cy_cache, []}}, State) ->\n    Delay = cb_config(trigger_interval, CBModule),\n    request_random_node_delayed(Delay, CBModule),\n    State;\n\n\nhandle_msg({selected_peer, CBModule, _Msg={cy_cache, Nodes}}, State) ->\n    log:log(info, \"selected_peer: ~w, ~w\", [CBModule, _Msg]),\n    {_Node, PData} = state_get({exch_data, CBModule}, State),\n    case PData of\n        undefined -> state_set({exch_data, CBModule}, {Nodes, undefined}, State);\n        _ -> start_p2p_exchange(Nodes, PData, CBModule, State)\n    end;\n\n\n%% This message is a reply from a callback module to CBModule:select_data()\nhandle_msg({selected_data, CBModule, PData}, State) ->\n    % check if a peer has been received already\n    {Peer, _PData} = state_get({exch_data, CBModule}, State),\n    case Peer of\n        undefined -> state_set({exch_data, CBModule}, {undefined, PData}, State);\n        _ -> start_p2p_exchange(Peer, PData, CBModule, State)\n    end;\n\n\n%% This message is a request from another peer (i.e. another gossip module) to\n%% exchange data, usually results in CBModule:select_reply_data()\nhandle_msg({p2p_exch, CBModule, SourcePid, PData, OtherRound}=Msg, State) ->\n    log:log(debug, \"[ Gossip ] p2p_exch msg received from ~w. PData: ~w\",[SourcePid, PData]),\n    State1 = state_set({reply_peer, Ref=uid:get_pids_uid()}, SourcePid, State),\n    case check_round(OtherRound, CBModule, State1) of\n        {ok, State2} ->\n            cb_select_reply_data(PData, Ref, OtherRound, Msg, CBModule, State2);\n        {start_new_round, State2} -> % self is leader\n            ?TRACE_ROUND(\"[ Gossip ] Starting a new round in p2p_exch\", []),\n            State3 = cb_notify_change(new_round, state_get({round, CBModule}, State2), CBModule, State2),\n            State4 = cb_select_reply_data(PData, Ref, OtherRound, Msg, CBModule, State3),\n            comm:send(SourcePid, {new_round, CBModule, state_get({round, CBModule}, State4)}),\n            State4;\n        {enter_new_round, State2} ->\n            ?TRACE_ROUND(\"[ Gossip ] Entering a new round in p2p_exch\", []),\n            State3 = cb_notify_change(new_round, state_get({round, CBModule}, State2), CBModule, State2),\n            State4 = cb_select_reply_data(PData, Ref, OtherRound, Msg, CBModule, State3),\n            State4;\n        {propagate_new_round, State2} -> % i.e. MyRound > OtherRound\n            ?TRACE_ROUND(\"[ Gossip ] propagate round in p2p_exch\", []),\n            State3 = cb_select_reply_data(PData, Ref, OtherRound, Msg, CBModule, State2),\n            comm:send(SourcePid, {new_round, CBModule, state_get({round, CBModule}, State3)}),\n            State3\n    end;\n\n\n%% This message is a reply from a callback module to CBModule:select_reply_data()\nhandle_msg({selected_reply_data, CBModule, QData, Ref, Round}, State)->\n    case take_reply_peer(Ref, State) of\n        {none, State1} ->\n            log:log(warn, \"[ Gossip ] Got 'selected_reply_data', but no matching reply peer stored in State.\");\n        {Peer, State1} ->\n            comm:send(Peer, {p2p_exch_reply, CBModule, comm:this(), QData, Round}, [{shepherd, self()}])\n    end,\n    log:log(debug, \"[ Gossip ] selected_reply_data. CBModule: ~w, QData ~w\",\n        [CBModule, QData]),\n    State1;\n\n\n%% This message is a reply from another peer (i.e. another gossip module) to\n%% a p2p_exch request, usually results in CBModule:integrate_data()\nhandle_msg({p2p_exch_reply, CBModule, SourcePid, QData, OtherRound}=Msg, State) ->\n    log:log(debug, \"[ Gossip ] p2p_exch_reply, CBModule: ~w, QData ~w\", [CBModule, QData]),\n    case check_round(OtherRound, CBModule, State) of\n        {ok, State1} ->\n            cb_integrate_data(QData, OtherRound, Msg, CBModule, State1);\n        {start_new_round, State1} -> % self is leader\n            ?TRACE_ROUND(\"[ Gossip ] Starting a new round p2p_exch_reply\", []),\n            State2 = cb_notify_change(new_round, state_get({round, CBModule}, State1), CBModule, State1),\n            State3 = cb_integrate_data(QData, OtherRound, Msg, CBModule, State2),\n            comm:send(SourcePid, {new_round, CBModule, state_get({round, CBModule}, State3)}),\n            State3;\n        {enter_new_round, State1} ->\n            ?TRACE_ROUND(\"[ Gossip ] Entering a new round p2p_exch_reply\", []),\n            State2 = cb_notify_change(new_round, state_get({round, CBModule}, State1), CBModule, State1),\n            cb_integrate_data(QData, OtherRound, Msg, CBModule, State2);\n        {propagate_new_round, State1} -> % i.e. MyRound > OtherRound\n            ?TRACE_ROUND(\"[ Gossip ] propagate round in p2p_exch_reply\", []),\n            comm:send(SourcePid, {new_round, CBModule, state_get({round, CBModule}, State1)}),\n            cb_integrate_data(QData, OtherRound, Msg, CBModule, State1)\n    end;\n\n\n%% This message is a reply from a callback module to CBModule:integrate_data()\n%% Markes the end of a cycle\nhandle_msg({integrated_data, CBModule, cur_round}, State) ->\n    state_update({cycle, CBModule}, fun (X) -> X+1 end, State);\n\n\n% finishing an old round should not affect cycle counter of current round\nhandle_msg({integrated_data, _CBModule, prev_round}, State) ->\n    State;\n\n\n%% pass messages for callback modules to the respective callback module\n%% messages to callback modules need to have the form {cb_msg, CBModule, Msg}.\n%% Use envelopes if necessary.\nhandle_msg({cb_msg, CBModule, Msg}, State) ->\n    cb_handle_msg(Msg, CBModule, State);\n\n\n% round propagation message\nhandle_msg({new_round, CBModule, NewRound}, State) ->\n    MyRound = state_get({round, CBModule}, State),\n    if\n        MyRound < NewRound ->\n            ?TRACE_ROUND(\"[ Gossip ] Entering new round via round propagation message\", []),\n            State1 = cb_notify_change(new_round, NewRound, CBModule, State),\n            State2 = state_set({round, CBModule}, NewRound, State1),\n            state_set({cycle, CBModule}, 0, State2);\n        MyRound =:= NewRound -> % i.e. the round propagation msg was already received\n            ?TRACE_ROUND(\"[ Gossip ] Received propagation msg for round i'm already in\", []),\n            State;\n        MyRound > NewRound ->\n            ?TRACE_ROUND(\"[ Gossip ] MyRound > OtherRound\", []),\n            State\n    end;\n\n\n%% Received through stop_gossip_task/bulkowner\n%% Stops gossip tasks and cleans state of all garbage\n%% sets tombstone to handle possible subsequent request for already stopped tasks\nhandle_msg({stop_gossip_task, CBModule}, State) ->\n    log:log(?SHOW, \"[ Gossip ] Stopping ~w\", [CBModule]),\n    % shutdown callback module\n    State1 = cb_shutdown(CBModule, State),\n\n    % delete callback module dependent entries from state\n    State2 = state_remove_cb(CBModule, State1),\n\n    % remove from list of modules\n    State3 = state_update(cb_modules, fun(Modules) -> lists:delete(CBModule, Modules) end, State2),\n\n    % remove from trigger group\n    Interval = cb_config(trigger_interval, CBModule) div 1000,\n    CBModules = state_get({trigger_group, Interval}, State),\n    NewCBModules = lists:delete(CBModule, CBModules),\n    State4 = case NewCBModules of\n        [] ->\n            NewState = state_set({trigger_group, Interval}, NewCBModules, State3),\n            state_update(trigger_remove, fun (Intervals) -> [Interval|Intervals] end, NewState);\n        _ ->\n            state_set({trigger_group, Interval}, NewCBModules, State3)\n    end,\n\n    % set tombstone\n    State5 = state_set({cb_status, CBModule}, tombstone, State4),\n    State5.\n\n\n%% @doc Renew trigger message and handle adding/removal of triggers.\n%%      To avoid infecting mpath traces with infinite triggers, a basetrigger with\n%%      interval 1s is started during the startup of the process. This trigger is\n%%      uninfected, even if it belongs to a node which was added during infection.\n%%\n%%      To add and remove callback module specific triggers without infecting the triggers,\n%%      the request for the addition/removel of a trigger is saved to the state and\n%%      processed during the handling of the base trigger.\n-spec handle_trigger(TriggerInterval::non_neg_integer(), state()) -> state().\nhandle_trigger(TriggerInterval, State) ->\n    % check for trigger removal\n    State1 = case lists:member(TriggerInterval, state_get(trigger_remove, State)) of\n        true -> % remove trigger by not renewing the trigger\n            ?TRACE_TRIGGER(\"Remove trigger: ~w\", [TriggerInterval]),\n\n            % unconditionally renew basetrigger, even if the last trigger was removed\n            if TriggerInterval =:= 1 ->\n                msg_delay:send_trigger(TriggerInterval, {gossip_trigger, TriggerInterval});\n               TriggerInterval =/= 1 -> ok\n            end,\n\n            state_update(trigger_remove,\n                fun (Triggers) -> lists:delete(TriggerInterval, Triggers) end, State);\n\n        false -> % renew trigger\n            msg_delay:send_trigger(TriggerInterval, {gossip_trigger, TriggerInterval}),\n            State\n    end,\n\n    % check for new new triggers to add\n    case state_get(trigger_add, State1) of\n        [] -> State1;\n        NewTriggerIntervals ->\n            ?TRACE_TRIGGER(\"Add triggers: ~w\", [NewTriggerIntervals]),\n            lists:foreach(fun (NewTriggerInterval) ->\n                                msg_delay:send_trigger(NewTriggerInterval, {gossip_trigger, NewTriggerInterval})\n                          end, NewTriggerIntervals),\n            state_set(trigger_add, [], State1)\n    end.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Msg Exchange with Peer\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% called by either on({selected_data,...}) or on({selected_peer, ...}),\n% depending on which finished first\n-spec start_p2p_exchange(Peers::[node:node_type(),...], PData::gossip_beh:exch_data(),\n    CBModule::cb_module(), State::state()) -> state().\nstart_p2p_exchange(Peers, PData, CBModule, State)  ->\n    SendToPeer = fun(Peer, StateIn) ->\n        case node:is_me(Peer) of\n            false ->\n                %% log:log(warn, \"starting p2p exchange. Peer: ~w~n\",[Peer]),\n                ?SEND_TO_GROUP_MEMBER(\n                        node:pidX(Peer), gossip,\n                        {p2p_exch, CBModule, comm:this(), PData, state_get({round, CBModule}, StateIn)}),\n                state_set({trigger_lock, CBModule}, free, StateIn);\n            true  ->\n                %% todo does this really happen??? cyclon should not have itself in the cache\n                log:log(?SHOW, \"[ Gossip ] Node was ME, requesting new node\"),\n                request_random_node(CBModule),\n                {Peer, Data} = state_get({exch_data, CBModule}, StateIn),\n                state_set({exch_data, CBModule}, {undefined, Data}, StateIn)\n        end\n    end,\n    lists:foldl(SendToPeer, State, Peers).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Interacting with the Callback Modules\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n\n%% called when activating gossip module\n-spec init_gossip_tasks(nodelist:neighborhood(), state()) -> state().\ninit_gossip_tasks(Neighbors, State) ->\n    Init = fun (CBModule, StateIn) ->\n            _StateOut = init_gossip_task(CBModule, [{neighbors, Neighbors}], StateIn)\n          end,\n    lists:foldl(Init, State, ?CBMODULES).\n\n\n%% initialises a gossip task / callback mdoule\n%% called on activation of gossip module or on start_gossip_task message\n-spec init_gossip_task(CBModule::cb_module(), Args::list(), State::state()) -> state().\ninit_gossip_task(CBModule, Args, State) ->\n\n    % initialize CBModule\n    CBState = cb_init([{instance, CBModule} | Args], CBModule),\n    State1 = state_set({cb_state, CBModule}, CBState, State),\n    State2 = lists:foldl(fun({Key, Value}, StateIn) -> state_set({Key, CBModule}, Value, StateIn) end,\n                         State1,\n                         [{cb_status, started}, {cycle, 0}, {exch_data,\n                          {undefined, undefined}}, {round, 0}, {trigger_lock, free}]\n                        ),\n\n    % notify cb module about leader state\n    MyRange = state_get(range, State2),\n    LeaderMsg = case is_leader(MyRange) of\n        true -> {is_leader, MyRange};\n        false -> {no_leader, MyRange}\n    end,\n\n    State3 = cb_notify_change(leader, LeaderMsg, CBModule, State2),\n\n    % configure and add trigger\n    TriggerInterval = cb_config(trigger_interval, CBModule) div 1000,\n    ?TRACE_TRIGGER(\"Initiating Trigger for ~w. Interval: ~w\", [CBModule, TriggerInterval]),\n    {TriggerGroup, State4}  =\n    case state_get({trigger_group, TriggerInterval}, State3) of\n        false ->\n            % create and init new trigger group and request the new trigger\n            NewState1 = state_update(trigger_add, fun (List) -> [TriggerInterval|List] end, State3),\n            {[CBModule], NewState1};\n        OldTriggerGroup ->\n            % add CBModule to existing trigger group\n            {[CBModule|OldTriggerGroup], State3}\n    end,\n    State5 = state_set({trigger_group, TriggerInterval}, TriggerGroup, State4),\n\n    % add CBModule to list of cbmodules\n    state_update(cb_modules, fun(CBModules) -> [CBModule|CBModules] end, State5).\n\n\n%% @doc Calls the config function FunName of the callback module.\n%%      Allowed config functions are:\n%%      <ul>\n%%          <li> fanout: the number of peers contacted per cycle </li>\n%%          <li> min_cycles_per_round: The minimum number of cycles per round </li>\n%%          <li> man_cycles_per_round: The maximum number of cycles per round </li>\n%%          <li> trigger_interval: The time interval in ms after which a new cycle\n%%                  is triggered </li>\n%%      </ul>\n-spec cb_config(FunName, CBModule) -> non_neg_integer() when\n    is_subtype(FunName, fanout | min_cycles_per_round | max_cycles_per_round | trigger_interval),\n    is_subtype(CBModule, cb_module()).\ncb_config(FunName, {ModuleName, _Id}) ->\n    apply(ModuleName, FunName, []).\n\n\n%% @doc Called upon startup and calls CBModule:init().\n%%      It calls the init function with the given arguments. It is usually used\n%%      to initialise the state of the callback module.\n-spec cb_init(Args::list(proplists:property()), cb_module()) -> CBState::any().\ncb_init(Args, {ModuleName, _Id}) ->\n    {ok, CBState} = apply(ModuleName, init, [Args]), CBState.\n\n\n%% @doc Called at the beginning of every cycle and calls CBModule:select_node().\n%%      Should return true, if the peer selection is to be done by behaviour module,\n%%      false otherwise. If false is returned, the behaviour module expects a\n%%      selected_peer message.\n-spec cb_select_node(cb_module(), state()) -> {boolean(), state()}.\ncb_select_node(CBModule, State) ->\n    cb_call(select_node, [], CBModule, State).\n\n\n%% @doc Called at the beginning of a cycle and calls CBModule:select_data().\n%%      The callback module has to select the exchange data to be sent to the\n%%      peer. The exchange data has to be sent back to the gossip module as a\n%%      message of the form {selected_data, Instance, ExchangeData}.\n%%      If 'discard_msg' is returned, the current trigger is ignored.\n%%      (Note: Storing the trigger in the message queue would lead to self-accelerating\n%%      recursion of storing and triggering)\n-spec cb_select_data(cb_module(), state()) -> state().\ncb_select_data(CBModule, State) ->\n    case cb_call(select_data, [], CBModule, State) of\n        {ok, State1} ->\n            State1;\n        {discard_msg, State1} ->\n            state_set({trigger_lock, CBModule}, free, State1)\n    end.\n\n\n\n%% @doc Called upon a p2p_exch message and calls CBModule::select_reply_data().\n%%      Passes the PData from a p2p_exch request to the callback module. The callback\n%%      module has to select the exchange data to be sent to the peer. The Ref is\n%%      used by the behaviour module to identify the request.\n%%      The RoundStatus and Round information can be used for special handling\n%%      of messages from previous rounds.\n%%      The selected reply data is to be sent back to the behaviour module as a\n%%      message of the form {selected_reply_data, Instance, QData, Ref, Round}.\n%%      On certain return values, the reply_peer is removed from the state of the\n%%      gossip module. This is necessary if the callback module will not send an\n%%      selected_reply_data message (because the message is dscarded or sent back directly).\n-spec cb_select_reply_data(PData::gossip_beh:exch_data(), Ref::pos_integer(),\n    Round::non_neg_integer(), Msg::message(), CBModule::cb_module(), State::state()) -> state().\ncb_select_reply_data(PData, Ref, Round, Msg, CBModule, State) ->\n    case cb_call(select_reply_data, [PData, Ref, Round], CBModule, State) of\n        {ok, State1} -> State1;\n        {discard_msg, State1} ->\n            {_Peer, State2} = take_reply_peer(Ref, State1),\n            State2;\n        {retry, State1} ->\n            {_Peer, State2} = take_reply_peer(Ref, State1),\n            msg_queue_add(Msg, State2);\n        {send_back, State1} ->\n            {p2p_exch, _, SourcePid, _, _} = Msg,\n            comm:send(SourcePid, {send_error, comm:this(), Msg, message_rejected}),\n            {_Peer, State2} = take_reply_peer(Ref, State1),\n            State2\n    end.\n\n\n%% @doc Called by the upon a p2p_exch message and calls CBModule:integrate_data().\n%%      Passes the QData from a p2p_exch_reply to the callback module. Upon finishing\n%%      the processing of the data, a message of the form\n%%      {integrated_ data, Instance, RoundStatus} is to be sent to the gossip module.\n-spec cb_integrate_data(QData::gossip_beh:exch_data(), OtherRound::non_neg_integer(),\n                        message(), cb_module(), state()) -> state().\ncb_integrate_data(QData, OtherRound, Msg, CBModule, State) ->\n    {RetVal, State1} = cb_call(integrate_data, [QData, OtherRound], CBModule, State),\n    if RetVal =:= retry ->\n           msg_queue_add(Msg, State1);\n       RetVal =:= send_back ->\n            {p2p_exch_reply,_,SourcePid,_,_} = Msg,\n            comm:send(SourcePid, {send_error, comm:this(), Msg, message_rejected}),\n            State1;\n       RetVal =:= ok orelse RetVal =:= discard_msg ->\n            State1\n    end.\n\n\n%% @doc Called upon messages of the form {cb_msg, CBModule, Msg} and calls\n%%      CBModule:handle_msg().\n%%      Passes the message Msg to the callback module, used to handle messages\n%%      for the callback module.\n-spec cb_handle_msg(comm:message(), cb_module(), state()) -> state().\ncb_handle_msg(Msg, CBModule, State) ->\n    {ok, State1} = cb_call(handle_msg, [Msg], CBModule, State), State1.\n\n\n%% @doc Called upon {web_debug_info} messages and calls CBModule:web_debug_info().\n%%      The callback module has to return debugging infos, to be displayed in the\n%%      Scalaris Web Debug Interface.\n-spec cb_web_debug_info(cb_module(), state()) ->\n    {[{Key::string(), Value::any()}], state()}.\ncb_web_debug_info(CBModule, State) ->\n    cb_call(web_debug_info, [], CBModule, State).\n\n\n%% @doc Called upon every p2p_exch/p2p_exch_reply message and calls\n%%      CBmodule:round_has_converged().\n%%      The callback module should return true if the current round has converged\n%%      to a stable value, false otherwise (refers to gossip based aggregation\n%%      protocols implementing a convergence criterion).\n-spec cb_round_has_converged(cb_module(), state()) -> {boolean(), state()}.\ncb_round_has_converged(CBModule, State) ->\n    cb_call(round_has_converged, [], CBModule, State).\n\n\n%% @doc Called to notify the callback module about certain state changes indepent\n%%      of the standard message loop.\n%%      Used to notify a callback module about\n%%      <ul>\n%%          <li> 'new_round': the starting of a new round </li>\n%%          <li> 'leader': changes in the key range of the node. The MsgTag indicates\n%%                  whether the node is a leader or not, the NewRange is the new\n%%                  key range of the node </li>\n%%          <li> 'exch_failure': a failed message delivery, including exchange\n%%                  Data and Round from the original message </li>\n%%      </ul>\n-spec cb_notify_change(Tag::new_round, Round::non_neg_integer(), cb_module(), state()) -> state();\n    (Tag::leader, {is_leader|no_leader, intervals:interval()}, cb_module(), state()) -> state();\n    (Tag::exch_failure, {MsgTag::atom(), Data::any(), Round::non_neg_integer()}, cb_module(), state()) -> state().\ncb_notify_change(Tag, Notification, CBModule, State) ->\n    {ok, State1} = cb_call(notify_change, [Tag,Notification], CBModule, State),\n    State1.\n\n\n%% @doc Called upon stop_gossip_task(CBModule) and calls CBModule:shutdown().\n%%      It should be the opposite of init() and do any necessary clean up.\n-spec cb_shutdown(cb_module(), state()) -> state().\ncb_shutdown(CBModule, State) ->\n    {ok, State1} = cb_call(shutdown, [], CBModule, State), State1.\n\n\n%% @doc Helper function for the cb_* functions.\n%%      Adds the state of the respective callback module to the Args, calls the\n%%      given function and repacks the returned client state.\n-spec cb_call(cb_fun_name(), list(), cb_module(), state()) -> {ReturnValue::any(), state()}.\ncb_call(FunName, Args, CBModule={ModuleName, _InstanceId}, State) when is_atom(ModuleName) ->\n    Args1 = Args ++ [state_get({cb_state, CBModule}, State)],\n    {ReturnValue, ReturnedCBState} = apply(ModuleName, FunName, Args1),\n    {ReturnValue, state_set({cb_state, CBModule}, ReturnedCBState, State)}.\n\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Requesting Peers\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Sends the local node's cyclon process an enveloped request for a random node.\n%%      on_active({selected_peer, CBModule, {cy_cache, Cache}}, State) will handle the response\n-spec request_random_node(CBModule::cb_module()) -> ok.\nrequest_random_node(CBModule) ->\n    EnvPid = comm:reply_as(self(), 3, {selected_peer, CBModule, '_'}),\n    Fanout = cb_config(fanout, CBModule),\n    comm:send_local(self(), {cb_msg, {gossip_cyclon, default},\n                             {get_subset_rand, Fanout, EnvPid}}).\n\n\n%% Used for rerequesting peers from cyclon when cyclon returned an empty list,\n%% which is usually the case during startup.\n%% The delay prohibits bombarding the cyclon process with requests.\n-spec request_random_node_delayed(Delay::0..4294967295, CBModule::cb_module()) ->\n    ok.\nrequest_random_node_delayed(Delay, CBModule) ->\n    EnvPid = comm:reply_as(self(), 3, {selected_peer, CBModule, '_'}),\n    Fanout = cb_config(fanout, CBModule),\n    msg_delay:send_local(Delay div 1000, self(), {cb_msg, {gossip_cyclon, default},\n                                          {get_subset_rand, Fanout, EnvPid}}).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Round Handling\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% called at every p2p_exch and p2p_exch_reply message\n-spec check_round(OtherRound::non_neg_integer(), CBModule::cb_module(), State::state())\n    -> {ok | start_new_round | enter_new_round | propagate_new_round, state()}.\ncheck_round(OtherRound, CBModule, State) ->\n    MyRound = state_get({round, CBModule}, State),\n    ?TRACE_ROUND(\"[ Gossip ] check_round. CBModule: ~w. MyRound: ~w. OtherRound: ~w\",\n                 [CBModule, MyRound, OtherRound]),\n    Leader = is_leader(state_get(range, State)),\n    case MyRound =:= OtherRound of\n        true when Leader ->\n            case is_end_of_round(CBModule, State) of\n                {true, State1} ->\n                    State2 = state_update({round, CBModule}, fun (X) -> X+1 end, State1),\n                    State3 = state_set({cycle, CBModule}, 0, State2),\n                    {start_new_round, State3};\n                {false, State1} -> {ok, State1}\n            end;\n        true ->\n            {ok, State};\n        false when MyRound < OtherRound ->\n            State1 = state_set({round, CBModule}, OtherRound, State),\n            State2 = state_set({cycle, CBModule}, 0, State1),\n            {enter_new_round, State2};\n        false when MyRound > OtherRound ->\n            {propagate_new_round, State}\n    end.\n\n\n%% checks the convergence of the current round (only called at leader)\n-spec is_end_of_round(CBModule::cb_module(), State::state()) -> {boolean(), state()}.\nis_end_of_round(CBModule, State) ->\n    Cycles = state_get({cycle, CBModule}, State),\n    ?TRACE_ROUND(\"[ Gossip ] check_end_of_round. Cycles: ~w\", [Cycles]),\n    {RoundHasConverged, State1} = cb_round_has_converged(CBModule, State),\n    IsEndOfRound = Cycles >= cb_config(min_cycles_per_round, CBModule) andalso\n        (   ( Cycles >= cb_config(max_cycles_per_round, CBModule)) orelse\n        ( RoundHasConverged ) ),\n    {IsEndOfRound, State1}.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Range/Leader Handling\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Checks whether the node is the current leader.\n-spec is_leader(MyRange::intervals:interval()) -> boolean().\nis_leader(MyRange) ->\n    intervals:in(?RT:hash_key(\"0\"), MyRange).\n\n\n%% @doc Checks whether the node's range has changed, i.e. either the node\n%%      itself or its pred changed.\n-spec rm_my_range_changed(OldNeighbors::nodelist:neighborhood(),\n                          NewNeighbors::nodelist:neighborhood(),\n                          IsSlide::rm_loop:reason()) -> boolean().\nrm_my_range_changed(OldNeighbors, NewNeighbors, _IsSlide) ->\n    nodelist:node(OldNeighbors) =/= nodelist:node(NewNeighbors) orelse\n        nodelist:pred(OldNeighbors) =/= nodelist:pred(NewNeighbors).\n\n\n%% @doc Notifies the node's gossip process of a changed range.\n%%      Used to subscribe to the ring maintenance.\n-spec rm_send_new_range(Subscriber::pid(), Tag::?MODULE,\n                        OldNeighbors::nodelist:neighborhood(),\n                        NewNeighbors::nodelist:neighborhood(),\n                        Reason::rm_loop:reason()) -> ok.\nrm_send_new_range(Pid, ?MODULE, _OldNeighbors, NewNeighbors, _Reason) ->\n    NewRange = nodelist:node_range(NewNeighbors),\n    comm:send_local(Pid, {update_range, NewRange}).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% State: Getters and Setters\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Gets the value for the given key from the given state.\n%%      Allowed keys:\n%%      <ul>\n%%          <li>`cb_modules', all active callback modules,</li>\n%%          <li>`cb_states', the states of all callback modules ,</li>\n%%          <li>`cb_stati', stati of all callback modules ,</li>\n%%          <li>`cycles', the cycles of all callback modules ,</li>\n%%          <li>`exch_datas', the exch_data of all callback modules ,</li>\n%%          <li>`msg_queue', the message queue of the gossip module, </li>\n%%          <li>`range', the key range of the node, </li>\n%%          <li>`reply_peers', all reply_peers, </li>\n%%          <li>`rounds', rounds of all callback modules ,</li>\n%%          <li>`status', the status of the gossip module, </li>\n%%          <li>`trigger_add', triggers to be added on next trigger, </li>\n%%          <li>`trigger_groups', a list of all trigger groups, </li>\n%%          <li>`trigger_groups', a list of all trigger locks, </li>\n%%          <li>`trigger_remove', triggers to be removed on next trigger, </li>\n%%      </ul>\n%%      The entries of the following keys are specified by a {Key, SecondaryKey}\n%%      Tuple, allowing to have more than one entry per key. These are implemented\n%%      as Tuplelists of [{SecondaryKey, Value}] and accessed by the singular of\n%%      the key of the according record field (e.g. \"groups\" -> \"group\").\n%%      <ul>\n%%          <li>`{cb_state, CBModule}', the state of the given callback module, </li>\n%%          <li>`{cb_status, CBModule}', indicates, if `init()' was called on given\n%%              callback module, </li>\n%           <li>`{cycle, CBModule}', the cycle counter for the given callback module, </li>\n%%          <li>`{exch_data, CBModule}', a tuple of the data to exchange and the peer to\n%%                  exchange the data with. </li>\n%%          <li>`{round, CBModule}', the round of the given callback module, </li>\n%%          <li>`{reply_peer, Ref}', the peer to send the g2p_exch_reply to, </li>\n%%          <li>`{trigger_group, TriggerInterval}', the trigger group (i.e. a list\n%%              of callback modules) to the given TriggerInterval, </li>\n%%          <li>`{trigger_lock, CBModule}', locks triggering while within prepare-request\n%%              phase for the given callback module, </li>\n%%      </ul>\n-spec state_get(cb_modules, state()) -> [cb_module()];\n               (cb_states, state()) -> [{cb_module(), any()}];\n               (cb_stati, state()) -> [{cb_module(), cb_status()}];\n               (cycles, state()) -> [{cb_module(), non_neg_integer()}];\n               (exch_datas, state()) -> [{cb_module(), exch_data()}];\n               (msg_queue, state()) -> msg_queue:msg_queue();\n               (range, state()) -> intervals:interval();\n               (reply_peers, state()) -> [{Ref::pos_integer(), Pid::comm:mypid()}];\n               (rounds, state()) -> [{cb_module(), non_neg_integer()}];\n               (status, state()) -> status();\n               (trigger_add, state()) -> [pos_integer()];\n               (trigger_groups, state()) -> [{TriggerInterval::pos_integer(), CBModules::[cb_module()]}];\n               (trigger_locks, state()) -> [{cb_module(), locked | free}];\n               (trigger_remove, state()) -> [pos_integer()];\n               ({cb_state, cb_module()}, state()) -> any();\n               ({cb_status, cb_module()}, state()) -> cb_status() | false;\n               ({cycle, cb_module()}, state()) -> non_neg_integer() | false;\n               ({exch_data, cb_module()}, state()) -> exch_data() | false;\n               ({round, cb_module()}, state()) -> non_neg_integer() | false;\n               ({trigger_group, TriggerInterval::pos_integer()}, state()) -> [cb_module()] | false;\n               ({trigger_lock, cb_module()}, state()) -> locked | free | false.\nstate_get(cb_modules, State=#state{cb_modules=CBModules}) when is_record(State, state) ->\n    CBModules;\nstate_get(cb_states, State=#state{cb_states=CBStates}) when is_record(State, state) ->\n    CBStates;\nstate_get(cb_stati, State=#state{cb_stati=CBStati}) when is_record(State, state) ->\n    CBStati;\nstate_get(cycles, State=#state{cycles=Cycles}) when is_record(State, state) ->\n    Cycles;\nstate_get(exch_datas, State=#state{exch_datas=ExchDatas}) when is_record(State, state) ->\n    ExchDatas;\nstate_get(msg_queue, State=#state{msg_queue=MsgQueue}) when is_record(State, state) ->\n    MsgQueue;\nstate_get(range, State=#state{range=Range}) when is_record(State, state) ->\n    Range;\nstate_get(reply_peers, State=#state{reply_peers=ReplyPeers}) when is_record(State, state) ->\n    ReplyPeers;\nstate_get(rounds, State=#state{rounds=Round}) when is_record(State, state) ->\n    Round;\nstate_get(status, State=#state{status=Status}) when is_record(State, state) ->\n    Status;\nstate_get(trigger_add, State=#state{trigger_add=Triggers}) when is_record(State, state) ->\n    Triggers;\nstate_get(trigger_groups, State=#state{trigger_groups=TriggerGroups}) when is_record(State, state) ->\n    TriggerGroups;\nstate_get(trigger_locks, State=#state{trigger_locks=TriggerLocks}) when is_record(State, state) ->\n    TriggerLocks;\nstate_get(trigger_remove, State=#state{trigger_remove=Triggers}) when is_record(State, state) ->\n    Triggers;\nstate_get({cb_state, CBModule}, State=#state{cb_states=CBStates}) when is_record(State, state) ->\n    state_get_helper(CBModule, CBStates);\nstate_get({cb_status, CBModule}, State=#state{cb_stati=CBStati}) when is_record(State, state) ->\n    state_get_helper(CBModule, CBStati);\nstate_get({cycle, CBModule}, State=#state{cycles=Cycles}) when is_record(State, state) ->\n    state_get_helper(CBModule, Cycles);\nstate_get({exch_data, CBModule}, State=#state{exch_datas=ExchDatas}) when is_record(State, state) ->\n    state_get_helper(CBModule, ExchDatas);\nstate_get({round, CBModule}, State=#state{rounds=Round}) when is_record(State, state) ->\n    state_get_helper(CBModule, Round);\nstate_get({trigger_group, Interval}, State=#state{trigger_groups=TriggerGroups}) when is_record(State, state) ->\n    state_get_helper(Interval, TriggerGroups);\nstate_get({trigger_lock, CBModule}, State=#state{trigger_locks=TriggerLocks}) when is_record(State, state) ->\n    state_get_helper(CBModule, TriggerLocks).\n\n\n%% @doc Helper for state_get, extracts a value to the given (secondary) key from the given Tuplelist\n-spec state_get_helper(pos_integer() | cb_module() , [{cb_module()|pos_integer(), ValueType::any()}]) -> ValueType::any().\nstate_get_helper(Key, TupleList) when is_list(TupleList) ->\n    case lists:keyfind(Key, 1, TupleList) of\n        {Key, Value} -> Value;\n        false -> false\n    end.\n\n\n%% @doc Sets the given value for the given key in the given state.\n%%      For a description of the keys see state_get/2.\n-spec state_set(cb_modules, [cb_module()], state()) -> state();\n               (msg_queue, msg_queue:msg_queue(), state()) -> state();\n               (range, intervals:interval(), state()) -> state();\n               (status, status(), state()) -> state();\n               (trigger_add, [pos_integer()], state()) -> state();\n               (trigger_remove, [pos_integer()], state()) -> state();\n               ({cb_state, cb_module()}, any(), state()) -> state();\n               ({cb_status, cb_module()}, cb_status(), state()) -> state();\n               ({cycle, cb_module()}, non_neg_integer(), state()) -> state();\n               ({exch_data, cb_module()}, exch_data(), state()) -> state();\n               ({reply_peer, Ref::pos_integer()}, comm:mypid(), state()) -> state();\n               ({round, cb_module()}, non_neg_integer(), state()) -> state();\n               ({trigger_group, TriggerInterval::pos_integer()}, [cb_module()], state()) -> state();\n               ({trigger_lock, cb_module()}, locked | free, state()) -> state().\nstate_set({Key, SecondaryKey}, Value, State) when is_record(State, state) ->\n    List = case Key of\n        cb_state -> State#state.cb_states;\n        cb_status -> State#state.cb_stati;\n        cycle -> State#state.cycles;\n        exch_data -> State#state.exch_datas;\n        reply_peer -> State#state.reply_peers;\n        round -> State#state.rounds;\n        trigger_group -> State#state.trigger_groups;\n        trigger_lock -> State#state.trigger_locks\n    end,\n    List1 = lists:keystore(SecondaryKey, 1, List, {SecondaryKey, Value}),\n    case Key of\n        cb_state -> State#state{cb_states=List1};\n        cb_status -> State#state{cb_stati=List1};\n        cycle -> State#state{cycles=List1};\n        exch_data -> State#state{exch_datas=List1};\n        reply_peer -> State#state{reply_peers=List1};\n        round -> State#state{rounds=List1};\n        trigger_group -> State#state{trigger_groups=List1};\n        trigger_lock -> State#state{trigger_locks=List1}\n    end;\n\nstate_set(Key, Value, State) when is_record(State, state) ->\n    case Key of\n        cb_modules -> State#state{cb_modules = Value};\n        msg_queue -> State#state{msg_queue = Value};\n        range -> State#state{range = Value};\n        status -> State#state{status = Value};\n        trigger_add -> State#state{trigger_add = Value};\n        trigger_remove -> State#state{trigger_remove = Value}\n    end.\n\n%% @doc Remove callback module dependent entries from state.\n-spec state_remove_cb(cb_module(), state()) -> state().\nstate_remove_cb(CBModule, State) when is_record(State, state) ->\n    State1 = State#state{cb_states=lists:keydelete(CBModule, 1, State#state.cb_states)},\n    State2 = State1#state{cb_stati=lists:keydelete(CBModule, 1, State1#state.cb_stati)},\n    State3 = State2#state{cycles=lists:keydelete(CBModule, 1, State2#state.cycles)},\n    State4 = State3#state{exch_datas=lists:keydelete(CBModule, 1, State3#state.exch_datas)},\n    State5 = State4#state{rounds=lists:keydelete(CBModule, 1, State4#state.rounds)},\n    State5#state{trigger_locks=lists:keydelete(CBModule, 1, State5#state.trigger_locks)}.\n\n\n%% @doc Remove the entry of the given key.\n-spec state_remove(Key::{cb_status, SecondaryKey::cb_module()}, state()) -> state().\nstate_remove({cb_status, CBModule}, #state{cb_stati=CBStati}=State) when is_record(State, state) ->\n    CBStati1 = lists:keydelete(CBModule, 1, CBStati),\n    State#state{cb_stati=CBStati1}.\n\n\n%% @doc Updates an entry with the given update function\n%%      For a description of the keys see state_get/2.\n-spec state_update(cb_modules, UpdateFun::fun(), state()) -> state();\n                  (msg_queue, UpdateFun::fun(), state()) -> state();\n                  (range, UpdateFun::fun(), state()) -> state();\n                  (status, UpdateFun::fun(), state()) -> state();\n                  (trigger_add, UpdateFun::fun(), state()) -> state();\n                  (trigger_remove, UpdateFun::fun(), state()) -> state();\n                  ({cycle, cb_module()}, UpdateFun::fun(), state()) -> state();\n                  ({reply_peer, Ref::pos_integer()}, UpdateFun::fun(), state()) -> state();\n                  ({round, cb_module()}, UpdateFun::fun(), state()) -> state();\n                  ({trigger_group, TriggerInterval::pos_integer()}, UpdateFun::fun(), state()) -> state();\n                  ({cb_state, cb_module()}, UpdateFun::fun(), state()) -> state().\nstate_update(Key, Fun, State) when is_record(State, state) ->\n    NewValue = apply(Fun, [state_get(Key, State)]),\n    state_set(Key, NewValue, State).\n\n\n%% @doc Retrieve and remove the reply peer given by the Ref from the state.\n-spec take_reply_peer(Ref::pos_integer(), State::state()) -> {comm:mypid()|none, state()}.\ntake_reply_peer(Ref, #state{reply_peers=Peers}=State) when is_record(State, state) ->\n    case lists:keytake(Ref, 1, Peers) of\n        false -> {none, State};\n        {value, {Ref, Peer}, Rest} -> {Peer, State#state{reply_peers=Rest}}\n    end.\n\n\n%% @doc Gets all the tombstones from the state of the gossip module.\n-spec get_tombstones(State::state()) -> [cb_module()].\nget_tombstones(State) ->\n    [CBModule || {CBModule, tombstone} <- state_get(cb_stati, State)].\n\n\n\n%%------------------------- Message Queue --------------------------%%\n\n%% add to message queue and create message queue if necessary\n-spec msg_queue_add(Msg::message(), State::state()) -> state().\nmsg_queue_add(Msg, State) ->\n    MsgQueue1 = state_get(msg_queue, State),\n    MsgQueue2 = msg_queue:add(MsgQueue1, Msg),\n    state_set(msg_queue, MsgQueue2, State).\n\n\n%% send the messages from the current message queue and create a new message queue\n-spec msg_queue_send(State::state()) -> state().\nmsg_queue_send(State) ->\n    msg_queue:send(state_get(msg_queue, State)),\n    state_set(msg_queue, msg_queue:new(), State).\n\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Misc\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% provide some debug information for the gossip moudle (to be added to the\n%% information of a the callback modules)\n-spec web_debug_info(State::state()) -> [{any(), any()}, ...].\nweb_debug_info(State) ->\n    CBModules = state_get(cb_modules, State),\n    Tombstones = get_tombstones(State),\n    _KeyValueList =\n        [{\"behaviour module\",   \"\"},\n         {\"msg_queue_len\",      length(state_get(msg_queue, State))},\n         {\"status\",             state_get(status, State)},\n         {\"registered modules\", to_string(CBModules)},\n         {\"trigger groups\", to_string(state_get(trigger_groups, State))},\n         {\"cycles\", to_string(state_get(cycles, State))},\n         {\"tombstones\",         to_string(Tombstones)}\n     ].\n\n\n%% @doc String conversion for sane outputs and debugging\n-spec to_string(list()|state()) -> string().\n%% Returns a list as string\nto_string(List) when is_list(List) ->\n    lists:flatten(io_lib:format(\"~w\", [List]));\n\n%% returns the state as string (comment/uncomment as necessary to control output)\nto_string(State) when is_record(State, state) ->\n    _CBModules = state_get(cb_modules, State),\n    _CBStates = state_get(cb_states, State),\n    RawMsgQueue = state_get(msg_queue, State),\n    _Range = state_get(range, State),\n    _Status = state_get(status, State),\n    _TriggerAdd = state_get(trigger_add, State),\n    _TriggerGroups = state_get(trigger_groups, State),\n    _TriggerLocks = state_get(trigger_locks, State),\n    _TriggerRemove = state_get(trigger_remove, State),\n    _CBStati = state_get(cb_stati, State),\n    _Cycles = state_get(cycles, State),\n    _ExchDatas = state_get(exch_datas, State),\n    _ReplyPeers = state_get(reply_peers, State),\n    _Rounds = state_get(rounds, State),\n    %% _MsgQueue = RawMsgQueue,\n    _MsgQueue = lists:map(fun(Tuple) -> {element(1, Tuple), '...'} end, RawMsgQueue),\n    Str =\n        io_lib:format(\"State: ~n\", []) ++\n        io_lib:format(\"\\tCBModules: ~w~n\", [_CBModules]) ++\n        io_lib:format(\"\\tMsgQueue: ~w~n\", [_MsgQueue]) ++\n        io_lib:format(\"\\tRange: ~w~n\", [_Range]) ++\n        io_lib:format(\"\\tStatus: ~w~n\", [_Status]) ++\n        io_lib:format(\"\\tTriggerAdd: ~w~n\", [_TriggerAdd]) ++\n        io_lib:format(\"\\tTriggerRemove: ~w~n\", [_TriggerRemove]) ++\n        io_lib:format(\"\\tReplyPeers: ~w~n\", [_ReplyPeers]) ++\n        io_lib:format(\"\\tTriggerGroups: ~w~n\", [_TriggerGroups]) ++\n        io_lib:format(\"\\tTriggerLocks: ~w~n\", [_TriggerLocks]) ++\n        io_lib:format(\"\\tCBStati: ~w~n\", [_CBStati]) ++\n        io_lib:format(\"\\tCycles: ~w~n\", [_Cycles]) ++\n        io_lib:format(\"\\tRounds: ~w~n\", [_Rounds]) ++\n        %% io_lib:format(\"\\tExchDatas: ~w~n\", [_ExchDatas]) ++\n        %% io_lib:format(\"\\tCBStates: ~w~n\", [_CBStates]) ++\n        io_lib:format(\"\", []),\n    lists:flatten(Str).\n\n\n%% @doc Check the config of the gossip module. <br/>\n%%      Calls the check_config functions of all callback modules.\n-spec check_config() -> boolean().\ncheck_config() ->\n    lists:foldl(fun({Module, _Args}, Acc) -> Acc andalso Module:check_config() end, true, ?CBMODULES).\n\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Testing and Debugging\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n\n%% @doc Globally starts a gossip task identified by CBModule. <br/>\n%%      Args is passed to the init function of the callback module. <br/>\n%%      CBModule is either the name of a callback module or an name-instance_id\n%%      tuple.\n-spec start_gossip_task(GossipTask, Args) -> ok when\n    is_subtype(GossipTask, cb_module_name() | cb_module()),\n    is_subtype(Args, list(proplists:property())).\nstart_gossip_task(ModuleName, Args) when is_atom(ModuleName) ->\n    Id = uid:get_global_uid(),\n    start_gossip_task({ModuleName, Id}, Args);\n\nstart_gossip_task({ModuleName, Id}, Args) when is_atom(ModuleName) ->\n    Msg = {?send_to_group_member, gossip,\n                {start_gossip_task, {ModuleName, Id}, Args}},\n    bulkowner:issue_bulk_owner(uid:get_global_uid(), intervals:all(), Msg).\n\n\n%% @doc Globally stop a gossip task.\n-spec stop_gossip_task(CBModule::cb_module()) -> ok.\nstop_gossip_task(CBModule) ->\n    Msg = {?send_to_group_member, gossip, {stop_gossip_task, CBModule}},\n    bulkowner:issue_bulk_owner(uid:get_global_uid(), intervals:all(), Msg).\n\n\n%% hack to be able to suppress warnings when testing via config:write()\n-spec warn() -> log:log_level().\nwarn() ->\n    case config:read(gossip_log_level_warn) of\n        failed -> info;\n        Level -> Level\n    end.\n\n\n%% hack to be able to suppress warnings when testing via config:write()\n-compile({nowarn_unused_function, {error, 0}}).\n-spec error() -> log:log_level().\nerror() ->\n    case config:read(gossip_log_level_error) of\n        failed -> warn;\n        Level -> Level\n    end.\n\n\n%% @doc Value creater for type_check_SUITE.\n-spec tester_create_cb_module_names(1) -> cb_module_name().\ntester_create_cb_module_names(1) ->\n    gossip_load.\n\n\n-compile({nowarn_unused_function, {init_gossip_task_feeder, 3}}).\n-spec init_gossip_task_feeder(cb_module(), NoOfBuckets::gossip_load:histogram_size(), state())\n        -> {cb_module(), list(), state()}.\ninit_gossip_task_feeder(CBModule, NoOfBuckets, State) ->\n    %% note: gossip_load (the only supported cb_module for now) requires an\n    %%       integer NoOfBuckets parameter\n    %%       (other modules may have different parameters)\n    {CBModule, [NoOfBuckets], State}.\n"
  },
  {
    "path": "src/gossip_beh.erl",
    "content": "%  @copyright 2008-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jens V. Fischer <jensvfischer@gmail.com>\n%% @doc    Gossiping behaviour\n%% @end\n%% @version $Id$\n-module(gossip_beh).\n-author('jensvfischer@gmail.com').\n-vsn('$Id$').\n\n% for behaviour\n-ifndef(have_callback_support).\n-export([behaviour_info/1]).\n-endif.\n\n-export_type([exch_data/0, round_status/0, instance/0]).\n\n%% cb: callback\n-type cb_state() :: any().\n-type instance() :: {CBModule::module(), InstanceId:: atom() | uid:global_uid()}.\n-type exch_data() :: any().\n-type round_status() :: 'current_round' | 'old_round'.\n\n% Erlang version >= R15B\n-ifdef(have_callback_support).\n\n-type round() :: non_neg_integer().\n-type notify_tag() :: new_round | leader | exch_failure.\n-type notify_msg() :: {is_leader|no_leader, NewRange::intervals:interval()} |\n        {new_round, round()} | {MsgTag::p2p_exch | p2p_exch_reply, exch_data(), round()}.\n\n% Startup & Shutdown\n\n-callback init([proplists:property()]) -> {ok, cb_state()}.\n-callback shutdown(State::cb_state()) -> {ok, shutdown}.\n\n% Gossiping Message Loop\n\n-callback select_node(State::cb_state()) -> {boolean(), cb_state()}.\n\n-callback select_data(State::cb_state()) ->\n    % has to initiate {selected_data, CBModule::module(), PData::exch_data()}\n    {ok | discard_msg, cb_state()}.\n\n-callback select_reply_data(PData::exch_data(), Ref::pos_integer(), Round::round(), State::cb_state()) ->\n    % has to initiate {selected_reply_data, CBModule::module(), QData::exch_data()}\n    {ok | discard_msg | retry | send_back, cb_state()}.\n\n-callback integrate_data(Data::exch_data(), Round::round(), State::cb_state()) ->\n    % has to initiate {integrated_data, CBModule::module()}\n    {ok | discard_msg | retry | send_back, cb_state()}.\n\n-callback handle_msg(Message::comm:message(), State::cb_state()) ->\n    {ok, cb_state()}.\n\n% Config and Misc\n\n-callback fanout() -> pos_integer().\n\n-callback trigger_interval() -> pos_integer().\n\n-callback min_cycles_per_round() -> non_neg_integer() | infinity.\n\n-callback max_cycles_per_round() -> pos_integer() | infinity.\n\n-callback round_has_converged(State::cb_state()) -> {boolean(), cb_state()}.\n\n-callback notify_change(notify_tag(), notify_msg(), State::cb_state()) ->\n    {ok, cb_state()}.\n\n-callback web_debug_info(State::cb_state()) ->\n    {KeyValueList::[{Key::string(), Value::any()},...], cb_state()}.\n\n-callback check_config() -> boolean().\n\n% Erlang version < R15B\n-else.\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n  [ {init, 1},\n    {shutdown, 1},\n    {select_node, 1},\n    {select_data, 1},\n    {select_reply_data, 4},\n    {integrate_data, 3},\n    {handle_msg, 2},\n    {trigger_interval, 0},\n    {fanout, 0},\n    {min_cycles_per_round, 0},\n    {max_cycles_per_round, 0},\n    {round_has_converged, 1},\n    {notify_change, 3},\n    {web_debug_info,1},\n    {check_config, 0}\n  ];\nbehaviour_info(_Other) ->\n  undefined.\n\n-endif.\n"
  },
  {
    "path": "src/gossip_cyclon.erl",
    "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%  @copyright 2008-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Christian Hennig <hennig@zib.de>\n%% @author Jens V. Fischer <jensvfischer@gmail.com>\n%% @doc Gossip based membership management using CYCLON.\n%%\n%% CYCLON provides an unstructured overlay which can be used to obtain the address\n%% of a random node from the entirety of all nodes. This is useful for instance for\n%% gossiping algorithms which need a random peers to communicate with.\n%%\n%% The basic idea is as follows: Every node maintains cache of known peers. At the\n%% beginning of every cycle a random peer is chosen from the cache and a random\n%% subset of the node's neighbours is exchanged with that peer. The receiving\n%% peer uses the subset to update its own cache and also sends back a subset of\n%% its cache, which is merged with the cache of the initial peer.\n%% For details refer to the given paper.\n%% @end\n%%\n%% @reference S. Voulgaris, D. Gavidia, M. van Steen. CYCLON:\n%% Inexpensive Membership Management for Unstructured P2P Overlays.\n%% Journal of Network and Systems Management, Vol. 13, No. 2, June 2005.\n%%\n%% @version $Id$\n\n-module(gossip_cyclon).\n-author('hennig@zib.de').\n-author('jensvfischer@gmail.com').\n-behaviour(gossip_beh).\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n% gossip_beh\n-export([init/1, check_config/0, trigger_interval/0, fanout/0,\n        select_node/1, select_data/1, select_reply_data/4, integrate_data/3,\n        handle_msg/2, notify_change/3, min_cycles_per_round/0, max_cycles_per_round/0,\n        round_has_converged/1, web_debug_info/1, shutdown/1]).\n\n-export([rm_check/3,\n         rm_send_changes/5]).\n\n% API\n-export([get_subset_rand/1, get_subset_rand/2, get_subset_rand/3]).\n\n%% for testing\n-export([select_data_feeder/1]).\n-export_type([data/0, state/0]).\n\n-define(SEND_TO_GROUP_MEMBER(Pid, Process, Msg), comm:send(Pid, Msg, [{group_member, Process},\n                                                                      {?quiet}, {channel, prio}, {no_keep_alive}])).\n\n-define(TRACE_DEBUG(FormatString, Data), ok).\n%% -define(TRACE_DEBUG(FormatString, Data),\n%%         log:pal(\"[ Cyclon ~.0p ] \" ++ FormatString, [ comm:this() | Data])).\n\n%% print cache at the beginnig of every cycle in a dot friednly format\n-define(PRINT_CACHE_FOR_DOT(MyNode, Cache), ok).\n%% -define(PRINT_CACHE_FOR_DOT(MyNode, Cache), print_cache_dot(MyNode, Cache)).\n\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Type Definitions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-type data() :: cyclon_cache:cache().\n-type round() :: non_neg_integer().\n\n-type state() :: {Nodes::cyclon_cache:cache(), %% the cache of random nodes\n                  MyNode::node:node_type()}. %% the scalaris node of this process\n\n-dialyzer({no_return, print_cache_dot/2}).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Config Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%%------- External config function (called by gossip module) -------%%\n\n%% @doc The time interval in ms after which a new cycle is triggered by the gossip\n%%      module.\n-spec trigger_interval() -> pos_integer().\ntrigger_interval() -> % in ms\n    config:read(gossip_cyclon_interval).\n\n\n%% @doc The fanout (in cyclon always 1).\n-spec fanout() -> pos_integer().\nfanout() ->\n    1.\n\n\n%% @doc The minimum number of cycles per round.\n%%      Returns infinity, as rounds are not implemented by cyclon.\n-spec min_cycles_per_round() -> infinity.\nmin_cycles_per_round() ->\n    infinity.\n\n\n%% @doc The maximum number of cycles per round.\n%%      Returns infinity, as rounds are not implemented by cyclon.\n-spec max_cycles_per_round() -> infinity.\nmax_cycles_per_round() ->\n    infinity.\n\n\n%% @doc Gets the cyclon_shuffle_length parameter that defines how many entries\n%%      of the cache are exchanged.\n-spec shuffle_length() -> pos_integer().\nshuffle_length() ->\n    config:read(gossip_cyclon_shuffle_length).\n\n\n%% @doc Gets the cyclon_cache_size parameter that defines how many entries a\n%%      cache should at most have.\n-spec cache_size() -> pos_integer().\ncache_size() ->\n    config:read(gossip_cyclon_cache_size).\n\n\n%% @doc Cyclon doesn't need instantiabilty, so {gossip_cyclon, default} is always\n%%      used.\n-spec instance() -> {gossip_cyclon, default}.\n-compile({inline, [instance/0]}).\ninstance() ->\n    {gossip_cyclon, default}.\n\n\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(gossip_cyclon_interval) and\n    config:cfg_is_greater_than(gossip_cyclon_interval, 0) and\n\n    config:cfg_is_integer(gossip_cyclon_cache_size) and\n    config:cfg_is_greater_than(gossip_cyclon_cache_size, 2) and\n\n    config:cfg_is_integer(gossip_cyclon_shuffle_length) and\n    config:cfg_is_greater_than_equal(gossip_cyclon_shuffle_length, 1) and\n    config:cfg_is_less_than_equal(gossip_cyclon_shuffle_length,\n                                  config:read(gossip_cyclon_cache_size)).\n\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Sends a (local) message to the gossip module of the requesting process'\n%%      group asking for a random subset of the stored nodes.\n%%      The response in the form {cy_cache, [Node]} will be send (local) to the\n%%      requesting process.\n-spec get_subset_rand(N::pos_integer()) -> ok.\nget_subset_rand(N) ->\n    get_subset_rand(N, self()).\n\n%% @doc Same as get_subset_rand/1 but sends the reply back to the given Pid.\n-spec get_subset_rand(N::pos_integer(), Pid::comm:erl_local_pid()) -> ok.\nget_subset_rand(N, SourcePid) ->\n    Pid = pid_groups:get_my(gossip),\n    comm:send_local(Pid, {cb_msg, instance(), {get_subset_rand, N, SourcePid}}).\n\n%% @doc Same as get_subset_rand/2 but adds the given delay using msg_delay.\n-spec get_subset_rand(N::pos_integer(), Pid::comm:erl_local_pid(),\n                      Delay::non_neg_integer()) -> ok.\nget_subset_rand(N, SourcePid, Delay) ->\n    Pid = pid_groups:get_my(gossip),\n    msg_delay:send_local(Delay, Pid,\n                         {cb_msg, instance(), {get_subset_rand, N, SourcePid}}).\n\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Callback Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Initiate the gossip_cyclon module. <br/>\n%%      Called by the gossip module upon startup. <br/>\n%%      The Instance information is ignored, {gossip_cyclon, default} is always used.\n-spec init(Args::[proplists:property()]) -> {ok, state()}.\ninit(Args) ->\n    Neighbors = proplists:get_value(neighbors, Args),\n    log:log(info, \"[ Cyclon ~.0p ] activating...~n\", [comm:this()]),\n    rm_loop:subscribe(self(), cyclon,\n                      fun gossip_cyclon:rm_check/3,\n                      fun gossip_cyclon:rm_send_changes/5, inf),\n    monitor:proc_set_value(?MODULE, 'shuffle', rrd:create(60 * 1000000, 3, counter)), % 60s monitoring interval\n    Cache = case nodelist:has_real_pred(Neighbors) andalso\n                     nodelist:has_real_succ(Neighbors) of\n                true  -> cyclon_cache:new(nodelist:pred(Neighbors),\n                                          nodelist:succ(Neighbors));\n                false -> cyclon_cache:new()\n            end,\n\n    %% send two manual triggers to speed up startup performance\n    %% (setting up a 5 sec trigger (default for cyclon) takes ~6 s)\n    comm:send_local(pid_groups:get_my(gossip), {trigger_action, instance()}),\n    Delay = trigger_interval() div (2*1000), % half a trigger interval in sec\n    msg_delay:send_local(Delay, pid_groups:get_my(gossip), {trigger_action, instance()}),\n    {ok, {Cache, nodelist:node(Neighbors)}}.\n\n\n%% @doc Returns true, i.e. peer selection is done by gossip_cyclon module.\n-spec select_node(State::state()) -> {true, state()}.\nselect_node(State) ->\n    {true, State}.\n\n\n%% @doc Select and prepare the subset of the cache to be sent to the peer. <br/>\n%%      Called by the gossip module at the beginning of every cycle. <br/>\n%%      The selected exchange data (i.e. the selected subset of the cache) is\n%%      sent back to the gossip module as a message of the form\n%%      {selected_data, Instance, ExchangeData}.\n%%      gossip_trigger -> select_data() is equivalent to cy_shuffle in the old\n%%      cyclon module.\n-spec select_data(State::state()) -> {ok | discard_msg, state()}.\nselect_data({Cache, Node}=State) ->\n    case check_state(State) of\n        fail ->\n            ?TRACE_DEBUG(\"Cycle: ~w: select_data -> fail.\", [get_cycle()]),\n            {discard_msg, State};\n        _    ->\n            ?TRACE_DEBUG(\"Cycle: ~w: select_data -> ok.\", [get_cycle()]),\n            ?PRINT_CACHE_FOR_DOT(Node, Cache),\n            monitor:proc_set_value(?MODULE, 'shuffle',\n                                   fun(Old) -> rrd:add_now(1, Old) end),\n            Cache1 = cyclon_cache:inc_age(Cache),\n            {Cache2, NodeQ} = cyclon_cache:pop_oldest_node(Cache1),\n            Subset = cyclon_cache:get_random_subset(shuffle_length() - 1, Cache2),\n            ForSend = cyclon_cache:add_node(Node, 0, Subset),\n            Pid = pid_groups:get_my(gossip),\n            comm:send_local(Pid, {selected_peer, instance(), {cy_cache, [NodeQ]}}),\n            comm:send_local(Pid, {selected_data, instance(), ForSend}),\n            {ok, {Cache2, Node}}\n    end.\n\n\n%% @doc Process the subset from the requestor (P) and select a subset as reply\n%%      data (at Q). <br/>\n%%      Called by the behaviour module upon a p2p_exch message. <br/>\n%%      PSubset: exchange data (subset) from the p2p_exch request <br/>\n%%      Ref: used by the gossip module to identify the request <br/>\n%%      Round: ignored, as cyclon does not implement round handling\n%%      p2p_exch msg -> seleft_reply_data() is equivalent to cy_subset msg in the\n%%      old cyclon module.\n-spec select_reply_data(PSubset::data(), Ref::pos_integer(), Round::round(),\n    State::state()) -> {ok, state()}.\nselect_reply_data(PSubset, Ref, Round, {Cache, Node}) ->\n    % this is received at node Q -> integrate results of node P\n    QSubset = cyclon_cache:get_random_subset(shuffle_length(), Cache),\n    Pid = pid_groups:get_my(gossip),\n    comm:send_local(Pid, {selected_reply_data, instance(), {QSubset, PSubset}, Ref, Round}),\n    Cache1 = cyclon_cache:merge(Cache, Node, PSubset, QSubset, cache_size()),\n    {ok, {Cache1, Node}}.\n\n\n%% @doc Integrate the received subset (at node P). <br/>\n%%      Called by the behaviour module upon a p2p_exch_reply message. <br/>\n%%      QData: the subset from the peer (QSubset) and the subset wich was sent\n%%          in the request (PSubset) <br/>\n%%      Round: ignored, as cyclon does not implement round handling\n%%      Upon finishing the processing of the data, a message of the form\n%%      {integrated_data, Instance, RoundStatus} is to be sent to the gossip module.\n%%      p2p_exch_reply msg -> integrate_data() is equivalent to the cy_subset_response\n%%      msg in the old cyclon module.\n-spec integrate_data(QData::{data(), data()}, Round::round(), State::state()) ->\n    {ok, state()}.\nintegrate_data({QSubset, PSubset}, _Round, {Cache, Node}) ->\n    Cache1 = cyclon_cache:merge(Cache, Node, QSubset, PSubset, cache_size()),\n    Pid = pid_groups:get_my(gossip),\n    comm:send_local(Pid, {integrated_data, instance(), cur_round}),\n    {ok, {Cache1, Node}}.\n\n\n%% @doc Handle messages\n-spec handle_msg(Message, State::state()) -> {ok, state()} when\n      is_subtype(Message, {rm_changed, NewNode::node:node_type()} |\n                          {get_ages, SourcePid::comm:erl_local_pid()} |\n                          {get_subset_rand, N::pos_integer(), SourcePid::comm:erl_local_pid()} |\n                          {get_node_details_response, node_details:node_details()} |\n                          {get_dht_nodes_response, Nodes::[comm:mypid()]}).\n\n%% replaces the reference to self's dht node with NewNode\nhandle_msg({rm_changed, NewNode}, {Cache, _Node}) ->\n    {ok, {Cache, NewNode}};\n\n%% msg from admin:print_ages()\n%% request needs to be sent to the gossip module in the following form:\n%% {cb_msg, instance(), {get_ages, Pid}}\nhandle_msg({get_ages, Pid}, {Cache, _Node} = State) ->\n    comm:send_local(Pid, {cy_ages, cyclon_cache:get_ages(Cache)}),\n    {ok, State};\n\n%% msg from get_subset_random() (api)\n%% also directly requested from api_vm:get_other_vms() (change?)\nhandle_msg({get_subset_rand, N, Pid}, {Cache, _Node} = State) ->\n    comm:send_local(Pid, {cy_cache, cyclon_cache:get_random_nodes(N, Cache)}),\n    {ok, State};\n\n%% Response to a get_node_details message from self (via request_node_details()).\n%% The node details are used to possibly update Me and the succ and pred are\n%% possibly used to populate the cache.\n%% Request_node_details() is called in check_state() (i.e. in on_active({cy_shuffle})).\nhandle_msg({get_node_details_response, NodeDetails}, {OldCache, Node} = State) ->\n    case cyclon_cache:size(OldCache) =< 2 of\n        true  ->\n            Pred = node_details:get(NodeDetails, pred),\n            Succ = node_details:get(NodeDetails, succ),\n            NewCache =\n                lists:foldl(\n                  fun(N, CacheX) ->\n                          case node:same_process(N, Node) of\n                              false -> cyclon_cache:add_node(N, 0, CacheX);\n                              true -> CacheX\n                          end\n                  end, OldCache, [Pred, Succ]),\n            case cyclon_cache:size(NewCache) of\n                0 -> % try to get the cyclon cache from one of the known_hosts\n                    case config:read(known_hosts) of\n                        [] -> ok;\n                        [_|_] = KnownHosts ->\n                            Pid = util:randomelem(KnownHosts),\n                            EnvPid = comm:reply_as(comm:this(), 3, {cb_msg, instance(), '_'}),\n                            comm:send(Pid, {get_dht_nodes, EnvPid}, [{?quiet}])\n                    end;\n                _ ->\n                    ok\n            end,\n            {ok, {NewCache, Node}};\n        false ->\n            {ok, State}\n    end;\n\n%% used by the gossip_cyclon_feeder to add nodes to the cyclon cache\nhandle_msg({add_nodes_to_cache, Nodes}, {OldCache, Node}) ->\n    NewCache =\n        lists:foldl(\n          fun(N, CacheX) ->\n                  case node:same_process(N, Node) of\n                      false -> cyclon_cache:add_node(N, 0, CacheX);\n                      true -> CacheX\n                  end\n          end, OldCache, Nodes),\n    {ok, {NewCache, Node}};\n\n%% Response to get_dht_nodes message from service_per_vm. Contains a list of\n%% registered dht nodes from service_per_vm. Initiated in\n%% handle_msg({get_node_details_response, _NodeDetails} if the cache is empty.\n%% Tries to get a cyclon cache from one of the received nodes if cache is\n%% still empty.\n%% This happens (i.a.?) when only one node is present. In this case the\n%% get_node_details and the get_dht_nodes request are repeated every cycle\n%% (TODO is this the intended behaviour?)\nhandle_msg({get_dht_nodes_response, Nodes}, {Cache, _Node} = State) ->\n    Size = cyclon_cache:size(Cache),\n    case Nodes of\n        [] ->\n            {ok, State};\n        [_|_] when Size > 0 ->\n            {ok, State};\n        [Pid | _] ->\n            ?SEND_TO_GROUP_MEMBER(Pid, gossip, {p2p_exch, instance(), comm:this(), Cache, 0}),\n            {ok, State}\n    end.\n\n\n%% @doc Always returns false, as cyclon does not implement rounds.\n-spec round_has_converged(State::state()) -> {false, state()}.\nround_has_converged(State) ->\n    {false, State}.\n\n\n%% @doc Notifies the module about changes. <br/>\n%%      Changes can be new rounds, leadership changes or exchange failures. All\n%%      of them are ignored, as cyclon doesn't use / implements this features.\n-spec notify_change(any(), any(), State::state()) -> {ok, state()}.\nnotify_change(_, _, State) ->\n    {ok, State}.\n\n\n%% @doc Returns a key-value list of debug infos for the Web Interface. <br/>\n%%      Called by the gossip module upon {web_debug_info} messages.\n-spec web_debug_info(state()) ->\n    {KeyValueList::[{Key::string(), Value::string()},...], state()}.\nweb_debug_info({Cache, _Node}=State) ->\n    KeyValueList =\n        [{\"gossip_cyclon\", \"\"},\n         {\"cache_size\", integer_to_list(cyclon_cache:size(Cache))},\n         {\"cache (age, node):\", \"\"} | cyclon_cache:debug_format_by_age(Cache)],\n    {KeyValueList, State}.\n\n\n%% @doc Shut down the gossip_cyclon module. <br/>\n%%      Called by the gossip module upon stop_gossip_task(CBModule).\n-spec shutdown(State::state()) -> {ok, shutdown}.\nshutdown(_State) ->\n    % nothing to do\n    {ok, shutdown}.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Miscellaneous\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Filter Function for subscribing to the rm loop\n-spec rm_check(Neighbors, Neighbors, Reason) -> boolean() when\n      is_subtype(Neighbors, nodelist:neighborhood()),\n      is_subtype(Reason, rm_loop:reason()).\nrm_check(OldNeighbors, NewNeighbors, _Reason) ->\n    nodelist:node(OldNeighbors) =/= nodelist:node(NewNeighbors).\n\n\n%% @doc Exec Function for subscribing to the rm loop.\n%%      Sends changes to a subscribed cyclon process when the neighborhood\n%%      changes.\n-spec rm_send_changes(Pid::pid(), Tag::cyclon,\n        OldNeighbors::nodelist:neighborhood(),\n        NewNeighbors::nodelist:neighborhood(),\n        Reason::rm_loop:reason()) -> ok.\nrm_send_changes(Pid, cyclon, _OldNeighbors, NewNeighbors, _Reason) ->\n    comm:send_local(Pid, {cb_msg, instance(), {rm_changed, nodelist:node(NewNeighbors)}}).\n\n\n%% @doc Checks the current state. If the cache is empty or the current node is\n%%      unknown, the local dht_node will be asked for these values and the check\n%%      will be re-scheduled after 1s.\n-spec check_state(state()) -> ok | fail.\ncheck_state({Cache, _Node} = _State) ->\n    NeedsInfo = case cyclon_cache:size(Cache) of\n                    0 -> [pred, succ];\n                    _ -> []\n                end,\n    case NeedsInfo of\n        [_|_] -> request_node_details(NeedsInfo),\n                 fail;\n        []    -> ok\n    end.\n\n\n%% @doc Sends the local node's dht_node a request to tell us some information\n%%      about itself.\n%%      The node will respond with a {get_node_details_response, NodeDetails}\n%%      message, which will be envoloped and passed to this module through the\n%%      gossip module.\n-spec request_node_details([node_details:node_details_name()]) -> ok.\nrequest_node_details(Details) ->\n    case pid_groups:get_my(dht_node) of\n        failed ->\n            ok;\n        DHT_Node ->\n            This = comm:this(),\n            case comm:is_valid(This) of\n                true ->\n                    EnvPid = comm:reply_as(This, 3, {cb_msg, instance(), '_'}),\n                    comm:send_local(DHT_Node, {get_node_details, EnvPid, Details}),\n                    ok;\n                false -> ok\n            end\n    end.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Debugging and Testing\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n\n%% @doc Print the cache in a dot compatible format.\n%%      Format: Self -> Reference1; Self -> Reference2 ; ...\n%%      Prints references to nodes as local pids, so this produces meaningful\n%%      results if all nodes are started in the same Erlang VM.\n%%      TODO the fun in foldl only throws a 'has no local return' dialyzer warning\n%%              if the PRINT_CACHE_FOR_DOT macro is set to ok\n-compile({nowarn_unused_function, {print_cache_dot, 2}}).\n-spec print_cache_dot(node:node_type(), data()) -> ok.\nprint_cache_dot(MyNode, Cache) ->\n    Cycle = get_cycle(),\n    MyPid = comm:make_local(node:pidX(MyNode)),\n    Graph = lists:foldl(\n                    fun({Node, _Age}, AccIn) ->\n                        % printing MyPid causes dialyzer to think that this method does not have a \"local return\"\n                        [AccIn, io_lib:format(\"~w -> ~w; \", [MyPid, comm:make_local(node:pidX(Node))])]\n                    end, io_lib:format(\"[Cycle: ~w] \", [Cycle]), Cache),\n    log:pal(lists:flatten(Graph)).\n\n\n%% @doc Simple cycle counting meachanism\n%%      This only works, if this function is called excactly once every cycle.\n%%      For debugging purposes only, the gossip module provides more\n%%      sophisticated cycle counting.\n-spec get_cycle() -> non_neg_integer().\n-compile({nowarn_unused_function, {get_cycle, 0}}).\nget_cycle() ->\n    case get(cycles) of\n        undefined -> put(cycles, 1), 0;\n        Cycle1 -> put(cycles, Cycle1+1), Cycle1\n    end.\n\n\n%% still fails\n%% -spec init_feeder(Neighbors::nodelist:neighborhood()) -> {[proplists:property()]}.\n%% init_feeder(Neighbors) ->\n%%     {[{neighbors, Neighbors}]}.\n\n\n-spec select_data_feeder(State::state()) -> {state()}.\nselect_data_feeder(State) ->\n    monitor:proc_set_value(?MODULE, 'shuffle', rrd:create(60 * 1000000, 3, counter)), % 60s monitoring interval\n    {State}.\n\n\n%% node_details:node_details_name allows new_key, but\n%% dht_node:on({get_node_details, Pid, Which}, State) doesn't\n-compile({nowarn_unused_function, {request_node_details_feeder, 1}}).\n-spec request_node_details_feeder([node_details:node_details_name()]) ->\n    {[node_details:node_details_name()]}.\nrequest_node_details_feeder(Details) ->\n    {lists:filter(fun(Detail) -> Detail =/= new_key end, Details)}.\n\n"
  },
  {
    "path": "src/gossip_cyclon_feeder.erl",
    "content": "% @copyright 2013-2015 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc    Finds nodes from pid_groups and known hosts and feeds them to cyclon\n%% @end\n%% @version $Id$\n-module(gossip_cyclon_feeder).\n-author('schintke@zib.de').\n-author('kruber@zib.de').\n-author('fajerski@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-behaviour(gen_component).\n\n-export([start_link/1, on/2, init/1]).\n\n-include(\"gen_component.hrl\").\n\n-record(state, {\n          local_nodes = ?required(state, local_nodes) :: [pid()],\n          remote_nodes = ?required(state, remote_nodes) :: [comm:mypid_plain()],\n          known_hosts = ?required(state, known_hosts) :: [comm:mypid_plain()]}).\n\n-type state_t() :: #state{}.\n\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [], \n                             [{wait_for_init},\n                              {pid_groups_join_as, DHTNodeGroup, ?MODULE}]).\n\n-spec init([]) -> state_t().\ninit([]) ->\n    case config:read(start_type) =:= recover of\n        true ->\n            msg_delay:send_trigger(10, {trigger_round});\n        false ->\n            ok\n    end,\n    #state{local_nodes = pid_groups:find_all(dht_node),\n           remote_nodes = [],\n           known_hosts = config:read(known_hosts)\n          }.\n\n-spec on(term(), state_t()) -> state_t().\non({trigger_round}, #state{local_nodes=LocalNodes, remote_nodes=RemoteNodes, \n                           known_hosts=KnownHosts} = State) ->\n    ?TRACE(\"gossip_cyclon_feeder:trigger: ~p ~p ~p ~n\", [LocalNodes, RemoteNodes, KnownHosts]),\n    % call local nodes\n    [comm:send_local(Pid, {get_node_details, \n                           comm:reply_as(comm:this(), 3, \n                                         {local_node_response, Pid, '_'}), [neighbors]})\n     || Pid <- LocalNodes, is_process_alive(Pid)],\n    % call remote nodes\n    [comm:send(Pid, {get_node_details, \n                           comm:reply_as(comm:this(), 3, \n                                         {remote_node_response, Pid, '_'}), [neighbors]},\n              [{?quiet}])\n     || Pid <- RemoteNodes],\n    % call known hosts\n    [comm:send(Pid, {get_dht_nodes, \n                           comm:reply_as(comm:this(), 3, {known_hosts_response, Pid, '_'})},\n              [{?quiet}])\n     || Pid <- KnownHosts],\n    msg_delay:send_trigger(10, {trigger_round}),\n    State;\n\non({local_node_response, Pid, {get_node_details_response, [{neighbors, Neighborhood}]}},\n   #state{local_nodes = LocalNodes} = State) ->\n    case util:lists_take(Pid, LocalNodes) of\n        false ->\n            %% is duplicate response\n            State;\n        NewLocalNodes ->\n            %% @todo: notify cyclon\n            notify_cyclon(Neighborhood),\n            State#state{local_nodes = NewLocalNodes}\n    end;\n\non({remote_node_response, Pid, {get_node_details_response, [{neighbors, Neighborhood}]}},\n   #state{remote_nodes = RemoteNodes} = State) ->\n    case util:lists_take(Pid, RemoteNodes) of\n        false ->\n            %% is duplicate response\n            State;\n        NewRemoteNodes ->\n            %% @todo: notify cyclon\n            notify_cyclon(Neighborhood),\n            State#state{remote_nodes = NewRemoteNodes}\n    end;\n\non({known_hosts_response, Pid, {get_dht_nodes_response, Nodes}}, \n   #state{known_hosts=KnownHosts, remote_nodes=RemoteNodes} = State) ->\n    case util:lists_take(Pid, KnownHosts) of\n        false ->\n            %% is duplicate response\n            State;\n        NewKnownHosts ->\n            %% add to remote_nodes\n            State#state{known_hosts = NewKnownHosts,\n                        remote_nodes = lists:append(Nodes, RemoteNodes)}\nend.\n\nnotify_cyclon(Neighborhood) ->\n    ?TRACE(\"gossip_cyclon_feeder:notify_cyclon: ~p ~p ~n\", \n           [pid_groups:get_my(gossip), Neighborhood]),\n    Pred = nodelist:pred(Neighborhood),\n    Node = nodelist:node(Neighborhood),\n    Succ = nodelist:succ(Neighborhood),\n    comm:send_local(pid_groups:get_my(gossip),\n                    {cb_msg, \n                     {gossip_cyclon, default}, \n                     {add_nodes_to_cache, [Pred, Node, Succ]}}).\n"
  },
  {
    "path": "src/gossip_load.erl",
    "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%  @copyright 2008-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jens V. Fischer <jensvfischer@gmail.com>\n%% @doc Gossip based aggregation of load information.\n%%      This module implements the symmetric push-sum protocol. The algorithm is\n%%      used to compute aggregates of the load information, which is measured as\n%%      the count of items currently in a node's key range. <br/>\n%%      The aggregation of load information is used in Scalaris for two purposes:\n%%      First, for passive load balancing. When a node  joins, the gossiped load\n%%      information is used to decide where to place the new node. The node will\n%%      be placed so that the standard deviation of the load is reduced the most.\n%%      Second, the gossiping is used for system monitoring. The local estimates\n%%      of the gossiping can be viewed for example in the Web Interface of every\n%%      Scalaris node. <br/>\n%%      Different metrics are computed on the load information:\n%%      <ul>\n%%          <li> average load, the arithmetic mean of all nodes load information </li>\n%%          <li> the maximum load </li>\n%%          <li> the minimum load </li>\n%%          <li> standard deviation of the average load </li>\n%%          <li> leader based size, based on counting </li>\n%%          <li> key range bases size, calculated as address space of keys / average key range per node </li>\n%%          <li> histogram of load per key range (load measured as number of items per node) </li>\n%%      </ul>\n%%      The module is initialised during the startup of the gossiping framework,\n%%      continuously aggregating load information in the background. Additionally,\n%%      is it possible to start instances of the module for the purpose of computing\n%%      different sizes histograms, request_histogram/1.\n%% @end\n%% @version $Id$\n-module(gossip_load).\n-behaviour(gossip_beh).\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n% API\n-export([get_values_best/1, request_histogram/2, load_info_get/2, load_info_other_get/3]).\n\n% gossip_beh\n-export([init/1, check_config/0, trigger_interval/0, fanout/0,\n        select_node/1, select_data/1, select_reply_data/4, integrate_data/3,\n        handle_msg/2, notify_change/3, min_cycles_per_round/0, max_cycles_per_round/0,\n        round_has_converged/1, web_debug_info/1, shutdown/1]).\n\n%% for testing\n-export([tester_create_round/1, tester_create_state/11, tester_create_histogram/1,\n         tester_create_load_data_list/1, is_histogram/1,\n         tester_create_histogram_size/1, init_feeder/1, get_values_best_feeder/1]).\n\n-export_type([state/0, histogram/0, histogram_size/0, bucket/0]). %% for config gossip_load_*.erl\n-export_type([avg/0, avg_kr/0, min/0, max/0]). %% for config gossip_load_*.erl\n-export_type([load_info/0, load_info_other/0, merged/0]).\n\n%% -define(SHOW, config:read(log_level)).\n-define(SHOW, debug).\n\n-define(DEFAULT_MODULE, gossip_load_default).\n\n%% List of module names for additional aggregation\n%% (Modules need to implement gossip_load_beh)\n-define(ADDITIONAL_MODULES, [lb_active_gossip_load_metric, lb_active_gossip_request_metric]).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Type Definitions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-type state_key() :: convergence_count | instance | leader | load_data | ring_data | merged |\n    no_of_buckets | prev_state | range | request | requestor | round | status.\n-type avg() :: {Value::float(), Weight::float()}.\n-type avg_kr() :: {Value::number(), Weight::float()}.\n-type min() :: non_neg_integer().\n-type max() :: non_neg_integer().\n-type merged() :: non_neg_integer().\n-type bucket() :: {Interval::intervals:interval(), Avg::avg()|unknown}.\n-type histogram() :: [bucket()].\n-type histogram_size() :: pos_integer().\n-type instance() :: {Module :: gossip_load, Id :: atom() | uid:global_uid()}.\n-type status() :: init | uninit.\n-type round() :: non_neg_integer().\n\n%% Record for ring related gossipping data\n-record(ring_data, {\n            size_inv  = unknown :: unknown | avg(), % 1 / size\n            avg_kr    = unknown :: unknown | avg_kr() % average key range (distance between nodes in the address space)})\n    }).\n\n-type ring_data_uninit() :: #ring_data{}.\n-type ring_data() :: {ring_data, SizeInv::avg(), AvgKr::avg_kr()}.\n-type ring_data2() :: ring_data_uninit() | ring_data().\n\n%% record of load data values for gossiping\n-record(load_data, {\n            name      = ?required(name, load_data) :: atom(), %% unique name\n            avg       = unknown :: unknown | avg(), % average load\n            avg2      = unknown :: unknown | avg(), % average of load^2\n            min       = unknown :: unknown | min(),\n            max       = unknown :: unknown | max(),\n            histo     = unknown :: unknown | histogram()\n    }).\n\n-type load_data_uninit() :: #load_data{}.\n-type load_data_skipped() :: {load_data, atom(), skip}.\n-type load_data() :: {load_data, Name::atom(), Avg::avg(), Avg2::avg(),\n                      Min::non_neg_integer(), Max::non_neg_integer(),\n                      Histogram::histogram()}.\n\n%% -type load_data_list() :: [ load_data_uninit() | load_data(), ...].\n-type load_data_list() :: [load_data() | load_data_skipped(), ...].\n-type load_data_list2() :: [load_data_uninit() | load_data_skipped() | load_data(), ...].\n\n-type data() :: {load_data_list(), ring_data()}.\n\n%% state record\n-record(state, {\n        status              = uninit :: status(),\n        instance            = ?required(state, instance) :: instance(),\n        load_data_list      = ?required(state, load_data_list2) :: load_data_list2(),\n        ring_data           = ?required(state, ring_data2) :: ring_data2(),\n        leader              = unknown :: unknown | boolean(),\n        range               = unknown :: unknown | intervals:interval(),\n        request             = false :: boolean(),\n        requestor           = none :: none | comm:mypid(),\n        no_of_buckets       = ?required(state, no_of_buckets) :: non_neg_integer(),\n        round               = 0 :: round(),\n        merged              = 0 :: non_neg_integer(),\n        convergence_count   = 0 :: non_neg_integer()\n    }).\n\n-type state() :: #state{}.\n-type full_state() :: {PrevState :: unknown | state(), CurrentState :: state()}.\n\n-record(load_info_other, { %% additional gossip values\n            name    = ?required(name, load_info_other) :: atom(),\n            avg     = unknown :: unknown | float(),  % average load\n            stddev  = unknown :: unknown | float(), % standard deviation of the load\n            min     = unknown :: unknown | min(), % minimum load\n            max     = unknown :: unknown | max() % maximum load\n    }).\n\n-type(load_info_other() :: #load_info_other{}).\n\n% record of load data values for use by other modules\n-record(load_info, { %% general info and default gossip values\n            avg       = unknown :: unknown | float(),  % average load\n            stddev    = unknown :: unknown | float(), % standard deviation of the load\n            size_ldr  = unknown :: unknown | float(), % estimated ring size: 1/(average of leader=1, others=0)\n            size_kr   = unknown :: unknown | float(), % estimated ring size based on average key ranges\n            min       = unknown :: unknown | min(), % minimum load\n            max       = unknown :: unknown | max(), % maximum load\n            merged    = unknown :: unknown | merged(), % how often the data was merged since the node entered/created the round\n            other     = []      :: [load_info_other()]\n    }).\n\n-opaque(load_info() :: #load_info{}).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Config Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%%--------- External config function (called by bh module) ---------%%\n\n%% @doc The time interval in ms after which a new cycle is trigger by the behaviour\n%%      module.\n-spec trigger_interval() -> pos_integer().\ntrigger_interval() -> % in ms\n    config:read(gossip_load_interval).\n\n\n%% @doc The fanout (number of peers contacted per cycle).\n-spec fanout() -> pos_integer().\nfanout() ->\n    config:read(gossip_load_fanout).\n\n\n%% @doc The minimum number of cycles per round.\n%%      Only full cycles (i.e. received replies) are counted (ignored triggers\n%%      do not count as cycle).\n%%      Only relevant for leader, all other nodes enter rounds when told to do so.\n-spec min_cycles_per_round() -> non_neg_integer().\nmin_cycles_per_round() ->\n    config:read(gossip_load_min_cycles_per_round).\n\n\n%% @doc The maximum number of cycles per round.\n%%      Only full cycles (i.e. received replies) are counted (ignored triggers\n%%      do not count as cycle).\n%%      Only relevant for leader, all other nodes enter rounds when told to do so.\n-spec max_cycles_per_round() -> pos_integer().\nmax_cycles_per_round() ->\n    config:read(gossip_load_max_cycles_per_round).\n\n\n%%------------------- Private config functions ---------------------%%\n\n-spec convergence_count_best_values() -> pos_integer().\nconvergence_count_best_values() ->\n% convergence is counted (on average) twice every cycle:\n% 1) When receiving the answer to an request (integrate_data()) (this happens\n%       once every cycle).\n% 2) When receiving a request (select_reply_data()) (this happens randomly,\n%       but *on average* once per round).\n    config:read(gossip_load_convergence_count_best_values).\n\n\n-spec convergence_count_new_round() -> pos_integer().\nconvergence_count_new_round() ->\n    config:read(gossip_load_convergence_count_new_round).\n\n-spec convergence_epsilon() -> float().\nconvergence_epsilon() ->\n    config:read(gossip_load_convergence_epsilon).\n\n-spec no_of_buckets() -> histogram_size().\nno_of_buckets() ->\n    config:read(gossip_load_number_of_buckets).\n\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(gossip_load_interval) and\n    config:cfg_is_greater_than(gossip_load_interval, 0) and\n\n    config:cfg_is_integer(gossip_load_min_cycles_per_round) and\n    config:cfg_is_greater_than_equal(gossip_load_min_cycles_per_round, 0) and\n\n    config:cfg_is_integer(gossip_load_max_cycles_per_round) and\n    config:cfg_is_greater_than_equal(gossip_load_max_cycles_per_round, 1) and\n\n    config:cfg_is_float(gossip_load_convergence_epsilon) and\n    config:cfg_is_in_range(gossip_load_convergence_epsilon, 0.0, 100.0) and\n\n    config:cfg_is_integer(gossip_load_convergence_count_best_values) and\n    config:cfg_is_greater_than(gossip_load_convergence_count_best_values, 0) and\n\n    config:cfg_is_integer(gossip_load_convergence_count_new_round) and\n    config:cfg_is_greater_than(gossip_load_convergence_count_new_round, 0) and\n\n    config:cfg_is_integer(gossip_load_number_of_buckets) and\n    config:cfg_is_greater_than(gossip_load_number_of_buckets, 0) and\n\n    config:cfg_is_integer(gossip_load_fanout) and\n    config:cfg_is_greater_than(gossip_load_fanout, 0) and\n\n    config:cfg_is_list(gossip_load_additional_modules,\n                       fun(El) -> lists:member(El, ?ADDITIONAL_MODULES) end,\n                       \"A valid additional gossip_load module\").\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n\n%% @doc Sends a (local) message to the gossip module of the requesting process'\n%%      group asking for the best aggregation results.\n%%      The response in the form {gossip_get_values_best_response, BestValues}\n%%      will be send (local) to the requesting process.\n%%      BestValues are either the aggregation restult from the current or previous round,\n%%      depending on convergence_count_best_values.\n%%      If the option {instance, Instance} is present, the Instance is used.\n%%      Otherwise {gossip_load, default} is used as instance.\n%%      If the option {source_pid, SourcePid} is present, response is sent to\n%%      the SourcePid instead of the pid of the requesting process.\n%%      If the option {delay, Seconds} is present, the request is delayed for (at\n%%      least) Seconds seconds.\n%%      If the option {send_after, Milliseconds} is present, the request is delayed\n%%      for excatly Milliseconds milliseconds.\n-spec get_values_best(Options::[proplists:property()]) -> ok.\nget_values_best(Options) when is_list(Options) ->\n    Instance = proplists:get_value(instance, Options, {gossip_load, default}),\n    SourcePid = proplists:get_value(source_pid, Options, self()),\n    Delay = proplists:get_value(msg_delay, Options, 0),\n    SendAfter = proplists:get_value(send_after, Options, 0),\n    Pid = pid_groups:get_my(gossip),\n    Msg = {cb_msg, Instance, {gossip_get_values_best, SourcePid}},\n    if Delay =:= 0 andalso SendAfter =:= 0 ->\n           comm:send_local(Pid, Msg);\n       Delay =:= 0 andalso SendAfter =/= 0 ->\n           comm:send_local_after(SendAfter, Pid, Msg), ok;\n       Delay =/= 0 ->\n           msg_delay:send_local(Delay, Pid, Msg)\n    end.\n\n\n%% @doc Request a histogram with Size number of Buckets. <br/>\n%%      The resulting histogram will be sent to SourceId, when all values have\n%%      properly converged.\n-spec request_histogram(Size::histogram_size(), SourcePid::comm:mypid()) -> ok.\nrequest_histogram(Size, SourcePid) when Size >= 1 ->\n    gossip:start_gossip_task(?MODULE, [{no_of_buckets, Size}, {requestor, SourcePid}]).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Callback Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Initiate the gossip_load module. <br/>\n%%      Instance (mandatory) makes the module aware of its own instance id, which\n%%      is saved in the state of the module.\n%%      NoOfBuckets (optional) defines the size of the histogram calculated.\n%%      Requestor (optinal) defines to whom the calculated histogram will be sent\n%%      (used for request_histogram/1, called through the gossip module).\n-spec init(Args::[proplists:property()]) -> {ok, full_state()}.\ninit(Args) ->\n    Instance = proplists:get_value(instance, Args),\n    NoOfBuckets = proplists:get_value(no_of_buckets, Args, no_of_buckets()),\n    Requestor = proplists:get_value(requestor, Args, none),\n    log:log(debug, \"[ ~w ] CBModule initiated. NoOfBuckets: ~w, Requestor: ~w\",\n        [Instance, NoOfBuckets, Requestor]),\n    CurState = #state{\n        load_data_list = load_data_list_new(),\n        ring_data = ring_data_new(),\n        no_of_buckets = NoOfBuckets,\n        request = Requestor =/= none,\n        requestor = Requestor,\n        instance = Instance\n    },\n    {ok, {unknown, CurState}}.\n\n\n%% @doc Returns false, i.e. peer selection is done by gossip module.\n%%      State: the state of the gossip_load module\n-spec select_node(FullState::full_state()) -> {boolean(), full_state()}.\nselect_node(FullState) ->\n    {false, FullState}.\n\n%% @doc Select and prepare the load information to be sent to the peer. <br/>\n%%      Called by the gossip module at the beginning of every cycle. <br/>\n%%      The selected exchange data is sent back to the gossip module as a message\n%%      of the form {selected_data, Instance, ExchangeData}.\n%%      State: the state of the gossip_load module\n-spec select_data(FullState::full_state()) -> {ok, full_state()}.\nselect_data({PrevState, CurState}) ->\n    log:log(debug, \"[ ~w ] select_data: State: ~w\", [state_get(instance, CurState), CurState]),\n    CurState1 = case state_get(status, CurState) of\n        uninit ->\n            request_node_details(CurState), CurState;\n        init ->\n            {Data, NewCurState1} = prepare_data([], CurState),\n            Pid = pid_groups:get_my(gossip),\n            comm:send_local(Pid, {selected_data, state_get(instance, NewCurState1), Data}),\n            NewCurState1\n    end,\n    {ok, {PrevState, CurState1}}.\n\n\n%% @doc Process the data from the requestor and select reply data. <br/>\n%%      Called by the behaviour module upon a p2p_exch message. <br/>\n%%      PData: exchange data from the p2p_exch request <br/>\n%%      Ref: used by the gossip module to identify the request <br/>\n%%      RoundStatus / Round: round information used for special handling of\n%%          messages from previous rounds <br/>\n%%      State: the state of the gossip_load module\n-spec select_reply_data(PData::data(), Ref::pos_integer(), Round::round(),\n    FullState::full_state()) -> {discard_msg | ok | retry | send_back, full_state()}.\nselect_reply_data(PData, Ref, Round, {PrevState, CurState}) ->\n\n    CurRound = state_get(round, CurState),\n    PrevRound = state_get(round, PrevState),\n\n    case state_get(status, CurState) of\n        uninit ->\n            log:log(?SHOW, \"[ ~w ] select_reply_data in uninit\", [?MODULE]),\n            {retry,{PrevState, CurState}};\n        init when Round =:= CurRound ->\n            CurState1 = select_reply_data_helper(PData, Ref, Round, CurState),\n            {ok, {PrevState, CurState1}};\n        init when Round =:= PrevRound ->\n            PrevState1 = select_reply_data_helper(PData, Ref, Round, PrevState),\n            {ok, {PrevState1, CurState}};\n        _ ->\n            log:log(warn(), \"[ ~w ] Discarded data in select_reply_data. Reason: invalid round.\",\n                [state_get(instance, CurState)]),\n            {send_back, {PrevState, CurState}}\n    end.\n\n\n%% @doc Helper for select_reply_data()\n-spec select_reply_data_helper(PData::data(), Ref::pos_integer(), round(),\n                               state()) -> state().\nselect_reply_data_helper(PData, Ref, Round, State) ->\n    PLoadList = element(1, PData),\n    PSkipped = [Module || {load_data, Module, skip} <- PLoadList],\n    {Data1, State1} = prepare_data(PSkipped, State),\n    Data2 = {replace_skipped(PLoadList, element(1, Data1)),\n             element(2, Data1)},\n    Pid = pid_groups:get_my(gossip),\n    comm:send_local(Pid, {selected_reply_data, state_get(instance, State1),\n                          Data2, Ref, Round}),\n    {_Data2, State2} = merge_load_data(PData, State1),\n    State2.\n\n\n%% @doc Integrate the reply data. <br/>\n%%      Called by the behaviour module upon a p2p_exch_reply message. <br/>\n%%      QData: the reply data from the peer <br/>\n%%      RoundStatus / Round: round information used for special handling of\n%%          messages from previous rounds <br/>\n%%      State: the state of the gossip_load module <br/>\n%%      Upon finishing the processing of the data, a message of the form\n%%      {integrated_data, Instance, RoundStatus} is to be sent to the gossip module.\n-spec integrate_data(QData::data(), Round::round(), FullState::full_state()) ->\n    {discard_msg | ok | retry | send_back, full_state()}.\nintegrate_data(QData, Round, {PrevState, CurState}=FullState) ->\n    log:log(debug, \"[ ~w ] Reply-Data: ~w~n\", [state_get(instance, CurState), QData]),\n\n    CurRound = state_get(round, CurState),\n    PrevRound = state_get(round, PrevState),\n\n    case state_get(status, CurState) of\n        uninit ->\n            log:log(?SHOW, \"[ ~w ] integrate_data in uninit\", [?MODULE]),\n            {retry, FullState};\n        init when Round =:= CurRound ->\n            CurState1 = element(2, merge_load_data(QData, CurState)),\n            log:log(info, \"Load-Info: ~s\", [to_string(get_load_info(CurState1))]),\n            comm:send_local(pid_groups:get_my(gossip),\n                            {integrated_data, state_get(instance, CurState1), cur_round}),\n            {ok, {PrevState, CurState1}};\n        init when Round =:= PrevRound ->\n            PrevState1 = element(2, merge_load_data(QData, PrevState)),\n            log:log(info, \"Load-Info: ~s\", [to_string(get_load_info(PrevState1))]),\n            comm:send_local(pid_groups:get_my(gossip),\n                            {integrated_data, state_get(instance, PrevState1), prev_round}),\n            {ok, {PrevState1, CurState}};\n        _ ->\n            log:log(warn(), \"[ ~w ] Discarded data in integrate_data. Reason: invalid round.\", [?MODULE]),\n            {send_back, FullState}\n    end.\n\n\n%% @doc Handle get_state_response messages from the dht_node. <br/>\n%%      The received load information is stored and the status is set to init,\n%%      allowing the start of a new gossip round.\n%%      State: the state of the gossip_load module <br/>\n-spec handle_msg(Message::{get_node_details_response, node_details:node_details()},\n    FullState::full_state()) -> {ok, full_state()}.\nhandle_msg({get_node_details_response, NodeDetails}, {PrevState, CurState}) ->\n\n    LoadDataNew =\n        [ begin\n              Module = data_get(name, LoadData),\n              Load = Module:get_load(NodeDetails),\n              case Load of\n                  unknown -> {load_data, Module, skip};\n                  Load ->\n                      log:log(?SHOW, \"[ ~w ] Load: ~w\", [state_get(instance, CurState), Load]),\n                      LoadData1 = data_set(avg, {float(Load), 1.0}, LoadData),\n                      LoadData2 = data_set(min, Load, LoadData1),\n                      LoadData3 = data_set(max, Load, LoadData2),\n                      LoadData4 = data_set(avg2, {float(Load*Load), 1.0}, LoadData3),\n                      % histogram\n                      Histo = init_histo(Module, NodeDetails, CurState),\n                      log:log(?SHOW,\"[ ~w ] Histo: ~s\", [state_get(instance, CurState), to_string(Histo)]),\n                      _LoadData5 = data_set(histo, Histo, LoadData4)\n              end\n          end\n        || LoadData <- state_get(load_data_list, CurState)],\n\n    CurState1 = state_set(load_data_list, LoadDataNew, CurState),\n\n    RingData = state_get(ring_data, CurState1),\n    RingData1 = case (state_get(leader, CurState1)) of\n                    true ->\n                        data_set(size_inv, {1.0, 1.0}, RingData);\n                    false ->\n                        data_set(size_inv, {1.0, 0.0}, RingData)\n                end,\n    AvgKr = calc_initial_avg_kr(state_get(range, CurState1)),\n    RingData2 = data_set(avg_kr, AvgKr, RingData1),\n    CurState2 = state_set(ring_data, RingData2, CurState1),\n\n    {NewData, CurState3} = prepare_data([], CurState2),\n    CurState4 = state_set(status, init, CurState3),\n\n    % send PData to BHModule\n    Pid = pid_groups:get_my(gossip),\n    comm:send_local(Pid, {selected_data, state_get(instance, CurState4), NewData}),\n    {ok, {PrevState, CurState4}};\n\nhandle_msg({gossip_get_values_best, SourcePid}, {PrevState, CurState}) ->\n    BestState = previous_or_current(PrevState, CurState),\n    BestValues = get_load_info(BestState),\n    comm:send_local(SourcePid, {gossip_get_values_best_response, BestValues}),\n    {ok, {PrevState, CurState}};\n\nhandle_msg({gossip_get_values_all, SourcePid}, {unknown, CurState}) ->\n    CurInfo = get_load_info(CurState),\n    AllValues = {#load_info{}, CurInfo, CurInfo},\n    comm:send_local(SourcePid, {gossip_get_values_all_response, AllValues}),\n    {ok, {unknown, CurState}};\n\nhandle_msg({gossip_get_values_all, SourcePid}, {PrevState, CurState}) ->\n    BestState = previous_or_current(PrevState, CurState),\n    AllValues = {get_load_info(PrevState), get_load_info(CurState),\n                  get_load_info(BestState)},\n    comm:send_local(SourcePid, {gossip_get_values_all_response, AllValues}),\n    {ok, {PrevState, CurState}}.\n\n\n%% @doc Checks if the current round has converged yet <br/>\n%%      Returns true if the round has converged, false otherwise.\n-spec round_has_converged(FullState::full_state()) -> {boolean(), full_state()}.\nround_has_converged({_PrevState, CurState}=FullState) ->\n    ConvergenceCount = convergence_count_new_round(),\n    {has_converged(ConvergenceCount, CurState), FullState}.\n\n\n%% @doc Notifies the gossip_load module about changes. <br/>\n%%      Changes can be one of the following:\n%%      <ol>\n%%          <li> new_round <br/>\n%%               Notifies the the callback module about the beginning of round </li>\n%%          <li> leader <br/>\n%%               Notifies the the callback module about a change in the key range\n%%               of the node. The MsgTag indicates whether the node is a leader\n%%               or not, the NewRange is the new key range of the node. </li>\n%%          <li> exch_failure <br/>\n%%               Notifies the the callback module about a failed message delivery,\n%%               including the exchange data and round from the original message. </li>\n%%      </ol>\n-spec notify_change(Keyword::new_round, NewRound::round(), FullState::full_state()) -> {ok, full_state()};\n    (Keyword::leader, {MsgTag::is_leader | no_leader, NewRange::intervals:interval()},\n            FullState::full_state()) -> {ok, full_state()};\n    (Keyword::exch_failure, {_MsgTag::atom(), Data::data(), _Round::round()},\n             FullState::full_state()) -> {ok, full_state()}.\nnotify_change(new_round, NewRound, {PrevState, CurState}) ->\n    log:log(debug, \"[ ~w ~w ] new_round notification. NewRound: ~w. Leader: ~w\",\n            [state_get(instance, CurState), comm:this(), NewRound, state_get(leader, CurState)]),\n    case state_get(request, CurState) of\n        true ->\n          {ok, {PrevState, finish_request(CurState)}};\n        false ->\n          new_round(NewRound, PrevState, CurState)\n    end;\n\nnotify_change(leader, {MsgTag, NewRange}, {PrevState, CurState}) when MsgTag =:= is_leader ->\n    log:log(?SHOW, \"[ ~w ] ~w is the leader\", [state_get(instance, CurState), comm:this()]),\n    CurState1 = state_set([{leader, true}, {range, NewRange}], CurState),\n    {ok, {PrevState, CurState1}};\n\nnotify_change(leader, {MsgTag, NewRange}, {PrevState, CurState}) when MsgTag =:= no_leader ->\n    log:log(?SHOW, \"[ ~w ] ~w is no leader\", [state_get(instance, CurState), comm:this()]),\n    CurState1 = state_set([{leader, false}, {range, NewRange}], CurState),\n    {ok, {PrevState, CurState1}};\n\n\nnotify_change(exch_failure, {_MsgTag, Data, Round}, {PrevState, CurState} = FullState) ->\n    CurRound = state_get(round, CurState),\n    PrevRound = state_get(round, PrevState),\n    FullState1 =\n        case {Round, Data} of\n            {CurRound, undefined} ->\n                FullState;\n            {CurRound, _} ->\n                log:log(debug, \" [ ~w ] exch_failure in valid current round\", [?MODULE]),\n                {_NewData, CurState1} = merge_failed_exch_data(Data, CurState),\n                {PrevState, CurState1};\n            {PrevRound, undefined} ->\n                FullState;\n            {PrevRound, _} ->\n                log:log(debug, \" [ ~w ] exch_failure in valid old round\", [?MODULE]),\n                {_NewData, PrevState1} = merge_failed_exch_data(Data, PrevState),\n                {PrevState1, CurState};\n            _ ->\n                log:log(warn(), \" [ ~w ] exch_failure in invalid round\", [?MODULE]),\n                FullState\n        end,\n    {ok, FullState1}.\n\n\n\n%% @doc Returns a key-value list of debug infos for the Web Interface. <br/>\n%%      Called by the gossip module upon {web_debug_info} messages.\n%%      State: the state of the gossip_load module.\n-spec web_debug_info(full_state()) ->\n    {KeyValueList::[{Key::string(), Value::any()},...], full_state()}.\nweb_debug_info({PrevState, CurState}=FullState) ->\n    PreviousLoadData = state_get(load_data_list, PrevState),\n    CurrentLoadData = state_get(load_data_list, CurState),\n    PreviousRingData = state_get(ring_data, PrevState),\n    CurrentRingData = state_get(ring_data, CurState),\n    BestState = previous_or_current(PrevState, CurState),\n    Best = case BestState of\n               CurState -> current_data;\n               PrevState -> previous_data\n           end,\n    KeyValueList =\n        [{to_string(state_get(instance, CurState)), \"\"},\n         {\"best\",                Best},\n         {\"leader\",              state_get(leader, CurState)},\n         %% previous round\n         {\"prev_round\",          state_get(round, PrevState)},\n         {\"prev_merged\",         state_get(merged, PrevState)},\n         {\"prev_conv_avg_count\", state_get(convergence_count, PrevState)},\n         {\"prev_size_ldr\",       get_current_estimate(size_inv, PreviousRingData)},\n         {\"prev_size_kr\",        calc_size_kr(PreviousRingData)}\n        ]\n        ++\n            lists:flatten(\n          [ begin\n                [\n                 {\"prev_name\",           data_get(name, LoadData)},\n                 {\"prev_avg\",            get_current_estimate(avg, LoadData)},\n                 {\"prev_min\",            data_get(min, LoadData)},\n                 {\"prev_max\",            data_get(max, LoadData)},\n                 {\"prev_stddev\",         calc_stddev(LoadData)},\n                 {\"prev_histo\",          to_string(data_get(histo, LoadData))}\n                ]\n            end\n            || PreviousLoadData =/= unknown, LoadData <- PreviousLoadData])\n        ++\n        [\n         %% current round\n         {\"cur_round\",           state_get(round, CurState)},\n         {\"cur_merged\",          state_get(merged, CurState)},\n         {\"cur_conv_avg_count\",  state_get(convergence_count, CurState)},\n         {\"cur_size_ldr\",        get_current_estimate(size_inv, CurrentRingData)},\n         {\"cur_size_kr\",         calc_size_kr(CurrentRingData)}\n        ]\n        ++\n            lists:flatten(\n          [ begin\n                [\n                 {\"cur_name\",           data_get(name, LoadData)},\n                 {\"cur_avg\",            get_current_estimate(avg, LoadData)},\n                 {\"cur_min\",            data_get(min, LoadData)},\n                 {\"cur_max\",            data_get(max, LoadData)},\n                 {\"cur_stddev\",         calc_stddev(LoadData)},\n                 {\"cur_histo\",          to_string(data_get(histo, LoadData))}\n                ]\n            end\n            || LoadData <- CurrentLoadData]),\n    {KeyValueList, FullState}.\n\n\n%% @doc Shutd down the gossip_load module. <br/>\n%%      Called by the gossip module upon stop_gossip_task(CBModule).\n-spec shutdown(State::full_state()) -> {ok, shutdown}.\nshutdown(_FullState) ->\n    % nothing to do\n    {ok, shutdown}.\n\n\n%%------------------- Callback Functions: Helpers ------------------%%\n\n-spec request_node_details(State::state()) -> ok.\nrequest_node_details(State) ->\n    % get state of dht node\n    case pid_groups:get_my(dht_node) of\n        failed ->\n            % our dht_node died and was removed from pid_groups\n            log:log(warn, \"[ ~w ~w ] request_node_details from ~p failed\",\n                   [state_get(instance, State), comm:this(), pid_groups:my_groupname()]),\n            ok;\n        DHT_Node ->\n            EnvPid = comm:reply_as(comm:this(), 3, {cb_msg, state_get(instance, State), '_'}),\n            comm:send_local(DHT_Node,\n                            {get_node_details, EnvPid, [load, load2, load3, db, my_range]})\n    end.\n\n\n-spec new_round(NewRound::round(), PrevState::state(), CurState::state())\n        -> {ok, FullState::full_state()}.\nnew_round(NewRound, PrevState, CurState) ->\n    % Only replace prev round with current round if current has converged.\n    % Cases in which current round has not converged: e.g. late joining, sleeped/paused.\n    % ConvergenceTarget should be less than covergence_count_new_round,\n    % otherwise non leader groups seldom replace prev_load_data\n    {PrevState1, CurState1} = case has_converged(convergence_count_best_values(), CurState) of\n        true ->\n            % make the current state the previous state\n            NewState1 = state_new(CurState),\n            {CurState, NewState1};\n        false ->\n            % make the old previous state the new previous state and discard the current state\n            ConvCount = state_get(convergence_count, CurState),\n            CurRound = state_get(round, CurState),\n            log:log(warn(), \"[ ~w ] Entering new round (~w), but current round (~w) \"++\n                \"has not converged. Convergence Count: ~w\", [?MODULE, NewRound, CurRound, ConvCount]),\n            NewState1 = state_new(CurState),\n            {PrevState, NewState1}\n    end,\n    CurState2 =  state_set(round, NewRound, CurState1),\n    {ok, {PrevState1, CurState2}}.\n\n\n-spec has_converged(TargetConvergenceCount::pos_integer(), CurState::state()) -> boolean().\nhas_converged(TargetConvergenceCount, CurState) ->\n    CurrentConvergenceCount = state_get(convergence_count, CurState),\n    CurrentConvergenceCount >= TargetConvergenceCount.\n\n\n-spec finish_request(CurState::state()) -> state().\nfinish_request(CurState) ->\n    case state_get(leader, CurState) of\n        true ->\n            LoadDataList = state_get(load_data_list, CurState),\n            Histo = data_get(histo, get_default_load_data(LoadDataList)),\n            Requestor = state_get(requestor, CurState),\n            comm:send(Requestor, {histogram, Histo}),\n            gossip:stop_gossip_task(state_get(instance, CurState));\n        false ->\n            do_nothing\n    end,\n    CurState.\n\n\n-spec update_convergence_count(OldData::data(), NewData::data(), CurState::state()) -> state().\nupdate_convergence_count({OldLoadList, OldRing}, {NewLoadList, NewRing}, CurState) ->\n\n    % check whether all average based values changed less than epsilon percent\n    AvgChangeEpsilon = convergence_epsilon(),\n    HasConvergedFun =\n        fun(AvgType, Old, New) ->\n                OldValue = calc_current_estimate(data_get(AvgType, Old)),\n                NewValue = calc_current_estimate(data_get(AvgType, New)),\n                log:log(debug, \"[ ~w ] ~w: OldValue: ~w, New Value: ~w\",\n                        [state_get(instance, CurState), AvgType, OldValue, NewValue]),\n                calc_change(OldValue, NewValue) < AvgChangeEpsilon\n        end,\n\n    %% Ring convergence\n    HaveConverged1 = HasConvergedFun(size_inv, OldRing, NewRing) andalso\n                        HasConvergedFun(avg_kr, OldRing, NewRing),\n\n    %% Load convergence\n    HaveConverged2 =\n        HaveConverged1 andalso\n            lists:all(\n              fun({OldLoad, NewLoad, AvgType}) ->\n                      ?ASSERT(data_get(name, OldLoad) =:= data_get(name, NewLoad)),\n                      HasConvergedFun(AvgType, OldLoad, NewLoad)\n              end,\n              [{OldLoad, NewLoad, AvgType}\n               || {OldLoad, NewLoad} <- lists:zip(OldLoadList, NewLoadList),\n                  % =/= {load_data, _Name, skip}\n                  not (tuple_size(NewLoad) =:= 3 andalso element(3, NewLoad) =:= skip),\n                  AvgType            <- [avg, avg2]\n              ]),\n\n    log:log(debug, \"Averages have converged: ~w\", [HaveConverged2]),\n\n    %% TODO: Fix convergence counting for histograms with skipped values\n    %% (atm, all histogrames are ignored in regard to convergence counting)\n\n    %% HistoModulesConverged =\n    %%     [ begin\n    %%         case lists:member(data_get(name, NewLoad), Modules) of\n    %%              true -> true;\n    %%              false ->\n    %%                   ?ASSERT(data_get(name, OldLoad) =:= data_get(name, NewLoad)),\n    %%                   % Combine the avg values of the buckets of two histograms into one tuple list\n    %%                   Combine = fun ({Interval, AvgOld}, {Interval, AvgNew}) -> {AvgOld, AvgNew} end,\n    %%                   CompList = lists:zipwith(Combine,\n    %%                                            data_get(histo, OldLoad), data_get(histo, NewLoad)),\n    %%\n    %%                   % check that all the buckets of histogram have changed less than epsilon percent\n    %%                   Fun1 = fun({OldAvg, NewAvg}) ->\n    %%                                  OldEstimate = calc_current_estimate(OldAvg),\n    %%                                  NewEstimate = calc_current_estimate(NewAvg),\n    %%                                  _HasConverged = calc_change(OldEstimate, NewEstimate) < AvgChangeEpsilon\n    %%                          end,\n    %%                   _HaveConverged2 = lists:all(Fun1, CompList)\n    %%         end\n    %%       end\n    %%       || {OldLoad, NewLoad} <- lists:zip(OldLoadList, NewLoadList)\n    %%     ],\n    %%\n    %% HaveConverged3 = HaveConverged2 andalso lists:all(fun(X) -> X end, HistoModulesConverged),\n    HaveConverged3 = HaveConverged2,\n    %% log:log(debug, \"Histogram has converged: ~w\", [self(), HaveConverged2]),\n\n    if HaveConverged3 ->\n           state_update(convergence_count, fun inc/1, CurState);\n       true ->\n           state_set(convergence_count, 0, CurState)\n    end.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% State of gossip_load: Getters, Setters and Helpers\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-compile({inline, [state_get/2, state_set/3, state_update/3]}).\n\n%% @doc Create a new state with information from another state record.\n%%      This is used to build a new state from the current state when a new round\n%%      is entered. Some values (mainly counters) are resetted, others are taken\n%%      from the Parent state.\n-spec state_new(Parent::state()) -> state().\nstate_new(Parent) ->\n    #state{\n        load_data_list = load_data_list_new(),\n        ring_data = ring_data_new(),\n        leader = state_get(leader, Parent),\n        range = state_get(range, Parent),\n        no_of_buckets = state_get(no_of_buckets, Parent),\n        request = state_get(request, Parent),\n        requestor = state_get(requestor, Parent),\n        instance = state_get(instance, Parent)\n    }.\n\n\n-spec state_set (status, status(), state()) -> state();\n                (instance, instance(), state()) -> state();\n                (load_data_list, load_data_list(), state()) -> state();\n                (ring_data, ring_data(), state()) -> state();\n                (leader, boolean(), state()) -> state();\n                (range, intervals:interval(), state()) -> state();\n                (request, boolean(), state()) -> state();\n                (requestor, none | comm:mypid(), state()) -> state();\n                (no_of_buckets, non_neg_integer(), state()) -> state();\n                (round, round(), state()) -> state();\n                (merged, non_neg_integer(), state()) -> state();\n                (convergence_count, non_neg_integer(), state()) -> state().\nstate_set(Key, Value, State) when is_record(State, state) ->\n    case Key of\n        status -> State#state{status = Value};\n        instance -> State#state{instance = Value};\n        load_data_list -> State#state{load_data_list = Value};\n        ring_data -> State#state{ring_data = Value};\n        leader -> State#state{leader = Value};\n        range -> State#state{range = Value};\n        request -> State#state{request = Value};\n        requestor -> State#state{requestor = Value};\n        no_of_buckets -> State#state{no_of_buckets = Value};\n        round -> State#state{round = Value};\n        merged -> State#state{merged = Value};\n        convergence_count -> State#state{convergence_count = Value}\n    end.\n\n-spec state_set([ {status, status()} | {instance, instance()} |\n                  {load_data_list, load_data_list()} | {ring_data, ring_data()} |\n                  {leader, boolean()} | {range, intervals:interval()} |\n                  {request, boolean()} | {requestor, none | comm:mypid()} |\n                  {no_of_buckets, non_neg_integer()} | {round, round()} |\n                  {merged, non_neg_integer()} | {convergence_count, non_neg_integer()}, ...], State::state()) -> state().\nstate_set(KeyValueTupleList, State) when is_record(State, state) andalso is_list(KeyValueTupleList) ->\n    Fun = fun ({Key, Value}, OldState) -> state_set(Key, Value, OldState) end,\n    lists:foldl(Fun, State, KeyValueTupleList).\n\n-spec state_get (any(), unknown) -> unknown;\n                (status, state()) -> status();\n                (instance, state()) -> instance();\n                (load_data_list, state()) -> load_data_list();\n                (ring_data, state()) -> ring_data();\n                (leader, state()) -> boolean();\n                (range, state()) -> intervals:interval();\n                (request, state()) -> boolean();\n                (requestor, state()) -> none | comm:mypid();\n                (no_of_buckets, state()) -> non_neg_integer();\n                (round, state()) -> non_neg_integer();\n                (merged, state()) -> non_neg_integer();\n                (convergence_count, state()) -> non_neg_integer().\nstate_get(_Key, unknown) -> unknown;\nstate_get(status, #state{status=Status}) -> Status;\nstate_get(instance, #state{instance=Instance}) -> Instance;\nstate_get(load_data_list, #state{load_data_list=LoadDataList}) -> LoadDataList;\nstate_get(ring_data, #state{ring_data=RingData}) -> RingData;\nstate_get(leader, #state{leader=Leader}) -> Leader;\nstate_get(range, #state{range=Range}) -> Range;\nstate_get(request, #state{request=Request}) -> Request;\nstate_get(requestor, #state{requestor=Requestor}) -> Requestor;\nstate_get(no_of_buckets, #state{no_of_buckets=NoOfBuckets}) -> NoOfBuckets;\nstate_get(round, #state{round=Round}) -> Round;\nstate_get(merged, #state{merged=Merged}) -> Merged;\nstate_get(convergence_count, #state{convergence_count=ConvergenceCount}) -> ConvergenceCount.\n\n\n-spec state_update(Key::state_key(), Fun::fun(), State::state()) -> state().\nstate_update(Key, Fun, State) ->\n    Value = state_get(Key, State),\n    NewValue = apply(Fun, [Value]),\n    state_set(Key, NewValue, State).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Load Data and Ring Data\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-compile({inline, [load_data_new/1, load_data_list_new/0, ring_data_new/0,\n                   divide2/2, merge_failed_exch_data/2, merge_load_data/3, merge_avg/3]}).\n\n%% @doc creates a new load_data (for internal use by gossip_load.erl)\n-spec load_data_new(Module::atom()) -> load_data_uninit().\nload_data_new(Module) ->\n    #load_data{name = Module}.\n\n-spec load_data_list_new() -> [load_data_uninit(), ...].\nload_data_list_new() ->\n    AdditionalModules = config:read(gossip_load_additional_modules),\n    Modules = [?DEFAULT_MODULE | AdditionalModules],\n    [load_data_new(Module) || Module <- Modules].\n\n-spec ring_data_new() -> ring_data_uninit().\nring_data_new() ->\n    #ring_data{}.\n\n-spec get_default_load_data(unknown) -> unknown;\n                           (load_data_list()) -> load_data() | load_data_skipped().\nget_default_load_data(unknown) -> unknown;\nget_default_load_data(LoadDataList) ->\n    %% name is the second element of record tuple\n    case lists:keyfind(?DEFAULT_MODULE, 2, LoadDataList) of\n        false    -> throw(default_load_module_not_available);\n        LoadData -> LoadData\n    end.\n\n%% @doc Gets information from a load_data record.\n%%      Allowed keys include:\n%%      <ul>\n%%        <li>avg = average load,</li>\n%%        <li>min = minimum load,</li>\n%%        <li>max = maximum load,</li>\n%%        <li>avg2 = average of load^2,</li>\n%%        <li>size_inv = 1 / size_ldr,</li>\n%%        <li>avg_kr = average key range (distance between nodes in the address space),</li>\n%%      </ul>\n%%      See type spec for details on which keys are allowed on which records.\n-spec data_get(any(), unknown) -> unknown;\n             (name, load_data() | load_data_skipped()) -> atom();\n             (avg, load_data()) -> avg();\n             (avg2, load_data()) -> avg();\n             (min, load_data()) -> min();\n             (max, load_data()) -> max();\n             (histo, load_data()) -> histogram();\n             (size_inv, ring_data()) -> avg();\n             (avg_kr, ring_data()) -> avg_kr();\n             (avg | avg2 | min | max | histo | size_inv | avg_kr, load_data_skipped()) -> unknown.\ndata_get(_Key, unknown) -> unknown;\ndata_get(name, {load_data, Name, skip}) ->\n    Name;\ndata_get(_Key, {load_data, _Module, skip}) -> unknown;\ndata_get(name, LoadData=#load_data{name = Name}) when is_record(LoadData, load_data) ->\n    Name;\ndata_get(avg, #load_data{avg = Avg}) ->\n    Avg;\ndata_get(avg2, #load_data{avg2 = Avg2}) ->\n    Avg2;\ndata_get(min, #load_data{min = Min}) ->\n    Min;\ndata_get(max, #load_data{max = Max}) ->\n    Max;\ndata_get(histo, #load_data{histo = Histo}) ->\n    Histo;\ndata_get(size_inv, #ring_data{size_inv = SizeInv}) ->\n    SizeInv;\ndata_get(avg_kr, #ring_data{avg_kr = AvgKr}) ->\n    AvgKr.\n\n-spec get_current_estimate(Key::avg | avg2,\n                           unknown | load_data() | load_data_skipped()) -> float() | unknown;\n                          (Key::size_inv | avg_kr,\n                           unknown | ring_data()) -> float() | unknown.\nget_current_estimate(_Key, unknown) -> unknown;\nget_current_estimate(_Key, {load_data, _Module, skip}) -> unknown;\nget_current_estimate(avg, #load_data{avg=Avg}) ->\n    calc_current_estimate(Avg);\nget_current_estimate(avg2, #load_data{avg2=Avg2}) ->\n    calc_current_estimate(Avg2);\nget_current_estimate(size_inv, #ring_data{size_inv = SizeInv}) ->\n    calc_current_estimate(SizeInv);\nget_current_estimate(avg_kr, #ring_data{avg_kr = AvgKr}) ->\n    calc_current_estimate(AvgKr).\n\n\n%% @doc Sets information in a load_data record.\n%%      Allowed keys are:\n%%      <ul>\n%%        <li>avg = average load,</li>\n%%        <li>avg2 = average of load^2,</li>\n%%        <li>size_inv = 1 / size_ldr,</li>\n%%        <li>avg_kr = average key range (distance between nodes in the address space),</li>\n%%        <li>min = minimum load,</li>\n%%        <li>max = maximum load,</li>\n%%      </ul>\n-spec data_set(avg, avg(), load_data()) -> load_data();\n              (min, min(), load_data()) -> load_data();\n              (max, max(), load_data()) -> load_data();\n              (avg2, avg(), load_data()) -> load_data();\n              (histo, histogram(), load_data()) -> load_data();\n              (size_inv, avg(), ring_data()) -> ring_data();\n              (avg_kr, avg_kr(), ring_data()) -> ring_data().\ndata_set(Key, Value, LoadData) when is_record(LoadData, load_data) ->\n    case Key of\n        avg -> LoadData#load_data{avg = Value};\n        avg2 -> LoadData#load_data{avg2 = Value};\n        min -> LoadData#load_data{min = Value};\n        max -> LoadData#load_data{max = Value};\n        histo -> LoadData#load_data{histo = Value}\n    end;\ndata_set(Key, Value, RingData) when is_record(RingData, ring_data) ->\n    case Key of\n         size_inv -> RingData#ring_data{size_inv = Value};\n         avg_kr -> RingData#ring_data{avg_kr = Value}\n    end.\n\n%% @doc Prepares a load_data record for sending it to a peer and updates the\n%%      load_data of self accordingly. Skips load data skipped in the given\n%%      state or from the MoreSkippedModules list.\n-spec prepare_data(MoreSkippedModules::[atom()], state())\n        -> {{load_data_list(), ring_data()}, state()}.\nprepare_data(MoreSkippedModules, State) ->\n    LoadDataNew =\n        [begin\n             LoadName = data_get(name, LoadData),\n             LoadSkipped = {load_data, LoadName, skip},\n             case LoadData =:= LoadSkipped orelse\n                      lists:member(LoadName, MoreSkippedModules) of\n                  true -> LoadData;\n                  false ->\n                     LoadData1 = divide2(avg, LoadData),\n                     LoadData2 = divide2(avg2, LoadData1),\n\n                     % Min and Max\n                     % do_nothing,\n\n                     % Histogram\n                     _LoadData3 = divide2(LoadData2)\n             end\n         end || LoadData <- state_get(load_data_list, State)],\n\n    State1 = state_set(load_data_list, LoadDataNew, State),\n\n    % Averages ring data\n    RingData1 = divide2(size_inv, state_get(ring_data, State1)),\n    RingData2 = divide2(avg_kr, RingData1),\n\n    State2 = state_set(ring_data, RingData2, State1),\n\n    {{LoadDataNew, RingData2}, State2}.\n\n\n%% @doc Cut the average values and associated weights in half.\n-spec divide2(AvgType:: avg | avg2, MyData::load_data()) -> load_data();\n             (AvgType:: size_inv | avg_kr, MyData::ring_data()) -> ring_data().\ndivide2(AvgType, MyData) ->\n    case data_get(AvgType, MyData) of\n        unknown -> MyData;\n        {Avg, Weight} ->\n            NewAvg = Avg/2,\n            NewWeight = Weight/2,\n            data_set(AvgType, {NewAvg, NewWeight}, MyData)\n    end.\n\n\n%% @doc Merges the load_data from a failed exchange back into self's load_data.\n%%      Does not update the merge and convergence count.\n-spec merge_failed_exch_data(Data::data(), CurState::state()) -> {data(), state()}.\nmerge_failed_exch_data(Data, State) ->\n    merge_load_data(noupdate, Data, State).\n\n\n%% @doc Merges the given load_data into self's load_data of the current or\n%%      previous round. State can be current or previous state.\n-spec merge_load_data(OtherData::data(), CurState::state()) -> {data(), state()}.\nmerge_load_data(OtherData, State) ->\n    merge_load_data(update, OtherData, State).\n\n%% @doc Helper function. Merge the given load_data with the given states load_data.\n%%      Set update is set, the merge and convergence count is updated, otherwise not.\n-spec merge_load_data(Update::update | noupdate, OtherData::data(),\n    CurState::state()) -> {data(), state()}.\nmerge_load_data(Update, {OtherLoadList, OtherRing}, State) ->\n\n    MyLoadList = state_get(load_data_list, State),\n    LoadDataListNew =\n        [begin\n             LoadName = data_get(name, MyLoad1),\n             ?ASSERT(LoadName =:= data_get(name, OtherLoad)),\n             LoadSkipped = {load_data, LoadName, skip},\n             case MyLoad1 =:= LoadSkipped orelse OtherLoad =:= LoadSkipped of\n                  true -> MyLoad1;\n                  false ->\n                     % Averages load\n                     MyLoad2 = merge_avg(avg, MyLoad1, OtherLoad),\n                     MyLoad3 = merge_avg(avg2, MyLoad2, OtherLoad),\n\n                     % Min\n                     MyNewMin = erlang:min(data_get(min, MyLoad3),\n                                           data_get(min, OtherLoad)),\n                     MyLoad4 = data_set(min, MyNewMin, MyLoad3),\n\n                     % Max\n                     MyNewMax = erlang:max(data_get(max, MyLoad4),\n                                           data_get(max, OtherLoad)),\n                     MyLoad5 = data_set(max, MyNewMax, MyLoad4),\n\n                     % Histogram\n                     _MyLoad5 = merge_histo(MyLoad5, OtherLoad)\n             end\n         end || {MyLoad1, OtherLoad} <- lists:zip(MyLoadList, OtherLoadList)],\n\n    State1 = state_set(load_data_list, LoadDataListNew, State),\n\n    %% Averages ring\n    MyRing1 = state_get(ring_data, State1),\n    MyRing2 = merge_avg(size_inv, MyRing1, OtherRing),\n    MyRing3 = merge_avg(avg_kr, MyRing2, OtherRing),\n    State2 = state_set(ring_data, MyRing3, State1),\n\n    NewData = {LoadDataListNew, MyRing3},\n    State4 = case Update of\n        update ->\n            State3 = state_update(merged, fun inc/1, State2),\n            _State4 = update_convergence_count({MyLoadList, MyRing1}, NewData, State3);\n        noupdate -> State2\n    end,\n    {NewData, State4}.\n\n\n%% @doc Helper function. Merges the given type of average from the given\n%%      load_data records.\n-spec merge_avg(AvgType:: avg | avg2, Data1::load_data_uninit(), Data2::load_data_uninit())\n                -> load_data_uninit();\n               (AvgType:: size_inv | avg_kr, Data1::ring_data_uninit(), Data2::ring_data_uninit())\n                -> ring_data_uninit().\nmerge_avg(AvgType, Data1, Data2) ->\n    %% avg values can be 'unknown' instead of {Load, Weight}\n    %% unknown values are ignored\n    NewAvg = case {data_get(AvgType, Data1), data_get(AvgType, Data2)} of\n                 {unknown, unknown} -> unknown;\n                 {Avg1,    unknown} -> Avg1;\n                 {unknown, Avg2   } -> Avg2;\n                 {Avg1,    Avg2   } -> {AvgLoad1, AvgWeight1} = Avg1,\n                                       {AvgLoad2, AvgWeight2} = Avg2,\n                                       {AvgLoad1 + AvgLoad2, AvgWeight1 + AvgWeight2}\n             end,\n    data_set(AvgType, NewAvg, Data1).\n\n\n%% @doc Returns the previous load data if the current load data has not\n%%      sufficiently converged, otherwise returns the current load data.\n%%      In round 0 (when no previous load data exists) always the current load\n%%      data is returned.\n%%\n%%      PrevState           CurState                            Return\n%%      -----------------------------------------------------------------\n%%      unknown             not initialised                     CurState\n%%      unknown             initialized (but not converged)     CurState\n%%      unknown             converged                           CurState\n%%      converged           not initialised                     PrevState\n%%      converged           initialized (but not converged)     PrevState\n%%      converged           converged                           CurState\n-spec previous_or_current(PrevState::state() | unknown, CurState::state()) -> BestState::state().\nprevious_or_current(unknown, CurState) when is_record(CurState, state) ->\n    CurState;\n\nprevious_or_current(PrevState, CurState) when is_record(PrevState, state) andalso is_record(CurState, state) ->\n    case has_converged(convergence_count_best_values(), CurState) of\n        false -> PrevState;\n        true -> CurState\n    end.\n\n\n%% @doc Replaces skipped metrics in My load list with non-skipped values from\n%%      the other load list if possible.\n-spec replace_skipped(Other::LoadDataList, My::LoadDataList) -> LoadDataList\n        when is_subtype(LoadDataList, [load_data() | load_data_skipped()]).\nreplace_skipped([], []) ->\n    [];\nreplace_skipped([H | OtherL], [{load_data, Metric, skip} | MyL]) ->\n    ?ASSERT(Metric =:= data_get(name, H)),\n    [H | replace_skipped(OtherL, MyL)];\nreplace_skipped([_OtherH | OtherL], [MyH | MyL]) ->\n    ?ASSERT(data_get(name, _OtherH) =:= data_get(name, MyH)),\n    [MyH | replace_skipped(OtherL, MyL)].\n\n\n%%---------------------------- Histogram ---------------------------%%\n\n-compile({inline, [init_histo/3, divide2/1, merge_histo/2, merge_bucket/2]}).\n\n-spec init_histo(Module::module(), node_details:node_details(),\n                 CurState::state()) -> histogram().\ninit_histo(Module, NodeDetails, CurState) ->\n    NumberOfBuckets = state_get(no_of_buckets, CurState),\n    Module:init_histo(NodeDetails, NumberOfBuckets).\n\n-spec divide2(LoadData::load_data()) -> NewLoadData::load_data();\n             (Bucket::bucket()) -> NewBucket::bucket().\n% divide the histogram in the given load_data\ndivide2(LoadData) when is_record(LoadData, load_data) ->\n    Histogram = data_get(histo, LoadData),\n    Histogram1 = lists:map(fun divide2/1, Histogram),\n    data_set(histo, Histogram1, LoadData);\n\n% divide a single bucket of a histogram\ndivide2({Interval, unknown}) ->\n    {Interval, unknown};\n\n% divide a single bucket of a histogram\ndivide2({Interval, {Value, Weight}}) ->\n    {Interval, {Value/2, Weight/2}}.\n\n\n-spec merge_histo(MyData::load_data(), OtherData::load_data()) -> load_data().\nmerge_histo(MyData, OtherData) when is_record(MyData, load_data)\n        andalso is_record(OtherData, load_data)->\n    MyHisto = data_get(histo, MyData),\n    OtherHisto = data_get(histo, OtherData),\n    MergedHisto = lists:zipwith(fun merge_bucket/2, MyHisto, OtherHisto),\n    data_set(histo, MergedHisto, MyData).\n\n\n\n\n%% @doc Merge two buckets of a histogram.\n-spec merge_bucket(Bucket1::bucket(), Bucket2::bucket()) -> bucket().\nmerge_bucket({Key, unknown}, {Key, unknown}) ->\n    {Key, unknown};\n\nmerge_bucket({Key, unknown}, {Key, Avg2}) ->\n    {Key, Avg2};\n\nmerge_bucket({Key, Avg1}, {Key, unknown}) ->\n    {Key, Avg1};\n\nmerge_bucket({Key, {Value1, Weight1}}, {Key, {Value2, Weight2}}) ->\n    {Key, {Value1+Value2, Weight1+Weight2}}.\n\n\n\n%%---------------------------- Load Info ---------------------------%%\n\n%% @doc Builds a load_info record from the given state (current or previous state)\n-spec get_load_info(State::state()) -> load_info().\nget_load_info(State) ->\n    LoadDataList = state_get(load_data_list, State),\n    %% select default load data for output\n    {value, DefaultLoadData, OtherLoadData} =\n        lists:keytake(?DEFAULT_MODULE, 2, LoadDataList),\n    RingData = state_get(ring_data, State),\n    OtherLoadInfo = [#load_info_other {\n                                       name   = data_get(name, LoadData),\n                                       avg    = get_current_estimate(avg, LoadData),\n                                       stddev = calc_stddev(LoadData),\n                                       min    = data_get(min, LoadData),\n                                       max    = data_get(max, LoadData)\n                                      }\n                    || LoadData <- OtherLoadData],\n    _LoadInfo =\n        #load_info{ avg       = get_current_estimate(avg, DefaultLoadData),\n                    stddev    = calc_stddev(DefaultLoadData),\n                    size_ldr  = get_current_estimate(size_inv, RingData),\n                    size_kr   = calc_size_kr(RingData),\n                    min       = data_get(min, DefaultLoadData),\n                    max       = data_get(max, DefaultLoadData),\n                    merged    = state_get(merged, State),\n                    other     = OtherLoadInfo\n                  }.\n\n\n%% @doc Gets the value to the given key from the given load_info record.\n-spec load_info_get(Key::avgLoad, LoadInfoRecord::load_info()) -> unknown | float();\n                   (Key::stddev, LoadInfoRecord::load_info()) -> unknown | float();\n                   (Key::size_ldr, LoadInfoRecord::load_info()) -> unknown | float();\n                   (Key::size_kr, LoadInfoRecord::load_info()) -> unknown | float();\n                   (Key::size, LoadInfoRecord::load_info()) -> unknown | float();\n                   (Key::minLoad, LoadInfoRecord::load_info()) -> unknown | min();\n                   (Key::maxLoad, LoadInfoRecord::load_info()) -> unknown | max();\n                   (Key::merged, LoadInfoRecord::load_info()) -> unknown | merged();\n                   (Key::other, LoadInfoRecord::load_info()) -> [load_info_other()].\nload_info_get(Key, #load_info{avg=Avg, stddev=Stddev, size_ldr=SizeLdr,\n        size_kr=SizeKr, min=Min, max=Max, merged=Merged, other = Other}) ->\n    case Key of\n        avgLoad -> Avg;\n        stddev -> Stddev;\n        size_ldr -> SizeLdr;\n        size_kr -> SizeKr;\n        size ->\n            % favor key range based calculations over leader-based\n            case SizeKr of\n                unknown -> SizeLdr;\n                _       -> SizeKr\n            end;\n        minLoad -> Min;\n        maxLoad -> Max;\n        merged -> Merged;\n        other  -> Other\n    end.\n\n%% @doc Gets values from a load_info_other record which can be found in the load_info record.\n-spec load_info_other_get(Key::avgLoad, Module::atom(), LoadInfoRecord::load_info()) -> unknown | float();\n                         (Key::stddev, Module::atom(), LoadInfoRecord::load_info()) -> unknown | float();\n                         (Key::minLoad, Module::atom(), LoadInfoRecord::load_info()) -> unknown | min();\n                         (Key::maxLoad, Module::atom(), LoadInfoRecord::load_info()) -> unknown | max().\nload_info_other_get(Key, Module, #load_info{other = Other}) ->\n    case lists:keyfind(Module, 2, Other) of\n        false -> unknown;\n        LoadInfoOther -> load_info_other_get(Key, LoadInfoOther)\n    end.\n\n-spec load_info_other_get(Key::name, load_info_other()) -> atom();\n                   (Key::avgLoad, load_info_other()) -> unknown | float();\n                   (Key::stddev, load_info_other()) -> unknown | float();\n                   (Key::minLoad, load_info_other()) -> unknown | min();\n                   (Key::maxLoad, load_info_other()) -> unknown | max().\nload_info_other_get(Key, #load_info_other{name = Name, avg=Avg,\n                                          stddev=Stddev, min=Min, max=Max}) ->\n    case Key of\n        name -> Name;\n        avgLoad -> Avg;\n        stddev -> Stddev;\n        minLoad -> Min;\n        maxLoad -> Max\n    end.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Calculations on Load Data Values\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-compile({inline, [calc_current_estimate/1, calc_current_estimate/2]}).\n\n%% @doc Calculate the current estimate from a given {Value, Weight} tuple.\n-spec calc_current_estimate(unknown | avg()) -> float() | unknown.\ncalc_current_estimate(unknown) -> unknown;\n\ncalc_current_estimate({Value, Weight}) ->\n    calc_current_estimate(Value, Weight).\n\n\n%% @doc Calculate the current estimate from the given value and, weight.\n-spec calc_current_estimate(Value::float(), Weight::float()) -> float() | unknown.\ncalc_current_estimate(Value, Weight) ->\n    try Value / Weight\n    catch\n        error:badarith -> unknown\n    end.\n\n\n%% @doc Calculates the change in percent from the Old value to the New value.\n-spec calc_change(OldValue::float() | unknown, NewValue::float() | unknown) -> float().\ncalc_change(unknown, _NewValue) -> 100.0;\ncalc_change(_OldValue, unknown) -> 100.0;\ncalc_change(Value, Value)       -> 0.0;\n%calc_change(0, _NewValue)       -> 100.0; we only have floats!\ncalc_change(0.0, _NewValue)     -> 100.0;\ncalc_change(OldValue, NewValue) ->\n    ((OldValue + abs(NewValue - OldValue)) * 100.0 / OldValue) - 100.\n\n\n%% @doc Calculates the difference between the key of a node and its\n%%      predecessor. If the second is larger than the first it wraps around and\n%%      thus the difference is the number of keys from the predecessor to the\n%%      end (of the ring) and from the start to the current node.\n%%      Pre: MyRange is continuous\n-spec calc_initial_avg_kr(MyRange::intervals:interval()) -> avg_kr() | unknown.\ncalc_initial_avg_kr(MyRange) ->\n    try\n        % get_bounds([]) throws 'no bounds in empty interval'\n        {_, PredKey, MyKey, _} = intervals:get_bounds(MyRange),\n        % we don't know whether we can subtract keys of type ?RT:key()\n        % -> try it and if it fails, return unknown\n        {?RT:get_range(PredKey, MyKey), 1.0}\n    catch\n        throw:not_supported -> unknown;\n        throw:'no bounds in empty interval' -> unknown\n    end.\n\n\n%% @doc Extracts and calculates the standard deviation from the load_data record\n-spec calc_stddev(load_data() | load_data_skipped()) -> unknown | float().\n\ncalc_stddev({load_data, _Module, skip}) -> unknown;\n\ncalc_stddev(Data) when is_record(Data, load_data) ->\n    Avg = get_current_estimate(avg, Data),\n    Avg2 = get_current_estimate(avg2, Data),\n    case (Avg =:= unknown) orelse (Avg2 =:= unknown) of\n        true -> unknown;\n        false ->\n            Tmp = Avg2 - (Avg * Avg),\n            case (Tmp >= 0) of\n                true  -> math:sqrt(Tmp);\n                false -> unknown\n            end\n    end.\n\n\n%% @doc Extracts and calculates the size_kr field from the load_data record\n-spec calc_size_kr(unknown | ring_data()) -> unknown | float().\ncalc_size_kr(unknown) -> unknown;\n\ncalc_size_kr(Data) when is_record(Data, ring_data) ->\n    AvgKR = get_current_estimate(avg_kr, Data),\n    try\n        if\n            AvgKR =< 0        -> 1.0;\n            AvgKR =:= unknown -> unknown;\n            true              -> ?RT:n() / AvgKR\n        end\n    catch % ?RT:n() might throw\n        throw:not_supported -> unknown\n    end.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Misc\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-compile({inline, inc/1}).\n\n%% @doc Increments given value with one.\n-spec inc(Value::integer()) -> integer().\ninc(Value) ->\n    Value+1.\n\n\n-spec to_string(unknown) -> unknown;\n               (Histogram::histogram()) -> string();\n               (LoadInfo::load_info()) -> string();\n               (Instance::instance()) -> string().\nto_string(unknown) -> unknown;\n\nto_string(#load_info{avg=Avg, stddev=Stddev, size_ldr=SizeLdr, size_kr=SizeKr,\n        min=Min, max=Max, merged=Merged}) ->\n    Labels1 = [\"avg\", \"stddev\", \"size_ldr\", \"size_kr\"],\n    Values1 = [Avg, Stddev, SizeLdr, SizeKr],\n    LVZib = lists:zip(Labels1, Values1),\n    Fun = fun({Label, unknown}, AccIn) -> AccIn ++ Label ++ \": unknown, \";\n             ({Label, Value}, AccIn) ->\n                AccIn ++ io_lib:format(\"~s: ~.2f, \", [Label, Value]) end,\n    L1 = lists:foldl(Fun, \"\", LVZib),\n    L2 = io_lib:format(\"min: ~w, max: ~w, merged: ~w\", [Min, Max, Merged]),\n    lists:flatten(L1++L2);\n\nto_string(Histogram) when is_list(Histogram) ->\n    Values = [ calc_current_estimate(VWTuple) || { _, VWTuple } <- Histogram ],\n    Fun = fun (unknown, AccIn) -> AccIn ++ \"    unknown\";\n              (Value, AccIn) ->  AccIn ++ io_lib:format(\" ~10.2. f\", [Value]) end,\n    HistoString = lists:foldl(Fun, \"[ \", Values) ++ \" ]\",\n    lists:flatten(HistoString);\n\nto_string({ModuleName, Id}) ->\n    lists:flatten(io_lib:format(\"~w:~w\", [ModuleName, Id])).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% For Testing\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n\n\n%% @doc Creates a state record with greatly reduced variety in the round numbers\n%%      to reduce warings.\n%%      Used as value_creator in tester.erl (property testing).\n-spec tester_create_state(status(), instance(), load_data_list(), ring_data(),\n                          Leader::boolean(), Range::intervals:non_empty_interval(),\n                          Request::boolean(), NoOfBuckets::histogram_size(), round(),\n                          Merged::non_neg_integer(),\n                          ConvergenceCount::non_neg_integer()) -> state().\ntester_create_state(Status, Instance, LoadDataList, RingData, Leader, Range,\n                    Request, NoOfBuckets, Round, Merged, ConvergenceCountRound) ->\n    #state{status = Status,\n            instance = Instance,\n            load_data_list = LoadDataList,\n            ring_data = RingData,\n            leader = Leader,\n            range = Range,\n            request = Request,\n            requestor = comm:this(),\n            no_of_buckets = NoOfBuckets,\n            round = Round,\n            merged = Merged,\n            convergence_count = ConvergenceCountRound\n           }.\n\n%% @doc Creates round values with greatly reduces variance, so that more rounds\n%%      are valid rounds (i.e. rounds from messages and from the state match).\n-spec tester_create_round(Round::0..10) -> round().\ntester_create_round(0) -> 0;\ntester_create_round(_) -> 1.\n\n\n%% @doc Creates a histogram() within the specifications of this modules, i.e.\n%%      in particular that all histograms need to have the same keys\n%%      (keyrange/no_of_buckets).\n-spec tester_create_histogram(ListOfAvgs::[avg() | unknown]) -> histogram().\ntester_create_histogram([]) ->\n    tester_create_histogram([unknown]);\ntester_create_histogram(ListOfAvgs) ->\n    Buckets = intervals:split(intervals:all(), no_of_buckets()),\n    Histo1 = [ {BucketInterval, {}} || BucketInterval <- Buckets],\n    Fun = fun ({BucketInterval, {}}) -> {BucketInterval,\n                lists:nth(randoms:uniform(length(ListOfAvgs)), ListOfAvgs)} end,\n    lists:map(Fun, Histo1).\n\n-spec tester_create_histogram_size(1..50) -> histogram_size().\ntester_create_histogram_size(Size) -> Size.\n\n\n%% @doc Creates a fixed sized list of load_data every time. Size must not change once created.\n%% Tester fails with massive memory leak here when input type is load_data_list().\n-spec tester_create_load_data_list({load_data(), load_data(), load_data(), load_data()}) -> load_data_list().\ntester_create_load_data_list(LoadDataTuple) ->\n    % make first LoadData the default\n    case erlang:get(load_data_list) of\n        undefined ->\n            LoadDataList = erlang:tuple_to_list(LoadDataTuple),\n            LoadDataListNew =\n                case LoadDataList of\n                    [First | Rest] ->\n                        Default = First#load_data{name = ?DEFAULT_MODULE},\n                        [Default | Rest]\n                end,\n            erlang:put(load_data_list, LoadDataListNew),\n            LoadDataListNew;\n        Existing -> Existing\n    end.\n\n%% @doc Checks if a given list is a valid histogram.\n%%      Used as type_checker in tester.erl (property testing).\n-spec is_histogram([{intervals:interval(), avg()}]) -> boolean().\nis_histogram([]) ->\n    false;\n\nis_histogram(Histogram) ->\n    Buckets = intervals:split(intervals:all(), no_of_buckets()),\n    EmptyHisto = [ {BucketInterval, {}} || BucketInterval <- Buckets],\n    compare(Histogram, EmptyHisto).\n\n\n%% @doc Compares to histograms for identical keys, helper function for\n%%      is_histogram().\n-spec compare(histogram(), histogram()) -> boolean().\ncompare([], []) -> true;\ncompare([], _List) -> false;\ncompare(_List, []) -> false;\ncompare([{BucketInterval, _Avg1} | List1], [{BucketInterval, _Avg2} | List2]) ->\n    compare(List1, List2);\ncompare([_|_], [_|_]) ->\n    false.\n\n\n-spec init_feeder({NoOfBuckets::non_neg_integer(), Requestor::comm:mypid(),\n                   Random1::boolean(), Random2::boolean()}) -> {[proplists:property()]}.\ninit_feeder({NoOfBuckets, Requestor, Random1, Random2}) ->\n    Prop1 = if Random1 -> [{no_of_buckets, NoOfBuckets}];\n               not Random1 -> []\n            end,\n    Prop2 = if Random2 -> [{requestor, Requestor}];\n               not Random2 -> []\n            end,\n    {lists:append([Prop1, Prop2, [{instance, {gossip_load,default}}]])}.\n\n-spec get_values_best_feeder({Options::[proplists:property()], Secs::0..1, Millisecs::0..1000}) ->\n    {[proplists:property()]}.\nget_values_best_feeder({Options, Secs, Millisecs}) ->\n    Options1 = case proplists:get_value(send_after, Options) of\n        true ->\n            NewOptions1 = proplists:delete(send_after, Options),\n            [ {send_after, Millisecs} | NewOptions1 ];\n        _ ->\n            Options\n    end,\n    Options2 = case proplists:get_value(msg_delay, Options1) of\n        true ->\n            NewOptions2 = proplists:delete(msg_delay, Options),\n            [ {msg_delay, Secs} | NewOptions2 ];\n        _ ->\n            Options1\n    end,\n    {Options2}.\n\n\n%% @doc For testing: ensure, that only buckets with identical keys are feeded to\n%%      merge_bucket().\n-compile({nowarn_unused_function, {merge_bucket_feeder, 2}}).\n-spec merge_bucket_feeder(Bucket1::bucket(), Bucket2::bucket()) -> {bucket(), bucket()}.\nmerge_bucket_feeder({Key1, Val1}, {_Key2, Val2}) ->\n    {{Key1, Val1}, {Key1, Val2}}.\n\n% hack to be able to suppress warnings when testing\n-spec warn() -> log:log_level().\nwarn() ->\n    case config:read(gossip_log_level_warn) of\n        failed -> warn;\n        Level -> Level\n    end.\n\n\n"
  },
  {
    "path": "src/gossip_load_beh.erl",
    "content": "%  @copyright 2008-2011 Zuse Institute Berlin\n%\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n%\n%% @doc    Behavior for load modules in gossip_load\n%% @end\n%% @version $Id$\n-module(gossip_load_beh).\n-vsn('$Id$').\n\n% for behaviour\n-ifndef(have_callback_support).\n-export([behaviour_info/1]).\n-endif.\n\n-export_type([load/0]).\n\n-type load() :: number().\n\n% Erlang version >= R15B\n-ifdef(have_callback_support).\n\n-callback get_load(node_details:node_details()) -> load().\n\n-callback init_histo(node_details:node_details(), NumberOfBuckets::pos_integer())\n                    -> gossip_load:histogram().\n\n% Erlang version < R15B\n-else.\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n  [\n    {get_load, 1},\n    {init_histo, 2}\n  ];\n\nbehaviour_info(_Other) ->\n  undefined.\n\n-endif.\n"
  },
  {
    "path": "src/gossip_load_default.erl",
    "content": "%  @copyright 2010-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n%% @version $Id$\n-module(gossip_load_default).\n-vsn('$Id$').\n\n-behaviour(gossip_load_beh).\n\n-export([get_load/1, init_histo/2]).\n\n-spec get_load(node_details:node_details()) -> gossip_load_beh:load().\nget_load(NodeDetails) ->\n    node_details:get(NodeDetails, load).\n\n-spec init_histo(node_details:node_details(), NumberOfBuckets::pos_integer())\n                    -> gossip_load:histogram().\ninit_histo(NodeDetails, NumberOfBuckets) ->\n    DB = node_details:get(NodeDetails, db),\n    MyRange = node_details:get(NodeDetails, my_range),\n    Buckets = intervals:split(intervals:all(), NumberOfBuckets),\n    [ {BucketInterval, get_load_for_interval(BucketInterval, MyRange, DB)}\n        || BucketInterval <- Buckets ].\n\n-spec get_load_for_interval(BucketInterval::intervals:interval(),\n    MyRange::intervals:interval(), DB::db_dht:db()) -> gossip_load:avg() | unknown.\nget_load_for_interval(BucketInterval, MyRange, DB) ->\n    Intersection = intervals:intersection(BucketInterval, MyRange),\n    case intervals:is_empty(Intersection) of\n        true -> unknown;\n        false ->\n            try\n                Load = db_dht:get_load(DB, BucketInterval),\n                {float(Load), 1.0}\n            catch\n                Level:Reason ->\n                    log:pal(\"[ gossip ~p ] error while accessing DB: thrown ~s, reason:~.2p\",\n                            [self(), Level, Reason]),\n                    unknown\n            end\n    end.\n"
  },
  {
    "path": "src/gossip_vivaldi.erl",
    "content": "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%  @copyright 2008-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Jens V. Fischer <jensvfischer@gmail.com>\n%% @doc Gossip based calculation of Vivaldi coordinates.\n%% @end\n%%\n%% @version $Id$\n-module(gossip_vivaldi).\n-author('schuett@zib.de').\n-author('jensvfischer@gmail.com').\n-behaviour(gossip_beh).\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n%API\n-export([get_coordinate/0]).\n\n% gossip_beh\n-export([init/1, check_config/0, trigger_interval/0, fanout/0,\n        select_node/1, select_data/1, select_reply_data/4, integrate_data/3,\n        handle_msg/2, notify_change/3, min_cycles_per_round/0, max_cycles_per_round/0,\n        round_has_converged/1, web_debug_info/1, shutdown/1]).\n\n%% for testing\n-export([update_coordinate/5]).\n\n-export_type([est_error/0, latency/0, network_coordinate/0]).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Type Definitions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-type(network_coordinate() :: [float()]).\n-type(est_error() :: float()).\n-type(latency() :: number()).\n\n-type state() :: {network_coordinate(), est_error()}.\n-type data() :: any().\n-type round() :: non_neg_integer().\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Config Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%%------- External config function (called by gossip module) -------%%\n\n%% @doc The time interval in ms after which a new cycle is triggered by the gossip\n%%      module.\n-spec trigger_interval() -> pos_integer().\ntrigger_interval() -> % in ms\n    config:read(gossip_vivaldi_interval).\n\n\n%% @doc The fanout (number of peers contacted per cycle).\n-spec fanout() -> pos_integer().\nfanout() ->\n    1.\n\n\n%% @doc The minimum number of cycles per round.\n%%      Returns infinity, as rounds are not implemented by vivaldi.\n-spec min_cycles_per_round() -> infinity.\nmin_cycles_per_round() ->\n    infinity.\n\n\n%% @doc The maximum number of cycles per round.\n%%      Returns infinity, as rounds are not implemented by vivaldi.\n-spec max_cycles_per_round() -> infinity.\nmax_cycles_per_round() ->\n    infinity.\n\n\n%%------------------- Private config functions ---------------------%%\n\n%% @doc The dimensions for the coordinate system.\n-spec vivaldi_dimensions() -> pos_integer().\nvivaldi_dimensions() ->\n    config:read(gossip_vivaldi_dimensions).\n\n\n%% @doc Vivaldi doesn't need instantiabilty, so {gossip_vivaldi, default} is\n%%      always used.\n-spec instance() -> {gossip_vivaldi, default}.\n-compile({inline, [instance/0]}).\ninstance() ->\n    {gossip_vivaldi, default}.\n\n\n%% @doc Checks whether config parameters of the vivaldi process exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(gossip_vivaldi_interval) and\n    config:cfg_is_greater_than(gossip_vivaldi_interval, 0) and\n\n    config:cfg_is_integer(gossip_vivaldi_dimensions) and\n    config:cfg_is_greater_than_equal(gossip_vivaldi_dimensions, 2).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Sends a (local) message with a vivaldi get_coordinate request to the gossip\n%%      module of the requesting process' group asking for the current coordinate\n%%      and confidence.\n-spec get_coordinate() -> ok.\nget_coordinate() ->\n    Pid = pid_groups:get_my(gossip),\n    comm:send_local(Pid, {cb_msg, instance(), {get_coordinate, comm:this()}}).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Callback Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Initiate the gossip_vivaldi module. <br/>\n%%      Called by the gossip module upon startup. <br/>\n%%      The Instance information is ignored, {gossip_vivaldi, default} is always used.\n-spec init(Args::[any()]) -> {ok, state()}.\ninit(_Args) ->\n    log:log(info, \"[ Vivaldi ~.0p ] activating...~n\", [comm:this()]),\n    comm:send_local(pid_groups:get_my(gossip), {trigger_action, instance()}),\n    {ok, {random_coordinate(), 1.0}}.\n\n\n%% @doc Returns false, i.e. peer selection is done by the gossip module.\n-spec select_node(State::state()) -> {false, state()}.\nselect_node(State) ->\n    {false, State}.\n\n\n%% @doc Select and prepare the data to be sent to the peer. <br/>\n%%      The data consists of the coordinates and confidence of this node.\n%%      Called by the gossip module at the beginning of every cycle. <br/>\n%%      The selected exchange data is to be sent back to the gossip module as a\n%%      message of the form {selected_data, Instance, ExchangeData}.\n-spec select_data(State::state()) -> {ok, state()}.\nselect_data(State={Coordinate, Confidence}) ->\n    Data = {comm:this(), Coordinate, Confidence},\n    comm:send_local(pid_groups:get_my(gossip), {selected_data, instance(), Data}),\n    {ok, State}.\n\n\n%% @doc Process the data from the requestor. <br/>\n%%      No reply-data is selected and no select_reply_data_response is sent to\n%%      the gossip module, because vivaldi implements a push-only scheme. <br/>\n%%      Called by the behaviour module upon a p2p_exch message. <br/>\n%%      PData: exchange data from the p2p_exch request <br/>\n%%      Ref: used by the gossip module to identify the request <br/>\n%%      Round: ignored, as vivaldi does not implement round handling\n-spec select_reply_data(PData::data(), Ref::pos_integer(), Round::round(), State::state()) ->\n    {discard_msg | ok | retry | send_back, state()}.\nselect_reply_data(PData, _Ref, _Round, State) ->\n    {SourcePid, RemoteCoordinate, RemoteConfidence} = PData,\n    %io:format(\"{shuffle, ~p, ~p}~n\", [RemoteCoordinate, RemoteConfidence]),\n    _ = vivaldi_latency:measure_latency(SourcePid, RemoteCoordinate, RemoteConfidence),\n    {ok, State}.\n\n\n%% @doc Ignored, vivaldi is a push-only scheme.\n-spec integrate_data(QData::data(), Round::round(), State::state()) ->\n    {discard_msg | ok | retry | send_back, state()}.\nintegrate_data(_QData, _Round, State) ->\n    {ok, State}.\n\n\n%% @doc Handle messages\n%%      Response from vivaldi_latency after finishing measuring.\n-spec handle_msg(Msg::comm:message(), State::state()) -> {ok, state()}.\nhandle_msg({update_vivaldi_coordinate, Latency, {RemoteCoordinate, RemoteConfidence}},\n           {Coordinate, Confidence}) ->\n    %io:format(\"latency is ~pus~n\", [Latency]),\n    {NewCoordinate, NewConfidence} =\n        try\n            update_coordinate(RemoteCoordinate, RemoteConfidence,\n                              Latency, Coordinate, Confidence)\n        catch\n            % ignore any exceptions, e.g. badarith\n            error:_ -> {Coordinate, Confidence}\n        end,\n    %% signal gossip module, that the cycle is finished\n    comm:send_local(pid_groups:get_my(gossip), {integrated_data, instance(), cur_round}),\n    {ok, {NewCoordinate, NewConfidence}};\n\n\n%% Request for coordinates, usually from get_coordinate() api function\nhandle_msg({get_coordinate, SourcePid}, State={Coordinate, Confidence}) ->\n    comm:send(SourcePid, {vivaldi_get_coordinate_response, Coordinate, Confidence},\n              [{channel, prio}]),\n    {ok, State}.\n\n\n%% @doc Always returns false, vivaldi does not implement rounds.\n-spec round_has_converged(State::state()) -> {boolean(), state()}.\nround_has_converged(State) -> {false, State}.\n\n\n%% @doc Ignored, vivaldi doesn't use / implements these features.\n-spec notify_change(_, _, State::state()) -> {ok, state()}.\nnotify_change(_, _, State) -> {ok, State}.\n\n\n%% @doc Returns a key-value list of debug infos for the Web Interface. <br/>\n%%      Called by the gossip module upon {web_debug_info} messages.\n-spec web_debug_info(state()) ->\n    {KeyValueList::[{Key::string(), Value::any()},...], state()}.\nweb_debug_info(State={Coordinate, Confidence}) ->\n    KeyValueList =\n        [{\"gossip_vivaldi\", \"\"},\n         {\"coordinate\", webhelpers:safe_html_string(\"~p\", [Coordinate])},\n         {\"confidence\", Confidence}],\n    {KeyValueList, State}.\n\n\n%% @doc Shut down the gossip_vivaldi module. <br/>\n%%      Called by the gossip module upon stop_gossip_task(CBModule).\n-spec shutdown(State::state()) -> {ok, shutdown}.\nshutdown(_State) ->\n    % nothing to do\n    {ok, shutdown}.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Helpers\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec random_coordinate() -> network_coordinate().\nrandom_coordinate() ->\n    % note: network coordinates are float vectors!\n    [ float(randoms:rand_uniform(1, 10)) || _ <- lists:seq(1, vivaldi_dimensions()) ].\n\n\n-spec update_coordinate(network_coordinate(), est_error(), latency(),\n                         network_coordinate(), est_error()) ->\n                            {network_coordinate(), est_error()}.\nupdate_coordinate(Coordinate, _RemoteError, _Latency, Coordinate, Error) ->\n    % same coordinate\n    {Coordinate, Error};\nupdate_coordinate(RemoteCoordinate, RemoteError, Latency, Coordinate, Error) ->\n    Cc = 0.5, Ce = 0.5,\n    % sample weight balances local and remote error\n    W = if (Error + RemoteError) =:= 0.0 -> 0.0;\n           (Error + RemoteError) =/= 0.0 -> Error/(Error + RemoteError)\n        end,\n    % relative error of sample\n    Es = abs(mathlib:euclideanDistance(RemoteCoordinate, Coordinate) - Latency) / Latency,\n    % update weighted moving average of local error\n    Error1 = Es * Ce * W + Error * (1 - Ce * W),\n    % update local coordinates\n    Delta = Cc * W,\n    %io:format('expected latency: ~p~n', [mathlib:euclideanDist(Coordinate, _RemoteCoordinate)]),\n    C1 = mathlib:u(mathlib:vecSub(Coordinate, RemoteCoordinate)),\n    C2 = mathlib:euclideanDistance(Coordinate, RemoteCoordinate),\n    C3 = Latency - C2,\n    C4 = C3 * Delta,\n    Coordinate1 = mathlib:vecAdd(Coordinate, mathlib:vecMult(C1, C4)),\n    %io:format(\"new coordinate ~p and error ~p~n\", [Coordinate1, Error1]),\n    {Coordinate1, Error1}.\n\n\n"
  },
  {
    "path": "src/histogram.erl",
    "content": "% @copyright 2011-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Basic Histogram.\n%%         Yael Ben-Haim and Elad Tom-Tov, \"A streaming parallel\n%%         decision tree algorithm\", J. Machine Learning Research 11\n%%         (2010), pp. 849--872.\n%% @end\n%% @version $Id$\n-module(histogram).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n% external API\n-export([create/1, add/2, add/3, get_data/1, get_size/1,\n         get_num_elements/1, get_num_inserts/1, merge/2]).\n-export([merge_weighted/3, normalize_count/2]).\n\n% private API for unit tests:\n-export([find_smallest_interval/1, merge_interval/2,\n         tester_create_histogram/2, tester_is_valid_histogram/1]).\n\n-export([foldl_until/2, foldr_until/2]).\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-export_type([histogram/0, value/0]).\n\n-type value() :: number().\n-type data_item() :: {value(), pos_integer()}.\n-type data_list() :: list(data_item()).\n-record(histogram, {size = ?required(histogram, size):: non_neg_integer(),\n                    data = [] :: data_list(),\n                    data_size = 0 :: non_neg_integer(),\n                    inserts = 0 :: non_neg_integer()\n                   }).\n\n-opaque histogram() :: #histogram{}.\n\n%% @doc Creates an empty Size sized histogram\n-spec create(Size::non_neg_integer()) -> histogram().\ncreate(Size) ->\n    #histogram{size = Size}.\n\n-spec add(Value::value(), Histogram::histogram()) -> histogram().\nadd(Value, Histogram) ->\n    add(Value, 1, Histogram).\n\n-spec add(Value::value(), Count::pos_integer(), Histogram::histogram()) -> histogram().\nadd(_Value, _Count, Histogram = #histogram{size = 0}) ->\n    Histogram;\nadd(Value, Count, Histogram = #histogram{data = OldData, data_size = DataSize, inserts = Inserts}) ->\n    DataNew = insert({Value, Count}, OldData),\n    resize(Histogram#histogram{data = DataNew, data_size = DataSize + 1,\n                               inserts = Inserts + Count}).\n\n-spec get_data(Histogram::histogram()) -> data_list().\nget_data(Histogram) ->\n    Histogram#histogram.data.\n\n-spec get_size(Histogram::histogram()) -> non_neg_integer().\nget_size(Histogram) ->\n    Histogram#histogram.size.\n\n-spec get_num_elements(Histogram::histogram()) -> non_neg_integer().\nget_num_elements(Histogram) ->\n    Histogram#histogram.data_size.\n\n-spec get_num_inserts(Histogram::histogram()) -> non_neg_integer().\nget_num_inserts(Histogram) ->\n    Histogram#histogram.inserts.\n\n%% @doc Merges the given two histograms by adding every data point of Hist2\n%%      to Hist1.\n-spec merge(Hist1::histogram(), Hist2::histogram()) -> histogram().\nmerge(Hist1 = #histogram{size = 0}, _Hist2) -> Hist1;\nmerge(Hist1 = #histogram{data = Hist1Data}, #histogram{data = Hist2Data}) ->\n    NewData = lists:foldl(fun insert/2, Hist1Data, Hist2Data),\n    resize(Hist1#histogram{data = NewData, data_size = length(NewData)}).\n\n%% @doc Merges Hist2 into Hist1 and applies a weight to the Count of Hist2\n-spec merge_weighted(Hist1::histogram(), Hist2::histogram(), Weight::pos_integer()) -> histogram().\nmerge_weighted(Hist1, #histogram{data = Hist2Data} = Hist2, Weight) ->\n    WeightedData = lists:keymap(fun(Count) -> Count * Weight end, 2, Hist2Data),\n    WeightedHist2 = Hist2#histogram{data = WeightedData},\n    merge(Hist1, WeightedHist2).\n\n%% @doc Normalizes the Count by a normalization constant N\n-spec normalize_count(N::pos_integer(), Histogram::histogram()) -> histogram().\nnormalize_count(N, Histogram) ->\n    Data = histogram:get_data(Histogram),\n    DataNew = lists:keymap(fun(Count) -> Count div N end, 2, Data),\n    DataNew2 = lists:filter(fun({_Value, Count}) ->\n                                    Count > 0\n                            end, DataNew),\n    resize(Histogram#histogram{data = DataNew2, data_size = length(DataNew2)}).\n\n%% @doc Traverses the histogram until TargetCount entries have been found\n%%      and returns the value at this position.\n%% TODO change this to expect non empty histogram\n-spec foldl_until(TargetCount::non_neg_integer(), histogram())\n        -> {fail, Value::value() | nil, SumSoFar::non_neg_integer()} |\n           {ok, Value::value() | nil, Sum::non_neg_integer()}.\nfoldl_until(TargetCount, Histogram) ->\n    HistData = get_data(Histogram),\n    foldl_until_helper(TargetCount, HistData, _SumSoFar = 0, _BestValue = nil).\n\n%% @doc Like foldl_until but traverses the list from the right\n-spec foldr_until(TargetCount::non_neg_integer(), histogram())\n        -> {fail, Value::value() | nil, SumSoFar::non_neg_integer()} |\n           {ok, Value::value() | nil, Sum::non_neg_integer()}.\nfoldr_until(TargetCount, Histogram) ->\n    HistData = get_data(Histogram),\n    foldl_until_helper(TargetCount, lists:reverse(HistData), _SumSoFar = 0, _BestValue = nil).\n\n%% @doc Private method only exported for histogram_rt\n-spec foldl_until_helper(TargetCount::non_neg_integer(), DataList::data_list(),\n                         SumSoFar::non_neg_integer(), BestValue::nil | non_neg_integer())\n        -> {fail, Value::value() | nil, SumSoFar::non_neg_integer()} |\n           {ok, Value::value() | nil, Sum::non_neg_integer()}.\nfoldl_until_helper(TargetCount, _List, SumSoFar, BestValue)\n  when SumSoFar >= TargetCount ->\n    {ok, BestValue, SumSoFar};\nfoldl_until_helper(_TargetVal, [], SumSoFar, BestValue) ->\n    {fail, BestValue, SumSoFar};\nfoldl_until_helper(TargetCount, [{Val, Count} | Other], SumSoFar, _BestValue) ->\n    foldl_until_helper(TargetCount, Other, SumSoFar + Count, Val).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% private\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Resizes the given histogram to fit its maximum size (reduces the data).\n%%      PRE: histogram maximum size > 0 (from create/1)\n-spec resize(Histogram::histogram()) -> histogram().\nresize(Histogram = #histogram{data = Data, size = Size, data_size = DataSize})\n  when DataSize > Size andalso DataSize > 1 ->\n    ?DBG_ASSERT(Size > 0),\n    %% we need at least two items to do the following:\n    MinFirstValue = find_smallest_interval(Data),\n    NewHistogram = Histogram#histogram{data = merge_interval(MinFirstValue, Data),\n                                       data_size = DataSize - 1},\n    resize(NewHistogram);\nresize(#histogram{} = Histogram) ->\n    Histogram.\n\n-spec insert(Value::data_item(), Data::data_list()) -> data_list().\ninsert({Value, _} = DataItem, [{Value2, _} | _] = Data) when Value =< Value2 ->\n    [DataItem | Data];\ninsert(DataItem, [DataItem2 | Rest]) ->\n    [DataItem2 | insert(DataItem, Rest)];\ninsert(DataItem, []) ->\n    [DataItem].\n\n%% @doc Finds the smallest interval between two consecutive values and returns\n%%      the position of the first value (in the list's order).\n%%      Returning the position instead of the value ensures that the correct\n%%      items are merged when duplicate entries are in the histogram.\n%%      PRE: length(Data) >= 2\n-spec find_smallest_interval(Data::data_list()) -> MinFirstValue::value().\nfind_smallest_interval([{Value, _}, {Value2, _} | Rest]) ->\n    find_smallest_interval_loop(Value2 - Value, Value, Value2, Rest).\n\n-spec find_smallest_interval_loop(MinInterval::value(), MinFirstValue::value(), LastValue::number(), Data::data_list()) -> MinValue::value().\nfind_smallest_interval_loop(MinInterval, MinFirstValue, LastValue, [{Value, _} | Rest]) ->\n    Diff = Value - LastValue,\n    case MinInterval =< Diff of\n        true -> NewMinInterval = MinInterval,\n                NewMinFirstValue = MinFirstValue;\n        _    -> NewMinInterval = Diff,\n                NewMinFirstValue = LastValue\n    end,\n    find_smallest_interval_loop(NewMinInterval, NewMinFirstValue, Value, Rest);\nfind_smallest_interval_loop(_MinInterval, MinFirstValue, _LastValue, []) ->\n    MinFirstValue.\n\n%% @doc Merges two consecutive values if the first one of them is at PosMinValue.\n%%      Stops after the first match.\n%%      PRE: length(Data) >= 2, two consecutive values with the given difference\n-spec merge_interval(MinFirstValue::value(), Data::data_list()) -> data_list().\nmerge_interval(Value, [{Value, Count}, {Value2, Count2} | Rest]) when is_float(Value) orelse is_float(Value2) ->\n    [{(Value * Count + Value2 * Count2) / (Count + Count2), Count + Count2} | Rest];\nmerge_interval(Value, [{Value, Count}, {Value2, Count2} | Rest]) ->\n    [{(Value * Count + Value2 * Count2) div (Count + Count2), Count + Count2} | Rest];\nmerge_interval(MinFirstValue, [DataItem | Rest]) ->\n    [DataItem | merge_interval(MinFirstValue, Rest)].\n\n-spec tester_create_histogram(Size::non_neg_integer(), Data::data_list()) -> histogram().\ntester_create_histogram(Size = 0, _Data) ->\n    #histogram{size = Size, data = [], data_size = 0};\ntester_create_histogram(Size, Data) ->\n    DataSize = length(Data),\n    if DataSize > Size ->\n           #histogram{size = DataSize, data = Data, data_size = DataSize};\n       true ->\n           #histogram{size = Size, data = Data, data_size = DataSize}\n    end.\n\n-spec tester_is_valid_histogram(X::term()) -> boolean().\ntester_is_valid_histogram(#histogram{size = 0, data = [], data_size = 0}) ->\n    true;\ntester_is_valid_histogram(#histogram{size = Size, data = Data, data_size = DataSize})\n  when Size > 0->\n    Size >= DataSize andalso is_list(Data) andalso length(Data) =:= DataSize;\ntester_is_valid_histogram(_) ->\n    false.\n"
  },
  {
    "path": "src/histogram_rt.erl",
    "content": "%  @copyright 2013-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maximilian Michels <michels@zib.de>\n%% @doc Like histogram.erl but takes ?RT:key() as value and operates\n%%      in the key space according to the used routing table.\n%%      The histogram's interval span is ['(', BaseKey, BaseKey, ']'].\n%% @version $Id$\n-module(histogram_rt).\n-author('michels@zib.de').\n-vsn('$Id$').\n\n-export([create/2, add/2, add/3, get_data/1, get_size/1, merge/2,\n         get_num_elements/1, get_num_inserts/1]).\n-export([merge_weighted/3, normalize_count/2]).\n-export([foldl_until/2, foldr_until/2]).\n-export([is_histogram/1]).\n\n%% for testing\n-export([tester_create_histogram/2]).\n\n-include(\"scalaris.hrl\").\n\n-export_type([histogram/0, base_key/0]).\n\n-type key() :: ?RT:key().\n-type internal_value() :: histogram:value().\n\n-type external_data_item() :: {key(), pos_integer()}.\n-type external_data_list() :: [external_data_item()].\n\n-type base_key() :: key().\n\n-opaque(histogram() :: {Histogram::histogram:histogram(),\n                        BaseKey::base_key()\n                       }).\n\n-spec create(Size::non_neg_integer(), BaseKey::base_key()) -> histogram().\ncreate(Size, BaseKey) ->\n    Histogram = histogram:create(Size),\n    {Histogram, BaseKey}.\n\n-spec add(Value::key(), Histogram::histogram()) -> histogram().\nadd(Value, Histogram) ->\n    add(Value, 1, Histogram).\n\n-spec add(Value::key(), Count::pos_integer(), Histogram::histogram()) -> histogram().\nadd(Value, Count, {Histogram, BaseKey}) ->\n    NewHistogram = histogram:add(normalize(Value, BaseKey), Count, Histogram),\n    {NewHistogram, BaseKey}.\n\n-spec get_data(Histogram::histogram()) -> external_data_list().\nget_data({Histogram, BaseKey}) ->\n    Data = histogram:get_data(Histogram),\n    lists:map(fun({Value, Count}) ->\n                      {denormalize(Value, BaseKey), Count}\n              end, Data).\n\n-spec get_size(Histogram::histogram()) -> non_neg_integer().\nget_size({Histogram, _BaseKey}) ->\n    histogram:get_size(Histogram).\n\n-spec get_num_elements(Histogram::histogram()) -> non_neg_integer().\nget_num_elements({Histogram, _BaseKey}) ->\n    histogram:get_num_elements(Histogram).\n\n-spec get_num_inserts(Histogram::histogram()) -> non_neg_integer().\nget_num_inserts({Histogram, _BaseKey}) ->\n    histogram:get_num_inserts(Histogram).\n\n%% @doc Merges the given two histograms by adding every data point of\n%%      Hist2 to Hist1\n-spec merge(Hist1::histogram(), Hist2::histogram()) -> histogram().\nmerge(Hist1, Hist2) ->\n    merge_weighted(Hist1, Hist2, 1).\n\n%% @doc Merges Hist2 into Hist1 and applies a weight to the Count of Hist2\n-spec merge_weighted(Hist1::histogram(), Hist2::histogram(), Weight::pos_integer()) -> histogram().\nmerge_weighted({Histogram1, BaseKey}, {Histogram2, BaseKey}, Weight) ->\n    {histogram:merge_weighted(Histogram1, Histogram2, Weight), BaseKey};\nmerge_weighted(Hist1, Hist2, Weight) ->\n    case get_size(Hist1) of\n        0 -> Hist1;\n        _ ->\n            DataHist2 = get_data(Hist2),\n            lists:foldl(fun({Value, Count}, Hist) ->\n                                add(Value, Count*Weight, Hist)\n                        end,\n                        Hist1, DataHist2)\n    end.\n\n%% @doc Normalizes the Count by a normalization constant N\n-spec normalize_count(N::pos_integer(), Histogram::histogram()) -> histogram().\nnormalize_count(N, {Histogram, BaseKey}) ->\n    {histogram:normalize_count(N, Histogram), BaseKey}.\n\n%% @doc Traverses the histogram until TargetCount entries have been found\n%%      and returns the value at this position.\n-spec foldl_until(TargetCount::non_neg_integer(), histogram())\n        -> {fail, Value::key() | nil, SumSoFar::non_neg_integer()} |\n           {ok, Value::key() | nil, Sum::non_neg_integer()}.\nfoldl_until(TargetCount, {Histogram, BaseKey}) ->\n    Result = histogram:foldl_until(TargetCount, Histogram),\n    case Result of\n        {_Status, nil, _Sum} -> Result;\n        {Status, Range, Sum} -> {Status, denormalize(Range, BaseKey), Sum}\n    end.\n\n%% @doc Like foldl_until but traverses the list from the right\n-spec foldr_until(TargetCount::non_neg_integer(), histogram())\n        -> {fail, Value::key() | nil, SumSoFar::non_neg_integer()} |\n           {ok, Value::key() | nil, Sum::non_neg_integer()}.\nfoldr_until(TargetCount, {Histogram, BaseKey}) ->\n    Result = histogram:foldr_until(TargetCount, Histogram),\n    case Result of\n        {_Status, nil, _Sum} -> Result;\n        {Status, Range, Sum} -> {Status, denormalize(Range, BaseKey), Sum}\n    end.\n\n-compile({inline, [normalize/2]}).\n-spec normalize(Value::key(), BaseKey::base_key()) -> internal_value().\nnormalize(Value, BaseKey) ->\n    ?RT:get_range(BaseKey, Value).\n\n-compile({inline, [denormalize/2]}).\n-spec denormalize(Value::internal_value(), BaseKey::base_key()) -> key().\ndenormalize(Value, BaseKey) ->\n    ?RT:get_split_key(BaseKey, BaseKey, {Value, trunc(?RT:n())}).\n\n-spec is_histogram(histogram() | any()) -> boolean().\nis_histogram({_Histogram, _BaseKey}) ->\n    true;\nis_histogram(_) ->\n    false.\n\n-spec tester_create_histogram(Entries::[{key(), pos_integer()}], BaseKey::base_key()) -> histogram().\ntester_create_histogram(Entries, BaseKey) ->\n    HistogramRT = create(length(Entries), BaseKey),\n    lists:foldl(fun({Value, Count}, Hist) -> add(Value, Count, Hist) end, HistogramRT, Entries).\n"
  },
  {
    "path": "src/intervals.erl",
    "content": "%  @copyright 2007-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc Interval data structure and handling functions.\n%%\n%% All intervals created by methods of this module are normalized, i.e. simple\n%% intervals having unambiguous representations and complex intervals being\n%% lists of simple intervals sorted by the order given by interval_sort/2.\n%% Such a list contains no adjacent intervals except for those wrapping around,\n%% i.e. the first and the last element of the list. This representation is\n%% thus unambiguous as well.\n%% @end\n%% @version $Id$\n-module(intervals).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-compile({inline, [in_simple/2, in/2, is_between/5, interval_sort/2, merge_adjacent/2]}).\n-compile({inline, [normalize_internal/1, normalize_simple/1, wraps_around/4]}).\n-compile({inline, [intersection_simple/2, intersection_simple_element/2]}).\n-compile({inline, [union_simple/2, union/2]}).\n-compile({inline, [minus_simple/2, minus_simple2/2]}).\n-compile({inline, [split2/7]}).\n\n-include(\"scalaris.hrl\").\n\n-export([empty/0, new/1, new/4, all/0, from_elements/1,\n         % testing / comparing intervals\n         is_empty/1, is_non_empty/1, is_all/1, is_subset/2, is_continuous/1,\n         is_adjacent/2, in/2,\n         is_left_of/2, is_right_of/2,\n         split/2,\n         % operations for intervals\n         intersection/2, union/1, union/2, minus/2,\n         % getters for certain intervals\n         get_bounds/1, get_elements/1, get_simple_intervals/1,\n         simple_interval_to_interval/1,\n         % various\n         wraps_around/4,\n         %\n         % for unit testing only\n         is_well_formed/1, tester_create_interval/1,\n         is_well_formed_simple/1,\n         tester_create_simple_interval/1, tester_create_continuous_interval/4,\n         tester_create_non_empty_interval/2,\n         split_feeder/2\n        ]).\n\n-export_type([interval/0, key/0, left_bracket/0, right_bracket/0,\n              continuous_interval/0, non_empty_interval/0]).\n% for tester:\n-export_type([invalid_interval/0, invalid_simple_interval/0, simple_interval/0,\n              simple_interval2/0]).\n\n-type left_bracket() :: '(' | '['.\n-type right_bracket() :: ')' | ']'.\n-type key() :: ?RT:key() | ?MINUS_INFINITY_TYPE. % ?MINUS_INFINITY_TYPE unnecessary (should be included in ?RT:key()) but needed for fewer dialyzer warnings\n-type simple_interval2() :: {left_bracket(), key(), key(), right_bracket()} |\n                            {left_bracket(), key(), ?PLUS_INFINITY_TYPE, ')'}.\n-type simple_interval() :: {key()} | all | simple_interval2().\n-type invalid_simple_interval() :: {key()} | all | simple_interval2().\n-opaque interval() :: [simple_interval()].\n-opaque invalid_interval() :: [simple_interval()].\n-type continuous_interval() :: interval().\n-type non_empty_interval() :: interval().\n\n% @type interval() = [simple_interval()].\n% [] -> empty interval\n% [simple_interval(),...] -> union of the simple intervals\n% @type simple_interval() = {key()} | {left_bracket(), key(), key(), right_bracket()} | {left_bracket(), key(), PLUS_INFINITY, ')'} | all.\n% {term()} -> one element interval\n% {'[', A::term(), B::term(), ']'} -> closed interval [A, B]\n% {'(', A::term(), B::term(), ']'} -> half-open interval (A, B], aka ]A, B]\n% {'[', A::term(), B::term(), ')'} -> half-open interval [A, B), aka [A, B[\n% {'(', A::term(), B::term(), ')'} -> open interval (A, B), aka ]A, B[\n% all -> half open interval [?MINUS_INFINITY, ?PLUS_INFINITY)\n\n% Note: the intervals module uses two special symbols (?MINUS_INFINITY\n% and ?PLUS_INFINITY) to define the first, i.e. smallest, valid key and the\n% first key that is not valid anymore.\n% In Scalaris these values are dependent on the routing table\n% implementation. Therefore it is not possible e.g. to use intervals\n% over integer ranges and strings at the same time!\n\n-dialyzer({no_contracts, split2_feeder/7}).\n\n%% @doc Creates an empty interval.\n-spec empty() -> interval().\nempty() -> [].\n\n%% @doc Creates an interval covering the whole key space.\n-spec all() -> interval().\nall() -> normalize_simple(all).\n\n%% @doc Creates an interval covering a single element.\n-spec new(key()) -> interval().\nnew(X) -> normalize_simple({X}).\n\n%% @doc Creates a new interval depending on the given brackets, i.e.:\n%%      - closed interval [A, B],\n%%      - half-open interval (A, B], aka ]A, B]\n%%      - half-open interval [A, B), aka [A, B[\n%%      - open interval (A, B), aka ]A, B[\n%%      The new interval may wrap around, e.g. if A > B.\n%%      If '[A,A]' is given, an interval with the element A is created.\n%%      The special cases '(A,A)', '[A,A)', '(A,A]' and\n%%      '(?PLUS_INFINITY,?MINUS_INFINITY,)' translate to an empty interval.\n%%      '[?MINUS_INFINITY,?PLUS_INFINITY)' translates to 'all'.\n-spec new(left_bracket(), key(),\n          key() | ?PLUS_INFINITY_TYPE, %% then right_bracket is ')'\n          right_bracket()) -> interval().\nnew(LeftBr, Begin, End, RightBr) when End =/= ?PLUS_INFINITY orelse RightBr =:= ')' ->\n    normalize_simple({LeftBr, Begin, End, RightBr}).\n\n%% @doc Creates an interval from a list of elements.\n-spec from_elements(Elements::[key()]) -> interval().\nfrom_elements(Elements) ->\n    normalize_internal([{E} || E <- Elements]).\n\n%% @doc Checks whether the given interval is empty.\n-spec is_empty(interval()) -> boolean().\nis_empty([]) -> true;\nis_empty(_) ->  false.\n\n%% @doc Checks whether the given interval is non-empty (mainly for tester type checker).\n-spec is_non_empty(interval()) -> boolean().\nis_non_empty(I) -> not is_empty(I).\n\n%% @doc Checks whether the given interval is covering everything.\n-spec is_all(interval()) -> boolean().\nis_all([all]) -> true;\nis_all(_) ->  false.\n\n%% @doc Creates the intersection of two intervals.\n%%      Precondition: is_well_formed(A) andalso is_well_formed(B).\n-spec intersection(A::interval(), B::interval()) -> interval().\nintersection([all], B)  -> B;\nintersection(A, [all])  -> A;\nintersection([] = A, _) -> A;\nintersection(_, [] = B) -> B;\nintersection([{_} = A], B) -> intersection_element(A, B);\nintersection(A, [{_} = B]) -> intersection_element(B, A);\nintersection(A, A) -> A;\nintersection(A, B) ->\n    normalize_internal([IS || IA <- A, IB <- B,\n                              [] =/= (IS = intersection_simple(IA, IB))]).\n\n%% @doc Intersection between an element and an interval.\n-spec intersection_element(A::{key()}, B::interval()) -> interval().\nintersection_element({X} = A, B) ->\n    case in(X, B) of\n        false -> empty();\n        true  -> [A]\n    end.\n\n%% @doc Creates the intersection of two simple intervals or empty lists.\n-spec intersection_simple(A::simple_interval(), B::simple_interval()) -> simple_interval() | [].\nintersection_simple(A, A) -> A;\nintersection_simple({A0Br, A0, A1, A1Br},\n                    {B0Br, B0, B1, B1Br}) ->\n    B0_in_A = is_between(A0Br, A0, B0, A1, A1Br),\n    B1_in_A = is_between(A0Br, A0, B1, A1, A1Br),\n    A0_in_B = is_between(B0Br, B0, A0, B1, B1Br),\n    A1_in_B = is_between(B0Br, B0, A1, B1, B1Br),\n    if % are the intervals overlapping?\n        B0_in_A orelse B1_in_A orelse A0_in_B orelse A1_in_B ->\n            {NewLeft, NewLeftBr} =\n                case A0 =:= B0 of\n                    true when A0Br =:= '(' -> {A0, '('};\n                    true -> {A0, B0Br};\n                    false ->\n                        case util:max(A0, B0) =:= A0 of\n                            true  -> {A0, A0Br};\n                            false -> {B0, B0Br}\n                        end\n                end,\n            {NewRight, NewRightBr} =\n                case A1 =:= B1 of\n                    true when A1Br =:= ')' -> {A1, ')'};\n                    true -> {A1, B1Br};\n                    false ->\n                        case util:min(A1, B1) =:= A1 of\n                            true  -> {A1, A1Br};\n                            false -> {B1, B1Br}\n                        end\n                end,\n            % note: if left and right are the same in a closed interval, this\n            % means 'single element' here, in (half) open intervals, the result\n            % is an empty interval\n            case NewLeft =:= NewRight of\n                true when (NewLeftBr =:= '[') andalso (NewRightBr =:= ']') ->\n                    {NewLeft};\n                true -> [];\n                false -> {NewLeftBr, NewLeft, NewRight, NewRightBr}\n            end;\n        true -> []\n    end;\nintersection_simple(all, B)    -> B;\nintersection_simple(A, all)    -> A;\nintersection_simple({_} = A, B) -> intersection_simple_element(A, B);\nintersection_simple(A, {_} = B) -> intersection_simple_element(B, A).\n\n%% @doc Intersection between an element and a simple interval.\n-spec intersection_simple_element(A::{key()}, B::simple_interval()) -> {key()} | [].\nintersection_simple_element({X} = A, B) ->\n    case in_simple(X, B) of\n        false -> [];\n        true  -> A\n    end.\n\n%% @doc Returns true if A is a subset of B, i.e. the intersection of both is A.\n%%      Precondition: is_well_formed(A) andalso is_well_formed(B).\n-spec(is_subset(A::interval(), B::interval()) -> boolean()).\nis_subset(A, B) -> A =:= intersection(A, B).\n\n%% @doc X \\in I. Precondition: is_well_formed_simple(I).\n-spec in_simple(X::key(), I::simple_interval()) -> boolean().\n% inline is_between/5 for performance (+15-20%):\nin_simple(X, {LBr, Begin, End, RBr}) ->\n    (X > Begin andalso End > X) orelse\n        (LBr =:= '[' andalso X =:= Begin) orelse\n        (RBr =:= ']' andalso X =:= End);\nin_simple(_, all) -> true;\nin_simple(X, {E}) -> X =:= E.\n\n%% @doc X \\in I. Precondition: is_well_formed(I).\n-spec in(X::key(), I::interval()) -> boolean().\nin(X, [I | Rest]) -> in_simple(X, I) orelse in(X, Rest);\nin(_, [])         -> false.\n\n%% @doc Brings a list of intervals into normal form, i.e. sort, eliminate empty\n%%      intervals from the list, convert intervals that wrap around into a set\n%%      of intervals not wrapping around, merge adjacent intervals.\n%%      Note: Outside this module, use only for testing - all intervals\n%%      generated by this module are normalized!\n%%      Note: This function also corrects values which are to small, i.e. less\n%%      than ?MINUS_INFINITY, or too large, i.e. greater than or equal to\n%%      ?PLUS_INFINITY which is only needed if types are created based on\n%%      (potentially) inprecise type defs, e.g. in the unit tests. For internal\n%%      use prefer normalize_internal/1 which does not fix values and is thus\n%%      faster.\n-spec tester_create_interval(invalid_interval() | interval()) -> interval().\ntester_create_interval(List) ->\n    List1 = [normalize_simple_bounds(I)\n            || I <- List,\n               % filter out {X >= ?PLUS_INFINITY} which is invalid:\n               not (is_tuple(I) andalso element(1, I) =:= element andalso element(2, I) >= ?PLUS_INFINITY)],\n    normalize_internal(List1).\n\n-spec tester_create_non_empty_interval([simple_interval(),...], FallbackElem::key())\n        -> non_empty_interval().\ntester_create_non_empty_interval(List, FallbackElem) ->\n    I = tester_create_interval(List),\n    case is_empty(I) of\n        true  -> new(FallbackElem);\n        false -> I\n    end.\n\n-spec tester_create_continuous_interval(left_bracket(), key(), key() | ?PLUS_INFINITY_TYPE, right_bracket()) -> continuous_interval().\ntester_create_continuous_interval(_LBr, Key, Key, _RBr) ->\n    new('[', Key, Key, ']');\ntester_create_continuous_interval(LBr, LKey, ?PLUS_INFINITY, _RBr) ->\n    new(LBr, LKey, ?PLUS_INFINITY, ')');\ntester_create_continuous_interval(LBr, LKey, RKey, RBr) ->\n    new(LBr, LKey, RKey, RBr).\n\n%% @doc Brings a simple interval into normal form, i.e. if it is a real\n%%      interval, its keys must be in order.\n%%      Note: Outside this module, use only for testing - all intervals\n%%      generated by this module are normalized!\n%%      Note: This function also corrects values which are to small, i.e. less\n%%      than ?MINUS_INFINITY, or too large, i.e. greater than or equal to\n%%      ?PLUS_INFINITY which is only needed if types are created based on\n%%      (potentially) inprecise type defs, e.g. in the unit tests.\n-spec tester_create_simple_interval(invalid_simple_interval()) -> simple_interval().\ntester_create_simple_interval(I0) ->\n    I1 = normalize_simple_bounds(I0),\n    case normalize_simple(I1) of\n        [] -> all;\n        [H|_] -> H\n    end.\n\n-spec normalize_internal([simple_interval()]) -> interval(). % for dialyzer\nnormalize_internal(List) ->\n    NormalizedList1 = lists:flatmap(fun normalize_simple/1, List),\n    merge_adjacent(lists:usort(fun interval_sort/2, NormalizedList1), []).\n\n%% @doc Normalizes simple intervals (see normalize_internal/1).\n-spec normalize_simple(invalid_simple_interval()) -> [simple_interval()].\nnormalize_simple({'(', X, X, _RightBr}) -> [];\nnormalize_simple({_LeftBr, X, X, ')'}) -> [];\nnormalize_simple({'[', X, X, ']'}) -> normalize_simple({X});\nnormalize_simple({'[', ?MINUS_INFINITY, ?PLUS_INFINITY, ')'}) ->\n    [all];\nnormalize_simple({LeftBr, X, ?MINUS_INFINITY, ')'}) ->\n    [{LeftBr, X, ?PLUS_INFINITY, ')'}];\nnormalize_simple({LeftBr, X, ?MINUS_INFINITY, ']'}) ->\n    [{?MINUS_INFINITY}, {LeftBr, X, ?PLUS_INFINITY, ')'}];\nnormalize_simple({LeftBr, Begin, End, RightBr} = I) ->\n    case wraps_around(LeftBr, Begin, End, RightBr) of\n        true ->  [{'[', ?MINUS_INFINITY, End, RightBr},\n                  {LeftBr, Begin, ?PLUS_INFINITY, ')'}];\n        false -> [I]\n    end;\nnormalize_simple({_A} = I) -> [I];\nnormalize_simple(all) -> [all].\n\n-spec normalize_simple_bounds({left_bracket(), key(), key() | ?PLUS_INFINITY_TYPE, right_bracket()} | {key()} | all)\n        -> invalid_simple_interval().\nnormalize_simple_bounds({_LeftBr, Begin, End, RightBr})\n  when Begin < ?MINUS_INFINITY orelse Begin >= ?PLUS_INFINITY ->\n    normalize_simple_bounds({'[', ?MINUS_INFINITY, End, RightBr});\nnormalize_simple_bounds({_LeftBr, _Begin, ?PLUS_INFINITY, ')'} = I) ->\n    I;\nnormalize_simple_bounds({LeftBr, Begin, End, _RightBr})\n  when End < ?MINUS_INFINITY orelse End >= ?PLUS_INFINITY ->\n    {LeftBr, Begin, ?PLUS_INFINITY, ')'};\nnormalize_simple_bounds({X}) when X < ?MINUS_INFINITY ->\n    {?MINUS_INFINITY};\nnormalize_simple_bounds(I) ->\n    I.\n  \n%% @doc Checks whether the given interval is normalized, i.e. not wrapping\n%%      around and no 'interval' with equal borders (see normalize_internal/1).\n%%      Use only for testing - all intervals generated by this module are\n%%      well-formed, i.e. normalized!\n-spec is_well_formed(invalid_interval() | interval()) ->  boolean().\nis_well_formed([]) -> true;\nis_well_formed([_|_] = List) ->\n    lists:all(fun is_well_formed_simple/1, List) andalso\n        % sorted and unique:\n        List =:= lists:usort(fun interval_sort/2, List) andalso\n        % pairwise non-overlapping:\n        (lists:flatten([intersection_simple(A, B) || A <- List, B <- List,\n                                                     A =/= B]) =:= []).\n\n%% @doc Checks whether a given simple interval is normalized. Complex intervals\n%%      or any other value are considered 'not normalized'.\n-spec is_well_formed_simple(simple_interval()) ->  boolean().\nis_well_formed_simple({_LeftBr, _X, ?MINUS_INFINITY, ')'}) ->\n    false;\nis_well_formed_simple({_LeftBr, _X, ?PLUS_INFINITY, ')'}) ->\n    true;\nis_well_formed_simple({_LeftBr, X, Y, _RightBr}) ->\n    X < Y;\nis_well_formed_simple({_X}) -> true;\nis_well_formed_simple(all) -> true;\nis_well_formed_simple(_) -> false.\n\n%% @doc Specifies an order over simple intervals (returns true if I1 &lt;= I2).\n%%      The order is based on the intervals' first components\n%%      and in case of elements based on their value. 'all' is the first and\n%%      elements are sorted before intervals with the same values, if two\n%%      intervals' first components compare equal the one with '[' is smaller\n%%      (to ease merge_adjacent/2), otherwise normal &lt;= from erlang is used.\n-spec interval_sort(I1::simple_interval(), I2::simple_interval()) -> boolean().\ninterval_sort({A0Br, A0, _A1, _A1Br} = A, {B0Br, B0, _B1, _B1Br} = B) ->\n    % beware of not accidentally making two intervals equal, which is defined\n    % as A==B <=> interval_sort(A, B) andalso interval_sort(B, A)\n    B0 > A0 orelse\n        (A0 =:= B0 andalso A0Br =:= '[' andalso B0Br =:= '(') orelse\n        (A0 =:= B0 andalso (not (A0Br =:= '(' andalso B0Br =:= '[')) andalso A =< B);\ninterval_sort({A}, {_B0Br, B0, _B1, _B1Br}) ->\n    B0 >= A;\ninterval_sort({_A0Br, A0, _A1, _A1Br}, {B}) ->\n    B > A0;\ninterval_sort({A}, {B}) ->\n    B >= A;\ninterval_sort(all, _Interval2) ->\n    true;\ninterval_sort(_Interval1, _Interval2) ->\n    false.\n\n%% @doc Merges adjacent intervals in a sorted list of simple intervals using\n%%      union_simple/2.\n-spec merge_adjacent([simple_interval()], [simple_interval()]) -> [simple_interval()].\nmerge_adjacent([all | _T], _Results) ->\n    [all];\nmerge_adjacent([H | T], []) ->\n    merge_adjacent(T, [H]);\nmerge_adjacent([HI | TI], [HR | TR]) ->\n    merge_adjacent(TI, union_simple(HI, HR) ++ TR);\nmerge_adjacent([], Results) ->\n    lists:reverse(Results).\n\n%% @doc Creates the union of two intervals.\n-spec union(A::interval(), B::interval()) -> interval().\nunion([all] = A, _B) -> A;\nunion(_A, [all] = B) -> B;\nunion([], B)         -> B;\nunion(A, [])         -> A;\nunion(A, A)          -> A;\nunion(A, B)          -> normalize_internal(lists:append(A, B)).\n\n%% @doc Creates the union of a list of intervals.\n-spec union([interval()]) -> interval().\nunion(List) -> normalize_internal(lists:append(List)).\n\n%% @doc Creates the union of two simple intervals or empty lists.\n-spec union_simple(A::simple_interval(), B::simple_interval()) -> [simple_interval()].\nunion_simple(all, _B) -> [all];\nunion_simple(_A, all) -> [all];\nunion_simple({A0Br, A0, A1, ']'}, {_B0Br, A1, B1, B1Br}) ->\n    new(A0Br, A0, B1, B1Br);\nunion_simple({A0Br, A0, A1, _A1Br}, {'[', A1, B1, B1Br}) ->\n    new(A0Br, A0, B1, B1Br);\nunion_simple({'[', A0, A1, A1Br}, {B0Br, B0, A0, _B1Br}) ->\n    new(B0Br, B0, A1, A1Br);\nunion_simple({_A0Br, A0, A1, A1Br}, {B0Br, B0, A0, ']'}) ->\n    new(B0Br, B0, A1, A1Br);\nunion_simple({A0Br, A0, A1, A1Br} = A, {B0Br, B0, B1, B1Br} = B) ->\n    B0_in_A = is_between(A0Br, A0, B0, A1, A1Br),\n    B1_in_A = is_between(A0Br, A0, B1, A1, A1Br),\n    A0_in_B = is_between(B0Br, B0, A0, B1, B1Br),\n    A1_in_B = is_between(B0Br, B0, A1, B1, B1Br),\n    if\n        B0_in_A orelse B1_in_A orelse A0_in_B orelse A1_in_B ->\n            {NewLeft, NewLeftBr} =\n                case A0 =:= B0 of\n                    true when A0Br =:= '[' -> {A0, '['};\n                    true -> {A0, B0Br};\n                    false ->\n                        case util:min(A0, B0) =:= A0 of\n                            true  -> {A0, A0Br};\n                            false -> {B0, B0Br}\n                        end\n                end,\n            {NewRight, NewRightBr} =\n                case A1 =:= B1 of\n                    true when A1Br =:= ']' -> {A1, ']'};\n                    true -> {A1, B1Br};\n                    false ->\n                        case util:max(A1, B1) =:= A1 of\n                            true  -> {A1, A1Br};\n                            false -> {B1, B1Br}\n                        end\n                end,\n            % note: if left and right are the same in a closed interval, this\n            % means 'single element' here, in (half) open intervals, the result\n            % is en empty interval\n            % (using new/4 would create 'all' instead)\n            % however, the union of two 'real' intervals should never be an element!\n            % @todo: add exception here?\n            case NewLeft =:= NewRight of\n                true when (NewLeftBr =:= '[') andalso (NewRightBr =:= ']') ->\n                    new(NewLeft);\n                true -> empty();\n                false -> new(NewLeftBr, NewLeft, NewRight, NewRightBr)\n            end;\n        true -> [A, B]\n    end;\nunion_simple({B0}, {_B0Br, B0, B1, B1Br}) ->\n    new('[', B0, B1, B1Br);\nunion_simple({B1}, {B0Br, B0, B1, _B1Br}) ->\n    new(B0Br, B0, B1, ']');\nunion_simple({A0Br, A0, A1, _A1Br}, {A1}) ->\n    new(A0Br, A0, A1, ']');\nunion_simple({_A0Br, A0, A1, A1Br}, {A0}) ->\n    new('[', A0, A1, A1Br);\nunion_simple({A_Value} = A, B) ->\n    % note: always place the first element (A) before the second\n    % element (B) in a union for merge_adjacent/2\n    case in_simple(A_Value, B) of\n        true  -> [B];\n        false -> [A, B]\n    end;\nunion_simple(A, {B_Value} = B) ->\n    % note: always place the first element (A) before the second\n    % element (B) in a union for merge_adjacent/2\n    case in_simple(B_Value, A) of\n        true  -> [A];\n        false -> [A, B]\n    end.\n\n%% @doc Checks whether the given interval is a continuous interval, i.e. simple\n%%      intervals are always continuous, complex intervals are continuous if\n%%      they contain 2 simple intervals which are adjacent and wrap around.\n%%      Note: empty intervals are not continuous!\n-spec is_continuous(interval()) -> boolean().\nis_continuous([{_LBr, _L, _R, _RBr}]) -> true;\n% complex intervals have adjacent intervals merged except for those wrapping around\n% -> if it contains only two simple intervals which are adjacent, it is continuous!\nis_continuous([{'[', ?MINUS_INFINITY, _B1, _B1Br},\n               {_A0Br, _A0, ?PLUS_INFINITY, ')'}]) -> true;\nis_continuous([{?MINUS_INFINITY},\n               {_A0Br, _A0, ?PLUS_INFINITY, ')'}]) -> true;\nis_continuous([all]) -> true;\nis_continuous([{_Key}]) -> true;\nis_continuous(_) -> false.\n\n%% @doc Gets the outer bounds of a given non-empty interval including\n%%      brackets. Note that here\n%%      'all' transfers to {'[', ?MINUS_INFINITY, ?PLUS_INFINITY, ')'},\n%%      {Key} to {'[', Key, Key, ']'} and\n%%      [{'[',?MINUS_INFINITY,Key,')'},{'(',Key,?PLUS_INFINITY,')'}] to {'(', Key, Key, ')'}.\n%%      Other continuous normalized intervals that wrap around (as well as the\n%%      first two) are returned the same way they can be constructed with new/4.\n%%      Note: the bounds of non-continuous intervals are not optimal!\n%%      Note: this method will only work on non-empty intervals\n%%      and will throw an exception otherwise!\n-spec get_bounds(interval()) -> simple_interval2().\nget_bounds([{LBr, L, R, RBr}]) -> {LBr, L, R, RBr};\nget_bounds([{'[', ?MINUS_INFINITY, B1, B1Br},\n            {A0Br, A0, ?PLUS_INFINITY, ')'}]) -> {A0Br, A0, B1, B1Br};\nget_bounds([{?MINUS_INFINITY},\n            {A0Br, A0, ?PLUS_INFINITY, ')'}]) -> {A0Br, A0, ?MINUS_INFINITY, ']'};\nget_bounds([all]) -> {'[', ?MINUS_INFINITY, ?PLUS_INFINITY, ')'};\nget_bounds([{Key}]) -> {'[', Key, Key, ']'};\nget_bounds([]) -> erlang:throw('no bounds in empty interval');\n% fast creation of bounds by using the first and last value of the interval:\nget_bounds([H | T]) ->\n    case H of\n        {X1} -> LBr = '[', L = X1, ok;\n        {LBr, L, _, _} -> ok\n    end,\n    case lists:last(T) of\n        {X2} -> RBr = ']', R = X2, ok;\n        {_, _, R, RBr} -> ok\n    end,\n    {LBr, L, R, RBr}.\n\n%% @doc Gets all elements inside the interval and returnes a \"rest\"-interval,\n%%      i.e. the interval without the elements.\n-spec get_elements(interval()) -> {Elements::[key()], RestInt::interval()}.\nget_elements(I) -> get_elements_list(I, [], []).\n\n%% @doc Helper for get_elements/1.\n-spec get_elements_list(Interval::interval(), Elements::[key()], RestInt::[simple_interval()]) -> {Elements::[key()], RestInt::[simple_interval()]}.\nget_elements_list([{Key} | T], Elements, RestInt) ->\n    get_elements_list(T, [Key | Elements], RestInt);\nget_elements_list([I | T], Elements, RestInt) ->\n    get_elements_list(T, Elements, [I | RestInt]);\nget_elements_list([], Elements, RestInt) ->\n    {lists:reverse(Elements), lists:reverse(RestInt)}.\n\n%% @doc Checks whether two intervals are adjacent, i.e. the intervals are both\n%%      continuous, their union is continuous and their intersection is empty,\n%%      e.g. ('(A,B]', '(B,C)') with A=/=B and B=/=C.\n%%      Note: intervals like (A,B), (B,C) are not considered adjacent because\n%%      the element b would be between these two.\n-spec is_adjacent(interval(), interval()) -> boolean().\nis_adjacent(A, B) ->\n    is_continuous(A) andalso is_continuous(B) andalso\n        is_empty(intersection(A, B)) andalso is_continuous(union(A, B)).\n\n%% @doc Subtracts the second from the first simple interval.\n-spec minus_simple(simple_interval(), simple_interval()) -> interval().\nminus_simple(A, A)   -> empty();\nminus_simple(A = {_A0Br, _A0, _A1, _A1Br},\n             B = {_B0Br, _B0, _B1, _B1Br}) ->\n    B_ = intersection_simple(A, B),\n    case B_ of\n        A                      -> empty();\n        {_, _, _, _} -> minus_simple2(A, B_);\n        []                     -> [A];\n        _                      -> minus_simple(A, B_)\n    end;\nminus_simple(A = {A0}, B = {_B0Br, _B0, _B1, _B1Br}) ->\n    case in_simple(A0, B) of\n        true -> empty();\n        _    -> [A]\n    end;\nminus_simple({'[', A0, A1, A1Br}, {A0}) ->\n    new('(', A0, A1, A1Br);\nminus_simple({A0Br, A0, A1, ']'}, {A1}) ->\n    new(A0Br, A0, A1, ')');\nminus_simple(A = {A0Br, A0, A1, A1Br}, {B0}) ->\n    case in_simple(B0, A) of\n        false -> [A];\n        true  -> union(new(A0Br, A0, B0, ')'), new('(', B0, A1, A1Br))\n    end;\nminus_simple(A = {_}, {_}) -> [A];\nminus_simple(_, all) -> empty();\nminus_simple(all, B = {_B0Br, _B0, _B1, _B1Br}) ->\n    % hack: use [?MINUS_INFINITY, ?PLUS_INFINITY) as 'all' and [B0, B0] as element - minus_simple2 can handle this though\n    minus_simple2({'[', ?MINUS_INFINITY, ?PLUS_INFINITY, ')'}, B);\nminus_simple(all, {B0}) ->\n    % hack: use [?MINUS_INFINITY, ?PLUS_INFINITY) as 'all' and [B0, B0] as element - minus_simple2 can handle this though\n    minus_simple2({'[', ?MINUS_INFINITY, ?PLUS_INFINITY, ')'},\n                  {'[', B0, B0, ']'}).\n\n%% @doc Subtracts the second from the first simple interval (no elements, no\n%%      'all', no empty interval). The second interval must be a subset of the\n%%      first interval!\n-spec minus_simple2(simple_interval2(), simple_interval2()) -> interval().\nminus_simple2({A0Br, A0, A1, A1Br}, {B0Br, B0, B1, B1Br}) ->\n    First = case B0Br of\n                '(' when B0 =:= A0 andalso A0Br =:= '[' ->\n                    new(A0);\n                '(' when B0 =:= A0 -> empty();\n                '('                -> new(A0Br, A0, B0, ']');\n                '[' when B0 =:= A0 -> empty();\n                '['                -> new(A0Br, A0, B0, ')')\n               end,\n    Second = case B1Br of\n                 ']' when B1 =:= A1 -> empty();\n                 ']'                -> new('(', B1, A1, A1Br);\n                 ')' when B1 =:= A1 andalso A1Br =:= ']' ->\n                     new(A1);\n                 ')' when B1 =:= A1 -> empty();\n                 ')'                -> new('[', B1, A1, A1Br)\n               end,\n    union(First, Second).\n\n%% @doc Subtracts the second from the first interval.\n-spec minus(interval(), interval()) -> interval().\nminus(_A, [all]) -> empty();\nminus(A, [])     -> A;\nminus(A, A)      -> empty();\nminus(A, [HB | TB]) ->\n    % from every simple interval in A, remove all simple intervals in B\n    % note: we cannot use minus_simple in foldl since the result may be a list again\n    normalize_internal(\n      lists:flatmap(fun(IA) -> minus(minus_simple(IA, HB), TB) end, A)).\n\n%% @doc Determines whether an interval with the given borders wraps around,\n%%      i.e. the interval would cover the (non-existing) gap between\n%%      ?PLUS_INFINITY and ?MINUS_INFINITY.\n-spec wraps_around(left_bracket(), key(), key() | ?PLUS_INFINITY_TYPE, right_bracket()) -> boolean().\nwraps_around(_LeftBr, X, X, _RightBr) ->\n    false;\nwraps_around(_LeftBr, ?MINUS_INFINITY, _, _RightBr) ->\n    false;\nwraps_around(_LeftBr, _, ?PLUS_INFINITY, _RightBr) ->\n    false;\nwraps_around(_LeftBr, _, ?MINUS_INFINITY, ')') ->\n    % same as [A, ?PLUS_INFINITY) or (A, ?PLUS_INFINITY)\n    false;\nwraps_around(_LeftBr, _, ?MINUS_INFINITY, _RightBr) ->\n    true;\nwraps_around('(', ?PLUS_INFINITY, _, _RightBr) ->\n    % same as [?MINUS_INFINITY, A] or [?MINUS_INFINITY, A)\n    false;\nwraps_around(_LeftBr, ?PLUS_INFINITY, _, _RightBr) ->\n    true;\nwraps_around(_LeftBr, First, Last, _RightBr) when First > Last ->\n    true;\nwraps_around(_LeftBr, _First, _Last, _RightBr) ->\n    false.\n\n% @doc Begin &lt;= X &lt;= End\n% precondition Begin &lt;= End\n-spec is_between(BeginBr::left_bracket(), Begin::key(), X::key(), End::key(), EndBr::right_bracket()) -> boolean().\nis_between(LBr, Begin, X, End, RBr) ->\n    (X > Begin andalso End > X) orelse\n        (LBr =:= '[' andalso X =:= Begin) orelse\n        (RBr =:= ']' andalso X =:= End).\n\n%% @doc X and Y are adjacent and Y follows X\n-spec is_left_of(interval(), interval()) -> boolean().\nis_left_of(X, Y) ->\n    case is_adjacent(X, Y) of\n        true ->\n            {_, _A,  B, _} = get_bounds(X),\n            {_,  C, _D, _} = get_bounds(Y),\n            % in(B, X) =/= in(B, Y) implied by is_adjacent\n            (B =:= C andalso (in(B, X) orelse in(B, Y)))\n                orelse\n            (B =:= ?PLUS_INFINITY andalso C =:= ?MINUS_INFINITY andalso\n             in(?MINUS_INFINITY, Y));\n        false ->\n            false\n    end.\n\n%% @doc X and Y are adjacent and X follows Y\n-spec is_right_of(interval(), interval()) -> boolean().\nis_right_of(X, Y) ->\n    is_left_of(Y, X).\n\n-spec split_feeder(continuous_interval(), 1..255)\n        -> {continuous_interval(), pos_integer()}.\nsplit_feeder(I, Parts) ->\n    {I, Parts}.\n\n%% @doc Splits a continuous interval in X roughly equally-sized subintervals,\n%%      the result of non-continuous intervals is undefined.\n%%      Returns: List of adjacent intervals\n-spec split(continuous_interval(), Parts::pos_integer()) -> [continuous_interval()].\nsplit(I, 1) -> [I];\nsplit(I, Parts) ->\n    {LBr, LKey, RKey, RBr} = get_bounds(I),\n    % keep brackets inside the split interval if they are different\n    % (i.e. one closed, the other open), otherwise exclude split keys\n    % from each first interval at each split\n    {InnerLBr, InnerRBr} =\n        if (LBr =:= '[' andalso RBr =:= ']') orelse\n               (LBr =:= '(' andalso RBr =:= ')') -> {'[', ')'};\n           true -> {LBr, RBr}\n        end,\n    case LKey of\n        RKey -> [I];\n        _ ->\n            SplitKeys = ?RT:get_split_keys(LKey, RKey, Parts),\n            split2(LBr, LKey, RKey, RBr, SplitKeys, InnerLBr, InnerRBr)\n    end.\n\n-compile({nowarn_unused_function, {split2_feeder, 7}}).\n-spec split2_feeder\n        (left_bracket(), key(), key(), right_bracket(), SplitKeys::[?RT:key()],\n         InnerLBr::left_bracket(), InnerRBr::right_bracket())\n        -> {left_bracket(), key(), key(), right_bracket(), SplitKeys::[?RT:key()],\n            InnerLBr::left_bracket(), InnerRBr::right_bracket()};\n        (left_bracket(), key(), ?PLUS_INFINITY_TYPE, ')', SplitKeys::[?RT:key()],\n         InnerLBr::left_bracket(), InnerRBr::right_bracket())\n        -> {left_bracket(), key(), ?PLUS_INFINITY_TYPE, ')', SplitKeys::[?RT:key()],\n            InnerLBr::left_bracket(), InnerRBr::right_bracket()}.\nsplit2_feeder(LBr, LKey, RKey, RBr, SplitKeys, InnerLBr, InnerRBr) ->\n    {LBr, LKey, RKey, RBr,\n     util:shuffle(lists:usort([X || X <- SplitKeys, X =/= LKey])),\n     InnerLBr, InnerRBr}.\n\n-spec split2(left_bracket(), key(), key() | ?PLUS_INFINITY_TYPE,  %% then right_bracket is ')'\n             right_bracket(), SplitKeys::[?RT:key()], InnerLBr::left_bracket(),\n             InnerRBr::right_bracket()) -> [interval()].\nsplit2(LBr, LKey, RKey, RBr, [], _InnerLBr, _InnerRBr) ->\n    [new(LBr, LKey, RKey, RBr)];\nsplit2(LBr, LKey, RKey, RBr, [SplitKey | SplitKeys], InnerLBr, InnerRBr) ->\n    ?DBG_ASSERT(LKey =/= SplitKey),\n    [new(LBr, LKey, SplitKey, InnerRBr) |\n         split2(InnerLBr, SplitKey, RKey, RBr, SplitKeys, InnerLBr, InnerRBr)].\n\n%% @doc returns a list of simple intervals that make up Interval\n-spec get_simple_intervals(Interval::interval()) -> [simple_interval()].\nget_simple_intervals(Interval) ->\n    Interval.\n\n%% @doc Converts a simple interval (e.g. from get_simple_intervals/1) to a\n%%      valid interval().\n-spec simple_interval_to_interval(SInterval::simple_interval()) -> interval().\nsimple_interval_to_interval(SInterval) ->\n    normalize_simple(SInterval).\n"
  },
  {
    "path": "src/json/api_json.erl",
    "content": "%% @copyright 2011-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc JSON API for transactional, consistent access to replicated DHT items\n-module(api_json).\n-author('schintke@zib.de').\n\n-export([handler/2]).\n\n%% for the api_json_* modules:\n-export([tuple_list_to_json/1]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% main handler for json calls\n-spec handler(atom(), list()) -> any().\nhandler(nop, [_Value]) -> \"ok\";\n\nhandler(range_read, [From, To])              -> api_json_dht_raw:range_read(From, To);\nhandler(delete, [Key])                       -> api_json_rdht:delete(Key);\nhandler(delete, [Key, Timeout])              -> api_json_rdht:delete(Key, Timeout);\nhandler(req_list, [Param])                   -> api_json_tx:req_list(Param);\nhandler(req_list, [TLog, ReqList])           -> api_json_tx:req_list(TLog, ReqList);\nhandler(read, [Key])                         -> api_json_tx:read(Key);\nhandler(write, [Key, Value])                 -> api_json_tx:write(Key, Value);\nhandler(add_del_on_list, [Key, ToAdd, ToRemove])\n                                             -> api_json_tx:add_del_on_list(Key, ToAdd, ToRemove);\nhandler(add_on_nr, [Key, ToAdd])             -> api_json_tx:add_on_nr(Key, ToAdd);\nhandler(test_and_set, [Key, OldV, NewV])     -> api_json_tx:test_and_set(Key, OldV, NewV);\nhandler(req_list_commit_each, [Param])       -> api_json_tx:req_list_commit_each(Param);\n\nhandler(get_node_info, [])                   -> api_json_monitor:get_node_info();\nhandler(get_node_performance, [])            -> api_json_monitor:get_node_performance();\nhandler(get_service_info, [])                -> api_json_monitor:get_service_info();\nhandler(get_service_performance, [])         -> api_json_monitor:get_service_performance();\n\nhandler(get_replication_factor, [])          -> api_json_rt:get_replication_factor();\n\nhandler(rbr_read, [Key])                     -> api_json_rbr:read(Key);\nhandler(rbr_write, [Key, Value])             -> api_json_rbr:write(Key, Value);\n\nhandler(get_ring_size, [TimeOut])            -> api_json_ring:get_ring_size(TimeOut);\nhandler(wait_for_ring_size, [Size, TimeOut]) -> api_json_ring:wait_for_ring_size(Size, TimeOut);\n\nhandler(run_benchmark_incr, [])              -> api_jsonclient:run_benchmark_incr();\nhandler(run_benchmark_read, [])              -> api_jsonclient:run_benchmark_read();\n\nhandler(AnyOp, AnyParams) ->\n    io:format(\"Unknown request = ~s:~p(~p)~n\", [?MODULE, AnyOp, AnyParams]),\n    {struct, [{failure, \"unknownreq\"}]}.\n\n%% @doc Recursively converts 2-tuple lists to JSON objects.\n-spec tuple_list_to_json([{Key::atom(), Value::term()}]) -> {struct, [{Key::atom(), Value::term()}]}.\ntuple_list_to_json([]) -> [];\ntuple_list_to_json([{Key, _} | _] = List) when is_atom(Key) ->\n    {struct, [{KeyX, tuple_list_to_json(ValueX)} || {KeyX, ValueX} <- List]};\ntuple_list_to_json(X) -> X.\n"
  },
  {
    "path": "src/json/api_json_autoscale.erl",
    "content": "%% @copyright 2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Ufuk Celebi <celebi@zib.de>\n%% @doc JSON API for polling autoscale VM requests.\n%% @version $Id$\n-module(api_json_autoscale).\n-author('celebi@zib.de').\n-vsn('$Id$').\n\n-export([handler/2]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% @doc main handler for json calls\n-spec handler(atom(), list()) -> any().\nhandler(nop, [_Value]) -> \"ok\";\n\nhandler(check_config , [])    -> check_config();\nhandler(pull_scale_req  , []) -> pull_scale_req();\nhandler(lock_scale_req, [])   -> lock_scale_req();\nhandler(unlock_scale_req, []) -> unlock_scale_req();\n\nhandler(AnyOp, AnyParams) ->\n    io:format(\"Unknown request = ~s:~p(~p)~n\", [?MODULE, AnyOp, AnyParams]),\n    {struct, [{failure, \"unknownreq\"}]}.\n\n%% @doc Call api_autoscale and return:\n%%        {'status': 'ok'} -or-\n%%        {'status': 'error'}\n-spec check_config() -> {struct, [{Key::atom(), Value::term()}]}.\ncheck_config() ->\n    {struct, [{status, case api_autoscale:check_config() of\n                           true  -> \"ok\";\n                           false -> \"error\"\n                       end}]}.\n\n%% @doc Call api_autoscale and return:\n%%        {'status': 'ok', 'value': number} -or-\n%%        {'status': 'error', 'reason': reason}\n-spec pull_scale_req() -> {struct, [{Key::atom(), Value::term()}]}.\npull_scale_req() ->\n    {Status, Value} = api_autoscale:pull_scale_req(),\n    {struct, [{status, atom_to_list(Status)},\n              case erlang:is_integer(Value) of\n                  true  -> {value, Value};\n                  false -> {reason, atom_to_list(Value)}\n              end]}.\n%% @doc Call api_autoscale and return:\n%%        {'status': 'ok'} -or-\n%%        {'status': 'error', 'reason': reason}\n-spec lock_scale_req() -> {struct, [{Key::atom(), Value::term()}]}.\nlock_scale_req() ->\n    case api_autoscale:lock_scale_req() of\n        ok ->\n            {struct, [{status, \"ok\"}]};\n        {error, Reason} ->\n            {struct, [{status, \"error\"}, {reason, atom_to_list(Reason)}]}\n    end.\n\n%% @doc Call api_autoscale and return:\n%%        {'status': 'ok'} -or-\n%%        {'status': 'error', 'reason': reason}\n-spec unlock_scale_req() -> {struct, [{Key::atom(), Value::term()}]}.\nunlock_scale_req() ->\n    case api_autoscale:unlock_scale_req() of\n        ok ->\n            {struct, [{status, \"ok\"}]};\n        {error, Reason} ->\n            {struct, [{status, \"error\"}, {reason, atom_to_list(Reason)}]}\n    end.\n"
  },
  {
    "path": "src/json/api_json_dht_raw.erl",
    "content": "%% @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc JSON API for access to raw DHT functions.\n%% @version $Id$\n-module(api_json_dht_raw).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-export([handler/2]).\n\n% for api_json:\n-export([range_read/2]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% main handler for json calls\n-spec handler(atom(), list()) -> any().\nhandler(nop, [_Value]) -> \"ok\";\n\nhandler(range_read, [From, To])          -> range_read(From, To);\n\nhandler(AnyOp, AnyParams) ->\n    io:format(\"Unknown request = ~s:~p(~p)~n\", [?MODULE, AnyOp, AnyParams]),\n    {struct, [{failure, \"unknownreq\"}]}.\n\n-spec range_read(intervals:key(), intervals:key()) -> api_json_tx:result().\nrange_read(From, To) ->\n    {ErrorCode, Data} = api_dht_raw:range_read(From, To),\n    {struct, [{status, atom_to_list(ErrorCode)}, {value, data_to_json(Data)}]}.\n\n-spec data_to_json(Data::[db_entry:entry()]) ->\n            {array, [{struct, [{key, ?RT:key()} | \n                               {value, api_json_tx:value()} |\n                               {version, client_version()}]\n                     }]}.\ndata_to_json(Data) ->\n    {array, [ {struct, [{key, db_entry:get_key(DBEntry)},\n                        api_json_tx:value_to_json(db_entry:get_value(DBEntry)),\n                        {version, db_entry:get_version(DBEntry)}]} ||\n              DBEntry <- Data]}.\n"
  },
  {
    "path": "src/json/api_json_monitor.erl",
    "content": "%% @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc JSON API for accessing monitoring data.\n%% @version $Id$\n-module(api_json_monitor).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-export([handler/2]).\n\n% for api_json:\n-export([get_node_info/0, get_node_performance/0,\n         get_service_info/0, get_service_performance/0]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% main handler for json calls\n-spec handler(atom(), list()) -> any().\nhandler(nop, [_Value]) -> \"ok\";\n\nhandler(get_node_info, [])               -> get_node_info();\nhandler(get_node_performance, [])        -> get_node_performance();\nhandler(get_service_info, [])            -> get_service_info();\nhandler(get_service_performance, [])     -> get_service_performance();\n\nhandler(AnyOp, AnyParams) ->\n    io:format(\"Unknown request = ~s:~p(~p)~n\", [?MODULE, AnyOp, AnyParams]),\n    {struct, [{failure, \"unknownreq\"}]}.\n\n%% interface for monitoring calls\n-spec get_node_info() -> {struct, [{Key::atom(), Value::term()}]}.\nget_node_info() ->\n    {struct, [{status, \"ok\"}, {value, api_json:tuple_list_to_json(api_monitor:get_node_info())}]}.\n\n-spec get_node_performance() -> {struct, [{Key::atom(), Value::term()}]}.\nget_node_performance() ->\n    {struct, [{status, \"ok\"}, {value, api_json:tuple_list_to_json(api_monitor:get_node_performance())}]}.\n\n-spec get_service_info() -> {struct, [{Key::atom(), Value::term()}]}.\nget_service_info() ->\n    {struct, [{status, \"ok\"}, {value, api_json:tuple_list_to_json(api_monitor:get_service_info())}]}.\n\n-spec get_service_performance() -> {struct, [{Key::atom(), Value::term()}]}.\nget_service_performance() ->\n    {struct, [{status, \"ok\"}, {value, api_json:tuple_list_to_json(api_monitor:get_service_performance())}]}.\n"
  },
  {
    "path": "src/json/api_json_rbr.erl",
    "content": "%% @copyright 2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc JSON API for using rbr.\n-module(api_json_rbr).\n-author('schuett@zib.de').\n\n-export([handler/2]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n% for api_json:\n-export([read/1, write/2]).\n\n%% main handler for json calls\n-spec handler(atom(), list()) -> any().\nhandler(nop, [_Value]) -> \"ok\";\n\nhandler(read, [Key])                     -> read(Key);\nhandler(write, [Key, Value])             -> write(Key, Value);\n\nhandler(AnyOp, AnyParams) ->\n    io:format(\"Unknown request = ~s:~p(~p)~n\", [?MODULE, AnyOp, AnyParams]),\n    {struct, [{failure, \"unknownreq\"}]}.\n\n-spec read(client_key()) -> read_result().\nread(Key) ->\n    %% api_tx:read_result()\n    Result = kv_on_cseq:read(Key),\n    Json = result_to_json(Result),\n    io:format(\"~w~n\", [Json]),\n    Json.\n\n-spec write(client_key(), term()) -> write_result().\nwrite(Key, Value) ->\n    Result = kv_on_cseq:write(Key, Value),\n    result_to_json(Result).\n\n%% TODO copied from api_json_tx\n\n-type value() :: {struct, [{type | string(), string()} | %% {\"type\", \"as_is\" | \"as_bin\"}\n                           {value | string(), client_value()} %% {\"value\", ...}\n                          ]}.\n-type read_result() ::\n        {struct, [{status, string()}       %% \"ok\", \"fail\"\n                  | {reason, string()}     %% \"timeout\", \"not_found\"\n                  | {value, value()} ]}.\n-type write_result() ::\n        {struct, [{status, string()}       %% \"ok\", \"fail\"\n                  | {reason, string()} ]}. %% \"timeout\"\n\n-type result() :: read_result() | write_result().\n\n-spec result_to_json(Result::api_tx:result()) -> result().\nresult_to_json(Result) ->\n    {struct,\n     case Result of\n         {ok}                       -> [{status, \"ok\"}];\n         {ok, Val}                  -> [{status, \"ok\"},\n                                        value_to_json(Val)];\n         {fail, Reason}             -> [{status, \"fail\"},\n                                        {reason, atom_to_list(Reason)}]\n     end\n    }.\n\n-spec value_to_json(client_value()) -> {value, value()}.\nvalue_to_json(Value) when is_binary(Value) ->\n    {value, {struct, [{type, \"as_bin\"}, {value, base64:encode_to_string(Value)}]}};\nvalue_to_json(Value) ->\n    {value, {struct, [{type, \"as_is\"}, {value, Value}]}}.\n\n"
  },
  {
    "path": "src/json/api_json_rdht.erl",
    "content": "%% @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc JSON API for accessing replicated DHT functions.\n%% @version $Id$\n-module(api_json_rdht).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-export([handler/2]).\n\n% for api_json:\n-export([delete/1, delete/2]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% main handler for json calls\n-spec handler(atom(), list()) -> any().\nhandler(nop, [_Value]) -> \"ok\";\nhandler(delete, [Key])                   -> delete(Key);\nhandler(delete, [Key, Timeout])          -> delete(Key, Timeout);\n\nhandler(AnyOp, AnyParams) ->\n    io:format(\"Unknown request = ~s:~p(~p)~n\", [?MODULE, AnyOp, AnyParams]),\n    {struct, [{failure, \"unknownreq\"}]}.\n\n-type delete_result() :: {struct, [{failure, string()} |\n                                        {ok, non_neg_integer()} |\n                                        {results, {array, [string()]}}]}.\n-spec delete(client_key()) -> delete_result().\ndelete(Key) ->\n    delete(Key, 2000).\n\n-spec delete(client_key(), Timeout::pos_integer()) -> delete_result().\ndelete(Key, Timeout) ->\n    case api_rdht:delete(Key, Timeout) of\n        {fail, Reason, NumOK, StateList} ->\n            {struct, [{failure, atom_to_list(Reason)},\n                      {ok, NumOK},\n                      {results, {array, [atom_to_list(X) || X <- StateList]}}]};\n        {ok, NumOk, StateList} ->\n            {struct, [{ok, NumOk},\n                      {results, {array, [atom_to_list(X) || X <- StateList ]}}]}\n    end.\n"
  },
  {
    "path": "src/json/api_json_ring.erl",
    "content": "%% @copyright 2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc JSON API for querying rings.\n-module(api_json_ring).\n-author('schuett@zib.de').\n\n% for api_json:\n-export([get_ring_size/1, wait_for_ring_size/2]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-spec get_ring_size(TimeOut::integer()) -> integer() | failed.\nget_ring_size(TimeOut) ->\n    api_ring:get_ring_size(TimeOut).\n\n-spec wait_for_ring_size(Size::integer(), TimeOut::integer()) -> [char(),...].\nwait_for_ring_size(Size, TimeOut) ->\n    api_ring:wait_for_ring_size(Size, TimeOut),\n    \"ok\".\n"
  },
  {
    "path": "src/json/api_json_rt.erl",
    "content": "%% @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc JSON API for using routing tables.\n%% @version $Id$\n-module(api_json_rt).\n-author('schuett@zib.de').\n\n-export([handler/2]).\n\n% for api_json:\n-export([get_replication_factor/0]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% main handler for json calls\n-spec handler(atom(), list()) -> any().\nhandler(nop, [_Value]) -> \"ok\";\n\nhandler(get_replication_factor, [])      -> get_replication_factor();\n\nhandler(AnyOp, AnyParams) ->\n    io:format(\"Unknown request = ~s:~p(~p)~n\", [?MODULE, AnyOp, AnyParams]),\n    {struct, [{failure, \"unknownreq\"}]}.\n\n%% interface for rt calls\n-spec get_replication_factor() -> {struct, [{Key::atom(), Value::term()}]}.\nget_replication_factor() ->\n    {struct, [{status, \"ok\"}, {value, api_rt:get_replication_factor()}]}.\n"
  },
  {
    "path": "src/json/api_json_tx.erl",
    "content": "%% @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc JSON API for transactional, consistent access to replicated DHT items.\n%% @version $Id$\n-module(api_json_tx).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-export([handler/2]).\n\n% for the other JSON-RPC modules:\n-export([value_to_json/1, json_to_value/1, \n         results_to_json/1, result_to_json/1]).\n\n% for api_json:\n-export([req_list/1, req_list/2,\n         read/1, write/2, add_del_on_list/3, add_on_nr/2, test_and_set/3,\n         req_list_commit_each/1]).\n\n-export_type([value/0, request/0,\n              read_result/0, write_result/0, commit_result/0, result/0]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% main handler for json calls\n-spec handler(atom(), list()) -> any().\nhandler(nop, [_Value]) -> \"ok\";\n\nhandler(req_list, [Param])               -> req_list(Param);\nhandler(req_list, [TLog, ReqList])       -> req_list(TLog, ReqList);\nhandler(read, [Key])                     -> read(Key);\nhandler(write, [Key, Value])             -> write(Key, Value);\nhandler(add_del_on_list, [Key, ToAdd, ToRemove])\n                                         -> add_del_on_list(Key, ToAdd, ToRemove);\nhandler(add_on_nr, [Key, ToAdd])         -> add_on_nr(Key, ToAdd);\nhandler(test_and_set, [Key, OldV, NewV]) -> test_and_set(Key, OldV, NewV);\nhandler(req_list_commit_each, [Param])   -> req_list_commit_each(Param);\n\nhandler(AnyOp, AnyParams) ->\n    io:format(\"Unknown request = ~s:~p(~p)~n\", [?MODULE, AnyOp, AnyParams]),\n    {struct, [{failure, \"unknownreq\"}]}.\n\n\n%% interface for api_tx calls\n% Public Interface\n-type value() :: {struct, [{type | string(), string()} | %% {\"type\", \"as_is\" | \"as_bin\"}\n                           {value | string(), client_value()} %% {\"value\", ...}\n                          ]}.\n\n-type request() :: {string(), client_key()} % {\"read\", ...}\n                 | {string(), {struct, [{client_key(), value()}]}} % {\"write\", {struct, [Key, Value]}}\n                   % {\"add_del_on_list\", {struct, [...]}}\n                 | {string(), {struct, [{string(), client_key()} | % {\"key\", Key}\n                                        {string(), {array, [client_value()]}} % {\"add\", {array, ToAdd}}, {\"del\", {array, ToRemove}}\n                                       ]}}\n                 | {string(), {struct, [{client_key(), client_value()}]}} % {\"add_on_nr\", {struct, [Key, ToAdd]}}\n\n                   % {\"test_and_set\", {struct, [...]}}\n                 | {string(), {struct, [{string(), client_key()} | % {\"key\", Key}\n                                        {string(), value()} % {\"new\", New}, {\"old\", Old}\n                                       ]}}\n                 | {string(), any()}. % {\"commit\", _}\n\n-type read_result() ::\n        {struct, [{status, string()}       %% \"ok\", \"fail\"\n                  | {reason, string()}     %% \"timeout\", \"not_found\"\n                  | {value, value()} ]}.\n-type write_result() ::\n        {struct, [{status, string()}       %% \"ok\", \"fail\"\n                  | {reason, string()} ]}. %% \"timeout\"\n-type commit_result() ::\n        {struct, [{status, string()}       %% \"ok\", \"fail\"\n                  | {reason, string()}     %% \"timeout\"\n                  | {keys, {array, [string()]}} ]}. %% \"abort\"\n\n-type result() :: read_result() | write_result() | commit_result().\n\n-spec req_list({array, [request()]})\n                 -> {struct, [{tlog, string()}\n                              | {results, {array, [result()]}}]}.\nreq_list(ReqList) ->\n    JSON_TLog = tlog_to_json(api_tx:new_tlog()),\n    req_list(JSON_TLog, ReqList).\n\n-spec req_list(string(), {array, [request()]})\n                 -> {struct, [{tlog, string()}\n                              | {results, {array, [result()]}}]}.\nreq_list(JSON_TLog, JSON_ReqList) ->\n    TLog = json_to_tlog(JSON_TLog),\n    ReqList = json_to_reqlist(JSON_ReqList, true),\n    {NewTLog, Res} = api_tx:req_list(TLog, ReqList),\n    {struct, [{tlog, tlog_to_json(NewTLog)},\n              {results, results_to_json(Res)}]}.\n\n-spec read(client_key()) -> read_result().\nread(Key) ->\n    Res = api_tx:read(Key),\n    result_to_json(Res).\n\n-spec write(client_key(), value()) -> commit_result().\nwrite(Key, Value) ->\n    Res = api_tx:write(Key, json_to_value(Value)),\n    result_to_json(Res).\n\n-spec add_del_on_list(client_key(), ToAdd::{array, [client_value()]},\n                         ToRemove::{array, [client_value()]})\n                     -> commit_result() |\n                        {struct, [{status, string()}  %% \"fail\"\n                                 | {reason, string()} ]}. %% \"not_a_list\"\nadd_del_on_list(Key, {array, ToAdd}, {array, ToRemove}) ->\n    Res = api_tx:add_del_on_list(Key, ToAdd, ToRemove),\n    result_to_json(Res);\nadd_del_on_list(_Key, _ToAdd, _ToRemove) ->\n    result_to_json({fail, not_a_list}).\n\n-spec add_on_nr(client_key(), ToAdd::number())\n                     -> commit_result() |\n                        {struct, [{status, string()}  %% \"fail\"\n                                 | {reason, string()} ]}. %% \"not_a_number\"\nadd_on_nr(Key, ToAdd) ->\n    Res = api_tx:add_on_nr(Key, ToAdd),\n    result_to_json(Res).\n\n-spec test_and_set(client_key(),\n                      OldValue::value(),\n                      NewValue::value())\n                     -> commit_result() |\n                        {struct, [{status, string()}  %% \"fail\"\n                                 | {reason, string()} %% \"key_changed\", \"not_found\"\n                                 | {value, value()} ]}. %% for key_changed\ntest_and_set(Key, OldValue, NewValue) ->\n    OldRealValue = json_to_value(OldValue),\n    NewRealValue = json_to_value(NewValue),\n    Res = api_tx:test_and_set(Key, OldRealValue, NewRealValue),\n    result_to_json(Res).\n\n-spec req_list_commit_each({array, [request()]}) -> {array, [result()]}.\nreq_list_commit_each(JSON_ReqList) ->\n    ReqList = json_to_reqlist(JSON_ReqList, false),\n    Res = api_tx:req_list_commit_each(ReqList),\n    results_to_json(Res).\n\n-spec results_to_json(Results::[api_tx:result()]) -> {array, [result()]}.\nresults_to_json(Results) ->\n    Entries = [ result_to_json(Result) || Result <- Results ],\n    {array, Entries}.\n\n-spec result_to_json(Result::api_tx:result()) -> result().\nresult_to_json(Result) ->\n    {struct,\n     case Result of\n         {ok}                       -> [{status, \"ok\"}];\n         {ok, Val}                  -> [{status, \"ok\"},\n                                        value_to_json(Val)];\n         {fail, {key_changed, Val}} -> [{status, \"fail\"},\n                                        {reason, \"key_changed\"},\n                                        value_to_json(Val)];\n         {fail, Reason}             -> [{status, \"fail\"},\n                                        {reason, atom_to_list(Reason)}];\n         {fail, abort, Keys}        -> [{status, \"fail\"},\n                                        {reason, \"abort\"},\n                                        {keys,   {array, Keys}}]\n     end\n    }.\n\n-spec json_to_reqlist(JSON_ReqList::{array, [request()]}, AllowCommit::boolean()) -> [api_tx:request()].\njson_to_reqlist({array, TmpReqList}, AllowCommit) ->\n    [ case Elem of\n          {\"read\", Key} ->\n              {read, Key};\n          {\"write\", {struct, [{Key, Val}]}} ->\n              {write, Key, json_to_value(Val)};\n          % note: struct properties may be arbitrarily sorted (common sorting: alphabetically)\n          {\"add_del_on_list\", {struct, PropList}} ->\n              case PropList of\n                  [{\"add\", {array, ToAdd}}, {\"del\", {array, ToRemove}}, {\"key\", Key}] -> ok;\n                  [{\"add\", {array, ToAdd}}, {\"key\", Key}, {\"del\", {array, ToRemove}}] -> ok;\n                  [{\"del\", {array, ToRemove}}, {\"add\", {array, ToAdd}}, {\"key\", Key}] -> ok;\n                  [{\"del\", {array, ToRemove}}, {\"key\", Key}, {\"add\", {array, ToAdd}}] -> ok;\n                  [{\"key\", Key}, {\"add\", {array, ToAdd}}, {\"del\", {array, ToRemove}}] -> ok;\n                  [{\"key\", Key}, {\"del\", {array, ToRemove}}, {\"add\", {array, ToAdd}}] -> ok\n              end,\n              {add_del_on_list, Key, ToAdd, ToRemove};\n          {\"add_on_nr\", {struct, [{Key, ToAdd}]}} ->\n              {add_on_nr, Key, ToAdd};\n          {\"test_and_set\", {struct, PropList}} ->\n              case PropList of\n                  [{\"key\", Key}, {\"new\", New}, {\"old\", Old}] -> ok;\n                  [{\"key\", Key}, {\"old\", Old}, {\"new\", New}] -> ok;\n                  [{\"new\", New}, {\"key\", Key}, {\"old\", Old}] -> ok;\n                  [{\"new\", New}, {\"old\", Old}, {\"key\", Key}] -> ok;\n                  [{\"old\", Old}, {\"new\", New}, {\"key\", Key}] -> ok;\n                  [{\"old\", Old}, {\"key\", Key}, {\"new\", New}] -> ok\n              end,\n              {test_and_set, Key, json_to_value(Old), json_to_value(New)};\n          {\"commit\", _} when AllowCommit ->\n              {commit}\n      end || {struct, [Elem]} <- TmpReqList ].\n\n-spec tlog_to_json(TLog::tx_tlog:tlog()) -> string().\ntlog_to_json(TLog) ->\n    base64:encode_to_string(term_to_binary(TLog, [compressed, {minor_version, 1}])).\n\n-spec json_to_tlog(JsonTLog::string()) -> tx_tlog:tlog().\njson_to_tlog(JsonTLog) ->\n    binary_to_term(base64:decode(JsonTLog)).\n\n-spec value_to_json(client_value()) -> {value, value()}.\nvalue_to_json(Value) when is_binary(Value) ->\n    {value, {struct, [{type, \"as_bin\"}, {value, base64:encode_to_string(Value)}]}};\nvalue_to_json(Value) ->\n    {value, {struct, [{type, \"as_is\"}, {value, Value}]}}.\n\n-spec json_to_value(value()) -> client_value().\njson_to_value({struct, PropList}) ->\n    case PropList of\n        [{\"type\", Type}, {\"value\", Value}] -> ok;\n        [{\"value\", Value}, {\"type\", Type}] -> ok\n    end,\n    case Type of\n        \"as_bin\" -> base64:decode(Value);\n        \"as_is\"  -> json_array_to_value(Value)\n    end.\n\n%% @doc Converts {array, List} to List recursively. If the given value is not a\n%%      JSON list the value is returned unchanged.\n-spec json_array_to_value(value()) -> client_value().\njson_array_to_value({array, List}) when is_list(List) ->\n    [json_array_to_value(X) || X <- List];\njson_array_to_value(Value) ->\n    Value.\n"
  },
  {
    "path": "src/json/api_json_vm.erl",
    "content": "%% @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc JSON API for accessing an Erlang VM.\n%% @version $Id$\n-module(api_json_vm).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-export([handler/2]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% main handler for json calls\n-spec handler(atom(), list()) -> any().\nhandler(nop, [_Value]) -> \"ok\";\n\nhandler(get_version, [])                 -> get_version();\nhandler(get_info, [])                    -> get_info();\n\nhandler(number_of_nodes, [])             -> number_of_nodes();\nhandler(get_nodes, [])                   -> get_nodes();\nhandler(add_nodes, [Number])             -> add_nodes(Number);\n\nhandler(shutdown_node, [Name])           -> shutdown_node(Name);\nhandler(shutdown_nodes, [Count])         -> shutdown_nodes(Count);\nhandler(shutdown_nodes_by_name, [Names]) -> shutdown_nodes_by_name(Names);\n\nhandler(kill_node, [Name])               -> kill_node(Name);\nhandler(kill_nodes, [Count])             -> kill_nodes(Count);\nhandler(kill_nodes_by_name, [Names])     -> kill_nodes_by_name(Names);\n\nhandler(get_other_vms, [MaxVMs])         -> get_other_vms(MaxVMs);\n\nhandler(shutdown_vm, [])                 -> shutdown_vm();\nhandler(kill_vm, [])                     -> kill_vm();\n\nhandler(AnyOp, AnyParams) ->\n    io:format(\"Unknown request = ~s:~p(~p)~n\", [?MODULE, AnyOp, AnyParams]),\n    {struct, [{failure, \"unknownreq\"}]}.\n\n%% interface for vm calls\n-spec get_version() -> {struct, [{Key::atom(), Value::term()}]}.\nget_version() ->\n    {struct, [{status, \"ok\"}, {value, api_vm:get_version()}]}.\n\n-spec get_info() -> {struct, [{Key::atom(), Value::term()}]}.\nget_info() ->\n    % need to transform node name and IP address to strings:\n    Info = [case I of\n                {erlang_node, Node} ->\n                    {erlang_node, lists:flatten(io_lib:format(\"~s\", [Node]))};\n                {ip, {IP1, IP2, IP3, IP4}} ->\n                    {ip, lists:flatten(io_lib:format(\"~B.~B.~B.~B\", [IP1, IP2, IP3, IP4]))};\n                X -> X\n            end || I <- api_vm:get_info()],\n    {struct, [{status, \"ok\"}, {value, api_json:tuple_list_to_json(Info)}]}.\n\n-spec number_of_nodes() -> {struct, [{Key::atom(), Value::term()}]}.\nnumber_of_nodes() ->\n    {struct, [{status, \"ok\"}, {value, api_vm:number_of_nodes()}]}.\n\n-spec get_nodes() -> {struct, [{Key::atom(), Value::term()}]}.\nget_nodes() ->\n    {struct, [{status, \"ok\"}, {value, {array, [pid_groups:group_to_string(Group) || Group <- api_vm:get_nodes()]}}]}.\n\n-spec add_nodes(Number::non_neg_integer()) -> {struct, [{Key::atom(), Value::term()}]}.\nadd_nodes(Number) ->\n    {Ok, Failed1} = api_vm:add_nodes(Number),\n    Failed = [lists:flatten(io_lib:format(\"~p\", [Reason])) || {error, Reason} <- Failed1],\n    {struct, [{status, \"ok\"},\n              {ok, {array, [pid_groups:group_to_string(Group) || Group <- Ok]}},\n              {failed, {array, Failed}}]}.\n\n-spec shutdown_node(Name::nonempty_string()) -> {struct, [{Key::atom(), Value::term()}]}.\nshutdown_node(Name) ->\n    {struct, [{status, erlang:atom_to_list(api_vm:shutdown_node(pid_groups:string_to_group(Name)))}]}.\n\n-spec shutdown_nodes(Count::non_neg_integer()) -> {struct, [{Key::atom(), Value::term()}]}.\nshutdown_nodes(Count) ->\n    {struct, [{status, \"ok\"}, {ok, {array, [pid_groups:group_to_string(Group) || Group <- api_vm:shutdown_nodes(Count)]}}]}.\n\n-spec shutdown_nodes_by_name(Names::{array, [nonempty_string()]}) -> {struct, [{Key::atom(), Value::term()}]}.\nshutdown_nodes_by_name({array, Names}) ->\n    {Ok, NotFound} = api_vm:shutdown_nodes_by_name(\n                       [pid_groups:string_to_group(Group) || Group <- Names]),\n    Ok2 = [pid_groups:group_to_string(Group) || Group <- Ok],\n    NotFound2 = [pid_groups:group_to_string(Group) || Group <- NotFound],\n    {struct, [{status, \"ok\"}, {ok, {array, Ok2}}, {not_found, {array, NotFound2}}]}.\n\n-spec kill_node(Name::nonempty_string()) -> {struct, [{Key::atom(), Value::term()}]}.\nkill_node(Name) ->\n    {struct, [{status, erlang:atom_to_list(api_vm:kill_node(pid_groups:string_to_group(Name)))}]}.\n\n-spec kill_nodes(Count::non_neg_integer()) -> {struct, [{Key::atom(), Value::term()}]}.\nkill_nodes(Count) ->\n    Ok = [pid_groups:group_to_string(Group) || Group <- api_vm:kill_nodes(Count)],\n    {struct, [{status, \"ok\"}, {ok, {array, Ok}}]}.\n\n-spec kill_nodes_by_name(Names::{array, [nonempty_string()]}) -> {struct, [{Key::atom(), Value::term()}]}.\nkill_nodes_by_name({array, Names}) ->\n    {Ok, NotFound} = api_vm:kill_nodes_by_name(\n                       [pid_groups:string_to_group(Group) || Group <- Names]),\n    Ok2 = [pid_groups:group_to_string(Group) || Group <- Ok],\n    NotFound2 = [pid_groups:group_to_string(Group) || Group <- NotFound],\n    {struct, [{status, \"ok\"}, {ok, {array, Ok2}}, {not_found, {array, NotFound2}}]}.\n\n-spec get_other_vms(MaxVMs::pos_integer()) -> {struct, [{Key::atom(), Value::term()}]}.\nget_other_vms(MaxVMs) ->\n    OtherVMs = [{struct, [{erlang_node, erlang:atom_to_list(ErlNode)},\n                          {ip, lists:flatten(io_lib:format(\"~B.~B.~B.~B\", [IP1, IP2, IP3, IP4]))},\n                          {port, Port},\n                          {yaws_port, YawsPort}]}\n               || {ErlNode, {IP1, IP2, IP3, IP4}, Port, YawsPort} <- api_vm:get_other_vms(MaxVMs)],\n    {struct, [{status, \"ok\"}, {value, {array, OtherVMs}}]}.\n\n-spec shutdown_vm() -> {struct, [{Key::atom(), Value::term()}]}.\nshutdown_vm() ->\n    Result = api_vm:shutdown_vm(),\n    {struct, [{status, erlang:atom_to_list(Result)}]}.\n\n-spec kill_vm() -> {struct, [{Key::atom(), Value::term()}]}.\nkill_vm() ->\n    Result = api_vm:kill_vm(),\n    {struct, [{status, erlang:atom_to_list(Result)}]}.\n"
  },
  {
    "path": "src/json/api_jsonclient.erl",
    "content": "%% @copyright 2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc JSON API for talking to Scalaris.\n-module(api_jsonclient).\n-author('schuett@zib.de').\n\n% for api_json:\n-export([run_benchmark_read/0, run_benchmark_incr/0]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-spec run_benchmark_incr() -> {struct, [{Key::atom(), Value::term()}]}.\nrun_benchmark_incr() ->\n    {ok, IncrResult} = bench:increment_o(10, 500, []),\n    bench_json_helper:result_to_json(IncrResult).\n\n-spec run_benchmark_read() -> {struct, [{Key::atom(), Value::term()}]}.\nrun_benchmark_read() ->\n    {ok, ReadResult} = bench:quorum_read_o(10, 500, []),\n    bench_json_helper:result_to_json(ReadResult).\n"
  },
  {
    "path": "src/json/bench_json_helper.erl",
    "content": "%% @copyright 2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Helper for jsonifying bench results.\n-module(bench_json_helper).\n-author('schuett@zib.de').\n\n-export([result_to_json/1, json_to_result/1]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-spec json_to_result(term()) -> term().\njson_to_result(Result) ->\n    jsonbench_to_result(Result).\n\n-spec result_to_json(list()) -> {struct, [{Key::atom(), Value::term()}]}.\nresult_to_json(List) ->\n    {struct, [value_to_json(KeyX, ValueX) || {KeyX, ValueX} <- List]}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% result to json\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nvalue_to_json(servers, ServerList) ->\n    {servers, {array, [ server_to_json(Server) || Server <- ServerList]}};\nvalue_to_json(threads_per_vm, ThreadsPerVM) ->\n    {threads_per_vm, ThreadsPerVM};\nvalue_to_json(iterations, Iterations) ->\n    {iterations, Iterations};\nvalue_to_json(statistics, Statistics) ->\n    {statistics, {array, [statistic_to_json(S) || S <- Statistics]}};\nvalue_to_json(wall_clock_time, WallClockTime) ->\n    {wall_clock_time, WallClockTime};\nvalue_to_json(wall_clock_throughput, WallClockTP) ->\n    {wall_clock_throughput, WallClockTP};\nvalue_to_json(wall_clock_latency, WallClockLat) ->\n    {wall_clock_latency, WallClockLat};\nvalue_to_json(min_throughput_overall, MinTP) ->\n    {min_throughput_overall, MinTP};\nvalue_to_json(min_throughput_each, MinTPAll) ->\n    {min_throughput_each, {array, MinTPAll}};\nvalue_to_json(mean_throughput_overall, MeanTP) ->\n    {mean_throughput_overall, MeanTP};\nvalue_to_json(mean_throughput_each, MeanTPAll) ->\n    {mean_throughput_each, {array, MeanTPAll}};\nvalue_to_json(max_throughput_overall, MaxTP) ->\n    {max_throughput_overall, MaxTP};\nvalue_to_json(max_throughput_each, MaxTPAll) ->\n    {max_throughput_each, {array, MaxTPAll}};\nvalue_to_json(min_latency_each, MinLatAll) ->\n    {min_latency_each, {array, MinLatAll}};\nvalue_to_json(avg_latency_each, AvgLatAll) ->\n    {avg_latency_each, {array, AvgLatAll}};\nvalue_to_json(max_latency_each, MaxLatAll) ->\n    {max_latency_each, {array, MaxLatAll}};\nvalue_to_json(avg_latency_overall, AvgLat) ->\n    {avg_latency_overall, AvgLat};\nvalue_to_json(avg_exec_time, AvgExTimeAll) ->\n    {avg_exec_time, {array, AvgExTimeAll}};\nvalue_to_json(aborts, Aborts) ->\n    {aborts, {array, Aborts}}.\n\nserver_to_json({{A,B,C,D},Port,Server}) ->\n    TheIP = lists:flatten(io_lib:format(\"~w.~w.~w.~w\", [A,B,C,D])),\n    {struct, [{port, Port},\n              {server, erlang:atom_to_list(Server)},\n              {ip, TheIP}]\n    }.\n\nstatistic_to_json({WallClockTime, MinTime, MeanTime, MaxTime, Variance, Aborts}) ->\n    {struct, [{wall_clock_time, WallClockTime}, {min_time, MinTime}, {mean_time, MeanTime},\n              {max_time, MaxTime}, {variance, Variance}, {aborts, Aborts}]}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% json to result\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\njsonbench_to_result({struct, KVL}) ->\n    [json_to_value(KV) || KV <- KVL].\n\n\njson_to_value({\"servers\", {array, ServerList}}) ->\n    {servers, [ json_to_server(Server) || Server <- ServerList]};\njson_to_value({\"statistics\", {array, Statistics}}) ->\n    {statistics, [json_to_statistics(Statistic) || Statistic <- Statistics]};\njson_to_value({\"min_throughput_each\", {array, MinTPAll}}) ->\n    {min_throughput_each, MinTPAll};\njson_to_value({\"mean_throughput_each\", {array, MeanTPAll}}) ->\n    {mean_throughput_each, MeanTPAll};\njson_to_value({\"max_throughput_each\", {array, MaxTPAll}}) ->\n    {max_throughput_each, MaxTPAll};\njson_to_value({\"min_latency_each\", {array, MinLatAll}}) ->\n    {min_latency_each, MinLatAll};\njson_to_value({\"avg_latency_each\", {array, AvgLatAll}}) ->\n    {avg_latency_each, AvgLatAll};\njson_to_value({\"max_latency_each\", {array, MaxLatAll}}) ->\n    {max_latency_each, MaxLatAll};\njson_to_value({\"avg_exec_time\", {array, AvgExTimeAll}}) ->\n    {avg_exec_time, AvgExTimeAll};\njson_to_value({\"aborts\", {array, Aborts}}) ->\n    {aborts, Aborts};\njson_to_value({Key, Value}) -> %% fallthrough\n    {list_to_atom(Key), Value}.\n\njson_to_server({struct, [{\"port\", Port}, {\"server\", Server}, {\"ip\", IP}]}) ->\n    {IP, Port, list_to_atom(Server)}.\n\njson_to_statistics({struct, [{\"wall_clock_time\", WallClockTime}, {\"min_time\", MinTime},\n                             {\"mean_time\", MeanTime}, {\"max_time\", MaxTime}, {\"variance\", Variance},\n                             {\"aborts\", Aborts}]}) ->\n    {WallClockTime, MinTime, MeanTime, MaxTime, Variance, Aborts}.\n"
  },
  {
    "path": "src/json/jsonclient.erl",
    "content": "%% @copyright 2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc JSON Client.\n-module(jsonclient).\n-author('schuett@zib.de').\n\n-export([get_ring_size/4, wait_for_ring_size/5, run_benchmark/3]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-spec get_ring_size(TimeOut::integer(), IP::{non_neg_integer(), non_neg_integer(), non_neg_integer(), non_neg_integer()}, Port::integer(), SSL::boolean()) -> integer().\nget_ring_size(TimeOut, _IP = {A,B,C,D}, Port, SSL) ->\n    TheIP = lists:flatten(io_lib:format(\"~w.~w.~w.~w\", [A,B,C,D])),\n    doJsonRPC(TheIP, Port, \"jsonrpc.yaws\", \"get_ring_size\", [TimeOut], SSL).\n\n-spec wait_for_ring_size(Size::integer(), TimeOut::integer(), IP::{non_neg_integer(), non_neg_integer(), non_neg_integer(), non_neg_integer()}, Port::integer(), SSL::boolean()) -> string().\nwait_for_ring_size(Size, TimeOut, _IP = {A,B,C,D}, Port, SSL) ->\n    TheIP = lists:flatten(io_lib:format(\"~w.~w.~w.~w\", [A,B,C,D])),\n    doJsonRPC(TheIP, Port, \"jsonrpc.yaws\", \"wait_for_ring_size\", [Size, TimeOut], SSL).\n\n-spec run_benchmark(IP::{non_neg_integer(), non_neg_integer(), non_neg_integer(), non_neg_integer()}, Port::integer(), SSL::boolean()) -> ok.\nrun_benchmark(_IP = {A,B,C,D}, Port, SSL) ->\n    TheIP = lists:flatten(io_lib:format(\"~w.~w.~w.~w\", [A,B,C,D])),\n    io:format(\"running bench:increment(10, 500)...~n\"),\n    Incr = doJsonRPC(TheIP, Port, \"jsonrpc.yaws\", \"run_benchmark_incr\", [], SSL),\n    ResultIncr = bench_json_helper:json_to_result(Incr),\n    bench:print_results(ResultIncr, [print, verbose]),\n    io:format(\"running bench:quorum_read(10, 5000)...~n\"),\n    Read = doJsonRPC(TheIP, Port, \"jsonrpc.yaws\", \"run_benchmark_read\", [], SSL),\n    ResultRead = bench_json_helper:json_to_result(Read),\n    bench:print_results(ResultRead, [print, verbose]),\n    ok.\n\n-spec doJsonRPC(IP::string(), Port::integer(), Path::string(), Call::string(), Params::list(), SSL::boolean()) -> term().\ndoJsonRPC(IP, Port, Path, Call, Params, SSL) ->\n    _ = ssl:start(), %% just in case.  ok | {error, Reason}\n    ContentType = \"application/json\",\n    Json = {struct, [{jsonrpc, \"2.0\"}, {method, Call}, {params, {array, Params}}, {id, 1}]},\n    Body = lists:flatten(json2:encode(Json)),\n    Headers = [{\"User-Agent\", \"Wget/1.19.4 (darwin17.3.0)\"},\n               {\"Accept\", \"*/*\"},\n               {\"Accept-Encoding\", \"identity\"},\n               {\"Connection\", \"Keep-Alive\"},\n               {\"Content-Type\", ContentType},\n               {\"Content-Length\", length(Body)}],\n    Request = { get_url_prefix(SSL) ++ IP ++ \":\" ++ integer_to_list(Port) ++ \"/\" ++ Path, Headers,\n                ContentType, Body},\n    io:format(\"~s~n\", [get_url_prefix(SSL) ++ IP ++ \":\" ++ integer_to_list(Port) ++ \"/\" ++ Path]),\n    HTTPOptions = [{version, \"HTTP/1.1\"}],\n    Options = [{body_format, string}],\n    Result = httpc:request(post, Request, HTTPOptions, Options),\n    case Result of\n        {ok, {_StatusLine, _Headers2, Body2}} ->\n            JsonResponse = json2:decode_string(trim_new_lines(Body2)), % cheap string:trim()\n            case JsonResponse of\n                {ok, {struct, List}} ->\n                    case lists:keyfind(\"result\", 1, List) of\n                        {\"result\", TheResult} ->\n                            TheResult;\n                        false ->\n                            failed\n                    end;\n                X ->\n                    io:format(\"~w~n\", [X])\n            end;\n        {error, Reason} ->\n            {error, Reason}\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% util\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ntrim_new_lines(S) ->\n    trim_new_lines_internal(lists:reverse(S)).\n\ntrim_new_lines_internal([]) ->\n    [];\ntrim_new_lines_internal(S = [First | Rest]) ->\n    case First of\n        [$\\r,$\\n] -> trim_new_lines_internal(Rest);\n        $\\n ->trim_new_lines_internal(Rest);\n        _ -> lists:reverse(S)\n    end.\n\nget_url_prefix(_SSL = true) ->\n    \"https://\";\nget_url_prefix(_SSL = false) ->\n    \"http://\".\n"
  },
  {
    "path": "src/lb_active.erl",
    "content": "%  @copyright 2014-2015 Zuse Institute Berlin\n%\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maximilian Michels <michels@zib.de>\n%% @doc Active load balancing core module\n%% @version $Id$\n-module(lb_active).\n-author('michels@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-define(TRACE(X,Y), ok).\n%-define(TRACE(X,Y), io:format(\"lb_active: \" ++ X, Y)).\n\n% debug expressions turned off\n-define(DEBUG(E), ok).\n\n%% startup\n-export([start_link/1, check_config/0, is_enabled/0, check_gossip_modules/2]).\n%% gen_component\n-export([init/1, on/2]).\n%% for calls from the dht node\n-export([handle_dht_msg/2]).\n% Load Balancing\n-export([balance_nodes/3, balance_nodes/4, balance_noop/1]).\n-export([get_last_db_monitor_init/1]).\n-export([requests_balance/0]).\n\n-export_type([dht_message/0, state/0]).\n\n-record(lb_op, {id = ?required(id, lb_op)                           :: uid:global_uid(),\n                type = ?required(type, lb_op)                       :: slide_pred | slide_succ | jump,\n                %% sheds load\n                heavy_node = ?required(heavy, lb_op)                :: node:node_type(),\n                %% receives load\n                light_node = ?required(light, lb_op)                :: node:node_type(),\n                light_node_succ = ?required(light_node_succ, lb_op) :: node:node_type(),\n                target = ?required(target, lb_op)                   :: ?RT:key(),\n                %% time of the oldest data used for the decision for this lb_op\n                data_time = ?required(data_time, lb_op)             :: erlang_timestamp(),\n                time = os:timestamp()                               :: erlang_timestamp()\n               }).\n\n-type lb_op() :: #lb_op{}.\n\n-type options() :: [tuple()].\n\n-type message() :: {lb_stats_trigger} |\n                   {reset_monitors} |\n                   {gossip_reply, LightNode::lb_info:lb_info(), HeavyNode::lb_info:lb_info(), LightNodeSucc::lb_info:lb_info(),\n                    Options::options(), {gossip_get_values_best_response, LoadInfo::gossip_load:load_info()}} |\n                   {balance_phase1, Op::lb_op()} |\n                   {balance_phase2a, Op::lb_op()} |\n                   {balance_phase2b, Op::lb_op()} |\n                   {balance_failed, OpId::uid:global_uid()} |\n                   {balance_success, OpId::uid:global_uid()} |\n                   {move, result, Tag::{jump | slide_pred | slide_succ, OpId::uid:global_uid()}, Result::ok | dht_node_move:abort_reason()} |\n                   {web_debug_info, Requestor::pid()}.\n\n-type dht_message() :: {lb_active, reset_db_monitors} |\n                       {lb_active, balance,\n                        HeavyNode::lb_info:lb_info(), LightNode::lb_info:lb_info(),\n                        LightNodeSucc::lb_info:lb_info(), Options::options()}.\n\n-type module_state() :: tuple().\n\n-record(my_state, {last_balance = os:timestamp() :: erlang_timestamp(),\n                   last_db_monitor_reset = os:timestamp() :: erlang_timestamp(),\n                   pending_op = nil :: lb_op() | nil\n                  }).\n\n-type my_state() :: #my_state{}.\n\n-opaque state() :: {my_state(), module_state()}.\n\n%% list of active load balancing modules available\n-define(MODULES, [lb_active_karger, lb_active_directories]).\n\n%% options for sending messages directly to the lb_active process\n-define(lb, [{group_member, ?MODULE}]).\n\n-include(\"gen_component.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Initialization %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Start this process as a gen component and register it in the dht node group\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun on/2, [],\n                             [{pid_groups_join_as, DHTNodeGroup, lb_active}]).\n\n\n%% @doc Initialization of core module and lb_stats\n-spec init([]) -> state().\ninit([]) ->\n    %% subscribe to ring maintenance for reseting and configuring monitors\n    rm_loop:subscribe(\n       self(), ?MODULE, fun rm_filter/3,\n       fun(_Pid, _Tag, _Old, _New, _Reason) -> reset_monitors() end, inf),\n    reset_monitors(),\n    lb_stats:init(),\n    lb_stats:trigger(),\n    ModuleInitState = call_module(init, []),\n    {_MyState = #my_state{}, _ModuleState = ModuleInitState}.\n\n-spec rm_filter(nodelist:neighborhood(), nodelist:neighborhood(), rm_loop:reason()) -> boolean().\nrm_filter(OldNeighbors, NewNeighbors, _Reason) ->\n  nodelist:node(OldNeighbors) =/= nodelist:node(NewNeighbors) orelse\n        nodelist:pred(OldNeighbors) =/= nodelist:pred(NewNeighbors).\n\n-spec reset_monitors() -> ok.\nreset_monitors() ->\n    DhtNode = pid_groups:get_my(dht_node),\n    LbActive = pid_groups:get_my(lb_active),\n    %% send reset message to dht node and lb_active process\n    comm:send_local(DhtNode, {lb_active, reset_db_monitors}),\n    comm:send_local(LbActive, {reset_monitors}).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Main message handler %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc On handler after initialization\n-spec on(message(), state()) -> state().\non({lb_stats_trigger}, State) ->\n    lb_stats:trigger_routine(),\n    State;\n\n%% Gossip response before balancing takes place\non({gossip_reply, LightNode, HeavyNode, LightNodeSucc, Options,\n    {gossip_get_values_best_response, LoadInfo}}, State) ->\n    %% check the load balancing configuration by using\n    %% the standard deviation from the gossip process.\n    Size = gossip_load:load_info_get(size, LoadInfo),\n    GossipModule = lb_active_gossip_request_metric,\n    ItemMetrics = [{avgItems, gossip_load:load_info_get(avgLoad, LoadInfo)},\n                   {stddevItems, gossip_load:load_info_get(stddev, LoadInfo)}],\n    RequestMetrics = [{avgRequests, gossip_load:load_info_other_get(avgLoad, GossipModule, LoadInfo)},\n                      {stddevRequests, gossip_load:load_info_other_get(stddev, GossipModule, LoadInfo)}],\n    GossipMetrics = [{dht_size, Size} | ItemMetrics ++ RequestMetrics],\n    case requests_balance() andalso lists:keyfind(unknown, 2, RequestMetrics) =/= false of\n        true -> ok;\n        _ -> OptionsNew = GossipMetrics ++ Options,\n             HeavyPid = node:pidX(lb_info:get_node(HeavyNode)),\n             comm:send(HeavyPid, {lb_active, balance, HeavyNode, LightNode, LightNodeSucc, OptionsNew})\n    end,\n    State;\n\n%% lb_op received from dht_node and to be executed\non({balance_phase1, Op}, {MyState, ModuleState} = State) ->\n    OpPending = op_pending(MyState),\n    OldData = old_data(Op, MyState),\n    if\n        OpPending orelse OldData ->\n            ?TRACE(\"Phase1: Pending op: ~p. Old data: ~p. Discarding op ~p.~n\", [OpPending, OldData, Op#lb_op.id]),\n            State;\n        true ->\n            MyState2 = set_pending_op(Op, MyState),\n            if Op#lb_op.type =:= jump ->\n                    %% tell the succ of the light node in case of a jump\n                    LightNodeSuccPid = node:pidX(Op#lb_op.light_node_succ),\n                    comm:send(LightNodeSuccPid, {balance_phase2a, Op}, ?lb);\n                true ->\n                    %% set pending op at other node\n                    LightNodePid = node:pidX(Op#lb_op.light_node),\n                    comm:send(LightNodePid, {balance_phase2b, Op}, ?lb)\n            end,\n            {MyState2, ModuleState}\n    end;\n\n%% Received by the succ of the light node which takes the light nodes' load\n%% in case of a jump.\non({balance_phase2a, Op}, {MyState, ModuleState} = State) ->\n    OpPending = op_pending(MyState),\n    OldData = old_data(Op, MyState),\n    if\n        OpPending orelse OldData ->\n            ?TRACE(\"Phase2a: Pending op: ~p. Old data: ~p. Discarding op ~p.~n\", [OpPending, OldData, Op#lb_op.id]),\n            HeavyNodePid = node:pidX(Op#lb_op.heavy_node),\n            comm:send(HeavyNodePid, {balance_failed, Op#lb_op.id}, ?lb),\n            State;\n        true ->\n            MyState2 = set_pending_op(Op, MyState),\n            LightNodePid = node:pidX(Op#lb_op.light_node),\n            comm:send(LightNodePid, {balance_phase2b, Op}, ?lb),\n            {MyState2, ModuleState}\n    end;\n\n%% The light node which receives load from the heavy node and initiates the lb op.\non({balance_phase2b, Op}, {MyState, ModuleState} = State) ->\n    OpPending = op_pending(MyState),\n    OldData = old_data(Op, MyState),\n    if\n        OpPending orelse OldData ->\n            ?TRACE(\"Phase2b: Pending op: ~p. Old data: ~p. Discarding op ~p.~n\", [OpPending, OldData, Op#lb_op.id]),\n            HeavyNodePid = node:pidX(Op#lb_op.heavy_node),\n            comm:send(HeavyNodePid, {balance_failed, Op#lb_op.id}, ?lb),\n            if Op#lb_op.type =:= jump ->\n                   LightNodeSuccPid = node:pidX(Op#lb_op.light_node_succ),\n                   comm:send(LightNodeSuccPid, {balance_failed, Op#lb_op.id}, ?lb);\n               true -> ok\n            end,\n            State;\n        true ->\n            OpId = Op#lb_op.id,\n            TargetKey = Op#lb_op.target,\n            MyState2 = set_pending_op(Op, MyState),\n            ?TRACE(\"OpId: ~p Type: ~p Heavy: ~p Light: ~p LightNodeSucc: ~p Target: ~p~n\",\n                   [Op#lb_op.id, Op#lb_op.type, Op#lb_op.heavy_node, Op#lb_op.light_node, Op#lb_op.light_node_succ, TargetKey]),\n            MyDHT = pid_groups:get_my(dht_node),\n            _Pid = node:pidX(Op#lb_op.light_node),\n            ?DBG_ASSERT(_Pid =:= comm:make_global(MyDHT)),\n            case Op#lb_op.type of\n                jump ->\n                    comm:send_local(MyDHT, {move, start_jump, TargetKey, {jump, OpId}, comm:this()});\n                slide_pred ->\n                    comm:send_local(MyDHT, {move, start_slide, pred, TargetKey, {slide_pred, OpId}, comm:this()});\n                slide_succ ->\n                    comm:send_local(MyDHT, {move, start_slide, succ, TargetKey, {slide_succ, OpId}, comm:this()})\n            end,\n            {MyState2, ModuleState}\n    end;\n\non({balance_failed, OpId}, {MyState, ModuleState} = State) ->\n    case get_pending_op(MyState) of\n        nil ->\n            ?TRACE(\"Received balance_failed but OpId ~p was not pending~n\", [OpId]),\n            State;\n        Op when Op#lb_op.id =:= OpId ->\n            ?TRACE(\"Clearing pending op because of balance_failed ~p~n\", [OpId]),\n            MyState2 = set_pending_op(nil, MyState),\n            {MyState2, ModuleState};\n        _Op ->\n            ?TRACE(\"Received balance_failed answer but OpId ~p didn't match pending id ~p~n\", [OpId, _Op#lb_op.id]),\n            State\n    end;\n\n%% success does not imply the slide or jump was successfull. however,\n%% slide or jump failures should very rarly occur because of the locking\n%% and stale data detection.\non({balance_success, OpId}, {MyState, ModuleState} = State) ->\n    case get_pending_op(MyState) of\n        nil ->\n            ?TRACE(\"Received answer but OpId ~p was not pending~n\", [OpId]),\n            State;\n        Op when Op#lb_op.id =:= OpId ->\n            ?TRACE(\"Clearing pending op ~p~n\", [OpId]),\n            MyState2 = set_pending_op(nil, MyState),\n            MyState3 = set_time_last_balance(MyState2),\n            {MyState3, ModuleState};\n        _Op ->\n            ?TRACE(\"Received answer but OpId ~p didn't match pending id ~p~n\", [OpId, _Op#lb_op.id]),\n            State\n    end;\n\n%% received reply at the sliding/jumping node\non({move, result, {_JumpOrSlide, OpId}, _Status}, {MyState, ModuleState} = State) ->\n    ?TRACE(\"~p status with id ~p: ~p~n\", [_JumpOrSlide, OpId, _Status]),\n    ?DEBUG(lb_logger:report_success(_JumpOrSlide, _Status)),\n    case get_pending_op(MyState) of\n        nil ->\n            ?TRACE(\"Received answer but OpId ~p was not pending~n\", [OpId]),\n            State;\n        Op when Op#lb_op.id =:= OpId ->\n            ?TRACE(\"Clearing pending op and replying to other node ~p~n\", [OpId]),\n            HeavyNodePid = node:pidX(Op#lb_op.heavy_node),\n            comm:send(HeavyNodePid, {balance_success, OpId}, ?lb),\n            case Op#lb_op.type of\n                jump ->\n                    %% also reply to light node succ in case of jump\n                    LightNodeSucc = Op#lb_op.light_node_succ,\n                    LightNodeSuccPid = node:pidX(LightNodeSucc),\n                    comm:send(LightNodeSuccPid, {balance_success, OpId}, ?lb);\n                _ ->\n                    ok\n            end,\n            MyState2 = set_pending_op(nil, MyState),\n            MyState3 = set_time_last_balance(MyState2),\n            {MyState3, ModuleState};\n        _Op ->\n            ?TRACE(\"Received answer but OpId ~p didn't match pending id ~p~n\", [OpId, _Op#lb_op.id]),\n            State\n    end;\n\non({reset_monitors}, {MyState, ModuleState}) ->\n    lb_stats:init(),\n    ?TRACE(\"Reseting monitors ~n\", []),\n    MyState2 = set_last_db_monitor_init(MyState),\n    {MyState2, ModuleState};\n\non({web_debug_info, Requestor}, {MyState, ModuleState} = State) ->\n    KVList =\n        [{\"active module\", webhelpers:safe_html_string(\"~p\", [get_lb_module()])},\n         {\"load metric\", webhelpers:safe_html_string(\"~p\", [config:read(lb_active_load_metric)])},\n         {\"load metric value:\", webhelpers:safe_html_string(\"~p\", [lb_stats:get_load_metric()])},\n         {\"request metric\", webhelpers:safe_html_string(\"~p\", [config:read(lb_active_request_metric)])},\n         {\"request metric value\", webhelpers:safe_html_string(\"~p\", [lb_stats:get_request_metric()])},\n         {\"balance with\", webhelpers:safe_html_string(\"~p\", [config:read(lb_active_balance)])},\n         {\"last balance:\", webhelpers:safe_html_string(\"~p\", [get_time_last_balance(MyState)])},\n         {\"pending op:\",   webhelpers:safe_html_string(\"~p\", [get_pending_op(MyState)])},\n         {\"last db monitor init:\", webhelpers:safe_html_string(\"~p\", [get_last_db_monitor_init(MyState)])},\n         {\"module\", webhelpers:safe_html_string(\"~p\", [get_lb_module()])}\n        ],\n    case call_module(get_web_debug_kv, [ModuleState]) of\n        [H|T] -> Return = KVList ++ [H|T];\n        _ -> Return = KVList\n    end,\n    comm:send_local(Requestor, {web_debug_info_reply, Return}),\n    State;\n\non(Msg, {MyState, ModuleState}) ->\n    ModuleState2 = call_module(handle_msg, [Msg, ModuleState]),\n    {MyState, ModuleState2}.\n\n%%%%%%%%%%%%%%%%%%%%%%% Load Balancing %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec balance_nodes(lb_info:lb_info(), lb_info:lb_info(), options()) -> ok.\nbalance_nodes(HeavyNode, LightNode, Options) ->\n    balance_nodes(HeavyNode, LightNode, nil, Options).\n\n-spec balance_nodes(lb_info:lb_info(), lb_info:lb_info(), lb_info:lb_info() | nil, options()) -> ok.\nbalance_nodes(HeavyNode, LightNode, LightNodeSucc, Options) ->\n    case config:read(lb_active_use_gossip) of\n        true -> %% Retrieve global info from gossip before balancing\n            LBActivePid = pid_groups:get_my(lb_active),\n            Envelope = {gossip_reply, LightNode, HeavyNode, LightNodeSucc, Options, '_'},\n            ReplyPid = comm:reply_as(LBActivePid, 6, Envelope),\n            gossip_load:get_values_best([{source_pid, ReplyPid}]);\n        _ ->\n            HeavyPid = node:pidX(lb_info:get_node(HeavyNode)),\n            comm:send(HeavyPid, {lb_active, balance, HeavyNode, LightNode, LightNodeSucc, Options})\n    end.\n\n-spec balance_noop(options()) -> ok.\n%% no op but we send back simulation results\nbalance_noop(Options) ->\n    case proplists:get_value(simulate, Options) of\n        undefined -> ok;\n        ReqId ->\n            ReplyTo = proplists:get_value(reply_to, Options),\n            Id = proplists:get_value(id, Options),\n            comm:send(ReplyTo, {simulation_result, Id, ReqId, {nil, 0}})\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%% Calls from dht_node %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Process load balancing messages sent to the dht node\n-spec handle_dht_msg(dht_message(), dht_node_state:state()) -> dht_node_state:state().\n\nhandle_dht_msg({lb_active, reset_db_monitors}, DhtState) ->\n    case requests_balance() of\n        true ->\n            MyPredId = dht_node_state:get(DhtState, pred_id),\n            DhtNodeMonitor = pid_groups:get_my(dht_node_monitor),\n            comm:send_local(DhtNodeMonitor, {db_histogram_init, MyPredId});\n        _ -> ok\n    end,\n    DhtState;\n\n%% We received a jump or slide operation from a LightNode.\n%% In either case, we'll compute the target id and send out\n%% the jump or slide message to the LightNode.\nhandle_dht_msg({lb_active, balance, HeavyNode, LightNode, LightNodeSucc, Options}, DhtState) ->\n    %% check if we are the correct node\n    IncorrectNode = lb_info:get_node(HeavyNode) =/= dht_node_state:get(DhtState, node),\n    Sliding = slide_op:is_slide(dht_node_state:get(DhtState, slide_pred)) orelse\n                  slide_op:is_slide(dht_node_state:get(DhtState, slide_succ)),\n    if IncorrectNode ->\n           ?TRACE(\"I was mistaken for the HeavyNode. Doing nothing~n\", []),\n           balance_noop(Options);\n       Sliding ->\n           ?TRACE(\"Currently performing a slide operation.~n\", []),\n           balance_noop(Options);\n       true ->\n           %% get our load info again to have the newest data available\n           MyNode = lb_info:new(dht_node_state:details(DhtState)),\n           JumpOrSlide = %case lb_info:neighbors(MyNode, LightNode) of\n               case LightNodeSucc =:= nil of\n                   true  -> slide;\n                   false -> jump\n               end,\n\n           ?TRACE(\"Load Heavy: ~p Load Light: ~p~n\", [lb_info:get_load(MyNode), lb_info:get_load(LightNode)]),\n           ?TRACE(\"Requests Heavy: ~p Requests Light: ~p~n\", [lb_info:get_reqs(MyNode), lb_info:get_reqs(LightNode)]),\n           ProposedTargetLoadItems = lb_info:get_target_load(items, JumpOrSlide, MyNode, LightNode),\n           ProposedTargetLoadRequests = lb_info:get_target_load(requests, JumpOrSlide, MyNode, LightNode),\n\n           {TargetLoadItems, TargetLoadRequests} =\n               case gossip_available(Options) of\n                   true -> AvgItems = proplists:get_value(avgItems, Options),\n                           AvgRequests = proplists:get_value(avgRequests, Options),\n                           %% don't take away more than the average to avoid making the light node heavy\n                           {?IIF(ProposedTargetLoadItems > AvgItems,\n                                 trunc(AvgItems), ProposedTargetLoadItems),\n                            ?IIF(ProposedTargetLoadRequests > AvgRequests,\n                                 trunc(AvgRequests), ProposedTargetLoadRequests)\n                           };\n                   false -> AvgItems = 0, AvgRequests = 0,\n                            {ProposedTargetLoadItems, ProposedTargetLoadRequests}\n               end,\n\n           {From, To, Direction} =\n               case JumpOrSlide =:= jump orelse lb_info:is_succ(MyNode, LightNode) of\n                   true  -> %% Jump or heavy node is succ of light node\n                       {dht_node_state:get(DhtState, pred_id), dht_node_state:get(DhtState, node_id), forward};\n                   false -> %% Light node is succ of heavy node\n                       {dht_node_state:get(DhtState, node_id), dht_node_state:get(DhtState, pred_id), backward}\n               end,\n\n           {Metric, {SplitKey, TakenLoad}} =\n               case requests_balance() of\n                   false ->\n                       {items, dht_node_state:get_split_key(DhtState, From, To, TargetLoadItems, Direction)};\n                   true ->\n                       case lb_stats:get_request_histogram_split_key(TargetLoadRequests, Direction,\n                                                                     lb_info:get_items(HeavyNode)) of\n                           failed ->\n                               case config:read(lb_active_fall_back_to_items) of\n                                   true ->\n                                        log:log(warn, \"get_request_histogram failed. falling back to item balancing.~n\", []),\n                                        {items, dht_node_state:get_split_key(DhtState, From, To, TargetLoadItems, Direction)};\n                                   _ -> {requests, {nil, 0}}\n                               end;\n                           Val -> {requests, Val}\n                       end\n               end,\n\n           ?TRACE(\"SplitKey: ~p TargetLoadItems: ~p TargetLoadRequests: ~p TakenLoad: ~p Metric: ~p~n\",\n                  [SplitKey, TargetLoadItems, TargetLoadRequests, TakenLoad, Metric]),\n\n           IsSimulation = is_simulation(Options),\n           InMyRange = intervals:in(SplitKey, dht_node_state:get(DhtState, my_range)),\n           NotMyId = SplitKey =/= dht_node_state:get(DhtState, node_id),\n\n           if IsSimulation -> %% compute result of simulation and reply\n                   ReqId = proplists:get_value(simulate, Options),\n                   LoadChange =\n                       case JumpOrSlide of\n                           slide -> lb_info:get_load_change_slide(Metric, TakenLoad, HeavyNode, LightNode);\n                           jump  -> lb_info:get_load_change_jump(Metric, TakenLoad, HeavyNode, LightNode, LightNodeSucc)\n                       end,\n                   ReplyTo = proplists:get_value(reply_to, Options),\n                   Id = proplists:get_value(id, Options),\n                   comm:send(ReplyTo, {simulation_result, Id, ReqId, {Metric, LoadChange}});\n\n               InMyRange andalso NotMyId -> %% perform balancing\n                   StdDevTest =\n                       case gossip_available(Options) of\n                           true ->\n                               S = config:read(lb_active_gossip_stddev_threshold),\n                               DhtSize = proplists:get_value(dht_size, Options),\n                               case Metric of\n                                   items ->\n                                       Avg = AvgItems,\n                                       StdDev = proplists:get_value(stddevItems, Options);\n                                   requests ->\n                                       Avg = AvgRequests,\n                                       StdDev = proplists:get_value(stddevRequests, Options)\n                               end,\n                               Variance = StdDev * StdDev,\n                               VarianceChange =\n                                   case JumpOrSlide of\n                                       slide -> lb_info:get_load_change_slide(Metric, TakenLoad, DhtSize, HeavyNode, LightNode);\n                                       jump -> lb_info:get_load_change_jump(Metric, TakenLoad, DhtSize, HeavyNode, LightNode, LightNodeSucc)\n                                   end,\n                               VarianceNew = Variance + VarianceChange,\n                               StdDevNew = ?IIF(VarianceNew >= 0, math:sqrt(VarianceNew), StdDev),\n                               ?TRACE(\"New StdDev: ~p Old StdDev: ~p DhtSize: ~p Metric: ~p~n\", [StdDevNew, StdDev, DhtSize, Metric]),\n                               %% Check for decrease in stddev but also take into account systems with a large stddev where moving the\n                               %% the average load will reduce the stddev only gradually.\n                               StdDevNew < StdDev * (1 - S / DhtSize) orelse TakenLoad >= 0.9 * Avg;\n                           %% gossip not available, skipping this test\n                           false -> true\n                       end,\n\n                   case StdDevTest andalso TakenLoad > 0 of\n                       false -> ?TRACE(\"No balancing: stddev was not reduced enough.~n\", []);\n                       true ->\n                           ?TRACE(\"Sending out lb op.~n\", []),\n                           OpId = uid:get_global_uid(),\n                           Type =  if  JumpOrSlide =:= jump -> jump;\n                                       Direction =:= forward -> slide_succ;\n                                       Direction =:= backward -> slide_pred\n                                   end,\n                           OldestDataTime = if Type =:= jump ->\n                                                   lb_info:get_oldest_data_time([LightNode, HeavyNode, LightNodeSucc]);\n                                               true ->\n                                                   lb_info:get_oldest_data_time([LightNode, HeavyNode])\n                                            end,\n                           Op = #lb_op{id = OpId, type = Type,\n                                       light_node = lb_info:get_node(LightNode),\n                                       light_node_succ = lb_info:get_succ(LightNode),\n                                       heavy_node = lb_info:get_node(HeavyNode),\n                                       target = SplitKey,\n                                       data_time = OldestDataTime},\n                           ?DEBUG(lb_logger:report_op(Type, Metric)),\n\n                           LBModule = pid_groups:get_my(?MODULE),\n                           comm:send_local(LBModule, {balance_phase1, Op})\n                   end;\n               true -> ?TRACE(\"Invalid target chosen: ~p InMyRange: ~p NotMyId: ~p~n\", [SplitKey, InMyRange, NotMyId])\n           end\n\n    end,\n    DhtState;\n\nhandle_dht_msg(Msg, DhtState) when element(1,Msg) =:= lb_active ->\n    call_module(handle_dht_msg, [Msg, DhtState]).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%% Util %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-compile({inline, [is_enabled/0]}).\n-spec is_enabled() -> boolean().\nis_enabled() ->\n    config:read(lb_active).\n\n-compile({inline, [requests_balance/0]}).\n-spec requests_balance() -> boolean().\nrequests_balance() ->\n    config:read(lb_active_balance) =:= requests.\n\n-spec call_module(atom(), list()) -> term().\ncall_module(Fun, Args) ->\n    case get_lb_module() of\n        none ->\n            {};\n        Module ->\n            apply(Module, Fun, Args)\n    end.\n\n-spec get_lb_module() -> atom() | failed.\nget_lb_module() ->\n    config:read(lb_active_module).\n\n-spec get_pending_op(my_state()) -> nil | lb_op().\nget_pending_op(MyState) ->\n    MyState#my_state.pending_op.\n\n-spec set_pending_op(nil | lb_op(), my_state()) -> my_state().\nset_pending_op(Op, MyState) ->\n    MyState#my_state{pending_op = Op}.\n\n-spec op_pending(my_state()) -> boolean().\nop_pending(MyState) ->\n    case MyState#my_state.pending_op of\n        nil -> false;\n        Op ->\n            case old_op(Op) of\n                false -> true;\n                true ->\n                    ?TRACE(\"Ignoring old op ~p~n\", [Op]),\n                    false\n            end\n    end.\n\n%% @doc Checks if an lb_op has been pending for a long time\n-spec old_op(lb_op()) -> boolean().\nold_op(Op) ->\n    Threshold = config:read(lb_active_wait_for_pending_ops),\n    timer:now_diff(os:timestamp(), Op#lb_op.time) div 1000 > Threshold.\n\n%% @doc Checks if an lb_op contains old data\n-spec old_data(lb_op(), my_state()) -> boolean().\nold_data(Op, MyState) ->\n    LastBalanceTime = get_time_last_balance(MyState),\n    DataTime = Op#lb_op.data_time,\n    timer:now_diff(LastBalanceTime, DataTime) > 0.\n\n-spec set_last_db_monitor_init(my_state()) -> my_state().\nset_last_db_monitor_init(MyState) ->\n    MyState#my_state{last_db_monitor_reset = os:timestamp()}.\n\n-spec get_last_db_monitor_init(my_state()) -> erlang_timestamp().\nget_last_db_monitor_init(MyState) ->\n    MyState#my_state.last_db_monitor_reset.\n\n-spec get_time_last_balance(my_state()) -> erlang_timestamp().\nget_time_last_balance(MyState) ->\n    MyState#my_state.last_balance.\n\n-spec set_time_last_balance(my_state()) -> my_state().\nset_time_last_balance(MyState) ->\n    MyState#my_state{last_balance = os:timestamp()}.\n\n-spec gossip_available(options()) -> boolean().\ngossip_available(Options) ->\n    proplists:is_defined(dht_size, Options) andalso\n        proplists:is_defined(avgItems, Options) andalso\n        proplists:is_defined(stddevItems, Options) andalso\n        proplists:is_defined(avgRequests, Options) andalso\n        proplists:is_defined(stddevRequests, Options).\n\n-spec is_simulation(options()) -> boolean().\nis_simulation(Options) ->\n    proplists:is_defined(simulate, Options).\n\n-spec check_gossip_modules(atom(), atom()) -> boolean().\ncheck_gossip_modules(RequiredModule, DependencyKey) ->\n    Fun = fun(Value) -> lists:member(RequiredModule, Value) end,\n    Dependency = config:read(DependencyKey),\n    Msg = io_lib:format(\"~p required when ~p =:= ~p.\",\n                        [RequiredModule, DependencyKey, Dependency]),\n    config:cfg_test_and_error(gossip_load_additional_modules, Fun, Msg).\n\n-spec check_module_config() -> boolean().\ncheck_module_config() ->\n    case get_lb_module() of\n        none -> true;\n        Module -> apply(Module, check_config, [])\n    end.\n\n%% @doc config check registered in config.erl\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_bool(lb_active) and\n\n    config:cfg_is_in(lb_active_module, [none | ?MODULES]) and\n\n    config:cfg_is_in(lb_active_balance, [items, requests]) and\n\n    config:cfg_is_bool(lb_active_fall_back_to_items) and\n\n    config:cfg_is_bool(lb_active_use_gossip) and\n    config:cfg_is_greater_than(lb_active_gossip_stddev_threshold, 0) and\n\n    config:cfg_is_integer(lb_active_wait_for_pending_ops) and\n    config:cfg_is_greater_than(lb_active_wait_for_pending_ops, 0) and\n\n    (config:read(lb_active_use_gossip) =:= false orelse\n         check_gossip_modules(lb_active_gossip_request_metric, lb_active_use_gossip)) and\n\n    lb_stats:check_config() and\n\n    check_module_config().\n"
  },
  {
    "path": "src/lb_active_beh.erl",
    "content": "%  @copyright 2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maximilian Michels <michels@zib.de>\n%% @doc    Active load balancing algorithm behavior\n%% @end\n%% @version $Id$\n-module(lb_active_beh).\n-author('michels@zib.de').\n-vsn('$Id$').\n\n% for behaviour\n-ifndef(have_callback_support).\n-export([behaviour_info/1]).\n-endif.\n\n-ifdef(have_callback_support).\n-include(\"scalaris.hrl\").\n\n%% callbacks\n\n-type state() :: term().\n\n-callback init() -> state().\n\n-callback handle_msg(comm:message(), state()) -> state().\n\n-callback handle_dht_msg(comm:message(), dht_node_state:state())\n        -> dht_node_state:state().\n\n-callback get_web_debug_kv(state()) -> [{string(), string()}].\n\n-callback check_config() -> boolean().\n\n-else.\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n    [\n     {init, 0},\n     % handle msg from lb_active module\n     {handle_msg, 2},\n     % process lb message at dht node\n     {handle_dht_msg, 2},\n     % returns key / value list to be shown in web debug interface\n     {get_web_debug_kv, 1},\n     % config check performed by lb_active.erl \n     {check_config, 0}\n    ];\nbehaviour_info(_Other) ->\n    undefined.\n-endif."
  },
  {
    "path": "src/lb_active_directories.erl",
    "content": "%  @copyright 2014-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maximilian Michels <michels@zib.de>\n%% @doc Implementation of a modified version of the paper below. This implementation \n%%      doesn't use virtual servers but can still benefit from the load balancing\n%%      algorithm's attributes, respectively the load directories and the emergency\n%%      transfer of load.\n%%\n%%      Many-to-Many scheme\n%% @end\n%% @reference B. Godfrey, S. Surana, K. Lakshminarayanan, R. Karp, and I. Stoica\n%%            \"Load balancing in dynamic structured peer-to-peer systems\"\n%%            Performance Evaluation, vol. 63, no. 3, pp. 217-240, 2006.\n%%\n%% @version $Id$\n-module(lb_active_directories).\n-author('michels@zib.de').\n-vsn('$Id$').\n\n-behaviour(lb_active_beh).\n%% implements\n-export([init/0, check_config/0]).\n-export([handle_msg/2, handle_dht_msg/2]).\n-export([get_web_debug_kv/1]).\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n%% Defines the number of directories\n%% e.g. 1 implies a central directory\n-define(NUM_DIRECTORIES, 2).\n\n-define(TRACE(X,Y), ok).\n%-define(TRACE(X,Y), io:format(X,Y)).\n\n-type directory_name() :: string().\n\n-record(directory, {name         = ?required(directory, name) :: directory_name(),\n                    pool         = gb_sets:new()              :: gb_sets:set(lb_info:lb_info()),\n                    num_reported = 0                          :: non_neg_integer()\n                    }).\n\n-record(reassign, {light = ?required(reassign, from) :: lb_info:lb_info(),\n                   heavy = ?required(reassign, to)   :: lb_info:lb_info()\n                   }).\n\n-type reassign() :: #reassign{}.\n-type schedule() :: [reassign()].\n\n-record(state, {my_dirs = [] :: [directory_name()],\n                %threshold_periodic  = 0.5, %% k_p = (1 + average directory utilization) / 2\n                %threshold_emergency = 1.0,  %% k_e\n                schedule = [] :: schedule()\n                }).\n\n-type directory() :: #directory{}.\n\n-type state() :: #state{}.\n\n-type trigger() :: publish_trigger | directory_trigger.\n\n-type message() :: {publish_trigger} |\n                   {post_load, lb_info:lb_info()} |\n                   {directory_trigger} |\n                   {get_state_response, intervals:interval()}.\n\n-type dht_message() :: {lb_active, request_load, pid()} |\n                       {lb_active, before_jump,\n                        HeavyNode::lb_info:lb_info(), LightNode::lb_info:lb_info()}.\n\n\n%%%%%%%%%%%%%%%% Initialization %%%%%%%%%%%%%%%%%%%%%%%\n\n-spec init() -> state().\ninit() ->\n    %% post load to random directory\n    request_dht_load(),\n    trigger(publish_trigger),\n    trigger(directory_trigger),\n    request_dht_range(),\n    This = comm:this(),\n    rm_loop:subscribe(\n       self(), ?MODULE, fun rm_loop:subscribe_dneighbor_change_slide_filter/3,\n       fun(_,_,_,_,_) -> comm:send_local(self(), {get_state, This, my_range}) end, inf),\n    #state{}.\n\n%%%%%%%%%%%%%%%% Process Messages %%%%%%%%%%%%%%%%%%%%%%%\n\n-spec handle_msg(message(), state()) -> state().\nhandle_msg({publish_trigger}, State) ->\n    trigger(publish_trigger),\n    case emergency of %% emergency when load(node) > k_e\n      % true ->\n         %post_load(),\n         %get && perform_transfer()\n      %   State;\n      _ ->\n          % get && perform transfer() without overloading\n          Schedule = State#state.schedule,\n          %% transfer (balance) up to MaxTransfer nodes\n          MaxTransfer = config:read(lb_active_directories_max_transfer),\n          perform_transfer(Schedule, 0, MaxTransfer),\n          % post_load()\n          % request dht load to post it in the directory afterwards\n          request_dht_load(),\n          State#state{schedule = []}\n    end;\n\n%% we received load because of publish load trigger or emergency\nhandle_msg({post_load, LoadInfo}, State) ->\n    ?TRACE(\"Posting load ~p~n\", [LoadInfo]),\n    Directory = get_random_directory(),\n    DirKey = Directory#directory.name,\n    post_load_to_directory(LoadInfo, DirKey, 0),\n    %% TODO Emergency Threshold has been already checked at the node overloaded...\n%%     EmergencyThreshold = State#state.threshold_emergency,\n%%     case lb_info:get_load(LoadInfo) > EmergencyThreshold of\n%%         true  ->\n%%             ?TRACE(\"Emergency in post_load~n\", []),\n%%             MySchedule = State#state.schedule,\n%%             Schedule = directory_routine(DirKey, emergency, MySchedule),\n%%             perform_transfer(Schedule);\n%%         false -> ok\n%%     end,\n    State;\n\nhandle_msg({directory_trigger}, State) ->\n    trigger(directory_trigger),\n    ?TRACE(\"~p My Directories: ~p~n\", [self(), State#state.my_dirs]),\n    %% Threshold k_p = Average laod in directory\n    %% Threshold k_e = 1 meaning full capacity\n    %% Upon receipt of load information:\n    %%      add_to_directory(load_information),\n    %%      case node_overloaded of\n    %%          true -> compute_reassign(directory_load, node, k_e);\n    %%          _    -> compute_reassign(directory_load, node, k_p),\n    %%                  clear_directory()\n    %%      end\n    %% compute_reassign:\n    %%  for every node from heavist to lightest in directory \n    %%    if l_n / c_n > k \n    %%       balance such that (l_n + l_x) / c_n gets minimized\n    %%  return assignment\n    MyDirKeys = State#state.my_dirs,\n    NewSchedule = manage_directories(MyDirKeys),\n    State#state{schedule = NewSchedule};\n\nhandle_msg({get_state_response, MyRange}, State) ->\n    Directories = get_all_directory_keys(),\n    MyDirectories = [int_to_str(Dir) || Dir <- Directories, intervals:in(Dir, MyRange)],\n    ?TRACE(\"~p: I am responsible for ~p~n\", [self(), MyDirectories]),\n    State#state{my_dirs = MyDirectories, schedule = []};\n\nhandle_msg(_Msg, State) ->\n    ?TRACE(\"Unknown message: ~p~n\", [_Msg]),\n    State.\n\n%%%%%%%%%%%%%%%%%% DHT Node interaction %%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Load balancing messages received by the dht node.\n-spec handle_dht_msg(dht_message(), dht_node_state:state()) -> dht_node_state:state().\nhandle_dht_msg({lb_active, request_load, ReplyPid}, DhtState) ->\n    NodeDetails = dht_node_state:details(DhtState),\n    LoadInfo = lb_info:new(NodeDetails),\n    case lb_info:is_valid(LoadInfo) of\n        true -> comm:send_local(ReplyPid, {post_load, LoadInfo});\n        _ -> ok\n    end,\n    DhtState;\n\n%% This handler is for requesting load information from the succ of\n%% the light node in case of a jump (we are the succ of the light node).\nhandle_dht_msg({lb_active, before_jump, HeavyNode, LightNode}, DhtState) ->\n    NodeDetails = dht_node_state:details(DhtState),\n    LightNodeSucc = lb_info:new(NodeDetails),\n    case lb_info:is_valid(LightNodeSucc) of\n        true -> lb_active:balance_nodes(HeavyNode, LightNode, LightNodeSucc, []);\n        _ -> ok\n    end,\n    DhtState;\n\nhandle_dht_msg(_Msg, DhtState) ->\n    ?TRACE(\"Unknown message: ~p~n\", [_Msg]),\n    DhtState.\n\n%%%%%%%%%%%%%%%%% Directory Management %%%%%%%%%%%%%%%\n\nmanage_directories(DirKeys) ->\n    manage_directories(DirKeys, []).\n\nmanage_directories([], Schedule) ->\n    Schedule;\nmanage_directories([DirKey | Other], Schedule) ->\n    DirSchedule = directory_routine(DirKey, periodic, Schedule),\n    manage_directories(Other, Schedule ++ DirSchedule).\n\n-spec directory_routine(directory_name(), periodic | emergency, schedule()) -> schedule().\ndirectory_routine(DirKey, _Type, Schedule) ->\n    %% Because of the lack of virtual servers/nodes, the load\n    %% balancing is differs from the paper here. We try to\n    %% balance the most loaded node with the least loaded\n    %% node.\n    %% TODO Some preference should be given to neighboring\n    %%      nodes to avoid too many jumps.\n    {_TLog, Directory} = get_directory(DirKey),\n    clear_directory(Directory, 0),\n    case dir_is_empty(Directory) of\n        true -> Schedule;\n        false ->\n            Pool = Directory#directory.pool,\n            ScheduleNew = find_matches(Pool),\n            ?TRACE(\"New schedule: ~p~n\", [ScheduleNew]),\n            ScheduleNew\n    end.\n\n-spec find_matches(gb_sets:set(lb_info:lb_info())) -> schedule().\nfind_matches(Nodes) ->\n    case gb_sets:size(Nodes) >= 2 of\n        true ->\n            {LightNode, NodesNew} = gb_sets:take_smallest(Nodes),\n            {HeavyNode, NodesNew2} = gb_sets:take_largest(NodesNew),\n            Epsilon = 0.24,\n            case lb_info:get_load(LightNode) =< Epsilon * lb_info:get_load(HeavyNode) of\n                true ->\n                    [#reassign{light = LightNode, heavy = HeavyNode} | find_matches(NodesNew2)];\n                _ ->\n                    find_matches(NodesNew2)\n            end;\n        false ->\n            %% return the result with the best match first\n            []\n    end.\n\n-spec get_all_directory_keys() -> [?RT:key()].\nget_all_directory_keys() ->\n    [get_directory_key_by_number(N) || N <- lists:seq(1, ?NUM_DIRECTORIES)].\n\n-spec get_random_directory_key() ->  ?RT:key().\nget_random_directory_key() ->\n    Rand = randoms:rand_uniform(1, ?NUM_DIRECTORIES+1),\n    get_directory_key_by_number(Rand).\n\n-spec get_directory_key_by_number(pos_integer()) -> ?RT:key().\nget_directory_key_by_number(N) when N > 0 ->\n    ?RT:hash_key(\"lb_active_dir\" ++ int_to_str(N)).\n\n%% selects two directories at random and returns the one which least nodes reported to\nget_random_directory() ->\n    {_TLog1, RandDir1} = get_directory(int_to_str(get_random_directory_key())),\n    {_TLog2, RandDir2} = get_directory(int_to_str(get_random_directory_key())),\n    case RandDir1#directory.num_reported >= RandDir2#directory.num_reported of\n        true -> RandDir1;\n        _    -> RandDir2\n    end.\n\n-spec post_load_to_directory(lb_info:lb_info(), directory_name(), non_neg_integer()) -> ok.\npost_load_to_directory(Load, DirKey, Retries) ->\n    {TLog, Dir} = get_directory(DirKey),\n    DirNew = dir_add_load(Load, Dir),\n    case set_directory(TLog, DirNew) of\n        ok -> ok;\n        failed ->\n            if Retries < 5 ->\n                    wait_randomly(),\n                    post_load_to_directory(Load, DirKey, Retries + 1);\n               true -> ok\n            end\n    end.\n\n-spec clear_directory(directory(), non_neg_integer()) -> ok.\nclear_directory(Directory, Retries) ->\n    DirNew = dir_clear_load(Directory),\n    case set_directory(api_tx:new_tlog(), DirNew) of\n        ok -> ok;\n        failed ->\n            if Retries < 5 ->\n                    wait_randomly(),\n                    clear_directory(Directory, Retries + 1);\n               true -> ok\n            end\n    end.\n\n-spec get_directory(directory_name()) -> {tx_tlog:tlog(), directory()}.\nget_directory(DirKey) ->\n    TLog = api_tx:new_tlog(),\n    case api_tx:read(TLog, DirKey) of\n        {TLog2, {ok, Directory}} ->\n            %?TRACE(\"~p: Got directory: ~p~n\", [?MODULE, Directory]),\n            {TLog2, Directory};\n        {TLog2, {fail, not_found}} ->\n            log:log(warn, \"~p: Directory not found: ~p\", [?MODULE, DirKey]),\n            {TLog2, #directory{name = DirKey}}\n    end.\n\n-spec set_directory(tx_tlog:tlog(), directory()) -> ok | failed.\nset_directory(TLog, Directory) ->\n    DirKey = Directory#directory.name,\n    case api_tx:req_list(TLog, [{write, DirKey, Directory}, {commit}]) of\n        {[], [{ok}, {ok}]} -> \n            ok;\n        Error ->\n            log:log(warn, \"~p: Failed to save directory ~p because of failed transaction: ~p\", [?MODULE, DirKey, Error]),\n            failed\n    end.\n\n%%%%%%%%%%%%%%%% Directory record %%%%%%%%%%%%%%%%%%%%%%\n\n-spec dir_add_load(lb_info:lb_info(), directory()) -> directory().\ndir_add_load(Load, Directory) ->\n    Pool = Directory#directory.pool,\n    PoolNew = gb_sets:add(Load, Pool),\n    NumReported = Directory#directory.num_reported,\n    Directory#directory{pool = PoolNew, num_reported = NumReported + 1}.\n\n-spec dir_clear_load(directory()) -> directory().\ndir_clear_load(Directory) ->\n    Directory#directory{pool = gb_sets:new(), num_reported = 0}.\n\n%% dir_set_schedule(Schedule, Directory) ->\n%%     Directory#directory{schedule = Schedule}.\n\n-spec dir_is_empty(directory()) -> boolean().\ndir_is_empty(Directory) ->\n    Pool = Directory#directory.pool,\n    gb_sets:is_empty(Pool).\n\n%%%%%%%%%%%%%% Reassignments %%%%%%%%%%%%%%%%%%%%%\n\n-spec perform_transfer(schedule(), non_neg_integer(), pos_integer()) -> ok.\nperform_transfer([], _, _) ->\n    ok;\nperform_transfer(_, MaxTransfer, MaxTransfer) ->\n    ok;\nperform_transfer([#reassign{light = LightNode, heavy = HeavyNode} | Other], Transferred, MaxTransfer) ->\n    ?TRACE(\"~p: Reassigning ~p (light: ~p) and ~p (heavy: ~p)~n\", [?MODULE, lb_info:get_node(LightNode), lb_info:get_load(LightNode), lb_info:get_node(HeavyNode), lb_info:get_load(HeavyNode)]),\n    case lb_info:neighbors(HeavyNode, LightNode) of\n        true ->\n            lb_active:balance_nodes(HeavyNode, LightNode, []);\n        false -> %% send message to succ of LightNode to get his load\n            LightNodeSucc = lb_info:get_succ(LightNode),\n            comm:send(node:pidX(LightNodeSucc), {lb_active, before_jump, HeavyNode, LightNode})\n    end,\n    perform_transfer(Other, Transferred + 1, MaxTransfer).\n\n%%%%%%%%%%%%\n%% Helpers\n%%\n\n-spec request_dht_range() -> ok.\nrequest_dht_range() ->\n    MyDHT = pid_groups:get_my(dht_node),\n    comm:send_local(MyDHT, {get_state, comm:this(), my_range}).\n\n-spec request_dht_load() -> ok.\nrequest_dht_load() ->\n    MyDHT = pid_groups:find_a(dht_node),\n    comm:send_local(MyDHT, {lb_active, request_load, self()}).\n\n-spec int_to_str(integer()) -> string().\nint_to_str(N) ->\n    erlang:integer_to_list(N).\n\n-spec wait_randomly() -> ok.\nwait_randomly() ->\n    timer:sleep(randoms:rand_uniform(1, 50)).\n\n-spec trigger(trigger()) -> ok.\ntrigger(Trigger) ->\n    Interval =\n        case Trigger of\n            publish_trigger -> config:read(lb_active_directories_publish_interval);\n            directory_trigger -> config:read(lb_active_directories_directory_interval)\n        end,\n    msg_delay:send_trigger(Interval div 1000, {Trigger}).\n\n-spec get_web_debug_kv(state()) -> [{string(), string()}].\nget_web_debug_kv(State) ->\n    [{\"state\", webhelpers:html_pre(\"~p\", [State])}].\n\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(lb_active_directories_publish_interval) and\n    config:cfg_is_greater_than(lb_active_directories_publish_interval, 1000) and\n\n    config:cfg_is_integer(lb_active_directories_directory_interval) and\n    config:cfg_is_greater_than(lb_active_directories_directory_interval, 1000) and\n\n    config:cfg_is_integer(lb_active_directories_max_transfer) and\n    config:cfg_is_greater_than(lb_active_directories_max_transfer, 0).\n"
  },
  {
    "path": "src/lb_active_gossip_load_metric.erl",
    "content": "%  @copyright 2010-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n%% @version $Id$\n-module(lb_active_gossip_load_metric).\n-author('michels@zib.de').\n-vsn('$Id$').\n\n-behaviour(gossip_load_beh).\n\n-export([get_load/1, init_histo/2]).\n\n-spec get_load(node_details:node_details()) -> gossip_load_beh:load().\nget_load(_NodeDetails) ->\n    lb_stats:get_load_metric().\n\n-spec init_histo(node_details:node_details(), NumberOfBuckets::pos_integer())\n                    -> gossip_load:histogram().\ninit_histo(NodeDetails, NumberOfBuckets) ->\n    MyRange = node_details:get(NodeDetails, my_range),\n    Buckets = intervals:split(intervals:all(), NumberOfBuckets),\n    [ {BucketInterval, get_load_for_interval(BucketInterval, MyRange)}\n        || BucketInterval <- Buckets ].\n\n-spec get_load_for_interval(BucketInterval::intervals:interval(),\n    MyRange::intervals:interval()) -> gossip_load:avg() | unknown.\nget_load_for_interval(BucketInterval, MyRange) ->\n    Intersection = intervals:intersection(BucketInterval, MyRange),\n    case intervals:is_empty(Intersection) of\n        true -> unknown;\n        false ->\n            Load = lb_stats:default_value(lb_stats:get_load_metric()),\n            {float(Load), 1.0}\n    end.\n"
  },
  {
    "path": "src/lb_active_gossip_request_metric.erl",
    "content": "%  @copyright 2010-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n%% @version $Id$\n-module(lb_active_gossip_request_metric).\n-author('michels@zib.de').\n-vsn('$Id$').\n\n-behaviour(gossip_load_beh).\n\n-export([get_load/1, init_histo/2]).\n\n-spec get_load(node_details:node_details()) -> gossip_load_beh:load().\nget_load(_NodeDetails) ->\n    lb_stats:get_request_metric().\n\n-spec init_histo(node_details:node_details(), NumberOfBuckets::pos_integer())\n                    -> gossip_load:histogram().\ninit_histo(NodeDetails, NumberOfBuckets) ->\n    MyRange = node_details:get(NodeDetails, my_range),\n    Buckets = intervals:split(intervals:all(), NumberOfBuckets),\n    [ {BucketInterval, get_load_for_interval(BucketInterval, MyRange)}\n        || BucketInterval <- Buckets ].\n\n-spec get_load_for_interval(BucketInterval::intervals:interval(),\n    MyRange::intervals:interval()) -> gossip_load:avg() | unknown.\nget_load_for_interval(BucketInterval, MyRange) ->\n    Intersection = intervals:intersection(BucketInterval, MyRange),\n    case intervals:is_empty(Intersection) of\n        true -> unknown;\n        false ->\n            Load = lb_stats:default_value(lb_stats:get_request_metric()),\n            {float(Load), 1.0}\n    end.\n"
  },
  {
    "path": "src/lb_active_karger.erl",
    "content": "%  @copyright 2013-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maximilian Michels <michels@zib.de>\n%% @doc Implementation of Karger and Ruhl's item balancing load balancing algorithm.\n%%      Modified to sample N nodes and use gossip information.\n%% @end\n%% @reference D. R. Karger and M. Ruhl,\n%%            \"Simple efficient load balancing algorithms for peer-to-peer systems,\"\n%%            in Proceedings of the sixteenth annual ACM symposium on Parallelism in algorithms and architectures,\n%%            2004, pp. 36-43.\n%% @version $Id$\n-module(lb_active_karger).\n-author('michels@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-define(TRACE(X,Y), ok).\n%-define(TRACE(X,Y), io:format(\"lb_active_karger: \" ++ X,Y)).\n\n-behaviour(lb_active_beh).\n\n-export([init/0, check_config/0]).\n-export([handle_msg/2, handle_dht_msg/2]).\n-export([get_web_debug_kv/1]).\n\n-record(state, {epsilon          = ?required(state, epsilon) :: float(),\n                rnd_node         = []                        :: [node:node_type()],\n                best_candidate   = []                        :: [{items | requests, {LoadChange::non_neg_integer(), node:node_type()}}],\n                round_id         = nil                       :: non_neg_integer() | nil,\n                my_lb_info       = nil                       :: lb_info:lb_info() | nil,\n                req_ids          = []                        :: [{integer(), node:node_type()}]\n               }).\n\n-type state() :: #state{}.\n\n-type(my_message() ::\n           %% trigger messages\n           {lb_active_karger_trigger} |\n           %% random node from cyclon\n           {cy_cache, [node:node_type()]} |\n           %% load response from dht node\n           {my_dht_response, DhtNode :: comm:mypid(), {get_state_response, Load :: number()}} |\n           %% Result from slide or jump\n           dht_node_move:result_message() |\n           %% simulation\n           {simulation_result, Id::integer(), ReqId::integer(), {items | requests, LoadChange::non_neg_integer()}} |\n           {pick_best_candidate, Id::integer()}).\n\n-type options() :: [{epsilon, float()} | {id, integer()} | {simulate} | {reply_to, comm:mypid()}].\n\n-type dht_message() ::\n\t\t   %% phase1\n\t\t   {lb_active, phase1, NodeX :: lb_info:lb_info(), options()} |\n\t\t   %% phase2\n\t\t   {lb_active, phase2, HeavyNode :: lb_info:lb_info(), LightNode :: lb_info:lb_info()}.\n\n\n%%%%%%%%%%%%%%%\n%%  Startup   %\n%%%%%%%%%%%%%%%\n\n%% @doc Initialization of module called by lb_active\n-spec init() -> state().\ninit() ->\n    trigger(),\n    Epsilon = config:read(lb_active_karger_epsilon),\n    #state{epsilon = Epsilon}.\n\n%%%%%%%%%%%%%%%\n%%  Trigger   %\n%%%%%%%%%%%%%%%\n\n-spec handle_msg(my_message(), state()) -> state().\nhandle_msg({lb_active_karger_trigger}, State) ->\n    trigger(),\n    %% Request N random nodes from cyclon\n    NumNodes = config:read(lb_active_karger_rnd_nodes),\n    gossip_cyclon:get_subset_rand(NumNodes),\n    State;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%  Handling of lb process related messages  %\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% No random nodes available via cyclon\nhandle_msg({cy_cache, []}, State) ->\n    ?TRACE(\"Cyclon returned no random node~n\", []),\n    State;\n\n%% Got a random node via cyclon\nhandle_msg({cy_cache, RandomNodes}, State) ->\n    ?TRACE(\"Got random node~n\", []),\n    case config:read(lb_active_karger_epsilon) of\n        self_tuning -> request_my_gossip_values();\n        _ -> request_my_node_details()\n    end,\n    State#state{rnd_node = RandomNodes};\n\n%% Process gossip values from gossip process to determine epsilon\n%% in case lb_active_karger_epsilon is set to self_tuning.\nhandle_msg({gossip_get_values_best_response, LoadInfo}, State) ->\n    Module = lb_active_gossip_load_metric,\n    Avg = gossip_load:load_info_other_get(avgLoad, Module, LoadInfo),\n    Stddev = gossip_load:load_info_other_get(stddev, Module, LoadInfo),\n    Max = gossip_load:load_info_other_get(maxLoad, Module, LoadInfo),\n    Epsilon =\n        try\n            lb_info:bound(0.01, Avg / erlang:max(Avg + Stddev, Max - Stddev), 0.24)\n        catch\n            error:badarith -> 0.24\n        end,\n    request_my_node_details(),\n    State#state{epsilon = Epsilon};\n\n%% Got load from my node\nhandle_msg({my_dht_response, {get_node_details_response, NodeDetails}}, State) ->\n\t?TRACE(\"Received node details for own node~n\", []),\n\tRandomNodes = State#state.rnd_node,\n\tEpsilon = State#state.epsilon,\n    ?TRACE(\"Epsilon: ~p~n\", [Epsilon]),\n    %% If we deal only with one random node, we don't have\n    %% any choice but to go to the next phase.\n    %% Otherwise, we ask all random nodes for their load\n    %% and calculate the load changes before going to the\n    %% next phase.\n    MyLBInfo = lb_info:new(NodeDetails),\n    IsValid = lb_info:is_valid(MyLBInfo),\n    Id = randoms:getRandomInt(), %%uid:get_global_uid(),\n    Options = [{id, Id}, {epsilon, Epsilon}],\n    case RandomNodes of\n        [RndNode] when IsValid ->\n            comm:send(node:pidX(RndNode), {lb_active, phase1, MyLBInfo, Options}, [{?quiet}]),\n            State#state{rnd_node = []};\n        RndNodes when IsValid ->\n            This = comm:this(),\n            ReqIds =\n                [begin\n                      ReqId = randoms:getRandomInt(),\n                      ?TRACE(\"Sending out simulate request with ReqId ~p to ~.0p~n\", [ReqId, node:pidX(RndNode)]),\n                      OptionsNew = [{simulate, ReqId}, {reply_to, This} | Options],\n                      comm:send(node:pidX(RndNode), {lb_active, phase1, MyLBInfo, OptionsNew}, [{?quiet}]),\n                      {ReqId, RndNode}\n                 end || RndNode <- RndNodes],\n            Timeout = config:read(lb_active_karger_simulation_timeout) div 1000,\n            msg_delay:send_local(Timeout, self(), {pick_best_candidate, Id}),\n            State#state{round_id = Id, my_lb_info = MyLBInfo, req_ids = ReqIds};\n        _ ->\n            State\n    end;\n\n%% collect all the load change responses and save the best candidate\nhandle_msg({simulation_result, Id, ThisReqId, {Metric, LoadChange}}, State) ->\n    ?TRACE(\"Received load change ~p in round ~p~n\", [LoadChange, Id]),\n    case State#state.round_id of\n        Id ->\n            ReqIds = State#state.req_ids,\n            ReqIdsNew = lists:keydelete(ThisReqId, 1, ReqIds),\n            case ReqIdsNew of\n                [] -> comm:send_local(self(), {pick_best_candidate, Id});\n                _  -> ok\n            end,\n            {ThisReqId, NodeX} = lists:keyfind(ThisReqId, 1, ReqIds),\n\n            Best = State#state.best_candidate,\n            {BestLoadChange, _Node} = proplists:get_value(Metric, Best, {0, nil}),\n\n            case LoadChange < BestLoadChange of\n                true  ->\n                    NewBest = lists:keystore(Metric, 1, Best, {Metric, {LoadChange, NodeX}}),\n                    State#state{req_ids = ReqIdsNew,\n                                best_candidate = NewBest};\n                _ ->\n                    State#state{req_ids = ReqIdsNew}\n            end;\n        _ ->\n           ?TRACE(\"Discarding old round with Id ~p~n\", [Id]),\n           State\n    end;\n\n%% In case we have a best candidate, start the actual\n%% load balancing algorithm.\nhandle_msg({pick_best_candidate, Id}, State) ->\n    ?TRACE(\"Deciding in round ~p~n\",[Id]),\n    case State#state.round_id of\n        Id ->\n            Best = State#state.best_candidate,\n            BestCandidate =\n                case lists:keyfind(requests, 1, Best) of\n                    {requests, {_LoadChange, Node}} -> Node;\n                    _ ->\n                        case lists:keyfind(items, 1, Best) of\n                            {items, {_LoadChange, Node}} -> Node;\n                            _ -> nil\n                        end\n                end,\n            case BestCandidate of\n                nil -> ?TRACE(\"No best candidate in Round ~p~n\", [Id]);\n                BestCandidate ->\n                    BestPid = node:pidX(BestCandidate),\n                    Epsilon = State#state.epsilon,\n                    MyLBInfo = State#state.my_lb_info,\n                    ?TRACE(\"Sending out decision in round ~p: BestCandidate: ~w~n\", [Id, BestCandidate]),\n                    Options = [{id, Id}, {epsilon, Epsilon}],\n                    comm:send(BestPid, {lb_active, phase1, MyLBInfo, Options})\n            end,\n            State#state{best_candidate = [], round_id = nil};\n        _ ->\n            ?TRACE(\"Old decision message for round ~p~n\", [Id]),\n            State\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%  Static methods called by dht_node message handler  %\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec handle_dht_msg(dht_message(), dht_node_state:state()) -> dht_node_state:state().\n%% First phase: We were contacted by another node who chose\n%% us as a random node. In this phase we'll determine if\n%% load balancing is necessary. If so, we'll try to balance\n%% assuming the two nodes are neighbors. If not we'll contact\n%% the light node's successor for more load information.\nhandle_dht_msg({lb_active, phase1, NodeX, Options}, DhtState) ->\n    {epsilon, Epsilon} = lists:keyfind(epsilon, 1, Options),\n\tMyLBInfo = lb_info:new(dht_node_state:details(DhtState)),\n    IsValid = lb_info:is_valid(MyLBInfo),\n\tMyLoad = lb_info:get_load(MyLBInfo),\n\tLoadX = lb_info:get_load(NodeX),\n    if IsValid andalso (MyLoad =/= 0 orelse LoadX =/= 0) ->\n           if\n               % first check if load balancing is necessary\n               MyLoad =< Epsilon * LoadX ->\n                   ?TRACE(\"My node is light~n\", []),\n                   balance_adjacent(NodeX, MyLBInfo, Options);\n               LoadX =< Epsilon * MyLoad ->\n                   ?TRACE(\"My node is heavy~n\", []),\n                   balance_adjacent(MyLBInfo, NodeX, Options);\n               true ->\n                   %% no balancing\n                   ?TRACE(\"Won't balance~n\", []),\n                   lb_active:balance_noop(Options)\n           end;\n       true -> lb_active:balance_noop(Options)\n    end,\n\tDhtState;\n\n%% Second phase: We are LightNode's successor. We might hold\n%% more load than the HeavyNode. If so, we'll slide with the\n%% LightNode. Otherwise we instruct the HeavyNode to set up\n%% a jump operation with the Lightnode.\nhandle_dht_msg({lb_active, phase2, HeavyNode, LightNode, Options}, DhtState) ->\n\t?TRACE(\"In phase 2~n\", []),\n\tMyLBInfo = lb_info:new(dht_node_state:details(DhtState)),\n    IsValid = lb_info:is_valid(MyLBInfo),\n\tMyLoad = lb_info:get_load(MyLBInfo),\n    LoadHeavyNode = lb_info:get_load(HeavyNode),\n    if IsValid andalso MyLoad > LoadHeavyNode ->\n           % slide\n           lb_active:balance_nodes(HeavyNode, LightNode, Options);\n       IsValid ->\n           % jump\n           lb_active:balance_nodes(HeavyNode, LightNode, MyLBInfo, Options);\n       true -> ok\n    end,\n\tDhtState.\n\n%%%%%%%%%%%%%%%%%%%%\n%% Helper methods  %\n%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Balance if the two nodes are adjacent, otherwise ask the light node's neighbor\n-spec balance_adjacent(lb_info:lb_info(), lb_info:lb_info(), options()) -> ok.\nbalance_adjacent(HeavyNode, LightNode, Options) ->\n\tcase lb_info:neighbors(HeavyNode, LightNode) of %%lb_info:is_succ(HeavyNode, LightNode) of\n\t\ttrue ->\n\t\t\t% neighbors, thus sliding\n\t\t\t?TRACE(\"We're neighbors~n\", []),\n            %% slide in phase1 or phase2\n            lb_active:balance_nodes(HeavyNode, LightNode, Options);\n\t\t_ ->\n\t\t\t% ask the successor of the light node how much load he carries\n\t\t\t?TRACE(\"Nodes not adjacent, requesting information about neighbors~n\", []),\n            LightNodeSucc = lb_info:get_succ(LightNode),\n\t\t\tcomm:send(node:pidX(LightNodeSucc), {lb_active, phase2, HeavyNode, LightNode, Options})\n\tend.\n\n-spec request_my_node_details() -> ok.\nrequest_my_node_details() ->\n    MyDhtNode = pid_groups:get_my(dht_node),\n    Envelope = comm:reply_as(comm:this(), 2, {my_dht_response, '_'}),\n    comm:send_local(MyDhtNode, {get_node_details, Envelope}).\n\n-spec request_my_gossip_values() -> ok.\nrequest_my_gossip_values() ->\n    gossip_load:get_values_best([]).\n\n-spec trigger() -> ok.\ntrigger() ->\n    Interval = config:read(lb_active_karger_interval) div 1000,\n    msg_delay:send_trigger(Interval, {lb_active_karger_trigger}).\n\n%% @doc Key/Value List for web debug\n-spec get_web_debug_kv(state()) -> [{string(), string()}].\nget_web_debug_kv(State) ->\n    [{\"state\", webhelpers:html_pre(\"~p\", [State])}].\n\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(lb_active_karger_interval) and\n    config:cfg_is_greater_than(lb_active_karger_interval, 0) and\n\n    (config:read(lb_active_karger_epsilon) =/= self_tuning orelse\n     lb_active:check_gossip_modules(lb_active_gossip_load_metric, lb_active_karger_epsilon)\n    ) and\n\n    (config:read(lb_active_karger_epsilon) =:= self_tuning orelse\n     config:cfg_is_float(lb_active_karger_epsilon) and\n     config:cfg_is_greater_than(lb_active_karger_epsilon, 0.0) and\n     config:cfg_is_less_than(lb_active_karger_epsilon, 0.25)\n    ) and\n\n    config:cfg_is_integer(lb_active_karger_rnd_nodes) and\n    config:cfg_is_greater_than_equal(lb_active_karger_rnd_nodes, 1) and\n\n    config:cfg_is_integer(lb_active_karger_simulation_timeout) and\n    config:cfg_is_greater_than(lb_active_karger_simulation_timeout, 0).\n"
  },
  {
    "path": "src/lb_common.erl",
    "content": "%  @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    TODO: Add description to lb_common\n%% @end\n%% @version $Id$\n-module(lb_common).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n-export([calculateStddev/2, bestStddev/2, bestStddev/3,\n         split_by_key/2, split_my_range/2]).\n\n\n% Avg2 = average of load ^2\n-spec calculateStddev(Avg::number(), Avg2::number()) -> float().\ncalculateStddev(Avg, Avg2) ->\n    math:sqrt(Avg2 - Avg * Avg).\n\n-spec default_sort_fun(Op1::{lb_op:lb_op(), integer()},\n                       Op2::{lb_op:lb_op(), integer()}) -> boolean().\ndefault_sort_fun({_Op1, Op1Change}, {_Op2, Op2Change}) ->\n    Op1Change =< Op2Change.\n\n% MinSum2Change = minimal required change of sum(load ^2)\n-spec bestStddev(Ops::[lb_op:lb_op()], MinSum2Change::integer() | plus_infinity)\n        -> [lb_op:lb_op()].\nbestStddev(Ops, MinSum2Change) ->\n    bestStddev(Ops, MinSum2Change, fun default_sort_fun/2).\n\n-spec bestStddev(Ops::[lb_op:lb_op()], MinSum2Change::integer() | plus_infinity,\n                 SortFun::fun((Op1::{lb_op:lb_op(), integer()},\n                               Op2::{lb_op:lb_op(), integer()})\n                                -> boolean()))\n        -> [lb_op:lb_op()].\nbestStddev(Ops, MinSum2Change, SortFun) ->\n%%     log:pal(\"[ ~.0p ] bestStddev(~.0p, ~.0p)~n\", [self(), Ops, MinSum2Change]),\n    OpsWithSum2Change =\n        [{Op, Sum2Change} || Op <- Ops,\n                             not lb_op:is_no_op(Op),\n                             Sum2Change <- [calc_sum2Change(Op)],\n                             MinSum2Change =:= plus_infinity orelse\n                                 Sum2Change < MinSum2Change],\n%%     log:pal(\"[ ~.0p ] OpsWithSum2Change: ~.0p~n\", [self(), OpsWithSum2Change]),\n    OpsWithSum2Change_sort = lists:sort(SortFun, OpsWithSum2Change),\n%%     log:pal(\"[ ~.0p ] OpsWithSum2Change_sort: ~.0p~n\", [self(), OpsWithSum2Change_sort]),\n    [Op || {Op, _Sum2Change} <- OpsWithSum2Change_sort].\n\n%% @doc Calculates the change of the sum of the square of the nodes' loads\n%%      after applying the given operation. \n-spec calc_sum2Change(Op::lb_op:lb_op()) -> integer().\ncalc_sum2Change(Op) ->\n    N1Load = node_details:get(lb_op:get(Op, n1), load),\n    N1NewLoad = node_details:get(lb_op:get(Op, n1_new), load),\n    N1SuccLoad = node_details:get(lb_op:get(Op, n1succ), load),\n    N1SuccNewLoad = node_details:get(lb_op:get(Op, n1succ_new), load),\n    Sum2ChangeOp1 = - (N1Load * N1Load) + (N1NewLoad * N1NewLoad)\n                    - (N1SuccLoad * N1SuccLoad) + (N1SuccNewLoad * N1SuccNewLoad),\n    case lb_op:is_jump(Op) of\n        true ->\n            N3Load = node_details:get(lb_op:get(Op, n3), load),\n            N3NewLoad = node_details:get(lb_op:get(Op, n3_new), load),\n            (Sum2ChangeOp1 - (N3Load * N3Load) + (N3NewLoad * N3NewLoad));\n        _ -> Sum2ChangeOp1\n    end.\n\n%% @doc Returns the given SplitKey and the load that would be split off by\n%%      using this key.\n-spec split_by_key(DhtNodeState::dht_node_state:state(), SelectedKey::?RT:key())\n        -> {SplitKey::?RT:key(), TargetLoadNew::non_neg_integer()}.\nsplit_by_key(DhtNodeState, SelectedKey) ->\n    MyPredId = dht_node_state:get(DhtNodeState, pred_id),\n    DB = dht_node_state:get(DhtNodeState, db),\n    Interval = node:mk_interval_between_ids(MyPredId, SelectedKey),\n    TargetLoadNew = db_dht:get_load(DB, Interval),\n%%     log:pal(\"[ ~.0p ]  TN: ~.0p, SK: ~.0p~n\", [self(), TargetLoadNew, SelectedKey]),\n    {SelectedKey, TargetLoadNew}.\n\n%% @doc Returns the SplitKey that splits the current node's address range in\n%%      two (almost) equal halves.\n-spec split_my_range(DhtNodeState::dht_node_state:state(), SelectedKey::?RT:key())\n        -> {SplitKey::?RT:key(), TargetLoadNew::non_neg_integer()}.\nsplit_my_range(DhtNodeState, SelectedKey) ->\n    MyNodeId = dht_node_state:get(DhtNodeState, node_id),\n    MyPredId = dht_node_state:get(DhtNodeState, pred_id),\n    % note: MyNodeId cannot be ?PLUS_INFINITY so this split key is valid\n    SplitKey = try ?RT:get_split_key(MyPredId, MyNodeId, {1, 2})\n               catch throw:not_supported -> SelectedKey\n               end,\n%%     log:pal(\"[ ~.0p ] Pred: ~.0p, My: ~.0p,~nSplit: ~.0p, Selected: ~.0p~n\",\n%%            [self(), MyPredId, MyNodeId, SplitKey, SelectedKey]),\n    Interval = node:mk_interval_between_ids(MyPredId, SplitKey),\n    DB = dht_node_state:get(DhtNodeState, db),\n    TargetLoadNew = db_dht:get_load(DB, Interval),\n    {SplitKey, TargetLoadNew}.\n"
  },
  {
    "path": "src/lb_info.erl",
    "content": "%  @copyright 2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maximilian Michels <michels@zib.de>\n%% @doc For active load balancing:\n%%          - contains information about a node\n%%          - provides functions to evaluate an lb operation\n%% @version $Id$\n-module(lb_info).\n-author('michels@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-export([new/1, is_valid/1]).\n-export([get_load/1, get_reqs/1, get_node/1, get_succ/1, get_items/1, get_time/1]).\n-export([is_succ/2, neighbors/2, get_target_load/3, get_target_load/4]).\n%% without dht size\n-export([get_load_change_slide/4, get_load_change_jump/5]).\n%% with dht size available\n-export([get_load_change_slide/5, get_load_change_jump/6]).\n-export([get_oldest_data_time/1]).\n% util\n-export([bound/3]).\n\n-export_type([lb_info/0]).\n\n-type load() :: unknown | number().\n\n-record(lb_info, {load  = unknown                   :: unknown | load(),\n                  reqs  = unknown                   :: unknown | load(),\n                  items = ?required(lb_info, items) :: load(),\n                  node  = ?required(lb_info, node)  :: node:node_type(),\n                  succ  = ?required(lb_info, succ)  :: node:node_type(),\n                  time  = os:timestamp()            :: erlang_timestamp()\n                 }).\n\n-opaque lb_info() :: #lb_info{}.\n\n%% @doc Creates a new record to hold essential load balancing values\n-spec new(NodeDetails::node_details:node_details()) -> lb_info().\nnew(NodeDetails) ->\n    Items = node_details:get(NodeDetails, load),\n    %% lb_stats:get_load_metric(), can be unknown\n    SystemLoad = node_details:get(NodeDetails, load2),\n    %% lb_stats:get_request_metric(), can be unknown\n    Requests = node_details:get(NodeDetails, load3),\n    Load = case config:read(lb_active_balance) of\n               items -> Items;\n               requests ->\n                   case config:read(lb_active_fall_back_to_items) of\n                       true ->\n                           try\n                               (erlang:round(math:sqrt(Items)) + Requests) * SystemLoad\n                           catch\n                               error:badarith -> unknown\n                           end;\n                       _ -> Requests\n                   end\n           end,\n    #lb_info{load  = Load,\n             reqs  = Requests,\n             items = Items,\n             node  = node_details:get(NodeDetails, node),\n             succ  = node_details:get(NodeDetails, succ)}.\n\n-spec get_load(LBInfo::lb_info()) -> load().\nget_load (#lb_info{load  = Load }) -> Load.\n\n-spec get_reqs(LBInfo::lb_info()) -> load().\nget_reqs (#lb_info{reqs  = Requests}) -> Requests.\n\n-spec get_items(LBInfo::lb_info()) -> load().\nget_items(#lb_info{items = Items}) -> Items.\n\n-spec get_node(LBInfo::lb_info()) -> node:node_type().\nget_node (#lb_info{node  = Node }) -> Node.\n\n-spec get_succ(LBInfo::lb_info()) -> node:node_type().\nget_succ (#lb_info{succ  = Succ }) -> Succ.\n\n-spec get_time(LBInfo::lb_info()) -> erlang_timestamp().\nget_time (#lb_info{time  = Time }) -> Time.\n\n-spec is_succ(Node1::lb_info(), Node2::lb_info()) -> boolean().\nis_succ(Succ, Node) ->\n    get_succ(Node) =:= get_node(Succ).\n\n-spec neighbors(Node1::lb_info(), Node2::lb_info()) -> boolean().\nneighbors(Node1, Node2) ->\n    is_succ(Node1, Node2) orelse is_succ(Node2, Node1).\n\n-spec is_valid(Info::lb_info()) -> boolean().\nis_valid(Info) -> is_number(get_load(Info)).\n\n%% @doc The number of db entries the heavy node will give to the light node\n-spec get_target_load(items | requests, Op::slide | jump, HeavyNode::lb_info(), LightNode::lb_info()) -> non_neg_integer().\nget_target_load(items, JumpOrSlide, HeavyNode, LightNode) ->\n    get_target_load(JumpOrSlide, get_items(HeavyNode), get_items(LightNode));\nget_target_load(requests, JumpOrSlide, HeavyNode, LightNode) ->\n    get_target_load(JumpOrSlide, get_reqs(HeavyNode), get_reqs(LightNode)).\n\n%% @doc The number of db entries the heavy node will give to the light node (weighted)\n-spec get_target_load(Op::slide | jump, HeavyNode::load(), LightNode::load())\n                    -> non_neg_integer().\nget_target_load(_Op, unknown, _LightNode) -> 0;\nget_target_load(_Op, _HeavyNode, unknown) -> 0;\nget_target_load(slide, HeavyNode, LightNode) ->\n    TotalItems = HeavyNode + LightNode,\n    AvgItems = trunc(TotalItems) div 2,\n    ItemsToShed = HeavyNode - AvgItems,\n    bound(0, ItemsToShed, HeavyNode);\nget_target_load(jump, HeavyNode, _LightNode) ->\n    AvgItems = trunc(HeavyNode) div 2,\n    ItemsToShed = HeavyNode - AvgItems,\n    bound(0, ItemsToShed, HeavyNode).\n\n%% @doc Calculates the change in Variance\n%% no dht size available\n-spec get_load_change_slide(Metric::items | requests, TakenLoad::non_neg_integer(),\n                            HeavyNode::lb_info(), LightNode::lb_info()) -> LoadChange::number().\nget_load_change_slide(Metric, TakenLoad, HeavyNode, LightNode) ->\n    get_load_change_slide(Metric, TakenLoad, 1, HeavyNode, LightNode).\n\n%% @doc Calculates the change in Variance\n%% dht size available\n-spec get_load_change_slide(Metric::items | requests, TakenLoad::non_neg_integer(), DhtSize::pos_integer(),\n                            HeavyNode::lb_info(), LightNode::lb_info()) -> LoadChange::number().\nget_load_change_slide(Metric, TakenLoad, DhtSize, HeavyNode, LightNode) ->\n    MetricFun = get_metric_fun(Metric),\n    get_load_change_diff(DhtSize, MetricFun(HeavyNode), MetricFun(HeavyNode) - TakenLoad) +\n        get_load_change_diff(DhtSize, MetricFun(LightNode), MetricFun(LightNode) + TakenLoad).\n\n%% @doc Calculates the change in Variance\n%% no dht size available\n-spec get_load_change_jump(Metric::items | requests, TakenLoad::non_neg_integer(),\n                           HeavyNode::lb_info(), LightNode::lb_info(), LightNodeSucc::lb_info()) -> LoadChange::number().\nget_load_change_jump(Metric, TakenLoad, HeavyNode, LightNode, LightNodeSucc) ->\n    get_load_change_jump(Metric, TakenLoad, 1, HeavyNode, LightNode, LightNodeSucc).\n\n%% @doc Calculates the change in Variance\n%% dht size available\n-spec get_load_change_jump(Metric::items | requests, TakenLoad::non_neg_integer(), DhtSize::pos_integer(),\n                           HeavyNode::lb_info(), LightNode::lb_info(), LightNodeSucc::lb_info()) -> LoadChange::number().\nget_load_change_jump(Metric, TakenLoad, DhtSize, HeavyNode, LightNode, LightNodeSucc) ->\n    MetricFun = get_metric_fun(Metric),\n    get_load_change_diff(DhtSize, MetricFun(LightNode), TakenLoad) +\n        get_load_change_diff(DhtSize, MetricFun(LightNodeSucc), MetricFun(LightNodeSucc) + MetricFun(LightNode)) +\n        get_load_change_diff(DhtSize, MetricFun(HeavyNode), MetricFun(HeavyNode) - TakenLoad).\n\n-spec get_load_change_diff(pos_integer(), non_neg_integer(), non_neg_integer()) -> load().\nget_load_change_diff(DhtSize, OldLoad, NewLoad) ->\n    NewLoad * NewLoad / DhtSize - OldLoad * OldLoad / DhtSize.\n\n-spec get_metric_fun(Metric::items | requests) -> fun((lb_info()) -> load()).\nget_metric_fun(items)    -> fun get_items/1;\nget_metric_fun(requests) -> fun get_reqs/1.\n\n-spec get_oldest_data_time([lb_info()]) -> OldestTime::erlang_timestamp().\nget_oldest_data_time([Node | Other]) ->\n    get_oldest_data_time(Other, get_time(Node)).\n\n-spec get_oldest_data_time([lb_info()], Oldest::erlang_timestamp()) -> OldestTime::erlang_timestamp().\nget_oldest_data_time([], Oldest) ->\n    Oldest;\nget_oldest_data_time([Node | Other], Oldest) ->\n    OldestNew = erlang:min(get_time(Node), Oldest),\n    get_oldest_data_time(Other, OldestNew).\n\n-spec bound(LowerBound::number(), Value::number(), UpperBound::number()) -> number().\nbound(LowerBound, Value, UpperBound) ->\n    if Value < LowerBound -> LowerBound;\n       Value > UpperBound -> UpperBound;\n       true -> Value\n    end.\n"
  },
  {
    "path": "src/lb_op.erl",
    "content": "%  @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Balance operation structure\n%% @end\n%% @version $Id$\n-module(lb_op).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-export([no_op/0, slide_op/5, jump_op/7,\n        is_no_op/1, is_slide/1, is_jump/1, get/2]).\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-export_type([lb_op/0, id/0]).\n\n-type id() :: any().\n\n-type type() :: slide | jump.\n\n-record(lb_op,\n        {type       = ?required(lb_op, type)       :: type(),\n         id         = ?required(lb_op, id)         :: id(),\n         % first node involved (node that slides or moves)\n         n1         = ?required(lb_op, n1)         :: node_details:node_details(),\n         % successor of node1\n         n1succ     = ?required(lb_op, n1succ)     :: node_details:node_details(),\n         % node to move to if type == jump\n         n3         = null                         :: node_details:node_details() | null,\n         n1_new     = ?required(lb_op, n1_new)     :: node_details:node_details(),\n         n1succ_new = ?required(lb_op, n1succ_new) :: node_details:node_details(),\n         n3_new     = null                         :: node_details:node_details() | null\n        }).\n-opaque lb_op() :: #lb_op{} | no_op.\n\n%% @doc Creates a no_op operation.\n-spec no_op() -> lb_op().\nno_op() -> no_op.\n\n%% @doc Creates a new slide operation with the given nodes and their details\n%%      after the slide.\n-spec slide_op(Id::id(),\n    Node::node_details:node_details(), Successor::node_details:node_details(),\n    NodeNew::node_details:node_details(), SuccessorNew::node_details:node_details())\n        -> lb_op().\nslide_op(Id, Node, Successor, NodeNew, SuccessorNew) ->\n    #lb_op{type = slide, id = Id,\n           n1 = Node, n1succ = Successor,\n           n1_new = NodeNew, n1succ_new = SuccessorNew}.\n\n%% @doc Creates a new jump operation with the given nodes and their details\n%%      after the jump.\n-spec jump_op(Id::id(),\n    NodeToMove::node_details:node_details(), NodeToMove_succ::node_details:node_details(), NodePosition::node_details:node_details(),\n    NodeToMoveNew::node_details:node_details(), NodeToMove_succNew::node_details:node_details(), NodePositionNew::node_details:node_details())\n        -> lb_op().\njump_op(Id, NodeToMove, NodeToMove_succ, NodePosition,\n        NodeToMoveNew, NodeToMove_succNew, NodePositionNew) ->\n    #lb_op{type = jump, id = Id,\n           n1 = NodeToMove, n1succ = NodeToMove_succ, n3 = NodePosition,\n           n1_new = NodeToMoveNew, n1succ_new = NodeToMove_succNew, n3_new = NodePositionNew}.\n\n%% @doc Determines whether the operation is a no_op, i.e. no operation.\n-spec is_no_op(Op::lb_op()) -> boolean().\nis_no_op(no_op) -> true;\nis_no_op(#lb_op{}) -> false.\n\n%% @doc Determines whether the operation is a slide.\n-spec is_slide(Op::lb_op()) -> boolean().\nis_slide(no_op) -> false;\nis_slide(Op) -> Op#lb_op.type =:= slide.\n\n%% @doc Determines whether the operation is a jump.\n-spec is_jump(Op::lb_op()) -> boolean().\nis_jump(no_op) -> false;\nis_jump(Op) -> Op#lb_op.type =:= jump.\n\n%% @doc Gets the selected property from the load balancing operation.\n-spec get(lb_op(), n1 | n1succ| n1_new | n1succ_new) -> node_details:node_details();\n         (lb_op(), n3 | n3_new) -> node_details:node_details() | null;\n         (lb_op(), id) -> id().\nget(#lb_op{id=Id, n1=N1, n1succ=N1Succ, n3=N3,\n           n1_new=N1New, n1succ_new=N1SuccNew, n3_new=N3New}, Key) ->\n    case Key of\n        id -> Id;\n        n1 -> N1;\n        n1succ -> N1Succ;\n        n3 -> N3;\n        n1_new -> N1New;\n        n1succ_new -> N1SuccNew;\n        n3_new -> N3New\n    end.\n"
  },
  {
    "path": "src/lb_psv_beh.erl",
    "content": "%  @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Passive load balancing algorithm behavior\n%% @end\n%% @version $Id$\n-module(lb_psv_beh).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n% for behaviour\n-ifndef(have_callback_support).\n-export([behaviour_info/1]).\n-endif.\n\n%% userdevguide-begin rt_beh:behaviour\n-ifdef(have_callback_support).\n-include(\"scalaris.hrl\").\n\n-callback get_number_of_samples(Conn::dht_node_join:connection()) -> ok.\n-callback get_number_of_samples_remote(\n        SourcePid::comm:mypid(), Conn::dht_node_join:connection()) -> ok.\n-callback create_join(DhtNodeState::dht_node_state:state(), SelectedKey::?RT:key(),\n                      SourcePid::comm:mypid(), Conn::dht_node_join:connection())\n        -> dht_node_state:state().\n-callback sort_candidates(Ops::[lb_op:lb_op()]) -> [lb_op:lb_op()].\n-callback process_join_msg(comm:message(), LbPsvState::term(),\n                           DhtNodeState::dht_node_state:state())\n        -> dht_node_state:state() | unknown_event.\n\n-callback check_config() -> boolean().\n\n-else.\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n    [\n     % at the joining node, get the number of IDs to sample\n     {get_number_of_samples, 1},\n     % at an existing node, send the number of IDs to sample to a joining node\n     {get_number_of_samples_remote, 2},\n     % at an existing node, simulate a join operation\n     {create_join, 4},\n     % sort a list of candidates so that the best ones are at the front of the list \n     {sort_candidates, 1},\n     % process join messages with signature {Msg, {join, LbPsv, LbPsvState}}\n     {process_join_msg, 3},\n     % common methods\n     {check_config, 0}\n    ];\nbehaviour_info(_Other) ->\n    undefined.\n-endif.\n%% userdevguide-end rt_beh:behaviour\n"
  },
  {
    "path": "src/lb_psv_beh.hrl",
    "content": "% @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Common types and function specs for passive load balancing algorithm\n%%         implementations.\n%% @end\n%% @version $Id$\n\n-include(\"scalaris.hrl\").\n\n-export([get_number_of_samples/1, get_number_of_samples_remote/2,\n         create_join/4, sort_candidates/1,\n         process_join_msg/3,\n         check_config/0]).\n"
  },
  {
    "path": "src/lb_psv_gossip.erl",
    "content": "%  @copyright 2010-2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Passive load balancing using gossip to sample O(log(N)) nodes (at\n%%         least lb_psv_samples samples) and choose the\n%%         one that reduces the standard deviation the most.\n%%         If there is no load address ranges are split according to\n%%         lb_psv_split_fallback, i.e. in halves or using the selected keys,\n%%         otherwise no more load is moved than is needed to have the average\n%%         load on one of the two nodes.\n%% @end\n%% @version $Id$\n-module(lb_psv_gossip).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-define(TRACE_SEND(Pid, Msg), ?TRACE(\"[ ~.0p ] to ~.0p: ~.0p)~n\", [self(), Pid, Msg])).\n-define(TRACE1(Msg, State), ?TRACE(\"[ ~.0p ]~n  Msg: ~.0p~n  State: ~.0p)~n\", [self(), Msg, State])).\n\n-behaviour(lb_psv_beh).\n-include(\"lb_psv_beh.hrl\").\n\n-export_type([custom_message/0]).\n\n-type custom_message() ::\n          {gossip_get_values_best_response, gossip_load:load_info()} |\n          {get_split_key_response, SplitKey::{?RT:key(), TakenLoad::non_neg_integer()}}.\n\n%% @doc Gets the number of IDs to sample during join.\n%%      Note: this is executed at the joining node.\n-spec get_number_of_samples(Conn::dht_node_join:connection()) -> ok.\nget_number_of_samples({_, ContactNode} = Connection) ->\n    Msg = {join, number_of_samples_request, comm:this(), ?MODULE, Connection},\n    ?TRACE_SEND(ContactNode, Msg),\n    comm:send(ContactNode, Msg).\n\n%% @doc Sends the number of IDs to sample during join to the joining node.\n%%      Note: this is executed at the existing node.\n-spec get_number_of_samples_remote(\n        SourcePid::comm:mypid(), Conn::dht_node_join:connection()) -> ok.\nget_number_of_samples_remote(SourcePid, Connection) ->\n    SPid = comm:reply_as(self(), 3,\n                         {join, ?MODULE, '_',\n                          {get_samples, SourcePid, Connection}}),\n    gossip_load:get_values_best([{source_pid, SPid}]).\n\n%% @doc Creates a join operation if a node would join at my node with the\n%%      given key. This will simulate the join operation and return a lb_op()\n%%      with all the data needed in sort_candidates/1.\n%%      Note: this is executed at an existing node.\n-spec create_join(DhtNodeState::dht_node_state:state(), SelectedKey::?RT:key(),\n                  SourcePid::comm:mypid(), Conn::dht_node_join:connection())\n        -> dht_node_state:state().\ncreate_join(DhtNodeState, SelectedKey, SourcePid, Conn) ->\n    case dht_node_state:get(DhtNodeState, slide_pred) of\n        null ->\n            SPid = comm:reply_as(self(), 3,\n                                 {join, ?MODULE, '_',\n                                  {create_join2, SelectedKey, SourcePid, Conn}}),\n            gossip_load:get_values_best([{source_pid, SPid}]);\n        _ ->\n            % postpone message:\n            Msg = {join, get_candidate, SourcePid, SelectedKey, ?MODULE, Conn},\n            ?TRACE_SEND(self(), Msg),\n            _ = comm:send_local_after(100, self(), Msg),\n            ok\n    end,\n    DhtNodeState.\n\n-spec create_join2(DhtNodeState::dht_node_state:state(), SelectedKey::?RT:key(),\n                   SourcePid::comm:mypid(), BestValues::gossip_load:load_info(),\n                   Conn::dht_node_join:connection()) -> dht_node_state:state().\ncreate_join2(DhtNodeState, SelectedKey, SourcePid, BestValues, Conn) ->\n    case dht_node_state:get(DhtNodeState, slide_pred) of\n        null ->\n            Neighbors = dht_node_state:get(DhtNodeState, neighbors),\n            MyNodeId = nodelist:nodeid(Neighbors),\n            try\n                MyLoad = dht_node_state:get(DhtNodeState, load),\n                if MyLoad >= 2 ->\n%%                        log:pal(\"[ ~.0p ] trying split by load\", [self()]),\n                       TargetLoad =\n                           case gossip_load:load_info_get(avgLoad, BestValues) of\n                               unknown -> util:floor(MyLoad / 2);\n                               AvgLoad when AvgLoad > 0 andalso MyLoad > AvgLoad ->\n                                   util:floor(erlang:min(MyLoad - AvgLoad, AvgLoad));\n                               _       -> util:floor(MyLoad / 2)\n                           end,\n%%                        log:pal(\"T: ~.0p, My: ~.0p, Avg: ~.0p~n\", [TargetLoad, MyLoad, gossip_load:load_info_get(avgLoad, BestValues)]),\n                       MyPredId = node:id(nodelist:pred(Neighbors)),\n                       SPid = comm:reply_as(self(), 3,\n                                            {join, ?MODULE, '_',\n                                             {create_join3, SelectedKey, SourcePid, Conn}}),\n                       DBCache = pid_groups:get_my(dht_node_db_cache),\n\n                       Msg = {get_split_key, dht_node_state:get(DhtNodeState, db),\n                              dht_node_state:get(DhtNodeState, full_range), MyPredId, MyNodeId,\n                              TargetLoad, forward, SPid},\n                       ?TRACE_SEND(DBCache, Msg),\n                       comm:send_local(DBCache, Msg),\n                       DhtNodeState;\n                   true -> % fall-back\n                       create_join3(DhtNodeState, SelectedKey, SourcePid, {MyNodeId, 0}, Conn)\n                end\n            catch\n                Error:Reason -> % fall-back\n                    log:log(error, \"[ Node ~w ] failed to get split key \"\n                                \"for another node: ~.0p:~.0p~n\"\n                                \"  SelectedKey: ~.0p, SourcePid: ~.0p~n  State: ~.0p\",\n                            [self(), Error, Reason, SelectedKey, SourcePid, DhtNodeState]),\n                    create_join3(DhtNodeState, SelectedKey, SourcePid, {MyNodeId, 0}, Conn)\n            end;\n        _ ->\n            % postpone message:\n            SPid = comm:reply_as(self(), 3,\n                                 {join, ?MODULE, '_',\n                                  {create_join2, SelectedKey, SourcePid, Conn}}),\n            gossip_load:get_values_best([{source_pid, SPid}, {send_after, 100}]),\n            DhtNodeState\n    end.\n\n-spec create_join3(DhtNodeState::dht_node_state:state(), SelectedKey::?RT:key(),\n                   SourcePid::comm:mypid(),\n                   SplitKey::{?RT:key(), TakenLoad::non_neg_integer()},\n                   Conn::dht_node_join:connection()) -> dht_node_state:state().\ncreate_join3(DhtNodeState, SelectedKey, SourcePid, SplitKey0, Conn) ->\n    case dht_node_state:get(DhtNodeState, slide_pred) of\n        null ->\n            Candidate =\n                try\n                    MyNode = dht_node_state:get(DhtNodeState, node),\n                    MyNodeId = node:id(MyNode),\n                    {SplitKey, OtherLoadNew} =\n                        case SplitKey0 of\n                            {MyNodeId, _TargetLoadNew} -> % fall-back\n%%                                 log:log(info, \"[ Node ~w ] could not split load - no key in my range, \"\n%%                                             \"using fall-back instead\", [self()]),\n                                case config:read(lb_psv_split_fallback) of\n                                    split_address -> % split address range:\n%%                                         log:pal(\"[ ~.0p ] trying split by address range\", [self()]),\n                                        lb_common:split_my_range(DhtNodeState, SelectedKey);\n                                    keep_key -> % keep (randomly selected) key:\n                                        lb_common:split_by_key(DhtNodeState, SelectedKey)\n                                end;\n                            X -> X\n                        end,\n                    MyPredId = node:id(dht_node_state:get(DhtNodeState, pred)),\n                    case SplitKey of\n                        MyNodeId ->\n                            log:log(warn, \"[ Node ~w ] join requested for my ID (~.0p), \"\n                                        \"sending no_op...\", [self(), MyNodeId]),\n                            lb_op:no_op();\n                        MyPredId ->\n                            log:log(warn, \"[ Node ~w ] join requested for my pred's ID (~.0p), \"\n                                        \"sending no_op...\", [self(), MyPredId]),\n                            lb_op:no_op();\n                        _ ->\n                            MyLoad = dht_node_state:get(DhtNodeState, load),\n                            MyLoadNew = MyLoad - OtherLoadNew,\n                            MyNodeDetails1 = node_details:set(node_details:new(), node, MyNode),\n                            MyNodeDetails = node_details:set(MyNodeDetails1, load, MyLoad),\n                            MyNodeDetailsNew = node_details:set(MyNodeDetails, load, MyLoadNew),\n                            OtherNodeDetails = node_details:set(node_details:new(), load, 0),\n                            OtherNodeDetailsNew1 = node_details:set(node_details:new(), new_key, SplitKey),\n                            OtherNodeDetailsNew2 = node_details:set(OtherNodeDetailsNew1, load, OtherLoadNew),\n                            Interval = node:mk_interval_between_ids(MyPredId, SplitKey),\n                            OtherNodeDetailsNew = node_details:set(OtherNodeDetailsNew2, my_range, Interval),\n                            lb_op:slide_op(comm:this(), OtherNodeDetails, MyNodeDetails,\n                                           OtherNodeDetailsNew, MyNodeDetailsNew)\n                    end\n                catch\n                    Error:Reason ->\n                        log:log(error, \"[ Node ~w ] could not create a candidate \"\n                                    \"for another node: ~.0p:~.0p~n\"\n                                    \"  SelectedKey: ~.0p, SourcePid: ~.0p~n  State: ~.0p\",\n                                [self(), Error, Reason, SelectedKey, SourcePid, DhtNodeState]),\n                        lb_op:no_op()\n                end,\n            Msg = {join, get_candidate_response, SelectedKey, Candidate, Conn},\n            ?TRACE_SEND(SourcePid, Msg),\n            comm:send(SourcePid, Msg);\n        _ ->\n            % re-start with create_join2 (it can then create a new up-to-date request for a split key):\n            SPid = comm:reply_as(self(), 3,\n                                 {join, ?MODULE, '_',\n                                  {create_join2, SelectedKey, SourcePid, Conn}}),\n            gossip_load:get_values_best([{source_pid, SPid}, {send_after, 100}]),\n            ok\n    end,\n    DhtNodeState.\n\n%% @doc Sorts all provided operations so that the one with the biggest change\n%%      of the standard deviation is at the front. In case of no load changes,\n%%      the operation with the largest address range at the joining node will\n%%      be at the front.\n%%      Note: this is executed at the joining node.\n-spec sort_candidates(Ops::[lb_op:lb_op()]) -> [lb_op:lb_op()].\nsort_candidates(Ops) ->\n    lb_common:bestStddev(util:shuffle(Ops), plus_infinity,\n                         fun lb_psv_split:my_sort_fun/2).\n\n-spec process_join_msg(custom_message(),\n        {get_samples, SourcePid::comm:mypid(), Conn::dht_node_join:connection()} |\n            {create_join2, SelectedKey::?RT:key(), SourcePid::comm:mypid(), Conn::dht_node_join:connection()} |\n            {create_join3, SelectedKey::?RT:key(), SourcePid::comm:mypid(), BestValues::gossip_load:load_info(), Conn::dht_node_join:connection()},\n        DhtNodeState::dht_node_state:state()) -> dht_node_state:state().\nprocess_join_msg({gossip_get_values_best_response, BestValues},\n                 {get_samples, SourcePid, Conn}, DhtNodeState) ->\n    MinSamples = conf_get_min_number_of_samples(),\n    Samples = case gossip_load:load_info_get(size, BestValues) of\n                  unknown -> MinSamples;\n                  % always round up:\n                  Size    -> erlang:max(util:ceil((util:log2(Size))), MinSamples)\n              end,\n    comm:send(SourcePid, {join, get_number_of_samples, Samples, Conn}),\n    DhtNodeState;\nprocess_join_msg({gossip_get_values_best_response, BestValues},\n                 {create_join2, SelectedKey, SourcePid, Conn}, DhtNodeState) ->\n    create_join2(DhtNodeState, SelectedKey, SourcePid, BestValues, Conn);\nprocess_join_msg({get_split_key_response, SplitKey},\n                 {create_join3, SelectedKey, SourcePid, Conn}, DhtNodeState) ->\n    create_join3(DhtNodeState, SelectedKey, SourcePid, SplitKey, Conn).\n\n%% @doc Checks whether config parameters of the passive load balancing\n%%      algorithm exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(lb_psv_samples) and\n    config:cfg_is_greater_than_equal(lb_psv_samples, 1) and\n    config:cfg_is_in(lb_psv_split_fallback, [split_address, keep_key]).\n\n%% @doc Gets the minnimum number of nodes to sample (set in the config files).\n-spec conf_get_min_number_of_samples() -> pos_integer().\nconf_get_min_number_of_samples() ->\n    config:read(lb_psv_samples).\n"
  },
  {
    "path": "src/lb_psv_simple.erl",
    "content": "%  @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Simple passive load balancing sampling k nodes and choosing the\n%%         one that reduces the standard deviation the most.\n%%         Uses randomly sampled IDs.\n%% @end\n%% @version $Id$\n-module(lb_psv_simple).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-define(TRACE_SEND(Pid, Msg), ?TRACE(\"[ ~.0p ] to ~.0p: ~.0p)~n\", [self(), Pid, Msg])).\n-define(TRACE1(Msg, State), ?TRACE(\"[ ~.0p ]~n  Msg: ~.0p~n  State: ~.0p)~n\", [self(), Msg, State])).\n\n-behaviour(lb_psv_beh).\n-include(\"lb_psv_beh.hrl\").\n\n-export_type([custom_message/0]).\n\n-type custom_message() :: none().\n\n%% @doc Gets the number of IDs to sample during join.\n%%      Note: this is executed at the joining node.\n-spec get_number_of_samples(Conn::dht_node_join:connection()) -> ok.\nget_number_of_samples(Connection) ->\n    ?TRACE_SEND(self(), {join, get_number_of_samples,\n                             conf_get_number_of_samples(), Connection}),\n    comm:send_local(self(), {join, get_number_of_samples,\n                             conf_get_number_of_samples(), Connection}).\n\n%% @doc Sends the number of IDs to sample during join to the joining node.\n%%      Note: this is executed at the existing node.\n-spec get_number_of_samples_remote(\n        SourcePid::comm:mypid(), Conn::dht_node_join:connection()) -> ok.\nget_number_of_samples_remote(SourcePid, Connection) ->\n    ?TRACE_SEND(SourcePid, {join, get_number_of_samples,\n                          conf_get_number_of_samples(), Connection}),\n    comm:send(SourcePid, {join, get_number_of_samples,\n                          conf_get_number_of_samples(), Connection}).\n\n%% @doc Creates a join operation if a node would join at my node with the\n%%      given key. This will simulate the join operation and return a lb_op()\n%%      with all the data needed in sort_candidates/1.\n%%      Note: this is executed at an existing node.\n-spec create_join(DhtNodeState::dht_node_state:state(), SelectedKey::?RT:key(),\n                  SourcePid::comm:mypid(), Conn::dht_node_join:connection())\n        -> dht_node_state:state().\ncreate_join(DhtNodeState, SelectedKey, SourcePid, Conn) ->\n    case dht_node_state:get(DhtNodeState, slide_pred) of\n        null ->\n            Candidate =\n                try\n                    MyNode = dht_node_state:get(DhtNodeState, node),\n                    MyNodeId = node:id(MyNode),\n                    case SelectedKey of\n                        MyNodeId ->\n                            log:log(warn, \"[ Node ~w ] join requested for my ID (~.0p), \"\n                                        \"sending no_op...\", [self(), MyNodeId]),\n                            lb_op:no_op();\n                        _ ->\n                            MyLoad = dht_node_state:get(DhtNodeState, load),\n                            {SelectedKey, OtherLoadNew} =\n                                lb_common:split_by_key(DhtNodeState, SelectedKey),\n                            MyLoadNew = MyLoad - OtherLoadNew,\n                            MyNodeDetails = node_details:set(\n                                              node_details:set(node_details:new(), node, MyNode), load, MyLoad),\n                            MyNodeDetailsNew = node_details:set(MyNodeDetails, load, MyLoadNew),\n                            OtherNodeDetails = node_details:set(node_details:new(), load, 0),\n                            OtherNodeDetailsNew = node_details:set(\n                                                    node_details:set(node_details:new(), new_key, SelectedKey), load, OtherLoadNew),\n                            lb_op:slide_op(comm:this(), OtherNodeDetails, MyNodeDetails,\n                                           OtherNodeDetailsNew, MyNodeDetailsNew)\n                    end\n                catch\n                    Error:Reason ->\n                        log:log(error, \"[ Node ~w ] could not create a candidate \"\n                                    \"for another node: ~.0p:~.0p~n\"\n                                    \"  SelectedKey: ~.0p, SourcePid: ~.0p~n  State: ~.0p\",\n                                [self(), Error, Reason, SelectedKey, SourcePid, DhtNodeState]),\n                        lb_op:no_op()\n                end,\n            Msg = {join, get_candidate_response, SelectedKey, Candidate, Conn},\n            ?TRACE_SEND(SourcePid, Msg),\n            comm:send(SourcePid, Msg);\n        _ ->\n            % postpone message:\n            Msg = {join, get_candidate, SourcePid, SelectedKey, ?MODULE, Conn},\n            ?TRACE_SEND(SourcePid, Msg),\n            _ = comm:send_local_after(100, self(), Msg),\n            ok\n    end,\n    DhtNodeState.\n\n%% @doc Sorts all provided operations so that the one with the biggest change\n%%      of the standard deviation is at the front.\n%%      Note: this is executed at the joining node.\n-spec sort_candidates(Ops::[lb_op:lb_op()]) -> [lb_op:lb_op()].\nsort_candidates(Ops) -> lb_common:bestStddev(util:shuffle(Ops), plus_infinity).\n\n-spec process_join_msg(comm:message(), State::any(),\n        DhtNodeState::dht_node_state:state()) -> unknown_event.\nprocess_join_msg(_Msg, _State, _DhtNodeState) ->\n    unknown_event.\n\n%% @doc Checks whether config parameters of the passive load balancing\n%%      algorithm exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(lb_psv_samples) and\n    config:cfg_is_greater_than_equal(lb_psv_samples, 1).\n\n%% @doc Gets the number of nodes to sample (set in the config files).\n-spec conf_get_number_of_samples() -> pos_integer().\nconf_get_number_of_samples() ->\n    config:read(lb_psv_samples).\n"
  },
  {
    "path": "src/lb_psv_split.erl",
    "content": "%  @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Simple passive load balancing sampling k nodes and choosing the\n%%         one that reduces the standard deviation the most.\n%%         Splits loads in half, if there is no load address ranges are split\n%%         according to lb_psv_split_fallback, i.e. in halves or using the\n%%         selected keys.\n%% @end\n%% @version $Id$\n-module(lb_psv_split).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-define(TRACE_SEND(Pid, Msg), ?TRACE(\"[ ~.0p ] to ~.0p: ~.0p)~n\", [self(), Pid, Msg])).\n-define(TRACE1(Msg, State), ?TRACE(\"[ ~.0p ]~n  Msg: ~.0p~n  State: ~.0p)~n\", [self(), Msg, State])).\n\n-behaviour(lb_psv_beh).\n-include(\"lb_psv_beh.hrl\").\n\n-export_type([custom_message/0]).\n\n-export([my_sort_fun/2]).\n\n-type custom_message() ::\n          {get_split_key_response, SplitKey::{?RT:key(), TakenLoad::non_neg_integer()}}.\n\n%% @doc Gets the number of IDs to sample during join.\n%%      Note: this is executed at the joining node.\n-spec get_number_of_samples(Conn::dht_node_join:connection()) -> ok.\nget_number_of_samples(Connection) ->\n    comm:send_local(self(), {join, get_number_of_samples,\n                             conf_get_number_of_samples(), Connection}).\n\n%% @doc Sends the number of IDs to sample during join to the joining node.\n%%      Note: this is executed at the existing node.\n-spec get_number_of_samples_remote(\n        SourcePid::comm:mypid(), Conn::dht_node_join:connection()) -> ok.\nget_number_of_samples_remote(SourcePid, Connection) ->\n    comm:send(SourcePid, {join, get_number_of_samples,\n                          conf_get_number_of_samples(), Connection}).\n\n%% @doc Creates a join operation if a node would join at my node with the\n%%      given key. This will simulate the join operation and return a lb_op()\n%%      with all the data needed in sort_candidates/1.\n%%      Note: this is executed at an existing node.\n-spec create_join(DhtNodeState::dht_node_state:state(), SelectedKey::?RT:key(),\n                  SourcePid::comm:mypid(), Conn::dht_node_join:connection())\n        -> dht_node_state:state().\ncreate_join(DhtNodeState, SelectedKey, SourcePid, Conn) ->\n    case dht_node_state:get(DhtNodeState, slide_pred) of\n        null ->\n            Neighbors = dht_node_state:get(DhtNodeState, neighbors),\n            MyNodeId = nodelist:nodeid(Neighbors),\n            try\n                MyLoad = dht_node_state:get(DhtNodeState, load),\n                if MyLoad >= 2 ->\n%%                        log:pal(\"[ ~.0p ] trying split by load\", [self()]),\n                       TargetLoad = util:floor(MyLoad / 2),\n%%                        log:pal(\"T: ~.0p, My: ~.0p~n\", [TargetLoad, MyLoad]),\n                       MyPredId = node:id(nodelist:pred(Neighbors)),\n                       SPid = comm:reply_as(self(), 3,\n                                            {join, ?MODULE, '_',\n                                             {create_join2, SelectedKey, SourcePid, Conn}}),\n                       DBCache = pid_groups:get_my(dht_node_db_cache),\n                       \n                       Msg = {get_split_key, dht_node_state:get(DhtNodeState, db),\n                              dht_node_state:get(DhtNodeState, full_range), MyPredId, MyNodeId,\n                              TargetLoad, forward, SPid},\n                       ?TRACE_SEND(DBCache, Msg),\n                       comm:send_local(DBCache, Msg),\n                       DhtNodeState;\n                   true -> % fall-back\n                       create_join2(DhtNodeState, SelectedKey, SourcePid, {MyNodeId, 0}, Conn)\n                end\n            catch\n                Error:Reason -> % fall-back\n                    log:log(error, \"[ Node ~w ] failed to get split key \"\n                                \"for another node: ~.0p:~.0p~n\"\n                                \"  SelectedKey: ~.0p, SourcePid: ~.0p~n  State: ~.0p\",\n                            [self(), Error, Reason, SelectedKey, SourcePid, DhtNodeState]),\n                    create_join2(DhtNodeState, SelectedKey, SourcePid, {MyNodeId, 0}, Conn)\n            end;\n        _ ->\n            % postpone message:\n            Msg = {join, get_candidate, SourcePid, SelectedKey, ?MODULE, Conn},\n            ?TRACE_SEND(SourcePid, Msg),\n            _ = comm:send_local_after(100, self(), Msg),\n            DhtNodeState\n    end.\n\n-spec create_join2(DhtNodeState::dht_node_state:state(), SelectedKey::?RT:key(),\n                   SourcePid::comm:mypid(),\n                   SplitKey::{?RT:key(), TakenLoad::non_neg_integer()},\n                   Conn::dht_node_join:connection()) -> dht_node_state:state().\ncreate_join2(DhtNodeState, SelectedKey, SourcePid, SplitKey0, Conn) ->\n    case dht_node_state:get(DhtNodeState, slide_pred) of\n        null ->\n            Candidate =\n                try\n                    MyNode = dht_node_state:get(DhtNodeState, node),\n                    MyNodeId = node:id(MyNode),\n                    {SplitKey, OtherLoadNew} =\n                        case SplitKey0 of\n                            {MyNodeId, _TargetLoadNew} -> % fall-back\n%%                                 log:log(info, \"[ Node ~w ] could not split load - no key in my range, \"\n%%                                             \"splitting address range instead\", [self()]),\n                                lb_common:split_my_range(DhtNodeState, SelectedKey);\n                            X -> X\n                        end,\n                    MyPredId = node:id(dht_node_state:get(DhtNodeState, pred)),\n                    case SplitKey of\n                        MyNodeId ->\n                            log:log(warn, \"[ Node ~w ] join requested for my ID (~.0p), \"\n                                        \"sending no_op...\", [self(), MyNodeId]),\n                            lb_op:no_op();\n                        MyPredId ->\n                            log:log(warn, \"[ Node ~w ] join requested for my pred's ID (~.0p), \"\n                                        \"sending no_op...\", [self(), MyPredId]),\n                            lb_op:no_op();\n                        _ ->\n%%                             log:pal(\"MK: ~.0p, SK: ~.0p~n\", [MyNodeId, SplitKey]),\n                            MyLoad = dht_node_state:get(DhtNodeState, load),\n                            MyLoadNew = MyLoad - OtherLoadNew,\n                            MyNodeDetails1 = node_details:set(node_details:new(), node, MyNode),\n                            MyNodeDetails = node_details:set(MyNodeDetails1, load, MyLoad),\n                            MyNodeDetailsNew = node_details:set(MyNodeDetails, load, MyLoadNew),\n                            OtherNodeDetails = node_details:set(node_details:new(), load, 0),\n                            OtherNodeDetailsNew1 = node_details:set(node_details:new(), new_key, SplitKey),\n                            OtherNodeDetailsNew2 = node_details:set(OtherNodeDetailsNew1, load, OtherLoadNew),\n                            Interval = node:mk_interval_between_ids(MyPredId, SplitKey),\n                            OtherNodeDetailsNew = node_details:set(OtherNodeDetailsNew2, my_range, Interval),\n                            lb_op:slide_op(comm:this(), OtherNodeDetails, MyNodeDetails,\n                                           OtherNodeDetailsNew, MyNodeDetailsNew)\n                    end\n                catch\n                    Error:Reason ->\n                        log:log(error, \"[ Node ~w ] could not create a candidate \"\n                                    \"for another node: ~.0p:~.0p~n\"\n                                    \"  SelectedKey: ~.0p, SourcePid: ~.0p~n  State: ~.0p\",\n                                [self(), Error, Reason, SelectedKey, SourcePid, DhtNodeState]),\n                        lb_op:no_op()\n                end,\n            Msg = {join, get_candidate_response, SelectedKey, Candidate, Conn},\n            ?TRACE_SEND(SourcePid, Msg),\n            comm:send(SourcePid, Msg);\n        _ ->\n            % re-start with create_join (it can then create a new up-to-date request for a split key):\n            Msg = {join, get_candidate, SourcePid, SelectedKey, ?MODULE, Conn},\n            ?TRACE_SEND(SourcePid, Msg),\n            _ = comm:send_local_after(100, self(), Msg),\n            ok\n    end,\n    DhtNodeState.\n\n%% @doc Sort function for two operations and their Sum2Change.\n%%      Op1 will be preferred over Op2, i.e. Op1 is smaller than Op2, if its\n%%      Sum2Change is smaller or if it is equal and its new node's address\n%%      space is larger.\n-spec my_sort_fun(Op1::{lb_op:lb_op(), integer()},\n                  Op2::{lb_op:lb_op(), integer()}) -> boolean().\nmy_sort_fun({Op1, Op1Change}, {Op2, Op2Change}) ->\n    if Op1Change < Op2Change -> true;\n       true ->\n           case config:read(lb_psv_split_fallback) of\n               split_address when Op1Change =:= Op2Change ->\n                   Op1NewInterval = node_details:get(lb_op:get(Op1, n1_new), my_range),\n                   {_, Op1NewPredId, Op1NewMyId, _} =\n                       intervals:get_bounds(Op1NewInterval),\n                   Op1NewRange = try ?RT:get_range(Op1NewPredId, Op1NewMyId)\n                                 catch throw:not_supported -> 0\n                                 end,\n                   Op2NewInterval = node_details:get(lb_op:get(Op2, n1_new), my_range),\n                   {_, Op2NewPredId, Op2NewMyId, _} =\n                       intervals:get_bounds(Op2NewInterval),\n                   Op2NewRange = try ?RT:get_range(Op2NewPredId, Op2NewMyId)\n                                 catch throw:not_supported -> 0\n                                 end,\n                   Op2NewRange =< Op1NewRange;\n               split_address -> false;\n               keep_key -> Op1Change =:= Op2Change\n           end\n    end.\n\n%% @doc Sorts all provided operations so that the one with the biggest change\n%%      of the standard deviation is at the front. In case of no load changes,\n%%      the operation with the largest address range at the joining node will\n%%      be at the front.\n%%      Note: this is executed at the joining node.\n-spec sort_candidates(Ops::[lb_op:lb_op()]) -> [lb_op:lb_op()].\nsort_candidates(Ops) ->\n    lb_common:bestStddev(util:shuffle(Ops), plus_infinity, fun my_sort_fun/2).\n\n-spec process_join_msg(comm:message(), State::any(),\n        DhtNodeState::dht_node_state:state()) -> unknown_event.\nprocess_join_msg({get_split_key_response, SplitKey},\n                 {create_join2, SelectedKey, SourcePid, Conn}, DhtNodeState) ->\n    create_join2(DhtNodeState, SelectedKey, SourcePid, SplitKey, Conn);\nprocess_join_msg(_Msg, _State, _DhtNodeState) ->\n    unknown_event.\n\n%% @doc Checks whether config parameters of the passive load balancing\n%%      algorithm exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(lb_psv_samples) and\n    config:cfg_is_greater_than_equal(lb_psv_samples, 1) and\n    config:cfg_is_in(lb_psv_split_fallback, [split_address, keep_key]).\n\n%% @doc Gets the number of nodes to sample (set in the config files).\n-spec conf_get_number_of_samples() -> pos_integer().\nconf_get_number_of_samples() ->\n    config:read(lb_psv_samples).\n"
  },
  {
    "path": "src/lb_stats.erl",
    "content": "%  @copyright 2014-2015 Zuse Institute Berlin\n%\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maximilian Michels <michels@zib.de>\n%% @doc Active load balancing stats module which implements collecting\n%%      and accessing stats for the lb_active module.\n%% @version $Id$\n-module(lb_stats).\n-author('michels@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n%-define(TRACE(X,Y), ok).\n-define(TRACE(X,Y), io:format(\"lb_stats: \" ++ X, Y)).\n\n%% get split key based on the request histogram\n-export([get_request_histogram_split_key/3]).\n\n%% for db monitoring\n-export([init/0, init_db_histogram/1, update_db_histogram/2, update_db_monitor/2]).\n-export([set_ignore_db_requests/1, get_ignore_db_requests/0]).\n%% Metrics\n-export([get_load_metric/0, get_request_metric/0, default_value/1]).\n%% Triggered by lb_active\n-export([trigger/0, trigger_routine/0]).\n%% config checked by lb_active\n-export([check_config/0]).\n\n-export_type([load/0]).\n\n-type load() :: number().\n\n-type load_metric() :: cpu | mem | reductions.\n-type request_metric() :: db_histogram.\n\n%% possible metrics\n% items, cpu, mem, db_reads, db_writes, db_requests,\n% transactions, tx_latency, net_throughput, net_latency\n%% available metrics\n-define(LOAD_METRICS, [cpu, mem, reductions]).\n-define(REQUEST_METRICS, [db_reads, db_writes, db_all]).\n\n%%%%%%%%%%%%%%%%%%%%%%%% Monitoring values %%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec init() -> ok.\ninit() ->\n    case collect_stats() of\n        true ->\n            %% cpu_sup not available on all OSs\n            catch cpu_sup:util(), %% throw away first util value\n            {InitialReductions, Timestamp} = get_reductions(),\n            set_last_reductions(InitialReductions, Timestamp),\n            ResolutionSecs = config:read(lb_active_monitor_resolution) div 1000,\n            {MegaSecs, Secs, _Microsecs} = os:timestamp(),\n            %% synchronize the start time for all monitors to a divisible of the monitor interval\n            StartTime = {MegaSecs, Secs - Secs rem ResolutionSecs + ResolutionSecs, 0},\n            % only store newest value in rrd, monitor stores more values\n            RRD = rrd:create(ResolutionSecs * 1000000, 1, gauge, StartTime),\n            monitor:client_monitor_set_value(lb_active, cpu, RRD),\n            monitor:client_monitor_set_value(lb_active, mem, RRD),\n            monitor:monitor_set_value(lb_active, reductions, RRD);\n        _ ->\n            ok\n    end.\n\n-spec trigger_routine() -> ok.\ntrigger_routine() ->\n    trigger(),\n    %% cpu_sup not available on all OSs\n    CPU = try\n              erlang:round(cpu_sup:util())\n          catch\n              _ -> 0\n              %% can we use the avg1 instead, somehow?\n              %% _ -> erlang:round(cpu_sup:avg1())\n          end,\n\n    %% actual keys memsup returns depends on the OS\n    MemInfo = memsup:get_system_memory_data(),\n    {free_memory, FreeMemory} = lists:keyfind(free_memory, 1, MemInfo),\n    {total_memory, TotalMemory} = lists:keyfind(total_memory, 1, MemInfo),\n\n    MEM = FreeMemory / TotalMemory * 100,\n    {NewReductions, NewTimestamp} = get_reductions(),\n    {OldReductions, OldTimestamp} = get_last_reductions(),\n    TimeDiff = timer:now_diff(NewTimestamp, OldTimestamp) div 1000000,\n    if TimeDiff > 0 -> % let at least one second pass\n           ReductionsPerSec = (NewReductions - OldReductions) div TimeDiff,\n           set_last_reductions(NewReductions, NewTimestamp),\n           DhtNodeMonitor = pid_groups:get_my(dht_node_monitor),\n           comm:send_local(DhtNodeMonitor, {db_report}),\n           monitor:monitor_set_value(lb_active, reductions, fun(Old) -> rrd:add(NewTimestamp, ReductionsPerSec, Old) end);\n       true -> ok\n    end,\n    monitor:client_monitor_set_value(lb_active, cpu, fun(Old) -> rrd:add(NewTimestamp, CPU, Old) end),\n    monitor:client_monitor_set_value(lb_active, mem, fun(Old) -> rrd:add(NewTimestamp, MEM, Old) end).\n\n-compile({inline, [update_db_monitor/2]}).\n%% @doc Updates the local rrd for reads or writes and checks for reporting\n-spec update_db_monitor(Type::db_reads | db_writes, Key::?RT:key()) -> ok.\nupdate_db_monitor(Type, Key) ->\n    case lb_active:is_enabled() andalso not get_ignore_db_requests() andalso\n             (config:read(lb_active_request_metric) =:= Type orelse\n                  config:read(lb_active_request_metric) =:= db_all) of\n        true ->\n            DhtNodeMonitor = pid_groups:get_my(dht_node_monitor),\n            comm:send_local(DhtNodeMonitor, {db_op, Key});\n        _ -> ok\n    end.\n\n-compile({inline, [init_db_histogram/1]}).\n%% @doc Called by dht node process to initialize the db monitors\n-spec init_db_histogram(PredId::?RT:key()) -> rrd:rrd().\ninit_db_histogram(PredId) ->\n    HistogramSize = config:read(lb_active_histogram_size),\n    HistogramType = {histogram_rt, HistogramSize, PredId},\n    MonitorResSecs = config:read(lb_active_monitor_resolution) div 1000,\n    {MegaSecs, Secs, _Microsecs} = os:timestamp(),\n    %% synchronize the start time for all monitors to a divisible of the monitor interval\n    StartTime = {MegaSecs, Secs - Secs rem MonitorResSecs + MonitorResSecs, 0},\n    % only store newest value in rrd, monitor stores more values\n    RRD  = rrd:create(MonitorResSecs*1000000, 1, HistogramType, StartTime),\n    Monitor = pid_groups:get_my(monitor),\n    monitor:clear_rrds(Monitor, [{lb_active, db_histogram}]),\n    monitor:monitor_set_value(lb_active, db_histogram, RRD),\n    RRD.\n\n-compile({inline, [update_db_histogram/2]}).\n%% @doc Updates the local rrd for reads or writes and checks for reporting\n-spec update_db_histogram(Key::?RT:key(), OldHistogram::rrd:rrd()) -> rrd:rrd().\nupdate_db_histogram(Key, OldHistogram) ->\n    NewHistogram = %% only update the histogram if necessary\n        case config:read(lb_active_balance) of\n            requests ->\n                NewHist = rrd:add_now(Key, OldHistogram),\n                monitor:check_report(lb_active, db_histogram, OldHistogram, NewHist),\n                NewHist;\n            _ -> OldHistogram\n        end,\n    NewHistogram.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%     Metrics       %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_load_metric() -> unknown | load().\nget_load_metric() ->\n    Metric = config:read(lb_active_load_metric),\n    Value = case get_load_metric(Metric) of\n                unknown -> unknown;\n                Val     -> erlang:round(Val)\n            end,\n    Value.\n\n-spec get_load_metric(load_metric()) -> unknown | load().\nget_load_metric(cpu)        -> get_vm_metric(cpu);\nget_load_metric(reductions) -> get_dht_metric(reductions);\nget_load_metric(mem)        -> get_vm_metric(mem);\nget_load_metric(_)          -> throw(metric_not_available).\n\n-spec get_request_metric() -> integer().\nget_request_metric() ->\n    case get_dht_metric(db_histogram) of\n        unknown -> unknown;\n        Val -> erlang:round(Val)\n    end.\n\n-spec get_vm_metric(load_metric()) -> unknown | load().\nget_vm_metric(Metric) ->\n    ClientMonitorPid = pid_groups:pid_of(clients_group, monitor),\n    get_metric(ClientMonitorPid, Metric).\n\n-spec get_dht_metric(load_metric() | request_metric()) -> unknown | load().\nget_dht_metric(Metric) ->\n    MonitorPid = pid_groups:get_my(monitor),\n    get_metric(MonitorPid, Metric).\n\n-spec get_metric(pid(), load_metric() | request_metric()) -> unknown | load().\nget_metric(MonitorPid, Metric) ->\n    case monitor:get_rrds(MonitorPid, [{lb_active, Metric}]) of\n        [{_Process, _Key, undefined}] ->\n            unknown;\n        [{_Process, _Key, RRD}] ->\n            RRDVals = rrd:get_all_values(asc, RRD),\n            %% max number of consecutive non-undefined values at the end of RRDVals\n            NumDefined = lists:foldl(fun(undefined, _Acc) -> 0;\n                                        (_, Acc)          -> Acc + 1\n                                     end, 0, RRDVals),\n            NumNeeded = config:read(lb_active_monitor_history_min),\n            if NumDefined >= NumNeeded ->\n                   NumValues = rrd:get_count(RRD),\n                   DefinedVals = lists:sublist(RRDVals, NumValues - NumDefined + 1, NumDefined),\n                   Type = rrd:get_type(RRD),\n                   UnpackedVals = lists:map(fun(Val) -> get_value_type(Val, Type) end, DefinedVals),\n                   avg_weighted(UnpackedVals);\n               true -> unknown\n            end\n    end.\n\n-spec get_value_type(RRD::rrd:data_type(), Type::rrd:timeseries_type()) -> unknown | number().\nget_value_type(undefined, _Type) ->\n    unknown;\nget_value_type(Value, _Type) when is_number(Value) ->\n    Value;\nget_value_type(Value, {histogram_rt, _Size, _BaseKey}) ->\n    histogram_rt:get_num_inserts(Value).\n\n%% @doc returns the weighted average of a list using an increasing weight\n-spec avg_weighted([number()]) -> number().\navg_weighted([]) ->\n    0;\navg_weighted(List) ->\n    avg_weighted(List, _Weight=1, _Normalize=0, _Sum=0).\n\n%% @doc returns the weighted average of a list using increasing weight\n-spec avg_weighted([number(),...], Weight::pos_integer(), Normalize::non_neg_integer(), Sum::number()) -> float().\navg_weighted([], _Weight, N, Sum) ->\n    Sum/N;\navg_weighted([Element | Other], Weight, N, Sum) ->\n    %% linear weight\n    CurrentWeight = Weight,\n    %% quadratic weight\n    %CurrentWeight = Weight * Weight,\n    %% exponential weight\n    %CurrentWeight = util:pow(2, Weight),\n    avg_weighted(Other, Weight + 1, N + CurrentWeight, Sum + CurrentWeight * Element).\n\n%% @doc returns a split key from the request histogram at a given time (if available)\n-spec get_request_histogram_split_key(TargetLoad::pos_integer(),\n                                      Direction::forward | backward,\n                                      Items::non_neg_integer())\n        -> {?RT:key(), TakenLoad::non_neg_integer()} | failed.\nget_request_histogram_split_key(TargetLoad, Direction, Items) ->\n    MonitorPid = pid_groups:get_my(monitor),\n    case monitor:get_rrds(MonitorPid, [{lb_active, db_histogram}]) of\n        [{_Process, _Key, undefined}] ->\n            ?TRACE(\"No request histogram available because rrd is undefined.~n\", []),\n            failed;\n        [{_Process, _Key, RRD}] ->\n            %% values from oldest to newest\n            AllValues = rrd:get_all_values(asc, RRD),\n            NumNeeded = config:read(lb_active_monitor_history_min),\n            %% check if we have enough recent values\n            NumDefined = lists:foldl(fun(El, Acc) ->\n                                             if El =:= undefined -> 0;\n                                                true -> Acc + 1\n                                             end\n                                     end, 0, AllValues),\n            if NumDefined >= NumNeeded ->\n                   ?TRACE(\"Got ~p histograms to compute split key~n\", [NumDefined]),\n                   %% merge all histograms with weight (the older the lower the weight)\n                   NumAll = rrd:get_count(RRD),\n                   Values = lists:sublist(AllValues, NumAll - NumDefined + 1, NumDefined),\n                   {AllHists, _} = lists:foldl(\n                                     fun(Hist, {AccHist, Weight}) ->\n                                             {histogram_rt:merge_weighted(AccHist, Hist, Weight), Weight + 1}\n                                     end, {hd(Values), 2}, tl(Values)),\n                   %% normalize afterwards\n                   Histogram = histogram_rt:normalize_count((NumDefined*(NumDefined+1)) div 2, AllHists),\n                   % check if enough requests have been inserted into the histogram\n                   EntriesAvailable = histogram_rt:get_num_inserts(Histogram),\n                   Confidence = config:read(lb_active_request_confidence),\n                   if Items =:= 0 orelse EntriesAvailable / Items < Confidence ->\n                          ?TRACE(\"Confidence too low (below ~p) for request balancing~n\", [Confidence]),\n                          failed;\n                      true ->\n                          {Status, Key, TakenLoad} =\n                              case Direction of\n                                  forward -> histogram_rt:foldl_until(TargetLoad, Histogram);\n                                  backward -> histogram_rt:foldr_until(TargetLoad, Histogram)\n                              end,\n                          case Status of\n                              fail -> failed;\n                              ok -> {Key, TakenLoad}\n                          end\n                   end;\n               true ->\n                   ?TRACE(\"Not enough request histograms available: ~p~n\", [NumDefined]),\n                   failed\n            end\n    end.\n\n%% @doc get the reductions of all processes in the pid group\n-spec get_reductions() -> {non_neg_integer(), erlang_timestamp()}.\nget_reductions() ->\n    MyGroupPids = pid_groups:my_members(),\n    AllReductions =\n        [begin\n             {reductions, N} = erlang:process_info(Pid, reductions),\n             N\n         end || Pid <- MyGroupPids, Pid =/= self()],\n    {lists:sum(AllReductions), os:timestamp()}.\n\n-spec set_last_reductions(non_neg_integer(), erlang_timestamp()) -> ok.\nset_last_reductions(Reductions, Timestamp) ->\n    erlang:put(reductions, {Reductions, Timestamp}),\n    ok.\n\n-spec get_last_reductions() -> {non_neg_integer(), erlang_timestamp()}.\nget_last_reductions() ->\n    erlang:get(reductions).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%% Util %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Sets a default value if the value is unknown\n-spec default_value(Val::unknown | number()) -> number().\ndefault_value(unknown) -> 0;\ndefault_value(Val)     -> Val.\n\n-compile({inline, [set_ignore_db_requests/1, get_ignore_db_requests/0]}).\n%% @doc Sets an indicator for lb_stats to stop monitoring requests during slides\n-spec set_ignore_db_requests(boolean()) -> ok.\nset_ignore_db_requests(Bool) -> erlang:put(ignore_db_requests, Bool), ok.\n\n%% @doc Flag for the dht_node process to check if the current message is a slide message\n-spec get_ignore_db_requests() -> boolean().\nget_ignore_db_requests() -> erlang:get(ignore_db_requests) =:= true.\n\n-spec collect_stats() -> boolean().\ncollect_stats() ->\n    lists:member(config:read(lb_active_load_metric), ?LOAD_METRICS).\n\n-spec trigger() -> ok.\ntrigger() ->\n    Interval = config:read(lb_active_monitor_interval) div 1000,\n    msg_delay:send_trigger(Interval, {lb_stats_trigger}).\n\n\n%% @doc config check by lb_active module\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_in(lb_active_load_metric, ?LOAD_METRICS) and\n\n    config:cfg_is_in(lb_active_request_metric, ?REQUEST_METRICS) and\n\n    config:cfg_is_integer(lb_active_histogram_size) and\n    config:cfg_is_greater_than(lb_active_histogram_size, 0) and\n\n    config:cfg_is_integer(lb_active_monitor_resolution) and\n    config:cfg_is_greater_than(lb_active_monitor_resolution, 0) and\n\n    config:cfg_is_integer(lb_active_monitor_interval) and\n    config:cfg_is_greater_than(lb_active_monitor_interval, 0) and\n\n    config:cfg_is_less_than(lb_active_monitor_interval, config:read(lb_active_monitor_resolution)) and\n\n    config:cfg_is_integer(lb_active_monitor_history_min) and\n    config:cfg_is_greater_than(lb_active_monitor_history_min, 0) and\n\n    config:cfg_is_integer(lb_active_monitor_history_max) and\n    config:cfg_is_greater_than_equal(lb_active_monitor_history_max, config:read(lb_active_monitor_history_min)) and\n\n    config:cfg_is_float(lb_active_request_confidence) and\n    config:cfg_is_greater_than_equal(lb_active_request_confidence, 0.0).\n"
  },
  {
    "path": "src/log.erl",
    "content": "% @copyright 2007-2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc log service\n%% @version $Id$\n-module(log).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"log4erl.hrl\").\n\n-export([start_link/0]).\n-export([log/1, log/2, log/3, log/4, set_log_level/1]).\n-export([pal/1, pal/2]).\n-export([check_config/0]).\n\n-export_type([log_level/0]).\n\n-type log_level() :: debug | info | warn | error | fatal.\n\n%% @doc Starts the log4erl process, removes the error_logger and\n%%      error_logger_file_h report handlers and registers itself as the (only)\n%%      report handler. Note: requires a running config process and can only be\n%%      run once!\n-spec start_link() -> {ok, Pid::pid()}.\nstart_link() ->\n    Link = log4erl:start(log4erl, []),\n    case Link of\n        {ok, _} ->\n            LogLevel = config:read(log_level),\n            LogLevelFile = config:read(log_level_file),\n            % determine lowest log level, adapt cut-off level accordingly\n            case log4erl_utils:to_log(LogLevel, LogLevelFile) of\n                true  -> % LogLevel >= LogLevelFile\n                    log_filter_codegen:set_cutoff_level(LogLevelFile);\n                false -> % LogLevel < LogLevelFile\n                    log_filter_codegen:set_cutoff_level(LogLevel)\n            end,\n            case util:is_unittest() of\n                true ->\n                    log4erl:add_appender(?DEFAULT_LOGGER,\n                                         {log4erl_ctpal_appender, ctpal},\n                                         {LogLevel,\n                                          config:read(log_format)});\n                _ ->\n                    log4erl:add_console_appender(stdout,\n                                                 {LogLevel,\n                                                  config:read(log_format)})\n            end,\n            ErrorLoggerFile = filename:join(config:read(log_path),\n                                            config:read(log_file_name_log4erl)),\n            ok = filelib:ensure_dir(ErrorLoggerFile),\n            log4erl:add_file_appender(file, {config:read(log_path),\n                                             config:read(log_file_name_log4erl),\n                                             {size, config:read(log_file_size)},\n                                             config:read(log_file_rotations),\n                                             \"txt\",\n                                             LogLevelFile}),\n\n%%             log4erl:change_format(stdout, config:read(log_format)),\n            log4erl:change_format(file, config:read(log_format_file)),\n\n            % remove the default error_logger's file and tty handlers\n            _ = error_logger:logfile(close),\n            error_logger:tty(false),\n            error_logger:delete_report_handler(error_logger),\n            % there should not be any previous log4erl handler - just in case, delete it:\n            error_logger:delete_report_handler(error_logger_log4erl_h),\n            % erlang >= R15B adds cth_log_redirect in common_test, delete it:\n            error_logger:delete_report_handler(cth_log_redirect),\n            % now register log4erl error logger:\n            log4erl:error_logger_handler(),\n            %% check whether erlang error_logger only reports to log4erl\n            case gen_event:which_handlers(error_logger) of\n                [error_logger_log4erl_h] -> ok;\n                Loggers -> log(info, \"additional error loggers installed: ~.0p\",\n                               [[L || L <- Loggers, L =/= error_logger_log4erl_h]])\n            end,\n            ok\n    end,\n    Link.\n\n-spec log(LogMsg::string()) -> any().\nlog(Log) ->\n    log(Log, []).\n\n-spec log(Level::log_level(), LogMsg::string()) -> any();\n         (LogMsg::string(), Data::list()) -> any().\nlog(Level, Log) when is_atom(Level) ->\n    log4erl:log(Level, Log);\nlog(Log, Data) ->\n    log4erl:log(warn, Log, Data).\n\n-spec log(Level::log_level(), LogMsg::string(), Data::list()) -> any().\nlog(Level, Log, Data) ->\n    log4erl:log(Level, Log, Data).\n\n-spec log(Logger::atom(), Level::log_level(), LogMsg::string(), Data::list()) -> any().\nlog(Logger, Level, Log, Data) ->\n    log4erl:log(Logger, Level, Log, Data).\n\n-spec set_log_level(Level::log_level() | none) -> any().\nset_log_level(Level) ->\n    log4erl:change_log_level(Level).\n\n% for (temporary) debugging, use log:pal/1 and log:pal/2\n-spec pal(LogMsg::string()) -> any().\npal(Log) -> log(Log).\n\n-spec pal(LogMsg::string(), Data::list()) -> any().\npal(Log, Data) -> log(Log, Data).\n\n%% @doc Checks whether config parameters of the log4erl process exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_in(log_level, [warn, info, error, fatal, debug, none]) and\n    config:cfg_is_in(log_level_file, [warn, info, error, fatal, debug, none]) and\n    \n    config:cfg_is_string(log_path) and\n    config:cfg_is_string(log_file_name_log4erl) and\n    \n    config:cfg_is_integer(log_file_size) and\n    config:cfg_is_greater_than(log_file_size, 0) and\n    \n    config:cfg_is_integer(log_file_rotations) and\n    config:cfg_is_greater_than(log_file_rotations, 0) and\n\n    config:cfg_is_string(log_format) and\n    config:cfg_is_string(log_format_file).\n"
  },
  {
    "path": "src/log4erl_ctpal_appender.erl",
    "content": "%% The contents of this file are subject to the Mozilla Public License\r\n%% Version 1.1 (the \"License\"); you may not use this file except in\r\n%% compliance with the License. You may obtain a copy of the License at\r\n%% http://www.mozilla.org/MPL/\r\n%% \r\n%% Software distributed under the License is distributed on an \"AS IS\"\r\n%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the\r\n%% License for the specific language governing rights and limitations\r\n%% under the License.\r\n%% \r\n%% The Original Code is available on https://code.google.com/p/log4erl/ \r\n%% file: console_appender.erl.\r\n%% \r\n%% The Initial Developer of the Original Code is Ahmed Nawras.\r\n\r\n%% @author Ahmed Nawras\r\n%% @author Nico Kruber <kruber@zib.de>\r\n%% @doc Log4erl appender using log:pal/1 instead if io:format/1 based on\r\n%%      console_appender.\r\n%% @version $Id$\r\n-module(log4erl_ctpal_appender).\r\n-author('Ahmed Nawras').\r\n-author('kruber@zib.de').\r\n-vsn('$Id$').\r\n\r\n-include(\"log4erl.hrl\").\r\n\r\n-behaviour(gen_event).\r\n%% gen_event callbacks\r\n-export([init/1, handle_event/2, handle_call/2, \r\n\t handle_info/2, terminate/2, code_change/3]).\r\n\r\n-spec init({conf, Conf::[term()]} | {log:log_level()} | {log:log_level(), string()}) -> {ok, #console_appender{}}.\r\ninit({conf, Conf}) when is_list(Conf) ->\r\n    CL = lists:foldl(fun(X, List) ->\r\n\t\t\t     [proplists:get_value(X,Conf)|List]\r\n\t\t     end,\r\n\t\t     [],\r\n\t\t     [level, format]),\r\n    \r\n    %% in case format doesn't exist\r\n    Res = case hd(CL) of\r\n\t      undefined ->\r\n\t\t  [_|CL2] = CL,\r\n\t\t  lists:reverse(CL2);\r\n\t      _ ->\r\n\t\t  lists:reverse(CL)\r\n\t  end,\r\n    init(list_to_tuple(Res));\r\ninit({Level}) ->\r\n    init({Level, ?DEFAULT_FORMAT});\r\ninit({Level, Format} = _Args) ->\r\n    ?LOG2(\"Initializing console_appender with args =  ~p~n\",[_Args]),\r\n    {ok, Toks} = log_formatter:parse(Format),\r\n    ?LOG2(\"Tokens received is ~p~n\",[Toks]),\r\n    State = #console_appender{level = Level, format = Toks},\r\n    ?LOG2(\"State is ~p~n\",[State]),\r\n    {ok, State}.\r\n\r\n-spec handle_event({change_level, Level::log:log_level()} |\r\n                   {log, #log{}}, State::#console_appender{})\r\n        -> {ok, #console_appender{}}.\r\nhandle_event({change_level, Level}, State) ->\r\n    State2 = State#console_appender{level = Level},\r\n    ?LOG2(\"Changed level to ~p~n\",[Level]),\r\n    {ok, State2};\r\nhandle_event({log,LLog}, State) ->\r\n    ?LOG2(\"handl_event:log = ~p~n\",[LLog]),\r\n    do_log(LLog, State),\r\n    {ok, State}.\r\n\r\n-spec handle_call({change_format, Format::string()} | \r\n                  {change_level, Level::log:log_level()} |\r\n                  term(), State::#console_appender{})\r\n        -> {ok, ok, #console_appender{}}.\r\nhandle_call({change_format, Format}, State) ->\r\n    ?LOG2(\"Old State in console_appender is ~p~n\",[State]),\r\n    {ok, Tokens} = log_formatter:parse(Format),\r\n    ?LOG2(\"Adding format of ~p~n\",[Tokens]),\r\n    State1 = State#console_appender{format=Tokens},\r\n    {ok, ok, State1};\r\nhandle_call({change_level, Level}, State) ->\r\n    State2 = State#console_appender{level = Level},\r\n    ?LOG2(\"Changed level to ~p~n\",[Level]),\r\n    {ok, ok, State2};\r\nhandle_call(_Request, State) ->\r\n    Reply = ok,\r\n    ?LOG2(\"Received unknown request ~p~n\", [_Request]),\r\n    {ok, Reply, State}.\r\n\r\n-spec handle_info(_Info::any(), State::#console_appender{}) -> {ok, #console_appender{}}.\r\nhandle_info(_Info, State) ->\r\n    {ok, State}.\r\n\r\n-spec terminate(Reason::any(), State::#console_appender{}) -> ok.\r\nterminate(_Reason, _State) ->\r\n    ok.\r\n\r\n-spec code_change(OldVsn::any(), State::#console_appender{}, Extra::any()) -> {ok, #console_appender{}}.\r\ncode_change(_OldVsn, State, _Extra) ->\r\n    {ok, State}.\r\n\r\n-spec do_log(Log::#log{}, State::#console_appender{}) -> ok.\r\ndo_log(#log{level = L} = Log,#console_appender{level=Level, format=Format}) ->\r\n    ToLog = log4erl_utils:to_log(L, Level),\r\n    case ToLog of\r\n\ttrue ->\r\n\t    M = log_formatter:format(Log, Format),\r\n\t    ?LOG2(\"console_appender result message is ~s~n\",[M]),\r\n\t    ct:pal(M);\r\n\tfalse ->\r\n\t    ok\r\n    end.\r\n"
  },
  {
    "path": "src/math_pos.erl",
    "content": "%  @copyright 2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Implements mathematical operations on numbers in positional\n%%         notations represented by lists, i.e.\n%%         [1,2,3] with Base 10 equals 1*10^0 + 2*10^-1 + 3*10^-2.\n%%         Note: valid list elements are: 0..(Base-1). \n%% @end\n%% @version $Id$\n-module(math_pos).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-type position_var() :: [non_neg_integer()].\n\n-export([plus/3, minus/3, divide/3,\n         multiply/3, multiply/4,\n         make_same_length/3, make_same_length/4, remove_zeros/3,\n         from_decimal/2, to_decimal/2]).\n\n%% @doc A + B\n-spec plus(A::position_var(), B::position_var(), Base::pos_integer()) -> position_var().\nplus(A, B, Base) ->\n    plus_rev(lists:reverse(A), lists:reverse(B), 0, [], Base).\n\n-spec plus_rev(A_rev::position_var(), B_rev::position_var(),\n        Carry::non_neg_integer(), Sum::position_var(), Base::pos_integer())\n    -> Sum::position_var().\nplus_rev([A1 | A_rev_Rest], [D1 | D_rev_Rest], Carry, Sum, Base) ->\n    S1_new1 = A1 + D1 + Carry,\n    NewCarry = S1_new1 div Base,\n    S1_new = S1_new1 - NewCarry * Base,\n    plus_rev(A_rev_Rest, D_rev_Rest, NewCarry, [S1_new | Sum], Base);\n% note: forget first carry (don't change length of lists)\nplus_rev([], [], _Carry, Sum, _Base) -> Sum.\n\n%% @doc A - B\n-spec minus(A::position_var(), B::position_var(), Base::pos_integer()) -> position_var().\nminus(A, B, Base) ->\n    minus_rev(lists:reverse(A), lists:reverse(B), 0, [], Base).\n  \n-spec minus_rev(A_rev::position_var(), B_rev::position_var(), Carry::non_neg_integer(),\n        Diff::position_var(), Base::pos_integer()) -> Diff::position_var().\nminus_rev([A1 | A_rev_Rest], [B1 | B_rev_Rest], Carry, Diff, Base) ->\n    {CurChar, NewCarry} = case (A1 - Carry - B1) of\n                              X when X >= 0 -> {X, 0};\n                              X when X < (-Base) -> {X + 2 * Base, 2};\n                              X -> {X + Base, 1}\n                          end,\n    minus_rev(A_rev_Rest, B_rev_Rest, NewCarry, [CurChar | Diff], Base);\n% note: forget first carry (only important inside the subtraction)\nminus_rev([], [], _Carry, Diff, _Base) -> Diff.\n\n\n%% @doc A * Factor, if Factor is a non-negative integer cutting off any carry\n%%      forwards.\n-spec multiply(A::position_var(), Factor::non_neg_integer(), Base::pos_integer())\n        -> position_var().\nmultiply(A, Factor, Base) ->\n    element(1, multiply(A, Factor, Base, cutoff)).\n\n%% @doc A * Factor, if Factor is a non-negative integer.\n-spec multiply(A::position_var(), Factor::non_neg_integer(), Base::pos_integer(),\n               Cut::cutoff) -> {Prod::position_var(), Added::0};\n              (A::position_var(), Factor::non_neg_integer(), Base::pos_integer(),\n               Cut::enlarge) -> {Prod::position_var(), Added::non_neg_integer()}.\nmultiply(A = [_|_], 0, _Base, _Cut) ->\n    {lists:duplicate(erlang:length(A), 0), 0};\nmultiply(A = [_|_], 1, _Base, _Cut) ->\n    {A, 0};\nmultiply(A = [_|_], Factor, Base, Cut) when is_integer(Factor) andalso Factor > 0 ->\n    multiply_rev1(lists:reverse(A), Factor, 0, [], Base, Cut, 0);\nmultiply([], _Factor, _Base, _Cut) ->\n    {[], 0}.\n\n-spec multiply_rev1(A_rev::position_var(), Factor::non_neg_integer(),\n                    Carry::non_neg_integer(), Prod::position_var(),\n                    Base::pos_integer(), Cut::cutoff,\n                    Added::0) -> {Prod::position_var(), Added::0};\n                   (A_rev::position_var(), Factor::non_neg_integer(),\n                    Carry::non_neg_integer(), Prod::position_var(),\n                    Base::pos_integer(), Cut::enlarge,\n                    Added::non_neg_integer()) -> {Prod::position_var(), Added::non_neg_integer()}.\nmultiply_rev1([A1 | A_rev_Rest], Factor, Carry, Prod, Base, Cut, Added) ->\n    P1_new1 = A1 * Factor + Carry,\n    NewCarry = P1_new1 div Base,\n    P1_new = P1_new1 - NewCarry * Base,\n    multiply_rev1(A_rev_Rest, Factor, NewCarry, [P1_new | Prod], Base, Cut, Added);\nmultiply_rev1([], _Factor, 0, Prod, _Base, enlarge, Added) ->\n    {Prod, Added};\nmultiply_rev1([], Factor, Carry, Prod, Base, enlarge = Cut, Added) ->\n    % enlarge list length to fit the result\n    NewCarry = Carry div Base,\n    P1_new = Carry - NewCarry * Base,\n    multiply_rev1([], Factor, NewCarry, [P1_new | Prod], Base, Cut, Added + 1);\nmultiply_rev1([], _Factor, _Carry, Prod, _Base, cutoff, 0) ->\n    % forget first carry (don't change length of lists)\n    {Prod, 0}.\n\n%% @doc A / Divisor (with rounding to nearest integer not larger than the\n%%      result in the last component). Divisor must be a positive integer.\n-spec divide(A::position_var(), Divisor::pos_integer(), Base::pos_integer()) -> position_var().\ndivide(A = [_|_], Divisor, Base) when is_integer(Divisor) andalso Divisor > 1 ->\n    divide_helper(A, Divisor, 0, Base);\ndivide(A = [_|_], 1, _Base) -> A;\ndivide([], _Divisor, _Base) -> [].\n\n-spec divide_helper(Diff::position_var(), Divisor::pos_integer(), Carry::non_neg_integer(),\n                   _Base) -> position_var().\ndivide_helper([D1 | DR], Divisor, Carry, Base) ->\n    Diff0 = Carry * Base + D1,\n    Diff2 = Diff0 div Divisor,\n    NewCarry = Diff0 rem Divisor,\n    [Diff2 | divide_helper(DR, Divisor, NewCarry, Base)];\ndivide_helper([], _Divisor, _Carry, _Base) -> [].\n\n%% @doc Bring two lists to the same length by appending or prepending zeros.\n-spec make_same_length(A::position_var(), B::position_var(), AddTo::front | back)\n        -> {A::position_var(), B::position_var(),\n            ALen::non_neg_integer(), BLen::non_neg_integer(),\n            AddedToA::non_neg_integer(), AddedToB::non_neg_integer()}.\nmake_same_length(A, B, AddTo) ->\n    make_same_length(A, B, AddTo, 0).\n\n%% @doc Bring two lists to the same length by appending or prepending at least MinAdd zeros.\n-spec make_same_length(A::position_var(), B::position_var(), AddTo::front | back, MinAdd::non_neg_integer())\n        -> {A::position_var(), B::position_var(),\n            ALen::non_neg_integer(), BLen::non_neg_integer(),\n            AddedToA::non_neg_integer(), AddedToB::non_neg_integer()}.\nmake_same_length(A, B, AddTo, MinAdd) ->\n    A_l = erlang:length(A), B_l = erlang:length(B),\n    MaxLength = erlang:max(A_l, B_l) + MinAdd,\n    AddToALength = MaxLength - A_l, AddToBLength = MaxLength - B_l,\n    AddToA = lists:duplicate(AddToALength, 0),\n    AddToB = lists:duplicate(AddToBLength, 0),\n    case AddTo of\n        back  -> {lists:append(A, AddToA), lists:append(B, AddToB),\n                  A_l, B_l, AddToALength, AddToBLength};\n        front -> {lists:append(AddToA, A), lists:append(AddToB, B),\n                  A_l, B_l, AddToALength, AddToBLength}\n    end.\n\n%% @doc Remove leading or trailing 0's.\n-spec remove_zeros(A::position_var(), RemoveFrom::front | back, MaxToRemove::non_neg_integer() | all)\n        -> A::position_var().\nremove_zeros(A, back, C) -> lists:reverse(remove_zeros_front(lists:reverse(A), C));\nremove_zeros(A, front, C) -> remove_zeros_front(A, C).\n\n-spec remove_zeros_front(A::position_var(), MaxToRemove::non_neg_integer() | all) -> position_var().\nremove_zeros_front(A, 0) -> A;\nremove_zeros_front([0 | R], all) -> remove_zeros_front(R, all);\nremove_zeros_front([0 | R], C) -> remove_zeros_front(R, C - 1);\nremove_zeros_front([_|_] = A, _) -> A;\nremove_zeros_front([], _) -> [].\n\n%% @doc Converts a (decimal, non-negative) integer to a position var with the\n%%      given Base.\n-spec from_decimal(X::non_neg_integer(), Base::pos_integer()) -> position_var().\nfrom_decimal(X, Base) ->\n    from_decimal_(X, Base).\n\nfrom_decimal_(X, Base) when X < Base ->\n    [X];\nfrom_decimal_(X, Base) ->\n    [X rem Base | from_decimal_(X div Base, Base)].\n\n%% @doc Converts a position var with the given Base to a (decimal, non-negative)\n%%      integer.\n-spec to_decimal(X::position_var(), Base::pos_integer()) -> non_neg_integer().\nto_decimal([], _Base) ->\n    0;\nto_decimal(X, Base) ->\n    element(2, lists:foldr(fun(A, {Fac, Result}) ->\n                                   {Fac * Base, Result + A * Fac}\n                           end, {1, 0}, X)).\n"
  },
  {
    "path": "src/mathlib.erl",
    "content": "% @copyright 2007-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Marie Hoffmann <ozymandiaz147@googlemail.com>\n%% TODO change from camel case to prefix_infix_postfix()\n%% @doc Math utility functions.\n%% @version $Id$\n-module(mathlib).\n-author('ozymandiaz147@googlemail.com').\n-vsn('$Id$').\n\n-export([closestPoints/1, euclideanDistance/1, euclideanDistance/2, u/1,\n         vecAdd/2, vecSub/2, vecMult/2, vecWeightedAvg/4, zeros/1, median/1,\n         nearestCentroid/2,\n        aggloClustering/2]).\n\n-export([factorial/1, binomial_coeff/2, gcd/2]).\n\n%% for type_check_SUITE\n-export([binomial_coeff_feeder/2,\n         factorial_feeder/1,\n         zeros_feeder/1]).\n\n-type(vector() :: [number(),...]).\n\n%% @doc Median of an unsorted non-empty list of numbers, i.e. a vector.\n-spec median(vector()) -> number().\nmedian(L) ->\n    L1 = lists:sort(L),\n    N = length(L1),\n    case N rem 2 of\n        1 -> lists:nth(round(N / 2), L1);\n        0 -> (lists:nth(trunc(N / 2), L1) + lists:nth(trunc(N / 2) + 1, L1)) / 2\n    end.\n\n%% @doc Add two vectors X,Y, i.e. X + Y.\n-spec vecAdd(X::vector(), Y::vector()) -> vector().\nvecAdd(X, Y) ->\n    lists:zipwith(fun(Xi, Yi) -> Xi + Yi end, X, Y).\n\n%% @doc Substract two vectors X,Y, i.e. X - Y.\n-spec vecSub(X::vector(), Y::vector()) -> vector().\nvecSub(X, Y) ->\n    lists:zipwith(fun(Xi, Yi) -> Xi - Yi end, X, Y).\n\n%% @doc Multiply vector V with a scalar S.\n-spec vecMult(V::vector(), S::float()) -> vector().\nvecMult(V, S) ->\n    lists:map(fun(X) -> S*X end, V).\n\n-spec vecWeightedAvg(V1::vector(), V2::vector(), W1::float(), W2::float()) -> vector().\nvecWeightedAvg(V1, V2, W1, W2) ->\n    vecMult(vecAdd(vecMult(V1, W1), vecMult(V2, W2)), 1 / (W1 + W2)).\n\n%% @doc Euclidean distance between origin and V.\n-spec euclideanDistance(V::vector()) -> Distance::float().\neuclideanDistance(V) ->\n    math:sqrt(lists:foldl(fun(Vi, OldDist) -> OldDist + math:pow(Vi, 2) end,\n                          0.0, V)).\n\n%% @doc Euclidean distance between two vectors.\n-spec euclideanDistance(V::vector(), W::vector()) -> Distance::float().\neuclideanDistance(V, W) ->\n    math:sqrt(util:zipfoldl(fun(Vi, Wi) -> math:pow(Vi - Wi, 2) end,\n                            fun(Dist, OldDist) -> OldDist + Dist end,\n                            V, W, 0.0)).\n\n%% @doc Unit vector u(v) = v/||v||\n-spec u(V::vector()) -> UV::vector().\nu(V) ->\n    vecMult(V, 1 / euclideanDistance(V)).\n\n%% @doc Get the nearest centroid to U from the list Centroids, including the euclidian\n%  distance. The function returns 'none' if no nearest centroid can be found. Ambiguity is\n%  resolved by picking the first one of the nearest centroids.\n-spec nearestCentroid(U::dc_centroids:centroid(), dc_centroids:centroids()) ->\n    {Distance::float(), NearestCentroid::dc_centroids:centroid()} | none.\nnearestCentroid(_U, []) -> none;\nnearestCentroid(U, [U|T]) -> nearestCentroid(U, T);\nnearestCentroid(U, [X|Centroids]) ->\n    CoordU = dc_centroids:get_coordinate(U),\n    CoordsX = dc_centroids:get_coordinate(X),\n    First = {euclideanDistance(CoordU, CoordsX), X},\n    lists:foldl(fun\n            (C, {CurrentDistance, _CurrentMin} = Current) ->\n                case C of\n                    U -> Current;\n                    _ -> CoordC = dc_centroids:get_coordinate(C),\n                         NewDistance = euclideanDistance(CoordU, CoordC),\n                         case NewDistance < CurrentDistance of\n                             true -> {NewDistance, C};\n                             false -> Current\n                         end\n                end\n        end, First, Centroids).\n\n%% @doc Find indices of closest centroids.\n-spec closestPoints(dc_centroids:centroids())\n    -> {float(), dc_centroids:centroid(), dc_centroids:centroid()} | none.\nclosestPoints([]) -> none;\nclosestPoints([_]) -> none;\nclosestPoints([First, Second|_] = Centroids) ->\n    FirstDist = dc_centroids:distance(First, Second),\n    lists:foldl(\n        fun\n            (Centroid, {CurrentMinDist, _, _} = Acc) ->\n                % get the centroid with minimum distance to Centroid. If this is less than\n                % the distance of the centroids in Acc, exchange\n                case nearestCentroid(Centroid, Centroids) of\n                    none -> Acc;\n                    {Dist, CentroidMin} ->\n                        case Dist < CurrentMinDist of\n                            true -> {Dist, Centroid, CentroidMin};\n                            false -> Acc\n                        end\n                end\n    end, {FirstDist, First, Second}, Centroids).\n\n-spec zeros_feeder(0..10000) -> {0..10000}.\nzeros_feeder(N) -> {N}.\n\n%% @doc Create a list with N zeros.\n-spec zeros(N::0) -> [];\n           (N::pos_integer()) -> [0,...].\nzeros(N) -> lists:duplicate(N, 0).\n\n%% @doc Get closest centroids and merge them if their distance is within Radius.\n-spec aggloClustering(Centroids::dc_centroids:centroids(), Radius::number()) -> dc_centroids:centroids().\naggloClustering(Centroids, Radius) when Radius >= 0 ->\n    case closestPoints(Centroids) of\n        none -> Centroids;\n        {Min, I, J} -> aggloClusteringHelper(Centroids, Radius, Min, I, J)\n    end.\n\n-spec aggloClusteringHelper\n        (Centroids::[dc_centroids:centroid(),...], Radius::number(),\n         Min::float(), I::dc_centroids:centroid(), J::dc_centroids:centroid()) ->\n         dc_centroids:centroids().\n% Note: closestPoints/1 creates Min, I, J and only returns {-1, -1, -1} if\n% Centroids contains less than two elements. This is not the case in the first\n% pattern and we can thus assume these values are pos_integer().\naggloClusteringHelper(Centroids, _Radius, 0.0, _, _) -> Centroids;\naggloClusteringHelper(Centroids, Radius, Min, I, J) when Min =< Radius ->\n    {C1, S1} = dc_centroids:get_coordinate_and_relative_size(I),\n    {C2, S2} = dc_centroids:get_coordinate_and_relative_size(J),\n    NewCoordinate = vecWeightedAvg(C1, C2, S1, S2),\n    NewCentroid = dc_centroids:new(NewCoordinate, S1+S2),\n\n    NewCentroids = [NewCentroid | lists:subtract(Centroids, [I,J])],\n    case closestPoints(NewCentroids) of\n        none -> NewCentroids;\n        {Min1, I1, J1} ->\n            aggloClusteringHelper(NewCentroids, Radius, Min1, I1, J1)\n    end;\naggloClusteringHelper(Centroids, _Radius, _Min, _I, _J) ->\n    Centroids.\n\n% @doc Calculates the binomial coefficient of n over k for n >= k.\n%      see http://rosettacode.org/wiki/Evaluate_binomial_coefficients#Erlang\n-spec binomial_coeff(non_neg_integer(), non_neg_integer()) -> integer().\nbinomial_coeff(_, 0) -> 1;\nbinomial_coeff(N, K) when N >= K ->\n  choose(N, K, 1, 1).\n\n-spec binomial_coeff_feeder(0..100, 0..100) ->\n                                   {non_neg_integer(), non_neg_integer()}.\nbinomial_coeff_feeder(X, Y) ->\n    {erlang:max(X, Y), erlang:min(X, Y)}.\n\n-spec choose(non_neg_integer(), non_neg_integer(),\n             non_neg_integer(), non_neg_integer()) -> non_neg_integer().\nchoose(N, K, K, Acc) ->\n  (Acc * (N-K+1)) div K;\nchoose(N, K, I, Acc) ->\n  choose(N, K, I+1, (Acc * (N-I+1)) div I).\n\n-spec factorial_feeder(0..20) -> {0..20}.\nfactorial_feeder(N) -> {N}.\n\n% @doc calculates N!\n-spec factorial(non_neg_integer()) -> pos_integer().\nfactorial(N) -> factorial(N, 1).\n\n-compile({nowarn_unused_function, {factorial_feeder, 2}}).\n-spec factorial_feeder(0..20, pos_integer()) -> {0..20, pos_integer()}.\nfactorial_feeder(N, Acc) -> {N, Acc}.\n\n-spec factorial(non_neg_integer(), pos_integer()) -> pos_integer().\nfactorial(0, Acc) -> Acc;\nfactorial(N, Acc) ->\n    factorial(N - 1, N * Acc).\n\n%% @doc Calculates the greatest common divisor of two integers.\n-spec gcd(non_neg_integer(), non_neg_integer()) -> non_neg_integer().\ngcd(A, 0) -> A;\ngcd(A, B) -> gcd(B, A rem B).\n"
  },
  {
    "path": "src/mgmt_server.erl",
    "content": "% @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc The management server maintains a list of scalaris nodes and\n%%      checks their availability using a failure_detector. Its main\n%%      purpose is to give new scalaris nodes a list of nodes already\n%%      in the system.\n%% @end\n%% @version $Id$\n-module(mgmt_server).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([start_link/2,\n         number_of_nodes/0,\n         node_list/0, node_list/1]).\n\n-behaviour(gen_component).\n-include(\"scalaris.hrl\").\n\n-export([init/1, on/2]).\n\n-include(\"gen_component.hrl\").\n\n% accepted messages of the mgmt_server process\n-type(message() ::\n    {fd_notify, fd:event(), PID::comm:mypid(), Reason::fd:reason()} |\n    {get_list, SourcePid::comm:mypid()} |\n    {get_list_length, SourcePid::comm:mypid()} |\n    {register, Node::node:node_type()}).\n\n% internal state (known nodes)\n-type(state()::Nodes::gb_trees:tree(comm:mypid(), node:node_type())).\n\n%% @doc trigger a message with the number of nodes known to the mgmt server\n-spec number_of_nodes() -> ok.\nnumber_of_nodes() ->\n    Pid = mgmtPid(),\n    This = comm:this(),\n    case comm:is_valid(Pid) andalso comm:is_valid(This) of\n        true -> comm:send(Pid, {get_list_length, This});\n        _    -> comm:send_local(self(), {get_list_length_response, 0})\n    end.\n\n%% @doc trigger a message with all nodes known to the mgmt server\n-spec node_list() -> ok.\nnode_list() -> node_list(false).\n\n-spec node_list(UseShepherd::boolean()) -> ok.\nnode_list(UseShepherd) ->\n    Pid = mgmtPid(),\n    case comm:is_valid(Pid) of\n        true -> comm:send(Pid, {get_list, comm:this()},\n                          ?IIF(UseShepherd, [{shepherd, self()}], []));\n        _    -> comm:send_local(self(), {get_list_response, []})\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Implementation\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec on(message(), state()) -> state().\non({fd_notify, crash, PID, _Reason}, Nodes) ->\n    case gb_trees:lookup(PID, Nodes) of\n        {value, Node} -> dn_cache:add_zombie_candidate(Node),\n                         gb_trees:delete(PID, Nodes);\n        none          -> Nodes\n    end;\non({fd_notify, leave, PID, _Reason}, Nodes) ->\n    % graceful leave - prevent the node being added as a zombie candidate by removing now:\n    gb_trees:delete_any(PID, Nodes);\non({fd_notify, _Event, _PID, _Reason}, Nodes) ->\n    Nodes;\n\non({get_list, SourcePid}, Nodes) ->\n    comm:send(SourcePid, {get_list_response, gb_trees:keys(Nodes)}),\n    Nodes;\n\non({get_list_length, SourcePid}, Nodes) ->\n    comm:send(SourcePid, {get_list_length_response, gb_trees:size(Nodes)}),\n    Nodes;\n\non({register, Node}, Nodes) ->\n    NodePid = node:pidX(Node),\n    case gb_trees:lookup(NodePid, Nodes) of\n        {value, _OldNode} -> gb_trees:update(NodePid, Node, Nodes);\n        none              -> fd:subscribe(self(), [NodePid]),\n                             gb_trees:insert(NodePid, Node, Nodes)\n    end;\n\n% dead-node-cache reported dead node to be alive again\non({zombie, Node}, Nodes) ->\n    on({register, Node}, Nodes);\n\non({web_debug_info, Requestor}, Nodes) ->\n    RegisteredPids = gb_trees:keys(Nodes),\n    % resolve (local and remote) pids to names:\n    PidNames = pid_groups:pids_to_names(RegisteredPids, 1000),\n    KeyValueList =\n        [{\"registered nodes\", length(RegisteredPids)},\n         {\"registered nodes (node):\", \"\"} |\n         [{\"\", Pid} || Pid <- PidNames]],\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    Nodes.\n\n-spec init(Options::[tuple()]) -> state().\ninit(_Options) ->\n    dn_cache:subscribe(),\n    gb_trees:empty().\n\n%% @doc starts the server; called by the mgmt supervisor\n%% @see sup_scalaris\n-spec start_link(pid_groups:groupname(), [tuple()]) -> {ok, pid()}.\nstart_link(ServiceGroup, Options) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, Options,\n                             [{erlang_register, mgmt_server},\n                              {pid_groups_join_as, ServiceGroup, ?MODULE}]).\n\n%% @doc pid of the mgmt server (may be invalid)\n-spec mgmtPid() -> comm:mypid() | any().\nmgmtPid() ->\n    config:read(mgmt_server).\n"
  },
  {
    "path": "src/modr.erl",
    "content": "%  @copyright 2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    helper module for modr mode\n%% @end\n-module(modr).\n-author('schuett@zib.de').\n\n-include(\"scalaris.hrl\").\n\n-export([is_enabled/0, fix_neighborhood/1, fix_state/1]).\n\n-spec is_enabled() -> boolean().\nis_enabled() ->\n    config:read(key_creator) =:= modr.\n\n-spec fix_neighborhood(Neighbors::nodelist:neighborhood()) ->\n                              nodelist:neighborhood().\nfix_neighborhood(Neighbors) ->\n    Neighbors.\n\n-spec fix_state(State::rm_tman_state:state()) ->\n                              rm_tman_state:state().\nfix_state(State) ->\n    State.\n\n%% notes on rm_tman.erl\n\n%% update_node:\n%%    nodelist:update_node changes basenode, triggered by slide_chord.erl\n%% trigger_update: changes node ids\n%%    nodelist:update_ids\n%% update_nodes:\n%%    nodelist:add_nodes\n%%    nodelist:filter\n%% remove_neighbors_in_interval: changes NodeInterval\n%%    nodelist:filter\n"
  },
  {
    "path": "src/monitor.erl",
    "content": "%  @copyright 2011-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Monitors data from different processes, e.g. for performance\n%%         evaluations.\n%% @end\n%% @version $Id$\n-module(monitor).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n% monitor process functions\n-export([start_link/1, init/1, on/2, check_config/0]).\n\n% functions (temporarily) storing monitoring values in the calling process:\n-export([proc_set_value/3, proc_get_value/2, proc_exists_value/2,\n         proc_clear_value/2, proc_check_timeslot/2, proc_check_all_timeslot/0]).\n\n% functions sending monitoring values directly to the monitor process\n-export([monitor_set_value/3, client_monitor_set_value/3]).\n\n% reporting to the monitor for manually-maintained rrd records\n-export([check_report/4]).\n\n-export([get_rrds/2\n         , clear_rrds/2\n         , get_rrd_keys/0\n         , get_rrd_keys/1\n        ]).\n\n% for monitor_perf:\n-export([web_debug_info_merge_values/2]).\n\n-export_type([key/0, table_index/0]).\n\n-include(\"gen_component.hrl\").\n\n-type key() :: atom().\n-type internal_key() :: {'$monitor$', Process::atom(), Key::string()}.\n-type table_index() :: {Process::atom(), Key::key()}.\n\n-type state() :: {Table::ets:tid() | atom(), ApiTxReqList::rrd:rrd()}.\n-type message() ::\n    {report_rrd, Process::atom(), Key::key(), OldValue::rrd:rrd(), Value::rrd:rrd()} |\n    {report_single, Process::atom(), Key::key(),\n     NewValue_or_UpdateFun::term() | fun((Old::Value | undefined) -> New::Value)} |\n    {check_timeslots} |\n    {web_debug_info, Requestor::comm:erl_local_pid()}.\n\n%% @doc Converts the given Key to avoid conflicts in erlang:put/get.\n-spec to_internal_key(Process::atom(), Key::key()) -> internal_key().\nto_internal_key(Process, Key) -> {'$monitor$', Process, Key}.\n\n-spec check_report(Process::atom(), Key::key(), Old::Value, New::Value) -> ok.\ncheck_report(_Process, _Key, undefined, _NewValue) -> ok;\ncheck_report(_Process, _Key, _OldValue, undefined) -> ok;\ncheck_report(Process, Key, OldValue, NewValue) ->\n    % check whether to report to the monitor\n    % (always report if a new time slot was started)\n    SlotOld = rrd:get_slot_start(0, OldValue),\n    SlotNew = rrd:get_slot_start(0, NewValue),\n    case SlotNew of\n        SlotOld -> ok; %nothing to do\n        _  -> % new slot -> report to monitor:\n            proc_report_to_my_monitor(Process, Key, OldValue, NewValue)\n    end.\n\n%% @doc Keep track of the available keys by adding Key to the list of keys\n%%      stored at '$monitor$:$keys$'.\n-spec proc_add_to_keys_avail(OldValue::term() | undefined, Process::atom(), Key::key()) -> ok.\nproc_add_to_keys_avail(undefined, Process, Key) ->\n    AvailKey = '$monitor$:$keys$',\n    OldKeys = case erlang:get(AvailKey) of\n                  undefined -> [];\n                  L -> L\n              end,\n    erlang:put(AvailKey, [{Process, Key} | OldKeys]),\n    ok;\nproc_add_to_keys_avail(_OldValue, _Process, _Key) ->\n    ok.\n\n%% @doc Sets the value at Key inside the current process.\n%%      Either specify a new value or an update function which generates the\n%%      new value from the old one.\n%%      If a new time slot is started by updating the value, then the rrd()\n%%      record is send to the monitor process.\n-spec proc_set_value(Process::atom(), Key::key(),\n                     NewValue_or_UpdateFun::term() | fun((Old::Value | undefined) -> New::Value)) -> ok.\nproc_set_value(Process, Key, UpdateFun) when is_function(UpdateFun, 1) ->\n    InternalKey = to_internal_key(Process, Key),\n    OldValue = erlang:get(InternalKey),\n    NewValue = UpdateFun(OldValue),\n    proc_add_to_keys_avail(OldValue, Process, Key),\n    check_report(Process, Key, OldValue, NewValue),\n    erlang:put(InternalKey, NewValue);\nproc_set_value(Process, Key, NewValue) ->\n    InternalKey = to_internal_key(Process, Key),\n    OldValue = erlang:put(InternalKey, NewValue),\n    proc_add_to_keys_avail(OldValue, Process, Key),\n    check_report(Process, Key, OldValue, NewValue).\n\n%% @doc Sets the value at Key inside the monitor process of the current group.\n%%      Either specify a new value or an update function which generates the\n%%      new value from the old one.\n-spec monitor_set_value(Process::atom(), Key::key(),\n                        NewValue_or_UpdateFun::term() | fun((Old::Value | undefined) -> New::Value)) -> ok.\nmonitor_set_value(Process, Key, NewValue_or_UpdateFun) ->\n    MyMonitor = pid_groups:get_my(monitor),\n    comm:send_local(MyMonitor, {report_single, Process, Key, NewValue_or_UpdateFun}).\n\n%% @doc Advances the stored timeslots of the value at Key inside the current\n%%      process (if necessary) to the current time.\n%%      If a new time slot is started by updating the value, then the rrd()\n%%      record is send to the monitor process.\n-spec proc_check_timeslot(Process::atom(), Key::key()) -> ok.\nproc_check_timeslot(Process, Key) ->\n    InternalKey = to_internal_key(Process, Key),\n    OldValue = erlang:get(InternalKey),\n    case OldValue of\n        undefined -> ok;\n        _ ->\n            NewValue = rrd:check_timeslot_now(OldValue),\n            check_report(Process, Key, OldValue, NewValue),\n            erlang:put(InternalKey, NewValue),\n            ok\n    end.\n\n%% @doc Advances the stored timeslots of the value at Key inside the current\n%%      process (if necessary) to the current time.\n%%      If a new time slot is started by updating the value, then the rrd()\n%%      record is send to the monitor process.\n-spec proc_check_all_timeslot() -> ok.\nproc_check_all_timeslot() ->\n    case erlang:get('$monitor$:$keys$') of\n        undefined -> ok;\n        AvailableKeys ->\n            _ = [proc_check_timeslot(Process, Key) || {Process, Key} <- AvailableKeys],\n            ok\n    end.\n\n%% @doc Sets the value at Key inside the monitor process of the 'clients_group'.\n%%      Either specify a new value or an update function which generates the\n%%      new value from the old one.\n-spec client_monitor_set_value(Process::atom(), Key::key(),\n                        NewValue_or_UpdateFun::term() | fun((Old::Value | undefined) -> New::Value)) -> ok.\nclient_monitor_set_value(Process, Key, NewValue_or_UpdateFun) ->\n    MyMonitor = pid_groups:pid_of(clients_group, monitor),\n    comm:send_local(MyMonitor, {report_single, Process, Key, NewValue_or_UpdateFun}).\n\n%% @doc Checks whether a value exists at Key.\n-spec proc_exists_value(Process::atom(), Key::key()) -> boolean().\nproc_exists_value(Process, Key) ->\n    erlang:get(to_internal_key(Process, Key)) =/= undefined.\n\n%% @doc Gets the value stored at Key. The key must exist, otherwise no rrd()\n%%      structure is returned!\n-spec proc_get_value(Process::atom(), Key::key()) -> rrd:rrd().\nproc_get_value(Process, Key) ->\n    erlang:get(to_internal_key(Process, Key)).\n\n-spec proc_clear_value(Process::atom(), Key::key()) -> ok.\nproc_clear_value(Process, Key) ->\n    erlang:erase(to_internal_key(Process, Key)),\n    ok.\n\n%% @doc Reports the given value to the process' monitor process.\n-spec proc_report_to_my_monitor(Process::atom(), Key::key(), OldValue::rrd:rrd(), Value::rrd:rrd()) -> ok.\nproc_report_to_my_monitor(Process, Key, OldValue, Value) ->\n    MyMonitor = pid_groups:get_my(monitor),\n    % note: it may happen that the new value created a new slot which already\n    % discarded all logged data from the previous (unreported) time slot\n    % -> send OldValue, too\n    comm:send_local(MyMonitor, {report_rrd, Process, Key, OldValue, Value}).\n\n%% @doc Retrieve individual RRDs from monitor\n-spec get_rrds(MonitorPid::comm:erl_local_pid(), Keys::list(table_index()))\n        -> [{atom(), key(), rrd:rrd() | undefined}].\nget_rrds(MonitorPid, Keys) ->\n    %% we peek into the ets table of the monitor process\n    %% look-up ets tables which the monitor-pid owns\n    Table = case erlang:get({monitor_table, MonitorPid}) of\n                undefined ->\n                    Tables = ets:all(),\n                    OwnedTables = [ X || X <- Tables,\n                                         MonitorPid =:= ets:info(X, owner) ],\n                    case OwnedTables of\n                        [Tab | _] ->\n                            erlang:put({monitor_table, MonitorPid}, Tab),\n                            Tab;\n                        [] ->\n                            % process not ready yet\n                            failed\n                    end;\n                X -> X\n            end,\n\n    case Table of\n        failed ->\n            [if Process =:= api_tx andalso Key =:= 'req_list' ->\n                    %% special case: always return an empty rrd for {api_tx, req_list}\n                    {Process, Key, init_apitx_reqlist_rrd(os:timestamp())};\n                true ->\n                    {Process, Key, undefined}\n             end || {Process, Key} <- Keys];\n        _ ->\n            [case get_rrd(Table, TableIndex) of\n                 undefined when Process =:= api_tx andalso Key =:= 'req_list' ->\n                     %% special case: always return an empty rrd for {api_tx, req_list}\n                     {Process, Key, init_apitx_reqlist_rrd(os:timestamp())};\n                 Value -> {Process, Key, Value}\n             end || {Process, Key} = TableIndex <- Keys]\n    end.\n\n-spec clear_rrds(MonitorPid::comm:erl_local_pid(), Keys::list(table_index())) -> ok.\nclear_rrds(MonitorPid, Keys) ->\n    comm:send_local(MonitorPid, {clear_rrds, Keys}).\n\n%% @doc Message handler when the rm_loop module is fully initialized.\n-spec on(message(), state()) -> state().\non({report_rrd, Process, Key, OldValue, _NewValue}, {Table, _ApiTxReqList} = State) ->\n    % note: reporting is always done when a new time slot is created\n    % -> use the values from the old value\n    TableIndex = {Process, Key},\n    MyData = case ets:lookup(Table, TableIndex) of\n                 [{TableIndex, X}] -> X;\n                 [] ->\n                     SlotLength = rrd:get_slot_length(OldValue),\n                     OldTime = rrd:get_current_time(OldValue),\n                     rrd:create(SlotLength, get_timeslots_to_keep(Process),\n                                rrd:get_type(OldValue),\n                                erlang:max(OldTime, OldTime - SlotLength))\n             end,\n    NewData = rrd:add_nonexisting_timeslots(MyData, OldValue),\n    ets:insert(Table, {TableIndex, NewData}),\n    State;\n\non({report_single, api_tx, 'req_list', TimeInMs}, {Table, OldApiTxReqList}) when is_number(TimeInMs) ->\n    NewApiTxReqList = rrd:add_now(TimeInMs, OldApiTxReqList),\n    check_report(api_tx, 'req_list', OldApiTxReqList, NewApiTxReqList),\n    {Table, NewApiTxReqList};\n\non({report_single, Process, Key, NewValue_or_UpdateFun}, State) ->\n    proc_set_value(Process, Key, NewValue_or_UpdateFun),\n    State;\n\non({trigger_check_timeslots}, State) ->\n    msg_delay:send_trigger(get_check_timeslots_interval(), {trigger_check_timeslots}),\n    gen_component:post_op({check_timeslots}, State);\n\non({check_timeslots}, {_Table, OldApiTxReqList} = State) ->\n    proc_check_all_timeslot(),\n    % manual check for {api_tx, req_list} necessary:\n    NewApiTxReqList = rrd:check_timeslot_now(OldApiTxReqList),\n    check_report(api_tx, 'req_list', OldApiTxReqList, NewApiTxReqList),\n    State;\n\non({get_rrd_keys, SourcePid}, {Table, _} = State) ->\n    Keys = get_all_keys(Table),\n    comm:send_local(SourcePid, {get_rrd_keys, Keys}),\n    State;\n\non({clear_rrds, TableIndexes}, {Table, _ApiTxReqList} = State) ->\n    _ = [begin ets:delete(Table, TableIndex),\n               {Process, Key} = TableIndex,\n               erlang:erase(to_internal_key(Process, Key))\n         end || TableIndex <- TableIndexes],\n    State;\n\non({web_debug_info, Requestor}, {Table, _ApiTxReqList} = State) ->\n    Keys = get_all_keys(Table),\n    GroupedLast5 = [begin\n                        KeyData5 = get_last_n(Table, Key, 5),\n                        web_debug_info_merge_values(Key, KeyData5)\n                    end || Key <- Keys],\n    comm:send_local(Requestor, {web_debug_info_reply, [{\"last 5 records per key:\", \"\"} | GroupedLast5]}),\n    State.\n\n-spec get_rrd(Table::ets:tid() | atom(), TableIndex::table_index()) -> rrd:rrd() | undefined.\nget_rrd(Table, TableIndex) ->\n    case ets:lookup(Table, TableIndex) of\n        [{TableIndex, X}] -> X;\n        [] -> undefined\n    end.\n\n-spec get_all_keys(Table::ets:tid() | atom()) -> [table_index()].\nget_all_keys(Table) ->\n    lists:usort(ets:select(Table, [{ {'$1', '$2'},\n                                     [],     % guard\n                                     ['$1']} % result\n                                  ])).\n\n% @doc Reduces the rrd() data to N time slots (the key _must_ exist in the table!).\n-spec get_last_n(Table::ets:tid() | atom(), Key::table_index(), N::pos_integer())\n        -> Value::rrd:rrd().\nget_last_n(Table, Key, N) ->\n    [{Key, Data}] = ets:lookup(Table, Key),\n    rrd:reduce_timeslots(N, Data).\n\n-spec web_debug_info_dump_fun(rrd:rrd(), From_us::rrd:internal_time(), To::rrd:internal_time(), Value::term())\n        -> {From::util:time_utc(), To::util:time_utc(), Diff_in_s::non_neg_integer(), ValueStr::string()}.\nweb_debug_info_dump_fun(DB, From_us, To_us, Value) ->\n    From = calendar:now_to_universal_time(util:us2timestamp(From_us)),\n    To = calendar:now_to_universal_time(util:us2timestamp(To_us)),\n    Diff_in_s = (To_us - From_us) div 1000000,\n    ValueStr =\n        case rrd:get_type(DB) of\n            {Type, Unit} when Type =:= timing orelse Type =:= timing_with_hist ->\n                {Sum, Sum2, Count, Min, Max, Hist} = Value,\n                AvgPerS = Count / Diff_in_s,\n                Avg = Sum / Count, Avg2 = Sum2 / Count,\n                Stddev = math:sqrt(Avg2 - (Avg * Avg)),\n                UnitStr = case Unit of\n                              count -> \"\";\n                              _     -> \" \" ++ erlang:atom_to_list(Unit)\n                          end,\n                Temp = io_lib:format(\"&nbsp;&nbsp;count: ~B (avg: ~.2f / s), avg: ~.2f~s, min: ~.2f~s, max: ~.2f~s, stddev: ~.2f~s<br />\",\n                              [Count, AvgPerS, Avg, UnitStr, float(Min), UnitStr, float(Max), UnitStr, Stddev, UnitStr]),\n                case histogram:get_num_elements(Hist) of \n                    0 ->\n                        Temp;\n                    _ ->\n                        %Temp ++ io_lib:format(\"&nbsp;&nbsp;hist:<pre>~.0p</pre><br />Largest Window: ~p\", [histogram:get_data(Hist), histogram:find_largest_window(N div 2, Hist)])\n                        Temp ++ io_lib:format(\"&nbsp;&nbsp;hist:<pre>~.0p</pre>\", [histogram:get_data(Hist)])\n                end;\n            counter ->\n                io_lib:format(\"&nbsp;&nbsp;sum: ~p (avg: ~.2f / s)\", [Value, Value / Diff_in_s]);\n            event ->\n                io_lib:format(\"&nbsp;&nbsp;events: ~p (avg: ~.2f / s)\", [Value, length(Value) / Diff_in_s]);\n            _       ->\n                io_lib:format(\"&nbsp;&nbsp;~p\", [Value])\n        end,\n    {From, To, Diff_in_s, lists:flatten(ValueStr)}.\n\n-spec web_debug_info_merge_values(table_index(), rrd:rrd())\n            -> {Key::string(), LastNValues::string()}.\nweb_debug_info_merge_values(Key, Data) ->\n    ValuesLastN =\n        [lists:flatten(io_lib:format(\"~p - ~p UTC (~p s):<br/>~s~n\",\n                                     [From, To, Diff_in_s, ValueStr]))\n         || {From, To, Diff_in_s, ValueStr} <- rrd:dump_with(Data, fun web_debug_info_dump_fun/4)],\n    {lists:flatten(io_lib:format(\"~p\", [Key])), string:join(ValuesLastN, \"<br />\")}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Startup\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Starts the monitor process, registers it with the process dictionary\n%%      and returns its pid for use by a supervisor.\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, null,\n                             [{wait_for_init}, %% uses protected ets table\n                              {pid_groups_join_as, DHTNodeGroup, monitor}]).\n\n%% @doc Initialises the module with an empty state.\n-spec init(null) -> state().\ninit(null) ->\n    msg_delay:send_trigger(get_check_timeslots_interval(), {trigger_check_timeslots}),\n    {ets:new(monitor, [ordered_set, protected]),\n     init_apitx_reqlist_rrd(os:timestamp())}.\n\n-spec init_apitx_reqlist_rrd(Time::erlang_timestamp() | rrd:internal_time()) -> rrd:rrd().\ninit_apitx_reqlist_rrd(Time) ->\n    % 10s monitoring interval, only keep newest in the client process\n    rrd:create(10 * 1000000, 1, {timing_with_hist, ms}, Time).\n\n%% @doc Checks whether config parameters of the monitor process exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(monitor_timeslots_to_keep) and\n    config:cfg_is_greater_than(monitor_timeslots_to_keep, 0).\n\n-spec get_timeslots_to_keep(Process::atom()) -> pos_integer().\nget_timeslots_to_keep(lb_active) ->\n    config:read(lb_active_monitor_history_max);\nget_timeslots_to_keep(_Process) ->\n    config:read(monitor_timeslots_to_keep).\n\n-spec get_check_timeslots_interval() -> 10.\nget_check_timeslots_interval() ->\n    10. % every 10s\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Convenience API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Get the available RRD keys\n-spec get_rrd_keys() -> [table_index()] | timeout.\nget_rrd_keys() ->\n    Monitor = case pid_groups:get_my(monitor) of\n        failed -> pid_groups:find_a(monitor);\n        M -> M\n    end,\n    get_rrd_keys(Monitor).\n\n-spec get_rrd_keys(comm:erl_local_pid()) -> [table_index()] | timeout.\nget_rrd_keys(MonitorPid) ->\n    comm:send_local(MonitorPid, {get_rrd_keys, self()}),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({get_rrd_keys, Keys}, Keys)\n    after 2000 ->\n            timeout\n    end.\n"
  },
  {
    "path": "src/monitor_perf.erl",
    "content": "%  @copyright 2011-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Periodically executes a small benchmark to monitor the overall\n%%         performance of Scalaris.\n%% @end\n%% @version $Id$\n-module(monitor_perf).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"record_helpers.hrl\").\n-include(\"scalaris.hrl\").\n\n% monitor process functions\n-export([start_link/1, init/1, on/2, check_config/0]).\n\n-include(\"gen_component.hrl\").\n\n-record(state,\n        {id        = ?required(state, id)        :: uid:global_uid(),\n         perf_rr   = ?required(state, perf_rr)   :: rrd:rrd(),\n         perf_lh   = ?required(state, perf_lh)   :: rrd:rrd(),\n         perf_tx   = ?required(state, perf_tx)   :: rrd:rrd()\n        }).\n\n-type state() :: {AllNodes::#state{}, CollectingAtLeader::#state{}, BenchPid::pid() | ok, IgnoreBenchTimeout::boolean()}.\n-type message() ::\n    {bench} |\n    {bench_result, Time::erlang_timestamp(), TimeInMs::non_neg_integer()} |\n    {bench_timeout, Time::erlang_timestamp(), BenchPid::pid()} |\n    {collect_system_stats} |\n    {propagate} |\n    {get_node_details_response, node_details:node_details()} |\n    {bulkowner, deliver, Id::uid:global_uid(), Range::intervals:interval(), {gather_stats, SourcePid::comm:mypid(), Id::uid:global_uid()}, Parents::[comm:mypid(),...]} |\n    {bulkowner, gather, Id::uid:global_uid(), Target::comm:mypid(), Msgs::[comm:message(),...], Parents::[comm:mypid()]} |\n    {bulkowner, reply, Id::uid:global_uid(), {gather_stats_response, Id::uid:global_uid(), [{Process::atom(), Key::monitor:key(), Data::rrd:timing_type()}]}} |\n    {bulkowner, deliver, Id::uid:global_uid(), Range::intervals:interval(), {report_value, StatsOneRound::#state{}}, Parents::[comm:mypid(),...]}.\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-define(TRACE1(Msg, State),\n        ?TRACE(\"[ ~.0p ]~n  Msg: ~.0p~n  State: ~.0p)~n\", [self(), Msg, State])).\n\n%% @doc Creates a monitoring value for benchmarks with a 1m monitoring interval\n%%      and a timing histogram, only keeping the newest value.\n-spec init_bench() -> ok.\ninit_bench() ->\n    monitor:proc_set_value(\n      ?MODULE, 'read_read', rrd:create(60 * 1000000, 1, {timing_with_hist, ms})).\n\n-spec bench_service(Owner::pid()) -> ok.\nbench_service(Owner) ->\n    % do not use gen_component:monitor/1 since this is not a gen_component\n    % -> tolerates that this side channel will not be traced by\n    %    trace_mpath/proto_sched if the monitor dies (it should not die unless\n    %    shutting down the VM anyway!)\n    erlang:monitor(process, Owner),\n    bench_service_loop(Owner).\n\n-spec bench_service_loop(Owner::comm:erl_local_pid()) -> ok.\nbench_service_loop(Owner) ->\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({bench}, %% ->\n            begin\n                Key1 = randoms:getRandomString(),\n                Key2 = randoms:getRandomString(),\n                ReqList = [{read, Key1}, {read, Key2}, {commit}],\n                Time = os:timestamp(),\n                {TimeInUs, _Result} = util:tc(fun api_tx:req_list/1, [ReqList]),\n                comm:send_local(Owner, {bench_result, Time, TimeInUs / 1000}),\n                bench_service_loop(Owner)\n            end);\n        ?SCALARIS_RECV({tx_tm_rtm_commit_reply, _, _}, %% ->\n            % left-over commit information from bench, more specifically api_tx:req_list/1\n            bench_service_loop(Owner));\n        {'DOWN', _MonitorRef, process, Owner, _Info1} -> ok\n    end.\n\n-spec init_system_stats() -> ok.\ninit_system_stats() ->\n    % system stats in 10s intervals:\n    monitor:monitor_set_value(\n      ?MODULE, 'mem_total', rrd:create(15 * 1000000, 1, gauge)),\n    monitor:monitor_set_value(\n      ?MODULE, 'mem_processes', rrd:create(15 * 1000000, 1, gauge)),\n    monitor:monitor_set_value(\n      ?MODULE, 'mem_system', rrd:create(15 * 1000000, 1, gauge)),\n    monitor:monitor_set_value(\n      ?MODULE, 'mem_atom', rrd:create(15 * 1000000, 1, gauge)),\n    monitor:monitor_set_value(\n      ?MODULE, 'mem_binary', rrd:create(15 * 1000000, 1, gauge)),\n    monitor:monitor_set_value(\n      ?MODULE, 'mem_ets', rrd:create(15 * 1000000, 1, gauge)),\n\n    monitor:monitor_set_value(\n      ?MODULE, 'rcv_count', rrd:create(15 * 1000000, 1, gauge)),\n    monitor:monitor_set_value(\n      ?MODULE, 'rcv_bytes', rrd:create(15 * 1000000, 1, gauge)),\n    monitor:monitor_set_value(\n      ?MODULE, 'send_count', rrd:create(15 * 1000000, 1, gauge)),\n    monitor:monitor_set_value(\n      ?MODULE, 'send_bytes', rrd:create(15 * 1000000, 1, gauge)),\n\n    collect_system_stats().\n\n-spec collect_system_stats() -> ok.\ncollect_system_stats() ->\n    [{total, MemTotal}, {processes, MemProcs}, {system, MemSys},\n     {atom, MemAtom}, {binary, MemBin}, {ets, MemEts}] =\n        erlang:memory([total, processes, system, atom, binary, ets]),\n\n    monitor:monitor_set_value(?MODULE, 'mem_total',\n                           fun(Old) -> rrd:add_now(MemTotal, Old) end),\n    monitor:monitor_set_value(?MODULE, 'mem_processes',\n                           fun(Old) -> rrd:add_now(MemProcs, Old) end),\n    monitor:monitor_set_value(?MODULE, 'mem_system',\n                           fun(Old) -> rrd:add_now(MemSys, Old) end),\n    monitor:monitor_set_value(?MODULE, 'mem_atom',\n                           fun(Old) -> rrd:add_now(MemAtom, Old) end),\n    monitor:monitor_set_value(?MODULE, 'mem_binary',\n                           fun(Old) -> rrd:add_now(MemBin, Old) end),\n    monitor:monitor_set_value(?MODULE, 'mem_ets',\n                           fun(Old) -> rrd:add_now(MemEts, Old) end),\n\n    {RcvCnt, RcvBytes, SendCnt, SendBytes} = comm_stats:get_stats(),\n    monitor:monitor_set_value(?MODULE, 'rcv_count',\n                           fun(Old) -> rrd:add_now(RcvCnt, Old) end),\n    monitor:monitor_set_value(?MODULE, 'rcv_bytes',\n                           fun(Old) -> rrd:add_now(RcvBytes, Old) end),\n    monitor:monitor_set_value(?MODULE, 'send_count',\n                           fun(Old) -> rrd:add_now(SendCnt, Old) end),\n    monitor:monitor_set_value(?MODULE, 'send_bytes',\n                           fun(Old) -> rrd:add_now(SendBytes, Old) end).\n\n%% @doc Message handler when the rm_loop module is fully initialized.\n-spec on(message(), state()) -> state().\non({bench} = Msg, {AllNodes, Leader, BenchPid, _IgnBenchT} = _State) ->\n    ?TRACE1(Msg, _State),\n    % periodic task to execute a mini-benchmark (in a separate process)\n    % (result will be send in a 'bench_result' message)\n    case get_bench_interval() of\n        0 -> ok;\n        I -> msg_delay:send_trigger(I, {bench}),\n             comm:send_local(BenchPid, Msg),\n             %% send a timeout so that a hanging bench service gets re-started\n             msg_delay:send_trigger(get_bench_timeout_interval(), {bench_timeout, os:timestamp(), BenchPid})\n    end,\n    {AllNodes, Leader, BenchPid, false};\n\non({bench_result, Time, TimeInMs} = _Msg,\n   {AllNodes, Leader, BenchPid, _IgnBenchT} = _State) ->\n    ?TRACE1(_Msg, _State),\n    % result from the mini-benchmark triggered by the {bench} message\n    monitor:proc_set_value(?MODULE, 'read_read',\n                           fun(Old) -> rrd:add(Time, TimeInMs, Old) end),\n    % ignore the bench_timeout message that will follow\n    {AllNodes, Leader, BenchPid, true};\n\non({bench_timeout, Time, BenchPid} = _Msg,\n   {AllNodes, Leader, BenchPid, false} = _State) ->\n    ?TRACE1(_Msg, _State),\n    % the bench service could not reply within get_bench_timeout_interval() seconds\n    % -> re-start service (it may hang) and assume this interval as the reported time\n    erlang:exit(BenchPid, kill),\n    monitor:proc_set_value(\n      ?MODULE, 'read_read',\n      fun(Old) -> rrd:add(Time, get_bench_timeout_interval() * 1000, Old) end),\n    Self = self(),\n    NewBenchPid = erlang:spawn(fun() -> bench_service(Self) end),\n    {AllNodes, Leader, NewBenchPid, false};\n\non({bench_timeout, _Time, _BenchPid} = _Msg, State) ->\n    %?TRACE1(_Msg, State),\n    % old or ignored timeout message\n    State;\n\non({collect_system_stats} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    % collect system stats, e.g. memory info, and send to monitor\n    msg_delay:send_trigger(10, {collect_system_stats}),\n    collect_system_stats(),\n    State;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% gather global performance data with bulkowner\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\non({propagate} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    % identify whether a DHT leader node is present in this VM\n    msg_delay:send_trigger(get_gather_interval(), {propagate}),\n    Msg = {get_node_details, comm:this(), [my_range]},\n    _ = [comm:send_local(Pid, Msg) || Pid <- pid_groups:find_all(dht_node)],\n    State;\n\non({get_node_details_response, NodeDetails} = _Msg,\n   {AllNodes, Leader, BenchPid, IgnBenchT} = State) ->\n    ?TRACE1(_Msg, State),\n    % response to {propagate} from all local DHT nodes\n    % if any of them is a leader node, start propagation with bulkowner\n    case is_leader(node_details:get(NodeDetails, my_range)) of\n        false -> State;\n        _ ->\n            % start a new timeslot and gather stats...\n            NewId = uid:get_global_uid(),\n            BMsg = {?send_to_registered_proc, monitor_perf, {gather_stats, comm:this()}},\n            bulkowner:issue_bulk_owner(NewId, intervals:all(), BMsg),\n            % create a new timeslot if required\n            Leader1 = check_timeslots(Leader),\n            % broadcast values from the previous timeslot if a new one was started\n            broadcast_previous_values(Leader, Leader1),\n            {AllNodes, Leader1#state{id = NewId}, BenchPid, IgnBenchT}\n    end;\n\non({bulkowner, deliver, Id, _Range, {gather_stats, SourcePid}, Parents} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    % retrieve stats for gather_stats\n    MyMonitor = pid_groups:get_my(monitor), % \"basic_services\" group\n    [Val_RR] = monitor:get_rrds(MyMonitor, [{?MODULE, 'read_read'}]),\n    ClientMonitor = pid_groups:pid_of(clients_group, monitor),\n    [Val_TX] = monitor:get_rrds(ClientMonitor, [{api_tx, 'req_list'}]),\n    % no need to reduce the multiple LH values - the next gather handler will do that\n    DB_LH = [begin\n                 [Val_LH] = monitor:get_rrds(Monitor, [{dht_node, 'lookup_hops'}]),\n                 Val_LH\n             end || Monitor <- pid_groups:find_all(monitor),\n                    Monitor =/= MyMonitor,\n                    Monitor =/= ClientMonitor],\n    case process_rrds([Val_RR, Val_TX | DB_LH ]) of\n        [] -> ok;\n        AllData ->\n            ReplyMsg = {?send_to_registered_proc, monitor_perf, {gather_stats_response, AllData}},\n            bulkowner:issue_send_reply(Id, SourcePid, ReplyMsg, Parents)\n    end,\n    State;\n\non({bulkowner, gather, Id, Target, Msgs, Parents} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    % gather replies from bulkowner_deliver with gather_stats\n    {PerfRR, PerfLH, PerfTX} =\n        lists:foldl(\n             fun({gather_stats_response, Data1}, {PerfRR1, PerfLH1, PerfTX1}) ->\n                     lists:foldl(\n                       fun({?MODULE, 'read_read', PerfRR3}, {PerfRR2, PerfLH2, PerfTX2}) ->\n                               {rrd:timing_with_hist_merge_fun(0, PerfRR2, PerfRR3), PerfLH2, PerfTX2};\n                          ({dht_node, 'lookup_hops', PerfLH3}, {PerfRR2, PerfLH2, PerfTX2}) ->\n                               {PerfRR2, rrd:timing_with_hist_merge_fun(0, PerfLH2, PerfLH3), PerfTX2};\n                          ({api_tx, 'req_list', PerfTX3}, {PerfRR2, PerfLH2, PerfTX2}) ->\n                               {PerfRR2, PerfLH2, rrd:timing_with_hist_merge_fun(0, PerfTX2, PerfTX3)}\n                       end, {PerfRR1, PerfLH1, PerfTX1}, Data1)\n             end, {undefined, undefined, undefined}, Msgs),\n\n    Msg = {?send_to_registered_proc, monitor_perf,\n           {gather_stats_response, [{?MODULE, 'read_read', PerfRR},\n                                    {dht_node, 'lookup_hops', PerfLH},\n                                    {api_tx, 'req_list', PerfTX}]}},\n    bulkowner:send_reply(Id, Target, Msg, Parents, self()),\n    State;\n\non({send_error, FailedTarget, {bulkowner, reply, Id, Target, BMsg, Parents}, _Reason} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    % if sending the reply from bulkowner gather fails\n    bulkowner:send_reply_failed(Id, Target, BMsg, Parents, self(), FailedTarget),\n    State;\n\non({send_error, FailedTarget,\n    {?send_to_registered_proc, monitor_perf,\n     {bulkowner, reply, Id, Target, BMsg, Parents}}, Reason} = _Msg, State) ->\n    %% redirect to other send_error\n    gen_component:post_op({send_error, FailedTarget, {bulkowner, reply, Id, Target, BMsg, Parents}, Reason}, State);\n\non({bulkowner, reply, Id, {gather_stats_response, DataL}} = _Msg,\n   {AllNodes, Leader, BenchPid, IgnBenchT} = _State)\n  when Id =:= Leader#state.id ->\n    ?TRACE1(_Msg, _State),\n    % final aggregation of gather_stats at the leader\n    Leader1 =\n        lists:foldl(\n          fun({?MODULE, 'read_read', PerfRR}, A = #state{perf_rr = DB}) ->\n                  T = rrd:get_current_time(DB),\n                  A#state{perf_rr = rrd:add_with(T, PerfRR, DB, fun rrd:timing_with_hist_merge_fun/3)};\n             ({dht_node, 'lookup_hops', PerfLH}, A = #state{perf_lh = DB}) ->\n                  T = rrd:get_current_time(DB),\n                  A#state{perf_lh = rrd:add_with(T, PerfLH, DB, fun rrd:timing_with_hist_merge_fun/3)};\n             ({api_tx, 'req_list', PerfTX}, A = #state{perf_tx = DB}) ->\n                  T = rrd:get_current_time(DB),\n                  A#state{perf_tx = rrd:add_with(T, PerfTX, DB, fun rrd:timing_with_hist_merge_fun/3)}\n          end, Leader, DataL),\n    {AllNodes, Leader1, BenchPid, IgnBenchT};\non({bulkowner, reply, _Id, {gather_stats_response, _Data}} = _Msg, State) ->\n    ?TRACE1(_Msg, State),\n    % final aggregation of gather_stats when not the leader or old ID -> ignore\n    State;\n\non({bulkowner, deliver, Id, _Range, {report_value, _OtherState}, _Parents} = _Msg,\n   {AllNodes, _Leader, _BenchPid, _IgnBenchT} = State)\n  when Id =:= AllNodes#state.id ->\n    % duplicate message\n    State;\n\non({bulkowner, deliver, Id, _Range, {report_value, OtherState}, _Parents} = _Msg,\n   {AllNodes, Leader, BenchPid, IgnBenchT} = State) ->\n    % if the leader has not changed, only accept newer GUIDs\n    % (there could be multiple deliver messages if more than one dht_node exist in this VM)\n    case (not uid:from_same_pid(Id, AllNodes#state.id)) orelse\n             uid:is_old_uid(AllNodes#state.id, Id) of\n        true ->\n            ?TRACE1(_Msg, State),\n            % integrate value send via broadcast from leader\n            #state{perf_rr = Perf_RR, perf_lh = Perf_LH, perf_tx = Perf_TX} =\n                AllNodes1 = integrate_values(AllNodes, OtherState),\n            % send to the own monitor\n            monitor:monitor_set_value(?MODULE,  'agg_read_read', Perf_RR),\n            monitor:monitor_set_value(dht_node, 'agg_lookup_hops', Perf_LH),\n            monitor:monitor_set_value(api_tx,   'agg_req_list', Perf_TX),\n            {AllNodes1#state{id = Id}, Leader, BenchPid, IgnBenchT};\n        _ ->\n            % drop old or duplicate message\n            State\n    end;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% misc.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\non({web_debug_info, Requestor} = _Msg,\n   {AllNodes, Leader, _BenchPid, _IgnBenchT} = State) ->\n    ?TRACE1(_Msg, State),\n    [KVAllNodes, KVLeader] =\n        [begin\n             PerfRR5 = rrd:reduce_timeslots(5, Data#state.perf_rr),\n             PerfLH5 = rrd:reduce_timeslots(5, Data#state.perf_lh),\n             PerfTX5 = rrd:reduce_timeslots(5, Data#state.perf_tx),\n             [monitor:web_debug_info_merge_values({?MODULE, perf_rr}, PerfRR5),\n              monitor:web_debug_info_merge_values({dht_node, perf_lh}, PerfLH5),\n              monitor:web_debug_info_merge_values({api_tx, perf_tx}, PerfTX5)]\n         end || Data <- [AllNodes, Leader]],\n    KeyValueList = lists:flatten([{\"all nodes:\", \"\"}, KVAllNodes,\n                                  {\"leader:\",    \"\"}, KVLeader]),\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    State.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Startup\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Starts the monitor process, registers it with the process dictionary\n%%      and returns its pid for use by a supervisor.\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, null,\n                             [{erlang_register, ?MODULE},\n                              {pid_groups_join_as, DHTNodeGroup, monitor_perf}]).\n\n%% @doc Initialises the module with an empty state.\n-spec init(null) -> state().\ninit(null) ->\n    BenchPid =\n        case get_bench_interval() of\n            0 -> ok;\n            I -> FirstDelay = randoms:rand_uniform(1, I + 1),\n                 msg_delay:send_trigger(FirstDelay, {bench}),\n                 msg_delay:send_trigger(get_gather_interval(), {propagate}),\n                 init_bench(),\n                 Self = self(),\n                 erlang:spawn(fun() -> bench_service(Self) end)\n        end,\n    init_system_stats(),\n    msg_delay:send_trigger(10, {collect_system_stats}),\n    Now = os:timestamp(),\n    Perf_RR = rrd:create(get_gather_interval() * 1000000, 60, {timing_with_hist, ms}, Now),\n    Perf_LH = rrd:create(get_gather_interval() * 1000000, 60, {timing, count}, Now),\n    Perf_TX = rrd:create(get_gather_interval() * 1000000, 60, {timing_with_hist, ms}, Now),\n    State = #state{id = uid:get_global_uid(),\n                   perf_rr = Perf_RR, perf_lh = Perf_LH, perf_tx = Perf_TX},\n    monitor:monitor_set_value(?MODULE,  'agg_read_read', Perf_RR),\n    monitor:monitor_set_value(dht_node, 'agg_lookup_hops', Perf_LH),\n    monitor:monitor_set_value(api_tx,   'agg_req_list', Perf_TX),\n    {State, State, BenchPid, false}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Miscellaneous\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec check_timeslots(#state{}) -> #state{}.\ncheck_timeslots(State = #state{perf_rr = PerfRR, perf_lh = PerfLH, perf_tx = PerfTX}) ->\n    State#state{perf_rr = rrd:check_timeslot_now(PerfRR),\n                perf_lh = rrd:check_timeslot_now(PerfLH),\n                perf_tx = rrd:check_timeslot_now(PerfTX)}.\n\n-spec broadcast_previous_values(OldState::#state{}, NewState::#state{}) -> ok.\nbroadcast_previous_values(OldState, NewState) ->\n    % broadcast the latest value only if a new time slot was started\n    PerfRRNewSlot = rrd:get_slot_start(0, OldState#state.perf_rr) =/=\n                        rrd:get_slot_start(0, NewState#state.perf_rr),\n    PerfLHNewSlot = rrd:get_slot_start(0, OldState#state.perf_lh) =/=\n                        rrd:get_slot_start(0, NewState#state.perf_lh),\n    PerfTXNewSlot = rrd:get_slot_start(0, OldState#state.perf_tx) =/=\n                        rrd:get_slot_start(0, NewState#state.perf_tx),\n    if PerfRRNewSlot orelse PerfLHNewSlot orelse PerfTXNewSlot ->\n           % new slot -> broadcast latest values only:\n           SendState = reduce_timeslots(1, OldState),\n           Msg = {?send_to_registered_proc, monitor_perf, {report_value, SendState}},\n           bulkowner:issue_bulk_owner(uid:get_global_uid(), intervals:all(), Msg);\n       true -> ok % nothing to do\n    end.\n\n-spec reduce_timeslots(N::pos_integer(), State::#state{}) -> #state{}.\nreduce_timeslots(N, State) ->\n    State#state{perf_rr = rrd:reduce_timeslots(N, State#state.perf_rr),\n                perf_lh = rrd:reduce_timeslots(N, State#state.perf_lh),\n                perf_tx = rrd:reduce_timeslots(N, State#state.perf_tx)}.\n\n%% @doc Integrates values from OtherState by merging all rrd records into MyState.\n-spec integrate_values(MyState::#state{}, OtherState::#state{}) -> #state{}.\nintegrate_values(MyState, OtherState) ->\n    MyPerfRR1 = rrd:merge(MyState#state.perf_rr, OtherState#state.perf_rr),\n    MyPerfLH1 = rrd:merge(MyState#state.perf_lh, OtherState#state.perf_lh),\n    MyPerfTX1 = rrd:merge(MyState#state.perf_tx, OtherState#state.perf_tx),\n    MyState#state{id = OtherState#state.id,\n                  perf_rr = MyPerfRR1, perf_lh = MyPerfLH1, perf_tx = MyPerfTX1}.\n\n%% @doc Checks whether the node is the current leader.\n-spec is_leader(MyRange::intervals:interval()) -> boolean().\nis_leader(MyRange) ->\n    intervals:in(?RT:hash_key(\"0\"), MyRange).\n\n%% @doc For each rrd in the given list, accumulate all values in our time span\n%%      into a single timing type value.\n-spec process_rrds(DBs::[{Process::atom(), Key::monitor:key(), DB::rrd:rrd() | undefined}]) ->\n          [{Process::atom(), Key::monitor:key(), Data::rrd:timing_type()}].\nprocess_rrds(DBs) ->\n    lists:flatten(\n      [begin\n           % DB slot length may be different -> try to gather data from our full time span:\n           Slots = erlang:max(1, (get_gather_interval() * 1000000) div rrd:get_slot_length(DB)),\n           DB2 = rrd:reduce_timeslots(Slots, DB),\n           DBDump = rrd:dump_with(DB2, fun(_DB, _From, _To, X) -> X end),\n           case DBDump of\n               []      -> [];\n               [H | T] ->\n                   Data = lists:foldl(fun(E, A) -> rrd:timing_with_hist_merge_fun(0, A, E) end, H, T),\n                   {Process, Key, Data}\n           end\n       end || {Process, Key, DB} <- DBs, DB =/= undefined]).\n\n%% @doc Checks whether config parameters of the rm_tman process exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(monitor_perf_interval) and\n    config:cfg_is_greater_than_equal(monitor_perf_interval, 0).\n\n-spec get_bench_interval() -> non_neg_integer().\nget_bench_interval() ->\n    config:read(monitor_perf_interval).\n\n%% @doc Timeout interval (in seconds) for a bench request.\n%%      NOTE: this must be smaller than get_bench_interval()!\n-spec get_bench_timeout_interval() -> non_neg_integer().\nget_bench_timeout_interval() ->\n    get_bench_interval() div 2.\n\n%% @doc Gets the interval of executing a broadcast gathering all nodes' stats\n%%      (in seconds).\n-spec get_gather_interval() -> pos_integer().\nget_gather_interval() -> 60.\n"
  },
  {
    "path": "src/mr.erl",
    "content": "%% @copyright 2007-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@informatik.hu-berlin.de>\n%% @doc Map Reduce functions\n%%      this is part of the dht node\n%%\n%% @end\n%% @version $Id$\n-module(mr).\n-author('fajerski@informatik.hu-berlin.de').\n-vsn('$Id$ ').\n\n%% -define(TRACE(X, Y), io:format(X, Y)).\n-define(TRACE(X, Y), ok).\n\n-export([\n        on/2,\n        work_on_phase/3,\n        neighborhood_succ_crash_filter/3,\n        neighborhood_succ_crash/5\n        ]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-type(bulk_message() :: {mr, job, mr_state:jobid(), MasterId::?RT:key(),\n                         Client::comm:mypid(),\n                         mr_state:job_description(), mr_state:data()} |\n                        {mr, next_phase_data, mr_state:jobid(), comm:mypid(),\n                         mr_state:data()}).\n\n-type(message() :: {bulk_distribute, uid:global_uid(), intervals:interval(),\n                    bulk_message(), Parents::[comm:mypid(),...]} |\n                   {mr, phase_results, mr_state:jobid(), comm:message(),\n                    intervals:interval()} |\n                   {mr, next_phase_data_ack, {mr_state:jobid(), reference(),\n                                              intervals:interval()},\n                    intervals:interval()} |\n                   {mr, next_phase, mr_state:jobid()} |\n                   {mr, terminate_job, mr_state:jobid()}).\n\n-spec on(message(), dht_node_state:state()) -> dht_node_state:state().\non({bulk_distribute, _Id, Interval,\n    {mr, job, JobId, MasterId, Client, Job, InitalData}, _Parents}, State) ->\n    %% this message starts the worker supervisor and adds a job specific state\n    %% to the dht node\n    rm_loop:subscribe(self(), {\"mr_succ_fd\", JobId, MasterId, Client},\n                      fun neighborhood_succ_crash_filter/3,\n                      fun neighborhood_succ_crash/5,\n                      inf),\n    MRState = case dht_node_state:get_mr_state(State, JobId) of\n        error ->\n            ?TRACE(\"~p mr_~s: received job init for ~p~n~p~n\",\n                   [self(), JobId, Interval, InitalData]),\n            mr_state:new(JobId, Client, MasterId, InitalData, Job,\n                       Interval);\n        {ok, ExState} ->\n            ?TRACE(\"~p mr_~s: second init for ~p~n~p~n\",\n                   [self(), JobId, Interval, InitalData]),\n            mr_state:add_data_to_phase(ExState, InitalData, Interval, 1)\n    end,\n    %% send acc to master\n    send_to_master(MRState, {mr_master, JobId, phase_completed, 0, Interval}),\n    dht_node_state:set_mr_state(State, JobId, MRState);\n\non({mr, phase_result, JobId, {work_done, Data}, Range, Round}, State) ->\n    %% processing of phase results from worker.\n    %% distribute data and start sync phase\n    %% ?TRACE(\"~p mr_~s: received phase results (round ~p) for interval ~p:~n~p...~ndistributing...~n\",\n    %%        [self(), JobId, Round, Range, Data]),\n    {ok, MRState} = dht_node_state:get_mr_state(State, JobId),\n    NewMRState = mr_state:interval_processed(MRState, Range, Round),\n    case mr_state:is_last_phase(MRState, Round) of\n        false ->\n            Ref = uid:get_global_uid(),\n            bulkowner:issue_bulk_distribute(Ref, dht_node,\n                                            5, {mr, next_phase_data, JobId,\n                                                Range, '_', Round}, Data);\n        _ ->\n            ?TRACE(\"~p jobs last phase done...sending to client ~p~n~p~n\",\n                   [self(), Data, Range]),\n            send_to_master(MRState, {mr_master, JobId, job_completed, Range}),\n            send_to_client(MRState, {mr_results,\n                               [{K, V} || {_HK, K, V} <- Data],\n                               Range, JobId})\n    end,\n    dht_node_state:set_mr_state(State, JobId, NewMRState);\n\non({mr, phase_result, JobId, {worker_died, Reason}, Range, _Round}, State) ->\n    %% processing of a failed worker result.\n    %% for now abort the job\n    ?TRACE(\"runtime error in phase ~p...terminating job~n\", [Round]),\n    {ok, MRState} = dht_node_state:get_mr_state(State, JobId),\n    send_to_master(MRState, {mr_master, JobId, job_error, Range}),\n    send_to_client(MRState, {mr_results, {error, Reason}, Range, JobId}),\n    State;\n\non({bulk_distribute, _Id, Interval,\n   {mr, next_phase_data, JobId, AckRange, Data, Round}, _Parents}, State) ->\n    %% processing of data for next phase.\n    %% save data and send ack\n    ?TRACE(\"~p mr_~s: received next phase data (round ~p) interval ~p: ~p~n\",\n           [self(), JobId, Round, Interval, Data]),\n    {ok, MRState} = dht_node_state:get_mr_state(State, JobId),\n    NewMRState = mr_state:add_data_to_phase(MRState, Data, Interval, Round + 1),\n    %% send ack with delivery interval\n    bulkowner:issue_bulk_owner(uid:get_global_uid(), AckRange, {mr,\n                                                                next_phase_data_ack,\n                                                               Interval, JobId,\n                                                               Round}),\n    dht_node_state:set_mr_state(State, JobId, NewMRState);\n\non({mr, next_phase_data_ack, AckInterval, JobId, Round, DeliveryInterval}, State) ->\n    %% ack from other mr nodes.\n    %% check if the whole interval waas acked. If so inform master, wait\n    %% otherwise.\n    {ok, MRState} = dht_node_state:get_mr_state(State, JobId),\n    NewMRState = mr_state:set_acked(MRState,\n                                    AckInterval),\n    NewMRState2 = case mr_state:is_acked_complete(NewMRState) of\n        true ->\n            send_to_master(NewMRState, {mr_master, JobId,\n                                      phase_completed, Round, DeliveryInterval}),\n            ?TRACE(\"Phase ~p complete...~p informing master at ~p~n\", [Round, self(),\n                                                                    mr_state:get(NewMRState,\n                                                                                master_id)]),\n            mr_state:reset_acked(NewMRState);\n        false ->\n            ?TRACE(\"~p is still waiting for phase ~p to complete~n\", [self(),\n                                                                      Round]),\n            NewMRState\n    end,\n    dht_node_state:set_mr_state(State, JobId, NewMRState2);\n\non({mr, next_phase, JobId, Round, _DeliveryInterval}, State) ->\n    %% master started next round.\n    ?TRACE(\"~p master initiated phase ~p in ~p~n~p\",\n              [self(), Round, JobId, DeliveryInterval]),\n    {ok, MRState} = dht_node_state:get_mr_state(State, JobId),\n    NewMRState = work_on_phase(JobId, MRState, Round),\n    dht_node_state:set_mr_state(State, JobId, NewMRState);\n\non({mr, terminate_job, JobId, _DeliveryInterval}, State) ->\n    %% master wants to terminate job.\n        ?TRACE(\"~p got terminate message from master for job ~p~n\",\n               [self(), JobId]),\n        case dht_node_state:get_mr_state(State, JobId) of\n            {ok, MRState} ->\n                Master = mr_state:get(MRState, master_id),\n                Client = mr_state:get(MRState, client),\n                rm_loop:unsubscribe(self(), {\"mr_succ_fd\", JobId, Master, Client}),\n                _ = mr_state:clean_up(MRState),\n                dht_node_state:delete_mr_state(State, JobId);\n            error ->\n                log:log(debug,\n                        \"Received terminate message for non-existing mr-job...ignoring!\"),\n                State\n        end;\n\non(_Msg, State) ->\n    ?TRACE(\"~p mr: unknown message ~p~n\", [comm:this(), _Msg]),\n    State.\n\n-spec work_on_phase(mr_state:jobid(), mr_state:state(), pos_integer()) ->\n    mr_state:state().\nwork_on_phase(JobId, State, Round) ->\n    {Round, FunTerm, ETS, Open, _Working} = mr_state:get_phase(State, Round),\n    case intervals:is_empty(Open) of\n        false ->\n            case db_ets:get_load(ETS) of\n                0 ->\n                    ?TRACE(\"~p mr_~s: no data for this phase...phase complete ~p~n\",\n                           [self(), JobId, Round]),\n                    case mr_state:is_last_phase(State, Round) of\n                        false ->\n                            %% io:format(\"no data for phase...done...~p informs master~n\", [self()]),\n                            send_to_master(State,\n                                           {mr_master, JobId, phase_completed, Round, Open}),\n                            mr_state:interval_empty(State, Open, Round);\n                        _ ->\n                            %% io:format(\"last phase and no data ~p~n\", [Round]),\n                            send_to_master(State, {mr_master, JobId, job_completed, Open}),\n\n                            send_to_client(State, {mr_results, [], Open, JobId}),\n                            mr_state:interval_empty(State, Open, Round)\n                    end;\n                _Load ->\n                    ?TRACE(\"~p mr_~s: starting to work on phase ~p\n                        sending work (~p)~n~p\n                               to ~p~n\", [self(), JobId, Round, Open,\n                                          ets:tab2list(ETS),\n                                          pid_groups:get_my(wpool)]),\n                    Reply = comm:reply_as(comm:this(), 4, {mr, phase_result, JobId, '_',\n                                                           Open, Round}),\n                    comm:send_local(pid_groups:get_my(wpool),\n                                    {do_work, Reply, {FunTerm, ETS, Open}}),\n                    mr_state:interval_processing(State, Open, Round)\n                end;\n        true ->\n            %% nothing to do\n            State\n    end.\n\n-spec neighborhood_succ_crash_filter(Old::nodelist:neighborhood(),\n                                     New::nodelist:neighborhood(),\n                                     rm_loop:reason()) -> boolean().\nneighborhood_succ_crash_filter(Old, New, {node_crashed, _Pid}) ->\n    %% only looking for succ changes\n    nodelist:succ(Old) /= nodelist:succ(New);\nneighborhood_succ_crash_filter(_Old, _New, _) -> false.\n\n-spec neighborhood_succ_crash(comm:mypid(), {string(), mr_state:jobid(),\n                                             ?RT:key(), comm:mypid()},\n                              Old::nodelist:neighborhood(),\n                              New::nodelist:neighborhood(),\n                              Reason::rm_loop:reason()) -> ok.\nneighborhood_succ_crash(Pid, {\"mr_succ_fd\", JobId, MasterId, Client}, _Old, _New, _Reason) ->\n    io:format(\"~p: succ crashed...informing master~n\",\n              [Pid]),\n    api_dht_raw:unreliable_lookup(MasterId,\n                                  {mr_master, JobId,\n                                   job_error,\n                                   intervals:empty()}),\n    comm:send(Client, {mr_results, {error, node_died}, intervals:empty(), JobId}).\n\n-spec send_to_master(mr_state:state(), mr_master:message()) -> ok.\nsend_to_master(State, Msg) ->\n    Key = mr_state:get(State, master_id),\n    api_dht_raw:unreliable_lookup(Key, Msg).\n\n-spec send_to_client(mr_state:state(), tuple()) -> ok.\nsend_to_client(State, Msg) ->\n    Client = mr_state:get(State, client),\n    comm:send(Client, Msg).\n"
  },
  {
    "path": "src/mr_master.erl",
    "content": "%% @copyright 2007-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@informatik.hu-berlin.de>\n%% @doc Map reduce master file\n%%      The master is in charge of orchestrating the single phases of the job.\n%% @end\n%%      So far it is the single point of failure. It should be replicated and\n%%      it should have a mechanism to recover to an earlier stage of the job\n%% @version $Id$\n-module(mr_master).\n-author('fajerski@zib.de').\n-vsn('$Id$ ').\n\n%% -define(TRACE(X, Y), io:format(X, Y)).\n-define(TRACE(X, Y), ok).\n\n-export([\n         init_job/4,\n         on/2,\n         dispatch_snapshot/1\n        ]).\n\n-export_type([message/0]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-type(message() :: {mr_master, init, Client::comm:mypid(), mr_state:jobid(),\n                    JobSpec::mr_state:job_description()} |\n                   {mr_master, mr_state:jobid(), snapshot,\n                    snapshot_leader:result_message(),\n                    mr_state:job_description(), comm:mypid()} |\n                   {mr_master, mr_state:jobid(), phase_completed, Round::non_neg_integer(), Range::intervals:interval()} |\n                   {mr_master, mr_state:jobid(), job_completed, intervals:interval()} |\n                   {mr_master, mr_state:jobid(), job_error, intervals:interval()}).\n\n-spec init_job(dht_node_state:state(), nonempty_string(), mr_state:job_description(),\n              comm:mypid()) ->\n    dht_node_state:state().\ninit_job(State, JobId, Job, Client) ->\n    Id = dht_node_state:get(State, node_id),\n    MasterState = mr_master_state:new(Id, Job, Client),\n    dispatch_snapshot(JobId),\n    dht_node_state:set_mr_master_state(State, JobId, MasterState).\n\n-spec dispatch_snapshot(mr_state:jobid()) -> ok.\ndispatch_snapshot(JobId) ->\n    Reply = comm:reply_as(comm:this(), 4, {mr_master, JobId, snapshot, '_'}),\n    comm:send_local(pid_groups:find_a(snapshot_leader), {init_snapshot,\n                                                  Reply}).\n\n-spec on(message(), dht_node_state:state()) -> dht_node_state:state().\non({mr_master, init, Client, JobId, Job}, State) ->\n    %% this is the inital message\n    %% it creates a JobId and starts the master process,\n    %% which in turn starts the worker supervisor on all nodes.\n    ?TRACE(\"mr_master: ~p~n received init message from ~p~n starting job ~p~n\",\n           [comm:this(), Client, Job]),\n    case validate_job(Job) of\n        ok ->\n            init_job(State, JobId, Job, Client);\n        {error, Reason} ->\n            comm:send(Client, {mr_results, {error, Reason}, intervals:all(),\n                               JobId}),\n            State\n    end;\n\non({mr_master, JobId, snapshot, {global_snapshot_done, Data}}, State) ->\n    MasterState = dht_node_state:get_mr_master_state(State, JobId),\n    Job = mr_master_state:get(job, MasterState),\n    FilteredData = case lists:keyfind(tag, 1, element(2, Job)) of\n                       false ->\n                           filter_data(Data);\n                       Tag ->\n                           filter_data(Data, Tag)\n                   end,\n    ?TRACE(\"mr_master: starting job ~p~n\", [JobId]),\n    bulkowner:issue_bulk_distribute(uid:get_global_uid(),\n                                    dht_node, 7,\n                                    {mr, job, JobId,\n                                     mr_master_state:get(id, MasterState),\n                                     mr_master_state:get(client, MasterState),\n                                     Job,\n                                     '_'},\n                                    FilteredData),\n    dht_node_state:set_mr_master_state(State,\n                                       JobId,\n                                       mr_master_state:set(MasterState,\n                                                           [{outstanding, false}]));\n\n%% in case the snapshot failed, try again\non({mr_master, JobId, snapshot, {global_snapshot_done_with_errors,\n                                 _ErrorInterval, _PartData}}, State) ->\n    dispatch_snapshot(JobId),\n    State;\n\non({mr_master, JobId, snapshot, {worker_died, Reason}}, State) ->\n    log:log(error, \"mr_master_~s: snapshot failed ~p\", [JobId, Reason]),\n    dht_node_state:delete_mr_master_state(State, JobId);\n\non({mr_master, JobId, phase_completed, Round, Range}, State) ->\n    MasterState = dht_node_state:get_mr_master_state(State, JobId),\n    I = mr_master_state:get(acked, MasterState),\n    NewInterval = intervals:union(I, Range),\n    NewMasterState = case intervals:is_all(NewInterval) of\n        false ->\n            ?TRACE(\"~p mr_master_~s: phase ~p not yet completed...~p~n\",\n                     [self(), JobId, Round, NewInterval]),\n            mr_master_state:set(MasterState, [{acked, NewInterval}]);\n        _ ->\n            ?TRACE(\"mr_master_~s: phase ~p completed...initiating next phase~n\",\n                     [JobId, Round]),\n            bulkowner:issue_bulk_owner(uid:get_global_uid(), intervals:all(),\n                                       {mr, next_phase, JobId, Round + 1}),\n            mr_master_state:set(MasterState, [{acked, intervals:empty()},\n                                        {round, Round + 1}])\n    end,\n    dht_node_state:set_mr_master_state(State, JobId, NewMasterState);\n\non({mr_master, JobId, job_completed, Range}, State) ->\n    MasterState = dht_node_state:get_mr_master_state(State, JobId),\n    I = mr_master_state:get(acked, MasterState),\n    NewInterval = intervals:union(I, Range),\n    case intervals:is_all(NewInterval) of\n        false ->\n            dht_node_state:set_mr_master_state(State,\n                                               JobId,\n                                               mr_master_state:set(MasterState,\n                                                                   [{acked,\n                                                                     NewInterval}]));\n        _ ->\n            ?TRACE(\"~p mr_master_~s: job completed...shutting down~n\",\n                     [self(), JobId]),\n            bulkowner:issue_bulk_owner(uid:get_global_uid(), intervals:all(),\n                                       {mr, terminate_job, JobId}),\n            dht_node_state:delete_mr_master_state(State, JobId)\n    end;\n\non({mr_master, JobId, job_error, _Range}, State) ->\n    ?TRACE(\"~p mr_master_~s: job crashed...shutting down~n\",\n             [self(), JobId]),\n    bulkowner:issue_bulk_owner(uid:get_global_uid(), intervals:all(),\n                               {mr, terminate_job, JobId}),\n    dht_node_state:delete_mr_master_state(State, JobId);\n\non(_Msg, State) ->\n    ?TRACE(\"~p mr_master: revceived ~p~n\",\n           [comm:this(), Msg]),\n    State.\n\n-spec filter_data([{?RT:key(),\n                   {{string(), term()} | {atom(), string(), term()}},\n                   client_version()}],\n                  {tag, atom()}) -> mr_state:data_list().\nfilter_data(Data, {tag, FilterTag}) ->\n    ?TRACE(\"Filtering! tag is ~p~ndata: ~p~n\", [FilterTag, Data]),\n    [{?RT:hash_key(K), K, V} || {_HashedKey, {Tag, K, V}, _Version} <- Data, Tag\n                                =:= FilterTag].\n\n-spec filter_data([{?RT:key(),\n                   {{string(), term()} | {atom(), string(), term()}},\n                   client_version()}]) -> mr_state:data_list().\nfilter_data(Data) ->\n    [{?RT:hash_key(K), K, V} || {_HashedKey, {K, V}, _Version} <- Data].\n\n-spec validate_job(mr_state:job_description()) -> ok | {error, term()}.\nvalidate_job({Phases, _Options}) ->\n    validate_phases(Phases).\n\n-spec validate_phases([mr_state:fun_term()]) -> ok | {error, term()}.\nvalidate_phases([]) -> ok;\nvalidate_phases([H | T]) ->\n    case validate_phase(H) of\n        ok ->\n            validate_phases(T);\n        Error ->\n            Error\n    end.\n\n-spec validate_phase(mr_state:fun_term()) -> ok | {error, term()}.\nvalidate_phase(Phase) ->\n    MoR = element(1, Phase),\n    FunTag = element(2, Phase),\n    Fun = element(3, Phase),\n    case MoR == map orelse MoR == reduce of\n        true ->\n            case FunTag of\n                erlanon ->\n                    case is_function(Fun, 1) of\n                        true ->\n                            ok;\n                        false ->\n                            {error ,{badfun, \"Fun should be a fun\"}}\n                    end;\n                jsanon ->\n                    case is_binary(Fun) of\n                        true ->\n                            ok;\n                        false ->\n                            {error ,{badfun, \"Fun should be a binary\"}}\n                    end;\n                Tag ->\n                    {error, {bad_tag, {Tag, Fun}}}\n            end;\n        false ->\n            {error, {bad_phase, \"phase must be either map or reduce\"}}\n    end.\n"
  },
  {
    "path": "src/mr_master_state.erl",
    "content": "%  @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc state for one map reduce master\n%% @version $Id$\n-module(mr_master_state).\n-author('fajerski@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n-type outstanding() :: false | snapshot.\n\n-record(state, {id             = 0 :: ?RT:key(),\n                acked = intervals:empty() :: intervals:interval(),\n                round  = 0 :: non_neg_integer(),\n                outstanding    = false :: outstanding(),\n                job            = [] :: mr_state:job_description(),\n                client         = false :: false | comm:mypid()\n               }).\n\n-type state() :: #state{}.\n\n-export_type([state/0]).\n\n-export([new/3\n        , get/2\n        , set/2\n        , get_slide_delta/2]).\n\n-spec new(Id::?RT:key(), Job::mr_state:job_description(), Client::comm:mypid()) ->\n    state().\nnew(Id, Job, Client) ->\n    #state{id          = Id,\n           outstanding =  snapshot,\n           job         = Job,\n           client      = Client}.\n\n%% TODO check if this is better vs dht_node_state\n-spec get(id, state()) -> ?RT:key();\n         (acked, state()) -> intervals:interval();\n         (round, state()) -> non_neg_integer();\n         (outstanding, state()) -> outstanding();\n         (job, state()) -> mr_state:job_description();\n         (client, state()) -> comm:mypid().\nget(id, #state{id = X}) ->\n    X;\nget(acked, #state{acked = X}) ->\n    X;\nget(round, #state{round = X}) ->\n    X;\nget(outstanding, #state{outstanding = X}) ->\n    X;\nget(job, #state{job = X}) ->\n    X;\nget(client, #state{client = X}) ->\n    X.\n\n-spec set(state(), [{id, ?RT:key()} |\n                        {acked, intervals:interval()} |\n                        {round, non_neg_integer()} |\n                        {outstanding, outstanding()} |\n                        {client, comm:mypid()}]) -> state().\nset(State, []) ->\n    State;\nset(State, [{id, Id} | T]) ->\n    set(State#state{id = Id}, T);\nset(State, [{acked, Acked} | T]) ->\n    set(State#state{acked = Acked}, T);\nset(State, [{round, Round} | T]) ->\n    set(State#state{round = Round}, T);\nset(State, [{outstanding, Out} | T]) ->\n    set(State#state{outstanding = Out}, T);\nset(State, [{client, Client} | T]) ->\n    set(State#state{client = Client}, T).\n\n-spec get_slide_delta(state(), intervals:interval()) -> {boolean(), state()}.\nget_slide_delta(State, Interval) ->\n    {intervals:in(get(id, State), Interval), State}.\n"
  },
  {
    "path": "src/mr_state.erl",
    "content": "%  @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc state for one map reduce node\n%% @version $Id$\n-module(mr_state).\n-author('fajerski@zib.de').\n-vsn('$Id$').\n\n-define(TRACE(X, Y), ok).\n%% -define(TRACE(X, Y), io:format(X, Y)).\n-define(TRACE_SLIDE(X, Y), ok).\n%% -define(TRACE_SLIDE(X, Y), io:format(X, Y)).\n\n-define(DEF_OPTIONS, []).\n\n-export([new/6\n        , get/2\n        , get_phase/2\n        , is_acked_complete/1\n        , set_acked/2\n        , reset_acked/1\n        , is_last_phase/2\n        , add_data_to_phase/4\n        , interval_processing/3\n        , interval_processed/3\n        , interval_empty/3\n        , accumulate_data/2\n        , clean_up/1\n        , get_slide_delta/2\n        , init_slide_state/1\n        , merge_states/2\n        , tester_is_valid_funterm/1\n        , tester_create_valid_funterm/2]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n%% for ?required macro\n-include(\"record_helpers.hrl\").\n\n-export_type([data/0, jobid/0, state/0, fun_term/0, data_list/0,\n              job_description/0]).\n\n-type(erl_fun() :: {map, erlanon, fun((Arg::{client_key(), term()}) ->\n                                          Res::[{client_key(), term()}])}\n                 | {reduce, erlanon, fun((Arg::[{client_key(), term()}]) ->\n                                            Res::[{client_key(), term()}])}).\n\n-type(js_fun() :: {map | reduce, jsanon, binary()}).\n\n-type(option() :: {tag, atom()}).\n\n-type(fun_term() :: erl_fun() | js_fun()).\n\n-type job_description() :: {[fun_term(),...], [option()]}.\n\n-type(data_list() :: [{?RT:key(), string(), term()}]).\n%% data in ets table has the same format\n-type(data_ets() :: db_ets:db()).\n\n-type(data() :: data_list() | data_ets()).\n\n-type(delta_phase() :: {PhaseNr::pos_integer(), fun_term(),\n                        Input::data_list(), ToWorkOn::intervals:interval(),\n                        WorkingOn::intervals:interval()}).\n\n-type(ets_phase() :: {PhaseNr::pos_integer(), fun_term(),\n                      Input::data_ets(), ToWorkOn::intervals:interval(),\n                      WorkingOn::intervals:interval()}).\n\n-type(phase() :: ets_phase() | delta_phase()).\n\n%% only allow strings that can be converted to atoms (used for ets names)\n-type(jobid() :: [0..255,...]).\n\n-record(state, {jobid       = ?required(state, jobid) :: jobid()\n                , client    = false :: comm:mypid() | false\n                , master_id = ?required(state, master_id) :: ?RT:key()\n                , phases    = ?required(state, phases) :: [phase(),...]\n                , options   = ?required(state, options) :: [option()]\n                , acked     = intervals:empty() :: intervals:interval()\n               }).\n\n-type(state() :: #state{}).\n\n-spec get(state(), client)          -> comm:mypid() | false;\n         (state(), jobid)           -> jobid();\n         (state(), phases)          -> [phase()];\n         (state(), master_id)       -> ?RT:key();\n         (state(), options)         -> [option()].\nget(#state{client        = Client\n           , master_id   = Master\n           , jobid       = JobId\n           , phases      = Phases\n           , options     = Options\n          }, Key) ->\n    case Key of\n        client      -> Client;\n        master_id   -> Master;\n        phases      -> Phases;\n        options     -> Options;\n        jobid       -> JobId\n    end.\n\n-spec new(jobid(), comm:mypid(), ?RT:key(), data_list(),\n          job_description(), intervals:interval()) ->\n    state().\nnew(JobId, Client, Master, InitalData, {Phases, Options}, Interval) ->\n    ?TRACE(\"mr_state: ~p~nnew state from: ~p~n\", [comm:this(), {JobId, Client,\n                                                                Master,\n                                                                InitalData,\n                                                                {Phases,\n                                                                 Options}}]),\n    InitalETS = db_ets:new(\n                    lists:append([\"mr_\", JobId, \"_1\"]), [ordered_set]),\n    DB = db_ets:put(InitalETS, InitalData),\n    ExtraData = [{1, DB, Interval, intervals:empty()} |\n                 [{I, db_ets:new(\n                        lists:flatten(io_lib:format(\"mr_~s_~p\", [JobId, I]))\n                        , [ordered_set]), intervals:empty(), intervals:empty()}\n                  || I <- lists:seq(2, length(Phases))]],\n    PhasesWithData = lists:zipwith(\n            fun(Fun, {Round, Data, Open, Working}) ->\n                    {Round, Fun, Data, Open, Working}\n            end, Phases, ExtraData),\n    JobOptions = merge_with_default_options(Options, ?DEF_OPTIONS),\n    NewState = #state{\n                  jobid      = JobId\n                  , client   = Client\n                  , master_id   = Master\n                  , phases   = PhasesWithData\n                  , options  = JobOptions\n          },\n    NewState.\n\n-spec is_last_phase(state(), pos_integer()) -> boolean().\nis_last_phase(#state{phases = Phases}, Round) ->\n    Round =:= length(Phases).\n\n-spec get_phase(state(), pos_integer()) -> phase() | false.\nget_phase(#state{phases = Phases}, Round) ->\n    lists:keyfind(Round, 1, Phases).\n\n-spec is_acked_complete(state()) -> boolean().\nis_acked_complete(#state{acked = Interval}) ->\n    intervals:is_all(Interval).\n\n-spec reset_acked(state()) -> state().\nreset_acked(State) ->\n    State#state{acked = intervals:empty()}.\n\n-spec set_acked(state(), intervals:interval()) -> state().\nset_acked(State = #state{acked = Interval}, NewInterval) ->\n    State#state{acked = intervals:union(Interval, NewInterval)}.\n\n-spec interval_processing(state(), intervals:interval(), pos_integer()) ->\n    state().\ninterval_processing(State = #state{phases = Phases}, Interval, Round) ->\n    case lists:keyfind(Round, 1, Phases) of\n        {Round, Fun, ETS, Open, Working} ->\n            NewPhases = lists:keyreplace(Round, 1, Phases,\n                                         {Round, Fun, ETS,\n                                          intervals:minus(Open, Interval),\n                                          intervals:union(Working, Interval)}),\n            ?TRACE(\"start working on ~p new open is ~p~n\", [Interval,\n                                                            intervals:minus(Open,\n                                                                            Interval)]),\n            State#state{phases = NewPhases};\n        false ->\n            State\n    end.\n\n-spec interval_processed(state(), intervals:interval(), pos_integer()) ->\n    state().\ninterval_processed(State = #state{phases = Phases}, Interval, Round) ->\n    case lists:keyfind(Round, 1, Phases) of\n        {Round, Fun, ETS, Open, Working} ->\n            NewPhases = lists:keyreplace(Round, 1, Phases,\n                                         {Round, Fun, ETS,\n                                          Open,\n                                          intervals:minus(Working, Interval)}),\n            State#state{phases = NewPhases};\n        false ->\n            State\n    end.\n\n-spec interval_empty(state(), intervals:interval(), pos_integer()) ->\n    state().\ninterval_empty(State = #state{phases = Phases}, Interval, Round) ->\n    case lists:keyfind(Round, 1, Phases) of\n        {Round, Fun, ETS, Open, Working} ->\n            NewPhases = lists:keyreplace(Round, 1, Phases,\n                                         {Round, Fun, ETS,\n                                          intervals:minus(Open, Interval),\n                                          Working}),\n            State#state{phases = NewPhases};\n        false ->\n            State\n    end.\n\n%% NEXT register new as value creator\n-spec add_data_to_phase(state(), data_list(), intervals:interval(),\n                             pos_integer()) -> state().\nadd_data_to_phase(State = #state{phases = Phases}, NewData,\n                      Interval, Round) ->\n    case lists:keyfind(Round, 1, Phases) of\n        {Round, Fun, ETS, Open, Working} ->\n            %% side effect is used here...only works with ets\n            _ = accumulate_data(NewData, ETS),\n            NextPhase = {Round, Fun, ETS, intervals:union(Open,\n                                                               Interval),\n                         Working},\n            State#state{phases = lists:keyreplace(Round, 1, Phases, NextPhase)};\n        false ->\n            %% someone tries to add data to nonexisting phase...do nothing\n            State\n    end.\n\n-spec accumulate_data([{client_key(), term()}], data()) -> data().\naccumulate_data(Data, List) when is_list(List) ->\n    lists:foldl(fun({K, V}, Acc) ->\n                        HK = ?RT:hash_key(K),\n                        acc_add_element(Acc, {HK, K, V});\n                   ({HK, K, V}, Acc) ->\n                        acc_add_element(Acc, {HK, K, V})\n                end,\n                List,\n                Data);\naccumulate_data(Data, ETS) ->\n    ?TRACE(\"accumulating ~p~n\", [Data]),\n    lists:foldl(fun({K, V}, ETSAcc) ->\n                        HK = ?RT:hash_key(K),\n                        acc_add_element(ETSAcc, {HK, K, V});\n                   ({HK, K, V}, ETSAcc) ->\n                        acc_add_element(ETSAcc, {HK, K, V})\n                end,\n                ETS,\n                Data).\n\n-spec acc_add_element(data(), {?RT:key(), client_key(), term()}) ->\n    data().\nacc_add_element(List, {HK, K, V} = T) when is_list(List) ->\n    case lists:keyfind(HK, 1, List) of\n        false ->\n            case is_list(V) of\n                true ->\n                    [T | List];\n                _ ->\n                    [{HK, K, [V]} | List]\n            end;\n        {HK, K, ExV} ->\n            case is_list(V) of\n                true ->\n                    [{HK, K, V ++ ExV} | List];\n                _ ->\n                    [{HK, K, [V | ExV]} | List]\n            end\n    end;\nacc_add_element(ETS, {HK, K, V}) ->\n    case db_ets:get(ETS, HK) of\n        {} ->\n            case is_list(V) of\n                true ->\n                    db_ets:put(ETS, {HK, K, V});\n                _ ->\n                    db_ets:put(ETS, {HK, K, [V]})\n            end;\n        {HK, K, ExV} ->\n            case is_list(V) of\n                true ->\n                    db_ets:put(ETS, {HK, K, V ++ ExV});\n                _ ->\n                    db_ets:put(ETS, {HK, K, [V | ExV]})\n            end\n    end.\n\n-spec merge_with_default_options(UserOptions::[option()],\n                                 DefaultOptions::[option()]) ->\n    JobOptions::[option()].\nmerge_with_default_options(UserOptions, DefaultOptions) ->\n    %% TODO merge by hand and skip everything that is not in DefaultOptions\n    lists:keymerge(1,\n                   lists:keysort(1, UserOptions),\n                   lists:keysort(1, DefaultOptions)).\n\n%% TODO fix types data as ets or list\n\n-spec clean_up(state()) -> [true].\nclean_up(#state{phases = Phases}) ->\n    lists:map(fun({_R, _Fun, ETS, _Interval, _Working}) ->\n                      db_ets:close(ETS)\n              end, Phases).\n\n-spec get_slide_delta(state(), intervals:interval()) -> {state(), state()}.\nget_slide_delta(State = #state{phases = Phases}, SlideInterval) ->\n    {NewPhases, SlidePhases} =\n    lists:foldl(\n      fun({Nr, Fun, ETS, Open, Working}, {PhaseAcc, SlideAcc}) ->\n              SlideData = lists:foldl(\n                            fun(SimpleInterval, AccI) ->\n                                    db_ets:foldl(ETS,\n                                                 fun(K, Acc) ->\n                                                         Entry = db_ets:get(ETS, K),\n                                                         %% _NewDB = db_ets:delete(ETS, K),\n                                                         [Entry | Acc]\n                                                 end,\n                                                 AccI,\n                                                 SimpleInterval)\n                            end, [],\n                            intervals:get_simple_intervals(SlideInterval)),\n              NewOpen = intervals:minus(Open, SlideInterval),\n              SlideOpen = intervals:intersection(Open, SlideInterval),\n              ?TRACE_SLIDE(\"~p Open: ~p~nSlideInterval: ~p~nNewOpen: ~p~nSlideOpen: ~p~n\",\n                     [self(), Open, SlideInterval, NewOpen, SlideOpen]),\n              {[{Nr, Fun, ETS, NewOpen, Working} | PhaseAcc],\n               [{Nr, Fun, SlideData, SlideOpen, intervals:empty()} | SlideAcc]}\n      end,\n      {[], []},\n      Phases),\n    {State#state{phases = NewPhases}, State#state{phases = SlidePhases}}.\n\n-spec init_slide_state(state()) -> state().\ninit_slide_state(State = #state{phases = Phases, jobid = JobId}) ->\n    PhasesETS = lists:foldl(\n                  fun({Nr, Fun, Data, Open, Working}, AccIn) ->\n                          ETS = db_ets:new(\n                                  lists:flatten(io_lib:format(\"mr_~s_~p\",\n                                                              [JobId, Nr]))\n                                  , [ordered_set]),\n                          _NewETS = db_ets:put(ETS, Data),\n                          [{Nr, Fun, ETS, Open, Working} | AccIn]\n                  end, [], Phases),\n    State#state{phases = PhasesETS}.\n\n-spec merge_states(state(), state()) -> state().\nmerge_states(State = #state{jobid = JobId,\n                            phases = Phases},\n             #state{jobid = JobId,\n                    phases = DeltaPhases}) ->\n    MergedPhases = lists:map(\n                  fun(DeltaPhase) ->\n                          merge_phase_delta(lists:keyfind(element(1, DeltaPhase),\n                                                          1,\n                                                          Phases),\n                                            DeltaPhase)\n                          end, DeltaPhases),\n    ?TRACE_SLIDE(\"~p mr_~p on: received delta: ~p~n\", [self(), JobId, DeltaPhases]),\n    trigger_work(State#state{phases = MergedPhases}, JobId).\n\n-spec merge_phase_delta(ets_phase(), delta_phase()) -> ets_phase().\nmerge_phase_delta({Round, Fun, ETS, Open, Working},\n                  {Round, Fun, Delta, DOpen, _DWorking}) ->\n    %% side effect\n    %% also should be db_ets, but ets can add lists\n    ets:insert(ETS, Delta),\n    {Round, Fun, ETS, intervals:union(Open, DOpen),\n     Working}.\n\n-spec trigger_work(state(), jobid()) -> state().\ntrigger_work(#state{phases = Phases} = State, JobId) ->\n    SmallestOpenPhase = lists:foldl(\n                         fun({Round, _, _, Open, _}, Acc) ->\n                                 case not intervals:is_empty(Open)\n                                      andalso Round < Acc of\n                                     true ->\n                                         Round;\n                                     _ ->\n                                         Acc\n                                 end\n                         end, length(Phases) + 1, Phases),\n    case SmallestOpenPhase of\n        N when N < length(Phases) ->\n            mr:work_on_phase(JobId, State, N);\n        _N ->\n            State\n    end.\n\n-spec tester_is_valid_funterm(fun_term()) -> boolean().\ntester_is_valid_funterm({map, erlanon, Fun}) when is_function(Fun, 1) -> true;\ntester_is_valid_funterm({reduce, erlanon, Fun}) when is_function(Fun, 1) -> true;\ntester_is_valid_funterm({map, jsanon, Fun}) when is_binary(Fun) -> true;\ntester_is_valid_funterm({reduce, jsanon, Fun}) when is_binary(Fun) -> true;\ntester_is_valid_funterm(_) -> false.\n\n-spec tester_create_valid_funterm(erlanon | jsanon, map | reduce) -> fun_term().\ntester_create_valid_funterm(erlanon, map) ->\n    {map, erlanon, fun({\"Foo\", bar}) -> [{\"Foo\", bar}] end};\ntester_create_valid_funterm(erlanon, reduce) ->\n    {reduce, erlanon, fun([{\"Foo\", bar}]) -> [{\"Foo\", bar}] end};\ntester_create_valid_funterm(jsanon, MoR) ->\n    {MoR, jsanon, <<\"some js function\">>}.\n"
  },
  {
    "path": "src/msg_delay.erl",
    "content": "% @copyright 2009-2015 Zuse Institute Berlin,\n%            2009 onScale solutions GmbH\n% @end\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Cheap message delay.\n%%      Instead of using send_after, which is slow in Erlang, as it\n%%      performs a system call, this module allows for a weaker\n%%      message delay.\n%%      You can specify the minimum message delay in seconds and the\n%%      component will send the message sometime afterwards.\n%%      Only local messages inside a VM are supported.\n%%      Internally it triggers itself periodically to schedule sending.\n%% @end\n%% @version $Id$\n-module(msg_delay).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(_X,_Y), ok).\n-behaviour(gen_component).\n\n%% public interface for delayed messages\n-export([send_local/3, send_local/4,\n         send_local_as_client/3, send_local_as_client/4]).\n\n%% public interface for self trigger messages using msg_delay\n-export([send_trigger/2]).\n\n%% functions for gen_component module and supervisor callbacks\n-export([start_link/1, on/2, init/1]).\n\n-include(\"gen_component.hrl\").\n\n% accepted messages of the msg_delay process\n-type message() ::\n    {msg_delay_req, Seconds::pos_integer(), Dest::comm:erl_local_pid(),\n     Msg::comm:message(), comm:send_local_options()} |\n    {msg_delay_periodic}.\n\n% internal state\n-type state() :: {TimeTable :: pdb:tableid(), Round :: non_neg_integer()}.\n\n-spec send_local_as_client(Seconds::non_neg_integer(),\n                           Dest::comm:erl_local_pid(),\n                           Msg::comm:message()) -> ok.\nsend_local_as_client(Seconds, Dest, Msg) ->\n    send_local_as_client(Seconds, Dest, Msg, []).\n\n-spec send_local_as_client(Seconds::non_neg_integer(),\n                           Dest::comm:erl_local_pid(),\n                           Msg::comm:message(), comm:send_local_options())\n        -> ok.\nsend_local_as_client(Seconds, Dest, Msg, Options) ->\n    Delayer = pid_groups:pid_of(clients_group, msg_delay),\n    %% TODO: if infected with proto_sched logging, immediately log msg\n    %% to proto_sched for the 'delayed' messages pool (which is not\n    %% implemented yet) and do *not* deliver to msg_delay process.\n    comm:send_local(Delayer, {msg_delay_req, Seconds, Dest, Msg, Options}).\n\n-spec send_local(Seconds::non_neg_integer(), Dest::comm:erl_local_pid(),\n                 Msg::comm:message()) -> ok.\nsend_local(Seconds, Dest, Msg) ->\n    send_local(Seconds, Dest, Msg, []).\n\n-spec send_local(Seconds::non_neg_integer(), Dest::comm:erl_local_pid(),\n                 Msg::comm:message(), comm:send_local_options()) -> ok.\nsend_local(Seconds, Dest, Msg, Options) ->\n    Delayer = pid_groups:find_a(msg_delay),\n    %% TODO: if infected with proto_sched logging, immediately deliver\n    %% to proto_sched for the 'delayed' messages pool (which is not\n    %% implemented yet) and do *not* deliver to msg_delay process.\n    comm:send_local(Delayer, {msg_delay_req, Seconds, Dest, Msg, Options}).\n\n%% be startable via supervisor, use gen_component\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2,\n                             [], % parameters passed to init\n                             [{pid_groups_join_as, DHTNodeGroup, msg_delay}]).\n\n-spec send_trigger(Seconds::non_neg_integer(), Msg::comm:message()) -> ok.\nsend_trigger(Seconds, Msg) ->\n    %% (1) Triggers are typically periodic messages. We do not want to\n    %%     infect a system forever, so we forbid trace_mpath here.\n    %% (2) We do not need a reply_as for self() in this case, as one can\n    %%     easily put all infos in the message itself, when inside the\n    %%     same process.\n    ?DBG_ASSERT2(erlang:get(trace_mpath) =:= undefined,\n                 send_trigger_infected_with_trace_mpath_or_proto_sched),\n    send_local(Seconds, self(), Msg, [{?quiet}]).\n\n%% userdevguide-begin gen_component:sample\n%% initialize: return initial state.\n-spec init([]) -> state().\ninit([]) ->\n    ?TRACE(\"msg_delay:init for pid group ~p~n\", [pid_groups:my_groupname()]),\n    %% For easier debugging, use a named table (generates an atom)\n    %%TableName = erlang:list_to_atom(pid_groups:group_to_filename(pid_groups:my_groupname()) ++ \"_msg_delay\"),\n    %%TimeTable = pdb:new(TableName, [set, protected, named_table]),\n    %% use random table name provided by ets to *not* generate an atom\n    TimeTable = pdb:new(?MODULE, [set]),\n    comm:send_local(self(), {msg_delay_periodic}),\n    _State = {TimeTable, _Round = 0}.\n\n-spec on(message(), state()) -> state().\non({msg_delay_req, Seconds, Dest, Msg, Options} = _FullMsg,\n   {TimeTable, Counter} = State) ->\n    ?TRACE(\"msg_delay:on(~.0p, ~.0p)~n\", [_FullMsg, State]),\n    Future = trunc(Counter + Seconds),\n    EMsg = case erlang:get(trace_mpath) of\n               undefined -> Msg;\n               PState -> trace_mpath:epidemic_reply_msg(PState, comm:this(), Dest, Msg)\n           end,\n    case pdb:get(Future, TimeTable) of\n        undefined ->\n            pdb:set({Future, [{Dest, EMsg, Options}]}, TimeTable);\n        {_, MsgQueue} ->\n            pdb:set({Future, [{Dest, EMsg, Options} | MsgQueue]}, TimeTable)\n    end,\n    State;\n\n%% periodic trigger\non({msg_delay_periodic} = Trigger, {TimeTable, Counter} = _State) ->\n    ?TRACE(\"msg_delay:on(~.0p, ~.0p)~n\", [Trigger, State]),\n    % triggers are not allowed to be infected!\n    ?DBG_ASSERT2(not trace_mpath:infected(), trigger_infected),\n    _ = case pdb:take(Counter, TimeTable) of\n        undefined -> ok;\n        {_, MsgQueue} ->\n            _ = [ case Msg of\n                      {'$gen_component', trace_mpath, PState, _From, _To, OrigMsg} ->\n                          case element(2, PState) of %% element 2 is the logger\n                              {proto_sched, _} ->\n                                  log:log(\"msg_delay: proto_sched not ready for delayed messages, so dropping this message: ~p\", [OrigMsg]),\n                                  %% these messages should not be\n                                  %% accepted to the database of\n                                  %% msg_delay anyway, but instead\n                                  %% should be immediately delivered\n                                  %% to proto_sched for the 'delayed'\n                                  %% messages pool (which is not\n                                  %% implemented yet) (see send_local,\n                                  %% send_local_as_client)\n                                  %% erlang:throw(redirect_proto_sched_msgs_at_submission_please)\n                                  ok;\n                              _ ->\n                                  trace_mpath:start(PState),\n                                  comm:send_local(Dest, OrigMsg, Options),\n                                  trace_mpath:stop()\n                          end;\n                      _ ->\n                          ?DBG_ASSERT2(not trace_mpath:infected(), infected_with_uninfected_msg),\n                          comm:send_local(Dest, Msg, Options)\n                  end || {Dest, Msg, Options} <- MsgQueue ]\n    end,\n    _ = comm:send_local_after(1000, self(), Trigger),\n    {TimeTable, Counter + 1};\n\non({web_debug_info, Requestor}, {TimeTable, Counter} = State) ->\n    KeyValueList =\n        [{\"queued messages (in 0-10s, messages):\", \"\"} |\n         [begin\n              Future = trunc(Counter + Seconds),\n              Queue = case pdb:get(Future, TimeTable) of\n                          undefined -> none;\n                          {_, Q}    -> Q\n                      end,\n              {webhelpers:safe_html_string(\"~p\", [Seconds]),\n               webhelpers:safe_html_string(\"~p\", [Queue])}\n          end || Seconds <- lists:seq(0, 10)]],\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    State.\n%% userdevguide-end gen_component:sample\n"
  },
  {
    "path": "src/msg_queue.erl",
    "content": "%  @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Implements helper functions for queuing messages and sending them\n%%         in the order they have been queued.\n%% @end\n%% @version $Id$\n-module(msg_queue).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-export_type([msg_queue/0]).\n\n-export([new/0, is_empty/1, add/2, send/1]).\n\n-include(\"scalaris.hrl\").\n\n-type msg_queue() :: [comm:message()].\n\n%% @doc Creates a new message queue.\n-spec new() -> msg_queue().\nnew() -> [].\n\n%% @doc Checks whether the message queue is empty.\n-spec is_empty(msg_queue()) -> boolean().\nis_empty(QueuedMessages) ->\n    QueuedMessages =:= [].\n\n%% @doc Adds a message to a given queue.\n-spec add(msg_queue(), comm:message()) -> msg_queue().\nadd(QueuedMessages, NewMessage) ->\n    [NewMessage | QueuedMessages].\n\n%% @doc Sends queued messages to the process calling the method, i.e. self(), in\n%%      the order the messages have been queued.\n-spec send(msg_queue()) -> ok.\nsend(QueuedMessages) ->\n    lists:foldr(fun(Msg, _) ->\n                        comm:send_local(self(), Msg)\n                end, ok, QueuedMessages).\n"
  },
  {
    "path": "src/mymaps.erl",
    "content": "%% @copyright 2016-2019 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc Thin wrapper around the maps module falling back to gb_trees if maps\n%%      are not available (Erlang versions below 17.0).\n%%      (extend as needed)\n%% @version $Id$\n-module(mymaps).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-export([new/0,\n         get/2, get/3, take/2, find/2, is_key/2,\n         put/3, remove/2, update/3,\n         from_list/1, to_list/1,\n         keys/1, values/1,\n         size/1]).\n\n-export_type([mymap/0]).\n\n-ifdef(with_maps).\n-type mymap() :: map().\n-else.\n-type mymap() :: gb_trees:tree().\n-endif.\n\n-spec get(Key::term(), Map::mymap()) -> Value::term().\n-ifdef(with_maps).\nget(Key, Map) -> maps:get(Key, Map).\n-else.\nget(Key, Map) ->\n    case gb_trees:lookup(Key, Map) of\n        {value, Value} -> Value;\n        none -> erlang:error({badkey, Key})\n    end.\n-endif.\n\n\n-spec get(Key::term(), Map::mymap(), Default) -> Value | Default\n        when is_subtype(Value, term()), is_subtype(Default, term()).\n-ifdef(with_maps).\nget(Key, Map, Default) -> maps:get(Key, Map, Default).\n-else.\nget(Key, Map, Default) ->\n    case gb_trees:lookup(Key, Map) of\n        {value, Value} -> Value;\n        none -> Default\n    end.\n-endif.\n\n-spec take(Key::term(), Map::mymap()) -> {Value::term(),  Map2::mymap()} | error.\n-ifdef(with_maps).\ntake(Key, Map) ->\n    try maps:take(Key, Map) of\n        Value -> Value\n    catch error:undef ->\n            %% no longer necessary when we drop support for Erlang < 19.0\n            %% Erlang 18.0 to 18.3 do not have a maps:take/2 function\n            %% get and delete can do the same logic\n            try maps:get(Key, Map) of\n                X ->\n                    NewMap = maps:remove(Key, Map),\n                    {X, NewMap}\n            catch error:_ -> error\n            end\n    end.\n-else.\ntake(Key, Map) ->\n    try gb_trees:take(Key, Map) of\n        Value -> Value\n    catch\n        error:_ -> error\n    end.\n-endif.\n\n-spec find(Key::term(), Map::mymap()) -> {ok, Value::term()} | error.\n-ifdef(with_maps).\nfind(Key, Map) -> maps:find(Key, Map).\n-else.\nfind(Key, Map) ->\n    case gb_trees:lookup(Key, Map) of\n        {value, Value} -> {ok, Value};\n        none -> error\n    end.\n-endif.\n\n-spec from_list([{Key::term(), Value::term()}]) -> Map::mymap().\n-ifdef(with_maps).\nfrom_list(List) -> maps:from_list(List).\n-else.\nfrom_list(List) -> gb_trees:from_orddict(orddict:from_list(List)).\n-endif.\n\n-spec is_key(Key::term(), Map::mymap()) -> boolean().\n-ifdef(with_maps).\nis_key(Key, Map) -> maps:is_key(Key, Map).\n-else.\nis_key(Key, Map) -> gb_trees:is_defined(Key, Map).\n-endif.\n\n-spec keys(Map::mymap()) -> [Key::term()].\n-ifdef(with_maps).\nkeys(Map) -> maps:keys(Map).\n-else.\nkeys(Map) -> gb_trees:keys(Map).\n-endif.\n\n-spec new() -> mymap().\n-ifdef(with_maps).\nnew() -> maps:new().\n-else.\nnew() -> gb_trees:empty().\n-endif.\n\n-spec put(Key::term(), Value::term(), Map1::mymap()) -> Map2::mymap().\n-ifdef(with_maps).\nput(Key, Value, Map) -> maps:put(Key, Value, Map).\n-else.\nput(Key, Value, Map) -> gb_trees:enter(Key, Value, Map).\n-endif.\n\n-spec remove(Key::term(), Map1::mymap()) -> Map2::mymap().\n-ifdef(with_maps).\nremove(Key, Map) -> maps:remove(Key, Map).\n-else.\nremove(Key, Map) -> gb_trees:delete_any(Key, Map).\n-endif.\n\n-spec to_list(Map::mymap()) -> [{Key::term(), Value::term()}].\n-ifdef(with_maps).\nto_list(Map) -> maps:to_list(Map).\n-else.\nto_list(Map) -> gb_trees:to_list(Map).\n-endif.\n\n-spec update(Key::term(), Value::term(), Map1::mymap()) -> Map2::mymap().\n-ifdef(with_maps).\nupdate(Key, Value, Map) -> maps:update(Key, Value, Map).\n-else.\nupdate(Key, Value, Map) -> gb_trees:update(Key, Value, Map).\n-endif.\n\n-spec values(Map::mymap()) -> [Value::term()].\n-ifdef(with_maps).\nvalues(Map) -> maps:values(Map).\n-else.\nvalues(Map) -> gb_trees:values(Map).\n-endif.\n\n-spec size(Map::mymap()) -> non_neg_integer().\n-ifdef(with_maps).\nsize(Map) -> maps:size(Map).\n-else.\nsize(Map) -> gb_trees:size(Map).\n-endif.\n"
  },
  {
    "path": "src/nifs/clocks.cpp",
    "content": "#include <sys/types.h>\n#include <sys/stat.h>\n#include <fcntl.h>\n#include <unistd.h>\n\n#include <cassert>\n#include <cstdlib>\n#include <iostream>\n#include <string>\n\n#include \"erl_nif.h\"\n\n#ifdef __MACH__\n#include <mach/mach.h>\n#include <mach/clock.h>\n#include <mach/mach_time.h>\n#include <mach/clock_types.h>\n#else\n#include <time.h>\n#endif\n\nextern \"C\" {\n\nERL_NIF_TERM timespec2term(ErlNifEnv* env, struct timespec tp) {\nreturn enif_make_tuple(env, 2,\n                         enif_make_int64(env, tp.tv_sec),\n                         enif_make_int64(env, tp.tv_nsec));\n}\n\n#ifdef __MACH__\nstatic ERL_NIF_TERM get_monotonic_clock(ErlNifEnv* env, int argc,\n                                        const ERL_NIF_TERM argv[]) {\n  clock_serv_t clk_srv;\n  kern_return_t res;\n  mach_timespec_t time_spec;\n\n  host_get_clock_service(mach_host_self(),\n                         SYSTEM_CLOCK,\n                         &clk_srv);\n  res = clock_get_time(clk_srv, &time_spec);\n  if (res != KERN_SUCCESS)\n    return enif_make_atom(env, \"failed\");\n  mach_port_deallocate(mach_task_self(), clk_srv);\n\n  return enif_make_tuple(env, 2,\n                         enif_make_int64(env, time_spec.tv_sec),\n                         enif_make_int64(env, time_spec.tv_nsec));\n}\n\nstatic ERL_NIF_TERM get_monotonic_clock_res(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {\n  clock_serv_t clk_srv;\n  kern_return_t res;\n  natural_t attr[1];\n  mach_msg_type_number_t cnt;\n\n  host_get_clock_service(mach_host_self(),\n                         SYSTEM_CLOCK,\n                         &clk_srv);\n\n  cnt = sizeof(attr);\n  res = clock_get_attributes(clk_srv, CLOCK_GET_TIME_RES, (clock_attr_t) attr, &cnt);\n  if (res != KERN_SUCCESS || cnt != 1)\n    return enif_make_atom(env, \"failed\");\n\n  mach_port_deallocate(mach_task_self(), clk_srv);\n\n  return enif_make_tuple(env, 2,\n                         enif_make_int64(env, 0),\n                         enif_make_int64(env, attr[0]));\n}\n\nstatic ERL_NIF_TERM get_realtime_clock(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {\n  clock_serv_t clk_srv;\n  kern_return_t res;\n  mach_timespec_t time_spec;\n\n  host_get_clock_service(mach_host_self(),\n                         REALTIME_CLOCK,\n                         &clk_srv);\n  res = clock_get_time(clk_srv, &time_spec);\n  if (res != KERN_SUCCESS)\n    return enif_make_atom(env, \"failed\");\n  mach_port_deallocate(mach_task_self(), clk_srv);\n\n  return enif_make_tuple(env, 2,\n                         enif_make_int64(env, time_spec.tv_sec),\n                         enif_make_int64(env, time_spec.tv_nsec));\n}\n\nstatic ERL_NIF_TERM get_realtime_clock_res(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {\n  clock_serv_t clk_srv;\n  kern_return_t res;\n  natural_t attr[1];\n  mach_msg_type_number_t cnt;\n\n  host_get_clock_service(mach_host_self(),\n                         REALTIME_CLOCK,\n                         &clk_srv);\n\n  cnt = sizeof(attr);\n  res = clock_get_attributes(clk_srv, CLOCK_GET_TIME_RES, (clock_attr_t) attr, &cnt);\n  if (res != KERN_SUCCESS || cnt != 1)\n    return enif_make_atom(env, \"failed\");\n\n  mach_port_deallocate(mach_task_self(), clk_srv);\n\n  return enif_make_tuple(env, 2,\n                         enif_make_int64(env, 0),\n                         enif_make_int64(env, attr[0]));\n}\n\nstatic ERL_NIF_TERM get_ptp0_clock(ErlNifEnv* env, int argc,\n                                       const ERL_NIF_TERM argv[]) {\n  return enif_make_atom(env, \"not_supported\");\n}\n\nstatic ERL_NIF_TERM get_ptp1_clock(ErlNifEnv* env, int argc,\n                                       const ERL_NIF_TERM argv[]) {\n  return enif_make_atom(env, \"not_supported\");\n}\n\nstatic ERL_NIF_TERM get_ptp2_clock(ErlNifEnv* env, int argc,\n                                       const ERL_NIF_TERM argv[]) {\n  return enif_make_atom(env, \"not_supported\");\n}\n\nstatic ERL_NIF_TERM get_ptp0_clock_res(ErlNifEnv* env, int argc,\n                                       const ERL_NIF_TERM argv[]) {\n  return enif_make_atom(env, \"not_supported\");\n}\n\nstatic ERL_NIF_TERM get_ptp1_clock_res(ErlNifEnv* env, int argc,\n                                       const ERL_NIF_TERM argv[]) {\n  return enif_make_atom(env, \"not_supported\");\n}\n\nstatic ERL_NIF_TERM get_ptp2_clock_res(ErlNifEnv* env, int argc,\n                                       const ERL_NIF_TERM argv[]) {\n  return enif_make_atom(env, \"not_supported\");\n}\n\n#else\nstatic ERL_NIF_TERM get_time(ErlNifEnv* env, clockid_t clk_id) {\n  struct timespec tp;\n\n  int res = clock_gettime(clk_id, &tp);\n  if(res != 0)\n    return enif_make_atom(env, \"failed\");\n\n  return timespec2term(env, tp);\n}\n\nstatic ERL_NIF_TERM get_ptp_time(ErlNifEnv* env, std::string dev) {\n  return enif_make_atom(env, \"not_supported\");\n\n  struct timespec tp;\n\n  int fd = ::open(dev.c_str(), O_RDONLY);\n  if(fd <= 0) {\n    perror(\"open: \");\n    return enif_make_atom(env, \"failed\");\n  }\n\n  int res = ::clock_gettime(fd, &tp);\n  if(res != 0) {\n    perror(\"clock_gettime: \");\n    return enif_make_atom(env, \"failed\");\n  }\n\n  res = ::close(fd);\n  if(res != 0) {\n    perror(\"close: \");\n    return enif_make_atom(env, \"failed\");\n  }\n\n  return timespec2term(env, tp);\n}\n\nstatic ERL_NIF_TERM get_ptp_resolution(ErlNifEnv* env, std::string dev) {\n  return enif_make_atom(env, \"not_supported\");\n\n  struct timespec tp;\n\n  int fd = open(dev.c_str(), O_RDONLY);\n  if(fd < 0)\n    return enif_make_atom(env, \"failed\");\n\n  int res = clock_getres(fd, &tp);\n  if(res != 0)\n    return enif_make_atom(env, \"failed\");\n\n  res = close(fd);\n  if(res != 0)\n    return enif_make_atom(env, \"failed\");\n\n  return timespec2term(env, tp);\n}\n\nstatic ERL_NIF_TERM get_resolution(ErlNifEnv* env, clockid_t clk_id) {\n  struct timespec tp;\n\n  int res = clock_getres(clk_id, &tp);\n  if(res != 0)\n    return enif_make_atom(env, \"failed\");\n\n  return timespec2term(env, tp);\n}\n\nstatic ERL_NIF_TERM get_monotonic_clock_res(ErlNifEnv* env, int argc,\n                                            const ERL_NIF_TERM argv[]) {\n  return get_resolution(env, CLOCK_MONOTONIC);\n}\n\n\nstatic ERL_NIF_TERM get_monotonic_clock(ErlNifEnv* env, int argc,\n                                        const ERL_NIF_TERM argv[]) {\n  return get_time(env, CLOCK_MONOTONIC);\n}\n\nstatic ERL_NIF_TERM get_realtime_clock(ErlNifEnv* env, int argc,\n                                       const ERL_NIF_TERM argv[]) {\n  return get_time(env, CLOCK_REALTIME);\n}\n\nstatic ERL_NIF_TERM get_realtime_clock_res(ErlNifEnv* env, int argc,\n                                           const ERL_NIF_TERM argv[]) {\n  return get_resolution(env, CLOCK_REALTIME);\n}\n\nstatic ERL_NIF_TERM get_ptp0_clock(ErlNifEnv* env, int argc,\n                                       const ERL_NIF_TERM argv[]) {\n  return get_ptp_time(env, \"/sys/class/ptp/ptp0/dev\");\n}\n\nstatic ERL_NIF_TERM get_ptp1_clock(ErlNifEnv* env, int argc,\n                                       const ERL_NIF_TERM argv[]) {\n  return get_ptp_time(env, \"/sys/class/ptp/ptp1/dev\");\n}\n\nstatic ERL_NIF_TERM get_ptp2_clock(ErlNifEnv* env, int argc,\n                                       const ERL_NIF_TERM argv[]) {\n  return get_ptp_time(env, \"/sys/class/ptp/ptp2/dev\");\n}\n\nstatic ERL_NIF_TERM get_ptp0_clock_res(ErlNifEnv* env, int argc,\n                                       const ERL_NIF_TERM argv[]) {\n  return get_ptp_resolution(env, \"/sys/class/ptp/ptp0/dev\");\n}\n\nstatic ERL_NIF_TERM get_ptp1_clock_res(ErlNifEnv* env, int argc,\n                                       const ERL_NIF_TERM argv[]) {\n  return get_ptp_resolution(env, \"/sys/class/ptp/ptp1/dev\");\n}\n\nstatic ERL_NIF_TERM get_ptp2_clock_res(ErlNifEnv* env, int argc,\n                                       const ERL_NIF_TERM argv[]) {\n  return get_ptp_resolution(env, \"/sys/class/ptp/ptp2/dev\");\n}\n#endif\n\n/**\n * the list of nifs\n */\nstatic ErlNifFunc nif_funcs[] = {\n  {\"get_monotonic_clock\", 0, get_monotonic_clock},\n  {\"get_monotonic_clock_res\", 0, get_monotonic_clock_res},\n  {\"get_realtime_clock\", 0, get_realtime_clock},\n  {\"get_realtime_clock_res\", 0, get_realtime_clock_res},\n  {\"get_ptp0_clock\", 0, get_ptp0_clock},\n  {\"get_ptp1_clock\", 0, get_ptp1_clock},\n  {\"get_ptp2_clock\", 0, get_ptp2_clock},\n  {\"get_ptp0_clock_res\", 0, get_ptp0_clock_res},\n  {\"get_ptp1_clock_res\", 0, get_ptp1_clock_res},\n  {\"get_ptp2_clock_res\", 0, get_ptp2_clock_res}\n};\n\n/**\n * initialise the nif module\n */\nERL_NIF_INIT(clocks,nif_funcs,NULL,NULL,NULL,NULL)\n\n}\n"
  },
  {
    "path": "src/nifs/prime.cpp",
    "content": "#include <stdio.h>\n#include <string.h>\n#include <stdint.h>\n#include <math.h>\n\n#include <erl_nif.h>\n\n// adapted from http://www.algorithmist.com/index.php/Prime_Sieve_of_Eratosthenes.c\n\nextern \"C\" {\n\n#define GET(sieve,b) ((sieve[(b)>>5]>>((b)&31))&1)\n\nuint32_t *make_sieve(uint64_t maxn) {\n  const size_t P1 = (maxn + 63) / 64; /* = ceil(MAXN/64) */\n  const uint64_t P2 = (maxn + 1) / 2; /* = ceil(MAXN/2) */\n  const uint32_t P3 = ceil(ceil(sqrt(maxn)) / 2); /* = ceil(ceil(sqrt(MAXN))/2) */\n  size_t sieve_size = sizeof(uint32_t)*P1;\n  uint32_t *sieve = (uint32_t*) malloc(sieve_size);\n  memset(sieve, 0, sieve_size);\n  for (uint32_t k = 1; k <= P3; k++) {\n    if (GET(sieve,k) == 0) {\n      for(uint32_t j = 2*k+1, i = 2*k*(k+1); i < P2; i += j) {\n        sieve[i>>5] |= 1<<(i&31);\n      }\n    }\n  }\n  return sieve;\n}\n\nbool sieve_is_prime(const uint32_t *sieve, int p) {\n  return p==2 || (p>2 && (p&1)==1 && (GET(sieve,(p-1)>>1)==0));\n}\n\n\n/**\n * @brief Returns the first prime larger than or equal to N.\n *\n * @param env erlang nif environment\n * @param argc function arity (only one allowed)\n * @param argv function parameters (only one parameter: N)\n *\n * @return a prime larger than the first parameter\n */\nstatic ERL_NIF_TERM get_nearest(ErlNifEnv* env, int argc,\n                                const ERL_NIF_TERM argv[]) {\n  ErlNifUInt64 i;\n  enif_get_uint64(env, argv[0], &i);\n  // from https://en.wikipedia.org/wiki/Bertrand%27s_postulate#Better_results\n  // Lowell Schoenfeld showed that for n ≥ 2010760, there is always a prime between n and (1 + 1/16597)n.\n  const uint64_t size = i >= 2010760 ? i + (i + 16596) / 16597 : 2010760;\n  const uint32_t *sieve = make_sieve(size);\n  // search for the prime\n  for (++i; i <= size; ++i) {\n    if (sieve_is_prime(sieve,i)) {\n      free(const_cast<uint32_t *>(sieve));\n      return enif_make_uint64(env, i);\n    }\n  }\n  free(const_cast<uint32_t *>(sieve));\n  // this should not happen ever!\n  return enif_make_badarg(env);\n}\n\n/**\n * @brief Checks whether the given number N is prime.\n *\n * @param env erlang nif environment\n * @param argc function arity (only one allowed)\n * @param argv function parameters (only one parameter: N)\n *\n * @return \"true\" or \"false\" atom\n */\nstatic ERL_NIF_TERM is_prime(ErlNifEnv* env, int argc,\n                             const ERL_NIF_TERM argv[]) {\n  ErlNifUInt64 i;\n  enif_get_uint64(env, argv[0], &i);\n  const uint32_t *sieve = make_sieve(i);\n  const bool isprime = sieve_is_prime(sieve, i);\n  free(const_cast<uint32_t *>(sieve));\n  return enif_make_atom(env, isprime ? \"true\" : \"false\");\n}\n\n/**\n * the list of nifs\n */\nstatic ErlNifFunc nif_funcs[] = {\n  {\"get_nearest\", 1, get_nearest},\n  {\"is_prime\", 1, is_prime}\n};\n\n/**\n * upgrade is called when the NIF library is loaded and there is old code of this module with a loaded NIF library\n */\nint upgrade(ErlNifEnv* env, void** priv_data, void** old_priv_data, ERL_NIF_TERM load_info) {\n  return 0;\n}\n\n/**\n * initialise the nif module\n */\nERL_NIF_INIT(prime,nif_funcs,NULL,NULL,upgrade,NULL)\n\n}\n"
  },
  {
    "path": "src/node.erl",
    "content": "%  @copyright 2007-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    node data structure + functions\n\n%% @version $Id$\n-module(node).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([id/1, id_version/1, pidX/1, yawsPort/1,\n         new/3, null/0, is_valid/1,\n         same_process/2, is_newer/2, newer/2, is_me/1,\n         update_id/2]).\n%% intervals between nodes\n-export([mk_interval_between_ids/2, mk_interval_between_nodes/2]).\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-export_type([node_type/0]).\n\n-record(node, {pid        = ?required(node, pid)        :: comm:mypid(),\n               id         = ?required(node, id)         :: ?RT:key(),\n               id_version = ?required(node, id_version) :: non_neg_integer(),\n               yaws_port  = config:read(yaws_port)      :: non_neg_integer()}).\n-opaque(node_type() :: #node{}).\n\n%% @doc Creates a new node.\n-spec new(Pid::comm:mypid(), Id::?RT:key(), IdVersion::non_neg_integer()) -> node_type().\nnew(Pid, Id, IdVersion) -> #node{pid=Pid, id=Id, id_version=IdVersion}.\n\n%% @doc Creates an invalid node.\n-spec null() -> null.\nnull() -> null.\n\n%% @doc Gets the pid of the node.\n-spec pidX(node_type()) -> comm:mypid().\npidX(#node{pid=PID}) -> PID.\n\n-spec yawsPort(node_type()) -> non_neg_integer().\nyawsPort(#node{yaws_port = Port}) -> Port.\n\n%% @doc Gets the node's ID.\n-spec id(node_type()) -> ?RT:key().\nid(#node{id=Id}) -> Id.\n\n%% @doc Gets the version of the node's ID.\n-spec id_version(node_type()) -> non_neg_integer().\nid_version(#node{id_version=IdVersion}) -> IdVersion.\n\n%% @doc Checks whether the given parameter is a valid node.\n-spec is_valid(node_type()) -> true;\n               (null | unknown) -> false.\nis_valid(X) when is_record(X, node) -> true;\nis_valid(_) -> false.\n\n%% @doc Checks whether two nodes are the same, i.e. contain references to the.\n%%      same process.\n-spec same_process(node_type() | comm:mypid() | pid(), node_type()) -> boolean();\n                  (node_type(), comm:mypid() | pid()) -> boolean();\n                  (null | unknown, node_type() | comm:mypid() | pid()) -> false;\n                  (node_type() | comm:mypid() | pid(), null | unknown) -> false.\nsame_process(Node1, Node2) when is_record(Node1, node) andalso is_record(Node2, node) ->\n    pidX(Node1) =:= pidX(Node2);\nsame_process(Node1, Node2) when (Node1 =:= null) orelse (Node1 =:= unknown) orelse\n                           (Node2 =:= null) orelse (Node2 =:= unknown) ->\n    false;\nsame_process(Pid1, Node2) when is_record(Node2, node) andalso is_pid(Pid1) ->\n    comm:make_global(Pid1) =:= pidX(Node2);\nsame_process(Pid1, Node2) when is_record(Node2, node) ->\n    Pid1 =:= pidX(Node2);\nsame_process(Node1, Pid2) when is_record(Node1, node) andalso is_pid(Pid2)->\n    pidX(Node1) =:= comm:make_global(Pid2);\nsame_process(Node1, Pid2) when is_record(Node1, node) ->\n    pidX(Node1) =:= Pid2.\n\n%% @doc Checks whether the given node is the same as the node associated with\n%%      the requesting process.\n-spec is_me(node_type() | null | unknown) -> boolean().\nis_me(Node) ->\n    same_process(Node, pid_groups:get_my(dht_node)).\n\n%% @doc Determines whether Node1 is a newer instance of Node2.\n%%      Note: Both nodes need to share the same PID, otherwise an exception of\n%%      type 'throw' is thrown!\n-spec is_newer(Node1::node_type(), Node2::node_type()) -> boolean().\nis_newer(#node{pid=PID, id=Id1, id_version=IdVersion1},\n         #node{pid=PID, id=Id2, id_version=IdVersion2}) ->\n    if\n        (IdVersion1 > IdVersion2) -> true;\n        IdVersion1 < IdVersion2 -> false;\n        Id1 =:= Id2 -> false;\n        true -> throw('got two nodes with same IDversion but different ID')\n    end.\n\n%% @doc Creates an interval that covers all keys a node with MyKey is\n%%      responsible for if his predecessor has PredKey, i.e. (PredKey, MyKey]\n-spec mk_interval_between_ids(PredKey::?RT:key(), MyKey::?RT:key())\n                             -> intervals:interval().\nmk_interval_between_ids(Key, Key) ->\n    intervals:all();\nmk_interval_between_ids(PredKey, MyKey) ->\n    intervals:new('(', PredKey, MyKey, ']').\n\n%% @doc Creates an interval that covers all keys a node is responsible for given\n%%      his predecessor, i.e. (node:id(PredKey), node:id(MyKey)]\n-spec mk_interval_between_nodes(Pred::node:node_type(),\n                                Node::node:node_type())\n                               -> intervals:interval().\nmk_interval_between_nodes(Pred, Node) ->\n    mk_interval_between_ids(node:id(Pred), node:id(Node)).\n\n%% @doc Determines the newer instance of two representations of the same node.\n%%      Note: Both nodes need to share the same PID, otherwise an exception of\n%%      type 'throw' is thrown!\n-spec newer(node_type(), node_type()) -> node_type().\nnewer(Node1, Node2) ->\n    case is_newer(Node1, Node2) of\n        true -> Node1;\n        _    -> Node2\n    end.\n\n%% @doc Updates the node's id and increases the id's version accordingly. \n-spec update_id(Node::node_type(), NewId::?RT:key()) -> node_type().\nupdate_id(Node = #node{id_version=IdVersion}, NewId) ->\n    Node#node{id=NewId, id_version=IdVersion + 1}.\n"
  },
  {
    "path": "src/node_details.erl",
    "content": "%  @copyright 2007-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Node summary for statistics\n%% @end\n%% @version $Id$\n-module(node_details).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([new/0, new/9,\n         contains/2, get/2, set/3]).\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-export_type([node_details/0, node_details_name/0,\n              load/0, load2/0, load3/0,\n              hostname/0, rt_size/0, message_log/0, memory/0]).\n\n-type(load() :: integer()).\n-type(load2() :: lb_stats:load()).\n-type(load3() :: lb_stats:load()).\n-type(hostname() :: string()).\n-type(rt_size() :: integer()).\n-type(message_log() :: any()).\n-type(memory() :: non_neg_integer()).\n\n-type(node_details_name() :: predlist | pred | node | my_range | succ |\n                             succlist | load | hostname | rt_size |\n                             message_log | memory | new_key | db | is_leaving).\n\n-record(node_details, {predlist = ?required(node_details, predlist) :: nodelist:non_empty_snodelist(),\n                       node     = ?required(node_details, node)     :: node:node_type(),\n                       succlist = ?required(node_details, succlist) :: nodelist:non_empty_snodelist(),\n                       load     = ?required(node_details, load)     :: load(),\n                       load2    = ?required(node_details, load2)    :: load2(),\n                       load3    = ?required(node_details, load3)    :: load3(),\n                       hostname = ?required(node_details, hostname) :: hostname(),\n                       rt_size  = ?required(node_details, rt_size)  :: rt_size(),\n                       memory   = ?required(node_details, memory)   :: memory()}).\n-type(node_details_record() :: #node_details{}).\n\n-type(node_details_list() ::\n    [{predlist, nodelist:non_empty_snodelist()} |\n     {node, node:node_type()} |\n     {my_range, intervals:interval()} |\n     {succlist, nodelist:non_empty_snodelist()} |\n     {load, load()} |\n     {load2, load2()} |\n     {load3, load3()} |\n     {hostname, hostname()} |\n     {rt_size, rt_size()} |\n     {message_log, message_log()} |\n     {memory, memory()} |\n     {new_key, ?RT:key()} |\n     {db, db_dht:db()} | % only useful locally for read access!\n     {is_leaving, boolean()}]).\n\n-opaque(node_details() :: node_details_record() | node_details_list()).\n\n%% @doc Creates an empty node details object.\n-spec new() -> node_details().\nnew() -> [].\n\n%% @doc Creates a new node details object with the given data.\n-spec new(PredList::nodelist:non_empty_snodelist(), Node::node:node_type(), SuccList::nodelist:non_empty_snodelist(), Load::load(), Load2::load2(), Load3::load3(), Hostname::hostname(), RTSize::rt_size(), Memory::memory()) -> node_details().\nnew(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory) ->\n    #node_details{\n     predlist = PredList,\n     node = Node,\n     succlist = SuccList,\n     load = Load,\n     load2 = Load2,\n     load3 = Load3,\n     hostname = Hostname,\n     rt_size = RTSize,\n     memory = Memory\n    }.\n\n%% @doc Converts a node details record into a list (only for internal use!).\n-spec to_list(node_details_record()) -> node_details_list().\nto_list(#node_details{predlist=PredList, node=Me, succlist=SuccList, load=Load, load2=Load2, load3=Load3,\n  hostname=HostName, rt_size=RTSize, memory=Memory} = _NodeDetails) ->\n    [{predlist, PredList},\n     {node, Me},\n     {succlist, SuccList},\n     {load, Load},\n     {load2, Load2},\n     {load3, Load3},\n     {hostname, HostName},\n     {rt_size, RTSize},\n     {memory, Memory}].\n\n%% @doc Adds the given data to the node list object.\n%%      Beware: Setting pred/succ will overwrite predlist/succlist!\n%%      (pred and succ are only shortcuts for hd(predlist)/hd(succlist))\n-spec set(node_details(), predlist, nodelist:non_empty_snodelist()) -> node_details();\n          (node_details(), pred, node:node_type()) -> node_details();\n          (node_details(), node, node:node_type()) -> node_details();\n          (node_details(), my_range, intervals:interval()) -> node_details();\n          (node_details(), is_leaving, boolean()) -> node_details();\n          (node_details(), succ, node:node_type()) -> node_details();\n          (node_details(), succlist, nodelist:non_empty_snodelist()) -> node_details();\n          (node_details(), db, db_dht:db()) -> node_details();\n          (node_details(), load, load()) -> node_details();\n          (node_details(), load2, load2()) -> node_details();\n          (node_details(), load3, load3()) -> node_details();\n          (node_details(), hostname, hostname()) -> node_details();\n          (node_details(), rt_size, rt_size()) -> node_details();\n          (node_details(), message_log, message_log()) -> node_details();\n          (node_details(), memory, memory()) -> node_details();\n          (node_details(), new_key, ?RT:key()) -> node_details().\nset(NodeDetails, Key, Value) when is_record(NodeDetails, node_details) ->\n    case Key of\n        % record members:\n        predlist -> NodeDetails#node_details{predlist = Value};\n        node -> NodeDetails#node_details{node = Value};\n        succlist -> NodeDetails#node_details{succlist = Value};\n        load -> NodeDetails#node_details{load = Value};\n        load2 -> NodeDetails#node_details{load2 = Value};\n        load3 -> NodeDetails#node_details{load3 = Value};\n        hostname -> NodeDetails#node_details{hostname = Value};\n        rt_size -> NodeDetails#node_details{rt_size = Value};\n        memory -> NodeDetails#node_details{memory = Value};\n        succ -> NodeDetails#node_details{succlist = [Value]};\n        pred -> NodeDetails#node_details{predlist = [Value]};\n        % list members:\n        my_range -> [{Key, Value} | to_list(NodeDetails)];\n        is_leaving -> [{Key, Value} | to_list(NodeDetails)];\n        message_log -> [{Key, Value} | to_list(NodeDetails)];\n        new_key -> [{Key, Value} | to_list(NodeDetails)];\n        db -> [{Key, Value} | to_list(NodeDetails)]\n    end;\nset(NodeDetails, Key, Value) when is_list(NodeDetails) ->\n    case Key of\n        pred -> lists:keystore(predlist, 1, NodeDetails, {predlist, [Value]});\n        succ -> lists:keystore(succlist, 1, NodeDetails, {succlist, [Value]});\n        _    -> lists:keystore(Key, 1, NodeDetails, {Key, Value})\n    end.\n\n%% @doc Checks whether the given data is available in a node details object.\n-spec contains(node_details(), node_details_name()) -> boolean().\ncontains(NodeDetails, Key) when is_record(NodeDetails, node_details) ->\n    lists:member(Key, [predlist, pred, node, succlist, succ, load, hostname,\n                       rt_size, memory]);\ncontains(NodeDetails, Key) when is_list(NodeDetails) ->\n    case Key of\n        pred -> lists:keymember(predlist, 1, NodeDetails);\n        succ -> lists:keymember(succlist, 1, NodeDetails);\n        _    -> lists:keymember(Key, 1, NodeDetails)\n    end.\n\n%% @doc Gets the value of the given data in a node details object. Will throw\n%%      an exception if the value can not be located.\n-spec get(node_details(), predlist | succlist) -> nodelist:non_empty_snodelist();\n          (node_details(), pred | node | succ) -> node:node_type();\n          (node_details(), my_range) -> intervals:interval();\n          (node_details(), is_leaving) -> boolean();\n          (node_details(), db) -> db_dht:db();\n          (node_details(), load) -> load();\n          (node_details(), load2) -> load2();\n          (node_details(), load3) -> load3();\n          (node_details(), hostname) -> hostname();\n          (node_details(), rt_size) -> rt_size();\n          (node_details(), message_log) -> message_log();\n          (node_details(), memory) -> memory();\n          (node_details(), new_key) -> ?RT:key().\nget(#node_details{predlist=PredList, node=Me, succlist=SuccList, load=Load, load2=Load2, load3=Load3,\n  hostname=HostName, rt_size=RTSize, memory=Memory} = _NodeDetails, Key) ->\n    case Key of\n        predlist -> PredList;\n        pred -> hd(PredList);\n        node -> Me;\n        succlist -> SuccList;\n        succ -> hd(SuccList);\n        load -> Load;\n        load2 -> Load2;\n        load3 -> Load3;\n        hostname -> HostName;\n        rt_size -> RTSize;\n        memory -> Memory;\n        _ -> throw('not_available')\n    end;\nget(NodeDetails, Key) when is_list(NodeDetails) ->\n    case Key of\n        pred -> hd(get_list(NodeDetails, predlist));\n        succ -> hd(get_list(NodeDetails, succlist));\n        _    -> get_list(NodeDetails, Key)\n    end.\n\n-spec get_list(node_details_list(), predlist | succlist) -> nodelist:non_empty_snodelist();\n              (node_details_list(), pred | node | succ) -> node:node_type();\n              (node_details_list(), my_range) -> intervals:interval();\n              (node_details_list(), is_leaving) -> boolean();\n              (node_details_list(), db) -> db_dht:db();\n              (node_details_list(), load) -> load();\n              (node_details_list(), hostname) -> hostname();\n              (node_details_list(), rt_size) -> rt_size();\n              (node_details_list(), message_log) -> message_log();\n              (node_details_list(), memory) -> memory();\n              (node_details_list(), new_key) -> ?RT:key().\nget_list(NodeDetails, Key) ->\n    case lists:keyfind(Key, 1, NodeDetails) of\n        {Key, Value} -> Value;\n        false -> throw('not_available')\n    end.\n"
  },
  {
    "path": "src/nodelist.erl",
    "content": "%  @copyright 2010-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Provides lists of nodes that are sorted by the nodes' ids.\n%% @end\n%% @version $Id$\n-module(nodelist).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile({inline, [node/1, nodeid/1, pred/1, preds/1, succ/1, succs/1,\n                   node_range/1, succ_range/1,\n                   create_pid_to_node_dict/2]}).\n\n-export([% constructors:\n         new_neighborhood/1, new_neighborhood/2, new_neighborhood/3,\n         mk_neighborhood/2, mk_neighborhood/4,\n         mk_nodelist/2,\n\n         % getters:\n         node/1, nodeid/1, pred/1, preds/1, succ/1, succs/1,\n         node_range/1, succ_range/1,\n         has_real_pred/1, has_real_succ/1,\n\n         % modifiers:\n         trunc/3, trunc_preds/2, trunc_succs/2,\n         remove/2, remove/3, filter/2, filter/3,\n         lremove/2, lremove/3,lfilter/2, lfilter/3,\n         lfilter_min_length/3, filter_min_length/4,\n         merge/4, add_node/4, add_nodes/4,\n\n         update_node/2, % update base node\n         update_ids/2, % update node ids\n\n         % converters:\n         to_list/1,\n\n         % miscellaneous:\n         succ_ord_node/2, succ_ord_id/2,\n         succ_ord_node/3, succ_ord_id/3,\n         lupdate_ids/2, lremove_outdated/1, lremove_outdated/2,\n         largest_smaller_than/2, largest_smaller_than/3,\n         create_pid_to_node_dict/2]).\n\n-include(\"scalaris.hrl\").\n\n-export_type([neighborhood/0, snodelist/0, non_empty_snodelist/0]).\n\n-type(snodelist() :: [node:node_type()]). % (sorted) node list\n-type(non_empty_snodelist() :: [node:node_type(),...]). % non-empty (sorted) node list\n-opaque(neighborhood() :: {Preds::non_empty_snodelist(), Node::node:node_type(), Succs::non_empty_snodelist(),\n                           NodeInterval::intervals:interval(), SuccInterval::intervals:interval()}).\n\n%% @doc Helper function that throws an exception if the given neighbor is newer\n%%      than the BaseNode (requires that both are equal!).\n-spec throw_if_newer(Neighbor::node:node_type(), BaseNode::node:node_type()) -> ok.\nthrow_if_newer(Neighbor, BaseNode) ->\n    case node:is_newer(Neighbor, BaseNode) of\n        true ->\n            throw('cannot create a neighborhood() with a neighbor newer than the node itself');\n        _ -> ok\n    end.\n\n-spec add_intervals(Preds::non_empty_snodelist(), Node::node:node_type(), Succs::non_empty_snodelist()) -> neighborhood().\nadd_intervals([Pred | _] = Preds, Node, [Succ | _] = Succs) ->\n    {Preds, Node, Succs,\n     node:mk_interval_between_nodes(Pred, Node),\n     node:mk_interval_between_nodes(Node, Succ)}.\n\n%% @doc Creates a new neighborhood structure for the given node.\n-spec new_neighborhood(Node::node:node_type()) -> neighborhood().\nnew_neighborhood(Node) ->\n    add_intervals([Node], Node, [Node]).\n\n%% @doc Creates a new neighborhood structure for the given node and a neighbor\n%%      (this will be its predecessor and successor).\n-spec new_neighborhood(Node::node:node_type(), Neighbor::node:node_type()) -> neighborhood().\nnew_neighborhood(Node, Neighbor) ->\n    case node:same_process(Node, Neighbor) of\n        true ->\n            % the node should always be the first to get a new ID!\n            % if not, something went wrong\n            throw_if_newer(Neighbor, Node),\n            new_neighborhood(Node);\n        false ->\n            add_intervals([Neighbor], Node, [Neighbor])\n    end.\n\n%% @doc Creates a new neighborhood structure for the given node, its predecessor\n%%      and successor. If the order is wrong, Pred and Succ will be exchanged.\n%%      (provided for convenience - special case of mk_neighborhood)\n-spec new_neighborhood(Pred::node:node_type(), Node::node:node_type(), Succ::node:node_type()) -> neighborhood().\nnew_neighborhood(Pred, Node, Succ) ->\n    case node:same_process(Pred, Node) of\n        false ->\n            case node:same_process(Succ, Node) of\n                false ->\n                    case node:same_process(Pred, Succ) of\n                        false ->\n                            % distinct nodes -> determine order:\n                            case succ_ord_node(Pred, Succ, Node) of\n                                true  -> add_intervals([Succ], Node, [Pred]);\n                                false -> add_intervals([Pred], Node, [Succ])\n                            end;\n                        true ->\n                            NewerNode = node:newer(Pred, Succ),\n                            new_neighborhood(Node, NewerNode)\n                    end;\n                true ->\n                    throw_if_newer(Succ, Node),\n                    new_neighborhood(Node, Pred)\n            end;\n        true ->\n            % the node should always be the first to get a new ID!\n            % if not, something went wrong\n            throw_if_newer(Pred, Node),\n            new_neighborhood(Node, Succ)\n    end.\n\n%% @doc Helper function to make sure a (temporary) neighborhood object has\n%%      non-empty predecessor and successor lists (fills them with itself if\n%%      necessary).\n-spec ensure_lists_not_empty(Preds::snodelist(), Node::node:node_type(), Succs::snodelist()) -> neighborhood().\nensure_lists_not_empty([_|_] = Preds, Node, [_|_] = Succs) ->\n    add_intervals(Preds, Node, Succs);\nensure_lists_not_empty([], Node, []) ->\n    add_intervals([Node], Node, [Node]);\nensure_lists_not_empty([], Node, [_|_] = Succs) ->\n    add_intervals([lists:last(Succs)], Node, Succs);\nensure_lists_not_empty([_|_] = Preds, Node, []) ->\n    add_intervals(Preds, Node, [lists:last(Preds)]).\n\n%% @doc Truncates the given neighborhood's predecessor and successor lists to\n%%      the given sizes.\n-spec trunc(Neighborhood::neighborhood(), PredsLength::pos_integer(), SuccsLength::pos_integer()) -> neighborhood().\ntrunc({Preds, Node, Succs, NodeIntv, SuccIntv}, PredsLength, SuccsLength) when (PredsLength > 0) andalso (SuccsLength > 0) ->\n    {lists:sublist(Preds, PredsLength), Node, lists:sublist(Succs, SuccsLength), NodeIntv, SuccIntv}.\n\n%% @doc Truncates the given neighborhood's predecessor list to the given size.\n-spec trunc_preds(neighborhood(), PredsLength::pos_integer()) -> neighborhood().\ntrunc_preds({Preds, Node, Succs, NodeIntv, SuccIntv}, PredsLength) when (PredsLength > 0) ->\n    {lists:sublist(Preds, PredsLength), Node, Succs, NodeIntv, SuccIntv}.\n\n%% @doc Truncates the given neighborhood's successor list to the given size.\n-spec trunc_succs(neighborhood(), SuccsLength::pos_integer()) -> neighborhood().\ntrunc_succs({Preds, Node, Succs, NodeIntv, SuccIntv}, SuccsLength) when (SuccsLength > 0) ->\n    {Preds, Node, lists:sublist(Succs, SuccsLength), NodeIntv, SuccIntv}.\n\n%% @doc Returns the neighborhood's node.\n-spec node(neighborhood()) -> node:node_type().\nnode(Neighborhood) -> element(2, Neighborhood).\n\n%% @doc Returns the ID of the neighborhood's node (provided for convenience).\n-spec nodeid(neighborhood()) -> ?RT:key().\nnodeid(Neighborhood) -> node:id(?MODULE:node(Neighborhood)).\n\n%% @doc Returns the node's range.\n-spec node_range(neighborhood()) -> intervals:interval().\nnode_range(Neighborhood) -> element(4, Neighborhood).\n\n%% @doc Updates the base node of the neighborhood if its ID is still between\n%%      the ID of the predecessor and successor. Otherwise throws an exception.\n%%      Note: pred and/or succ could change due to (outdated) other preds/succs.\n-spec update_node(neighborhood(), NewBaseNode::node:node_type()) -> neighborhood().\nupdate_node({[Node], Node, [Node], _NodeIntv, _SuccIntv}, NewBaseNode) ->\n    add_intervals([NewBaseNode], NewBaseNode, [NewBaseNode]);\nupdate_node({[Pred] = Neighbors, _Node, Neighbors, _NodeIntv, _SuccIntv}, NewBaseNode) ->\n    case node:id(NewBaseNode) =:= node:id(Pred) of\n        false -> add_intervals(Neighbors, NewBaseNode, Neighbors);\n        _     -> throw(new_node_not_in_pred_succ_interval)\n    end;\nupdate_node({[Pred | _] = Preds, _Node, [Succ | _] = Succs, _NodeIntv, _SuccIntv}, NewBaseNode) ->\n    PredSuccInt = intervals:new('(', node:id(Pred), node:id(Succ), ')'),\n    case intervals:in(node:id(NewBaseNode), PredSuccInt) of\n        true -> % sort preds, succs\n                SuccOrd = fun(N1, N2) -> succ_ord_node(N1, N2, NewBaseNode) end,\n                PredOrd = fun(N1, N2) -> succ_ord_node(N2, N1, NewBaseNode) end,\n                SuccsSorted = lists:usort(SuccOrd, Succs),\n                PredsSorted = lists:usort(PredOrd, Preds),\n                add_intervals(PredsSorted, NewBaseNode, SuccsSorted);\n        _    -> throw(new_node_not_in_pred_succ_interval)\n    end.\n\n%% @doc Returns the neighborhood's predecessor list.\n-spec preds(neighborhood()) -> non_empty_snodelist().\npreds(Neighborhood) -> element(1, Neighborhood).\n\n%% @doc Returns a neighborhood's or a node lists's predecessor (may be the node\n%%      itself).\n-spec pred(neighborhood()) -> node:node_type().\npred(Neighborhood) -> hd(preds(Neighborhood)).\n\n%% @doc Returns whether the neighborhood contains a real predecessor (one not\n%%      equal to the own node) or not (provided for convenience).\n-spec has_real_pred(neighborhood()) -> boolean().\nhas_real_pred(Neighborhood) -> pred(Neighborhood) =/= ?MODULE:node(Neighborhood).\n\n%% @doc Returns the neighborhood's successor list.\n-spec succs(neighborhood()) -> non_empty_snodelist().\nsuccs(Neighborhood) -> element(3, Neighborhood).\n\n%% @doc Returns the neighborhood's or a node lists's successor (may be the node\n%%      itself).\n-spec succ(neighborhood()) -> node:node_type().\nsucc(Neighborhood) -> hd(succs(Neighborhood)).\n\n%% @doc Returns the successor's range.\n-spec succ_range(neighborhood()) -> intervals:interval().\nsucc_range(Neighborhood) -> element(5, Neighborhood).\n\n%% @doc Returns whether the neighborhood contains a real predecessor (one not\n%%      equal to the own node) or not (provided for convenience).\n-spec has_real_succ(neighborhood()) -> boolean().\nhas_real_succ(Neighborhood) -> succ(Neighborhood) =/= ?MODULE:node(Neighborhood).\n\n%% @doc Splits the given (unsorted) node list into sorted lists with nodes that\n%%      have smaller, equal and larger IDs than the given node. The NodeList may\n%%      contain duplicates, i.e. nodes with the same pid but different IDs (and\n%%      IDVersions).\n-spec lsplit_nodelist(NodeList::[node:node_type()], Node::node:node_type()) -> {Smaller::snodelist(), Equal::snodelist(), Larger::snodelist()}.\nlsplit_nodelist(NodeList, Node) ->\n    lusplit_nodelist(lremove_outdated(NodeList, Node), Node).\n\n%% @doc Splits the given (unsorted) \"unique\" node list, i.e. a list containing\n%%      no two nodes with the same pid but different IDs (and IDVersions), into\n%%      three sorted lists with nodes that have smaller, equal and larger IDs\n%%      than the given node.\n-spec lusplit_nodelist(NodeList::[node:node_type()], Node::node:node_type()) -> {Smaller::snodelist(), Equal::snodelist(), Larger::snodelist()}.\nlusplit_nodelist(NodeList, Node) ->\n    NodeId = node:id(Node),\n    {Smaller, Equal, Larger} =\n        util:lists_partition3(\n          fun(N) ->\n                  case node:same_process(N, Node) of\n                      true -> throw_if_newer(N, Node), 2;\n                      _    -> NId = node:id(N),\n                              if NId < NodeId -> 1;\n                                 NId > NodeId -> 3;\n                                 true -> 2\n                              end\n                  end\n          end, NodeList),\n\n    SmallerSorted = lists:usort(fun succ_ord_node/2, Smaller),\n    EqualSorted = lists:usort(fun succ_ord_node/2, Equal),\n    LargerSorted = lists:usort(fun succ_ord_node/2, Larger),\n    {SmallerSorted, EqualSorted, LargerSorted}.\n\n%% @doc Creates a sorted nodelist starting at the given node and going clockwise\n%%      along the ring (also see succ_ord_node/3).\n-spec mk_nodelist(UnorderedNodeList::[node:node_type()], Node::node:node_type()) -> OrderedNodeList::snodelist().\nmk_nodelist(NodeList, Node) ->\n    {SmallerSorted, EqualSorted, LargerSorted} =\n        lsplit_nodelist(NodeList, Node),\n    lists:append([EqualSorted, LargerSorted, SmallerSorted]).\n\n%% @doc Creates a neighborhood structure for the given node from a given\n%%      (unsorted) node list and limits its predecessor and successor lists to\n%%      the given sizes.\n-spec mk_neighborhood(NodeList::[node:node_type()], Node::node:node_type(), PredsLength::pos_integer(), SuccsLength::pos_integer()) -> neighborhood().\nmk_neighborhood(NodeList, Node, PredsLength, SuccsLength) ->\n    NeighborHood = mk_neighborhood(NodeList, Node),\n    trunc(NeighborHood, PredsLength, SuccsLength).\n\n%% @doc Creates a neighborhood structure for the given node from a given\n%%      (unsorted) node list. Note that in this case, the predecessor and\n%%      successor lists will effectively be the same!\n-spec mk_neighborhood(NodeList::[node:node_type()], Node::node:node_type()) -> neighborhood().\nmk_neighborhood(NodeList, Node) ->\n    {SmallerSorted, EqualSorted, LargerSorted} =\n        lsplit_nodelist(NodeList, Node),\n\n    if (LargerSorted =:= []) andalso (SmallerSorted =:= []) ->\n           Preds = lists:reverse(EqualSorted),\n           Succs = EqualSorted;\n       true ->\n           Neighbors = lists:append(LargerSorted, SmallerSorted),\n           Preds = lists:reverse(Neighbors),\n           Succs = Neighbors\n    end,\n    ensure_lists_not_empty(Preds, Node, Succs).\n\n-type filter_fun() :: fun((node:node_type()) -> boolean()).\n-type eval_fun() :: fun((node:node_type()) -> any()).\n\n%% @doc Removes the given node (or node with the given Pid) from a node list\n%%      (provided for convenience - see lfilter/2).\n-spec lremove(NodeOrPid::node:node_type() | comm:mypid() | pid(), snodelist()) -> snodelist().\nlremove(NodeOrPid, NodeList) ->\n    lfilter(NodeList, fun(N) -> not node:same_process(NodeOrPid, N) end).\n\n%% @doc Removes the given node (or node with the given Pid) from a neighborhood\n%%      (provided for convenience - see filter/2).\n%%      Note: A neighborhood's base node is never removed!\n-spec remove(NodeOrPid::node:node_type() | comm:mypid() | pid(), neighborhood()) -> neighborhood().\nremove(NodeOrPid, Neighborhood) ->\n    filter(Neighborhood, fun(N) -> not node:same_process(NodeOrPid, N) end).\n\n%% @doc Removes the given node (or node with the given Pid) from a node list\n%%      and executes EvalFun for any such occurrence (provided for convenience\n%%      - see filter/3).\n-spec lremove(NodeOrPid::node:node_type() | comm:mypid() | pid(), snodelist(),\n              EvalFun::eval_fun()) -> snodelist().\nlremove(NodeOrPid, NodeList, EvalFun) ->\n    lfilter(NodeList, fun(N) -> not node:same_process(NodeOrPid, N) end, EvalFun).\n\n%% @doc Removes the given node (or node with the given Pid) from a neighborhood\n%%      and executes EvalFun for any such occurrence (provided for convenience,\n%%      see filter/3).\n%%      Note: A neighborhood's base node is never removed!\n-spec remove(NodeOrPid::node:node_type() | comm:mypid() | pid(), neighborhood(),\n             EvalFun::eval_fun()) -> neighborhood().\nremove(NodeOrPid, Neighborhood, EvalFun) ->\n    filter(Neighborhood, fun(N) -> not node:same_process(NodeOrPid, N) end, EvalFun).\n\n%% @doc Keeps any node for which FilterFun returns true in a node list.\n-spec lfilter(snodelist(), FilterFun::filter_fun()) -> snodelist().\nlfilter(NodeList, FilterFun) ->\n    [N || N <- NodeList, FilterFun(N)].\n\n%% @doc Keeps any node for which FilterFun returns true in a neighborhood.\n%%      Note: A neighborhood's base node is never removed!\n-spec filter(neighborhood(), FilterFun::filter_fun()) -> neighborhood().\nfilter({Preds, Node, Succs, _NodeIntv, _SuccIntv}, FilterFun) ->\n    NewPreds = lfilter(Preds, FilterFun),\n    NewSuccs = lfilter(Succs, FilterFun),\n    ensure_lists_not_empty(NewPreds, Node, NewSuccs).\n\n%% @doc Keeps any node for which FilterFun returns true in a node list and\n%%      executes EvalFun for any other node.\n-spec lfilter(snodelist(), FilterFun::filter_fun(), EvalFun::eval_fun()) -> snodelist().\nlfilter([N | Rest], FilterFun, EvalFun) ->\n    case FilterFun(N) of\n        true -> [N | lfilter(Rest, FilterFun, EvalFun)];\n        _    -> EvalFun(N),\n                lfilter(Rest, FilterFun, EvalFun)\n    end;\nlfilter([], _FilterFun, _EvalFun) ->\n    [].\n\n%% @doc Keeps any node for which FilterFun returns true in a neighborhood\n%%      and executes EvalFun for any other node.\n%%      Note: A neighborhood's base node is never removed!\n-spec filter(neighborhood(), FilterFun::filter_fun(),\n             EvalFun::eval_fun()) -> neighborhood().\nfilter({Preds, Node, Succs, _NodeIntv, _SuccIntv}, FilterFun, EvalFun) ->\n    % note: cannot call lfilter/3 on the succ/pred lists (a node may be in both\n    %       lists but should have EvalFun evaluated only once)\n    {NewPreds, PredsRemoved} = lists:partition(FilterFun, Preds),\n    {NewSuccs, SuccsRemoved} = lists:partition(FilterFun, Succs),\n    Removed = lists:usort(lists:append(PredsRemoved, SuccsRemoved)),\n    lists:foreach(EvalFun, Removed),\n    ensure_lists_not_empty(NewPreds, Node, NewSuccs).\n\n%% @doc Keeps any node for which FilterFun returns true in a node list but\n%%      produces a node list with at least MinLength elements by adding enough\n%%      unmatching nodes. Preserves the order of the list.\n-spec lfilter_min_length(snodelist(), FilterFun::filter_fun(), MinLength::non_neg_integer())\n        -> snodelist().\nlfilter_min_length(NodeList, FilterFun, MinLength) ->\n    % first count the number of nodes that match the FilterFun\n    SatisfyingCount = lists:foldl(fun(N, Count) ->\n                                          case FilterFun(N) of\n                                              true  -> Count + 1;\n                                              false -> Count\n                                          end\n                                  end,\n                                  0, NodeList),\n    % then collect matching nodes and as many unmatching nodes as needed to\n    % have a result of at least MinLength elements\n    % -> beware not to destroy the order of the list!\n    UnsatisfyingNodesToAdd = MinLength - SatisfyingCount,\n    NewNodeList =\n        case UnsatisfyingNodesToAdd =< 0 of\n            true -> [Node || Node <- NodeList, FilterFun(Node)];\n            false ->\n                AddIfSatisfyingOrMinFun =\n                    fun(Node, {ResultList, UnsatisfyingNodesToAdd1}) ->\n                            case FilterFun(Node) of\n                                true  ->\n                                    {[Node | ResultList], UnsatisfyingNodesToAdd1};\n                                false when UnsatisfyingNodesToAdd1 =/= 0 ->\n                                    {[Node | ResultList], UnsatisfyingNodesToAdd1 - 1};\n                                false ->\n                                    {ResultList, UnsatisfyingNodesToAdd1}\n                            end\n                    end,\n                % elements are in wrong order, but lists:foldr cannot be applied\n                {ResultsReverse, _} = lists:foldl(AddIfSatisfyingOrMinFun,\n                                                  {[], UnsatisfyingNodesToAdd},\n                                                  NodeList),\n                lists:reverse(ResultsReverse)\n        end,\n    NewNodeList.\n\n%% @doc Keeps any node for which FilterFun returns true in a predecessor and\n%%      successor lists but produces a neighborhood with at least MinPredsLength\n%%      predecessors and at least MinSuccsLength successors by adding enough\n%%      unmatching nodes. If the predecessor or successor list is smaller than\n%%      MinPredsLength and MinSuccsLength respectively, the whole list will be\n%%      used.\n%%      Note: A neighborhood's base node is never removed!\n-spec filter_min_length(neighborhood(), FilterFun::filter_fun(), MinPredsLength::non_neg_integer(), MinSuccsLength::non_neg_integer()) -> neighborhood().\nfilter_min_length({Preds, Node, Succs, _NodeIntv, _SuccIntv}, FilterFun, MinPredsLength, MinSuccsLength) ->\n    NewPreds = lfilter_min_length(Preds, FilterFun, MinPredsLength),\n    NewSuccs = lfilter_min_length(Succs, FilterFun, MinSuccsLength),\n    ensure_lists_not_empty(NewPreds, Node, NewSuccs).\n\n%% @doc Helper function that removes the head of NodeList if is is equal to\n%%      Node (using node:same_process/2).\n-spec lremove_head_if_eq(NodeList::snodelist(), Node::node:node_type()) -> snodelist().\nlremove_head_if_eq([H | T] = NodeList, Node) ->\n    case node:same_process(H, Node) of\n        false -> NodeList;\n        true  -> T\n    end;\nlremove_head_if_eq([] = NodeList, _Node) ->\n    NodeList.\n\n%% @doc Converts a neighborhood to a sorted list of nodes including the\n%%      predecessors, the node and its successors. The first element of the\n%%      resulting list will be the node itself, afterwards every known node\n%%      along the ring towards the first node.\n-spec to_list(neighborhood()) -> non_empty_snodelist().\nto_list({Preds, Node, Succs, _NodeIntv, _SuccIntv}) ->\n    Ord = fun(N1, N2) -> succ_ord_node(N1, N2, Node) end,\n    CleanPreds = lremove_head_if_eq(Preds, Node),\n    CleanSuccs = lremove_head_if_eq(Succs, Node),\n    CleanPredsReversed = lists:reverse(CleanPreds),\n    [Node | util:smerge2(CleanSuccs, CleanPredsReversed, Ord)].\n\n%% @doc Removes NodeToRemove from the given list and additionally gets the\n%%      resulting list's last element.\n-spec lget_last_and_remove(NodeList::snodelist(), NodeToRemove::node:node_type(), ResultList::snodelist()) ->\n        {LastNode::node:node_type(), FilteredList::non_empty_snodelist()} |\n        {null, []}.\nlget_last_and_remove([H | T], NodeToRemove, ResultList) ->\n    case node:same_process(H, NodeToRemove) of\n        true ->\n            lget_last_and_remove(T, NodeToRemove, ResultList);\n        false ->\n            lget_last_and_remove(T, NodeToRemove, [H | ResultList])\n    end;\nlget_last_and_remove([], _NodeToRemove, []) ->\n    {node:null(), []};\nlget_last_and_remove([], _NodeToRemove, [Last | _] = ResultList) ->\n    {Last, lists:reverse(ResultList)}.\n\n%% @doc Helper function that adds NewHead to the head of NodeList if is is not\n%%      equal to Node (using node:same_process/2).\n-spec ladd_head_if_noteq(NewHead::node:node_type(), NodeList::snodelist(), Node::node:node_type()) -> snodelist().\nladd_head_if_noteq(NewHead, NodeList, Node) ->\n    case node:same_process(NewHead, Node) of\n        false -> [NewHead | NodeList];\n        true  -> NodeList\n    end.\n\n%% @doc Rebases a sorted node list (as returned by to_list/1, for example) to\n%%      the sorted list based on a new first node (without including it).\n%%      Removes the NewFirstNode from this list on the fly.\n-spec lrebase_list(non_empty_snodelist(), NewFirstNode::node:node_type()) -> snodelist().\nlrebase_list([First] = NodeList, NewFirstNode) ->\n    case node:same_process(First, NewFirstNode) of\n        false -> NodeList;\n        true  -> []\n    end;\nlrebase_list([NewFirstNode | T], NewFirstNode) ->\n    lremove(NewFirstNode, T); % just to be sure, remove NewFirstNode from the Tail\nlrebase_list([First | T], NewFirstNode) ->\n    {LastNode, TFilt} = lget_last_and_remove(T, NewFirstNode, []),\n    if LastNode =/= null ->\n           NodeListInterval = node:mk_interval_between_nodes(First, LastNode),\n           case intervals:in(node:id(NewFirstNode), NodeListInterval) of\n               false ->\n                   ladd_head_if_noteq(First, TFilt, NewFirstNode);\n               true->\n                   NewPredsInterval = node:mk_interval_between_nodes(First, NewFirstNode),\n                   {L1T, L2} = lists:splitwith(fun(N) -> intervals:in(node:id(N), NewPredsInterval) end, TFilt),\n                   L1 = ladd_head_if_noteq(First, L1T, NewFirstNode),\n                   lists:append(L2, L1)\n           end;\n       true ->\n           ladd_head_if_noteq(First, [], NewFirstNode)\n    end.\n\n%% @doc Merges two lists of nodes into a neighborhood structure with the given\n%%      node. Predecessor and successor lists are truncated to the given sizes,\n%%      node IDs are updated with the most up-to-date ID from any list.\n%%      Neither Node1View nor Node2View should contain BaseNode!\n-spec lmerge_helper(Node1View::snodelist(), Node2View::snodelist(), BaseNode::node:node_type(),\n                    PredsLength::pos_integer(), SuccsLength::pos_integer()) -> neighborhood().\nlmerge_helper(Node1View, Node2View, BaseNode, PredsLength, SuccsLength) ->\n    {Node1ViewUpd, Node2ViewUpd} = lupdate_ids(Node1View, Node2View),\n    % due to updated IDs, the lists might not be sorted anymore...\n    Ord = fun(N1, N2) -> succ_ord_node(N1, N2, BaseNode) end,\n    Node1ViewUpdSorted = lists:usort(Ord, Node1ViewUpd),\n    Node2ViewUpdSorted = lists:usort(Ord, Node2ViewUpd),\n\n    % favour the newly provided nodes over the present ones (they are probably newer)\n    MergedView = util:smerge2(Node1ViewUpdSorted, Node2ViewUpdSorted, Ord,\n                              fun(_E1, E2) ->\n                                      ?DBG_ASSERT(_E1 =:= E2 orelse not node:same_process(_E1, E2)),\n                                      [E2]\n                              end),\n    Preds = lists:sublist(lists:reverse(MergedView), PredsLength),\n    Succs = lists:sublist(MergedView, SuccsLength),\n    ensure_lists_not_empty(Preds, BaseNode, Succs).\n\n%% @doc Merges nodes of Neighbors2 into Neighbors1 and truncates the predecessor\n%%      and successor lists to the given sizes.\n-spec merge(Neighbors1::neighborhood(), Neighbors2::neighborhood(), PredsLength::pos_integer(),\n            SuccsLength::pos_integer()) -> neighborhood().\nmerge(Neighbors1, Neighbors2, PredsLength, SuccsLength) ->\n    % note: similar to mk_neighborhood/4\n    % create a sorted list of nodes in Neighbors1 and Neighbours2\n    [Node1 | Neighbors1View] = to_list(Neighbors1),\n    ?DBG_ASSERT(Node1 =:= ?MODULE:node(Neighbors1)),\n    Neighbors2View = lrebase_list(to_list(Neighbors2), Node1),\n    lmerge_helper(Neighbors1View, Neighbors2View, Node1, PredsLength, SuccsLength).\n\n%% @doc Adds a node to a neighborhood structure and truncates the predecessor\n%%      and successor list to the given sizes.\n%%      Note: nodes which have only been present in the predecessor (successor)\n%%      list may now also appear in the successor (predecessor) list if a list\n%%      has been too small.\n-spec add_node(Neighbors::neighborhood(), NodeToAdd::node:node_type(), PredsLength::pos_integer(),\n               SuccsLength::pos_integer()) -> neighborhood().\nadd_node({Preds, BaseNode, Succs, _NodeIntv, _SuccIntv}, NodeToAdd, PredsLength, SuccsLength) ->\n    case node:same_process(BaseNode, NodeToAdd) of\n        false ->\n            CleanPreds = lremove_head_if_eq(Preds, BaseNode),\n            CleanSuccs = lremove_head_if_eq(Succs, BaseNode),\n            UpdateFun = fun(N) ->\n                                case node:same_process(N, NodeToAdd) of\n                                    true -> node:newer(N, NodeToAdd);\n                                    _    -> N\n                                end\n                        end,\n            % create a view of all know (and updated) nodes:\n            ViewUpd = lists:append([NodeToAdd | lists:map(UpdateFun, CleanPreds)],\n                                   lists:map(UpdateFun, CleanSuccs)),\n            % sort the list again\n            SuccOrd = fun(N1, N2) -> succ_ord_node(N1, N2, BaseNode) end,\n            SuccsUpdSorted = lists:usort(SuccOrd, ViewUpd),\n            PredsUpdSorted = lists:reverse(SuccsUpdSorted),\n\n            trunc(add_intervals(PredsUpdSorted, BaseNode, SuccsUpdSorted), PredsLength, SuccsLength);\n        true ->\n            throw_if_newer(NodeToAdd, BaseNode),\n            % eventually\n            mk_neighborhood(lists:append(Preds, Succs), BaseNode, PredsLength, SuccsLength)\n    end.\n\n%% @doc Adds nodes from the given node list to the given neighborhood structure\n%%      and truncates the predecessor and successor list to the given sizes.\n%%      Note: nodes which have only been present in the predecessor (successor)\n%%      list may now also appear in the successor (predecessor) list if a list\n%%      has been too small.\n-spec add_nodes(Neighbors::neighborhood(), NodeList::[node:node_type()],\n                PredsLength::pos_integer(), SuccsLength::pos_integer()) -> neighborhood().\nadd_nodes(Neighbors, [NodeToAdd], PredsLength, SuccsLength) ->\n    add_node(Neighbors, NodeToAdd, PredsLength, SuccsLength);\nadd_nodes(Neighbors, [_|_] = NodeList, PredsLength, SuccsLength) ->\n    % note: similar to mk_neighborhood/4 and merge/4\n    [Node | NeighborsView] = to_list(Neighbors),\n    ?DBG_ASSERT(Node =:= ?MODULE:node(Neighbors)),\n    {SmallerSorted, EqualSorted, LargerSorted} =\n        lsplit_nodelist(NodeList, Node),\n\n    OtherView = if (LargerSorted =:= []) andalso (SmallerSorted =:= []) ->\n                       [N || N <- EqualSorted, not node:same_process(N, Node)];\n                   true ->\n                       lists:append(LargerSorted, SmallerSorted)\n                end,\n    lmerge_helper(NeighborsView, OtherView, Node, PredsLength, SuccsLength);\nadd_nodes(Neighbors, [], PredsLength, SuccsLength) ->\n    trunc(Neighbors, PredsLength, SuccsLength).\n\n%% @doc Updates the node IDs of the nodes in the neighborhood with the most\n%%      up-to-date ID in either its own lists or the given node list.\n%%      Note: throws if any node in the NodeList is the same as the base node\n%%      but with an updated ID!\n-spec update_ids(Neighbors::neighborhood(), NodeList::[node:node_type()]) -> neighborhood().\nupdate_ids({Preds, BaseNode, Succs, _NodeIntv, _SuccIntv}, [_|_] = NodeList) ->\n    {PredsUpd, _} = lupdate_ids(Preds, NodeList),\n    {SuccsUpd, _} = lupdate_ids(Succs, NodeList),\n    _ = [throw_if_newer(N, BaseNode) || N <- NodeList, node:same_process(BaseNode, N)],\n    % due to updated IDs, the lists might not be sorted anymore...\n    SuccOrd = fun(N1, N2) -> succ_ord_node(N1, N2, BaseNode) end,\n    PredOrd = fun(N1, N2) -> succ_ord_node(N2, N1, BaseNode) end,\n    SuccsUpdSorted = lists:usort(SuccOrd, SuccsUpd),\n    PredsUpdSorted = lists:usort(PredOrd, PredsUpd),\n    ensure_lists_not_empty(PredsUpdSorted, BaseNode, SuccsUpdSorted);\nupdate_ids(Neighbors, []) ->\n    Neighbors.\n\n%% @doc Defines that N1 is less than or equal to N2 if their IDs are (provided\n%%      for convenience).\n-spec succ_ord_node(N1::node:node_type(), N2::node:node_type()) -> boolean().\nsucc_ord_node(N1, N2) -> succ_ord_id(node:id(N1), node:id(N2)).\n\n%% @doc Defines that K1 is less than or equal to K2 if their IDs are.\n-spec succ_ord_id(K1::?RT:key(), K2::?RT:key()) -> boolean().\nsucc_ord_id(K1, K2) -> K1 =< K2.\n\n%% @doc Defines a 'less than or equal' order starting from a base node going\n%%      along the ring towards the successor where nodes that are further away\n%%      are said to be larger than nodes with smaller distances.\n-spec succ_ord_node(node:node_type(), node:node_type(), BaseNode::node:node_type()) -> boolean().\nsucc_ord_node(N1, N2, BaseNode) ->\n    succ_ord_id(node:id(N1), node:id(N2), node:id(BaseNode)).\n\n%% @doc Defines a 'less than or equal' order starting from a base node going\n%%      along the ring towards the successor where nodes that are further away\n%%      are said to be larger than nodes with smaller distances.\n-spec succ_ord_id(K1::?RT:key(), K2::?RT:key(), BaseKey::?RT:key()) -> boolean().\nsucc_ord_id(K1, K2, BaseKey) ->\n    % more understandable version:\n%%     (K1 > BaseKey andalso K2 > BaseKey andalso K1 =< K2) orelse\n%%     (K1 < BaseKey andalso K2 < BaseKey andalso K1 =< K2) orelse\n%%     (K1 > BaseKey andalso K2 < BaseKey) orelse\n%%     (K1 =:= BaseKey).\n    % (slightly) faster version:\n    (K1 =:= BaseKey) orelse\n    (K1 > BaseKey andalso K2 < BaseKey) orelse\n    (K1 =< K2 andalso (K1 >= BaseKey orelse K2 < BaseKey)).\n\n%% %%  doc Defines a 'less than or equal' order starting from a base node going\n%% %%      along the ring towards the predecessor where nodes that are further away\n%% %%      are said to be larger than nodes with smaller distances.\n%% -spec pred_ord(node:node_type(), node:node_type(), BaseNode::node:node_type()) -> boolean().\n%% pred_ord(N1, N2, BaseNode) ->\n%%     BaseNodeId = node:id(BaseNode),\n%%     (node:id(N1) > BaseNodeId andalso node:id(N2) > BaseNodeId andalso node:id(N1) >= node:id(N2)) orelse\n%%     (node:id(N1) < BaseNodeId andalso node:id(N2) < BaseNodeId andalso node:id(N1) >= node:id(N2)) orelse\n%%     (node:id(N1) < BaseNodeId andalso node:id(N2) > BaseNodeId) orelse\n%%     (node:id(N1) =:= BaseNodeId).\n\n%% @doc Creates a dictionary mapping a node PID to the newest version of its\n%%      node object from the given lists.\n-spec create_pid_to_node_dict(Dict, NodeLists::[[node:node_type()]]) -> Dict\n        when is_subtype(Dict, dict:dict(comm:mypid(), node:node_type())).\ncreate_pid_to_node_dict(Dict, []) ->\n    dict:map(fun(_PidX, [NodeX]) -> NodeX;\n                (_PidX, [H | RestX]) ->\n                     lists:foldl(fun node:newer/2, H, RestX)\n             end, Dict);\ncreate_pid_to_node_dict(Dict, [NodeList | Rest]) ->\n    DictNew = lists:foldl(\n                fun(NodeX, DictX) ->\n                        case node:is_valid(NodeX) of\n                            true ->\n                                dict:append(node:pidX(NodeX), NodeX, DictX);\n                            false ->\n                                DictX\n                        end\n                end, Dict, NodeList),\n    create_pid_to_node_dict(DictNew, Rest).\n\n%% @doc Removes any node with outdated ID information from the list as well as\n%%      any outdated node that shares the same process as Node and any invalid\n%%      node.\n-spec lremove_outdated(NodeList::[node:node_type()], Node::node:node_type() | null)\n        -> [node:node_type()].\nlremove_outdated(NodeList, Node) ->\n    % make a unique set of updated pids:\n    UpdNodes = create_pid_to_node_dict(dict:new(), [NodeList]),\n    % now remove all out-dated nodes:\n    NodeIsUpToDate = fun(N) ->\n                             NInDict = dict:fetch(node:pidX(N), UpdNodes),\n                             not node:is_newer(NInDict, N)\n                     end,\n    NodeListUpd = [N || N <- NodeList, node:is_valid(N),\n                        not (node:same_process(N, Node) andalso (node:is_newer(Node, N))),\n                        NodeIsUpToDate(N)],\n    NodeListUpd.\n\n%% @doc Removes any node with outdated ID information from the list as well as\n%%      any invalid node.\n-spec lremove_outdated(NodeList::[node:node_type()]) -> [node:node_type()].\nlremove_outdated(NodeList) ->\n     lremove_outdated(NodeList, node:null()).\n\n%% @doc Updates the node IDs of the nodes in both lists with the most up-to-date\n%%      ID in any of the two lists. The returned lists are in the same order as\n%%      as the input lists and may now contain duplicates (we could not decide\n%%      which to remove here!). Note that due to the updated IDs the order might\n%%      not be correct, i.e. according to some ordering function, anymore!\n-spec lupdate_ids(L1::[node:node_type()], L2::[node:node_type()])\n        -> {L1Upd::[node:node_type()], L2Upd::[node:node_type()]}.\nlupdate_ids(L1, L2) ->\n    % make a unique set of updated pids:\n    UpdNodes = create_pid_to_node_dict(dict:new(), [L1, L2]),\n\n    GetNewNode = fun(Node) -> dict:fetch(node:pidX(Node), UpdNodes) end,\n    L1Upd = lists:map(GetNewNode, L1),\n    L2Upd = lists:map(GetNewNode, L2),\n\n    {L1Upd, L2Upd}.\n\n%% @doc Returns whether NId is between MyId and Id and not equal to Id.\n%%      If MyId and Id are the same, every other id is less than Id.\n-spec less_than_id(NId::?RT:key(), Id::?RT:key(), MyId::?RT:key()) -> boolean().\nless_than_id(NId, Id, MyId) ->\n    % note: succ_ord_id = less than or equal\n    (NId =/= Id) andalso (MyId =:= Id orelse\n                              succ_ord_id(NId, Id, MyId)).\n\n%% @doc Look-up largest node in the NodeList that has an ID smaller than Id.\n%%      NodeList must be sorted with the largest key first (order of a\n%%      neighborhood's predecessor list).\n%%      Note: this implementation does not use intervals because comparing keys\n%%      with succ_ord is (slightly) faster. Also this is faster than a\n%%      lists:fold*/3.\n-spec best_node_maxfirst(MyId::?RT:key(), Id::?RT:key(), NodeList::snodelist(),\n                         LastFound::node:node_type()) -> Result::node:node_type().\nbest_node_maxfirst(MyId, Id, [H | T], LastFound) ->\n    % note: succ_ord_id = less than or equal\n    HId = node:id(H),\n    case less_than_id(HId, Id, MyId) of\n        false -> best_node_maxfirst(MyId, Id, T, LastFound);\n        _     -> case succ_ord_id(node:id(LastFound), HId, Id) of\n                     true -> H;\n                     _    -> LastFound\n                 end\n    end;\nbest_node_maxfirst(_MyId, _Id, [], LastFound) -> LastFound.\n\n%% @doc Similar to best_node_maxfirst/4 but with a NodeList that must be sorted\n%%      with the smallest key first (order of a neighborhood's successor list).\n-spec best_node_minfirst(MyId::?RT:key(), Id::?RT:key(), NodeList::snodelist(),\n                         LastFound1::node:node_type(), LastFound2::node:node_type())\n        -> Result::node:node_type().\nbest_node_minfirst(MyId, Id, [H | T], LastFound1, LastFound2) ->\n    case less_than_id(node:id(H), Id, MyId) of\n        true -> best_node_minfirst(MyId, Id, T, H, LastFound2);\n        _    -> best_node_minfirst(MyId, Id, [], LastFound1, LastFound2)\n    end;\nbest_node_minfirst(_MyId, Id, [], LastFound1, LastFound2) ->\n    % note: succ_ord_id = less than or equal\n    case succ_ord_id(node:id(LastFound1), node:id(LastFound2), Id) of\n        true -> LastFound2;\n        _    -> LastFound1\n    end.\n\n%% @doc Returns the node among all know neighbors (including the base node)\n%%      with the largest id smaller than Id.\n-spec largest_smaller_than(Neighbors::neighborhood(), Id::?RT:key()) -> node:node_type() | null.\nlargest_smaller_than({Preds, BaseNode, Succs, _NodeIntv, _SuccIntv}, Id) ->\n    BaseNodeId = node:id(BaseNode),\n    case BaseNodeId =/= Id of\n        true -> largest_smaller_than(Preds, BaseNode, Succs, Id, BaseNode);\n        _ -> % take first pred if valid\n            Pred = hd(Preds),\n            case node:id(Pred) =/= Id of\n                true -> Pred;\n                _    -> node:null()\n            end\n    end.\n\n%% @doc Returns the node among all know neighbors (including the base node)\n%%      with the largest id smaller than Id given that a previous search found\n%%      LastFound as its Node with the largest id smaller than id.\n-spec largest_smaller_than(Neighbors::neighborhood(), Id::?RT:key(),\n                           LastFound::node:node_type()) -> node:node_type().\nlargest_smaller_than({Preds, BaseNode, Succs, _NodeIntv, _SuccIntv}, Id, LastFound) ->\n    largest_smaller_than(Preds, BaseNode, Succs, Id, LastFound).\n\n%% @doc Helper for largest_smaller_than/2 and largest_smaller_than/3.\n-spec largest_smaller_than(Preds::non_empty_snodelist(), BaseNode::node:node_type(),\n                           Succs::non_empty_snodelist(), Id::?RT:key(),\n                           LastFound::node:node_type()) -> node:node_type().\nlargest_smaller_than(Preds, BaseNode, Succs, Id, LastFound) ->\n    MyId = node:id(BaseNode),\n    BestSucc = best_node_minfirst(MyId, Id, [BaseNode | Succs], LastFound, LastFound),\n    _Best = best_node_maxfirst(MyId, Id, Preds, BestSucc).\n"
  },
  {
    "path": "src/paxos/acceptor.erl",
    "content": "% @copyright 2009-2015 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Part of a generic Paxos-Consensus implementation\n%%      The role of a acceptor.\n%% @end\n%% @version $Id$\n-module(acceptor).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n%%% public interface for initiating a paxos acceptor for a new PaxosID\n-export([start_paxosid/2, start_paxosid_local/3, start_paxosid/3]).\n-export([stop_paxosids/2]).\n-export([add_learner/3]).\n-export([msg_accepted/4]).\n%%% functions for gen_component module and supervisor callbacks\n-export([start_link/2]).\n-export([on/2, init/1]).\n-export([check_config/0]).\n\n-include(\"gen_component.hrl\").\n-include(\"acceptor_state.hrl\").\n\n-compile({inline, [initialize/4, inform_learner/3]}).\n\n%% Messages to expect from this module\n-spec msg_ack(comm:mypid(), any(), non_neg_integer(), any(), non_neg_integer())\n             -> ok.\nmsg_ack(Proposer, PaxosID, InRound, Val, Raccepted) ->\n    comm:send(Proposer, {acceptor_ack, PaxosID, InRound, Val, Raccepted}).\n\n-spec msg_nack(comm:mypid(), any(), non_neg_integer()) -> ok.\nmsg_nack(Proposer, PaxosID, NewerRound) ->\n    comm:send(Proposer, {acceptor_nack, PaxosID, NewerRound}).\n\n-spec msg_naccepted(comm:mypid(), any(), non_neg_integer()) -> ok.\nmsg_naccepted(Proposer, PaxosID, NewerRound) ->\n    comm:send(Proposer, {acceptor_naccepted, PaxosID, NewerRound}).\n\n-spec msg_accepted(comm:mypid(), any(), non_neg_integer(), any()) -> ok.\nmsg_accepted(Learner, PaxosID, Raccepted, Val) ->\n    comm:send(Learner, {?acceptor_accept, PaxosID, Raccepted, Val}).\n\n%%% public function to initiate a new paxos instance\n%%% gets a\n%%%   PaxosID: has to be unique in the system, user has to care about this\n%%%   Acceptors: a list of paxos_acceptor processes, that are used\n%%%   Proposal: if no consensus is available beforehand, this proposer proposes this\n%%%   Majority: how many responses from acceptors have to be collected?\n%%%   InitialRound (optional): start with paxos round number (default 1)\n%%%     if InitialRound is 0, a Fast-Paxos is executed\n-spec start_paxosid(any(), [ comm:mypid() ]) -> ok.\nstart_paxosid(PaxosID, Learners) ->\n    Acceptor = pid_groups:get_my(paxos_acceptor),\n    start_paxosid_local(Acceptor, PaxosID, Learners).\n\n-spec start_paxosid_local(pid(), any(), [ comm:mypid() ]) -> ok.\nstart_paxosid_local(LAcceptor, PaxosID, Learners) ->\n    %% find the groups acceptor process\n    Message = {acceptor_initialize, PaxosID, Learners},\n    comm:send_local(LAcceptor, Message).\n\n-spec start_paxosid(comm:mypid(), any(), [ comm:mypid() ]) -> ok.\nstart_paxosid(Acceptor, PaxosID, Learners) ->\n    comm:send(Acceptor, {acceptor_initialize, PaxosID, Learners}).\n\n-spec stop_paxosids(comm:mypid(), list(any())) -> ok.\nstop_paxosids(Acceptor, PaxosIds) ->\n    comm:send(Acceptor, {acceptor_deleteids, PaxosIds}).\n\n-spec add_learner(comm:mypid(), any(), comm:mypid()) -> ok.\nadd_learner(Acceptor, PaxosID, Learner) ->\n    comm:send(Acceptor, {acceptor_add_learner, PaxosID, Learner}).\n\n%% be startable via supervisor, use gen_component\n-spec start_link(pid_groups:groupname(), pid_groups:pidname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup, PidName) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2,\n                             [],\n                             [{pid_groups_join_as, DHTNodeGroup, PidName},\n                              {spawn_opts, [{fullsweep_after, 0},\n                                            {min_heap_size, 16383}]}]).\n\n%% initialize: return initial state.\n-spec init([]) -> atom().\ninit([]) ->\n    ?TRACE(\"Starting acceptor for DHT node: ~p~n\", [pid_groups:my_groupname()]),\n    %% For easier debugging, use a named table (generates an atom)\n    %%TableName = erlang:list_to_atom(pid_groups:group_to_filename(pid_groups:my_groupname()) ++ \"_acceptor\"),\n    %%pdb:new(TableName, [set, protected, named_table]),\n    %% use random table name provided by ets to *not* generate an atom\n    TableName = pdb:new(?MODULE, [set]),\n    _State = TableName.\n\n-spec on(comm:message(), atom()) -> atom().\non({acceptor_initialize, PaxosID, Learners}, ETSTableName = State) ->\n    ?TRACE(\"acceptor:initialize for paxos id: Pid ~p Learners ~p~n\", [PaxosID, Learners]),\n    case pdb:get(PaxosID, ETSTableName) of\n        undefined when Learners =:= [] -> % just in case\n            log:log(error, \"dupl. acceptor init for id ~p\", [PaxosID]);\n        undefined ->\n            initialize(state_new(PaxosID), ETSTableName, PaxosID, Learners);\n        StateForID ->\n            case state_get_learners(StateForID) of\n                Learners ->\n                    log:log(error, \"dupl. acceptor init for id ~p\", [PaxosID]);\n                _ ->\n                    initialize(StateForID, ETSTableName, PaxosID, Learners)\n            end\n    end,\n    State;\n\n% need Sender & PaxosID\non({proposer_prepare, Proposer, PaxosID, InRound}, ETSTableName = State) ->\n    ?TRACE(\"acceptor:prepare for paxos id: ~p round ~p~n\", [PaxosID,InRound]),\n    case pdb:get(PaxosID, ETSTableName) of\n        undefined -> StateForID = state_new(PaxosID),\n                     msg_delay:send_local(\n                       config:read(acceptor_noinit_timeout) div 1000, self(),\n                       {acceptor_delete_if_no_learner, PaxosID});\n        StateForID -> ok\n    end,\n    case state_add_prepare_msg(StateForID, InRound) of\n        {ok, NewState} ->\n            pdb:set(NewState, ETSTableName),\n            msg_ack(Proposer, PaxosID, InRound,\n                    state_get_value(NewState),\n                    state_get_raccepted(NewState));\n        {dropped, NewerRound} -> msg_nack(Proposer, PaxosID, NewerRound)\n    end,\n    State;\n\non({?proposer_accept, Proposer, PaxosID, InRound, InProposal}, ETSTableName = State) ->\n    ?TRACE(\"acceptor:accept for paxos id: ~p round ~p~n\", [PaxosID, InRound]),\n    case pdb:get(PaxosID, ETSTableName) of\n        undefined -> StateForID = state_new(PaxosID),\n                     msg_delay:send_local(\n                       (config:read(tx_timeout) * 4) div 1000, self(),\n                       {acceptor_delete_if_no_learner, PaxosID});\n        StateForID -> ok\n    end,\n    case state_add_accept_msg(StateForID, InRound, InProposal) of\n        {ok, NewState} ->\n            pdb:set(NewState, ETSTableName),\n            inform_learners(PaxosID, NewState);\n        {dropped, NewerRound} -> msg_naccepted(Proposer, PaxosID, NewerRound)\n    end,\n    State;\n\non({acceptor_deleteids, ListOfPaxosIDs}, ETSTableName = State) ->\n    ?TRACE(\"acceptor:deleteids~n\", []),\n    _ = [pdb:delete(Id, ETSTableName) || Id <- ListOfPaxosIDs],\n    State;\n\non({acceptor_delete_if_no_learner, PaxosID}, ETSTableName = State) ->\n    ?TRACE(\"acceptor:delete_if_no_learner~n\", []),\n    case pdb:get(PaxosID, ETSTableName) of\n        undefined -> ok; %% already deleted\n        StateForID ->\n            case state_get_learners(StateForID) of\n                [] ->\n                    %% io:format(\"Deleting unhosted acceptor id~n\"),\n                    pdb:delete(PaxosID, ETSTableName);\n                [_|_] -> ok %% learners are registered\n            end\n    end,\n    State;\n\non({acceptor_add_learner, PaxosID, Learner}, ETSTableName = State) ->\n    ?TRACE(\"acceptor:add_learner~n\", []),\n    case pdb:get(PaxosID, ETSTableName) of\n        undefined -> ok; %% do not support adding learners without prior initialize\n        StateForID ->\n            case state_accepted(StateForID) of\n                true -> inform_learner(Learner, PaxosID, StateForID);\n                false -> ok\n            end,\n            NewLearners = [Learner | state_get_learners(StateForID)],\n            NStateForID = state_set_learners(StateForID, NewLearners),\n            pdb:set(NStateForID, ETSTableName)\n    end,\n    State.\n\n-spec initialize(StateForID::acceptor_state(), ETSTableName::atom(),\n                 PaxosID::any(), Learners::[comm:mypid()]) -> ok.\ninitialize(StateForID, ETSTableName, PaxosID, Learners) ->\n    NewState = state_set_learners(StateForID, Learners),\n    pdb:set(NewState, ETSTableName),\n    case state_accepted(NewState) of\n        true  -> inform_learners(PaxosID, NewState);\n        false -> ok\n    end.\n\n-spec inform_learners(PaxosID::any(), acceptor_state()) -> ok.\ninform_learners(PaxosID, State) ->\n    ?TRACE(\"acceptor:inform_learners: PaxosID ~p Learners ~p Decision ~p~n\",\n           [PaxosID, state_get_learners(State), state_get_value(State)]),\n    _ = [ inform_learner(X, PaxosID, State)\n            || X <- state_get_learners(State) ],\n    ok.\n\ninform_learner(Learner, PaxosID, StateForID) ->\n    msg_accepted(Learner, PaxosID,\n                 state_get_raccepted(StateForID),\n                 state_get_value(StateForID)).\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(acceptor_noinit_timeout) and\n    config:cfg_is_greater_than_equal(acceptor_noinit_timeout, 1000) and\n    config:cfg_is_greater_than_equal(tx_timeout, 1000/4) and\n    config:cfg_is_greater_than(acceptor_noinit_timeout, tx_timeout).\n"
  },
  {
    "path": "src/paxos/acceptor_state.hrl",
    "content": "% @copyright 2009-2012 Zuse Institute Berlin,\n%                      onScale solutions GmbH\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Part of generic Paxos-Consensus implementation\n%%           The state needed for a single acceptor instance.\n%% @end\n%% @version $Id$\n\n-compile({inline, [state_get_learners/1, state_set_learners/2,\n                   state_get_rack/1, state_set_rack/2,\n                   state_get_raccepted/1, state_set_raccepted/2,\n                   state_get_value/1, state_set_value/2\n                   ]}).\n\n-type acceptor_state() ::\n        { any(),             %% PaxosID,\n          [ comm:mypid() ],  %% Learners,\n          non_neg_integer(), %% Rack\n          non_neg_integer() | -1, %% Raccepted\n          any()              %% Value\n        }.\n\n-spec state_new(any()) -> acceptor_state().\nstate_new(PaxosID) ->\n    {PaxosID, _Learners = [], _Rack = 0, _Raccepted = -1, paxos_no_value_yet}.\n-spec state_get_learners(acceptor_state()) -> [ comm:mypid() ].\nstate_get_learners(State) ->           element(2, State).\n-spec state_set_learners(acceptor_state(), [ comm:mypid() ]) -> acceptor_state().\nstate_set_learners(State, Learners) -> setelement(2, State, Learners).\n-spec state_get_rack(acceptor_state()) -> non_neg_integer().\nstate_get_rack(State) ->             element(3, State).\n-spec state_set_rack(acceptor_state(), non_neg_integer()) -> acceptor_state().\nstate_set_rack(State, Round) ->      setelement(3, State, Round).\n-spec state_get_raccepted(acceptor_state()) -> non_neg_integer() | -1.\nstate_get_raccepted(State) ->        element(4, State).\n-spec state_set_raccepted(acceptor_state(), non_neg_integer()) -> acceptor_state().\nstate_set_raccepted(State, Round) -> setelement(4, State, Round).\n-spec state_get_value(acceptor_state()) -> any().\nstate_get_value(State) ->            element(5, State).\n-spec state_set_value(acceptor_state(), any()) -> acceptor_state().\nstate_set_value(State, Value) ->     setelement(5, State, Value).\n\n-spec state_add_prepare_msg(acceptor_state(), non_neg_integer())\n        -> {ok, acceptor_state()} | {dropped, non_neg_integer()}.\nstate_add_prepare_msg(State, InRound) ->\n    Rack = state_get_rack(State),\n    case (InRound > Rack) andalso (InRound > state_get_raccepted(State)) of\n        true -> {ok, state_set_rack(State, InRound)};\n        false -> {dropped, Rack}\n    end.\n\n-spec state_add_accept_msg(acceptor_state(), non_neg_integer(), any())\n        -> {ok, acceptor_state()} | {dropped, non_neg_integer()}.\nstate_add_accept_msg(State, InRound, InProposal) ->\n    Rack = state_get_rack(State),\n    case (InRound >= Rack) andalso (InRound > state_get_raccepted(State)) of\n        true ->\n            NewState1 = state_set_raccepted(State, InRound),\n            NewState2 = state_set_value(NewState1, InProposal),\n            {ok, NewState2};\n        false -> {dropped, Rack}\n    end.\n\n-spec state_accepted(acceptor_state()) -> boolean().\nstate_accepted(State) -> paxos_no_value_yet =/= state_get_value(State).\n"
  },
  {
    "path": "src/paxos/learner.erl",
    "content": "% @copyright 2009-2015 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Part of a generic Paxos-Consensus implementation\n%%           The role of a learner.\n%% @end\n%% @version $Id$\n-module(learner).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n-export([start_paxosid/5, start_paxosid_local/5]).\n-export([stop_paxosids/2]).\n\n-export([start_link/2]).\n-export([on/2, init/1]).\n-export([check_config/0]).\n\n-include(\"gen_component.hrl\").\n-include(\"learner_state.hrl\").\n\n-type state() :: atom(). % table name\n\n-spec msg_decide(comm:mypid(), any(), any(), any()) -> ok.\nmsg_decide(Client, ClientCookie, PaxosID, Val) ->\n    comm:send(Client, {learner_decide, ClientCookie, PaxosID, Val}).\n\n-spec start_paxosid(comm:mypid(), any(), pos_integer(), comm:mypid(), any())\n                   -> ok.\nstart_paxosid(Learner, PaxosID, Majority, ProcessToInform, ClientCookie) ->\n    comm:send(Learner, {learner_initialize, PaxosID, Majority,\n                           ProcessToInform, ClientCookie}).\n\n-spec start_paxosid_local(pid(), any(), pos_integer(), comm:mypid(), any())\n                         -> ok.\nstart_paxosid_local(Learner, PaxosID, Majority, ProcessToInform, ClientCookie) ->\n    comm:send_local(Learner, {learner_initialize, PaxosID, Majority,\n                              ProcessToInform, ClientCookie}).\n\n-spec stop_paxosids(comm:mypid(), [any()]) -> ok.\nstop_paxosids(Learner, ListOfPaxosIDs) ->\n    comm:send(Learner, {learner_deleteids, ListOfPaxosIDs}).\n\n%% startable via supervisor, use gen_component\n-spec start_link(pid_groups:groupname(), pid_groups:pidname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup, PidName) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2,\n                             [],\n                             [{pid_groups_join_as, DHTNodeGroup, PidName},\n                              {spawn_opts, [{fullsweep_after, 0},\n                                            {min_heap_size, 16383}]}]).\n\n%% initialize: return initial state.\n-spec init([]) -> atom().\ninit([]) ->\n    ?TRACE(\"Starting learner for DHT node: ~p~n\", [pid_groups:my_groupname()]),\n    %% For easier debugging, use a named table (generates an atom)\n    %%TableName = erlang:list_to_atom(pid_groups:group_to_filename(pid_groups:my_groupname()) ++ \"_learner\"),\n    %%pdb:new(TableName, [set, protected, named_table]),\n    %% use random table name provided by ets to *not* generate an atom\n    TableName = pdb:new(?MODULE, [set]),\n    _State = TableName.\n\n-spec on(comm:message(), state()) -> state().\non({learner_initialize, PaxosID, Majority, ProcessToInform, ClientCookie},\n   ETSTableName = State) ->\n    ?TRACE(\"learner:initialize for paxos id: ~p~n\", [PaxosID]),\n    case pdb:get(PaxosID, ETSTableName) of\n        undefined -> pdb:set(state_new(PaxosID, Majority,\n                                       ProcessToInform, ClientCookie),\n                            ETSTableName);\n        StateForID ->\n            %% set Majority and ProcessInfo and check whether finished already\n            case Majority =:= state_get_majority(StateForID)\n                andalso ProcessToInform =:= state_get_process_to_inform(StateForID)\n            of\n                false ->\n                    TmpState = state_set_majority(StateForID, Majority),\n                    Tmp2State = state_set_process_to_inform(TmpState, ProcessToInform),\n                    NewState = state_set_client_cookie(Tmp2State, ClientCookie),\n                    case (Majority =< state_get_accepted_count(NewState)) of\n                        true -> decide(PaxosID, NewState);\n                        false -> ok\n                    end,\n                    pdb:set(NewState, ETSTableName);\n                true ->\n                    log:log(error,\n                            \"duplicate learner initialize for id ~p\",\n                            [PaxosID])\n            end\n    end,\n    State;\n\non({?acceptor_accept, PaxosID, Round, Value}, ETSTableName = State) ->\n    ?TRACE(\"learner:accepted for paxosid '~p' and round '~p' value '~p'~n\",\n           [PaxosID, Round, Value]),\n    MyState = case pdb:get(PaxosID, ETSTableName) of\n                  undefined ->\n                      msg_delay:send_local(\n                        config:read(learner_noinit_timeout) div 1000, self(),\n                        {learner_deleteid_if_still_no_client, PaxosID}),\n                      state_new(PaxosID, 128, none, no_cookie);\n                  StateForID -> StateForID\n              end,\n    case state_add_accepted_msg(MyState, Round, Value) of\n        {majority_accepted, NewState} ->\n            decide(PaxosID, NewState),\n            pdb:set(NewState, ETSTableName);\n        {ok, NewState} -> pdb:set(NewState, ETSTableName);\n        dropped -> ok\n    end,\n    State;\n\non({learner_deleteids, ListOfPaxosIDs}, ETSTableName = State) ->\n    _ = [pdb:delete(Id, ETSTableName) || Id <- ListOfPaxosIDs],\n    State;\n\non({learner_deleteid_if_still_no_client, PaxosID}, ETSTableName = State) ->\n    case pdb:get(PaxosID, ETSTableName) of\n        undefined -> ok;\n        StateForId ->\n            case state_get_process_to_inform(StateForId) of\n                none ->\n                    %% io:format(\"Deleting unhosted learner id~n\"),\n                    pdb:delete(PaxosID, ETSTableName);\n                _ -> ok\n            end\n    end,\n    State;\n\non(_, _State) ->\n    unknown_event.\n\ndecide(PaxosID, State) ->\n    ?TRACE(\"learner:decide for paxosid '~p' in round '~p' and value '~p'~n\",\n           [PaxosID, state_get_round(State), state_get_value(State)]),\n    case state_get_process_to_inform(State) of\n        none -> ok; % will be informed later when learner is initialized\n        Pid -> msg_decide(Pid, state_get_client_cookie(State),\n                          PaxosID, state_get_value(State))\n    end.\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(learner_noinit_timeout) and\n    config:cfg_is_greater_than_equal(learner_noinit_timeout, 1000) and\n    config:cfg_is_greater_than(learner_noinit_timeout, tx_timeout).\n"
  },
  {
    "path": "src/paxos/learner_state.hrl",
    "content": "% @copyright 2009-2012 Zuse Institute Berlin,\n%                      onScale solutions GmbH\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Part of generic Paxos-Consensus implementation\n%%      The state needed for a single learner instance.\n%% @end\n%% @version $Id$\n\n-compile({inline, [%state_get_paxosid/1,\n                   state_get_majority/1, state_set_majority/2,\n                   state_get_process_to_inform/1, state_set_process_to_inform/2,\n                   state_get_client_cookie/1, state_set_client_cookie/2,\n                   state_get_accepted_count/1, state_set_accepted_count/2,\n                   state_inc_accepted_count/1,\n                   state_get_round/1, state_set_round/2,\n                   state_get_value/1, state_set_value/2\n                  ]}).\n\n-type proc_to_inform() :: comm:mypid() | none.\n-type learner_state() ::\n        {any(),             % PaxosID,\n         pos_integer(),     % Majority,\n         proc_to_inform(),  % ProcessToInform\n         any(),             % ClientCookie\n         non_neg_integer(), % AcceptedCount\n         non_neg_integer(), % Round\n         any()              % Value\n        }.\n%% Value stored to accept messages for a paxos id before learner is\n%% initialized. (and for sanity checks)\n\n-spec state_new(any(), pos_integer(), proc_to_inform(), any()) -> learner_state().\nstate_new(PaxosID, Majority, ProcessToInform, ClientCookie) ->\n    {PaxosID, Majority, ProcessToInform, ClientCookie,\n     0, 0, paxos_no_value_yet}.\n\n%% -spec state_get_paxosid(learner_state()) -> any().\n%% state_get_paxosid(State) -> element(1, State).\n-spec state_get_majority(learner_state()) -> pos_integer().\nstate_get_majority(State) -> element(2, State).\n-spec state_set_majority(learner_state(), pos_integer()) -> learner_state().\nstate_set_majority(State, Majority) -> setelement(2, State, Majority).\n-spec state_get_process_to_inform(learner_state()) -> proc_to_inform().\nstate_get_process_to_inform(State) -> element(3, State).\n-spec state_set_process_to_inform(learner_state(), comm:mypid()) -> learner_state().\nstate_set_process_to_inform(State, Pid) -> setelement(3, State, Pid).\n-spec state_get_client_cookie(learner_state()) -> any().\nstate_get_client_cookie(State) -> element(4, State).\n-spec state_set_client_cookie(learner_state(), any()) -> learner_state().\nstate_set_client_cookie(State, Pid) -> setelement(4, State, Pid).\n-spec state_get_accepted_count(learner_state()) -> non_neg_integer().\nstate_get_accepted_count(State) -> element(5, State).\n-spec state_set_accepted_count(learner_state(), non_neg_integer()) -> learner_state().\nstate_set_accepted_count(State, Num) -> setelement(5, State, Num).\n-spec state_inc_accepted_count(learner_state()) -> learner_state().\nstate_inc_accepted_count(State) -> setelement(5, State, element(5, State) + 1).\n-spec state_get_round(learner_state()) -> non_neg_integer().\nstate_get_round(State) -> element(6, State).\n-spec state_set_round(learner_state(), non_neg_integer()) -> learner_state().\nstate_set_round(State, Round) -> setelement(6, State, Round).\n-spec state_get_value(learner_state()) -> any().\nstate_get_value(State) -> element(7, State).\n-spec state_set_value(learner_state(), any()) -> learner_state().\nstate_set_value(State, Value) -> setelement(7, State, Value).\n\n-spec state_add_accepted_msg(learner_state(), non_neg_integer(), any())\n        -> dropped | {ok | majority_accepted, learner_state()}.\nstate_add_accepted_msg(State, Round, Value) ->\n    StateRound = state_get_round(State),\n    if Round >= StateRound ->\n           TmpState = if Round > StateRound ->\n                             % reset round and accepted:\n                             state_set_round(\n                               state_set_accepted_count(State, 0), Round);\n                         true -> State\n                      end,\n           Tmp2State = state_set_value(TmpState, Value),\n           NewState = state_inc_accepted_count(Tmp2State),\n           case state_get_accepted_count(NewState)\n                    =:= state_get_majority(NewState) of\n               true -> {majority_accepted, NewState};\n               false -> {ok, NewState}\n           end;\n       true -> dropped % outdated round, silently drop it\n    end.\n"
  },
  {
    "path": "src/paxos/proposer.erl",
    "content": "% @copyright 2009-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Description : Part of generic Paxos-Consensus implementation\n%%           The role of a proposer.\n%% @end\n%% @version $Id$\n-module(proposer).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n%%% public interface for triggering a paxos proposer executed in any process\n%%% a Fast-Paxos is triggered by giving 0 as initial round number explicitly\n-export([start_paxosid/6,start_paxosid/7]).\n-export([stop_paxosids/2]).\n-export([trigger/2]).\n-export([msg_accept/5]).\n\n%%% functions for gen_component module and supervisor callbacks\n-export([start_link/2]).\n-export([on/2, init/1]).\n\n-include(\"gen_component.hrl\").\n-include(\"proposer_state.hrl\").\n\n-type state() :: atom(). % TableName\n\n%%% public function to start a new paxos instance gets as parameters:\n%%%   PaxosID: has to be unique in the system, user has to care about this\n%%%   Acceptors: a list of paxos_acceptor processes, that are used\n%%%   Proposal: if no consensus is available beforehand, this proposer proposes this\n%%%   Majority: how many responses from acceptors have to be collected?\n%%%   MaxProposers: the maximum number of proposers used for this paxos instance\n%%%     (it is used to generate unique round numbers. Offset is the given initialRound)\n%%%   InitialRound (optional): start with paxos round number (default 1)\n%%%     if InitialRound is 0, a Fast-Paxos is executed\n%%%     InitialRound must be unique for all proposers of a paxosid:\n%%%       1 &lt;= initialRound &lt;= MaxProposers for normal paxos\n%%%       0 &lt;= initialRound &lt; MaxProposers if 1 proposer uses fast paxos\n\nmsg_prepare(Dest, ReplyTo, PaxosID, Round) ->\n    ?TRACE(\"Sending proposer_prepare: ~p, ~p~n\", [PaxosID, Round]),\n    Msg = {proposer_prepare, ReplyTo, PaxosID, Round},\n    comm:send(Dest, Msg).\n\n-spec msg_accept(comm:mypid(), comm:mypid(), any(),\n                 non_neg_integer(), any()) -> ok.\nmsg_accept(Dest, ReplyTo, PaxosID, Round, Value) ->\n    ?TRACE(\"Sending proposer_accept ~p, ~p Proposal ~p~n\", [PaxosID, Round, Value]),\n    Msg = {?proposer_accept, ReplyTo, PaxosID, Round, Value},\n    comm:send(Dest, Msg).\n\n-spec start_paxosid(comm:mypid(), any(), [ comm:mypid() ], any(),\n                    pos_integer(), pos_integer()) -> ok.\nstart_paxosid(Proposer, PaxosID, Acceptors, Proposal,\n              Majority, MaxProposers) ->\n    start_paxosid(Proposer, PaxosID, Acceptors, Proposal,\n                  Majority, MaxProposers, 1).\n-spec start_paxosid(comm:mypid(), any(), [ comm:mypid() ], any(),\n                    pos_integer(), pos_integer(), non_neg_integer()) -> ok.\nstart_paxosid(Proposer, PaxosID, Acceptors, Proposal,\n              Majority, MaxProposers, InitialRound) ->\n    Msg = {?proposer_initialize, PaxosID, Acceptors, Proposal,\n           Majority, MaxProposers, InitialRound},\n    comm:send(Proposer, Msg).\n\n-spec stop_paxosids(comm:mypid(), any()) -> ok.\nstop_paxosids(Proposer, PaxosIds) ->\n    comm:send(Proposer, {?proposer_deleteids, PaxosIds}).\n\n-spec trigger(comm:mypid(), any()) -> ok.\ntrigger(Proposer, PaxosID) ->\n    comm:send(Proposer, {proposer_trigger, PaxosID}).\n\n%% be startable via supervisor, use gen_component\n-spec start_link(pid_groups:groupname(), pid_groups:pidname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup, PidName) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2,\n                             [],\n                             [{pid_groups_join_as, DHTNodeGroup, PidName},\n                              {spawn_opts, [{fullsweep_after, 0},\n                                            {min_heap_size, 16383}]}]).\n\n%% initialize: return initial state.\n-spec init([]) -> state().\ninit([]) ->\n    ?TRACE(\"Starting proposer for DHT node: ~p~n\", [pid_groups:my_groupname()]),\n    %% For easier debugging, use a named table (generates an atom)\n    %%TableName = erlang:list_to_atom(pid_groups:group_to_filename(pid_groups:my_groupname()) ++ \"_proposer\"),\n    %%pdb:new(TableName, [set, protected, named_table]),\n    %% use random table name provided by ets to *not* generate an atom\n    TableName = pdb:new(?MODULE, [set]),\n    _State = TableName.\n\n-spec on(comm:message(), state()) -> state().\non({?proposer_initialize, PaxosID, Acceptors, Proposal,\n    Majority, MaxProposers, InitialRound},\n   State) ->\n    on({?proposer_initialize, PaxosID, Acceptors, Proposal,\n        Majority, MaxProposers, InitialRound,\n        _ReplyTo = comm:this()},\n       State);\n\non({?proposer_initialize, PaxosID, Acceptors, Proposal, Majority,\n    MaxProposers, InitialRound, ReplyTo},\n   ETSTableName = State) ->\n    ?TRACE(\"proposer:initialize for paxos id: ~p round ~p~n\", [PaxosID,InitialRound]),\n    case pdb:get(PaxosID, ETSTableName) of\n        undefined ->\n            StateForID = state_new(PaxosID, ReplyTo, Acceptors, Proposal,\n                                   Majority, MaxProposers, InitialRound),\n            pdb:set(StateForID, ETSTableName);\n        StateForID ->\n            log:log(error, \"Duplicate proposer:initialize for paxos id ~p\"\n                           \"Just triggering instead~n\", [PaxosID])\n    end,\n    proposer_trigger(StateForID, PaxosID, InitialRound, State);\n\n% trigger new proposer round\non({proposer_trigger, PaxosID}, ETSTableName = State) ->\n    ?TRACE(\"proposer:trigger for paxos id ~p with auto round increment~n\", [PaxosID]),\n    case pdb:get(PaxosID, ETSTableName) of\n        undefined -> State;\n        StateForID ->\n            TmpState = state_reset_state(StateForID),\n            NewState = state_inc_round(TmpState),\n            pdb:set(NewState, ETSTableName),\n            proposer_trigger(StateForID, PaxosID, state_get_round(NewState), State)\n    end;\n\n%% trigger for given round is needed for initial round without auto-increment\n%% and fast forward, but be careful:\n%% Rounds must always have the form \"InitialRound + x * MaxProposers\"\non({proposer_trigger, PaxosID, Round}, ETSTableName = State) ->\n    ?TRACE(\"proposer:trigger for paxos id ~p and round ~p~n\", [PaxosID, Round]),\n    case pdb:get(PaxosID, ETSTableName) of\n        undefined  -> State;\n        StateForID -> proposer_trigger(StateForID, PaxosID, Round, ETSTableName)\n    end;\n\non({acceptor_ack, PaxosID, Round, Value, RLast}, ETSTableName = State) ->\n    ?TRACE(\"proposer:ack for paxos id ~p round ~p~n\", [PaxosID, Round]),\n    _ = case pdb:get(PaxosID, ETSTableName) of\n        undefined ->\n            %% What to do when this PaxosID does not already exist? Think!\n            %% -> Proposers don't get messages, they not requested.\n            ok;\n        StateForID ->\n            case state_add_ack_msg(StateForID, Round, Value, RLast) of\n                {ok, NewState} ->\n                    %% ?TRACE(\"NEW State: ~p~n\", [NewState]),\n                    pdb:set(NewState, ETSTableName);\n                {majority_acked, NewState} ->\n                    %%   multicast accept(Round, Latest_value) to Acceptors\n                    %% ?TRACE(\"NEW State: ~p majority accepted~n\", [NewState]),\n                    pdb:set(NewState, ETSTableName),\n                    Acceptors = state_get_acceptors(NewState),\n                    ReplyTo = state_get_replyto(NewState),\n                    LatestVal = state_get_latest_value(NewState),\n                    [msg_accept(X, ReplyTo, PaxosID, Round, LatestVal)\n                     || X <- Acceptors]\n            end\n    end,\n    State;\n\non({acceptor_nack, PaxosID, Round}, _ETSTableName = State) ->\n    ?TRACE(\"proposer:nack for paxos id ~p and round ~p is newest seen~n\",\n           [PaxosID, Round]),\n    start_new_higher_round(PaxosID, Round, State),\n    State;\n\non({acceptor_naccepted, PaxosID, Round}, _ETSTableName = State) ->\n    ?TRACE(\"proposer:naccepted for paxos id ~p and round ~p is newest seen~n\",\n           [PaxosID, Round]),\n    start_new_higher_round(PaxosID, Round, State),\n    State;\n\non({?proposer_deleteids, ListOfPaxosIDs}, ETSTableName = State) ->\n    _ = [pdb:delete(Id, ETSTableName) || Id <- ListOfPaxosIDs],\n    State;\n\non(_, _State) ->\n    unknown_event.\n\n-spec proposer_trigger(StateForID::proposer_state(), PaxosID::any(),\n                       Round::non_neg_integer(), state()) -> state().\nproposer_trigger(StateForID, PaxosID, Round, ETSTableName = State) ->\n    Acceptors = state_get_acceptors(StateForID),\n    ReplyTo = state_get_replyto(StateForID),\n    Proposal = state_get_proposal(StateForID),\n    _ = case Round of\n            0 -> [msg_accept(X, ReplyTo,\n                             PaxosID, Round,\n                             Proposal)\n                    || X <- Acceptors];\n            _ -> [msg_prepare(X, ReplyTo, PaxosID, Round)\n                    || X <- Acceptors]\n        end,\n    case Round > state_get_round(StateForID) of\n        true ->\n            pdb:set(state_set_round(StateForID, Round),\n                    ETSTableName);\n        false -> ok\n    end,\n    State.\n\nstart_new_higher_round(PaxosID, Round, ETSTableName) ->\n    case pdb:get(PaxosID, ETSTableName) of\n        undefined -> ok;\n        StateForID ->\n            MyRound = state_get_round(StateForID),\n            %% check whether outdated nack message? (we get them from each acceptor)\n            case MyRound < Round of\n                true ->\n                    MaxProposers = state_get_max_proposers(StateForID),\n                    Factor = (Round - MyRound) div MaxProposers + 1,\n                    NextRound = MyRound + Factor * MaxProposers,\n                    %% let other prop. more time (NextRound ms) to achieve consensus\n                    TmpState = state_reset_state(StateForID),\n                    pdb:set(state_set_round(TmpState, NextRound), ETSTableName),\n                    NewMsg = {proposer_trigger, PaxosID, NextRound},\n%%                     _ = comm:send_local_after(NextRound, self(),\n%%                                               {proposer_trigger, PaxosID,\n%%                                                NextRound});\n                    case randoms:rand_uniform(0, 2) of\n                        0 -> comm:send_local(self(), NewMsg);\n                        1 -> msg_delay:send_local(0, self(), NewMsg) % delay < 1s\n                    end;\n                false -> dropped\n            end\n    end.\n"
  },
  {
    "path": "src/paxos/proposer_state.hrl",
    "content": "% @copyright 2009-2012 Zuse Institute Berlin,\n%                      onScale solutions GmbH\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Part of generic Paxos-Consensus implementation\n%%       The state needed for a single proposer instance.\n%% @end\n%% @version $Id$\n\n-compile({inline, [state_get_replyto/1,\n                   state_get_acceptors/1,\n                   state_get_proposal/1,\n                   state_get_majority/1,\n                   state_get_max_proposers/1,\n                   state_get_round/1, state_set_round/2,\n                   state_get_latest_value/1, state_set_latest_value/2,\n                   state_get_ack_count/1, %state_inc_ack_count/1,\n                   state_inc_round/1,\n                   state_reset_state/1\n                   ]}).\n\n-type proposer_state() ::\n        { any(),            % paxos_id\n          comm:mypid(),     % ReplyTo,\n          [ comm:mypid() ], % Acceptors,\n          any(),            % Own_proposal (e.g. prepared / abort),\n          pos_integer(),    % Majority,\n          pos_integer(),    % MaxProposers,\n          non_neg_integer(),    % Round, (current round)\n          non_neg_integer(),    % RLast_highest,\n          any(),            % Latest_value,\n          non_neg_integer()     % Ack_rec_count\n        }.\n\n-spec state_new(any(), comm:mypid(), [comm:mypid()], any(), pos_integer(), pos_integer(), non_neg_integer()) -> proposer_state().\nstate_new(PaxosID, ReplyTo, Acceptors, Proposal, Majority, MaxProposers, Round) ->\n    {PaxosID, ReplyTo, Acceptors, Proposal, Majority, MaxProposers,\n     Round, 0, paxos_no_value_yet, 0}.\n\n-spec state_get_replyto(proposer_state()) -> comm:mypid().\nstate_get_replyto(State) ->           element(2, State).\n-spec state_get_acceptors(proposer_state()) -> [ comm:mypid() ].\nstate_get_acceptors(State) ->         element(3, State).\n-spec state_get_proposal(proposer_state()) -> any().\nstate_get_proposal(State) ->          element(4, State).\n-spec state_get_majority(proposer_state()) -> pos_integer().\nstate_get_majority(State) ->          element(5, State).\n-spec state_get_max_proposers(proposer_state()) -> pos_integer().\nstate_get_max_proposers(State) ->     element(6, State).\n-spec state_get_round(proposer_state()) -> non_neg_integer().\nstate_get_round(State) ->             element(7, State).\n-spec state_set_round(proposer_state(), non_neg_integer()) -> proposer_state().\nstate_set_round(State, Round) ->      setelement(7, State, Round).\n-spec state_get_latest_value(proposer_state()) -> any().\nstate_get_latest_value(State) ->      element(9, State).\n-spec state_set_latest_value(proposer_state(), any()) -> proposer_state().\nstate_set_latest_value(State, Val) -> setelement(9, State, Val).\n-spec state_get_ack_count(proposer_state()) -> non_neg_integer().\nstate_get_ack_count(State) ->         element(10, State).\n%% -spec state_inc_ack_count(proposer_state()) -> proposer_state().\n%% state_inc_ack_count(State) ->         setelement(10, State,\n%%                                            state_get_ack_count(State) + 1).\n-spec state_inc_round(proposer_state()) -> proposer_state().\nstate_inc_round(State) ->\n    state_set_round(State, state_get_round(State) + state_get_max_proposers(State)).\n\n-spec state_reset_state(proposer_state()) -> proposer_state().\nstate_reset_state(State) ->           setelement(10, State, 0).\n\n-compile({nowarn_unused_function, state_add_ack_msg_feeder/4}).\n-spec state_add_ack_msg_feeder(InState, InAckRound, InAckValue, InAckRLast)\n        -> {InState, InAckRound, InAckValue, InAckRLast}\n            when is_subtype(InState, proposer_state()),\n                 is_subtype(InAckRound, non_neg_integer()),\n                 is_subtype(InAckValue, any()),\n                 is_subtype(InAckRLast, non_neg_integer()).\nstate_add_ack_msg_feeder(InState, InAckRound, InAckValue, InAckRLast) ->\n    StateRound = state_get_round(InState),\n    {state_set_round(InState, erlang:max(StateRound, InAckRound)),\n     erlang:min(StateRound, InAckRound), InAckValue, InAckRLast}.\n\n-spec state_add_ack_msg(proposer_state(), non_neg_integer(), any(),\n                        non_neg_integer()) -> {ok | majority_acked, proposer_state()}.\nstate_add_ack_msg(InState, InAckRound, InAckValue, InAckRLast) ->\n    {PaxosID, ReplyTo, Acceptors, Proposal, Majority, MaxProposers,\n     StateRound, StateRLast_highest, StateLatestValue, AckRecCount}\n        = InState,\n    NewState =\n        %% check r == state.Round == current?\n        if InAckRound =:= StateRound ->\n               %% increment AckRecCount\n               %% if Rlast > state.RLast_highest\n               %%   update Rlast_highest and Latest_value\n               {NewRLast_highest, NewLatestValue} =\n                   if InAckRLast > StateRLast_highest ->\n                          {InAckRLast, InAckValue};\n                      true ->\n                          {StateRLast_highest, StateLatestValue}\n                   end,\n               {PaxosID, ReplyTo, Acceptors, Proposal, Majority, MaxProposers,\n                StateRound, NewRLast_highest, NewLatestValue, AckRecCount + 1};\n           true ->\n               %% InAckRound =< StateRound -> old ack message, drop it\n               %% InAckRound > StateRound should not happen\n               %% (we only receive msgs for round we started ourselves)\n               ?DBG_ASSERT2(InAckRound =< StateRound,\n                            {got_higher_round, InAckRound, StateRound}),\n               InState\n        end,\n    case state_get_ack_count(NewState) =:= state_get_majority(NewState) of\n        %% if collected majority of answers\n        %% ignore acks greater than majority. An 'accept'\n        %%   was sent to all acceptors when majority was gathered\n        true ->\n            %% io:format(\"Proposer: majority acked in round ~p~n\",\n            %%           [get_round(NewState)]),\n            {majority_acked,\n             case state_get_latest_value(NewState) of\n                 paxos_no_value_yet ->\n                     state_set_latest_value(NewState, Proposal);\n                 _ -> NewState\n             end\n            };\n        false -> {ok, NewState}\n    end.\n    %% multicast accept(Round, Latest_value) to Acceptors\n    %% (done in on() handler)\n"
  },
  {
    "path": "src/pdb.erl",
    "content": "%% @copyright 2010-2012 Zuse Institute Berlin\n%%            and onScale solutions GmbH\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc DB for a process internal state (lika a gen_component).\n%% This abstraction allows for easy switching between\n%% erlang:put/get/erase and ets:insert/lookup/delete\n%% @end\n%% @version $Id$\n-module(pdb).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-behaviour(pdb_beh).\n\n-export_type([tableid/0]).\n\n-export([new/2, get/2, set/2, delete/2, take/2, tab2list/1]).\n-include(\"scalaris.hrl\").\n\n-type tableid() :: atom().\n\n-spec new(TableName::tableid(),\n          [set | ordered_set | bag | duplicate_bag |\n           public | protected | private |\n           named_table | {keypos, integer()} |\n           {heir, pid(), term()} | {heir,none} |\n           {write_concurrency, boolean()}]) -> tableid().\nnew(TableName, _Params) -> TableName.\n\n-spec get(term(), tableid()) -> tuple() | undefined.\nget(Key, _TableName) -> erlang:get(Key).\n\n-spec set(tuple(), tableid()) -> ok.\nset({}, _TableName) -> ok; %% empty_tuple is forbidden as key\nset(NewTuple, _TableName) ->\n    erlang:put(element(1, NewTuple), NewTuple),\n    ok.\n\n-spec delete(term(), tableid()) -> ok.\ndelete(Key, _TableName) -> erlang:erase(Key), ok.\n\n-spec take(term(), tableid()) -> term() | undefined.\ntake(Key, _TableName) -> erlang:erase(Key).\n\n-spec tab2list(tableid()) -> [term()].\ntab2list(_TableName) -> [ X || {_,X} <- erlang:get()].\n\n"
  },
  {
    "path": "src/pdb_beh.erl",
    "content": "%% @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Behaviour for pdb (see pdb.erl and pdb_ets.erl)\n%% @end\n%% @version $Id$\n-module(pdb_beh).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n% for behaviour\n-ifndef(have_callback_support).\n-export([behaviour_info/1]).\n-endif.\n\n-ifdef(have_callback_support).\n-type tableid() :: term().\n\n-callback new(TableName::atom(),\n              [set | ordered_set | bag | duplicate_bag |\n               public | protected | private |\n               named_table | {keypos, integer()} |\n               {heir, pid(), term()} | {heir,none} |\n               {write_concurrency, boolean()}]) -> tableid().\n\n-callback get(term(), tableid()) -> tuple() | undefined.\n-callback set(tuple(), tableid()) -> ok.\n-callback delete(term(), tableid()) -> ok.\n-callback take(term(), tableid()) -> term() | undefined.\n-callback tab2list(tableid()) -> [term()].\n\n-else.\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n    [ {new, 2},\n      {get, 2},\n      {set, 2},\n      {delete, 2},\n      {take, 2},\n      {tab2list, 1}\n    ];\nbehaviour_info(_Other) ->\n    undefined.\n-endif.\n\n"
  },
  {
    "path": "src/pdb_ets.erl",
    "content": "%% @copyright 2010-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc DB for a process internal state (lika a gen_component).\n%% This abstraction allows for easy switching between\n%% erlang:put/get/erase and ets:insert/lookup/delete\n%% @end\n%% @version $Id$\n-module(pdb_ets).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-behaviour(pdb_beh).\n\n-export_type([tableid/0]).\n\n-export([new/2, get/2, set/2, delete/2, take/2, tab2list/1]).\n-include(\"scalaris.hrl\").\n\n-type tableid() :: ets:tid() | atom().\n\n-spec new(TableName::atom(),\n          [set | ordered_set | bag | duplicate_bag |\n           public | protected | private |\n           named_table | {keypos, integer()} |\n           {heir, pid(), term()} | {heir,none} |\n           {write_concurrency, boolean()}]) -> tableid().\nnew(TableName, Params) ->\n    ets:new(TableName, Params).\n\n-spec get(term(), tableid()) -> tuple() | undefined.\nget(Key, TableName) ->\n    case ets:lookup(TableName, Key) of\n        [] -> undefined;\n        [Entry] -> Entry\n    end.\n\n-spec set(tuple(), tableid()) -> ok.\nset(NewTuple, TableName) ->\n    ets:insert(TableName, NewTuple),\n    ok.\n\n-spec delete(term(), tableid()) -> ok.\ndelete(Key, TableName) ->\n    ets:delete(TableName, Key),\n    ok.\n\n-spec take(term(), tableid()) -> term() | undefined.\ntake(Key, TableName) ->\n    Old = get(Key, TableName),\n    ets:delete(TableName, Key),\n    Old.\n\n-spec tab2list(tableid()) -> [term()].\ntab2list(TableName) ->\n  ets:tab2list(TableName).\n"
  },
  {
    "path": "src/pid_groups.erl",
    "content": "% @copyright 2007-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Process groups.\n%%\n%%      This module provides a mechanism to implement process\n%%      groups. Within a process group, the names of processes have to\n%%      be unique, but the same name can be used in different\n%%      groups. The motivation for this module was to run several\n%%      scalaris nodes in one erlang vm for debugging. But for the\n%%      processes forming a scalaris node being able to talk to each\n%%      other, they have to know their names (dht_node, config,\n%%      etc.). This module allows the processes to keep their names.\n%%\n%%      When a new process group is created, a unique \"groupname\" is\n%%      created, which has to be shared by all nodes in this group.\n%% @end\n%% @version $Id$\n-module(pid_groups).\n-author('schuett@zib.de').\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n-export([start_link/0]).\n\n%% gen_server callbacks\n-export([init/1,on/2]).\n\n%% group creation\n-export([new/0,     %% () -> GrpName\n         new/1]).   %% (PrefixAtom) -> GrpName\n\n%% group membership management\n-export([join/1,          %% (GrpName) -> ok               deprecated\n         join_as/2,       %% (GrpName, PidName) -> ok\n         add/3]).         %% (GrpName, PidName, Pid) -> ok\n\n%% operations inside a group (PRE: caller is group member):\n-export([my_groupname/0,  %% () -> GrpName\n         my_pidname/0,    %% () -> PidName\n         get_my/1,        %% (PidName) -> Pid\n         my_members/0,    %% () -> [Pids]\n         my_tab2list/0]). %% () -> [{GrpName, PidName, Pid}]\n\n%% get information on groups for non-members\n-export([pid_of/2,            %% (GrpName, PidName) -> Pid\n         group_and_name_of/1, %% (Pid) -> {GrpName, PidName}\n         group_of/1,          %% (Pid) -> GrpName\n         group_with/1,        %% (PidName) -> GrpName\n         groups_with/1,       %% (PidName) -> [GrpName]\n\n         find_a/1,            %% (PidName) -> Pid\n         find_all/1,          %% (PidName) -> [Pid]\n\n         members/1,           %% (GrpName) -> [Pids]\n         members_by_name/1,   %% (GrpName) -> [pidname()]\n\n         pid_to_name/1,       %% translation of pids to strings for debugging\n         pids_to_names/2]).\n\n%% global information\n-export([processes/0,     %% () -> [Pid]    %% for fprof\n         groups/0,        %% () -> [GrpName]\n         tab2list/0]).    %% () -> [{GrpName, PidName, Pid}]\n\n%% for persistency\n-export([group_to_filename/1,   %% (GrpName) -> string()\n         filename_to_group/1]). %% (string()) -> GrpName\n\n%% for the monitoring in the web interface\n-export([get_web_debug_info/2,    %% (Group, PidName) -> any()\n         groups_as_json/0,\n         members_by_name_as_json/1,\n         group_to_string/1,\n         string_to_group/1]).\n\n%% for unittests when group is in breakpoint\n-export([hide/1, unhide/1]).\n\n-export_type([pidname/0, groupname/0]).\n\n-type(pidname() :: nonempty_string()                       % vivaldi and fd\n                 | atom()\n                 | {atom(), atom()}\n                 | {atom(), pos_integer()}                 % the nth lease_db\n                 | {{atom(), pos_integer()}, atom()}).     % the acceptor of the nths tm\n-type(groupname() :: atom()\n                   | pos_integer()\n                   | {atom(), pos_integer() | atom()}).\n\n-type(message() ::\n    {pid_groups_add, groupname(), pidname(), pid()} |\n    {drop_state} |\n    {group_and_name_of, Pid::comm:mypid(), Source::comm:mypid()} |\n    {'EXIT', pid(), Reason::any()}).\n\n-include(\"gen_component.hrl\").\n\n%%% group creation\n%% @doc create a new group with a random name.\n-spec new() -> groupname().\nnew() ->\n    randoms:getRandomInt().\n\n%% @doc create a new group with a given prefix.\n-spec new(atom()) -> groupname().\nnew(Prefix) ->\n    {Prefix, new()}.\n\n%%% group membership management\n%% @doc Current process joins the group GrpName, but has no process name.\n%%\n%% DEPRECATED! client processes should not join groups.  Other\n%% processes should have a unique role name and join via join_as or\n%% add.\n-spec join(groupname()) -> ok.\njoin(GrpName) ->\n    %% @todo eliminate this function\n    erlang:put('$@groupname', GrpName), ok.\n\n%% @doc Current process joins the group GrpName with process name Pidname.\n-spec join_as(groupname(), pidname()) -> ok.\njoin_as(GrpName, PidName) ->\n    erlang:put('$@groupname', GrpName), %% for my_groupname()\n    erlang:put('$@pidname', PidName),   %% for my_pidname()\n    add(GrpName, PidName, self()).\n\n%% @doc Third party register: add a pid with a given pidname (role) to a group.\n-spec add(groupname(), pidname(), pid()) -> ok.\nadd(GrpName, PidName, Pid) ->\n    comm:send_local(pid_groups_manager(),\n                    {pid_groups_add, GrpName, PidName, Pid, self()}),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({pid_groups_add_done}, ok);\n        ?SCALARIS_RECV(\n            {pid_groups_add_done, OldGroup, OldName}, %% ->\n            throw({Pid, already_registered_as, OldGroup, OldName, was_trying, GrpName, PidName})\n          )\n    end.\n\n\n%%% operations inside a group (PRE_COND: caller is group member):\n-spec my_groupname() -> groupname() | undefined.\nmy_groupname() -> erlang:get('$@groupname').\n\n-spec my_pidname() -> pidname() | undefined.\nmy_pidname() -> erlang:get('$@pidname').\n\n%% @doc Gets the Pid of the current process' group member with the given name.\n-spec get_my(pidname()) -> pid() | failed.\nget_my(PidName) ->\n    case my_groupname() of\n        undefined -> failed;\n        GrpName   -> cached_lookup(GrpName, PidName)\n    end.\n\n-spec my_members() -> [pid()].\nmy_members() -> members(my_groupname()).\n\n-spec my_tab2list() -> [{{groupname(), pidname()}, pid()}].\nmy_tab2list() ->\n    GrpName = my_groupname(),\n    [ X || X <- tab2list(), GrpName =:= element(1, element(1, X)) ].\n\n%% get information on groups for non-members\n\n%% @doc lookup a pid via its group name and pid name.\n-spec pid_of(groupname(), pidname()) -> pid() | failed.\npid_of(GrpName, PidName) ->\n    cached_lookup(GrpName, PidName).\n\n%% @doc lookup group and pid name of a process via its pid.\n-spec group_and_name_of(pid()) -> {groupname(), pidname()} | failed.\n%% maybe used with reg_name() but '_' would violates result type.\ngroup_and_name_of('_') -> failed;\ngroup_and_name_of(Pid) ->\n    case ets:match(?MODULE, {'$1',Pid}) of\n        [[{GrpName, PidName}]] -> {GrpName, PidName};\n        []                -> failed\n    end.\n\n-spec group_of(pid()) -> groupname() | failed.\ngroup_of(Pid) ->\n    case group_and_name_of(Pid) of\n        failed -> failed;\n        {GrpName, _PidName} -> GrpName\n    end.\n\n%% @doc find a process group with a given process name inside\n-spec group_with(pidname()) -> groupname() | failed.\ngroup_with(PidName) ->\n    case ets:match(?MODULE, {{'$1', PidName}, '_'}) of\n        [[GrpName] | _] -> GrpName;\n        []            -> failed\n    end.\n\n%% @doc find all process groups with a given process name inside\n-spec groups_with(pidname()) -> [groupname()] | failed.\ngroups_with(PidName) ->\n    case ets:match(?MODULE, {{'$1', PidName}, '_'}) of\n        [_|_] = Groups -> lists:append(Groups);\n        []   -> failed\n    end.\n\n%% @doc find a pid with the given name (search in own group first), otherwise\n%% allow round-robin returns of all the pids with this name in any group\n-spec find_a(pidname()) -> pid() | failed.\nfind_a(PidName) ->\n    cached_lookup('_', PidName).\n\n%% @doc perform a lookup with caching and cheking whether the cache is up to date.\n%% Commonly used by find_a/1 and get_my/1. get_my should only search in the own\n%% group, while find_a/1 first searches the own group and then searches in other\n%% groups (parameter SearchGrp =:= '_' instead of a groupname()). For find_a/1,\n%% if several matching pidnames are found, we use a round-robin cache per client\n%% process to spread the load.\n%% @private\n-spec cached_lookup('_' | groupname(), pidname()) -> pid() | failed.\ncached_lookup(SearchGrp, PidName) ->\n    CachedName = {'$?scalaris_pid_groups_cache', SearchGrp, PidName},\n    case erlang:get(CachedName) of\n        undefined ->\n            %% try in my own group first\n            X = case my_groupname() of\n                    undefined -> failed;\n                    SearchGrp ->\n                        %% lookup in own group\n                        case ets:lookup(?MODULE, {SearchGrp, PidName}) of\n                            [{{SearchGrp, PidName}, FPid}] -> FPid;\n                            [] -> failed\n                        end;\n                    GrpName ->\n                        %% generic search, but lookup in own group first.\n                        %% recursive call to ask cache of own group before doing a lookup\n                        cached_lookup(GrpName, PidName)\n                end,\n            %% when (failed =:= X) -> search in own group did not help\n            case X of\n                failed ->\n                    %% search others (when SearchGrp =:= '_' as in find_a/1)\n                    case ets:match(?MODULE, {{SearchGrp, PidName}, '$1'}) of\n                        [[FoundPid]] ->\n                            erlang:put(CachedName, FoundPid),\n                            FoundPid;\n                        [[_TPid] | _] = Pids ->\n                            %% fill process local cache\n                            ?DBG_ASSERT(SearchGrp =:= '_'),\n                            %% This should only happen for client\n                            %% processes not registered in a pid_group (failed)\n                            %% io:format(\"Ring is created to find a ~p in ~p ~p~n\",\n                            %% [PidName, self(), pid_groups:group_and_name_of(self())]),\n                            Ring = ring_new([PidX || [PidX] <- util:shuffle(Pids),\n                                                     erlang:is_process_alive(PidX),\n                                                     erlang:process_info(PidX, priority) =/= {priority, low}]),\n                            {Pid, NewPids} = ring_get(Ring),\n                            erlang:put(CachedName, NewPids),\n                            Pid;\n                        [] ->\n                            log:log(info, \"***No pid registered for ~p ~p~n\",\n                                    [PidName, util:get_stacktrace()]),\n                            failed\n                    end;\n                Pid ->\n                    erlang:put(CachedName, Pid),\n                    Pid\n            end;\n        Pid when is_pid(Pid) ->\n            %% clean process local cache if entry is outdated\n            case erlang:is_process_alive(Pid) andalso\n                     erlang:process_info(Pid, priority) =/= {priority, low} of\n                true ->\n                    Pid;\n                false -> erlang:erase(CachedName),\n                         cached_lookup(SearchGrp, PidName)\n            end;\n        Pids ->\n            %% clean process local cache if entry is outdated\n            {Pid, NewPids} = ring_get(Pids),\n            case erlang:is_pid(Pid)\n                andalso erlang:is_process_alive(Pid)\n                andalso\n                erlang:process_info(Pid, priority) =/= {priority, low} of\n                true ->\n                    erlang:put(CachedName, NewPids),\n                    Pid;\n                false -> erlang:erase(CachedName),\n                         cached_lookup(SearchGrp, PidName)\n            end\n    end.\n\n-spec ring_get({list(), list()}) -> {any(), {list(), list()}}.\nring_get({[], []})        -> {'$dead_code', {[], []}};\nring_get({[E], []} = Q)   -> {E, Q};\nring_get({[], L}) -> ring_get({lists:reverse(L), []});\nring_get({[ E | R ], L})  -> {E, {R, [E|L]}}.\n\n-spec ring_new(list()) -> {list(), list()}.\nring_new(L) -> {L, []}.\n\n-spec find_all(pidname()) -> [pid()].\nfind_all(PidName) ->\n    PidList = ets:match(?MODULE, {{'_', PidName}, '$1'}),\n    lists:append(PidList).\n\n-spec members(groupname()) -> [pid()].\nmembers(GrpName) ->\n    PidList = ets:match(?MODULE, {{GrpName, '_'}, '$1'}),\n    lists:append(PidList).\n\n-spec members_by_name(groupname()) -> [pidname()].\nmembers_by_name(GrpName) ->\n    MatchList = ets:match(?MODULE, {{GrpName, '$1'}, '_'}),\n    PidNameList = [ X || [X] <- MatchList ],\n    lists:sort(PidNameList).\n\n%% global information\n-spec processes() -> [pid()]. %% for fprof for example\nprocesses() ->\n    %% alternative implementation, when each pid is only in one group:\n    %% [X || [X]<- ets:match(?MODULE, {'_','$1'})].\n    lists:usort([element(2, X) || X <- tab2list()]).\n\n-spec groups() -> [groupname()].\ngroups() ->\n    lists:usort([element(1, element(1,X)) || X <- tab2list()]).\n\n-spec tab2list() -> [{{groupname(), pidname()}, pid()}].\ntab2list() -> ets:tab2list(?MODULE).\n\n%% @doc only supports the form {atom(), pos_integer()} | atom(). Not\n%%      deeper nested structures.\n-spec group_to_filename(groupname()) -> nonempty_string().\ngroup_to_filename('') ->\n    ?ASSERT(util:is_unittest()),\n    \"''\"; %% make tester happy by not delivering an empty string\ngroup_to_filename(GrpName) when is_atom(GrpName) ->\n    atom_to_list(GrpName);\ngroup_to_filename(GrpName) when is_integer(GrpName) ->\n    integer_to_list(GrpName);\ngroup_to_filename({Prefix, Postfix}) ->\n    group_to_filename(Prefix) ++ \"@\" ++ group_to_filename(Postfix).\n\n%% @doc only supports the form {atom(), pos_integer()} | atom(). Not\n%%      deeper nested structures.\n-spec filename_to_group(nonempty_string()) -> groupname().\n%% filename_to_group([]) -> {'',''};\nfilename_to_group(GrpName) ->\n    case string:sub_word(GrpName, 1, $@) of\n        GrpName ->\n            list_to_atom(GrpName);\n        Part1 ->\n            {list_to_atom(Part1), list_to_integer(string:sub_word(GrpName, 2, $@))}\n    end.\n\n%% @doc Resolve a local pid to its name.\n-spec pid_to_name(pid() | {groupname(), pidname()}) -> string().\npid_to_name(Pid) when is_pid(Pid) ->\n    case pid_groups:group_and_name_of(Pid) of\n        failed -> erlang:pid_to_list(Pid);\n        X      -> pid_to_name(X)\n    end;\npid_to_name({GrpName, PidName}) ->\n    lists:flatten(io_lib:format(\"~.0p:~.0p\", [GrpName, PidName])).\n\n%% @doc Resolve (local and remote) pids to names.\n-spec pids_to_names(Pids::[comm:mypid()], Timeout::pos_integer()) -> [string()].\npids_to_names(Pids, Timeout) ->\n    Refs = [begin\n                case comm:is_local(Pid) of\n                    true -> comm:make_local(Pid);\n                    _    -> comm:send(comm:get(pid_groups, Pid),\n                                      {group_and_name_of, Pid,\n                                       comm:reply_as(comm:this(), 2, {ok, '_', Pid})}),\n                            comm:send_local_after(Timeout, self(), {timeout, Pid})\n                end\n            end || Pid <- Pids],\n    [begin\n         case erlang:is_reference(Ref) of\n             false -> pid_to_name(Ref);\n             _     -> trace_mpath:thread_yield(),\n                      receive\n                          ?SCALARIS_RECV(\n                              {ok, {group_and_name_of_response, Name}, Pid}, %% ->\n                              begin\n                                  _ = erlang:cancel_timer(Ref),\n                                  receive {timeout, Pid} -> ok\n                                      after 0 -> ok\n                                  end,\n                                  pid_to_name(Name)\n                              end\n                            );\n                          {timeout, Pid} ->\n                              lists:flatten(\n                                io_lib:format(\"~.0p (timeout)\", [Pid]))\n                      end\n         end\n     end || Ref <- Refs].\n\n%% for the monitoring in the web interface\n%% @doc get info about a process\n-spec get_web_debug_info(string(), string()) ->\n   {struct, [{pairs, {array, [{struct, [{key | value, nonempty_string()}]}]}}]}.\nget_web_debug_info([], []) ->\n    {struct, [{pairs, {array, [ {struct, [{key, \"process\"}, {value, \"unknown\"}]} ]}}]};\nget_web_debug_info(GrpStr, PidNameString) ->\n    GrpName = string_to_group(GrpStr),\n    KVs = case pid_of_string(GrpName, PidNameString) of\n              failed -> [{\"process\", \"unknown\"}];\n              Pid    -> util:debug_info(Pid)\n          end,\n    JsonKVs = [ {struct, [{key, K}, {value, V}]} || {K, V} <- KVs],\n    {struct, [{pairs, {array, JsonKVs}}]}.\n\n%% for web interface\n-spec groups_as_json() ->\n   {array, [{struct, [{id | text, nonempty_string()} | {leaf, false}]}]}.\ngroups_as_json() ->\n    GroupList = groups(),\n    GroupListAsStructs =\n        [ {struct, [{id, group_to_string(El)}, {text, group_to_string(El)}, {leaf, false}]} || El <- GroupList ],\n    {array, GroupListAsStructs}.\n\n%% @doc find processes in a group (for web interface)\n-spec members_by_name_as_json(nonempty_string()) ->\n   {array, [{struct, [{id | text, nonempty_string()} | {leaf, true}]}]}.\nmembers_by_name_as_json(GrpString) ->\n    GrpName = string_to_group(GrpString),\n    PidList = members_by_name(GrpName),\n    PidListAsJson =\n        [ begin\n              ElStr = pidname_to_string(El),\n              {struct, [{id, GrpString ++ \".\" ++ ElStr},\n                        {text, ElStr}, {leaf, true}]}\n          end || El <- PidList ],\n    {array, PidListAsJson}.\n\n-spec group_to_string(groupname()) -> string().\ngroup_to_string(Group) when is_atom(Group) ->\n    atom_to_list(Group);\ngroup_to_string(Group) when is_integer(Group) ->\n    integer_to_list(Group);\ngroup_to_string(Group) when is_tuple(Group) ->\n    lists:flatten(io_lib:format(\"~p\", [Group])).\n\n-spec string_to_group(nonempty_string()) -> groupname() | failed.\nstring_to_group(GrpStr) ->\n    case [Name || Name <- groups(),\n                  group_to_string(Name) =:= GrpStr] of\n        [GrpName] -> GrpName;\n        [] -> failed\n    end.\n\n%% @doc hide a group of processes temporarily (for paused groups in\n%% unit tests)\n-spec hide(groupname()) -> ok.\nhide(GrpName) ->\n    comm:send_local(pid_groups_manager(),\n                    {pid_groups_hide, GrpName, self()}),\n    trace_mpath:thread_yield(),\n    receive ?SCALARIS_RECV({pid_groups_hide_done, GrpName}, ok) end.\n\n-spec unhide(groupname()) -> ok.\nunhide(GrpName) ->\n    comm:send_local(pid_groups_manager(),\n                    {pid_groups_unhide, GrpName, self()}),\n    trace_mpath:thread_yield(),\n    receive ?SCALARIS_RECV({pid_groups_unhide_done, GrpName}, ok) end.\n\n%%\n%% pid_groups manager process (gen_component)\n%%\n\n%% @doc Starts the server\n-spec start_link() -> {ok, pid()}.\nstart_link() ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                             [{erlang_register, ?MODULE},\n                              %% we provide a protected ets table,\n                              %% that should be available when we\n                              %% started..., so we better wait for its\n                              %% creation.\n                              {wait_for_init}]).\n\n%% @doc Initiates the server\n%% @private\n-spec init([]) -> null.\ninit(_Args) ->\n%%  ets:new(call_counter, [set, public, named_table]),\n%%  ets:insert(call_counter, {lookup_pointer, 0}),\n    _ = ets:new(?MODULE, [set, protected, named_table]),\n    _ = ets:new(pid_groups_hidden, [set, protected, named_table]),\n    %% required to gracefully eliminate dead, but registered processes\n    %% from the ets-table\n    process_flag(trap_exit, true),\n    _State = null.\n\n%% @doc Handling call messages\n%% @private\n-spec on(message(), null) -> null.\non({pid_groups_add, GrpName, PidName, Pid, ReplyTo}, State) ->\n    case group_and_name_of(Pid) of\n        failed ->\n            case ets:insert_new(?MODULE, {{GrpName, PidName}, Pid}) of\n                true -> ok;\n                false ->\n                    OldPid = ets:lookup_element(?MODULE, {GrpName, PidName}, 2),\n                    unlink(OldPid),\n                    ets:insert(?MODULE, {{GrpName, PidName}, Pid})\n            end,\n            link(Pid),\n            comm:send_local(ReplyTo, {pid_groups_add_done});\n        {GrpName, PidName} ->\n            comm:send_local(ReplyTo, {pid_groups_add_done});\n        {OldGroup, OldName} ->\n            comm:send_local(ReplyTo, {pid_groups_add_done, OldGroup, OldName})\n    end,\n    State;\n\non({drop_state}, State) ->\n    % only for unit tests\n    Links = ets:match(?MODULE, {'_', '$1'}),\n    _ = [unlink(Pid) || [Pid] <- Links],\n    ets:delete_all_objects(?MODULE),\n    State;\n\non({group_and_name_of, Pid, Source}, State) ->\n    % only for web debug interface\n    Name = pid_groups:group_and_name_of(comm:make_local(Pid)),\n    comm:send(Source, {group_and_name_of_response, Name}),\n    State;\n\non({pid_groups_hide, GrpName, Source}, State) ->\n    _ = [ begin\n              ets:delete_object(?MODULE, X),\n              ets:insert(pid_groups_hidden, X)\n          end || X <- tab2list(), GrpName =:= element(1, element(1, X)) ],\n    comm:send_local(Source, {pid_groups_hide_done, GrpName}),\n    State;\n\non({pid_groups_unhide, GrpName, Source}, State) ->\n    _ = [ begin\n              ets:delete_object(pid_groups_hidden, X),\n              ets:insert(?MODULE, X)\n          end || X <- ets:tab2list(pid_groups_hidden), GrpName =:= element(1, element(1, X)) ],\n    ets:delete(pid_groups_hidden, GrpName),\n    comm:send_local(Source, {pid_groups_unhide_done, GrpName}),\n    State;\n\non({'EXIT', FromPid, _Reason}, State) ->\n    Processes = ets:match(?MODULE, {'$1', FromPid}),\n    _ = [ets:delete(?MODULE, {InstanceId, Name}) || [{InstanceId, Name}] <- Processes],\n    State.\n\n\n%% Internal functions\n%% @doc Helper for the web debug interface to convert a pidname.\n-spec pidname_to_string(pidname()) -> nonempty_string().\npidname_to_string('') ->\n    ?ASSERT(util:is_unittest()),\n    \"''\"; %% make tester happy by not delivering an empty string\npidname_to_string(X) when is_atom(X) ->\n    atom_to_list(X);\npidname_to_string(Name) when is_tuple(Name) ->\n    lists:flatten(io_lib:format(\"~p\", [Name]));\npidname_to_string(Name) when is_list(Name) ->\n    Name.\n\n-spec pid_of_string(groupname() | failed, nonempty_string()) -> pid() | failed.\npid_of_string(failed, _) -> failed;\npid_of_string(GrpName, [${ | _] = PidNameStr) ->\n    case [Name || Name <- members_by_name(GrpName),\n                  pidname_to_string(Name) =:= PidNameStr] of\n        [PidName] -> pid_of(GrpName, PidName);\n        [] -> failed\n    end;\npid_of_string(GrpName, \"''\") ->\n    pid_of(GrpName, '');\npid_of_string(GrpName, PidNameStr) ->\n    PidName = try erlang:list_to_existing_atom(PidNameStr)\n              catch error:badarg -> PidNameStr\n              end,\n    case pid_of(GrpName, PidName) of\n        failed when is_atom(PidName) ->\n            % fallback for string pidnames with matching atoms somewhere\n            pid_of(GrpName, PidNameStr);\n        X -> X\n    end.\n\n%% @doc Returns the pid of the pid_groups manager process (a gen_component).\n-spec pid_groups_manager() -> pid() | failed.\npid_groups_manager() ->\n    case whereis(?MODULE) of\n        undefined ->\n            log:log(error, \"[ PG ] no pid_groups gen_component process found\"),\n            failed;\n        PID ->\n            %log:log(info, \"[ PG ] pid_groups gen_component process found\"),\n            PID\n    end.\n"
  },
  {
    "path": "src/prime.erl",
    "content": "% @copyright 2012-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @doc    prime number module\n%%          implemented method: trail division\n%% @end\n%% @reference 2009 - Melissa E. O'Neill\n%%          <em> The Genuine Sieve of Eratosthenes </em>\n%%          Journal of Functional Programming 19(1) pp: 95-106\n%%          Implemented Method: Trial Division\n%% @version $Id$\n-module(prime).\n-author('malange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-include(\"record_helpers.hrl\").\n-include(\"scalaris.hrl\").\n\n-export([get_nearest/1, is_prime/1]).\n\n% feeder for tester\n-export([is_prime_feeder/1, get_nearest_feeder/1]).\n\n-type prime() :: pos_integer().\n\n% last prime in prime_cache/0\n-define(PrimeCache, 5003).\n-define(TESTER_MAX_PRIME, 5250).\n\n-on_load(init/0).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec init() -> ok | {error, term()}.\ninit() ->\n    %% loads the shared library\n    SoName =\n        case code:priv_dir(scalaris) of\n            {error, bad_name} ->\n                Dir1 = filename:join(\n                         [filename:dirname(code:where_is_file(\"scalaris.beam\")),\n                          \"..\", priv]),\n                case filelib:is_dir(Dir1) of\n                    true ->\n                        filename:join(Dir1, ?MODULE);\n                    _ ->\n                        filename:join([\"..\", priv, ?MODULE])\n                end;\n            Dir ->\n                filename:join(Dir, ?MODULE)\n        end,\n    % tolerate some errors trying to load the library:\n    case erlang:load_nif(SoName, 0) of\n        ok -> ok;\n        {error, {load_failed, Text}} ->\n            log:log(warn,\n                    \"~s - falling back to the (considerably) slower Erlang native code\",\n                    [Text]),\n            ok;\n        Error -> Error\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_nearest_feeder(1..?TESTER_MAX_PRIME) -> {1..?TESTER_MAX_PRIME}.\nget_nearest_feeder(N) -> {N}.\n\n%% @doc Returns the first prime larger than or equal to N.\n-spec get_nearest(pos_integer()) -> prime().\nget_nearest(N) when N > ?PrimeCache ->\n    MaxN =\n        if N >= 396738 ->\n               % for x >= 396738 there is at least one prime between x and (1 + 1/(25ln^2(x)))x\n               % Dusart, Pierre (2010). \"Estimates of Some Functions Over Primes without R.H.\". arXiv:1002.0442\n               LogN = math:log(N),\n               util:ceil((1 + 1 / (25 * LogN*LogN)) * N);\n           N >= 25 ->\n             % x >= 25, there is always a prime between n and (1 + 1/5)n\n             % Nagura, J. \"On the interval containing at least one prime number.\" Proceedings of the Japan Academy, Series A 28 (1952), pp. 177-181.\n             N + (N + 4) div 5\n        end,\n    sieve_num(tl(prime_cache()), lists:seq(?PrimeCache + 2, MaxN, 2), N);\nget_nearest(N) when N =< ?PrimeCache ->\n    find_in_cache(N, prime_cache()).\n\n%% @doc Finds a number at least as large as N in the given prime list.\n%%      Such a number must exist!\n-spec find_in_cache(pos_integer(), [pos_integer()]) -> prime().\nfind_in_cache(N, [P | _Cache]) when P >= N ->\n    P;\nfind_in_cache(N, [_ | Cache]) ->\n    find_in_cache(N, Cache).\n\n%% @doc Sieves out all non-primes of the given list and returns the first\n%%      number bigger than Num. Throws if there is no such number.\n-spec sieve_num(Primes::[non_neg_integer()], Candidates::[non_neg_integer()],\n                Num::non_neg_integer()) -> Prime::non_neg_integer().\nsieve_num([H | _TL], _Candidates, Num) when H >= Num ->\n    H;\nsieve_num([H | TL], Candidates, Num) ->\n    sieve_num(TL, sieve_filter(Candidates, H*H, H), Num);\nsieve_num([], [H | _TL], Num) when H >= Num ->\n    H;\nsieve_num([], [H | TL], Num) ->\n    sieve_num([], sieve_filter(TL, H*H, H), Num).\n\n%% @doc Removes all factors of Inc from List, starting with Num.\n-spec sieve_filter(List::[non_neg_integer()], Num::non_neg_integer(),\n                   Inc::non_neg_integer()) -> [non_neg_integer()].\nsieve_filter([], _Num, _Inc) ->\n    [];\nsieve_filter([Num | TL], Num, Inc) ->\n    sieve_filter(TL, Num + Inc, Inc);\nsieve_filter([H | _TL] = All, Num, Inc) when H > Num ->\n    sieve_filter(All, Num + Inc, Inc);\nsieve_filter([H | TL], Num, Inc) ->\n    [H | sieve_filter(TL, Num, Inc)].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec is_prime_feeder(1..?TESTER_MAX_PRIME) -> {1..?TESTER_MAX_PRIME}.\nis_prime_feeder(N) -> {N}.\n\n-spec is_prime(pos_integer()) -> boolean().\nis_prime(V) when V =< ?PrimeCache ->\n    lists:member(V, prime_cache());\nis_prime(V) ->\n    get_nearest(V-1) =:= V.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc list of all primes less than 5003 from\n%      http://primes.utm.edu/lists/small/10000.txt\n-spec prime_cache() -> [pos_integer()].\nprime_cache() ->\n    [ 2,      3,      5,      7,     11,     13,     17,     19,     23,     29,\n     31,     37,     41,     43,     47,     53,     59,     61,     67,     71,\n     73,     79,     83,     89,     97,    101,    103,    107,    109,    113,\n    127,    131,    137,    139,    149,    151,    157,    163,    167,    173,\n    179,    181,    191,    193,    197,    199,    211,    223,    227,    229,\n    233,    239,    241,    251,    257,    263,    269,    271,    277,    281,\n    283,    293,    307,    311,    313,    317,    331,    337,    347,    349,\n    353,    359,    367,    373,    379,    383,    389,    397,    401,    409,\n    419,    421,    431,    433,    439,    443,    449,    457,    461,    463,\n    467,    479,    487,    491,    499,    503,    509,    521,    523,    541,\n    547,    557,    563,    569,    571,    577,    587,    593,    599,    601,\n    607,    613,    617,    619,    631,    641,    643,    647,    653,    659,\n    661,    673,    677,    683,    691,    701,    709,    719,    727,    733,\n    739,    743,    751,    757,    761,    769,    773,    787,    797,    809,\n    811,    821,    823,    827,    829,    839,    853,    857,    859,    863,\n    877,    881,    883,    887,    907,    911,    919,    929,    937,    941,\n    947,    953,    967,    971,    977,    983,    991,    997,   1009,   1013,\n   1019,   1021,   1031,   1033,   1039,   1049,   1051,   1061,   1063,   1069,\n   1087,   1091,   1093,   1097,   1103,   1109,   1117,   1123,   1129,   1151,\n   1153,   1163,   1171,   1181,   1187,   1193,   1201,   1213,   1217,   1223,\n   1229,   1231,   1237,   1249,   1259,   1277,   1279,   1283,   1289,   1291,\n   1297,   1301,   1303,   1307,   1319,   1321,   1327,   1361,   1367,   1373,\n   1381,   1399,   1409,   1423,   1427,   1429,   1433,   1439,   1447,   1451,\n   1453,   1459,   1471,   1481,   1483,   1487,   1489,   1493,   1499,   1511,\n   1523,   1531,   1543,   1549,   1553,   1559,   1567,   1571,   1579,   1583,\n   1597,   1601,   1607,   1609,   1613,   1619,   1621,   1627,   1637,   1657,\n   1663,   1667,   1669,   1693,   1697,   1699,   1709,   1721,   1723,   1733,\n   1741,   1747,   1753,   1759,   1777,   1783,   1787,   1789,   1801,   1811,\n   1823,   1831,   1847,   1861,   1867,   1871,   1873,   1877,   1879,   1889,\n   1901,   1907,   1913,   1931,   1933,   1949,   1951,   1973,   1979,   1987,\n   1993,   1997,   1999,   2003,   2011,   2017,   2027,   2029,   2039,   2053,\n   2063,   2069,   2081,   2083,   2087,   2089,   2099,   2111,   2113,   2129,\n   2131,   2137,   2141,   2143,   2153,   2161,   2179,   2203,   2207,   2213,\n   2221,   2237,   2239,   2243,   2251,   2267,   2269,   2273,   2281,   2287,\n   2293,   2297,   2309,   2311,   2333,   2339,   2341,   2347,   2351,   2357,\n   2371,   2377,   2381,   2383,   2389,   2393,   2399,   2411,   2417,   2423,\n   2437,   2441,   2447,   2459,   2467,   2473,   2477,   2503,   2521,   2531,\n   2539,   2543,   2549,   2551,   2557,   2579,   2591,   2593,   2609,   2617,\n   2621,   2633,   2647,   2657,   2659,   2663,   2671,   2677,   2683,   2687,\n   2689,   2693,   2699,   2707,   2711,   2713,   2719,   2729,   2731,   2741,\n   2749,   2753,   2767,   2777,   2789,   2791,   2797,   2801,   2803,   2819,\n   2833,   2837,   2843,   2851,   2857,   2861,   2879,   2887,   2897,   2903,\n   2909,   2917,   2927,   2939,   2953,   2957,   2963,   2969,   2971,   2999,\n   3001,   3011,   3019,   3023,   3037,   3041,   3049,   3061,   3067,   3079,\n   3083,   3089,   3109,   3119,   3121,   3137,   3163,   3167,   3169,   3181,\n   3187,   3191,   3203,   3209,   3217,   3221,   3229,   3251,   3253,   3257,\n   3259,   3271,   3299,   3301,   3307,   3313,   3319,   3323,   3329,   3331,\n   3343,   3347,   3359,   3361,   3371,   3373,   3389,   3391,   3407,   3413,\n   3433,   3449,   3457,   3461,   3463,   3467,   3469,   3491,   3499,   3511,\n   3517,   3527,   3529,   3533,   3539,   3541,   3547,   3557,   3559,   3571,\n   3581,   3583,   3593,   3607,   3613,   3617,   3623,   3631,   3637,   3643,\n   3659,   3671,   3673,   3677,   3691,   3697,   3701,   3709,   3719,   3727,\n   3733,   3739,   3761,   3767,   3769,   3779,   3793,   3797,   3803,   3821,\n   3823,   3833,   3847,   3851,   3853,   3863,   3877,   3881,   3889,   3907,\n   3911,   3917,   3919,   3923,   3929,   3931,   3943,   3947,   3967,   3989,\n   4001,   4003,   4007,   4013,   4019,   4021,   4027,   4049,   4051,   4057,\n   4073,   4079,   4091,   4093,   4099,   4111,   4127,   4129,   4133,   4139,\n   4153,   4157,   4159,   4177,   4201,   4211,   4217,   4219,   4229,   4231,\n   4241,   4243,   4253,   4259,   4261,   4271,   4273,   4283,   4289,   4297,\n   4327,   4337,   4339,   4349,   4357,   4363,   4373,   4391,   4397,   4409,\n   4421,   4423,   4441,   4447,   4451,   4457,   4463,   4481,   4483,   4493,\n   4507,   4513,   4517,   4519,   4523,   4547,   4549,   4561,   4567,   4583,\n   4591,   4597,   4603,   4621,   4637,   4639,   4643,   4649,   4651,   4657,\n   4663,   4673,   4679,   4691,   4703,   4721,   4723,   4729,   4733,   4751,\n   4759,   4783,   4787,   4789,   4793,   4799,   4801,   4813,   4817,   4831,\n   4861,   4871,   4877,   4889,   4903,   4909,   4919,   4931,   4933,   4937,\n   4943,   4951,   4957,   4967,   4969,   4973,   4987,   4993,   4999,   5003].\n"
  },
  {
    "path": "src/proto_sched.erl",
    "content": "% @copyright 2013-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Allow to centrally schedule all messages involved in a certain\n%% protocol execution and calculate the number of possible\n%% interleavings for convenience (so one can guess how often the\n%% protocol has to be run randomly to cover a good fraction of all\n%% possible interleavings).\n\n%% How it works: We use the same mechanism as trace_mpath and redirect\n%% all messages to the scheduler. The scheduler maintains a mailbox\n%% for each pair of (Src, Dest) to maintain FIFO ordering of channels,\n%% but allowing all other possible message interleavings (FIFO ordering can\n%% be optinally disabled). It delivers\n%% - steered by the given random seed - a single message for execution\n%% and receives a confirmation when the corresponding message handler\n%% is done. All messages generated by this message handler will arrive\n%% at the scheduler before the on_handler_done message, as Erlang\n%% provides FIFO on channels and all comm:sends are redirected\n%% to the central scheduler.\n\n%% How is the number of possible message interleavings calculated: At\n%% each step, we know how many messages we can choose from. So the\n%% number of different interleavings is the product of the\n%% possibilities in each step.\n\n%% How to detect the end of the protocol? When no more messages are\n%% queued, the protocol is finished. You can easily wait for this with\n%% the wait_for_end() function.\n\n%% Why this cannot be done with the breakpoints that gen_components\n%% provide? Breakpoints only have the execution of a message handler\n%% under its control. If a message handler generates new messages in\n%% the system, the VM directly enqueues them to the corresponding\n%% mailboxes of the receivers. As there is no shuffeling in the\n%% mailboxes of each receiver, there is only a limited amount of\n%% message interleaving simulated using breakpoints. In contrast, with\n%% proto_sched, messages generated by different message handlers can\n%% overhaul each other, as long as they do not correspond to the same\n%% communication channel, where FIFO ordering is maintained.\n\n%% Fast tests for timeouts: For msg_delay, we could add a modification\n%% so timeouts are just directly put in the pool of deliverable\n%% messages. So this would simulate a long lasting execution in a\n%% short timeframe (time compression). Actually msg_delay events\n%% somehow must be sorted according to the time they 'can' be\n%% delivered?  if req A is send and wants to be delivered after 10 sec\n%% and B is then send and wants to be delivered after 5 seconds, each\n%% could be delivered first?!\n\n%% @version $Id:$\n-module(proto_sched).\n-author('schintke@zib.de').\n-vsn('$Id:$').\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%\n%% Quick start:\n%% 1. call proto_sched:thread_num(N) %% seed has to be put somehow in\n%% 2. in each process/thread participating call\n%% 2.1. call proto_sched:thread_begin()\n%% 2.2. perform a synchronous request like api_tx:read(\"a\")\n%% 2.3. call proto_sched:thread_end()\n%% 3. call proto_sched:wait_for_end()\n%% 4. call proto_sched:get_infos() to retrieve some statistics like\n%%    the number of possible interleavings, the number of local or\n%%    globally send messages, etc.\n%% 5. call proto_sched:cleanup() to forget about the run and\n%%    delete statistics data.\n%%\n%% immediately before every receive statement using SCALARIS_RECV\n%% insert a trace_mpath:thread_yield() to pass the control flow back\n%% to the proto_sched.\n%%\n%% You can also provide a trace_id, so that the proto_sched can be used\n%% independently for several protocols at the same time (e.g. in\n%% concurrent unittests). See the interfaces and exported functions\n%% below.\n%%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%%-define(TRACE(X,Y), log:log(\"proto_sched: \" ++ X,Y)).\n-define(TRACE(X,Y), ok).\n-define(ORDERED_LINKS_DEFAULT, config:read(ordered_links)).\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n-behaviour(gen_component).\n\n%% client functions\n\n%% declare how many threads you will have (with optional trace_id):\n%% when thread_num threads called thread_begin(), proto_sched starts\n%% the scheduled execution\n-export([thread_num/1, thread_num/2, thread_num/3]).\n\n%% in each thread, give control to proto_sched (with optional trace_id)\n-export([thread_begin/0, thread_begin/1]).\n%% in each thread, declare its end (with optional trace_id)\n-export([thread_end/0, thread_end/1]).\n\n%% (1) before a receive, yield each thread to pass control to central\n%%     scheduler\n-export([thread_yield/0]).\n\n-export([get_infos/0, get_infos/1, info_shorten_messages/2]).\n-export([register_callback/2, register_callback/3]).\n-export([infected/0]).\n-export([clear_infection/0, restore_infection/0]).\n-export([wait_for_end/0, wait_for_end/1]).\n-export([cleanup/0, cleanup/1]).\n\n%% report messages from other modules\n-export([start/2]).\n-export([log_send/6, log_info/3]).\n-export([epidemic_reply_msg/4]).\n\n%% gen_component behaviour\n-export([start_link/1, init/1]).\n-export([on/2]). %% internal message handler as gen_component\n\n-export([check_config/0]).\n\n-type logger()       :: {proto_sched, comm:mypid()}.\n-type anypid()       :: pid() | comm:mypid().\n-type trace_id()     :: term().\n-type send_event()   :: {log_send, Time::'_', trace_id(),\n                         Source::anypid(), Dest::anypid(), comm:message(),\n                         local | global, comm:send_options()}.\n-type info_event()   :: {log_info, Time::'_', trace_id(),\n                         Source::anypid(), Info::comm:message()}.\n\n-type passed_state() :: {trace_id(), logger()}.\n-type gc_mpath_msg() :: {'$gen_component', trace_mpath, passed_state(),\n                         Src::anypid(), Dest::anypid(), comm:message()}.\n\n-export_type([logger/0]).\n-export_type([passed_state/0]).\n-export_type([callback_on_msg/0]).\n\n-type queue_key()        :: {Src :: anypid(), Dest :: anypid()}.\n-type delay_queue_key()  :: {Dest :: anypid()}.\n-type msg_queues()       :: [queue_key()].\n-type msg_delay_queues() :: [delay_queue_key()].\n\n-type callback_on_msg() ::\n        fun((Src::comm:mypid(), Dest::comm:mypid(), Msg::comm:message()) -> ok).\n\n-record(state,\n        {msg_queues              = ?required(state, msg_queues)\n         :: msg_queues(),\n         msg_delay_queues        = ?required(state, msg_delay_queues)\n         :: msg_delay_queues(),\n         status                  = ?required(state, status)\n         :: new | stopped | running\n          | {delivered, comm:mypid(), reference(), erlang_timestamp()},\n         ordered_links           = ?required(state, ordered_links)\n         :: boolean(),\n         to_be_cleaned           = ?required(state, to_be_cleaned)\n         :: false | {to_be_cleaned, pid()},\n         passed_state            = ?required(state, passed_state)\n         :: none | passed_state(),\n         num_possible_executions = ?required(state, num_possible_executions)\n         :: pos_integer(),\n         num_delivered_msgs      = ?required(state, num_delivered_msgs)\n         :: non_neg_integer(),\n         delivered_msgs          = ?required(state, delivered_msgs)\n         :: [{Source::anypid(), Dest::anypid(), comm:message(), local | global} |\n             {Source::anypid(), info, comm:message()}], %% delivered messages (and infos) in reverse order\n         nums_chosen_from        = ?required(state, nums_chosen_from)\n         :: [pos_integer()], %% #possibilities for each delivered msg in reverse order\n         callback_on_send        = ?required(state, callback_on_send)\n         :: callback_on_msg(),\n         callback_on_deliver     = ?required(state, callback_on_deliver)\n         :: callback_on_msg(),\n         thread_num              = ?required(state, thread_num)\n         :: non_neg_integer(),\n         threads_registered      = ?required(state, threads_registered)\n         :: non_neg_integer(),\n         inform_on_end           = ?required(state, inform_on_end)\n         :: pid() | none\n        }).\n\n-include(\"gen_component.hrl\").\n\n-type state_t() :: #state{}.\n-type state()   :: [{trace_id(), state_t()}].\n\n-spec thread_num(pos_integer()) -> ok.\nthread_num(N) -> thread_num(N, _TraceId=default).\n\n-spec thread_num(pos_integer(), trace_id()) -> ok.\nthread_num(N, TraceId) -> thread_num(N, TraceId, _OrderedLinks = ?ORDERED_LINKS_DEFAULT).\n\n-spec thread_num(pos_integer(), trace_id(), boolean()) -> ok.\nthread_num(N, TraceId, OrderedLinks) ->\n    send_steer_msg({thread_num, TraceId, OrderedLinks, N, comm:this()}),\n    receive\n        ?SCALARIS_RECV({thread_num_done}, ok);\n        ?SCALARIS_RECV({thread_num_failed},\n                       util:do_throw('proto_sched:thread_num_failed'))\n        end.\n\n-spec thread_begin() -> ok.\nthread_begin() -> thread_begin(default).\n\n-spec thread_begin(trace_id()) -> ok.\nthread_begin(TraceId) ->\n    ?ASSERT2(not infected(), duplicate_thread_begin),\n    %% We could send this as normal traced client message to\n    %% ourselves?!  But we better send in a special way to be able to\n    %% detect these thread_begin messages in a separate handler\n    %% clause as we want to detect when thread_num was set to small.\n    send_steer_msg({thread_begin, TraceId, comm:this()}),\n    %% proto_sched will then schedule itself a proper infected\n    %% message, that we then receive, which atomatically infects this\n    %% client thread\n    receive\n        ?SCALARIS_RECV({thread_begin_but_already_running},\n                       util:do_throw('proto_sched:thread_begin-but_already_running'));\n        ?SCALARIS_RECV(\n           {thread_release_to_run},\n           %% Yippie, we were chosen for execution, so we go on now up\n           %% to the next trace_mpath:thread_yield() (in front of a\n           %% receive) or proto_sched:thread_stop() that we pass.\n           ok)\n    end,\n    ?DBG_ASSERT2(infected(), not_infected_after_thread_begin),\n    ok.\n\n-spec thread_yield() -> ok.\nthread_yield() ->\n    ?ASSERT2(infected(), yield_outside_thread_start_thread_end),\n    trace_mpath:thread_yield().\n\n-spec thread_end() -> ok.\nthread_end() -> thread_end(default).\n\n-spec thread_end(trace_id()) -> ok.\nthread_end(TraceId) ->\n    ?ASSERT2(infected(), duplicate_or_uninfected_thread_end),\n    %% inform proto_sched that we are finished.\n    send_steer_msg({on_handler_done, TraceId, thread_end, comm:this()}),\n    %% switch off the infection\n    clear_infection(),\n    ?DBG_ASSERT2(not infected(), infected_after_thread_end),\n    ok.\n\n-spec wait_for_end() -> ok.\nwait_for_end() -> wait_for_end(default).\n\n-spec wait_for_end(trace_id()) -> ok.\nwait_for_end(TraceId) ->\n    ?ASSERT2(not infected(), wait_for_end_when_infected),\n    send_steer_msg({wait_for_end, TraceId, self()}),\n    receive\n        ?SCALARIS_RECV({proto_sched_done}, ok);\n        ?SCALARIS_RECV({wait_for_end_trace_not_found},\n           util:do_throw('proto_sched:wait_for_end-trace not found'))\n        end.\n\n-spec start(trace_id(), logger()) -> ok.\nstart(TraceId, Logger) ->\n    PState = passed_state_new(TraceId, Logger),\n    own_passed_state_put(PState).\n\n-spec register_callback(CallbackFun::callback_on_msg(), on_send | on_deliver)\n        -> ok | failed.\nregister_callback(CallbackFun, OnX) ->\n    register_callback(CallbackFun, OnX, default).\n\n-spec register_callback(CallbackFun::callback_on_msg(), on_send | on_deliver,\n                        trace_id()) -> ok | failed.\nregister_callback(CallbackFun, OnX, TraceId) ->\n    %% clear infection\n    clear_infection(),\n    %% register the callback function\n    LoggerPid = pid_groups:find_a(proto_sched),\n    comm:send_local(LoggerPid, {register_callback, CallbackFun, OnX, TraceId, comm:this()}),\n    X = receive\n            ?SCALARIS_RECV({register_callback_reply, Result}, Result)\n        end,\n    %% restore infection\n    restore_infection(),\n    X.\n\n-spec info_shorten_messages(Infos, CharsPerMsg::pos_integer()) -> Infos\n        when is_subtype(Infos, [tuple()]).\ninfo_shorten_messages([], _CharsPerMsg) ->\n    [];\ninfo_shorten_messages(Infos, CharsPerMsg) ->\n    {value, {delivered_msgs, DeliveredMsgs}, RestInfos} =\n        lists:keytake(delivered_msgs, 1, Infos),\n    DeliveredMsgs1 =\n        [begin\n             MsgStr = lists:flatten(io_lib:format(\"~111610.0p\", [Msg])),\n             element(1, util:safe_split(CharsPerMsg, MsgStr))\n         end || Msg <- DeliveredMsgs],\n    [{delivered_msgs, DeliveredMsgs1} | RestInfos].\n\n-spec get_infos() -> [tuple()].\nget_infos() -> get_infos(default).\n\n-spec get_infos(trace_id()) -> [tuple()].\nget_infos(TraceId) ->\n    case pid_groups:find_a(proto_sched) of\n        failed -> [];\n        LoggerPid ->\n            This = comm:this(),\n            case comm:is_valid(This) of\n                true ->\n                    clear_infection(),\n                    comm:send_local(LoggerPid, {get_infos, This, TraceId}),\n                    receive\n                        ?SCALARIS_RECV({get_infos_reply, Infos}, Infos)\n                    end,\n                    restore_infection(),\n                    Infos;\n                false ->\n                    log:log(\"Requesting proto_sched trace infos without a valid Pid (~p)\",\n                            [This]),\n                    []\n            end\n    end.\n\n-spec infected() -> boolean().\ninfected() ->\n    trace_mpath:infected().\n\n-spec clear_infection() -> ok.\nclear_infection() ->\n    trace_mpath:clear_infection().\n\n-spec restore_infection() -> ok.\nrestore_infection() ->\n    trace_mpath:restore_infection().\n\n-spec cleanup() -> ok.\ncleanup() -> cleanup(default).\n\n-spec cleanup(trace_id()) -> ok.\ncleanup(TraceId) ->\n     %% clear infection\n    ?ASSERT2(not infected(), 'proto_sched:cleanup_called_infected'),\n    ProtoSchedPid = pid_groups:find_a(?MODULE),\n    comm:send_local(ProtoSchedPid, {cleanup, TraceId, self()}),\n    receive {cleanup_done} -> ok;\n            {cleanup_trace_not_found} ->\n            erlang:throw('proto_sched:cleanup_trace_not_found')\n    end,\n    ok.\n\n%% Functions used to report tracing events from other modules\n-spec epidemic_reply_msg(passed_state(), anypid(), anypid(), comm:message()) ->\n                                gc_mpath_msg().\nepidemic_reply_msg(PState, FromPid, ToPid, Msg) ->\n    {'$gen_component', trace_mpath, PState, FromPid, ToPid, Msg}.\n\n-spec log_send(passed_state(), anypid(), anypid(), comm:message(),\n               local | global | local_after, comm:send_options()) -> ok.\nlog_send(PState, FromPid, ToPid, Msg, LocalOrGlobal, SendOptions) ->\n    case passed_state_logger(PState) of\n        {proto_sched, LoggerPid} ->\n            TraceId = passed_state_trace_id(PState),\n            send_log_msg(\n              PState,\n              LoggerPid,\n              {log_send, '_', TraceId, FromPid, ToPid, Msg, LocalOrGlobal, SendOptions})\n    end,\n    ok.\n\n-spec log_info(passed_state(), anypid(), comm:message()) -> ok.\nlog_info(PState, FromPid, Info) ->\n    case passed_state_logger(PState) of\n        {proto_sched, LoggerPid} ->\n            TraceId = passed_state_trace_id(PState),\n            send_log_msg(\n              PState,\n              LoggerPid,\n              {log_info, '_', TraceId, FromPid, Info})\n    end,\n    ok.\n\n-spec send_log_msg(passed_state(), comm:mypid(), send_event() | info_event() | comm:message()) -> ok.\nsend_log_msg(RestoreThis, LoggerPid, Msg) ->\n    %% don't log the sending of log messages ...\n    clear_infection(),\n    comm:send(LoggerPid, Msg),\n    own_passed_state_put(RestoreThis).\n\n\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(ServiceGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                             [{erlang_register, ?MODULE},\n                              {pid_groups_join_as, ServiceGroup, ?MODULE}]).\n\n-spec init(any()) -> state().\ninit(_Arg) ->\n    msg_delay:send_trigger(1, {check_slow_handler_trigger}),\n    [].\n\n-spec on(send_event() | info_event() | comm:message(), state()) -> state().\non({thread_begin, TraceId, Client}, State) ->\n    ?TRACE(\"proto_sched:on({thread_begin, ~p, ~p})\", [TraceId, Client]),\n    ?DBG_ASSERT2(not infected(), infected_in_on_handler),\n    T1 = get_or_create(TraceId, State),\n    case T1#state.status of\n        new ->\n            T2 = T1#state{\n                   threads_registered = 1 + T1#state.threads_registered},\n            %% trigger start_deliver when thread_num = threads_registered\n            T3 = start_deliver_when_ready(TraceId, T2),\n            NewState = lists:keystore(TraceId, 1, State, {TraceId, T3}),\n            gen_component:post_op({log_send, os:timestamp(),\n                                   TraceId, Client, Client,\n                                   {thread_release_to_run}, global, []}, NewState);\n        _ ->\n            log:log(\"Wrong proto_sched:thread_begin, found state is: ~.0p\",\n                    [T1]),\n            %% wrong call to proto_sched:thread_begin(),\n            %% send fail message to raise exception at caller code\n            %% position.\n            comm:send(Client, {thread_begin_but_already_running}),\n            State\n    end;\n\non({thread_num, TraceId, OrderedLinks, N, Client}, State) ->\n    ?TRACE(\"proto_sched:on({thread_num, ~p, ~p})\", [TraceId, N]),\n    ?DBG_ASSERT(not infected()),\n    T1 = get_or_create(TraceId, State),\n    case new =:= T1#state.status andalso 0 =:= T1#state.thread_num of\n        true ->\n            T2 = T1#state{thread_num = N, ordered_links = OrderedLinks},\n            %% trigger start_deliver when thread_num = threads_registered\n            T3 = start_deliver_when_ready(TraceId, T2),\n            comm:send(Client, {thread_num_done}),\n            lists:keystore(TraceId, 1, State, {TraceId, T3});\n        _ ->\n            log:log(\"Wrong proto_sched:thread_num, \"\n                    \"(duplicate call or already running) \"\n                    \"- found state is: ~.0p\",\n                    [T1]),\n            %% wrong call to proto_sched:thread_begin(),\n            %% send fail message to raise exception at\n            %% caller code position.\n            comm:send(Client, {thread_num_failed}),\n            State\n    end;\n\non({log_send, _Time, TraceId, From, To, UMsg, LorG, SendOptions}, State) ->\n    ?TRACE(\"proto_sched:on({log_send ... ~.0p (~.0p) -> ~.0p (~.0p): ~.0p})\",\n           [From,\n            pid_groups:group_and_name_of(From),\n            To,\n            pid_groups:group_and_name_of(To),\n            UMsg]),\n    FromGPid = comm:make_global(From),\n    ?DBG_ASSERT2(not infected(), infected_in_on_handler),\n    TmpEntry = case lists:keyfind(TraceId, 1, State) of\n                   false ->\n                       add_message(From, To, UMsg, LorG, SendOptions, new(TraceId));\n                   {TraceId, OldTrace} ->\n                       add_message(From, To, UMsg, LorG, SendOptions, OldTrace)\n               end,\n    %% call the callback function (if any)\n    CallbackFun = TmpEntry#state.callback_on_send,\n    ?TRACE(\"executing callback function ~p.\", [CallbackFun]),\n    CallbackFun(From, To, UMsg),\n    case TmpEntry#state.status of\n        new ->\n            %% still waiting for all threads to join\n            ?DBG_ASSERT2(UMsg =:= {thread_release_to_run}, wrong_starting_msg),\n            lists:keystore(TraceId, 1, State, {TraceId, TmpEntry});\n        {delivered, FromGPid, _Ref, _DeliverTime} ->\n            %% only From is allowed to enqueue messages\n            %% only when delivered or to_be_cleaned (during execution\n            %% of a scheduled piece of code) new arbitrary messages\n            %% can be added to the schedule\n            lists:keystore(TraceId, 1, State, {TraceId, TmpEntry})\n    end;\n\non({log_info, _Time, TraceId, From, Info}, State) ->\n    ?TRACE(\"proto_sched:on({log_info ... ~.0p (~.0p): ~.0p})\",\n           [From, pid_groups:group_and_name_of(From), Info]),\n    FromGPid = comm:make_global(From),\n    ?DBG_ASSERT2(not infected(), infected_in_on_handler),\n    TmpEntry = case lists:keyfind(TraceId, 1, State) of\n                   false -> new(TraceId);\n                   {TraceId, OldTrace} -> OldTrace\n               end,\n    NewEntry =\n        TmpEntry#state{delivered_msgs\n                           = [ {From, info, Info} | TmpEntry#state.delivered_msgs] },\n    case TmpEntry#state.status of\n        new ->\n            %% still waiting for all threads to join\n            lists:keystore(TraceId, 1, State, {TraceId, NewEntry});\n        {delivered, FromGPid, _Ref, _DeliverTime} ->\n            %% only From is allowed to enqueue infos\n            %% only when delivered or to_be_cleaned (during execution\n            %% of a scheduled piece of code) new arbitrary infos\n            %% can be added to the schedule\n            lists:keystore(TraceId, 1, State, {TraceId, NewEntry})\n    end;\n\non({start_deliver, TraceId}, State) ->\n    ?TRACE(\"proto_sched:on({start_deliver, ~p})\", [TraceId]),\n    ?DBG_ASSERT2(not infected(), infected_in_on_handler),\n    %% initiate delivery: if messages are already queued, deliver\n    %% first message, otherwise when first message arrives, start\n    %% delivery with that message.\n    case lists:keyfind(TraceId, 1, State) of\n        %% Entry is always there, as start_deliver is only called after\n        %% enough thread_num and thread_begin calls\n        {TraceId, OldTrace} ->\n            case new =/= OldTrace#state.status of\n                true ->\n                    log:log(\"Duplicate proto_sched:start_deliver() call\"\n                            \" probably not what you intend to do for\"\n                            \" reproducible results~n\"),\n                    case util:is_unittest() of\n                        true ->\n                            erlang:throw(proto_sched_duplicate_start_deliver);\n                        false ->\n                            ok\n                    end;\n                _ -> ok\n            end,\n            NewEntry = OldTrace#state{status = running},\n            NewState = lists:keystore(TraceId, 1, State, {TraceId, NewEntry}),\n            ?TRACE(\"proto_sched:on({start_deliver, ~p}) postop deliver\", [TraceId]),\n            gen_component:post_op({deliver, TraceId}, NewState)\n    end;\n\non({deliver, TraceId}, State) ->\n    ?TRACE(\"proto_sched:on({deliver, ~p})\", [TraceId]),\n    ?DBG_ASSERT2(not infected(), infected_in_on_handler),\n    case lists:keyfind(TraceId, 1, State) of\n        false ->\n            ?TRACE(\"proto_sched:on({deliver, ~p}) Nothing to deliver, unknown trace id!\", [TraceId]),\n            State;\n        {TraceId, TraceEntry} ->\n            case TraceEntry#state.status of\n                {delivered, _ToPid, _Ref, _Time} ->\n                    ?TRACE(\"There is already message delivered to ~.0p\",\n                           [_ToPid]),\n                    erlang:throw(proto_sched_already_in_delivered_mode);\n                _ -> ok\n            end,\n            case TraceEntry#state.msg_queues of\n                [] ->\n                    ?TRACE(\"Running out of messages, \"\n                           \"waiting for further ones to arrive on id '~p'.~n\"\n                           \"When protocol is finished, call proto_sched:stop(~p) and~n\"\n                           \"proto_sched:cleanup(~p)\",\n                           [TraceId, TraceId, TraceId]),\n                    ?TRACE(\"Seen ~p possible executions so far for id '~p'.\",\n                           [TraceEntry#state.num_possible_executions, TraceId]),\n                    case TraceEntry#state.inform_on_end of\n                        none -> ok;\n                        Client ->\n                            comm:send_local(Client, {proto_sched_done})\n                    end,\n                    NewEntry = TraceEntry#state{status = stopped},\n                    lists:keystore(TraceId, 1, State, {TraceId, NewEntry});\n                _ ->\n                    {From, To, LorG, Msg, SendOptions, NumPossible, TmpEntry} =\n                        pop_random_message(TraceEntry),\n                    ?TRACE(\"Chosen from ~p possible next messages.\", [NumPossible]),\n                    Monitor = case comm:is_local(comm:make_global(To)) of\n                                  true -> erlang:monitor(process,\n                                                         comm:make_local(comm:get_plain_pid(To)));\n                                  false -> none\n                              end,\n                    NewEntry =\n                        TmpEntry#state{num_possible_executions\n                                       = NumPossible * TmpEntry#state.num_possible_executions,\n                                      status = {delivered,\n                                                comm:make_global(To),\n                                                Monitor,\n                                                os:timestamp()},\n                                      num_delivered_msgs\n                                       = 1 + TmpEntry#state.num_delivered_msgs,\n                                      delivered_msgs\n                                       = [ {From, To, LorG, Msg}\n                                           | TmpEntry#state.delivered_msgs],\n                                      nums_chosen_from\n                                      = [ NumPossible\n                                          | TmpEntry#state.nums_chosen_from] },\n                    %% we want to get raised messages, so we have to infect this message\n                    PState = TraceEntry#state.passed_state,\n                    InfectedMsg = epidemic_reply_msg(PState, From, To, Msg),\n                    ?TRACE(\"delivering msg to execute:\"\n                           \" ~.0p (~.0p) -> ~.0p (~.0p): ~.0p.\",\n                           [From,\n                            pid_groups:group_and_name_of(comm:make_local(From)),\n                            To,\n                            pid_groups:group_and_name_of(comm:make_local(To)),\n                            Msg]),\n                    %% call the callback function (if any) before sending out the msg\n                    CallbackFun = TraceEntry#state.callback_on_deliver,\n                    ?TRACE(\"executing callback function ~p.\", [CallbackFun]),\n                    CallbackFun(From, To, Msg),\n                    %% Send infected message with a shepherd. In case of send errors,\n                    %% we will be informed by a {send_error, Pid, Msg, Reason} message.\n                    ShepherdPid = comm:reply_as(self(), 2, {send_error, '_', SendOptions}),\n                    comm:send(comm:make_global(To), InfectedMsg, [{shepherd, ShepherdPid}]),\n                    lists:keystore(TraceId, 1, State, {TraceId, NewEntry})\n            end\n    end;\n\non({on_handler_done, TraceId, _Tag, To}, State) ->\n     ?TRACE(\"proto_sched:on({on_handler_done, ~p}).\", [TraceId]),\n     %% do not use gen_component:post_op to allow a pending cleanup\n     %% call to interrupt us early.\n     case lists:keyfind(TraceId, 1, State) of\n         false ->\n             %% this is a bug\n             log:log(\"This is a bug\"),\n             State;\n         {TraceId, TraceEntry} ->\n             case TraceEntry#state.status of\n                 {delivered, To, Ref, _Time} ->\n                     %% this delivered was done, so we can schedule a new msg.\n                     erlang:demonitor(Ref, [flush]),\n\n                     %% enqueue a new deliver request for this TraceId\n                     ?TRACE(\"~p proto_sched:on({on_handler_done, ~p})\"\n                            \" trigger next deliver 1.\", [To, TraceId]),\n                     comm:send_local(self(), {deliver, TraceId}),\n                     %% set status to running\n                     NewEntry = TraceEntry#state{status = running},\n                     NewState = lists:keystore(TraceId, 1, State,\n                                               {TraceId, NewEntry}),\n                     case NewEntry#state.to_be_cleaned of\n                         {to_be_cleaned, CallerPid} ->\n                             ?TRACE(\"proto_sched:on({on_handler_done, ~p})\"\n                                    \" doing cleanup.\", [TraceId]),\n                             gen_component:post_op({do_cleanup,\n                                                    TraceId,\n                                                    CallerPid}, NewState);\n                         false -> NewState\n                     end;\n                  new ->\n                      %% proto_sched:end() immediately after proto_sched:start()?\n                      %% enqueue a new deliver request for this TraceId\n                      ?TRACE(\"proto_sched:on({on_handler_done, ~p})\"\n                             \"trigger next deliver 2 ~p.\", [TraceId, new]),\n                      comm:send_local(self(), {deliver, TraceId}),\n                      State\n             end\n     end;\n\non({send_error, {send_error, Pid, Msg, Reason}, SendOptions} = _ShepherdMsg, State) ->\n    %% call on_handler_done and continue with message delivery\n    TraceId = get_trace_id(get_passed_state(Msg)),\n    ?TRACE(\"send error for trace id ~p: ~p calling on_handler_done.\", [TraceId, _ShepherdMsg]),\n    case lists:keyfind(TraceId, 1, State) of\n        false -> State;\n        {TraceId, TraceEntry} ->\n            %% NOTE: if the monitor has hit before, its 'DOWN' message with\n            %%       reason 'noproc' is ignored (see below) and we can assume\n            %%       a delivered state\n            {delivered, Pid, _Ref, _Time} = TraceEntry#state.status,\n            %% enqueue a send_error message if a shepherd was used\n            State1 =\n                case lists:keyfind(shepherd, 1, SendOptions) of\n                    {shepherd, ShepherdPid} ->\n                        % work with envelopes\n                        {RealShepherdPid, RealMsg} =\n                            comm:unpack_cookie(\n                              ShepherdPid,\n                              {send_error, Pid, get_orig_msg(Msg), Reason}),\n                        TmpEntry =\n                            add_message(whereis(comm_server),\n                                        RealShepherdPid, RealMsg,\n                                        local, [], TraceEntry),\n                        lists:keystore(TraceId, 1, State, {TraceId, TmpEntry});\n                    false ->\n                        State\n                end,\n            %% process not alive -> generate on_handler_done\n            gen_component:post_op({on_handler_done, TraceId, send_error, Pid}, State1)\n    end;\n\non({register_callback, CallbackFun, OnX, TraceId, Client}, State) ->\n    ?TRACE(\"proto_sched:on({register_callback, ~p, ~p, ~p, ~p}).\",\n           [CallbackFun, OnX, TraceId, Client]),\n    ?DBG_ASSERT2(not infected(), infected_in_on_handler),\n    case lists:keyfind(TraceId, 1, State) of\n        false ->\n            comm:send(Client, {register_callback_reply, failed}),\n            State;\n        {TraceId, TraceEntry} ->\n            comm:send(Client, {register_callback_reply, ok}),\n            NewEntry =\n                case OnX of\n                    on_send ->\n                        TraceEntry#state{callback_on_send = CallbackFun};\n                    on_deliver ->\n                        TraceEntry#state{callback_on_deliver = CallbackFun}\n                end,\n            lists:keyreplace(TraceId, 1, State, {TraceId, NewEntry})\n    end;\n\non({get_infos, Client, TraceId}, State) ->\n    ?TRACE(\"proto_sched:on({get_infos, ~p, ~p}).\", [Client, TraceId]),\n    ?DBG_ASSERT2(not infected(), infected_in_on_handler),\n    case lists:keyfind(TraceId, 1, State) of\n        false ->\n            comm:send(Client, {get_infos_reply, []});\n        {TraceId, TraceEntry} ->\n            BranchingFactor =\n                case length(TraceEntry#state.nums_chosen_from) of\n                    0 -> 0;\n                    N -> lists:sum(TraceEntry#state.nums_chosen_from) / N\n                end,\n            Infos =\n                [{delivered_msgs,\n                  lists:reverse(TraceEntry#state.delivered_msgs)},\n                 {nums_chosen_from,\n                  lists:reverse(TraceEntry#state.nums_chosen_from)},\n                 {avg_branching_factor, BranchingFactor},\n                 {num_delivered_msgs,\n                  TraceEntry#state.num_delivered_msgs},\n                 {num_possible_executions,\n                  TraceEntry#state.num_possible_executions}],\n            comm:send(Client, {get_infos_reply, Infos})\n    end,\n    State;\n\non({wait_for_end, TraceId, CallerPid}, State) ->\n    ?TRACE(\"proto_sched:on({wait_for_end, ~p, ~p}).\", [TraceId, CallerPid]),\n    case lists:keyfind(TraceId, 1, State) of\n        false ->\n            comm:send_local(CallerPid, {wait_for_end_trace_not_found}),\n            State;\n        {TraceId, TraceEntry} ->\n            case TraceEntry#state.status of\n                stopped ->\n                    comm:send_local(CallerPid, {proto_sched_done}),\n                    State;\n                _ ->\n                    ?ASSERT2(none =:=  TraceEntry#state.inform_on_end,\n                             'proto_sched:wait_for_end_already_called'),\n                    NewEntry = TraceEntry#state{inform_on_end = CallerPid},\n                    lists:keyreplace(TraceId, 1, State, {TraceId, NewEntry})\n            end\n    end;\n\non({cleanup, TraceId, CallerPid}, State) ->\n    ?TRACE(\"proto_sched:on({cleanup, ~p, ~p}).\", [TraceId, CallerPid]),\n    ?DBG_ASSERT2(not infected(), infected_in_on_handler),\n    case lists:keyfind(TraceId, 1, State) of\n        false ->\n            comm:send_local(CallerPid, {cleanup_trace_not_found}),\n            State;\n        {TraceId, TraceEntry} ->\n            case TraceEntry#state.status of\n                {delivered, _To, _Ref, _Time} ->\n                    ?TRACE(\"proto_sched:on({cleanup, ~p, ~p}) set status to to_be_cleaned.\", [TraceId, CallerPid]),\n                    NewEntry = TraceEntry#state{\n                                 to_be_cleaned = {to_be_cleaned, CallerPid}},\n                    lists:keyreplace(TraceId, 1, State, {TraceId, NewEntry});\n                _ ->\n                    gen_component:post_op({do_cleanup, TraceId, CallerPid}, State)\n            end\n    end;\n\non({do_cleanup, TraceId, CallerPid}, State) ->\n    ?TRACE(\"proto_sched:on({do_cleanup, ~p, ~p}).\", [TraceId, CallerPid]),\n    ?DBG_ASSERT2(not infected(), infected_in_on_handler),\n    case lists:keytake(TraceId, 1, State) of\n        {value, {TraceId, TraceEntry}, TupleList2} ->\n            send_out_pending_messages(TraceEntry#state.msg_queues),\n            send_out_pending_messages(TraceEntry#state.msg_delay_queues),\n            comm:send_local(CallerPid, {cleanup_done}),\n            TupleList2;\n        false ->\n            comm:send_local(CallerPid, {cleanup_done}),\n            State\n    end;\n\non({'DOWN', Ref, process, _Pid, noproc = _Reason}, State) ->\n    ?TRACE(\"proto_sched:on({'DOWN', ~p, process, ~p, ~p}).\",\n           [Ref, _Pid, _Reason]),\n    %% the process did not exist when the monitor was opened and we should thus\n    %% also have a send_error in our message box which will clean up\n    %% search for trace with status delivered and Ref\n    true = lists:any(\n             % note: _Pid is always a real local pid, but _PidX is a comm:mypid()!\n             %       since Ref is unique though, we do not need to check the pid, too\n             fun({_TraceId, #state{status = {delivered, _PidX, RefX, _Time}}})\n                  when RefX =:= Ref ->\n                     true;\n                ({_TraceId, _}) ->\n                     false\n             end, State),\n    State;\n\non({'DOWN', Ref, process, Pid, _Reason}, State) ->\n    ?TRACE(\"proto_sched:on({'DOWN', ~p, process, ~p, ~p}).\",\n           [Ref, Pid, _Reason]),\n    %% NOTE: do not create a send_error message since the message may have been\n    %% received, otherwise, our shepherd should have kicked in (a send_error\n    %% will then be in our message queue)\n\n    %% search for trace with status delivered and Ref\n    [TraceEntry | _] =\n        lists:dropwhile(\n          % note: Pid is always a real local pid, but _PidX is a comm:mypid()!\n          %       since Ref is unique though, we do not need to check the pid, too\n          fun({_TraceId, #state{status = {delivered, _PidX, RefX, _Time}}})\n               when RefX =:= Ref ->\n                  false;\n             ({_TraceId, _}) ->\n                  true\n          end, State),\n    %% log:log(\"proto_sched:on({'DOWN', ~p, process, ~p, ~p}).\",\n    %%         [Ref, Pid, Reason]),\n    %% the process we delivered to has died, so we generate a\n    %% gc_on_done message ourselves.\n    %% use post_op to avoid concurrency with send_error\n    %% message when delivering to already dead nodes.\n    gen_component:post_op({on_handler_done,\n                           element(1, TraceEntry),\n                           pid_ended_died_or_killed,\n                           comm:make_global(Pid)}, State);\n\non({check_slow_handler_trigger}, State) ->\n    msg_delay:send_trigger(1, {check_slow_handler_trigger}),\n    gen_component:post_op({check_slow_handler_action}, State);\n\non({check_slow_handler_action}, State) ->\n    %% check for delivered messages that take longer than a second\n    %% and output diagnostic information on it.\n\n    %% trace id\n    %% executing process\n    %% message to process\n    %% amount of time the response is pending\n    %% current function of the process delivered to (when local)\n    [ case TState#state.status of\n          {delivered, _GPid, _Ref, StartTime} ->\n              Delta = timer:now_diff(os:timestamp(), StartTime) div 1000000,\n              case 1 =< Delta of\n                  true -> report_slow_handler(TId, TState);\n                  _ ->    ok\n              end;\n          _ -> ok\n      end\n      || {TId, TState} <- State ],\n    State.\n\n-spec report_slow_handler(trace_id(), state_t()) -> ok.\nreport_slow_handler(Tid, Entry) ->\n    {delivered, GPid, _Ref, StartTime} = Entry#state.status,\n    Delta = (timer:now_diff(os:timestamp(), StartTime) div 100000)/10,\n    {PidGrpName, StackTrace} =\n        case comm:is_local(GPid) of\n            true ->\n                LPid = comm:make_local(GPid),\n                {comm:make_local(LPid),\n                 try erlang:process_info(LPid,\n                                         current_stacktrace)\n                 catch error:badarg ->\n                         %% older erlang version\n                         %% -> fall back to current function\n                         catch(erlang:process_info(LPid,\n                                                   current_function))\n                 end};\n            _ ->\n                {not_local, no_stackstrace}\n        end,\n    log:log(\"proto_sched: Msg takes longer than ~p s to process:~n\"\n            \"TraceId: ~p~n\"\n            \"Process: ~p (~p)~n\"\n            \"DeliMsg: ~10000.0p~n\"\n            \"Msg No:  ~p~n\"\n            \"StackTr: ~p~n\",\n            [ Delta,\n              Tid,\n              GPid, PidGrpName,\n              hd(Entry#state.delivered_msgs),\n              Entry#state.num_delivered_msgs,\n              StackTrace\n            ]),\n    ok.\n\npassed_state_new(TraceId, Logger) ->\n    {TraceId, Logger}.\n\npassed_state_trace_id(State)      -> element(1, State).\npassed_state_logger(State)        -> element(2, State).\n\nown_passed_state_put(State)       -> erlang:put(trace_mpath, State), ok.\n%%own_passed_state_get()            -> erlang:get(trace_mpath).\n\n\nnew(TraceId) ->\n    LoggerPid = pid_groups:find_a(?MODULE),\n    Logger = comm:make_global(LoggerPid),\n    #state{ msg_queues = [],\n            msg_delay_queues = [],\n            status = new,\n            ordered_links = true,\n            to_be_cleaned = false,\n            passed_state = passed_state_new(TraceId, {proto_sched, Logger}),\n            num_possible_executions = 1,\n            num_delivered_msgs = 0,\n            delivered_msgs = [],\n            nums_chosen_from = [],\n            callback_on_send = fun(_From, _To, _Msg) -> ok end,\n            callback_on_deliver = fun(_From, _To, _Msg) -> ok end,\n            thread_num = 0,\n            threads_registered = 0,\n            inform_on_end = none\n          }.\n\n%% @doc Sends out all messages remaining in queues\nsend_out_pending_messages(Queues) ->\n    lists:foreach(fun(Key) ->\n                          {_Src, Dest} = Key,\n                          Queue = queue:to_list(erlang:erase(Key)),\n                          To = comm:make_global(Dest),\n                          _ = [comm:send(To, Msg) || {_LorG, Msg} <- Queue]\n                  end,\n                  Queues).\n\n-spec add_message(anypid(), anypid(), comm:message(), local | global,\n                  comm:send_options(), state_t()) -> state_t().\nadd_message(Src, Dest, Msg, LorG, SendOptions, #state{msg_queues = OldQueues} = State) ->\n    Key = {Src, Dest},\n    NewQueues = add_to_list_of_queues(Key, {LorG, Msg, SendOptions}, OldQueues),\n    State#state{msg_queues = NewQueues}.\n\n%% -spec add_delay_message(comm:mypid(), comm:message(), state_t()) -> state_t().\n%% add_delay_message(Dest, Msg, #state{msg_delay_queues = OldQueues} =\n%% State) ->\n%%     Key = {Dest},\n%%     NewQueues = add_to_list_of_queues(Key, Msg, OldQueues),\n%%     State#state{msg_delay_queues = NewQueues}.\n\n-spec pop_random_message(state_t())\n        -> {Src::comm:mypid(), Dest::comm:mypid(), local | global,\n            Msg::comm:message(), comm:send_options(),\n            Possibilities::pos_integer(), state_t()}.\npop_random_message(#state{msg_queues = OldQueues, ordered_links = false} = State) ->\n    {{Src, Dest} = Key, Len} = util:randomelem_and_length(OldQueues),\n    Q = erlang:get(Key),\n    QLen = queue:len(Q),\n    MessagePos = randoms:rand_uniform(0, QLen),\n    {FrontQ, BackQ} = queue:split(MessagePos, Q),\n    {{value, {LorG, M, SendOptions}}, BackQ2} = queue:out(BackQ),\n    Q2 = queue:join(FrontQ, BackQ2),\n    NewQueues = update_queue_in_list_of_queues(Key, Q2, OldQueues),\n    State2 = State#state{msg_queues = NewQueues},\n    {Src, Dest, LorG, M, SendOptions, QLen*Len, State2};\npop_random_message(#state{msg_queues = OldQueues, ordered_links = true} = State) ->\n    {{Src, Dest} = Key, Len} = util:randomelem_and_length(OldQueues),\n    Q = erlang:get(Key),\n    {{value, {LorG, M, SendOptions}}, Q2} = queue:out(Q),\n    NewQueues = update_queue_in_list_of_queues(Key, Q2, OldQueues),\n    State2 = State#state{msg_queues = NewQueues},\n    {Src, Dest, LorG, M, SendOptions, Len, State2}.\n\n%% -spec pop_random_delay_message(state_t()) -> {comm:mypid(), comm:message(), state_t()}.\n%% pop_random_delay_message(#state{msg_delay_queues = OldQueues} = State) ->\n%%     {{Dest} = Key, Q} = util:randomelem(OldQueues),\n%%     {{value, M}, Q2} = queue:out(Q),\n%%     NewQueues = update_queue_in_list_of_queues(Key, Q2, OldQueues),\n%%     State2 = State#state{msg_delay_queues = NewQueues},\n%%     {Dest, M, State2}.\n\n-spec add_to_list_of_queues\n        (queue_key(), {local | global, comm:message(), comm:send_options()}, msg_queues()) -> msg_queues().%;\n        %(delay_queue_key(), comm:message(), msg_delay_queues()) -> msg_delay_queues().\nadd_to_list_of_queues(Key, M, Queues) ->\n    case erlang:get(Key) of\n        undefined ->\n            _ = erlang:put(Key, queue:from_list([M])),\n            [Key | Queues];\n        Queue ->\n            _ = erlang:put(Key, queue:in(M, Queue)),\n            Queues\n    end.\n\n-spec update_queue_in_list_of_queues\n        (queue_key(), queue:queue({local | global, comm:message()}), msg_queues()) -> msg_queues().%;\n        %(delay_queue_key(), queue:queue(comm:message()), msg_delay_queues()) -> msg_delay_queues().\nupdate_queue_in_list_of_queues(Key, Q, Queues) ->\n    case queue:is_empty(Q) of\n        true ->\n            erlang:erase(Key),\n            lists:delete(Key, Queues);\n        false ->\n            _ = erlang:put(Key, Q),\n            Queues\n    end.\n\n-spec get_passed_state(gc_mpath_msg()) -> passed_state().\nget_passed_state(Msg) ->\n    element(3, Msg).\n\n-spec get_orig_msg(gc_mpath_msg()) -> comm:message().\nget_orig_msg(Msg) ->\n    element(6, Msg).\n\n-spec get_trace_id(passed_state()) -> trace_id().\nget_trace_id(State) ->\n    element(1, State).\n\nsend_steer_msg(Msg) ->\n    LoggerPid = pid_groups:find_a(?MODULE),\n    Logger = comm:make_global(LoggerPid),\n    %% send not as an infected message, but directly to the logger process\n    send_log_msg(erlang:get(trace_mpath), Logger, Msg).\n\n-spec get_or_create(trace_id(), state()) -> state_t().\nget_or_create(TraceId, State) ->\n    case lists:keyfind(TraceId, 1, State) of\n        false ->            new(TraceId);\n        {TraceId, Entry} -> Entry\n    end.\n\n-spec start_deliver_when_ready(trace_id(), state_t()) -> state_t().\nstart_deliver_when_ready(TraceId, Entry) ->\n    case Entry#state.thread_num =:= Entry#state.threads_registered of\n        true ->\n            comm:send_local(self(), {start_deliver, TraceId}),\n            Entry;\n        false ->\n            Entry\n    end.\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_bool(ordered_links).\n\n"
  },
  {
    "path": "src/quorum.erl",
    "content": "% @copyright 2010-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Service functions for handling quorums.\n%% @end\n%% @version $Id$\n-module(quorum).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-export([majority_for_accept/1,\n         majority_for_deny/1,\n         minority/1]).\n\n%% @doc Gives the majority required for a quorum to accept.\n-spec majority_for_accept(integer()) -> integer().\nmajority_for_accept(SetSize) -> SetSize div 2 + 1.\n\n%% @doc Gives the majority required for a quorum to deny.\n-spec majority_for_deny(integer()) -> integer().\nmajority_for_deny(SetSize) -> SetSize div 2 + SetSize rem 2.\n\n%% @doc Gives the minority.\n-spec minority(integer()) -> integer().\nminority(SetSize) -> SetSize - majority_for_accept(SetSize).\n"
  },
  {
    "path": "src/randoms.erl",
    "content": "%  @copyright 2007-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Helper functions to create random numbers.\n%% @end\n%% @version $Id$\n-module(randoms).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n-export([start/0, stop/0, getRandomString/0, getRandomInt/0,\n         rand_uniform/2, rand_uniform/3, uniform/0, uniform/1]).\n\n%% for tester\n-export([rand_uniform_feeder/2, rand_uniform_feeder/3]).\n\n%% @doc Starts the crypto module's server.\n-spec start() -> ok.\nstart() -> crypto:start().\n\n%% @doc Generates a random string in the range 1 =&lt; Id &lt; 2^32\n-spec getRandomString() -> string().\ngetRandomString() ->\n    integer_to_list(getRandomInt()).\n\n%% @doc Generates a random integer in the range 1 =&lt; Id &lt; 2^32\n-spec getRandomInt() -> pos_integer().\n-ifdef(with_crypto_bytes_to_integer).\ngetRandomInt() ->\n    util:min(1 + crypto:bytes_to_integer(crypto:strong_rand_bytes(4)), 16#100000000).\n-else.\ngetRandomInt() ->\n    rand_uniform(1, 16#100000000).\n-endif.\n\n-spec rand_uniform_feeder(integer(), integer()) -> {Lo::integer(), Hi::integer()}.\nrand_uniform_feeder(X, Y) when X > Y -> {Y, X};\nrand_uniform_feeder(X, Y) when X < Y -> {X, Y};\nrand_uniform_feeder(X, X) -> {X, X + 1}.\n\n%% @doc Generates a random number N with Lo &lt;= N &lt; Hi using the crypto\n%%      library pseudo-random number generator.\n-spec rand_uniform(Lo::integer(), Hi::integer()) -> integer().\nrand_uniform(Lo, Hi) ->\n    crypto_rand_uniform(Lo, Hi).\n\n-spec rand_uniform_feeder(integer(), integer(), Count::1..1000)\n        -> {Lo::integer(), Hi::integer(), Count::non_neg_integer()}.\nrand_uniform_feeder(X, Y, C) when X > Y -> {Y, X, C};\nrand_uniform_feeder(X, Y, C) when X < Y -> {X, Y, C};\nrand_uniform_feeder(X, X, C) -> {X, X + 1, C}.\n\n%% @doc Generates Count random numbers N with Lo &lt;= N &lt; Hi using the crypto\n%%      library pseudo-random number generator.\n-spec rand_uniform(Lo::integer(), Hi::integer(), Count::non_neg_integer()) -> [integer()].\nrand_uniform(Lo, Hi, Count) when Lo < Hi ->\n    util:for_to_ex(1, Count, fun(_) -> crypto_rand_uniform(Lo, Hi) end).\n\n%% @doc Stops the crypto module's server.\n-spec stop() -> ok.\nstop() -> crypto:stop().\n\n-spec uniform() -> float().\n-ifdef(with_rand).\nuniform() ->\n    rand:uniform().\n-else.\nuniform() ->\n    random:uniform().\n-endif.\n\n-spec uniform(X::pos_integer()) -> pos_integer().\n-ifdef(with_rand).\nuniform(X) ->\n    rand:uniform(X).\n-else.\nuniform(X) ->\n    random:uniform(X).\n-endif.\n\n-ifdef(have_crypto_randuniform_support).\ncrypto_rand_uniform(Lo, Hi) ->\n    crypto:rand_uniform(Lo, Hi).\n-else.\ncrypto_rand_uniform(Lo, Hi) ->\n    Range = Hi - Lo,\n    Result = uniform(Range) + Lo - 1,\n    %% ct:pal(\"~w-~w:~w -> ~w~n\", [Hi, Lo, Range, Result]),\n    Result.\n-endif.\n"
  },
  {
    "path": "src/rbr/inc_on_cseq.erl",
    "content": "% @copyright 2014-2017 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    Provides a minimalistic interface to read or increment an integer\n%%         value based on rbrcseq.\n%% @end\n%% @version $Id$\n-module(inc_on_cseq).\n-author('skrzypczak@zib.de').\n-vsn('$Id:$ ').\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([read/1]).\n-export([inc/1]).\n\n-export([rf_val/1]).\n-export([rf_none/1]).\n-export([cc_noop/3]).\n-export([wf_inc/3]).\n\n-spec read(client_key()) -> {ok, client_value()} | {fail, not_found}.\nread(Key) ->\n    rbrcseq:qread(kv_db, self(), ?RT:hash_key(Key), ?MODULE, fun ?MODULE:rf_val/1),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({qread_done, _ReqId, _NextFastWriteRound, _OldWriteRound, Value},\n                       case Value of\n                           no_value_yet -> {fail, not_found};\n                           _ -> {ok, Value}\n                           end\n                      )\n    after 1000 ->\n        log:log(\"read hangs ~p~n\", [erlang:process_info(self(), messages)]),\n        receive\n            ?SCALARIS_RECV({qread_done, _ReqId, _NextFastWriteRound, _OldWriteRound, Value},\n                            case Value of\n                                no_value_yet -> {fail, not_found};\n                                _ -> {ok, Value}\n                            end\n                          )\n       end\n end.\n\n%% increments value stored at Key. For simplicity, does not fail if current\n%% value is not an integer. Instead, the value will be replaced by 1.\n-spec inc(client_key()) -> {ok}.\ninc(Key) ->\n    rbrcseq:qwrite(kv_db, self(), ?RT:hash_key(Key), ?MODULE,\n                   fun ?MODULE:rf_none/1, fun ?MODULE:cc_noop/3, fun ?MODULE:wf_inc/3, 1),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({qwrite_done, _ReqId, _NextFastWriteRound, _Value, _WriteRet}, {ok});\n        ?SCALARIS_RECV({qwrite_deny, _ReqId, _NextFastWriteRound, _Value, Reason},\n                       begin log:log(\"Write failed on key ~p: ~p~n\", [Key, Reason]),\n                       {ok} end)\n    after 1000 ->\n            log:log(\"~p write hangs at key ~p, ~p~n\",\n                    [self(), Key, erlang:process_info(self(), messages)]),\n            receive\n                ?SCALARIS_RECV({qwrite_done, _ReqId, _NextFastWriteRound, _Value, _WriteRet},\n                               begin\n                                   log:log(\"~p write was only slow at key ~p~n\",\n                                           [self(), Key]),\n                                   {ok}\n                               end); %%;\n                ?SCALARIS_RECV({qwrite_deny, _ReqId, _NextFastWriteRound, _Value, Reason},\n                               begin log:log(\"~p Write failed: ~p~n\",\n                                             [self(), Reason]),\n                                     {ok} end)\n                end\n    end.\n\n-spec rf_val(prbr_bottom | non_neg_integer()) -> non_neg_integer().\nrf_val(prbr_bottom) -> 0;\nrf_val(X) -> X.\n\n-spec rf_none(any()) -> none.\nrf_none(_) -> none.\n\n-spec cc_noop(any(), any(), any()) -> {true, none}.\ncc_noop(_,_,_) -> {true, none}.\n\n-spec wf_inc(prbr_bottom | non_neg_integer(), none, non_neg_integer()) -> {non_neg_integer(), none}.\nwf_inc(prbr_bottom, none, ToAdd) -> {ToAdd, none};\nwf_inc(Val, none, ToAdd) when not is_integer(Val) -> {ToAdd, none};\nwf_inc(Val, none, ToAdd) ->  {Val + ToAdd, none}.\n"
  },
  {
    "path": "src/rbr/json_on_cseq.erl",
    "content": "% @copyright 2012-2017 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak  <skrzypczak@zib.de>\n%% @doc    Replicated storage of JSON objects based on rbrcseq. Implements\n%%         a superset of RFC 6902 (JSON Patch) which allows  partial updates on\n%%         the stored objects. This is based on dotto. See scalaris/contrib or:\n%%              https://github.com/rumoe/dotto\n%%\n%%         In its current state, erlang dicts are used to represent JSON objects.\n%% @end\n%% @version $Id$\n-module(json_on_cseq).\n-author('skrzypczak@zib.de').\n-vsn('$Id:$ ').\n\n-define(NON_EXIST_VAL, no_value_yet).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export_type([json/0]).\n-export_type([path/0]).\n-export_type([patch/0]).\n-export_type([patch_cmd/0]).\n\n%% PUBLIC API\n-export([new_json/0]).\n\n-export([read/1]).\n-export([fetch/2]).\n-export([write/2]).\n\n-export([patch/2]).\n-export([add/3]).\n-export([remove/2]).\n-export([replace/3]).\n-export([move/3]).\n-export([copy/3]).\n-export([test/3]).\n\n%% READ FILTER\n-export([rf_empty/1]).\n-export([rf_val/1]).\n\n%% CONTENT CHECK\n-export([cc_noop/3]).\n\n%% WRITE FILTER\n-export([wf_val/3]).\n-export([wf_patch/3]).\n\n%% TESTER\n-export([create_rand_json/1]).\n-export([is_json/1]).\n\n-ifdef(namespaced_dict).\n-type json() :: dict:dict().\n-else.\n-type json() :: dict().\n-endif.\n\n%% A JSON patch is list of sequentially executed patch commands.\n-type patch() :: [patch_cmd()].\n\n%% Valid patch commands as per RFC 6902.\n%% Consult https://tools.ietf.org/html/rfc6902 for the effects of the respective\n%% commands on a given JSON object.\n-type patch_cmd() :: {Op :: remove, Path :: path()} |\n                     {Op :: add | replace | test, Path :: path(), Value :: any()} |\n                     {Op :: move | copy, From :: path(), Path :: path()}.\n\n%% JSON Path similar to RFC 6901 (https://tools.ietf.org/html/rfc6901).\n%% Howerver, instead of strings (\"/this/is/a/path\"), lists are used\n%% ([this, is, a, path]).\n-type path() :: [any()].\n\n%% @doc Creates a new, empty JSON object.\n-spec new_json() -> json().\nnew_json() -> dict:new().\n\n%% @doc Read full JSON object stored at given key.\n%% Returns\n%%      {ok, JsonObject}    - JSON Object stored at given key.\n%%      {fail, not_fount}   - No value at this key exists.\n-spec read(client_key()) -> {ok, client_value()} | {fail, not_found}.\nread(Key) ->\n    read_helper(Key, fun ?MODULE:rf_val/1).\n\n%% @doc Partially read a JSON object stored at a given key.\n-spec fetch(client_key(), path()) ->\n    {ok, client_value()} | {fail, not_found} | {error, any()}.\nfetch(Key, Path) ->\n    read_helper(Key, get_rf_fetch_fun(Path)).\n\n%% @doc Write full JSON object to a given key.\n%% Returns:\n%%      ok                  - JSON Object was successfully written\n%%      {fail, Reason}      - The write was denied by scalaris.\n-spec write(client_key(), json()) -> ok | {fail, any()}.\nwrite(Key, Value) ->\n    write_helper(Key, Value, fun ?MODULE:wf_val/3).\n\n%% %%%%%%%%%%%%%%%%\n%% JSON PATCH SPECIFIC API\n%% %%%%%%%%%%%%%%%%\n\n%% @doc Apply a JSON patch to the JSON object at a given key. A patch consists\n%% of a list of patch commands, which are applied sequentially to the JSON\n%% object. If all operations are performed successfully, then the resulting\n%% JSON object replaces the currently stored value. If *any* command fails,\n%% the stored object will not be modified.\n%% Returns:\n%%      ok                  - The patch was successfully applied\n%%      {error, ErrorList}  - At least one patch command failed.\n%%      {fail, Reason}      - The write was denied by scalaris.\n-spec patch(client_key(), patch_cmd() | patch()) ->\n    ok | {fail, any()} | {error, [any()]}.\npatch(Key, PatchCommand) when not is_list(PatchCommand) ->\n    patch(Key, [PatchCommand]);\npatch(Key, Patch) ->\n    write_helper(Key, Patch, fun ?MODULE:wf_patch/3).\n\n-spec add(client_key(), path(), client_value()) ->\n    ok | {fail | error, any()}.\nadd(Key, Path, Value) ->\n    unpack_if_error_list(patch(Key, {add, Path, Value})).\n\n-spec remove(client_key(), path()) ->\n    ok | {fail | error, any()}.\nremove(Key, Path) ->\n    unpack_if_error_list(patch(Key, {remove, Path})).\n\n-spec replace(client_key(), path(), client_value()) ->\n    ok | {fail | error, any()}.\nreplace(Key, Path, Value) ->\n    unpack_if_error_list(patch(Key, {replace, Path, Value})).\n\n-spec move(client_key(), path(), path()) ->\n    ok | {fail | error, any()}.\nmove(Key, From, Path) ->\n    unpack_if_error_list(patch(Key, {move, From, Path})).\n\n-spec copy(client_key(), path(), path()) ->\n    ok | {fail | error, any()}.\ncopy(Key, From, Path) ->\n    unpack_if_error_list(patch(Key, {copy, From, Path})).\n\n-spec test(client_key(), path(), client_value()) ->\n    {ok, boolean()} | {fail | error, any()}.\ntest(Key, Path, Value) ->\n    read_helper(Key, get_rf_test_fun(Path, Value)).\n\n%% %%%%%%%%%%%%%%%%\n%% READ FILTER\n%% %%%%%%%%%%%%%%%%\n%% @doc RedFilter that always returns an empty value\n-spec rf_empty(any()) -> none.\nrf_empty(_Any) -> none.\n\n%% @doc ReadFilter returning the full JSON object\n-spec rf_val(json() | prbr_bottom) -> client_value() | ?NON_EXIST_VAL.\nrf_val(prbr_bottom) -> ?NON_EXIST_VAL;\nrf_val(X)           -> X.\n\n%% @doc Returns a ReadFilter to partially read a JSON object.\n-spec get_rf_fetch_fun(path()) -> prbr:read_filter().\nget_rf_fetch_fun(Path) ->\n    fun\n        (prbr_bottom) -> ?NON_EXIST_VAL;\n        (Obj)         ->\n            case dotto:fetch(Obj, Path) of\n                {ok, Result} -> Result;\n                Any -> Any\n            end\n    end.\n\n%% @doc Returns a ReadFilter which tests the existens of a value\n%% in a stored JSON object.\n-spec get_rf_test_fun(path(), client_value()) -> prbr:read_filter().\nget_rf_test_fun(Path, Value) ->\n    fun\n        (prbr_bottom) -> ?NON_EXIST_VAL;\n        (Obj)         ->\n            case dotto:test(Obj, Path, Value) of\n                {ok, Result} -> Result;\n                Any -> Any\n            end\n    end.\n\n%% %%%%%%%%%%%%%%%%\n%% CONTENT CHECK\n%% %%%%%%%%%%%%%%%%\n%% @doc Empty ContentCheck that always passes. And gives no update information\n%% (none) to the WriteFilter\n-spec cc_noop(any(), prbr:write_filter(), any()) -> {true, none}.\ncc_noop(_ReadVal, _WriteFilter, _ValToWrite) -> {true, none}.\n\n%% %%%%%%%%%%%%%%%%\n%% WRITE FILTER\n%% %%%%%%%%%%%%%%%%\n%% @doc WriteFilter that replaces the current value with the client provided one.\n-spec wf_val(json() | prbr_bottom, any(), json()) -> {json(), ok}.\nwf_val(_OldVal, _UpdateInfo, ValToWrite) -> {ValToWrite, ok}.\n\n%% @doc WriteFilter that applies a list of patch commands to a JSON object.\n%% A non-existing JSON object is treated as an empty object (dict:new()).\n-spec wf_patch(json() | prbr_bottom, any(), patch()) ->  {json(), ok | {error, [any()]}}.\nwf_patch(prbr_bottom, UI, Patch) -> wf_patch(dict:new(), UI, Patch);\nwf_patch(Json, _UI, Patch) ->\n    case dotto:apply(Patch, Json) of\n        {ok, ResultJson} -> {ResultJson, ok};\n        {error, _, Errors} -> {Json, {error, Errors}}\n    end.\n\n\n%% %%%%%%%%%%%%%%%%\n%% INTERNAL HELPER\n%% %%%%%%%%%%%%%%%%\n-spec read_helper(client_key(), prbr:read_filter()) ->\n    {ok, client_value()} | {fail, not_found} | {error, any()}.\nread_helper(Key, ReadFilter) ->\n    rbrcseq:qread(kv_db, self(), ?RT:hash_key(Key), ?MODULE, ReadFilter),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({qread_done, _ReqId, _NextFastWriteRound, _OldWriteRound, Value},\n                       case Value of\n                           ?NON_EXIST_VAL -> {fail, not_found};\n                           {error, Reason} -> {error, Reason};\n                           _ -> {ok, Value}\n                           end\n                      )\n    after 1000 ->\n        log:log(\"read hangs ~p~n\", [erlang:process_info(self(), messages)]),\n        receive\n            ?SCALARIS_RECV({qread_done, _ReqId, _NextFastWriteRound, _OldWriteRound, Value},\n                            case Value of\n                                ?NON_EXIST_VAL -> {fail, not_found};\n                                {error, Reason} -> {error, Reason};\n                                _ -> {ok, Value}\n                            end\n                          )\n        end\n    end.\n\n-spec write_helper(client_key(), client_value(), prbr:write_filter()) ->\n    ok | {fail | error, any()}.\nwrite_helper(Key, Value, WriteFilter) ->\n    write_helper(Key, Value, fun ?MODULE:rf_empty/1, fun ?MODULE:cc_noop/3,\n                  WriteFilter).\n\n-spec write_helper(client_key(), client_value(), prbr:read_filter(),\n                   fun((any(), any(), any()) -> any()), prbr:write_filter()) ->\n    ok | {fail | error, any()}.\nwrite_helper(Key, Value, ReadFilter, ContentCheck, WriteFilter) ->\n    rbrcseq:qwrite(kv_db, self(), ?RT:hash_key(Key), ?MODULE,\n                   ReadFilter, ContentCheck, WriteFilter, Value),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({qwrite_done, _ReqId, _NextFastWriteRound, _Value, WriteRet}, WriteRet);\n        ?SCALARIS_RECV({qwrite_deny, _ReqId, _NextFastWriteRound, _Value, Reason},\n                        begin\n                            log:log(\"Write failed on key ~p: ~p~n\", [Key, Reason]),\n                            {fail, Reason}\n                        end)\n    after 1000 ->\n        log:log(\"~p write hangs at key ~p, ~p~n\",\n                [self(), Key, erlang:process_info(self(), messages)]),\n        receive\n            ?SCALARIS_RECV({qwrite_done, _ReqId, _NextFastWriteRound, Value, WriteRet},\n                            begin\n                                log:log(\"~p write was only slow at key ~p~n\", [self(), Key]),\n                                WriteRet\n                            end);\n            ?SCALARIS_RECV({qwrite_deny, _ReqId, _NextFastWriteRound, _Value, Reason},\n                            begin\n                                log:log(\"~p Write failed: ~p~n\", [self(), Reason]),\n                                {fail, Reason}\n                            end)\n            end\n    end.\n\n%% @doc Helper that unpacks a list of errors if this list contains only one\n%% error. Does nothing otherwise.\n-spec unpack_if_error_list(any()) -> any().\nunpack_if_error_list({error, [Error]}) -> {error, Error};\nunpack_if_error_list(Any) -> Any.\n\n%% %%%%%%%%%%%%%%%%\n%% TESTER\n%% %%%%%%%%%%%%%%%%\n\n-spec is_json(term()) -> boolean().\nis_json({dict, _, _, _, _, _, _ ,_, _}) -> true;\nis_json(_) -> false.\n\n-spec create_rand_json(integer()) -> json().\ncreate_rand_json(_Seed) -> new_json(). %% TODO\n"
  },
  {
    "path": "src/rbr/kv_on_cseq.erl",
    "content": "% @copyright 2012-2015 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc    key value store based on rbrcseq.\n%% @end\n%% @version $Id$\n-module(kv_on_cseq).\n-author('schintke@zib.de').\n-vsn('$Id:$ ').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n\n-define(LOG_CC_FAILS, false).\n%%-define(LOG_CC_FAILS, true).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([read/1]).\n-export([write/2]).\n\n-export([work_phase_async/4]).\n\n-export([set_lock/3]).\n-export([commit_read/5, commit_read_feeder/5]).\n-export([commit_write/5, commit_write_feeder/5]).\n-export([abort_read/5, abort_read_feeder/5]).\n-export([abort_write/5, abort_write_feeder/5]).\n\n%% for selection among differing values in quorums with same paxos\n%% round (as they can occur in this data type, as we allow partial\n%% operations as setting or unlocking write locks (on maybe outdated\n%% replicas).\n-export([max/2]).\n\n%% filters and checks for rbr_cseq operations\n%% consistency\n%% -export([is_valid_next_req/3]).\n%% read filters\n-export([rf_rl_vers/1]).\n-export([rf_rl_wl_vers/1]).\n-export([rf_val/1]).\n-export([rf_val_vers/1]).\n-export([rf_wl_vers/1]).\n\n%% content checks\n-export([cc_abort_read/3]).\n-export([cc_abort_write/3]).\n-export([cc_commit_read/3]).\n-export([cc_commit_write/3]).\n-export([cc_set_rl/3]).\n-export([cc_set_wl/3]).\n-export([cc_single_write/3]).\n-export([cc_return_val/4]).\n\n%% write filters\n-export([wf_set_rl/3]).\n-export([wf_set_vers_val/3]).\n-export([wf_set_wl/3]).\n-export([wf_unset_rl/3]).\n-export([wf_unset_wl/3]).\n-export([wf_val_unset_wl/3]).\n\n-export([get_commuting_wf_for_rf/1]).\n\n\n-type txid() :: ?RT:key().\n\n-type readlock()  :: [txid()].\n-type writelock() :: txid() | no_write_lock.\n\n-type version()   :: non_neg_integer() | -1.\n-type value()     :: any().\n\n-type db_entry()  :: { %% plainkey(),\n                readlock(),\n                writelock(),\n                version(),\n                value()\n               }.\n\n%% @TODO add support for ?random_from_list | {{?sublist, Start::pos_integer() | neg_integer(), Len::integer()}\n-spec work_phase_async(comm:erl_local_pid(), any(), ?RT:key(), read | write) -> ok.\nwork_phase_async(ClientPid, ReqId, HashedKey, _Op) ->\n    ReplyTo = comm:reply_as(ClientPid, 3,\n                            {work_phase_async_done, ReqId, '_'}),\n    rbrcseq:qread(kv_db, ReplyTo, HashedKey, ?MODULE,\n                  fun ?MODULE:rf_val_vers/1),\n    ok.\n\n-spec rf_val_vers(db_entry() | prbr_bottom) -> {client_value(), version()}.\nrf_val_vers(prbr_bottom) -> {?value_dropped, -1};\nrf_val_vers(X)          -> {val(X), vers(X)}.\n\n\n\n%% %%%%%%%%%%%%%%%%%%%%%%\n%% functions for read\n%% %%%%%%%%%%%%%%%%%%%%%%\n-spec read(client_key()) -> api_tx:read_result().\nread(Key) ->\n    rbrcseq:qread(kv_db, self(), ?RT:hash_key(Key), ?MODULE,\n                  fun ?MODULE:rf_val/1),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({qread_done, _ReqId, _NextFastWriteRound, _OldWriteRound, Value},\n                       case Value of\n                           no_value_yet -> {fail, not_found};\n                           _ -> {ok, Value}\n                           end\n                      )\n    after 1000 ->\n            log:log(\"read hangs ~p~n\", [erlang:process_info(self(), messages)]),\n                receive\n                    ?SCALARIS_RECV({qread_done, _ReqId, _NextFastWriteRound, _OldWriteRound, Value},\n                                   case Value of\n                                       no_value_yet -> {fail, not_found};\n                                       _ -> {ok, Value}\n                                   end\n                                  )\n                    end\n        end.\n\n-spec rf_val(db_entry() | prbr_bottom) -> client_value().\nrf_val(prbr_bottom) -> no_value_yet;\nrf_val(X)          -> val(X).\n\n\n\n\n\n%% %%%%%%%%%%%%%%%%%%%%%%\n%% functions for write: this write ensures not to conflict with running tx\n%% %%%%%%%%%%%%%%%%%%%%%%\n-spec write(client_key(), client_value()) -> api_tx:write_result().\nwrite(Key, Value) ->\n    rbrcseq:qwrite(kv_db, self(), ?RT:hash_key(Key), ?MODULE,\n                   fun ?MODULE:rf_rl_wl_vers/1,\n                   fun ?MODULE:cc_single_write/3,\n                   fun ?MODULE:wf_set_vers_val/3, Value),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({qwrite_done, _ReqId, _NextFastWriteRound, _Value, _WriteRet}, {ok}); %%;\n        ?SCALARIS_RECV({qwrite_deny, _ReqId, _NextFastWriteRound, _Value, Reason},\n                       begin log:log(\"Write failed on key ~p: ~p~n\", [Key, Reason]),\n                       {ok} end) %% TODO: extend write_result type {fail, Reason} )\n    after 1000 ->\n            log:log(\"~p write hangs at key ~p, ~p~n\",\n                    [self(), Key, erlang:process_info(self(), messages)]),\n            receive\n                ?SCALARIS_RECV({qwrite_done, _ReqId, _NextFastWriteRound, Value, _WriteRet},\n                               begin\n                                   log:log(\"~p write was only slow at key ~p~n\",\n                                           [self(), Key]),\n                                   {ok}\n                               end); %%;\n                ?SCALARIS_RECV({qwrite_deny, _ReqId, _NextFastWriteRound, _Value, Reason},\n                               begin log:log(\"~p Write failed: ~p~n\",\n                                             [self(), Reason]),\n                                     {ok} end)\n                end\n    end.\n\n-spec cc_single_write(no_value_yet | {readlock(), writelock(), version()},\n                      prbr:write_filter(), Value :: any()) ->\n                             {true, UI :: version()} |\n                             {false, [readlock_is_set | writelock_is_set]}.\ncc_single_write(no_value_yet, _WriteFilter, _Val) ->\n    {true, -1};\ncc_single_write({RL, WL, Vers}, _WriteFilter, _Val) ->\n    Checks = [ { [] =:= RL,    readlock_is_set },\n               { no_write_lock =:= WL, writelock_is_set } ],\n    cc_return_val(cc_single_write, Checks, _UI_if_ok = Vers, ?LOG_CC_FAILS).\n\n-spec wf_set_vers_val(db_entry() | prbr_bottom, version(),\n                      client_value()) -> {db_entry(), none}.\nwf_set_vers_val(prbr_bottom, Version, WriteValue) ->\n    {{_RL = [], _WL = no_write_lock, Version, WriteValue}, none};\nwf_set_vers_val(Entry, Version, WriteValue) ->\n    T = set_vers(Entry, Version + 1),\n    {set_val(T, WriteValue), none}.\n\n\n\n\n%% %%%%%%%%%%%%%%%%%%%%%%\n%% functions for set_lock\n%% %%%%%%%%%%%%%%%%%%%%%%\n\n-spec set_lock(tx_tlog:tlog_entry(), txid(), comm:erl_local_pid()) -> ok.\nset_lock(TLogEntry, TxId, ReplyTo) ->\n    Key = tx_tlog:get_entry_key(TLogEntry),\n    % TODO: this is potentially unsafe depending on the RT implementation!\n    HashedKey = if is_list(Key) -> ?RT:hash_key(Key);\n                   true         -> Key\n                end,\n    case tx_tlog:get_entry_operation(TLogEntry) of\n        ?read ->\n            ReadFilter = fun ?MODULE:rf_wl_vers/1, %% read lock is irrelevant for reads\n            ContentCheck = fun ?MODULE:cc_set_rl/3,\n            WriteFilter = fun ?MODULE:wf_set_rl/3,\n            rbrcseq:qwrite(kv_db, ReplyTo, HashedKey, ?MODULE,\n                           ReadFilter,\n                           ContentCheck,\n                           WriteFilter,\n                           _Value = {TxId,\n                                     tx_tlog:get_entry_version(TLogEntry)});\n        ?write ->\n            ReadFilter = fun ?MODULE:rf_rl_wl_vers/1,\n            ContentCheck = fun ?MODULE:cc_set_wl/3,\n            WriteFilter = fun ?MODULE:wf_set_wl/3,\n            rbrcseq:qwrite(kv_db, ReplyTo, HashedKey, ?MODULE,\n                           ReadFilter,\n                           ContentCheck,\n                           WriteFilter, _Value = {TxId,\n                                                  tx_tlog:get_entry_version(TLogEntry)})\n    end.\n\n%% set_lock read: read filter, content check and write filter\n-spec rf_wl_vers(db_entry() | prbr_bottom) -> {writelock(), version()} | no_value_yet.\nrf_wl_vers(prbr_bottom) -> no_value_yet;\nrf_wl_vers(X) -> {writelock(X), vers(X)}.\n\n-spec cc_set_rl(no_value_yet | {writelock(), version()}, prbr:write_filter(),\n                {txid_on_cseq:txid(), version()}) ->\n                       {true, UI :: none} |\n                       {false, any()}.\ncc_set_rl(no_value_yet, _WF, _Val = {_TxId, _TLogVers}) ->\n    {true, none};\ncc_set_rl({WL, Vers}, _WF, _Val = {_TxId, TLogVers}) ->\n    Checks = [ { TLogVers =:= Vers, {cc_set_rl_version_mismatch, TLogVers, Vers} },\n               { no_write_lock =:= WL,      cc_set_rl_writelock_is_set } ],\n    cc_return_val(cc_set_rl, Checks, _UI_if_ok = none, ?LOG_CC_FAILS).\n\n-spec wf_set_rl(db_entry(), UI :: {writelock(), version()} | no_value_yet,\n                {txid_on_cseq:txid(), version()}) -> {db_entry(), none}.\nwf_set_rl(DBEntry, _UI, {TxId, _Vers}) ->\n    {set_readlock(DBEntry, TxId), none}.\n\n\n%% set_lock write: read filter, content check and write filter\n-spec rf_rl_wl_vers(db_entry() | prbr_bottom) ->\n        {readlock(), writelock(), version()} | no_value_yet. %% drop -1 in the future?\nrf_rl_wl_vers(prbr_bottom) -> no_value_yet;\nrf_rl_wl_vers(X)          -> {readlock(X),\n                              writelock(X),\n                              vers(X)}.\n\n-spec cc_set_wl({readlock, writelock(), version()}, prbr:write_filter(),\n                {txid_on_cseq:txid(), version()}) ->\n                       {true, UI :: none} |\n                       {false, any()}.\ncc_set_wl(no_value_yet, _WF, _Val = {_TxId, _TLogVers}) ->\n    {true, none};\ncc_set_wl({RL, WL, Vers}, _WF, _Val = {TxId, TLogVers}) ->\n    Checks =\n        [ { TLogVers =:= Vers,\n            {cc_set_wl_version_mismatch, TLogVers, Vers} },\n          { no_write_lock =:= WL\n            %% accept already existing WL from write through\n            orelse TxId =:= WL %% old comment (do not rewrite TxId)?\n          , {cc_set_wl_txid_mismatch, WL, TxId} },\n          { [] =:= RL,\n            {cc_set_wl_readlock_not_empty, RL} } ],\n    cc_return_val(cc_set_wl, Checks, _UI_if_ok = none, ?LOG_CC_FAILS).\n\n-spec wf_set_wl\n       (prbr_bottom, UI :: none, {txid_on_cseq:txid(), -1}) -> {db_entry(), none};\n       (db_entry(),  UI :: none, {txid_on_cseq:txid(), version()}) -> {db_entry(), none}.\nwf_set_wl(prbr_bottom, _UI = none, {TxId, -1}) ->\n    {set_writelock(new_entry(), TxId), none};\nwf_set_wl(prbr_bottom, _UI = none, {TxId, _}) ->\n%%    ct:pal(\"should only happen in tester unittests~n\"),\n    {set_writelock(new_entry(), TxId), none};\nwf_set_wl(DBEntry, _UI = none, {TxId, _TLogVers}) ->\n    {set_writelock(DBEntry, TxId), none}.\n\n%% %%%%%%%%%%%%%%%%%%%%%%%%%\n%% functions for commit_read\n%% %%%%%%%%%%%%%%%%%%%%%%%%%\n-spec commit_read_feeder(tx_tlog:tlog_entry(), txid(), comm:erl_local_pid(),\n                         pr:pr(), {txid_on_cseq:txid(), version()}) ->\n                                {tx_tlog:tlog_entry(),\n                                 txid(),\n                                 comm:erl_local_pid(),\n                                 pr:pr(),\n                                 {txid_on_cseq:txid(), version()}}.\ncommit_read_feeder(TLogEntry, TxId, Pid, Round, OldVal) ->\n    {TLogEntry, TxId, Pid, Round, OldVal}.\n\n%% @doc Releases the read lock of a given entry, if it was set.\n-spec commit_read(tx_tlog:tlog_entry(), txid(), comm:erl_local_pid(),\n                  pr:pr(), any()) -> ok.\ncommit_read(TLogEntry, TxId, ReplyTo, NextRound, OldVal) ->\n    Key = tx_tlog:get_entry_key(TLogEntry),\n    % TODO: this is potentially unsafe depending on the RT implementation!\n    HashedKey = if is_list(Key) -> ?RT:hash_key(Key);\n                   true         -> Key\n                end,\n    ReadFilter = fun ?MODULE:rf_rl_vers/1,\n    ContentCheck = fun ?MODULE:cc_commit_read/3,\n    WriteFilter = fun ?MODULE:wf_unset_rl/3,\n    rbrcseq:qwrite_fast(kv_db, ReplyTo, HashedKey, ?MODULE,\n                        ReadFilter,\n                        ContentCheck,\n                        WriteFilter,\n                        _Value = {TxId, tx_tlog:get_entry_version(TLogEntry)},\n                        NextRound, OldVal),\n    ok.\n\n-spec rf_rl_vers(db_entry() | prbr_bottom) ->\n                           {readlock(), version()} | no_value_yet.\nrf_rl_vers(prbr_bottom) -> no_value_yet;\nrf_rl_vers(X) ->\n    {readlock(X), vers(X)}.\n\n-spec cc_commit_read(no_value_yet | {readlock, version()}, prbr:write_filter(),\n                {txid_on_cseq:txid(), version()}) ->\n                            {true, UI :: none} |\n                            {false, any()}.\ncc_commit_read(no_value_yet, _WF, _Val = {_TxId, _TLogVers}) ->\n    {true, none};\ncc_commit_read({_RL, Vers}, _WF, _Val = {_TxId, TLogVers}) ->\n    Checks =\n        [ { TLogVers =:= Vers,  {cc_commit_read_version_mismatch, TLogVers, Vers} } ],\n    cc_return_val(cc_commit_read, Checks, _UI_if_ok = none, ?LOG_CC_FAILS);\ncc_commit_read({_RL, _WL, Vers}, _WF, _Val = {_TxId, TLogVers}) ->\n    Checks =\n        [ { TLogVers =:= Vers,  {cc_commit_read_version_mismatch, TLogVers, Vers} } ],\n    cc_return_val(cc_commit_read, Checks, _UI_if_ok = none, ?LOG_CC_FAILS).\n\n-spec wf_unset_rl\n       (prbr_bottom, UI :: none, {txid_on_cseq:txid(), 0}) -> {prbr_bottom, none};\n       (db_entry(), UI :: none, {txid_on_cseq:txid(), version()}) -> {db_entry(), none}.\nwf_unset_rl(prbr_bottom, _UI = none, {_TxId, _Vers}) -> {prbr_bottom, none};\nwf_unset_rl(DBEntry, _UI = none, {TxId, _TLogVers}) ->\n    {unset_readlock(DBEntry, TxId), none}.\n\n\n\n\n\n%% %%%%%%%%%%%%%%%%%%%%%%%%%\n%% functions for commit_write\n%% %%%%%%%%%%%%%%%%%%%%%%%%%\n-spec commit_write_feeder(tx_tlog:tlog_entry_write(), txid(), comm:erl_local_pid(),\n                          pr:pr(), {txid_on_cseq:txid(), version()}) ->\n                                 {tx_tlog:tlog_entry_write(), txid(),\n                                  comm:erl_local_pid(),\n                                  pr:pr(),\n                                  {txid_on_cseq:txid(), version()}}.\ncommit_write_feeder(TLogEntry, TxId, Pid, Round, OldVal) ->\n    {_, Val} = tx_tlog:get_entry_value(TLogEntry),\n    SaneCommitWriteTLogEntry = tx_tlog:set_entry_value(TLogEntry, ?value, Val),\n    {SaneCommitWriteTLogEntry, TxId, Pid, Round, OldVal}.\n\n%% @doc Releases the write lock of a given entry, if it was\n%%      set, and writes the new value.\n%% erlang shell test call: api_tx:write(\"a\", 1).\n-spec commit_write(tx_tlog:tlog_entry_write(), txid(), comm:erl_local_pid(),\n                   pr:pr(), any()) -> ok.\ncommit_write(TLogEntry, TxId, ReplyTo, NextRound, OldVal) ->\n    Key = tx_tlog:get_entry_key(TLogEntry),\n    % TODO: this is potentially unsafe depending on the RT implementation!\n    HashedKey = if is_list(Key) -> ?RT:hash_key(Key);\n                   true         -> Key\n                end,\n    ReadFilter = fun ?MODULE:rf_wl_vers/1,\n    ContentCheck = fun ?MODULE:cc_commit_write/3,\n    WriteFilter = fun ?MODULE:wf_val_unset_wl/3,\n    {?value, Value} = tx_tlog:get_entry_value(TLogEntry),\n    %% {?value, Value} = tx_tlog:get_entry_value(TLogEntry),\n    rbrcseq:qwrite_fast(kv_db, ReplyTo, HashedKey, ?MODULE,\n                        ReadFilter,\n                        ContentCheck,\n                        WriteFilter,\n                        _Value = {TxId, tx_tlog:get_entry_version(TLogEntry),\n                                  Value},\n                        NextRound, OldVal),\n    ok.\n\n-spec cc_commit_write(no_value_yet\n                      | {writelock(), version()}\n                      | {readlock(), writelock(), version()},\n                      prbr:write_filter(),\n                      {txid_on_cseq:txid(), version(), NewVal :: any()}) ->\n                             {true, UI :: none} |\n                             {false, Reason :: any()}.\ncc_commit_write(no_value_yet, _WF, _Val = {_TxId, _TLogVers, _NewVal}) ->\n    {true, none};\ncc_commit_write({WL, Vers}, _WF, _Val = {TxId, TLogVers, _NewVal}) ->\n    %% normal write:\n\n    %% The system may already went on, as the consensus may already be\n    %% free for others when the value was written half and another process\n    %% performed a write through.\n    %% It is ensured, that our write value was a consensus once, because\n    %% otherwise the content checkes of concurrent actions would fail:\n    %% (1) this tx was committed, so we proved that we were able to acquire\n    %%     the write_lock\n    %% (2) only one tx can acquire a write lock.\n\n    %% content check is performed on the latest quorum value, but we\n    %% have to accept also the newly written value, which may have\n    %% been propagated by a write through of concurrent operation, but\n    %% then we do not want to write again (let the cc fail).\n    Checks =\n        [ {  %% we release the lock\n             %% (Vers =:= TLogVers andalso\n             (WL =:= TxId)\n             orelse (Vers =< TLogVers) %% updated outdated replicas\n             %% we were interrupted during the release lock and\n             %% another process made a write through? But then the\n             %% register is free to be used for further ops... and any\n             %% value would be ok? So the outcome of the commit_write\n             %% is not important at all?\n             %%orelse (Vers >= TLogVers), %% Vers =:= (1 + TLogVers)),\n  ,          {cc_commit_write_tlog_version_to_small, WL, TxId, TLogVers, Vers} } ],\n    cc_return_val(cc_commit_write, Checks, _UI_if_ok = none, ?LOG_CC_FAILS);\n\n\ncc_commit_write({_RL, WL, Vers}, _WF, _Val = {TxId, TLogVers, _NewVal}) ->\n    %% fast_write:\n    %% in case of fast write we get the value of the last read as\n    %% write value, which here was produced by the read filter of\n    %% set_lock.\n\n    %% As we are in the case of fast_write, we can expect the\n    %% writelock to be in place. The version usually matches, but\n    %% could be outdated, when the write part of the setting of the\n    %% write_lock was successful on a quorum which included an\n    %% outdated replica (there, we cannot guarantee anything on the\n    %% value). But we will write our own value anyway and habe\n    %% exclusive write access, so until we get a quorum updated with\n    %% the right value, no one else will substatially interfere.\n    %% But what is when decided abort? How to rollback if the values\n    %% are no longer consistent across the replicas holding the WL?\n    Checks =\n        [ { Vers =< TLogVers,  {cc_commit_write_tlog_version_to_small,\n                                 TLogVers, Vers} },\n          { WL =:= TxId, {cc_commit_write_lock_is_not_txid, WL, TxId} } ],\n    cc_return_val(cc_commit_write, Checks, _UI_if_ok = none, ?LOG_CC_FAILS).\n\n\n-spec wf_val_unset_wl\n       (prbr_bottom, UI :: none, {txid_on_cseq:txid(), 0, value()}) -> {prbr_bottom, none};\n       (db_entry(), UI :: none, {txid_on_cseq:txid(), version(), value()}) -> {db_entry(), none}.\nwf_val_unset_wl(prbr_bottom, _UI = none, {_TxId, _Vers, _Val}) -> {prbr_bottom, none};\nwf_val_unset_wl(DBEntry, _UI = none, {_TxId, TLogVers, Val}) ->\n    case vers(DBEntry) =< TLogVers of\n        true ->\n            T1 = set_writelock(DBEntry, no_write_lock),\n            T2 = set_val(T1, Val),\n            T3 = reset_readlock(T2),\n            %% increment version counter on write\n            {set_vers(T3, 1 + TLogVers), none};\n        _ ->\n            {DBEntry, none}\n    end.\n\n\n\n\n\n%% %%%%%%%%%%%%%%%%%%%%%%%%%\n%% functions for abort_read\n%% %%%%%%%%%%%%%%%%%%%%%%%%%\n-spec abort_read_feeder(tx_tlog:tlog_entry(), txid(), comm:erl_local_pid(),\n                         pr:pr(), {txid_on_cseq:txid(), version()}) ->\n                                {tx_tlog:tlog_entry(),\n                                 txid(),\n                                 comm:erl_local_pid(),\n                                 pr:pr(),\n                                 {txid_on_cseq:txid(), version()}}.\nabort_read_feeder(TLogEntry, TxId, Pid, Round, OldVal) ->\n    {TLogEntry, TxId, Pid, Round, OldVal}.\n\n%% @doc Releases the read lock of a given entry, if it was set.\n-spec abort_read(tx_tlog:tlog_entry(), txid(), comm:erl_local_pid(),\n                  pr:pr(), any()) -> ok.\nabort_read(TLogEntry, TxId, ReplyTo, NextRound, OldVal) ->\n    Key = tx_tlog:get_entry_key(TLogEntry),\n    % TODO: this is potentially unsafe depending on the RT implementation!\n    HashedKey = if is_list(Key) -> ?RT:hash_key(Key);\n                   true         -> Key\n                end,\n    ReadFilter = fun ?MODULE:rf_rl_vers/1,\n    ContentCheck = fun ?MODULE:cc_abort_read/3,\n    WriteFilter = fun ?MODULE:wf_unset_rl/3,\n    rbrcseq:qwrite_fast(kv_db, ReplyTo, HashedKey, ?MODULE,\n                        ReadFilter,\n                        ContentCheck,\n                        WriteFilter,\n                        _Value = {TxId, tx_tlog:get_entry_version(TLogEntry)},\n                        NextRound, OldVal),\n    ok.\n\n%% @doc The content check for abort_read has only to ensure, that the\n%% readlock is eliminated when it is there. The write operation (unset\n%% read lock) is performed, when the result is {true, _}. Otherwise,\n%% the tx still passes and it is assumed that either the readlock was\n%% not acquired, or was already eliminated by a write_through\n%% operation of the rbrcseq module due to concurrency.\n-spec cc_abort_read(no_value_yet | {readlock, version()}, prbr:write_filter(),\n                {txid_on_cseq:txid(), version()}) ->\n                       {boolean(), UI :: none}.\ncc_abort_read(no_value_yet, _WF, _Val = {_TxId, _TLogVers}) ->\n    %% we do not see a readlock, so it was not acquired\n    {false, none};\ncc_abort_read({_WL = no_write_lock, _Vers}, _WF, _Val = {_TxId, _TLogVers}) ->\n    %% we can get the write_filter value...\n    %% we do not see a readlock, so it was not acquired\n    {false, none};\ncc_abort_read({RL, _Vers}, _WF, _Val = {TxId, _TLogVers}) when is_list(RL) ->\n    %% ensure readlock is gone...\n    case lists:member(TxId, RL) of\n        true -> {true, none};\n        false -> {false, none}\n                 %% RL is already gone or was not acquired, so we do not\n                 %% need to update the entry\n    end;\ncc_abort_read({_WL, _Vers}, _WF, _Val = {_TxId, _TLogVers}) ->\n    %% we can get the write_filter value...\n    %% we do not see a readlock, so it was not acquired\n    {false, none};\ncc_abort_read({RL, _WL, _Vers}, _WF, _Val = {TxId, _TLogVers}) ->\n    %% we can get the write_filter value...\n    %% ensure readlock is gone...\n    case lists:member(TxId, RL) of\n        true -> {true, none};\n        false -> {false, none}\n                 %% RL is already gone or was not acquired, so we do\n                 %% not need to update the entry\n    end.\n\n\n%% %%%%%%%%%%%%%%%%%%%%%%%%%\n%% functions for abort_write\n%% %%%%%%%%%%%%%%%%%%%%%%%%%\n-spec abort_write_feeder(tx_tlog:tlog_entry_write(), txid(), comm:erl_local_pid(),\n                          pr:pr(), {txid_on_cseq:txid(), version()}) ->\n                                 {tx_tlog:tlog_entry_write(), txid(),\n                                  comm:erl_local_pid(),\n                                  pr:pr(),\n                                  {txid_on_cseq:txid(), version()}}.\nabort_write_feeder(TLogEntry, TxId, Pid, Round, OldVal) ->\n    {_, Val} = tx_tlog:get_entry_value(TLogEntry),\n    SaneCommitWriteTLogEntry = tx_tlog:set_entry_value(TLogEntry, ?value, Val),\n    {SaneCommitWriteTLogEntry, TxId, Pid, Round, OldVal}.\n\n%% @doc Releases the write lock of a given entry, if it was set.\n%% erlang shell test call: api_tx:write(\"a\", 1).\n-spec abort_write(tx_tlog:tlog_entry_write(), txid(), comm:erl_local_pid(),\n                   pr:pr(), any()) -> ok.\nabort_write(TLogEntry, TxId, ReplyTo, NextRound, OldVal) ->\n    Key = tx_tlog:get_entry_key(TLogEntry),\n    % TODO: this is potentially unsafe depending on the RT implementation!\n    HashedKey = if is_list(Key) -> ?RT:hash_key(Key);\n                   true         -> Key\n                end,\n    ReadFilter = fun ?MODULE:rf_wl_vers/1,\n    ContentCheck = fun ?MODULE:cc_abort_write/3,\n    WriteFilter = fun ?MODULE:wf_unset_wl/3,\n    {?value, Value} = tx_tlog:get_entry_value(TLogEntry),\n    %% {?value, Value} = tx_tlog:get_entry_value(TLogEntry),\n    rbrcseq:qwrite_fast(kv_db, ReplyTo, HashedKey, ?MODULE,\n                        ReadFilter,\n                        ContentCheck,\n                        WriteFilter,\n                        _Value = {TxId, tx_tlog:get_entry_version(TLogEntry),\n                                  Value},\n                        NextRound, OldVal),\n    ok.\n\n-spec cc_abort_write(no_value_yet\n                     | {writelock(), version()}\n                     | {readlock(), writelock(), version()},\n                     prbr:write_filter(),\n                     {txid_on_cseq:txid(), version(), NewVal :: any()}) ->\n                            {true, UI :: none} |\n                            {false, any()}.\ncc_abort_write(no_value_yet, _WF, _Val = {_TxId, _TLogVers, _NewVal}) ->\n    {false, none};\ncc_abort_write({WL, Vers}, _WF, _Val = {TxId, TLogVers, _NewVal}) ->\n    %% normal qwrite\n\n    %% if there is our lock, we release it, otherwise we do not touch the value\n    Checks =\n        [ { Vers =:= TLogVers,  {cc_abort_write_tlog_version_mismatch,\n                                 TLogVers, Vers} },\n          { WL =:= TxId, {cc_abort_write_lock_is_not_txid, WL, TxId} } ],\n    cc_return_val(cc_abort_write, Checks, _UI_if_ok = none, ?LOG_CC_FAILS);\ncc_abort_write({_RL, WL, Vers}, _WF, _Val = {TxId, TLogVers, _NewVal}) ->\n    %% qwrite_fast\n    %% if there is our lock, we release it, otherwise we were not able\n    %% to acquire the lock during validation, so we do not touch the\n    %% value (let the cc fail).\n    Checks =\n        [ { Vers =:= TLogVers,  {cc_abort_write_tlog_version_mismatch,\n                                 TLogVers, Vers} },\n          { WL =:= TxId, {cc_abort_write_lock_is_not_txid, WL, TxId} } ],\n    cc_return_val(cc_abort_write, Checks, _UI_if_ok = none, ?LOG_CC_FAILS).\n\n-spec wf_unset_wl\n       (prbr_bottom, UI :: none, {txid_on_cseq:txid(), 0, value()}) -> {prbr_bottom, none};\n       (db_entry(), UI :: none, {txid_on_cseq:txid(), version(), value()}) -> {db_entry(), none}.\nwf_unset_wl(prbr_bottom, _UI = none, {_TxId, _Vers, _Val}) -> {prbr_bottom, none};\nwf_unset_wl(DBEntry, _UI = none, {TxId, _TLogVers, _Val}) ->\n    case writelock(DBEntry) of\n       TxId -> {set_writelock(DBEntry, no_write_lock), none};\n        _ -> {DBEntry, none}\n    end.\n\n\n\n\n-spec cc_return_val(atom(), [{boolean(), tuple()|atom()}], UI :: any(), boolean()) ->\n                           {true, UI :: any()} |\n                           {false, any()}.\ncc_return_val(WhichCC, Checks, UI, true) ->\n    %% copy cuntion from below to reduce overhead when not logging\n\n    %% log is defined as macro ?LOG_CC_FAILS. Unfortunately\n    %% dialyzer claims Log to be always false. But for\n    %% debugging purposes we can alter this for individual\n    %% content checks or enable logging for all of them by\n    %% redefining the macro. So it seems we have to live\n    %% with dialyzer's warning here.\n    lists:foldl(\n      fun({true, _Xreason}, {true, _UI_or_Reasons}) ->\n              {true, UI};\n         ({true, _Xreason}, {false, UI_or_Reasons}) ->\n              {false, UI_or_Reasons};\n         ({false, Xreason}, {true, _UI_or_Reasons}) ->\n              log:log(\"~p cc failed: ~.0p~n\", [WhichCC, Xreason]),\n              {false, [Xreason]};\n         ({false, Xreason}, {false, UI_or_Reasons}) ->\n              log:log(\"~p cc failed: ~.0p~n\", [WhichCC, Xreason]),\n              {false, [Xreason | UI_or_Reasons]}\n      end, {true, UI}, Checks);\ncc_return_val(_WhichCC, Checks, UI, false) ->\n    lists:foldl(\n      fun({true, _Xreason}, {true, _UI_or_Reasons}) ->\n              {true, UI};\n         ({true, _Xreason}, {false, UI_or_Reasons}) ->\n              {false, UI_or_Reasons};\n         ({false, Xreason}, {true, _UI_or_Reasons}) ->\n              {false, [Xreason]};\n         ({false, Xreason}, {false, UI_or_Reasons}) ->\n              {false, [Xreason | UI_or_Reasons]}\n      end, {true, UI}, Checks).\n\n\n\n\n%% abstract data type db_entry()\n\n-spec new_entry() -> db_entry().\nnew_entry() ->\n    {[], no_write_lock, -1, ?value_dropped}.\n\n-spec readlock(db_entry()) -> readlock().\nreadlock(Entry) -> element(1, Entry).\n-spec set_readlock(db_entry(), txid()) -> db_entry().\nset_readlock(prbr_bottom, TxId) ->\n    set_readlock(new_entry(), TxId);\nset_readlock(Entry, TxId) ->\n    NewRL = [TxId | element(1, Entry)],\n    setelement(1, Entry, NewRL).\n-spec unset_readlock(db_entry(), txid()) -> db_entry().\nunset_readlock(Entry, TxId) ->\n    %% delete all occurrences of TxId\n    NewRL = [ X || X <- element(1, Entry), X =/= TxId ],\n    setelement(1, Entry, NewRL).\n-spec reset_readlock(db_entry()) -> db_entry().\nreset_readlock(Entry) ->\n    setelement(1, Entry, []).\n\n\n-spec writelock(db_entry()) -> writelock().\nwritelock(Entry) -> element(2, Entry).\n-spec set_writelock(db_entry(), writelock()) -> db_entry().\nset_writelock(Entry, TxId) -> setelement(2, Entry, TxId).\n-spec vers(db_entry()) -> version().\nvers(Entry) -> element(3, Entry).\n-spec set_vers(db_entry(), version()) -> db_entry().\nset_vers(Entry, Vers) -> setelement(3, Entry, Vers).\n-spec val(db_entry()) -> value().\nval(Entry) -> element(4, Entry).\n-spec set_val(db_entry(), value()) -> db_entry().\nset_val(Entry, Val) -> setelement(4, Entry, Val).\n\n-spec max(db_entry(), db_entry()) -> db_entry();\n         ({client_value(), version()}, {client_value(), version()}) -> {client_value(), version()}.\nmax({_ValA, VersA} = A, {_ValB, VersB} = B) ->\n    %% partial entries A and B produced by read filter rf_val_vers\n    ?TRACE(\"A (~p) > B (~p)?\", [A,B]),\n    if VersA > VersB ->\n           A;\n       true ->\n           %% do we have further to distinguish on same version\n           %% values whether locks are set or not?\n           %% @TODO Think about it...\n           B\n    end;\nmax(A, B) ->\n    ?TRACE(\"A (~p) > B (~p)?\", [A,B]),\n    VersA = vers(A), VersB = vers(B),\n    if VersA > VersB ->\n           A;\n       true ->\n           %% do we have further to distinguish on same version\n           %% values whether locks are set or not?\n           %% @TODO Further think about it...\n           B\n    end.\n\n-spec get_commuting_wf_for_rf(prbr:read_filter()) ->\n        [prbr:write_filter()].\nget_commuting_wf_for_rf(RF) ->\n    {name, Name} = erlang:fun_info(RF, name),\n    {module, Module} = erlang:fun_info(RF, module),\n    case {Module, Name} of\n        {?MODULE, rf_val} ->\n            [fun ?MODULE:wf_set_rl/3,\n             fun ?MODULE:wf_unset_rl/3,\n             fun ?MODULE:wf_set_wl/3,\n             fun ?MODULE:wf_unset_wl/3];\n        {?MODULE, rf_val_vers} ->\n            [fun ?MODULE:wf_set_rl/3,\n             fun ?MODULE:wf_unset_rl/3,\n             fun ?MODULE:wf_set_wl/3,\n             fun ?MODULE:wf_unset_wl/3];\n        {?MODULE, rf_rl_vers} ->\n            [fun ?MODULE:wf_set_wl/3,\n             fun ?MODULE:wf_unset_wl/3];\n        {?MODULE, rf_wl_vers} ->\n            [fun ?MODULE:wf_set_rl/3,\n             fun ?MODULE:wf_unset_rl/3];\n        _ -> []\n    end.\n"
  },
  {
    "path": "src/rbr/pr.erl",
    "content": "% @copyright 2014-2016 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc    Abstract data type for a paxos round (pr).\n%% @end\n%% @version $Id$\n-module(pr).\n-author('schintke@zib.de').\n-vsn('$Id:$ ').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n-include(\"scalaris.hrl\").\n\n-export([new/2]).\n-export([get_r/1]).\n-export([get_id/1]).\n-export([get_wti/1, set_wti/2]).\n%% let users retrieve their smallest possible round for fast_write on\n%% entry creation.\n-export([smallest_round/1]).\n\n%% let users retrieve their uid from an assigned round number.\n\n%% let users retrieve their smallest possible round for fast_write on\n%% entry creation.\n%%-export([smallest_round/1]).\n\n-export_type([pr/0]).\n\n-type write_through_info() ::\n          {\n           WriteReturn :: any(),\n           OriginalLearner :: any()\n          }.\n\n%% pr() has to be unique for this key system wide\n%% pr() has to be comparable with < and =<\n%% if the caller process may handle more than one request at a time for the\n%% same key, it has to be unique for each request.\n%% for example\n%% -type pr() :: {pos_integer(), comm:mypid(), none}.\n%% -type pr() :: {pos_integer(), {dht_node_id(), lease_epoch()}, none}.\n-type pr() ::\n        {non_neg_integer(),         %% the actual round number\n         any(),                     %% the clients identifier to make unique\n         %% the used write_filter for write through and its parameters\n         none | write_through_info() %% ...\n        }.\n\n-spec new(non_neg_integer(), any()) -> pr().\nnew(Counter, ProposerUID) -> {Counter, ProposerUID, none}.\n\n-spec get_r(pr()) -> non_neg_integer().\nget_r(RwId) -> element(1, RwId).\n\n-spec get_id(pr()) -> any().\nget_id(RwId) -> element(2, RwId).\n\n-spec get_wti(pr()) -> none | write_through_info().\nget_wti(RwId) -> element(3, RwId).\n\n-spec set_wti(pr(), none | write_through_info()) -> pr().\nset_wti(R, WTI) -> setelement(3, R, WTI).\n\n%% As the round number contains the client's pid, they are still\n%% unique.  Two clients using their smallest_round for a fast write\n%% concurrently are separated, because we do not use plain Paxos, but\n%% assign a succesful writer immediately the next round_number for a\n%% follow up fast_write by increasing our read_round already on the\n%% write.  So, the smallest_round of the second client becomes invalid\n%% when the first one writes.  In consequence, at most one proposer\n%% can perform a successful fast_write with its smallest_round. Voila!\n-spec smallest_round(comm:mypid()) -> pr().\nsmallest_round(Pid) -> new(1, Pid).\n\n"
  },
  {
    "path": "src/rbr/prbr.erl",
    "content": "% @copyright 2012-2018 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc    Generic paxos round based register (prbr) implementation.\n%%         The read/write store alias acceptor.\n%% @end\n%% @version $Id$\n-module(prbr).\n-author('schintke@zib.de').\n-vsn('$Id:$ ').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n-include(\"scalaris.hrl\").\n\n-define(PDB, db_prbr).\n-define(REPAIR, config:read(prbr_repair_on_write)).\n-define(REPAIR_DELAY, 100). % in ms\n-define(REDUNDANCY, (config:read(redundancy_module))).\n\n%%% the prbr has to be embedded into a gen_component using it.\n%%% The state it operates on has to be passed to the on handler\n%%% correctly. All messages it handles start with the token\n%%% prbr to support a generic, embeddable on-handler trigger.\n\n%%% functions for module where embedded into\n-export([on/2, init/1, close/1, close_and_delete/1]).\n-export([check_config/0]).\n-export([noop_read_filter/1]).  %% See rbrcseq for explanation.\n-export([noop_write_filter/3]). %% See rbrcseq for explanation.\n-export([new/2]).\n-export([set_entry/2]).\n-export([get_entry/2]).\n-export([delete_entry/2]).\n-export([entry_key/1]).\n-export([entry_val/1]).\n-export([entry_set_val/2]).\n\n-export([tester_create_write_filter/1]).\n\n%% let fetch the number of DB entries\n-export([get_load/1]).\n\n%% only for unittests\n-export([tab2list_raw_unittest/1]).\n\n%% only during recover\n-export([tab2list/1]).\n\n-export_type([message/0]).\n-export_type([state/0]).\n-export_type([read_filter/0]).\n-export_type([write_filter/0]).\n-export_type([entry/0]).\n\n%% read_filter(custom_data() | no_value_yet) -> read_info()\n-type read_filter() :: fun((term()) -> term()).\n\n%% write_filter(OldLocalDBentry :: custom_data(),\n%%              InfosToUpdateOutdatedEntry :: info_passed_from_read_to_write(),\n%%              ValueForWriteOperation:: Value())\n%% -> {custom_data(), value_returned_to_caller()}\n-type write_filter() :: fun((term(), term(), term()) -> {term(), term()}).\n\n-type state() :: ?PDB:db().\n\n-type message() ::\n        {prbr, read, DB :: dht_node_state:db_selector(),\n         WasConsistentLookup :: boolean(),\n         Proposer :: comm:mypid(), ?RT:key(), DataType :: module(),\n         InRound :: pr:pr(),\n         read_filter()}\n      | {prbr, write, DB :: dht_node_state:db_selector(),\n         WasConsistentLookup :: boolean(),\n         Proposer :: comm:mypid(), ?RT:key(), DataType :: module(),\n         InRound :: pr:pr(), Value :: term(), PassedToUpdate :: term(),\n         write_filter()}\n      | {prbr, delete_key, DB :: dht_node_state:db_selector(),\n         Client :: comm:mypid(), Key :: ?RT:key()}\n      | {prbr, tab2list_raw, DB :: dht_node_state:db_selector(),\n         Client :: comm:mypid()}.\n\n\n%% improvements to usual paxos:\n\n%% for reads the client has just to send a unique identifier and the\n%% acceptor provides a valid round number. The actual round number of\n%% the request is then the tuple {round_number, unique_id}.\n\n%% A proposer then may receive answers with different round numbers\n%% and selects that one with the highest round, where he also takes\n%% the value from.\n\n%% on read the acceptor can assign the next round number. They remain\n%% unique as we get the node_id in the read request and that is part of\n%% the round number.\n\n%% so there are no longer any read denies, all reads succeed.\n-type entry() :: { any(), %% key\n                   pr:pr(), %% r_read\n                   pr:pr(), %% r_write\n                   any(), %% value\n                   prbr:write_filter() %% last applied write filter\n                 }.\n\n%% Messages to expect from this module\n-spec msg_round_request_reply(comm:mypid(), boolean(), pr:pr(), pr:pr(),\n                              any(), prbr:write_filter()) -> ok.\nmsg_round_request_reply(Client, Cons, ReadRound, WriteRound, Value, LastWF) ->\n    comm:send(Client, {round_request_reply, Cons,  ReadRound, WriteRound, Value, LastWF}).\n\n-spec msg_read_reply(comm:mypid(), Consistency::boolean(), pr:pr(), atom(),\n                     any(), prbr:write_filter()) -> ok.\nmsg_read_reply(Client, Cons, YourRound, Val, LastWriteRound, LastWF) ->\n    comm:send(Client, {read_reply, Cons, YourRound, Val, LastWriteRound, LastWF}).\n\n-spec msg_read_deny(comm:mypid(), Consistency::boolean(), pr:pr(), pr:pr()) -> ok.\nmsg_read_deny(Client, Cons, YourRound, LargerRound) ->\n    comm:send(Client, {read_deny, Cons, YourRound, LargerRound}).\n\n-spec msg_write_reply(comm:mypid(), Consistency::boolean(),\n                      any(), pr:pr(), pr:pr(), any()) -> ok.\nmsg_write_reply(Client, Cons, Key, UsedWriteRound, YourNextRoundForWrite, WriteRet) ->\n    comm:send(Client, {write_reply, Cons, Key, UsedWriteRound, YourNextRoundForWrite, WriteRet}).\n\n-spec msg_learned_reply(comm:mypid(),  pr:pr(), pr:pr(), any()) -> ok.\nmsg_learned_reply(Client, UsedWriteRound, YourNextRoundForWrite, WriteRet) ->\n    comm:send(Client, {learned_reply, UsedWriteRound, YourNextRoundForWrite, WriteRet}).\n\n-spec msg_write_deny(comm:mypid(), Consistency::boolean(), any(), pr:pr())\n                    -> ok.\nmsg_write_deny(Client, Cons, Key, WriteRoundTried) ->\n    comm:send(Client, {write_deny, Cons, Key, WriteRoundTried}).\n\n-spec noop_read_filter(term()) -> term().\nnoop_read_filter(X) -> X.\n\n-spec noop_write_filter(Old :: term(), UpdateInfo :: term(), Val :: term()) -> {term(), term()}.\nnoop_write_filter(_, UI, X) -> {X, UI}.\n\n%% initialize: return initial state.\n-spec init(atom() | tuple()) -> state().\ninit(DBName) -> ?PDB:new(DBName).\n\n%% @doc Closes the given DB (it may be recoverable using open/1 depending on\n%%      the DB back-end).\n-spec close(state()) -> true.\nclose(State) -> ?PDB:close(State).\n\n%% @doc Closes the given DB and deletes all contents (this DB can thus not be\n%%      re-opened using open/1).\n-spec close_and_delete(state()) -> true.\nclose_and_delete(State) -> ?PDB:close_and_delete(State).\n\n-spec on(message(), state()) -> state().\non({prbr, round_request, _DB, Cons, Proposer, Key, DataType, ProposerUID, ReadFilter, OpType}, TableName) ->\n    ?TRACE(\"prbr:round_request: ~p in round ~p~n\", [Key, ProposerUID]),\n    KeyEntry = get_entry(Key, TableName),\n\n    %% custom prbr read handler might change value (e.g. due to GCing)\n    {NewKeyEntryVal, ReadVal, ValueHasChanged} =\n        case erlang:function_exported(DataType, prbr_read_handler, 3) of\n            true -> DataType:prbr_read_handler(KeyEntry, TableName, ReadFilter);\n            _    -> {entry_val(KeyEntry), ReadFilter(entry_val(KeyEntry)), false}\n        end,\n\n    %% Assign a valid next read round number\n    %% If this round_request is part a read it is not necessary\n    %% to increment the read round number, since a read does not\n    %% interfere with other reads or writes.\n    %% This allows concurrent reads and prevents\n    %% reads to interfere with concurrent writes.\n    AssignedReadRound =\n        case OpType =:= read of\n            true ->\n                OldRound = entry_r_read(KeyEntry),\n                pr:new(pr:get_r(OldRound), ProposerUID);\n            _ ->\n                next_read_round(KeyEntry, ProposerUID)\n        end,\n    trace_mpath:log_info(self(), {'prbr:on(round_request)',\n                                  %% key, Key,\n                                  round, AssignedReadRound,\n                                  val, NewKeyEntryVal,\n                                  read_filter, ReadFilter}),\n\n    msg_round_request_reply(Proposer, Cons, AssignedReadRound, entry_r_write(KeyEntry),\n                            ReadVal, entry_last_wf(KeyEntry)),\n\n    _ = case OpType =:= read andalso not ValueHasChanged of\n        true ->\n            %% No updates must be performed; save a DB write operation\n            %% (Above, the proposer UID of the read round is changed even\n            %% for a read. The new round must be returned to the client since\n            %% it contains the request id for the round_request, but it does\n            %% not have to be stored in DB since there are no follow-up requests\n            %% in a read and reads do not interfere with other ops)\n            ok;\n        false ->\n            NewKeyEntry = entry_set_r_read(KeyEntry, AssignedReadRound),\n            NewKeyEntry2 = entry_set_val(NewKeyEntry, NewKeyEntryVal),\n            set_entry(NewKeyEntry2, TableName)\n    end,\n\n    TableName;\n\non({prbr, read, _DB, Cons, Proposer, Key, DataType, ProposerUID, ReadFilter,\n   ReadRound}, TableName) ->\n    ?TRACE(\"prbr:read: ~p in round ~p~n\", [Key, ReadRound]),\n    KeyEntry = get_entry(Key, TableName),\n\n    _ = case pr:get_r(ReadRound) > pr:get_r(entry_r_read(KeyEntry)) of\n         true ->\n             %% prbr read handler might change value (e.g. due to GCing)\n            {NewKeyEntryVal, ReadVal, _ValueHasChanged} =\n                case erlang:function_exported(DataType, prbr_read_handler, 3) of\n                    true -> DataType:prbr_read_handler(KeyEntry, TableName, ReadFilter);\n                    _    -> {entry_val(KeyEntry), ReadFilter(entry_val(KeyEntry)), false}\n                end,\n            trace_mpath:log_info(self(), {'prbr:on(read)',\n                                          %% key, Key,\n                                          round, ReadRound,\n                                          val, NewKeyEntryVal,\n                                          read_filter, ReadFilter}),\n\n            EntryWriteRound = entry_r_write(KeyEntry),\n            % A prbr read has a different request ID than the previously executed\n            % round_request, and therefore a different ProposerUID.\n            NewReadRound = pr:new(pr:get_r(ReadRound), ProposerUID),\n\n            msg_read_reply(Proposer, Cons, NewReadRound, ReadVal,\n                           EntryWriteRound, entry_last_wf(KeyEntry)),\n\n            NewKeyEntry = entry_set_r_read(KeyEntry, NewReadRound),\n            NewKeyEntry2 = entry_set_val(NewKeyEntry, NewKeyEntryVal),\n            _ = set_entry(NewKeyEntry2, TableName);\n         _ ->\n            msg_read_deny(Proposer, Cons, ReadRound, entry_r_read(KeyEntry))\n    end,\n    TableName;\n\non({prbr, write, DB, Cons, Proposer, Key, DataType, ProposerUID, InRound, OldWriteRound, Value,\n    PassedToUpdate, WriteFilter, IsWriteThrough}, TableName) ->\n    ?TRACE(\"prbr:write for key: ~p in round ~p~n\", [Key, InRound]),\n    trace_mpath:log_info(self(), {prbr_on_write}),\n    KeyEntry = get_entry(Key, TableName),\n\n    %% When this is part of a write through keep the old proposer,\n    %% so that the original proposer of the write gets notified of\n    %% write progress as well.\n    LearnerForWTI =\n        case IsWriteThrough andalso ?REDUNDANCY:notify_orig_learner_on_wt() of\n                false ->\n                    case pr:get_wti(OldWriteRound) of\n                        {Return, OrigLearner} ->\n                            msg_learned_reply(OrigLearner, OldWriteRound, InRound, Return);\n                        _ ->\n                            %% first write has no predecessor\n                            ok\n                    end,\n                    Proposer;\n                true ->\n                    {_, OrigLearner} = pr:get_wti(OldWriteRound),\n                    %% The follow-up behaviour of a WT does not change if it is\n                    %% successful or denied. In both cases the original request\n                    %% is retried. Therefore we do need to keep old WT learner around\n                    %% since sending updated progress is of no benefit. This prevents\n                    %% a long list of WT-Learner caused by write throughs followed by\n                    %% write throughs due to duelling requests (which is likely for a\n                    %% high replication factor with a high number of concurrent requests).\n                    OrigLearner\n              end,\n    _ = case writable(KeyEntry, InRound, OldWriteRound, WriteFilter) of\n            true ->\n                {NewVal, Ret} =\n                    case erlang:function_exported(DataType, prbr_write_handler, 5) of\n                        true ->\n                            DataType:prbr_write_handler(KeyEntry, PassedToUpdate,\n                                                        Value, TableName, WriteFilter);\n                        _    ->\n                            WriteFilter(entry_val(KeyEntry), PassedToUpdate, Value)\n                    end,\n\n                TWriteRound = pr:new(pr:get_r(InRound), ProposerUID),\n                %% store information to be able to reproduce the request in\n                %% write_throughs. We modify the InRound here to avoid duplicate\n                %% transfer of the Value etc.\n                NewWriteRound = pr:set_wti(TWriteRound, {Ret, LearnerForWTI}),\n                TEntry = entry_set_r_write(KeyEntry, NewWriteRound),\n\n                %% prepare for fast write\n                NextWriteRound = next_read_round(TEntry, ProposerUID),\n                T2Entry = entry_set_r_read(TEntry, NextWriteRound),\n\n                NewEntry = entry_set_last_wf(T2Entry, WriteFilter),\n\n                trace_mpath:log_info(self(), {'prbr:on(write)',\n                                              round, NewWriteRound,\n                                              passed_to_update, PassedToUpdate,\n                                              val, Value,\n                                              write_filter, WriteFilter,\n                                              newval, NewVal}),\n                msg_write_reply(Proposer, Cons, Key, NewWriteRound,\n                                  NextWriteRound, Ret),\n\n                set_entry(entry_set_val(NewEntry, NewVal), TableName);\n            {false, Reason} ->\n                RoundTried = pr:new(pr:get_r(InRound), ProposerUID),\n                trace_mpath:log_info(self(), {'prbr:on(write) denied',\n                                              round, RoundTried}),\n\n                case Reason of\n                    repair_required ->\n                        %% Start repair on write\n                        ?TRACE(\"Initiating on-write repair for entry~n~p\", [KeyEntry]),\n                        comm:send_local_after(?REPAIR_DELAY, self(),\n                                              {prbr,\n                                               init_repair_on_write,\n                                               DB,\n                                               entry_key(KeyEntry),\n                                               entry_r_write(KeyEntry)}),\n                        ok;\n                    _ -> ok\n                end,\n\n                %% When the client recieves enough denies, it must completely restart\n                %% its write, therefore it is not necessary to send any \"newer round\".\n                %% The round the client used to write is send back, because the client\n                %% must distinguish between denies from its own request and possible\n                %% write through attempts based on its partially completed write.\n                msg_write_deny(Proposer, Cons, Key, RoundTried)\n        end,\n    TableName;\n\non({prbr, init_repair_on_write, DB, Key, KnownWriteRound}, TableName) ->\n    %% Triggers a repair process for this replica.\n    %% The repair process will send a qread to retrieve the most recent rounds\n    %% and value. This data will be used to refresh the replica under the condition\n    %% that no other process has written on this replica in the meantime (because\n    %% this is only possible if the most recent value was already established).\n    KeyEntry = get_entry(Key, TableName),\n    _ = case KnownWriteRound =:= entry_r_write(KeyEntry) of\n          true ->\n            ?TRACE(\"Starting read for on-write repair on key~n~p\", [Key]),\n            Envelope = {prbr, repair_on_write, DB, Key, KnownWriteRound, '_'},\n            SendTo = comm:reply_as(self(), 6, Envelope),\n            rbrcseq:qread(DB, SendTo, Key, prbr_on_use_repair);\n          false -> ok\n        end,\n    TableName;\n\non({prbr, repair_on_write, _DB, Key, KnownWriteRound,\n    {qread_done, _ReqId, ReadRound, WriteRound, Value}}, TableName) ->\n    KeyEntry = get_entry(Key, TableName),\n    _ = case KnownWriteRound =:= entry_r_write(KeyEntry) of\n            true ->\n                %% Change only if this replica was not modified yet\n                ?TRACE(\"On-write repair of replica on key ~n~p\", [Key]),\n                KeyEntry2 = entry_set_r_read(KeyEntry, ReadRound),\n                KeyEntry3 = entry_set_r_write(KeyEntry2, WriteRound),\n                KeyEntry4 = entry_set_val(KeyEntry3, Value),\n                _ = set_entry(KeyEntry4, TableName);\n            false -> ok\n        end,\n    TableName;\n\non({prbr, delete_key, _DB, Client, Key}, TableName) ->\n    %% for normal delete we will have to have a special write operation taking\n    %% the Paxos round numbers into account...\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    ct:pal(\"R~p deleted~n\", [?RT:get_key_segment(Key)]),\n    Entry = get_entry(Key, TableName),\n    _ = delete_entry(TableName, Entry),\n    comm:send_local(Client, {delete_key_reply, Key}),\n    TableName;\n\non({prbr, get_entry, _DB, Client, Key}, TableName) ->\n    comm:send_local(Client, {entry, get_entry(Key, TableName)}),\n    TableName;\n\non({prbr, tab2list_raw, DB, Client}, TableName) ->\n    comm:send_local(Client, {DB, tab2list_raw(TableName)}),\n    TableName.\n\n-spec get_entry(any(), state()) -> entry().\nget_entry(Id, TableName) ->\n    case ?PDB:get(TableName, Id) of\n        {}    -> new(Id);\n        Entry -> Entry\n    end.\n\n-spec set_entry(entry(), state()) -> state().\nset_entry(NewEntry, TableName) ->\n    _ = ?PDB:set(TableName, NewEntry),\n    TableName.\n\n-spec delete_entry(state(), entry()) -> db_prbr:db().\ndelete_entry(TableName, Entry) ->\n    ?PDB:delete_entry(TableName, Entry).\n\n-spec get_load(state()) -> non_neg_integer().\nget_load(State) -> ?PDB:get_load(State).\n\n-spec tab2list(state()) -> [{any(),any()}].\ntab2list(State) ->\n    %% without prbr own data\n    Entries = tab2list_raw(State),\n    [ {entry_key(X), entry_val(X)} || X <- Entries].\n\n-spec tab2list_raw_unittest(state()) -> [entry()].\ntab2list_raw_unittest(State) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    tab2list_raw(State).\n\n-spec tab2list_raw(state()) -> [entry()].\ntab2list_raw(State) ->\n    %% with prbr own data\n    ?PDB:tab2list(State).\n\n%% operations for abstract data type entry()\n\n-spec new(any()) -> entry().\nnew(Key) ->\n    new(Key, prbr_bottom).\n\n-spec new(any(), any()) -> entry().\nnew(Key, Val) ->\n    {Key,\n     %% Note: atoms < pids, so this is a good default.\n     _R_Read = pr:new(0, '_'),\n     %% Note: atoms < pids, so this is a good default.\n     _R_Write = pr:new(0, '_'),\n     _Value = Val,\n     fun prbr:noop_write_filter/3\n     }.\n\n-spec entry_key(entry()) -> any().\nentry_key(Entry) -> element(1, Entry).\n%% -spec entry_set_key(entry(), any()) -> entry().\n%% entry_set_key(Entry, Key) -> setelement(2, Entry, Key).\n-spec entry_r_read(entry()) -> pr:pr().\nentry_r_read(Entry) -> element(2, Entry).\n-spec entry_set_r_read(entry(), pr:pr()) -> entry().\nentry_set_r_read(Entry, Round) -> setelement(2, Entry, Round).\n-spec entry_r_write(entry()) -> pr:pr().\nentry_r_write(Entry) -> element(3, Entry).\n-spec entry_set_r_write(entry(), pr:pr()) -> entry().\nentry_set_r_write(Entry, Round) -> setelement(3, Entry, Round).\n-spec entry_val(entry()) -> any().\nentry_val(Entry) -> element(4, Entry).\n-spec entry_set_val(entry(), any()) -> entry().\nentry_set_val(Entry, Value) -> setelement(4, Entry, Value).\n-spec entry_set_last_wf(entry(), prbr:write_filter()) -> entry().\nentry_set_last_wf(Entry, WriteFilter) -> setelement(5, Entry, WriteFilter).\n-spec entry_last_wf(entry()) -> prbr:write_filter().\nentry_last_wf(Entry) -> element(5, Entry).\n\n-spec next_read_round(entry(), any()) -> pr:pr().\nnext_read_round(Entry, ProposerUID) ->\n    LatestSeenRead = pr:get_r(entry_r_read(Entry)),\n    LatestSeenWrite = pr:get_r(entry_r_write(Entry)),\n    pr:new(util:max(LatestSeenRead, LatestSeenWrite) + 1, ProposerUID).\n\n\n\n-spec writable(entry(), pr:pr(), pr:pr(), prbr:write_filter()) ->\n          true | {false, repair_required | denied}.\nwritable(Entry, InRound, OldWriteRound, WF) ->\n    LatestSeenRead = entry_r_read(Entry),\n    LatestSeenWrite = entry_r_write(Entry),\n    InRoundR = pr:get_r(InRound),\n    InRoundId = pr:get_id(InRound),\n    LatestSeenReadR = pr:get_r(LatestSeenRead),\n    LatestSeenReadId = pr:get_id(LatestSeenRead),\n    LatestSeenWriteR = pr:get_r(LatestSeenWrite),\n    LatestSeenWriteId = pr:get_id(LatestSeenWrite),\n    OldWriteR = pr:get_r(OldWriteRound),\n    OldWriteId = pr:get_id(OldWriteRound),\n    IsNotPartialWrite = not is_partial_write(WF),\n    %% make sure that no write filter operations are missing in the\n    %% sequence of writes\n    GaplessWriteSequence = IsNotPartialWrite orelse\n                               (OldWriteR =:= LatestSeenWriteR andalso\n                                    OldWriteId =:= LatestSeenWriteId),\n\n    if ((InRoundR =:= LatestSeenReadR andalso InRoundId =:= LatestSeenReadId)\n            orelse (InRoundR > LatestSeenReadR))\n       andalso (InRoundR > LatestSeenWriteR)\n       andalso GaplessWriteSequence ->\n\n            true;\n       true ->\n            %% If this replica is empty due to node failure, and\n            %% therefore lags behind other replicas, a repair process\n            %% can be started which will send a qread to the other\n            %% replicas to refresh its rounds and value.\n            RepairRequired = ?REPAIR\n                                andalso ?REDUNDANCY =:= replication\n                                andalso not GaplessWriteSequence\n                                andalso LatestSeenWriteR =:= 0,\n            Reason = case RepairRequired of\n                        true -> repair_required;\n                        false -> denied\n                     end,\n\n            {false, Reason}\n    end.\n\n-spec is_partial_write(prbr:write_filter()) -> boolean().\nis_partial_write(Any) -> fun prbr:noop_write_filter/3 =/= Any.\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_atom(redundancy_module) andalso\n        config:cfg_is_bool(prbr_repair_on_write).\n\n-spec tester_create_write_filter(0) -> write_filter().\ntester_create_write_filter(0) -> fun prbr:noop_write_filter/3.\n\n"
  },
  {
    "path": "src/rbr/rbrcseq.erl",
    "content": "% @copyright 2012-2018 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc    Allow a sequence of consensus using a prbr.\n%% @end\n%% @version $Id$\n-module(rbrcseq).\n-author('schintke@zib.de').\n-vsn('$Id:$ ').\n\n%% req_for_retrigger can handle incdelay or noincdelay for retriggering\n%% requests. noincdelay is not used right now, but throwing away logic seems premature\n-dialyzer({no_match, req_for_retrigger/2}).\n\n%%-define(PDB, pdb_ets).\n-define(PDB, pdb).\n-define(REDUNDANCY, (config:read(redundancy_module))).\n-define(READ_RETRY_COUNT, (config:read(read_attempts_without_progress))).\n\n%%-define(TRACE(X,Y), log:pal(\"~p\" X,[self()|Y])).\n%%-define(TRACE(X,Y),\n%%        Name = pid_groups:my_pidname(),\n%%        case Name of\n%%            kv_db -> log:pal(\"~p ~p \" X, [self(), Name | Y]);\n%%            _ -> ok\n%%        end).\n-define(TRACE(X,Y), ok).\n-define(TRACE_PERIOD(X,Y), ok).\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-behaviour(gen_component).\n\n%% api:\n-export([qread/4, qread/5]).\n-export([qwrite/6, qwrite/8]).\n-export([qwrite_fast/8, qwrite_fast/10]).\n-export([get_db_for_id/2]).\n\n-export([check_config/0]).\n-export([start_link/3]).\n-export([init/1, on/2]).\n\n-type state() :: { ?PDB:tableid(),\n                   dht_node_state:db_selector(),\n                   non_neg_integer() %% period this process is in\n                 }.\n\n%% Aggregation of the distributed write state received in round_request\n%% or read replies.\n-record(write_state, {% The highest write round received in replies\n                      highest_write_round :: pr:pr(),\n                      % The number of replies seen with this round\n                      highest_write_count :: non_neg_integer(),\n                      % The write_filter used in this round\n                      highest_write_filter :: prbr:write_filter() | none,\n                      %% Value aggregation based on the seen replies. The content depends\n                      %% on the chosen redundancy strategy. In the case of replication,\n                      %% it is simply equal to the value stored at any acceptor with the\n                      %% highest seen write round. Aggregation behaviour is defined in\n                      %% the respective ?REDUNDANCY module.\n                      value :: any()\n                     }).\n\n%% Aggregator for round_request replies.\n%% rr_replies are received after executing a read without explicit round number\n%% to receive the currently highest rounds from a quorum of replicas.\n-record(rr_replies, {reply_count :: non_neg_integer(),        %% total number of replies recieved\n                     highest_read_count :: non_neg_integer(), %% number of replies seen with highest read round\n                     highest_read_round :: pr:pr(),           %% highest read round received in replies\n                     write_state :: [#write_state{}]\n                    }).\n\n%% Aggregator for read replies.\n%% If qread phase 1 has not found a consistent quorum, a read with an explicit\n%% round number is startet. The replies are aggregated in this record.\n-record(r_replies, {ack_count :: non_neg_integer(),  %% number of acks reveiced\n                    deny_count :: non_neg_integer(), %% number of denies received\n                    write_state :: #write_state{}\n                   }).\n\n%% Aggregator for write replies.\n-record(w_replies, {ack_count :: non_neg_integer(),  %% number of ack reveiced\n                    deny_count :: non_neg_integer(), %% number of denies received\n                    highest_write_round :: pr:pr()   %% highest write round reveiced\n                   }).\n\n-type replies() :: #rr_replies{} | #r_replies{} | #w_replies{}.\n\n-type entry() :: {any(), %% ReqId\n                  any(), %% key of entry keeping track of open requests for this client\n                  any(), %% debug field\n                  non_neg_integer(), %% period of last retriggering / starting\n                  non_neg_integer(), %% period of next retriggering\n                  ?RT:key(), %% key\n                  module(), %% data type\n                  comm:erl_local_pid(), %% client\n                  any(), %% filter (read) or tuple of filters (write)\n                  is_read | any(), %% value to write if entry belongs to write\n                  read | write | denied_write, %% operation type\n                  pr:pr(), %% my round\n                  none | read_retry_info(), %% read retry information\n                  replies() %% maintains replies to check for consistent quorums\n%%% Attention: There is a case that checks the size of this tuple below!!\n                 }.\n\n-type read_retry_info() :: {\n      non_neg_integer(), %% number of retries left\n      non_neg_integer(), %% maximum read round number seen in prev iteration\n      non_neg_integer()  %% maximum write round number seen in prev iteration\n}.\n\n\n-type check_next_step() :: fun((term(), term()) -> term()).\n\n-export_type([check_next_step/0]).\n\n-include(\"gen_component.hrl\").\n\n%% quorum read protocol for consensus sequence\n%%\n%% user definable functions and types for qread and abbreviations:\n%% RF = ReadFilter(dbdata() | no_value_yet) -> read_info().\n%% r = replication degree.\n%%\n%% qread(Client, Key, RF) ->\n%%   % read phase\n%%     r * lookup -> r * dbaccess ->\n%%     r * read_filter(DBEntry)\n%%   collect quorum, select newest (prbr knows that itself) ->\n%%     send newest to client.\n\n%% This variant works on whole dbentries without filtering.\n-spec qread(pid_groups:pidname(), comm:erl_local_pid(), ?RT:key(), module()) -> ok.\nqread(CSeqPidName, Client, Key, DataType) ->\n    RF = fun prbr:noop_read_filter/1,\n    qread(CSeqPidName, Client, Key, DataType, RF).\n\n-spec qread(pid_groups:pidname(), comm:erl_local_pid(), any(), module(), prbr:read_filter()) -> ok.\nqread(CSeqPidName, Client, Key, DataType, ReadFilter) ->\n    Pid = pid_groups:find_a(CSeqPidName),\n\n    ReqMsg  = {qround_request, Client, '_', Key, DataType,\n                ReadFilter, read, _RetriggerAfter = 1},\n    start_request(Pid, ReqMsg),\n    %% the process will reply to the client directly\n    ok.\n\n%% quorum write protocol for consensus sequence\n%%\n%% user definable functions and types for qwrite and abbreviations:\n%% RF = ReadFilter(dbdata() | no_value_yet) -> read_info().\n%% CC = ContentCheck(read_info(), WF, value()) ->\n%%         {true, UI}\n%%       | {false, Reason}.\n%% WF = WriteFilter(old_dbdata(), UI, value()) -> dbdata().\n%% RI = ReadInfo produced by RF\n%% UI = UpdateInfo (data that could be used to update/detect outdated replicas)\n%% r = replication degree\n%%\n%% qwrite(Client, RF, CC, WF, Val) ->\n%%   % read phase\n%%     r * lookup -> r * dbaccess ->\n%%     r * RF(DBEntry) -> read_info();\n%%   collect quorum, select newest read_info() = RI\n%%   % allowed next value? (version outdated for example?)\n%%   CC(RI, WF, Val) -> {IsValid = boolean(), UI}\n%%   if false =:= IsValid => return abort to the client\n%%   if true =:= IsValid =>\n%%   % write phase\n%%     r * lookup -> r * dbaccess ->\n%%     r * WF(OldDBEntry, UI, Val) -> NewDBEntry\n%%   collect quorum of 'written' acks\n%%   inform client on done.\n\n%% if the paxos register is changed concurrently or a majority of\n%% answers cannot be collected, rbrcseq automatically restarts with\n%% the read phase. Either the CC fails then (which informs the client\n%% and ends the protocol) or the operation passes through (or another\n%% retry will happen).\n\n%% This variant works on whole dbentries without filtering.\n-spec qwrite(pid_groups:pidname(),\n             comm:erl_local_pid(),\n             ?RT:key(),\n             module(),\n             fun ((any(), any(), any()) -> {boolean(), any()}), %% CC (Content Check)\n             client_value()) -> ok.\nqwrite(CSeqPidName, Client, Key, DataType, CC, Value) ->\n    RF = fun prbr:noop_read_filter/1,\n    WF = fun prbr:noop_write_filter/3,\n    qwrite(CSeqPidName, Client, Key, DataType, RF, CC, WF, Value).\n\n-spec qwrite_fast(pid_groups:pidname(),\n                  comm:erl_local_pid(),\n                  ?RT:key(),\n                  module(),\n                  fun ((any(), any(), any()) -> {boolean(), any()}), %% CC (Content Check)\n                  client_value(), pr:pr(),\n                  client_value() | prbr_bottom) -> ok.\nqwrite_fast(CSeqPidName, Client, Key, DataType, CC, Value, Round, OldVal) ->\n    RF = fun prbr:noop_read_filter/1,\n    WF = fun prbr:noop_write_filter/3,\n    qwrite_fast(CSeqPidName, Client, Key, DataType, RF, CC, WF, Value, Round, OldVal).\n\n-spec qwrite(pid_groups:pidname(),\n             comm:erl_local_pid(),\n             ?RT:key(),\n             module(),\n             fun ((any()) -> any()), %% read filter\n             fun ((any(), any(), any()) -> {boolean(), any()}), %% content check\n             fun ((any(), any(), any()) -> {any(), any()}), %% write filter\n%%              %% select what you need to read for the operation\n%%              fun ((CustomData) -> ReadInfo),\n%%              %% is it an allowed follow up operation? and what info is\n%%              %% needed to update outdated replicas (could be rather old)?\n%%              fun ((CustomData, ReadInfo,\n%%                    {fun ((ReadInfo, WriteValue) -> CustomData),\n%%                     WriteValue}) -> {boolean(), PassedInfo}),\n%%              %% update the db entry with the given infos, must\n%%              %% generate a valid custom datatype. ReturnValue is included\n%%              %% in qwrite_done message for the caller.\n%%              fun ((PassedInfo, WriteValue) -> {CustomData, ReturnValue}),\n%%              %%module(),\n             client_value()) -> ok.\nqwrite(CSeqPidName, Client, Key, DataType, ReadFilter, ContentCheck,\n       WriteFilter, Value) ->\n    Pid = pid_groups:find_a(CSeqPidName),\n\n    ReqMsg = {qwrite, Client, '_', Key,\n                   DataType, {ReadFilter, ContentCheck, WriteFilter},\n                   Value, _RetriggerAfter = 20},\n    start_request(Pid, ReqMsg),\n    %% the process will reply to the client directly\n    ok.\n\n-spec qwrite_fast(pid_groups:pidname(),\n                  comm:erl_local_pid(),\n                  ?RT:key(),\n                  module(),\n                  fun ((any()) -> any()), %% read filter\n                      fun ((any(), any(), any()) -> {boolean(), any()}), %% content check\n                          fun ((any(), any(), any()) -> any()), %% write filter\n                              client_value(), pr:pr(), client_value() | prbr_bottom)\n-> ok.\nqwrite_fast(CSeqPidName, Client, Key, DataType, ReadFilter, ContentCheck,\n            WriteFilter, Value, Round, OldValue) ->\n    Pid = pid_groups:find_a(CSeqPidName),\n\n    %% Writes need the old write round to ensure every replica has the same\n    %% sequence of applied write filter. A qwrite_fast is only sucessfull\n    %% if no other process has touched the the replica's in the mean time.\n    %% In this case, the round received (parameter Round) from the last\n    %% write is the used write round inrecemented by one.\n    OldWriteRound = pr:new(pr:get_r(Round)-1, pr:get_id(Round)),\n\n    ReqMsg =  {qwrite_fast, Client, '_', Key,\n                          DataType, {ReadFilter, ContentCheck, WriteFilter},\n                          Value, _RetriggerAfter = 20, Round, OldWriteRound,\n                          OldValue},\n    start_request(Pid, ReqMsg),\n    %% the process will reply to the client directly\n    ok.\n\n%% start the request with message to on({request_init... before starting\n%% the first step of the protocol\n-spec start_request(pid_groups:pidname(), any()) -> ok.\nstart_request(CseqPid, Msg) ->\n    comm:send_local(CseqPid, {request_init, _ClientPosInMsg=2,\n                              _OpenReqEntryPlaceHolder=3, Msg}).\n\n%% @doc spawns a rbrcseq, called by the scalaris supervisor process\n-spec start_link(pid_groups:groupname(), pid_groups:pidname(), dht_node_state:db_selector())\n                -> {ok, pid()}.\nstart_link(DHTNodeGroup, Name, DBSelector) ->\n    gen_component:start_link(\n      ?MODULE, fun ?MODULE:on/2, DBSelector,\n      [{pid_groups_join_as, DHTNodeGroup, Name}]).\n\n-spec init(dht_node_state:db_selector()) -> state().\ninit(DBSelector) ->\n    _ = code:ensure_loaded(?REDUNDANCY),\n    case erlang:function_exported(?REDUNDANCY, init, 0) of\n        true ->\n            ?REDUNDANCY:init();\n        _ -> ok\n    end,\n    msg_delay:send_trigger(1, {next_period, 1}),\n    {?PDB:new(?MODULE, [set]), DBSelector, 0}.\n\n-spec on(comm:message(), state()) -> state().\n%% ; ({qread, any(), client_key(), fun ((any()) -> any())},\n%% state()) -> state().\n\n\non({request_init, ClientPosInMsg, OpenReqEntryPos, InitMsg}, State) ->\n    %% will be used to track open entries belonging to this request\n    EntryReg = create_open_entry_register(tablename(State)),\n    %% fill placeholder in msg with EntryReg\n    NewMsg = setelement(OpenReqEntryPos, InitMsg, EntryReg),\n    %% once the request is done, the result is sent to Client. However, cleanup\n    %% is requires to close eventual open entries (happens if proposer receives\n    %% a quorum of success replies from a write through caused by its incomplete\n    %% write). Replaces the client in the message, so that it is sent to the\n    %% proposer's request_cleanup on-handler which in turn will forward the message\n    %% to the client after it is done with its cleanup.\n    Client = element(ClientPosInMsg, NewMsg),\n    InjectedCleanupClient = comm:reply_as(self(), 4,\n                                          {request_cleanup, Client, EntryReg, '_'}),\n    NewMsg2 = setelement(ClientPosInMsg, NewMsg, InjectedCleanupClient),\n    comm:send_local(self(), NewMsg2),\n    State;\n\non({request_cleanup, Client, EntryReg, Result}, State) ->\n    case delete_all_entries(EntryReg, tablename(State),_DeleteOpenEntryReqToo=true) of\n        ok ->\n            comm:send_local(Client, Result);\n        error ->\n            %% Open request register does not exist anymore. This means\n            %% cleanup was already called and the client has received a reply.\n            ok\n    end,\n    State;\n\n%% qread step 1: request the current read/write rounds of + values from replicas.\non({qround_request, Client, EntryReg, Key, DataType, ReadFilter, OpType=read, RetriggerAfter}, State) ->\n    ReadRetryInfo =  {?READ_RETRY_COUNT, _MaxSeenReadRoundNum = 0, _MaxSeenWriteRoundNum = 0},\n    gen_component:post_op({qround_request, Client, EntryReg, Key, DataType, ReadFilter,\n                           OpType, ReadRetryInfo, RetriggerAfter}, State);\n\non({qround_request, Client, EntryReg, Key, DataType, ReadFilter, OpType, RetriggerAfter}, State) ->\n    gen_component:post_op({qround_request, Client, EntryReg, Key, DataType, ReadFilter,\n                           OpType, _ReadRetryInfo=none, RetriggerAfter}, State);\n\non({qround_request, Client, EntryReg, Key, DataType, ReadFilter, OpType, ReadRetryInfo,\n        RetriggerAfter}, State) ->\n    ?TRACE(\"rbrcseq:read step 1: round_request, Client ~p~n\", [Client]),\n    %% assign new reqest-id; (also assign new ReqId when retriggering)\n\n    %% if the caller process may handle more than one request at a\n    %% time for the same key, the pids id has to be unique for each\n    %% request to use the prbr correctly.\n    ReqId = uid:get_pids_uid(),\n\n    ?TRACE(\"rbrcseq:read step 1: round_request ReqId ~p~n\", [ReqId]),\n    %% initiate lookups for replicas(Key) and perform\n    %% rbr reads there apply the content filter to only retrieve the required information\n    This = comm:reply_as(comm:this(), 2, {qround_request_collect, '_'}),\n\n    %% add the ReqId in case we concurrently perform several requests\n    %% for the same key from the same process, which may happen.\n    %% later: retrieve the request id from the assigned round number\n    %% to get the entry from the pdb\n    MyId = {my_id(), ReqId},\n    Dest = pid_groups:find_a(routing_table),\n    DB = db_selector(State),\n\n    %% create local state for the request\n    Entry = entry_new_round_request(qround_request, ReqId, EntryReg, Key, DataType, Client,\n                                    period(State), ReadFilter, RetriggerAfter, OpType,\n                                    ReadRetryInfo),\n    _ = case save_entry(Entry, tablename(State)) of\n            ok ->\n                [ begin\n                    %% let fill in whether lookup was consistent\n                    LookupEnvelope = dht_node_lookup:envelope(4,\n                            {prbr, round_request, DB, '_', This, X,\n                             DataType, MyId, ReadFilter, OpType}),\n                    comm:send_local(Dest,{?lookup_aux, X, 0, LookupEnvelope})\n                  end\n                || X <- ?REDUNDANCY:get_keys(Key) ];\n            error ->\n                %% client has already received an reply.\n                ok\n        end,\n    %% retriggering of the request is done via the periodic dictionary scan\n    %% {next_period, ...}\n\n    State;\n\n%% qread step 2: collect round_request replies (step 1) from replicas.\n%% If a majority replied and it is a consistent quorum (i.e. all received rounds are the same)\n%% we can deliver the read. If not, a qread with explicit round number is started.\non({qround_request_collect,\n    {round_request_reply, Cons, ReceivedReadRound, ReceivedWriteRound, ReadValue, LastWF}}, State) ->\n    ?TRACE(\"rbrcseq:on round_request_collect reply with r_round: ~p~n\", [ReceivedReadRound]),\n\n    {_Round, ReqId} = pr:get_id(ReceivedReadRound),\n    case get_entry(ReqId, tablename(State)) of\n        undefined ->\n            %% drop replies for unknown requests, as they must be outdated as all replies\n            %% run through the same process.\n            State;\n        Entry ->\n            Replies = entry_replies(Entry),\n            {Result, NewReplies, NewRound} =\n                add_rr_reply(Replies, db_selector(State), ReceivedReadRound,\n                             ReceivedWriteRound, ReadValue, entry_optype(Entry), LastWF,\n                             entry_datatype(Entry), entry_filters(Entry), Cons),\n            TEntry = entry_set_my_round(Entry, NewRound),\n            NewEntry = entry_set_replies(TEntry, NewReplies),\n            case Result of\n                false ->\n                    %% no majority replied yet\n                    update_entry(NewEntry, tablename(State)),\n                    State;\n                consistent ->\n                    WriteState = get_highest_seen_write_state(NewReplies#rr_replies.write_state),\n                    %% majority replied and we have a consistent quorum ->\n                    %% we can skip read with explicit round number and deliver directly\n                    trace_mpath:log_info(self(),\n                                          {qread_done, readval,\n                                           WriteState#write_state.highest_write_round,\n                                           WriteState#write_state.value}),\n                    delete_entry(NewEntry, tablename(State)),\n                    inform_client(qread_done, NewEntry,\n                                  WriteState#write_state.highest_write_round,\n                                  WriteState#write_state.value),\n                    State;\n                inconsistent ->\n                    %% majority replied, but we do not have a consistent quorum\n                    {RetryWithoutIncrement, ReadRetryInfo} =\n                        case entry_optype(NewEntry) =:= read of\n                            true ->\n                                %% as this is a read, keep this entry open for lagging behind messages\n                                %% we might get a consistent quorum with the remaining replies, which\n                                %% would save us the need to do a write-through\n                                update_entry(NewEntry, tablename(State)),\n\n                                %% check if read can be retried because we made progress since\n                                %% last read or our attempt number is not depleted\n                                WriteState = get_highest_seen_write_state(NewReplies#rr_replies.write_state),\n                                SeenMaxRR = pr:get_r(NewReplies#rr_replies.highest_read_round),\n                                SeenMaxWR = pr:get_r(WriteState#write_state.highest_write_round),\n                                {RetriesRemaining, PrevMaxRR, PrevMaxWR} =\n                                    entry_get_read_retry_info(NewEntry),\n                                NewReadRetryInfo =\n                                    case SeenMaxRR =< PrevMaxRR andalso\n                                         SeenMaxWR =< PrevMaxWR of\n                                                     true -> %% no progress since last retry\n                                                        {RetriesRemaining - 1, PrevMaxRR,\n                                                         PrevMaxWR};\n                                                     false ->\n                                                        {?READ_RETRY_COUNT,\n                                                         max(PrevMaxRR, SeenMaxRR),\n                                                         max(PrevMaxRR, SeenMaxWR)}\n                                             end,\n                                {RetriesRemaining > 0, NewReadRetryInfo};\n                            false ->\n                                %% writes always modify acceptor states during\n                                %% first phase\n                                delete_entry(NewEntry, tablename(State)),\n                                {false, none}\n                        end,\n\n                    case RetryWithoutIncrement of\n                        true ->\n                            %% retry the read without causing a state change in acceptors\n                            gen_component:post_op({qround_request,\n                                            entry_client(NewEntry),\n                                            entry_openreqentry(NewEntry),\n                                            entry_key(NewEntry),\n                                            entry_datatype(NewEntry),\n                                            entry_filters(NewEntry),\n                                            entry_optype(NewEntry),\n                                            ReadRetryInfo,\n                                            entry_retrigger(NewEntry)}, State);\n                        false ->\n                            %% do a qread with highest received read round + 1\n                            gen_component:post_op({qread,\n                                           entry_client(NewEntry),\n                                           entry_openreqentry(NewEntry),\n                                           entry_key(NewEntry),\n                                           entry_datatype(NewEntry),\n                                           entry_filters(NewEntry),\n                                           entry_retrigger(NewEntry),\n                                           1+pr:get_r(entry_my_round(NewEntry)),\n                                           entry_optype(NewEntry)}, State)\n                    end\n            end\n    end;\n\n%% qread step 3 with explicit read round number\non({qread, Client, OpenReqEntry, Key, DataType, ReadFilter, RetriggerAfter, ReadRound, OpType}, State) ->\n    ?TRACE(\"rbrcseq:on qread ReqId ~p~n\", [ReqId]),\n    %% if the caller process may handle more than one request at a\n    %% time for the same key, the pids id has to be unique for each\n    %% request to use the prbr correctly.\n    ReqId = uid:get_pids_uid(),\n\n    %% initiate lookups for replicas(Key) and perform\n    %% rbr reads in a certain round (work as paxos proposer)\n    %% there apply the content filter to only retrieve the required information\n    This = comm:reply_as(comm:this(), 2, {qread_collect, '_'}),\n\n    %% add the ReqId in case we concurrently perform several requests\n    %% for the same key from the same process, which may happen.\n    %% later: retrieve the request id from the assigned round number\n    %% to get the entry from the pdb\n    MyId = {my_id(), ReqId},\n    Dest = pid_groups:find_a(routing_table),\n    DB = db_selector(State),\n    %% create local state for the request\n    Entry = entry_new_read(qread, ReqId, OpenReqEntry, Key, DataType, Client, period(State),\n                           ReadFilter, RetriggerAfter, OpType),\n    _ = case save_entry(Entry, tablename(State)) of\n        ok ->\n            [ begin\n                %% let fill in whether lookup was consistent\n                LookupEnvelope = dht_node_lookup:envelope(4,\n                    {prbr, read, DB, '_', This, X, DataType, MyId, ReadFilter,\n                     pr:new(ReadRound, MyId)}),\n                comm:send_local(Dest, {?lookup_aux, X, 0, LookupEnvelope})\n              end\n            || X <- ?REDUNDANCY:get_keys(Key) ];\n        error ->\n            %% client has already received a reply\n            ok\n    end,\n\n    %% retriggering of the request is done via the periodic dictionary scan\n    %% {next_period, ...}\n\n    State;\n\n%% qread step 4: a replica replied to read from step 3\n%%               when      majority reached\n%%                  -> finish when consens is stable enough or\n%%                  -> trigger write_through to stabilize an open consens\n%%               otherwise just register the reply.\non({qread_collect,\n    {read_reply, Cons, MyRwithId, Val, SeenWriteRound, SeenLastWF}}, State) ->\n    ?TRACE(\"rbrcseq:on qread_collect read_reply MyRwithId: ~p~n\", [MyRwithId]),\n    %% collect a majority of answers and select that one with the highest\n    %% round number.\n    {_Round, ReqId} = pr:get_id(MyRwithId),\n    case get_entry(ReqId, tablename(State)) of\n        undefined ->\n            %% drop replies for unknown requests, as they must be\n            %% outdated as all replies run through the same process.\n            State;\n        Entry ->\n            Replies = entry_replies(Entry),\n            {Result, NewReplies, NewRound} =\n                add_read_reply(Replies, db_selector(State), MyRwithId, Val, SeenWriteRound,\n                               entry_my_round(Entry), entry_optype(Entry), SeenLastWF,\n                               entry_datatype(Entry), entry_filters(Entry), Cons),\n            TE = entry_set_my_round(Entry, NewRound),\n            NewEntry = entry_set_replies(TE, NewReplies),\n            WriteState = NewReplies#r_replies.write_state,\n            case Result of\n                false ->\n                    update_entry(NewEntry, tablename(State)),\n                    State;\n                true ->\n                    trace_mpath:log_info(self(),\n                                         {qread_done,\n                                          readval, WriteState#write_state.highest_write_round,\n                                          WriteState#write_state.value}),\n                    %% In contrast to retrying the request, in which case we are interestet in\n                    %% the highest seen round number, the second phase (write) must be performed\n                    %% with the same round that was accepted in the first phase.\n                    %% This prevents multiple proposals with the same round number but differing IDs\n                    NewEntry2 = entry_set_my_round(NewEntry, MyRwithId),\n                    inform_client(qread_done, NewEntry2, WriteState#write_state.highest_write_round,\n                                   WriteState#write_state.value),\n                    delete_entry(NewEntry2, tablename(State)),\n                    State;\n                write_through ->\n                    %% in case a consensus was started, but not yet finished,\n                    %% we first have to finish it\n                    trace_mpath:log_info(self(), {qread_write_through_necessary}),\n                    case randoms:rand_uniform(1,5) of\n                        1 ->\n                            %% delete entry, so outdated answers from minority\n                            %% are not considered\n                            delete_entry(NewEntry, tablename(State)),\n                            gen_component:post_op({qread_initiate_write_through,\n                                                   MyRwithId, NewEntry}, State);\n                        3 ->\n                            %% delay a bit\n                            _ = comm:send_local_after(\n                                  15 + randoms:rand_uniform(1,10), self(),\n                                  {qread_initiate_write_through, MyRwithId, NewEntry}),\n                            delete_entry(NewEntry, tablename(State)),\n                            State;\n                        2 ->\n                            delete_entry(NewEntry, tablename(State)),\n                            comm:send_local(self(), {qread_initiate_write_through,\n                                                    MyRwithId, NewEntry}),\n                            State;\n                        4 ->\n                            delete_entry(NewEntry, tablename(State)),\n                            %% retry read\n                            gen_component:post_op({qread,\n                                                   entry_client(NewEntry),\n                                                   entry_openreqentry(NewEntry),\n                                                   entry_key(NewEntry),\n                                                   entry_datatype(NewEntry),\n                                                   entry_filters(NewEntry),\n                                                   entry_retrigger(NewEntry),\n                                                   1+pr:get_r(entry_my_round(NewEntry)),\n                                                   entry_optype(NewEntry)}, State);\n                        5 ->\n                            delete_entry(NewEntry, tablename(State)),\n                            %% retry read\n                            comm:send_local_after(15 + randoms:rand_uniform(1,10), self(),\n                                                  {qread,\n                                                   entry_client(NewEntry),\n                                                   entry_openreqentry(NewEntry),\n                                                   entry_key(NewEntry),\n                                                   entry_datatype(NewEntry),\n                                                   entry_filters(NewEntry),\n                                                   entry_retrigger(NewEntry),\n                                                   1+pr:get_r(entry_my_round(NewEntry)),\n                                                   entry_optype(NewEntry)}),\n                            State\n                        end\n            end\n        end;\n\non({qread_collect, {read_deny, Cons, MyRwithId, LargerRound}}, State) ->\n    {_Round, ReqId} = pr:get_id(MyRwithId),\n    case get_entry(ReqId, tablename(State)) of\n        undefined ->\n            State;\n        Entry ->\n            Replies = entry_replies(Entry),\n            {Result, NewReplies, NewRound} =\n                add_read_deny(Replies, db_selector(State), entry_my_round(Entry),\n                              LargerRound, Cons),\n            TEntry = entry_set_my_round(Entry, NewRound),\n            NewEntry = entry_set_replies(TEntry, NewReplies),\n            case Result of\n                false ->\n                    update_entry(NewEntry, tablename(State)),\n                    State;\n                retry ->\n                    % we can no longer achieve a quorum accept because of a\n                    % concurrent request\n                    NextMsg = {qread,\n                               entry_client(NewEntry),\n                               entry_openreqentry(NewEntry),\n                               entry_key(NewEntry),\n                               entry_datatype(NewEntry),\n                               entry_filters(NewEntry),\n                               entry_retrigger(NewEntry),\n                               1 + pr:get_r(entry_my_round(NewEntry)),\n                               entry_optype(NewEntry)},\n                    delete_entry(NewEntry, tablename(State)),\n\n                    case randoms:rand_uniform(1, 2) of\n                        1 ->\n                            %% retry read immediately\n                            gen_component:post_op(NextMsg, State);\n                        2 ->\n                            %% delay before retry\n                            Delay = 15 + randoms:rand_uniform(1, 10),\n                            comm:send_local_after(Delay, self(), NextMsg),\n                            State\n                    end\n            end\n    end;\n\non({qread_initiate_write_through, RoundTried, ReadEntry}, State) ->\n    ?TRACE(\"rbrcseq:on qread_initiate_write_through ~p~n\", [ReadEntry]),\n    %% if a read_filter was active, we cannot take over the value for\n    %% a write_through.  We then have to retrigger the read without a\n    %% read-filter, but in the end have to reply with a filtered\n    %% value!\n    case entry_filters(ReadEntry) =:= fun prbr:noop_read_filter/1 of\n        true ->\n            %% we are only allowed to try the write once in exactly\n            %% this Round otherwise we may overwrite newer values with\n            %% older ones?! If it fails, we have to restart with the\n            %% read as we observed concurrency happening.\n\n            %% we need a new id to collect the answers of this write\n            %% the client in this new id will be ourselves, so we can\n            %% proceed, when we got enough replies.\n            This = comm:reply_as(\n                     self(), 4,\n                     {qread_write_through_done, ReadEntry, no_filtering, '_'}),\n\n            ReqId = uid:get_pids_uid(),\n            MyId = {my_id(), ReqId},\n\n            %% we only try to re-write a consensus in exactly this\n            %% round without retrying, so having no content check is\n            %% fine here\n            ReadReplies = entry_replies(ReadEntry),\n            WriteState = ReadReplies#r_replies.write_state,\n            ReadVal = WriteState#write_state.value,\n            PreviousWTI = pr:get_wti(WriteState#write_state.highest_write_round),\n            {WTWF, WTUI, WTVal} = %% WT.. means WriteThrough here\n                case PreviousWTI of\n                    none ->\n                        {fun prbr:noop_write_filter/3, none, ReadVal};\n                    {WriteRet, _} ->\n                        WTI = {fun prbr:noop_write_filter/3, WriteRet, ReadVal},\n                        ?TRACE(\"Setting write through write filter ~p\",\n                               [WTI]),\n                        WTI\n                 end,\n            WriteRound = pr:set_wti(RoundTried, PreviousWTI),\n\n            Filters = {fun prbr:noop_read_filter/1,\n                       fun(_,_,_) -> {true, WTUI} end,\n                       WTWF},\n\n            Entry = entry_new_write(write_through, ReqId,\n                                    entry_openreqentry(ReadEntry),\n                                    entry_key(ReadEntry),\n                                    entry_datatype(ReadEntry),\n                                    This,\n                                    period(State),\n                                    Filters, WTVal,\n                                    entry_retrigger(ReadEntry)\n                                    - entry_period(ReadEntry)),\n\n            Collector = comm:reply_as(\n                          comm:this(), 3,\n                          {qread_write_through_collect, ReqId, '_'}),\n\n            Dest = pid_groups:find_a(routing_table),\n            DB = db_selector(State),\n            Keys = ?REDUNDANCY:get_keys(entry_key(Entry)),\n            WTVals = ?REDUNDANCY:write_values_for_keys(Keys,  WTVal),\n            _ = case save_entry(Entry, tablename(State)) of\n                    ok ->\n                        [ begin\n                            %% let fill in whether lookup was consistent\n                            LookupEnvelope =\n                                dht_node_lookup:envelope(\n                                    4,\n                                    {prbr, write, DB, '_', Collector, K,\n                                     entry_datatype(ReadEntry), MyId, WriteRound,\n                                     pr:set_wti(RoundTried, PreviousWTI),\n                                     V, WTUI, WTWF, _IsWriteThrough = true}),\n                            comm:send_local(Dest, {?lookup_aux, K, 0, LookupEnvelope})\n                          end\n                        || {K, V} <- lists:zip(Keys, WTVals) ];\n                    error ->\n                        %% client already received a reply\n                        ok\n                end,\n            State;\n        false ->\n            %% apply the read-filter after the write_through: just\n            %% initiate a read without filtering, which then - if the\n            %% consens is still open - can trigger a repair and will\n            %% reply to us with a full entry, that we can filter\n            %% ourselves before sending it to the original client\n            This = comm:reply_as(\n                     self(), 4,\n                     {qread_write_through_done, ReadEntry, apply_filter, '_'}),\n\n            NextRoundNumber = 1 + pr:get_r(RoundTried),\n            gen_component:post_op({qread, This, entry_openreqentry(ReadEntry),\n               entry_key(ReadEntry), entry_datatype(ReadEntry), fun prbr:noop_read_filter/1,\n               entry_retrigger(ReadEntry) - entry_period(ReadEntry), NextRoundNumber, write},\n              State)\n    end;\n\non({qread_write_through_collect, ReqId,\n    {write_reply, Cons, _Key, Round, NextRound, WriteRet}}, State) ->\n    ?TRACE(\"rbrcseq:on qread_write_through_collect reply ~p~n\", [ReqId]),\n    Entry = get_entry(ReqId, tablename(State)),\n    _ = case Entry of\n        undefined ->\n            %% drop replies for unknown requests, as they must be\n            %% outdated as all replies run through the same process.\n            State;\n        _ ->\n            ?TRACE(\"rbrcseq:on qread_write_through_collect Client: ~p~n\", [entry_client(Entry)]),\n\n            Replies = entry_replies(Entry),\n            {Done, NewReplies, _HigherRound} = add_write_reply(Replies, Round, Cons),\n            NewEntry = entry_set_replies(Entry, NewReplies),\n            case Done of\n                false -> update_entry(NewEntry, tablename(State));\n                true ->\n                    ?TRACE(\"rbrcseq:on qread_write_through_collect infcl: ~p~n\", [entry_client(Entry)]),\n                    ReplyEntry = entry_set_my_round(NewEntry, NextRound),\n                    inform_client(qwrite_done, ReplyEntry, WriteRet),\n                    delete_entry(Entry, tablename(State))\n            end\n    end,\n    State;\n\non({qread_write_through_collect, ReqId,\n    {write_deny, Cons, Key, RoundTried}}, State) ->\n    ?TRACE(\"rbrcseq:on qread_write_through_collect deny ~p~n\", [ReqId]),\n    TableName = tablename(State),\n    Entry = get_entry(ReqId, TableName),\n    _ = case Entry of\n        undefined ->\n            %% drop replies for unknown requests, as they must be\n            %% outdated as all replies run through the same process.\n            State;\n        _ ->\n            ?TRACE(\"rbrcseq:on qread_write_through_collect deny Client: ~p~n\", [entry_client(Entry)]),\n            Replies = entry_replies(Entry),\n            {_Done=true, _NewReplies} = add_write_deny(Replies, RoundTried, Cons),\n\n            %% the first deny we receive requires us to retry the request\n            delete_entry(Entry, TableName),\n\n            %% we want to retry with the read, the original\n            %% request is packed in the client field of the\n            %% entry as we created a reply_as with\n            %% qread_write_through_done The 2nd field of the\n            %% reply_as was filled with the original state\n            %% entry (including the original client and the\n            %% original read filter.\n            {_Pid, Msg1} = comm:unpack_cookie(entry_client(Entry), {whatever}),\n            %% reply_as from qread write through without filtering\n            qread_write_through_done = comm:get_msg_tag(Msg1),\n            UnpackedEntry = element(2, Msg1),\n            UnpackedClient = entry_client(UnpackedEntry),\n\n            %% In case of filters enabled, we packed once more\n            %% to write through without filters and applying\n            %% the filters afterwards. Let's check this by\n            %% unpacking and seeing whether the reply msg tag\n            %% is still a qread_write_through_done. Then we\n            %% have to use the 2nd unpacking to get the\n            %% original client entry.\n            {_Pid2, Msg2} = comm:unpack_cookie(\n                              UnpackedClient, {whatever2}),\n\n            {Client, Filter} =\n                case comm:get_msg_tag(Msg2) of\n                    qread_write_through_done ->\n                        %% we also have to delete this request\n                        %% as no one will answer it.\n                        UnpackedEntry2 = element(2, Msg2),\n                        delete_entry(UnpackedEntry2, TableName),\n                        {entry_client(UnpackedEntry2),\n                         entry_filters(UnpackedEntry2)};\n                    _ ->\n                        {UnpackedClient,\n                         entry_filters(UnpackedEntry)}\n                end,\n            NextRound = 1 + pr:get_r(RoundTried),\n            gen_component:post_op({qread, Client, entry_openreqentry(Entry), Key, entry_datatype(Entry),\n                                   Filter, entry_retrigger(Entry) - entry_period(Entry), NextRound, write},\n              State)\n    end;\n\non({qread_write_through_done, ReadEntry, _Filtering,\n    {qwrite_done, _ReqId, _Round, _Val, _WriteRet}}, State) ->\n    ?TRACE(\"rbrcseq:on qread_write_through_done qwrite_done ~p ~p~n\", [_ReqId, ReadEntry]),\n    %% as we applied a write filter, the actual distributed consensus\n    %% result may be different from the highest paxos version, that we\n    %% collected in the beginning. So we have to initiate the qread\n    %% again to get the latest value.\n\n%%    Old:\n%%    ClientVal =\n%%        case Filtering of\n%%            apply_filter -> F = entry_filters(ReadEntry), F(Val);\n%%            _ -> Val\n%%        end,\n%%    TReplyEntry = entry_set_val(ReadEntry, ClientVal),\n%%    ReplyEntry = entry_set_my_round(TReplyEntry, Round),\n%%    %% log:pal(\"Write through of write request done informing ~p~n\", [ReplyEntry]),\n%%    inform_client(qread_done, ReplyEntry),\n\n    Client = entry_client(ReadEntry),\n    OpenReqEntry = entry_openreqentry(ReadEntry),\n    Key = entry_key(ReadEntry),\n    DataType = entry_datatype(ReadEntry),\n    ReadFilter = entry_filters(ReadEntry),\n    RetriggerAfter = entry_retrigger(ReadEntry) - entry_period(ReadEntry),\n    TriedRound = entry_my_round(ReadEntry),\n\n    gen_component:post_op(\n      {qread, Client, OpenReqEntry, Key, DataType, ReadFilter, RetriggerAfter, 1 + pr:get_r(TriedRound), write},\n      State);\n\non({qread_write_through_done, ReadEntry, Filtering,\n    {qread_done, _ReqId, Round, OldWriteRound, Val}}, State) ->\n    ?TRACE(\"rbrcseq:on qread_write_through_done qread_done ~p ~p~n\", [_ReqId, ReadEntry]),\n    ClientVal =\n        case Filtering of\n            apply_filter -> F = entry_filters(ReadEntry), F(Val);\n            _ -> Val\n        end,\n    Replies = entry_replies(ReadEntry),\n    WriteState = Replies#r_replies.write_state,\n    NewWriteState = WriteState#write_state{value=ClientVal},\n    NewReplies = Replies#r_replies{write_state=NewWriteState},\n    TReplyEntry = entry_set_replies(ReadEntry, NewReplies),\n    ReplyEntry = entry_set_my_round(TReplyEntry, Round),\n\n    inform_client(qread_done, ReplyEntry, OldWriteRound,\n                  NewWriteState#write_state.value),\n\n    State;\n\n%% normal qwrite step 1: preparation and starting read-phase\non({qwrite, Client, OpenReqEntry, Key, DataType, Filters, WriteValue, RetriggerAfter}, State) ->\n    gen_component:post_op({qwrite, Client, OpenReqEntry, Key, DataType,\n                           Filters, WriteValue, RetriggerAfter, incremental}, State);\n\non({qwrite, Client, OpenReqEntry, Key, DataType, Filters, WriteValue, RetriggerAfter, RoundToTry}, State) ->\n    ?TRACE(\"rbrcseq:on qwrite~n\", []),\n    %% assign new reqest-id\n    ReqId = uid:get_pids_uid(),\n    ?TRACE(\"rbrcseq:on qwrite c ~p uid ~p ~n\", [Client, ReqId]),\n\n    %% create local state for the request id, including used filters\n    Entry = entry_new_write(qwrite, ReqId, OpenReqEntry, Key, DataType, Client, period(State),\n                            Filters, WriteValue, RetriggerAfter),\n\n    This = comm:reply_as(self(), 3, {qwrite_read_done, ReqId, '_'}),\n    case save_entry(Entry, tablename(State)) of\n        ok ->\n            case RoundToTry of\n                incremental ->\n                    gen_component:post_op({qround_request, This, OpenReqEntry, Key, DataType,\n                        element(1, Filters), write, _RetriggerAfter = 1}, State);\n                _ ->\n                    gen_component:post_op({qread, This, OpenReqEntry, Key, DataType,\n                        element(1, Filters), _RetriggerAfter = 1, RoundToTry, write}, State)\n            end;\n        error ->\n            %% client has already received reply\n            State\n    end;\n\n%% qwrite step 2: qread is done, we trigger a quorum write in the given Round\non({qwrite_read_done, ReqId,\n    {qread_done, _ReadId, Round, OldWriteRound, ReadValue}}, State) ->\n    ?TRACE(\"rbrcseq:on qwrite_read_done qread_done~n\", []),\n\n    NewState = gen_component:post_op({do_qwrite_fast, ReqId, Round, OldWriteRound, ReadValue}, State),\n    %% check if we should send a learned message to ourselves before executing phase 2\n    %% to prevent applying the same update twice\n    case pr:get_wti(OldWriteRound) of\n        {PrevReturn, PrevProposer} ->\n            case element(1, PrevProposer)  =:= comm:this() of\n                true ->\n                    %% the previous update in the sequence was done by the same proposer -> send learned\n                    %% msg to ourselves to be safe\n                    Msg = element(4, PrevProposer),\n                    NewMsg = setelement(3, Msg,\n                                         {learned_reply, OldWriteRound, Round, PrevReturn}),\n                    gen_component:post_op(NewMsg, NewState);\n                false ->\n                    NewState\n            end;\n        _ ->\n            %% first write in this sequence\n            NewState\n    end;\n\non({qwrite_fast, Client, OpenReqEntry, Key, DataType, Filters = {_RF, _CC, _WF},\n    WriteValue, RetriggerAfter, Round, OldWriteRound, ReadFilterResultValue}, State) ->\n\n    %% create state and ReqId, store it and trigger 'do_qwrite_fast'\n    %% which is also the write phase of a slow write.\n        %% assign new reqest-id\n    ReqId = uid:get_pids_uid(),\n    ?TRACE(\"rbrcseq:on qwrite c ~p uid ~p ~n\", [Client, ReqId]),\n\n    %% create local state for the request id, including used filters\n    Entry = entry_new_write(qwrite, ReqId, OpenReqEntry, Key, DataType, Client, period(State),\n                            Filters, WriteValue, RetriggerAfter),\n\n    case save_entry(Entry, tablename(State)) of\n        ok ->\n            gen_component:post_op({do_qwrite_fast, ReqId, Round, OldWriteRound,\n                                  ReadFilterResultValue}, State);\n        error ->\n            %% client hjas already received reply\n            State\n    end;\n\non({do_qwrite_fast, ReqId, Round, OldWriteRound, OldRFResultValue}, State) ->\n    %% What if ReqId does no longer exist? Can that happen? How?\n    %% a) Lets analyse the paths to do_qwrite_fast:\n    %% b) Lets analyse when entries are removed from the database:\n    Entry = get_entry(ReqId, tablename(State)),\n    MyId = {my_id(), ReqId},\n    _ = case Entry of\n        undefined ->\n          %% drop actions for unknown requests, as they must be\n          %% outdated. The retrigger mechanism may delete entries at\n          %% any time, so we have to be prepared for that.\n          State;\n        _ ->\n          NewEntry = entry_set_debug(Entry, do_qwrite_fast),\n          ContentCheck = element(2, entry_filters(NewEntry)),\n          WriteFilter = element(3, entry_filters(NewEntry)),\n          WriteValue = entry_write_val(NewEntry),\n          DataType = entry_datatype(NewEntry),\n\n          _ = case ContentCheck(OldRFResultValue,\n                                WriteFilter,\n                                WriteValue) of\n              {true, PassedToUpdate} ->\n                %% own proposal possible as next instance in the\n                %% consens sequence\n                This = comm:reply_as(comm:this(), 3, {qwrite_collect, ReqId, '_'}),\n                DB = db_selector(State),\n                Keys = ?REDUNDANCY:get_keys(entry_key(NewEntry)),\n                WrVals = ?REDUNDANCY:write_values_for_keys(Keys,  WriteValue),\n\n                [ begin\n                    %% let fill in whether lookup was consistent\n                    LookupEnvelope =\n                      dht_node_lookup:envelope(\n                        4,\n                        {prbr, write, DB, '_', This, K, DataType, MyId, Round, OldWriteRound,\n                        V, PassedToUpdate, WriteFilter, _IsWriteThrough = false}),\n                    api_dht_raw:unreliable_lookup(K, LookupEnvelope)\n                  end\n                  || {K, V} <- lists:zip(Keys, WrVals)];\n                {false, Reason} = _Err ->\n                  %% own proposal not possible as of content check\n                  comm:send_local(entry_client(NewEntry),\n                            {qwrite_deny, ReqId, Round, OldRFResultValue,\n                            {content_check_failed, Reason}}),\n                  delete_entry(NewEntry, tablename(State))\n                end,\n            State\n        end;\n\n%% qwrite step 3: a replica replied to write from step 2\n%%                when      majority reached, -> finish.\n%%                otherwise just register the reply.\non({qwrite_collect, ReqId,\n    {write_reply, Cons, _Key, Round, NextRound, WriteRet}}, State) ->\n    ?TRACE(\"rbrcseq:on qwrite_collect write_reply~n\", []),\n    Entry = get_entry(ReqId, tablename(State)),\n    _ = case Entry of\n        undefined ->\n            %% drop replies for unknown requests, as they must be\n            %% outdated as all replies run through the same process.\n            State;\n        _ ->\n            Replies = entry_replies(Entry),\n            {Done, NewReplies, IsHigherRound} = add_write_reply(Replies, Round, Cons),\n            NewEntry = case IsHigherRound andalso entry_optype(Entry) =:= denied_write of\n                            true ->\n                                %% If we entered this branch that means we received\n                                %% the first success reply from a writethrough\n                                %% that tries to establish our previous failed\n                                %% write attempt. It is possible that this proposer\n                                %% already retried its write in a new request\n                                %% (howerver, it cannot have started the write phase yet)\n                                %% To prevent that the write is applied twice, delete\n                                %% the newer request. In case that this WT fails\n                                %% this proposer will be notified of it\n                                %% and will then start a new try.\n                                TEntry = entry_set_optype(Entry, write),\n                                delete_newer_entries(TEntry, tablename(State)),\n                                entry_set_replies(TEntry, NewReplies);\n                            false ->\n                                entry_set_replies(Entry, NewReplies)\n                       end,\n            case Done of\n                false -> update_entry(NewEntry, tablename(State));\n                true ->\n                    ReplyEntry = entry_set_my_round(NewEntry, NextRound),\n                    trace_mpath:log_info(self(),\n                                         {qwrite_done,\n                                          value, entry_write_val(ReplyEntry)}),\n                    inform_client(qwrite_done, ReplyEntry, WriteRet),\n\n                    %% a successfull write is always the end of the protocol.\n                    %% immediately clear all open requests associated\n                    %% with the client request, and do not wait unitl this\n                    %% process received the cleanup msg. This must be done\n                    %% to prevent a second write beeing submitted by this proposer\n                    delete_all_entries(entry_openreqentry(NewEntry),\n                                       tablename(State), _DeleteReqEntryToo=false)\n            end\n    end,\n    State;\n\n%% qwrite step 3: a replica replied to write from step 2\n%%                when      majority reached, -> finish.\n%%                otherwise just register the reply.\non({qwrite_collect, ReqId,\n    {write_deny, Cons, _Key, RoundTried}}, State) ->\n    ?TRACE(\"rbrcseq:on qwrite_collect write_deny~n\", []),\n    TableName = tablename(State),\n    Entry = get_entry(ReqId, TableName),\n    case Entry of\n        undefined ->\n            %% drop replies for unknown requests, as they must be\n            %% outdated as all replies run through the same process.\n            State;\n        _ ->\n            Replies = entry_replies(Entry),\n            {Done, NewReplies} = add_write_deny(Replies, RoundTried, Cons),\n            NewEntry = entry_set_replies(Entry, NewReplies),\n            %% If this request was already denied, do not deny again\n            %% (see true case why this might happen)\n            case Done andalso entry_optype(NewEntry) =/= denied_write of\n                false ->\n                    update_entry(NewEntry, TableName),\n                    State;\n                true ->\n                    %% retry\n                    %% log:pal(\"Concurrency detected, retrying~n\"),\n\n                    %% we have to reshuffle retries a bit, so no two\n                    %% proposers using the same rbrcseq process steal\n                    %% each other the token forever.\n                    %% On a random basis, we either reenqueue the\n                    %% request to ourselves or we retry the request\n                    %% directly via a post_op.\n\n                    %% As this happens only when concurrency is detected (not the critical\n                    %% failure- and concurrency-free path), we have the time to choose a\n                    %% random number. This is still faster than using msg_delay or\n                    %% comm:local_send_after() with a random delay.\n                    %% TODO: random is not allowed for proto_sched reproducability...\n                    %% log:log(\"Concurrency retry\"),\n                    UpperLimit = case proto_sched:infected() of\n                                     true -> 2;\n                                     false -> 3\n                                 end,\n\n                    %% The request is retriggered. However, the current entry\n                    %% representing *this* request should NOT be deleted. This\n                    %% write attempt may have written some (albeit the minority of)\n                    %% replicas. A concurrent proposer might see an inconsistent write state,\n                    %% thus triggering a write through. If acceptors vote for this\n                    %% WT-proposal, the original proposer (i.e. this process) is notified.\n                    %% These messages are aggregated using the current req-id. Thus\n                    %% we need to keep it around for now until some write suceeded.\n                    %%\n                    %% To prevent a deny msg that lags behind from triggering\n                    %% this branch again, mark this request as denied\n                    %% Also we need to prevent a retrigger of this request\n                    %% since it already spawns a successor request.\n                    %% Do NOT pass these updates to req_for_retrigger below!\n                    NewEntry2 = entry_set_optype(NewEntry, denied_write),\n                    NewEntry3 = entry_disable_retrigger(NewEntry2),\n                    update_entry(NewEntry3, TableName),\n\n                    NextMsg = {qwrite,\n                               entry_client(Entry),\n                               entry_openreqentry(Entry),\n                               entry_key(Entry),\n                               entry_datatype(Entry),\n                               entry_filters(Entry),\n                               entry_write_val(Entry),\n                               entry_retrigger(NewEntry),\n                               % round to retry in... + 2 to account for incremented\n                               % read round prepared for fast writes\n                               2 + pr:get_r(Replies#w_replies.highest_write_round)\n                              },\n\n                    case randoms:rand_uniform(1, UpperLimit) of\n                        1 ->\n                            %% retry read immediately\n                            gen_component:post_op(NextMsg, State);\n                        2 ->\n                            %% delay before retry\n                            Delay = 15 + randoms:rand_uniform(1, 10),\n                            comm:send_local_after(Delay, self(), NextMsg),\n                            State\n                    end\n            end\n    end\n    %% decide somehow whether a fast paxos or a normal paxos is necessary\n    %% if full paxos: perform qread(self(), Key, ContentReadFilter)\n\n    %% if can propose andalso ContentValidNextStep(Result,\n    %% ContentWriteFilter, Value)?\n    %%   initiate lookups to replica keys and perform rbr:write on each\n\n    %% collect a majority of ok answers or maj -1 of deny answers.\n    %% inform the client\n    %% delete the local state of the request\n\n    %% reissue the write if not enough replies collected (with higher\n    %% round number)\n\n    %% drop replies for unknown requests, as they must be outdated\n    %% as all initiations run through the same process.\n    ;\n\non({qwrite_collect, ReqId,\n    {learned_reply, Round, NextRound, WriteRet}}, State) ->\n    %% receiving this message means that the request was definitely completed\n    Entry = get_entry(ReqId, tablename(State)),\n    _ = case Entry of\n        undefined ->\n            %% drop replies for unknown requests, as they must be\n            %% outdated as all replies run through the same process.\n            State;\n        _ ->\n            Replies = entry_replies(Entry),\n            {_, NewReplies, _} = add_write_reply(Replies, Round, true),\n            NewEntry = entry_set_replies(Entry, NewReplies),\n            ReplyEntry = entry_set_my_round(NewEntry, NextRound),\n            inform_client(qwrite_done, ReplyEntry, WriteRet),\n\n            delete_all_entries(entry_openreqentry(NewEntry),\n                                tablename(State), _DeleteReqEntryToo=false)\n    end,\n    State;\n\n%% periodically scan the local states for long lasting entries and\n%% retrigger them\non({next_period, NewPeriod}, State) ->\n    ?TRACE_PERIOD(\"~p ~p rbrcseq:on next_period~n\", [self(), pid_groups:my_pidname()]),\n    %% reissue (with higher round number) the read if not enough\n    %% replies collected somehow take care of the event and retrigger,\n    %% if it takes to long. Either use msg_delay or record a timestamp\n    %% and periodically revisit all open requests and check whether\n    %% they remain unanswered to long in the system, (could be\n    %% combined with a field of allowed duration which is increased\n    %% per retriggering to catch slow or overloaded systems)\n\n    %% could also be done in another (spawned?) process to avoid\n    %% longer service interruptions in this process?\n\n    %% scan for open requests older than NewPeriod and initiate\n    %% retriggering for them\n    Table = tablename(State),\n    _ = [ retrigger(X, Table, incdelay)\n          || X <- ?PDB:tab2list(Table), is_tuple(X), 14 =:= erlang:tuple_size(X),\n             0 =/= entry_retrigger(X), NewPeriod > entry_retrigger(X)],\n\n    %% re-trigger next next_period\n    msg_delay:send_trigger(1, {next_period, NewPeriod + 1}),\n    set_period(State, NewPeriod).\n\n-spec req_for_retrigger(entry(), incdelay|noincdelay) ->\n                               {qround_request,\n                                Client :: comm:erl_local_pid(),\n                                OpenReqEntry :: any(),\n                                Key :: ?RT:key(),\n                                DataType :: module(),\n                                Filters :: any(),\n                                OpType :: atom(),\n                                Delay :: non_neg_integer()}\n                               | {qwrite,\n                                Client :: comm:erl_local_pid(),\n                                OPenReqEntry :: any(),\n                                Key :: ?RT:key(),\n                                DataType :: module(),\n                                Filters :: any(),\n                                Val :: any(),\n                                Delay :: non_neg_integer()}.\nreq_for_retrigger(Entry, IncDelay) ->\n    RetriggerDelay = case IncDelay of\n                         incdelay -> erlang:max(1, (entry_retrigger(Entry) - entry_period(Entry)) + 1);\n                         noincdelay -> entry_retrigger(Entry)\n                     end,\n    ?ASSERT(erlang:tuple_size(Entry) =:= 14),\n    Filters = entry_filters(Entry),\n    if is_tuple(Filters) -> %% write request\n           {qwrite, entry_client(Entry), entry_openreqentry(Entry),\n            entry_key(Entry), entry_datatype(Entry),\n            entry_filters(Entry), entry_write_val(Entry),\n            RetriggerDelay};\n       true -> %% read request\n           {qround_request, entry_client(Entry), entry_openreqentry(Entry),\n            entry_key(Entry), entry_datatype(Entry), entry_filters(Entry),\n            entry_optype(Entry), RetriggerDelay}\n    end.\n\n-spec retrigger(entry(), ?PDB:tableid(), incdelay|noincdelay) -> ok.\nretrigger(Entry, TableName, IncDelay) ->\n    Request = req_for_retrigger(Entry, IncDelay),\n    ?TRACE(\"Retrigger caused by timeout or concurrency for ~.0p~n\", [Request]),\n    comm:send_local(self(), Request),\n    delete_entry(Entry, TableName),\n    ok.\n\n-spec get_entry(any(), ?PDB:tableid()) -> entry() | {any(), [any()]} | undefined.\nget_entry(ReqId, TableName) ->\n    ?PDB:get(ReqId, TableName).\n\n%% Refreshs the content of an existing entry. ATTENTION: No check if this entry\n%% already exists will be performed.\n-spec update_entry(entry(), ?PDB:tableid()) -> ok.\nupdate_entry(NewEntry, TableName) ->\n    ?PDB:set(NewEntry, TableName).\n\n%% Stores a new entry in ?PDB as long as the open-request-entry\n%% still exists. Does nothing otherwise. If such entry does not exist,\n%% this means that the client has already reveiced a reply and no further\n%% work should be done for this request (Otherwise a write might be\n%% applied twice.)\n-spec save_entry(entry(), ?PDB:tableid()) -> ok | error.\nsave_entry(NewEntry, TableName) ->\n    ReqId = entry_reqid(NewEntry),\n    OpenReqEntry = entry_openreqentry(NewEntry),\n    case get_entry(OpenReqEntry, TableName) of\n        {OpenReqEntry, OpenReqList} ->\n            ?PDB:set({OpenReqEntry, [ReqId | OpenReqList]}, TableName),\n            ?PDB:set(NewEntry, TableName),\n            ok;\n        _ -> error\n    end.\n\n%% Deletes an request entry, as well as its entry in the open-request-entry.\n-spec delete_entry(entry(), ?PDB:tableid()) -> ok.\ndelete_entry(Entry, TableName) ->\n    ReqId = entry_reqid(Entry),\n    OpenReqEntry = entry_openreqentry(Entry),\n    {OpenReqEntry, OpenReqList} = get_entry(OpenReqEntry, TableName),\n    ?PDB:set({OpenReqEntry, lists:delete(ReqId, OpenReqList)}, TableName),\n    ?PDB:delete(ReqId, TableName).\n\n%% Deletes all entries that where added to the open request list after\n%% the given entry.\n-spec delete_newer_entries(entry(), ?PDB:tableid()) -> ok.\ndelete_newer_entries(Entry, TableName) ->\n    {OpenReqEntryId, OpenReqList} = get_entry(entry_openreqentry(Entry), TableName),\n    EntryReqId = entry_reqid(Entry),\n    {ToDelete, ToKeep} = lists:splitwith(fun(E) -> E =/= EntryReqId end, OpenReqList),\n    lists:foreach(fun(E) -> ?PDB:delete(E, TableName) end, ToDelete),\n    ?PDB:set({OpenReqEntryId, ToKeep}, TableName).\n\n%% Deletes all entries included in the open-request-entry. No more can be done.\n%% Return an error if entry is malformed or does not exist\n-spec delete_all_entries(any(), ?PDB:tableid(), boolean()) -> ok | error.\ndelete_all_entries(OpenReqEntryId, TableName, DeleteOpenReqEntryToo) ->\n    case get_entry(OpenReqEntryId, TableName) of\n        {OpenReqEntryId, EntryList} ->\n            lists:foreach(fun(E) -> ?PDB:delete(E, TableName) end, EntryList),\n            case DeleteOpenReqEntryToo of\n                true ->\n                    ?PDB:delete(OpenReqEntryId, TableName);\n                false ->\n                    ?PDB:set({OpenReqEntryId, []}, TableName),\n                    ok\n            end;\n        _ -> error\n    end.\n\n%% Creates entry which will be used to keep track of open requests and returns\n%% its keys by which it can be retrieved.\n-spec create_open_entry_register(?PDB:tableid()) -> any().\ncreate_open_entry_register(TableName) ->\n    RegId = uid:get_pids_uid(),\n    InitVal = [],\n    ?PDB:set({RegId, InitVal}, TableName),\n    RegId.\n\n%% abstract data type to collect quorum read/write replies\n-spec entry_new_round_request(any(), any(), any(), ?RT:key(), module(),\n                     comm:erl_local_pid(), non_neg_integer(), any(),\n                     non_neg_integer(), read | write, read_retry_info())\n                    -> entry().\nentry_new_round_request(Debug, ReqId, EntryReg, Key, DataType, Client, Period, Filter, RetriggerAfter,\n                        OpType, ReadRetryInfo) ->\n    {ReqId, EntryReg, Debug, Period, Period + RetriggerAfter + 20, Key, DataType, Client,\n     Filter, _ValueToWrite = is_read, OpType, _MyRound = pr:new(0,0), ReadRetryInfo, new_rr_replies()}.\n\n-spec entry_new_read(any(), any(), any(), ?RT:key(), module(),\n                     comm:erl_local_pid(), non_neg_integer(), any(),\n                     non_neg_integer(), read | write)\n                    -> entry().\nentry_new_read(Debug, ReqId, EntryReg,  Key, DataType, Client, Period, Filter, RetriggerAfter, OpType) ->\n    {ReqId, EntryReg, Debug, Period, Period + RetriggerAfter + 20, Key, DataType, Client,\n     Filter, _ValueToWrite = is_read, OpType, _MyRound = pr:new(0,0), none, new_read_replies()}.\n\n-spec entry_new_write(any(), any(), any(), ?RT:key(), module(), comm:erl_local_pid(),\n                      non_neg_integer(), tuple(), any(), non_neg_integer())\n                     -> entry().\nentry_new_write(Debug, ReqId, EntryReg, Key, DataType, Client, Period, Filters, Value, RetriggerAfter) ->\n    {ReqId, EntryReg, Debug, Period, Period + RetriggerAfter, Key, DataType, Client,\n     Filters, _ValueToWrite = Value, write, _MyRound = pr:new(0,0), none, new_write_replies()}.\n\n-spec new_rr_replies() -> #rr_replies{}.\nnew_rr_replies() ->\n    #rr_replies{reply_count = 0, highest_read_count = 0,\n                highest_read_round = pr:new(0,0), write_state = []}. %% keep all replies from different rounds\n\n-spec new_read_replies() -> #r_replies{}.\nnew_read_replies() ->\n    #r_replies{ack_count = 0, deny_count = 0, write_state = new_write_state()}.\n\n-spec new_write_state() -> #write_state{}.\nnew_write_state() ->\n    #write_state{highest_write_count = 0, highest_write_round = pr:new(0,0),\n                  highest_write_filter = none, value = new_empty_value}.\n\n-spec new_write_replies() -> #w_replies{}.\nnew_write_replies() ->\n    #w_replies{ack_count = 0, deny_count = 0,\n               highest_write_round = pr:new(0,0)}.\n\n-spec entry_reqid(entry())        -> any().\nentry_reqid(Entry)                -> element(1, Entry).\n-spec entry_openreqentry(entry()) -> any().\nentry_openreqentry(Entry)         -> element(2, Entry).\n-spec entry_set_debug(entry(), any()) -> entry().\nentry_set_debug(Entry,Debug)      -> setelement(3, Entry, Debug).\n-spec entry_period(entry())       -> non_neg_integer().\nentry_period(Entry)               -> element(4, Entry).\n-spec entry_retrigger(entry())    -> non_neg_integer().\nentry_retrigger(Entry)            -> element(5, Entry).\n-spec entry_disable_retrigger(entry()) -> entry().\nentry_disable_retrigger(Entry) -> setelement(5, Entry, 0).\n-spec entry_key(entry())          -> any().\nentry_key(Entry)                  -> element(6, Entry).\n-spec entry_datatype(entry())     -> module().\nentry_datatype(Entry)             -> element(7, Entry).\n-spec entry_client(entry())       -> comm:erl_local_pid().\nentry_client(Entry)               -> element(8, Entry).\n-spec entry_filters(entry())      -> any().\nentry_filters(Entry)              -> element(9, Entry).\n-spec entry_write_val(entry())    -> is_read | any().\nentry_write_val(Entry)            -> element(10, Entry).\n-spec entry_optype(entry())       -> read | write | denied_write.\nentry_optype(Entry)               -> element(11, Entry).\n-spec entry_set_optype(entry(), read | write | denied_write) -> entry().\nentry_set_optype(Entry, OpType)   -> setelement(11, Entry, OpType).\n-spec entry_my_round(entry())     -> pr:pr().\nentry_my_round(Entry)             -> element(12, Entry).\n-spec entry_set_my_round(entry(), pr:pr()) -> entry().\nentry_set_my_round(Entry, Round)  -> setelement(12, Entry, Round).\n-spec entry_get_read_retry_info(entry()) -> read_retry_info() | none.\nentry_get_read_retry_info(Entry) -> element(13, Entry).\n-spec entry_replies(entry())      -> replies().\nentry_replies(Entry)              -> element(14, Entry).\n-spec entry_set_replies(entry(), replies()) -> entry().\nentry_set_replies(Entry, Replies) -> setelement(14, Entry, Replies).\n\n-spec add_rr_reply(#rr_replies{}, dht_node_state:db_selector(),\n                   pr:pr(), pr:pr(), client_value(), atom(), prbr:write_filter(),\n                   module(), any(), boolean())\n                   -> {false | consistent | inconsistent | write_through,\n                       #rr_replies{}, pr:pr()}.\nadd_rr_reply(Replies, _DBSelector, SeenReadRound, SeenWriteRound, Value,\n             OpType, SeenLastWF, Datatype, Filters, _Cons) ->\n    %% increment number of replies received\n    ReplyCount = Replies#rr_replies.reply_count + 1,\n    R1 = Replies#rr_replies{reply_count=ReplyCount},\n\n    %% update number of newest read rounds received\n    PrevMaxReadR = Replies#rr_replies.highest_read_round,\n    R2 =\n        if PrevMaxReadR =:= SeenReadRound ->\n                MaxRCount = R1#rr_replies.highest_read_count + 1,\n                R1#rr_replies{highest_read_count=MaxRCount};\n           PrevMaxReadR < SeenReadRound ->\n                R1#rr_replies{highest_read_count=1,\n                              highest_read_round=SeenReadRound};\n           true ->\n                R1\n        end,\n\n    %% update write rounds and value\n    NewWriteState = update_write_state(Replies#rr_replies.write_state,\n                                       SeenWriteRound, SeenLastWF, Value, Datatype),\n    R3 = R2#rr_replies{write_state=NewWriteState},\n\n    %% If enough replicas have replied, decide on the next action\n    %% to take.\n    {Result, R4} =\n        case {?REDUNDANCY:quorum_accepted(ReplyCount), ?REDUNDANCY:quorum_accepted(ReplyCount-1)} of\n            {true, true} ->\n              %% We have received more replies than necessary for a quorum. This means, that an\n              %% the initial quorum was inconsistent. Collect lagging behind message to mavbe reach\n              %% an consistent quorum, as this would help us to prevent write-throughs in case of a read.\n              case OpType =:= read of\n                true ->\n                  case get_write_state_quorum(NewWriteState) of\n                    none -> {false, R3};\n                    WS ->\n                        ReadFilter =\n                          case Filters of\n                            {RF, _, _}   -> RF;\n                            RF           -> RF\n                          end,\n                        CollectedVal = WS#write_state.value,\n                        ReadValue = ?REDUNDANCY:get_read_value(CollectedVal, ReadFilter),\n                        T1 = R3#rr_replies{write_state=[WS#write_state{value=ReadValue}]},\n                        {consistent, T1}\n                    end;\n                _ -> \n                  {false, R3}\n              end;\n            {true, false} -> %% we have received the minimum number of replies necessary for a quorum\n                HighestWriteState = get_highest_seen_write_state(NewWriteState),\n                NewHighestWF = HighestWriteState#write_state.highest_write_filter,\n                ReadFilter =\n                    case Filters of\n                        {RF, _, _}   -> RF;\n                        RF           -> RF\n                    end,\n\n                ConsReadRounds = ReplyCount =:= R3#rr_replies.highest_read_count,\n                ConsWriteRounds = ReplyCount =:= HighestWriteState#write_state.highest_write_count,\n                ConsQuorum = ConsReadRounds andalso ConsWriteRounds,\n\n                IsRead = OpType =:= read,\n                IsCommutingRead = IsRead andalso is_read_commuting(ReadFilter, NewHighestWF, Datatype),\n\n                if  %% A consistent quorum is the base case for successful delivery\n                    ConsQuorum orelse\n                    %% Reads never interfere with writes. Here, read rounds are inconsistent\n                    %% which means there is a write in progress. But we can be certain that\n                    %% the write is not done because the write rounds are consistent. Thus,\n                    %% this read is concurrent to the write and the 'old' state can be safely\n                    %% delivered.\n                    (IsRead andalso ConsWriteRounds) orelse\n                    %% this is a read operation that commutes with in-progress write\n                    IsCommutingRead ->\n                        %% construct value from received replies (has no effect when\n                        %% data is simply replicated).\n                        CollectedVal = HighestWriteState#write_state.value,\n                        ReadValue = ?REDUNDANCY:get_read_value(CollectedVal, ReadFilter),\n                        T1 = R3#rr_replies{write_state=[HighestWriteState#write_state{value=ReadValue}]},\n                        {consistent, T1};\n\n                    %% For everything else, the default is starting a qread which\n                    %% might receive a consistent state\n                    true ->\n                        {inconsistent, R3}\n                end;\n            {false, _} ->\n                %% no majority yet\n                {false, R3}\n        end,\n    {Result, R4, R4#rr_replies.highest_read_round}.\n\n-spec add_read_reply(#r_replies{}, dht_node_state:db_selector(),\n                     pr:pr(),  client_value(),  pr:pr(), pr:pr(), atom(),\n                     prbr:write_filter(), module(), any(), Consistency::boolean())\n                    -> {Done::boolean() | write_through, #r_replies{}, pr:pr()}.\nadd_read_reply(Replies, _DBSelector, AssignedRound, Val, SeenWriteRound,\n               CurrentRound, OpType, SeenLastWF, Datatype, Filters, _Cons) ->\n    %% either decide on a majority of consistent replies, than we can\n    %% just take the newest consistent value and do not need a\n    %% write_through?\n    %% Otherwise we decide on a consistent quorum (a majority agrees\n    %% on the same version). We ensure this by write_through on odd\n    %% cases.\n    NewAckCount = Replies#r_replies.ack_count + 1,\n    R1 = Replies#r_replies{ack_count=NewAckCount},\n\n    NewWriteState = update_write_state(Replies#r_replies.write_state,\n                                       SeenWriteRound, SeenLastWF, Val, Datatype),\n    R2 = R1#r_replies{write_state=NewWriteState},\n\n\n    {Result, R4} =\n        case ?REDUNDANCY:quorum_accepted(NewAckCount) of\n            true ->\n                %% we have the majority of acks and do not have to wait for\n                %% more replies\n\n                ReadFilter =\n                    case Filters of\n                        {RF, _, _} -> RF;\n                        RF         -> RF\n                    end,\n\n                %% construct read value from replies and update reply aggregator\n                Collected = NewWriteState#write_state.value,\n                Constructed = ?REDUNDANCY:get_read_value(Collected, ReadFilter),\n                R3 = R2#r_replies{write_state=NewWriteState#write_state{value=Constructed}},\n\n                SawConsWriteState = NewWriteState#write_state.highest_write_count =:= NewAckCount,\n                IsCommutingRead = OpType =:= read andalso\n                                      is_read_commuting(ReadFilter,\n                                                        NewWriteState#write_state.highest_write_filter,\n                                                        Datatype),\n\n                if SawConsWriteState orelse IsCommutingRead ->\n                        %% yay, we can deliver the result!\n                        {true, R3};\n                   true ->\n                        %% Due to the inconsistent write state, we must help\n                        %% to establish the current value in a write-through\n                        {write_through, R3}\n                end;\n            _ ->\n                {false, R2}\n        end,\n\n    NewRound = erlang:max(CurrentRound, AssignedRound),\n    {Result, R4, NewRound}.\n\n%%\n-spec get_highest_seen_write_state(nonempty_list(#write_state{})) -> #write_state{}.\nget_highest_seen_write_state(_WriteStateList=[H |_]) -> H.\n\n-spec update_write_state(#write_state{} | [#write_state{}], pr:pr(), prbr:write_filter(),\n                        any(), module()) -> #write_state{} | [#write_state{}].\n% full list of write state entries as sometimes all replies are keepts\nupdate_write_state([], SeenRound, SeenLastWF, SeenValue, Datatype) ->\n    [update_write_state(new_write_state(), SeenRound, SeenLastWF, SeenValue, Datatype)];\nupdate_write_state(_WriteStates=[H|T], SeenRound, SeenLastWF, SeenValue, Datatype) ->\n    CurrentRoundNoWTI = pr:set_wti(H#write_state.highest_write_round, none),\n    SeenRoundNoWTI = pr:set_wti(SeenRound, none),  \n\n    if  CurrentRoundNoWTI =:= SeenRoundNoWTI ->\n          [update_write_state(H, SeenRound, SeenLastWF, SeenValue, Datatype) | T];\n        CurrentRoundNoWTI < SeenRoundNoWTI ->\n          [update_write_state(new_write_state(), SeenRound, SeenLastWF, SeenValue, Datatype) | [H|T]];\n        true ->\n          [H | update_write_state(T, SeenRound, SeenLastWF, SeenValue, Datatype)]\n    end;\n% update single write state entry.\nupdate_write_state(WriteState, SeenRound, SeenLastWF, SeenValue, Datatype) ->\n    %% extract write through info for round comparisons since\n    %% they can be key-dependent if something different than\n    %% replication is used for redundancy\n    CurrentRoundNoWTI = pr:set_wti(WriteState#write_state.highest_write_round, none),\n    SeenRoundNoWTI = pr:set_wti(SeenRound, none),\n\n    CurrentValue = WriteState#write_state.value,\n    if CurrentRoundNoWTI =:= SeenRoundNoWTI ->\n            WriteState#write_state{\n              highest_write_count=WriteState#write_state.highest_write_count+1,\n              value=?REDUNDANCY:collect_read_value(CurrentValue, SeenValue,Datatype)\n             };\n       CurrentRoundNoWTI < SeenRoundNoWTI ->\n            WriteState#write_state{\n              highest_write_round=SeenRound,\n              highest_write_count=1,\n              highest_write_filter=SeenLastWF,\n              value=?REDUNDANCY:collect_newer_read_value(CurrentValue,SeenValue, Datatype)\n             };\n       true ->\n            WriteState#write_state{\n              value=?REDUNDANCY:collect_older_read_value(CurrentValue, SeenValue, Datatype)\n             }\n    end.\n\n%% checks if we collected writes states where a quorum has voted in the same round\n-spec get_write_state_quorum([#write_state{}]) -> none | #write_state{}.\nget_write_state_quorum([]) -> none;\nget_write_state_quorum([H|T]) ->\n  case ?REDUNDANCY:quorum_accepted(H#write_state.highest_write_count) of\n    true -> H;\n    false -> get_write_state_quorum(T)\n  end.\n\n%%\n%% functions for aggregating denies start here\n%%\n-spec add_read_deny(#r_replies{}, dht_node_state:db_selector(), pr:pr(), pr:pr(), boolean())\n                    -> {retry | false, #r_replies{}, pr:pr()}.\nadd_read_deny(Replies, _DBSelector, CurrentRound, ReceivedRound, _Cons) ->\n    %% increment deny count\n    NewDenies = Replies#r_replies.deny_count + 1,\n    R1 = Replies#r_replies{deny_count = NewDenies},\n\n    %% the entries new round will be the maximum received round\n    NewRound = erlang:max(CurrentRound, ReceivedRound),\n\n    %% retry the read if enough acceptors have denied\n    Result = case ?REDUNDANCY:quorum_denied(NewDenies) of\n                     true -> retry;\n                     false -> false\n             end,\n    {Result, R1, NewRound}.\n\n-spec add_write_reply(#w_replies{}, pr:pr(), Consistency::boolean())\n                     -> {Done::boolean(), #w_replies{}, IsHigherRound::boolean()}.\nadd_write_reply(Replies, Round, _Cons) ->\n    RepliesMaxWriteR = Replies#w_replies.highest_write_round,\n    RepliesRoundCmp = {pr:get_r(RepliesMaxWriteR), pr:get_id(RepliesMaxWriteR)},\n    RoundCmp = {pr:get_r(Round), pr:get_id(Round)},\n    {R1, IsHigherRound} =\n        case RoundCmp > RepliesRoundCmp of\n            false -> {Replies, false};\n            true ->\n                %% Running into this case can mean two things:\n                %% 1. This is the first reply received\n                %% 2. This is a reply of a write through which tries to repair the\n                %% partially written value of this request\n                %% If this is case 2, all previous received replies are obsolete.\n                {Replies#w_replies{highest_write_round=Round,\n                                   ack_count=0, deny_count=0}, true}\n        end,\n    R2 =\n        case RoundCmp >= RepliesRoundCmp of\n            %% We must ignore all replies based on the original write (older round), if\n            %% we already received write through replies.\n            false -> R1;\n            true ->\n                NewAckCount = R1#w_replies.ack_count + 1,\n                R1#w_replies{ack_count=NewAckCount}\n        end,\n    Done = ?REDUNDANCY:quorum_accepted(R2#w_replies.ack_count),\n    {Done, R2, IsHigherRound}.\n\n-spec add_write_deny(#w_replies{}, pr:pr(), Consistency::boolean())\n                    -> {Done::boolean(), #w_replies{}}.\nadd_write_deny(Replies, RoundTried, _Cons) ->\n    RepliesMaxWriteR = Replies#w_replies.highest_write_round,\n    RepliesRoundCmp = {pr:get_r(RepliesMaxWriteR), pr:get_id(RepliesMaxWriteR)},\n    RoundCmp = {pr:get_r(RoundTried), pr:get_id(RoundTried)},\n    R1 =\n        case RoundCmp > RepliesRoundCmp of\n            false -> Replies;\n            true ->\n                %% Running into this case can mean two things:\n                %% 1. This is the first reply received\n                %% 2. This is a reply of a write through which tries to repair the\n                %% partially written value of this request\n                %% If this is case 2, all previous received replies are obsolete.\n                Replies#w_replies{highest_write_round=RoundTried,\n                                  ack_count=0,\n                                  deny_count=0}\n        end,\n    R2 =\n        case RoundCmp >= RepliesRoundCmp of\n            %% We must ignore all replies based on the original write (older round), if\n            %% we already received write through replies.\n            false -> R1;\n            true ->\n                NewDenyCount = R1#w_replies.deny_count + 1,\n                R1#w_replies{deny_count=NewDenyCount}\n        end,\n\n    % even one deny requires us to retry. otherwise we risk waiting forever\n    % if one acceptor has failed\n    Done = true,\n    {Done, R2}.\n\n-spec is_read_commuting(prbr:read_filter(), prbr:write_filter(), module()) -> boolean().\nis_read_commuting(ReadFilter, HighestWriteFilterSeen, Datatype) ->\n    %% A WF is considered commuting to a RF iff RF(v) =:= RF(WF(v)) for any v\n    %% To decide if a read can be can be deliverd when seeing inconsistent\n    %% write rounds, it is enough to check if the latest WF of the highest received\n    %% reply does commute with the RF of the current read.\n    %% Proof sketch:\n    %% (v_x, wf_x -> value/write_filter of reply with write round x)\n    %% Assume a set of replies with arbitrary write rounds. h* -> highest round recieved\n    %% Assume wf_h* commutes with current RF (therefore it is not a write through)\n    %% Assume knowing the previous write round in replica of h* -> let this round be c\n    %% - There once was a consistent write quorum in round c\n    %% - All replies with rounds smaller c can be ignored (a newer val was delivered)\n    %% - For every round h greater c but smaller h* it can be shown:\n    %%      - h was and will never be consistent quorum\n    %%      - the previous write round in its replica was also c\n    %%      - therefore wf_h(v_c) =:= v_h\n    %%      - no read with RF rf delivered v_h if rf(v_c) =/= rf(v_h)\n    case erlang:function_exported(Datatype, get_commuting_wf_for_rf, 1) of\n        true ->\n            CommutingWF = Datatype:get_commuting_wf_for_rf(ReadFilter),\n            lists:member(HighestWriteFilterSeen, CommutingWF);\n        false ->\n            false\n    end.\n\n-spec inform_client(qread_done, entry(), pr:pr(), any()) -> ok.\ninform_client(qread_done, Entry, WriteRound, ReadValue) ->\n    comm:send_local(\n      entry_client(Entry),\n      {qread_done,\n       entry_reqid(Entry),\n       entry_my_round(Entry), %% here: round for client's next fast qwrite\n       WriteRound, %% round which client must provide to ensure that full sequence of consensus is guaranteed\n       ReadValue\n      }).\n-spec inform_client(qwrite_done, entry(), any()) -> ok.\ninform_client(qwrite_done, Entry, WriteRet) ->\n    comm:send_local(\n      entry_client(Entry),\n      {qwrite_done,\n       entry_reqid(Entry),\n       entry_my_round(Entry), %% here: round for client's next fast qwrite\n       entry_write_val(Entry),\n       WriteRet\n      }).\n\n%% @doc needs to be unique for this process in the whole system\n-spec my_id() -> any().\nmy_id() ->\n    %% TODO: use the id of the dht_node and later the current lease_id\n    %% and epoch number which should be shorter on the wire than\n    %% comm:this(). Changes in the node id or current lease and epoch\n    %% have to be pushed to this process then.\n    comm:this().\n\n-spec tablename(state()) -> ?PDB:tableid().\ntablename(State) -> element(1, State).\n-spec db_selector(state()) -> dht_node_state:db_selector().\ndb_selector(State) -> element(2, State).\n-spec period(state()) -> non_neg_integer().\nperiod(State) -> element(3, State).\n-spec set_period(state(), non_neg_integer()) -> state().\nset_period(State, Val) -> setelement(3, State, Val).\n\n-spec get_db_for_id(atom(), ?RT:key()) -> {atom(), pos_integer()}.\nget_db_for_id(DBName, Key) ->\n    {DBName, ?RT:get_key_segment(Key)}.\n\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(read_attempts_without_progress).\n\n"
  },
  {
    "path": "src/rbr/replication.erl",
    "content": "% @copyright 2014-2016 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    Implementation of replication as redundancy strategy\n%% @end\n%% @version $Id$\n-module(replication).\n-author('skrzypczak@zib.de').\n-vsn('$Id:$ ').\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([get_keys/1]).\n-export([write_values_for_keys/2]).\n-export([quorum_accepted/1, quorum_denied/1]).\n-export([collect_read_value/3]).\n-export([collect_newer_read_value/3]).\n-export([collect_older_read_value/3]).\n-export([get_read_value/2]).\n-export([skip_write_through/1]).\n-export([notify_orig_learner_on_wt/0]).\n\n%% @doc Returns the replicas of the given key.\n-spec get_keys(?RT:key()) -> [?RT:key()].\nget_keys(Key) ->\n    ?RT:get_replica_keys(Key).\n\n%% @doc Returns a list of values (based on WriteValue) wich should\n%% be written to the Keys passed as argument\n-spec write_values_for_keys([?RT:key()], client_value()) -> [client_value()].\nwrite_values_for_keys(Keys, WriteValue) ->\n    [WriteValue || _K <- Keys].\n\n%% @doc Returns if enough acks for majority have been collected.\n-spec quorum_accepted(integer()) -> boolean().\nquorum_accepted(AccCount) ->\n    R = config:read(replication_factor),\n    quorum:majority_for_accept(R) =< AccCount.\n\n%% @doc Returns if enough denies for majority have been collected.\n-spec quorum_denied(integer()) -> boolean().\nquorum_denied(DeniedCount) ->\n    R = config:read(replication_factor),\n    quorum:majority_for_deny(R) =< DeniedCount.\n\n%% @doc Handles a new read reply of a round newer than the newest round\n%% seen in the past.\n-spec collect_newer_read_value(client_value(), client_value(), module()) -> client_value().\ncollect_newer_read_value(_Collected, NewValue, _DataType) ->\n    NewValue.\n\n%% @doc Handles a new read reply of a previous round.\n-spec collect_older_read_value(client_value(), client_value(), module()) -> client_value().\ncollect_older_read_value(Collected, _NewValue, _DataType) ->\n    Collected.\n\n%% @doc Handles a new read reply of the current round.\n-spec collect_read_value(client_value(), client_value(), module()) -> client_value().\ncollect_read_value(Collected, NewValue, DataType) ->\n    case NewValue of\n        Collected -> Collected;\n        DifferingVal ->\n            ct:log(\"Consistency based on value comparison: ~p <-> ~p (~p)\",\n                   [Collected, NewValue, DataType]),\n            %% if this happens, consistency is probably broken by\n            %% too weak (wrong) content checks...?\n\n            %% unfortunately the following statement is not always\n            %% true: As we also update parts of the value (set\n            %% write lock) it can happen that the write lock is\n            %% set on a quorum inluding an outdated replica, which\n            %% can store a different value than the replicas\n            %% up-to-date. This is not a consistency issue, as the\n            %% tx write the new value to the entry.  But what in\n            %% case of rollback? We cannot safely restore the\n            %% latest value then by just removing the write_lock,\n            %% but would have to actively write the former value!\n\n            %% We use a user defined value selector to chose which\n            %% value is newer. If a data type (leases for example)\n            %% only allows consistent values at any time, this\n            %% callback can check for violation by checking\n            %% equality of all replicas. If a data type allows\n            %% partial write access (like kv_on_cseq for its\n            %% locks) we need an ordering of the values, as it\n            %% might be the case, that a writelock which was set\n            %% on an outdated replica had to be rolled back. Then\n            %% replicas with newest paxos time stamp may exist\n            %% with differing value and we have to chose that one\n            %% with the highest version number for a quorum read.\n            MaxFunModule =\n                case erlang:function_exported(DataType, max, 2) of\n                    true  -> DataType;\n                        %% this datatype has not defined their own max fun\n                        %% therefore use the default util:max\n                     _     -> util\n                end,\n            MaxFunModule:max(Collected, DifferingVal)\n    end.\n\n%% @doc Returns the read value base on all previously collected read replies\n-spec get_read_value(client_value(), prbr:read_filter()) -> client_value().\nget_read_value(ReadValue, _ReadFilter) -> ReadValue.\n\n%% @doc Decide whether to skip a write through based on the read value\n-spec skip_write_through(client_value()) -> boolean().\nskip_write_through(_ReadValue) -> false.\n\n-spec notify_orig_learner_on_wt() -> boolean().\nnotify_orig_learner_on_wt() -> true.\n\n"
  },
  {
    "path": "src/rm_beh.erl",
    "content": "%  @copyright 2008-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Ring maintenance behaviour\n%% @end\n%% @version $Id$\n-module(rm_beh).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-ifndef(have_callback_support).\n-export([behaviour_info/1]).\n-endif.\n\n-ifdef(have_callback_support).\n-include(\"scalaris.hrl\").\n-type state() :: term().\n-type custom_message() :: comm:message().\n\n-callback get_neighbors(state()) -> nodelist:neighborhood().\n-callback init_first() -> ok.\n-callback init(Me::node:node_type(), Pred::node:node_type(), Succ::node:node_type())\n        -> state().\n-callback trigger_action(State::state())\n        -> {ChangeReason::rm_loop:reason(), state()}.\n-callback handle_custom_message(custom_message(), state())\n        -> {ChangeReason::rm_loop:reason(), state()} | unknown_event.\n-callback zombie_node(State::state(), Node::node:node_type())\n        -> {ChangeReason::rm_loop:reason(), state()}.\n-callback fd_notify(State::state(), Event::fd:event(), DeadPid::comm:mypid(), Data::term())\n        -> {ChangeReason::rm_loop:reason(), state()}.\n-callback new_pred(State::state(), NewPred::node:node_type())\n        -> {ChangeReason::rm_loop:reason(), state()}.\n-callback new_succ(State::state(), NewSucc::node:node_type())\n        -> {ChangeReason::rm_loop:reason(), state()}.\n-callback update_node(State::state(), NewMe::node:node_type())\n        -> {ChangeReason::rm_loop:reason(), state()}.\n-callback contact_new_nodes(NewNodes::[node:node_type()]) -> ok.\n-callback trigger_interval() -> pos_integer().\n\n-callback get_web_debug_info(State::state()) -> [{string(), string()}].\n-callback check_config() -> boolean().\n-callback unittest_create_state(Neighbors::nodelist:neighborhood()) -> state().\n\n-else.\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n    [\n     {get_neighbors, 1},\n     {init_first, 0}, {init, 3}, {trigger_action, 1}, {handle_custom_message, 2},\n     {trigger_interval, 0},\n     {zombie_node, 2}, {fd_notify, 4},\n     {new_pred, 2}, {new_succ, 2},\n     {update_node, 2},\n     {contact_new_nodes, 1},\n     {get_web_debug_info, 1},\n     {check_config, 0},\n     {unittest_create_state, 1}\n    ];\nbehaviour_info(_Other) ->\n    undefined.\n-endif.\n"
  },
  {
    "path": "src/rm_beh.hrl",
    "content": "% @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Common types and function specs for ring maintenance implementations.\n%% @end\n%% @version $Id$\n\n-export_type([state/0, custom_message/0]).\n\n-export([init_first/0, init/3, trigger_action/1, handle_custom_message/2,\n         trigger_interval/0,\n         zombie_node/2, fd_notify/4,\n         new_pred/2, new_succ/2,\n         update_node/2, contact_new_nodes/1,\n         get_neighbors/1,\n         get_web_debug_info/1,\n         check_config/0,\n         unittest_create_state/1]).\n"
  },
  {
    "path": "src/rm_chord.erl",
    "content": "% @copyright 2008-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Chord-like ring maintenance\n%% @end\n%% @version $Id$\n-module(rm_chord).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n-behaviour(rm_beh).\n\n-opaque state() :: {Neighbors :: nodelist:neighborhood()}.\n\n% accepted messages of an initialized rm_chord process in addition to rm_loop\n-type(custom_message() ::\n    {rm, get_succlist, Source_Pid::comm:mypid()} |\n    {rm, {rm, node_info_response, NodeDetails::node_details:node_details()}, from_succ | from_node} |\n    {rm, get_succlist_response, Succ::node:node_type(), SuccsSuccList::nodelist:non_empty_snodelist()}).\n\n-define(SEND_OPTIONS, [{channel, prio}, {?quiet}]).\n\n% note include after the type definitions for erlang < R13B04!\n-include(\"rm_beh.hrl\").\n\n-spec get_neighbors(state()) -> nodelist:neighborhood().\nget_neighbors({Neighbors}) ->\n    Neighbors.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Startup\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Nothing to do.\n-spec init_first() -> ok.\ninit_first() ->\n    ok.\n\n%% @doc Initialises the state when rm_loop receives an init_rm message.\n-spec init(Me::node:node_type(), Pred::node:node_type(),\n           Succ::node:node_type()) -> state().\ninit(Me, Pred, Succ) ->\n    Neighborhood = nodelist:new_neighborhood(Pred, Me, Succ),\n    get_successorlist(node:pidX(Succ)),\n    {Neighborhood}.\n\n-spec unittest_create_state(Neighbors::nodelist:neighborhood()) -> state().\nunittest_create_state(Neighbors) ->\n    {Neighbors}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Message Loop\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Message handler when the module is fully initialized.\n-spec handle_custom_message(custom_message(), state())\n        -> {ChangeReason::{unknown} | {node_discovery}, state()} | unknown_event.\nhandle_custom_message({rm, get_succlist, Source_Pid}, {Neighborhood} = State) ->\n    comm:send(Source_Pid, {rm, get_succlist_response,\n                           nodelist:node(Neighborhood),\n                           nodelist:succs(Neighborhood)},\n              ?SEND_OPTIONS),\n    {{unknown}, State};\n\n% got node_details from our successor\nhandle_custom_message({rm, {rm, node_info_response, NodeDetails}, from_succ}, State)  ->\n    SuccsPred = node_details:get(NodeDetails, pred),\n    ThisWithCookie = comm:reply_as(comm:this(), 2, {rm, '_', from_node}),\n    comm:send(node:pidX(SuccsPred),\n              {rm, node_info, ThisWithCookie, [node, is_leaving]},\n              ?SEND_OPTIONS),\n    {{unknown}, State};\n\n% we asked another node we wanted to add for its node object -> now add it\n% (if it is not in the process of leaving the system)\nhandle_custom_message({rm, {rm, node_info_response, NodeDetails}, from_node},\n   {OldNeighborhood} = State)  ->\n    case node_details:get(NodeDetails, is_leaving) of\n        false ->\n            Node = node_details:get(NodeDetails, node),\n            NewNeighborhood = nodelist:add_nodes(OldNeighborhood, [Node],\n                                                 predListLength(), succListLength()),\n            OldSucc = nodelist:succ(OldNeighborhood),\n            NewSucc = nodelist:succ(NewNeighborhood),\n            %% @TODO if(length(NewSuccs) < succListLength() / 2) do something right now\n            case OldSucc =/= NewSucc of\n                true ->\n                    get_successorlist(node:pidX(NewSucc)),\n                    rm_loop:notify_new_pred(node:pidX(NewSucc),\n                                            nodelist:node(NewNeighborhood));\n                false -> ok\n            end,\n            {{node_discovery}, {NewNeighborhood}};\n        true  -> {{unknown}, State}\n    end;\n\nhandle_custom_message({rm, get_succlist_response, Succ, SuccsSuccList},\n   {OldNeighborhood} = State) ->\n\n    NewNeighborhood = nodelist:add_nodes(OldNeighborhood, [Succ | SuccsSuccList],\n                                         predListLength(), succListLength()),\n    OldView = nodelist:to_list(OldNeighborhood),\n    NewView = nodelist:to_list(NewNeighborhood),\n    ViewOrd = fun(A, B) ->\n                      nodelist:succ_ord_node(A, B, nodelist:node(OldNeighborhood))\n              end,\n    {_, _, NewNodes} = util:ssplit_unique(OldView, NewView, ViewOrd),\n    contact_new_nodes(NewNodes),\n    {{unknown}, State};\n\nhandle_custom_message({rm, update_node, Node}, {OldNeighborhood}) ->\n    NewNeighborhood = nodelist:update_ids(OldNeighborhood, [Node]),\n    {{unknown}, {NewNeighborhood}};\n\nhandle_custom_message(_, _State) -> unknown_event.\n\n-spec trigger_action(State::state())\n        -> {ChangeReason::rm_loop:reason(), state()}.\ntrigger_action({Neighborhood} = State) ->\n    % new stabilization interval\n    case nodelist:has_real_succ(Neighborhood) of\n        true ->\n            ThisWithCookie = comm:reply_as(comm:this(), 2, {rm, '_', from_succ}),\n            comm:send(node:pidX(nodelist:succ(Neighborhood)),\n                      {rm, node_info, ThisWithCookie, [pred]},\n                      ?SEND_OPTIONS);\n        _    -> ok\n    end,\n    {{unknown}, State}.\n\n-spec new_pred(State::state(), NewPred::node:node_type())\n        -> {ChangeReason::rm_loop:reason(), state()}.\nnew_pred({OldNeighborhood}, NewPred) ->\n    NewNeighborhood = nodelist:add_node(OldNeighborhood, NewPred,\n                                        predListLength(), succListLength()),\n    {{node_discovery}, {NewNeighborhood}}.\n\n-spec new_succ(State::state(), NewSucc::node:node_type())\n        -> {ChangeReason::rm_loop:reason(), state()}.\nnew_succ({OldNeighborhood}, NewSucc) ->\n    NewNeighborhood = nodelist:add_node(OldNeighborhood, NewSucc,\n                                        predListLength(), succListLength()),\n    {{node_discovery}, {NewNeighborhood}}.\n\n-spec update_node(State::state(), NewMe::node:node_type())\n        -> {ChangeReason::rm_loop:reason(), state()}.\nupdate_node({OldNeighborhood}, NewMe) ->\n    NewNeighborhood = nodelist:update_node(OldNeighborhood, NewMe),\n    % only send pred and succ the new node\n    Message = {rm, update_node, NewMe},\n    Pred = nodelist:pred(NewNeighborhood),\n    Succ = nodelist:succ(NewNeighborhood),\n    comm:send(node:pidX(Succ), Message, ?SEND_OPTIONS),\n    case Pred =/= Succ of\n        true -> comm:send(node:pidX(Pred), Message, ?SEND_OPTIONS);\n        _    -> ok\n    end,\n    {{unknown}, {NewNeighborhood}}.\n\n-spec contact_new_nodes(NewNodes::[node:node_type()]) -> ok.\ncontact_new_nodes(NewNodes) ->\n    % TODO: add a local cache of contacted nodes in order not to contact them again\n    ThisWithCookie = comm:reply_as(comm:this(), 2, {rm, '_', from_node}),\n    case comm:is_valid(ThisWithCookie) of\n        true ->\n            _ = [begin\n                     Msg = {rm, node_info, ThisWithCookie, [node, is_leaving]},\n                     comm:send(node:pidX(Node), Msg, ?SEND_OPTIONS)\n                 end || Node <- NewNodes],\n            ok;\n        false -> ok\n    end.\n\n%% @doc Failure detector reported dead/changed node.\n-spec fd_notify(State::state(), Event::fd:event(), DeadPid::comm:mypid(),\n                Data::term())\n        -> {ChangeReason::rm_loop:reason(), state()}.\nfd_notify({OldNeighborhood}, leave, _DeadPid, OldNode) ->\n    % graceful leave -> do not add as zombie candidate!\n    NewNeighborhood = nodelist:remove(OldNode, OldNeighborhood),\n    % TODO: find replacement?\n    {{graceful_leave, OldNode}, {NewNeighborhood}};\nfd_notify({OldNeighborhood}, jump, _DeadPid, OldNode) ->\n    % remove old node while jumping -> do not add as zombie candidate!\n    FilterFun = fun(N) ->\n                        ?implies(node:same_process(N, OldNode),\n                                 node:is_newer(N, OldNode))\n                end,\n    NewNeighborhood = nodelist:filter(OldNeighborhood, FilterFun),\n    % TODO: find replacement?\n    {{graceful_leave, OldNode}, {NewNeighborhood}};\nfd_notify({OldNeighborhood}, crash, DeadPid, _Reason) ->\n    % crash, i.e. non-graceful leave -> add as zombie candidate\n    FilterFun = fun(N) -> not node:same_process(N, DeadPid) end,\n    NewNeighborhood = nodelist:filter(OldNeighborhood, FilterFun,\n                                      fun dn_cache:add_zombie_candidate/1),\n    % TODO: find replacement?\n    {{node_crashed, DeadPid}, {NewNeighborhood}};\nfd_notify(State, _Event, _DeadPid, _Data) ->\n    {{unknown}, State}.\n\n% dead-node-cache reported dead node to be alive again\n-spec zombie_node(State::state(), Node::node:node_type())\n        -> {ChangeReason::rm_loop:reason(), state()}.\nzombie_node({OldNeighborhood}, Node) ->\n    % this node could potentially be useful as it has been in our state before\n    NewNeighborhood = nodelist:add_node(OldNeighborhood, Node,\n                                        predListLength(), succListLength()),\n    {{node_discovery}, {NewNeighborhood}}.\n\n-spec get_web_debug_info(State::state()) -> [{string(), string()}].\nget_web_debug_info(_State) -> [].\n\n%% @doc Checks whether config parameters of the rm_chord process exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(stabilization_interval_base) and\n    config:cfg_is_greater_than(stabilization_interval_base, 0) and\n\n    config:cfg_is_integer(succ_list_length) and\n    config:cfg_is_greater_than(succ_list_length, 0).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Internal Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @private\n\n%% @doc Sends a message to the remote node's dht_node process asking for\n%%      its list of successors.\n-spec get_successorlist(comm:mypid()) -> ok.\nget_successorlist(RemoteDhtNodePid) ->\n    comm:send(RemoteDhtNodePid, {rm, get_succlist, comm:this()}, ?SEND_OPTIONS).\n\n%% @doc the length of the successor list\n-spec predListLength() -> pos_integer().\npredListLength() -> 1.\n\n%% @doc the length of the successor list\n-spec succListLength() -> pos_integer().\nsuccListLength() -> config:read(succ_list_length).\n\n%% @doc the interval between two stabilization runs\n-spec trigger_interval() -> pos_integer().\ntrigger_interval() -> config:read(stabilization_interval_base) div 1000.\n"
  },
  {
    "path": "src/rm_loop.erl",
    "content": "%  @copyright 2010-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Shared process for the ring maintenance implementations.\n%% @end\n%% @version $Id$\n-module(rm_loop).\n-author('kruber@zib.de').\n\n-include(\"scalaris.hrl\").\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-define(TRACE_SEND(Pid, Msg), ?TRACE(\"[ ~.0p ] to ~.0p: ~.0p~n\", [self(), Pid, Msg])).\n-define(TRACE1(Msg, State),\n        ?TRACE(\"[ ~.0p ]~n  Msg: ~.0p~n  State: ~.0p~n\", [self(), Msg, State])).\n%% -define(TRACE_STATE(OldState, NewState, Reason),\n%%         case element(1, OldState) =/= element(1, NewState) of\n%%             true -> log:pal(\"[ ~.0p ] ~p~n  new Neighbors: ~.0p~n\",\n%%                [self(), Reason, nodelist:to_list(get_neighbors(NewState))]);\n%%             _    -> ok\n%%         end).\n%% -define(TRACE_STATE(OldState, NewState, Reason),\n%%         case element(1, OldState) =/= element(1, NewState) of\n%%             true -> trace_mpath:log_info(self(), {rm_changed, Reason,\n%%                                                   nodelist:to_list(get_neighbors(NewState))});\n%%             _    -> ok\n%%         end).\n-define(TRACE_STATE(OldState, NewState, Reason), ok).\n\n-export([send_trigger/0, init_first/0, init/4, cleanup/1, on/2,\n         leave/1, update_id/1,\n         get_neighbors/1, has_left/1, is_responsible/2,\n         notify_new_pred/2, notify_new_succ/2,\n         notify_slide_finished/1,\n         propose_new_neighbors/1,\n         % received at dht_node, (also) handled here:\n         fd_notify/4, zombie_node/2,\n         % node/neighborhood change subscriptions:\n         subscribe/5, unsubscribe/2,\n         subscribe_dneighbor_change_filter/3,\n         subscribe_dneighbor_change_slide_filter/3,\n         % web debug info:\n         get_web_debug_info/1,\n         % unit tests:\n         unittest_create_state/2]).\n\n-export_type([state/0, reason/0]).\n\n-type reason() :: {slide_finished, pred | succ | none} | % a slide finished\n                  {graceful_leave, Node::node:node_type()} | % the given node is about to leave\n                  {node_crashed, Node::comm:mypid()} | % the given node crashed\n                  {add_subscriber} | % a subscriber was added\n                  {node_discovery} | % a new/changed node was discovered\n                  {update_id_failed} | % a request to update the node's ID failed\n                  {unknown}. % any other reason, e.g. changes during slides\n\n-type subscriber_filter_fun() :: fun((OldNeighbors::nodelist:neighborhood(),\n                                      NewNeighbors::nodelist:neighborhood(),\n                                      Reason::reason()) -> boolean()).\n-type subscriber_exec_fun() :: fun((Subscriber::pid() | null, Tag::any(),\n                                    OldNeighbors::nodelist:neighborhood(),\n                                    NewNeighbors::nodelist:neighborhood(),\n                                    Reason::reason()) -> any()).\n\n-opaque state() ::\n          {RM_State    :: ?RM:state(),\n           HasLeft     :: boolean(),\n           % subscribers to node change events, i.e. node ID changes:\n           SubscrTable :: ets:tid()}.\n\n% accepted messages of an initialized rm_loop process\n-type(message() ::\n    {rm, trigger} |\n    {rm, trigger_action} |\n    {rm, notify_new_pred, NewPred::node:node_type()} |\n    {rm, notify_new_succ, NewSucc::node:node_type()} |\n    {rm, notify_slide_finished, SlideType::pred | succ} |\n    {rm, propose_new_neighbors, NewNodes::[node:node_type(),...]} |\n    {rm, node_info, SourcePid::comm:mypid(), Which::[is_leaving | succlist | succ | predlist | pred | node,...]} |\n    {rm, leave, Tag::jump | leave} |\n    {rm, update_my_id, NewId::?RT:key()} |\n    {web_debug_info, Requestor::comm:erl_local_pid()} |\n    {rm, subscribe, Pid::pid(), Tag::any(), subscriber_filter_fun(), subscriber_exec_fun(), MaxCalls::pos_integer() | inf} |\n    {rm, unsubscribe, Pid::pid(), Tag::any()} |\n    {rm, get_move_state, Pid::pid()}).\n\n-define(SEND_OPTIONS, [{channel, prio}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Public Interface\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Returns the current neighborhood structure.\n-spec get_neighbors(state()) -> nodelist:neighborhood().\nget_neighbors({RM_State, _HasLeft, _SubscrTable}) ->\n    ?RM:get_neighbors(RM_State).\n\n%% @doc Returns whether the current node has already left the ring\n%%      (intermediate state before the node is killed or jumping to another\n%%      ID).\n-spec has_left(state()) -> boolean().\nhas_left({_RM_State, HasLeft, _SubscrTable}) -> HasLeft.\n\n%% @doc Convenience method checking whether the current node is responsible\n%%      for the given key, i.e. has not left and Key is in range.\n%%      Improves performance over two calls in dht_node_state/is_responsible/2.\n-spec is_responsible(Key::intervals:key(), state()) -> boolean().\nis_responsible(Key, {RM_State, HasLeft, _SubscrTable}) ->\n    not HasLeft andalso\n        intervals:in(Key, nodelist:node_range(?RM:get_neighbors(RM_State))).\n\n\n%% @doc Notifies fd-subscribed nodes that the current dht_node is\n%%      going to leave. Will inform the dht_node process (message handled in\n%%      dht_node_move).\n%%      Note: only call this method from inside the dht_node process!\n-spec leave(Tag::jump | leave) -> ok.\nleave(Tag) ->\n    Pid = pid_groups:get_my(dht_node),\n    comm:send_local(Pid, {rm, leave, Tag}).\n\n%% @doc Sends a message to the remote node's dht_node process notifying\n%%      it of a new successor.\n-spec notify_new_succ(Node::comm:mypid(), NewSucc::node:node_type()) -> ok.\nnotify_new_succ(Node, NewSucc) ->\n    comm:send(Node, {rm, notify_new_succ, NewSucc}, ?SEND_OPTIONS).\n\n%% @doc Sends a message to the remote node's dht_node process notifying\n%%      it of a new predecessor.\n-spec notify_new_pred(Node::comm:mypid(), NewPred::node:node_type()) -> ok.\nnotify_new_pred(Node, NewPred) ->\n    comm:send(Node, {rm, notify_new_pred, NewPred}, ?SEND_OPTIONS).\n\n%% @doc Sends a message to the local node's dht_node process notifying\n%%      it of a finished slide.\n-spec notify_slide_finished(SlideType::pred | succ) -> ok.\nnotify_slide_finished(SlideType) ->\n    Pid = pid_groups:get_my(dht_node),\n    comm:send_local(Pid, {rm, notify_slide_finished, SlideType}).\n\n%% @doc Sends a message to the local node's dht_node process notifying\n%%      it of a potential new neighbor.\n-spec propose_new_neighbors(NewNodes::[node:node_type(),...]) -> ok.\npropose_new_neighbors(NewNodes) ->\n    Pid = pid_groups:get_my(dht_node),\n    comm:send_local(Pid, {rm, propose_new_neighbors, NewNodes}).\n\n%% @doc Updates a dht node's id and sends the ring maintenance a message about\n%%      the change.\n%%      Beware: the only allowed node id changes are between the node's\n%%      predecessor and successor!\n-spec update_id(NewId::?RT:key()) -> ok.\nupdate_id(NewId) ->\n    %TODO: do not send message, include directly\n    Pid = pid_groups:get_my(dht_node),\n    comm:send_local(Pid, {rm, update_my_id, NewId}).\n\n%% @doc Filter function for subscriptions that returns true if a\n%%      direct neighbor, i.e. pred, succ or base node, changed.\n-spec subscribe_dneighbor_change_filter(\n        OldNeighbors::nodelist:neighborhood(), NewNeighbors::nodelist:neighborhood(),\n        Reason::reason()) -> boolean().\nsubscribe_dneighbor_change_filter(OldNeighbors, NewNeighbors, _Reason) ->\n    nodelist:node(OldNeighbors) =/= nodelist:node(NewNeighbors) orelse\n        nodelist:pred(OldNeighbors) =/= nodelist:pred(NewNeighbors) orelse\n        nodelist:succ(OldNeighbors) =/= nodelist:succ(NewNeighbors).\n\n%% @doc Filter function for subscriptions that returns true if a\n%%      direct neighbor, i.e. pred, succ or base node, changed or a slide\n%%      operation finished.\n-spec subscribe_dneighbor_change_slide_filter(\n        OldNeighbors::nodelist:neighborhood(), NewNeighbors::nodelist:neighborhood(),\n        Reason::reason()) -> boolean().\nsubscribe_dneighbor_change_slide_filter(_OldNeighbors, _NewNeighbors, {slide_finished, _}) ->\n    true;\nsubscribe_dneighbor_change_slide_filter(OldNeighbors, NewNeighbors, _Reason) ->\n    nodelist:node(OldNeighbors) =/= nodelist:node(NewNeighbors) orelse\n        nodelist:pred(OldNeighbors) =/= nodelist:pred(NewNeighbors) orelse\n        nodelist:succ(OldNeighbors) =/= nodelist:succ(NewNeighbors).\n\n%% @doc Registers the given function to be called when the dht_node changes its\n%%      id. It will get the given Pid and the new node as its parameters.\n-spec subscribe(Pid::pid() | null, Tag::any(), subscriber_filter_fun(), subscriber_exec_fun(),\n                MaxCalls::pos_integer() | inf) -> ok.\nsubscribe(RegPid, Tag, FilterFun, ExecFun, MaxCalls) ->\n    Pid = pid_groups:get_my(dht_node),\n    comm:send_local(Pid, {rm, subscribe, RegPid, Tag, FilterFun, ExecFun, MaxCalls}).\n\n%% @doc Un-registers the given process with the given tag from node change\n%%      updates.\n-spec unsubscribe(Pid::pid() | null, Tag::any()) -> ok.\nunsubscribe(RegPid, Tag) ->\n    Pid = pid_groups:get_my(dht_node),\n    comm:send_local(Pid, {rm, unsubscribe, RegPid, Tag}).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Startup\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Starts the RM trigger.\n-spec send_trigger() -> ok.\nsend_trigger() ->\n    msg_delay:send_trigger(?RM:trigger_interval(), {rm, trigger}).\n\n%% @doc Initializes the rm_loop trigger and whatever the RM-specific code wants.\n%%      NOTE: this is called during dht_node:init/1 and thus is not infected\n%%            with trace_mpath.\n-spec init_first() -> ok.\ninit_first() ->\n    send_trigger(),\n    ?RM:init_first().\n\n%% @doc Initializes the rm_loop state.\n-spec init(Me::node:node_type(), Pred::node:node_type(),\n           Succ::node:node_type(), OldSubscrTable::null | ets:tid()) -> state().\ninit(Me, Pred, Succ, OldSubscrTable) ->\n    % do not wait for the first trigger to arrive here\n    % -> execute trigger action immediately\n    comm:send_local(self(), {rm, trigger_action}),\n    % create the ets table storing the subscriptions\n    if OldSubscrTable =/= null -> SubscrTable = OldSubscrTable;\n       true -> SubscrTable = ets:new(rm_subscribers, [ordered_set, private])\n    end,\n    dn_cache:subscribe(),\n    RM_State = ?RM:init(Me, Pred, Succ),\n    set_failuredetector(?RM:get_neighbors(RM_State)),\n    NewState = {RM_State, false, SubscrTable},\n    ?TRACE_STATE({null, null, null}, NewState, init),\n    NewState.\n\n%% @doc Cleans up before the state is deleted, e.g. removes fd subscriptions\n%%      for a rejoin operation.\n%%      NOTE: only valid when HasLeft is set in the state!\n-spec cleanup(State::state()) -> ok.\ncleanup({RM_State, true, _SubscrTable} = _State) ->\n    Neighborhood = ?RM:get_neighbors(RM_State),\n    View = lists:append(nodelist:preds(Neighborhood),\n                        nodelist:succs(Neighborhood)),\n    Pids = [node:pidX(Node) || Node <- View,\n                               not node:same_process(Node, nodelist:node(Neighborhood))],\n    fd:update_subscriptions(self(), Pids, []),\n    ok.\n\n%% @doc Creates a state() object for a unit test.\n%%      Pre: the process must have joined a group. See pid_groups:join_as/2.\n-spec unittest_create_state(Neighbors::nodelist:neighborhood(), HasLeft::boolean()) -> state().\nunittest_create_state(Neighbors, HasLeft) ->\n    SubscrTable = ets:new(rm_subscribers, [ordered_set, protected]),\n    {?RM:unittest_create_state(Neighbors), HasLeft, SubscrTable}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Message Loop\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Message handler when the rm_loop module is fully initialized.\n-spec on(message() | ?RM:custom_message(), state()) -> state().\non({rm, trigger}, State) ->\n    send_trigger(),\n    RMFun = fun(RM_State) -> ?RM:trigger_action(RM_State) end,\n    update_state(State, RMFun);\n\non({rm, trigger_action}, State) ->\n    RMFun = fun(RM_State) -> ?RM:trigger_action(RM_State) end,\n    update_state(State, RMFun);\n\non({rm, notify_new_pred, NewPred}, State) ->\n    RMFun = fun(RM_State) -> ?RM:new_pred(RM_State, NewPred) end,\n    update_state(State, RMFun);\n\non({rm, notify_new_succ, NewSucc}, State) ->\n    RMFun = fun(RM_State) -> ?RM:new_succ(RM_State, NewSucc) end,\n    update_state(State, RMFun);\n\non({rm, notify_slide_finished, SlideType}, State = {RM_State, _HasLeft, SubscrTable}) ->\n    Neighborhood = ?RM:get_neighbors(RM_State),\n    call_subscribers(Neighborhood, Neighborhood, {slide_finished, SlideType}, SubscrTable),\n    State;\n\non({rm, propose_new_neighbors, NewNodes}, State) ->\n    ?RM:contact_new_nodes(NewNodes),\n    State;\n\non({rm, update_my_id, NewId}, State) ->\n    Neighborhood = ?RM:get_neighbors(element(1, State)),\n    OldMe = nodelist:node(Neighborhood),\n    case node:id(OldMe) of\n        NewId -> State;\n        _ ->\n            NewMe = node:update_id(OldMe, NewId),\n            % note: nodelist can't update the base node if the new id is not\n            % between pred id and succ id\n            try begin\n                    RMFun = fun(RM_State) -> ?RM:update_node(RM_State, NewMe) end,\n                    update_state(State, RMFun)\n                end\n            catch\n                throw:(Reason = new_node_not_in_pred_succ_interval) ->\n                    log:log(error, \"[ RM ] can't update dht node ~w with id ~w (pred=~w, succ=~w): ~.0p\",\n                            [nodelist:node(Neighborhood), NewId,\n                             nodelist:pred(Neighborhood),\n                             nodelist:succ(Neighborhood),\n                             Reason]),\n                    update_state(State, fun(RM_State) -> {{update_id_failed}, RM_State} end)\n            end\n    end;\n\non({rm, node_info, SourcePid, Which}, {RM_State, HasLeft, _SubscrTable} = State) ->\n    Neighborhood = ?RM:get_neighbors(RM_State),\n    ExtractValuesFun =\n        fun(Elem, NodeDetails) ->\n                Value =\n                    case Elem of\n                        is_leaving  -> HasLeft;\n                        succlist    -> nodelist:succs(Neighborhood);\n                        succ        -> nodelist:succ(Neighborhood);\n                        predlist    -> nodelist:preds(Neighborhood);\n                        pred        -> nodelist:pred(Neighborhood);\n                        node        -> nodelist:node(Neighborhood)\n                    end,\n                node_details:set(NodeDetails, Elem, Value)\n        end,\n    NodeDetails = lists:foldl(ExtractValuesFun, node_details:new(), Which),\n    comm:send(SourcePid, {rm, node_info_response, NodeDetails}, ?SEND_OPTIONS),\n    State;\n\non({rm, leave, Tag}, {RM_State, _HasLeft, SubscrTable}) ->\n    Neighborhood = ?RM:get_neighbors(RM_State),\n    Me = nodelist:node(Neighborhood),\n    SupDhtNode = pid_groups:get_my(sup_dht_node),\n    fd:report(Tag, sup:sup_get_all_children(SupDhtNode), Me),\n    % also update the pred in the successor:\n    Pred = nodelist:pred(Neighborhood),\n    notify_new_pred(node:pidX(nodelist:succ(Neighborhood)), Pred),\n    % msg to dht_node to continue the slide:\n    comm:send_local(self(), {move, node_leave}),\n    {RM_State, true, SubscrTable};\n\n%% requests the move state of the rm, e.g. before rejoining the ring\non({rm, get_move_state, Pid}, {_RM_State, _HasLeft, SubscrTable} = State) ->\n    MoveState = [{subscr_table, SubscrTable}],\n    comm:send_local(Pid, {get_move_state_response, MoveState}),\n    State;\n\n%% add Pid to the node change subscriber list\non({rm, subscribe, Pid, Tag, FilterFun, ExecFun, MaxCalls}, {RM_State, _HasLeft, SubscrTable} = State) ->\n    SubscrTuple = {{Pid, Tag}, FilterFun, ExecFun, MaxCalls},\n    ets:insert(SubscrTable, SubscrTuple),\n    % check if the condition is already met:\n    Neighborhood = ?RM:get_neighbors(RM_State),\n    call_subscribers_check(Neighborhood, Neighborhood, {add_subscriber}, SubscrTuple, SubscrTable),\n    State;\n\non({rm, unsubscribe, Pid, Tag}, {_RM_State, _HasLeft, SubscrTable} = State) ->\n    ets:delete(SubscrTable, {Pid, Tag}),\n    State;\n\non(Message, {RM_State, HasLeft, SubscrTable} = OldState) ->\n    % similar to update_state/2 but handle unknown_event differently\n    OldNeighborhood = ?RM:get_neighbors(RM_State),\n    case ?RM:handle_custom_message(Message, RM_State) of\n        unknown_event ->\n            log:log(error, \"unknown message: ~.0p~n in Module: ~p and handler ~p~n in State ~.0p\",\n                    [Message, ?MODULE, on, OldState]),\n            OldState;\n        {Reason, NewRM_State}   ->\n            NewNeighborhood = ?RM:get_neighbors(NewRM_State),\n            call_subscribers(OldNeighborhood, NewNeighborhood, Reason, SubscrTable),\n            update_failuredetector(OldNeighborhood, NewNeighborhood, null),\n            NewState = {NewRM_State, HasLeft, SubscrTable},\n            ?TRACE_STATE(RM_State, NewState, Reason),\n            NewState\n    end.\n\n% failure detector reported dead node\n-spec fd_notify(State::state(), Event::fd:event(), DeadPid::comm:mypid(),\n                Data::term()) -> state().\nfd_notify(State, Event, DeadPid, Data) ->\n    RMFun = fun(RM_State) -> ?RM:fd_notify(RM_State, Event, DeadPid, Data) end,\n    % note: only a crashed pid is unsubscribed by fd itself!\n    NewState = update_state(State, RMFun, ?IIF(Event =:= crash, DeadPid, null)),\n    % do rrepair in case of non-graceful leaves only\n    case config:read(rrepair_after_crash) andalso\n             Event =:= crash of\n        true ->\n            OldPred = nodelist:pred(?RM:get_neighbors(element(1, State))),\n            NewNeighb = ?RM:get_neighbors(element(1, NewState)),\n            NewPred = nodelist:pred(NewNeighb),\n            case NewPred of\n                OldPred -> ok;\n                _       ->\n                    case intervals:in(node:id(OldPred), nodelist:node_range(NewNeighb)) of\n                        true ->\n                            CrashInterval = node:mk_interval_between_nodes(NewPred, OldPred),\n                            case pid_groups:get_my(rrepair) of\n                                failed -> ok;\n                                Pid ->\n                                    comm:send_local(\n                                      Pid,\n                                      {request_resolve, {interval_upd_my, CrashInterval}, []})\n                            end;\n                        false ->\n                            ok\n                    end\n            end;\n        false -> ok\n    end,\n    NewState.\n\n% dead-node-cache reported dead node to be alive again\n-spec zombie_node(State::state(), Node::node:node_type()) -> state().\nzombie_node(State, Node) ->\n    RMFun = fun(RM_State) -> ?RM:zombie_node(RM_State, Node) end,\n    update_state(State, RMFun).\n\n-spec get_web_debug_info(State::state()) -> [{string(), string()}].\nget_web_debug_info({RM_State, _HasLeft, SubscrTable}) ->\n    Neighborhood = ?RM:get_neighbors(RM_State),\n    Preds = [{\"preds:\", \"\"} | make_indexed_nodelist(nodelist:preds(Neighborhood))],\n    Succs = [{\"succs:\", \"\"} | make_indexed_nodelist(nodelist:succs(Neighborhood))],\n    PredsSuccs = lists:append(Preds, Succs),\n    RM_Info = ?RM:get_web_debug_info(RM_State),\n    Subscribers = [begin\n                       case Pid of\n                           null -> {\"null\", Tag};\n                           _    -> {pid_groups:pid_to_name(Pid), Tag}\n                       end\n                   end\n                   || {{Pid, Tag}, _FilterFun, _ExecFun} <- ets:tab2list(SubscrTable)],\n    [{\"rm_state:\", \"\"},\n     {\"algorithm\", webhelpers:safe_html_string(\"~p\", [?RM])},\n     {\"self\", webhelpers:safe_html_string(\"~p\", [nodelist:node(Neighborhood)])},\n     {\"nc_subscr\", webhelpers:safe_html_string(\"~p\", [Subscribers])} |\n         lists:append(PredsSuccs, RM_Info)].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Internal Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Calls RMFun (which may update the Neighborhood), then calls all\n%%      subscribers and updates the failure detector if necessary.\n-spec update_state(OldState::state(), RMFun::fun((?RM:state()) -> {reason(), ?RM:state()}))\n        -> NewState::state().\nupdate_state(OldState, RMFun) ->\n    update_state(OldState, RMFun, null).\n\n-spec update_state(OldState::state(), RMFun::fun((?RM:state()) -> {reason(), ?RM:state()}),\n                   CrashedPid::comm:mypid() | null) -> NewState::state().\nupdate_state({OldRM_State, HasLeft, SubscrTable} = _OldState, RMFun, CrashedPid) ->\n    OldNeighborhood = ?RM:get_neighbors(OldRM_State),\n    {Reason, NewRM_State} = RMFun(OldRM_State),\n    NewNeighborhood = ?RM:get_neighbors(NewRM_State),\n    call_subscribers(OldNeighborhood, NewNeighborhood, Reason, SubscrTable),\n    update_failuredetector(OldNeighborhood, NewNeighborhood, CrashedPid),\n    NewState = {NewRM_State, HasLeft, SubscrTable},\n    ?TRACE_STATE(_OldState, NewState, Reason),\n    NewState.\n\n% @doc Subscribe all PIDs in the neighborhood with the failuredetector.\n-spec set_failuredetector(Neighborhood::nodelist:neighborhood()) -> ok.\nset_failuredetector(Neighborhood) ->\n    [_ | View] = nodelist:to_list(Neighborhood),\n    NewPids = [node:pidX(Node) || Node <- View],\n    fd:subscribe(self(), NewPids).\n\n% @doc Check if change of failuredetector is necessary and subscribe the new\n%%     nodes' pids.\n-spec update_failuredetector(OldNeighborhood::nodelist:neighborhood(),\n                             NewNeighborhood::nodelist:neighborhood(),\n                             CrashedPid::comm:mypid() | null) -> ok.\nupdate_failuredetector(OldNeighborhood, NewNeighborhood, CrashedPid) ->\n    % Note: nodelist:to_list/1 would provide similar functionality to determine\n    % the view but at a higher cost and we need neither unique nor sorted lists.\n    OldView = lists:append(nodelist:preds(OldNeighborhood),\n                           nodelist:succs(OldNeighborhood)),\n    NewView = lists:append(nodelist:preds(NewNeighborhood),\n                           nodelist:succs(NewNeighborhood)),\n    OldNodePid = node:pidX(nodelist:node(OldNeighborhood)),\n    NewNodePid = node:pidX(nodelist:node(NewNeighborhood)),\n    OldPids = [node:pidX(Node) || Node <- OldView,\n                                  not node:same_process(Node, OldNodePid),\n                                  % note: crashed pid already unsubscribed by fd, do not unsubscribe again\n                                  not node:same_process(Node, CrashedPid)],\n    NewPids = [node:pidX(Node) || Node <- NewView,\n                                  not node:same_process(Node, NewNodePid)],\n    fd:update_subscriptions(self(), OldPids, NewPids),\n    ok.\n\n%% @doc Inform the dht_node of a new neighborhood.\n-spec call_subscribers(OldNeighbors::nodelist:neighborhood(),\n        NewNeighbors::nodelist:neighborhood(), Reason::reason(),\n        SubscrTable::ets:tid()) -> ok.\ncall_subscribers(OldNeighborhood, NewNeighborhood, Reason, SubscrTable) ->\n    call_subscribers_iter(OldNeighborhood, NewNeighborhood, Reason, SubscrTable,\n                          ets:first(SubscrTable)).\n\n%% @doc Iterates over all susbcribers and calls their subscribed functions.\n-spec call_subscribers_iter(OldNeighbors::nodelist:neighborhood(),\n        NewNeighbors::nodelist:neighborhood(), Reason::reason(), SubscrTable::ets:tid(),\n        CurrentKey::{Pid::pid() | null, Tag::any()} | '$end_of_table') -> ok.\ncall_subscribers_iter(_OldNeighborhood, _NewNeighborhood, _Reason,\n                      _SubscrTable, '$end_of_table') ->\n    ok;\ncall_subscribers_iter(OldNeighborhood, NewNeighborhood, Reason, SubscrTable, CurrentKey) ->\n    % assume the key exists (it should since we are iterating over the table!)\n    [SubscrTuple] = ets:lookup(SubscrTable, CurrentKey),\n    call_subscribers_check(OldNeighborhood, NewNeighborhood, Reason, SubscrTuple, SubscrTable),\n    call_subscribers_iter(OldNeighborhood, NewNeighborhood, Reason, SubscrTable,\n                          ets:next(SubscrTable, CurrentKey)).\n\n%% @doc Checks whether FilterFun for the current subscription tuple is true\n%%      and executes ExecFun. Unsubscribes the tuple, if ExecFun has been\n%%      called MaxCalls times.\n-spec call_subscribers_check(OldNeighbors::nodelist:neighborhood(),\n        NewNeighbors::nodelist:neighborhood(), Reason::reason(),\n        {{Pid::pid() | null, Tag::any()}, FilterFun::subscriber_filter_fun(),\n         ExecFun::subscriber_exec_fun(), MaxCalls::pos_integer() | inf},\n        SubscrTable::ets:tid()) -> ok.\ncall_subscribers_check(OldNeighborhood, NewNeighborhood, Reason,\n        {{Pid, Tag}, FilterFun, ExecFun, MaxCalls}, SubscrTable) ->\n    case FilterFun(OldNeighborhood, NewNeighborhood, Reason) of\n        true -> ExecFun(Pid, Tag, OldNeighborhood, NewNeighborhood, Reason),\n                case MaxCalls of\n                    inf -> ok;\n                    1   -> ets:delete(SubscrTable, {Pid, Tag}); % unsubscribe\n                    _   -> % subscribe with new max\n                           ets:insert(SubscrTable, {{Pid, Tag}, FilterFun, ExecFun, MaxCalls - 1})\n                end;\n        _    -> ok\n    end.\n\n%% @doc Helper for the web_debug_info handler. Converts the given node list to\n%%      an indexed node list containing string representations of the nodes.\n-spec make_indexed_nodelist(NodeList::[node:node_type()]) -> [{Index::string(), Node::string()}].\nmake_indexed_nodelist(NodeList) ->\n    IndexedList = lists:zip(lists:seq(1, length(NodeList)), NodeList),\n    [{webhelpers:safe_html_string(\"~p\", [Index]),\n      webhelpers:safe_html_string(\"~p\", [Node])} || {Index, Node} <- IndexedList].\n"
  },
  {
    "path": "src/rm_tman.erl",
    "content": "%  @copyright 2009-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Christian Hennig <hennig@zib.de>\n%% @doc    T-Man ring maintenance\n%% @end\n%% @reference Mark Jelasity, Ozalp Babaoglu. T-Man: Gossip-Based Overlay\n%% Topology Management. Engineering Self-Organising Systems 2005:1-15\n%% @version $Id$\n-module(rm_tman).\n-author('hennig@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n-behaviour(rm_beh).\n\n-opaque state() :: rm_tman_state:state().\n\n-define(FIXMODRN(NEIGHBORS), case modr:is_enabled() of true -> modr:fix_neighborhood(NEIGHBORS); false -> NEIGHBORS end).\n-define(FIXMODRS(STATE), case modr:is_enabled() of true -> modr:fix_state(STATE); false -> STATE end).\n\n% accepted messages of an initialized rm_tman process in addition to rm_loop\n-type(custom_message() ::\n    {rm, once, {cy_cache, Cache::[node:node_type()]}} |\n    {rm, {cy_cache, Cache::[node:node_type()]}} |\n    {rm, node_info_response, NodeDetails::node_details:node_details()} |\n    {rm, buffer, OtherNeighbors::nodelist:neighborhood(), RequestPredsMinCount::non_neg_integer(), RequestSuccsMinCount::non_neg_integer()} |\n    {rm, buffer_response, OtherNodes::nodelist:non_empty_snodelist()}).\n\n-define(SEND_OPTIONS, [{channel, prio}, {?quiet}]).\n\n% note include after the type definitions for erlang < R13B04!\n-include(\"rm_beh.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Startup\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Initializes the cyclon cache retrieval \"trigger\" (un-infected).\n-spec init_first() -> ok.\ninit_first() ->\n    gossip_cyclon:get_subset_rand(1, comm:reply_as(self(), 2, {rm, '_'}),\n                                  config:read(tman_cyclon_interval)),\n    ok.\n\n%% @doc Initialises the state when rm_loop receives an init_rm message.\n-spec init(Me::node:node_type(), Pred::node:node_type(),\n           Succ::node:node_type()) -> rm_tman_state:state().\ninit(Me, Pred, Succ) ->\n    Neighborhood = nodelist:new_neighborhood(Pred, Me, Succ),\n    % ask cyclon once (a repeating trigger is already started in init_first/0)\n    gossip_cyclon:get_subset_rand(1, comm:reply_as(self(), 3, {rm, once, '_'})),\n    % start by using all available nodes reported by cyclon\n    RandViewSize = config:read(gossip_cyclon_cache_size),\n    rm_tman_state:init(Neighborhood, RandViewSize, [], true).\n\n-spec unittest_create_state(Neighbors::nodelist:neighborhood()) -> rm_tman_state:state().\nunittest_create_state(Neighbors) ->\n    rm_tman_state:init(Neighbors, 1, [], true).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Message Loop (custom messages not already handled by rm_loop:on/2)\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Message handler when the module is fully initialized.\n-spec handle_custom_message(custom_message(), rm_tman_state:state())\n        -> {ChangeReason::{unknown} | {node_discovery}, rm_tman_state:state()} | unknown_event.\n% got empty cyclon cache\nhandle_custom_message({rm, once, {cy_cache, [] = NewCache}}, State) ->\n    % loop with msg_delay until a non-empty cache is received\n    gossip_cyclon:get_subset_rand(1, comm:reply_as(self(), 3, {rm, once, '_'}), 0),\n    add_cyclon_cache(NewCache, State);\nhandle_custom_message({rm, once, {cy_cache, NewCache}}, State) ->\n    add_cyclon_cache(NewCache, State);\n\n% got cyclon cache (as part of a repeating call)\nhandle_custom_message({rm, {cy_cache, NewCache}}, State) ->\n    {ChangeReason, NewState} = add_cyclon_cache(NewCache, State),\n    NewRandViewSize = rm_tman_state:get_randview_size(NewState),\n    % trigger new cyclon cache request\n    gossip_cyclon:get_subset_rand(NewRandViewSize, comm:reply_as(self(), 2, {rm, '_'}),\n                                  config:read(tman_cyclon_interval)),\n    {ChangeReason, NewState};\n\n% got shuffle request\nhandle_custom_message({rm, buffer, OtherNeighbors, RequestPredsMinCount, RequestSuccsMinCount},\n                      State) ->\n    Cache = rm_tman_state:get_cache(State),\n    RandViewSize = rm_tman_state:get_randview_size(State),\n    Neighborhood = rm_tman_state:get_neighbors(State),\n    OtherNodes = nodelist:to_list(OtherNeighbors),\n    CacheUpd = element(1, nodelist:lupdate_ids(Cache, OtherNodes)),\n    % use only a subset of the cyclon cache in order not to fill the send buffer\n    % with potentially non-existing nodes (in contrast, the nodes from our\n    % neighbourhood are more likely to still exist!)\n    MyRndView = get_RndView(RandViewSize, CacheUpd),\n    % note: we can not use the result from the util:split_unique/2 call below\n    %       since we also need to (re-)integrate nodes which are in our cyclon\n    %       cache but came inside the other nodes's neighbourhood\n    %       -> instead integrate the whole other view and use the updated own\n    %          neighbourhood\n    NewNeighborhood = trigger_update(Neighborhood, MyRndView, OtherNodes),\n\n    MyNodes = nodelist:to_list(NewNeighborhood),\n    % do not send nodes already known to the other node:\n    {MyViewUpd, _, _} = util:split_unique(\n                          lists:append(MyRndView, MyNodes), OtherNodes),\n    OtherNode = nodelist:node(OtherNeighbors),\n    NeighborsToSendTmp = nodelist:mk_neighborhood(MyViewUpd, OtherNode,\n                                                  get_pred_list_length(),\n                                                  get_succ_list_length()),\n    % only send nodes in between the range of the other node's neighborhood\n    % but at least a given number\n    OtherNodeUpdId = node:id(OtherNode),\n    OtherLastPredId = node:id(lists:last(nodelist:preds(OtherNeighbors))),\n    OtherLastSuccId = node:id(lists:last(nodelist:succs(OtherNeighbors))),\n    OtherRange = intervals:union(\n                   intervals:new('(', OtherNodeUpdId, OtherLastSuccId, ')'),\n                   intervals:new('(', OtherLastPredId, OtherNodeUpdId, ')')),\n    NeighborsToSend =\n        tl(nodelist:to_list(\n             nodelist:filter_min_length(\n               NeighborsToSendTmp,\n               fun(N) -> intervals:in(node:id(N), OtherRange) end,\n               RequestPredsMinCount, RequestSuccsMinCount))),\n    comm:send(node:pidX(OtherNode),\n              {rm, buffer_response, NeighborsToSend}, ?SEND_OPTIONS),\n    {{node_discovery}, rm_tman_state:set_neighbors(?FIXMODRN(NewNeighborhood),\n                       rm_tman_state:set_cache(CacheUpd, State))};\n\nhandle_custom_message({rm, buffer_response, OtherNodes},\n                      State) ->\n    % similar to \"{rm, buffer,...}\" handling above:\n    Cache = rm_tman_state:get_cache(State),\n    RandViewSize = rm_tman_state:get_randview_size(State),\n    Neighborhood = rm_tman_state:get_neighbors(State),\n    CacheUpd = element(1, nodelist:lupdate_ids(Cache, OtherNodes)),\n    MyRndView = get_RndView(RandViewSize, CacheUpd),\n    NewNeighborhood = trigger_update(Neighborhood, MyRndView, OtherNodes),\n\n    % increase RandViewSize (no error detected):\n    NewRandViewSize =\n        case RandViewSize < config:read(gossip_cyclon_cache_size) of\n            true ->  RandViewSize + 1;\n            false -> RandViewSize\n        end,\n    {{node_discovery}, rm_tman_state:set_neighbors(?FIXMODRN(NewNeighborhood),\n                       rm_tman_state:set_randview_size(NewRandViewSize,\n                       rm_tman_state:set_cache(CacheUpd, State)))};\n\n% we asked another node we wanted to add for its node object -> now add it\n% (if it is not in the process of leaving the system)\nhandle_custom_message({rm, node_info_response, NodeDetails}, State) ->\n    case node_details:get(NodeDetails, is_leaving) of\n        false ->\n            Node = node_details:get(NodeDetails, node),\n            NewState = update_nodes(State, [Node], [], null),\n            {{node_discovery}, NewState};\n        true ->\n            {{unknown}, ?FIXMODRS(State)}\n    end;\n\nhandle_custom_message({rm, update_node, Node}, State) ->\n    Neighborhood = rm_tman_state:get_neighbors(State),\n    NewNeighborhood1 = nodelist:update_ids(Neighborhood, [Node]),\n    % message from pred or succ\n    % -> update any out-dated nodes between old and new ID of the given node to\n    %    prevent wrong pred/succ changed:\n    NodePid = node:pidX(Node),\n    OldPred = nodelist:pred(Neighborhood),\n    OldPredPid = node:pidX(OldPred),\n    I = case OldPredPid =:= NodePid andalso\n                 OldPredPid =/= node:pidX(nodelist:pred(NewNeighborhood1)) of\n            true ->\n                MyNodeId = nodelist:nodeid(NewNeighborhood1),\n                intervals:new('(', node:id(Node), MyNodeId, ')');\n            _ ->\n                OldSucc = nodelist:succ(Neighborhood),\n                OldSuccPid = node:pidX(OldSucc),\n                case OldSuccPid =:= NodePid andalso\n                         OldSuccPid =/= node:pidX(nodelist:succ(NewNeighborhood1)) of\n                    true ->\n                        MyNodeId = nodelist:nodeid(NewNeighborhood1),\n                        intervals:new('(', MyNodeId, node:id(Node), ')');\n                    _ ->\n                        intervals:empty()\n                end\n        end,\n    % now remove all potentially out-dated nodes and try to re-add them with\n    % updated information\n    NewNeighborhood2 = remove_neighbors_in_interval(NewNeighborhood1, I, null),\n    {{unknown}, rm_tman_state:set_neighbors(?FIXMODRN(NewNeighborhood2), State)};\n\nhandle_custom_message(_, _State) -> unknown_event.\n\n%% @doc Integrates a non-empty cyclon cache into the own random view and\n%%      neighborhood structures and updates the random view size accordingly.\n%%      Ignores empty cyclon caches.\n-spec add_cyclon_cache(Cache::[node:node_type()], rm_tman_state:state())\n        -> {ChangeReason::{unknown} | {node_discovery}, rm_tman_state:state()}.\nadd_cyclon_cache([], State) ->\n    % ignore empty cache from cyclon\n    {{unknown}, State};\nadd_cyclon_cache(NewCache, State) ->\n    % increase RandViewSize (no error detected):\n    RandViewSize = rm_tman_state:get_randview_size(State),\n    Neighborhood = rm_tman_state:get_neighbors(State),\n    RandViewSizeNew =\n        case (RandViewSize < config:read(gossip_cyclon_cache_size)) of\n            true  -> RandViewSize + 1;\n            false -> RandViewSize\n        end,\n    NewNeighborhood = trigger_update(Neighborhood, [], NewCache),\n    {{node_discovery}, rm_tman_state:set_neighbors(?FIXMODRN(NewNeighborhood),\n                       rm_tman_state:set_randview_size(RandViewSizeNew,\n                       rm_tman_state:set_cache(NewCache, State)))}.\n\n-spec trigger_action(State::rm_tman_state:state())\n        -> {ChangeReason::rm_loop:reason(), rm_tman_state:state()}.\ntrigger_action(State) ->\n    % Trigger an update of the Random view\n    % use only a subset of the cyclon cache in order not to fill the send buffer\n    % with potentially non-existing nodes (in contrast, the nodes from our\n    % neighbourhood are more likely to still exist!)\n    % Test for being alone:\n    Neighborhood = rm_tman_state:get_neighbors(State),\n    RandViewSize = rm_tman_state:get_randview_size(State),\n    Cache = rm_tman_state:get_cache(State),\n    Me = nodelist:node(Neighborhood),\n    RndView = get_RndView(RandViewSize, Cache),\n    {Pred, Succ} = get_safe_pred_succ(Neighborhood, RndView),\n    case node:same_process(Pred, Me) andalso node:same_process(Succ, Me) of\n        false -> % there is another node in the system\n            %log:log(debug, \" [RM | ~p ] RNDVIEW: ~p\", [self(),RndView]),\n            %io:format(\"~p~n\",[{Preds,Succs,RndView,Me}]),\n            RequestPredsMinCount =\n                case nodelist:has_real_pred(Neighborhood) of\n                    true -> get_pred_list_length() - length(nodelist:preds(Neighborhood));\n                    _    -> get_pred_list_length()\n                end,\n            RequestSuccsMinCount =\n                case nodelist:has_real_succ(Neighborhood) of\n                    true -> get_succ_list_length() - length(nodelist:succs(Neighborhood));\n                    _    -> get_succ_list_length()\n                end,\n            % send succ and pred our known nodes and request their nodes\n            Message = {rm, buffer, Neighborhood, RequestPredsMinCount, RequestSuccsMinCount},\n            comm:send(node:pidX(Succ), Message, ?SEND_OPTIONS),\n            case Pred =/= Succ of\n                true -> comm:send(node:pidX(Pred), Message, ?SEND_OPTIONS);\n                _    -> ok\n            end;\n        _ -> % our node is the only node in the system\n            % nothing to do here - we will be actively called by any new node\n            % (see new_succ/2 and new_pred/2)\n            ok\n    end,\n    {{unknown}, State}.\n\n-spec new_pred(State::rm_tman_state:state(), NewPred::node:node_type()) ->\n          {ChangeReason::rm_loop:reason(), rm_tman_state:state()}.\nnew_pred(State, NewPred) ->\n    % if we do not want to trust notify_new_pred messages to provide an alive node, use this instead:\n%%     trigger_update(OldNeighborhood, [], [NewPred]),\n    % we trust NewPred to be alive -> integrate node:\n    {{node_discovery}, update_nodes(State, [NewPred], [], null)}.\n\n-spec new_succ(State::rm_tman_state:state(), NewSucc::node:node_type())\n        -> {ChangeReason::rm_loop:reason(), rm_tman_state:state()}.\nnew_succ(State, NewSucc) ->\n    % similar to new_pred/2\n    {{node_discovery}, update_nodes(State, [NewSucc], [], null)}.\n\n-spec update_node(State::rm_tman_state:state(), NewMe::node:node_type())\n        -> {ChangeReason::rm_loop:reason(), rm_tman_state:state()}.\nupdate_node(State, NewMe) ->\n    Cache = rm_tman_state:get_cache(State),\n    Neighborhood = rm_tman_state:get_neighbors(State),\n    RandViewSize = rm_tman_state:get_randview_size(State),\n    NewNeighborhood1 = nodelist:update_node(Neighborhood, NewMe),\n    % -> update any out-dated nodes between old and new ID to prevent wrong\n    %    pred/succ changed:\n    OldId = nodelist:nodeid(Neighborhood),\n    NewId = node:id(NewMe),\n    I = case intervals:in(NewId, nodelist:node_range(Neighborhood)) of\n            true  -> intervals:new('[', NewId, OldId, ')');\n            false -> ?DBG_ASSERT(intervals:in(node:id(NewMe), nodelist:succ_range(Neighborhood))),\n                     intervals:new('(', OldId, NewId, ']')\n        end,\n    NewNeighborhood2 = remove_neighbors_in_interval(NewNeighborhood1, I, null),\n\n    ?DBG_ASSERT2(node:pidX(nodelist:pred(Neighborhood)) =:= node:pidX(nodelist:pred(NewNeighborhood2)),\n                 no_pred_change_allowed),\n    ?DBG_ASSERT2(node:pidX(nodelist:succ(Neighborhood)) =:= node:pidX(nodelist:succ(NewNeighborhood2)),\n                 no_succ_change_allowed),\n    % only send pred and succ the new node\n    Message = {rm, update_node, NewMe},\n    RndView = get_RndView(RandViewSize, Cache),\n    {Pred, Succ} = get_safe_pred_succ(NewNeighborhood2, RndView),\n    comm:send(node:pidX(Succ), Message, ?SEND_OPTIONS),\n    case Pred =/= Succ of\n        true -> comm:send(node:pidX(Pred), Message, ?SEND_OPTIONS);\n        _    -> ok\n    end,\n    %% ignore for modr-mode, it is triggered by slide_chord.erl\n    {{unknown}, rm_tman_state:set_neighbors(NewNeighborhood2, State)}.\n\n%% @doc Removes all nodes from the given neighborhood which are in the\n%%      interval I but keep TolerateNode.\n-spec remove_neighbors_in_interval(Neighborhood::nodelist:neighborhood(),\n                                   I::intervals:interval(),\n                                   TolerateNode::node:node_type() | null)\n        -> NewNeighborhood::nodelist:neighborhood().\nremove_neighbors_in_interval(Neighborhood, I, TolerateNode) ->\n    case intervals:is_empty(I) of\n        false ->\n            nodelist:filter(\n              Neighborhood,\n              % note: be resilient in case we have a more up-to-date TolerateNode node info!\n              fun(N) -> (not intervals:in(node:id(N), I)) orelse\n                            node:same_process(N, TolerateNode) end,\n              fun(N) -> contact_new_nodes([N]) end);\n        true -> Neighborhood\n    end.\n\n-spec contact_new_nodes(NewNodes::[node:node_type()]) -> ok.\ncontact_new_nodes([_|_] = NewNodes) ->\n    % TODO: add a local cache of contacted nodes in order not to contact them again\n    This = comm:this(),\n    case comm:is_valid(This) of\n        true ->\n            _ = [begin\n                     Pid = node:pidX(Node),\n                     comm:send(Pid, {rm, node_info, This, [node, is_leaving]},\n                               ?SEND_OPTIONS)\n                 end || Node <- NewNodes],\n            ok;\n        false -> ok\n    end;\ncontact_new_nodes([]) ->\n    ok.\n\n%% @doc Failure detector reported dead/changed node.\n-spec fd_notify(State::rm_tman_state:state(), Event::fd:event(), DeadPid::comm:mypid(),\n                Data::term())\n        -> {ChangeReason::rm_loop:reason(), rm_tman_state:state()}.\nfd_notify(State, leave, _DeadPid, OldNode) ->\n    % graceful leave -> do not add as zombie candidate!\n    State2 =\n        update_nodes(State, [], [OldNode], null),\n    Cache = rm_tman_state:get_cache(State2),\n    Neighborhood = rm_tman_state:get_neighbors(State2),\n    % try to find a replacement in the cache:\n    NewNeighborhood = trigger_update(Neighborhood, [], Cache),\n    {{graceful_leave, OldNode},\n     rm_tman_state:set_neighbors(?FIXMODRN(NewNeighborhood), State2)};\nfd_notify(State, jump, _DeadPid, OldNode) ->\n    % remove old node while jumping -> do not add as zombie candidate!\n    % the node will be added again (or might already have been added)\n    % -> only remove from neighbours if older!\n    FilterFun = fun(N) ->\n                        ?implies(node:same_process(N, OldNode),\n                                 node:is_newer(N, OldNode))\n                end,\n    State2 =\n        update_nodes2(State, [], true, FilterFun, null),\n    Cache = rm_tman_state:get_cache(State2),\n    Neighborhood = rm_tman_state:get_neighbors(State2),\n    % try to find a replacement in the cache:\n    NewNeighborhood = trigger_update(Neighborhood, [], Cache),\n    {{graceful_leave, OldNode},\n     rm_tman_state:set_neighbors(?FIXMODRN(NewNeighborhood), State2)};\nfd_notify(State, crash, DeadPid, _Reason) ->\n    % crash, i.e. non-graceful leave -> add as zombie candidate\n    State2 =\n        update_nodes(State, [], [DeadPid], fun dn_cache:add_zombie_candidate/1),\n    Cache = rm_tman_state:get_cache(State2),\n    Neighborhood = rm_tman_state:get_neighbors(State2),\n    % try to find a replacement in the cache:\n    NewNeighborhood = trigger_update(Neighborhood, [], Cache),\n    {{node_crashed, DeadPid},\n     rm_tman_state:set_neighbors(?FIXMODRN(NewNeighborhood), State2)};\nfd_notify(State, _Event, _DeadPid, _Data) ->\n    {{unknown}, State}.\n\n% dead-node-cache reported dead node to be alive again\n-spec zombie_node(State::rm_tman_state:state(), Node::node:node_type())\n        -> {ChangeReason::rm_loop:reason(), rm_tman_state:state()}.\nzombie_node(State, Node) ->\n    % this node could potentially be useful as it has been in our state before\n    {{node_discovery}, update_nodes(State, [Node], [], null)}.\n\n-spec get_web_debug_info(State::rm_tman_state:state()) -> [{string(), string()}].\nget_web_debug_info(_State) -> [].\n\n%% @doc Checks whether config parameters of the rm_tman process exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(stabilization_interval_base) and\n    config:cfg_is_greater_than(stabilization_interval_base, 0) and\n\n    config:cfg_is_integer(gossip_cyclon_cache_size) and\n    config:cfg_is_greater_than(gossip_cyclon_cache_size, 2) and\n\n    config:cfg_is_integer(tman_cyclon_interval) and\n    config:cfg_is_greater_than(tman_cyclon_interval, 0) and\n\n    config:cfg_is_integer(succ_list_length) and\n    config:cfg_is_greater_than_equal(succ_list_length, 1) and\n\n    config:cfg_is_integer(pred_list_length) and\n    config:cfg_is_greater_than_equal(pred_list_length, 1).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Internal Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Get N peers from the cyclon cache.\n-spec get_RndView(integer(), [node:node_type()]) -> [node:node_type()].\nget_RndView(N, Cache) ->\n    lists:sublist(Cache, N).\n\n%% @doc Gets the node's current successor and predecessor in a safe way.\n%%      If either is unknown, the random view is used to get a replacement. If\n%%      this doesn't help either, the own node is returned as this is the\n%%      current node's view.\n-spec get_safe_pred_succ(\n        Neighborhood::nodelist:neighborhood(), RndView::[node:node_type()]) ->\n              {Pred::node:node_type(), Succ::node:node_type()}.\nget_safe_pred_succ(Neighborhood, RndView) ->\n    case nodelist:has_real_pred(Neighborhood) andalso\n             nodelist:has_real_succ(Neighborhood) of\n        true -> {nodelist:pred(Neighborhood), nodelist:succ(Neighborhood)};\n        _    -> NewNeighbors = nodelist:add_nodes(Neighborhood, RndView, 1, 1),\n                {nodelist:pred(NewNeighbors), nodelist:succ(NewNeighbors)}\n    end.\n\n% @doc is there churn in the system\n-spec has_churn(OldNeighborhood::nodelist:neighborhood(),\n                NewNeighborhood::nodelist:neighborhood()) -> boolean().\nhas_churn(OldNeighborhood, NewNeighborhood) ->\n    OldNeighborhood =/= NewNeighborhood.\n\n%% @doc Triggers the integration of new nodes from OtherNeighborhood and\n%%      RndView into our Neighborhood by contacting every useful node.\n%%      NOTE: nodes from OtherNeighborhood and RndView compete for the actual\n%%            number of contacted nodes, so the (potentially more outdated)\n%%            random view should be limited\n%%      NOTE: no node is (directly) added by this function, the returned\n%%            neighborhood may contain updated node IDs though!\n-spec trigger_update(OldNeighborhood::nodelist:neighborhood(),\n                     RndView::[node:node_type()],\n                     OtherNodes::[node:node_type()])\n        -> NewNeighborhood::nodelist:neighborhood().\ntrigger_update(OldNeighborhood, MyRndView, OtherNodes) ->\n    % update node ids with information from the other node's neighborhood\n    OldNeighborhood2 = nodelist:update_ids(OldNeighborhood, OtherNodes),\n    NewNeighborhood2 =\n        nodelist:add_nodes(OldNeighborhood2, lists:append(MyRndView, OtherNodes),\n                           get_pred_list_length(), get_succ_list_length()),\n\n    OldView = nodelist:to_list(OldNeighborhood2),\n    NewView = nodelist:to_list(NewNeighborhood2),\n    ViewOrd = fun(A, B) ->\n                      nodelist:succ_ord_node(A, B, nodelist:node(OldNeighborhood2))\n              end,\n    {_, _, NewNodes} = util:ssplit_unique(OldView, NewView, ViewOrd),\n\n    contact_new_nodes(NewNodes),\n    OldNeighborhood2.\n\n%% @doc Adds and removes the given nodes from the rm_tman state.\n%%      Note: Sets the new RandViewSize to 0 if NodesToRemove is not empty and\n%%      the new neighborhood is different to the old one. If the successor or\n%%      predecessor changes, the trigger action will be called immediately.\n-spec update_nodes(State::rm_tman_state:state(),\n                   NodesToAdd::[node:node_type()],\n                   NodesToRemove::[node:node_type() | comm:mypid() | pid()],\n                   RemoveNodeEvalFun::fun((node:node_type()) -> any()) | null)\n        -> NewState::rm_tman_state:state().\nupdate_nodes(State, NodesToAdd, [], RemoveNodeEvalFun) ->\n    update_nodes2(State, NodesToAdd, false, fun(_N) -> true end, RemoveNodeEvalFun);\nupdate_nodes(State, NodesToAdd, [Node], RemoveNodeEvalFun) ->\n    FilterFun = fun(N) -> not node:same_process(N, Node) end,\n    update_nodes2(State, NodesToAdd, true, FilterFun, RemoveNodeEvalFun);\nupdate_nodes(State, NodesToAdd, [_,_|_] = NodesToRemove, RemoveNodeEvalFun) ->\n    FilterFun = fun(N) -> not lists:any(\n                            fun(B) -> node:same_process(N, B) end,\n                            NodesToRemove)\n                end,\n    update_nodes2(State, NodesToAdd, true, FilterFun, RemoveNodeEvalFun).\n\n%% @doc Helper for update_nodes/4 with a more generic interface.\n%% @see update_nodes/4\n-spec update_nodes2(State::rm_tman_state:state(),\n                    NodesToAdd::[node:node_type()], MayRemove::boolean(),\n                    NodesFilterFun::fun((node:node_type()) -> boolean()),\n                    RemoveNodeEvalFun::fun((node:node_type()) -> any()) | null)\n        -> NewState::rm_tman_state:state().\nupdate_nodes2(State, [], false, _FilterFun, _RemoveNodeEvalFun) ->\n    State;\nupdate_nodes2(OldState,\n             NodesToAdd, MayRemove, FilterFun, RemoveNodeEvalFun) ->\n    % keep all nodes that are not in NodesToRemove\n    % note: NodesToRemove should have 0 or 1 element in most cases\n    OldNeighborhood = rm_tman_state:get_neighbors(OldState),\n    OldCache = rm_tman_state:get_cache(OldState),\n    RandViewSize = rm_tman_state:get_randview_size(OldState),\n    OldPredPid = node:pidX(nodelist:pred(OldNeighborhood)),\n    OldSuccPid = node:pidX(nodelist:succ(OldNeighborhood)),\n    Nbh1 = if is_function(RemoveNodeEvalFun) ->\n                  nodelist:filter(OldNeighborhood, FilterFun, RemoveNodeEvalFun);\n              true ->\n                  nodelist:filter(OldNeighborhood, FilterFun)\n           end,\n    NewCache = nodelist:lfilter(OldCache, FilterFun),\n\n    NewNeighborhood = nodelist:add_nodes(Nbh1, NodesToAdd,\n                                         get_pred_list_length(),\n                                         get_succ_list_length()),\n\n    NewChurn = has_churn(OldNeighborhood, NewNeighborhood),\n    NewRandViewSize = case NewChurn andalso MayRemove of\n                          true -> 0;\n                          _    -> RandViewSize\n                      end,\n    NewPredPid = node:pidX(nodelist:pred(NewNeighborhood)),\n    NewSuccPid = node:pidX(nodelist:succ(NewNeighborhood)),\n    NewState = rm_tman_state:set_neighbors(NewNeighborhood,\n               rm_tman_state:set_randview_size(NewRandViewSize,\n               rm_tman_state:set_cache(NewCache,\n               rm_tman_state:set_churn(NewChurn, OldState)))),\n    if OldPredPid =/= NewPredPid orelse OldSuccPid =/= NewSuccPid ->\n           element(2, trigger_action(NewState));\n       true -> NewState\n    end.\n\n-spec trigger_interval() -> pos_integer().\ntrigger_interval() -> config:read(stabilization_interval_base) div 1000.\n\n-spec get_pred_list_length() -> pos_integer().\nget_pred_list_length() -> config:read(pred_list_length).\n\n-spec get_succ_list_length() -> pos_integer().\nget_succ_list_length() -> config:read(succ_list_length).\n\n-spec get_neighbors(rm_tman_state:state()) -> nodelist:neighborhood().\nget_neighbors(State) ->\n    rm_tman_state:get_neighbors(State).\n"
  },
  {
    "path": "src/rm_tman_state.erl",
    "content": "%  @copyright 2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    the state of rm_tman\n%% @end\n-module(rm_tman_state).\n-author('schuett@zib.de').\n\n-include(\"scalaris.hrl\").\n\n-export([get_neighbors/1, get_randview_size/1, get_cache/1, get_churn/1,\n         set_neighbors/2, set_randview_size/2, set_cache/2, set_churn/2,\n         init/4]).\n\n-export_type([state/0]).\n\n\n-opaque state() :: {Neighbors      :: nodelist:neighborhood(),\n                    RandomViewSize :: pos_integer(),\n                    Cache          :: [node:node_type()], % random cyclon nodes\n                    Churn          :: boolean()}.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% getter\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_neighbors(state()) -> nodelist:neighborhood().\nget_neighbors({Neighbors, _RandViewSize, _Cache, _Churn}) ->\n    Neighbors.\n\n-spec get_randview_size(state()) -> pos_integer().\nget_randview_size({_Neighbors, RandViewSize, _Cache, _Churn}) ->\n    RandViewSize.\n\n-spec get_cache(state()) -> [node:node_type()].\nget_cache({_Neighbors, _RandViewSize, Cache, _Churn}) ->\n    Cache.\n\n-spec get_churn(state()) -> boolean().\nget_churn({_Neighbors, _RandViewSize, _Cache, Churn}) ->\n    Churn.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% setter\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec set_neighbors(nodelist:neighborhood(), state()) -> state().\nset_neighbors(Neighbors, {_Neighbors2, RandViewSize2, Cache2, Churn2}) ->\n    {Neighbors, RandViewSize2, Cache2, Churn2}.\n\n-spec set_randview_size(pos_integer(), state()) -> state().\nset_randview_size(RandViewSize, {Neighbors2, _RandViewSize2, Cache2, Churn2}) ->\n    {Neighbors2, RandViewSize, Cache2, Churn2}.\n\n-spec set_cache([node:node_type()], state()) -> state().\nset_cache(Cache, {Neighbors2, RandViewSize2, _Cache2, Churn2}) ->\n    {Neighbors2, RandViewSize2, Cache, Churn2}.\n\n-spec set_churn(boolean(), state()) -> state().\nset_churn(Churn, {Neighbors2, RandViewSize2, Cache2, _Churn2}) ->\n    {Neighbors2, RandViewSize2, Cache2, Churn}.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% misc.\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec init(Neighbors      :: nodelist:neighborhood(),\n          RandomViewSize :: pos_integer(),\n          Cache          :: [node:node_type()], % random cyclon nodes\n          Churn          :: boolean()) -> state().\ninit(Neighbors, RandomViewSize, Cache, Churn) ->\n    {Neighbors, RandomViewSize, Cache, Churn}.\n"
  },
  {
    "path": "src/rrd.erl",
    "content": "% @copyright 2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Round-Robin-Database (RRD) clone.\n%% @end\n%% @version $Id$\n-module(rrd).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-export_type([rrd/0, data_type/0,\n              gauge_type/0, counter_type/0, timing_type/0, event_type/1,\n              timeseries_type/0, internal_time/0, timespan/0]).\n\n% external API with transparent time handling\n-export([create/3,\n         add_now/2, check_timeslot_now/1,\n         merge/2,\n         dump/1, dump_with/2, dump_with/3]).\n\n% external API without transparent time handling\n-export([create/4, add/3, check_timeslot/2]).\n\n% internal API for the monitor process\n-export([get_slot_start/2, reduce_timeslots/2, add_nonexisting_timeslots/2,\n         get_type/1, get_count/1, get_slot_length/1, get_current_time/1,\n         get_value/2, get_value_by_offset/2, get_all_values/2,\n         add_with/4, timing_with_hist_merge_fun/3]).\n\n% misc\n-export([check_config/0]).\n\n% gauge: record newest value of a time slot,\n% counter: sum up all values of a time slot,\n% event: record every event (incl. timestamp) in a time slot,\n% histogram: record all values in a time slot using an approximative histogram\n% timing: record time spans, store {sum(x), sum(x^2), count(x), min(x), max(x)}\n% timing_with_hist: record time spans, store {sum(x), sum(x^2), count(x), min(x), max(x), histogram(x)}\n-type timeseries_type() :: gauge | counter | event |\n                           {histogram, Size::non_neg_integer()} |\n                           {histogram_rt, Size::non_neg_integer(), BaseKey::histogram_rt:base_key()} |\n                           {timing | timing_with_hist, us | ms | s | count | percent}.\n-type fill_policy_type() :: set_undefined | keep_last_value.\n-type time() :: erlang_timestamp().\n-type internal_time() :: non_neg_integer(). % default: micro seconds since Epoch\n-type timespan() :: pos_integer().\n-type update_fun(T, NewV) :: fun((Time::internal_time(), Old::T | undefined, NewV) -> T).\n\n-type gauge_type() :: number().\n-type counter_type() :: number().\n-type histogram_type() :: histogram:histogram() | histogram_rt:histogram().\n-type timing_type() :: {Sum::number(), Sum2::number(), Count::pos_integer(), Min::number(), Max::number(), Hist::histogram:histogram()}.\n-type event_type(T) :: [{internal_time(), T}].\n-type data_type() :: gauge_type() | counter_type() | histogram_type() | timing_type() | event_type(term()).\n\n-record(rrd, {slot_length   = ?required(rrd, slot_length)   :: timespan(),\n              count         = ?required(rrd, count)         :: pos_integer(),\n              type          = ?required(rrd, type)          :: timeseries_type(),\n              % index of current slot\n              current_index = ?required(rrd, current_index) :: non_neg_integer(),\n              % current slot starts at current_time and lasts for step_size\n              % time units\n              current_time  = ?required(rrd, current_time)  :: internal_time(),\n              data          = ?required(rrd, data)          :: array:array(data_type() | undefined),\n              fill_policy   = ?required(rrd, fill_policy)   :: fill_policy_type()\n            }).\n\n-opaque rrd() :: #rrd{}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% External API with transparent time handling\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% @doc Creates a new rrd() record. SlotLength in microseconds, Count is the\n%%      number of time slots to keep, type the type of rrd to create.\n-spec create(SlotLength::timespan(), Count::pos_integer(), Type::timeseries_type()) ->\n    rrd().\ncreate(SlotLength, Count, Type) ->\n    create(SlotLength, Count, Type, os:timestamp()).\n\n% @doc Note: gauge, counter and timing types accept only number() as value, event\n%      accepts any value.\n-spec add_now(Value::term(), rrd()) -> rrd().\nadd_now(Value, DB) ->\n    add(os:timestamp(), Value, DB).\n\n% @doc Advances the stored timeslots (if necessary) to the current time.\n-spec check_timeslot_now(rrd()) -> rrd().\ncheck_timeslot_now(DB) ->\n    check_timeslot(os:timestamp(), DB).\n\n-spec dump(rrd()) -> [{From::time(), To::time(), data_type()}].\ndump(DB) ->\n    dump_with(DB, fun(_DB, From, To, X) ->\n                          {util:us2timestamp(From), util:us2timestamp(To), X}\n                  end).\n\n-type dump_fun_existing(T) :: fun((rrd(), From::internal_time(), To::internal_time(), Value::data_type()) -> T).\n-type dump_fun_nonexisting(T) :: fun((rrd(), From::internal_time(), To::internal_time()) -> ignore | {keep, T}).\n\n-spec dump_with(rrd(), dump_fun_existing(T)) -> [T].\ndump_with(DB, FunExist) ->\n    dump_with(DB, FunExist, fun(_DB, _From, _To) -> ignore end).\n\n-spec dump_with(rrd(), dump_fun_existing(T), dump_fun_nonexisting(U)) -> [T | U].\ndump_with(DB, FunExist, FunNotExist) ->\n    CurrentIndex = DB#rrd.current_index,\n    Count = DB#rrd.count,\n    dump_internal(DB, (CurrentIndex + 1) rem Count, CurrentIndex,\n                  DB#rrd.current_time - (Count - 1) * DB#rrd.slot_length, [],\n                  FunExist, FunNotExist).\n\n%% @doc Merges any value of DB2 which is in the current or a future time slot\n%%      of DB1 into it and returns a new rrd.\n%%      Note: gauge rrd values will only be updated if no previous value\n%%      existed since we can not determine which value is newer.\n%%      For any value from DB2 uses (StartTime + EndTime) div 2 as the time for\n%%      adding it.\n-spec merge(DB1::rrd(), DB2::rrd()) -> rrd().\nmerge(DB1 = #rrd{type = Type}, DB2 = #rrd{type = Type}) ->\n    MergeFun = case Type of\n        gauge   -> fun gauge_merge_fun/3;\n        counter -> fun counter_merge_fun/3;\n        event   -> fun event_merge_fun/3;\n        {timing, _} -> fun timing_merge_fun/3;\n        {timing_with_hist, _} -> fun timing_with_hist_merge_fun/3\n    end,\n    DataL = dump_with(DB2,\n                      fun(_DB, StartTime, EndTime, X) ->\n                              {(StartTime + EndTime) div 2, X}\n                      end),\n    lists:foldl(fun({TimeX, ValueX}, DBX) ->\n                        add_with(TimeX, ValueX, DBX, MergeFun)\n                end, DB1, DataL).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% External API without transparent time handling and some internal API\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%@doc StepSize in milliseconds\n-spec create(SlotLength::timespan(), Count::pos_integer(), Type::timeseries_type(),\n             StartTime::time() | internal_time()) -> rrd().\ncreate(SlotLength, Count, Type, {_, _, _} = StartTime) ->\n    create(SlotLength, Count, Type, util:timestamp2us(StartTime));\ncreate(SlotLength, Count, Type, StartTime) ->\n    #rrd{slot_length = SlotLength, count = Count, type = Type, current_index = 0,\n         current_time = StartTime, data = array:new(Count),\n         fill_policy = set_undefined}.\n\n% @doc Note: gauge, counter and timing types accept only number() as value, event\n%      accepts any value.\n-spec add(Time::time() | internal_time(), Value::term(), rrd()) -> rrd().\nadd({_, _, _} = ExternalTime, Value, DB) ->\n    add_with(util:timestamp2us(ExternalTime), Value, DB, select_fun(DB));\nadd(Time, Value, DB) ->\n    add_with(Time, Value, DB, select_fun(DB)).\n\n-spec select_fun(DB::rrd()) -> update_fun(data_type(), term()).\nselect_fun(DB) ->\n    case DB#rrd.type of\n        gauge -> fun gauge_update_fun/3;\n        counter -> fun counter_update_fun/3;\n        event -> fun event_update_fun/3;\n        {histogram, Size} ->\n            fun(Time, OldValue, Value) ->\n                    histogram_update_fun(Time, OldValue, Value, Size)\n            end;\n        {histogram_rt, Size, BaseKey} ->\n            fun(Time, OldValue, Value) ->\n                    histogram_rt_update_fun(Time, OldValue, Value, Size, BaseKey)\n            end;\n        {timing, _} -> fun timing_update_fun/3;\n        {timing_with_hist, _} -> fun timing_with_hist_update_fun/3\n    end.\n\n% @doc Advances the stored timeslots (if necessary) to the given time.\n-spec check_timeslot(Time::time() | internal_time(), rrd()) -> rrd().\ncheck_timeslot({_, _, _} = ExternalTime, DB) ->\n    add_with(util:timestamp2us(ExternalTime), undefined, DB, fun keep_old_update_fun/3);\ncheck_timeslot(Time, DB) ->\n    add_with(Time, undefined, DB, fun keep_old_update_fun/3).\n\n-spec add_with(Time::internal_time(), NewV, rrd(), update_fun(data_type(), NewV)) -> rrd().\nadd_with(Time, Value, DB = #rrd{slot_length = SlotLength,\n                                current_time = CurrentTime,\n                                current_index = CurrentIndex,\n                                data = Data}, F) ->\n    case is_current_slot(Time, CurrentTime, SlotLength) of\n        true ->\n            OldValue = array:get(CurrentIndex, Data),\n            DB#rrd{data = array:set(CurrentIndex, F(Time, OldValue, Value), Data)};\n        _ ->\n            case is_future_slot(Time, CurrentTime, SlotLength) of\n                true ->\n                    Count = DB#rrd.count,\n                    FillPolicy = DB#rrd.fill_policy,\n                    Delta = (Time - CurrentTime) div SlotLength,\n                    NewIndex = (CurrentIndex + Delta) rem Count,\n                    FilledData =\n                        if Delta =< Count ->\n                                fill(Data, FillPolicy, Count,\n                                     (CurrentIndex + 1) rem Count,\n                                     NewIndex, CurrentIndex);\n                           FillPolicy =:= set_undefined ->\n                               % need to wipe all data clean so that old data\n                               % is not falsely presented for new time slots\n                               % as an optimisation, simply create a new array here\n                               array:new(Count);\n                           true ->\n                               % fill everything:\n                               fill(Data, FillPolicy, Count,\n                                    (CurrentIndex + 1) rem Count,\n                                    CurrentIndex, CurrentIndex)\n                        end,\n                    DB#rrd{data = array:set(NewIndex, F(Time, undefined, Value), FilledData),\n                           current_index = NewIndex,\n                           current_time = DB#rrd.current_time + Delta * SlotLength};\n                _ -> % PastTimeSlot; ignore\n                    DB\n            end\n    end.\n\n-spec fill(Array, fill_policy_type(), Count::pos_integer(),\n           StartIndex::non_neg_integer(), StopIndex::non_neg_integer(),\n           IndexLastValue::non_neg_integer()) -> Array\n        when is_subtype(Array, array:array(data_type() | undefined)).\nfill(Data, FillPolicy, Count, StartIndex, StopIndex, IndexLastValue) ->\n    case FillPolicy of\n        set_undefined ->\n            fill(Data, Count, StartIndex, StopIndex, undefined);\n        keep_last_value ->\n            fill(Data, Count, StartIndex, StopIndex, array:get(IndexLastValue, Data))\n    end.\n\n-spec fill(Array, Count::pos_integer(), StartIndex::non_neg_integer(),\n           StopIndex::non_neg_integer(), NewValue::data_type() | undefined) -> Array\n        when is_subtype(Array, array:array(data_type() | undefined)).\nfill(Data, _Count, StopIndex, StopIndex, _NewValue) ->\n    Data;\nfill(Data, Count, CurrentGapIndex, StopIndex, NewValue) ->\n    fill(array:set(CurrentGapIndex, NewValue, Data), Count,\n         (CurrentGapIndex + 1) rem Count,\n         StopIndex, NewValue).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% Internal API for the monitor\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Gets the start of the given slot.\n%%      Some examples of values for SlotIdx:\n%%      previous slot: -1, current slot: 0, next slot: 1\n-spec get_slot_start(SlotIdx::integer(), DB::rrd()) -> internal_time().\nget_slot_start(0, DB) ->\n    DB#rrd.current_time; % minor optimization\nget_slot_start(SlotIdx, DB) ->\n    DB#rrd.current_time + SlotIdx * DB#rrd.slot_length.\n\n%% @doc Reduces the number of time slots.\n-spec reduce_timeslots(SlotCount::pos_integer(), DB::rrd()) -> rrd().\nreduce_timeslots(SlotCount, DB) when SlotCount >= DB#rrd.count ->\n    DB#rrd{data = array:resize(SlotCount, DB#rrd.data),\n           count = SlotCount};\nreduce_timeslots(SlotCount, DB) ->\n    Count = DB#rrd.count,\n    CurrentIndex = DB#rrd.current_index,\n    NewDB = DB#rrd{data = array:new(SlotCount), count = SlotCount,\n                   current_index = 0},\n    EndIdx = (CurrentIndex + Count - SlotCount + 1) rem Count,\n    copy_data(DB, NewDB#rrd.current_index, CurrentIndex, EndIdx, NewDB).\n\n%% @doc Adds to DB all non-existing time slots from OtherDB that are newer than\n%%      or are in the current time slot of DB. Both structures must have the\n%%      same type and the same slot borders!\n-spec add_nonexisting_timeslots(DB::rrd(), OtherDB::rrd()) -> rrd().\nadd_nonexisting_timeslots(#rrd{type = T, slot_length = L} = DB,\n                          #rrd{type = T, slot_length = L} = OtherDB) ->\n    OtherCurrentIdx = OtherDB#rrd.current_index,\n    OtherStartIdx = (OtherCurrentIdx + 1) rem OtherDB#rrd.count,\n    add_nonexisting_timeslots_internal(DB, OtherDB, OtherStartIdx, OtherCurrentIdx).\n\n-spec add_nonexisting_timeslots_internal(DB::rrd(), OtherDB::rrd(), OtherStart::non_neg_integer(), OtherEnd::non_neg_integer()) -> rrd().\nadd_nonexisting_timeslots_internal(DB, OtherDB, OtherEnd, OtherEnd) ->\n    add_nonexisting_timeslots_internal2(DB, OtherDB, OtherEnd);\nadd_nonexisting_timeslots_internal(DB, OtherDB, OtherIndex, OtherEnd) ->\n    NewDB = add_nonexisting_timeslots_internal2(DB, OtherDB, OtherIndex),\n    add_nonexisting_timeslots_internal(NewDB, OtherDB, (OtherIndex + 1) rem OtherDB#rrd.count, OtherEnd).\n\n-spec add_nonexisting_timeslots_internal2(DB::rrd(), OtherDB::rrd(), OtherIndex::non_neg_integer()) -> rrd().\nadd_nonexisting_timeslots_internal2(DB, OtherDB, OtherIndex) ->\n    OtherCount = OtherDB#rrd.count,\n    OtherCurrentIdx = OtherDB#rrd.current_index,\n    OtherIndexTime = get_slot_start((OtherIndex - OtherCurrentIdx - OtherCount) rem OtherCount, OtherDB),\n    OtherValue = array:get(OtherIndex, OtherDB#rrd.data),\n    add_with(OtherIndexTime, OtherValue, DB, fun keep_old_update_fun/3).\n\n-spec get_slot_length(DB::rrd()) -> timespan().\nget_slot_length(DB) -> DB#rrd.slot_length.\n\n-spec get_type(DB::rrd()) -> timeseries_type().\nget_type(DB) -> DB#rrd.type.\n\n-spec get_count(DB::rrd()) -> pos_integer().\nget_count(DB) -> DB#rrd.count.\n\n-spec get_current_time(DB::rrd()) -> internal_time().\nget_current_time(DB) ->\n    DB#rrd.current_time.\n\n%% @doc Gets the value at the given time or 'undefined' if there is no value.\n-spec get_value(DB::rrd(), Time::time() | internal_time()) -> undefined | data_type().\nget_value(DB, {_, _, _} = Time) ->\n    get_value(DB, util:timestamp2us(Time));\nget_value(DB, InternalTime) when is_integer(InternalTime) ->\n    case time2slotidx(InternalTime, DB#rrd.current_time, DB#rrd.slot_length,\n                      DB#rrd.current_index, DB#rrd.count) of\n        future -> undefined;\n        past -> undefined;\n        Slot ->\n            array:get(Slot, DB#rrd.data)\n    end.\n\n%% @doc If SlotOffset is 0, gets the current value, otherwise the value in a\n%%      previous slot the given offset away from the current one.\n%%      May return 'undefined' if there is no value.\n-spec get_value_by_offset(DB::rrd(), SlotOffset::non_neg_integer()) -> undefined | data_type().\nget_value_by_offset(DB, 0) ->\n    array:get(DB#rrd.current_index, DB#rrd.data); % minor optimization\nget_value_by_offset(DB, SlotOffset) when SlotOffset < DB#rrd.count ->\n    Count = DB#rrd.count,\n    Index = (DB#rrd.current_index + Count - SlotOffset) rem Count,\n    array:get(Index, DB#rrd.data);\nget_value_by_offset(DB, SlotOffset) when SlotOffset >= DB#rrd.count ->\n    % rare use case\n    get_value_by_offset(DB, SlotOffset rem DB#rrd.count).\n\n%% @doc Gets all values as list from newest to oldest (desc) or from\n%%      oldest to newest (asc). Values may be undefined.\n-spec get_all_values(asc | desc, DB::rrd()) -> [undefined | data_type()].\nget_all_values(desc, DB) ->\n    [get_value_by_offset(DB, Offset) || Offset <- lists:seq(0, DB#rrd.count-1)];\nget_all_values(asc, DB) ->\n    [get_value_by_offset(DB, Offset) || Offset <- lists:seq(DB#rrd.count-1, 0, -1)].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% Private API\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec is_current_slot(Time::internal_time(), CurrentTime::internal_time(), StepSize::timespan()) -> boolean().\nis_current_slot(Time, CurrentTime, StepSize) ->\n    CurrentTime =< Time andalso Time < CurrentTime + StepSize.\n\n-spec is_future_slot(Time::internal_time(), CurrentTime::internal_time(), StepSize::timespan()) -> boolean().\nis_future_slot(Time, CurrentTime, StepSize) ->\n    CurrentTime + StepSize =< Time.\n\n-spec time2slotidx(Time::internal_time(), CurrentTime::internal_time(),\n                   StepSize::timespan(), CurrentSlot::non_neg_integer(),\n                   SlotCount::pos_integer()) -> non_neg_integer()| future | past.\ntime2slotidx(Time, CurrentTime, StepSize, CurrentSlot, SlotCount) ->\n    case is_current_slot(Time, CurrentTime, StepSize) of\n        true -> CurrentSlot;\n        _ ->\n            case is_future_slot(Time, CurrentTime, StepSize) of\n                true -> future;\n                _ ->\n                    SlotOffset = SlotCount + (Time - CurrentTime) div StepSize,\n                    if SlotOffset > 0 -> (CurrentSlot + SlotOffset) rem SlotCount;\n                       true           -> past\n                    end\n            end\n    end.\n\n-spec dump_internal(rrd(), CurrentIdx::non_neg_integer(), EndIdx::non_neg_integer(),\n                    CurrentTime::internal_time(), Rest::[T],\n                    dump_fun_existing(T), dump_fun_nonexisting(U)) -> [T | U].\ndump_internal(DB, EndIndex, EndIndex, CurrentTime, Rest, FunExist, FunNotExist) ->\n    dump_internal2(DB, EndIndex, CurrentTime, Rest, FunExist, FunNotExist);\ndump_internal(DB, IndexToFetch, EndIndex, CurrentTime, Rest, FunExist, FunNotExist) ->\n    NewRest = dump_internal2(DB, IndexToFetch, CurrentTime, Rest, FunExist, FunNotExist),\n    dump_internal(DB, (IndexToFetch + 1) rem DB#rrd.count, EndIndex,\n                  CurrentTime + DB#rrd.slot_length, NewRest, FunExist, FunNotExist).\n\ndump_internal2(DB, CurrentIndex, CurrentTime, Rest, FunExist, FunNotExist) ->\n    From = CurrentTime,\n    To = CurrentTime + DB#rrd.slot_length,\n    case array:get(CurrentIndex, DB#rrd.data) of\n        undefined -> case FunNotExist(DB, From, To) of\n                         ignore -> Rest;\n                         {keep, X} -> [X | Rest]\n                     end;\n        Value     -> [FunExist(DB, From, To, Value) | Rest]\n    end.\n\n-spec copy_data(rrd(), IndexToWrite::non_neg_integer(), Start::non_neg_integer(), End::non_neg_integer(), Acc::rrd()) -> rrd().\ncopy_data(DB, IndexToWrite, EndIndex, EndIndex, AccDB) ->\n    copy_data2(DB, IndexToWrite, EndIndex, AccDB);\ncopy_data(DB, IndexToWrite, CurrentIndex, EndIndex, AccDB) ->\n    NewAccDB = copy_data2(DB, IndexToWrite, CurrentIndex, AccDB),\n    AccCount = AccDB#rrd.count,\n    Count = DB#rrd.count,\n    copy_data(DB, (IndexToWrite + AccCount - 1) rem AccCount,\n              (CurrentIndex + Count - 1) rem Count, EndIndex, NewAccDB).\n\n-spec copy_data2(rrd(), IndexToWrite::non_neg_integer(), Start::non_neg_integer(), Acc::rrd()) -> rrd().\ncopy_data2(DB, IndexToWrite, CurrentIndex, AccDB) ->\n    CurrentData = array:get(CurrentIndex, DB#rrd.data),\n    AccData = array:set(IndexToWrite, CurrentData, AccDB#rrd.data),\n    AccDB#rrd{data = AccData}.\n\n-spec gauge_update_fun(Time::internal_time(), Old::T | undefined, New::T) -> T when is_subtype(T, number()).\ngauge_update_fun(_Time, _Old, New) -> New.\n\n-spec counter_update_fun(Time::internal_time(), Old::T | undefined, New::T) -> T when is_subtype(T, number()).\ncounter_update_fun(_Time, undefined, New) -> New;\ncounter_update_fun(_Time, Old, New) -> Old + New.\n\n-spec event_update_fun(Time::internal_time(), Old::event_type(T) | undefined, NewV)\n        -> event_type(T | NewV).\nevent_update_fun(Time, undefined, New) -> [{Time, New}];\nevent_update_fun(Time, Old, New) -> lists:append(Old, [{Time, New}]).\n\n-spec histogram_update_fun(Time::internal_time(), Old::histogram_type() | undefined, NewV::number(), Size::non_neg_integer())\n        -> histogram:histogram().\nhistogram_update_fun(_Time, undefined, New, Size) ->\n    Hist = histogram:create(Size),\n    histogram:add(New, Hist);\nhistogram_update_fun(_Time, Hist, New, _Size) ->\n    histogram:add(New, Hist).\n\n-spec histogram_rt_update_fun(Time::internal_time(), Old::histogram_type() | undefined, NewV::no_op | number(), Size::non_neg_integer(), BaseKey::histogram_rt:base_key())\n        -> histogram_rt:histogram().\nhistogram_rt_update_fun(Time, undefined, New, Size, BaseKey) ->\n    Hist = histogram_rt:create(Size, BaseKey),\n    histogram_rt_update_fun(Time, Hist, New, Size, BaseKey);\nhistogram_rt_update_fun(_Time, Hist, no_op, _Size, _BaseKey) ->\n    Hist;\nhistogram_rt_update_fun(_Time, Hist, New, _Size, _BaseKey) ->\n    histogram_rt:add(New, Hist).\n\n-spec timing_update_fun(Time::internal_time(), Old::timing_type() | undefined, New::number())\n        -> timing_type().\ntiming_update_fun(_Time, undefined, New) ->\n    {New, New*New, 1, New, New, histogram:create(0)};\ntiming_update_fun(_Time, {Sum, Sum2, Count, Min, Max, Hist}, New) ->\n    {Sum + New, Sum2 + New*New, Count + 1, erlang:min(Min, New), erlang:max(Max, New), Hist}.\n\n-spec timing_with_hist_update_fun(Time::internal_time(), Old::timing_type() | undefined, New::number())\n        -> timing_type().\ntiming_with_hist_update_fun(_Time, undefined, New) ->\n    Hist = histogram:create(get_timing_hist_size()),\n    {New, New*New, 1, New, New, histogram:add(New, Hist)};\ntiming_with_hist_update_fun(_Time, {Sum, Sum2, Count, Min, Max, Hist}, New) ->\n    {Sum + New, Sum2 + New*New, Count + 1, erlang:min(Min, New), erlang:max(Max, New),\n     histogram:add(New, Hist)}.\n\n-spec keep_old_update_fun(Time::internal_time(), Old::T | undefined, NewV) -> T | NewV.\nkeep_old_update_fun(_Time, undefined, New) -> New;\nkeep_old_update_fun(_Time, Old, _New) -> Old.\n\n-spec gauge_merge_fun(Time::internal_time(), Old::T | undefined, New::T | undefined)\n        -> T | undefined when is_subtype(T, number()).\ngauge_merge_fun(_Time, undefined, New) -> New;\ngauge_merge_fun(_Time, Old, _New) -> Old.\n\n-spec counter_merge_fun(Time::internal_time(), Old::T | undefined, New::T | undefined)\n        -> T | undefined when is_subtype(T, number()).\ncounter_merge_fun(_Time, undefined, New) ->\n    New;\ncounter_merge_fun(_Time, Old, undefined) ->\n    Old;\ncounter_merge_fun(_Time, Old, New) ->\n    Old + New.\n\n-spec event_merge_fun(Time::internal_time(), Old::event_type(T) | undefined, event_type(NewV) | undefined)\n        -> event_type(T | NewV) | undefined.\nevent_merge_fun(_Time, undefined, New) ->\n    New;\nevent_merge_fun(_Time, Old, undefined) ->\n    Old;\nevent_merge_fun(_Time, Old, New) ->\n    lists:usort(lists:append(Old, New)).\n\n-spec timing_merge_fun(Time::internal_time(), Old::timing_type() | undefined, New::timing_type() | undefined)\n        -> timing_type() | undefined.\ntiming_merge_fun(_Time, undefined, New) ->\n    New;\ntiming_merge_fun(_Time, Old, undefined) ->\n    Old;\ntiming_merge_fun(_Time, {Sum, Sum2, Count, Min, Max, Hist},\n                  {NewSum, NewSum2, NewCount, NewMin, NewMax, _NewHist}) ->\n    {Sum + NewSum, Sum2 + NewSum2, Count + NewCount,\n     erlang:min(Min, NewMin), erlang:max(Max, NewMax), Hist}.\n\n-spec timing_with_hist_merge_fun(Time::internal_time(), Old::timing_type() | undefined, New::timing_type() | undefined)\n        -> timing_type() | undefined.\ntiming_with_hist_merge_fun(_Time, undefined, New) ->\n    New;\ntiming_with_hist_merge_fun(_Time, Old, undefined) ->\n    Old;\ntiming_with_hist_merge_fun(_Time, {Sum, Sum2, Count, Min, Max, Hist},\n                  {NewSum, NewSum2, NewCount, NewMin, NewMax, NewHist}) ->\n    {Sum + NewSum, Sum2 + NewSum2, Count + NewCount,\n     erlang:min(Min, NewMin), erlang:max(Max, NewMax), histogram:merge(Hist, NewHist)}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% miscellaneous\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Checks whether config parameters of the rrd module exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(rrd_timing_hist_size) and\n    config:cfg_is_greater_than_equal(rrd_timing_hist_size, 0).\n\n-spec get_timing_hist_size() -> non_neg_integer().\nget_timing_hist_size() ->\n    config:read(rrd_timing_hist_size).\n"
  },
  {
    "path": "src/rrepair/art.erl",
    "content": "% @copyright 2011,2012 Zuse Institute Berlin\n%\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n%\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @doc    Approximate Reconciliation Tree (ART) implementation\n%%         Represents a packed merkle tree.\n%% @end\n%% @reference\n%%         2002 - J.Byers, J.Considine, M.Mitzenmachen\n%%              Fast Approximate Reconcilication of Set Differences\n%%              - BU Computer Science TR\n%% @end\n%% @version $Id$\n-module(art).\n-author('malange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-include(\"record_helpers.hrl\").\n-include(\"scalaris.hrl\").\n\n-export([new/1, new/2, default_config/0,\n         get_interval/1, get_config/1,\n         get_property/2,\n         lookup/2]).\n-export([merkle_leaf_hf/2]).\n\n-export_type([art/0, config/0]).\n\n%-define(TRACE(X,Y), io:format(\"~w: [~p] \" ++ X ++ \"~n\", [?MODULE, self()] ++ Y)).\n-define(TRACE(X,Y), ok).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Types\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-type config_param() :: {correction_factor, non_neg_integer()} |\n                        {inner_bf_fpr,      float()} |\n                        {leaf_bf_fpr,       float()}.\n-type config()       :: [config_param()].\n\n-type art() :: { art,\n                 CorrectionFactor :: non_neg_integer(),\n                 InnerBfFpr :: float(),\n                 LeafBfFpr  :: float(),\n                 Interval   :: intervals:interval(),\n                 InnerNodes :: bloom:bloom_filter(),\n                 Leafs      :: bloom:bloom_filter()}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc Returns default art configuration.\n-spec default_config() -> config().\ndefault_config() ->\n    [{correction_factor, 1},\n     {inner_bf_fpr, 0.01},\n     {leaf_bf_fpr, 0.1}].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec new(merkle_tree:merkle_tree()) -> art().\nnew(Tree) ->\n    new(Tree, default_config()).\n\n-spec new(merkle_tree:merkle_tree(), config()) -> art().\nnew(Tree, Config0) ->\n    Config = merge_prop_lists(default_config(), Config0),\n    CorrectionFactor = proplists:get_value(correction_factor, Config),\n    InnerBfFpr = proplists:get_value(inner_bf_fpr, Config),\n    LeafBfFpr = proplists:get_value(leaf_bf_fpr, Config),\n    {InnerCount, LeafCount, _EmptyLeafCount, _ItemCount} = merkle_tree:size_detail(Tree),\n    InnerBF = bloom:new_fpr(InnerCount, InnerBfFpr),\n    LeafBF = bloom:new_fpr(LeafCount, LeafBfFpr),\n    {IBF, LBF} = fill_bloom(merkle_tree:iterator(Tree), InnerBF, LeafBF),\n    ?TRACE(\"INNER=~p~nLeaf=~p\", [bloom:print(IBF), bloom:print(LBF)]),\n    {art, CorrectionFactor, InnerBfFpr, LeafBfFpr,\n     merkle_tree:get_interval(Tree), IBF, LBF}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc returns interval of the packed merkle tree (art-structure)\n-spec get_interval(art()) -> intervals:interval().\nget_interval({art, _CorFac, _IBfFpr, _LBfFpr, Interval, _IBF, _LBF}) -> Interval.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_property(art(), correction_factor) -> non_neg_integer();\n                  (art(), items_count) -> non_neg_integer();\n                  (art(), inner_bf | leaf_bf) -> bloom:bloom_filter().\nget_property({art, CorFac, _IBfFpr, _LBfFpr, _Interval, _IBF, _LBF}, correction_factor) ->\n    CorFac;\nget_property({art, _CorFac, _IBfFpr, _LBfFpr, _Interval, _IBF, LBF}, items_count) ->\n    bloom:get_property(LBF, items_count);\nget_property({art, _CorFac, _IBfFpr, _LBfFpr, _Interval, IBF, _LBF}, inner_bf) ->\n    IBF;\nget_property({art, _CorFac, _IBfFpr, _LBfFpr, _Interval, _IBF, LBF}, leaf_bf) ->\n    LBF.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_config(art()) -> config().\nget_config({art, CorFac, IBfFpr, LBfFpr, _Interval, _IBF, _LBF}) ->\n    [{correction_factor, CorFac},\n     {inner_bf_fpr,      IBfFpr},\n     {leaf_bf_fpr,       LBfFpr}].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec lookup(merkle_tree:mt_node(), art()) -> boolean().\nlookup(Node, {art, CorrectionFactor, _IBfFpr, _LBfFpr, _Interval, IBF, LBF}) ->\n    lookup_cf([{Node, CorrectionFactor}], IBF, LBF).\n\n-spec lookup_cf([{Node, CF}], IBF::BF, LBF::BF) -> Result when\n    is_subtype(BF,     bloom:bloom_filter()),\n    is_subtype(Node,   merkle_tree:mt_node()),\n    is_subtype(CF,     non_neg_integer()),        %correction factor\n    is_subtype(Result, boolean()).\nlookup_cf([{Node, 0} | L], IBF, LBF) ->\n    NodeHash = merkle_tree:get_hash(Node),\n    BF = ?IIF(merkle_tree:is_leaf(Node), LBF, IBF),\n    ?TRACE(\"NodeHash=~p\", [NodeHash]),\n    bloom:is_element(BF, NodeHash) andalso lookup_cf(L, IBF, LBF);\nlookup_cf([{Node, CF} | L], IBF, LBF) ->\n    NodeHash = merkle_tree:get_hash(Node),\n    IsLeaf = merkle_tree:is_leaf(Node),\n    BF = ?IIF(IsLeaf, LBF, IBF),\n    ?TRACE(\"NodeHash=~p~nIsLeaf=~p\", [NodeHash, IsLeaf]),\n    case bloom:is_element(BF, NodeHash) of\n        false -> false;\n        true  -> if IsLeaf -> lookup_cf(L, IBF, LBF);\n                    true   -> Childs = merkle_tree:get_childs(Node),\n                              NL = prepend_merkle_childs(L, Childs, CF - 1),\n                              lookup_cf(NL, IBF, LBF)\n                 end\n    end;\nlookup_cf([], _IBF, _LBF) ->\n    true.\n\n%% @doc Prepends the given merkle_tree Childs to the LookupList with the given\n%%      correction factor.\n-spec prepend_merkle_childs(LookupList::[{Node, CF}], Childs::[Node],\n                            ChildCF::non_neg_integer()) -> [{Node, CF}] when\n    is_subtype(Node,   merkle_tree:mt_node()),\n    is_subtype(CF,     non_neg_integer()).        %correction factor\nprepend_merkle_childs(L, [], _ChildCF) -> L;\nprepend_merkle_childs(L, [Child | Rest], ChildCF) ->\n    [{Child, ChildCF} | prepend_merkle_childs(L, Rest, ChildCF)].\n    \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Helper\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec fill_bloom(merkle_tree:mt_iter(), Inner1::BF, Leaf1::BF) -> {Inner2::BF, Leaf2::BF} when\n      is_subtype(BF,        bloom:bloom_filter()).\nfill_bloom(Iter, IBF, LBF) ->\n    {InnerHashes, LeafHashes} = merkle_get_hashes(Iter, [], []),\n    {bloom:add_list(IBF, InnerHashes), bloom:add_list(LBF, LeafHashes)}.\n\n-spec merkle_get_hashes(merkle_tree:mt_iter(), InnerHashes1::HashL, LeafHashes1::HashL)\n        -> {InnerHashes2::HashL, LeafHashes2::HashL} when\n          is_subtype(HashL, [merkle_tree:mt_node_key()]).\nmerkle_get_hashes(Iter, InnerHashes, LeafHashes) ->\n    case merkle_tree:next(Iter) of\n        none -> {InnerHashes, LeafHashes};\n        {Node, Iter2} ->\n            NodeHash = merkle_tree:get_hash(Node),\n            case merkle_tree:is_leaf(Node) of\n                true ->\n                    merkle_get_hashes(Iter2, InnerHashes, [NodeHash | LeafHashes]);\n                false ->\n                    merkle_get_hashes(Iter2, [NodeHash | InnerHashes], LeafHashes)\n            end\n    end.\n\nmerge_prop_lists(DefList, ListB) ->\n    lists:foldl(fun({Key, Val}, Acc) ->\n                        [{Key, proplists:get_value(Key, ListB, Val)} | Acc]\n                end, [], DefList).\n\n%% @doc Leaf hash fun to use for the embedded merkle tree.\n-spec merkle_leaf_hf(merkle_tree:mt_bucket(), intervals:interval()) -> binary().\nmerkle_leaf_hf([], I) ->\n    ?CRYPTO_SHA(term_to_binary(I));\nmerkle_leaf_hf([_|_] = Bucket, _I) ->\n    ?CRYPTO_SHA(term_to_binary(Bucket)).\n"
  },
  {
    "path": "src/rrepair/bloom.erl",
    "content": "% @copyright 2011-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @doc    Bloom Filter implementation\n%% @end\n%% @reference A. Broder, M. Mitzenmacher\n%%          <em>Network Applications of Bloom Filters: A Survey</em>\n%%          2004 Internet Mathematics 1(4)\n%% @version $Id$\n-module(bloom).\n-author('malange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-include(\"record_helpers.hrl\").\n-include(\"scalaris.hrl\").\n\n-define(REP_HFS, hfs_plain). %HashFunctionSet selection for usage by bloom filter\n\n-export([new_fpr/2, new_fpr/3, new_bpi/3, new_bin/3, new/2,\n         add/2, add_list/2, is_element/2, filter/2, filter_neg/2,\n         filter_neg_and_add/3,\n         item_count/1]).\n-export([equals/2, join/2, get_property/2, print/1]).\n\n% needed by other Bloom filter implementations or rr_recon:\n-export([calc_least_size/3,\n         calc_HF_num_Size_opt/2, calc_FPR/3,\n         select_best/7,\n         p_add_positions/3,\n         resize/2]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Types\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-record(bloom, {\n                filter        = ?required(bloom, filter) :: bitstring(),\n                hfs           = ?required(bloom, hfs)    :: ?REP_HFS:hfs(),    %HashFunctionSet\n                items_count   = 0                        :: non_neg_integer()  %number of inserted items\n               }).\n-opaque bloom_filter() :: #bloom{}.\n-type key() :: any().\n\n-export_type([bloom_filter/0, key/0]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Creates a new bloom filter with the default (optimal) hash function set\n%%      based on the given false positive rate.\n-spec new_fpr(MaxItems::non_neg_integer(), FPR::float()) -> bloom_filter().\nnew_fpr(MaxItems, FPR) ->\n    {K, Size} = calc_HF_num_Size_opt(MaxItems, FPR),\n    new(Size, ?REP_HFS:new(K)).\n\n%% @doc Creates a new bloom filter with the given hash function set\n%%      based on the given false positive rate.\n-spec new_fpr(MaxItems::non_neg_integer(), FPR::float(), ?REP_HFS:hfs() | non_neg_integer())\n        -> bloom_filter().\nnew_fpr(MaxItems, FPR, Hfs) ->\n    Size = calc_least_size(MaxItems, FPR, ?REP_HFS:size(Hfs)),\n    new(Size, Hfs).\n\n%% @doc Creates a new bloom filter with the given hash function set and a fixed\n%%      number of bits per item.\n-spec new_bpi(MaxItems::non_neg_integer(), BitsPerItem::number(), ?REP_HFS:hfs() | non_neg_integer())\n        -> bloom_filter().\nnew_bpi(MaxItems, BitPerItem, Hfs) ->\n    new(util:ceil(BitPerItem * MaxItems), Hfs).\n\n%% @doc Creates a new bloom filter with the given binary, hash function set and\n%%      item count.\n-spec new_bin(Filter::bitstring(), ?REP_HFS:hfs() | non_neg_integer(), ItemsCount::non_neg_integer())\n        -> bloom_filter().\nnew_bin(Filter, HfCount, ItemsCount) when is_integer(HfCount) ->\n    new_bin(Filter, ?REP_HFS:new(HfCount), ItemsCount);\nnew_bin(Filter, Hfs, ItemsCount) ->\n    #bloom{filter = Filter, hfs = Hfs, items_count = ItemsCount}.\n\n%% @doc Creates a new bloom filter.\n-spec new(BitSize::pos_integer(), ?REP_HFS:hfs() | non_neg_integer()) -> bloom_filter().\nnew(BitSize, HfCount) when is_integer(HfCount) ->\n    new(BitSize, ?REP_HFS:new(HfCount));\nnew(BitSize, Hfs) ->\n    #bloom{filter = <<0:BitSize>>, hfs = Hfs, items_count = 0}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Adds one item to the bloom filter.\n-spec add(bloom_filter(), key()) -> bloom_filter().\nadd(#bloom{hfs = Hfs, items_count = FilledCount,\n           filter = Filter} = Bloom, Item) ->\n    BFSize = erlang:bit_size(Filter),\n    Bloom#bloom{filter = p_add_list(Hfs, BFSize, Filter, [Item]),\n                items_count = FilledCount + 1}.\n\n%% @doc Adds multiple items to the bloom filter.\n-spec add_list(bloom_filter(), [key()]) -> bloom_filter().\nadd_list(#bloom{hfs = Hfs,\n                items_count = FilledCount,\n                filter = Filter\n               } = Bloom, Items) ->\n    BFSize = erlang:bit_size(Filter),\n    F = p_add_list(Hfs, BFSize, Filter, Items),\n    ItemsL = length(Items),\n    Bloom#bloom{filter = F, items_count = FilledCount + ItemsL}.\n\n-compile({inline, [p_add_list/4, p_add_positions/3]}).\n\n%% @doc Helper to set item bits, optimised for a large number of items /\n%%      positions to set.\n-spec p_add_list(Hfs::?REP_HFS:hfs(), BFSize::pos_integer(),\n                 BF1::bitstring(), Items::[key()]) -> BF2::bitstring().\np_add_list(_Hfs, _BFSize, BF, []) -> BF;\np_add_list(_Hfs, 1, _BF, _Items = [_|_]) -> <<1:1>>;\np_add_list(Hfs, BFSize, BF, Items = [_|_]) ->\n    Positions = lists:flatmap(fun(Item) ->\n                                      ?REP_HFS:apply_val_rem(Hfs, Item, BFSize)\n                              end, Items),\n    p_add_positions(Positions, BF, BFSize).\n\n-spec p_add_positions(Positions::[non_neg_integer(),...], BF1::bitstring(),\n                      BFSize::pos_integer()) -> BF2::bitstring().\np_add_positions(Positions, BF, BFSize) ->\n    [Pos | Rest] = lists:usort(Positions),\n    PosInByte = Pos rem 8,\n    PreBitsNum = Pos - PosInByte,\n    AccPosBF = (2#10000000 bsr PosInByte),\n    p_add_list_(Rest, AccPosBF, [<<0:PreBitsNum>>], PreBitsNum, BF, BFSize).\n\np_add_list_([Pos | Rest], AccPosBF, AccBF, AccBFSize, BF, BFSize) when Pos - AccBFSize < 8 ->\n    % Pos in same byte\n    PosInByte = Pos rem 8,\n    AccPosBF2 = AccPosBF bor (2#10000000 bsr PosInByte),\n    p_add_list_(Rest, AccPosBF2, AccBF, AccBFSize, BF, BFSize);\np_add_list_([Pos | Rest], AccPosBF, AccBF, AccBFSize, BF, BFSize) ->\n    % Pos in next byte\n    PosInByte = Pos rem 8,\n    PreBitsNum2 = Pos - PosInByte,\n    DiffBits = PreBitsNum2 - AccBFSize - 8,\n    % add AccPosBF to AccBF\n    AccBF2 = [<<AccPosBF:8, 0:DiffBits>> | AccBF],\n    % make new AccPosBF\n    AccPosBF2 = (2#10000000 bsr PosInByte),\n    p_add_list_(Rest, AccPosBF2, AccBF2, AccBFSize + 8 + DiffBits, BF, BFSize);\np_add_list_([], AccPosBF, AccBF, AccBFSize, BF, BFSize) ->\n    RestBits = BFSize - AccBFSize,\n    LastBitString = if RestBits >= 8 ->\n                           <<AccPosBF:RestBits/little>>;\n                       true ->\n                           % be careful with incomplete bytes (see set_bits/3)\n                           AccPosBF1 = AccPosBF bsr (8 - RestBits),\n                           <<AccPosBF1:RestBits/little>>\n                    end,\n    AccBFBin = erlang:list_to_bitstring(lists:reverse([LastBitString | AccBF])),\n    case BF of\n        <<0:BFSize>> ->\n            AccBFBin;\n        _ ->\n            % merge AccBF2 and BF:\n            Result = util:bin_or(BF, AccBFBin),\n            % this also makes suce that BF and AccBFBin both have the same size:\n            ?ASSERT(erlang:bit_size(Result) =:= BFSize),\n            Result\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Returns true if the bloom filter contains this item.\n-spec is_element(bloom_filter(), key()) -> boolean().\nis_element(#bloom{items_count = 0}, _Item) ->\n    false;\nis_element(#bloom{hfs = Hfs, filter = Filter}, Item) ->\n    BFSize = erlang:bit_size(Filter),\n    Positions = ?REP_HFS:apply_val_rem(Hfs, Item, BFSize),\n    check_bits(Filter, Positions).\n\n%% @doc Returns all items present in the bloom filter.\n-spec filter(bloom_filter(), Items::[key()]) -> [key()].\nfilter(#bloom{items_count = 0}, _Items) ->\n    [];\nfilter(#bloom{hfs = Hfs, filter = Filter}, Items) ->\n    BFSize = erlang:bit_size(Filter),\n    [Item || Item <- Items,\n             check_bits(Filter, ?REP_HFS:apply_val_rem(Hfs, Item, BFSize))].\n\n%% @doc Returns all items not present in the bloom filter.\n-spec filter_neg(bloom_filter(), Items::[key()]) -> [key()].\nfilter_neg(#bloom{items_count = 0}, Items) ->\n    Items;\nfilter_neg(#bloom{hfs = Hfs, filter = Filter}, Items) ->\n    BFSize = erlang:bit_size(Filter),\n    [Item || Item <- Items,\n             not check_bits(Filter, ?REP_HFS:apply_val_rem(Hfs, Item, BFSize))].\n\n%% @doc Returns all items not present in the bloom filter and adds all items\n%%      to a Bloom filter of the same structure in parallel.\n%%      This way, the bit positions do not need to be determined twice\n%%      (using costly hash functions) compared to executing these two\n%%      functions separately.\n%% @see filter_neg/2\n%% @see add_list/2\n-spec filter_neg_and_add(BFFilter::bloom_filter(), Items::[key()],\n                         BFAdd::bloom_filter()) -> {[key()], BFAdd2::bloom_filter()}.\nfilter_neg_and_add(#bloom{items_count = 0, hfs = Hfs, filter = Filter}, Items,\n                   BFAdd = #bloom{hfs = Hfs, filter = FilterAdd}) ->\n    ?ASSERT(erlang:bit_size(Filter) =:= erlang:bit_size(FilterAdd)),\n    {Items, bloom:add_list(BFAdd, Items)};\nfilter_neg_and_add(#bloom{hfs = Hfs, filter = Filter}, Items,\n                   BFAdd = #bloom{hfs = Hfs, filter = FilterAdd, items_count = FilledCountAdd}) ->\n    BFSize = erlang:bit_size(Filter),\n    BFSize = erlang:bit_size(FilterAdd), % needs to be equal!\n    IPositions = [{Item, ?REP_HFS:apply_val_rem(Hfs, Item, BFSize)} || Item <- Items],\n    FilteredItems = [Item || {Item, Pos} <- IPositions,\n                             not check_bits(Filter, Pos)],\n    % from p_add_list/4:\n    FilterAdd2 =\n        if Items =:= [] -> FilterAdd;\n           BFSize =:= 1 -> <<1:1>>;\n           true ->\n               Positions = lists:flatmap(fun({_Item, Pos}) -> Pos end, IPositions),\n               p_add_positions(Positions, FilterAdd, BFSize)\n        end,\n    % from add_list/2:\n    ItemsL = length(Items),\n    BFAdd2 = BFAdd#bloom{filter = FilterAdd2, items_count = FilledCountAdd + ItemsL},\n    {FilteredItems, BFAdd2}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Gets the number of items inserted into this bloom filter.\n-spec item_count(bloom_filter()) -> non_neg_integer().\nitem_count(#bloom{items_count = ItemsCount}) -> ItemsCount.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Joins two bloom filter so that the returned bloom filter represents\n%%      their union.\n-spec join(bloom_filter(), bloom_filter()) -> bloom_filter().\njoin(#bloom{items_count = 0, hfs = Hfs1} = _BF1,\n     #bloom{hfs = Hfs2} = BF2) ->\n    ?ASSERT(?REP_HFS:size(Hfs1) =:= ?REP_HFS:size(Hfs2)),\n    ?ASSERT(get_property(BF2, size) =:= get_property(_BF1, size)),\n    BF2;\njoin(#bloom{hfs = Hfs1} = BF1,\n     #bloom{items_count = 0, hfs = Hfs2} = _BF2) ->\n    ?ASSERT(?REP_HFS:size(Hfs1) =:= ?REP_HFS:size(Hfs2)),\n    ?ASSERT(get_property(BF1, size) =:= get_property(_BF2, size)),\n    BF1;\njoin(#bloom{items_count = Items1, filter = F1, hfs = Hfs},\n     #bloom{items_count = Items2, filter = F2}) ->\n    Size = erlang:bit_size(F1),\n    ?ASSERT(Size =:= erlang:bit_size(F2)),\n    NewF = util:bin_or(F1, F2),\n    ?ASSERT(erlang:bit_size(NewF) =:= Size),\n    #bloom{filter = NewF, hfs = Hfs,\n           items_count = Items1 + Items2 %approximation\n           }.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Checks whether two bloom filters are equal.\n-spec equals(bloom_filter(), bloom_filter()) -> boolean().\nequals(#bloom{ items_count = Items1, filter = Filter1 },\n       #bloom{ items_count = Items2, filter = Filter2 }) ->\n    Items1 =:= Items2 andalso\n        Filter1 =:= Filter2.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Returns bloom filter debug information.\n-spec print(bloom_filter()) -> [{atom(), any()}].\nprint(#bloom{filter = Filter, hfs = Hfs, items_count = NumItems} = Bloom) ->\n    Size = erlang:bit_size(Filter),\n    HCount = ?REP_HFS:size(Hfs),\n    [{filter_bit_size, Size},\n     {struct_byte_size, byte_size(term_to_binary(Bloom))},\n     {hash_fun_num, HCount},\n     {items_inserted, NumItems},\n     {act_fpr, get_property(Bloom, fpr)},\n     {compression_rate, ?IIF(NumItems =:= 0, 0.0, Size / NumItems)}].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_property(bloom_filter(), fpr) -> float();\n                  (bloom_filter(), size) -> non_neg_integer();\n                  (bloom_filter(), filter) -> bitstring();\n                  (bloom_filter(), hfs_size) -> non_neg_integer();\n                  (bloom_filter(), hfs) -> ?REP_HFS:hfs();\n                  (bloom_filter(), items_count) -> non_neg_integer().\nget_property(#bloom{filter = Filter, hfs = Hfs, items_count = NumItems}, fpr) ->\n    Size = erlang:bit_size(Filter),\n    calc_FPR(Size, NumItems, ?REP_HFS:size(Hfs));\nget_property(#bloom{filter = Filter}, size)        ->\n    erlang:bit_size(Filter);\nget_property(#bloom{filter = X}     , filter)      -> X;\nget_property(#bloom{hfs = X}        , hfs_size)    -> ?REP_HFS:size(X);\nget_property(#bloom{hfs = X}        , hfs)         -> X;\nget_property(#bloom{items_count = X}, items_count) -> X.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% bit operations\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc Checks whether all bits are set at the given positions.\n-spec check_bits(bitstring(), Positions::[non_neg_integer()]) -> boolean().\n% V1\n%% check_bits(Filter, [Pos | Positions]) ->\n%%     PreBytes = Pos div 8,\n%%     <<_:PreBytes/binary, CheckByte:8, _/binary>> = Filter,\n%%     case 0 =/= CheckByte band (1 bsl (Pos rem 8)) of\n%%         true -> check_bits(Filter, Positions);\n%%         false -> false\n%%     end;\n%% check_bits(_, []) ->\n%%     true.\n\n% V 2 - 12 % faster than V1\ncheck_bits(Filter, [Pos | Positions]) ->\n    case Filter of\n        <<_:Pos/bitstring, 1:1, _/bitstring>> ->\n            check_bits(Filter, Positions);\n        _ -> false\n    end;\ncheck_bits(_, []) ->\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Common bloom filter calculations\n%%\n%%       Definitions:\n%%       N  = BF maximum number of elements\n%%       M  = BF bit size\n%%       FP = probability of a false positive, 0<=FP<=1 (also: FPR)\n%%       k  = number of hash functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Calculates the optimal number of hash functions K and Bloom filter\n%%      size M when K = ln(2)*M/N and M = N * log_2(1/FP) / ln(2)\n%%      with N elements and a false positive probability FP.\n-spec calc_HF_num_Size_opt(N::non_neg_integer(), FP::float())\n        -> {K::pos_integer(), M::pos_integer()}.\ncalc_HF_num_Size_opt(N, FP) ->\n    K0 = - util:log2(FP), % = log_2(1 / FP)\n    K_Min = util:floor(K0),\n    K_Max = util:ceil(K0),\n    M_Min = calc_least_size(N, FP, K_Min),\n    M_Max = calc_least_size(N, FP, K_Max),\n    FPR_Min = calc_FPR(M_Min, N, K_Min),\n    FPR_Max = calc_FPR(M_Max, N, K_Max),\n    % unfortunately, we can't ensure the following two conditions due to\n    % floating point precision issues near 1 in both calc_least_size/3 and calc_FPR/3\n    % instead, use the best fitting option (see below)\n    %?DBG_ASSERT(FPR_Min =< FP),\n    %?DBG_ASSERT(FPR_Max =< FP),\n%%     log:pal(\"Bloom (~g): K=~B, M=~B (FP=~g) vs. K=~B, M=~B (FP=~g)\",\n%%             [FP, K_Min, M_Min, FPR_Min, K_Max, M_Max, FPR_Max]),\n    {K, M, _FPX} = select_best(FP, K_Min, M_Min, FPR_Min,\n                               K_Max, M_Max, FPR_Max),\n    {K, M}.\n\n%% @doc Given two different options to create a Bloom filter with false\n%%      positive rates (or any other measure, e.g. failure rate) FP1 and FP2\n%%      and a target false positive rate FP, selects the best option that has\n%%      FPX =&lt; FP and the lowest MX.\n-spec select_best(FP::float(), K1::pos_integer(), M1::pos_integer(), FP1::float(),\n                  K2::pos_integer(), M2::pos_integer(), FP2::float())\n        -> {K::pos_integer(), M::pos_integer(), FP::float()}.\nselect_best(FP, K1, M1, FP1, K2, M2, FP2) ->\n    if FP1 =< FP andalso FP2 =< FP ->\n           if M1 < M2 ->\n                  {K1, M1, FP1};\n              M1 =:= M2 andalso FP1 < FP2 ->\n                  {K1, M1, FP1};\n              true ->\n                  {K2, M2, FP2}\n           end;\n       FP1 =< FP -> % andalso FP2 > FP\n           {K1, M1, FP1};\n       FP2 =< FP -> % andalso FP1 > FP\n           {K2, M2, FP2};\n       % all below have FP1 > FP andalso FP2 > FP\n       FP1 < FP2 ->\n           {K1, M1, FP1};\n       FP1 == FP2 andalso M1 =< M2 ->\n           {K1, M1, FP1};\n       true ->\n           {K2, M2, FP2}\n    end.\n\n%% @doc Calculates the number of bits needed by a bloom filter to have a false\n%%      positive probability of FP using K hash functions and up to N-Elements.\n%%      M = 1/(1-(1-(FP)^(1/K))^(1/(KN)))\n-spec calc_least_size(N::non_neg_integer(), FP::float(), K::pos_integer())\n        -> M::pos_integer().\ncalc_least_size(_N, FP, _K) when FP == 0 -> 1;\ncalc_least_size(0, _FP, _K) -> 1;\ncalc_least_size(N, FP, K) ->\n    % util:ceil(1 / (1 - math:pow(1 - math:pow(FP, 1 / K), 1 / (K * N))))\n    util:ceil(1 / util:pow1p(1 - math:pow(FP, 1 / K), 1 / (K * N))).\n\n%% @doc Calculates FP for an M-bit large bloom filter with K hash funtions\n%%      and a maximum number of N elements.\n%%      FP = (1-(1-1/M)^(K*N))^K\n-spec calc_FPR(M::pos_integer(), N::non_neg_integer(), K::pos_integer())\n        -> FP::float().\ncalc_FPR(_M, 0, _K) ->\n    0.0;\ncalc_FPR(1, _N, _K) ->\n    1.0;\ncalc_FPR(M, N, K) ->\n    %math:pow(1 - math:pow(1 - 1/M, K * N), K).\n    % more precise version taking floating point precision near 1 into acccount:\n    Inner = -math:exp(K * N * util:log1p(-1 / M)),\n    if Inner == -1.0 -> 0.0;\n       true -> math:exp(K * util:log1p(Inner))\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% helper functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc Increases Val until Val rem Div == 0.\n-spec resize(Val::non_neg_integer(), Div::pos_integer()) -> NewVal::non_neg_integer().\nresize(Val, Div) ->\n    case Val rem Div of\n        0   -> Val;\n        Rem -> Val + Div - Rem\n    end.\n"
  },
  {
    "path": "src/rrepair/cbf.erl",
    "content": "% @copyright 2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @doc    Counting Bloom Filter implementation\n%% @end\n%% @reference D. Guo, M. Li\n%%          <em>Set Reconciliation via Counting Bloom Filters</em>\n%%          2013 IEEE Transactions on Knowledge and Data Engineering 25.10\n%% @version $Id$\n-module(cbf).\n-author('kruber@zib.de').\n-author('mlange@informatik.hu-berlin.de').\n\n-include(\"record_helpers.hrl\").\n-include(\"scalaris.hrl\").\n\n-define(REP_HFS, hfs_plain). % hash function set implementation to use\n\n-export([new_fpr/2, new_fpr/3, new_bpi/3, new_bin/3, new/2,\n         add/2, add_list/2, remove/2, remove_list/2,\n         is_element/2, item_count/1,\n         to_bloom/1]).\n-export([equals/2, join/2, minus/2, print/1]).\n\n% for tests:\n-export([get_property/2]).\n-export([p_add_list/4]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Types\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-record(cbf, {\n                filter        = ?required(cbf, filter) :: array:array(integer()),\n                hfs           = ?required(cbf, hfs)    :: ?REP_HFS:hfs(),    %HashFunctionSet\n                items_count   = 0                      :: non_neg_integer()  %number of inserted items\n               }).\n-opaque cbf() :: #cbf{}.\n-type key() :: any().\n\n-export_type([cbf/0, key/0]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Creates a new counting bloom filter with the default (optimal) hash\n%%      function set based on the given false positive rate.\n-spec new_fpr(MaxItems::non_neg_integer(), FPR::float()) -> cbf().\nnew_fpr(MaxItems, FPR) ->\n    {K, Size} = bloom:calc_HF_num_Size_opt(MaxItems, FPR),\n    new(Size, ?REP_HFS:new(K)).\n\n%% @doc Creates a new counting bloom filter with the given hash function set\n%%      based on the given false positive rate.\n-spec new_fpr(MaxItems::non_neg_integer(), FPR::float(), ?REP_HFS:hfs() | non_neg_integer())\n        -> cbf().\nnew_fpr(MaxItems, FPR, Hfs) ->\n    Size = bloom:calc_least_size(MaxItems, FPR, ?REP_HFS:size(Hfs)),\n    new(Size, Hfs).\n\n%% @doc Creates a new counting bloom filter with the given hash function set and\n%%      a fixed number of positions (bits in standard bloom filters) per item.\n-spec new_bpi(MaxItems::non_neg_integer(), BitsPerItem::number(), ?REP_HFS:hfs() | non_neg_integer())\n        -> cbf().\nnew_bpi(MaxItems, BitPerItem, Hfs) ->\n    new(util:ceil(BitPerItem * MaxItems), Hfs).\n\n%% @doc Creates a new counting bloom filter with the given binary, hash\n%%      function set and item count.\n-spec new_bin(Filter::array:array(integer()), ?REP_HFS:hfs() | non_neg_integer(), ItemsCount::non_neg_integer())\n        -> cbf().\nnew_bin(Filter, HfCount, ItemsCount) when is_integer(HfCount) ->\n    new_bin(Filter, ?REP_HFS:new(HfCount), ItemsCount);\nnew_bin(Filter, Hfs, ItemsCount) ->\n    #cbf{filter = Filter, hfs = Hfs, items_count = ItemsCount}.\n\n%% @doc Creates a new counting bloom filter.\n-spec new(BitSize::pos_integer(), ?REP_HFS:hfs() | non_neg_integer()) -> cbf().\nnew(BitSize, HfCount) when is_integer(HfCount) ->\n    new(BitSize, ?REP_HFS:new(HfCount));\nnew(BitSize, Hfs) ->\n    #cbf{filter = array:new(BitSize, {default,0}), hfs = Hfs, items_count = 0}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Adds one item to the counting bloom filter.\n-spec add(cbf(), key()) -> cbf().\nadd(#cbf{hfs = Hfs, items_count = FilledCount,\n         filter = Filter} = Bloom, Item) ->\n    BFSize = array:size(Filter),\n    Bloom#cbf{filter = p_add_list(Hfs, BFSize, Filter, [Item]),\n              items_count = FilledCount + 1}.\n\n%% @doc Adds multiple items to the counting bloom filter.\n-spec add_list(cbf(), [key()]) -> cbf().\nadd_list(#cbf{hfs = Hfs,\n              items_count = FilledCount,\n              filter = Filter\n             } = Bloom, Items) ->\n    BFSize = array:size(Filter),\n    ItemsL = length(Items),\n    F = p_add_list(Hfs, BFSize, Filter, Items),\n    Bloom#cbf{filter = F, items_count = FilledCount + ItemsL}.\n\n-compile({inline, [p_add_list/4, p_change_list_/6]}).\n\n%% @doc Helper to add items to the counting bloom filter.\n-spec p_add_list(Hfs::?REP_HFS:hfs(), BFSize::pos_integer(),\n                 BF1::array:array(integer()), Items::[key()])\n-> BF2::array:array(integer()).\np_add_list(_Hfs, _BFSize, BF, []) -> BF;\np_add_list(Hfs, 1, BF, _Items = [_|_]) -> array:set(0, ?REP_HFS:size(Hfs), BF);\np_add_list(Hfs, BFSize, BF, Items = [_|_]) ->\n    Positions = lists:flatmap(fun(Item) ->\n                                      ?REP_HFS:apply_val_rem(Hfs, Item, BFSize)\n                              end, Items),\n    [Pos | Rest] = lists:sort(Positions),\n    p_change_list_(Rest, Pos, [1 | lists:duplicate(Pos, 0)],\n                   BFSize, BF, fun erlang:'+'/2).\n\n%% @doc Helper increasing or decreasing counters by first accumulating all\n%%      counters in a list and merging it with the old list.\n-spec p_change_list_(Positions::[non_neg_integer()], CurPos::non_neg_integer(),\n                     Counters::[non_neg_integer(),...], BFSize::non_neg_integer(),\n                     BF::array:array(integer()),\n                     ChangeFun::fun((non_neg_integer(), non_neg_integer())\n                                   -> integer()))\n-> BF2::array:array(integer()).\np_change_list_([], CurPos, Counters, BFSize, BF, ChangeFun) ->\n    Counters1 = lists:reverse(Counters, lists:duplicate(erlang:max(0, BFSize - CurPos - 1), 0)),\n    Counters2 = lists:zipwith(ChangeFun, array:to_list(BF), Counters1),\n    array:from_list(Counters2);\np_change_list_([CurPos | Positions], CurPos, [CurCount | Counters], BFSize, BF, ChangeFun) ->\n    p_change_list_(Positions, CurPos, [CurCount + 1 | Counters], BFSize, BF, ChangeFun);\np_change_list_([NewPos | Positions], CurPos, Counters, BFSize, BF, ChangeFun) ->\n    Counters1 = lists:duplicate(NewPos - CurPos - 1, 0) ++ Counters,\n    p_change_list_(Positions, NewPos, [1 | Counters1], BFSize, BF, ChangeFun).\n\n%% @doc Removes one item from the counting bloom filter.\n%%      (may introduce false negatives if removing an item not added previously)\n-spec remove(cbf(), key()) -> cbf().\nremove(#cbf{hfs = Hfs, items_count = FilledCount,\n              filter = Filter} = Bloom, Item) ->\n    BFSize = array:size(Filter),\n    Bloom#cbf{filter = p_remove_list(Hfs, BFSize, Filter, [Item]),\n              items_count = FilledCount + 1}.\n\n%% @doc Removes multiple items from the counting bloom filter.\n%%      (may introduce false negatives if removing an item not added previously)\n-spec remove_list(cbf(), [key()]) -> cbf().\nremove_list(#cbf{hfs = Hfs,\n                 items_count = FilledCount,\n                 filter = Filter\n                } = Bloom, Items) ->\n    BFSize = array:size(Filter),\n    ItemsL = length(Items),\n    F = p_remove_list(Hfs, BFSize, Filter, Items),\n    Bloom#cbf{filter = F, items_count = FilledCount + ItemsL}.\n\n-compile({inline, [p_remove_list/4]}).\n\n%% @doc Helper to remove items from the counting bloom filter.\n-spec p_remove_list(Hfs::?REP_HFS:hfs(), BFSize::pos_integer(),\n                    BF1::array:array(integer()), Items::[key()])\n        -> BF2::array:array(integer()).\np_remove_list(_Hfs, _BFSize, BF, []) -> BF;\np_remove_list(_Hfs, 1, BF, _Items = [_|_]) -> array:set(0, 0, BF);\np_remove_list(Hfs, BFSize, BF, Items = [_|_]) ->\n    Positions = lists:flatmap(fun(Item) ->\n                                      ?REP_HFS:apply_val_rem(Hfs, Item, BFSize)\n                              end, Items),\n    [Pos | Rest] = lists:sort(Positions),\n    p_change_list_(Rest, Pos, [1 | lists:duplicate(Pos, 0)],\n                   BFSize, BF, fun erlang:'-'/2).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Returns true if the counting bloom filter contains this item.\n-spec is_element(cbf(), key()) -> boolean().\nis_element(#cbf{items_count = 0}, _Item) ->\n    false;\nis_element(#cbf{hfs = Hfs, filter = Filter}, Item) ->\n    BFSize = array:size(Filter),\n    Positions = ?REP_HFS:apply_val_rem(Hfs, Item, BFSize),\n    check_counters(Filter, Positions).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Gets the number of items inserted into this counting bloom filter.\n-spec item_count(cbf()) -> non_neg_integer().\nitem_count(#cbf{items_count = ItemsCount}) -> ItemsCount.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Joins two counting bloom filters so that the returned counting bloom\n%%      filter represents their union.\n-spec join(cbf(), cbf()) -> cbf().\njoin(#cbf{items_count = 0, hfs = Hfs1} = _BF1,\n     #cbf{hfs = Hfs2} = BF2) ->\n    ?ASSERT(?REP_HFS:size(Hfs1) =:= ?REP_HFS:size(Hfs2)),\n    ?ASSERT(get_property(BF2, size) =:= get_property(_BF1, size)),\n    BF2;\njoin(#cbf{hfs = Hfs1} = BF1,\n     #cbf{items_count = 0, hfs = Hfs2} = _BF2) ->\n    ?ASSERT(?REP_HFS:size(Hfs1) =:= ?REP_HFS:size(Hfs2)),\n    ?ASSERT(get_property(BF1, size) =:= get_property(_BF2, size)),\n    BF1;\njoin(#cbf{items_count = Items1, filter = F1, hfs = Hfs},\n     #cbf{items_count = Items2, filter = F2}) ->\n    Size = array:size(F1),\n    ?ASSERT(Size =:= array:size(F2)),\n    if Items1 > Items2 ->\n           FSmall = F2, FBig = F1, ok;\n       true ->\n           FSmall = F1, FBig = F2, ok\n    end,\n    NewF = array:sparse_foldl(fun(I, X, Acc) ->\n                                      array:set(I, array:get(I, Acc) + X, Acc)\n                              end, FBig, FSmall),\n    #cbf{filter = NewF, hfs = Hfs,\n         items_count = Items1 + Items2 %approximation\n        }.\n\n%% @doc Subtracts counting bloom filter A from B so that the returned\n%%      counting bloom filter that approximates the set difference (with false\n%%      positives and false negatives!).\n-spec minus(A::cbf(), B::cbf()) -> cbf().\nminus(#cbf{items_count = 0, hfs = Hfs1} = BF1,\n      #cbf{hfs = Hfs2} = _BF2) ->\n    ?ASSERT(?REP_HFS:size(Hfs1) =:= ?REP_HFS:size(Hfs2)),\n    ?ASSERT(get_property(_BF2, size) =:= get_property(BF1, size)),\n    BF1;\nminus(#cbf{hfs = Hfs1} = BF1,\n      #cbf{items_count = 0, hfs = Hfs2} = _BF2) ->\n    ?ASSERT(?REP_HFS:size(Hfs1) =:= ?REP_HFS:size(Hfs2)),\n    ?ASSERT(get_property(BF1, size) =:= get_property(_BF2, size)),\n    BF1;\nminus(#cbf{items_count = Items1, filter = F1, hfs = Hfs},\n      #cbf{items_count = Items2, filter = F2}) ->\n    Size = array:size(F1),\n    ?ASSERT(Size =:= array:size(F2)),\n    NewF = array:sparse_foldl(fun(I, X, Acc) ->\n                                      array:set(I, array:get(I, Acc) - X, Acc)\n                              end, F1, F2),\n    #cbf{filter = NewF, hfs = Hfs,\n         items_count = Items1 - Items2 %approximation\n        }.\n\n-spec to_bloom(cbf()) -> bloom:bloom_filter().\nto_bloom(#cbf{items_count = 0, filter = F, hfs = Hfs}) ->\n    bloom:new(array:size(F), Hfs);\nto_bloom(#cbf{items_count = ItemsCount, filter = F, hfs = Hfs}) ->\n%%     % note: this is faster than using array:sparse_fold[r,l]/3 to get the positions\n%%     Positions0 = array:sparse_to_list(array:sparse_map(fun(I, _) -> I end, F)),\n%%     % position 0 may be missing since 0 is the default value, too\n%%     Positions = case array:get(0, F) of\n%%                     0 -> Positions0;\n%%                     _ -> [0 | Positions0]\n%%                 end,\n    % this seems even faster:\n    Positions = lists:reverse(\n                  element(2, lists:foldl(\n                            fun(0, {N, L}) -> {N+1, L};\n                               (_, {N, L}) -> {N+1, [N | L]}\n                            end, {0, []}, array:to_list(F)))),\n    BFSize = array:size(F),\n    BFBits = bloom:p_add_positions(Positions, <<0:BFSize>>, BFSize),\n    bloom:new_bin(BFBits, Hfs, ItemsCount).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Checks whether two counting bloom filters are equal.\n-spec equals(cbf(), cbf()) -> boolean().\nequals(#cbf{ items_count = Items1, filter = Filter1 },\n       #cbf{ items_count = Items2, filter = Filter2 }) ->\n    Items1 =:= Items2 andalso\n        Filter1 =:= Filter2.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Returns counting bloom filter debug information.\n-spec print(cbf()) -> [{atom(), any()}].\nprint(#cbf{filter = Filter, hfs = Hfs, items_count = NumItems} = Bloom) ->\n    Size = array:size(Filter),\n    HCount = ?REP_HFS:size(Hfs),\n    [{filter_size, Size},\n     {filter_byte_size, byte_size(term_to_binary(Filter, [compressed]))},\n     {filter_as_list_byte_size, byte_size(term_to_binary(array:to_list(Filter), [compressed]))},\n     {hash_fun_num, HCount},\n     {items_inserted, NumItems},\n     {act_fpr, get_property(Bloom, fpr)},\n     {compression_rate, ?IIF(NumItems =:= 0, 0.0, Size / NumItems)}].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_property(cbf(), fpr) -> float();\n                  (cbf(), size) -> non_neg_integer();\n                  (cbf(), filter) -> array:array(integer());\n                  (cbf(), hfs_size) -> non_neg_integer();\n                  (cbf(), hfs) -> ?REP_HFS:hfs();\n                  (cbf(), items_count) -> non_neg_integer().\nget_property(#cbf{filter = Filter, hfs = Hfs, items_count = NumItems}, fpr) ->\n    Size = array:size(Filter),\n    bloom:calc_FPR(Size, NumItems, ?REP_HFS:size(Hfs));\nget_property(#cbf{filter = Filter}, size)        ->\n    array:size(Filter);\nget_property(#cbf{filter = X}     , filter)      -> X;\nget_property(#cbf{hfs = X}        , hfs_size)    -> ?REP_HFS:size(X);\nget_property(#cbf{hfs = X}        , hfs)         -> X;\nget_property(#cbf{items_count = X}, items_count) -> X.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% bit/counter position operations\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc Checks whether all counters are non-zero at the given positions.\n-spec check_counters(array:array(integer()), Positions::[non_neg_integer()]) -> boolean().\ncheck_counters(Filter, [Pos | Positions]) ->\n    array:get(Pos, Filter) =/= 0 andalso check_counters(Filter, Positions);\ncheck_counters(_, []) ->\n    true.\n"
  },
  {
    "path": "src/rrepair/hfs_beh.erl",
    "content": "% @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @doc    HashFunctionSet Behaviour\n%% @end\n%% @version $Id$\n-module(hfs_beh).\n-author('malange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n-ifndef(have_callback_support).\n-export([behaviour_info/1]).\n-endif.\n-export([tester_create_hfs_fun/1]).\n\n-export_type([hfs_fun/0]).\n\n-ifdef(have_callback_support).\n-type itemKey() :: any().\n-type hfs()     :: term().\n-callback new(pos_integer()) -> hfs().\n-callback apply_val(hfs(), itemKey()) -> [non_neg_integer(),...].\n-callback apply_val(hfs(), pos_integer(), itemKey()) -> non_neg_integer().\n-callback apply_val_rem(hfs(), itemKey(), pos_integer()) -> [non_neg_integer(),...].\n-callback size(hfs()) -> pos_integer().\n-else.\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n    [\n     {new, 1},\n     {apply_val, 2},\n     {apply_val, 3},\n     {apply_val_rem, 3},\n     {size, 1}\n    ];\nbehaviour_info(_Other) ->\n    undefined.\n-endif.\n\n-type hfs_fun() :: {fun((binary()) -> non_neg_integer() | binary()),\n                    HashBits::pos_integer()}.\n\n-spec tester_create_hfs_fun(1..3) -> hfs_fun().\ntester_create_hfs_fun(1) -> {fun erlang:adler32/1, 32};\ntester_create_hfs_fun(2) -> {fun erlang:md5/1, 128};\ntester_create_hfs_fun(3) -> {fun(X) -> ?CRYPTO_SHA(X) end, 160}.\n"
  },
  {
    "path": "src/rrepair/hfs_lhsp.erl",
    "content": "% @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @doc    Less hashing, same performance hash function set container\n%% @end\n%% @reference\n%%         Implementation of a hash function set proposed in\n%%         2006 by A. Kirsch, M. Mitzenmacher -\n%%         \"Less Hashing, Same Performance: Building a Better Bloom Filter\n%%         Build k Hash functions of the form g_i(x) = h_1(X) + i * h_2(X)\n%%\n%%         Used MD5 Hash-Function like in\n%%         2000 - L.Fan, P. Cao., JA, ZB :\n%%               \"Summary Cache: A Scalable Wide-Area Web Cache Sharing Protocol\"\n%%               (Counting Bloom Filters Paper)\n%% @version $Id$\n-module(hfs_lhsp).\n-author('malange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n% types\n-behaviour(hfs_beh).\n\n-type itemKey() :: any().\n-opaque hfs()   :: {hfs_lhsp, Hf_count::pos_integer(), H1_fun::hfs_beh:hfs_fun(), H2_fun::hfs_beh:hfs_fun()}.\n\n-export_type([hfs/0]).\n\n-export([new/1, apply_val/2, apply_val/3, apply_val_rem/3]).\n-export([size/1]).\n\n% for tester:\n-export([apply_val_feeder/3,\n         tester_create_hfs/1]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% API functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc returns a new lhsp hfs with default functions\n-spec new(pos_integer()) -> hfs().\nnew(HFCount) ->\n    {hfs_lhsp, HFCount, {fun erlang:adler32/1, 32}, {fun erlang:md5/1, 128}}.\n\n% @doc Applies Val to all hash functions in container HC\n-spec apply_val(hfs(), itemKey()) -> [non_neg_integer(),...].\napply_val({hfs_lhsp, K, H1, H2}, Val) ->\n    ValBin = erlang:term_to_binary(Val),\n    HV1 = hash_value(ValBin, H1),\n    if K > 1 ->\n           HV2 = hash_value(ValBin, H2),\n           apply_val_helper(K - 1, HV2, [HV1]);\n       true ->\n           [HV1]\n    end.\n\n-compile({nowarn_unused_function, apply_val_helper_feeder/3}).\n\n-spec apply_val_helper_feeder(\n        Hf_count::1..100, HashValue2::non_neg_integer(), Acc::Hashes)\n        -> {Hf_count::1..100, HashValue2::non_neg_integer(), Acc::Hashes}\n        when is_subtype(Hashes, [non_neg_integer(),...]).\napply_val_helper_feeder(HF_count, HV2, Acc) ->\n    {HF_count, HV2, Acc}.\n\n%% @doc Helper for apply_val/2.\n-spec apply_val_helper(Hf_count::pos_integer(), HashValue2::non_neg_integer(),\n                       Acc::Hashes) -> Hashes\n        when is_subtype(Hashes, [non_neg_integer(),...]).\napply_val_helper(0, _HV2, Acc) ->\n    Acc;\napply_val_helper(N, HV2, [H|_] = L) ->\n    apply_val_helper(N - 1, HV2, [H + HV2 | L]).\n\n%% @doc Applies Val to all hash functions in container HC and returns only\n%%      remainders of divisions by Rem.\n-spec apply_val_rem(HC::hfs(), Val::itemKey(), Rem::2..16#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF)\n        -> [non_neg_integer(),...].\napply_val_rem({hfs_lhsp, K, H1, H2}, Val, Rem) when Rem > 1 ->\n    ValBin = erlang:term_to_binary(Val),\n    HV1 = hash_value(ValBin, H1) rem Rem,\n    % TODO: what if both hashes are 0?\n    if K > 1 ->\n           HV2 = hash_value(ValBin, H2) rem Rem,\n           if HV2 =:= 0 ->\n                  apply_val_rem_helper(K - 1, HV1, Rem, [HV2]);\n              true ->\n                  apply_val_rem_helper(K - 1, HV2, Rem, [HV1])\n           end;\n       true ->\n           [HV1]\n    end.\n\n-compile({nowarn_unused_function, apply_val_rem_helper_feeder/4}).\n\n-spec apply_val_rem_helper_feeder(\n        Hf_count::1..100, HashValue2::non_neg_integer(), Rem::pos_integer(),\n        Acc::Hashes)\n        -> {Hf_count::1..100, HashValue2::non_neg_integer(), Rem::pos_integer(),\n            Acc::Hashes}\n        when is_subtype(Hashes, [non_neg_integer(),...]).\napply_val_rem_helper_feeder(HF_count, HV2, Rem, Acc) ->\n    {HF_count, HV2, Rem, Acc}.\n\n%% @doc Helper for apply_val_rem/3.\n-spec apply_val_rem_helper(Hf_count::pos_integer(), HashValue2::non_neg_integer(),\n                           Rem::pos_integer(), Acc::Hashes) -> Hashes\n        when is_subtype(Hashes, [non_neg_integer(),...]).\napply_val_rem_helper(0, _HV2, _Rem, Acc) ->\n    Acc;\napply_val_rem_helper(N, HV2, Rem, [H|_] = L) ->\n    apply_val_rem_helper(N - 1, HV2, Rem, [(H + HV2) rem Rem | L]).\n\n-spec apply_val_feeder(hfs(), pos_integer(), itemKey())\n        -> {hfs(), pos_integer(), itemKey()}.\napply_val_feeder({hfs_lhsp, K, H1, H2}, I, Val) ->\n    {{hfs_lhsp, K, H1, H2}, erlang:min(K, I), Val}.\n    \n%% @doc Apply hash function I to given value; I = 1..hfs_size.\n%%      NOTE: When multiple different I are needed, prefer apply_val/2 since\n%%            that function is faster.\n-spec apply_val(hfs(), pos_integer(), itemKey()) -> non_neg_integer().\napply_val({hfs_lhsp, K, H1, H2}, I, Val) when I =< K ->\n    ValBin = erlang:term_to_binary(Val),\n    HV1 = hash_value(ValBin, H1),\n    if I > 1 ->\n           HV2 = hash_value(ValBin, H2),\n           HV1 + (I - 1) * HV2;\n       true ->\n           HV1\n    end.\n\n%% @doc Returns number of hash functions in the container\n-spec size(hfs()) -> pos_integer().\nsize({hfs_lhsp, K, _, _}) ->\n    K.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% private functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-compile({inline, [hash_value/2]}).\n\n-spec hash_value(binary(), hfs_beh:hfs_fun()) -> non_neg_integer().\nhash_value(Val, {HashFun, HashBits}) ->\n    H = HashFun(Val),\n    if erlang:is_binary(H) ->\n           <<R:HashBits>> = H,\n           R;\n       true -> H\n    end.\n\n-spec tester_create_hfs({hfs_lhsp, Hf_count::1..100, H1_fun::hfs_beh:hfs_fun(),\n                         H2_fun::hfs_beh:hfs_fun()}) -> hfs().\ntester_create_hfs({hfs_lhsp, Hf_count, H1_fun, H2_fun}) ->\n    {hfs_lhsp, Hf_count, H1_fun, H2_fun}.\n"
  },
  {
    "path": "src/rrepair/hfs_plain.erl",
    "content": "% @copyright 2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Plain hashing, k-times using md5\n%% @end\n%% @version $Id$\n-module(hfs_plain).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n% types\n-behaviour(hfs_beh).\n\n-type itemKey() :: any().\n-opaque hfs()   :: {hfs_plain, Hf_count::pos_integer(), HashFun::hfs_beh:hfs_fun()}.\n\n-export_type([hfs/0]).\n\n-export([new/1, apply_val/2, apply_val/3, apply_val_rem/3]).\n-export([size/1]).\n\n% for tester:\n-export([apply_val_feeder/3, apply_val_rem_feeder/3,\n         tester_create_hfs/1]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% API functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc returns a new hfs with default functions\n-spec new(pos_integer()) -> hfs().\nnew(HFCount) ->\n    {hfs_plain, HFCount, {fun(X) -> ?CRYPTO_SHA(X) end, 160}}.\n\n%% @doc Hashes Val K-times as defined by the HFS.\n-spec apply_val(hfs(), itemKey()) -> [non_neg_integer(),...].\napply_val({hfs_plain, K, {HashFun, HashBits}}, Val) ->\n    lists:flatten(\n      util:for_to_ex(1, K,\n                     fun(I) ->\n                             hash_value(erlang:term_to_binary({Val, I}),\n                                        HashFun, HashBits, 1)\n                     end)).\n\n-spec apply_val_rem_feeder(hfs(), itemKey(), Rem::pos_integer())\n        -> {hfs(), itemKey(), Rem::pos_integer()}.\napply_val_rem_feeder({hfs_plain, _K, {_HashFun, HashBits}} = HFS, Val, Rem) ->\n    {HFS, Val, erlang:max(2, Rem rem (1 bsl HashBits))}.\n\n%% @doc Hashes Val K-times as defined by the HFS and returns only\n%%      remainders of divisions by Rem.\n-spec apply_val_rem(hfs(), Val::itemKey(), Rem::2..16#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF)\n        -> [non_neg_integer(),...].\napply_val_rem({hfs_plain, K, {HashFun, HashBits}}, Val, Rem) when Rem > 1 ->\n    RemBits = util:ceil(util:log2(Rem)),\n    Partitions = HashBits div RemBits,\n    ?ASSERT2(Partitions > 0, 'hashfun not suitable for chosen remainder'), %% too few bits\n    HashBitsEach = HashBits div Partitions,\n    ?DBG_ASSERT(HashBitsEach > 0),\n    Hashes =\n        lists:flatten(\n          util:for_to_ex(1, (K + Partitions - 1) div Partitions,\n                         fun(I) ->\n                                 L = hash_value(\n                                       erlang:term_to_binary({Val, I}),\n                                       HashFun, HashBitsEach, Partitions),\n                                 [X rem Rem || X <- L]\n                         end)),\n    element(1, lists:split(K, Hashes)).\n\n-spec apply_val_feeder(hfs(), pos_integer(), itemKey())\n        -> {hfs(), pos_integer(), itemKey()}.\napply_val_feeder({hfs_plain, K, _HashFun} = HFS, I, Val) ->\n    {HFS, erlang:min(K, I), Val}.\n    \n%% @doc Hashes Val with the I'th hash function as defined by the HFS.\n%%      (I = 1..hfs_size).\n%%      NOTE: When multiple different I are needed, prefer apply_val/2 since\n%%            that function is faster.\n-spec apply_val(hfs(), pos_integer(), itemKey()) -> non_neg_integer().\napply_val({hfs_plain, K, {HashFun, HashBits}}, I, Val) when I =< K ->\n    [H] = hash_value(erlang:term_to_binary({Val, I}), HashFun, HashBits, 1),\n    H.\n\n%% @doc Returns number of hash functions in the container\n-spec size(hfs()) -> pos_integer().\nsize({hfs_plain, K, _HashFun}) ->\n    K.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% private functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-compile({inline, [hash_value/4, split_bin/3]}).\n\n-spec hash_value(binary(), fun((binary()) -> non_neg_integer() | binary()),\n                 HashBitsEach::pos_integer(), Partitions::pos_integer())\n        -> [non_neg_integer()].\nhash_value(Val, HashFun, HashBitsEach, Partitions) ->\n    H = HashFun(Val),\n    if erlang:is_binary(H) ->\n           split_bin(H, Partitions, HashBitsEach);\n       true ->\n           split_bin(<<H:(Partitions*HashBitsEach)>>, Partitions, HashBitsEach)\n    end.\n\n%% @doc Splits Bin into Remaining chunks of Size each.\n-spec split_bin(Bin::bitstring(), Remaining::non_neg_integer(),\n                Size::pos_integer()) -> [non_neg_integer()].\nsplit_bin(_Bin, 0, _Size) -> [];\nsplit_bin(Bin, Remaining, Size) ->\n    <<R1:Size, Rest/bitstring>> = Bin,\n    [R1 | split_bin(Rest, Remaining - 1, Size)].\n\n-spec tester_create_hfs({hfs_plain, Hf_count::1..100, HashFun::hfs_beh:hfs_fun()}) -> hfs().\ntester_create_hfs({hfs_plain, Hf_count, HashFun}) ->\n    {hfs_plain, Hf_count, HashFun}.\n"
  },
  {
    "path": "src/rrepair/iblt.erl",
    "content": "% @copyright 2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @doc    Invertible Bloom Lookup Table\n%%         Operations: Insert, Delete, Get, ListEntries\n%% @end\n%% @reference\n%%          1) M.T. Goodrich, M. Mitzenmacher\n%%             <em>Invertible Bloom Lookup Tables</em>\n%%             2011 ArXiv e-prints. 1101.2245\n%%          2) D. Eppstein, M.T. Goodrich, F. Uyeda, G. Varghese\n%%             <em>Whats the Difference? Efficient Set Reconciliation without Prior Context</em>\n%%             2011 SIGCOMM'11 Vol.41(4)\n%% @version $Id$\n-module(iblt).\n-author('malange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-include(\"record_helpers.hrl\").\n-include(\"scalaris.hrl\").\n\n-define(REP_HFS, hfs_lhsp). %HashFunctionSet selection for usage by bloom filter\n\n-export([new/2, new/3, insert/3, delete/3, get/2, list_entries/1]).\n-export([is_element/2]).\n-export([get_fpr/1, get_prop/2]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Types\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-export_type([options/0]).\n\n-type value()   :: integer().\n-type cell()    :: {Count       :: non_neg_integer(),\n                    KeySum      :: binary(),\n                    KeyHashSum  :: non_neg_integer(),   %sum c(x) of all inserted keys x, for c = any hashfunction not in hfs\n                    ValSum      :: value(),\n                    ValHashSum  :: non_neg_integer()}.  %sum c(y) of all inserted values y, for c = any hashfunction not in hfs\n\n-type table() :: [] | [{ColNr :: pos_integer(), Cells :: [cell()]}].\n\n-record(iblt, {\n               hfs        = ?required(iblt, hfs) :: ?REP_HFS:hfs(),    %HashFunctionSet\n               table      = []                   :: table(),\n               cell_count = 0                    :: non_neg_integer(),\n               col_size   = 0                    :: non_neg_integer(), %cells per column\n               item_count = 0                    :: non_neg_integer()  %number of inserted items\n               }).\n\n-type iblt() :: #iblt{}.\n\n-type option()         :: prime.\n-type options()        :: [] | [option()].\n-type cell_operation() :: add | remove.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec new(?REP_HFS:hfs() | non_neg_integer(), pos_integer()) -> iblt().\nnew(Hfs, CellCount) ->\n    new(Hfs, CellCount, [prime]).\n\n-spec new(?REP_HFS:hfs() | non_neg_integer(), pos_integer(), options()) -> iblt().\nnew(HfCount, CellCount, Options) when is_integer(HfCount) ->\n    new(?REP_HFS:new(HfCount), CellCount, Options);\nnew(Hfs, CellCount, Options) ->\n    K = ?REP_HFS:size(Hfs),\n    {Cells, ColSize} = case proplists:get_bool(prime, Options) of\n                            true ->\n                                CCS = prime:get_nearest(util:ceil(CellCount / K)),\n                                {CCS * K, CCS};\n                            false ->\n                                RCC = bloom:resize(CellCount, K),\n                                {RCC, erlang:round(RCC / K)}\n                        end,\n    SubTable = [{0, <<0>> ,0, 0, 0} || _ <- lists:seq(1, ColSize)],\n    Table = [ {I, SubTable} || I <- lists:seq(1, K)],\n    #iblt{\n          hfs = Hfs,\n          table = Table,\n          cell_count = Cells,\n          col_size = ColSize,\n          item_count = 0\n          }.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec insert(iblt(), ?RT:key(), value()) -> iblt().\ninsert(IBLT, Key, Value) ->\n    change_table(IBLT, add, Key, Value).\n\n-spec delete(iblt(), ?RT:key(), value()) -> iblt().\ndelete(IBLT, Key, Value) ->\n    change_table(IBLT, remove, Key, Value).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec operation_val(cell_operation()) -> 1 | -1.\noperation_val(add) -> 1;\noperation_val(remove) -> -1.\n\n-spec change_table(iblt(), cell_operation(), ?RT:key(), value()) -> iblt().\nchange_table(#iblt{ hfs = Hfs, table = T, item_count = ItemCount, col_size = ColSize } = IBLT,\n             Operation, Key, Value) ->\n    NT = lists:foldl(\n           fun({ColNr, Col}, NewT) ->\n                   NCol = change_cell(Col,\n                                      ?REP_HFS:apply_val(Hfs, ColNr, Key) rem ColSize,\n                                      encode_key(Key), Value, Operation),\n                   [{ColNr, NCol} | NewT]\n           end, [], T),\n    IBLT#iblt{ table = NT, item_count = ItemCount + operation_val(Operation)}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec change_cell([cell()], pos_integer(), Key::binary(), value(), cell_operation()) -> [cell()].\nchange_cell(Column, CellNr, Key, Value, Operation) ->\n    {HeadL, [Cell | TailL]} = lists:split(CellNr, Column),\n    {Count, KeySum, KHSum, ValSum, VHSum} = Cell,\n    lists:append(HeadL, [{Count + operation_val(Operation),\n                          util:bin_xor(KeySum, Key),\n                          KHSum bxor checksum_fun(Key),\n                          ValSum bxor Value,\n                          VHSum bxor checksum_fun(Value)} | TailL]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get(iblt(), ?RT:key()) -> value() | not_found.\nget(#iblt{ table = T, hfs = Hfs, col_size = ColSize }, Key) ->\n    p_get(T, Hfs, ColSize, Key).\n\n-spec p_get(table(), ?REP_HFS:hfs(), non_neg_integer(), ?RT:key()) -> value() | not_found.\np_get([], _, _, _) -> not_found;\np_get([{ColNr, Col} | T], Hfs, ColSize, Key) ->\n    Cell = lists:nth((?REP_HFS:apply_val(Hfs, ColNr, Key) rem ColSize) + 1, Col),\n    {_, _, _, Val, _} = Cell,\n    case is_pure(Cell) of\n        true -> Val;\n        false -> p_get(T, Hfs, ColSize, Key)\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc a cell is pure if count = -1 or 1, and checksum of key and value correspond to their checksums\n-spec is_pure(cell()) -> boolean().\nis_pure({Count, Key, KeyCheck, Val, ValCheck}) ->\n    (Count =:= 1 orelse Count =:= -1) andalso\n        checksum_fun(Key) =:= KeyCheck andalso\n        checksum_fun(Val) =:= ValCheck.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc lists all correct entries of this structure\n%     correct entries can be retrieved out of pure cells\n%     a pure cell := count = 1 and check_sum(keySum)=keyHashSum and check_sum(valSum)=valHashSum\n-spec list_entries(iblt()) -> [{?RT:key(), value()}].\nlist_entries(IBLT) ->\n    p_list_entries(IBLT, []).\n\n-spec p_list_entries(iblt(), Acc) -> Acc when is_subtype(Acc, [{?RT:key(), value()} | [{?RT:key(), value()}]]).\np_list_entries(#iblt{ table = T } = IBLT, Acc) ->\n    case get_any_entry(T, []) of\n        [] -> lists:flatten(Acc);\n        [_|_] = L ->\n            NewIBLT = lists:foldl(fun({Key, Val}, NT) ->\n                                          delete(NT, Key, Val)\n                                  end, IBLT, L),\n            p_list_entries(NewIBLT, [L, Acc])\n    end.\n\n% tries to find any pure entry\n-spec get_any_entry(table(), [{?RT:key(), value()}]) -> [{?RT:key(), value()}].\nget_any_entry([], Acc) ->\n    Acc;\nget_any_entry([{_, Col} | T], Acc) ->\n    Result = [{decode_key(Key), Val} || {_, Key, _, Val, _} = Cell <- Col,\n                                        is_pure(Cell)],\n    if\n        Result =:= [] -> get_any_entry(T, lists:append(Result, Acc));\n        true -> Result\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec is_element(iblt(), ?RT:key()) -> boolean().\nis_element(#iblt{ hfs = Hfs, table = T, col_size = ColSize }, Key) ->\n    Found = lists:foldl(\n              fun({ColNr, Col}, Count) ->\n                      {C, _, _, _, _} = lists:nth((?REP_HFS:apply_val(Hfs, ColNr, Key) rem ColSize) + 1, Col),\n                      Count + if C > 0 -> 1;\n                                 true -> 0 end\n                      end, 0, T),\n    Found =:= ?REP_HFS:size(Hfs).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc calculates actual false positive rate depending on saturation degree\n-spec get_fpr(iblt()) -> float().\nget_fpr(#iblt{  hfs = Hfs, cell_count = M, item_count = N }) ->\n    K = ?REP_HFS:size(Hfs),\n    math:pow(1 - math:exp((-K * N) / M), K).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_prop(atom(), iblt()) -> any().\nget_prop(Prop, IBLT) ->\n    case Prop of\n        item_count -> IBLT#iblt.item_count;\n        col_size -> IBLT#iblt.col_size;\n        cell_count -> IBLT#iblt.cell_count\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% helpers\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Hash function for checksum building.\n-spec checksum_fun(binary() | integer()) -> non_neg_integer().\nchecksum_fun(X) when is_integer(X) -> erlang:crc32(integer_to_list(X));\nchecksum_fun(X) -> erlang:crc32(X).\n\n-spec encode_key(?RT:key()) -> binary().\nencode_key(Key) ->\n    term_to_binary(Key).\n\n-spec decode_key(binary()) -> ?RT:key().\ndecode_key(BinKey) ->\n    binary_to_term(BinKey).\n"
  },
  {
    "path": "src/rrepair/merkle_tree.erl",
    "content": "% @copyright 2011-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @doc    Merkle tree implementation\n%%         with configurable bucketing, branching and hashing.\n%%         Underlaying tree structure is an n-ary tree.\n%%         After calling gen_hashes the tree is ready to use and sealed.\n%% @end\n%% @version $Id$\n-module(merkle_tree).\n-author('malange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-include(\"record_helpers.hrl\").\n-include(\"scalaris.hrl\").\n\n-export([new/1, new/2, new/3,\n         keys_to_intervals/2,\n         insert/2, insert_list/2,\n         lookup/2, size/1, size_detail/1, gen_hash/1, gen_hash/2,\n         iterator/1, next/1,\n         is_empty/1, is_leaf/1, is_merkle_tree/1,\n         get_bucket/1, get_hash/1, get_interval/1, get_childs/1, get_root/1,\n         get_item_count/1, get_items/1, get_leaf_count/1, get_leaves/1,\n         get_bucket_size/1, get_branch_factor/1,\n         get_opt_bucket_size/3,\n         store_to_DOT/2, store_graph/2]).\n-export([leaf_hash_sha/2]).\n\n% exports for tests\n-export([tester_create_hash_fun/1, tester_create_inner_hash_fun/1]).\n\n-compile({inline, [get_hash/1, get_interval/1, node_size/1, decode_key/1,\n                   run_leaf_hf/4, run_inner_hf/2]}).\n\n%-define(TRACE(X,Y), io:format(\"~w: [~p] \" ++ X ++ \"~n\", [?MODULE, self()] ++ Y)).\n-define(TRACE(X,Y), ok).\n\n%-define(DOT_SHORTNAME_HASH(X), X). % short node hash in exported merkle tree dot-pictures\n-define(DOT_SHORTNAME_HASH(X), X rem 100000000). % last 8 digits\n\n%-define(DOT_SHORTNAME_KEY(X), X). % short node keys in exported merkle tree dot-pictures\n-define(DOT_SHORTNAME_KEY(X), b).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Types\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-export_type([mt_config/0, merkle_tree/0, mt_node/0, mt_node_key/0, mt_size/0,\n              mt_bucket/0]).\n-export_type([mt_iter/0, mt_config_params/0]).\n\n-type mt_node_key()     :: non_neg_integer().\n-type mt_bucket_entry() :: {?RT:key()} | {?RT:key(), any()} | {?RT:key(), any(), any()}. % add more, if needed\n-type mt_bucket()       :: [mt_bucket_entry()].\n-type mt_size()         :: {InnerNodes::non_neg_integer(),\n                            LeafNodes::non_neg_integer(),\n                            EmptyLeafNodes::non_neg_integer(),\n                            Items::non_neg_integer()}.\n-type leaf_hash_fun()   :: fun((mt_bucket(), intervals:interval()) -> binary()).\n-type inner_hash_fun()  :: fun(([mt_node_key(),...]) -> mt_node_key()).\n\n-record(mt_config,\n        {\n         branch_factor  = 2                 :: pos_integer(),   %number of childs per inner node\n         bucket_size    = 24                :: pos_integer(),   %max items in a leaf\n         leaf_hf        = fun leaf_hash_sha/2  :: leaf_hash_fun(), %hash function for leaf signature creation\n         inner_hf       = fun inner_hash_XOR/1 :: inner_hash_fun(),%hash function for inner node signature creation\n         keep_bucket    = false             :: boolean()        %false=bucket will be empty after p_bulk_build; true=bucket will be filled\n         }).\n-type mt_config() :: #mt_config{}.\n%only key value pairs of mt_config allowed:\n-type mt_config_params() :: [{branch_factor, pos_integer()}\n                                 | {bucket_size, pos_integer()}\n                                 | {leaf_hf, leaf_hash_fun()}\n                                 | {inner_hf, inner_hash_fun()}\n                                 | {keep_bucket, boolean()}] | [].\n\n-type mt_leaf() :: { Hash        :: mt_node_key() | nil, %hash of childs/containing items\n                     ItemCount   :: non_neg_integer(),   %number of items in the leaf nodes below or in this node\n                     Bucket      :: mt_bucket(),         %item storage\n                     Interval    :: intervals:interval() %represented interval\n                    }.\n-type mt_inner() :: {Hash        :: mt_node_key() | nil, %hash of childs/containing items\n                     Count       :: non_neg_integer(),   %number of subnodes including itself\n                     ItemCount   :: non_neg_integer(),   %number of items in the leaf nodes below or in this node\n                     Interval    :: intervals:interval(),%represented interval\n                     Child_list  :: [mt_leaf() | mt_inner(),...] %child nodes (in arbitrary order)\n                    }.\n-type mt_node() :: mt_inner() | mt_leaf().\n\n-type mt_iter()     :: [mt_node() | [mt_node()]].\n-type merkle_tree() :: {merkle_tree, mt_config(), Root::mt_node()}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Gets the given merkle tree's bucket size (number of elements in a leaf).\n-spec get_bucket_size(merkle_tree()) -> pos_integer().\nget_bucket_size({merkle_tree, Config, _}) ->\n    Config#mt_config.bucket_size.\n\n%% @doc Gets the given merkle tree's branch factor (max number of children of\n%%      an inner node).\n-spec get_branch_factor(merkle_tree()) -> pos_integer().\nget_branch_factor({merkle_tree, Config, _}) ->\n    Config#mt_config.branch_factor.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Gets the root node of a merkle tree.\n-spec get_root(merkle_tree()) -> mt_node().\nget_root({merkle_tree, _, Root}) -> Root.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Checks whether the merkle tree or tree node has any children or elements.\n-spec is_empty(merkle_tree() | mt_node()) -> boolean().\nis_empty({merkle_tree, _, Root}) -> is_empty(Root);\nis_empty({_H, _ICnt = 0, _Bkt = [], _I}) -> true;\nis_empty({_H, _Cnt, _ICnt, _I, _CL}) -> false;\nis_empty({_H, _ICnt, _Bkt, _I}) -> false. % inner node (must have children!)\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Creates a new empty merkle tree with default params for the given\n%%      interval. Also creates the hash values.\n-spec new(intervals:interval()) -> merkle_tree().\nnew(I) ->\n    new(I, []).\n\n%% @doc Creates a new empty merkle tree with the given params and interval.\n%%      Also creates the hash values.\n%%      ConfParams = list of tuples defined by {config field name, value}\n%%      e.g. [{branch_factor, 32}, {bucket_size, 16}]\n-spec new(intervals:interval(), mt_config_params()) -> merkle_tree().\nnew(I, ConfParams) ->\n    new(I, [], ConfParams).\n\n%% @doc Creates a new merkle tree with the given params, interval and\n%%      entries from EntryList. Also creates the hash values.\n%%      ConfParams = list of tuples defined by {config field name, value}\n%%      e.g. [{branch_factor, 32}, {bucket_size, 16}]\n-spec new(intervals:interval(), EntryList::mt_bucket(), mt_config_params())\n        -> merkle_tree().\nnew(I, EntryList, ConfParams) ->\n    % remove duplicates and speed up keys_to_intervals/2 by sorting\n    KeyList = lists:ukeysort(1, EntryList),\n    Config = build_config(ConfParams),\n    {[Root], _} = build_childs([{I, length(KeyList), KeyList}], Config, [], 0),\n    {merkle_tree, Config, Root}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Checks whether an interval is present in the merkle tree and returns\n%%      the node responsible for it (exact match!).\n-spec lookup(intervals:interval(), merkle_tree()) -> mt_node() | not_found.\nlookup(I, {merkle_tree, _, Root}) ->\n    case intervals:is_subset(I, get_interval(Root)) of\n        true  -> lookup_(I, Root);\n        false -> not_found\n    end.\n\n%% @doc Helper for lookup/2. In contrast to lookup/2, assumes that I is a\n%%      subset of the current node's interval.\n-spec lookup_(intervals:interval(), mt_node()) -> mt_node() | not_found.\nlookup_(I, {_H, _ICnt, _Bkt, I} = Node) ->\n    Node;\nlookup_(_I, {_H, _ICnt, _Bkt, _NodeI}) ->\n    not_found;\nlookup_(I, {_H, _Cnt, _ICnt, I, _CL} = Node) ->\n    Node;\nlookup_(I, {_H, _Cnt, _ICnt, _NodeI, ChildList = [_|_]}) ->\n    case lists:dropwhile(fun(X) ->\n                                 not intervals:is_subset(I, get_interval(X))\n                         end, ChildList) of\n        [IChild | _] -> lookup_(I, IChild);\n        []       -> not_found\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Gets the node's hash value.\n-spec get_hash(merkle_tree() | mt_node()) -> mt_node_key().\nget_hash({merkle_tree, _, Root}) -> get_hash(Root);\nget_hash({Hash, _Cnt, _ICnt, _I, _CL}) -> Hash;\nget_hash({Hash, _ICnt, _Bkt, _I}) -> Hash.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_interval(merkle_tree() | mt_node()) -> intervals:interval().\nget_interval({merkle_tree, _, Root}) -> get_interval(Root);\nget_interval({_H, _Cnt, _ICnt, I, _CL}) -> I;\nget_interval({_H, _ICnt, _Bkt, I}) -> I.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_childs(merkle_tree() | mt_node()) -> [mt_node()].\nget_childs({merkle_tree, _, Root}) -> get_childs(Root);\nget_childs({_H, _Cnt, _ICnt, _I, [_|_] = Childs}) -> Childs;\nget_childs({_H, _ICnt, _Bkt, _I}) -> [].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Returns the number of items in all buckets in or below this node.\n-spec get_item_count(merkle_tree() | mt_node()) -> non_neg_integer().\nget_item_count({merkle_tree, _, Root}) -> get_item_count(Root);\nget_item_count({_H, _Cnt, ICnt, _I, _CL}) -> ICnt;\nget_item_count({_H, ICnt, _Bkt, _I}) -> ICnt.\n\n%% @doc Gets all items in the buckets of the given merkle tree or node list\n%%      (in arbitrary order).\n-spec get_items(merkle_tree() | mt_node() | [mt_node()])\n        -> {Items::mt_bucket(), LeafCount::non_neg_integer()}.\nget_items({merkle_tree, _, Root}) -> get_items(Root);\nget_items({_H, _Cnt, _ICnt, _I, _CL} = N) -> get_items_([N], [], 0);\nget_items({_H, _ICnt, Bkt, _I}) -> {Bkt, 1};\nget_items(Nodes) when is_list(Nodes) -> get_items_(Nodes, [], 0).\n\n%% @doc Helper for get_items/1.\n-spec get_items_(Iter::mt_iter(), Items::mt_bucket(), LCnt::non_neg_integer())\n        -> {Items::mt_bucket(), LeafCount::non_neg_integer()}.\nget_items_(Iter, Items, LCnt) ->\n    case next(Iter) of\n        {Node, Next} ->\n            case is_leaf(Node) of\n                true  -> get_items_(Next, get_bucket(Node) ++ Items, LCnt + 1);\n                false -> get_items_(Next, Items, LCnt)\n            end;\n        none ->\n            {Items, LCnt}\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Returns the number of leafs under a bucket (1 if the requested node is\n%%      a leaf node).\n%%      NOTE: This scans through the whole sub-tree!\n-spec get_leaf_count(merkle_tree() | mt_node()) -> pos_integer().\nget_leaf_count({merkle_tree, _, Root}) -> get_leaf_count(Root);\nget_leaf_count({_H, _Cnt, _ICnt, _I, _CL} = N) -> get_leaf_count_([N], 0);\nget_leaf_count({_H, _ICnt, _Bkt, _I}) -> 1.\n\n%% @doc Helper for get_leaf_count/1.\n-spec get_leaf_count_(Iter::mt_iter(), LCnt::non_neg_integer()) -> pos_integer().\nget_leaf_count_(Iter, LCnt) ->\n    case next(Iter) of\n        {Node, Next} ->\n            get_leaf_count_(Next, LCnt + ?IIF(is_leaf(Node), 1, 0));\n        none -> LCnt\n    end.\n\n%% @doc Gets all leaves in the given merkle tree or node list\n%%      (in arbitrary order).\n-spec get_leaves(merkle_tree() | mt_node() | [mt_node()]) -> [mt_node()].\nget_leaves({merkle_tree, _, Root}) -> get_leaves(Root);\nget_leaves({_H, _Cnt, _ICnt, _I, _CL} = N) -> get_leaves_([N], []);\nget_leaves({_H, _ICnt, _Bkt, _I} = N) -> [N];\nget_leaves(Nodes) when is_list(Nodes) -> get_leaves_(Nodes, []).\n\n%% @doc Helper for get_leaves/1.\n-spec get_leaves_(Iter::mt_iter(), Acc::[mt_node()]) -> [mt_node()].\nget_leaves_(Iter, Acc) ->\n    case next(Iter) of\n        {Node, Next} ->\n            case is_leaf(Node) of\n                true  -> get_leaves_(Next, [Node | Acc]);\n                false -> get_leaves_(Next, Acc)\n            end;\n        none -> Acc\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Checks whether the given merkle_tree or node is a leaf.\n-spec is_leaf(merkle_tree() | mt_node()) -> boolean().\nis_leaf({merkle_tree, _, Root}) -> is_leaf(Root);\nis_leaf({_H, _ICnt, _Bkt, _I}) -> true;\nis_leaf(_) -> false.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc Returns true if given term is a merkle tree otherwise false.\n-spec is_merkle_tree(any()) -> boolean().\nis_merkle_tree({merkle_tree, _, _Root}) -> true;\nis_merkle_tree(_) -> false.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_bucket(merkle_tree() | mt_node()) -> mt_bucket().\nget_bucket({merkle_tree, _, Root}) -> get_bucket(Root);\nget_bucket({_H, _ICnt, Bucket, _I}) -> Bucket;\nget_bucket(_) -> [].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec insert_list(mt_bucket(), merkle_tree()) -> merkle_tree().\ninsert_list(Terms, Tree) ->\n    lists:foldl(fun insert/2, Tree, Terms).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec insert(Key::mt_bucket_entry(), merkle_tree()) -> merkle_tree().\ninsert(Key, {merkle_tree, Config = #mt_config{keep_bucket = true}, Root} = Tree) ->\n    CheckKey = decode_key(Key),\n    case intervals:in(CheckKey, get_interval(Root)) of\n        true -> {merkle_tree, Config, insert_to_node(Key, CheckKey, Root, Config)};\n        false -> Tree\n    end.\n\n%% @doc Helper for insert/2 assuming that CheckKey is in the given node's\n%%      interval.\n-spec insert_to_node(Key::mt_bucket_entry(), CheckKey::?RT:key(),\n                     Node::mt_node(), Config::mt_config())\n        -> NewNode::mt_node().\ninsert_to_node(Key, CheckKey, {_H, ItemCount, Bucket, Interval} = N,\n               #mt_config{ branch_factor = BranchFactor,\n                           bucket_size = BucketSize } = Config) ->\n    if ItemCount < Config#mt_config.bucket_size ->\n           % leaf node will stay leaf node\n           case lists:keymember(element(1, Key), 1, Bucket) of\n               false -> {nil, ItemCount + 1, [Key | Bucket], Interval};\n               _     -> N\n           end;\n       ItemCount =:= BucketSize ->\n           % former leaf node which will become an inner node\n           % (only split here, insert in next iteration)\n           ChildI = intervals:split(Interval, BranchFactor),\n           NewLeafs = [{nil, CX, BX, IX}\n                       || {IX, CX, BX} <- keys_to_intervals(Bucket, ChildI)],\n           ?DBG_ASSERT(length(NewLeafs) =:= BranchFactor),\n           insert_to_node(Key, CheckKey, {nil, 1 + BranchFactor, BucketSize,\n                                          Interval, NewLeafs}, Config)\n    end;\ninsert_to_node(Key, CheckKey, {Hash, Count, ItemCount, Interval,\n                               Childs = [_|_]} = Node, Config) ->\n    % inner node; insert into a child\n    case util:lists_takewith(fun(N) ->\n                                     intervals:in(CheckKey, get_interval(N))\n                             end, Childs) of\n        false ->\n            log:log(\"InsertFailed!\"),\n            Node;\n        {Dest, Rest} ->\n            OldSize = node_size(Dest),\n            NewDest = insert_to_node(Key, CheckKey, Dest, Config),\n            {Hash, Count - OldSize + node_size(NewDest),\n             ItemCount + 1, Interval, [NewDest | Rest]}\n    end.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Builds a merkle node from the given KeyList assuming that the current\n%%      node needs to be split because the KeyList is larger than\n%%      Config#mt_config.bucket_size.\n-spec p_bulk_build(CurNode::mt_node(), mt_config(), KeyList::mt_bucket())\n        -> mt_node().\np_bulk_build({_H, ItemCount, _Bkt, Interval}, Config, KeyList) ->\n    ChildsI = intervals:split(Interval, Config#mt_config.branch_factor),\n    IKList = keys_to_intervals(KeyList, ChildsI),\n    {ChildNodes, NCount} = build_childs(IKList, Config, [], 0),\n    % hash the bucket now:\n    Hash = run_inner_hf(ChildNodes, Config#mt_config.inner_hf),\n    {Hash, 1 + NCount, ItemCount + length(KeyList), Interval, ChildNodes}.\n\n-spec build_childs([{I::intervals:continuous_interval(), Count::non_neg_integer(),\n                     mt_bucket()}],\n                   mt_config(), Acc::[mt_node()], NCount::non_neg_integer())\n        -> {[mt_node()], NCount::non_neg_integer()}.\nbuild_childs([{Interval, Count, Bucket} | T], Config, Acc, NCount) ->\n    Node = if Count > Config#mt_config.bucket_size ->\n                  p_bulk_build({nil, 0, [], Interval}, Config, Bucket);\n              true ->\n                  % hash the bucket now:\n                  {Bucket1, Hash} = run_leaf_hf(Bucket, Interval,\n                                                Config#mt_config.leaf_hf, false),\n                  {Hash, Count,\n                   ?IIF(Config#mt_config.keep_bucket, Bucket1, []), Interval}\n           end,\n    build_childs(T, Config, [Node | Acc], NCount + node_size(Node));\nbuild_childs([], _, Acc, NCount) ->\n    {Acc, NCount}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Assigns hash values to all nodes in the tree.\n-spec gen_hash(merkle_tree()) -> merkle_tree().\ngen_hash(Tree = {merkle_tree, #mt_config{keep_bucket = KeepBucket}, _Root}) ->\n    gen_hash(Tree, not KeepBucket).\n\n%% @doc Assigns hash values to all nodes in the tree and removes the buckets\n%%      afterwards if CleanBuckets is set.\n-spec gen_hash(merkle_tree(), CleanBuckets::boolean()) -> merkle_tree().\ngen_hash({merkle_tree, Config = #mt_config{inner_hf = InnerHf,\n                                           leaf_hf = LeafHf,\n                                           keep_bucket = KeepBucket},\n          Root}, CleanBuckets) ->\n    RootNew = gen_hash_node(Root, InnerHf, LeafHf, KeepBucket, CleanBuckets),\n    {merkle_tree, Config#mt_config{keep_bucket = not CleanBuckets}, RootNew}.\n\n%% @doc Helper for gen_hash/2.\n-spec gen_hash_node(mt_node(), InnerHf::inner_hash_fun(), LeafHf::leaf_hash_fun(),\n                    KeepBucket::boolean(),\n                    CleanBuckets::boolean()) -> mt_node().\ngen_hash_node({_H, Count, ItemCount, Interval, ChildList = [_|_]},\n              InnerHf, LeafHf, OldKeepBucket, CleanBuckets) ->\n    % inner node\n    NewChilds = [gen_hash_node(X, InnerHf, LeafHf, OldKeepBucket,\n                               CleanBuckets) || X <- ChildList],\n    Hash = run_inner_hf(NewChilds, InnerHf),\n    {Hash, Count, ItemCount, Interval, NewChilds};\ngen_hash_node({Hash, _ICnt, _Bkt = [], _I} = N, _InnerHf,\n              _LeafHf, false, _CleanBuckets) when Hash =/= nil ->\n    % leaf node, no bucket contents, keep_bucket false\n    % -> we already hashed the value in p_bulk_build and cannot insert any more\n    %    values\n    N;\ngen_hash_node({_OldHash, ICnt, Bucket, Interval},\n              _InnerHf, LeafHf, true, CleanBuckets) ->\n    % leaf node, keep_bucket true\n    {Bucket1, Hash} = run_leaf_hf(Bucket, Interval, LeafHf, true),\n    {Hash, ICnt, ?IIF(CleanBuckets, [], Bucket1), Interval}.\n\n%% @doc Hashes an inner node based on its childrens' hashes.\n-spec run_inner_hf([mt_node(),...], InnerHf::inner_hash_fun()) -> mt_node_key().\nrun_inner_hf(Childs, InnerHf) ->\n    InnerHf([get_hash(C) || C <- Childs]).\n\n%% @doc Hashes a leaf with the given bucket.\n-spec run_leaf_hf(mt_bucket(), intervals:interval(), LeafHf::leaf_hash_fun(),\n                  MaybeUnsorted::boolean()) -> {mt_bucket(), mt_node_key()}.\nrun_leaf_hf(Bucket0, I, LeafHf, false) ->\n    % either reversed or correct (during bulk build due to keys_to_intervals/2)\n    Bucket = case Bucket0 of\n                 [E1, E2 | _] when element(1, E1) > element(1, E2) ->\n                     lists:reverse(Bucket0);\n                 _ ->\n                     Bucket0\n             end,\n    ?DBG_ASSERT2(lists:ukeysort(1, Bucket) =:= Bucket, {not_sorted, Bucket}),\n    Hash = LeafHf(Bucket, I),\n    Size = erlang:bit_size(Hash),\n    <<SmallHash:Size/integer-unit:1>> = Hash,\n    {Bucket, SmallHash};\nrun_leaf_hf(Bucket0, I, LeafHf, true) ->\n    run_leaf_hf(lists:keysort(1, Bucket0), I, LeafHf, false).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc Returns the total number of nodes in a tree or node (inner nodes and leaves)\n-spec size(merkle_tree() | mt_node()) -> non_neg_integer().\nsize({merkle_tree, _, Root}) -> node_size(Root);\nsize(Node) -> node_size(Node).\n\n-spec node_size(mt_node()) -> non_neg_integer().\nnode_size({_H, Count, _ICnt, _I, _CL = [_|_]}) ->\n    Count;\nnode_size({_H, _ICnt, _Bkt, _I}) ->\n    1.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc Returns a triple with number of inner nodes, leaf nodes and hashed items.\n-spec size_detail(merkle_tree() | [mt_node()]) -> mt_size().\nsize_detail({merkle_tree, _, Root}) ->\n    size_detail_node([Root], 0, 0, 0, 0);\nsize_detail(Nodes) when is_list(Nodes) ->\n    Result = {_Inner, _Leafs, _EmptyLeafs, _Items} = size_detail_node(Nodes, 0, 0, 0, 0),\n    % already tested via unit test:\n%%     ?DBG_ASSERT(_Leafs =:= lists:sum([get_leaf_count(N) || N <- Nodes])),\n%%     ?DBG_ASSERT(_Items =:= lists:sum([get_item_count(N) || N <- Nodes])),\n    Result.\n\n-spec size_detail_node([mt_node() | [mt_node()]], InnerNodes::non_neg_integer(),\n                       Leafs::non_neg_integer(), EmptyLeafs::non_neg_integer(),\n                       Items::non_neg_integer())\n        -> mt_size().\nsize_detail_node([{_H, _Cnt, _ICnt, _I, Childs = [_|_]} | R], Inner, Leafs, EmptyLeafs, Items) ->\n    size_detail_node([Childs | R], Inner + 1, Leafs, EmptyLeafs, Items);\nsize_detail_node([{_H, _ICnt = 0, _Bkt, _I} | R], Inner, Leafs, EmptyLeafs, Items) ->\n    size_detail_node(R, Inner, Leafs + 1, EmptyLeafs + 1, Items);\nsize_detail_node([{_H, ICnt, _Bkt, _I} | R], Inner, Leafs, EmptyLeafs, Items) ->\n    size_detail_node(R, Inner, Leafs + 1, EmptyLeafs, Items + ICnt);\nsize_detail_node([], InnerNodes, Leafs, EmptyLeafs, Items) ->\n    {InnerNodes, Leafs, EmptyLeafs, Items};\nsize_detail_node([[{_H, _Cnt, _ICnt, _I, Childs = [_|_]} | R1] | R2], Inner, Leafs, EmptyLeafs, Items) ->\n    size_detail_node([Childs, R1 | R2], Inner + 1, Leafs, EmptyLeafs, Items);\nsize_detail_node([[{_H, _ICnt = 0, _Bkt, _I} | R1] | R2], Inner, Leafs, EmptyLeafs, Items) ->\n    size_detail_node([R1 | R2], Inner, Leafs + 1, EmptyLeafs + 1, Items);\nsize_detail_node([[{_H, ICnt, _Bkt, _I} | R1] | R2], Inner, Leafs, EmptyLeafs, Items) ->\n    size_detail_node([R1 | R2], Inner, Leafs + 1, EmptyLeafs, Items + ICnt);\nsize_detail_node([[] | R2], Inner, Leafs, EmptyLeafs, Items) ->\n    size_detail_node(R2, Inner, Leafs, EmptyLeafs, Items).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc Returns an iterator to visit all tree nodes with next/1.\n%      Iterates over all tree nodes from left to right in a deep first manner.\n-spec iterator(Tree::merkle_tree()) -> Iter::mt_iter().\niterator({merkle_tree, _, Root}) -> [Root].\n\n-spec iterator_node(Node::mt_node(), mt_iter()) -> mt_iter().\niterator_node({_H, _Cnt, _ICnt, _I, Childs = [_|_]}, Iter1) ->\n    [Childs | Iter1];\niterator_node({_H, _ICnt, _Bkt, _I}, Iter1) ->\n    Iter1.\n\n-spec next(mt_iter()) -> none | {Node::mt_node(), mt_iter()}.\nnext([Node | R]) when is_tuple(Node) ->\n    {Node, iterator_node(Node, R)};\nnext([[Node | R1] | R2]) when is_tuple(Node) ->\n    {Node, iterator_node(Node, [R1 | R2])};\nnext([[] | R2]) ->\n    next(R2);\nnext([]) ->\n    none.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc Stores the tree graph into a png file.\n-spec store_graph(merkle_tree(), string()) -> ok.\nstore_graph(MerkleTree, FileName) ->\n    erlang:spawn(fun() -> store_to_DOT_p(MerkleTree, FileName, true) end),\n    ok.\n\n% @doc Stores the tree graph into a file in DOT language (for Graphviz or other visualization tools).\n-spec store_to_DOT(merkle_tree(), string()) -> ok.\nstore_to_DOT(MerkleTree, FileName) ->\n    erlang:spawn(fun() -> store_to_DOT_p(MerkleTree, FileName, false) end),\n    ok.\n\nstore_to_DOT_p({merkle_tree, Conf, Root}, FileName, ToPng) ->\n    case file:open(\"../\" ++ FileName ++ \".dot\", [write]) of\n        {ok, Fileid} ->\n            io:fwrite(Fileid, \"digraph merkle_tree { ~n\", []),\n            io:fwrite(Fileid, \"    style=filled;~n\", []),\n            store_node_to_DOT(Root, Fileid, 1, 2, Conf),\n            io:fwrite(Fileid, \"} ~n\", []),\n            _ = file:truncate(Fileid),\n            _ = file:close(Fileid),\n            _ = if ToPng ->\n                       _ = os:cmd(io_lib:format(\"dot ../~s.dot -Tpng > ../~s.png\", [FileName, FileName])),\n                       os:cmd(io_lib:format(\"rm -f ../~s.dot\", [FileName]));\n                   true -> ok\n                end,\n            ok;\n        {_, _} ->\n            io_error\n    end.\n\n-spec store_node_to_DOT(mt_node(), pid(), pos_integer(), pos_integer(), mt_config()) -> pos_integer().\nstore_node_to_DOT({H, ICnt, _Bkt, I}, Fileid, MyId,\n                  NextFreeId, #mt_config{ bucket_size = BuckSize }) ->\n    {LBr, _LKey, _RKey, RBr} = intervals:get_bounds(I),\n    io:fwrite(Fileid, \"    ~p [label=\\\"~p\\\\n~s~p,~p~s ; ~p/~p\\\", shape=box]~n\",\n              [MyId, ?DOT_SHORTNAME_HASH(H), erlang:atom_to_list(LBr),\n               ?DOT_SHORTNAME_KEY(_LKey), ?DOT_SHORTNAME_KEY(_RKey),\n               erlang:atom_to_list(RBr), ICnt, BuckSize]),\n    NextFreeId;\nstore_node_to_DOT({H, _Cnt, _ICnt, I, Childs = [_ | RChilds]}, Fileid, MyId,\n                  NextFreeId, TConf) ->\n    io:fwrite(Fileid, \"    ~p -> { ~p\", [MyId, NextFreeId]),\n    NNFreeId = lists:foldl(fun(_, Acc) ->\n                                    io:fwrite(Fileid, \";~p\", [Acc]),\n                                    Acc + 1\n                           end, NextFreeId + 1, RChilds),\n    io:fwrite(Fileid, \" }~n\", []),\n    {_, NNNFreeId} = lists:foldl(fun(Node, {NodeId, NextFree}) ->\n                                         {NodeId + 1 , store_node_to_DOT(Node, Fileid, NodeId, NextFree, TConf)}\n                                 end, {NextFreeId, NNFreeId}, Childs),\n    {LBr, _LKey, _RKey, RBr} = intervals:get_bounds(I),\n    io:fwrite(Fileid, \"    ~p [label=\\\"~p\\\\n~s~p,~p~s\\\"\"\"]~n\",\n              [MyId, ?DOT_SHORTNAME_HASH(H), erlang:atom_to_list(LBr),\n               ?DOT_SHORTNAME_KEY(_LKey), ?DOT_SHORTNAME_KEY(_RKey),\n               erlang:atom_to_list(RBr)]),\n    NNNFreeId.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% Calculates min bucket size to remove S tree levels.\n% Formula: N / (v^(log_v(N) - S))\n% S = 1.. has to be smaller than log_v(N)\n-spec get_opt_bucket_size(N::non_neg_integer(), V::non_neg_integer(), S::pos_integer()) -> pos_integer().\nget_opt_bucket_size(_N, 0, _S) -> 1;\nget_opt_bucket_size(0, _V, _S) -> 1;\nget_opt_bucket_size(N, V, S) ->\n    Height = erlang:max(util:ceil(util:log(N, V)) - S, 1),\n    util:ceil(N / math:pow(V, Height)).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Local Functions\n\n-spec build_config(mt_config_params()) -> mt_config().\nbuild_config(ParamList) ->\n    lists:foldl(\n      fun({Key, Val}, Conf) ->\n              case Key of\n                  branch_factor  -> ?ASSERT(Val >= 2), Conf#mt_config{ branch_factor = Val };\n                  bucket_size    -> Conf#mt_config{ bucket_size = Val };\n                  leaf_hf        -> Conf#mt_config{ leaf_hf = Val };\n                  inner_hf       -> Conf#mt_config{ inner_hf = Val };\n                  keep_bucket    -> Conf#mt_config{ keep_bucket = Val}\n              end\n      end,\n      #mt_config{}, ParamList).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Decodes a key for use by the merkle_tree.\n-spec decode_key(Key::mt_bucket_entry()) -> ?RT:key().\ndecode_key(Key) -> element(1, Key).\n\n%% @doc Inserts Key into its matching interval (represented as a tuple of\n%%      bucket tuples). For each key, it starts at Last and continuously\n%%      checks further tuples until index High is reached.\n%%      PreCond: Key fits into one of the given intervals,\n%%               CheckKey is the decoded Key (see decode_key/1)\n-spec p_key_in_I(Key::mt_bucket_entry(), CheckKey::?RT:key(), Acc) -> Acc when\n    is_subtype(Acc,  {ITuple::tuple(), Last::pos_integer(), High::pos_integer()}).\np_key_in_I(Key, CheckKey, {ITuple, Last, High}) ->\n    % find first tuple for key\n    {I, Len, Keys} = element(Last, ITuple),\n    case intervals:in(CheckKey, I) of\n        true  ->\n            {setelement(Last, ITuple, {I, Len + 1, [Key | Keys]}), Last, Last};\n        false ->\n            Next = Last rem tuple_size(ITuple) + 1,\n            ?ASSERT2(Next =/= High, loop_detected), % item does not belong in any interval!\n            p_key_in_I(Key, CheckKey, {ITuple, Next, High})\n    end.\n\n%% @doc Inserts the given keys into the given intervals.\n-spec keys_to_intervals(mt_bucket(), [I,...])\n        -> Buckets::[{I, Count::non_neg_integer(), mt_bucket()}] when\n     is_subtype(I, intervals:continuous_interval()).\nkeys_to_intervals(KList, IList) ->\n    IBucket = [{I, 0, []} || I <- IList],\n    % pay attention to the key order - run_leaf_hf/4 relies on it being either\n    % correct or reversed (which is exactly what p_key_in_I/3 does)\n    {IBucket2, _, _} =\n        lists:foldl(fun(Key, Acc) ->\n                            p_key_in_I(Key, decode_key(Key), Acc)\n                    end, {list_to_tuple(IBucket), 1, 1}, KList),\n    tuple_to_list(IBucket2).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec inner_hash_XOR(ChildHashes::[mt_node_key(),...]) -> mt_node_key().\ninner_hash_XOR([H|T]) ->\n    lists:foldl(fun(X, Acc) -> X bxor Acc end, H, T).\n\n%% @doc Leaf hash fun to use for the embedded merkle tree.\n-spec leaf_hash_sha(merkle_tree:mt_bucket(), intervals:interval()) -> binary().\nleaf_hash_sha([], _I) ->\n    <<0:160>>;\nleaf_hash_sha([_|_] = Bucket, _I) ->\n    ?CRYPTO_SHA(term_to_binary(Bucket)).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% allow future versions to create more hash funs by having an integer parameter\n-spec tester_create_hash_fun(I::1) -> leaf_hash_fun().\ntester_create_hash_fun(1) -> fun leaf_hash_sha/2.\n\n% allow future versions to create more hash funs by having an integer parameter\n-spec tester_create_inner_hash_fun(I::1) -> inner_hash_fun().\ntester_create_inner_hash_fun(1) -> fun inner_hash_XOR/1.\n"
  },
  {
    "path": "src/rrepair/rr_recon.erl",
    "content": "% @copyright 2011-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    replica reconcilication module\n%% @end\n%% @version $Id$\n-module(rr_recon).\n-author('malange@informatik.hu-berlin.de').\n\n-behaviour(gen_component).\n\n-include(\"record_helpers.hrl\").\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([init/1, on/2, start/2, check_config/0]).\n-export([map_key_to_interval/2, map_key_to_quadrant/2, map_rkeys_to_quadrant/2,\n         map_interval/2,\n         quadrant_intervals/0]).\n-export([get_chunk_kv/1, get_chunk_filter/1]).\n%-export([compress_kv_list/5]).\n\n%export for testing\n-export([find_sync_interval/2, quadrant_subints_/3, key_dist/2]).\n-export([merkle_compress_hashlist/4, merkle_decompress_hashlist/3]).\n-export([pos_to_bitstring/4, bitstring_to_k_list_k/3, bitstring_to_k_list_kv/3]).\n-export([calc_n_subparts_FR/2, calc_n_subparts_FR/3]).\n%% -export([calc_max_different_hashes/3,\n%%          trivial_signature_sizes/4, trivial_worst_case_failrate/4,\n%%          shash_signature_sizes/4, shash_worst_case_failrate/4,\n%%          calc_max_different_items_node/3, calc_max_different_items_total/3,\n%%          bloom_target_fp/4, bloom_worst_case_failrate_/5]).\n-export([tester_create_kvi_tree/1, tester_is_kvi_tree/1]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% debug\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-define(TRACE(X,Y), ok).\n%-define(TRACE(X,Y), log:pal(\"~w: [ ~.0p:~.0p ] \" ++ X ++ \"~n\", [?MODULE, pid_groups:my_groupname(), self()] ++ Y)).\n-define(TRACE_SEND(Pid, Msg), ?TRACE(\"to ~.0p:~.0p: ~.0p~n\", [pid_groups:group_of(comm:make_local(comm:get_plain_pid(Pid))), Pid, Msg])).\n-define(TRACE_RCV(Msg, State),\n        ?TRACE(\"~n  Msg: ~.0p~n\"\n                 \"State: method: ~.0p;  stage: ~.0p;  initiator: ~.0p~n\"\n                 \"      syncI@I: ~.0p~n\"\n                 \"       params: ~.0p~n\",\n               [Msg, State#rr_recon_state.method, State#rr_recon_state.stage,\n                State#rr_recon_state.initiator, State#rr_recon_state.'sync_interval@I',\n                ?IIF(is_list(State#rr_recon_state.struct), State#rr_recon_state.struct, [])])).\n-define(ALG_DEBUG(X,Y), ok).\n%-define(ALG_DEBUG(X,Y), log:pal(\"~w: [ ~.0p:~.0p ] \" ++ X, [?MODULE, pid_groups:my_groupname(), self()] ++ Y)).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% type definitions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-export_type([method/0, request/0]).\n\n-type method()         :: trivial | shash | bloom | merkle_tree | art.% | iblt.\n-type stage()          :: req_shared_interval | build_struct | reconciliation | resolve.\n\n-type exit_reason()    :: empty_interval |      %interval intersection between initator and client is empty\n                          recon_node_crash |    %sync partner node crashed\n                          sync_finished |       %finish recon on local node\n                          sync_finished_remote. %client-side shutdown by merkle-tree recon initiator\n\n-type db_chunk_kv()    :: [{?RT:key(), client_version()}].\n\n-type signature_size() :: 0..160. % use an upper bound of 160 (SHA-1) to limit automatic testing\n-type kvi_tree()       :: mymaps:mymap(). % KeyShort::non_neg_integer() => {VersionShort::non_neg_integer(), Idx::non_neg_integer()}\n\n-record(trivial_sync,\n        {\n         interval       = intervals:empty()                       :: intervals:interval(),\n         reconPid       = undefined                               :: comm:mypid() | undefined,\n         exp_delta      = ?required(trivial_sync, exp_delta)      :: number(),\n         db_chunk       = ?required(trivial_sync, db_chunk)       :: {bitstring(), bitstring()} | {bitstring(), bitstring(), ResortedKV::db_chunk_kv(), Dupes::db_chunk_kv()}, % two binaries for transfer, the four-tuple only temporarily (locally)\n         '+item_count'  = ?required(trivial_sync, '+item_count')  :: non_neg_integer(), % number of items not present in db_chunk\n         sig_size       = ?required(trivial_sync, sig_size)       :: signature_size()\n        }).\n\n-record(shash_sync,\n        {\n         interval       = intervals:empty()                       :: intervals:interval(),\n         reconPid       = undefined                               :: comm:mypid() | undefined,\n         exp_delta      = ?required(shash_sync, exp_delta)        :: number(),\n         db_chunk       = ?required(shash_sync, db_chunk)         :: bitstring() | {bitstring(), ResortedKV::db_chunk_kv(), Dupes::db_chunk_kv()}, % binary for transfer, the three-tuple only temporarily (locally)\n         '+item_count'  = ?required(shash_sync, '+item_count')    :: non_neg_integer(), % number of items not present in db_chunk\n         sig_size       = ?required(shash_sync, sig_size)         :: signature_size(),\n         fail_rate      = ?required(shash_sync, fail_rate)        :: float()\n        }).\n\n-record(bloom_sync,\n        {\n         interval       = intervals:empty()                       :: intervals:interval(),\n         reconPid       = undefined                               :: comm:mypid() | undefined,\n         exp_delta      = ?required(bloom_sync, exp_delta)        :: number(),\n         bf             = ?required(bloom_sync, bloom)            :: bitstring() | bloom:bloom_filter(), % bitstring for transfer, the full filter locally\n         item_count     = ?required(bloom_sync, item_count)       :: non_neg_integer(),\n         hf_count       = ?required(bloom_sync, hf_count)         :: pos_integer(),\n         fail_rate      = ?required(bloom_sync, fail_rate)        :: float()\n        }).\n\n-record(merkle_params,\n        {\n         interval       = ?required(merkle_param, interval)       :: intervals:interval(),\n         reconPid       = undefined                               :: comm:mypid() | undefined,\n         exp_delta      = ?required(merkle_param, exp_delta)      :: number(),\n         branch_factor  = ?required(merkle_param, branch_factor)  :: pos_integer(),\n         num_trees      = ?required(merkle_param, num_trees)      :: pos_integer(),\n         bucket_size    = ?required(merkle_param, bucket_size)    :: pos_integer(),\n         fail_rate      = ?required(merkle_param, fail_rate)      :: float(),\n         ni_item_count  = ?required(merkle_param, ni_item_count)  :: non_neg_integer(),\n         ni_max_ic      = ?required(merkle_param, ni_max_ic)      :: non_neg_integer()\n        }).\n\n-record(art_recon_struct,\n        {\n         art            = ?required(art_recon_struct, art)            :: art:art(),\n         reconPid       = undefined                                   :: comm:mypid() | undefined,\n         branch_factor  = ?required(art_recon_struct, branch_factor)  :: pos_integer(),\n         bucket_size    = ?required(art_recon_struct, bucket_size)    :: pos_integer()\n        }).\n\n-type sync_struct() :: #trivial_sync{} |\n                       #shash_sync{} |\n                       #bloom_sync{} |\n                       merkle_tree:merkle_tree() |\n                       [merkle_tree:mt_node()] |\n                       #art_recon_struct{}.\n-type parameters() :: #trivial_sync{} |\n                      #shash_sync{} |\n                      #bloom_sync{} |\n                      #merkle_params{} |\n                      #art_recon_struct{}.\n-type recon_dest() :: ?RT:key() | random.\n\n-type merkle_sync_rcv() ::\n          {MyMaxItemsCount::non_neg_integer(),\n           MyKVItems::merkle_tree:mt_bucket()}.\n-type merkle_sync_send() ::\n          {OtherMaxItemsCount::non_neg_integer(),\n           MyKVItems::merkle_tree:mt_bucket()}.\n-type merkle_sync_direct() ::\n          % mismatches with an empty leaf on either node\n          % -> these are resolved directly\n          {MyKItems::[?RT:key()], MyLeafCount::non_neg_integer(), OtherNodeCount::non_neg_integer()}.\n-type merkle_sync() :: {[merkle_sync_send()], [merkle_sync_rcv()],\n                        SynRcvLeafCount::non_neg_integer(), merkle_sync_direct()}.\n\n-record(rr_recon_state,\n        {\n         ownerPid           = ?required(rr_recon_state, ownerPid)    :: pid(),\n         dest_rr_pid        = ?required(rr_recon_state, dest_rr_pid) :: comm:mypid(), %dest rrepair pid\n         dest_recon_pid     = undefined                              :: comm:mypid() | undefined, %dest recon process pid\n         method             = undefined                              :: method() | undefined,\n         'sync_interval@I'  = intervals:empty()                      :: intervals:interval(),\n         'max_items@I'      = undefined                              :: non_neg_integer() | undefined,\n         params             = {}                                     :: parameters() | {}, % parameters from the other node\n         struct             = {}                                     :: sync_struct() | {}, % my recon structure\n         stage              = req_shared_interval                    :: stage(),\n         initiator          = false                                  :: boolean(),\n         merkle_sync        = {[], [], 0, {[], 0, 0}}                :: merkle_sync(),\n         misc               = []                                     :: [{atom(), term()}], % any optional parameters an algorithm wants to keep\n         kv_list            = []                                     :: db_chunk_kv() | [db_chunk_kv()], % list of KV chunks only temporarily when retrieving the DB in pieces\n         k_list             = []                                     :: [?RT:key()],\n         next_phase_kv      = []                                     :: db_chunk_kv(), % KV items that go directly into the next phase, e.g. items from colliding hashes (dupes)\n         stats              = ?required(rr_recon_state, stats)       :: rr_recon_stats:stats()\n         }).\n-type state() :: #rr_recon_state{}.\n\n% keep in sync with merkle_check_node/22\n-define(recon_ok,                       1). % match\n-define(recon_fail_stop_leaf,           2). % mismatch, sending node has leaf node\n-define(recon_fail_stop_inner,          3). % mismatch, sending node has inner node\n-define(recon_fail_cont_inner,          0). % mismatch, both inner nodes (continue!)\n\n-type merkle_cmp_request() :: {Hash::merkle_tree:mt_node_key() | none, IsLeaf::boolean()}.\n\n-type request() ::\n    {start, method(), DestKey::recon_dest()} |\n    {create_struct, method(), SenderI::intervals:interval(), SenderMaxItems::non_neg_integer()} | % from initiator\n    {start_recon, bloom, #bloom_sync{}} | % to initiator\n    {start_recon, merkle_tree, #merkle_params{}} | % to initiator\n    {start_recon, art, #art_recon_struct{}}. % to initiator\n\n-type message() ::\n    % API\n    request() |\n    % trivial/shash/bloom sync messages\n    {reconcile_req, DiffBFBin::bitstring(), OtherBFCount::non_neg_integer(),\n     OtherDiffCount::non_neg_integer(), SenderPid::comm:mypid()} |\n    {resolve_req, BinReqIdxPos::bitstring()} |\n    {resolve_req, DBChunk::{bitstring(), bitstring()}, DupesCount::non_neg_integer(),\n     Payload::any(), SigSize::signature_size(), SenderPid::comm:mypid()} |\n    {resolve_req, shutdown, Payload::any()} |\n    % merkle tree sync messages\n    {?check_nodes, SenderPid::comm:mypid(), ToCheck::bitstring(), MaxItemsCount::non_neg_integer()} |\n    {?check_nodes, ToCheck::bitstring(), MaxItemsCount::non_neg_integer()} |\n    {?check_nodes_response, FlagsBin::bitstring(), MaxItemsCount::non_neg_integer()} |\n    {resolve_req, HashesK::bitstring(), HashesV::bitstring()} |\n    {resolve_req, BinKeyList::bitstring(), SyncSendFr_real::float()} |\n    % dht node response\n    {create_struct2, SenderI::intervals:interval(), {get_state_response, MyI::intervals:interval()}} |\n    {process_db, {get_chunk_response, {intervals:interval(), db_chunk_kv()}}} |\n    % internal\n    {shutdown, exit_reason()} |\n    {fd_notify, fd:event(), DeadPid::comm:mypid(), Reason::fd:reason()} |\n    {'DOWN', MonitorRef::reference(), process, Owner::pid(), Info::any()}\n    .\n\n-include(\"gen_component.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Message handling\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec on(message(), state()) -> state() | kill.\n\non({create_struct, RMethod, SenderI, SenderMaxItems} = _Msg, State) ->\n    ?TRACE_RCV(_Msg, State),\n    % (first) request from initiator to create a sync struct\n    This = comm:reply_as(comm:this(), 3, {create_struct2, SenderI, '_'}),\n    comm:send_local(pid_groups:get_my(dht_node), {get_state, This, my_range}),\n    State#rr_recon_state{method = RMethod, initiator = false,\n                         'max_items@I' = SenderMaxItems};\n\non({create_struct2, SenderI, {get_state_response, MyI}} = _Msg,\n   State = #rr_recon_state{stage = req_shared_interval, initiator = false,\n                           method = RMethod,            dest_rr_pid = DestRRPid}) ->\n    ?TRACE_RCV(_Msg, State),\n    % target node got sync request, asked for its interval\n    % dest_interval contains the interval of the initiator\n    % -> client creates recon structure based on common interval, sends it to initiator\n    SyncI = find_sync_interval(MyI, SenderI),\n    case intervals:is_empty(SyncI) of\n        false ->\n            send_chunk_req(pid_groups:get_my(dht_node), self(),\n                           SyncI, get_max_items()),\n            case RMethod of\n                art -> ok; % legacy (no integrated trivial sync yet)\n                _   -> fd:subscribe(self(), [DestRRPid])\n            end,\n            % reduce SenderI to the sub-interval matching SyncI, i.e. a mapped SyncI\n            SenderSyncI = map_interval(SenderI, SyncI),\n            State#rr_recon_state{stage = build_struct,\n                                 'sync_interval@I' = SenderSyncI};\n        true ->\n            shutdown(empty_interval, State)\n    end;\n\non({process_db, {get_chunk_response, {RestI, DBList0}}} = _Msg,\n   State = #rr_recon_state{stage = build_struct,       initiator = false,\n                           'sync_interval@I' = SenderSyncI}) ->\n    ?TRACE_RCV(_Msg, State),\n    % create recon structure based on all elements in sync interval\n    DBList = [{Key, VersionX} || {KeyX, VersionX} <- DBList0,\n                                 none =/= (Key = map_key_to_interval(KeyX, SenderSyncI))],\n    build_struct(DBList, RestI, State);\n\non({start_recon, RMethod, Params} = _Msg,\n   State = #rr_recon_state{dest_rr_pid = DestRRPid, misc = Misc}) ->\n    ?TRACE_RCV(_Msg, State),\n    % initiator got other node's sync struct or parameters over sync interval\n    % (mapped to the initiator's range)\n    % -> create own recon structure based on sync interval and reconcile\n    % note: sync interval may be an outdated sub-interval of this node's range\n    %       -> pay attention when saving values to DB!\n    %       (it could be outdated then even if we retrieved the current range now!)\n    case RMethod of\n        trivial ->\n            #trivial_sync{interval = MySyncI, reconPid = DestReconPid,\n                          db_chunk = CKV, sig_size = SigSize} = Params,\n            Params1 = Params#trivial_sync{db_chunk = {<<>>, <<>>}},\n            ?DBG_ASSERT(DestReconPid =/= undefined),\n            fd:subscribe(self(), [DestRRPid]),\n            % convert db_chunk to a map for faster access checks\n            {CKVTree, VSize} = decompress_kv_list(CKV, unknown, SigSize),\n            ?DBG_ASSERT(Misc =:= []),\n            Misc1 = [{db_chunk, CKVTree},\n                     {ver_size, VSize}];\n        shash ->\n            #shash_sync{interval = MySyncI, reconPid = DestReconPid,\n                        db_chunk = SH, sig_size = SigSize} = Params,\n            ?DBG_ASSERT(DestReconPid =/= undefined),\n            fd:subscribe(self(), [DestRRPid]),\n            % convert db_chunk to a map for faster access checks\n            {SHTree, 0} = decompress_kv_list({SH, <<>>}, unknown, SigSize),\n            Params1 = Params#shash_sync{db_chunk = <<>>},\n            ?DBG_ASSERT(Misc =:= []),\n            Misc1 = [{db_chunk, SHTree}];\n        bloom ->\n            #bloom_sync{interval = MySyncI, reconPid = DestReconPid,\n                        bf = BFBin, item_count = BFCount,\n                        hf_count = HfCount} = Params,\n            ?DBG_ASSERT(DestReconPid =/= undefined),\n            fd:subscribe(self(), [DestRRPid]),\n            ?DBG_ASSERT(Misc =:= []),\n            BF = bloom:new_bin(BFBin, HfCount, BFCount),\n            Params1 = Params#bloom_sync{bf = BF},\n            Misc1 = [{item_count, 0},\n                     {my_bf, bloom:new(erlang:max(1, erlang:bit_size(BFBin)), HfCount)}];\n        merkle_tree ->\n            #merkle_params{interval = MySyncI, reconPid = DestReconPid} = Params,\n            Params1 = Params,\n            ?DBG_ASSERT(DestReconPid =/= undefined),\n            fd:subscribe(self(), [DestRRPid]),\n            Misc1 = Misc;\n        art ->\n            #art_recon_struct{art = ART, reconPid = DestReconPid} = Params,\n            MySyncI = art:get_interval(ART),\n            Params1 = Params,\n            ?DBG_ASSERT(DestReconPid =/= undefined),\n            Misc1 = Misc\n    end,\n    % client only sends non-empty sync intervals or exits\n    ?DBG_ASSERT(not intervals:is_empty(MySyncI)),\n    \n    send_chunk_req(pid_groups:get_my(dht_node), self(),\n                   MySyncI, get_max_items()),\n    State#rr_recon_state{stage = reconciliation, params = Params1,\n                         method = RMethod, initiator = true,\n                         'sync_interval@I' = MySyncI,\n                         dest_recon_pid = DestReconPid,\n                         misc = Misc1};\n\non({process_db, {get_chunk_response, {RestI, DBList}}} = _Msg,\n   State = #rr_recon_state{stage = reconciliation,        initiator = true,\n                           method = trivial,              kv_list = KVList}) ->\n    ?TRACE_RCV(_Msg, State),\n    % we need all items first and the process them further (just as on the non-initiator)\n    % collect them in the 'kv_list' property of #rr_recon_state{}\n\n    % if rest interval is non-empty get another chunk\n    SyncFinished = intervals:is_empty(RestI),\n    if not SyncFinished ->\n           send_chunk_req(pid_groups:get_my(dht_node), self(),\n                          RestI, get_max_items()),\n           State#rr_recon_state{kv_list = [DBList | KVList]};\n       true ->\n           #rr_recon_state{params = #trivial_sync{exp_delta = ExpDelta,\n                                                  sig_size = SigSize,\n                                                  '+item_count' = AddToChunkSize},\n                           dest_rr_pid = DestRRPid, stats = Stats,\n                           ownerPid = OwnerL,\n                           misc = [{db_chunk, CKVTree},\n                                   {ver_size, VSize}],\n                           dest_recon_pid = DestReconPid} = State,\n           FullKVList = lists:append([DBList | KVList]),\n           MyDBSize1 = length(FullKVList),\n           ChunkSize = mymaps:size(CKVTree),\n           % identify items to send, request and the remaining (non-matched) DBChunk:\n           % remove colliding hashes in DBList and add them to phase 2 (in turn,\n           % colliding items in CKV are also added to phase 2 on the non-initiator)\n           {ToSend, ToReqIdx, OtherDBChunk1, _Dupes} =\n               get_diff_with_dupes(FullKVList, CKVTree, [], [], SigSize, VSize,\n                                   fun get_full_diff/4),\n           ItemCountNI_p1 = ChunkSize + AddToChunkSize,\n           ?ALG_DEBUG(\"CheckCKV ~B+~Bckv vs. ~B+~Bcmp items\",\n                      [ChunkSize, AddToChunkSize, MyDBSize1 - length(_Dupes), length(_Dupes)]),\n           ?DBG_ASSERT2(length(ToSend) =:= length(lists:usort(ToSend)),\n                        {non_unique_send_list, ToSend, _Dupes}),\n           % note: the actual acc(phase1) may be different from what the non-initiator expected\n           Fr_p1_real = trivial_worst_case_failrate(\n                          SigSize, MyDBSize1, ItemCountNI_p1, ExpDelta),\n           Stats1  = rr_recon_stats:set([{fail_rate_p1, Fr_p1_real}], Stats),\n           NewStats = send_resolve_request(Stats1, ToSend, OwnerL, DestRRPid,\n                                           true, true),\n           % let the non-initiator's rr_recon process identify the remaining keys\n           ReqIdx = lists:usort([Idx || {_Version, Idx} <- mymaps:values(OtherDBChunk1)]\n                                    ++ ToReqIdx),\n           ToReq2 = compress_idx_list(ReqIdx, ChunkSize - 1, [], 0, 0, integrate_size),\n           NewStats2 =\n               if ReqIdx =/= [] orelse AddToChunkSize > 0 ->\n                      % the non-initiator will use key_upd_send and we must thus increase\n                      % the number of resolve processes here!\n                      rr_recon_stats:inc([{rs_expected, 1}], NewStats);\n                  true -> NewStats\n               end,\n\n           ?ALG_DEBUG(\"resolve_req Trivial~n  Session=~.0p ; ToReq=~p (~p bits)\",\n                      [rr_recon_stats:get(session_id, Stats1), length(ReqIdx),\n                       erlang:bit_size(ToReq2)]),\n           comm:send(DestReconPid, {resolve_req, ToReq2}),\n           \n           shutdown(sync_finished,\n                    State#rr_recon_state{stats = NewStats2, misc = []})\n    end;\n\non({process_db, {get_chunk_response, {RestI, DBList}}} = _Msg,\n   State = #rr_recon_state{stage = reconciliation,    initiator = true,\n                           method = shash,            kv_list = KVList}) ->\n    ?TRACE_RCV(_Msg, State),\n    % this is similar to the trivial sync above:\n    % we need all items first and the process them further (just as on the non-initiator)\n    % collect them in the 'kv_list' property of #rr_recon_state{}\n\n    %if rest interval is non empty start another sync\n    SyncFinished = intervals:is_empty(RestI),\n    if not SyncFinished ->\n           send_chunk_req(pid_groups:get_my(dht_node), self(),\n                          RestI, get_max_items()),\n           State#rr_recon_state{kv_list = [DBList | KVList]};\n       true ->\n           #rr_recon_state{params = #shash_sync{exp_delta = ExpDelta,\n                                                sig_size = SigSize,\n                                                fail_rate = FR,\n                                                '+item_count' = AddToChunkSize},\n                           stats = Stats, misc = [{db_chunk, SHTree}]} = State,\n           FullKVList = lists:append([DBList | KVList]),\n           MyDBSize1 = length(FullKVList),\n           ChunkSize = mymaps:size(SHTree),\n           % identify differing items and the remaining (non-matched) DBChunk:\n           % remove colliding hashes in DBList and add them to phase 2 (in turn,\n           % colliding items in SH are also added to phase 2 on the non-initiator)\n           {FullKVList1, Dupes0} =\n               if FullKVList =:= [] -> {[], []};\n                  true -> sort_extractdupes([{compress_key(KV, SigSize), KV}\n                                             || KV <- FullKVList])\n               end,\n           ItemCountNI_p1 = ChunkSize + AddToChunkSize,\n           ?ALG_DEBUG(\"CheckSH ~B+~Bckv vs. ~B+~Bcmp items\",\n                      [ChunkSize, AddToChunkSize, MyDBSize1 - length(Dupes0), length(Dupes0)]),\n           {NewKVList, OtherDBChunk1} =\n               shash_get_full_diff(FullKVList1, SHTree,\n                                   [element(2, D) || D <- Dupes0], SigSize),\n           % note: the actual fr(phase1) may be different from what the non-initiator expected\n           Fr_p1_real =\n               shash_worst_case_failrate(SigSize, MyDBSize1, ItemCountNI_p1, ExpDelta),\n           FR_p2 = calc_n_subparts_FR(1, FR, Fr_p1_real),\n           Stats1  = rr_recon_stats:set([{fail_rate_p1, Fr_p1_real}], Stats),\n           CKidxSize = mymaps:size(OtherDBChunk1),\n           ItemCountNI_p2 = CKidxSize + AddToChunkSize,\n           ReqIdx = lists:usort([Idx || {_Version, Idx} <- mymaps:values(OtherDBChunk1)]),\n           OtherDiffIdx = compress_idx_list(ReqIdx, ChunkSize - 1, [], 0, 0, integrate_size),\n           phase2_run_trivial_on_diff(\n             NewKVList, OtherDiffIdx, ItemCountNI_p1 =:= 0, FR_p2,\n             ItemCountNI_p2,\n             State#rr_recon_state{kv_list = [], stats = Stats1, misc = []})\n    end;\n\non({process_db, {get_chunk_response, {RestI, DBList}}} = _Msg,\n   State = #rr_recon_state{stage = reconciliation,    initiator = true,\n                           method = bloom,            kv_list = KVList,\n                           params = #bloom_sync{bf = BF} = Params,\n                           dest_recon_pid = DestReconPid, stats = Stats,\n                           dest_rr_pid = DestRRPid,   ownerPid = OwnerL,\n                           misc = [{item_count, MyDBSize}, {my_bf, MyBF}]}) ->\n    ?TRACE_RCV(_Msg, State),\n    % if rest interval is non empty start another sync\n    % (do this early to allow some parallelism)\n    SyncFinished = intervals:is_empty(RestI),\n    if not SyncFinished ->\n           send_chunk_req(pid_groups:get_my(dht_node), self(),\n                          RestI, get_max_items());\n       true -> ok\n    end,\n    \n    MyDBSize1 = MyDBSize + length(DBList),\n    % no need to map keys since the other node's bloom filter was created with\n    % keys mapped to our interval\n    BFCount = bloom:item_count(BF),\n    % get a subset of Delta without what is missing on this node:\n    % NOTE: The only errors which might occur are bloom:filter_neg_and_add/3\n    %       omitting items which are not on the other node (with the filter's\n    %       false positive probability/rate). These are the items we would miss.\n    {Diff, MyBF1} = bloom:filter_neg_and_add(BF, DBList, MyBF),\n    NewKVList = lists:append(KVList, Diff),\n\n    if not SyncFinished ->\n           State#rr_recon_state{kv_list = NewKVList,\n                                misc = [{item_count, MyDBSize1}, {my_bf, MyBF1}]};\n       BFCount =:= 0 ->\n           % in this case, the other node has already shut down\n           % -> we need to send our diff (see phase2_run_trivial_on_diff/7)\n           KList = [element(1, KV) || KV <- NewKVList],\n           NewStats = send_resolve_request(\n                        Stats, KList, OwnerL, DestRRPid, _IsInitiator = true, false),\n           NewState = State#rr_recon_state{stage = resolve, kv_list = NewKVList,\n                                           stats = NewStats},\n           shutdown(sync_finished, NewState);\n       true ->\n           DiffBF = util:bin_xor(bloom:get_property(BF, filter),\n                                 bloom:get_property(MyBF1, filter)),\n           send(DestReconPid, {reconcile_req, DiffBF, MyDBSize1, length(NewKVList),\n                               comm:this()}),\n           % allow the garbage collector to free the original Bloom filter here\n           Params1 = Params#bloom_sync{bf = <<>>},\n           State#rr_recon_state{stage = resolve, params = Params1, kv_list = NewKVList}\n    end;\n\non({process_db, {get_chunk_response, {RestI, DBList}}} = _Msg,\n   State = #rr_recon_state{stage = reconciliation,     initiator = true,\n                           method = RMethod})\n  when RMethod =:= merkle_tree orelse RMethod =:= art->\n    ?TRACE_RCV(_Msg, State),\n    build_struct(DBList, RestI, State);\n\non({fd_notify, crash, _Pid, _Reason} = _Msg, State) ->\n    ?TRACE_RCV(_Msg, State),\n    shutdown(recon_node_crash, State);\n\non({fd_notify, _Event, _Pid, _Reason} = _Msg, State) ->\n    State;\n\non({shutdown, Reason}, State) ->\n    shutdown(Reason, State);\n\non({'DOWN', MonitorRef, process, _Owner, _Info}, _State) ->\n    log:log(info, \"[ ~p - ~p] shutdown due to rrepair shut down\", [?MODULE, comm:this()]),\n    gen_component:demonitor(MonitorRef),\n    kill;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% trivial/shash/bloom/art reconciliation sync messages\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\non({resolve_req, shutdown, _Payload} = _Msg,\n   State = #rr_recon_state{stage = resolve,           initiator = false,\n                           method = RMethod})\n  when (RMethod =:= shash orelse RMethod =:= art) ->\n    ?DBG_ASSERT(?implies(RMethod =:= shash, _Payload =:= <<>>)),\n    ?DBG_ASSERT(?implies(RMethod =:= art, _Payload =:= none)),\n    shutdown(sync_finished, State);\n\non({resolve_req, shutdown, Fr_p1_real} = _Msg,\n   State = #rr_recon_state{stage = resolve,           initiator = true,\n                           method = bloom,            stats = Stats}) ->\n    % note: no real phase 2 since the other node has no items\n    Stats1  = rr_recon_stats:set([{fail_rate_p1, Fr_p1_real},\n                                  {fail_rate_p2, 0.0}], Stats),\n    shutdown(sync_finished, State#rr_recon_state{stats = Stats1});\n\non({resolve_req, BinReqIdxPos} = _Msg,\n   State = #rr_recon_state{stage = resolve,           initiator = false,\n                           method = trivial,\n                           dest_rr_pid = DestRRPid,   ownerPid = OwnerL,\n                           k_list = KList,            next_phase_kv = AdditionalItems,\n                           stats = Stats}) ->\n    ?TRACE_RCV(_Msg, State),\n    ToSend0 = decompress_idx_to_list(BinReqIdxPos, KList),\n    ToSend = [element(1, KV) || KV <- AdditionalItems] ++ ToSend0,\n    NewStats =\n        send_resolve_request(Stats, ToSend,\n                             OwnerL, DestRRPid, _Initiator = false, true),\n    shutdown(sync_finished, State#rr_recon_state{stats = NewStats});\n\non({resolve_req, OtherDBChunk, DupesCount, MyDiffIdx, SigSize, DestReconPid} = _Msg,\n   State = #rr_recon_state{stage = resolve,           initiator = false,\n                           method = shash,            stats = Stats,\n                           kv_list = KVList,          next_phase_kv = AdditionalItems}) ->\n    ?TRACE_RCV(_Msg, State),\n%%     log:pal(\"[ ~p ] CKIdx1: ~B (~B compressed)\",\n%%             [self(), erlang:byte_size(MyDiffIdx),\n%%              erlang:byte_size(erlang:term_to_binary(MyDiffIdx, [compressed]))]),\n    ?DBG_ASSERT(SigSize >= 0),\n\n    {DBChunkTree, VSize} = decompress_kv_list(OtherDBChunk, unknown, SigSize),\n    OrigDBChunkLen = mymaps:size(DBChunkTree),\n    % worst case diff here is 100% since both nodes now operate on the\n    % differences only and each node may have what the other node does not\n    Fr_p2_real = trivial_worst_case_failrate(\n                   % note: initiator and non-initiator are exchanged here (in phase 2)\n                   SigSize, length(KVList), OrigDBChunkLen + DupesCount, 100),\n    Stats1  = rr_recon_stats:set([{fail_rate_p2, Fr_p2_real}], Stats),\n\n    MyDiffKV = AdditionalItems ++ decompress_idx_to_list(MyDiffIdx, KVList),\n    % items not in CKidx cannot be in the diff! (except for the AdditionalItems)\n    State1 = State#rr_recon_state{kv_list = MyDiffKV, stats = Stats1},\n    NewStats2 = shash_bloom_perform_resolve(\n                  State1, DBChunkTree, DupesCount, SigSize, VSize, DestReconPid,\n                  fun get_full_diff/4),\n\n    shutdown(sync_finished, State1#rr_recon_state{stats = NewStats2});\n\non({reconcile_req, DiffBFBin, OtherBFCount, OtherDiffCount, DestReconPid} = _Msg,\n   State = #rr_recon_state{stage = reconciliation,    initiator = false,\n                           method = bloom,            kv_list = KVList,\n                           struct = #bloom_sync{bf = BF, hf_count = MyHfCount,\n                                                exp_delta = ExpDelta,\n                                                fail_rate = FR} = Struct,\n                           stats = Stats}) ->\n    ?TRACE_RCV(_Msg, State),\n    \n    OtherBF = bloom:new_bin(util:bin_xor(bloom:get_property(BF, filter),\n                                         DiffBFBin), MyHfCount, OtherBFCount),\n    % get a subset of Delta without what is missing on this node:\n    % NOTE: The only errors which might occur are bloom:filter_neg/2\n    %       omitting items which are not on the other node (with the filter's\n    %       false positive probability/rate). These are the items we would miss.\n    Diff = bloom:filter_neg(OtherBF, KVList),\n    % allow the garbage collector to free the original Bloom filter\n    Struct1 = Struct#bloom_sync{bf = <<>>},\n    State1 = State#rr_recon_state{kv_list = Diff, struct = Struct1,\n                                  dest_recon_pid = DestReconPid},\n    \n    % here, the failure probability is correct (in contrast to the\n    % initiator) since we know the number of item checks and BF sizes of both\n    % Bloom filters\n    Fr_p1_real = bloom_worst_case_failrate(BF, OtherBF, ExpDelta),\n    % NOTE: use left-over failure rate after phase 1 (bloom) for phase 2 (trivial RC)\n    FR_p2 = calc_n_subparts_FR(1, FR, Fr_p1_real),\n    ?ALG_DEBUG(\"I:~.0p, FR(phase1)=~p~n\"\n               \"  Bloom1: m=~B k=~B BFCount=~B Checks=~B\"\n               \"  Bloom2: m=~B k=~B BFCount=~B Checks=~B\"\n               \"->  fr(phase1)=~p, FR(phase2)=~p\",\n               [State1#rr_recon_state.dest_recon_pid,\n                calc_n_subparts_FR(2, FR),\n                bloom:get_property(BF, size), bloom:get_property(BF, hfs_size),\n                bloom:item_count(BF), OtherBFCount,\n                bloom:get_property(OtherBF, size), bloom:get_property(OtherBF, hfs_size),\n                bloom:item_count(OtherBF), length(KVList), Fr_p1_real, FR_p2]),\n    Stats1 = rr_recon_stats:set([{fail_rate_p1, Fr_p1_real}], Stats),\n    phase2_run_trivial_on_diff(\n      Diff, Fr_p1_real, false, FR_p2, OtherDiffCount,\n      State1#rr_recon_state{params = {}, stats = Stats1});\n\non({resolve_req, DBChunk, DupesCount, Fr_p1_real, SigSize, _DestReconPid} = _Msg,\n   State = #rr_recon_state{stage = resolve,           initiator = true,\n                           method = bloom,            kv_list = KVList,\n                           stats = Stats,\n                           dest_recon_pid = DestReconPid}) ->\n    ?TRACE_RCV(_Msg, State),\n    \n    {DBChunkTree, VSize} = decompress_kv_list(DBChunk, unknown, SigSize),\n    OrigDBChunkLen = mymaps:size(DBChunkTree),\n    % worst case diff here is 100% since both nodes now operate on the\n    % differences only and each node may have what the other node does not\n    Fr_p2_real = trivial_worst_case_failrate(\n                   SigSize, length(KVList), OrigDBChunkLen + DupesCount, 100),\n    Stats1  = rr_recon_stats:set([{fail_rate_p1, Fr_p1_real},\n                                  {fail_rate_p2, Fr_p2_real}], Stats),\n    State1 = State#rr_recon_state{stats = Stats1},\n    NewStats2 = shash_bloom_perform_resolve(\n                  State1, DBChunkTree, DupesCount, SigSize, VSize, DestReconPid,\n                  fun get_full_diff/4),\n    shutdown(sync_finished, State1#rr_recon_state{stats = NewStats2});\n\non({resolve_req, DBChunk, DupesCount, none, SigSize, DestReconPid} = _Msg,\n   State = #rr_recon_state{stage = resolve,           initiator = false,\n                           method = art}) ->\n    ?TRACE_RCV(_Msg, State),\n    \n    {DBChunkTree, VSize} = decompress_kv_list(DBChunk, unknown, SigSize),\n\n    NewStats2 = shash_bloom_perform_resolve(\n                  State, DBChunkTree, DupesCount, SigSize, VSize, DestReconPid,\n                  fun get_part_diff/4),\n    shutdown(sync_finished, State#rr_recon_state{stats = NewStats2});\n\non({resolve_req, BinReqIdxPos} = _Msg,\n   State = #rr_recon_state{stage = resolve,           initiator = IsInitiator,\n                           method = RMethod,\n                           dest_rr_pid = DestRRPid,   ownerPid = OwnerL,\n                           k_list = KList,            next_phase_kv = AdditionalItems,\n                           stats = Stats,\n                           misc = []})\n  when (RMethod =:= bloom orelse RMethod =:= shash orelse RMethod =:= art) andalso\n           ((RMethod =/= bloom) =:= IsInitiator) -> % non-initiator with Bloom, otherwise initiator\n    ?TRACE_RCV(_Msg, State),\n%%     log:pal(\"[ ~p ] CKIdx2: ~B (~B compressed)\",\n%%             [self(), erlang:byte_size(BinReqIdxPos),\n%%              erlang:byte_size(erlang:term_to_binary(BinReqIdxPos, [compressed]))]),\n    ToSend0 = bitstring_to_k_list_k(BinReqIdxPos, KList, []),\n    ToSend = [element(1, KV) || KV <- AdditionalItems] ++ ToSend0,\n    NewStats = send_resolve_request(\n                 Stats, ToSend, OwnerL, DestRRPid, IsInitiator, true),\n    shutdown(sync_finished, State#rr_recon_state{stats = NewStats});\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% merkle tree sync messages\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\non({?check_nodes, SenderPid, ToCheck, OtherItemsCount, OtherMaxItemsCount},\n   State = #rr_recon_state{params = #merkle_params{exp_delta = ExpDelta},\n                           misc = [{fail_rate_target_per_node, LastFRPerNode},\n                                   {fail_rate_target_phase1, FR_p1},\n                                   {prev_used_fr, PrevUsedFr},\n                                   {icount, MyMaxItemCount},\n                                   {total_icount, MyItemCount}]}) ->\n    % this is the first check_nodes message from the initiator and we finally\n    % learn about the exact number of items to synchronise with\n    MaxAffectedItems = calc_max_different_items_total(\n                         MyItemCount, OtherItemsCount, ExpDelta),\n    on({?check_nodes, ToCheck, OtherMaxItemsCount},\n       State#rr_recon_state{dest_recon_pid = SenderPid,\n                            misc = [{max_affected_items, MaxAffectedItems},\n                                    {fail_rate_target_per_node, LastFRPerNode},\n                                    {fail_rate_target_phase1, FR_p1},\n                                    {prev_used_fr, PrevUsedFr},\n                                    {icount, MyMaxItemCount}]});\n\non({?check_nodes, ToCheck0, OtherMaxItemsCount},\n   State = #rr_recon_state{stage = reconciliation,    initiator = false,\n                           method = merkle_tree,      merkle_sync = Sync,\n                           params = #merkle_params{} = Params,\n                           struct = TreeNodes,        stats = Stats,\n                           dest_recon_pid = DestReconPid,\n                           misc = [{max_affected_items, MaxAffectedItems},\n                                   {fail_rate_target_per_node, LastFRPerNode},\n                                   {fail_rate_target_phase1, FR_p1},\n                                   {prev_used_fr, PrevUsedFr},\n                                   {icount, MyLastMaxItemsCount}]}) ->\n    ?DBG_ASSERT(comm:is_valid(DestReconPid)),\n    {_FR_I, _FR_L, SigSizeI, SigSizeL, EffectiveFr_I, EffectiveFr_L} =\n        merkle_next_signature_sizes(Params, LastFRPerNode, MyLastMaxItemsCount,\n                                    OtherMaxItemsCount, MaxAffectedItems),\n    ToCheck = merkle_decompress_hashlist(ToCheck0, SigSizeI, SigSizeL),\n    {FlagsBin, RTree, SyncNew, NStats, MyMaxItemsCount,\n     NextLvlNodesAct, HashCmpI, HashCmpL} =\n        merkle_check_node(ToCheck, TreeNodes, SigSizeI, SigSizeL,\n                          MyLastMaxItemsCount, OtherMaxItemsCount, Params, Stats,\n                          <<>>, [], [], [], 0, [], 0, 0, Sync, 0, 0, 0,\n                          0, 0),\n    UsedFr = merkle_calc_used_fr(PrevUsedFr, EffectiveFr_I,\n                                 EffectiveFr_L, HashCmpI, HashCmpL),\n    NewState = State#rr_recon_state{struct = RTree, merkle_sync = SyncNew,\n                                    stats = NStats},\n    ?ALG_DEBUG(\"merkle (NI)~n  fail_rate(p1): ~p -> ~p - NextNodes: ~B\",\n               [PrevUsedFr, UsedFr, NextLvlNodesAct]),\n    send(DestReconPid, {?check_nodes_response, FlagsBin, MyMaxItemsCount}),\n    \n    if RTree =:= [] ->\n           % start a (parallel) resolve (if items to resolve)\n           merkle_resolve_leaves_send(NewState, UsedFr);\n       true ->\n           % calculate the remaining trees' target failure rate based on the\n           % already used failure prob\n           FRPerNode = calc_n_subparts_FR(NextLvlNodesAct, FR_p1, element(1, UsedFr)),\n           NewState#rr_recon_state{misc = [{max_affected_items, MaxAffectedItems},\n                                           {fail_rate_target_per_node, FRPerNode},\n                                           {fail_rate_target_phase1, FR_p1},\n                                           {prev_used_fr, UsedFr},\n                                           {icount, MyMaxItemsCount}]}\n    end;\n\non({?check_nodes_response, FlagsBin, OtherMaxItemsCount},\n   State = #rr_recon_state{stage = reconciliation,        initiator = true,\n                           method = merkle_tree,          merkle_sync = Sync,\n                           params = #merkle_params{} = Params,\n                           struct = TreeNodes,            stats = Stats,\n                           dest_recon_pid = DestReconPid,\n                           misc = [{signature_size, {SigSizeI, SigSizeL}},\n                                   {max_affected_items, MaxAffectedItems},\n                                   {fail_rate_target_per_node, {EffectiveFr_I, EffectiveFr_L}},\n                                   {fail_rate_target_phase1, FR_p1},\n                                   {prev_used_fr, PrevUsedFr},\n                                   {icount, MyLastMaxItemsCount},\n                                   {oicount, OtherLastMaxItemsCount}]}) ->\n    {RTree, SyncNew, NStats, MyMaxItemsCount,\n     NextLvlNodesAct, HashCmpI, HashCmpL} =\n        merkle_cmp_result(FlagsBin, TreeNodes, SigSizeI, SigSizeL,\n                          MyLastMaxItemsCount, OtherLastMaxItemsCount,\n                          Sync, Params, Stats, [], [], [], 0, [], 0, 0, 0, 0, 0,\n                          0, 0),\n    UsedFr = merkle_calc_used_fr(PrevUsedFr, EffectiveFr_I,\n                                 EffectiveFr_L, HashCmpI, HashCmpL),\n    NewState = State#rr_recon_state{struct = RTree, merkle_sync = SyncNew,\n                                    stats = NStats},\n    ?ALG_DEBUG(\"merkle (I)~n  fail_rate(p1): ~p -> ~p - NextNodes: ~B\",\n               [PrevUsedFr, UsedFr, NextLvlNodesAct]),\n\n    if RTree =:= [] ->\n           % start a (parallel) resolve (if items to resolve)\n           merkle_resolve_leaves_send(NewState, UsedFr);\n       true ->\n           % calculate the remaining trees' failure prob based on the already\n           % used failure prob\n           NextFRPerNode = calc_n_subparts_FR(NextLvlNodesAct, FR_p1, element(1, UsedFr)),\n           {_FR_I, _FR_L, NextSigSizeI, NextSigSizeL, NextEffectiveFr_I, NextEffectiveFr_L} =\n               merkle_next_signature_sizes(Params, NextFRPerNode, MyMaxItemsCount,\n                                           OtherMaxItemsCount, MaxAffectedItems),\n           Req = merkle_compress_hashlist(RTree, <<>>, NextSigSizeI, NextSigSizeL),\n           send(DestReconPid, {?check_nodes, Req, MyMaxItemsCount}),\n           NewState#rr_recon_state{misc = [{signature_size, {NextSigSizeI, NextSigSizeL}},\n                                           {max_affected_items, MaxAffectedItems},\n                                           {fail_rate_target_per_node, {NextEffectiveFr_I, NextEffectiveFr_L}},\n                                           {fail_rate_target_phase1, FR_p1},\n                                           {prev_used_fr, UsedFr},\n                                           {icount, MyMaxItemsCount},\n                                           {oicount, OtherMaxItemsCount}]}\n    end;\n\non({resolve_req, SyncStruct_p2} = _Msg,\n   State = #rr_recon_state{stage = resolve,           method = merkle_tree})\n  when is_tuple(SyncStruct_p2) ->\n    ?TRACE_RCV(_Msg, State),\n    % NOTE: FIFO channels ensure that the {resolve_req, BinKeyList} is always\n    %       received after the {resolve_req, Hashes} message from the other node!\n    merkle_resolve_leaves_receive(State, SyncStruct_p2);\n\non({resolve_req, BinKeyList, SyncSendFr_real} = _Msg,\n   State = #rr_recon_state{stage = resolve,           initiator = IsInitiator,\n                           method = merkle_tree,\n                           merkle_sync = {SyncSend, [], SyncRcvLeafCount, DirectResolve},\n                           params = Params,           next_phase_kv = ToSend,\n                           dest_rr_pid = DestRRPid,   ownerPid = OwnerL,\n                           stats = Stats})\n    % NOTE: FIFO channels ensure that the {resolve_req, BinKeyList} is always\n    %       received after the {resolve_req, Hashes} message from the other node!\n  when is_bitstring(BinKeyList) ->\n    ?TRACE_RCV(_Msg, State),\n    PrevFr_p2 = rr_recon_stats:get(fail_rate_p2, Stats), % from sync_receive\n    NextFr_p2 = PrevFr_p2 + SyncSendFr_real,\n    ?ALG_DEBUG(\"merkle (~s) - BinKeyListSize: ~B compressed~n\"\n               \"  fail_rate(p2): ~p (rcv) + ~p (send) = ~p\",\n               [?IIF(IsInitiator, \"I\", \"NI\"),\n                erlang:byte_size(\n                  erlang:term_to_binary(BinKeyList, [compressed])),\n                PrevFr_p2, SyncSendFr_real, NextFr_p2]),\n    Stats1  = rr_recon_stats:set([{fail_rate_p2, NextFr_p2}], Stats),\n    ToSendKeys = [K || {K, _V} <- ToSend],\n    NStats = if BinKeyList =:= <<>> andalso ToSendKeys =:= [] ->\n                    Stats1;\n                BinKeyList =:= <<>> ->\n                    merkle_resolve_leaves_ckidx(\n                      [], BinKeyList, DestRRPid, Stats1, OwnerL,\n                      Params, ToSendKeys, IsInitiator);\n                true ->\n                    merkle_resolve_leaves_ckidx(\n                      SyncSend, BinKeyList, DestRRPid, Stats1, OwnerL,\n                      Params, ToSendKeys, IsInitiator)\n             end,\n    NewState = State#rr_recon_state{merkle_sync = {[], [], SyncRcvLeafCount, DirectResolve},\n                                    stats = NStats},\n    shutdown(sync_finished, NewState).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec build_struct(DBList::db_chunk_kv(),\n                   RestI::intervals:interval(), state()) -> state() | kill.\nbuild_struct(DBList, RestI,\n             State = #rr_recon_state{method = RMethod, params = Params,\n                                     struct = {},\n                                     initiator = Initiator, stats = Stats,\n                                     kv_list = KVList,\n                                     'sync_interval@I' = SyncI,\n                                     'max_items@I' = InitiatorMaxItems}) ->\n    ?DBG_ASSERT(?implies(RMethod =/= merkle_tree andalso RMethod =/= art,\n                         not Initiator)),\n    % note: RestI already is a sub-interval of the sync interval\n    BeginSync = intervals:is_empty(RestI),\n    NewKVList0 = [DBList | KVList],\n    if BeginSync ->\n           NewKVList = lists:append(lists:reverse(NewKVList0)),\n           ToBuild = if Initiator andalso RMethod =:= art -> merkle_tree;\n                        true -> RMethod\n                     end,\n           {BuildTime, {SyncStruct, Fr_p1}} =\n               util:tc(\n                 fun() -> build_recon_struct(ToBuild, SyncI, NewKVList,\n                                             InitiatorMaxItems, Params)\n                 end),\n           Stats1 = rr_recon_stats:inc([{build_time, BuildTime}], Stats),\n           Stats2  = rr_recon_stats:set([{fail_rate_p1, Fr_p1}], Stats1),\n           NewState = State#rr_recon_state{struct = SyncStruct, stats = Stats2,\n                                           kv_list = NewKVList},\n           begin_sync(NewState#rr_recon_state{stage = reconciliation});\n       true ->\n           send_chunk_req(pid_groups:get_my(dht_node), self(),\n                          RestI, get_max_items()),\n           % keep stage (at initiator: reconciliation, at other: build_struct)\n           State#rr_recon_state{kv_list = NewKVList0}\n    end.\n\n-spec begin_sync(state()) -> state() | kill.\nbegin_sync(State = #rr_recon_state{method = trivial, params = {}, initiator = false,\n                                   struct = MySyncStruct,\n                                   ownerPid = OwnerL, stats = Stats,\n                                   dest_rr_pid = DestRRPid, kv_list = _KVList}) ->\n    ?ALG_DEBUG(\"BEGIN SYNC\", []),\n    SID = rr_recon_stats:get(session_id, Stats),\n    {KList, VList, ResortedKVOrigList, Dupes} = MySyncStruct#trivial_sync.db_chunk,\n    MySyncStruct1 = MySyncStruct#trivial_sync{db_chunk = {KList, VList}},\n    send(DestRRPid, {continue_recon, comm:make_global(OwnerL), SID,\n                     {start_recon, trivial, MySyncStruct1}}),\n    State#rr_recon_state{struct = {}, stage = resolve, kv_list = [],\n                         k_list = [K || {K, _V} <- ResortedKVOrigList],\n                         next_phase_kv = Dupes};\nbegin_sync(State = #rr_recon_state{method = shash, params = {}, initiator = false,\n                                   struct = MySyncStruct,\n                                   ownerPid = OwnerL, stats = Stats,\n                                   dest_rr_pid = DestRRPid, kv_list = _KVList}) ->\n    ?ALG_DEBUG(\"BEGIN SYNC\", []),\n    SID = rr_recon_stats:get(session_id, Stats),\n    {KList, ResortedKVOrigList, Dupes} = MySyncStruct#shash_sync.db_chunk,\n    MySyncStruct1 = MySyncStruct#shash_sync{db_chunk = KList},\n    send(DestRRPid, {continue_recon, comm:make_global(OwnerL), SID,\n                     {start_recon, shash, MySyncStruct1}}),\n    if KList =:= <<>> andalso Dupes =:= [] -> % no items on this node\n           shutdown(sync_finished, State#rr_recon_state{struct = {}, kv_list = []});\n       true ->\n           State#rr_recon_state{struct = {}, stage = resolve,\n                                kv_list = ResortedKVOrigList,\n                                next_phase_kv = Dupes}\n    end;\nbegin_sync(State = #rr_recon_state{method = bloom, params = {}, initiator = false,\n                                   struct = MySyncStruct,\n                                   ownerPid = OwnerL, stats = Stats,\n                                   dest_rr_pid = DestRRPid}) ->\n    ?ALG_DEBUG(\"BEGIN SYNC\", []),\n    SID = rr_recon_stats:get(session_id, Stats),\n    BFBin = bloom:get_property(MySyncStruct#bloom_sync.bf, filter),\n    MySyncStruct1 = MySyncStruct#bloom_sync{bf = BFBin},\n    send(DestRRPid, {continue_recon, comm:make_global(OwnerL), SID,\n                     {start_recon, bloom, MySyncStruct1}}),\n    case MySyncStruct#bloom_sync.item_count of\n        0 -> shutdown(sync_finished, State#rr_recon_state{kv_list = []});\n        _ -> State#rr_recon_state{stage = reconciliation}\n    end;\nbegin_sync(State = #rr_recon_state{method = merkle_tree, params = {}, initiator = false,\n                                   struct = MySyncStruct,\n                                   ownerPid = OwnerL, stats = Stats,\n                                   dest_rr_pid = DestRRPid}) ->\n    ?ALG_DEBUG(\"BEGIN SYNC\", []),\n    % tell the initiator to create its struct first, and then build ours\n    % (at this stage, we do not have any data in the merkle tree yet!)\n    DBItems = State#rr_recon_state.kv_list,\n    MerkleI = merkle_tree:get_interval(MySyncStruct),\n    MerkleV = merkle_tree:get_branch_factor(MySyncStruct),\n    MerkleB = merkle_tree:get_bucket_size(MySyncStruct),\n    NumTrees = get_merkle_num_trees(),\n    FRTotal = get_failure_rate(),\n    FRTotal_p1 = calc_n_subparts_FR(2, FRTotal),\n    \n    % split interval first and create NumTrees merkle trees later\n    {BuildTime1, ICBList} =\n        util:tc(fun() ->\n                        merkle_tree:keys_to_intervals(\n                          DBItems, intervals:split(MerkleI, NumTrees))\n                end),\n    ItemCount = lists:sum([Count || {_SubI, Count, _Bucket} <- ICBList]),\n    MaxItemCount = lists:max([0 | [Count || {_SubI, Count, _Bucket} <- ICBList]]),\n    FRTotal_p1_perNode = calc_n_subparts_FR(NumTrees, FRTotal_p1),\n    \n    MySyncParams = #merkle_params{exp_delta = get_max_expected_delta(),\n                                  interval = MerkleI,\n                                  branch_factor = MerkleV,\n                                  bucket_size = MerkleB,\n                                  num_trees = NumTrees,\n                                  fail_rate = FRTotal,\n                                  ni_item_count = ItemCount,\n                                  ni_max_ic = MaxItemCount},\n    SyncParams = MySyncParams#merkle_params{reconPid = comm:this()},\n    SID = rr_recon_stats:get(session_id, Stats),\n    send(DestRRPid, {continue_recon, comm:make_global(OwnerL), SID,\n                     {start_recon, merkle_tree, SyncParams}}),\n    \n    % finally create the real merkle tree containing data\n    % -> this way, the initiator can create its struct in parallel!\n    {BuildTime2, SyncStruct} =\n        util:tc(fun() ->\n                        [merkle_tree:get_root(\n                           merkle_tree:new(SubI, Bucket,\n                                           [{keep_bucket, true},\n                                            {branch_factor, MerkleV},\n                                            {bucket_size, MerkleB}]))\n                           || {SubI, _Count, Bucket} <- ICBList]\n                end),\n    MTSize = merkle_tree:size_detail(SyncStruct),\n    Stats1 = rr_recon_stats:set([{tree_size, MTSize}], Stats),\n    Stats2 = rr_recon_stats:inc([{build_time, BuildTime1 + BuildTime2}], Stats1),\n    ?ALG_DEBUG(\"merkle (NI) - NextNodes: ~B~n\"\n               \"  Inner/Leaf/EmptyLeaves/Items: ~p\",\n               [length(SyncStruct), MTSize]),\n    \n    State#rr_recon_state{struct = SyncStruct,\n                         stats = Stats2, params = MySyncParams,\n                         misc = [{fail_rate_target_per_node, FRTotal_p1_perNode},\n                                 {fail_rate_target_phase1, FRTotal_p1},\n                                 {prev_used_fr, {0.0, 0.0}},\n                                 {icount, MaxItemCount},\n                                 {total_icount, ItemCount}],\n                         kv_list = []};\nbegin_sync(State = #rr_recon_state{method = merkle_tree, params = Params, initiator = true,\n                                   struct = MySyncStruct, stats = Stats,\n                                   dest_recon_pid = DestReconPid}) ->\n    ?ALG_DEBUG(\"BEGIN SYNC\", []),\n    MTSize = merkle_tree:size_detail(MySyncStruct),\n    Stats1 = rr_recon_stats:set([{tree_size, MTSize}], Stats),\n    #merkle_params{fail_rate = FRTotal, num_trees = NumTrees, exp_delta = ExpDelta,\n                   ni_item_count = OtherItemsCount, ni_max_ic = OtherMaxItemsCount} = Params,\n    MyItemCount =\n        lists:sum([merkle_tree:get_item_count(Node) || Node <- MySyncStruct]),\n    MyMaxItemCount =\n        lists:max([0 | [merkle_tree:get_item_count(Node) || Node <- MySyncStruct]]),\n    ?ALG_DEBUG(\"merkle (I) - NextNodes: ~B~n\"\n               \"  Inner/Leaf/EmptyLeaves/Items: ~p\",\n               [length(MySyncStruct), MTSize]),\n    FRTotal_p1 = calc_n_subparts_FR(2, FRTotal),\n    FRTotal_p1_perNode = calc_n_subparts_FR(NumTrees, FRTotal_p1),\n    MaxAffectedItems = calc_max_different_items_total(\n                         MyItemCount, OtherItemsCount, ExpDelta),\n    \n    {_FR_I, _FR_L, NextSigSizeI, NextSigSizeL, EffectiveFr_I, EffectiveFr_L} =\n        merkle_next_signature_sizes(Params, FRTotal_p1_perNode, MyMaxItemCount,\n                                    OtherMaxItemsCount, MaxAffectedItems),\n    \n    Req = merkle_compress_hashlist(MySyncStruct, <<>>, NextSigSizeI, NextSigSizeL),\n    send(DestReconPid, {?check_nodes, comm:this(), Req, MyItemCount, MyMaxItemCount}),\n    State#rr_recon_state{stats = Stats1,\n                         misc = [{signature_size, {NextSigSizeI, NextSigSizeL}},\n                                 {max_affected_items, MaxAffectedItems},\n                                 {fail_rate_target_per_node, {EffectiveFr_I, EffectiveFr_L}},\n                                 {fail_rate_target_phase1, FRTotal_p1},\n                                 {prev_used_fr, {0.0, 0.0}},\n                                 {icount, MyMaxItemCount},\n                                 {oicount, OtherMaxItemsCount}],\n                         kv_list = []};\nbegin_sync(State = #rr_recon_state{method = art, params = {}, initiator = false,\n                                   struct = MySyncStruct,\n                                   ownerPid = OwnerL, stats = Stats,\n                                   dest_rr_pid = DestRRPid}) ->\n    ?ALG_DEBUG(\"BEGIN SYNC\", []),\n    SID = rr_recon_stats:get(session_id, Stats),\n    send(DestRRPid, {continue_recon, comm:make_global(OwnerL), SID,\n                     {start_recon, art, MySyncStruct}}),\n    case art:get_property(MySyncStruct#art_recon_struct.art, items_count) of\n        0 -> shutdown(sync_finished, State#rr_recon_state{kv_list = []});\n        _ -> State#rr_recon_state{struct = {}, stage = resolve}\n    end;\nbegin_sync(State = #rr_recon_state{method = art, params = Params, initiator = true,\n                                   struct = MySyncStruct, stats = Stats,\n                                   dest_recon_pid = DestReconPid}) ->\n    ?ALG_DEBUG(\"BEGIN SYNC\", []),\n    ART = Params#art_recon_struct.art,\n    Stats1 = rr_recon_stats:set(\n               [{tree_size, merkle_tree:size_detail(MySyncStruct)}], Stats),\n    OtherItemCount = art:get_property(ART, items_count),\n    case merkle_tree:get_interval(MySyncStruct) =:= art:get_interval(ART) of\n        true ->\n            {ASyncLeafs, NComp, NSkip, NLSync} =\n                art_get_sync_leaves([merkle_tree:get_root(MySyncStruct)], ART,\n                                    [], 0, 0, 0),\n            Diff = lists:append([merkle_tree:get_bucket(N) || N <- ASyncLeafs]),\n            MyItemCount = merkle_tree:get_item_count(MySyncStruct),\n            FRTotal = get_failure_rate(),\n            % TODO: correctly calculate the probabilities and select appropriate parameters beforehand\n            LeafBf = art:get_property(ART, leaf_bf),\n            LeafBfFpr = bloom:get_property(LeafBf, fpr),\n            NrChecksNotInBF =\n                calc_max_different_items_node(\n                  bloom:get_property(LeafBf, items_count),\n                  MyItemCount, get_max_expected_delta()),\n            EffectiveFr_p1 = NrChecksNotInBF * LeafBfFpr,\n            FR_p2 =\n                if EffectiveFr_p1 > FRTotal ->\n                       log:log(\"~w: [ ~.0p:~.0p ] FR constraint broken (phase 1 overstepped?)~n\"\n                               \"  continuing with smallest possible failure rate\",\n                               [?MODULE, pid_groups:my_groupname(), self()]),\n                       1.0e-16;\n                   true ->\n                       % NOTE: use left-over failure rate after phase 1 (ART) for phase 2 (trivial RC)\n                       calc_n_subparts_FR(1, FRTotal, EffectiveFr_p1)\n                end,\n            Stats2  = rr_recon_stats:set([{fail_rate_p1, EffectiveFr_p1}], Stats1),\n            \n            Stats3 = rr_recon_stats:inc([{tree_nodesCompared, NComp},\n                                         {tree_compareSkipped, NSkip},\n                                         {tree_leavesSynced, NLSync}], Stats2),\n            \n            phase2_run_trivial_on_diff(Diff, none, OtherItemCount =:= 0,\n                                       FR_p2, OtherItemCount,\n                                       State#rr_recon_state{stats = Stats3});\n        false when OtherItemCount =/= 0 ->\n            % must send resolve_req message for the non-initiator to shut down\n            send(DestReconPid, {resolve_req, shutdown, none}),\n            shutdown(sync_finished, State#rr_recon_state{stats = Stats1,\n                                                         kv_list = []});\n        false ->\n            shutdown(sync_finished, State#rr_recon_state{stats = Stats1,\n                                                         kv_list = []})\n    end.\n\n-spec shutdown(exit_reason(), state()) -> kill.\nshutdown(Reason, #rr_recon_state{ownerPid = OwnerL, stats = Stats,\n                                 initiator = Initiator, dest_rr_pid = DestRR,\n                                 dest_recon_pid = DestRC, method = RMethod,\n                                 'sync_interval@I' = SyncI}) ->\n    ?ALG_DEBUG(\"SHUTDOWN~n  Session=~.0p Reason=~p\",\n               [rr_recon_stats:get(session_id, Stats), Reason]),\n\n    % unsubscribe from fd if a subscription was made:\n    case Initiator orelse (not intervals:is_empty(SyncI)) of\n        true ->\n            case RMethod of\n                trivial -> fd:unsubscribe(self(), [DestRR]);\n                bloom   -> fd:unsubscribe(self(), [DestRR]);\n                merkle_tree -> fd:unsubscribe(self(), [DestRR]);\n                _ -> ok\n            end;\n        false -> ok\n    end,\n\n    Status = exit_reason_to_rc_status(Reason),\n    NewStats = rr_recon_stats:set([{status, Status}], Stats),\n    send_local(OwnerL, {recon_progress_report, comm:this(), Initiator, DestRR,\n                        DestRC, NewStats}),\n    kill.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% KV-List compression\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Helper calculating 1 - (1 - Z)^X.\n-spec calc_one_m_xpow_one_m_z(X::number(), Z::float()) -> float().\ncalc_one_m_xpow_one_m_z(X, _Z) when X == 0 ->\n    0.0;\ncalc_one_m_xpow_one_m_z(_X, Z) when Z == 0 ->\n    0.0;\ncalc_one_m_xpow_one_m_z(_X, Z) when Z == 1 ->\n    1.0;\ncalc_one_m_xpow_one_m_z(X, Z) when Z < 1.0e-8 ->\n    % BEWARE: we cannot use (1-z) since it is near 1 and its floating\n    %         point representation is sub-optimal!\n    % => use Taylor expansion of 1 - (1 - z)^x  at z = 0\n    % http://www.wolframalpha.com/input/?i=Taylor+expansion+of+1+-+(1+-+z)%5Ex++at+z+%3D+0\n    X2 = X * X, X3 = X2 * X, X4 = X3 * X, X5 = X4 * X,\n    Z2 = Z * Z, Z3 = Z2* Z, Z4 = Z3 * Z, Z5 = Z4 * Z,\n    % small terms first:\n%%     Z5 * (X5 - 10*X4 + 35*X3 - 50*X2 + 24*X) / 120\n%%         + Z4 * (X4 - 6*X3 + 11*X2 - 6*X) / 24\n%%         + Z3 * (X3 - 3*X2 + 2*X) / 6\n%%         + Z2 * (X2 - X) / 2 + Z * X; % +O[p^6]\n    {S5, C5} = util:kahan_sum([X5, -10*X4, 35*X3, -50*X2, 24*X], 0.0, 0.0),\n    {S4, C4} = util:kahan_sum([X4, -6*X3, 11*X2, -6*X], 0.0, 0.0),\n    {S3, C3} = util:kahan_sum([X3, -3*X2, 2*X], 0.0, 0.0),\n    {S2, C2} = util:kahan_sum([X2, -X], 0.0, 0.0),\n    \n    {Sum, _C} = util:kahan_sum(\n                  [Z5 * C5 / 120, Z4 * C4 / 24, Z3 * C3 / 6, Z2 * C2 / 2,\n                   Z5 * S5 / 120, Z4 * S4 / 24, Z3 * S3 / 6, Z2 * S2 / 2,\n                   Z * X % +O[z^6]\n                  ], 0.0, 0.0),\n    Sum;\ncalc_one_m_xpow_one_m_z(X, Z) ->\n    1 - math:pow(1 - Z, X).\n\n%% @doc Helper for calculating the maximum number of different hashes when\n%%      an upper bound on the delta is known, ignoring cases where it is\n%%      known to be wrong based on N and M.\n-spec calc_max_different_hashes_(N::non_neg_integer(), M::non_neg_integer(),\n                                 ExpDelta::number()) -> non_neg_integer().\ncalc_max_different_hashes_(N, M, ExpDelta) when ExpDelta >= 0 andalso ExpDelta =< 100 ->\n    if ExpDelta == 0 ->\n           % M and N may differ anyway if the actual delta is higher\n           % -> target no collisions among items on any node!\n           erlang:max(M, N);\n       ExpDelta == 100 ->\n           M + N; % special case of the ones below\n       is_float(ExpDelta) ->\n           % assume the worst case, i.e. ExpDelta percent different hashes\n           % on both nodes together due to missing items (out-dated items have\n           % the same key!), and thus:\n           % N = NT * (100 - ExpDelta * alpha) / 100 and\n           % M = NT * (100 - ExpDelta * (1-alpha)) / 100\n           util:ceil(((M + N) * 100) / (200 - ExpDelta));\n       is_integer(ExpDelta) ->\n           % -> use integer division (and round up) for higher precision:\n           ((M + N) * 100 + 200 - ExpDelta - 1) div (200 - ExpDelta)\n    end.\n\n%% @doc Helper for calculating the maximum number of different hashes when\n%%      an upper bound on the delta is known. This method also handles cases\n%%      where ExpDelta is known to be wrong and sets the minimum known value\n%%      after printing a warning.\n-spec calc_max_different_hashes(N::non_neg_integer(), M::non_neg_integer(),\n                                ExpDelta::number()) -> non_neg_integer().\ncalc_max_different_hashes(N, M, ExpDelta) when ExpDelta >= 0 andalso ExpDelta =< 100 ->\n    Differences0 = calc_max_different_hashes_(N, M, ExpDelta),\n    % in case ExpDelta is wrong, at least |N-M| items must differ and are missing on one node\n    MinDiff = erlang:abs(N - M),\n    if MinDiff =< Differences0 ->\n           Differences0;\n       true ->\n           log:log(\"~w: [ ~.0p:~.0p ] expected delta ~B must be wrong,~n\"\n                   \"  continuing with the minimum number of differences we know: ~B\",\n                   [?MODULE, pid_groups:my_groupname(), self(), Differences0, MinDiff]),\n           MinDiff\n    end.\n\n%% @doc Calculates the maximum number of different items on both nodes (with N\n%%      and M items each) which may differ, i.e. which may not exist on either\n%%      node when an upper bound on the delta is known.\n-spec calc_max_different_items_total(N::non_neg_integer(), M::non_neg_integer(),\n                               ExpDelta::number()) -> non_neg_integer().\ncalc_max_different_items_total(N, M, ExpDelta) ->\n    % the worst case with the maximal number of item checks is when\n    % this node has all ExpDelta items missing\n    % -> calculate the (expected) original item count\n    % -> have ExpDelta percent different items on either node\n    % (if items are only outdated, there are ExpDelta differences from (N+M)/2\n    % items which is less than the worst case above)\n\n    % note: avoid duplicate warnings if ExpDelta is wrong - if it is wrong here,\n    %       we do not need to fix MaxItems because the actual Differences are\n    %       set below either way:\n    MaxItems = calc_max_different_hashes_(N, M, ExpDelta),\n    Differences0 =\n        if ExpDelta == 0   -> 0;\n           ExpDelta == 100 -> MaxItems; % special case of the one below\n           is_float(ExpDelta) ->\n               % worst case: we have all the ExpDelta percent items the other node does not have\n               util:ceil(MaxItems * ExpDelta / 100);\n           is_integer(ExpDelta) ->\n               % -> use integer division (and round up) for higher precision:\n               (MaxItems * ExpDelta + 99) div 100\n        end,\n    % in case ExpDelta is wrong, at least |N-M| items must differ and are missing on one node\n    MinDiff = erlang:abs(N - M),\n    Differences =\n        if MinDiff =< Differences0 ->\n               Differences0;\n           true ->\n               log:log(\"~w: [ ~.0p:~.0p ] expected delta ~B must be wrong,~n\"\n                       \"  continuing with the minimum number of differences we know: ~B\",\n                       [?MODULE, pid_groups:my_groupname(), self(), Differences0, MinDiff]),\n               MinDiff\n        end,\n%%     log:pal(\"[ ~p ] MaxItems: ~B Differences: ~B -> ~B\", [self(), MaxItems, Differences0, Differences]),\n    Differences.\n\n%% @doc Calculates the maximum number of items on the first node (N items)\n%%      which do not exist on the second node (M items) when an upper bound\n%%      on the delta is known. Note that the result is limited by N which\n%%      is the maximum number of differences on the first node.\n%% @see calc_max_different_items_total/3\n-spec calc_max_different_items_node(N::non_neg_integer(), M::non_neg_integer(),\n                               ExpDelta::number()) -> non_neg_integer().\ncalc_max_different_items_node(N, M, ExpDelta) ->\n    erlang:min(calc_max_different_items_total(N, M, ExpDelta), N).\n\n-spec calc_items_in_chunk(DBChunk::bitstring(), BitsPerItem::non_neg_integer())\n-> NrItems::non_neg_integer().\ncalc_items_in_chunk(<<>>, 0) -> 0;\ncalc_items_in_chunk(DBChunk, BitsPerItem) ->\n    ?ASSERT(erlang:bit_size(DBChunk) rem BitsPerItem =:= 0),\n    erlang:bit_size(DBChunk) div BitsPerItem.\n\n%% @doc Sorts the given list of tuples, and extracts duplicates (non-sorted)\n%%      based on the first tuple element.\n-spec sort_extractdupes(List::[Tuple,...])\n        -> {SortedList::[Tuple], Dupes::[Tuple]}\n   when is_subtype(Tuple, {term()} | {term(), term()} | {term(), term(), term()}).\nsort_extractdupes([_ | _] = List) ->\n    SortedList0 = lists:sort(List),\n    % detect and remove duplicates:\n    {Last, LastDup, Rest, Dupes0} =\n        lists:foldl(fun(X, {LastX, _, RestX, Dupes})\n                         when element(1, X) =:= element(1, LastX) ->\n                            {X, dup, RestX, [LastX | Dupes]};\n                       (X, {LastX, nodup, RestX, Dupes}) ->\n                            {X, nodup, [LastX | RestX], Dupes};\n                       (X, {LastX, dup, RestX, Dupes}) ->\n                            {X, nodup, RestX, [LastX | Dupes]}\n                    end, {hd(SortedList0), nodup, [], []}, tl(SortedList0)),\n    if LastDup =:= nodup ->\n           {lists:reverse(Rest, [Last]), Dupes0};\n       true ->\n           {lists:reverse(Rest),\n            % this is reversed but the order does not matter for duplicates:\n            [Last | Dupes0]}\n    end.\n\n%% @doc Transforms a list of key and version tuples (with unique keys), into a\n%%      compact binary representation for transfer.\n-spec compress_kv_list\n        (KVList::db_chunk_kv(), SigSize, VSize::signature_size(),\n         KeyComprFun::fun(({?RT:key(), client_version()}, SigSize) -> bitstring()),\n         integrate_size)\n    -> {KeyDiff::Bin, VBin::Bin, ResortedKOrigList::db_chunk_kv(), Dupes::db_chunk_kv()};\n        (KVList::db_chunk_kv(), SigSize, VSize::signature_size(),\n         KeyComprFun::fun(({?RT:key(), client_version()}, SigSize) -> bitstring()),\n         return_size)\n    -> {{KeyDiff::Bin, DeltaSigSizeBits::non_neg_integer(), DeltaSigSize::non_neg_integer()},\n        VBin::Bin, ResortedKOrigList::db_chunk_kv(), Dupes::db_chunk_kv()}\n    when is_subtype(Bin, bitstring()),\n         is_subtype(SigSize, signature_size()).\ncompress_kv_list([_ | _], 0, 0, _KeyComprFun, integrate_size) ->\n    {<<>>, <<>>, [], []};\ncompress_kv_list([_ | _], 0, 0, _KeyComprFun, return_size) ->\n    {{<<>>, 0, 0}, <<>>, [], []};\ncompress_kv_list([_ | _] = KVList, SigSize, VSize, KeyComprFun, SizeOption) ->\n    {SortedKVList, Dupes0} =\n        sort_extractdupes([{KeyComprFun(X, SigSize), V, X}\n                           || X = {_K0, V} <- KVList]),\n    {KList, VList, KV0List} = lists:unzip3(SortedKVList),\n    DiffKStruct = compress_idx_list(KList, util:pow(2, SigSize) - 1, [], 0, 0, SizeOption),\n    DiffVBin = if VSize =:= 0 -> <<>>;\n                  true -> lists:foldl(fun(V, Acc) ->\n                                              <<Acc/bitstring, V:VSize>>\n                                      end, <<>>, VList)\n               end,\n    Dupes = [element(3, D) || D <- Dupes0],\n    {DiffKStruct, DiffVBin, KV0List, Dupes};\ncompress_kv_list([], _SigSize, _VSize, _KeyComprFun, integrate_size) ->\n    {<<>>, <<>>, [], []};\ncompress_kv_list([], _SigSize, _VSize, _KeyComprFun, return_size) ->\n    {{<<>>, 0, 0}, <<>>, [], []}.\n\n%% @doc De-compresses the binary from compress_kv_list/5 into a map with a\n%%      binary key representation and the integer of the (shortened) version.\n-spec decompress_kv_list(CompressedBin::{KeyDiff::bitstring(), VBin::bitstring()},\n                         DeltaSigSize::signature_size() | unknown,\n                         SigSize::signature_size())\n        -> {ResTree::kvi_tree(), VSize::signature_size()}.\ndecompress_kv_list({<<>>, <<>>}, _DeltaSigSize, _SigSize) ->\n    {mymaps:new(), 0};\ndecompress_kv_list({KeyDiff, VBin}, DeltaSigSize, SigSize) ->\n    {KList, KListLen} = decompress_idx_list(KeyDiff, DeltaSigSize, util:pow(2, SigSize) - 1),\n    VBinSize = erlang:bit_size(VBin),\n    VSize = VBinSize div KListLen,\n    ?ASSERT(VBinSize rem KListLen =:= 0),\n    {<<>>, Res, _} =\n        lists:foldl(\n          fun(CurKeyX, {<<Version:VSize, T/bitstring>>, AccX, CurPosX}) ->\n                  {T, [{CurKeyX, {Version, CurPosX}} | AccX], CurPosX + 1}\n          end, {VBin, [], 0}, KList),\n    KVMap = mymaps:from_list(Res),\n    % hashes, i.e. compressed keys, should be unique in KeyDiff!\n    ?DBG_ASSERT(mymaps:size(KVMap) =:= KListLen),\n    {KVMap, VSize}.\n\n%% @doc Wrapper for GetDiffFun which also extracts items with colliding\n%%      hashes in MyEntries. These are added to the items to send and since\n%%      they are not matched with MyIOtherKvTree, colliding items there remain. \n%% @see get_full_diff/4\n%% @see get_part_diff/4\n-spec get_diff_with_dupes(\n        MyEntries::db_chunk_kv(), MyIOtherKvTree::kvi_tree(),\n        AccFBItems::[?RT:key()], AccReqItems::[non_neg_integer()],\n        SigSize::signature_size(), VSize::signature_size(),\n        GetDiffFun::fun((MyEntries::[EntryWithHash], MyIOtherKvTree::kvi_tree(),\n                         AccFBItems::[?RT:key()], AccReqItems::[non_neg_integer()])\n                       -> {FBItems::[?RT:key()], ReqItemsIdx::[non_neg_integer()],\n                           MyIOtherKvTree::kvi_tree()}))\n        -> {FBItems::[?RT:key()], ReqItemsIdx::[non_neg_integer()],\n            MyIOtherKvTree::kvi_tree(), Dupes::[EntryWithHash]}\n    when is_subtype(EntryWithHash, {HashedKey::non_neg_integer(), Key::?RT:key(), VShort::integer()}).\nget_diff_with_dupes([], MyIOtKvTree, FBItems, ReqItemsIdx, _SigSize, _VSize, _GetDiffFun) ->\n    {FBItems, ReqItemsIdx, MyIOtKvTree, []};\nget_diff_with_dupes([_ | _] = KVList, MyIOtKvTree, FBItems, ReqItemsIdx, SigSize, VSize, GetDiffFun) ->\n    VMod = util:pow(2, VSize),\n    {KVList1, Dupes} =\n        sort_extractdupes([begin\n                               {KeyShort, VersionShort} =\n                                   compress_kv_pair(Key, Version, SigSize, VMod),\n                               {KeyShort, Key, VersionShort}\n                           end || {Key, Version} <- KVList]),\n    {ToSend1, ToReqIdx1, OtherDBChunk1} =\n        GetDiffFun(KVList1, MyIOtKvTree,\n                   [element(2, D) || D <- Dupes] ++ FBItems, ReqItemsIdx),\n    {ToSend1, ToReqIdx1, OtherDBChunk1, Dupes}.\n\n%% @doc Gets all entries from MyEntries which are not encoded in MyIOtherKvTree\n%%      or the entry in MyEntries has a newer version than the one in the tree\n%%      and returns them as FBItems. ReqItems contains items in the tree but\n%%      where the version in MyEntries is older than the one in the tree.\n%% @see get_part_diff/4\n-spec get_full_diff(MyEntries::[{HashedKey::non_neg_integer(), Key::?RT:key(), VShort::integer()}],\n                    MyIOtherKvTree::kvi_tree(), AccFBItems::[?RT:key()],\n                    AccReqItems::[non_neg_integer()])\n        -> {FBItems::[?RT:key()], ReqItemsIdx::[non_neg_integer()],\n            MyIOtherKvTree::kvi_tree()}.\nget_full_diff([], MyIOtKvTree, FBItems, ReqItemsIdx) ->\n    {FBItems, ReqItemsIdx, MyIOtKvTree};\nget_full_diff([{KeyShort, Key, VersionShort} | Rest], MyIOtKvTree, FBItems, ReqItemsIdx) ->\n    case mymaps:find(KeyShort, MyIOtKvTree) of\n        error ->\n            get_full_diff(Rest, MyIOtKvTree, [Key | FBItems], ReqItemsIdx);\n        {ok, {OtherVersionShort, Idx}} ->\n            MyIOtKvTree2 = mymaps:remove(KeyShort, MyIOtKvTree),\n            if VersionShort > OtherVersionShort ->\n                   get_full_diff(Rest, MyIOtKvTree2, [Key | FBItems], ReqItemsIdx);\n               VersionShort =:= OtherVersionShort ->\n                   get_full_diff(Rest, MyIOtKvTree2, FBItems, ReqItemsIdx);\n               true -> % VersionShort < OtherVersionShort\n                   get_full_diff(Rest, MyIOtKvTree2, FBItems, [Idx | ReqItemsIdx])\n            end\n    end.\n\n%% @doc Gets all entries from MyEntries which are in MyIOtherKvTree\n%%      and the entry in MyEntries has a newer version than the one in the tree\n%%      and returns them as FBItems. ReqItems contains items in the tree but\n%%      where the version in MyEntries is older than the one in the tree.\n%% @see get_full_diff/4\n-spec get_part_diff(MyEntries::[{HashedKey::non_neg_integer(), Key::?RT:key(), VShort::integer()}],\n                    MyIOtherKvTree::kvi_tree(), AccFBItems::[?RT:key()],\n                    AccReqItems::[non_neg_integer()])\n        -> {FBItems::[?RT:key()], ReqItemsIdx::[non_neg_integer()],\n            MyIOtherKvTree::kvi_tree()}.\nget_part_diff([], MyIOtKvTree, FBItems, ReqItemsIdx) ->\n    {FBItems, ReqItemsIdx, MyIOtKvTree};\nget_part_diff([{KeyShort, Key, VersionShort} | Rest], MyIOtKvTree, FBItems, ReqItemsIdx) ->\n    case mymaps:find(KeyShort, MyIOtKvTree) of\n        error ->\n            get_part_diff(Rest, MyIOtKvTree, FBItems, ReqItemsIdx);\n        {ok, {OtherVersionShort, Idx}} ->\n            MyIOtKvTree2 = mymaps:remove(KeyShort, MyIOtKvTree),\n            if VersionShort > OtherVersionShort ->\n                   get_part_diff(Rest, MyIOtKvTree2, [Key | FBItems], ReqItemsIdx);\n               VersionShort =:= OtherVersionShort ->\n                   get_part_diff(Rest, MyIOtKvTree2, FBItems, ReqItemsIdx);\n               true ->\n                   get_part_diff(Rest, MyIOtKvTree2, FBItems, [Idx | ReqItemsIdx])\n            end\n    end.\n\n%% @doc Transforms a single key and version into compact representations\n%%      based on the given size and VMod, respectively.\n%% @see compress_kv_list/5.\n-spec compress_kv_pair(Key::?RT:key(), Version::client_version(),\n                        SigSize::signature_size(), VMod::pos_integer())\n        -> {KeyShort::non_neg_integer(), VersionShort::integer()}.\ncompress_kv_pair(Key, Version, SigSize, VMod) ->\n    {compress_key(Key, SigSize), Version rem VMod}.\n\n%% @doc Transforms a key or a KV-tuple into a compact binary representation\n%%      based on the given size.\n%% @see compress_kv_list/5.\n-spec compress_key(Key::?RT:key() | {Key::?RT:key(), Version::client_version()},\n                   SigSize::signature_size()) -> KeyShort::non_neg_integer().\ncompress_key(Key, SigSize) ->\n    KBin = erlang:md5(erlang:term_to_binary(Key)),\n    RestSize = erlang:bit_size(KBin) - SigSize,\n    % return an integer based on the last SigSize bits:\n    if RestSize >= 0  ->\n           <<_:RestSize/bitstring, KeyShort:SigSize/integer-unit:1>> = KBin,\n           KeyShort;\n       true ->\n           FillSize = -RestSize,\n           <<KeyShort:SigSize/integer-unit:1>> = <<0:FillSize, KBin/binary>>,\n           KeyShort\n    end.\n\n%% @doc Transforms a key from a KV-tuple into a compact binary representation\n%%      based on the given size.\n%% @see compress_kv_list/5.\n-spec trivial_compress_key({Key::?RT:key(), Version::client_version()},\n                   SigSize::signature_size()) -> KeyShort::non_neg_integer().\ntrivial_compress_key(KV, SigSize) ->\n    compress_key(element(1, KV), SigSize).\n\n%% @doc Creates a compressed version of a (key-)position list.\n%%      MaxPosBound represents an upper bound on the biggest value in the list;\n%%      when decoding, the same bound must be known!\n-spec compress_idx_list\n        (SortedIdxList::[non_neg_integer()], MaxPosBound::non_neg_integer(),\n         ResultIdx::[non_neg_integer()], LastPos::non_neg_integer(),\n         Max::non_neg_integer(), integrate_size)\n    -> CompressedIndices::bitstring();\n        (SortedIdxList::[non_neg_integer()], MaxPosBound::non_neg_integer(),\n         ResultIdx::[non_neg_integer()], LastPos::non_neg_integer(),\n         Max::non_neg_integer(), return_size)\n    -> {CompressedIndices::bitstring(), DeltaSigSizeBits::non_neg_integer(),\n        DeltaSigSize::non_neg_integer()}.\ncompress_idx_list([Pos | Rest], MaxPosBound, AccResult, LastPos, Max, SizeOption) ->\n    CurIdx0 = Pos - LastPos,\n    % need a positive value to encode:\n    CurIdx = if CurIdx0 >= 0 -> CurIdx0;\n                true -> Mod = MaxPosBound + 1,\n                        ((CurIdx0 rem Mod) + Mod) rem Mod\n             end,\n    compress_idx_list(Rest, MaxPosBound, [CurIdx | AccResult], Pos + 1,\n                      erlang:max(CurIdx, Max), SizeOption);\ncompress_idx_list([], MaxPosBound, AccResult, _LastPos, Max, SizeOption) ->\n    IdxSize = if Max =:= 0 -> 1;\n                 true      -> bits_for_number(Max)\n              end,\n    Bin = lists:foldr(fun(Pos, Acc) ->\n                              <<Acc/bitstring, Pos:IdxSize/integer-unit:1>>\n                      end, <<>>, AccResult),\n    if Bin =/= <<>> andalso SizeOption =:= integrate_size ->\n           IdxBitsSize = bits_for_number(bits_for_number(MaxPosBound)),\n           <<IdxSize:IdxBitsSize/integer-unit:1, Bin/bitstring>>;\n       Bin =/= <<>> andalso SizeOption =:= return_size ->\n           IdxBitsSize = bits_for_number(bits_for_number(MaxPosBound)),\n           {Bin, IdxBitsSize, IdxSize};\n       Bin =:= <<>> andalso SizeOption =:= integrate_size ->\n           Bin;\n       Bin =:= <<>> andalso SizeOption =:= return_size ->\n           {Bin, 0, 0}\n    end.\n\n%% @doc De-compresses a bitstring with indices from compress_idx_list/5\n%%      into a list of indices encoded by that function.\n-spec decompress_idx_list(CompressedBin::bitstring(),\n                          DeltaSigSize::signature_size() | unknown,\n                          MaxPosBound::non_neg_integer())\n        -> {[non_neg_integer()], Count::non_neg_integer()}.\ndecompress_idx_list(<<>>, _, _MaxPosBound) ->\n    {[], 0};\ndecompress_idx_list(Bin, unknown, MaxPosBound) ->\n    IdxBitsSize = bits_for_number(bits_for_number(MaxPosBound)),\n    <<SigSize0:IdxBitsSize/integer-unit:1, Bin2/bitstring>> = Bin,\n    decompress_idx_list(Bin2, SigSize0, MaxPosBound);\ndecompress_idx_list(Bin, DeltaSigSize, MaxPosBound) ->\n    SigSize = erlang:max(1, DeltaSigSize),\n    Count = calc_items_in_chunk(Bin, SigSize),\n    IdxList = decompress_idx_list_(Bin, 0, SigSize, MaxPosBound + 1),\n    ?DBG_ASSERT(Count =:= length(IdxList)),\n    {IdxList, Count}.\n\n%% @doc Helper for decompress_idx_list/3.\n-spec decompress_idx_list_(CompressedBin::bitstring(), LastPos::non_neg_integer(),\n                           SigSize::signature_size(), Mod::pos_integer())\n        -> ResKeys::[non_neg_integer()].\ndecompress_idx_list_(<<>>, _LastPos, _SigSize, _Mod) ->\n    [];\ndecompress_idx_list_(Bin, LastPos, SigSize, Mod) ->\n    <<Diff:SigSize/integer-unit:1, T/bitstring>> = Bin,\n    CurPos = (LastPos + Diff) rem Mod,\n    [CurPos | decompress_idx_list_(T, CurPos + 1, SigSize, Mod)].\n\n%% @doc De-compresses a bitstring with indices from compress_idx_list/5\n%%      into the encoded sub-list of the original list.\n%%      NOTE: in contrast to decompress_idx_list/3 (which is used for\n%%            decompressing KV lists as well), we do not support duplicates\n%%            in the original list fed into compress_idx_list/5 and will fail\n%%            during the decode!\n-spec decompress_idx_to_list(CompressedBin::bitstring(), [X]) -> [X].\ndecompress_idx_to_list(<<>>, _List) ->\n    [];\ndecompress_idx_to_list(Bin, List) ->\n    IdxBitsSize = bits_for_number(bits_for_number(length(List) - 1)),\n    <<SigSize0:IdxBitsSize/integer-unit:1, Bin2/bitstring>> = Bin,\n    SigSize = erlang:max(1, SigSize0),\n    decompress_idx_to_list_(Bin2, List, SigSize).\n\n%% @doc Helper for decompress_idx_to_list/2.\n-spec decompress_idx_to_list_(CompressedBin::bitstring(), [X],\n                              SigSize::signature_size()) -> [X].\ndecompress_idx_to_list_(<<>>, _, _SigSize) ->\n    [];\ndecompress_idx_to_list_(Bin, List, SigSize) ->\n    <<KeyPosInc:SigSize/integer-unit:1, T/bitstring>> = Bin,\n    % note: this fails if there have been duplicates in the original list or\n    %       KeyPosInc was negative!\n    [X | List2] = lists:nthtail(KeyPosInc, List),\n    [X | decompress_idx_to_list_(T, List2, SigSize)].\n\n%% @doc Converts a list of positions to a bitstring where the x'th bit is set\n%%      if the x'th position is in the list. The final bitstring may be\n%%      created with erlang:list_to_bitstring(lists:reverse(Result)).\n%%      A total of FinalSize bits will be used.\n%%      PreCond: sorted list Pos, 0 &lt;= every pos &lt; FinalSize\n-spec pos_to_bitstring(Pos::[non_neg_integer()], AccBin::[bitstring()],\n                       BitsSet::non_neg_integer(), FinalSize::non_neg_integer())\n        -> Result::[bitstring()].\npos_to_bitstring([Pos | Rest], AccBin, BitsSet, FinalSize) ->\n    New = <<0:(Pos - BitsSet), 1:1>>,\n    pos_to_bitstring(Rest, [New | AccBin], Pos + 1, FinalSize);\npos_to_bitstring([], AccBin, BitsSet, FinalSize) ->\n    [<<0:(FinalSize - BitsSet)>> | AccBin].\n\n%% @doc Converts the bitstring from pos_to_bitstring/4 into keys at the\n%%      appropriate positions in KList. Result is reversly sorted.\n%%      NOTE: This is essentially the same as bitstring_to_k_list_kv/3 but we\n%%            need the separation because of the opaque RT keys.\n%% @see pos_to_bitstring/4\n-spec bitstring_to_k_list_k(PosBitString::bitstring(), KList::[?RT:key()],\n                            Acc::[?RT:key()]) -> Result::[?RT:key()].\nbitstring_to_k_list_k(<<1:1, RestBits/bitstring>>, [Key | RestK], Acc) ->\n    bitstring_to_k_list_k(RestBits, RestK, [Key | Acc]);\nbitstring_to_k_list_k(<<0:1, RestBits/bitstring>>, [_Key | RestK], Acc) ->\n    bitstring_to_k_list_k(RestBits, RestK, Acc);\nbitstring_to_k_list_k(<<>>, _KList, Acc) ->\n    Acc; % last 0 bits may  be truncated, e.g. by setting FinalSize in pos_to_bitstring/4 accordingly\nbitstring_to_k_list_k(RestBits, [], Acc) ->\n    % there may be rest bits, but all should be 0:\n    BitCount = erlang:bit_size(RestBits),\n    ?ASSERT(<<0:BitCount>> =:= RestBits),\n    Acc.\n\n%% @doc Converts the bitstring from pos_to_bitstring/4 into keys at the\n%%      appropriate positions in KVList. Result is reversly sorted.\n%% @see pos_to_bitstring/4\n-spec bitstring_to_k_list_kv(PosBitString::bitstring(), KVList::db_chunk_kv(),\n                             Acc::[?RT:key()]) -> Result::[?RT:key()].\nbitstring_to_k_list_kv(<<1:1, RestBits/bitstring>>, [{Key, _Version} | RestKV], Acc) ->\n    bitstring_to_k_list_kv(RestBits, RestKV, [Key | Acc]);\nbitstring_to_k_list_kv(<<0:1, RestBits/bitstring>>, [{_Key, _Version} | RestKV], Acc) ->\n    bitstring_to_k_list_kv(RestBits, RestKV, Acc);\nbitstring_to_k_list_kv(<<>>, _KVList, Acc) ->\n    Acc; % last 0 bits may  be truncated, e.g. by setting FinalSize in pos_to_bitstring/4 accordingly\nbitstring_to_k_list_kv(RestBits, [], Acc) ->\n    % there may be rest bits, but all should be 0:\n    BitCount = erlang:bit_size(RestBits),\n    ?ASSERT(<<0:BitCount>> =:= RestBits),\n    Acc.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% SHash specific\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Gets all entries from MyEntries which are not encoded in MyIOtKvSet.\n%%      Also returns the tree with all these matches removed.\n-spec shash_get_full_diff(KVList::[{HashedKey::non_neg_integer(),\n                                    {Key::?RT:key(), Version::client_version()}}],\n                          MyIOtherKvTree::kvi_tree(),\n                          AccDiff::KVList, SigSize::signature_size())\n        -> {Diff::KVList, MyIOtherKvTree::kvi_tree()}\n    when is_subtype(KVList, db_chunk_kv()).\nshash_get_full_diff([], MyIOtKvSet, AccDiff, _SigSize) ->\n    {AccDiff, MyIOtKvSet};\nshash_get_full_diff([{CurKey, KV} | Rest], MyIOtKvSet, AccDiff, SigSize) ->\n    OldSize = mymaps:size(MyIOtKvSet),\n    MyIOtKvSet2 = mymaps:remove(CurKey, MyIOtKvSet),\n    case mymaps:size(MyIOtKvSet2) of\n        OldSize ->\n            shash_get_full_diff(Rest, MyIOtKvSet2, [KV | AccDiff], SigSize);\n        _ ->\n            shash_get_full_diff(Rest, MyIOtKvSet2, AccDiff, SigSize)\n    end.\n\n%% @doc Part of the resolve_req message processing of the SHash, Bloom, ART RC\n%%      processes in phase 2 (trivial RC) at the initiator for Bloom and at\n%%      the non-initiator for the other protocols.\n-spec shash_bloom_perform_resolve(\n        State::state(), DBChunkTree::kvi_tree(), DupesCount::non_neg_integer(),\n        SigSize::signature_size(), VSize::signature_size(),\n        DestReconPid::comm:mypid(),\n        GetDiffFun::fun((MyEntries::[{HashedKey::non_neg_integer(), Key::?RT:key(), VShort::integer()}],\n                         MyIOtherKvTree::kvi_tree(),\n                         AccFBItems::[?RT:key()], AccReqItems::[non_neg_integer()])\n                       -> {FBItems::[?RT:key()], ReqItemsIdx::[non_neg_integer()],\n                           MyIOtherKvTree::kvi_tree()}))\n        -> rr_recon_stats:stats().\nshash_bloom_perform_resolve(\n  #rr_recon_state{dest_rr_pid = DestRRPid,   ownerPid = OwnerL,\n                  kv_list = KVList,          stats = Stats,\n                  method = _RMethod,         initiator = IsInitiator},\n  DBChunkTree, DupesCount, SigSize, VSize, DestReconPid, GetDiffFun) ->\n    {ToSendKeys1, ToReqIdx1, DBChunkTree1, _Dupes} =\n        get_diff_with_dupes(KVList, DBChunkTree, [], [], SigSize, VSize,\n                            GetDiffFun),\n    ?ALG_DEBUG(\"CheckCKV ~B+~Bckv vs. ~B+~Bcmp items\",\n               [mymaps:size(DBChunkTree), DupesCount,\n                length(KVList) - length(_Dupes), length(_Dupes)]),\n\n    NewStats1 = send_resolve_request(Stats, ToSendKeys1, OwnerL, DestRRPid,\n                                     IsInitiator, false),\n\n    % let the initiator's rr_recon process identify the remaining keys\n    ReqIdx = lists:usort([Idx || {_Version, Idx} <- mymaps:values(DBChunkTree1)] ++ ToReqIdx1),\n    ToReq2 = erlang:list_to_bitstring(\n               lists:reverse(\n                 pos_to_bitstring(% note: ReqIdx positions start with 0\n                   ReqIdx, [], 0, ?IIF(ReqIdx =:= [], 0, lists:last(ReqIdx) + 1)))),\n    ?ALG_DEBUG(\"resolve_req ~s~n  Session=~.0p ; ToReq= ~p bytes\",\n               [_RMethod, rr_recon_stats:get(session_id, NewStats1), erlang:byte_size(ToReq2)]),\n    comm:send(DestReconPid, {resolve_req, ToReq2}),\n    % the other node will use key_upd_send and we must thus increase\n    % the number of resolve processes here!\n    if ReqIdx =/= [] orelse DupesCount > 0 ->\n           rr_recon_stats:inc([{rs_expected, 1}], NewStats1);\n       true -> NewStats1\n    end.\n\n%% @doc Sets up a phase2 trivial synchronisation on the identified differences\n%%      where the mapping of the differences is not clear yet and only the\n%%      current node knows them.\n%%      NOTE: Payload is only sent if the other node has not shutdown yet!\n-spec phase2_run_trivial_on_diff(\n  UnidentifiedDiff::db_chunk_kv(), Payload::any(), OtherHasShutdown::boolean(),\n  FR_p2::float(), OtherCmpItemCount::non_neg_integer(), State::state())\n        -> NewState::state().\nphase2_run_trivial_on_diff(\n  UnidentifiedDiff, Payload, OtherHasShutdown, FR_p2,\n  OtherCmpItemCount, % number of items the other nodes compares CKV entries with\n  State = #rr_recon_state{stats = Stats, dest_recon_pid = DestReconPid,\n                          initiator = IsInitiator,\n                          dest_rr_pid = DestRRPid, ownerPid = OwnerL}) ->\n    CKVItems = UnidentifiedDiff,\n    CKVSize = length(CKVItems),\n    StartResolve = CKVSize + OtherCmpItemCount > 0,\n    ?ALG_DEBUG(\"Reconcile SHash/Bloom/ART~n  Session=~.0p ; Phase2=~B vs. ~B\",\n               [rr_recon_stats:get(session_id, Stats), CKVSize, OtherCmpItemCount]),\n    if StartResolve andalso not OtherHasShutdown ->\n           % send idx of non-matching other items & KV-List of my diff items\n           % start resolve similar to a trivial recon but using the full diff!\n           % (as if non-initiator in trivial recon)\n           ExpDelta = 100, % the sets at I and NI may be distinct (in the worst-case)\n           {BuildTime, {MyDiffK, MyDiffV, ResortedKVOrigList, Dupes, SigSizeT, _VSizeT}} =\n               util:tc(fun() ->\n                               compress_kv_list_fr(\n                                 CKVItems, OtherCmpItemCount, CKVSize, ExpDelta, FR_p2,\n                                 fun trivial_signature_sizes/4, fun trivial_compress_key/2)\n                       end),\n           Fr_p2_real = trivial_worst_case_failrate(\n                          SigSizeT, OtherCmpItemCount, CKVSize, ExpDelta),\n\n           send(DestReconPid,\n                {resolve_req, {MyDiffK, MyDiffV}, _DupesCount = length(Dupes),\n                 Payload, SigSizeT, comm:this()}),\n           % the other node will use key_upd_send and we must thus increase\n           % the number of resolve processes here!\n           NewStats1 = rr_recon_stats:inc([{rs_expected, 1},\n                                           {build_time, BuildTime}], Stats),\n           NewStats  = rr_recon_stats:set([{fail_rate_p2, Fr_p2_real}], NewStats1),\n           KList = [element(1, KV) || KV <- ResortedKVOrigList],\n           State#rr_recon_state{stats = NewStats, stage = resolve,\n                                kv_list = [], k_list = KList,\n                                next_phase_kv = Dupes, misc = []};\n       OtherHasShutdown ->\n           % no need to send resolve_req message - the non-initiator already shut down\n           % the other node does not have any items but there may be a diff at our node!\n           % start a resolve here:\n           KList = [element(1, KV) || KV <- CKVItems],\n           NewStats = send_resolve_request(\n                        Stats, KList, OwnerL, DestRRPid, IsInitiator, true),\n           NewState = State#rr_recon_state{stats = NewStats, stage = resolve},\n           shutdown(sync_finished, NewState);\n       true -> % not OtherHasShutdown, CKVSize =:= 0\n           % must send resolve_req message for the non-initiator to shut down\n           send(DestReconPid, {resolve_req, shutdown, Payload}),\n           shutdown(sync_finished, State)\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Merkle Tree specific\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Calculates from a total failure rate target FR the (next) failure rate\n%%      targets to use for signature and sub-tree reconciliations.\n-spec merkle_next_fr_targets(BranchFactor::pos_integer(), FRPerNode::float())\n    -> {FR_I::float(), FR_L::float()}.\nmerkle_next_fr_targets(BranchFactor, FRPerNode) ->\n    % mistakes caused by:\n    % inner node: current node or any of its BranchFactor children (B=BranchFactor+1)\n    % leaf node: current node only (B=1) and thus FRPerNode\n    FR_I = calc_n_subparts_FR(BranchFactor + 1, FRPerNode),\n    FR_L = FRPerNode,\n%%     log:pal(\"merkle [ ~p ]~n  FRPerNode: ~p, \\tFR_I: ~p, \\tFR_L: ~p\",\n%%             [self(), FRPerNode, FR_I, FR_L]),\n    {FR_I, FR_L}.\n\n%% @doc Calculates the new signature sizes based on the next failure targets\n%%      as in merkle_next_fr_targets/2\n-spec merkle_next_signature_sizes(\n        Params::#merkle_params{}, FRPerNode::float(),\n        MyMaxItemsCount::non_neg_integer(), OtherMaxItemsCount::non_neg_integer(),\n        MaxAffectedItems::non_neg_integer())\n    -> {FR_I::float(), FR_L::float(),\n        NextSigSizeI::signature_size(), NextSigSizeL::signature_size(),\n        EffectiveFr_I::float(), EffectiveFr_L::float()}.\nmerkle_next_signature_sizes(\n  #merkle_params{bucket_size = BucketSize, branch_factor = BranchFactor},\n  FRPerNode, MyMaxItemsCount, OtherMaxItemsCount, MaxAffectedItems) ->\n    {FR_I, FR_L} = merkle_next_fr_targets(BranchFactor, FRPerNode),\n\n    % note: we need to use the same failure rate for this level's signature\n    %       comparison as a children's tree has in total!\n    if MaxAffectedItems =:= 0 ->\n           NextSigSizeL = NextSigSizeI = get_min_hash_bits(),\n           _AffectedItemsI = _AffectedItemsL = 0,\n           EffectiveFr_L = EffectiveFr_I = float(0 / util:pow(2, NextSigSizeI));\n       MyMaxItemsCount =/= 0 andalso OtherMaxItemsCount =/= 0 ->\n           _AffectedItemsI =\n               AffectedItemsI = erlang:min(MyMaxItemsCount + OtherMaxItemsCount,\n                                           MaxAffectedItems),\n           NextSigSizeI = min_max(util:ceil(util:log2(AffectedItemsI / FR_I)),\n                                  get_min_hash_bits(), 160),\n           EffectiveFr_I = float(AffectedItemsI / util:pow(2, NextSigSizeI)),\n           _AffectedItemsL =\n               AffectedItemsL = lists:min([MyMaxItemsCount + OtherMaxItemsCount,\n                                           2 * BucketSize, MaxAffectedItems]),\n           NextSigSizeL = min_max(util:ceil(util:log2(AffectedItemsL / FR_L)),\n                                  get_min_hash_bits(), 160),\n           EffectiveFr_L = float(AffectedItemsL / util:pow(2, NextSigSizeL));\n       true ->\n           % should only occur during the reconciliation initiation in begin_sync/1\n           NextSigSizeL = NextSigSizeI = 0,\n           _AffectedItemsI = _AffectedItemsL = 0,\n           EffectiveFr_L = EffectiveFr_I = 0.0\n    end,\n\n    ?ALG_DEBUG(\"merkle - signatures~n  MyMI: ~B,\\tOtMI: ~B,\\tMaxAffected: ~B,\\tMaxAffectedI: ~B,\\tMaxAffectedL: ~B~n\"\n               \"  FR_I: ~g,\\tFR_L: ~g,\\tSigSizeI: ~B,\\tSigSizeL: ~B~n\"\n               \"  -> eff. FR_I: ~g,\\teff. FR_L: ~g\",\n               [MyMaxItemsCount, OtherMaxItemsCount, MaxAffectedItems, _AffectedItemsI, _AffectedItemsL,\n                FR_I, FR_L, NextSigSizeI, NextSigSizeL,\n                EffectiveFr_I, EffectiveFr_L]),\n    {FR_I, FR_L, NextSigSizeI, NextSigSizeL, EffectiveFr_I, EffectiveFr_L}.\n\n-compile({nowarn_unused_function, {min_max_feeder, 3}}).\n-spec min_max_feeder(X::number(), Min::number(), Max::number())\n        -> {X::number(), Min::number(), Max::number()}.\nmin_max_feeder(X, Min, Max) when Min > Max -> {X, Max, Min};\nmin_max_feeder(X, Min, Max) -> {X, Min, Max}.\n\n%% @doc Sets min and max boundaries for X and returns either Min, Max or X.\n-spec min_max(X::number(), Min::number(), Max::number()) -> number().\nmin_max(X, _Min, Max) when X >= Max ->\n    ?DBG_ASSERT(_Min =< Max orelse _Min =:= get_min_hash_bits()),\n    Max;\nmin_max(X, Min, _Max) when X =< Min ->\n    ?DBG_ASSERT(Min =< _Max orelse Min =:= get_min_hash_bits()),\n    Min;\nmin_max(X, _Min, _Max) ->\n    % dbg_assert must be true:\n    %?DBG_ASSERT(_Min =< _Max orelse _Min =:= get_min_hash_bits()),\n    X.\n\n%% @doc Transforms a list of merkle keys, i.e. hashes, into a compact binary\n%%      representation for transfer.\n-spec merkle_compress_hashlist(Nodes::[merkle_tree:mt_node()], Bin,\n                               SigSizeI::signature_size(),\n                               SigSizeL::signature_size()) -> Bin\n    when is_subtype(Bin, bitstring()).\nmerkle_compress_hashlist([], Bin, _SigSizeI, _SigSizeL) ->\n    Bin;\nmerkle_compress_hashlist([N1 | TL], Bin, SigSizeI, SigSizeL) ->\n    H1 = merkle_tree:get_hash(N1),\n    case merkle_tree:is_leaf(N1) of\n        true ->\n            Bin2 = case merkle_tree:is_empty(N1) of\n                       true  -> <<Bin/bitstring, 1:1, 0:1>>;\n                       false -> <<Bin/bitstring, 1:1, 1:1, H1:SigSizeL>>\n                   end,\n            merkle_compress_hashlist(TL, Bin2, SigSizeI, SigSizeL);\n        false ->\n            merkle_compress_hashlist(TL, <<Bin/bitstring, 0:1, H1:SigSizeI>>,\n                                     SigSizeI, SigSizeL)\n    end.\n\n%% @doc Transforms the compact binary representation of merkle hash lists from\n%%      merkle_compress_hashlist/2 back into the original form.\n-spec merkle_decompress_hashlist(bitstring(), SigSizeI::signature_size(),\n                                 SigSizeL::signature_size())\n        -> Hashes::[merkle_cmp_request()].\nmerkle_decompress_hashlist(<<>>, _SigSizeI, _SigSizeL) ->\n    [];\nmerkle_decompress_hashlist(Bin, SigSizeI, SigSizeL) ->\n    IsLeaf = case Bin of\n                 <<1:1, 1:1, Hash:SigSizeL/integer-unit:1, Bin2/bitstring>> ->\n                     true;\n                 <<1:1, 0:1, Bin2/bitstring>> ->\n                     Hash = none,\n                     true;\n                 <<0:1, Hash:SigSizeI/integer-unit:1, Bin2/bitstring>> ->\n                     false\n             end,\n    [{Hash, IsLeaf} | merkle_decompress_hashlist(Bin2, SigSizeI, SigSizeL)].\n\n%% @doc Compares the given Hashes from the other node with my merkle_tree nodes\n%%      (executed on non-initiator).\n%%      Returns the comparison results and the rest nodes to check in a next\n%%      step.\n-spec merkle_check_node(\n        Hashes::[merkle_cmp_request()], MyNodes::NodeList,\n        SigSizeI::signature_size(), SigSizeL::signature_size(),\n        MyMaxItemsCount::non_neg_integer(), OtherMaxItemsCount::non_neg_integer(),\n        #merkle_params{}, Stats, FlagsAcc::bitstring(), RestTreeAcc::NodeList,\n        SyncAccSend::[merkle_sync_send()], SyncAccRcv::[merkle_sync_rcv()],\n        SyncAccRcvLeafCount::Count,\n        MySyncAccDRK::[?RT:key()], MySyncAccDRLCount::Count, OtherSyncAccDRLCount::Count,\n        SyncIn::merkle_sync(), AccCmp::Count, AccSkip::Count,\n        NextLvlNodesActIN::Count, HashCmpI_IN::Count, HashCmpL_IN::Count)\n        -> {FlagsOUT::bitstring(), RestTreeOut::NodeList,\n            SyncOUT::merkle_sync(), Stats, MaxItemsCount::Count,\n            NextLvlNodesActOUT::Count, HashCmpI_OUT::Count, HashCmpL_OUT::Count}\n    when\n      is_subtype(NodeList, [merkle_tree:mt_node()]),\n      is_subtype(Stats,    rr_recon_stats:stats()),\n      is_subtype(Count,    non_neg_integer()).\nmerkle_check_node([], [], _SigSizeI, _SigSizeL,\n                  _MyMaxItemsCount, _OtherMaxItemsCount, _Params, Stats, FlagsAcc, RestTreeAcc,\n                  SyncAccSend, SyncAccRcv, SyncAccRcvLeafCount,\n                  MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount,\n                  {SyncInSend, SyncInRcv, SyncInRcvLeafCount,\n                   {MySyncInDRK, MySyncInDRLCount, OtherSyncInDRLCount}},\n                  AccCmp, AccSkip, NextLvlNodesActIN,\n                  HashCmpI_IN, HashCmpL_IN) ->\n    NStats = rr_recon_stats:inc([{tree_nodesCompared, AccCmp},\n                                 {tree_compareSkipped, AccSkip}], Stats),\n    % note: we can safely include all leaf nodes here although only inner nodes\n    %       should go into MIC - every inner node always has more items than\n    %       any leaf node (otherwise it would have been a leaf node)\n    AccMIC = lists:max([0 | [merkle_tree:get_item_count(Node) || Node <- RestTreeAcc]]),\n    ?DBG_ASSERT(NextLvlNodesActIN =:= length(RestTreeAcc)),\n    {FlagsAcc, lists:reverse(RestTreeAcc),\n     {lists:reverse(SyncAccSend, SyncInSend),\n      lists:reverse(SyncAccRcv, SyncInRcv),\n      SyncInRcvLeafCount + SyncAccRcvLeafCount,\n      {MySyncAccDRK ++ MySyncInDRK, MySyncInDRLCount + MySyncAccDRLCount,\n       OtherSyncInDRLCount + OtherSyncAccDRLCount}},\n     NStats, AccMIC, NextLvlNodesActIN, HashCmpI_IN, HashCmpL_IN};\nmerkle_check_node([{Hash, IsLeafHash} | TK], [Node | TN], SigSizeI, SigSizeL,\n                  MyMaxItemsCount, OtherMaxItemsCount, Params, Stats, FlagsAcc, RestTreeAcc,\n                  SyncAccSend, SyncAccRcv, SyncAccRcvLeafCount,\n                  MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount,\n                  SyncIN, AccCmp, AccSkip, NextLvlNodesActIN,\n                  HashCmpI_IN, HashCmpL_IN) ->\n    IsLeafNode = merkle_tree:is_leaf(Node),\n    EmptyNode = merkle_tree:is_empty(Node),\n    EmptyLeafNode = IsLeafNode andalso EmptyNode,\n    NonEmptyLeafNode = IsLeafNode andalso (not EmptyNode),\n    NodeHash =\n        if EmptyLeafNode ->\n               none; % to match with the hash from merkle_decompress_hashlist/3\n           IsLeafNode ->\n               <<X:SigSizeL/integer-unit:1>> = <<(merkle_tree:get_hash(Node)):SigSizeL>>,\n               X;\n           true ->\n               <<X:SigSizeI/integer-unit:1>> = <<(merkle_tree:get_hash(Node)):SigSizeI>>,\n               X\n        end,\n    EmptyLeafHash = IsLeafHash andalso Hash =:= none,\n    NonEmptyLeafHash = IsLeafHash andalso Hash =/= none,\n    % note that only hash matches(!) count for HashCmpI_OUT and HashCmpL_OUT\n    % since mismatches are always correct (the items must differ!)\n    % -> don't count mismatches since these values ebter the failure probability\n    if Hash =:= NodeHash andalso IsLeafHash =:= IsLeafNode ->\n           Skipped = merkle_tree:size(Node) - 1,\n           if EmptyLeafHash -> % empty leaf hash on both nodes - this was exact!\n                  HashCmpI_OUT = HashCmpI_IN,\n                  HashCmpL_OUT = HashCmpL_IN;\n              IsLeafHash -> % both non-empty leaf nodes\n                  HashCmpI_OUT = HashCmpI_IN,\n                  HashCmpL_OUT = HashCmpL_IN + 1;\n              true -> % both inner nodes\n                  HashCmpI_OUT = HashCmpI_IN + 1,\n                  HashCmpL_OUT = HashCmpL_IN\n           end,\n           merkle_check_node(TK, TN, SigSizeI, SigSizeL,\n                             MyMaxItemsCount, OtherMaxItemsCount, Params, Stats,\n                             <<FlagsAcc/bitstring, ?recon_ok:2>>, RestTreeAcc,\n                             SyncAccSend, SyncAccRcv, SyncAccRcvLeafCount,\n                             MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount,\n                             SyncIN, AccCmp + 1, AccSkip + Skipped,\n                             NextLvlNodesActIN, HashCmpI_OUT, HashCmpL_OUT);\n       (not IsLeafNode) andalso (not IsLeafHash) ->\n           % both inner nodes\n           Childs = merkle_tree:get_childs(Node),\n           NextLvlNodesActOUT = NextLvlNodesActIN + Params#merkle_params.branch_factor,\n           merkle_check_node(TK, TN, SigSizeI, SigSizeL,\n                             MyMaxItemsCount, OtherMaxItemsCount, Params, Stats,\n                             <<FlagsAcc/bitstring, ?recon_fail_cont_inner:2>>, lists:reverse(Childs, RestTreeAcc),\n                             SyncAccSend, SyncAccRcv, SyncAccRcvLeafCount,\n                             MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount,\n                             SyncIN, AccCmp + 1, AccSkip,\n                             NextLvlNodesActOUT, HashCmpI_IN, HashCmpL_IN);\n       (not IsLeafNode) andalso NonEmptyLeafHash ->\n           % inner node here, non-empty leaf there\n           % no need to compare hashes - this is an exact process based on the tags\n           {MyKVItems, LeafCount} = merkle_tree:get_items(Node),\n           Sync = {MyMaxItemsCount, MyKVItems},\n           merkle_check_node(TK, TN, SigSizeI, SigSizeL,\n                             MyMaxItemsCount, OtherMaxItemsCount, Params, Stats,\n                             <<FlagsAcc/bitstring, ?recon_fail_stop_inner:2>>, RestTreeAcc,\n                             SyncAccSend, [Sync | SyncAccRcv], SyncAccRcvLeafCount + LeafCount,\n                             MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount,\n                             SyncIN, AccCmp + 1, AccSkip,\n                             NextLvlNodesActIN, HashCmpI_IN, HashCmpL_IN);\n       NonEmptyLeafNode andalso (NonEmptyLeafHash orelse not IsLeafHash) ->\n           % non-empty leaf here, non-empty leaf or inner node there\n           OtherMaxItemsCount1 =\n               if NonEmptyLeafHash -> % both non-empty leaf nodes\n                      erlang:min(Params#merkle_params.bucket_size,\n                                 OtherMaxItemsCount);\n                  true -> % inner node there\n                      OtherMaxItemsCount\n               end,\n           SyncAccSend1 =\n               [{OtherMaxItemsCount1, merkle_tree:get_bucket(Node)} | SyncAccSend],\n           merkle_check_node(TK, TN, SigSizeI, SigSizeL,\n                             MyMaxItemsCount, OtherMaxItemsCount, Params, Stats,\n                             <<FlagsAcc/bitstring, ?recon_fail_stop_leaf:2>>, RestTreeAcc,\n                             SyncAccSend1, SyncAccRcv, SyncAccRcvLeafCount,\n                             MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount,\n                             SyncIN, AccCmp + 1, AccSkip,\n                             NextLvlNodesActIN, HashCmpI_IN, HashCmpL_IN);\n       (NonEmptyLeafNode orelse not IsLeafNode) andalso EmptyLeafHash ->\n           % non-empty leaf or inner node here, empty leaf there\n           % no need to compare hashes - this is an exact process based on the tags\n           % -> resolve directly here, i.e. without a trivial sub process\n           ResultCode = if not IsLeafNode -> ?recon_fail_stop_inner; % stop_empty_leaf1\n                           NonEmptyLeafNode -> ?recon_fail_stop_leaf % stop_empty_leaf2\n                        end,\n           {MyKVItems, LeafCount} = merkle_tree:get_items(Node),\n           MySyncAccDRK1 = [element(1, X) || X <- MyKVItems] ++ MySyncAccDRK,\n           merkle_check_node(TK, TN, SigSizeI, SigSizeL,\n                             MyMaxItemsCount, OtherMaxItemsCount, Params, Stats,\n                             <<FlagsAcc/bitstring, ResultCode:2>>, RestTreeAcc,\n                             SyncAccSend, SyncAccRcv, SyncAccRcvLeafCount,\n                             MySyncAccDRK1, MySyncAccDRLCount + LeafCount,\n                             OtherSyncAccDRLCount,\n                             SyncIN, AccCmp + 1, AccSkip,\n                             NextLvlNodesActIN, HashCmpI_IN, HashCmpL_IN);\n       EmptyLeafNode andalso (NonEmptyLeafHash orelse not IsLeafHash) ->\n           % empty leaf here, non-empty leaf or inner node there\n           % no need to compare hashes - this is an exact process based on the tags\n           % -> resolved directly at the other node, i.e. without a trivial sub process\n           ResultCode = if not IsLeafHash -> ?recon_fail_stop_inner; % stop_empty_leaf3\n                           NonEmptyLeafHash -> ?recon_fail_cont_inner % stop_empty_leaf4\n                        end,\n           merkle_check_node(TK, TN, SigSizeI, SigSizeL,\n                             MyMaxItemsCount, OtherMaxItemsCount, Params, Stats,\n                             <<FlagsAcc/bitstring, ResultCode:2>>, RestTreeAcc,\n                             SyncAccSend, SyncAccRcv, SyncAccRcvLeafCount,\n                             MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount + 1,\n                             SyncIN, AccCmp + 1, AccSkip,\n                             NextLvlNodesActIN, HashCmpI_IN, HashCmpL_IN)\n    end.\n\n%% @doc Processes compare results from merkle_check_node/22 on the initiator.\n%       Note that only hash matches(!) count for HashCmpI_OUT and HashCmpL_OUT\n%       since mismatches are always correct (the items must differ!)\n%       -> don't count mismatches since these values ebter the failure probability\n-spec merkle_cmp_result(\n        bitstring(), RestTree::NodeList,\n        SigSizeI::signature_size(), SigSizeL::signature_size(),\n        MyMaxItemsCount::non_neg_integer(), OtherMaxItemsCount::non_neg_integer(),\n        SyncIn::merkle_sync(), #merkle_params{}, Stats, RestTreeAcc::NodeList,\n        SyncAccSend::[merkle_sync_send()], SyncAccRcv::[merkle_sync_rcv()],\n        SyncAccRcvLeafCount::Count,\n        MySyncAccDRK::[?RT:key()], MySyncAccDRLCount::Count, OtherSyncAccDRLCount::Count,\n        AccCmp::Count, AccSkip::Count,\n        NextLvlNodesActIN::Count, HashCmpI_IN::Count, HashCmpL_IN::Count)\n        -> {RestTreeOut::NodeList, MerkleSyncOut::merkle_sync(),\n            NewStats::Stats, MaxItemsCount::Count,\n            NextLvlNodesActOUT::Count, HashCmpI_OUT::Count, HashCmpL_OUT::Count}\n    when\n      is_subtype(NodeList, [merkle_tree:mt_node()]),\n      is_subtype(Stats,    rr_recon_stats:stats()),\n      is_subtype(Count,    non_neg_integer()).\nmerkle_cmp_result(<<>>, [], _SigSizeI, _SigSizeL,\n                  _MyMaxItemsCount, _OtherMaxItemsCount,\n                  {SyncInSend, SyncInRcv, SyncInRcvLeafCount,\n                   {MySyncInDRK, MySyncInDRLCount, OtherSyncInDRLCount}},\n                  _Params, Stats,\n                  RestTreeAcc, SyncAccSend, SyncAccRcv, SyncAccRcvLeafCount,\n                  MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount, AccCmp, AccSkip,\n                  NextLvlNodesActIN, HashCmpI_IN, HashCmpL_IN) ->\n    NStats = rr_recon_stats:inc([{tree_nodesCompared, AccCmp},\n                                 {tree_compareSkipped, AccSkip}], Stats),\n    % note: we can safely include all leaf nodes here although only inner nodes\n    %       should go into MIC - every inner node always has more items than\n    %       any leaf node (otherwise it would have been a leaf node)\n    AccMIC = lists:max([0 | [merkle_tree:get_item_count(Node) || Node <- RestTreeAcc]]),\n    ?DBG_ASSERT(NextLvlNodesActIN =:= length(RestTreeAcc)),\n    {lists:reverse(RestTreeAcc),\n     {lists:reverse(SyncAccSend, SyncInSend),\n      lists:reverse(SyncAccRcv, SyncInRcv),\n      SyncInRcvLeafCount + SyncAccRcvLeafCount,\n      {MySyncAccDRK ++ MySyncInDRK, MySyncInDRLCount + MySyncAccDRLCount,\n       OtherSyncInDRLCount + OtherSyncAccDRLCount}},\n     NStats, AccMIC, NextLvlNodesActIN, HashCmpI_IN, HashCmpL_IN};\nmerkle_cmp_result(<<?recon_ok:2, TR/bitstring>>, [Node | TN], SigSizeI, SigSizeL,\n                  MyMaxItemsCount, OtherMaxItemsCount, MerkleSyncIn, Params, Stats,\n                  RestTreeAcc, SyncAccSend, SyncAccRcv, SyncAccRcvLeafCount,\n                  MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount,\n                  AccCmp, AccSkip, NextLvlNodesActIN, HashCmpI_IN, HashCmpL_IN) ->\n    case merkle_tree:is_leaf(Node) of\n        true ->\n            case merkle_tree:is_empty(Node) of\n                true ->\n                    % empty leaf hash on both nodes - this was exact!\n                    HashCmpI_OUT = HashCmpI_IN,\n                    HashCmpL_OUT = HashCmpL_IN;\n                false ->\n                    HashCmpI_OUT = HashCmpI_IN,\n                    HashCmpL_OUT = HashCmpL_IN + 1\n            end;\n        false ->\n            HashCmpI_OUT = HashCmpI_IN + 1,\n            HashCmpL_OUT = HashCmpL_IN\n    end,\n    Skipped = merkle_tree:size(Node) - 1,\n    merkle_cmp_result(TR, TN, SigSizeI, SigSizeL,\n                      MyMaxItemsCount, OtherMaxItemsCount, MerkleSyncIn, Params, Stats,\n                      RestTreeAcc, SyncAccSend, SyncAccRcv, SyncAccRcvLeafCount,\n                      MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount,\n                      AccCmp + 1, AccSkip + Skipped,\n                      NextLvlNodesActIN, HashCmpI_OUT, HashCmpL_OUT);\nmerkle_cmp_result(<<?recon_fail_cont_inner:2, TR/bitstring>>, [Node | TN],\n                  SigSizeI, SigSizeL,\n                  MyMaxItemsCount, OtherMaxItemsCount, SyncIn, Params, Stats,\n                  RestTreeAcc, SyncAccSend, SyncAccRcv, SyncAccRcvLeafCount,\n                  MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount,\n                  AccCmp, AccSkip, NextLvlNodesActIN, HashCmpI_IN, HashCmpL_IN) ->\n    % either cont_inner or stop_empty_leaf4\n    case merkle_tree:is_leaf(Node) of\n        false -> % cont_inner\n            % inner hash on both nodes\n            Childs = merkle_tree:get_childs(Node),\n            NextLvlNodesActOUT = NextLvlNodesActIN + Params#merkle_params.branch_factor,\n            RestTreeAcc1 = lists:reverse(Childs, RestTreeAcc),\n            MySyncAccDRK1 = MySyncAccDRK,\n            MySyncAccDRLCount1 = MySyncAccDRLCount;\n        true -> % stop_empty_leaf4\n            ?DBG_ASSERT(not merkle_tree:is_empty(Node)),\n            % non-empty leaf on this node, empty leaf on the other node\n            % -> resolve directly here, i.e. without a trivial sub process\n            NextLvlNodesActOUT = NextLvlNodesActIN,\n            RestTreeAcc1 = RestTreeAcc,\n            MyKVItems = merkle_tree:get_bucket(Node),\n            MySyncAccDRK1 = [element(1, X) || X <- MyKVItems] ++ MySyncAccDRK,\n            MySyncAccDRLCount1 = MySyncAccDRLCount + 1\n    end,\n    merkle_cmp_result(TR, TN, SigSizeI, SigSizeL,\n                      MyMaxItemsCount, OtherMaxItemsCount, SyncIn, Params, Stats,\n                      RestTreeAcc1, SyncAccSend, SyncAccRcv, SyncAccRcvLeafCount,\n                      MySyncAccDRK1, MySyncAccDRLCount1, OtherSyncAccDRLCount,\n                      AccCmp + 1, AccSkip,\n                      NextLvlNodesActOUT, HashCmpI_IN, HashCmpL_IN);\nmerkle_cmp_result(<<?recon_fail_stop_inner:2, TR/bitstring>>, [Node | TN],\n                  SigSizeI, SigSizeL,\n                  MyMaxItemsCount, OtherMaxItemsCount, SyncIn, Params, Stats,\n                  RestTreeAcc, SyncAccSend, SyncAccRcv, SyncAccRcvLeafCount,\n                  MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount, AccCmp, AccSkip,\n                  NextLvlNodesActIN, HashCmpI_IN, HashCmpL_IN) ->\n    % either stop_inner or stop_empty_leaf1 or stop_empty_leaf3\n    % NOTE: all these mismatches are exact process based on the tags\n    IsLeafNode = merkle_tree:is_leaf(Node),\n    EmptyLeafNode = IsLeafNode andalso merkle_tree:is_empty(Node),\n    \n    if IsLeafNode andalso (not EmptyLeafNode) -> % stop_inner\n           SyncAccSend1 =\n               [{OtherMaxItemsCount, merkle_tree:get_bucket(Node)} | SyncAccSend],\n           OtherSyncAccDRLCount1 = OtherSyncAccDRLCount,\n           MySyncAccDRK1 = MySyncAccDRK,\n           MySyncAccDRLCount1 = MySyncAccDRLCount;\n       EmptyLeafNode -> % stop_empty_leaf1\n           % -> resolved directly at the other node, i.e. without a trivial sub process\n           SyncAccSend1 = SyncAccSend,\n           OtherSyncAccDRLCount1 = OtherSyncAccDRLCount + 1, % this will deviate from the other node!\n           MySyncAccDRK1 = MySyncAccDRK,\n           MySyncAccDRLCount1 = MySyncAccDRLCount;\n       not IsLeafNode -> % stop_empty_leaf3\n           % -> resolve directly here, i.e. without a trivial sub process\n           SyncAccSend1 = SyncAccSend,\n           OtherSyncAccDRLCount1 = OtherSyncAccDRLCount,\n           {MyKVItems, LeafCount} = merkle_tree:get_items(Node),\n           MySyncAccDRK1 = [element(1, X) || X <- MyKVItems] ++ MySyncAccDRK,\n           MySyncAccDRLCount1 = MySyncAccDRLCount + LeafCount\n    end,\n    merkle_cmp_result(TR, TN, SigSizeI, SigSizeL,\n                      MyMaxItemsCount, OtherMaxItemsCount, SyncIn, Params, Stats,\n                      RestTreeAcc, SyncAccSend1, SyncAccRcv, SyncAccRcvLeafCount,\n                      MySyncAccDRK1, MySyncAccDRLCount1, OtherSyncAccDRLCount1,\n                      AccCmp + 1, AccSkip,\n                      NextLvlNodesActIN, HashCmpI_IN, HashCmpL_IN);\nmerkle_cmp_result(<<?recon_fail_stop_leaf:2, TR/bitstring>>, [Node | TN],\n                  SigSizeI, SigSizeL,\n                  MyMaxItemsCount, OtherMaxItemsCount, SyncIn, Params, Stats,\n                  RestTreeAcc, SyncAccSend, SyncAccRcv, SyncAccRcvLeafCount,\n                  MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount, AccCmp, AccSkip,\n                  NextLvlNodesActIN, HashCmpI_IN, HashCmpL_IN) ->\n    % either stop_leaf or stop_empty_leaf2\n    case merkle_tree:is_leaf(Node) of\n        true  ->\n            case merkle_tree:is_empty(Node) of\n                false -> % stop_leaf\n                    MaxItemsCount = erlang:min(Params#merkle_params.bucket_size,\n                                               MyMaxItemsCount),\n                    SyncAccRcv1 =\n                        [{MaxItemsCount, merkle_tree:get_bucket(Node)} | SyncAccRcv],\n                    SyncAccRcvLeafCount1 = SyncAccRcvLeafCount + 1,\n                    OtherSyncAccDRLCount1 = OtherSyncAccDRLCount;\n                true -> % stop_empty_leaf2\n                    % -> resolved directly at the other node, i.e. without a trivial sub process\n                    SyncAccRcv1 = SyncAccRcv,\n                    SyncAccRcvLeafCount1 = SyncAccRcvLeafCount,\n                    OtherSyncAccDRLCount1 = OtherSyncAccDRLCount + 1\n            end;\n        false -> % stop_leaf\n            {MyKVItems, LeafCount} = merkle_tree:get_items(Node),\n            SyncAccRcv1 =\n                [{MyMaxItemsCount, MyKVItems} | SyncAccRcv],\n            SyncAccRcvLeafCount1 = SyncAccRcvLeafCount + LeafCount,\n            OtherSyncAccDRLCount1 = OtherSyncAccDRLCount\n    end,\n    merkle_cmp_result(TR, TN, SigSizeI, SigSizeL,\n                      MyMaxItemsCount, OtherMaxItemsCount, SyncIn, Params, Stats,\n                      RestTreeAcc, SyncAccSend, SyncAccRcv1, SyncAccRcvLeafCount1,\n                      MySyncAccDRK, MySyncAccDRLCount, OtherSyncAccDRLCount1,\n                      AccCmp + 1, AccSkip,\n                      NextLvlNodesActIN, HashCmpI_IN, HashCmpL_IN).\n\n%% @doc Helper for calculating the actual used failure rate during merkle\n%%      hash comparisons.\n-spec merkle_calc_used_fr(PrevUsedFr::UsedFrSum, EffectiveFr_I::float(),\n                          EffectiveFr_L::float(), HashCmpI::non_neg_integer(),\n                          HashCmpL::non_neg_integer()) -> UsedFrSum\n        when is_subtype(UsedFrSum, {Sum::float(), Compensation::float()}).\nmerkle_calc_used_fr({PrevUsedFr, PrevCompensation}, EffectiveFr_I,\n                    EffectiveFr_L, HashCmpI, HashCmpL) ->\n    % PrevUsedFr + HashCmpI * EffectiveFr_I +  HashCmpL * EffectiveFr_L\n    util:kahan_sum([HashCmpI * EffectiveFr_I,  HashCmpL * EffectiveFr_L],\n                   PrevUsedFr, PrevCompensation).\n\n%% @doc Helper for adding a leaf node's KV-List to a compressed binary\n%%      during merkle sync.\n%% @see merkle_resolve_retrieve_leaf_hashes/14\n-spec merkle_resolve_add_leaf_hashes(\n        [SyncSend], FRAllLeaves::float(), NumRestLeaves::non_neg_integer(),\n        BucketSizeBits::non_neg_integer(), DupesSizeBits::pos_integer(),\n        Params::#merkle_params{}, PrevFR::UsedFrSum,\n        HashesK::Bin, HashesV::Bin, BucketSizesBin::Bin, DiffSigSizesBin::Bin, DupesCount::Bin,\n        NewSyncAcc::[SyncSend], Dupes::[db_chunk_kv()])\n        -> {{HashesK::Bin, HashesV::Bin, BucketSizesBin::Bin, DiffSigSizesBin::Bin, DupesCount::Bin},\n            NewSyncSend::[SyncSend], Dupes::db_chunk_kv()}\n    when is_subtype(Bin, bitstring()),\n         is_subtype(SyncSend, {OtherMaxItemsCount::non_neg_integer(),\n                               Bucket::merkle_tree:mt_bucket()}),\n         is_subtype(UsedFrSum, {Sum::float(), Compensation::float()}).\nmerkle_resolve_add_leaf_hashes(\n  [{OtherMaxItemsCount, Bucket} | Rest], FRAllLeaves, NumRestLeaves,\n  BucketSizeBits, DupesSizeBits, Params, {PrevFRSum, PrevFrC} = _PrevFR,\n  HashesK, HashesV, BucketSizesBin, DiffSigSizesBin, DupesCount, SyncAcc, Dupes) ->\n    BucketSize = length(Bucket),\n    ExpDelta = 100, % TODO: establish a bound of maximum MaxAffectedItems items for very low (initial) ExpDelta?\n    ?DBG_ASSERT(BucketSize > 0),\n    ?DBG_ASSERT(BucketSize =< util:pow(2, BucketSizeBits)),\n    % note: this includes duplicates which are actually not encoded!\n    ?DBG_ASSERT(?implies(BucketSizeBits =:= 0, BucketSize =:= 1)),\n    BucketSizesBinNew = <<BucketSizesBin/bitstring, (BucketSize - 1):BucketSizeBits>>,\n    FR_next = calc_n_subparts_FR(NumRestLeaves, FRAllLeaves, PrevFRSum),\n    {SigSize, VSize} =\n        trivial_signature_sizes(OtherMaxItemsCount, BucketSize, ExpDelta, FR_next),\n    % note: we can only estimate the real FR of this part here - the other\n    %       node will report back the exact probability based on its actual\n    %       number of items (we nonetheless need this value to adjust the\n    %       individual trivial syncs' signatures!)\n    ThisFR_upper_bound =\n        trivial_worst_case_failrate(SigSize, OtherMaxItemsCount, BucketSize, ExpDelta),\n    NextFR = util:kahan_sum([ThisFR_upper_bound], PrevFRSum, PrevFrC),\n%%     log:pal(\"merkle_send [ ~p ] (rest: ~B):~n  bits: ~p, FR: ~p vs. ~p~n  acc total: ~p -> ~p\",\n%%             [self(), NumRestLeaves, {SigSize, VSize}, FR_next, FR_p1, _PrevFR, NextFR]),\n    {{HashesKBucket, IdxBitsSize, DiffSigSize1}, HashesVBucket, ResortedBucket, AddDupes} =\n        compress_kv_list(Bucket, SigSize, VSize, fun trivial_compress_key/2, return_size),\n    AddDupesL = length(AddDupes),\n    % beware: buckets with 0 encoded items are optimised to <<>>\n    if (BucketSize - AddDupesL) > 0 ->\n           DiffSigSizesBinNew = <<DiffSigSizesBin/bitstring, DiffSigSize1:IdxBitsSize/integer-unit:1>>,\n           HashesKNew = <<HashesK/bitstring, HashesKBucket/bitstring>>,\n           HashesVNew = <<HashesV/bitstring, HashesVBucket/bitstring>>,\n           ok;\n       true ->\n           ?DBG_ASSERT(HashesKBucket =:= <<>>),\n           ?DBG_ASSERT(HashesVBucket =:= <<>>),\n           DiffSigSizesBinNew = DiffSigSizesBin,\n           HashesKNew = HashesK,\n           HashesVNew = HashesV,\n           ok\n    end,\n%%     log:pal(\"merkle_send [ ~p ]:~n  ~p~n  ~p\",\n%%             [self(), {NumRestLeaves, FRAllLeaves, _PrevFR},\n%%              {BucketSize, AddDupesL, OtherMaxItemsCount, FR_next}]),\n    DupesCountNew = <<DupesCount/bitstring, AddDupesL:DupesSizeBits>>,\n    merkle_resolve_add_leaf_hashes(\n      Rest, FRAllLeaves, NumRestLeaves - 1, BucketSizeBits, DupesSizeBits, Params, NextFR,\n      HashesKNew, HashesVNew, BucketSizesBinNew, DiffSigSizesBinNew, DupesCountNew, [{OtherMaxItemsCount, ResortedBucket} | SyncAcc],\n      [AddDupes | Dupes]);\nmerkle_resolve_add_leaf_hashes(\n    [], _FRAllLeaves, _NumRestLeaves, _BucketSizeBits, _DupesSizeBits,\n    _Params, _PrevFR, HashesK, HashesV, BucketSizesBin, DiffSigSizesBin, DupesCount, SyncAcc, Dupes) ->\n    ?DBG_ASSERT(HashesK =/= <<>> orelse HashesV =/= <<>> orelse DupesCount > 0),\n    {{HashesK, HashesV, BucketSizesBin, DiffSigSizesBin, DupesCount},\n     lists:reverse(SyncAcc), lists:flatten(Dupes)}.\n\n%% @doc Helper for retrieving a leaf node's KV-List from the compressed binary\n%%      returned by merkle_resolve_add_leaf_hashes/9 during merkle sync.\n%% @see merkle_resolve_add_leaf_hashes/9\n-spec merkle_resolve_retrieve_leaf_hashes(\n        [SyncRcv], HashesK::Bin, HashesV::Bin, BucketSizesBin::Bin, DiffSigSizesBin::Bin, DupesCount::Bin, FRAllLeaves::float(),\n        NumRestLeaves::non_neg_integer(), PrevFR::UsedFrSum, PrevFR_real::UsedFrSum,\n        BucketSizeBits::non_neg_integer(), DupesSizeBits::pos_integer(), Params::#merkle_params{},\n        ToSendKeysAcc::[?RT:key()], ToResolveIdxAcc::[bitstring()],\n        ResolveNonEmpty::boolean(), DupesCountTotal::non_neg_integer())\n        -> {ToSendKeys::[?RT:key()], ToResolveIdx::Bin,\n            OtherHasResolve::boolean(), ThisFR_real::float()}\n    when is_subtype(Bin, bitstring()),\n         is_subtype(SyncRcv, {MyMaxItemsCount::non_neg_integer(),\n                              Bucket::merkle_tree:mt_bucket()}),\n         is_subtype(UsedFrSum, {Sum::float(), Compensation::float()}).\nmerkle_resolve_retrieve_leaf_hashes(\n  [{MyMaxItemsCount, MyKVItems} | Rest],\n  HashesK, HashesV, BucketSizesBin, DiffSigSizesBin, DupesCount, FRAllLeaves, NumRestLeaves,\n  {PrevFRSum, PrevFrC} = _PrevFR, {PrevFR_real_Sum, PrevFr_real_C} = _PrevFR_real,\n  BucketSizeBits, DupesSizeBits, Params, ToSendKeys, ToResolveIdx,\n  ResolveNonEmpty, DupesCountTotal) ->\n    MyActualItemsCount = length(MyKVItems),\n    ExpDelta = 100, % TODO: establish a bound of maximum MaxAffectedItems items for very low (initial) ExpDelta?\n    <<BucketSize0:BucketSizeBits/integer-unit:1, NBucketSizesBin/bitstring>> = BucketSizesBin,\n    <<Dupes:DupesSizeBits/integer-unit:1, NDupesCount/bitstring>> = DupesCount,\n    OtherActualItemsCount = BucketSize0 + 1,\n    BucketSize = OtherActualItemsCount - Dupes,\n    FR_next = calc_n_subparts_FR(NumRestLeaves, FRAllLeaves, PrevFRSum),\n%%     log:pal(\"merkle_receive [ ~p ]:~n  ~p~n  ~p\",\n%%             [self(), {NumRestLeaves, FRAllLeaves, _PrevFR},\n%%              {BucketSize, Dupes, MyMaxItemsCount, FR_next}]),\n    {SigSize, VSize} =\n        trivial_signature_sizes(MyMaxItemsCount, OtherActualItemsCount, ExpDelta, FR_next),\n    ThisFR_upper_bound =\n        trivial_worst_case_failrate(SigSize, MyMaxItemsCount, OtherActualItemsCount, ExpDelta),\n    NextFR = util:kahan_sum([ThisFR_upper_bound], PrevFRSum, PrevFrC),\n    ThisFR_real =\n        trivial_worst_case_failrate(SigSize, MyActualItemsCount, OtherActualItemsCount, ExpDelta),\n    NextFR_real = util:kahan_sum([ThisFR_real], PrevFR_real_Sum, PrevFr_real_C),\n%%     log:pal(\"merkle_receive [ ~p ] (rest: ~B):~n  bits: ~p, acc: ~p vs. ~p~n  acc total: ~p -> ~p\",\n%%             [self(), NumRestLeaves, {SigSize, VSize}, FR_next, ThisFR_upper_bound, _PrevFR, NextFR]),\n    % beware: buckets with 0 encoded items are optimised to <<>>\n    if BucketSize > 0 ->\n           IdxBitsSize = bits_for_number(SigSize),\n           <<DiffSigSize1:IdxBitsSize/integer-unit:1, NDiffSigSizesBin/bitstring>> = DiffSigSizesBin,\n           OBucketKBinSize = BucketSize * DiffSigSize1,\n           %log:pal(\"merkle: ~B\", [OBucketKBinSize]),\n           OBucketVBinSize = BucketSize * VSize,\n           <<OBucketKBin:OBucketKBinSize/bitstring, NHashesK/bitstring>> = HashesK,\n           <<OBucketVBin:OBucketVBinSize/bitstring, NHashesV/bitstring>> = HashesV,\n           {OBucketTree, VSize} = decompress_kv_list({OBucketKBin, OBucketVBin}, DiffSigSize1, SigSize),\n           ok;\n       true ->\n           OBucketTree = mymaps:new(),\n           NHashesK = HashesK,\n           NHashesV = HashesV,\n           NDiffSigSizesBin = DiffSigSizesBin,\n           ok\n    end,\n\n    % calc diff (trivial sync)\n    ?DBG_ASSERT(MyKVItems =/= []),\n%%     log:pal(\"Tree: ~.2p\", [mymaps:values(OBucketTree)]),\n    % note: dupes are already added to ToSendKeys1\n    {ToSendKeys1, ToReqIdx1, OBucketTree1, _Dupes} =\n        get_diff_with_dupes(\n          MyKVItems, OBucketTree, ToSendKeys, [], SigSize, VSize,\n          fun get_full_diff/4),\n    ReqIdx = lists:usort(\n               [Idx || {_Version, Idx} <- mymaps:values(OBucketTree1)]\n                   ++ ToReqIdx1),\n%%     log:pal(\"pos_to_bitstring(~.2p, ~.2p, 0, ~B)\",\n%%             [ReqIdx, ToResolveIdx, Params#merkle_params.bucket_size]),\n    ToResolveIdx1 = pos_to_bitstring(ReqIdx, ToResolveIdx, 0,\n                                     Params#merkle_params.bucket_size),\n    merkle_resolve_retrieve_leaf_hashes(\n      Rest, NHashesK, NHashesV, NBucketSizesBin, NDiffSigSizesBin, NDupesCount, FRAllLeaves, NumRestLeaves - 1, NextFR,\n      NextFR_real, BucketSizeBits, DupesSizeBits, Params, ToSendKeys1, ToResolveIdx1,\n      ?IIF(ReqIdx =/= [], true, ResolveNonEmpty), DupesCountTotal + Dupes);\nmerkle_resolve_retrieve_leaf_hashes(\n  [], HashesK, HashesV, BucketSizesBin, DiffSigSizesBin, DupesCount, _FRAllLeaves, _NumRestLeaves, _PrevFR,\n  PrevFR_real, _BucketSizeBits, _DupesSizeBits, _Params, ToSendKeys, ToResolveIdx,\n  ResolveNonEmpty, DupesCountTotal) ->\n    ?ASSERT(HashesK =:= <<>>),\n    ?ASSERT(HashesV =:= <<>>),\n    ?ASSERT(BucketSizesBin =:= <<>>),\n    ?ASSERT(DiffSigSizesBin =:= <<>>),\n    ?ASSERT(DupesCount =:= <<>>),\n    ToResolveIdx1 =\n        if ResolveNonEmpty ->\n               erlang:list_to_bitstring(lists:reverse(ToResolveIdx));\n           true ->\n               % optimise this case away (otherwise all entries will be 0)\n               % (also ref. the calls of merkle_resolve_leaves_ckidx/8)\n               <<>>\n        end,\n    {ToSendKeys, ToResolveIdx1, DupesCountTotal > 0 orelse ResolveNonEmpty,\n     element(1, PrevFR_real)}.\n\n%% @doc Creates a compact binary consisting of bitstrings with trivial\n%%      reconciliations for all sync requests to send.\n-spec merkle_resolve_leaves_send(\n        State::state(), UsedFr::{Sum::float(), Compensation::float()})\n    -> NewState::state().\nmerkle_resolve_leaves_send(\n  State = #rr_recon_state{params = Params, initiator = IsInitiator,\n                          stats = Stats, dest_recon_pid = DestReconPid,\n                          dest_rr_pid = DestRRPid,   ownerPid = OwnerL,\n                          merkle_sync = {SyncSend, SyncRcv, SyncRcvLeafCount,\n                                         {MySyncDRK, MySyncDRLCount, OtherSyncDRLCount}}},\n  {UsedFrSum, _UsedFrC}) ->\n%%     log:pal(\"Sync (~s):~n  send:~.2p~n  rcv:~.2p\",\n%%             [?IIF(IsInitiator, \"I\", \"NI\"), SyncSend, SyncRcv]),\n    % resolve items from emptyLeaf-* comparisons with empty leaves on any node as key_upd:\n    NStats0 = send_resolve_request(Stats, MySyncDRK, OwnerL, DestRRPid, IsInitiator, true),\n    NStats1 = rr_recon_stats:set([{fail_rate_p1, UsedFrSum}], NStats0),\n    NStats2 = rr_recon_stats:inc(\n                [{tree_leavesSynced, MySyncDRLCount + OtherSyncDRLCount},\n                 {rs_expected, ?IIF(OtherSyncDRLCount > 0, 1, 0)}], NStats1),\n    % allow the garbage collector to free the SyncDRK list here\n    SyncNew1 = {SyncSend, SyncRcv, SyncRcvLeafCount, {[], MySyncDRLCount, OtherSyncDRLCount}},\n\n    SyncSendL = length(SyncSend),\n    SyncRcvL = length(SyncRcv),\n    TrivialProcs = SyncSendL + SyncRcvL,\n    FRAllLeaves = calc_n_subparts_FR(1, Params#merkle_params.fail_rate, UsedFrSum),\n    ?ALG_DEBUG(\"merkle (~s) - LeafSync~n  ~B (send), ~B (receive), ~B direct (~s)\\tFRAllLeaves: ~g\\t\"\n               \"ItemsToSend: ~B (~g per leaf)~n  send: ~B (L-L) + ~B (L-I), receive: ~B (L-L) + ~B (I-L)\",\n               [?IIF(IsInitiator, \"I\", \"NI\"), SyncSendL, SyncRcvL,\n                MySyncDRLCount, ?IIF(IsInitiator, \"in\", \"out\"),\n                FRAllLeaves,\n                lists:sum([length(MyKVItems) || {_, MyKVItems} <- SyncSend]),\n                ?IIF(SyncSend =/= [],\n                     lists:sum([length(MyKVItems) || {_, MyKVItems} <- SyncSend]) /\n                         SyncSendL, 0.0),\n                length([ok || {OtherMaxIC, _MyKVItems} <- SyncSend, OtherMaxIC =< Params#merkle_params.bucket_size]),\n                SyncSendL - length([ok || {OtherMaxIC, _MyKVItems} <- SyncSend, OtherMaxIC =< Params#merkle_params.bucket_size]),\n                length([ok || {MyMaxIC, _MyKVItems} <- SyncRcv, MyMaxIC =< Params#merkle_params.bucket_size]),\n                SyncRcvL - length([ok || {MyMaxIC, _MyKVItems} <- SyncRcv, MyMaxIC =< Params#merkle_params.bucket_size])]),\n    \n    if SyncSendL =:= 0 andalso SyncRcvL =:= 0 ->\n           % nothing to do\n           shutdown(sync_finished,\n                    State#rr_recon_state{stats = NStats2,\n                                         merkle_sync = SyncNew1, misc = []});\n       SyncSend =/= [] ->\n           % note: we do not have empty buckets here and thus always store (BucketSize - 1)\n           BucketSizeBits = bits_for_number(Params#merkle_params.bucket_size - 1),\n           % note: duplicates can be 0 up to BucketSize\n           DupesSizeBits = bits_for_number(Params#merkle_params.bucket_size),\n           {SyncStruct, NewSyncSend, Dupes} =\n               merkle_resolve_add_leaf_hashes(\n                 SyncSend, FRAllLeaves, TrivialProcs, BucketSizeBits, DupesSizeBits,\n                 Params, {0.0, 0.0}, <<>>, <<>>, <<>>, <<>>, <<>>, [], []),\n           % note: 1 trivial proc contains 1 leaf\n           % the other node will send its items from this CKV list - increase rs_expected, too\n           NStats3 = rr_recon_stats:inc([{tree_leavesSynced, SyncSendL},\n                                         {rs_expected, 1}], NStats2),\n           \n           MerkleSyncNew1 = {NewSyncSend, SyncRcv, SyncRcvLeafCount,\n                             {[], MySyncDRLCount, OtherSyncDRLCount}},\n           ?ALG_DEBUG(\"merkle (~s) - HashesSize: ~B (~B compressed)\",\n                      [?IIF(IsInitiator, \"I\", \"NI\"),\n                       erlang:byte_size(erlang:term_to_binary(SyncStruct)),\n                       erlang:byte_size(\n                         erlang:term_to_binary(SyncStruct, [compressed]))]),\n           send(DestReconPid, {resolve_req, SyncStruct}),\n           State#rr_recon_state{stage = resolve, stats = NStats3,\n                                merkle_sync = MerkleSyncNew1,\n                                next_phase_kv = Dupes,\n                                misc = [{all_leaf_acc, FRAllLeaves},\n                                        {trivial_procs, TrivialProcs}]};\n       true ->\n           % only wait for the other node's resolve_req\n           State#rr_recon_state{stage = resolve, merkle_sync = SyncNew1,\n                                stats = NStats2,\n                                misc = [{all_leaf_acc, FRAllLeaves},\n                                        {trivial_procs, TrivialProcs}]}\n    end.\n\n%% @doc Decodes the trivial reconciliations from merkle_resolve_leaves_send/5\n%%      and resolves them returning a compressed idx list each with keys to\n%%      request.\n-spec merkle_resolve_leaves_receive(\n        State::state(), {HashesK::Bin, HashesV::Bin, BucketSizesBin::Bin,\n                         DiffSigSizesBin::Bin, DupesCount::Bin}) -> NewState::state()\n    when is_subtype(Bin, bitstring()).\nmerkle_resolve_leaves_receive(\n  State = #rr_recon_state{initiator = IsInitiator,\n                          merkle_sync = {SyncSend, SyncRcv, SyncRcvLeafCount, DirectResolve},\n                          params = Params,\n                          dest_rr_pid = DestRRPid,   ownerPid = OwnerL,\n                          dest_recon_pid = DestRCPid, stats = Stats,\n                          misc = [{all_leaf_acc, FRAllLeaves},\n                                  {trivial_procs, TrivialProcs}]},\n  {HashesK, HashesV, BucketSizesBin, DiffSigSizesBin, DupesCount}) ->\n    ?DBG_ASSERT(HashesK =/= <<>> orelse HashesV =/= <<>> orelse DupesCount > 0),\n    % note: we do not have empty buckets here and thus always store (BucketSize - 1)\n    BucketSizeBits = bits_for_number(Params#merkle_params.bucket_size - 1),\n    % note: duplicates can be 0 up to BucketSize\n    DupesSizeBits = bits_for_number(Params#merkle_params.bucket_size),\n    % mismatches to resolve:\n    % * at initiator    : inner(I)-leaf(NI) or leaf(NI)-non-empty-leaf(I)\n    % * at non-initiator: inner(NI)-leaf(I)\n    % note: 1 trivial proc may thus contain more than 1 leaf!\n    {ToSend, ToResolve, OtherHasResolve, ThisFR_real} =\n        merkle_resolve_retrieve_leaf_hashes(\n          SyncRcv, HashesK, HashesV, BucketSizesBin, DiffSigSizesBin, DupesCount, FRAllLeaves, TrivialProcs,\n          {0.0, 0.0}, {0.0, 0.0},\n          BucketSizeBits, DupesSizeBits, Params, [], [], false, 0),\n\n    % send resolve message:\n    % resolve items we should send as key_upd:\n    % NOTE: the other node does not know whether our ToSend is empty and thus\n    %       always expects a following resolve process!\n    Stats1 = send_resolve_request(Stats, ToSend, OwnerL, DestRRPid, IsInitiator, false),\n    % let the other node's rr_recon process identify the remaining keys;\n    % it will use key_upd_send (if non-empty) and we must thus increase\n    % the number of resolve processes here!\n    MerkleResReqs = ?IIF(OtherHasResolve, 1, 0),\n    Stats2 = rr_recon_stats:inc([{tree_leavesSynced, SyncRcvLeafCount},\n                                 {rs_expected, MerkleResReqs}], Stats1),\n    ?DBG_ASSERT(rr_recon_stats:get(fail_rate_p2, Stats2) =:= 0.0),\n    NStats  = rr_recon_stats:set([{fail_rate_p2, ThisFR_real}], Stats2),\n    ?ALG_DEBUG(\"resolve_req Merkle~n  Session=~.0p ; resolve expexted=~B\",\n               [rr_recon_stats:get(session_id, NStats),\n                rr_recon_stats:get(rs_expected, NStats)]),\n\n    comm:send(DestRCPid, {resolve_req, ToResolve, ThisFR_real}),\n    % free up some memory:\n    NewState = State#rr_recon_state{merkle_sync = {SyncSend, [], SyncRcvLeafCount, DirectResolve},\n                                    stats = NStats},\n    % shutdown if nothing was sent (otherwise we need to wait for the returning CKidx):\n    if SyncSend =:= [] -> shutdown(sync_finished, NewState);\n       true -> NewState\n    end.\n\n%% @doc Decodes all requested keys from merkle_resolve_leaves_receive/2 (as a\n%%      result of sending resolve requests) and resolves the appropriate entries\n%%      (if non-empty) with our data using a key_upd_send.\n-spec merkle_resolve_leaves_ckidx(\n        Sync::[merkle_sync_send()], BinKeyList::bitstring(), DestRRPid::comm:mypid(),\n        Stats, OwnerL::comm:erl_local_pid(), Params::#merkle_params{},\n        ToSend::[?RT:key()], IsInitiator::boolean()) -> NewStats::Stats\n    when is_subtype(Stats, rr_recon_stats:stats()).\nmerkle_resolve_leaves_ckidx([{_OtherMaxItemsCount, MyKVItems} | TL],\n                             BinKeyList0,\n                             DestRRPid, Stats, OwnerL, Params, ToSend, IsInitiator) ->\n    Positions = Params#merkle_params.bucket_size,\n    <<ReqKeys:Positions/bitstring-unit:1, BinKeyList/bitstring>> = BinKeyList0,\n    ToSend1 = bitstring_to_k_list_kv(ReqKeys, MyKVItems, ToSend),\n    merkle_resolve_leaves_ckidx(TL, BinKeyList, DestRRPid, Stats, OwnerL, Params,\n                                ToSend1, IsInitiator);\nmerkle_resolve_leaves_ckidx([], <<>>, DestRRPid, Stats, OwnerL, _Params,\n                            [_|_] = ToSend, IsInitiator) ->\n    send_resolve_request(Stats, ToSend, OwnerL, DestRRPid, IsInitiator, false).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% art recon\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Gets all leaves in the merkle node list (recursively) which are not\n%%      present in the art structure.\n-spec art_get_sync_leaves(Nodes::NodeL, art:art(), ToSyncAcc::NodeL,\n                          NCompAcc::non_neg_integer(), NSkipAcc::non_neg_integer(),\n                          NLSyncAcc::non_neg_integer())\n        -> {ToSync::NodeL, NComp::non_neg_integer(), NSkip::non_neg_integer(),\n            NLSync::non_neg_integer()} when\n    is_subtype(NodeL,  [merkle_tree:mt_node()]).\nart_get_sync_leaves([], _Art, ToSyncAcc, NCompAcc, NSkipAcc, NLSyncAcc) ->\n    {ToSyncAcc, NCompAcc, NSkipAcc, NLSyncAcc};\nart_get_sync_leaves([Node | Rest], Art, ToSyncAcc, NCompAcc, NSkipAcc, NLSyncAcc) ->\n    NComp = NCompAcc + 1,\n    IsLeaf = merkle_tree:is_leaf(Node),\n    case art:lookup(Node, Art) of\n        true ->\n            NSkip = NSkipAcc + ?IIF(IsLeaf, 0, merkle_tree:size(Node) - 1),\n            art_get_sync_leaves(Rest, Art, ToSyncAcc, NComp, NSkip, NLSyncAcc);\n        false ->\n            if IsLeaf ->\n                   art_get_sync_leaves(Rest, Art, [Node | ToSyncAcc],\n                                       NComp, NSkipAcc, NLSyncAcc + 1);\n               true ->\n                   art_get_sync_leaves(\n                     lists:append(merkle_tree:get_childs(Node), Rest), Art,\n                     ToSyncAcc, NComp, NSkipAcc, NLSyncAcc)\n            end\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Sends a request_resolve message to the rrepair layer which sends the\n%%      entries from the given keys to the other node with a feedback request.\n-spec send_resolve_request(Stats, ToSend::[?RT:key()], OwnerL::comm:erl_local_pid(),\n                           DestRRPid::comm:mypid(), IsInitiator::boolean(),\n                           SkipIfEmpty::boolean()) -> Stats\n    when is_subtype(Stats, rr_recon_stats:stats()).\nsend_resolve_request(Stats, [] = _ToSend, _OwnerL, _DestRRPid, _IsInitiator,\n                     true = _SkipIfEmpty) ->\n    ?ALG_DEBUG(\"Resolve~n  Session=~.0p ; ToSend=~p\",\n               [rr_recon_stats:get(session_id, Stats), 0]),\n    Stats;\nsend_resolve_request(Stats, ToSend, OwnerL, DestRRPid, IsInitiator,\n                     _SkipIfEmpty) ->\n    SID = rr_recon_stats:get(session_id, Stats),\n    ?ALG_DEBUG(\"Resolve~n  Session=~.0p ; ToSend=~p\", [SID, length(ToSend)]),\n    % note: the resolve request is counted at the initiator and\n    %       thus from_my_node must be set accordingly on this node!\n    ?DBG_ASSERT2(length(ToSend) =:= length(lists:usort(ToSend)),\n                 {non_unique_send_list, ToSend}),\n    send_local(OwnerL, {request_resolve, SID,\n                        {key_upd_send, DestRRPid, ToSend, _ToReq = []},\n                        [{from_my_node, ?IIF(IsInitiator, 1, 0)},\n                         {feedback_request, comm:make_global(OwnerL)}]}),\n    % key_upd_send + one reply from a subsequent feedback response (?key_upd)\n    rr_recon_stats:inc([{rs_expected, 2}], Stats).\n\n%% @doc Gets the number of bits needed to encode the given number.\n-spec bits_for_number(Number::pos_integer()) -> pos_integer();\n                     (0) -> 0.\nbits_for_number(0) -> 0;\nbits_for_number(Number) ->\n    util:ceil(util:log2(Number + 1)).\n\n%% @doc Splits the target failure rate FR into N (not necessarily independent)\n%%      sub-processes with equal failure rates and returns the FR to use for\n%%      each of them: FR_sub = FR / N.\n%%      This is based on the linearity of the expected number of failures.\n-spec calc_n_subparts_FR(N::pos_integer(), FR::float()) -> FR_sub::float().\ncalc_n_subparts_FR(N, FR) when FR > 0 andalso N >= 1 ->\n    _FR_sub = FR / N.\n\n%% @doc Splits the target failure rate FR into N further almost equal (not\n%%      necessarily independent) independent sub-processes and returns the\n%%      target FR use for the next of these sub-processes with all the previous\n%%      sub-processes having a combined failure rate of PrevFr.\n%%      This is based on the linearity of the expected number of failures.\n-spec calc_n_subparts_FR(N::pos_integer(), FR::float(), PrevFr::float())\n        -> FR_sub::float().\ncalc_n_subparts_FR(N, FR, PrevFr)\n  when FR > 0 andalso PrevFr >= 0 andalso N >= 1 ->\n    FR_sub = (FR - PrevFr) / N,\n    % a previous phase may overstep (if item counts change during the setup)\n    % -> allow that and print a warning\n    if FR_sub > 0 ->\n           FR_sub;\n       true ->\n           log:log(\"~w: [ ~.0p:~.0p ] FR constraint broken (phase 1 overstepped?)~n\"\n                   \"  continuing with ~p instead (~p, ~p, ~B)\",\n                   [?MODULE, pid_groups:my_groupname(), self(), 1.0e-16,\n                    FR, PrevFr, N]),\n           1.0e-16\n    end.\n\n%% @doc Calculates the signature sizes for comparing every item in Items\n%%      (at most ItemCount) with OtherItemCount other items and expecting at\n%%      most min(ItemCount, OtherItemCount) version comparisons.\n%%      Sets the bit sizes to have a failure rate below FR.\n%% @see shash_signature_sizes/4\n-spec trivial_signature_sizes\n        (ItemCountI::non_neg_integer(), ItemCountNI::non_neg_integer(),\n         ExpDelta::number(), FR::float())\n        -> {SigSize::signature_size(), VSize::signature_size()}.\ntrivial_signature_sizes(0, _ItemCountNI, _ExpDelta, FR) when FR > 0 ->\n    {0, 0}; % invalid but since there are 0 items, this is ok!\ntrivial_signature_sizes(_ItemCountI, 0, _ExpDelta, FR) when FR > 0 ->\n    {0, 0}; % invalid but since there are 0 items, this is ok!\ntrivial_signature_sizes(ItemCountI, ItemCountNI, ExpDelta, FR) when FR > 0 ->\n    MaxKeySize = 128, % see compress_key/2\n    VSize = get_min_version_bits(),\n    NT = calc_max_different_hashes(ItemCountI, ItemCountNI, ExpDelta),\n    SigSize =\n        if NT > 1 ->\n               %% log_2(1 / (1 - (1 - FR / (2*NT))^(1 / (NT-1))))\n               Y = calc_one_m_xpow_one_m_z(1 / (NT - 1), FR / (2 * NT)),\n               min_max(util:ceil(util:log2(1 / Y)), get_min_hash_bits(), MaxKeySize);\n           NT =:= 1 ->\n               % should only happen for ExpDelta == 0, but may occur in other\n               % cases due to floating point issues\n               get_min_hash_bits()\n        end,\n%%     log:pal(\"trivial [ ~p ] - FR: ~p, \\tSigSize: ~B, \\tVSizeL: ~B~n\"\n%%             \"IC@I: ~B, \\tIC@NI: ~B\",\n%%             [self(), FR, SigSize, VSize, ItemCountI, ItemCountNI]),\n    {SigSize, VSize}.\n\n%% @doc Calculates the worst-case failure rate of the trivial algorithm\n%%      with the given signature size, item counts and expected delta.\n%%      NOTE: Precision loss may occur for very high values!\n%% @see shash_worst_case_failrate/4\n-spec trivial_worst_case_failrate(\n        SigSize::signature_size(), ItemCountI::non_neg_integer(),\n        ItemCountNI::non_neg_integer(), ExpDelta::number()) -> float().\ntrivial_worst_case_failrate(0, 0, _ItemCountNI, _ExpDelta) ->\n    % this is exact! (see special case in trivial_signature_sizes/4)\n    0.0;\ntrivial_worst_case_failrate(0, _ItemCountI, 0, _ExpDelta) ->\n    % this is exact! (see special case in trivial_signature_sizes/4)\n    0.0;\ntrivial_worst_case_failrate(SigSize, ItemCountI, ItemCountNI, ExpDelta) ->\n    BK2 = util:pow(2, SigSize),\n    NT = calc_max_different_hashes(ItemCountI, ItemCountNI, ExpDelta),\n    % exact but with problems for small 1 / BK2:\n%%     2 * NT * (1 - math:pow(1 - 1 / BK2, NT - 1)).\n    2 * NT * calc_one_m_xpow_one_m_z(NT - 1, 1 / BK2).\n\n%% @doc Creates a compressed key-value list for comparing every item in Items\n%%      (at most ItemCount) with OtherItemCount other items and expecting at\n%%      most min(ItemCount, OtherItemCount) version comparisons.\n%%      Sets the bit sizes to have an error below FR.\n-spec compress_kv_list_fr(\n        Items::db_chunk_kv(), ItemCountI::IC, ItemCountNI::IC, ExpDelta, FR,\n        SigFun::fun((ItemCountI::IC, ItemCountNI::IC, ExpDelta, FR) -> {SigSize, VSize::signature_size()}),\n        KeyComprFun::fun(({?RT:key(), client_version()}, SigSize) -> bitstring()))\n        -> {KeyDiff::Bin, VBin::Bin, ResortedKOrigList::db_chunk_kv(),\n            Dupes::db_chunk_kv(), SigSize::signature_size(), VSize::signature_size()}\n    when is_subtype(Bin, bitstring()),\n         is_subtype(IC, non_neg_integer()),\n         is_subtype(ExpDelta, number()),\n         is_subtype(FR, float()),\n         is_subtype(SigSize, signature_size()).\ncompress_kv_list_fr(DBItems, ItemCountI, ItemCountNI, ExpDelta, FR, SigFun, KeyComprFun) ->\n    {SigSize, VSize} = SigFun(ItemCountI, ItemCountNI, ExpDelta, FR),\n    {HashesKNew, HashesVNew, ResortedBucket, Dupes} =\n        compress_kv_list(DBItems, SigSize, VSize, KeyComprFun, integrate_size),\n    % debug compressed and uncompressed sizes:\n    ?ALG_DEBUG(\"compress_kv_list (ExpDelta = ~p)~n\"\n               \"  ~Bckv vs. ~Bcmp items (~B dupes), SigSize: ~B, VSize: ~B, ChunkSize: ~B+~B / ~B+~B bits\",\n               [ExpDelta, ItemCountNI, ItemCountI, length(Dupes), SigSize, VSize,\n                erlang:bit_size(erlang:term_to_binary(HashesKNew)),\n                erlang:bit_size(erlang:term_to_binary(HashesVNew)),\n                erlang:bit_size(\n                  erlang:term_to_binary(HashesKNew,\n                                        [{minor_version, 1}, {compressed, 2}])),\n                erlang:bit_size(\n                  erlang:term_to_binary(HashesVNew,\n                                        [{minor_version, 1}, {compressed, 2}]))]),\n    ?DBG_ASSERT(?implies(VSize =/= 0, (HashesKNew =:= <<>>) =:= (HashesVNew =:= <<>>))),\n    {HashesKNew, HashesVNew, ResortedBucket, Dupes, SigSize, VSize}.\n\n%% @doc Calculates the signature size for comparing ItemCount items with\n%%      OtherItemCount other items (including versions into the hashes).\n%%      Sets the bit size to have a failure rate below FR.\n%% @see trivial_signature_sizes/4\n-spec shash_signature_sizes(\n        ItemCountI::non_neg_integer(), ItemCountNI::non_neg_integer(),\n        ExpDelta::number(), FR::float())\n        -> {SigSize::signature_size(), _VSize::0}.\nshash_signature_sizes(0, _ItemCountNI, _ExpDelta, FR) when FR > 0 ->\n    {0, 0}; % invalid but since there are 0 items, this is ok!\nshash_signature_sizes(_ItemCountI, 0, _ExpDelta, FR) when FR > 0 ->\n    {0, 0}; % invalid but since there are 0 items, this is ok!\nshash_signature_sizes(ItemCountI, ItemCountNI, ExpDelta, FR) when FR > 0 ->\n    MaxSize = 128, % see compress_key/2\n    N_delta_I = calc_max_different_items_node(ItemCountI, ItemCountNI, ExpDelta),\n    N_delta_NI = calc_max_different_items_node(ItemCountNI, ItemCountI, ExpDelta),\n    SigSize0 = util:ceil(\n                 util:log2(\n                   % if the delta is low and/or FR is high, this term is negative\n                   % and we cannot apply the logarithm - here, any signature\n                   % size suffices and we choose the lowest non-zero value\n                   erlang:max(1,\n                              2 * N_delta_I * N_delta_NI / FR\n                                  - N_delta_NI - ItemCountI + 2))),\n    SigSize = min_max(SigSize0, get_min_hash_bits(), MaxSize),\n%%     log:pal(\"shash [ ~p ] - FR: ~p, \\tSigSize: ~B, \\tIC@I: ~B, \\tIC@NI: ~B~n\"\n%%             \"  Ndelta_@I: ~B, \\tNdelta_@I: ~B\",\n%%             [self(), FR, SigSize, ItemCountI, ItemCountNI, N_delta_I, N_delta_NI]),\n    {SigSize, 0}.\n\n%% @doc Calculates the worst-case failure rate of the SHash algorithm\n%%      with the given signature size, item counts and expected delta.\n%%      NOTE: Precision loss may occur for very high values!\n%% @see trivial_worst_case_failrate/4\n-spec shash_worst_case_failrate(\n        SigSize::signature_size(), ItemCountI::non_neg_integer(),\n        ItemCountNI::non_neg_integer(), ExpDelta::number()) -> float().\nshash_worst_case_failrate(0, 0, _ItemCountNI, _ExpDelta) ->\n    % this is exact! (see special case in trivial_signature_sizes/4)\n    0.0;\nshash_worst_case_failrate(0, _ItemCountI, 0, _ExpDelta) ->\n    % this is exact! (see special case in trivial_signature_sizes/4)\n    0.0;\nshash_worst_case_failrate(SigSize, ItemCountI, ItemCountNI, ExpDelta) ->\n    BK2 = util:pow(2, SigSize),\n    N_delta_I = calc_max_different_items_node(ItemCountI, ItemCountNI, ExpDelta),\n    N_delta_NI = calc_max_different_items_node(ItemCountNI, ItemCountI, ExpDelta),\n    % exact but with problems for small 1 / BK2:\n%%     2 * N_delta_I * N_delta_NI / BK2 * math:pow(1 - 1 / BK2, N_delta_NI + ItemCountI - 2),\n    2 * N_delta_I * N_delta_NI / BK2 * math:exp((N_delta_NI + ItemCountI - 2) * util:log1p(-1 / BK2)).\n\n%% @doc Calculates the bloom target FP for each of two Bloom filters so that\n%%      their combined expected failure rate is below FR, assuming the worst\n%%      case in the number of item checks that could yield false positives,\n%%      i.e. items that are not encoded in the Bloom filter, taking the\n%%      expected delta into account.\n-spec bloom_target_fp(\n        ItemCountI::non_neg_integer(), ItemCountNI::non_neg_integer(),\n        ExpDelta::number(), FR::float()) -> TargetFP::float().\nbloom_target_fp(ItemCountI, ItemCountNI, ExpDelta, FR) ->\n    NrChecksNotInBFI = calc_max_different_items_node(ItemCountI, ItemCountNI, ExpDelta),\n    NrChecksNotInBFNI = calc_max_different_items_node(ItemCountNI, ItemCountI, ExpDelta),\n    % assume at least one comparison, even if empty sets are reconciled\n    FR / erlang:max(NrChecksNotInBFI + NrChecksNotInBFNI, 1).\n\n%% @doc Calculates the worst-case failure probability of the bloom\n%%      reconciliation with the given two Bloom filters from either node.\n%%      NOTE: Precision loss may occur for very high values!\n-spec bloom_worst_case_failrate(\n        BFI::bloom:bloom_filter(), BFNI::bloom:bloom_filter(),\n        ExpDelta::number()) -> float().\nbloom_worst_case_failrate(BFI, BFNI, ExpDelta) ->\n    FprI = bloom:get_property(BFI, fpr),\n    FprNI = bloom:get_property(BFNI, fpr),\n    ItemCountI = bloom:get_property(BFI, items_count),\n    ItemCountNI = bloom:get_property(BFNI, items_count),\n    bloom_worst_case_failrate_(FprI, FprNI, ItemCountI, ItemCountNI, ExpDelta).\n\n%% @doc Helper for bloom_worst_case_failrate/3.\n%% @see bloom_worst_case_failrate/3\n-spec bloom_worst_case_failrate_(\n        FprI::float(), FprNI::float(),\n        ItemCountI::non_neg_integer(), ItemCountNI::non_neg_integer(),\n        ExpDelta::number()) -> float().\nbloom_worst_case_failrate_(FprI, FprNI, ItemCountI, ItemCountNI, ExpDelta) ->\n    ?DBG_ASSERT2(FprI >= 0 andalso FprI =< 1, FprI),\n    ?DBG_ASSERT2(FprNI >= 0 andalso FprNI =< 1, FprNI),\n    NrChecksNotInBFI = calc_max_different_items_node(ItemCountI, ItemCountNI, ExpDelta),\n    NrChecksNotInBFNI = calc_max_different_items_node(ItemCountNI, ItemCountI, ExpDelta),\n    NrChecksNotInBFI * FprNI + NrChecksNotInBFNI * FprI.\n\n-spec build_recon_struct(\n        method(), DestI::intervals:non_empty_interval(), db_chunk_kv(),\n        InitiatorMaxItems::non_neg_integer() | undefined, % not applicable on iniator\n        Params::parameters() | {})\n        -> {sync_struct(), Fr_p1::float()}.\nbuild_recon_struct(trivial, I, DBItems, InitiatorMaxItems, _Params) ->\n    % at non-initiator\n    ?DBG_ASSERT(not intervals:is_empty(I)),\n    ?DBG_ASSERT(InitiatorMaxItems =/= undefined),\n    ItemCount = length(DBItems),\n    ExpDelta = get_max_expected_delta(),\n    {MyDiffK, MyDiffV, ResortedKVOrigList, Dupes, SigSize, _VSize} =\n        compress_kv_list_fr(DBItems, InitiatorMaxItems, ItemCount,\n                             ExpDelta, get_failure_rate(),\n                             fun trivial_signature_sizes/4, fun trivial_compress_key/2),\n    {#trivial_sync{interval = I, reconPid = comm:this(), exp_delta = ExpDelta,\n                   db_chunk = {MyDiffK, MyDiffV, ResortedKVOrigList, Dupes},\n                   '+item_count' = length(Dupes), sig_size = SigSize},\n     % Note: we can only guess the number of items of the initiator here, so\n     %       this is not exactly the failure rate of phase 1!\n     _Fr_p1 = trivial_worst_case_failrate(SigSize, InitiatorMaxItems, ItemCount, ExpDelta)};\nbuild_recon_struct(shash, I, DBItems, InitiatorMaxItems, _Params) ->\n    % at non-initiator\n    ?DBG_ASSERT(not intervals:is_empty(I)),\n    ?DBG_ASSERT(InitiatorMaxItems =/= undefined),\n    ItemCount = length(DBItems),\n    FR = get_failure_rate(),\n    FR_p1 = calc_n_subparts_FR(2, FR),\n    ExpDelta = get_max_expected_delta(),\n    {MyDiffK, <<>>, ResortedKVOrigList, Dupes, SigSize, 0} =\n        compress_kv_list_fr(DBItems, InitiatorMaxItems, ItemCount,\n                             ExpDelta, FR_p1,\n                             fun shash_signature_sizes/4, fun compress_key/2),\n    {#shash_sync{interval = I, reconPid = comm:this(), exp_delta = ExpDelta,\n                 db_chunk = {MyDiffK, ResortedKVOrigList, Dupes},\n                 '+item_count' = length(Dupes), sig_size = SigSize,\n                 fail_rate = FR},\n     % Note: we can only guess the number of items of the initiator here, so\n     %       this is not exactly the failure rate of phase 1!\n     _Fr_p1 = shash_worst_case_failrate(SigSize, InitiatorMaxItems, ItemCount, ExpDelta)};\nbuild_recon_struct(bloom, I, DBItems, InitiatorMaxItems, _Params) ->\n    % at non-initiator\n    ?DBG_ASSERT(not intervals:is_empty(I)),\n    ?DBG_ASSERT(InitiatorMaxItems =/= undefined),\n    MyMaxItems = length(DBItems),\n    FR = get_failure_rate(),\n    FR_p1 = calc_n_subparts_FR(2, FR),\n    ExpDelta = get_max_expected_delta(),\n    % decide for a common Bloom filter size (and number of hash functions)\n    % for an efficient diff BF - use a combination where both Bloom filters\n    % are below the targeted FR_p1_bf (we may thus not reach FR_p1 exactly)\n    % based on a common target FPt:\n    FPt = bloom_target_fp(InitiatorMaxItems, MyMaxItems, ExpDelta, FR_p1),\n    {K1, M1} = bloom:calc_HF_num_Size_opt(MyMaxItems, FPt),\n    {K2, M2} = bloom:calc_HF_num_Size_opt(InitiatorMaxItems, FPt),\n    % total failure rates in phase 1 for the two options {K1, M1} vs. {K2, M2}\n    FP1_I = bloom:calc_FPR(M1, InitiatorMaxItems, K1),\n    FP1_NI = bloom:calc_FPR(M1, MyMaxItems, K1),\n    Fr1_p1 = bloom_worst_case_failrate_(FP1_I, FP1_NI, InitiatorMaxItems,\n                                        MyMaxItems, ExpDelta),\n    FP2_I = bloom:calc_FPR(M2, InitiatorMaxItems, K2),\n    FP2_NI = bloom:calc_FPR(M2, MyMaxItems, K2),\n    Fr2_p1 = bloom_worst_case_failrate_(FP2_I, FP2_NI, InitiatorMaxItems,\n                                        MyMaxItems, ExpDelta),\n    ?ALG_DEBUG(\"Bloom My: ~B OtherMax: ~B FPt: ~p~n  bloom1: ~p~n  bloom2: ~p\",\n               [MyMaxItems, InitiatorMaxItems, FPt,\n                {K1, M1, FP1_I, FP1_NI}, {K2, M2, FP2_I, FP2_NI}]),\n    {K, M, Fr_p1} = bloom:select_best(FR_p1, K1, M1, Fr1_p1, K2, M2, Fr2_p1),\n    BF0 = bloom:new(M, K),\n    if Fr_p1 > FR_p1 ->\n           log:log(\"~w: [ ~.0p:~.0p ] FR constraint for phase 1 probably broken\",\n                   [?MODULE, pid_groups:my_groupname(), self()]);\n       true -> ok\n    end,\n    ?ALG_DEBUG(\"NI:~.0p, FR(phase1)=~p, ExpDelta = ~p~n\"\n               \"  m=~B k=~B NICount=~B ICount=~B~n\"\n               \"  fr(phase1)=~p\",\n               [comm:this(), FR_p1, ExpDelta, M, K,\n                MyMaxItems, InitiatorMaxItems, Fr_p1]),\n    BF = bloom:add_list(BF0, DBItems),\n    {#bloom_sync{interval = I, reconPid = comm:this(), exp_delta = ExpDelta,\n                 bf = BF, item_count = MyMaxItems, hf_count = K,\n                 fail_rate = FR},\n     % Note: we can only guess the number of items of the initiator here, so\n     %       this is not exactly the failure rate of phase 1!\n     Fr_p1};\nbuild_recon_struct(merkle_tree, I, DBItems, _InitiatorMaxItems, Params) ->\n    ?DBG_ASSERT(not intervals:is_empty(I)),\n    Fr_p1 = 0.0, % needs to be set at the end of phase 1!\n    case Params of\n        {} ->\n            % merkle_tree - at non-initiator!\n            ?DBG_ASSERT(_InitiatorMaxItems =/= undefined),\n            MOpts = [{branch_factor, get_merkle_branch_factor()},\n                     {bucket_size, get_merkle_bucket_size()}],\n            % do not build the real tree here - build during begin_sync so that\n            % the initiator can start creating its struct earlier and in parallel\n            % the actual build process is executed in begin_sync/2\n            {merkle_tree:new(I, [{keep_bucket, true} | MOpts]), Fr_p1};\n        #merkle_params{branch_factor = BranchFactor,\n                       bucket_size = BucketSize,\n                       num_trees = NumTrees} ->\n            % merkle_tree - at initiator!\n            ?DBG_ASSERT(_InitiatorMaxItems =:= undefined),\n            MOpts = [{branch_factor, BranchFactor},\n                     {bucket_size, BucketSize}],\n            % build now\n            RootsI = intervals:split(I, NumTrees),\n            ICBList = merkle_tree:keys_to_intervals(DBItems, RootsI),\n            {[merkle_tree:get_root(\n               merkle_tree:new(SubI, Bucket, [{keep_bucket, true} | MOpts]))\n               || {SubI, _Count, Bucket} <- ICBList], Fr_p1};\n        #art_recon_struct{branch_factor = BranchFactor,\n                          bucket_size = BucketSize} ->\n            % ART at initiator\n            MOpts = [{branch_factor, BranchFactor},\n                     {bucket_size, BucketSize},\n                     {leaf_hf, fun art:merkle_leaf_hf/2}],\n            {merkle_tree:new(I, DBItems, [{keep_bucket, true} | MOpts]),\n             Fr_p1\n            }\n    end;\nbuild_recon_struct(art, I, DBItems, _InitiatorMaxItems, _Params = {}) ->\n    % ART at non-initiator\n    ?DBG_ASSERT(not intervals:is_empty(I)),\n    ?DBG_ASSERT(_InitiatorMaxItems =/= undefined),\n    BranchFactor = get_merkle_branch_factor(),\n    BucketSize = merkle_tree:get_opt_bucket_size(length(DBItems), BranchFactor, 1),\n    Tree = merkle_tree:new(I, DBItems, [{branch_factor, BranchFactor},\n                                        {bucket_size, BucketSize},\n                                        {leaf_hf, fun art:merkle_leaf_hf/2},\n                                        {keep_bucket, true}]),\n    % create art struct:\n    {#art_recon_struct{art = art:new(Tree, get_art_config()),\n                       reconPid = comm:this(),\n                       branch_factor = BranchFactor,\n                       bucket_size = BucketSize},\n     _Fr_p1 = 0.0\n    }.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% HELPER\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec send(Pid::comm:mypid(), Msg::comm:message() | comm:group_message()) -> ok.\nsend(Pid, Msg) ->\n    ?TRACE_SEND(Pid, Msg),\n    comm:send(Pid, Msg).\n\n-spec send_local(Pid::comm:erl_local_pid(), Msg::comm:message() | comm:group_message()) -> ok.\nsend_local(Pid, Msg) ->\n    ?TRACE_SEND(Pid, Msg),\n    comm:send_local(Pid, Msg).\n\n%% @doc Sends a get_chunk request to the local DHT_node process.\n%%      Request responds with a list of {Key, Version, Value} tuples (if set\n%%      for resolve) or {Key, Version} tuples (anything else).\n%%      The mapping to DestI is not done here!\n-spec send_chunk_req(DhtPid::LPid, AnswerPid::LPid, ChunkI::intervals:interval(),\n                     MaxItems::pos_integer() | all) -> ok when\n    is_subtype(LPid,        comm:erl_local_pid()).\nsend_chunk_req(DhtPid, SrcPid, I, MaxItems) ->\n    SrcPidReply = comm:reply_as(SrcPid, 2, {process_db, '_'}),\n    send_local(DhtPid,\n               {get_chunk, SrcPidReply, I, fun get_chunk_filter/1,\n                fun get_chunk_kv/1, MaxItems}).\n\n-spec get_chunk_filter(db_entry:entry()) -> boolean().\nget_chunk_filter(DBEntry) -> db_entry:get_version(DBEntry) =/= -1.\n-spec get_chunk_kv(db_entry:entry()) -> {?RT:key(), client_version() | -1}.\nget_chunk_kv(DBEntry) -> {db_entry:get_key(DBEntry), db_entry:get_version(DBEntry)}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec exit_reason_to_rc_status(exit_reason()) -> rr_recon_stats:status().\nexit_reason_to_rc_status(sync_finished) -> finish;\nexit_reason_to_rc_status(sync_finished_remote) -> finish;\nexit_reason_to_rc_status(_) -> abort.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Maps any key (K) into a given interval (I). If K is already in I, K is returned.\n%%      If K has more than one associated key in I, the closest one is returned.\n%%      If all associated keys of K are not in I, none is returned.\n-spec map_key_to_interval(?RT:key(), intervals:interval()) -> ?RT:key() | none.\nmap_key_to_interval(Key, I) ->\n    RGrp = [K || K <- ?RT:get_replica_keys(Key), intervals:in(K, I)],\n    case RGrp of\n        [] -> none;\n        [R] -> R;\n        [H|T] ->\n            element(1, lists:foldl(fun(X, {_KeyIn, DistIn} = AccIn) ->\n                                           DistX = key_dist(X, Key),\n                                           if DistX < DistIn -> {X, DistX};\n                                              true -> AccIn\n                                           end\n                                   end, {H, key_dist(H, Key)}, T))\n    end.\n\n-compile({inline, [key_dist/2]}).\n\n-spec key_dist(Key1::?RT:key(), Key2::?RT:key()) -> number().\nkey_dist(Key, Key) -> 0;\nkey_dist(Key1, Key2) ->\n    Dist1 = ?RT:get_range(Key1, Key2),\n    Dist2 = ?RT:get_range(Key2, Key1),\n    erlang:min(Dist1, Dist2).\n\n%% @doc Maps an abitrary key to its associated key in replication quadrant Q.\n-spec map_key_to_quadrant(?RT:key(), rt_beh:segment()) -> ?RT:key().\nmap_key_to_quadrant(Key, Q) ->\n    RKeys = ?RT:get_replica_keys(Key),\n    map_rkeys_to_quadrant(RKeys, Q).\n\n%% @doc Returns a key in the given replication quadrant Q from a list of\n%%      replica keys.\n-spec map_rkeys_to_quadrant([?RT:key(),...], rt_beh:segment()) -> ?RT:key().\nmap_rkeys_to_quadrant(RKeys, Q) ->\n    SegM = case lists:member(?MINUS_INFINITY, RKeys) of\n               true -> Q rem config:read(replication_factor) + 1;\n               _ -> Q\n           end,\n    hd(lists:dropwhile(fun(K) -> ?RT:get_key_segment(K) =/= SegM end, RKeys)).\n\n%% @doc Gets the quadrant intervals.\n-spec quadrant_intervals() -> [intervals:non_empty_interval(),...].\nquadrant_intervals() ->\n    case ?RT:get_replica_keys(?MINUS_INFINITY) of\n        [_]               -> [intervals:all()];\n        [HB,_|_] = Borders -> quadrant_intervals_(Borders, [], HB)\n    end.\n\n%% @doc Internal helper for quadrant_intervals/0 - keep in sync with\n%%      map_key_to_quadrant/2!\n%%      PRE: keys in Borders and HB must be unique (as created in quadrant_intervals/0)!\n%% TODO: use intervals:new('[', A, B, ')') instead so ?MINUS_INFINITY is in quadrant 1?\n%%       -> does not fit ranges that well as they are normally defined as (A,B]\n-spec quadrant_intervals_(Borders::[?RT:key(),...], ResultIn::[intervals:non_empty_interval()],\n                          HeadB::?RT:key()) -> [intervals:non_empty_interval(),...].\nquadrant_intervals_([K], Res, HB) ->\n    lists:reverse(Res, [intervals:new('(', K, HB, ']')]);\nquadrant_intervals_([A | [B | _] = TL], Res, HB) ->\n    quadrant_intervals_(TL, [intervals:new('(', A, B, ']') | Res], HB).\n\n%% @doc Gets all sub intervals of the given interval which lay only in\n%%      a single quadrant.\n-spec quadrant_subints_(A::intervals:interval(), Quadrants::[intervals:interval()],\n                        AccIn::[intervals:interval()]) -> AccOut::[intervals:interval()].\nquadrant_subints_(_A, [], Acc) -> Acc;\nquadrant_subints_(A, [Q | QT], Acc) ->\n    Sec = intervals:intersection(A, Q),\n    case intervals:is_empty(Sec) of\n        false when Sec =:= Q ->\n            % if a quadrant is completely covered, only return this\n            % -> this would reconcile all the other keys, too!\n            % it also excludes non-continuous intervals\n            [Q];\n        false -> quadrant_subints_(A, QT, [Sec | Acc]);\n        true  -> quadrant_subints_(A, QT, Acc)\n    end.\n\n%% @doc Gets all replicated intervals of I.\n%%      PreCond: interval (I) is continuous and is inside a single quadrant!\n-spec replicated_intervals(intervals:continuous_interval())\n        -> [intervals:continuous_interval()].\nreplicated_intervals(I) ->\n    ?DBG_ASSERT(intervals:is_continuous(I)),\n    ?DBG_ASSERT(1 =:= length([ok || Q <- quadrant_intervals(),\n                                not intervals:is_empty(\n                                  intervals:intersection(I, Q))])),\n    case intervals:is_all(I) of\n        false ->\n            case intervals:get_bounds(I) of\n                {_LBr, ?MINUS_INFINITY, ?PLUS_INFINITY, _RBr} ->\n                    [I]; % this is the only interval possible!\n                {'[', Key, Key, ']'} ->\n                    [intervals:new(X) || X <- ?RT:get_replica_keys(Key)];\n                {LBr, LKey, RKey0, RBr} ->\n                    LKeys = lists:sort(?RT:get_replica_keys(LKey)),\n                    % note: get_bounds may also return ?PLUS_INFINITY but this is not a valid key in ?RT!\n                    RKey = ?IIF(RKey0 =:= ?PLUS_INFINITY, ?MINUS_INFINITY, RKey0),\n                    RKeys = case lists:sort(?RT:get_replica_keys(RKey)) of\n                                [?MINUS_INFINITY | RKeysTL] ->\n                                    lists:append(RKeysTL, [?MINUS_INFINITY]);\n                                X -> X\n                            end,\n                    % since I is in a single quadrant, RKey >= LKey\n                    % -> we can zip the sorted keys to get the replicated intervals\n                    lists:zipwith(\n                      fun(LKeyX, RKeyX) ->\n                              % this debug statement only holds for replication factors that are a power of 2:\n%%                               ?DBG_ASSERT(?RT:get_range(LKeyX, ?IIF(RKeyX =:= ?MINUS_INFINITY, ?PLUS_INFINITY, RKeyX)) =:=\n%%                                           ?RT:get_range(LKey, RKey0)),\n                              intervals:new(LBr, LKeyX, RKeyX, RBr)\n                      end, LKeys, RKeys)\n            end;\n        true -> [I]\n    end.\n\n%% @doc Gets a randomly selected sync interval as an intersection of the two\n%%      given intervals as a sub interval of A inside a single quadrant.\n%%      Result may be empty, otherwise it is also continuous!\n-spec find_sync_interval(intervals:continuous_interval(), intervals:continuous_interval())\n        -> intervals:interval().\nfind_sync_interval(A, B) ->\n    ?DBG_ASSERT(intervals:is_continuous(A)),\n    ?DBG_ASSERT(intervals:is_continuous(B)),\n    Quadrants = quadrant_intervals(),\n    InterSecs = [I || AQ <- quadrant_subints_(A, Quadrants, []),\n                      BQ <- quadrant_subints_(B, Quadrants, []),\n                      RBQ <- replicated_intervals(BQ),\n                      not intervals:is_empty(\n                        I = intervals:intersection(AQ, RBQ))],\n    case InterSecs of\n        [] -> intervals:empty();\n        [_|_] -> util:randomelem(InterSecs)\n    end.\n\n%% @doc Maps interval B into interval A.\n%%      PreCond: the second (continuous) interval must be in a single quadrant!\n%%      The result is thus also only in a single quadrant.\n%%      Result may be empty, otherwise it is also continuous!\n-spec map_interval(intervals:continuous_interval(), intervals:continuous_interval())\n        -> intervals:interval().\nmap_interval(A, B) ->\n    ?DBG_ASSERT(intervals:is_continuous(A)),\n    ?DBG_ASSERT(intervals:is_continuous(B)),\n    ?DBG_ASSERT(1 =:= length([ok || Q <- quadrant_intervals(),\n                                not intervals:is_empty(\n                                  intervals:intersection(B, Q))])),\n    \n    % note: The intersection may only be non-continuous if A is covering more\n    %       than a quadrant. In this case, another intersection will be larger\n    %       and we can safely ignore this one!\n    InterSecs = [I || RB <- replicated_intervals(B),\n                      not intervals:is_empty(\n                        I = intervals:intersection(A, RB)),\n                      intervals:is_continuous(I)],\n    case InterSecs of\n        [] -> intervals:empty();\n        [_|_] -> util:randomelem(InterSecs)\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% STARTUP\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc init module\n-spec init(state()) -> state().\ninit(State) ->\n    _ = gen_component:monitor(State#rr_recon_state.ownerPid),\n    State.\n\n-spec start(SessionId::rrepair:session_id(), SenderRRPid::comm:mypid())\n        -> {ok, pid()}.\nstart(SessionId, SenderRRPid) ->\n    State = #rr_recon_state{ ownerPid = self(),\n                             dest_rr_pid = SenderRRPid,\n                             stats = rr_recon_stats:new(SessionId) },\n    PidName = lists:flatten(io_lib:format(\"~s_~p.~s\", [?MODULE, SessionId, randoms:getRandomString()])),\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, State,\n                             [{pid_groups_join_as, pid_groups:my_groupname(),\n                               {short_lived, PidName}}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Config parameter handling\n%\n% rr_recon_failure_rate     - expected number of failures during the\n%                             (approximate) reconciliation process\n% rr_art_inner_fpr          - \n% rr_art_leaf_fpr           -  \n% rr_art_correction_factor  - \n% rr_merkle_branch_factor   - merkle tree branching factor thus number of childs per node\n% rr_merkle_bucket_size     - size of merkle tree leaf buckets\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Checks whether a config parameter is float and in [0,1].\n-spec check_percent(atom()) -> boolean().\ncheck_percent(Atom) ->\n    config:cfg_is_float(Atom) andalso\n        config:cfg_is_greater_than(Atom, 0) andalso\n        config:cfg_is_less_than(Atom, 1).\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n     % deprecated, old, and unused config parameter\n    case config:read(rr_recon_p1e) of\n        failed -> true;\n        _ -> error_logger:error_msg(\"rr_recon_p1e parameter exists - \"\n                                    \"please migrate to rr_recon_failure_rate \"\n                                    \"(see scalaris.cfg and scalaris.local.cfg)~n\"),\n             false\n    end andalso\n        config:cfg_is_in(rr_recon_method, [trivial, shash, bloom, merkle_tree, art]) andalso\n        config:cfg_is_number(rr_recon_failure_rate) andalso\n        config:cfg_is_greater_than(rr_recon_failure_rate, 0) andalso\n        config:cfg_is_number(rr_recon_expected_delta) andalso\n        config:cfg_is_in_range(rr_recon_expected_delta, 0, 100) andalso\n        config:cfg_is_integer(rr_recon_version_bits) andalso\n        config:cfg_is_greater_than(rr_recon_version_bits, 0) andalso\n        config:cfg_test_and_error(rr_max_items,\n                                  fun(all) -> true;\n                                     (X) -> erlang:is_integer(X) andalso X > 0\n                                  end, \"is not 'all' or an integer > 0\"),\n        config:cfg_is_integer(rr_recon_min_sig_size) andalso\n        config:cfg_is_greater_than(rr_recon_min_sig_size, 0) andalso\n        config:cfg_is_integer(rr_merkle_branch_factor) andalso\n        config:cfg_is_greater_than(rr_merkle_branch_factor, 1) andalso\n        config:cfg_is_integer(rr_merkle_bucket_size) andalso\n        config:cfg_is_greater_than(rr_merkle_bucket_size, 0) andalso\n        config:cfg_is_integer(rr_merkle_num_trees) andalso\n        config:cfg_is_greater_than(rr_merkle_num_trees, 0) andalso\n        check_percent(rr_art_inner_fpr) andalso\n        check_percent(rr_art_leaf_fpr) andalso\n        config:cfg_is_integer(rr_art_correction_factor) andalso\n        config:cfg_is_greater_than(rr_art_correction_factor, 0).\n\n-spec get_failure_rate() -> float().\nget_failure_rate() ->\n    config:read(rr_recon_failure_rate).\n\n%% @doc Specifies what the maximum expected delta is (in percent between 0 and\n%%      100, inclusive). The failure probabilities will take this into account.\n-spec get_max_expected_delta() -> number().\nget_max_expected_delta() ->\n    config:read(rr_recon_expected_delta).\n\n%% @doc Use at least these many bits for compressed version numbers.\n-spec get_min_version_bits() -> pos_integer().\nget_min_version_bits() ->\n    config:read(rr_recon_version_bits).\n\n%% @doc Use at least these many bits for hashes.\n-spec get_min_hash_bits() -> pos_integer().\nget_min_hash_bits() ->\n    config:read(rr_recon_min_sig_size).\n\n%% @doc Specifies how many items to retrieve from the DB at once.\n%%      Tries to reduce the load of a single request in the dht_node process.\n-spec get_max_items() -> pos_integer() | all.\nget_max_items() ->\n    config:read(rr_max_items).\n\n%% @doc Merkle number of childs per inner node.\n-spec get_merkle_branch_factor() -> pos_integer().\nget_merkle_branch_factor() ->\n    config:read(rr_merkle_branch_factor).\n\n%% @doc Merkle number of childs per inner node.\n-spec get_merkle_num_trees() -> pos_integer().\nget_merkle_num_trees() ->\n    config:read(rr_merkle_num_trees).\n\n%% @doc Merkle max items in a leaf node.\n-spec get_merkle_bucket_size() -> pos_integer().\nget_merkle_bucket_size() ->\n    config:read(rr_merkle_bucket_size).\n\n-spec get_art_config() -> art:config().\nget_art_config() ->\n    [{correction_factor, config:read(rr_art_correction_factor)},\n     {inner_bf_fpr, config:read(rr_art_inner_fpr)},\n     {leaf_bf_fpr, config:read(rr_art_leaf_fpr)}].\n\n-spec tester_create_kvi_tree(\n        [{KeyShort::non_neg_integer(),\n          {VersionShort::non_neg_integer(), Idx::non_neg_integer()}}]) -> kvi_tree().\ntester_create_kvi_tree(KVList) ->\n    mymaps:from_list(KVList).\n\n-spec tester_is_kvi_tree(Map::any()) -> boolean().\ntester_is_kvi_tree(Map) ->\n    try mymaps:to_list(Map) of\n        KVList -> lists:all(fun({K, {V, Idx}}) when is_integer(K) andalso K >= 0\n                                 andalso is_integer(V) andalso V >= 0\n                                 andalso is_integer(Idx) andalso Idx >= 0 ->\n                                    true;\n                               ({_, _}) ->\n                                    false\n                            end, KVList)\n    catch _:_ -> false % probably no map\n    end.\n"
  },
  {
    "path": "src/rrepair/rr_recon_stats.erl",
    "content": "% @copyright 2011-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Replica Repair Reconciliation Statistics\n%% @end\n%% @version $Id$\n-module(rr_recon_stats).\n-author('malange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Exported functions and types\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-export([new/1, new/2, inc/2, set/2, get/2, print/1]).\n\n-export_type([stats/0, status/0]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Types\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-type status() :: wait | abort | finish.\n-record(rr_recon_stats,\n        {\n         session_id         = ?required(rr_recon_stats, session_id) :: rrepair:session_id(),\n         tree_size          = {0,0,0,0} :: merkle_tree:mt_size(),\n         tree_nodesCompared = 0       :: non_neg_integer(),\n         tree_compareSkipped= 0       :: non_neg_integer(),\n         tree_leavesSynced  = 0       :: non_neg_integer(),\n         fail_rate_p1       = 0.0     :: float(),\n         fail_rate_p2       = 0.0     :: float(),\n         build_time         = 0       :: non_neg_integer(),      %in us\n         recon_time         = 0       :: non_neg_integer(),      %in us\n         rs_expected        = 0       :: non_neg_integer(),      %number of resolve expected requests\n         status             = wait    :: status()\n         }).\n-type stats() :: #rr_recon_stats{}.\n\n-type field_list1()  ::\n          [{tree_size, merkle_tree:mt_size()} |\n               {tree_nodesCompared, non_neg_integer()} |\n               {tree_compareSkipped, non_neg_integer()} |\n               {tree_leavesSynced, non_neg_integer()} |\n               {build_time, non_neg_integer()} |\n               {recon_time, non_neg_integer()} |\n               {rs_expected, non_neg_integer()}].\n\n-type field_list2()  ::\n          [{status, status()} | {fail_rate_p1, float()} | {fail_rate_p2, float()}]\n            | field_list1().\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% API Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec new(rrepair:session_id()) -> stats().\nnew(SID) ->\n    ?DBG_ASSERT(SID =/= null),\n    #rr_recon_stats{session_id = SID}.\n\n-spec new(rrepair:session_id(), field_list2()) -> stats().\nnew(SID, KVList) ->\n    set(KVList, #rr_recon_stats{session_id = SID}).\n\n% @doc increases the record field with name key by value\n-spec inc(field_list1(), Old::stats()) -> New::stats().\ninc([], Stats) ->\n    Stats;\ninc([{K, V} | L], Stats) ->\n    NS = case K of\n             tree_size ->\n                 {OldIn, OldLn, OldELn, OldIt} = Stats#rr_recon_stats.tree_size,\n                 {IncIn, IncLn, IncELn, IncIt} = V,\n                 X = {OldIn + IncIn, OldLn + IncLn, OldELn + IncELn, OldIt + IncIt},\n                 Stats#rr_recon_stats{tree_size = X};\n             tree_nodesCompared ->\n                 X = V + Stats#rr_recon_stats.tree_nodesCompared,\n                 Stats#rr_recon_stats {tree_nodesCompared = X};\n             tree_leavesSynced ->\n                 X = V + Stats#rr_recon_stats.tree_leavesSynced,\n                 Stats#rr_recon_stats{tree_leavesSynced = X};\n             tree_compareSkipped ->\n                 X = V + Stats#rr_recon_stats.tree_compareSkipped,\n                 Stats#rr_recon_stats{tree_compareSkipped = X};\n             build_time ->\n                 X = V + Stats#rr_recon_stats.build_time,\n                 Stats#rr_recon_stats{build_time = X};\n             recon_time ->\n                 X = V + Stats#rr_recon_stats.recon_time,\n                 Stats#rr_recon_stats{recon_time = X};\n             rs_expected ->\n                 X = V + Stats#rr_recon_stats.rs_expected,\n                 Stats#rr_recon_stats{rs_expected = X}\n         end,\n    inc(L, NS).\n\n% @doc sets the value of record field with name of key to the given value\n-spec set(field_list2(), Old::stats()) -> New::stats().\nset([], Stats) ->\n    Stats;\nset([{K, V} | L], Stats) ->\n    NS = case K of\n             tree_size           -> Stats#rr_recon_stats{tree_size = V};\n             tree_nodesCompared  -> Stats#rr_recon_stats{tree_nodesCompared = V};\n             tree_leavesSynced   -> Stats#rr_recon_stats{tree_leavesSynced = V};\n             tree_compareSkipped -> Stats#rr_recon_stats{tree_compareSkipped = V};\n             fail_rate_p1        -> Stats#rr_recon_stats{fail_rate_p1 = V};\n             fail_rate_p2        -> Stats#rr_recon_stats{fail_rate_p2 = V};\n             build_time          -> Stats#rr_recon_stats{build_time = V};\n             recon_time          -> Stats#rr_recon_stats{recon_time = V};\n             rs_expected         -> Stats#rr_recon_stats{rs_expected = V};\n             status              -> Stats#rr_recon_stats{status = V}\n         end,\n    set(L, NS).\n\n-spec get(session_id, stats())         -> rrepair:session_id();\n         (tree_size, stats())          -> merkle_tree:mt_size();\n         (tree_nodesCompared, stats()) -> non_neg_integer();\n         (tree_compareSkipped, stats())-> non_neg_integer();\n         (tree_leavesSynced, stats())  -> non_neg_integer();\n         (fail_rate_p1 | fail_rate_p2 | fail_rate, stats()) -> float();\n         (build_time, stats())         -> non_neg_integer();\n         (recon_time, stats())         -> non_neg_integer();\n         (rs_expected, stats())        -> non_neg_integer();\n         (status, stats())             -> status().\nget(session_id         , #rr_recon_stats{session_id          = X}) -> X;\nget(tree_size          , #rr_recon_stats{tree_size           = X}) -> X;\nget(tree_nodesCompared , #rr_recon_stats{tree_nodesCompared  = X}) -> X;\nget(tree_leavesSynced  , #rr_recon_stats{tree_leavesSynced   = X}) -> X;\nget(tree_compareSkipped, #rr_recon_stats{tree_compareSkipped = X}) -> X;\nget(fail_rate_p1       , #rr_recon_stats{fail_rate_p1        = X}) -> X;\nget(fail_rate_p2       , #rr_recon_stats{fail_rate_p2        = X}) -> X;\nget(build_time         , #rr_recon_stats{build_time          = X}) -> X;\nget(recon_time         , #rr_recon_stats{recon_time          = X}) -> X;\nget(rs_expected        , #rr_recon_stats{rs_expected         = X}) -> X;\nget(status             , #rr_recon_stats{status              = X}) -> X;\nget(fail_rate          , #rr_recon_stats{fail_rate_p1 = Fr_p1, fail_rate_p2 = Fr_p2}) ->\n    Fr_p1 + Fr_p2.\n\n-spec print(stats()) -> [any()].\nprint(Stats) ->\n    StatsL = tl(erlang:tuple_to_list(Stats)),\n    FieldNames = record_info(fields, rr_recon_stats),\n    [rr_recon_stats, lists:zip(FieldNames, StatsL)].\n"
  },
  {
    "path": "src/rrepair/rr_resolve.erl",
    "content": "% @copyright 2011-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable request_resolvelaw or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @doc    replica update resolve module\n%%         Updates local and/or remote Key-Value-Pairs (kv-pair)\n%%         Modes:\n%%           1) key_upd: updates local db entries with received kvv-list, if received kv is newer\n%%           2) key_upd_send: creates kvv-list out of a given key-list and sends it to dest\n%%           3) interval_upd_my: tries to resolve items from the given interval by\n%%                               requesting data from replica nodes\n%%         Options:\n%%           1) Feedback: sends data ids to Node (A) which are outdated at (A)\n%%           2) Send_Stats: sends resolution stats to given pid\n%%         Usage:\n%%           rrepair process provides API for resolve requests\n%% @end\n%% @version $Id$\n-module(rr_resolve).\n-author('malange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"record_helpers.hrl\").\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([init/1, on/2, start/1]).\n-export([get_stats_session_id/1, merge_stats/2]).\n-export([print_resolve_stats/1]).\n\n% for tester:\n-export([merge_stats_feeder/2]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% type definitions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-export_type([operation/0, options/0, kvv_list/0]).\n-export_type([stats/0]).\n\n-type option()   :: {feedback_request, comm:mypid()} |\n                    {from_my_node, 0 | 1}.\n-type options()  :: [option()].\n-type kvv_list() :: [{?RT:key(), db_dht:value(), client_version()}].\n-type exit_reason() :: resolve_ok | resolve_abort.\n\n-record(resolve_stats,\n        {\n         session_id       = ?required(resolve_stats, session_id) :: rrepair:session_id() | null,\n         diff_size        = 0      :: non_neg_integer(),\n         regen_count      = 0      :: non_neg_integer(),\n         update_count     = 0      :: non_neg_integer(),\n         upd_fail_count   = 0      :: non_neg_integer(),\n         regen_fail_count = 0      :: non_neg_integer()\n         }).\n-type stats() :: #resolve_stats{}.\n\n-type operation() ::\n    {?key_upd, KvvListInAnyQ::kvv_list(), ReqKeys::[?RT:key()]} |\n    {key_upd_send, DestPid::comm:mypid(), SendKeys::[?RT:key()], ReqKeys::[?RT:key()]} |\n    {interval_upd_my, intervals:interval()}.\n\n-record(rr_resolve_state,\n        {\n         ownerPid       = ?required(rr_resolve_state, ownerPid)   :: pid(),\n         operation      = undefined                               :: undefined | operation(),\n         my_range       = undefined                               :: undefined | intervals:interval(),\n         fb_dest_pid    = undefined                               :: undefined | comm:mypid(),\n         fb_send_kvv    = []                                      :: OutdatedOnOther::kvv_list(),\n         fb_had_kvv_req = false                                   :: NonEmptyReqList::boolean(),\n         fb_send_kvv_req= []                                      :: RequestedByOther::kvv_list(),\n         other_kv_tree  = mymaps:new()                            :: MyIOtherKvTree::mymaps:mymap(), % ?RT:key() => client_version()\n         stats          = ?required(rr_resolve_state, stats)      :: stats(),\n         from_my_node   = 1                                       :: 0 | 1\n         }).\n-type state() :: #rr_resolve_state{}.\n\n-type message() ::\n    % API\n    {start, operation(), options(), StartTag::atom()} |\n    % internal\n    {get_entries_response, db_dht:db_as_list()} |\n    {get_state_response, intervals:interval()} |\n    {update_key_entries_ack, [{db_entry:entry_ex(), Exists::boolean(), Done::boolean()}]} |\n    {'DOWN', MonitorRef::reference(), process, Owner::pid(), Info::any()}.\n\n-include(\"gen_component.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% debug\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-define(TRACE(X,Y), ok).\n%-define(TRACE(X,Y), log:pal(\"~w [~p:~p] \" ++ X ++ \"~n\", [?MODULE, pid_groups:my_groupname(), self()] ++ Y)).\n-define(TRACE_START_END(X,Y), ok).\n%-define(TRACE_START_END(X,Y), log:pal(\"~w [~p:~p] \" ++ X ++ \"~n\", [?MODULE, pid_groups:my_groupname(), self()] ++ Y)).\n-define(TRACE_SEND(Pid, Msg), ?TRACE(\"to ~p:~.0p: ~.0p~n\", [pid_groups:group_of(comm:make_local(comm:get_plain_pid(Pid))), Pid, Msg])).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Message handling\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec on(message(), state()) -> state() | kill.\n\non({start, Operation, Options, _StartCont}, State) ->\n    FBDest = proplists:get_value(feedback_request, Options, undefined),\n    FromMyNode = proplists:get_value(from_my_node, Options, 1),\n    NewState = State#rr_resolve_state{ operation = Operation,\n                                       fb_dest_pid = FBDest,\n                                       from_my_node = FromMyNode },\n    ?TRACE_START_END(\"RESOLVE ~s - Operation=~p~n FeedbackTo=~p~n SessionId:~p (MyNode: ~p)\",\n           [_StartCont, util:extint2atom(element(1, Operation)), FBDest,\n            NewState#rr_resolve_state.stats#resolve_stats.session_id,\n            FromMyNode]),\n    send_local(pid_groups:get_my(dht_node), {get_state, comm:this(), my_range}),\n    NewState;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% MODE: key_upd\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\non({get_state_response, MyI}, State =\n       #rr_resolve_state{ operation = {?key_upd, KvvList, ReqKeys},\n                          stats = _Stats }) ->\n    MyIOtherKvvList = map_kvv_list(KvvList, MyI),\n    ?TRACE(\"GET INTERVAL - Operation=~p~n SessionId:~p~n MyInterval=~p~n KVVListLen=~p\",\n           [key_upd, _Stats#resolve_stats.session_id, MyI, length(KvvList)]),\n\n    % send requested entries (similar to key_upd_send handling)\n    RepKeyInt = intervals:from_elements(map_key_list(ReqKeys, MyI)),\n    send_local(pid_groups:get_my(dht_node), {get_entries, self(), RepKeyInt}),\n\n    % send entries in sender interval but not in sent KvvList\n    % convert keys KvvList to a map for faster access checks\n    MyIOtherKvTree = make_other_kv_tree(MyIOtherKvvList),\n\n    % allow the garbage collection to clean up the ReqKeys here:\n    % also update the KvvList\n    State#rr_resolve_state{operation = {?key_upd, MyIOtherKvvList, []},\n                           other_kv_tree = MyIOtherKvTree,\n                           fb_had_kvv_req = ReqKeys =/= []};\n\non({get_entries_response, EntryList}, State =\n       #rr_resolve_state{ operation = {?key_upd, MyIOtherKvvList, []},\n                          fb_send_kvv_req = [],\n                          stats = Stats}) ->\n    % note: EntryList may not be unique! - it is made unique in shutdown/2 though\n    KvvList = [entry_to_kvv(E) || E <- EntryList],\n    ToUpdate = start_update_key_entries(MyIOtherKvvList, comm:this(),\n                                        pid_groups:get_my(dht_node)),\n    ?TRACE(\"GET ENTRIES - Operation=~p~n SessionId:~p ; ToUpdate=~p - #Items: ~p\",\n           [key_upd, Stats#resolve_stats.session_id, ToUpdate, length(EntryList)]),\n\n    % allow the garbage collection to clean up the KvvList here:\n    NewState =\n        State#rr_resolve_state{operation = {?key_upd, [], []},\n                               stats = Stats#resolve_stats{diff_size = ToUpdate},\n                               fb_send_kvv_req = KvvList},\n\n    if ToUpdate =:= 0 ->\n           % use the same options as above in get_state_response:\n           shutdown(resolve_ok, NewState);\n       true ->\n           % note: shutdown and feedback handled by update_key_entries_ack\n           NewState\n    end;\n\non({get_state_response, MyI}, State =\n       #rr_resolve_state{ operation = {key_upd_send, _Dest, SendKeys, _ReqKeys},\n                          stats = _Stats }) ->\n    ?TRACE(\"GET INTERVAL - Operation=~p~n SessionId:~p~n MyInterval=~p\",\n           [key_upd_send, _Stats#resolve_stats.session_id, MyI]),\n    SendKeysMappedInterval = intervals:from_elements(map_key_list(SendKeys, MyI)),\n    send_local(pid_groups:get_my(dht_node), {get_entries, self(), SendKeysMappedInterval}),\n    State;\n\non({get_entries_response, EntryList}, State =\n       #rr_resolve_state{ operation = {key_upd_send, Dest, _, ReqKeys},\n                          fb_dest_pid = FBDest, from_my_node = FromMyNode,\n                          stats = Stats }) ->\n    SID = Stats#resolve_stats.session_id,\n    ?TRACE(\"GET ENTRIES - Operation=~p~n SessionId:~p - #Items: ~p\",\n           [key_upd_send, SID, length(EntryList)]),\n\n    % note: EntryList may not be unique!\n    KvvList = make_unique_kvv([entry_to_kvv(E) || E <- EntryList]),\n    ?DBG_ASSERT2(length(ReqKeys) =:= length(lists:usort(ReqKeys)),\n                 {non_unique_req_list, ReqKeys}),\n    ?TRACE_START_END(\"sending key_upd with ~B items and ~B requests\",\n                     [length(KvvList), length(ReqKeys)]),\n    send_request_resolve(Dest, {?key_upd, KvvList, ReqKeys}, SID,\n                         FromMyNode, FBDest, []),\n    NewState = State#rr_resolve_state{fb_dest_pid = undefined},\n    shutdown(resolve_ok, NewState);\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% MODE: interval_upd_my\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\non({get_state_response, MyI} = _Msg,\n   State = #rr_resolve_state{operation = {interval_upd_my, I}}) ->\n    ?TRACE(\"GET INTERVAL - Operation=~p~n IntervalBounds=~p~n MyInterval=~p\",\n           [interval_upd_my, intervals:get_bounds(I), MyI]),\n    ?DBG_ASSERT(State#rr_resolve_state.fb_dest_pid =:= undefined),\n    ISec = intervals:intersection(MyI, I),\n    NewState = State#rr_resolve_state{ my_range = MyI },\n    case intervals:is_empty(ISec) of\n        false -> case rrepair:select_sync_node(ISec, true) of\n                     not_found ->\n                         shutdown(resolve_abort, NewState);\n                     DKey ->\n                         % TODO: keep trying to resolve the whole intersection\n                         %       e.g. by removing each sync interval and\n                         %       continuing with the rest until the whole\n                         %       interval is covered (at each step check with\n                         %       the range reported from the dht_node!)\n                         % -> the current implementation only tries once!\n                         % note: bloom and art may not fully re-generate the\n                         %       own range -> choose merkle_tree instead\n                         send_local(pid_groups:get_my(rrepair),\n                                    {request_sync, merkle_tree, DKey}),\n                         shutdown(resolve_ok, NewState)\n                 end;\n        true  -> shutdown(resolve_abort, NewState)\n    end;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\non({update_key_entries_ack, NewEntryList}, State =\n       #rr_resolve_state{ operation = Op,\n                          stats = #resolve_stats{ diff_size = _Diff,\n                                                  regen_count = RegenOk,\n                                                  update_count = UpdOk,\n                                                  upd_fail_count = UpdFail,\n                                                  regen_fail_count = RegenFail\n                                                } = Stats,\n                          fb_dest_pid = FBDest, fb_send_kvv = MissingOnOther,\n                          other_kv_tree = MyIOtherKvTree\n                        })\n  when element(1, Op) =:= ?key_upd ->\n    ?TRACE(\"GET ENTRY_ACK - Operation=~p~n SessionId:~p - #NewItems: ~p\",\n           [util:extint2atom(element(1, Op)), Stats#resolve_stats.session_id,\n            length(NewEntryList)]),\n\n    {NewUpdOk, NewUpdFail, NewRegenOk, NewRegenFail, NewFBItems} =\n        integrate_update_key_entries_ack(\n          NewEntryList, UpdOk, UpdFail, RegenOk, RegenFail, MissingOnOther, MyIOtherKvTree,\n          FBDest =/= undefined),\n\n    NewStats = Stats#resolve_stats{update_count     = NewUpdOk,\n                                   regen_count      = NewRegenOk,\n                                   upd_fail_count   = NewUpdFail,\n                                   regen_fail_count = NewRegenFail},\n    NewState = State#rr_resolve_state{stats = NewStats, fb_send_kvv = NewFBItems},\n    ?DBG_ASSERT(_Diff =:= (NewRegenOk + NewUpdOk + NewUpdFail + NewRegenFail)),\n    ?TRACE(\"UPDATED = ~p - Regen=~p\", [NewUpdOk, NewRegenOk]),\n    shutdown(resolve_ok, NewState);\n\non({'DOWN', MonitorRef, process, _Owner, _Info}, _State) ->\n    log:log(info, \"[ ~p - ~p] shutdown due to rrepair shut down\", [?MODULE, comm:this()]),\n    gen_component:demonitor(MonitorRef),\n    kill.\n\n%% @doc Maps the given tuple list (with keys as first elements) to the MyI\n%%      interval.\n%%      Precond: tuple list with unique keys\n%%      Note: mapped tuples are not unique if MyI covers multiple replicas!\n-spec map_kvv_list([Tpl], MyI::intervals:interval()) -> [Tpl]\n        when is_subtype(Tpl, {?RT:key()} |\n                             {?RT:key(), term()} |\n                             {?RT:key(), term(), term()}).\nmap_kvv_list(TplList, MyI) ->\n    ?DBG_ASSERT(length(TplList) =:= length(lists:ukeysort(1, TplList))),\n    [setelement(1, E, RKey) || E <- TplList,\n                               RKey <- ?RT:get_replica_keys(element(1, E)),\n                               intervals:in(RKey, MyI)].\n\n%% @doc Maps the given (unique!) key list to the MyI interval.\n%%      Note: mapped keys are not unique if MyI covers multiple replicas!\n-spec map_key_list([?RT:key()], MyI::intervals:interval()) -> [?RT:key()].\nmap_key_list(KeyList, MyI) ->\n    ?DBG_ASSERT(length(KeyList) =:= length(lists:usort(KeyList))),\n    [RKey || Key <- KeyList,\n             RKey <- ?RT:get_replica_keys(Key),\n             intervals:in(RKey, MyI)].\n\n%% @doc Starts updating the local entries with the given KvvList.\n%%      -> Returns number of send update requests.\n%%      PreCond: KvvList contains only unique keys\n-spec start_update_key_entries(MyIOtherKvvList::kvv_list(), comm:mypid(),\n                               comm:erl_local_pid()) -> UpdRequests::non_neg_integer().\nstart_update_key_entries([], _MyPid, _DhtPid) -> 0;\nstart_update_key_entries(MyIOtherKvvList, MyPid, DhtPid) ->\n    send_local(DhtPid, {update_key_entries, MyPid, MyIOtherKvvList}),\n    length(MyIOtherKvvList).\n\n-spec integrate_update_key_entries_ack(\n        [{Entry::db_entry:entry_ex(), Exists::boolean(), Done::boolean()}],\n        UpdOk::non_neg_integer(), UpdFail::non_neg_integer(),\n        RegenOk::non_neg_integer(), RegenFail::non_neg_integer(),\n        FBItems::kvv_list(), OtherKvTree::mymaps:mymap(), FBOn::boolean())\n        -> {UpdOk::non_neg_integer(), UpdFail::non_neg_integer(),\n            RegenOk::non_neg_integer(), RegenFail::non_neg_integer(),\n            FBItems::kvv_list()}.\nintegrate_update_key_entries_ack([], UpdOk, UpdFail, RegenOk, RegenFail, FBItems,\n                               _OtherKvTree, _FBOn) ->\n    {UpdOk, UpdFail, RegenOk, RegenFail, FBItems};\nintegrate_update_key_entries_ack([{Entry, Exists, Done} | Rest], UpdOk, UpdFail,\n                               RegenOk, RegenFail, FBItems, OtherKvTree, FBOn) ->\n    NewFBItems =\n        if not Done andalso Exists andalso FBOn ->\n               case any_replica_in_tree(?RT:get_replica_keys(db_entry:get_key(Entry)), OtherKvTree) of\n                   none -> [entry_to_kvv(Entry) | FBItems];\n                   {value, OtherVersion} ->\n                       MyVersion = db_entry:get_version(Entry),\n                       if MyVersion > OtherVersion ->\n                              [entry_to_kvv(Entry) | FBItems];\n                          true ->\n                              FBItems\n                       end\n               end;\n           true -> FBItems\n        end,\n    if Done andalso Exists ->\n           integrate_update_key_entries_ack(\n             Rest, UpdOk + 1, UpdFail, RegenOk, RegenFail, NewFBItems, OtherKvTree, FBOn);\n       Done andalso not Exists ->\n           integrate_update_key_entries_ack(\n             Rest, UpdOk, UpdFail, RegenOk + 1, RegenFail, NewFBItems, OtherKvTree, FBOn);\n       not Done and Exists ->\n           integrate_update_key_entries_ack(\n             Rest, UpdOk, UpdFail + 1, RegenOk, RegenFail, NewFBItems, OtherKvTree, FBOn);\n       not Done and not Exists ->\n           integrate_update_key_entries_ack(\n             Rest, UpdOk, UpdFail, RegenOk, RegenFail + 1, NewFBItems, OtherKvTree, FBOn)\n    end.\n\n%% @doc Tries to find any key from a list in the tree.\n-spec any_replica_in_tree(RKeys::[Key], Tree::mymaps:mymap())\n        -> none | {value, Val} when is_subtype(Key, any()), is_subtype(Val, any()).\nany_replica_in_tree([], _Tree) ->\n    none;\nany_replica_in_tree([Key | Rest], Tree) ->\n    case mymaps:find(Key, Tree) of\n        error -> any_replica_in_tree(Rest, Tree);\n        {ok, X} -> {value, X}\n    end.\n\n-spec shutdown(exit_reason(), state()) -> kill.\nshutdown(_Reason, #rr_resolve_state{ownerPid = Owner,\n                                    stats = Stats,\n                                    operation = _Op, fb_dest_pid = FBDest,\n                                    fb_send_kvv = FbKVV,\n                                    fb_had_kvv_req = SendReqKeyReply,\n                                    fb_send_kvv_req = FbReqKVV,\n                                    from_my_node = FromMyNode} = _State) ->\n    ?TRACE_START_END(\"SHUTDOWN ~p - Operation=~p~n SessionId:~p (MyNode: ~p)~n\"\n                     \" feedback: ~.2p\",\n           [_Reason, util:extint2atom(element(1, _Op)), Stats#resolve_stats.session_id,\n            FromMyNode, ?IIF(FBDest =/= undefined,\n                             {length(FbKVV), \"items to\", FBDest, \"via key_upd:\", FbKVV},\n                             none)]),\n        case FBDest of\n            undefined ->\n                ok;\n            _ when not SendReqKeyReply ->\n                FbKVV1 = make_unique_kvv(FbKVV),\n                send_request_resolve(FBDest, {?key_upd, FbKVV1, []},\n                                     Stats#resolve_stats.session_id,\n                                     FromMyNode, undefined, []);\n            _ ->\n                % feedback item lists may not be unique -> make them!\n                FbKVV1 = make_unique_kvv(FbKVV),\n                FbReqKVV1 = make_unique_kvv(FbReqKVV),\n                send_request_resolve(FBDest, {?key_upd, FbKVV1, []},\n                                     Stats#resolve_stats.session_id,\n                                     FromMyNode, undefined, []),\n                send_request_resolve(FBDest, {?key_upd, FbReqKVV1, []},\n                                     Stats#resolve_stats.session_id,\n                                     FromMyNode, comm:make_global(Owner), [])\n        end,\n    % note: do not propagate the SessionId unless we report to the node\n    %       the request came from (indicated by FromMyNode =:= 1),\n    %       otherwise the resolve_progress_report on the other node will\n    %       be counted for the session's rs_finish and it will not match\n    %       its rs_expected anymore!\n    if FromMyNode =:= 1 ->\n           send_local(Owner, {resolve_progress_report, self(), Stats});\n       true ->\n           send_local(Owner, {resolve_progress_report, self(),\n                              Stats#resolve_stats{session_id = null}})\n    end,\n    kill.\n\n%% @doc Makes the given tuple list (with keys as first elements) unique, i.e.\n%%      there are no duplicate replicas of keys in the result tuple list.\n%%      Note: Assumes, KVs at the same node have the same version and chooses\n%%            an arbitrary key for each replica group.\n-spec make_unique_kvv([Tpl]) -> [Tpl]\n        when is_subtype(Tpl, {?RT:key()} |\n                             {?RT:key(), term()} |\n                             {?RT:key(), term(), term()}).\nmake_unique_kvv([]) -> [];\nmake_unique_kvv([_|_] = KVV) ->\n    MKVV = [{rr_recon:map_key_to_quadrant(element(1, Val), 1), Val} || Val <- KVV],\n    [Val || {_MappedKeyX, Val} <- lists:ukeysort(1, MKVV)].\n\n%% @doc Creates a Key-Version tree from another node's KVV list containing one\n%%      entry for every replica key of the KVV list.\n%%      Note: Assumes, KVs at the same node have the same version and sets\n%%            an (arbitrary) version from these for each replica group.\n-spec make_other_kv_tree([{?RT:key(), Val::term(), client_version()}])\n        -> mymaps:mymap().\nmake_other_kv_tree(KVV) ->\n    mymaps:from_list([{KeyX, VersionX} || {KeyX, _ValX, VersionX} <- KVV]).\n\n-spec send_request_resolve(Dest::comm:mypid(), Op::operation(),\n                           SID::rrepair:session_id() | null,\n                           FromMyNode::0 | 1, FBDest::comm:mypid() | undefined,\n                           Options::options())\n        -> ok.\nsend_request_resolve(Dest, Op, SID, FromMyNode, FBDest, Options) ->\n    Options1 = case FBDest of\n                   undefined -> Options;\n                   _         -> [{feedback_request, FBDest} | Options]\n               end,\n    Options2 = [{from_my_node, FromMyNode bxor 1} | Options1],\n    send(Dest, {continue_resolve, SID, Op, Options2}),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% resolve stats operations\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_stats_session_id(stats()) -> rrepair:session_id() | null.\nget_stats_session_id(Stats) -> Stats#resolve_stats.session_id.\n\n-spec merge_stats_feeder(stats(), stats()) -> {stats(), stats()}.\nmerge_stats_feeder(A, B) ->\n    {A, B#resolve_stats{session_id = A#resolve_stats.session_id}}.\n\n%% @doc Merges two stats records with an identical session_id\n%%      (otherwise error will be raised).\n-spec merge_stats(stats(), stats()) -> stats().\nmerge_stats(#resolve_stats{ session_id = SID,\n                            diff_size = ADiff,\n                            regen_count = ARC,\n                            regen_fail_count = AFC,\n                            upd_fail_count = AUFC,\n                            update_count = AUC },\n            #resolve_stats{ session_id = SID,\n                            diff_size = BDiff,\n                            regen_count = BRC,\n                            regen_fail_count = BFC,\n                            upd_fail_count = BUFC,\n                            update_count = BUC }) ->\n    #resolve_stats{ session_id = SID,\n                    diff_size = ADiff + BDiff,\n                    regen_count = ARC + BRC,\n                    regen_fail_count = AFC + BFC,\n                    upd_fail_count = AUFC + BUFC,\n                    update_count = AUC + BUC }.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% HELPER\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec send(Pid::comm:mypid(), Msg::comm:message() | comm:group_message()) -> ok.\nsend(Pid, Msg) ->\n    ?TRACE_SEND(Pid, Msg),\n    comm:send(Pid, Msg).\n\n-spec send_local(Pid::comm:erl_local_pid(), Msg::comm:message() | comm:group_message()) -> ok.\nsend_local(Pid, Msg) ->\n    ?TRACE_SEND(Pid, Msg),\n    comm:send_local(Pid, Msg).\n\n-spec entry_to_kvv(db_entry:entry_ex()) -> {?RT:key(), db_dht:value(), client_version()}.\nentry_to_kvv(Entry) ->\n    {db_entry:get_key(Entry),\n     db_entry:get_value(Entry),\n     db_entry:get_version(Entry)}.\n\n-spec print_resolve_stats(stats()) -> [any()].\nprint_resolve_stats(Stats) ->\n    StatsL = tl(erlang:tuple_to_list(Stats)),\n    FieldNames = record_info(fields, resolve_stats),\n    [resolve_stats, lists:zip(FieldNames, StatsL)].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% STARTUP\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec init(state()) -> state().\ninit(State) ->\n    _ = gen_component:monitor(State#rr_resolve_state.ownerPid),\n    State.\n\n-spec start(SessionId::rrepair:session_id() | null) -> {ok, MyPid::pid()}.\nstart(SessionId) ->\n    State = #rr_resolve_state{ownerPid = self(),\n                              stats = #resolve_stats{session_id = SessionId}},\n    PidName = lists:flatten(io_lib:format(\"~s.~s\", [?MODULE, randoms:getRandomString()])),\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, State,\n                             [{pid_groups_join_as, pid_groups:my_groupname(),\n                               {short_lived, PidName}}]).\n"
  },
  {
    "path": "src/rrepair/rrepair.erl",
    "content": "% @copyright 2011-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @doc    replica repair module\n%%         Replica sets will be synchronized in two steps.\n%%          I) reconciliation   - find set differences  (rr_recon.erl)\n%%         II) resolution       - resolve found differences (rr_resolve.erl)\n%%\n%%         Examples:\n%%            1) remote node should get a single kvv-pair (Key, Value, Version)\n%%               with Key mapped into first quadrant\n%%               >>comm:send(RemoteRRepairPid, {request_resolve, {?key_upd, [{Key, Value, Version}], []}, []}).\n%%\n%% @end\n%% @version $Id$\n-module(rrepair).\n-author('malange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"record_helpers.hrl\").\n-include(\"scalaris.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% debug\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-define(TRACE(X,Y), ok).\n%-define(TRACE(X,Y), log:pal(\"~w [~p:~.0p] \" ++ X ++ \"~n\", [?MODULE, pid_groups:my_groupname(), self()] ++ Y)).\n\n-define(TRACE_RECON(X,Y), ok).\n%-define(TRACE_RECON(X,Y), log:pal(\"~w [~p:~.0p] \" ++ X ++ \"~n\", [?MODULE, pid_groups:my_groupname(), self()] ++ Y)).\n\n-define(TRACE_RESOLVE(X,Y), ok).\n%-define(TRACE_RESOLVE(X,Y), log:pal(\"~w [~p:~.0p] \" ++ X ++ \"~n\", [?MODULE, pid_groups:my_groupname(), self()] ++ Y)).\n\n-define(TRACE_COMPLETE(X,Y), ok).\n%-define(TRACE_COMPLETE(X,Y), log:pal(\"~w [~p:~.0p] \" ++ X ++ \"~n\", [?MODULE, pid_groups:my_groupname(), self()] ++ Y)).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% export\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-export([start_link/1, init/1, on/2, check_config/0,\n         select_sync_node/2,\n         session_get/2]).\n\n-export_type([session_id/0, session/0]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% type definitions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-type round()       :: non_neg_integer().\n-type session_id()  :: {round(), comm:mypid()}.\n-type principal_id():: comm:mypid() | none.\n\n% @doc session contains only data of the sync request initiator thus rs_stats:regen_count represents only\n%      number of regenerated db items on the initator\n-record(session,\n        { id                = ?required(session, id)            :: session_id(),\n          principal         = none                              :: principal_id(),\n          rc_method         = ?required(session, rc_method)     :: rr_recon:method(),\n          rc_stats          = none                              :: rr_recon_stats:stats() | none,\n          rs_stats          = none                              :: rr_resolve:stats() | none,\n          rs_expected       = 0                                 :: non_neg_integer(),\n          rs_finish         = 0                                 :: non_neg_integer(),\n          ttl               = ?required(session, ttl)           :: pos_integer()    %time to live in milliseconds\n        }).\n-type session() :: #session{}.\n\n-record(rrepair_state,\n        {\n         round          = 0                                         :: round(),\n         open_recon     = 0                                         :: non_neg_integer(),\n         open_resolve   = 0                                         :: non_neg_integer(),\n         open_sessions  = []                                        :: [session()]   % List of running request_sync calls (only rounds initiated by this process)\n         }).\n-type state() :: #rrepair_state{}.\n\n-type state_field() :: round |           %next round id\n                       open_recon |      %number of open recon processes\n                       open_resolve |    %number of open resolve processes\n                       open_sessions.    %list of current running sync sessions\n\n-type message() ::\n    % API\n    {request_sync, DestKey::random | ?RT:key()} |\n    {request_sync, Method::rr_recon:method(), DestKey::random | ?RT:key()} |\n    {request_sync, Method::rr_recon:method(), DestKey::random | ?RT:key(), Principal::principal_id()} |\n    {request_resolve, rr_resolve:operation(), rr_resolve:options()} |\n    {get_state, Sender::comm:mypid(), Keys::state_field() | [state_field(),...]} |\n    % internal\n    {rr_trigger} |\n    {rr_gc_trigger} |\n    {start_sync, get_range, session_id(), rr_recon:method(), DestKey::random | ?RT:key(),\n     {get_state_response, [{my_range, intervals:interval()} | {load, non_neg_integer()},...]}} |\n\t{start_recon | continue_recon, SenderRRPid::comm:mypid(), session_id(), ReqMsg::rr_recon:request()} |\n    {request_resolve | continue_resolve, session_id() | null, rr_resolve:operation(), rr_resolve:options()} |\n    % misc\n    {web_debug_info, Requestor::comm:erl_local_pid()} |\n    % report\n    {recon_progress_report, Sender::comm:mypid(), Initiator::boolean(),\n     DestRR::comm:mypid(), DestRC::comm:mypid() | undefined, Stats::rr_recon_stats:stats()} |\n    {resolve_progress_report, Sender::comm:erl_local_pid(), Stats::rr_resolve:stats()}.\n\n-include(\"gen_component.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% API messages\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec on(message(), state()) -> state().\n\n% Requests db sync with DestKey using default recon method (given in config).\non({request_sync, DestKey}, State) ->\n    request_sync(State, get_recon_method(), DestKey, none);\n\non({request_sync, Method, DestKey}, State) ->\n    request_sync(State, Method, DestKey, none);\n\non({request_sync, Method, DestKey, Principal}, State) ->\n    request_sync(State, Method, DestKey, Principal);\n\n% initial resolve request\non({request_resolve, Operation, Options}, State) ->\n    gen_component:post_op({request_resolve, null, Operation, Options}, State);\n\n% request replica repair status\non({get_state, Sender, Key}, State =\n       #rrepair_state{ open_recon = Recon, open_resolve = Resolve,\n                       round = Round, open_sessions = Sessions }) ->\n    Keys = if is_list(Key) -> Key;\n              is_atom(Key) -> [Key]\n           end,\n    Values0 = [case KeyX of\n                   open_recon -> Recon;\n                   open_resolve -> Resolve;\n                   round -> Round;\n                   open_sessions -> Sessions\n               end || KeyX <- Keys],\n    Value = if is_list(Key) -> Values0;\n               is_atom(Key) -> hd(Values0)\n            end,\n    comm:send(Sender, {get_state_response, Value}),\n    State;\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% internal messages\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\non({rr_trigger} = Msg, State) ->\n    ?TRACE(\"RR: SYNC TRIGGER\", []),\n    Prob = get_start_prob(),\n    Random = randoms:rand_uniform(1, 100),\n    if Random =< Prob ->\n           comm:send_local(self(), {request_sync, get_recon_method(), random});\n       true -> ok\n    end,\n    case get_trigger_interval() of\n        0 -> ok;\n        X -> msg_delay:send_trigger(X, Msg)\n    end,\n    State;\n\non({rr_gc_trigger} = Msg, State = #rrepair_state{ open_sessions = Sessions }) ->\n    Elapsed = get_gc_interval() * 1000,\n    NewSessions = [S#session{ ttl = S#session.ttl - Elapsed }\n                            || S <- Sessions,\n                               S#session.ttl - Elapsed > 0],\n    msg_delay:send_trigger(Elapsed div 1000, Msg),\n    State#rrepair_state{ open_sessions = NewSessions };\n\non({start_sync, get_range, SessionId, Method, DestKey,\n    {get_state_response, [{my_range, MyI}, {load, MyLoad}]}}, State) ->\n    Msg = {?send_to_group_member, rrepair,\n           {start_recon, comm:this(), SessionId,\n            {create_struct, Method, MyI, MyLoad}}},\n    DKey = case DestKey of\n               random -> select_sync_node(MyI, true);\n               _ -> DestKey\n           end,\n    % skip if no key outside my range found\n    case DKey of\n        not_found ->\n            #rrepair_state{open_recon = OR, open_sessions = OS} = State,\n            % similar to handling of recon_progress_report as initiator\n            % assume the session is present (request_sync has created it!)\n            {S, TSessions} = extract_session(SessionId, OS),\n            ?TRACE_RECON(\"~nRECON OK3 - ~p\", [S]),\n            % this session is aborted, so it is complete!\n            Stats = rr_recon_stats:new(SessionId, [{status, finish}]),\n            SUpd = update_session_recon(S, Stats),\n            true = check_session_complete(SUpd),\n            State#rrepair_state{open_recon = OR - 1,\n                                open_sessions = TSessions};\n        _ ->\n            ?TRACE_RECON(\"START_TO_DEST ~p\", [DKey]),\n            api_dht_raw:unreliable_lookup(DKey, Msg),\n            State\n    end;\n\n%% @doc receive sync request at the non-initiator and spawn a new process\n%%      which executes a sync protocol\non({start_recon, Sender, SessionID, Msg}, State) ->\n    ?TRACE_RECON(\"START RECON FROM ~p\", [Sender]),\n    {ok, Pid} = rr_recon:start(SessionID, Sender),\n    comm:send_local(Pid, Msg),\n    State#rrepair_state{ open_recon = State#rrepair_state.open_recon + 1 };\n\n%% @doc receive sync request at the initiator (after setting up the request via\n%%      request_sync) and spawn a new process which executes a sync protocol\non({continue_recon, Sender, SessionID, Msg}, State) ->\n    ?TRACE_RECON(\"CONTINUE RECON FROM ~p\", [Sender]),\n    {ok, Pid} = rr_recon:start(SessionID, Sender),\n    comm:send_local(Pid, Msg),\n    % note: don't increase open_recon (this is a continued process previously\n    %       counted by request_sync)\n    State;\n\n% (first or continued) resolve request\non({Tag, SessionID, Operation, Options},\n   State = #rrepair_state{open_resolve = OpenResolve})\n  when Tag =:= request_resolve orelse Tag =:= continue_resolve ->\n    {ok, Pid} = rr_resolve:start(SessionID),\n    comm:send_local(Pid, {start, Operation, Options, Tag}),\n    State#rrepair_state{ open_resolve = OpenResolve + 1 };\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% report messages\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\non({recon_progress_report, Sender, _Initiator = false, DestRR, DestRC, Stats},\n   State = #rrepair_state{ open_recon = OR }) ->\n    ?TRACE_RECON(\"~nRECON-NI OK - Sender=~p~nStats=~p~nOpenRecon=~p~nSessions=~p,~nDestRR: ~p, DestRC: ~p\",\n                 [Sender, rr_recon_stats:print(Stats), OR - 1, State#rrepair_state.open_sessions, DestRR, DestRC]),\n    % TODO: integrate non-initiator stats into the stats of the initiator?\n    %       -> may need to be integrated into the 'continue_recon' message of rr_recon\n    case rr_recon_stats:get(status, Stats) of\n        abort when DestRC =:= undefined ->\n            % report to Initiator since he still has a session laying around\n            % and no local rr_recon process to terminate it\n            % use empty stats with status abort for now since non-initiator\n            % stats are not integrated in the successful case either\n            SID = rr_recon_stats:get(session_id, Stats),\n            StatsToSend = rr_recon_stats:new(SID, [{status, abort}]),\n            comm:send(DestRR, {recon_progress_report, Sender, true,\n                               comm:this(), undefined, StatsToSend});\n        _ -> ok\n    end,\n    State#rrepair_state{ open_recon = OR - 1 };\non({recon_progress_report, _Sender, _Initiator = true, _DestRR, _DestRC, Stats},\n   State = #rrepair_state{open_recon = ORC, open_resolve = _ORS,\n                          open_sessions = OS}) ->\n    {S, TSessions} = extract_session(rr_recon_stats:get(session_id, Stats), OS),\n    SUpd = update_session_recon(S, Stats),\n    ?TRACE_RECON(\"~nRECON-I OK - Sender=~p~nStats=~p~nSession=~.2p~nOpenRecon=~p ; OpenResolve=~p~nOldSessions=~p,~n~p\",\n                 [_Sender, rr_recon_stats:print(Stats), SUpd, ORC - 1, _ORS, OS, rr_recon_stats:print(Stats)]),\n    NewOS = case check_session_complete(SUpd) of\n                true -> TSessions;\n                _    -> [SUpd | TSessions]\n            end,\n    State#rrepair_state{open_recon = ORC - 1, open_sessions = NewOS};\n\non({resolve_progress_report, _Sender, Stats},\n   State = #rrepair_state{open_resolve = OR, open_sessions = OS}) ->\n    SID = rr_resolve:get_stats_session_id(Stats),\n    NSessions = case extract_session(SID, OS) of\n                    not_found when SID =:= null ->\n                        % all other session IDs should be there!\n                        OS;\n                    {S, TSessions} ->\n                        SUpd = update_session_resolve(S, Stats),\n                        case check_session_complete(SUpd) of\n                            true -> TSessions;\n                            _    -> [SUpd | TSessions]\n                        end\n                end,\n    ?TRACE_RESOLVE(\"~nRESOLVE OK - Sender=~p ~nStats=~p~nOpenRecon=~p ; OpenResolve=~p~nOldSessions=~p~nNewSessions=~p\",\n                   [_Sender, rr_resolve:print_resolve_stats(Stats),\n                    State#rrepair_state.open_recon, OR - 1, OS, NSessions]),\n    State#rrepair_state{open_resolve = OR - 1,\n                        open_sessions = NSessions};\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% misc info messages\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\non({web_debug_info, Requestor}, #rrepair_state{ round = Round,\n                                                open_recon = OpenRecon,\n                                                open_resolve = OpenResol,\n                                                open_sessions = Sessions } = State) ->\n    ?TRACE(\"WEB DEBUG INFO\", []),\n    KeyValueList =\n        [{\"Recon Method:\",      webhelpers:safe_html_string(\"~p\", [get_recon_method()])},\n         {\"Sync Round:\",        webhelpers:safe_html_string(\"~p\", [Round])},\n         {\"Open Recon Jobs:\",   webhelpers:safe_html_string(\"~p\", [OpenRecon])},\n         {\"Open Resolve Jobs:\", webhelpers:safe_html_string(\"~p\", [OpenResol])},\n         {\"Open Sessions:\",     webhelpers:safe_html_string(\"~p\", [length(Sessions)])}\n        ],\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    State.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% internal functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% - Requests database synchronization with DestPid (DestPid=DhtNodePid or random).\n%   Random leads to sync with a node which is associated with this (e.g. symmetric partner)\n% - Principal will eventually get an request_sync_complete message\n%   (no result message will be send if request receiver dies etc.).\n-spec request_sync(State::state(), Method::rr_recon:method(),\n                   DestKey::random | ?RT:key(), Principal::principal_id()) -> state().\nrequest_sync(State = #rrepair_state{round = Round, open_recon = OpenRecon,\n                                    open_sessions = Sessions},\n             Method, DestKey, Principal) ->\n    ?TRACE(\"RR: REQUEST SYNC WITH ~p\", [DestKey]),\n    This0 = comm:this(),\n    S = new_session(Round, This0, Method, Principal),\n    This = comm:reply_as(This0, 6, {start_sync, get_range, S#session.id, Method, DestKey, '_'}),\n    comm:send_local(pid_groups:get_my(dht_node), {get_state, This, [my_range, load]}),\n    State#rrepair_state{ round = next_round(Round),\n                         open_recon = OpenRecon + 1,\n                         open_sessions = [S | Sessions] }.\n\n%% @doc Selects a random key in the given (continuous) interval and returns one\n%%      of its replicas which is not in the interval (if ExcludeInterval is true).\n%%      If ExcludeInterval is false, any of its replica keys is returned.\n-spec select_sync_node\n        (intervals:continuous_interval(), ExcludeInterval::false) -> ?RT:key();\n        (intervals:continuous_interval(), ExcludeInterval::true)  -> ?RT:key() | not_found.\nselect_sync_node(Interval, ExcludeInterval) ->\n    ?DBG_ASSERT(intervals:is_continuous(Interval)),\n    case intervals:is_all(Interval) of\n        true when ExcludeInterval -> not_found; % no sync partner here!\n        _ ->\n            Bounds = intervals:get_bounds(Interval),\n            Key = ?RT:get_random_in_interval(Bounds),\n            Keys = if ExcludeInterval ->\n                          [K || K <- ?RT:get_replica_keys(Key),\n                                not intervals:in(K, Interval)];\n                      true -> ?RT:get_replica_keys(Key)\n                   end,\n            case Keys of\n                [] -> not_found;\n                [_|_] -> util:randomelem(Keys)\n            end\n    end.\n\n-spec next_round(round()) -> round().\nnext_round(R) -> R + 1.\n\n-spec new_session(round(), comm:mypid(), rr_recon:method(), principal_id()) -> session().\nnew_session(Round, Pid, RCMethod, Principal) ->\n    #session{ id = {Round, Pid}, rc_method = RCMethod, ttl = get_session_ttl(), principal = Principal }.\n\n-spec extract_session(session_id() | null, [session()]) -> {session(), Remain::[session()]} | not_found.\nextract_session(null, _Sessions) -> not_found;\nextract_session(Id, Sessions) ->\n    {Satis, NotSatis} = lists:partition(fun(#session{ id = I }) ->\n                                                Id =:= I\n                                        end,\n                                        Sessions),\n    case Satis of\n        [X] -> {X, NotSatis};\n        [] -> not_found;\n        _ ->\n            log:log(error, \"[ ~p ] SESSION NOT UNIQUE! ~p - OpenSessions=~p\", [?MODULE, Id, Sessions]),\n            not_found\n    end.\n\n-spec update_session_recon(session(), rr_recon_stats:stats()) -> session().\nupdate_session_recon(Session = #session{rs_expected = 0}, New) ->\n    ?ASSERT(rr_recon_stats:get(status, New) =/= wait),\n    NewRS = rr_recon_stats:get(rs_expected, New),\n    Session#session{rc_stats  = New, rs_expected = NewRS}.\n\n%% @doc Increases the rs_finish field and merges the new stats.\n-spec update_session_resolve(session(), rr_resolve:stats()) -> session().\nupdate_session_resolve(#session{ rs_stats = none, rs_finish = RSCount } = S, Stats) ->\n    S#session{rs_stats = Stats, rs_finish = RSCount + 1};\nupdate_session_resolve(#session{ rs_stats = Old, rs_finish = RSCount} = S, New) ->\n    Merge = rr_resolve:merge_stats(Old, New),\n    S#session{rs_stats = Merge, rs_finish = RSCount + 1}.\n\n%% @doc Checks if the session is complete (rs_called =:= rs_finish, stats\n%%      available) and in this case informs the principal and returns 'true'.\n%%      Otherwise 'false'.\n-spec check_session_complete(session()) -> boolean().\ncheck_session_complete(#session{rc_stats = RCStats, principal = PrincipalPid,\n                                rs_expected = C, rs_finish = C} = S)\n  when RCStats =/= none ->\n    case rr_recon_stats:get(status, RCStats) of\n        X when X =:= finish orelse X =:= abort ->\n            ?TRACE_COMPLETE(\"--SESSION COMPLETE--~n~p\", [S]),\n            case PrincipalPid of\n                none -> ok;\n                _ -> comm:send(PrincipalPid, {request_sync_complete, S})\n            end,\n            true;\n        wait ->\n            false\n    end;\ncheck_session_complete(_Session) ->\n    false.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Startup\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Starts the replica update process,\n%%      registers it with the process dictionary\n%%      and returns its pid for use by a supervisor.\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                             [{pid_groups_join_as, DHTNodeGroup, ?MODULE}]).\n\n%% @doc Initialises the module and starts the trigger\n-spec init([]) -> state().\ninit([]) ->\n    case get_trigger_interval() of\n        0 -> ok;\n        X -> msg_delay:send_trigger(X, {rr_trigger})\n    end,\n    msg_delay:send_trigger(get_gc_interval(), {rr_gc_trigger}),\n    #rrepair_state{}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% API functions for the session record\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec session_get(id, session())          -> session_id();\n                 (principal, session())   -> principal_id();\n                 (rc_method, session())   -> rr_recon:method();\n                 (rc_stats, session())    -> rr_recon_stats:stats() | none;\n                 (rs_stats, session())    -> rr_resolve:stats() | none;\n                 (rs_expected, session()) -> non_neg_integer();\n                 (rs_finish, session())   -> non_neg_integer();\n                 (ttl, session())         -> pos_integer().\nsession_get(id         , #session{id          = X}) -> X;\nsession_get(principal  , #session{principal   = X}) -> X;\nsession_get(rc_method  , #session{rc_method   = X}) -> X;\nsession_get(rc_stats   , #session{rc_stats    = X}) -> X;\nsession_get(rs_stats   , #session{rs_stats    = X}) -> X;\nsession_get(rs_expected, #session{rs_expected = X}) -> X;\nsession_get(rs_finish  , #session{rs_finish   = X}) -> X;\nsession_get(ttl        , #session{ttl         = X}) -> X.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Config handling\n%\n% USED CONFIG FIELDS\n%\t* rr_trigger_interval: integer duration until next triggering (milliseconds) (0 = de-activate)\n%\t* rr_recon_method: set reconciliation algorithm name\n%   * rr_trigger_probability: this is the probability of starting a synchronisation\n%                             with a random node if trigger has fired. ]0,100]\n%   * rr_session_ttl: time to live for sessions until they are garbage collected (milliseconds)\n%   * rr_gc_interval: garbage collector execution interval (milliseconds)\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_in(rr_recon_method, [trivial, shash, bloom, merkle_tree, art]) andalso % theoretically also 'iblt', but no full support for that yet\n        config:cfg_is_integer(rr_session_ttl) andalso\n        config:cfg_is_greater_than(rr_session_ttl, 0) andalso\n        config:cfg_is_integer(rr_trigger_probability) andalso\n        config:cfg_is_greater_than(rr_trigger_probability, 0) andalso\n        config:cfg_is_less_than_equal(rr_trigger_probability, 100) andalso\n        config:cfg_is_integer(rr_gc_interval) andalso\n        config:cfg_is_greater_than(rr_gc_interval, 0) andalso\n        config:cfg_is_integer(rr_trigger_interval) andalso\n        config:cfg_is_greater_than_equal(rr_trigger_interval, 0).\n\n-spec get_recon_method() -> rr_recon:method().\nget_recon_method() ->  config:read(rr_recon_method).\n\n-spec get_trigger_interval() -> non_neg_integer().\nget_trigger_interval() ->\n    %% deactivated when 0,so ceil when larger than 0\n    util:ceil(config:read(rr_trigger_interval) / 1000).\n\n-spec get_start_prob() -> pos_integer().\nget_start_prob() -> config:read(rr_trigger_probability).\n\n-spec get_session_ttl() -> pos_integer().\nget_session_ttl() -> config:read(rr_session_ttl).\n\n-spec get_gc_interval() -> pos_integer().\nget_gc_interval() -> config:read(rr_gc_interval) div 1000.\n"
  },
  {
    "path": "src/rt_beh.erl",
    "content": "% @copyright 2007-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc routing table behaviour\n%% @end\n%% @version $Id$\n-module(rt_beh).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n% for behaviour\n-ifndef(have_callback_support).\n-export([behaviour_info/1]).\n-endif.\n\n-export([tester_create_segment/1,\n         tester_is_segment/1]).\n-export_type([segment/0]).\n\n-type segment() :: pos_integer(). % 1..config:read(replication_factor)\n\n%% userdevguide-begin rt_beh:behaviour\n-ifdef(have_callback_support).\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n-type rt() :: term().\n-type external_rt() :: term().\n-type key() :: term().\n\n-callback empty_ext(nodelist:neighborhood()) -> external_rt().\n-callback init() -> ok.\n-callback activate(nodelist:neighborhood()) -> rt().\n-callback hash_key(client_key() | binary()) -> key().\n-callback get_random_node_id() -> key().\n-callback next_hop(nodelist:neighborhood(), external_rt(), key()) -> succ | comm:mypid().\n-callback succ(external_rt(), nodelist:neighborhood()) -> comm:mypid().\n\n-callback init_stabilize(nodelist:neighborhood(), rt()) -> rt().\n-callback update(OldRT::rt(), OldNeighbors::nodelist:neighborhood(),\n                 NewNeighbors::nodelist:neighborhood())\n        -> {trigger_rebuild, rt()} | {ok, rt()}.\n-callback filter_dead_node(rt(), DeadPid::comm:mypid(), Reason::fd:reason()) -> rt().\n\n-callback to_pid_list(rt()) -> [comm:mypid()].\n-callback get_size(rt()) -> non_neg_integer().\n-callback get_size_ext(external_rt()) -> non_neg_integer().\n-callback get_replica_keys(key()) -> [key()].\n-callback get_replica_keys(key(), pos_integer()) -> [key()].\n-callback get_key_segment(key()) -> pos_integer().\n-callback get_key_segment(key(), pos_integer()) -> pos_integer().\n\n-callback n() -> number().\n-callback get_range(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE) -> number().\n-callback get_split_key(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE,\n                        SplitFraction::{Num::number(), Denom::pos_integer()})\n        -> key() | ?PLUS_INFINITY_TYPE.\n-callback get_split_keys(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE,\n                         Parts::pos_integer()) -> [key()].\n-callback get_random_in_interval(intervals:simple_interval2()) -> key().\n\n-callback dump(RT::rt()) -> KeyValueList::[{Index::string(), Node::string()}].\n\n-callback to_list(dht_node_state:state()) -> [{key(), comm:mypid()}].\n-callback export_rt_to_dht_node(rt(), Neighbors::nodelist:neighborhood()) -> external_rt().\n-callback handle_custom_message_inactive(comm:message(), msg_queue:msg_queue()) -> msg_queue:msg_queue().\n-callback handle_custom_message(comm:message(), rt_loop:state_active()) -> rt_loop:state_active() | unknown_event.\n\n-callback check(OldRT::rt(), NewRT::rt(), OldERT::external_rt(), Neighbors::nodelist:neighborhood(),\n            ReportToFD::boolean()) -> NewERT::external_rt().\n-callback check(OldRT::rt(), NewRT::rt(), OldERT::external_rt(),\n            OldNeighbors::nodelist:neighborhood(), NewNeighbors::nodelist:neighborhood(),\n            ReportToFD::boolean()) -> NewERT::external_rt().\n\n-callback check_config() -> boolean().\n-callback wrap_message(Key::key(), Msg::comm:message(), MyERT::external_rt(),\n                       Neighbors::nodelist:neighborhood(),\n                       Hops::non_neg_integer()) -> comm:message().\n-callback unwrap_message(Msg::comm:message(), State::dht_node_state:state()) ->\n    comm:message().\n\n-else.\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n    [\n     % create a default routing table\n     {empty_ext, 1},\n     % initialize a routing table\n     {init, 0},\n     % activate a routing table\n     {activate, 1},\n     % mapping: key space -> identifier space\n     {hash_key, 1}, {get_random_node_id, 0},\n     % routing\n     {next_hop, 3},\n     % return the succ\n     {succ, 2},\n     % trigger for new stabilization round\n     {init_stabilize, 2},\n     % adapt RT to changed neighborhood\n     {update, 3},\n     % dead nodes filtering\n     {filter_dead_node, 3},\n     % statistics\n     {to_pid_list, 1}, {get_size, 1}, {get_size_ext, 1},\n     % gets all (replicated) keys for a given (hashed) key\n     % (for symmetric replication)\n     {get_replica_keys, 1}, {get_replica_keys, 2},\n     % get the segment of the ring a key belongs to (1-4)\n     {get_key_segment, 1}, {get_key_segment, 2},\n     % address space size, range and split key\n     % (may all throw 'throw:not_supported' if unsupported by the RT)\n     {n, 0}, {get_range, 2}, {get_split_key, 3},\n     % get a random key wihtin the requested interval\n     {get_random_in_interval, 1},\n     % for debugging and web interface\n     {dump, 1},\n     % for bulkowner\n     {to_list, 1},\n     % convert from internal representation to version for dht_node\n     {export_rt_to_dht_node, 2},\n     % handle messages specific to a certain routing-table implementation if\n     % rt_loop is in on_inactive state\n     {handle_custom_message_inactive, 2},\n     % handle messages specific to a certain routing-table implementation\n     {handle_custom_message, 2},\n     % common methods\n     {check, 5}, {check, 6},\n     {check_config, 0},\n     % wrap and unwrap lookup messages\n     {wrap_message, 5},\n     {unwrap_message, 2}\n    ];\nbehaviour_info(_Other) ->\n    undefined.\n-endif.\n%% userdevguide-end rt_beh:behaviour\n\n-spec tester_create_segment(pos_integer()) -> segment().\ntester_create_segment(Int) ->\n    Int rem config:read(replication_factor) + 1.\n\n-spec tester_is_segment(segment()) -> boolean().\ntester_is_segment(Segment) when Segment < 1 ->\n    false;\ntester_is_segment(Segment) ->\n    Segment =< config:read(replication_factor).\n"
  },
  {
    "path": "src/rt_beh.hrl",
    "content": "% @copyright 2010-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Common types and function specs for routing table implementations.\n%% @end\n%% @version $Id$\n\n-include(\"client_types.hrl\").\n\n-export_type([key/0, rt/0, custom_message/0, external_rt/0]).\n\n-export([empty_ext/1, init/0, activate/1,\n         hash_key/1, get_random_node_id/0, next_hop/3, succ/2,\n         init_stabilize/2, update/3,\n         filter_dead_node/3, to_pid_list/1, get_size/1, get_size_ext/1,\n         get_replica_keys/1, get_replica_keys/2, get_key_segment/1,\n         get_key_segment/2,\n         n/0, get_range/2, get_random_in_interval/1, get_random_in_interval/2,\n         get_split_key/3, get_split_keys/3,\n         dump/1, to_list/1, export_rt_to_dht_node/2,\n         handle_custom_message_inactive/2, handle_custom_message/2,\n         check/5, check/6,\n         check_config/0,\n         client_key_to_binary/1,\n         wrap_message/5,\n         unwrap_message/2\n     ]).\n\n-spec client_key_to_binary(Key::client_key()) -> binary().\nclient_key_to_binary(Key) when is_binary(Key) ->\n    Key;\nclient_key_to_binary(Key) ->\n    case unicode:characters_to_binary(Key) of\n        {incomplete, Encoded, Rest} ->\n            RestBin = unicode:characters_to_binary(Rest, latin1),\n            <<Encoded/binary, RestBin/binary>>;\n        {error, Encoded, [Invalid | Rest]} when Invalid >= 16#D800 andalso Invalid =< 16#DFFF ->\n            % nevertheless encode the invalid unicode character in range\n            % 16#D800 to 16#DFFF\n            <<X1, X2:2/binary>> = unicode:characters_to_binary([Invalid - 4096]),\n            % map to the \"correct\" binary:\n            RestBin = client_key_to_binary(Rest),\n            <<Encoded/binary, (X1 + 1), X2/binary, RestBin/binary>>;\n        {error, Encoded, [Invalid | Rest]} when Invalid =:= 16#FFFE orelse Invalid =:= 16#FFFF ->\n            % nevertheless encode the invalid unicode character\n            <<X1:2/binary, X2>> = unicode:characters_to_binary([Invalid - 2]),\n            % map to the \"correct\" binary:\n            RestBin = client_key_to_binary(Rest),\n            <<Encoded/binary, X1/binary, (X2 + 1), RestBin/binary>>;\n        Bin -> Bin\n    end.\n\n% note: can not use wrapper methods for all methods to make dialyzer happy\n% about the opaque types since it doesn't recognize the module's own opaque\n% type if returned by a method outside the module's scope, e.g. node:id/1.\n% If required, e.g. a method is used internally and externally, use a wrapper\n% in the implementing module.\n"
  },
  {
    "path": "src/rt_chord.erl",
    "content": "% @copyright 2007-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Routing Table\n%% @end\n-module(rt_chord).\n-author('schuett@zib.de').\n\n-behaviour(rt_beh).\n-include(\"scalaris.hrl\").\n\n%% userdevguide-begin rt_chord:types\n-type key() :: 0..16#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF. % 128 bit numbers\n-type index() :: {pos_integer(), non_neg_integer()}.\n-opaque rt() :: gb_trees:tree(index(), {Node::node:node_type(), PidRT::comm:mypid()}).\n-opaque external_rt() :: gb_trees:tree(NodeId::key(), PidRT::comm:mypid()).\n-type custom_message() ::\n       {rt_get_node, Source_PID::comm:mypid(), Index::index()} |\n       {rt_get_node_response, Index::index(), Node::node:node_type()}.\n%% userdevguide-end rt_chord:types\n\n-define(SEND_OPTIONS, [{channel, prio}]).\n\n-export([add_range/2]).\n\n% Note: must include rt_beh.hrl AFTER the type definitions for erlang < R13B04\n% to work.\n-include(\"rt_beh.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Key Handling\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% userdevguide-begin rt_chord:empty\n%% @doc Creates an empty routing table.\n-spec empty(nodelist:neighborhood()) -> rt().\nempty(_Neighbors) -> gb_trees:empty().\n%% userdevguide-end rt_chord:empty\n\n%% @doc This function is called during the startup of the rt_loop process and\n%%      is allowed to send trigger messages.\n%%      Noop in chord.\n-spec init() -> ok.\ninit() -> ok.\n\n%% @doc Activate the routing table.\n%%      This function is called during the activation of the routing table process.\n-spec activate(nodelist:neighborhood()) -> rt().\nactivate(Neighbors) -> empty(Neighbors).\n\n%% @doc Hashes the key to the identifier space.\n-spec hash_key(binary() | client_key()) -> key().\nhash_key(Key) when not is_binary(Key) ->\n    hash_key(client_key_to_binary(Key));\nhash_key(Key) ->\n    <<N:128>> = ?CRYPTO_MD5(Key),\n    N.\n\n%% @doc Generates a random node id, i.e. a random 128-bit number, based on the\n%%      parameters set in the config file (key_creator and key_creator_bitmask).\n-spec get_random_node_id() -> key().\nget_random_node_id() ->\n    case config:read(key_creator) of\n        random -> hash_key(randoms:getRandomString());\n        random_with_bit_mask ->\n            {Mask1, Mask2} = config:read(key_creator_bitmask),\n            (hash_key(randoms:getRandomString()) band Mask2) bor Mask1;\n        modr ->\n            %% put random key into first quarter\n            Key = hash_key(randoms:getRandomString()) div config:read(replication_factor),\n            %% select the quarter based on the availability zone id\n            Quarter = config:read(availability_zone_id) rem config:read(replication_factor),\n            %% calculate the final key\n            Key + Quarter * n() div config:read(replication_factor)\n    end.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% RT Management\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% userdevguide-begin rt_chord:init_stabilize\n%% @doc Starts the stabilization routine.\n-spec init_stabilize(nodelist:neighborhood(), rt()) -> rt().\ninit_stabilize(Neighbors, RT) ->\n    % calculate the longest finger\n    case first_index(Neighbors) of\n        null -> ok;\n        {Key, Index} ->\n            % trigger a lookup for Key\n            api_dht_raw:unreliable_lookup(\n              Key, {?send_to_group_member, routing_table,\n                    {rt_get_node, comm:this(), Index}})\n    end,\n    RT.\n%% userdevguide-end rt_chord:init_stabilize\n\n%% userdevguide-begin rt_chord:filter_dead_node\n%% @doc Removes dead nodes from the routing table.\n-spec filter_dead_node(rt(), DeadPid::comm:mypid(), Reason::fd:reason()) -> rt().\nfilter_dead_node(RT, DeadPid, _Reason) ->\n    DeadIndices = [Index || {Index, {Node, _PidRT}}  <- gb_trees:to_list(RT),\n                            node:same_process(Node, DeadPid)],\n    lists:foldl(fun(Index, Tree) -> gb_trees:delete(Index, Tree) end,\n                RT, DeadIndices).\n%% userdevguide-end rt_chord:filter_dead_node\n\n%% @doc Returns the pids of the routing table entries.\n-spec to_pid_list(rt()) -> [comm:mypid()].\nto_pid_list(RT) ->\n    [node:pidX(Node) || {Node, _PidRT} <- gb_trees:values(RT)].\n\n%% @doc Returns the size of the routing table.\n-spec get_size(rt()) -> non_neg_integer().\nget_size(RT) ->\n    gb_trees:size(RT).\n\n%% @doc Returns the size of the external routing table.\n-spec get_size_ext(external_rt()) -> non_neg_integer().\nget_size_ext(RT) ->\n    gb_trees:size(RT).\n\n%% @doc Keep a key in the address space. See n/0.\n-spec normalize(non_neg_integer()) -> key().\nnormalize(Key) -> Key band 16#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.\n\n%% @doc Returns the size of the address space.\n-spec n() -> 16#100000000000000000000000000000000.\nn() -> 16#100000000000000000000000000000000.\n\n%% @doc Adds the given range, i.e. Range/n() parts of the key space, to the\n%%      given key.\n-spec add_range(key(), Range::non_neg_integer()) -> key().\nadd_range(Key, Range) -> normalize(Key + Range).\n\n%% @doc Gets the number of keys in the interval (Begin, End]. In the special\n%%      case of Begin==End, the whole key range as specified by n/0 is returned.\n-spec get_range(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE) -> non_neg_integer().\nget_range(Begin, Begin) -> n(); % I am the only node\nget_range(?MINUS_INFINITY, ?PLUS_INFINITY) -> n(); % special case, only node\nget_range(Begin, End) when End > Begin -> End - Begin;\nget_range(Begin, End) when End < Begin -> (n() - Begin) + End.\n\n%% @doc Gets the key that splits the interval (Begin, End] so that the first\n%%      interval will (roughly) be (Num/Denom) * range(Begin, End). In the\n%%      special case of Begin==End, the whole key range is split.\n%%      Beware: (Num/Denom) must be in [0, 1]; the final key will be rounded\n%%      down and may thus be Begin.\n-spec get_split_key(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE,\n                    SplitFraction::{Num::number(), Denom::pos_integer()}) -> key() | ?PLUS_INFINITY_TYPE.\nget_split_key(Begin, _End, {Num, _Denom}) when Num == 0 -> Begin;\nget_split_key(_Begin, End, {Num, Denom}) when Num == Denom -> End;\nget_split_key(Begin, End, {Num, Denom}) ->\n    normalize(Begin + trunc(get_range(Begin, End) * Num) div Denom).\n\n%% @doc Splits the range between Begin and End into up to Parts equal parts and\n%%      returning the according split keys.\n-spec get_split_keys(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE,\n                     Parts::pos_integer()) -> [key()].\nget_split_keys(Begin, End, Parts) ->\n    get_split_keys_helper(Begin, End, Parts).\n\n-spec get_split_keys_helper(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE,\n                            Parts::pos_integer()) -> [key()].\nget_split_keys_helper(_Begin, _End, 1) ->\n    [];\nget_split_keys_helper(Begin, End, Parts) ->\n    SplitKey = get_split_key(Begin, End, {1, Parts}),\n    if SplitKey =:= Begin ->\n           get_split_keys_helper(Begin, End, Parts - 1);\n       true ->\n           [SplitKey | get_split_keys_helper(SplitKey, End, Parts - 1)]\n    end.\n\n%% @doc Gets input similar to what intervals:get_bounds/1 returns and\n%%      calculates a random key in this range. Fails with an exception if there\n%%      is no key.\n-spec get_random_in_interval(intervals:simple_interval2()) -> key().\nget_random_in_interval(I) ->\n    hd(get_random_in_interval(I, 1)).\n\n%% @doc Gets input similar to what intervals:get_bounds/1 returns and\n%%      calculates Count number of random keys in this range (duplicates may\n%%      exist!). Fails with an exception if there is no key.\n-spec get_random_in_interval(intervals:simple_interval2(), Count::pos_integer()) -> [key(),...].\nget_random_in_interval({LBr, L, R, RBr}, Count) ->\n    case intervals:wraps_around(LBr, L, R, RBr) of\n        false -> get_random_in_interval2(LBr, L, R, RBr, Count);\n        true  -> [normalize(Key) || Key <- get_random_in_interval2(LBr, L, ?PLUS_INFINITY + R, RBr, Count)]\n    end.\n\n% TODO: return a failure constant if the interval is empty? (currently fails with an exception)\n-spec get_random_in_interval2(intervals:left_bracket(), key(), non_neg_integer(),\n                              intervals:right_bracket(), Count::pos_integer()) -> [non_neg_integer()].\nget_random_in_interval2('(', L, R, ']', Count) ->\n    L2 = L + 1,\n    R2 = R + 1,\n    randoms:rand_uniform(L2, R2, Count);\nget_random_in_interval2('[', L, R, ')', Count) ->\n    randoms:rand_uniform(L, R, Count);\nget_random_in_interval2('[', X, X, ']', Count) ->\n    lists:duplicate(Count, X);\nget_random_in_interval2('[', L, R, ']', Count) ->\n    R2 = R + 1,\n    randoms:rand_uniform(L, R2, Count);\nget_random_in_interval2('(', L, R, ')', Count) ->\n    L2 = L + 1,\n    randoms:rand_uniform(L2, R, Count).\n\n%% @doc Returns the replicas of the given key.\n-spec get_replica_keys(key()) -> [key()].\nget_replica_keys(Key) ->\n    rt_simple:get_replica_keys(Key).\n\n-spec get_replica_keys(key(), pos_integer()) -> [key()].\nget_replica_keys(Key, ReplicationFactor) ->\n    rt_simple:get_replica_keys(Key, ReplicationFactor).\n\n-spec get_key_segment(key()) -> pos_integer().\nget_key_segment(Key) ->\n    rt_simple:get_key_segment(Key).\n\n-spec get_key_segment(key(), pos_integer()) -> pos_integer().\nget_key_segment(Key, ReplicationFactor) ->\n    rt_simple:get_key_segment(Key, ReplicationFactor).\n\n%% @doc Dumps the RT state for output in the web interface.\n-spec dump(RT::rt()) -> KeyValueList::[{Index::string(), Node::string()}].\ndump(RT) ->\n    [{webhelpers:safe_html_string(\"~p\", [Index]),\n      webhelpers:safe_html_string(\"~p\", [Node])} || {Index, {Node, _PidRT}} <- gb_trees:to_list(RT)].\n\n%% userdevguide-begin rt_chord:stabilize\n%% @doc Updates one entry in the routing table and triggers the next update.\n%%      Changed indicates whether a new node was inserted (the RT structure may\n%%      change independently from this indicator!).\n-spec stabilize(Neighbors::nodelist:neighborhood(), OldRT::rt(), Index::index(),\n                Node::node:node_type(), PidRT::comm:mypid()) -> {NewRT::rt(), Changed::boolean()}.\nstabilize(Neighbors, RT, Index, Node, PidRT) ->\n    MyId = nodelist:nodeid(Neighbors),\n    Succ = nodelist:succ(Neighbors),\n    case (node:id(Succ) =/= node:id(Node))   % reached succ?\n        andalso (not intervals:in(           % there should be nothing shorter\n                   node:id(Node),            %   than succ\n                   nodelist:succ_range(Neighbors))) of\n        true ->\n            NextIndex = next_index(Index),\n            NextKey = calculateKey(MyId, NextIndex),\n            CurrentKey = calculateKey(MyId, Index),\n            case CurrentKey =/= NextKey of\n                true ->\n                    Msg = {rt_get_node, comm:this(), NextIndex},\n                    api_dht_raw:unreliable_lookup(\n                      NextKey, {?send_to_group_member, routing_table, Msg});\n                _ -> ok\n            end,\n            Changed = (Index =:= first_index() orelse\n                       (gb_trees:lookup(prev_index(Index), RT) =/= {value, {Node, PidRT}})),\n            {gb_trees:enter(Index, {Node, PidRT}, RT), Changed};\n        false ->\n            %% there should be nothing shorter than succ\n            case intervals:in(node:id(Node), nodelist:succ_range(Neighbors)) of\n                %% ignore message\n                false -> {RT, false};\n                %% add succ to RT\n                true -> {gb_trees:enter(Index, {Node, PidRT}, RT), true}\n            end\n    end.\n%% userdevguide-end rt_chord:stabilize\n\n%% userdevguide-begin rt_chord:update\n%% @doc Updates the routing table due to a changed neighborhood.\n-spec update(OldRT::rt(), OldNeighbors::nodelist:neighborhood(),\n             NewNeighbors::nodelist:neighborhood())\n        -> {ok | trigger_rebuild, rt()}.\nupdate(OldRT, OldNeighbors, NewNeighbors) ->\n    NewPred = nodelist:pred(NewNeighbors),\n    OldSucc = nodelist:succ(OldNeighbors),\n    NewSucc = nodelist:succ(NewNeighbors),\n    NewNodeId = nodelist:nodeid(NewNeighbors),\n    % only re-build if a new successor occurs or the new node ID is not between\n    % Pred and Succ any more (which should not happen since this must come from\n    % a slide!)\n    case node:same_process(OldSucc, NewSucc) andalso\n             intervals:in(NewNodeId, node:mk_interval_between_nodes(NewPred, NewSucc)) of\n        true ->\n            % -> if not rebuilding, update the node IDs though\n            UpdNodes = nodelist:create_pid_to_node_dict(\n                         dict:new(), [nodelist:preds(NewNeighbors),\n                                      nodelist:succs(NewNeighbors)]),\n            NewRT = gb_trees:map(\n                      fun(_K, {Node, PidRT}) ->\n                              % check neighbors for newer version of the node\n                              case dict:find(node:pidX(Node), UpdNodes) of\n                                  {ok, N} -> {node:newer(Node, N), PidRT};\n                                  error -> {Node, PidRT}\n                              end\n                      end, OldRT),\n            {ok, NewRT};\n        false ->\n            % to be on the safe side ...\n            {trigger_rebuild, empty(NewNeighbors)}\n    end.\n%% userdevguide-end rt_chord:update\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Finger calculation\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% @private\n-spec calculateKey(key(), index()) -> key().\ncalculateKey(Id, {I, J}) ->\n    % N / K^I * (J + 1)\n    Offset = (n() div util:pow(config:read(chord_base), I)) * (J + 1),\n    %io:format(\"~p: ~p + ~p~n\", [{I, J}, Id, Offset]),\n    normalize(Id + Offset).\n\n%% @doc Gets the first index not pointing to the own node.\n-spec first_index(Neighbors::nodelist:neighborhood()) -> {key(), index()} | null.\nfirst_index(Neighbors) ->\n    MyRange = nodelist:node_range(Neighbors),\n    case intervals:is_all(MyRange) of\n        true  -> null;\n        false -> Id = nodelist:nodeid(Neighbors),\n                 first_index_(Id, MyRange, first_index())\n    end.\n\n%% @doc Helper for first_index/1.\n-spec first_index_(Id::key(), MyRange::intervals:interval(), index())\n        -> {key(), index()}.\nfirst_index_(Id, MyRange, Index) ->\n    Key = calculateKey(Id, Index),\n    case intervals:in(Key, MyRange) of\n        true  -> first_index_(Id, MyRange, next_index(Index));\n        false -> {Key, Index}\n    end.\n\n%% @doc Returns the first possible index, i.e. the index of the longest finger,\n%%      for the configured chord_base.\n-spec first_index() -> index().\nfirst_index() ->\n   {1, config:read(chord_base) - 2}.\n\n%% @doc Calculates the next index, i.e. the index for the next shorter finger,\n%%      for the configured chord_base.\n-spec next_index(index()) -> index().\nnext_index({I, 0}) ->\n    {I + 1, config:read(chord_base) - 2};\nnext_index({I, J}) ->\n    {I, J - 1}.\n\n%% @doc Calculates the previous index, i.e. the index for the next longer finger,\n%%      for the configured chord_base.\n-spec prev_index(index()) -> index().\nprev_index({I, J}) ->\n    MaxJ = config:read(chord_base) - 2,\n    if J =:= MaxJ andalso I > 1 -> {I - 1, 0};\n       J =/= MaxJ -> {I, J + 1}\n    end.\n\n%% @doc Checks whether config parameters of the rt_chord process exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(chord_base) and\n        config:cfg_is_greater_than_equal(chord_base, 2) and\n        config:cfg_is_integer(replication_factor) and\n        config:cfg_is_greater_than_equal(replication_factor, 2) and\n        config:cfg_is_in(key_creator, [random, random_with_bit_mask, modr]) and\n        case config:read(key_creator) of\n            random -> true;\n            random_with_bit_mask ->\n                config:cfg_is_tuple(key_creator_bitmask, 2,\n                                fun({Mask1, Mask2}) ->\n                                        erlang:is_integer(Mask1) andalso\n                                            erlang:is_integer(Mask2) end,\n                                \"{int(), int()}\");\n            modr -> config:cfg_is_integer(replication_factor) and\n                        config:cfg_is_integer(availability_zone_id);\n            _ -> false\n        end.\n\n%% @doc No special handling of messages, i.e. all messages are queued.\n-spec handle_custom_message_inactive(custom_message(), msg_queue:msg_queue()) ->\n    msg_queue:msg_queue().\nhandle_custom_message_inactive(Msg, MsgQueue) ->\n    msg_queue:add(MsgQueue, Msg).\n\n%% userdevguide-begin rt_chord:handle_custom_message\n%% @doc Chord reacts on 'rt_get_node_response' messages in response to its\n%%      'rt_get_node' messages.\n-spec handle_custom_message(custom_message(), rt_loop:state_active()) ->\n                                   rt_loop:state_active() | unknown_event.\nhandle_custom_message({rt_get_node, Source_PID, Index}, State) ->\n    MyNode = nodelist:node(rt_loop:get_neighb(State)),\n    comm:send(Source_PID, {rt_get_node_response, Index, MyNode, comm:this()}, ?SEND_OPTIONS),\n    State;\nhandle_custom_message({rt_get_node_response, Index, Node, PidRT}, State) ->\n    OldRT = rt_loop:get_rt(State),\n    OldERT = rt_loop:get_ert(State),\n    Neighbors = rt_loop:get_neighb(State),\n    NewERT = case stabilize(Neighbors, OldRT, Index, Node, PidRT) of\n                 {NewRT, true} ->\n                     check_do_update(OldRT, NewRT, OldERT, Neighbors, true);\n                 {NewRT, false} ->\n                     OldERT\n             end,\n    rt_loop:set_ert(rt_loop:set_rt(State, NewRT), NewERT);\nhandle_custom_message(_Message, _State) ->\n    unknown_event.\n%% userdevguide-end rt_chord:handle_custom_message\n\n%% userdevguide-begin rt_chord:check\n%% @doc Notifies the dht_node and failure detector if the routing table changed.\n%%      Provided for convenience (see check/5).\n-spec check(OldRT::rt(), NewRT::rt(), OldERT::external_rt(), Neighbors::nodelist:neighborhood(),\n            ReportToFD::boolean()) -> NewERT::external_rt().\ncheck(OldRT, OldRT, OldERT, _Neighbors, _ReportToFD) ->\n    OldERT;\ncheck(OldRT, NewRT, OldERT, Neighbors, ReportToFD) ->\n    check_do_update(OldRT, NewRT, OldERT, Neighbors, ReportToFD).\n\n%% @doc Notifies the dht_node if the (external) routing table changed.\n%%      Also updates the failure detector if ReportToFD is set.\n%%      Note: the external routing table also changes if the neighborhood changes.\n-spec check(OldRT::rt(), NewRT::rt(), OldERT::external_rt(),\n            OldNeighbors::nodelist:neighborhood(), NewNeighbors::nodelist:neighborhood(),\n            ReportToFD::boolean()) -> NewERT::external_rt().\ncheck(OldRT, NewRT, OldERT, OldNeighbors, NewNeighbors, ReportToFD) ->\n    case OldNeighbors =:= NewNeighbors andalso OldRT =:= NewRT of\n        true -> OldERT;\n        _ -> check_do_update(OldRT, NewRT, OldERT, NewNeighbors, ReportToFD)\n    end.\n\n%% @doc Helper for check/4 and check/5.\n-spec check_do_update(OldRT::rt(), NewRT::rt(), OldERT::external_rt(),\n                      NewNeighbors::nodelist:neighborhood(),\n                      ReportToFD::boolean()) -> ERT::external_rt().\ncheck_do_update(OldRT, NewRT, OldERT, NewNeighbors, ReportToFD) ->\n    % update failure detector:\n    case ReportToFD of\n        true ->\n            NewPids = to_pid_list(NewRT),\n            OldPids = to_pid_list(OldRT),\n            fd:update_subscriptions(self(), OldPids, NewPids);\n        _ -> ok\n    end,\n    case pid_groups:get_my(dht_node) of\n        failed ->\n            % TODO: can this really happen?!\n            OldERT;\n        Pid ->\n            NewERT = export_rt_to_dht_node(NewRT, NewNeighbors),\n            comm:send_local(Pid, {rt_update, NewERT}),\n            NewERT\n    end.\n%% userdevguide-end rt_chord:check\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Communication with dht_node\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% userdevguide-begin rt_chord:empty_ext\n-spec empty_ext(nodelist:neighborhood()) -> external_rt().\nempty_ext(_Neighbors) -> gb_trees:empty().\n%% userdevguide-end rt_chord:empty_ext\n\n%% userdevguide-begin rt_chord:next_hop\n%% @doc Returns the next hop to contact for a lookup.\n%%      Note, that this code will be called from the dht_node process and\n%%      it will thus have an external_rt!\n-spec next_hop(nodelist:neighborhood(), external_rt(), key()) -> succ | comm:mypid().\nnext_hop(Neighbors, RT, Id) ->\n    case intervals:in(Id, nodelist:succ_range(Neighbors)) of\n        true ->\n            succ;\n        false ->\n            % check routing table:\n            RTSize = get_size_ext(RT),\n            case util:gb_trees_largest_smaller_than(Id, RT) of\n                {value, _Key, Node} ->\n                    Node;\n                nil when RTSize =:= 0 ->\n                    Succ = nodelist:succ(Neighbors),\n                    node:pidX(Succ);\n                nil -> % forward to largest finger\n                    {_Key, Node} = gb_trees:largest(RT),\n                    Node\n            end\n    end.\n%% userdevguide-end rt_chord:next_hop\n\n%% @doc Return the succ, but get the pid from ERT if possible\n%%      (to hopefully get a rt_loop pid instead of a dht_node state pid)\n-spec succ(ERT::external_rt(), Neighbors::nodelist:neighborhood()) -> comm:mypid().\nsucc(ERT, Neighbors) ->\n    Succ = nodelist:succ(Neighbors),\n    case gb_trees:lookup(node:id(Succ), ERT) of\n        {value, Pid} -> Pid;\n        none -> node:pidX(Succ)\n    end.\n\n%% userdevguide-begin rt_chord:export_rt_to_dht_node\n-spec export_rt_to_dht_node(rt(), Neighbors::nodelist:neighborhood()) -> external_rt().\nexport_rt_to_dht_node(RT, Neighbors) ->\n    Id = nodelist:nodeid(Neighbors),\n    %% include whole neighbourhood external routing table (ert)\n    %% note: we are subscribed at the RM for changes to whole neighborhood\n    Preds = nodelist:preds(Neighbors),\n    Succs = nodelist:succs(Neighbors),\n    EnterDhtNode = fun(Node, Tree) ->\n                           gb_trees:enter(node:id(Node), node:pidX(Node), Tree)\n                   end,\n    Tree0 = lists:foldl(EnterDhtNode, gb_trees:empty(), Preds),\n    Tree1 = lists:foldl(EnterDhtNode, Tree0, Succs),\n    ERT = util:gb_trees_foldl(\n            fun (_Key, {Node, PidRT}, Acc) ->\n                     % only store the id and the according PidRT Pid\n                     case node:id(Node) of\n                         Id -> Acc;\n                         _  -> gb_trees:enter(node:id(Node), PidRT, Acc)\n                     end\n            end, Tree1, RT),\n    ERT.\n%% userdevguide-end rt_chord:export_rt_to_dht_node\n\n%% @doc Converts the (external) representation of the routing table to a list of\n%%      {Id, Pid} tuples, in the order of the fingers, i.e. first=succ,\n%%      second=shortest finger, third=next longer finger,...\n-spec to_list(dht_node_state:state()) -> [{key(), comm:mypid()}].\nto_list(State) ->\n    ERT = dht_node_state:get(State, rt),\n    MyNodeId = dht_node_state:get(State, node_id),\n    lists:usort(fun({AId, _APid}, {BId, _BPid}) ->\n                        nodelist:succ_ord_id(AId, BId, MyNodeId)\n                end, gb_trees:to_list(ERT)).\n\n%% userdevguide-begin rt_chord:wrap_message\n%% @doc Wrap lookup messages. This is a noop in Chord.\n-spec wrap_message(Key::key(), Msg::comm:message(), MyERT::external_rt(),\n                   Neighbors::nodelist:neighborhood(),\n                   Hops::non_neg_integer()) -> comm:message().\nwrap_message(_Key, Msg, _ERT, _Neighbors, _Hops) -> Msg.\n%% userdevguide-end rt_chord:wrap_message\n\n%% userdevguide-begin rt_chord:unwrap_message\n%% @doc Unwrap lookup messages. This is a noop in Chord.\n-spec unwrap_message(Msg::comm:message(), State::dht_node_state:state()) -> comm:message().\nunwrap_message(Msg, _State) -> Msg.\n%% userdevguide-end rt_chord:unwrap_message\n"
  },
  {
    "path": "src/rt_frt.erl",
    "content": "% @copyright 2013-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Magnus Mueller <mamuelle@informatik.hu-berlin.de>\n%% @author Jens Fischer <jensvfischer@gmail.com>\n\n%% @doc Flexible routing table. Two flexible routing tables (FRTs) are implemented,\n%% FRT-chord and grouped FRT-chord (GFRT). The functions specific to these two\n%% implementations can be found at the end of this file (seperated by ifdefs).\n%%\n%%\n%% @end\n%% @version $Id$\n-module(rt_frt).\n-author('mamuelle@informatik.hu-berlin.de').\n-behaviour(rt_beh).\n\n-export([dump_to_csv/1, get_source_id/1, get_source_node/1]).\n\n% exports for unit tests\n-export([check_rt_integrity/1, check_well_connectedness/1, get_random_key_from_generator/3]).\n\n% Functions which are specific to the frt/gfrt implementation\n-export([allowed_nodes/1, frt_check_config/0, rt_entry_info/4]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%% RT Implementation\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-type key() :: rt_chord:key().\n-type external_rt_t_tree() :: gb_trees:tree(NodeId::key(), Node::mynode()).\n-opaque external_rt() :: {Size::unknown | float(), external_rt_t_tree()}. %% @todo: make opaque\n\n% define the possible types of nodes in the routing table:\n%  - normal nodes are nodes which have been added by entry learning\n%  - source is the node at which this RT is\n%  - sticky is a node which is not to be deleted with entry filtering\n-type entry_type() :: normal | source | sticky.\n-type custom_info() :: undefined | term().\n\n-type(mynode() :: {Id::key(), IdVersion::non_neg_integer(),\n                   DHTNodePid::comm:mypid(), RTLoopPid::comm:mypid() | none}).\n-record(rt_entry, {\n        node :: mynode(),\n        type :: entry_type(),\n        adjacent_fingers = {undefined, undefined} :: {key() |\n                                                      'undefined', key() |\n                                                      'undefined'},\n        custom = undefined :: custom_info()\n    }).\n\n-type(rt_entry() :: #rt_entry{}).\n\n-export_type([rt_entry/0]).\n\n-type rt_t_tree() :: gb_trees:tree(NodeId::key(), rt_entry()).\n-record(rt_t, {\n        source = undefined :: key() | undefined\n        , num_active_learning_lookups = 0 :: non_neg_integer()\n        , nodes = gb_trees:empty() :: rt_t_tree()\n        , nodes_in_ring = unknown :: unknown | float()\n        %% , nodes_in_ring = unknown :: Size :: unknown | float()\n    }).\n\n-opaque rt() :: #rt_t{}.\n\n-type custom_message() :: {get_rt, SourcePID::comm:mypid()}\n                        | {get_rt_reply, RT::rt()}\n                        | {trigger_random_lookup}\n                        | {rt_get_node, From::comm:mypid()}\n                        | {rt_learn_node, NewNode::mynode()}\n                        | {rt_get_neighbor, From::comm:mypid()}\n                        | {rt_learn_neighbor, NewNode::mynode()}.\n\n-include(\"scalaris.hrl\").\n-include(\"rt_beh.hrl\").\n\n% @doc Maximum number of entries in a routing table\n-spec maximum_entries() -> non_neg_integer().\nmaximum_entries() -> config:read(rt_frt_max_entries).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Key Handling\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc This function is called during the startup of the rt_loop process and\n%%      is allowed to send trigger messages.\n-spec init() -> ok.\ninit() ->\n    % initializse the trigger for random lookups\n    case config:read(rt_frt_al) of\n        true -> msg_delay:send_trigger(0, {trigger_random_lookup});\n        false -> ok\n    end.\n\n%% @doc Activate the routing table.\n%%      This function is called during the activation of the routing table process.\n-spec activate(nodelist:neighborhood()) -> rt().\nactivate(Neighbors) ->\n    % ask the successor node for its routing table\n    Msg = {?send_to_group_member, routing_table, {get_rt, comm:this()}},\n    comm:send(node:pidX(nodelist:succ(Neighbors)), Msg),\n\n    % request approximated ring size\n    gossip_load:get_values_best([]),\n\n    MyNode = node2mynode(nodelist:node(Neighbors), comm:this()),\n    update_entries(Neighbors, add_source_entry(MyNode, #rt_t{})).\n\n%% @doc Hashes the key to the identifier space.\n-spec hash_key(client_key() | binary()) -> key().\nhash_key(Key) -> rt_chord:hash_key(Key).\n%% userdevguide-end rt_frtchord:hash_key\n\n%% userdevguide-begin rt_frtchord:get_random_node_id\n%% @doc Generates a random node id, i.e. a random 128-bit number.\n-spec get_random_node_id() -> key().\nget_random_node_id() -> rt_chord:get_random_node_id().\n%% userdevguide-end rt_frtchord:get_random_node_id\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% RT Management\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% userdevguide-begin rt_frtchord:init_stabilize\n%% @doc Triggered by a new stabilization round, renews the routing table.\n%% Check:\n%% - if node id didn't change, i.e. only preds/succs changed, update sticky entries\n%% - if node id changed, just renew the complete table and maybe tell known nodes\n%%  that something changed (async and optimistic -> if they don't care, we don't care)\n-spec init_stabilize(nodelist:neighborhood(), rt()) -> rt().\ninit_stabilize(Neighbors, RT) ->\n    %% log:pal(\"[~w] init_stabilize\", [self()]),\n    case node:id(nodelist:node(Neighbors)) =:= entry_nodeid(get_source_node(RT)) of\n        true ->\n            update_entries(Neighbors, RT) ;\n        false -> % source node changed, replace the complete table\n            activate(Neighbors)\n    end\n    .\n%% userdevguide-end rt_frtchord:init_stabilize\n\n%% @doc Update sticky entries\n%% This function converts sticky nodes from the RT which aren't in the neighborhood\n%% anymore to normal nodes. Afterwards it adds new nodes from the neighborhood.\n-spec update_entries(NewNeighborhood::nodelist:neighborhood(), RT::rt()) -> rt().\nupdate_entries(NewNeighborhood, RT) ->\n    %% neighborhood nodes (sticky nodes) from RT\n    OldNeighbors1 = lists:map(fun rt_entry_node/1, get_sticky_entries(RT)),\n    OldNeighbors2 = lists:map(fun(Node) -> {id(Node), pid_dht(Node)} end, OldNeighbors1),\n    OldNeighbors3 = sets:from_list(OldNeighbors2),\n    %% OldNeighbors = sets:from_list(tl(nodelist:to_list(OldNeighborhood))),\n\n    %% neighborhood nodes (succs + preds w/o self) from new neighborhood from rm\n    NewNeighbors1 = tl(nodelist:to_list(NewNeighborhood)),\n    NewNeighbors2 = lists:map(fun(Node) -> {node:id(Node), node:pidX(Node)} end, NewNeighbors1),\n    NewNeighbors3 = sets:from_list(NewNeighbors2),\n\n    %% former neighborhood nodes which aren't neighboorhod nodes anymore\n    ConvertNodes = sets:subtract(OldNeighbors3, NewNeighbors3),\n    ConvertNodesIds = util:sets_map(fun({Id, _Pid}) -> Id end, ConvertNodes),\n\n    %% new neighboorhod nodes\n    ToBeAddedNodes1 = sets:subtract(NewNeighbors3, OldNeighbors3),\n    %% Add PidRTs to the nodes to be added. There might be PidRTs in the old RT\n    %% even for \"new\" nodes, e.g. for nodes added through get_rt_reply (i.e. for\n    %% normal nodes that are converted to sticky nodes). Get the VersionIds from\n    %% the list of new nodes.\n    OldNodes = lists:map(fun rt_entry_node/1, gb_trees:values(get_rt_tree(RT))),\n    NewNodes = lists:map(fun(Node) -> {node:id(Node), node:id_version(Node),\n                                       node:pidX(Node)}\n                         end, NewNeighbors1),\n    ToBeAddedNodes2 =\n        util:sets_map(fun({Id, PidDHT}) ->\n                              PidRT = case lists:keyfind(Id, 1, OldNodes) of\n                                  {_Id, _IdVersion, _PidDHT, PidRT0} -> PidRT0;\n                                  _else -> none\n                              end,\n                              {_, IdVersion, _} = lists:keyfind(Id, 1, NewNodes),\n                              {Id, IdVersion, PidDHT, PidRT}\n                      end, ToBeAddedNodes1),\n\n    %% uncomment for debugging\n    %% case sets:size(ConvertNodes) > 0 orelse sets:size(ToBeAddedNodes1) > 0 of\n    %%     true ->\n    %%         log:pal(\"~w~nOldNeighbors:~n~190.2p~nNewNeighors:~n~190.2p~n\" ++\n    %%                 \"ConvertNodes:~n~190.2p~nToBeAddedNodes~n~190.2p~n\",\n    %%                 [self(), OldNeighbors1, NewNeighbors1, sets:to_list(ConvertNodes),\n    %%                  ToBeAddedNodes2]);\n    %%     false -> ok\n    %% end,\n\n    %% convert former neighboring nodes to normal nodes and add sticky nodes\n    FilteredRT = lists:foldl(fun sticky_entry_to_normal_node/2, RT, ConvertNodesIds),\n    NewRT = lists:foldl(fun add_sticky_entry/2, FilteredRT, ToBeAddedNodes2),\n\n    %% request rt_loop pids of new neighbours with unknown PidRTs\n    ToBeAddedNodes3 = lists:filter(fun({_Id, _IdVersion, _PidDHT, PidRT}) ->\n                                           PidRT =:= none\n                                   end, ToBeAddedNodes2),\n    lists:foreach(fun(Node) -> comm:send(pid_dht(Node),\n                                         {?send_to_group_member, routing_table,\n                                          {rt_get_neighbor, comm:this()}})\n                  end, ToBeAddedNodes3),\n\n    check_helper(RT, NewRT, true),\n    NewRT.\n\n%% userdevguide-begin rt_frtchord:update\n%% @doc Updates the routing table due to a changed node ID, pred and/or succ.\n%% - We must rebuild the complete routing table when the source node id changed\n%% - If only the preds/succs changed, adapt the old routing table\n-spec update(OldRT::rt(), OldNeighbors::nodelist:neighborhood(),\n    NewNeighbors::nodelist:neighborhood()) -> {trigger_rebuild, rt()} | {ok, rt()}.\nupdate(OldRT, Neighbors, Neighbors) -> {ok, OldRT};\nupdate(OldRT, OldNeighbors, NewNeighbors) ->\n    case nodelist:node(OldNeighbors) =:= nodelist:node(NewNeighbors) of\n        true -> % source node didn't change\n            % update the sticky nodes: delete old nodes and add new nodes\n            {ok, update_entries(NewNeighbors, OldRT)};\n        _Else -> % source node changed, rebuild the complete table\n            {trigger_rebuild, OldRT}\n    end\n    .\n%% userdevguide-end rt_frtchord:update\n\n%% userdevguide-begin rt_frtchord:filter_dead_node\n%% @doc Removes dead nodes from the routing table (rely on periodic\n%%      stabilization here).\n-spec filter_dead_node(rt(), DeadPid::comm:mypid(), Reason::fd:reason()) -> rt().\nfilter_dead_node(RT, DeadPid, _Reason) ->\n    % find the node id of DeadPid and delete it from the RT\n    case [N || N <- internal_to_list(RT), pid_dht(N) =:= DeadPid] of\n        [Node] -> entry_delete(id(Node), RT);\n        [] -> RT\n    end\n    .\n%% userdevguide-end rt_frtchord:filter_dead_node\n\n%% userdevguide-begin rt_frtchord:to_pid_list\n%% @doc Returns the pids of the routing table entries.\n-spec to_pid_list(rt()) -> [comm:mypid()].\nto_pid_list(RT) -> [pid_dht(N) || N <- internal_to_list(RT)].\n%% userdevguide-end rt_frtchord:to_pid_list\n\n%% @doc Get the size of the RT excluding entries which are not tagged as normal entries.\n-spec get_size_without_special_nodes(rt()) -> non_neg_integer().\nget_size_without_special_nodes(#rt_t{} = RT) ->\n    util:gb_trees_foldl(\n        fun(_Key, Val, Acc) ->\n                Acc + case entry_type(Val) of\n                    normal -> 1;\n                    _else -> 0\n                end\n        end, 0, get_rt_tree(RT)).\n\n%% userdevguide-begin rt_frtchord:get_size\n%% @doc Returns the size of the routing table.\n-spec get_size(rt()) -> non_neg_integer().\nget_size(#rt_t{} = RT) -> gb_trees:size(get_rt_tree(RT)).\n\n%% @doc Returns the size of the external routing table.\n-spec get_size_ext(external_rt()) -> non_neg_integer().\nget_size_ext(RT) ->\n    gb_trees:size(external_rt_get_tree(RT)).\n%% userdevguide-end rt_frtchord:get_size\n\n%% userdevguide-begin rt_frtchord:n\n%% @doc Returns the size of the address space.\n-spec n() -> 16#100000000000000000000000000000000.\nn() -> 16#100000000000000000000000000000000.\n%% userdevguide-end rt_frtchord:n\n\n%% @doc Gets the number of keys in the interval (Begin, End]. In the special\n%%      case of Begin==End, the whole key range as specified by n/0 is returned.\n-spec get_range(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE) -> non_neg_integer().\nget_range(Begin, End) -> rt_chord:get_range(Begin, End).\n\n%% @doc Gets the key that splits the interval (Begin, End] so that the first\n%%      interval will be (Num/Denom) * range(Begin, End). In the special case of\n%%      Begin==End, the whole key range is split.\n%%      Beware: SplitFactor must be in [0, 1]; the final key will be rounded\n%%      down and may thus be Begin.\n-spec get_split_key(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE,\n                    SplitFraction::{Num::non_neg_integer(), Denom::pos_integer()}) -> key() | ?PLUS_INFINITY_TYPE.\nget_split_key(Begin, End, SplitFraction) ->\n    rt_chord:get_split_key(Begin, End, SplitFraction).\n\n%% @doc Splits the range between Begin and End into up to Parts equal parts and\n%%      returning the according split keys.\n-spec get_split_keys(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE,\n                     Parts::pos_integer()) -> [key()].\nget_split_keys(Begin, End, Parts) ->\n    rt_chord:get_split_keys(Begin, End, Parts).\n\n%% @doc Gets input similar to what intervals:get_bounds/1 returns and\n%%      calculates a random key in this range. Fails with an exception if there\n%%      is no key.\n-spec get_random_in_interval(intervals:simple_interval2()) -> key().\nget_random_in_interval(SimpleI) ->\n    rt_chord:get_random_in_interval(SimpleI).\n\n%% @doc Gets input similar to what intervals:get_bounds/1 returns and\n%%      calculates Count number of random keys in this range (duplicates may\n%%      exist!). Fails with an exception if there is no key.\n-spec get_random_in_interval(intervals:simple_interval2(), Count::pos_integer()) -> [key(),...].\nget_random_in_interval(SimpleI, Count) ->\n    rt_chord:get_random_in_interval(SimpleI, Count).\n\n%% userdevguide-begin rt_frtchord:get_replica_keys\n%% @doc Returns the replicas of the given key.\n-spec get_replica_keys(key()) -> [key()].\nget_replica_keys(Key) ->\n    rt_simple:get_replica_keys(Key).\n\n-spec get_replica_keys(key(), pos_integer()) -> [key()].\nget_replica_keys(Key, ReplicationFactor) ->\n    rt_simple:get_replica_keys(Key, ReplicationFactor).\n%% userdevguide-end rt_frtchord:get_replica_keys\n\n-spec get_key_segment(key()) -> pos_integer().\nget_key_segment(Key) ->\n    rt_simple:get_key_segment(Key).\n\n-spec get_key_segment(key(), pos_integer()) -> pos_integer().\nget_key_segment(Key, ReplicationFactor) ->\n    rt_simple:get_key_segment(Key, ReplicationFactor).\n\n%% userdevguide-begin rt_frtchord:dump\n%% @doc Dumps the RT state for output in the web interface.\n-spec dump(RT::rt()) -> KeyValueList::[{Index::string(), Node::string()}].\ndump(RT) -> [{\"0\", webhelpers:safe_html_string(\"~p\", [RT])}].\n%% userdevguide-end rt_frtchord:dump\n\n% @doc Dump the routing table into a CSV string\n-spec dump_to_csv(RT::rt()) -> [char()].\ndump_to_csv(RT) ->\n    Fingers = internal_to_list(RT),\n    IndexedFingers = lists:zip(lists:seq(1,length(Fingers)), Fingers),\n    MyId = get_source_id(RT),\n    lists:flatten(\n        [\n            \"Finger,Id\\n\"\n            , io_lib:format(\"0,~p~n\", [MyId])\n        ] ++\n        [\n            io_lib:format(\"~p,~p~n\",[Index,id(Finger)])\n            || {Index, Finger} <- IndexedFingers\n        ]\n    )\n    .\n\n\n%% @doc Checks whether config parameters of the rt_frtchord process exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_in(key_creator, [random, random_with_bit_mask]) and\n        case config:read(key_creator) of\n            random -> true;\n            random_with_bit_mask ->\n                config:cfg_is_tuple(key_creator_bitmask, 2,\n                                fun({Mask1, Mask2}) ->\n                                        erlang:is_integer(Mask1) andalso\n                                            erlang:is_integer(Mask2) end,\n                                \"{int(), int()}\");\n            _ -> false\n        end and\n    config:cfg_is_bool(rt_frt_al) and\n    config:cfg_is_greater_than_equal(rt_frt_al_interval, 0) and\n    config:cfg_is_integer(rt_frt_max_entries) and\n    config:cfg_is_greater_than(rt_frt_max_entries, 0) and\n    config:cfg_is_integer(rt_frt_max_entries) and\n    config:cfg_is_greater_than(rt_frt_max_entries, 0) and\n    config:cfg_is_integer(rt_frt_gossip_interval) and\n    config:cfg_is_greater_than(rt_frt_gossip_interval, 0) and\n    config:cfg_is_in(rt_frt_reduction_ratio_strategy,\n        [best_rt_reduction_ratio, convergent_rt_reduction_ratio]) and\n    frt_check_config()\n    .\n\n%% @doc Generate a random key from the pdf as defined in (Nagao, Shudo, 2011)\n%% TODO I floor the key for now; the key generator should return ints, but returns\n%float. It is currently unclear if this is a bug in the original paper by Nagao and\n%Shudo. Using erlang:trunc/1 should be enough for flooring, as X >= 0\n-spec get_random_key_from_generator(SourceNodeId::key(),\n                                    PredId::key(),\n                                    SuccId::key()\n                                   ) -> key().\nget_random_key_from_generator(SourceNodeId, PredId, SuccId) ->\n    Rand = randoms:uniform(),\n    X = erlang:trunc(get_range(SourceNodeId, SuccId) *\n                         math:pow(get_range(SourceNodeId, PredId) /\n                                      get_range(SourceNodeId, SuccId),\n                                  Rand\n                                 )),\n    rt_chord:add_range(SourceNodeId, X).\n\n%% @doc Trigger need to be resend here w/o queuing to avoid trace infestation.\n-spec handle_custom_message_inactive(custom_message(), msg_queue:msg_queue()) ->\n    msg_queue:msg_queue().\n% resend trigger\nhandle_custom_message_inactive({trigger_random_lookup}, MsgQueue) ->\n    Interval = config:read(rt_frt_al_interval),\n    msg_delay:send_trigger(Interval, {trigger_random_lookup}),\n    MsgQueue;\n% queue all other messages\nhandle_custom_message_inactive(Msg, MsgQueue) ->\n    msg_queue:add(MsgQueue, Msg).\n\n\n%% userdevguide-begin rt_frtchord:handle_custom_message\n%% @doc Handle custom messages. The following messages can occur:\n%%      - TODO explain messages\n\n% send the RT to a node asking for it\n-spec handle_custom_message(custom_message(), rt_loop:state_active()) ->\n                                   rt_loop:state_active() | unknown_event.\nhandle_custom_message({get_rt, Pid}, State) ->\n    comm:send(Pid, {get_rt_reply, rt_loop:get_rt(State)}),\n    State\n    ;\n\nhandle_custom_message({get_rt_reply, RT}, State) ->\n    %% merge the routing tables. Note: We don't care if this message is not from our\n    %% current successor. We just have to make sure that the merged entries are valid.\n    OldRT = rt_loop:get_rt(State),\n    NewRT = case OldRT =/= RT of\n        true ->\n            % - add each entry from the other RT if it doesn't already exist\n            % - entries are added as normal entries. RM (-> update()) invokes\n            %   adding sticky nodes or converting normal nodes to sticky nodes\n            %   respectively\n            util:gb_trees_foldl(\n                fun(Key, Entry, Acc) ->\n                        case entry_exists(Key, Acc) of\n                            true -> Acc;\n                            false -> add_normal_entry(rt_entry_node(Entry), Acc)\n                        end\n                end,\n                OldRT, get_rt_tree(RT));\n        false -> OldRT\n    end,\n    NewERT = check(OldRT, NewRT, rt_loop:get_ert(State),\n                    rt_loop:get_neighb(State), true),\n    rt_loop:set_ert(rt_loop:set_rt(State, NewRT), NewERT);\n\n% lookup a random key chosen with a pdf:\n% x = sourcenode + d(s,succ)*(d(s,pred)/d(s,succ))^rnd\n% where rnd is chosen uniformly from [0,1)\nhandle_custom_message({trigger_random_lookup}, State) ->\n    RT = rt_loop:get_rt(State),\n    SourceNode = get_source_node(RT),\n    SourceNodeId = entry_nodeid(SourceNode),\n    {PredId, SuccId} = adjacent_fingers(SourceNode),\n    Key = get_random_key_from_generator(SourceNodeId, PredId, SuccId),\n\n    % schedule the next random lookup\n    Interval = config:read(rt_frt_al_interval),\n    msg_delay:send_trigger(Interval, {trigger_random_lookup}),\n\n    api_dht_raw:unreliable_lookup(Key, {?send_to_group_member, routing_table,\n                                        {rt_get_node, comm:this()}}),\n    State\n    ;\n\nhandle_custom_message({rt_get_node, From}, State) ->\n    MyNode = nodelist:node(rt_loop:get_neighb(State)),\n    comm:send(From, {rt_learn_node, node2mynode(MyNode, comm:this())}),\n    State;\n\nhandle_custom_message({rt_learn_node, NewNode}, State) ->\n    OldRT = rt_loop:get_rt(State),\n    {NewRT, NewERT} =\n        case rt_lookup_node(id(NewNode), OldRT) of\n            none ->\n                RT = add_normal_entry(NewNode, OldRT),\n                ERT = check(OldRT, RT, rt_loop:get_ert(State),\n                                rt_loop:get_neighb(State), true),\n                {RT, ERT};\n            {value, _RTEntry} -> {OldRT, rt_loop:get_ert(State)}\n        end,\n    rt_loop:set_ert(rt_loop:set_rt(State, NewRT), NewERT);\n\nhandle_custom_message({rt_get_neighbor, From}, State) ->\n    MyNode = nodelist:node(rt_loop:get_neighb(State)),\n    comm:send(From, {rt_learn_neighbor, node2mynode(MyNode, comm:this())}),\n    State;\n\nhandle_custom_message({rt_learn_neighbor, {Id, IdVersion, PidDHT, PidRT}}, State) ->\n    OldRT = rt_loop:get_rt(State),\n    {NewRT, NewERT} =\n        case rt_lookup_node(Id, OldRT) of\n            %% if no entry exists: don't add\n            %% this could be due to\n            %% 1) nodes which were neighbors when sending the rt_get_neighbor\n            %%    request, but aren't neighbors anymore and have been filtered\n            %%    out of the rt since\n            %% 2) Interleavings of requesting the rt pids and crashing nodes\n            none -> {OldRT, rt_loop:get_ert(State)};\n\n            %% only update the RT if the PidRT is unknown\n            {value, OldEntry} ->\n                OldNode = rt_entry_node(OldEntry),\n                case OldNode =:= {Id, IdVersion, PidDHT, none} of\n                    true ->\n                        %% update the rt pid\n                        OldEntry= rt_get_node(Id, OldRT),\n                        NewNode = {Id, IdVersion, PidDHT, PidRT},\n                        NewEntry = rt_entry_set_node(OldEntry, NewNode),\n                        OldNodeTree = get_rt_tree(OldRT),\n                        NewNodeTree = gb_trees:update(Id, NewEntry, OldNodeTree),\n                        RT = rt_set_nodes(OldRT, NewNodeTree),\n                        ERT = check(OldRT, RT, rt_loop:get_ert(State),\n                                    rt_loop:get_neighb(State), true),\n                        %% print_debug(true, OldNode, {Id, PidDHT, PidRT}, ERT),\n                        {RT, ERT};\n                    false ->\n                        %% print_debug(false, OldNode, {Id, PidDHT, PidRT}, rt_loop:get_ert(State)),\n                        {OldRT, rt_loop:get_ert(State)}\n                end\n        end,\n    rt_loop:set_ert(rt_loop:set_rt(State, NewRT), NewERT);\n\nhandle_custom_message({gossip_get_values_best_response, Vals}, State) ->\n    RT = rt_loop:get_rt(State),\n    NewRT = rt_set_ring_size(RT, gossip_load:load_info_get(size_kr, Vals)),\n    gossip_load:get_values_best([{msg_delay, config:read(rt_frt_gossip_interval)}]),\n    ERT = export_rt_to_dht_node(NewRT, rt_loop:get_neighb(State)),\n    comm:send_local(rt_loop:get_pid_dht(State), {rt_update, ERT}),\n    rt_loop:set_ert(rt_loop:set_rt(State, NewRT), ERT);\n\nhandle_custom_message(_Message, _State) -> unknown_event.\n%% userdevguide-end rt_frtchord:handle_custom_message\n\n%% userdevguide-begin rt_frtchord:check\n%% @doc Notifies the dht_node and failure detector if the routing table changed.\n%%      Provided for convenience (see check/5).\n-spec check(OldRT::rt(), NewRT::rt(), OldERT::external_rt(),\n                Neighbors::nodelist:neighborhood(),\n                ReportToFD::boolean()) -> NewERT::external_rt().\ncheck(OldRT, NewRT, OldERT, Neighbors, ReportToFD) ->\n    check(OldRT, NewRT, OldERT, Neighbors, Neighbors, ReportToFD).\n\n%% @doc Notifies the dht_node if the (external) routing table changed.\n%%      Also updates the failure detector if ReportToFD is set.\n%%      Note: the external routing table only changes if the internal RT has\n%%      changed.\n-spec check(OldRT::rt(), NewRT::rt(), OldERT::external_rt(),\n                OldNeighbors::nodelist:neighborhood(),\n                NewNeighbors::nodelist:neighborhood(),\n                ReportToFD::boolean()) -> NewERT::external_rt().\ncheck(OldRT, NewRT, OldERT, OldNeighbors, NewNeighbors, ReportToFD) ->\n    % if the routing tables haven't changed and the successor/predecessor haven't changed\n    % as well, do nothing\n    case OldRT =:= NewRT andalso\n         nodelist:succ(OldNeighbors) =:= nodelist:succ(NewNeighbors) andalso\n         nodelist:pred(OldNeighbors) =:= nodelist:pred(NewNeighbors) of\n        true -> OldERT;\n        _Else -> export_to_dht(NewRT, ReportToFD)\n    end.\n\n% @doc Helper to send the new routing table to the dht node\n-spec export_to_dht(rt(), ReportToFD::boolean()) ->\n    NewERT::external_rt().\nexport_to_dht(NewRT, ReportToFD) ->\n    Pid = pid_groups:get_my(dht_node),\n    ERT = export_rt_to_dht_node_helper(NewRT),\n    case Pid of\n        failed -> ok;\n        _ -> comm:send_local(Pid, {rt_update, ERT})\n    end,\n    % update failure detector:\n    case ReportToFD of\n        true -> add_fd(NewRT);\n        _ -> ok\n    end,\n    ERT.\n\n% @doc Helper to check for routing table changes, excluding changes to the neighborhood.\n-spec check_helper(OldRT::rt(), NewRT::rt(), ReportToFD::boolean()) -> ok.\ncheck_helper(OldRT, NewRT, ReportToFD) ->\n    case OldRT =:= NewRT\n    of\n        true -> ok;\n        false -> export_to_dht(NewRT, ReportToFD)\n    end.\n\n%% @doc Filter the source node's pid from a list.\n-spec filter_source_pid(rt(), [comm:mypid()]) -> [comm:mypid()].\nfilter_source_pid(RT, ListOfPids) ->\n    SourcePid = pid_dht(rt_entry_node(get_source_node(RT))),\n    [P || P <- ListOfPids, P =/= SourcePid].\n\n%% @doc Set up a set of subscriptions from a routing table\n-spec add_fd(RT::rt())  -> ok.\nadd_fd(#rt_t{} = RT) ->\n    NewPids = to_pid_list(RT),\n    % Filter the source node from the Pids, as we don't need an FD for that node. If the\n    % source node crashes (which is the process calling this function), we are done\n    % for.\n    fd:subscribe(self(), filter_source_pid(RT, NewPids)).\n\n%% @doc Update subscriptions\n-spec update_fd(OldRT::rt(), NewRT::rt()) -> ok.\nupdate_fd(OldRT, OldRT) -> ok;\nupdate_fd(#rt_t{} = OldRT, #rt_t{} = NewRT) ->\n    OldPids = filter_source_pid(OldRT, to_pid_list(OldRT)),\n    NewPids = filter_source_pid(NewRT, to_pid_list(NewRT)),\n    fd:update_subscriptions(self(), OldPids, NewPids).\n\n%% userdevguide-end rt_frtchord:check\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Communication with routing_table and dht_node\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% userdevguide-begin rt_frtchord:empty_ext\n-spec empty_ext(nodelist:neighborhood()) -> external_rt().\nempty_ext(_Neighbors) -> {unknown, gb_trees:empty()}.\n%% userdevguide-end rt_frtchord:empty_ext\n\n%% userdevguide-begin rt_frtchord:next_hop\n%% @doc Returns the next hop to contact for a lookup.\n%% Beware: Sometimes called from dht_node\n-spec next_hop_node(nodelist:neighborhood(), external_rt(), key()) -> succ | mynode().\nnext_hop_node(Neighbors, ERT, Id) ->\n    case intervals:in(Id, nodelist:succ_range(Neighbors)) of\n        true -> succ;\n        _else -> RT = external_rt_get_tree(ERT),\n                 RTSize = get_size_ext(ERT),\n                 case util:gb_trees_largest_smaller_than(Id, RT) of\n                     {value, _Id1, Node} -> Node;\n                     nil when RTSize =:= 0 ->\n                         node2mynode(nodelist:succ(Neighbors));\n                     nil -> % forward to largest finger\n                         {_Id, Node} = gb_trees:largest(RT),\n                         Node\n                 end\n    end.\n\n-spec next_hop(nodelist:neighborhood(), external_rt(), key()) -> succ | comm:mypid().\nnext_hop(Neighbors, ERT, Id) ->\n    case next_hop_node(Neighbors, ERT, Id) of\n        succ -> succ;\n        %% prefer rt pid\n        Node -> case pid_rt(Node) of\n                    none -> pid_dht(Node);\n                    Pid -> Pid\n                end\n    end.\n%% userdevguide-end rt_frtchord:next_hop\n\n%% @doc Return the succ\n-spec succ(ERT::external_rt(), Neighbors::nodelist:neighborhood()) -> comm:mypid().\nsucc(ERT, Neighbors) ->\n    Succ = nodelist:succ(Neighbors),\n    ERTtree = external_rt_get_tree(ERT),\n    %% prefer pid of rt_loop\n    case gb_trees:lookup(node:id(Succ), ERTtree) of\n        {value, {_Key, _IdVersion, PidDHT, none}} -> PidDHT;\n        {value, {_Key, _IdVersion, _PidDHT, PidRT}} -> PidRT;\n        none -> node:pidX(Succ)\n    end.\n\n\n\n%% userdevguide-begin rt_frtchord:export_rt_to_dht_node\n%% @doc Converts the internal RT to the external RT used by the dht_node.\n%% The external routing table is optimized to speed up next_hop/2. For this, it is\n%%  only a gb_tree with keys being node ids and values being of type mynode().\n-spec export_rt_to_dht_node_helper(rt()) -> external_rt().\nexport_rt_to_dht_node_helper(RT) ->\n    % From each rt_entry, we extract only the field \"node\" and add it to the tree\n    % under the node id. The source node is filtered.\n    {RT#rt_t.nodes_in_ring, util:gb_trees_foldl(\n            fun(_K, V, Acc) ->\n                    case entry_type(V) of\n                        source -> Acc;\n                        _Else -> Node = rt_entry_node(V),\n                            gb_trees:enter(id(Node), Node, Acc)\n                    end\n            end, gb_trees:empty(),get_rt_tree(RT))}.\n\n-spec export_rt_to_dht_node(rt(), Neighbors::nodelist:neighborhood()) -> external_rt().\nexport_rt_to_dht_node(RT, _Neighbors) ->\n    export_rt_to_dht_node_helper(RT).\n%% userdevguide-end rt_frtchord:export_rt_to_dht_node\n\n%% userdevguide-begin rt_frtchord:to_list\n%% @doc Converts the external representation of the routing table to a list\n%%      in the order of the fingers, i.e. first=succ, second=shortest finger,\n%%      third=next longer finger,...\n-spec to_list(dht_node_state:state()) -> [{Id::key(), DHTPid::comm:mypid()}].\nto_list(State) ->\n    ERT = external_rt_get_tree(dht_node_state:get(State, rt)),\n    MyNodeId = dht_node_state:get(State, node_id),\n    L1 = util:gb_trees_foldl(fun(Id, {Id, _IdV, PidDHT, _PidRT}, Acc) ->\n                                     [{Id, PidDHT}|Acc]\n                             end, [], ERT),\n    lists:usort(fun({AId, _APid}, {BId, _PPid}) ->\n                        nodelist:succ_ord_id(AId, BId, MyNodeId)\n                end, L1).\n\n%% @doc Converts the internal representation of the routing table to a list\n%%      in the order of the fingers, i.e. first=succ, second=shortest finger,\n%%      third=next longer finger,...\n-spec internal_to_list(rt()) -> [mynode()].\ninternal_to_list(#rt_t{} = RT) ->\n    SourceNode = get_source_node(RT),\n    ListOfNodes = [rt_entry_node(N) || N <- gb_trees:values(get_rt_tree(RT))],\n    sorted_nodelist(ListOfNodes, id(rt_entry_node(SourceNode)))\n    .\n\n% @doc Helper to do the actual work of converting a list of mynode() records\n%      to list beginning with the source node and wrapping around at 0\n-spec sorted_nodelist(ListOfNodes::[mynode()], SourceNode::key()) -> [mynode()].\nsorted_nodelist(ListOfNodes, SourceNode) ->\n    % sort\n    Sorted = lists:sort(fun(A, B) -> id(A) =< id(B) end,\n        ListOfNodes),\n    % rearrange elements: all until the source node must be attached at the end\n    {Front, Tail} = lists:splitwith(fun(N) -> id(N) =< SourceNode end, Sorted),\n    Tail ++ Front\n    .\n%% userdevguide-end rt_frtchord:to_list\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% FRT specific algorithms and functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc Filter one element from a set of nodes. Do it in a way that filters such a node\n% that the resulting routing table is the best one under all _possible_ routing tables.\n\n-spec entry_filtering(rt()) -> rt().\nentry_filtering(#rt_t{} = RT) ->\n    entry_filtering(RT, allowed_nodes(RT)).\n\n-spec entry_filtering(rt(),[#rt_entry{type :: 'normal'}]) -> rt().\nentry_filtering(RT, []) -> RT; % only sticky entries and the source node given; nothing to do\nentry_filtering(RT, [_|_] = AllowedNodes) ->\n    Spacings = [\n        begin PredNode = predecessor_node(RT,Node),\n              {Node, spacing(Node, RT) + spacing(PredNode, RT)}\n        end || Node <- AllowedNodes],\n    % remove the element with the smallest canonical spacing range between its predecessor\n    % and its successor. TODO beware of numerical errors!\n    {FilterEntry, _Spacing} = hd(lists:sort(\n            fun ({_,SpacingA}, {_, SpacingB})\n                -> SpacingA =< SpacingB\n            end, Spacings)\n    ),\n    FilteredNode = rt_entry_node(FilterEntry),\n    entry_delete(id(FilteredNode), RT)\n    .\n\n% @doc Delete an entry from the routing table\n-spec entry_delete(EntryKey::key(), RT::rt()) -> RefinedRT::rt().\nentry_delete(EntryKey, RT) ->\n    Tree = gb_trees:delete(EntryKey, get_rt_tree(RT)),\n    Node = rt_get_node(EntryKey, RT),\n\n    % update affected routing table entries (adjacent fingers)\n    {PredId, SuccId} = adjacent_fingers(Node),\n    Pred = rt_get_node(PredId, RT),\n    Succ = rt_get_node(SuccId, RT),\n\n    UpdatedTree = case PredId =:= SuccId of\n        true ->\n            Entry = set_adjacent_fingers(Pred, PredId, PredId),\n            gb_trees:enter(PredId, Entry, Tree);\n        false ->\n            NewPred = set_adjacent_succ(Pred, SuccId),\n            NewSucc = set_adjacent_pred(Succ, PredId),\n            gb_trees:enter(PredId, NewPred,\n                gb_trees:enter(SuccId, NewSucc, Tree))\n    end,\n    % Note: We don't update the subscription here, as it is unclear at this point wether\n    % the node died and the FD informed us on that or if the node is alive and was\n    % filtered due to a full routing table. If the node died, the FD\n    % already removed the subscription.\n\n    RT#rt_t{nodes=UpdatedTree}\n    .\n\n% @doc Convert a sticky entry to a normal node\n-spec sticky_entry_to_normal_node(EntryKey::key(), RT::rt()) -> rt().\nsticky_entry_to_normal_node(EntryKey, RT) ->\n    Node = #rt_entry{type=sticky} = rt_get_node(EntryKey, RT),\n    NewNode = Node#rt_entry{type=normal},\n    UpdatedTree = gb_trees:enter(EntryKey, NewNode, get_rt_tree(RT)),\n    RT#rt_t{nodes=UpdatedTree}.\n\n\n-spec rt_entry_from(Node::mynode(), Type::entry_type(),\n                    PredId::key(), SuccId::key()) -> rt_entry().\nrt_entry_from(Node, Type, PredId, SuccId) ->\n    #rt_entry{node=Node , type=Type , adjacent_fingers={PredId, SuccId},\n             custom=rt_entry_info(Node, Type, PredId, SuccId)}.\n\n% @doc Create an rt_entry and return the entry together with the Pred und Succ node, where\n% the adjacent fingers are changed for each node.\n-spec create_entry(Node::mynode(), Type::entry_type(), RT::rt()) ->\n    {rt_entry(), rt_entry(), rt_entry()}.\ncreate_entry(Node, Type, RT) ->\n    NodeId = id(Node),\n    Tree = get_rt_tree(RT),\n    FirstNode = id(Node) =:= get_source_id(RT),\n    case util:gb_trees_largest_smaller_than(NodeId, Tree)of\n        nil when FirstNode -> % this is the first entry of the RT\n            NewEntry = rt_entry_from(Node, Type, NodeId, NodeId),\n            {NewEntry, NewEntry, NewEntry};\n        nil -> % largest finger\n            {_PredId, Pred} = gb_trees:largest(Tree),\n            get_adjacent_fingers_from(Pred, Node, Type, RT);\n        {value, _PredId, Pred} ->\n            get_adjacent_fingers_from(Pred, Node, Type, RT)\n    end.\n\n% Get the tuple of adjacent finger ids with Node being in the middle:\n% {Predecessor, Node, Successor}\n-spec get_adjacent_fingers_from(Pred::rt_entry(), Node::mynode(),\n    Type::entry_type(), RT::rt()) -> {rt_entry(), rt_entry(), rt_entry()}.\nget_adjacent_fingers_from(Pred, Node, Type, RT) ->\n    PredId = entry_nodeid(Pred),\n    Succ = successor_node(RT, Pred),\n    SuccId = entry_nodeid(Succ),\n    NodeId = id(Node),\n    NewEntry = rt_entry_from(Node, Type, PredId, SuccId),\n    case PredId =:= SuccId of\n        false ->\n            {set_adjacent_succ(Pred, NodeId),\n             NewEntry,\n             set_adjacent_pred(Succ, NodeId)\n            };\n        true ->\n            AdjacentNode = set_adjacent_fingers(Pred, NodeId, NodeId),\n            {AdjacentNode, NewEntry, AdjacentNode}\n    end\n    .\n\n% @doc Add a new entry to the routing table. A source node is only allowed to be added\n% once.\n-spec entry_learning(Entry::mynode(), Type::entry_type(), RT::rt()) -> RefinedRT::rt().\nentry_learning(Entry, Type, RT) ->\n    % only add the entry if it doesn't exist yet or if it is a sticky node. If its a\n    % stickynode, RM told us about a new neighbour -> if the neighbour was already added\n    % as a normal node, convert it to a sticky node now.\n    case gb_trees:lookup(id(Entry), get_rt_tree(RT)) of\n        none ->\n            % change the type to 'sticky' if the node is between the neighbors of the source\n            % node\n            AdaptedType = case Type of\n                sticky -> Type;\n                source -> Type;\n                normal ->\n                    {Pred, Succ} = adjacent_fingers(get_source_node(RT)),\n                    ShouldBeAStickyNode = case Pred =/= Succ of\n                        true ->\n                            case Pred =< Succ of\n                                true ->\n                                    Interval = intervals:new('[', Pred, Succ, ']'),\n                                    intervals:in(id(Entry), Interval);\n                                false ->\n                                    Interval = intervals:new('[', Pred, 0, ']'),\n                                    Interval2 = intervals:new('[', 0, Succ, ']'),\n                                    intervals:in(id(Entry), Interval) orelse\n                                        intervals:in(id(Entry), Interval2)\n                            end;\n                        false ->\n                            % Only two nodes are existing in the ring (otherwise, Pred == Succ\n                            % means there is a bug somewhere!). When two nodes are in the\n                            % system, another third node will be either the successor or\n                            % predecessor of the source node when added.\n                            true\n                    end,\n                    case ShouldBeAStickyNode of\n                        true -> sticky;\n                        false -> Type\n                    end\n            end,\n\n            Ns = {NewPred, NewNode, NewSucc} = create_entry(Entry, AdaptedType, RT),\n            % - if the nodes are all the same, we entered the first node and thus only enter\n            % a single node to the tree\n            % - if pred and succ are the same, we enter the second node: add that node and\n            % an updated pred\n            % - else, add the new node and update succ and pred\n            Nodes = case Ns of\n                {NewNode, NewNode, NewNode} ->\n                    gb_trees:enter(entry_nodeid(NewNode), NewNode, get_rt_tree(RT));\n                {NewPred, NewNode, NewPred} ->\n                    gb_trees:enter(entry_nodeid(NewNode), NewNode,\n                            gb_trees:enter(entry_nodeid(NewPred), NewPred,\n                                get_rt_tree(RT)));\n                _Else ->\n                    gb_trees:enter(entry_nodeid(NewSucc), NewSucc,\n                        gb_trees:enter(entry_nodeid(NewNode), NewNode,\n                            gb_trees:enter(entry_nodeid(NewPred), NewPred, get_rt_tree(RT))\n                        )\n                    )\n            end,\n            rt_set_nodes(RT, Nodes);\n        {value, ExistingEntry} ->\n            % Always update a sticky entry, as that information was send from ring\n            % maintenance.\n            case Type of\n                sticky -> % update entry\n                    StickyEntry = rt_get_node(id(Entry), RT),\n                    Nodes = gb_trees:enter(id(Entry),\n                        StickyEntry#rt_entry{type=sticky},\n                        get_rt_tree(RT)),\n                    rt_set_nodes(RT, Nodes);\n                _ ->\n                    Nodes = gb_trees:enter(rt_entry_id(ExistingEntry),\n                                           rt_entry_set_node(ExistingEntry, Entry),\n                                           get_rt_tree(RT)),\n                    rt_set_nodes(RT, Nodes)\n            end\n    end.\n\n% @doc Combines entry learning and entry filtering.\n-spec entry_learning_and_filtering(mynode(), entry_type(), rt()) -> rt().\nentry_learning_and_filtering(Entry, Type, RT) ->\n    IntermediateRT = entry_learning(Entry, Type, RT),\n\n    SizeOfRT = get_size_without_special_nodes(IntermediateRT),\n    MaxEntries = maximum_entries(),\n    case SizeOfRT > MaxEntries of\n        true ->\n            NewRT = entry_filtering(IntermediateRT),\n            %% only delete the subscription if the newly added node was not filtered;\n            %otherwise, there isn't a subscription yet\n            case rt_lookup_node(id(Entry), NewRT) of\n                none -> ok;\n                {value, _RTEntry} -> update_fd(RT, NewRT)\n            end,\n            NewRT;\n        false ->\n            add_fd(IntermediateRT),\n            IntermediateRT\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% RT and RT entry record accessors\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc Get the source node of a routing table\n-spec get_source_node(RT::rt()) -> rt_entry().\nget_source_node(#rt_t{source=undefined}) -> erlang:error(\"routing table source unknown\");\nget_source_node(#rt_t{source=NodeId, nodes=Nodes}) ->\n    case gb_trees:is_empty(Nodes) of\n            false -> gb_trees:get(NodeId, Nodes);\n            true  -> exit(rt_broken_tree_empty)\n    end.\n\n% @doc Get the id of the source node.\n-spec get_source_id(RT::rt()) -> key().\nget_source_id(#rt_t{source=NodeId}) -> NodeId.\n\n% @doc Set the source node of a routing table\n-spec set_source_node(SourceId::key(), RT::rt()) -> rt().\nset_source_node(SourceId, #rt_t{source=undefined}=RT) ->\n    RT#rt_t{source=SourceId}.\n\n% @doc Get the gb_tree of the routing table containing its nodes\n-spec get_rt_tree(Nodes::rt()) -> rt_t_tree().\nget_rt_tree(#rt_t{nodes=Nodes}) -> Nodes.\n\n% @doc Get all sticky entries of a routing table\n-spec get_sticky_entries(rt()) -> [rt_entry()].\nget_sticky_entries(#rt_t{nodes=Nodes}) ->\n    util:gb_trees_foldl(fun(_K, #rt_entry{type=sticky} = E, Acc) -> [E|Acc];\n                      (_,_,Acc) -> Acc\n                  end, [], Nodes).\n\n% @doc Check if a node exists in the routing table\n-spec entry_exists(EntryKey::key(), rt()) -> boolean().\nentry_exists(EntryKey, #rt_t{nodes=Nodes}) ->\n    case gb_trees:lookup(EntryKey, Nodes) of\n        none -> false;\n        _Else -> true\n    end.\n\n% @doc Add an entry of a specific type to the routing table\n-spec add_entry(Node::mynode(), 'normal' | 'source' | 'sticky', rt()) -> rt().\nadd_entry(Node, Type, RT) ->\n    %% TODO Optimize? Atm linear traversal of the RT\n    RTList = lists:map(fun rt_entry_node/1, rt_get_nodes(RT)),\n    %% check if a node with an identical PidDHT (3 element in mynode tuple)\n    %% already exists.\n    case lists:keyfind(pid_dht(Node), 3, RTList) of\n        false -> %% no node with same PidDHT exists -> add entry\n            entry_learning_and_filtering(Node, Type, RT);\n        OldNode -> %% node already exists in RT -> check id version\n            %% Only nodes with different id but same PidDHT are found here, nodes\n            %% with identical ids are already filtered out higher up. Nodes with\n            %% different id but same DHTPid can occur during slides/jumps.\n            case is_newer(Node, OldNode) of\n                false -> %% ignore node w/ outdated id version\n                    RT;\n                true -> %% remove the old (outdated) entry before adding the new one\n                    SourceNode = rt_entry_node(get_source_node(RT)),\n                    {Id1, IdV1, PidDHT1, _PidRT1} = SourceNode,\n                    {Id2, IdV2, PidDHT2, _PidRT2} = OldNode,\n                    case {Id1, IdV1, PidDHT1} =:= {Id2, IdV2, PidDHT2} of\n                        %% If the new entry is the source node of the current node,\n                        %% we need to change the source node manually.\n                        %% This can happen, when messages from the learning on\n                        %% forwarding (of lookup messages) reach the node faster,\n                        %% than messages from the RM, i.e. in this case, the node\n                        %% learns its new Id from a rt_learn_node message, not the RM.\n                        true ->\n                            RT1 = entry_delete(id(OldNode), RT),\n                            add_source_entry(Node, RT1#rt_t{source=undefined});\n                        false ->\n                            RT1 = entry_delete(id(OldNode), RT),\n                            entry_learning_and_filtering(Node, Type, RT1)\n                    end\n            end\n    end.\n\n\n% @doc Add a sticky entry to the routing table\n-spec add_sticky_entry(Entry::mynode(), rt()) -> rt().\nadd_sticky_entry(Entry, RT) -> add_entry(Entry, sticky, RT).\n\n% @doc Add the source entry to the routing table\n-spec add_source_entry(Entry::mynode(), rt()) -> rt().\nadd_source_entry(Entry, #rt_t{source=undefined} = RT) ->\n    IntermediateRT = set_source_node(id(Entry), RT),\n    % only learn; the RT must be empty, so there is no filtering needed afterwards\n    NewRT = entry_learning(Entry, source, IntermediateRT),\n    add_fd(NewRT),\n    NewRT.\n\n% @doc Add a normal entry to the routing table\n-spec add_normal_entry(Entry::mynode(), rt()) -> rt().\nadd_normal_entry(Entry, RT) ->\n    add_entry(Entry, normal, RT).\n\n% @doc Get the inner mynode() of a rt_entry\n-spec rt_entry_node(N::rt_entry()) -> mynode().\nrt_entry_node(#rt_entry{node=N}) -> N.\n\n% @doc Set the inner mynode() of a rt_entry\n-spec rt_entry_set_node(Entry::rt_entry(), Node::mynode()) -> rt_entry().\nrt_entry_set_node(#rt_entry{} = Entry, Node) -> Entry#rt_entry{node=Node}.\n\n% @doc Get all nodes within the routing table\n-spec rt_get_nodes(RT::rt()) -> [rt_entry()].\nrt_get_nodes(RT) -> gb_trees:values(get_rt_tree(RT)).\n\n% @doc Set the tree of nodes of the routing table.\n-spec rt_set_nodes(RT :: rt(), Nodes::rt_t_tree()) -> rt().\nrt_set_nodes(#rt_t{source=undefined}, _) -> erlang:error(source_node_undefined);\nrt_set_nodes(#rt_t{} = RT, Nodes) -> RT#rt_t{nodes=Nodes}.\n\n% @doc Set the size estimate of the ring\n-spec rt_set_ring_size(RT::rt(), Size::unknown | float()) -> rt().\nrt_set_ring_size(#rt_t{}=RT, Size) -> RT#rt_t{nodes_in_ring=Size}.\n\n% @doc Get the ring size estimate from the external routing table\n-spec external_rt_get_ring_size(RT::external_rt()) -> Size::float() | unknown.\nexternal_rt_get_ring_size(RT)\n  when element(1, RT) >= 0 orelse element(1, RT) == unknown ->\n    element(1, RT).\n\n% @doc Get the tree of an external rt\n-spec external_rt_get_tree(RT::external_rt()) -> external_rt_t_tree().\nexternal_rt_get_tree(RT) when is_tuple(RT) ->\n    element(2, RT).\n\n%% Get the node with the given Id. This function will crash if the node doesn't exist.\n-spec rt_get_node(NodeId::key(), RT::rt()) -> rt_entry().\nrt_get_node(NodeId, RT)  -> gb_trees:get(NodeId, get_rt_tree(RT)).\n\n% @doc Similar to rt_get_node/2, but doesn't crash when the id doesn't exist\n-spec rt_lookup_node(NodeId::key(), RT::rt()) -> {value, rt_entry()} | none.\nrt_lookup_node(NodeId, RT) -> gb_trees:lookup(NodeId, get_rt_tree(RT)).\n\n% @doc Get the id of a given node\n-spec rt_entry_id(Entry::rt_entry()) -> key().\nrt_entry_id(Entry) -> id(rt_entry_node(Entry)).\n\n%% @doc Check if the given routing table entry is of the given entry type.\n-spec entry_is_of_type(rt_entry(), Type::entry_type()) -> boolean().\nentry_is_of_type(#rt_entry{type=Type}, Type) -> true;\nentry_is_of_type(_,_) -> false.\n\n%% @doc Check if the given routing table entry is a source entry.\n-spec is_source(Entry::rt_entry()) -> boolean().\nis_source(Entry) -> entry_is_of_type(Entry, source).\n\n%% @doc Check if the given routing table entry is a sticky entry.\n-spec is_sticky(Entry::rt_entry()) -> boolean().\nis_sticky(Entry) -> entry_is_of_type(Entry, sticky).\n\n-spec entry_type(Entry::rt_entry()) -> entry_type().\nentry_type(Entry) -> Entry#rt_entry.type.\n\n%% @doc Get the node id of a routing table entry\n-spec entry_nodeid(Node::rt_entry()) -> key().\nentry_nodeid(#rt_entry{node=Node}) -> id(Node).\n\n% @doc Get the adjacent fingers from a routing table entry\n-spec adjacent_fingers(rt_entry()) -> {key(), key()}.\nadjacent_fingers(#rt_entry{adjacent_fingers=Fingers}) -> Fingers.\n\n%% @doc Get the adjacent predecessor key() of the current node.\n-spec adjacent_pred(rt_entry()) -> key().\nadjacent_pred(#rt_entry{adjacent_fingers={Pred,_Succ}}) -> Pred.\n\n%% @doc Get the adjacent successor key of the current node\n-spec adjacent_succ(rt_entry()) -> key().\nadjacent_succ(#rt_entry{adjacent_fingers={_Pred,Succ}}) -> Succ.\n\n%% @doc Set the adjacent fingers of a node\n-spec set_adjacent_fingers(rt_entry(), key(), key()) -> rt_entry().\nset_adjacent_fingers(#rt_entry{} = Entry, PredId, SuccId) ->\n    Entry#rt_entry{adjacent_fingers={PredId, SuccId}}.\n\n%% @doc Set the adjacent successor of the finger\n-spec set_adjacent_succ(rt_entry(), key()) -> rt_entry().\nset_adjacent_succ(#rt_entry{adjacent_fingers={PredId, _Succ}} = Entry, SuccId) ->\n    set_adjacent_fingers(Entry, PredId, SuccId).\n\n%% @doc Set the adjacent predecessor of the finger\n-spec set_adjacent_pred(rt_entry(), key()) -> rt_entry().\nset_adjacent_pred(#rt_entry{adjacent_fingers={_Pred, SuccId}} = Entry, PredId) ->\n    set_adjacent_fingers(Entry, PredId, SuccId).\n\n%% @doc Get the adjacent predecessor rt_entry() of the given node.\n-spec predecessor_node(RT::rt(), Node::rt_entry()) -> rt_entry().\npredecessor_node(RT, Node) ->\n    gb_trees:get(adjacent_pred(Node), get_rt_tree(RT)).\n\n-spec successor_node(RT::rt(), Node::rt_entry()) -> rt_entry().\nsuccessor_node(RT, Node) ->\n    try gb_trees:get(adjacent_succ(Node), get_rt_tree(RT)) catch\n         error:function_clause -> exit('stale adjacent fingers')\n    end.\n\n-spec spacing(Node::rt_entry(), RT::rt()) -> float().\nspacing(Node, RT) ->\n    SourceNodeId = entry_nodeid(get_source_node(RT)),\n    canonical_spacing(SourceNodeId, entry_nodeid(Node),\n        adjacent_succ(Node)).\n\n%% @doc Calculate the canonical spacing, which is defined as\n%%  S_i = log_2(distance(SourceNode, SuccId) / distance(SourceNode, Node))\ncanonical_spacing(SourceId, NodeId, SuccId) ->\n    util:log2(get_range(SourceId, SuccId) / get_range(SourceId, NodeId)).\n\n% @doc Check that all entries in an rt are well connected by their adjacent fingers\n-spec check_rt_integrity(RT::rt()) -> boolean().\ncheck_rt_integrity(#rt_t{} = RT) ->\n    Nodes = [id(N) || N <- internal_to_list(RT)],\n\n    %  make sure that the entries are well-connected\n    Currents = Nodes,\n    Last = lists:last(Nodes),\n    Preds = [Last| lists:filter(fun(E) -> E =/= Last end, Nodes)],\n    Succs = tl(Nodes) ++ [hd(Nodes)],\n\n    % for each 3-tuple of pred, current, succ, check if the RT obeys the fingers\n    Checks = [begin Node = rt_get_node(C, RT),\n                case adjacent_fingers(Node) of\n                    {P, S} ->\n                        true;\n                    _Else ->\n                        false\n                end end || {P, C, S} <- lists:zip3(Preds, Currents, Succs)],\n    lists:all(fun(X) -> X end, Checks).\n\n%% userdevguide-begin rt_frtchord:wrap_message\n%% @doc Wrap lookup messages.\n%% For node learning in lookups, a lookup message is wrapped with the global Pid of the\n-spec wrap_message(Key::key(), Msg::comm:message(), MyERT::external_rt(),\n                   Neighbors::nodelist:neighborhood(),\n                   Hops::non_neg_integer()) -> {'$wrapped', mynode(),\n                                                comm:message()} | comm:message().\nwrap_message(_Key, Msg, _ERT, Neighbors, 0) ->\n    MyNode = nodelist:node(Neighbors),\n    RTLoopPid = comm:make_global(pid_groups:get_my(routing_table)),\n    {'$wrapped', {node:id(MyNode), node:id_version(MyNode), node:pidX(MyNode),\n                  RTLoopPid}, Msg};\n\nwrap_message(Key, {'$wrapped', Issuer, _} = Msg, ERT, Neighbors, 1) ->\n    %% The reduction ratio is only useful if this is not the last hop\n    case intervals:in(Key, nodelist:succ_range(Neighbors)) of\n        true -> ok;\n        false ->\n            MyNode = nodelist:node(Neighbors),\n            NextHop = next_hop_node(Neighbors, ERT, Key),\n            SendMsg = case external_rt_get_ring_size(ERT) of\n                unknown -> true;\n                RingSize ->\n                    FirstDist = get_range(id(Issuer), node:id(MyNode)),\n                    TotalDist = get_range(id(Issuer), id(NextHop)),\n                    % reduction ratio > optimal/convergent ratio?\n                    1 - FirstDist / TotalDist >\n                    case config:read(rt_frt_reduction_ratio_strategy) of\n                        best_rt_reduction_ratio -> best_rt_reduction_ratio(RingSize);\n                        convergent_rt_reduction_ratio -> convergent_rt_reduction_ratio(RingSize)\n                    end\n            end,\n\n            case SendMsg of\n                true -> send2rt(Issuer, {rt_learn_node, NextHop});\n                false -> ok\n            end\n    end,\n\n    learn_on_forward(Issuer, nodelist:node(Neighbors)),\n    Msg;\n\nwrap_message(_Key, {'$wrapped', Issuer, _} = Msg, _ERT, Neighbors, _Hops) ->\n    learn_on_forward(Issuer, nodelist:node(Neighbors)),\n    Msg.\n\nbest_rt_reduction_ratio(RingSize) ->\n    1 - math:pow(1 / RingSize, 2 / (maximum_entries() - 1)).\nconvergent_rt_reduction_ratio(RingSize) ->\n    1 - math:pow(1 / RingSize, 4 / (maximum_entries() - 2)).\n\n-spec learn_on_forward(Issuer::mynode(), MyNode::node:node_type()) -> ok.\nlearn_on_forward(Issuer, MyNode) ->\n    PidDHT = comm:make_local(node:pidX(MyNode)),\n    case self() of\n        PidDHT ->\n            comm:send_local(pid_groups:get_my(routing_table), {rt_learn_node, Issuer});\n        _else ->\n            comm:send_local(self(), {rt_learn_node, Issuer})\n    end.\n\n%% userdevguide-end rt_frtchord:wrap_message\n\n%% userdevguide-begin rt_frtchord:unwrap_message\n%% @doc Unwrap lookup messages.\n%% The Pid is retrieved and the Pid of the current node is sent to the retrieved Pid\n-spec unwrap_message(Msg::comm:message(), State::dht_node_state:state()) -> comm:message().\nunwrap_message({'$wrapped', _Issuer, UnwrappedMessage}, _State) -> UnwrappedMessage.\n%% userdevguide-end rt_frtchord:unwrap_message\n\n% @doc Check that the adjacent fingers of a RT are building a ring\n-spec check_well_connectedness(RT::rt()) -> boolean().\ncheck_well_connectedness(RT) ->\n    Nodes = [N || {_, N} <- gb_trees:to_list(get_rt_tree(RT))],\n    NodeIds = lists:sort([Id || {Id, _} <- gb_trees:to_list(get_rt_tree(RT))]),\n    % traverse the adjacent fingers of the nodes and add each visited node to a list\n    % NOTE: each node should only be visited once\n    InitVisitedNodes = ordsets:from_list([{N, false} || N <- Nodes]),\n    %% check forward adjacent fingers\n    Visit = fun(Visit, Current, Visited, Direction) ->\n            case ordsets:is_element({Current, false}, Visited) of\n                true -> % node wasn't visited\n                    AccVisited = ordsets:add_element({Current, true},\n                        ordsets:del_element({Current, false}, Visited)),\n                    Next = case Direction of\n                        succ -> rt_get_node(adjacent_succ(Current), RT);\n                        pred -> rt_get_node(adjacent_pred(Current), RT)\n                    end,\n                    Visit(Visit, Next, AccVisited, Direction);\n                false ->\n                    Filtered = ordsets:filter(fun({_, true}) -> true; (_) -> false end,\n                        Visited),\n                    lists:sort([entry_nodeid(N) || {N, true} <- ordsets:to_list(Filtered)])\n            end\n    end,\n    Succs = Visit(Visit, get_source_node(RT), InitVisitedNodes, succ),\n    Preds = Visit(Visit, get_source_node(RT), InitVisitedNodes, pred),\n    try\n        NodeIds = Succs,\n        NodeIds = Preds,\n        true\n    catch\n        _:_ -> false\n    end\n    .\n\n%% @doc Transform a node:node_type() to mynode().\n-spec node2mynode(Node::node:node_type()) -> mynode().\nnode2mynode(Node) -> node2mynode(Node, none).\n\n%% @doc Transform a node:node_type() to mynode().\n-spec node2mynode(Node::node:node_type(), PidRT::comm:mypid() | none) -> mynode().\nnode2mynode(Node, PidRT) ->\n    {node:id(Node), node:id_version(Node), node:pidX(Node), PidRT}.\n\n%% @doc Get the id from a mynode().\n-spec id(Node::mynode()) -> key().\nid({Id, _IdVersion, _PidDHT, _PidRT}) -> Id.\n\n%% @doc Get the Pid from an mynode().\n-spec pid_dht(Node::mynode()) -> comm:mypid().\npid_dht({_Id, _IdVersion, PidDHT, _PidRT}) -> PidDHT.\n\n%% @doc Get the Pid from an mynode().\n-spec pid_rt(Node::mynode()) -> comm:mypid() | none.\npid_rt({_Id, _IdVersion, _PidDHT, PidRT}) -> PidRT.\n\n%% @doc Determines whether Node1 is a newer instance of Node2.\n%%      Note: Both nodes need to share the same PidDHT, otherwise an exception of\n%%      type 'throw' is thrown! PidRT is ignored for the comparison.\n-spec is_newer(Node1::mynode(), Node2::mynode()) -> boolean().\nis_newer(_Node1 = {Id1, IdVersion1, PidDHT, _PidRT1},\n         _Node2 = {Id2, IdVersion2, PidDHT, _PidRT2}) ->\n    if\n        (IdVersion1 > IdVersion2) -> true;\n        IdVersion1 < IdVersion2 -> false;\n        Id1 =:= Id2 -> false;\n        true -> throw('got two nodes with same IDversion but different ID')\n    end.\n\n\n%% @doc Send Msg to the routing table.\n%%      Send directly if rt_loop pid is known, otherwise as group_member msg.\n-spec send2rt(Node::mynode(), Msg::comm:message()) -> ok.\nsend2rt(Node, Msg) ->\n    {Pid, NewMsg} = case pid_rt(Node) of\n                        none ->\n                            MsgNew = {?send_to_group_member, routing_table, Msg},\n                            {pid_dht(Node), MsgNew};\n                        PidDHT ->\n                            {PidDHT, Msg}\n                    end,\n    comm:send(Pid, NewMsg).\n\n%% print_debug(Case, OldNode, NewNode, ERT) ->\n%%     OldPidDHT = comm:make_local(pid_dht(OldNode)),\n%%     OldPidRT = comm:make_local(pid_rt(OldNode)),\n%%     NewPidDHT = comm:make_local(pid_dht(NewNode)),\n%%     NewPidRT = comm:make_local(pid_rt(NewNode)),\n%%     log:pal(\"{~w, ~w} Update Neighbor: ~w. KnownNode: ~w. NewNode: ~w~n\",\n%%             [pid_groups:get_my(dht_node), self(), Case, {OldPidDHT, OldPidRT},\n%%              {NewPidDHT, NewPidRT}]).\n\n-ifdef(GFRT).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% GFRT-chord\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% Additional information appended to an rt_entry()\n-record(rt_entry_info, {group = other_dc :: other_dc | same_dc}).\n-type rt_entry_info_t()::#rt_entry_info{}.\n\n\n-spec allowed_nodes(RT::rt()) -> [rt_entry()].\nallowed_nodes(RT) ->\n    Source = get_source_node(RT),\n    SourceId = rt_entry_id(Source),\n    Nodes = rt_get_nodes(RT),\n\n    % $E_{\\bar{G}}$ and $E_{G}$\n    {E_NG, E_G} = lists:partition(fun is_from_other_group/1, Nodes),\n\n    % If $E_G = \\emptyset$, we know that we can allow all nodes to be filtered.\n    % Otherwise, check if $E_\\text{leap} \\neq \\emptyset$.\n    {OnlyNonGroupMembers, {E_a, E_b}} = case E_G of\n        [] -> {true, {ignore, ignore}};\n        [First|_] ->\n            Predecessor = predecessor_node(RT, Source),\n            FirstDist = get_range(SourceId, rt_entry_id(First)),\n\n            % Compute $e_\\alpha$, $e_\\beta$ and the respective distances\n            {{E_alpha, E_alphaDist}, {E_beta, _E_betaDist}} = lists:foldl(\n                fun (Node, {{MinNode, MinDist}, {MaxNode, MaxDist}}) ->\n                        NodeDist = get_range(SourceId, rt_entry_id(Node)),\n                        NewE_alpha = case erlang:min(MinDist, NodeDist) of\n                            MinDist -> {MinNode, MinDist};\n                            _ -> {Node, NodeDist}\n                        end,\n                        NewE_beta = case erlang:max(MaxDist, NodeDist) of\n                            MinDist -> {MaxNode, MaxDist};\n                            _ -> {Node, NodeDist}\n                        end,\n                        {NewE_alpha, NewE_beta}\n                end, {{First, FirstDist}, {First, FirstDist}}, E_G),\n\n            % Is there any non-group entry $n$ such that $d(s, e_\\alpha) \\leq d(s, n)$ and\n            % $n \\neq s.pred$? The following line basically computes $E_leap$ and checks\n            % if that set is empty.\n            {lists:any(fun(P) when P =:= Predecessor -> false;\n                         (N) -> get_range(SourceId, rt_entry_id(N)) >= E_alphaDist andalso\n                            not is_sticky(N) andalso not is_source(N)\n                      end, E_NG),\n                  {E_alpha, E_beta}}\n    end,\n\n    if OnlyNonGroupMembers -> [N || N <- E_NG, not is_sticky(N),\n                                               not is_source(N)];\n       not OnlyNonGroupMembers andalso E_a =/= ignore andalso E_b =/= ignore ->\n           [N || N <- Nodes, not is_sticky(N), not is_source(N),\n                                               N =/= E_a,\n                                               N =/= E_b]\n    end.\n\n-spec rt_entry_info(Node::mynode(), Type::entry_type(),\n                    PredId::key(), SuccId::key()) -> rt_entry_info_t().\nrt_entry_info(Node, _Type, _PredId, _SuccId) ->\n    #rt_entry_info{group=case comm:get_ip(pid_dht(Node)) =:= comm:get_ip(comm:this()) of\n            true -> same_dc;\n            false -> other_dc\n        end}.\n\n%% Check if the given node belongs to another group of nodes\n-spec is_from_other_group(rt_entry()) -> boolean().\nis_from_other_group(Node) ->\n    CustomInfo = Node#rt_entry.custom,\n    CustomInfo#rt_entry_info.group =:= same_dc.\n\n-else.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% FRT-chord\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% Additional information appended to an rt_frt_helpers:rt_entry()\n-type rt_entry_info_t() :: undefined.\n\n-spec allowed_nodes(RT::rt()) -> [rt_entry()].\nallowed_nodes(RT) ->\n    [N || N <- rt_get_nodes(RT), not is_sticky(N) and not is_source(N)].\n\n-spec rt_entry_info(Node::mynode(), Type::entry_type(),\n                    PredId::key(), SuccId::key()) -> rt_entry_info_t().\nrt_entry_info(_Node, _Type, _PredId, _SuccId) ->\n    undefined.\n\n-endif.\n\n%% @doc Checks whether config parameters of the rt_frtchord/rt_gfrtchord\n%%      process exist and are valid.\n-spec frt_check_config() -> boolean().\nfrt_check_config() -> true.\n"
  },
  {
    "path": "src/rt_loop.erl",
    "content": "%  @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    routing table process\n%% @end\n%% @version $Id$\n-module(rt_loop).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n-include(\"lookup.hrl\").\n\n% for routing table implementation\n-export([start_link/1]).\n-export([init/1, on_inactive/2, on_active/2,\n         activate/1, deactivate/0,\n         check_config/0,\n         get_neighb/1, get_pid_dht/1, get_rt/1, set_rt/2, get_ert/1, set_ert/2,\n         rm_neighbor_change/3, rm_send_update/5]).\n\n-export_type([state_active/0]).\n\n-include(\"gen_component.hrl\").\n\n% state of the routing table loop\n%% userdevguide-begin rt_loop:state\n-opaque(state_active() :: {Neighbors    :: nodelist:neighborhood(),\n                           RT           :: ?RT:rt(),\n                           ERT          :: ?RT:external_rt(),\n                           DHTNodePid   :: comm:mypid()}).\n-type(state_inactive() :: {inactive,\n                           MessageQueue::msg_queue:msg_queue()}).\n%% userdevguide-end rt_loop:state\n\n% accepted messages of rt_loop processes\n-type(message() ::\n    {trigger_rt} | %% just for periodic wake up\n    {periodic_rt_rebuild} | %% actually initiate a rt rebuild\n    {update_rt, OldNeighbors::nodelist:neighborhood(), NewNeighbors::nodelist:neighborhood()} |\n    {fd_notify, fd:event(), DeadPid::comm:mypid(), Reason::fd:reason()} |\n    {web_debug_info, Requestor::comm:erl_local_pid()} |\n    {dump, Pid::comm:erl_local_pid()} |\n    ?RT:custom_message()).\n\n%% @doc Activates the routing table process. If not activated, it will\n%%      queue most messages without processing them.\n%%      Pre: dht_node must be up and running\n-spec activate(Neighbors::nodelist:neighborhood()) -> ok.\nactivate(Neighbors) ->\n    Pid = pid_groups:get_my(routing_table),\n    comm:send_local(Pid, {activate_rt, Neighbors}).\n\n%% @doc Deactivates the re-register process.\n-spec deactivate() -> ok.\ndeactivate() ->\n    Pid = pid_groups:get_my(routing_table),\n    comm:send_local(Pid, {deactivate_rt}).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Startup\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Starts the routing tabe maintenance process, registers it with the\n%%      process dictionary and returns its pid for use by a supervisor.\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on_inactive/2, [],\n                             [{pid_groups_join_as, DHTNodeGroup, routing_table}]).\n\n%% @doc Initialises the module with an empty state.\n-spec init([]) -> state_inactive().\ninit([]) ->\n    %% generate trigger msg only once and then keep it repeating\n    msg_delay:send_trigger(get_base_interval(), {trigger_rt}),\n    ?RT:init(),\n    {inactive, msg_queue:new()}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Message Loop\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Message handler during start up phase (will change to on_active/2 when a\n%%      'activate_rt' message is received).\n-spec on_inactive(message(), state_inactive()) -> state_inactive();\n                 ({activate_rt, Neighbors::nodelist:neighborhood()}, state_inactive())\n                    -> {'$gen_component', [{on_handler, Handler::gen_component:handler()},...],\n                        State::state_active()}.\non_inactive({activate_rt, Neighbors}, {inactive, QueuedMessages}) ->\n    log:log(info, \"[ RT ~.0p ] activating...~n\", [comm:this()]),\n    comm:send_local(self(), {periodic_rt_rebuild}),\n    rm_loop:subscribe(self(), ?MODULE,\n                      fun ?MODULE:rm_neighbor_change/3,\n                      fun ?MODULE:rm_send_update/5, inf),\n    msg_queue:send(QueuedMessages),\n    DHTNodePid = comm:make_local(node:pidX(nodelist:node(Neighbors))),\n    gen_component:change_handler(\n      {Neighbors, ?RT:activate(Neighbors), ?RT:empty_ext(Neighbors), DHTNodePid},\n      fun ?MODULE:on_active/2);\n\non_inactive({trigger_rt}, State) ->\n    %% keep trigger active to avoid generating new triggers when\n    %% frequently jumping between inactive and active state.\n    msg_delay:send_trigger(get_base_interval(), {trigger_rt}),\n    State;\n\non_inactive({web_debug_info, Requestor}, {inactive, QueuedMessages} = State) ->\n    % get a list of up to 50 queued messages to display:\n    MessageListTmp = [{\"\", webhelpers:safe_html_string(\"~p\", [Message])}\n                  || Message <- lists:sublist(QueuedMessages, 50)],\n    MessageList = case length(QueuedMessages) > 50 of\n                      true -> lists:append(MessageListTmp, [{\"...\", \"\"}]);\n                      _    -> MessageListTmp\n                  end,\n    KeyValueList = [{\"\", \"\"}, {\"inactive RT process\", \"\"}, {\"queued messages:\", \"\"} | MessageList],\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    State;\n\n% unknown message\non_inactive(Msg, {inactive, MsgQueue}) ->\n    MsgQueueNew = ?RT:handle_custom_message_inactive(Msg, MsgQueue),\n    {inactive, MsgQueueNew}.\n\n%% @doc Message handler when the module is fully initialized.\n-spec on_active(message(), state_active())\n                  -> state_active() |\n                         {'$gen_component', [{post_op, Msg::{periodic_rt_rebuild}},...], state_active()} |\n                         unknown_event;\n               ({deactivate_rt}, state_active())\n                  -> {'$gen_component', [{on_handler, Handler::gen_component:handler()},...], State::state_inactive()}.\non_active({deactivate_rt}, {Neighbors, _OldRT, _ERT, DHTPid})  ->\n    log:log(info, \"[ RT ~.0p ] deactivating...~n\", [comm:this()]),\n    rm_loop:unsubscribe(self(), ?MODULE),\n    % send new empty RT to the dht_node so that all routing messages\n    % must be passed to the successor:\n    comm:send_local(DHTPid, {rt_update, ?RT:empty_ext(Neighbors)}),\n    gen_component:change_handler({inactive, msg_queue:new()},\n                                 fun ?MODULE:on_inactive/2);\n\n%% userdevguide-begin rt_loop:update_rt\n% Update routing table with changed neighborhood from the rm\non_active({update_rt, OldNeighbors, NewNeighbors}, {_Neighbors, OldRT, OldERT, DHTPid}) ->\n    case ?RT:update(OldRT, OldNeighbors, NewNeighbors) of\n        {trigger_rebuild, NewRT} ->\n            NewERT = ?RT:check(OldRT, NewRT, OldERT, OldNeighbors, NewNeighbors, true),\n            % trigger immediate rebuild\n            gen_component:post_op({periodic_rt_rebuild}, {NewNeighbors, NewRT, NewERT, DHTPid})\n        ;\n        {ok, NewRT} ->\n            NewERT = ?RT:check(OldRT, NewRT, OldERT, OldNeighbors, NewNeighbors, true),\n            {NewNeighbors, NewRT, NewERT, DHTPid}\n    end;\n%% userdevguide-end rt_loop:update_rt\n\n%% userdevguide-begin rt_loop:trigger\n% Message handler to manage the trigger\non_active({trigger_rt}, State) ->\n    msg_delay:send_trigger(get_base_interval(), {trigger_rt}),\n    gen_component:post_op({periodic_rt_rebuild}, State);\n\n% Actual periodic rebuilding of the RT\non_active({periodic_rt_rebuild}, {Neighbors, OldRT, OldERT, DHTPid}) ->\n    % start periodic stabilization\n    % log:log(debug, \"[ RT ] stabilize\"),\n    NewRT = ?RT:init_stabilize(Neighbors, OldRT),\n    NewERT = ?RT:check(OldRT, NewRT, OldERT, Neighbors, true),\n    {Neighbors, NewRT, NewERT, DHTPid};\n%% userdevguide-end rt_loop:trigger\n\n% failure detector reported dead node\non_active({fd_notify, crash, DeadPid, Reason}, {Neighbors, OldRT, OldERT, DHTPid}) ->\n    NewRT = ?RT:filter_dead_node(OldRT, DeadPid, Reason),\n    NewERT = ?RT:check(OldRT, NewRT, OldERT, Neighbors, false),\n    {Neighbors, NewRT, NewERT, DHTPid};\non_active({fd_notify, _Event, _DeadPid, _Reason}, State) ->\n    State;\n\n% debug_info for web interface\non_active({web_debug_info, Requestor},\n   {_Neighbors, RT, ERT, _DHTPid} = State) ->\n    KeyValueList =\n        [{\"rt_size\", ?RT:get_size(RT)},\n         {\"ert_size\", ?RT:get_size_ext(ERT)},\n         {\"rt (index, node):\", \"\"} | ?RT:dump(RT)],\n    comm:send_local(Requestor, {web_debug_info_reply, KeyValueList}),\n    State;\n\non_active({dump, Pid}, {_Neighbors, RT, _ERT, _DHTPid} = State) ->\n    comm:send_local(Pid, {dump_response, RT}),\n    State;\n\non_active({?lookup_aux, Key, Hops, Msg}, {Neighbors, _RT, ERT, DHTPid} = State) ->\n    case config:read(leases) of\n        true ->\n            lookup_aux_leases(Neighbors, ERT, DHTPid, Key, Hops, Msg);\n        _ ->\n            lookup_aux_chord(Neighbors, ERT, DHTPid, Key, Hops, Msg)\n    end,\n    State;\n\non_active({send_error, _Target, {?send_to_group_member, routing_table,\n                                 {?lookup_aux, Key, Hops, Msg}} = _Message, _Reason}, State) ->\n    log:log(warn, \"[routing_table] lookup_aux failed 1. Target: ~p. Msg: ~p.\", [_Target, _Message]),\n    _ = comm:send_local_after(100, self(), {?lookup_aux, Key, Hops + 1, Msg}),\n    State;\n\non_active({send_error, _Target, {?lookup_aux, Key, Hops, Msg} = _Message, _Reason}, State) ->\n    log:log(warn, \"[routing_table] lookup_aux failed 2. Target: ~p~nMsg: ~.2p.\", [_Target, _Message]),\n    _ = comm:send_local_after(100, self(), {?lookup_aux, Key, Hops + 1, Msg}),\n    State;\n\non_active({send_error, _Target, {?lookup_fin, Key, Data, Msg} = _Message, _Reason}, State) ->\n    _ = comm:send_local_after(100, self(), {?lookup_aux, Key, ?HOPS_FROM_DATA(Data) + 1, Msg}),\n    State;\n\n% unknown message\non_active(Message, State) ->\n    ?RT:handle_custom_message(Message, State).\n\n-spec lookup_aux_chord(Neighbors::nodelist:neighborhood(), ERT::?RT:external_rt(),\n                       DHTNodePid::comm:mypid(), Key::intervals:key(),\n                       Hops::non_neg_integer(), Msg::comm:message()) -> ok.\nlookup_aux_chord(Neighbors, ERT, DHTPid, Key, Hops, Msg) ->\n    WrappedMsg = ?RT:wrap_message(Key, Msg, ERT, Neighbors, Hops),\n    % NOTE: chord-like routing requires routing through predecessor -> only decide at pred:\n    case ?RT:next_hop(Neighbors, ERT, Key) of\n        succ ->\n            comm:send_local(DHTPid, {lookup_decision, Key, Hops, WrappedMsg});\n        NextHop ->\n            NewMsg = {?lookup_aux, Key, Hops + 1, WrappedMsg},\n            comm:send(NextHop, NewMsg, [{shepherd, self()}])\n    end.\n\n-spec lookup_aux_leases(Neighbors::nodelist:neighborhood(), ERT::?RT:external_rt(),\n                        DHTNodePid::comm:mypid(), Key::intervals:key(),\n                        Hops::non_neg_integer(), Msg::comm:message()) -> ok.\nlookup_aux_leases(Neighbors, ERT, DHTPid, Key, Hops, Msg) ->\n    WrappedMsg = ?RT:wrap_message(Key, Msg, ERT, Neighbors, Hops),\n    % NOTE: leases do not require routing through predecessor -> let the own node decide:\n    case intervals:in(Key, nodelist:node_range(Neighbors)) of\n        true ->\n            comm:send_local(DHTPid, {?lookup_fin, Key, ?HOPS_TO_DATA(Hops),\n                                     WrappedMsg});\n        false ->\n            %% next_hop and nodelist:succ(Neighbors) return different nodes if\n            %% key is in the interval of the succ.\n            NextHop = case ?RT:next_hop(Neighbors, ERT, Key) of\n                          succ -> ?RT:succ(ERT, Neighbors);\n                          Pid -> Pid\n                      end,\n            NewMsg = {?lookup_aux, Key, Hops + 1, WrappedMsg},\n            comm:send(NextHop, NewMsg, [{shepherd, self()}])\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% rt_loop:state_active() handling\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% handling rt_loop's (opaque) state - these handlers should at least be used\n% outside this module:\n\n-spec get_neighb(State::state_active()) -> nodelist:neighborhood().\nget_neighb({Neighbors, _RT, _ERT, _DHTPid}) ->\n    Neighbors.\n\n-spec get_pid_dht(State::state_active()) -> comm:mypid().\nget_pid_dht({_Neighbors, _RT, _ERT, DHTPid}) -> DHTPid.\n\n-spec get_rt(State::state_active()) -> ?RT:rt().\nget_rt({_Neighbors, RT, _ERT, _DHTPid}) -> RT.\n\n-spec set_rt(State::state_active(), RT::?RT:rt()) -> NewState::state_active().\nset_rt({Neighbors, _OldRT, ERT, DHTPid}, NewRT) ->\n    {Neighbors, NewRT, ERT, DHTPid}.\n\n-spec get_ert(State::state_active()) -> ?RT:external_rt().\nget_ert({_Neighbors, _RT, ERT, _DHTPid}) -> ERT.\n\n-spec set_ert(State::state_active(), ERT::?RT:external_rt()) -> NewState::state_active().\nset_ert({Neighbors, RT, _OldERT, DHTPid}, ERT) ->\n    {Neighbors, RT, ERT, DHTPid}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Misc.\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Filter function for subscriptions that returns true if a\n%%      any neighbor changes.\n-spec rm_neighbor_change(\n        OldNeighbors::nodelist:neighborhood(), NewNeighbors::nodelist:neighborhood(),\n        Reason::rm_loop:reason()) -> boolean().\nrm_neighbor_change(OldNeighbors, NewNeighbors, _Reason) ->\n    OldNeighbors =/= NewNeighbors.\n\n%% @doc Notifies the node's routing table of a changed neighborhood.\n%%      Used to subscribe to the ring maintenance.\n-spec rm_send_update(Subscriber::pid(), Tag::?MODULE,\n                     OldNeighbors::nodelist:neighborhood(),\n                     NewNeighbors::nodelist:neighborhood(),\n                     Reason::rm_loop:reason()) -> ok.\nrm_send_update(Pid, ?MODULE, OldNeighbors, NewNeighbors, _Reason) ->\n    comm:send_local(Pid, {update_rt, OldNeighbors, NewNeighbors}).\n\n-spec get_base_interval() -> pos_integer().\nget_base_interval() ->\n    config:read(pointer_base_stabilization_interval) div 1000.\n\n%% @doc Checks whether config parameters of the rt_loop process exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(pointer_base_stabilization_interval) and\n        config:cfg_is_greater_than_equal(pointer_base_stabilization_interval, 1000).\n"
  },
  {
    "path": "src/rt_simple.erl",
    "content": "% @copyright 2008-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Simple implementation of a routing table with linear routing.\n%% @end\n%% @version $Id$\n-module(rt_simple).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(rt_beh).\n-include(\"scalaris.hrl\").\n\n%% userdevguide-begin rt_simple:types\n-type key() :: 0..16#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF. % 128 bit numbers\n-opaque rt() :: Succ::node:node_type().\n-type external_rt() :: Succ::node:node_type(). %% @todo: make opaque\n-type custom_message() :: none().\n%% userdevguide-end rt_simple:types\n\n% Note: must include rt_beh.hrl AFTER the type definitions for erlang < R13B04\n% to work.\n-include(\"rt_beh.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Key Handling\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% userdevguide-begin rt_simple:empty\n%% @doc Creates an \"empty\" routing table containing the successor.\n-spec empty(nodelist:neighborhood()) -> rt().\nempty(Neighbors) -> nodelist:succ(Neighbors).\n%% userdevguide-end rt_simple:empty\n\n%% @doc This function is called during the startup of the rt_loop process and\n%%      is allowed to send trigger messages.\n%%      Noop in rt_simple.\n-spec init() -> ok.\ninit() -> ok.\n\n%% @doc Activate the routing table.\n%%      This function is called during the activation of the routing table process.\n-spec activate(nodelist:neighborhood()) -> rt().\nactivate(Neighbors) -> empty(Neighbors).\n\n%% @doc Hashes the key to the identifier space.\n-spec hash_key(client_key() | binary()) -> key().\nhash_key(Key) when not is_binary(Key) ->\n    hash_key(client_key_to_binary(Key));\nhash_key(Key) ->\n    <<N:128>> = ?CRYPTO_MD5(Key),\n    N.\n%% userdevguide-end rt_simple:hash_key\n\n%% userdevguide-begin rt_simple:get_random_node_id\n%% @doc Generates a random node id, i.e. a random 128-bit number.\n-spec get_random_node_id() -> key().\nget_random_node_id() ->\n    case config:read(key_creator) of\n        random -> hash_key(randoms:getRandomString());\n        random_with_bit_mask ->\n            {Mask1, Mask2} = config:read(key_creator_bitmask),\n            (hash_key(randoms:getRandomString()) band Mask2) bor Mask1\n    end.\n%% userdevguide-end rt_simple:get_random_node_id\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% RT Management\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% userdevguide-begin rt_simple:init_stabilize\n%% @doc Triggered by a new stabilization round, renews the routing table.\n-spec init_stabilize(nodelist:neighborhood(), rt()) -> rt().\ninit_stabilize(Neighbors, _RT) -> empty(Neighbors).\n%% userdevguide-end rt_simple:init_stabilize\n\n%% userdevguide-begin rt_simple:update\n%% @doc Updates the routing table due to a changed node ID, pred and/or succ.\n-spec update(OldRT::rt(), OldNeighbors::nodelist:neighborhood(),\n             NewNeighbors::nodelist:neighborhood()) -> {ok, rt()}.\nupdate(_OldRT, _OldNeighbors, NewNeighbors) ->\n    {ok, nodelist:succ(NewNeighbors)}.\n%% userdevguide-end rt_simple:update\n\n%% userdevguide-begin rt_simple:filter_dead_node\n%% @doc Removes dead nodes from the routing table (rely on periodic\n%%      stabilization here).\n-spec filter_dead_node(rt(), DeadPid::comm:mypid(), Reason::fd:reason()) -> rt().\nfilter_dead_node(RT, _DeadPid, _Reason) -> RT.\n%% userdevguide-end rt_simple:filter_dead_node\n\n%% userdevguide-begin rt_simple:to_pid_list\n%% @doc Returns the pids of the routing table entries.\n-spec to_pid_list(rt()) -> [comm:mypid()].\nto_pid_list(Succ) -> [node:pidX(Succ)].\n%% userdevguide-end rt_simple:to_pid_list\n\n%% userdevguide-begin rt_simple:get_size\n%% @doc Returns the size of the routing table.\n-spec get_size(rt()) -> non_neg_integer().\nget_size(_RT) -> 1.\n\n%% @doc Returns the size of the external routing table.\n-spec get_size_ext(external_rt()) -> non_neg_integer().\nget_size_ext(_RT) -> 1.\n%% userdevguide-end rt_simple:get_size\n\n%% userdevguide-begin rt_simple:n\n%% @doc Returns the size of the address space.\n-spec n() -> 16#100000000000000000000000000000000.\nn() -> 16#100000000000000000000000000000000.\n%% userdevguide-end rt_simple:n\n\n%% @doc Keep a key in the address space. See n/0.\n-spec normalize(Key::key()) -> key().\nnormalize(Key) -> Key band 16#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.\n\n%% @doc Gets the number of keys in the interval (Begin, End]. In the special\n%%      case of Begin==End, the whole key range as specified by n/0 is returned.\n-spec get_range(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE) -> non_neg_integer().\nget_range(Begin, Begin) -> n(); % I am the only node\nget_range(?MINUS_INFINITY, ?PLUS_INFINITY) -> n(); % special case, only node\nget_range(Begin, End) when End > Begin -> End - Begin;\nget_range(Begin, End) when End < Begin -> (n() - Begin) + End.\n\n%% @doc Gets the key that splits the interval (Begin, End] so that the first\n%%      interval will be (Num/Denom) * range(Begin, End). In the special case of\n%%      Begin==End, the whole key range is split.\n%%      Beware: SplitFactor must be in [0, 1]; the final key will be rounded\n%%      down and may thus be Begin.\n-spec get_split_key(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE,\n                    SplitFraction::{Num::number(), Denom::pos_integer()}) -> key() | ?PLUS_INFINITY_TYPE.\nget_split_key(Begin, _End, {Num, _Denom}) when Num == 0 -> Begin;\nget_split_key(_Begin, End, {Num, Denom}) when Num == Denom -> End;\nget_split_key(Begin, End, {Num, Denom}) ->\n    normalize(Begin + trunc(get_range(Begin, End) * Num) div Denom).\n\n%% @doc Splits the range between Begin and End into up to Parts equal parts and\n%%      returning the according split keys.\n-spec get_split_keys(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE,\n                     Parts::pos_integer()) -> [key()].\nget_split_keys(Begin, End, Parts) ->\n    get_split_keys_helper(Begin, End, Parts).\n\n-spec get_split_keys_helper(Begin::key(), End::key() | ?PLUS_INFINITY_TYPE,\n                            Parts::pos_integer()) -> [key()].\nget_split_keys_helper(_Begin, _End, 1) ->\n    [];\nget_split_keys_helper(Begin, End, Parts) ->\n    SplitKey = get_split_key(Begin, End, {1, Parts}),\n    if SplitKey =:= Begin ->\n           get_split_keys_helper(Begin, End, Parts - 1);\n       true ->\n           [SplitKey | get_split_keys_helper(SplitKey, End, Parts - 1)]\n    end.\n\n%% @doc Gets input similar to what intervals:get_bounds/1 returns and\n%%      calculates a random key in this range. Fails with an exception if there\n%%      is no key.\n-spec get_random_in_interval(intervals:simple_interval2()) -> key().\nget_random_in_interval(I) ->\n    hd(get_random_in_interval(I, 1)).\n\n%% @doc Gets input similar to what intervals:get_bounds/1 returns and\n%%      calculates Count number of random keys in this range (duplicates may\n%%      exist!). Fails with an exception if there is no key.\n-spec get_random_in_interval(intervals:simple_interval2(), Count::pos_integer()) -> [key(),...].\nget_random_in_interval({LBr, L, R, RBr}, Count) ->\n    case intervals:wraps_around(LBr, L, R, RBr) of\n        false -> get_random_in_interval2(LBr, L, R, RBr, Count);\n        true  -> [normalize(Key) || Key <- get_random_in_interval2(LBr, L, ?PLUS_INFINITY + R, RBr, Count)]\n    end.\n\n% TODO: return a failure constant if the interval is empty? (currently fails with an exception)\n-spec get_random_in_interval2(intervals:left_bracket(), key(), non_neg_integer(),\n                              intervals:right_bracket(), Count::pos_integer()) -> [non_neg_integer()].\nget_random_in_interval2('(', L, R, ']', Count) ->\n    L2 = L + 1,\n    R2 = R + 1,\n    randoms:rand_uniform(L2, R2, Count);\nget_random_in_interval2('[', L, R, ')', Count) ->\n    randoms:rand_uniform(L, R, Count);\nget_random_in_interval2('[', X, X, ']', Count) ->\n    lists:duplicate(Count, X);\nget_random_in_interval2('[', L, R, ']', Count) ->\n    R2 = R + 1,\n    randoms:rand_uniform(L, R2, Count);\nget_random_in_interval2('(', L, R, ')', Count) ->\n    L2 = L + 1,\n    randoms:rand_uniform(L2, R, Count).\n\n%% userdevguide-begin rt_simple:get_replica_keys\n%% @doc Returns the replicas of the given key.\n-spec get_replica_keys(key()) -> [key()].\nget_replica_keys(Key) ->\n    get_replica_keys(Key, config:read(replication_factor)).\n\n-spec get_replica_keys(key(), pos_integer()) -> [key()].\nget_replica_keys(Key, ReplicationFactor) ->\n    case ReplicationFactor of\n        2 ->\n            [Key,\n             Key bxor 16#80000000000000000000000000000000\n            ];\n        4 ->\n            [Key,\n             Key bxor 16#40000000000000000000000000000000,\n             Key bxor 16#80000000000000000000000000000000,\n             Key bxor 16#C0000000000000000000000000000000\n            ];\n        8 ->\n            [Key,\n             Key bxor 16#20000000000000000000000000000000,\n             Key bxor 16#40000000000000000000000000000000,\n             Key bxor 16#60000000000000000000000000000000,\n             Key bxor 16#80000000000000000000000000000000,\n             Key bxor 16#A0000000000000000000000000000000,\n             Key bxor 16#C0000000000000000000000000000000,\n             Key bxor 16#E0000000000000000000000000000000\n            ];\n        16 ->\n            [Key,\n             Key bxor 16#10000000000000000000000000000000,\n             Key bxor 16#20000000000000000000000000000000,\n             Key bxor 16#30000000000000000000000000000000,\n             Key bxor 16#40000000000000000000000000000000,\n             Key bxor 16#50000000000000000000000000000000,\n             Key bxor 16#60000000000000000000000000000000,\n             Key bxor 16#70000000000000000000000000000000,\n\n             Key bxor 16#80000000000000000000000000000000,\n             Key bxor 16#90000000000000000000000000000000,\n             Key bxor 16#A0000000000000000000000000000000,\n             Key bxor 16#B0000000000000000000000000000000,\n             Key bxor 16#C0000000000000000000000000000000,\n             Key bxor 16#D0000000000000000000000000000000,\n             Key bxor 16#E0000000000000000000000000000000,\n             Key bxor 16#F0000000000000000000000000000000\n            ];\n        R ->\n            Step = n() div R,\n            MappedToFirstSegment = Key rem Step,\n            [MappedToFirstSegment + I * Step || I <- lists:seq(0, R-1)]\n    end.\n%% userdevguide-end rt_simple:get_replica_keys\n-spec get_key_segment(key()) -> pos_integer().\nget_key_segment(Key) ->\n    get_key_segment(Key, config:read(replication_factor)).\n\n-spec get_key_segment(key(), pos_integer()) -> pos_integer().\nget_key_segment(Key, ReplicationFactor) ->\n    case ReplicationFactor of\n        2  -> (Key bsr 127) + 1;\n        4  -> (Key bsr 126) + 1;\n        8  -> (Key bsr 125) + 1;\n        16 -> (Key bsr 124) + 1;\n        R ->\n            Step = n() div R,\n            Key div Step + 1\n    end.\n\n%% userdevguide-begin rt_simple:dump\n%% @doc Dumps the RT state for output in the web interface.\n-spec dump(RT::rt()) -> KeyValueList::[{Index::string(), Node::string()}].\ndump(Succ) -> [{\"0\", webhelpers:safe_html_string(\"~p\", [Succ])}].\n%% userdevguide-end rt_simple:dump\n\n%% @doc Checks whether config parameters of the rt_simple process exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_in(key_creator, [random, random_with_bit_mask]) and\n        case config:read(key_creator) of\n            random -> true;\n            random_with_bit_mask ->\n                config:cfg_is_tuple(key_creator_bitmask, 2,\n                                fun({Mask1, Mask2}) ->\n                                        erlang:is_integer(Mask1) andalso\n                                            erlang:is_integer(Mask2) end,\n                                \"{int(), int()}\");\n            _ -> false\n        end.\n\n%% @doc No special handling of messages, i.e. all messages are queued.\n-spec handle_custom_message_inactive(comm:message(), msg_queue:msg_queue()) ->\n    msg_queue:msg_queue().\nhandle_custom_message_inactive(Msg, MsgQueue) ->\n    msg_queue:add(MsgQueue, Msg).\n\n%% userdevguide-begin rt_simple:handle_custom_message\n%% @doc There are no custom messages here.\n-spec handle_custom_message\n        (custom_message() | any(), rt_loop:state_active()) -> unknown_event.\nhandle_custom_message(_Message, _State) -> unknown_event.\n%% userdevguide-end rt_simple:handle_custom_message\n\n%% userdevguide-begin rt_simple:check\n%% @doc Notifies the dht_node and failure detector if the routing table changed.\n%%      Provided for convenience (see check/5).\n-spec check(OldRT::rt(), NewRT::rt(), OldERT::external_rt(),\n                Neighbors::nodelist:neighborhood(),\n                ReportToFD::boolean()) -> NewERT::external_rt().\ncheck(OldRT, NewRT, OldERT, Neighbors, ReportToFD) ->\n    check(OldRT, NewRT, OldERT, Neighbors, Neighbors, ReportToFD).\n\n%% @doc Notifies the dht_node if the (external) routing table changed.\n%%      Also updates the failure detector if ReportToFD is set.\n%%      Note: the external routing table only changes the internal RT has\n%%      changed.\n-spec check(OldRT::rt(), NewRT::rt(), OldERT::external_rt(),\n                OldNeighbors::nodelist:neighborhood(), NewNeighbors::nodelist:neighborhood(),\n                ReportToFD::boolean()) -> NewERT::external_rt().\ncheck(OldRT, NewRT, OldERT, _OldNeighbors, NewNeighbors, ReportToFD) ->\n    case OldRT =:= NewRT of\n        true -> OldERT;\n        _ ->\n            Pid = node:pidX(nodelist:node(NewNeighbors)),\n            NewERT = export_rt_to_dht_node(NewRT, NewNeighbors),\n            comm:send_local(comm:make_local(Pid), {rt_update, NewERT}),\n            % update failure detector:\n            case ReportToFD of\n                true ->\n                    NewPids = to_pid_list(NewRT),\n                    OldPids = to_pid_list(OldRT),\n                    fd:update_subscriptions(self(), OldPids, NewPids);\n                _ -> ok\n            end,\n            NewERT\n    end.\n%% userdevguide-end rt_simple:check\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Communication with dht_node\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% userdevguide-begin rt_simple:empty_ext\n-spec empty_ext(nodelist:neighborhood()) -> external_rt().\nempty_ext(Neighbors) -> empty(Neighbors).\n%% userdevguide-end rt_simple:empty_ext\n\n%% userdevguide-begin rt_simple:next_hop\n%% @doc Returns the next hop to contact for a lookup.\n-spec next_hop(nodelist:neighborhood(), external_rt(), key()) -> succ | comm:mypid().\nnext_hop(Neighbors, RT, Id) ->\n    case intervals:in(Id, nodelist:succ_range(Neighbors)) of\n        true -> succ;\n        _    -> node:pidX(RT)\n    end.\n%% userdevguide-end rt_simple:next_hop\n\n\n%% @doc Return the succ\n%%      No need to lookup the succ in the ERT, only dht_node pids are used anyway.\n-spec succ(ERT::external_rt(), Neighbors::nodelist:neighborhood()) -> comm:mypid().\nsucc(_ERT, Neighbors) ->\n    node:pidX(nodelist:succ(Neighbors)).\n\n%% userdevguide-begin rt_simple:export_rt_to_dht_node\n%% @doc Converts the internal RT to the external RT used by the dht_node. Both\n%%      are the same here.\n-spec export_rt_to_dht_node(rt(), Neighbors::nodelist:neighborhood()) -> external_rt().\nexport_rt_to_dht_node(RT, _Neighbors) -> RT.\n%% userdevguide-end rt_simple:export_rt_to_dht_node\n\n%% userdevguide-begin rt_simple:to_list\n%% @doc Converts the (external) representation of the routing table to a list\n%%      {Id, Pid} tuples in the order of the fingers, i.e. first=succ,\n%%      second=shortest finger, third=next longer finger,...\n-spec to_list(dht_node_state:state()) -> [{key(), comm:mypid()}].\nto_list(State) ->\n    Succ = dht_node_state:get(State, rt), % ERT = Succ\n    [{node:id(Succ), node:pidX(Succ)}].\n%% userdevguide-end rt_simple:to_list\n\n%% userdevguide-begin rt_simple:wrap_message\n%% @doc Wrap lookup messages. This is a noop in rt_simple.\n-spec wrap_message(Key::key(), Msg::comm:message(), MyERT::external_rt(),\n                   Neighbors::nodelist:neighborhood(),\n                   Hops::non_neg_integer()) -> comm:message().\nwrap_message(_Key, Msg, _MyERT, _Neighbors, _Hops) -> Msg.\n%% userdevguide-end rt_simple:wrap_message\n\n%% userdevguide-begin rt_simple:unwrap_message\n%% @doc Unwrap lookup messages. This is a noop in rt_simple.\n-spec unwrap_message(Msg::comm:message(), State::dht_node_state:state()) -> comm:message().\nunwrap_message(Msg, _State) -> Msg.\n%% userdevguide-end rt_simple:unwrap_message\n"
  },
  {
    "path": "src/scalaris.erl",
    "content": "% @copyright 2007-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Starting and stopping the scalaris app.\n-module(scalaris).\n-author('schuett@zib.de').\n\n-include(\"scalaris.hrl\").\n\n%% functions called by Erlangs init module, triggered via command line\n%% (bin/scalarisctl and erl ... '-s scalaris')\n-export([start/0, stop/0, load/0]).\n%% (bin/scalarisctl and erl ... '-s scalaris cli')\n-export([cli/0, process/1]).\n\n%% functions called by application:start(scalaris)\n%% triggered by ?MODULE:start/0.\n-behaviour(application).\n-export([start/2, stop/1]).\n\n%% functions called by Erlangs init module, triggered via command line\n%% (bin/scalarisctl and erl ... '-s scalaris')\n-spec start() -> ok | {error, Reason::term()}.\nstart() ->\n    _ = load(),\n    %% preload at least the API modules (for Erlang shell usage)\n    _ = code:ensure_loaded(api_dht),\n    _ = code:ensure_loaded(api_dht_raw),\n    _ = code:ensure_loaded(api_monitor),\n    _ = code:ensure_loaded(api_rdht),\n    _ = code:ensure_loaded(api_ring),\n    _ = code:ensure_loaded(api_tx),\n    _ = code:ensure_loaded(api_vm),\n    _ = code:ensure_loaded(api_autoscale),\n    _ = code:ensure_loaded(api_mr),\n    _ = code:ensure_loaded(trace_mpath),\n    application:start(scalaris).\n\n-spec stop() -> ok | {error, Reason::term()}.\nstop() ->\n    application:stop(scalaris).\n\n-spec load() -> 'ok' | {'error', term()}.\nload() ->\n    %% provide load/0 as a separate function (called from\n    %% 'bin/scalarisctl checkinstallation').\n    %% We need to have the scalaris application loaded to get access\n    %% to the scalaris application environment variables provided with\n    %% the '-scalaris key value' syntax on the 'erl' command line.\n    %% The mechanism is used to check whether config files can be\n    %% found without actually starting the application.\n    application:load(\n      {application, scalaris,\n       [{description, \"scalaris\"},\n        {vsn, ?SCALARIS_VERSION},\n        {mod, {scalaris, []}},\n        {registered, []},\n        {applications, [kernel, stdlib]},\n        {env, []}\n       ]}).\n\n%% functions called by application:start(scalaris)\n%% triggered by ?MODULE:start/0.\n-spec start(StartType::normal, StartArgs::[])\n        -> {ok, Pid::pid()}.\nstart(normal, []) ->\n    try\n        util:if_verbose(\"~nAlready registered: ~p.~n\", [erlang:registered()]),\n        util:if_verbose(\"Running with node name ~p.~n\", [node()]),\n        config:init([]),\n        {ok, _LogPid} = log:start_link(),\n        case config:read(start_type) of\n            client ->\n                _ = inets:start(), % for jsonrpc\n                ok;\n            _ ->\n                {ok, _PidGroupsPid} = pid_groups:start_link(),\n                {ok, _YawsPid} = sup_yaws:start_link(),\n                case sup_scalaris:start_link() of\n                    %% ignore -> {error, ignore}; % no longer needed as dialyzer states\n                    X = {ok, Pid} when is_pid(Pid) ->\n                        comm:send_local(service_per_vm, {scalaris_says_hi}),\n                        X\n                end\n        end\n    catch\n        Error:Reason ->\n            io:format(\"Ups, Scalaris crashed with ~p:~p~n~p~n\",\n                      [Error, Reason, util:get_stacktrace()]),\n            halt(1)\n    end.\n\n-spec stop(any()) -> ok.\nstop(_State) ->\n    sup:sup_terminate(main_sup),\n%%  would prevent proper VM termination: (so leave it as a comment for now)\n%%  sup_scalaris:stop_first_services(),\n    ok.\n\n%% functions called by Erlangs init module, triggered via command line\n%% (bin/scalarisctl and erl ... '-s scalaris cli')\n-spec cli() -> ok.\ncli() ->\n    case init:get_plain_arguments() of\n        [NodeName | Args] ->\n            Node = list_to_atom(NodeName),\n            io:format(\"~p~n\", [Node]),\n            case rpc:call(Node, ?MODULE, process, [Args]) of\n                {badrpc, Reason} ->\n                    io:format(\"badrpc to ~p: ~p~n\", [Node, Reason]),\n                    init:stop(1),\n                    receive nothing -> ok end;\n                _ ->\n                    init:stop(0),\n                    receive nothing -> ok end\n            end;\n        _ ->\n            print_usage(),\n            init:stop(1),\n            receive nothing -> ok end\n    end.\n\n-spec print_usage() -> ok.\nprint_usage() ->\n    io:format(\"usage info~n\", []).\n\n-spec process([string()]) -> ok.\nprocess([\"stop\"]) ->\n    init:stop();\nprocess(Args) ->\n    io:format(\"got process(~p)~n\", [Args]).\n"
  },
  {
    "path": "src/service_per_vm.erl",
    "content": "% @copyright 2010-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Handling all Scalaris nodes inside an Erlang VM and Erlang VM wide\n%%      maintenance tasks.\n%% @version $Id$\n-module(service_per_vm).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-export([dump_node_states/0, kill_nodes/1, kill_nodes_by_name/1, register_dht_node/1,\n         deregister_dht_node/1, is_scalaris_ready/0]).\n\n-export([start_link/1, init/1, on/2]).\n\n% state of the module\n-type state() :: {[{MonitorRef::reference(), DhtNode::comm:mypid()}], boolean(), boolean()}.\n\n% accepted messages the module\n-type message() ::\n    {get_dht_nodes, ReplyPid :: comm:mypid()} |\n    {register_dht_node, PidToAdd :: comm:mypid()} |\n    {deregister_dht_node, PidToRemove :: comm:mypid()} |\n    {'DOWN', MonitorRef::reference(), process, Owner::comm:erl_local_pid(), Info::any()} |\n    {delete_node, SupPid::pid() | atom(), SupId::pid() | term()} |\n    {trigger_gc} | %% only internally inside the process\n    {hi}.\n\n-include(\"scalaris.hrl\").\n-include(\"gen_component.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Public API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% @doc ask all local nodes for their state\n-spec dump_node_states() -> [term()].\ndump_node_states() ->\n    [gen_component:get_state(Pid)\n     || Pid <- pid_groups:find_all(dht_node)].\n\n-spec kill_nodes(No::non_neg_integer()) -> ok.\nkill_nodes(No) ->\n    Childs = lists:sublist([X || X <- supervisor:which_children(main_sup),\n                                 is_list(element(1, X))], No),\n    _ = [begin\n             SupDhtNode = element(2, Child),\n             Id = element(1, Child),\n             sup:sup_terminate_childs(SupDhtNode),\n             _ = supervisor:terminate_child(main_sup, Id),\n             supervisor:delete_child(main_sup, Id)\n         end || Child <- Childs],\n    ok.\n\n% @doc kills Scalaris nodes from the current VM \n-spec kill_nodes_by_name(Names::list(pid_groups:groupname())) -> ok.\nkill_nodes_by_name(Names) ->\n    comm:send_local(service_per_vm, {kill_nodes_by_name, Names}),\n    ok.\n\n%% @doc Sends a register message to a running service_per_vm to register a\n%%      local(!) dht_node process.\n-spec register_dht_node(comm:mypid()) -> ok.\nregister_dht_node(Pid) ->\n    case whereis(service_per_vm) of\n        undefined -> ok;\n        Service   -> comm:send_local(Service, {register_dht_node, Pid})\n    end.\n\n%% @doc Sends a deregister message to a running service_per_vm to remove a\n%%      local(!) dht_node process.\n-spec deregister_dht_node(comm:mypid()) -> ok.\nderegister_dht_node(Pid) ->\n    case whereis(service_per_vm) of\n        undefined -> ok;\n        Service   -> comm:send_local(Service, {deregister_dht_node, Pid})\n    end.\n\n-spec is_scalaris_ready() -> boolean().\nis_scalaris_ready() ->\n    case whereis(service_per_vm) of\n        undefined ->\n            false;\n        ServicePid ->\n            %% comm:this() can't be trusted yet\n            comm:send_local(ServicePid, {is_ready_local, self()}),\n            trace_mpath:thread_yield(),\n            receive\n                ?SCALARIS_RECV({is_ready_local_response, Ready}, %% ->\n                               Ready)\n            end\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Server process\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(ServiceGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                             [{erlang_register, service_per_vm},\n                              {pid_groups_join_as, ServiceGroup, ?MODULE},\n                              {wait_for_init}]).\n\n-spec init(any()) -> state().\ninit(_Arg) ->\n    %% find my IP address\n    comm:init_and_wait_for_valid_IP(),\n    msg_delay:send_trigger(10, {trigger_gc}),\n    {[], false, false}.\n\n-spec on(Message :: message(), State :: state()) -> state().\n\n% @doc registers a dht node\non({register_dht_node, Pid}, {Nodes, CommHasStarted, SupHasStarted}) ->\n    % only local processes may register!\n    MonRef = gen_component:monitor(comm:make_local(comm:get_plain_pid(Pid))),\n    {[{MonRef, Pid} | Nodes], CommHasStarted, SupHasStarted};\n\n% @doc de-registers a dht node\non({deregister_dht_node, Pid}, {Nodes, CommHasStarted, SupHasStarted} = State) ->\n    case lists:keytake(Pid, 2, Nodes) of\n        {value, {MonRef, Pid}, Rest} ->\n            % allow erlang_demonitor to take its DOWN message off the message\n            % queue if present!\n            gen_component:demonitor(MonRef, [flush]),\n            {Rest, CommHasStarted, SupHasStarted};\n        false -> State\n    end;\n\non({'DOWN', MonitorRef, process, _LocalPid, _Info1}, {Nodes, CommHasStarted, SupHasStarted}) ->\n    % compare the monitor reference only (it is tied to the Pid and unique)\n    gen_component:demonitor(MonitorRef),\n    {[X || {MonRef, _Node} = X <- Nodes, MonRef =/= MonitorRef], CommHasStarted, SupHasStarted};\n\n% @doc replies with the list of registered dht nodes\non({get_dht_nodes, Pid}, {Nodes, _CommHasStarted, _SupHasStarted} = State) ->\n    case comm:is_valid(Pid) of\n        true ->\n            Pids = [Node || {_MonRef, Node} <- Nodes],\n            comm:send(Pid, {get_dht_nodes_response, Pids});\n        false ->\n            ok\n    end,\n    State;\n\non({delete_node, SupPid, SupId}, State) ->\n    _ = admin:del_node({SupId, SupPid, supervisor, []}, false),\n    State;\n\non({kill_nodes_by_name, Names}, State) ->\n    _ = admin:del_nodes_by_name(Names, false),\n    State;\n\non({add_node, Options}, State) ->\n    _ = admin:add_node(Options),\n    State;\n\non({trigger_gc}, State) ->\n    %% Periodically garbage collect all processes of the VM.  This\n    %% helps especially when running several VMs on the same machine.\n    %% Otherwise the Erlang VM will not release allocated memory and\n    %% the operating system may start swapping virtual memory,\n    %% although the Erlang VM could easily reduce its memory footprint\n    %% and swapping would not be necessary.  With periodic garbage\n    %% collection, Erlang becomes much more cooperative with other\n    %% processes running on the same machine.\n    msg_delay:send_trigger(10, {trigger_gc}),\n    %% io:format(\"Garbage collect all processes~n\"),\n    _ = [garbage_collect(X) || X <- processes()],\n    State;\n\n%% message from comm:init_and_wait_for_valid_IP/0 (no reply needed)\non({comm_says_hi}, {Nodes, _CommHasStarted, SupHasStarted}) ->\n    {Nodes, true, SupHasStarted};\n\non({scalaris_says_hi}, {Nodes, CommHasStarted, _SupHasStarted}) ->\n    {Nodes, CommHasStarted, true};\n\non({is_ready_local, Pid}, {_Nodes, CommHasStarted, SupHasStarted} = State) ->\n    comm:send_local(Pid, {is_ready_local_response, CommHasStarted andalso SupHasStarted}),\n    State.\n"
  },
  {
    "path": "src/slide_beh.erl",
    "content": "%  @copyright 2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Behaviour for rm-dependant slide functionality.\n%% @end\n%% @version $Id$\n-module(slide_beh).\n-author('kruber@zib.de').\n-vsn('$Id$ ').\n\n-ifndef(have_callback_support).\n-export([behaviour_info/1]).\n-endif.\n\n-ifdef(have_callback_support).\n-callback prepare_join_send(\n            State::dht_node_state:state(), SlideOp::slide_op:slide_op())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()} |\n           {abort, AbortReason::dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}.\n-callback prepare_rcv_data(\n            State::dht_node_state:state(), SlideOp::slide_op:slide_op())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()} |\n           {abort, AbortReason::dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}.\n-callback prepare_send_data1(\n            State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n            ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()} |\n           {abort, AbortReason::dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}.\n-callback prepare_send_data2(\n            State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n            EmbeddedMsg::{continue})\n        -> {ok, dht_node_state:state(), slide_op:slide_op()} |\n           {abort, AbortReason::dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}.\n-callback update_rcv_data1(\n            State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n            ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()} |\n           {abort, AbortReason::dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}.\n-callback update_rcv_data2(\n            State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n            EmbeddedMsg::comm:message())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()} |\n           {abort, AbortReason::dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}.\n-callback prepare_send_delta1(\n            State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n            ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()} |\n           {abort, AbortReason::dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}.\n-callback prepare_send_delta2(\n            State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n            EmbeddedMsg::comm:message())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()} |\n           {abort, AbortReason::dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}.\n-callback finish_delta1(\n            State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n            ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()} |\n           {abort, AbortReason::dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}.\n-callback finish_delta2(\n            State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n            EmbeddedMsg::comm:message())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()} |\n           {abort, AbortReason::dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}.\n-callback finish_delta_ack1(\n            State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n            ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()} |\n           {abort, AbortReason::dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}.\n-callback finish_delta_ack2(\n            State::dht_node_state:state(), SlideOp::slide_op:slide_op(), NextOpMsg,\n            EmbeddedMsg::comm:message())\n        -> {ok, dht_node_state:state(), slide_op:slide_op(), NextOpMsg} |\n           {abort, AbortReason::dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}\n        when is_subtype(NextOpMsg, dht_node_move:next_op_msg()).\n-callback abort_slide(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                      Reason::dht_node_move:abort_reason(), MoveMsgTag::atom())\n        -> dht_node_state:state().\n-else.\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n    [\n     {prepare_join_send, 2},\n     {prepare_rcv_data, 2},\n     {prepare_send_data1, 3},\n     {prepare_send_data2, 3},\n     {update_rcv_data1, 3},\n     {update_rcv_data2, 3},\n     {prepare_send_delta1, 3},\n     {prepare_send_delta2, 3},\n     {finish_delta1, 3},\n     {finish_delta2, 3},\n     {finish_delta_ack1, 3},\n     {finish_delta_ack2, 4},\n     {abort_slide, 4}\n    ];\nbehaviour_info(_Other) ->\n    undefined.\n-endif.\n"
  },
  {
    "path": "src/slide_chord.erl",
    "content": "%% @copyright 2010-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Slide protocol used for Chord ring maintenance (also see the\n%%         dht_node_move module).\n%%         Note: assumes that the neighborhood does not change during the\n%%         handling of a message.\n%% @end\n%% @version $Id$\n-module(slide_chord).\n-author('kruber@zib.de').\n-vsn('$Id$').\n-behaviour(slide_beh).\n\n-include(\"scalaris.hrl\").\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-define(TRACE_SEND(Pid, Msg), ?TRACE(\"[ ~.0p ] to ~.0p: ~.0p~n\", [self(), Pid, Msg])).\n\n-export([prepare_join_send/2, prepare_rcv_data/2,\n         prepare_send_data1/3, prepare_send_data2/3,\n         update_rcv_data1/3, update_rcv_data2/3,\n         prepare_send_delta1/3, prepare_send_delta2/3,\n         finish_delta1/3, finish_delta2/3,\n         finish_delta_ack1/3, finish_delta_ack2/4,\n         abort_slide/4]).\n\n-export([rm_exec/5]).\n\n-spec prepare_join_send(State::dht_node_state:state(), SlideOp::slide_op:slide_op())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nprepare_join_send(State, SlideOp) ->\n    % in ordinary slides, the DB range is extended right before the\n    % other node is instructed to change the ID - during a join this is\n    % the case after receiving the initial join_request -> set it now!\n    State1 = dht_node_state:add_db_range(\n               State, slide_op:get_interval(SlideOp),\n               slide_op:get_id(SlideOp)),\n    {ok, State1, SlideOp}.\n\n-spec prepare_rcv_data(State::dht_node_state:state(), SlideOp::slide_op:slide_op())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nprepare_rcv_data(State, SlideOp) ->\n    Type = slide_op:get_type(SlideOp),\n    % message forward on succ is set before the pred can change its ID;\n    % on pred in ordinary slides, the message forward will be set right before\n    % the ID is going to be changed;\n    % during a join we already set the new ID -> set message forward now, too!\n    SlideOp1 = case (slide_op:get_sendORreceive(Type) =:= 'rcv' andalso\n                         slide_op:get_predORsucc(Type) =:= pred)\n                        orelse slide_op:is_join(Type, 'rcv') of\n                   true ->\n                       slide_op:set_msg_fwd(SlideOp);\n                   _ -> SlideOp\n               end,\n    {ok, State, SlideOp1}.\n\n%% @doc Change the local node's ID to the given TargetId by calling the ring\n%%      maintenance and sending a continue message when the node is up-to-date.\n-spec change_my_id(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                   ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nchange_my_id(State, SlideOp, ReplyPid) ->\n    case slide_op:get_sendORreceive(SlideOp) of\n        'send' ->\n            State1 = dht_node_state:add_db_range(\n                       State, slide_op:get_interval(SlideOp),\n                       slide_op:get_id(SlideOp)),\n            SlideOp2 = SlideOp;\n        'rcv'  ->\n            State1 = State,\n            SlideOp2 = slide_op:set_msg_fwd(SlideOp)\n    end,\n    TargetId = slide_op:get_target_id(SlideOp2),\n    case slide_op:is_leave(SlideOp2) andalso\n             slide_op:get_next_op(SlideOp2) =:= {none} of\n        true ->\n            Tag = ?IIF(slide_op:is_jump(SlideOp2), jump, leave),\n            rm_loop:leave(Tag),\n            % de-activate processes not needed anymore:\n            dht_node_reregister:deactivate(),\n            % note: do not deactivate gossip, vivaldi or dc_clustering -\n            % their values are still valid and still count!\n            dn_cache:unsubscribe(),\n            rt_loop:deactivate(),\n            service_per_vm:deregister_dht_node(comm:this()),\n            {ok, State1, SlideOp2};\n        _ ->\n            rm_loop:subscribe(\n              ReplyPid, {move, slide_op:get_id(SlideOp2)},\n              fun(_OldN, NewN, Reason) ->\n                      nodelist:nodeid(NewN) =:= TargetId orelse\n                          Reason =:= {update_id_failed}\n              % note: no need to check the id version\n              end,\n              fun ?MODULE:rm_exec/5,\n              1),\n            rm_loop:update_id(TargetId),\n            {ok, State1, SlideOp2}\n    end.\n\n%% @doc Prepares sending data for the given (existing!) slide operation and\n%%      changes the own ID if necessary.\n%% @see prepare_send_data2/3\n%% @see dht_node_move:prepare_send_data1/2\n-spec prepare_send_data1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                         ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nprepare_send_data1(State, SlideOp, ReplyPid) ->\n    case slide_op:get_predORsucc(SlideOp) of\n        succ -> change_my_id(State, SlideOp, ReplyPid);\n        pred -> State1 =\n                    case slide_op:get_type(SlideOp) of\n                        {join, 'send'} -> % already set\n                            State;\n                        _ ->\n                            dht_node_state:add_db_range(\n                              State, slide_op:get_interval(SlideOp),\n                              slide_op:get_id(SlideOp))\n                    end,\n                send_continue_msg(ReplyPid),\n                {ok, State1, SlideOp}\n    end.\n\n%% @doc Cleans up after prepare_send_data1/3 once the RM is up-to-date, (no-op here).\n%% @see prepare_send_data1/3\n%% @see dht_node_move:prepare_send_data2/3\n-spec prepare_send_data2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                         EmbeddedMsg::{continue})\n        -> {ok, dht_node_state:state(), slide_op:slide_op()};\n                        (State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                         EmbeddedMsg::{abort})\n        -> {abort, dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}.\nprepare_send_data2(State, SlideOp, {continue}) ->\n    {ok, State, SlideOp};\nprepare_send_data2(State, SlideOp, {abort}) ->\n    {abort, target_id_not_in_range, State, SlideOp}.\n\n%% @doc Accepts data received during the given (existing!) slide operation,\n%%      writes it to the DB and changes the own ID if necessary.\n%% @see update_rcv_data2/3\n%% @see dht_node_move:update_rcv_data1/5\n-spec update_rcv_data1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                       ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nupdate_rcv_data1(State, SlideOp, ReplyPid) ->\n    case slide_op:get_predORsucc(SlideOp) of\n        succ -> change_my_id(State, SlideOp, ReplyPid);\n        pred -> send_continue_msg(ReplyPid),\n                {ok, State, SlideOp}\n    end.\n\n%% @doc Cleans up after update_rcv_data1/3 once the RM is up-to-date, (no-op here).\n%% @see update_rcv_data1/3\n%% @see dht_node_move:update_rcv_data2/3\n-spec update_rcv_data2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                       EmbeddedMsg::{continue})\n        -> {ok, dht_node_state:state(), slide_op:slide_op()};\n                      (State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                       EmbeddedMsg::{abort})\n        -> {abort, dht_node_move:abort_reason(), dht_node_state:state(), slide_op:slide_op()}.\nupdate_rcv_data2(State, SlideOp, {continue}) ->\n    {ok, State, SlideOp};\nupdate_rcv_data2(State, SlideOp, {abort}) ->\n    {abort, target_id_not_in_range, State, SlideOp}.\n\n-spec send_continue_msg(Pid::comm:erl_local_pid()) -> ok.\nsend_continue_msg(Pid) ->\n    ?TRACE_SEND(Pid, {continue}),\n    comm:send_local(Pid, {continue}).\n\n-spec send_abort_msg(Pid::comm:erl_local_pid()) -> ok.\nsend_abort_msg(Pid) ->\n    ?TRACE_SEND(Pid, {abort}),\n    comm:send_local(Pid, {abort}).\n\n-spec send_continue_msg_when_pred_ok(\n        State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n        ReplyPid::comm:erl_local_pid()) -> ok.\nsend_continue_msg_when_pred_ok(State, SlideOp, ReplyPid) ->\n    ExpPredId = slide_op:get_target_id(SlideOp),\n    case dht_node_state:get(State, pred_id) of\n        ExpPredId ->\n            send_continue_msg(ReplyPid);\n        _OldPredId ->\n            OldPred = dht_node_state:get(State, pred),\n            rm_loop:subscribe(\n              ReplyPid, {move, slide_op:get_id(SlideOp)},\n              fun(_RMOldN, RMNewN, _Reason) ->\n                      RMNewPred = nodelist:pred(RMNewN),\n                      % new pred pid or same pid but (updated) ID\n                      PredChanged = RMNewPred =/= OldPred,\n                      ?DBG_ASSERT2(not (PredChanged andalso\n                                            node:pidX(RMNewPred) =:= node:pidX(OldPred)) orelse\n                                       node:id(RMNewPred) =:= ExpPredId,\n                                   {\"unexpected pred ID change\", _OldPredId,\n                                    node:id(RMNewPred), ExpPredId}),\n                      PredChanged\n              end,\n              fun ?MODULE:rm_exec/5,\n              1)\n    end.\n\n%% @doc Accepts data_ack received during the given (existing!) slide operation\n%%      and continues be sending a message to ReplyPid (if sending to pred,\n%%      right after the RM is up-to-date).\n%% @see prepare_send_delta2/3\n%% @see dht_node_move:prepare_send_delta1/2\n-spec prepare_send_delta1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                          ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nprepare_send_delta1(State, OldSlideOp, ReplyPid) ->\n    case slide_op:get_predORsucc(OldSlideOp) of\n        succ -> send_continue_msg(ReplyPid);\n        pred -> send_continue_msg_when_pred_ok(State, OldSlideOp, ReplyPid)\n    end,\n    {ok, State, OldSlideOp}.\n\n%% @doc Cleans up after prepare_send_delta1/3 once the RM is up-to-date, e.g.\n%%      removes temporary additional db_range entries.\n%% @see prepare_send_delta1/3\n%% @see dht_node_move:prepare_send_delta2/3\n-spec prepare_send_delta2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                          EmbeddedMsg::{continue})\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nprepare_send_delta2(State, SlideOp, {continue}) ->\n    MoveFullId = slide_op:get_id(SlideOp),\n    {ok, dht_node_state:rm_db_range(State, MoveFullId), SlideOp}.\n\n%% @doc Removes the dht_node's message forward for the slide operation's\n%%      interval and continues by sending a message to ReplyPid\n%%      (if receiving from pred, right after the RM is up-to-date).\n%% @see finish_delta2/3\n%% @see dht_node_move:finish_delta1/3\n-spec finish_delta1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                    ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nfinish_delta1(State, OldSlideOp, ReplyPid) ->\n    SlideOp = slide_op:remove_msg_fwd(OldSlideOp),\n    case slide_op:get_predORsucc(SlideOp) of\n        succ ->\n            send_continue_msg(ReplyPid),\n            {ok, State, SlideOp};\n        pred ->\n            send_continue_msg_when_pred_ok(State, SlideOp, ReplyPid),\n            % optimization: until we know about the new id of our pred (or a\n            % new pred or the continue message), add the range to the db_range\n            % so our node already responds to such messages\n            {ok, dht_node_state:add_db_range(\n               State, slide_op:get_interval(SlideOp), slide_op:get_id(SlideOp)),\n             SlideOp}\n    end.\n\n%% @doc Cleans up after finish_delta1/4 once the RM is up-to-date, e.g. removes\n%%      temporary additional db_range entries.\n%% @see finish_delta1/4\n%% @see dht_node_move:finish_delta2/3\n-spec finish_delta2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                    EmbeddedMsg::{continue}) -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nfinish_delta2(State, SlideOp, {continue}) ->\n    MoveFullId = slide_op:get_id(SlideOp),\n    {ok, dht_node_state:rm_db_range(State, MoveFullId), SlideOp}.\n\n%% @doc No-op with chord RT.\n%% @see finish_delta_ack2/3\n%% @see dht_node_move:finish_delta_ack1/2\n-spec finish_delta_ack1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                        ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nfinish_delta_ack1(State, OldSlideOp, ReplyPid) ->\n    send_continue_msg(ReplyPid),\n    {ok, State, OldSlideOp}.\n\n%% @doc No-op with chord RT.\n%% @see finish_delta_ack1/3\n%% @see dht_node_move:finish_delta_ack2/3\n-spec finish_delta_ack2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                        NextOpMsg, EmbeddedMsg::{continue})\n        -> {ok, dht_node_state:state(), slide_op:slide_op(), NextOpMsg}\n        when is_subtype(NextOpMsg, dht_node_move:next_op_msg()).\nfinish_delta_ack2(State, SlideOp, NextOpMsg, {continue}) ->\n    {ok, State, SlideOp, NextOpMsg}.\n\n%% @doc Executed when aborting the given slide operation (assumes the SlideOp\n%%      has already been set up).\n-spec abort_slide(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                  Reason::dht_node_move:abort_reason(), MoveMsgTag::atom())\n        -> dht_node_state:state().\nabort_slide(State, SlideOp, Reason, MoveMsgTag) ->\n    case slide_op:get_phase(SlideOp) of\n        {wait_for_continue, Phase} -> ok;\n        Phase -> ok\n    end,\n    SendOrRcv = slide_op:get_sendORreceive(SlideOp),\n\n    % revert ID change?\n    case slide_op:get_predORsucc(SlideOp) of\n        succ ->\n            % try to change ID back (if not the first receiving join slide)\n            case SendOrRcv of\n                'rcv' when Reason =:= target_down ->\n                    % if ID already changed, keep it; as well as the already incorporated data\n                    ok;\n                _ ->\n                    MyId = dht_node_state:get(State, node_id),\n                    case slide_op:get_my_old_id(SlideOp) of\n                        null -> ok;\n                        MyId -> ok;\n                        _ when Phase =:= wait_for_delta_ack -> ok;\n                        MyOldId ->\n                            log:log(warn, \"[ dht_node_move ~.0p ] trying to revert ID change from ~p to ~p~n\",\n                                    [comm:this(), MyOldId, MyId]),\n                            rm_loop:update_id(MyOldId),\n                            ok\n                    end\n            end;\n        pred ->\n            % nothing we can do on this side\n            ok\n    end,\n\n    % remove incomplete/useless data?\n    case SendOrRcv of\n        'rcv' when Reason =:= target_down ->\n            % nothing to do\n            State;\n        'rcv' when (Phase =/= wait_for_delta orelse MoveMsgTag =/= delta_ack) ->\n            % remove incomplete data (note: this function will be called again\n            % as part of dht_node_move:abort_slide/4 but without deleting items!)\n            dht_node_state:slide_stop_record(State, slide_op:get_interval(SlideOp), true);\n        _ ->\n            State\n    end.\n\n-spec rm_exec(pid(), term(),\n              OldNeighbors::nodelist:neighborhood(),\n              NewNeighbors::nodelist:neighborhood(),\n              Reason::rm_loop:reason()) -> ok.\nrm_exec(Pid, {move, _RMSlideId}, _RMOldNeighbors, _RMNewNeighbors, Reason) ->\n    case Reason of\n        {update_id_failed} -> send_abort_msg(Pid);\n        _ -> send_continue_msg(Pid)\n    end.\n"
  },
  {
    "path": "src/slide_leases.erl",
    "content": "%% @copyright 2010-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schütt <schuett@zib.de>\n%% @doc    Slide protocol used for lease-based ring maintenance (also see the\n%%         dht_node_move module).\n%% @end\n%% @version $Id$\n-module(slide_leases).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(slide_beh).\n\n-include(\"scalaris.hrl\").\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-define(TRACE_SEND(Pid, Msg), ?TRACE(\"[ ~.0p ] to ~.0p: ~.0p~n\", [self(), Pid, Msg])).\n\n-export([prepare_join_send/2, prepare_rcv_data/2,\n         prepare_send_data1/3, prepare_send_data2/3,\n         update_rcv_data1/3, update_rcv_data2/3,\n         prepare_send_delta1/3, prepare_send_delta2/3,\n         finish_delta1/3, finish_delta2/3,\n         finish_delta_ack1/3, finish_delta_ack2/4,\n         abort_slide/4]).\n\n-spec prepare_join_send(State::dht_node_state:state(), SlideOp::slide_op:slide_op())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nprepare_join_send(State, SlideOp) ->\n    %log:log(\"prepare_join_send\", []),\n    % can be ignored for leases\n    {ok, State, SlideOp}.\n\n-spec prepare_rcv_data(State::dht_node_state:state(), SlideOp::slide_op:slide_op())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nprepare_rcv_data(State, SlideOp) ->\n    % do nothing\n    %io:format(\"prepare_rcv_data~n\", []),\n    {ok, State, SlideOp}.\n\n-spec prepare_send_data1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                         ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nprepare_send_data1(State, SlideOp, ReplyPid) ->\n    % do nothing\n    %io:format(\"prepare_send_data1~n\", []),\n    send_continue_msg(ReplyPid),\n    {ok, State, SlideOp}.\n\n-spec prepare_send_data2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                         EmbeddedMsg::{continue})\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nprepare_send_data2(State, SlideOp, {continue}) ->\n    % do nothing\n    %io:format(\"prepare_send_data2~n\", []),\n    {ok, State, SlideOp}.\n\n-spec update_rcv_data1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                       ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nupdate_rcv_data1(State, SlideOp, ReplyPid) ->\n    % do nothing\n    %io:format(\"update_rcv_data1~n\", []),\n    send_continue_msg(ReplyPid),\n    {ok, State, SlideOp}.\n\n-spec update_rcv_data2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                       EmbeddedMsg::{continue})\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nupdate_rcv_data2(State, SlideOp, {continue}) ->\n    % do nothing\n    %io:format(\"update_rcv_data2~n\", []),\n    {ok, State, SlideOp}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% split lease and disable lease\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prepare_send_delta1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                          ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()} |\n           {abort, Reason::{protocol_error, EmbeddedMsg::any()}, State::dht_node_state:state(), SlideOp1::slide_op:slide_op()}.\nprepare_send_delta1(State, OldSlideOp, ReplyPid) ->\n    % start to split own range\n    %log:log(\"prepare_send_delta1 ~p~n\", [slide_op:get_type(OldSlideOp)]),\n    case find_lease(State, OldSlideOp, active) of\n        {ok, Lease} ->\n            Id = l_on_cseq:id(l_on_cseq:get_range(Lease)),\n            % check slide direction\n            Interval = slide_op:get_interval(OldSlideOp),\n            case Interval =:= l_on_cseq:get_range(Lease) of\n                false ->\n                    case intervals:in(Id, Interval) of\n                        true ->\n                                                % ->\n                            log:log(\"unsupported slide direction in prepare_send_delta1~n\"),\n                            log:log(\"~p in ~p~n == false\", [Id, Interval]),\n                            log:log(\"~p~n\", [Lease]),\n                            {abort, {protocol_error, unsupported_slide_direction}, State, OldSlideOp};\n                        false ->\n                                                % <-\n                            {R1, R2} =  {Interval, intervals:minus(l_on_cseq:get_range(Lease), Interval)},\n                            NewOwner = node:pidX(slide_op:get_node(OldSlideOp)),\n                            l_on_cseq:lease_split_and_change_owner(Lease, R1, R2, NewOwner, ReplyPid),\n                            {ok, State, OldSlideOp}\n                    end;\n                true ->\n                    log:log(\"only change owner instead of split and change owner\", []),\n                    %% @todo\n                    %log:log(\"Id ~p ~n\", [Id]),\n                    %log:log(\"Interval     ~p\", [Interval]),\n                    %log:log(\"intervals:in ~p\", [intervals:in(Id, Interval)]),\n                    %log:log(\"lease        ~p\", [Lease]),\n                    %log:log(\"lease range  ~p\", [l_on_cseq:get_range(Lease)]),\n                    %log:log(\"bounds       ~p\", [intervals:get_bounds(l_on_cseq:get_range(Lease))]),\n                    %log:log(\"id           ~p\", [l_on_cseq:id(l_on_cseq:get_range(Lease))]),\n                    NewOwner = node:pidX(slide_op:get_node(OldSlideOp)),\n                    l_on_cseq:lease_handover(Lease, NewOwner, ReplyPid),\n                    {ok, State, OldSlideOp}\n            end;\n        error ->\n            % @todo\n            LeaseList = dht_node_state:get(State, lease_list),\n            ActiveLease = lease_list:get_active_lease(LeaseList),\n            PassiveLeaseList = lease_list:get_passive_leases(LeaseList),\n            Interval = slide_op:get_interval(OldSlideOp),\n            log:log(\"unknown lease in prepare_send_delta1~n\"),\n            log:log(\"~p:~p~n\", [ActiveLease, PassiveLeaseList]),\n            log:log(\"~p~n\", [Interval]),\n            {abort, {protocol_error, unknown_lease_in_prepare_send_delta}, State, OldSlideOp}\n    end.\n\n\n-spec prepare_send_delta2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                          EmbeddedMsg::any())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()} |\n           {abort, Reason::{protocol_error, EmbeddedMsg::any()}, State::dht_node_state:state(), SlideOp1::slide_op:slide_op()}.\nprepare_send_delta2(State, SlideOp, Msg) ->\n    % check that split has been done\n    case Msg of\n        {handover, success, _NewLease} ->\n            % disable new lease\n            %log:log(\"prepare_send_delta2 ~p~n\", [Msg]),\n            %State1 = locally_disable_lease(State, NewLease),\n            {ok, State, SlideOp};\n        {split, fail, _Lease} ->\n            log:log(\"prepare_send_delta2: split failed~n\", []),\n            {abort, {protocol_error, Msg}, State, SlideOp}\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec finish_delta1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                    ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nfinish_delta1(State, OldSlideOp, ReplyPid) ->\n    % do nothing\n    send_continue_msg(ReplyPid),\n    {ok, State, OldSlideOp}.\n\n-spec finish_delta2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                    EmbeddedMsg::{continue}) -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nfinish_delta2(State, SlideOp, {continue}) ->\n    % do nothing\n    {ok, State, SlideOp}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% handover\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec finish_delta_ack1(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                        ReplyPid::comm:erl_local_pid())\n        -> {ok, dht_node_state:state(), slide_op:slide_op()}.\nfinish_delta_ack1(State, OldSlideOp, ReplyPid) ->\n    % handover lease to succ\n    comm:send_local(ReplyPid, {continue}),\n    %log:log(\"finish_delta_ack1~n\", []),\n    {ok, State, OldSlideOp}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec finish_delta_ack2(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                        NextOpMsg, EmbeddedMsg::{continue})\n        -> {ok, dht_node_state:state(), slide_op:slide_op(), NextOpMsg}\n        when is_subtype(NextOpMsg, dht_node_move:next_op_msg()).\nfinish_delta_ack2(State, SlideOp, NextOpMsg, Msg) ->\n    % notify neighbor on successful handover\n    %log:log(\"finish_delta_ack2 ~p~n\", [_Msg]),\n    % notify succ\n    case find_lease(State, SlideOp, passive) of\n        {ok, Lease} ->\n            Owner = l_on_cseq:get_owner(Lease),\n            l_on_cseq:lease_send_lease_to_node(Owner, Lease, active),\n            State1 = lease_list:remove_lease_from_dht_node_state(Lease, l_on_cseq:get_id(Lease),\n                                                                 State, passive),\n            {ok, State1, SlideOp, NextOpMsg};\n        error ->\n            log:log(\"error in finish_delta_ack2 (~p)\", [comm:this()]),\n            {abort, {protocol_error, Msg}, State, SlideOp}\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Executed when aborting the given slide operation (assumes the SlideOp\n%%      has already been set up).\n-spec abort_slide(State::dht_node_state:state(), SlideOp::slide_op:slide_op(),\n                  Reason::dht_node_move:abort_reason(), MoveMsgTag::atom())\n        -> dht_node_state:state().\nabort_slide(State, _SlideOp, _Reason, _MoveMsgTag) ->\n    State.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% utility functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec send_continue_msg(Pid::comm:erl_local_pid()) -> ok.\nsend_continue_msg(Pid) ->\n    ?TRACE_SEND(Pid, {continue}),\n    comm:send_local(Pid, {continue}).\n\nfind_lease(State, SlideOp, Mode) ->\n    LeaseList = dht_node_state:get(State, lease_list),\n    Interval = slide_op:get_interval(SlideOp),\n    Pred = fun (L) ->\n                   intervals:is_subset(Interval, l_on_cseq:get_range(L))\n                       andalso intervals:is_continuous(\n                                   intervals:intersection(Interval,l_on_cseq:get_range(L)))\n           end,\n    ActiveLease = lease_list:get_active_lease(LeaseList),\n    PassiveLeases = lease_list:get_passive_leases(LeaseList),\n    case Mode of\n        active ->\n            case Pred(ActiveLease) of\n                true ->\n                    {ok, ActiveLease};\n                false ->\n                    error\n            end;\n        passive ->\n            case lists:filter(Pred, PassiveLeases) of\n                [Lease] ->\n                    {ok, Lease};\n                _ ->\n                    log:log(\"did not found requested lease in passive list:~n~w~n~w~n~w (~p)~n\",\n                            [Interval, ActiveLease, PassiveLeases, comm:this()]),\n                    error\n            end\n    end.\n"
  },
  {
    "path": "src/slide_op.erl",
    "content": "%  @copyright 2010-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Slide operation structure for node moves, joins and leaves.\n%% @end\n%% @version $Id$\n-module(slide_op).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-export([new_slide/8, new_slide_i/8,\n         new_receiving_slide_join/5,\n         new_sending_slide_join/4, new_sending_slide_join_i/5,\n         new_sending_slide_leave/5,\n         new_sending_slide_jump/6,\n         update_target_id/4,\n         other_type_to_my_type/1,\n         is_slide/1,\n         is_join/1, is_join/2, is_leave/1, is_leave/2, is_jump/1, is_jump/2,\n         is_incremental/1,\n         get_id/1, get_node/1, get_interval/1,\n         get_my_old_id/1, get_target_id/1, get_jump_target_id/1,\n         get_source_pid/1, get_tag/1, get_sendORreceive/1, get_type/1,\n         get_predORsucc/1,\n         get_time_last_send/1, get_time_next_warn/1,\n         set_time_last_send/2, set_time_next_warn/2,\n         get_send_errors/1, inc_send_errors/1, reset_send_errors/1,\n         get_phase/1, set_phase/2,\n         is_setup_at_other/1, set_setup_at_other/1,\n         get_next_op/1, set_next_op/2,\n         get_other_max_entries/1, set_other_max_entries/2,\n         get_msg_fwd/1, set_msg_fwd/1, remove_msg_fwd/1]).\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n%-define(TRACE(X,Y), log:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n\n-export_type([slide_op/0, id/0, phase/0, type/0, next_op/0]).\n\n-type id() :: uid:global_uid().\n\n-type type() ::\n        {slide, pred | succ, 'send' | 'rcv'} |\n        {join, 'send' | 'rcv'} |\n        {leave, 'send' | 'rcv'} |\n        {jump, 'send' | 'rcv'}.\n\n-type phase_plain() ::\n        null | % should only occur as an intermediate state, otherwise equal to \"no slide op\"\n        wait_for_other | % a node initiated a slide but needs more info from its partner\n        wait_for_data_ack | wait_for_delta_ack | % sending node\n        wait_for_data | wait_for_delta. % receiving node\n-type phase() ::\n        phase_plain() |\n        {wait_for_continue, phase_plain()}. % async (local) slide message to rm-specific implementation\n\n-type next_op() ::\n        {slide, continue, Id::?RT:key()} |\n        {jump, continue, Id::?RT:key()} |\n        {leave, continue} |\n        {slide, pred | succ, Id::?RT:key(), Tag::any(), SourcePid::comm:mypid() | null} |\n        {jump, Id::?RT:key(), Tag::any(), SourcePid::comm:mypid() | null} |\n        {leave, SourcePid::comm:mypid() | null} |\n        {none}.\n\n-record(slide_op,\n        {type              = ?required(slide_op, type)      :: type(),\n         id                = ?required(slide_op, id)        :: id(),\n         node              = ?required(slide_op, node)      :: node:node_type(), % the node, data is sent to/received from\n         interval          = ?required(slide_op, interval)  :: intervals:interval(), % send/receive data in this range\n         my_old_id         = ?required(slide_op, my_old_id) :: ?RT:key() | null, % ID before changing the own ID (null in case of receiving joins)\n         target_id         = ?required(slide_op, target_id) :: ?RT:key(), % ID to move the predecessor of the two participating nodes to\n         jump_target_id    = null                           :: ?RT:key() | null, % ID to jump to in case of a jump operation which is preceeded by a slide to leave\n         tag               = ?required(slide_op, tag)       :: any(),\n         source_pid        = null              :: comm:mypid() | null, % pid of the process that requested the move (and will thus receive a message about its state)\n         time_last_send    = never             :: erlang_timestamp() | never,\n         time_next_warn    = never             :: erlang_timestamp() | never,\n         send_errors       = 0                 :: non_neg_integer(),\n         phase             = null              :: phase(),\n         setup_at_other    = false             :: boolean(),\n         % note: use a format which does not require conversion when read\n         % -> the pid is already contained in node, but this should be faster\n         msg_fwd           = []                :: [{intervals:interval(), comm:mypid()}],\n         next_op           = {none}            :: next_op(),\n         other_max_entries = unknown           :: unknown | pos_integer()\n        }).\n-opaque slide_op() :: #slide_op{}.\n\n%% @doc Sets up a slide operation of the given type. One of the nodes will\n%%      change its ID to TargetId.\n-spec new_slide(MoveId::uid:global_uid(), Type::type(), CurTargetId::?RT:key(),\n                Tag::any(), SourcePid::comm:mypid() | null,\n                OtherMTE::unknown | pos_integer(), NextOp::next_op(),\n                Neighbors::nodelist:neighborhood())\n        -> slide_op().\nnew_slide(MoveId, Type, CurTargetId, Tag, SourcePid, OtherMTE, NextOp, Neighbors) ->\n    {PredOrSucc, SendOrReceive} =\n        case Type of\n            {slide, PoS, SoR} -> {PoS, SoR};\n            % do not handle \"join\" here -> use the specialized new_*_slide_join methods!\n            % creating slides for the leaving/jumping node must be done with new_slide_leave/jump!\n            {leave, 'rcv'} -> {pred, 'rcv'};\n            {jump, 'rcv'} ->  {pred, 'rcv'}\n        end,\n    {Interval, TargetNode} =\n        get_interval_tnode(PredOrSucc, SendOrReceive, CurTargetId, Neighbors),\n    #slide_op{type = Type,\n              id = MoveId,\n              node = TargetNode,\n              interval = Interval,\n              my_old_id = nodelist:nodeid(Neighbors),\n              target_id = CurTargetId,\n              tag = Tag,\n              source_pid = SourcePid,\n              next_op = NextOp,\n              other_max_entries = OtherMTE}.\n\n%% @doc Sets up an incremental slide operation of the given type. One of the\n%%      nodes will change its ID to CurTargetId and finally FinalTargetId.\n-spec new_slide_i(MoveId::uid:global_uid(), Type::type(),\n                CurTargetId::?RT:key(), FinalTargetId::?RT:key(),\n                Tag::any(), SourcePid::comm:erl_local_pid() | null,\n                OtherMTE::unknown | pos_integer(), Neighbors::nodelist:neighborhood())\n        -> slide_op().\nnew_slide_i(MoveId, Type, CurTargetId, FinalTargetId, Tag, SourcePid, OtherMTE, Neighbors) ->\n    NextOp = case FinalTargetId of\n                 CurTargetId -> {none};\n                 _           -> {slide, continue, FinalTargetId}\n             end,\n    new_slide(MoveId, Type, CurTargetId, Tag, SourcePid, OtherMTE, NextOp, Neighbors).\n\n-spec get_interval_tnode(PredOrSucc::pred | succ, SendOrReceive::'send' | 'rcv',\n                         TargetId::?RT:key(), Neighbors::nodelist:neighborhood())\n        -> {intervals:interval(), node:node_type()}.\nget_interval_tnode(PredOrSucc, SendOrReceive, TargetId, Neighbors) ->\n    case PredOrSucc of\n        pred ->\n            Pred = nodelist:pred(Neighbors),\n            PredId = node:id(Pred),\n            I = case SendOrReceive of\n                    'rcv'  -> node:mk_interval_between_ids(TargetId, PredId);\n                    'send' -> node:mk_interval_between_ids(PredId, TargetId)\n                end,\n            {I, Pred};\n        succ ->\n            NodeId = nodelist:nodeid(Neighbors),\n            I = case SendOrReceive of\n                    'rcv'  -> node:mk_interval_between_ids(NodeId, TargetId);\n                    'send' -> node:mk_interval_between_ids(TargetId, NodeId)\n                end,\n            {I, nodelist:succ(Neighbors)}\n    end.\n\n%% @doc Sets up a new slide operation for a joining node (see\n%%      dht_node_join.erl). MyKey is the joining node's new Id and will be used\n%%      as the target id of the slide operation.\n-spec new_receiving_slide_join(MoveId::uid:global_uid(), TargetId::?RT:key(),\n        Tag::any(), SourcePid::comm:mypid(), Neighbors::nodelist:neighborhood()) -> slide_op().\nnew_receiving_slide_join(MoveId, TargetId, Tag, SourcePid, Neighbors) ->\n    Pred = nodelist:pred(Neighbors),\n    TargetNode = nodelist:succ(Neighbors),\n    IntervalToReceive = node:mk_interval_between_ids(node:id(Pred), TargetId),\n    #slide_op{type = {join, 'rcv'},\n              id = MoveId,\n              node = TargetNode,\n              interval = IntervalToReceive,\n              my_old_id = null,\n              target_id = TargetId,\n              tag = Tag,\n              source_pid = SourcePid}.\n\n%% @doc Sets up a new slide operation for a node which sends a joining node\n%%      some of its data.\n%%      Throws 'throw:not_responsible' if the current node is not responsible\n%%      for the ID of JoiningNode.\n-spec new_sending_slide_join(MoveId::uid:global_uid(), JoiningNode::node:node_type(),\n        Tag::any(), Neighbors::nodelist:neighborhood()) -> slide_op().\nnew_sending_slide_join(MoveId, JoiningNode, Tag, Neighbors) ->\n    JoiningNodeId = node:id(JoiningNode),\n    new_sending_slide_join(MoveId, JoiningNode, JoiningNodeId, Tag, Neighbors).\n\n%% @doc Sets up an incremental slide operation of the given type. One of the\n%%      nodes will change its ID to CurTargetId and finally FinalTargetId.\n-spec new_sending_slide_join_i(\n        MoveId::uid:global_uid(), JoiningNode::node:node_type(),\n        CurTargetId::?RT:key(), Tag::any(), Neighbors::nodelist:neighborhood())\n        -> slide_op().\nnew_sending_slide_join_i(MoveId, JoiningNode, CurTargetId, Tag, Neighbors) ->\n    FinalTargetId = node:id(JoiningNode),\n    NextOp = case FinalTargetId of\n                 CurTargetId -> {none};\n                 _           -> {slide, continue, FinalTargetId}\n             end,\n    Slide = new_sending_slide_join(MoveId, JoiningNode, CurTargetId, Tag, Neighbors),\n    Slide#slide_op{next_op = NextOp}.\n\n-spec new_sending_slide_join(\n        MoveId::uid:global_uid(), JoiningNode::node:node_type(),\n        TargetId::?RT:key(), Tag::any(), Neighbors::nodelist:neighborhood()) -> slide_op().\nnew_sending_slide_join(MoveId, JoiningNode, JoiningNodeId, Tag, Neighbors) ->\n    case intervals:in(JoiningNodeId, nodelist:node_range(Neighbors)) of\n        false -> erlang:throw(not_responsible);\n        _ ->\n            IntervalToSend = node:mk_interval_between_ids(\n                               node:id(nodelist:pred(Neighbors)), JoiningNodeId),\n            #slide_op{type = {join, 'send'},\n                      id = MoveId,\n                      node = JoiningNode,\n                      interval = IntervalToSend,\n                      my_old_id = nodelist:nodeid(Neighbors),\n                      target_id = JoiningNodeId,\n                      tag = Tag,\n                      source_pid = null}\n    end.\n\n%% @doc Sets up a new slide operation for a node which is about to leave its\n%%      position in the ring incrementally (current step is to move to\n%%      CurTargetId) and transfer its data to its successor.\n-spec new_sending_slide_leave(MoveId::id(), CurTargetId::?RT:key(), Tag::any(),\n        SourcePid::comm:erl_local_pid() | null,\n        Neighbors::nodelist:neighborhood()) -> slide_op().\nnew_sending_slide_leave(MoveId, CurTargetId, Tag, SourcePid, Neighbors) ->\n    {Interval, TargetNode} =\n        get_interval_tnode('succ', 'send', CurTargetId, Neighbors),\n    NextOp = case node:id(nodelist:pred(Neighbors)) of\n                 CurTargetId -> {none};\n                 _           -> {leave, continue}\n             end,\n    #slide_op{type = {leave, 'send'},\n              id = MoveId,\n              node = TargetNode,\n              interval = Interval,\n              my_old_id = nodelist:nodeid(Neighbors),\n              target_id = CurTargetId,\n              tag = Tag,\n              source_pid = SourcePid,\n              next_op = NextOp}.\n\n%% @doc Sets up a new slide operation for a node which is about to leave its\n%%      position in the ring, transfer its data to its successor\n%%      incrementally (current step is to move to CurTargetId) and afterwards\n%%      join somewhere else.\n-spec new_sending_slide_jump(MoveId::id(), CurTargetId::?RT:key(),\n        JumpTargetId::?RT:key(), SourcePid::comm:erl_local_pid(),\n                              Tag::any(), Neighbors::nodelist:neighborhood())\n        -> slide_op().\nnew_sending_slide_jump(MoveId, CurTargetId, JumpTargetId, SourcePid, Tag, Neighbors) ->\n    {Interval, TargetNode} =\n        get_interval_tnode('succ', 'send', CurTargetId, Neighbors),\n    NextOp =\n         case node:id(nodelist:pred(Neighbors)) of\n             CurTargetId -> {none};\n             _           -> {leave, continue}\n         end,\n    #slide_op{type = {jump, 'send'},\n              id = MoveId,\n              node = TargetNode,\n              interval = Interval,\n              my_old_id = nodelist:nodeid(Neighbors),\n              target_id = CurTargetId,\n              jump_target_id = JumpTargetId,\n              tag = Tag,\n              source_pid = SourcePid,\n              next_op = NextOp}.\n\n%% @doc Updates the slide op with a new TargetId and NextOp adapting message\n%%      forwards and intervals accordingly.\n-spec update_target_id(slide_op(), TargetId::?RT:key(), NextOp::next_op(),\n                       Neighbors::nodelist:neighborhood()) -> slide_op().\nupdate_target_id(SlideOp = #slide_op{target_id=TargetId}, TargetId, NextOp, _Neighbors) ->\n    SlideOp#slide_op{next_op = NextOp};\nupdate_target_id(SlideOp = #slide_op{type=Type, node=TargetNode, msg_fwd=OldMsgFwd},\n                 TargetId, NextOp, Neighbors) ->\n    PredOrSucc = get_predORsucc(Type),\n    SendOrReceive = get_sendORreceive(Type),\n    {Interval, TargetNode2} =\n        case is_join(Type, 'rcv') of\n            false ->\n                Pred = nodelist:pred(Neighbors),\n                case SendOrReceive =:= 'rcv' andalso\n                         node:id(Pred) =:= TargetId of\n                    false ->\n                        get_interval_tnode(PredOrSucc, SendOrReceive, TargetId, Neighbors);\n                    true ->\n                        % the leaving node may not be in Neighbors any more!\n                        % in this case, the TargetId is the same ID as the pred ID!\n                        % -> use the ID of the TargetNode instead (should be up-to-date!)\n                        {node:mk_interval_between_ids(TargetId, node:id(TargetNode)),\n                         TargetNode}\n                end;\n            true ->\n                % note: we always need to transfer everything from Pred to TargetId\n                {node:mk_interval_between_ids(node:id(nodelist:pred(Neighbors)), TargetId),\n                 nodelist:succ(Neighbors)}\n        end,\n    false = intervals:is_all(Interval),\n    % TargetNode2 may be more up to date - check that this is the same node though\n    true = (node:same_process(TargetNode, TargetNode2) orelse is_leave(SlideOp, 'rcv')),\n    SlideOp1 = SlideOp#slide_op{interval = Interval,\n                                target_id = TargetId,\n                                next_op = NextOp},\n    case OldMsgFwd of\n        []  -> SlideOp1;\n        [_] -> set_msg_fwd(SlideOp1)\n    end.\n\n%% @doc Returns the id of a receiving or sending slide operation.\n-spec get_id(SlideOp::slide_op()) -> id().\nget_id(#slide_op{id=Id}) -> Id.\n\n%% @doc Returns the node to exchange data with.\n-spec get_node(SlideOp::slide_op()) -> node:node_type().\nget_node(#slide_op{node=Node}) -> Node.\n\n%% @doc Returns the interval of data to receive or send.\n-spec get_interval(SlideOp::slide_op()) -> intervals:interval().\nget_interval(#slide_op{interval=Interval}) -> Interval.\n\n%% @doc Returns the old ID before any changes, i.e. at the start of the slide\n%%      (null in case of receiving joins).\n-spec get_my_old_id(SlideOp::slide_op()) -> ?RT:key() | null.\nget_my_old_id(#slide_op{my_old_id=MyOldId}) -> MyOldId.\n\n%% @doc Returns the target id a node participating in a receiving or sending\n%%      slide operation moves to (note: this may be the other node).\n-spec get_target_id(SlideOp::slide_op()) -> ?RT:key().\nget_target_id(#slide_op{target_id=TargetId}) -> TargetId.\n\n-spec get_jump_target_id(SlideOp::slide_op()) -> ?RT:key().\nget_jump_target_id(#slide_op{jump_target_id=TargetId}) -> TargetId.\n\n%% @doc Gets the pid of the (local) process that requested the move or null if\n%%      no local process initiated it.\n-spec get_source_pid(SlideOp::slide_op()) -> comm:erl_local_pid() | null.\nget_source_pid(#slide_op{source_pid=Pid}) -> Pid.\n\n%% @doc Returns the tag of a slide operation. This will be send to the\n%%      originating process (along with the result message).\n%% @see get_source_pid/1\n-spec get_tag(SlideOp::slide_op()) -> any().\nget_tag(#slide_op{tag=Tag}) -> Tag.\n\n%% @doc Returns whether the given slide operation sends or receives data.\n-spec get_sendORreceive(SlideOp::slide_op() | type()) -> 'send' | 'rcv'.\nget_sendORreceive(#slide_op{type=Type}) -> get_sendORreceive(Type);\nget_sendORreceive({slide, _, SendOrReceive}) -> SendOrReceive;\nget_sendORreceive({_TypeTag, SendOrReceive}) -> SendOrReceive.\n\n%% @doc Returns whether the given slide operation works with the successor or\n%%      predecessor.\n-spec get_predORsucc(SlideOp::slide_op() | type()) -> pred | succ.\nget_predORsucc(#slide_op{type=Type}) -> get_predORsucc(Type);\nget_predORsucc({slide, PredOrSucc, _}) -> PredOrSucc;\nget_predORsucc({join, 'send'}) -> pred;\nget_predORsucc({join, 'rcv'}) -> succ;\nget_predORsucc({leave, 'send'}) -> succ;\nget_predORsucc({leave, 'rcv'}) -> pred;\nget_predORsucc({jump, 'send'}) -> succ;\nget_predORsucc({jump, 'rcv'}) -> pred.\n\n%% @doc Returns the given slide operation's (full) type.\n-spec get_type(SlideOp::slide_op()) -> type().\nget_type(#slide_op{type=Type}) -> Type.\n\n%% @doc Converts the given slide type to the type the other participating node\n%%      can use.\n-spec other_type_to_my_type(type()) -> type().\nother_type_to_my_type({slide, pred, SendOrReceive}) ->\n    {slide, succ, switch_sendORreceive2(SendOrReceive)};\nother_type_to_my_type({slide, succ, SendOrReceive}) ->\n    {slide, pred, switch_sendORreceive2(SendOrReceive)};\nother_type_to_my_type({TypeTag, SendOrReceive}) ->\n    {TypeTag, switch_sendORreceive2(SendOrReceive)}.\n\n%% @doc Helper to change 'send' to 'rcv' and the other way around.\n-spec switch_sendORreceive2('send') -> 'rcv';\n                           ('rcv') -> 'send'.\nswitch_sendORreceive2('send') -> 'rcv';\nswitch_sendORreceive2('rcv') -> 'send'.\n\n%% @doc Returns whether the given term is a slide op or not.\n-spec is_slide(any()) -> boolean().\nis_slide(#slide_op{}) -> true;\nis_slide(_) -> false.\n\n%% @doc Returns whether the given slide op or type is a join operation.\n-spec is_join(SlideOp::slide_op() | type()) -> boolean().\nis_join(#slide_op{type=Type}) -> is_join(Type);\nis_join(Type) -> element(1, Type) =:= join.\n\n%% @doc Returns whether the given slide op or type is a join operation sending\n%%      or receiving data as provided.\n-spec is_join(SlideOp::slide_op() | type(), 'send' | 'rcv') -> boolean().\nis_join(#slide_op{type=Type}, SendOrReceive) -> is_join(Type, SendOrReceive);\nis_join({join, SendOrReceive}, SendOrReceive) -> true;\nis_join(_Type, _SendOrReceive) -> false.\n\n%% @doc Returns whether the given slide op or type is a leave operation.\n-spec is_leave(SlideOp::slide_op() | type()) -> boolean().\nis_leave(#slide_op{type=Type}) -> is_leave(Type);\nis_leave({leave, _}) -> true;\nis_leave({jump, _}) -> true;\nis_leave(_Type) -> false.\n\n%% @doc Returns whether the given slide op or type is a leave operation sending\n%%      or receiving data as provided.\n-spec is_leave(SlideOp::slide_op() | type(), 'send' | 'rcv') -> boolean().\nis_leave(#slide_op{type=Type}, SendOrReceive) -> is_leave(Type, SendOrReceive);\nis_leave({leave, SendOrReceive}, SendOrReceive) -> true;\nis_leave({jump, SendOrReceive}, SendOrReceive) -> true;\nis_leave(_Type, _SendOrReceive) -> false.\n\n%% @doc Returns whether the given slide op or type is a jump operation.\n-spec is_jump(SlideOp::slide_op() | type()) -> boolean().\nis_jump(#slide_op{type=Type}) -> is_jump(Type);\nis_jump({jump, _SendOrReceive}) -> true;\nis_jump(_Type) -> false.\n\n%% @doc Returns whether the given slide op or type is a jump operation sending\n%%      or receiving data as provided.\n-spec is_jump(SlideOp::slide_op() | type(), 'send' | 'rcv') -> boolean().\nis_jump(#slide_op{type=Type}, SendOrReceive) -> is_jump(Type, SendOrReceive);\nis_jump({jump, SendOrReceive}, SendOrReceive) -> true;\nis_jump(_Type, _SendOrReceive) -> false.\n\n%% @doc Returns whether the given slide op is part of an incremental slide.\n-spec is_incremental(SlideOp::slide_op()) -> boolean().\nis_incremental(#slide_op{next_op={slide, continue, _Id}}) -> true;\nis_incremental(#slide_op{next_op={jump, continue, _Id}}) -> true;\nis_incremental(#slide_op{next_op={leave, continue}}) -> true;\nis_incremental(_) -> false.\n\n%% @doc Returns the time of the last send operation.\n-spec get_time_last_send(SlideOp::slide_op()) -> erlang_timestamp() | never.\nget_time_last_send(#slide_op{time_last_send = X}) -> X.\n\n%% @doc Returns the time the next warning should be emitted if no further send\n%%      operation occurs.\n-spec get_time_next_warn(SlideOp::slide_op()) -> erlang_timestamp() | never.\nget_time_next_warn(#slide_op{time_next_warn = X}) -> X.\n\n%% @doc Sets the time of the last send operation.\n-spec set_time_last_send(SlideOp::slide_op(), erlang_timestamp()) -> slide_op().\nset_time_last_send(SlideOp, X) -> SlideOp#slide_op{time_last_send = X}.\n\n%% @doc Sets the time the next warning should be emitted if no further send\n%%      operation occurs.\n-spec set_time_next_warn(SlideOp::slide_op(), erlang_timestamp()) -> slide_op().\nset_time_next_warn(SlideOp, X) -> SlideOp#slide_op{time_next_warn = X}.\n\n%% @doc Returns the number of send errors from messages with shepherd.\n-spec get_send_errors(SlideOp::slide_op()) -> non_neg_integer().\nget_send_errors(#slide_op{send_errors = X}) -> X.\n\n%% @doc Increases the number of send_errors from messages with shepherd by 1.\n-spec inc_send_errors(SlideOp::slide_op()) -> slide_op().\ninc_send_errors(SlideOp = #slide_op{send_errors = X}) ->\n    SlideOp#slide_op{send_errors = X + 1}.\n\n%% @doc Resets the number of send_errors from messages with shepherd to 0.\n-spec reset_send_errors(SlideOp::slide_op()) -> slide_op().\nreset_send_errors(SlideOp) ->\n    SlideOp#slide_op{send_errors = 0}.\n\n%% @doc Returns the current phase of the slide operation.\n-spec get_phase(SlideOp::slide_op()) -> phase().\nget_phase(#slide_op{phase=Phase}) -> Phase.\n\n%% @doc Sets the slide operation's current phase.\n-spec set_phase(SlideOp::slide_op(), NewPhase::phase()) -> slide_op().\nset_phase(SlideOp, NewPhase) -> SlideOp#slide_op{phase = NewPhase}.\n\n%% @doc Returns wether the current slide op has already been set up at the\n%%      other node.\n-spec is_setup_at_other(SlideOp::slide_op()) -> boolean().\nis_setup_at_other(#slide_op{setup_at_other=SetupAtOther}) -> SetupAtOther.\n\n%% @doc Sets that the current slide op has already been set up at the\n%%      other node.\n-spec set_setup_at_other(SlideOp::slide_op()) -> slide_op().\nset_setup_at_other(SlideOp) -> SlideOp#slide_op{setup_at_other = true}.\n\n-spec get_next_op(SlideOp::slide_op()) -> next_op().\nget_next_op(#slide_op{next_op=Op}) -> Op.\n\n-spec set_next_op(SlideOp::slide_op(), NextOp::next_op()) -> slide_op().\nset_next_op(SlideOp, NextOp) -> SlideOp#slide_op{next_op = NextOp}.\n\n-spec get_other_max_entries(SlideOp::slide_op()) -> unknown | pos_integer().\nget_other_max_entries(#slide_op{other_max_entries=OtherMTE}) -> OtherMTE.\n\n-spec set_other_max_entries(SlideOp::slide_op(), OtherMTE::pos_integer()) -> slide_op().\nset_other_max_entries(SlideOp, OtherMTE) -> SlideOp#slide_op{other_max_entries = OtherMTE}.\n\n-spec get_msg_fwd(SlideOp::slide_op() | null) -> [{intervals:interval(), comm:mypid()}].\nget_msg_fwd(null) -> [];\nget_msg_fwd(#slide_op{msg_fwd=MsgFwd}) -> MsgFwd.\n\n-spec set_msg_fwd(SlideOp::slide_op()) -> slide_op().\nset_msg_fwd(SlideOp = #slide_op{interval=Interval}) ->\n    ?TRACE(\"[ ~.0p ] set_msg_fwd: ~.0p~n\", [self(), Interval]),\n    SlideOp#slide_op{msg_fwd = [{Interval, node:pidX(get_node(SlideOp))}]}.\n\n-spec remove_msg_fwd(SlideOp::slide_op()) -> slide_op().\nremove_msg_fwd(SlideOp) ->\n    ?TRACE(\"[ ~.0p ] remove_msg_fwd: ~.0p~n\", [self(), SlideOp#slide_op.msg_fwd]),\n    SlideOp#slide_op{msg_fwd = []}.\n"
  },
  {
    "path": "src/snapshot.erl",
    "content": "%  @copyright 2012-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Stefan Keidel <keidel@informatik.hu-berlin.de>\n%% @doc Generic snapshot-related functions (utils)\n%% @version $Id$\n-module(snapshot).\n-author('keidel@informatik.hu-berlin.de').\n-vsn('$Id$').\n-include(\"scalaris.hrl\").\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(_X,_Y), ok).\n\n-export([on_do_snapshot/3, on_local_snapshot_is_done/1]).\n\n-spec snapshot_is_done(dht_node_state:state()) -> boolean().\nsnapshot_is_done(DHTNodeState) ->\n    SnapState = dht_node_state:get(DHTNodeState, snapshot_state),\n    DB = dht_node_state:get(DHTNodeState, db),\n    case snapshot_state:is_in_progress(SnapState) andalso\n            db_dht:snapshot_is_running(DB) andalso\n            db_dht:snapshot_is_lockfree(DB) of\n        true ->\n            true;\n        _ ->\n            ?TRACE(\"~p snapshot:snapshot_is_done db: ~p~n\",[comm:this(),DB]),\n            ?TRACE(\"~p snapshot:snapshot_is_done db data: ~p~n\",[comm:this(),db_dht:get_data(DB)]),\n            ?TRACE(\"~p snapshot:snapshot_is_done snapshot data: ~p~n\",[comm:this(),db_dht:get_snapshot_data(DB)]),\n            false\n    end.\n\n-spec on_do_snapshot(non_neg_integer(),any(),dht_node_state:state()) -> dht_node_state:state().\non_do_snapshot(SnapNumber, Leader, DHTNodeState) ->\n    SnapState = dht_node_state:get(DHTNodeState, snapshot_state),\n    NewState = case snapshot_state:is_in_progress(SnapState) of\n        true -> % old snapshot is still running or current snapshot is already running\n            case snapshot_state:get_number(SnapState) < SnapNumber of\n                true -> % currently running snapshot is old\n                    ?TRACE(\"snapshot: on_do_snapshot: starting new ~p dumping\n                           old ~p~n\",[SnapNumber, snapshot_state:get_number(SnapState)]),\n                    msg_snapshot_leaders_err(\"New snapshot arrived\",\n                                             snapshot_state:get_number(SnapState),\n                                             dht_node_state:get(DHTNodeState,my_range),\n                                             snapshot_state:get_leaders(SnapState)),\n                    delete_and_init_snapshot(SnapNumber,Leader,DHTNodeState);\n                false -> % the current snapshot is the same as the incoming one or newer\n                    case snapshot_state:get_number(SnapState) =:= SnapNumber of\n                        true ->\n                            % additional msg for current snapshot -> add leader to dht node state for later messaging\n                            NewSnapState = snapshot_state:add_leader(SnapState, Leader),\n                            dht_node_state:set_snapshot_state(DHTNodeState, NewSnapState);\n                        false ->\n                            ?TRACE(\"snapshot: on_do_snapshot: ignoring old snapshot message ~p~n\", [SnapNumber]),\n                            % old snapshot -> ignore (or error msg?)\n                             DHTNodeState\n                    end\n            end;\n        false ->\n            % no snapshot is progress -> init new\n            ?TRACE(\"snapshot: on_do_snapshot: init new snapshot~n\",[]),\n            delete_and_init_snapshot(SnapNumber, Leader, DHTNodeState)\n    end,\n    % check if snapshot is already done (i.e. there were no active transactions when the snapshot arrived)\n    case snapshot_is_done(NewState) of\n        true ->\n            comm:send(comm:this(), {local_snapshot_is_done});\n        false ->\n            ?TRACE(\"~p snapshot:on_do_snapshot: snapshot is not done~n\",[comm:this()]),\n            ok\n    end,\n    % return\n    NewState.\n\n-spec on_local_snapshot_is_done(dht_node_state:state()) -> dht_node_state:state().\non_local_snapshot_is_done(DHTNodeState) ->\n    Db = dht_node_state:get(DHTNodeState,db),\n    SnapState = dht_node_state:get(DHTNodeState,snapshot_state),\n\n    % collect local state and send it\n    SnapNumber = snapshot_state:get_number(SnapState),\n    ?TRACE(\"snapshot: local snapshot ~p done~n\",[SnapNumber]),\n    Data = db_dht:join_snapshot_data(Db),\n    Leaders = snapshot_state:get_leaders(SnapState),\n    DBRange = dht_node_state:get(DHTNodeState,my_range),\n    msg_snapshot_leaders(Data,SnapNumber,DBRange,Leaders),\n\n    % cleanup\n    NewDB = db_dht:delete_snapshot(dht_node_state:get(DHTNodeState,db)),\n    NewSnapState = snapshot_state:stop_progress(SnapState),\n    NewState = dht_node_state:set_snapshot_state(DHTNodeState, NewSnapState),\n\n    dht_node_state:set_db(NewState, NewDB).\n\nmsg_snapshot_leaders(Data,SnapNumber,DBRange,[Leader | RestOfLeaders]) ->\n    comm:send(Leader, {local_snapshot_done,comm:this(),SnapNumber,DBRange,Data}),\n    msg_snapshot_leaders(Data,SnapNumber,DBRange,RestOfLeaders);\nmsg_snapshot_leaders(_Data,_SnapNumber,_DBRange,[]) ->\n    ok.\n\nmsg_snapshot_leaders_err(Msg,SnapNumber,DBRange,[Leader | RestOfLeaders]) ->\n    comm:send(Leader, {local_snapshot_failed,comm:this(),SnapNumber,DBRange,Msg}),\n    msg_snapshot_leaders_err(Msg,SnapNumber,DBRange,RestOfLeaders);\nmsg_snapshot_leaders_err(_Msg,_SnapNumber,_DBRange,[]) ->\n    ok.\n\n-spec delete_and_init_snapshot(non_neg_integer(),any(),dht_node_state:state()) -> dht_node_state:state().\ndelete_and_init_snapshot(SnapNumber,Leader,DHTNodeState) ->\n    NewDB = db_dht:init_snapshot(dht_node_state:get(DHTNodeState,db)),\n    TmpSnapState = snapshot_state:new(SnapNumber, true, [Leader]),\n    %% inform tx_tm_rtm on new SnapNumber\n    comm:send_local(pid_groups:get_my(tx_tm), {update_snapno, SnapNumber}),\n    TmpState = dht_node_state:set_db(DHTNodeState, NewDB),\n    dht_node_state:set_snapshot_state(TmpState, TmpSnapState).\n"
  },
  {
    "path": "src/snapshot_leader.erl",
    "content": "%  @copyright 2012-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Stefan Keidel <keidel@informatik.hu-berlin.de>\n%% @doc gen_component for (potential) leader of the snapshot algorithm\n%% @version $Id$\n-module(snapshot_leader).\n-author('keidel@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n%-define(TRACE(X, Y), io:format(X, Y)).\n-define(TRACE(_X, _Y), ok).\n-behaviour(gen_component).\n\n%% functions for gen_component module and supervisor callbacks\n-export([start_link/1, on/2, init/1]).\n\n-include(\"scalaris.hrl\").\n\n% accepted messages of the snapshot_leader process\n-type init_message() :: {init_snapshot, Client::comm:erl_local_pid()}.\n\n-type result_message() :: {local_snapshot_done, From::comm:erl_local_pid(),\n                           SnapNumber::non_neg_integer(),\n                           DBRange::intervals:interval(),\n                           Snapshot::db_dht:db_as_list()} |\n    {local_snapshot_failed, From::comm:erl_local_pid(),\n     SnapNumber::non_neg_integer(), Msg::string()}.\n\n-type message() :: init_message() | result_message().\n\n-export_type([result_message/0]).\n\n-include(\"gen_component.hrl\").\n\n%% be startable via supervisor, use gen_component\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2,\n                             [], % parameters passed to init\n                             [{pid_groups_join_as, DHTNodeGroup, snapshot_leader}]).\n\n%% initialize: return initial state.\n-spec init([]) -> snapshot_leader_state:state().\ninit([]) ->\n    snapshot_leader_state:new().\n\n-spec on(message(), snapshot_leader_state:state()) -> snapshot_leader_state:state().\n\non({init_snapshot, Client}, State) ->\n    NewSnapNum = snapshot_leader_state:get_number(State) + 1,\n    ?TRACE(\"snapshot_leader: init_snapshot with number ~p~n\", [NewSnapNum]),\n    % send init_snapshot to all dht_nodes\n    bulkowner:issue_bulk_owner(uid:get_global_uid(), intervals:all(), {do_snapshot, NewSnapNum, comm:this()}),\n    snapshot_leader_state:new(NewSnapNum, true, Client, State);\n\n% TODO: too much redundant code below -> break this up into several functions\n\non({local_snapshot_done, _From, SnapNumber, Range, Snapshot}, State) ->\n    case (snapshot_leader_state:is_in_progress(State)\n         andalso SnapNumber =:= snapshot_leader_state:get_number(State)) of\n        true ->\n            ?TRACE(\"snapshot_leader: local_snapshot_done ~p from ~p for range ~p~n\",\n                   [SnapNumber, From, Range]),\n            TmpState = snapshot_leader_state:add_interval(State, Range),\n            NewState = snapshot_leader_state:add_snapshot(TmpState, Snapshot),\n            case snapshot_leader_state:interval_union_is_all(NewState) of\n                true -> % snapshot done, message client and \"reset\" local state\n                    Data = snapshot_leader_state:get_global_snapshot(NewState),\n                    ?TRACE(\"snapshot_leader: snapshot ~p is done. sending\n                           data...~n\", [SnapNumber]),\n                    ErrorInterval = snapshot_leader_state:get_error_interval(NewState),\n                    case intervals:is_empty(ErrorInterval) of\n                        true ->\n                            comm:send(snapshot_leader_state:get_client(NewState),\n                                      {global_snapshot_done, Data});\n                        _ ->\n                            comm:send(snapshot_leader_state:get_client(NewState),\n                                      {global_snapshot_done_with_errors, ErrorInterval, Data})\n                    end,\n                    snapshot_leader_state:new(SnapNumber, false, false, NewState);\n                false ->\n                    NewState\n            end;\n        false -> % late/random snapshot_done message -> ignore\n            %% ?TRACE(\"snapshot_leader: local_snapshot_done ~p but stale ~n\",\n            %%        [SnapNumber]),\n            State\n    end;\n\non({local_snapshot_failed, _From, SnapNumber, Range, _Msg}, State) ->\n    case (snapshot_leader_state:is_in_progress(State)\n         andalso SnapNumber =:= snapshot_leader_state:get_number(State)) of\n        true ->\n            NewState = snapshot_leader_state:add_error_interval(State, Range),\n            case snapshot_leader_state:interval_union_is_all(NewState) of\n                true -> % snapshot done, message client and \"reset\" local state\n                    Data = snapshot_leader_state:get_global_snapshot(NewState),\n                    ?TRACE(\"snapshot_leader: Snapshot ~p failed sending data...~n\", [SnapNumber]),\n                    ErrorInterval = snapshot_leader_state:get_error_interval(NewState),\n                    case intervals:is_empty(ErrorInterval) of\n                        true ->\n                            comm:send(snapshot_leader_state:get_client(NewState),\n                                      {global_snapshot_done, Data});\n                        _ ->\n                            comm:send(snapshot_leader_state:get_client(NewState),\n                                      {global_snapshot_done_with_errors, ErrorInterval, Data})\n                    end,\n                    snapshot_leader_state:new(SnapNumber, false, false, NewState);\n                false ->\n                    NewState\n            end;\n        false -> % late/random snapshot_failed message\n            State\n    end;\n\non(_, _) ->\n    unknown_event.\n"
  },
  {
    "path": "src/snapshot_leader_state.erl",
    "content": "%  @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Stefan Keidel <keidel@informatik.hu-berlin.de>\n%% @doc Local state information needed for the leader of the S3 snapshot algorithm\n%% @version $Id$\n-module(snapshot_leader_state).\n-author('keidel@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-export([new/0, new/4,\n         get_number/1, is_in_progress/1, get_global_snapshot/1, get_client/1,\n         interval_union_is_all/1, get_error_interval/1,\n         set_number/2, add_snapshot/2, add_interval/2, add_error_interval/2,\n         start_progress/1, stop_progress/1]).\n\n-export_type([state/0]).\n\n%-define(TRACE(X, Y), io:format(X, Y)).\n-define(TRACE(_X, _Y), ok).\n\n-include(\"scalaris.hrl\").\n\n-type(state() :: {SnapNo::non_neg_integer(), InProgress::boolean(),\n                  Snapshots::ets:tid() | atom(), Interval::intervals:interval(),\n                  ErrorInterval::intervals:interval(), Client::comm:mypid() | false}).\n\n% constructors\n\n-spec new() -> state().\nnew() ->\n    new(0, false, false).\n\n%% need to clean up ets table from old state\n-spec new(non_neg_integer(), boolean(), comm:mypid() | false, state()) -> state().\nnew(Number, InProgress, Client, {_, _, OldTable, _, _, _}) -> \n    ets:delete(OldTable),\n    new(Number, InProgress, Client).\n\n-spec new(non_neg_integer(), boolean(), comm:mypid() | false) -> state().\nnew(Number, InProgress, Client) ->\n    {Number, InProgress, ets:new(snapshot_leader_db, [ordered_set, private]),\n     intervals:empty(), intervals:empty(), Client}.\n\n% getters\n\n-spec get_number(state()) -> non_neg_integer().\nget_number(State) -> element(1, State).\n\n-spec is_in_progress(state()) -> boolean().\nis_in_progress(State) -> element(2, State).\n\n-spec get_global_snapshot(state()) -> list().\nget_global_snapshot(State) -> ets:tab2list(element(3, State)).\n\n-spec get_client(state()) -> comm:mypid().\nget_client(State) -> element(6, State).\n\n-spec interval_union_is_all(state()) -> boolean().\ninterval_union_is_all({_, _, _, Interval, ErrorInterval, _} = _State) ->\n    intervals:is_all(intervals:union(Interval, ErrorInterval)).\n\n-spec get_error_interval(state()) -> intervals:interval().\nget_error_interval(State) -> element(5, State).\n\n% setters\n\n-spec set_number(state(), non_neg_integer()) -> state().\nset_number(SnapInfo, NewVal) -> \n    erlang:put(local_snap_number, NewVal),\n    setelement(1, SnapInfo, NewVal).\n\n-spec add_snapshot(state(), any()) -> state().\nadd_snapshot({Number, InProgress, SnapshotDB, Interval, ErrorInterval, Client}, NewSnapshot) ->\n    add_snapshot_entries_to_db(SnapshotDB, NewSnapshot),\n    {Number, InProgress, SnapshotDB, Interval, ErrorInterval, Client}.\n\n-spec add_interval(state(), intervals:interval()) -> state().\nadd_interval({_Number, _InProgress, _SnapshotDB, Interval, _ErrorInterval, _Client}, NewInterval) ->\n    {_Number, _InProgress, _SnapshotDB, intervals:union(Interval, NewInterval), _ErrorInterval, _Client}.\n\n-spec add_error_interval(state(), intervals:interval()) -> state().\nadd_error_interval({_Number, _InProgress, _SnapshotDB, _Interval, ErrorInterval, _Client}, NewInterval) ->\n    {_Number, _InProgress, _SnapshotDB, _Interval, intervals:union(ErrorInterval, NewInterval), _Client}.\n\n-spec start_progress(state()) -> state().\nstart_progress(SnapInfo) -> setelement(2, SnapInfo, true).\n\n-spec stop_progress(state()) -> state().\nstop_progress(SnapInfo) -> setelement(2, SnapInfo, false).\n\n% helpers\n\nadd_snapshot_entries_to_db(SnapshotDB, [Tuple | RestOfSnapshot]) ->\n    Key = get_unique_key(db_entry:get_key(Tuple)),\n    case ets:lookup(SnapshotDB, Key) of\n        [{_Key, _Val, EntryVersion}] -> \n            case db_entry:get_version(Tuple) > EntryVersion of\n                true ->\n                    InsertTuple = {Key, rdht_tx:decode_value(db_entry:get_value(Tuple)), db_entry:get_version(Tuple)},\n                    ets:insert(SnapshotDB, InsertTuple);\n                false -> % we already stored same verion or newer -> do nothing\n                    ok\n            end;\n        [] -> \n            InsertTuple = {Key, rdht_tx:decode_value(db_entry:get_value(Tuple)), db_entry:get_version(Tuple)},\n            ets:insert(SnapshotDB, InsertTuple)\n    end,\n    add_snapshot_entries_to_db(SnapshotDB, RestOfSnapshot);\nadd_snapshot_entries_to_db(_SnapshotDB, []) ->\n    ok.\n\nget_unique_key(Key) ->\n    Keys = ?RT:get_replica_keys(Key),\n    ?TRACE(\"unique key of ~p is ~p (query ~p)~n\", [Keys, lists:nth(1, lists:sort(Keys)), Key]),\n    lists:nth(1, lists:sort(Keys)).\n"
  },
  {
    "path": "src/snapshot_state.erl",
    "content": "%  @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Stefan Keidel <keidel@informatik.hu-berlin.de>\n%% @doc Local state information needed for the S3 snapshot algorithm\n%% @version $Id$\n-module(snapshot_state).\n-author('keidel@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-export([new/0, new/3, get_number/1, is_in_progress/1, get_leaders/1,\n         set_number/2, add_leader/2, start_progress/1, stop_progress/1]).\n\n-export_type([snapshot_state/0]).\n\n-type(snapshot_state() :: {SnapNo::non_neg_integer(), InProgress::boolean(), Leaders::[comm:mypid()]}).\n\n% constructors\n\n-spec new() -> snapshot_state().\nnew() ->\n    erlang:put(local_snap_number, 0),\n    {0, false, []}.\n\n-spec new(non_neg_integer(), boolean(), [comm:mypid() | none]) -> snapshot_state().\nnew(Number, InProgress, Leaders) ->\n    erlang:put(local_snap_number, Number),\n    {Number, InProgress, [X || X <- Leaders, X =/= none]}.\n\n% getters\n\n-spec get_number(snapshot_state()) -> non_neg_integer().\nget_number({Number, _, _}) -> Number.\n\n-spec is_in_progress(snapshot_state()) -> boolean().\nis_in_progress({_, InProgress, _}) -> InProgress.\n\n-spec get_leaders(snapshot_state()) -> [comm:mypid()].\nget_leaders({_, _, Leaders}) -> Leaders.\n\n% setters\n\n-spec set_number(snapshot_state(), non_neg_integer()) -> snapshot_state().\nset_number(SnapInfo, NewVal) ->\n    erlang:put(local_snap_number, NewVal),\n    setelement(1, SnapInfo, NewVal).\n\n-spec add_leader(snapshot_state(), comm:mypid() | none) -> snapshot_state().\nadd_leader(State, none) -> State;\nadd_leader({Number, InProgress, Leaders}, NewLeader) ->\n    {Number, InProgress, [NewLeader | Leaders]}.\n\n-spec start_progress(snapshot_state()) -> snapshot_state().\nstart_progress(SnapInfo) -> setelement(2, SnapInfo, true).\n\n-spec stop_progress(snapshot_state()) -> snapshot_state().\nstop_progress(SnapInfo) -> setelement(2, SnapInfo, false).\n"
  },
  {
    "path": "src/statistics.erl",
    "content": "% @copyright 2007-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Statistics Module for mgmt server\n%% @version $Id$\n-module(statistics).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([get_ring_details/0, get_ring_details_neighbors/1,\n         get_total_load/2, get_average_load/2, get_load_std_deviation/2,\n         get_average_rt_size/1, get_rt_size_std_deviation/1,\n         get_memory_usage/1, get_max_memory_usage/1,\n         getTimingMonitorStats/3, getGaugeMonitorStats/4]).\n\n-include(\"scalaris.hrl\").\n\n-export_type([ring/0, ring_element/0]).\n\n-type ring_element() :: {ok, Details::node_details:node_details()} | {failed, comm:mypid()}.\n-type ring() :: [ring_element()].\n-type load() :: load | load2 | load3.\n-type load_type() :: node_details:load() | node_details:load2() | node_details:load3().\n\n-spec get_total_load(Which::load(), Ring::ring()) -> load_type().\nget_total_load(Which, Ring) ->\n    lists:foldl(fun (X, Sum) -> X + Sum end, 0,\n                lists:map(fun(Node) -> get_load(Which, Node) end, Ring)).\n\n-spec get_average_load(Which::load(), Ring::ring()) -> float().\nget_average_load(Which, Ring) ->\n    FilteredRing = lists:filter(fun (X) -> is_valid(X) end, Ring),\n    case length(FilteredRing) of\n        0 -> 0.0;\n        _ ->\n            get_total_load(Which, FilteredRing) / length(FilteredRing)\n    end.\n\n-spec get_memory_usage(Ring::ring()) -> float().\nget_memory_usage(Ring) ->\n    FilteredRing = lists:filter(fun (X) -> is_valid(X) end, Ring),\n    case length(FilteredRing) of\n        0 -> 0.0;\n        _ ->\n            lists:foldl(fun (X, Sum) -> X + Sum end, 0,\n                        lists:map(fun get_memory/1, FilteredRing))\n                / length(FilteredRing)\n    end.\n\n-spec get_max_memory_usage(Ring::ring()) -> node_details:memory().\nget_max_memory_usage(Ring) ->\n    lists:foldl(fun (X, Sum) -> erlang:max(X, Sum) end, 0,\n                lists:map(fun get_memory/1, Ring)).\n\n-spec get_load_std_deviation(Which::load(), Ring::ring()) -> float().\nget_load_std_deviation(Which, Ring) ->\n    FilteredRing = lists:filter(fun (X) -> is_valid(X) end, Ring),\n    case length(FilteredRing) of\n        0 -> 0.0;\n        _ ->\n            Average = get_average_load(Which, FilteredRing),\n            math:sqrt(lists:foldl(fun (Load, Acc) ->\n                                          Acc + (Load - Average)\n                                              * (Load - Average)\n                                  end, 0,\n                                  lists:map(fun(Node) ->\n                                                    get_load(Which, Node)\n                                            end,\n                                            FilteredRing))\n                      / length(FilteredRing))\n    end.\n\n-spec get_load(Which::load(), ring_element()) -> node_details:load().\nget_load(Which, {ok, Details}) ->\n    case node_details:get(Details, Which) of\n        unknown -> 0;\n        Val -> Val\n    end;\nget_load(_Which, {failed, _}) ->\n    0.\n\n-spec get_memory(ring_element()) -> node_details:memory().\nget_memory({ok, Details}) ->\n    node_details:get(Details, memory);\nget_memory({failed, _}) ->\n    0.\n\n%% @doc Returns a sorted list of all known nodes.\n%%      See compare_node_details/2 for a definition of the order.\n%%      Note: throws 'mgmt_server_timeout' if the mgmt server does not respond\n%%      within 2s.\n-spec get_ring_details() -> ring().\nget_ring_details() ->\n    mgmt_server:node_list(true),\n    Nodes = begin\n                trace_mpath:thread_yield(),\n                receive\n                ?SCALARIS_RECV({get_list_response, N}, N);\n                ?SCALARIS_RECV({send_error, _, {get_list, _}, Reason},\n                               begin\n                                 log:log(error,\"[ ST ] Mgmt server unavailable: ~p\", [Reason]),\n                                 throw('mgmt_server_timeout')\n                               end)\n                after 2000 ->\n                        log:log(error,\"[ ST ] Timeout getting node list from mgmt server\"),\n                        throw('mgmt_server_timeout')\n                end\n            end,\n    lists:sort(fun compare_node_details/2, get_ring_details(Nodes)).\n\n%% @doc Returns a sorted list of all known nodes in the neighborhoods of the\n%%      dht_node processes in this VM, recurses to their neighboring nodes if\n%%      requested.\n%%      See compare_node_details/2 for a definition of the order.\n-spec get_ring_details_neighbors(RecursionLvl::non_neg_integer()) -> ring().\nget_ring_details_neighbors(RecursionLvl) ->\n    Nodes = [comm:make_global(Pid) || Pid <- pid_groups:find_all(dht_node)],\n    get_ring_details_neighbors(RecursionLvl, [], Nodes).\n\n-spec get_ring_details_neighbors(RecursionLvl::non_neg_integer(), Ring::ring(), Nodes::[comm:mypid()]) -> ring().\nget_ring_details_neighbors(RecursionLvl, Ring, Nodes) ->\n    % first get the nodes with no details yet:\n    RingNodes = [begin\n                     case RingE of\n                         {ok, Details} ->\n                             node:pidX(node_details:get(Details, node));\n                         {failed, Pid} ->\n                             Pid\n                     end\n                 end || RingE <- Ring],\n    {_OnlyRing, _Both, NewNodes} = util:split_unique(RingNodes, Nodes),\n    % then get their node details:\n    NewRing = lists:sort(fun compare_node_details/2,\n                         lists:append(Ring, get_ring_details(NewNodes))),\n    case RecursionLvl < 1 of\n        true -> NewRing;\n        _ -> % gather nodes for the next recursion:\n            NextNodes =\n                lists:flatmap(\n                  fun(RingE) ->\n                          case RingE of\n                              {ok, Details} ->\n                                  PredList = node_details:get(Details, predlist),\n                                  SuccList = node_details:get(Details, succlist),\n                                  [node:pidX(Node) || Node <- PredList] ++\n                                      [node:pidX(Node) || Node <- SuccList];\n                              {failed, _Pid} ->\n                                  []\n                          end\n                  end, NewRing),\n            get_ring_details_neighbors(RecursionLvl - 1, NewRing, NextNodes)\n    end.\n\n%% @doc Returns a sorted list of details about the given nodes.\n%%      See compare_node_details/2 for a definition of the order.\n-spec get_ring_details(Nodes::[comm:mypid()]) -> ring().\nget_ring_details(Nodes) ->\n    UUID = uid:get_global_uid(),\n    _ = [begin\n             SourcePid = comm:reply_as(comm:this(), 2, {ok, '_', Pid, UUID}),\n             comm:send(Pid, {get_node_details, SourcePid})\n         end || Pid <- Nodes],\n    get_node_details(Nodes, UUID, [], 0).\n\n%% @doc Defines an order of ring_element() terms so that {failed, Pid} terms\n%%      are considered the smallest but sorted by their pids.\n%%      Terms like {ok, node_details:node_details()} are compared using the\n%%      order of their node ids.\n-spec compare_node_details(ring_element(), ring_element()) -> boolean().\ncompare_node_details({ok, X}, {ok, Y}) ->\n    node:id(node_details:get(X, node)) < node:id(node_details:get(Y, node));\ncompare_node_details({failed, X}, {failed, Y}) ->\n    X =< Y;\ncompare_node_details({failed, _}, {ok, _}) ->\n    true;\ncompare_node_details({ok, _}, {failed, _}) ->\n    false.\n\n-spec get_node_details(Pids::[comm:mypid()], uid:global_uid(), ring(),\n                       TimeInMS::non_neg_integer()) -> ring().\nget_node_details([_|_] = Pids, UUID, Ring, TimeInMS) ->\n    Continue =\n        if\n            TimeInMS =:= 2000 ->\n                log:log(error,\"[ ST ]: 2sec Timeout waiting for get_node_details_response from ~p\",[Pids]),\n                continue;\n            TimeInMS >= 6000 ->\n                log:log(error,\"[ ST ]: 6sec Timeout waiting for get_node_details_response from ~p\",[Pids]),\n                stop;\n            true -> continue\n    end,\n    case Continue of\n        continue ->\n            trace_mpath:thread_yield(),\n            receive\n                ?SCALARIS_RECV(\n                    {ok, {get_node_details_response, Details}, Pid, UUID}, %% ->\n                    get_node_details(lists:delete(Pid, Pids), UUID,\n                                     [{ok, Details} | Ring],\n                                     TimeInMS)\n                  );\n                % clean up old / unrelated messages\n                ?SCALARIS_RECV(\n                    {ok, {get_node_details_response, _Details}, _Pid, _OldUUID}, %% ->\n                    get_node_details(Pids, UUID, Ring, TimeInMS)\n                  )\n            after\n                10 ->\n                    get_node_details(Pids, UUID, Ring, TimeInMS + 10)\n            end;\n        _ -> Failed = [{failed, Pid} || Pid <- Pids],\n             lists:append(Failed, Ring)\n    end;\nget_node_details([], _UUID, Ring, _TimeInMS) -> Ring.\n\n%%%-------------------------------RT----------------------------------\n\n-spec get_total_rt_size(Ring::ring()) -> node_details:rt_size().\nget_total_rt_size(Ring) ->\n    lists:foldl(fun (X, Sum) -> X + Sum end, 0, lists:map(fun get_rt/1, Ring)).\n\n-spec get_average_rt_size(Ring::ring()) -> float().\nget_average_rt_size(Ring) ->\n    FilteredRing = lists:filter(fun (X) -> is_valid(X) end, Ring),\n    case length(FilteredRing) of\n        0 -> 0.0;\n        _ ->\n            get_total_rt_size(FilteredRing) / length(FilteredRing)\n    end.\n\n-spec get_rt_size_std_deviation(Ring::ring()) -> float().\nget_rt_size_std_deviation(Ring) ->\n    FilteredRing = lists:filter(fun (X) -> is_valid(X) end, Ring),\n    case length(FilteredRing) of\n        0 -> 0.0;\n        _ ->\n            Average = get_average_rt_size(FilteredRing),\n            math:sqrt(lists:foldl(fun (RTSize, Acc) ->\n                                          Acc + (RTSize - Average)\n                                              * (RTSize - Average)\n                                  end, 0,\n                                  lists:map(fun get_rt/1, FilteredRing))\n                      / length(FilteredRing))\n    end.\n\n-spec get_rt(ring_element()) -> node_details:rt_size().\nget_rt({ok, Details}) ->\n    node_details:get(Details, rt_size);\nget_rt({failed, _}) ->\n    0.\n\n-spec is_valid({ok, Details::node_details:node_details()}) -> true;\n              ({failed, _}) -> false.\nis_valid({ok, _}) ->\n    true;\nis_valid({failed, _}) ->\n    false.\n\n%%%-----------------------------Monitor-------------------------------\n-type time_list(Value) :: [[Time1_Value2::non_neg_integer() | Value]].\n-type tuple_list(Value) :: [{Time1_Value2::non_neg_integer(), Value}].\n\n-spec getMonitorData(Monitor::pid(), [{Process::atom(), Key::monitor:key()}]) -> [{Process::atom(), Key::monitor:key(), rrd:rrd()}].\ngetMonitorData(Monitor, Keys) ->\n    [Data || Data = {_Process, _Key, Value} <- monitor:get_rrds(Monitor, Keys),\n             Value =/= undefined].\n\n-spec monitor_timing_dump_fun_exists(rrd:rrd(), From_us::rrd:internal_time(), To_us::rrd:internal_time(), Value::term())\n        -> {TimestampMs::integer(), Count::non_neg_integer(), CountPerS::float(),\n            Avg::float(), Min::float(), Max::float(), Stddev::float(),\n            Hist::tuple_list(pos_integer())}.\nmonitor_timing_dump_fun_exists(_DB, From_us, To_us, {Sum, Sum2, Count, Min, Max, Hist}) ->\n    Diff_in_s = (To_us - From_us) div 1000000,\n    CountPerS = Count / Diff_in_s,\n    Avg = Sum / Count, Avg2 = Sum2 / Count,\n    Stddev = math:sqrt(Avg2 - (Avg * Avg)),\n    HistTpl = [X || X <- histogram:get_data(Hist)],\n    {round(To_us / 1000), Count, CountPerS, Avg, Min, Max, Stddev, HistTpl}.\n\n-spec monitor_timing_dump_fun_notexists(rrd:rrd(), From_us::rrd:internal_time(), To_us::rrd:internal_time())\n        -> {keep, {TimestampMs::integer(), Count::0, CountPerS::float(),\n                   Avg::float(), Min::float(), Max::float(), Stddev::float(),\n                   Hist::[{Time::float(), Count::pos_integer()}]}}.\nmonitor_timing_dump_fun_notexists(_DB, _From_us, To_us) ->\n    {keep, {round(To_us / 1000), 0, 0.0, 0.0, 0.0, 0.0, 0.0, []}}.\n\n%% @doc Gets the difference in seconds from UTC time to local time.\n-spec get_utc_local_diff_s() -> integer().\nget_utc_local_diff_s() ->\n    SampleTime = os:timestamp(),\n    UTC_s = calendar:datetime_to_gregorian_seconds(\n              calendar:now_to_universal_time(SampleTime)),\n    Local_s = calendar:datetime_to_gregorian_seconds(\n                calendar:now_to_local_time(SampleTime)),\n    Local_s - UTC_s.\n\n%% @doc Gets monitor stats from 'timing' and 'timing_with_hist' values in an\n%%      easy format for the web interface.\n-spec getTimingMonitorStats\n        (Monitor::pid(), [{Process::atom(), Key::monitor:key()}], list)\n        -> [{Process::atom(), Key::monitor:key(),\n             {CountD::time_list(non_neg_integer()),\n              CountPerSD::time_list(float()), AvgD::time_list(float()),\n              MinD::time_list(float()), MaxD::time_list(float()),\n              StddevD::time_list(float()),\n              HistD::time_list(time_list(pos_integer()))}}];\n        (Monitor::pid(), [{Process::atom(), Key::monitor:key()}], tuple)\n        -> [{Process::atom(), Key::monitor:key(),\n             {CountD::tuple_list(non_neg_integer()),\n              CountPerSD::tuple_list(float()), AvgD::tuple_list(float()),\n              MinD::tuple_list(float()), MaxD::tuple_list(float()),\n              StddevD::tuple_list(float()),\n              HistD::tuple_list(tuple_list(pos_integer()))}}].\ngetTimingMonitorStats(Monitor, Keys, Type) ->\n    UtcToLocalDiff_ms = get_utc_local_diff_s() * 1000,\n    [begin\n         Dump = rrd:dump_with(DB, fun monitor_timing_dump_fun_exists/4,\n                              fun monitor_timing_dump_fun_notexists/3),\n         Value =\n             lists:foldr(\n               fun({TimeUTC, Count, CountPerS, Avg, Min, Max, Stddev, Hist},\n                   {CountD, CountPerSD, AvgD, MinD, MaxD, StddevD, HistD}) ->\n                       Time = TimeUTC + UtcToLocalDiff_ms,\n                       case Type of\n                           list ->\n                               CountAcc = [Time, Count],\n                               CountPerSAcc = [Time, CountPerS],\n                               AvgAcc = [Time, Avg],\n                               MinAcc = [Time, Min],\n                               MaxAcc = [Time, Max],\n                               StddevAcc = [Time, Stddev],\n                               HistL = [erlang:tuple_to_list(X) || X <- Hist],\n                               HistAcc = [Time, HistL];\n                           tuple ->\n                               CountAcc = {Time, Count},\n                               CountPerSAcc = {Time, CountPerS},\n                               AvgAcc = {Time, Avg},\n                               MinAcc = {Time, Min},\n                               MaxAcc = {Time, Max},\n                               StddevAcc = {Time, Stddev},\n                               HistAcc = {Time, Hist}\n                       end,\n                       {[CountAcc | CountD], [CountPerSAcc | CountPerSD],\n                        [AvgAcc | AvgD], [MinAcc | MinD], [MaxAcc | MaxD],\n                        [StddevAcc | StddevD], [HistAcc | HistD]}\n               end, {[], [], [], [], [], [], []}, Dump),\n         {Process, Key, Value}\n     end || {Process, Key, DB} <- getMonitorData(Monitor, Keys)].\n\n-spec monitor_gauge_dump_fun_exists(rrd:rrd(), From_us::rrd:internal_time(), To_us::rrd:internal_time(), Value)\n        -> {TimestampMs::integer(), Value}.\nmonitor_gauge_dump_fun_exists(_DB, _From_us, To_us, Value) ->\n    {round(To_us / 1000), Value}.\n\n-spec monitor_gauge_dump_fun_notexists(rrd:rrd(), From_us::rrd:internal_time(), To_us::rrd:internal_time())\n        -> {keep, {TimestampMs::integer(), Value::0}}.\nmonitor_gauge_dump_fun_notexists(_DB, _From_us, To_us) ->\n    {keep, {round(To_us / 1000), 0}}.\n\n%% @doc Gets monitor stats from 'gauge' values in an easy format for the web\n%%      interface. Scales the original values by dividing them by Div.\n-spec getGaugeMonitorStats\n        (Monitor::pid(), [{Process::atom(), Key::monitor:key()}], list, Div::number())\n        -> [{Process::atom(), Key::monitor:key(),\n             ValueD::time_list(non_neg_integer())}];\n        (Monitor::pid(), [{Process::atom(), Key::monitor:key()}], tuple, Div::number())\n        -> [{Process::atom(), Key::monitor:key(),\n             ValueD::tuple_list(non_neg_integer())}].\ngetGaugeMonitorStats(Monitor, Keys, Type, Div) ->\n    UtcToLocalDiff_ms = get_utc_local_diff_s() * 1000,\n    [begin\n         Dump = rrd:dump_with(DB, fun monitor_gauge_dump_fun_exists/4,\n                              fun monitor_gauge_dump_fun_notexists/3),\n         Value =\n             lists:foldr(\n               fun({TimeUTC, ValueX}, ValueD) ->\n                       Time = TimeUTC + UtcToLocalDiff_ms,\n                       case Type of\n                           list ->\n                               ValueAcc = [Time, ValueX / Div];\n                           tuple ->\n                               ValueAcc = {Time, ValueX / Div}\n                       end,\n                       [ValueAcc | ValueD]\n               end, [], Dump),\n         {Process, Key, Value}\n     end || {Process, Key, DB} <- getMonitorData(Monitor, Keys)].\n"
  },
  {
    "path": "src/sup.erl",
    "content": "%  @copyright 2007-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Supervisor helper functions for better control of startup phase.\n%%\n%%      Start the supervisor separately and then add its childs one\n%%      after the other.\n%% @end\n%% @version $Id$\n-module(sup).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-include(\"types.hrl\").\n\n-export([sup_start/3,\n         sup_get_all_children/1,\n         sup_terminate/1,\n         sup_terminate_childs/1]).\n-export([worker_desc/3,\n         worker_desc/4,\n         supervisor_desc/3,\n         supervisor_desc/4]).\n\n%% for admin:add_node and unittests\n-export([start_sup_as_child/3]).\n\n%% Typical call structure is:\n%% sup_start(SupName, Module, Options)\n%% Module:supspec(Options)\n%% Erlang supervisor:start_link(SupName, Module, Options) is called\n%% Erlamg supervisor calls Module:init(Options) in spawned process\n%%   which return the supervisor spec\n%%   and in normal Erlang its childs\n%%   when childs are given, supervisor:start_link also start the childs\n%%   with sup it is intended to add the childs later...\n%% Module:childs(Options) is called to get the childs\n%% for each child, the given spec is used to start it.\n\n%% when a child is a supervisor: how is the group info passed from\n%% Module:init to the childs? Only via the Options, so the group cannot be created in Module:init anymore...\n\n-type prefix() :: nonempty_maybe_improper_list(string(), string()).\n\n%% taken from supervisor.erl (not exported there)\n-type startlink_err() :: {'already_started', pid()} | 'shutdown' | term().\n-type startlink_ret() :: {'ok', pid()} | {'error', startlink_err()}.\n-type sup_name() :: {'local', Name :: atom()} | {'global', Name :: atom()}.\n-type sup_ref()  :: (Name :: atom())\n                  | {Name :: atom(), Node :: node()}\n                  | {'global', Name :: atom()}\n                  | pid().\n\n-spec sup_start(sup_name(), module(), term()) -> startlink_ret().\nsup_start(Supervisor, Module, Options) ->\n    InitialPrefix = \"+\",\n    Ret = sup_start([InitialPrefix], Supervisor, Module, Options),\n    progress(\"~n\"),\n    Ret.\n\n-spec sup_start(prefix(),\n                sup_name() | no_name, module(), term())\n               -> startlink_ret().\nsup_start(Prefix, Supervisor, Module, Options) ->\n    progress(Prefix ++ \"~.0p ~.0p.erl~n\", [Supervisor, Module]),\n    SupSpec = trycall(Prefix, Module, supspec, Options, unknown_supspec),\n    Res = case Supervisor of\n              no_name -> supervisor:start_link(Module, Options);\n              _ -> supervisor:start_link(Supervisor, Module, Options)\n          end,\n    progress(Prefix ++ \"~.0p done~n\", [Supervisor]),\n    case Res of\n        {ok, SupRef} ->\n            progress(Prefix ++ \"`-~.0p ~.0p\", [SupSpec, SupRef]),\n            Childs = trycall(Prefix, Module, childs, Options, []),\n            ChildPrefix = last_prefix_to_space(Prefix),\n            ChildsRes = add_childs(ChildPrefix ++ [\"  +-\"], SupRef, Childs),\n            if is_tuple(ChildsRes) andalso element(1, ChildsRes) =:= ok ->\n                   Res; %% return pid of supervisor as it may be linked to externally;\n               true ->\n                   io:format(\"Startup raised ~.0p.~n\", [ChildsRes]),\n                   sup_terminate(SupRef),\n                   ChildsRes\n            end;\n        Error ->\n            progress(Prefix ++ \"~.0p\", [Error]),\n            Error\n    end.\n\n-spec start_sup_as_child(prefix(),\n                         sup_ref() | no_name,\n                         supervisor:child_spec())\n               -> supervisor:startchild_ret().\nstart_sup_as_child(Prefix, AtSup, SupAsChild) ->\n    progress(Prefix ++ \"supervisor ~.0p ~.0p\",\n             [element(1, SupAsChild), element(2, SupAsChild)]),\n    Module = element(1, element(2, SupAsChild)),\n    Args = element(3, element(2, SupAsChild)),\n    PipePrefix = last_prefix_to_pipe(Prefix),\n    SupSpec = trycall(PipePrefix,\n                      Module, supspec, Args, unknown_supspec),\n    case SupSpec of\n        unknown_supspec ->\n            progress(PipePrefix ++ \"childs are: ~.0p\", [element(6, SupAsChild)]);\n        _ -> ok\n    end,\n    Res = supervisor:start_child(AtSup, SupAsChild),\n%%    io:format(\"Res is ~.0p\", [Res]),\n    case Res of\n        %% {ok, SupRef} | {ok, SupRef, _GroupInfo}:\n        X when is_tuple(X) andalso element(1, X) =:= ok ->\n            SupRef = element(2, X),\n            case SupSpec of\n                unknown_supspec ->\n                    progress(PipePrefix ++ \"started childs at ~.0p:~n\",\n                             [SupRef]),\n                    show_started_childs(PipePrefix ++ [\"+-\"], SupRef),\n                    Res;\n                _ ->\n                    progress(PipePrefix\n                             ++ \"~.0p ~.0p\", [SupSpec, SupRef]),\n                    Childs = trycall(PipePrefix, Module, childs, Args, []),\n                    ChildsRes = add_childs(PipePrefix ++ [\"+-\"], SupRef, Childs),\n                    if is_tuple(ChildsRes) andalso element(1, ChildsRes) =:= ok ->\n                           Res; %% return pid of supervisor as it may be linked to externally;\n                       true ->\n                           io:format(\"Startup raised ~.0p.~n\", [ChildsRes]),\n                           SupName = element(1, SupAsChild),\n                           sup_terminate_childs(SupRef),\n                           _ = supervisor:terminate_child(AtSup, SupName),\n                           _ = supervisor:delete_child(AtSup, SupName),\n                           ChildsRes\n                    end\n               end;\n        Error ->\n            progress(Prefix ++ \" ~.0p~n\", [Error]),\n            Error\n    end.\n\ntrycall(Prefix, Module, Func, Args, DefaultReturnValue) ->\n    try Module:Func(Args)\n    catch error:undef ->\n            FlatPrefix = lists:flatten(Prefix),\n            progress(FlatPrefix ++ \"~.0p provides no ~.0p/1 function. Fall back to normal supervisor startup here.\", [Module, Func]),\n            DefaultReturnValue;\n          X:Y ->\n            io:format(\"~.0p:~.0p failed with ~.0p:~.0p ~.0p~n\",\n                      [Module, Func, X, Y, util:get_stacktrace()]),\n            DefaultReturnValue\n    end.\n\nadd_childs(_Prefix, SupRef, []) -> {ok, SupRef};\nadd_childs(Prefix, SupRef, [Child]) ->\n    LastChildPrefix = last_prefix_to_backslash(Prefix),\n    add_child(LastChildPrefix, SupRef, Child);\n\nadd_childs(Prefix, SupRef, [Child | Tail]) ->\n    Res = add_child(Prefix, SupRef, Child),\n    case Res of\n        {ok, _Pid} ->\n            add_childs(Prefix, SupRef, Tail);\n        {ok, _Pid, _GroupInfo} ->\n            add_childs(Prefix, SupRef, Tail);\n        Error ->\n            Error\n    end.\n\nadd_child(Prefix, SupRef, Child) ->\n    case Child of\n        {Name, MFA, _Kind, _Kill, worker, _Opts} ->\n            progress(Prefix ++ \"~.0p ~.0p \", [Name, MFA]),\n            Res = supervisor:start_child(SupRef, Child),\n            _ = case Res of\n                {ok, undefined} ->\n                    progress(\"not started\");\n                {ok, Pid} -> progress(\"~.0p.\", [Pid]);\n                {ok, Pid, GroupInfo} ->\n                        progress(\"~.0p ~.0p.\", [Pid, GroupInfo]);\n                Error ->\n                    progress(\"~nFailed to start ~.0p reason ~.0p~n\", [Child, Error]),\n                    progress(\"Supervisor ~.0p has childs: ~.0p~n\",\n                              [SupRef, sup_which_children(SupRef)]),\n                    Error\n            end,\n            Res;\n        {_Name, _MFA, _Kind, _Kill, supervisor, _Opts} ->\n            %% output is done by sup_start\n            start_sup_as_child(Prefix, SupRef, Child)\n    end.\n\nshow_started_childs(Prefix, SupRef) ->\n    Childs = sup_which_children(SupRef),\n    show_childs(Prefix, Childs).\n\nshow_childs(_Prefix, []) -> ok;\nshow_childs(Prefix, [LastChild]) ->\n    show_child(last_prefix_to_backslash(Prefix), LastChild);\nshow_childs(Prefix, [X | Tail]) ->\n    show_child(Prefix, X),\n    show_childs(Prefix, Tail).\n\nshow_child(Prefix, Child) ->\n    case element(3, Child) of\n        worker ->\n            progress(Prefix ++ \"~.0p~n\", [Child]);\n        supervisor ->\n            progress(Prefix ++ \"~.0p~n\", [Child]),\n            show_started_childs(last_prefix_to_pipe(Prefix) ++ [\"+-\"], element(2, Child))\n    end.\n\n\nlast_prefix_to_space([LastElem]) ->\n    [[case X of\n         $+ -> $\n     end || X <- LastElem]];\nlast_prefix_to_space([X | T]) ->\n    [X | last_prefix_to_space(T)].\n\nlast_prefix_to_pipe([LastElem]) ->\n    [[case X of\n         $+ -> $|;\n         $- -> $ ;\n         $` -> $ ;\n         _ -> X\n     end || X <- LastElem]];\nlast_prefix_to_pipe([X | T]) ->\n    [X | last_prefix_to_pipe(T)].\n\nlast_prefix_to_backslash([LastElem]) ->\n    [[case X of\n         $+ -> $`;\n         _ -> X\n     end || X <- LastElem]];\nlast_prefix_to_backslash([X | T]) ->\n    [X | last_prefix_to_backslash(T)].\n\n-spec progress(prefix()) -> ok.\nprogress(Fmt) ->\n    util:if_verbose(lists:flatten(Fmt)),\n    ok.\n-spec progress(prefix(), list()) -> ok.\nprogress(Fmt, Args) ->\n    util:if_verbose(lists:flatten(Fmt), Args),\n    ok.\n\n%% @doc Creates a worker description for a supervisor.\n-spec worker_desc(Name::atom() | string() | {atom(), pos_integer()}, Module::module(),\n                  Function::atom())\n        -> {Name::atom() | string() | {atom(), pos_integer()}, {Module::module(), Function::atom(), Options::[]},\n            permanent, brutal_kill, worker, []}.\nworker_desc(Name, Module, Function) ->\n    worker_desc(Name, Module, Function, []).\n\n%% @doc Creates a worker description for a supervisor.\n-spec worker_desc(Name::atom() | string() | {atom(), pos_integer()}, Module::module(),\n                  Function::atom(), Options::list())\n        -> {Name::atom() | string() | {atom(), pos_integer()}, {Module::module(), Function::atom(), Options::list()},\n            permanent, brutal_kill, worker, []}.\nworker_desc(Name, Module, Function, Options) ->\n    {Name, {Module, Function, Options}, permanent, brutal_kill, worker, []}.\n\n%% @doc Creates a supervisor description for a supervisor.\n-spec supervisor_desc(Name::atom() | string() | {atom(), pos_integer()}, Module::module(),\n                      Function::atom())\n        -> {Name::atom() | string() | {atom(), pos_integer()},\n            {Module::module(), Function::atom(), Options::[]},\n            permanent, brutal_kill, supervisor, []}.\nsupervisor_desc(Name, Module, Function) ->\n    supervisor_desc(Name, Module, Function, []).\n\n%% @doc Creates a supervisor description for a supervisor.\n-spec supervisor_desc(Name::atom() | string() | {atom(), pos_integer()}, Module::module(),\n                      Function::atom(), Options::list())\n        -> {Name::atom() | string() | {atom(), pos_integer()},\n            {Module::module(), Function::atom(), Options::list()},\n            permanent, brutal_kill, supervisor, []}.\nsupervisor_desc(Name, Module, Function, Args) ->\n    {Name, {Module, Function, Args}, permanent, brutal_kill, supervisor, []}.\n\n-spec sup_terminate(Supervisor::pid() | atom()) -> ok.\nsup_terminate(SupPid) ->\n    sup_terminate_childs(SupPid),\n    case is_pid(SupPid) of\n        true -> exit(SupPid, kill);\n        false ->\n            case whereis(SupPid) of\n                SupPid2 when is_pid(SupPid2) -> exit(SupPid2, kill);\n                _ -> ok\n            end\n    end,\n    util:wait_for_process_to_die(SupPid),\n    ok.\n\n%% @doc Terminates all children of the given supervisor(s) gracefully, i.e. first\n%%      stops all gen_component processes and then terminates all children\n%%      recursively.\n%%      Note: the children beneath the given supervisor pids MUST NOT overlap!\n-spec sup_terminate_childs(Supervisor::Proc | [Proc]) -> ok\n    when is_subtype(Proc, pid() | atom()).\nsup_terminate_childs(SupPid) when is_pid(SupPid) orelse is_atom(SupPid) ->\n    sup_terminate_childs([SupPid]);\nsup_terminate_childs(SupPids) when is_list(SupPids) ->\n    _ = [sup_pause_childs(SupPid) || SupPid <- SupPids],\n    _ = [sup_kill_childs(SupPid) || SupPid <- SupPids],\n    ok.\n\n%% @doc Pauses all children of the given supervisor (recursively) by setting\n%%      an appropriate breakpoint.\n-spec sup_pause_childs(Supervisor::pid() | atom()) -> ok.\nsup_pause_childs(SupPid) ->\n    ChildSpecs = sup_which_children(SupPid),\n    Self = self(),\n    _ = [ begin\n              case Type of\n                  supervisor ->\n                      sup_pause_childs(Pid);\n                  worker ->\n                      case gen_component:is_gen_component(Pid) of\n                          true -> gen_component:bp_about_to_kill(Pid);\n                          _    -> ok\n                      end\n              end\n          end ||  {_Id, Pid, Type, _Module} <- ChildSpecs,\n                  Pid =/= undefined, Pid =/= Self, is_process_alive(Pid) ],\n    ok.\n\n-spec get_tables_of(pid()) -> {MNesia::[atom()], Ets::[ets:tid() | atom()]} | [ets:tid() | atom()].\nget_tables_of(Pid)->\n    case config:read(db_backend) of\n        db_mnesia ->\n            {db_mnesia:mnesia_tables_of(pid_groups:my_groupname()),\n             util:ets_tables_of(Pid)};\n        _ ->\n            util:ets_tables_of(Pid)\n    end.\n\n-spec delete_tables(pid(), {MNesia::[atom()], Ets::[ets:tid() | atom()]} | [ets:tid() | atom()] ) -> ok.\ndelete_tables(Pid, Db) ->\n    case config:read(db_backend) of\n        db_mnesia ->\n            {MNesia, Ets} = Db,\n            _ = db_mnesia:delete_mnesia_tables(MNesia),\n            ok;\n        _ ->\n            Ets = Db,\n            ok\n    end,\n    _ = [ util:wait_for_ets_table_to_disappear(Pid, Tab) || Tab <- Ets ],\n    ok.\n\n%% @doc Kills all children of the given supervisor (recursively) after they\n%%      have been paused by sup_pause_childs/1.\n-spec sup_kill_childs(Supervisor::pid() | atom()) -> ok.\nsup_kill_childs(SupPid) ->\n    ChildSpecs = sup_which_children(SupPid),\n    _ = [ try\n              case Type of\n                  supervisor -> sup_kill_childs(Pid);\n                  worker     -> ok\n              end,\n              Tables = get_tables_of(Pid),\n              _ = supervisor:terminate_child(SupPid, Id),\n              _ = supervisor:delete_child(SupPid, Id),\n              util:wait_for_process_to_die(Pid),\n              delete_tables(Pid, Tables),\n              ok\n          catch\n              % child may not exist any more due to a parallel process terminating it\n              exit:{killed, _} -> ok;\n              exit:{noproc, _} -> ok;\n              % exit reason may encapsulate a previous failure\n              exit:{{killed, _}, _} -> ok;\n              exit:{{noproc, _}, _} -> ok\n          end ||  {Id, Pid, Type, _Module} <- ChildSpecs,\n                  Pid =/= undefined, is_process_alive(Pid) ],\n    ok.\n\n-spec sup_get_all_children(Supervisor::pid() | atom()) -> [pid()].\nsup_get_all_children(Supervisor) ->\n    AllChilds = [X || X = {_, Pid, _, _} <- sup_which_children(Supervisor),\n                      Pid =/= undefined],\n    WorkerChilds = [Pid ||  {_Id, Pid, worker, _Modules} <- AllChilds],\n    SupChilds = [Pid || {_Id, Pid, supervisor, _Modules} <- AllChilds],\n    lists:flatten([WorkerChilds | [sup_get_all_children(S) || S <- SupChilds]]).\n\n%% @doc Hardened version of supervisor:which_children/1 which returns an empty\n%%      list of childs if the supervisor has been killed, e.g. by another\n%%      process.\n-spec sup_which_children(Supervisor::pid() | atom()) ->\n          [{Id::term(), Pid::pid() | undefined, Type::worker | supervisor,\n            Modules::[module()] | dynamic}].\nsup_which_children(Supervisor) ->\n    try supervisor:which_children(Supervisor)\n    catch exit:{killed, _} -> [];\n          exit:{noproc, _} -> []\n    end.\n"
  },
  {
    "path": "src/sup_comm_layer.erl",
    "content": "%% @copyright 2008-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc    Supervisor for the comm layer processes\n%% @end\n%% @version $Id$\n-module(sup_comm_layer).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(supervisor).\n\n-export([start_link/0, init/1]).\n-export([supspec/1, childs/1]).\n\n-spec start_link() -> {ok, Pid::pid()} | ignore |\n                      {error, Error::{already_started, Pid::pid()} | shutdown | term()}.\nstart_link() ->\n    supervisor:start_link(?MODULE, []).\n\n-spec init([]) -> {ok, {{one_for_all, MaxRetries::pos_integer(),\n                                      PeriodInSeconds::pos_integer()},\n                         [ProcessDescr::supervisor:child_spec()]}}.\ninit(X) ->\n    CommLayerGroup = comm_layer,\n    pid_groups:join_as(CommLayerGroup, ?MODULE),\n    supspec(X).\n\n-spec supspec(any()) -> {ok, {{one_for_all, MaxRetries::pos_integer(),\n                         PeriodInSeconds::pos_integer()}, []}}.\nsupspec(_) ->\n    {ok, {{one_for_all, 10, 1}, []}}.\n\n-spec childs([]) ->\n                    [ProcessDescr::supervisor:child_spec()].\nchilds([]) ->\n    CommLayerGroup = comm_layer,\n    Delayer =\n        sup:worker_desc(comm_layer_msg_delay, msg_delay, start_link,\n                             [CommLayerGroup]),\n    CommServer =\n        sup:worker_desc(comm_server, comm_server, start_link,\n                             [CommLayerGroup]),\n    CommAcceptor =\n        case config:read(comm_backend) of\n            ssl ->\n                ok = ssl:start(), % ssl was requested otherwise fail\n                sup:worker_desc(comm_ssl_acceptor, comm_ssl_acceptor, start_link,\n                                [CommLayerGroup]);\n            gen_tcp ->\n                sup:worker_desc(comm_tcp_acceptor, comm_tcp_acceptor, start_link,\n                                [CommLayerGroup])\n        end,\n\n    CommLogger =\n        sup:worker_desc(comm_logger, comm_logger, start_link),\n    [\n     Delayer,\n     CommServer,\n     CommLogger,\n     CommAcceptor\n    ].\n"
  },
  {
    "path": "src/sup_dht_node.erl",
    "content": "%  @copyright 2007-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Supervisor for each DHT node that is responsible for keeping\n%%         processes running that run for themselves.\n%%\n%%         If one of the supervised processes fails, only the failed process\n%%         will be re-started!\n%% @end\n%% @version $Id$\n-module(sup_dht_node).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(supervisor).\n-include(\"scalaris.hrl\").\n\n-export([start_link/1, init/1]).\n-export([supspec/1, childs/1]).\n\n-spec start_link({DHTNodeGroup::pid_groups:groupname(), Options::[tuple()]})\n        -> {ok, Pid::pid(), pid_groups:groupname()} | ignore |\n               {error, Error::{already_started, Pid::pid()} | shutdown | term()}.\nstart_link({DHTNodeGroup, Options}) ->\n    case supervisor:start_link(?MODULE, [{DHTNodeGroup, Options}]) of\n        {ok, Pid} -> {ok, Pid, DHTNodeGroup};\n        X         -> X\n    end.\n\n%% userdevguide-begin sup_dht_node:init\n-spec init([{pid_groups:groupname(), [tuple()]}])\n        -> {ok, {{one_for_one, MaxRetries::pos_integer(),\n                  PeriodInSeconds::pos_integer()}, []}}.\ninit([{DHTNodeGroup, _Options}] = X) ->\n    pid_groups:join_as(DHTNodeGroup, ?MODULE),\n    supspec(X).\n%% userdevguide-end sup_dht_node:init\n\n-spec supspec(any()) -> {ok, {{one_for_one, MaxRetries::pos_integer(),\n                         PeriodInSeconds::pos_integer()}, []}}.\nsupspec(_) ->\n    {ok, {{one_for_one, 10, 1}, []}}.\n\n-spec childs([{pid_groups:groupname(), Options::[tuple()]}]) ->\n                    [ProcessDescr::supervisor:child_spec()].\nchilds([{DHTNodeGroup, Options}]) ->\n    Autoscale =\n        case config:read(autoscale) of\n            true -> sup:worker_desc(autoscale, autoscale, start_link, [DHTNodeGroup]);\n            _ -> []\n        end,\n    DBValCache =\n        sup:worker_desc(dht_node_db_cache, dht_node_db_cache, start_link,\n                             [DHTNodeGroup]),\n    DC_Clustering =\n        sup:worker_desc(dc_clustering, dc_clustering, start_link,\n                             [DHTNodeGroup]),\n    DeadNodeCache =\n        sup:worker_desc(deadnodecache, dn_cache, start_link,\n                             [DHTNodeGroup]),\n    Delayer =\n        sup:worker_desc(msg_delay, msg_delay, start_link,\n                             [DHTNodeGroup]),\n    Gossip =\n        sup:worker_desc(gossip, gossip, start_link, [DHTNodeGroup]),\n\n    VivaldiLatency =\n        sup:worker_desc(vivaldi_latency, vivaldi_latency, start_link, [DHTNodeGroup]),\n\n    GossipCyclonFeeder =\n        sup:worker_desc(gossip_cyclon_feeder, gossip_cyclon_feeder, start_link, [DHTNodeGroup]),\n\n    LBActive =\n        case config:read(lb_active) of\n            true -> sup:worker_desc(lb_active, lb_active, start_link, [DHTNodeGroup]);\n            _ -> []\n        end,\n    Reregister =\n        sup:worker_desc(dht_node_reregister, dht_node_reregister,\n                             start_link, [DHTNodeGroup]),\n    RMLeases =\n        sup:worker_desc(rm_leases, rm_leases,\n                             start_link, [DHTNodeGroup]),\n    RoutingTable =\n        sup:worker_desc(routing_table, rt_loop, start_link,\n                             [DHTNodeGroup]),\n    DHTNodeCache =\n        case config:read(cache_dht_nodes) of\n            true ->\n                sup:worker_desc(dht_node_cache, dht_node_cache, start_link, [DHTNodeGroup]);\n            false ->\n                []\n        end,\n    SupDHTNodeCore_AND =\n        sup:supervisor_desc(sup_dht_node_core, sup_dht_node_core,\n                                 start_link, [DHTNodeGroup, Options]),\n    %% SupMr = case config:read(mr_enable) of\n    %%     true ->\n    %%         util:sup_supervisor_desc(sup_mr, sup_mr, start_link, [DHTNodeGroup]);\n    %%     _ -> []\n    %% end,\n    Monitor =\n        sup:worker_desc(monitor, monitor, start_link, [DHTNodeGroup]),\n    RepUpdate =\n        case config:read(rrepair_enabled) of\n            true -> sup:worker_desc(rrepair, rrepair, start_link, [DHTNodeGroup]);\n            _ -> []\n        end,\n    SnapshotLeader =\n        sup:worker_desc(snapshot_leader, snapshot_leader, start_link,\n                        [DHTNodeGroup]),\n    SupWPool =\n        sup:supervisor_desc(sup_wpool, sup_wpool, start_link, [DHTNodeGroup]),\n    WPool = sup:worker_desc(wpool, wpool, start_link, [DHTNodeGroup, Options]),\n\n    lists:flatten([ %% RepUpd may be [] and lists:flatten eliminates this\n                    Monitor,\n                    Delayer,\n                    Reregister,\n                    DBValCache,\n                    DeadNodeCache,\n                    DHTNodeCache,\n                    RoutingTable,\n                    DC_Clustering,\n                    VivaldiLatency,\n                    Gossip,\n                    GossipCyclonFeeder,\n                    SnapshotLeader,\n                    SupWPool,\n                    WPool,\n                    SupDHTNodeCore_AND,\n                    RepUpdate,\n                    Autoscale,\n                    RMLeases,\n                    LBActive\n           ]).\n"
  },
  {
    "path": "src/sup_dht_node_core.erl",
    "content": "%  @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Supervisor for each DHT node that is responsible for keeping\n%%         processes running that are essential to the operation of the node.\n%%\n%%         If one of the supervised processes (dht_node, msg_delay or\n%%         sup_dht_node_core_tx) fails, all will be re-started!\n%%         Note that the DB is needed by the dht_node (and not vice-versa) and\n%%         is thus started at first.\n%% @end\n%% @version $Id$\n-module(sup_dht_node_core).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(supervisor).\n\n-export([start_link/2, init/1, check_config/0]).\n-export([supspec/1, childs/1]).\n\n-spec start_link(pid_groups:groupname(), Options::[tuple()]) ->\n                        {ok, Pid::pid()} | ignore |\n                        {error, Error::{already_started, Pid::pid()} |\n                                       shutdown | term()}.\nstart_link(DHTNodeGroup, Options) ->\n    supervisor:start_link(?MODULE, [DHTNodeGroup, Options]).\n\n%% userdevguide-begin sup_dht_node_core:init\n-spec init([{pid_groups:groupname(), Options::[tuple()]}]) ->\n                  {ok, {{one_for_all, MaxRetries::pos_integer(),\n                         PeriodInSeconds::pos_integer()},\n                        [ProcessDescr::supervisor:child_spec()]}}.\ninit([DHTNodeGroup, _Options] = X) ->\n    pid_groups:join_as(DHTNodeGroup, ?MODULE),\n    supspec(X).\n%% userdevguide-end sup_dht_node_core:init\n\n-spec supspec(any()) -> {ok, {{one_for_all, MaxRetries::pos_integer(),\n                         PeriodInSeconds::pos_integer()}, []}}.\nsupspec(_) ->\n    {ok, {{one_for_all, 10, 1}, []}}.\n\n-spec childs([{pid_groups:groupname(), Options::[tuple()]}]) ->\n                    [ProcessDescr::supervisor:child_spec()].\nchilds([DHTNodeGroup, Options]) ->\n    PaxosProcesses = sup:supervisor_desc(sup_paxos, sup_paxos,\n                                              start_link, [{DHTNodeGroup, []}]),\n    DHTNodeModule = config:read(dht_node),\n    DHTNode = sup:worker_desc(dht_node, DHTNodeModule, start_link,\n                                   [DHTNodeGroup, Options]),\n    %% rbrcseq process working on the kv DB\n\n    KV_RBRcseq = sup:worker_desc(\n                   kv_db, rbrcseq,\n                   start_link,\n                   [DHTNodeGroup,\n                    _PidGroupsNameKV = kv_db,\n                    _DBSelectorKV = kv_db]),\n    %% rbrcseq process working on the lease_db1 DB\n    Lease_RBRcseqs = [sup:worker_desc(\n                   {lease_db, Id}, rbrcseq,\n                   start_link,\n                   [DHTNodeGroup,\n                    _PidGroupsNameL1 = {lease_db, Id},\n                    _DBSelectorL1 = {lease_db, Id}])\n        || Id <- lists:seq(1, config:read(replication_factor))],\n\n    Tx_RBRcseqs = [sup:worker_desc(\n                   {tx_id, Id}, rbrcseq,\n                   start_link,\n                   [DHTNodeGroup,\n                    _PidGroupsNameL1 = {tx_id, Id},\n                    _DBSelectorL1 = {tx_id, Id}])\n                   || Id <- lists:seq(1, config:read(replication_factor))],\n\n    %% crdt process  working on the kv DB\n    Crdt_KVcseq = sup:worker_desc(crdt_db, crdt:proposer_module(),\n                                start_link,\n                                [DHTNodeGroup,\n                                 _PidGroupsNameCrdt = crdt_db,\n                                 _DBSelectorCrdt = crdt_db]),\n\n    DHTNodeMonitor = sup:worker_desc(\n                       dht_node_monitor, dht_node_monitor, start_link,\n                       [DHTNodeGroup, Options]),\n    TX =\n        sup:supervisor_desc(sup_dht_node_core_tx, sup_dht_node_core_tx, start_link,\n                                 [DHTNodeGroup]),\n    lists:flatten([\n     PaxosProcesses,\n     KV_RBRcseq,\n     Lease_RBRcseqs,\n     Tx_RBRcseqs,\n     Crdt_KVcseq,\n     DHTNodeMonitor,\n     DHTNode,\n     TX\n    ]).\n\n\n%% @doc Checks whether config parameters for the sup_dht_node_core supervisor\n%%      exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_module(dht_node).\n"
  },
  {
    "path": "src/sup_dht_node_core_tx.erl",
    "content": "%  @copyright 2009-2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc    Supervisor for each DHT node that is responsible for keeping\n%%         its transaction processes running.\n%%\n%%         If one of the supervised processes fails, it will be re-started!\n%% @end\n%% @version $Id$\n-module(sup_dht_node_core_tx).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-behaviour(supervisor).\n\n-export([start_link/1, init/1]).\n-export([supspec/1, childs/1]).\n\n-spec start_link(pid_groups:groupname())\n        -> {ok, Pid::pid()} | ignore |\n           {error, Error::{already_started, Pid::pid()} | shutdown | term()}.\nstart_link(DHTNodeGroup) ->\n    supervisor:start_link(?MODULE, [DHTNodeGroup]).\n\n-spec init([pid_groups:groupname()]) ->\n                  {ok, {{one_for_one, MaxRetries::pos_integer(),\n                         PeriodInSeconds::pos_integer()},\n                        [ProcessDescr::supervisor:child_spec()]}}.\ninit([DHTNodeGroup]) ->\n    pid_groups:join_as(DHTNodeGroup, ?MODULE),\n    supspec([DHTNodeGroup]).\n\n-spec supspec(any()) -> {ok, {{one_for_one, MaxRetries::pos_integer(),\n                         PeriodInSeconds::pos_integer()}, []}}.\nsupspec(_) ->\n    {ok, {{one_for_one, 10, 1}, []}}.\n\n-spec childs([pid_groups:groupname()]) ->\n                    [ProcessDescr::supervisor:child_spec()].\nchilds([DHTNodeGroup]) ->\n    RDHT_tx_read = sup:worker_desc(rdht_tx_read, rdht_tx_read, start_link,\n                                        [DHTNodeGroup]),\n    RDHT_tx_write = sup:worker_desc(rdht_tx_write, rdht_tx_write,\n                                         start_link, [DHTNodeGroup]),\n    TX_TM = sup:worker_desc(tx_tm, tx_tm_rtm, start_link,\n                                 [DHTNodeGroup, tx_tm]),\n    TX_TM_Paxos = sup:supervisor_desc(\n                    sup_paxos_tm, sup_paxos, start_link,\n                    [{DHTNodeGroup, [{sup_paxos_parent, tx_tm}]}]),\n\n\n    TX_RTMs = [sup:worker_desc(\n                   {tx_rtm, Id}, tx_tm_rtm,\n                   start_link,\n                   [DHTNodeGroup, {tx_rtm, Id}])\n               || Id <- lists:seq(1, config:read(replication_factor))],\n\n    TX_RTM_Paxi = [sup:supervisor_desc(\n                {sup_paxos_rtm, Id}, sup_paxos, start_link,\n                [{DHTNodeGroup, [{sup_paxos_parent, {tx_rtm, Id}}]}])\n                   || Id <- lists:seq(1, config:read(replication_factor))],\n\n    TX_TM_New = sup:worker_desc(tx_tm_new, tx_tm, start_link,\n                                 [DHTNodeGroup, tx_tm_new]),\n    lists:flatten([\n                   RDHT_tx_read, RDHT_tx_write,\n                   %% start paxos supervisors before tx processes to create used atoms\n                   TX_TM_Paxos, TX_TM,\n                   TX_RTM_Paxi,\n                   TX_RTMs,\n                   TX_TM_New\n    ]).\n"
  },
  {
    "path": "src/sup_paxos.erl",
    "content": "%  @copyright 2011, 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n\n%% @doc Supervisor to instantiate paxos processes (proposer, acceptor,\n%%      learner). Each process that uses paxos should instantiate\n%%      its own set of paxos processes to avoid clashing of paxos ids.\n%%\n%%      To name the processes inside a pid_group, a naming prefix can\n%%      be passed %% to the start_link/2 call. The supervisor will\n%%      append '_proposer', '_acceptor', and '_learner' to the given\n%%      prefix for the individual processes. If no prefix is given,\n%%      the processes register as 'paxos_proposer', 'paxos_acceptor',\n%%      and 'paxos_learner' directly.\n%% @end\n%% @version $Id$\n-module(sup_paxos).\n-author('schintke@zib.de').\n-vsn('$Id$').\n-behaviour(supervisor).\n-export([start_link/1, init/1]).\n-export([supspec/1, childs/1]).\n\n-spec start_link({pid_groups:groupname(), Options::[tuple()]}) ->\n                        {ok, Pid::pid()} | ignore |\n                        {error, Error::{already_started, Pid::pid()} |\n                                       shutdown | term()}.\nstart_link({PidGroup, Options}) ->\n    supervisor:start_link(?MODULE, [{PidGroup, Options}]).\n\n%% userdevguide-begin sup_dht_node_core:init\n-spec init([{pid_groups:groupname(), Options::[tuple()]}]) ->\n                  {ok, {{one_for_one, MaxRetries::pos_integer(),\n                         PeriodInSeconds::pos_integer()},\n                        [ProcessDescr::supervisor:child_spec()]}}.\ninit([{PidGroup, Options}] = X) ->\n    SupervisorName =\n        case lists:keyfind(sup_paxos_parent, 1, Options) of\n            {sup_paxos_parent, Parent} ->\n                {Parent, ?MODULE};\n            false -> ?MODULE\n        end,\n    pid_groups:join_as(PidGroup, SupervisorName),\n    supspec(X).\n\n-spec supspec(any()) -> {ok, {{one_for_one, MaxRetries::pos_integer(),\n                         PeriodInSeconds::pos_integer()}, []}}.\nsupspec(_) ->\n    {ok, {{one_for_one, 10, 1}, []}}.\n\n-spec childs([{pid_groups:groupname(), Options::[tuple()]}]) ->\n                    [ProcessDescr::supervisor:child_spec()].\nchilds([{PidGroup, Options}]) ->\n    {ProposerName, AcceptorName, LearnerName} =\n        case lists:keyfind(sup_paxos_parent, 1, Options) of\n            {sup_paxos_parent, Parent} ->\n                {{Parent, proposer},\n                 {Parent, acceptor},\n                 {Parent, learner}};\n            false ->\n                {paxos_proposer, paxos_acceptor, paxos_learner}\n        end,\n    Proposer = sup:worker_desc(proposer, proposer,\n                                    start_link, [PidGroup, ProposerName]),\n    Acceptor = sup:worker_desc(acceptor, acceptor,\n                                    start_link, [PidGroup, AcceptorName]),\n    Learner = sup:worker_desc(learner, learner,\n                                   start_link, [PidGroup, LearnerName]),\n    [Proposer, Acceptor, Learner].\n"
  },
  {
    "path": "src/sup_scalaris.erl",
    "content": "%  @copyright 2007-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Supervisor for a mgmt server or/and \"ordinary\" node that is\n%%      responsible for keeping its processes running.\n%%\n%%      If one of the supervised processes fails, only the failed\n%%      process will be re-started!\n%% @end\n-module(sup_scalaris).\n-author('schuett@zib.de').\n\n-include(\"scalaris.hrl\").\n\n-behaviour(supervisor).\n-export([init/1]).\n\n-export([supspec/1, childs/1]).\n\n-export([start_link/0, check_config/0]).\n\n%% used in unittest_helper.erl\n-export([start_link/1]).\n-export([stop_first_services/0]).\n\n-spec start_link() -> {ok, Pid::pid()}\n                         | {error, Error::{already_started, Pid::pid()}\n                                        | term()}.\nstart_link() -> start_link([]).\n\n%% called by scalaris.erl and by unittest_helper.erl\n-spec start_link([tuple()])\n        -> {ok, Pid::pid()} |\n           {error, Error::{already_started, Pid::pid()} | shutdown | term()}.\nstart_link(Options) ->\n    ServiceGroup = basic_services,\n    %% start watching the startup\n    %% (StartupWatcher is killed below when we succeed)\n    StartupWatcher = spawn(fun() -> timer:sleep(5*1000), dump_processes() end),\n    Link = sup:sup_start({local, main_sup}, ?MODULE,\n                         [{service_group, ServiceGroup} | Options]),\n    case Link of\n        {ok, SupRef} when is_pid(SupRef) ->\n            case config:read(start_type) of\n                recover -> ok;\n                _ ->\n                    case pid_groups:find_all(dht_node) of\n                        [DhtNodePid] ->\n                            comm:send_local(DhtNodePid, {join, start});\n                        [] -> ok\n                    end\n            end,\n            add_additional_nodes(),\n            %% startup successful, stop watching the startup\n            exit(StartupWatcher, kill),\n            case util:is_unittest() of\n                true -> ok;\n                _    -> io:format(\"Scalaris started successfully.\"\n                                  \" Hit <return> to see the erlang shell prompt.~n\")\n            end,\n            ok;\n%%        ignore ->\n%%            log:log(\n%%              \"error in starting sup_scalaris supervisor:\"\n%%              \" supervisor should not return ignore~n\",\n%%              []);\n        {error, Error} ->\n            log:log(\"error in starting sup_scalaris supervisor: ~p~n\", [Error])\n   end,\n   Link.\n\n-spec init([tuple()])\n        -> {ok, {{one_for_one, MaxRetries::pos_integer(),\n                  PeriodInSeconds::pos_integer()},\n                 [ProcessDescr::supervisor:child_spec()]}}.\ninit(Options) ->\n    start_first_services(),\n    supspec(Options).\n\n-spec supspec(any()) -> {ok, {{one_for_one, MaxRetries::pos_integer(),\n                  PeriodInSeconds::pos_integer()}, []}}.\nsupspec(_) ->\n    {ok, {{one_for_one, 10, 1}, []}}.\n\n-spec get_dht_node_descs([tuple()]) -> [ProcessDescr::supervisor:child_spec()].\nget_dht_node_descs(Options) ->\n  case config:read(start_type) of\n    recover ->\n      %% creating tuples with DB_names different parts : {DB_type, PID_group, DB_name}\n      DB_list = db_util:get_recoverable_dbs(),\n      %% creating list of all nodes per vm and removing duplicates\n      PID_groups = lists:usort([PidGroup || {_, PidGroup, _} <- DB_list]),\n\n      %% create descriptions for all dht nodes to recover:\n      [begin\n           Option_new =\n               [{list_to_atom(Type), DBName}\n               || {Type, X, DBName} <- DB_list, X =:= PID_group],\n           DhtNodeId = randoms:getRandomString(),\n           TheOptions = [{my_sup_dht_node_id, DhtNodeId} | lists:append(Options, Option_new)],\n           sup:supervisor_desc(DhtNodeId, sup_dht_node, start_link,\n                               [{PID_group, TheOptions}])\n       end || PID_group <- PID_groups];\n    _ ->\n      DHTNodeJoinAt = case {util:app_get_env(join_at, random),\n                            util:app_get_env(join_at_list, no_list)} of\n                          {random, no_list} ->\n                              [];\n                          {_, List} when is_list(List) ->\n                              [{{dht_node, id}, hd(List)}, {skip_psv_lb}];\n                           {Id, no_list} ->\n                              [{{dht_node, id}, Id},       {skip_psv_lb}]\n                      end,\n      DhtNodeId = randoms:getRandomString(),\n      DHTNodeOptions = DHTNodeJoinAt ++ [{first} | Options], % this is the first dht_node in this VM\n      DHTNodeGroup = pid_groups:new(dht_node),\n      sup:supervisor_desc(DhtNodeId, sup_dht_node, start_link,\n        [{DHTNodeGroup, [{my_sup_dht_node_id, DhtNodeId}\n          | DHTNodeOptions]}])\n  end.\n\n-spec childs(list(tuple())) -> [any()].\nchilds(Options) ->\n    {service_group, ServiceGroup} = lists:keyfind(service_group, 1, Options),\n    StartMgmtServer = case config:read(start_mgmt_server) of\n                          failed -> false;\n                          X -> X\n                      end,\n\n    AdminServer = sup:worker_desc(admin_server, admin, start_link),\n    BenchServer = sup:worker_desc(bench_server, bench_server, start_link, [ServiceGroup]),\n    MgmtServer = sup:worker_desc(mgmt_server, mgmt_server, start_link,\n                                      [ServiceGroup, []]),\n    MgmtServerDNCache =\n        sup:worker_desc(deadnodecache, dn_cache, start_link,\n                             [ServiceGroup]),\n    CommLayer =\n        sup:supervisor_desc(sup_comm_layer, sup_comm_layer, start_link),\n    CommStats =\n        sup:worker_desc(comm_stats, comm_stats, start_link, [comm_layer]),\n    ClientsDelayer =\n        sup:worker_desc(clients_msg_delay, msg_delay, start_link,\n                             [clients_group]),\n    BasicServicesDelayer =\n        sup:worker_desc(basic_services_msg_delay, msg_delay, start_link,\n                             [ServiceGroup]),\n    ClientsMonitor =\n        sup:worker_desc(clients_monitor, monitor, start_link, [clients_group]),\n    %% Moves several lines to get_dht_node_descs() for recovery mechanims\n    DHTNodes = get_dht_node_descs(Options),\n    FailureDetector = sup:worker_desc(fd, fd, start_link, [ServiceGroup]),\n    Ganglia = case config:read(ganglia_enable) of\n                  true -> sup:worker_desc(ganglia_server, ganglia, start_link, [ServiceGroup]);\n                  _ -> []\n              end,\n    Monitor =\n        sup:worker_desc(monitor, monitor, start_link, [ServiceGroup]),\n    MonitorPerf =\n        sup:worker_desc(monitor_perf, monitor_perf, start_link, [ServiceGroup]),\n    Service =\n        sup:worker_desc(service_per_vm, service_per_vm, start_link,\n                             [ServiceGroup]),\n    TraceMPath =\n        sup:worker_desc(trace_mpath, trace_mpath, start_link,\n                             [ServiceGroup]),\n    ProtoSched =\n        sup:worker_desc(proto_sched, proto_sched, start_link,\n                             [ServiceGroup]),\n\n    Top =\n        sup:worker_desc(top, top, start_link,\n                             [ServiceGroup]),\n\n    ServicePaxosGroup = sup:supervisor_desc(\n                          sup_service_paxos_group, sup_paxos, start_link,\n                          [{ServiceGroup, []}]),\n    AutoscaleServer =\n        case (config:read(autoscale_server) =:= true) andalso\n                 StartMgmtServer of\n            true -> sup:worker_desc(autoscale_server, autoscale_server,\n                                         start_link, [ServiceGroup]);\n            _    -> []\n        end,\n    %% order in the following list is the start order\n    BasicServers = [TraceMPath,\n                    ClientsDelayer,\n                    BasicServicesDelayer,\n                    ProtoSched,\n                    ClientsMonitor,\n                    Top,\n                    Monitor,\n                    CommStats,\n                    CommLayer,\n                    FailureDetector,\n                    AdminServer,\n                    ServicePaxosGroup,\n                    AutoscaleServer],\n    Servers = [BenchServer],\n    MgmtServers =\n        case StartMgmtServer orelse util:is_unittest() of\n            true -> [MgmtServerDNCache, MgmtServer];\n            false -> []\n        end,\n    DHTNodeServer = case config:read(start_type) of\n                        nostart -> []; %% no dht node requested\n                        first_nostart -> []; %% no dht node requested\n                        failed -> [];\n                        _ -> DHTNodes\n                    end,\n    lists:flatten([BasicServers, MgmtServers, Service, Servers, DHTNodeServer, Ganglia, MonitorPerf]).\n\n-spec add_additional_nodes() -> ok.\nadd_additional_nodes() ->\n    Size = config:read(nodes_per_vm),\n    log:log(info, \"Starting ~B nodes\", [Size]),\n    _ = case util:app_get_env(join_at_list, no_list) of\n            no_list ->\n                api_vm:add_nodes(Size - 1);\n            List ->\n                api_vm:add_nodes_at_ids(tl(List))\n        end,\n    ok.\n\nstart_first_services() ->\n    util:if_verbose(\"~p start first services...~n\", [?MODULE]),\n    util:if_verbose(\"~p start randoms...~n\", [?MODULE]),\n    randoms:start(),\n    util:if_verbose(\"~p start inets~n\", [?MODULE]),\n    _ = inets:start(),\n\n    %% for mnesia\n    start_mnesia(),\n\n    %% for lb_stats and wpool\n    _ = application:load(sasl),\n    application:set_env(sasl, sasl_error_logger, false),\n    _ = application:start(sasl),\n\n    %% for lb_stats\n    _ = application:load(os_mon),\n    IsLbActive = config:read(lb_active) =:= true,\n    ExplicitOsMon = config:read(start_os_mon) =:= true,\n    case IsLbActive or ExplicitOsMon of\n        true -> %% for lb_stats\n            application:set_env(os_mon, start_os_sup, false),\n            _ = application:start(os_mon),\n            ok;\n        _ -> ok\n    end,\n    case config:read(wpool_js) of\n        true -> %% for wpool\n            _ = application:start(erlang_js),\n            ok;\n        _ -> ok\n    end,\n    util:if_verbose(\"~p start first services done.~n\", [?MODULE]).\n\n%% stop the services we started outside the supervisor tree\n%% (those who materialized as processes)\n-spec stop_first_services() -> ok.\nstop_first_services() ->\n    %% config seems not available here, so we stop unconditionally\n    _ = application:stop(erlang_js),\n    _ = application:stop(os_mon),\n    _ = application:stop(sasl),\n    _ = inets:stop(),\n    ok.\n\n-spec start_mnesia() -> ok.\nstart_mnesia() ->\n  case config:read(db_backend) of\n    db_mnesia -> db_mnesia:start();\n    _ -> ok\n  end.\n\ndump_processes() ->\n    Processes = [ {X, process_info(X, [initial_call,current_location, current_stacktrace])}\n                  || X <- processes()],\n    io:format(\"~p~n\", [lists:last(Processes)]),\n    ok.\n\n%% @doc Checks whether config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(nodes_per_vm) and\n    config:cfg_is_port(yaws_port) and\n    config:cfg_is_string(docroot) and\n    config:cfg_is_in(start_type, [first, joining, quorum, recover, nostart, first_nostart, client]).\n"
  },
  {
    "path": "src/sup_wpool.erl",
    "content": "%% @copyright 2007-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@informatik.hu-berlin.de>\n%% @doc    supervisor for wpool workers\n%% @end\n%% @version $Id$\n-module(sup_wpool).\n-author('fajerski@zib.de').\n-vsn('$Id$ ').\n\n-behaviour(supervisor).\n-include(\"scalaris.hrl\").\n\n-export([start_link/1, init/1]).\n-export([supspec/1]).\n\n-spec start_link(pid_groups:groupname())\n        -> {ok, Pid::pid(), pid_groups:groupname()} | ignore |\n               {error, Error::{already_started, Pid::pid()} | shutdown | term()}.\nstart_link(DHTNodeGroup) ->\n    case supervisor:start_link(?MODULE, DHTNodeGroup) of\n        {ok, Pid} -> {ok, Pid, DHTNodeGroup};\n        X         -> X\n    end.\n\n-spec init(pid_groups:groupname()) ->\n                  {ok, {{one_for_one, MaxRetries::pos_integer(),\n                         PeriodInSeconds::pos_integer()}, []}}.\ninit(DHTNodeGroup) ->\n    pid_groups:join_as(DHTNodeGroup, sup_wpool),\n    supspec([DHTNodeGroup]).\n\n-spec supspec(any()) -> {ok, {{one_for_one, MaxRetries::pos_integer(),\n                         PeriodInSeconds::pos_integer()}, []}}.\nsupspec(_) ->\n    {ok, {{one_for_one, 10, 1}, []}}.\n"
  },
  {
    "path": "src/sup_yaws.erl",
    "content": "%% @copyright 2011-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n-module(sup_yaws).\n-author('kruber@zib.de').\n\n-behaviour(supervisor).\n\n-export([start_link/0, init/1, check_config/0]).\n\n-include(\"yaws.hrl\").\n\n-spec start_link() -> {ok, Pid::pid()} | ignore |\n                      {error, Error::{already_started, Pid::pid()} | shutdown | term()}.\nstart_link() ->\n    % preload api_json modules to make atoms known to list_to_existing_atom/1\n    _ = api_json:module_info(),\n    _ = api_json_dht_raw:module_info(),\n    _ = api_json_monitor:module_info(),\n    _ = api_json_rdht:module_info(),\n    _ = api_json_tx:module_info(),\n    Id = io_lib:format(\"yaws@~p\", [node()]),\n    Docroot = config:read(docroot),\n    AuthDirs = case config:read(yaws_auth) of\n                   UserPwds = [_|_] -> [#auth{dir = \"/\",\n                                              docroot = Docroot,\n                                              realm = \"Scalaris Web Interface\",\n                                              users = UserPwds}];\n                   _                -> []\n               end,\n    GconfList = [{max_connections, 800},\n                 {logdir, config:read(log_path)},\n                 {include_dir, [Docroot ++ \"/api\"]},\n                 {id, Id}\n                ],\n    SconfList2 = [{docroot, Docroot},\n                 {port, config:read(yaws_port)},\n                 {listen, {0,0,0,0}},\n                 {allowed_scripts, [yaws]},\n                 {partial_post_size, config:read(yaws_max_post_data)},\n                 {authdirs, AuthDirs},\n                 {flags, [{access_log, false}]} % {deflate, true}\n                ],\n    SconfList = case config:read(yaws_ssl) of\n                    true ->\n                        _ = application:ensure_all_started(ssl), %% >= R16B02\n                        SSL= {ssl, [\n%% https://jamielinux.com/docs/openssl-certificate-authority/\n                                    {keyfile, config:read(yaws_keyfile)},\n                                    {certfile, config:read(yaws_certfile)},\n                                    {cacertfile, config:read(yaws_cacertfile)},\n                                    {verify, verify_peer},\n                                    {fail_if_no_peer_cert, true},\n                                    {password, config:read(yaws_sslpassword)}\n                                   ]},\n                        io:format(\"~p~n\", [SSL]),\n                        [SSL | SconfList2];\n                    _ -> SconfList2\n                end,\n\n    % load yaws application (required by yaws_api:embedded_start_conf)\n    _ = application:load(\n          {application, yaws,\n           [{description, \"yaws WWW server\"},\n            {vsn, \"%VSN%\"},\n            {modules, []},\n            {registered, []},\n            {mod, {yaws_app, []}},\n            {env, []},\n            {applications, [kernel, stdlib]}]}),\n\n    {ok, SCList, GC, ChildSpecs} =\n        yaws_api:embedded_start_conf(Docroot, SconfList, GconfList, Id),\n\n    util:if_verbose(\"Yaws listening at port ~p.~n\", [config:read(yaws_port)]),\n    X = supervisor:start_link(?MODULE, ChildSpecs),\n\n    %% now configure Yaws\n    ok = yaws_api:setconf(GC, SCList),\n    %% remove yaws error logger, as we have our own\n    error_logger:delete_report_handler(yaws_log_file_h),\n    X.\n\n-spec init(ChildSpecs) -> {ok, {{one_for_all, MaxRetries::pos_integer(),\n                                      PeriodInSeconds::pos_integer()},\n                         ChildSpecs}} when is_subtype(ChildSpecs, [supervisor:child_spec()]).\ninit(ChildSpecs) ->\n    {ok, {{one_for_all, 10, 1}, ChildSpecs}}.\n\n%% @doc Checks whether yaws config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_string(docroot) and\n    config:cfg_is_string(log_path) and\n    config:cfg_is_port(yaws_port) and\n    config:cfg_is_list(yaws_auth,\n                       fun(X) ->\n                               case X of\n                                   {User, Password} when is_list(User)\n                                     andalso is_list(Password) -> true;\n                                   _ -> false\n                               end\n                       end, \"{\\\"User\\\", \\\"Password\\\"}\") and\n\n    case config:read(yaws_max_post_data) of\n        nolimit -> true;\n        _ -> config:cfg_is_integer(yaws_max_post_data) and\n             config:cfg_is_greater_than(yaws_max_post_data, 0)\n    end.\n"
  },
  {
    "path": "src/time/clocks.erl",
    "content": "%  @copyright 2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    functions for querying clocks\n\n%% @version $Id$\n-module(clocks).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n%-on_load(on_load/0).\n\n-export([init/0,\n         get_monotonic_clock/0, get_monotonic_clock_res/0,\n         get_realtime_clock/0, get_realtime_clock_res/0,\n         get_ptp0_clock/0, get_ptp0_clock_res/0,\n         get_ptp1_clock/0, get_ptp1_clock_res/0,\n         get_ptp2_clock/0, get_ptp2_clock_res/0]).\n\n-export([test/0]).\n\n-spec init() -> ok.\ninit() ->\n    %% loads the shared library\n    SoName =\n        case code:priv_dir(scalaris) of\n            {error, bad_name} ->\n                Dir1 = filename:join(\n                         [filename:dirname(code:where_is_file(\"scalaris.beam\")),\n                          \"..\", priv]),\n                case filelib:is_dir(Dir1) of\n                    true ->\n                        filename:join(Dir1, ?MODULE);\n                    _ ->\n                        filename:join([\"..\", priv, ?MODULE])\n                end;\n            Dir ->\n                filename:join(Dir, ?MODULE)\n        end,\n    erlang:load_nif(SoName, 0).\n\n-spec get_monotonic_clock() -> failed | {non_neg_integer(), non_neg_integer()}.\nget_monotonic_clock() ->\n    erlang:nif_error(undef).\n\n-spec get_monotonic_clock_res() -> failed | {non_neg_integer(), non_neg_integer()}.\nget_monotonic_clock_res() ->\n    erlang:nif_error(undef).\n\n-spec get_realtime_clock() -> failed | {non_neg_integer(), non_neg_integer()}.\nget_realtime_clock() ->\n    erlang:nif_error(undef).\n\n-spec get_realtime_clock_res() -> failed | {non_neg_integer(), non_neg_integer()}.\nget_realtime_clock_res() ->\n    erlang:nif_error(undef).\n\n-spec get_ptp0_clock() -> failed | {non_neg_integer(), non_neg_integer()}.\nget_ptp0_clock() ->\n    erlang:nif_error(undef).\n\n-spec get_ptp0_clock_res() -> failed | {non_neg_integer(), non_neg_integer()}.\nget_ptp0_clock_res() ->\n    erlang:nif_error(undef).\n\n-spec get_ptp1_clock() -> failed | {non_neg_integer(), non_neg_integer()}.\nget_ptp1_clock() ->\n    erlang:nif_error(undef).\n\n-spec get_ptp1_clock_res() -> failed | {non_neg_integer(), non_neg_integer()}.\nget_ptp1_clock_res() ->\n    erlang:nif_error(undef).\n\n-spec get_ptp2_clock() -> failed | {non_neg_integer(), non_neg_integer()}.\nget_ptp2_clock() ->\n    erlang:nif_error(undef).\n\n-spec get_ptp2_clock_res() -> failed | {non_neg_integer(), non_neg_integer()}.\nget_ptp2_clock_res() ->\n    erlang:nif_error(undef).\n\n-spec test() -> ok.\ntest() ->\n    init(),\n    io:format(\"monotonic ~p:~p~n\", [get_monotonic_clock(), get_monotonic_clock_res()]),\n    io:format(\"realtime ~p:~p~n\", [get_realtime_clock(), get_realtime_clock_res()]),\n    io:format(\"ptp0 ~p:~p~n\", [get_ptp0_clock(), get_ptp0_clock_res()]),\n    io:format(\"ptp1 ~p:~p~n\", [get_ptp1_clock(), get_ptp1_clock_res()]),\n    io:format(\"ptp2 ~p:~p~n\", [get_ptp2_clock(), get_ptp2_clock_res()]),\n    ok.\n"
  },
  {
    "path": "src/top.erl",
    "content": "% @copyright 2012-2015 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc    Text mode top for an Erlang VM. Just call top:top() in the eshell.\n%% @end\n%% @version $Id$\n-module(top).\n-author('schintke@zib.de').\n-vsn('$Id:$ ').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n-include(\"scalaris.hrl\").\n\n-behaviour(gen_component).\n\n-export([start_link/1]).\n-export([on/2, init/1]).\n\n-export([top/0]).\n-export([trace_fwd/2]).\n\n-include(\"gen_component.hrl\").\n\n-type state() :: { pdb:tableid(),\n                   false | all | pid, %% sampling: no, all or a single pid\n                   false | all | pid, %% output: no, all, or a single pid\n                   erlang_timestamp(), %% last output\n                   non_neg_integer(), %% # of measures\n                   non_neg_integer(), %% sort by\n                   no_exclude | pid()\n                 }.\n\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(Group) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                             [{erlang_register, ?MODULE},\n                              {pid_groups_join_as, Group, ?MODULE}]).\n\n-spec init([]) -> state().\ninit([]) ->\n    try\n        Candidates = [ {X, process_info(X, [initial_call])}\n                       || X <- processes()],\n        Leaders = [ {Pid, process_info(Pid, [registered_name])}\n                    || {Pid, [{initial_call, Call}]} <- Candidates,\n                       Call =:= {group, server, 3}],\n        [Leader] = [ Pid ||  {Pid, [{registered_name, []}]} <- Leaders],\n        erlang:group_leader(Leader, self())\n    catch _:_ -> ok\n    end,\n    {pdb:new(?MODULE, [set]), false, false,\n     os:timestamp(), 0, _SortBy = 4, self()}.\n\n-spec top() -> ok.\ntop() ->\n    io:format(\"~p~n\", [io:rows(standard_io)]),\n    Pid = pid_groups:find_a(?MODULE),\n    comm:send_local(Pid, {enable_sampling, all}),\n    comm:send_local(Pid, {sample_all}),\n    comm:send_local(Pid, {enable_output, all}),\n    comm:send_local(Pid, {output_all}),\n    top_interactive(Pid).\n\ntop_interactive(Pid) ->\n    {ok, [Char]} = io:fread(\"> \", \"~c\"),\n    Quit =\n        case Char of\n            \"q\" -> quit;\n            \"e\" ->\n                comm:send_local(Pid, {output_off}),\n                Expr = io:parse_erl_exprs('expr>'),\n                comm:send_local(Pid, {enable_output, all}),\n                comm:send_local(Pid, {output_all}),\n                case element(1, Expr) of\n                    ok ->\n                        spawn(fun() -> erl_eval:exprs(element(2, Expr), erl_eval:new_bindings()) end),\n                          ok;\n                    _ -> ok\n                end;\n            \"p\" -> comm:send_local(Pid, {sort_by, pid});\n            \"w\" -> comm:send_local(Pid, {sort_by, mqlen});\n            \"c\" -> comm:send_local(Pid, {sort_by, cpuusage});\n            \"m\" -> comm:send_local(Pid, {sort_by, memory});\n            \"t\" -> comm:send_local(Pid, {toggle_exclude_self});\n            \"i\" -> top_inspect_pid(Pid);\n            \"g\" -> [garbage_collect(X) || X <- processes()];\n            _ -> ok\n    end,\n    case Quit of\n        quit ->\n            comm:send_local(Pid, {output_off}),\n            comm:send_local(Pid, {stop_sampling});\n        _ -> top_interactive(Pid)\n    end.\n\ntop_inspect_pid(Pid) ->\n    %% Read pid and start inspect mode\n    comm:send_local(Pid, {output_off}),\n    comm:send_local(Pid, {stop_sampling}),\n    InspectPid = pid_prompt(),\n    comm:send_local(Pid, {enable_sampling, InspectPid}),\n    comm:send_local(Pid, {sample_pid, InspectPid}),\n    comm:send_local(Pid, {enable_output, pid}),\n    comm:send_local(Pid, {output_pid, InspectPid}),\n    top_inspect_pid_interactive(Pid, InspectPid),\n    comm:send_local(Pid, {stop_sampling}),\n    comm:send_local(Pid, {enable_sampling, all}),\n    comm:send_local(Pid, {sample_all}),\n    comm:send_local(Pid, {enable_output, all}),\n    comm:send_local(Pid, {output_all}).\n\npid_prompt() ->\n    {ok, [InspectPidString]} = io:fread(\"pid or name > \", \"~s\"),\n    try list_to_pid(InspectPidString)\n    catch _:_ ->\n            try\n                InspectPidAtom = list_to_existing_atom(InspectPidString),\n                case whereis(InspectPidAtom) of\n                    undefined ->\n                        case pid_groups:find_all(InspectPidAtom) of\n                            [Pid] -> Pid;\n                            [] -> throw({no_pid, no_pid});\n                            Pids ->\n                                SPids = lists:sort(Pids),\n                                io:format(\"Which of the ~ps do you mean?~n\",\n                                          [InspectPidAtom]),\n                                lists:foldl(\n                                  fun(X, Acc) ->\n                                          io:format(\n                                            \" ~p: ~p ~p~n\",\n                                            [Acc, X,\n                                             pid_groups:group_and_name_of(X)]),\n                                          1 + Acc end, 1, SPids),\n                                {ok, [Nth]} = io:fread(\"num > \", \"~d\"),\n                                lists:nth(Nth, SPids)\n                        end;\n                    Pid -> Pid\n                end\n            catch _:_ ->\n                    io:format(\"No such pid found. Please retry.~n\"),\n                    pid_prompt()\n            end\n    end.\n\ntop_inspect_pid_interactive(TopPid, Pid) ->\n    {ok, [Char]} = io:fread(\"> \", \"~c\"),\n    Quit =\n        case Char of\n            \"q\" -> quit;\n            \"d\" -> %% show dictionary\n                comm:send_local(TopPid, {output_off}),\n                comm:send_local(TopPid, {stop_sampling}),\n                print_process_dictionary(Pid),\n                _ = io:get_chars(\"Hit return to continue> \", 1),\n                comm:send_local(TopPid, {enable_sampling, Pid}),\n                comm:send_local(TopPid, {sample_pid, Pid}),\n                comm:send_local(TopPid, {enable_output, pid}),\n                comm:send_local(TopPid, {output_pid, Pid}),\n                ok;\n            \"m\" -> %% show messages\n                comm:send_local(TopPid, {output_off}),\n                comm:send_local(TopPid, {stop_sampling}),\n                print_process_messages(Pid),\n                _ = io:get_chars(\"Hit return to continue> \", 1),\n                comm:send_local(TopPid, {enable_sampling, Pid}),\n                comm:send_local(TopPid, {sample_pid, Pid}),\n                comm:send_local(TopPid, {enable_output, pid}),\n                comm:send_local(TopPid, {output_pid, Pid}),\n                ok;\n            \"e\" ->\n                comm:send_local(TopPid, {output_off}),\n                Expr = io:parse_erl_exprs('expr>'),\n                comm:send_local(TopPid, {enable_output, pid}),\n                comm:send_local(TopPid, {output_pid, Pid}),\n                case element(1, Expr) of\n                    ok ->\n                        spawn(fun() -> erl_eval:exprs(element(2, Expr), erl_eval:new_bindings()) end),\n                          ok;\n                    _ -> ok\n                end;\n            \"c\" -> comm:send_local(TopPid, {sort_by, cpuusage});\n            \"g\" -> [garbage_collect(X) || X <- processes()];\n            _ -> ok\n        end,\n    case Quit of\n        quit ->\n            comm:send_local(TopPid, {output_off}),\n            comm:send_local(TopPid, {stop_sampling}),\n            ok;\n        _ -> top_inspect_pid_interactive(TopPid, Pid)\n    end.\n\nusage() ->\n    io:format(\n      \"~n\"\n      \" (q)uit, (e)val expr, (g)arb-coll. (i)nspect a pid\"\n      %% \", exclude (t)op itself\" %% undocumented feature for top development\n      \"~n sort by: (c)pu usage (m)emory (p)ids (w)aiting messages~n\").\n\nusage_inspect() ->\n    io:format(\n      \"~n\"\n      \" (q)uit, (e)val expr, (g)arb-coll, (d)ictionary, (m)essages.\"\n      %% \", exclude (t)op itself\" %% undocumented feature for top development\n      \"~n sort by: (c)pu usage~n\").\n\nprint_process_dictionary(Pid) ->\n    Infos = try_process_info(\n              Pid, [dictionary, registered_name]),\n\n    io:format(\"Pid: ~p~n\", [ Pid ]),\n    io:format(\"Name: ~p\",\n              [process_info_get(Infos, registered_name, [])]),\n    case gen_component:is_gen_component(Pid) of\n        true -> io:format(\" ~p\", [readable_grp_and_pid_name(Pid)]);\n        false -> ok\n    end,\n    io:format(\"~n\"),\n    io:format(\"Process Dictionary:~n\"),\n    {ok, Chars} = io:columns(), %% terminal columns\n    CharsForVal = Chars - 21 - 1,\n    io:format(\"~20s ~-\" ++ integer_to_list(CharsForVal) ++ \"s~n\",\n              [\"Key\", \"Value\"]),\n    _ = [ io:format(\"~20s ~-\" ++ integer_to_list(CharsForVal) ++ \"s~n\",\n                    [ lists:flatten(io_lib:format(\"~1210.0p\",\n                                                  [element(1, X)])),\n                      lists:flatten(io_lib:format(\"~111610.0p\",\n                                                  [element(2, X)]))])\n          || X <- process_info_get(Infos, dictionary, [])],\n    ok.\n\nprint_process_messages(Pid) ->\n    Infos = try_process_info(\n              Pid, [messages, registered_name]),\n\n    io:format(\"Pid: ~p~n\", [ Pid ]),\n    io:format(\"Name: ~p\",\n              [process_info_get(Infos, registered_name, [])]),\n    case gen_component:is_gen_component(Pid) of\n        true -> io:format(\" ~p\", [readable_grp_and_pid_name(Pid)]);\n        false -> ok\n    end,\n    io:format(\"~n\"),\n    {ok, Chars} = io:columns(), %% terminal columns\n    {MsgCount, AllMessages0} =\n        lists:foldl(fun(MsgX, {I, TreeX}) ->\n                            % note: don't use comm:get_msg_tag/1 - we want to keep the group_message tags\n                            Tag = erlang:element(1, MsgX),\n                            Size = erlang:byte_size(erlang:term_to_binary(MsgX, [{minor_version, 1}])),\n                            case gb_trees:lookup(Tag, TreeX) of\n                                none ->\n                                    MsgX2 = prettyprint_msg(MsgX),\n                                    {I + 1, gb_trees:insert(Tag, {1, I, I, Size, MsgX2}, TreeX)};\n                                {value, {OldCount, OldFirstPos, _OldLastPos, OldSize, OldMsg}} ->\n                                    {I + 1, gb_trees:update(Tag, {OldCount + 1, OldFirstPos, I, Size + OldSize, OldMsg}, TreeX)}\n                            end\n                    end, {0, gb_trees:empty()}, process_info_get(Infos, messages, [])),\n    AllMessages = lists:reverse(\n                    lists:keysort(1, util:gb_trees_foldl(\n                                    fun(_Tag, {TagCnt, TagFirstPos, TagLastPos, TagSize, Example}, L) ->\n                                            [{TagCnt, TagFirstPos, TagLastPos, TagSize, Example} | L]\n                                    end, [], AllMessages0))),\n    io:format(\"Process Messages (total ~p):~n\", [MsgCount]),\n    CharsForVal = Chars - 7 - 9 - 13 - 1,\n    io:format(\"~6s ~8s ~12s ~-\" ++ integer_to_list(CharsForVal) ++ \"s~n\",\n              [\"Count\", \"Size\", \"Positions\", \"First Message\"]),\n    _ = [begin\n             FirstMessage = lists:flatten(io_lib:format(\"~111610.0p\", [Example])),\n             {FirstMessage1, FirstMessage2} = util:safe_split(CharsForVal, FirstMessage),\n             {AddStr, AddVal} =\n                 case FirstMessage2 of\n                     [] -> {[], []};\n                     _  ->\n                         {FirstMessage2a, FirstMessage2b} = util:safe_split(CharsForVal - 1, FirstMessage2),\n                         case FirstMessage2b of\n                             [] ->\n                                 {\"~29s ~-\" ++ integer_to_list(CharsForVal-1) ++ \"s~n\",\n                                  [\"\", FirstMessage2a]};\n                             _  ->\n                                 {\"~29s ~-\" ++ integer_to_list(CharsForVal-1) ++ \"s~n\"\n                                  \"~29s ~-\" ++ integer_to_list(CharsForVal-1) ++ \"s~n\",\n                                  [\"\", FirstMessage2a, \"\", FirstMessage2b]}\n                         end\n                 end,\n             io:format(\"~6s ~7sk ~12s ~-\" ++ integer_to_list(CharsForVal) ++ \"s~n\" ++ AddStr,\n                       [ lists:flatten(io_lib:format(\"~1210.0p\", [TagCnt])),\n                         lists:flatten(io_lib:format(\"~1210.0p\", [TagSize div 1024])),\n                         lists:flatten(io_lib:format(\"~1210.0p-~1210.0p\", [TagFirstPos, TagLastPos])),\n                         FirstMessage1] ++ AddVal)\n         end\n            || {TagCnt, TagFirstPos, TagLastPos, TagSize, Example} <- AllMessages],\n    ok.\n\n-spec prettyprint_msg(comm:message()) -> comm:message().\nprettyprint_msg({?lookup_aux, Key, Hops, Msg}) ->\n    {util:extint2atom(?lookup_aux), Key, Hops, prettyprint_msg(Msg)};\nprettyprint_msg({?lookup_fin, Key, Hops, Msg}) ->\n    {util:extint2atom(?lookup_fin), Key, Hops, prettyprint_msg(Msg)};\nprettyprint_msg({?send_to_group_member, ProcessName, Msg}) ->\n    {util:extint2atom(?send_to_group_member), ProcessName, prettyprint_msg(Msg)};\nprettyprint_msg({?send_to_registered_proc, ProcessName, Msg}) ->\n    {util:extint2atom(?send_to_registered_proc), ProcessName, prettyprint_msg(Msg)};\nprettyprint_msg(Msg) ->\n    case Msg of\n        {X, f, Y} when is_integer(X) andalso is_tuple(Y) ->\n            %% handle lookup envelope separately\n            {X, f, prettyprint_msg(Y)};\n        _ ->\n            setelement(1, Msg, util:extint2atom(element(1, Msg)))\n    end.\n\n-spec on(comm:message(), state()) -> state().\n\non({enable_sampling, Val}, State) ->\n    case is_pid(Val) of\n        true ->\n            dbg:stop_clear(),\n            dbg:tracer(process, {fun top:trace_fwd/2, self()}),\n            _ = dbg:tpl('_', []),\n            dbg:p(Val, [c, return_to]);\n        false -> ok\n    end,\n    set_sampling(State, Val);\n\non({enable_output, Val}, State)   -> set_output(State, Val);\non({sort_by, Type}, State)        -> set_sort_by(State, Type);\non({toggle_exclude_self}, State)  -> toggle_exclude_self(State);\non({output_off}, State)           -> set_output(State, false);\n\n%% gather information on all local pids for periodic statistical output\non({sample_all}, State) when all =:= element(2, State) ->\n    T = [ catch({X, erlang:process_info(X, [message_queue_len, status])})\n          || X <- erlang:processes(), X =/= exclude_self(State) ],\n\n    TableName = table(State),\n    _ = [ begin\n          Y = pdb:get(P, TableName),\n          case Y of\n              undefined ->\n                  V1 = pdbe_new(P, NewVals),\n                  pdb:set(V1, TableName);\n              _ ->\n%%                  io:format(\"X:~p Y:~p~n\", [X, Y]),\n                  V1 = pdbe_add_vals(Y, NewVals),\n                  pdb:set(V1, TableName)\n          end\n      end || {P, NewVals} <- T, is_pid(P), undefined =/= NewVals],\n    _ = comm:send_local_after(1, self(), {sample_all}),\n    inc_counter(State);\n\non({sample_all}, State) -> State;\n\n%% gather information on a single pid for periodic statistical output\non({sample_pid, Pid}, State) when is_pid(element(2, State)) ->\n    TableName = table(State),\n    NewVals = case try_process_info(\n                     Pid,\n                     [message_queue_len, status]) of\n                  undefined -> [];\n                  Infos -> Infos\n              end,\n    TakeVals = case process_info_get(NewVals, status, dead_pid) of\n                   running -> true;\n                   runnable -> true;\n                   _ -> false\n               end,\n    %% _ = comm:send_local_after(1, self(), {sample_pid, Pid}),\n    comm:send_local(self(), {sample_pid, Pid}),\n    case TakeVals of\n        true ->\n            %P = {Pid, process_info_get(NewVals, current_function, dead)},\n            P = {Pid, erlang:get(current_function)},\n            Entry = pdb:get(P, TableName),\n            case Entry of\n                undefined ->\n                    V1 = pdbe_new(P, NewVals),\n                    pdb:set(V1, TableName);\n                _ ->\n                    %% io:format(\"X:~p Y:~p~n\", [X, Y]),\n                    V1 = pdbe_add_vals(Entry, NewVals),\n                    pdb:set(V1, TableName)\n            end;\n        false ->\n            P = {Pid, not_running},\n            Entry = pdb:get(P, TableName),\n            TVals = lists:keyreplace(status, 1,\n                                     NewVals, {status, runnable}),\n            case Entry of\n                undefined ->\n                    V1 = pdbe_new(P, TVals),\n                    pdb:set(V1, TableName);\n                _ ->\n                    V1 = pdbe_add_vals(Entry, TVals),\n                    pdb:set(V1, TableName)\n            end\n    end,\n    inc_counter(State);\n\non({sample_pid, _Pid}, State) -> State;\n\non({trace, _Pid, Fun}, State) ->\n    erlang:put(current_function, Fun),\n    State;\n\non({output_all}, State) when all =:= element(3, State) ->\n    TableName = table(State),\n    Now = os:timestamp(),\n    Delay = timer:now_diff(Now, timestamp(State)) / 1000000,\n    Count = counter(State),\n    ProcsMem = erlang:memory(processes),\n    ProcsMemUsed = erlang:memory(processes_used),\n    R1 = pdb:tab2list(TableName),\n    PlotData = [ begin\n                     Pid = element(1, X),\n                     pdb:delete(Pid, TableName),\n                     %% plot data is:\n                     Infos = try_process_info(Pid, [status, memory, stack_size]),\n                     Status = process_info_get(Infos, status, dead_pid),\n                     {vals_pid(X),\n                      Status,\n                      vals_mqlen(X)/Count,\n                      vals_cpu_usage(X, Count),\n                      process_info_get(Infos, memory, 0),\n                      process_info_get(Infos, stack_size, 0)}\n                 end || X <- R1, is_tuple(X), is_pid(element(1, X)) ],\n    SortedPlotData = lists:keysort(sort_by(State), PlotData),\n    L = length(SortedPlotData),\n    {ok, Lines} = io:rows(), %% terminal rows\n    {_, PlotThese} =\n        lists:split(erlang:max(0, L - (Lines - 9)), SortedPlotData),\n    Time = element(2, calendar:local_time()),\n    WallClock = element(1, erlang:statistics(wall_clock)) div 60000,\n    io:format(\"top - ~2..0B:~2..0B:~2..0B up ~B days, ~B:~2..0B, load average: ~.2f~n\",\n             [element(1, Time), element(2, Time), element(3, Time),\n              WallClock div (60*24),\n              WallClock rem (60*24) div 60,\n              (WallClock rem (60*60)) rem 60,\n              lists:foldl(fun(X, Acc) -> element(4, X)/100 + Acc end,\n                          0, PlotData)\n             ]),\n    io:format(\"Tasks: ~p total, ~p running, ~p sleeping, ~p garb-coll.~n\",\n             [L,\n              lists:foldl(fun(X, Acc) -> case element(2, X) of\n                                             runnable -> 1 + Acc;\n                                             running -> 1 + Acc;\n                                             _ -> Acc\n                                         end end, 0, PlotData),\n              lists:foldl(fun(X, Acc) ->\n                                  case element(2, X) of\n                                      waiting -> 1 + Acc;\n                                      suspended -> 1 + Acc;\n                                      exiting -> 1 + Acc;\n                                      dead_pid -> 1 + Acc;\n                                      _ -> Acc\n                                  end end, 0, PlotData),\n              lists:foldl(fun(X, Acc) ->\n                                  case element(2, X) of\n                                      garbage_collecting -> 1 + Acc;\n                                      _ -> Acc\n                                  end end, 0, PlotData)\n              ]),\n    {{_, IOIn}, {_, IOOut}} = erlang:statistics(io),\n    %% {GCNum, GCFreed, _} = erlang:statistics(garbage_collection),\n    %% io:format(\"Context Switches: ~p Garb.Coll: ~p ~s~n\",\n    %%           [element(1, erlang:statistics(context_switches)),\n    %%            GCNum, readable_mem(GCFreed)]),\n\n    io:format(\"IO: ~s/~s in/out Mem: ~s procs, ~s atom, ~s binary, ~s ets~n\",\n              [readable_mem(IOIn), readable_mem(IOOut),\n               readable_mem(ProcsMem),\n               readable_mem(erlang:memory(atom)),\n               readable_mem(erlang:memory(binary)),\n               readable_mem(erlang:memory(ets))\n              ]),\n    io:format(\"FPS: ~.1f~n\", [Count / Delay]),\n    WordSize = erlang:system_info(wordsize),\n    io:format(\"~12s ~1s ~1s ~-20s ~9s ~6s ~6s ~5s ~5s~n\",\n              [ \"PID\", \"T\", \"S\", \"NAME\", \"W-Msg\", \"%CPU\", \"%MEM\", \"MEM\", \"STACK\" ]),\n    _ = [ begin\n          Type = case catch(gen_component:is_gen_component(Pid)) of\n                     true -> \"G\";\n                     _ -> \"E\"\n                 end,\n          io:format(\"~12w ~1s ~1s ~-20s ~9.2f ~6.2f ~6.2f ~5s ~5s~n\",\n                    [Pid, Type, readable_status(Status),\n                     readable_pid_name(Pid), MQLen, CPUUsage,\n                     MemUsage / ProcsMemUsed * 100,\n                     readable_mem(MemUsage),\n                     readable_mem(WordSize * StackSize)])\n      end\n      || {Pid, Status, MQLen, CPUUsage, MemUsage, StackSize} <- lists:reverse(PlotThese)],\n    usage(),\n    msg_delay:send_trigger(2, {output_all}),\n    S1 = set_counter(State, 1),\n    set_timestamp(S1, os:timestamp());\n\non({output_all}, State) -> State;\n\non({output_pid, Pid}, State) when pid =:= element(3, State) ->\n    TableName = table(State),\n    Now = os:timestamp(),\n    Delay = timer:now_diff(Now, timestamp(State)) / 1000000,\n    Count = counter(State),\n    Infos = try_process_info(\n              Pid, [dictionary, registered_name, memory, total_heap_size,\n                    stack_size, status, message_queue_len, current_function]),\n    PD = process_info_get(Infos, dictionary, []),\n    R1 = pdb:tab2list(TableName),\n    PlotData = [ begin\n                     pdb:delete(PidAndCFun, TableName),\n                     %% plot data is:\n                     {PidAndCFun,\n                      no_status,\n                      no_mqlen,\n                      vals_cpu_usage(X, Count),\n                      no_mem}\n                 end || {PidAndCFun, MQLen, Status} = X <- R1,\n                        is_tuple(Status), is_number(MQLen)\n               ],\n    SortedPlotData = lists:keysort(sort_by(State), PlotData),\n    L = length(SortedPlotData),\n    {ok, Lines} = io:rows(), %% terminal rows\n    SkipLines = 11,\n    {_, PlotThese} =\n        lists:split(erlang:max(0, L - (Lines - SkipLines)), SortedPlotData),\n\n    Time = element(2, calendar:local_time()),\n    WallClock = element(1, erlang:statistics(wall_clock)) div 60000,\n    io:format(\"top - ~2..0B:~2..0B:~2..0B up ~B days, ~B:~2..0B~n\",\n              [element(1, Time), element(2, Time), element(3, Time),\n               WallClock div (60*24),\n               WallClock rem (60*24) div 60,\n               (WallClock rem (60*60)) rem 60\n              ]),\n    IdlePlotDataEntry = lists:keyfind({Pid, not_running}, 1, PlotData),\n    Idle = case IdlePlotDataEntry of\n        false -> 0.0;\n        _ -> element(4, IdlePlotDataEntry)\n    end,\n    io:format(\"Pid: ~p, ~6.2f% run, ~6.2f% idle~n\",\n              [ Pid, 100.0 - Idle, Idle ]),\n    io:format(\"Name: ~p\",\n              [process_info_get(Infos, registered_name, [])]),\n    case gen_component:is_gen_component(Pid) of\n        true -> io:format(\" ~p\", [readable_grp_and_pid_name(Pid)]);\n        false -> ok\n    end,\n    io:format(\"~n\"),\n    MemUsage = process_info_get(Infos, memory, 0),\n    HeapSize = process_info_get(Infos, total_heap_size, 0),\n    StackSize = process_info_get(Infos, stack_size, 0),\n    PDSize = erlang:byte_size(erlang:term_to_binary(PD, [{minor_version, 1}])),\n    WordSize = erlang:system_info(wordsize),\n    io:format(\"Mem: ~s total, ~s heap, ~s stack, ~s dictionary (~p entries)~n\",\n              [readable_mem(MemUsage),\n               readable_mem((HeapSize - StackSize) * WordSize),\n               readable_mem(StackSize * WordSize),\n               readable_mem(PDSize), length(PD)\n              ]),\n\n    io:format(\"Status: ~s MQLen: ~p Current Function: ~p~n\",\n              [readable_status(\n                 process_info_get(Infos, status, dead_pid)),\n               process_info_get(Infos, message_queue_len, 0),\n               erlang:get(current_function)\n              ]),\n    io:format(\"FPS: ~.1f~n\", [Count / Delay]),\n\n    io:format(\"~12s ~-35s ~6s~n\",\n              [ \"PID\", \"FUNCTION\", \"%RTIME\"]),\n    _ = [ begin\n              CFun = element(2, PlotPid),\n              CFunString =\n                  case is_tuple(CFun) of\n                      true ->\n                          io_lib:write(element(1, CFun))\n                              ++ \":\" ++ io_lib:write(element(2, CFun))\n                              ++ \"/\" ++ io_lib:write(element(3, CFun));\n                      false -> io_lib:write(CFun)\n                  end,\n              FunRTime = if 0.004 > (100.0 - Idle) ->\n                                 case L > 1 of\n                                     true -> 100.0 / (L-1);\n                                     false -> 100.0\n                                 end;\n                            true -> CPUUsage * 100.0 / (100.0 - Idle)\n                         end,\n              io:format(\"~12w ~-35s ~6.2f~n\",\n                        [element(1, PlotPid), CFunString, FunRTime ])\n          end\n          || {PlotPid, _Status, _MQLen, CPUUsage, _MemUsage}\n                 <- lists:reverse(PlotThese), not_running =/= element(2, PlotPid)],\n    util:for_to(1, Lines - SkipLines - length(PlotThese),\n                fun(_) -> io:format(\"~n\") end),\n    usage_inspect(),\n    msg_delay:send_local(4, self(), {output_pid, Pid}),\n    S1 = set_counter(State, 0),\n    set_timestamp(S1, os:timestamp());\n\non({output_pid, _Pid}, State) -> State;\n\non({stop_sampling}, State) ->\n    TableName = table(State),\n    R1 = pdb:tab2list(TableName),\n    _ = [ pdb:delete(element(1, X), TableName)\n          || {_,_,_} = X <- R1, is_pid(element(1, X)) ],\n    S1 = set_counter(State, 0),\n    dbg:stop_clear(),\n    set_sampling(S1, false).\n\n\ntable(State) -> element(1, State).\nset_sampling(State, Val) -> setelement(2, State, Val).\nset_output(State, Val) -> setelement(3, State, Val).\ntimestamp(State) -> element(4, State).\nset_timestamp(State, Val) -> setelement(4, State, Val).\ncounter(State) -> element(5, State).\ninc_counter(State) -> setelement(5, State, 1 + element(5, State)).\nset_counter(State, Val) -> setelement(5, State, Val).\nsort_by(State) -> element(6, State).\nset_sort_by(State, Type) ->\n    Val = case Type of\n              pid -> 1;\n              mqlen -> 3;\n              cpuusage -> 4;\n              memory -> 5\n          end,\n    setelement(6, State, Val).\nexclude_self(State) -> element(7, State).\ntoggle_exclude_self(State) ->\n    Self = self(),\n    case element(7, State) of\n        Self -> setelement(7, State, no_exclude);\n        _ -> setelement(7, State, self())\n    end.\n\nvals_pid(Entry) -> element(1, Entry).\nvals_mqlen(Entry) -> element(2, Entry).\n\nvals_cpu_usage(Entry, Count) ->\n    Status = element(3, Entry),\n    Runs = element(6, Status),\n    Runnable = element(4, Status),\n    case Count of\n        0 -> 0;\n        _ -> (Runs + Runnable) / Count * 100\n    end.\n\nvals_add_mqlen(Entry, Amount) ->\n    setelement(2, Entry, Amount + element(2, Entry)).\nvals_add_status(Entry, Val) ->\n    OldStatus = element(3, Entry),\n    NewStatus =\n        case Val of\n            exiting ->\n                setelement(1, OldStatus, 1 + element(1, OldStatus));\n            garbage_collecting ->\n                setelement(2, OldStatus, 1 + element(2, OldStatus));\n            waiting            ->\n                setelement(3, OldStatus, 1 + element(3, OldStatus));\n            runnable           ->\n                setelement(4, OldStatus, 1 + element(4, OldStatus));\n            suspended          ->\n                setelement(5, OldStatus, 1 + element(5, OldStatus));\n            running            ->\n                setelement(6, OldStatus, 1 + element(6, OldStatus));\n            dead_pid           ->\n                setelement(7, OldStatus, 1 + element(7, OldStatus))\n        end,\n    setelement(3, Entry, NewStatus).\n\npdbe_new(Pid, Vals) ->\n    Entry = {Pid,\n             process_info_get(Vals, message_queue_len, 0),\n             {_E = 0,_G = 0, _W = 0, _I = 0, _S = 0, _R= 0, _D = 0} %% status\n            },\n    vals_add_status(Entry, process_info_get(Vals, status, dead_pid)).\n\n\npdbe_add_vals(Entry, Vals) ->\n    MQLen = process_info_get(Vals, message_queue_len, 0),\n    V1 = case MQLen > 1 of\n             false -> Entry;\n             true -> vals_add_mqlen(Entry, MQLen)\n         end,\n    Status = process_info_get(Vals, status, dead_pid),\n    _V2 = vals_add_status(V1, Status).\n\nreadable_status(Status) ->\n    case Status of\n        exiting -> \"E\";\n        garbage_collecting -> \"G\";\n        waiting -> \"W\";\n        runnable -> \"R\";\n        suspended -> \"S\";\n        running -> \"R\";\n        dead_pid -> \"D\"\n    end.\n\nreadable_mem(0) -> \"0B\";\nreadable_mem(Mem) ->\n    case math:log(Mem)/math:log(2) of\n        L when L < 10.0 ->\n            lists:flatten(io_lib:format(\"~p\", [Mem]))\n            ++ \"B\";\n        L when L < 20.0 ->\n            lists:flatten(io_lib:format(\"~p\", [Mem div 1024]))\n            ++ \"KB\";\n        L when L < 30.0 ->\n            lists:flatten(io_lib:format(\"~p\", [Mem div 1024 div 1024]))\n            ++ \"MB\";\n        L when L >= 30.0 ->\n            lists:flatten(\n              io_lib:format(\n                \"~p\", [Mem div 1024 div 1024 div 1024]))\n            ++ \"GB\"\n    end.\n\nreadable_pid_name(Pid) ->\n    Name = case pid_groups:group_and_name_of(Pid) of\n               failed ->\n                   try element(2, process_info(Pid, registered_name))\n                   catch _Level:_Reason ->\n                           try_process_info(Pid, initial_call, dead_pid)\n                   end;\n               N -> element(2, N)\n           end,\n    case is_list(Name) of\n        false -> io_lib:write(Name);\n        true -> Name\n    end.\n\nreadable_grp_and_pid_name(Pid) ->\n    case pid_groups:group_and_name_of(Pid) of\n        failed ->\n            try element(2, process_info(Pid, registered_name))\n            catch _Level:_Reason ->\n                    try_process_info(Pid, initial_call, dead_pid)\n            end;\n        N -> N\n    end.\n\ntry_process_info(Pid, Types) ->\n    try erlang:process_info(Pid, Types)\n    catch _:_ -> []\n    end.\n\ntry_process_info(Pid, Type, DefaultVal) ->\n    try element(2, erlang:process_info(Pid, Type))\n    catch _:_ -> DefaultVal\n    end.\n\nprocess_info_get(undefined, _Type, DefaultVal) ->\n    DefaultVal;\nprocess_info_get(PropList, Type, DefaultVal) ->\n    case lists:keyfind(Type, 1, PropList) of\n        false -> DefaultVal;\n        X -> element(2, X)\n    end.\n\n-spec trace_fwd({trace, pid(), call | return_to, mfa()}, pid()) -> pid().\ntrace_fwd({trace, Pid, call, {Mod, Fun, Params}}, TopPid) ->\n    TopPid ! {trace, Pid, {Mod, Fun, length(Params)}},\n    TopPid;\ntrace_fwd({trace, Pid, return_to, {Mod, Fun, Arity}}, TopPid) ->\n    TopPid ! {trace, Pid, {Mod, Fun, Arity}},\n    TopPid.\n\n"
  },
  {
    "path": "src/trace_mpath.erl",
    "content": "% @copyright 2012-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Trace what a message triggers in the system by tracing all\n%% generated subsequent messages.\n%% @version $Id$\n-module(trace_mpath).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%\n%% 1. call trace_mpath:start(your_trace_id)\n%% 2. perform a request like api_tx:read(\"a\")\n%% 3. call trace_mpath:stop() %% trace_id is taken from the calling\n%%                               process implicitly\n%% 4. call trace_mpath:get_trace(your_trace_id) to retrieve the trace,\n%%    when you think everything is recorded\n%% 5. call trace_mpath:cleanup(your_trace_id) to free the memory\n%%\n%% Optional Startup for reduced memory consumption:\n%%   1.1 Use message map fun to normalize or shrink messages\n%%       call: trace_mpath:start(your_trace_id, [{map_fun, your_map_fun()}])\n%%   1.2 Use filter fun to log only messages your are interested in\n%%       call: trace_mpath:start(your_trace_id, [{filter_fun, your_filter_fun()}])\n%%       attention: filter_fun operates on mapped messages if map_fun() is used\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-include(\"scalaris.hrl\").\n-behaviour(gen_component).\n\n%% client functions\n-export([start/0, start/1, start/2, stop/0]).\n-export([infected/0]).\n-export([clear_infection/0, restore_infection/0]).\n-export([get_trace/0, get_trace/1, get_trace/2,\n         get_trace_raw/1, get_trace_raw/2,\n         cleanup/0, cleanup/1]).\n-export([thread_yield/0]).\n\n%% trace analysis\n-export([send_histogram/1]).\n-export([time_delta/1]).\n-export([to_texfile/2, to_texfile/3,\n         to_texfile_no_time/2, to_texfile_no_time/3]).\n\n%% report tracing events from other modules\n-export([log_send/6]).\n-export([log_info/2, log_info/3]).\n-export([log_recv/4]).\n-export([epidemic_reply_msg/4]).\n-export([get_msg_tag/1]).\n\n%% useful in e.g. filter funs\n-export([normalize_pidinfo/1]).\n\n%% gen_component behaviour\n-export([start_link/1, init/1]).\n-export([on/2]). %% internal message handler as gen_component\n\n-type time()         :: erlang_timestamp() | non_neg_integer().\n-type logger()       :: io_format                       %% | ctpal\n                      | {log_collector, comm:mypid()}\n                      | {proto_sched, comm:mypid()}.\n-type pidinfo()      :: {comm:mypid(),\n                         {pid_groups:groupname(), pid_groups:pidname()} |\n                             no_pid_name |\n                             non_local_pid_name_unknown}.\n-type anypid()       :: pid() | comm:mypid().\n-type trace_id()     :: atom().\n-type send_event0()  :: {log_send, time(), trace_id(), %internal\n                         Source::anypid(), Dest::anypid(), comm:message(),\n                         local | global}.\n-type send_event()   :: {log_send, time(), trace_id(),\n                         Source::pidinfo(), Dest::pidinfo(), comm:message(),\n                         local | global}.\n-type info_event0()  :: {log_info, time(), trace_id(), %internal\n                         anypid(), comm:message()}.\n-type info_event()   :: {log_info, time(), trace_id(),\n                         pidinfo(), comm:message()}.\n-type recv_event0()  :: {log_recv, time(), trace_id(), %internal\n                         Source::anypid(), Dest::anypid(), comm:message()}.\n-type recv_event()   :: {log_recv, time(), trace_id(),\n                         Source::pidinfo(), Dest::pidinfo(), comm:message()}.\n-type trace_event0() :: send_event0() | info_event0() | recv_event0(). %internal\n-type trace_event()  :: send_event() | info_event() | recv_event().\n-type trace()        :: [trace_event()].\n-type msg_map_fun()  :: fun((comm:message(), Source::pid() | comm:mypid(),\n                             Dest::pid() | comm:mypid()) -> comm:message()).\n-type filter_fun()   :: fun((trace_event()) -> boolean()).\n-type passed_state1():: {trace_id(), logger(), msg_map_fun(), filter_fun()}.\n-type passed_state() :: passed_state1()\n                        | {trace_id(), logger()}.\n-type gc_mpath_msg() :: {'$gen_component', trace_mpath, passed_state(),\n                         Source::anypid(), Dest::anypid(), comm:message()}.\n-type options()      :: [{logger, logger() | comm:mypid()} |\n                         {map_fun, msg_map_fun()} |\n                         {filter_fun, filter_fun()}].\n\n-export_type([trace/0, trace_event/0]).\n-export_type([pidinfo/0, logger/0, passed_state/0]).\n\n-include(\"gen_component.hrl\").\n\n-spec start() -> ok.\nstart() -> start(default).\n\n-spec start(trace_id() | passed_state()) -> ok.\nstart(TraceId) when is_atom(TraceId) ->\n    start(TraceId, []);\nstart({TraceId, Logger} = _PState) ->\n    proto_sched:start(TraceId, Logger);\nstart({TraceId, Logger, MsgMapFun, FilterFun} = _PState) ->\n    start(TraceId, Logger, MsgMapFun, FilterFun).\n\n-spec start(trace_id(), logger() | comm:mypid() | options()) -> ok.\nstart(TraceId, Options) when is_list(Options) ->\n    InLogger = proplists:get_value(logger, Options, nil),\n    Logger = if InLogger =:= nil ->\n                    LoggerPid = pid_groups:find_a(?MODULE),\n                    comm:make_global(LoggerPid);\n                true -> InLogger\n             end,\n    MsgFun = proplists:get_value(map_fun, Options, fun(Msg, _Src, _Dest) -> Msg end),\n    FilterFun = proplists:get_value(filter_fun, Options, fun(_) -> true end),\n    start(TraceId, Logger, MsgFun, FilterFun);\nstart(TraceId, Logger) ->\n    start(TraceId, [{logger, Logger}]).\n\n-spec start(trace_id(), logger() | comm:mypid(), msg_map_fun(), filter_fun()) -> ok.\nstart(TraceId, InLogger, MsgMapFun, FilterFun) ->\n    Logger = case comm:is_valid(InLogger) of\n                 true -> {log_collector, InLogger}; %% just a pid was given\n                 false -> InLogger\n             end,\n    PState = passed_state_new(TraceId, Logger, MsgMapFun, FilterFun),\n    own_passed_state_put(PState).\n\n-spec stop() -> ok.\nstop() ->\n    % assert infection when stop/0 is called\n    ?DBG_ASSERT(erlang:get(trace_mpath) =/= undefined),\n    %% stop sending epidemic messages\n    erlang:erase(trace_mpath),\n    ok.\n\n-spec infected() -> boolean().\ninfected() ->\n    case erlang:get(trace_mpath) of\n        {_TraceId, _Logger} -> true;\n        {_TraceId, _Logger, _MsgMapFun, _FilterFun} -> true;\n        _                   -> false\n    end.\n\n-spec clear_infection() -> ok.\nclear_infection() ->\n    case erlang:erase(trace_mpath) of\n        undefined ->\n            %% remove _bak entry as otherwise an old _bak may be\n            %% restored in restore_infection\n            erlang:erase(trace_mpath_bak);\n        PState    ->\n            erlang:put(trace_mpath_bak, PState)\n    end,\n    ok.\n\n-spec restore_infection() -> ok.\nrestore_infection() ->\n    case erlang:get(trace_mpath_bak) of\n        undefined -> ok;\n        PState    -> erlang:put(trace_mpath, PState),\n                     ok\n    end.\n\n-spec get_trace() -> trace().\nget_trace() -> get_trace(default).\n\n-spec get_trace(trace_id()) -> trace().\nget_trace(TraceId) ->\n    get_trace(TraceId, none).\n\n-spec get_trace(trace_id(), Option::cleanup | none) -> trace().\nget_trace(TraceId, Option) ->\n    LogRaw = get_trace_raw(TraceId, Option),\n    Trace =\n        [case Event of\n             {log_send, Time, TraceId, Source, Dest, {Tag, Key, Hops, Msg}, LorG}\n               when Tag =:= ?lookup_aux orelse Tag =:= ?lookup_fin ->\n                 {log_send, Time, TraceId,\n                  normalize_pidinfo(Source),\n                  normalize_pidinfo(Dest),\n                  convert_msg({Tag, Key, Hops, convert_msg(Msg)}), LorG};\n             {log_send, Time, TraceId, Source, Dest, Msg, LorG} ->\n                 {log_send, Time, TraceId,\n                  normalize_pidinfo(Source),\n                  normalize_pidinfo(Dest), convert_msg(Msg), LorG};\n             {log_recv, Time, TraceId, Source, Dest, {Tag, Key, Hops, Msg}}\n               when Tag =:= ?lookup_aux orelse Tag =:= ?lookup_fin ->\n                 {log_recv, Time, TraceId,\n                  normalize_pidinfo(Source),\n                  normalize_pidinfo(Dest),\n                  convert_msg({Tag, Key, Hops, convert_msg(Msg)})};\n             {log_recv, Time, TraceId, Source, Dest, Msg} ->\n                 {log_recv, Time, TraceId,\n                  normalize_pidinfo(Source),\n                  normalize_pidinfo(Dest), convert_msg(Msg)};\n             {log_info, Time, TraceId, Pid, Msg} ->\n                 case Msg of\n                     {gc_on_done, Tag} ->\n                         {log_info, Time, TraceId,\n                          normalize_pidinfo(Pid),\n                          {gc_on_done, util:extint2atom(Tag)}};\n                     _ ->\n                         {log_info, Time, TraceId,\n                          normalize_pidinfo(Pid), convert_msg(Msg)}\n                 end\n         end || Event <- LogRaw],\n    resolve_remote_pids(Trace).\n\n-spec resolve_remote_pids(Trace::trace()) -> trace().\nresolve_remote_pids(Trace) ->\n    % remote pids are present in two forms (with each of them present!):\n    % (a) fully resolved pidinfo() and (b) non-resolved pidinfo()\n    % -> find (a) and replace (b):\n    DictResolved =\n        dict:from_list(\n          lists:flatmap(\n            fun(Event) ->\n                    Pid1 = element(4, Event),\n                    Pid2 = ?IIF(element(1, Event) =:= log_info, unknown, element(5, Event)),\n                    % known pids\n                    [{PidX, X} || X = {PidX, InfoX} <- [Pid1, Pid2], is_tuple(InfoX)]\n            end, Trace)),\n    [begin\n         Pid1 = element(4, Event),\n         Pid2 = ?IIF(element(1, Event) =:= log_info, unknown, element(5, Event)),\n         lists:foldl(fun({I, PidX}, EventX) ->\n                             case dict:find(PidX, DictResolved) of\n                                 error -> EventX;\n                                 {ok, Res} -> setelement(I, EventX, Res)\n                             end\n                     end, Event,\n                     % unknown pids:\n                     [{I, PidX} || {I, {PidX, InfoX}} <- [{4, Pid1}, {5, Pid2}],\n                                   is_atom(InfoX)])\n     end || Event <- Trace].\n\n-spec convert_msg(Msg::comm:message()) -> comm:message().\nconvert_msg(Msg) when is_tuple(Msg)\n                      andalso tuple_size(Msg) =:= 3\n                      andalso f =:= element(2, Msg) ->\n    %% lookup envelope\n    setelement(1, Msg, util:extint2atom(element(1,element(3, Msg))));\nconvert_msg(Msg) when is_tuple(Msg) andalso tuple_size(Msg) >= 1 ->\n    setelement(1, Msg, util:extint2atom(element(1, Msg)));\nconvert_msg(Msg) -> Msg.\n\n-spec get_trace_raw(trace_id()) -> trace().\nget_trace_raw(TraceId) ->\n    get_trace_raw(TraceId, none).\n\n-spec get_trace_raw(trace_id(), Option::cleanup | none) -> trace().\nget_trace_raw(TraceId, Option) ->\n    LoggerPid = pid_groups:find_a(trace_mpath),\n    comm:send_local(LoggerPid, {get_trace, comm:this(), TraceId, Option}),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({get_trace_reply, Log}, Log)\n    end.\n\n-spec cleanup() -> ok.\ncleanup() -> cleanup(default).\n\n-spec cleanup(trace_id()) -> ok.\ncleanup(TraceId) ->\n    LoggerPid = pid_groups:find_a(trace_mpath),\n    comm:send_local(LoggerPid, {cleanup, TraceId}),\n    ok.\n\n%% for proto_sched and to mark end of client processing before\n%% receives\n-spec thread_yield() -> ok.\nthread_yield() ->\n    %% report done for proto_sched to go on...\n    log_info(self(), {gc_on_done, scalaris_recv}).\n    %% clear_infection().\n    %% we need to remain infected for receive after N clauses.\n\n\n%% Functions for trace analysis\n-spec send_histogram(trace()) -> list().\nsend_histogram(Trace) ->\n    %% only send events\n    Sends = [ X || X <- Trace, element(1, X) =:= log_send],\n    %% only message tags\n    Tags = [ element(1,element(6,X)) || X <- Sends],\n    SortedTags = lists:sort(Tags),\n    %% reduce tags\n    CountedTags = lists:foldl(fun(X, Acc) ->\n                                      case Acc of\n                                          [] -> [{X, 1}];\n                                          [{Y, Count} | Tail] ->\n                                              case X =:= Y of\n                                                  true ->\n                                                      [{Y, Count + 1} | Tail];\n                                                  false ->\n                                                      [{X, 1}, {Y, Count} | Tail]\n                                              end\n                                      end\n                              end,\n                              [], SortedTags),\n    lists:reverse(lists:keysort(2, CountedTags)).\n\n-spec time_delta(trace()) -> trace().\ntime_delta(Trace) ->\n    SortedTrace = lists:keysort(2, Trace),\n    StartTime = element(2, hd(SortedTrace)),\n    [ setelement(2, X, timer:now_diff(element(2, X), StartTime))\n      || X <- SortedTrace].\n\n-spec notime_delta(trace()) -> trace().\nnotime_delta(Trace) ->\n    SortedTrace = lists:keysort(2, Trace),\n    util:map_with_nr(fun(X, I) -> setelement(2, X, I*10) end, SortedTrace, 0).\n\n%% sample call sequence to get a tx trace:\n%% tex traces have to be relatively short, so paste all at once to the\n%% erlang shell.\n%% api_tx:write(\"b\", 1). trace_mpath:start(). api_tx:write(\"a\", 1). trace_mpath:stop(). timer:sleep(10). T = trace_mpath:get_trace(). trace_mpath:to_texfile(T, \"trace.tex\").\n%%  P = pid_groups:find_a(trace_mpath). gen_component:bp_set_cond(P, fun(_,_) -> true end, mybp). api_tx:write(\"b\", 1). trace_mpath:start(). api_tx:write(\"a\", 1). trace_mpath:stop(). timer:sleep(1). gen_component:bp_del(P, mybp). gen_component:bp_cont(P). timer:sleep(10).  T = trace_mpath:get_trace(). trace_mpath:to_texfile(T, \"trace.tex\").\n%% P = pid_groups:find_a(trace_mpath). gen_component:bp_set_cond(P, fun(_,_)-> true end, mybp). rbrcseq:qread(self(), \"b\"). receive _ -> ok end. trace_mpath:start(). rbrcseq:qread(self(), \"a\"). receive _ -> ok end. trace_mpath:stop(). gen_component:bp_del(P, mybp). gen_component:bp_cont(P). T = trace_mpath:get_trace(). trace_mpath:to_texfile(T, \"trace.tex\").\n\n%% @doc Write the trace to a LaTeX file (20 microseconds in the trace are 1 cm\n%%      in the plot).\n-spec to_texfile(trace(), file:name())\n        -> ok | {error, file:posix() | badarg | terminated}.\nto_texfile(Trace, Filename) ->\n    to_texfile(Trace, Filename, 20).\n\n%% @doc Write the trace to a LaTeX file (ScaleX microseconds in the trace are 1\n%%      cm in the plot).\n-spec to_texfile(trace(), file:name(), ScaleX::pos_integer())\n        -> ok | {error, file:posix() | badarg | terminated}.\nto_texfile(Trace, Filename, ScaleX) ->\n    to_texfile(Trace, Filename, fun time_delta/1, true, ScaleX).\n\n%% @doc Write the trace to a LaTeX file (20 microseconds in the trace are 1 cm\n%%      in the plot). The representation will not have a time representation\n%%      but rather only show successive messages.\n-spec to_texfile_no_time(trace(), file:name())\n        -> ok | {error, file:posix() | badarg | terminated}.\nto_texfile_no_time(Trace, Filename) ->\n    to_texfile_no_time(Trace, Filename, 20).\n\n%% @doc Write the trace to a LaTeX file (ScaleX microseconds in the trace are 1\n%%      cm in the plot). The representation will not have a time representation\n%%      but rather only show successive messages.\n-spec to_texfile_no_time(trace(), file:name(), ScaleX::pos_integer())\n        -> ok | {error, file:posix() | badarg | terminated}.\nto_texfile_no_time(Trace, Filename, ScaleX) ->\n    %% we do not need the gc_on_done messages\n    F = fun(X) ->\n                case X of\n                    {log_info, _TimeStamp, _TraceId, _From, {gc_on_done, _MsgTag}} -> false;\n                    _ -> true\n                end\n        end,\n    FilteredTrace = [X || X <- Trace, F(X)],\n    to_texfile(FilteredTrace, Filename, fun notime_delta/1, false, ScaleX).\n\n%% @doc Write the trace to a LaTeX file (ScaleX microseconds in the trace are\n%%      1 cm in the plot).\n-spec to_texfile(trace(), file:name(), DeltaFun::fun((trace()) -> trace()),\n                 HaveRealTime::boolean(), ScaleX::pos_integer())\n        -> ok | {error, file:posix() | badarg | terminated}.\nto_texfile(Trace, Filename, DeltaFun, HaveRealTime, ScaleX0) ->\n    {ok, File} = file:open(Filename, [write]),\n    io:format(File,\n      \"\\\\documentclass[10pt]{article}~n\"\n      \"\\\\usepackage[paperwidth=\\\\maxdimen,paperheight=\\\\maxdimen]{geometry}~n\"\n      \"\\\\usepackage{tikz}~n\"\n      \"\\\\usetikzlibrary{arrows}~n\"\n      \"\\\\usepackage[T1]{fontenc}~n\"\n      \"\\\\usepackage{lmodern}~n\"\n      \"\\\\usepackage[tightpage,active]{preview}~n\"\n      \"\\\\PreviewEnvironment{tikzpicture}~n\"\n      \"\\\\begin{document}~n\"\n      \"\\\\makeatletter~n\"\n      \"\\\\renewcommand{\\\\tiny}{\\\\@setfontsize\\\\miniscule{4}{5}}~n\"\n      \"\\\\makeatother~n\"\n      \"\\\\pagestyle{empty}\\\\sf\\\\scriptsize\"\n      \"\\\\begin{tikzpicture}~n\",\n              []),\n    %% create nodes for processes:\n    %% all processes in that order, but only once\n\n    NodesR = lists:foldl(\n               fun(X, Acc) ->\n                       Pid = case element(1, X) of\n                                 log_send -> element(4, X);\n                                 log_recv -> element(5, X);\n                                 log_info -> element(4, X)\n                             end,\n                       case lists:member(Pid, Acc) of\n                           true -> Acc;\n                           false -> [ Pid | Acc ]\n                       end\n               end,\n               [], Trace),\n    Nodes = lists:reverse(NodesR),\n    DrawTrace = DeltaFun(Trace),\n\n    EndTime = element(2, lists:last(DrawTrace)),\n\n    ScaleX =\n        if (EndTime div ScaleX0) > 565 ->\n               NewScale = util:ceil(EndTime / 5650) * 10,\n               io:format(\"Warning: adapting scale to 1cm : ~Bus to fit \"\n                         \"the maximum LaTeX width of 565cm.~n\", [NewScale]),\n               NewScale;\n           true ->\n               ScaleX0\n        end,\n    TicsFreq = ScaleX, %% draw x-tics every TicsFreq microseconds\n\n    %% draw nodes and timelines\n    _ = lists:foldl(\n          fun(X, Acc) ->\n                  LatexNode = term_to_latex_string(X),\n                  io:format(File,\n                            \"\\\\draw (0, -~f) node[anchor=east] {~s};~n\",\n                            [length(Acc)/2, LatexNode]),\n                  io:format(File,\n                            \"\\\\draw[color=gray,very thin] (0, -~f) -- (~fcm, -~f);~n\",\n                            [length(Acc)/2, (EndTime+10)/ScaleX, length(Acc)/2]),\n                  case EndTime > 600 of\n                      true ->\n                          io:format(File,\n                                    \"\\\\foreach \\\\x in {~B,~B,...,~B}~n\"\n                                    \"  \\\\node[anchor=south east,gray,inner sep=0pt] at (\\\\x, -~f) {\\\\tiny ~s};~n\",\n                                    [15, 30,\n                                     ((trunc((EndTime+10)) div 15) * 15) div ScaleX,\n                                     length(Acc)/2, LatexNode]);\n                      false -> ok\n                  end,\n                  [X | Acc]\n          end,\n          [], Nodes),\n    %% draw key\n    EndSlot = (EndTime div TicsFreq),\n    util:for_to(\n      1, EndSlot,\n      fun(I) ->\n              io:format(File,\n              \"  \\\\draw[color=gray, very thin] (~fcm, 0.5)\"\n              \"  node[above] {~B\" ++ ?IIF(HaveRealTime, \"$\\\\mu$s\", \"\") ++ \"} --\"\n              \" (~f, -~f);~n\",\n              [I*TicsFreq/ScaleX,\n               I*TicsFreq,\n               I*TicsFreq/ScaleX, length(Nodes)/2])\n      end),\n\n    io:format(File,\n              \"  \\\\draw[color=green!30!black,->] (-4cm, 0.5)\"\n              \"  -- (-3.5cm, 0.5) node[anchor=west] {local send};~n\"\n              \"  \\\\draw[color=red!50!black,->] (-2cm, 0.5)\"\n              \"  -- (-1.5cm, 0.5) node[anchor=west] {global send};~n\", []),\n\n    draw_messages(File, Nodes, ScaleX, HaveRealTime, DrawTrace),\n\n    io:format(\n      File,\n      \"\\\\end{tikzpicture}~n\"\n      \"\\\\end{document}~n\",\n      []),\n    file:close(File).\n\nterm_to_latex_string(Term) ->\n    quote_latex(lists:flatten(io_lib:format(\"~0.0p\", [Term]))).\n\nquote_latex([]) -> [];\nquote_latex([Char | Tail]) ->\n    NewChars =\n        case Char of\n            $_ ->  \"\\\\_\";\n            ${ ->  \"\\\\{\";\n            $} ->  \"\\\\}\";\n%%            $[ ->  \"\\\\[\";\n%%            $] ->  \"\\\\]\";\n            $# ->  \"\\\\#\";\n            %% $< ->  lists:reverse(\"$\\lt$\");\n            %% $> ->  lists:reverse(\"$\\gt$\");\n            _ -> [Char]\n        end,\n    NewChars ++ quote_latex(Tail).\n\n%% @doc Gets the message tag of the given message. Special dht_node messages of\n%%      embedded processes get translated into a tuple of two message tags.\n-spec get_msg_tag(Msg::comm:message()) -> atom() | {atom(), atom()}.\nget_msg_tag(Msg) when tuple_size(Msg) =< 1 ->\n    element(1, Msg);\nget_msg_tag(Msg) ->\n    case element(1, Msg) of\n        TagX when (TagX =:= join orelse TagX =:= move orelse\n                           TagX =:= l_on_cseq orelse TagX =:= rm) ->\n            Element2 = element(2, Msg),\n            case is_tuple(Element2) andalso tuple_size(Element2) >= 1\n                andalso is_atom(element(1, Element2)) of\n                true  ->\n                    {TagX, element(1, Element2)};\n                false when is_atom(Element2) ->\n                    {TagX, Element2};\n                false ->\n                    TagX\n            end;\n        ?send_to_group_member_atom ->\n            {?send_to_group_member_atom, _PidName, EmbeddedMsg} = Msg,\n            {send_to_group_member, get_msg_tag(EmbeddedMsg)};\n        ?send_to_registered_proc_atom ->\n            {?send_to_registered_proc_atom, _PidName, EmbeddedMsg} = Msg,\n            {send_to_registered_proc, get_msg_tag(EmbeddedMsg)};\n        TagX when TagX =:= ?lookup_aux_atom orelse\n                      TagX =:= ?lookup_fin_atom ->\n            {TagX, _Key, _Hops, WrappedMsg} = Msg,\n            {TagX, get_msg_tag(WrappedMsg)};\n        TagX ->\n            TagX\n    end.\n\ndraw_messages(_File, _Nodes, _ScaleX, _HaveRealTime, []) -> ok;\ndraw_messages(File, Nodes, ScaleX, HaveRealTime, [X | DrawTrace]) ->\n    RemainingTrace =\n    case element(1, X) of\n        log_send ->\n            %% search for corresponding receive and draw a line\n            SrcNum = length(lists:takewhile(\n                              fun(Y) -> element(4, X) =/= Y end, Nodes)),\n            DestNum = length(lists:takewhile(\n                               fun(Y) -> element(5, X) =/= Y end, Nodes)),\n            Recv = [ Y || Y <- DrawTrace,\n                          log_recv =:= element(1, Y),\n                          element(4, X) =:= element(4, Y),\n                          element(5, X) =:= element(5, Y),\n                          %% element(6, X) =:= element(6, Y), %% we have fifo\n                          element(2, X) =< element(2, Y)\n                   ],\n            SendTime = element(2, X),\n            SendMsg = element(6, X),\n            RecvEvent =\n                case Recv of\n                    [] -> setelement(2, X, SendTime + 10);\n                    _ -> hd(Recv)\n                end,\n            RecvTime = element(2, RecvEvent),\n            SendTag = term_to_latex_string(get_msg_tag(SendMsg)),\n            RecvTag = term_to_latex_string(get_msg_tag(element(6, RecvEvent))),\n            Color = case element(7, X) of\n                        local -> \"green!30!black\";\n                        global -> \"red!50!black\"\n                    end,\n            MsgSizeBytes = erlang:byte_size(erlang:term_to_binary(SendMsg, [{minor_version, 1}])),\n            MsgSize =\n                case math:log(MsgSizeBytes)/math:log(2) of\n                    L when L < 10.0 ->\n                        term_to_latex_string(MsgSizeBytes) ++ \"\\\\,B\";\n                    L when L < 20.0 ->\n                        term_to_latex_string(MsgSizeBytes div 1024) ++ \"\\\\,KB\";\n                    L when L < 30.0 ->\n                        term_to_latex_string(\n                          MsgSizeBytes div 1024 div 1024) ++ \"\\\\,MB\";\n                    L when L < 40.0 ->\n                        term_to_latex_string(\n                          MsgSizeBytes div 1024 div 1024 div 1024) ++ \"\\\\,GB\"\n                end,\n\n            case SrcNum of\n                SrcNum when (SrcNum < DestNum) ->\n                    MsgTag =\n                        case SendTag =:= RecvTag of\n                            true -> SendTag;\n                            false -> SendTag ++ \"\\\\\\\\[-0.5em]\\\\tiny \" ++ RecvTag\n                        end,\n                    io:format(File,\n                              \"\\\\draw[->, color=~s] (~fcm, -~f)\"\n                              \" to node[inner sep=1pt, anchor=west,sloped,rotate=90,align=left] \"\n                              \"{\\\\tiny ~s} (~fcm, -~f) \"\n                              \"node [anchor=north, inner sep=1pt] {\\\\tiny ~s};~n\",\n                              [Color, SendTime/ScaleX, SrcNum/2,\n                               MsgTag,\n                               RecvTime/ScaleX, DestNum/2,\n                               MsgSize]);\n                SrcNum when (SrcNum > DestNum) ->\n                    MsgTag =\n                        case SendTag =:= RecvTag of\n                            true -> SendTag;\n                            false -> RecvTag ++ \"\\\\\\\\[-0.5em]\\\\tiny \" ++ SendTag\n                        end,\n                    io:format(File,\n                              \"\\\\draw[->, color=~s] (~fcm, -~f)\"\n                              \" to node[inner sep=1pt, anchor=west,sloped,rotate=-90, align=left] \"\n                              \"{\\\\tiny ~s} (~fcm, -~f) \"\n                              \"node [anchor=west, inner sep=1pt, rotate=60] {\\\\tiny ~s};~n\",\n                              [Color, SendTime/ScaleX, SrcNum/2,\n                               MsgTag,\n                               RecvTime/ScaleX, DestNum/2,\n                               MsgSize]);\n                SrcNum when (SrcNum =:= DestNum) ->\n                    MsgTag =\n                        case SendTag =:= RecvTag of\n                            true -> SendTag;\n                            false -> RecvTag ++ \"\\\\\\\\[-0.5em]\\\\tiny \" ++ SendTag\n                        end,\n                    io:format(File,\n                              \"\\\\draw[->, color=~s] (~fcm, -~f)\"\n                              \" .. controls +(~fcm,-0.3) ..\"\n                              \" node[inner sep=1pt,anchor=west,sloped,rotate=-90, align=left]\"\n                              \"{\\\\tiny ~s} (~fcm, -~f) \"\n                              \"node [anchor=west, inner sep=1pt, rotate=60] {\\\\tiny ~s};~n\",\n                              [Color, element(2, X)/ScaleX, SrcNum/2,\n                               (RecvTime - element(2, X))/ScaleX/2,\n                               MsgTag,\n                               RecvTime/ScaleX, DestNum/2,\n                               MsgSize])\n            end,\n            NewDrawTrace =\n                case Recv of\n                    [] -> DrawTrace;\n                    _ -> lists:delete(hd(Recv), DrawTrace)\n                end,\n            case Recv of\n                [] -> DrawTrace;\n                _ ->\n                    %% draw process busy until gc_on_done log_info event\n                    DoneEvents = [ Y || Y <- NewDrawTrace,\n                                        log_info =:= element(1, Y),\n                                        element(2, Y) >= RecvTime,\n                                        element(4, Y) =:= element(5, hd(Recv)),\n                                        element(1, element(5, Y)) =:= gc_on_done\n                                ],\n                    case DoneEvents of\n                        [] -> NewDrawTrace;\n                        _ ->\n                            DoneTime = element(2, hd(DoneEvents)),\n                            if HaveRealTime ->\n                                   io:format(File,\n                                             \"\\\\draw[semithick] (~fcm, -~f)\"\n                                                 \" -- \"\n                                                 \" (~fcm, -~f) node[inner sep=1pt, anchor=south] {\\\\tiny ~B};~n\",\n                                             [RecvTime/ScaleX, DestNum/2,\n                                              DoneTime/ScaleX, DestNum/2,\n                                              DoneTime - RecvTime]);\n                               true -> ok\n                            end,\n                            lists:delete(hd(DoneEvents), NewDrawTrace)\n                    end\n            end;\n        log_recv ->\n            %% found a receive without a send?\n            %% not yet implemented\n            io:format(\"Found receive without send?~n\"),\n          _ = element(5, X),\n            DrawTrace;\n        log_info ->\n            %% print info somewhere\n            SrcNum = length(lists:takewhile(\n                              fun(Y) -> element(4, X) =/= Y end, Nodes)),\n            EventTime = element(2, X),\n            io:format(\n              File, \"\\\\draw [color=blue] (~fcm, -~f) ++(0, 0.1cm) node[rotate=60, anchor=west, inner sep=1pt] {\\\\tiny ~s}-- ++(0, -0.2cm);~n\",\n              [EventTime/ScaleX, SrcNum/2, term_to_latex_string(element(5, X))]),\n            DrawTrace\n    end,\n    draw_messages(File, Nodes, ScaleX, HaveRealTime, RemainingTrace).\n\n%% Functions used to report tracing events from other modules\n-spec epidemic_reply_msg(passed_state(), anypid(), anypid(), comm:message()) ->\n                                gc_mpath_msg().\nepidemic_reply_msg(PState, FromPid, ToPid, Msg) ->\n    {'$gen_component', trace_mpath, PState, comm:make_global(FromPid), ToPid, Msg}.\n\n-spec log_send(passed_state(), anypid(), anypid(), comm:message(),\n               local | global | local_after, comm:send_options())\n        -> DeliverAlsoDirectly :: boolean().\nlog_send(PState, FromPid, ToPid, Msg, LocalOrGlobal, SendOptions) ->\n    Now = os:timestamp(),\n    case passed_state_logger(PState) of\n        io_format ->\n            MsgMapFun = passed_state_msg_map_fun(PState),\n            io:format(\"~p send ~.0p -> ~.0p:~n  ~.0p.~n\",\n                      [util:readable_utc_time(Now),\n                       normalize_pidinfo(FromPid),\n                       normalize_pidinfo(ToPid), MsgMapFun(Msg, FromPid, ToPid)]),\n            true;\n        {log_collector, LoggerPid} ->\n            MsgMapFun = passed_state_msg_map_fun(PState),\n            TraceId = passed_state_trace_id(PState),\n            case comm:is_local(LoggerPid) of\n                true ->\n                    % normalise names later...\n                    FromPid1 = FromPid, ToPid1 = ToPid, ok;\n                false ->\n                    % normalise names now (non-local loggers cannot resolve them)\n                    FromPid1 = normalize_pidinfo(FromPid),\n                    ToPid1 = normalize_pidinfo(ToPid),\n                    ok\n            end,\n            send_log_msg(\n              PState,\n              LoggerPid,\n              {log_send, Now, TraceId, FromPid1, ToPid1, MsgMapFun(Msg, FromPid, ToPid),\n               ?IIF(LocalOrGlobal =:= local_after, local, LocalOrGlobal)}),\n            true;\n        {proto_sched, _} when LocalOrGlobal =:= local_after ->\n            %% Do delivery via proto_sched and *not* via\n            %% erlang:send_after (returning false here). Otherwise the\n            %% msg gets delivered a second time shortly after outside\n            %% the schedule.\n            %% TODO: see comm:send_local_after\n            %% TODO: Should be queued to another queue in the proto_sched to\n            %% allow violation of FIFO ordering\n            proto_sched:log_send(PState, FromPid, ToPid, Msg, LocalOrGlobal,\n                                 SendOptions),\n            false;\n        {proto_sched, _} ->\n            proto_sched:log_send(PState, FromPid, ToPid, Msg, LocalOrGlobal,\n                                 SendOptions),\n            false\n    end.\n\n-spec log_info(anypid(), comm:message()) -> ok.\nlog_info(FromPid, Info) ->\n    case own_passed_state_get() of\n        undefined -> ok;\n        PState -> log_info(PState, FromPid, Info)\n    end.\n-spec log_info(passed_state(), anypid(), comm:message()) -> ok.\nlog_info(PState, FromPid, Info) ->\n    Now = os:timestamp(),\n    case passed_state_logger(PState) of\n        io_format ->\n            io:format(\"~p info ~.0p:~n  ~.0p.~n\",\n                      [util:readable_utc_time(Now),\n                       normalize_pidinfo(FromPid),\n                       Info]);\n        {log_collector, LoggerPid} ->\n            TraceId = passed_state_trace_id(PState),\n            case comm:is_local(LoggerPid) of\n                true ->\n                    % normalise names later...\n                    FromPid1 = FromPid, ok;\n                false ->\n                    % normalise names now (non-local loggers cannot resolve them)\n                    FromPid1 = normalize_pidinfo(FromPid),\n                    ok\n            end,\n            send_log_msg(PState, LoggerPid, {log_info, Now, TraceId, FromPid1, Info});\n        {proto_sched, LoggerPid} ->\n            case Info of\n                {gc_on_done, Tag} ->\n                    TraceId = passed_state_trace_id(PState),\n                    send_log_msg(PState, LoggerPid,\n                                 {on_handler_done, TraceId, Tag,\n                                  element(1, normalize_pidinfo(FromPid))});\n                _ ->\n                    proto_sched:log_info(PState, FromPid, Info)\n            end\n    end,\n    ok.\n\n-spec log_recv(passed_state(), anypid(), anypid(), comm:message()) -> ok.\nlog_recv(PState, FromPid, ToPid, Msg) ->\n    Now = os:timestamp(),\n    case passed_state_logger(PState) of\n        io_format ->\n            MsgMapFun = passed_state_msg_map_fun(PState),\n            io:format(\"~p recv ~.0p -> ~.0p:~n  ~.0p.~n\",\n                      [util:readable_utc_time(Now),\n                       normalize_pidinfo(FromPid),\n                       normalize_pidinfo(ToPid),\n                       MsgMapFun(Msg, FromPid, ToPid)]);\n        {log_collector, LoggerPid} ->\n            MsgMapFun = passed_state_msg_map_fun(PState),\n            TraceId = passed_state_trace_id(PState),\n            case comm:is_local(LoggerPid) of\n                true ->\n                    % normalise names later...\n                    FromPid1 = FromPid, ToPid1 = ToPid, ok;\n                false ->\n                    % normalise names now (non-local loggers cannot resolve them)\n                    FromPid1 = normalize_pidinfo(FromPid),\n                    ToPid1 = normalize_pidinfo(ToPid),\n                    ok\n            end,\n            send_log_msg(\n              PState,\n              LoggerPid,\n              {log_recv, Now, TraceId, FromPid1, ToPid1, MsgMapFun(Msg, FromPid, ToPid)});\n        {proto_sched, _} ->\n            ok\n    end,\n    ok.\n\n-spec send_log_msg(passed_state(), comm:mypid(),\n                   trace_event0() |\n                       {on_handler_done, trace_id(),\n                        MsgTag::atom() | integer(), anypid()})\n        -> ok.\nsend_log_msg(RestoreThis, LoggerPid, Msg) ->\n    %% don't log the sending of log messages ...\n    case passed_state_logger(RestoreThis) of\n        {proto_sched, _} ->\n            clear_infection(),\n            comm:send(LoggerPid, Msg),\n            own_passed_state_put(RestoreThis);\n        _ ->\n            FilterFun = passed_state_filter_fun(RestoreThis),\n            case FilterFun(Msg) of\n                true ->\n                    clear_infection(),\n                    comm:send(LoggerPid, Msg),\n                    own_passed_state_put(RestoreThis);\n                false -> ok\n            end\n    end.\n\n-spec normalize_pidinfo(anypid() | pidinfo()) -> pidinfo().\nnormalize_pidinfo(Pid) ->\n    case is_pid(Pid) of\n        true ->\n            ?ASSERT(node(Pid) =:= node()),\n            PidName = try pid_groups:group_and_name_of(Pid) of\n                          failed -> no_pid_name;\n                          Name -> Name\n                      catch _:_ -> no_pid_name\n                      end,\n            {comm:make_global(Pid), PidName};\n        false ->\n            case comm:is_valid(Pid) of\n                true ->\n                    case comm:is_local(Pid) of\n                        true ->\n                            PidName =\n                                try pid_groups:group_and_name_of(\n                                       comm:make_local(Pid)) of\n                                    failed -> no_pid_name;\n                                    Name -> Name\n                                catch _:_ -> no_pid_name\n                                end,\n                            {Pid, PidName};\n                        false -> {Pid, non_local_pid_name_unknown}\n                    end;\n                false -> %% already a pidinfo()\n                    Pid\n            end\n    end.\n\n-type state() :: [{trace_id(), trace()}].\n\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(ServiceGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                             [{erlang_register, trace_mpath},\n                              {pid_groups_join_as, ServiceGroup, ?MODULE}]).\n\n-spec init(any()) -> state().\ninit(_Arg) -> [].\n\n-spec on(trace_event() | comm:message(), state()) -> state().\non({log_send, _Time, TraceId, _From, _To, _UMsg, _LorG} = Msg, State) ->\n    state_add_log_event(State, TraceId, Msg);\non({log_recv, _Time, TraceId, _From, _To, _UMsg} = Msg, State) ->\n    state_add_log_event(State, TraceId, Msg);\non({log_info, _Time, TraceId, _From, _UMsg} = Msg, State) ->\n    state_add_log_event(State, TraceId, Msg);\n\non({get_trace, Pid, TraceId, cleanup}, State) ->\n    case lists:keytake(TraceId, 1, State) of\n        false ->\n            comm:send(Pid, {get_trace_reply, no_trace_found}),\n            State;\n        {value, {TraceId, Msgs}, TupleList2} ->\n            comm:send(Pid, {get_trace_reply, lists:reverse(Msgs)}),\n            TupleList2\n    end;\non({get_trace, Pid, TraceId, none}, State) ->\n    case lists:keyfind(TraceId, 1, State) of\n        false ->\n            comm:send(Pid, {get_trace_reply, no_trace_found});\n        {TraceId, Msgs} ->\n            comm:send(Pid, {get_trace_reply, lists:reverse(Msgs)})\n    end,\n    State;\non({cleanup, TraceId}, State) ->\n    case lists:keytake(TraceId, 1, State) of\n        {value, _Tuple, TupleList2} -> TupleList2;\n        false                       -> State\n    end.\n\n-spec passed_state_new(trace_id(), logger(), msg_map_fun(), filter_fun())\n        -> passed_state1().\npassed_state_new(TraceId, Logger, MsgMapFun, FilterFun) ->\n    {TraceId, Logger, MsgMapFun, FilterFun}.\n\n-spec passed_state_trace_id(passed_state()) -> trace_id().\npassed_state_trace_id(State)      -> element(1, State).\n-spec passed_state_logger(passed_state()) -> logger().\npassed_state_logger(State)        -> element(2, State).\n-spec passed_state_msg_map_fun(passed_state1()) -> msg_map_fun().\npassed_state_msg_map_fun(State)   -> element(3, State).\n-spec passed_state_filter_fun(passed_state1()) -> filter_fun().\npassed_state_filter_fun(State)    -> element(4, State).\n\n-spec own_passed_state_put(passed_state()) -> ok.\nown_passed_state_put(State)       -> erlang:put(trace_mpath, State), ok.\n\n-spec own_passed_state_get() -> passed_state() | undefined.\nown_passed_state_get()            -> erlang:get(trace_mpath).\n\nstate_add_log_event(State, TraceId, Msg) ->\n    NewEntry = case lists:keyfind(TraceId, 1, State) of\n                   false ->\n                       {TraceId, [Msg]};\n                   {TraceId, OldTrace} ->\n                       {TraceId, [Msg | OldTrace]}\n               end,\n    lists:keystore(TraceId, 1, State, NewEntry).\n"
  },
  {
    "path": "src/tracer.erl",
    "content": "% @copyright 2009-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Simple Profiler for Scalaris.\n%% @version $Id$\n-module(tracer).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([tracer/1, start/0, dump/0,\n         tracer_perf/1, start_perf/0, dump_perf/0]).\n\n-include(\"scalaris.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% 1. put tracer:start() and/or tracer:start_perf() into scalaris_app.erl\n%    before application:start(scalaris)\n% 2. run benchmark\n% 3. call tracer:dump() or tracer:dump_perf()\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec start() -> ok.\nstart() ->\n    spawn(?MODULE, tracer, [self()]),\n    trace_mpath:thread_yield(),\n    receive ?SCALARIS_RECV({done}, ok) end,\n    ok.\n\n-spec start_perf() -> ok.\nstart_perf() ->\n    spawn(?MODULE, tracer_perf, [self()]),\n    trace_mpath:thread_yield(),\n    receive ?SCALARIS_RECV({done}, ok) end,\n    ok.\n\n-spec tracer(Pid::comm:erl_local_pid()) -> no_return().\ntracer(Pid) ->\n    erlang:trace(all, true, [send, procs]),\n    comm:send_local(Pid, {done}),\n    _ = ets:new(tracer, [set, protected, named_table]),\n    loop().\n\n-spec tracer_perf(Pid::comm:erl_local_pid()) -> no_return().\ntracer_perf(Pid) ->\n    erlang:trace(all, true, [running, timestamp]),\n    comm:send_local(Pid, {done}),\n    _ = ets:new(tracer_perf, [set, protected, named_table]),\n    loop_perf().\n\n-spec loop() -> no_return().\nloop() ->\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n            {trace, Pid, send_to_non_existing_process, Msg, To}, %% ->\n            begin\n                log:log(error,\"send_to_non_existing_process: ~p -> ~p (~p)\", [Pid, To, Msg]),\n                loop()\n            end\n          );\n        ?SCALARIS_RECV(\n            {trace, Pid, exit, Reason}, %% ->\n            case Reason of\n                normal ->\n                    loop();\n                {ok, _Stack,_Num} ->\n                    io:format(\" EXIT: ~p | ~p~n\", [Pid, Reason]),\n                    loop();\n                _ ->\n                    io:format(\" EXIT: ~p | ~p~n\", [Pid, Reason]),\n                    %io:format(\"~p~n\", [dump()]),\n                    %log:log(warn,\"EXIT: ~p | ~p\", [Pid, Reason]),\n                    loop()\n            end\n          );\n        ?SCALARIS_RECV(\n            {trace, Pid, spawn, Pid2, {M, F, Args}}, %% ->\n            begin\n                %io:format(\" SPAWN: ~p -> ~p in ~p~n\", [Pid, Pid2, {M, F, Args}]),\n                %log:log(debug \"[ TRACER ] SPAWN: ~p -> ~p in ~p\", [Pid, Pid2, {M, F, Args}]),\n                ets:insert(tracer, {Pid, Pid2, {M, F, Args}}),\n                loop()\n            end\n          );\n        _X ->\n            loop()\n    end.\n\n-spec loop_perf() -> no_return().\nloop_perf() ->\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n            {trace_ts, Pid, in, _, TS}, %% ->\n            begin\n                case ets:lookup(tracer_perf, Pid) of\n                    [] ->\n                        ets:insert(tracer_perf, {Pid, TS, 0});\n                    [{Pid, _, Sum}] ->\n                        ets:insert(tracer_perf, {Pid, TS, Sum})\n                end,\n                loop_perf()\n            end\n          );\n        ?SCALARIS_RECV(\n            {trace_ts, Pid, out, _, TS}, %% ->\n            begin\n                case ets:lookup(tracer_perf, Pid) of\n                    [] ->\n                        ets:insert(tracer_perf, {Pid, ok, 0});\n                    [{Pid, In, Sum}] ->\n                        ets:insert(tracer_perf, {Pid, ok, timer:now_diff(TS, In) + Sum})\n                end,\n                loop_perf()\n            end\n          );\n        _X ->\n            io:format(\"unknown message: ~p~n\", [_X]),\n            loop_perf()\n    end.\n-spec dump() -> [{Pid::pid(), Pid2::pid(), {M::module(), F::atom(), Args::list()}}].\ndump() ->\n    ets:tab2list(tracer).\n\n-spec dump_perf() -> [{Pid::pid(), ScheduledIn::{MegaSecs::integer(), Secs::integer(), MicroSecs::integer()} | ok, Runtime::integer()}].\ndump_perf() ->\n    lists:reverse(lists:keysort(3, ets:tab2list(tracer_perf))).\n"
  },
  {
    "path": "src/transactions/rdht_tx.erl",
    "content": "% @copyright 2009-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    API for transactions on replicated DHT items.\n%% @version $Id$\n-module(rdht_tx).\n-author('schintke@zib.de').\n-author('kruber@zib.de').\n-vsn('$Id$ ').\n\n-compile({inline, [req_get_op/1, req_get_key/1]}).\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n\n-export([req_list/3]).\n-export([check_config/0]).\n-export([encode_value/1, decode_value/1, is_encoded_value/1]).\n-export([req_props/1,\n         req_get_key/1, req_get_op/1]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export_type([req_id/0, request_on_key/0, encoded_value/0]).\n\n-type req_id() :: uid:global_uid().\n-type request_on_key() :: api_tx:request_on_key().\n\n-type encoded_value() :: atom() | boolean() | number() | binary().\n\n%% @doc Perform several requests inside a transaction.\n-spec req_list(tx_tlog:tlog_ext(), [api_tx:request()], EnDecode::true)      -> {tx_tlog:tlog_ext(), [api_tx:result()]};\n              (tx_tlog:tlog_ext(), [api_tx:request_enc()], EnDecode::false) -> {tx_tlog:tlog_ext(), [api_tx:result()]}.\nreq_list([], [{commit}], _EnDecode) -> {[], [{ok}]};\nreq_list(TLog, ReqList, EnDecode) ->\n    %% PRE: TLog is sorted by key, implicitly given, as\n    %%      we only generate sorted TLogs.\n    ?TRACE(\"rdht_tx:req_list(~p, ~p, ~p)~n\", [TLog, ReqList, EnDecode]),\n\n    %% (0) Check TLog? Costs performance, may save some requests\n\n    %% (1) Ensure commit is only at end of req_list (otherwise abort),\n    %% (2) drop {commit} request at end and remember whether to\n    %%     commit or not\n    case rl_chk(ReqList, []) of\n        abort -> tlog_and_results_to_abort(TLog, ReqList);\n        {ReqList1, FoundCommitAtEnd} ->\n            TLog2 = upd_tlog_via_rdht(TLog, ReqList1),\n\n            %% perform all requests based on TLog to compute result\n            %% entries\n%%            io:format(\"Doing work on TLog~n\"),\n            {NewClientTLog, Results} = do_reqs_on_tlog(TLog2, ReqList1, EnDecode),\n%%            io:format(\"Doing commit on ~p~n\", [NewClientTLog]),\n\n            %% do commit (if requested) and append the commit result\n            %% to result list\n            case FoundCommitAtEnd of\n                true ->\n                    CommitRes = commit(NewClientTLog),\n                    {tx_tlog:empty(), Results ++ [CommitRes]};\n                false ->\n                    {NewClientTLog, Results}\n            end\n    end.\n\n%% @doc Check whether commit is only at end (OKorAbort).\n%%      Cut commit at end and inform caller via boolean (CommitAtEnd).\n-spec rl_chk(InTodo::[api_tx:request()], Acc::[api_tx:request()])\n        -> abort | {Acc::[api_tx:request()], CommitAtEnd::boolean()}.\nrl_chk([], Acc) ->\n    {lists:reverse(Acc), false};\nrl_chk([{commit}], Acc) ->\n    {lists:reverse(Acc), true};\nrl_chk([{commit} | _RL], _Acc) ->\n    log:log(info, \"Commit not at end of a req_list. Deciding abort.\"),\n    abort;\nrl_chk([Op | RL], Acc) ->\n    rl_chk(RL, [Op | Acc]).\n\n%% @doc Fill all fields with {fail, abort} information.\n-spec tlog_and_results_to_abort(tx_tlog:tlog(), [api_tx:request()]) ->\n                                       {tx_tlog:tlog(), [api_tx:result()]}.\ntlog_and_results_to_abort(TLog, ReqList) ->\n    tlog_and_results_to_abort_iter(TLog, ReqList, []).\n\n-spec tlog_and_results_to_abort_iter(tx_tlog:tlog(), [api_tx:request()], [api_tx:result()])\n        -> {tx_tlog:tlog(), [api_tx:result()]}.\ntlog_and_results_to_abort_iter(TLog, [], AccRes) ->\n    {TLog, lists:reverse(AccRes)};\ntlog_and_results_to_abort_iter(TLog, [Req | ReqListT], AccRes) ->\n    case Req of\n        {commit} ->\n            Res = {fail, abort, []},\n            tlog_and_results_to_abort_iter(TLog, ReqListT, [Res | AccRes]);\n        _ ->\n            Res = case req_get_op(Req) of\n                      read -> {fail, not_found};\n                      write -> {ok};\n                      add_del_on_list -> {ok};\n                      add_on_nr -> {ok};\n                      test_and_set -> {ok}\n                  end,\n            NewTLog = tx_tlog:add_or_update_status_by_key(\n                        TLog, req_get_key(Req), ?fail),\n            tlog_and_results_to_abort_iter(NewTLog, ReqListT, [Res | AccRes])\n    end.\n\n%% @doc Send requests to the DHT, gather replies and merge TLogs.\n-spec upd_tlog_via_rdht(tx_tlog:tlog(), [request_on_key()]) -> tx_tlog:tlog().\nupd_tlog_via_rdht(TLog, ReqList) ->\n    %% what to get from rdht? (also check old TLog)\n    SReqList = lists:keysort(2, ReqList),\n    ReqListonRDHT = tx_tlog:first_req_per_key_not_in_tlog(TLog, SReqList),\n\n    %% perform RDHT operations to collect missing TLog entries\n    %% rdht requests for independent keys are processed in parallel.\n    ReqIds = initiate_rdht_ops(ReqListonRDHT),\n\n    RTLog = collect_replies(tx_tlog:empty(), ReqIds),\n\n    %% merge TLogs (insert fail, abort, when version mismatch\n    %% in reads for same key is detected)\n    _MTLog = tx_tlog:merge(TLog, RTLog).\n\n-spec req_props(Req::request_on_key()) -> {NeedsFullRead::boolean(), WorksAfterAnyPartialRead::boolean(), ProvidesFullRead::boolean()}.\nreq_props(Req) ->\n    case req_get_op(Req) of\n        read when tuple_size(Req) =:= 2 -> {true, false, true}; % full read\n        read when tuple_size(Req) =:= 3 -> {false, false, false}; % partial read\n        test_and_set -> {true, false, true};\n        add_on_nr -> {true, false, true};\n        add_del_on_list -> {true, false, true};\n        write -> {false, true, true}\n    end.\n\n%% @doc Trigger operations for the DHT.\n-spec initiate_rdht_ops([request_on_key()]) -> [req_id()].\ninitiate_rdht_ops(ReqList) ->\n    ?TRACE(\"rdht_tx:initiate_rdht_ops(~p)~n\", [ReqList]),\n    %% @todo should choose a dht_node in the local VM at random or even\n    %% better round robin.\n    [ begin\n          NewReqId = uid:get_global_uid(), % local id not sufficient\n          case req_get_op(Entry) of\n              write           -> rdht_tx_write:work_phase(self(), NewReqId, Entry);\n              read            -> rdht_tx_read:work_phase(self(), NewReqId, Entry);\n              test_and_set    -> rdht_tx_test_and_set:work_phase(self(), NewReqId, Entry);\n              add_del_on_list -> rdht_tx_add_del_on_list:work_phase(self(), NewReqId, Entry);\n              add_on_nr       -> rdht_tx_add_on_nr:work_phase(self(), NewReqId, Entry)\n          end,\n          NewReqId\n      end || Entry <- ReqList ].\n\n%% @doc Collect replies from the quorum DHT operations.\n-spec collect_replies(tx_tlog:tlog(), [req_id()]) -> tx_tlog:tlog().\ncollect_replies(TLog, [_H | _T] = ReqIdsList) ->\n    ?TRACE(\"rdht_tx:collect_replies(~p, ~p)~n\", [TLog, ReqIdsList]),\n    % receive only matching replies\n    {ReqId, RdhtTlogEntry} = receive_answer(),\n    case util:lists_take(ReqId, ReqIdsList) of\n        false -> collect_replies(TLog, ReqIdsList);\n        RemainingReqIds ->\n            NewTLog = tx_tlog:add_entry(TLog, RdhtTlogEntry),\n            collect_replies(NewTLog, RemainingReqIds)\n    end;\ncollect_replies(TLog, []) ->\n    %% Drop outdated results...\n    receive_old_answers(),\n%%    io:format(\"Tlog collected ~p~n\", [TLog]),\n    tx_tlog:sort_by_key(TLog).\n\n%% @doc Perform all operations on the TLog and generate list of results.\n-spec do_reqs_on_tlog(tx_tlog:tlog(), [request_on_key()], EnDecode::boolean()) ->\n                             {tx_tlog:tlog(), [api_tx:result()]}.\ndo_reqs_on_tlog(TLog, ReqList, EnDecode) ->\n    do_reqs_on_tlog_iter(TLog, ReqList, [], EnDecode).\n\n%% @doc Helper to perform all operations on the TLog and generate list\n%%      of results.\n%%      TODO: sort the req list similar to the tlog list and parse through both at the same time!\n-spec do_reqs_on_tlog_iter(tx_tlog:tlog(), [request_on_key()], [api_tx:result()], EnDecode::boolean())\n        -> {tx_tlog:tlog(), [api_tx:result()]}.\ndo_reqs_on_tlog_iter(TLog, [Req | ReqTail], Acc, EnDecode) ->\n    {NewTLogEntry, ResultEntry} =\n        case Req of\n            %% native functions first:\n            {read, Key} ->\n                Entry = tx_tlog:find_entry_by_key(TLog, Key),\n                rdht_tx_read:extract_from_tlog(Entry, Key, read, EnDecode);\n            {read, Key, Op} ->\n                Entry = tx_tlog:find_entry_by_key(TLog, Key),\n                rdht_tx_read:extract_from_tlog(Entry, Key, Op, EnDecode);\n            {write, Key, Value} ->\n                Entry = tx_tlog:find_entry_by_key(TLog, Key),\n                rdht_tx_write:extract_from_tlog(Entry, Key, Value, EnDecode);\n            %% non-native functions:\n            {add_del_on_list, Key, ToAdd, ToDel} ->\n                Entry = tx_tlog:find_entry_by_key(TLog, Key),\n                rdht_tx_add_del_on_list:extract_from_tlog(Entry, Key, ToAdd, ToDel, EnDecode);\n            {add_on_nr, Key, X} ->\n                Entry = tx_tlog:find_entry_by_key(TLog, Key),\n                rdht_tx_add_on_nr:extract_from_tlog(Entry, Key, X, EnDecode);\n            {test_and_set, Key, Old, New} ->\n                Entry = tx_tlog:find_entry_by_key(TLog, Key),\n                rdht_tx_test_and_set:extract_from_tlog(Entry, Key, Old, New, EnDecode)\n        end,\n    NewTLog = tx_tlog:update_entry(TLog, NewTLogEntry),\n    do_reqs_on_tlog_iter(NewTLog, ReqTail, [ResultEntry | Acc], EnDecode);\ndo_reqs_on_tlog_iter(TLog, [], Acc, _EnDecode) ->\n    {tx_tlog:cleanup(TLog), lists:reverse(Acc)}.\n\n-spec is_encoded_value(term()) -> boolean().\nis_encoded_value(Value) when is_atom(Value) -> true;\nis_encoded_value(Value) when is_boolean(Value) -> true;\nis_encoded_value(Value) when is_number(Value) -> true;\nis_encoded_value(Value) when is_binary(Value) ->\n    try decode_value(Value) of _ -> true\n    catch error:badarg -> false\n    end;\nis_encoded_value(_) -> false.\n\n%% @doc Encode the given client value to its internal representation which is\n%%      compressed for all values except atom, boolean, number or binary.\n-spec encode_value(client_value()) -> encoded_value().\nencode_value(Value) when is_atom(Value) -> Value;\nencode_value(Value) when is_boolean(Value) -> Value;\nencode_value(Value) when is_number(Value) -> Value;\nencode_value(Value) when is_binary(Value) ->\n    %% do not compress a binary\n    erlang:term_to_binary(Value, [{minor_version, 1}]);\nencode_value(Value) ->\n    erlang:term_to_binary(Value, [{compressed, 6}, {minor_version, 1}]).\n\n%% @doc Decodes the given internal representation of a client value.\n%%      TODO: use the stronger variant (commented out for the bug in rdht_tx_read:extract_from_tlog/4)\n-spec decode_value(encoded_value()) -> client_value().\n%% decode_value(Value) when is_atom(Value) -> Value;\n%% decode_value(Value) when is_boolean(Value) -> Value;\n%% decode_value(Value) when is_number(Value) -> Value;\ndecode_value(Value) when is_binary(Value) -> erlang:binary_to_term(Value);\ndecode_value(Value)                       -> Value.\n\n%% commit phase\n-spec commit(tx_tlog:tlog()) -> api_tx:commit_result().\n-ifdef(TXNEW).\ncommit(TLog) ->\n    %% set steering parameters, we need for the transactions engine:\n    %% number of retries, etc?\n    %% some parameters are checked via the individual operations\n    %% read, write which implement the behaviour tx_op_beh.\n    case tx_tlog:is_sane_for_commit(TLog) of\n        false -> {fail, abort, tx_tlog:get_insane_keys(TLog)};\n        true ->\n            Client = comm:this(),\n            ClientsId = {?commit_client_id, uid:get_global_uid()},\n            ?TRACE(\"rdht_tx:commit(Client ~p, ~p, TLog ~p)~n\", [Client, ClientsId, TLog]),\n            case pid_groups:find_a(tx_tm_new) of\n                failed ->\n                    Msg = io_lib:format(\"No tx_tm_new found.~n\", []),\n                    tx_tm:msg_commit_reply(Client, ClientsId, {abort, Msg});\n                TM ->\n                    tx_tm:commit(TM, Client, ClientsId, TLog)\n            end,\n            _Result =\n                begin\n                    trace_mpath:thread_yield(),\n                    %% log:log(\"Got execution right and finished commit ~p\", [self()]),\n                    receive\n                    ?SCALARIS_RECV(\n                       {tx_tm_commit_reply, ClientsId, commit}, %% ->\n                         {ok}  %% commit / abort;\n                      );\n                    ?SCALARIS_RECV(\n                       {tx_tm_commit_reply, ClientsId, {abort, FailedKeys}}, %% ->\n                         {fail, abort, FailedKeys} %% commit / abort;\n                       )\n                end end\n    end.\n-else.\ncommit(TLog) ->\n    %% set steering parameters, we need for the transactions engine:\n    %% number of retries, etc?\n    %% some parameters are checked via the individual operations\n    %% read, write which implement the behaviour tx_op_beh.\n    case tx_tlog:is_sane_for_commit(TLog) of\n        false -> {fail, abort, tx_tlog:get_insane_keys(TLog)};\n        true ->\n            Client = comm:this(),\n            ClientsId = {?commit_client_id, uid:get_global_uid()},\n            ?TRACE(\"rdht_tx:commit(Client ~p, ~p, TLog ~p)~n\", [Client, ClientsId, TLog]),\n            case pid_groups:find_a(tx_tm) of\n                failed ->\n                    Msg = io_lib:format(\"No tx_tm found.~n\", []),\n                    tx_tm_rtm:msg_commit_reply(Client, ClientsId, {abort, Msg});\n                TM ->\n                    tx_tm_rtm:commit(TM, Client, ClientsId, TLog)\n            end,\n            _Result =\n                begin\n                    trace_mpath:thread_yield(),\n                    receive\n                        ?SCALARIS_RECV(\n                           {tx_tm_rtm_commit_reply, ClientsId, commit}, %% ->\n                           {ok}  %% commit / abort;\n                          );\n                        ?SCALARIS_RECV(\n                           {tx_tm_rtm_commit_reply, ClientsId, {abort, FailedKeys}}, %% ->\n                           {fail, abort, FailedKeys} %% commit / abort;\n                          )\n                    end end\n    end.\n-endif.\n\n-spec receive_answer() -> {req_id(), tx_tlog:tlog_entry()}.\n-ifdef(TXNEW).\n receive_answer() ->\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n%%%% OLD TX\n% {tx_tm_rtm_commit_reply, _, _}, %%->\n           {tx_tm_commit_reply, _, _}, %%->\n           %% probably an outdated commit reply: drop it.\n             receive_answer()\n          );\n        ?SCALARIS_RECV(\n           {_Op, ReqId, RdhtTlog}, %% ->\n             {ReqId, RdhtTlog}\n          )\n    end.\n-else.\nreceive_answer() ->\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n           {tx_tm_rtm_commit_reply, _, _}, %%->\n           %% probably an outdated commit reply: drop it.\n           receive_answer()\n          );\n        ?SCALARIS_RECV(\n           {_Op, ReqId, RdhtTlog}, %% ->\n           {ReqId, RdhtTlog}\n          )\n        end.\n-endif.\n\n-spec receive_old_answers() -> ok.\n-ifdef(TXNEW).\nreceive_old_answers() ->\n    % note: do not yield trace_mpath thread with \"after 0\"!\n%%    receive\n%%%%%% OLD TX\n%%%   ?SCALARIS_RECV({tx_tm_rtm_commit_reply, _, _}, receive_old_answers());\n%%        ?SCALARIS_RECV({tx_tm_commit_reply, _, _}, receive_old_answers());\n%%        ?SCALARIS_RECV({_Op, _RdhtId, _RdhtTlog}, receive_old_answers())\n%%    after 0 -> ok\n%%    end.\n    ok.\n-else.\nreceive_old_answers() ->\n    % note: do not yield trace_mpath thread with \"after 0\"!\n    receive\n        ?SCALARIS_RECV({tx_tm_commit_reply, _, _}, receive_old_answers());\n        ?SCALARIS_RECV({_Op, _RdhtId, _RdhtTlog}, receive_old_answers())\n    after 0 -> ok\n    end.\n-endif.\n\n-spec req_get_op(api_tx:request_on_key())\n                -> read | write | add_del_on_list | add_on_nr | test_and_set.\nreq_get_op(Request) -> element(1, Request).\n-spec req_get_key(api_tx:request_on_key())\n                 -> api_tx:client_key().\nreq_get_key(Request) -> element(2, Request).\n\n%% @doc Checks whether used config parameters exist and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(tx_timeout) and\n    config:cfg_is_greater_than_equal(tx_timeout, 1000).\n\n"
  },
  {
    "path": "src/transactions/rdht_tx_add_del_on_list.erl",
    "content": "%% @copyright 2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc Part of replicated DHT implementation.\n%%      The add_del_on_list operation.\n%%      This \"two-phase\" operation uses rdht_tx_read to first read the value\n%%      and then alters the tlog entry so that it appears to be a write\n%%      operation from rdht_tx_write. Changes are performed in the context of\n%%      the client, not on the responsible node(s)!\n%% @version $Id$\n-module(rdht_tx_add_del_on_list).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([work_phase/3, extract_from_tlog/5]).\n\n% feeder for tester\n-export([extract_from_tlog_feeder/5]).\n\n\n-spec work_phase(pid(), rdht_tx:req_id() | rdht_tx_write:req_id(),\n                 api_tx:request()) -> ok.\nwork_phase(ClientPid, ReqId, Request) ->\n    rdht_tx_read:work_phase(ClientPid, ReqId, Request).\n\n-spec extract_from_tlog_feeder\n        (tx_tlog:tlog_entry(), client_key(), client_value(), client_value(), EnDecode::true)\n        -> {tx_tlog:tlog_entry(), client_key(), client_value(), client_value(), EnDecode::true};\n        (tx_tlog:tlog_entry(), client_key(), rdht_tx:encoded_value(), rdht_tx:encoded_value(), EnDecode::false)\n        -> {tx_tlog:tlog_entry(), client_key(), rdht_tx:encoded_value(), rdht_tx:encoded_value(), EnDecode::false}.\nextract_from_tlog_feeder(Entry, Key, ToAdd, ToDel, EnDecode) ->\n    % the result in the tlog is essentially a read op\n    % -> similar to rdht_tx_read:extract_from_tlog_feeder/4\n    {ValType, EncVal} = tx_tlog:get_entry_value(Entry),\n    NewEntry =\n        % transform unknow value types to a valid ?value type:\n        case not lists:member(ValType, [?value, ?not_found]) of\n            true -> tx_tlog:set_entry_value(Entry, ?value, EncVal);\n            _    -> Entry\n        end,\n    {NewEntry, Key, ToAdd, ToDel, EnDecode}.\n\n%% @doc Simulate a change on a set via read and write requests.\n%%      Update the TLog entry accordingly.\n-spec extract_from_tlog\n        (tx_tlog:tlog_entry(), client_key(), client_value(), client_value(), EnDecode::true)\n            -> {tx_tlog:tlog_entry(), api_tx:listop_result()};\n        (tx_tlog:tlog_entry(), client_key(), rdht_tx:encoded_value(), rdht_tx:encoded_value(), EnDecode::false)\n            -> {tx_tlog:tlog_entry(), api_tx:listop_result()}.\nextract_from_tlog(Entry, _Key, ToAdd, ToDel, true) when\n      (not erlang:is_list(ToAdd)) orelse\n      (not erlang:is_list(ToDel)) ->\n    %% input type error\n    {tx_tlog:set_entry_status(Entry, ?fail), {fail, not_a_list}};\nextract_from_tlog(Entry0, Key, ToAdd, ToDel, true) ->\n    {Entry, Res0} = rdht_tx_read:extract_from_tlog(Entry0, Key, read, true),\n    case Res0 of\n        {ok, OldValue} when erlang:is_list(OldValue) ->\n            %% types ok\n            case ToAdd =:= [] andalso ToDel =:= [] of\n                true -> {Entry, {ok}}; % no op\n                _ ->\n                    NewValue1 = lists:append(ToAdd, OldValue),\n                    NewValue2 = util:minus_first(NewValue1, ToDel),\n                    rdht_tx_write:extract_from_tlog(Entry, Key, NewValue2, true)\n            end;\n        {fail, not_found} -> %% key creation\n            NewValue2 = util:minus_first(ToAdd, ToDel),\n            rdht_tx_write:extract_from_tlog(Entry, Key, NewValue2, true);\n        {ok, _} -> %% value is not a list\n            {tx_tlog:set_entry_status(Entry, ?fail),\n             {fail, not_a_list}}\n    end;\nextract_from_tlog(Entry, Key, ToAdd, ToDel, false) ->\n    % note: we can only work with decoded values here\n    extract_from_tlog(Entry, Key, rdht_tx:decode_value(ToAdd), rdht_tx:decode_value(ToDel), true).\n"
  },
  {
    "path": "src/transactions/rdht_tx_add_on_nr.erl",
    "content": "%% @copyright 2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc Part of replicated DHT implementation.\n%%      The add_on_nr operation.\n%%      This \"two-phase\" operation uses rdht_tx_read to first read the value\n%%      and then alters the tlog entry so that it appears to be a write\n%%      operation from rdht_tx_write. Changes are performed in the context of\n%%      the client, not on the responsible node(s)!\n%% @version $Id$\n-module(rdht_tx_add_on_nr).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([work_phase/3, extract_from_tlog/4]).\n\n% feeder for tester\n-export([extract_from_tlog_feeder/4]).\n\n\n-spec work_phase(pid(), rdht_tx:req_id() | rdht_tx_write:req_id(),\n                 api_tx:request()) -> ok.\nwork_phase(ClientPid, ReqId, Request) ->\n    rdht_tx_read:work_phase(ClientPid, ReqId, Request).\n\n-spec extract_from_tlog_feeder\n        (tx_tlog:tlog_entry(), client_key(), client_value(), EnDecode::true)\n        -> {tx_tlog:tlog_entry(), client_key(), client_value(), EnDecode::true};\n        (tx_tlog:tlog_entry(), client_key(), rdht_tx:encoded_value(), EnDecode::false)\n        -> {tx_tlog:tlog_entry(), client_key(), rdht_tx:encoded_value(), EnDecode::false}.\nextract_from_tlog_feeder(Entry, Key, X, EnDecode) ->\n    % the result in the tlog is essentially a read op\n    % -> similar to rdht_tx_read:extract_from_tlog_feeder/4\n    {ValType, EncVal} = tx_tlog:get_entry_value(Entry),\n    NewEntry =\n        % transform unknow value types to a valid ?value type:\n        case not lists:member(ValType, [?value, ?not_found]) of\n            true -> tx_tlog:set_entry_value(Entry, ?value, EncVal);\n            _    -> Entry\n        end,\n    {NewEntry, Key, X, EnDecode}.\n\n%% @doc Simulate a change on a set via read and write requests.\n%%      Update the TLog entry accordingly.\n-spec extract_from_tlog\n        (tx_tlog:tlog_entry(), client_key(), client_value(), EnDecode::true)\n            -> {tx_tlog:tlog_entry(), api_tx:numberop_result()};\n        (tx_tlog:tlog_entry(), client_key(), rdht_tx:encoded_value(), EnDecode::false)\n            -> {tx_tlog:tlog_entry(), api_tx:numberop_result()}.\nextract_from_tlog(Entry, _Key, X, true) when (not erlang:is_number(X)) ->\n    %% check type of input data\n    {tx_tlog:set_entry_status(Entry, ?fail),\n     {fail, not_a_number}};\nextract_from_tlog(Entry0, Key, X, true) ->\n    {Entry, Res0} = rdht_tx_read:extract_from_tlog(Entry0, Key, read, true),\n    case Res0 of\n        {ok, OldValue} when erlang:is_number(OldValue) ->\n            %% types ok\n            case X == 0 of %% also accepts 0.0\n                true -> {Entry, {ok}}; % no op\n                _ ->\n                    NewValue = OldValue + X,\n                    rdht_tx_write:extract_from_tlog(Entry, Key, NewValue, true)\n            end;\n        {fail, not_found} -> %% key creation\n            rdht_tx_write:extract_from_tlog(Entry, Key, X, true);\n        {ok, _} -> %% value is not a number\n            {tx_tlog:set_entry_status(Entry, ?fail),\n             {fail, not_a_number}}\n    end;\nextract_from_tlog(Entry, Key, X, false) ->\n    % note: we can only work with decoded values here\n    extract_from_tlog(Entry, Key, rdht_tx:decode_value(X), true).\n"
  },
  {
    "path": "src/transactions/rdht_tx_read.erl",
    "content": "%% @copyright 2009-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Part of replicated DHT implementation.\n%%      The read operation.\n%% @version $Id$\n-module(rdht_tx_read).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n%% -define(TRACE_SNAP(X, Y), ct:pal(X, Y)).\n-define(TRACE_SNAP(X, Y), ?TRACE(X, Y)).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-behaviour(tx_op_beh).\n-export([work_phase/3, work_phase_key/5,\n         validate_prefilter/1, validate/3,\n         commit/5, abort/5,\n         extract_from_value/3, extract_from_tlog/4]).\n\n-behaviour(gen_component).\n-export([init/1, on/2]).\n\n-export([start_link/1]).\n-export([check_config/0]).\n\n% feeder for tester\n-export([extract_from_tlog_feeder/4]).\n\n-include(\"gen_component.hrl\").\n-include(\"rdht_tx_read_state.hrl\").\n\n%% reply messages a client should expect (when calling asynch work_phase/3)\n-spec msg_reply(rdht_tx:req_id() | rdht_tx_write:req_id(),\n                tx_tlog:tlog_entry()) -> comm:message().\nmsg_reply(Id, TLogEntry) ->\n    {rdht_tx_read_reply, Id, TLogEntry}.\n\n-spec work_phase(pid(), rdht_tx:req_id() | rdht_tx_write:req_id(),\n                 api_tx:request()) -> ok.\nwork_phase(ClientPid, ReqId, Request) ->\n    Key = element(2, Request),\n    Op = if element(1, Request) =/= read -> ?read; % e.g. {add_del_on_list, Key, ToAdd, ToRemove}\n            tuple_size(Request) =:= 3 ->\n                case element(3, Request) of\n                    random_from_list -> ?random_from_list;\n                     % let client crash if input data is not correct:\n                    {sublist, Start, Len} when is_integer(Start)\n                      andalso Start =/= 0 andalso is_integer(Len)\n                      -> {?sublist, Start, Len}\n                end;\n            true -> ?read\n         end,\n    HashedKey = ?RT:hash_key(Key),\n    work_phase_key(ClientPid, ReqId, Key, HashedKey, Op).\n\n-spec work_phase_key(comm:erl_local_pid(), rdht_tx:req_id() | rdht_tx_write:req_id(),\n                     client_key(), ?RT:key(),\n                     Op::?read | ?write | ?random_from_list | {?sublist, Start::pos_integer() | neg_integer(), Len::integer()})\n        -> ok.\n-ifdef(TXNEW).\nwork_phase_key(ClientPid, ReqId, Key, HashedKey, Op) ->\n    ?TRACE(\"rdht_tx_read:work_phase asynch~n\", []),\n    %% PRE: No entry for key in TLog\n    %% find rdht_tx_read process as collector\n    CollectorPid = pid_groups:find_a(?MODULE),\n    %% it is ok to pack the client_is info into the collector pid as\n    %% it is only send around in the local VM but not across the LAN\n    %% or WAN: first to the rbrcseq process and then to the\n    %% rdht_tx_read process. So the reply for the client can be build\n    %% in a single on handler and we do not need to store a state for\n    %% each request.\n    MyCollectorPid =\n        comm:reply_as(CollectorPid, 5,\n                      {work_phase_done, ClientPid, Key, Op, '_'}),\n    kv_on_cseq:work_phase_async(MyCollectorPid, ReqId, HashedKey, Op),\n\n    ok.\n-else.\nwork_phase_key(ClientPid, ReqId, Key, HashedKey, Op) ->\n    ?TRACE(\"rdht_tx_read:work_phase asynch~n\", []),\n    %% PRE: No entry for key in TLog\n    %% find rdht_tx_read process as collector\n    CollectorPid = pid_groups:find_a(?MODULE),\n    %% trigger quorum read\n    quorum_read(comm:make_global(CollectorPid), ReqId, HashedKey, Op),\n    %% inform CollectorPid on whom to inform after quorum reached\n    comm:send_local(CollectorPid, {client_is, ReqId, ClientPid, Key, Op}),\n    ok.\n-endif.\n\n%% not needed for newtx, but required by type_check_SUITE (excluded private fun)\n-spec quorum_read(CollectorPid::comm:mypid(), ReqId::rdht_tx:req_id() | rdht_tx_write:req_id(),\n                  HashedKey::?RT:key(),\n                  Op::?read | ?write | ?random_from_list | {?sublist, Start::pos_integer() | neg_integer(), Len::integer()})\n        -> ok.\nquorum_read(CollectorPid, ReqId, HashedKey, Op) ->\n    ?TRACE(\"rdht_tx_read:quorum_read ~p Collector: ~p~n\", [self(), CollectorPid]),\n    RKeys = ?RT:get_replica_keys(HashedKey),\n    _ = [ api_dht_raw:unreliable_lookup(\n            RKey, {?read_op, CollectorPid, ReqId, RKey, Op})\n        || RKey <- RKeys ],\n    ok.\n\n%% @doc Performs the requested operation in the dht_node context.\n-spec extract_from_value\n        (rdht_tx:encoded_value(), client_version(), Op::?read) -> Result::{?ok, rdht_tx:encoded_value(), client_version()};\n        (rdht_tx:encoded_value(), client_version(), Op::?random_from_list)\n            -> Result::{?ok, rdht_tx:encoded_value(), client_version()} | {?fail, ?empty_list | ?not_a_list, client_version()};\n        (rdht_tx:encoded_value(), client_version(), Op::{?sublist, Start::pos_integer() | neg_integer(), Len::integer()})\n            -> Result::{?ok, rdht_tx:encoded_value(), client_version()} | {?fail, ?not_a_list, client_version()};\n        (rdht_tx:encoded_value(), client_version(), Op::?write) -> Result::{?ok, ?value_dropped, client_version()};\n        (empty_val, -1, Op::?read | ?write) -> Result::{?ok, ?value_dropped, -1};\n        (empty_val, -1, Op::?random_from_list | {?sublist, Start::pos_integer() | neg_integer(), Len::integer()})\n            -> Result::{?fail, ?not_found, -1}.\nextract_from_value(empty_val, Version = -1, ?read) ->\n    {?ok, ?value_dropped, Version};\nextract_from_value(Value, Version, ?read) ->\n    {?ok, Value, Version};\nextract_from_value(empty_val, Version = -1, ?write) ->\n    {?ok, ?value_dropped, Version};\nextract_from_value(_Value, Version, ?write) ->\n    {?ok, ?value_dropped, Version};\nextract_from_value(empty_val, Version = -1, ?random_from_list) ->\n    {?fail, ?not_found, Version};\nextract_from_value(ValueEnc, Version, ?random_from_list) ->\n    Value = rdht_tx:decode_value(ValueEnc),\n    case Value of\n        [_|_]     -> RandVal_ListLen = util:randomelem_and_length(Value),\n                     {?ok, rdht_tx:encode_value(RandVal_ListLen), Version};\n        []        -> {?fail, ?empty_list, Version};\n        _         -> {?fail, ?not_a_list, Version}\n    end;\nextract_from_value(empty_val, Version = -1, {?sublist, _Start, _Len}) ->\n    {?fail, ?not_found, Version};\nextract_from_value(ValueEnc, Version, {?sublist, Start, Len}) ->\n    Value = rdht_tx:decode_value(ValueEnc),\n    if is_list(Value) ->\n           SubList_ListLen = util:sublist(Value, Start, Len),\n           {?ok, rdht_tx:encode_value(SubList_ListLen), Version};\n       true ->\n           {?fail, ?not_a_list, Version}\n    end.\n\n-spec extract_from_tlog_feeder(\n        tx_tlog:tlog_entry(), client_key(),\n        Op::read | random_from_list | {sublist, Start::pos_integer() | neg_integer(), Len::integer()},\n        EnDecode::boolean())\n        -> {tx_tlog:tlog_entry(), client_key(),\n            Op::read | random_from_list | {sublist, Start::pos_integer() | neg_integer(), Len::integer()},\n            EnDecode::boolean()}.\nextract_from_tlog_feeder(Entry, Key, read = Op, EnDecode) ->\n    {ValType, EncVal} = tx_tlog:get_entry_value(Entry),\n    NewEntry =\n        % transform unknow value types to a valid ?value type (this is allowed for any entry operation):\n        case not lists:member(ValType, [?value, ?not_found]) of\n            true -> tx_tlog:set_entry_value(Entry, ?value, EncVal);\n            _    -> Entry\n        end,\n    {NewEntry, Key, Op, EnDecode};\nextract_from_tlog_feeder(Entry, Key, random_from_list = Op, EnDecode) ->\n    {ValType, EncVal} = tx_tlog:get_entry_value(Entry),\n    NewEntry =\n        % transform unknow value types to a valid ?value type (this is allowed for any entry operation):\n        case not lists:member(ValType, [?value, ?not_found, {?fail, ?empty_list}, {?fail, ?not_a_list}]) of\n            true -> tx_tlog:set_entry_value(Entry, ?value, EncVal);\n            _    -> Entry\n        end,\n    {NewEntry, Key, Op, EnDecode};\nextract_from_tlog_feeder(Entry, Key, {sublist, _Start, _Len} = Op, EnDecode) ->\n    {ValType, EncVal} = tx_tlog:get_entry_value(Entry),\n    NewEntry =\n        % transform unknow value types to a valid ?value type (this is allowed for any entry operation):\n        case not lists:member(ValType, [?value, ?not_found, {?fail, ?not_a_list}]) of\n            true -> tx_tlog:set_entry_value(Entry, ?value, EncVal);\n            _    -> Entry\n        end,\n    {NewEntry, Key, Op, EnDecode}.\n\n%% @doc Get a result entry for a read from the given TLog entry.\n-spec extract_from_tlog\n        (tx_tlog:tlog_entry(), client_key(), Op::read, EnDecode::boolean())\n            -> {tx_tlog:tlog_entry(), api_tx:read_result()};\n        (tx_tlog:tlog_entry(), client_key(), Op::random_from_list, EnDecode::boolean())\n            -> {tx_tlog:tlog_entry(), api_tx:read_random_from_list_result()};\n        (tx_tlog:tlog_entry(), client_key(), Op::{sublist, Start::pos_integer() | neg_integer(), Len::integer()}, EnDecode::boolean())\n            -> {tx_tlog:tlog_entry(), api_tx:read_sublist_result()}.\nextract_from_tlog(Entry, _Key, read, EnDecode) ->\n    {ValType, EncVal} = tx_tlog:get_entry_value(Entry),\n    Res = case ValType of\n              ?value -> {ok, ?IIF(EnDecode, rdht_tx:decode_value(EncVal), EncVal)};\n              ?not_found -> {fail, not_found}\n          end,\n    {Entry, Res};\nextract_from_tlog(Entry, _Key, Op, EnDecode) ->\n    {ValType, EncVal} = tx_tlog:get_entry_value(Entry),\n    % note: Value and ValType can either contain the partial read or a failure\n    %       from the requested op - nothing else is possible since otherwise a\n    %       full read would have been executed!\n    case ValType of\n        ?partial_value ->\n            ClientVal =\n                case Op of\n                    random_from_list ->\n                        ?IIF(EnDecode, rdht_tx:decode_value(EncVal), EncVal);\n                    {sublist, _Start, _Len} ->\n                        ?IIF(EnDecode, rdht_tx:decode_value(EncVal), EncVal)\n                end,\n            {Entry, {ok, ClientVal}};\n        ?value ->\n            case Op of\n                random_from_list ->\n                    Value = rdht_tx:decode_value(EncVal),\n                    case Value of\n                        [_|_] ->\n                            DecodedVal = util:randomelem_and_length(Value),\n                            % note: if not EnDecode is given, an encoded value is\n                            % expected like the original value!\n                            ResVal = ?IIF(not EnDecode, rdht_tx:encode_value(DecodedVal),\n                                          DecodedVal),\n                            {Entry, {ok, ResVal}};\n                        _ ->\n                            Res = case Value of\n                                      []        -> {fail, empty_list};\n                                      _         -> {fail, not_a_list}\n                                  end,\n                            {tx_tlog:set_entry_status(Entry, ?fail), Res}\n                    end;\n                {sublist, Start, Len} ->\n                    Value = rdht_tx:decode_value(EncVal),\n                    if is_list(Value) ->\n                           DecodedVal = util:sublist(Value, Start, Len),\n                           % note: if not EnDecode is given, an encoded value is\n                           % expected like the original value!\n                            ResVal = ?IIF(not EnDecode, rdht_tx:encode_value(DecodedVal),\n                                          DecodedVal),\n                           {Entry, {ok, ResVal}};\n                       true ->\n                           {tx_tlog:set_entry_status(Entry, ?fail),\n                            {fail, not_a_list}}\n                    end\n            end;\n        ?not_found ->\n            Res = {fail, not_found},\n            case Op of\n                random_from_list ->\n                    {tx_tlog:set_entry_status(Entry, ?fail), Res};\n                {sublist, _Start, _Len} ->\n                    {tx_tlog:set_entry_status(Entry, ?fail), Res};\n                _ -> {Entry, Res}\n            end;\n        {?fail, ?empty_list} when Op =:= random_from_list -> {Entry, {fail, empty_list}};\n        {?fail, ?not_a_list} -> {Entry, {fail, not_a_list}}\n    end.\n\n%% May make several ones from a single TransLog item (item replication)\n%% validate_prefilter(TransLogEntry) ->\n%%   [TransLogEntries] (replicas)\n-spec validate_prefilter(tx_tlog:tlog_entry()) -> [tx_tlog:tlog_entry()].\nvalidate_prefilter(TLogEntry) ->\n    ?TRACE(\"rdht_tx_read:validate_prefilter(~p)~n\", [TLogEntry]),\n    Key = tx_tlog:get_entry_key(TLogEntry),\n    RKeys = ?RT:get_replica_keys(?RT:hash_key(Key)),\n    [ tx_tlog:set_entry_key(TLogEntry, X) || X <- RKeys ].\n\n%% validate the translog entry and return the proposal\n-spec validate(db_dht:db(), tx_tlog:snap_number(), tx_tlog:tlog_entry()) ->\n    {db_dht:db(), ?prepared | ?abort}.\nvalidate(DB, OwnSnapNumber, RTLogEntry) ->\n    ?TRACE(\"rdht_tx_read:validate)~n\", []),\n    %% contact DB to check entry\n    DBEntry = db_dht:get_entry(DB, tx_tlog:get_entry_key(RTLogEntry)),\n    VersionOK =\n        (tx_tlog:get_entry_version(RTLogEntry)\n         >= db_entry:get_version(DBEntry)),\n    Lockable = (false =:= db_entry:get_writelock(DBEntry)),\n    TLogSnapNo = tx_tlog:get_entry_snapshot(RTLogEntry),\n    SnapNumbersOK = (TLogSnapNo >= OwnSnapNumber),\n    if VersionOK andalso Lockable andalso SnapNumbersOK ->\n           %% if a snapshot instance is running, copy old value to snapshot db before setting lock\n           %% set locks on entry\n           NewEntry = db_entry:inc_readlock(DBEntry),\n           NewDB = db_dht:set_entry(DB, NewEntry, TLogSnapNo, OwnSnapNumber),\n           {NewDB, ?prepared};\n       true ->\n           {DB, ?abort}\n    end.\n\n-spec commit(db_dht:db(), tx_tlog:tlog_entry(), OwnProposalWas::?prepared | ?abort,\n             tx_tlog:snap_number(), tx_tlog:snap_number()) -> db_dht:db().\ncommit(DB, _RTLogEntry, ?abort, _TMSnapNo, _OwnSnapNo) ->\n    %% own proposal was abort, so no lock to clean up\n    %% we could compare DB with RTLogEntry and update if outdated\n    %% as this commit confirms the status of a majority of the\n    %% replicas. Could also be possible already in the validate req?\n    DB;\ncommit(DB, RTLogEntry, ?prepared, _TMSnapNo, OwnSnapNo) ->\n    ?TRACE(\"rdht_tx_read:commit)~n\", []),\n    %% perform op: nothing to do for 'read'\n    %% release locks\n    DBEntry = db_dht:get_entry(DB, tx_tlog:get_entry_key(RTLogEntry)),\n    RTLogVers = tx_tlog:get_entry_version(RTLogEntry),\n    DBVers = db_entry:get_version(DBEntry),\n    if RTLogVers =:= DBVers ->\n            NewEntry = db_entry:dec_readlock(DBEntry),\n            TLogSnapNo = tx_tlog:get_entry_snapshot(RTLogEntry),\n            db_dht:set_entry(DB, NewEntry, TLogSnapNo, OwnSnapNo);\n        true -> DB %% a write has already deleted this lock\n    end.\n\n-spec abort(db_dht:db(), tx_tlog:tlog_entry(), ?prepared | ?abort,\n            tx_tlog:snap_number(), tx_tlog:snap_number()) -> db_dht:db().\nabort(DB, RTLogEntry, OwnProposalWas, TMSnapNo, OwnSnapNo) ->\n    ?TRACE(\"rdht_tx_read:abort)~n\", []),\n    %% same as when committing\n    commit(DB, RTLogEntry, OwnProposalWas, TMSnapNo, OwnSnapNo).\n\n\n%% be startable via supervisor, use gen_component\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2,\n                             [],\n                             [{pid_groups_join_as, DHTNodeGroup, ?MODULE}]).\n\n-type state() :: {pos_integer(), pos_integer(), pos_integer(), atom()}.\n\n%% initialize: return initial state.\n-spec init([]) -> state().\ninit([]) ->\n    ?TRACE(\"rdht_tx_read: Starting rdht_tx_read for DHT node: ~p~n\",\n           [pid_groups:my_groupname()]),\n    %% For easier debugging, use a named table (generates an atom)\n    %%TableName = erlang:list_to_atom(pid_groups:group_to_filename(pid_groups:my_groupname()) ++ \"_rdht_tx_read\"),\n    %%Table = pdb:new(TableName, [set, protected, named_table]),\n    %% use random table name provided by ets to *not* generate an atom\n    Table = pdb:new(?MODULE, [set]),\n    Reps = config:read(replication_factor),\n    MajOk = quorum:majority_for_accept(Reps),\n    MajDeny = quorum:majority_for_deny(Reps),\n\n    _State = {Reps, MajOk, MajDeny, Table}.\n\n-spec on(comm:message(), state()) -> state().\n%% reply triggered by api_dht_raw:unreliable_get_key/3\non({?read_op_with_id_reply, Id, SnapNumber, Ok_Fail, Val_Reason, Vers},\n   {Reps, MajOk, MajDeny, Table} = State) ->\n    ?TRACE(\"~p rdht_tx_read:on(get_key_with_id_reply) ID ~p~n\", [self(), Id]),\n    Entry = get_entry(Id, Table),\n    %% @todo inform sender when its entry is outdated?\n    %% @todo inform former sender on outdated entry when we\n    %% get a newer entry?\n    TmpEntry = state_add_reply(Entry, {Ok_Fail, Val_Reason, Vers}, SnapNumber),\n    _ = case state_get_client(TmpEntry) of\n            unknown ->\n                %% when we get a client, we will inform it\n                pdb:set(TmpEntry, Table);\n            Client ->\n                decide_set_and_inform_client_if_ready(Client, TmpEntry, Reps, MajOk, MajDeny, Table)\n        end,\n    State;\n\n%% triggered by ?MODULE:work_phase/3\non({client_is, Id, Pid, Key, Op}, {Reps, MajOk, MajDeny, Table} = State) ->\n    ?TRACE(\"~p rdht_tx_read:on(client_is)~n\", [self()]),\n    Entry = get_entry(Id, Table),\n    Tmp0Entry = state_set_op(Entry, Op),\n    Tmp1Entry = state_set_client(Tmp0Entry, Pid),\n    TmpEntry = state_set_key(Tmp1Entry, Key),\n    decide_set_and_inform_client_if_ready(Pid, TmpEntry, Reps, MajOk, MajDeny, Table),\n%    State;\n%\n%%% triggered periodically\n%on({timeout_id, Id}, {_Reps, _MajOk, _MajDeny, Table} = State) ->\n%    ?TRACE(\"~p rdht_tx_read:on(timeout) Id ~p~n\", [self(), Id]),\n%    case pdb:get(Id, Table) of\n%        undefined -> ok;\n%        Entry ->\n%            %% inform client on timeout if Id exists and client is not informed\n%            timeout_inform(Entry),\n%            pdb:delete(Id, Table)\n%    end,\n    State;\n\n%% used by new tx protocol\non({work_phase_done, ClientPid, Key, Op,\n    {work_phase_async_done, Id, {qread_done, _, _, _, {Val, Vers}}}},\n   {_Reps, _MajOk, _MajDeny, Table} = State) ->\n    Entry = get_entry(Id, Table),\n    T1 = state_set_op(Entry, Op),\n    T2 = state_set_client(T1, ClientPid),\n    T3 = state_set_key(T2, Key),\n\n    T4 = state_set_result(T3, {?ok, Val, Vers}),\n%%    log:log(\"Seeing: ~p ~p~n\", [Val, Vers]),\n    ValType = case Vers of\n                  -1 -> ?not_found;\n                  _ -> ?value\n              end,\n    T5 = case ValType of\n             ?not_found ->\n                 case Op of\n                     {?sublist, _, _} ->\n                         state_set_decided(T4, ?fail);\n                     ?random_from_list ->\n                         state_set_decided(T4, ?fail);\n                     _ ->\n                         state_set_decided(T4, ?ok)\n                 end;\n             ?value -> state_set_decided(T4, ?ok)\n         end,\n    set_and_inform_client(ClientPid, T5, 1, Table, ValType),\n    State.\n\n-spec get_entry(rdht_tx:req_id(), atom()) -> read_state().\nget_entry(Id, Table) ->\n    case pdb:get(Id, Table) of\n        undefined ->\n%%            msg_delay:send_local(config:read(transaction_lookup_timeout) div 1000,\n%%                                 self(), {timeout_id, Id}),\n            state_new(Id);\n        Any -> Any\n    end.\n\n% -spec timeout_inform(read_state()) -> ok.\n% %% inform client on timeout if Id exists and client is not informed yet\n% timeout_inform(Entry) ->\n%     case {state_is_client_informed(Entry),\n%           state_get_client(Entry)} of\n%         {_, unknown} -> ok;\n%         {false, Client} ->\n%             TmpEntry = state_set_decided(Entry, {fail, timeout}),\n%             inform_client(Client, TmpEntry);\n%         _ -> ok\n%     end.\n\n%% @doc Takes a decision (if possible) and informs the client once a decision\n%%      has been taken and the client is not\n%%      informed yet. Also sets the state into the pdb.\n-spec decide_set_and_inform_client_if_ready(pid(), read_state(), Reps::pos_integer(),\n                                     MajOk::pos_integer(), MajDeny::pos_integer(),\n                                     Table::atom()) -> ok.\ndecide_set_and_inform_client_if_ready(Client, Entry, Reps, MajOk, MajDeny, Table) ->\n    case state_is_client_informed(Entry) of\n        false ->\n            % false = state_get_decided(Entry), % should always be true here!\n            NumReplied = state_get_numreplied(Entry),\n\n            % if majority replied, we can given an answer!\n            % (but: need all Reps replicas for not_found)\n            if NumReplied >= MajOk ->\n                   {Ok_Fail, Val_Reason, Vers} = state_get_result(Entry),\n                   % ?read | ?write | ?random_from_list | {?sublist, Start::pos_integer() | neg_integer(), Len::integer()}\n                   Op = state_get_op(Entry),\n                   if Vers >= 0 ->\n                          NumAbort = state_get_numfailed(Entry),\n                          % note: MajDeny =:= 2, so 2 times not_found results\n                          % in {fail, not_found} although the 3rd reply may have\n                          % contained an actual value. This is to make sure that\n                          % if 'a' is written, then 'b' and then 'c', no matter\n                          % what happens during the write of 'c', 'a' is never\n                          % read again!\n                          % Example: b is written       => (a, b, b, b), then\n                          %          c is being written => (a, b+wl, b+wl, c)\n                          %          (crash during write), then\n                          %          churn happens,     => (a, not_found, not_found, c)\n                          %          => could read (a, not_found, not_found)\n                          %             but should not return a!\n                          % note: 2 failures are normally not supported but we\n                          % want to be on the safe side here\n                          Entry2 =\n                              if NumAbort >= MajDeny andalso Op =:= ?write ->\n                                     % note: there is no error for write ops so\n                                     % far (the user does not get a value) but\n                                     % we can send our known version for write\n                                     % ops to be committed\n                                     ValType = ?value_dropped,\n                                     state_set_decided(Entry, Ok_Fail);\n                                 NumAbort >= MajDeny ->\n                                     % note: not_found is reported to the user,\n                                     % but we know there is a newer version\n                                     % -> decide for fail\n                                     % also make sure to report the according\n                                     % result, especially the version!\n                                     Entry1 = state_set_decided(Entry, ?fail),\n                                     ValType = ?not_found,\n                                     state_set_result(Entry1, {Ok_Fail, empty_val, -1});\n                                 Ok_Fail =:= ?ok ->\n                                      NumOk = state_get_numok(Entry),\n                                      ValType = if Op =:= ?read andalso NumOk >= MajDeny ->\n                                                        %% a value visible in every majority r=4: at least in 2 visible\n                                                        ?value;\n                                                   Op =:= ?read ->\n                                 %% maybe a value not visible in every majority\n\n                                 %% (a) if we return not_found a read after a\n                                 %% write could return not_found, which is wrong\n                                 %% (majority has value, but not yet the\n                                 %% complete remaining minority).\n\n                                 %% (b) Returning the partly written value\n                                 %% (persistent in a minority of the replicas)\n                                 %% is also problematic, a subsequent read may\n                                 %% return not_found when it accesses other\n                                 %% replicas.\n\n                                 %% We have no correct solution here.\n                                 %% The solution is done in the new\n                                 %% rbrcseq.\n\n                                 %% We return the value as\n                                 %% api_tx_SUITE:random_write_read needs it.\n\n                                                        %% ?not_found;\n                                                        ?value;\n                                                   true ->\n                                                        ?partial_value\n                                               end,\n                                     state_set_decided(Entry, Ok_Fail);\n                                 Ok_Fail =:= ?fail ->\n                                     ValType = {?fail, Val_Reason},\n                                     state_set_decided(Entry, Ok_Fail)\n                              end,\n                          % a decision was taken in any of the cases\n                          % -> inform the client\n                          set_and_inform_client(Client, Entry2, Reps, Table, ValType);\n                      true ->\n                          if NumReplied =:= Reps ->\n                                 % all replied with -1\n                                 ValType = if Op =:= ?write -> ?value_dropped;\n                                              true          -> ?not_found\n                                           end,\n                                 Entry2 = state_set_decided(Entry, Ok_Fail),\n                                 set_and_inform_client(Client, Entry2, Reps, Table, ValType);\n                             true -> pdb:set(Entry, Table)\n                          end\n                   end;\n               true -> pdb:set(Entry, Table)\n            end;\n        true -> set_or_delete_if_all_replied(Entry, Reps, Table)\n    end.\n\n%% @doc Informs the client, updates the read state accordingly and sets it in\n%%      the pdb.\n-spec set_and_inform_client(pid(), read_state_decided(), Reps::pos_integer(),\n                            Table::atom(), ValType::?value | ?partial_value | ?not_found | ?value_dropped)\n        -> ok.\nset_and_inform_client(Client, Entry, Reps, Table, ValType) ->\n    Id = state_get_id(Entry),\n    TLogEntry = make_tlog_entry(Entry, ValType),\n    Msg = msg_reply(Id, TLogEntry),\n    comm:send_local(Client, Msg),\n    Entry2 = state_set_client_informed(Entry),\n    set_or_delete_if_all_replied(Entry2, Reps, Table).\n\n-compile({nowarn_unused_function, {make_tlog_entry_feeder, 2}}).\n-spec make_tlog_entry_feeder(\n        read_state_decided(), {ValType::?value | ?partial_value | ?not_found | ?value_dropped, ?RT:key()})\n        -> {read_state_decided(), ?value | ?partial_value | ?not_found | ?value_dropped}.\nmake_tlog_entry_feeder(Entry, {ValType, Key}) ->\n    %% when make_tlog_entry is called, the key of the state is never\n    %% 'unknown'\n    {state_set_key(Entry, Key), ValType}.\n\n%% @doc Creates a tlog entry from a read state.\n%% Pre: a decision must be present in the read state\n-spec make_tlog_entry(read_state_decided(), ValType::?value | ?partial_value | ?not_found | ?value_dropped)\n        -> tx_tlog:tlog_entry().\nmake_tlog_entry(Entry, ValType) ->\n    {_, Val0, Vers} = state_get_result(Entry),\n    Key = state_get_key(Entry),\n    Status = state_get_decided(Entry),\n    Val = case ValType of\n              ?value_dropped -> ?value_dropped;\n              ?not_found     -> ?value_dropped;\n              {?fail, _}     -> ?value_dropped;\n              _              -> Val0\n          end,\n    SnapNumber = state_get_snapshot_number(Entry),\n    tx_tlog:new_entry(?read, Key, Vers, Status, SnapNumber, ValType, Val).\n\n-spec set_or_delete_if_all_replied(read_state_decided(),\n                                   pos_integer(), atom()) -> ok.\nset_or_delete_if_all_replied(Entry, Reps, Table) ->\n    ?TRACE(\"rdht_tx_read:delete_if_all_replied Reps: ~p =?= ~p, ClientInformed: ~p Client: ~p~n\",\n              [Reps, state_get_numreplied(Entry), state_is_client_informed(Entry), state_get_client(Entry)]),\n    case (Reps =:= state_get_numreplied(Entry))\n        andalso (state_is_client_informed(Entry)) of\n        true  -> pdb:delete(state_get_id(Entry), Table);\n        false -> pdb:set(Entry, Table)\n    end.\n\n%% @doc Checks whether config parameters for rdht_tx_read exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(replication_factor) and\n    config:cfg_is_greater_than(replication_factor, 0).\n"
  },
  {
    "path": "src/transactions/rdht_tx_read_state.hrl",
    "content": "% @copyright 2009-2012 Zuse Institute Berlin,\n%                      onScale solutions GmbH\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@onscale.de>\n%% @doc Part of replicated DHT implementation.\n%%      The abstract datatype for the state for processing read operations.\n%% @version $Id$\n\n-compile({inline, [state_get_id/1,\n                   state_get_client/1, state_set_client/2,\n                   state_get_key/1, state_set_key/2,\n                   state_get_numok/1, state_inc_numok/1,\n                   state_get_numfailed/1, state_inc_numfailed/1,\n                   state_get_result/1, state_set_result/2,\n                   state_get_decided/1, state_set_decided/2,\n                   state_is_client_informed/1, state_set_client_informed/1,\n                   state_get_numreplied/1,\n                   state_get_snapshot_number/1, state_set_snapshot_number/2\n                  ]}).\n\n% {?ok|fail, Val|FailReason, Vers}\n-type result() :: {?ok | ?fail,\n                   rdht_tx:encoded_value() | 0 | ?not_found | ?empty_list | ?not_a_list,\n                   -1 | client_version()}.\n-type read_state() ::\n                  { ID               :: rdht_tx:req_id(),\n                    ClientPid        :: pid() | unknown,\n                    Key              :: ?RT:key() | unknown,\n                    NumOk            :: non_neg_integer(),\n                    NumFail          :: non_neg_integer(),\n                    Result           :: result() | {?fail, 0, -2},\n                    IsDecided        :: tx_tlog:tx_status() | false,\n                    IsClientInformed :: boolean(),\n                    SnapNumber       :: tx_tlog:snap_number(),\n                    Op               :: ?read | ?write | ?random_from_list | {?sublist, Start::pos_integer() | neg_integer(), Len::integer()}\n                  }.\n-type read_state_decided() ::\n                  { ID               :: rdht_tx:req_id(),\n                    ClientPid        :: pid() | unknown,\n                    Key              :: ?RT:key() | unknown,\n                    NumOk            :: non_neg_integer(),\n                    NumFail          :: non_neg_integer(),\n                    Result           :: result(),\n                    IsDecided        :: tx_tlog:tx_status(),\n                    IsClientInformed :: boolean(),\n                    SnapNumber       :: tx_tlog:snap_number(),\n                    Op               :: ?read | ?write | ?random_from_list | {?sublist, Start::pos_integer() | neg_integer(), Len::integer()}\n                  }.\n\n-spec state_new(rdht_tx:req_id()) -> read_state().\nstate_new(Id) ->\n    {Id, unknown, unknown, 0, 0, {?fail, 0, -2}, false, false, 0, ?read}.\n\n-spec state_get_id(read_state()) -> rdht_tx:req_id().\nstate_get_id(State) ->              element(1, State).\n-spec state_get_client(read_state()) -> pid() | unknown.\nstate_get_client(State) ->          element(2, State).\n-spec state_set_client(read_state(), pid()) -> read_state().\nstate_set_client(State, Pid) ->     setelement(2, State, Pid).\n-spec state_get_key(read_state()) -> ?RT:key() | unknown.\nstate_get_key(State) ->             element(3, State).\n-spec state_set_key(read_state(), ?RT:key()) -> read_state().\nstate_set_key(State, Key) ->        setelement(3, State, Key).\n\n-spec state_get_numok(read_state()) -> non_neg_integer().\nstate_get_numok(State) ->           element(4, State).\n-spec state_inc_numok(read_state()) -> read_state().\nstate_inc_numok(State) ->           setelement(4, State, element(4, State) + 1).\n-spec state_get_numfailed(read_state()) -> non_neg_integer().\nstate_get_numfailed(State) ->       element(5, State).\n-spec state_inc_numfailed(read_state()) -> read_state().\nstate_inc_numfailed(State) ->       setelement(5, State, element(5, State) + 1).\n-spec state_get_result(read_state()) -> result() | {?fail, 0, -2}.\nstate_get_result(State) ->          element(6, State).\n-spec state_set_result(read_state(), result()) -> read_state().\nstate_set_result(State, Val) ->     setelement(6, State, Val).\n-spec state_get_decided(read_state()) -> tx_tlog:tx_status() | false.\nstate_get_decided(State) ->         element(7, State).\n-spec state_set_decided(read_state(), tx_tlog:tx_status()) -> read_state().\nstate_set_decided(State, Val) ->    setelement(7, State, Val).\n-spec state_is_client_informed(read_state()) -> boolean().\nstate_is_client_informed(State) ->  element(8, State).\n-spec state_set_client_informed(read_state()) -> read_state().\nstate_set_client_informed(State) -> setelement(8, State, true).\n-spec state_get_snapshot_number(read_state()) -> tx_tlog:snap_number().\nstate_get_snapshot_number(State) -> element(9, State).\n-spec state_set_snapshot_number(read_state(), tx_tlog:snap_number()) -> read_state().\nstate_set_snapshot_number(State, Val) -> setelement(9, State, Val).\n-spec state_get_op(read_state()) -> ?read | ?write | ?random_from_list | {?sublist, Start::pos_integer() | neg_integer(), Len::integer()}.\nstate_get_op(State) ->          element(10, State).\n-spec state_set_op(read_state(), ?read | ?write | ?random_from_list | {?sublist, Start::pos_integer() | neg_integer(), Len::integer()}) -> read_state().\nstate_set_op(State, Op) ->     setelement(10, State, Op).\n\n-spec state_get_numreplied(read_state()) -> non_neg_integer().\nstate_get_numreplied(State) ->\n    state_get_numok(State) + state_get_numfailed(State).\n\n-spec state_add_reply(read_state(),\n                      Result::{?ok, rdht_tx:encoded_value(), client_version()} | {?ok, empty_val, -1} |\n                          {?fail, ?not_found | ?empty_list | ?not_a_list,\n                           client_version()}, tx_tlog:snap_number())\n        -> read_state().\nstate_add_reply(State, Result, SnapNumber) ->\n    ?TRACE(\"state_add_reply state res majok majdeny ~p ~p ~n\", [State, Result]),\n    Vers = element(3, Result),\n    OldVers = element(3, state_get_result(State)),\n    TmpState = if Vers > OldVers -> state_set_result(State, Result);\n                  true           -> State\n               end,\n    NewState = if Vers >= 0 -> state_inc_numok(TmpState);\n       true      -> state_inc_numfailed(TmpState)\n    end,\n    state_set_snapshot_number(NewState, SnapNumber).\n"
  },
  {
    "path": "src/transactions/rdht_tx_test_and_set.erl",
    "content": "%% @copyright 2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc Part of replicated DHT implementation.\n%%      The test_and_set operation.\n%%      This \"two-phase\" operation uses rdht_tx_read to first read the value\n%%      and then alters the tlog entry so that it appears to be a write\n%%      operation from rdht_tx_write. Changes are performed in the context of\n%%      the client, not on the responsible node(s)!\n%% @version $Id$\n-module(rdht_tx_test_and_set).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([work_phase/3, extract_from_tlog/5]).\n\n% feeder for tester\n-export([extract_from_tlog_feeder/5]).\n\n\n-spec work_phase(pid(), rdht_tx:req_id() | rdht_tx_write:req_id(),\n                 api_tx:request()) -> ok.\nwork_phase(ClientPid, ReqId, Request) ->\n    rdht_tx_read:work_phase(ClientPid, ReqId, Request).\n\n-spec extract_from_tlog_feeder\n        (tx_tlog:tlog_entry(), client_key(), client_value(), client_value(), EnDecode::true)\n        -> {tx_tlog:tlog_entry(), client_key(), client_value(), client_value(), EnDecode::true};\n        (tx_tlog:tlog_entry(), client_key(), rdht_tx:encoded_value(), rdht_tx:encoded_value(), EnDecode::false)\n        -> {tx_tlog:tlog_entry(), client_key(), rdht_tx:encoded_value(), rdht_tx:encoded_value(), EnDecode::false}.\nextract_from_tlog_feeder(Entry, Key, Old, New, EnDecode) ->\n    % the result in the tlog is essentially a read op\n    % -> similar to rdht_tx_read:extract_from_tlog_feeder/4\n    {ValType, EncVal} = tx_tlog:get_entry_value(Entry),\n    NewEntry =\n        % transform unknow value types to a valid ?value type:\n        case not lists:member(ValType, [?value, ?not_found]) of\n            true -> tx_tlog:set_entry_value(Entry, ?value, EncVal);\n            _    -> Entry\n        end,\n    {NewEntry, Key, Old, New, EnDecode}.\n\n%% @doc Get a result entry for a test_and_set operation from the given TLog entry.\n-spec extract_from_tlog\n        (tx_tlog:tlog_entry(), client_key(), client_value(), client_value(), EnDecode::true)\n            -> {tx_tlog:tlog_entry(), api_tx:testandset_result()};\n        (tx_tlog:tlog_entry(), client_key(), rdht_tx:encoded_value(), rdht_tx:encoded_value(), EnDecode::false)\n            -> {tx_tlog:tlog_entry(), api_tx:testandset_result()}.\nextract_from_tlog(Entry0, Key, Old, New, EnDecode) ->\n    {Entry, Res0} = rdht_tx_read:extract_from_tlog(Entry0, Key, read, EnDecode),\n    case Res0 of\n        {ok, Old} ->\n            rdht_tx_write:extract_from_tlog(Entry, Key, New, EnDecode);\n        {ok, RealOldValue} when not EnDecode ->\n            % maybe there is just a different compression scheme\n            % -> test decoded values\n            Res0Decoded = rdht_tx:decode_value(RealOldValue),\n            OldDecoded = rdht_tx:decode_value(Old),\n            if Res0Decoded =:= OldDecoded ->\n                   rdht_tx_write:extract_from_tlog(Entry, Key, New, EnDecode);\n               true ->\n                   {tx_tlog:set_entry_status(Entry, ?fail),\n                    {fail, {key_changed, RealOldValue}}}\n            end;\n        {ok, RealOldValue} ->\n            {tx_tlog:set_entry_status(Entry, ?fail),\n             {fail, {key_changed, RealOldValue}}};\n        {fail, not_found} = X ->\n            {tx_tlog:set_entry_status(Entry, ?fail), X}\n    end.\n"
  },
  {
    "path": "src/transactions/rdht_tx_write.erl",
    "content": "%% @copyright 2009-2015 Zuse Institute Berlin\n%%                 onScale solutions GmbH\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@onscale.de>\n%% @doc Part of replicated DHT implementation.\n%%      The write operation.\n%% @version $Id$\n-module(rdht_tx_write).\n-author('schintke@onscale.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n%% -define(TRACE_SNAP(X, Y), ct:pal(X, Y)).\n-define(TRACE_SNAP(X, Y), ?TRACE(X, Y)).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-behaviour(tx_op_beh).\n-export([work_phase/3,\n         validate_prefilter/1, validate/3,\n         commit/5, abort/5,\n         extract_from_tlog/4]).\n\n-behaviour(gen_component).\n-export([init/1, on/2]).\n-export([start_link/1]).\n-export([check_config/0]).\n\n-export_type([req_id/0]).\n\n-include(\"gen_component.hrl\").\n\n-type req_id() :: {rdht_tx:req_id(), pid(), any()}.\n\n%% reply messages a client should expect (when calling asynch work_phase/3)\n-spec msg_reply(req_id(), tx_tlog:tlog_entry())\n               -> comm:message().\nmsg_reply(Id, TLogEntry) ->\n    {rdht_tx_write_reply, Id, TLogEntry}.\n\n-spec work_phase(pid(), rdht_tx:req_id(), api_tx:request()) -> ok.\nwork_phase(ClientPid, ReqId, Request) ->\n    ?TRACE(\"rdht_tx_write:work_phase asynch~n\", []),\n    %% Find rdht_tx_write process\n    RdhtTxWritePid = pid_groups:find_a(?MODULE),\n    % hash key here so that any error during the process is thrown in the client's context\n    Key = element(2, Request),\n    HashedKey = ?RT:hash_key(Key),\n    comm:send_local(RdhtTxWritePid, {start_work_phase, ReqId, ClientPid, HashedKey, Key}).\n\n%% @doc Get a result entry for a write from the given TLog entry.\n%%      Update the TLog entry accordingly.\n-spec extract_from_tlog\n        (tx_tlog:tlog_entry(), client_key(), client_value(), EnDecode::true) ->\n                       {tx_tlog:tlog_entry(), api_tx:write_result()};\n        (tx_tlog:tlog_entry(), client_key(), rdht_tx:encoded_value(), EnDecode::false) ->\n                       {tx_tlog:tlog_entry(), api_tx:write_result()}.\nextract_from_tlog(Entry, _Key, Value1, EnDecode) ->\n    Value = ?IIF(EnDecode, rdht_tx:encode_value(Value1), Value1),\n    E1 = tx_tlog:set_entry_value(Entry, ?value, Value),\n    case tx_tlog:get_entry_operation(E1) of\n        ?write -> {E1, {ok}};\n        ?read  -> {tx_tlog:set_entry_operation(E1, ?write), {ok}}\n    end.\n\n%% May make several ones from a single TransLog item (item replication)\n%% validate_prefilter(TransLogEntry) ->\n%%   [TransLogEntries] (replicas)\n-spec validate_prefilter(tx_tlog:tlog_entry()) -> [tx_tlog:tlog_entry()].\nvalidate_prefilter(TLogEntry) ->\n    ?TRACE(\"rdht_tx_write:validate_prefilter(~p)~n\", [TLogEntry]),\n    Key = tx_tlog:get_entry_key(TLogEntry),\n    RKeys = ?RT:get_replica_keys(?RT:hash_key(Key)),\n    [ tx_tlog:set_entry_key(TLogEntry, X) || X <- RKeys ].\n\n%% validate the translog entry and return the proposal\n-spec validate(db_dht:db(), tx_tlog:snap_number(), tx_tlog:tlog_entry()) ->\n    {db_dht:db(), ?prepared | ?abort}.\nvalidate(DB, OwnSnapNumber, RTLogEntry) ->\n    %% contact DB to check entry\n    %% set locks on DB\n    DBEntry = db_dht:get_entry(DB, tx_tlog:get_entry_key(RTLogEntry)),\n\n    RTVers = tx_tlog:get_entry_version(RTLogEntry),\n    DBVers = db_entry:get_version(DBEntry),\n\n    %% Note: RTVers contains the latest version in the system (if not outdated).\n    %%       We can not update the DB entry's version with it though since this\n    %%       would need the old value in the rtlog in case of rollback and to\n    %%       serve read requests properly (version and value have to be changed\n    %%       atomically as a pair).\n    ReadLocks = db_entry:get_readlock(DBEntry),\n    WriteLock = db_entry:get_writelock(DBEntry),\n    ?TRACE(\"rdht_tx_write:validate: local snapnumber is ~p; snapnumber in tlog\n           entry is ~p~n\",[OwnSnapNumber,tx_tlog:get_entry_snapshot(RTLogEntry)]),\n    TLogSnapNo = tx_tlog:get_entry_snapshot(RTLogEntry),\n    SnapNumbersOK = (TLogSnapNo >= OwnSnapNumber),\n    if ((RTVers =:= DBVers andalso ReadLocks =:= 0) orelse RTVers > DBVers) andalso\n           (WriteLock =:= false orelse WriteLock < RTVers) andalso SnapNumbersOK->\n           %% set locks on entry (use RTVers for write locks to allow proper\n           %% handling of outdated commit and abort messages - only clean up\n           %% if the write lock version matches!)\n           NewEntry = db_entry:set_writelock(DBEntry, RTVers),\n           NewDB = db_dht:set_entry(DB, NewEntry, TLogSnapNo, OwnSnapNumber),\n            ?TRACE_SNAP(\"rdht_tx_write:validate prepare: ~p~n~p  ~p~n~p~n~p~n~p\",\n                        [comm:this(), tx_tlog:get_entry_snapshot(RTLogEntry),\n                         OwnSnapNumber, NewEntry, DB, NewDB]),\n           {NewDB, ?prepared};\n       true ->\n            ?TRACE_SNAP(\"rdht_tx_write:validate abort: ~p~n~p  ~p\",\n                        [comm:this(), tx_tlog:get_entry_snapshot(RTLogEntry),\n                         OwnSnapNumber]),\n           {DB, ?abort}\n    end.\n\n-spec commit(db_dht:db(), tx_tlog:tlog_entry(), ?prepared | ?abort,\n             tx_tlog:snap_number(), tx_tlog:snap_number()) -> db_dht:db().\ncommit(DB, RTLogEntry, _OwnProposalWas, _TMSnapNo, OwnSnapNo) ->\n    ?TRACE(\"rdht_tx_write:commit)~n\", []),\n    DBEntry = db_dht:get_entry(DB, tx_tlog:get_entry_key(RTLogEntry)),\n    %% perform op\n    RTLogVers = tx_tlog:get_entry_version(RTLogEntry),\n    %% Note: if false =/= WriteLock -> WriteLock is always >= own DBVers!\n    DBVers = db_entry:get_version(DBEntry),\n    WriteLock = db_entry:get_writelock(DBEntry),\n    if DBVers =< RTLogVers ->\n            {?value, Value} = tx_tlog:get_entry_value(RTLogEntry),\n            T2DBEntry = db_entry:set_value(DBEntry, Value, RTLogVers + 1),\n            NewEntry =\n                if WriteLock =/= false andalso WriteLock =< RTLogVers ->\n                        %% op that created the write lock or outdated WL?\n                        db_entry:reset_locks(T2DBEntry);\n                   true -> T2DBEntry\n                end,\n            TLogSnapNo = tx_tlog:get_entry_snapshot(RTLogEntry),\n            ?TRACE_SNAP(\"rdht_tx_write:commit ~p~ncommiting entry~n~p~n~p\",\n                [comm:this(), NewEntry, DB]),\n            db_dht:set_entry(DB, NewEntry, TLogSnapNo, OwnSnapNo);\n       true ->\n            ?TRACE_SNAP(\"rdht_tx_write:commit ~p~noutdated commit~n~p\",\n                [comm:this(), DB]),\n            DB %% outdated commit\n    end.\n\n-spec abort(db_dht:db(), tx_tlog:tlog_entry(), OwnProposalWas::?prepared | ?abort,\n            tx_tlog:snap_number(), tx_tlog:snap_number()) -> db_dht:db().\nabort(DB, _RTLogEntry, ?abort, _TMSnapNo, _OwnSnapNo) ->\n    %% our own proposal was abort so no locks to clean up\n    DB;\nabort(DB, RTLogEntry, ?prepared, _TMSnapNo, OwnSnapNo) ->\n    ?TRACE(\"rdht_tx_write:abort)~n\", []),\n    %% abort operation when we voted prepared\n    %% need to release locks\n    DBEntry = db_dht:get_entry(DB, tx_tlog:get_entry_key(RTLogEntry)),\n    RTLogVers = tx_tlog:get_entry_version(RTLogEntry),\n    % Note: WriteLock is always >= DBVers! - old check:\n    %%             DBVers = db_entry:get_version(DBEntry),\n    %%             if RTLogVers =:= DBVers ->\n    WriteLock = db_entry:get_writelock(DBEntry),\n    if WriteLock =/= false andalso WriteLock =< RTLogVers ->\n            %% op that created the write lock or outdated WL?\n            NewEntry = db_entry:unset_writelock(DBEntry),\n            TLogSnapNo = tx_tlog:get_entry_snapshot(RTLogEntry),\n            db_dht:set_entry(DB, NewEntry, TLogSnapNo, OwnSnapNo);\n        true -> DB\n    end.\n\n%% be startable via supervisor, use gen_component\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2,\n                             [],\n                             [{pid_groups_join_as, DHTNodeGroup, ?MODULE}]).\n\n%% initialize: return initial state.\n-spec init([]) -> null.\ninit([]) ->\n    ?TRACE(\"rdht_tx_write: Starting rdht_tx_write for DHT node: ~p~n\",\n           [pid_groups:my_groupname()]),\n    %% For easier debugging, use a named table (generates an atom)\n    %%TableName = erlang:list_to_atom(pid_groups:group_to_filename(pid_groups:my_groupname()) ++ \"_rdht_tx_write\"),\n    %%_Table = pdb:new(TableName, [set, protected, named_table]).\n    %% use random table name provided by ets to *not* generate an atom\n    _Table = pdb:new(?MODULE, [set]).\n\n-spec on(comm:message(), pdb:tableid()) -> pdb:tableid().\non({start_work_phase, ReqId, ClientPid, HashedKey, Key}, TableName) ->\n    %% PRE: No entry for key in TLog\n    %% build translog entry from quorum read\n    rdht_tx_read:work_phase_key(self(), ReqId, Key, HashedKey, ?write),\n    pdb:set({ReqId, ClientPid}, TableName),\n    TableName;\n\n%% reply triggered by rdht_tx_write:work_phase/3\non({rdht_tx_read_reply, Id, TLogEntry}, TableName) ->\n    {Id, ClientPid} = pdb:take(Id, TableName),\n    NewTLogEntry = update_tlog_entry(TLogEntry),\n    Msg = msg_reply(Id, NewTLogEntry),\n    comm:send_local(ClientPid, Msg),\n    TableName.\n\n-spec update_tlog_entry(tx_tlog:tlog_entry()) -> tx_tlog:tlog_entry().\nupdate_tlog_entry(TLogEntry) ->\n    %% we keep always the read version and expect equivalence during\n    %% validation and increment then in case of write.\n    T2 = tx_tlog:set_entry_operation(TLogEntry, ?write),\n    % should always be true (a partial read for the version never fails)!\n    % ?ok =:= tx_tlog:get_entry_status(T2),\n    tx_tlog:set_entry_value(T2, ?value, ?value_dropped).\n\n%% @doc Checks whether used config parameters exist and are valid.\n-spec check_config() -> true.\ncheck_config() -> true.\n"
  },
  {
    "path": "src/transactions/tx_item_state.hrl",
    "content": "% @copyright 2009-2015 Zuse Institute Berlin,\n%                      onScale solutions GmbH\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@onscale.de>\n%% @doc Part of generic transaction implementation -\n%%      The state for a single request in a transaction of a TM and RTM.\n%% @version $Id$\n\n-compile({inline, [tx_item_get_itemid/1, %tx_item_set_itemid/2,\n                   tx_item_get_txid/1,\n                   tx_item_get_tlog/1,\n                   tx_item_get_maj_for_prepared/1,\n                   tx_item_get_maj_for_abort/1,\n                   tx_item_get_decided/1, tx_item_set_decided/2,\n                   tx_item_get_numprepared/1, tx_item_inc_numprepared/1,\n                   tx_item_get_numabort/1, tx_item_inc_numabort/1,\n                   tx_item_get_paxosids_rtlogs_tps/1,\n                   tx_item_set_paxosids_rtlogs_tps/2,\n                   tx_item_get_status/1, tx_item_set_status/2,\n                   tx_item_get_hold_back/1, tx_item_hold_back/2,\n                   %tx_item_set_hold_back/2,\n                   tx_item_get_numcommitted/1, tx_item_inc_numcommitted/1\n                  ]}).\n\n-type tx_item_id() :: {?tx_item_id, TLogUid::uid:global_uid(), ItemId::non_neg_integer()}.\n-opaque paxos_id() :: {?paxos_id, TLogUid::uid:global_uid(), ItemId::non_neg_integer(), PaxosId::non_neg_integer()}.\n-type tx_item_state() ::\n {\n   TxItemId::tx_item_id(), %%  1 id of the item\n   ?tx_item_state,     %%  2 tx_item_state, data type tag for debugging\n   TxId::tx_id() | undefined_tx_id,  %%  3 part of transaction with id TxId\n   TLogEntry::tx_tlog:tlog_entry() | empty_tlog_entry, %%  4 corresponding transaction log entry\n   Maj_for_prepared::non_neg_integer(), %%  5 prepare votes to decide prepared\n   Maj_for_abort::non_neg_integer(), %%  6 abort votes to decide abort\n   Decided::false | ?prepared | ?abort, %%  7 current decision status\n   Numprepared::non_neg_integer(), %%  8 number of received prepare votes\n   Numabort::non_neg_integer(), %%  9 number of received abort votes\n   [{paxos_id(), RTLogEntry::tx_tlog:tlog_entry(), TP::comm:mypid() | unknown}], %% 10 involved PaxosIDs\n   Status::new | uninitialized | ok, %% 11 item status\n   HoldBackQueue::[comm:message()],         %% 12 when not initialized\n   NumCommitted::non_neg_integer()  %% 13\n }.\n%% @TODO maybe the following entries are also necessary?:\n%%      tx_tm_helper_behaviour to use? needed? for what?,\n%%      timeout before the first RTM takes over\n\n-spec tx_item_new(tx_item_id()) -> tx_item_state().\ntx_item_new(ItemId) ->\n    ReplDeg = config:read(replication_factor),\n    {ItemId, ?tx_item_state, undefined_tx_id, empty_tlog_entry,\n     quorum:majority_for_accept(ReplDeg), quorum:majority_for_deny(ReplDeg),\n     false, 0, 0, _no_paxIds = [], uninitialized, _HoldBack = [], 0}.\n\n-spec tx_item_new(tx_item_id(), tx_id(), tx_tlog:tlog_entry())\n         -> tx_item_state().\ntx_item_new(ItemId, TxId, TLogEntry) ->\n    ReplDeg = config:read(replication_factor),\n    tx_item_new(ItemId, TxId, TLogEntry, quorum:majority_for_accept(ReplDeg),\n        quorum:majority_for_deny(ReplDeg)).\n\n%% pre: if paxos IDs are given, their order must match the order of the created RTLogEntries!\n-spec tx_item_new(tx_item_id(), tx_id(), tx_tlog:tlog_entry(),\n                  Maj_for_prepared::non_neg_integer(),\n                  Maj_for_abort::non_neg_integer())\n        -> tx_item_state().\ntx_item_new({?tx_item_id, TLogUid, ItemNr} = ItemId, TxId, TLogEntry,\n    Maj_for_prepared, Maj_for_abort) ->\n    %% expand TransLogEntry to replicated translog entries\n    RTLogEntries =\n        case tx_tlog:get_entry_operation(TLogEntry) of\n            ?read ->\n                rdht_tx_read:validate_prefilter(TLogEntry);\n            ?write ->\n                rdht_tx_write:validate_prefilter(TLogEntry)\n        end,\n    PaxIDsRTLogsTPs =\n        util:map_with_nr(\n          fun(RTLogEntry, NrX) ->\n                  {{?paxos_id, TLogUid, ItemNr, NrX}, RTLogEntry, unknown}\n          end, RTLogEntries, 0),\n    {ItemId, ?tx_item_state, TxId, TLogEntry,\n     Maj_for_prepared, Maj_for_abort,\n     false, 0, 0, PaxIDsRTLogsTPs,\n     uninitialized, _HoldBack = [], 0}.\n\n-spec tx_item_get_itemid(tx_item_state()) -> tx_item_id().\ntx_item_get_itemid(State) ->         element(1, State).\n%% -spec tx_item_set_itemid(tx_item_state(), tx_item_id()) -> tx_item_state().\n%% tx_item_set_itemid(State, Val) ->    setelement(1, State, Val).\n-spec tx_item_get_txid(tx_item_state()) -> tx_id() | undefined_tx_id.\ntx_item_get_txid(State) ->           element(3, State).\n-spec tx_item_get_tlog(tx_item_state()) -> tx_tlog:tlog_entry() | empty_tlog_entry.\ntx_item_get_tlog(State) ->           element(4, State).\n-spec tx_item_get_maj_for_prepared(tx_item_state()) -> non_neg_integer().\ntx_item_get_maj_for_prepared(State) -> element(5, State).\n-spec tx_item_get_maj_for_abort(tx_item_state()) -> non_neg_integer().\ntx_item_get_maj_for_abort(State) ->  element(6, State).\n-spec tx_item_get_decided(tx_item_state()) -> false | ?prepared | ?abort.\ntx_item_get_decided(State) ->        element(7, State).\n-spec tx_item_set_decided(tx_item_state(), false | ?prepared | ?abort) -> tx_item_state().\ntx_item_set_decided(State, Val) ->   setelement(7, State, Val).\n-spec tx_item_get_numprepared(tx_item_state()) -> non_neg_integer().\ntx_item_get_numprepared(State) ->    element(8, State).\n-spec tx_item_inc_numprepared(tx_item_state()) -> tx_item_state().\ntx_item_inc_numprepared(State) ->    setelement(8, State, element(8,State) + 1).\n-spec tx_item_get_numabort(tx_item_state()) -> non_neg_integer().\ntx_item_get_numabort(State) ->       element(9, State).\n-spec tx_item_inc_numabort(tx_item_state()) -> tx_item_state().\ntx_item_inc_numabort(State) ->       setelement(9, State, element(9,State) + 1).\n\n-spec tx_item_get_paxosids_rtlogs_tps(tx_item_state())\n        -> [{paxos_id(), tx_tlog:tlog_entry(), comm:mypid() | unknown}].\ntx_item_get_paxosids_rtlogs_tps(State) -> element(10, State).\n-spec tx_item_set_paxosids_rtlogs_tps(tx_item_state(), [{paxos_id(), tx_tlog:tlog_entry(), comm:mypid()}]) -> tx_item_state().\ntx_item_set_paxosids_rtlogs_tps(State, NewTPList) -> setelement(10, State, NewTPList).\n\n-spec tx_item_get_status(tx_item_state()) -> new | uninitialized | ok.\ntx_item_get_status(State) ->         element(11, State).\n-spec tx_item_set_status(tx_item_state(), new | uninitialized | ok) -> tx_item_state().\ntx_item_set_status(State, Status) -> setelement(11, State, Status).\n-spec tx_item_get_hold_back(tx_item_state()) -> [comm:message()].\ntx_item_get_hold_back(State) ->      element(12, State).\n-spec tx_item_hold_back(comm:message(), tx_item_state()) -> tx_item_state().\ntx_item_hold_back(Msg, State) ->     setelement(12, State, [Msg | element(12, State)]).\n%% -spec tx_item_set_hold_back(tx_item_state(), [comm:message()]) -> tx_item_state().\n%% tx_item_set_hold_back(State, Queue) -> setelement(12, State, Queue).\n\n-spec tx_item_get_numcommitted(tx_item_state()) -> non_neg_integer().\ntx_item_get_numcommitted(State) ->    element(13, State).\n-spec tx_item_inc_numcommitted(tx_item_state()) -> tx_item_state().\ntx_item_inc_numcommitted(State) ->       setelement(13, State, element(13,State) + 1).\n\n-spec tx_item_newly_decided(tx_item_state()) -> false | ?prepared | ?abort.\ntx_item_newly_decided(State) ->\n    case tx_item_get_decided(State) of\n        false ->\n            Prepared = tx_item_get_numprepared(State) =:= tx_item_get_maj_for_prepared(State),\n            Abort =    tx_item_get_numabort(State) =:= tx_item_get_maj_for_abort(State),\n            Bad = quorum:majority_for_accept(config:read(replication_factor)) + 1,\n            case tx_item_get_numprepared(State) + tx_item_get_numabort(State) of\n                Bad -> log:log(\"Deciding at ~pth answer!!!\", [Bad]);\n                _ -> ok\n            end,\n            if Prepared andalso not Abort -> ?prepared;\n               not Prepared andalso Abort -> ?abort;\n               true -> false\n            end;\n        _Any -> false\n    end.\n\n-spec tx_item_set_tp_for_paxosid(tx_item_state(), comm:mypid(), paxos_id()) -> tx_item_state().\ntx_item_set_tp_for_paxosid(State, TP, PaxosId) ->\n    TPList = tx_item_get_paxosids_rtlogs_tps(State),\n    NewTPList = util:lists_keystore2(PaxosId, 1, TPList, 3, TP),\n    tx_item_set_paxosids_rtlogs_tps(State, NewTPList).\n\n-spec tx_item_add_learner_decide(tx_item_state(),\n                                 {learner_decide, tx_item_id(),\n                                  paxos_id(), ?prepared | ?abort})\n        -> {hold_back | state_updated | {item_newly_decided, ?prepared | ?abort},\n            tx_item_state()}.\ntx_item_add_learner_decide(State, {learner_decide, _ItemId, _PaxosID, Value} = Msg) ->\n    case ok =/= tx_item_get_status(State) of\n        true -> %% new | uninitialized\n            TmpState = tx_item_hold_back(Msg, State),\n            NewState = tx_item_set_status(TmpState, uninitialized),\n            {hold_back, NewState};\n        false -> %% ok\n            TmpState =\n                case Value of\n                    ?prepared -> tx_item_inc_numprepared(State);\n                    ?abort ->    tx_item_inc_numabort(State)\n                end,\n            case tx_item_newly_decided(TmpState) of\n                false -> {state_updated, TmpState};\n                Decision ->\n                    NewState = tx_item_set_decided(TmpState, Decision),\n                    {{item_newly_decided, Decision}, NewState}\n            end\n    end.\n\n-spec tx_item_add_commit_ack(tx_item_state())\n        -> {item_newly_safe | state_updated, tx_item_state()}.\ntx_item_add_commit_ack(State) ->\n    NewState = tx_item_inc_numcommitted(State),\n    Maj = tx_item_get_maj_for_prepared(NewState),\n    case tx_item_get_numcommitted(NewState) of\n        Maj -> {item_newly_safe, NewState};\n        _   -> {state_updated, NewState}\n    end.\n"
  },
  {
    "path": "src/transactions/tx_op_beh.erl",
    "content": "%% @copyright 2012 Zuse Institute Berlin\n%%            2009, 2010 onScale solutions GmbH\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@onscale.de>\n%% @doc Part of generic transactions implementation.\n%%      The behaviour of an operation in a transaction.\n%% @version $Id$\n-module(tx_op_beh).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n% for behaviour\n-ifndef(have_callback_support).\n-export([behaviour_info/1]).\n-endif.\n\n-ifdef(have_callback_support).\n-include(\"scalaris.hrl\").\n-callback work_phase(pid(), rdht_tx:req_id() | rdht_tx_write:req_id(),\n                     api_tx:request()) -> ok.\n-callback validate_prefilter(tx_tlog:tlog_entry()) -> [tx_tlog:tlog_entry()].\n-callback validate(db_dht:db(), tx_tlog:snap_number(), tx_tlog:tlog_entry()) -> {db_dht:db(), ?prepared | ?abort}.\n-callback commit(db_dht:db(), tx_tlog:tlog_entry(), ?prepared | ?abort,\n                 tx_tlog:snap_number(), tx_tlog:snap_number()) -> db_dht:db().\n-callback abort(db_dht:db(), tx_tlog:tlog_entry(), ?prepared | ?abort,\n                tx_tlog:snap_number(), tx_tlog:snap_number()) -> db_dht:db().\n-else.\n-spec behaviour_info(atom()) -> [{atom(), arity()}] | undefined.\nbehaviour_info(callbacks) ->\n    [\n     %% do the work phase *asynchronously*, replies to local client with a msg\n     %% work_phase(ClientPid, Id, Request) ->\n     %%   msg {work_phase_reply, Id, TLogEntry}\n     {work_phase, 3},\n     %% May make several ones from a single TransLog item (item replication)\n     %% validate_prefilter(TransLogEntry) ->\n     %%   [TransLogEntries] (replicas)\n     {validate_prefilter, 1},\n     %% validate a single item\n     %% validate(DB, RTLogentry) -> {DB, Proposal (prepared/abort)}\n     {validate, 3},\n     %% commit(DB, RTLogentry, OwnProposalWas, TMSnapNo, OwnSnapNo)\n     {commit, 5},\n     %% abort(DB, RTLogentry, OwnProposalWas, TMSnapNo, OwnSnapNo)\n     {abort, 5}\n    ];\nbehaviour_info(_Other) ->\n    undefined.\n-endif.\n"
  },
  {
    "path": "src/transactions/tx_state.hrl",
    "content": "% @copyright 2009-2015 Zuse Institute Berlin.\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@onscale.de>\n%% @doc Part of generic transaction implementation -\n%%      The state for a whole transaction in a TM and RTM.\n%% @end\n%% @version $Id$\n\n-compile({inline, [tx_state_get_tid/1, %tx_state_set_tid/2,\n                   tx_state_get_client/1, %tx_state_set_client/2,\n                   tx_state_get_clientsid/1, %tx_state_set_clientsid/2,\n                   tx_state_get_tm/1, %tx_state_set_tm/2,\n                   tx_state_get_rtms/1, %tx_state_set_rtms/2,\n                   tx_state_get_txitemids/1, %tx_state_set_txitemids/2,\n                   tx_state_get_learners/1,\n                   tx_state_is_decided/1, tx_state_set_decided/2,\n                   tx_state_get_numids/1,\n                   tx_state_get_numprepared/1, tx_state_inc_numprepared/1,\n                   tx_state_get_numabort/1, tx_state_inc_numabort/1,\n                   tx_state_get_numinformed/1, tx_state_inc_numinformed/1,\n                   tx_state_set_numinformed/2,\n                   tx_state_get_numcommitack/1, tx_state_inc_numcommitack/1,\n                   tx_state_get_numtpsregistered/1, tx_state_inc_numtpsregistered/1,\n                   tx_state_get_status/1, tx_state_set_status/2,\n                   tx_state_get_hold_back/1, tx_state_hold_back/2,\n                   %tx_state_set_hold_back/2,\n                   tx_state_all_tps_informed/1, tx_state_all_tps_registered/1\n                  ]}).\n\n-type tx_state() ::\n        {tx_id(),                  %% Tid\n         ?tx_state,                 %% type-marker (unnecessary in principle)\n         comm:mypid() | unknown,   %% Client\n         any(),                    %% ClientsId,\n         comm:mypid() | unknown,   %% TM\n         rtms(), %% [{Key, RTM, Nth, Acceptor}]\n         [tx_item_id()], %% _txitemids = [TxItemId],\n         [comm:mypid()],           %% Learners,\n         ?undecided | false | ?abort | ?commit, %% Status decided?\n         non_neg_integer(),        %% NumIds,\n         non_neg_integer(),        %% NumPrepared,\n         non_neg_integer(),        %% NumAbort,\n         non_neg_integer(),        %% NumTPsInformed\n         non_neg_integer(),        %% Number of items committed\n         non_neg_integer(),        %% NumTpsRegistered\n         new | uninitialized | ok, %% status: new / uninitialized / ok\n         [comm:message()]                   %% HoldBack\n         }.\n%% @TODO also necessary?\n%%               Majority of RTMs,\n%%               timeout before RTM takes over\n%%               }\n\n%% -spec is_tx_state(tx_state()) -> boolean().\n%% is_tx_state(TxState) ->\n%%     is_tuple(TxState)\n%%         andalso erlang:tuple_size(TxState) >= 2\n%%         andalso ?tx_state =:= element(2, TxState).\n\n-spec tx_state_new(tx_id(), comm:mypid() | unknown,\n                   comm:mypid() | unknown, comm:mypid() | unknown,\n                   tx_tm_rtm:rtms(),\n                   [tx_item_state()],\n                   [comm:mypid()]) -> tx_state().\ntx_state_new(Tid, Client, ClientsID, TM, RTMs, ItemStates, Learners) ->\n    {Tid, ?tx_state, Client, ClientsID, TM, RTMs,\n     [tx_item_get_itemid(ItemState) || ItemState <- ItemStates],\n     Learners, ?undecided, length(ItemStates),\n     _Prepared = 0, _Aborts = 0, _Informed = 0, _CommitsAcked = 0,\n     _TpsRegistered = 0, _Status = uninitialized, _HoldBackQueue = []}.\n\n-spec tx_state_new(tx_id()) -> tx_state().\ntx_state_new(Tid) ->\n    tx_state_new(Tid, unknown, unknown, unknown, _RTMs = [], _ItemStates = [], []).\n\n-spec tx_state_get_tid(tx_state()) -> tx_id().\ntx_state_get_tid(State) ->                 element(1, State).\n%% -spec tx_state_set_tid(tx_state(), tx_id()) -> tx_state().\n%% tx_state_set_tid(State, Val) ->            setelement(1, State, Val).\n-spec tx_state_get_client(tx_state()) -> comm:mypid() | unknown.\ntx_state_get_client(State) ->              element(3, State).\n%% -spec tx_state_set_client(tx_state(), comm:mypid() | unknown) -> tx_state().\n%% tx_state_set_client(State, Val) ->         setelement(3, State, Val).\n-spec tx_state_get_clientsid(tx_state()) -> any().\ntx_state_get_clientsid(State) ->           element(4, State).\n%% -spec tx_state_set_clientsid(tx_state(), any()) -> tx_state().\n%% tx_state_set_clientsid(State, Val) ->      setelement(4, State, Val).\n\n-spec tx_state_get_tm(tx_state()) -> comm:mypid() | unknown.\ntx_state_get_tm(State) ->                  element(5, State).\n%% -spec tx_state_set_tm(tx_state(), comm:mypid() | unknown) -> tx_state().\n%% tx_state_set_tm(State, Val) ->             setelement(5, State, Val).\n\n-spec tx_state_get_rtms(tx_state()) -> tx_tm_rtm:rtms().\ntx_state_get_rtms(State) ->                element(6, State).\n%% -spec tx_state_set_rtms(tx_state(), tx_tm_rtm:rtms()) -> tx_state().\n%% tx_state_set_rtms(State, Val) ->           setelement(6, State, Val).\n-spec tx_state_get_txitemids(tx_state()) -> [tx_tm_rtm:tx_item_id()].\ntx_state_get_txitemids(State) ->      element(7, State).\n%% -spec tx_state_set_txitemids(tx_state(), [tx_tm_rtm:tx_item_id()]) -> tx_state().\n%% tx_state_set_txitemids(State, Val) -> setelement(7, State, Val).\n-spec tx_state_get_learners(tx_state()) -> [comm:mypid()].\ntx_state_get_learners(State) ->            element(8, State).\n-spec tx_state_is_decided(tx_state()) -> ?undecided | false | ?abort | ?commit.\ntx_state_is_decided(State) ->              element(9, State).\n-spec tx_state_set_decided(tx_state(), ?undecided | false | ?abort | ?commit) -> tx_state().\ntx_state_set_decided(State, Val) ->        setelement(9, State, Val).\n-spec tx_state_get_numids(tx_state()) -> non_neg_integer().\ntx_state_get_numids(State) ->              element(10, State).\n-spec tx_state_get_numprepared(tx_state()) -> non_neg_integer().\ntx_state_get_numprepared(State) ->         element(11, State).\n-spec tx_state_inc_numprepared(tx_state()) -> tx_state().\ntx_state_inc_numprepared(State) ->         setelement(11, State, 1 + element(11, State)).\n-spec tx_state_get_numabort(tx_state()) -> non_neg_integer().\ntx_state_get_numabort(State) ->            element(12, State).\n-spec tx_state_inc_numabort(tx_state()) -> tx_state().\ntx_state_inc_numabort(State) ->            setelement(12, State, 1 + element(12, State)).\n-spec tx_state_get_numinformed(tx_state()) -> non_neg_integer().\ntx_state_get_numinformed(State) ->         element(13, State).\n-spec tx_state_inc_numinformed(tx_state()) -> tx_state().\ntx_state_inc_numinformed(State) ->         setelement(13, State, 1 + element(13, State)).\n-spec tx_state_set_numinformed(tx_state(), non_neg_integer()) -> tx_state().\ntx_state_set_numinformed(State, Val) ->    setelement(13, State, Val).\n-spec tx_state_get_numcommitack(tx_state()) -> non_neg_integer().\ntx_state_get_numcommitack(State) ->        element(14, State).\n-spec tx_state_inc_numcommitack(tx_state()) -> tx_state().\ntx_state_inc_numcommitack(State) ->        setelement(14, State, 1 + element(14, State)).\n-spec tx_state_get_numtpsregistered(tx_state()) -> non_neg_integer().\ntx_state_get_numtpsregistered(State) ->    element(15, State).\n-spec tx_state_inc_numtpsregistered(tx_state()) -> tx_state().\ntx_state_inc_numtpsregistered(State) ->    setelement(15, State, 1 + element(15, State)).\n\n-spec tx_state_get_status(tx_state()) -> new | uninitialized | ok.\ntx_state_get_status(State) -> element(16, State).\n-spec tx_state_set_status(tx_state(), new | uninitialized | ok) -> tx_state().\ntx_state_set_status(State, Status) -> setelement(16, State, Status).\n-spec tx_state_get_hold_back(tx_state()) -> [comm:message()].\ntx_state_get_hold_back(State) -> element(17, State).\n-spec tx_state_hold_back(comm:message(), tx_state()) -> tx_state().\ntx_state_hold_back(Msg, State) -> setelement(17, State, [Msg | element(17, State)]).\n%% -spec tx_state_set_hold_back(tx_state(), [comm:message()]) -> tx_state().\n%% tx_state_set_hold_back(State, Queue) -> setelement(17, State, Queue).\n\n\n-spec tx_state_newly_decided(tx_state()) -> ?undecided | false | ?abort | ?commit.\ntx_state_newly_decided(State) ->\n    case (?undecided =:= tx_state_is_decided(State)) of\n        false -> false; %% was already decided\n        true ->\n            %% maybe: calculate decision:\n            case tx_state_get_numabort(State) > 0 of\n                true -> ?abort;\n                _ ->\n                    case tx_state_get_numprepared(State) =:= tx_state_get_numids(State) of\n                        true -> ?commit;\n                        _ -> ?undecided\n                    end\n            end\n    end.\n\n-spec tx_state_all_tps_informed(tx_state()) -> boolean().\ntx_state_all_tps_informed(State) ->\n    tx_state_get_numinformed(State)\n        =:= (tx_state_get_numids(State)\n             * config:read(replication_factor)).\n\n-spec tx_state_all_tps_registered(tx_state()) -> boolean().\ntx_state_all_tps_registered(State) ->\n    tx_state_get_numtpsregistered(State)\n        =:= (tx_state_get_numids(State)\n             * config:read(replication_factor)).\n\n-spec tx_state_add_item_decision(tx_state(), ?prepared | ?abort)\n        -> {?undecided | false | {tx_newly_decided, ?abort | ?commit},\n            tx_state()}.\ntx_state_add_item_decision(State, Decision) ->\n    T1 =\n        case Decision of\n            ?prepared -> tx_state_inc_numprepared(State);\n            ?abort    -> tx_state_inc_numabort(State)\n        end,\n    case tx_state_newly_decided(T1) of\n        ?undecided -> {?undecided, T1};\n        false -> {false, T1};\n        Result -> %% commit or abort\n            T2 = tx_state_set_decided(T1, Result),\n            {{tx_newly_decided, Result}, T2}\n    end.\n\n-spec tx_state_add_item_acked(tx_state())\n        -> {all_items_acked | open, tx_state()}.\ntx_state_add_item_acked(State) ->\n    NewState = tx_state_inc_numcommitack(State),\n    NumAll = tx_state_get_numids(NewState),\n    case tx_state_get_numcommitack(NewState) of\n        NumAll -> {all_items_acked, NewState};\n        _      -> {open, NewState}\n    end.\n"
  },
  {
    "path": "src/transactions/tx_tlog.erl",
    "content": "% @copyright 2009-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n% @author Florian Schintke <schintke@zib.de>\n% @doc operations on the end user transaction log\n% @version $Id$\n-module(tx_tlog).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-compile({inline, [new_entry/5, new_entry/6, new_entry/7,\n                   get_entry_operation/1, set_entry_operation/2,\n                   get_entry_key/1,       set_entry_key/2,\n                   get_entry_status/1,    set_entry_status/2,\n                   get_entry_snapshot/1,  set_entry_snapshot/2,\n                   get_entry_value/1,     set_entry_value/3,\n                   get_entry_value_type/1,\n                   drop_value/1,\n                   get_entry_version/1\n                  ]}).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% Operations on TLogs\n-export([empty/0]).\n-export([add_entry/2]).\n-export([add_or_update_status_by_key/3]).\n-export([update_entry/2]).\n-export([sort_by_key/1]).\n-export([find_entry_by_key/2]).\n-export([is_sane_for_commit/1]).\n-export([get_insane_keys/1]).\n-export([merge/2, first_req_per_key_not_in_tlog/2, cleanup/1]).\n\n%% Operations on entries of TLogs\n-export([new_entry/5, new_entry/6, new_entry/7]).\n-export([get_entry_operation/1, set_entry_operation/2]).\n-export([get_entry_key/1,       set_entry_key/2]).\n-export([get_entry_status/1,    set_entry_status/2]).\n-export([get_entry_snapshot/1,  set_entry_snapshot/2]).\n-export([get_entry_value/1,     set_entry_value/3,\n         get_entry_value_type/1]).\n-export([drop_value/1]).\n-export([get_entry_version/1]).\n\n-export_type([tlog/0, tlog_entry/0, tlog_entry_write/0]).\n-export_type([tlog_ext/0, tlog_entry_ext/0]).\n-export_type([tx_status/0, tx_op/0]).\n-export_type([snap_number/0]).\n\n-type tx_status() :: ?ok | ?fail. % TODO: add 'undefined'?!\n-type tx_op()     :: ?read | ?write.\n-type snap_number() :: non_neg_integer().\n% note: from all the value types, only ?value and ?value_dropped remain in the\n%       user tlog - the rest are intermediate states!\n-type value_type_r() :: ?value | ?partial_value | ?not_found | {?fail, atom() | integer()} | ?value_dropped.\n-type value_type_w() :: ?value | {?fail, atom() | integer()} | ?value_dropped.\n\n-type tlog_key() :: client_key() | ?RT:key().\n%% TODO this does not make sense anymore\n%% TLogEntry: {Operation, Key, Version, Status, Value}\n%% Sample: {?read,\"key3\",?value,\"value3\",0}\n-type tlog_entry_read() ::\n          { Op        :: ?read,\n            Key       :: tlog_key(),\n            Version   :: non_neg_integer() | -1,\n            Status    :: tx_status(),\n            SnapshotNumber :: snap_number(),\n            % note: only partial reads allow ?partial_value:\n            ValueType :: value_type_r(),\n            Value     :: rdht_tx:encoded_value() | ?value_dropped % ?value_dropped if ValueType =:= ?not_found or {?fail, _} or ?value_dropped\n          }.\n-type tlog_entry_write() ::\n          { Op        :: ?write,\n            Key       :: tlog_key(),\n            Version   :: non_neg_integer() | -1,\n            Status    :: tx_status(),\n            SnapshotNumber :: snap_number(),\n            ValueType :: value_type_w(),\n            Value     :: rdht_tx:encoded_value() | ?value_dropped % ?value_dropped if ValueType =:= ?not_found or {?fail, _} or ?value_dropped\n          }.\n-type tlog_entry() :: tlog_entry_read() | tlog_entry_write().\n-type tlog_entry_ext() ::\n          { Op        :: ?read,\n            Key       :: client_key(),\n            Version   :: non_neg_integer() | -1,\n            Status    :: tx_status(),\n            SnapshotNumber :: snap_number(),\n            ValueType :: ?value_dropped,\n            Value     :: ?value_dropped\n          }\n        | { Op        :: ?write,\n            Key       :: client_key(),\n            Version   :: non_neg_integer() | -1,\n            Status    :: tx_status(),\n            SnapshotNumber :: snap_number(),\n            ValueType :: ?value,\n            Value     :: rdht_tx:encoded_value()\n          }.\n%% -opaque tlog() :: [tlog_entry()]. % creates a false warning in add_or_update_status_by_key/3\n-type tlog() :: [tlog_entry()].\n-type tlog_ext() :: [tlog_entry_ext()].\n\n% @doc create an empty list\n-spec empty() -> tlog().\nempty() -> [].\n\n-spec add_entry(tlog(), tlog_entry()) -> tlog().\nadd_entry(TransLog, Entry) -> [ Entry | TransLog ].\n\n-spec add_or_update_status_by_key(tlog(), tlog_key(), tx_status()) -> tlog().\nadd_or_update_status_by_key([], Key, Status) ->\n    Entry = new_entry(?read, Key, _Vers = 0, _ValType = ?value_dropped, _Val = ?value_dropped),\n    [set_entry_status(Entry, Status)];\nadd_or_update_status_by_key([Entry | T], Key, Status)\n  when element(2, Entry) =:= Key ->\n    [set_entry_status(Entry, Status) | T];\nadd_or_update_status_by_key([Entry | T], Key, Status) ->\n    [Entry | add_or_update_status_by_key(T, Key, Status)].\n\n-spec update_entry(tlog(), tlog_entry()) -> tlog().\nupdate_entry(TLog, Entry) ->\n    lists:keyreplace(get_entry_key(Entry), 2, TLog, Entry).\n\n-spec sort_by_key(tlog()) -> tlog().\nsort_by_key(TLog) -> lists:keysort(2, TLog).\n\n-spec find_entry_by_key(tlog(), tlog_key()) -> tlog_entry() | false.\nfind_entry_by_key(TLog, Key) ->\n    lists:keyfind(Key, 2, TLog).\n\n-spec entry_is_sane_for_commit(tlog_entry()) -> boolean().\nentry_is_sane_for_commit(Entry) ->\n    get_entry_status(Entry) =:= ?ok.\n\n-spec is_sane_for_commit(tlog()) -> boolean().\nis_sane_for_commit(TLog) ->\n    lists:all(fun entry_is_sane_for_commit/1, TLog).\n\n-spec get_insane_keys(tlog_ext()) -> [client_key()].\nget_insane_keys(TLog) ->\n    [get_entry_key(X) || X <- TLog, not entry_is_sane_for_commit(X)].\n\n%% @doc Merge TLog entries from sorted translogs (see sort_by_key/1), if same\n%%      key. Check for version mismatch, take over values.\n%%      Duplicate keys are only allowedif the old TLog only read the value!\n%%      SortedTlog is old TLog\n%%      SortedRTlog is TLog received from newer RDHT operations\n-spec merge(tlog(), tlog()) -> tlog().\nmerge(TLog1, TLog2) ->\n    merge_tlogs_iter(TLog1, TLog2).\n\n-spec merge_tlogs_iter([tlog_entry()], [tlog_entry()]) -> [tlog_entry()].\nmerge_tlogs_iter([TEntry | TTail] = SortedTLog,\n                 [RTEntry | RTTail] = SortedRTLog) ->\n    TKey = get_entry_key(TEntry),\n    RTKey = get_entry_key(RTEntry),\n    if TKey < RTKey ->\n           [TEntry | merge_tlogs_iter(TTail, SortedRTLog)];\n       TKey > RTKey ->\n           [RTEntry | merge_tlogs_iter(SortedTLog, RTTail)];\n       true -> % TKey =:= RTKey ->\n           %% key was in TLog, new entry is newer and contains value\n           %% for read?\n           case get_entry_operation(TEntry) of\n               ?read ->\n                   %% check versions: if mismatch -> change status to abort\n                   TVersion = get_entry_version(TEntry),\n                   RTVersion = get_entry_version(RTEntry),\n                   NewTLogEntry =\n                       if TVersion =:= RTVersion ->\n                              {ValType, Val} = get_entry_value(RTEntry),\n                              TEntry2 = set_entry_value(TEntry, ValType, Val),\n                              TStatus = get_entry_status(TEntry2),\n                              if TStatus =:= ?ok ->\n                                      % overwrite with new status which may be failed!\n                                     set_entry_status(TEntry2, get_entry_status(RTEntry));\n                                 true -> TEntry2\n                              end;\n                          true ->\n                              set_entry_status(RTEntry, ?fail)\n                       end,\n                   [NewTLogEntry | merge_tlogs_iter(TTail, RTTail)];\n               _ ->\n                   log:log(warn,\n                           \"Duplicate key in TLog merge should not happen ~p ~p\", [TEntry, RTEntry]),\n                   [RTEntry | merge_tlogs_iter(TTail, RTTail)]\n           end\n    end;\nmerge_tlogs_iter([], [])                  -> [];\nmerge_tlogs_iter([], [_|_] = SortedRTLog) -> SortedRTLog;\nmerge_tlogs_iter([_|_] = SortedTLog, [])  -> SortedTLog.\n\n%% @doc Filters a request list with unique keys so that only operations reside\n%%      that need data from the DHT which is not yet present in the TLog.\n%%      Note: uses functions from rdht_tx to cope with requests.\n-spec first_req_per_key_not_in_tlog(tlog(), [rdht_tx:request_on_key()])\n        -> [rdht_tx:request_on_key()].\nfirst_req_per_key_not_in_tlog(SortedTLog, SortedReqList) ->\n    %% PRE: no {commit} requests in SortedReqList\n    %% POST: returns requests in arbitrary order.\n    first_req_per_key_not_in_tlog_iter(SortedTLog, SortedReqList, []).\n\n%% @doc Helper to acquire requests, that need information from the DHT.\n-spec first_req_per_key_not_in_tlog_iter(\n        [tlog_entry()], SortedReqList::[rdht_tx:request_on_key()], Acc::[rdht_tx:request_on_key()])\n        -> [rdht_tx:request_on_key()].\nfirst_req_per_key_not_in_tlog_iter(_, [], Acc) -> Acc;\nfirst_req_per_key_not_in_tlog_iter([] = USTLog, [Req | _] = SReqList, Acc) ->\n    ReqKey = rdht_tx:req_get_key(Req),\n    {Req2, RTail2} = read_op_for_key(ReqKey, SReqList, [], false, false),\n    first_req_per_key_not_in_tlog_iter(USTLog, RTail2, Req2 ++ Acc);\nfirst_req_per_key_not_in_tlog_iter([TEntry | TTail] = USTLog,\n                                   [Req | _RTail] = SReqList,\n                                   Acc) ->\n    TKey = get_entry_key(TEntry),\n    ReqKey = rdht_tx:req_get_key(Req),\n    if TKey =:= ReqKey ->\n           HaveValue = get_entry_operation(TEntry) =:= ?write,\n           {Req2, RTail2} = read_op_for_key(ReqKey, SReqList, [], HaveValue, true),\n           first_req_per_key_not_in_tlog_iter\n             (USTLog, RTail2, Req2 ++ Acc);\n       TKey < ReqKey ->\n           %% jump to next Tlog entry\n           first_req_per_key_not_in_tlog_iter(TTail, SReqList, Acc);\n       true -> % TKey > ReqKey\n           {Req2, RTail2} = read_op_for_key(ReqKey, SReqList, [], false, false),\n           first_req_per_key_not_in_tlog_iter(USTLog, RTail2, Req2 ++ Acc)\n    end.\n\n%% @doc Filters the (sorted) request list for all ops with the given Key and\n%%      returns a full read request if required or otherwise the partial read\n%%      as requested.\n%% note: it does not matter which request is actually returned here as long as\n%%       we get a full read op if needed!\nread_op_for_key(_Key, [] = SReqList, TmpReq, _HaveValue, _HavePartialValue) ->\n    {TmpReq, SReqList};\nread_op_for_key(Key, [Req | RTail] = SReqList, TmpReq, HaveValue, HavePartialValue) ->\n    ReqKey = rdht_tx:req_get_key(Req),\n    if Key =:= ReqKey andalso HaveValue ->\n           % consume further requests with the same key (there already is a full read)\n           read_op_for_key(Key, RTail, TmpReq, HaveValue, HavePartialValue);\n       Key =:= ReqKey -> %andalso not HaveValue\n           % no full read yet\n           {ReqNeedsFullRead, ReqWorksAfterAnyPartialRead, ReqProvidesFullRead} = rdht_tx:req_props(Req),\n           if ReqNeedsFullRead ->\n                  % req in TmpReq (if any) does not matter\n                  % -> this op requires a full read and there is none yet!\n                  read_op_for_key(Key, RTail, [Req], ReqProvidesFullRead, true);\n              HavePartialValue andalso ReqWorksAfterAnyPartialRead ->\n                  % there is a previous request or TLog entry but it does not provide a full value\n                  % -> must be a partial read\n                  % this op is e.g. a write which works after any partial read\n                  read_op_for_key(Key, RTail, TmpReq, ReqProvidesFullRead, true);\n              HavePartialValue ->\n                  % there is a previous request or TLog entry but it does not provide a full value\n                  % -> must be a partial read\n                  % this op can not operate on any partial read\n                  % -> create a full read\n                  NewReq = {read, ReqKey},\n                  read_op_for_key(Key, RTail, [NewReq], true, true);\n              true -> % no information available yet -> use the given request\n                  read_op_for_key(Key, RTail, [Req], ReqProvidesFullRead, true)\n           end;\n       true -> {TmpReq, SReqList}\n    end.\n\n%% @doc Strips the tlog from read values (sets those values to ?value_dropped).\n-spec cleanup(tlog()) -> tlog().\ncleanup(TLog) ->\n    [ case get_entry_operation(TLogEntry) of\n          ?read -> drop_value(TLogEntry);\n          ?write -> TLogEntry\n      end || TLogEntry <- TLog ].\n\n%% Operations on Elements of TransLogs (public)\n-spec new_entry(tx_op(), ?RT:key(), non_neg_integer() | -1,\n                value_type_r() | value_type_w(), rdht_tx:encoded_value())\n            -> tlog_entry().\nnew_entry(Op, Key, Vers, ValType, Val) ->\n    {Op, Key, Vers, ?ok, 1, ValType, Val}.\n\n-spec new_entry(tx_op(), ?RT:key(), non_neg_integer() | -1,\n                tx_status(), value_type_r() | value_type_w(), rdht_tx:encoded_value())\n            -> tlog_entry().\nnew_entry(Op, Key, Vers, Status, ValType, Val) ->\n    {Op, Key, Vers, Status, 1, ValType, Val}.\n\n-spec new_entry(tx_op(), ?RT:key(), non_neg_integer() | -1,\n                tx_status(), snap_number(), value_type_r() | value_type_w(), rdht_tx:encoded_value())\n            -> tlog_entry().\nnew_entry(Op, Key, Vers, Status, SnapNumber, ValType, Val) ->\n    {Op, Key, Vers, Status, SnapNumber, ValType, Val}.\n\n-spec get_entry_operation(tlog_entry()) -> tx_op().\nget_entry_operation(Element) -> element(1, Element).\n\n-spec set_entry_operation(tlog_entry(), tx_op()) -> tlog_entry().\nset_entry_operation(Element, Val) -> setelement(1, Element, Val).\n\n-spec get_entry_key(tlog_entry()) -> client_key() | ?RT:key().\nget_entry_key(Element)       -> element(2, Element).\n\n-spec set_entry_key(tlog_entry(), client_key() | ?RT:key()) -> tlog_entry().\nset_entry_key(Entry, Val)    -> setelement(2, Entry, Val).\n\n-spec get_entry_version(tlog_entry()) -> non_neg_integer() | -1.\nget_entry_version(Element)   -> element(3, Element).\n\n-spec get_entry_status(tlog_entry()) -> tx_status().\nget_entry_status(Element)    -> element(4, Element).\n\n-spec set_entry_status(tlog_entry(), tx_status()) -> tlog_entry().\nset_entry_status(Element, Val) -> setelement(4, Element, Val).\n\n-spec get_entry_snapshot(tlog_entry()) -> snap_number().\nget_entry_snapshot(Element)    -> element(5, Element).\n\n-spec set_entry_snapshot(tlog_entry(), snap_number()) -> tlog_entry().\nset_entry_snapshot(Element, Val)    -> setelement(5, Element, Val).\n\n-spec get_entry_value_type(tlog_entry()) -> value_type_r() | value_type_w().\nget_entry_value_type(Element)  -> element(6, Element).\n\n-spec get_entry_value(tlog_entry()) -> {value_type_r() | value_type_w(), rdht_tx:encoded_value()}.\nget_entry_value(Element)       -> {element(6, Element), element(7, Element)}.\n\n-spec set_entry_value(tlog_entry(), value_type_r() | value_type_w(), rdht_tx:encoded_value()) -> tlog_entry().\nset_entry_value(Element, ValType, Val) ->\n    setelement(7, setelement(6, Element, ValType), Val).\n\n-spec drop_value(tlog_entry()) -> tlog_entry().\ndrop_value(Element) ->\n    % TODO: keep ?not_found value type so a future op does not have to go to DB?\n    set_entry_value(Element, ?value_dropped, ?value_dropped).\n"
  },
  {
    "path": "src/transactions/tx_tm_rtm.erl",
    "content": "% @copyright 2009-2015 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Part of a generic implementation of transactions using Paxos Commit -\n%%      the roles of the (replicated) transaction manager TM and RTM.\n%% @end\n%% @version $Id$\n-module(tx_tm_rtm).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-compile({inline, [get_item_entry/2]}).\n\n%%-define(TRACE_RTM_MGMT(X,Y), io:format(X,Y)).\n%%-define(TRACE_RTM_MGMT(X,Y), log:pal(X,Y)).\n-define(TRACE_RTM_MGMT(X,Y), ok).\n%-define(TRACE(X,Y), log:pal(X,Y)).\n-define(TRACE(_X,_Y), ok).\n-behaviour(gen_component).\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% public interface for transaction validation using Paxos-Commit.\n-export([commit/4]).\n-export([msg_commit_reply/3]).\n-export([rm_send_update/5]).\n\n%% functions for gen_component module, supervisor callbacks and config\n-export([start_link/2]).\n-export([on/2, init/1]).\n-export([on_init/2]).\n-export([check_config/0]).\n\n%% functions for dht_node\n-export([get_my/2]).\n\n-export_type([rtms/0]).\n-export_type([tx_item_id/0, paxos_id/0]).\n-export_type([tx_id/0]).\n\n-type tx_id() :: {?tx_id, uid:global_uid()}.\n\n-type rtm() :: {?RT:key(),\n                {comm:mypid()} | unknown,\n                Role :: pos_integer(),\n                {Acceptor :: comm:mypid()} | unknown}.\n\n-type rtms() :: [rtm()].\n\n-include(\"gen_component.hrl\").\n-include(\"tx_item_state.hrl\").\n-include(\"tx_state.hrl\").\n\n-type state() ::\n    {RTMs           :: rtms(),\n     TableName      :: pdb:tableid(),\n     Role           :: pid_groups:pidname(),\n     LocalAcceptor  :: pid(),\n     GLocalLearner  :: comm:mypid(),\n     OpenTxNum      :: non_neg_integer(),\n     LocalSnapNo    :: non_neg_integer()}.\n\n%% messages a client has to expect when using this module\n-spec msg_commit_reply(comm:mypid(), any(), any()) -> ok.\nmsg_commit_reply(Client, ClientsID, Result) ->\n    comm:send(Client, {tx_tm_rtm_commit_reply, ClientsID, Result}).\n\n-spec msg_tp_do_commit_abort(comm:mypid(), any(), ?commit | ?abort, non_neg_integer()) -> ok.\nmsg_tp_do_commit_abort(TP, Id, Result, LocalSnapNumber) ->\n    comm:send(TP, {?tp_do_commit_abort, Id, Result, LocalSnapNumber}).\n\n%% public interface for transaction validation using Paxos-Commit.\n%% ClientsID may be nil, its not used by tx_tm. It will be repeated in\n%% replies to allow to map replies to the right requests in the\n%% client.\n-spec commit(comm:erl_local_pid(), comm:mypid(), any(), tx_tlog:tlog()) -> ok.\ncommit(TM, Client, ClientsID, TLog) ->\n    Msg = {tx_tm_rtm_commit, Client, ClientsID, TLog},\n    comm:send_local(TM, Msg).\n\n%% @doc Notifies the tx_tm_rtm of a changed node ID.\n-spec rm_send_update(Subscriber::pid(), Tag::?MODULE,\n                     OldNeighbors::nodelist:neighborhood(),\n                     NewNeighbors::nodelist:neighborhood(),\n                     Reason::rm_loop:reason()) -> ok.\nrm_send_update(Pid, ?MODULE, OldNeighbors, NewNeighbors, _Reason) ->\n    OldId = node:id(nodelist:node(OldNeighbors)),\n    NewId = node:id(nodelist:node(NewNeighbors)),\n    case OldId =/= NewId of\n        true  -> comm:send_local(Pid, {new_node_id, NewId});\n        false -> ok\n    end.\n\n%% be startable via supervisor, use gen_component\n-spec start_link(pid_groups:groupname(), any()) -> {ok, pid()}.\nstart_link(DHTNodeGroup, Role) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2,\n                             [],\n                             [{pid_groups_join_as, DHTNodeGroup, Role},\n                              {spawn_opts, [{fullsweep_after, 0},\n                                            {min_heap_size, 16383}]}]).\n\n%% initialize: return initial state.\n-spec init([]) -> state() | {'$gen_component', [{on_handler, gen_component:handler()},...],\n                             state()}.\ninit([]) ->\n    Role = pid_groups:my_pidname(),\n    ?TRACE(\"tx_tm_rtm:init for instance: ~p ~p~n\",\n           [pid_groups:my_groupname(), Role]),\n    %% For easier debugging, use a named table (generates an atom)\n    %%TableName = erlang:list_to_atom(pid_groups:group_to_filename(pid_groups:my_groupname()) ++ \"_tx_tm_rtm_\" ++ atom_to_list(Role)),\n    %%Table = pdb:new(TableName, [set, protected, named_table]),\n    %% use random table name provided by ets to *not* generate an atom\n    Table = pdb:new(?MODULE, [set]),\n    LAcceptor = get_my(Role, acceptor),\n    GLLearner = comm:make_global(get_my(Role, learner)),\n    %% start getting rtms and maintain them.\n    InitialState = {_RTMs = [], Table, Role, LAcceptor, GLLearner,\n                    _OpenTx = 0, _LocalSnapNo = 0},\n    case Role of\n        tx_tm ->\n            comm:send_local(self(), {get_node_details}),\n            rtm_update_trigger(state_get_RTMs(InitialState)),\n            msg_delay:send_trigger(1, {update_RTMs_on_init}),\n            %% subscribe to id changes\n            rm_loop:subscribe(self(), ?MODULE,\n                              fun rm_loop:subscribe_dneighbor_change_filter/3,\n                              fun ?MODULE:rm_send_update/5, inf),\n            gen_component:change_handler(InitialState, fun ?MODULE:on_init/2);\n        _ -> InitialState\n    end.\n\n-spec on(comm:message(), state()) -> state().\n%% a TP comitted/roled back the item as requested\non({?tp_committed, ItemId} = _Msg, State) ->\n    ?TRACE(\"tx_tm_rtm:on(~p)~n\", [_Msg]),\n\n    %% retrieve the item and handle the commit ack therein\n    {OrigItemResult, ItemState} = get_item_entry(ItemId, State),\n    case OrigItemResult of\n        new -> State;\n        _ ->\n            {ItemResult, NewItemState} = tx_item_add_commit_ack(ItemState),\n            _ = set_entry(NewItemState, State),\n            %% retrieve the tx entry and increment the number of commit acks\n            TxId = tx_item_get_txid(NewItemState),\n            {_, TxState} = get_tx_entry(TxId, State),\n            _ = case ItemResult of\n                    state_updated ->\n                        _ = set_entry(TxState, State);\n                        % trigger_delete_if_done(TxState, State);\n                    item_newly_safe ->\n                        {TxResult, TmpTxState} = tx_state_add_item_acked(TxState),\n                        NewTxState =\n                            case TxResult of\n                                open -> TmpTxState;\n                                all_items_acked ->\n                                    %% to inform, we do not need to know the new state\n                                    Result = tx_state_is_decided(TmpTxState),\n                                    inform_client(TmpTxState, State, Result),\n                                    inform_rtms(TxId, TmpTxState, Result),\n                                    TmpTxState\n                            end,\n                        _ = set_entry(NewTxState, State),\n                        trigger_delete_if_done(NewTxState, State)\n                end,\n            State\n    end;\n\non({delete_if_still_uninitialized, ItemId}, State) ->\n    {NewUninitializedOK, _Item} = get_item_entry(ItemId, State),\n    case NewUninitializedOK of\n        uninitialized ->\n            TableName = state_get_tablename(State),\n            pdb:delete(ItemId, TableName);\n        _ -> ok\n    end,\n    State;\n\n%% a paxos consensus is decided (msg generated by learner.erl)\non({learner_decide, ItemId, _PaxosID, _Value} = Msg, State) ->\n    ?TRACE(\"tx_tm_rtm:on(~p)~n\", [Msg]),\n\n    %% retrieve the item and handle the learner decide therein\n    {NewUnitializedOK, ItemState} = get_item_entry(ItemId, State),\n    case NewUnitializedOK of\n        new ->\n            %% we are the first or the tx is already done...?\n            %% (delete_if_still_uninitialized is only triggered when the\n            %% delayed trigger_delete_if_done does not catch the fourth\n            %% learner_decide - see below)\n            msg_delay:send_local(30, self(),\n                                 {delete_if_still_uninitialized, ItemId});\n        _ -> ok\n    end,\n    {ItemResult, NewItemState} = tx_item_add_learner_decide(ItemState, Msg),\n    _ = set_entry(NewItemState, State),\n\n    %% retrieve the tx entry and increment the number of paxos decided\n    %% The TxState may be dropped, when hold_back (then it was newly created...).\n    TxId = tx_item_get_txid(NewItemState),\n    {_, TxState} = get_tx_entry(TxId, State),\n\n    _ = case ItemResult of\n            hold_back -> ok;\n            state_updated ->\n                _ = set_entry(TxState, State);\n            {item_newly_decided, Decision} ->\n                {TxResult, TmpTxState} = tx_state_add_item_decision(TxState, Decision),\n                NewTxState =\n                    case TxResult of\n                        ?undecided -> TmpTxState;\n                        false -> TmpTxState;\n                        {tx_newly_decided, Result} ->\n                            T1TxState = inform_tps(TmpTxState, State, Result),\n                            %% to inform, we do not need to know the new state\n\n                            %% client and rtms are informed, when\n                            %% maj. of tps comitted\n                            %% inform_client(T1TxState, State, Result),\n                            %% inform_rtms(TxId, TmpTxState, Result),\n                            T1TxState\n                    end,\n                set_entry(NewTxState, State)\n        end,\n    State;\n\non({tx_tm_rtm_commit, Client, ClientsID, TransLog}, State) ->\n    ?TRACE(\"tx_tm_rtm:on({commit, ...}) for TLog ~p as ~p~n\",\n           [TransLog, state_get_role(State)]),\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    Maj = quorum:majority_for_accept(config:read(replication_factor)),\n    GLLearner = state_get_gllearner(State),\n    TLogUid = uid:get_global_uid(),\n    NewTid = {?tx_id, TLogUid},\n    This = comm:this(),\n    ItemStates =\n        util:map_with_nr(\n          fun(TLogEntry, NrX) ->\n                  ItemId = {?tx_item_id, TLogUid, NrX},\n                  TItemState = tx_item_new(ItemId, NewTid, TLogEntry),\n                  ItemState = tx_item_set_status(TItemState, ok),\n                  %% initialize local learner\n                  _ = [ learner:start_paxosid(GLLearner, element(1, X),\n                                              Maj, This, ItemId)\n                          || X <- tx_item_get_paxosids_rtlogs_tps(ItemState) ],\n                  ItemState\n          end, TransLog, 0),\n\n    RTMs = state_get_RTMs(State),\n    TmpTxState = tx_state_new(NewTid, Client, ClientsID, This, RTMs,\n                              ItemStates, [GLLearner]),\n    TxState = tx_state_set_status(TmpTxState, ok),\n\n    init_TPs(TxState, ItemStates, state_get_local_snapno(State)),\n    init_RTMs(TxState, ItemStates),\n\n    _ = [ set_entry(ItemState, State) || ItemState <- ItemStates ],\n    _ = set_entry(TxState, State),\n\n    %% send a weak timeout to ourselves to take further care\n    %% if the tx is slow (or a majority of tps failed)\n    msg_delay:send_local((config:read(tx_timeout) * 2) div 1000, self(),\n                         {tx_tm_rtm_tid_isdone, NewTid}),\n    state_inc_opentxnum(State);\n\n%% is the txid done?\non({tx_tm_rtm_tid_isdone, TxId}, State) ->\n    ?TRACE(\"tx_tm_rtm_tid_isdone ~p as ~p~n\",\n           [TxId, state_get_role(State)]),\n    {ErrCode, TxState} = get_tx_entry(TxId, State),\n    case ErrCode of\n        new -> ok; %% already finished, nothing to be done\n        uninitialized ->\n            %% can happen when already cleaned up at tm\n            ok;\n        ok ->\n            Maj = quorum:majority_for_accept(config:read(replication_factor)),\n            WhatToDo =\n                case tx_state_is_decided(TxState)\n                of\n                    ?abort ->\n                        case Maj > tx_state_get_numcommitack(TxState) of\n                            true -> delay;\n                            false -> {delete, ?abort}\n                        end;\n                    ?commit ->\n                        case Maj > tx_state_get_numcommitack(TxState) of\n                            true -> delay;\n                            false -> {delete, ?commit}\n                        end;\n                    ?undecided ->\n                        %% This tx is a bit slow. Start fds on the participants\n                        %% and then take over on crash messages.  When not enough\n                        %% tps have registered? propose yourself.\n\n                        %% TODO: instead of direct takeover, fd:subscribe to the\n                        %% participants. And then takeover when a crash is\n                        %% actually reported.\n                        QLen = element(2, erlang:process_info(self(), message_queue_len)),\n\n                        %% TODO: check if any messages in the queue contribute to this TxId,\n                        %%       then, delay, otherwise trigger take over\n                        case QLen > state_get_opentxnum(State) of\n                            true ->\n                                                % we have replies other than 'isdone' for some\n                                                % of the tx in the queue\n                                delay;\n                            false ->\n                                takeover\n                        end\n                end,\n            case WhatToDo of\n                {delete, Decision} ->\n                    %% we delete the local state. As we have to maintain opentxnum\n                    %% we send a message to ourselves:\n                    %% log:log(\"Trigger tx_tm_rtm_delete in ~p~n\", [state_get_role(State)]),\n                    comm:send_local(self(), {?tx_tm_rtm_delete, TxId, Decision});\n                delay ->\n                    msg_delay:send_local((config:read(tx_timeout) * 2)\n                                             div 1000,\n                                         self(), {tx_tm_rtm_tid_isdone, TxId});\n                takeover ->\n                    %% FDSubscribe = enough_tps_registered(TxState, State),\n                    ValidRTMs = [ X || X <- tx_state_get_rtms(TxState),\n                                       unknown =/= get_rtmpid(X) ],\n                    send_to_rtms(ValidRTMs,\n                                 fun(_X) -> {tx_tm_rtm_propose_yourself,\n                                            TxId} end)\n            end,\n            ok\n    end,\n    State;\n\n%% this tx is finished and enough TPs were informed, delete the state\non({?tx_tm_rtm_delete, TxId, Decision}, State) ->\n    ?TRACE(\"tx_tm_rtm:on({delete, ~p, ~p}) in ~p ~n\",\n           [TxId, Decision, state_get_role(State)]),\n    {ErrCode, TxState} = get_tx_entry(TxId, State),\n    %% inform RTMs on delete\n    Role = state_get_role(State),\n    {DeleteIt, NewState} =\n        case ErrCode of\n            ok when Role =:= tx_tm ->\n                %% deletes to RTMS are already send in inform_RTMs.\n                %% inform used learner to delete paxosids.\n                AllPaxIds = get_paxos_ids(State, TxState),\n                %% We could delete immediately, but we still miss the\n                %% minority of learner_decides, which would re-create the\n                %% id in the learner, which then would have to be deleted\n                %% separately, so we give the minority a second to arrive\n                %% and then send the delete request.\n                %% learner:stop_paxosids(GLLearner, AllPaxIds),\n                GLLearner = state_get_gllearner(State),\n                msg_delay:send_local(1, comm:make_local(GLLearner),\n                                     {learner_deleteids, AllPaxIds}),\n                {_DeleteIt = true, State};\n            ok ->\n                %% the test trigger_delete was passed, at least by the TM\n                %% RTMs only wait for all tp register messages, to not miss them\n                %% record, that every TP was informed and all paxids decided\n                TmpTxState = tx_state_set_numinformed(\n                               TxState, tx_state_get_numids(TxState) *\n                                   config:read(replication_factor)),\n                Tmp2TxState = tx_state_set_decided(TmpTxState, Decision),\n                _ = set_entry(Tmp2TxState, State),\n                Delete = tx_state_all_tps_registered(TxState),\n                TmpState =\n                    case Delete of\n                        true ->\n                            state_unsubscribe(State, tx_state_get_tm(TxState));\n                        false -> State\n                    end,\n                %% inform used acceptors to delete paxosids.\n                AllPaxIds = get_paxos_ids(State, TxState),\n                LAcceptor = state_get_lacceptor(State),\n                %% msg_delay:send_local((config:read(tx_timeout) * 2) div 1000, LAcceptor,\n                %%                      {acceptor_deleteids, AllPaxIds});\n                comm:send_local(LAcceptor, {acceptor_deleteids, AllPaxIds}),\n                {Delete, TmpState};\n            new -> {false, State}; %% already deleted\n            uninitialized ->\n                {false, State} %% will be deleted when msg_delay triggers it\n        end,\n    case DeleteIt of\n        false ->\n            %% @TODO if we are a rtm, we still wait for register TPs\n            %% trigger same delete later on, as we do not get a new\n            %% request to delete from the tm\n            NewState;\n        true ->\n            TableName = state_get_tablename(NewState),\n            %% delete locally\n            _ = [ pdb:delete(ItemId, TableName)\n              || ItemId <- tx_state_get_txitemids(TxState)],\n            pdb:delete(TxId, TableName),\n            state_dec_opentxnum(NewState)\n            %% @TODO failure cases are not handled yet. If some\n            %% participants do not respond, the state is not deleted.\n            %% In the future, we will handle this using msg_delay for\n            %% outstanding txids to trigger a delete of the items.\n    end;\n\n%% sent by init_RTMs\non({?tx_tm_rtm_init_RTM, TxState, ItemStates, _InRole} = _Msg, State) ->\n   ?TRACE(\"tx_tm_rtm:on({init_RTM, ...}) ~n\", []),\n\n    %% lookup transaction id locally and merge with given TxState\n    Tid = tx_state_get_tid(TxState),\n    {LocalTxStatus, LocalTxEntry} = get_tx_entry(Tid, State),\n    {TmpEntry, NewEntryHoldBackQ, NewState} =\n        case LocalTxStatus of\n            new ->\n                %% nothing known locally\n                %% rtms subscribe to tm crashes for running tx.\n                TmpState = state_subscribe(State, tx_state_get_tm(TxState)),\n                {TxState, [], TmpState};\n            uninitialized ->\n                %%io:format(\"initRTM takes over hold back queue for id ~p in ~p~n\", [Tid, Role]),\n                %% take hold back from existing entry\n                {TxState, tx_state_get_hold_back(LocalTxEntry), State};\n            ok ->\n                log:log(error, \"Duplicate init_RTM\", []),\n                %% hold back queue should be empty!\n                {LocalTxEntry, [], State}\n        end,\n    % note: do not need to set the holdback queue for the entry\n    %       -> we will process it below\n    NewEntry = tx_state_set_status(TmpEntry, ok),\n    _ = set_entry(NewEntry, NewState),\n\n    %% lookup items locally, merge with given ItemStates,\n    %% initiate local paxos acceptors (with received paxos_ids)\n    Learners = tx_state_get_learners(TxState),\n    LAcceptor = state_get_lacceptor(NewState),\n    NewItemsHoldBackQ = lists:append(\n                          merge_item_states(Tid, tx_state_get_txitemids(NewEntry),\n                                            ItemStates, NewState, Learners, LAcceptor)),\n\n    %% process hold back messages for tx_state\n    %% io:format(\"Starting hold back queue processing~n\"),\n    NewState2 = lists:foldr(fun on/2, NewState, NewEntryHoldBackQ),\n    %% process hold back messages for tx_items\n    NewState3 = lists:foldr(fun on/2, NewState2, NewItemsHoldBackQ),\n    %% io:format(\"Stopping hold back queue processing~n\"),\n\n    %% set timeout and remember timerid to cancel, if finished earlier?\n    %%msg_delay:send_local(1 + InRole, self(), {tx_tm_rtm_propose_yourself, Tid}),\n    %% after timeout take over and initiate new paxos round as proposer\n    %% done in on({tx_tm_rtm_propose_yourself...}) handler\n    NewState3;\n\n% received by RTMs\non({?register_TP, {Tid, ItemId, PaxosID, TP}} = Msg, State) ->\n    Role = state_get_role(State),\n    %% TODO merge ?register_TP and accept messages to a single message\n    ?TRACE(\"tx_tm_rtm:on(~p) as ~p~n\", [Msg, Role]),\n    {ErrCodeTx, TmpTxState} = get_tx_entry(Tid, State),\n    _ = case ok =/= ErrCodeTx of\n        true -> %% new / uninitialized\n            %% hold back and handle when corresponding tx_state is\n            %% created in init_RTM\n            %% io:format(\"Holding back a registerTP for id ~p in ~p~n\", [Tid, Role]),\n            T2TxState = tx_state_hold_back(Msg, TmpTxState),\n            NewTxState = tx_state_set_status(T2TxState, uninitialized),\n            set_entry(NewTxState, State);\n        false -> %% ok\n            TxState = tx_state_inc_numtpsregistered(TmpTxState),\n            _ = set_entry(TxState, State),\n            {ok, ItemState} = get_item_entry(ItemId, State),\n\n            case tx_state_is_decided(TxState) of\n                ?undecided ->\n                    %% store TP info to corresponding PaxosId\n                    NewEntry =\n                        tx_item_set_tp_for_paxosid(ItemState, TP, PaxosID),\n                    trigger_delete_if_done(TxState, State),\n                    set_entry(NewEntry, State);\n                Decision when Role =:= tx_tm ->\n                    %% if ?register_TP arrives after tx decision, inform the\n                    %% slowly client directly\n                    %% find matching RTLogEntry and send commit_reply\n                    {PaxosID, RTLogEntry, _TP} =\n                        lists:keyfind(PaxosID, 1,\n                          tx_item_get_paxosids_rtlogs_tps(ItemState)),\n                    msg_tp_do_commit_abort(TP, {PaxosID, RTLogEntry, comm:this(), ItemId}, Decision, state_get_local_snapno(State)),\n                    %% record in txstate and try to delete entry?\n                    NewTxState = tx_state_inc_numinformed(TxState),\n                    trigger_delete_if_done(NewTxState, State),\n                    set_entry(NewTxState, State);\n                _ ->\n                    %% RTMs check whether everything is done\n                    trigger_delete_if_done(TxState, State)\n            end\n    end,\n    State;\n\n% timeout on Tid maybe a proposer crashed? Force proposals with abort.\non({tx_tm_rtm_propose_yourself, Tid}, State) ->\n    ?TRACE(\"tx_tm_rtm:propose_yourself(~p) as ~p~n\", [Tid, state_get_role(State)]),\n    %% only on in rtm processes!\n    ?DBG_ASSERT(tx_tm =/= state_get_role(State)),\n    %% after timeout take over and initiate new paxos round as proposer\n    {ErrCodeTx, TxState} = get_tx_entry(Tid, State),\n%%    log:pal(\"propose yourself (~.0p/~.0p) for: ~.0p ~.0p~n\",\n%%           [self(),\n%%            pid_groups:group_and_name_of(self()),\n%%            Tid, {ErrCodeTx, TxState}]),\n    _ =\n    case ErrCodeTx of\n        new -> ok; %% takeover is not necessary. Was finished successfully.\n        _Any ->\n            log:log(info, \"Takeover by RTM was necessary.\"),\n            Maj = quorum:majority_for_accept(config:read(replication_factor)),\n            RTMs = tx_state_get_rtms(TxState),\n            Role = state_get_role(State),\n            ValidAccs = rtms_get_valid_accpids(RTMs),\n            MaxProposers = length(ValidAccs) + 1,\n            This = comm:this(),\n            case comm:is_valid(This) of\n                false ->\n                    log:log(warn, \"Cannot discover my comm:this().~n\");\n                true ->\n                    {tx_rtm, ThisRTMsNumber} = Role,\n\n            %% add ourselves as learner and\n            %% trigger paxos proposers for new round with own proposal 'abort'\n            GLLearner = state_get_gllearner(State),\n            Proposer = comm:make_global(get_my(Role, proposer)),\n            [ begin\n                  {_, ItemState} = get_item_entry(ItemId, State),\n                  case tx_item_get_decided(ItemState) of\n                      false ->\n%%                          log:pal(\"initiating proposer~n\"),\n                          [ begin\n                                learner:start_paxosid(GLLearner, PaxId, Maj,\n                                                      This, ItemId),\n                                %% add learner to running paxos acceptors\n                                _ = [ comm:send(X,\n                                                {acceptor_add_learner,\n                                                 PaxId, GLLearner})\n                                      || X <- ValidAccs],\n                                proposer:start_paxosid(\n                                  Proposer, PaxId, _Acceptors = ValidAccs, ?abort,\n                                  Maj, MaxProposers, _InitialRound = ThisRTMsNumber),\n                                ok\n                            end\n                            || {PaxId, _RTLog, _TP}\n                                   <- tx_item_get_paxosids_rtlogs_tps(ItemState) ];\n                      _Decision -> % already decided to prepared / abort\n                          log:pal(\"Already decided~n\"),\n                          ok\n                  end\n              end || ItemId <- tx_state_get_txitemids(TxState) ]\n            end\n        end,\n    State;\n\n%% sent by snapshot.erl to update tx_tm on new local snapshot numbers\non({update_snapno, SnapNo}, State) ->\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    state_set_local_snapno(State, SnapNo);\n\n%% failure detector events\non({fd, Cookie, {fd_notify, crash, Pid, Reason}}, State) ->\n    %% in tx_tm and rtm processes! (rtms subscribe to the tm for running tx)\n    ?TRACE_RTM_MGMT(\"tx_tm_rtm:on({crash,...}) of Pid ~p~n\", [Pid]),\n    %% in tx_tm and rtm processes! (rtms subscribe to the tm for running tx)\n    handle_crash(Pid, Reason, Cookie, State, on);\n%% on({fd, _Cookie, {fd_notify, crash, _Pid, _Reason}}\n%%    {_RTMs, _TableName, _Role, _LAcceptor, _GLLearner} = State) ->\n%%     ?TRACE(\"tx_tm_rtm:on:crash of ~p in Transaction ~p~n\", [_Pid, binary_to_term(_Cookie)]),\n%%     %% @todo should we take over, if the TM failed?\n%%     %% Takeover done by timeout (propose yourself). Doing it here could\n%%     %% improve speed, but really necessary!?\n%%     %%\n%%     %% for all Tids make a fold with\n%%     %% NewState = lists:foldr(fun(X, XState) ->\n%%     %%   on({tx_tm_rtm_propose_yourself, Tid}, XState)\n%%     %%                        end, State, listwithalltids),\n%%     State;\non({fd, _Cookie, {fd_notify, _Event, _Pid, _Reason}}, State) ->\n    State;\n\non({new_node_id, Id}, State) ->\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    RTMs = state_get_RTMs(State),\n    IDs = ?RT:get_replica_keys(Id),\n    NewRTMs = [ set_rtmkey(R, I) || {R, I} <- lists:zip(RTMs, IDs) ],\n    state_set_RTMs(State, NewRTMs);\n%% periodic RTM update\non({update_RTMs}, State) ->\n    ?TRACE_RTM_MGMT(\"tx_tm_rtm:on:update_RTMs in Pid ~p ~n\", [self()]),\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    rtm_update_trigger(state_get_RTMs(State)),\n    State;\non({update_RTMs_on_init}, State) ->\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    %% keep trigger alive to avoid restarting it (possibly infected)\n    msg_delay:send_trigger(1, {update_RTMs_on_init}),\n    State;\n%% accept RTM updates\non({get_rtm_reply, InKey, InPid, InAcceptor}, State) ->\n    ?TRACE_RTM_MGMT(\"tx_tm_rtm:on:get_rtm_reply in Pid ~p for Pid ~p and State ~p~n\", [self(), InPid, State]),\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    RTMs = state_get_RTMs(State),\n    NewRTMs = rtms_upd_entry(RTMs, InKey, InPid, InAcceptor),\n    rtms_of_same_dht_node(NewRTMs),\n    state_set_RTMs(State, NewRTMs).\n\n\n-spec on_init(comm:message(), state())\n    -> state() |\n       {'$gen_component', [{on_handler, Handler::gen_component:handler()}], State::state()}.\non_init({get_node_details}, State) ->\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    util:wait_for(fun() -> comm:is_valid(comm:this()) end),\n    comm:send_local(pid_groups:get_my(dht_node),\n                    {get_node_details, comm:this(), [node]}),\n    % update gllearner with determined ip-address\n    state_set_gllearner(State,\n                        comm:make_global(get_my(state_get_role(State),\n                                                learner)));\n\n%% While initializing\non_init({get_node_details_response, NodeDetails}, State) ->\n    ?TRACE(\"tx_tm_rtm:on_init:get_node_details_response State; ~p~n\", [_State]),\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    IdSelf = node:id(node_details:get(NodeDetails, node)),\n    %% provide ids for RTMs (sorted by increasing latency to them).\n    %% first entry is the locally hosted replica of IdSelf\n    RTM_ids = ?RT:get_replica_keys(IdSelf),\n    {NewRTMs, _} = lists:foldr(\n                fun(X, {Acc, I}) ->\n                  {[rtm_entry_new(X, unknown, I, unknown) | Acc ], I - 1}\n                end,\n                {[], length(RTM_ids)}, RTM_ids),\n    rtm_update_once(NewRTMs),\n    state_set_RTMs(State, NewRTMs);\n\non_init({update_RTMs}, State) ->\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    rtm_update_trigger(state_get_RTMs(State)),\n    State;\non_init({update_RTMs_on_init}, State) ->\n    ?TRACE_RTM_MGMT(\"tx_tm_rtm:on_init:update_RTMs in Pid ~p ~n\", [self()]),\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    rtm_update_once(state_get_RTMs(State)),\n    msg_delay:send_trigger(1, {update_RTMs_on_init}),\n    State;\n\non_init({get_rtm_reply, InKey, InPid, InAcceptor}, State) ->\n    ?TRACE_RTM_MGMT(\"tx_tm_rtm:on_init:get_rtm_reply in Pid ~p for Pid ~p State ~p~n\", [self(), InPid, State]),\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    RTMs = state_get_RTMs(State),\n    NewRTMs = rtms_upd_entry(RTMs, InKey, InPid, InAcceptor),\n    case lists:keymember(unknown, 2, NewRTMs) of %% filled all entries?\n        false ->\n            rtms_of_same_dht_node(NewRTMs),\n            gen_component:change_handler(state_set_RTMs(State, NewRTMs), fun ?MODULE:on/2);\n        _ -> state_set_RTMs(State, NewRTMs)\n    end;\n\non_init({new_node_id, Id}, State) ->\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    case state_get_RTMs(State) of\n        [] ->\n            %% new_node_id before first get_node_details: delay this info.\n            comm:send_local(self(), {new_node_id, Id}),\n            State;\n        [_|_] = RTMs ->\n            %% tx_tm processes have always #replicas entries here,\n            %% crashed rtms are listed as 'unknown' (so using\n            %% lists:zip is safe here)\n            IDs = ?RT:get_replica_keys(Id),\n            NewRTMs = [ set_rtmkey(R, I) || {R, I} <- lists:zip(RTMs, IDs) ],\n            state_set_RTMs(State, NewRTMs)\n    end;\n\n%% do not accept new commit requests when not enough rtms are valid\non_init({tx_tm_rtm_commit, _Client, _ClientsID, _TransLog} = Msg, State) ->\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    %% forward request to a node which is ready to serve requests\n    %% there, redirect message to tx_tm\n    RedirectMsg = {?send_to_group_member, tx_tm, Msg},\n    comm:send_local(pid_groups:get_my(routing_table),\n                    {?lookup_aux, ?RT:get_random_node_id(), 0, RedirectMsg}),\n    State;\n\n%% perform operations on already running transactions, should not be a problem\non_init({learner_decide, _ItemId, _PaxosID, _Value} = Msg, State) ->\n    on(Msg, State);\non_init({tx_tm_rtm_tid_isdone, _TxId} = Msg, State) ->\n    on(Msg, State);\non_init({?tp_committed, _ItemId} = Msg, State) ->\n    on(Msg, State);\non_init({delete_if_still_uninitialized, _ItemId} = Msg, State) ->\n    on(Msg, State);\non_init({?tx_tm_rtm_delete, _TxId, _Decision} = Msg, State) ->\n    on(Msg, State);\non_init({?register_TP, {Tid, _ItemId, _PaxosID, _TP}} = Msg, State) ->\n    {ErrCodeTx, _TxState} = get_tx_entry(Tid, State),\n    case new =:= ErrCodeTx of\n        true ->\n            msg_delay:send_local(1, self(), Msg),\n            State;\n        false ->\n            on(Msg, State)\n    end;\non_init({update_snapno, SnapNo}, State) ->\n    %% only in tx_tm not in rtm processes!\n    ?DBG_ASSERT(tx_tm =:= state_get_role(State)),\n    state_set_local_snapno(State, SnapNo);\non_init({fd, Cookie, {fd_notify, crash, Pid, Reason}}, State) ->\n    %% only in tx_tm\n    handle_crash(Pid, Reason, Cookie, State, on_init);\non_init({fd, _Cookie, {fd_notify, _Event, _Pid, _Reason}}, State) ->\n    State.\n\n%% functions for periodic RTM updates\n-spec rtm_update_trigger(rtms()) -> ok.\nrtm_update_trigger(RTMs) ->\n    rtm_update_once(RTMs),\n    msg_delay:send_trigger(config:read(tx_rtm_update_interval) div 1000,\n                           {update_RTMs}),\n    ok.\n\n-spec rtm_update_once(rtms()) -> ok.\nrtm_update_once(RTMs) ->\n    This = comm:this(),\n    _ = [ begin\n              Name = get_nth_rtm_name(get_nth(RTM)),\n              Key = get_rtmkey(RTM),\n              api_dht_raw:unreliable_lookup(Key, {get_rtm, This, Key, Name})\n          end\n          || RTM <- RTMs],\n    ok.\n\n%% functions for tx processing\n-spec init_RTMs(tx_state(), [tx_item_state()]) -> ok.\ninit_RTMs(TxState, ItemStates) ->\n    ?TRACE(\"tx_tm_rtm:init_RTMs~n\", []),\n    ItemStatesForRTM =\n        [ {tx_item_get_itemid(ItemState),\n           tx_item_get_maj_for_prepared(ItemState),\n           tx_item_get_maj_for_abort(ItemState),\n           tx_item_get_tlog(ItemState)}\n        || ItemState <- ItemStates],\n    RTMs = tx_state_get_rtms(TxState),\n    send_to_rtms(\n      RTMs, fun(X) -> {?tx_tm_rtm_init_RTM, TxState, ItemStatesForRTM, get_nth(X)}\n            end).\n\n-spec init_TPs(tx_state(), [tx_item_state()], non_neg_integer()) -> ok.\ninit_TPs(TxState, ItemStates, LocalSnapNumber) ->\n    ?TRACE(\"tx_tm_rtm:init_TPs~n\", []),\n    %% send to each TP its own record / request including the RTMs to\n    %% be used\n    Tid = tx_state_get_tid(TxState),\n    RTMs = tx_state_get_rtms(TxState),\n    CleanRTMs = rtms_get_valid_rtmpids(RTMs),\n    Accs = rtms_get_valid_accpids(RTMs),\n    TM = comm:this(),\n    _ = [ begin\n          %% ItemState = lists:keyfind(ItemId, 1, ItemStates),\n          ItemId = tx_item_get_itemid(ItemState),\n          [ begin\n                Key = tx_tlog:get_entry_key(RTLog),\n                Msg1 = {?init_TP, {Tid, CleanRTMs, Accs, TM,\n                                   tx_tlog:drop_value(RTLog), ItemId, PaxId,\n                                   LocalSnapNumber}},\n                %% delivers message to a dht_node process, which has\n                %% also the role of a TP\n                api_dht_raw:unreliable_lookup(Key, Msg1)\n            end\n            || {PaxId, RTLog, _TP} <- tx_item_get_paxosids_rtlogs_tps(ItemState) ]\n              %%      end || ItemId <- tx_state_get_txitemids(TxState) ],\n      end || ItemState <- ItemStates ],\n    ok.\n\n-spec get_tx_entry(tx_id(), state())\n                     -> {new | ok | uninitialized, tx_state()}.\nget_tx_entry(Id, State) ->\n    case pdb:get(Id, state_get_tablename(State)) of\n        undefined -> {new, tx_state_new(Id)};\n        Entry -> {tx_state_get_status(Entry), Entry}\n    end.\n\n-spec get_item_entry(tx_item_id(), state())\n        -> {new | uninitialized | ok, tx_item_state()}.\nget_item_entry(Id, State) ->\n    case pdb:get(Id, state_get_tablename(State)) of\n        undefined -> {new, tx_item_new(Id)};\n        Entry -> {tx_item_get_status(Entry), Entry}\n    end.\n\n-spec set_entry(tx_state() | tx_item_state(), state()) -> ok.\nset_entry(NewEntry, State) ->\n    pdb:set(NewEntry, state_get_tablename(State)).\n\n-spec get_paxos_ids(State::state(), TxState::tx_state()) -> [paxos_id()].\nget_paxos_ids(State, TxState) ->\n    lists:flatmap(\n      fun(ItemId) ->\n              {ok, ItemState} = get_item_entry(ItemId, State),\n              [PaxId || {PaxId, _RTLog, _TP}\n                            <- tx_item_get_paxosids_rtlogs_tps(ItemState)]\n      end, tx_state_get_txitemids(TxState)).\n\n-spec inform_client(tx_state(), state(), ?commit | ?abort) -> ok.\ninform_client(TxState, State, Result) ->\n    ?TRACE(\"tx_tm_rtm:inform client~n\", []),\n    Client = tx_state_get_client(TxState),\n    ClientsId = tx_state_get_clientsid(TxState),\n    ClientResult = case Result of\n                       ?commit -> commit;\n                       ?abort -> {abort, get_failed_keys(TxState, State)}\n                   end,\n    case Client of\n        unknown -> ok;\n        _       -> msg_commit_reply(Client, ClientsId, ClientResult)\n    end,\n    ok.\n\n-spec inform_tps(tx_state(), state(), ?commit | ?abort) ->\n                           tx_state().\ninform_tps(TxState, State, Result) ->\n    ?TRACE(\"tx_tm_rtm:inform tps~n\", []),\n    LocalSnapNumber = state_get_local_snapno(State),\n    %% inform TPs\n    This = comm:this(),\n    Informed =\n        lists:foldl(\n          fun(ItemId, Sum) ->\n                  {ok, ItemState} = get_item_entry(ItemId, State),\n                  lists:foldl(\n                    fun({PaxId, RTLogEntry, TP}, Sum2) ->\n                            case comm:is_valid(TP) of\n                                true ->\n                                    msg_tp_do_commit_abort(\n                                      TP,\n                                      {PaxId, RTLogEntry, This, ItemId},\n                                      Result, LocalSnapNumber),\n                                    Sum2 + 1;\n                                _ -> Sum2\n                            end\n                    end, Sum, tx_item_get_paxosids_rtlogs_tps(ItemState))\n          end, 0, tx_state_get_txitemids(TxState)),\n    tx_state_set_numinformed(TxState, Informed).\n\n-spec inform_rtms(tx_id(), tx_state(), ?commit | ?abort) -> ok.\ninform_rtms(TxId, TxState, Result) ->\n    ?TRACE(\"tx_tm_rtm:inform rtms~n\", []),\n    %% inform the rtms stored in the txid:\n    RTMs = tx_state_get_rtms(TxState),\n    _ = [ begin\n              case get_rtmpid(RTM) of\n                  unknown -> ok;\n                  {RTMPid} -> comm:send(RTMPid,\n                                        {?tx_tm_rtm_delete, TxId, Result})\n              end\n          end\n          || RTM <- RTMs ],\n    ok.\n\n-spec trigger_delete_if_done(tx_state(), state()) -> ok.\ntrigger_delete_if_done(TxState, State) ->\n    ?TRACE(\"tx_tm_rtm:trigger delete?~n\", []),\n    case (tx_state_is_decided(TxState)) of\n        ?undecided -> ok;\n        false -> ok;\n        Decision -> %% commit / abort\n            %% @TODO majority informed is sufficient?!\n            case tx_state_all_tps_informed(TxState)\n                     andalso ((tx_state_get_numids(TxState) =:= tx_state_get_numcommitack(TxState))\n                              orelse (tx_tm =/= state_get_role(State)))\n                %%    andalso tx_state_all_tps_registered(TxState)\n            of\n                true ->\n                    TxId = tx_state_get_tid(TxState),\n                    %% wait a bit for slow minority of answers to arrive\n                    %% (delete_if_still_uninitialized is only triggered when\n                    %% this is not enough to catch the fourth learner_decide)\n                    msg_delay:send_local(2, self(),\n                                         {?tx_tm_rtm_delete, TxId, Decision});\n                false -> ok\n            end\n    end, ok.\n\n%% @doc Merges the item states transferred in a ?tx_tm_rtm_init_RTM message\n%%      into the locally known state, initiates new paxos processes and\n%%      returns the holdback (message) queue.\n-spec merge_item_states(Tid::tx_id(), [tx_item_id()],\n                        [{EntryId::tx_item_id(),\n                          Maj_for_prepared::non_neg_integer(),\n                          Maj_for_abort::non_neg_integer(), tx_tlog:tlog_entry()}],\n                         State::state(), Learners::[comm:mypid()], LAcceptor::pid())\n        -> HoldBackQ::[[comm:message()]].\nmerge_item_states(_Tid, [], [], _State, _Learners, _LAcceptor) -> [];\nmerge_item_states(Tid, [EntryId | RestLocal],\n                  [{EntryId, Maj_for_prepared, Maj_for_abort, TLogEntry} | RestNew],\n                  State, Learners, LAcceptor) ->\n    {LocalItemStatus, LocalItem} = get_item_entry(EntryId, State),\n    {TmpItem, TmpItemHoldBackQR} =\n        case LocalItemStatus of\n            new -> %% nothing known locally\n                {tx_item_new(\n                  EntryId, Tid, TLogEntry, Maj_for_prepared,\n                  Maj_for_abort), []};\n            uninitialized ->\n                %% take over hold back from existing entry\n                IHoldBQ = tx_item_get_hold_back(LocalItem),\n                Entry = tx_item_new(\n                          EntryId, Tid, TLogEntry, Maj_for_prepared,\n                          Maj_for_abort),\n                {Entry, lists:reverse(IHoldBQ)};\n            ok ->\n                log:log(error, \"Duplicate init_RTM for an item\", []),\n                %% hold back queue should be empty!\n                {LocalItem, []}\n        end,\n    NewItem = tx_item_set_status(TmpItem, ok),\n    _ = set_entry(NewItem, State),\n    _ = [ acceptor:start_paxosid_local(LAcceptor, PaxId, Learners)\n            || {PaxId, _RTlog, _TP} <- tx_item_get_paxosids_rtlogs_tps(NewItem) ],\n    [TmpItemHoldBackQR |\n         merge_item_states(Tid, RestLocal, RestNew, State, Learners, LAcceptor)].\n\n%% -spec count_messages_for_type(Type::term()) ->\n%%                                      {TypeCount::non_neg_integer(),\n%%                                       OtherCount::non_neg_integer()}.\n%% count_messages_for_type(Type) ->\n%%     {_, Msg} = erlang:process_info(self(), messages),\n%%     lists:foldl(fun(X, {AccType, AccOther} = Acc) ->\n%%                         if is_tuple(X) ->\n%%                                Tag = element(1, X),\n%%                                case Tag of\n%%                                    Type -> {AccType + 1, AccOther};\n%%                                    _    -> {AccType, AccOther + 1}\n%%                                end;\n%%                            true -> Acc\n%%                         end\n%%                 end, {0, 0}, Msg).\n\n%% enough_tps_registered(TxState, State) ->\n%%     BoolV =\n%%         [ begin\n%%               {ok, ItemState} = get_item_entry(X, State),\n%%               {_,_,TPs} =\n%%                   lists:unzip3(tx_item_get_paxosids_rtlogs_tps(ItemState)),\n%%               % TODO: if activated, rather fold over the list of TPs to get the number of valid TPs for better performance(?)\n%%               ValidTPs = [ Y || Y <- TPs, unknown =/= Y],\n%%               length(ValidTPs) >= tx_item_get_maj_for_prepared(ItemState)\n%%           end\n%%           || X <- tx_state_get_txitemids(TxState)],\n%%     lists:foldl(fun(X, Acc) -> Acc andalso X end, true, BoolV).\n\n-spec rtms_of_same_dht_node(rtms()) -> boolean().\nrtms_of_same_dht_node(InRTMs) ->\n    %% group_of may return failed, don't include these\n    Groups =\n        lists:usort(\n          [G || X <- InRTMs, unknown =/= (RTMPid = get_rtmpid(X)),\n                failed =/= (G = pid_groups:group_of(\n                                  comm:make_local(element(1, RTMPid))))]),\n    case length(Groups) of\n        4 -> false;\n        _ ->\n            log:log(info, \"RTMs of same DHT node are used. Please start more Scalaris nodes.~n\"),\n            true\n    end.\n\n-compile({inline, [rtm_entry_new/4, get_rtmkey/1, set_rtmkey/2, get_rtmpid/1,\n                   get_nth/1, %get_accpid/1,\n                   rtms_get_valid_rtmpids/1, rtms_get_valid_accpids/1]}).\n\n-spec rtm_entry_new(?RT:key(), {comm:mypid()} | unknown,\n                    pos_integer(), {comm:mypid()} | unknown) -> rtm().\nrtm_entry_new(Key, RTMPid, Nth, AccPid) -> {Key, RTMPid, Nth, AccPid}.\n-spec get_rtmkey(rtm()) -> ?RT:key().\nget_rtmkey(RTMEntry) -> element(1, RTMEntry).\n-spec set_rtmkey(rtm(), ?RT:key()) -> rtm().\nset_rtmkey(RTMEntry, Val) -> setelement(1, RTMEntry, Val).\n-spec get_rtmpid(rtm()) -> {comm:mypid()} | unknown.\nget_rtmpid(RTMEntry) -> element(2, RTMEntry).\n-spec get_nth(rtm()) -> pos_integer().\nget_nth(RTMEntry)    -> element(3, RTMEntry).\n%% -spec get_accpid(rtm()) -> {comm:mypid()} | unknown.\n%% get_accpid(RTMEntry) -> element(4, RTMEntry).\n\n-spec rtms_get_valid_rtmpids(rtms()) -> [comm:mypid()].\nrtms_get_valid_rtmpids(RTMs) ->\n    [ RTMPid || {_Key, {RTMPid}, _Nth, _AccPid} <- RTMs ].\n-spec rtms_get_valid_accpids(rtms()) -> [comm:mypid()].\nrtms_get_valid_accpids(RTMs) ->\n    [ AccPid || {_Key, _RTMPid, _Nth, {AccPid}} <- RTMs ].\n\n-spec rtms_upd_entry(rtms(), ?RT:key(), comm:mypid(), comm:mypid()) -> rtms().\nrtms_upd_entry(RTMs, InKey, InPid, InAccPid) ->\n    [ case get_rtmkey(Entry) of\n          InKey ->\n              RTMPid = {InPid},\n              case get_rtmpid(Entry) of\n                  RTMPid  -> ok;\n                  unknown -> fd:subscribe_refcount(\n                               comm:reply_as(self(), 3, {fd, tx_tm_rtm, '_'}),\n                               [InPid]);\n                  {XPid}  -> Self = comm:reply_as(self(), 3, {fd, tx_tm_rtm, '_'}),\n                             fd:unsubscribe_refcount(Self, [XPid]),\n                             fd:subscribe_refcount(Self, [InPid])\n              end,\n              rtm_entry_new(InKey, RTMPid, get_nth(Entry), {InAccPid});\n          _ -> Entry\n      end || Entry <- RTMs ].\n\n-spec send_to_rtms(rtms(), fun((rtm()) -> comm:message())) -> ok.\nsend_to_rtms(RTMs, MsgGen) ->\n    _ = [ case get_rtmpid(RTM) of\n              unknown  -> ok;\n              {RTMPid} -> comm:send(RTMPid, MsgGen(RTM))\n          end || RTM <- RTMs ],\n    ok.\n\n-spec get_nth_rtm_name(pos_integer()) -> pid_groups:pidname().\nget_nth_rtm_name(N) -> {tx_rtm, N}.\n\n-spec get_my(pid_groups:pidname(), atom()) -> pid() | failed.\nget_my(Role, PaxosRole) ->\n    pid_groups:get_my({Role, PaxosRole}).\n\n-compile({inline, [state_get_RTMs/1, state_set_RTMs/2,\n                   state_get_tablename/1, state_get_role/1, state_get_lacceptor/1,\n                   state_get_gllearner/1, state_set_gllearner/2,\n                   state_get_opentxnum/1, state_inc_opentxnum/1, state_dec_opentxnum/1,\n                   state_get_local_snapno/1, state_set_local_snapno/2,\n                   state_subscribe/2, state_unsubscribe/2]}).\n\n-spec state_get_RTMs(state())      -> rtms().\nstate_get_RTMs(State)          -> element(1, State).\n-spec state_set_RTMs(state(), rtms())\n                    -> state().\nstate_set_RTMs(State, Val)     -> setelement(1, State, Val).\n-spec state_get_tablename(state()) -> pdb:tableid().\nstate_get_tablename(State)     -> element(2, State).\n-spec state_get_role(state())       -> pid_groups:pidname().\nstate_get_role(State)          -> element(3, State).\n-spec state_get_lacceptor(state())  -> pid().\nstate_get_lacceptor(State)     -> element(4, State).\n-spec state_get_gllearner(state()) -> comm:mypid().\nstate_get_gllearner(State) -> element(5, State).\n-spec state_set_gllearner(state(), comm:mypid()) -> state().\nstate_set_gllearner(State, Pid) -> setelement(5, State, Pid).\n-spec state_get_opentxnum(state()) -> non_neg_integer().\nstate_get_opentxnum(State) -> element(6, State).\n-spec state_inc_opentxnum(state()) -> state().\nstate_inc_opentxnum(State) -> setelement(6, State, element(6, State) + 1).\n-spec state_dec_opentxnum(state()) -> state().\nstate_dec_opentxnum(State) ->\n    setelement(6, State, erlang:max(0, element(6, State) - 1)).\n-spec state_get_local_snapno(state()) -> non_neg_integer().\nstate_get_local_snapno(State) -> element(7, State).\n-spec state_set_local_snapno(state(), non_neg_integer()) -> state().\nstate_set_local_snapno(State, No) -> setelement(7, State, No).\n\n-spec state_subscribe(state(), comm:mypid()) -> state().\nstate_subscribe(State, Pid) ->\n    Self = comm:reply_as(self(), 3, {fd, state_get_role(State), '_'}),\n    fd:subscribe_refcount(Self, [Pid]),\n    State.\n\n-spec state_unsubscribe(state(), comm:mypid()) -> state().\nstate_unsubscribe(State, Pid) ->\n    Self = comm:reply_as(self(), 3, {fd, state_get_role(State), '_'}),\n    fd:unsubscribe_refcount(Self, [Pid]),\n    State.\n\n-spec get_failed_keys(tx_state(), state()) -> [client_key()].\nget_failed_keys(TxState, State) ->\n    NumAbort = tx_state_get_numabort(TxState),\n    Result =\n    case NumAbort of\n        0 -> [];\n        _ ->\n            [ tx_tlog:get_entry_key(tx_item_get_tlog(TxItem))\n                || TxItemId <- tx_state_get_txitemids(TxState),\n                   ?abort =:= tx_item_get_decided(\n                     TxItem = element(2, get_item_entry(TxItemId, State)))]\n    end,\n    ?DBG_ASSERT2(length(Result) =:= NumAbort,\n                 {'NumAbort counter differs from actual TxItem states',\n                  length(Result), '=/=', NumAbort}),\n    Result.\n\n-spec handle_crash(comm:mypid(), Reason::fd:reason(), Cookie::tx_tm_rtm | pid_groups:pidname(),\n                   state(), on | on_init)\n                  -> state() |\n                     {'$gen_component',\n                      [{on_handler, fun((comm:message(), state()) -> state())}],\n                      state()}.\nhandle_crash(Pid, _Reason, _Cookie, State, Handler) ->\n    %% tm: update rtms\n    %% rtm: take over tx\n\n    %% in case an rtm crashed: (RTMs is an empty list for rtms)\n    RTMs = state_get_RTMs(State),\n    This = comm:this(),\n    NewRTMs = [ case get_rtmpid(RTM) of\n                    X when unknown =:= X orelse {Pid} =:= X ->\n                        I = get_nth(RTM),\n                        Name = get_nth_rtm_name(I),\n                        Key = get_rtmkey(RTM),\n                        api_dht_raw:unreliable_lookup(\n                          Key, {get_rtm, This, Key, Name}),\n                        rtm_entry_new(Key, unknown, I, unknown);\n                    _ -> RTM\n                end\n                || RTM <- RTMs ],\n\n    %% in case a tm crashed:\n    %% scan over all running transactions and delete this Pid\n    %% if necessary, takeover the tx and try deciding with abort\n    NewState =\n        case state_get_role(State) of\n            tx_tm -> State;\n            _ -> %% tx_rtm\n                lists:foldl(\n                  fun(X,StateIter) ->\n                          case is_tx_state(X) of\n                              true ->\n                                  log:pal(\n                                    \"propose yourself (~.0p/~.0p) for: ~.0p~n\",\n                                    [self(),\n                                     pid_groups:group_and_name_of(self()),\n                                     tx_state_get_tid(X)]),\n                                  on({tx_tm_rtm_propose_yourself, tx_state_get_tid(X)}, StateIter);\n                              false -> StateIter\n                          end\n                  end, State, pdb:tab2list(state_get_tablename(State)))\n        end,\n\n    %% in case an rtm crashed: (RTMs is an empty list for rtms)\n    %% no longer use this RTM\n    ValidRTMs = [ X || X <- NewRTMs, unknown =/= get_rtmpid(X) ],\n    case length(ValidRTMs) < 3\n        andalso tx_tm =:= state_get_role(NewState)\n        andalso on =:= Handler of\n        true ->\n            gen_component:change_handler(\n              state_set_RTMs(NewState, NewRTMs),\n              fun ?MODULE:on_init/2);\n        false -> state_set_RTMs(NewState, NewRTMs)\n    end.\n\n-spec is_tx_state(tx_state() | tx_item_state()) -> boolean().\nis_tx_state(X) when is_tuple(X) ->\n    ?tx_state =:= element(2, X);\nis_tx_state(_) -> false.\n\n%% @doc Checks whether config parameters for tx_tm_rtm exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(replication_factor) and\n    config:cfg_is_greater_than(replication_factor, 0) and\n\n    config:cfg_is_integer(tx_timeout) and\n    config:cfg_is_greater_than(tx_timeout, 0) and\n    config:cfg_is_integer(tx_rtm_update_interval) and\n    config:cfg_is_greater_than(tx_rtm_update_interval, 0) and\n\n    config:cfg_is_greater_than_equal(tx_timeout, 1000 div 3)\n%%     config:cfg_is_greater_than_equal(tx_timeout, 1000 div 2)\n    .\n"
  },
  {
    "path": "src/transactions/tx_tp.erl",
    "content": "%% @copyright 2009-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Part of generic transactions implementation using Paxos Commit\n%%           The role of a transaction participant.\n%% @version $Id$\n-module(tx_tp).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n%% -define(TRACE_SNAP(X, Y), ct:pal(X, Y)).\n-define(TRACE_SNAP(X, Y), ?TRACE(X, Y)).\n\n%%% public interface\n\n%%% functions for gen_component module and supervisor callbacks\n-export([init/0, on_init_TP/2]).\n-export([on_do_commit_abort/4, on_do_commit_abort_fwd/7]).\n\n-spec init() -> pdb:tableid().\ninit() ->\n    %% For easier debugging, use a named table (generates an atom)\n    %%TableName = erlang:list_to_atom(pid_groups:group_to_filename(pid_groups:my_groupname()) ++ \"_tx_tp\"),\n    %%Table = pdb:new(TableName, [set, protected, named_table]).\n    %% use random table name provided by ets to *not* generate an atom\n    _Table = pdb:new(?MODULE, [set]).\n\n%%\n%% Attention: this is not a separate process!!\n%%            It runs inside the dht_node to get access to the db_dht\n%%\n\n-spec on_init_TP({tx_tm_rtm:tx_id(),\n                  [comm:mypid()], [comm:mypid()], comm:mypid(),\n                  tx_tlog:tlog_entry(),\n                  tx_tm_rtm:tx_item_id(),\n                  tx_tm_rtm:paxos_id(), tx_tlog:snap_number()},\n                  dht_node_state:state()) -> dht_node_state:state().\n%% messages handled in dht_node context:\n%% PreCond: check for DB responsibility must still be valid (ref. lookup_fin handling)\non_init_TP({Tid, RTMs, Accs, TM, RTLogEntry, ItemId, PaxId, _SnapNo}, DHT_Node_State) ->\n    %?TRACE(\"tx_tp:on_init_TP({..., ...})~n\", []),\n    %% validate locally via callback\n    DB = dht_node_state:get(DHT_Node_State, db),\n    LocalSnapNumber = snapshot_state:get_number(dht_node_state:get(DHT_Node_State,snapshot_state)),\n    %% check only necessary in case of damaged routing\n    %% but: check already done by lookup_fin which uses post_op and thus the check is still valid\n%%     Key = tx_tlog:get_entry_key(RTLogEntry),\n%%     case dht_node_state:is_db_responsible(Key, DHT_Node_State) of\n%%         true ->\n            {TmpDB, Proposal} =\n                case tx_tlog:get_entry_operation(RTLogEntry) of\n                    ?read ->\n                        rdht_tx_read:validate(DB, LocalSnapNumber, RTLogEntry);\n                    ?write ->\n                        rdht_tx_write:validate(DB, LocalSnapNumber, RTLogEntry)\n                end,\n            %% initiate a paxos proposer round 0 with the proposal\n            R = config:read(replication_factor),\n            Proposer = comm:make_global(pid_groups:get_my(paxos_proposer)),\n            proposer:start_paxosid(Proposer, PaxId,\n                                   _Acceptors = Accs, Proposal,\n                                   _Maj = quorum:majority_for_accept(R),\n                                   _MaxProposers = R + 1, %% rtms + client\n                                   0),\n            %% send registerTP to each RTM (send with it the learner id)\n            This = comm:this(),\n            _ = [ comm:send(X, {?register_TP, {Tid, ItemId, PaxId, This}})\n                    || X <- [TM | RTMs], unknown =/= X],\n            %% (optimized: embed the proposer's accept message in registerTP message)\n            %% remember own proposal for lock release\n            TP_DB = dht_node_state:get(DHT_Node_State, tx_tp_db),\n            pdb:set({PaxId, Proposal}, TP_DB),\n\n            dht_node_state:set_db(DHT_Node_State, TmpDB)%;\n%%         false ->\n%%             %% forward commit to now responsible node\n%%             dht_node_lookup:lookup_aux(\n%%               DHT_Node_State, Key, 0, {?init_TP, _Params}),\n%%             DHT_Node_State\n%%     end\n    .\n-spec on_do_commit_abort({tx_tm_rtm:paxos_id(),\n                          tx_tlog:tlog_entry(),\n                          comm:mypid(),\n                          tx_tm_rtm:tx_item_id()},\n                         ?commit | ?abort, tx_tlog:snap_number(), dht_node_state:state())\n                        -> dht_node_state:state().\non_do_commit_abort({PaxosId, RTLogEntry, TM, TMItemId} = Id, Result, TMSnapNo, DHT_Node_State) ->\n    %?TRACE(\"tx_tp:on_do_commit_abort({, ...})~n\", []),\n    %% inform callback on commit/abort to release locks etc.\n    % get own proposal for lock release\n    TP_DB = dht_node_state:get(DHT_Node_State, tx_tp_db),\n    case pdb:get(PaxosId, TP_DB) of\n        {PaxosId, Proposal} ->\n            NewDB = update_db_or_forward(TM, TMItemId, RTLogEntry, Result, Proposal, TMSnapNo, DHT_Node_State),\n            %% delete corresponding proposer state\n            Proposer = comm:make_global(pid_groups:get_my(paxos_proposer)),\n            proposer:stop_paxosids(Proposer, [PaxosId]),\n            pdb:delete(PaxosId, TP_DB),\n            dht_node_state:set_db(DHT_Node_State, NewDB);\n        undefined ->\n            %% delay or forward commit until corresponding validate seen\n            Key = tx_tlog:get_entry_key(RTLogEntry),\n            case dht_node_state:is_db_responsible(Key, DHT_Node_State) of\n                true ->\n                    %% tx is already commited and we either are the\n                    %% slow minority or we already got such a request\n                    %% and already deleted the state,\n                    %% so we claim we committed\n                    %% msg_delay:send_local(\n                    %%   1, self(), {?tp_do_commit_abort, Id, Result});\n                    comm:send(TM, {?tp_committed, TMItemId});\n                false ->\n                    % we don't have an own proposal yet (no validate seen), so we forward msg as is.\n                    api_dht_raw:unreliable_lookup(Key, {?tp_do_commit_abort, Id,\n                                                        Result, TMSnapNo})\n            end,\n            DHT_Node_State\n    end.\n\n-spec on_do_commit_abort_fwd(comm:mypid(), tx_tm_rtm:tx_item_id(),\n                             tx_tlog:tlog_entry(),\n                             ?commit | ?abort, ?prepared | ?abort, non_neg_integer(),\n                             dht_node_state:state())\n                           -> dht_node_state:state().\non_do_commit_abort_fwd(TM, TMItemId, RTLogEntry, Result, OwnProposal, TMSnapNo, DHT_Node_State) ->\n    NewDB = update_db_or_forward(TM, TMItemId, RTLogEntry, Result, OwnProposal, TMSnapNo, DHT_Node_State),\n    dht_node_state:set_db(DHT_Node_State, NewDB).\n\nupdate_db_or_forward(TM, TMItemId, RTLogEntry, Result, OwnProposal, TMSnapNo, DHT_Node_State) ->\n    %% Check for DB responsibility:\n    DB = dht_node_state:get(DHT_Node_State, db),\n    Key = tx_tlog:get_entry_key(RTLogEntry),\n    SnapState = dht_node_state:get(DHT_Node_State,snapshot_state),\n    OwnSnapNo = snapshot_state:get_number(SnapState),\n    case dht_node_state:is_db_responsible(Key, DHT_Node_State) of\n        true ->\n            ?TRACE(\"~p tx_tp:update_db_or_forward before commit/abort~n\",[comm:this()]),\n            ?TRACE(\"~p tx_tp:update_db_or_forward before db: ~p~n\",[comm:this(),DB]),\n            ?TRACE(\"~p tx_tp:update_db_or_forward before db data: ~p~n\",[comm:this(),db_dht:get_data(DB)]),\n            ?TRACE(\"~p tx_tp:update_db_or_forward before snapshot data: ~p~n\",[comm:this(),db_dht:get_snapshot_data(DB)]),\n            ?TRACE(\"~p tx_tp:update_db_or_forward incoming operation: ~p~n\",[comm:this(),{tx_tlog:get_entry_operation(RTLogEntry), Result}]),\n            Res =\n                case tx_tlog:get_entry_operation(RTLogEntry) of\n                    ?read when Result =:= ?abort ->\n                        rdht_tx_read:abort(DB, RTLogEntry, OwnProposal, TMSnapNo, OwnSnapNo);\n                    ?read when Result =:= ?commit ->\n                        rdht_tx_read:commit(DB, RTLogEntry, OwnProposal, TMSnapNo, OwnSnapNo);\n                    ?write when Result =:= ?abort ->\n                        rdht_tx_write:abort(DB, RTLogEntry, OwnProposal, TMSnapNo, OwnSnapNo);\n                    ?write when Result =:= ?commit ->\n                        rdht_tx_write:commit(DB, RTLogEntry, OwnProposal, TMSnapNo, OwnSnapNo)\n                end,\n            comm:send(TM, {?tp_committed, TMItemId}),\n            % check if snapshot is running and if so, if it's already done\n            case snapshot_state:is_in_progress(SnapState) andalso db_dht:snapshot_is_running(Res) of\n                true ->\n                    case db_dht:snapshot_is_lockfree(Res) of\n                        true ->\n                            ?TRACE_SNAP(\"~p tx_tp:update_db_or_forward snapshot is finally done~n\",[comm:this()]),\n                            comm:send_local(self(), {local_snapshot_is_done});\n                        _ ->\n                            ?TRACE_SNAP(\"~p tx_tp:update_db_or_forward snapshot is still\n                                        not done~n~p\",[comm:this(), Res]),\n                            ok\n                    end;\n                _ ->\n                    ?TRACE_SNAP(\"~p tx_tp:update_db_or_forward something\n                                else~n~p~n~p\",[comm:this(), X, Res]),\n                    ok\n            end,\n            Res;\n        false ->\n            %% forward commit to now responsible node\n            api_dht_raw:unreliable_lookup(Key, {?tp_do_commit_abort_fwd,\n                                                TM, TMItemId, RTLogEntry,\n                                                Result, OwnProposal, TMSnapNo}),\n           DB\n    end.\n"
  },
  {
    "path": "src/tx/tx_tm.erl",
    "content": "% @copyright 2012-2015 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc Transaction validation based on a sequence of Paxos\n%%      round based registers (kv_on_cseq).\n%% @end\n%% @version $Id:$\n-module(tx_tm).\n-author('schintke@zib.de').\n-vsn('$Id:$ ').\n\n%-define(TRACE(X,Y), ct:pal(X,Y)).\n-define(TRACE(_X,_Y), ok).\n-behaviour(gen_component).\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n%% public interface for transaction validation.\n-export([commit/4]).           %% commit a transaction log\n-export([msg_commit_reply/3]). %% expect this message as reply to a commit\n-export([redecide/1]).         %% take over an open tx based on a txid\n-export([redecide_on_key/1]).  %% take over an open tx based on an involved key\n\n%% detect whether txnew is enabled or not\n-export([is_txnew_enabled/0]).\n\n%% TODO: commit should return the txid (maybe on the layer above) and\n%% include that in its replies, so duplicate replies can be detected\n%% and eliminated / ignored by the client or a proxy process. A client\n%% could retrigger the tx if it takes to long. But, if the inform\n%% client message is lost, the system would not be able to reanswer\n%% the request, but would reply with {fail, tx_not_open}.\n\n%% functions for dht embedding\n%%-export([get_my/2]).\n-export([rm_send_update/5]). %% subscribe to rm changes for local tx_ids\n\n%% functions for gen_component module, supervisor callbacks and config\n-export([start_link/2]).\n-export([on/2, init/1]).\n-export([on_init/2]).\n-export([check_config/0]).\n\n-export_type([tx_id/0]).\n\n-include(\"gen_component.hrl\").\n\n\n%% TM on kv_on_cseq\n\n%% commit\n%% 1. assign txid\n%% 2. get locks *and* persist txid\n%%    store TLog (for big values in a tlog use two steps here\n%%      to avoid value transfer via lookups: first lookup the\n%%      responsible node, then send value directly)\n%%    a) write txid (content check ensures, it does not exist already)\n%%    b) read lock: write txid and client into readlock,\n%%                  content check ensures no write lock is set\n%%                  and item is untouched (has same version as in TLog).\n%%       write lock: write txid, planned value, and client into write lock\n%%                  content check ensures no write lock is set,\n%%                  and no read lock is set,\n%%                  and item is untouched (has same version as in TLog).\n%% 3. wait for txid persisted and all keys replied\n%% 4. if we got all locks, decide on commit else abort.\n%%    write decision to txid record in the dht to be able to decide later\n%%    in the same way. (otherwise arguments below would become true [*])\n%%    wait for that to finish.\n%% 5. release all gotten locks and actually perform the planned value\n%%    change when decided commit or rollback in case of abort.\n%%    read lock: simply remove the readlock\n%%    write lock: write the value and increase the version number\n%%                in the database field by one.\n%% 6. if all items released the locks (majority of replicas) and txid is written\n%%    a) inform client\n%%    b) remove txid\n%% Done\n\n%% it could happen that a client is informed twice on the outcome of a\n%% tx?! Example: read-read-commit: tm performs steps 1-5 and informs\n%% in step 6 the client and then crashes. Another tm takes over and\n%% cannot decide whether the client was informed or not.\n\n%% This scenario may also happen in the old transaction protocol?!\n\n\n%% What happens if a single txid entry survives?\n%% 1. the tx start is gone wrong\n%% 2. everything was done and the delete failed.\n%% We can distiguish both cases on the content of the entry.\n%% 1. the txid status is open\n%% 2. the txid statis is commit/abort\n%% We do not know whether the client was informed or not, so we should\n%% inform it once more?!.\n\n\n\n\n%% [*]\n\n%% 1. Inconsistent decisions when not persisting decision to txid entry:\n\n%% Example: read-read-commit: tm performs steps 1-5 and informs in\n%% step 6 the client and then crashes. Another tm takes over and\n%% cannot detect whether 2-5 were done or not (to decide in the same\n%% way) (he cannot decide whether the tx was actually commited or\n%% aborted (maybe we now would have to abort, but the first tm was\n%% able to commit - the system state changed in the meantime))?! There\n%% remained no system state change to detect that.\n\n%% This scenario may also happen in the old transaction protocol?!\n\n\n%% 2. Inconsistent decision when writing decision and freeing locks is\n%%    done in a single step concurrently:\n%%\n%% Scenario 1. may happen when all locks are removed and the txid\n%% remains without the decision persisted. (Why can that happen: Too\n%% many messages lost...? / The TM crashed in that moment).\n\n%% ** Why store the complete TLog in the txid field?\n\n%% We could distribute it to the respective items and store the\n%% necessary information there in the read and write locks.\n\n\n%% recover:\n%% a lock is found + no txid\n%% -> try to write txid with abort and release lock when successful\n\n%% a lock is found + a txid is found with status open (lock acquisition phase)\n%% -> contact other items and propose abort as we do not know the content\n%%    if all are prepared - decide commit (they all were already locked)\n%%    else decide abort\n\n%% a lock is found + a txid is found with status commit/abort\n%% -> quorum free the locks + inform client + delete txid\n\n%% a txid is found in state open\n%% -> contact tx participants and try to decide abort\n\n%% a txid is found in state decided/abort\n%% -> look items up and check whether the locks are gone (resend\n%%    decision with txid, content check will do the rest)\n\n%% getting rid of a single remaining txid:\n%%   if it is decided and the other replicas are empty, delete it.\n%%   if it is open and the other replicas are empty\n\n\n%% concurrency control\n%% recover a crash that happened during the locking phase\n\n%% recover a crash that happened during decision phase\n\n%% recover a crash that happened during actual commit phase\n\n\n\n\n\n-type tx_id() :: ?RT:key(). %% was {?tx_id, uid:global_uid()}.\n\n-type state() ::\n    {TableName      :: pdb:tableid(),\n     Id             :: ?RT:key(),\n     OpenTxNum      :: non_neg_integer()}.\n\n%% getter and setters for tx_state implemented at the end of this file.\n-define(enum_id,              1).\n-define(enum_marker,          2).\n-define(enum_client,          3).\n-define(enum_clientid,        4).\n-define(enum_status,          5).\n-define(enum_tlog,            6).\n-define(enum_txid_round,      7).\n-define(enum_txid_writtenval, 8).\n-define(enum_open_locks,      9).\n-define(enum_open_commits,   10).\n-define(enum_failed_locks,   11).\n\n-type tx_state() ::\n        {tx_id(),                  %% Tid\n         ?tx_state,                %% type-marker (unnecessary in principle)\n         comm:mypid(),             %% Client\n         any(),                    %% ClientsId,\n         open | ?commit | ?abort,  %% status\n         tx_tlog:tlog(),           %% tlog to be committed\n         pr:pr() | '_',   %% next txid write token for fast write\n         any(),                    %% last written txid value for fast write\n         %% keys of locks to be confirmed.\n         [?RT:key() | client_key() | binary()],\n         %% keys, rounds, and oldvalues of commits to be confirmed,\n         [{?RT:key() | client_key() | binary(),\n           pr:pr() | '_', %% for fast write, set on lock reply\n           any() %% old value for fast write\n          }],\n         %% keys that failed to be locked,\n         [?RT:key() | client_key() | binary()]\n        }.\n\n\n%% non_neg_integer(),        %% NumIds,\n         %% non_neg_integer(),        %% NumPrepared,\n         %% non_neg_integer(),        %% NumAbort,\n         %% non_neg_integer(),        %% NumTPsInformed\n         %% non_neg_integer(),        %% Number of items committed\n         %% non_neg_integer(),        %% NumTpsRegistered\n         %% new | uninitialized | ok  %% status: new / uninitialized / ok\n         %% }.\n\n%% public interface for transaction validation.\n%% ClientsID may be nil, its not used by tx_tm. It will be repeated in\n%% replies to allow to map replies to the right requests in the\n%% client.\n%% TODO: we could eliminate ClientsID from this implementation, as we\n%%       have comm:reply_as - maybe that would be a bit slower.\n-spec commit(comm:erl_local_pid(), comm:mypid(), any(), tx_tlog:tlog()) -> ok.\ncommit(TM, Client, ClientsID, TLog) ->\n    Msg = {tx_tm_commit, Client, ClientsID, TLog},\n    comm:send_local(TM, Msg).\n\n%% messages a client has to expect when using this module\n-spec msg_commit_reply(comm:mypid(), any(),\n                       commit | {abort, [client_key() | binary]}) -> ok.\nmsg_commit_reply(Client, ClientsID, Result) ->\n    comm:send(Client, {tx_tm_commit_reply, ClientsID, Result}).\n\n%% @doc takes care of a transaction with the given transaction id.\n%% TODO: to be implemented\n-spec redecide(tx_id()) -> ok.\nredecide(_TxId) ->\n    %% quorum read txid from dht to learn about involved keys.\n    %% if not existing: nothing is known -> {fail, tx_not_open}.\n    %% else\n    %% quorum read all involved keys.\n\n    %% If all items locked with tx_id make progress: decide to prepared,\n    %% release the locks, inform client and release txid entry in dht.\n\n    %% If only some items are locked, the original tm may still try to\n    %% lock the others. If we decide to abort, we have to ensure that\n    %% concurrent tms for the same tx will also decide to abort. This\n    %% can be done by increasing the version number\n    ok.\n\n-spec redecide_on_key(?RT:key()) -> ok.\nredecide_on_key(_Key) -> ok.\n\n-spec is_txnew_enabled() -> boolean().\n-ifdef(TXNEW).\nis_txnew_enabled() -> true.\n-else.\nis_txnew_enabled() -> false.\n-endif.\n\n%% @doc Notifies the tx_tm of a changed node ID.\n-spec rm_send_update(Subscriber::pid(), Tag::?MODULE,\n                     OldNeighbors::nodelist:neighborhood(),\n                     NewNeighbors::nodelist:neighborhood(),\n                     Reason::rm_loop:reason()) -> ok.\nrm_send_update(Pid, ?MODULE, OldNeighbors, NewNeighbors, _Reason) ->\n    OldId = node:id(nodelist:node(OldNeighbors)),\n    NewId = node:id(nodelist:node(NewNeighbors)),\n    if OldId =/= NewId -> comm:send_local(Pid, {new_node_id, NewId});\n       true            -> ok\n    end.\n\n%% be startable via supervisor, use gen_component\n%% TODO: for now: be compatible with old tx startup,\n%%       later we can eliminate the role (the atom tx_tm_new)?\n-spec start_link(pid_groups:groupname(), any()) -> {ok, pid()}.\nstart_link(DHTNodeGroup, tx_tm_new) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2,\n                             [], %% empty list as parameter to init\n                             [{pid_groups_join_as, DHTNodeGroup, tx_tm_new},\n                              {spawn_opts, [{fullsweep_after, 0},\n                                            {min_heap_size, 16383}]}]).\n\n%% initialize: return initial state.\n-spec init([]) -> state() | {'$gen_component', [{on_handler, gen_component:handler()},...],\n                             state()}.\ninit([]) ->\n    _Role = pid_groups:my_pidname(),\n    ?TRACE(\"tx_tm:init for instance: ~p ~p~n\",\n           [pid_groups:my_groupname(), _Role]),\n    %% For easier debugging, use a named table (generates an atom)\n    %%TableName = erlang:list_to_atom(pid_groups:group_to_filename(pid_groups:my_groupname()) ++ \"_tx_tm_\" ++ atom_to_list(_Role)),\n    %%Table = pdb:new(TableName, [set, protected, named_table]),\n    %% use random table name provided by ets to *not* generate an atom\n    Table = pdb:new(?MODULE, [set]),\n\n    comm:send_local(self(), {get_node_details}),\n    %% subscribe to id changes\n    rm_loop:subscribe(self(), ?MODULE,\n                      fun rm_loop:subscribe_dneighbor_change_filter/3,\n                      fun ?MODULE:rm_send_update/5, inf),\n    State = {Table, _ID = unknown, _OpenTxNum = 0},\n    gen_component:change_handler(State, fun ?MODULE:on_init/2).\n\n-spec on(comm:message(), state()) -> state().\n\n%% step 1: Write the tx to the txid store\non({tx_tm_commit, Client, ClientsID, TLog}, State) ->\n    ?TRACE(\"tx_tm:on({commit, ...}) for TLog ~p as ~p~n\",\n           [TLog, state_get_role(State)]),\n\n    TLogUid = uid:get_global_uid(),\n    NewTid = {?tx_id, TLogUid},\n\n    NewTidKey = ?RT:hash_key(term_to_binary(NewTid)),\n    TxState = tx_state_new(NewTidKey, Client, ClientsID, TLog),\n    _ = set_entry(TxState, State),\n\n    ReplyAsTxIdStored = comm:reply_as(self(), 3,\n                                      {tx_tm_txid_stored, NewTidKey, '_'}),\n\n    %% initiate storage of txid\n    Keys = [ tx_tlog:get_entry_key(X) || X <- TLog ],\n    txid_on_cseq:new(NewTidKey, Keys, ReplyAsTxIdStored),\n\n    state_inc_opentxnum(State);\n\n%% tx is stored in txid store\non({tx_tm_txid_stored, TxId,\n    {qwrite_done, _ReqId, NextWriteRound, WrittenVal, _WriteRet}}, State) ->\n    %% store NextWriteRound and WrittenValue for fast write of decision\n    TxState = get_entry(TxId, State),\n    T1TxState = tx_state_set_txid_next_write_token(TxState, NextWriteRound),\n    NewTxState = tx_state_set_txid_written_value(T1TxState, WrittenVal),\n    set_entry(NewTxState, State),\n    %% progress to the next step in the protocol\n    gen_component:post_op({tx_tm_acquire_locks, TxId}, State);\n\n%% step 2: acquire locks for tlog entries (this cannot be done\n%% overlapping with step 1 because when locks were acquired without\n%% the tx_id stored properly, we cannot safely cleanup remaining\n%% locks, as we do not know whether the TM is just slow (and will\n%% later write the tx_id and decide the tx and relies on the set\n%% locks) or crashed and will never write and decide the tx_id).\non({tx_tm_acquire_locks, TxId}, State) ->\n    %% initiate lock operations on all keys\n    TxState = get_entry(TxId, State),\n    TLog = tx_state_tlog(TxState),\n    _ = [\n     begin\n         ReplyAs =\n             comm:reply_as(self(), 4,\n                           {tx_tm_lock_get_done, TxId,\n                            tx_tlog:get_entry_key(X), '_'}),\n         kv_on_cseq:set_lock(X, TxId, ReplyAs)\n     end || X <- TLog ],\n    State;\n\n%% lock for a tlog entry was acquired (or not)\non({tx_tm_lock_get_done, TxId, Key,\n    {qwrite_done, _ReqId, NextRound, WrittenVal, _WriteRet}}, State) ->\n    case get_entry(TxId, State) of\n        undefined ->\n            State;\n        TxState ->\n            NewOpenLocks = lists:delete(Key, tx_state_open_locks(TxState)),\n            T1TxState = tx_state_set_open_locks(TxState, NewOpenLocks),\n            %% remember nextround and writtenval for tx execution\n            NewTxState = tx_state_add_nextround_writtenval_for_commit(\n                           T1TxState, Key, NextRound, WrittenVal),\n            set_entry(NewTxState, State),\n            case NewOpenLocks of\n                [] ->\n                    %% all locks acquired? -> proceed to next step\n                    Round = tx_state_txid_next_write_token(NewTxState),\n                    OldVal = tx_state_txid_written_value(NewTxState),\n                    case tx_state_failed_locks(NewTxState) of\n                        [] ->\n                            %% io:format(\"decide commit~n\"),\n                            gen_component:post_op(\n                              {tx_tm_write_decision, TxId, commit, Round, OldVal}, State);\n                        _ ->\n                            gen_component:post_op(\n                              {tx_tm_write_decision, TxId, abort, Round, OldVal}, State)\n                    end;\n                _ -> State\n            end\n    end;\n\non({tx_tm_lock_get_done, TxId, Key,\n    {qwrite_deny, _ReqId, NextRound, WrittenVal, _Reason}}, State) ->\n    %% a lock was not acquirable -> abort the tx\n    case get_entry(TxId, State) of\n        undefined ->\n            State;\n        TxState ->\n            %% for performance reasons we only record the first key\n            %% that made the tx aborting\n            NewOpenLocks = lists:delete(Key, tx_state_open_locks(TxState)),\n            NewFailedLocks = [Key | tx_state_failed_locks(TxState)],\n            T1TxState = tx_state_set_open_locks(TxState, NewOpenLocks),\n            T2TxState = tx_state_set_failed_locks(T1TxState, NewFailedLocks),\n            NewTxState = tx_state_add_nextround_writtenval_for_commit(\n                           T2TxState, Key, NextRound, WrittenVal),\n            set_entry(NewTxState, State),\n            case NewOpenLocks of\n                [] ->\n                    %% all locks acquired? -> proceed to next step\n                    Round = tx_state_txid_next_write_token(NewTxState),\n                    OldVal = tx_state_txid_written_value(NewTxState),\n                    %%log:log(\"decide abort~n\"),\n                    gen_component:post_op(\n                      {tx_tm_write_decision, TxId, abort, Round, OldVal}, State);\n                _ -> State\n            end\n    end;\n\n\n%% step 2: Write the decision to the txid store\non({tx_tm_write_decision, TxId, Decision, WriteRound, OldVal}, State) ->\n    ReplyAs = comm:reply_as(self(), 3,\n                            {tx_tm_decision_stored, TxId, '_'}),\n\n%%    log:log(\"Decision ~p~n\", [Decision]),\n    txid_on_cseq:decide(TxId, Decision, ReplyAs, WriteRound, OldVal),\n    State;\n\non({tx_tm_decision_stored, TxId,\n    {qwrite_done, _ReqId, _Round, Decision, _WriteRet}}, State) ->\n    gen_component:post_op({tx_tm_execute_decision, TxId, Decision}, State);\n\non({tx_tm_execute_decision, TxId, Decision}, State) ->\n    TxState = get_entry(TxId, State),\n    TLog = tx_state_tlog(TxState),\n    _ = [ begin\n              ReplyAs = comm:reply_as(self(), 5,\n                                      {tx_tm_executed_decision, TxId,\n                                       Key, Decision, '_'}),\n              %% retrieve corresponding TLog Entry\n              TLogEntry = tx_tlog:find_entry_by_key(TLog, Key),\n              Op = tx_tlog:get_entry_operation(TLogEntry),\n              case {Op, Decision} of\n                  {?read, commit} ->\n                      kv_on_cseq:commit_read(\n                        TLogEntry, TxId, ReplyAs, NextRound, OldVal);\n                  {?write, commit} ->\n                      kv_on_cseq:commit_write(\n                        TLogEntry, TxId, ReplyAs, NextRound, OldVal);\n                  {?read, abort} ->\n                      kv_on_cseq:abort_read(\n                        TLogEntry, TxId, ReplyAs, NextRound, OldVal);\n                  {?write, abort} ->\n                      kv_on_cseq:abort_write(\n                        TLogEntry, TxId, ReplyAs, NextRound, OldVal)\n              end\n          end\n          || {Key, NextRound, OldVal} <- tx_state_open_commits(TxState) ],\n    State;\n\non({tx_tm_executed_decision, TxId, Key, Decision,\n    _QwriteAnswer},\n    %% either: {qwrite_done, _ReqId, _Round, _Val, _WriteRet}}\n    %% or:     {qwrite_deny, _ReqId, _Round, _Val, Reason}}\n    %% deny is also ok, probably the decision was propagated otherwise\n    %% (write_through during concurrency, so we accept a write deny).\n    %% We had the lock, and only we can release it.\n    State) ->\n    case get_entry(TxId, State) of\n        %% this should not happen!\n        %% undefined ->\n        %%     State;\n        TxState ->\n            NewOpenCommits = lists:keydelete(Key, 1, tx_state_open_commits(TxState)),\n            case NewOpenCommits of\n                [] ->\n                    %% all keys committed -> inform client\n                    Client = tx_state_client(TxState),\n                    ClientsId = tx_state_clients_id(TxState),\n                    case Decision of\n                        commit ->\n                            msg_commit_reply(Client, ClientsId, commit);\n                        abort ->\n                            msg_commit_reply(Client, ClientsId, {abort, tx_state_failed_locks(TxState)})\n                    end,\n                    %% proceed to next step\n                    gen_component:post_op({tx_tm_delete_txid, TxId}, State);\n                _ ->\n                    NewTxState = tx_state_set_open_commits(TxState, NewOpenCommits),\n                    set_entry(NewTxState, State),\n                    State\n            end\n    end;\n\non({tx_tm_delete_txid, TxId}, State) ->\n    %% TODO: delete in dht\n    %% delete locally:\n    pdb:delete(TxId, state_get_tablename(State)),\n    State;\n\non({new_node_id, Id}, State) ->\n    %% change txid range to remain local with the txids\n    state_set_id(State, Id).\n\n-spec on_init(comm:message(), state())\n    -> state() |\n       {'$gen_component', [{on_handler, Handler::gen_component:handler()}], State::state()}.\non_init({get_node_details}, State) ->\n    util:wait_for(fun() -> comm:is_valid(comm:this()) end),\n    comm:send_local(pid_groups:get_my(dht_node),\n                    {get_node_details, comm:this(), [node]}),\n    State;\n\n%% While initializing\non_init({get_node_details_response, NodeDetails}, State) ->\n    ?TRACE(\"tx_tm:on_init:get_node_details_response State; ~p~n\", [_State]),\n    IdSelf = node:id(node_details:get(NodeDetails, node)),\n    NewState = state_set_id(State, IdSelf),\n    gen_component:change_handler(NewState, fun ?MODULE:on/2);\n\non_init({new_node_id, Id}, State) ->\n    state_set_id(State, Id);\n\n%% do not accept new commit requests when not enough rtms are valid\non_init({tx_tm_commit, _Client, _ClientsID, _TransLog} = Msg, State) ->\n    %% forward request to a node which is ready to serve requests.\n    %% There, redirect message to tx_tm_new\n    RedirectMsg = {?send_to_group_member, tx_tm_new, Msg},\n    api_dht_raw:unreliable_lookup(?RT:get_random_node_id(), RedirectMsg),\n    State.\n\n\n\n-spec get_entry(any(), state()) -> tx_state() | undefined.\nget_entry(ReqId, State) ->\n    pdb:get(ReqId, state_get_tablename(State)).\n\n-spec set_entry(tx_state(), state()) -> ok.\nset_entry(NewEntry, State) ->\n    pdb:set(NewEntry, state_get_tablename(State)).\n\n%% -spec inform_client(tx_state(), state(), ?commit | ?abort) -> ok.\n%% inform_client(TxState, State, Result) ->\n%%     ?TRACE(\"tx_tm:inform client~n\", []),\n%%     Client = tx_state_get_client(TxState),\n%%     ClientsId = tx_state_get_clientsid(TxState),\n%%     ClientResult = case Result of\n%%                        ?commit -> commit;\n%%                        ?abort -> {abort, get_failed_keys(TxState, State)}\n%%                    end,\n%%     case Client of\n%%         unknown -> ok;\n%%         _       -> msg_commit_reply(Client, ClientsId, ClientResult)\n%%     end,\n%%    ok.\n\n%% -spec get_my(, %% pid_groups:pidname(),\n%%              atom()) -> pid() | failed.\n%% get_my(Role, PaxosRole) ->\n%%     PidName = list_to_existing_atom(\n%%                 atom_to_list(Role) ++ \"_\" ++ atom_to_list(PaxosRole)),\n%%     pid_groups:get_my(PidName).\n%%\n-spec state_get_tablename(state()) -> pdb:tableid().\nstate_get_tablename(State)     -> element(1, State).\n\n-spec state_set_id(state(), ?RT:key()) -> state().\nstate_set_id(State, Id) -> setelement(2, State, Id).\n\n%%-spec state_get_opentxnum(state()) -> non_neg_integer().\n%% state_get_opentxnum(State) -> element(3, State).\n-spec state_inc_opentxnum(state()) -> state().\nstate_inc_opentxnum(State) -> setelement(3, State, element(3, State) + 1).\n%%-spec state_dec_opentxnum(state()) -> state().\n%% state_dec_opentxnum(State) ->\n%%     setelement(3, State, erlang:max(0, element(3, State) - 1)).\n\n\n%% @doc Checks whether config parameters for tx_tm exist and are\n%%      valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(replication_factor) and\n    config:cfg_is_greater_than(replication_factor, 0)\n    .\n\n-spec tx_state_new(tx_id(), comm:mypid(), any(), tx_tlog:tlog()) -> tx_state().\ntx_state_new(TxId, Client, ClientId, TLog) ->\n    InvolvedKeys = [ tx_tlog:get_entry_key(X) || X <- TLog ],\n    {TxId, ?tx_state, Client, ClientId, open, TLog,\n     _NextWriteRound= '_', %% assigned by reply to txid_on_cseq:new\n     prbr_bottom, %% last written val for txid\n     _ToBeLockedKeys = InvolvedKeys,\n     _ToBeCommittedKeys =\n         [ {X, _NextRoundPlaceholder = '_', no_value_yet} %% written_value_placeholder}\n           || X <- InvolvedKeys],\n     _FailedLocks = []}.\n\n%% ?enum_marker\n%% ?enum_id\n%% ?enum_client\n%% ?enum_clientid\n%% ?enum_status\n%% ?enum_tlog\n%% ?enum_txid_round\n%% ?enum_txid_writtenval\n%% ?enum_open_locks\n%% ?enum_open_commits\n%% ?enum_failed_locks\n\n-spec tx_state_client(tx_state()) -> comm:mypid().\ntx_state_client(TxState) -> element(?enum_client, TxState).\n\n-spec tx_state_clients_id(tx_state()) -> any().\ntx_state_clients_id(TxState) -> element(?enum_clientid, TxState).\n\n-spec tx_state_tlog(tx_state()) -> tx_tlog:tlog().\ntx_state_tlog(TxState) -> element(?enum_tlog, TxState).\n\n-spec tx_state_txid_next_write_token(tx_state()) -> pr:pr() | '_'.\ntx_state_txid_next_write_token(TxState) ->\n    element(?enum_txid_round, TxState).\n\n-spec tx_state_set_txid_next_write_token(tx_state(), pr:pr())\n                             -> tx_state().\ntx_state_set_txid_next_write_token(TxState, NextRound) ->\n    setelement(?enum_txid_round, TxState, NextRound).\n\n-spec tx_state_txid_written_value(tx_state()) -> any().\ntx_state_txid_written_value(TxState) ->\n    element(?enum_txid_writtenval, TxState).\n\n-spec tx_state_set_txid_written_value(tx_state(), any()) -> tx_state().\ntx_state_set_txid_written_value(TxState, WrittenVal) ->\n    setelement(?enum_txid_writtenval, TxState, WrittenVal).\n\n-spec tx_state_open_locks(tx_state()) -> [client_key() | binary() | ?RT:key()].\ntx_state_open_locks(TxState) -> element(?enum_open_locks, TxState).\n\n-spec tx_state_set_open_locks(tx_state(), [client_key() | binary() | ?RT:key()])\n                             -> tx_state().\ntx_state_set_open_locks(TxState, OpenLocks) ->\n    setelement(?enum_open_locks, TxState, OpenLocks).\n\n-spec tx_state_open_commits(tx_state()) ->\n                                   [{client_key() | binary() | ?RT:key(),\n                                     pr:pr() | '_',\n                                     any()}].\ntx_state_open_commits(TxState)    -> element(?enum_open_commits, TxState).\n\n-spec tx_state_set_open_commits(tx_state(),\n                                [{client_key() | binary() | ?RT:key(),\n                                  pr:pr(),\n                                  any()}])\n                             -> tx_state().\ntx_state_set_open_commits(TxState, FailedLocks) ->\n    setelement(?enum_open_commits, TxState, FailedLocks).\n\n-spec tx_state_add_nextround_writtenval_for_commit(\n        tx_state(), client_key() | binary() | ?RT:key(),\n        pr:pr(), any()) -> tx_state().\ntx_state_add_nextround_writtenval_for_commit(TxState, Key, NextRound, WrittenVal) ->\n    OpenCommits = tx_state_open_commits(TxState),\n    NewOpenCommits = lists:keyreplace(Key, 1, OpenCommits,\n                                      {Key, NextRound, WrittenVal}),\n    tx_state_set_open_commits(TxState, NewOpenCommits).\n\n-spec tx_state_failed_locks(tx_state())\n                           -> [client_key() | binary() | ?RT:key()].\ntx_state_failed_locks(TxState)    -> element(?enum_failed_locks, TxState).\n\n-spec tx_state_set_failed_locks(tx_state(),\n                                [client_key() | binary() | ?RT:key()])\n                               -> tx_state().\ntx_state_set_failed_locks(TxState, FailedLocks) ->\n    setelement(?enum_failed_locks, TxState, FailedLocks).\n"
  },
  {
    "path": "src/tx/txid_on_cseq.erl",
    "content": "% @copyright 2012-2015 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc txid store based on rbrcseq.\n%% @end\n%% @version $Id:$\n-module(txid_on_cseq).\n-author('schintke@zib.de').\n-vsn('$Id:$').\n\n%-define(TRACE(X,Y), io:format(X,Y)).\n-define(TRACE(X,Y), ok).\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([read/2]).\n-export([new/3]).\n-export([decide/5]).\n-export([delete/2]).\n\n%% filters and checks for rbr_cseq operations consistency\n-export([is_valid_new/3]).\n-export([is_valid_decide/3]).\n-export([is_valid_delete/3]).\n\n%% write filters\n-export([wf_decide/3]).\n\n-type txid() :: ?RT:key().\n\n-type txid_entry() ::\n        { txid(),\n          comm:mypid(),         %% client\n          [client_key()],       %% involved keys\n          erlang_timestamp(),   %% creation time\n          open | commit | abort %% status\n        }.\n\n-export_type([txid/0]).\n\n-spec read(txid(), comm:erl_local_pid()) -> ok.\nread(Key, ReplyTo) ->\n    %% decide which db is responsible, ie. if the key is from\n    %% the first quarter of the ring, use lease_db1, if from 2nd\n    %% quarter -> use lease_db2, ...\n    DB = rbrcseq:get_db_for_id(tx_id, Key),\n    %% perform qread\n    rbrcseq:qread(DB, ReplyTo, Key, ?MODULE).\n\n-spec new(txid(), [client_key()], comm:erl_local_pid()) -> ok.\nnew(Key, InvolvedKeys, ReplyTo) ->\n    DB = rbrcseq:get_db_for_id(tx_id, Key),\n    Value = new_entry(Key, InvolvedKeys, comm:make_global(ReplyTo)),\n    RBRCseqPid = comm:make_global(pid_groups:find_a(DB)),\n    rbrcseq:qwrite_fast(DB, ReplyTo, Key, ?MODULE,\n                        fun txid_on_cseq:is_valid_new/3, Value,\n                        pr:smallest_round(RBRCseqPid), prbr_bottom).\n\n-spec decide(txid(), commit | abort, comm:erl_local_pid(),\n             pr:pr(), any()) -> ok.\ndecide(Key, Decision, ReplyTo, Round, OldVal) ->\n    DB = rbrcseq:get_db_for_id(tx_id, Key),\n    rbrcseq:qwrite_fast(DB, ReplyTo, Key, ?MODULE,\n                        fun txid_on_cseq:is_valid_decide/3, Decision,\n                        Round, OldVal).\n\n-spec delete(txid(), comm:erl_local_pid()) -> ok.\ndelete(Key, ReplyTo) ->\n    DB = rbrcseq:get_db_for_id(tx_id, Key),\n    rbrcseq:qwrite(DB, ReplyTo, Key, ?MODULE,\n                   fun txid_on_cseq:is_valid_delete/3, prbr_bottom).\n\n%% content checks\n-spec is_valid_new(prbr_bottom | txid_entry(),\n                   prbr:write_filter(), New::txid_entry()) ->\n                          {boolean(), null}.\nis_valid_new(prbr_bottom, _WriteFilter, _NewTxIdEntry) ->\n    {true, null};\nis_valid_new(ExistingEntry, _WriteFilter, ExistingEntry) ->\n    %% recreating same entry is ok (write through may happened)\n    {true, null};\nis_valid_new(_ExistingEntry, _WriteFilter, _DifferingNewEntry) ->\n    {false, null}.\n\n-spec is_valid_decide(prbr_bottom | txid_entry(),\n                      prbr:write_filter(), commit | abort) ->\n                               {boolean(), null}.\nis_valid_decide(prbr_bottom, _WriteFilter, _Decision) ->\n    {false, null};\nis_valid_decide(ExistingEntry, _WriteFilter, Decision) ->\n    case status(ExistingEntry) of\n        open -> {true, null};\n        %% recreating same entry is ok (write through may happened)\n        Decision -> {true, null};\n        _ ->\n            log:log(\"You try to overwrite a tx with another decision.~n\"),\n            {false, null}\n    end.\n\n-spec is_valid_delete(prbr_bottom | txid_entry(),\n                      prbr:write_filter(), New::prbr_bottom %% =:= txid_entry()\n                                                ) ->\n                               {boolean(), null}.\nis_valid_delete(prbr_bottom, _WriteFilter, prbr_bottom) ->\n    {false, null}; %% not necessary to delete deleted entry\n%%is_valid_delete(ExistingEntry, _WriteFilter, ExistingEntry) ->\n%%    %% deleting same entry is ok (write through may happened)\n%%    {false, null};\nis_valid_delete(ExistingEntry, _WriteFilter, prbr_bottom) ->\n    case status(ExistingEntry) of\n        open -> {false, null};\n        _ -> {true, null} %% comit | abort\n    end.\n\n%% write filters\n%% WF(old_dbdata(), UpdateInfo, value()) -> {dbdata(), val_passed_to_caller()}.\n-spec wf_decide(txid_entry(), null, commit | abort) -> {txid_entry(), none}.\nwf_decide(Old, null, Decision) ->\n    {set_status(Old, Decision), none}.\n\n%% abstract data type: txid_entry\n-spec new_entry(?RT:key(), [client_key()], comm:mypid()) -> txid_entry().\nnew_entry(TxId, Keys, Client) ->\n    {TxId, Client, Keys, os:timestamp(), open}.\n\n-spec status(txid_entry()) -> open | commit | abort.\nstatus(Entry) -> element(5, Entry).\n\n-spec set_status(txid_entry(), commit|abort) -> txid_entry().\nset_status(Entry, Decision) -> setelement(5, Entry, Decision).\n\n"
  },
  {
    "path": "src/uid.erl",
    "content": "%  @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Provides unique process-local and global IDs.\n%% @end\n%% @version $Id$\n-module(uid).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-export_type([global_uid/0]).\n\n-export([get_pids_uid/0, get_global_uid/0,\n         is_my_old_uid/1, is_old_uid/2,\n         from_same_pid/2]).\n\n-opaque global_uid() :: {pos_integer(), pid()}.\n\n-spec get_pids_uid() -> pos_integer().\nget_pids_uid() ->\n    Result = case erlang:get(pids_uid_counter) of\n                 undefined ->\n                     %% Same pid may be reused in the same VM, so we\n                     %% get a VM unique offset to start\n                     %% It is not completely safe, but safe enough\n                     element(1, erlang:statistics(reductions));\n                 Any -> Any + 1\n             end,\n    erlang:put(pids_uid_counter, Result),\n    Result.\n\n-spec get_global_uid() -> global_uid().\nget_global_uid() ->\n    % note: Erlang makes the local pid() globally unique by adding the node\n    %       name when transferring it\n    _Result = {get_pids_uid(), self()}\n    %% , term_to_binary(_Result, [{minor_version, 1}])\n    .\n\n%% @doc Checks whether the given GUID is an old incarnation of a GUID from\n%%      my node.\n-spec is_my_old_uid(pos_integer() | global_uid()) -> boolean() | remote.\nis_my_old_uid({LocalUid, Pid}) ->\n    case comm:this() of\n        Pid -> is_my_old_uid(LocalUid);\n        _   -> remote\n    end;\nis_my_old_uid(Id) when is_integer(Id) ->\n    LastUid = case erlang:get(pids_uid_counter) of\n                  undefined -> 0;\n                  Any -> Any\n              end,\n    Id =< LastUid;\nis_my_old_uid(_Id) ->\n    false.\n\n%% @doc Checks whether GUID1 is an old incarnation of GUID2.\n-spec is_old_uid(GUID1::global_uid(), GUID2::global_uid()) -> boolean().\nis_old_uid({LocalUid1, Pid}, {LocalUid2, Pid}) when LocalUid1 < LocalUid2 ->\n    true;\nis_old_uid(_GUID1, _GUID2) ->\n    false.\n\n%% @doc Checks whether GUID1 is from the same process as GUID2.\n-spec from_same_pid(GUID1::global_uid(), GUID2::global_uid()) -> boolean().\nfrom_same_pid({_LocalUid1, Pid}, {_LocalUid2, Pid}) ->\n    true;\nfrom_same_pid(_GUID1, _GUID2) ->\n    false.\n"
  },
  {
    "path": "src/util.erl",
    "content": "% @copyright 2007-2019 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Utility Functions.\n%% @end\n%% @version $Id$\n-module(util).\n\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n-export_type([time_utc/0, us_timestamp/0]).\n\n-export([escape_quotes/1,\n         kahan_sum/3,\n         min/2, max/2, log/2, log2/1, log1p/1, ceil/1, floor/1, pow1p/2,\n         logged_exec/1,\n         randomelem/1, randomelem_and_length/1,\n         pop_randomelem/1, pop_randomelem/2, pop_randomsubset/2,\n         get_stacktrace/0, get_linetrace/0, get_linetrace/1,\n         do_throw/1,\n         extract_from_list_may_not_exist/2,\n         minus_all/2, minus_first/2,\n         map_with_nr/3,\n         par_map/2, par_map/3,\n         lists_take/2, lists_takewith/2,\n         lists_split/2, lists_keystore2/5,\n         lists_partition3/2,\n         lists_remove_at_indices/2,\n         sublist/3, lists_index_of/2,\n         lists_check_min_length/2,\n         sleep_for_ever/0, shuffle/1, get_proc_in_vms/1,random_subset/2,\n         gb_trees_largest_smaller_than/2, gb_trees_foldl/3, pow/2,\n         zipfoldl/5, safe_split/2, '=:<'/2,\n         split_unique/2, split_unique/3, split_unique/4,\n         ssplit_unique/2, ssplit_unique/3, ssplit_unique/4,\n         smerge2/2, smerge2/3, smerge2/4, smerge2/6,\n         is_unittest/0, make_filename/1,\n         app_get_env/2,\n         app_check_running/1,\n         timestamp2us/1, us2timestamp/1,\n         time_plus_s/2, time_plus_ms/2, time_plus_us/2,\n         readable_utc_time/1,\n         for_to/3, for_to_ex/3, for_to_ex/4, for_to_fold/5,\n         collect_while/1]).\n-export([list_set_nth/3]).\n-export([debug_info/0, debug_info/1]).\n-export([print_bits/2, bin_xor/2, bin_or/2, bin_and/2]).\n-export([if_verbose/1, if_verbose/2]).\n-export([tc/3, tc/2, tc/1]).\n-export([wait_for/1, wait_for/2,\n         wait_for_process_to_die/1,\n         wait_for_ets_table_to_disappear/2,\n         ets_tables_of/1]).\n-export([round/2]).\n\n-export([repeat/3, repeat/4, parallel_run/5]).\n\n-export([empty/1]).\n\n-export([extint2atom/1]).\n\n% RRD helpers which don't belong to the rrd datastructure\n-export([ rrd_combine_timing_slots/3\n         , rrd_combine_timing_slots/4\n         , rrd_combine_gauge_slots/3\n         , rrd_combine_gauge_slots/4\n         , rrd_combine_slots/6\n    ]).\n\n% feeder for tester\n-export([log_feeder/2, log2_feeder/1, log1p_feeder/1, pow1p_feeder/2]).\n-export([readable_utc_time_feeder/1]).\n-export([map_with_nr_feeder/3]).\n-export([par_map_feeder/2, par_map_feeder/3]).\n-export([lists_takewith_feeder/2]).\n-export([lists_partition3_feeder/2]).\n\n-export([sets_map/2, sets_equal/2]).\n\n-type us_timestamp() :: non_neg_integer(). % micro seconds since Epoch\n\n-type time_utc() :: {{1970..10000, 1..12, 1..31}, {0..23, 0..59, 0..59}}.\n\n-type args() :: [term()].\n-type accumulatorFun(T, U) :: fun((T, U) -> U).\n-type repeat_params() :: parallel |\n                         collect |\n                         {accumulate, accumulatorFun(any(), R), R}. %{accumulate, fun, accumulator init value}\n\n-dialyzer([{[no_opaque, no_contracts], gb_trees_largest_smaller_than/2},\n           {no_return, lists_takewith_iter_feeder/3}]).\n\n%% @doc Executes wait_for/2 with a WaitTime of 10ms.\n-spec wait_for(fun(() -> boolean())) -> ok.\nwait_for(F) -> wait_for(F, 10).\n\n%% @doc Waits for F/0 to become true and checks every WaitTime Milliseconds.\n%%      Uses (send_local_after/2 and receive) or timer:sleep/1 to wait to\n%%      return control flow to e.g. proto_sched depending on whether the caller\n%%      is a gen_component or not.\n-spec wait_for(fun(() -> boolean()), WaitTimeInMs::pos_integer()) -> ok.\nwait_for(F, WaitTime) ->\n    case gen_component:is_gen_component(self()) of\n        true  -> wait_for2(F, WaitTime);\n        false -> wait_for1(F, WaitTime)\n    end.\n\n%% @doc Waits for F/0 to become true and checks every WaitTime Milliseconds.\n%%      Uses send_local_after/2 and receive to wait to return control flow to\n%%       e.g. proto_sched and is thus _NOT_ suitable for gen_components.\n-spec wait_for1(fun(() -> boolean()), WaitTimeInMs::pos_integer()) -> ok.\nwait_for1(F, WaitTime) ->\n    case F() of\n        true  -> ok;\n        false ->\n            WaitID = uid:get_pids_uid(),\n            comm:send_local_after(WaitTime, self(), {continue_wait, WaitID}),\n            trace_mpath:thread_yield(),\n            receive\n                ?SCALARIS_RECV({continue_wait, WaitID},% ->\n                               wait_for1(F, WaitTime))\n            end\n    end.\n\n%% @doc Waits for F/0 to become true and checks every WaitTime Milliseconds.\n%%      Uses timer:sleep/1 to wait and is thus suitable for gen_components.\n-spec wait_for2(fun(() -> boolean()), WaitTimeInMs::pos_integer()) -> ok.\nwait_for2(F, WaitTime) ->\n    case F() of\n        true  -> ok;\n        false -> timer:sleep(WaitTime),\n                 wait_for2(F, WaitTime)\n    end.\n\n%% @doc Waits for the given process (name or pid) to die.\n-spec wait_for_process_to_die(pid() | atom()) -> ok.\nwait_for_process_to_die(Name) when is_atom(Name) ->\n    wait_for(fun() ->\n                     case erlang:whereis(Name) of\n                         undefined -> true;\n                         Pid       -> not is_process_alive(Pid)\n                     end\n             end);\nwait_for_process_to_die(Pid) when is_pid(Pid) ->\n    wait_for(fun() -> not is_process_alive(Pid) end).\n\n%% @doc Waits for the given ets table to disappear.\n-spec wait_for_ets_table_to_disappear(Pid::pid(), ets:tid() | atom()) -> ok.\nwait_for_ets_table_to_disappear(Pid, Table) ->\n    wait_for(fun() ->\n                     case ets:info(Table, owner) of\n                         undefined -> true;\n                         Pid -> false;\n                         _ -> true\n                     end\n             end).\n\n-spec ets_tables_of(pid()) -> [ets:tid() | atom()].\nets_tables_of(Pid) ->\n    Tabs = ets:all(),\n    [ Tab || Tab <- Tabs, ets:info(Tab, owner) =:= Pid ].\n\n%% @doc Escapes quotes in the given string.\n-spec escape_quotes(String::string()) -> string().\nescape_quotes(String) ->\n    lists:foldr(fun escape_quotes_/2, [], String).\n\n%-spec escape_quotes_(String::string(), Rest::string()) -> string().\n-spec escape_quotes_(char(), string()) -> string().\nescape_quotes_($\", Rest) -> [$\\\\, $\" | Rest];\nescape_quotes_(Ch, Rest) -> [Ch | Rest].\n\n%% @doc Variant of erlang:max/2 also taking ?PLUS_INFINITY_TYPE and\n%%      ?MINUS_INFINITY_TYPE into account, e.g. for comparing keys.\n%% @end\n%%-spec max(?PLUS_INFINITY_TYPE, any()) -> ?PLUS_INFINITY_TYPE;\n%%         (any(), ?PLUS_INFINITY_TYPE) -> ?PLUS_INFINITY_TYPE;\n%%         (T | ?MINUS_INFINITY_TYPE, T | ?MINUS_INFINITY_TYPE) -> T.\n-spec max(any(), any()) -> any().\nmax(?PLUS_INFINITY, _) -> ?PLUS_INFINITY;\nmax(_, ?PLUS_INFINITY) -> ?PLUS_INFINITY;\nmax(?MINUS_INFINITY, X) -> X;\nmax(X, ?MINUS_INFINITY) -> X;\nmax(A, B) when A > B -> A;\nmax(_A, B) -> B.\n\n%% @doc Variant of erlang:min/2 also taking ?PLUS_INFINITY_TYPE and\n%%      ?MINUS_INFINITY_TYPE into account, e.g. for comparing keys.\n%% @end\n%%-spec min(?MINUS_INFINITY_TYPE, any()) -> ?MINUS_INFINITY_TYPE;\n%%         (any(), ?MINUS_INFINITY_TYPE) -> ?MINUS_INFINITY_TYPE;\n%%         (T | ?PLUS_INFINITY_TYPE, T | ?PLUS_INFINITY_TYPE) -> T.\n-spec min(any(), any()) -> any().\nmin(?MINUS_INFINITY, _) -> ?MINUS_INFINITY;\nmin(_, ?MINUS_INFINITY) -> ?MINUS_INFINITY;\nmin(?PLUS_INFINITY, X) -> X;\nmin(X, ?PLUS_INFINITY) -> X;\nmin(A, B) when A < B -> A;\nmin(_A, B) -> B.\n\n%% @doc Implements the Kahan summation algorithm for a low numerical error\n%%      when adding a sequence of finite precision floating point numbers.\n%%      (ref. [https://en.wikipedia.org/wiki/Kahan_summation_algorithm])\n-spec kahan_sum([number()], AccSum::float(), AccCompensation::float()) ->\n          {Sum::float(), Compensation::float()}.\nkahan_sum([], AccSum, AccCompensation) ->\n    {AccSum, AccCompensation};\nkahan_sum([X | Rest], Sum, C) ->\n    Y = X - C,\n    T = Sum + Y,\n    kahan_sum(Rest, T, (T - Sum) - Y).\n\n-spec pow(integer(), non_neg_integer()) -> integer();\n         (float(), non_neg_integer()) -> number().\npow(_X, 0) ->\n    1;\npow(X, 1) ->\n    X;\npow(X, 2) ->\n    X * X;\npow(X, 3) ->\n    X * X * X;\npow(X, Y) when (Y rem 2) =:= 0 ->\n    Half = pow(X, Y div 2),\n    Half * Half;\npow(X, Y) ->\n    Half = pow(X, Y div 2),\n    Half * Half * X.\n\n-spec log_feeder(X::number(), Base::number()) -> {number(), number()}.\nlog_feeder(X0, B0) ->\n    X = case X0 of\n            0   -> 1;   % not allowed\n            0.0 -> 1.0; % not allowed\n            X1  -> erlang:abs(X1)\n        end,\n    B = case erlang:abs(B0) of\n           1    -> 2;   % 1 not allowed\n           1.0  -> 2.0; % 1 not allowed\n           0    -> 2;   % 0 not allowed\n           0.0  -> 2.0; % 0 not allowed\n           B1   -> B1\n        end,\n    {X, B}.\n\n%% @doc Logarithm of X to the base of Base.\n-spec log(X::number(), Base::number()) -> float().\nlog(X, B) -> math:log10(X) / math:log10(B).\n\n-spec log2_feeder(X::number()) -> {number()}.\nlog2_feeder(0)   -> {1}; % 0 not allowed\nlog2_feeder(0.0) -> {1.0}; % 0 not allowed\nlog2_feeder(X) when X < 0 -> {-X};\nlog2_feeder(X)   -> {X}.\n\n%% @doc Logarithm of X to the base of 2.\n-spec log2(X::number()) -> float().\nlog2(X) -> math:log10(X) / 0.3010299956639812. % use hard-coded math:log10(2)\n\n-spec log1p_feeder(X::number()) -> {number()}.\nlog1p_feeder(-1)   -> {0}; % -1 not allowed\nlog1p_feeder(-1.0) -> {0.0}; % -1 not allowed\nlog1p_feeder(X) when X < -1 -> {-X};\nlog1p_feeder(X)    -> {X}.\n\n%% @doc More precise version of ln(1+x) for small x.\n%%\n%% from: David Goldberg. 1991. What every computer scientist should know\n%%       about floating-point arithmetic. ACM Comput. Surv. 23, 1\n%%       (March 1991), 5-48. DOI=<a href=\"http://dx.doi.org/10.1145/103162.103163\">10.1145/103162.103163</a>\n-spec log1p(X::number()) -> float().\nlog1p(X) ->\n    W = 1 + X,\n    if W == 1 -> float(X);\n       true   -> X * math:log(W) / (W-1)\n    end.\n\n-spec pow1p_feeder(X0::float(), Y0::float()) -> {X::float(), Y::float()}.\npow1p_feeder(X0, Y0) ->\n    X = if X0 < 0 -> -X0;\n           true   -> X0\n        end,\n    {X, Y0}.\n\n%% @doc Calculates (1 - X^Y) more exactly, especially for X^Y near 1\n%%      (only really usefull for 0 &lt; X &lt; 1 - for the rest, use math:pow/2).\n%%      Uses the series representation of 1 - X^Y\n%%      1-X^Y = sum_(i=1)^infinity (- Y^i * log^i(X) / (i!))\n%% from: <a href=\"http://www.wolframalpha.com/input/?i=series+x^y\">Wolfram Alpha for x^y</a>\n-spec pow1p(X::float(), Y::float()) -> float().\npow1p(X, Y) when X >= 0 andalso Y == 0 ->\n    0.0;\npow1p(X, Y) when X == 0 andalso Y /= 0 ->\n    1.0;\npow1p(X, Y) when X > 0 andalso Y /= 0 ->\n    % the difference between the terms for i and (i+1) is ln(X)/((i+1)*N)\n    YxLnX = Y * math:log(X),\n    pow1p_(-YxLnX, YxLnX, 2, -YxLnX).\n\n%% @doc Helper for pow1p/2.\n-spec pow1p_(Prev::float(), YxLnX::float(), I::pos_integer(), PrevSum::float())\n        -> float().\npow1p_(Prev, YxLnX, CurI, Sum) ->\n    Cur = Prev * YxLnX / CurI,\n    NextSum = Sum + Cur,\n    if NextSum == Sum ->\n           % log:pal(\"end at ~B\", [CurI]),\n           Sum;\n       true ->\n           % log:pal(\"cur ~p ~p\", [Sum, NextSum]),\n           pow1p_(Cur, YxLnX, CurI + 1, NextSum)\n    end.\n\n%% @doc Returns the largest integer not larger than X.\n-spec floor(X::number()) -> integer().\nfloor(X) ->\n    T = erlang:trunc(X),\n    if X < T -> T - 1;\n       true  -> T\n    end.\n\n%% @doc Returns the smallest integer not smaller than X.\n-spec ceil(X::number()) -> integer().\nceil(X) ->\n    T = erlang:trunc(X),\n    if X > T -> T + 1;\n       true  -> T\n    end.\n\n-spec logged_exec(Cmd::string() | atom()) -> ok.\nlogged_exec(Cmd) ->\n    Output = os:cmd(Cmd),\n    OutputLength = length(Output),\n    if\n        OutputLength > 10 ->\n            log:log(info, \"exec\", Cmd),\n            log:log(info, \"exec\", Output),\n            ok;\n        true ->\n            ok\n    end.\n\n-ifdef(have_new_stacktrace).\n-spec get_stacktrace() -> [{Module::atom(), Function::atom(), ArityOrArgs::byte() | [term()]} |\n                           {Module::atom(), Function::atom(), ArityOrArgs::byte() | [term()], Sources::[term()]}].\nget_stacktrace() ->\n    try\n        erlang:exit(a)\n    catch\n        _Class:_ExceptionPattern:Stk ->\n            Stk\n    end.\n-else.\n%% @doc Gets the current stack trace. Use this method in order to get a stack\n%%      trace if no exception was thrown.\n-spec get_stacktrace() -> [{Module::atom(), Function::atom(), ArityOrArgs::byte() | [term()]} |\n                           {Module::atom(), Function::atom(), ArityOrArgs::byte() | [term()], Sources::[term()]}].\nget_stacktrace() ->\n    % throw an exception for erlang:get_stacktrace/0 to return the actual stack trace\n    case (try erlang:exit(a)\n          catch exit:_ -> erlang:get_stacktrace()\n          end) of\n        % erlang < R15 : {util, get_stacktrace, 0}\n        % erlang >= R15: {util, get_stacktrace, 0, _}\n        %% drop head element as it was generated just above\n        [T | ST] when erlang:element(1, T) =:= util andalso\n                          erlang:element(2, T) =:= get_stacktrace andalso\n                          erlang:element(3, T) =:= 0 -> ok;\n        ST -> ST % just in case\n    end,\n    ST.\n-endif.\n\n-spec get_linetrace() -> term() | undefined.\nget_linetrace() ->\n    erlang:get(test_server_loc).\n\n-spec get_linetrace(Pid::pid()) -> term() | undefined.\nget_linetrace(Pid) ->\n    {dictionary, Dict} = erlang:process_info(Pid, dictionary),\n    extract_from_list_may_not_exist(Dict, test_server_loc).\n\n-spec do_throw(term()) -> no_return().\ndo_throw(Exception) ->\n    log:log(\"Exception ~p at ~.0p\", [Exception, get_stacktrace()]),\n    erlang:throw(Exception).\n\n%% @doc Extracts a given ItemInfo from an ItemList or returns 'undefined' if\n%%      there is no such item.\n-spec extract_from_list_may_not_exist\n        ([{Item::term(), Info::any()}], ItemInfo::term()) -> Info::any() | undefined.\nextract_from_list_may_not_exist(List, Key) ->\n    case lists:keyfind(Key, 1, List) of\n        false -> undefined;\n        X     -> element(2, X)\n    end.\n\n%% @doc minus_all(M,N) : { x | x in M and x notin N}\n-spec minus_all(List::[T], Excluded::[T]) -> [T]\n    when is_subtype(T, any()).\nminus_all([_|_] = L, [Excluded]) ->\n    [E || E <- L, E =/= Excluded];\nminus_all([_|_] = L, ExcludeList) ->\n    ExcludeSet = sets:from_list(ExcludeList),\n    [E || E <- L, not sets:is_element(E, ExcludeSet)];\nminus_all([], _ExcludeList) ->\n    [].\n\n%% @doc Deletes the first occurrence of each element in Excluded from List.\n%%      Similar to lists:foldl(fun lists:delete/2, NewValue1, ToDel) but more\n%%      performant for out case.\n-spec minus_first(List::[T], Excluded::[T]) -> [T]\n    when is_subtype(T, any()).\nminus_first([_|_] = L, [Excluded]) ->\n    lists:delete(Excluded, L);\nminus_first([_|_] = L, ExcludeList) ->\n    minus_first2(L, ExcludeList);\nminus_first([], _ExcludeList) ->\n    [].\n\n%% @doc Removes every item in Excluded only once from List.\n-spec minus_first2(List::[T], Excluded::[T]) -> [T]\n    when is_subtype(T, any()).\nminus_first2([H | T], [_|_] = Excluded) ->\n    case lists_take(H, Excluded) of\n        false     -> [H | minus_first2(T, Excluded)];\n        Excluded2 -> minus_first2(T, Excluded2)\n    end;\nminus_first2([], _Excluded) ->\n    [];\nminus_first2(L, []) ->\n    L.\n\n-spec get_proc_in_vms(atom()) -> [comm:mypid()].\nget_proc_in_vms(Proc) ->\n    mgmt_server:node_list(),\n    Nodes =\n        begin\n            trace_mpath:thread_yield(),\n            receive\n                ?SCALARIS_RECV({get_list_response, X}, X)\n        after 2000 ->\n            log:log(error,\"[ util ] Timeout getting node list from mgmt server\"),\n            throw('mgmt_server_timeout')\n        end end,\n    lists:usort([comm:get(Proc, DHTNode) || DHTNode <- Nodes]).\n\n-spec sleep_for_ever() -> no_return().\nsleep_for_ever() ->\n    timer:sleep(5000),\n    sleep_for_ever().\n\n%% @doc Returns a random element from the given (non-empty!) list according to\n%%      a uniform distribution.\n-spec randomelem(List::[X,...]) -> X\n    when is_subtype(X, any()).\nrandomelem([X]) -> X;\nrandomelem(List) ->\n    element(1, randomelem_and_length(List)).\n\n%% @doc Returns a random element from the given (non-empty!) list according to\n%%      a uniform distribution (also returns the list's length).\n-spec randomelem_and_length(List::[X,...]) -> {X, Length::pos_integer()}\n    when is_subtype(X, any()).\nrandomelem_and_length([X]) -> {X, 1};\nrandomelem_and_length(List) ->\n    Length = length(List) + 1,\n    RandomNum = randoms:rand_uniform(1, Length),\n    {lists:nth(RandomNum, List), Length - 1}.\n\n%% @doc Removes a random element from the (non-empty!) list and returns the\n%%      resulting list and the removed element.\n-spec pop_randomelem(List::[X,...]) -> {NewList::[X], PoppedElement::X}\n    when is_subtype(X, any()).\npop_randomelem([X]) -> {[], X};\npop_randomelem(List) ->\n    pop_randomelem(List, length(List)).\n\n%% @doc Removes a random element from the first Size elements of a (non-empty!)\n%%      list and returns the resulting list and the removed element.\n%%      If Size is 0, the first element will be popped.\n%%      Size must not exceed the length of the list!\n-spec pop_randomelem(List::[X,...], Size::non_neg_integer())\n        -> {NewList::[X], PoppedElement::X}\n    when is_subtype(X, any()).\npop_randomelem([X | TL], 0) -> {TL, X};\npop_randomelem([X | TL], 1) -> {TL, X};\npop_randomelem(List, Size) ->\n    {Leading, [H | T]} = lists:split(randoms:rand_uniform(0, Size), List),\n    {lists:append(Leading, T), H}.\n\n%% @doc Removes a random subset of Size elements from the given list and returns\n%%      the resulting list and the removed subset.\n%%      If Size is larger than the list length, all elements will be returned\n%%      (in random order).\n-spec pop_randomsubset(Size::pos_integer(), [X])\n        -> {NewList::[X], PoppedSet::[X]}\n    when is_subtype(X, any()).\npop_randomsubset(0, List) -> {List, []};\npop_randomsubset(_Size, []) -> {[], []};\npop_randomsubset(_Size, [X]) -> {[], [X]};\npop_randomsubset(Size0, List) ->\n    A = array:fix(array:from_list(List)),\n    Size = erlang:min(Size0, array:size(A)),\n    LNew = array:to_list(shuffle_helperA(A, Size, 0)),\n    {PoppedSet, Rest} = lists:split(Size, LNew),\n    {Rest, PoppedSet}.\n\n%% @doc Returns a random subset of Size elements from the given list.\n-spec random_subset(Size::pos_integer(), [T]) -> [T]\n    when is_subtype(T, any()).\nrandom_subset(0, _List) -> [];\nrandom_subset(_Size, []) -> [];\nrandom_subset(_Size, [X]) -> [X];\nrandom_subset(Size0, List) ->\n    A = array:fix(array:from_list(List)),\n    Size = erlang:min(Size0, array:size(A)),\n    A2 = shuffle_helperA(A, Size, 0),\n    array:to_list(array:resize(Size, A2)).\n\n%% @doc Fisher-Yates shuffling for lists.\n-spec shuffle([T]) -> [T]\n    when is_subtype(T, any()).\nshuffle([]) -> [];\nshuffle([X]) -> [X];\nshuffle(List) ->\n    A = array:fix(array:from_list(List)),\n    A2 = shuffle_helperA(A, array:size(A), 0),\n    array:to_list(A2).\n\n%% @doc Fisher-Yates shuffling for lists helper function: creates a shuffled\n%%      list of length ShuffleSize.\n%%      PreCond: a non-empty array, ShuffleSize &lt; size(Array)\n-spec shuffle_helperA(array:array(T), ShuffleSize::non_neg_integer(),\n                      StartPos::non_neg_integer()) -> array:array(T)\n    when is_subtype(T, any()).\nshuffle_helperA(Array, CurPos, CurPos) ->\n    Array;\nshuffle_helperA(Array, ShuffleSize, N1) ->\n    E1 = array:get(N1, Array),\n    N2 = randoms:rand_uniform(N1, array:size(Array)),\n    E2 = array:get(N2, Array),\n    A2 = array:set(N1, E2, array:set(N2, E1, Array)),\n    shuffle_helperA(A2, ShuffleSize, N1 + 1).\n\n%% @doc Find the largest key in GBTree that is smaller than Key.\n%%      Note: gb_trees offers only linear traversal or lookup of exact keys -\n%%      we implement a more flexible binary search here despite gb_tree being\n%%      defined as opaque.\n-spec gb_trees_largest_smaller_than(Key, gb_trees:tree(Key, Value))\n        -> {value, Key, Value} | nil\n    when is_subtype(Key, any()),\n         is_subtype(Value, any()).\ngb_trees_largest_smaller_than(_Key, {0, _Tree}) ->\n    nil;\ngb_trees_largest_smaller_than(MyKey, {_Size, InnerTree}) ->\n    gb_trees_largest_smaller_than_iter(MyKey, InnerTree, true).\n\n-spec gb_trees_largest_smaller_than_iter(Key, {Key, Value, Smaller::term(), Bigger::term()}, RightTree::boolean()) -> {value, Key, Value} | nil\n    when is_subtype(Key, any()),\n         is_subtype(Value, any()).\ngb_trees_largest_smaller_than_iter(SearchKey, {Key, Value, _Smaller, nil},\n                                   true) when Key < SearchKey ->\n    % we reached the right end of the whole tree\n    % -> there is no larger item than the current item\n    {value, Key, Value};\ngb_trees_largest_smaller_than_iter(SearchKey, {Key, Value, _Smaller, Bigger},\n                                   RightTree) when Key < SearchKey ->\n    case gb_trees_largest_smaller_than_iter(SearchKey, Bigger, RightTree) of\n        {value, _, _} = AValue -> AValue;\n        nil -> {value, Key, Value}\n    end;\ngb_trees_largest_smaller_than_iter(SearchKey, {_Key, _Value, Smaller, _Bigger},\n                                   _RightTree) ->\n  gb_trees_largest_smaller_than_iter(SearchKey, Smaller, false);\ngb_trees_largest_smaller_than_iter(_SearchKey, nil, _RightTree) ->\n    nil.\n\n%% @doc Foldl over gb_trees.\n-spec gb_trees_foldl(fun((Key, Value, Acc) -> Acc), Acc, gb_trees:tree(Key, Value)) -> Acc\n    when is_subtype(Acc, any()),\n         is_subtype(Key, any()),\n         is_subtype(Value, any()).\ngb_trees_foldl(F, Acc, GBTree) ->\n    gb_trees_foldl_iter(F, Acc, gb_trees:next(gb_trees:iterator(GBTree))).\n\n-spec gb_trees_foldl_iter(fun((Key, Value, Acc) -> Acc), Acc,\n                          {Key, Value, Iter::gb_trees:iter(Key, Value)} | none) -> Acc.\ngb_trees_foldl_iter(_F, Acc, none) ->\n    Acc;\ngb_trees_foldl_iter(F, Acc, {Key, Val, Iter}) ->\n    gb_trees_foldl_iter(F, F(Key, Val, Acc), gb_trees:next(Iter)).\n\n%% @doc Measures the execution time (in microseconds) for an MFA\n%%      (does not catch exceptions as timer:tc/3 in older Erlang versions).\n-spec tc(module(), atom(), list()) -> {integer(), term()}.\ntc(M, F, A) ->\n    Before = os:timestamp(),\n    Val = apply(M, F, A),\n    After = os:timestamp(),\n    {timer:now_diff(After, Before), Val}.\n\n%% @doc Measures the execution time (in microseconds) for Fun(Args)\n%%      (does not catch exceptions as timer:tc/3 in older Erlang versions).\n-spec tc(Fun::fun(), Args::list()) -> {integer(), term()}.\ntc(Fun, Args) ->\n    Before = os:timestamp(),\n    Val = apply(Fun, Args),\n    After = os:timestamp(),\n    {timer:now_diff(After, Before), Val}.\n\n%% @doc Measures the execution time (in microseconds) for Fun()\n%%      (does not catch exceptions as timer:tc/3 in older Erlang versions).\n-spec tc(Fun::fun()) -> {integer(), term()}.\ntc(Fun) ->\n    Before = os:timestamp(),\n    Val = Fun(),\n    After = os:timestamp(),\n    {timer:now_diff(After, Before), Val}.\n\n-spec zipfoldl(ZipFun::fun((X, Y) -> Z), FoldFun::fun((Z, Acc) -> Acc), L1::[X], L2::[Y], Acc) -> Acc.\nzipfoldl(ZipFun, FoldFun, [L1H | L1R], [L2H | L2R], AccIn) ->\n    zipfoldl(ZipFun, FoldFun, L1R, L2R, FoldFun(ZipFun(L1H, L2H), AccIn));\nzipfoldl(_ZipFun, _FoldFun, [], [], AccIn) ->\n    AccIn.\n\n%% @doc Sorts like erlang:'=&lt;'/2 but also defines the order of integers/floats\n%%      representing the same value.\n-spec '=:<'(T, T) -> boolean()\n                     when is_subtype(T, any()).\n'=:<'(T1, T2) ->\n    case (T1 == T2) andalso (T1 =/= T2) of\n        true when erlang:is_number(T1) andalso erlang:is_number(T2) ->\n            erlang:is_integer(T1);\n        true when erlang:is_tuple(T1) andalso erlang:is_tuple(T2) ->\n            '=:<'(erlang:tuple_to_list(T1), erlang:tuple_to_list(T2));\n        true when erlang:is_list(T1) andalso erlang:is_list(T2) ->\n            % recursively check '=<'\n            '=:<_lists'(T1, T2);\n        _ -> erlang:'=<'(T1, T2)\n    end.\n\n%% @doc Compare two lists which are equal based on erlang:'=='/2.\n-spec '=:<_lists'(T::list(), T::list()) -> boolean().\n'=:<_lists'([H1 | R1], [H2 | R2]) ->\n    case (H1 == H2) andalso (H1 =/= H2) of\n        true  -> '=:<'(H1, H2);\n        false -> '=:<_lists'(R1, R2)\n    end;\n'=:<_lists'([], []) -> true.\n\n%% @doc Splits off N elements from List. If List is not large enough, the whole\n%%      list is returned.\n-spec safe_split(non_neg_integer(), [T]) -> {FirstN::[T], Rest::[T]}\n    when is_subtype(T, any()).\nsafe_split(N, List) when is_integer(N), N >= 0, is_list(List) ->\n    safe_split(N, List, []).\n\n-spec safe_split(non_neg_integer(), [T], [T]) -> {FirstN::[T], Rest::[T]}\n    when is_subtype(T, any()).\nsafe_split(0, L, R) ->\n    {lists:reverse(R, []), L};\nsafe_split(N, [H | T], R) ->\n    safe_split(N - 1, T, [H | R]);\nsafe_split(_N, [], R) ->\n    {lists:reverse(R, []), []}.\n\n%% @doc Splits L1 into a list of elements that are not contained in L2, a list\n%%      of elements that both lists share and a list of elements unique to L2.\n%%      Returned lists are sorted and contain no duplicates.\n-spec split_unique(L1::[X], L2::[X]) -> {UniqueL1::[X], Shared::[X], UniqueL2::[X]}\n    when is_subtype(X, any()).\nsplit_unique(L1, L2) ->\n    split_unique(L1, L2, fun erlang:'=<'/2).\n\n%% @doc Splits L1 into a list of elements that are not contained in L2, a list\n%%      of elements that are equal in both lists (according to the ordering\n%%      function Lte) and a list of elements unique to L2.\n%%      When two elements compare equal, the element from List1 is picked.\n%%      Lte(A, B) should return true if A compares less than or equal to B in\n%%      the ordering, false otherwise.\n%%      Returned lists are sorted according to Lte and contain no duplicates.\n-spec split_unique(L1::[X], L2::[X], Lte::fun((X, X) -> boolean())) -> {UniqueL1::[X], Shared::[X], UniqueL2::[X]}\n    when is_subtype(X, any()).\nsplit_unique(L1, L2, Lte) ->\n    split_unique(L1, L2, Lte, fun(E1, _E2) -> E1 end).\n\n%% @doc Splits L1 into a list of elements that are not contained in L2, a list\n%%      of elements that are equal in both lists (according to the ordering\n%%      function Lte) and a list of elements unique to L2.\n%%      When two elements compare equal, EqSelect(element(L1), element(L2))\n%%      chooses which of them to take.\n%%      Lte(A, B) should return true if A compares less than or equal to B in\n%%      the ordering, false otherwise.\n%%      Returned lists are sorted according to Lte and contain no duplicates.\n-spec split_unique(L1::[X], L2::[X], Lte::fun((X, X) -> boolean()), EqSelect::fun((X, X) -> X)) -> {UniqueL1::[X], Shared::[X], UniqueL2::[X]}\n    when is_subtype(X, any()).\nsplit_unique(L1, L2, Lte, EqSelect) ->\n    L1Sorted = lists:usort(Lte, L1),\n    L2Sorted = lists:usort(Lte, L2),\n    ssplit_unique_helper(L1Sorted, L2Sorted, Lte, EqSelect, {[], [], []}).\n\n%% @doc Splits L1 into a list of elements that are not contained in L2, a list\n%%      of elements that both lists share and a list of elements unique to L2.\n%%      Both lists must be sorted. Returned lists are sorted as well.\n-spec ssplit_unique(L1::[X], L2::[X]) -> {UniqueL1::[X], Shared::[X], UniqueL2::[X]}\n    when is_subtype(X, any()).\nssplit_unique(L1, L2) ->\n    ssplit_unique(L1, L2, fun erlang:'=<'/2).\n\n%% @doc Splits L1 into a list of elements that are not contained in L2, a list\n%%      of elements that are equal in both lists (according to the ordering\n%%      function Lte) and a list of elements unique to L2.\n%%      When two elements compare equal, the element from List1 is picked.\n%%      Both lists must be sorted according to Lte. Lte(A, B) should return\n%%      true if A compares less than or equal to B in the ordering, false\n%%      otherwise.\n%%      Returned lists are sorted according to Lte.\n-spec ssplit_unique(L1::[X], L2::[X], Lte::fun((X, X) -> boolean())) -> {UniqueL1::[X], Shared::[X], UniqueL2::[X]}\n    when is_subtype(X, any()).\nssplit_unique(L1, L2, Lte) ->\n    ssplit_unique(L1, L2, Lte, fun(E1, _E2) -> E1 end).\n\n%% @doc Splits L1 into a list of elements that are not contained in L2, a list\n%%      of elements that are equal in both lists (according to the ordering\n%%      function Lte) and a list of elements unique to L2.\n%%      When two elements compare equal, EqSelect(element(L1), element(L2))\n%%      chooses which of them to take.\n%%      Both lists must be sorted according to Lte. Lte(A, B) should return true\n%%      if A compares less than or equal to B in the ordering, false otherwise.\n%%      Returned lists are sorted according to Lte.\n-spec ssplit_unique(L1::[X], L2::[X], Lte::fun((X, X) -> boolean()), EqSelect::fun((X, X) -> X)) -> {UniqueL1::[X], Shared::[X], UniqueL2::[X]}\n    when is_subtype(X, any()).\nssplit_unique(L1, L2, Lte, EqSelect) ->\n    ssplit_unique_helper(L1, L2, Lte, EqSelect, {[], [], []}).\n\n%% @doc Helper function for ssplit_unique/4.\n-spec ssplit_unique_helper(L1::[X], L2::[X], Lte::fun((X, X) -> boolean()), EqSelect::fun((X, X) -> X), {UniqueOldL1::[X], SharedOld::[X], UniqueOldL2::[X]}) -> {UniqueL1::[X], Shared::[X], UniqueL2::[X]}\n    when is_subtype(X, any()).\nssplit_unique_helper(L1 = [H1 | T1], L2 = [H2 | T2], Lte, EqSelect, {UniqueL1, Shared, UniqueL2}) ->\n    LteH1H2 = Lte(H1, H2),\n    LteH2H1 = Lte(H2, H1),\n    case LteH1H2 andalso LteH2H1 of\n        true ->\n            ssplit_unique_helper(T1, L2, Lte, EqSelect, {UniqueL1, [EqSelect(H1, H2) | Shared], UniqueL2});\n        false when LteH1H2 ->\n            ssplit_unique_helper(T1, L2, Lte, EqSelect, {[H1 | UniqueL1], Shared, UniqueL2});\n        false when LteH2H1 ->\n            % the top of the shared list could be the same as the top of L2!\n            case (Shared =:= []) orelse not (Lte(hd(Shared), H2) andalso Lte(H2, hd(Shared))) of\n                true  -> ssplit_unique_helper(L1, T2, Lte, EqSelect, {UniqueL1, Shared, [H2 | UniqueL2]});\n                false -> ssplit_unique_helper(L1, T2, Lte, EqSelect, {UniqueL1, Shared, UniqueL2})\n            end\n    end;\nssplit_unique_helper(L1, [], _Lte, _EqSelect, {UniqueL1, Shared, UniqueL2}) ->\n    {lists:reverse(UniqueL1, L1), lists:reverse(Shared), lists:reverse(UniqueL2)};\nssplit_unique_helper([], L2 = [H2 | T2], Lte, EqSelect, {UniqueL1, Shared, UniqueL2}) ->\n    % the top of the shared list could be the same as the top of L2 since\n    % elements are only removed from L2 if an element of L1 is larger\n    case Shared =:= [] orelse not (Lte(hd(Shared), H2) andalso Lte(H2, hd(Shared))) of\n        true  ->\n            {lists:reverse(UniqueL1), lists:reverse(Shared), lists:reverse(UniqueL2, L2)};\n        false ->\n            ssplit_unique_helper([], T2, Lte, EqSelect, {UniqueL1, Shared, UniqueL2})\n    end.\n\n%% @doc Merges two unique sorted lists into a single list.\n-spec smerge2(L1::[X], L2::[X]) -> MergedList::[X]\n    when is_subtype(X, any()).\nsmerge2(L1, L2) ->\n    smerge2(L1, L2, fun erlang:'=<'/2).\n\n%% @doc Merges two unique Lte-sorted lists into a single list.\n-spec smerge2(L1::[X], L2::[X], Lte::fun((X, X) -> boolean())) -> MergedList::[X]\n    when is_subtype(X, any()).\nsmerge2(L1, L2, Lte) ->\n    smerge2(L1, L2, Lte, fun(E1, _E2) -> [E1] end).\n\n%% @doc Merges two unique Lte-sorted lists into a single list.\n-spec smerge2(L1::[X], L2::[X], Lte::fun((X, X) -> boolean()), EqSelect::fun((X, X) -> [X])) -> MergedList::[X]\n    when is_subtype(X, any()).\nsmerge2(L1, L2, Lte, EqSelect) ->\n    smerge2(L1, L2, Lte, EqSelect, fun(X) -> [X] end, fun(X) -> [X] end).\n\n%% @doc Merges two unique Lte-sorted lists into a single list.\n-spec smerge2(L1::[X], L2::[X], Lte::fun((X, X) -> boolean()), EqSelect::fun((X, X) -> [X]),\n              FirstExist::fun((X) -> [X]), SecondExist::fun((X) -> [X])) -> MergedList::[X]\n    when is_subtype(X, any()).\nsmerge2(L1, L2, Lte, EqSelect, FirstExist, SecondExist) ->\n    smerge2_helper(L1, L2, Lte, EqSelect, FirstExist, SecondExist).\n\n%% @doc Helper function for merge2/4.\n-spec smerge2_helper(L1::[X], L2::[X], Lte::fun((X, X) -> boolean()),\n        EqSelect::fun((X, X) -> [X]), FirstExist::fun((X) -> [X]),\n        SecondExist::fun((X) -> [X])) -> MergedList::[X]\n    when is_subtype(X, any()).\nsmerge2_helper(L1 = [H1 | T1], L2 = [H2 | T2], Lte, EqSelect, FirstExist, SecondExist) ->\n    LteH1H2 = Lte(H1, H2),\n    LteH2H1 = Lte(H2, H1),\n    % note: need to reverse the results of EqSelect, FirstExist, SecondExist since ML is reversed\n    if LteH1H2 andalso LteH2H1 ->\n           EqSelect(H1, H2) ++ smerge2_helper(T1, T2, Lte, EqSelect, FirstExist, SecondExist);\n       LteH1H2 ->\n           FirstExist(H1) ++ smerge2_helper(T1, L2, Lte, EqSelect, FirstExist, SecondExist);\n       LteH2H1 ->\n           SecondExist(H2) ++ smerge2_helper(L1, T2, Lte, EqSelect, FirstExist, SecondExist)\n    end;\nsmerge2_helper(L1, [], _Lte, _EqSelect, FirstExist, _SecondExist) ->\n    lists:flatmap(FirstExist, L1);\nsmerge2_helper([], L2, _Lte, _EqSelect, _FirstExist, SecondExist) ->\n    lists:flatmap(SecondExist, L2).\n\n%% @doc Try to check whether common-test is running.\n-spec is_unittest() -> boolean().\nis_unittest() ->\n    case erlang:get({util, is_unittest}) of\n        undefined ->\n            Pid = self(),\n            % old erlang versions, e.g. R14B04, may not clean up old DOWN messages in\n            % demonitor and thus pollute the message queue and cause 'unknown message'\n            % warnings in gen_components\n            % -> spawn the ct:get_status() call into a separate process\n            % Note: no comm:send_local and no SCALARIS_RECV needed (we are not\n            % interested in tracing this)\n            spawn(fun () ->\n                           case catch ct:get_status() of\n                               no_tests_running -> Pid ! {is_unittest, false};\n                               {error, _} -> Pid ! {is_unittest, false};\n                               {'EXIT', {undef, _}} -> Pid ! {is_unittest, false};\n                               _ -> Pid ! {is_unittest, true}\n                           end\n                  end),\n            receive\n                {is_unittest, Result} ->\n                    erlang:put({util, is_unittest}, Result),\n                    Result\n            end;\n        Value -> Value\n  end.\n\n-spec make_filename([byte()]) -> string().\nmake_filename(Name) ->\n    re:replace(Name, \"[^a-zA-Z0-9\\-_@\\.]\", \"_\", [{return, list}, global]).\n\n%% @doc Get an application environment variable for the 'scalaris' application.\n%%      If the variable is undefined, Default is returned.\n-spec app_get_env(Var::atom(), Default::T) -> T\n    when is_subtype(T, any()).\napp_get_env(Var, Default) ->\n    case application:get_env(scalaris, Var) of\n        {ok, Val} -> Val;\n        _         -> app_check_known(),\n                     Default\n    end.\n\n-spec app_check_known() -> ok.\napp_check_known() ->\n    case application:get_application() of\n        {ok, scalaris } -> ok;\n        undefined ->\n            case is_unittest() of\n                true -> ok;\n                _    ->\n                    %% log:log(\"undefined application but no unittest~n\"),\n                    ok\n            end;\n        {ok, App} ->\n            log:log(\"unknown application: ~.0p~n\", [App]),\n            ok\n    end.\n\n%% @doc check if App is allready running.\n-spec app_check_running(App::atom()) -> boolean().\napp_check_running(App) ->\n    Running = application:which_applications(),\n    case lists:keyfind(App, 1, Running) of\n        false ->\n            false;\n        _Tuple ->\n            true\n    end.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% time calculations\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% @doc convert os:timestamp() to microsecs\n% See http://erlang.org/pipermail/erlang-questions/2008-December/040368.html\n-spec timestamp2us(erlang_timestamp()) -> us_timestamp().\ntimestamp2us({MegaSecs, Secs, MicroSecs}) ->\n    (MegaSecs*1000000 + Secs)*1000000 + MicroSecs.\n\n% @doc convert microsecs to os:timestamp()\n-spec us2timestamp(us_timestamp()) -> erlang_timestamp().\nus2timestamp(Time) ->\n    MicroSecs = Time rem 1000000,\n    Time2 = (Time - MicroSecs) div 1000000,\n    Secs = Time2 rem 1000000,\n    MegaSecs = (Time2 - Secs) div 1000000,\n    {MegaSecs, Secs, MicroSecs}.\n\n-spec time_plus_us(Time::erlang_timestamp(), Delta_MicroSeconds::non_neg_integer()) -> erlang_timestamp().\ntime_plus_us({MegaSecs, Secs, MicroSecs}, Delta) ->\n    MicroSecs1 = MicroSecs + Delta,\n    NewMicroSecs = MicroSecs1 rem 1000000,\n    Secs1 = Secs + (MicroSecs1 div 1000000),\n    NewSecs = Secs1 rem 1000000,\n    MegaSecs1 = MegaSecs + (Secs1 div 1000000),\n    NewMegaSecs = MegaSecs1 rem 1000000,\n    {NewMegaSecs, NewSecs, NewMicroSecs}.\n\n-spec time_plus_ms(Time::erlang_timestamp(), Delta_MilliSeconds::non_neg_integer()) -> erlang_timestamp().\ntime_plus_ms(Time, Delta) ->\n    time_plus_us(Time, Delta * 1000).\n\n-spec time_plus_s(Time::erlang_timestamp(), Delta_Seconds::non_neg_integer()) -> erlang_timestamp().\ntime_plus_s({MegaSecs, Secs, MicroSecs}, Delta) ->\n    Secs1 = Secs + Delta,\n    NewSecs = Secs1 rem 1000000,\n    MegaSecs1 = MegaSecs + (Secs1 div 1000000),\n    NewMegaSecs = MegaSecs1 rem 1000000,\n    {NewMegaSecs, NewSecs, MicroSecs}.\n\n-spec readable_utc_time_feeder({0..1000, 0..1000, 0..1000}) -> {erlang_timestamp()}.\nreadable_utc_time_feeder({A, B, C}) ->\n    {{A, B, C}}.\n\n-spec readable_utc_time(erlang_timestamp()) -> tuple().\nreadable_utc_time(TimeTriple) ->\n    DateTime = calendar:now_to_universal_time(TimeTriple),\n    erlang:append_element(DateTime, element(3, TimeTriple)).\n\n%% acc=AccIn, for(i; I<=n; i++) { acc=AccFun(fun(i), acc) }\n-spec for_to_fold(integer(), integer(), fun((integer()) -> X),\n                  AccFun::fun((X, Acc) -> Acc), AccIn::Acc) -> Acc.\nfor_to_fold(I, N, Fun, AccFun, AccIn) when I =< N ->\n    AccOut = AccFun(Fun(I), AccIn),\n    for_to_fold(I + 1, N, Fun, AccFun, AccOut);\nfor_to_fold(_I, _N, _Fun, _AccFun, AccIn) ->\n    AccIn.\n\n%% @doc for(i; i &lt;= n; i++) { fun(i) }\n-spec for_to(integer(), integer(), fun((integer()) -> any())) -> ok.\nfor_to(I, N, Fun) when I =< N ->\n    Fun(I),\n    for_to(I + 1, N, Fun);\nfor_to(_I, _N, _Fun) ->\n    ok.\n\n%% @doc for(i; i &lt;= n; i++) { Acc = [fun(i) | Acc] }\n%%      (equal to <tt>lists:map(Fun, lists:seq(N,I,-1)) ++ Acc</tt>).\n-spec for_to_ex(integer(), integer(), fun((integer()) -> T), [T]) -> [T].\nfor_to_ex(I, N, Fun, Acc) ->\n    for_to_fold(I, N, Fun, fun(X, XAcc) -> [X | XAcc] end, Acc).\n\n%% @doc for(i; i &lt;= n; i++) { Acc = [fun(i) | Acc] }\n%%      (equal to <tt>lists:map(Fun, lists:seq(N,I,-1))</tt>).\n-spec for_to_ex(integer(), integer(), fun((integer()) -> T)) -> [T].\nfor_to_ex(I, N, Fun) ->\n    for_to_ex(I, N, Fun, []).\n\n-spec map_with_nr_feeder(1..2, [number()], integer()) -> {Fun::fun((number(), integer()) -> number()), List::[number()], integer()}.\nmap_with_nr_feeder(1, List, StartNr) ->\n    {fun(X, I) -> X * I end, List, StartNr};\nmap_with_nr_feeder(2, List, StartNr) ->\n    {fun(X, I) -> X + I end, List, StartNr}.\n\n%% @doc Similar to lists:map/2 but also passes the current number to the fun:\n%%      <tt>[a, b, c,...]</tt> maps to\n%%      <tt>[fun(a, StartNr), fun(b, StartNr+1), fun(c, StartNr+2),...]</tt>\n-spec map_with_nr(fun((A, integer()) -> B), List::[A], StartNr::integer()) -> [B]\n     when is_subtype(A, any()),\n          is_subtype(B, any()).\nmap_with_nr(F, [H | T], Nr) ->\n    [F(H, Nr) | map_with_nr(F, T, Nr + 1)];\nmap_with_nr(F, [], _Nr) when is_function(F, 2) -> [].\n\n-type try_catch_result() :: ok | throw | error | exit.\n\n%% @doc Helper for par_map/2.\n-spec par_map_recv(Id::term(), {try_catch_result(), [B]}) -> {try_catch_result(), [B]}\n    when is_subtype(B, any()).\npar_map_recv(E, {ErrorX, ListX}) ->\n    trace_mpath:thread_yield(),\n    receive ?SCALARIS_RECV({parallel_result, E, {ok, ResultY}},\n                           {ErrorX, [ResultY | ListX]});\n            ?SCALARIS_RECV({parallel_result, E, ErrorY},\n                           {ErrorY, ListX})\n    end.\n\n-spec par_map_feeder(1..2, [number()]) -> {Fun::fun((number()) -> number()), List::[number()]}.\npar_map_feeder(1, List) ->\n    {fun(X) -> X * X end, List};\npar_map_feeder(2, List) ->\n    {fun(X) -> X + X end, List}.\n\n%% @doc Parallel version of lists:map/2. Spawns a new process for each element\n%%      in the list!\n-spec par_map(Fun::fun((A) -> B), List::[A]) -> [B]\n     when is_subtype(A, any()),\n          is_subtype(B, any()).\npar_map(Fun, [E]) -> [Fun(E)];\npar_map(Fun, [_|_] = List) ->\n    _ = [erlang:spawn(?MODULE, parallel_run, [self(), Fun, [E], true, E]) || E <- List],\n    case lists:foldr(fun par_map_recv/2, {{ok, ok}, []}, List) of\n        {{ok, ok}, Result}   -> Result;\n        {{Level, Reason}, _} -> erlang:Level(Reason) % throw the error here again\n    end;\npar_map(Fun, []) when is_function(Fun, 1)-> [].\n\n%% @doc Helper for par_map/3.\n-spec par_map_recv2(ListElem::term(), {try_catch_result(), [B], Id::non_neg_integer()})\n        -> {try_catch_result(), [B], Id::non_neg_integer()}\n     when is_subtype(B, any()).\npar_map_recv2(_E, {ErrorX, ListX, Id}) ->\n    trace_mpath:thread_yield(),\n    receive ?SCALARIS_RECV({parallel_result, Id, {ok, ResultY}},\n                           {ErrorX, lists:reverse(ResultY, ListX), Id + 1});\n            ?SCALARIS_RECV({parallel_result, Id, ErrorY},\n                           {ErrorY, ListX, Id + 1})\n    end.\n\n-spec par_map_feeder(1..2, [number()], 1..50) -> {Fun::fun((number()) -> number()), List::[number()], 1..50}.\npar_map_feeder(FunNr, List, MaxThreads) ->\n    {Fun, List} = par_map_feeder(FunNr, List),\n    {Fun, List, MaxThreads}.\n\n%% @doc Parallel version of lists:map/2 with the possibility to limit the\n%%      maximum number of processes being spawned.\n-spec par_map(Fun::fun((A) -> B), List::[A], MaxThreads::pos_integer()) -> [B]\n     when is_subtype(A, any()),\n          is_subtype(B, any()).\npar_map(Fun, [E], _MaxThreads) -> [Fun(E)];\npar_map(Fun, [_|_] = List, 1) -> lists:map(Fun, List);\npar_map(Fun, [_|_] = List, MaxThreads) ->\n    SplitList = lists_split(List, MaxThreads),\n    lists:foldl(\n      fun(E, Id) ->\n              erlang:spawn(?MODULE, parallel_run,\n                           [self(), fun(X) -> lists:map(Fun, X) end, [E], true, Id]),\n              Id + 1\n      end, 0, SplitList),\n    % note: lists are reversed!\n    case lists:foldl(fun par_map_recv2/2, {{ok, ok}, [], 0}, SplitList) of\n        {{ok, ok}, Result, _}   -> Result;\n        {{Level, Reason}, _, _} -> erlang:Level(Reason) % throw the error here again\n    end;\npar_map(Fun, [], _MaxThreads) when is_function(Fun, 1) -> [].\n\n%% @doc Delete an element from a list (once). When not found, return false.\n-spec lists_take(T, [T]) -> [T] | false when is_subtype(T, any()).\nlists_take(Elem, L) ->\n    lists_take_iter(Elem, L, []).\n\n-spec lists_take_iter(T, [T], [T]) -> [T] | false\n   when is_subtype(T, any()).\nlists_take_iter(Elem, [Elem|T], Acc) -> lists:reverse(Acc, T);\nlists_take_iter(Elem, [H|T], Acc)    -> lists_take_iter(Elem, T, [H|Acc]);\nlists_take_iter(_Elem, [], _Acc)     -> false.\n\n-spec lists_takewith_feeder(Elem::T, [T]) -> {fun((T) -> boolean()), [T]}\n     when is_subtype(T, any()).\nlists_takewith_feeder(Elem, L) ->\n    {fun(X) -> X =:= Elem end, L}.\n\n%% @doc Delete an element from a list (once) based on a predicate. When not\n%%      found, return false.\n-spec lists_takewith(fun((T) -> boolean()), [T]) -> {T, [T]} | false\n     when is_subtype(T, any()).\nlists_takewith(Pred, L) ->\n    lists_takewith_iter(Pred, L, []).\n\n-compile({nowarn_unused_function, {lists_takewith_iter_feeder, 3}}).\n-spec lists_takewith_iter_feeder(Elem::T, [T], [T]) -> {fun((T) -> boolean()), [T], [T]}\n     when is_subtype(T, any()).\nlists_takewith_iter_feeder(Elem, L, Acc) ->\n    {fun(X) -> X =:= Elem end, L, Acc}.\n\n-spec lists_takewith_iter(fun((T) -> boolean()), [T], [T]) -> {T, [T]} | false\n   when is_subtype(T, any()).\nlists_takewith_iter(Pred, [Elem | T], Acc) ->\n    case Pred(Elem) of\n        true  -> {Elem, lists:reverse(Acc, T)};\n        false -> lists_takewith_iter(Pred, T, [Elem | Acc])\n    end;\nlists_takewith_iter(_Pred, [], _Acc) ->\n    false.\n\n%% @doc Splits the given list into several partitions, returning a list of parts\n%%      of the original list. Both the parts and their contents are reversed\n%%      compared to the original list!\n-spec lists_split([A], Partitions::pos_integer()) -> [[A]]\n     when is_subtype(A, any()).\nlists_split([X], _Partitions) -> [[X]]; %% redundant shortcut\nlists_split([_|_] = List, 1) -> [lists:reverse(List)];\nlists_split([_|_] = List, Partitions) ->\n    Len = length(List),\n    BlockSize = Len div Partitions,\n    Rem = Len rem Partitions,\n    case BlockSize =< 1 of\n        true ->\n            %% more partitions than elements, return list of single element lists\n            lists:foldl(fun(E, Acc) -> [[E] | Acc] end, [], List);\n        _ -> lists_split(List, BlockSize, Rem, 0, [], []) % do fair partitioning\n    end;\nlists_split([], _Partitions) -> [].\n\n%% @doc Helper for lists_split/2.\n-spec lists_split([A], BlockSize::pos_integer(), Rem::non_neg_integer(),\n                  CurBlockSize::non_neg_integer(), [A], [[A]]) -> [[A]]\n     when is_subtype(A, any()).\nlists_split([_|_] = List, BlockSize, 0, BlockSize, CurBlock, Result) ->\n    %% blocksize reached and no remainder to distribute: start new block\n    lists_split(List, BlockSize, 0, 0, [], [CurBlock | Result]);\nlists_split([H | T], BlockSize, Rem, BlockSize, CurBlock, Result) ->\n    %% blocksize reached but remainder to distribute: add this element\n    %% to the current block and start the next block\n    lists_split(T, BlockSize, Rem-1, 0, [], [[H | CurBlock] | Result]);\nlists_split([H | T], BlockSize, Rem, CurBlockSize, CurBlock, Result) ->\n    %% add an element to the currently build block\n    lists_split(T, BlockSize, Rem, CurBlockSize + 1, [H | CurBlock], Result);\nlists_split([], _BlockSize, _Rem, _CurBlockSize, CurBlock, Result) ->\n    %% return last block and previously build ones as complete result\n    [CurBlock | Result].\n\n-spec lists_keystore2(Key::term(), NC::pos_integer(), List::[tuple()],\n                      NS::pos_integer(), NewValue::term()) -> [tuple()].\nlists_keystore2(Key, NC, [H | T], NS, NewValue) when element(NC, H) == Key ->\n    [setelement(NS, H, NewValue) | T];\nlists_keystore2(Key, NC, [H | T], NS, NewValue) ->\n    [H | lists_keystore2(Key, NC, T, NS, NewValue)];\nlists_keystore2(_Key, _N, [], _NS, _NewValue) ->\n    [].\n\n-spec lists_partition3_feeder(will_fill_pred, [integer()])\n        -> {fun((integer()) -> 1..3), [integer()]}.\nlists_partition3_feeder(will_fill_pred, List) ->\n    {fun(I) -> abs(I rem 3) + 1 end, List}.\n\n-spec lists_partition3(Pred::fun((Elem :: T) -> 1..3), List::[T])\n    -> {Pred1::[T], Pred2::[T], Pred3::[T]}\n     when is_subtype(T, any()).\nlists_partition3(Pred, L) ->\n    lists_partition3(Pred, L, [], [], []).\n\n-compile({nowarn_unused_function, {lists_partition3_feeder, 5}}).\n-spec lists_partition3_feeder(will_fill_pred, [integer()],\n                              [integer()], [integer()], [integer()])\n        -> {fun((integer()) -> 1..3), [integer()], [integer()], [integer()], [integer()]}.\nlists_partition3_feeder(will_fill_pred, List, As, Bs, Cs) ->\n    {fun(I) -> abs(I rem 3) + 1 end, List, As, Bs, Cs}.\n\n-spec lists_partition3(Pred::fun((Elem :: T) -> 1..3), List::[T],\n                      Acc1::[T], Acc2::[T], Acc3::[T])\n                     ->  {Pred1::[T], Pred2::[T], Pred3::[T]}\n     when is_subtype(T, any()).\nlists_partition3(Pred, [H | T], As, Bs, Cs) ->\n    case Pred(H) of\n        1 -> lists_partition3(Pred, T, [H | As], Bs, Cs);\n        2 -> lists_partition3(Pred, T, As, [H | Bs], Cs);\n        3 -> lists_partition3(Pred, T, As, Bs, [H | Cs])\n    end;\nlists_partition3(Pred, [], As, Bs, Cs) when is_function(Pred, 1) ->\n    {lists:reverse(As), lists:reverse(Bs), lists:reverse(Cs)}.\n\n-spec lists_remove_at_indices([any(),...], [non_neg_integer(),...]) -> [any()].\nlists_remove_at_indices([_|_] = List, [_|_] = Indices) ->\n    lists_remove_at_indices(List, Indices, 0).\n\n% PRED: Indices list should be non-empty\n-spec lists_remove_at_indices([T], Indices::[non_neg_integer()], non_neg_integer())\n        -> [T] when is_subtype(T, any()).\nlists_remove_at_indices(List, [], _CurrentIndex) ->\n    List;\nlists_remove_at_indices([_|ListTail], [CurrentIndex|IndexTail], CurrentIndex) ->\n    lists_remove_at_indices(ListTail, IndexTail, CurrentIndex + 1);\nlists_remove_at_indices([X|L], Indices, CurrentIndex) ->\n    [X | lists_remove_at_indices(L, Indices, CurrentIndex + 1)].\n\n%% @doc A more flexible sublist function than lists:sublist/3.\n%%      Extracts a sublist of length Length starting at Start.\n%%      If Start is negative, we count from the end, e.g. -1 is the last\n%%      element, -2 the second last.\n%%      If Length is negative, the sublist is created in reversed\n%%      direction, e.g. sublist([a,b,c], -1, -2) gets [c, b].\n%%      If Start is less than -ListLength and Length is non-negative, it will be\n%%      set to 1. If Length is negative in this case, an empty sublist will be\n%%      returned.\n%%      If Start is greater than ListLength and Length is non-negative, an empty\n%%      sublist will be returned. If Length is negative in this case, it will\n%%      be set to ListLength.\n%%      Note: sublists never wrap between start and end, i.e.\n%%      sublist([a,b,c], 1, -2) gets []!\n%%      Examples:\n%%       * first 10: sublist(L, 1, 10) | sublist(L, 10, -10) (reverse order)\n%%       * last 10 : sublist(L, -10, 10) | sublist(L, -1, -10) (reverse order)\n-spec sublist(List::[X,...], Start::pos_integer() | neg_integer(), Length::integer()) -> {[X], Length::non_neg_integer()}\n     when is_subtype(X, any()).\nsublist(List, Start, Length) ->\n    ListLen = length(List),\n    NewStart = if Start > ListLen andalso Length >= 0  -> 0;\n                  Start > ListLen andalso Length < 0   -> ListLen;\n                  Start >= 1                           -> Start;\n                  Start < -ListLen andalso Length >= 0 -> 1;\n                  Start < -ListLen andalso Length < 0  -> 0;\n                  Start =< -1                          -> ListLen + Start + 1\n               end,\n    sublist_(List, ListLen, NewStart, Length).\n\n-compile({nowarn_unused_function, {sublist__feeder, 4}}).\n-spec sublist__feeder(List::[X,...], ListLength::non_neg_integer(), Start::non_neg_integer(), Length::integer())\n        -> {List::[X,...], ListLength::non_neg_integer(), Start::non_neg_integer(), Length::integer()}\n     when is_subtype(X, any()).\nsublist__feeder(List, _, Start, Length) ->\n    ListLength = length(List),\n    {List, ListLength, erlang:min(Start, ListLength), Length}.\n\n%% @doc Helper for sublist/3.\n%%      Pre: ListLength =:= length(List), 0 =&lt; Start =&lt; ListLength\n-spec sublist_(List::[X,...], ListLength::non_neg_integer(), Start::non_neg_integer(), Length::integer()) -> {[X], Length::non_neg_integer()}\n     when is_subtype(X, any()).\nsublist_(_List, ListLength, 0, _Length) ->\n    {[], ListLength};\nsublist_(List, ListLength, Start, Length) when Length >= 0 ->\n    {lists:sublist(List, Start, Length), ListLength};\nsublist_(List, ListLength, Start, Length) when Length < 0 ->\n    RevList = lists:reverse(List),\n    NewStart = ListLength - Start + 1, % note: reverse order!\n    {lists:sublist(RevList, NewStart, -Length), ListLength}.\n\n%% @doc If Element is in List, its index is returned (1..length(List) as in lists:nth/2),\n%%      otherwise 'not_found'.\n-spec lists_index_of(Element::T, List::[T]) -> pos_integer() | not_found\n     when is_subtype(T, any()).\nlists_index_of(Element, List) when is_list(List) ->\n    lists_index_of_(Element, List, 1).\n\n%% @doc Helper for lists_index_of/2\n-spec lists_index_of_(Element::T, List::[T], pos_integer()) -> pos_integer() | not_found\n     when is_subtype(T, any()).\nlists_index_of_(_E, [], _N)      -> not_found;\nlists_index_of_(E, [E | _TL], N) -> N;\nlists_index_of_(E, [_H | TL], N) -> lists_index_of_(E, TL, N + 1).\n\n%% @doc Checks whether the given list is at least the given length long without\n%%      going through the whole list like erlang:length/1.\n-spec lists_check_min_length(List::[any()], non_neg_integer()) -> boolean().\nlists_check_min_length([], I) when I > 0 -> false;\nlists_check_min_length(_, 0) -> true;\nlists_check_min_length([_|T], I) ->\n    lists_check_min_length(T, I - 1).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% repeat\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Sequencial (default) or parallel run of function FUN with arguments ARGS TIMES-fold.\n%%      Options as list/propertylist: collect, parallel, accumulate\n%%          * collect (atom): all results of FUN will returned as a list\n%%          * accumulate (tuple): {accumulate, accFun, accInit}\n%%                                all results will be accumulated with accFun\n%%          * parallel (atom): FUN will be called TIMES-fold in parallel.\n%%                             Combination with collect and accumulate is supported.\n%% @end\n-spec repeat(fun(), args(), pos_integer()) -> ok.\nrepeat(Fun, Args, Times) ->\n    NoAccFun = fun(_, _) -> ok end,\n    i_repeat(Fun, Args, Times, NoAccFun, ok).\n-spec repeat(fun(), args(), pos_integer(), [repeat_params()]) -> ok | any().\nrepeat(Fun, Args, Times, Params) ->\n    NoAccFun = fun(_, _) -> ok end,\n    case lists:member(collect, Params) of\n        true ->\n            AccFun = fun(I, R) -> [I | R] end,\n            AccInit = [],\n            ok;\n        false ->\n            case lists:keyfind(accumulate, 1, Params) of\n                false ->\n                    AccFun = NoAccFun,\n                    AccInit = ok;\n                {accumulate, AccFun, AccInit} ->\n                    ok\n            end\n    end,\n    case lists:member(parallel, Params) of\n        true ->\n            repeat(fun spawn/3, [?MODULE, parallel_run,\n                                 [self(), Fun, Args, AccFun =/= NoAccFun, ok]],\n                   Times),\n            case AccFun of\n                NoAccFun -> ok;\n                _ -> parallel_collect(Times, AccFun, AccInit)\n            end;\n        _ -> i_repeat(Fun, Args, Times, AccFun, AccInit)\n    end.\n\n-spec i_repeat(fun(), args(), non_neg_integer(), accumulatorFun(any(), R), R) -> R.\ni_repeat(_, _, 0, _AccFun, Acc) -> Acc;\ni_repeat(Fun, Args, Times, AccFun, Acc) ->\n    R = apply(Fun, Args),\n    i_repeat(Fun, Args, Times - 1, AccFun, AccFun(R, Acc)).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% parallel repeat helper functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec parallel_run(pid(), fun(), args(), boolean(), Id::any()) -> ok.\nparallel_run(SrcPid, Fun, Args, DoAnswer, Id) ->\n    Res = try {ok, apply(Fun, Args)}\n          catch Level:Reason -> {Level, Reason}\n          end,\n    case DoAnswer of\n        true -> comm:send_local(SrcPid, {parallel_result, Id, Res});\n        _ -> ok\n    end,\n    ok.\n\n-spec parallel_collect(non_neg_integer(), accumulatorFun(any(), U), U) -> U.\nparallel_collect(0, _, Accumulator) ->\n    Accumulator;\nparallel_collect(ExpectedResults, AccuFun, Accumulator) ->\n    trace_mpath:thread_yield(),\n    receive ?SCALARIS_RECV({parallel_result, ok, {ok, Result}}, ok);\n            ?SCALARIS_RECV({parallel_result, ok, Result}, ok) % TODO: throw the error here again?\n    end,\n    parallel_collect(ExpectedResults - 1, AccuFun, AccuFun(Result, Accumulator)).\n\n-spec collect_while(GatherFun::fun((non_neg_integer()) -> {boolean(), T} | boolean())) -> [T].\ncollect_while(GatherFun) ->\n    collect_while(GatherFun, 0).\n\n-spec collect_while(GatherFun::fun((non_neg_integer()) -> {boolean(), T} | boolean()), non_neg_integer()) -> [T].\ncollect_while(GatherFun, Count) ->\n    case GatherFun(Count) of\n        {true, Data}  -> [Data | GatherFun(Count + 1)];\n        {false, Data} -> [Data];\n        true          -> GatherFun(Count + 1);\n        false         -> []\n    end.\n\n-spec list_set_nth([A], pos_integer(), B) -> [A | B]\n     when is_subtype(A, any()),\n          is_subtype(B, any()).\nlist_set_nth(L, Pos, Val) ->\n    list_set_nth(L, Pos, Val, 1).\n\n-spec list_set_nth([A], pos_integer(), B, pos_integer()) -> [A | B]\n     when is_subtype(A, any()),\n          is_subtype(B, any()).\nlist_set_nth([_H | T], Pos, Val, Pos) ->\n    [Val | T];\nlist_set_nth([H | T], Pos, Val, Cur) ->\n    [H | list_set_nth(T, Pos, Val, Cur + 1)];\nlist_set_nth([], _Pos, _Val, _Cur) -> [].\n\n-spec debug_info() -> [[{string(), string() | number()}]].\ndebug_info() ->\n    [ [ debug_info(Y) || Y <- pid_groups:members(X)] || X <- pid_groups:groups()].\n\n-spec debug_info(pid()) -> [{string(), term()}];\n                (atom()) -> [[{string(), term()}]].\ndebug_info(PidName) when is_atom(PidName) ->\n    [ debug_info(X) || X <- pid_groups:find_all(PidName)];\ndebug_info(Pid) when is_pid(Pid) ->\n    {GenCompDesc, GenCompInfo} =\n        case gen_component:is_gen_component(Pid) of\n            true ->\n                {Grp, Name} = pid_groups:group_and_name_of(Pid),\n                comm:send_local(Pid , {web_debug_info, self()}),\n                trace_mpath:thread_yield(),\n                receive\n                    ?SCALARIS_RECV({web_debug_info_reply, LocalKVs}, %% ->\n                                   {[{\"pidgroup\", pid_groups:group_to_string(Grp)},\n                                     {\"pidname\", webhelpers:safe_html_string(\"~p\", [Name])}],\n                                    LocalKVs})\n                after 1000 -> {[], []}\n                end;\n            false -> {[], []}\n        end,\n    [{_, Memory}, {_, Reductions}, {_, QueueLen}] =\n        process_info(Pid, [memory, reductions, message_queue_len]),\n    [{\"pid\", pid_to_list(Pid)}]\n        ++   GenCompDesc\n        ++ [{\"memory\", Memory},\n            {\"reductions\", Reductions},\n            {\"message_queue_len\", QueueLen}]\n        ++ GenCompInfo.\n\n%% empty shell_prompt_func\n-spec empty(any()) -> [].\nempty(_) -> \"\".\n\n-spec print_bits(fun((string(), [term()]) -> Result), binary()) -> Result.\nprint_bits(FormatFun, Binary) ->\n    BitSize = erlang:bit_size(Binary),\n    <<BinNr:BitSize>> = Binary,\n    NrBits = lists:flatten(io_lib:format(\"~B\", [BitSize])),\n    FormatFun(\"~\" ++ NrBits ++ \".2B\", [BinNr]).\n\n-spec if_verbose(string()) -> ok.\nif_verbose(String) ->\n    case app_get_env(verbose, false) of\n        true ->  io:format(String);\n        false -> ok\n    end.\n\n-spec if_verbose(string(), list()) -> ok.\nif_verbose(String, Fmt) ->\n    case app_get_env(verbose, false) of\n        true ->  io:format(String, Fmt);\n        false -> ok\n    end.\n\n%% @doc Binary XOR for the two bitstrings, even for big bitstrings where the\n%%      conversion to an integer fails.\n%%      Note: 0's are appended if the sizes do not match.\n-spec bin_xor(bitstring(), bitstring()) -> bitstring().\nbin_xor(Binary1, Binary2) ->\n    bin_op(Binary1, Binary2, fun erlang:'bxor'/2).\n\n%% @doc Binary OR for the two bitstrings, even for big bitstrings where the\n%%      conversion to an integer fails.\n%%      Note: 0's are appended if the sizes do not match.\n-spec bin_or(bitstring(), bitstring()) -> bitstring().\nbin_or(Binary1, Binary2) ->\n    bin_op(Binary1, Binary2, fun erlang:'bor'/2).\n\n%% @doc Binary AND for the two bitstrings, even for big bitstrings where the\n%%      conversion to an integer fails.\n%%      Note: 0's are appended if the sizes do not match.\n-spec bin_and(bitstring(), bitstring()) -> bitstring().\nbin_and(Binary1, Binary2) ->\n    bin_op(Binary1, Binary2, fun erlang:'band'/2).\n\n%% @doc Generic binary operations for the two bitstrings, even for big\n%%      bitstrings where the conversion to an integer fails.\n%%      Note: 0's are appended if the sizes do not match.\n-spec bin_op(bitstring(), bitstring(), fun((integer(), integer()) -> integer()))\n        -> bitstring().\nbin_op(Binary1, Binary2, BinOp) ->\n    BitSize1 = erlang:bit_size(Binary1),\n    BitSize2 = erlang:bit_size(Binary2),\n    ResSize = erlang:max(BitSize1, BitSize2),\n    % up to (at least) Erlang 18.3, there is an upper limit of converting\n    % binaries to integers or if this works the following bxor/2 will fail\n    if ResSize =< 16#1FFFFC0 ->\n           <<BinNr1:BitSize1/little>> = Binary1,\n           <<BinNr2:BitSize2/little>> = Binary2,\n           ResNr = BinOp(BinNr1, BinNr2),\n           <<ResNr:ResSize/little>>;\n       BitSize1 =:= BitSize2 ->\n           % split the binary and bxor each part\n           RestSize = BitSize1 rem 16#1FFFFC0,\n           <<BinNr1:RestSize/little, Bin1TL/binary>> = Binary1,\n           <<BinNr2:RestSize/little, Bin2TL/binary>> = Binary2,\n           ResNr = BinOp(BinNr1, BinNr2),\n           bin_op(Bin1TL, Bin2TL, BinOp, <<ResNr:RestSize/little>>);\n       true ->\n           % first bring the binaries to the same size, then try again:\n           Bin1Large = <<Binary1/bitstring, 0:(ResSize - BitSize1)/little>>,\n           Bin2Large = <<Binary2/bitstring, 0:(ResSize - BitSize2)/little>>,\n           bin_op(Bin1Large, Bin2Large, BinOp)\n    end.\n\n%% @doc Helper for bin_op/3.\n%%      Note: We cannot use erlang:list_to_binary/1 either since that suffers\n%%            from the same problem with big binaries.\n-spec bin_op(binary(), binary(), fun((integer(), integer()) -> integer()),\n              ResultAcc::bitstring()) -> bitstring().\nbin_op(<<>>, <<>>, _BinOp, Acc) ->\n    Acc;\nbin_op(Binary1, Binary2, BinOp, Acc) ->\n    <<BinNr1:16#1FFFFC0/little, Bin1TL/binary>> = Binary1,\n    <<BinNr2:16#1FFFFC0/little, Bin2TL/binary>> = Binary2,\n    ResNr = BinOp(BinNr1, BinNr2),\n    bin_op(Bin1TL, Bin2TL, BinOp, <<Acc/bitstring, ResNr:16#1FFFFC0/little>>).\n\n-ifdef(enable_debug).\n-spec extint2atom(atom()) -> atom().\nextint2atom(X) when is_atom(X) -> X.\n-else.\n-spec extint2atom(atom() | integer()) -> atom().\nextint2atom(X) when is_atom(X) -> X;\nextint2atom(X) when is_integer(X) ->\n    case X of\n        %% lookup\n        ?lookup_aux -> ?lookup_aux_atom;\n        ?lookup_fin -> ?lookup_fin_atom;\n        %% comm\n        ?send_to_group_member -> ?send_to_group_member_atom;\n        ?send_to_registered_proc -> ?send_to_registered_proc_atom;\n        ?deliver -> ?deliver_atom;\n        ?unpack_msg_bundle -> ?unpack_msg_bundle_atom;\n        ?quiet -> ?quiet_atom;\n        %% dht_node\n        ?get_key_with_id_reply -> ?get_key_with_id_reply_atom;\n        ?get_key -> ?get_key_atom;\n        ?read_op -> ?read_op_atom;\n        ?read_op_with_id_reply -> ?read_op_with_id_reply_atom;\n        %% paxos\n        ?proposer_accept -> ?proposer_accept_atom;\n        ?acceptor_accept -> ?acceptor_accept_atom;\n        ?paxos_id -> ?paxos_id_atom;\n        ?proposer_initialize -> ?proposer_initialize_atom;\n        ?proposer_deleteids -> ?proposer_deleteids_atom;\n        %% transactions\n        ?register_TP -> ?register_TP_atom;\n        ?tx_tm_rtm_init_RTM -> ?tx_tm_rtm_init_RTM_atom;\n        ?tp_do_commit_abort -> ?tp_do_commit_abort_atom;\n        ?tx_tm_rtm_delete -> ?tx_tm_rtm_delete_atom;\n        ?tp_committed -> ?tp_committed_atom;\n        ?tx_state -> ?tx_state_atom;\n        ?tx_id -> ?tx_id_atom;\n        ?tx_item_id -> ?tx_item_id_atom;\n        ?tx_item_state -> ?tx_item_state_atom;\n        ?commit_client_id -> ?commit_client_id_atom;\n        ?undecided -> ?undecided_atom;\n        ?prepared -> ?prepared_atom;\n        ?commit -> ?commit_atom;\n        ?abort -> ?abort_atom;\n        ?value -> ?value_atom;\n        ?partial_value -> ?partial_value_atom;\n        ?read -> ?read_atom;\n        ?write -> ?write_atom;\n        ?init_TP -> ?init_TP_atom;\n        ?tp_do_commit_abort_fwd -> ?tp_do_commit_abort_fwd_atom;\n        ?random_from_list -> ?random_from_list_atom;\n        ?sublist -> ?sublist_atom;\n        ?ok -> ?ok_atom;\n        ?fail -> ?fail_atom;\n        ?not_found -> ?not_found_atom;\n        ?empty_list -> ?empty_list_atom;\n        ?not_a_list -> ?not_a_list_atom;\n        %% rrepair\n        ?check_nodes -> ?check_nodes_atom;\n        ?check_nodes_response -> ?check_nodes_response_atom;\n        ?key_upd -> ?key_upd_atom\n    end.\n-endif.\n\n-spec sets_map(Fun::fun((V) -> X), Set::sets:set(V)) -> [X]\n     when is_subtype(V, any()),\n          is_subtype(X, any()).\nsets_map(Fun, Set) ->\n    lists:reverse(sets:fold(fun (El, Acc) ->\n                [Fun(El) | Acc]\n        end, [], Set)).\n\n%% @doc Compare two sets for equality.\n-spec sets_equal(sets:set(V), sets:set(V)) -> boolean()\n     when is_subtype(V, any()).\nsets_equal(Set1, Set2) ->\n    sets:is_subset(Set1, Set2) andalso sets:is_subset(Set2, Set1).\n\n%% @doc Combine the last N slots from a dump into one tuple. The number of slots to\n%% combine is determined by Interval (in us): Take as many slots as needed to look\n%% Interval-Epsilon microseconds back into the past.\n\n-spec rrd_combine_timing_slots(DB :: rrd:rrd(),\n                               CurrentTS :: erlang_timestamp(),\n                               Interval :: non_neg_integer()) ->\n    {Sum :: number(), SquaresSum :: number(), Count :: non_neg_integer(),\n     Min :: number(), Max :: number()} | undefined.\nrrd_combine_timing_slots(DB, CurrentTS, Interval) ->\n    rrd_combine_timing_slots(DB, CurrentTS, Interval, 0). % Epsilon = 0ms\n\n-spec rrd_combine_timing_slots(DB :: rrd:rrd(),\n                               CurrentTS :: erlang_timestamp(),\n                               Interval :: non_neg_integer(),\n                               Epsilon :: non_neg_integer()) ->\n    {Sum :: number(), SquaresSum :: number(), Count :: non_neg_integer(),\n     Min :: number(), Max :: number()} | undefined.\nrrd_combine_timing_slots(DB, CurrentTS, Interval, Epsilon) ->\n    InitialValue = fun({SlotSum,SlotSquared,SlotCount,SlotMin,SlotMax,_}) ->\n            {SlotSum, SlotSquared, SlotCount, SlotMin, SlotMax }\n    end,\n    UpdateValue = fun({Sum, SquaresSum, Count, Min, Max} = _Acc,\n                      {SlotSum,SlotSquared,SlotCount,SlotMin,SlotMax,_}) ->\n            { Sum+SlotSum\n             , SquaresSum + SlotSquared\n             , Count + SlotCount\n             , erlang:min(Min, SlotMin)\n             , erlang:max(Max, SlotMax)\n            }\n    end,\n    rrd_combine_slots(DB, CurrentTS, Interval, Epsilon, InitialValue, UpdateValue).\n\n%% @doc This function does the same as rrd_combine_timing_slots, but for RRDs of type\n%% gauge. It sums up slots until enough slots have been read.\n-spec rrd_combine_gauge_slots(DB :: rrd:rrd(),\n                               CurrentTS :: erlang_timestamp(),\n                               Interval :: non_neg_integer()) -> Value :: number() | undefined.\nrrd_combine_gauge_slots(DB, CurrentTS, Interval) ->\n    rrd_combine_gauge_slots(DB, CurrentTS, Interval, 0). % Epsilon = 0ms\n\n-spec rrd_combine_gauge_slots(DB :: rrd:rrd(),\n                               CurrentTS :: erlang_timestamp(),\n                               Interval :: non_neg_integer(),\n                               Epsilon :: non_neg_integer()) -> Value :: number() | undefined.\nrrd_combine_gauge_slots(DB, CurrentTS, Interval, Epsilon) ->\n    InitialValue = fun(Value) -> Value end,\n    UpdateValue = fun(AccValue, CurrentValue) -> AccValue + CurrentValue end,\n    rrd_combine_slots(DB, CurrentTS, Interval, Epsilon, InitialValue, UpdateValue).\n\n%% @doc Accumulates data over RRD slots. This function allows to reduce a number of slots\n%% into a single accumulator value. The amount of slots is determined by Interval and\n%% Epsilon: it reads as many slots as needed to cover the last Interval - Epsilon us.\n-spec rrd_combine_slots(DB :: rrd:rrd(),\n                        CurrentTS :: erlang_timestamp(),\n                        Interval :: non_neg_integer(),\n                        Epsilon :: non_neg_integer(),\n                        InitialValue :: fun((rrd:data_type()) -> Acc),\n                        UpdateValue :: fun((Acc, rrd:data_type()) -> Acc)\n                       ) -> term() | undefined.\nrrd_combine_slots(DB, CurrentTS, Interval, Epsilon, InitialValue, UpdateValue) ->\n    Slots = rrd:dump(DB),\n    CalcStepLength = fun(Current, From, To) ->\n            case timer:now_diff(Current,From) >= 0\n                andalso timer:now_diff(To, Current) >= 0 of\n                true  -> timer:now_diff(Current, From);\n                false -> timer:now_diff(To, From)\n            end\n    end,\n    Acc = lists:foldl(\n            fun\n                (_, {RemainingUS, _} = Acc) when (RemainingUS - Epsilon) =< 0 -> Acc;\n                ({From, To, Value}, {RemainingUS, Acc}) ->\n                    StepLength = CalcStepLength(CurrentTS, From, To),\n                    {RemainingUS - StepLength, UpdateValue(Acc, Value)};\n                ({From, To, Value}, {RemainingUS}) ->\n                    StepLength = CalcStepLength(CurrentTS, From, To),\n                    {RemainingUS - StepLength, InitialValue(Value)}\n                end, {Interval}, Slots),\n        case Acc of\n            {_, Results} -> Results;\n            {Interval} -> undefined\n        end.\n\n%% @doc Rounds a float according to some precision\n-spec round(Number::float(), Precision::pos_integer()) -> float().\nround(Number, Precision) ->\n    P = pow(10, Precision),\n    erlang:round(Number * P) / P.\n"
  },
  {
    "path": "src/vivaldi_latency.erl",
    "content": "%  @copyright 2009-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Vivaldi helper module for measuring latency between nodes.\n%% @end\n%% @version $Id$\n-module(vivaldi_latency).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n\n-export([start_link/1, on/2, init/1]).\n\n-export([measure_latency/3, check_config/0]).\n\n-include(\"gen_component.hrl\").\n\n%% state of the vivaldi loop\n-type state() :: {gb_trees:tree(task_id(), task_state()), task_id()}.\n\n-type task_id() :: non_neg_integer().\n\n%% state per task\n-type task_state() ::\n    {Owner::comm:erl_local_pid(),\n     RemotePid::comm:mypid(),\n     Token::{gossip_vivaldi:network_coordinate(), gossip_vivaldi:est_error()},\n     Start::erlang_timestamp() | unknown,\n     Count::non_neg_integer(),\n     Latencies::[gossip_vivaldi:latency()]}.\n\n-type token() :: {gossip_vivaldi:network_coordinate(), gossip_vivaldi:est_error()}.\n\n% accepted messages of vivaldi_latency processes\n-type message() ::\n    {ping_reply, {pong, term()}, TaskId::task_id()} |\n    {start_ping, Owner::comm:erl_local_pid(), RemotePid::comm:mypid(), Token::token()} |\n    {shutdown, TaskId::task_id()}.\n\n-define(SEND_OPTIONS, [{channel, prio}, {?quiet}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Message Loop\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc message handler\n-spec on(Message::message(), State::state()) -> state().\non({ping_reply, {pong, gossip}, TaskId}, State = {Tasks, NextTask}) ->\n    case gb_trees:lookup(TaskId, Tasks) of\n        {value, Task} ->\n            {Owner, RemotePid, Token, Start, Count, Latencies} = Task,\n            Stop = os:timestamp(),\n            NewLatencies = [timer:now_diff(Stop, Start) | Latencies],\n            case Count =:= config:read(gossip_vivaldi_count_measurements) of\n                true ->\n                    Msg = {cb_msg, {gossip_vivaldi, default},\n                           {update_vivaldi_coordinate, calc_latency(NewLatencies), Token}},\n                    comm:send_local(Owner, Msg),\n                    {gb_trees:delete(TaskId, Tasks), NextTask};\n                false ->\n                    SPid = comm:reply_as(comm:this(), 2, {ping_reply, '_', TaskId}),\n                    comm:send(RemotePid, {ping, SPid}, ?SEND_OPTIONS),\n                    UpdatedTask = {Owner, RemotePid, Token, os:timestamp(),\n                                   Count+1, NewLatencies},\n                    {gb_trees:enter(TaskId, UpdatedTask, Tasks), NextTask}\n            end;\n        none ->\n            State\n    end;\n\non({ping_reply, {pong, _PidName}, _TaskId}, State) ->\n    % ignore unrelated pong messages\n    State;\n\non({start_ping, Owner, RemotePid, Token}, {Tasks, NextTask}) ->\n    %%NewCount = Count + 1,\n    SPid = comm:reply_as(comm:this(), 2, {ping_reply, '_', NextTask}),\n    comm:send(RemotePid, {ping, SPid}, ?SEND_OPTIONS),\n    Count = config:read(gossip_vivaldi_count_measurements),\n    msg_delay:send_local(Count * config:read(gossip_vivaldi_latency_timeout),\n                         self(), {shutdown, NextTask}, [{?quiet}]),\n    Task = {Owner, RemotePid, Token, os:timestamp(), 1, []},\n    {gb_trees:enter(NextTask, Task, Tasks), NextTask+1};\n\non({shutdown, TaskId}, {Tasks, NextTask}) ->\n    %% log:log(info, \"shutdown vivaldi_latency due to timeout\", []),\n    {gb_trees:delete_any(TaskId, Tasks), NextTask};\n\non(Msg, State) ->\n    io:format(\"vivaldi unknown message: ~p~n\", [Msg]),\n    State.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Init\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec init(Args::term()) -> state().\ninit([]) ->\n    %% {Owner, RemotePid, Token, unknown, 0, []}.\n    {gb_trees:empty(), 0}.\n\n-spec start_link(pid_groups:groupname()) -> {ok, pid()}.\nstart_link(DHTNodeGroup) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                             [{pid_groups_join_as, DHTNodeGroup, vivaldi_latency},\n                              {wait_for_init}]).\n\n-spec measure_latency(comm:mypid(), gossip_vivaldi:network_coordinate(),\n                      gossip_vivaldi:est_error()) -> ok.\nmeasure_latency(RemotePid, RemoteCoordinate, RemoteConfidence) ->\n    Pid = pid_groups:find_a(vivaldi_latency),\n    Token = {RemoteCoordinate, RemoteConfidence},\n    comm:send_local(Pid, {start_ping, self(), RemotePid, Token}),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Helper functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec calc_latency([gossip_vivaldi:latency(),...]) -> number().\ncalc_latency(Latencies) ->\n    mathlib:median(Latencies).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Miscellaneous\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Checks whether config parameters of the vivaldi_latency process exist\n%%      and are valid.\n-spec check_config() -> boolean().\ncheck_config() ->\n    config:cfg_is_integer(gossip_vivaldi_count_measurements) and\n    config:cfg_is_greater_than(gossip_vivaldi_count_measurements, 0) and\n\n    config:cfg_is_integer(gossip_vivaldi_measurements_delay) and\n    config:cfg_is_greater_than_equal(gossip_vivaldi_measurements_delay, 0) and\n\n    config:cfg_is_integer(gossip_vivaldi_latency_timeout) and\n    config:cfg_is_greater_than(gossip_vivaldi_latency_timeout, 0).\n"
  },
  {
    "path": "src/webhelpers.erl",
    "content": "% @copyright 2007-2018 Zuse Institute Berlin,\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc web helpers module for mgmt server to generate the web interface\n%% @version $Id$\n-module(webhelpers).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"yaws_api.hrl\").\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\n-export([getRingChart/0, getRingRendered/0, getIndexedRingRendered/0,\n         get_and_cache_ring/0, flush_ring_cache/0,\n         getDCClustersAndNodes/0,\n         getGossipRendered/0, getVivaldiMap/0,\n         getMonitorClientData/0, getMonitorRingData/0,\n         lookup/1, set_key/2, delete_key/2, isPost/1,\n         safe_html_string/1, safe_html_string/2, html_pre/2,\n         format_nodes/1, format_centroids/1,\n         lookup_kvoncseq/1, set_key_kvoncseq/2\n     ]).\n\n-type attribute_type() :: {atom(), string()}.\n-type html_type() :: {atom(), [attribute_type()], html_type() | [html_type()] | string()}.\n\n% countries for which icons are available in the docroot/icons directory:\n-define(COUNTRIES, [\"aa\", \"au\", \"br\", \"ch\", \"cz\", \"edu\", \"fr\", \"hk\", \"il\",\n                    \"it\", \"kr\", \"no\", \"pr\", \"se\", \"tw\", \"at\", \"be\", \"ca\", \"cn\",\n                    \"de\", \"es\", \" gr\", \"hu\", \"in\", \"jp\", \"nl\", \"pl\", \"pt\",\n                    \"sg\", \"uk\", \"us\"]).\n\n% @doc Checks whether the current request is a post operation.\n-spec isPost(A::#arg{req::#http_request{method::atom()}}) -> boolean().\nisPost(A) ->\n    Method = (A#arg.req)#http_request.method,\n    Method =:= 'POST'.\n\n%%%-----------------------------Lookup/Put/delete---------------------\n\n-spec lookup(Key::?RT:key())\n        -> {TimeInMs::integer(),\n            Result::{Value::db_dht:value(), Version::client_version()} |\n                    {fail, not_found} | {fail, timeout} | {fail, fail}}.\nlookup(Key) ->\n    util:tc(api_tx, read, [Key]).\n\n-spec set_key(Key::?RT:key(), Value::db_dht:value())\n        -> {TimeInMs::integer(),\n            Result::commit | userabort | {fail, not_found} | {fail, timeout} |\n                    {fail, fail} | {fail, abort}}.\nset_key(Key, Value) ->\n    util:tc(api_tx, write, [Key, Value]).\n\n-spec delete_key(Key::?RT:key(), Timeout::pos_integer())\n        -> {TimeInMs::integer(),\n            Result::{ok, pos_integer(), list()} | {fail, timeout} |\n                    {fail, timeout, pos_integer(), list()}}.\ndelete_key(Key, Timeout) ->\n    util:tc(api_rdht, delete, [Key, Timeout]).\n\n%%%-----------------------------Lookup/Put/delete (kvoncseq)----------\n\n-spec lookup_kvoncseq(string()) -> {ok, _} | {fail, not_found}.\nlookup_kvoncseq(Key) ->\n    kv_on_cseq:read(Key).\n\n-spec set_key_kvoncseq(string(), client_value()) -> {ok}.\nset_key_kvoncseq(Key, Value) ->\n    kv_on_cseq:write(Key, Value).\n\n%%%--------------------------Vivaldi-Map------------------------------\n-spec getVivaldiMap() -> [{{comm:mypid(), node_details:hostname()}, gossip_vivaldi:network_coordinate()}].\ngetVivaldiMap() ->\n    Nodes = [{\n                node:pidX(node_details:get(Node, node))\n                , node_details:get(Node, hostname)\n            } || {ok, Node} <- get_and_cache_ring()],\n    NodePids = [P || {P, _} <- Nodes],\n    This = comm:this(),\n    _ = [erlang:spawn(\n           fun() ->\n                   SourcePid = comm:reply_as(This, 2, {webhelpers, '_', Pid}),\n                   comm:send(Pid, {cb_msg, {gossip_vivaldi, default}, {get_coordinate, SourcePid}}, [{group_member, gossip}])\n           end) || Pid <- NodePids],\n    lists:zip(Nodes, get_vivaldi(NodePids, [], 0))\n    .\n\n-spec get_vivaldi(Pids::[comm:mypid()], [gossip_vivaldi:network_coordinate()],\n                  TimeInMS::non_neg_integer()) -> [gossip_vivaldi:network_coordinate()].\nget_vivaldi([], Coords, _TimeInS) -> Coords;\nget_vivaldi(Pids, Coords, TimeInMS) ->\n    Continue =\n        if\n            TimeInMS =:= 2000 ->\n                log:log(error,\"[ WH ]: 2sec Timeout waiting for vivaldi_get_coordinate_response from ~p\",[Pids]),\n                continue;\n            TimeInMS >= 6000 ->\n                log:log(error,\"[ WH ]: 6sec Timeout waiting for vivaldi_get_coordinate_response from ~p\",[Pids]),\n                stop;\n            true -> continue\n    end,\n    case Continue of\n        continue ->\n            trace_mpath:thread_yield(),\n            receive\n                ?SCALARIS_RECV(\n                    {webhelpers, {vivaldi_get_coordinate_response, Coordinate, _Confidence}, Pid}, %% ->\n                    get_vivaldi(lists:delete(Pid, Pids), [Coordinate | Coords],\n                                TimeInMS)\n                  )\n            after\n                10 ->\n                    get_vivaldi(Pids, Coords, TimeInMS + 10)\n            end;\n        _ -> Coords\n    end.\n\n% @doc Choose a random color for a Pid\n-spec color(comm:mypid()) -> string().\ncolor(Pid) ->\n    Hash = erlang:phash2(Pid, 256*256*256),\n    R = Hash rem 256,\n    G = (Hash div 256) rem 256,\n    B = (Hash div (256*256)) rem 256,\n    io_lib:format(\"rgb(~p,~p,~p)\",[R,G,B]).\n\n% @doc Get a string representation for a vivaldi coordinate\n-spec format_coordinate([gossip_vivaldi:network_coordinate(),...]) -> string().\nformat_coordinate([X,Y]) ->\n    io_lib:format(\"[~p,~p]\", [X,Y]).\n\n% @doc Format Nodes as returned by getVivaldiMap() into JSON.\n-spec format_nodes([{{comm:mypid(), node_details:hostname()}\n                     , gossip_vivaldi:network_coordinate()}]) -> string().\nformat_nodes(Nodes) ->\n    % order nodes according to their datacenter (designated by color)\n    NodesTree = lists:foldl(\n        fun({{NodeName, NodeHost}, Coords}, Acc) ->\n            Key = color(NodeName),\n            PriorValue = gb_trees:lookup(Key, Acc),\n            V = case PriorValue of\n                none -> [];\n                {value, PV} -> PV\n            end,\n            gb_trees:enter(Key, [io_lib:format(\"{\\\"name\\\":\\\"~p\\\",\\\"coords\\\":~s,\\\"host\\\":\\\"~s\\\"}\"\n                                               , [comm:make_local(NodeName),format_coordinate(Coords),NodeHost]) | V]\n                           , Acc)\n    end, gb_trees:empty(), Nodes),\n\n    \"[\" ++ util:gb_trees_foldl(fun(Color, DCNodes, Acc) ->\n                NodeString = string:join(\n                        [io_lib:format(\"{\\\"color\\\":\\\"~s\\\",\\\"info\\\":~s}\",\n                                        [Color,NodeInfo])\n                         || NodeInfo <- DCNodes ], \",\"),\n                Sep = case Acc of\n                    \"\" -> \"\";\n                    _ -> \",\"\n                end,\n                NodeString ++ Sep ++ Acc\n    end, \"\", NodesTree) ++ \"]\".\n%%%--------------------------DC Clustering------------------------------\n-spec getDCClustersAndNodes() -> {[{comm:mypid(), gossip_vivaldi:network_coordinate()}],\n        dc_centroids:centroids(), non_neg_integer(), float()} | disabled.\ngetDCClustersAndNodes() ->\n    case config:read(dc_clustering_enable) of\n        true ->\n            This = comm:this(),\n            ClusteringProcess = pid_groups:find_a(dc_clustering),\n            comm:send_local(ClusteringProcess, {query_clustering, This}),\n            comm:send_local(ClusteringProcess, {query_my, local_epoch, This}),\n            comm:send_local(ClusteringProcess, {query_my, radius, This}),\n\n            % note: receive wrapped in anonymous functions to allow\n            %       ?SCALARIS_RECV in multiple receive statements\n            Centroids = fun() ->\n                trace_mpath:thread_yield(),\n                receive\n                ?SCALARIS_RECV({query_clustering_response, Cs}, Cs)\n            after 2000 ->\n                    log:log(error,\"[ WH ] Timeout getting query_clustering_response from dc_clustering\"),\n                    throw('dc_clustering_timeout')\n            end end(),\n\n            Epoch = fun() ->\n                trace_mpath:thread_yield(),\n                receive\n                ?SCALARIS_RECV({query_my_response, local_epoch, E}, E)\n            after 2000 ->\n                    log:log(error,\"[ WH ] Timeout getting local_epoch from dc_clustering\"),\n                    throw('dc_clustering_timeout')\n            end end(),\n\n            Radius = fun() ->\n                trace_mpath:thread_yield(),\n                receive\n                ?SCALARIS_RECV({query_my_response, radius, R}, R)\n            after 2000 ->\n                    log:log(error,\"[ WH ] Timeout getting radius from dc_clustering\"),\n                    throw('dc_clustering_timeout')\n            end end(),\n\n            Nodes = getVivaldiMap(),\n\n            {Nodes, Centroids, Epoch, Radius};\n        _ -> disabled\n    end.\n\nformat_centroid(Centroid) ->\n    {Coords, Radius} = dc_centroids:get_coordinate_and_relative_size(Centroid),\n    FormatString = \"{\\\"coords\\\":~p, \\\"radius\\\":~f}\",\n    io_lib:format(FormatString, [Coords, Radius])\n    .\n\n%% @doc Convert a list of centroids into a JSON string\n-spec format_centroids(Centroids :: [dc_centroids:centroid()]) -> string().\nformat_centroids(Centroids) ->\n    \"[\"\n    ++ lists:flatten([format_centroid(C) || C <- Centroids])\n    ++ \"]\"\n    .\n\n%%%-----------------------------Ring----------------------------------\n\n-spec get_and_cache_ring() -> statistics:ring().\nget_and_cache_ring() ->\n    case erlang:get(web_scalaris_ring) of\n        undefined ->\n            Ring =\n                case whereis(mgmt_server) of\n                    undefined -> statistics:get_ring_details_neighbors(1); % get neighbors only\n                    _         -> statistics:get_ring_details()\n                end,\n            % mapping node_id -> index\n            _ = util:map_with_nr(\n                  fun(RingE, NrX) ->\n                          case RingE of\n                              {ok, NodeDetailsX} ->\n                                  NodeX = node_details:get(NodeDetailsX, node),\n                                  erlang:put({web_scalaris_ring_idx, node:id(NodeX)}, NrX),\n                                  erlang:put(web_scalaris_ring_size, NrX),\n                                  ok;\n                              _ -> ok\n                          end\n                  end, Ring, 1),\n            erlang:put(web_scalaris_ring, Ring);\n        Ring -> ok\n    end,\n    Ring.\n\n-spec get_indexed_id_by_node(Node::node:node_type()) -> non_neg_integer() | undefined.\nget_indexed_id_by_node(Node) ->\n    get_indexed_id_by_id(node:id(Node)).\n\n-spec get_indexed_id_by_id(NodeId::?RT:key()) -> non_neg_integer() | undefined.\nget_indexed_id_by_id(NodeId) ->\n    erlang:get({web_scalaris_ring_idx, NodeId}).\n\n-spec get_ring_size() -> non_neg_integer() | undefined.\nget_ring_size() ->\n    erlang:get(web_scalaris_ring_size).\n\n-spec flush_ring_cache() -> ok.\nflush_ring_cache() ->\n    erlang:erase(web_scalaris_ring),\n    ok.\n\n-spec extract_ring_info([node_details:node_details()])\n        -> [{Label::string(), Value::string(), Known::boolean()}].\nextract_ring_info([]) ->\n    [];\nextract_ring_info([RingE]) ->\n    Node = node_details:get(RingE, node),\n    MyId = node:id(Node),\n    PredId = node:id(node_details:get(RingE, pred)),\n    MyIndexStr = case get_indexed_id_by_id(MyId) of\n                     undefined -> \"\";\n                     MyIndex -> integer_to_list(MyIndex) ++ \": \"\n                 end,\n    NodePid = node:pidX(Node),\n    case comm:get_ip(NodePid) of\n        {NodeIP1, NodeIP2, NodeIP3, NodeIP4} -> ok;\n        _ -> NodeIP1 = NodeIP2 = NodeIP3 = NodeIP4 = 0\n    end,\n    NodePort = comm:get_port(NodePid),\n    YawsPort = node:yawsPort(Node),\n    Label = lists:flatten(io_lib:format(\"~s<a href=\\\\\\\"http://~B.~B.~B.~B:~B/ring.yaws\\\\\\\">~s / ~B.~B.~B.~B:~B</a> (~B)\",\n                  [MyIndexStr,\n                   NodeIP1, NodeIP2, NodeIP3, NodeIP4, YawsPort,\n                   node_details:get(RingE, hostname),\n                   NodeIP1, NodeIP2, NodeIP3, NodeIP4, NodePort,\n                   node_details:get(RingE, load)])),\n    case MyId =:= PredId of\n        true -> [{Label, \"100.00\", true}];\n        _ ->\n            Diff = ?RT:get_range(PredId, MyId) * 100 / ?RT:n(),\n            Me_val = io_lib:format(\"~f\", [Diff]),\n            Unknown_val = io_lib:format(\"~f\", [100 - Diff]),\n            [{Label, Me_val, true}, {\"unknown\", Unknown_val, false}]\n    end;\nextract_ring_info([First | _Rest] = Ring) ->\n    extract_ring_info(Ring, First, []).\n\n-spec extract_ring_info(Ring::[node_details:node_details()], First::node_details:node_details(),\n                        Acc::[T | [T]]) -> [T]\n        when is_subtype(T, {Label::string(), Value::string(), Known::boolean()}).\nextract_ring_info([RingE], First, Acc) ->\n    NewAcc = [extract_ring_info2(RingE, First) | Acc],\n    lists:flatten(lists:reverse(NewAcc));\nextract_ring_info([RingE1, RingE2 | Rest], First, Acc) ->\n    NewAcc = [extract_ring_info2(RingE1, RingE2) | Acc],\n    extract_ring_info([RingE2 | Rest], First, NewAcc).\n\n-spec extract_ring_info2(RingE1::node_details:node_details(), RingE2::node_details:node_details())\n        -> [{Label::string(), Value::string(), Known::boolean()}].\nextract_ring_info2(RingE1, RingE2) ->\n    E1_Node = node_details:get(RingE1, node),\n    E1_MyId = node:id(E1_Node),\n    E1_PredId = node:id(node_details:get(RingE1, pred)),\n    E2_PredId = node:id(node_details:get(RingE2, pred)),\n    E1_Diff = ?RT:get_range(E1_PredId, E1_MyId) * 100 / ?RT:n(),\n    E1_MyIndexStr = case get_indexed_id_by_id(E1_MyId) of\n                        undefined -> \"\";\n                        E1_MyIndex -> integer_to_list(E1_MyIndex) ++ \": \"\n                    end,\n    E1_NodePid = node:pidX(E1_Node),\n    case comm:get_ip(E1_NodePid) of\n        {E1_NodeIP1, E1_NodeIP2, E1_NodeIP3, E1_NodeIP4} -> ok;\n        _ -> E1_NodeIP1 = E1_NodeIP2 = E1_NodeIP3 = E1_NodeIP4 = 0\n    end,\n    E1_NodePort = comm:get_port(E1_NodePid),\n    E1_YawsPort = node:yawsPort(E1_Node),\n    E1_Label = lists:flatten(io_lib:format(\"~s<a href=\\\\\\\"http://~B.~B.~B.~B:~B/ring.yaws\\\\\\\">~s / ~B.~B.~B.~B:~B</a> (~B)\",\n                  [E1_MyIndexStr,\n                   E1_NodeIP1, E1_NodeIP2, E1_NodeIP3, E1_NodeIP4, E1_YawsPort,\n                   node_details:get(RingE1, hostname),\n                   E1_NodeIP1, E1_NodeIP2, E1_NodeIP3, E1_NodeIP4, E1_NodePort,\n                   node_details:get(RingE1, load)])),\n    E1_Me_val = io_lib:format(\"~f\", [E1_Diff]),\n    case E1_MyId =:= E2_PredId of\n        true -> [{E1_Label, E1_Me_val, true}];\n        _ -> % add an \"unknown\" slice as there is another (but unknown) node:\n            Diff2 = ?RT:get_range(E1_MyId, E2_PredId) * 100 / ?RT:n(),\n            Unknown_val = io_lib:format(\"~f\", [Diff2]),\n            [{E1_Label, E1_Me_val, true}, {\"unknown\", Unknown_val, false}]\n    end.\n\n-spec getRingChart() -> [html_type()].\ngetRingChart() ->\n    RealRing = get_and_cache_ring(),\n    Ring = [NodeDetails || {ok, NodeDetails} <- RealRing],\n    case length(Ring) of\n        0 -> [];\n        RingSize ->\n            try\n                Data = extract_ring_info(Ring),\n                ColorAlphaInc = 1 / RingSize,\n                {_, DataStr0, ColorStr0} =\n                    lists:foldr(\n                      fun({Label, Value, Known}, {I, DStr, CStr}) ->\n                              {CurColor, NewI} =\n                                  case Known of\n                                      true ->\n                                          Alpha = case (1 - I * ColorAlphaInc) of\n                                                      X when X > 0.2 -> X;\n                                                      _ -> 0.2\n                                                  end,\n                                          AlphaStr = io_lib:format(\"~.2f\", [Alpha]),\n                                          {lists:flatten([\"$.color.make(0, 128, 128, \", AlphaStr, \").toString()\"]), I -1};\n                                      _ -> {\"$.color.make(255, 255, 255, 1).toString()\", I}\n                                  end,\n                              CurData = lists:append([\"{ label: \\\"\", Label, \"\\\", data: \", Value, \" }\"]),\n                              NewData = case DStr of\n                                            [] -> CurData;\n                                            _  -> lists:append([CurData, \", \", DStr])\n                                        end,\n                              NewColors = case CStr of\n                                              [] -> CurColor;\n                                              _  -> lists:append([CurColor, \", \", CStr])\n                                          end,\n                              {NewI, NewData, NewColors}\n                      end, {RingSize - 1, \"\", \"\"}, Data),\n                DataStr = lists:flatten([\"var ring = [\", DataStr0, \"];\\n\"\n                                         \"var colors = [\", ColorStr0, \"];\\n\"]),\n                [{script, [{type, \"text/javascript\"}], lists:append([\"$(function() {\\n\", DataStr, \"plot_ring(ring, colors);\\n});\\n\"])},\n                 {table, [],\n                  [{tr, [],\n                    [{td, [], {'div', [{id, \"ring\"}, {style, \"width: 600px; height: 350px\"}], []}},\n                     {td, [], {'div', [{id, \"ringhover\"}, {style, \"width: 100px; height: 350px\"}], []}}]}]}\n                ]\n            catch % ?RT methods might throw\n                throw:not_supported -> [{p, [], \"Sorry, pie chart not available (unknown error).\"}]\n            end\n    end.\n\n-spec getRingRendered() -> html_type().\ngetRingRendered() ->\n    RealRing = get_and_cache_ring(),\n    Ring = [X || X = {ok, _} <- RealRing],\n    RingSize = length(Ring),\n    if\n        RingSize =:= 0 ->\n            {p, [], \"empty ring\"};\n        true ->\n            {p, [],\n              [\n              {table, [{bgcolor, \"#CCDCEE\"}, {width, \"100%\"}],\n               [%% items\n                {tr, [{bgcolor, \"#000099\"}],\n                 [\n                  {td, [{width, \"10%\"}], {strong, [], {font, [{color, \"white\"}], \"\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Total\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Average\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Std. Deviation\"}}}\n                 ]\n                },\n                {tr, [],\n                 [\n                  {td, [{bgcolor, \"#000099\"}], {strong, [], {font, [{color, \"white\"}], \"Items\"}}},\n                  {td, [], io_lib:format(\"~p\", [statistics:get_total_load(load, RealRing)])},\n                  {td, [], io_lib:format(\"~p\", [erlang:round(statistics:get_average_load(load, RealRing))])},\n                  {td, [], io_lib:format(\"~p\", [erlang:round(statistics:get_load_std_deviation(load, RealRing))])}\n                 ]\n                },\n                %% load\n                {tr, [],\n                 [\n                  {td, [{bgcolor, \"#000099\"}], {strong, [], {font, [{color, \"white\"}], \"Load\"}}},\n                  {td, [], io_lib:format(\"-\", [])},\n                  {td, [], io_lib:format(\"~p\", [erlang:round(statistics:get_average_load(load2, RealRing))])},\n                  {td, [], io_lib:format(\"~p\", [erlang:round(statistics:get_load_std_deviation(load2, RealRing))])}\n                 ]\n                },\n                %% requests\n                {tr, [],\n                 [\n                  {td, [{bgcolor, \"#000099\"}], {strong, [], {font, [{color, \"white\"}], \"Requests\"}}},\n                  {td, [], io_lib:format(\"~p\", [statistics:get_total_load(load3, RealRing)])},\n                  {td, [], io_lib:format(\"~p\", [erlang:round(statistics:get_average_load(load3, RealRing))])},\n                  {td, [], io_lib:format(\"~p\", [erlang:round(statistics:get_load_std_deviation(load3, RealRing))])}\n                 ]\n                }\n               ]\n              },\n              {br, []},\n              {table, [{bgcolor, \"#CCDCEE\"}, {width, \"100%\"}],\n               [{tr, [{bgcolor, \"#000099\"}],\n                 [\n                  {td, [{align, \"center\"}, {width,\"350px\"}], {strong, [], {font, [{color, \"white\"}], \"Host\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Preds\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Node\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Succs\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"RTSize\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Items\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Load\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Requests\"}}}\n                 ]},\n                lists:append([renderRing(X) ||  X = {ok, _} <- RealRing],\n                             [renderRing(X) ||  X = {failed, _} <- RealRing])\n               ]\n              }\n             ]\n            }\n    end.\n\n-spec renderRing(Node::statistics:ring_element()) -> html_type().\nrenderRing({ok, Details}) ->\n    Hostname = node_details:get(Details, hostname),\n    PredList = node_details:get(Details, predlist),\n    Node = node_details:get(Details, node),\n    SuccList = node_details:get(Details, succlist),\n    RTSize = node_details:get(Details, rt_size),\n    Items = node_details:get(Details, load),\n    Load = node_details:get(Details, load2),\n    Requests = node_details:get(Details, load3),\n    MyIndexStr = case get_indexed_id_by_node(Node) of\n                     undefined -> dead_node();\n                     MyIndex -> MyIndex\n                 end,\n    NodeListFun =\n        fun(NodeX) ->\n                case get_indexed_id_by_node(NodeX) of\n                    undefined -> lists:flatten(\n                                   io_lib:format(\"<b>~p</b>\", [node:id(NodeX)]));\n                    X         -> X\n                end\n        end,\n    NodePid = node:pidX(Node),\n    case comm:get_ip(NodePid) of\n        {NodeIP1, NodeIP2, NodeIP3, NodeIP4} -> ok;\n        _ -> NodeIP1 = NodeIP2 = NodeIP3 = NodeIP4 = 0\n    end,\n    NodePort = comm:get_port(NodePid),\n    YawsPort = node:yawsPort(Node),\n    {tr, [],\n      [\n       {td, [], [get_flag(Hostname),\n                 io_lib:format(\"<a href=\\\"http://~B.~B.~B.~B:~B/ring.yaws\\\">~p (~B.~B.~B.~B:~B)</a>\",\n                               [NodeIP1, NodeIP2, NodeIP3, NodeIP4, YawsPort,\n                                Hostname,\n                                NodeIP1, NodeIP2, NodeIP3, NodeIP4, NodePort])]},\n       {td, [], io_lib:format(\"~.100p\", [[NodeListFun(N) || N <- PredList]])},\n       {td, [], io_lib:format(\"~p:&nbsp;~p\", [MyIndexStr, node:id(Node)])},\n       {td, [], io_lib:format(\"~.100p\", [[NodeListFun(N) || N <- SuccList]])},\n       {td, [], io_lib:format(\"~p\", [RTSize])},\n       {td, [], io_lib:format(\"~p\", [Items])},\n       {td, [], io_lib:format(\"~p\", [Load])},\n       {td, [], io_lib:format(\"~p\", [Requests])}\n      ]};\nrenderRing({failed, Pid}) ->\n    {tr, [],\n      [\n       {td, [], \"-\"},\n       {td, [], \"-\"},\n       {td, [], io_lib:format(\"- (~p)\", [Pid])},\n       {td, [], \"-\"},\n       {td, [], \"-\"},\n       {td, [], \"-\"},\n       {td, [], \"-\"}\n      ]}.\n\n-spec getIndexedRingRendered() -> html_type().\ngetIndexedRingRendered() ->\n    RealRing = get_and_cache_ring(),\n    RingSize = get_ring_size(),\n    EHtml =\n        if\n        RingSize =:= 0 ->\n            {p, [], \"empty ring\"};\n        true ->\n            {p, [],\n              [\n              {table, [{bgcolor, \"#CCDCEE\"}, {width, \"100%\"}],\n               [%% items\n                {tr, [{bgcolor, \"#000099\"}],\n                 [\n                  {td, [{width, \"10%\"}], {strong, [], {font, [{color, \"white\"}], \"\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Total\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Average\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Std. Deviation\"}}}\n                 ]\n                },\n                {tr, [],\n                 [\n                  {td, [{bgcolor, \"#000099\"}], {strong, [], {font, [{color, \"white\"}], \"Items\"}}},\n                  {td, [], io_lib:format(\"~p\", [statistics:get_total_load(load, RealRing)])},\n                  {td, [], io_lib:format(\"~p\", [erlang:round(statistics:get_average_load(load, RealRing))])},\n                  {td, [], io_lib:format(\"~p\", [erlang:round(statistics:get_load_std_deviation(load, RealRing))])}\n                 ]\n                },\n                %% load\n                {tr, [],\n                 [\n                  {td, [{bgcolor, \"#000099\"}], {strong, [], {font, [{color, \"white\"}], \"Load\"}}},\n                  {td, [], io_lib:format(\"-\", [])},\n                  {td, [], io_lib:format(\"~p\", [erlang:round(statistics:get_average_load(load2, RealRing))])},\n                  {td, [], io_lib:format(\"~p\", [erlang:round(statistics:get_load_std_deviation(load2, RealRing))])}\n                 ]\n                },\n                %% requests\n                {tr, [],\n                 [\n                  {td, [{bgcolor, \"#000099\"}], {strong, [], {font, [{color, \"white\"}], \"Requests\"}}},\n                  {td, [], io_lib:format(\"~p\", [statistics:get_total_load(load3, RealRing)])},\n                  {td, [], io_lib:format(\"~p\", [erlang:round(statistics:get_average_load(load3, RealRing))])},\n                  {td, [], io_lib:format(\"~p\", [erlang:round(statistics:get_load_std_deviation(load3, RealRing))])}\n                 ]\n                }\n               ]\n              },\n              {br, []},\n              {table, [{bgcolor, \"#CCDCEE\"}, {width, \"100%\"}],\n               [{tr, [{bgcolor, \"#000099\"}],\n                 [\n                  {td, [{align, \"center\"}, {width,\"350px\"}], {strong, [], {font, [{color, \"white\"}], \"Host\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Preds Offset\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Node\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Succs Offsets\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"RTSize\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Items\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Load\"}}},\n                  {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Requests\"}}}\n                 ]},\n                lists:append([renderIndexedRing(X) ||  X = {ok, _} <- RealRing],\n                             [renderIndexedRing(X) ||  X = {failed, _} <- RealRing])\n               ]\n              }\n             ]\n            }\n    end,\n    EHtml.\n\n%% @doc Renders an indexed ring into ehtml.\n%%      Precond: existing Ring from get_and_cache_ring/0 with\n%%      node_id -> index mapping.\n-spec renderIndexedRing(Node::statistics:ring_element()) -> html_type().\nrenderIndexedRing({ok, Details}) ->\n    Hostname = node_details:get(Details, hostname),\n    PredList = node_details:get(Details, predlist),\n    Node = node_details:get(Details, node),\n    SuccList = node_details:get(Details, succlist),\n    RTSize = node_details:get(Details, rt_size),\n    Items = node_details:get(Details, load),\n    Load = node_details:get(Details, load2),\n    Requests = node_details:get(Details, load3),\n    MyIndex = get_indexed_id_by_node(Node),\n    PredIndex = lists:map(fun(Pred) -> get_indexed_pred_id(Pred, MyIndex) end, PredList),\n    SuccIndices = lists:map(fun(Succ) -> get_indexed_succ_id(Succ, MyIndex) end, SuccList),\n    [FirstSuccIndex|_] = SuccIndices,\n    MyIndexStr = case MyIndex of\n                     undefined -> dead_node();\n                     X -> X\n                 end,\n    NodePid = node:pidX(Node),\n    case comm:get_ip(NodePid) of\n        {NodeIP1, NodeIP2, NodeIP3, NodeIP4} -> ok;\n        _ -> NodeIP1 = NodeIP2 = NodeIP3 = NodeIP4 = 0\n    end,\n    NodePort = comm:get_port(NodePid),\n    YawsPort = node:yawsPort(Node),\n    {tr, [],\n      [\n       {td, [], [get_flag(Hostname),\n                 io_lib:format(\"<a href=\\\"http://~B.~B.~B.~B:~B/ring.yaws\\\">~p (~B.~B.~B.~B:~B)</a>\",\n                               [NodeIP1, NodeIP2, NodeIP3, NodeIP4, YawsPort,\n                                Hostname,\n                                NodeIP1, NodeIP2, NodeIP3, NodeIP4, NodePort])]},\n       case hd(PredIndex) =:= -1 of\n           true->\n               {td, [], io_lib:format(\"~p\", [PredIndex])};\n           false ->\n               {td, [], io_lib:format(\"<span style=\\\"color:red\\\">~p</span>\", [PredIndex])}\n       end,\n       {td, [], io_lib:format(\"~p:&nbsp;~p\", [MyIndexStr, node:id(Node)])},\n       case is_list(FirstSuccIndex) orelse FirstSuccIndex =/= 1 of\n           true -> {td, [], io_lib:format(\"<span style=\\\"color:red\\\">~p</span>\", [SuccIndices])};\n           false -> {td, [], io_lib:format(\"~p\", [SuccIndices])}\n       end,\n       {td, [], io_lib:format(\"~p\", [RTSize])},\n       {td, [], io_lib:format(\"~p\", [Items])},\n       {td, [], io_lib:format(\"~p\", [Load])},\n       {td, [], io_lib:format(\"~p\", [Requests])}\n      ]};\n\nrenderIndexedRing({failed, Pid}) ->\n    {tr, [],\n      [\n       {td, [], \"-\"},\n       {td, [], \"-\"},\n       {td, [], io_lib:format(\"- (~p)\", [Pid])},\n       {td, [], \"-\"},\n       {td, [], \"-\"},\n       {td, [], \"-\"},\n       {td, [], \"-\"},\n       {td, [], \"-\"}\n      ]}.\n\n\n%%%-----------------------------Gossip----------------------------------\n\n-type gossip_pv() :: {PidName::string(), gossip_load:load_info()}.\n-type gossip_key() :: avgLoad | stddev | size_ldr | size_kr | minLoad | maxLoad.\n\n-spec getGossip() -> [gossip_pv()].\ngetGossip() ->\n    GossipPids = pid_groups:find_all(gossip),\n    [begin\n         comm:send_local(Pid, {cb_msg, {gossip_load, default}, {gossip_get_values_best, self()}}),\n         trace_mpath:thread_yield(),\n         receive\n             ?SCALARIS_RECV(\n                 {gossip_get_values_best_response, BestValues}, %% ->\n                 {pid_groups:pid_to_name(Pid), BestValues}\n               )\n         end\n     end || Pid <- GossipPids].\n\n-spec getGossipRendered() -> html_type().\ngetGossipRendered() ->\n    Values = getGossip(),\n    { table, [{bgcolor, \"#CCDCEE\"}, {width, \"100%\"}], renderGossip(Values) }.\n\n-spec renderGossip([gossip_pv()]) -> [html_type()].\nrenderGossip([]) -> [];\nrenderGossip([V1]) ->\n    renderGossip2(V1, {false, V1}, {false, V1});\nrenderGossip([V1, V2]) ->\n    renderGossip2(V1, {true, V2}, {false, V2});\nrenderGossip([V1, V2, V3 | Rest]) ->\n    lists:append(\n      renderGossip2(V1, {true, V2}, {true, V3}),\n      renderGossip(Rest)).\n\n-spec renderGossip2(gossip_pv(), {boolean(), gossip_pv()}, {boolean(), gossip_pv()}) -> [html_type()].\nrenderGossip2(PV1, PVE2, PVE3) ->\n    [renderGossipHead(PV1, PVE2, PVE3),\n     renderGossipData(PV1, PVE2, PVE3, \"Size (leader election)\",\n                      size_ldr, fun(V) -> safe_html_string(\"~.2f\", [V]) end),\n     renderGossipData(PV1, PVE2, PVE3, \"Size (key range)\",\n                      size_kr, fun(V) -> safe_html_string(\"~.2f\", [V]) end),\n     renderGossipData(PV1, PVE2, PVE3, \"Average load\",\n                      avgLoad, fun(V) -> safe_html_string(\"~.2f\", [V]) end),\n     renderGossipData(PV1, PVE2, PVE3, \"Maximum load\",\n                      maxLoad, fun(V) -> safe_html_string(\"~B\", [V]) end),\n     renderGossipData(PV1, PVE2, PVE3, \"Minimum load\",\n                      minLoad, fun(V) -> safe_html_string(\"~B\", [V]) end),\n     renderGossipData(PV1, PVE2, PVE3, \"Standard deviation of the load\",\n                      stddev, fun(V) -> safe_html_string(\"~.2f\", [V]) end)\n     ].\n\n-spec renderGossipHead(gossip_pv(), {boolean(), gossip_pv()}, {boolean(), gossip_pv()}) -> html_type().\nrenderGossipHead({P1, _V1}, {P2Exists, PV2}, {P3Exists, PV3}) ->\n    ValueP1 = P1,\n    {TD_V2_1, TD_V2_2} =\n        case P2Exists of\n            true ->\n                P2 = element(1, PV2),\n                {{td, [{align, \"left\"}], {strong, [], {font, [{color, \"white\"}], P2}}},\n                 {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Value\"}}}};\n            _ ->\n                {{td, [{bgcolor, \"#D5DEDE\"}, {align, \"left\"}], \"&nbsp;\"},\n                 {td, [{bgcolor, \"#D5DEDE\"}, {align, \"center\"}], \"&nbsp;\"}}\n              end,\n    {TD_V3_1, TD_V3_2} =\n        case P3Exists of\n            true ->\n                P3 = element(1, PV3),\n                {{td, [{align, \"left\"}], {strong, [], {font, [{color, \"white\"}], P3}}},\n                 {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Value\"}}}};\n            _ ->\n                {{td, [{bgcolor, \"#D5DEDE\"}, {align, \"left\"}], \"&nbsp;\"},\n                 {td, [{bgcolor, \"#D5DEDE\"}, {align, \"center\"}], \"&nbsp;\"}}\n              end,\n    {tr, [{bgcolor, \"#000099\"}],\n     [{td, [{align, \"left\"}], {strong, [], {font, [{color, \"white\"}], ValueP1}}},\n      {td, [{align, \"center\"}], {strong, [], {font, [{color, \"white\"}], \"Value\"}}},\n      {td, [{bgcolor, \"#D5DEDE\"}, {width, \"25pt\"}], \"&nbsp;\"},\n      TD_V2_1, TD_V2_2,\n      {td, [{bgcolor, \"#D5DEDE\"}, {width, \"25pt\"}], \"&nbsp;\"},\n      TD_V3_1, TD_V3_2\n     ]}.\n\n-spec renderGossipData(gossip_pv(), {boolean(), gossip_pv()}, {boolean(), gossip_pv()},\n                       string(), gossip_key(), fun((term()) -> string())) -> html_type().\nrenderGossipData({_P1, V1}, {P2Exists, PV2}, {P3Exists, PV3}, Name, Key, Fun) ->\n    ValueV1 = format_gossip_value(V1, Key, Fun),\n    {TD_V2_1, TD_V2_2} =\n        case P2Exists of\n            true ->\n                V2 = element(2, PV2),\n                {{td, [{align, \"left\"}], Name},\n                 {td, [{align, \"left\"}], format_gossip_value(V2, Key, Fun)}};\n            _ ->\n                {{td, [{bgcolor, \"#D5DEDE\"}, {align, \"left\"}], \"&nbsp;\"},\n                 {td, [{bgcolor, \"#D5DEDE\"}, {align, \"left\"}], \"&nbsp;\"}}\n        end,\n    {TD_V3_1, TD_V3_2} =\n        case P3Exists of\n            true ->\n                V3 = element(2, PV3),\n                {{td, [{align, \"left\"}], Name},\n                 {td, [{align, \"left\"}], format_gossip_value(V3, Key, Fun)}};\n            _ ->\n                {{td, [{bgcolor, \"#D5DEDE\"}, {align, \"left\"}], \"&nbsp;\"},\n                 {td, [{bgcolor, \"#D5DEDE\"}, {align, \"left\"}], \"&nbsp;\"}}\n        end,\n    {tr, [],\n     [{td, [{align, \"left\"}], Name},\n      {td, [{align, \"left\"}], ValueV1},\n      {td, [{bgcolor, \"#D5DEDE\"}, {width, \"25pt\"}], \"&nbsp;\"},\n      TD_V2_1, TD_V2_2,\n      {td, [{bgcolor, \"#D5DEDE\"}, {width, \"25pt\"}], \"&nbsp;\"},\n      TD_V3_1, TD_V3_2\n     ]}.\n\n-spec format_gossip_value(gossip_load:load_info(), Key::gossip_key(),\n                          fun((term()) -> string())) -> string().\nformat_gossip_value(Value, Key, Fun) ->\n    case gossip_load:load_info_get(Key, Value) of\n        unknown -> \"n/a\";\n        X       -> Fun(X)\n    end.\n\n\n%%%-----------------------------Monitoring----------------------------------\n\n-spec getMonitorClientData() -> html_type().\ngetMonitorClientData() ->\n    ClientMonitor = pid_groups:pid_of(clients_group, monitor),\n    {_CountD, CountPerSD, AvgMsD, MinMsD, MaxMsD, StddevMsD, _HistMsD} =\n        case statistics:getTimingMonitorStats(ClientMonitor, [{api_tx, 'req_list'}], list) of\n            []                           -> {[], [], [], [], [], [], []};\n            [{api_tx, 'req_list', Data}] -> Data\n        end,\n    AvgMinMaxMsD = lists:zipwith3(fun([Time, Avg], [Time, Min], [Time, Max]) ->\n                                          [Time, Avg, Avg - Min, Max - Avg]\n                                  end, AvgMsD, MinMsD, MaxMsD),\n\n    VMMonitor = pid_groups:pid_of(basic_services, monitor),\n    MemMonKeys = [{monitor_perf, X} || X <- [mem_total, mem_processes, mem_system,\n                                             mem_atom, mem_binary, mem_ets]],\n    case statistics:getGaugeMonitorStats(VMMonitor, MemMonKeys, list, 1024.0 * 1024.0) of\n        [] -> MemTotalD = MemProcD = MemSysD = MemAtomD = MemBinD = MemEtsD = [];\n        [{monitor_perf, mem_total, MemTotalD},\n         {monitor_perf, mem_processes, MemProcD},\n         {monitor_perf, mem_system, MemSysD},\n         {monitor_perf, mem_atom, MemAtomD},\n         {monitor_perf, mem_binary, MemBinD},\n         {monitor_perf, mem_ets, MemEtsD}] -> ok\n    end,\n\n    IOMonKeys = [{monitor_perf, X} || X <- [%rcv_count,\n                                            rcv_bytes, %send_count,\n                                            send_bytes]],\n    case statistics:getGaugeMonitorStats(VMMonitor, IOMonKeys, list, 1) of\n        [] -> %RcvCntD0 = SendCntD0 =\n            RcvBytesD0 = SendBytesD0 = [];\n        [%{monitor_perf, rcv_count, RcvCntD0},\n         {monitor_perf, rcv_bytes, RcvBytesD0},\n         %{monitor_perf, send_count, SendCntD0},\n         {monitor_perf, send_bytes, SendBytesD0}] -> ok\n    end,\n%%     RcvCntD = get_diff_data(lists:reverse(RcvCntD0)),\n    RcvBytesD = get_diff_data(lists:reverse(RcvBytesD0)),\n%%     SendCntD = get_diff_data(lists:reverse(SendCntD0)),\n    SendBytesD = get_diff_data(lists:reverse(SendBytesD0)),\n\n    DataStr =\n        lists:flatten(\n          [\"\\n\",\n%%            \"var count_data = \",           io_lib:format(\"~p\", [CountD]), \";\\n\",\n           \"var count_per_s_data = \",     io_lib:format(\"~p\", [CountPerSD]), \";\\n\",\n           \"var avg_min_max_ms_data = \",  io_lib:format(\"~p\", [AvgMinMaxMsD]), \";\\n\",\n%%            \"var avg_ms_data = \",          io_lib:format(\"~p\", [AvgMsD]), \";\\n\",\n%%            \"var min_ms_data = \",          io_lib:format(\"~p\", [MinMsD]), \";\\n\",\n%%            \"var max_ms_data = \",          io_lib:format(\"~p\", [MaxMsD]), \";\\n\",\n%%            \"var hist_ms_data = \",         io_lib:format(\"~p\", [HistMsD]), \";\\n\",\n           \"var stddev_ms_data = \",       io_lib:format(\"~p\", [StddevMsD]), \";\\n\",\n\n           \"var mem_total_data = \",       io_lib:format(\"~p\", [MemTotalD]), \";\\n\",\n           \"var mem_processes_data = \",   io_lib:format(\"~p\", [MemProcD]), \";\\n\",\n           \"var mem_system_data = \",      io_lib:format(\"~p\", [MemSysD]), \";\\n\",\n           \"var mem_atom_data = \",        io_lib:format(\"~p\", [MemAtomD]), \";\\n\",\n           \"var mem_binary_data = \",      io_lib:format(\"~p\", [MemBinD]), \";\\n\",\n           \"var mem_ets_ms_data = \",      io_lib:format(\"~p\", [MemEtsD]), \";\\n\"\n\n%%            \"var io_rcv_count_data = \",    io_lib:format(\"~p\", [RcvCntD]), \";\\n\",\n           \"var io_rcv_bytes_data = \",    io_lib:format(\"~p\", [RcvBytesD]), \";\\n\",\n%%            \"var io_send_count_data = \",   io_lib:format(\"~p\", [SendCntD]), \";\\n\",\n           \"var io_send_bytes_data = \",   io_lib:format(\"~p\", [SendBytesD]), \";\\n\"\n          ]),\n    {script, [{type, \"text/javascript\"}], DataStr}.\n\n%% @doc In a list containing [Time, Value] elements, compute the differences of\n%%      any two consecutive values and return a list of [Time, DiffToPrevious].\n%%      PreCond: values must increase monotonically!\n-spec get_diff_data([T]) -> [T].\nget_diff_data([]) -> [];\nget_diff_data([H | T]) -> lists:reverse([H | get_diff_data(H, T)]).\n\n-spec get_diff_data(Last::T, Rest::[T]) -> [T].\nget_diff_data([_TimeLast, _ValueLast], []) -> [];\nget_diff_data([_TimeLast, ValueLast], [Cur = [TimeCur, ValueCur] | Rest])\n  when ValueCur >= ValueLast ->\n    [[TimeCur, ValueCur - ValueLast] | get_diff_data(Cur, Rest)].\n\n-spec getMonitorRingData() -> [html_type()].\ngetMonitorRingData() ->\n    case pid_groups:pid_of(basic_services, monitor) of\n        failed ->\n            Prefix = {p, [], \"NOTE: no monitor_perf in this VM\"},\n            DataRR = DataLH = DataTX = {[], [], [], [], [], [], []}, ok;\n        Monitor ->\n            Prefix = [],\n            ReqKeys = [{monitor_perf, 'agg_read_read'}, {dht_node, 'agg_lookup_hops'}, {api_tx, 'agg_req_list'}],\n            case statistics:getTimingMonitorStats(Monitor, ReqKeys, list) of\n                [] -> DataRR = DataLH = DataTX = {[], [], [], [], [], [], []}, ok;\n                [{monitor_perf, 'agg_read_read', DataRR},\n                 {dht_node, 'agg_lookup_hops', DataLH},\n                 {api_tx, 'agg_req_list', DataTX}] -> ok\n            end\n    end,\n    {_RRCountD, _RRCountPerSD, RRAvgMsD, RRMinMsD, RRMaxMsD, RRStddevMsD, _RRHistMsD} = DataRR,\n    {_LHCountD, _LHCountPerSD, LHAvgCountD, LHMinCountD, LHMaxCountD, LHStddevCountD, _LHHistCountD} = DataLH,\n    {_TXCountD, TXCountPerSD, TXAvgCountD, TXMinCountD, TXMaxCountD, TXStddevCountD, _TXHistCountD} = DataTX,\n    RRAvgMinMaxMsD = lists:zipwith3(fun([Time, Avg], [Time, Min], [Time, Max]) ->\n                                            [Time, Avg, Avg - Min, Max - Avg]\n                                    end, RRAvgMsD, RRMinMsD, RRMaxMsD),\n    LHAvgMinMaxCountD = lists:zipwith3(fun([Time, Avg], [Time, Min], [Time, Max]) ->\n                                               [Time, Avg, Avg - Min, Max - Avg]\n                                       end, LHAvgCountD, LHMinCountD, LHMaxCountD),\n    TXAvgMinMaxCountD = lists:zipwith3(fun([Time, Avg], [Time, Min], [Time, Max]) ->\n                                               [Time, Avg, Avg - Min, Max - Avg]\n                                       end, TXAvgCountD, TXMinCountD, TXMaxCountD),\n    DataStr =\n        lists:flatten(\n          [\"\\n\",\n           \"var rr_avg_min_max_ms_data = \",    io_lib:format(\"~p\", [RRAvgMinMaxMsD]), \";\\n\",\n           \"var rr_stddev_ms_data = \",         io_lib:format(\"~p\", [RRStddevMsD]), \";\\n\",\n           \"var lh_avg_min_max_count_data = \", io_lib:format(\"~p\", [LHAvgMinMaxCountD]), \";\\n\",\n           \"var lh_stddev_count_data = \",      io_lib:format(\"~p\", [LHStddevCountD]), \";\\n\",\n           \"var tx_count_per_s_data = \",       io_lib:format(\"~p\", [TXCountPerSD]), \";\\n\",\n           \"var tx_avg_min_max_ms_data = \",    io_lib:format(\"~p\", [TXAvgMinMaxCountD]), \";\\n\",\n           \"var tx_stddev_ms_data = \",         io_lib:format(\"~p\", [TXStddevCountD]), \";\\n\"]),\n    lists:flatten([Prefix, {script, [{type, \"text/javascript\"}], DataStr}]).\n\n\n%%%-----------------------------Misc----------------------------------\n\ndead_node() ->\n    \"dead node?\".\n\n-spec get_indexed_pred_id(Node::node:node_type(),\n                          MyIndex::non_neg_integer() | string()) -> integer() | string().\nget_indexed_pred_id(Node, MyIndex) ->\n    NodeIndex = get_indexed_id_by_node(Node),\n    RingSize = get_ring_size(),\n    if NodeIndex =:= undefined orelse MyIndex =:= undefined -> dead_node();\n       true ->\n           case NodeIndex =:= MyIndex of\n               true -> 0;\n               _    ->\n                   ((NodeIndex - MyIndex + RingSize) rem RingSize) - RingSize\n           end\n    end.\n\n-spec get_indexed_succ_id(Node::node:node_type(),\n                          MyIndex::non_neg_integer() | string()) -> integer() | string().\nget_indexed_succ_id(Node, MyIndex) ->\n    NodeIndex = get_indexed_id_by_node(Node),\n    RingSize = get_ring_size(),\n    if NodeIndex =:= undefined orelse MyIndex =:= undefined -> dead_node();\n       true -> (NodeIndex - MyIndex + RingSize) rem RingSize\n    end.\n\n-spec get_flag(Hostname::node_details:hostname()) -> html_type().\nget_flag(Hostname) ->\n    Country = string:substr(Hostname, 1 + string:rchr(Hostname, $.)),\n    URL = case lists:member(Country, ?COUNTRIES) of\n              true  -> \"icons/\" ++ Country ++ \".gif\";\n              false -> \"icons/unknown.gif\"\n          end,\n    {img, [{src, URL}, {width, 26}, {height, 16}], []}.\n\n%% @doc Formats the data with the format string and escapes angle brackets with\n%%      their HTML counter parts so that content is not mis-interpreted as HTML\n%%      tags.\n-spec safe_html_string(io:format(), Data::[term()]) -> string().\nsafe_html_string(Format, Data) ->\n    safe_html_string(io_lib:format(Format, Data)).\n\n%% @doc Escapes angle brackets within the string with their HTML counter parts\n%%      so that content is not mis-interpreted as HTML tags.\n-spec safe_html_string(io_lib:chars()) -> string().\nsafe_html_string(String) ->\n    lists:flatten([case C of\n                       $< -> \"&lt;\";\n                       $> -> \"&gt;\";\n                       _ -> C\n                   end || C <- lists:flatten(String)]).\n\n%% @doc Pre-formats a string (useful for reading erlang terms)\n-spec html_pre(io:format(), Data::[term()]) -> string().\nhtml_pre(Format, Data) ->\n    \"<pre>\" ++ lists:flatten(io_lib:format(Format, Data)) ++ \"</pre>\".\n"
  },
  {
    "path": "src/wpool.erl",
    "content": "%% @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc worker pool implementation.\n%%      This gen_component handles arbitrary workloads for other processes. It\n%%      can be send a do_work message with a job specification and will return\n%%      the results of the computation or an error.\n%%      Jobs will run concurrently in seperate processes. The maximum number of\n%%      concurrent workers can be configured via the wpool_maxw setting in\n%%      scalaris.[local.]config.\n%%      when the maximum number of jobs is reached new jobs are queued and run\n%%      as soon as a running job finishes.\n%%\n%% @end\n%% @version $Id$\n-module(wpool).\n-author('fajerski@zib.de').\n-vsn('$Id$ ').\n\n-define(TRACE(X, Y), ok).\n%% -define(TRACE(X, Y), io:format(X, Y)).\n\n-behaviour(gen_component).\n\n-export([\n        start_link/2\n        , on/2\n        , init/1\n        ]).\n\n-include(\"scalaris.hrl\").\n\n-export_type([job/0]).\n\n-include(\"gen_component.hrl\").\n\n-type(mr_job() :: {mr_state:fun_term(),\n                   Data::db_ets:db(), Interval::intervals:interval()}).\n\n%% -type(generic_job() :: {erlanon | jsanon, binary(), [tuple()]}).\n\n-type(job() :: mr_job()).\n\n-type(message() :: {do_work, Source::comm:mypid(), job()} |\n                   %% TODO find better spec for Info\n                   {'DOWN', reference(), process, pid(), Info::any()} |\n                   {data, pid(), [tuple()]}).\n\n%% running jobs are defined by the pid the worker process has and the source of\n%% the job\n-type(avail_workers() :: [comm:mypid()]).\n-type(working() :: [{Worker::comm:mypid(), Source::comm:mypid()}]).\n\n%% a waiting job is defined by the source and the job spec\n-type(waiting_jobs() :: [{comm:mypid(), job()}]).\n\n-type(state() :: {avail_workers(), working(), waiting_jobs()}).\n\n-spec init([]) -> state().\ninit([]) ->\n    {[start_worker(X) || X <- lists:seq(1, config:read(wpool_maxw))],\n     [],\n     []}.\n\n-spec start_link(pid_groups:groupname(), tuple()) -> {ok, pid()}.\nstart_link(DHTNodeGroup, _Options) ->\n    ?TRACE(\"wpool: starting on node ~p~n\", [DHTNodeGroup]),\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [],\n                             [{pid_groups_join_as, DHTNodeGroup, wpool}]).\n\n-spec on(message(), state()) -> state().\n%% new job arrives...either start it right away or queue it\non({do_work, Source, Workload}, {[], Working, Waiting}) ->\n    {[], Working, lists:append(Waiting, [{Source, Workload}])};\non({do_work, Source, Workload}, {[Pid | R], Working, Waiting}) ->\n    comm:send_local(Pid, {work, self(), Workload}),\n    {R, [{Pid, Source} | Working], Waiting};\n\n%% worker threw an exception. send exception info back to origin\non({'DOWN', exc_subs, wpool_worker, Pid, Reason}, {Avail, Working, Waiting}) ->\n    %% normal termination\n    ?TRACE(\"worker ~p threw exception ~p~n\", [Pid, Reason]),\n    {value, {Pid, Source}, NewWorking} = lists:keytake(Pid, 1, Working),\n    comm:send(Source, {worker_died, Reason}),\n    worker_finished({Avail, NewWorking, Waiting}, Pid);\n\n%% worker sends results; forward to source\n%% TODO hsould be better if worker sends directly to source and just informs\n%% wpool that it is available for new work\non({data, Pid, Data}, {Avail, Working, Waiting}) ->\n    ?TRACE(\"wpool: received data from ~p:~n~p~n\",\n            [Pid, Data]),\n    %% send results to source\n    {value, {_Pid, Source}, NewWorking} = lists:keytake(Pid, 1, Working),\n    comm:send(Source, {work_done, Data}),\n    worker_finished({Avail, NewWorking, Waiting}, Pid);\n\non(Msg, State) ->\n    io:format(\"~200p~nwpool: unknown message~n\", [Msg]),\n    State.\n\n%% @doc start a worker\n%%     starts worker under the supervisor sup_wpool\n-spec start_worker(pos_integer()) -> pid().\nstart_worker(ID) ->\n    Sup = pid_groups:get_my(sup_wpool),\n    case supervisor:start_child(Sup, {{wpool_w, ID}, {wpool_worker, start_link,\n                                                [{exception_subscription,\n                                                  [comm:this()]}]}, permanent,\n                                            brutal_kill, worker, []}) of\n        {ok, Pid} ->\n            Pid;\n        {ok, Pid, _Info} ->\n            Pid\n    end.\n\n-spec worker_finished(state(), comm:mypid()) -> state().\nworker_finished({Avail, Working, []}, Pid) ->\n    {[Pid | Avail], Working, []};\nworker_finished({Avail, Working, [{Source, Workload} | MoreWaiting]}, Pid) ->\n    comm:send_local(Pid, {work, self(), Workload}),\n    {Avail, [{Pid, Source} | Working], MoreWaiting}.\n"
  },
  {
    "path": "src/wpool_worker.erl",
    "content": "%% @copyright 2007-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc worker pool worker implementation.\n%%\n%% @end\n%% @version $Id$\n-module(wpool_worker).\n-author('fajerski@zib.de').\n-vsn('$Id$ ').\n\n-define(TRACE(X, Y), ok).\n%% -define(TRACE(X, Y), io:format(X, Y)).\n\n-behaviour(gen_component).\n\n-export([\n        start_link/1\n        , on/2\n        , init/1\n        ]).\n\n-include(\"scalaris.hrl\").\n-include(\"gen_component.hrl\").\n\n-type(message() :: {work, Source::comm:erl_local_pid(), wpool:job()}).\n\n-type(state() :: {}).\n\n-spec init([]) -> state().\ninit([]) ->\n    {}.\n\n-spec start_link(tuple()) -> {ok, pid()}.\nstart_link(Options) ->\n    ?TRACE(\"~p wpool_worker: starting...~n\", [self()]),\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, [], [Options]).\n\n-spec on(message(), state()) -> state().\non({work, Source, Workload}, State) ->\n    work(Source, Workload),\n    State;\n\non(_Msg, State) ->\n    ?TRACE(\"~200p~nwpool_worker: unknown message~n\", [Msg]),\n    State.\n\n\n%% @doc do the actual work.\n%%      executes Job and returns results to the local wpool. wpool associates\n%%      the worker pid to the jobs client and knows where the results go.\n-spec work(comm:erl_local_pid(), wpool:job()) -> ok.\nwork(Source, {{map, erlanon, Fun}, Data, Interval}) ->\n    ?TRACE(\"worker: should apply map ~p to ~p in ~p~n\", [Fun, Data, Interval]),\n    Results =\n    lists:foldl(fun(SimpleInterval, Acc1) ->\n                        db_ets:foldl(Data,\n                                     fun(HK, Acc) ->\n                                             {_HK, K, V} = db_ets:get(Data, HK),\n                                             ?TRACE(\"mapping ~p~n\", [{K, V}]),\n                                             Res = apply_erl(Fun, {K, V}),\n                                             mr_state:accumulate_data(Res, Acc)\n                                     end,\n                                     Acc1, SimpleInterval)\n                end,\n                [],\n                intervals:get_simple_intervals(Interval)),\n    return(Source, Results);\nwork(Source, {{reduce, erlanon, Fun}, Data, Interval}) ->\n    ?TRACE(\"~p worker: should apply reduce ~p~n    in ~p~n\",\n           [self(), ets:tab2list(Data), Interval]),\n    Args = lists:foldl(fun(SimpleInterval, Acc1) ->\n                               db_ets:foldl(Data,\n                                            fun(HK, AccFold) ->\n                                                    {_HK, K, V} =\n                                                    db_ets:get(Data, HK),\n                                                    [{K, V} | AccFold]\n                                            end,\n                                         Acc1,\n                                         SimpleInterval)\n                       end,\n                       [],\n                       intervals:get_simple_intervals(Interval)),\n    ?TRACE(\"~p reducing ~p~n\", [self(), Args]),\n    Res = apply_erl(Fun, Args),\n    return(Source, [{?RT:hash_key(K), K, V} || {K, V} <- Res]);\n\nwork(Source, {{map, jsanon, Fun}, Data, Interval}) ->\n    %% ?TRACE(\"worker: should apply ~p to ~p~n\", [FunBin, Data]),\n    {ok, VM} = js_driver:new(),\n    Results =\n    lists:foldl(fun(SimpleInterval, Acc1) ->\n                        db_ets:foldl(Data,\n                                     fun(HK, Acc) ->\n                                             {_HK, K, V} = db_ets:get(Data, HK),\n                                             Res = apply_js(Fun, [{K, V}], VM),\n                                             mr_state:accumulate_data(Res, Acc)\n                                     end,\n                                     Acc1, SimpleInterval)\n                end,\n                [],\n                intervals:get_simple_intervals(Interval)),\n    return(Source, Results);\nwork(Source, {{reduce, jsanon, Fun}, Data, Interval}) ->\n    {ok, VM} = js_driver:new(),\n    Args = lists:foldl(fun(SimpleInterval, Acc1) ->\n                               db_ets:foldl(Data,\n                                            fun(HK, AccFold) ->\n                                                    {_HK, K, V} =\n                                                    db_ets:get(Data, HK),\n                                                    [{K, V} | AccFold]\n                                            end,\n                                         Acc1,\n                                         SimpleInterval)\n                       end,\n                       [],\n                       intervals:get_simple_intervals(Interval)),\n    Res = apply_js(Fun, [Args], VM),\n    return(Source, [{?RT:hash_key(K), K, V} || {K, V} <- Res]).\n%% TODO add generic work loads\n\n%% @doc applies Fun with Args.\n-spec apply_erl(Fun::fun((Arg::term()) -> Res::A), Args::term()) -> A.\napply_erl(Fun, Data) ->\n    Fun(Data).\n\n%% @doc applies Fun with Args withing JSVM.\n-spec apply_js(Fun::binary(), Args::[term()], JSVM::port()) -> term().\napply_js(FunBin, Data, VM) ->\n    AutoJS = define_auto_js(FunBin, Data),\n    {ok, Result} = js:eval(VM, AutoJS),\n    decode(Result).\n\n%% @doc create a self calling JS function.\n%%      takes a anonymous JS function (as a binary string) and a list of\n%%      arguments and returns a self calling JS (`function(arg) {...}(args)')\n%%      function as a binary string.\n-spec define_auto_js(Fun::binary(), [term()]) -> binary().\ndefine_auto_js(FunBin, Args) ->\n    EncodedArgs = encode_args(Args),\n    iolist_to_binary([FunBin, \"(\", EncodedArgs, \")\"]).\n\n%% @doc build argument list for JS function.\n%%      takes a list of terms and returns an iolist of the encoded (mochijson2)\n%%      terms with commas in between.\n-spec encode_args([term()]) -> iolist().\nencode_args([]) ->\n    [];\nencode_args([H]) ->\n    js_mochijson2:encode(encode(H));\nencode_args([H | T]) ->\n    [js_mochijson2:encode(encode(H)), \",\" | encode_args(T)].\n\n-spec encode(term()) -> term().\nencode({K, V}) ->\n    {struct, [{encode(list_to_binary(K)), encode(V)}]};\nencode([{_K, _V} | _T] = KVList) ->\n    %% {struct, [{encode(K), encode(V)} || {K, V} <- KVList]};\n    %% [{struct, [{key, encode(K)}, {value, encode(V)}]} || {K, V} <- KVList];\n    Fun = fun({K, V}, AccIn) ->\n                  %% strings need to be encoded as binaries\n                  %% keys are always strings; if value is a string it needs to\n                  %% be passed as a binary\n                  EnK = encode(list_to_binary(K)), EnV = encode(V),\n                  [{EnK, EnV} | AccIn]\n          end,\n    Data = lists:foldl(Fun, [], KVList),\n    {struct, Data};\n%% this poses a problem with lists as values. if value is a list it gets encoded\n%% to a binary and does not get encoded as an array.\n%% encode(String) when is_list(String) ->\n%%     list_to_binary(String);\nencode(X) -> X.\n\n-spec decode(term()) -> term().\ndecode({struct, [{K, V}]}) ->\n    {decode(K), decode(V)};\ndecode({struct, [{_K, _V} | _T] = KVList}) ->\n    [{decode(K), decode(V)} || {K, V} <- KVList];\ndecode([{struct, _KV} | _T] = List) ->\n    [{decode(K), decode(V)} || {struct, [{K, V}]} <- List];\ndecode(BinString) when is_binary(BinString) ->\n    binary_to_list(BinString);\ndecode(X) -> X.\n\n%% send results back to wpool\n-spec return(comm:erl_local_pid(), [term()]) -> ok.\nreturn(Source, Data) ->\n    comm:send_local(Source, {data, self(), Data}).\n\n"
  },
  {
    "path": "test/.gitignore",
    "content": "/*.beam\n"
  },
  {
    "path": "test/0001-rt_chord-reduce-key-size-to-16-bits-for-better-debug.patch",
    "content": "From f79c8c8ce5ba28952d7f9406cbdf39e1aca5c197 Mon Sep 17 00:00:00 2001\nFrom: Nico Kruber <kruber@zib.de>\nDate: Sat, 9 Nov 2013 16:11:10 +0100\nSubject: [PATCH] rt_chord: reduce key size to 16 bits for better debugging\n\n---\n include/scalaris.hrl |  4 ++--\n src/rt_chord.erl     | 14 +++++++-------\n 2 files changed, 9 insertions(+), 9 deletions(-)\n\ndiff --git a/include/scalaris.hrl b/include/scalaris.hrl\nindex 17fb0d2..ced216e 100644\n--- a/include/scalaris.hrl\n+++ b/include/scalaris.hrl\n@@ -31,8 +31,8 @@\n -define(MINUS_INFINITY, 0).\n -define(MINUS_INFINITY_TYPE, 0).\n % first invalid key:\n--define(PLUS_INFINITY, 16#100000000000000000000000000000000).\n--define(PLUS_INFINITY_TYPE, 16#100000000000000000000000000000000).\n+-define(PLUS_INFINITY, 16#10000).\n+-define(PLUS_INFINITY_TYPE, 16#10000).\n \n %%Simple routingtable\n %-define(RT, rt_simple).\ndiff --git a/src/rt_chord.erl b/src/rt_chord.erl\nindex a3260df..cb32c19 100644\n--- a/src/rt_chord.erl\n+++ b/src/rt_chord.erl\n@@ -64,7 +64,7 @@ hash_key(Key) -> hash_key_(Key).\n hash_key_(Key) when not is_binary(Key) ->\n     hash_key_(client_key_to_binary(Key));\n hash_key_(Key) ->\n-    <<N:128>> = ?CRYPTO_MD5(Key),\n+    <<N:16,_:112>> = ?CRYPTO_MD5(Key),\n     N.\n \n %% @doc Generates a random node id, i.e. a random 128-bit number, based on the\n@@ -117,14 +117,14 @@ get_size(RT) ->\n \n %% @doc Keep a key in the address space. See n/0.\n -spec normalize(non_neg_integer()) -> key_t().\n-normalize(Key) -> Key band 16#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.\n+normalize(Key) -> Key band 16#FFFF.\n \n %% @doc Returns the size of the address space.\n -spec n() -> integer().\n n() -> n_().\n %% @doc Helper for n/0 to make dialyzer happy with internal use of n/0.\n--spec n_() -> 16#100000000000000000000000000000000.\n-n_() -> 16#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF + 1.\n+-spec n_() -> 16#10000.\n+n_() -> 16#FFFF + 1.\n \n %% @doc Gets the number of keys in the interval (Begin, End]. In the special\n %%      case of Begin==End, the whole key range as specified by n/0 is returned.\n@@ -179,9 +179,9 @@ get_random_in_interval2('[', L, R, ']') ->\n -spec get_replica_keys(key()) -> [key()].\n get_replica_keys(Key) ->\n     [Key,\n-     Key bxor 16#40000000000000000000000000000000,\n-     Key bxor 16#80000000000000000000000000000000,\n-     Key bxor 16#C0000000000000000000000000000000\n+     Key bxor 16#4000,\n+     Key bxor 16#8000,\n+     Key bxor 16#C000\n     ].\n \n -spec get_key_segment(key()) -> pos_integer().\n-- \n1.8.1.4\n\n"
  },
  {
    "path": "test/all_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{suites, tests, all}.\n{skip_suites, tests, [performance_SUITE], \"ignore benchmarks\"}.\n{skip_suites, tests, [scalaris_cth_SUITE], \"only for debugging scalaris_cth.erl\"}.\n"
  },
  {
    "path": "test/all_with_cover_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n{cover, \"scalaris.coverspec\"}.\n\n{suites, tests, all}.\n{skip_suites, tests, [performance_SUITE], \"ignore benchmarks\"}.\n{skip_suites, tests, [scalaris_cth_SUITE], \"only for debugging scalaris_cth.erl\"}.\n"
  },
  {
    "path": "test/api_json_SUITE.erl",
    "content": "%  @copyright 2008-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Unit tests for the JSON-API\n%% @end\n%% @version $Id$\n-module(api_json_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id$').\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n\nall()   -> [get_node_info, get_node_performance, get_service_info, get_service_performance].\nsuite() -> [ {timetrap, {seconds, 120}} ].\n\ninit_per_suite(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(4, [{config, [{log_path, PrivDir}]}]),\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\nget_node_info(_Config) ->\n     {struct, [{status, \"ok\"}, {value, {struct, Value}}]} = api_json:handler(get_node_info, []),\n    ?assert(erlang:is_list(Value)),\n    ok.\n\nget_node_performance(_Config) ->\n     {struct, [{status, \"ok\"}, {value, {struct, Value}}]} = api_json:handler(get_node_performance, []),\n    ?assert(erlang:is_list(Value)),\n    ok.\n\nget_service_info(_Config) ->\n     {struct, [{status, \"ok\"}, {value, {struct, Value}}]} = api_json:handler(get_service_info, []),\n    ?assert(erlang:is_list(Value)),\n    ok.\n\nget_service_performance(_Config) ->\n     {struct, [{status, \"ok\"}, {value, {struct, Value}}]} = api_json:handler(get_service_performance, []),\n    ?assert(erlang:is_list(Value)),\n    ok.\n"
  },
  {
    "path": "test/api_tx_SUITE.erl",
    "content": "%% @copyright 2012-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @version $Id$\n-module(api_tx_SUITE).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\n%% disable proto_sched statements for this suite\n-define(proto_sched(Action), ok).\n-include(\"api_tx_SUITE.hrl\").\n\ngroups() ->\n    [%% implementation in api_tx_SUITE.hrl\n     %% (shared with api_tx_proto_sched_SUITE.erl)\n     {proto_sched_ready, [sequence],\n      proto_sched_ready_tests()},\n     %% implementation below\n     {not_proto_sched_ready, [sequence],\n      [ write_test_race_mult_rings, %% uses timings?\n        read_write_2old,            %% uses wait_for_dht_entries/1\n        read_write_2old_locked,     %% uses wait_for_dht_entries/1\n        read_write_notfound,        %% uses wait_for_dht_entries/1\n        tester_encode_decode,       %% no messages send (does not make sense with proto_sched)\n        tester_read_not_existing,\n        tester_write_read_not_existing,\n        tester_write_read,\n        tester_add_del_on_list_not_existing,\n        tester_add_del_on_list,\n        tester_add_del_on_list_maybe_invalid,\n        tester_add_on_nr_not_existing,\n        tester_add_on_nr,\n        tester_add_on_nr_maybe_invalid,\n        tester_test_and_set_not_existing,\n        tester_test_and_set,\n        tester_tlog_add_del_on_list_not_existing,\n        tester_tlog_add_del_on_list,\n        tester_tlog_add_del_on_list_maybe_invalid,\n        tester_tlog_add_on_nr_not_existing,\n        tester_tlog_add_on_nr,\n        tester_tlog_add_on_nr_maybe_invalid,\n        tester_tlog_test_and_set_not_existing,\n        tester_tlog_test_and_set,\n        tester_random_from_list,\n        tester_req_list,\n        tester_req_list_on_same_key,\n        req_list_parallelism\n      ]}].\n\nall()   -> [\n            {group, proto_sched_ready},\n            {group, not_proto_sched_ready}\n           ].\n\nsuite() -> [ {timetrap, {seconds, 200}} ].\n\ninit_per_testcase(TestCase, Config) ->\n    case TestCase of\n        write_test_race_mult_rings -> %% this case creates its own ring\n            ok;\n        tester_encode_decode -> %% this case does not need a ring\n            ok;\n        _ ->\n            {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n            unittest_helper:make_ring(4, [{config, [{log_path, PrivDir}]}]),\n            ok\n    end,\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n-spec adapt_tx_runs(N::pos_integer()) -> pos_integer().\nadapt_tx_runs(N) -> N.\n\n%% @doc Test for api_tx:write taking at least 2s after stopping a ring\n%%      and starting a new one.\nwrite_test_race_mult_rings(Config) ->\n    % first ring:\n    write_test(Config),\n    % second ring and more:\n    write_test(Config),\n    write_test(Config),\n    write_test(Config),\n    write_test(Config),\n    write_test(Config),\n    write_test(Config),\n    write_test(Config).\n\n-spec write_test(Config::[tuple()]) -> true.\nwrite_test(Config) ->\n    OldRegistered = erlang:registered(),\n    OldProcesses = unittest_helper:get_processes(),\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(1, [{config, [{log_path, PrivDir}, {monitor_perf_interval, 0}]}]),\n    Self = self(),\n    BenchPid1 = erlang:spawn(fun() ->\n                                     {Time, _} = util:tc(api_tx, write, [\"1\", 1]),\n                                     comm:send_local(Self, {time, Time}),\n                                     ct:pal(\"~.0pus~n\", [Time])\n                             end),\n    receive {time, FirstWriteTime} -> ok\n    end,\n    util:wait_for_process_to_die(BenchPid1),\n    BenchPid2 = erlang:spawn(fun() ->\n                                     {Time, _} = util:tc(api_tx, write, [\"2\", 2]),\n                                     comm:send_local(Self, {time, Time}),\n                                     ct:pal(\"~.0pus~n\", [Time])\n                             end),\n    receive {time, SecondWriteTime} -> ok\n    end,\n    util:wait_for_process_to_die(BenchPid2),\n    unittest_helper:check_ring_load(config:read(replication_factor) * 2),\n    unittest_helper:check_ring_data(),\n    unittest_helper:stop_ring(),\n%%     randoms:stop(), %doesn't matter\n    _ = inets:stop(),\n    unittest_helper:kill_new_processes(OldProcesses),\n    {_, _, OnlyNewReg} =\n        util:split_unique(OldRegistered, erlang:registered()),\n    ct:pal(\"NewReg: ~.0p~n\", [OnlyNewReg]),\n    ?equals_pattern_w_note(FirstWriteTime, X when X =< 1000000,\n       \"We need more than a second to become operational?!\"),\n    ?equals_pattern(SecondWriteTime, X when X =< 1000000).\n\n-spec read_write_2old(Config::[tuple()]) -> ok.\nread_write_2old(_Config) ->\n    Key = \"read_write_2old_a\",\n    GSelf = comm:make_global(self()),\n    ?equals_w_note(api_tx:write(Key, 1), {ok}, \"write_1_a\"),\n    R = config:read(replication_factor),\n    wait_for_dht_entries(R),\n    RKeys = ?RT:get_replica_keys(?RT:hash_key(Key)),\n    MDeny = quorum:majority_for_deny(R),\n    DeleteKeys = lists:sublist(RKeys, MDeny),\n    DHTNodes = pid_groups:find_all(dht_node),\n    _ = [comm:send_local(DhtNode, {delete_keys, GSelf, DeleteKeys}) || DhtNode <- DHTNodes ],\n    _ = [ receive {delete_keys_reply} -> ok end || _ <- DHTNodes ],\n\n    ?equals(api_tx:write(Key, 2), {ok}),\n    ok.\n\n-spec read_write_2old_locked(Config::[tuple()]) -> ok.\nread_write_2old_locked(_Config) ->\n    Key = \"read_write_2old_a\",\n    GSelf = comm:make_global(self()),\n    ?equals_w_note(api_tx:write(Key, 1), {ok}, \"write_1_a\"),\n    R = config:read(replication_factor),\n    wait_for_dht_entries(R),\n    RKeys = ?RT:get_replica_keys(?RT:hash_key(Key)),\n    MDeny = quorum:majority_for_deny(R),\n    ModKeys = lists:sublist(RKeys, MDeny),\n\n    % get ModKeys entries\n    ModEntries = [ begin\n                    api_dht_raw:unreliable_lookup(X, {get_key_entry, GSelf, X}),\n                    receive {get_key_entry_reply, Entry} ->\n                            ?assert_w_note(not db_entry:is_empty(Entry), io_lib:format(\"~p\", [Entry])),\n                            Entry\n                    end\n                end || X <- ModKeys ],\n    OldVersion = db_entry:get_version(erlang:hd(ModEntries)),\n    %% all entries have same version\n    ?assert(lists:all(fun(E) -> db_entry:get_version(E) =:= OldVersion end, ModEntries)),\n\n    % write new value\n    ?equals_w_note(api_tx:write(Key, 2), {ok}, \"write_2_a\"),\n    util:wait_for(\n      fun() ->\n              {Status, Values} = api_dht_raw:range_read(0, 0),\n              Status =:= ok andalso\n                  lists:all(fun(E) ->\n                                    db_entry:get_version(E) =:= (OldVersion + 1)\n                            end, Values)\n      end),\n\n    % set majority for Deny outdated, locked entries:\n    [ begin\n          EntryL = db_entry:set_writelock(X, OldVersion - 1),\n          api_dht_raw:unreliable_lookup(db_entry:get_key(EntryL), {set_key_entry, GSelf, EntryL}),\n          receive {set_key_entry_reply, EntryL} -> ok end\n      end || X <- ModEntries ],\n\n    % now try to write\n    ?equals_w_note(api_tx:write(Key, 3), {ok}, \"write_3_a\"),\n    ok.\n\n-spec read_write_notfound(Config::[tuple()]) -> ok.\nread_write_notfound(_Config) ->\n    Key = \"read_write_notfound_test\",\n    _ = [read_write_notfound_test(Key ++ lists:flatten(io_lib:format(\"_~p_~p\", [X, M])), X, M)\n           || X <- [none | lists:seq(1,config:read(replication_factor))],\n              M <- [single, req_list]],\n    ok.\n\n-spec read_write_notfound_test(Key::client_key(), NthKeyToExclude::1..4 | none, Mode::single | req_list) -> ok.\nread_write_notfound_test(Key, NthKeyToExclude, Mode) ->\n    R = config:read(replication_factor),\n    RKeys = ?RT:get_replica_keys(?RT:hash_key(Key)),\n    MDeny = quorum:minority(R),\n    DelKeys = lists:sublist(RKeys, MDeny),\n\n    Note = io_lib:format(\"Key: ~p, Hashed: ~p, Excl.: ~p, Mode: ~p\",\n                         [Key, RKeys, NthKeyToExclude, Mode]),\n    % init\n    ?equals_w_note(api_tx:write(Key, 1), {ok}, Note ++ \" (write_0_a)\"),\n    wait_for_dht_entries(RKeys),\n    _ = [begin\n             comm:send_local(DhtNode, {delete_keys, comm:make_global(self()), DelKeys}),\n             receive {delete_keys_reply} -> ok end\n         end || DhtNode <- pid_groups:find_all(dht_node)],\n\n    % test\n    case NthKeyToExclude of\n        none -> ok;\n        _    ->\n            HashedKeyToExclude = lists:nth(NthKeyToExclude, RKeys),\n            drop_read_op_on_key(HashedKeyToExclude)\n    end,\n    case Mode of\n        single ->\n            ct:pal(\"read ~p\", [Key]),\n            {T1, R1} = api_tx:read(api_tx:new_tlog(), Key),\n            ct:pal(\"read result ~p\", [{T1, R1}]),\n            ct:pal(\"write ~p\", [T1]),\n            {T2, R2} = api_tx:write(T1, Key, 2),\n            ct:pal(\"write result ~p\", [{T2, R2}]),\n            ct:pal(\"commit ~p\", [T2]),\n            R3 = api_tx:commit(T2),\n            ct:pal(\"commit result ~p\", [R3]),\n            ok;\n        req_list ->\n            ct:pal(\"req_list\"),\n            {_T1, [R1, R2, R3]} = api_tx:req_list(api_tx:new_tlog(), [{read, Key}, {write, Key, 2}, {commit}]),\n            ok\n    end,\n\n    ?equals_w_note(R2, {ok}, Note ++ \" (write result)\"),\n    % the following should be true but is not at the moment:\n    case R1 of\n        {fail, not_found} ->\n            ?equals_pattern_w_note(R3, {fail, abort, _}, Note ++ \" (commit result)\");\n        {ok, 1} ->\n            ?equals_pattern_w_note(R3, {ok}, Note ++ \" (commit result)\")\n    end,\n\n    % cleanup\n    case NthKeyToExclude of\n        none -> ok;\n        _    ->\n            HKeyToExclude = lists:nth(NthKeyToExclude, RKeys),\n            stop_drop_read_op_on_key(HKeyToExclude)\n    end,\n    ok.\n\ndrop_read_op_on_key(HashedKey) ->\n    ct:pal(\"Silencing key ~p~n\", [HashedKey]),\n    Self = self(),\n    SkipHashedKeyFun =\n        fun (Message, _State) ->\n                 case Message of\n                     {?read_op, _Source_PID, _SourceId, HashedKey, _Op} ->\n                         ct:pal(\"Detected read, dropping it ~p, key ~p~n\",\n                                [self(), HashedKey]),\n                         comm:send_local(Self, {drop_read_op_on_key, HashedKey, done}),\n                         drop_single;\n                     {?read_op, _Source_PID, _SourceId, HashedKey2, _Op} ->\n                         ct:pal(\"Detected read ~p, key ~p~n\",\n                                [self(), HashedKey2]),\n                         false;\n                     _ ->\n%%                          ct:pal(\"Let pass ~p~n\", [Message]),\n                         false\n                 end\n        end,\n\n    _ = [gen_component:bp_set_cond(DhtNode, SkipHashedKeyFun, drop_read_op_on_key)\n        || DhtNode <- pid_groups:find_all(dht_node)],\n    ok.\n\nstop_drop_read_op_on_key(HashedKey) ->\n    ct:pal(\"Reactivating ~p~n\", [HashedKey]),\n    _ = [gen_component:bp_del(DhtNode, drop_read_op_on_key)\n        || DhtNode <- pid_groups:find_all(dht_node)],\n    cleanup_drop_read_op_on_key(HashedKey).\n\ncleanup_drop_read_op_on_key(HashedKey) ->\n    receive {drop_read_op_on_key, HashedKey, done} -> cleanup_drop_read_op_on_key(HashedKey)\n    after 0 -> ok end.\n\n-spec prop_encode_decode(Value::client_value()) -> true.\nprop_encode_decode(Value) ->\n    Value =:= rdht_tx:decode_value(rdht_tx:encode_value(Value)).\n\ntester_encode_decode(_Config) ->\n    tester:test(?MODULE, prop_encode_decode, 1, 10000).\n\n-spec prop_read_not_existing(Key::client_key()) -> true.\nprop_read_not_existing(Key) ->\n    case api_tx:read(Key) of\n        {fail, not_found} -> true;\n        {ok, _Value} -> true % may happen as we do not clear the ring after every op\n    end.\n\ntester_read_not_existing(_Config) ->\n    tester:test(?MODULE, prop_read_not_existing, 1, 10000).\n\n-spec prop_write_read_not_existing(Key::client_key(), Value::client_value()) -> true | no_return().\nprop_write_read_not_existing(Key, Value) ->\n    ?equals(api_tx:write(Key, Value), {ok}),\n    ?equals(api_tx:read(Key), {ok, Value}).\n\ntester_write_read_not_existing(_Config) ->\n    tester:test(?MODULE, prop_write_read_not_existing, 2, 5000).\n\n-spec prop_write_read(Key::client_key(), Value1::client_value(), Value2::client_value()) -> true | no_return().\nprop_write_read(Key, Value1, Value2) ->\n    ?equals(api_tx:write(Key, Value1), {ok}),\n    ?equals(api_tx:write(Key, Value2), {ok}),\n    ?equals(api_tx:read(Key), {ok, Value2}).\n\ntester_write_read(_Config) ->\n    tester:test(?MODULE, prop_write_read, 3, 5000).\n\n-spec prop_add_del_on_list2(Key::client_key(), Initial::client_value(), OldExists::boolean(), ToAdd::client_value(), ToRemove::client_value()) -> true | no_return().\nprop_add_del_on_list2(Key, Initial, OldExists, ToAdd, ToRemove) ->\n    if (not erlang:is_list(Initial)) orelse\n           (not erlang:is_list(ToAdd)) orelse\n           (not erlang:is_list(ToRemove)) ->\n           ?equals(api_tx:add_del_on_list(Key, ToAdd, ToRemove), {fail, not_a_list}),\n           Result = api_tx:read(Key),\n           if OldExists -> ?equals(Result, {ok, Initial});\n              true      -> ?equals(Result, {fail, not_found})\n           end;\n       true ->\n           ?equals(api_tx:add_del_on_list(Key, ToAdd, ToRemove), {ok}),\n           Result = api_tx:read(Key),\n           ?equals_pattern(Result, {ok, _List}),\n           {ok, List} = Result,\n           SortedList = lists:sort(fun util:'=:<'/2, List),\n           ?equals(SortedList, lists:sort(fun util:'=:<'/2, util:minus_first(lists:append(Initial, ToAdd), ToRemove)))\n    end.\n\n-spec prop_add_del_on_list_not_existing(Key::client_key(), ToAdd::[client_value()], ToRemove::[client_value()]) -> true | no_return().\nprop_add_del_on_list_not_existing(Key, ToAdd, ToRemove) ->\n    case api_tx:read(Key) of\n        {ok, OldValue} -> OldExists = true;\n        _ -> OldValue = [], OldExists = false\n    end,\n    prop_add_del_on_list2(Key, OldValue, OldExists, ToAdd, ToRemove).\n\ntester_add_del_on_list_not_existing(_Config) ->\n    tester:test(?MODULE, prop_add_del_on_list_not_existing, 3, 5000).\n\n-spec prop_add_del_on_list(Key::client_key(), Initial::client_value(), ToAdd::[client_value()], ToRemove::[client_value()]) -> true | no_return().\nprop_add_del_on_list(Key, Initial, ToAdd, ToRemove) ->\n    ?equals(api_tx:write(Key, Initial), {ok}),\n    prop_add_del_on_list2(Key, Initial, true, ToAdd, ToRemove).\n\ntester_add_del_on_list(_Config) ->\n    tester:test(?MODULE, prop_add_del_on_list, 4, 5000).\n\n-spec prop_add_del_on_list_maybe_invalid(Key::client_key(), Initial::client_value(), ToAdd::client_value(), ToRemove::client_value()) -> true | no_return().\nprop_add_del_on_list_maybe_invalid(Key, Initial, ToAdd, ToRemove) ->\n    ?equals(api_tx:write(Key, Initial), {ok}),\n    prop_add_del_on_list2(Key, Initial, true, ToAdd, ToRemove).\n\ntester_add_del_on_list_maybe_invalid(_Config) ->\n    tester:test(?MODULE, prop_add_del_on_list_maybe_invalid, 4, 5000).\n\n-spec prop_add_on_nr2(Key::client_key(), Existing::boolean(), Initial::client_value(), ToAdd::client_value()) -> true | no_return().\nprop_add_on_nr2(Key, Existing, Initial, ToAdd) ->\n    if (not erlang:is_number(Initial)) orelse\n           (not erlang:is_number(ToAdd)) ->\n           ?equals(api_tx:add_on_nr(Key, ToAdd), {fail, not_a_number}),\n           Result = api_tx:read(Key),\n           if Existing -> ?equals(Result, {ok, Initial});\n              true     -> ?equals(Result, {fail, not_found})\n           end;\n       true ->\n           ?equals(api_tx:add_on_nr(Key, ToAdd), {ok}),\n           Result = api_tx:read(Key),\n           ?equals_pattern(Result, {ok, _Number}),\n           {ok, Number} = Result,\n           case Existing of\n               false -> ?equals(Number, ToAdd);\n               % note: Initial+ToAdd could be float when Initial is not and thus Number is neither\n               true when ToAdd == 0 -> ?equals(Number, Initial);\n               _     -> ?equals(Number, (Initial + ToAdd))\n           end\n    end.\n\n-spec prop_add_on_nr_not_existing(Key::client_key(), ToAdd::number()) -> true | no_return().\nprop_add_on_nr_not_existing(Key, ToAdd) ->\n    {Existing, OldValue} = case api_tx:read(Key) of\n                               {ok, Value} -> {true, Value};\n                               _ -> {false, 0}\n                           end,\n    prop_add_on_nr2(Key, Existing, OldValue, ToAdd).\n\ntester_add_on_nr_not_existing(_Config) ->\n    tester:test(?MODULE, prop_add_on_nr_not_existing, 2, 5000).\n\n-spec prop_add_on_nr(Key::client_key(), Initial::client_value(), ToAdd::number()) -> true | no_return().\nprop_add_on_nr(Key, Initial, ToAdd) ->\n    ?equals(api_tx:write(Key, Initial), {ok}),\n    prop_add_on_nr2(Key, true, Initial, ToAdd).\n\ntester_add_on_nr(_Config) ->\n    tester:test(?MODULE, prop_add_on_nr, 3, 5000).\n\n-spec prop_add_on_nr_maybe_invalid(Key::client_key(), Initial::client_value(), ToAdd::client_value()) -> true | no_return().\nprop_add_on_nr_maybe_invalid(Key, Initial, ToAdd) ->\n    ?equals(api_tx:write(Key, Initial), {ok}),\n    prop_add_on_nr2(Key, true, Initial, ToAdd).\n\ntester_add_on_nr_maybe_invalid(_Config) ->\n    tester:test(?MODULE, prop_add_on_nr_maybe_invalid, 3, 5000).\n\n-spec prop_test_and_set2(Key::client_key(), Existing::boolean(), RealOldValue::client_value(), OldValue::client_value(), NewValue::client_value()) -> true | no_return().\nprop_test_and_set2(Key, Existing, RealOldValue, OldValue, NewValue) ->\n    if not Existing ->\n           ?equals(api_tx:test_and_set(Key, OldValue, NewValue), {fail, not_found}),\n           ?equals(api_tx:read(Key), {fail, not_found});\n       RealOldValue =:= OldValue ->\n           ?equals(api_tx:test_and_set(Key, OldValue, NewValue), {ok}),\n           ?equals(api_tx:read(Key), {ok, NewValue});\n       true ->\n           ?equals(api_tx:test_and_set(Key, OldValue, NewValue), {fail, {key_changed, RealOldValue}}),\n           ?equals(api_tx:read(Key), {ok, RealOldValue})\n    end.\n\n-spec prop_test_and_set_not_existing(Key::client_key(), OldValue::client_value(), NewValue::client_value()) -> true | no_return().\nprop_test_and_set_not_existing(Key, OldValue, NewValue) ->\n    {Existing, RealOldValue} = case api_tx:read(Key) of\n                                   {ok, Value} -> {true, Value};\n                                   _ -> {false, unknown}\n                               end,\n    prop_test_and_set2(Key, Existing, RealOldValue, OldValue, NewValue).\n\ntester_test_and_set_not_existing(_Config) ->\n    tester:test(?MODULE, prop_test_and_set_not_existing, 3, 5000).\n\n-spec prop_test_and_set(Key::client_key(), RealOldValue::client_value(), OldValue::client_value(), NewValue::client_value()) -> true | no_return().\nprop_test_and_set(Key, RealOldValue, OldValue, NewValue) ->\n    ?equals(api_tx:write(Key, RealOldValue), {ok}),\n    prop_test_and_set2(Key, true, RealOldValue, OldValue, NewValue).\n\ntester_test_and_set(_Config) ->\n    tester:test(?MODULE, prop_test_and_set, 4, 5000).\n\n-spec prop_random_from_list(Key::client_key(), Value::client_value()) -> true.\nprop_random_from_list(Key, Value) ->\n    _ = api_tx:write(Key,  [Value]),\n    ?equals_pattern(\n        api_tx:req_list([{read, Key, random_from_list}]),\n        {[{?read, Key, Version, ?ok, SnapNumber, ?value_dropped, ?value_dropped}],\n         [{ok, { Value, 1 } }]} when is_integer(Version)\n                                    andalso Version >= 0\n                                    andalso is_integer(SnapNumber)\n                                    andalso SnapNumber >= 0),\n    ValueEnc = rdht_tx:encode_value({Value, 1}),\n    ?equals_pattern(\n        api_txc:req_list([{read, Key, random_from_list}]),\n        {[{?read, Key, Version, ?ok, SnapNumber, ?value_dropped, ?value_dropped}],\n         [{ok, ValueEnc}]} when is_integer(Version)\n                                    andalso Version >= 0\n                                    andalso is_integer(SnapNumber)\n                                    andalso SnapNumber >= 0),\n    true.\n\ntester_random_from_list(_Config) ->\n    tester:test(?MODULE, prop_random_from_list, 2, 5000).\n\n%%% operations with TLOG:\n\n-spec prop_tlog_add_del_on_list2(TLog::tx_tlog:tlog(), Key::client_key(), Initial::client_value(), OldExists::boolean(), ToAdd::client_value(), ToRemove::client_value()) -> true | no_return().\nprop_tlog_add_del_on_list2(TLog0, Key, Initial, OldExists, ToAdd, ToRemove) ->\n    {TLog1, Result1} = api_tx:add_del_on_list(TLog0, Key, ToAdd, ToRemove),\n    {TLog2, Result2} = api_tx:read(TLog1, Key),\n    if (not erlang:is_list(Initial)) orelse\n           (not erlang:is_list(ToAdd)) orelse\n           (not erlang:is_list(ToRemove)) ->\n           ?equals(Result1, {fail, not_a_list}),\n           if OldExists -> ?equals(Result2, {ok, Initial});\n              true      -> ?equals(Result2, {fail, not_found})\n           end,\n           Result3 = api_tx:commit(TLog2),\n           ?equals(Result3, {fail, abort, [Key]}),\n           if OldExists -> ?equals(api_tx:read(Key), {ok, Initial});\n              true      -> ?equals(api_tx:read(Key), {fail, not_found})\n           end;\n       true ->\n           ?equals(Result1, {ok}),\n           ?equals_pattern(Result2, {ok, _List}),\n           {ok, List} = Result2,\n           SortedList = lists:sort(fun util:'=:<'/2, List),\n           ?equals(SortedList, lists:sort(fun util:'=:<'/2, util:minus_first(lists:append(Initial, ToAdd), ToRemove))),\n           Result3 = api_tx:commit(TLog2),\n           ?equals(Result3, {ok}),\n           ?equals(api_tx:read(Key), Result2)\n    end.\n\n-spec prop_tlog_add_del_on_list_not_existing(Key::client_key(), ToAdd::[client_value()], ToRemove::[client_value()]) -> true | no_return().\nprop_tlog_add_del_on_list_not_existing(Key, ToAdd, ToRemove) ->\n    case api_tx:read(Key) of\n        {ok, OldValue} -> OldExists = true;\n        _ -> OldValue = [], OldExists = false\n    end,\n    prop_tlog_add_del_on_list2(api_tx:new_tlog(), Key, OldValue, OldExists, ToAdd, ToRemove).\n\ntester_tlog_add_del_on_list_not_existing(_Config) ->\n    tester:test(?MODULE, prop_tlog_add_del_on_list_not_existing, 3, 5000).\n\n-spec prop_tlog_add_del_on_list(Key::client_key(), Initial::client_value(), ToAdd::[client_value()], ToRemove::[client_value()]) -> true | no_return().\nprop_tlog_add_del_on_list(Key, Initial, ToAdd, ToRemove) ->\n    ?equals(api_tx:write(Key, Initial), {ok}),\n    prop_tlog_add_del_on_list2(api_tx:new_tlog(), Key, Initial, true, ToAdd, ToRemove).\n\ntester_tlog_add_del_on_list(_Config) ->\n    tester:test(?MODULE, prop_tlog_add_del_on_list, 4, 5000).\n\n-spec prop_tlog_add_del_on_list_maybe_invalid(Key::client_key(), Initial::client_value(), ToAdd::client_value(), ToRemove::client_value()) -> true | no_return().\nprop_tlog_add_del_on_list_maybe_invalid(Key, Initial, ToAdd, ToRemove) ->\n    ?equals(api_tx:write(Key, Initial), {ok}),\n    prop_tlog_add_del_on_list2(api_tx:new_tlog(), Key, Initial, true, ToAdd, ToRemove).\n\ntester_tlog_add_del_on_list_maybe_invalid(_Config) ->\n    tester:test(?MODULE, prop_tlog_add_del_on_list_maybe_invalid, 4, 5000).\n\n-spec prop_tlog_add_on_nr2(TLog::tx_tlog:tlog(), Key::client_key(), Existing::boolean(), Initial::client_value(), ToAdd::client_value()) -> true | no_return().\nprop_tlog_add_on_nr2(TLog0, Key, Existing, Initial, ToAdd) ->\n    {TLog1, Result1} = api_tx:add_on_nr(TLog0, Key, ToAdd),\n    {TLog2, Result2} = api_tx:read(TLog1, Key),\n    if (not erlang:is_number(Initial)) orelse\n           (not erlang:is_number(ToAdd)) ->\n           ?equals(Result1, {fail, not_a_number}),\n           if Existing -> ?equals(Result2, {ok, Initial});\n              true     -> ?equals(Result2, {fail, not_found})\n           end,\n           Result3 = api_tx:commit(TLog2),\n           ?equals(Result3, {fail, abort, [Key]}),\n           if Existing -> ?equals(api_tx:read(Key), {ok, Initial});\n              true     -> ?equals(api_tx:read(Key), {fail, not_found})\n           end;\n       true ->\n           ?equals(Result1, {ok}),\n           ?equals_pattern(Result2, {ok, _Number}),\n           {ok, Number} = Result2,\n           case Existing of\n               false -> ?equals(Number, ToAdd);\n               % note: Initial+ToAdd could be float when Initial is not and thus Number is neither\n               true when ToAdd == 0 -> ?equals(Number, Initial);\n               _     -> ?equals(Number, (Initial + ToAdd))\n           end,\n           Result3 = api_tx:commit(TLog2),\n           ?equals(Result3, {ok}),\n           ?equals(api_tx:read(Key), Result2)\n    end.\n\n-spec prop_tlog_add_on_nr_not_existing(Key::client_key(), ToAdd::number()) -> true | no_return().\nprop_tlog_add_on_nr_not_existing(Key, ToAdd) ->\n    {Existing, OldValue} = case api_tx:read(Key) of\n                               {ok, Value} -> {true, Value};\n                               _ -> {false, 0}\n                           end,\n    prop_tlog_add_on_nr2(api_tx:new_tlog(), Key, Existing, OldValue, ToAdd).\n\ntester_tlog_add_on_nr_not_existing(_Config) ->\n    tester:test(?MODULE, prop_tlog_add_on_nr_not_existing, 2, 5000).\n\n-spec prop_tlog_add_on_nr(Key::client_key(), Initial::client_value(), ToAdd::number()) -> true | no_return().\nprop_tlog_add_on_nr(Key, Initial, ToAdd) ->\n    ?equals(api_tx:write(Key, Initial), {ok}),\n    prop_tlog_add_on_nr2(api_tx:new_tlog(), Key, true, Initial, ToAdd).\n\ntester_tlog_add_on_nr(_Config) ->\n    tester:test(?MODULE, prop_tlog_add_on_nr, 3, 5000).\n\n-spec prop_tlog_add_on_nr_maybe_invalid(Key::client_key(), Initial::client_value(), ToAdd::client_value()) -> true | no_return().\nprop_tlog_add_on_nr_maybe_invalid(Key, Initial, ToAdd) ->\n    ?equals(api_tx:write(Key, Initial), {ok}),\n    prop_tlog_add_on_nr2(api_tx:new_tlog(), Key, true, Initial, ToAdd).\n\ntester_tlog_add_on_nr_maybe_invalid(_Config) ->\n    tester:test(?MODULE, prop_tlog_add_on_nr_maybe_invalid, 3, 5000).\n\n-spec prop_tlog_test_and_set2(TLog::tx_tlog:tlog(), Key::client_key(), Existing::boolean(), RealOldValue::client_value(), OldValue::client_value(), NewValue::client_value()) -> true | no_return().\nprop_tlog_test_and_set2(TLog0, Key, Existing, RealOldValue, OldValue, NewValue) ->\n    {TLog1, Result1} = api_tx:test_and_set(TLog0, Key, OldValue, NewValue),\n    {TLog2, Result2} = api_tx:read(TLog1, Key),\n    if not Existing ->\n           ?equals(Result1, {fail, not_found}),\n           ?equals(Result2, {fail, not_found}),\n           Result3 = api_tx:commit(TLog2),\n           ?equals(Result3, {fail, abort, [Key]}),\n           ?equals(api_tx:read(Key), {fail, not_found});\n       RealOldValue =:= OldValue ->\n           ?equals(Result1, {ok}),\n           ?equals(Result2, {ok, NewValue}),\n           Result3 = api_tx:commit(TLog2),\n           ?equals(Result3, {ok}),\n           ?equals(api_tx:read(Key), Result2);\n       true ->\n           ?equals(Result1, {fail, {key_changed, RealOldValue}}),\n           ?equals(Result2, {ok, RealOldValue}),\n           Result3 = api_tx:commit(TLog2),\n           ?equals(Result3, {fail, abort, [Key]}),\n           ?equals(api_tx:read(Key), {ok, RealOldValue})\n    end.\n\n-spec prop_tlog_test_and_set_not_existing(Key::client_key(), OldValue::client_value(), NewValue::client_value()) -> true | no_return().\nprop_tlog_test_and_set_not_existing(Key, OldValue, NewValue) ->\n    {Existing, RealOldValue} = case api_tx:read(Key) of\n                                   {ok, Value} -> {true, Value};\n                                   _ -> {false, unknown}\n                               end,\n    prop_tlog_test_and_set2(api_tx:new_tlog(), Key, Existing, RealOldValue, OldValue, NewValue).\n\ntester_tlog_test_and_set_not_existing(_Config) ->\n    tester:test(?MODULE, prop_tlog_test_and_set_not_existing, 3, 5000).\n\n-spec prop_tlog_test_and_set(Key::client_key(), RealOldValue::client_value(), OldValue::client_value(), NewValue::client_value()) -> true | no_return().\nprop_tlog_test_and_set(Key, RealOldValue, OldValue, NewValue) ->\n    ?equals(api_tx:write(Key, RealOldValue), {ok}),\n    prop_tlog_test_and_set2(api_tx:new_tlog(), Key, true, RealOldValue, OldValue, NewValue).\n\ntester_tlog_test_and_set(_Config) ->\n    tester:test(?MODULE, prop_tlog_test_and_set, 4, 5000).\n\n%% @doc Checks that the same result is returned when executing a req_list in a\n%%      bunch or as sequential single requests (partial reads with random data\n%%      are not supported).\n-spec prop_tester_req_list([api_tx:read_request() | api_tx:write_request() |\n                            api_tx:add_del_on_list_request() |\n                            api_tx:add_on_nr_request() |\n                            api_tx:test_and_set_request()]) -> true | no_return().\nprop_tester_req_list(ReqList) ->\n    {TLogAll, ResultAll} = api_tx:req_list(ReqList),\n    {TLogSeq, ResultSeq} =\n        lists:foldl(fun(Req, {TLog, Res}) ->\n                            {NTlog, NRes} = api_tx:req_list(TLog, [Req]),\n                            {NTlog, lists:append(Res, NRes)}\n                    end, {api_tx:new_tlog(), []}, ReqList),\n    ?equals(ResultAll, ResultSeq),\n    ?equals(TLogAll, TLogSeq).\n\ntester_req_list(_Config) ->\n    prop_tester_req_list([{add_del_on_list,[354334],{},[-1]},\n                          {test_and_set,[677315],{[]},3},\n                          {read,[677315]}]),\n    prop_tester_req_list([{add_del_on_list,[354334],{},[-1]},\n                          {read,[677315]},\n                          {read,[677315]}]),\n    tester:test(?MODULE, prop_tester_req_list, 1, 5000).\n\n%% @doc Compares Res with ExpRes for the given request Req. Takes care of not\n%%      comparing results from reading random values, e.g. random_from_list.\n%%      If ORes (1-element list) has failed, Res (1|2-element list) should also\n%%      be the same as ExpRes (1|2-element list).\n-spec same_result_if_not_random(Req::api_tx:request_on_key(), ORes::[api_tx:result(),...],\n                                Res::[api_tx:result(),...], ExpRes::[api_tx:result(),...],\n                                Note::iolist()) -> true.\nsame_result_if_not_random(Req, [ORes], Res, ExpRes, Note) ->\n    case Req of\n        {read, _Key0, ReadOp}\n          when ReadOp =:= random_from_list orelse\n                   (is_tuple(ReadOp) andalso element(1, ReadOp) =:= sublist) ->\n            case ORes of\n                {fail, _} ->\n                    ?equals_w_note(Res, ExpRes, Note);\n                {ok, {_RandomVal, ListLength}} ->\n                    % can only guarantee the list length here\n                    case length(ExpRes) of\n                        2 -> ?equals_pattern_w_note(Res, [{ok}, {ok, {_, ListLength}}], Note);\n                        1 -> ?equals_pattern_w_note(Res, [{ok, {_, ListLength}}], Note)\n                    end\n            end;\n        _ ->\n            ?equals_w_note(Res, ExpRes, Note)\n    end.\n\n-spec check_op_on_tlog(tx_tlog:tlog(), api_tx:request_on_key(), tx_tlog:tlog(),\n                       [api_tx:result(),...], none | client_value()) -> true | no_return().\ncheck_op_on_tlog(TLog, Req, NTLog, NRes, RingVal) ->\n    ReqKey = element(2, Req),\n    case tx_tlog:find_entry_by_key(TLog, ReqKey) of\n        false ->\n            % TODO: implement some checks here, e.g. create OldEntry as read op and continue with the checks below\n            true;\n        OldEntry ->\n            NewEntry = tx_tlog:find_entry_by_key(NTLog, ReqKey),\n            ?assert_w_note(NewEntry =/= false, io_lib:format(\"NewEntry: ~.0p\", [NewEntry])),\n            Note = io_lib:format(\"Entry: ~.0p, Res: ~.0p, Req: ~.0p, RingVal: ~.0p\", [OldEntry, NRes, Req, RingVal]),\n            case tx_tlog:get_entry_status(OldEntry) of\n                ?fail ->\n                    % despite the status 'failed', the operation's result\n                    % should be the same as if there was no failure!\n                    TLog2 = [tx_tlog:set_entry_status(OldEntry, ?ok)],\n                    {NTLog2, NRes2} = api_tx:req_list(TLog2, [Req]),\n                    same_result_if_not_random(Req, NRes, NRes2, NRes, Note),\n                    % while we are at it, check the op on an ok TLog, too:\n                    check_op_on_tlog(TLog2, Req, NTLog2, NRes2, RingVal),\n                    % status must not change between OldEntry and NewEntry!\n                    ?equals(?fail, tx_tlog:get_entry_status(NewEntry));\n                ?ok ->\n\n                    % result must be the same as if executed alone\n                    % note: previous write may have changed the value!\n                    {ExpResAlone, ReqsAlone} =\n                        case tx_tlog:get_entry_operation(OldEntry) of\n                            ?read -> {NRes, [Req]};\n                            ?write ->\n                                ValueAfterWrite = rdht_tx:decode_value(element(2, tx_tlog:get_entry_value(OldEntry))),\n                                {[{ok}, hd(NRes)], [{write, ReqKey, ValueAfterWrite}, Req]}\n                        end,\n                    {_, NRes2} = api_tx:req_list(ReqsAlone),\n                    same_result_if_not_random(Req, NRes, NRes2, ExpResAlone, Note),\n\n                    {_, OReadRes} = api_tx:read(TLog, ReqKey),\n                    case OReadRes of\n                        {fail, not_found} ->\n                            % no write can result in not_found:\n                            ?equals(tx_tlog:get_entry_operation(OldEntry), ?read),\n                            ?assert(RingVal =:= none),\n                            case Req of\n                                {write, _Key, Value} ->\n                                    ?equals(tx_tlog:get_entry_status(NewEntry), ?ok),\n                                    ?equals(tx_tlog:get_entry_value(NewEntry),\n                                            {?value, rdht_tx:encode_value(Value)}),\n                                    ?equals(NRes, [{ok}]);\n                                {read, _Key, _} -> % random_from_list | {sublist, _Start, Len}\n                                    ?equals(tx_tlog:get_entry_status(NewEntry), ?fail),\n                                    ?equals(tx_tlog:get_entry_value_type(NewEntry), ?value_dropped),\n                                    ?equals(NRes, [{fail, not_found}]);\n                                {read, _Key} ->\n                                    ?equals(tx_tlog:get_entry_status(NewEntry), ?ok),\n                                    ?equals(tx_tlog:get_entry_value_type(NewEntry), ?value_dropped),\n                                    ?equals(NRes, [{fail, not_found}]);\n                                {test_and_set, _Key, _Old, _New} ->\n                                    ?equals(tx_tlog:get_entry_status(NewEntry), ?fail),\n                                    ?equals(tx_tlog:get_entry_value_type(NewEntry), ?value_dropped),\n                                    ?equals(NRes, [{fail, not_found}]);\n                                {add_on_nr, _Key, X} when NRes =:= [{ok}] ->\n                                    % check value content (op will create the value)\n                                    ?equals(tx_tlog:get_entry_status(NewEntry), ?ok),\n                                    ?equals(tx_tlog:get_entry_value(NewEntry),\n                                            {?value, rdht_tx:encode_value(X)}),\n                                    ?equals(NRes, [{ok}]);\n                                {add_on_nr, _Key, _X} when NRes =:= [{fail, not_a_number}] ->\n                                    ?equals(tx_tlog:get_entry_status(NewEntry), ?fail),\n                                    ?equals(tx_tlog:get_entry_value_type(NewEntry), ?value_dropped);\n                                {add_del_on_list, _Key, ToAdd, ToDel} when NRes =:= [{ok}] ->\n                                    % check value content (op will create the value)\n                                    ?equals(tx_tlog:get_entry_status(NewEntry), ?ok),\n                                    ?equals(tx_tlog:get_entry_value(NewEntry),\n                                            {?value, rdht_tx:encode_value(util:minus_first(ToAdd, ToDel))}),\n                                    ?equals(NRes, [{ok}]);\n                                {add_del_on_list, _Key, _ToAdd, _ToRemove} when NRes =:= [{fail, not_a_list}] ->\n                                    ?equals(tx_tlog:get_entry_status(NewEntry), ?fail),\n                                    ?equals(tx_tlog:get_entry_value_type(NewEntry), ?value_dropped)\n                            end;\n                        {ok, OValue} ->\n                            case tx_tlog:get_entry_operation(OldEntry) of\n                                ?read ->\n                                    % ?assert(RingVal =/= none) % 'none' may be the value used\n                                    % (anyway, we check the value below so this check is not important)\n                                    ?equals(OValue, RingVal);\n                                ?write ->\n                                    ?equals({?value, rdht_tx:encode_value(OValue)},\n                                            tx_tlog:get_entry_value(OldEntry))\n                            end,\n                            case NRes of\n                                [OkTpl] when is_tuple(OkTpl) andalso element(1, OkTpl) =:= ok ->\n                                    case Req of\n                                        {write, _Key, Value} ->\n                                            ?equals(tx_tlog:get_entry_status(NewEntry), ?ok),\n                                            ?equals(tx_tlog:get_entry_value(NewEntry),\n                                                    {?value, rdht_tx:encode_value(Value)}),\n                                            ?equals(NRes, [{ok}]);\n                                        {read, _Key} ->\n                                            ?equals(OldEntry, NewEntry),\n                                            ?equals_pattern(tx_tlog:get_entry_value_type(NewEntry),\n                                                            XType when XType =:= ?value_dropped orelse XType =:= ?value),\n                                            ?equals(NRes, [{ok, OValue}]);\n                                        {test_and_set, _Key, _Old, New} ->\n                                            ?equals(tx_tlog:get_entry_status(NewEntry), ?ok),\n                                            ?equals(tx_tlog:get_entry_value(NewEntry),\n                                                    {?value, rdht_tx:encode_value(New)}),\n                                            ?equals(NRes, [{ok}]);\n                                        {add_on_nr, _Key, X} when X == 0 -> % no-op (int or float)\n                                            ?equals(tx_tlog:get_entry_status(NewEntry), ?ok),\n                                            ?equals(tx_tlog:get_entry_value(NewEntry),\n                                                    tx_tlog:get_entry_value(OldEntry)),\n                                            ?equals(NRes, [{ok}]);\n                                        {add_on_nr, _Key, X} ->\n                                            ?equals(tx_tlog:get_entry_status(NewEntry), ?ok),\n                                            ?equals(tx_tlog:get_entry_value(NewEntry),\n                                                    {?value, rdht_tx:encode_value(OValue + X)}),\n                                            ?equals(NRes, [{ok}]);\n                                        {add_del_on_list, _Key, ToAdd, ToDel} when ToAdd =:= [] andalso ToDel =:= [] -> % no-op\n                                            ?equals(tx_tlog:get_entry_status(NewEntry), ?ok),\n                                            ?equals(tx_tlog:get_entry_value(NewEntry),\n                                                    tx_tlog:get_entry_value(OldEntry)),\n                                            ?equals(NRes, [{ok}]);\n                                        {add_del_on_list, _Key, ToAdd, ToDel} ->\n                                            ?equals(tx_tlog:get_entry_status(NewEntry), ?ok),\n                                            NValue = util:minus_first(lists:append(ToAdd, OValue), ToDel),\n                                            ?equals(tx_tlog:get_entry_value(NewEntry),\n                                                    {?value, rdht_tx:encode_value(NValue)}),\n                                            ?equals(NRes, [{ok}]);\n                                        {read, _Key, random_from_list} ->\n                                            ?equals_pattern(NRes, [{ok, {_RandomValX, _ListLengthX}}]),\n                                            ?equals(tx_tlog:get_entry_status(NewEntry), ?ok),\n                                            [{ok, {RandomVal, ListLengthX}}] = NRes,\n                                            Note2 = io_lib:format(\"RandomVal: ~p (~p), StoredVal: ~p (~p)\",\n                                                                 [RandomVal, ListLengthX, OValue, length(OValue)]),\n                                            ?assert_w_note(lists:member(RandomVal, OValue), Note2),\n                                            ?equals_w_note(ListLengthX, length(OValue), Note2),\n                                            ?equals_pattern(tx_tlog:get_entry_value_type(NewEntry),\n                                                            XType when XType =:= ?value_dropped orelse XType =:= ?value);\n                                        {read, _Key, {sublist, _Start, Len}} ->\n                                            ?equals_pattern(NRes, [{ok, {_SubListX, _ListLengthX}}]),\n                                            ?equals(tx_tlog:get_entry_status(NewEntry), ?ok),\n                                            [{ok, {SubList, ListLengthX}}] = NRes,\n                                            Note2 = io_lib:format(\"SubList: ~p (~p), StoredVal: ~p (~p)\",\n                                                                 [SubList, ListLengthX, OValue, length(OValue)]),\n                                            ?assert_w_note(length(SubList) =< erlang:abs(Len), Note2),\n                                            ?equals_w_note(lists:subtract(SubList, OValue), [], Note2),\n                                            ?equals_w_note(ListLengthX, length(OValue), Note2),\n                                            ?equals_pattern(tx_tlog:get_entry_value_type(NewEntry),\n                                                            XType when XType =:= ?value_dropped orelse XType =:= ?value)\n                                    end;\n                                [{fail, Reason}] ->\n                                    case Req of\n                                        {write, _Key, _Value} ->\n                                            ?ct_fail(\"a write should never fail without a commit\", []);\n                                        {read, _Key} ->\n                                            ?ct_fail(\"a read should not fail on an existing value\", []);\n                                        {test_and_set, _Key, _Old, _New} ->\n                                            ?equals(Reason, {key_changed, OValue});\n                                        {add_on_nr, _Key, _X} ->\n                                            ?equals(Reason, not_a_number);\n                                        {add_del_on_list, _Key, _ToAdd, _ToDel} ->\n                                            ?equals(Reason, not_a_list);\n                                        {read, _Key, random_from_list} ->\n                                            ?equals_pattern(Reason, X when X =:= empty_list orelse X =:= not_a_list);\n                                        {read, _Key, {sublist, _Start, _Len}} ->\n                                            ?equals(Reason, not_a_list)\n                                    end,\n                                    % note: all ops except read fail the transaction if the op fails\n                                    %       a read never gets to this point though so we can check it here\n                                    ?equals(tx_tlog:get_entry_status(NewEntry), ?fail),\n                                    ?equals(tx_tlog:get_entry_value(NewEntry),\n                                            tx_tlog:get_entry_value(OldEntry))\n                            end\n                    end\n            end\n    end.\n\n\ncheck_commit([], {ok}, _RingVal) -> true;\ncheck_commit(TLog, CommitRes, RingVal) ->\n    TEntry = hd(TLog),\n    Key = tx_tlog:get_entry_key(TEntry),\n    case CommitRes of\n        {ok} ->\n            ?equals(?ok, tx_tlog:get_entry_status(TEntry)),\n            case tx_tlog:get_entry_operation(TEntry) of\n                ?read ->\n                    NewRingVal = case api_tx:read(Key) of\n                                     {fail, not_found} -> none;\n                                     {ok, NewVal} -> NewVal\n                                 end,\n                    ?equals(RingVal, NewRingVal);\n                ?write ->\n                    {ok, NewRingVal} = api_tx:read(Key),\n                    ?equals(tx_tlog:get_entry_value(TEntry),\n                            {?value, rdht_tx:encode_value(NewRingVal)})\n            end;\n        {fail, abort, _} ->\n            % no concurrency in the test case, so the commit result should depend on the entry's tx status!\n            ?equals(?fail, tx_tlog:get_entry_status(TEntry)),\n            NewRingVal = case api_tx:read(Key) of\n                             {fail, not_found} -> none;\n                             {ok, NewVal} -> NewVal\n                         end,\n            ?equals(RingVal, NewRingVal)\n    end.\n\n%% same result when executing a req_list in a bunch or as sequential\n%% single requests.\n-spec prop_tester_req_list_on_same_key(client_key(), [api_tx:request_on_key()]) -> true | no_return().\nprop_tester_req_list_on_same_key(Key, InReqList) ->\n    ReqList = [ setelement(2, Req, Key) || Req <- InReqList ],\n\n    RingVal = case api_tx:read(Key) of\n                  {fail, not_found} -> none;\n                  {ok, Val} -> Val\n              end,\n    %% perform on key not in DHT\n    {TLogSeqE, _ResultSeqE} =\n        lists:foldl(fun(Req, {TLog, Res}) ->\n                            {NTLog, NRes} = api_tx:req_list(TLog, [Req]),\n                            check_op_on_tlog(TLog, Req, NTLog, NRes, RingVal),\n                            {NTLog, lists:append(Res, NRes)}\n                    end, {api_tx:new_tlog(), []}, ReqList),\n\n    CommitE = api_tx:commit(TLogSeqE),\n    check_commit(TLogSeqE, CommitE, RingVal),\n\n    %% perform on key as int\n    {ok} = api_tx:write(Key, 42),\n    {TLogSeqI, _ResultSeqI} =\n        lists:foldl(fun(Req, {TLog, Res}) ->\n                            {NTLog, NRes} = api_tx:req_list(TLog, [Req]),\n                            check_op_on_tlog(TLog, Req, NTLog, NRes, 42),\n                            {NTLog, lists:append(Res, NRes)}\n                    end, {api_tx:new_tlog(), []}, ReqList),\n\n    CommitI = api_tx:commit(TLogSeqI),\n    check_commit(TLogSeqI, CommitI, 42),\n\n    %% perform on key as list\n    {ok} = api_tx:write(Key, [42]),\n    {TLogSeqL, _ResultSeqL} =\n        lists:foldl(fun(Req, {TLog, Res}) ->\n                            {NTLog, NRes} = api_tx:req_list(TLog, [Req]),\n                            check_op_on_tlog(TLog, Req, NTLog, NRes, [42]),\n                            {NTLog, lists:append(Res, NRes)}\n                    end, {api_tx:new_tlog(), []}, ReqList),\n\n    CommitL = api_tx:commit(TLogSeqL),\n    check_commit(TLogSeqL, CommitL, [42]),\n\n    true.\n\ntester_req_list_on_same_key(_Config) ->\n    prop_tester_req_list_on_same_key(\"a\", [{read,\"a\"},{add_on_nr,\"a\",\"*\"},{test_and_set,\"a\",[],{42}}]),\n    tester:test(?MODULE, prop_tester_req_list_on_same_key, 2, 3000).\n\nreq_list_parallelism(_Config) ->\n    Partitions = 25,\n    WriteReqsPart = [{write, lists:flatten(io_lib:format(\"articles:count:~B\", [X])), 200}\n                      || X <- lists:seq(1, Partitions)],\n    ReadReqsPart = [{read, lists:flatten(io_lib:format(\"articles:count:~B\", [X]))}\n                     || X <- lists:seq(1, Partitions)],\n\n    X = lists:duplicate(Partitions, {ok}),\n    X = api_tx:req_list_commit_each(WriteReqsPart),\n    {ok} = api_tx:write(\"articles:count\", 200 * Partitions),\n\n    Iters = 500,\n\n    ReadResPart = lists:sum(util:for_to_ex(1, Iters, fun(_) -> element(1, util:tc(api_tx, req_list_commit_each, [ReadReqsPart])) end)),\n    ReadRes = lists:sum(util:for_to_ex(1, Iters, fun(_) -> element(1, util:tc(api_tx, req_list_commit_each, [[{read, \"articles:count\"}]])) end)),\n    AvgReadResPart = ReadResPart / Iters,\n    AvgReadRes = ReadRes / Iters,\n    ct:pal(\"api_tx:req_list_commit_each~n  1 key : ~.2f~n 25 keys: ~.2f~n\", [AvgReadRes, AvgReadResPart]),\n\n    TxReadResPart = lists:sum(util:for_to_ex(1, Iters, fun(_) -> element(1, util:tc(api_tx, req_list, [ReadReqsPart])) end)),\n    TxReadRes = lists:sum(util:for_to_ex(1, Iters, fun(_) -> element(1, util:tc(api_tx, req_list, [[{read, \"articles:count\"}]])) end)),\n    AvgTxReadResPart = TxReadResPart / Iters,\n    AvgTxReadRes = TxReadRes / Iters,\n    ct:pal(\"api_tx:req_list~n  1 key : ~.2f~n 25 keys: ~.2f~n\", [AvgTxReadRes, AvgTxReadResPart]),\n\n    % parallel reads should not be much slower than a single read (tolerate (Partitions / 2) * time)\n    if AvgReadResPart >= (Partitions / 2) * AvgReadRes ->\n           {comment, lists:flatten(\n              io_lib:format(\n                \"api_tx:req_list_commit_each/1: 1 key: ~.2fus, ~B keys: ~.2fus~n\",\n                [AvgReadRes, Partitions, AvgReadResPart]))};\n       AvgTxReadResPart >= (Partitions / 2) * AvgTxReadRes ->\n           {comment, lists:flatten(\n              io_lib:format(\n                \"api_tx:req_list/1: 1 key: ~.2fus, ~B keys: ~.2fus~n\",\n                [AvgTxReadRes, Partitions, AvgTxReadResPart]))};\n        true -> ok\n    end.\n\n%% @doc Wait until (exactly) the given number of DHT entries are stored.\n%%      This may be necessary, to make sure (late) write messages have arrived\n%%      at the original nodes.\n%%      Note: DHT entries = 4 * client entries!\n-spec wait_for_dht_entries(Count::non_neg_integer() | [?RT:key()]) -> ok.\nwait_for_dht_entries(Count) when is_integer(Count) andalso Count >= 0 ->\n    util:wait_for(\n      fun() ->\n              {Status, Values} = api_dht_raw:range_read(0, 0),\n              Status =:= ok andalso erlang:length(Values) =:= Count\n      end);\nwait_for_dht_entries(HashedKeys) when is_list(HashedKeys) ->\n    GSelf = comm:make_global(self()),\n    util:wait_for(\n      fun() ->\n              Entries = [begin\n                             api_dht_raw:unreliable_lookup(HK, {get_key_entry, GSelf, HK}),\n                             receive {get_key_entry_reply, Entry} -> Entry end\n                         end || HK <- HashedKeys],\n              NonEmptyEntries = [E || E <- Entries, not db_entry:is_empty(E)],\n              length(NonEmptyEntries) =:= length(HashedKeys)\n      end).\n"
  },
  {
    "path": "test/api_tx_SUITE.hrl",
    "content": "%% @copyright 2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Nico Kruber <kruber@zib.de>\n\n-dialyzer({no_fail_call, write_2/1}).\n\n%% deliver list of all test cases defined in here:\nproto_sched_ready_tests() ->\n    [ new_tlog_0,\n      req_list_2,\n      read_2,\n      write_3,\n      commit_1,\n      read_1,\n      write_2,\n      test_and_set_3,\n      conflicting_tx,\n      conflicting_tx2,\n      write2_read2,\n      multi_write,\n      ops_on_not_found,\n      random_write_read\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\nnew_tlog_0(_Config) ->\n    ?proto_sched(start),\n    ?equals(api_tx:new_tlog(), []),\n    ?proto_sched(stop).\nreq_list_2(_Config) ->\n    ?proto_sched(start),\n    EmptyTLog = api_tx:new_tlog(),\n\n    %% execute empty request list\n    ?equals(api_tx:req_list(EmptyTLog, []), {[], []}),\n\n    %% write new item\n    ?equals_pattern(api_tx:req_list(EmptyTLog,\n                                    [{write, \"req_list_2_B\", 7}, {commit}]),\n                    {_TLog, [_WriteRes = {ok}, _CommitRes = {ok}]}),\n    %% read existing item\n    ?equals_pattern(api_tx:req_list(EmptyTLog,\n                                    [{read, \"req_list_2_B\"}, {commit}]),\n                    {_TLog, [_ReadRes = {ok, _ReadVal=7}, _CommitRes = {ok}]}),\n    %% read non-existing item\n    ?equals_pattern(api_tx:req_list(EmptyTLog,\n                                    [{read, \"non-existing\"}, {commit}]),\n                    {_TLog, [_ReadRes = {fail, not_found},\n                             %% allow test for existance of a key to be ok\n                             _CommitRes = {ok}]}),\n    %% read non-existing item and write to that item afterwards\n    ?equals_pattern(api_tx:req_list(EmptyTLog,\n                                    [{read, \"non-existing1\"},\n                                     {write, \"non-existing1\", \"value\"},\n                                     {commit}]),\n                    {_TLog, [_ReadRes = {fail, not_found},\n                             _WriteRes = {ok},\n                             _CommitRes = {ok}]}),\n    %% exec more complex transaction with repeated requests\n    ?equals_pattern(api_tx:req_list(EmptyTLog,\n                                    [{read, \"B\"}, {read, \"B\"},\n                                     {write, \"A\", 8}, {read, \"A\"}, {read, \"A\"},\n                                     {read, \"A\"}, {write, \"B\", 9},\n                                     {commit}]),\n                    {_TLog, [{fail,not_found}, {fail,not_found},\n                             {ok}, {ok, 8}, {ok, 8},\n                             {ok, 8}, {ok},\n                             {ok}]}),\n\n    %% exec empty commit\n    ?equals_pattern(api_tx:req_list(EmptyTLog, [{commit}]),\n                    {_TLog, [{ok}]}),\n\n    %% exec empty double commit\n    ?equals_pattern(api_tx:req_list(EmptyTLog, [{commit}, {commit}]),\n                    {_TLog, [{fail, abort, []}, {fail, abort, []}]}),\n\n    %% try commit not as last operation in request list\n    ?equals_pattern(api_tx:req_list(EmptyTLog, [{commit}, {read, \"A\"}]),\n                    {_TLog, [{fail, abort, []}, _]}),\n\n    %% try commit not as last operation in request list with longer list\n    ?equals_pattern(api_tx:req_list(EmptyTLog,\n                                    [{commit}, {read, \"A\"}, {read, \"B\"}]),\n                    {_TLog, [{fail, abort, []}, _, _]}),\n\n    %% ops based on tlog\n    {NonExistReadTLog, _Res1} = api_tx:read(EmptyTLog, \"req_list_2_C\"),\n    %% write new item which is already in tlog\n    ?equals_pattern(api_tx:req_list(NonExistReadTLog,\n                                    [{write, \"req_list_2_C\", 42}, {commit}]),\n                    {_TLog, [_WriteRes = {ok}, _CommitRes = {ok}]}),\n    %% read existing item which is already in tlog\n    {ExistReadTLog, _Res2} = api_tx:read(EmptyTLog, \"req_list_2_C\"),\n    ?equals_pattern(api_tx:req_list(ExistReadTLog,\n                                    [{read, \"req_list_2_C\"}, {commit}]),\n                    {_TLog, [_ReadRes = {ok, _ReadVal=42}, _CommitRes = {ok}]}),\n    %% read non-existing item\n    {NonExistReadTLog2, _Res3} = api_tx:read(EmptyTLog, \"non-existing\"),\n    ?equals_pattern(api_tx:req_list(NonExistReadTLog2,\n                                    [{read, \"non-existing\"}, {commit}]),\n                    {_TLog, [_ReadRes = {fail, not_found},\n                             %% allow test for existance of a key to be ok\n                             _CommitRes = {ok}]}),\n    ?proto_sched(stop),\n    ok.\n\nread_2(_Config) ->\n    ?proto_sched(start),\n    _ = api_tx:write(\"A\", 7),\n    %% read existing key\n    ?equals_pattern(api_tx:read(api_tx:new_tlog(), \"A\"),\n                    {_, {ok, 7}}),\n    %% read non existing key\n    ?equals_pattern(api_tx:read(api_tx:new_tlog(), \"non-existing\"),\n                    {_, {fail, not_found}}),\n\n    ?proto_sched(stop),\n    ok.\n\nwrite_3(_Config) ->\n    ?proto_sched(start),\n    %% write new key\n    {_, {ok}} = api_tx:write(api_tx:new_tlog(), \"write_3_newkey\", 7),\n    %% modify existing key\n    {_, {ok}} = api_tx:write(api_tx:new_tlog(), \"write_3_newkey\", 8),\n    %% write a key that is already in tlog\n    {TLogA, _} = api_tx:read(api_tx:new_tlog(), \"write_3_newkey\"),\n    {_, {ok}} = api_tx:write(TLogA, \"write_3_newkey\", 9),\n    %% write key that does not exist and the read in tlog failed\n    {TLogB, {fail, not_found}} =\n        api_tx:read(api_tx:new_tlog(), \"write_3_newkey2\"),\n    {_, {ok}} = api_tx:write(TLogB, \"write_3_newkey2\", 9),\n    ?proto_sched(stop),\n    ok.\n\ncommit_1(_Config) ->\n    ?proto_sched(start),\n    EmptyTLog = api_tx:new_tlog(),\n    %% commit empty tlog\n    ?equals(api_tx:commit(EmptyTLog), {ok}),\n\n    %% commit a tlog\n    {WriteTLog, _} = api_tx:write(api_tx:new_tlog(), \"commit_1_A\", 7),\n    ?equals(api_tx:commit(WriteTLog), {ok}),\n\n    _ = api_tx:write(\"commit_1_B\", 7),\n    {ReadTLog, _} = api_tx:read(api_tx:new_tlog(), \"commit_1_B\"),\n    ?equals(api_tx:commit(ReadTLog), {ok}),\n\n    %% commit a failed TLog\n    TimeoutReadTLog =\n        [ tx_tlog:set_entry_status(X, ?fail) || X <- ReadTLog ],\n    ?equals(api_tx:commit(TimeoutReadTLog), {fail, abort, [\"commit_1_B\"]}),\n\n    {WriteTLog2, _} = api_tx:write(api_tx:new_tlog(), \"commit_1_C\", 7),\n    TimeoutWriteTLog =\n        [ tx_tlog:set_entry_status(X, ?fail) || X <- WriteTLog2 ],\n    ?equals(api_tx:commit(TimeoutWriteTLog), {fail, abort, [\"commit_1_C\"]}),\n\n    %% commit a non-existing tlog\n    {NonExistReadTLog, _} = api_tx:read(EmptyTLog, \"non-existing\"),\n    %% allow test for existance of a key to be ok\n    ?equals(api_tx:commit(NonExistReadTLog), {ok}),\n\n    ?proto_sched(stop),\n    ok.\n\nread_1(_Config) ->\n    ?proto_sched(start),\n    ?equals(api_tx:read(\"non-existing\"), {fail, not_found}),\n    ?equals(api_tx:read(\"read_1_ReadKey\"), {fail, not_found}),\n    ?equals(api_tx:write(\"read_1_ReadKey\", \"IsSet\"), {ok}),\n    ?equals(api_tx:read(\"read_1_ReadKey\"), {ok, \"IsSet\"}),\n    ?proto_sched(stop),\n    ok.\n\nwrite_2(_Config) ->\n    ?proto_sched(start),\n    ?equals(api_tx:write(\"write_2_WriteKey\", \"Value\"), {ok}),\n    ?equals(api_tx:read(\"write_2_WriteKey\"), {ok, \"Value\"}),\n    ?equals(api_tx:write(\"write_2_WriteKey\", \"Value2\"), {ok}),\n    ?equals(api_tx:read(\"write_2_WriteKey\"), {ok, \"Value2\"}),\n\n    %% invalid key\n    try ?RT:hash_key([a,b,c]) of\n        _ -> ?equals(catch api_tx:write([a,b,c], \"Value\"), {ok})\n    catch\n        error:badarg ->\n            ?equals_pattern(catch api_tx:write([a,b,c], \"Value\"), {'EXIT',{badarg, _}})\n    end,\n    ?proto_sched(stop),\n    ok.\n\ntest_and_set_3(_Config) ->\n    ?proto_sched(start),\n    ?equals(api_tx:test_and_set(\"test_and_set_3\", \"Value\", \"NextValue\"),\n            {fail, not_found}),\n    ?equals(api_tx:write(\"test_and_set_3\", \"Value\"), {ok}),\n    ?equals(api_tx:test_and_set(\"test_and_set_3\", \"Value\", \"NextValue\"), {ok}),\n    ?equals(api_tx:test_and_set(\"test_and_set_3\", \"wrong\", \"NewValue\"),\n            {fail, {key_changed, \"NextValue\"}}),\n    ?proto_sched(stop),\n    ok.\n\nconflicting_tx(_Config) ->\n    ?proto_sched(start),\n    EmptyTLog = api_tx:new_tlog(),\n    %% ops with other interleaving tx\n    %% prepare an account\n    _ = api_tx:write(\"Account A\", 100),\n\n    %% Tx1: read the balance and later try to modify it\n    {Tx1TLog, {ok, Bal1}} = api_tx:read(EmptyTLog, \"Account A\"),\n\n    %% Tx3: read the balance and later try to commit the read\n    {Tx3TLog, {ok, _Bal3}} = api_tx:read(EmptyTLog, \"Account A\"),\n\n    %% Tx2 reads the balance and increases it\n    {Tx2TLog, {ok, Bal2}} = api_tx:read(EmptyTLog, \"Account A\"),\n    ?equals_pattern(\n       api_tx:req_list(Tx2TLog, [{write, \"Account A\", Bal2 + 100}, {commit}]),\n       {_, [_WriteRes = {ok}, _CommitRes = {ok}]}),\n\n    %% Tx1 tries to increases it atomically and fails\n    ?equals_pattern(\n       api_tx:req_list(Tx1TLog, [{write, \"Account A\", Bal1 + 100}, {commit}]),\n       {_, [_WriteRes = {ok}, _CommitRes = {fail, abort, [\"Account A\"]}]}),\n    io:format(\"DOne~n\"),\n    %% Tx3: try to commit the read and fail (value changed in the meantime)\n    ?equals_pattern(api_tx:commit(Tx3TLog), {fail, abort, [\"Account A\"]}),\n\n    %% check that two reading transactions can coexist\n    %% Tx4: read the balance and later try to commit the read\n    {Tx4TLog, {ok, _Bal4}} = api_tx:read(EmptyTLog, \"Account A\"),\n\n    %% Tx5: read the balance and commit the read\n    {Tx5TLog, {ok, _Bal5}} = api_tx:read(EmptyTLog, \"Account A\"),\n    ?equals_pattern(api_tx:commit(Tx5TLog), {ok}),\n\n    %% Tx4: try to commit a read and succeed (no updates in the meantime)\n    ?equals_pattern(api_tx:commit(Tx4TLog), {ok}),\n    ?proto_sched(stop),\n    ok.\n\nconflicting_tx2(_Config) ->\n    ?proto_sched(start),\n    %% read non-existing item\n%%% log:log(\"1 doing read~n\"),\n    {TLog1a, [ReadRes1a]} =\n        api_tx:req_list([{read, \"conflicting_tx2_non-existing\"}]),\n    ?equals(ReadRes1a, {fail, not_found}),\n%%% log:log(\"2 doing commit for read TLog~n\"),\n    ?equals(api_tx:commit(TLog1a), {ok}),\n\n%%% log:log(\"3 doing single write for key (creating it)~n\"),\n    _ = api_tx:write(\"conflicting_tx2_non-existing\", \"Value\"),\n    %% verify not_found of tlog in commit phase? key now exists!\n%%% log:log(\"4 doing commit for outdated 'not_found' TLog~n\"),\n    ?equals(api_tx:commit(TLog1a),\n            {fail, abort, [\"conflicting_tx2_non-existing\"]}),\n\n%%% log:log(\"5 doing write and commit on outdated 'not_found' TLog~n\"),\n    ?equals_pattern(api_tx:req_list(TLog1a,\n                                    [{write, \"conflicting_tx2_non-existing\", \"NewValue\"},\n                                     {commit}]),\n                    {_TLog, [_WriteRes = {ok},\n                             _CommitRes = {fail, abort, [\"conflicting_tx2_non-existing\"]}]}),\n%%% log:log(\"6 reading same key again, should return initially written value~n\"),\n    ?equals(api_tx:read(\"conflicting_tx2_non-existing\"), {ok, \"Value\"}),\n\n    ?proto_sched(stop),\n    ok.\n\nwrite2_read2(_Config) ->\n    ?proto_sched(start),\n    KeyA = \"KeyA\",\n    KeyB = \"KeyB\",\n    ValueA = \"Value1\",\n    ValueB = \"Value2\",\n\n    log:log(\"Write KeyA\"),\n    {TLog1, _} = api_tx:write(api_tx:new_tlog(), KeyA, ValueA),\n    log:log(\"Write KeyB\"),\n    {TLog2, _} = api_tx:write(TLog1, KeyB, ValueB),\n    log:log(\"commit tlog ~p\", [TLog2]),\n    {ok} = api_tx:commit(TLog2),\n    log:log(\"reading A and B\"),\n\n    ?equals_pattern(api_tx:req_list([{read, KeyA}, {read, KeyB}, {commit}]),\n                    {_TLog4, [{ok, ValueA}, {ok, ValueB}, {ok}]}),\n    ?proto_sched(stop),\n    ok.\n\nmulti_write(_Config) ->\n    ?proto_sched(start),\n    Key = \"MultiWrite\",\n    Value1 = \"Value1\",\n    Value2 = \"Value2\",\n    {TLog1, _} = api_tx:write(api_tx:new_tlog(), Key, Value1),\n    {TLog2, _} = api_tx:write(TLog1, Key, Value2),\n    ?equals(api_tx:commit(TLog2), {ok}),\n    ?equals(api_tx:read(Key), {ok, Value2}),\n    ?proto_sched(stop),\n    ok.\n\n-spec ops_on_not_found(Config::[tuple()]) -> ok.\nops_on_not_found(_Config) ->\n    ?proto_sched(start),\n    %% perform operations on non existing key (TLog of that) and\n    %% check for the return values\n    %% also documents expected behaviour a bit\n    {NotFoundTLog,_} = api_tx:read(api_tx:new_tlog(), \"a\"),\n\n    [ ?equals_w_note(api_tx:req_list(NotFoundTLog, [Req]),\n                     {ExpectedTLog, ExpectedRes},\n                     {'NotFoundTLog: ', NotFoundTLog, ' Req: ', Req})\n      || {Req, {ExpectedTLog, ExpectedRes}} <-\n             [{ {read, \"a\"},\n                % {[{76,\"a\",-1,84,0,78,78}],[{fail,not_found}]} }\n                {[{?read,\"a\",-1,?ok,0,?value_dropped,?value_dropped}],\n                 [{fail,not_found}]}\n              },\n              { {read, \"a\", random_from_list},\n                {[{?read,\"a\",-1,?fail,0,?value_dropped,?value_dropped}],\n                 [{fail,not_found}]}\n              },\n              { {read, \"a\", {sublist, 1, 2}},\n                {[{?read,\"a\",-1,?fail,0,?value_dropped,?value_dropped}],\n                 [{fail,not_found}]}\n              },\n              { {write, \"a\", 7},\n                %% {77,\"a\",-1,84,0,75,7}\n                {[{?write,\"a\",-1,?ok,0,?value,7}],\n                 [{ok}]}\n              },\n              { {add_del_on_list, \"a\", [7], [8]},\n                %% {77,\\\"a\\\",-1,84,0,75,<<131,107,0,1,7>>}]\n                {[{?write,\"a\",-1,?ok,0,?value,term_to_binary([7])}],\n                 [{ok}]}\n              },\n              { {add_del_on_list, \"a\", 7, 8},\n                {[{?read,\"a\",-1,?fail,0,?value_dropped,?value_dropped}],\n                 [{fail, not_a_list}]}\n              },\n              { {add_on_nr, \"a\", 7},\n                {[{?write,\"a\",-1,?ok,0,?value,7}],\n                 [{ok}]}\n              },\n              { {add_on_nr, \"a\", [7]},\n                {[{?read,\"a\",-1,?fail,0,?value_dropped,?value_dropped}],\n                 [{fail, not_a_number}]}\n              },\n              { {test_and_set, \"a\", 0, 7},\n                {[{?read,\"a\",-1,?fail,0,?value_dropped,?value_dropped}],\n                 [{fail,not_found}]}\n              }\n             ]\n    ],\n    ?proto_sched(stop),\n    ok.\n\nrandom_write_read2(0) -> ok;\nrandom_write_read2(Count) ->\n    ?proto_sched(start),\n    Key = io_lib:format(\"~p\", [Count]),\n    ?equals_w_note(api_tx:write(Key, Count), {ok}, Key),\n    ?equals_w_note(api_tx:read(Key), {ok, Count}, Key),\n    ?proto_sched(stop),\n    random_write_read2(Count -1).\n\nrandom_write_read(_) ->\n    random_write_read2(adapt_tx_runs(10000)).\n"
  },
  {
    "path": "test/api_tx_concurrent_SUITE.erl",
    "content": "%  @copyright 2008-2014 Zuse Institute Berlin\n%\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for src/api_tx\n%% @end\n%% @version $Id$\n-module(api_tx_concurrent_SUITE).\n\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n\nall() ->\n    [increment_test_2, increment_test_4, increment_test_8].\n\nsuite() -> [ {timetrap, {seconds, 620}} ].\n\ninit_per_suite(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(4, [{config, [{log_path, PrivDir}]}]),\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninc(Key) ->\n    {TLog1, [ReadResult]} = api_tx:req_list([{read, Key}]),\n    case ReadResult of\n        {ok, Value} ->\n            {_TLog, [{ok}, CommitResult]} =\n                api_tx:req_list(TLog1, [{write, Key, Value + 1}, {commit}]),\n            CommitResult;\n        Fail -> Fail\n    end.\n\nprocess(Parent, Key, Count) ->\n    process_iter(Parent, Key, Count, 0).\n\nprocess_iter(Parent, _Key, 0, AbortCount) ->\n    Parent ! {done, AbortCount};\nprocess_iter(Parent, Key, Count, AbortCount) ->\n    Result = inc(Key),\n    case Result of\n        {ok}             -> process_iter(Parent, Key, Count - 1, AbortCount);\n        {fail, abort, _} -> process_iter(Parent, Key, Count, AbortCount + 1)\n    end.\n\nincrement_test_8(_Config) -> increment_test_n(_Config, 8).\nincrement_test_4(_Config) -> increment_test_n(_Config, 4).\nincrement_test_2(_Config) -> increment_test_n(_Config, 2).\n\nincrement_test_n(_Config, N) ->\n    Key = \"i\",\n    ?equals(api_tx:write(\"i\", 0), {ok}),\n\n    Self = self(),\n    Count = 200 div N,\n    _ = [ spawn(api_tx_concurrent_SUITE, process, [Self, Key, Count])\n          || _ <- lists:seq(1, N) ],\n\n    Aborts = wait_for_done(N),\n    ct:pal(\"aborts: ~w~n\", [Aborts]),\n    {ok, Total} = api_tx:read(Key),\n    ?equals(N*Count, Total),\n    ok.\n\nwait_for_done(0) ->\n    [];\nwait_for_done(Count) ->\n    receive\n        {done, Aborts} ->\n            [Aborts |wait_for_done(Count - 1)]\n    end.\n"
  },
  {
    "path": "test/api_tx_proto_sched_SUITE.erl",
    "content": "%% @copyright 2014-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @version $Id: api_tx_SUITE.erl 5051 2013-07-26 12:21:14Z kruber@zib.de $\n-module(api_tx_proto_sched_SUITE).\n-author('schintke@zib.de').\n-vsn('$Id: api_tx_SUITE.erl 5051 2013-07-26 12:21:14Z kruber@zib.de $').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\n-dialyzer({no_match, concurrent_2/1}). % tx_tm:is_txnew_enabled() looks constant\n\n%% start proto scheduler for this suite\n-define(proto_sched(Action), proto_sched_fun(Action)).\n-include(\"api_tx_SUITE.hrl\").\n\n\ngroups() ->\n    [%% implementation in api_tx_SUITE.hrl\n     %% (shared with api_tx_proto_sched_SUITE.erl)\n     {proto_sched_ready, [sequence],\n      proto_sched_ready_tests()},\n     {only_in_proto_sched,\n      %% [sequence, {repeat_until_any_fail, forever}],\n      %% [sequence, {repeat_until_any_fail, 500}],\n      [sequence],\n      [concurrent_2]}\n    ].\n\n\nall()   -> [ {group, proto_sched_ready},\n             {group, only_in_proto_sched}\n           ].\n\nsuite() -> [ {timetrap, {seconds, 200}} ].\n\ninit_per_testcase(TestCase, Config) ->\n    case TestCase of\n        write_test_race_mult_rings -> %% this case creates its own ring\n            ok;\n        tester_encode_decode -> %% this case does not need a ring\n            ok;\n        _ ->\n            {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n            unittest_helper:make_ring(4, [{config, [{log_path, PrivDir}]}]),\n            timer:sleep(1000),\n            ?ASSERT(ok =:= unittest_helper:check_ring_size_fully_joined(4)),\n            unittest_helper:wait_for_stable_ring_deep(),\n            ok\n    end,\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n-spec proto_sched_fun(start | stop) -> ok.\nproto_sched_fun(start) ->\n    %% ct:pal(\"Starting proto scheduler\"),\n    proto_sched:thread_num(1),\n    proto_sched:thread_begin();\nproto_sched_fun(stop) ->\n    %% is a ring running?\n    case erlang:whereis(pid_groups) =:= undefined\n             orelse pid_groups:find_a(proto_sched) =:= failed of\n        true -> ok;\n        false ->\n            %% then finalize proto_sched run:\n            %% try to call thread_end(): if this\n            %% process was running the proto_sched\n            %% thats fine, otherwise thread_end()\n            %% will raise an exception\n            proto_sched:thread_end(),\n            proto_sched:wait_for_end(),\n            ct:pal(\"Proto scheduler stats: ~.2p\",\n                   %%[proto_sched:info_shorten_messages(\n                   %%   proto_sched:get_infos(), 200)]),\n                   [lists:keydelete(nums_chosen_from, 1,\n                                    lists:keydelete(delivered_msgs, 1,\n                                                    proto_sched:get_infos()))]),\n            proto_sched:cleanup()\n    end.\n\n%% @doc Reduces the number of tx calls so that the time taken by test methods\n%%      with proto_sched does not increase too much.\n-spec adapt_tx_runs(N::pos_integer()) -> pos_integer().\nadapt_tx_runs(N) ->\n    erlang:max(N div (10 * config:read(replication_factor)), 1).\n\nconcurrent_2(_Config) ->\n    %% let two increments run concurrently\n\n    %% initialize a key\n    {ok} = api_tx:write(\"a\", 0),\n\n    Iterations = 6,\n    Threads = 2,\n    log:log(\"Concurrency test begins\"),\n    proto_sched:thread_num(Threads),\n    case tx_tm:is_txnew_enabled() of\n        true ->\n            proto_sched:register_callback(\n              fun api_tx_proto_sched_SUITE:rbr_invariant/3, on_deliver);\n        _ -> ok\n    end,\n    F = fun(_X, 0) -> {ok};\n           (X, Count) ->\n                {TLog, {ok, Val}} = api_tx:read(api_tx:new_tlog(), \"a\"),\n                case Val >= (Iterations - Count) of\n                    false ->\n                        %% I performed (Iterations - Count)\n                        %% increments, so the counter should be at\n                        %% least that high\n                        log:log(\"Oops? Missing value\"),\n                        %% log:log(\"~p\", [proto_sched:get_infos()])\n                        ?DBG_ASSERT2(Val >= (Iterations - Count),\n                                     {'missing increment'});\n                    true -> ok\n                end,\n                {[], [{ok}, CommitRes]} =\n                    api_tx:req_list(TLog, [{write, \"a\", Val+1}, {commit}]),\n                case CommitRes of\n                    {ok} ->\n                        X(X, Count-1);\n                    _ ->\n                        log:log(\"Retry ~p, Val ~p, StillTodo ~p, CommitRes ~p\",\n                                [self(), Val, Count, CommitRes]),\n                        X(X, Count)\n                end\n        end,\n    [ spawn(fun() ->\n                    proto_sched:thread_begin(),\n                    Res = F(F, Iterations),\n                    log:log(\"Res~p ~p\", [X, Res]),\n                    proto_sched:thread_end()\n            end) || X <- lists:seq(1,Threads)],\n    proto_sched:wait_for_end(),\n    proto_sched:cleanup(),\n    ?equals_w_note(api_tx:read(\"a\"), {ok, Threads*Iterations},\n                   wrong_result),\n    ok.\n\n%% callback function for proto_sched\n-spec rbr_invariant(Src::comm:mypid(), Dest::comm:mypid(), Msg::comm:message()) -> ok.\nrbr_invariant(_From, _To, _Msg) ->\n    %% we are a callback function, which is not infected!\n    %% we are executed in the context of proto_sched and are allowed\n    %% to use receive, etc, as where the callback is called, no\n    %% other important things happen...\n    %% as the proto_sched mainly receives {log_send,...} events, we do\n    %% not pick messages from its inbox, that we do not want to pick\n    %% when using receive.\n\n    %% get the key\n    HashedKey = api_dht:hash_key(\"a\"),\n    Replicas = api_dht_raw:get_replica_keys(HashedKey),\n\n    %% retrieve all replicas: we simply retrieve the whole ring and\n    %% filter for the interesting keys\n    DHTNodes = pid_groups:find_all(dht_node),\n\n    RingData = [ begin\n                     N ! {prbr, tab2list_raw, kv, self()},\n                     receive\n                         {kv, List} -> List\n                     end\n                 end\n                 || N <- DHTNodes ],\n    FlatRingData = lists:flatten(RingData),\n\n    MyEntries = [ begin\n                      Entry = lists:keyfind(X, 1, FlatRingData),\n                      setelement(1, Entry, ShortName)\n                  end || {X, ShortName} <- lists:zip(Replicas, [r0, r1, r2, r3]) ],\n\n    %% print all replicas in a structured way\n    case erlang:get(entries) of\n        MyEntries -> ok;\n        [A,B,C,D] ->\n            %% print all changed replicas in a structured way\n            [NA, NB, NC, ND] = MyEntries,\n            X = [NA =:= A, NB =:= B, NC =:= C, ND =:= D],\n            Format = lists:flatten([ \"~n~10000.0p\" || Y <- X, Y =:= false]),\n            log:log(Format,\n                    [ element(2, Y) || Y <- lists:zip(X, MyEntries),\n                                       element(1, Y) =:= false]);\n        _ ->\n            %% print all replicas (more or less than 4?)\n            log:log(\"~n~10000.0p\", [MyEntries])\n    end,\n\n    erlang:put(entries, MyEntries),\n\n    %% anaylse the replicas\n    ok.\n\n\n"
  },
  {
    "path": "test/art_SUITE.erl",
    "content": "%  @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <MLange@informatik.hu-berlin.de>\n%% @doc    Tests for art module (approximate reconciliation tree).\n%% @end\n%% @version $Id$\n-module(art_SUITE).\n-author('mlange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nall() -> [\n          tester_new,\n          tester_lookup\n          %eprof\n         ].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 20}}\n    ].\n\ninit_per_suite(Config) ->\n    rt_SUITE:register_value_creator(),\n    unittest_helper:start_minimal_procs(Config, [], true).\n\nend_per_suite(Config) ->\n    rt_SUITE:unregister_value_creator(),\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_new(intervals:key(), intervals:key()) -> true.\nprop_new(L, R) ->\n    I = unittest_helper:build_interval(L, R),\n    DB = db_generator:get_db(I, 400, uniform, [{output, list_keytpl}]),\n    Conf1 = art:default_config(),\n    Art2 = art:new(merkle_tree:new(I, DB, [{leaf_hf, fun art:merkle_leaf_hf/2}]),\n                   [{correction_factor, proplists:get_value(correction_factor, Conf1) + 1},\n                    {inner_bf_fpr, proplists:get_value(inner_bf_fpr, Conf1) + 0.1},\n                    {leaf_bf_fpr, proplists:get_value(leaf_bf_fpr, Conf1) + 0.1}]),\n    Conf2 = art:get_config(Art2),\n    ?equals(proplists:get_value(correction_factor, Conf1) + 1,\n            proplists:get_value(correction_factor, Conf2)),\n    ?equals(proplists:get_value(inner_bf_fpr, Conf1) + 0.1,\n            proplists:get_value(inner_bf_fpr, Conf2)),\n    ?equals(proplists:get_value(leaf_bf_fpr, Conf1) + 0.1,\n            proplists:get_value(leaf_bf_fpr, Conf2)).\n\ntester_new(_) ->\n    tester:test(?MODULE, prop_new, 2, 100).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_lookup(intervals:key(), intervals:key()) -> true.\nprop_lookup(L, R) ->    \n    I = unittest_helper:build_interval(L, R),\n    DB = db_generator:get_db(I, 400, uniform, [{output, list_keytpl}]),\n    Tree = merkle_tree:new(I, DB, [{leaf_hf, fun art:merkle_leaf_hf/2}]),\n    Art = art:new(Tree),\n    Found = nodes_in_art(merkle_tree:iterator(Tree), Art, 0),\n    ct:pal(\"TreeNodes=~p ; Found=~p\", [merkle_tree:size(Tree), Found]),\n    ?assert(Found > 0).\n\n-spec nodes_in_art(merkle_tree:mt_iter(), art:art(), non_neg_integer()) -> non_neg_integer().\nnodes_in_art(Iter, Art, Acc) ->\n    case merkle_tree:next(Iter) of\n        none -> Acc;\n        {Node, NewIter} -> \n            nodes_in_art(NewIter, Art, Acc + ?IIF(art:lookup(Node, Art), 1, 0))\n    end.\n\ntester_lookup(_) ->\n  tester:test(?MODULE, prop_lookup, 2, 100).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\neprof(_) ->\n    L=0,\n    R=193307343591240590005637476551917548364,\n    ToAdd=1273,\n    \n    I = intervals:new('[', L, R, ']'),\n    Keys = db_generator:get_db(I, ToAdd, uniform, [{output, list_keytpl}]),\n    Merkle = merkle_tree:new(I, Keys, [{leaf_hf, fun art:merkle_leaf_hf/2}]),\n        \n    _ = eprof:start(),\n    Fun = fun() -> art:new(Merkle) end,\n    eprof:profile([], Fun),\n    eprof:stop_profiling(),\n    eprof:analyze(procs, [{sort, time}]),\n    \n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n"
  },
  {
    "path": "test/autoconf_SUITE.erl",
    "content": "% @copyright 2018-2020 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Unit tests for autoconf\n%% @end\n\n\n%% |----------+------------|\n%% | Erlang   |    Release |\n%% |----------+------------|\n%% | R14B04   | 2011-10-04 |\n%% | R15B     | 2011-12-14 |\n%% | R16A     | 2013-01-29 |\n%% | OTP 17.0 | 2014-04-07 |\n%% | OTP 18.0 | 2015-06-24 |\n%% | OTP 19.0 | 2016-06-21 |\n%% | OTP 20.0 | 2017-06-21 |\n%% | OTP 21.0 | 2018-06-19 |\n%% | OTP 22.0 | 2019-05-14 |\n%% | OTP 23.0 | 2020-05-13 |\n%% |----------+------------|\n\n-module(autoconf_SUITE).\n-author('schuett@zib.de').\n\n-compile(export_all).\n\n-include_lib(\"unittest.hrl\").\n\nall() ->\n    [test_has_maps_get_2,\n     test_has_mnesia_sync_log_0,\n     test_has_cerl_sets_new_0,\n     test_has_maps_take_2,\n     test_has_maps_iterator_1,\n     test_has_maps_next_1,\n     test_has_logger_add_handler_3,\n     test_have_crypto_randuniform_support,\n     test_with_crypto_hash,\n     test_with_crypto_bytes_to_integer,\n     test_with_maps,\n     test_with_rand,\n     test_have_ssl_handshake,\n     test_have_ssl_getstat,\n     test_have_new_stacktrace,\n     test_namespaced_dict,\n     test_HAVE_ERLANG_NOW,\n     test_have_ctline_support,\n     test_have_callback_support,\n     test_have_socket_open,\n     test_have_persistent_term_get,\n     test_have_counters_get,\n     test_have_atomics_new\n     ].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 10}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\notp_rel() ->\n    erlang:system_info(otp_release).\n\notp_rel_long() ->\n   Name = filename:join([code:root_dir(), \"releases\",\n                         erlang:system_info(otp_release), \"OTP_VERSION\"]),\n    case file:open(Name, [read]) of\n        {ok, IO} ->\n            {ok, VSN} = file:read_line(IO),\n            string:strip(VSN, right, $\\n);\n        {error, _Reason} ->\n            \"unknown\"\nend.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% maps:get/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% https://github.com/erlang/otp/commit/6fdad74f41803089a0f9026c98f319daecda9a50\n% erts,stdlib: Change map module name to maps\n\n% maps:get/2, OTP-17\nhas_maps_get_2() ->\n    _ = code:ensure_loaded(maps),\n    erlang:function_exported(maps, get, 2).\n\ntest_has_maps_get_2(_Config) ->\n    FalseReleases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\",\n                     \"R16B01\", \"R16B02\", \"R16B03-1\"],\n    TrueReleases = [\"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\"],\n    case has_maps_get_2() of\n        true ->\n            ?assert_w_note(lists:member(otp_rel(), TrueReleases), otp_rel());\n        false ->\n            ?assert_w_note(lists:member(otp_rel(), FalseReleases), otp_rel())\n    end,\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% mnesia:sync_log/0\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% https://github.com/scalaris-team/scalaris/commit/659eb6e44d82d4fbcba133b31ca7a372daf9bfc4\n\n% mnesia:sync_log/0, OTP-17\nhas_mnesia_sync_log_0() ->\n    _ = code:ensure_loaded(mnesia),\n    erlang:function_exported(mnesia, sync_log, 0).\n\ntest_has_mnesia_sync_log_0(_Config) ->\n    FalseReleases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\",\n                     \"R16B01\", \"R16B02\", \"R16B03-1\"],\n    TrueReleases = [\"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\"],\n    case has_mnesia_sync_log_0() of\n        true ->\n            ?assert_w_note(lists:member(otp_rel(), TrueReleases), otp_rel());\n        false ->\n            ?assert_w_note(lists:member(otp_rel(), FalseReleases), otp_rel())\n    end,\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% cerl_sets:new/0\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% https://github.com/erlang/otp/commit/c957cb3887aaadffa75b5bb70f12e79edc841396\n% compiler: Add cerl_sets module\n\n% cerl_sets:new/0, OTP-18\nhas_cerl_sets_new_0() ->\n    _ = code:ensure_loaded(cerl_sets),\n    erlang:function_exported(cerl_sets, new, 0).\n\ntest_has_cerl_sets_new_0(_Config) ->\n    FalseReleases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\",\n                     \"R16B01\", \"R16B02\", \"R16B03-1\", \"17\"],\n    TrueReleases = [\"18\", \"19\", \"20\", \"21\", \"22\", \"23\"],\n    case has_cerl_sets_new_0() of\n        true ->\n            ?assert_w_note(lists:member(otp_rel(), TrueReleases), otp_rel());\n        false ->\n            ?assert_w_note(lists:member(otp_rel(), FalseReleases), otp_rel())\n    end,\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% maps:take/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% https://github.com/erlang/otp/commit/65bd8ade865eebe0d8a3c3210a4e2e9f334e229f\n% erts: Add BIF maps:take/2\n\n% maps:take/2, OTP-19\nhas_maps_take_2() ->\n    _ = code:ensure_loaded(maps),\n    erlang:function_exported(maps, take, 2).\n\ntest_has_maps_take_2(_Config) ->\n    FalseReleases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\",\n                     \"R16B01\", \"R16B02\", \"R16B03-1\", \"17\", \"18\"],\n    TrueReleases = [\"19\", \"20\", \"21\", \"22\", \"23\"],\n    case has_maps_take_2() of\n        true ->\n            ?assert_w_note(lists:member(otp_rel(), TrueReleases), otp_rel());\n        false ->\n            ?assert_w_note(lists:member(otp_rel(), FalseReleases), otp_rel())\n    end,\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% maps:iterator/1\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% https://github.com/erlang/otp/commit/0149a73d15df1f80cb46752ec3829f48c38dd230\n% erts: Implement maps path iterator\n\n% maps:iterator/1, OTP-21\nhas_maps_iterator_1() ->\n    _ = code:ensure_loaded(maps),\n    erlang:function_exported(maps, iterator, 1).\n\ntest_has_maps_iterator_1(_Config) ->\n    FalseReleases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\",\n                     \"R16B01\", \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\"],\n    TrueReleases = [\"21\", \"22\", \"23\"],\n    case has_maps_iterator_1() of\n        true ->\n            ?assert_w_note(lists:member(otp_rel(), TrueReleases), otp_rel());\n        false ->\n            ?assert_w_note(lists:member(otp_rel(), FalseReleases), otp_rel())\n    end,\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% maps:next/1\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% https://github.com/erlang/otp/commit/d945d6f1c71d5442a25e4be60f84fc49ae8b6b4e\n% stdlib: Introduce maps iterator API\n\n% maps:next/1, OTP-21\nhas_maps_next_1() ->\n    _ = code:ensure_loaded(maps),\n    erlang:function_exported(maps, next, 1).\n\ntest_has_maps_next_1(_Config) ->\n    FalseReleases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\",\n                     \"R16B01\", \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\"],\n    TrueReleases = [\"21\", \"22\", \"23\"],\n    case has_maps_next_1() of\n        true ->\n            ?assert_w_note(lists:member(otp_rel(), TrueReleases), otp_rel());\n        false ->\n            ?assert_w_note(lists:member(otp_rel(), FalseReleases), otp_rel())\n    end,\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% logger:add_handler/3\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% https://github.com/erlang/otp/commit/0deea4a8f369013ec00e231d0c2c37e4ab3f0ba1\n% Add logger\n\n% logger:add_handler/3, OTP-21\nhas_logger_add_handler_3() ->\n    _ = code:ensure_loaded(logger),\n    erlang:function_exported(logger, add_handler, 3).\n\ntest_has_logger_add_handler_3(_Config) ->\n    FalseReleases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\",\n                     \"R16B01\", \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\"],\n    TrueReleases = [\"21\", \"22\", \"23\"],\n    case has_logger_add_handler_3() of\n        true ->\n            ?assert_w_note(lists:member(otp_rel(), TrueReleases), otp_rel());\n        false ->\n            ?assert_w_note(lists:member(otp_rel(), FalseReleases), otp_rel())\n    end,\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% have_crypto_randuniform_support\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-ifdef(have_crypto_randuniform_support).\ntest_have_crypto_randuniform_support(_Config) ->\n    Releases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_have_crypto_randuniform_support(_Config) ->\n    ?assert_w_note(lists:member(otp_rel(), []), otp_rel()),\n    ok.\n-endif.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% with_crypto_hash\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-ifdef(with_crypto_hash).\ntest_with_crypto_hash(_Config) ->\n    Releases = [\"R15B02\", \"R15B03\", \"R16B\", \"R16B01\", \"R16B02\", \"R16B03-1\",\n                \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_with_crypto_hash(_Config) ->\n    Releases = [\"R14B04\", \"R15B\", \"R15B01\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% with_crypto_bytes_to_integer\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-ifdef(with_crypto_bytes_to_integer).\ntest_with_crypto_bytes_to_integer(_Config) ->\n    Releases = [\"R16B01\", \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\", \"21\",\n                \"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_with_crypto_bytes_to_integer(_Config) ->\n    Releases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% with_maps\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-ifdef(with_maps).\ntest_with_maps(_Config) ->\n    Releases = [\"18\", \"19\", \"20\", \"21\", \"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_with_maps(_Config) ->\n    Releases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\", \"17\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% with_rand\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-ifdef(with_rand).\ntest_with_rand(_Config) ->\n    Releases = [\"18\", \"19\", \"20\", \"21\", \"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_with_rand(_Config) ->\n    Releases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\", \"17\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% have_ssl_handshake\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-ifdef(have_ssl_handshake).\ntest_have_ssl_handshake(_Config) ->\n    Releases = [\"21\", \"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_have_ssl_handshake(_Config) ->\n    Releases = [\"R14B04\" , \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% have_ssl_getstat\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-ifdef(have_ssl_getstat).\ntest_have_ssl_getstat(_Config) ->\n    Releases = [\"19\", \"20\", \"21\", \"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_have_ssl_getstat(_Config) ->\n    Releases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\", \"17\", \"18\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% have_new_stacktrace\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-ifdef(have_new_stacktrace).\ntest_have_new_stacktrace(_Config) ->\n    Releases = [\"21\", \"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_have_new_stacktrace(_Config) ->\n    Releases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% namespaced_dict\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-ifdef(namespaced_dict).\ntest_namespaced_dict(_Config) ->\n    Releases = [\"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_namespaced_dict(_Config) ->\n    Releases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% HAVE_ERLANG_NOW\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-ifdef(HAVE_ERLANG_NOW).\ntest_HAVE_ERLANG_NOW(_Config) ->\n    ?assert_w_note(lists:member(otp_rel(), []), otp_rel()),\n    ok.\n-else.\ntest_HAVE_ERLANG_NOW(_Config) ->\n    Releases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% have_ctline_support\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-ifdef(have_ctline_support).\ntest_have_ctline_support(_Config) ->\n    Releases = [\"R14B04\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_have_ctline_support(_Config) ->\n    Releases = [\"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% have_callback_support\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-ifdef(have_callback_support).\ntest_have_callback_support(_Config) ->\n    Releases = [\"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\", \"21\", \"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_have_callback_support(_Config) ->\n    Releases = [\"R14B04\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% have_socket_open\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% https://github.com/erlang/otp/commit/3ca71520bfb664f0ea809ffdf41505936e4d5e90\n% [socket-nif] preliminary version of the new socket interface (nififying)\n\n-ifdef(have_socket_open).\ntest_have_socket_open(_Config) ->\n    Releases = [\"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_have_socket_open(_Config) ->\n    Releases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\", \"21\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% have_persistent_term_get\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% https://github.com/erlang/otp/commit/805748eb668d5562fe17f3172cdae07a86166c3f\n% Add a persistent term storage\n\n%% OTP 21.2\n\n-ifdef(have_persistent_term_get).\ntest_have_persistent_term_get(_Config) ->\n    Releases = [\"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_have_persistent_term_get(_Config) ->\n    Releases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\", \"21\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% have_counters_get\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% https://github.com/erlang/otp/commit/fefb5d039e87ff7137e78b3d5f2eaf01e498ec4d\n% erts: Add new module 'counters'\n\n%% OTP 21.2\n\n-ifdef(have_counters_get).\ntest_have_counters_get(_Config) ->\n    Releases = [\"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_have_counters_get(_Config) ->\n    Releases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\", \"21\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% have_atomics_new\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% https://github.com/erlang/otp/commit/1315c6457e49595fdd3f91693c0506964416c9f0\n% erts: Add new module 'atomics'\n\n%% OTP 21.2\n\n-ifdef(have_atomics_new).\ntest_have_atomics_new(_Config) ->\n    Releases = [\"22\", \"23\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-else.\ntest_have_atomics_new(_Config) ->\n    Releases = [\"R14B04\", \"R15B\", \"R15B01\", \"R15B02\", \"R15B03\", \"R16B\", \"R16B01\",\n                \"R16B02\", \"R16B03-1\", \"17\", \"18\", \"19\", \"20\", \"21\"],\n    ?assert_w_note(lists:member(otp_rel(), Releases), otp_rel()),\n    ok.\n-endif.\n"
  },
  {
    "path": "test/banking_account_SUITE.erl",
    "content": "%  @copyright 2008-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Unit tests testing banking transactions\n%% @end\n%% @version $Id$\n-module(banking_account_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id$').\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n\nall()   -> [banking_account].\nsuite() -> [ {timetrap, {seconds, 120}} ].\n\ninit_per_suite(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(4, [{config, [{log_path, PrivDir}]}]),\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\nmoney_transfer(A, B) ->\n    TLog0 = api_tx:new_tlog(),\n    Req1 = [{read, A},\n            {read, B}],\n    {TLog1, [{ok, ABalance}, {ok, BBalance}]} = api_tx:req_list(TLog0, Req1),\n    AtoB = case ABalance > BBalance of\n               true  ->  1;\n               false -> -1\n           end,\n    Req2 = [{write, A, ABalance - AtoB},\n            {write, B, BBalance + AtoB},\n            {commit}],\n    {_TLog2, [_, _, ResCommit]} = api_tx:req_list(TLog1, Req2),\n    ResCommit.\n\nprocess(Parent, A, B, Count) ->\n    process_iter(Parent, A, B, Count, 0).\n\nprocess_iter(Parent, _A, _B, 0, AbortCount) ->\n    Parent ! {done, AbortCount};\nprocess_iter(Parent, A, B, Count, AbortCount) ->\n    case money_transfer(A, B) of\n        {ok} -> process_iter(Parent, A, B, Count - 1, AbortCount);\n        _ -> process_iter(Parent, A, B, Count, AbortCount + 1)\n    end.\n\nbanking_account(_Config) ->\n    Total = 1000,\n    ?equals(api_tx:write(\"a\", Total), {ok}),\n    ?equals(api_tx:write(\"b\", 0), {ok}),\n    ?equals(api_tx:write(\"c\", 0), {ok}),\n    Self = self(),\n    Count = 1000,\n    spawn(banking_account_SUITE, process, [Self, \"a\", \"b\", Count]),\n    spawn(banking_account_SUITE, process, [Self, \"b\", \"c\", Count]),\n    spawn(banking_account_SUITE, process, [Self, \"c\", \"a\", Count]),\n    Aborts = wait_for_done(3),\n    ct:pal(\"Committed Tx: ~p~n\", [3 * Count]),\n    ct:pal(\"Aborted and restarted Tx due to concurrent conflicting Txs: ~p~n\",\n           [lists:sum(Aborts)]),\n    {ok, A} = api_tx:read(\"a\"),\n    {ok, B} = api_tx:read(\"b\"),\n    {ok, C} = api_tx:read(\"c\"),\n    ct:pal(\"balance: ~p ~p ~p~n\", [A, B, C]),\n    ?equals(A + B + C, Total),\n    ok.\n\nwait_for_done(0) ->\n    [];\nwait_for_done(Count) ->\n    receive\n        {done, Aborts} -> [Aborts | wait_for_done(Count - 1)]\n    end.\n"
  },
  {
    "path": "test/benchmark_SUITE.erl",
    "content": "%  @copyright 2008, 2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Runs the basic benchmarks from src/bench.erl\n%%         The results are stored in several files in the main directory, so\n%%         that the buildbot can fetch the data from there.\n%% @end\n%% @version $Id$\n-module(benchmark_SUITE).\n\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n\nall() ->\n    [run_increment_1_1000, run_increment_10_100,\n     run_read_1_100000, run_read_10_10000].\n\nsuite() -> [ {timetrap, {seconds, 120}} ].\n\ninit_per_suite(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(4, [{config, [{log_path, PrivDir}]}]),\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\nrun_increment_1_1000(_Config) ->\n    Threads    = 1,\n    Iterations = 10000,\n    Start = os:timestamp(),\n    {ok, _} = bench:increment(Threads, Iterations),\n    Stop = os:timestamp(),\n    RunTime = erlang:max(1, timer:now_diff(Stop, Start)),\n    write_result(\"result_increment_1_10000.txt\", Threads * Iterations / RunTime * 1000000.0),\n    ok.\n\nrun_increment_10_100(_Config) ->\n    Threads    = 10,\n    Iterations = 1000,\n    Start = os:timestamp(),\n    {ok, _} = bench:increment(Threads, Iterations),\n    Stop = os:timestamp(),\n    RunTime = erlang:max(1, timer:now_diff(Stop, Start)),\n    write_result(\"result_increment_10_1000.txt\", Threads * Iterations / RunTime * 1000000.0),\n    ok.\n\nrun_read_1_100000(_Config) ->\n    Threads    = 1,\n    Iterations = 100000,\n    Start = os:timestamp(),\n    {ok, _} = bench:quorum_read(Threads, Iterations),\n    Stop = os:timestamp(),\n    RunTime = erlang:max(1, timer:now_diff(Stop, Start)),\n    write_result(\"result_read_1_100000.txt\", Threads * Iterations / RunTime * 1000000.0),\n    ok.\n\nrun_read_10_10000(_Config) ->\n    Threads    = 10,\n    Iterations = 10000,\n    Start = os:timestamp(),\n    {ok, _} = bench:quorum_read(Threads, Iterations),\n    Stop = os:timestamp(),\n    RunTime = erlang:max(1, timer:now_diff(Stop, Start)),\n    write_result(\"result_read_10_10000.txt\", Threads * Iterations / RunTime * 1000000.0),\n    ok.\n\n-spec write_result(Filename::string(), Result::term()) -> ok.\nwrite_result(Filename, Result) ->\n    % make_ring switched to the bin sub-dir...go to top-level:\n    {ok, F} = file:open(\"../\" ++ Filename, [write]),\n    io:fwrite(F, \"~p~n\", [Result]),\n    _ = file:close(F),\n    ok.\n"
  },
  {
    "path": "test/bloom_SUITE.erl",
    "content": "%  @copyright 2010-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <MLange@informatik.hu-berlin.de>\n%% @doc    Tests for bloom filter module.\n%% @end\n%% @version $Id$\n-module(bloom_SUITE).\n-author('mlange@informatik.hu-berlin.de').\n\n-define(BLOOM, bloom).\n-define(Fpr_Test_NumTests, 25).\n\n-include(\"bloom_SUITE.hrl\").\n\nall() -> [\n          tester_p_add_list,\n          tester_add,\n          tester_add_list,\n          tester_join,\n          tester_equals\n          %tester_fpr\n          %eprof\n          %fprof\n         ].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 45}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n"
  },
  {
    "path": "test/bloom_SUITE.hrl",
    "content": "%  @copyright 2010-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <MLange@informatik.hu-berlin.de>\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_p_add_list(BF0Items::[?BLOOM:key()], Items::[?BLOOM:key()]) -> true.\nprop_p_add_list(BF0Items, Items) ->\n    BF0 = newBloom(erlang:max(10, erlang:length(Items)), 0.1),\n    BF = ?BLOOM:add_list(BF0, BF0Items),\n    \n    ?equals(lists:foldl(fun(Item, Acc) -> ?BLOOM:add(Acc, Item) end, BF, Items),\n            ?BLOOM:add_list(BF, Items)).\n\ntester_p_add_list(_) ->\n    prop_p_add_list([], []),\n    prop_p_add_list([], [6,7,8]),\n    prop_p_add_list([6,7,8], [88,103,15,128,219]),\n    tester:test(?MODULE, prop_p_add_list, 2, 10000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_add(?BLOOM:key(), ?BLOOM:key()) -> true.\nprop_add(X, Y) ->\n    B1 = newBloom(10, 0.1),\n    B2 = ?BLOOM:add(B1, X),\n    ?assert(?BLOOM:is_element(B2, X)),\n    B3 = ?BLOOM:add(B2, Y),\n    ?assert(?BLOOM:is_element(B3, X)),\n    ?assert(?BLOOM:is_element(B3, Y)),\n    ?equals(?BLOOM:get_property(B1, items_count), 0),\n    ?equals(?BLOOM:get_property(B2, items_count), 1),\n    ?equals(?BLOOM:get_property(B3, items_count), 2).\n\ntester_add(_) ->\n    prop_add(one, 0.5359298222471391),\n    tester:test(?MODULE, prop_add, 2, 100, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_add_list([?BLOOM:key(),...]) -> true.\nprop_add_list(Items) ->\n    B1 = newBloom(erlang:length(Items), 0.1),\n    B2 = ?BLOOM:add_list(B1, Items),\n    lists:foreach(fun(X) -> ?assert(?BLOOM:is_element(B2, X)) end, Items),\n    ?equals(?BLOOM:get_property(B2, items_count), length(Items)).\n\ntester_add_list(_) ->\n    tester:test(?MODULE, prop_add_list, 1, 10, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_join([?BLOOM:key(),...], [?BLOOM:key(),...]) -> true.\nprop_join(List1, List2) ->\n    BSize = erlang:length(List1) + erlang:length(List2),\n    B1 = ?BLOOM:add_list(newBloom(BSize, 0.1), List1),\n    B2 = ?BLOOM:add_list(newBloom(BSize, 0.1), List2),\n    B3 = ?BLOOM:join(B1, B2),\n    lists:foreach(fun(X) -> ?assert(?BLOOM:is_element(B1, X) andalso\n                                        ?BLOOM:is_element(B3, X)) end, List1),\n    lists:foreach(fun(X) -> ?assert(?BLOOM:is_element(B2, X) andalso\n                                        ?BLOOM:is_element(B3, X)) end, List2),\n    true.\n\ntester_join(_) ->\n    tester:test(?MODULE, prop_join, 2, 100, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_equals([?BLOOM:key(),...]) -> true.\nprop_equals(List) ->\n    B1 = ?BLOOM:add_list(newBloom(erlang:length(List), 0.1), List),\n    B2 = ?BLOOM:add_list(newBloom(erlang:length(List), 0.1), List),\n    ?assert(?BLOOM:equals(B1, B2)).\n\ntester_equals(_) ->\n    tester:test(?MODULE, prop_equals, 1, 100, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_fpr(500..10000, string | int) -> true.\nprop_fpr(ItemCount, ItemType) ->\n    InList = random_list(ItemType, ItemCount),\n    \n    DestFpr = randoms:rand_uniform(1, 100) / 1000,\n    DestFPRList = [DestFpr, DestFpr*0.8, DestFpr*0.5],\n    ExampleBF = ?BLOOM:new_fpr(ItemCount, DestFpr),\n    \n    FPs = [{util:repeat(\n              fun measure_fpr/3, [Fpr, {InList, ItemCount}, ItemType],\n              ?Fpr_Test_NumTests,\n              [parallel, {accumulate, fun(X, Y) -> X + Y end, 0}])\n               / ?Fpr_Test_NumTests,\n            Fpr}\n           || Fpr <- DestFPRList],\n    FPs2 = [{D, M, (1 - D/M) * 100,\n             if M-D =< 0 -> \"ok\"; true -> \"fail\" end }\n           || {M, D} <- FPs],\n    ct:pal(\"ItemCount=~p ; ItemType=~p ; Tests=~p ; Functions=~p ; CompressionRate=~.2f~n\"\n               \"DestFpr, Measured, Diff in %, Status~n~p\",\n               [ItemCount, ItemType, ?Fpr_Test_NumTests,\n                ?BLOOM:get_property(ExampleBF, hfs_size),\n                ?BLOOM:get_property(ExampleBF, size) / ItemCount, FPs2]),\n    true.\n\nmeasure_fpr(DestFpr, {InList, ItemCount}, ListItemType) ->\n    InitBF = ?BLOOM:new_fpr(ItemCount, DestFpr),\n    BF = ?BLOOM:add_list(InitBF, InList),\n    \n    Count = trunc(10 / ?BLOOM:get_property(BF, fpr)),\n    _NotInList = random_list(ListItemType, Count),\n    NotInList = lists:filter(fun(I) -> not lists:member(I, InList) end, _NotInList),\n    Found = lists:foldl(fun(I, Acc) ->\n                                Acc + case ?BLOOM:is_element(BF, I) of\n                                          true -> 1;\n                                          false -> 0\n                                      end\n                        end, 0, NotInList),\n    Found / Count.\n\ntester_fpr(_) ->\n    tester:test(?MODULE, prop_fpr, 2, 2, [{threads, 1}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\neprof(_) ->\n    Count = 1000,\n    BF = newBloom(Count, 0.1),\n    Items = [randoms:getRandomInt() || _ <- lists:seq(1, Count)],\n        \n    _ = eprof:start(),\n    Fun = fun() -> ?BLOOM:add_list(BF, Items) end,\n    eprof:profile([], Fun),\n    eprof:stop_profiling(),\n    eprof:analyze(procs, [{sort, time}]),\n    \n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nfprof(_) ->\n    Count = 1000,\n    BF = newBloom(Count, 0.1),\n    Items = [randoms:getRandomInt() || _ <- lists:seq(1, Count)],\n        \n    fprof:apply(?BLOOM, add, [BF, Items]),\n    fprof:profile(),\n    fprof:analyse([{cols, 120}]),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nnewBloom(ElementNum, Fpr) ->\n    ?BLOOM:new_fpr(ElementNum, Fpr).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% UTILS\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec random_list(ItemType :: int | string, pos_integer()) -> [string() | pos_integer()].\nrandom_list(int, Count) ->\n    util:for_to_ex(1, Count, fun(_) -> randoms:getRandomInt() end);\nrandom_list(string, Count) ->\n    util:for_to_ex(1, Count, fun(_) -> randoms:getRandomString() end).\n\nfor_to_ex(I, N, Fun, AccuFun, Accu) ->\n    NewAccu = AccuFun(Fun(I), Accu),\n    if\n        I < N ->\n            for_to_ex(I + 1, N, Fun, AccuFun, NewAccu);\n        I =:= N ->\n            NewAccu;\n        I > N ->\n            failed\n    end.\n"
  },
  {
    "path": "test/buildbot_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{suites, tests, all}.\n\n{skip_suites, tests, [benchmark_SUITE], \"ignore benchmarks\"}.\n{skip_suites, tests, [performance_SUITE], \"ignore benchmarks\"}.\n{skip_suites, tests, [rr_recon_performance_SUITE], \"ignore benchmarks\"}.\n{skip_cases, tests, histogram_SUITE, [perf_add], \"ignore benchmarks\"}.\n{skip_suites, tests, [scalaris_cth_SUITE], \"only for debugging scalaris_cth.erl\"}.\n\n"
  },
  {
    "path": "test/bulkowner_SUITE.erl",
    "content": "%  @copyright 2008, 2011, 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for src/bulkowner.erl\n%% @end\n%% @version $Id$\n-module(bulkowner_SUITE).\n\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\nall() -> [tester_count].\n\nsuite() -> [ {timetrap, {seconds, 10}} ].\n\ninit_per_suite(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(4, [{config, [{log_path, PrivDir}]}]),\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ntester_count(_Config) ->\n    tester:test(?MODULE, count, 0, 1000),\n    ?expect_no_message().\n\n-spec count() -> ok.\ncount() ->\n    Entries = [begin\n                   RandInt = uid:get_pids_uid(),\n                   db_entry:new(\n                     ?RT:hash_key(erlang:integer_to_list(RandInt)),\n                     X, RandInt)\n               end || X <- lists:seq(1, 16)],\n    db_generator:insert_db(Entries),\n    Total = reduce(Entries),\n    Keys = [db_entry:get_key(Entry) || Entry <- Entries],\n    Id = uid:get_global_uid(),\n    I = intervals:from_elements(Keys),\n    bulkowner:issue_bulk_owner(Id, I, {bulk_read_entry, comm:this()}),\n    ?equals(collect(Id, 0, Total, 0), Total),\n    db_generator:remove_keys(Keys),\n    ok.\n\ncollect(Id, Sum, ExpSum, Msgs) ->\n%%     if Msgs =< 0 -> ok;\n%%        true      -> ct:pal(\"sum after ~p msgs: ~p~n\", [Msgs, Sum])\n%%     end,\n    if\n        Msgs =:= 4 -> Sum; % 4 nodes -> max 4 messages\n        Sum =:= ExpSum -> Sum;\n        true ->\n            receive\n                {bulkowner, reply, Id, {bulk_read_entry_response, _NowDone, Data}} ->\n                    collect(Id, Sum + reduce(Data), ExpSum, Msgs + 1)\n            end\n    end.\n\n-spec reduce(db_dht:db_as_list()) -> integer().\nreduce(Entries) ->\n    lists:foldl(fun(E, Acc) -> db_entry:get_value(E) + Acc end, 0, Entries).\n"
  },
  {
    "path": "test/cbf_SUITE.erl",
    "content": "%  @copyright 2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @author Maik Lange <MLange@informatik.hu-berlin.de>\n%% @doc    Tests for bloom filter module.\n%% @end\n%% @version $Id$\n-module(cbf_SUITE).\n-author('kruber@zib.de').\n-author('mlange@informatik.hu-berlin.de').\n\n-define(BLOOM, cbf).\n-define(Fpr_Test_NumTests, 25).\n\n-include(\"bloom_SUITE.hrl\").\n\nall() -> [\n          tester_p_add_list,\n          tester_add,\n          tester_add_list,\n          tester_join,\n          tester_equals,\n          tester_to_bloom\n          %tester_fpr\n          %eprof\n          %fprof\n         ].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 45}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_to_bloom([?BLOOM:key(),...]) -> true.\nprop_to_bloom(Items) ->\n    CBF1 = newBloom(erlang:length(Items), 0.1),\n    CBF2 = ?BLOOM:add_list(CBF1, Items),\n    BF2 = ?BLOOM:to_bloom(CBF2),\n    lists:foreach(fun(X) -> ?assert(bloom:is_element(BF2, X)) end, Items),\n    ?equals(bloom:get_property(BF2, items_count), length(Items)).\n\ntester_to_bloom(_) ->\n    tester:test(?MODULE, prop_to_bloom, 1, 10, [{threads, 2}]).\n"
  },
  {
    "path": "test/churn_SUITE.erl",
    "content": "% @copyright 2008-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc : Unit tests for transactions under churn\n%% @end\n%% @version $Id$\n-module(churn_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\ntest_cases() ->\n    [\n     transactions_1_failure_4_nodes_read,\n     transactions_2_failures_4_nodes_read,\n     transactions_3_failures_4_nodes_read,\n     transactions_1_failure_4_nodes_networksplit_write,\n     transactions_2_failures_4_nodes_networksplit_write,\n     transactions_3_failures_4_nodes_networksplit_write\n    ].\n\nall() ->\n%%     unittest_helper:create_ct_all(test_cases()).\n    test_cases().\n\ngroups() ->\n%%     unittest_helper:create_ct_groups(test_cases(), [{transactions_1_failure_4_nodes_read, [sequence, {repeat_until_any_fail, forever}]}]).\n    [].\n\nsuite() -> [ {timetrap, {seconds, 30}} ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(TestCase, Config) ->\n    case TestCase of\n        %% the ring maintenance fixes network split situations, which\n        %% can lead to two separate rings and consistency violation.\n        transactions_2_failures_4_nodes_networksplit_write ->\n            {skip, \"ring maint. cannot handle network split yet - see issue 59\"};\n        transactions_3_failures_4_nodes_networksplit_write ->\n            {skip, \"ring maint. cannot handle network split yet - see issue 59\"};\n        _ ->\n            {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n            unittest_helper:make_symmetric_ring(\n              [{config, [{log_path, PrivDir}, {rrepair_after_crash, false},\n                         {replication_factor, 4}]}]),\n            unittest_helper:check_ring_size_fully_joined(4),\n            [{stop_ring, true} | Config]\n%%             {skip, \"temporarily\"}\n    end.\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\ntransactions_1_failure_4_nodes_read(_) ->\n    transactions_X_failures_4_nodes_read(1, ok, ok).\n\ntransactions_2_failures_4_nodes_read(_) ->\n    transactions_X_failures_4_nodes_read(2, ok_or_abort, ok).\n\ntransactions_3_failures_4_nodes_read(_) ->\n    transactions_X_failures_4_nodes_read(3, abort, ok_or_abort).\n\n-spec transactions_X_failures_4_nodes_read(\n        FailedNodes::1..3, RAfterFail::ok | abort | ok_or_abort,\n        WAfterFail::ok | abort | ok_or_abort) -> true.\ntransactions_X_failures_4_nodes_read(FailedNodes, RAfterFail, WAfterFail) ->\n    ?equals_w_note(api_tx:write(\"0\", 1), {ok}, \"write_0_a\"),\n    ?equals_w_note(api_tx:read(\"0\"), {ok, 1}, \"read_0_a\"),\n    % wait for late write messages to arrive at the original nodes\n    api_tx_SUITE:wait_for_dht_entries(4),\n    _ = api_vm:kill_nodes(FailedNodes),\n    unittest_helper:check_ring_size(4 - FailedNodes),\n    unittest_helper:wait_for_stable_ring(),\n    unittest_helper:wait_for_stable_ring_deep(),\n    \n    RAfterFailRes = api_tx:read(\"0\"),\n    case RAfterFail of\n        ok    -> ?equals_w_note(RAfterFailRes, {ok, 1}, \"read_0_b\");\n        abort -> ?equals_w_note(RAfterFailRes, {fail, not_found}, \"read_0_b\");\n        ok_or_abort -> ok\n    end,\n    \n    WAfterFailRes = api_tx:write(\"0\", 2),\n    case WAfterFail of\n        ok    -> ?equals_w_note(WAfterFailRes, {ok}, \"write_0_b\");\n        abort -> ?equals_w_note(WAfterFailRes, {fail, abort, [\"0\"]}, \"write_0_b\");\n        ok_or_abort -> ok\n    end,\n\n    RAfterWAfterFailRes = api_tx:read(\"0\"),\n    case WAfterFailRes of\n        {ok} -> ?equals_w_note(RAfterWAfterFailRes, {ok, 2}, \"read_0_c\");\n        _    -> ?equals_w_note(RAfterWAfterFailRes, {fail, not_found}, \"read_0_c\")\n    end.\n\ntransactions_1_failure_4_nodes_networksplit_write(_) ->\n    % pause some dht_node:\n    Node = pid_groups:find_a(sup_dht_node),\n    PauseSpec = pause_node(Node),\n    unittest_helper:check_ring_size(3),\n    unittest_helper:wait_for_stable_ring(),\n    unittest_helper:wait_for_stable_ring_deep(),\n\n    ct:pal(\"attempting write_0_a~n\"),\n    ?equals_w_note(api_tx:write(\"0\", 1), {ok}, \"write_0_a\"),\n    ct:pal(\"attempting read_0_a~n\"),\n    ?equals_w_note(api_tx:read(\"0\"), {ok, 1}, \"read_0_a\"),\n\n    unpause_node(PauseSpec),\n\n    ct:pal(\"attempting read_0_b~n\"),\n    ?equals_w_note(api_tx:read(\"0\"), {ok, 1}, \"read_0_b\"),\n    ok.\n\ntransactions_2_failures_4_nodes_networksplit_write(_) ->\n    transactions_more_failures_4_nodes_networksplit_write(2).\n\ntransactions_3_failures_4_nodes_networksplit_write(_) ->\n    transactions_more_failures_4_nodes_networksplit_write(3).\n\n-spec transactions_more_failures_4_nodes_networksplit_write(FailedNodes::2 | 3) -> ok.\ntransactions_more_failures_4_nodes_networksplit_write(FailedNodes) ->\n    PauseSpecs = [pause_node(DhtNodeSupPid) || DhtNodeSupPid <- util:random_subset(FailedNodes, pid_groups:find_all(sup_dht_node))],\n    unittest_helper:check_ring_size(4 - FailedNodes),\n    unittest_helper:wait_for_stable_ring(),\n    unittest_helper:wait_for_stable_ring_deep(),\n\n    ct:pal(\"attempting write_0_a~n\"),\n    ?equals_w_note(api_tx:write(\"0\", 1), {fail, abort, [\"0\"]}, \"write_0_a\"),\n    ct:pal(\"attempting read_0_a~n\"),\n    ?equals_w_note(api_tx:read(\"0\"), {fail, abort, [\"0\"]}, \"read_0_a\"),\n\n    _ = [unpause_node(PauseSpec) || PauseSpec <- PauseSpecs],\n\n    ct:pal(\"attempting write_0_b~n\"),\n    ?equals_w_note(api_tx:write(\"0\", 2), {ok}, \"write_0_b\"),\n    ?equals_w_note(api_tx:read(\"0\"), {ok, 2}, \"read_0_b\"),\n\n    ok.\n\n-type pause_spec() :: {pid_groups:groupname(), [pid()]}.\n\n-spec pause_node(DhtNodeSupPid::pid()) -> pause_spec().\npause_node(DhtNodeSupPid) ->\n    GroupName = pid_groups:group_of(DhtNodeSupPid),\n    DhtNodePid = pid_groups:pid_of(GroupName, dht_node),\n    ct:log(\"Pausing pid ~p~n\", [DhtNodePid]),\n    DhtNodeSupChilds = sup:sup_get_all_children(DhtNodeSupPid),\n    _ = [case gen_component:is_gen_component(Pid) of\n             true ->\n                 gen_component:bp_set_cond(Pid, fun(_Msg, _State) -> true end, sleep),\n                 gen_component:bp_barrier(Pid);\n             false -> ok\n         end || Pid <- DhtNodeSupChilds],\n\n    fd:report(crash, DhtNodeSupChilds, 'DOWN'),\n\n    pid_groups:hide(GroupName),\n    {GroupName, DhtNodeSupChilds}.\n\n-spec unpause_node(pause_spec()) -> ok.\nunpause_node({GroupName, DhtNodeSupChilds}) ->\n    pid_groups:unhide(GroupName),\n    DhtNodePid = pid_groups:pid_of(GroupName, dht_node),\n    ct:pal(\"Restarting pid ~p~n\", [DhtNodePid]),\n    % restart the node again:\n    _ = [case gen_component:is_gen_component(Pid) of\n             true ->\n                 %% all have to have reached a breakpoint, otherwise this\n                 %% blocks because of the bp_barrier...\n                 gen_component:bp_del_async(Pid, sleep),\n                 gen_component:bp_cont(Pid);\n             false -> ok\n         end || Pid <- DhtNodeSupChilds],\n    ok.\n"
  },
  {
    "path": "test/collect_module_info_SUITE.erl",
    "content": "% @copyright 2015-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Test collect_function_info module\n%% @end\n\n-module(collect_module_info_SUITE).\n-author('schuett@zib.de').\n\n-compile(export_all).\n\n-include_lib(\"kernel/include/file.hrl\").\n\n-include_lib(\"unittest.hrl\").\n\nall() ->\n    [test_collect_function_info].\n\n\nsuite() ->\n    [\n     {timetrap, {seconds, 300}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ntest_collect_function_info(_Config) ->\n    Modules = get_modules(code:get_path()),\n    ParseState = tester_parse_state:new_parse_state(),\n    CollectorFun =\n        fun (Module, InnerParseState) ->\n                tester_collect_function_info:unittest_collect_module_info(Module, InnerParseState)\n        end,\n    _ParseState2 = lists:foldl(CollectorFun, ParseState, Modules),\n    ok.\n\nget_modules(Paths) ->\n    lists:flatten([get_modules_intern(Path) || Path <- Paths]).\n\nget_modules_intern(Path) ->\n    {ok, Files} = file:list_dir(Path),\n    [erlang:list_to_atom(string:left(File, length(File) - 5)) || File <- Files,\n                                                                 length(File) > 5,\n                                                                 string:right(File, 5) =:= \".beam\",\n                                                                 has_abstract_code(File)].\n\n-spec has_abstract_code(string()) -> boolean().\nhas_abstract_code(FileName) ->\n    ModuleFile = code:where_is_file(FileName),\n    case beam_lib:chunks(ModuleFile, [abstract_code]) of\n        {ok, {_Module, [{abstract_code, {_AbstVersion, _AbstractCode}}]}} ->\n            true;\n        {ok, {_Module, [{abstract_code, no_abstract_code}]}} ->\n            false\n    end.\n"
  },
  {
    "path": "test/consistent_lookup_chord_SUITE.erl",
    "content": "%% @copyright 2012-2019 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for consistent lookups with Chord\n%% @end\n%% @version $Id$\n-module(consistent_lookup_chord_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\ngroups() ->\n    [{send_tests, [sequence], [\n                              test_consistent_send\n                              ]}\n    ].\n\nall() ->\n    [\n     {group, send_tests}\n     ].\n\nsuite() -> [ {timetrap, {seconds, 120}} ].\n\ngroup(send_tests) ->\n    [{timetrap, {seconds, 400}}];\ngroup(_) ->\n    suite().\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(_TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(1, [{config, [{log_path, PrivDir}]}]),\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\ntest_consistent_send(_Config) ->\n    Self = self(),\n    Ev = dht_node_lookup:envelope(3, {unittest_consistent_send, Self, '_'}),\n    api_dht_raw:unreliable_lookup(?RT:hash_key(\"0\"),\n                                  Ev),\n    ExpectConsistent = case config:read(leases) of\n                           true -> true;\n                           _ -> false\n                       end,\n    receive\n        {unittest_consistent_send, Self, ExpectConsistent} = Msg ->\n            ct:pal(\"message ~p\", [Msg])\n    end.\n"
  },
  {
    "path": "test/consistent_lookup_leases_SUITE.erl",
    "content": "%% @copyright 2012-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for consistent lookups with leases\n%% @end\n%% @version $Id$\n-module(consistent_lookup_leases_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\ngroups() ->\n    [{send_tests, [sequence], [\n                              test_consistent_send\n                              ]}\n    ].\n\nall() ->\n    [\n     {group, send_tests}\n     ].\n\nsuite() -> [ {timetrap, {seconds, 120}} ].\n\ngroup(send_tests) ->\n    [{timetrap, {seconds, 400}}];\ngroup(_) ->\n    suite().\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(_TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(1, [{config, [{log_path, PrivDir},\n                                            {leases, true}]}]),\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\ntest_consistent_send(_Config) ->\n    Ev = dht_node_lookup:envelope(3, {unittest_consistent_send, self(), '_'}),\n    api_dht_raw:unreliable_lookup(?RT:hash_key(\"0\"),\n                                  Ev),\n    receive\n        {unittest_consistent_send, _Self, true} = Msg ->\n            ct:pal(\"message ~p\", [Msg]);\n        {unittest_consistent_send, _Self, false} = Msg ->\n            ct:pal(\"message ~p\", [Msg])\n    end.\n"
  },
  {
    "path": "test/crash_recovery_SUITE.erl",
    "content": "%% @copyright 2012-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for crash recovery with leases\n%% @end\n%% @version $Id$\n-module(crash_recovery_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\ngroups() ->\n    [{crash_recovery_tests,[sequence], [\n                                        test_crash_recovery,\n                                        test_crash_recovery_one_new_node,\n                                        test_crash_recovery_one_outdated_node,\n                                        test_crash_recovery_bad_owner_pids%,\n                                        %% requires node rejoin after lost active lease\n                                        %% test_crash_recovery_two_outdated_nodes\n                                       ]},\n     {repeater, [{repeat, 30}], [{group, crash_recovery_tests}]}\n    ].\n\nall() ->\n    [\n     {group, crash_recovery_tests}\n     ].\n\nsuite() -> [ {timetrap, {seconds, 40}} ].\n\ngroup(crash_recovery_tests) ->\n    [{timetrap, {seconds, 400}}];\ngroup(_) ->\n    suite().\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\n\ninit_per_testcase(_TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_symmetric_ring([{config, [{log_path, PrivDir},\n                                                   {leases, true}]}]),\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% crash recovery test\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ntest_crash_recovery(_Config) ->\n    % do nothing\n    F = fun(_DHTNodes) ->\n                ok\n        end,\n\n    generic_crash_recovery_test(F).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% crash recovery test with one 'new' node\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ntest_crash_recovery_one_new_node(_Config) ->\n    % delete the lease dbs on the first node\n    F = fun(DHTNodes) ->\n                % drop the four lease_dbs on the first node\n                erase_lease_dbs(hd(DHTNodes))\n        end,\n\n    generic_crash_recovery_test(F).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% crash recovery test with one node having outdated data\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ntest_crash_recovery_one_outdated_node(_Config) ->\n    % delete the lease dbs on the first node\n    F = fun(DHTNodes) ->\n                % manipulate rounds on the first node\n                change_lease_replicas(hd(DHTNodes))\n        end,\n\n    generic_crash_recovery_test(F).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% crash recovery test with *two* nodes having outdated data\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ntest_crash_recovery_two_outdated_nodes(_Config) ->\n    % delete the manipulate dbs on *two* nodes\n    F = fun(DHTNodes) ->\n                % manipulate rounds on two nodes\n                [N1, N2 | _Tail] = DHTNodes,\n                change_lease_replicas(N1),\n                change_lease_replicas(N2)\n        end,\n\n    generic_crash_recovery_test(F).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% crash recovery test with change all owner pids\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ntest_crash_recovery_bad_owner_pids(_Config) ->\n    % XXXX\n    F = fun(DHTNodes) ->\n                change_owner_pids(DHTNodes)\n        end,\n\n    generic_crash_recovery_test(F).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% generic crash recovery test\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ngeneric_crash_recovery_test(DoBadThings) ->\n    % we create a ring with four nodes. We stop lease renewal on all\n    % nodes and wait for the leases to timeout.\n    DHTNodes = pid_groups:find_all(dht_node),\n\n    % stop all nodes\n    ct:pal(\"cr: stop all nodes\"),\n    _ = [lease_helper:intercept_lease_renew(Node) || Node <- DHTNodes],\n    lease_helper:wait_for_number_of_valid_active_leases(0),\n    [gen_component:bp_del(Node, block_trigger) || Node <- DHTNodes],\n\n    % do bad things\n    DoBadThings(DHTNodes),\n\n    % trigger renewal on all nodes\n    ct:pal(\"cr: renew all nodes ~p\", [DHTNodes]),\n    [ comm:send_local(Node, {l_on_cseq, renew_leases}) || Node <- DHTNodes],\n\n    % wait for leases to reappear\n    ct:pal(\"cr: wait for leases to reappear\"),\n    %% tolerate one failed node, as long as a correct ring is created\n    lease_checker2:wait_for_clean_leases(400, [{ring_size_range,\n                                                config:read(replication_factor)-1,\n                                                config:read(replication_factor)}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% functions manipulating the node state\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nerase_lease_dbs(Node) ->\n    F = fun(State) ->\n                LeaseDBs = [{lease_db, I} || I <- lists:seq(1, config:read(replication_factor))],\n                lists:foldl(fun (DB, Acc) ->\n                                    dht_node_state:set_prbr_state(Acc, DB, prbr:init(DB))\n                            end, State, LeaseDBs)\n        end,\n    change_node_state(Node, F).\n\n\nchange_lease_replicas(Node) ->\n    F = fun(State) ->\n                [reset_read_and_write_rounds(State, {lease_db, I})\n                 || I <- lists:seq(1, config:read(replication_factor))],\n                State\n        end,\n    change_node_state(Node, F).\n\nchange_node_state(Node, F) ->\n    comm:send_local(Node, {set_state, comm:this(), F}),\n    receive\n        {set_state_response, _NewState} ->\n            ok\n    end.\n\nchange_owner_pids(DHTNodes) ->\n    Self = comm:this(),\n    F = fun(State) ->\n                [change_owner_pid(Self, State, {lease_db, I})\n                 || I <- lists:seq(1, config:read(replication_factor))],\n                State\n        end,\n    lists:foreach(fun (Node) ->\n                          comm:send_local(Node, {set_state, comm:this(), F}),\n                          receive\n                              {set_state_response, _NewState} ->\n                                  ok\n                          end\n                  end, DHTNodes).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% functions manipulating prbr state\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nchange_owner_pid(Pid, State, DBName) ->\n    LeaseDB = dht_node_state:get(State, DBName),\n    ct:pal(\"LeaseDB ~p\", [LeaseDB]),\n    _ = [\n         case l_on_cseq:is_a_lease(Lease) of\n             true ->\n                 prbr:set_entry({Key,\n                                 {ReadRound, ReadClientId, ReadWriteFilter},\n                                 {WriteRound, WriteClientId, WriteWriteFilter},\n                                 l_on_cseq:set_owner(Lease, Pid), LastWriteFilter\n                                }, LeaseDB);\n             false ->\n                 ct:fail(\"the lease db contains a ~p, which is not a lease record\", [Lease])\n         end\n      || {Key,\n          {ReadRound, ReadClientId, ReadWriteFilter},\n          {WriteRound, WriteClientId, WriteWriteFilter},\n          Lease, LastWriteFilter}\n             <- prbr:tab2list_raw_unittest(LeaseDB)],\n    ok.\n\n\nreset_read_and_write_rounds(State, DBName) ->\n    LeaseDB = dht_node_state:get(State, DBName),\n    _ = [ prbr:set_entry(prbr:new(Key, Value), LeaseDB)\n          || {Key, _R_Read, _R_Write, Value, _LastWriteFilter}\n                 <- prbr:tab2list_raw_unittest(LeaseDB)],\n    ok.\n"
  },
  {
    "path": "test/crdt_SUITE.erl",
    "content": "%% @copyright 2013-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    Unit tests for crdt-paxos.\n%% @end\n-module(crdt_SUITE).\n-author('skrzypczak.de').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\n-define(NUM_REPEATS, 5).\n-define(RANDOMIZE_RING, true).\n-define(CMD_PER_TEST, 1000).\n-define(CMD_PER_PROTO_TEST, 100).\n-define(IMPLEMENTATIONS,\n    [%zheng, not working properly yet\n    wf_crdt_paxos,\n    crdt_paxos]).\n-define(DEFAULT_IMPLEMENTATION, crdt_paxos).\n\n\nall()   -> \n    [\n        {group, Implementation}\n    || Implementation <- ?IMPLEMENTATIONS] ++ [tester_type_check_crdt].\n\ngroups() ->\n    [{I, [],\n        [sanity,\n            {gcounter_group, [sequence, {repeat, ?NUM_REPEATS}],\n             [\n                crdt_gcounter_inc,\n                crdt_gcounter_read_your_write,\n                crdt_gcounter_read_monotonic,\n                crdt_gcounter_read_monotonic2,\n                crdt_gcounter_concurrent_read_monotonic,\n                crdt_gcounter_ordered_concurrent_read,\n                crdt_submit_update_command_list,\n                crdt_submit_query_command_list\n             ]},\n            {pncounter_group, [sequence, {repeat, ?NUM_REPEATS}],\n             [\n              crdt_pncounter_banking\n             ]},\n            {optorset_group, [sequence, {repeat, ?NUM_REPEATS}],\n             [\n              crdt_optorset_lteq\n             ]},\n            {proto_sched_group, [sequence, {repeat, ?NUM_REPEATS}],\n             [\n              crdt_proto_sched_write,\n              crdt_proto_sched_concurrent_read_monotonic,\n              crdt_proto_sched_concurrent_read_ordered,\n              crdt_proto_sched_read_your_write\n            ]}\n        ]\n    } || I <- ?IMPLEMENTATIONS].\n\nsuite() -> [ {timetrap, {seconds, 400}} ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) ->\n    case lists:member(Group, ?IMPLEMENTATIONS) of\n        true -> [{crdt_rsm, Group} | Config];\n        false -> Config\n    end.\n\nend_per_group(_Group, _Config) ->\n    ok.\n\ninit_per_testcase(_TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    {crdt_rsm, RSM} =\n        case lists:keyfind(crdt_rsm, 1, Config) of\n            false -> {crdt_rsm, ?DEFAULT_IMPLEMENTATION};\n            Any -> Any\n        end,\n    {Size, R} =\n        case ?RANDOMIZE_RING of\n            true ->\n                {randoms:rand_uniform(2, 8), randoms:rand_uniform(2, 8)};\n            false ->\n                {3, 3}\n        end,\n\n    unittest_helper:make_ring(Size, [{config, [\n                                        {log_path, PrivDir},\n                                        {replication_factor, R},\n                                        {crdt_rsm, RSM},\n                                        {ordered_links, false}]\n                                    }]),\n\n    ct:pal(\"Start test with ringsize ~p, replication factor ~p and CRDT implementation ~p\",\n        [Size, R, RSM]),\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\nsanity(_Config) ->\n\n    Key = randoms:getRandomString(),\n\n    ok = gcounter_on_cseq:inc(Key),\n    {ok, Result} = gcounter_on_cseq:read(Key),\n\n    ct:pal(\"gcounter state is: ~p\", [gcounter_on_cseq:read_state(Key)]),\n    ?equals(Result, 1),\n    ok.\n\ncrdt_submit_update_command_list(_Config) ->\n    %% submit a list of update commands\n    Key = randoms:getRandomString(),\n\n    Count = randoms:rand_uniform(1, ?CMD_PER_TEST),\n    ct:pal(\"Submitting lists of ~p updates\", [Count]),\n    UpdateList =\n        [fun(ReplicaId, Crdt) ->\n            gcounter:update_add(ReplicaId, I, Crdt)\n         end || I <- lists:seq(1, Count)],\n    ok = crdt_on_cseq:write(Key, gcounter, UpdateList),\n\n    {ok, Result} = gcounter_on_cseq:read(Key),\n    ct:pal(\"gcounter state is: ~p\", [gcounter_on_cseq:read_state(Key)]),\n    ?equals(Count*(Count+1) div 2, Result).\n\ncrdt_submit_query_command_list(_Config) ->\n    %% Check that query ordering is preserverd when submitting a list of\n    %% queries\n    Key = randoms:getRandomString(),\n\n    %% filling counter\n    Count = randoms:rand_uniform(1, 1000),\n    Update = fun(RepId, Crdt) -> gcounter:update_add(RepId, Count, Crdt) end,\n    ok = crdt_on_cseq:write(Key, gcounter, Update),\n\n    %% sanity check\n    {ok, Result} = gcounter_on_cseq:read(Key),\n    ?equals(Count, Result),\n\n    QueryListLength = randoms:rand_uniform(1, ?CMD_PER_TEST div 10),\n    QueryList =\n        [case randoms:rand_uniform(1, 10000) rem 2 of\n            0 ->\n                fun crdt:query_noop/1;\n            1 ->\n               fun gcounter:query_counter/1\n        end || _ <- lists:seq(1, QueryListLength)],\n\n    ct:pal(\"Submitting lists of ~p queries\", [QueryListLength]),\n    {ok, Results} = crdt_on_cseq:read(Key, gcounter, QueryList),\n\n    %% check that result matches queries\n    ?equals(QueryListLength, length(Results)),\n    [?equals(Result, lists:sum(lists:flatten([R]))) || R <- Results],\n\n    List = lists:zip(QueryList, Results),\n    [?equals(Q =:= fun crdt:query_noop/1, is_list(R)) || {Q, R} <- List].\n\n\ncrdt_gcounter_inc(_Config) ->\n    %% start random number of writers\n    %% select a key to operate on\n    %% start random number of increments (non transactional)\n    %% check if number of increments = value in key\n    Key = randoms:getRandomString(),\n    UnitTestPid = self(),\n\n    Parallel = randoms:rand_uniform(2, 10),\n    Count = ?CMD_PER_TEST div Parallel,\n    WriteFun = fun(_I) -> ok = gcounter_on_cseq:inc(Key) end,\n    _ = spawn_writers(UnitTestPid, Parallel, Count, WriteFun),\n    wait_writers_completion(Parallel),\n\n    {ok, Result} = gcounter_on_cseq:read(Key),\n    ct:pal(\"gcounter state is: ~p\", [gcounter_on_cseq:read_state(Key)]),\n    ct:pal(\"Planned ~p increments, done ~p~n\", [Count*Parallel, Result]),\n    ?equals(Result, Count*Parallel),\n    ok.\n\ncrdt_pncounter_banking(_Config) ->\n    %% Emulation of banking account suite (without transaction...i.e. no atomicity\n    %% of a money transfer).\n    %% 1. Create multiple accounts\n    %% 2. Spawn multiple worker that transfer money between accounts\n    %% 3. Check if all the money is still there\n    UnitTestPid = self(),\n\n    AccountNum = randoms:rand_uniform(2, 10),\n    Accounts = [randoms:getRandomString() || _ <- lists:seq(1, AccountNum)],\n    %% init all accounts with some money\n    StartMoney = randoms:rand_uniform(100, 10000),\n    TotalMoney = StartMoney * AccountNum,\n    _ = [pncounter_on_cseq:add(Account, StartMoney) || Account <- Accounts],\n\n    %% spawn worker\n    Parallel = randoms:rand_uniform(1, 50),\n    Count = ?CMD_PER_TEST div Parallel,\n    WriteFun =  fun(_I) ->\n                    TransactAmount = randoms:rand_uniform(1, StartMoney),\n                    From = lists:nth(randoms:rand_uniform(1, AccountNum+1), Accounts),\n                    To = lists:nth(randoms:rand_uniform(1, AccountNum+1), Accounts),\n                    %% allows From =:= To, but shouldn't be a problem\n                    ok = pncounter_on_cseq:subtract(From, TransactAmount),\n                    ok = pncounter_on_cseq:add(To, TransactAmount)\n                end,\n    _ = spawn_writers(UnitTestPid, Parallel, Count, WriteFun),\n    wait_writers_completion(Parallel),\n    %% check if nothing is lost or gained\n    Balances = [begin\n                    {ok, Money} = pncounter_on_cseq:read(Account),\n                    Money\n                end || Account <- Accounts],\n    EndMoney = lists:sum(Balances),\n    ct:pal(\"The individual account balances are: ~n~p\", [Balances]),\n    ct:pal(\"Start balance: ~p~nEnd balance: ~p~n\",\n           [TotalMoney, EndMoney]),\n    ?equals(TotalMoney, EndMoney),\n    ok.\n\ncrdt_gcounter_read_your_write(_Config) ->\n    %% starts concurrent worker writing/reading repeateadly\n    %% each update should be visibile immediatly when using the same worker\n    Key = randoms:getRandomString(),\n    UnitTestPid = self(),\n\n    %% Start writer\n    Parallel = randoms:rand_uniform(20, 21),\n    Count = ?CMD_PER_TEST div Parallel,\n    WriteFun = fun (_) ->\n                    {ok, Old} = gcounter_on_cseq:read(Key),\n                    ok = gcounter_on_cseq:inc(Key),\n                    {ok, New} = gcounter_on_cseq:read(Key),\n                    ?equals(true, Old < New)\n               end,\n    _ = spawn_writers(UnitTestPid, Parallel, Count, WriteFun),\n    wait_writers_completion(Parallel),\n\n    ok.\n\ncrdt_gcounter_read_monotonic(_Config) ->\n    %% starts random number of writers\n    %% start one reader\n    %% values read should increase monotonic\n    Key = randoms:getRandomString(),\n    UnitTestPid = self(),\n\n    %% read for infinity\n    ReadFun = fun() -> {ok, Val} = gcounter_on_cseq:read_state(Key), Val end,\n    CmpFun = fun gcounter:lteq/2,\n    Reader = spawn_monotonic_reader(UnitTestPid, ReadFun, CmpFun),\n\n    %% Start writer\n    Parallel = randoms:rand_uniform(1, 50),\n    Count = ?CMD_PER_TEST div Parallel,\n    WriteFun = fun (_) ->\n                    ok = gcounter_on_cseq:inc(Key)\n               end,\n    _ = spawn_writers(UnitTestPid, Parallel, Count, WriteFun),\n    wait_writers_completion(Parallel),\n\n    %% stop reader\n    stop_readers(Reader),\n    check_monotonic_reader_failure(),\n\n    ok.\n\ncrdt_gcounter_read_monotonic2(_Config) ->\n    %% starts random number of writers\n    %% start multiple reader, which submit reads sequentially\n    %% values read should increase monotonic\n    Key = randoms:getRandomString(),\n    UnitTestPid = self(),\n\n    ReaderCount = randoms:rand_uniform(2, 50),\n    ct:pal(\"Start ~p readers\", [ReaderCount]),\n    Reader = [spawn(fun() ->\n                        Loop =\n                            fun(F) ->\n                                receive\n                                    {read, Pids, Prev} ->\n                                        {ok, Result} = gcounter_on_cseq:read_state(Key),\n                                        case gcounter:lteq(Prev, Result) of\n                                            true -> ok;\n                                            false ->\n                                                ct:pal(\"~n~w ~nis not smaller or equals than ~n~w\", [Prev, Result]),\n                                                UnitTestPid ! {compare_failed, Prev, Result},\n                                                ok\n                                        end,\n                                        NextReader = lists:nth(randoms:rand_uniform(1, length(Pids)+1), Pids),\n                                        NextReader ! {read, Pids, Result},\n                                        F(F);\n                                    {reader_done} ->\n                                        UnitTestPid ! {reader_terminated}\n                                end\n                            end,\n                        Loop(Loop)\n                    end)\n              || _ <- lists:seq(1, ReaderCount)],\n    {ok, Init} = gcounter_on_cseq:read_state(Key),\n    hd(Reader) ! {read, Reader, Init},\n\n    %% do all the writes\n    Parallel = randoms:rand_uniform(1, 50),\n    Count = ?CMD_PER_TEST div Parallel,\n    WriteFun = fun (_) ->\n                    ok = gcounter_on_cseq:inc(Key)\n               end,\n    _ = spawn_writers(UnitTestPid, Parallel, Count, WriteFun),\n    wait_writers_completion(Parallel),\n    stop_readers(Reader),\n\n    receive {compare_failed, A, B} ->\n        ?ct_fail(\"~n~w ~nis not smaller or equals than ~n~w\", [A, B])\n    after 100 ->\n        ok\n    end,\n    ok.\n\n\n\ncrdt_gcounter_concurrent_read_monotonic(_Config) ->\n    %% starts random number of writers\n    %% start multiple reader\n    %% for each reader, the sequence of seen values should increase\n    Key = randoms:getRandomString(),\n    UnitTestPid = self(),\n\n    %% read for infinity\n    ReaderCount = randoms:rand_uniform(2, 10),\n    ct:pal(\"Start ~p readers\", [ReaderCount]),\n    ReadFun = fun() -> {ok, Val} = gcounter_on_cseq:read_state(Key), Val end,\n    CmpFun = fun gcounter:lteq/2,\n    Readers = [spawn_monotonic_reader(UnitTestPid, ReadFun, CmpFun) || _ <- lists:seq(1, ReaderCount)],\n\n    %% Start writer\n    Parallel = randoms:rand_uniform(1, 50),\n    Count = ?CMD_PER_TEST div Parallel,\n    WriteFun = fun (_) ->\n                    ok = gcounter_on_cseq:inc(Key)\n               end,\n    _ = spawn_writers(UnitTestPid, Parallel, Count, WriteFun),\n    wait_writers_completion(Parallel),\n    stop_readers(Readers),\n\n    check_monotonic_reader_failure(),\n\n    ok.\n\ncrdt_gcounter_ordered_concurrent_read(_Config) ->\n    %% Starts random number of writer and two readers\n    %% For two concurrent reads returning r1 and r2, it must alsways hold that\n    %% r1 <= r2 or r2 <= r1. (of course, the same must hold for non-concurrent reads)\n    Key = randoms:getRandomString(),\n    UnitTestPid = self(),\n\n    %% start two readers which will report their read results back to main process\n    ReaderCount = randoms:rand_uniform(2, 5),\n    ct:pal(\"Start ~p readers\", [ReaderCount]),\n    ReaderPids =\n        [spawn(\n           fun() ->\n                   Loop = fun(F) ->\n                            receive\n                                {reader_done} ->\n                                    UnitTestPid ! {reader_terminated}\n                            after 0 ->\n                                {ok, Result} = gcounter_on_cseq:read_state(Key),\n                                UnitTestPid ! {testreturn, Id, Result},\n                                F(F)\n                            end\n                          end,\n                   Loop(Loop)\n           end)\n        || Id <- lists:seq(1, ReaderCount)],\n\n    %% Start writers\n    WriterCount = randoms:rand_uniform(1, 20),\n    Count = (?CMD_PER_TEST div 10) div WriterCount,\n    WriteFun = fun (_) ->\n                    ok = gcounter_on_cseq:inc(Key)\n               end,\n    _ = spawn_writers(UnitTestPid, WriterCount, Count, WriteFun),\n    wait_writers_completion(WriterCount),\n    stop_readers(ReaderPids),\n\n    ReadResults = [\n                    begin\n                        Loop = fun(F, Collected) ->\n                                    receive {testreturn, Id, Result} ->\n                                        F(F, [Result | Collected])\n                                    after 0 ->\n                                        Collected\n                                    end\n                               end,\n                        lists:reverse(Loop(Loop, []))\n                    end\n                  || Id <- lists:seq(1, ReaderCount)],\n    %% check each pair of returns... it should be enough to only check a subset\n    %% of pairs but this is good enough for now\n    ct:pal(\"Check if all ~p reads preformend can be ordered...\", [length(lists:flatten(ReadResults))]),\n    _ = [begin\n            {L1, L2} = {lists:nth(L1Idx, ReadResults), lists:nth(L2Idx, ReadResults)},\n            [\n             ?equals_w_note(gcounter:lteq(E1, E2) orelse gcounter:lteq(E2, E1), true,\n                            lists:flatten(io_lib:format(\"(E1: ~p, E2: ~p)\", [E1, E2])))\n            || E1 <- L1, E2 <- L2]\n         end\n        || L1Idx <- lists:seq(1, ReaderCount), L2Idx <- lists:seq(1, ReaderCount), L1Idx =< L2Idx],\n\n    ok.\n\ncrdt_optorset_lteq(_Config) ->\n    %% Checks if the optimized lteq implementation is equivalent to the paper\n    %% version\n\n    %% Generate some Sets\n    InitSeeds = 1,\n    SetNumber = ?CMD_PER_TEST div 10,\n\n    ct:pal(\"Generating sets...\"),\n    ReplicaCount = config:read(replication_factor),\n    MaxElement = 50, %% do not choose to large.. no sets will be removed otherwise\n\n    InitSets = [optorset:new() || _ <- lists:seq(1, InitSeeds)],\n    Generated = lists:foldl(\n                  fun(_, Acc) ->\n                        % take a random existing set\n                        Idx = randoms:rand_uniform(1, min(length(Acc) + 1, 30)),\n                        S = lists:nth(Idx, Acc),\n\n                        % add or remove some random element on a random replica\n                        E = randoms:rand_uniform(1, MaxElement + 1),\n                        R = randoms:rand_uniform(1, ReplicaCount + 1),\n                        NewS =\n                            case optorset:query_contains(E, S) of\n                                false -> optorset:update_add(R, E, S);\n                                true -> optorset:update_remove(E, S)\n                            end,\n                        [NewS | Acc]\n                  end, InitSets, lists:seq(1, SetNumber)),\n\n    %% check all pairs of sets in generated list and look for lteq lteq/2\n    %% discrepencies\n\n    ct:pal(\"Generating set pairs...\"),\n    Pairs = [{A, B} || A <- Generated, B <- Generated],\n    ct:pal(\"verifying equivalence...\"),\n    Mismatches = lists:foldl(\n                   fun(E={A, B}, Acc) ->\n                        R1 = optorset:lteq(A, B),\n                        R2 = optorset:lteq2(A, B),\n                        case R1 =:= R2 of\n                            true -> Acc;\n                            false ->\n                                ct:pal(\"ERROR: ~n ~p ~n ~p ~n~n lteq: ~p lteq2: ~p\",\n                                       [A, B, R1, R2]),\n                                [E | Acc]\n                        end\n                   end, [], Pairs),\n\n    ?equals_w_note(length(Mismatches), 0, \"Results form lteq and lteq2 does not match\"),\n\n    ok.\n\n\ncrdt_proto_sched_write(_Config) ->\n    Key = randoms:getRandomString(),\n    TraceId = default,\n    UnitTestPid = self(),\n\n    Parallel = randoms:rand_uniform(1, 10),\n    Count = ?CMD_PER_PROTO_TEST div Parallel,\n    WriteFun = fun(_I) -> ok = gcounter_on_cseq:inc(Key) end,\n\n    _ = spawn_writers(UnitTestPid, Parallel, Count, WriteFun, TraceId),\n    proto_sched:thread_num(Parallel, TraceId),\n    ?ASSERT(not proto_sched:infected()),\n    wait_writers_completion(Parallel),\n    proto_sched:wait_for_end(TraceId),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed, TraceId),\n    proto_sched:cleanup(TraceId),\n\n    {ok, Result} = gcounter_on_cseq:read(Key),\n    ct:pal(\"gcounter state is: ~p\", [gcounter_on_cseq:read_state(Key)]),\n    ct:pal(\"Planned ~p increments, done ~p~n\", [Count*Parallel, Result]),\n    ?equals(Count*Parallel, Result),\n\n    ok.\n\ncrdt_proto_sched_concurrent_read_monotonic(_Config) ->\n    %% starts random number of writers\n    %% start one reader\n    %% values read should increase monotonic\n    Key = randoms:getRandomString(),\n    TraceId = default,\n    UnitTestPid = self(),\n\n    %% read for infinity\n    ParallelReader = randoms:rand_uniform(1, 5),\n    ReadFun = fun() -> {ok, Val} = gcounter_on_cseq:read_state(Key), Val end,\n    CmpFun = fun gcounter:lteq/2,\n    Readers = [spawn_monotonic_reader(UnitTestPid, ReadFun, CmpFun, TraceId)\n               || _ <- lists:seq(1, ParallelReader)],\n\n    %% Start writer\n    ParallelWriter = randoms:rand_uniform(1, 10),\n    Count = ?CMD_PER_PROTO_TEST div ParallelWriter,\n    WriteFun = fun(_I) -> ok = gcounter_on_cseq:inc(Key) end,\n    _ = spawn_writers(UnitTestPid, ParallelWriter, Count, WriteFun, TraceId),\n\n    %% run ptoto sched\n    Parallel = ParallelReader + ParallelWriter,\n    proto_sched:thread_num(Parallel, TraceId),\n    ?ASSERT(not proto_sched:infected()),\n    wait_writers_completion(ParallelWriter),\n    stop_readers(Readers),\n\n    check_monotonic_reader_failure(),\n\n    proto_sched:wait_for_end(TraceId),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed, TraceId),\n    proto_sched:cleanup(TraceId),\n\n    ok.\n\ncrdt_proto_sched_concurrent_read_ordered(_Config) ->\n    %% Starts random number of writer and two readers\n    %% For two concurrent reads returning r1 and r2, it must alsways hold that\n    %% r1 <= r2 or r2 <= r1. (of course, the same must hold for non-concurrent reads)\n    Key = randoms:getRandomString(),\n    TraceId = default,\n    UnitTestPid = self(),\n\n    %% start readers which will report their read results back to main process\n    ReaderCount = randoms:rand_uniform(1, 5),\n    ct:pal(\"Start ~p readers\", [ReaderCount]),\n    ReaderPids =\n        [spawn(\n           fun() ->\n                   Loop = fun(F) ->\n                            receive\n                                {reader_done} ->\n                                    UnitTestPid ! {reader_terminated}\n                            after 0 ->\n                                {ok, Result} = gcounter_on_cseq:read_state(Key),\n                                UnitTestPid ! {testreturn, Id, Result},\n                                F(F)\n                            end\n                          end,\n                   proto_sched:thread_begin(TraceId),\n                   Loop(Loop),\n                   proto_sched:thread_end(TraceId)\n           end)\n        || Id <- lists:seq(1, ReaderCount)],\n\n    %% Start writers\n    WriterCount = randoms:rand_uniform(1, 20),\n    Count = ?CMD_PER_PROTO_TEST div WriterCount,\n    WriteFun = fun (_) ->\n                    gcounter_on_cseq:inc(Key)\n               end,\n    _ = spawn_writers(UnitTestPid, WriterCount, Count, WriteFun, TraceId),\n\n    %% start proto_sched\n    Parallel = ReaderCount + WriterCount,\n    proto_sched:thread_num(Parallel, TraceId),\n    ?ASSERT(not proto_sched:infected()),\n    wait_writers_completion(WriterCount),\n    stop_readers(ReaderPids),\n\n    %% terminate proto_sched\n    proto_sched:wait_for_end(TraceId),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed, TraceId),\n    proto_sched:cleanup(TraceId),\n\n    %% verify result\n    ReadResults = [\n                    begin\n                        Loop = fun(F, Collected) ->\n                                    receive {testreturn, Id, Result} ->\n                                        F(F, [Result | Collected])\n                                    after 0 ->\n                                        Collected\n                                    end\n                               end,\n                        lists:reverse(Loop(Loop, []))\n                    end\n                  || Id <- lists:seq(1, ReaderCount)],\n    %% check each pair of returns... it should be enough to only check a subset\n    %% of pairs but this is good enough for now\n    ct:pal(\"Check if all ~p reads preformend can be ordered...\", [length(lists:flatten(ReadResults))]),\n    _ = [begin\n            {L1, L2} = {lists:nth(L1Idx, ReadResults), lists:nth(L2Idx, ReadResults)},\n            [\n             ?equals_w_note(gcounter:lteq(E1, E2) orelse gcounter:lteq(E2, E1), true,\n                            lists:flatten(io_lib:format(\"(E1: ~p, E2: ~p)\", [E1, E2])))\n            || E1 <- L1, E2 <- L2]\n         end\n        || L1Idx <- lists:seq(1, ReaderCount), L2Idx <- lists:seq(1, ReaderCount), L1Idx =< L2Idx],\n\n    ok.\n\ncrdt_proto_sched_read_your_write(_Config) ->\n    %% starts concurrent worker writing/reading repeateadly\n    %% each update should be visibile immediatly when using the same worker\n    Key = randoms:getRandomString(),\n    TraceId = default,\n    UnitTestPid = self(),\n\n    %% Start clinets\n    Parallel = randoms:rand_uniform(1, 10),\n    Count = ?CMD_PER_PROTO_TEST div Parallel,\n    WriteFun = fun (_) ->\n                {ok, Old} = gcounter_on_cseq:read(Key),\n                ok = gcounter_on_cseq:inc(Key),\n                {ok, New} = gcounter_on_cseq:read(Key),\n                ?equals(true, Old < New)\n               end,\n    _ = spawn_writers(UnitTestPid, Parallel, Count, WriteFun, TraceId),\n\n    proto_sched:thread_num(Parallel, TraceId),\n    ?ASSERT(not proto_sched:infected()),\n    wait_writers_completion(Parallel),\n\n    proto_sched:wait_for_end(TraceId),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed, TraceId),\n    proto_sched:cleanup(TraceId),\n\n    ok.\n\ntester_type_check_crdt(_Config) ->\n    Count = 100,\n    config:write(no_print_ring_data, true),\n\n    tester:register_value_creator({typedef, crdt, update_fun, []},\n                                  crdt, tester_create_update_fun, 1),\n    tester:register_value_creator({typedef, crdt, query_fun, []},\n                                  crdt, tester_create_query_fun, 1),\n    tester:register_value_creator({typedef, gcounter, crdt, []},\n                                  gcounter, new, 0),\n    tester:register_value_creator({typedef, pncounter, crdt, []},\n                                  pncounter, new, 0),\n    tester:register_value_creator({typedef, gset, crdt, []},\n                                  gset, new, 0),\n\n    %% [{modulename, [excludelist = {fun, arity}]}]\n    Modules =\n        [ {crdt, [], []},\n          {crdt_beh, [], []},\n          {crdt_proposer,\n           [\n            {start_link, 3},            % starts processes\n            {start_gen_component, 5},   % unsupported types\n            {on, 2},                    % sends messages\n            {read, 5},                  % needs fun as input\n            {write, 5},                 % needs fun as input\n            {send_to_all_replicas, 2}  % sends messages\n           ],\n           [\n            {add_read_reply, 5},        % needs value matching db_type\n            {send_to_all_replicas, 3},  % sends messages\n            {send_to_local_replica, 2}, % sends messages\n            {send_to_local_replica, 3}, % sends messages\n            {start_request, 2},         % sends messages\n            {inform_client, 3},         % cannot create valid envelopes\n            {inform_client, 2},         % cannot create valid envelopes\n            {get_entry, 2},             % needs valid ets:tid(),\n            {save_entry, 2},            % needs valid ets:tid(),\n            {delete_entry, 2}           % needs valid ets:tid(),\n           ]},\n          {crdt_acceptor,\n           [\n            {init, 1},                  % needs to be in a pidgroup for db_name\n            {close, 1},                 % needs valid ets:tid()\n            {close_and_delete, 1},      % needs valid ets:tid()\n            {on, 2},                    % sends messages\n            {get_load, 1},              % needs valid ets:tid()\n            {set_entry, 2},             % needs valid ets:tid()\n            {get_entry, 2},             % needs valid ets:tid()\n            {tab2list, 1},              % needs valid ets:tid()\n            {tab2list_raw_unittest, 1}  % needs valid ets:tid()\n           ],\n           [\n            {tab2list_raw, 1},          % needs valid ets:tid()\n            {msg_update_reply, 3},      % sends messages\n            {msg_merge_reply, 2},       % sends messages\n            {msg_prepare_reply, 5},     % sends messages\n            {msg_prepare_deny, 4},      % sends messages\n            {msg_vote_deny, 4},         % sends messages\n            {msg_vote_reply, 2}         % sends messages\n           ]\n          },\n          {crdt_wait_free_wrapper,\n            [\n             {init, 1},                 % sends messages\n             {start_link, 3},           % starts processes\n             {start_gen_component, 5},  % unsupported types\n             {on, 2},                   % sends messages\n             {read, 5},                 % needs fun as input\n             {write, 5}                 % needs fun as input\n            ],\n            [\n             {get_field, 2},            %% needs in bounds index\n             {set_field, 3},            %% needs in bounds index\n             {add_to_buffer, 5},        %% needs in bounds index\n             {get_pstate, 2},           % needs valid ets:tid()\n             {save_pstate, 3},          % needs valid ets:tid()\n             {send_to_all_proposers, 2}, % sends messages\n             {notify_waiting_progress, 2}, % sends messages\n             {inform_all_clients, 2},   % sends messages\n             {inform_client, 2},        % sends messagse\n             {inform_client, 3}         % sends messagse\n            ]\n          },\n          {gla_proposer,\n           [\n            {start_link, 3},            % starts processes\n            {start_gen_component, 5},   % unsupported types\n            {on, 2},                    % sends messages\n            {read, 5},                  % needs fun as input\n            {write, 5}                  % needs fun as input\n           ],\n           [\n            {send_to_all_replicas, 2},  % sends messages\n            {send_to_all_replicas, 3},  % sends messages\n            {start_request, 2},         % sends messages\n            {inform_client, 3},         % cannot create valid envelopes\n            {inform_client, 2},         % cannot create valid envelopes\n            {get_entry, 2},             % needs valid ets:tid(),\n            {save_entry, 2}             % needs valid ets:tid(),\n           ]},\n          {gla_acceptor,\n           [\n            {init, 1},                  % needs to be in a pidgroup for db_name\n            {close, 1},                 % needs valid ets:tid()\n            {close_and_delete, 1},      % needs valid ets:tid()\n            {on, 2},                    % sends messages\n            {set_entry, 2},             % needs valid ets:tid()\n            {get_entry, 2}              % needs valid ets:tid()\n           ],\n           [\n            {msg_ack_reply, 3},          % sends messages\n            {msg_learner_ack_reply, 6},  % sends messages\n            {msg_nack_reply, 4},       % sends messages\n            {entry_proposed, 2},\n            {entry_update_proposed, 3}\n           ]},\n          {gcounter,\n           [\n            {update_nth, 3}             % requires args in bounds\n           ],\n           [\n            {update_nth, 4}             % requires args in bounds\n           ]\n          },\n          {pncounter, [],[] },\n          {gset,\n           [{exists,2},\n            {fold,3}\n           ],\n           []\n          }\n          %% cannot test both pncounter_on_cseq and gcounter_on_cseq at the same time,\n          %% as this would cause a crash if the same key is used for both\n        ],\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n\n    tester:unregister_value_creator({typedef, crdt, query_fun, []}),\n    tester:unregister_value_creator({typedef, crdt, update_fun, []}),\n    tester:unregister_value_creator({typedef, gcounter, crdt, []}),\n    tester:unregister_value_creator({typedef, pncounter, crdt, []}),\n    tester:unregister_value_creator({typedef, gset, crdt, []}),\n\n    true.\n\n-spec spawn_writers(pid(), non_neg_integer(), non_neg_integer(), fun((non_neg_integer()) -> ok)) -> [pid()].\nspawn_writers(UnitTestPid, NumberOfWriters, IterationsPerWriter, WriteFun) ->\n    spawn_writers(UnitTestPid, NumberOfWriters, IterationsPerWriter, WriteFun, none).\n\n-spec spawn_writers(pid(), non_neg_integer(), non_neg_integer(), fun((non_neg_integer()) -> ok),\n                    atom() | none) -> [pid()].\nspawn_writers(UnitTestPid, NumberOfWriters, IterationsPerWriter, WriteFun, ProtoSchedTraceId) ->\n    ct:pal(\"Starting concurrent writers: ~p~n\"\n           \"Performing iterations: ~p~n\",\n           [NumberOfWriters, IterationsPerWriter]),\n    [spawn(\n        fun() ->\n            _ = case ProtoSchedTraceId of\n                none ->\n                    _ = [WriteFun(I) || I <- lists:seq(1, IterationsPerWriter)];\n                TraceId ->\n                    proto_sched:thread_begin(TraceId),\n                    _ = [WriteFun(I) || I <- lists:seq(1, IterationsPerWriter)],\n                    proto_sched:thread_end(TraceId)\n            end,\n            UnitTestPid ! {done}\n        end)\n     || _Nth <- lists:seq(1, NumberOfWriters)].\n\n\n-spec wait_writers_completion(non_neg_integer()) -> ok.\nwait_writers_completion(NumberOfWriter) ->\n    [receive {done} ->\n        ct:pal(\"Finished ~p/~p.~n\", [Nth, NumberOfWriter]),\n        ok\n    end || Nth <- lists:seq(1, NumberOfWriter)],\n    ok.\n\n-spec stop_readers(pid() | [pid()]) -> ok.\nstop_readers(Reader) when is_pid(Reader) ->\n    stop_readers([Reader]);\nstop_readers(Readers) ->\n    _ = [Reader ! {reader_done} || Reader <- Readers],\n    [receive {reader_terminated} -> ok end || _ <- Readers],\n    ok.\n\n-spec spawn_monotonic_reader(pid(), fun(() -> crdt:crdt()), fun((term(), term()) -> boolean())) -> pid().\nspawn_monotonic_reader(UnitTestPid, ReadFun, LTEQCompareFun) ->\n    spawn_monotonic_reader(UnitTestPid, ReadFun, LTEQCompareFun, none).\n\n-spec spawn_monotonic_reader(pid(), fun(() -> crdt:crdt()), fun((term(), term()) -> boolean()),\n                             atom() | none ) -> pid().\nspawn_monotonic_reader(UnitTestPid, ReadFun, LTEQCompareFun, TraceId) ->\n    ct:pal(\"Starting monotonic reader...\"),\n    spawn(fun() ->\n        Init = ReadFun(),\n        Loop =\n            fun(F, Prev) ->\n                receive {reader_done} ->\n                    UnitTestPid ! {reader_terminated}\n                after 0 ->\n                    V = ReadFun(),\n                    case LTEQCompareFun(Prev, V) of\n                        true ->\n                            F(F, V);\n                        false ->\n                            ct:pal(\"~n~w ~nis not smaller or equals than ~n~w\", [Prev, V]),\n                            UnitTestPid ! {compare_failed, Prev, V},\n                            F(F, V)\n                    end\n                end\n            end,\n        case TraceId of\n            none ->  Loop(Loop, Init);\n            TraceId ->\n                proto_sched:thread_begin(TraceId),\n                Loop(Loop, Init),\n                proto_sched:thread_end(TraceId)\n        end\n    end).\n\n-spec check_monotonic_reader_failure() -> ok.\ncheck_monotonic_reader_failure() ->\n    receive {compare_failed, A, B} ->\n        ?ct_fail(\"~n~w ~nis not smaller or equals than ~n~w\", [A, B])\n    after 100 ->\n        ok\n    end.\n\n"
  },
  {
    "path": "test/db_backend_SUITE.hrl",
    "content": "% @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc    Unit tests for db backends that fullfill src/db_backend_beh.erl.\n%%         just define ?TEST_DB and include this file.\n%%         The default to determine equality of keys is == (ets with ordered_set\n%%         uses this). If a backend only considers matching keys equal (i.e.\n%%         =:=) the EQ macro has to defined to =:= in the file that includes\n%%         this one.\n%% @end\n%% @version $Id$\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n\n%% default equality\n-ifndef(EQ).\n-define(EQ, ==).\n-endif.\n\ntests_avail() ->\n    [tester_put,\n     tester_get,\n     tester_delete,\n     tester_foldl,\n     tester_foldr].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% test put/2 of available backends\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_put([db_backend_beh:entry()]) -> true.\nprop_put(Data) ->\n    {DB1, ScrubbedData} =\n        write_scrubbed_to_db(?TEST_DB:new(randoms:getRandomString()),\n                             Data),\n    check_db(DB1, ScrubbedData, \"check_db_put1\"),\n    ?TEST_DB:?CLOSE(DB1),\n    true.\n\ntester_put(_Config) ->\n    tester:test(?MODULE, prop_put, 1, rw_suite_runs(10000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% test get/2 of available backends\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_get([db_backend_beh:entry()]) -> true.\nprop_get(Data) ->\n    {DB1, ScrubbedData} =\n        write_scrubbed_to_db(?TEST_DB:new(randoms:getRandomString()), Data),\n    GetData = lists:foldl(\n            fun(Entry, AccIn) ->\n                [?TEST_DB:get(DB1, element(1, Entry)) | AccIn]\n            end, [], ScrubbedData),\n    compare_lists(ScrubbedData, GetData, \"check_db_put1\"),\n    check_db(DB1, GetData, \"check_db_put1\"),\n    ?TEST_DB:?CLOSE(DB1),\n    true.\n\ntester_get(_Config) ->\n    tester:test(?MODULE, prop_get, 1, rw_suite_runs(10000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% test delete/2 of available backends\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_delete([db_backend_beh:entry()], [db_backend_beh:key()]) -> true.\nprop_delete(Data, ToDelete) ->\n    {DB1, ScrubbedData} =\n        write_scrubbed_to_db(?TEST_DB:new(randoms:getRandomString()), Data),\n    DB2 = lists:foldl(\n            fun(Key, DBAcc) ->\n                ?TEST_DB:delete(DBAcc, Key)\n            end, DB1, ToDelete),\n    ExpData = [El || El <- ScrubbedData,\n                     not lists:any(fun(Key) -> element(1, El) ?EQ Key end,\n                                   ToDelete)],\n    check_db(DB2, ExpData, \"check_db_delete\"),\n    ?TEST_DB:?CLOSE(DB2),\n    true.\n\ntester_delete(_Config) ->\n    tester:test(?MODULE, prop_delete, 2, rw_suite_runs(10000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% test foldl/2 of available backends\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_foldl([db_backend_beh:entry()], db_backend_beh:interval(),\n                 non_neg_integer()) -> true.\nprop_foldl(Data, Interval, MaxNum) ->\n    {DB1, ScrubbedData} =\n        write_scrubbed_to_db(?TEST_DB:new(randoms:getRandomString()), Data),\n    ExpInInterval = [El || El <- ScrubbedData,\n                           is_in(Interval, element(1, El))],\n    ExpInIntervalCounted = lists:sublist(ExpInInterval, MaxNum),\n    %% We have certain combination of keys (e.g. {0} and {0.0}]) which are returned\n    %% in an unspecified order by certain DBs (lets call that a sequenze). If MaxNum\n    %% is set to a value which would split such a sequenz the test might fail because\n    %% the DBs returns unexpected keys. In such case all remaining elements of the sequenze\n    %% are added as well (which effectivly increases MaxNum).\n    ExpInIntervalCountedTailed =\n        case MaxNum >= length(ExpInInterval) orelse MaxNum == 0 of\n            true ->\n                ExpInIntervalCounted;\n            _ ->\n                Tail = lists:takewhile(fun(E) ->\n                                            element(1, lists:last(ExpInIntervalCounted)) ==\n                                            element(1, E)\n                                       end, lists:nthtail(MaxNum, ExpInInterval)),\n                lists:append(ExpInIntervalCounted, Tail)\n        end,\n\n    AllFold = ?TEST_DB:foldl(DB1, fun(K, AccIn) -> [?TEST_DB:get(DB1, K) | AccIn] end, []),\n    IntervalFold = ?TEST_DB:foldl(DB1,\n                                 fun(K, AccIn) -> [?TEST_DB:get(DB1, K) | AccIn] end,\n                                 [],\n                                 Interval),\n    IntervalCountFold = ?TEST_DB:foldl(DB1,\n                            fun(K, AccIn) -> [?TEST_DB:get(DB1, K) | AccIn] end,\n                            [],\n                            Interval,\n                            length(ExpInIntervalCountedTailed)),\n    UnorderedFold = ?TEST_DB:foldl_unordered(DB1, fun(E, AccIn) -> [E | AccIn] end, []),\n    %% we expect all data from fold to be in reversed order because of list\n    %% accumulation. we need to reverse ScrubbedData, ExpInInterval and\n    %% ExpInIntervalCounted.\n    compare_lists(ScrubbedData, AllFold, \"test_foldl1\"),\n    compare_lists(ExpInInterval, IntervalFold, \"test_foldl2\"),\n    compare_lists(ExpInIntervalCountedTailed, IntervalCountFold, \"test_foldl3\"),\n    compare_lists(ScrubbedData, UnorderedFold, \"test_foldl4\"),\n    ?TEST_DB:?CLOSE(DB1),\n    true.\n\ntester_foldl(_Config) ->\n    prop_foldl([{0.0,{{},[0.0],-3},[],one,{}}, {0,[{}]}], all, 1),\n    prop_foldl([{0.0}, {0}, {0.0}], {'(', {'*'}, {0}, ']'}, 4),\n    prop_foldl([{foo}, {\"bar\"}, {42},{-1},{{\"foobar\"}},{{{0.3820862051907793}}},{0.39125416350624936}], all, 2),\n    prop_foldl([{42},{-1},{{{0.3820862051907793}}},{0.39125416350624936}], all, 2),\n    prop_foldl([{{42,-5,{0,{},{-1,{{},[5],42},[[[]]]}}}},{[5]},{0.0}], all, 3),\n    tester:test(?MODULE, prop_foldl, 3, rw_suite_runs(10000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% test foldr/2 of available backends\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_foldr([db_backend_beh:entry()], db_backend_beh:interval(),\n                 non_neg_integer()) -> true.\nprop_foldr(Data, Interval, MaxNum) ->\n    {DB1, ScrubbedData} =\n        write_scrubbed_to_db(?TEST_DB:new(randoms:getRandomString()), Data),\n    ExpInInterval = [El || El <- ScrubbedData,\n                           is_in(Interval, element(1, El))],\n    ExpInIntervalCounted =\n        case MaxNum >= length(ExpInInterval) of\n            true ->\n                ExpInInterval;\n            _ ->\n                EndIndex = length(ExpInInterval) - MaxNum,\n                Counted = lists:nthtail(EndIndex, ExpInInterval),\n                %% The same reason as for prop_foldl, but instead of adding, drop\n                %% all elements of the splitted sequence.\n                case MaxNum == 0 of\n                    true ->\n                        Counted;\n                    _ ->\n                        lists:dropwhile(fun(E) ->\n                                element(1, lists:nth(EndIndex, ExpInInterval)) ==\n                                element(1, E)\n                        end, Counted)\n                end\n        end,\n\n    AllFold = ?TEST_DB:foldr(DB1, fun(K, AccIn) -> [?TEST_DB:get(DB1, K) | AccIn] end, []),\n    IntervalFold = ?TEST_DB:foldr(DB1,\n                                 fun(K, AccIn) -> [?TEST_DB:get(DB1, K) | AccIn] end,\n                                 [],\n                                 Interval),\n    IntervalCountFold = ?TEST_DB:foldr(DB1,\n                            fun(K, AccIn) -> [?TEST_DB:get(DB1, K) | AccIn] end,\n                            [],\n                            Interval,\n                            length(ExpInIntervalCounted)),\n    %% ct:pal(\"ExpInInterval: ~p~nIntervalFold: ~p~nInterval: ~p~n\", [ExpInInterval,\n    %%                                                  IntervalFold, Interval]),\n    compare_lists(ScrubbedData, AllFold, \"test_foldr1\"),\n    compare_lists(ExpInInterval, IntervalFold, \"test_foldr2\"),\n    compare_lists(ExpInIntervalCounted, IntervalCountFold, \"test_foldr3\"),\n    ?TEST_DB:?CLOSE(DB1),\n    true.\n\ntester_foldr(_Config) ->\n    prop_foldr([{{0.0},[],[],42,42,[],one,{}},{{0},[{}]}], all, 1),\n    tester:test(?MODULE, prop_foldr, 3, rw_suite_runs(10000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% helper functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nwrite_scrubbed_to_db(DB, Data) ->\n    ScrubbedData = scrub_data(Data),\n    DB1 = lists:foldl(\n            fun(Entry, DBAcc) ->\n                ?TEST_DB:put(DBAcc, Entry)\n            end, DB, ScrubbedData),\n    {DB1, ScrubbedData}.\n\nscrub_data(Data) ->\n    %% Entries should be unique\n    SortFun = fun(A, B) -> ((element(1, A) < element(1, B)) orelse\n                            (element(1, A) ?EQ element(1, B)))\n                            orelse\n                            (is_integer(element(1, A))\n                            andalso is_float(element(1, B))\n                            andalso element(1, A) == element(1, B))\n              end,\n    lists:usort(SortFun, lists:reverse(Data)).\n\ncheck_db(DB, ExpData, Note) ->\n    InDb = ?TEST_DB:foldl(DB, fun(K, AIn) -> [?TEST_DB:get(DB, K) | AIn] end, []),\n    compare_lists(InDb, ExpData, Note),\n    ?equals_w_note(?TEST_DB:get_load(DB), length(ExpData), Note).\n\ncompare_lists(List1, List2, Note) ->\n    ?equals_w_note(length(List1), length(List2), \"length of lists \" ++ Note),\n    ?compare_w_note(fun eq_lenient/2, scrub_data(List1), scrub_data(List2), \"content of lists \" ++\n                   Note).\n\nis_in({Key}, OtherKey) -> Key ?EQ OtherKey;\nis_in(all, _Key) -> true;\nis_in({'(', L, R, ')'}, Key) -> Key > L andalso Key < R;\nis_in({'(', L, R, ']'}, Key) -> Key > L andalso ((Key < R) orelse (Key ?EQ R));\nis_in({'[', L, R, ')'}, Key) -> ((Key > L) orelse (Key ?EQ L)) andalso Key < R;\nis_in({'[', L, R, ']'}, Key) -> ((Key > L) orelse (Key ?EQ L)) andalso ((Key < R) orelse (Key ?EQ R)).\n\n%% Test two lists L1 and L2 for equality while taking into consideration that elements E1, E2 might be\n%% out of order iff E1 == E2 but E1 =/= E2 (eg. {0.0} and {0})\neq_lenient(L1, L2) -> eq_lenient(L1, [], L2).\n\neq_lenient([], [], []) -> true;\neq_lenient([H1|L1], [], [H2|L2]) when H1 ?EQ H2 ->\n    eq_lenient(L1, [], L2);\neq_lenient([H1|L1], Pref, [H2|L2]) when H1 ?EQ H2 ->\n    eq_lenient(L1, [], lists:append(Pref, L2));\neq_lenient([H1|L1], Pref, [H2|L2]) when element(1, H1) == element(1, H2) ->\n    eq_lenient([H1|L1], [H2|Pref], L2);\neq_lenient(_L1, _Pref, _L2) -> false.\n"
  },
  {
    "path": "test/db_bitcask_SUITE.erl",
    "content": "% @copyright 2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for the bitcask backend.\n%% @end\n%% @version $Id$\n-module(db_bitcask_SUITE).\n\n-author('schuett@zib.de').\n\n-compile(export_all).\n\n-define(TEST_DB, db_bitcask).\n-define(CLOSE, close).\n-define(EQ, =:=).\n\n-include(\"db_backend_SUITE.hrl\").\n\nall() ->\n    case code:which(bitcask) of\n        non_existing -> [];\n        _            -> tests_avail()\n    end.\n\nsuite() -> [ {timetrap, {seconds, 60}} ].\n\ninit_per_suite(Config) ->\n    %% need config here to get db path\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_symmetric_ring([{config, [{log_path, PrivDir}]}]),\n    unittest_helper:check_ring_size_fully_joined(config:read(replication_factor)),\n\n    %% cleanup schema generated possibly in earlier failed run\n    PWD = os:cmd(pwd),\n    WorkingDir = string:sub_string(PWD, 1, string:len(PWD) - 1) ++\n        \"/\" ++ config:read(db_directory) ++ \"/\" ++ atom_to_list(erlang:node()) ++ \"/\",\n    cleanup(WorkingDir),\n\n    tester:register_type_checker({typedef, db_backend_beh, key, []}, db_backend_beh, tester_is_valid_db_key),\n    tester:register_value_creator({typedef, db_backend_beh, key, []}, db_backend_beh, tester_create_db_key, 1),\n    Config.\n\nend_per_suite(_Config) ->\n    tester:unregister_type_checker({typedef, db_backend_beh, key, []}),\n    tester:unregister_value_creator({typedef, db_backend_beh, key, []}),\n\n    %% cleanup schema generated in this run\n    PWD = os:cmd(pwd),\n    WorkingDir = string:sub_string(PWD, 1, string:len(PWD) - 1) ++\n        \"/\" ++ config:read(db_directory) ++ \"/\" ++ atom_to_list(erlang:node()) ++ \"/\",\n    cleanup(WorkingDir),\n\n    _ = unittest_helper:stop_ring(),\n    ok.\n\nrw_suite_runs(N) ->\n    erlang:min(N, 200).\n\n\ncleanup(Path) ->\n    Re = \".*\",\n    Files = filelib:fold_files(Path, Re, true, fun(File, Acc) -> [File | Acc] end, []),\n    _ = [file:delete(File) || File <- Files],\n    ok.\n"
  },
  {
    "path": "test/db_dht_SUITE.erl",
    "content": "% @copyright 2010-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Unit tests for src/db_dht.erl.\n%% @end\n%% @version $Id$\n-module(db_dht_SUITE).\n\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n-include(\"unittest.hrl\").\n\ngroups() ->\n    [{hand_written_tests, [sequence], [\n                                       read,\n                                       write,\n                                       delete,\n                                       get_load_and_middle,\n                                       split_data,\n                                       update_entries,\n                                       changed_keys\n                                      ]},\n     {tester_tests, [sequence], [\n                                 tester_new, tester_set_entry, tester_update_entry,\n                                 tester_delete_entry1, tester_delete_entry2,\n                                 tester_write,\n                                 tester_delete, tester_add_data,\n                                 tester_get_entries2, tester_get_entries3_1,\n                                 tester_get_entries3_2,\n                                 tester_get_load2,\n                                 tester_split_data, tester_update_entries,\n                                 tester_delete_entries1, tester_delete_entries2,\n                                 tester_changed_keys_get_entry,\n                                 tester_changed_keys_set_entry,\n                                 tester_changed_keys_update_entry,\n                                 tester_changed_keys_delete_entry,\n                                 tester_changed_keys_read,\n                                 tester_changed_keys_write,\n                                 tester_changed_keys_delete,\n                                 tester_changed_keys_get_entries2,\n                                 tester_changed_keys_get_entries4,\n                                 tester_get_chunk_precond,\n                                 tester_get_chunk4,\n                                 tester_get_split_key5,\n                                 tester_changed_keys_update_entries,\n                                 tester_changed_keys_delete_entries1,\n                                 tester_changed_keys_delete_entries2,\n                                 tester_changed_keys_get_load,\n                                 tester_changed_keys_get_load2,\n                                 tester_changed_keys_split_data1,\n                                 tester_changed_keys_split_data2,\n                                 tester_changed_keys_get_data,\n                                 tester_changed_keys_add_data,\n                                 tester_changed_keys_check_db,\n                                 tester_changed_keys_mult_interval,\n                                 tester_stop_record_changes]}].\n\nall() ->\n    [\n     {group, hand_written_tests},\n     {group, tester_tests}\n    ].\n\n\nsuite() -> [ {timetrap, {seconds, 15}} ].\n\n-define(KEY(Key), ?RT:hash_key(Key)).\n-define(VALUE(Val), rdht_tx:encode_value(Val)).\n\n%% @doc Specify how often a read/write suite can be executed in order not to\n%%      hit a timeout (depending on the speed of the DB implementation).\n-spec max_rw_tests_per_suite() -> pos_integer().\nmax_rw_tests_per_suite() ->\n    10000.\n\n%% @doc Returns the min of Desired and max_rw_tests_per_suite().\n%%      Should be used to limit the number of executions of r/w suites.\n-spec rw_suite_runs(Desired::pos_integer()) -> pos_integer().\nrw_suite_runs(Desired) ->\n    erlang:min(Desired, max_rw_tests_per_suite()).\n\ninit_per_suite(Config) ->\n    Config3 = unittest_helper:start_minimal_procs(Config, [], false),\n    tester:register_type_checker({typedef, intervals, interval, []}, intervals, is_well_formed),\n    tester:register_value_creator({typedef, intervals, interval, []}, intervals, tester_create_interval, 1),\n    Config3.\n\nend_per_suite(Config) ->\n    tester:unregister_type_checker({typedef, intervals, interval, []}),\n    tester:unregister_value_creator({typedef, intervals, interval, []}),\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(_TestCase, Config) ->\n    pid_groups:join_as(ct_tests, ?MODULE),\n    Config.\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n-define(db_equals_pattern(Actual, ExpectedPattern),\n        % wrap in function so that the internal variables are out of the calling function's scope\n        fun() ->\n                case Actual of\n                    {DB_EQUALS_PATTERN_DB, ExpectedPattern} -> DB_EQUALS_PATTERN_DB;\n                    {DB_EQUALS_PATTERN_DB, Any} ->\n                        ct:pal(\"Failed: Stacktrace ~p~n\",\n                               [util:get_stacktrace()]),\n                        ?ct_fail(\"~p evaluated to \\\"~p\\\" which is \"\n                               \"not the expected ~p\",\n                               [??Actual, Any, ??ExpectedPattern]),\n                        DB_EQUALS_PATTERN_DB\n                end\n        end()).\n\nread(_Config) ->\n    prop_new(?KEY(\"Unknown\")).\n\nwrite(_Config) ->\n    prop_write(?KEY(\"Key1\"), ?VALUE(\"Value1\"), 1, ?KEY(\"Key2\")).\n\ndelete(_Config) ->\n    prop_delete(?KEY(\"DeleteKey1\"), ?VALUE(\"Value1\"), false, 0, 1, ?KEY(\"DeleteKey2\")),\n    prop_delete(?KEY(\"DeleteKey1\"), ?VALUE(\"Value1\"), false, 1, 1, ?KEY(\"DeleteKey2\")).\n\nget_load_and_middle(_Config) ->\n    DB = db_dht:new(db_dht),\n    ?equals(db_dht:get_load(DB), 0),\n    DB2 = db_dht:write(DB, rt_SUITE:number_to_key(1), ?VALUE(\"Value1\"), 1),\n    ?equals(db_dht:get_load(DB2), 1),\n    DB3 = db_dht:write(DB2, rt_SUITE:number_to_key(1), ?VALUE(\"Value1\"), 2),\n    ?equals(db_dht:get_load(DB3), 1),\n    DB4 = db_dht:write(DB3, rt_SUITE:number_to_key(2), ?VALUE(\"Value2\"), 1),\n    ?equals(db_dht:get_load(DB4), 2),\n    DB5 = db_dht:write(DB4, rt_SUITE:number_to_key(3), ?VALUE(\"Value3\"), 1),\n    DB6 = db_dht:write(DB5, rt_SUITE:number_to_key(4), ?VALUE(\"Value4\"), 1),\n    OrigFullList = db_dht:get_data(DB6),\n    {DB7, HisList} = db_dht:split_data(DB6, node:mk_interval_between_ids(rt_SUITE:number_to_key(2), rt_SUITE:number_to_key(4))),\n    ?equals(db_dht:read(DB7, rt_SUITE:number_to_key(3)), {ok, ?VALUE(\"Value3\"), 1}),\n    ?equals(db_dht:read(DB7, rt_SUITE:number_to_key(4)), {ok, ?VALUE(\"Value4\"), 1}),\n    ?equals(db_dht:get_load(DB7), 2),\n    ?equals(length(HisList), 2),\n    ?equals(length(db_dht:get_data(DB7)), 2),\n    DB8 = db_dht:add_data(DB7, HisList),\n    % lists could be in arbitrary order -> sort them\n    ?equals(lists:sort(OrigFullList), lists:sort(db_dht:get_data(DB8))),\n    db_dht:close(DB8).\n\n%% @doc Some split_data tests using fixed values.\n%% @see prop_split_data/2\nsplit_data(_Config) ->\n    prop_split_data([db_entry:new(rt_SUITE:number_to_key(1), ?VALUE(\"Value1\"), 1),\n                     db_entry:new(rt_SUITE:number_to_key(2), ?VALUE(\"Value2\"), 2)],\n                    intervals:empty()),\n    prop_split_data([db_entry:new(rt_SUITE:number_to_key(1), ?VALUE(\"Value1\"), 1),\n                     db_entry:new(rt_SUITE:number_to_key(2), ?VALUE(\"Value2\"), 2)],\n                    intervals:all()),\n    prop_split_data([db_entry:new(rt_SUITE:number_to_key(1), ?VALUE(\"Value1\"), 1),\n                     db_entry:new(rt_SUITE:number_to_key(2), ?VALUE(\"Value2\"), 2)],\n                    intervals:new(rt_SUITE:number_to_key(2))),\n    prop_split_data([db_entry:new(rt_SUITE:number_to_key(1), ?VALUE(\"Value1\"), 1),\n                     db_entry:new(rt_SUITE:number_to_key(2), ?VALUE(\"Value2\"), 2)],\n                    intervals:new(rt_SUITE:number_to_key(5))),\n    prop_split_data([db_entry:new(rt_SUITE:number_to_key(1), ?VALUE(\"Value1\"), 1),\n                     db_entry:new(rt_SUITE:number_to_key(2), ?VALUE(\"Value2\"), 2),\n                     db_entry:new(rt_SUITE:number_to_key(3), ?VALUE(\"Value3\"), 3),\n                     db_entry:new(rt_SUITE:number_to_key(4), ?VALUE(\"Value4\"), 4),\n                     db_entry:new(rt_SUITE:number_to_key(5), ?VALUE(\"Value5\"), 5)],\n                    intervals:new('[', rt_SUITE:number_to_key(2), rt_SUITE:number_to_key(4), ')')),\n    prop_split_data([db_entry:new(rt_SUITE:number_to_key(1), ?VALUE(\"Value1\"), 1),\n                     db_entry:new(rt_SUITE:number_to_key(2), ?VALUE(\"Value2\"), 2),\n                     db_entry:new(rt_SUITE:number_to_key(3), ?VALUE(\"Value3\"), 3),\n                     db_entry:new(rt_SUITE:number_to_key(4), ?VALUE(\"Value4\"), 4),\n                     db_entry:new(rt_SUITE:number_to_key(5), ?VALUE(\"Value5\"), 5)],\n                    intervals:union(intervals:new('[', rt_SUITE:number_to_key(1), rt_SUITE:number_to_key(3), ')'),\n                                    intervals:new(rt_SUITE:number_to_key(4)))),\n    prop_split_data([db_entry:set_writelock(db_entry:new(rt_SUITE:number_to_key(1), ?VALUE(\"Value1\"), 1), 1),\n                     db_entry:inc_readlock(db_entry:new(rt_SUITE:number_to_key(2), ?VALUE(\"Value2\"), 2)),\n                     db_entry:new(rt_SUITE:number_to_key(3), ?VALUE(\"Value3\"), 3),\n                     db_entry:new(rt_SUITE:number_to_key(4), ?VALUE(\"Value4\"), 4),\n                     db_entry:new(rt_SUITE:number_to_key(5), ?VALUE(\"Value5\"), 5)],\n                    intervals:union(intervals:new('[', rt_SUITE:number_to_key(1), rt_SUITE:number_to_key(3), ')'),\n                                    intervals:new(rt_SUITE:number_to_key(4)))).\n\n%% @doc Some update_entries tests using fixed values.\n%% @see prop_update_entries_helper/3\nupdate_entries(_Config) ->\n    prop_update_entries_helper([db_entry:new(?KEY(\"1\"), ?VALUE(\"Value1\"), 1),\n                                db_entry:new(?KEY(\"2\"), ?VALUE(\"Value2\"), 1),\n                                db_entry:new(?KEY(\"3\"), ?VALUE(\"Value3\"), 1),\n                                db_entry:new(?KEY(\"4\"), ?VALUE(\"Value4\"), 1),\n                                db_entry:new(?KEY(\"5\"), ?VALUE(\"Value5\"), 1)],\n                               [db_entry:new(?KEY(\"1\"), ?VALUE(\"Value1\"), 2),\n                                db_entry:new(?KEY(\"2\"), ?VALUE(\"Value2\"), 2),\n                                db_entry:new(?KEY(\"3\"), ?VALUE(\"Value3\"), 2),\n                                db_entry:new(?KEY(\"4\"), ?VALUE(\"Value4\"), 2),\n                                db_entry:new(?KEY(\"5\"), ?VALUE(\"Value5\"), 2)],\n                               [db_entry:new(?KEY(\"1\"), ?VALUE(\"Value1\"), 2),\n                                db_entry:new(?KEY(\"2\"), ?VALUE(\"Value2\"), 2),\n                                db_entry:new(?KEY(\"3\"), ?VALUE(\"Value3\"), 2),\n                                db_entry:new(?KEY(\"4\"), ?VALUE(\"Value4\"), 2),\n                                db_entry:new(?KEY(\"5\"), ?VALUE(\"Value5\"), 2)]),\n    prop_update_entries_helper([db_entry:new(?KEY(\"1\"), ?VALUE(\"Value1\"), 1),\n                                db_entry:new(?KEY(\"2\"), ?VALUE(\"Value2\"), 1),\n                                db_entry:new(?KEY(\"3\"), ?VALUE(\"Value3\"), 1),\n                                db_entry:new(?KEY(\"4\"), ?VALUE(\"Value4\"), 1),\n                                db_entry:new(?KEY(\"5\"), ?VALUE(\"Value5\"), 1)],\n                               [db_entry:new(?KEY(\"1\"), ?VALUE(\"Value1\"), 2),\n                                db_entry:new(?KEY(\"4\"), ?VALUE(\"Value4\"), 2),\n                                db_entry:new(?KEY(\"5\"), ?VALUE(\"Value5\"), 3)],\n                               [db_entry:new(?KEY(\"1\"), ?VALUE(\"Value1\"), 2),\n                                db_entry:new(?KEY(\"2\"), ?VALUE(\"Value2\"), 1),\n                                db_entry:new(?KEY(\"3\"), ?VALUE(\"Value3\"), 1),\n                                db_entry:new(?KEY(\"4\"), ?VALUE(\"Value4\"), 2),\n                                db_entry:new(?KEY(\"5\"), ?VALUE(\"Value5\"), 3)]),\n    prop_update_entries_helper([db_entry:new(?KEY(\"1\"), ?VALUE(\"Value1\"), 2),\n                                db_entry:new(?KEY(\"2\"), ?VALUE(\"Value2\"), 2),\n                                db_entry:new(?KEY(\"3\"), ?VALUE(\"Value3\"), 2),\n                                db_entry:new(?KEY(\"4\"), ?VALUE(\"Value4\"), 2),\n                                db_entry:new(?KEY(\"5\"), ?VALUE(\"Value5\"), 2)],\n                               [db_entry:new(?KEY(\"1\"), ?VALUE(\"Value1\"), 1),\n                                db_entry:new(?KEY(\"4\"), ?VALUE(\"Value4\"), 1),\n                                db_entry:new(?KEY(\"5\"), ?VALUE(\"Value5\"), 1)],\n                               [db_entry:new(?KEY(\"1\"), ?VALUE(\"Value1\"), 2),\n                                db_entry:new(?KEY(\"2\"), ?VALUE(\"Value2\"), 2),\n                                db_entry:new(?KEY(\"3\"), ?VALUE(\"Value3\"), 2),\n                                db_entry:new(?KEY(\"4\"), ?VALUE(\"Value4\"), 2),\n                                db_entry:new(?KEY(\"5\"), ?VALUE(\"Value5\"), 2)]),\n    prop_update_entries_helper([db_entry:set_writelock(db_entry:new(?KEY(\"1\"), ?VALUE(\"Value1\"), 1), 1),\n                                db_entry:inc_readlock(db_entry:new(?KEY(\"2\"), ?VALUE(\"Value2\"), 2)),\n                                db_entry:new(?KEY(\"3\"), ?VALUE(\"Value3\"), 1),\n                                db_entry:new(?KEY(\"4\"), ?VALUE(\"Value4\"), 1),\n                                db_entry:new(?KEY(\"5\"), ?VALUE(\"Value5\"), 1)],\n                               [db_entry:new(?KEY(\"1\"), ?VALUE(\"Value1\"), 2),\n                                db_entry:new(?KEY(\"2\"), ?VALUE(\"Value2\"), 2),\n                                db_entry:new(?KEY(\"3\"), ?VALUE(\"Value3\"), 2),\n                                db_entry:new(?KEY(\"4\"), ?VALUE(\"Value4\"), 2),\n                                db_entry:new(?KEY(\"5\"), ?VALUE(\"Value5\"), 2)],\n                               [db_entry:set_writelock(db_entry:new(?KEY(\"1\"), ?VALUE(\"Value1\"), 1), 1),\n                                db_entry:inc_readlock(db_entry:new(?KEY(\"2\"), ?VALUE(\"Value2\"), 2)),\n                                db_entry:new(?KEY(\"3\"), ?VALUE(\"Value3\"), 2),\n                                db_entry:new(?KEY(\"4\"), ?VALUE(\"Value4\"), 2),\n                                db_entry:new(?KEY(\"5\"), ?VALUE(\"Value5\"), 2)]),\n    prop_update_entry({?KEY(\"239309376718523519117394992299371645018\"), empty_val, false, 0, -1},\n                      <<6>>, false, 7, 4).\n\nchanged_keys(_Config) ->\n    DB = db_dht:new(db_dht),\n\n    ?equals(db_dht:get_changes(DB), {[], []}),\n\n    DB2 = db_dht:stop_record_changes(DB),\n    ?equals(db_dht:get_changes(DB2), {[], []}),\n\n    DB3 = db_dht:record_changes(DB2, intervals:empty()),\n    ?equals(db_dht:get_changes(DB3), {[], []}),\n\n    db_dht:close(DB3).\n\n% tester-based functions below:\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:new/0, db_dht getters\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_new(Key::?RT:key()) -> true.\nprop_new(Key) ->\n    DB = db_dht:new(db_dht),\n    check_db(DB, {true, []}, 0, [], \"check_db_new_1\"),\n    ?equals(db_dht:read(DB, Key), {ok, empty_val, -1}),\n    check_entry(DB, Key, db_entry:new(Key), {ok, empty_val, -1}, false, \"check_entry_new_1\"),\n    db_dht:close(DB),\n    true.\n\ntester_new(_Config) ->\n    tester:test(?MODULE, prop_new, 1, rw_suite_runs(10), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:set_entry/2, db_dht getters\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_set_entry(DBEntry::db_entry:entry()) -> true.\nprop_set_entry(DBEntry) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:set_entry(DB, DBEntry),\n    IsNullEntry = db_entry:is_null(DBEntry),\n    check_entry(DB2, db_entry:get_key(DBEntry), DBEntry,\n                {ok, db_entry:get_value(DBEntry), db_entry:get_version(DBEntry)},\n                not IsNullEntry, \"check_entry_set_entry_1\"),\n    case not db_entry:is_empty(DBEntry) andalso\n             not (db_entry:get_writelock(DBEntry) =/= false andalso db_entry:get_readlock(DBEntry) > 0) andalso\n             db_entry:get_version(DBEntry) >= 0 of\n        true -> check_db(DB2, {true, []}, 1, [DBEntry], \"check_db_set_entry_0\");\n        _ when IsNullEntry ->\n                check_db(DB2, {true, []}, 0, [], \"check_db_set_entry_1\");\n        _    -> check_db(DB2, {false, [DBEntry]}, 1, [DBEntry], \"check_db_set_entry_2\")\n    end,\n    db_dht:close(DB2),\n    true.\n\ntester_set_entry(_Config) ->\n    tester:test(?MODULE, prop_set_entry, 1, rw_suite_runs(10), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:update_entry/2, db_dht getters\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_update_entry(DBEntry1::db_entry:entry(), Value2::db_dht:value(), WriteLock2::boolean(),\n                        ReadLock2::0..10, Version2::client_version()) -> true.\nprop_update_entry(DBEntry1, Value2, WriteLock2, ReadLock2, Version2) ->\n    DBEntry2 = create_db_entry(db_entry:get_key(DBEntry1), Value2, WriteLock2, ReadLock2, Version2),\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:set_entry(DB, DBEntry1),\n    case db_entry:is_null(DBEntry1) of\n        true -> % update not possible\n            DB3 = DB2,\n            ok;\n        false ->\n            DB3 = db_dht:update_entry(DB2, DBEntry2),\n            IsNullEntry = db_entry:is_null(DBEntry2),\n            check_entry(DB3, db_entry:get_key(DBEntry2), DBEntry2,\n                        {ok, db_entry:get_value(DBEntry2), db_entry:get_version(DBEntry2)},\n                        not IsNullEntry, \"check_entry_update_entry_1\"),\n            case not db_entry:is_empty(DBEntry2) andalso\n                     not (db_entry:get_writelock(DBEntry2) =/= false andalso db_entry:get_readlock(DBEntry2) > 0) andalso\n                     db_entry:get_version(DBEntry2) >= 0 of\n                true -> check_db(DB3, {true, []}, 1, [DBEntry2], \"check_db_update_entry_0\");\n                _ when IsNullEntry ->\n                    check_db(DB3, {true, []}, 0, [], \"check_db_update_entry_1\");\n                _    -> check_db(DB3, {false, [DBEntry2]}, 1, [DBEntry2], \"check_db_update_entry_2\")\n            end\n    end,\n    db_dht:close(DB3),\n    true.\n\ntester_update_entry(_Config) ->\n    tester:test(?MODULE, prop_update_entry, 5, rw_suite_runs(10), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:delete_entry/2, db_dht getters\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_delete_entry1(DBEntry1::db_entry:entry()) -> true.\nprop_delete_entry1(DBEntry1) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:set_entry(DB, DBEntry1),\n    DB3 = db_dht:delete_entry(DB2, DBEntry1),\n    check_entry(DB3, db_entry:get_key(DBEntry1), db_entry:new(db_entry:get_key(DBEntry1)),\n                {ok, empty_val, -1}, false, \"check_entry_delete_entry1_1\"),\n    check_db(DB3, {true, []}, 0, [], \"check_db_delete_entry1_1\"),\n    db_dht:close(DB3),\n    true.\n\n-spec prop_delete_entry2(DBEntry1::db_entry:entry(), DBEntry2::db_entry:entry()) -> true.\nprop_delete_entry2(DBEntry1, DBEntry2) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:set_entry(DB, DBEntry1),\n    % note: DBEntry2 may not be the same\n    DB3 = db_dht:delete_entry(DB2, DBEntry2),\n    case db_entry:get_key(DBEntry1) =/= db_entry:get_key(DBEntry2) of\n        true ->\n            IsNullEntry = db_entry:is_null(DBEntry1),\n            check_entry(DB3, db_entry:get_key(DBEntry1), DBEntry1,\n                {ok, db_entry:get_value(DBEntry1), db_entry:get_version(DBEntry1)},\n                not IsNullEntry, \"check_entry_delete_entry2_1\"),\n            case not db_entry:is_empty(DBEntry1) andalso\n                     not (db_entry:get_writelock(DBEntry1) =/= false andalso db_entry:get_readlock(DBEntry1) > 0) andalso\n                     db_entry:get_version(DBEntry1) >= 0 of\n                true -> check_db(DB3, {true, []}, 1, [DBEntry1], \"check_db_delete_entry2_1a\");\n                _ when IsNullEntry ->\n                        check_db(DB3, {true, []}, 0, [], \"check_db_delete_entry2_1b\");\n                _    -> check_db(DB3, {false, [DBEntry1]}, 1, [DBEntry1], \"check_db_delete_entry2_1c\")\n            end;\n        _    ->\n            check_entry(DB3, db_entry:get_key(DBEntry1), db_entry:new(db_entry:get_key(DBEntry1)),\n                        {ok, empty_val, -1}, false, \"check_entry_delete_entry2_2\"),\n            check_db(DB3, {true, []}, 0, [], \"check_db_delete_entry2_2\")\n    end,\n    db_dht:close(DB3),\n    true.\n\ntester_delete_entry1(_Config) ->\n    tester:test(?MODULE, prop_delete_entry1, 1, rw_suite_runs(10), [{threads, 2}]).\n\ntester_delete_entry2(_Config) ->\n    tester:test(?MODULE, prop_delete_entry2, 2, rw_suite_runs(10), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:write/2, db_dht getters\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_write(Key::?RT:key(), Value::db_dht:value(), Version::client_version(), Key2::?RT:key()) -> true.\nprop_write(Key, Value, Version, Key2) ->\n    DBEntry = db_entry:new(Key, Value, Version),\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:write(DB, Key, Value, Version),\n    check_entry(DB2, Key, DBEntry, {ok, Value, Version}, true, \"check_entry_write_1\"),\n    check_db(DB2, {true, []}, 1, [DBEntry], \"check_db_write_1\"),\n    case Key =/= Key2 of\n        true -> check_entry(DB2, Key2, db_entry:new(Key2), {ok, empty_val, -1}, false, \"write_2\");\n        _    -> ok\n    end,\n    db_dht:close(DB2),\n    true.\n\ntester_write(_Config) ->\n    tester:test(?MODULE, prop_write, 4, rw_suite_runs(1000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:delete/2, also validate using different getters\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_delete(Key::?RT:key(), Value::db_dht:value(), WriteLock::boolean(),\n                  ReadLock::0..10, Version::client_version(), Key2::?RT:key()) -> true.\nprop_delete(Key, Value, WriteLock, ReadLock, Version, Key2) ->\n    DB = db_dht:new(db_dht),\n    DBEntry = create_db_entry(Key, Value, WriteLock, ReadLock, Version),\n    DB2 = db_dht:set_entry(DB, DBEntry),\n\n    % delete DBEntry:\n    DB3 =\n        case db_entry:is_locked(DBEntry) of\n            true ->\n                DBA1 = ?db_equals_pattern(db_dht:delete(DB2, Key), locks_set),\n                check_entry(DBA1, Key, DBEntry, {ok, Value, Version}, true, \"check_entry_delete_1a\"),\n                case not db_entry:is_empty(DBEntry) andalso\n                         not (db_entry:get_writelock(DBEntry) =/= false andalso db_entry:get_readlock(DBEntry) > 0) andalso\n                         db_entry:get_version(DBEntry) >= 0 of\n                    true -> check_db(DBA1, {true, []}, 1, [DBEntry], \"check_db_delete_1ax\");\n                    _    -> check_db(DBA1, {false, [DBEntry]}, 1, [DBEntry], \"check_db_delete_1ay\")\n                end,\n                case Key =/= Key2 of\n                    true ->\n                        DBTmp = ?db_equals_pattern(db_dht:delete(DBA1, Key2), undef),\n                        check_entry(DBTmp, Key, DBEntry, {ok, Value, Version}, true, \"check_entry_delete_2a\"),\n                        case not db_entry:is_empty(DBEntry) andalso\n                                 not (db_entry:get_writelock(DBEntry) =/= false andalso db_entry:get_readlock(DBEntry) > 0) andalso\n                                 db_entry:get_version(DBEntry) >= 0 of\n                            true -> check_db(DBTmp, {true, []}, 1, [DBEntry], \"check_db_delete_2ax\");\n                            _    -> check_db(DBTmp, {false, [DBEntry]}, 1, [DBEntry], \"check_db_delete_2ay\")\n                        end,\n                        DBTmp;\n                    _ -> DBA1\n                end;\n            _ ->\n                DBA1 = ?db_equals_pattern(db_dht:delete(DB2, Key), ok),\n                check_entry(DBA1, Key, db_entry:new(Key), {ok, empty_val, -1}, false, \"check_entry_delete_1b\"),\n                check_db(DBA1, {true, []}, 0, [], \"check_db_delete_1b\"),\n                DBA2 = ?db_equals_pattern(db_dht:delete(DBA1, Key), undef),\n                check_entry(DBA2, Key, db_entry:new(Key), {ok, empty_val, -1}, false, \"check_entry_delete_2b\"),\n                check_db(DBA2, {true, []}, 0, [], \"check_db_delete_2b\"),\n                DBA2\n        end,\n\n    db_dht:close(DB3),\n    true.\n\ntester_delete(_Config) ->\n    tester:test(?MODULE, prop_delete, 6, rw_suite_runs(1000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:add_data/2, also validate using different getters\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_add_data(Data::db_dht:db_as_list()) -> true.\nprop_add_data(Data) ->\n    DB = db_dht:new(db_dht),\n\n    UniqueCleanData = unittest_helper:scrub_data(Data),\n\n    DB2 = db_dht:add_data(DB, Data),\n    check_db2(DB2, length(UniqueCleanData), UniqueCleanData, \"check_db_add_data_1\"),\n\n    db_dht:close(DB2),\n    true.\n\ntester_add_data(_Config) ->\n    prop_add_data([create_db_entry(?KEY(\"3\"), empty_val, false, 0, -1),\n                   create_db_entry(?KEY(\"4\"), <<\"foo\">>, false, 5, 230),\n                   create_db_entry(?KEY(\"5\"), empty_val, false, 3, -1)]),\n    tester:test(?MODULE, prop_add_data, 1, rw_suite_runs(1000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:get_entries/3 emulating the former get_range_kv/2 method\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_get_entries3_1(Data::db_dht:db_as_list(), Range::intervals:interval()) -> true.\nprop_get_entries3_1(Data, Range) ->\n    DB = db_dht:new(db_dht),\n    % lists:usort removes all but first occurrence of equal elements\n    % -> reverse list since db_dht:add_data will keep the last element\n    UniqueData = lists:usort(fun(A, B) ->\n                                     db_entry:get_key(A) =< db_entry:get_key(B)\n                             end, lists:reverse(Data)),\n    DB2 = db_dht:add_data(DB, UniqueData),\n\n    FilterFun = fun(A) -> (not db_entry:is_empty(A)) andalso\n                              intervals:in(db_entry:get_key(A), Range)\n                end,\n    ValueFun = fun(DBEntry) -> {db_entry:get_key(DBEntry),\n                                db_entry:get_value(DBEntry)}\n               end,\n\n    ?equals_w_note(lists:sort(db_dht:get_entries(DB2, FilterFun, ValueFun)),\n                   lists:sort([ValueFun(A) || A <- lists:filter(FilterFun, UniqueData)]),\n                   \"check_get_entries3_1_1\"),\n\n    db_dht:close(DB2),\n    true.\n\ntester_get_entries3_1(_Config) ->\n    tester:test(?MODULE, prop_get_entries3_1, 2, rw_suite_runs(1000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:get_entries/3 emulating the former get_range_kvv/2 method\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_get_entries3_2(Data::db_dht:db_as_list(), Range::intervals:interval()) -> true.\nprop_get_entries3_2(Data, Range) ->\n    DB = db_dht:new(db_dht),\n    % lists:usort removes all but first occurrence of equal elements\n    % -> reverse list since db_dht:add_data will keep the last element\n    UniqueData = lists:usort(fun(A, B) ->\n                                     db_entry:get_key(A) =< db_entry:get_key(B)\n                             end, lists:reverse(Data)),\n    DB2 = db_dht:add_data(DB, UniqueData),\n\n    FilterFun = fun(A) -> (not db_entry:is_empty(A)) andalso\n                              db_entry:get_writelock(A) =:= false andalso\n                              intervals:in(db_entry:get_key(A), Range)\n                end,\n    ValueFun = fun(DBEntry) -> {db_entry:get_key(DBEntry),\n                                db_entry:get_value(DBEntry),\n                                db_entry:get_version(DBEntry)}\n               end,\n\n    ?equals_w_note(lists:sort(db_dht:get_entries(DB2, FilterFun, ValueFun)),\n                   lists:sort([ValueFun(A) || A <- lists:filter(FilterFun, UniqueData)]),\n                   \"check_get_entries3_2_1\"),\n\n    db_dht:close(DB2),\n    true.\n\ntester_get_entries3_2(_Config) ->\n    tester:test(?MODULE, prop_get_entries3_2, 2, rw_suite_runs(1000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:get_entries/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_get_entries2(Data::db_dht:db_as_list(), Range::intervals:interval()) -> true.\nprop_get_entries2(Data, Range) ->\n    DB = db_dht:new(db_dht),\n    % lists:usort removes all but first occurrence of equal elements\n    % -> reverse list since db_dht:add_data will keep the last element\n    UniqueData = lists:usort(fun(A, B) ->\n                                     db_entry:get_key(A) =< db_entry:get_key(B)\n                             end, lists:reverse(Data)),\n    DB2 = db_dht:add_data(DB, UniqueData),\n\n    InRangeFun = fun(A) -> (not db_entry:is_empty(A)) andalso\n                               intervals:in(db_entry:get_key(A), Range)\n                 end,\n\n    ?equals_w_note(lists:sort(db_dht:get_entries(DB2, Range)),\n                   lists:sort(lists:filter(InRangeFun, UniqueData)),\n                   \"check_get_entries2_1\"),\n\n    db_dht:close(DB2),\n    true.\n\ntester_get_entries2(_Config) ->\n    tester:test(?MODULE, prop_get_entries2, 2, rw_suite_runs(1000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:get_load/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_get_load2(Data::db_dht:db_as_list(), LoadInterval::intervals:interval()) -> true.\nprop_get_load2(Data, LoadInterval) ->\n    DB = db_dht:new(db_dht),\n    UniqueCleanData = unittest_helper:scrub_data(Data),\n\n    DB2 = db_dht:add_data(DB, Data),\n\n    FilterFun = fun(A) -> intervals:in(db_entry:get_key(A), LoadInterval) end,\n    ValueFun = fun(_DBEntry) -> 1 end,\n\n    ?equals_w_note(db_dht:get_load(DB2, LoadInterval),\n                   length(lists:filter(FilterFun, UniqueCleanData)),\n                   \"check_get_load2_1\"),\n    ?equals_w_note(db_dht:get_load(DB2, LoadInterval),\n                   length(db_dht:get_entries(DB2, FilterFun, ValueFun)),\n                   \"check_get_load2_2\"),\n\n    db_dht:close(DB2),\n    true.\n\ntester_get_load2(_Config) ->\n    tester:test(?MODULE, prop_get_load2, 2, rw_suite_runs(1000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:split_data/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_split_data(Data::db_dht:db_as_list(), Range::intervals:interval()) -> true.\nprop_split_data(Data, Range) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    UniqueCleanData = unittest_helper:scrub_data(Data),\n\n    InHisRangeFun = fun(A) -> (not db_entry:is_empty(A)) andalso\n                                  (not intervals:in(db_entry:get_key(A), Range))\n                    end,\n    InMyRangeFun = fun(A) -> intervals:in(db_entry:get_key(A), Range) end,\n\n    {DB3, HisList} = db_dht:split_data(DB2, Range),\n    ?equals_w_note(lists:sort(HisList),\n                   lists:sort(lists:filter(InHisRangeFun, UniqueCleanData)),\n                   \"check_split_data_1\"),\n    ?equals_w_note(lists:sort(db_dht:get_data(DB3)),\n                   lists:sort(lists:filter(InMyRangeFun, UniqueCleanData)),\n                   \"check_split_data_2\"),\n\n    db_dht:close(DB3),\n    true.\n\ntester_split_data(_Config) ->\n    tester:test(?MODULE, prop_split_data, 2, rw_suite_runs(1000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:update_entries/4\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_update_entries(Data::db_dht:db_as_list(), ItemsToUpdate::pos_integer()) -> true.\nprop_update_entries(Data, ItemsToUpdate) ->\n    UniqueCleanData = unittest_helper:scrub_data(Data),\n    {UniqueUpdateData, UniqueOldData} =\n    case length(UniqueCleanData) < ItemsToUpdate of\n        true ->\n            lists:split(length(UniqueCleanData), UniqueCleanData);\n        _ ->\n            lists:split(ItemsToUpdate, UniqueCleanData)\n    end,\n\n    ExpUpdatedData = UniqueUpdateData ++ UniqueOldData,\n\n    prop_update_entries_helper(UniqueCleanData, UniqueUpdateData, ExpUpdatedData).\n\n-spec prop_update_entries_helper(UniqueData::db_dht:db_as_list(), UniqueUpdateData::db_dht:db_as_list(), ExpUpdatedData::db_dht:db_as_list()) -> true.\nprop_update_entries_helper(UniqueData, UniqueUpdateData, ExpUpdatedData) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, UniqueData),\n\n    UpdatePred = fun(OldEntry, NewEntry) ->\n                         db_entry:get_version(OldEntry) < db_entry:get_version(NewEntry)\n                 end,\n    UpdateVal = fun(_OldEntry, NewEntry) -> NewEntry end,\n\n    DB3 = db_dht:update_entries(DB2, UniqueUpdateData, UpdatePred, UpdateVal),\n\n    ?equals_w_note(lists:sort(db_dht:get_data(DB3)),\n                   lists:sort(ExpUpdatedData),\n                   \"check_update_entries_1\"),\n\n    db_dht:close(DB3),\n    true.\n\ntester_update_entries(_Config) ->\n    tester:test(?MODULE, prop_update_entries, 2, rw_suite_runs(1000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:delete_entries/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_delete_entries1(Data::db_dht:db_as_list(), Range::intervals:interval()) -> true.\nprop_delete_entries1(Data, Range) ->\n    % use a range to delete entries\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n\n    DB3 = db_dht:delete_entries(DB2, Range),\n\n    UniqueCleanData = unittest_helper:scrub_data(Data),\n    UniqueRemainingData = [DBEntry || DBEntry <- UniqueCleanData,\n                                      not intervals:in(db_entry:get_key(DBEntry), Range)],\n    check_db2(DB3, length(UniqueRemainingData), UniqueRemainingData, \"check_db_delete_entries1_1\"),\n\n    db_dht:close(DB3),\n    true.\n\n-spec prop_delete_entries2(Data::db_dht:db_as_list(), Range::intervals:interval()) -> true.\nprop_delete_entries2(Data, Range) ->\n    % use a range to delete entries\n    FilterFun = fun(DBEntry) -> not intervals:in(db_entry:get_key(DBEntry), Range) end,\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n\n    DB3 = db_dht:delete_entries(DB2, FilterFun),\n\n    UniqueCleanData = unittest_helper:scrub_data(Data),\n    UniqueRemainingData = [DBEntry || DBEntry <- UniqueCleanData,\n                                      intervals:in(db_entry:get_key(DBEntry), Range)],\n    check_db2(DB3, length(UniqueRemainingData), UniqueRemainingData, \"check_db_delete_entries2_1\"),\n\n    db_dht:close(DB3),\n    true.\n\ntester_delete_entries1(_Config) ->\n    tester:test(?MODULE, prop_delete_entries1, 2, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_delete_entries2(_Config) ->\n    tester:test(?MODULE, prop_delete_entries2, 2, rw_suite_runs(1000), [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_dht:record_changes/2, stop_record_changes/1 and get_changes/1\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_changed_keys_get_entry(\n        Data::db_dht:db_as_list(), ChangesInterval::intervals:interval(),\n        Key::?RT:key()) -> true.\nprop_changed_keys_get_entry(Data, ChangesInterval, Key) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    _ = db_dht:get_entry(DB3, Key),\n    ?equals_w_note(db_dht:get_changes(DB3), {[], []}, \"changed_keys_get_entry_1\"),\n\n    DB4 = check_stop_record_changes(DB3, ChangesInterval, \"changed_keys_get_entry_2\"),\n\n    db_dht:close(DB4),\n    true.\n\n-spec prop_changed_keys_set_entry(\n        Data::db_dht:db_as_list(), ChangesInterval::intervals:interval(),\n        Entry::db_entry:entry()) -> true.\nprop_changed_keys_set_entry(Data, ChangesInterval, Entry) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    Old = db_dht:get_entry(DB2, db_entry:get_key(Entry)),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    DB4 = db_dht:set_entry(DB3, Entry),\n    check_changes(DB4, ChangesInterval, \"changed_keys_set_entry_1\"),\n    check_entry_in_changes(DB4, ChangesInterval, Entry, Old, \"changed_keys_set_entry_2\"),\n\n    DB5 = check_stop_record_changes(DB4, ChangesInterval, \"changed_keys_set_entry_3\"),\n\n    db_dht:close(DB5),\n    true.\n\n\n-spec prop_changed_keys_update_entry(\n        Data::[db_entry:entry(),...], ChangesInterval::intervals:interval(),\n        UpdateVal::db_dht:value()) -> true.\nprop_changed_keys_update_entry(Data, ChangesInterval, UpdateVal) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    % lists:usort removes all but first occurrence of equal elements\n    % -> reverse list since db_dht:add_data will keep the last element\n    UniqueData = lists:usort(fun(A, B) ->\n                                     db_entry:get_key(A) =< db_entry:get_key(B)\n                             end, lists:reverse(Data)),\n    UpdateElement = util:randomelem(UniqueData),\n    Old = db_dht:get_entry(DB2, db_entry:get_key(UpdateElement)),\n    UpdatedElement = db_entry:set_value(UpdateElement, UpdateVal, db_entry:get_version(UpdateElement) + 1),\n\n    case db_entry:is_null(Old) of\n        true -> % element does not exist, i.e. was a null entry, -> cannot update\n            DB5 = DB2;\n        _ ->\n            DB3 = db_dht:record_changes(DB2, ChangesInterval),\n            DB4 = db_dht:update_entry(DB3, UpdatedElement),\n            check_changes(DB4, ChangesInterval, \"changed_update_entry_1\"),\n            check_entry_in_changes(DB4, ChangesInterval, UpdatedElement, Old, \"changed_update_entry_2\"),\n\n            DB5 = check_stop_record_changes(DB4, ChangesInterval, \"changed_update_entry_3\")\n    end,\n\n    db_dht:close(DB5),\n    true.\n\n-spec prop_changed_keys_delete_entry(\n        Data::db_dht:db_as_list(), ChangesInterval::intervals:interval(),\n        Entry::db_entry:entry()) -> true.\nprop_changed_keys_delete_entry(Data, ChangesInterval, Entry) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    Old = db_dht:get_entry(DB2, db_entry:get_key(Entry)),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    DB4 = db_dht:delete_entry(DB3, Entry),\n    check_changes(DB4, ChangesInterval, \"delete_entry_1\"),\n    check_key_in_deleted_no_locks(DB4, ChangesInterval, db_entry:get_key(Entry), Old, \"delete_entry_2\"),\n\n    DB5 = check_stop_record_changes(DB4, ChangesInterval, \"delete_entry_3\"),\n\n    db_dht:close(DB5),\n    true.\n\n-spec prop_changed_keys_read(\n        Data::db_dht:db_as_list(), ChangesInterval::intervals:interval(),\n        Key::?RT:key()) -> true.\nprop_changed_keys_read(Data, ChangesInterval, Key) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    _ = db_dht:read(DB3, Key),\n    ?equals_w_note(db_dht:get_changes(DB3), {[], []}, \"changed_keys_read_1\"),\n\n    DB4 = check_stop_record_changes(DB3, ChangesInterval, \"changed_keys_read_2\"),\n\n    db_dht:close(DB4),\n    true.\n\n-spec prop_changed_keys_write(\n        Data::db_dht:db_as_list(), ChangesInterval::intervals:interval(),\n        Key::?RT:key(), Value::db_dht:value(), Version::client_version()) -> true.\nprop_changed_keys_write(Data, ChangesInterval, Key, Value, Version) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    Old = db_dht:get_entry(DB2, Key),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    DB4 = db_dht:write(DB3, Key, Value, Version),\n    check_changes(DB4, ChangesInterval, \"changed_keys_write_1\"),\n    ChangedEntry = db_dht:get_entry(DB4, Key),\n    check_entry_in_changes(DB4, ChangesInterval, ChangedEntry, Old, \"changed_keys_write_2\"),\n\n    DB5 = check_stop_record_changes(DB4, ChangesInterval, \"changed_keys_write_3\"),\n\n    db_dht:close(DB5),\n    true.\n\n-spec prop_changed_keys_delete(\n        Data::db_dht:db_as_list(), ChangesInterval::intervals:interval(),\n        Key::?RT:key()) -> true.\nprop_changed_keys_delete(Data, ChangesInterval, Key) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    Old = db_dht:get_entry(DB2, Key),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    {DB4, _Status} = db_dht:delete(DB3, Key),\n    check_changes(DB4, ChangesInterval, \"delete_1\"),\n    check_key_in_deleted_no_locks(DB4, ChangesInterval, Key, Old, \"delete_2\"),\n\n    DB5 = check_stop_record_changes(DB4, ChangesInterval, \"delete_3\"),\n\n    db_dht:close(DB5),\n    true.\n\n-spec prop_changed_keys_get_entries2(\n        Data::db_dht:db_as_list(), ChangesInterval::intervals:interval(),\n        Interval::intervals:interval()) -> true.\nprop_changed_keys_get_entries2(Data, ChangesInterval, Interval) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    _ = db_dht:get_entries(DB3, Interval),\n    ?equals_w_note(db_dht:get_changes(DB3), {[], []}, \"changed_keys_get_entries2_1\"),\n\n    DB4 = check_stop_record_changes(DB3, ChangesInterval, \"changed_keys_get_entries2_2\"),\n\n    db_dht:close(DB4),\n    true.\n\n-spec prop_changed_keys_get_entries4(\n        Data::db_dht:db_as_list(), ChangesInterval::intervals:interval(),\n        Interval::intervals:interval()) -> true.\nprop_changed_keys_get_entries4(Data, ChangesInterval, Interval) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    FilterFun = fun(E) -> (not db_entry:is_empty(E)) andalso\n                              (not intervals:in(db_entry:get_key(E), Interval))\n                end,\n    ValueFun = fun(E) -> db_entry:get_key(E) end,\n\n    _ = db_dht:get_entries(DB3, FilterFun, ValueFun),\n    ?equals_w_note(db_dht:get_changes(DB3), {[], []}, \"changed_keys_get_entries4_1\"),\n\n    DB4 = check_stop_record_changes(DB3, ChangesInterval, \"changed_keys_get_entries4_2\"),\n\n    db_dht:close(DB4),\n    true.\n\n-spec prop_get_chunk4(Keys::[?RT:key()], StartId::?RT:key(), Interval::intervals:interval(), ChunkSize::pos_integer() | all) -> true.\nprop_get_chunk4(Keys2, StartId, Interval, ChunkSize) ->\n    Keys = lists:usort(Keys2),\n    %% ct:pal(\"prop_get_chunk4(~w, ~w, ~w, ~w)\", [Keys2, StartId, Interval, ChunkSize]),\n    DB = db_dht:new(db_dht),\n    DB2 = lists:foldl(fun(Key, DBA) -> db_dht:write(DBA, Key, ?VALUE(\"Value\"), 1) end, DB, Keys),\n    {Next, Chunk} = db_dht:get_chunk(DB2, StartId, Interval, ChunkSize),\n    % note: prevent erlang default printing from converting small int lists to strings:\n    ChunkKeys = [{db_entry:get_key(C)} || C <- Chunk],\n    %% ct:pal(\"-> ~.2p\", [{Next, ChunkKeys}]),\n    db_dht:close(DB2),\n    ?equals(lists:usort(ChunkKeys), lists:sort(ChunkKeys)), % check for duplicates\n    KeysInRange = [{Key} || Key <- Keys, intervals:in(Key, Interval)],\n    KeysInRangeCount = length(KeysInRange),\n    ExpectedChunkSize =\n        case ChunkSize of\n            all -> KeysInRangeCount;\n            _   -> erlang:min(KeysInRangeCount, ChunkSize)\n        end,\n    if ExpectedChunkSize =/= length(ChunkKeys) ->\n           ?ct_fail(\"chunk has wrong size ~w ~w ~w, expected size: ~w\",\n                    [ChunkKeys, Keys, Interval, ExpectedChunkSize]);\n       true -> ok\n    end,\n    % elements in Chunk must be in Interval\n    ?equals([X || X = {Key} <- ChunkKeys, not intervals:in(Key, Interval)],\n            []),\n    % Next is subset of Interval, no chunk entry is in Next:\n    ?equals_w_note(intervals:is_subset(Next, Interval), true,\n                   io_lib:format(\"Next ~.0p is not subset of ~.0p\",\n                                 [Next, Interval])),\n    ?compare(fun(N, I) ->\n                     ?implies(intervals:is_continuous(I) andalso (not intervals:in(StartId, I)),\n                              intervals:is_continuous(N) orelse intervals:is_empty(N))\n             end, Next, Interval),\n    ?equals_w_note([X || X = {Key} <- ChunkKeys, intervals:in(Key, Next)],\n                   [], io_lib:format(\"Next: ~.0p\", [Next])),\n    % if ChunkSize is all, Next must be empty!\n    ?IIF(ChunkSize =:= all, ?equals(Next, intervals:empty()), true),\n    % keys in Chunk plus keys in Next must be all keys in Interval\n    KeysInChunkPlusNext = lists:usort(ChunkKeys\n                                          ++ [{Key} || Key <- Keys, intervals:in(Key, Next)]),\n    ?equals(KeysInChunkPlusNext, KeysInRange),\n    % if Chunk not empty, first key must be first after StartId:\n    if KeysInRangeCount > 0 ->\n           FirstAfterStartId =\n               case lists:partition(fun({Key}) -> Key >= StartId end, KeysInRange) of\n                   {[], [H|_]} -> H;\n                   {[H|_], _}  -> H\n               end,\n           ?compare(fun(ChunkKeysX, FirstAfterStartIdX) ->\n                            hd(ChunkKeysX) =:= FirstAfterStartIdX\n                    end, ChunkKeys, FirstAfterStartId);\n       true -> true\n    end.\n\n-spec prop_get_split_key5(Keys::[?RT:key(),...], Begin::?RT:key(), End::?RT:key(), TargetLoad::pos_integer(), forward | backward) -> true.\nprop_get_split_key5(Keys2, Begin, End, TargetLoad, ForwardBackward) ->\n    Keys = lists:usort(Keys2),\n    DB = db_dht:new(db_dht),\n    DB2 = lists:foldl(fun(Key, DBA) -> db_dht:write(DBA, Key, ?VALUE(\"Value\"), 1) end, DB, Keys),\n    {SplitKey, TakenLoad} = db_dht:get_split_key(DB2, Begin, End, TargetLoad, ForwardBackward),\n    SplitInterval = case ForwardBackward of\n                        forward  ->\n                            intervals:new('(', Begin, SplitKey, ']');\n                        backward ->\n                            intervals:new('(', SplitKey, Begin, ']')\n                    end,\n    {_Next, Chunk} = db_dht:get_chunk(DB2, Begin, SplitInterval, all),\n    db_dht:close(DB2),\n\n    ?compare(fun erlang:'=<'/2, TakenLoad, TargetLoad),\n    ?compare(fun erlang:'=<'/2, length(Chunk), TargetLoad),\n    case TakenLoad < TargetLoad of\n        true ->\n            ?equals(SplitKey, End);\n        _ ->\n            ok\n    end,\n    ?equals(length(Chunk), TakenLoad),\n    true.\n\n-spec prop_changed_keys_update_entries(\n        Data::db_dht:db_as_list(), ChangesInterval::intervals:interval(),\n        Entry1::db_entry:entry(), Entry2::db_entry:entry()) -> true.\nprop_changed_keys_update_entries(Data, ChangesInterval, Entry1, Entry2) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    Old1 = db_dht:get_entry(DB2, db_entry:get_key(Entry1)),\n    Old2 = db_dht:get_entry(DB2, db_entry:get_key(Entry2)),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    UpdatePred = fun(OldEntry, NewEntry) ->\n                         db_entry:get_version(OldEntry) < db_entry:get_version(NewEntry)\n                 end,\n    UpdateVal = fun(_OldEntry, NewEntry) -> NewEntry end,\n\n    DB4 = db_dht:update_entries(DB3, [Entry1, Entry2], UpdatePred, UpdateVal),\n    NewEntry1 = db_dht:get_entry(DB4, db_entry:get_key(Entry1)),\n    NewEntry2 = db_dht:get_entry(DB4, db_entry:get_key(Entry2)),\n    check_changes(DB4, ChangesInterval, \"update_entries_1\"),\n    ?implies(db_entry:get_version(Old1) < db_entry:get_version(Entry1),\n             check_entry_in_changes(DB4, ChangesInterval, NewEntry1, Old1, \"update_entries_2\")),\n    ?implies(db_entry:get_version(Old2) < db_entry:get_version(Entry2),\n             check_entry_in_changes(DB4, ChangesInterval, NewEntry2, Old2, \"update_entries_3\")),\n\n    DB5 = check_stop_record_changes(DB4, ChangesInterval, \"update_entries_4\"),\n\n    db_dht:close(DB5),\n    true.\n\n-spec prop_changed_keys_delete_entries1(\n        Data::db_dht:db_as_list(), Range::intervals:interval(),\n        ChangesInterval::intervals:interval()) -> true.\nprop_changed_keys_delete_entries1(Data, ChangesInterval, Range) ->\n    % use a range to delete entries\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    DB4 = db_dht:delete_entries(DB3, Range),\n\n    UniqueCleanData = unittest_helper:scrub_data(Data),\n    DeletedKeys = [{db_entry:get_key(DBEntry), true}\n                  || DBEntry <- UniqueCleanData,\n                     intervals:in(db_entry:get_key(DBEntry), Range)],\n    check_changes(DB4, ChangesInterval, \"delete_entries1_1\"),\n    check_keys_in_deleted(DB4, ChangesInterval, DeletedKeys, \"delete_entries1_2\"),\n\n    db_dht:close(DB3),\n    true.\n\n-spec prop_changed_keys_delete_entries2(\n        Data::db_dht:db_as_list(), Range::intervals:interval(),\n        ChangesInterval::intervals:interval()) -> true.\nprop_changed_keys_delete_entries2(Data, ChangesInterval, Range) ->\n    % use a range to delete entries\n    FilterFun = fun(DBEntry) -> not intervals:in(db_entry:get_key(DBEntry), Range) end,\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    DB4 = db_dht:delete_entries(DB3, FilterFun),\n\n    UniqueCleanData = unittest_helper:scrub_data(Data),\n    DeletedKeys = [{db_entry:get_key(DBEntry), true}\n                  || DBEntry <- UniqueCleanData,\n                     not intervals:in(db_entry:get_key(DBEntry), Range)],\n    check_changes(DB4, ChangesInterval, \"delete_entries2_1\"),\n    check_keys_in_deleted(DB4, ChangesInterval, DeletedKeys, \"delete_entries2_2\"),\n\n    db_dht:close(DB3),\n    true.\n\n-spec prop_changed_keys_get_load(\n        Data::db_dht:db_as_list(), ChangesInterval::intervals:interval()) -> true.\nprop_changed_keys_get_load(Data, ChangesInterval) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    db_dht:get_load(DB3),\n    ?equals_w_note(db_dht:get_changes(DB3), {[], []}, \"changed_keys_get_load_1\"),\n\n    DB4 = check_stop_record_changes(DB3, ChangesInterval, \"changed_keys_get_load_2\"),\n\n    db_dht:close(DB4),\n    true.\n\n-spec prop_changed_keys_get_load2(\n        Data::db_dht:db_as_list(), LoadInterval::intervals:interval(),\n        ChangesInterval::intervals:interval()) -> true.\nprop_changed_keys_get_load2(Data, LoadInterval, ChangesInterval) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    db_dht:get_load(DB3, LoadInterval),\n    ?equals_w_note(db_dht:get_changes(DB3), {[], []}, \"changed_keys_get_load2_1\"),\n\n    DB4 = check_stop_record_changes(DB3, ChangesInterval, \"changed_keys_get_load2_2\"),\n\n    db_dht:close(DB4),\n    true.\n\n-spec prop_changed_keys_split_data1(\n        Data::db_dht:db_as_list(),\n        ChangesInterval::intervals:interval(),\n        MyNewInterval1::intervals:interval()) -> true.\nprop_changed_keys_split_data1(Data, ChangesInterval, MyNewInterval) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    {DB4, _HisList} = db_dht:split_data(DB3, MyNewInterval),\n    ?equals_w_note(db_dht:get_changes(DB4), {[], []}, \"split_data1_1\"),\n\n    DB5 = check_stop_record_changes(DB4, ChangesInterval, \"split_data1_2\"),\n\n    db_dht:close(DB5),\n    true.\n\n-spec prop_changed_keys_split_data2(\n        Data::db_dht:db_as_list(),\n        ChangesInterval::intervals:interval(),\n        MyNewInterval1::intervals:interval()) -> true.\nprop_changed_keys_split_data2(Data, ChangesInterval, MyNewInterval) ->\n    %% db_entries that are null won't be inserted into db anymore\n    CleanData = unittest_helper:scrub_data(Data),\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:record_changes(DB, ChangesInterval),\n    DB3 = db_dht:add_data(DB2, CleanData),\n\n    {DB4, _HisList} = db_dht:split_data(DB3, MyNewInterval),\n\n    check_changes(DB4, intervals:intersection(ChangesInterval, MyNewInterval), \"split_data2_1\"),\n\n    DB5 = check_stop_record_changes(DB4, ChangesInterval, \"split_data2_2\"),\n\n    db_dht:close(DB5),\n    true.\n\n-spec prop_changed_keys_get_data(\n        Data::db_dht:db_as_list(), ChangesInterval::intervals:interval()) -> true.\nprop_changed_keys_get_data(Data, ChangesInterval) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    UniqueCleanData = unittest_helper:scrub_data(Data),\n\n    ?equals(lists:sort(db_dht:get_data(DB3)), lists:sort(UniqueCleanData)),\n    ?equals_w_note(db_dht:get_changes(DB3), {[], []}, \"changed_keys_get_data_1\"),\n\n    DB4 = check_stop_record_changes(DB3, ChangesInterval, \"changed_keys_get_data_2\"),\n\n    db_dht:close(DB4),\n    true.\n\n-spec prop_changed_keys_add_data(\n        Data::db_dht:db_as_list(),\n        ChangesInterval::intervals:interval()) -> true.\nprop_changed_keys_add_data(Data, ChangesInterval) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:record_changes(DB, ChangesInterval),\n\n    DB3 = db_dht:add_data(DB2, Data),\n    check_changes(DB3, ChangesInterval, \"add_data_1\"),\n\n    % lists:usort removes all but first occurrence of equal elements\n    % -> reverse list since db_dht:add_data will keep the last element\n    UniqueData = lists:usort(fun(A, B) ->\n                                     db_entry:get_key(A) =< db_entry:get_key(B)\n                             end, lists:reverse(Data)),\n    _ = [check_entry_in_changes(DB3, ChangesInterval, E, db_entry:new(db_entry:get_key(E)), \"add_data_2\")\n           || E <- UniqueData],\n\n    DB4 = check_stop_record_changes(DB3, ChangesInterval, \"add_data_3\"),\n\n    db_dht:close(DB4),\n    true.\n\n-spec prop_changed_keys_check_db(\n        Data::db_dht:db_as_list(), ChangesInterval::intervals:interval()) -> true.\nprop_changed_keys_check_db(Data, ChangesInterval) ->\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n    DB3 = db_dht:record_changes(DB2, ChangesInterval),\n\n    _ = db_dht:check_db(DB3),\n    ?equals_w_note(db_dht:get_changes(DB3), {[], []}, \"changed_keys_check_db_1\"),\n\n    DB4 = check_stop_record_changes(DB3, ChangesInterval, \"changed_keys_check_db_2\"),\n\n    db_dht:close(DB4),\n    true.\n\n-spec prop_changed_keys_mult_interval(\n        Data::db_dht:db_as_list(), Entry1::db_entry:entry(),\n        Entry2::db_entry:entry(), Entry3::db_entry:entry(),\n        Entry4::db_entry:entry()) -> true.\nprop_changed_keys_mult_interval(Data, Entry1, Entry2, Entry3, Entry4) ->\n    CI1 = intervals:union(intervals:new(db_entry:get_key(Entry1)),\n                          intervals:new(db_entry:get_key(Entry2))),\n    CI2 = intervals:union(intervals:new(db_entry:get_key(Entry3)),\n                          intervals:new(db_entry:get_key(Entry4))),\n    CI1_2 = intervals:union(CI1, CI2),\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n\n    DB3 = db_dht:record_changes(DB2, CI1),\n    Old1 = db_dht:get_entry(DB3, db_entry:get_key(Entry1)),\n    DB4 = db_dht:set_entry(DB3, Entry1),\n    check_changes(DB4, CI1, \"changed_keys_mult_interval_1\"),\n    check_entry_in_changes(DB4, CI1, Entry1, Old1, \"changed_keys_mult_interval_2\"),\n\n    DB5 = db_dht:record_changes(DB4, CI2),\n    Old2 = db_dht:get_entry(DB5, db_entry:get_key(Entry2)),\n    DB6 = db_dht:set_entry(DB5, Entry2),\n    check_changes(DB6, CI1_2, \"changed_keys_mult_interval_3\"),\n    check_entry_in_changes(DB6, CI1_2, Entry2, Old2, \"changed_keys_mult_interval_4\"),\n\n    DB7 = db_dht:record_changes(DB6, CI2),\n    Old3 = db_dht:get_entry(DB7, db_entry:get_key(Entry3)),\n    DB8 = db_dht:set_entry(DB7, Entry3),\n    check_changes(DB8, CI1_2, \"changed_keys_mult_interval_5\"),\n    check_entry_in_changes(DB8, CI1_2, Entry3, Old3, \"changed_keys_mult_interval_6\"),\n\n    DB9 = db_dht:record_changes(DB8, CI2),\n    Old4 = db_dht:get_entry(DB9, db_entry:get_key(Entry4)),\n    DB10 = db_dht:set_entry(DB9, Entry4),\n    check_changes(DB10, CI1_2, \"changed_keys_mult_interval_7\"),\n    check_entry_in_changes(DB10, CI1_2, Entry4, Old4, \"changed_keys_mult_interval_8\"),\n\n    DB11 = check_stop_record_changes(DB10, CI1_2, \"changed_keys_mult_interval_9\"),\n\n    db_dht:close(DB11),\n    true.\n\n-spec prop_stop_record_changes(\n        Data::db_dht:db_as_list(), Entry1::db_entry:entry(),\n        Entry2::db_entry:entry(), Entry3::db_entry:entry(),\n        Entry4::db_entry:entry()) -> true.\nprop_stop_record_changes(Data, Entry1, Entry2, Entry3, Entry4) ->\n    CI1 = intervals:union(intervals:new(db_entry:get_key(Entry1)),\n                          intervals:new(db_entry:get_key(Entry2))),\n    CI2 = intervals:union(intervals:new(db_entry:get_key(Entry3)),\n                          intervals:new(db_entry:get_key(Entry4))),\n    CI1_2 = intervals:union(CI1, CI2),\n    CI1_wo2 = intervals:minus(CI1, CI2),\n    DB = db_dht:new(db_dht),\n    DB2 = db_dht:add_data(DB, Data),\n\n    DB3 = db_dht:record_changes(DB2, CI1_2),\n    Old1 = db_dht:get_entry(DB3, db_entry:get_key(Entry1)),\n    DB4 = db_dht:set_entry(DB3, Entry1),\n    check_changes(DB4, CI1_2, \"stop_record_changes_1\"),\n    check_entry_in_changes(DB4, CI1_2, Entry1, Old1, \"stop_record_changes_2\"),\n\n    Old3 = db_dht:get_entry(DB4, db_entry:get_key(Entry3)),\n    DB5 = db_dht:set_entry(DB4, Entry3),\n    check_changes(DB5, CI1_2, \"stop_record_changes_3\"),\n    check_entry_in_changes(DB5, CI1_2, Entry3, Old3, \"stop_record_changes_4\"),\n\n    DB6 = db_dht:stop_record_changes(DB5, CI2),\n    check_changes(DB6, CI1_wo2, \"stop_record_changes_5\"),\n    check_entry_in_changes(DB6, CI1_wo2, Entry1, Old1, \"stop_record_changes_6\"),\n\n    Old2 = db_dht:get_entry(DB6, db_entry:get_key(Entry2)),\n    DB7 = db_dht:set_entry(DB6, Entry2),\n    check_changes(DB7, CI1_wo2, \"stop_record_changes_7\"),\n    check_entry_in_changes(DB7, CI1_wo2, Entry2, Old2, \"stop_record_changes_8\"),\n\n    Old4 = db_dht:get_entry(DB7, db_entry:get_key(Entry4)),\n    DB8 = db_dht:set_entry(DB7, Entry4),\n    check_changes(DB8, CI1_wo2, \"stop_record_changes_9\"),\n    check_entry_in_changes(DB8, CI1_wo2, Entry4, Old4, \"stop_record_changes_10\"),\n\n    DB9 = db_dht:stop_record_changes(DB8),\n    ?equals_w_note(db_dht:get_changes(DB9), {[], []}, \"stop_record_changes_11\"),\n\n    db_dht:close(DB9),\n    true.\n\ntester_changed_keys_get_entry(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_get_entry, 3, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_set_entry(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_set_entry, 3, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_update_entry(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_update_entry, 3, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_delete_entry(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_delete_entry, 3, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_read(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_read, 3, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_write(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_write, 5, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_delete(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_delete, 3, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_get_entries2(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_get_entries2, 3, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_get_entries4(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_get_entries4, 3, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_get_chunk_precond(_Config) ->\n    Table = ets:new(ets_test_SUITE, [ordered_set | ?DB_ETS_ADDITIONAL_OPS]),\n    ets:insert(Table, {5}),\n    ets:insert(Table, {6}),\n    ets:insert(Table, {7}),\n    ?equals(ets:next(Table, 7), '$end_of_table'),\n    ?equals(ets:next(Table, 6), 7),\n    ?equals(ets:next(Table, 5), 6),\n    ?equals(ets:next(Table, 4), 5),\n    ?equals(ets:next(Table, 3), 5),\n    ?equals(ets:next(Table, 2), 5),\n    ?equals(ets:first(Table), 5).\n\ntester_get_chunk4(_Config) ->\n    case rt_SUITE:default_rt_has_chord_keys() of\n        true ->\n            prop_get_chunk4([0, 4, 31], 0, intervals:new('[', 0, 4, ']'), 2),\n            prop_get_chunk4([1, 5, 127, 13], 3, intervals:new('[', 3, 2, ']'), 4),\n            prop_get_chunk4([30, 20, 8, 4], 9, intervals:union(intervals:new('[',0,10,']'), intervals:new('[',28,32,')')), all),\n            prop_get_chunk4([321412035892863292970556376746395450950,178033137068077382596514331220271255735,36274679037320551674149151592760931654,24467032062604602002936599440583551943],\n                            39662566533623950601697671725795532001,\n                            intervals:union([intervals:new('[', 0, 117488216920678280505356111701746995698, ']'),\n                                             intervals:new(161901968021578670353994653229245016552),\n                                             intervals:new('[', 225156471921460939006161924022031177737, 340282366920938463463374607431768211456, ')')]),\n                            all),\n            prop_get_chunk4([12, 13, 14, 15, 16],0,intervals:new('[', 10, 0, ']'),2),\n            prop_get_chunk4([12, 8, 6, 4], 4, intervals:new('[', 10, 0, ']'), 1),\n            prop_get_chunk4([3, 4], 10, intervals:new('[', 0, 9, ']'), 1),\n            prop_get_chunk4([2, 6, 7], 5, intervals:new(4), 3),\n            prop_get_chunk4([2, 6, 7], 5, intervals:new('[',5,9,']'), 3);\n        _ -> ok\n    end,\n    % TODO: fix this test\n    prop_get_chunk4(\n      [rt_SUITE:number_to_key(510),\n       rt_SUITE:number_to_key(545)],\n      rt_SUITE:number_to_key(530),\n      intervals:new('(', rt_SUITE:number_to_key(532), rt_SUITE:number_to_key(530), ')'),\n      all),\n\n    tester:test(?MODULE, prop_get_chunk4, 4, rw_suite_runs(10000), [{threads, 2}]).\n\ntester_get_split_key5(_Config) ->\n    case rt_SUITE:default_rt_has_chord_keys() of\n        true ->\n            prop_get_split_key5([2], 6, 4, 12, backward),\n            prop_get_split_key5([12, 10, 4], 6, 8, 1, backward),\n            prop_get_split_key5([10, 9], 10, 6, 2, backward),\n            prop_get_split_key5([10, 9, 8], 10, 6, 2, backward),\n            prop_get_split_key5([10, 6, 5], 6, 9, 1, forward),\n            prop_get_split_key5([10, 6, 5], 9, 6, 1, backward),\n            prop_get_split_key5([10, 9, 8, 7], 10, 6, 2, backward),\n            prop_get_split_key5([10, 9, 8, 7, 4], 10, 6, 2, backward),\n            prop_get_split_key5([11, 10, 9, 8, 7, 4], 10, 6, 2, backward);\n        _ -> ok\n    end,\n    % TODO: fix this test\n    prop_get_split_key5([rt_SUITE:number_to_key(510),\n                         rt_SUITE:number_to_key(520),\n                         rt_SUITE:number_to_key(545)],\n                        rt_SUITE:number_to_key(543),\n                        rt_SUITE:number_to_key(520),\n                        1, backward),\n\n    tester:test(?MODULE, prop_get_split_key5, 5, rw_suite_runs(10000), [{threads, 2}]).\n\ntester_changed_keys_update_entries(_Config) ->\n    prop_changed_keys_update_entries(\n      [{?KEY(\"200\"),empty_val,false,0,-1}], intervals:all(),\n      {?KEY(\"100\"),empty_val,false,1,-1}, {?KEY(\"200\"),empty_val,false,296,-1}),\n    tester:test(?MODULE, prop_changed_keys_update_entries, 4, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_delete_entries1(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_delete_entries1, 3, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_delete_entries2(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_delete_entries2, 3, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_get_load(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_get_load, 2, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_get_load2(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_get_load2, 3, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_split_data1(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_split_data1, 3, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_split_data2(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_split_data2, 3, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_get_data(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_get_data, 2, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_add_data(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_add_data, 2, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_check_db(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_check_db, 2, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_changed_keys_mult_interval(_Config) ->\n    tester:test(?MODULE, prop_changed_keys_mult_interval, 5, rw_suite_runs(1000), [{threads, 2}]).\n\ntester_stop_record_changes(_Config) ->\n    tester:test(?MODULE, prop_stop_record_changes, 5, rw_suite_runs(1000), [{threads, 2}]).\n\n\n% helper functions:\n\n-spec check_entry(DB::db_dht:db(), Key::?RT:key(), ExpDBEntry::db_entry:entry(),\n                  ExpRead::{ok, Value::db_dht:value(), Version::client_version()} | {ok, empty_val, -1},\n                  ExpExists::boolean(), Note::string()) -> true.\ncheck_entry(DB, Key, ExpDBEntry, ExpRead, _ExpExists, Note) ->\n    ?equals_w_note(db_dht:get_entry(DB, Key), ExpDBEntry, Note),\n    ?equals_w_note(db_dht:read(DB, Key), ExpRead, Note).\n\n% note: use manageable values for ReadLock!\n-spec create_db_entry(Key::?RT:key(), Value::db_dht:value(), WriteLock::false | client_version(),\n                      ReadLock::0..1000, Version::client_version()) -> db_entry:entry();\n                     (Key::?RT:key(), Value::empty_val, WriteLock::false,\n                      ReadLock::0..1000, Version::-1) -> db_entry:entry().\ncreate_db_entry(Key, Value, WriteLock, ReadLock, Version) ->\n    E1 = if Value =:= empty_val andalso Version =:= -1 -> db_entry:new(Key);\n            true -> db_entry:new(Key, Value, Version)\n         end,\n    E2 = case WriteLock of\n             true -> db_entry:set_writelock(E1, db_entry:get_version(E1));\n             _    -> E1\n         end,\n    _E3 = inc_readlock(E2, ReadLock).\n\n-spec inc_readlock(DBEntry::db_entry:entry(), Count::non_neg_integer()) -> db_entry:entry().\ninc_readlock(DBEntry, 0) -> DBEntry;\ninc_readlock(DBEntry, Count) -> inc_readlock(db_entry:inc_readlock(DBEntry), Count - 1).\n\n-spec check_db(DB::db_dht:db(),\n               ExpCheckDB::{true, []} | {false, InvalidEntries::db_dht:db_as_list()},\n               ExpLoad::integer(),\n               ExpData::db_dht:db_as_list(), Note::string()) -> true.\ncheck_db(DB, ExpCheckDB, ExpLoad, ExpData, Note) ->\n    check_db(DB, ExpCheckDB, ExpLoad, ExpData, {[], []}, Note).\n\n-spec check_db(DB::db_dht:db(),\n               ExpCheckDB::{true, []} | {false, InvalidEntries::db_dht:db_as_list()},\n               ExpLoad::integer(),\n               ExpData::db_dht:db_as_list(),\n               ExpCKData::{UpdatedEntries::db_dht:db_as_list(), DeletedKeys::[?RT:key()]},\n               Note::string()) -> true.\ncheck_db(DB, ExpCheckDB, ExpLoad, ExpData, ExpCKData, Note) ->\n    ?equals_w_note(db_dht:check_db(DB), ExpCheckDB, Note),\n    ?equals_w_note(db_dht:get_load(DB), ExpLoad, Note),\n    ?equals_w_note(lists:sort(db_dht:get_data(DB)), lists:sort(ExpData), Note),\n    ?equals_w_note(db_dht:get_changes(DB), ExpCKData, Note).\n\n%% @doc Like check_db/5 but do not check DB using db_dht:check_db.\n-spec check_db2(DB::db_dht:db(), ExpLoad::integer(),\n               ExpData::db_dht:db_as_list(), Note::string()) -> true.\ncheck_db2(DB, ExpLoad, ExpData, Note) ->\n    check_db2(DB, ExpLoad, ExpData, {[], []}, Note).\n\n%% @doc Like check_db/5 but do not check DB using db_dht:check_db.\n-spec check_db2(DB::db_dht:db(), ExpLoad::integer(),\n                ExpData::db_dht:db_as_list(),\n                ExpCKData::{UpdatedEntries::db_dht:db_as_list(), DeletedKeys::[?RT:key()]},\n                Note::string()) -> true.\ncheck_db2(DB, ExpLoad, ExpData, ExpCKData, Note) ->\n    ?equals_w_note(db_dht:get_load(DB), ExpLoad, Note),\n    ?equals_w_note(lists:sort(db_dht:get_data(DB)), lists:sort(ExpData), Note),\n    ?equals_w_note(db_dht:get_changes(DB), ExpCKData, Note).\n\nget_random_interval_from_changes(DB) ->\n    {ChangedEntries, DeletedKeys} = db_dht:get_changes(DB),\n    case ChangedEntries =/= [] orelse DeletedKeys =/= [] of\n        true ->\n            intervals:new(util:randomelem(\n                            lists:append(\n                              [db_entry:get_key(E) || E <- ChangedEntries],\n                              DeletedKeys)));\n        _    -> intervals:empty()\n    end.\n\n-spec check_stop_record_changes(DB::db_dht:db(), ChangesInterval::intervals:interval(), Note::string()) -> db_dht:db().\ncheck_stop_record_changes(DB, ChangesInterval, Note) ->\n    I1 = get_random_interval_from_changes(DB),\n    DB2 = db_dht:stop_record_changes(DB, I1),\n    check_changes(DB2, intervals:minus(ChangesInterval, I1), Note ++ \"a\"),\n\n    DB3 = db_dht:stop_record_changes(DB2),\n    ?equals_w_note(db_dht:get_changes(DB3), {[], []}, Note ++ \"b\"),\n\n    DB3.\n\n\n%% @doc Checks that all entries returned by db_dht:get_changes/1 are in the\n%%      given interval.\n-spec check_changes(DB::db_dht:db(), ChangesInterval::intervals:interval(), Note::string()) -> true.\ncheck_changes(DB, ChangesInterval, Note) ->\n    {ChangedEntries1, DeletedKeys1} = db_dht:get_changes(DB),\n    case lists:all(fun(E) -> intervals:in(db_entry:get_key(E), ChangesInterval) end,\n               ChangedEntries1) of\n        false ->\n            ?ct_fail(\"~s evaluated to \\\"~w\\\" and contains elements not in ~w~n(~s)~n\",\n                     [\"element(1, db_dht:get_changes(DB))\", ChangedEntries1,\n                      ChangesInterval, lists:flatten(Note)]);\n        _ -> ok\n    end,\n    case lists:all(fun(E) -> intervals:in(E, ChangesInterval) end, DeletedKeys1) of\n        false ->\n            ?ct_fail(\"~s evaluated to \\\"~w\\\" and contains elements not in ~w~n(~s)~n\",\n                     [\"element(2, db_dht:get_changes(DB))\", DeletedKeys1,\n                      ChangesInterval, lists:flatten(Note)]);\n        _ -> ok\n    end,\n    check_changes2(DB, ChangesInterval, ChangesInterval, Note),\n    % select some random key from the changed entries and try get_changes/2\n    % with an interval that does not contain this key\n    case ChangedEntries1 =/= [] orelse DeletedKeys1 =/= [] of\n        true ->\n            SomeKey = util:randomelem(\n                        lists:append(\n                          [db_entry:get_key(E) || E <- ChangedEntries1],\n                          DeletedKeys1)),\n            check_changes2(DB, ChangesInterval, intervals:minus(ChangesInterval, intervals:new(SomeKey)), Note);\n        _    -> true\n    end.\n\n%% @doc Checks that all entries returned by db_dht:get_changes/2 are in the\n%%      given interval GetChangesInterval and also ChangesInterval.\n-spec check_changes2(DB::db_dht:db(), ChangesInterval::intervals:interval(), GetChangesInterval::intervals:interval(), Note::string()) -> true.\ncheck_changes2(DB, ChangesInterval, GetChangesInterval, Note) ->\n    {ChangedEntries2, DeletedKeys2} = db_dht:get_changes(DB, GetChangesInterval),\n    FinalInterval = intervals:intersection(ChangesInterval, GetChangesInterval),\n    case lists:all(fun(E) -> intervals:in(db_entry:get_key(E), FinalInterval) end,\n               ChangedEntries2) of\n        false ->\n            ?ct_fail(\"~s evaluated to \\\"~w\\\" and contains elements not in ~w~n(~s)~n\",\n                     [\"element(1, db_dht:get_changes(DB, FinalInterval))\",\n                      ChangedEntries2, FinalInterval, lists:flatten(Note)]);\n        _ -> ok\n    end,\n    case lists:all(fun(E) -> intervals:in(E, FinalInterval) end, DeletedKeys2) of\n        false ->\n            ?ct_fail(\"~s evaluated to \\\"~w\\\" and contains elements not in ~w~n(~s)~n\",\n                     [\"element(2, db_dht:get_changes(DB, FinalInterval))\",\n                      DeletedKeys2, FinalInterval, lists:flatten(Note)]);\n        _ -> ok\n    end,\n    true.\n\n%% @doc Checks that a key is present exactly once in the list of deleted\n%%      keys returned by db_dht:get_changes/1 if no lock is set on a\n%%      previously existing entry.\n-spec check_key_in_deleted_no_locks(\n        DB::db_dht:db(), ChangesInterval::intervals:interval(), Key::?RT:key(),\n        OldEntry::db_entry:entry(), Note::string()) -> true.\ncheck_key_in_deleted_no_locks(DB, ChangesInterval, Key, Old, Note) ->\n    case intervals:in(Key, ChangesInterval) andalso not db_entry:is_null(Old) andalso\n             not db_entry:is_locked(Old) of\n        true ->\n            {_ChangedEntries, DeletedKeys} = db_dht:get_changes(DB),\n            check_key_in_deleted_internal(DeletedKeys, ChangesInterval, Key,\n                                          not db_entry:is_null(Old), Note);\n        _    -> true\n    end.\n\n%% @doc Checks that all given keys are present exactly once in the list of\n%%      deleted keys returned by db_dht:get_changes/1.\n-spec check_keys_in_deleted(\n        DB::db_dht:db(), ChangesInterval::intervals:interval(),\n        Keys::[{?RT:key(), OldExists::boolean()}], Note::string()) -> true.\ncheck_keys_in_deleted(DB, ChangesInterval, Keys, Note) ->\n    {_ChangedEntries, DeletedKeys} = db_dht:get_changes(DB),\n    [check_key_in_deleted_internal(DeletedKeys, ChangesInterval, Key, OldExists, Note)\n    || {Key, OldExists} <- Keys],\n    true.\n\n-spec check_key_in_deleted_internal(\n        DeletedKeys::[?RT:key()], ChangesInterval::intervals:interval(),\n        Key::?RT:key(), OldExists::boolean(), Note::string()) -> true.\ncheck_key_in_deleted_internal(DeletedKeys, ChangesInterval, Key, OldExists, Note) ->\n    case intervals:in(Key, ChangesInterval) andalso OldExists of\n        true ->\n            case length([K || K <- DeletedKeys, K =:= Key]) of\n                1 -> ok;\n                _ -> ?ct_fail(\"element(2, db_dht:get_changes(DB)) evaluated \"\n                              \"to \\\"~w\\\" and did not contain 1x deleted key ~w~n(~s)~n\",\n                              [DeletedKeys, Key, lists:flatten(Note)])\n            end;\n        _    -> true\n    end.\n\n%% @doc Checks that an entry is present exactly once in the list of changed\n%%      entries returned by db_dht:get_changes/1.\n-spec check_entry_in_changed_entries(\n        DB::db_dht:db(), ChangesInterval::intervals:interval(), Entry::db_entry:entry(),\n        OldEntry::db_entry:entry(), Note::string()) -> ok.\ncheck_entry_in_changed_entries(DB, ChangesInterval, NewEntry, OldEntry, Note) ->\n    {ChangedEntries, _DeletedKeys} = db_dht:get_changes(DB),\n    check_entry_in_changed_entries_internal(ChangedEntries, ChangesInterval, NewEntry, OldEntry, Note).\n\n-spec check_entry_in_changed_entries_internal(\n        ChangedEntries::db_dht:db_as_list(), ChangesInterval::intervals:interval(),\n        NewEntry::db_entry:entry(), OldEntry::db_entry:entry(), Note::string()) -> ok.\ncheck_entry_in_changed_entries_internal(ChangedEntries, ChangesInterval, NewEntry, OldEntry, Note) ->\n    case intervals:in(db_entry:get_key(NewEntry), ChangesInterval) andalso\n             OldEntry =/= NewEntry of\n        true ->\n            case length([E || E <- ChangedEntries, E =:= NewEntry]) of\n                1 -> ok;\n                _ -> ?ct_fail(\"element(1, db_dht:get_changes(DB)) evaluated \"\n                              \"to \\\"~w\\\" and did not contain 1x changed entry ~w~n(~s)~n\",\n                              [ChangedEntries, NewEntry, lists:flatten(Note)])\n            end;\n        _    -> ok\n    end.\n\n%% @doc Checks that an entry is present exactly once in the list of changed\n%%      entries returned by db_dht:get_changes/1.\n-spec check_entry_in_changes(\n        DB::db_dht:db(), ChangesInterval::intervals:interval(), Entry::db_entry:entry(),\n        OldEntry::db_entry:entry(), Note::string()) -> ok.\ncheck_entry_in_changes(DB, ChangesInterval, NewEntry, OldEntry, Note) ->\n    {ChangedEntries, DeletedKeys} = db_dht:get_changes(DB),\n    case db_entry:is_null(NewEntry) of\n        true ->\n            check_key_in_deleted_internal(DeletedKeys, ChangesInterval,\n                                          db_entry:get_key(NewEntry),\n                                          not db_entry:is_null(OldEntry), Note);\n        _ ->\n            check_entry_in_changed_entries_internal(ChangedEntries, ChangesInterval, NewEntry, OldEntry, Note)\n    end.\n\n-spec count_keys_in_range(Keys::[?RT:key()], Interval::intervals:interval()) -> non_neg_integer().\ncount_keys_in_range(Keys, Interval) ->\n    lists:foldl(fun(Key, Count) ->\n                        case intervals:in(Key, Interval) of\n                            true -> Count + 1;\n                            _    -> Count\n                        end\n                end, 0, Keys).\n"
  },
  {
    "path": "test/db_ets_SUITE.erl",
    "content": "% @copyright 2010-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc    Unit tests for ets backend.\n%% @end\n-module(db_ets_SUITE).\n\n-author('fajerski@zib.de').\n\n-compile(export_all).\n\n-define(TEST_DB, db_ets).\n-define(CLOSE, close).\n\n-include(\"db_backend_SUITE.hrl\").\n\ngroups() ->\n    [{tester_tests, [sequence], tests_avail()}].\n\nall() ->\n    [\n     {group, tester_tests}\n    ].\n\nsuite() -> [ {timetrap, {seconds, 50}} ].\n\ninit_per_suite(Config) ->\n    tester:register_type_checker({typedef, db_backend_beh, key, []}, db_backend_beh, tester_is_valid_db_key),\n    tester:register_value_creator({typedef, db_backend_beh, key, []}, db_backend_beh, tester_create_db_key, 1),\n    Config.\n\nend_per_suite(_Config) ->\n    tester:unregister_type_checker({typedef, db_backend_beh, key, []}),\n    tester:unregister_value_creator({typedef, db_backend_beh, key, []}),\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\nrw_suite_runs(N) ->\n    erlang:min(N, 10000).\n"
  },
  {
    "path": "test/db_generator.erl",
    "content": "%  @copyright 2010-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <MLange@informatik.hu-berlin.de>\n%% @doc    Creates test-DBs for rrepair.\n%% @end\n%% @version $Id$\n-module(db_generator).\n-author('malange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-include(\"scalaris.hrl\").\n\n-export([get_db/3, get_db/4]).\n-export([fill_ring/3]).\n-export([insert_db/1, remove_keys/1]).\n-export([is_old_value/1]).\n\n% for tester:\n-export([get_db_feeder/3, get_db_feeder/4]).\n-export([fill_ring_feeder/3]).\n-export([feeder_fix_rangen/2]).\n\n-define(VersionMin, 1).\n-define(VersionMax, 1048576). % 1024*1024\n-define(VersionDiffMax, 512).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% TYPES\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-export_type([distribution/0, db_distribution/0, option/0,\n              db_type/0, db_parameter/0, db_status/0, failure_type/0]).\n\n-type distribution()    :: random |\n                           uniform |\n                           {non_uniform, random_bias:generator()}.\n-type db_distribution() :: distribution().\n-type result_k()  :: ?RT:key().\n-type result_ktpl():: {?RT:key()}.\n-type result_kv() :: {?RT:key(), db_dht:value()}.\n-type result()    :: result_k() | result_kv() | result_ktpl().\n-type option()    :: {output, list_key_val | list_key | list_keytpl}.\n\n-type failure_type()    :: update | regen | mixed.\n-type failure_quadrant():: rt_beh:segment().\n-type failure_dest()    :: all | [failure_quadrant(),...].\n-type db_type()         :: wiki | random.\n-type db_parameter()    :: {ftype, failure_type()} |\n                           {fprob, 0..100} |                  %failure probability\n                           {fdest, failure_dest()} |          %quadrants in which failures are inserted / if missing all is assumed\n                           {fdistribution, distribution()} |  %distribution of failures\n                           {distribution, db_distribution()}. %data distribution in every quadrant\n-type db_status() :: {Entries   :: non_neg_integer(),\n                      Existing  :: non_neg_integer(),\n                      Missing   :: non_neg_integer(),\n                      Outdated  :: non_neg_integer()}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_db_feeder(intervals:continuous_interval(), 0..1000, db_distribution())\n        -> {intervals:continuous_interval(), non_neg_integer(), db_distribution()}.\nget_db_feeder(I0, Count0, Distribution0) ->\n    {I, Count, Distribution, _} = get_db_feeder(I0, Count0, Distribution0, []),\n    {I, Count, Distribution}.\n\n%% @doc This will generate a list of up to [ItemCount] keys with the requested\n%%      distribution in the given interval.\n-spec get_db(intervals:continuous_interval(), non_neg_integer(), db_distribution()) -> [result_k()].\nget_db(I, Count, Distribution) ->\n    get_db(I, Count, Distribution, []).\n\n-spec get_db_feeder(intervals:continuous_interval(), 0..1000, db_distribution(), [option()])\n        -> {intervals:continuous_interval(), non_neg_integer(), db_distribution(), [option()]}.\nget_db_feeder(I, Count, Distribution = random, Options) ->\n    {I, Count, Distribution, Options};\nget_db_feeder(I, Count, Distribution = uniform, Options) ->\n    {I, Count, Distribution, Options};\nget_db_feeder(I, Count0, Distribution0 = {non_uniform, RanGen}, Options) ->\n    Count = erlang:min(Count0, random_bias:numbers_left(RanGen)),\n    {I, Count, feeder_fix_rangen(Distribution0, Count), Options}.\n\n-spec get_db(intervals:continuous_interval(), non_neg_integer(), db_distribution(), [option()]) -> [result()].\nget_db(Interval, ItemCount, Distribution, Options) ->\n    ?DBG_ASSERT(intervals:is_continuous(Interval)),\n    ?DBG_ASSERT(Distribution =:= feeder_fix_rangen(Distribution, ItemCount)),\n    OutputType = proplists:get_value(output, Options, list_key),\n    case Distribution of\n        random -> gen_random(Interval, ItemCount, OutputType);\n        uniform -> uniform_key_list([{Interval, ItemCount}], [], OutputType);\n        {non_uniform, RanGen} -> non_uniform_key_list(Interval, ItemCount, RanGen, [], OutputType)\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec gen_random\n        (Interval::intervals:continuous_interval(), ToAdd::non_neg_integer(),\n         OutputType::list_key) -> [result_k()];\n        (Interval::intervals:continuous_interval(), ToAdd::non_neg_integer(),\n         OutputType::list_keytpl) -> [result_ktpl()];\n        (Interval::intervals:continuous_interval(), ToAdd::non_neg_integer(),\n         OutputType::list_key_val) -> [result_kv()].\ngen_random(I, Add, OutputType) ->\n    SimpleI = intervals:get_bounds(I),\n    ToAdd0 = gb_sets:from_list(?RT:get_random_in_interval(SimpleI, Add)),\n    gen_random_gb_sets(SimpleI, Add - gb_sets:size(ToAdd0), OutputType, ToAdd0, 1).\n\n-spec gen_random_gb_sets\n        (Interval::intervals:simple_interval2(), ToAdd::non_neg_integer(),\n         OutputType::list_key, Set::gb_sets:set(?RT:key()), Retries::non_neg_integer())\n        -> [result_k()];\n        (Interval::intervals:simple_interval2(), ToAdd::non_neg_integer(),\n         OutputType::list_keytpl, Set::gb_sets:set(?RT:key()), Retries::non_neg_integer())\n        -> [result_ktpl()];\n        (Interval::intervals:simple_interval2(), ToAdd::non_neg_integer(),\n         OutputType::list_key_val, Set::gb_sets:set(?RT:key()), Retries::non_neg_integer())\n        -> [result_kv()].\ngen_random_gb_sets(_I, ToAdd, OutputType, Set, Retry)\n  when ToAdd =:= 0 orelse Retry =:= 3 ->\n    % abort after 3 random keys already in Tree / probably no more free keys in I\n    KeyList = gb_sets:to_list(Set),\n    case OutputType of\n        list_key     -> [Key || Key <- KeyList];\n        list_keytpl  -> [{Key} || Key <- KeyList];\n        list_key_val -> [{Key, gen_value()} || Key <- KeyList]\n    end;\ngen_random_gb_sets(I, ToAdd, OutputType, Set, Retry) ->\n    NewKey = ?RT:get_random_in_interval(I),\n    case gb_sets:is_member(NewKey, Set) of\n        true ->\n            gen_random_gb_sets(I, ToAdd, OutputType, Set, Retry + 1);\n        false ->\n            NewSet = gb_sets:add(NewKey, Set),\n            gen_random_gb_sets(I, ToAdd - 1, OutputType, NewSet, 0)\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec uniform_key_list\n        ([{Interval::intervals:continuous_interval(), ToAdd::non_neg_integer()}],\n         Acc::[result_k()], OutputType::list_key) -> [result_k()];\n        ([{Interval::intervals:continuous_interval(), ToAdd::non_neg_integer()}],\n         Acc::[result_ktpl()], OutputType::list_keytpl) -> [result_ktpl()];\n        ([{Interval::intervals:continuous_interval(), ToAdd::non_neg_integer()}],\n         Acc::[result_kv()], OutputType::list_key_val) -> [result_kv()].\nuniform_key_list([], Acc, _) -> Acc;\nuniform_key_list([{I, Add} | R] = Cmd, Acc, AccType) ->\n    if Add > 100 ->\n           case intervals:split(I, 2) of\n               [I1, I2] ->\n                   AddD2 = Add div 2,\n                   uniform_key_list([{I1, AddD2}, {I2, AddD2 + (Add rem 2)} | R],\n                                    Acc, AccType);\n               [I] ->\n                   uniform_key_list_no_split(Cmd, Acc, AccType)\n           end;\n       true ->\n           uniform_key_list_no_split(Cmd, Acc, AccType)\n    end.\n\n-spec uniform_key_list_no_split\n        ([{Interval::intervals:continuous_interval(), ToAdd::non_neg_integer()}],\n         Acc::[result_k()], OutputType::list_key) -> [result_k()];\n        ([{Interval::intervals:continuous_interval(), ToAdd::non_neg_integer()}],\n         Acc::[result_ktpl()], OutputType::list_keytpl) -> [result_ktpl()];\n        ([{Interval::intervals:continuous_interval(), ToAdd::non_neg_integer()}],\n         Acc::[result_kv()], OutputType::list_key_val) -> [result_kv()].\nuniform_key_list_no_split([{_I, 0} | R], Acc, AccType) ->\n    uniform_key_list(R, Acc, AccType);\nuniform_key_list_no_split([{I, Add} | R], Acc, AccType) ->\n    {LBr, IL, IR, RBr} = intervals:get_bounds(I),\n    Denom = Add + ?IIF(RBr =:= ')' andalso LBr =:= '(', 1, 0),\n    ToAddKeys = lists:usort(\n                  util:for_to_ex(\n                    ?IIF(LBr =:= '(', 1, 0),\n                    Add + ?IIF(LBr =:= '(', 0, -1),\n                    fun(Index) ->\n                            ?RT:get_split_key(IL, IR, {Index, Denom})\n                    end)),\n    % note: ?RT:get_split_key/3 considers the interval (IL, IR] which may not\n    %       be correct here\n    ToAdd =\n        case AccType of\n            list_key ->\n                [Key || Key <- ToAddKeys, intervals:in(Key, I)];\n            list_keytpl ->\n                [{Key} || Key <- ToAddKeys, intervals:in(Key, I)];\n            list_key_val ->\n                [{Key, gen_value()} || Key <- ToAddKeys, intervals:in(Key, I)]\n        end,\n    uniform_key_list(R, lists:append(ToAdd, Acc), AccType).\n\n-spec gen_value() -> db_dht:value().\ngen_value() ->\n    tester_value_creator:create_value(integer, 0, tester_parse_state:new_parse_state()).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec non_uniform_key_list\n        (Interval::intervals:continuous_interval(), ToAdd::non_neg_integer(),\n         RanGen::random_bias:generator(), Acc::[result_k()],\n         OutputType::list_key) -> [result_k()];\n        (Interval::intervals:continuous_interval(), ToAdd::non_neg_integer(),\n         RanGen::random_bias:generator(), Acc::[result_ktpl()],\n         OutputType::list_keytpl) -> [result_ktpl()];\n        (Interval::intervals:continuous_interval(), ToAdd::non_neg_integer(),\n         RanGen::random_bias:generator(), Acc::[result_kv()],\n         OutputType::list_key_val) -> [result_kv()].\nnon_uniform_key_list(_I, 0, _RanGen, Acc, _AccType) -> Acc;\nnon_uniform_key_list(I, ToAdd, RanGen, Acc, AccType) ->\n    ?DBG_ASSERT(ToAdd =:= 1 orelse random_bias:numbers_left(RanGen) =:= ToAdd),\n    SubIntervals = intervals:split(I, ToAdd),\n    non_uniform_key_list_(SubIntervals, ToAdd, RanGen, Acc, length(Acc), AccType, 0.0).\n\n-spec non_uniform_key_list_\n        (SubIs::[intervals:continuous_interval(),...], ToAdd::non_neg_integer(),\n         Fun::random_bias:generator(), Acc::[result_k()], AccLen::non_neg_integer(),\n         OutputType::list_key, RoundingError::float()) -> [result_k()];\n        (SubIs::[intervals:continuous_interval(),...], ToAdd::non_neg_integer(),\n         Fun::random_bias:generator(), Acc::[result_ktpl()], AccLen::non_neg_integer(),\n         OutputType::list_keytpl, RoundingError::float()) -> [result_ktpl()];\n        (SubIs::[intervals:continuous_interval(),...], ToAdd::non_neg_integer(),\n         Fun::random_bias:generator(), Acc::[result_kv()], AccLen::non_neg_integer(),\n         OutputType::list_key_val, RoundingError::float()) -> [result_kv()].\nnon_uniform_key_list_([SubI | R], ToAdd, RanGen, Acc, AccLen, AccType, RoundingError) ->\n    ?DBG_ASSERT(not intervals:is_empty(SubI)),\n    {Status, V, RanGen1} = random_bias:next(RanGen),\n    Add0 = V * ToAdd + RoundingError,\n    Add1 = erlang:max(0, erlang:trunc(Add0)),\n    Add = if Add1 + AccLen > ToAdd -> ToAdd - AccLen;\n              R =:= [] ->\n                  % try to add the missing items to the last sub interval\n                  ToAdd - AccLen;\n             true -> Add1\n          end,\n    %log:pal(\"non_uniform_key_list: add: ~p, SubI: ~p\", [Add, SubI]),\n    NAcc0 = if Add >= 1 -> uniform_key_list([{SubI, Add}], [], AccType);\n               true     -> []\n            end,\n    NAcc0Len = length(NAcc0),\n    NAccLen = AccLen + NAcc0Len,\n    % note: the ordering of the returned keys is not important\n    %       -> append to smaller list (better performance)\n    case Status of\n        _ when NAccLen =:= ToAdd ->\n            lists:append(NAcc0, Acc);\n        ok when R =/= [] ->\n            NAcc = lists:append(NAcc0, Acc),\n            NewRoundingError = float(Add0 - Add + Add - NAcc0Len),\n            non_uniform_key_list_(R, ToAdd, RanGen1, NAcc, NAccLen, AccType, NewRoundingError);\n        _ when NAccLen < ToAdd ->\n            % there may still be missing items -> try to add them to the last\n            % sub interval again - beware not to create duplicate keys!\n            Acc1 = uniform_key_list([{SubI, ToAdd - NAccLen}], NAcc0, AccType),\n            lists:append(lists:usort(Acc1), Acc)\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Helper functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec fill_ring_feeder(random, {1..1000, 1..1000}, [db_parameter()])\n        -> {db_type(), pos_integer(), [db_parameter()]}.\nfill_ring_feeder(Type, {Size0, Size1}, Params) ->\n    % failures must be less than DB size!\n    if Size0 >= Size1 -> DBSize = Size0, FSize = Size1;\n       true           -> DBSize = Size1, FSize = Size0\n    end,\n    Params1 =\n        case proplists:get_value(distribution, Params, uniform) of\n            {non_uniform, _DBRanGen} = DBDist0 ->\n                [{distribution, feeder_fix_rangen(DBDist0, DBSize)} |\n                     proplists:delete(distribution, Params)];\n            _ -> Params\n        end,\n    % set the given number of failures into the random number generator\n    Params2 =\n        case proplists:get_value(fdistribution, Params1, uniform) of\n            {non_uniform, _FRanGen} = EDist0 ->\n                [{fdistribution, feeder_fix_rangen(EDist0, FSize)} |\n                     proplists:delete(fdistribution, Params1)];\n            _ -> Params1\n        end,\n    {Type, DBSize, Params2}.\n\n% @doc  DBSize=Number of Data Entities in DB (without replicas)\n-spec fill_ring(db_type(), pos_integer(), [db_parameter()]) -> db_status().\nfill_ring(Type, DBSize, Params) ->\n    case Type of\n        random -> fill_random(DBSize, Params);\n        wiki -> fill_wiki(Params, \"barwiki-latest-pages-meta-current.db\")\n    end.\n\n-compile({nowarn_unused_function, {fill_random_feeder, 2}}).\n-spec fill_random_feeder({1..1000, 1..1000}, [db_parameter()])\n        -> {pos_integer(), [db_parameter()]}.\nfill_random_feeder(Sizes0, Params0) ->\n    {random, DBSize, Params} = fill_ring_feeder(random, Sizes0, Params0),\n    {DBSize, Params}.\n\n-spec fill_random(DBSize::pos_integer(), [db_parameter()]) -> db_status().\nfill_random(DBSize, Params) ->\n    Distr = proplists:get_value(distribution, Params, uniform),\n    I = hd(intervals:split(intervals:all(), config:read(replication_factor))),\n    {_LBr, LKey, RKey, _RBr} = intervals:get_bounds(I),\n    I2 = intervals:new('(', LKey, RKey, ')'),\n    Keys = get_db(I2, DBSize, Distr),\n    {DB, DBStatus} = gen_kvv(proplists:get_value(fdistribution, Params, uniform), Keys, Params),\n    insert_db(DB),\n    DBStatus.\n\n-spec fill_wiki([db_parameter()], DBFile::string()) -> db_status().\nfill_wiki(_Params, DBFile) ->\n    Result = os:cmd(\"ant -buildfile ../contrib/wikipedia/build.xml import-db \"\n                   \"-Ddata=\\\"../contrib/wikipedia/\" ++ DBFile ++ \"\\\" \"\n                   \"-Dnumber_of_importers=1 -Dmy_import_number=1 \"\n                   \"-Dscalaris.node=\\\"\" ++ atom_to_list(erlang:node()) ++ \"\\\" \"\n                   \"-Dscalaris.cookie=\\\"\" ++ atom_to_list(erlang:get_cookie()) ++ \"\\\"\"),\n    io:format(\"~s~n\", [Result]),\n    %TODO\n    {0, 0, 0, 0}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Adds the given db entries to the DBs of the corresponding dht_node\n%%      processes.\n-spec insert_db(db_dht:db_as_list()) -> ok.\ninsert_db(KVV) ->\n    Nodes = get_node_list(),\n    _ = lists:foldl(\n          fun(Node, ActKVV) ->\n                  comm:send(Node, {get_state, comm:this(), my_range}),\n                  trace_mpath:thread_yield(),\n                  receive ?SCALARIS_RECV({get_state_response, NRange}, ok) end,\n                  {NKVV, RestKVV} = lists:partition(\n                                      fun(Entry) ->\n                                              intervals:in(db_entry:get_key(Entry), NRange)\n                                      end, ActKVV),\n                  comm:send(Node, {add_data, comm:this(), NKVV}),\n                  RestKVV\n          end,\n          KVV, Nodes),\n    _ = [begin\n             trace_mpath:thread_yield(),\n             receive ?SCALARIS_RECV({add_data_reply}, ok) end\n         end || _Node <- Nodes],\n    ok.\n\n%% @doc Removes all DB entries with the given keys from the corresponding\n%%      dht_node processes.\n-spec remove_keys([?RT:key()]) -> ok.\nremove_keys(Keys) ->\n    _ = [begin\n             comm:send(Node, {delete_keys, comm:this(), Keys})\n         end || Node <- get_node_list()],\n    _ = [begin\n             trace_mpath:thread_yield(),\n             receive ?SCALARIS_RECV({delete_keys_reply}, ok) end\n         end || _Node <- get_node_list()],\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Binomial distributions should have an N so that the number of\n%%      random numbers matches the number of e.g. items they should be used for.\n%%      Use this to fix the value of auto-generated generators.\n%%      Also pay attention that if no approximation is used in the binomial\n%%      calculation, getting random numbers for high N is expensive!\n-spec feeder_fix_rangen(distribution(), pos_integer()) -> distribution().\nfeeder_fix_rangen({non_uniform, {{binom, _N, P, X, Approx}, CalcFun, NewStateFun}}, MaxN) ->\n    {non_uniform, {{binom, erlang:max(1, MaxN - 1), P, X, Approx}, CalcFun, NewStateFun}};\nfeeder_fix_rangen(RanGen, _MaxN) ->\n    RanGen.\n\n-compile({nowarn_unused_function, {gen_kvv_feeder, 3}}).\n-spec gen_kvv_feeder(ErrorDist::distribution(), [?RT:key()], [db_parameter()])\n        -> {ErrorDist::distribution(), [?RT:key()], [db_parameter()]}.\ngen_kvv_feeder(EDist0, Keys0, Params) ->\n    Keys = lists:usort(Keys0),\n    KeysLen0 = length(Keys),\n    case EDist0 of\n        {non_uniform, RanGen} ->\n            KeysLen = erlang:min(KeysLen0, random_bias:numbers_left(RanGen)),\n            EDist = feeder_fix_rangen(EDist0, erlang:max(1, KeysLen)),\n            {EDist, lists:sublist(Keys, KeysLen), Params};\n        _ ->\n            {EDist0, Keys, Params}\n    end.\n\n%% @doc Generates a consistent db with errors\n%%      Note: the random number generator in {non_uniform, RanGen} will start\n%%            from scratch each time this function is called based on the\n%%            initial state!\n-spec gen_kvv(ErrorDist::distribution(), [?RT:key()], [db_parameter()]) -> {db_dht:db_as_list(), db_status()}.\ngen_kvv(EDist, Keys, Params) ->\n    case length(Keys) of\n        0 -> {[], {_Entries = 0, _Existing = 0, _Missing = 0, _Outdated = 0}};\n        KeyCount ->\n            FType = proplists:get_value(ftype, Params, update),\n            FProb = proplists:get_value(fprob, Params, 50),\n            FDest = proplists:get_value(fdest, Params, all),\n            FCount =  erlang:round(KeyCount * (FProb / 100)),\n            p_gen_kvv(EDist, Keys, KeyCount, FType, FDest, FCount)\n    end.\n\n-compile({nowarn_unused_function, {p_gen_kvv_feeder, 6}}).\n-spec p_gen_kvv_feeder(ErrorDist::distribution(), [?RT:key(),...], KeyCount::0,\n                       failure_type(), failure_dest(), FailCount::non_neg_integer())\n        -> {ErrorDist::distribution(), [?RT:key(),...], KeyCount::pos_integer(),\n            failure_type(), failure_dest(), FailCount::non_neg_integer()}.\np_gen_kvv_feeder(EDist0, Keys0, _WrongKeyCount, FType, FDest, FCount) ->\n    Keys = lists:usort(Keys0),\n    KeysLen0 = length(Keys),\n    case EDist0 of\n        {non_uniform, RanGen} ->\n            KeysLen = erlang:min(KeysLen0, random_bias:numbers_left(RanGen)),\n            EDist = feeder_fix_rangen(EDist0, KeysLen),\n            {EDist, lists:sublist(Keys, KeysLen), KeysLen, FType, FDest, FCount};\n        _ ->\n            {EDist0, Keys, KeysLen0, FType, FDest, FCount}\n    end.\n\n-spec p_gen_kvv(ErrorDist::distribution(), [?RT:key(),...],\n                KeyCount::pos_integer(), failure_type(),\n                failure_dest(), FailCount::non_neg_integer()) -> {db_dht:db_as_list(), db_status()}.\np_gen_kvv(random, Keys, KeyCount, FType, FDest, FCount) ->\n    ?DBG_ASSERT(length(Keys) =:= length(lists:usort(Keys))), % unique keys\n    KeysWithVersions =\n        lists:zip(Keys, randoms:rand_uniform(?VersionMin, ?VersionMax, KeyCount)),\n    {GoodKeys, FKeys} = util:pop_randomsubset(FCount, KeysWithVersions),\n    GoodDB = lists:flatmap(fun get_rep_group/1, GoodKeys),\n    {DB, O} = lists:foldl(fun(FKey, {AccAll, Out}) ->\n                                  {RList, NewOut} = get_failure_rep_group(FKey, FType, FDest),\n                                  {lists:append(RList, AccAll), Out + NewOut}\n                          end,\n                          {GoodDB, 0}, FKeys),\n    Insert = length(DB),\n    DBSize = KeyCount * config:read(replication_factor),\n    {DB, {DBSize, Insert, DBSize - Insert, O}};\np_gen_kvv({non_uniform, RanGen}, Keys, KeyCount, FType, FDest, FCount) ->\n    ?DBG_ASSERT(length(Keys) =:= length(lists:usort(Keys))), % unique keys\n    ?DBG_ASSERT(KeyCount =:= 1 orelse random_bias:numbers_left(RanGen) =< KeyCount),\n    KeysWithVersions =\n        lists:zip(Keys, randoms:rand_uniform(?VersionMin, ?VersionMax, KeyCount)),\n    FProbList = get_non_uniform_probs(RanGen),\n    % note: don't use RanGen any more - we don't get the new state in the last call!\n    CellLength = case length(FProbList) of\n                   0 -> KeyCount + 1;\n                   FProbL -> erlang:round(KeyCount / FProbL)\n               end,\n    FCells = lists:reverse(\n               lists:keysort(1, build_failure_cells(FProbList, KeysWithVersions,\n                                                    CellLength, []))),\n    {DB, _, Out} =\n        lists:foldl(fun({_P, Cell}, {DB, RestF, ROut}) ->\n                            {NewEntry, NewOut, NewF} =\n                                add_failures_to_cell(Cell, RestF, FType, FDest, {[], ROut}),\n                            {lists:append(NewEntry, DB), NewF, NewOut}\n                    end,\n                    {[], FCount, 0}, FCells),\n    Insert = length(DB),\n    DBSize = KeyCount * config:read(replication_factor),\n    {DB, {DBSize, Insert, DBSize - Insert, Out}};\np_gen_kvv(uniform, Keys, KeyCount, FType, FDest, FCount) ->\n    ?DBG_ASSERT(length(Keys) =:= length(lists:usort(Keys))), % unique keys\n    KeysWithVersions =\n        lists:zip(Keys, randoms:rand_uniform(?VersionMin, ?VersionMax, KeyCount)),\n    FRate = case FCount of\n                0 -> KeyCount + 1;\n                _ -> erlang:max(1, erlang:trunc(KeyCount / FCount))\n            end,\n    {DB, O, _, _} =\n        lists:foldl(\n          fun(Key, {AccDb, Out, Count, FCRest}) ->\n                  {{RList, AddOut}, FCNew} =\n                      case Count rem FRate of\n                          0 when FCRest > 0 ->\n                              {get_failure_rep_group(Key, FType, FDest), FCRest - 1};\n                          _ -> {{get_rep_group(Key), 0}, FCRest}\n                      end,\n                  {lists:append(RList, AccDb), Out + AddOut, Count + 1, FCNew}\n          end,\n          {[], 0, 1, FCount}, KeysWithVersions),\n    Insert = length(DB),\n    DBSize = KeyCount * config:read(replication_factor),\n    {DB, {DBSize, Insert, DBSize - Insert, O}}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec add_failures_to_cell([{?RT:key(), Version::db_dht:version()}], non_neg_integer(), failure_type(), failure_dest(),\n                           Acc::{[db_entry:entry()], Outdated::non_neg_integer()})\n        -> Result::{[db_entry:entry()], Outdated::non_neg_integer(), RestFCount::non_neg_integer()}.\nadd_failures_to_cell([], FCount, _FType, _FDest, {AccE, AccO}) ->\n    {AccE, AccO, FCount};\nadd_failures_to_cell([H | T], 0, FType, FDest, {DB, Out}) ->\n    add_failures_to_cell(T, 0, FType, FDest,\n                         {lists:append(get_rep_group(H), DB), Out});\nadd_failures_to_cell([H | T], FCount, FType, FDest, {AccEntry, AccOut}) ->\n    {AddEntrys, AddOut} = get_failure_rep_group(H, FType, FDest),\n    add_failures_to_cell(T, FCount - 1, FType, FDest,\n                         {lists:append(AddEntrys, AccEntry), AddOut + AccOut}).\n\n%% @doc Groups Keys into cells of equal size with failure probabilities\n%%      assigned from FProbs.\n-spec build_failure_cells(FProbs::[float()], Keys::[{?RT:key(), Version::db_dht:version()}],\n                          CellLength::pos_integer(),\n                          Acc::[{float(), [{?RT:key(), Version::db_dht:version()}]}])\n        -> Result::[{float(), [{?RT:key(), Version::db_dht:version()}]}].\nbuild_failure_cells([], [], _CellLength, Acc) ->\n    Acc;\nbuild_failure_cells([], T, _CellLength, []) ->\n    [{0.0, T}];\nbuild_failure_cells([], T, _CellLength, [{P, Cell} | Acc]) ->\n    [{P, lists:append(T, Cell)} | Acc];\nbuild_failure_cells(_P, [], _CellLength, Acc) ->\n    Acc;\nbuild_failure_cells([P | T], List, CellLength, Acc) ->\n    ?DBG_ASSERT(CellLength > 0), % otherwise no progress and endless loop!\n    {Cell, LT} = util:safe_split(CellLength, List),\n    build_failure_cells(T, LT, CellLength, [{P, Cell}|Acc]).\n\n-spec get_non_uniform_probs(random_bias:generator()) -> [float()].\nget_non_uniform_probs(RanGen) ->\n    case random_bias:next(RanGen) of\n        {ok, V, RanGen1} -> [V | get_non_uniform_probs(RanGen1)];\n        {last, V, exit}  -> [V]\n    end.\n\n%% TODO: we need to create random values (prefixed them with old/new for better\n%%       debugging) if we want to measure bytes to transfer\n-spec get_synthetic_value(old | new) -> term().\nget_synthetic_value(new) ->\n    val;\nget_synthetic_value(old) ->\n    old.\n\n%% @doc Determines whether a given value is outdated as created by\n%%      get_synthetic_value/1.\n-spec is_old_value(db_dht:value()) -> boolean().\nis_old_value(old) -> true;\nis_old_value(_) -> false.\n\n-spec get_rep_group({?RT:key(), Version::db_dht:version()}) -> [db_entry:entry()].\nget_rep_group({Key, Version}) ->\n    Value = get_synthetic_value(new),\n    [db_entry:new(K, Value, Version) || K <- ?RT:get_replica_keys(Key)].\n\n-spec get_failure_rep_group({?RT:key(), Version::db_dht:version()},\n                            failure_type(), failure_dest()) ->\n          {[db_entry:entry()], Outdated::0 | 1}.\nget_failure_rep_group({Key, VersionNew}, FType, FDest) ->\n    RepKeys = ?RT:get_replica_keys(Key),\n    EKey = get_error_key(RepKeys, FDest),\n    ValueNew = get_synthetic_value(new),\n    RGrp = [db_entry:new(K, ValueNew, VersionNew) || K <- RepKeys, K =/= EKey],\n    case get_failure_type(FType) of\n        update ->\n            VersionOld = erlang:max(0, VersionNew - randoms:rand_uniform(1, ?VersionDiffMax)),\n            ValueOld = get_synthetic_value(old),\n            {[db_entry:new(EKey, ValueOld, VersionOld) | RGrp], 1};\n        regen -> {RGrp, 0}\n    end.\n\n% @doc Resolves failure type mixed.\n-spec get_failure_type(failure_type()) -> regen | update.\nget_failure_type(mixed) ->\n    case randoms:rand_uniform(1, 3) of\n        1 -> update;\n        _ -> regen\n    end;\nget_failure_type(Type) -> Type.\n\n% @doc Returns one key of KeyList which is in any quadrant out of DestList.\n-spec get_error_key([?RT:key(),...], failure_dest()) -> ?RT:key().\nget_error_key(RepKeys, all) ->\n    util:randomelem(RepKeys);\nget_error_key(RepKeys, QList) ->\n    rr_recon:map_rkeys_to_quadrant(RepKeys, util:randomelem(QList)).\n\n-spec get_node_list() -> [comm:mypid()].\nget_node_list() ->\n    mgmt_server:node_list(),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n            {get_list_response, List}, %% ->\n            List\n          )\n    end.\n"
  },
  {
    "path": "test/db_generator_SUITE.erl",
    "content": "%  @copyright 2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc Unit tests for test/db_generator.erl.\n%% @end\n%% @version $Id$\n-module(db_generator_SUITE).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\nall() ->\n    [\n     tester_get_db3,\n     tester_get_db4\n     ].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 90}}\n    ].\n\ninit_per_suite(Config) ->\n    tester:register_type_checker({typedef, intervals, interval, []}, intervals, is_well_formed),\n    tester:register_type_checker({typedef, intervals, continuous_interval, []}, intervals, is_continuous),\n    tester:register_value_creator({typedef, random_bias, generator, []},\n                                  random_bias, tester_create_generator, 3),\n    tester:register_value_creator({typedef, intervals, interval, []}, intervals, tester_create_interval, 1),\n    tester:register_value_creator({typedef, intervals, continuous_interval, []}, intervals, tester_create_continuous_interval, 4),\n    rt_SUITE:register_value_creator(),\n    unittest_helper:start_minimal_procs(Config, [], true).\n\nend_per_suite(Config) ->\n    tester:unregister_value_creator({typedef, random_bias, generator, []}),\n    tester:unregister_value_creator({typedef, intervals, interval, []}),\n    tester:unregister_value_creator({typedef, intervals, continuous_interval, []}),\n    tester:unregister_type_checker({typedef, intervals, interval, []}),\n    tester:unregister_type_checker({typedef, intervals, continuous_interval, []}),\n    rt_SUITE:unregister_value_creator(),\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_generator:get_db/3 and db_generator:get_db/4\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_get_db3(I::intervals:continuous_interval(), ItemCount::1..1000,\n                   db_generator:db_distribution()) -> true.\nprop_get_db3(Interval, ItemCount0, Distribution0 = {non_uniform, RanGen}) ->\n    ItemCount = erlang:min(ItemCount0, random_bias:numbers_left(RanGen)),\n    prop_get_db3_(Interval, ItemCount, db_generator:feeder_fix_rangen(Distribution0, ItemCount));\nprop_get_db3(Interval, ItemCount, Distribution) ->\n    prop_get_db3_(Interval, ItemCount, Distribution).\n\n-spec prop_get_db3_(I::intervals:continuous_interval(), ItemCount::1..1000,\n                    db_generator:db_distribution()) -> true.\nprop_get_db3_(Interval, ItemCount, Distribution) ->\n    Result = db_generator:get_db(Interval, ItemCount, Distribution),\n    ?equals([Key || Key <- Result,\n                    not intervals:in(Key, Interval)],\n            []),\n    ?compare(fun erlang:'=<'/2, length(Result), ItemCount),\n    ?equals(length(Result), length(lists:usort(Result))),\n    ?implies(length(intervals:split(Interval, ItemCount)) == ItemCount,\n             ?equals(length(Result), ItemCount)).\n\n-spec prop_get_db4(I::intervals:continuous_interval(), ItemCount::1..1000,\n                   db_generator:db_distribution(), Options::[db_generator:option()]) -> true.\nprop_get_db4(Interval, ItemCount0, Distribution0 = {non_uniform, RanGen}, Options) ->\n    ItemCount = erlang:min(ItemCount0, random_bias:numbers_left(RanGen)),\n    prop_get_db4_(Interval, ItemCount, db_generator:feeder_fix_rangen(Distribution0, ItemCount), Options);\nprop_get_db4(Interval, ItemCount, Distribution, Options) ->\n    prop_get_db4_(Interval, ItemCount, Distribution, Options).\n\n\n-spec prop_get_db4_(I::intervals:continuous_interval(), ItemCount::1..1000,\n                    db_generator:db_distribution(), Options::[db_generator:option()]) -> true.\nprop_get_db4_(Interval, ItemCount, Distribution, Options) ->\n    Result = db_generator:get_db(Interval, ItemCount, Distribution, Options),\n    case proplists:get_value(output, Options, list_key) of\n        list_key ->\n            ?equals([Key || Key <- Result,\n                            not intervals:in(Key, Interval)],\n                    []);\n        list_keytpl ->\n            ?equals([Key || {Key} <- Result,\n                            not intervals:in(Key, Interval)],\n                    []);\n        list_key_val ->\n            ?equals([Key || {Key, _Val} <- Result,\n                            not intervals:in(Key, Interval)],\n                    [])\n    end,\n    ?compare(fun erlang:'=<'/2, length(Result), ItemCount),\n    ?equals(length(Result), length(lists:usort(Result))),\n    ?implies(length(intervals:split(Interval, ItemCount)) == ItemCount,\n             ?equals(length(Result), ItemCount)).\n\ntester_get_db3(_Config) ->\n    prop_get_db3(intervals:new(?MINUS_INFINITY), 844, random),\n    tester:test(?MODULE, prop_get_db3, 3, 500, [{threads, 4}]).\n\ntester_get_db4(_Config) ->\n    prop_get_db4(intervals:new(?MINUS_INFINITY), 1,\n                 {non_uniform, random_bias:binomial(50, 0.06755763133001705)},\n                 [{output,list_key}]),\n    tester:test(?MODULE, prop_get_db4, 4, 500, [{threads, 4}]).\n"
  },
  {
    "path": "test/db_hanoidb_SUITE.erl",
    "content": "% @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    Unit tests for hanoidb backend.\n%% @end\n%% @version $Id$\n-module(db_hanoidb_SUITE).\n\n-author('skrzypczak@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-define(TEST_DB, db_hanoidb).\n-define(CLOSE, close_and_delete).\n-define(EQ, =:=).\n\n-include(\"db_backend_SUITE.hrl\").\n\nall() ->\n    case code:which(hanoidb) of\n        non_existing -> [];\n        _            -> lists:append(tests_avail(), [tester_reopen])\n    end.\n\nsuite() -> [ {timetrap, {seconds, 30}} ].\n\ninit_per_suite(Config) ->\n    tester:register_type_checker({typedef, db_backend_beh, key, []}, db_backend_beh, tester_is_valid_db_key),\n    tester:register_value_creator({typedef, db_backend_beh, key, []}, db_backend_beh, tester_create_db_key, 1),\n    unittest_helper:start_minimal_procs(Config, [], false).\n\nend_per_suite(Config) ->\n    tester:unregister_type_checker({typedef, db_backend_beh, key, []}),\n    tester:unregister_value_creator({typedef, db_backend_beh, key, []}),\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\nrw_suite_runs(N) ->\n    erlang:min(N, 200).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_hanoidb:open/1, db_hanoidb getters\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_reopen(Key::?RT:key()) -> true.\nprop_reopen(Key) ->\n    DB = db_hanoidb:new(randoms:getRandomString()),\n    FileName = db_hanoidb:get_name(DB),\n    check_db(DB, [], \"check_db_reopen_1\"),\n    ?equals(db_hanoidb:get(DB, Key), {}),\n    DB1 = db_hanoidb:put(DB, {Key}),\n    db_hanoidb:close(DB1),\n    DB2 = db_hanoidb:open(FileName),\n    check_db(DB2, [{Key}], \"check_db_reopen_2\"),\n    ?equals(db_hanoidb:get(DB2, Key), {Key}),\n    db_hanoidb:close_and_delete(DB2),\n    true.\n\ntester_reopen(_Config) ->\n    tester:test(?MODULE, prop_reopen, 1, rw_suite_runs(10)).\n\n\n"
  },
  {
    "path": "test/db_mnesia_SUITE.erl",
    "content": "% @copyright 2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @author Tanguy Racinet <tanracinet@gmail.com>\n%% @doc    Unit tests for the mnesia backend.\n%% @end\n%% @version $Id$\n-module(db_mnesia_SUITE).\n\n-author('kruber@zib.de').\n-author('tanracinet@gmail.com').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-define(TEST_DB, db_mnesia).\n-define(CLOSE, close).\n-define(EQ, ==).\n\n-include(\"db_backend_SUITE.hrl\").\n\nall() -> tests_avail().\n\nsuite() -> [ {timetrap, {seconds, 60}} ].\n\ninit_per_suite(Config) ->\n    %% need config here to get db path\n    Config2 = unittest_helper:start_minimal_procs(Config, [], false),\n\n    %% cleanup schema generated possibly in earlier failed run\n    PWD = os:cmd(pwd),\n    WorkingDir = string:sub_string(PWD, 1, string:len(PWD) - 1) ++\n        \"/\" ++ config:read(db_directory) ++ \"/\" ++ atom_to_list(erlang:node()) ++ \"/\",\n    _ = file:delete(WorkingDir ++ \"schema.DAT\"),\n\n    ok = db_mnesia:start(),\n    tester:register_type_checker({typedef, db_backend_beh, key, []}, db_backend_beh, tester_is_valid_db_key),\n    tester:register_value_creator({typedef, db_backend_beh, key, []}, db_backend_beh, tester_create_db_key, 1),\n    Config2.\n\nend_per_suite(Config) ->\n    tester:unregister_type_checker({typedef, db_backend_beh, key, []}),\n    tester:unregister_value_creator({typedef, db_backend_beh, key, []}),\n\n    _ = application:stop(mnesia),\n    %% cleanup schema generated in this run\n    PWD = os:cmd(pwd),\n    WorkingDir = string:sub_string(PWD, 1, string:len(PWD) - 1) ++\n        \"/\" ++ config:read(db_directory) ++ \"/\" ++ atom_to_list(erlang:node()) ++ \"/\",\n    _ = file:delete(WorkingDir ++ \"schema.DAT\"),\n\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\nrw_suite_runs(N) ->\n    erlang:min(N, 200).\n"
  },
  {
    "path": "test/db_toke_SUITE.erl",
    "content": "% @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc    Unit tests for tokyo cabinet backend.\n%% @end\n%% @version $Id$\n-module(db_toke_SUITE).\n\n-author('fajerski@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-define(TEST_DB, db_toke).\n-define(CLOSE, close_and_delete).\n-define(EQ, =:=).\n\n-include(\"db_backend_SUITE.hrl\").\n\nall() ->\n    case code:which(toke_drv) of\n        non_existing -> [];\n        _            -> lists:append(tests_avail(), [tester_reopen])\n    end.\n\nsuite() -> [ {timetrap, {seconds, 30}} ].\n\ninit_per_suite(Config) ->\n    tester:register_type_checker({typedef, db_backend_beh, key, []}, db_backend_beh, tester_is_valid_db_key),\n    tester:register_value_creator({typedef, db_backend_beh, key, []}, db_backend_beh, tester_create_db_key, 1),\n    unittest_helper:start_minimal_procs(Config, [], false).\n\nend_per_suite(Config) ->\n    tester:unregister_type_checker({typedef, db_backend_beh, key, []}),\n    tester:unregister_value_creator({typedef, db_backend_beh, key, []}),\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\nrw_suite_runs(N) ->\n    erlang:min(N, 200).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% db_toke:open/1, db_toke getters\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_reopen(Key::?RT:key()) -> true.\nprop_reopen(Key) ->\n    DB = db_toke:new(randoms:getRandomString()),\n    FileName = db_toke:get_name(DB),\n    check_db(DB, [], \"check_db_reopen_1\"),\n    ?equals(db_toke:get(DB, Key), {}),\n    DB1 = db_toke:put(DB, {Key}),\n    db_toke:close(DB1),\n    DB2 = db_toke:open(FileName),\n    check_db(DB2, [{Key}], \"check_db_reopen_2\"),\n    ?equals(db_toke:get(DB2, Key), {Key}),\n    db_toke:close_and_delete(DB2),\n    true.\n\ntester_reopen(_Config) ->\n    tester:test(?MODULE, prop_reopen, 1, rw_suite_runs(10)).\n"
  },
  {
    "path": "test/dc_centroids_SUITE.erl",
    "content": "%  @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Magnus Mueller <mamuelle@informatik.hu-berlin.de>\n%% @doc    Test suite for the dc_centroids module.\n%% @end\n%% @version $Id$\n\n-module(dc_centroids_SUITE).\n-author('mamuelle@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\nall() -> [\n        distance\n    ].\n\nsuite() ->\n    [\n        {timetrap, {seconds, 30}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\n% Test dc_centroids:distance/2\n%\n% This is the euclidian distance between two centroids\ndistance(_Config) ->\n    U = dc_centroids:new([0.0,0.0], 1.0),\n    V = dc_centroids:new([1.0,1.0], 1.0),\n    ?equals(dc_centroids:distance(U,V), math:sqrt(2)),\n    ok.\n"
  },
  {
    "path": "test/dc_clustering_SUITE.erl",
    "content": "% @copyright 2012-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Magnus Mueller <mamuelle@informatik.hu-berlin.de>\n%% @doc    Unit tests for the dc_clustering module.\n%% @end\n%% @version $Id$\n-module(dc_clustering_SUITE).\n-author('mamuelle@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-compile(export_all).\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\nall() -> [\n        single_node\n        , two_nodes\n].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 20}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_testcase(Testcase, Config) ->\n    % dc_clustering must be activated and a radius must exist\n    EnableClustering = {dc_clustering_enable, true},\n    ClusterRadius = {dc_clustering_radius, 1000.0},\n    case Testcase of\n        single_node ->\n            {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n            unittest_helper:make_ring(1, [{config, [\n                            {log_path, PrivDir}\n                            , EnableClustering\n                            , ClusterRadius\n                        ]}]),\n            timer:sleep(500);\n        two_nodes ->\n            {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n            unittest_helper:make_ring(2, [{config, [\n                            {log_path, PrivDir}\n                            , EnableClustering\n                            , ClusterRadius\n                        ]}]),\n            timer:sleep(500)\n    end,\n\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n\n%% Helper function to retrieve the vivaldi coordinate and centroid information of a node\nget_vivaldi_and_centroids(Gossip, Clustering) ->\n    comm:send_local(Gossip, {cb_msg, {gossip_vivaldi, default}, {get_coordinate, comm:this()}}),\n    comm:send_local(Clustering, {query_clustering, comm:this()}),\n\n    % get vivaldi coordinate first\n    Coordinate = receive\n        {vivaldi_get_coordinate_response, Coord, _Confidence} ->\n            Coord\n    end,\n\n    Centroids = receive\n        {query_clustering_response, C} ->\n            C\n    end,\n    {Coordinate, Centroids}.\n\nsingle_node(_) ->\n    % in a ring with only one node, only a single cluster shall exist with the centroid\n    % being the node of the ring\n\n    %% get the node which forms the ring\n    Clustering = pid_groups:find_a(dc_clustering),\n    Group = pid_groups:group_of(Clustering),\n    Gossip = pid_groups:pid_of(Group, gossip),\n\n    {Coordinate, [{centroid, Center, Size}]} = get_vivaldi_and_centroids(Gossip, Clustering),\n    ?equals(Coordinate, Center),\n    ?equals(1.0, Size),\n    ok\n    .\n\ntwo_nodes(_) ->\n    Clustering = pid_groups:find_a(dc_clustering),\n    Group = pid_groups:group_of(Clustering),\n    Gossip = pid_groups:pid_of(Group, gossip),\n\n    {_Coordinate, Centroids} = get_vivaldi_and_centroids(Gossip, Clustering),\n    ?assert(length(Centroids) > 0),\n    ?assert(length(Centroids) < 3),\n    ?equals(1.0, lists:foldl(fun(C, Acc) -> dc_centroids:get_relative_size(C) + Acc end,\n            0, Centroids)),\n    ok\n    .\n"
  },
  {
    "path": "test/default_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{suites, tests, all}.\n\n{skip_suites, tests, [benchmark_SUITE], \"ignore benchmarks\"}.\n{skip_suites, tests, [performance_SUITE], \"ignore benchmarks\"}.\n{skip_suites, tests, [rr_recon_performance_SUITE], \"ignore benchmarks\"}.\n{skip_cases, tests, histogram_SUITE, [perf_add], \"ignore benchmarks\"}.\n\n{skip_suites, tests, [api_tx_proto_sched_SUITE], \"ignore tests with protocol scheduler\"}.\n{skip_suites, tests, [proto_sched_SUITE], \"ignore tests with protocol scheduler\"}.\n{skip_suites, tests, [dht_node_move_proto_sched_SUITE], \"ignore tests with protocol scheduler\"}.\n{skip_suites, tests, [join_leave_proto_sched_SUITE], \"ignore tests with protocol scheduler\"}.\n{skip_suites, tests, [mr_proto_sched_SUITE], \"ignore tests with protocol scheduler\"}.\n{skip_suites, tests, [rrepair_proto_sched_SUITE], \"ignore tests with protocol scheduler\"}.\n\n{skip_suites, tests, [memtest_SUITE], \"ignore memory leak tests\"}.\n{skip_suites, tests, [scalaris_cth_SUITE], \"only for debugging scalaris_cth.erl\"}.\n\n{skip_suites, tests, [slide_leases_SUITE], \"ignore non-deterministic tests\"}.\n\n{skip_suites, tests, [type_check_SUITE], \"ignore type check tests\"}.\n{skip_cases, tests, l_on_cseq_SUITE, [tester_type_check_l_on_cseq], \"ignore type check tests\"}.\n{skip_cases, tests, prbr_SUITE, [tester_type_check_rbr], \"ignore type check tests\"}.\n{skip_cases, tests, rm_leases_SUITE, [tester_type_check_rm_leases], \"ignore type check tests\"}.\n{skip_cases, tests, slide_leases_SUITE, [tester_type_check_slide_leases], \"ignore type check tests\"}.\n"
  },
  {
    "path": "test/dht_node_move_SUITE.erl",
    "content": "% @copyright 2010-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Unit tests for src/dht_node_move.erl (slide and jump operations).\n%% @end\n%% @version $Id$\n-module(dht_node_move_SUITE).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n%% no proto scheduler for this suite\n-define(proto_sched(_Action), ok).\n%% number of slides without timeouts\n-define(NUM_SLIDES, 50).\n\n-include(\"dht_node_move_SUITE.hrl\").\n\nall() ->\n    [\n     {group, send_to_pred},\n     {group, send_to_pred_incremental},\n     {group, send_to_succ},\n     {group, send_to_succ_incremental},\n     {group, send_to_both},\n     {group, send_to_both_incremental},\n     {group, slide_illegally},\n     {group, jump_slide}\n    ].\n\nsuite() -> [ {timetrap, {seconds, 30}} ].\n\n-spec additional_ring_config() -> [].\nadditional_ring_config() ->\n    [].\n"
  },
  {
    "path": "test/dht_node_move_SUITE.hrl",
    "content": "% @copyright 2010-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Unit tests for src/dht_node_move.erl (slide and jump operations).\n%%   dht_node_move_SUITE.erl:\n%%       The regular execution of the suite.\n%%   dht_node_move_proto_sched_SUITE.erl:\n%%       Executes using the proto scheduler which serializes all messages to\n%%       generate a random interleaving of messages on different channels.\n%% @end\n%% @version $Id$\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\n\n-type(node_type() :: predspred | pred | node | succ).\n\n-type(node_tuple() :: {node:node_type(), node:node_type(), node:node_type(), node:node_type()}).\n\n-type(targetid_fun() :: fun((node_tuple()) -> ?RT:key())).\n\n-record(slideconf, {name :: atom(),\n                    node :: node_type(),\n                    slide_with :: node_type(),\n                    targetid ::  targetid_fun()}).\n\n-type(slide_config_record() :: #slideconf{}).\n\n\ntest_cases() -> [].\n\ngroups() ->\n    MoveConfig = {move_config, move_config_parameters()},\n    MoveConfigInc = {move_config, move_config_parameters_incremental()},\n    MoveConfigInc2 = {move_config, move_config_parameters_incremental2()},\n    GroupOptions = [sequence, {repeat, 1}],\n    Config = [MoveConfig | GroupOptions],\n    ConfigInc = [MoveConfigInc | GroupOptions],\n    ConfigInc2 = [MoveConfigInc2 | GroupOptions],\n    SendToPredTestCases =\n        [\n         symm4_slide_succ_rcv_load,\n         symm4_slide_pred_send_load,\n         tester_symm4_slide_succ_rcv_load_timeouts_succ,\n         tester_symm4_slide_succ_rcv_load_timeouts_node,\n         tester_symm4_slide_pred_send_load_timeouts_pred,\n         tester_symm4_slide_pred_send_load_timeouts_node\n        ],\n    SendToSuccTestCases =\n        [\n         symm4_slide_succ_send_load,\n         symm4_slide_pred_rcv_load,\n         tester_symm4_slide_succ_send_load_timeouts_succ,\n         tester_symm4_slide_succ_send_load_timeouts_node,\n         tester_symm4_slide_pred_rcv_load_timeouts_pred,\n         tester_symm4_slide_pred_rcv_load_timeouts_node\n        ],\n    SendToBothTestCases =\n        [\n         test_slide_adjacent,\n         test_slide_conflict\n        ],\n    SlideIllegallyTestCases =\n        [\n         test_slide_illegally\n        ],\n    JumpSlideTestCases =\n        [\n         tester_test_jump\n        ],\n    [\n      {send_to_pred, Config, SendToPredTestCases},\n      {send_to_pred_incremental, ConfigInc, SendToPredTestCases},\n      {send_to_succ, Config, SendToSuccTestCases},\n      {send_to_succ_incremental, ConfigInc, SendToSuccTestCases},\n      {send_to_both, Config, SendToBothTestCases},\n      {send_to_both_incremental, ConfigInc2, SendToBothTestCases},\n      {slide_illegally, Config, SlideIllegallyTestCases},\n      {jump_slide, Config, JumpSlideTestCases},\n      {jump_slide_incremental, ConfigInc, JumpSlideTestCases}\n    ]\n      ++\n%%         unittest_helper:create_ct_groups(test_cases(), [{tester_symm4_slide_pred_send_load_timeouts_pred_incremental, [sequence, {repeat_until_any_fail, forever}]}]).\n        [].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) ->\n    unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(_TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    GroupConfig = proplists:get_value(tc_group_properties, Config, []),\n    {move_config, MoveConf} = lists:keyfind(move_config, 1, GroupConfig),\n    unittest_helper:make_symmetric_ring([{config, [{log_path, PrivDir},\n                                                   {dht_node, mockup_dht_node},\n                                                   {monitor_perf_interval, 0},\n                                                   {join_lb_psv, lb_psv_simple},\n                                                   {lb_psv_samples, 1},\n                                                   {replication_factor, 4}]\n                                          ++ MoveConf ++ additional_ring_config()}]),\n    % wait for all nodes to finish their join before writing data\n    unittest_helper:check_ring_size_fully_joined(4),\n    %% write some data (use a function because left-over tx_timeout messages can disturb the tests):\n    Pid = erlang:spawn(fun() ->\n                               _ = [api_tx:write(erlang:integer_to_list(X), X) || X <- lists:seq(1, 100)]\n                       end),\n    util:wait_for_process_to_die(Pid),\n    % wait a bit for the rm-processes to settle\n    timer:sleep(500),\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n%% @doc Sets tighter timeouts for slides\n-spec move_config_parameters() -> unittest_helper:kv_opts().\nmove_config_parameters() ->\n    [{move_use_incremental_slides, false},\n     {move_wait_for_reply_timeout, 1000},\n     {move_send_msg_retries, 2},\n     {move_send_msg_retry_delay, 0}].\n\n%% @doc Sets tighter timeouts for slides\n-spec move_config_parameters_incremental() -> unittest_helper:kv_opts().\nmove_config_parameters_incremental() ->\n    % note: later parameters in the list override the first ones in config\n    move_config_parameters() ++\n        [{move_use_incremental_slides, true},\n         {move_max_transport_entries, 2},\n         {move_send_msg_retry_delay, 0}].\n\n%% @doc Config like above but more entries are moved at once\n-spec move_config_parameters_incremental2() -> unittest_helper:kv_opts().\nmove_config_parameters_incremental2() ->\n    move_config_parameters_incremental() ++\n        [{move_max_transport_entries, 10}].\n\n%% @doc Test slide with successor, receiving data from it (using api_tx in the bench server).\nsymm4_slide_succ_rcv_load(_Config) ->\n    stop_time(fun() ->\n                      BenchPid = erlang:spawn(fun() -> bench:increment(10, 10000) end),\n                      api_tx_SUITE:wait_for_dht_entries(440),\n                      symm4_slide_load_test(1, succ, \"slide_succ_rcv\", fun id_my_to_succ_1_100/6, ?NUM_SLIDES),\n                      symm4_slide_load_test(2, succ, \"slide_succ_rcv\", fun id_my_to_succ_1_100/6, ?NUM_SLIDES),\n                      symm4_slide_load_test(3, succ, \"slide_succ_rcv\", fun id_my_to_succ_1_100/6, ?NUM_SLIDES),\n                      symm4_slide_load_test(4, succ, \"slide_succ_rcv\", fun id_my_to_succ_1_100/6, ?NUM_SLIDES),\n                      erlang:exit(BenchPid, 'kill'),\n                      util:wait_for_process_to_die(BenchPid)\n              end, \"symm4_slide_succ_rcv_load\"),\n    unittest_helper:check_ring_load(440),\n    unittest_helper:check_ring_data().\n\n%% @doc Test slide with successor, sending data to it (using api_tx in the bench server).\nsymm4_slide_succ_send_load(_Config) ->\n    stop_time(fun() ->\n                      BenchPid = erlang:spawn(fun() -> bench:increment(10, 10000) end),\n                      api_tx_SUITE:wait_for_dht_entries(440),\n                      symm4_slide_load_test(1, succ, \"slide_succ_send\", fun id_pred_to_my_99_100/6, ?NUM_SLIDES),\n                      symm4_slide_load_test(2, succ, \"slide_succ_send\", fun id_pred_to_my_99_100/6, ?NUM_SLIDES),\n                      symm4_slide_load_test(3, succ, \"slide_succ_send\", fun id_pred_to_my_99_100/6, ?NUM_SLIDES),\n                      symm4_slide_load_test(4, succ, \"slide_succ_send\", fun id_pred_to_my_99_100/6, ?NUM_SLIDES),\n                      erlang:exit(BenchPid, 'kill'),\n                      util:wait_for_process_to_die(BenchPid)\n              end, \"symm4_slide_succ_send_load\"),\n    unittest_helper:check_ring_load(440),\n    unittest_helper:check_ring_data().\n\n%% @doc Test slide with predecessor, sending data to it (using api_tx in the bench server).\nsymm4_slide_pred_send_load(_Config) ->\n    stop_time(fun() ->\n                      BenchPid = erlang:spawn(fun() -> bench:increment(10, 10000) end),\n                      api_tx_SUITE:wait_for_dht_entries(440),\n                      symm4_slide_load_test(1, pred, \"slide_pred_send\", fun id_pred_to_my_1_100/6, ?NUM_SLIDES),\n                      symm4_slide_load_test(2, pred, \"slide_pred_send\", fun id_pred_to_my_1_100/6, ?NUM_SLIDES),\n                      symm4_slide_load_test(3, pred, \"slide_pred_send\", fun id_pred_to_my_1_100/6, ?NUM_SLIDES),\n                      symm4_slide_load_test(4, pred, \"slide_pred_send\", fun id_pred_to_my_1_100/6, ?NUM_SLIDES),\n                      erlang:exit(BenchPid, 'kill'),\n                      util:wait_for_process_to_die(BenchPid)\n              end, \"symm4_slide_pred_send_load\"),\n    unittest_helper:check_ring_load(440),\n    unittest_helper:check_ring_data().\n\n%% @doc Test slide with predecessor, receiving data from it (using api_tx in the bench server).\nsymm4_slide_pred_rcv_load(_Config) ->\n    stop_time(fun() ->\n                      BenchPid = erlang:spawn(fun() -> bench:increment(10, 10000) end),\n                      api_tx_SUITE:wait_for_dht_entries(440),\n                      symm4_slide_load_test(1, pred, \"slide_pred_rcv\", fun id_predspred_to_pred_99_100/6, ?NUM_SLIDES),\n                      symm4_slide_load_test(2, pred, \"slide_pred_rcv\", fun id_predspred_to_pred_99_100/6, ?NUM_SLIDES),\n                      symm4_slide_load_test(3, pred, \"slide_pred_rcv\", fun id_predspred_to_pred_99_100/6, ?NUM_SLIDES),\n                      symm4_slide_load_test(4, pred, \"slide_pred_rcv\", fun id_predspred_to_pred_99_100/6, ?NUM_SLIDES),\n                      erlang:exit(BenchPid, 'kill'),\n                      util:wait_for_process_to_die(BenchPid)\n              end, \"symm4_slide_pred_rcv_load\"),\n    unittest_helper:check_ring_load(440),\n    unittest_helper:check_ring_data().\n\n%%%%%%%%%%%%%%%%%%%%\n\n-spec reply_with_send_error(Msg::comm:message(), State::dht_node_state:state())\n        -> State::dht_node_state:state().\nreply_with_send_error(Msg, State) when element(1, Msg) =:= move ->\n    % just in case, if there are two slides, then send two send errors\n    SlidePred = dht_node_state:get(State, slide_pred),\n    SlideSucc = dht_node_state:get(State, slide_succ),\n    % note: only send a message once -> use lists:usort/1 to filter out duplicates\n    FailMsgs =\n        [begin\n             case Slide of\n                 null ->\n                     case element(2, Msg) of\n                         slide ->\n                             {node:pidX(element(5, Msg)), element(4, Msg)};\n                         _ ->\n                             {null, ok}\n                     end;\n                 _    ->\n                     Target = node:pidX(slide_op:get_node(Slide)),\n                     FailMsgCookie =\n                         case element(2, Msg) of\n                             slide_abort   -> {timeouts, 0};\n                             delta_ack     -> {timeouts, 0};\n                             slide         -> element(4, Msg);\n                             _             -> slide_op:get_id(Slide)\n                         end,\n                     {Target, FailMsgCookie}\n             end\n         end || Slide <- lists:usort([SlidePred, SlideSucc])],\n    _ = [comm:send(Target, {move, {send_error, comm:this(), Msg, unittest}, FailMsgCookie})\n        || X = {Target, FailMsgCookie} <- lists:usort(FailMsgs),\n           X =/= {null, ok}],\n    State;\nreply_with_send_error(_Msg, State) ->\n    State.\n\n% keep in sync with dht_node_move and the timeout config parameters set in set_move_config_parameters/0\n-type move_message() ::\n%{move, slide, OtherType::slide_op:type(), MoveFullId::slide_op:id(), InitNode::node:node_type(), TargetNode::node:node_type(), TargetId::?RT:key(), Tag::any(), NextOp::slide_op:next_op(), MaxTransportEntries::pos_integer()} |\n    {{move, slide, '_', '_', '_', '_', '_', '_', '_', '_'}, [], 1..2, reply_with_send_error} |\n%{move, slide_abort, pred | succ, MoveFullId::slide_op:id(), Reason::abort_reason()} |\n    {{move, slide_abort, '_', '_', '_'}, [], 1, reply_with_send_error} |\n% note: do not loose local messages:\n%{move, node_update, Tag::{move, slide_op:id()}} | % message from RM that it has changed the node's id to TargetId\n%{move, rm_new_pred, Tag::{move, slide_op:id()}} | % message from RM that it knows the pred we expect\n%{move, data, MovingData::db_dht:db_as_list(), MoveFullId::slide_op:id(), TargetId::?RT:key(), NextOp::slide_op:next_op()} |\n    {{move, data, '_', '_', '_', '_'}, [], 1..5, reply_with_send_error} |\n%{move, data_ack, MoveFullId::slide_op:id()} |\n    {{move, data_ack, '_'}, [], 1..5, reply_with_send_error} |\n%{move, delta, ChangedData::db_dht:db_as_list(), DeletedKeys::[?RT:key()], MoveFullId::slide_op:id()} |\n    {{move, delta, '_', '_', '_'}, [], 1..5, reply_with_send_error} |\n%%{move, delta_ack, MoveFullId::slide_op:id(), {none}} |\n    {{move, delta_ack, '_', {none}}, [], 1..2, reply_with_send_error} |\n%{move, delta_ack, MoveFullId::slide_op:id(), {continue, NewSlideId::slide_op:id()}} |\n    {{move, delta_ack, '_', {continue, '_'}}, [], 1..2, reply_with_send_error}.\n% note: do not lose local messages:\n%{move, rm_db_range, MoveFullId::slide_op:id()} |\n% note: there's no timeout for this message\n%{move, done, MoveFullId::slide_op:id()} |\n\n%% @doc Makes IgnoredMessages unique, i.e. only one msg per msg type.\n-spec fix_tester_ignored_msg_list(IgnoredMessages::[move_message(),...]) -> [move_message(),...].\nfix_tester_ignored_msg_list(IgnoredMessages) ->\n    IMsg2 = lists:usort(fun(E1, E2) ->\n                                erlang:element(2, erlang:element(1, E1)) =<\n                                    erlang:element(2, erlang:element(1, E2))\n                        end, IgnoredMessages),\n    [begin\n         NewAction =\n             case Action of\n                 reply_with_send_error -> fun reply_with_send_error/2;\n                 X -> X\n             end,\n         {Msg, Conds, Count, NewAction}\n     end || {Msg, Conds, Count, Action} <- IMsg2].\n\n-spec send_ignore_msg_list_to(NthNode::1..4, PredOrSuccOrNode::pred | succ | node, IgnoredMessages::[move_message(),...]) -> ok.\nsend_ignore_msg_list_to(NthNode, PredOrSuccOrNode, IgnoredMessages) ->\n    % cleanup, just in case:\n    _ = [comm:send_local(DhtNodePid, {mockup_dht_node, clear_match_specs})\n           || DhtNodePid <- pid_groups:find_all(dht_node)],\n\n    FailMsg = lists:flatten(io_lib:format(\"~.0p (~B.*) ignoring messages: ~.0p\",\n                                          [PredOrSuccOrNode, NthNode, IgnoredMessages])),\n    {_PredsPred, Pred, Node, Succ} = get_pred_node_succ(NthNode, FailMsg),\n    Other = case PredOrSuccOrNode of\n                succ -> Succ;\n                pred -> Pred;\n                node -> Node\n            end,\n    comm:send(node:pidX(Other), {mockup_dht_node, add_match_specs, IgnoredMessages}).\n\n%% @doc Test slide with successor, receiving data from it (using api_tx in the bench server), ignore (some) messages on succ.\n-spec prop_symm4_slide_succ_rcv_load_timeouts_succ(IgnoredMessages::[move_message(),...]) -> true.\nprop_symm4_slide_succ_rcv_load_timeouts_succ(IgnoredMessages_) ->\n    IgnoredMessages = fix_tester_ignored_msg_list(IgnoredMessages_),\n\n    send_ignore_msg_list_to(1, succ, IgnoredMessages),\n    symm4_slide_load_test(1, succ, \"slide_succ_rcv_timeouts_succ\", fun id_my_to_succ_1_100/6, 1),\n    send_ignore_msg_list_to(2, succ, IgnoredMessages),\n    symm4_slide_load_test(2, succ, \"slide_succ_rcv_timeouts_succ\", fun id_my_to_succ_1_100/6, 1),\n    send_ignore_msg_list_to(3, succ, IgnoredMessages),\n    symm4_slide_load_test(3, succ, \"slide_succ_rcv_timeouts_succ\", fun id_my_to_succ_1_100/6, 1),\n    send_ignore_msg_list_to(4, succ, IgnoredMessages),\n    symm4_slide_load_test(4, succ, \"slide_succ_rcv_timeouts_succ\", fun id_my_to_succ_1_100/6, 1),\n\n    % cleanup, just in case:\n    _ = [comm:send_local(DhtNodePid, {mockup_dht_node, clear_match_specs})\n           || DhtNodePid <- pid_groups:find_all(dht_node)],\n    true.\n\n%% @doc Test slide with successor, receiving data from it (using api_tx in the bench server), ignore (some) messages on node.\n-spec prop_symm4_slide_succ_rcv_load_timeouts_node(IgnoredMessages::[move_message(),...]) -> true.\nprop_symm4_slide_succ_rcv_load_timeouts_node(IgnoredMessages_) ->\n    IgnoredMessages = fix_tester_ignored_msg_list(IgnoredMessages_),\n\n    send_ignore_msg_list_to(1, node, IgnoredMessages),\n    symm4_slide_load_test(1, succ, \"slide_succ_rcv_timeouts_node\", fun id_my_to_succ_1_100/6, 1),\n    send_ignore_msg_list_to(2, node, IgnoredMessages),\n    symm4_slide_load_test(2, succ, \"slide_succ_rcv_timeouts_node\", fun id_my_to_succ_1_100/6, 1),\n    send_ignore_msg_list_to(3, node, IgnoredMessages),\n    symm4_slide_load_test(3, succ, \"slide_succ_rcv_timeouts_node\", fun id_my_to_succ_1_100/6, 1),\n    send_ignore_msg_list_to(4, node, IgnoredMessages),\n    symm4_slide_load_test(4, succ, \"slide_succ_rcv_timeouts_node\", fun id_my_to_succ_1_100/6, 1),\n\n    % cleanup, just in case:\n    _ = [comm:send_local(DhtNodePid, {mockup_dht_node, clear_match_specs})\n           || DhtNodePid <- pid_groups:find_all(dht_node)],\n    true.\n\ntester_symm4_slide_succ_rcv_load_timeouts_succ(_Config) ->\n    BenchPid = erlang:spawn(fun() -> bench:increment(10, 10000) end),\n    api_tx_SUITE:wait_for_dht_entries(440),\n    tester:test(?MODULE, prop_symm4_slide_succ_rcv_load_timeouts_succ, 1, 25),\n    unittest_helper:check_ring_load(440),\n    unittest_helper:check_ring_data(),\n    erlang:exit(BenchPid, 'kill'),\n    util:wait_for_process_to_die(BenchPid).\n\ntester_symm4_slide_succ_rcv_load_timeouts_node(_Config) ->\n    BenchPid = erlang:spawn(fun() -> bench:increment(10, 10000) end),\n    api_tx_SUITE:wait_for_dht_entries(440),\n    tester:test(?MODULE, prop_symm4_slide_succ_rcv_load_timeouts_node, 1, 25),\n    unittest_helper:check_ring_load(440),\n    unittest_helper:check_ring_data(),\n    erlang:exit(BenchPid, 'kill'),\n    util:wait_for_process_to_die(BenchPid).\n\n%% @doc Test slide with successor, sending data to it (using api_tx in the bench server), ignore (some) messages on succ.\n-spec prop_symm4_slide_succ_send_load_timeouts_succ(IgnoredMessages::[move_message(),...]) -> true.\nprop_symm4_slide_succ_send_load_timeouts_succ(IgnoredMessages_) ->\n    IgnoredMessages = fix_tester_ignored_msg_list(IgnoredMessages_),\n\n    send_ignore_msg_list_to(1, succ, IgnoredMessages),\n    symm4_slide_load_test(1, succ, \"slide_succ_send_timeouts_succ\", fun id_pred_to_my_99_100/6, 1),\n    send_ignore_msg_list_to(2, succ, IgnoredMessages),\n    symm4_slide_load_test(2, succ, \"slide_succ_send_timeouts_succ\", fun id_pred_to_my_99_100/6, 1),\n    send_ignore_msg_list_to(3, succ, IgnoredMessages),\n    symm4_slide_load_test(3, succ, \"slide_succ_send_timeouts_succ\", fun id_pred_to_my_99_100/6, 1),\n    send_ignore_msg_list_to(4, succ, IgnoredMessages),\n    symm4_slide_load_test(4, succ, \"slide_succ_send_timeouts_succ\", fun id_pred_to_my_99_100/6, 1),\n\n    % cleanup, just in case:\n    _ = [comm:send_local(DhtNodePid, {mockup_dht_node, clear_match_specs})\n           || DhtNodePid <- pid_groups:find_all(dht_node)],\n    true.\n\n%% @doc Test slide with successor, sending data to it (using api_tx in the bench server), ignore (some) messages on node.\n-spec prop_symm4_slide_succ_send_load_timeouts_node(IgnoredMessages::[move_message(),...]) -> true.\nprop_symm4_slide_succ_send_load_timeouts_node(IgnoredMessages_) ->\n    IgnoredMessages = fix_tester_ignored_msg_list(IgnoredMessages_),\n\n    send_ignore_msg_list_to(1, node, IgnoredMessages),\n    symm4_slide_load_test(1, succ, \"slide_succ_send_timeouts_node\", fun id_pred_to_my_99_100/6, 1),\n    send_ignore_msg_list_to(2, node, IgnoredMessages),\n    symm4_slide_load_test(2, succ, \"slide_succ_send_timeouts_node\", fun id_pred_to_my_99_100/6, 1),\n    send_ignore_msg_list_to(3, node, IgnoredMessages),\n    symm4_slide_load_test(3, succ, \"slide_succ_send_timeouts_node\", fun id_pred_to_my_99_100/6, 1),\n    send_ignore_msg_list_to(4, node, IgnoredMessages),\n    symm4_slide_load_test(4, succ, \"slide_succ_send_timeouts_node\", fun id_pred_to_my_99_100/6, 1),\n\n    % cleanup, just in case:\n    _ = [comm:send_local(DhtNodePid, {mockup_dht_node, clear_match_specs})\n           || DhtNodePid <- pid_groups:find_all(dht_node)],\n    true.\n\ntester_symm4_slide_succ_send_load_timeouts_succ(_Config) ->\n    BenchPid = erlang:spawn(fun() -> bench:increment(10, 10000) end),\n    api_tx_SUITE:wait_for_dht_entries(440),\n    tester:test(?MODULE, prop_symm4_slide_succ_send_load_timeouts_succ, 1, 25),\n    unittest_helper:check_ring_load(440),\n    unittest_helper:check_ring_data(),\n    erlang:exit(BenchPid, 'kill'),\n    util:wait_for_process_to_die(BenchPid).\n\ntester_symm4_slide_succ_send_load_timeouts_node(_Config) ->\n    BenchPid = erlang:spawn(fun() -> bench:increment(10, 10000) end),\n    api_tx_SUITE:wait_for_dht_entries(440),\n    tester:test(?MODULE, prop_symm4_slide_succ_send_load_timeouts_node, 1, 25),\n    unittest_helper:check_ring_load(440),\n    unittest_helper:check_ring_data(),\n    erlang:exit(BenchPid, 'kill'),\n    util:wait_for_process_to_die(BenchPid).\n\n%% @doc Test slide with predecessor, sending data to it (using api_tx in the bench server), ignore (some) messages on pred.\n-spec prop_symm4_slide_pred_send_load_timeouts_pred(IgnoredMessages::[move_message(),...]) -> true.\nprop_symm4_slide_pred_send_load_timeouts_pred(IgnoredMessages_) ->\n    IgnoredMessages = fix_tester_ignored_msg_list(IgnoredMessages_),\n\n    send_ignore_msg_list_to(1, pred, IgnoredMessages),\n    symm4_slide_load_test(1, pred, \"slide_pred_send_timeouts_pred\", fun id_pred_to_my_1_100/6, 1),\n    send_ignore_msg_list_to(2, pred, IgnoredMessages),\n    symm4_slide_load_test(2, pred, \"slide_pred_send_timeouts_pred\", fun id_pred_to_my_1_100/6, 1),\n    send_ignore_msg_list_to(3, pred, IgnoredMessages),\n    symm4_slide_load_test(3, pred, \"slide_pred_send_timeouts_pred\", fun id_pred_to_my_1_100/6, 1),\n    send_ignore_msg_list_to(4, pred, IgnoredMessages),\n    symm4_slide_load_test(4, pred, \"slide_pred_send_timeouts_pred\", fun id_pred_to_my_1_100/6, 1),\n\n    % cleanup, just in case:\n    _ = [comm:send_local(DhtNodePid, {mockup_dht_node, clear_match_specs})\n           || DhtNodePid <- pid_groups:find_all(dht_node)],\n    true.\n\n%% @doc Test slide with predecessor, sending data to it (using api_tx in the bench server), ignore (some) messages on node.\n-spec prop_symm4_slide_pred_send_load_timeouts_node(IgnoredMessages::[move_message(),...]) -> true.\nprop_symm4_slide_pred_send_load_timeouts_node(IgnoredMessages_) ->\n    IgnoredMessages = fix_tester_ignored_msg_list(IgnoredMessages_),\n\n    send_ignore_msg_list_to(1, node, IgnoredMessages),\n    symm4_slide_load_test(1, pred, \"slide_pred_send_timeouts_node\", fun id_pred_to_my_1_100/6, 1),\n    send_ignore_msg_list_to(2, node, IgnoredMessages),\n    symm4_slide_load_test(2, pred, \"slide_pred_send_timeouts_node\", fun id_pred_to_my_1_100/6, 1),\n    send_ignore_msg_list_to(3, node, IgnoredMessages),\n    symm4_slide_load_test(3, pred, \"slide_pred_send_timeouts_node\", fun id_pred_to_my_1_100/6, 1),\n    send_ignore_msg_list_to(4, node, IgnoredMessages),\n    symm4_slide_load_test(4, pred, \"slide_pred_send_timeouts_node\", fun id_pred_to_my_1_100/6, 1),\n\n    % cleanup, just in case:\n    _ = [comm:send_local(DhtNodePid, {mockup_dht_node, clear_match_specs})\n           || DhtNodePid <- pid_groups:find_all(dht_node)],\n    true.\n\ntester_symm4_slide_pred_send_load_timeouts_pred(_Config) ->\n    BenchPid = erlang:spawn(fun() -> bench:increment(10, 10000) end),\n    tester:test(?MODULE, prop_symm4_slide_pred_send_load_timeouts_pred, 1, 25),\n    unittest_helper:check_ring_load(440),\n    unittest_helper:check_ring_data(),\n    erlang:exit(BenchPid, 'kill'),\n    util:wait_for_process_to_die(BenchPid).\n\ntester_symm4_slide_pred_send_load_timeouts_node(_Config) ->\n    BenchPid = erlang:spawn(fun() -> bench:increment(10, 10000) end),\n    api_tx_SUITE:wait_for_dht_entries(440),\n    tester:test(?MODULE, prop_symm4_slide_pred_send_load_timeouts_node, 1, 25),\n    unittest_helper:check_ring_load(440),\n    unittest_helper:check_ring_data(),\n    erlang:exit(BenchPid, 'kill'),\n    util:wait_for_process_to_die(BenchPid).\n\n%% @doc Test slide with successor, receiving data from it (using api_tx in the bench server), ignore (some) messages on pred.\n-spec prop_symm4_slide_pred_rcv_load_timeouts_pred(IgnoredMessages::[move_message(),...]) -> true.\nprop_symm4_slide_pred_rcv_load_timeouts_pred(IgnoredMessages_) ->\n    IgnoredMessages = fix_tester_ignored_msg_list(IgnoredMessages_),\n\n    send_ignore_msg_list_to(1, pred, IgnoredMessages),\n    symm4_slide_load_test(1, pred, \"slide_pred_rcv_timeouts_pred\", fun id_predspred_to_pred_99_100/6, 1),\n    send_ignore_msg_list_to(2, pred, IgnoredMessages),\n    symm4_slide_load_test(2, pred, \"slide_pred_rcv_timeouts_pred\", fun id_predspred_to_pred_99_100/6, 1),\n    send_ignore_msg_list_to(3, pred, IgnoredMessages),\n    symm4_slide_load_test(3, pred, \"slide_pred_rcv_timeouts_pred\", fun id_predspred_to_pred_99_100/6, 1),\n    send_ignore_msg_list_to(4, pred, IgnoredMessages),\n    symm4_slide_load_test(4, pred, \"slide_pred_rcv_timeouts_pred\", fun id_predspred_to_pred_99_100/6, 1),\n\n    % cleanup, just in case:\n    _ = [comm:send_local(DhtNodePid, {mockup_dht_node, clear_match_specs})\n           || DhtNodePid <- pid_groups:find_all(dht_node)],\n    true.\n\n%% @doc Test slide with predecessor, receiving data from it (using api_tx in the bench server), ignore (some) messages on node.\n-spec prop_symm4_slide_pred_rcv_load_timeouts_node(IgnoredMessages::[move_message(),...]) -> true.\nprop_symm4_slide_pred_rcv_load_timeouts_node(IgnoredMessages_) ->\n    IgnoredMessages = fix_tester_ignored_msg_list(IgnoredMessages_),\n\n    send_ignore_msg_list_to(1, node, IgnoredMessages),\n    symm4_slide_load_test(1, pred, \"slide_pred_rcv_timeouts_node\", fun id_predspred_to_pred_99_100/6, 1),\n    send_ignore_msg_list_to(2, node, IgnoredMessages),\n    symm4_slide_load_test(2, pred, \"slide_pred_rcv_timeouts_node\", fun id_predspred_to_pred_99_100/6, 1),\n    send_ignore_msg_list_to(3, node, IgnoredMessages),\n    symm4_slide_load_test(3, pred, \"slide_pred_rcv_timeouts_node\", fun id_predspred_to_pred_99_100/6, 1),\n    send_ignore_msg_list_to(4, node, IgnoredMessages),\n    symm4_slide_load_test(4, pred, \"slide_pred_rcv_timeouts_node\", fun id_predspred_to_pred_99_100/6, 1),\n\n    % cleanup, just in case:\n    _ = [comm:send_local(DhtNodePid, {mockup_dht_node, clear_match_specs})\n           || DhtNodePid <- pid_groups:find_all(dht_node)],\n    true.\n\ntester_symm4_slide_pred_rcv_load_timeouts_pred(_Config) ->\n    BenchPid = erlang:spawn(fun() -> bench:increment(10, 10000) end),\n    api_tx_SUITE:wait_for_dht_entries(440),\n    tester:test(?MODULE, prop_symm4_slide_pred_rcv_load_timeouts_pred, 1, 25),\n    unittest_helper:check_ring_load(440),\n    unittest_helper:check_ring_data(),\n    erlang:exit(BenchPid, 'kill'),\n    util:wait_for_process_to_die(BenchPid).\n\ntester_symm4_slide_pred_rcv_load_timeouts_node(_Config) ->\n    BenchPid = erlang:spawn(fun() -> bench:increment(10, 10000) end),\n    api_tx_SUITE:wait_for_dht_entries(440),\n    tester:test(?MODULE, prop_symm4_slide_pred_rcv_load_timeouts_node, 1, 25),\n    unittest_helper:check_ring_load(440),\n    unittest_helper:check_ring_data(),\n    erlang:exit(BenchPid, 'kill'),\n    util:wait_for_process_to_die(BenchPid).\n\n%%%%%%%%%%%%%%%%%%%%\n\n-spec id_my_to_succ_1_100(PredsPredId::?RT:key(), PredId::?RT:key(), MyId::?RT:key(),\n                          SuccId::?RT:key(), N::pos_integer(), Count::pos_integer())\n        -> NewKey::?RT:key().\nid_my_to_succ_1_100(_PredsPredId, _PredId, MyId, SuccId, _N, _Count) ->\n    ?RT:get_split_key(MyId, SuccId, {1, 100}).\n\n-spec id_pred_to_my_99_100(PredsPredId::?RT:key(), PredId::?RT:key(), MyId::?RT:key(),\n                           SuccId::?RT:key(), N::pos_integer(), Count::pos_integer())\n        -> NewKey::?RT:key().\nid_pred_to_my_99_100(_PredsPredId, PredId, MyId, _SuccId, _N, _Count) ->\n    ?RT:get_split_key(PredId, MyId, {99, 100}).\n\n-spec id_pred_to_my_1_100(PredsPredId::?RT:key(), PredId::?RT:key(), MyId::?RT:key(),\n                          SuccId::?RT:key(), N::pos_integer(), Count::pos_integer())\n        -> NewKey::?RT:key().\nid_pred_to_my_1_100(_PredsPredId, PredId, MyId, _SuccId, _N, _Count) ->\n    ?RT:get_split_key(PredId, MyId, {1, 100}).\n\n-spec id_predspred_to_pred_99_100(PredsPredId::?RT:key(), PredId::?RT:key(), MyId::?RT:key(),\n                                  SuccId::?RT:key(), N::pos_integer(), Count::pos_integer())\n        -> NewKey::?RT:key().\nid_predspred_to_pred_99_100(PredsPredId, PredId, _MyId, _SuccId, _N, _Count) ->\n    ?RT:get_split_key(PredsPredId, PredId, {99, 100}).\n\n-spec symm4_slide_load_test(\n        NthNode::1..4, PredOrSucc::pred | succ, Tag::string(),\n        TargetIdFun::fun((PredsPredId::?RT:key(), PredId::?RT:key(), MyId::?RT:key(),\n                          SuccId::?RT:key(), N::pos_integer(), Count::pos_integer()) -> TargetId::?RT:key()),\n        Count::pos_integer()) -> ok.\nsymm4_slide_load_test(NthNode, PredOrSucc, Tag, TargetIdFun, Count) ->\n    % get a random DHT node and let it slide with its successor/predecessor (Count times)\n    DhtNode = lists:nth(NthNode, pid_groups:find_all(dht_node)),\n    _ = [begin\n             FailMsg = lists:flatten(\n                         io_lib:format(\"slide_~.0p(~B.~B, ~.0p)\",\n                                       [PredOrSucc, NthNode, N, DhtNode])),\n             {PredsPred, Pred, Node, Succ} = get_pred_node_succ(NthNode, FailMsg),\n             TargetId = TargetIdFun(node:id(PredsPred), node:id(Pred),\n                                    node:id(Node), node:id(Succ), N, Count),\n             NewTag = lists:flatten(io_lib:format(\"~s-~B.~B\", [Tag, NthNode, N])),\n             Other = case PredOrSucc of\n                         succ -> Succ;\n                         pred -> Pred\n                     end,\n             ?proto_sched(start),\n             symm4_slide_load_test_slide(DhtNode, PredOrSucc, TargetId, NewTag, NthNode, N, Node, Other),\n             ?proto_sched(stop),\n             % note: do not yield trace_mpath thread with \"after 0\"!\n             receive\n                 ?SCALARIS_RECV(Z,\n                                ?ct_fail(\"slide_~.0p(~B.~B, ~.0p) unexpected message: ~.0p\",\n                                         [PredOrSucc, NthNode, N, DhtNode, Z]))\n             after 0 -> ok\n             end\n         end || N <- lists:seq(1, Count)],\n    ok.\n\n\n-spec get_pred_node_succ(NthNode::1..4, FailMsg::string())\n        -> {PredsPred::node:node_type(), Pred::node:node_type(),\n            Node::node:node_type(), Succ::node:node_type()}.\nget_pred_node_succ(NthNode, FailMsg) ->\n    DhtNodePid = lists:nth(NthNode, pid_groups:find_all(dht_node)),\n    {Pred1, Node1, Succ} = get_pred_node_succ2(DhtNodePid, FailMsg),\n    {PredsPred, Pred2, Node2} = get_pred_node_succ2(node:pidX(Pred1), FailMsg),\n    {PredsPred, node:newer(Pred1, Pred2), node:newer(Node1, Node2), Succ}.\n\n-spec get_pred_node_succ2(NodePid::pid() | comm:mypid(), FailMsg::string())\n        -> {Pred::node:node_type(), Node::node:node_type(), Succ::node:node_type()}.\nget_pred_node_succ2(NodePid, FailMsg) when is_pid(NodePid) ->\n    get_pred_node_succ2(comm:make_global(NodePid), FailMsg);\nget_pred_node_succ2(NodePid, FailMsg) ->\n    comm:send(NodePid, {get_node_details, comm:this(), [node, pred, succ]}),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({get_node_details_response, NodeDetails},\n                       begin\n                           Node = node_details:get(NodeDetails, node),\n                           Pred = node_details:get(NodeDetails, pred),\n                           Succ = node_details:get(NodeDetails, succ),\n                           {Pred, Node, Succ}\n                       end);\n        ?SCALARIS_RECV(Y, ?ct_fail(\"~s: unexpected message while \"\n                                   \"waiting for get_node_details_response: ~.0p\",\n                                   [FailMsg, Y]))\n    end.\n\n-spec symm4_slide_load_test_slide(DhtNode::pid(), PredOrSucc::pred | succ,\n        TargetId::?RT:key(), Tag::any(), NthNode::1..4, N::pos_integer(),\n        Node::node:node_type(), Other::node:node_type()) -> ok.\nsymm4_slide_load_test_slide(DhtNode, PredOrSucc, TargetId, Tag, NthNode, N, Node, Other) ->\n    comm:send_local(DhtNode, {move, start_slide, PredOrSucc, TargetId, Tag, comm:this()}),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(\n        {move, result, Tag, ok}, begin\n                                     %% ct:pal(\"~p.~p ~.0p -> ~.0p~n\", [NthNode, N, node:id(Node), TargetId]),\n                                     ok\n                                 end);\n        ?SCALARIS_RECV({move, result, Tag, Result},\n                       begin\n                           case lists:member(Result, [ongoing_slide, wrong_pred_succ_node]) of\n                               true ->\n                                   ct:pal(\"slide_~.0p(~B.~B, ~.0p, ~.0p, ~.0p) result: ~.0p~nretrying...~n\",\n                                          [PredOrSucc, NthNode, N, Node, Other, TargetId, Result]),\n                                   timer:sleep(25), % wait a bit before trying again\n                                   symm4_slide_load_test_slide(DhtNode, PredOrSucc, TargetId, Tag, NthNode, N, Node, Other);\n                               _ ->\n                                   ?ct_fail(\"slide_~.0p(~B.~B, ~.0p, ~.0p, ~.0p) result: ~.0p\",\n                                            [PredOrSucc, NthNode, N, Node, Other, TargetId, Result])\n                           end\n                       end);\n        ?SCALARIS_RECV(X,\n                       ?ct_fail(\"slide_~.0p(~B.~B, ~.0p, ~.0p, ~.0p) unexpected message: ~.0p\",\n                                [PredOrSucc, NthNode, N, Node, Other, TargetId, X]))\n        end.\n\n-spec stop_time(F::fun(() -> any()), Tag::string()) -> ok.\nstop_time(F, Tag) ->\n    Start = os:timestamp(),\n    F(),\n    Stop = os:timestamp(),\n    ElapsedTime = erlang:max(1, timer:now_diff(Stop, Start)) / 1000000.0,\n    Frequency = 1 / ElapsedTime,\n    ct:pal(\"~p took ~ps: ~p1/s~n\",\n           [Tag, ElapsedTime, Frequency]),\n    ok.\n\n-spec check_size(Size::pos_integer()) -> ok.\ncheck_size(Size) ->\n    unittest_helper:check_ring_size(Size),\n    unittest_helper:wait_for_stable_ring(),\n    unittest_helper:check_ring_size(Size).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Helper functions simultaneous slides %%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec gen_split_key_fun(node_type(), {integer(), integer()}) -> targetid_fun().\ngen_split_key_fun(Selector, Fraction) ->\n    fun(Nodes) ->\n            From = select_from_nodes(Selector, Nodes),\n            To = select_from_nodes(succ(Selector), Nodes),\n            ?RT:get_split_key(node:id(From), node:id(To), Fraction)\n    end.\n\n-spec pred(node_type()) -> node_type().\npred(pred) -> predspred;\npred(node) -> pred;\npred(succ) -> node.\n\n-spec succ(node_type()) -> node_type().\nsucc(predspred) -> pred;\nsucc(pred)      -> node;\nsucc(node)      -> succ.\n\n-spec generate_slide_variation(slide_config_record()) -> [slide_config_record(),...].\ngenerate_slide_variation(SlideConf) ->\n    SlideWith = ?IIF(SlideConf#slideconf.slide_with =:= succ, pred, succ),\n    Node =\n        case SlideWith of\n            pred -> succ(SlideConf#slideconf.node);\n            succ -> pred(SlideConf#slideconf.node)\n        end,\n    Name = list_to_atom(string:concat(atom_to_list(SlideConf#slideconf.name), \"var\")),\n    Variation = #slideconf{name = Name,\n                           node = Node,\n                           slide_with = SlideWith,\n                           targetid = SlideConf#slideconf.targetid},\n    [SlideConf, Variation].\n\n-spec select_from_nodes(Selector::node_type(), Nodes::node_tuple()) -> node:node_type().\nselect_from_nodes(Selector, Nodes) ->\n    N = selector_to_idx(Selector),\n    element(N, Nodes).\n\n-spec select_from_nodes(Node::node_type(), Direction::pred | succ, Nodes::node_tuple()) -> node:node_type().\nselect_from_nodes(Node, Direction, Nodes) ->\n    NIdx = selector_to_idx(Node),\n    Idx = case Direction of\n              pred -> NIdx - 1;\n              succ -> NIdx + 1\n          end,\n    element(Idx, Nodes).\n\n-spec selector_to_idx(node_type()) -> 1..4.\nselector_to_idx(predspred) -> 1;\nselector_to_idx(pred) -> 2;\nselector_to_idx(node) -> 3;\nselector_to_idx(succ) -> 4.\n\n-spec get_node_details(DhtNode::pid() | comm:mypid()) -> {node:node_type(), node:node_type(), node:node_type()}.\nget_node_details(DhtNode) ->\n    erlang:list_to_tuple(get_node_details(DhtNode, [pred, node, succ])).\n\n-spec get_node_details(DhtNode::pid() | comm:mypid(), Which::[node | pred | succ,...])\n        -> [node:node_type(),...].\nget_node_details(DhtNode, Which) ->\n    comm:send(comm:make_global(DhtNode), {get_node_details, comm:this(), Which}),\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({get_node_details_response, NodeDetails},\n                       [node_details:get(NodeDetails, X) || X <- Which])\n    end.\n\n-spec get_predspred_pred_node_succ(DhtNode::pid()) -> node_tuple().\nget_predspred_pred_node_succ(DhtNode) ->\n    {Pred, Node, Succ} = get_node_details(DhtNode),\n    {PredsPred, Pred2, Node2} = get_node_details(node:pidX(Pred)),\n    % the nodes' RM info must be correct after each slide and thus before the\n    % next one (which is when this function is called)\n    ?equals_w_note(Pred, Pred2, wrong_pred_info_in_node),\n    ?equals_w_note(Node2, Node, wrong_succ_info_in_pred),\n\n    % make sure, all pred/succ info is correct:\n    case admin:check_ring() of\n        ok -> ok;\n        _ ->\n            %% only temporarily wrong?\n            timer:sleep(20),\n            ?equals(admin:check_ring(), ok)\n    end,\n    {PredsPred, Pred, Node, Succ}.\n\n-spec set_breakpoint(Pid::pid(), gen_component:bp_name()) -> ok.\nset_breakpoint(Pid, Tag) ->\n    gen_component:bp_set(Pid, move, Tag),\n    gen_component:bp_barrier(Pid).\n\nwait(Pid, Tag) ->\n    gen_component:bp_del(Pid, Tag).\n\ncontinue(Pid) ->\n    gen_component:bp_cont(Pid).\n\n-spec proto_sched_callback_on_send() -> proto_sched:callback_on_msg().\nproto_sched_callback_on_send() ->\n    This = self(),\n    fun(_From, _To, {move, result, Tag, _Result}) ->\n            comm:send_local(This, {end_of_slide,   Tag});\n       (_From, _To, _Msg) ->\n            ok\n    end.\n\n-spec proto_sched_callback_on_deliver() -> proto_sched:callback_on_msg().\nproto_sched_callback_on_deliver() ->\n    This = self(),\n    fun(_From, _To, {move, start_slide, _, _, Tag, _}) ->\n            comm:send_local(This, {begin_of_slide, Tag});\n       (_From, _To, _Msg) ->\n            ok\n    end.\n\nreceive_result() ->\n    % do not use SCALARIS_RECV since we receive messages from proto_sched\n    % callbacks and we must not interfere with proto_sched here!\n    receive\n        X -> X\n    end.\n\n%% check for interleaving of slides using\n%% FIFO properties of erlang message channels\n-spec slide_interleaving() -> boolean().\nslide_interleaving() ->\n    case proto_sched:infected() of\n        false -> true;\n        true  ->\n            %% first message\n            {begin_of_slide, Tag} = receive_result(),\n            %% next message\n            Result =\n                case receive_result() of\n                    {end_of_slide, Tag}     -> false;\n                    {begin_of_slide, _Tag2} -> true\n                end,\n            %% clear pending messages\n            _ = receive_result(),\n            _ = receive_result(),\n            Result\n    end.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Execute simultaneous slides %%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Execute simultaneous slides between two adjacent nodes,\n%%      i.e. send data to the pred and receive from the succ\n-spec slide_simultaneously(DhtNode::pid(), {SlideConf1::slide_config_record(), SlideConf2::slide_config_record()}, VerifyFun::fun()) -> ok.\nslide_simultaneously(DhtNode, {SlideConf1, SlideConf2} = _Action, VerifyFun) ->\n    SlideVariations1 = generate_slide_variation(SlideConf1),\n    SlideVariations2 = generate_slide_variation(SlideConf2),\n    _ = [ begin\n              Nodes = get_predspred_pred_node_succ(DhtNode),\n              Node1 = select_from_nodes(Slide1#slideconf.node, Nodes),\n              Node2 = select_from_nodes(Slide2#slideconf.node, Nodes),\n              Direction1 = Slide1#slideconf.slide_with,\n              Direction2 = Slide2#slideconf.slide_with,\n              TargetId1 = (Slide1#slideconf.targetid)(Nodes),\n              TargetId2 = (Slide2#slideconf.targetid)(Nodes),\n              Tag1 = Slide1#slideconf.name,\n              Tag2 = Slide2#slideconf.name,\n              PidLocal1 = comm:make_local(node:pidX(Node1)),\n              PidLocal2 = comm:make_local(node:pidX(Node2)),\n              ?proto_sched(start),\n              ct:pal(\"Beginning~n\"\n                    \" ~p~n  ~s (~p) with~n  ~s (~p)~n  to ~p,~n\"\n                    \" ~p~n  ~s (~p) with~n  ~s (~p)~n  to ~p\",\n                     [Tag1, Slide1#slideconf.node, Node1, Direction1,\n                      select_from_nodes(Slide1#slideconf.node, Direction1, Nodes), TargetId1,\n                      Tag2, Slide2#slideconf.node, Node2, Direction2,\n                      select_from_nodes(Slide2#slideconf.node, Direction2, Nodes), TargetId2]),\n              %% We use breakpoints to assure simultaneous slides.\n              %% As these don't work with the proto scheduler, we test the timing of the slides using\n              %% a callback function instead. The function sends out messages begin_of_slide and\n              %% end_of_slide messages for each slide.\n              case proto_sched:infected() of\n                  false ->\n                      set_breakpoint(PidLocal1, slide1),\n                      ?IIF(PidLocal1 =/= PidLocal2, set_breakpoint(PidLocal2, slide2), ok);\n                  true  ->\n                      proto_sched:register_callback(\n                        proto_sched_callback_on_send(), on_send),\n                      proto_sched:register_callback(\n                        proto_sched_callback_on_deliver(), on_deliver)\n              end,\n              %% send out slides\n              comm:send(node:pidX(Node1), {move, start_slide, Direction1, TargetId1, {slide1, Direction1, Tag1}, comm:this()}),\n              comm:send(node:pidX(Node2), {move, start_slide, Direction2, TargetId2, {slide2, Direction2, Tag2}, comm:this()}),\n              %% Continue once both slides have reached the gen_component\n              case proto_sched:infected() of\n                  false ->\n                      wait(PidLocal1, slide1),\n                      ?IIF(PidLocal1 =/= PidLocal2, wait(PidLocal2, slide2), ok),\n                      continue(PidLocal1),\n                      ?IIF(PidLocal1 =/= PidLocal2, continue(PidLocal2), ok);\n                  true  ->\n                      ok\n              end,\n              %% receive results\n              % we need to process any slide result message and cannot filter\n              % for a tag, otherwise we might hang if running with proto_sched\n              % -> Result1 may be the result of Tag2, similar for Result2\n              ReceiveResultFun =\n                  fun() ->\n                          trace_mpath:thread_yield(),\n                          receive\n                              ?SCALARIS_RECV({move, result, _Tag, Result},% ->\n                                             Result)\n                          end\n                  end,\n              Result1 = ReceiveResultFun(),\n              Result2 = ReceiveResultFun(),\n              % NOTE: must check interleaving inside proto_sched for\n              %       slide_interleaving() to be able to distinguish between\n              %       proto_sched and non-proto_sched runs\n              Interleaving = slide_interleaving(),\n              ct:pal(\"Result1: ~p,~nResult2: ~p~nInterleaving: ~p\",\n                     [Result1, Result2, Interleaving]),\n              ?proto_sched(stop),\n              _ = get_predspred_pred_node_succ(DhtNode),\n              ct:pal(\"checked pred/succ info\"),\n              VerifyFun(Result1, Result2, Interleaving),\n              timer:sleep(10)\n          end || Slide1 <- SlideVariations1, Slide2 <- SlideVariations2],\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Test cases for simultaneous slides %%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Test slides for two adjacent nodes\ntest_slide_adjacent(_Config) ->\n    BenchPid = erlang:spawn(fun() -> bench:increment(10, 100000) end),\n    % make sure all keys have been created...\n    api_tx_SUITE:wait_for_dht_entries(440),\n    VerifyFun =\n        fun(Result1, Result2, _Interleaved) ->\n                unittest_helper:check_ring_load(440),\n                unittest_helper:check_ring_data(),\n                %% both slides should be successfull\n                ?assert_w_note(\n                    Result1 =:= ok andalso Result2 =:= ok\n                        orelse\n                        %% rarely, one slide fails because the neighbour\n                        %% information hasn't been updated yet\n                          ((Result1 =:= wrong_pred_succ_node) xor\n                               (Result2 =:= wrong_pred_succ_node)),\n                    [{result1, Result1}, {result2, Result2}])\n        end,\n    Actions =\n        [{\n           #slideconf{name = action1slide1,\n                      node = pred,\n                      slide_with = succ,\n                      targetid = gen_split_key_fun(pred, {1, 2})},\n           #slideconf{name = action1slide2,\n                      node = node,\n                      slide_with = succ,\n                      targetid = gen_split_key_fun(node, {1, 2})}\n         },\n         {\n           #slideconf{name = action2slide1,\n                      node = pred,\n                      slide_with = succ,\n                      targetid = gen_split_key_fun(pred, {1, 3})},\n           #slideconf{name = action2slide2,\n                      node = node,\n                      slide_with = succ,\n                      targetid = gen_split_key_fun(pred, {2, 3})}\n         },\n         {\n           #slideconf{name = action3slide1,\n                      node = pred,\n                      slide_with = succ,\n                      targetid = gen_split_key_fun(predspred, {1, 2})},\n           #slideconf{name = action3slide2,\n                      node = node,\n                      slide_with = succ,\n                      targetid = gen_split_key_fun(pred, {1, 2})}\n         },\n         {\n           #slideconf{name = action4slide1,\n                      node = pred,\n                      slide_with = succ,\n                      targetid = gen_split_key_fun(predspred, {1, 2})},\n           #slideconf{name = action4slide2,\n                      node = node,\n                      slide_with = succ,\n                      targetid = gen_split_key_fun(node, {1, 2})}\n         }],\n    DhtNodes = pid_groups:find_all(dht_node),\n    _ = [begin\n             slide_simultaneously(DhtNode, Action, VerifyFun)\n         end || DhtNode <- DhtNodes, Action <- Actions],\n    erlang:exit(BenchPid, 'kill'),\n    util:wait_for_process_to_die(BenchPid).\n\n\n%% @doc Test for two slides in conflict with each other\ntest_slide_conflict(_Config) ->\n    BenchPid = erlang:spawn(fun() -> bench:increment(10, 100000) end),\n    % make sure all keys have been created...\n    api_tx_SUITE:wait_for_dht_entries(440),\n    DhtNodes = pid_groups:find_all(dht_node),\n    VerifyFun =\n        fun (Result1, Result2, Interleaved) ->\n                unittest_helper:check_ring_load(440),\n                unittest_helper:check_ring_data(),\n                %% Outcome: at most one slide may succeed.\n                %% We ensure slide interleaving using breakpoints. When using the proto scheduler\n                %% we test for interleaving as we cannot use breakpoints.\n                case Interleaved of\n                    true ->\n                        ?assert_w_note(Result1 =/= ok orelse Result2 =/= ok,\n                                       [{result1, Result1}, {result2, Result2}]);\n                    false ->\n                        Vals = [ok, ongoing_slide, target_id_not_in_range, wrong_pred_succ_node],\n                        ?assert_w_note(lists:member(Result1, Vals),\n                                       {result1, Result1, 'not', Vals}),\n                        ?assert_w_note(lists:member(Result2, Vals),\n                                       {result2, Result2, 'not', Vals})\n                end\n        end,\n    Actions = [\n               {\n                 #slideconf{name = action1slide1,\n                            node = pred,\n                            slide_with = succ,\n                            targetid = gen_split_key_fun(pred, {3, 4})},\n                 #slideconf{name = action1slide2,\n                            node = node,\n                            slide_with = succ,\n                            targetid = gen_split_key_fun(pred, {1, 4})}\n               },\n               {\n                 #slideconf{name = action2slide1,\n                            node = node,\n                            slide_with = succ,\n                            targetid = gen_split_key_fun(pred, {1, 2})},\n                 #slideconf{name = action2slide2,\n                            node = node,\n                            slide_with = succ,\n                            targetid = gen_split_key_fun(node, {1, 2})}\n               }],\n    _ = [ begin\n              slide_simultaneously(DhtNode, Action, VerifyFun)\n          end || DhtNode <- DhtNodes, Action <- Actions],\n    erlang:exit(BenchPid, 'kill'),\n    util:wait_for_process_to_die(BenchPid).\n\n%% @doc slide illegally, i.e. provide a target id impossible to slide to\ntest_slide_illegally(_Config) ->\n    DhtNodes = pid_groups:find_all(dht_node),\n    %% Two slides that can't be performed\n    Slide1 = #slideconf{name = illegalslide1,\n                        node = node,\n                        slide_with = succ,\n                        targetid = gen_split_key_fun(predspred, {1, 2})},\n    Slide2 = #slideconf{name = illegalslide2,\n                        node = pred,\n                        slide_with = succ,\n                        targetid = gen_split_key_fun(node, {1, 2})},\n    %% Variations (different startup messages)\n    Slides = generate_slide_variation(Slide1) ++ generate_slide_variation(Slide2),\n    _ = [ begin\n              Nodes = get_predspred_pred_node_succ(DhtNode),\n              Node = select_from_nodes(Slide#slideconf.node, Nodes),\n              Direction = Slide#slideconf.slide_with,\n              TargetId = (Slide#slideconf.targetid)(Nodes),\n              Tag = Slide#slideconf.name,\n              ct:pal(\"Beginning ~p\", [Tag]),\n              ?proto_sched(start),\n              comm:send(node:pidX(Node), {move, start_slide, Direction, TargetId, {slide, Direction, Tag}, comm:this()}),\n              trace_mpath:thread_yield(),\n              receive\n                  ?SCALARIS_RECV({move, result, {slide, Direction, Tag}, target_id_not_in_range}, ok);\n                  ?SCALARIS_RECV(X, ?ct_fail(\"Illegal Slide ~p\", [X]))\n              end,\n              ?proto_sched(stop)\n          end || DhtNode <- DhtNodes, Slide <- Slides].\n\n%% @doc lets the tester run prop_jump_slide with different keys\ntester_test_jump(_Config) ->\n    tester:test(?MODULE, prop_jump_slide, 1, _Iterations=100).\n\n%% @doc tests the jump operation.\n-spec prop_jump_slide(TargetKey::?RT:key()) -> true.\nprop_jump_slide(TargetKey) ->\n    DhtNodes = pid_groups:find_all(dht_node),\n    %% get a random node\n    JumpingNode = util:randomelem(DhtNodes),\n    %% check if we chose a valid key\n    InvalidTarget = key_taken_as_node_id(TargetKey, JumpingNode),\n    perform_jump(JumpingNode, TargetKey, InvalidTarget),\n    true.\n\n-spec perform_jump(JumpingNode::pid(), ?RT:key(), InvalidTarget::boolean()) -> true.\nperform_jump(JumpingNode, TargetKey, InvalidTarget) ->\n    %% get neighborhood to check if jump will be a slide\n    comm:send_local(JumpingNode, {get_state, comm:this(), neighbors}),\n    Neighbors = fun() -> receive ?SCALARIS_RECV({get_state_response, Neighb}, Neighb) end end(),\n    ConvertedToSlide =\n        case intervals:in(TargetKey, nodelist:node_range(Neighbors)) orelse\n             intervals:in(TargetKey, nodelist:succ_range(Neighbors)) of\n            true -> ct:pal(\"Jump will be converted to slide.\"),\n                    true;\n            _ -> false\n        end,\n    %% debug output in case of pending slide/jump operations\n    comm:send_local(JumpingNode, {get_node_details, comm:this(), [slide_pred, slide_succ]}),\n    SlideInfo = fun() -> receive ?SCALARIS_RECV({get_node_details_response, Details}, Details) end end(),\n    [{slide_pred, SlidePred}, {slide_succ, SlideSucc}] = SlideInfo,\n    ct:pal(\"SlidePred: ~p SlideSucc: ~p\", [SlidePred, SlideSucc]),\n\n    %% start jump\n    Ring = lists:sort(fun(A, B) -> node:id(A) =< node:id(B) end,\n                      [hd(get_node_details(DhtNode, [node])) || DhtNode <- pid_groups:find_all(dht_node)]),\n    ct:pal(\"Ring: ~.7p~nNode ~p jumping to~n             ~p\", [Ring, JumpingNode, TargetKey]),\n    ?proto_sched(start),\n    comm:send_local(JumpingNode, {move, start_jump, TargetKey, prop_jump_slide, comm:this()}),\n    trace_mpath:thread_yield(),\n    Result = fun() ->\n                receive\n                    ?SCALARIS_RECV({move, result, prop_jump_slide, Res}, Res)\n                end\n             end(),\n    ct:pal(\"Result: ~p~n\", [Result]),\n    ?proto_sched(stop),\n    % when the protocol finishes, the pred/succ infos in the ring should be correct!\n    ?proto_sched(fun admin:check_ring/0), % only safe with proto_sched!\n    %% check result\n    if Result =:= wrong_pred_succ_node ->\n            % should not happen with proto_sched!\n            ?proto_sched(fun() -> ?ct_fail(\"Result should not be wrong_pred_succ_node\", []) end),\n            ct:pal(\"Retrying because of wrong_pred_succ_node\"),\n            timer:sleep(10),\n            perform_jump(JumpingNode, TargetKey, InvalidTarget);\n       InvalidTarget ->\n            ?assert_w_note((ConvertedToSlide andalso Result =:= target_id_not_in_range)\n                               orelse Result =:= ok, {ConvertedToSlide, Result}),\n            %% check if the invalid target is really taken by another node\n            ?assert(key_taken_as_node_id(TargetKey, JumpingNode)),\n            %% and that it is different from ours\n            comm:send_local(JumpingNode, {get_state, comm:this(), node_id}),\n            MyId = fun() -> receive ?SCALARIS_RECV({get_state_response, Id}, Id) end end(),\n            ?compare(fun erlang:'=/='/2, MyId, TargetKey);\n       true ->\n            ?equals(Result, ok)\n    end.\n\n-spec key_taken_as_node_id(?RT:key(), JumpingNode::comm:erl_local_pid()) -> boolean().\nkey_taken_as_node_id(Key, JumpingNode) ->\n    DhtNodes = pid_groups:find_all(dht_node),\n    lists:any(fun(Node) when Node =:= JumpingNode -> false;\n                 (Node) ->\n                %% get node information\n                comm:send_local(Node, {get_state, comm:this(), node_id}),\n                NodeId = fun() -> receive ?SCALARIS_RECV({get_state_response, Id}, Id) end end(),\n                NodeId =:= Key\n              end, DhtNodes).\n"
  },
  {
    "path": "test/dht_node_move_proto_sched_SUITE.erl",
    "content": "% @copyright 2010-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Unit tests for src/dht_node_move.erl (slide and jump operations).\n%%   Executes tests using the protocol scheduler producing a random interleaving\n%%   between the different channels.\n%% @end\n%% @version $Id$\n-module(dht_node_move_proto_sched_SUITE).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n%% start proto scheduler for this suite\n-define(proto_sched(Action), proto_sched_fun(Action)).\n\n%% number of slides without timeouts\n-define(NUM_SLIDES, 25).\n\n-include(\"dht_node_move_SUITE.hrl\").\n\nall() ->\n    [\n     {group, send_to_pred},\n     {group, send_to_pred_incremental},\n     {group, send_to_succ},\n     {group, send_to_succ_incremental},\n     {group, send_to_both},\n     {group, send_to_both_incremental},\n     {group, slide_illegally},\n     {group, jump_slide}\n    ].\n\nsuite() -> [ {timetrap, {seconds, 60}} ].\n\n-spec additional_ring_config() -> [{stabilization_interval_base, 100000},...].\nadditional_ring_config() ->\n    % increase ring stabilisation interval since proto_sched infections get\n    % lost if rm subscriptions are triggered instead of continuing due to our\n    % direct (and infected) messages!\n    [{stabilization_interval_base, 100000}, % ms\n     {tman_cyclon_interval, 100} % s\n    ].\n\n-spec proto_sched_fun(start | stop | fun(() -> ok)) -> ok.\nproto_sched_fun(start) ->\n    %% ct:pal(\"Starting proto scheduler\"),\n    proto_sched:thread_num(1),\n    proto_sched:thread_begin();\nproto_sched_fun(stop) ->\n    %% is a ring running?\n    case erlang:whereis(pid_groups) =:= undefined\n             orelse pid_groups:find_a(proto_sched) =:= failed of\n        true -> ok;\n        false ->\n            %% then finalize proto_sched run:\n            %% try to call thread_end(): if this\n            %% process was running the proto_sched\n            %% thats fine, otherwise thread_end()\n            %% will raise an exception\n            proto_sched:thread_end(),\n            proto_sched:wait_for_end(),\n            unittest_helper:print_proto_sched_stats(at_end_if_failed),\n            proto_sched:cleanup()\n    end;\nproto_sched_fun(Fun) when is_function(Fun) ->\n    Fun().\n"
  },
  {
    "path": "test/dn_cache_SUITE.erl",
    "content": "%  @copyright 2010-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Test suite for the dn_cache module.\n%% @end\n%% @version $Id$\n-module(dn_cache_SUITE).\n\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n\nall() ->\n    [dn_detection].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 30}}\n    ].\n\ninit_per_suite(Config) ->\n    unittest_helper:start_minimal_procs(Config, [], true).\n\nend_per_suite(Config) ->\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\ndn_detection(Config) ->\n    pid_groups:join_as(dn_cache_group, node),\n    config:write(zombieDetectorInterval, 10),\n    NodePid = fake_node(),\n    NodePidG = comm:make_global(NodePid),\n    Node = node:new(NodePidG, ?RT:hash_key(\"0\"), 0),\n    {ok, _DNCachePid} = dn_cache:start_link(dn_cache_group),\n    \n    dn_cache:subscribe(),\n    comm:send(NodePidG, {sleep}),\n    dn_cache:add_zombie_candidate(Node),\n    ?expect_no_message(),\n    \n    comm:send(NodePidG, {continue}),\n    ?expect_message({zombie, Node}),\n    \n    exit(NodePid, kill),\n    % there might be more zombie messages:\n    ?consume_all_messages({zombie, Node}),\n    \n    Config.\n\nfake_node() ->\n    element(1, unittest_helper:start_subprocess(\n              fun() -> pid_groups:join_as(dn_cache_group, node) end,\n              fun fake_process/0)).\n\nfake_process() ->\n    receive\n        {ping, Pid} ->\n            comm:send(Pid, {pong, dht_node}),\n            fake_process();\n        {sleep} ->\n            receive\n                {continue} -> fake_process()\n            end\n    end.\n"
  },
  {
    "path": "test/erl_id_trans.erl",
    "content": "%% ``The contents of this file are subject to the Erlang Public License,\n%% Version 1.1, (the \"License\"); you may not use this file except in\n%% compliance with the License. You should have received a copy of the\n%% Erlang Public License along with this software. If not, it can be\n%% retrieved via the world wide web at http://www.erlang.org/.\n%% \n%% Software distributed under the License is distributed on an \"AS IS\"\n%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See\n%% the License for the specific language governing rights and limitations\n%% under the License.\n%% \n%% The Initial Developer of the Original Code is Ericsson Utvecklings AB.\n%% Portions created by Ericsson are Copyright 1999, Ericsson Utvecklings\n%% AB. All Rights Reserved.''\n%% \n%%     $Id$\n%%\n-module(erl_id_trans).\n\n%% A identity transformer of Erlang abstract syntax.\n\n%% This module only traverses legal Erlang code. This is most noticeable\n%% in guards where only a limited number of expressions are allowed.\n%% N.B. if this module is to be used as a basis for tranforms then\n%% all the error cases must be handled otherwise this module just crashes!\n\n-export([parse_transform/2,\n\n        farity_list/1, record_defs/1, function/3]).\n\n-spec parse_transform(any(), any()) -> any().\nparse_transform(Forms, _Options) ->\n    forms(Forms).\n\n%% forms(Fs) -> lists:map(fun (F) -> form(F) end, Fs).\n\nforms([F0|Fs0]) ->\n    F1 = form(F0),\n    Fs1 = forms(Fs0),\n    [F1|Fs1];\nforms([]) -> [].\n\n%% -type form(Form) -> Form.\n%%  Here we show every known form and valid internal structure. We do not\n%%  that the ordering is correct!\n\n%% First the various attributes.\nform({attribute,Line,module,Mod}) ->\n    {attribute,Line,module,Mod};\nform({attribute,Line,file,{File,Line}}) ->\t%This is valid anywhere.\n    {attribute,Line,file,{File,Line}};\nform({attribute,Line,export,Es0}) ->\n    Es1 = farity_list(Es0),\n    {attribute,Line,export,Es1};\nform({attribute,Line,import,{Mod,Is0}}) ->\n    Is1 = farity_list(Is0),\n    {attribute,Line,import,{Mod,Is1}};\nform({attribute,Line,compile,C}) ->\n    {attribute,Line,compile,C};\nform({attribute,Line,record,{Name,Defs0}}) ->\n    Defs1 = record_defs(Defs0),\n    {attribute,Line,record,{Name,Defs1}};\nform({attribute,Line,asm,{function,N,A,Code}}) ->\n    {attribute,Line,asm,{function,N,A,Code}};\nform({attribute,Line,Attr,Val}) ->\t\t%The general attribute.\n    {attribute,Line,Attr,Val};\nform({function,Line,Name0,Arity0,Clauses0}) ->\n    {Name,Arity,Clauses} = function(Name0, Arity0, Clauses0),\n    {function,Line,Name,Arity,Clauses};\n% Mnemosyne, ignore...\nform({rule,Line,Name,Arity,Body}) ->\n    {rule,Line,Name,Arity,Body}; % Dont dig into this\n%% Extra forms from the parser.\nform({error,E}) -> {error,E};\nform({warning,W}) -> {warning,W};\nform({eof,Line}) -> {eof,Line}.\n\n%% -type farity_list([Farity]) -> [Farity] when Farity <= {atom(),integer()}.\n\n-spec farity_list(list()) -> list().\nfarity_list([{Name,Arity}|Fas]) ->\n    [{Name,Arity}|farity_list(Fas)];\nfarity_list([]) -> [].\n\n%% -type record_defs([RecDef]) -> [RecDef].\n%%  N.B. Field names are full expressions here but only atoms are allowed\n%%  by the *parser*!\n\n-spec record_defs(list()) -> list().\nrecord_defs([{record_field,Line,{atom,La,A},Val0}|Is]) ->\n    Val1 = expr(Val0),\n    [{record_field,Line,{atom,La,A},Val1}|record_defs(Is)];\nrecord_defs([{record_field,Line,{atom,La,A}}|Is]) ->\n    [{record_field,Line,{atom,La,A}}|record_defs(Is)];\nrecord_defs([]) -> [].\n\n%% -type function(atom(), integer(), [Clause]) -> {atom(),integer(),[Clause]}.\n\n-spec function(atom(), integer(), any()) -> {atom(), integer(), any()}.\nfunction(Name, Arity, Clauses0) ->\n    Clauses1 = clauses(Clauses0),\n    {Name,Arity,Clauses1}.\n\n%% -type clauses([Clause]) -> [Clause].\n\nclauses([C0|Cs]) ->\n    C1 = clause(C0),\n    [C1|clauses(Cs)];\nclauses([]) -> [].\n\n%% -type clause(Clause) -> Clause.\n\nclause({clause,Line,H0,G0,B0}) ->\n    H1 = head(H0),\n    G1 = guard(G0),\n    B1 = exprs(B0),\n    {clause,Line,H1,G1,B1}.\n\n%% -type head([Pattern]) -> [Pattern].\n\nhead(Ps) -> patterns(Ps).\n\n%% -type patterns([Pattern]) -> [Pattern].\n%%  These patterns are processed \"sequentially\" for purposes of variable\n%%  definition etc.\n\npatterns([P0|Ps]) ->\n    P1 = pattern(P0),\n    [P1|patterns(Ps)];\npatterns([]) -> [].\n\n%% -type pattern(Pattern) -> Pattern.\n%%  N.B. Only valid patterns are included here.\n\npattern({var,Line,V}) -> {var,Line,V};\npattern({match,Line,L0,R0}) ->\n    L1 = pattern(L0),\n    R1 = pattern(R0),\n    {match,Line,L1,R1};\npattern({integer,Line,I}) -> {integer,Line,I};\npattern({char,Line,C}) -> {char,Line,C};\npattern({float,Line,F}) -> {float,Line,F};\npattern({atom,Line,A}) -> {atom,Line,A};\npattern({string,Line,S}) -> {string,Line,S};\npattern({nil,Line}) -> {nil,Line};\npattern({cons,Line,H0,T0}) ->\n    H1 = pattern(H0),\n    T1 = pattern(T0),\n    {cons,Line,H1,T1};\npattern({tuple,Line,Ps0}) ->\n    Ps1 = pattern_list(Ps0),\n    {tuple,Line,Ps1};\n%%pattern({struct,Line,Tag,Ps0}) ->\n%%    Ps1 = pattern_list(Ps0),\n%%    {struct,Line,Tag,Ps1};\npattern({record,Line,Name,Pfs0}) ->\n    Pfs1 = pattern_fields(Pfs0),\n    {record,Line,Name,Pfs1};\npattern({record_index,Line,Name,Field0}) ->\n    Field1 = pattern(Field0),\n    {record_index,Line,Name,Field1};\n%% record_field occurs in query expressions\npattern({record_field,Line,Rec0,Name,Field0}) ->\n    Rec1 = expr(Rec0),\n    Field1 = expr(Field0),\n    {record_field,Line,Rec1,Name,Field1};\npattern({record_field,Line,Rec0,Field0}) ->\n    Rec1 = expr(Rec0),\n    Field1 = expr(Field0),\n    {record_field,Line,Rec1,Field1};\npattern({bin,Line,Fs}) ->\n    Fs2 = pattern_grp(Fs),\n    {bin,Line,Fs2};\npattern({op,Line,Op,A}) ->\n    {op,Line,Op,A};\npattern({op,Line,Op,L,R}) ->\n    {op,Line,Op,L,R}.\n\npattern_grp([{bin_element,L1,E1,S1,T1} | Fs]) ->\n    S2 = case S1 of\n\t     default ->\n\t\t default;\n\t     _ ->\n\t\t expr(S1)\n\t end,\n    T2 = case T1 of\n\t     default ->\n\t\t default;\n\t     _ ->\n\t\t bit_types(T1)\n\t end,\n    [{bin_element,L1,expr(E1),S2,T2} | pattern_grp(Fs)];\npattern_grp([]) ->\n    [].\n\nbit_types([]) ->\n    [];\nbit_types([Atom | Rest]) when is_atom(Atom) ->\n    [Atom | bit_types(Rest)];\nbit_types([{Atom, Integer} | Rest]) when is_atom(Atom), is_integer(Integer) ->\n    [{Atom, Integer} | bit_types(Rest)].\n\n\n\n%% -type pattern_list([Pattern]) -> [Pattern].\n%%  These patterns are processed \"in parallel\" for purposes of variable\n%%  definition etc.\n\npattern_list([P0|Ps]) ->\n    P1 = pattern(P0),\n    [P1|pattern_list(Ps)];\npattern_list([]) -> [].\n\n%% -type pattern_fields([Field]) -> [Field].\n%%  N.B. Field names are full expressions here but only atoms are allowed\n%%  by the *linter*!.\n\npattern_fields([{record_field,Lf,{atom,La,F},P0}|Pfs]) ->\n    P1 = pattern(P0),\n    [{record_field,Lf,{atom,La,F},P1}|pattern_fields(Pfs)];\npattern_fields([{record_field,Lf,{var,La,'_'},P0}|Pfs]) ->\n    P1 = pattern(P0),\n    [{record_field,Lf,{var,La,'_'},P1}|pattern_fields(Pfs)];\npattern_fields([]) -> [].\n\n%% -type guard([GuardTest]) -> [GuardTest].\n\nguard([G0|Gs]) when is_list(G0) ->\n    [guard0(G0) | guard(Gs)];\nguard(L) ->\n    guard0(L).\n\nguard0([G0|Gs]) ->\n    G1 =  guard_test(G0),\n    [G1|guard0(Gs)];\nguard0([]) -> [].\n\nguard_test(Expr={call,Line,{atom,La,F},As0}) ->\n    case erl_internal:type_test(F, length(As0)) of\n\ttrue -> \n\t    As1 = gexpr_list(As0),\n\t    {call,Line,{atom,La,F},As1};\n\t_ ->\n\t    gexpr(Expr)\n    end;\nguard_test(Any) ->\n    gexpr(Any).\n\n%% Before R9, there were special rules regarding the expressions on\n%% top level in guards. Those limitations are now lifted - therefore\n%% there is no need for a special clause for the toplevel expressions.\n%% -type gexpr(GuardExpr) -> GuardExpr.\n\ngexpr({var,Line,V}) -> {var,Line,V};\ngexpr({integer,Line,I}) -> {integer,Line,I};\ngexpr({char,Line,C}) -> {char,Line,C};\ngexpr({float,Line,F}) -> {float,Line,F};\ngexpr({atom,Line,A}) -> {atom,Line,A};\ngexpr({string,Line,S}) -> {string,Line,S};\ngexpr({nil,Line}) -> {nil,Line};\ngexpr({cons,Line,H0,T0}) ->\n    H1 = gexpr(H0),\n    T1 = gexpr(T0),\t\t\t\t%They see the same variables\n    {cons,Line,H1,T1};\ngexpr({tuple,Line,Es0}) ->\n    Es1 = gexpr_list(Es0),\n    {tuple,Line,Es1};\ngexpr({record_index,Line,Name,Field0}) ->\n    Field1 = gexpr(Field0),\n    {record_index,Line,Name,Field1};\ngexpr({record_field,Line,Rec0,Name,Field0}) ->\n    Rec1 = gexpr(Rec0),\n    Field1 = gexpr(Field0),\n    {record_field,Line,Rec1,Name,Field1};\ngexpr({record,Line,Name,Inits0}) ->\n    Inits1 = grecord_inits(Inits0),\n    {record,Line,Name,Inits1};\ngexpr({call,Line,{atom,La,F},As0}) ->\n    case erl_internal:guard_bif(F, length(As0)) of\n\ttrue -> As1 = gexpr_list(As0),\n\t\t{call,Line,{atom,La,F},As1}\n    end;\n% Guard bif's can be remote, but only in the module erlang...\ngexpr({call,Line,{remote,La,{atom,Lb,erlang},{atom,Lc,F}},As0}) ->\n    case erl_internal:guard_bif(F, length(As0)) or\n\t erl_internal:arith_op(F, length(As0)) or \n\t erl_internal:comp_op(F, length(As0)) or\n\t erl_internal:bool_op(F, length(As0)) of\n\ttrue -> As1 = gexpr_list(As0),\n\t\t{call,Line,{remote,La,{atom,Lb,erlang},{atom,Lc,F}},As1}\n    end;\n% Unfortunately, writing calls as {M,F}(...) is also allowed.\ngexpr({call,Line,{tuple,La,[{atom,Lb,erlang},{atom,Lc,F}]},As0}) ->\n    case erl_internal:guard_bif(F, length(As0)) or\n\t erl_internal:arith_op(F, length(As0)) or \n\t erl_internal:comp_op(F, length(As0)) or\n\t erl_internal:bool_op(F, length(As0)) of\n\ttrue -> As1 = gexpr_list(As0),\n\t\t{call,Line,{tuple,La,[{atom,Lb,erlang},{atom,Lc,F}]},As1}\n    end;\ngexpr({bin,Line,Fs}) ->\n    Fs2 = pattern_grp(Fs),\n    {bin,Line,Fs2};\ngexpr({op,Line,Op,A0}) ->\n    case erl_internal:arith_op(Op, 1) or \n\t erl_internal:bool_op(Op, 1) of\n\ttrue -> A1 = gexpr(A0),\n\t\t{op,Line,Op,A1}\n    end;\ngexpr({op,Line,Op,L0,R0}) when Op =:= 'andalso'; Op =:= 'orelse' ->\n    %% R11B: andalso/orelse are now allowed in guards.\n    L1 = gexpr(L0),\n    R1 = gexpr(R0),\t\t\t%They see the same variables\n    {op,Line,Op,L1,R1};\ngexpr({op,Line,Op,L0,R0}) ->\n    case erl_internal:arith_op(Op, 2) or\n\t  erl_internal:bool_op(Op, 2) or \n\t  erl_internal:comp_op(Op, 2) of\n\ttrue ->\n\t    L1 = gexpr(L0),\n\t    R1 = gexpr(R0),\t\t\t%They see the same variables\n\t    {op,Line,Op,L1,R1}\n    end.\n\n%% -type gexpr_list([GuardExpr]) -> [GuardExpr].\n%%  These expressions are processed \"in parallel\" for purposes of variable\n%%  definition etc.\n\ngexpr_list([E0|Es]) ->\n    E1 = gexpr(E0),\n    [E1|gexpr_list(Es)];\ngexpr_list([]) -> [].\n\ngrecord_inits([{record_field,Lf,{atom,La,F},Val0}|Is]) ->\n    Val1 = gexpr(Val0),\n    [{record_field,Lf,{atom,La,F},Val1}|grecord_inits(Is)];\ngrecord_inits([{record_field,Lf,{var,La,'_'},Val0}|Is]) ->\n    Val1 = gexpr(Val0),\n    [{record_field,Lf,{var,La,'_'},Val1}|grecord_inits(Is)];\ngrecord_inits([]) -> [].\n\n%% -type exprs([Expression]) -> [Expression].\n%%  These expressions are processed \"sequentially\" for purposes of variable\n%%  definition etc.\n\nexprs([E0|Es]) ->\n    E1 = expr(E0),\n    [E1|exprs(Es)];\nexprs([]) -> [].\n\n%% -type expr(Expression) -> Expression.\n\nexpr({var,Line,V}) -> {var,Line,V};\nexpr({integer,Line,I}) -> {integer,Line,I};\nexpr({float,Line,F}) -> {float,Line,F};\nexpr({atom,Line,A}) -> {atom,Line,A};\nexpr({string,Line,S}) -> {string,Line,S};\nexpr({char,Line,C}) -> {char,Line,C};\nexpr({nil,Line}) -> {nil,Line};\nexpr({cons,Line,H0,T0}) ->\n    H1 = expr(H0),\n    T1 = expr(T0),\t\t\t\t%They see the same variables\n    {cons,Line,H1,T1};\nexpr({lc,Line,E0,Qs0}) ->\n    Qs1 = lc_bc_quals(Qs0),\n    E1 = expr(E0),\n    {lc,Line,E1,Qs1};\nexpr({bc,Line,E0,Qs0}) ->\n    Qs1 = lc_bc_quals(Qs0),\n    E1 = expr(E0),\n    {bc,Line,E1,Qs1};\nexpr({tuple,Line,Es0}) ->\n    Es1 = expr_list(Es0),\n    {tuple,Line,Es1};\n%%expr({struct,Line,Tag,Es0}) ->\n%%    Es1 = pattern_list(Es0),\n%%    {struct,Line,Tag,Es1};\nexpr({record_index,Line,Name,Field0}) ->\n    Field1 = expr(Field0),\n    {record_index,Line,Name,Field1};\nexpr({record,Line,Name,Inits0}) ->\n    Inits1 = record_inits(Inits0),\n    {record,Line,Name,Inits1};\nexpr({record_field,Line,Rec0,Name,Field0}) ->\n    Rec1 = expr(Rec0),\n    Field1 = expr(Field0),\n    {record_field,Line,Rec1,Name,Field1};\nexpr({record,Line,Rec0,Name,Upds0}) ->\n    Rec1 = expr(Rec0),\n    Upds1 = record_updates(Upds0),\n    {record,Line,Rec1,Name,Upds1};\nexpr({record_field,Line,Rec0,Field0}) ->\n    Rec1 = expr(Rec0),\n    Field1 = expr(Field0),\n    {record_field,Line,Rec1,Field1};\nexpr({block,Line,Es0}) ->\n    %% Unfold block into a sequence.\n    Es1 = exprs(Es0),\n    {block,Line,Es1};\nexpr({'if',Line,Cs0}) ->\n    Cs1 = icr_clauses(Cs0),\n    {'if',Line,Cs1};\nexpr({'case',Line,E0,Cs0}) ->\n    E1 = expr(E0),\n    Cs1 = icr_clauses(Cs0),\n    {'case',Line,E1,Cs1};\nexpr({'receive',Line,Cs0}) ->\n    Cs1 = icr_clauses(Cs0),\n    {'receive',Line,Cs1};\nexpr({'receive',Line,Cs0,To0,ToEs0}) ->\n    To1 = expr(To0),\n    ToEs1 = exprs(ToEs0),\n    Cs1 = icr_clauses(Cs0),\n    {'receive',Line,Cs1,To1,ToEs1};\nexpr({'try',Line,Es0,Scs0,Ccs0,As0}) ->\n    Es1 = exprs(Es0),\n    Scs1 = icr_clauses(Scs0),\n    Ccs1 = icr_clauses(Ccs0),\n    As1 = exprs(As0),\n    {'try',Line,Es1,Scs1,Ccs1,As1};\nexpr({'fun',Line,Body}) ->\n    case Body of\n\t{clauses,Cs0} ->\n\t    Cs1 = fun_clauses(Cs0),\n\t    {'fun',Line,{clauses,Cs1}};\n\t{function,F,A} ->\n\t    {'fun',Line,{function,F,A}};\n\t{function,M,F,A} ->\t\t\t%R10B-6: fun M:F/A.\n\t    {'fun',Line,{function,M,F,A}}\n    end;\nexpr({call,Line,F0,As0}) ->\n    %% N.B. If F an atom then call to local function or BIF, if F a\n    %% remote structure (see below) then call to other module,\n    %% otherwise apply to \"function\".\n    F1 = expr(F0),\n    As1 = expr_list(As0),\n    {call,Line,F1,As1};\nexpr({'catch',Line,E0}) ->\n    %% No new variables added.\n    E1 = expr(E0),\n    {'catch',Line,E1};\nexpr({'query', Line, E0}) ->\n    %% lc expression\n    E = expr(E0),\n    {'query', Line, E};\nexpr({match,Line,P0,E0}) ->\n    E1 = expr(E0),\n    P1 = pattern(P0),\n    {match,Line,P1,E1};\nexpr({bin,Line,Fs}) ->\n    Fs2 = pattern_grp(Fs),\n    {bin,Line,Fs2};\nexpr({op,Line,Op,A0}) ->\n    A1 = expr(A0),\n    {op,Line,Op,A1};\nexpr({op,Line,Op,L0,R0}) ->\n    L1 = expr(L0),\n    R1 = expr(R0),\t\t\t\t%They see the same variables\n    {op,Line,Op,L1,R1};\n%% The following are not allowed to occur anywhere!\nexpr({remote,Line,M0,F0}) ->\n    M1 = expr(M0),\n    F1 = expr(F0),\n    {remote,Line,M1,F1}.\n\n%% -type expr_list([Expression]) -> [Expression].\n%%  These expressions are processed \"in parallel\" for purposes of variable\n%%  definition etc.\n\nexpr_list([E0|Es]) ->\n    E1 = expr(E0),\n    [E1|expr_list(Es)];\nexpr_list([]) -> [].\n\n%% -type record_inits([RecordInit]) -> [RecordInit].\n%%  N.B. Field names are full expressions here but only atoms are allowed\n%%  by the *linter*!.\n\nrecord_inits([{record_field,Lf,{atom,La,F},Val0}|Is]) ->\n    Val1 = expr(Val0),\n    [{record_field,Lf,{atom,La,F},Val1}|record_inits(Is)];\nrecord_inits([{record_field,Lf,{var,La,'_'},Val0}|Is]) ->\n    Val1 = expr(Val0),\n    [{record_field,Lf,{var,La,'_'},Val1}|record_inits(Is)];\nrecord_inits([]) -> [].\n\n%% -type record_updates([RecordUpd]) -> [RecordUpd].\n%%  N.B. Field names are full expressions here but only atoms are allowed\n%%  by the *linter*!.\n\nrecord_updates([{record_field,Lf,{atom,La,F},Val0}|Us]) ->\n    Val1 = expr(Val0),\n    [{record_field,Lf,{atom,La,F},Val1}|record_updates(Us)];\nrecord_updates([]) -> [].\n\n%% -type icr_clauses([Clause]) -> [Clause].\n\nicr_clauses([C0|Cs]) ->\n    C1 = clause(C0),\n    [C1|icr_clauses(Cs)];\nicr_clauses([]) -> [].\n\n%% -type lc_bc_quals([Qualifier]) -> [Qualifier].\n%%  Allow filters to be both guard tests and general expressions.\n\nlc_bc_quals([{generate,Line,P0,E0}|Qs]) ->\n    E1 = expr(E0),\n    P1 = pattern(P0),\n    [{generate,Line,P1,E1}|lc_bc_quals(Qs)];\nlc_bc_quals([{b_generate,Line,P0,E0}|Qs]) ->\n    E1 = expr(E0),\n    P1 = pattern(P0),\n    [{b_generate,Line,P1,E1}|lc_bc_quals(Qs)];\nlc_bc_quals([E0|Qs]) ->\n    E1 = expr(E0),\n    [E1|lc_bc_quals(Qs)];\nlc_bc_quals([]) -> [].\n\n%% -type fun_clauses([Clause]) -> [Clause].\n\nfun_clauses([C0|Cs]) ->\n    C1 = clause(C0),\n    [C1|fun_clauses(Cs)];\nfun_clauses([]) -> [].\n"
  },
  {
    "path": "test/gossip_SUITE.erl",
    "content": "%  @copyright 2010-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jens V. Fischer <jensvfischer@gmail.com>\n%% @doc    Integrationstests for the gossip and gossip_load modules.\n%% @end\n%% @version $Id$\n-module(gossip_SUITE).\n\n-author('jensvfischer@gmail.com').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\n-dialyzer({no_fail_call, test_request_histogram1/1}).\n\n-define(NO_OF_NODES, 5).\n\nall() ->\n    [\n        test_no_load,\n        test_load,\n        test_request_histogram1,\n        test_request_histogram2\n    ].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 80}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) ->\n    unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) ->\n    unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(_TestCase, Config) ->\n    unittest_helper:make_ring_with_ids(\n      [?MINUS_INFINITY],\n      [{config, [\n                 {monitor_perf_interval, 0},  % deactivate monitor_perf\n                 {gossip_load_interval, 100}, % truncated to 0, i.e. immediate delivery\n                 {gossip_load_convergence_count_new_round, 5},\n                 {gossip_load_convergence_count_best_values, 1},\n                 {gossip_log_level_warn, warn},\n                 {gossip_log_level_error, error}\n                ]\n       }]),\n    {_, []} = api_vm:add_nodes(?NO_OF_NODES - 1),\n    unittest_helper:wait_for_stable_ring_deep(),\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Testcases\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ntest_no_load(_Config) ->\n\n    % get values from gossiping (after round finishes)\n    wait_n_rounds(1),\n    send2gossip({cb_msg, {gossip_load, default}, {gossip_get_values_all, self()}}, 1),\n\n    %                 {load_info, avg, stddev, size_ldr, size_kr, minLoad, maxLoad, merged}\n    LoadInfoExpected ={load_info, 0.0, 0.0, 5.0, 5.0, 0, 0, whatever, []},\n    receive {gossip_get_values_all_response, {PrevLoadInfo, _, _}} ->\n            ?compare(fun compare/2, PrevLoadInfo, LoadInfoExpected)\n    end.\n\n\ntest_load(_Config) ->\n    write(100),\n\n    % request load from all nodes and calc expected values manually\n    Loads = lists:map(fun (State) -> dht_node_state:get(State, load) end, get_node_states()),\n    Avg = lists:sum(Loads)/?NO_OF_NODES,\n    Stddev = calc_stddev(Loads),\n    Size = ?NO_OF_NODES,\n    Min = lists:min(Loads),\n    Max = lists:max(Loads),\n\n    %                  {load_info, avg, stddev, size_ldr, size_kr, minLoad, maxLoad, merged, other}\n    LoadInfoExpected = {load_info, Avg, Stddev, Size,     Size,    Min,     Max,      dc,    []},\n\n    % get values from gossiping (after round finishes)\n    % first round might be interrupted by node joins, thus wait two rounds\n    wait_n_rounds(1),\n    send2gossip({cb_msg, {gossip_load, default}, {gossip_get_values_all, self()}}, 1),\n\n    receive {gossip_get_values_all_response, {PrevLoadInfo, _, _}} ->\n                ?compare(fun compare/2, PrevLoadInfo, LoadInfoExpected)\n    end.\n\n\ntest_request_histogram1(_Config) ->\n    ?expect_exception(gossip_load:request_histogram(0, comm:this()), error, function_clause).\n\n\ntest_request_histogram2(_Config) ->\n    write(100),\n    NoOfBuckets = 10,\n\n    % get the states from all nodes and calc a histogram manually\n    DHTStates = get_node_states(),\n    Histos = lists:map(fun(State) -> init_histo(State, NoOfBuckets) end, DHTStates),\n    MergeFun =\n        fun(Histo, Acc) ->\n            Combine = fun({Load1, N1}, {Load2, N2}) -> {Load1+Load2, N1+N2};\n                         (unknown, {Load, N}) -> {Load, N};\n                         ({Load, N}, unknown) -> {Load, N} end,\n            lists:zipwith(Combine, Histo, Acc)\n        end,\n    InitialAcc = [ {0,0} || _X <- lists:seq(1, NoOfBuckets)],\n    MergedHisto = lists:foldl(MergeFun, InitialAcc, Histos),\n\n    % calculate the histogram from Sums and Numbers of Nodes\n    Histo = lists:map(fun({Sum, N}) -> Sum/N end, MergedHisto),\n    %% log:log(\"Histo: ~w\", [Histo]),\n\n    gossip_load:request_histogram(NoOfBuckets, comm:this()),\n    %% GossipedHisto = receive {histogram, Histo} -> Histo end, % doesn't work, don't know why!\n    GossipedHisto = receive Msg -> element(2, Msg) end,\n\n    % remove the intervals from gossiped histo:\n    GossipedHisto1 = lists:map(fun({_Interval, Value}) -> Value end, GossipedHisto),\n\n    % calc the estimates\n    GossipedHisto2 = lists:map(fun({Value, Weight}) -> Value/Weight end, GossipedHisto1),\n    %% log:log(\"GossipedHisto: ~w\", [GossipedHisto2]),\n    ?compare(fun compare/2, GossipedHisto2, Histo).\n\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Helper\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Sends a Msg to the gossip process\n-spec send2gossip(Msg::comm:message(), Delay::non_neg_integer()) -> ok.\nsend2gossip(Msg, Delay) ->\n    Group = pid_groups:group_with(gossip),\n    Pid = pid_groups:pid_of(Group, gossip),\n    msg_delay:send_local(Delay, Pid, Msg).\n\n%% @doc Waits n rounds, pulling the web_debug_info every second.\n-spec wait_n_rounds(NoOfRounds::pos_integer()) -> ok.\nwait_n_rounds(NoOfRounds) ->\n    Round = get_current_round(),\n    wait_for_round(Round+NoOfRounds).\n\n\n%% @doc Helper for wait_n_round/0\n-spec wait_for_round(TargetRound::pos_integer()) -> ok.\nwait_for_round(TargetRound) ->\n    Round = get_current_round(),\n    %% log:log(\"CurrentRound: ~w, TargetRound: ~w\", [Round, TargetRound]),\n    if Round >= TargetRound -> ok;\n       Round =/= TargetRound -> wait_for_round(TargetRound)\n    end.\n\n\n%% @doc Get the current round (with one second delay).\n-spec get_current_round() -> non_neg_integer().\nget_current_round() ->\n    send2gossip({web_debug_info, self()}, 1),\n    receive {web_debug_info_reply, KeyValueList} ->\n        case lists:keyfind(\"cur_round\", 1, KeyValueList) of\n            {\"cur_round\", Round} -> Round;\n            false -> get_current_round() % happens if cb module not yet initiated\n        end\n    end.\n\n\n%% @doc Writes the given number of dummy entries to scalaris.\n-spec write(NoOfEntries::pos_integer()) -> ok.\nwrite(NoOfEntries) ->\n    _CommitLog = [ begin\n        K = io_lib:format(\"k~w\", [N]),\n        V = io_lib:format(\"v~w\", [N]),\n        api_tx:write(K,V)\n      end || N <- lists:seq(1,NoOfEntries) ],\n    %% log:log(\"CommitLog: ~w\", [_CommitLog]),\n    ok.\n\n\n%% @doc Get the states from all dht nodes.\n-spec get_node_states() -> [dht_node_state:state(), ...].\nget_node_states() ->\n    % request load from all nodes and calc expected values manually\n    [gen_component:get_state(Pid) || Pid <- pid_groups:find_all(dht_node)].\n\n\n%% @doc Builds a initial histogram, i.e. a histogram for a single node.\n-spec init_histo(DHTNodeState::dht_node_state:state(), NoOfBuckets::pos_integer()) -> [{non_neg_integer(), 1}, ...].\ninit_histo(DHTNodeState, NoOfBuckets) ->\n    DB = dht_node_state:get(DHTNodeState, db),\n    MyRange = dht_node_state:get(DHTNodeState, my_range),\n    Buckets = intervals:split(intervals:all(), NoOfBuckets),\n    _Histo = [ get_load_for_interval(BucketInterval, MyRange, DB) || BucketInterval <- Buckets ],\n    %% log:log(\"Histo ~w\", [_Histo]),\n    _Histo.\n\n\n%% @doc Gets the load for a given interval.\n-spec get_load_for_interval(BucketInterval::intervals:interval(),\n    MyRange::intervals:interval(), DB::db_dht:db()) -> {non_neg_integer(), 1}.\nget_load_for_interval(BucketInterval, MyRange, DB) ->\n    case intervals:is_empty(intervals:intersection(BucketInterval, MyRange)) of\n        true -> unknown;\n        false ->\n            Load = db_dht:get_load(DB, BucketInterval),\n            {Load, 1}\n    end.\n\n\n%% @doc Compares LoadInfo records and Histo. Returns true if the difference is less than 5 %.\n-spec compare(LoadInfo1::gossip_load:load_info(), LoadInfo1::gossip_load:load_info()) -> boolean();\n             (Histo1::[float(),...], Histo2::[float(),...]) -> boolean().\ncompare(LoadInfo1, LoadInfo2) when is_tuple(LoadInfo1) andalso is_tuple(LoadInfo2) ->\n    %% log:log(\"LoadInfo1: ~w~n\", [LoadInfo1]),\n    %% log:log(\"LoadInfo2: ~w~n\", [LoadInfo2]),\n    Fun = fun(Key, {Acc, LI1, LI2}) ->\n            Value1 = gossip_load:load_info_get(Key, LI1),\n            Value2 = gossip_load:load_info_get(Key, LI2),\n            {Acc andalso calc_diff(Value1, Value2) < 5.0, LI1, LI2}\n    end,\n    % merged counter is excluded from comparison\n    element(1, lists:foldl(Fun, {true, LoadInfo1, LoadInfo2},\n                           [avgLoad, stddev, size_ldr, size_kr, minLoad, maxLoad]));\n\ncompare(Histo1, Histo2) when is_list(Histo1) andalso is_list(Histo2) ->\n    Fun = fun(Val1, Val2) -> calc_diff(Val1, Val2) < 5.0 end,\n    ComparisonResult = lists:zipwith(Fun, Histo1, Histo2),\n    lists:all(fun(X) -> X end, ComparisonResult).\n\n\n%% @doc Calculates the difference in percent from one value to another value.\n-spec calc_diff(Value1::float()|unknown, Value2::float()|unknown) -> float().\ncalc_diff(Value1, Value2) ->\n    if\n        (Value1 =/= unknown) andalso (Value1 =:= Value2) -> 0.0;\n        (Value1 =:= unknown) orelse (Value2 =:= unknown) orelse (Value1 == 0) -> 100.0;\n        true -> ((Value1 + abs(Value2 - Value1)) * 100.0 / Value1) - 100\n    end.\n\n%% @doc Calculate the standard deviation for a given list of numbers.\n-spec calc_stddev(Loads::[number()]) -> float().\ncalc_stddev(Loads) ->\n    Avg = lists:sum(Loads)/?NO_OF_NODES,\n    LoadsSquared = lists:map(fun (Load) -> math:pow(Avg-Load, 2) end, Loads),\n    _Stddev = math:sqrt(lists:sum(LoadsSquared)/?NO_OF_NODES).\n\n"
  },
  {
    "path": "test/gossip_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{cases, tests, type_check_SUITE, [tester_type_check_gossip]}.\n{suites, tests, [gossip_SUITE, gossip_cyclon_SUITE, gossip_vivaldi_SUITE]}.\n"
  },
  {
    "path": "test/gossip_cyclon_SUITE.erl",
    "content": "%  @copyright 2010-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jens V. Fischer <jensvfischer@gmail.com>\n%% @doc    Tests for the gossip_cyclon module\n%% @end\n%% @version $Id$\n-module(gossip_cyclon_SUITE).\n\n-author('jensvfischer@gmail.com').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\n-define(NO_OF_NODES, 2).\n\nall() ->\n    [\n     test_age_inc,\n     test_pop_oldest,\n     test_cache_size,\n     test_select_data,\n     test_select_reply_data,\n     test_integrate_data\n    ].\n\n\nsuite() ->\n    [\n     {timetrap, {seconds, 20}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_testcase(_TestCase, Config) ->\n    %% monitor needs ring :-(\n    unittest_helper:make_ring(\n      ?NO_OF_NODES,\n      [{config, [\n                 {monitor_perf_interval, 0},  % deactivate monitor_perf\n                 {gossip_log_level_warn, info},\n                 {gossip_log_level_error, error}\n                ]\n       }]),\n    unittest_helper:wait_for_stable_ring_deep(),\n    monitor:proc_set_value(gossip_cyclon, 'shuffle', rrd:create(60 * 1000000, 3, counter)), % 60s monitoring interval\n    Group = pid_groups:group_with(gossip),\n    pid_groups:join_as(Group, gossip_cyclon_tester),\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Testcases\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Test, that the age field in the returned cache is incremented by select_data\ntest_age_inc(_Config) ->\n    tester:test(?MODULE, age_inc, 1, 250, []).\n\n-spec age_inc(gossip_cyclon:state()) -> true.\nage_inc(State) ->\n    State1 = {Cache1, _} = consolidate_nodes(State),\n    case gossip_cyclon:select_data(State1) of\n        {ok, {NewCache, _Node}} ->\n            %% check that the age was incremented\n            lists:all(fun({Node, Age}) ->\n                              {Node, OldAge} = lists:keyfind(Node, 1, Cache1),\n                              OldAge =:= Age - 1\n                      end, NewCache) andalso\n            length(Cache1) =:= length(NewCache) + 1;\n        {discard_msg, State2} ->\n            ?equals(State2, State1)\n    end.\n\n\n%% @doc Test, that the node sent as selected_peer by select_data is the oldest\n%%      node in the given cache.\ntest_pop_oldest(_Config) ->\n    tester:test(?MODULE, pop_oldest, 1, 250, []).\n\n-spec pop_oldest(gossip_cyclon:state()) -> true.\npop_oldest(State) ->\n    {Cache1, MyNode} = consolidate_nodes(State),\n\n    %% don't allow multiple entries with same age, otherwise the selection is non-deterministic\n    Cache2 = lists:foldl(fun({Node, Age}, AccIn) ->\n                                 case lists:keymember(Age, 2, AccIn) of\n                                     true -> AccIn;\n                                     false -> [{Node, Age}|AccIn]\n                                 end\n                         end, [], Cache1),\n\n    trace_mpath:start(test_pop_oldest, [{filter_fun, generate_message_filter(selected_peer, 1)}]),\n    Ret = gossip_cyclon:select_data({Cache2, MyNode}),\n    trace_mpath:stop(),\n    Result = case Ret of\n        {ok, _State2} ->\n            timer:sleep(5),\n            [SendEvent] = trace_mpath:get_trace(test_pop_oldest),\n            {Oldest, _Age} = lists:last(lists:keysort(2, Cache2)),\n            {selected_peer,{gossip_cyclon,default},{cy_cache,[SendNode]}} = element(6, SendEvent),\n            Oldest =:= SendNode;\n        {discard_msg, State2} ->\n            {Cache2, MyNode} =:= State2\n    end,\n    trace_mpath:cleanup(test_pop_oldest),\n    Result.\n\n\n%% @doc Test, that the cache size stays within the bounds given by the config\n%%      and the algorithm.\ntest_cache_size(_Config) ->\n    tester:test(?MODULE, check_cache_size, 5, 250, []).\n\n-spec check_cache_size(gossip_cyclon:data(), gossip_cyclon:data(), pos_integer(),\n                       non_neg_integer(), gossip_cyclon:state()) -> true.\ncheck_cache_size(PSubset, QSubset, Ref, Round, State) ->\n    PSubset1 = consolidate_nodes(PSubset),\n    QSubset1 = consolidate_nodes(QSubset),\n    State1 = consolidate_nodes(State),\n    ExpectedMaxCacheSize = config:read(gossip_cyclon_cache_size),\n    {_Tag, {NewCache1, _Node1}} = gossip_cyclon:select_data(State1),\n    {ok, {NewCache2, _Node2}} = gossip_cyclon:select_reply_data(PSubset1, Ref, Round, State1),\n    {ok, {NewCache3, _Node3}} = gossip_cyclon:integrate_data({QSubset1, PSubset1}, Round, State1),\n\n    length(NewCache1) =< ExpectedMaxCacheSize andalso\n    length(NewCache2) =< ExpectedMaxCacheSize andalso\n    length(NewCache3) =< ExpectedMaxCacheSize andalso\n    ((L1=length(NewCache1)) + 1 =:= (L2=length(element(1, State1))) orelse L1 =:= L2) andalso\n    length(NewCache2) >= length(element(1, State1)) andalso\n    length(NewCache3) >= length(element(1, State1)).\n\n\n%% @doc Test, that the data sent by select_data is a subset of the given Cache.\ntest_select_data(_Config) ->\n    tester:test(?MODULE, select_data, 1, 250, []).\n\n-spec select_data(gossip_cyclon:state()) -> true.\nselect_data(State) ->\n    {Cache1, MyNode} = consolidate_nodes(State),\n    trace_mpath:start(test_select_data,\n                      [{filter_fun, generate_message_filter(selected_data, 1)}]),\n    Ret = gossip_cyclon:select_data({Cache1, MyNode}),\n    trace_mpath:stop(),\n    Result = case Ret of\n        {ok, _} ->\n            timer:sleep(5),\n            [SendEvent] = trace_mpath:get_trace(test_select_data),\n            {selected_data,{gossip_cyclon,default},Subset} = element(6, SendEvent),\n            is_subset(Subset, [{MyNode, 0}|Cache1]);\n        {discard_msg, State2} ->\n            {Cache1, MyNode} =:= State2\n    end,\n    trace_mpath:cleanup(test_select_data),\n    Result.\n\n\n%% @doc Test, that the data sent by select_reply_data is a subset of the given Cache.\n%%      Also tests, that the cache returned by select_reply_data is a subset of\n%%      the union of the given Cache and the received Cache (implicit test of the\n%%      cache merging).\ntest_select_reply_data(_Config) ->\n    tester:test(?MODULE, select_reply_data, 4, 250, []).\n\n-spec select_reply_data(gossip_cyclon:data(), pos_integer(),non_neg_integer(),\n                        gossip_cyclon:state()) -> true.\nselect_reply_data(PSubset, Ref, Round, State) ->\n    State1 = {Cache1, _} = consolidate_nodes(State),\n    PSubset1 = consolidate_nodes(PSubset),\n\n    trace_mpath:start(test_select_reply_data,\n                      [{filter_fun, generate_message_filter(selected_reply_data, 1)}]),\n    {ok, {RetCache, _}} = gossip_cyclon:select_reply_data(PSubset1, Ref, Round, State1),\n    trace_mpath:stop(),\n    timer:sleep(5),\n    [SendEvent] = trace_mpath:get_trace(test_select_reply_data),\n    trace_mpath:cleanup(test_select_reply_data),\n    {selected_reply_data,{gossip_cyclon,default},{QSubset, PSubset1}, Ref, Round} = element(6, SendEvent),\n    is_subset(QSubset, Cache1) andalso\n    is_subset(RetCache, lists:usort(Cache1++PSubset1)).\n\n\n%% @doc Test, that the cache returned by integrate_data is a subset of the union\n%%      of the given Cache and the received Cache (implicit test of the cache merging).\ntest_integrate_data(_Config) ->\n    tester:test(?MODULE, integrate_data, 4, 250, []).\n\n-spec integrate_data(gossip_cyclon:data(), gossip_cyclon:data(), non_neg_integer(),\n                     gossip_cyclon:state()) -> true.\nintegrate_data(PSubset, QSubset, Round, State) ->\n    State1 = {Cache1, _} = consolidate_nodes(State),\n    PSubset1 = consolidate_nodes(PSubset),\n    QSubset1 = consolidate_nodes(QSubset),\n\n    {ok, {RetCache, _}} = gossip_cyclon:integrate_data(\n                                    {QSubset1, PSubset1}, Round, State1),\n    is_subset(RetCache, lists:usort(QSubset1 ++ Cache1)).\n\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Helper\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Ensure globally (i.e. for all pid's produced by this testsuite), that all\n%%      nodes with the same pid are identical in id and version id.\n%%      This prevents 'got two nodes with same IDversion but different ID' errors.\n%%      Uses the process dictionary to maintain state across calls.\n-spec consolidate_nodes(gossip_cyclon:data()) -> gossip_cyclon:data();\n                       (gossip_cyclon:state()) -> gossip_cyclon:state().\nconsolidate_nodes(Data) when is_list(Data) ->\n    Data1 = lists:map(fun({Node, Age}) -> {check_node(Node), Age} end, Data),\n    deduplicate(Data1);\n\nconsolidate_nodes(_State={Nodes, null}) ->\n    Nodes1 = consolidate_nodes(Nodes),\n    {Nodes1, null};\n\nconsolidate_nodes(_State={Nodes, MyNode}) ->\n    Nodes1 = consolidate_nodes(Nodes),\n    MyNode1 = check_node(MyNode),\n    {Nodes1, MyNode1}.\n\n\n%% @doc Check if a node with the same pid is already present in the process\n%%      dictionary.\n-spec check_node(node:node_type()) -> node:node_type().\ncheck_node(Node) ->\n    case get(Pid=node:pidX(Node)) of\n        undefined ->\n            put(Pid, Node), Node;\n        NodeFromPD ->\n            NodeFromPD\n    end.\n\n\n%% @doc Don't allow duplictes (i.e. a cache entry with the same pid (as ensured\n%%      by consolidate_node), but with different ages) within one cache.\n-spec deduplicate(cyclon_cache:cache()) -> cyclon_cache:cache().\ndeduplicate(Cache) ->\n    lists:foldl(fun(Entry={Node, _}, AccIn) ->\n                        case lists:keyfind(Node, 1, AccIn) of\n                            false -> [Entry|AccIn];\n                            {Node, _} -> AccIn\n                        end\n                end,\n                [], Cache).\n\n\n%% @doc Checks whether the given Subset is a subset of the given Cache.\n%%      Ignores the age field, as this might be updated by the age incrementation.\n-spec is_subset(cyclon_cache:cache(), cyclon_cache:cache()) -> boolean().\nis_subset(Subset, Cache) ->\n    lists:all(fun({Node, _Age}) -> lists:keymember(Node, 1, Cache) end, Subset).\n\n\n%% @doc Generate a filter function for trace_mpath.\n%%      Filters for send events with the given message tag at the given position\n%%      (within the message of a send event).\n-spec generate_message_filter(atom(), pos_integer()) ->\n    fun((trace_mpath:trace_event()) -> boolean()).\ngenerate_message_filter(Tag, Pos) ->\n    fun({log_send, _, _, _, _, Message, _}) ->\n            element(Pos, Message) =:= Tag;\n       (_) ->\n            false\n    end.\n\n"
  },
  {
    "path": "test/gossip_types_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{cases, tests, type_check_SUITE, [tester_type_check_gossip]}.\n"
  },
  {
    "path": "test/gossip_vivaldi_SUITE.erl",
    "content": "% @copyright 2010-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jens V. Fischer <jensvfischer@gmail.com>\n%% @doc Unit tests for src/vivaldi.erl\n%% @end\n%% @version $Id$\n-module(gossip_vivaldi_SUITE).\n-author('jensvfischer@gmail.com').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n\n\nall() ->\n    [\n     test_init,\n     test_get_coordinate,\n     test_select_node,\n     test_select_data,\n     test_select_reply_data,\n     test_update_coordinate\n    ].\n\n\nsuite() ->\n    [\n     {timetrap, {seconds, 10}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_testcase(TestCase, Config) ->\n    NeedsRing = [test_get_coordinate, test_select_reply_data],\n    case lists:member(TestCase, NeedsRing) of\n        true ->\n            unittest_helper:make_ring(2, [{config, [{monitor_perf_interval, 0},  % deactivate monitor_perf\n                                                    {gossip_vivaldi_interval, 100}\n\n                                                   ]}]),\n            unittest_helper:wait_for_stable_ring_deep(),\n            [{stop_ring, true} | Config];\n        false ->\n            unittest_helper:start_minimal_procs(Config, [], true)\n    end.\n\n\nend_per_testcase(_TestCase, Config) ->\n    {save_config, unittest_helper:stop_minimal_procs(Config)}.\n\ntest_get_coordinate(_Config) ->\n    pid_groups:join_as(pid_groups:group_with(gossip), ?MODULE),\n    gossip_vivaldi:get_coordinate(),\n    ?expect_message({vivaldi_get_coordinate_response, _Coordinate, _Confidence}).\n\n\ntest_init(_Config) ->\n    pid_groups:join_as(?MODULE, gossip),\n    ?expect_no_message(),\n    Ret = gossip_vivaldi:init([]),\n    ?equals_pattern(Ret, {ok, {_RandomCoordinate, 1.0}}),\n    ?expect_message({trigger_action, {gossip_vivaldi, default}}),\n    ?expect_no_message().\n\n\ntest_select_node(_Config) ->\n    Ret = gossip_vivaldi:select_node({[0.5, 0.5], 0.5}),\n    ?equals(Ret, {false, {[0.5, 0.5], 0.5}}).\n\n\ntest_select_data(_Config) ->\n    pid_groups:join_as(?MODULE, gossip),\n    Ret = gossip_vivaldi:select_data({[0.1, 0.1], 0.2}),\n    ?equals(Ret, {ok, {[0.1, 0.1], 0.2}}),\n    This = comm:this(),\n    ?expect_message({selected_data, {gossip_vivaldi, default},\n                     {This, [0.1, 0.1], 0.2}}).\n\n\ntest_select_reply_data(_Config) ->\n    config:write(gossip_vivaldi_count_measurements, 1),\n    config:write(gossip_vivaldi_measurements_delay, 0),\n\n    Data = {comm:this(), [0.0, 0.0], 0.77},\n    Ret = gossip_vivaldi:select_reply_data(Data, 1, 0, {[1.0, 1.0], 1.0}),\n    ?equals(Ret, {ok, {[1.0, 1.0], 1.0}}),\n\n    receive\n        {ping, SourcePid} ->\n            comm:send(SourcePid, {pong, gossip}, [{channel, prio}])\n    end,\n    ?expect_message({cb_msg, {gossip_vivaldi,default},\n                     {update_vivaldi_coordinate, _Latency, {[0.0, 0.0], 0.77}}}),\n\n    config:write(gossip_vivaldi_count_measurements, 10),\n    config:write(gossip_vivaldi_measurements_delay, 1000).\n\n\ntest_update_coordinate(_Config) ->\n    tester:test(?MODULE, update_coordinate, 7, 250, []).\n\n\n-spec update_coordinate(Float, Float, Float, number(), Float, Float, Float) -> true when\n      is_subtype(Float, float()).\nupdate_coordinate(Coord1x, Coord1y, Conf1, Latency, Coord2x, Coord2y, Conf2) ->\n    Coord1 = [Coord1x, Coord1y], Coord2 = [Coord2x, Coord2y],\n    config:write(gossip_vivaldi_dimensions, 2),\n    pid_groups:join_as(?MODULE, gossip),\n    Ret = gossip_vivaldi:handle_msg({update_vivaldi_coordinate,\n                                     Latency, {Coord1, Conf1}}, {Coord2, Conf2}),\n    ?expect_message({integrated_data, {gossip_vivaldi, default},  cur_round}),\n    case Latency == 0 of\n        true when Coord1 =/= Coord2 ->\n            ?expect_exception(gossip_vivaldi:update_coordinate(Coord1, Conf1, Latency, Coord2, Conf2),\n                              error, badarith);\n        _ ->\n            NewState = gossip_vivaldi:update_coordinate(Coord1, Conf1, Latency, Coord2, Conf2),\n            ?equals(Ret, {ok, NewState})\n    end.\n\n\n"
  },
  {
    "path": "test/histogram_SUITE.erl",
    "content": "%% @copyright 2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for src/histogram.erl\n%% @end\n%% @version $Id$\n-module(histogram_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"unittest.hrl\").\n\n-compile(export_all).\n\nall()   -> [\n            add2,\n            add3,\n            add2_identical,\n            add3_identical,\n            add_integer,\n            resize,\n            get_num_inserts,\n            find_smallest_interval,\n            merge_interval,\n            perf_add,\n            foldl_until,\n            foldr_until\n           ].\n\nsuite() -> [ {timetrap, {seconds, 40}} ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\nadd2(_Config) ->\n    H = histogram:create(10),\n    Values = [3.5, 3.0, 2.0, 1.0],\n    H2 = lists:foldl(fun histogram:add/2, H, Values),\n    ?equals(histogram:get_data(H2), [{1.0,1}, {2.0,1}, {3.0,1}, {3.5,1}]),\n    ok.\n\nadd3(_Config) ->\n    H = histogram:create(10),\n    Values = [{3.5, 2}, {3.0, 1}, {2.0, 5}, {1.0, 7}],\n    H2 = lists:foldl(fun ({Value, Count}, Histogram) ->\n                              histogram:add(Value, Count, Histogram)\n                     end, H, Values),\n    ?equals(histogram:get_data(H2), [{1.0, 7}, {2.0, 5}, {3.0, 1}, {3.5, 2}]),\n    ok.\n\nadd2_identical(_Config) ->\n    H = histogram:create(2),\n    Values = [1.25, 5.0, 5.0],\n    H2 = lists:foldl(fun histogram:add/2, H, Values),\n    ?equals(histogram:get_data(H2), [{1.25, 1}, {5.0, 2}]),\n    H3 = lists:foldl(fun histogram:add/2, H2, Values),\n    ?equals(histogram:get_data(H3), [{1.25, 2}, {5.0, 4}]),\n    ok.\n\nadd3_identical(_Config) ->\n    H = histogram:create(4),\n    Values = [{2.0, 1}, {3.5, 2}, {3.0, 1}, {2.0, 6}, {1.0, 3}],\n    H2 = lists:foldl(fun({Value, Count}, Histogram) ->\n                             histogram:add(Value, Count, Histogram)\n                     end,\n                     H, Values),\n    ?equals(histogram:get_data(H2), [{1.0, 3}, {2.0, 7}, {3.0, 1}, {3.5, 2}]),\n    ok.\n\nadd_integer(_Config) ->\n    H = histogram:create(5),\n    Values = [1, 2, 3, 4, 5],\n    H2 = lists:foldl(fun histogram:add/2, H, Values),\n    ?equals(histogram:get_data(H2), [{1, 1}, {2, 1}, {3, 1}, {4, 1}, {5, 1}]),\n    Values2 = [3, 4, 10],\n    H3 = lists:foldl(fun histogram:add/2, H2, Values2),\n    ?equals(histogram:get_data(H3), [{1, 2}, {3, 2}, {4, 2}, {5, 1}, {10, 1}]),\n    ok.\n\n\nresize(_Config) ->\n    H = histogram:create(3),\n    Values = [3.5, 3.0, 2.0, 1.0],\n    H2 = lists:foldl(fun histogram:add/2, H, Values),\n    ?equals(histogram:get_data(H2), [{1.0,1}, {2.0,1}, {3.25,2}]),\n    ok.\n\nget_num_inserts(_Config) ->\n    H = histogram:create(10),\n    ?equals(histogram:get_num_inserts(H), 0),\n    Values = [3.5, 3.0, 2.0, 1.0],\n    H2 = lists:foldl(fun histogram:add/2, H, Values),\n    ?equals(histogram:get_num_inserts(H2), 4),\n    Values2 = [{2.0, 1}, {3.5, 2}, {3.0, 1}, {2.0, 6}, {1.0, 3}],\n    H3 = lists:foldl(fun({Value, Count}, Histogram) ->\n                             histogram:add(Value, Count, Histogram)\n                     end,\n                     H2, Values2),\n    ?equals(histogram:get_num_inserts(H3), 17),\n    ok.\n\nfind_smallest_interval(_Config) ->\n    H1a = histogram:create(10),\n    Values1a = [3.5, 3.0, 2.0, 1.0],\n    H1b = lists:foldl(fun histogram:add/2, H1a, Values1a),\n    ?equals(3.0, histogram:find_smallest_interval(histogram:get_data(H1b))),\n    Values2a = [4.0, 2.5, 2.0, 1.0],\n    H2b = lists:foldl(fun histogram:add/2, H1a, Values2a),\n    ?equals(2.0, histogram:find_smallest_interval(histogram:get_data(H2b))),\n    ok.\n\nmerge_interval(_Config) ->\n    H = histogram:create(10),\n    Values = [3.5, 3.0, 2.0, 1.0],\n    H2 = lists:foldl(fun histogram:add/2, H, Values),\n    MinFirstValue = histogram:find_smallest_interval(histogram:get_data(H2)),\n    H3 = histogram:merge_interval(MinFirstValue, histogram:get_data(H2)),\n    ?equals(3.0, MinFirstValue),\n    ?equals(H3, [{1.0,1}, {2.0,1}, {3.25,2}]),\n    ok.\n\nperf_add(_Config) ->\n    Hist = histogram:create(10),\n    AddIntFun = fun(I, AccH) -> histogram:add(float(I), AccH) end,\n    Hist2 = performance_SUITE:iter2_foldl(performance_SUITE:count(), AddIntFun, Hist, \"histogram:add (1)\"),\n    _Hist3 = performance_SUITE:iter2_foldl(performance_SUITE:count(), AddIntFun, Hist2, \"histogram:add (2)\"),\n    ok.\n\nfoldl_until(_Config) ->\n    H1 = histogram:create(10),\n    Values = [1, 1, 2, 2, 3, 3, 4, 4, 4, 5, 5, 5],\n    H2 = lists:foldl(fun histogram:add/2, H1, Values),\n    %% sum found\n    {ok, Val, Sum} = histogram:foldl_until(6, H2),\n    ?equals(Val, 3),\n    ?equals(Sum, 6),\n    %% target value too high\n    {fail, Val2, Sum2} = histogram:foldl_until(1000, H2),\n    ?equals(Val2, 5),\n    ?equals(Sum2, 12),\n    %% target value too low\n    {ok, Val3, Sum3} = histogram:foldl_until(0, H2),\n    ?equals(Val3, nil),\n    ?equals(Sum3, 0),\n    ok.\n\nfoldr_until(_Config) ->\n    H1 = histogram:create(10),\n    Values = [1, 1, 2, 2, 3, 3, 4, 4, 4, 5, 5, 5],\n    H2 = lists:foldl(fun histogram:add/2, H1, Values),\n    %% sum found\n    {ok, Val, Sum} = histogram:foldr_until(6, H2),\n    ?equals(Val, 4),\n    ?equals(Sum, 6),\n    %% target value too high\n    {fail, Val2, Sum2} = histogram:foldr_until(1000, H2),\n    ?equals(Val2, 1),\n    ?equals(Sum2, 12),\n    %% target value too low\n    {ok, Val3, Sum3} = histogram:foldr_until(0, H2),\n    ?equals(Val3, nil),\n    ?equals(Sum3, 0),\n    ok."
  },
  {
    "path": "test/histogram_rt_SUITE.erl",
    "content": "%% @copyright 2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maximilian Michels <michels@zib.de>\n%% @doc    Unit tests specifically for histogram_rt in addition to the\n%%         tests in histogram_SUITE.\n%% @end\n%% @version $Id$\n-module(histogram_rt_SUITE).\n-author('michels@zib.de').\n-vsn('$Id$').\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\n-compile(export_all).\n\nall()    ->\n    [\n     add_keys,\n     merge_keys\n    ].\n\nsuite() -> [ {timetrap, {seconds, 30}} ].\n\n-define(EPSILON, 1.0e-7 * ?RT:n()).\n\ninit_per_suite(Config) ->\n    rt_SUITE:register_value_creator(),\n    unittest_helper:start_minimal_procs(Config, [], true).\n\nend_per_suite(Config) ->\n    rt_SUITE:unregister_value_creator(),\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\n\n-spec prop_add_keys(BaseKey::?RT:key(), Size::1..50, Values::[?RT:key(),...]) -> true.\nprop_add_keys(BaseKey, Size, Values0) ->\n    H = histogram_rt:create(Size, BaseKey),\n    Values = lists:sublist(Values0, Size),\n    H2 = lists:foldl(fun histogram_rt:add/2, H, Values),\n    %% check results\n    SortedValues = lists:sort(fun(Val, Val2) ->\n                                      succ_ord_key(Val, Val2, BaseKey)\n                              end, Values),\n    %ct:pal(\"BaseKey: ~p SortedValues: ~p Result: ~p\", [BaseKey, SortedValues, histogram_rt:get_data(H2)]),\n    Result = lists:map(fun(Value) -> {Value, 1} end, SortedValues),\n    ?compare(fun(Actual, Expected) ->\n                     check_result(Actual, Expected, BaseKey)\n             end, histogram_rt:get_data(H2), Result),\n    true.\n\nadd_keys(_Config) ->\n    prop_add_keys(?RT:hash_key(\"0\"), 17, [?RT:hash_key(\"0\")]),\n    tester:test(?MODULE, prop_add_keys, 3, 250, [{threads, 2}]).\n\n-spec prop_merge_keys(BaseKey::?RT:key(), Key1::?RT:key(), Key2::?RT:key()) -> true.\nprop_merge_keys(BaseKey, Key1, Key2) ->\n    %% insert two keys which will\n    H = histogram_rt:create(1, BaseKey),\n    H2 = histogram_rt:add(Key1, H),\n    H3 = histogram_rt:add(Key2, H2),\n    %% check results\n    SplitKey =\n        if Key1 =:= Key2 -> Key1;\n           true ->\n               case succ_ord_key(Key1, Key2, BaseKey) of\n                   true  -> ?RT:get_split_key(Key1, Key2, {1,2});\n                   false -> ?RT:get_split_key(Key2, Key1, {1,2})\n               end\n        end,\n    %ct:pal(\"Key1: ~p (Range: ~p) Key2: ~p (Range: ~p) BaseKey: ~p SplitKey: ~p Result: ~p, Raw: ~p\", [Key1, ?RT:get_range(Key1, BaseKey), Key2, ?RT:get_range(Key2, BaseKey), BaseKey, SplitKey, histogram_rt:get_data(H3), histogram:get_data(element(1, H3))]),\n    ?compare(fun(Actual, Expected) ->\n                     check_result(Actual, Expected, BaseKey)\n             end, histogram_rt:get_data(H3), [{SplitKey, 2}]),\n    true.\n\nmerge_keys(_Config) ->\n    tester:test(?MODULE, prop_merge_keys, 3, 250, [{threads, 2}]).\n\ncheck_result(Actual, Expected, BaseKey) ->\n    case rt_SUITE:default_rt_has_chord_keys() of\n        true  -> ?equals(Actual, Expected);\n        false -> check_elements(Actual, Expected, BaseKey)\n    end.\n\ncheck_elements([], [], _BaseKey) ->\n    true;\ncheck_elements([{El1, Count1} | Rest], [{El2, Count2} | Rest2], BaseKey) ->\n    %ct:pal(\"El: ~p, El2:~p\", [El1, El2]),\n    case nodelist:succ_ord_id(El1, El2, BaseKey) of\n        true -> Range = ?RT:get_range(El1, El2);\n        false -> Range = ?RT:get_range(El2, El1)\n    end,\n    %ct:pal(\"Range: ~p\", [Range]),\n    %ct:pal(\"Check: ~p\", [Range < ?EPSILON orelse El1 =:= El2]),\n    ?assert_w_note(Range < ?EPSILON orelse El1 =:= El2,\n                   {Range, '>=', ?EPSILON, ';', El1, '=/=', El2}),\n    ?equals(Count1, Count2),\n    check_elements(Rest, Rest2, BaseKey).\n\n%% @doc Like nodelist:succ_ord_id/3 but here the BaseKey is the biggest\n%% possible key and not the smallest.\n-spec succ_ord_key(K1::?RT:key(), K2::?RT:key(), BaseKey::?RT:key()) -> boolean().\nsucc_ord_key(K1, K2, BaseKey) ->\n    (K1 > BaseKey andalso K2 > BaseKey andalso K1 =< K2) orelse\n    (K1 < BaseKey andalso K2 =< BaseKey andalso K1 =< K2) orelse\n    (K1 > BaseKey andalso K2 =< BaseKey).\n"
  },
  {
    "path": "test/iblt_SUITE.erl",
    "content": "%  @copyright 2010-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <MLange@informatik.hu-berlin.de>\n%% @doc    Tests for iblt module (invertible bloom lookup table).\n%% @end\n-module(iblt_SUITE).\n-author('mlange@informatik.hu-berlin.de').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n-include(\"unittest.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nall() -> [\n          tester_insert,\n          tester_delete,\n          %tester_prime_vs_noprime,\n          tester_prime_option,\n          tester_list_entries\n         ].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 120}}\n    ].\n\ninit_per_suite(Config) ->\n    rt_SUITE:register_value_creator(),\n    unittest_helper:start_minimal_procs(Config, [], true).\n\nend_per_suite(Config) ->\n    rt_SUITE:unregister_value_creator(),\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_insert(10..100, ?RT:key(), client_version()) -> true.\nprop_insert(CellCount, Key, Value) ->\n    IBLT = iblt:new(get_hfs(), CellCount),\n    IBLT2 = iblt:insert(IBLT, Key, Value),\n    ?equals(iblt:get_prop(item_count, IBLT), 0),\n    ?equals(iblt:get_prop(item_count, IBLT2), 1),\n    ?equals(iblt:get(IBLT, Key), not_found),\n    ?equals(iblt:get(IBLT2, Key), Value).\n\ntester_insert(_) ->\n    tester:test(?MODULE, prop_insert, 3, 1000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_delete(10..100, ?RT:key(), client_version()) -> true.\nprop_delete(CellCount, Key, Value) ->\n    IBLT = iblt:new(get_hfs(), CellCount),\n    IBLT2 = iblt:insert(IBLT, Key, Value),\n    ?equals(iblt:get_prop(item_count, IBLT2), 1),\n    ?equals(iblt:get(IBLT2, Key), Value),\n    IBLT3 = iblt:delete(IBLT2, Key, Value),\n    ?equals(iblt:get_prop(item_count, IBLT3), 0),\n    ?equals(iblt:get(IBLT3, Key), not_found),\n    %detail check: count of every cell must be zero\n    {iblt, _, T, _, _, _} = IBLT3,\n    Count = lists:foldl(fun({_, Col}, Acc) ->\n                                Acc + lists:foldl(fun({C, _, _, _, _}, A) -> A+C end, 0, Col)\n                        end, 0, T),\n    ?equals(Count, 0).\n\ntester_delete(_) ->\n    tester:test(?MODULE, prop_delete, 3, 10000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ntester_prime_vs_noprime(_) ->\n    Calls = 10000,\n    Pid = spawn(?MODULE, collector, [{0, 0, 0, 0}, {0, 0}]),\n    register(pvp_iblt_test, Pid),\n    ct:pal(\"tester_prime_vs_noprime - Calls=~p\", [Calls]),\n    tester:test(?MODULE, prop_prime, 2, Calls, [{threads, 1}]),\n    pvp_iblt_test ! {reset, \"Prime-ColSize - COLLISIONS:\"},\n    tester:test(?MODULE, prop_prime, 2, Calls, [{threads, 1}]),\n    pvp_iblt_test ! {reset, \"Prime-ColSize - COLLISIONS:\"},\n    tester:test(?MODULE, prop_prime, 2, Calls, [{threads, 1}]),\n    pvp_iblt_test ! {reset, \"Prime-ColSize - COLLISIONS:\"},\n\n    tester:test(?MODULE, prop_noprime, 2, Calls, [{threads, 1}]),\n    pvp_iblt_test ! {reset, \"NON-Prime-ColSize - COLLISIONS:\"},\n    tester:test(?MODULE, prop_noprime, 2, Calls, [{threads, 1}]),\n    pvp_iblt_test ! {reset, \"NON-Prime-ColSize - COLLISIONS:\"},\n    tester:test(?MODULE, prop_noprime, 2, Calls, [{threads, 1}]),\n    pvp_iblt_test ! {reset, \"NON-Prime-ColSize - COLLISIONS:\"},\n    true.\n\n-spec prop_prime(10..100, [{?RT:key(), client_version()}]) -> true.\nprop_prime(Cells, Items) -> prop_get(Cells, Items, [prime]).\n\n-spec prop_noprime(10..100, [{?RT:key(), client_version()}]) -> true.\nprop_noprime(Cells, Items) -> prop_get(Cells, Items, []).\n\n-spec prop_get(10..100, [{?RT:key(), client_version()}], iblt:options()) -> true.\nprop_get(CellCount, Items, Options) ->\n    IBLT = lists:foldl(fun({Key, Ver}, _IBLT) ->\n                               iblt:insert(_IBLT, Key, Ver)\n                       end,\n                       iblt:new(get_hfs(), erlang:round(1.5 * CellCount), Options),\n                       Items),\n    ?equals(iblt:get_prop(item_count, IBLT), length(Items)),\n    Found = lists:foldl(fun({Key, _}, Sum) ->\n                                case iblt:get(IBLT, Key) of\n                                    not_found -> Sum;\n                                    _ -> Sum + 1\n                                end\n                        end, 0, Items),\n    _ = if\n            Found =:= 0 andalso length(Items) > 0 -> pvp_iblt_test ! count_zero;\n            Found > 0 andalso Found =:= length(Items) -> pvp_iblt_test ! count_all;\n            length(Items) > 0 -> pvp_iblt_test ! {some, Found, length(Items)};\n            true -> ok\n        end,\n    _ = pvp_iblt_test ! {found_sum, Found},\n    _ = pvp_iblt_test ! {items_sum, length(Items)},\n    true.\n\ncollector({ZeroFound, SomeFound, SomeAll, AllFound} = D, {FoundSum, ItemsSum} = A) ->\n    receive\n        count_zero -> collector({ZeroFound + 1, SomeFound, SomeAll, AllFound}, A);\n        count_all -> collector({ZeroFound, SomeFound, SomeAll, AllFound + 1}, A);\n        {found_sum, FS} -> collector(D, {FoundSum + FS, ItemsSum});\n        {items_sum, IS} -> collector(D, {FoundSum, ItemsSum + IS});\n        {some, Found, All} -> collector({ZeroFound, SomeFound + Found, SomeAll + All, AllFound}, A);\n        {reset, Title} -> ct:pal(\"~s: SumFound=~p/~p (~f%) | DETAILS:  ZeroFound=~p ; AllFound=~p ; Some=~p/~p (~f%)\",\n                                 [Title,\n                                  FoundSum, ItemsSum, 100*(FoundSum / ItemsSum),\n                                  ZeroFound, AllFound,\n                                  SomeFound, SomeAll, 100*(SomeFound / SomeAll)]),\n                          collector({0, 0, 0, 0}, {0, 0})\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_prime_option(10..5000) -> true.\nprop_prime_option(Cells) ->\n    PIBLT = iblt:new(get_hfs(), Cells, [prime]),\n    ?assert(prime:is_prime(iblt:get_prop(col_size, PIBLT))).\n\ntester_prime_option(_) ->\n    tester:test(?MODULE, prop_prime_option, 1, 20, [{threads, 2}]).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_list_entries(?RT:key(), ?RT:key(), 10..200) -> true.\nprop_list_entries(L, R, Count) ->\n    I = intervals:new('[', L, R, ']'),\n    DB = db_generator:get_db(I, Count, uniform, [{output, list_key_val}]),\n    IBLT = lists:foldl(fun({Key, Val}, Acc) -> iblt:insert(Acc, Key, Val) end,\n                       iblt:new(get_hfs(), erlang:round(1.5 * Count)),\n                       DB),\n    List = iblt:list_entries(IBLT),\n    Found = length(List),\n     ct:pal(\"--IBLT--\n             ItemsInserted=~p/~p\n             ListEntrySize=~p/~p (~f%)\n             FPR=~p\",\n            [iblt:get_prop(item_count, IBLT), Count,\n             Found, Count, (Found / Count) * 100,\n             iblt:get_fpr(IBLT)]),\n    ?assert(Found > 0 andalso Found =< Count).\n\ntester_list_entries(_) ->\n    tester:test(?MODULE, prop_list_entries, 3, 2, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nget_hfs() -> 5.\n"
  },
  {
    "path": "test/intervals_SUITE.erl",
    "content": "%  @copyright 2008-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc Unit tests for src/intervals.erl.\n%% @end\n%% @version $Id$\n-module(intervals_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\n-dialyzer({[no_opaque, no_return], [tc1/1, normalize/1]}).\n\ngroups() ->\n    [{hand_written_tests, [sequence], [\n                                       new, is_empty, intersection, tc1, \n                                       normalize, is_left_right_of]},\n\n     {tester_tests, [sequence], [\n     tester_empty_well_formed, tester_empty, tester_empty_continuous,\n     tester_all_well_formed, tester_all_continuous, tester_all,\n     tester_new1_well_formed, tester_new1, tester_new1_continuous,\n     tester_new1_bounds,\n     tester_new2_well_formed, tester_new2, tester_new2_continuous,\n     tester_new2_bounds,\n     tester_new4_well_formed, tester_new4, tester_new4_continuous,\n     tester_new4_bounds,\n     tester_from_elements1_well_formed,\n     tester_from_elements1,\n     tester_normalize_well_formed, tester_normalize,\n     tester_intersection_well_formed, tester_intersection,\n     tester_not_intersection, tester_not_intersection2,\n     tester_union2_well_formed, tester_union2, tester_not_union2,\n     tester_union1_well_formed, tester_union1, tester_not_union1,\n     tester_union2_same_as_union1, tester_union_continuous,\n     tester_is_adjacent, tester_is_adjacent_union,\n     tester_is_left_right_of,\n     tester_is_subset, tester_is_subset2, tester_is_double_subset_equality,\n     tester_minus_well_formed, tester_minus, tester_minus2, tester_minus3,\n     tester_not_minus, tester_not_minus2,\n     tester_get_bounds_continuous, tester_get_bounds, tester_get_elements,\n     tester_mk_from_node_ids_well_formed, tester_mk_from_node_ids, tester_mk_from_node_ids_continuous,\n     tester_mk_from_nodes,\n     tester_split_is_continuous, tester_split_is_well_formed, tester_split, split_bounds]}].\n\nall() ->\n    [\n     {group, tester_tests},\n     {group, hand_written_tests}\n].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 90}}\n    ].\n\ninit_per_suite(Config) ->\n    Config3 = unittest_helper:start_minimal_procs(Config, [], true),\n    tester:register_type_checker({typedef, intervals, interval, []}, intervals, is_well_formed),\n    tester:register_type_checker({typedef, intervals, continuous_interval, []}, intervals, is_continuous),\n    tester:register_value_creator({typedef, intervals, interval, []}, intervals, tester_create_interval, 1),\n    tester:register_value_creator({typedef, intervals, continuous_interval, []}, intervals, tester_create_continuous_interval, 4),\n    rt_SUITE:register_value_creator(),\n    Config3.\n\nend_per_suite(Config) ->\n    tester:unregister_type_checker({typedef, intervals, interval, []}),\n    tester:unregister_type_checker({typedef, intervals, continuous_interval, []}),\n    tester:unregister_value_creator({typedef, intervals, interval, []}),\n    tester:unregister_value_creator({typedef, intervals, continuous_interval, []}),\n    rt_SUITE:unregister_value_creator(),\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\nnew(_Config) ->\n    ?assert(intervals:is_well_formed(intervals:new('[', ?RT:hash_key(\"a\"), ?RT:hash_key(\"b\"), ']'))),\n    ?equals(intervals:new(?MINUS_INFINITY), intervals:new('[',?MINUS_INFINITY,?MINUS_INFINITY,']')).\n\nis_empty(_Config) ->\n    NotEmpty = intervals:new('[', ?RT:hash_key(\"a\"), ?RT:hash_key(\"b\"), ']'),\n    Empty = intervals:empty(),\n    ?assert(not intervals:is_empty(NotEmpty)),\n    ?assert(intervals:is_empty(Empty)).\n\nintersection(_Config) ->\n    NotEmpty = intervals:new('[', ?RT:hash_key(\"a\"), ?RT:hash_key(\"b\"), ']'),\n    Empty = intervals:empty(),\n    All = intervals:all(),\n    ?assert(intervals:is_empty(intervals:intersection(NotEmpty, Empty))),\n    ?assert(intervals:is_empty(intervals:intersection(Empty, NotEmpty))),\n    ?assert(intervals:is_empty(intervals:intersection(NotEmpty, Empty))),\n    ?assert(not intervals:is_empty(intervals:intersection(NotEmpty, NotEmpty))),\n    ?equals(intervals:intersection(NotEmpty, NotEmpty), NotEmpty),\n    ?equals(intervals:intersection(All, All), All),\n    ok.\n\ntc1(_Config) ->\n    % some tests that have once failed:\n    Key1 = ?RT:hash_key(\"b\"),\n    Key2 = ?RT:get_split_key(?MINUS_INFINITY, Key1, {1, 2}), % smaller than Key1\n    Key3 = ?RT:get_split_key(Key1, ?PLUS_INFINITY, {1, 2}), % larger than Key1\n    ?assert(intervals:is_subset(\n              intervals:union(\n                intervals:new('[', ?MINUS_INFINITY, Key2, ']'),\n                intervals:new('[', Key3, ?PLUS_INFINITY, ')')),\n              intervals:new('[', Key3, Key1, ']'))),\n\n    ?assert(intervals:is_subset(\n              intervals:union(intervals:new(Key1), intervals:new(Key3)),\n              intervals:new('[', Key1, Key3, ']'))),\n\n    ?equals(intervals:union([{'(', ?MINUS_INFINITY, ?PLUS_INFINITY,')'}],\n                            [{'[', ?MINUS_INFINITY, ?RT:hash_key(\"4\"), ']'}]),\n            intervals:all()),\n\n    ok.\n\nnormalize(_Config) ->\n    % some tests that have once failed:\n    % attention: manually defined intervals may have to be adapted to changes in the intervals module!\n    ?equals(intervals:tester_create_interval([{'[', ?MINUS_INFINITY, ?PLUS_INFINITY,')'}, {?MINUS_INFINITY}, {'[', ?PLUS_INFINITY,4,']'}, {'(', ?MINUS_INFINITY, ?PLUS_INFINITY, ')'}]),\n            intervals:all()),\n    \n    ?equals(intervals:tester_create_interval([{'[', ?MINUS_INFINITY, ?PLUS_INFINITY,')'}, {?MINUS_INFINITY}, {'[', ?PLUS_INFINITY,4,']'}]),\n            intervals:all()),\n    \n    ?equals(intervals:tester_create_interval([{'[', ?MINUS_INFINITY, ?PLUS_INFINITY,')'}, {'[', ?PLUS_INFINITY, 4, ']'}]),\n            intervals:all()),\n\n    ?equals(intervals:tester_create_interval([{'[',3,2,']'}, {'[',?PLUS_INFINITY,3,')'}, {'(',3,?PLUS_INFINITY,')'}]),\n            intervals:all()).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:empty/0, intervals:is_empty/1, intervals:in/2 and intervals:is_continuous/1\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_empty_well_formed() -> true.\nprop_empty_well_formed() ->\n    intervals:is_well_formed(intervals:empty()).\n\n-spec prop_empty_continuous() -> true.\nprop_empty_continuous() ->\n    not intervals:is_continuous(intervals:empty()).\n\n-spec prop_empty(intervals:key()) -> true.\nprop_empty(X) ->\n    ?assert(intervals:is_empty(intervals:empty())),\n    ?assert(not intervals:in(X, intervals:empty())).\n\ntester_empty_well_formed(_Config) ->\n    tester:test(?MODULE, prop_empty_well_formed, 0, 1, [{threads, 2}]).\n\ntester_empty_continuous(_Config) ->\n    tester:test(?MODULE, prop_empty_continuous, 0, 1, [{threads, 2}]).\n\ntester_empty(_Config) ->\n    tester:test(?MODULE, prop_empty, 1, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:all/0, intervals:is_all/1, intervals:in/2 and intervals:is_continuous/1\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_all_well_formed() -> true.\nprop_all_well_formed() ->\n    intervals:is_well_formed(intervals:all()).\n\n-spec prop_all_continuous() -> true.\nprop_all_continuous() ->\n    intervals:is_continuous(intervals:all()).\n\n-spec prop_all(intervals:key()) -> true.\nprop_all(X) ->\n    ?assert(intervals:is_all(intervals:all())),\n    ?assert(intervals:in(X, intervals:all())).\n\ntester_all_well_formed(_Config) ->\n    tester:test(?MODULE, prop_all_well_formed, 0, 1, [{threads, 2}]).\n\ntester_all_continuous(_Config) ->\n    tester:test(?MODULE, prop_all_continuous, 0, 1, [{threads, 2}]).\n\ntester_all(_Config) ->\n    tester:test(?MODULE, prop_all, 1, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:new/1, intervals:in/2 and intervals:is_continuous/1\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_new1_well_formed(intervals:key()) -> true.\nprop_new1_well_formed(X) ->\n    intervals:is_well_formed(intervals:new(X)).\n\n-spec prop_new1_continuous(intervals:key()) -> true.\nprop_new1_continuous(X) ->\n    intervals:is_continuous(intervals:new(X)).\n\n-spec prop_new1_bounds(intervals:key()) -> true.\nprop_new1_bounds(X) ->\n    ?equals(intervals:get_bounds(intervals:new(X)), {'[', X, X, ']'}).\n\n-spec prop_new1(intervals:key()) -> true.\nprop_new1(X) ->\n    ?equals(intervals:new(X), intervals:new('[', X, X, ']')),\n    ?assert(intervals:in(X, intervals:new(X))),\n    ?assert(not intervals:is_empty(intervals:new(X))).\n\ntester_new1_well_formed(_Config) ->\n    tester:test(?MODULE, prop_new1_well_formed, 1, 5000, [{threads, 2}]).\n\ntester_new1_continuous(_Config) ->\n    tester:test(?MODULE, prop_new1_continuous, 1, 5000, [{threads, 2}]).\n\ntester_new1_bounds(_Config) ->\n    tester:test(?MODULE, prop_new1_bounds, 1, 5000, [{threads, 2}]).\n\ntester_new1(_Config) ->\n    tester:test(?MODULE, prop_new1, 1, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:new/4, intervals:in/2 and intervals:is_continuous/1\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_new2_well_formed(intervals:key(), intervals:key()) -> true.\nprop_new2_well_formed(X, Y) ->\n    intervals:is_well_formed(intervals:new('[', X, Y, ']')).\n\n-spec prop_new2_continuous(intervals:key(), intervals:key()) -> true.\nprop_new2_continuous(X, Y) ->\n    intervals:is_continuous(intervals:new('[', X, Y, ']')).\n\n-spec prop_new2_bounds(intervals:key(), intervals:key()) -> true.\nprop_new2_bounds(X, Y) ->\n    I = intervals:new('[', X, Y, ']'),\n    case intervals:is_all(I) of\n        false -> ?equals(intervals:get_bounds(I), {'[', X, Y, ']'});\n        true  -> ?equals(intervals:get_bounds(I), {'[', ?MINUS_INFINITY, ?PLUS_INFINITY, ')'})\n    end.\n\n-spec prop_new2(intervals:key(), intervals:key()) -> true.\nprop_new2(X, Y) ->\n    % check deterministic behavior\n    A = intervals:new('[', X, Y, ']'),\n    B = intervals:new('[', X, Y, ']'),\n    ?equals(A, B),\n    ?assert(intervals:in(X, A)),\n    ?assert(intervals:in(Y, A)),\n    ?assert(not intervals:is_empty(A)).\n\ntester_new2_well_formed(_Config) ->\n    tester:test(?MODULE, prop_new2_well_formed, 2, 5000, [{threads, 2}]).\n\ntester_new2_continuous(_Config) ->\n    tester:test(?MODULE, prop_new2_continuous, 2, 5000, [{threads, 2}]).\n\ntester_new2_bounds(_Config) ->\n    tester:test(?MODULE, prop_new2_bounds, 2, 5000, [{threads, 2}]).\n\ntester_new2(_Config) ->\n    tester:test(?MODULE, prop_new2, 2, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:new/4, intervals:in/2 and intervals:is_continuous/1\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_new4_well_formed(intervals:left_bracket(), intervals:key(), intervals:key(), intervals:right_bracket()) -> true.\nprop_new4_well_formed(XBr, X, Y, YBr) ->\n    intervals:is_well_formed(intervals:new(XBr, X, Y, YBr)).\n\n-spec prop_new4_continuous(intervals:left_bracket(), intervals:key(), intervals:key(), intervals:right_bracket()) -> true.\nprop_new4_continuous(XBr, X, Y, YBr) ->\n    Interval = intervals:new(XBr, X, Y, YBr),\n    ?implies(not intervals:is_empty(Interval), intervals:is_continuous(Interval)).\n\n-spec prop_new4_bounds(intervals:left_bracket(), intervals:key(), {intervals:key(), intervals:right_bracket()} | {plus_infinity, ')'}) -> true.\nprop_new4_bounds(XBr, X, {Y_, YBr}) ->\n    Y = case Y_ of\n            plus_infinity -> ?PLUS_INFINITY; % workaround if this is not precise enough for a type def\n            _             -> Y_\n        end,\n    I = intervals:new(XBr, X, Y, YBr),\n    % convert brackets if ?MINUS_INFINITY or ?PLUS_INFINITY are involved\n    % (they will be converted due to normalization)\n    {Ye, YBre} = case {Y, YBr} of\n                     {?MINUS_INFINITY, ')'} -> {?PLUS_INFINITY, ')'};\n                     _                      -> {Y, YBr}\n                 end,\n    {Xe, XBre} = case {X, XBr} of\n                     {?PLUS_INFINITY, '('} -> {?MINUS_INFINITY, '['};\n                     _                     -> {X, XBr}\n                 end,\n    case intervals:is_empty(I) of\n        true -> true;\n        false ->\n            case intervals:is_all(I) of\n                false -> ?equals(intervals:get_bounds(I), {XBre, Xe, Ye, YBre});\n                true  -> ?equals(intervals:get_bounds(I), {'[', ?MINUS_INFINITY, ?PLUS_INFINITY, ')'})\n            end,\n            true\n    end.\n\n-spec prop_new4(intervals:left_bracket(), intervals:key(), intervals:key(), intervals:right_bracket()) -> true.\nprop_new4(XBr, X, Y, YBr) ->\n    I = intervals:new(XBr, X, Y, YBr),\n    case (X =:= Y andalso (XBr =:= '(' orelse YBr =:= ')')) orelse\n             ({XBr, X, Y, YBr} =:= {'(', ?PLUS_INFINITY, ?MINUS_INFINITY, ')'}) of\n        true  -> ?assert(intervals:is_empty(I));\n        false -> \n            ?assert(not intervals:is_empty(I)),\n            ?equals(?implies(XBr =:= '[', intervals:in(X, I)), true),\n            ?equals(?implies(XBr =:= '(', not intervals:in(X, I)), true),\n            ?equals(?implies(YBr =:= ']' andalso Y =/= ?PLUS_INFINITY, intervals:in(Y, I)), true),\n            ?equals(?implies(YBr =:= ')' orelse Y =:= ?PLUS_INFINITY, not intervals:in(Y, I)), true)\n    end.\n\ntester_new4_well_formed(_Config) ->\n    tester:test(?MODULE, prop_new4_well_formed, 4, 5000, [{threads, 2}]).\n\ntester_new4_continuous(_Config) ->\n    tester:test(?MODULE, prop_new4_continuous, 4, 5000, [{threads, 2}]).\n\ntester_new4_bounds(_Config) ->\n    tester:test(?MODULE, prop_new4_bounds, 3, 5000, [{threads, 2}]).\n\ntester_new4(_Config) ->\n    tester:test(?MODULE, prop_new4, 4, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:from_elements/1 and intervals:in/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_from_elements1_well_formed([intervals:key()]) -> true.\nprop_from_elements1_well_formed(X) ->\n    intervals:is_well_formed(intervals:from_elements(X)).\n\n-spec prop_from_elements1([intervals:key()]) -> true.\nprop_from_elements1(X) ->\n    I = intervals:from_elements(X),\n    ?equals(?implies(X =/= [], not intervals:is_empty(I)), true),\n    _ = [?equals(intervals:in(K, I), true) || K <- X],\n    true.\n\ntester_from_elements1_well_formed(_Config) ->\n    tester:test(?MODULE, prop_from_elements1_well_formed, 1, 5000, [{threads, 2}]).\n\ntester_from_elements1(_Config) ->\n    tester:test(?MODULE, prop_from_elements1, 1, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:tester_create_interval/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_normalize_well_formed(intervals:invalid_interval()) -> true.\nprop_normalize_well_formed(Is) ->\n    intervals:is_well_formed(intervals:tester_create_interval(Is)).\n\n-spec prop_normalize(intervals:interval(), intervals:key()) -> true.\nprop_normalize(I, X) ->\n    intervals:in(X, I) =:= intervals:in(X, intervals:tester_create_interval(I)).\n\ntester_normalize_well_formed(_Config) ->\n    tester:test(?MODULE, prop_normalize_well_formed, 1, 5000, [{threads, 2}]).\n\ntester_normalize(_Config) ->\n    tester:test(?MODULE, prop_normalize, 2, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:intersection/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_intersection_well_formed(intervals:interval(), intervals:interval()) -> true.\nprop_intersection_well_formed(A, B) ->\n    intervals:is_well_formed(intervals:intersection(A, B)).\n\n-spec prop_intersection(intervals:interval(), intervals:interval(), intervals:key()) -> true.\nprop_intersection(A, B, X) ->\n    ?implies(intervals:in(X, A) andalso intervals:in(X, B),\n             intervals:in(X, intervals:intersection(A, B))).\n\n-spec prop_not_intersection(intervals:interval(), intervals:interval(), intervals:key()) -> true.\nprop_not_intersection(A, B, X) ->\n    ?implies(intervals:in(X, A) xor intervals:in(X, B),\n             not intervals:in(X, intervals:intersection(A, B))).\n\n-spec prop_not_intersection2(intervals:interval(), intervals:interval(), intervals:key()) -> true.\nprop_not_intersection2(A, B, X) ->\n    ?implies(not intervals:in(X, A) andalso not intervals:in(X, B),\n             not intervals:in(X, intervals:intersection(A, B))).\n\ntester_intersection_well_formed(_Config) ->\n    tester:test(?MODULE, prop_intersection_well_formed, 2, 5000, [{threads, 2}]).\n\ntester_intersection(_Config) ->\n    tester:test(?MODULE, prop_intersection, 3, 5000, [{threads, 2}]).\n\ntester_not_intersection(_Config) ->\n    tester:test(?MODULE, prop_not_intersection, 3, 5000, [{threads, 2}]).\n\ntester_not_intersection2(_Config) ->\n    tester:test(?MODULE, prop_not_intersection2, 3, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:union/2, intervals:union/1 and intervals:is_continuous/1\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_union2_well_formed(intervals:interval(), intervals:interval()) -> true.\nprop_union2_well_formed(A, B) ->\n    intervals:is_well_formed(intervals:union(A, B)).\n\n-spec prop_union1_well_formed([intervals:interval()]) -> true.\nprop_union1_well_formed(List) ->\n    intervals:is_well_formed(intervals:union(List)).\n\n-spec prop_union2(intervals:interval(), intervals:interval(), intervals:key()) -> true.\nprop_union2(A, B, X) ->\n    ?implies(intervals:in(X, A) orelse intervals:in(X, B),\n             intervals:in(X, intervals:union(A, B))).\n\n-spec prop_union1([intervals:interval()], intervals:key()) -> true.\nprop_union1(List, X) ->\n    ?implies(lists:any(fun(I) -> intervals:in(X, I) end, List),\n             intervals:in(X, intervals:union(List))).\n\n-spec prop_not_union2(intervals:interval(), intervals:interval(), intervals:key()) -> true.\nprop_not_union2(A, B, X) ->\n    ?implies(not intervals:in(X, A) andalso not intervals:in(X, B),\n             not intervals:in(X, intervals:union(A, B))).\n\n-spec prop_not_union1([intervals:interval()], intervals:key()) -> true.\nprop_not_union1(List, X) ->\n    ?implies(lists:all(fun(I) -> not intervals:in(X, I) end, List),\n             not intervals:in(X, intervals:union(List))).\n\n-spec prop_union2_same_as_union1(intervals:interval(), intervals:interval()) -> true.\nprop_union2_same_as_union1(A, B) ->\n    ?equals(intervals:union(A, B), intervals:union([A, B])).\n\n-spec prop_union_continuous(A0Br::intervals:left_bracket(), A0::intervals:key(), A1::intervals:key(), A1Br::intervals:right_bracket(),\n                            B0Br::intervals:left_bracket(), B0::intervals:key(), B1::intervals:key(), B1Br::intervals:right_bracket()) -> true.\nprop_union_continuous(A0Br, A0, A1, A1Br, B0Br, B0, B1, B1Br) ->\n    A = intervals:new(A0Br, A0, A1, A1Br),\n    B = intervals:new(B0Br, B0, B1, B1Br),\n    ContinuousUnion = intervals:is_continuous(intervals:union(A, B)),\n    % list conditions when union(A,B) is continuous\n    C1 = A =:= B andalso intervals:is_continuous(A),\n    C2 = intervals:is_empty(A) andalso intervals:is_continuous(B),\n    C3 = intervals:is_empty(B) andalso intervals:is_continuous(A),\n    % if the (continuous) intervals overlap, then their union is continuous:\n    % (note: if both are empty, none of the following conditions is true)\n    C4 = intervals:in(B0, A) orelse intervals:in(B1, A) orelse\n             intervals:in(A0, B) orelse intervals:in(A1, B),\n    % unions of continuous intervals wrapping around are continuous\n    C5 = intervals:is_continuous(A) andalso intervals:is_continuous(B) andalso\n             {A1, A1Br, B0, B0Br} =:= {?PLUS_INFINITY, ')', ?MINUS_INFINITY, '['},\n    C6 = intervals:is_continuous(A) andalso intervals:is_continuous(B) andalso\n             {B1, B1Br, A0, A0Br} =:= {?PLUS_INFINITY, ')', ?MINUS_INFINITY, '['},\n    % if any C* is true, the union must be continuous, if all are false it is not\n    % note: split this into several checks for better error messages\n    ?equals(?implies(C1, ContinuousUnion), true),\n    ?equals(?implies(C2, ContinuousUnion), true),\n    ?equals(?implies(C3, ContinuousUnion), true),\n    ?equals(?implies(C4, ContinuousUnion), true),\n    ?equals(?implies(C5, ContinuousUnion), true),\n    ?equals(?implies(C6, ContinuousUnion), true),\n    case intervals:is_continuous(intervals:union(A, B)) of\n        false -> ?equals(C1, false),\n                 ?equals(C2, false),\n                 ?equals(C3, false),\n                 ?equals(C4, false),\n                 ?equals(C5, false),\n                 ?equals(C6, false),\n                 true;\n        true  -> true\n    end.\n\ntester_union2_well_formed(_Config) ->\n    tester:test(?MODULE, prop_union2_well_formed, 2, 5000, [{threads, 2}]).\n\ntester_union1_well_formed(_Config) ->\n    tester:test(?MODULE, prop_union1_well_formed, 1, 5000, [{threads, 2}]).\n\ntester_union2(_Config) ->\n    tester:test(?MODULE, prop_union2, 3, 5000, [{threads, 2}]).\n\ntester_union1(_Config) ->\n    tester:test(?MODULE, prop_union1, 2, 5000, [{threads, 2}]).\n\ntester_not_union2(_Config) ->\n    tester:test(?MODULE, prop_not_union2, 3, 5000, [{threads, 2}]).\n\ntester_not_union1(_Config) ->\n    tester:test(?MODULE, prop_not_union1, 2, 5000, [{threads, 2}]).\n\ntester_union2_same_as_union1(_Config) ->\n    tester:test(?MODULE, prop_union2_same_as_union1, 2, 5000, [{threads, 2}]).\n\ntester_union_continuous(_Config) ->\n    tester:test(?MODULE, prop_union_continuous, 8, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:is_adjacent/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ntester_is_adjacent(_Config) ->\n    tester:test(?MODULE, prop_is_adjacent, 8, 5000, [{threads, 2}]).\n\n-spec(prop_is_adjacent(A0Br::intervals:left_bracket(), A0::intervals:key(), A1::intervals:key(), A1Br::intervals:right_bracket(),\n                       B0Br::intervals:left_bracket(), B0::intervals:key(), B1::intervals:key(), B1Br::intervals:right_bracket()) -> boolean()).\nprop_is_adjacent(A0Br, A0, A1, A1Br, B0Br, B0, B1, B1Br) ->\n    A = intervals:new(A0Br, A0, A1, A1Br),\n    B = intervals:new(B0Br, B0, B1, B1Br),\n    IsAdjacent =\n        not intervals:is_empty(A) andalso\n        not intervals:is_empty(B) andalso\n        intervals:is_empty(intervals:intersection(A, B)) andalso\n            (% real intervals, a border is in one interval but not the other:\n             {A0Br, A0, B1Br} =:= {'(', B1, ']'} orelse\n             {A0Br, A0, B1Br} =:= {'[', B1, ')'} orelse\n             {B0Br, B0, A1Br} =:= {'(', A1, ']'} orelse\n             {B0Br, B0, A1Br} =:= {'[', A1, ')'} orelse\n             % one interval is an element:\n             {A, B0Br, B0} =:= {intervals:new(A0), '(', A0} orelse\n             {A, B1Br, B1} =:= {intervals:new(A0), ')', A0} orelse\n             {A, B0Br, B0} =:= {intervals:new(A1), '(', A1} orelse\n             {A, B1Br, B1} =:= {intervals:new(A1), ')', A1} orelse\n             % special cases for intervals with ?PLUS_INFINITY or ?MINUS_INFINITY\n             % (cases with A0=:=B1 or A1=:=B0 have already been handled above)\n             {A0Br, A0, B1, B1Br} =:= {'[', ?MINUS_INFINITY, ?PLUS_INFINITY, ')'} orelse\n             {B0Br, B0, A1, A1Br} =:= {'[', ?MINUS_INFINITY, ?PLUS_INFINITY, ')'} orelse\n             {A0Br, A0, B1, B1Br} =:= {'(', ?PLUS_INFINITY, ?MINUS_INFINITY, ')'} orelse\n             {B0Br, B0, A1, A1Br} =:= {'(', ?PLUS_INFINITY, ?MINUS_INFINITY, ')'} orelse\n             {A, B} =:= {intervals:new(?MINUS_INFINITY), intervals:new(?PLUS_INFINITY)} orelse\n             {B, A} =:= {intervals:new(?MINUS_INFINITY), intervals:new(?PLUS_INFINITY)}),\n    IsAdjacent =:= intervals:is_adjacent(A, B).\n\ntester_is_adjacent_union(_Config) ->\n    tester:test(?MODULE, prop_is_adjacent_union, 8, 5000, [{threads, 2}]).\n\n-spec prop_is_adjacent_union(A0Br::intervals:left_bracket(), A0::intervals:key(), A1::intervals:key(), A1Br::intervals:right_bracket(),\n                             B0Br::intervals:left_bracket(), B0::intervals:key(), B1::intervals:key(), B1Br::intervals:right_bracket()) -> true.\nprop_is_adjacent_union(A0Br, A0, A1, A1Br, B0Br, B0, B1, B1Br) ->\n    A = intervals:new(A0Br, A0, A1, A1Br),\n    B = intervals:new(B0Br, B0, B1, B1Br),\n    ?implies(intervals:is_adjacent(A, B), intervals:is_continuous(intervals:union(A, B))).\n\n-spec(prop_is_left_right_of(A0Br::intervals:left_bracket(), A0::intervals:key(),\n                            A1::intervals:key(), A1Br::intervals:right_bracket(),\n                            B0Br::intervals:left_bracket(), B0::intervals:key(),\n                            B1::intervals:key(), B1Br::intervals:right_bracket()) -> true).\nprop_is_left_right_of(A0Br, A0, A1, A1Br, B0Br, B0, B1, B1Br) ->\n    A = intervals:new(A0Br, A0, A1, A1Br),\n    B = intervals:new(B0Br, B0, B1, B1Br),\n    ?equals(?implies(intervals:is_left_of(A, B), intervals:is_empty(intervals:minus(intervals:new(A0Br, A0, B1, B1Br), intervals:union(A, B)))), true),\n    ?equals(?implies(intervals:is_right_of(A, B), intervals:is_empty(intervals:minus(intervals:new(B0Br, B0, A1, A1Br), intervals:union(A, B)))), true),\n    ?equals(intervals:is_adjacent(A, B), intervals:is_left_of(A, B) orelse intervals:is_right_of(A, B)).\n\ntester_is_left_right_of(_Config) ->\n    tester:test(intervals_SUITE, prop_is_left_right_of, 8, 5000, [{threads, 2}]).\n\nis_left_right_of(_Config) ->\n    MiddleKey = ?RT:hash_key(\"17\"),\n    X = intervals:new('[', MiddleKey, ?PLUS_INFINITY, ')'),\n    Y = intervals:new('[', ?MINUS_INFINITY, MiddleKey, ')'),\n    ?equals(intervals:is_adjacent(X, Y), true), % @17\n    ?equals(intervals:is_left_of(X, Y), true),\n    ?equals(intervals:is_left_of(Y, X), true),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:is_subset/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_is_subset(intervals:interval(), intervals:interval()) -> true.\nprop_is_subset(X, Y) ->\n    % Y covers X iff X \\cup Y covers X\n    intervals:is_subset(X, Y) =:= intervals:is_subset(X, intervals:intersection(X, Y)).\n\ntester_is_subset(_Config) ->\n    tester:test(?MODULE, prop_is_subset, 2, 5000, [{threads, 2}]).\n\n-spec prop_is_subset2(intervals:interval(), intervals:interval()) -> true.\nprop_is_subset2(X, Y) ->\n    Z = intervals:intersection(X, Y),\n    % X as well as Y cover X \\cup Y\n    intervals:is_subset(Z, X) andalso intervals:is_subset(Z, Y).\n\ntester_is_subset2(_Config) ->\n    tester:test(?MODULE, prop_is_subset2, 2, 5000, [{threads, 2}]).\n\n-spec prop_is_double_subset_equality(intervals:interval(), intervals:interval()) -> true.\nprop_is_double_subset_equality(X, Y) ->\n    (X =:= Y) =:= (intervals:is_subset(X, Y) andalso intervals:is_subset(Y, X)).\n\ntester_is_double_subset_equality(_Config) ->\n    tester:test(?MODULE, prop_is_double_subset_equality, 2, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:minus/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_minus_well_formed(intervals:interval(), intervals:interval()) -> true.\nprop_minus_well_formed(A, B) ->\n    intervals:is_well_formed(intervals:minus(A, B)).\n\n-spec prop_minus(intervals:interval(), intervals:interval(), intervals:key()) -> true.\nprop_minus(A, B, X) ->\n    ?implies(intervals:in(X, A) andalso not intervals:in(X, B),\n             intervals:in(X, intervals:minus(A, B))).\n\n-spec prop_minus2(intervals:left_bracket(), intervals:key(), intervals:key(), intervals:right_bracket()) -> true.\nprop_minus2(XBr, X, Y, YBr) ->\n    I = intervals:new(XBr, X, Y, YBr),\n    I_inv = case intervals:is_empty(I) of\n                true  -> intervals:all();\n                false ->\n                    case I =:= intervals:new(X) of\n                        true  -> intervals:union(\n                                   intervals:new('[', ?MINUS_INFINITY, X, ')'),\n                                   intervals:new('(', X, ?PLUS_INFINITY, ')'));\n                        false -> intervals:new(switch_br2(YBr), Y, X, switch_br2(XBr))\n                    end\n            end,\n    ?equals(intervals:minus(intervals:all(), I), I_inv).\n\n-spec prop_minus3(intervals:interval()) -> true.\nprop_minus3(A) ->\n    A_inv = intervals:minus(intervals:all(), A),\n    ?equals(intervals:union(A, A_inv), intervals:all()).\n\n-spec prop_not_minus(intervals:interval(), intervals:interval(), intervals:key()) -> true.\nprop_not_minus(A, B, X) ->\n    ?implies(intervals:in(X, B),\n             not intervals:in(X, intervals:minus(A, B))).\n\n-spec prop_not_minus2(intervals:interval(), intervals:interval(), intervals:key()) -> true.\nprop_not_minus2(A, B, X) ->\n    ?implies(not intervals:in(X, A),\n             not intervals:in(X, intervals:minus(A, B))).\n\ntester_minus_well_formed(_Config) ->\n    tester:test(?MODULE, prop_minus_well_formed, 2, 5000, [{threads, 2}]).\n\ntester_minus(_Config) ->\n    tester:test(?MODULE, prop_minus, 3, 5000, [{threads, 2}]).\n\ntester_minus2(_Config) ->\n    tester:test(?MODULE, prop_minus2, 4, 5000, [{threads, 2}]).\n\ntester_minus3(_Config) ->\n    tester:test(?MODULE, prop_minus3, 1, 5000, [{threads, 2}]).\n\ntester_not_minus(_Config) ->\n    tester:test(?MODULE, prop_not_minus, 3, 5000, [{threads, 2}]).\n\ntester_not_minus2(_Config) ->\n    tester:test(?MODULE, prop_not_minus2, 3, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:get_bounds/1\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_get_bounds_continuous(intervals:interval()) -> true.\nprop_get_bounds_continuous(I) ->\n    ?implies(intervals:is_continuous(I),\n             case intervals:get_bounds(I) of\n                 {'(', Key, Key, ')'} -> I =:= intervals:union(intervals:new('(', Key, ?PLUS_INFINITY, ')'),\n                                                               intervals:new('[', ?MINUS_INFINITY, Key, ')'));\n                 {LBr, L, R, RBr}     -> I =:= intervals:new(LBr, L, R, RBr)\n             end).\n\n-spec prop_get_bounds(intervals:interval()) -> true.\nprop_get_bounds(I) ->\n    ?implies(not intervals:is_empty(I),\n             begin\n                 BoundsInterval =\n                     case intervals:get_bounds(I) of\n                         {'(', Key, Key, ')'} -> \n                             intervals:union(intervals:new('(', Key, ?PLUS_INFINITY, ')'),\n                                             intervals:new('[', ?MINUS_INFINITY, Key, ')'));\n                         {LBr, L, R, RBr} ->\n                             intervals:new(LBr, L, R, RBr)\n                     end,\n                 intervals:is_subset(I, BoundsInterval)\n             end).\n\ntester_get_bounds_continuous(_Config) ->\n    tester:test(?MODULE, prop_get_bounds_continuous, 1, 5000, [{threads, 2}]).\n\ntester_get_bounds(_Config) ->\n    tester:test(?MODULE, prop_get_bounds, 1, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:get_elements/1\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_get_elements(intervals:interval()) -> true.\nprop_get_elements(I) ->\n    {Elements, RestInt} = intervals:get_elements(I),\n    ?equals(lists:foldl(fun(E, Acc) -> intervals:union(intervals:new(E), Acc) end, RestInt, Elements), I).\n\ntester_get_elements(_Config) ->\n    tester:test(?MODULE, prop_get_elements, 1, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intervals:split/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_split_is_well_formed(intervals:continuous_interval(), 1..1000) -> true.\nprop_split_is_well_formed(I, Parts) ->\n    S = intervals:split(I, Parts),\n    lists:foldl(fun(SubI, Acc) -> Acc andalso intervals:is_well_formed(SubI) end, true, S).\n\n-spec prop_split_is_continuous(intervals:continuous_interval(), 1..1000) -> true.\nprop_split_is_continuous(I, Parts) ->\n    S = intervals:split(I, Parts),\n    lists:foldl(fun(SubI, Acc) -> \n                        Acc andalso ?implies(not intervals:is_empty(SubI), intervals:is_continuous(SubI)) \n                end, true, S).    \n\n%% @doc Checks that each split interval is a subset of the original,\n%%      the union of all is the original, split-off intervals do not overlap.\n-spec prop_split(intervals:continuous_interval(), 1..1000) -> true.\nprop_split(I, Parts) ->\n    S = intervals:split(I, Parts),\n    lists:foreach(fun(SubI) -> ?equals(intervals:is_subset(SubI, I), true) end, S),\n    ?equals(lists:foldl(fun(SubI, UI) -> intervals:union(SubI, UI) end, intervals:empty(), S), I),\n    S_nonempty = [X || X <- S, not intervals:is_empty(X)],\n    _ = lists:foldl(fun(SubI, LastI) -> \n                            ?assert(intervals:is_adjacent(SubI, LastI)),\n                            SubI \n                    end, hd(S_nonempty), tl(S_nonempty)),\n    UniqueRanges = gb_sets:from_list(\n                     [begin\n                          {_, Begin, End, _} = intervals:get_bounds(X),\n                          ?RT:get_range(Begin, End)\n                      end || X <- S_nonempty]),\n    case gb_sets:fold(fun(X, Acc) -> Acc andalso is_integer(X) end, true, UniqueRanges) of\n        true ->\n            ?assert_w_note(lists:member(gb_sets:size(UniqueRanges), [1, 2]),\n                           io_lib:format(\"UniqueRanges: ~.0p\", [gb_sets:to_list(UniqueRanges)]));\n        false -> true\n    end.\n\ntester_split_is_continuous(_Config) ->\n    tester:test(?MODULE, prop_split_is_continuous, 2, 5000, [{threads, 2}]).\n\ntester_split_is_well_formed(_Config) ->\n    tester:test(?MODULE, prop_split_is_well_formed, 2, 5000, [{threads, 2}]).\n\ntester_split(_Config) ->\n    tester:test(?MODULE, prop_split, 2, 3000, [{threads, 2}]).\n\nsplit_bounds(_) ->\n    Q = intervals:split(intervals:all(), 4), %4=ReplicationFactor\n    Min = lists:min(lists:map(fun(I) -> {_, L, _, _} = intervals:get_bounds(I), L end, Q)),\n    Max = lists:max(lists:map(fun(I) -> {_, _, R, _} = intervals:get_bounds(I), R end, Q)),\n    ?equals(Min, ?MINUS_INFINITY) andalso ?equals(Max, ?PLUS_INFINITY).\n    \n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% node:mk_interval_between_ids/2, intervals:in/2 and intervals:is_continuous/1\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_mk_from_node_ids_well_formed(?RT:key(), ?RT:key()) -> true.\nprop_mk_from_node_ids_well_formed(X, Y) ->\n    intervals:is_well_formed(node:mk_interval_between_ids(X, Y)).\n\n-spec prop_mk_from_node_ids_continuous(?RT:key(), ?RT:key()) -> true.\nprop_mk_from_node_ids_continuous(X, Y) ->\n    Interval = node:mk_interval_between_ids(X, Y),\n    ?implies(not intervals:is_empty(Interval), intervals:is_continuous(Interval)).\n\n-spec prop_mk_from_node_ids(?RT:key(), ?RT:key()) -> true.\nprop_mk_from_node_ids(X, Y) ->\n    I = node:mk_interval_between_ids(X, Y),\n    ?assert(not intervals:is_empty(I)),\n    ?assert(intervals:in(Y, I)),\n    ?assert(intervals:is_continuous(I)),\n    case X =:= Y of\n        true  ->\n            ?assert(intervals:is_all(I) andalso\n                        intervals:in(X, I));\n        false ->\n            ?assert(intervals:new('(', X, Y, ']') =:= I andalso\n                        not intervals:in(X, I))\n    end.\n\ntester_mk_from_node_ids_well_formed(_Config) ->\n    tester:test(?MODULE, prop_mk_from_node_ids_well_formed, 2, 5000, [{threads, 2}]).\n\ntester_mk_from_node_ids_continuous(_Config) ->\n    tester:test(?MODULE, prop_mk_from_node_ids_continuous, 2, 5000, [{threads, 2}]).\n\ntester_mk_from_node_ids(_Config) ->\n    tester:test(?MODULE, prop_mk_from_node_ids, 2, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% node:mk_interval_between_nodes/2, intervals:in/2 and intervals:is_continuous/1\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_mk_from_nodes(node:node_type(), node:node_type()) -> true.\nprop_mk_from_nodes(X, Y) ->\n    node:mk_interval_between_nodes(X, Y) =:=\n        node:mk_interval_between_ids(node:id(X), node:id(Y)).\n\ntester_mk_from_nodes(_Config) ->\n    tester:test(?MODULE, prop_mk_from_nodes, 2, 5000, [{threads, 2}]).\n\n% helpers:\n\n-spec switch_br('(') -> '['; ('[') -> '(';\n               (')') -> ']'; (']') -> ')'.\nswitch_br('(') -> '[';\nswitch_br('[') -> '(';\nswitch_br(')') -> ']';\nswitch_br(']') -> ')'.\n\n-spec switch_br2('(') -> ']'; ('[') -> ')';\n                (')') -> '['; (']') -> '('.\nswitch_br2('(') -> ']';\nswitch_br2('[') -> ')';\nswitch_br2(')') -> '[';\nswitch_br2(']') -> '('.\n"
  },
  {
    "path": "test/join_leave_SUITE.erl",
    "content": "% @copyright 2010-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Unit tests for src/dht_node_join.erl in combination with\n%%         src/dht_node_move.erl.\n%% @end\n%% @version $Id$\n-module(join_leave_SUITE).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n%% no proto scheduler for this suite\n-define(proto_sched(Action), ok).\n\n%% spawn a process running the given function\n-define(proto_sched2(Action, Arg), proto_sched2_fun(Action, Arg)).\n\n%% Use different bench parameters when using proto sched\n%%  bench:increment(Threads1, Iterations1) for proto sched\n%%  bench:increment(Threads2, Iterations2) otherwise\n-define(bench(Threads1, Iterations1, Threads2, Iterations2),\n        ct:pal(\"Threads: ~w. Its: ~w\", [Threads2, Iterations2]),\n        bench:increment(Threads2, Iterations2)).\n\nsuite() -> [ {timetrap, {seconds, 60}} ].\n\ntest_cases() ->\n    [\n     tester_join_at,\n     add_9, rm_5, add_9_rm_5,\n     add_2x3_load,\n     tester_join_at_timeouts\n    ].\n\ngroups() ->\n%%     unittest_helper:create_ct_groups(test_cases(), [{add_9_rm_5, [sequence, {repeat_until_any_fail, forever}]}]).\n    unittest_helper:create_ct_groups([join_lookup], [{join_lookup, [sequence, {repeat_until_any_fail, 5}]}]) ++\n    unittest_helper:create_ct_groups([add_3_rm_3_data], [{add_3_rm_3_data, [sequence, {repeat_until_any_fail, 5}]}]) ++\n    unittest_helper:create_ct_groups([add_3_rm_3_data_inc], [{add_3_rm_3_data_inc, [sequence, {repeat_until_any_fail, 5}]}]) ++\n    [{graceful_leave_load, [sequence, {repeat_until_any_fail, 5}], [make_4_add_1_rm_1_load, make_4_add_2_rm_2_load, make_4_add_3_rm_3_load]}].\n\nall() ->\n%%     unittest_helper:create_ct_all(test_cases()).\n    unittest_helper:create_ct_all([join_lookup]) ++\n        unittest_helper:create_ct_all([add_3_rm_3_data]) ++\n        unittest_helper:create_ct_all([add_3_rm_3_data_inc]) ++\n        [{group, graceful_leave_load}] ++\n        test_cases().\n\n-spec additional_ring_config() -> [].\nadditional_ring_config() ->\n    [].\n\n-include(\"join_leave_SUITE.hrl\").\n\n-spec proto_sched2_fun(start, Fun::fun(() -> term())) -> pid();\n                      (stop, Pid::pid()) -> ok.\nproto_sched2_fun(start, Fun) ->\n    erlang:spawn(Fun);\nproto_sched2_fun(stop, Pid) ->\n    wait_for_process_to_die(Pid).\n"
  },
  {
    "path": "test/join_leave_SUITE.hrl",
    "content": "% @copyright 2010-2019 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Unit tests for src/dht_node_join.erl in combination with\n%%         src/dht_node_move.erl.\n%% @end\n%% @version $Id$\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(graceful_leave_load, _Config) ->\n    % TODO: occasionally the bench server (as a Scalaris client) hangs because\n    %       it connected to the node being shut down\n    %       -> in a real-life system where the client is on the same node as\n    %          the Scalaris node and both are being shut down together this has\n    %          no influence on the rest of the nodes (a stale transaction is\n    %          cleaned up by the RTMs after some time)\n    {skip, \"graceful leave not fully supported yet\"};\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(_TestCase, Config) ->\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\njoin_parameters_list() ->\n    [{move_wait_for_reply_timeout, 300},\n     {move_send_msg_retries, 3},\n     {move_send_msg_retry_delay, 0},\n     {join_request_timeout, 1000},\n     {join_request_timeouts, 3},\n     {join_lookup_timeout, 1000},\n     {join_known_hosts_timeout, 1000},\n     {join_timeout, 3000},\n     {join_get_number_of_samples_timeout, 1000}].\n\n%% check whether we may loose some lookup when join is finished and\n%% slide operations are still going on.\njoin_lookup(Config) ->\n    %% need config to get random node id\n    Config2 = unittest_helper:start_minimal_procs(Config, [], false),\n    Config3 = unittest_helper:stop_minimal_procs(Config2),\n\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config3),\n    unittest_helper:make_ring(4, [{config, [{log_path, PrivDir},\n                                            {rrepair_after_crash, false}]\n                                       ++ additional_ring_config()}]),\n    Keys = ?RT:get_replica_keys(?RT:get_random_node_id()),\n    ?proto_sched(start),\n    %% do as less as possible between make_ring and sending the lookups\n    This = comm:this(),\n    _ = [ comm:send_local(pid_groups:find_a(dht_node),\n                          {?lookup_aux, X, 0, {ping, This}}) || X <- Keys ],\n\n    %% got all replies? ok\n    [ begin\n          trace_mpath:thread_yield(),\n          receive\n              ?SCALARIS_RECV({pong, dht_node}, ok) end\n      end || _ <- Keys ],\n    ?proto_sched(stop).\n\nadd_9(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(1, [{config, [{log_path, PrivDir},\n                                            {rrepair_after_crash, false}\n                                           | join_parameters_list()]\n                                       ++ additional_ring_config()}]),\n    %% stop_time calls ?proto_sched(start)\n    stop_time(fun add_9_test/0, \"add_9\").\n\nadd_9_test() ->\n    _ = api_vm:add_nodes(9),\n    check_size(10).\n\nrm_5(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(1, [{config, [{log_path, PrivDir},\n                                            {rrepair_after_crash, false}\n                                           | join_parameters_list()]\n                                       ++ additional_ring_config()}]),\n    _ = api_vm:add_nodes(9),\n    check_size(10),\n    %% stop_time calls ?proto_sched(start)\n    stop_time(fun rm_5_test/0, \"rm_5\").\n\nrm_5_test() ->\n    _ = api_vm:kill_nodes(5),\n    check_size(5).\n\nadd_9_rm_5(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(1, [{config, [{log_path, PrivDir},\n                                            {rrepair_after_crash, false}\n                                           | join_parameters_list()]\n                                       ++ additional_ring_config()}]),\n    %% stop_time calls ?proto_sched(start)\n    stop_time(fun add_9_rm_5_test/0, \"add_9_rm_5\").\n\nadd_9_rm_5_test() ->\n    _ = api_vm:add_nodes(9),\n    check_size(10),\n    _ = api_vm:kill_nodes(5),\n    check_size(5).\n\nadd_3_rm_3_data(Config) ->\n    add_3_rm_3_data(Config, false).\n\nadd_3_rm_3_data_inc(Config) ->\n    add_3_rm_3_data(Config, true).\n\nadd_3_rm_3_data(Config, Incremental) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(1, [{config, [{move_max_transport_entries, 25},\n                                            {move_use_incremental_slides, Incremental},\n                                            {log_path, PrivDir},\n                                            {monitor_perf_interval, 0},\n                                            {rrepair_after_crash, false}\n                                           | join_parameters_list()]\n                                       ++ additional_ring_config()}]),\n\n    RandomKeys = [randoms:getRandomString() || _ <- lists:seq(1,100)],\n    % note: there may be hash collisions -> count the number of unique DB entries!\n    RandomHashedKeys = lists:flatmap(fun(K) -> ?RT:get_replica_keys(?RT:hash_key(K)) end, RandomKeys),\n    ExpLoad = length(lists:usort(RandomHashedKeys)),\n\n    _ = util:map_with_nr(fun(Key, X) -> {ok} = api_tx:write(Key, X) end, RandomKeys, 10000001),\n    % wait for late write messages to arrive at the original nodes\n    api_tx_SUITE:wait_for_dht_entries(ExpLoad),\n\n    ?proto_sched(start),\n    ct:pal(\"######## starting join ########\"),\n    _ = api_vm:add_nodes(3),\n    ?proto_sched(stop),\n    unittest_helper:check_ring_size_fully_joined(4),\n    ?proto_sched(restart),\n    ct:pal(\"######## starting graceful leave ########\"),\n    _ = api_vm:shutdown_nodes(3),\n    ?proto_sched(stop),\n    check_size(1),\n    ct:pal(\"######## checking load ########\"),\n    unittest_helper:check_ring_load(ExpLoad),\n    ct:pal(\"######## done ########\"),\n    ok.\n\nadd_2x3_load(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(1, [{config, [{log_path, PrivDir},\n                                            {rrepair_after_crash, false},\n                                            {monitor_perf_interval, 0}\n                                           | join_parameters_list()]\n                                       ++ additional_ring_config()}]),\n    %% stop_time calls ?proto_sched2(start/stop)\n    stop_time(fun add_2x3_load_test/0,\n               fun() -> ?bench(1, 100, 1, 5000) end,\n               \"add_2x3_load\"),\n    unittest_helper:check_ring_load(config:read(replication_factor)),\n    unittest_helper:check_ring_data().\n\nadd_2x3_load_test() ->\n    ct:pal(\"######## starting join 1 ########\"),\n    _ = api_vm:add_nodes(3),\n    check_size(4),\n    timer:sleep(500),\n    ct:pal(\"######## starting join 2 ########\"),\n    _ = api_vm:add_nodes(3),\n    check_size(7),\n    ct:pal(\"######## waiting for bench finish ########\").\n\nmake_4_add_1_rm_1_load(Config) ->\n    make_4_add_x_rm_y_load(Config, 1, 1, true).\n\nmake_4_add_2_rm_2_load(Config) ->\n    make_4_add_x_rm_y_load(Config, 2, 2, true).\n\nmake_4_add_3_rm_3_load(Config) ->\n    make_4_add_x_rm_y_load(Config, 3, 3, true).\n\n-spec make_4_add_x_rm_y_load(Config::[tuple()], X::non_neg_integer(), Y::pos_integer(),\n                             StartOnlyAdded::boolean()) -> boolean().\nmake_4_add_x_rm_y_load(Config, X, Y, StartOnlyAdded) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring_with_ids(\n      ?RT:get_replica_keys(?MINUS_INFINITY),\n      [{config, [{log_path, PrivDir}, {rrepair_after_crash, false},\n                 {monitor_perf_interval, 0} | join_parameters_list()]\n            ++ additional_ring_config()}]),\n    %% stop_time calls ?proto_sched(start)\n    stop_time(fun() -> add_x_rm_y_load_test(X, Y, StartOnlyAdded) end,\n               fun() -> ?bench(10, 100, 10, 1000) end,\n               lists:flatten(io_lib:format(\"add_~B_rm_~B_load\", [X, Y]))),\n    unittest_helper:check_ring_load(config:read(replication_factor)*10),\n    unittest_helper:check_ring_data().\n\n-spec add_x_rm_y_load_test(X::non_neg_integer(), Y::pos_integer(), StartOnlyAdded::boolean()) -> ok.\nadd_x_rm_y_load_test(X, Y, StartOnlyAdded) ->\n    %% BenchPid = erlang:spawn(fun() -> bench:increment(10, 1000) end),\n    {Started, []} = api_vm:add_nodes(X),\n    check_size(X + 4),\n    timer:sleep(500),\n    Ring = [begin\n                Pred = hd(node_details:get(Details, predlist)),\n                Node = node_details:get(Details, node),\n                {comm:make_local(node:pidX(Node)), node:id(Node),\n                 ?RT:get_range(node:id(Pred), node:id(Node)) / ?RT:n()}\n            end || {ok, Details} <- statistics:get_ring_details()],\n    ct:pal(\"RING: ~.2p\", [Ring]),\n    ct:pal(\"######## finish join ########\"),\n    % let Y nodes gracefully leave, one after another - not more than minority of tm_tms can die!\n    ToShutdown =\n        if StartOnlyAdded -> util:random_subset(Y, Started);\n           true -> [pid_groups:group_of(Pid)|| Pid <- util:random_subset(Y, pid_groups:find_all(dht_node))]\n        end,\n    [begin\n         Pid = pid_groups:pid_of(Group, dht_node),\n         ct:pal(\"Killing #~p: ~p (~p)\", [I, Pid, Group]),\n         comm:send_local(Pid, {leave, null}),\n         check_size(X + 4 - I)\n     end || {I, Group} <- lists:zip(lists:seq(1, Y), ToShutdown)],\n    check_size(X + 4 - Y),\n    ct:pal(\"######## waiting for bench finish ########\").\n\n-spec prop_join_at(FirstId::?RT:key(), SecondId::?RT:key(), Incremental::boolean()) -> true.\nprop_join_at(SameId, SameId, _Incremental) ->\n    %% tester chose same id twice, which is not supported by this test\n    %% (join at an existing id), so we skip this case\n    true;\nprop_join_at(FirstId, SecondId, Incremental) ->\n    OldProcesses = unittest_helper:get_processes(),\n    BenchSlaves = 2,\n\n    unittest_helper:make_ring_with_ids(\n      [FirstId],\n      [{config, [{move_max_transport_entries, 25},\n                 {move_use_incremental_slides, Incremental},\n                 pdb:get(log_path, ?MODULE), {rrepair_after_crash, false},\n                 {monitor_perf_interval, 0} | join_parameters_list()]\n            ++ additional_ring_config()}]),\n\n    RandomKeys = [randoms:getRandomString() || _ <- lists:seq(1,100)],\n    % note: there may be hash collisions -> count the number of unique DB entries!\n    RandomHashedKeys = lists:flatmap(fun(K) -> ?RT:get_replica_keys(?RT:hash_key(K)) end, RandomKeys),\n    ExpLoad = length(lists:usort(RandomHashedKeys)),\n\n    _ = util:map_with_nr(fun(Key, X) -> {ok} = api_tx:write(Key, X) end, RandomKeys, 10000001),\n    % wait for late write messages to arrive at the original nodes\n    api_tx_SUITE:wait_for_dht_entries(ExpLoad),\n\n    %% stop_time calls ?proto_sched2(start/stop)\n    stop_time(fun() -> _ = admin:add_node_at_id(SecondId), check_size(2) end,\n               fun() -> ?bench(BenchSlaves, 15, BenchSlaves, 50) end,\n               \"join_at\"),\n\n    unittest_helper:check_ring_load(ExpLoad + BenchSlaves * config:read(replication_factor)),\n    unittest_helper:check_ring_data(),\n    unittest_helper:stop_ring(),\n    unittest_helper:kill_new_processes(OldProcesses),\n    true.\n\ntester_join_at(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    pdb:set({log_path, PrivDir}, ?MODULE),\n    tester:test(?MODULE, prop_join_at, _Arity = 3, _Iterations = 5).\n\n-spec reply_to_join_response(Msg::comm:message(), State, Reason::abort | crash | send_error) -> State\n        when is_subtype(State, dht_node_state:state() | dht_node_join:join_state()).\nreply_to_join_response(Msg, State, Reason) when is_tuple(State) andalso element(1, State) =:= state ->\n    case Msg of\n        {join, join_response, Succ, Pred, MoveId, CandId, _TargetId, _NextOp} ->\n            % joined nodes only reply with a reject! (crash and send_error are always possible)\n            case Reason of\n                send_error ->\n                    comm:send(node:pidX(Succ), {move, {send_error, comm:this(), Msg, unittest}, MoveId});\n                crash ->\n                    comm:send(node:pidX(Succ), {move, MoveId, {fd_notify, crash, comm:this(), 'DOWN'}});\n                abort ->\n                    dht_node_join:reject_join_response(Succ, Pred, MoveId, CandId)\n            end;\n        {join, join_response, _MsgReason, _CandId} ->\n            % joined nodes only ignore this message! (not sent with shepherd, no fd used)\n            case Reason of\n                _ ->\n                    ok\n            end\n    end,\n    State;\nreply_to_join_response(Msg, State, Reason) when is_tuple(State) andalso element(1, State) =:= join ->\n    case Msg of\n        {join, join_response, Succ, Pred, MoveId, CandId, _TargetId, _NextOp} ->\n            case Reason of\n                send_error ->\n                    comm:send(node:pidX(Succ), {move, {send_error, comm:this(), Msg, unittest}, MoveId});\n                abort ->\n                    dht_node_join:reject_join_response(Succ, Pred, MoveId, CandId);\n                crash ->\n                    comm:send(node:pidX(Succ), {move, MoveId, {fd_notify, crash, comm:this(), 'DOWN'}})\n            end;\n        _ -> ok\n    end,\n    State.\n\n-spec reply_to_join_request(Msg::comm:message(), State, Reason::not_responsible | busy) -> State\n        when is_subtype(State, dht_node_state:state() | dht_node_join:join_state()).\nreply_to_join_request(Msg, State, Reason) when is_tuple(State) andalso element(1, State) =:= state ->\n    case Msg of\n        {join, join_request, NewPred, CandId, _MaxTransportEntries} ->\n            comm:send(node:pidX(NewPred), {join, join_response, Reason, CandId});\n        _ -> ok\n    end,\n    State;\nreply_to_join_request(_Msg, State, _Reason) when is_tuple(State) andalso element(1, State) =:= join ->\n    State.\n\n% TODO: simulate more message drops,\n% TODO: simulate certain protocol runs, e.g. the other node replying with noop, etc.\n% keep in sync with dht_node_join and the timeout config parameters of join_parameters_list/0\n-type join_message() ::\n%% messages at the joining node:\n%{get_dht_nodes_response, Nodes::[node:node_type()]} |\n%{join, get_number_of_samples, Samples::non_neg_integer(), Conn::connection()} |\n%{join, get_candidate_response, OrigJoinId::?RT:key(), Candidate::lb_op:lb_op(), Conn::connection()} |\n%{join, join_response, Succ::node:node_type(), Pred::node:node_type(), MoveFullId::slide_op:id(),\n% CandId::lb_op:id(), TargetId::?RT:key(), NextOp::slide_op:next_op()} |\n    {{join, join_response, '_', '_', '_', '_', '_', '_'}, [], 1..2,\n     {reply_to_join_response, abort | crash | send_error}} |\n%{join, join_response, not_responsible, CandId::lb_op:id()} |\n%{join, lookup_timeout, Conn::connection()} |\n%{join, known_hosts_timeout} |\n%{join, get_number_of_samples_timeout, Conn::connection()} |\n%{join, join_request_timeout, Timeouts::non_neg_integer(), CandId::lb_op:id()} |\n%{join, timeout} |\n%% messages at the existing node:\n%{join, number_of_samples_request, SourcePid::comm:mypid(), LbPsv::module(), Conn::connection()} |\n%{join, get_candidate, Source_PID::comm:mypid(), Key::?RT:key(), LbPsv::module(), Conn::connection()} |\n%{join, join_request, NewPred::node:node_type(), CandId::lb_op:id(), MaxTransportEntries::unknown | pos_integer()} |\n    {{join, join_request, '_', '_', '_'}, [], 1..2,\n     drop_msg | {reply_to_join_request, not_responsible | busy}}.\n%{Msg::lb_psv_simple:custom_message() | lb_psv_split:custom_message() |\n%      lb_psv_gossip:custom_message(),\n% {join, LbPsv::module(), LbPsvState::term()}}\n\n%% @doc Makes IgnoredMessages unique, i.e. only one msg per msg type.\n-spec fix_tester_ignored_msg_list(IgnoredMessages::[join_message(),...]) -> [join_message(),...].\nfix_tester_ignored_msg_list(IgnoredMessages) ->\n    IMsg2 = lists:usort(fun(E1, E2) ->\n                                erlang:element(2, erlang:element(1, E1)) =<\n                                    erlang:element(2, erlang:element(1, E2))\n                        end, IgnoredMessages),\n    [begin\n         NewAction =\n             case Action of\n                 {reply_to_join_response, Reason} ->\n                     fun(MsgX, StateX) -> reply_to_join_response(MsgX, StateX, Reason) end;\n                 {reply_to_join_request, Reason} ->\n                     fun(MsgX, StateX) -> reply_to_join_request(MsgX, StateX, Reason) end;\n                 X -> X\n             end,\n         {Msg, Conds, Count, NewAction}\n     end || {Msg, Conds, Count, Action} <- IMsg2].\n\n-spec send_ignore_msg_list_to(NthNode::1..4, PredOrSuccOrNode::pred | succ | node, IgnoredMessages::[join_message(),...]) -> ok.\nsend_ignore_msg_list_to(NthNode, PredOrSuccOrNode, IgnoredMessages) ->\n    % cleanup, just in case:\n    _ = [comm:send_local(DhtNodePid, {mockup_dht_node, clear_match_specs})\n           || DhtNodePid <- pid_groups:find_all(dht_node)],\n\n    FailMsg = lists:flatten(io_lib:format(\"~.0p (~B.*) ignoring messages: ~.0p\",\n                                          [PredOrSuccOrNode, NthNode, IgnoredMessages])),\n    {_PredsPred, Pred, Node, Succ} = dht_node_move_SUITE:get_pred_node_succ(NthNode, FailMsg),\n    Other = case PredOrSuccOrNode of\n                succ -> Succ;\n                pred -> Pred;\n                node -> Node\n            end,\n    comm:send(node:pidX(Other), {mockup_dht_node, add_match_specs, IgnoredMessages}).\n\n-spec prop_join_at_timeouts(FirstId::?RT:key(), SecondId::?RT:key(),\n        IgnoredMessages::[join_message(),...], IgnMsgAt1st::boolean(),\n        IgnMsgAt2nd::boolean()) -> true.\nprop_join_at_timeouts(FirstId, SecondId, IgnoredMessages_, IgnMsgAt1st, IgnMsgAt2nd) ->\n    OldProcesses = unittest_helper:get_processes(),\n    BenchSlaves = 2, BenchRuns = 50,\n    IgnoredMessages = fix_tester_ignored_msg_list(IgnoredMessages_),\n\n    unittest_helper:make_ring_with_ids(\n      [FirstId],\n      [{config, [{dht_node, mockup_dht_node}, pdb:get(log_path, ?MODULE),\n                 {rrepair_after_crash, false}, {monitor_perf_interval, 0}\n                | join_parameters_list()]\n            ++ additional_ring_config()}]),\n    if IgnMsgAt1st -> send_ignore_msg_list_to(1, node, IgnoredMessages);\n       true        -> ok\n    end,\n    BenchPid = erlang:spawn(fun() -> bench:increment(BenchSlaves, BenchRuns) end),\n    wait_for_process_to_die(BenchPid),\n    MoreOpts = if IgnMsgAt2nd -> [{match_specs, IgnoredMessages}];\n                  true        -> []\n               end,\n    _ = admin:add_node([{{dht_node, id}, SecondId}, {skip_psv_lb}] ++ MoreOpts),\n    check_size(2),\n    unittest_helper:check_ring_load(BenchSlaves * config:read(replication_factor)),\n    unittest_helper:check_ring_data(),\n    unittest_helper:stop_ring(),\n%%     randoms:stop(), % tester may need it\n    _ = inets:stop(),\n    unittest_helper:kill_new_processes(OldProcesses),\n    true.\n\ntester_join_at_timeouts(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    pdb:set({log_path, PrivDir}, ?MODULE),\n    tester:test(?MODULE, prop_join_at_timeouts, 5, 10).\n\n-spec stop_time(F::fun(() -> any()), Tag::string()) -> ok.\nstop_time(F, Tag) ->\n    Start = os:timestamp(),\n    ?proto_sched(start),\n    F(),\n    ?proto_sched(stop),\n    Stop = os:timestamp(),\n    ElapsedTime = erlang:max(1, timer:now_diff(Stop, Start)) / 1000000.0,\n    Frequency = 1 / ElapsedTime,\n    ct:pal(\"~p took ~ps: ~p1/s~n\",\n           [Tag, ElapsedTime, Frequency]),\n    ok.\n\n-spec stop_time(F1::fun(() -> any()), F2::fun(() -> any()), Tag::string()) -> ok.\nstop_time(F1, F2, Tag) ->\n    Start = os:timestamp(),\n    Pid = ?proto_sched2(start, fun() -> F2() end),\n    F1(),\n    ?proto_sched2(stop, Pid),\n    Stop = os:timestamp(),\n    ElapsedTime = erlang:max(1, timer:now_diff(Stop, Start)) / 1000000.0,\n    Frequency = 1 / ElapsedTime,\n    ct:pal(\"~p took ~ps: ~p1/s~n\",\n           [Tag, ElapsedTime, Frequency]),\n    ok.\n\n-spec check_size(Size::pos_integer()) -> ok.\ncheck_size(Size) ->\n    Infected = trace_mpath:infected(),\n    ?IIF(Infected, ?proto_sched(stop), ok),\n    unittest_helper:check_ring_size(Size),\n    unittest_helper:wait_for_stable_ring(),\n    unittest_helper:check_ring_size_fully_joined(Size),\n    ?IIF(Infected, ?proto_sched(restart), ok).\n\n%% @doc Waits for the given process to die (without putting the wait messages\n%%      into the proto_sched if enabled).\n-spec wait_for_process_to_die(Pid::pid()) -> ok.\nwait_for_process_to_die(Pid) ->\n    Infected = trace_mpath:infected(),\n    ?IIF(Infected, ?proto_sched(stop), ok),\n    util:wait_for_process_to_die(Pid),\n    ?IIF(Infected, ?proto_sched(restart), ok).\n"
  },
  {
    "path": "test/join_leave_proto_sched_SUITE.erl",
    "content": "% @copyright 2010-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Unit tests for src/dht_node_join.erl in combination with\n%%         src/dht_node_move.erl.\n%% @end\n%% @version $Id$\n-module(join_leave_proto_sched_SUITE).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n%% start proto scheduler for this suite\n-define(proto_sched(Action), proto_sched_fun(Action)).\n\n%% start proto scheduler with a second thread running function given as Arg\n%% Action :: start | restart | stop.  Arg :: fun()\n-define(proto_sched2(Action, Arg), proto_sched2_fun(Action, Arg)).\n\n%% Use different bench parameters when using proto sched\n%%  bench:increment(Threads1, Iterations1) for proto sched\n%%  bench:increment(Threads2, Iterations2) otherwise\n-define(bench(Threads1, Iterations1, Threads2, Iterations2),\n        bench:increment(Threads1, Iterations1)).\n\nsuite() -> [ {timetrap, {seconds, 60}} ].\n\ntest_cases() ->\n    ReplicationFactor = config:read(replication_factor),\n    if\n        ReplicationFactor > 16 -> [];\n        true                   -> [ tester_join_at ]\n    end.\n\ngroups() ->\n    ReplicationFactor = config:read(replication_factor),\n    AddRMGroup =\n        if\n            ReplicationFactor > 16 -> [ add_9, rm_5, add_9_rm_5               ];\n            true                   -> [ add_9, rm_5, add_9_rm_5, add_2x3_load ]\n        end,\n    unittest_helper:create_ct_groups([join_lookup], [{join_lookup, [sequence, {repeat_until_any_fail, 20}]}]) ++\n    unittest_helper:create_ct_groups([add_3_rm_3_data], [{add_3_rm_3_data, [sequence, {repeat_until_any_fail, 20}]}]) ++\n    unittest_helper:create_ct_groups([add_3_rm_3_data_inc], [{add_3_rm_3_data_inc, [sequence, {repeat_until_any_fail, 20}]}]) ++\n    [{add_rm, [sequence, {repeat_until_any_fail, 5}], AddRMGroup}] ++\n    [{graceful_leave_load, [sequence, {repeat_until_any_fail, 5}], [make_4_add_1_rm_1_load, make_4_add_2_rm_2_load, make_4_add_3_rm_3_load]}].\n\nall() ->\n    unittest_helper:create_ct_all([join_lookup]) ++\n        unittest_helper:create_ct_all([add_3_rm_3_data]) ++\n%        unittest_helper:create_ct_all([add_3_rm_3_data_inc]) ++ % TODO: too heavy for proto_sched to finish within 120s\n        [{group, add_rm},\n         {group, graceful_leave_load}] ++\n        test_cases().\n\n-spec additional_ring_config() -> [{stabilization_interval_base, 100000},...].\nadditional_ring_config() ->\n    % increase ring stabilisation interval since proto_sched infections get\n    % lost if rm subscriptions are triggered instead of continuing due to our\n    % direct (and infected) messages!\n    [{stabilization_interval_base, 100000}, % ms\n     {tman_cyclon_interval, 100} % s\n    ].\n\n-include(\"join_leave_SUITE.hrl\").\n\n-spec proto_sched_fun(start | restart | stop) -> ok.\nproto_sched_fun(start) ->\n    %% ct:pal(\"Starting proto scheduler\"),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed), % clear previous stats\n    proto_sched:thread_num(1),\n    proto_sched:thread_begin();\nproto_sched_fun(restart) ->\n    %% ct:pal(\"Starting proto scheduler\"),\n    proto_sched:thread_num(1),\n    proto_sched:thread_begin();\nproto_sched_fun(stop) ->\n    %% is a ring running?\n    case erlang:whereis(pid_groups) =:= undefined\n             orelse pid_groups:find_a(proto_sched) =:= failed of\n        true -> ok;\n        false ->\n            %% then finalize proto_sched run:\n            %% try to call thread_end(): if this\n            %% process was running the proto_sched\n            %% thats fine, otherwise thread_end()\n            %% will raise an exception\n            proto_sched:thread_end(),\n            proto_sched:wait_for_end(),\n            unittest_helper:print_proto_sched_stats(at_end_if_failed_append),\n            proto_sched:cleanup()\n    end.\n\n-spec proto_sched2_fun(start, Fun::fun(() -> term())) -> pid();\n                      (stop, Pid::pid()) -> ok.\nproto_sched2_fun(start, Fun) ->\n    unittest_helper:print_proto_sched_stats(at_end_if_failed), % clear previous stats\n    proto_sched:thread_num(2),\n    Pid = erlang:spawn(fun() ->\n                               proto_sched:thread_begin(),\n                               Fun(),\n                               proto_sched:thread_end()\n                       end),\n    proto_sched:thread_begin(),\n    Pid;\nproto_sched2_fun(stop, _Pid) ->\n    proto_sched_fun(stop).\n"
  },
  {
    "path": "test/json_on_cseq_SUITE.erl",
    "content": "%% @copyright 2013-2017 Zuse Institute Berlin\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    Unit tests for json_on_cseq\n%% @end\n%% @version $Id$\n-module(json_on_cseq_SUITE).\n-author('skrzypczak@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\nall() -> [\n          tester_type_check_json_on_cseq,\n          test_dotto_avail\n         ].\n\nsuite() -> [ {timetrap, {seconds, 400}} ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_testcase(_TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_symmetric_ring([{config, [{log_path, PrivDir},\n                                                   {replication_factor, 4}]}]),\n    unittest_helper:check_ring_size_fully_joined(config:read(replication_factor)),\n    [{stop_ring, true} | Config].\n\ntest_dotto_avail(_Config) ->\n    Data = dict:from_list([{name, \"bob\"}, {age, 29}, {friends, [\"sandy\", \"patrick\"]},\n                           {data, dict:from_list([{numbers, [10, 11, 12]}])}]),\n    Expected = dict:from_list([{name, \"bob\"}, {age, 29}, {friends, [\"sandy\"]},\n                           {data, dict:from_list([{numbers, [10, 11, 12]}])}]),\n    {ok, Result} = dotto:remove(Data, [friends, 1]),\n    ?equals(dict:to_list(Expected), dict:to_list(Result)).\n\ntester_type_check_json_on_cseq(_Config) ->\n    Count = 250,\n\n    JsonType = {typedef, json_on_cseq, json, []},\n    tester:register_type_checker(JsonType, json_on_cseq, is_json),\n    tester:register_value_creator(JsonType, json_on_cseq, create_rand_json, 1),\n\n    %% [{modulename, [publicexeludelist = {fun, arity}], [privexcludelist..}]\n    %% Excluded these functions because cannot create funs\n    Modules = [{json_on_cseq,\n                 [{cc_noop,3}],\n                 [{read_helper, 2},\n                  {write_helper, 3},\n                  {write_helper, 5}\n                 ]}\n              ],\n\n    _ = [tester:type_check_module(Mod, Excl, ExclPriv, Count)\n         || {Mod, Excl, ExclPriv} <- Modules],\n\n    tester:unregister_type_checker(JsonType),\n    tester:unregister_value_creator(JsonType),\n    true.\n\n"
  },
  {
    "path": "test/l_on_cseq_SUITE.erl",
    "content": "%% @copyright 2012-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for l_on_cseq\n%% @end\n-module(l_on_cseq_SUITE).\n-author('schuett@zib.de').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\n-dialyzer({[no_opaque, no_return],\n           [test_renew_with_concurrent_range_change/1,\n            test_renew_with_concurrent_aux_change_invalid_split/1,\n            test_renew_with_concurrent_aux_change_valid_split/1,\n            test_renew_with_concurrent_aux_change_invalid_merge/1,\n            test_renew_with_concurrent_aux_change_valid_merge/1,\n            test_handover_with_concurrent_aux_change/1]}).\n\ngroups() ->\n    [{tester_tests, [sequence], [\n                              tester_type_check_l_on_cseq\n                              ]},\n     {renew_tests, [sequence], [\n                             test_renew_with_concurrent_renew,\n                             %test_renew_with_concurrent_owner_change,\n                             test_renew_with_concurrent_range_change,\n                             test_renew_with_concurrent_aux_change_invalid_split,\n                             test_renew_with_concurrent_aux_change_valid_split,\n                             test_renew_with_concurrent_aux_change_invalid_merge,\n                             test_renew_with_concurrent_aux_change_valid_merge\n                             ]},\n     {split_tests, [sequence], [\n                             test_split,\n                             test_split_with_concurrent_renew,\n                             test_split_but_lease_already_exists,\n                             %test_split_with_owner_change_in_step1,\n                             %test_split_with_owner_change_in_step2,\n                             %test_split_with_owner_change_in_step3,\n                             test_split_with_aux_change_in_step1\n                            ]},\n     {merge_tests, [sequence], [\n                               ]}, % @todo\n     {takeover_tests, [sequence], [\n                                   test_takeover\n                               ]},\n     {handover_tests, [sequence], [\n                                test_handover,\n                                test_handover_with_concurrent_renew,\n                                test_handover_with_concurrent_aux_change%,\n                                %test_handover_with_concurrent_owner_change\n                                ]}\n    ].\n\nall() ->\n    [\n     {group, tester_tests},\n     {group, renew_tests},\n     {group, split_tests},\n     {group, handover_tests},\n     {group, takeover_tests}\n     ].\n\nsuite() -> [ {timetrap, {seconds, 180}} ].\n\ngroup(tester_tests) ->\n    [{timetrap, {seconds, 400}}];\ngroup(renew_tests) ->\n    [{timetrap, {seconds, 60}}];\ngroup(split_tests) ->\n    [{timetrap, {seconds, 60}}];\ngroup(takeover_tests) ->\n    [{timetrap, {seconds, 60}}];\ngroup(handover_tests) ->\n    [{timetrap, {seconds, 60}}];\ngroup(_) ->\n    suite().\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    Config2 = unittest_helper:start_minimal_procs(Config, [], true),\n    RingSize = config:read(replication_factor),\n    Config3 = unittest_helper:stop_minimal_procs(Config2),\n\n    case TestCase of\n        test_garbage_collector ->\n            unittest_helper:make_ring(RingSize, [{config, [{log_path, PrivDir},\n                                                           {leases, true}]}]),\n            unittest_helper:check_ring_size_fully_joined(RingSize),\n            ok;\n        _ ->\n            unittest_helper:make_ring(RingSize, [{config, [{log_path, PrivDir},\n                                                           {leases, true}]}]),\n            unittest_helper:check_ring_size_fully_joined(RingSize),\n            ok\n    end,\n    [{stop_ring, true} | Config3].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\ntester_type_check_l_on_cseq(_Config) ->\n    Count = 500,\n    config:write(no_print_ring_data, true),\n    tester:register_value_creator({typedef, prbr, write_filter, []},\n                                  prbr, tester_create_write_filter, 1),\n\n    %% [{modulename, [excludelist = {fun, arity}]}]\n    Modules =\n        [ {l_on_cseq,\n           [ {add_first_lease_to_db, 2}, %% cannot create DB refs for State\n             {lease_renew, 2}, %% sends messages\n             {lease_renew, 3}, %% sends messages\n             {lease_handover, 3}, %% sends messages\n             {lease_takeover, 2}, %% sends messages\n             {lease_takeover_after, 3}, %% sends messages\n             {lease_split, 4}, %% sends messages\n             {lease_merge, 3}, %% sends messages\n             {lease_send_lease_to_node, 3}, %% sends messages\n             {lease_split_and_change_owner, 5}, %% sends messages\n             {id, 1}, %% todo\n             {split_range, 1}, %% todo\n             {unittest_lease_update, 4}, %% only for unittests\n             {unittest_lease_update_unsafe, 3}, %% only for unittests\n             {unittest_clear_lease_list, 1}, %% only for unittests\n             {disable_lease, 2}, %% requires dht_node_state\n             {on, 2}, %% cannot create dht_node_state (reference for bulkowner)\n             {get_pretty_timeout, 1}, %% cannot create valid timestamps\n             {read, 2} %% cannot create pids\n           ],\n           [\n             {get_active_lease, 1}, %% cannot create reference (bulkowner uses one in dht_node_state\n             {update_lease, 5}, %% cannot create reference (bulkowner uses one in dht_node_state\n             {renew_and_update_round, 4}, %% cannot create reference (bulkowner uses one in dht_node_state\n             {format_utc_timestamp, 1} %% cannot create valid timestamps\n           ]},\n          {lease_list,\n           [\n             {update_lease_in_dht_node_state, 4}, %% cannot create dht_node_state (reference for bulkowner)\n             {remove_lease_from_dht_node_state, 4}, %% cannot create dht_node_state (reference for bulkowner)\n             {get_next_round, 2}, %% cannot create dht_node_state (reference for bulkowner)\n             {update_next_round, 3} %% cannot create dht_node_state (reference for bulkowner)\n           ],\n           [\n            {update_lease_in_dht_node_state, 3}, %% cannot create dht_node_state (reference for bulkowner)\n            {update_active_lease, 2}, %% assert fails for random input\n            {remove_next_round, 2}, %% cannot create dht_node_state (reference for bulkowner)\n            {remove_passive_lease_from_dht_node_state, 3}, %% cannot create dht_node_state (reference for bulkowner)\n            {remove_active_lease_from_dht_node_state, 3} %% cannot create dht_node_state (reference for bulkowner)\n           ]},\n          {leases,\n           [\n            {is_responsible, 2} %% cannot create dht_node_state (reference for bulkowner)\n           ],\n           [\n           ]\n          }\n        ],\n    %% join a dht_node group to be able to call lease trigger functions\n    pid_groups:join(pid_groups:group_with(dht_node)),\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    tester:unregister_value_creator({typedef, prbr, write_filter, []}),\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% renew unit tests\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ntest_renew_with_concurrent_renew(_Config) ->\n    ModifyF =\n        fun(Old) ->\n                l_on_cseq:set_timeout(\n                  l_on_cseq:set_version(Old, l_on_cseq:get_version(Old)+1))\n        end,\n    WaitF = fun wait_for_simple_update/2,\n    test_renew_helper(_Config, ModifyF, WaitF),\n    true.\n\ntest_renew_with_concurrent_owner_change(_Config) ->\n    ModifyF =\n        fun(Old) ->\n                l_on_cseq:set_owner(\n                  l_on_cseq:set_timeout(\n                    l_on_cseq:set_version(\n                      l_on_cseq:set_epoch(Old, l_on_cseq:get_epoch(Old)+1),\n                      0)),\n                  comm:this())\n        end,\n    WaitF = fun wait_for_delete/2,\n    test_renew_helper(_Config, ModifyF, WaitF),\n    true.\n\ntest_renew_with_concurrent_range_change(_Config) ->\n    ModifyF =\n        fun(Old) ->\n                l_on_cseq:set_range(\n                  l_on_cseq:set_timeout(\n                    l_on_cseq:set_version(\n                      l_on_cseq:set_epoch(Old, l_on_cseq:get_epoch(Old)+1),\n                      0)),\n                  obfuscated_intervals_all())\n        end,\n    WaitF = fun wait_for_epoch_update/2,\n    test_renew_helper(_Config, ModifyF, WaitF),\n    true.\n\ntest_renew_with_concurrent_aux_change_invalid_split(_Config) ->\n    ModifyF =\n        fun(Old) ->\n                Aux = {invalid, split, r1, r2},\n                l_on_cseq:set_aux(\n                  l_on_cseq:set_timeout(\n                    l_on_cseq:set_version(\n                      l_on_cseq:set_epoch(Old, l_on_cseq:get_epoch(Old)+1),\n                    0)),\n                  Aux)\n        end,\n    WaitF = fun wait_for_epoch_update/2,\n    test_renew_helper(_Config, ModifyF, WaitF),\n    true.\n\ntest_renew_with_concurrent_aux_change_valid_split(_Config) ->\n    ModifyF =\n        fun(Old) ->\n                Aux = {valid, split, r1, r2},\n                l_on_cseq:set_aux(\n                  l_on_cseq:set_timeout(\n                    l_on_cseq:set_version(\n                      l_on_cseq:set_epoch(Old, l_on_cseq:get_epoch(Old)+1),\n                    0)),\n                  Aux)\n        end,\n    WaitF = fun wait_for_epoch_update/2,\n    test_renew_helper(_Config, ModifyF, WaitF),\n    true.\n\ntest_renew_with_concurrent_aux_change_invalid_merge(_Config) ->\n    ModifyF =\n        fun(Old) ->\n                Aux = {invalid, merge, r1, r2},\n                l_on_cseq:set_aux(\n                  l_on_cseq:set_timeout(\n                    l_on_cseq:set_version(\n                      l_on_cseq:set_epoch(Old, l_on_cseq:get_epoch(Old)+1),\n                    0)),\n                  Aux)\n        end,\n    WaitF = fun wait_for_epoch_update/2,\n    test_renew_helper(_Config, ModifyF, WaitF),\n    true.\n\ntest_renew_with_concurrent_aux_change_valid_merge(_Config) ->\n    ModifyF =\n        fun(Old) ->\n                Aux = {valid, merge, r1, r2},\n                l_on_cseq:set_aux(\n                  l_on_cseq:set_timeout(\n                    l_on_cseq:set_version(\n                      l_on_cseq:set_epoch(Old, l_on_cseq:get_epoch(Old)+1),\n                    0)),\n                  Aux)\n        end,\n    WaitF = fun wait_for_epoch_update/2,\n    test_renew_helper(_Config, ModifyF, WaitF),\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% split unit tests\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ntest_split(_Config) ->\n    NullF = fun (_Id, _Lease, _DHTNode) -> ok end,\n    WaitRightLeaseF = fun (Id, Lease) ->\n                             OldEpoch   = l_on_cseq:get_epoch(Lease),\n                             wait_for_lease_version(Id, OldEpoch+2, 0)\n                     end,\n    WaitLeftLeaseF = fun (Id) ->\n                             wait_for_lease_version(Id, 2, 0)\n                      end,\n    FinalWaitF = fun wait_for_split_success_msg/0,\n    test_split_helper_for_4_steps(_Config,\n                                  NullF, NullF,NullF, NullF,\n                                  WaitLeftLeaseF, WaitRightLeaseF, FinalWaitF),\n    true.\n\ntest_split_with_concurrent_renew(_Config) ->\n    NullF = fun (_Id, _Lease, _DHTNode) -> ok end,\n    RenewLeaseLeftF = fun (_Id, Lease, _DHTNode) ->\n                              log:log(\"left renew lease with ~w ~w\", [_Id, Lease]),\n                              l_on_cseq:lease_renew(Lease, passive),\n                              wait_for_lease_version(l_on_cseq:get_id(Lease),\n                                                     l_on_cseq:get_epoch(Lease),\n                                                     l_on_cseq:get_version(Lease)+1)\n                  end,\n    RenewLeaseRightF = fun (_Id, Lease, _DHTNode) ->\n                               log:log(\"right renew lease with ~w ~w\", [_Id, Lease]),\n                               l_on_cseq:lease_renew(Lease, active),\n                               wait_for_lease_version(l_on_cseq:get_id(Lease),\n                                                      l_on_cseq:get_epoch(Lease),\n                                                      l_on_cseq:get_version(Lease)+1)\n                  end,\n    WaitRightLeaseF = fun (Id, Lease) ->\n                              OldEpoch   = l_on_cseq:get_epoch(Lease),\n                              wait_for_lease_version(Id, OldEpoch+2, 0)\n                     end,\n    WaitLeftLeaseF = fun (Id) ->\n                             wait_for_lease_version(Id, 2, 0)\n                      end,\n    FinalWaitF = fun wait_for_split_success_msg/0,\n    test_split_helper_for_4_steps(_Config,\n                                  NullF, NullF, RenewLeaseLeftF, RenewLeaseRightF,\n                                  WaitLeftLeaseF, WaitRightLeaseF, FinalWaitF),\n    true.\n\ntest_split_but_lease_already_exists(_Config) ->\n    ContentCheck =\n        fun (Current, _WriteFilter, _Next) ->\n                case Current == prbr_bottom of\n                    true ->\n                        {true, null};\n                    false ->\n                        {false, lease_already_exists}\n                end\n        end,\n    CreateLeaseF =\n        fun(LeftId) ->\n                New = l_on_cseq:set_version(\n                        l_on_cseq:set_epoch(\n                          l_on_cseq:unittest_create_lease(LeftId),\n                          47),\n                        11),\n                DB = rbrcseq:get_db_for_id(lease_db, LeftId),\n                rbrcseq:qwrite(DB, self(), LeftId, l_on_cseq,\n                               ContentCheck,\n                               New),\n                receive\n                    {qwrite_done, _ReqId, _Round, _, _} -> ok;\n                    X -> ct:pal(\"wrong message ~p\", [X]),\n                          timer:sleep(4000)\n                end\n        end,\n    WaitRightLeaseF = fun (Id, Lease) ->\n                             OldEpoch = l_on_cseq:get_epoch(Lease),\n                             OldVersion = l_on_cseq:get_version(Lease),\n                             wait_for_lease_version(Id, OldEpoch, OldVersion)\n                     end,\n    WaitLeftLeaseF = fun (Id) ->\n                             wait_for_lease_version(Id, 47, 11)\n                      end,\n    FinalWaitF = fun wait_for_split_fail_msg/0,\n    test_split_helper_for_1_step(_Config,\n                                 CreateLeaseF,\n                                 WaitLeftLeaseF, WaitRightLeaseF, FinalWaitF),\n    true.\n\ntest_split_with_owner_change_in_step1(_Config) ->\n    ChangeOwnerF =\n        fun (Id, Lease, DHTNode) ->\n                ct:pal(\"changing owner: ~p ~p\", [Id, Lease]),\n                New = l_on_cseq:set_owner(\n                        l_on_cseq:set_timeout(\n                          l_on_cseq:set_version(\n                            l_on_cseq:set_epoch(Lease, l_on_cseq:get_epoch(Lease)+1),\n                            0)),\n                        comm:this()),\n                l_on_cseq:unittest_lease_update(Lease, New, active, DHTNode)\n        end,\n    NullF = fun (_Id, _Lease, _DHTNode) -> ok end,\n    WaitRightLeaseF = fun wait_for_delete/2,\n    WaitLeftLeaseF = fun (_Id) -> ok end,\n                            % we cannot read the left lease anymore, because\n                            % consistent routing will prevent the delivery of\n                            % messages\n\n    %fun (Id) ->\n    %        wait_for_lease_version(Id, 1, 0)\n    %end,\n    FinalWaitF = fun wait_for_split_fail_msg/0,\n    test_split_helper_for_2_steps(_Config,\n                                  NullF, ChangeOwnerF,\n                                  WaitLeftLeaseF, WaitRightLeaseF, FinalWaitF),\n    true.\n\ntest_split_with_owner_change_in_step2(Config) ->\n    ChangeOwnerF =\n        fun (Id, Lease, DHTNode) ->\n                ct:pal(\"changing owner: ~p ~p\", [Id, Lease]),\n                New = l_on_cseq:set_owner(\n                        l_on_cseq:set_timeout(\n                          l_on_cseq:set_version(\n                            l_on_cseq:set_epoch(Lease, l_on_cseq:get_epoch(Lease)+1),\n                            0)),\n                        comm:this()),\n                l_on_cseq:unittest_lease_update(Lease, New, passive, DHTNode)\n        end,\n    NullF = fun (_Id, _Lease, _DHTNode) -> ok end,\n    WaitRightLeaseF = fun (_Id, _Lease) -> ok end,\n                            % we cannot read the left lease anymore, because\n                            % consistent routing will prevent the delivery of\n                            % messages\n    %fun (Id, Lease) ->\n    %        OldEpoch = l_on_cseq:get_epoch(Lease),\n    %        wait_for_lease_version(Id, OldEpoch + 1, 0)\n    %end,\n    WaitLeftLeaseF = fun wait_for_delete/1,\n    FinalWaitF = fun wait_for_split_fail_msg/0,\n    test_split_helper_for_3_steps(Config,\n                                  NullF, NullF, ChangeOwnerF,\n                                  WaitLeftLeaseF, WaitRightLeaseF, FinalWaitF),\n    true.\n\ntest_split_with_owner_change_in_step3(Config) ->\n    ChangeOwnerF =\n        fun (Id, Lease, DHTNode) ->\n                ct:pal(\"changing owner: ~p ~p\", [Id, Lease]),\n                New = l_on_cseq:set_owner(\n                        l_on_cseq:set_timeout(\n                          l_on_cseq:set_version(\n                            l_on_cseq:set_epoch(Lease, l_on_cseq:get_epoch(Lease)+1),\n                            0)),\n                        comm:this()),\n                l_on_cseq:unittest_lease_update(Lease, New, active, DHTNode)\n        end,\n    NullF = fun (_Id, _Lease, _DHTNode) -> ok end,\n    WaitLeftLeaseF = fun (_Id) -> ok end,\n                            % we cannot read the left lease anymore, because\n                            % consistent routing will prevent the delivery of\n                            % messages\n    %fun (Id) ->\n    %        wait_for_lease_version(Id, 2, 0)\n    %end,\n    WaitRightLeaseF = fun wait_for_delete/2,\n    FinalWaitF = fun wait_for_split_fail_msg/0,\n    test_split_helper_for_4_steps(Config,\n                                  NullF, NullF, NullF, ChangeOwnerF,\n                                  WaitLeftLeaseF, WaitRightLeaseF, FinalWaitF),\n    true.\n\ntest_split_with_aux_change_in_step1(_Config) ->\n    ChangeOwnerF =\n        fun (Id, Lease, DHTNode) ->\n                ct:pal(\"changing aux: ~p ~p\", [Id, Lease]),\n                New = l_on_cseq:set_aux(\n                        l_on_cseq:set_timeout(\n                          l_on_cseq:set_version(\n                            l_on_cseq:set_epoch(Lease, l_on_cseq:get_epoch(Lease)+1),\n                            0)),\n                        {invalid, merge, intervals:empty(), intervals:empty()}),\n                l_on_cseq:unittest_lease_update(Lease, New, passive, DHTNode)\n        end,\n    NullF = fun (_Id, _Lease, _DHTNode) -> ok end,\n    WaitRightLeaseF = fun (Id, Lease) ->\n                             wait_for_lease_version(Id, l_on_cseq:get_epoch(Lease) + 1, 0)\n                      end,\n    WaitLeftLeaseF = fun (Id) ->\n                             wait_for_lease_version(Id, 1, 0)\n                      end,\n    FinalWaitF = fun wait_for_split_fail_msg/0,\n    test_split_helper_for_2_steps(_Config,\n                                  NullF, ChangeOwnerF,\n                                  WaitLeftLeaseF, WaitRightLeaseF, FinalWaitF),\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% handover unit tests\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ntest_handover(_Config) ->\n    ModifyF = fun(Old) -> Old end,\n    WaitF = fun (Id, _Lease) ->\n                    wait_for_lease_owner(Id, comm:this()),\n                    receive\n                        {handover, success, _} -> ok\n                    end\n            end,\n    test_handover_helper(_Config, ModifyF, WaitF),\n    true.\n\ntest_handover_with_concurrent_renew(_Config) ->\n    ModifyF = fun(Old) ->\n                      l_on_cseq:set_version(\n                        l_on_cseq:set_epoch(Old, l_on_cseq:get_epoch(Old)+1),\n                        0)\n                      end,\n    WaitF = fun (Id, _Lease) ->\n                    wait_for_lease_owner(Id, comm:this()),\n                    receive\n                        {handover, success, _} -> ok\n                    end\n            end,\n    test_handover_helper(_Config, ModifyF, WaitF),\n    true.\n\ntest_handover_with_concurrent_aux_change(_Config) ->\n    ModifyF = fun(Old) ->\n                      l_on_cseq:set_aux(\n                        l_on_cseq:set_version(\n                          l_on_cseq:set_epoch(Old, l_on_cseq:get_epoch(Old)+1),\n                          0),\n                        {valid, merge, foo, bar})\n                      end,\n    WaitF = fun (_Id, _Lease) ->\n                    receive\n                        {handover, failed, _} -> ok\n                    end\n            end,\n    test_handover_helper(_Config, ModifyF, WaitF),\n    true.\n\ntest_handover_with_concurrent_owner_change(_Config) ->\n    ModifyF = fun(Old) ->\n                      l_on_cseq:set_owner(\n                        l_on_cseq:set_version(\n                          l_on_cseq:set_epoch(Old, l_on_cseq:get_epoch(Old)+1),\n                          0),\n                        comm:this())\n                      end,\n    WaitF = fun (_Id, _Lease) ->\n                    receive\n                        {handover, failed, _} -> ok\n                    end\n            end,\n    test_handover_helper(_Config, ModifyF, WaitF),\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% takeover unit tests\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ntest_takeover(_Config) ->\n    log:log(\"start test_takeover\"),\n    ModifyF = fun(Old) -> Old end,\n    WaitF = fun (Id, _Lease, OriginalOwner) ->\n                    ct:pal(\"takeover: wait_for_lease_owner ~p\", [OriginalOwner]),\n                    wait_for_lease_owner(Id, OriginalOwner),\n                    ct:pal(\"takeover: wait_for_lease_owner done\")\n            end,\n    test_takeover_helper(_Config, ModifyF, WaitF),\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% takeover helper\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ntest_takeover_helper(_Config, ModifyF, WaitF) ->\n    DHTNode = pid_groups:find_a(dht_node),\n    pid_groups:join(pid_groups:group_of(DHTNode)),\n\n    % intercept lease renew\n    {l_on_cseq, renew, Old, _Mode} = lease_helper:intercept_lease_renew(DHTNode),\n    %OriginalOwner = l_on_cseq:get_owner(Old),\n    ct:pal(\"takeover: old lease ~p\", [Old]),\n    Id = l_on_cseq:get_id(Old),\n    % now we change the owner of the lease\n    l_on_cseq:lease_handover(Old, comm:this(), self()),\n    ct:pal(\"new owner ~p\", [comm:this()]),\n    HandoverWaitF = fun (_Id, _Lease) ->\n                            wait_for_lease_owner(_Id, comm:this()),\n                            receive\n                                {handover, success, _} -> ok\n                            end\n                    end,\n    HandoverWaitF(Id, Old),\n    ct:pal(\"takeover: now we update the lease\"),\n    % now we update the lease\n    {ok, Current} = l_on_cseq:read(Id),\n    ct:pal(\"takeover: current lease: ~p\", [Current]),\n    New = ModifyF(Current),\n    case New =/= Current of\n        true ->\n            Res = l_on_cseq:unittest_lease_update(Current, New, active, DHTNode),\n            ct:pal(\"takeover: lease_update: ~p (~p -> ~p)\", [Res, Current, New]),\n            wait_for_lease(New);\n        false ->\n            ok\n    end,\n    ct:pal(\"takeover: takeover\"),\n    % now the error handling of lease_takeover is going to be tested\n    takeover_loop(Current),\n    ct:pal(\"takeover: wait_for_lease2\"),\n    WaitF(Id, Current, comm:make_global(DHTNode)),\n    ct:pal(\"takeover: done\"),\n    true.\n\ntakeover_loop(L) ->\n    l_on_cseq:lease_takeover(L, self()),\n    M = receive\n            {takeover, _ , _} = _M -> _M;\n            {takeover, _ , _, _} = _M -> _M\n        end,\n    case M of\n        {takeover, success, L2} ->\n            ct:pal(\"takeover succeed ~w\", [L2]),\n            ok;\n        {takeover, failed, L2, _Result} ->\n            ct:pal(\"retrying takeover ~p ~p\", [L2, l_on_cseq:get_pretty_timeout(L2)]),\n            %% we repeat until the lease expired and then hopefully succeed\n            timer:sleep(500),\n            takeover_loop(L2)\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% handover helper\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ntest_handover_helper(_Config, ModifyF, WaitF) ->\n    DHTNode = pid_groups:find_a(dht_node),\n    pid_groups:join(pid_groups:group_of(DHTNode)),\n\n    % intercept lease renew\n    {l_on_cseq, renew, Old, _Mode} = lease_helper:intercept_lease_renew(DHTNode),\n    Id = l_on_cseq:get_id(Old),\n    % now we update the lease\n    New = ModifyF(Old),\n    l_on_cseq:unittest_lease_update(Old, New, active, DHTNode),\n    wait_for_lease(New),\n    % now the error handling of lease_handover is going to be tested\n    l_on_cseq:lease_handover(Old, comm:this(), self()),\n    WaitF(Id, Old),\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% split helper\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ntest_split_prepare(DHTNode) ->\n    % intercept lease renew\n    {l_on_cseq, renew, _Old, _Mode} = lease_helper:intercept_lease_renew(DHTNode),\n    % prepeare split\n    comm:send_local(DHTNode, {get_state, comm:this(), lease_list}),\n    L = receive\n            {get_state_response, LeaseList} ->\n                lease_list:get_active_lease(LeaseList)\n             end,\n    {ok, R1, R2} = l_on_cseq:split_range(l_on_cseq:get_range(L)),\n    log:log(\"split under test:~n~w~n~w~n~w~n\", [l_on_cseq:get_range(L), R1, R2]),\n    %[R1, R2] = intervals:split(l_on_cseq:get_range(L), 2),\n    LeftId = l_on_cseq:id(R1),\n    RightId = l_on_cseq:id(R2),\n    intercept_split_request(DHTNode),                                   % install intercepts\n    intercept_split_reply(DHTNode, split_reply_step1),                    %\n    intercept_split_reply(DHTNode, split_reply_step2),                    %\n    intercept_split_reply(DHTNode, split_reply_step3),                    %\n    intercept_split_reply(DHTNode, split_reply_step4),                    %\n\n    % step1\n    log:log(\"starting the split under test\"),\n    l_on_cseq:lease_split(L, R1, R2, self()),                    % trigger step\n    ct:pal(\"intercepting msg\"),\n    StartMsg = receive                                           % intercept msg\n                   M = {l_on_cseq, split, _Lease, __R1, __R2,\n                        __ReplyTo, __PostAux} ->\n                       M\n               end,\n    ct:pal(\"intercepted msg\"),\n    {l_on_cseq, split, Lease, _R1, _R2, _ReplyTo, _PostAux} = StartMsg,\n    {Lease, LeftId, RightId, StartMsg}.\n\n\ntest_split_helper_for_1_step(_Config,\n                             ModifyBeforeStep1,\n                             WaitLeftLease, WaitRightLease, FinalWaitF) ->\n    DHTNode = pid_groups:find_a(dht_node),\n    pid_groups:join(pid_groups:group_of(DHTNode)),\n    {Lease, LeftId, RightId, StartMsg} = test_split_prepare(DHTNode),\n    ModifyBeforeStep1(LeftId),                           % modify world\n    gen_component:bp_del(DHTNode, block_split_request),\n    gen_component:bp_del(DHTNode, split_reply_step1),\n    comm:send_local(DHTNode, StartMsg),                          % release msg\n    % wait for result\n    ct:pal(\"wait left\"),\n    WaitLeftLease(LeftId),\n    ct:pal(\"wait right\"),\n    WaitRightLease(RightId, Lease),\n    FinalWaitF().\n\ntest_split_helper_for_2_steps(_Config,\n                              ModifyBeforeStep1,\n                              ModifyBeforeStep2,\n                              WaitLeftLease, WaitRightLease, FinalWaitF) ->\n    DHTNode = pid_groups:find_a(dht_node),\n    pid_groups:join(pid_groups:group_of(DHTNode)),\n    {Lease, LeftId, RightId, StartMsg} = test_split_prepare(DHTNode),\n    ct:pal(\"0\"),\n    ModifyBeforeStep1(LeftId, Lease, DHTNode),                   % modify world\n    gen_component:bp_del(DHTNode, block_split_request),\n    comm:send_local(DHTNode, StartMsg),                          % release msg\n    % step 2\n    split_helper_do_step(DHTNode, split_reply_step1, ModifyBeforeStep2, RightId),\n    wait_for_split_message(DHTNode, split_reply_step2),\n    % wait for result\n    ct:pal(\"wait left\"),\n    WaitLeftLease(LeftId),\n    ct:pal(\"wait right\"),\n    WaitRightLease(RightId, Lease),\n    FinalWaitF().\n\ntest_split_helper_for_3_steps(_Config,\n                              ModifyBeforeStep1,\n                              ModifyBeforeStep2,\n                              ModifyBeforeStep3,\n                              WaitLeftLease, WaitRightLease, FinalWaitF) ->\n    DHTNode = pid_groups:find_a(dht_node),\n    pid_groups:join(pid_groups:group_of(DHTNode)),\n    {Lease, LeftId, RightId, StartMsg} = test_split_prepare(DHTNode),\n    ModifyBeforeStep1(RightId, Lease, DHTNode),                  % modify world\n    gen_component:bp_del(DHTNode, block_split_request),\n    comm:send_local(DHTNode, StartMsg),                          % release msg\n    % step 2\n    split_helper_do_step(DHTNode, split_reply_step1, ModifyBeforeStep2, LeftId),\n    % step 3\n    split_helper_do_step(DHTNode, split_reply_step2, ModifyBeforeStep3, LeftId),\n    wait_for_split_message(DHTNode, split_reply_step3),\n    % wait for result\n    log:pal(\"wait left\"),\n    WaitLeftLease(LeftId),\n    log:pal(\"wait right\"),\n    WaitRightLease(RightId, Lease),\n    FinalWaitF().\n\ntest_split_helper_for_4_steps(_Config,\n                              ModifyBeforeStep1,\n                              ModifyBeforeStep2,\n                              ModifyBeforeStep3,\n                              ModifyBeforeStep4,\n                              WaitLeftLease, WaitRightLease, FinalWaitF) ->\n    DHTNode = pid_groups:find_a(dht_node),\n    pid_groups:join(pid_groups:group_of(DHTNode)),\n    {Lease, LeftId, RightId, StartMsg} = test_split_prepare(DHTNode),\n    log:log(\"left and right-id:~w~n~w~n\", [LeftId, RightId]),\n    ModifyBeforeStep1(RightId, Lease, DHTNode),                  % modify world\n    gen_component:bp_del(DHTNode, block_split_request),\n    comm:send_local(DHTNode, StartMsg),                          % release msg\n    % step2\n    split_helper_do_step(DHTNode, split_reply_step1, ModifyBeforeStep2, LeftId),\n    log:log(\"finished step2\"),\n    % step3\n    split_helper_do_step(DHTNode, split_reply_step2, ModifyBeforeStep3, RightId),\n    log:log(\"finished step3\"),\n    % step4\n    split_helper_do_step(DHTNode, split_reply_step3, ModifyBeforeStep4, LeftId),\n    log:log(\"finished step4\"),\n    wait_for_split_message(DHTNode, split_reply_step4),\n    log:log(\"got split message\"),\n    % wait for result\n    ct:pal(\"wait left\"),\n    WaitLeftLease(LeftId),\n    ct:pal(\"wait right\"),\n    WaitRightLease(RightId, Lease),\n    FinalWaitF().\n\nsplit_helper_do_step(DHTNode, StepTag, ModifyBeforeStep, Id) ->\n    log:pal(\"doing ~p\", [StepTag]),\n    ReplyMsg = receive\n                   M = {l_on_cseq, StepTag, Lease, _R1, _R2, _ReplyTo, _PostAux, _Resp} ->\n                       M\n               end,\n    ModifyBeforeStep(Id, Lease, DHTNode),\n    gen_component:bp_del(DHTNode, StepTag),\n    watch_message(DHTNode, ReplyMsg).\n\nwait_for_split_message(DHTNode, StepTag) ->\n    log:pal(\"waiting for ~p\", [StepTag]),\n    receive\n        M = {l_on_cseq, StepTag, _Lease, _R1, _R2, _ReplyTo, _PostAux, _Resp} ->\n            %log:pal(\"got ~p\", [M]),\n            gen_component:bp_del(DHTNode, StepTag),\n            watch_message(DHTNode, M)\n    end.\n\nwait_for_split_success_msg() ->\n    log:pal(\"wait_for_split_success_msg() ~p\", [self()]),\n    receive\n        {split, success, _, _} ->\n            ok\n    end.\n\nwait_for_split_fail_msg() ->\n    receive\n        {split, fail, _} ->\n            ok\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% renew helper\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ntest_renew_helper(_Config, ModifyF, WaitF) ->\n    %% pid_groups:join(pid_groups:group_with(dht_node)),\n    DHTNode = pid_groups:find_a(dht_node),\n\n    % intercept lease renew\n    M = {l_on_cseq, renew, Old, _Mode} = lease_helper:intercept_lease_renew(DHTNode),\n    Id = l_on_cseq:get_id(Old),\n    % now we update the lease\n    New = ModifyF(Old),\n    l_on_cseq:unittest_lease_update(Old, New, active, DHTNode),\n    wait_for_lease(New),\n    % now the error handling of lease_renew is going to be tested\n    comm:send_local(DHTNode, M),\n    WaitF(Id, Old),\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% wait helper\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nwait_for(F) ->\n    case F() of\n        true ->\n            ok;\n        false ->\n            wait_for(F)\n    end.\n\nwait_for_lease(Lease) ->\n    Id = l_on_cseq:get_id(Lease),\n    wait_for_lease_helper(Id, fun (L) -> L == Lease end).\n\nwait_for_lease_version(Id, Epoch, Version) ->\n    ct:pal(\"wait_for_lease_version ~p\", [Id]),\n    wait_for_lease_helper(Id,\n                          fun (Lease) ->\n                                  ct:pal(\"want ~p:~p; have ~p:~p\", [Epoch, Version, l_on_cseq:get_epoch(Lease), l_on_cseq:get_version(Lease)]),\n                                  Epoch   == l_on_cseq:get_epoch(Lease)\n                          andalso Version == l_on_cseq:get_version(Lease)\n                          end).\n\nwait_for_lease_owner(Id, NewOwner) ->\n    wait_for_lease_helper(Id,\n                          fun (Lease) ->\n                                  NewOwner   == l_on_cseq:get_owner(Lease)\n                          end).\n\nwait_for_lease_helper(Id, F) ->\n    wait_for(fun () ->\n                     %DHTNode = pid_groups:find_a(dht_node),\n                     %comm:send_local(DHTNode, {get_state, comm:this(), lease_list}),\n                     %{A, P} = receive\n                     %        {get_state_response, {ActiveList, PassiveList}} ->\n                     %            {ActiveList, PassiveList}\n                     %    end,\n                     %ct:pal(\"~p ~p\", [A, P]),\n                     case l_on_cseq:read(Id) of\n                         {ok, Lease} ->\n                             F(Lease);\n                         _ ->\n                             false\n                     end\n             end).\n\nget_dht_node_state(Pid, What) ->\n    comm:send_local(Pid, {get_state, comm:this(), What}),\n    receive\n        {get_state_response, Data} ->\n            Data\n    end.\n\nget_all_active_leases() ->\n    [ get_active_lease(DHTNode) || DHTNode <- pid_groups:find_all(dht_node) ].\n\nget_active_lease(Pid) ->\n    LeaseList = get_dht_node_state(Pid, lease_list),\n    lease_list:get_active_lease(LeaseList).\n\nwait_for_simple_update(Id, Old) ->\n    OldVersion = l_on_cseq:get_version(Old),\n    OldEpoch   = l_on_cseq:get_epoch(Old),\n    wait_for_lease_version(Id, OldEpoch, OldVersion+1).\n\nwait_for_epoch_update(Id, Old) ->\n    OldEpoch   = l_on_cseq:get_epoch(Old),\n    wait_for_lease_version(Id, OldEpoch+1, 0).\n\nwait_for_delete(Id, _Old) ->\n    DHTNode = pid_groups:find_a(dht_node),\n    ct:pal(\"wait_for_delete ~p\", [Id]),\n    wait_for(fun () ->\n                     LeaseList = get_dht_node_state(DHTNode, lease_list),\n                     L = lease_list:get_active_lease(LeaseList),\n                     case L of\n                         empty ->\n                             true;\n                         _ ->\n                             l_on_cseq:get_id(L) =/= Id\n                     end\n             end).\n\nwait_for_delete(Id) ->\n    ct:pal(\"wait_for_delete ~p\", [Id]),\n    DHTNode = pid_groups:find_a(dht_node),\n    wait_for(fun () ->\n                     LeaseList = get_dht_node_state(DHTNode, lease_list),\n                     L = lease_list:get_active_lease(LeaseList),\n                     case L of\n                         empty ->\n                             true;\n                         _ ->\n                             l_on_cseq:get_id(L) =/= Id\n                     end\n             end).\n\nwait_for_number_of_leases(Nr) ->\n    wait_for(fun() ->\n                     length(get_all_active_leases()) == Nr\n             end).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intercepting and blocking\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nwatch_message(Pid, Message) ->\n    gen_component:bp_set_cond(Pid, block_message(self(), Message), watch_message),\n    comm:send_local(Pid, Message),\n    receive\n        {saw_message} ->\n            _ = gen_component:bp_step(Pid),\n            gen_component:bp_del(Pid, watch_message),\n            gen_component:bp_cont(Pid)\n    end.\n\nintercept_split_request(DHTNode) ->\n    % we wait for the next periodic trigger\n    gen_component:bp_set_cond(DHTNode, block_split_request(self()), block_split_request).\n\nintercept_split_reply(DHTNode, StepTag) ->\n    % we wait for the next periodic trigger\n    gen_component:bp_set_cond(DHTNode, block_split_reply(self(), StepTag), StepTag).\n\nblock_message(Pid, WatchedMessage) ->\n    fun (Message, _State) ->\n            case Message of\n                WatchedMessage ->\n                    comm:send_local(Pid, {saw_message}),\n                    true;\n                _ ->\n                    false\n            end\n    end.\n\nblock_split_request(Pid) ->\n    fun (Message, _State) ->\n            case Message of\n                {l_on_cseq, split, _Lease, _R1, _R2, _ReplyTo, _PostAux} ->\n                    comm:send_local(Pid, Message),\n                    drop_single;\n                _ ->\n                    false\n            end\n    end.\n\nblock_split_reply(Pid, StepTag) ->\n    fun (Message, _State) ->\n            case Message of\n                {l_on_cseq, StepTag, _Lease, _R1, _R2, _ReplyTo, _PostAux, _Resp} ->\n                    comm:send_local(Pid, Message),\n                    drop_single;\n                _ ->\n                    false\n            end\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% utility functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nobfuscated_intervals_all() ->\n    [{'(',0,5,']'},\n     {0},\n     {'(',5,340282366920938463463374607431768211456,')'}\n    ].\n"
  },
  {
    "path": "test/lease_helper.erl",
    "content": "%% @copyright 2012-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for slide_leases\n%% @end\n%% @version $Id$\n-module(lease_helper).\n-author('schuett@zib.de').\n-vsn('$Id').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\n-export([wait_for_correct_leases/1,\n         wait_for_ring_size/1,\n         wait_for_correct_ring/0,\n         print_all_active_leases/0,\n         print_all_passive_leases/0,\n         intercept_lease_renew/1\n        ]).\n\n-spec wait_for_correct_leases(pos_integer()) -> ok.\nwait_for_correct_leases(TargetSize) ->\n    util:wait_for(fun () -> admin:check_leases(TargetSize) end, 1000),\n    ct:pal(\"have correct leases\"),\n    util:wait_for(fun check_leases_per_node/0, 1000),\n    ct:pal(\"have correct leases_per_node\"),\n    ok.\n\n-spec wait_for_ring_size(pos_integer()) -> ok.\nwait_for_ring_size(Size) ->\n    util:wait_for(fun () ->\n                          Nodes = api_vm:number_of_nodes(),\n                          log:log(\"ring size: expected ~w, found ~w\", [Size, Nodes]),\n                          Nodes == Size end).\n\n-spec wait_for_correct_ring() -> ok.\nwait_for_correct_ring() ->\n    util:wait_for(fun () -> ct:pal(\"->admin:check_ring_deep()\"),\n                            Res = admin:check_ring_deep(),\n                            ct:pal(\"<-admin:check_ring_deep()\"),\n                            Res == ok\n             end).\n\n-spec wait_for_number_of_valid_active_leases(integer()) -> ok.\nwait_for_number_of_valid_active_leases(Count) ->\n    util:wait_for(fun () ->\n                          AllLeases = get_all_leases(),\n                          ActiveLeases = [ lease_list:get_active_lease(LL) || LL <- AllLeases ],\n                          ValidLeases = [ L || L <- ActiveLeases, l_on_cseq:is_valid(L)],\n                          NrOfValidLeases = length(ValidLeases),\n                          log:log(\"active leases: expected ~w, found ~w\", [Count, NrOfValidLeases]),\n                          Count =:= NrOfValidLeases\n                  end).\n\n-spec print_all_active_leases() -> ok.\nprint_all_active_leases() ->\n    AllLeases = get_all_leases(),\n    ActiveLeases = [ lease_list:get_active_lease(LL) || LL <- AllLeases ],\n    log:log(\"active leases: ~w\", [ActiveLeases]),\n    ok.\n\n-spec print_all_passive_leases() -> ok.\nprint_all_passive_leases() ->\n    AllLeases = get_all_leases(),\n    PassiveLeases = [ lease_list:get_passive_leases(LL) || LL <- AllLeases ],\n    log:log(\"passive leases: ~w\", [PassiveLeases]),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% wait helper\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nget_dht_node_state(Pid, What) ->\n    comm:send_local(Pid, {get_state, comm:this(), What}),\n    receive\n        {get_state_response, Data} ->\n            Data\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% message blocking\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec intercept_lease_renew(comm:erl_local_pid_plain()) -> comm:message().\nintercept_lease_renew(DHTNode) ->\n    %DHTNode = pid_groups:find_a(dht_node),\n    % we wait for the next periodic trigger\n    gen_component:bp_set_cond(DHTNode, block_renew(self()), block_renew),\n    Msg = receive\n              M = {l_on_cseq, renew, _Lease, _Mode} ->\n                  M\n          end,\n    gen_component:bp_set_cond(DHTNode, block_trigger(self()), block_trigger),\n    gen_component:bp_del(DHTNode, block_renew),\n    Msg.\n\nblock_renew(Pid) ->\n    fun (Message, _State) ->\n            case Message of\n                {l_on_cseq, renew, _Lease, _Mode} ->\n                    comm:send_local(Pid, Message),\n                    drop_single;\n                _ ->\n                    false\n            end\n    end.\n\nblock_trigger(Pid) ->\n    fun (Message, _State) ->\n            case Message of\n                {l_on_cseq, renew_leases} ->\n                    comm:send_local(Pid, Message),\n                    drop_single;\n                _ ->\n                    false\n            end\n    end.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% utility functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nget_all_leases() ->\n    [ get_leases(DHTNode) || DHTNode <- pid_groups:find_all(dht_node) ].\n\nget_leases(Pid) ->\n    get_dht_node_state(Pid, lease_list).\n\ncheck_leases_per_node() ->\n    lists:all(fun (B) -> B end, [ check_local_leases(DHTNode) || DHTNode <- pid_groups:find_all(dht_node) ]).\n\n-spec is_disjoint([intervals:interval()]) -> boolean().\nis_disjoint([]) ->\n    true;\nis_disjoint([H | T]) ->\n    is_disjoint(H, T) andalso\n        is_disjoint(T).\n\n-spec is_disjoint(intervals:interval(), [intervals:interval()]) -> boolean().\nis_disjoint(_I, []) ->\n    true;\nis_disjoint(I, [H | T]) ->\n    intervals:is_empty(intervals:intersection(I, H))\n        andalso is_disjoint(I, T).\n\n\ncheck_local_leases(DHTNode) ->\n    LeaseList = get_dht_node_state(DHTNode, lease_list),\n    ActiveLease = lease_list:get_active_lease(LeaseList),\n    PassiveLeases = lease_list:get_passive_leases(LeaseList),\n    %log:log(\"active lease: ~w\", [ActiveLease]),\n    ActiveInterval = case ActiveLease of\n                         empty ->\n                             intervals:empty();\n                         _ ->\n                             l_on_cseq:get_range(ActiveLease)\n                     end,\n    MyRange = get_dht_node_state(DHTNode, my_range),\n    LocalCorrect = MyRange =:= ActiveInterval,\n    length(PassiveLeases) == 0 andalso LocalCorrect.\n"
  },
  {
    "path": "test/leases_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{suites, tests, [l_on_cseq_SUITE]}.\n{suites, tests, [mockup_l_on_cseq_SUITE]}.\n\n{suites, tests, [slide_leases_SUITE]}.\n{suites, tests, [rm_leases_SUITE]}.\n\n{suites, tests, [consistent_lookup_leases_SUITE]}.\n\n{suites, tests, [crash_recovery_SUITE]}.\n\n{suites, tests, [recover_mnesia_SUITE]}.\n"
  },
  {
    "path": "test/leases_proto_sched_SUITE.erl",
    "content": "% @copyright 2010-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schütt <schuett@zib.de>\n%% @doc    Unit tests for leases.\n%% @end\n%% @version $Id$\n\n-module(leases_proto_sched_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\ngroups() ->\n    [{join_tests, [sequence], [\n                                 test_single_join\n                              ]}\n    ].\n\nall() ->\n    [\n     {group, join_tests}\n     ].\n\nsuite() -> [ {timetrap, {seconds, 300}} ].\n\ngroup(join_tests) ->\n    [{timetrap, {seconds, 10}}];\ngroup(_) ->\n    suite().\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(_TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(4, [{config, [{log_path, PrivDir},\n                                            {leases, true}]}]),\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n-spec proto_sched_fun(start | stop) -> ok.\nproto_sched_fun(start) ->\n    proto_sched:thread_begin();\nproto_sched_fun(stop) ->\n    %% is a ring running?\n    case erlang:whereis(pid_groups) =:= undefined\n             orelse pid_groups:find_a(proto_sched) =:= failed of\n        true -> ok;\n        false ->\n            %% then finalize proto_sched run:\n            %% try to call thread_end(): if this\n            %% process was running the proto_sched\n            %% thats fine, otherwise thread_end()\n            %% will raise an exception\n            proto_sched:thread_end(),\n            proto_sched:wait_for_end()\n    end.\n\n-spec proto_sched2_fun(setup, ThreadNum::pos_integer()) -> ok;\n                      (cleanup, PIDs::[pid() | atom()]) -> ok.\nproto_sched2_fun(setup, Arg) ->\n    proto_sched:thread_num(Arg);\nproto_sched2_fun(cleanup, _Arg) ->\n    proto_sched:wait_for_end(),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed),\n    proto_sched:cleanup().\n\ntest_single_join(_Config) ->\n    proto_sched2_fun(setup, 1),\n    proto_sched_fun(start),\n    {[_], []} = api_vm:add_nodes(1),\n    lease_helper:wait_for_ring_size(5),\n    proto_sched_fun(stop),\n    proto_sched2_fun(cleanup, []),\n    ?assert(admin:check_leases()),\n    ok.\n"
  },
  {
    "path": "test/math_pos_SUITE.erl",
    "content": "%  @copyright 2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Test suite for the math_pos module.\n%% @end\n%% @version $Id$\n-module(math_pos_SUITE).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\nall() ->\n    [plus, minus, multiply, divide,\n     tester_make_same_length,\n     tester_plus_symm, tester_plus_valid,\n     tester_minus, tester_minus_valid,\n     tester_divide_valid, tester_divide_decimals,\n     tester_multiply_valid, tester_multiply_decimals].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 20}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\n\n-spec plus(Config::[tuple()]) -> true.\nplus(_Config) ->\n    ?equals(math_pos:plus(  [1],   [1], 10),   [2]),\n    ?equals(math_pos:plus(  [1],   [2], 10),   [3]),\n    ?equals(math_pos:plus(  [1],   [9], 10),   [0]),\n    ?equals(math_pos:plus([0,1], [0,9], 10), [1,0]),\n    ?equals(math_pos:plus(  [2],   [1], 10),   [3]),\n    ?equals(math_pos:plus(  [9],   [1], 10),   [0]),\n    ?equals(math_pos:plus([0,9], [0,1], 10), [1,0]).\n\n-spec minus(Config::[tuple()]) -> true.\nminus(_Config) ->\n    ?equals(math_pos:minus(  [1],   [1], 10),   [0]),\n    ?equals(math_pos:minus(  [1],   [2], 10),   [9]),\n    ?equals(math_pos:minus(  [1],   [9], 10),   [2]),\n    ?equals(math_pos:minus([0,1], [0,9], 10), [9,2]),\n    ?equals(math_pos:minus(  [2],   [1], 10),   [1]),\n    ?equals(math_pos:minus(  [9],   [1], 10),   [8]),\n    ?equals(math_pos:minus([0,9], [0,1], 10), [0,8]).\n\n-spec multiply(Config::[tuple()]) -> true.\nmultiply(_Config) ->\n    ?equals(math_pos:multiply(  [1], 2, 10),   [2]),\n    ?equals(math_pos:multiply(  [2], 2, 10),   [4]),\n    ?equals(math_pos:multiply(  [3], 2, 10),   [6]),\n    ?equals(math_pos:multiply(  [4], 2, 10),   [8]),\n    ?equals(math_pos:multiply(  [5], 2, 10),   [0]),\n    ?equals(math_pos:multiply(  [6], 2, 10),   [2]),\n    ?equals(math_pos:multiply(  [7], 2, 10),   [4]),\n    ?equals(math_pos:multiply([0,1], 2, 10), [0,2]),\n    ?equals(math_pos:multiply([0,2], 2, 10), [0,4]),\n    ?equals(math_pos:multiply([0,3], 2, 10), [0,6]),\n    ?equals(math_pos:multiply([0,4], 2, 10), [0,8]),\n    ?equals(math_pos:multiply([0,5], 2, 10), [1,0]),\n    ?equals(math_pos:multiply([0,6], 2, 10), [1,2]),\n    ?equals(math_pos:multiply([0,7], 2, 10), [1,4]),\n    \n    ?equals(math_pos:multiply(  [1], 3, 10),   [3]),\n    ?equals(math_pos:multiply(  [2], 3, 10),   [6]),\n    ?equals(math_pos:multiply(  [3], 3, 10),   [9]),\n    ?equals(math_pos:multiply(  [4], 3, 10),   [2]),\n    ?equals(math_pos:multiply(  [5], 3, 10),   [5]),\n    ?equals(math_pos:multiply(  [6], 3, 10),   [8]),\n    ?equals(math_pos:multiply(  [7], 3, 10),   [1]),\n    ?equals(math_pos:multiply([0,1], 3, 10), [0,3]),\n    ?equals(math_pos:multiply([0,2], 3, 10), [0,6]),\n    ?equals(math_pos:multiply([0,3], 3, 10), [0,9]),\n    ?equals(math_pos:multiply([0,4], 3, 10), [1,2]),\n    ?equals(math_pos:multiply([0,5], 3, 10), [1,5]),\n    ?equals(math_pos:multiply([0,6], 3, 10), [1,8]),\n    ?equals(math_pos:multiply([0,7], 3, 10), [2,1]),\n    \n    ?equals(math_pos:multiply(  [1], 15, 10),   [5]),\n    ?equals(math_pos:multiply(  [2], 15, 10),   [0]),\n    ?equals(math_pos:multiply(  [3], 15, 10),   [5]),\n    ?equals(math_pos:multiply(  [4], 15, 10),   [0]),\n    ?equals(math_pos:multiply(  [5], 15, 10),   [5]),\n    ?equals(math_pos:multiply(  [6], 15, 10),   [0]),\n    ?equals(math_pos:multiply(  [7], 15, 10),   [5]),\n    ?equals(math_pos:multiply([0,1], 15, 10), [1,5]),\n    ?equals(math_pos:multiply([0,2], 15, 10), [3,0]),\n    ?equals(math_pos:multiply([0,3], 15, 10), [4,5]),\n    ?equals(math_pos:multiply([0,4], 15, 10), [6,0]),\n    ?equals(math_pos:multiply([0,5], 15, 10), [7,5]),\n    ?equals(math_pos:multiply([0,6], 15, 10), [9,0]),\n    ?equals(math_pos:multiply([0,7], 15, 10), [0,5]),\n    \n    ?equals(math_pos:multiply(  [1], 15, 10, enlarge),   {[1,5],1}),\n    ?equals(math_pos:multiply(  [2], 15, 10, enlarge),   {[3,0],1}),\n    ?equals(math_pos:multiply(  [3], 15, 10, enlarge),   {[4,5],1}),\n    ?equals(math_pos:multiply(  [4], 15, 10, enlarge),   {[6,0],1}),\n    ?equals(math_pos:multiply(  [5], 15, 10, enlarge),   {[7,5],1}),\n    ?equals(math_pos:multiply(  [6], 15, 10, enlarge),   {[9,0],1}),\n    ?equals(math_pos:multiply(  [7], 15, 10, enlarge), {[1,0,5],2}),\n    ?equals(math_pos:multiply([0,1], 15, 10, enlarge),   {[1,5],0}),\n    ?equals(math_pos:multiply([0,2], 15, 10, enlarge),   {[3,0],0}),\n    ?equals(math_pos:multiply([0,3], 15, 10, enlarge),   {[4,5],0}),\n    ?equals(math_pos:multiply([0,4], 15, 10, enlarge),   {[6,0],0}),\n    ?equals(math_pos:multiply([0,5], 15, 10, enlarge),   {[7,5],0}),\n    ?equals(math_pos:multiply([0,6], 15, 10, enlarge),   {[9,0],0}),\n    ?equals(math_pos:multiply([0,7], 15, 10, enlarge), {[1,0,5],1}).\n\n-spec divide(Config::[tuple()]) -> true.\ndivide(_Config) ->\n    ?equals(math_pos:divide(  [1], 2, 10),   [0]),\n    ?equals(math_pos:divide(  [2], 2, 10),   [1]),\n    ?equals(math_pos:divide(  [3], 2, 10),   [1]),\n    ?equals(math_pos:divide(  [4], 2, 10),   [2]),\n    ?equals(math_pos:divide([1,0], 2, 10), [0,5]),\n    ?equals(math_pos:divide([1,1], 2, 10), [0,5]),\n    ?equals(math_pos:divide([1,2], 2, 10), [0,6]),\n    ?equals(math_pos:divide([1,3], 2, 10), [0,6]),\n    ?equals(math_pos:divide([1,4], 2, 10), [0,7]),\n    ?equals(math_pos:divide([2,0], 2, 10), [1,0]),\n    ?equals(math_pos:divide([2,1], 2, 10), [1,0]),\n    ?equals(math_pos:divide([2,2], 2, 10), [1,1]),\n    ?equals(math_pos:divide([2,3], 2, 10), [1,1]),\n    ?equals(math_pos:divide([2,4], 2, 10), [1,2]),\n    \n    ?equals(math_pos:divide(  [1], 3, 10),   [0]),\n    ?equals(math_pos:divide(  [2], 3, 10),   [0]),\n    ?equals(math_pos:divide(  [3], 3, 10),   [1]),\n    ?equals(math_pos:divide(  [4], 3, 10),   [1]),\n    ?equals(math_pos:divide([1,0], 3, 10), [0,3]),\n    ?equals(math_pos:divide([1,1], 3, 10), [0,3]),\n    ?equals(math_pos:divide([1,2], 3, 10), [0,4]),\n    ?equals(math_pos:divide([1,3], 3, 10), [0,4]),\n    ?equals(math_pos:divide([1,4], 3, 10), [0,4]),\n    ?equals(math_pos:divide([2,0], 3, 10), [0,6]),\n    ?equals(math_pos:divide([2,1], 3, 10), [0,7]),\n    ?equals(math_pos:divide([3,0], 3, 10), [1,0]),\n    ?equals(math_pos:divide([3,1], 3, 10), [1,0]),\n    ?equals(math_pos:divide([3,2], 3, 10), [1,0]),\n    ?equals(math_pos:divide([3,3], 3, 10), [1,1]),\n    ?equals(math_pos:divide([3,4], 3, 10), [1,1]),\n    ?equals(math_pos:divide([3,5], 3, 10), [1,1]),\n    ?equals(math_pos:divide([3,6], 3, 10), [1,2]).\n\n%% make_same_length\n\n-spec prop_make_same_length(A::string(), B::string(), front | back) -> true.\nprop_make_same_length(A, B, Pos) ->\n    {A1, B1, ALen, BLen, A1Added, B1Added} = math_pos:make_same_length(A, B, Pos),\n    ?equals(ALen, erlang:length(A)),\n    ?equals(BLen, erlang:length(B)),\n    A1Len = erlang:length(A1),\n    ?equals(A1Len, erlang:length(B1)),\n    ?equals(ALen + A1Added, A1Len),\n    ?equals(math_pos:remove_zeros(A1, Pos, all), math_pos:remove_zeros(A, Pos, all)),\n    ?equals(math_pos:remove_zeros(B1, Pos, all), math_pos:remove_zeros(B, Pos, all)),\n    ?equals(math_pos:remove_zeros(A1, Pos, A1Added), A),\n    ?equals(math_pos:remove_zeros(B1, Pos, B1Added), B).\n\n-spec tester_make_same_length(Config::[tuple()]) -> ok.\ntester_make_same_length(_Config) ->\n    tester:test(?MODULE, prop_make_same_length, 3, 10000, [{threads, 2}]).\n\n%% plus\n\n-spec prop_plus_valid_base(X, X, Pos::front | back, Base::pos_integer()) -> true when is_subtype(X, list(non_neg_integer())).\nprop_plus_valid_base(A_, B_, Pos, Base) ->\n    {A, B, _, _, _, _} = math_pos:make_same_length(A_, B_, Pos),\n    A_plus_B = math_pos:plus(A, B, Base),\n    ?equals(erlang:length(A_plus_B), erlang:length(A)),\n    case lists:all(fun(E) -> E >= 0 andalso E < Base end, A_plus_B) of\n        true -> true;\n        _ -> ?ct_fail(\"math_pos:plus(A, B, ~B) evaluated to \\\"~.0p\\\" and \"\n                      \"contains invalid elements~n\",\n                      [Base, A_plus_B])\n    end.\n\n-spec prop_plus_valid1(A::[0..9], B::[0..9]) -> true.\nprop_plus_valid1(A_, B_) -> prop_plus_valid_base(A_, B_, front, 10).\n\n-spec prop_plus_valid2(A::string(), B::string()) -> true.\nprop_plus_valid2(A_, B_) -> prop_plus_valid_base(A_, B_, back, 16#10ffff + 1).\n\n-spec prop_plus_valid3(A::nonempty_string(), B::nonempty_string()) -> true.\nprop_plus_valid3(A_, B_) ->\n    Base = erlang:max(lists:max(A_), lists:max(B_)) + 1,\n    prop_plus_valid_base(A_, B_, back, Base).\n\n-spec tester_plus_valid(Config::[tuple()]) -> ok.\ntester_plus_valid(_Config) ->\n    tester:test(?MODULE, prop_plus_valid1, 2, 10000, [{threads, 2}]),\n    tester:test(?MODULE, prop_plus_valid2, 2, 10000, [{threads, 2}]),\n    tester:test(?MODULE, prop_plus_valid3, 2, 10000, [{threads, 2}]).\n\n-spec prop_plus_symm_base(X, X, Pos::front | back, Base::pos_integer()) -> true when is_subtype(X, list(non_neg_integer())).\nprop_plus_symm_base(A_, B_, Pos, Base) ->\n    {A, B, _, _, _, _} = math_pos:make_same_length(A_, B_, Pos),\n    ?equals(math_pos:plus(A, B, Base), math_pos:plus(B, A, Base)).\n\n-spec prop_plus_symm1(A::[0..9], B::[0..9]) -> true.\nprop_plus_symm1(A_, B_) -> prop_plus_symm_base(A_, B_, front, 10).\n\n-spec prop_plus_symm2(A::string(), B::string()) -> true.\nprop_plus_symm2(A_, B_) -> prop_plus_symm_base(A_, B_, back, 16#10ffff + 1).\n\n-spec prop_plus_symm3(A::nonempty_string(), B::nonempty_string()) -> true.\nprop_plus_symm3(A_, B_) ->\n    Base = erlang:max(lists:max(A_), lists:max(B_)) + 1,\n    prop_plus_symm_base(A_, B_, back, Base).\n\n-spec tester_plus_symm(Config::[tuple()]) -> ok.\ntester_plus_symm(_Config) ->\n    tester:test(?MODULE, prop_plus_symm1, 2, 10000, [{threads, 2}]),\n    tester:test(?MODULE, prop_plus_symm2, 2, 10000, [{threads, 2}]),\n    tester:test(?MODULE, prop_plus_symm3, 2, 10000, [{threads, 2}]).\n\n%% minus\n\n-spec prop_minus_valid_base(X, X, Pos::front | back, Base::pos_integer()) -> true when is_subtype(X, list(non_neg_integer())).\nprop_minus_valid_base(A_, B_, Pos, Base) ->\n    {A, B, _, _, _, _} = math_pos:make_same_length(A_, B_, Pos),\n    A_minus_B = math_pos:minus(A, B, Base),\n    ?equals(erlang:length(A_minus_B), erlang:length(A)),\n    case lists:all(fun(E) -> E >= 0 andalso E < Base end, A_minus_B) of\n        true -> true;\n        _ -> ?ct_fail(\"math_pos:minus(A, B, ~B) evaluated to \\\"~.0p\\\" and \"\n                      \"contains invalid elements~n\",\n                      [Base, A_minus_B])\n    end.\n\n-spec prop_minus_valid1(A::[0..9], B::[0..9]) -> true.\nprop_minus_valid1(A_, B_) -> prop_minus_valid_base(A_, B_, front, 10).\n\n-spec prop_minus_valid2(A::string(), B::string()) -> true.\nprop_minus_valid2(A_, B_) -> prop_minus_valid_base(A_, B_, back, 16#10ffff + 1).\n\n-spec prop_minus_valid3(A::nonempty_string(), B::nonempty_string()) -> true.\nprop_minus_valid3(A_, B_) ->\n    Base = erlang:max(lists:max(A_), lists:max(B_)) + 1,\n    prop_minus_valid_base(A_, B_, back, Base).\n\n-spec tester_minus_valid(Config::[tuple()]) -> ok.\ntester_minus_valid(_Config) ->\n    tester:test(?MODULE, prop_minus_valid1, 2, 10000, [{threads, 2}]),\n    prop_minus_valid2([557056,0,0,0,0,1], [557055,1114111,1114111,1114111,1114111,1114111]),\n    tester:test(?MODULE, prop_minus_valid2, 2, 10000, [{threads, 2}]),\n    tester:test(?MODULE, prop_minus_valid3, 2, 10000, [{threads, 2}]).\n\n-spec prop_minus_base(X, X, Pos::front | back, Base::pos_integer()) -> true when is_subtype(X, list(non_neg_integer())).\nprop_minus_base(A_, B_, Pos, Base) ->\n    {A, B, _, _, _, _} = math_pos:make_same_length(A_, B_, Pos),\n    A_B = math_pos:minus(A, B, Base),\n    ?equals(math_pos:plus(A_B, B, Base), A).\n\n-spec prop_minus1(A::[0..9], B::[0..9]) -> true.\nprop_minus1(A_, B_) -> prop_minus_base(A_, B_, front, 10).\n\n-spec prop_minus2(A::string(), B::string()) -> true.\nprop_minus2(A_, B_) -> prop_minus_base(A_, B_, back, 16#10ffff + 1).\n\n-spec prop_minus3(A::nonempty_string(), B::nonempty_string()) -> true.\nprop_minus3(A_, B_) ->\n    Base = erlang:max(lists:max(A_), lists:max(B_)) + 1,\n    prop_minus_base(A_, B_, back, Base).\n\n-spec tester_minus(Config::[tuple()]) -> ok.\ntester_minus(_Config) ->\n    tester:test(?MODULE, prop_minus1, 2, 10000, [{threads, 2}]),\n    tester:test(?MODULE, prop_minus2, 2, 10000, [{threads, 2}]),\n    tester:test(?MODULE, prop_minus3, 2, 10000, [{threads, 2}]).\n\n%% divide\n\n-spec prop_divide_valid_base(X, Div::pos_integer(), Base::pos_integer()) -> true when is_subtype(X, list(non_neg_integer())).\nprop_divide_valid_base(A, Div, Base) ->\n    A_div = math_pos:divide(A, Div, Base),\n    ?equals(erlang:length(A_div), erlang:length(A)),\n    case lists:all(fun(E) -> E >= 0 andalso E < Base end, A_div) of\n        true -> true;\n        _ -> ?ct_fail(\"math_pos:divide(A, Div, ~B) evaluated to \\\"~.0p\\\" and \"\n                      \"contains invalid elements~n\",\n                      [Base, A_div])\n    end.\n\n-spec prop_divide_valid1(A::[0..9], Div::pos_integer()) -> true.\nprop_divide_valid1(A, Div) -> prop_divide_valid_base(A, Div, 10).\n\n-spec prop_divide_valid2(A::string(), Div::pos_integer()) -> true.\nprop_divide_valid2(A, Div) -> prop_divide_valid_base(A, Div, 16#10ffff + 1).\n\n-spec prop_divide_valid3(A::nonempty_string(), Div::pos_integer()) -> true.\nprop_divide_valid3(A, Div) ->\n    Base = lists:max(A) + 1,\n    prop_divide_valid_base(A, Div, Base).\n\n-spec tester_divide_valid(Config::[tuple()]) -> ok.\ntester_divide_valid(_Config) ->\n    tester:test(?MODULE, prop_divide_valid1, 2, 10000, [{threads, 2}]),\n    tester:test(?MODULE, prop_divide_valid2, 2, 10000, [{threads, 2}]),\n    tester:test(?MODULE, prop_divide_valid3, 2, 10000, [{threads, 2}]).\n\n-spec prop_divide_decimals(A::non_neg_integer(), Div::pos_integer()) -> true.\nprop_divide_decimals(A, Div) ->\n    APos = decimal_to_pos(A),\n    ?equals(pos_to_decimal(math_pos:divide(APos, Div, 10)), A div Div).\n\n-spec tester_divide_decimals(Config::[tuple()]) -> ok.\ntester_divide_decimals(_Config) ->\n    tester:test(?MODULE, prop_divide_decimals, 2, 10000, [{threads, 2}]).\n\n%% multiply\n\n-spec prop_multiply_valid_base(A::[non_neg_integer()], Fac::non_neg_integer(), Base::pos_integer()) -> true.\nprop_multiply_valid_base(A, Fac, Base) ->\n    A_prod = math_pos:multiply(A, Fac, Base),\n    ?equals(erlang:length(A_prod), erlang:length(A)),\n    case lists:all(fun(E) -> E >= 0 andalso E < Base end, A_prod) of\n        true -> true;\n        _ -> ?ct_fail(\"math_pos:multiply(A, Div, ~B) evaluated to \\\"~.0p\\\" and \"\n                      \"contains invalid elements~n\",\n                      [Base, A_prod])\n    end.\n\n-spec prop_multiply_valid1(A::[0..9], Fac::0..9) -> true.\nprop_multiply_valid1(A, Fac) -> prop_multiply_valid_base(A, Fac, 10).\n\n-spec prop_multiply_valid2(A::string(), Fac::0..16#10ffff) -> true.\nprop_multiply_valid2(A, Fac) -> prop_multiply_valid_base(A, Fac, 16#10ffff + 1).\n\n-spec prop_multiply_valid3(A::nonempty_string(), Fac::non_neg_integer()) -> true.\nprop_multiply_valid3(A, Fac) ->\n    A_max = lists:max(A), Base = A_max + 1,\n    prop_multiply_valid_base(A, Fac, Base).\n\n-spec tester_multiply_valid(Config::[tuple()]) -> ok.\ntester_multiply_valid(_Config) ->\n    tester:test(?MODULE, prop_multiply_valid1, 2, 10000, [{threads, 2}]),\n    tester:test(?MODULE, prop_multiply_valid2, 2, 10000, [{threads, 2}]),\n    tester:test(?MODULE, prop_multiply_valid3, 2, 10000, [{threads, 2}]).\n\n-spec prop_multiply_decimals(A::non_neg_integer(), Fac::0..100000) -> true.\nprop_multiply_decimals(A, Fac) ->\n    APos = decimal_to_pos(A),\n    {ProdPos, Added} = math_pos:multiply(APos, Fac, 10, enlarge),\n    ?equals(pos_to_decimal(ProdPos), A * Fac),\n    ?equals(Added, length(ProdPos) - length(APos)).\n\n-spec tester_multiply_decimals(Config::[tuple()]) -> ok.\ntester_multiply_decimals(_Config) ->\n    tester:test(?MODULE, prop_multiply_decimals, 2, 10000, [{threads, 2}]).\n\n%% helpers\n\ndecimal_to_pos(X) -> [A - 48 || A <- lists:flatten(io_lib:format(\"~B\", [X]))].\n\npos_to_decimal(X) -> element(2, lists:foldr(fun(A, {Fac, Res}) ->\n                                                    {Fac * 10, Res + A * Fac}\n                                            end, {1, 0}, X)).\n"
  },
  {
    "path": "test/mathlib_SUITE.erl",
    "content": "%  @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Magnus Mueller <mamuelle@informatik.hu-berlin.de>\n%% @doc    Test suite for the mathlib module.\n%% @end\n%% @version $Id$\n\n-module(mathlib_SUITE).\n-author('mamuelle@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\nall() -> [\n        euclidian_distance\n        , nearest_centroid\n        , closest_points\n        , agglomerative_clustering\n    ].\n\nsuite() ->\n    [\n        {timetrap, {seconds, 30}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\n\n%% helper functions\n-spec in(X::any(), List::[any()]) -> boolean().\nin(X, List) ->\n    lists:foldl(fun(_, true) -> true;\n            (E, false) when E == X -> true;\n            (_, _) -> false\n        end, false, List)\n    .\n\n% Test mathlib:euclideanDistance/2\neuclidian_distance(_Config) ->\n    {U1, V1, V2, V3, V4} = {\n        [0.0,0.0],\n        [0, 1.0],\n        [1.0, 0],\n        [1.0, 1.0],\n        [-1.0,-1.0]\n    },\n    ?equals(mathlib:euclideanDistance(U1, U1), 0.0),\n    ?equals(mathlib:euclideanDistance(U1, V1), 1.0),\n    ?equals(mathlib:euclideanDistance(U1, V2), 1.0),\n    ?equals(mathlib:euclideanDistance(U1, V3), math:sqrt(2)),\n    ?equals(mathlib:euclideanDistance(U1, V4), math:sqrt(2)),\n    ok.\n\n\n% Test mathlib:nearestCentroid/2\n%\n% Testcases:\n% - Centroids empty (problemcase)\n% - Centroids non-empty and items are unique, U not contained (good case)\n% - Centroids contains U again (problemcase)\n% - Centroids non-empty, not unique, doesn't contain U (should be ok)\n% - Centroids non-empty, not unique, contains U\nnearest_centroid(_Config) ->\n    % Note: The relative size will be ignored in this unittest, so we set it to zero\n\n    U = dc_centroids:new([0.0, 0.0], 0.0),\n\n    %% ----- Good cases which should work ------\n\n    %% List of Centroids is not empty. U is not an element and only one\n    %   element in the list is nearest to U\n    C1 = [C1Nearest1, C1Nearest2 | _] = [dc_centroids:new([float(X), float(X)], 0.0)\n                                           || X <- lists:seq(1,5)],\n    ?equals(mathlib:nearestCentroid(U, C1), {math:sqrt(2), C1Nearest1}),\n    ?equals(mathlib:nearestCentroid(U, tl(C1)), {math:sqrt(8), C1Nearest2}),\n\n    %% List not empty, U is an element, only one nearest element\n    C2 = [U|C1],\n    ?equals(mathlib:nearestCentroid(U, C2), {math:sqrt(2), C1Nearest1}),\n    C3 = C1 ++ [U],\n    ?equals(mathlib:nearestCentroid(U, C3), {math:sqrt(2), C1Nearest1}),\n\n    %% List contains the same entry multiple times\n    C4 = C1 ++ C1 ++ C1,\n    ?equals(mathlib:nearestCentroid(U, C4), {math:sqrt(2), C1Nearest1}),\n\n    %% Order of the list should not be important\n    RC4 = lists:reverse(C4),\n    ?equals(mathlib:nearestCentroid(U, RC4), {math:sqrt(2), C1Nearest1}),\n\n    %% \n\n    %% ----- Cases that should behave well ------\n    % empty centroids: return 'none'\n    ?equals(mathlib:nearestCentroid(U, []), none),\n\n    % ambiguous centroids: return a node from a set of nodes\n    Ambiguous1 = [dc_centroids:new([float(X), float(Y)], 0.0)\n                    || X <- lists:seq(1,5), Y <- lists:seq(1,5)],\n    {AmbiguousDistance1, Element} = mathlib:nearestCentroid(U, Ambiguous1),\n    ?equals(AmbiguousDistance1, math:sqrt(2)),\n    AllowedElements1 = [dc_centroids:new([X, Y], 0.0) || {X, Y} <- [\n            {1.0,1.0},{1.0,-1.0},{-1.0,1.0},{-1.0,-1.0}\n        ]],\n    ?assert_w_note(in(Element, AllowedElements1),\n        \"Nearest element not in list of allowed coordinates\"),\n\n    % regression test\n    U2 = dc_centroids:new([0.0, 0.0], 1.0),\n    V2 = dc_centroids:new([1.0, 1.0], 1.0),\n    FarAway = dc_centroids:new([100.0, 100.0], 1.0),\n    ?equals(mathlib:nearestCentroid(U2, [V2, FarAway]), {dc_centroids:distance(U2, V2),\n            V2}),\n    ?equals(mathlib:nearestCentroid(U2, [FarAway, V2]), {dc_centroids:distance(U2, V2),\n            V2}),\n    ok.\n\n% Test mathlib:closestPoints/1\n%\n% Testcases:\n% - Return none for an empty list of centroids\n% - Return none for a list containing only one centroid\n% - Return the two elements with the smallest distance\n% - When ambiguous, pick any two elements with a smallest distance\nclosest_points(_Config) ->\n    %% ----- Good cases which should work ------\n    C1 = [C1_1, C1_2 | _] = [dc_centroids:new([float(X), float(X)], 0.0)\n                               || X <- lists:seq(1,5)],\n    Dist1 = dc_centroids:distance(C1_1, C1_2),\n    ?equals(mathlib:closestPoints(C1), {Dist1, C1_1, C1_2}),\n\n    %% ----- Cases that should behave well ------\n    % empty list\n    ?equals(mathlib:closestPoints([]), none),\n\n    % list with only one element\n    U = dc_centroids:new([0.0, 0.0], 0.0),\n    ?equals(mathlib:closestPoints([U]), none),\n\n    % ambiguous list\n    C2 = [C2_1, C2_2 | _] = [dc_centroids:new([float(X), float(Y)], 0.0)\n                               || X <- lists:seq(1,5),\n                                  Y <- lists:seq(1,5)],\n    Dist2 = dc_centroids:distance(C2_1, C2_2),\n    ?equals(mathlib:closestPoints(C2), {Dist2, C2_1, C2_2}),\n\n    % shuffled ambiguous list\n    C3 = util:shuffle(C2),\n    {Dist3, _A, _B} = mathlib:closestPoints(C3),\n    ?equals(Dist3, 1.0),\n\n    % regression test\n    U2 = dc_centroids:new([0.0, 0.0], 1.0),\n    V2 = dc_centroids:new([1.0, 1.0], 1.0),\n    FarAway = dc_centroids:new([100.0, 100.0], 1.0),\n    ?equals(mathlib:closestPoints([U2, V2, FarAway]),\n        {dc_centroids:distance(U2,V2), U2, V2}),\n    ?equals(mathlib:closestPoints([U2, FarAway, V2]),\n        {dc_centroids:distance(U2,V2), U2, V2}),\n    ok.\n\n\n% Test mathlib:aggloClustering/1\n%\n% Testcases:\n% - Clustering should fail with error when Radius < 0\n% - Clustering an empty list should return an empty list\n% - Clustering of one centroid should return the same centroid\n% - Clustering two centroids with a distance less/equal Radius should return a merged centroid\n% - Clustering two centroids with a distance > Radius should return both centroids\n% - Clustering a set of centroids should return the correct set of merged centroids\n% - The sum of the relative size over all centroids should remain the same\n% - XXX What should it do if elements/coordinates are duplicated?\nagglomerative_clustering(_Config) ->\n    % crash when radius < 0\n    ?expect_exception(mathlib:aggloClustering([], -1), error, function_clause),\n\n    % empty centroid list\n    ?equals(mathlib:aggloClustering([], 0), []),\n\n    % single node\n    U = dc_centroids:new([0.0, 0.0], 1.0),\n    ?equals(mathlib:aggloClustering([U], 0), [U]),\n\n    % merge two nodes\n    V = dc_centroids:new([1.0, 1.0], 1.0),\n    MergedUV = dc_centroids:new([0.5, 0.5], 2.0),\n    ?equals(mathlib:aggloClustering([U,V], 2), [MergedUV]),\n    ?equals(mathlib:aggloClustering([V,U], 2), [MergedUV]),\n\n    % don't merge far-away nodes\n    FarAway = dc_centroids:new([100.0, 100.0], 1.0),\n    ?equals(mathlib:aggloClustering([V, FarAway], 50), [V, FarAway]),\n    ?equals(mathlib:aggloClustering([V, U, FarAway], 99), [MergedUV, FarAway]),\n    ?equals(mathlib:aggloClustering([V, FarAway, U], 99), [MergedUV, FarAway]),\n\n    % merge many nodes, relative size sum should remain the same\n    C = [dc_centroids:new([float(X), float(X)], 1/6) || X <- lists:seq(1,6)],\n    ?equals(mathlib:aggloClustering(C, 7), [dc_centroids:new([3.5, 3.5], 1.0)]),\n    ok.\n"
  },
  {
    "path": "test/measure_util.erl",
    "content": "%  @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <MLange@informatik.hu-berlin.de>\n%% @doc    Utility functions for execution time measurement.\n%%         Src = http://www.trapexit.org/Measuring_Function_Execution_Time\n%% @end\n%% @version $Id$\n-module(measure_util).\n-author('mlange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-export([time_avg/3, time_with_result/3]).\n-export([print/1, print/2, get/2, get/3]).\n-export([add/2]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% type definitions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-export_type([result/0, options/0]).\n\n-type result() :: { Min::non_neg_integer(), \n                    Max::non_neg_integer(),\n                    Med::non_neg_integer(), \n                    Avg::float(),\n                    Sum::non_neg_integer(),\n                    Iterations::pos_integer()\n                  }.\n\n-type options() :: skip_first_value.\n\n-type time_unit() :: us | ms | s.\n-type mr_type() :: min | max | med | avg | sum.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% real type spec (not compatible with R13):\n%-spec time_with_result(fun((...) -> Result), pos_integer(), [options()]) -> {Result, Time::result()}.\n-spec time_with_result(fun(), pos_integer(), [options()]) -> {Result::term(), Time::result()}.\ntime_with_result(Fun, Iterations, Options) ->\n    Time = time_avg(Fun, Iterations, Options),\n    Result = Fun(),\n    {Result, Time}.\n\n% @doc Measures average execution time with possibiliy of skipping \n%      the first measured value.\n%      i.e.: time_avg(fun() -> myFun(A, B, C) end, 100, []).\n-spec time_avg(fun(), pos_integer(), [options()]) -> result().\ntime_avg(Fun, Iterations, Options) ->\n    L = util:repeat(fun() -> erlang:element(1, util:tc(Fun, [])) end, [], Iterations, [collect]),\n    Times = case lists:member(skip_first_value, Options) of\n                true -> lists:nthtail(1, L);\n                _ -> L\n            end,   \n    Length = length(Times),\n    Min = lists:min(Times),\n    Max = lists:max(Times),\n    Med = lists:nth(((Length + 1) div 2), lists:sort(Times)),\n    Sum = lists:foldl(fun(X, Sum) -> X + Sum end, 0, Times),\n    Avg = Sum / Length,\n    {Min, Max, Med, Avg, Sum, Iterations}.\n\n-spec add(result(), result()) -> result().\nadd({AMin, AMax, AMed, AAvg, ASum, AIt}, {BMin, BMax, BMed, BAvg, BSum, BIt}) ->\n    {AMin + BMin,\n     AMax + BMax,\n     AMed + BMed,\n     AAvg + BAvg,\n     ASum + BSum,\n     erlang:round((AIt + BIt) / 2)}.\n\n-spec print(result()) -> [{atom(), any()}].\nprint({Min, Max, Med, Avg, _Sum, _} = Values) ->\n    MaxVal = lists:max([Min, Max, Med, Avg]),\n    if\n        MaxVal > 1000000 -> print(Values, s);\n        MaxVal > 1000    -> print(Values, ms);\n        true             -> print(Values, us)\n    end.\n        \n\n-spec print(result(), time_unit()) -> [{atom(), any()}].\nprint({Min, Max, Med, Avg, Sum, Iter}, Unit) ->\n    [{unit, Unit},\n     {min, value_to_unit(Min, Unit)},\n     {max, value_to_unit(Max, Unit)},\n     {med, value_to_unit(Med, Unit)},\n     {avg, value_to_unit(Avg, Unit)},\n     {sum, value_to_unit(Sum, Unit)},\n     {iterations, Iter}].\n\n-spec get(result(), mr_type()) -> number().\nget({Min, Max, Med, Avg, Sum, _}, Type) ->\n    case Type of\n        min -> Min;\n        max -> Max;\n        med -> Med;\n        avg -> Avg;\n        sum -> Sum\n    end.\n\n-spec get(result(), mr_type(), time_unit()) -> float().\nget(Value, Type, Unit) ->\n    value_to_unit(get(Value, Type), Unit).\n\n-spec value_to_unit(number(), time_unit()) -> float().\nvalue_to_unit(Val, Unit) ->\n    case Unit of\n        us -> erlang:float(Val);\n        ms -> Val / 1000;\n        s -> Val / 1000000\n    end.\n"
  },
  {
    "path": "test/memtest_SUITE.erl",
    "content": "%% @copyright 2013-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @version $Id$\n-module(memtest_SUITE).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n-include(\"record_helpers.hrl\").\n\nall()   -> [\n            {group, api_tx},\n            {group, ring}\n           ].\n\ngroups() ->\n    [\n     {api_tx, [], [write_1000, fill_1000, fill_2000, modify_100x100]},\n     {ring,   [], [create_ring_100, add_remove_nodes_50, add_kill_nodes_50]}\n    ].\n\nsuite() -> [ {timetrap, {seconds, 300}} ].\n\ninit_per_suite(Config) ->\n    _ = code:ensure_loaded(gossip_load_default), % otherwise loaded too late\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) ->\n    ct:comment(io_lib:format(\"BEGIN ~p\", [Group])),\n    Config.\n\nend_per_group(Group, Config) ->\n    ct:comment(io_lib:format(\"END ~p\", [Group])),\n    Config.\n\ninit_per_testcase(_TestCase, Config) ->\n%%     Group = case proplists:get_value(tc_group_properties, Config) of\n%%                 undefined                 -> undefined;\n%%                 Props when is_list(Props) -> proplists:get_value(name, Props)\n%%             end,\n    %% stop ring from previous test case (it may have run into a timeout)\n    unittest_helper:stop_ring(),\n    % start ring:\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    %% add_remove_nodes_50 and add_kill_nodes_50 throw error:system_limit for larger r\n    unittest_helper:make_ring(1, [{config, [{log_path, PrivDir}, {monitor_perf_interval, 0},\n                                            {replication_factor, 4}]}]),\n    config:write(no_print_ring_data, true),\n    % load necessary code for the atom check to work (we do not create additional atoms):\n    InitKey = \"memtest_SUITE\",\n    {ok} = api_tx:write(InitKey, 1),\n    api_tx_SUITE:wait_for_dht_entries(?RT:get_replica_keys(?RT:hash_key(InitKey))),\n    _ = code:ensure_loaded(gossip_load_default), % otherwise loaded too late\n    garbage_collect_all(),\n    Config.\n\nend_per_testcase(_TestCase, _Config) ->\n    ct:pal(\"Memory after test:~n~p~n\", [erlang:memory()]),\n    unittest_helper:stop_ring(), % note: we do not use scalaris_cth's stop_ring!\n    garbage_collect_all(),\n    ct:pal(\"Memory after unittest_helper:stop_ring/0:~n~p~n\", [erlang:memory()]),\n    ok.\n\n-record(mem_info, {binary         = ?required(mem_info, binary)         :: pos_integer(),\n                   atom_used      = ?required(mem_info, atom_used)      :: pos_integer(),\n                   processes_used = ?required(mem_info, processes_used) :: pos_integer(),\n                   ets            = ?required(mem_info, ets)            :: pos_integer()\n                  }).\n\n%% @doc Creates an uncompressed binary of a list of Size integers starting at I.\n-spec make_binary(I::integer(), Size::non_neg_integer()) -> binary().\nmake_binary(I, Size) when Size >= 0->\n    Res = erlang:term_to_binary(lists:seq(I, I + Size)),\n%%     ct:pal(\"binary size: ~B\", [erlang:byte_size(Res)]),\n    Res.\n\n%% @doc Prints information about the given ets table ID and compares it with\n%%      the total Erlang VM memory stats.\nprint_table_info(Table) ->\n    print_table_info(Table, \"\").\n\n%% @doc Prints information about the given ets table ID and compares it with\n%%      the total Erlang VM memory stats (prefixes the message with Status).\nprint_table_info(Table, Status) ->\n    Memory = ets:info(Table, memory),\n    ContentSize = lists:sum([byte_size(element(2,X)) || X <- ets:tab2list(Table), is_binary(element(2, X))]),\n    ct:pal(\"~s~nerlang total: ~B, binary: ~B~nets table ~p, memory: ~B, contents: ~B~n\",\n           [Status, erlang:memory(total), erlang:memory(binary), Table, Memory, ContentSize]),\n    garbage_collect_all().\n\n%% @doc Gets memory statistics.\n-spec get_meminfo() -> #mem_info{}.\nget_meminfo() ->\n    [{binary, BinSize}, {atom_used, AtomUSize}, {processes_used, ProcUSize}, {ets, EtsSize}] =\n        erlang:memory([binary, atom_used, processes_used, ets]),\n    #mem_info{binary = BinSize, atom_used = AtomUSize,\n              processes_used = ProcUSize, ets = EtsSize}.\n\n%% @doc Checks whether the NewMemInfo field indicates increased memory use\n%%      compared to PrevMemInfo and returns whether this is the case or not.\n%%      AddedSize is the expected binary size difference.\n%%      Note: Tolerates 250k binary, 250k processes_used and 250k ets overhead\n%%            (keep in sync with check_memory_inc_/5).\n-spec check_memory_inc_bool_(\n        PrevMemInfo::#mem_info{}, NewMemInfo::#mem_info{},\n        NewItems::non_neg_integer(), AddedSize::non_neg_integer(),\n        CheckAtoms::boolean()) -> boolean().\ncheck_memory_inc_bool_(PrevMemInfo, NewMemInfo, NewItems, AddedSize, CheckAtoms) ->\n    % assume an entry for an item uses 4 * 1k memory for ets (excluding the binary size)\n    EntryEtsSize = 4 * 1000,\n\n    % we do not create additional atoms!\n    ?implies(CheckAtoms, NewMemInfo#mem_info.atom_used =:= PrevMemInfo#mem_info.atom_used) andalso\n    % tolerate 250k binary memory overhead for running maintenance processes and binary messages\n    NewMemInfo#mem_info.binary =< (PrevMemInfo#mem_info.binary + AddedSize + 250000) andalso\n    % tolerate 250k processes_used memory overhead for running maintenance processes like RT rebuild etc.\n    NewMemInfo#mem_info.processes_used =< (PrevMemInfo#mem_info.processes_used + 250000) andalso\n    % tolerate 250k ets memory overhead for DB data etc.\n    NewMemInfo#mem_info.ets =< (PrevMemInfo#mem_info.ets + 250000 + (EntryEtsSize * NewItems)).\n\n%% @doc If debugging is disabled, execute check_memory_inc_bool_/5 with atom_used\n%%      check, otherwise do not check atom_used if the test starts nodes (with\n%%      debugging, atoms in the form of registered processes are generated).\n-spec check_memory_inc_bool(\n        PrevMemInfo::#mem_info{}, NewMemInfo::#mem_info{},\n        NewItems::non_neg_integer(), AddedSize::non_neg_integer(),\n        TestStartsNodes::boolean()) -> boolean().\n-ifdef(enable_debug).\ncheck_memory_inc_bool(PrevMemInfo, NewMemInfo, NewItems, AddedSize, TestStartsNodes) ->\n    check_memory_inc_bool_(PrevMemInfo, NewMemInfo, NewItems, AddedSize, not TestStartsNodes).\n-else.\ncheck_memory_inc_bool(PrevMemInfo, NewMemInfo, NewItems, AddedSize, _TestStartsNodes) ->\n    check_memory_inc_bool_(PrevMemInfo, NewMemInfo, NewItems, AddedSize, true).\n-endif.\n\n%% @doc Helper for check_memory_inc/5.\n%%      Note: Tolerates 250k binary, 250k processes_used and 250k ets overhead.\n-spec check_memory_inc_(\n        PrevMemInfo::#mem_info{}, NewMemInfo::#mem_info{},\n        NewItems::non_neg_integer(), AddedSize::non_neg_integer(),\n        CheckAtoms::boolean()) -> ok.\ncheck_memory_inc_(PrevMemInfo, NewMemInfo, NewItems, AddedSize, CheckAtoms) ->\n    % assume an entry for an item uses 4 * 1k memory for ets (excluding the binary size)\n    EntryEtsSize = 4 * 1000,\n\n    % we do not create additional atoms!\n    ?IIF(CheckAtoms,\n         ?equals(NewMemInfo#mem_info.atom_used, PrevMemInfo#mem_info.atom_used),\n         ok),\n\n    % tolerate 250k binary memory overhead for running maintenance processes and binary messages\n    ?equals_pattern_w_note(NewMemInfo#mem_info.binary, X when X =< PrevMemInfo#mem_info.binary + AddedSize + 250000,\n                           lists:flatten(io_lib:format(\"PrevBinSize: ~B, AddedSize: ~B, NewBinSize: ~B, Diff: ~B\",\n                                         [PrevMemInfo#mem_info.binary, NewMemInfo#mem_info.binary, AddedSize,\n                                          NewMemInfo#mem_info.binary - AddedSize - PrevMemInfo#mem_info.binary]))),\n\n    case erlang:system_info(version) of\n        %% R18.1 and the current dev (8.0) have a memory leak\"\n        \"7.1\" -> %% R18.1\n            ok;\n        %%\"8.0\" -> %% current dev\n        %%    %% tolerate 700k processes_used memory overhead for running maintenance processes like RT rebuild etc.\n        %%    ?equals_pattern_w_note(NewMemInfo#mem_info.processes_used, X when X =< PrevMemInfo#mem_info.processes_used + 700000,\n        %%                           lists:flatten(io_lib:format(\"PrevProcUSize: ~B, NewProcUSize: ~B, Diff: ~B\",\n        %%                                         [PrevMemInfo#mem_info.processes_used, NewMemInfo#mem_info.processes_used,\n        %%                                          NewMemInfo#mem_info.processes_used - PrevMemInfo#mem_info.processes_used]))),\n        %%\n        %%    %% tolerate 270k ets memory overhead for DB data etc.\n        %%    ?equals_pattern_w_note(NewMemInfo#mem_info.ets, X when X =< PrevMemInfo#mem_info.ets + 270000 + (EntryEtsSize * NewItems),\n        %%                           lists:flatten(io_lib:format(\"PrevEtsSize: ~B, NewEtsSize: ~B, Diff: ~B\",\n        %%                                         [PrevMemInfo#mem_info.ets, NewMemInfo#mem_info.ets,\n        %%                                          NewMemInfo#mem_info.ets - PrevMemInfo#mem_info.ets])));\n\n        _ ->\n            %% tolerate 250k processes_used memory overhead for running maintenance processes like RT rebuild etc.\n            ?equals_pattern_w_note(NewMemInfo#mem_info.processes_used, X when X =< PrevMemInfo#mem_info.processes_used + 250000,\n                                   lists:flatten(io_lib:format(\"PrevProcUSize: ~B, NewProcUSize: ~B, Diff: ~B\",\n                                                 [PrevMemInfo#mem_info.processes_used, NewMemInfo#mem_info.processes_used,\n                                                  NewMemInfo#mem_info.processes_used - PrevMemInfo#mem_info.processes_used]))),\n            %% tolerate 250k ets memory overhead for DB data etc.\n            ?equals_pattern_w_note(NewMemInfo#mem_info.ets, X when X =< PrevMemInfo#mem_info.ets + 250000 + (EntryEtsSize * NewItems),\n                                   lists:flatten(io_lib:format(\"PrevEtsSize: ~B, NewEtsSize: ~B, Diff: ~B\",\n                                                 [PrevMemInfo#mem_info.ets, NewMemInfo#mem_info.ets,\n                                                  NewMemInfo#mem_info.ets - PrevMemInfo#mem_info.ets])))\n    end,\n    ok.\n\n%% @doc If debugging is disabled, execute check_memory_inc_/5 with atom_used\n%%      check, otherwise do not check atom_used if the test starts nodes (with\n%%      debugging, atoms in the form of registered processes are generated).\n-spec check_memory_inc(\n        PrevMemInfo::#mem_info{}, NewMemInfo::#mem_info{},\n        NewItems::non_neg_integer(), AddedSize::non_neg_integer(),\n        TestStartsNodes::boolean()) -> ok.\n-ifdef(enable_debug).\ncheck_memory_inc(PrevMemInfo, NewMemInfo, NewItems, AddedSize, TestStartsNodes) ->\n    check_memory_inc_(PrevMemInfo, NewMemInfo, NewItems, AddedSize, not TestStartsNodes).\n-else.\ncheck_memory_inc(PrevMemInfo, NewMemInfo, NewItems, AddedSize, _TestStartsNodes) ->\n    check_memory_inc_(PrevMemInfo, NewMemInfo, NewItems, AddedSize, true).\n-endif.\n\nwrite_1000(_Config) ->\n    write(1000, 200000). % ca. 1MiB RAM\n\n%% @doc Writes newly created binaries to the same key thus overwriting previous\n%%      binaries. The result should not occupy more memory than the last\n%%      written binary.\nwrite(Number, Size) when Number >= 1 ->\n    Key = lists:flatten(io_lib:format(\"write_~B\", [Number])),\n    DHTNodes = pid_groups:find_all(dht_node),\n%%     ct:pal(\"~p~n\", [erlang:memory()]),\n    Table = hd([Tab || Tab <- ets:all(),\n                       ets:info(Tab, name) =:= dht_node_db,\n                       lists:member(ets:info(Tab, owner), DHTNodes)]),\n    print_table_info(Table),\n    PrevMemInfo = get_meminfo(),\n    % only the last binary should be kept!\n    MyBinSize =\n        util:for_to_fold(1, Number,\n                         fun(I) ->\n                                 Bin = make_binary(I, Size),\n                                 {ok} = api_tx:write(Key, Bin),\n                                 % wait for late write messages:\n                                 api_tx_SUITE:wait_for_dht_entries(?RT:get_replica_keys(?RT:hash_key(Key))),\n                                 %print_table_info(Table),\n                                 erlang:byte_size(Bin)\n                         end,\n                         fun(BSize, _PrevSize) -> BSize end,\n                         0),\n    NewMemInfo = garbage_collect_all_and_check(40, PrevMemInfo, 1, MyBinSize, false),\n%%     ct:pal(\"~p~n\", [erlang:memory()]),\n    print_table_info(Table),\n    check_memory_inc(PrevMemInfo, NewMemInfo, 1, MyBinSize, false),\n    ok.\n\nfill_1000(_Config) ->\n    fill(1000, 20000). % ca. 100MiB RAM\n\nfill_2000(_Config) ->\n    fill(2000, 20000). % ca. 200MiB RAM\n\nfill(Number, Size) when Number >= 1 ->\n    DHTNodes = pid_groups:find_all(dht_node),\n%%     ct:pal(\"~p~n\", [erlang:memory()]),\n    Table = hd([Tab || Tab <- ets:all(),\n                       ets:info(Tab, name) =:= dht_node_db,\n                       lists:member(ets:info(Tab, owner), DHTNodes)]),\n    print_table_info(Table),\n    PrevMemInfo = get_meminfo(),\n    MyBinSize =\n        util:for_to_fold(1, Number,\n                         fun(I) ->\n                                 Key = lists:flatten(io_lib:format(\"fill_~B\", [I])),\n                                 Bin = make_binary(I, Size),\n                                 {ok} = api_tx:write(Key, Bin),\n                                 % wait for late write messages:\n                                 api_tx_SUITE:wait_for_dht_entries(?RT:get_replica_keys(?RT:hash_key(Key))),\n                                 %print_table_info(Table, lists:flatten(io_lib:format(\"~B\", [I]))),\n                                 erlang:byte_size(Bin)\n                         end,\n                         fun(BSize, TotalSize) -> TotalSize + BSize end,\n                         0),\n    NewMemInfo = garbage_collect_all_and_check(40, PrevMemInfo, Number, MyBinSize, false),\n%%     ct:pal(\"~p~n\", [erlang:memory()]),\n    print_table_info(Table),\n    check_memory_inc(PrevMemInfo, NewMemInfo, Number, MyBinSize, false),\n    ok.\n\n% @doc Modifies 100 items 100 times.\nmodify_100x100(_Config) ->\n    modify(100, 100, 20000). % ca. 10MiB RAM\n\nmodify(Number, Repeat, Size) when Number >= 1 andalso Repeat >= 1 ->\n    DHTNodes = pid_groups:find_all(dht_node),\n%%     ct:pal(\"~p~n\", [erlang:memory()]),\n    Table = hd([Tab || Tab <- ets:all(),\n                       ets:info(Tab, name) =:= dht_node_db,\n                       lists:member(ets:info(Tab, owner), DHTNodes)]),\n    print_table_info(Table),\n    PrevMemInfo = get_meminfo(),\n    MyBinSize =\n        util:for_to_fold(\n          1, Number,\n          fun(I) ->\n                  util:for_to_fold(\n                    1, Repeat,\n                    fun(J) ->\n                            Key = lists:flatten(io_lib:format(\"modify_~B\", [I])),\n                            Bin = make_binary(I+J, Size),\n                            {ok} = api_tx:write(Key, Bin),\n                            % wait for late write messages:\n                            api_tx_SUITE:wait_for_dht_entries(?RT:get_replica_keys(?RT:hash_key(Key))),\n                            %print_table_info(Table, lists:flatten(io_lib:format(\"~B\", [I]))),\n                            erlang:byte_size(Bin)\n                    end,\n                    fun(BSize, _PrevSize) -> BSize end,\n                    0)\n                  end,\n          fun(BSize, TotalSize) -> TotalSize + BSize end,\n          0),\n    NewMemInfo = garbage_collect_all_and_check(40, PrevMemInfo, Number, MyBinSize, false),\n%%     ct:pal(\"~p~n\", [erlang:memory()]),\n    print_table_info(Table),\n    check_memory_inc(PrevMemInfo, NewMemInfo, Number, MyBinSize, false),\n    ok.\n\ncreate_ring_100(Config) ->\n    % stop the initial ring (the ring is needed to initialise atoms etc.)\n    unittest_helper:stop_ring(),\n    garbage_collect_all(),\n\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    OldProcesses = unittest_helper:get_processes(),\n    PrevMemInfo = get_meminfo(),\n    util:for_to(\n      1, 100,\n      fun(_I) ->\n              unittest_helper:make_ring(1, [{config, [{log_path, PrivDir}, {monitor_perf_interval, 0}]}]),\n              unittest_helper:stop_ring()\n      end),\n    NewMemInfo = garbage_collect_all_and_check(10, PrevMemInfo, 0, 0, true),\n    NewProcesses = unittest_helper:get_processes(),\n    {_OnlyOld, _Both, OnlyNew} =\n        util:split_unique(OldProcesses, NewProcesses,\n                          fun(P1, P2) ->\n                                  element(1, P1) =< element(1, P2)\n                          end, fun(_P1, P2) -> P2 end),\n    ?equals(OnlyNew, []),\n    %%     ct:pal(\"~p~n\", [erlang:memory()]),\n    check_memory_inc(PrevMemInfo, NewMemInfo, 0, 0, true),\n    ok.\n\nadd_remove_nodes_50(_Config) ->\n    config:write(gossip_vivaldi_latency_timeout, 0),\n    {[TmpNode], []} = api_vm:add_nodes(1), % loads all code needed for joins\n    {[TmpNode], []} = api_vm:shutdown_nodes_by_name([TmpNode]), % loads all code needed for graceful leaves\n    garbage_collect_all(),\n    OldProcesses = unittest_helper:get_processes(),\n    PrevMemInfo = get_meminfo(),\n    {NewNodes, []} = api_vm:add_nodes(50),\n    unittest_helper:wait_for_stable_ring_deep(),\n    _ = api_vm:shutdown_nodes_by_name(NewNodes),\n    timer:sleep(1000), % wait for vivaldi_latency timeouts\n    unittest_helper:wait_for_stable_ring_deep(),\n    _ = garbage_collect_all_and_check(10, PrevMemInfo, 0, 0, true),\n    NewProcesses = unittest_helper:get_processes(),\n    {_OnlyOld, _Both, OnlyNew} =\n        util:split_unique(OldProcesses, NewProcesses,\n                          fun(P1, P2) ->\n                                  element(1, P1) =< element(1, P2)\n                          end, fun(_P1, P2) -> P2 end),\n    ?equals(OnlyNew, []),\n%%     ct:pal(\"~p~n\", [erlang:memory()]),\n    % TODO: add memory check? (process state may increase, e.g. dead nodes in fd)\n%%     check_memory_inc(PrevMemInfo, NewMemInfo, 0, 0, true),\n    ok.\n\nadd_kill_nodes_50(_Config) ->\n    config:write(gossip_vivaldi_latency_timeout, 0),\n    {[TmpNode], []} = api_vm:add_nodes(1), % loads all code needed for joins\n    {[TmpNode], []} = api_vm:kill_nodes_by_name([TmpNode]), % loads all code needed for killing nodes\n    garbage_collect_all(),\n    OldProcesses = unittest_helper:get_processes(),\n    PrevMemInfo = get_meminfo(),\n    {NewNodes, []} = api_vm:add_nodes(50),\n    unittest_helper:wait_for_stable_ring_deep(),\n    _ = api_vm:kill_nodes_by_name(NewNodes),\n    timer:sleep(1000), % wait for vivaldi_latency timeouts\n    unittest_helper:wait_for_stable_ring_deep(),\n    _ = garbage_collect_all_and_check(10, PrevMemInfo, 0, 0, true),\n    NewProcesses = unittest_helper:get_processes(),\n    {_OnlyOld, _Both, OnlyNew} =\n        util:split_unique(OldProcesses, NewProcesses,\n                          fun(P1, P2) ->\n                                  element(1, P1) =< element(1, P2)\n                          end, fun(_P1, P2) -> P2 end),\n    ?equals(OnlyNew, []),\n%%     ct:pal(\"~p~n\", [erlang:memory()]),\n    % TODO: add memory check? (process state may increase, e.g. dead nodes in fd)\n%%     check_memory_inc(PrevMemInfo, NewMemInfo, 0, 0, true),\n    ok.\n\n%% @doc Starts garbage collection for all processes.\ngarbage_collect_all() ->\n    _ = [erlang:garbage_collect(Pid) || Pid <- processes()],\n    % wait a bit for the gc to finish\n    timer:sleep(1000).\n\ngarbage_collect_all_and_check(0, _MemInfo, _NewItems, _AddedSize, _TestStartsNodes) ->\n    get_meminfo();\ngarbage_collect_all_and_check(Retries, MemInfo, NewItems, AddedSize, TestStartsNodes) when Retries > 0 ->\n    garbage_collect_all_and_check_(1, Retries + 1, MemInfo, NewItems, AddedSize, TestStartsNodes).\n\ngarbage_collect_all_and_check_(Max, Max, _MemInfo, _NewItems, _AddedSize, _TestStartsNodes) ->\n    get_meminfo();\ngarbage_collect_all_and_check_(N, Max, MemInfo, NewItems, AddedSize, TestStartsNodes) when N < Max ->\n    garbage_collect_all(),\n    MemInfo2 = get_meminfo(),\n    ct:pal(\"gc changes (~B):~nbin: ~12.B, atom: ~8.B, procs: ~10.B, ets: ~10.B~n    (+ ~10.B)      (+ ~6.B)       (+ ~8.B)     (+ ~8.B)\",\n           [N, MemInfo2#mem_info.binary, MemInfo2#mem_info.atom_used,\n            MemInfo2#mem_info.processes_used, MemInfo2#mem_info.ets,\n            MemInfo2#mem_info.binary - MemInfo#mem_info.binary,\n            MemInfo2#mem_info.atom_used - MemInfo#mem_info.atom_used,\n            MemInfo2#mem_info.processes_used - MemInfo#mem_info.processes_used,\n            MemInfo2#mem_info.ets - MemInfo#mem_info.ets]),\n    case check_memory_inc_bool(MemInfo, MemInfo2, NewItems, AddedSize, TestStartsNodes) of\n        true  -> MemInfo2;\n        false -> garbage_collect_all_and_check_(N + 1, Max, MemInfo, NewItems, AddedSize, TestStartsNodes)\n    end.\n"
  },
  {
    "path": "test/merkle_tree_SUITE.erl",
    "content": "%  @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <MLange@informatik.hu-berlin.de>\n%% @doc    Tests for merkle tree module.\n%% @end\n%% @version $Id$\n-module(merkle_tree_SUITE).\n-author('mlange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n\nall() -> [\n          %tester_branch_bucket,\n          tester_size,\n          tester_store_to_dot,\n          tester_tree_hash,\n          tester_insert_list,\n          tester_bulk_insert,\n          tester_iter,\n          tester_lookup,\n          test_empty\n          %eprof,\n          %fprof\n         ].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 20}}\n    ].\n\ninit_per_suite(Config) ->\n    rt_SUITE:register_value_creator(),\n    unittest_helper:start_minimal_procs(Config, [], true).\n\nend_per_suite(Config) ->\n    rt_SUITE:unregister_value_creator(),\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\neprof(_) ->\n    L=0,\n    R=193307343591240590005637476551917548364,\n    ToAdd=1273,\n    \n    I = intervals:new('[', L, R, ']'),\n    ct:pal(\"L=~p ; R=~p ; ToAdd=~p\", [L, R, ToAdd]),\n    Keys = db_generator:get_db(I, ToAdd, uniform, [{output, list_keytpl}]),\n    ct:pal(\"DB GEN OK\"),\n\n    _ = eprof:start(),\n    Fun = fun() -> merkle_tree:new(I, Keys, []) end,\n    eprof:profile([], Fun),\n    eprof:stop_profiling(),\n    eprof:analyze(procs, [{sort, time}]),\n    \n    ok.\n\nfprof(_) ->\n    L=0,\n    R=193307343591240590005637476551917548364,\n    ToAdd=1273,\n    \n    I = intervals:new('[', L, R, ']'),\n    ct:pal(\"L=~p ; R=~p ; ToAdd=~p\", [L, R, ToAdd]),\n    Keys = db_generator:get_db(I, ToAdd, uniform, [{output, list_keytpl}]),\n    ct:pal(\"DB GEN OK\"),\n\n    fprof:apply(merkle_tree, new, [I, Keys, []]),\n    fprof:profile(),\n    fprof:analyse(),\n    \n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_lookup(intervals:key(), intervals:key()) -> true.\nprop_lookup(L, R) ->\n    ToAdd = 200,\n    I = unittest_helper:build_interval(L, R),\n    Tree = build_tree(I, [], ToAdd, uniform),\n    Branch = merkle_tree:get_branch_factor(Tree),\n    SplitI = intervals:split(I, Branch),\n    SplitI2 = intervals:split(I, Branch + 1),\n    lists:foreach(\n      fun(SubI) ->\n              ?assert(merkle_tree:lookup(SubI, Tree) =/= not_found)\n      end, SplitI),\n    lists:foreach(\n      fun(SubI) ->\n              ?assert(merkle_tree:lookup(SubI, Tree) =:= not_found)\n      end, SplitI2),\n    true.\n\ntester_lookup(_) ->\n    case ?RT:get_replica_keys(?MINUS_INFINITY) of\n        [_K1, K2, _K3, K4] -> prop_lookup(K4, K2);\n        _ -> ok\n    end,\n    tester:test(?MODULE, prop_lookup, 2, 100, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ntest_empty(_) ->\n    Tree1 = merkle_tree:new(intervals:empty()),\n    Tree1a = merkle_tree:gen_hash(Tree1),\n    Tree2 = merkle_tree:new(intervals:all()),\n    Tree2a = merkle_tree:gen_hash(Tree2),\n    Tree3 = merkle_tree:new(intervals:empty(), [], []),\n    Tree3a = merkle_tree:gen_hash(Tree3),\n    Tree4 = merkle_tree:new(intervals:all(), [], []),\n    Tree4a = merkle_tree:gen_hash(Tree4),\n    Tree5 = merkle_tree:new(intervals:empty(), [], [{keep_bucket, true}]),\n    Tree5a = merkle_tree:gen_hash(Tree5),\n    Tree6 = merkle_tree:new(intervals:all(), [], [{keep_bucket, true}]),\n    Tree6a = merkle_tree:gen_hash(Tree6),\n\n    ?assert(merkle_tree:is_empty(Tree1)),\n    ?assert(merkle_tree:is_empty(Tree1a)),\n    ?assert(merkle_tree:is_empty(Tree2)),\n    ?assert(merkle_tree:is_empty(Tree2a)),\n    ?assert(merkle_tree:is_empty(Tree3)),\n    ?assert(merkle_tree:is_empty(Tree3a)),\n    ?assert(merkle_tree:is_empty(Tree4)),\n    ?assert(merkle_tree:is_empty(Tree4a)),\n    ?assert(merkle_tree:is_empty(Tree5)),\n    ?assert(merkle_tree:is_empty(Tree5a)),\n    ?assert(merkle_tree:is_empty(Tree6)),\n    ?assert(merkle_tree:is_empty(Tree6a)),\n\n    ?equals(Tree1, Tree1a),\n    ?equals(Tree2, Tree2a),\n    ?equals(Tree3, Tree3a),\n    ?equals(Tree4, Tree4a),\n    ?equals(Tree5, Tree5a),\n    ?equals(Tree6, Tree6a),\n\n    ?equals(Tree1, Tree3),\n    ?equals(Tree2, Tree4),\n    ?equals(merkle_tree:get_root(Tree3), merkle_tree:get_root(Tree5)),\n    ?equals(merkle_tree:get_root(Tree4), merkle_tree:get_root(Tree6)).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc Tests branching and bucketing\n-spec prop_branch_bucket(intervals:key(), intervals:key(),\n                         BranchFactor::2..16, BucketSize::24..512) -> true.\nprop_branch_bucket(L, R, Branch, Bucket) ->\n    I = unittest_helper:build_interval(L, R),\n    Config = [{branch_factor, Branch}, {bucket_size, Bucket}],\n    Tree1 = build_tree(I, Config, Bucket, uniform),\n    Tree2 = build_tree(I, Config, Bucket + 1, uniform),\n    ct:pal(\"Branch=~p ; Bucket=~p ; Tree1Size=~p ; Tree2Size=~p\",\n           [Branch, Bucket, merkle_tree:size(Tree1), merkle_tree:size(Tree2)]),\n    ?equals(merkle_tree:size(Tree1), 1),\n    ?equals(merkle_tree:size(Tree2), Branch + 1).\n\ntester_branch_bucket(_) ->\n    tester:test(?MODULE, prop_branch_bucket, 4, 10, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc Tests hash generation\n-spec prop_tree_hash(intervals:key(), intervals:key(), 1..100) -> true.\nprop_tree_hash(L, R, ToAdd) ->\n    I = unittest_helper:build_interval(L, R),\n    DB1 = db_generator:get_db(I, ToAdd, uniform, [{output, list_keytpl}]),\n    DB1a = util:shuffle(DB1),\n    DB1Size = length(DB1),\n    \n    DB1Tree1 = merkle_tree:new(I, DB1, []),\n    DB1Tree2 = merkle_tree:new(I, DB1, []),\n    DB1Tree2a = merkle_tree:gen_hash(merkle_tree:new(I, DB1, [])),\n    DB1Tree2b = merkle_tree:gen_hash(merkle_tree:insert_list(DB1, merkle_tree:new(I, [{keep_bucket, true}]))),\n    DB1Tree2c = merkle_tree:gen_hash(merkle_tree:insert_list(DB1, merkle_tree:new(I, [{keep_bucket, true}])), true),\n    \n    DB1aTree1 = merkle_tree:new(I, DB1a, []),\n    DB1aTree2 = merkle_tree:new(I, DB1a, []),\n    DB1aTree2a = merkle_tree:gen_hash(merkle_tree:new(I, DB1a, [])),\n    DB1aTree2b = merkle_tree:gen_hash(merkle_tree:insert_list(DB1a, merkle_tree:new(I, [{keep_bucket, true}]))),\n    DB1aTree2c = merkle_tree:gen_hash(merkle_tree:insert_list(DB1a, merkle_tree:new(I, [{keep_bucket, true}])), true),\n    \n    DB3Size = DB1Size + 1,\n    DB3Tree1 = build_tree(I, DB3Size, uniform),\n    \n    DB1RootHash1 = merkle_tree:get_hash(DB1Tree1),\n    DB1RootHash2 = merkle_tree:get_hash(DB1Tree2),\n    DB1RootHash2a = merkle_tree:get_hash(DB1Tree2a),\n    DB1RootHash2b = merkle_tree:get_hash(DB1Tree2b),\n    DB1RootHash2c = merkle_tree:get_hash(DB1Tree2c),\n    \n    DB1aRootHash1 = merkle_tree:get_hash(DB1aTree1),\n    DB1aRootHash2 = merkle_tree:get_hash(DB1aTree2),\n    DB1aRootHash2a = merkle_tree:get_hash(DB1aTree2a),\n    DB1aRootHash2b = merkle_tree:get_hash(DB1aTree2b),\n    DB1aRootHash2c = merkle_tree:get_hash(DB1aTree2c),\n    \n    DB3RootHash1 = merkle_tree:get_hash(DB3Tree1),\n    \n    ?equals(DB1RootHash1, DB1RootHash2),\n    ?equals(DB1RootHash2, DB1RootHash2a),\n    ?equals(DB1RootHash2, DB1RootHash2b),\n    ?equals(DB1RootHash2, DB1RootHash2c),\n    \n    ?equals(DB1aRootHash1, DB1aRootHash2),\n    ?equals(DB1aRootHash2, DB1aRootHash2a),\n    ?equals(DB1aRootHash2, DB1aRootHash2b),\n    ?equals(DB1aRootHash2, DB1aRootHash2c),\n    \n    ?equals(DB1RootHash1, DB1aRootHash1),\n    \n    ?compare(fun erlang:'>'/2, DB1RootHash1, 0),\n    ?compare(fun erlang:'>'/2, DB3RootHash1, 0),\n\n    MTSize = merkle_tree:size_detail(DB3Tree1),\n    case element(4, MTSize) of\n        DB1Size ->\n            % could not split the interval any further to add one more item\n            % -> items should be equal and thus also the hashes\n            ?equals(DB3RootHash1, DB1RootHash1);\n        DB3Size ->\n            % there is one more item and thus the hashes should differ\n            ?compare(fun erlang:'=/='/2, DB3RootHash1, DB1RootHash1);\n        _ ->\n            ?ct_fail(\"DB1: ~B, DB3: ~B, Other: ~p\", [DB1Size, DB3Size, MTSize])\n    end.\n\ntester_tree_hash(_) ->\n    case rt_SUITE:default_rt_has_chord_keys() of\n        true ->\n            prop_tree_hash(310893024101176788593096495898246585537, 310893024101176788593096495898246585538, 83),\n            prop_tree_hash(310893024101176788593096495898246585537, 56222597632513189683777859800513757781, 83);\n        _ -> ok\n    end,\n    tester:test(?MODULE, prop_tree_hash, 3, 100, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_insert_list(intervals:key(), intervals:key(), 1..2500) -> true.\nprop_insert_list(L, R, Count) ->\n    I = unittest_helper:build_interval(L, R),\n    DB = db_generator:get_db(I, Count, uniform, [{output, list_keytpl}]),\n    Tree = merkle_tree:insert_list(DB, merkle_tree:new(I, [{keep_bucket, true}])),\n    ItemCount = merkle_tree:get_item_count(Tree),\n    ?equals_w_note(Count, ItemCount,\n                   io_lib:format(\"DB=~p~nTree=~p~nCount=~p; DBSize=~p; TreeCount=~p\",\n                                 [DB, Tree, Count, length(DB), ItemCount])).\n\ntester_insert_list(_) ->\n    tester:test(?MODULE, prop_insert_list, 3, 100, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_bulk_insert(intervals:key(), intervals:key(), 1..50) -> true.\nprop_bulk_insert(L, R, BucketSize) ->\n    I = unittest_helper:build_interval(L, R),\n    DB = db_generator:get_db(I, BucketSize, uniform, [{output, list_keytpl}]),\n    Tree1 = merkle_tree:new(I, DB, [{bucket_size, BucketSize}]),\n    Tree2 = merkle_tree:new(I, DB, [{bucket_size, BucketSize}]),\n    Tree3 = build_tree(I, [{bucket_size, BucketSize}], BucketSize * 2 + 1, uniform),\n    Size1 = merkle_tree:size(Tree1),\n    Size2 = merkle_tree:size(Tree2),\n    Size3 = merkle_tree:size(Tree3),\n    ?equals(Size1, Size2),\n    ?assert(Size1 < Size3).\n\ntester_bulk_insert(_) ->\n    tester:test(?MODULE, prop_bulk_insert, 3, 100, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_size(intervals:key(), intervals:key(), 1..100) -> true.\nprop_size(L, R, ToAdd) ->\n    I = unittest_helper:build_interval(L, R),\n    Tree = build_tree(I, ToAdd, uniform),\n    Size = merkle_tree:size(Tree),\n    {Inner, Leafs, EmptyLeafs, Items} = merkle_tree:size_detail(Tree),\n    ?equals_w_note(Size, Inner + Leafs,\n                   io_lib:format(\"TreeSize~nItemsAdded: ~p\n                                  Simple: ~p Nodes\n                                  InnerNodes: ~B   ;   Leafs: ~B   ;   Items: ~B\",\n                                 [ToAdd, Size, Inner, Leafs, Items])),\n    ?equals_w_note(Leafs, merkle_tree:get_leaf_count(Tree),\n                   io_lib:format(\"TreeSize~nItemsAdded: ~p\n                                  Simple: ~p Nodes\n                                  InnerNodes: ~B   ;   Leafs: ~B   ;   Items: ~B\",\n                                 [ToAdd, Size, Inner, Leafs, Items])),\n    ?equals_w_note(Items, merkle_tree:get_item_count(Tree),\n                   io_lib:format(\"TreeSize~nItemsAdded: ~p\n                                  Simple: ~p Nodes\n                                  InnerNodes: ~B   ;   Leafs: ~B   ;   Items: ~B\",\n                                 [ToAdd, Size, Inner, Leafs, Items])),\n    ?equals_w_note(EmptyLeafs,\n                   length([ok || Leaf <- merkle_tree:get_leaves(Tree),\n                                 merkle_tree:is_empty(Leaf)]),\n                   io_lib:format(\"TreeSize~nItemsAdded: ~p\n                                  Simple: ~p Nodes\n                                  InnerNodes: ~B   ;   Leafs: ~B   ;   Items: ~B\",\n                                 [ToAdd, Size, Inner, Leafs, Items])).\n    \ntester_size(_) ->\n  tester:test(?MODULE, prop_size, 3, 100, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_iter(intervals:key(), intervals:key(), 1000..2000) -> true.\nprop_iter(L, R, ToAdd) ->\n    I = unittest_helper:build_interval(L, R),\n    Tree = build_tree(I, ToAdd, uniform),\n    {Inner, Leafs, _EmptyLeafs, _Items} = merkle_tree:size_detail(Tree),\n    Count = iterate(merkle_tree:iterator(Tree), fun(_, Acc) -> Acc + 1 end, 0),\n    ?equals_w_note(Count, Inner + Leafs,\n                   io_lib:format(\"Args: Interval=[~p, ~p] - ToAdd =~p~n\",\n                                 [L, R, ToAdd])).\n\ntester_iter(_Config) ->\n    tester:test(?MODULE, prop_iter, 3, 100, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_store_to_dot(intervals:key(), intervals:key(), 1..1000) -> true.\nprop_store_to_dot(L, R, ToAdd) ->\n    ct:pal(\"PARAMS: L=~p ; R=~p ; ToAdd=~p\", [L, R, ToAdd]),\n    I = unittest_helper:build_interval(L, R),\n    Tree = build_tree(I, ToAdd, uniform),\n    {Inner, Leafs, EmptyLeafs, Items} = merkle_tree:size_detail(Tree),\n    ct:pal(\"Tree Size Added =~B - Inner=~B ; Leafs=~B (empty: ~B) ; Items=~B\n            Saved to ../MerkleTree.png\", [ToAdd, Inner, Leafs, EmptyLeafs, Items]),\n    merkle_tree:store_graph(Tree, \"MerkleTree\"),\n    true.\n\ntester_store_to_dot(_) ->\n  tester:test(?MODULE, prop_store_to_dot, 3, 1, [{threads, 1}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Helper\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec build_tree(I, ToAdd, Distribution) -> Tree when\n    is_subtype(I,            intervals:interval()),\n    is_subtype(ToAdd,        non_neg_integer()),\n    is_subtype(Distribution, db_generator:distribution()),\n    is_subtype(Tree,         merkle_tree:merkle_tree()).\nbuild_tree(Interval, ToAdd, Distribution) ->\n    build_tree(Interval, [], ToAdd, Distribution).\n\n-spec build_tree(I, Config, ToAdd, Distribution) -> Tree when\n    is_subtype(I,            intervals:interval()),\n    is_subtype(Config,       merkle_tree:mt_config_params()),\n    is_subtype(ToAdd,        non_neg_integer()),\n    is_subtype(Distribution, db_generator:distribution()),\n    is_subtype(Tree,         merkle_tree:merkle_tree()).\nbuild_tree(Interval, Config, ToAdd, Distribution) ->\n    Keys = db_generator:get_db(Interval, ToAdd, Distribution, [{output, list_keytpl}]),\n    merkle_tree:new(Interval, Keys, Config).\n\n-spec iterate(merkle_tree:mt_iter(), fun((merkle_tree:mt_node(), T) -> T), T) -> T.\niterate(none, _, Acc) -> Acc;\niterate(Iter, Fun, Acc) ->\n    Next = merkle_tree:next(Iter),\n    case Next of\n        none -> Acc;\n        {Node, Iter2} -> iterate(Iter2, Fun, Fun(Node, Acc))\n    end.\n"
  },
  {
    "path": "test/mockup.erl",
    "content": "%  @copyright 2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Useful methods for mockup modules, e.g. matching of messages.\n%% @end\n%% @version $Id$\n-module(mockup).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n-export([match_any/2]).\n\n-export_type([match_spec/0]).\n\n-type match_variable() ::\n    '$1' | '$2' | '$3' | '$4' | '$5' | '$6' | '$7' | '$8' | '$9'.\n-type match_head_part() ::  '_' | match_variable() |term().\n-type match_head_wc() :: '_'.\n% note: head and actions currently only defined up to 9-tuples\n-type match_head_1() :: {match_head_part()}.\n-type match_head_2() :: {match_head_part(), match_head_part()}.\n-type match_head_3() :: {match_head_part(), match_head_part(), match_head_part()}.\n-type match_head_4() :: {match_head_part(), match_head_part(), match_head_part(), match_head_part()}.\n-type match_head_5() :: {match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part()}.\n-type match_head_6() :: {match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part()}.\n-type match_head_7() :: {match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part()}.\n-type match_head_8() :: {match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part()}.\n-type match_head_9() :: {match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part(), match_head_part()}.\n%% -type match_condition() :: none().\n-type match_counter() :: pos_integer() | infinity.\n-type match_action_wc() :: drop_msg | fun((Msg::comm:message(), ModuleState) -> ModuleState).\n-type match_action_1() :: drop_msg | fun((Msg::{atom()}, ModuleState) -> ModuleState).\n-type match_action_2() :: drop_msg | fun((Msg::{atom(), term()}, ModuleState) -> ModuleState).\n-type match_action_3() :: drop_msg | fun((Msg::{atom(), term(), term()}, ModuleState) -> ModuleState).\n-type match_action_4() :: drop_msg | fun((Msg::{atom(), term(), term(), term()}, ModuleState) -> ModuleState).\n-type match_action_5() :: drop_msg | fun((Msg::{atom(), term(), term(), term(), term()}, ModuleState) -> ModuleState).\n-type match_action_6() :: drop_msg | fun((Msg::{atom(), term(), term(), term(), term(), term()}, ModuleState) -> ModuleState).\n-type match_action_7() :: drop_msg | fun((Msg::{atom(), term(), term(), term(), term(), term(), term()}, ModuleState) -> ModuleState).\n-type match_action_8() :: drop_msg | fun((Msg::{atom(), term(), term(), term(), term(), term(), term(), term()}, ModuleState) -> ModuleState).\n-type match_action_9() :: drop_msg | fun((Msg::{atom(), term(), term(), term(), term(), term(), term(), term(), term()}, ModuleState) -> ModuleState).\n-type match_spec() ::\n    {Head::match_head_wc(), Conditions::[], Count::match_counter(), Action::match_action_wc()} |\n    {Head::match_head_1(), Conditions::[], Count::match_counter(), Action::match_action_1()} |\n    {Head::match_head_2(), Conditions::[], Count::match_counter(), Action::match_action_2()} |\n    {Head::match_head_3(), Conditions::[], Count::match_counter(), Action::match_action_3()} |\n    {Head::match_head_4(), Conditions::[], Count::match_counter(), Action::match_action_4()} |\n    {Head::match_head_5(), Conditions::[], Count::match_counter(), Action::match_action_5()} |\n    {Head::match_head_6(), Conditions::[], Count::match_counter(), Action::match_action_6()} |\n    {Head::match_head_7(), Conditions::[], Count::match_counter(), Action::match_action_7()} |\n    {Head::match_head_8(), Conditions::[], Count::match_counter(), Action::match_action_8()} |\n    {Head::match_head_9(), Conditions::[], Count::match_counter(), Action::match_action_9()}.\n\n-spec match_any(Msg::comm:message(), MatchSpecs::[match_spec()]) -> false | {true, Match::match_spec(), NewMatchSpecs::[match_spec()]}.\nmatch_any(Msg, MatchSpecs) ->\n    match_any(Msg, MatchSpecs, []).\n\n-spec match_any(Msg::comm:message(), MatchSpecs::[match_spec()], ProcessedMatchSpecs::[match_spec()]) -> false | {true, Match::match_spec(), NewMatchSpecs::[match_spec()]}.\nmatch_any(_Msg, [], _ProcessedMatchSpecs) ->\n    false;\nmatch_any(Msg, MatchSpecs = [First = {Head, Conditions, Count, Action} | Rest], ProcessedMatchSpecs) ->\n    case match(Msg, First) of\n        false ->\n            match_any(Msg, Rest, [First | ProcessedMatchSpecs]);\n        true when Count =:= infinity ->\n            {true, First, lists:append(ProcessedMatchSpecs, MatchSpecs)};\n        true when (Count - 1) > 0 ->\n            {true, First, lists:append([ProcessedMatchSpecs, [{Head, Conditions, Count - 1, Action}], Rest])};\n        true ->\n            {true, First, lists:append([ProcessedMatchSpecs, Rest])}\n    end.\n\n-spec match(Msg::comm:message(), match_spec()) -> boolean().\nmatch(_Msg, {'_', _Conditions = [], _Count, _Action}) ->\n    true;\nmatch(Msg, {Head, _Conditions = [], _Count, _Action}) when tuple_size(Msg) =/= tuple_size(Head) ->\n    false;\nmatch(Msg, {Head, _Conditions = [], _Count, _Action}) ->\n    lists:all(fun(X) ->\n                      match_head(Msg, erlang:element(X, Msg), erlang:element(X, Head))\n              end, lists:seq(1, tuple_size(Msg))).\n\nmatch_head(_Msg, _MsgElem, '_') -> true;\nmatch_head(Msg, MsgElem, '$1') -> MsgElem =:= erlang:element(1, Msg);\nmatch_head(Msg, _MsgElem, '$2') when tuple_size(Msg) < 2 -> false;\nmatch_head(Msg, MsgElem, '$2') -> MsgElem =:= erlang:element(2, Msg);\nmatch_head(Msg, _MsgElem, '$3') when tuple_size(Msg) < 3 -> false;\nmatch_head(Msg, MsgElem, '$3') -> MsgElem =:= erlang:element(3, Msg);\nmatch_head(Msg, _MsgElem, '$4') when tuple_size(Msg) < 4 -> false;\nmatch_head(Msg, MsgElem, '$4') -> MsgElem =:= erlang:element(4, Msg);\nmatch_head(Msg, _MsgElem, '$5') when tuple_size(Msg) < 5 -> false;\nmatch_head(Msg, MsgElem, '$5') -> MsgElem =:= erlang:element(5, Msg);\nmatch_head(Msg, _MsgElem, '$6') when tuple_size(Msg) < 6 -> false;\nmatch_head(Msg, MsgElem, '$6') -> MsgElem =:= erlang:element(6, Msg);\nmatch_head(Msg, _MsgElem, '$7') when tuple_size(Msg) < 7 -> false;\nmatch_head(Msg, MsgElem, '$7') -> MsgElem =:= erlang:element(7, Msg);\nmatch_head(Msg, _MsgElem, '$8') when tuple_size(Msg) < 8 -> false;\nmatch_head(Msg, MsgElem, '$8') -> MsgElem =:= erlang:element(8, Msg);\nmatch_head(Msg, _MsgElem, '$9') when tuple_size(Msg) < 9 -> false;\nmatch_head(Msg, MsgElem, '$9') -> MsgElem =:= erlang:element(9, Msg);\nmatch_head(_Msg, MsgElem, HeadElem) -> MsgElem =:= HeadElem.\n"
  },
  {
    "path": "test/mockup_dht_node.erl",
    "content": "%  @copyright 2011-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    A dht_node mockup that can ignore some messages or process them\n%%         differently compared to dht_node.\n%% @end\n%% @version $Id$\n-module(mockup_dht_node).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-include(\"scalaris.hrl\").\n\n-behaviour(gen_component).\n\n-export([start_link/2, on/2, init/1,\n         is_alive/1, is_alive_no_slide/1, is_alive_fully_joined/1]).\n\n-type message() ::\n        comm:message() |\n        {mockup_dht_node, add_match_specs, DropSpecs::[mockup:match_spec()]} |\n        {mockup_dht_node, clear_match_specs}.\n-type module_state() ::\n        dht_node_join:join_state() | dht_node_state:state() |\n        {'$gen_component', [{on_handler, Handler::gen_component:handler()}], dht_node_state:state() | dht_node_join:join_state()}.\n-type state() :: {Module::module(), Handler::gen_component:handler(), ModuleState::module_state(), MsgDropSpecs::[mockup:match_spec()]}.\n\n%-define(TRACE(X,Y), ct:pal(X,Y)).\n-define(TRACE(X,Y), ok).\n-define(TRACE_SEND(Pid, Msg), ?TRACE(\"[ ~.0p ] to ~.0p: ~.0p~n\", [self(), Pid, Msg])).\n-define(TRACE1(Msg, State),\n        ?TRACE(\"[ ~.0p ]~n  Msg: ~.0p~n  State: ~.0p~n\", [self(), Msg, State])).\n\n-spec on(message(), state()) -> state().\non(_Msg = {mockup_dht_node, add_match_specs, DropSpecs},\n    _State = {Module, Handler, ModuleState, MsgDropSpecs}) ->\n    ?TRACE1(_Msg, _State),\n    {Module, Handler, ModuleState, lists:append(MsgDropSpecs, DropSpecs)};\non(_Msg = {mockup_dht_node, clear_match_specs},\n    _State = {Module, Handler, ModuleState, _MsgDropSpecs}) ->\n    ?TRACE1(_Msg, _State),\n    {Module, Handler, ModuleState, []};\non(Msg, State = {Module, Handler, ModuleState, MsgDropSpecs}) ->\n%%     ?TRACE1(Msg, State),\n    case mockup:match_any(Msg, MsgDropSpecs) of\n        {true, {_Head, _Conditions, _Count, drop_msg}, NewMatchSpecs} ->\n            ?TRACE(\"[ ~.0p ] ignoring ~.0p~n\", [self(), Msg]),\n            {Module, Handler, ModuleState, NewMatchSpecs};\n        {true, {_Head, _Conditions, _Count, ActionFun}, NewMatchSpecs}\n          when is_function(ActionFun) ->\n            ?TRACE(\"[ ~.0p ] calling ~.0p for message ~.0p~n\", [self(), ActionFun, Msg]),\n            NewModuleState = ActionFun(Msg, ModuleState),\n            module_state_to_my_state(NewModuleState, {Module, Handler, ModuleState, NewMatchSpecs});\n        false ->\n            NewModuleState = Handler(Msg, ModuleState),\n            module_state_to_my_state(NewModuleState, State)\n    end.\n\n-spec start_link(pid_groups:groupname(), [tuple()]) -> {ok, pid()}.\nstart_link(DHTNodeGroup, Options) ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, Options,\n                             [{pid_groups_join_as, DHTNodeGroup, dht_node}, {wait_for_init}]).\n\n-spec init(Options::[tuple()]) -> state().\ninit(Options) ->\n    case lists:keytake(match_specs, 1, Options) of\n        {value, {match_specs, MatchSpecs}, RestOptions} ->\n            ok;\n        false -> RestOptions = Options,\n                 MatchSpecs = [],\n                 ok\n    end,\n    ModuleState = dht_node:init(RestOptions),\n    module_state_to_my_state(ModuleState, {dht_node, fun dht_node:on/2, ModuleState, MatchSpecs}).\n\n-spec module_state_to_my_state(module_state(), state()) -> state().\nmodule_state_to_my_state(ModuleState, {Module, OldHandler, _, NewMatchSpecs}) ->\n    case ModuleState of\n        {'$gen_component', Commands, ModuleRealState} ->\n            case lists:keyfind(on_handler, 1, Commands) of\n                {on_handler, NewHandler} ->\n                    {Module, NewHandler, ModuleRealState, NewMatchSpecs};\n                false ->\n                    {'$gen_component', Commands,\n                     {Module, OldHandler, ModuleRealState, NewMatchSpecs}}\n            end;\n        _ -> {Module, OldHandler, ModuleState, NewMatchSpecs}\n    end.\n\n-spec is_alive(State::state() | term()) -> boolean().\nis_alive({_Module, _Handler, ModuleState, _MsgDropSpecs}) ->\n    dht_node:is_alive(ModuleState);\nis_alive(_) -> false.\n\n-spec is_alive_no_slide(State::state() | term()) -> boolean().\nis_alive_no_slide({_Module, _Handler, ModuleState, _MsgDropSpecs}) ->\n    dht_node:is_alive_no_slide(ModuleState);\nis_alive_no_slide(_) -> false.\n\n-spec is_alive_fully_joined(State::state() | term()) -> boolean().\nis_alive_fully_joined({_Module, _Handler, ModuleState, _MsgDropSpecs}) ->\n    dht_node:is_alive_fully_joined(ModuleState);\nis_alive_fully_joined(_) -> false.\n"
  },
  {
    "path": "test/mockup_l_on_cseq.erl",
    "content": "%  @copyright 2009-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Vivaldi helper module for measuring latency between nodes.\n%% @end\n%% @version $$\n-module(mockup_l_on_cseq).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-behaviour(gen_component).\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-dialyzer({[no_opaque, no_return], init/1}).\n\n-export([on/2, init/1, start_link/0]).\n\n% lease mgmt.\n-export([create_two_adjacent_leases/1]).\n-export([create_lease/3]).\n\n% public api\n-export([get_renewal_counter/0, get_lease_list/0,\n         set_message_filter/2, reset_message_filter/0]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% public API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec create_two_adjacent_leases(comm:mypid_plain()) -> {l_on_cseq:lease_t(), l_on_cseq:lease_t()}.\ncreate_two_adjacent_leases(Owner) ->\n    % we have a symmetric ring with four nodes, so we are going to use\n    % the ranges: 0-1 and 1-2.\n    Key1 = rt_SUITE:number_to_key(0),\n    Key2 = rt_SUITE:number_to_key(2),\n    SplitKey = ?RT:get_split_key(Key1, Key2, {1, 2}),\n    L1 = create_lease(Key1, SplitKey, Owner),\n    L2 = create_lease(SplitKey, Key2, Owner),\n    ct:pal(\"mock leases~n~w~n~w~n\", [L1, L2]),\n    {L1, L2}.\n\n-spec create_lease(?RT:key(), ?RT:key(), comm:mypid_plain()) -> l_on_cseq:lease_t().\ncreate_lease(From, To, Owner) ->\n    L = l_on_cseq:unittest_create_lease_with_range(From, To, Owner),\n    Range = node:mk_interval_between_ids(From, To),\n    Id = l_on_cseq:id(Range),\n    DB = rbrcseq:get_db_for_id(lease_db, Id),\n    rbrcseq:qwrite(DB, self(), Id, l_on_cseq,\n                   fun (_Current, _WriteFile, _Next) ->\n                           {true, null}\n                   end,\n                   L),\n    receive\n        X -> ct:log(\"qwrite~n~w~n\", [X])\n    end,\n    %trace_mpath:thread_yield(),\n    %receive\n    %    ?SCALARIS_RECV(\n    %       {mock, ok}, %% ->\n    %       ok\n    %      )\n    %end,\n    L.\n\n-spec get_renewal_counter() -> non_neg_integer().\nget_renewal_counter() ->\n    comm:send_local(get_mock_pid(), {get_renewal_counter, self()}),\n    receive\n        {get_renewal_counter_response, Counter} ->\n             Counter\n    end.\n\n-spec get_lease_list() -> lease_list:lease_list().\nget_lease_list() ->\n    comm:send_local(get_mock_pid(), {get, lease_list, self()}),\n    receive\n        {get_response, List} ->\n             List\n    end.\n\n-spec set_message_filter(F::fun((comm:message()) -> boolean()),\n                         Owner::pid()) -> ok.\nset_message_filter(F, Owner) ->\n    comm:send_local(get_mock_pid(), {set_message_filter, F, Owner, self()}),\n    receive\n        {set_message_filter_response} ->\n             ok\n    end.\n\n-spec reset_message_filter() -> ok.\nreset_message_filter() ->\n    comm:send_local(get_mock_pid(), {reset_message_filter, self()}),\n    receive\n        {reset_message_filter_response} ->\n             ok\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Message Loop\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec on(comm:message(), dht_node_state:state()) ->\n                dht_node_state:state() | kill.\n% get renewal counter\non({get_renewal_counter, Pid}, State) ->\n    comm:send_local(Pid,\n                    {get_renewal_counter_response,\n                     erlang:get(renewal_counter)}),\n    State;\n\non({set_message_filter, F, Owner, Pid}, State) ->\n    erlang:put(message_filter, F),\n    erlang:put(owner, Owner),\n    comm:send_local(Pid, {set_message_filter_response}),\n    State;\n\non({reset_message_filter, Pid}, State) ->\n    erlang:put(message_filter, fun(_Msg) -> false end),\n    erlang:put(owner, undefined),\n    comm:send_local(Pid, {reset_message_filter_response}),\n    State;\n\n% get from dht_node_state\non({get, Key, Pid}, State) ->\n    comm:send_local(Pid,\n                    {get_response,\n                     dht_node_state:get(State, Key)}),\n    State;\n\n% intercept l_on_cseq messages\non(Msg, State) when element(1, Msg) =:= l_on_cseq ->\n    Filter = erlang:get(message_filter),\n    Intercept = Filter(Msg),\n    if\n        Intercept ->\n            intercept_message(Msg, State);\n        true ->\n            case Msg of\n                % intercept renew\n                {l_on_cseq, renew, _OldLease, _Mode} ->\n                    increment_renewal_counter(),\n                    l_on_cseq:on(Msg, State);\n                _ ->\n                    l_on_cseq:on(Msg, State)\n            end\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Init\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec init({}) -> dht_node_state:state().\ninit({}) ->\n    DHTNodeGrp = pid_groups:group_with(dht_node),\n    pid_groups:join_as(DHTNodeGrp, ?MODULE),\n    ct:log(\"mock_l_on_cseq running on ~w~n\", [self()]),\n    erlang:put(message_filter, fun(_Msg) -> false end),\n    erlang:put(renewal_counter, 0),\n    dht_node_state:new(rt_external_rt, rm_loop_state, dht_db).\n\n-spec start_link() -> {ok, pid()}.\nstart_link() ->\n    gen_component:start_link(?MODULE, fun ?MODULE:on/2, {}, [{wait_for_init}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Helper\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_mock_pid() -> pid() | failed.\nget_mock_pid() ->\n    pid_groups:find_a(?MODULE).\n\n-spec increment_renewal_counter() -> ok.\nincrement_renewal_counter() ->\n    erlang:put(renewal_counter, erlang:get(renewal_counter) + 1).\n\n-spec intercept_message(comm:message(), dht_node_state:state()) ->\n                dht_node_state:state().\nintercept_message(Msg, State) ->\n    Owner = erlang:get(owner),\n    comm:send_local(Owner, {intercepted_message, Msg}),\n    State.\n"
  },
  {
    "path": "test/mockup_l_on_cseq_SUITE.erl",
    "content": "%% @copyright 2012-2013, 2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for l_on_cseq based on a mockup dht_node\n%% @end\n%% @version $Id$\n-module(mockup_l_on_cseq_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\ngroups() ->\n    [{merge_tests, [sequence], [\n                                test_merge,\n                                test_merge_with_renewal_before_step1,\n                                test_merge_with_renewal_after_step1,\n                                test_merge_with_renewal_after_step2,\n                                test_merge_with_renewal_after_step3\n                               ]},\n     {split_tests, [sequence], [\n                                test_split\n                               ]}\n    ].\n\nall() ->\n    [\n     {group, merge_tests},\n     {group, split_tests}\n     ].\n\nsuite() -> [ {timetrap, {seconds, 90}} ].\n\ngroup(merge_tests) ->\n    [{timetrap, {seconds, 400}}];\ngroup(split_tests) ->\n    [{timetrap, {seconds, 400}}];\ngroup(_) ->\n    suite().\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) ->\n    unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(_TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_symmetric_ring([{config, [{log_path, PrivDir},\n                                                   {leases, true}]}]),\n    {ok, _} = mockup_l_on_cseq:start_link(),\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% merge unit tests\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ntest_merge(_Config) ->\n    % join group\n    Pid = pid_groups:find_a(mockup_l_on_cseq),\n    pid_groups:join(pid_groups:group_of(Pid)),\n    % create leases\n    {L1, L2} = mockup_l_on_cseq:create_two_adjacent_leases(comm:make_global(Pid)),\n    % do merge\n    % evil, but l_on_cseq:lease_merge sends to a real dht_node\n    comm:send_local(Pid, {l_on_cseq, merge, L1, L2, self()}),\n    receive\n        {merge, success, _L2, _L1} -> ok\n    end,\n    % no renews during merge!\n    ?equals(mockup_l_on_cseq:get_renewal_counter(), 0),\n    % check L1\n    ?equals(l_on_cseq:read(l_on_cseq:get_id(L1)), {ok, prbr_bottom}),\n    % check L2\n    case l_on_cseq:read(l_on_cseq:get_id(L2)) of\n        {ok, L2_} ->\n            ?equals(l_on_cseq:get_range(L2_),\n                    intervals:union(l_on_cseq:get_range(L1),\n                                    l_on_cseq:get_range(L2)));\n        {fail, not_found} -> ?ct_fail(\"{fail, not_found}\", [])\n    end,\n    true.\n\ntest_merge_with_renewal_before_step1(_Config) ->\n    test_merge_with_renewal_at(_Config, merge, first).\n\ntest_merge_with_renewal_after_step1(_Config) ->\n    test_merge_with_renewal_at(_Config, merge_reply_step1, second).\n\ntest_merge_with_renewal_after_step2(_Config) ->\n    test_merge_with_renewal_at(_Config, merge_reply_step2, first).\n\ntest_merge_with_renewal_after_step3(_Config) ->\n    test_merge_with_renewal_at(_Config, merge_reply_step3, second).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% split unit tests\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ntest_split(_Config) ->\n    % join group\n    Pid = pid_groups:find_a(mockup_l_on_cseq),\n    pid_groups:join(pid_groups:group_of(Pid)),\n    % create lease\n    L = mockup_l_on_cseq:create_lease(rt_SUITE:number_to_key(0),\n                                      rt_SUITE:number_to_key(16), comm:make_global(Pid)),\n    % do split\n    Range = l_on_cseq:get_range(L),\n    [R1, R2] = intervals:split(Range, 2),\n    % evil, but l_on_cseq:lease_merge sends to a real dht_node\n    comm:send_local(Pid, {l_on_cseq, split, L, R1, R2, self(), empty}),\n    {L1, L2} = receive\n                   {split, success, NewL1, NewL2} ->\n                       {NewL1, NewL2}\n               end,\n    % check L1\n    case l_on_cseq:read(l_on_cseq:get_id(L1)) of\n        {ok, L1_} ->\n            ?equals(l_on_cseq:get_range(L1_), R1),\n            ?equals(l_on_cseq:get_aux(L1_), empty),\n            true;\n        {fail, not_found} ->\n            ?ct_fail(\"{fail, not_found}\", [])\n    end,\n    % check L2\n    case l_on_cseq:read(l_on_cseq:get_id(L2)) of\n        {ok, L2_} ->\n            ?equals(l_on_cseq:get_range(L2_), R2),\n            ?equals(l_on_cseq:get_aux(L2_), empty),\n            true;\n        {fail, not_found} ->\n            ?ct_fail(\"{fail, not_found}\", [])\n    end,\n    LeaseList = mockup_l_on_cseq:get_lease_list(),\n    ActiveLease = lease_list:get_active_lease(LeaseList),\n    [PassiveLease] = lease_list:get_passive_leases(LeaseList),\n    ?equals(l_on_cseq:get_id(ActiveLease), l_on_cseq:get_id(L2)),\n    ?equals(l_on_cseq:get_id(PassiveLease), l_on_cseq:get_id(L1)),\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% merge unittest helper\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ntest_merge_with_renewal_at(_Config, Step, FirstOrSecond) ->\n    % join group\n    Pid = pid_groups:find_a(mockup_l_on_cseq),\n    pid_groups:join(pid_groups:group_of(Pid)),\n    % create lease\n    {L1, L2} = mockup_l_on_cseq:create_two_adjacent_leases(comm:make_global(Pid)),\n    % prepare message filter\n    mockup_l_on_cseq:set_message_filter(fun (Msg) ->\n                                            element(2, Msg) =:= Step\n                                        end,\n                                        self()),\n    % do merge\n    % evil, but l_on_cseq:lease_merge sends to a real dht_node\n    comm:send_local(Pid, {l_on_cseq, merge, L1, L2, self()}),\n    % wait for message_filter\n    %ct:pal(\"waiting for message filter\", []),\n    receive\n        {intercepted_message, Msg} ->\n            mockup_l_on_cseq:reset_message_filter(),\n            case FirstOrSecond of\n                first ->\n                    synchronous_renew(L1, passive);\n                second ->\n                    synchronous_renew(L2, active)\n            end,\n            comm:send_local(Pid, Msg);\n        X -> ct:pal(\"unknown message ~w\", [X])\n    end,\n    % wait for finish\n    ct:pal(\"waiting for merge success ~w\", [self()]),\n    receive\n        {merge, success, _L2, _L1} -> ok\n    end,\n    % no renews during merge!\n    ?equals(mockup_l_on_cseq:get_renewal_counter(), 1),\n    % check L1\n    case l_on_cseq:read(l_on_cseq:get_id(L1)) of\n        {ok, L1_} -> ?equals(L1_, prbr_bottom);\n        {fail, not_found} -> ?ct_fail(\"{fail, not_found}\", [])\n    end,\n    % check L2\n    case l_on_cseq:read(l_on_cseq:get_id(L2)) of\n        {ok, L2_} ->\n            ?equals(l_on_cseq:get_range(L2_),\n                    intervals:union(l_on_cseq:get_range(L1),\n                                    l_on_cseq:get_range(L2)));\n        {fail, not_found} -> ?ct_fail(\"{fail, not_found}\", [])\n    end,\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% helper\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% trigger lease renewal and wait until the renew happened\nsynchronous_renew(Lease, Mode) ->\n    Pid = pid_groups:find_a(mockup_l_on_cseq),\n    {ok, Current} = l_on_cseq:read(l_on_cseq:get_id(Lease)),\n    ct:pal(\"sync. renew ~w ~w~n~w\", [l_on_cseq:get_id(Lease), Mode, Current]),\n    mockup_l_on_cseq:set_message_filter(fun (Msg) ->\n                                            element(2, Msg) =:= renew_reply\n                                        end,\n                                        self()),\n    %comm:send_local(Pid,\n    %                {l_on_cseq, renew, Lease, Mode}),\n\n    l_on_cseq:lease_renew(Pid, Current, Mode),\n    receive\n        {intercepted_message, Msg} ->\n            mockup_l_on_cseq:reset_message_filter(),\n            comm:send_local(Pid, Msg)\n    end,\n    ct:pal(\"sync. renew after~n~w\", [l_on_cseq:read(l_on_cseq:get_id(Lease))]).\n\n"
  },
  {
    "path": "test/mr_SUITE.erl",
    "content": "% @copyright 2010-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc    Unit tests for the map reduce protocol.\n%% @end\n%% @version $Id$\n-module(mr_SUITE).\n-author('fajerski@zib.de').\n-vsn('$Id$').\n\n-compile([export_all]).\n\n%% no proto scheduler for this suite\n-define(proto_sched(_Action), ok).\n-define(proto_sched2(Action, Arg), proto_sched2_fun(Action, Arg)).\n\n-include(\"mr_SUITE.hrl\").\n\nall() ->\n    tests_avail() ++ [test_join, test_leave].\n\nsuite() -> [ {timetrap, {seconds, 15}} ].\n\n-spec proto_sched2_fun(setup, ThreadNum::pos_integer()) -> ok;\n                      (cleanup, PIDs::[pid() | atom()]) -> ok.\nproto_sched2_fun(setup, _Arg) ->\n    ok;\nproto_sched2_fun(cleanup, Arg) ->\n    _ = [util:wait_for_process_to_die(Pid) || Pid <- Arg],\n    ok.\n\ntest_join(_Config) ->\n    Pids = pid_groups:find_all(dht_node),\n    ct:pal(\"setting breakpoint before starting reduce phase\"),\n    NextPhase = fun(Msg, _State) ->\n            case Msg of\n                {bulkowner, start, _Id, _I, {mr, next_phase, _JobId, 1, _Interval}} ->\n                    comm:send_local(self(), Msg),\n                    drop_single;\n                _ ->\n                    false\n            end\n    end,\n    [gen_component:bp_set_cond(Pid, NextPhase, mr_bp) || Pid <- Pids],\n    ct:pal(\"starting job that triggers breakpoint\"),\n    MrPid = spawn_link(fun() ->\n                    ct:pal(\"starting mr job\"),\n                    Res = api_mr:start_job(get_wc_job_erl()),\n                    ct:pal(\"mr job finished\"),\n                    check_results(Res)\n            end),\n    timer:sleep(1000),\n    ct:pal(\"adding node to provoke slide\"),\n    _ = api_vm:add_nodes(2),\n    unittest_helper:check_ring_size_fully_joined(4),\n    unittest_helper:wait_for_stable_ring_deep(),\n    ct:pal(\"ring fully joined (4)\"),\n    ct:pal(\"removing breakpoints\"),\n    [gen_component:bp_del(Pid, mr_bp) || Pid <- Pids],\n    util:wait_for_process_to_die(MrPid),\n    ok.\n\ntest_leave(_Config) ->\n    [_] = api_vm:shutdown_nodes(1),\n    {[AddedNode], _} = api_vm:add_nodes(1),\n    unittest_helper:check_ring_size_fully_joined(2),\n    unittest_helper:wait_for_stable_ring_deep(),\n    Pids = pid_groups:find_all(dht_node),\n    ct:pal(\"setting breakpoint before starting reduce phase on ~p\", [Pids]),\n    NextPhase = fun(Msg, _State) ->\n            case Msg of\n                {bulkowner, start, _Id, _I, {mr, next_phase, _JobId, 1, _Interval}} ->\n                    comm:send_local(self(), Msg),\n                    drop_single;\n                _ ->\n                    false\n            end\n    end,\n    [gen_component:bp_set_cond(Pid, NextPhase, mr_bp) || Pid <- Pids],\n    ct:pal(\"starting job that triggers breakpoint\"),\n    MrPid = spawn_link(fun() ->\n                    ct:pal(\"starting mr job\"),\n                    Res = api_mr:start_job(get_wc_job_erl()),\n                    ct:pal(\"mr job finished\"),\n                    check_results(Res)\n            end),\n    timer:sleep(1000),\n    ct:pal(\"shutting down node to provoke slide\"),\n    ok = api_vm:shutdown_node(AddedNode),\n    [ct:pal(\"removing breakpoints on ~p...~p\", [Pid, gen_component:bp_del_async(Pid, mr_bp)]) || Pid <- Pids],\n    util:wait_for_process_to_die(MrPid),\n    ok.\n"
  },
  {
    "path": "test/mr_SUITE.hrl",
    "content": "% @copyright 2010-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc    Unit tests for the map reduce protocol.\n%% @end\n%% @version $Id$\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n\n-define(DATA, [{\"1\", \"MapReduce allows for distributed processing of the map and reduction operations\"}\n                , {\"3\", \"Provided each mapping operation can run independently all maps may be performed in parallel\"}\n                , {\"5\", \"This example data set contains one single word only once\"}\n                , {\"7\", \"Now I am too lazy to construct another sentence that actually makes sense\"}\n           ]).\n\ntests_avail() ->\n    [test_sane_result]. %%, test_error_on_kill].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_testcase(_TestCase, Config) ->\n    %% make new ring\n    unittest_helper:make_ring(2),\n    %% wait for all nodes to finish their join before writing data\n    unittest_helper:check_ring_size_fully_joined(2),\n    unittest_helper:wait_for_stable_ring_deep(),\n    Pid = erlang:spawn(fun add_data/0),\n    util:wait_for_process_to_die(Pid),\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\ntest_sane_result(_Config) ->\n    ?proto_sched2(setup, 1),\n    ?proto_sched(start),\n\n    Res = api_mr:start_job(get_wc_job_erl()),\n\n    %% each word only occurs once\n    ?proto_sched(stop),\n    ?proto_sched2(cleanup, []),\n    check_results(Res),\n    ok.\n\ntest_error_on_kill(_Config) ->\n    ?proto_sched2(setup, 2),\n    MrPid = spawn_link(fun() ->\n                               ?proto_sched(start),\n                               Res = api_mr:start_job(get_wc_job_erl()),\n                               ?equals(Res, {error, node_died}),\n                               ?proto_sched(stop)\n                       end),\n    KillPid = spawn_link(fun() ->\n                                 ?proto_sched(start),\n                                 [_] = api_vm:kill_nodes(1),\n                                 ?proto_sched(stop)\n                         end),\n    ?proto_sched2(cleanup, [MrPid, KillPid]),\n    ok.\n\nget_wc_job_erl() ->\n    Map = fun({_Key, Line}) ->\n        Tokens = string:tokens(Line, \" \\n,.;:?!()\\\"'-_\"),\n        [{string:to_lower(X),1} || X <- Tokens]\n    end,\n    Reduce = fun(KVList) ->\n            lists:map(fun({K, V}) ->\n                              {K, lists:sum(V)}\n                        end, KVList)\n    end,\n    {[{map, erlanon, Map},\n      {reduce, erlanon, Reduce}],\n     []}.\n\nadd_data() ->\n    X = lists:duplicate(length(?DATA), {ok}),\n    X = [api_tx:write(Key, {Key, Value}) || {Key, Value} <- ?DATA],\n    ok.\n\ncheck_results(Results) ->\n    Mapped = lists:foldl(fun({_K, V}, Acc) ->\n                                   T = string:tokens(string:to_lower(V), \" \"),\n                                   [{W, 1} || W <- T] ++ Acc\n                           end, [], ?DATA),\n    Expected = lists:foldl(fun({K, V}, Acc) ->\n                                   case lists:keyfind(K, 1, Acc) of\n                                       false ->\n                                           [{K, V} | Acc];\n                                       {T, C} ->\n                                           lists:keyreplace(T, 1, Acc, {T, C + V})\n                                   end\n                           end, [], Mapped),\n    ?equals(length(Expected),\n            length(Results)),\n    Zipped = lists:zip(lists:sort(Expected),\n                       lists:sort(Results)),\n    _ = [?equals(X, Y) || {X, Y} <- Zipped],\n    ok.\n"
  },
  {
    "path": "test/mr_proto_sched_SUITE.erl",
    "content": "% @copyright 2010-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Fajerski <fajerski@zib.de>\n%% @doc    Unit tests for the map reduce protocol.\n%% @end\n%% @version $Id$\n-module(mr_proto_sched_SUITE).\n-author('fajerski@zib.de').\n-vsn('$Id$').\n\n-compile([export_all]).\n\n%% start proto scheduler for this suite\n-define(proto_sched(Action), proto_sched_fun(Action)).\n-define(proto_sched2(Action, Arg), proto_sched2_fun(Action, Arg)).\n\n-include(\"mr_SUITE.hrl\").\n\nall() ->\ntests_avail() ++ [test_join, test_leave].\n\nsuite() -> [ {timetrap, {seconds, 15}} ].\n\n-spec proto_sched_fun(start | stop) -> ok.\nproto_sched_fun(start) ->\n    proto_sched:thread_begin();\nproto_sched_fun(stop) ->\n    %% is a ring running?\n    case erlang:whereis(pid_groups) =:= undefined\n             orelse pid_groups:find_a(proto_sched) =:= failed of\n        true -> ok;\n        false ->\n            %% then finalize proto_sched run:\n            %% try to call thread_end(): if this\n            %% process was running the proto_sched\n            %% thats fine, otherwise thread_end()\n            %% will raise an exception\n            proto_sched:thread_end()\n    end.\n\n-spec proto_sched2_fun(setup, ThreadNum::pos_integer()) -> ok;\n                      (cleanup, PIDs::[pid() | atom()]) -> ok.\nproto_sched2_fun(setup, Arg) ->\n    proto_sched:thread_num(Arg);\nproto_sched2_fun(cleanup, _Arg) ->\n    proto_sched:wait_for_end(),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed),\n    proto_sched:cleanup().\n\ntest_join(_Config) ->\n    ?proto_sched2(setup, 2),\n    spawn(fun() ->\n                       ?proto_sched(start),\n                       ct:pal(\"starting mr job\"),\n                       Res = api_mr:start_job(get_wc_job_erl()),\n                       ct:pal(\"mr job finished\"),\n                       check_results(Res),\n                       ?proto_sched(stop)\n               end),\n    ct:pal(\"adding node to provoke slide\"),\n    spawn(fun() ->\n                       ?proto_sched(start),\n                       {[_, _], []} = api_vm:add_nodes(2),\n                       ?proto_sched(stop)\n               end),\n    ?proto_sched2(cleanup, []),\n    ct:pal(\"ring fully joined (4)\"),\n    ok.\n\ntest_leave(_Config) ->\n    [_] = api_vm:shutdown_nodes(1),\n    {[AddedNode], _} = api_vm:add_nodes(1),\n    unittest_helper:check_ring_size_fully_joined(2),\n    unittest_helper:wait_for_stable_ring_deep(),\n    ?proto_sched2(setup, 2),\n    spawn_link(fun() ->\n                       ?proto_sched(start),\n                       ct:pal(\"starting mr job\"),\n                       Res = api_mr:start_job(get_wc_job_erl()),\n                       ct:pal(\"mr job finished\"),\n                       check_results(Res),\n                       ?proto_sched(stop)\n               end),\n    ct:pal(\"shutting down node ~p to provoke slide\", [AddedNode]),\n    spawn_link(fun() ->\n                       ?proto_sched(start),\n                       api_vm:shutdown_node(AddedNode),\n                       ?proto_sched(stop)\n               end),\n    ?proto_sched2(cleanup, []),\n    ok.\n"
  },
  {
    "path": "test/node_details_SUITE.erl",
    "content": "%  @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Test suites for the node_details module.\n%% @end\n%% @version $Id$\n-module(node_details_SUITE).\n\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n\ngroups() ->\n    [{tester_tests, [sequence],\n      [tester_new0, tester_new9,\n       tester_set_get_pred,\n       tester_set_get_predlist,\n       tester_set_get_node,\n       tester_set_get_my_range,\n       tester_set_get_succ,\n       tester_set_get_succlist,\n       tester_set_get_load,\n       tester_set_get_hostname,\n       tester_set_get_rt_size,\n       tester_set_get_message_log,\n       tester_set_get_memory]}\n    ].\n\nall() ->\n    [\n     {group, tester_tests}\n    ].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 10}}\n    ].\n\ninit_per_suite(Config) ->\n    unittest_helper:start_minimal_procs(Config, [], true).\n\nend_per_suite(Config) ->\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\n-spec safe_compare(NodeDetails::node_details:node_details(),\n                   Tag::node_details:node_details_name(),\n                   ExpValue::node:node_type() |\n                             nodelist:non_empty_snodelist() |\n                             intervals:interval() |\n                             node_details:load() |\n                             node_details:hostname() |\n                             node_details:rt_size() |\n                             node_details:message_log() |\n                             node_details:memory() |\n                             unknown,\n                   Unknown::[node_details:node_details_name()],\n                   Known::[node_details:node_details_name()]) -> true.\nsafe_compare(NodeDetails, Tag, ExpValue, Unknown, Known) ->\n    ValIsUnknown = (ExpValue =:= unknown) andalso\n                       ((Known =:= [] andalso lists:member(Tag, Unknown)) orelse\n                        (Unknown =:= [] andalso not lists:member(Tag, Known))),\n    case ValIsUnknown of\n        true -> ?equals_w_note(node_details:contains(NodeDetails, Tag), false, atom_to_list(Tag));\n        _    -> ?equals_w_note(node_details:get(NodeDetails, Tag), ExpValue, atom_to_list(Tag))\n    end.\n\n%% @doc Compares NodeDetails with the given values. Either Unknown or Known\n%%      must be non-empty. A value is unknown (and thus not part of the\n%%      NodeDetails object) if its tag is either in Unknown and Known is emty\n%%      or if its tag is not in Known and Unknown is empty.\n-spec node_details_equals(NodeDetails::node_details:node_details(),\n                          Pred::node:node_type() | unknown,\n                          PredList::nodelist:non_empty_snodelist() | unknown,\n                          Node::node:node_type() | unknown,\n                          MyRange::intervals:interval() | unknown,\n                          Succ::node:node_type() | unknown,\n                          SuccList::nodelist:non_empty_snodelist() | unknown,\n                          Load::node_details:load() | unknown,\n                          Load2::node_details:load() | unknown,\n                          Load3::node_details:load() | unknown,\n                          Hostname::node_details:hostname() | unknown,\n                          RTSize::node_details:rt_size() | unknown,\n                          MsgLog::node_details:message_log() | unknown,\n                          Memory::node_details:memory() | unknown,\n                          Unknown::[node_details:node_details_name()],\n                          Known::[node_details:node_details_name()]) -> true.\nnode_details_equals(NodeDetails, Pred, PredList, Node, MyRange, Succ, SuccList, Load, Load2, Load3, Hostname, RTSize, MsgLog, Memory, Unknown, Known) ->\n    safe_compare(NodeDetails, pred, Pred, Unknown, Known),\n    safe_compare(NodeDetails, predlist, PredList, Unknown, Known),\n    safe_compare(NodeDetails, node, Node, Unknown, Known),\n    safe_compare(NodeDetails, my_range, MyRange, Unknown, Known),\n    safe_compare(NodeDetails, succ, Succ, Unknown, Known),\n    safe_compare(NodeDetails, succlist, SuccList, Unknown, Known),\n    safe_compare(NodeDetails, load, Load, Unknown, Known),\n    safe_compare(NodeDetails, load2, Load2, Unknown, Known),\n    safe_compare(NodeDetails, load3, Load3, Unknown, Known),\n    safe_compare(NodeDetails, hostname, Hostname, Unknown, Known),\n    safe_compare(NodeDetails, rt_size, RTSize, Unknown, Known),\n    safe_compare(NodeDetails, message_log, MsgLog, Unknown, Known),\n    safe_compare(NodeDetails, memory, Memory, Unknown, Known).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% node_details:new/0\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec test_new0() -> true.\ntest_new0() ->\n    NodeDetails = node_details:new(),\n    node_details_equals(NodeDetails, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, [], []).\n\ntester_new0(Config) ->\n    tester:test(node_details_SUITE, test_new0, 0, 10, [{threads, 2}]),\n    Config.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% node_details:new/7\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec test_new9(nodelist:non_empty_snodelist(), node:node_type(), nodelist:non_empty_snodelist(), node_details:load(), node_details:load2(), node_details:load3(), node_details:hostname(), node_details:rt_size(), node_details:memory()) -> true.\ntest_new9(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory) ->\n    NodeDetails = node_details:new(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory),\n    node_details_equals(NodeDetails, hd(PredList), PredList, Node, unknown, hd(SuccList), SuccList, Load, Load2, Load3, Hostname, RTSize, unknown, Memory, [my_range, message_log], []).\n\ntester_new9(Config) ->\n    tester:test(node_details_SUITE, test_new9, 9, 10, [{threads, 2}]),\n    Config.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% node_details:set/3 and node_details:get/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec test_set_get_pred(Pred::node:node_type()) -> true.\ntest_set_get_pred(PredTest) ->\n    NodeDetails1 = node_details:new(),\n    Node = node:new(comm:this(), 0, 0), PredList = [Node], SuccList = [Node],\n    Load = 0, Load2 = 0, Load3 = 0, Hostname = \"localhost\", RTSize = 0, Memory = 0,\n    NodeDetails2 = node_details:new(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory),\n    NodeDetails1_new = node_details:set(NodeDetails1, pred, PredTest),\n    NodeDetails2_new = node_details:set(NodeDetails2, pred, PredTest),\n    node_details_equals(NodeDetails1_new, PredTest, [PredTest], unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, [], [pred, predlist]),\n    node_details_equals(NodeDetails2_new, PredTest, [PredTest], Node, unknown, hd(SuccList), SuccList, Load, Load2, Load3, Hostname, RTSize, unknown, Memory, [my_range, message_log], []).\n\n-spec test_set_get_predlist(PredList::nodelist:non_empty_snodelist()) -> true.\ntest_set_get_predlist(PredListTest) ->\n    NodeDetails1 = node_details:new(),\n    Node = node:new(comm:this(), 0, 0), PredList = [Node], SuccList = [Node],\n    Load = 0, Load2 = 0, Load3 = 0, Hostname = \"localhost\", RTSize = 0, Memory = 0,\n    NodeDetails2 = node_details:new(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory),\n    NodeDetails1_new = node_details:set(NodeDetails1, predlist, PredListTest),\n    NodeDetails2_new = node_details:set(NodeDetails2, predlist, PredListTest),\n    node_details_equals(NodeDetails1_new, hd(PredListTest), PredListTest, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, [], [pred, predlist]),\n    node_details_equals(NodeDetails2_new, hd(PredListTest), PredListTest, Node, unknown, hd(SuccList), SuccList, Load, Load2, Load3, Hostname, RTSize, unknown, Memory, [my_range, message_log], []).\n\n-spec test_set_get_node(Node::node:node_type()) -> true.\ntest_set_get_node(NodeTest) ->\n    NodeDetails1 = node_details:new(),\n    Node = node:new(comm:this(), 0, 0), PredList = [Node], SuccList = [Node],\n    Load = 0, Load2 = 0, Load3 = 0, Hostname = \"localhost\", RTSize = 0, Memory = 0,\n    NodeDetails2 = node_details:new(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory),\n    NodeDetails1_new = node_details:set(NodeDetails1, node, NodeTest),\n    NodeDetails2_new = node_details:set(NodeDetails2, node, NodeTest),\n    node_details_equals(NodeDetails1_new, unknown, unknown, NodeTest, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, [], [node]),\n    node_details_equals(NodeDetails2_new, hd(PredList), PredList, NodeTest, unknown, hd(SuccList), SuccList, Load, Load2, Load3, Hostname, RTSize, unknown, Memory, [my_range, message_log], []).\n\n-spec test_set_get_my_range(MyRange::intervals:interval()) -> true.\ntest_set_get_my_range(MyRangeTest) ->\n    NodeDetails1 = node_details:new(),\n    Node = node:new(comm:this(), 0, 0), PredList = [Node], SuccList = [Node],\n    Load = 0, Load2 = 0, Load3 = 0, Hostname = \"localhost\", RTSize = 0, Memory = 0,\n    NodeDetails2 = node_details:new(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory),\n    NodeDetails1_new = node_details:set(NodeDetails1, my_range, MyRangeTest),\n    NodeDetails2_new = node_details:set(NodeDetails2, my_range, MyRangeTest),\n    node_details_equals(NodeDetails1_new, unknown, unknown, unknown, MyRangeTest, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, [], [my_range]),\n    node_details_equals(NodeDetails2_new, hd(PredList), PredList, Node, MyRangeTest, hd(SuccList), SuccList, Load, Load2, Load3, Hostname, RTSize, unknown, Memory, [message_log], []).\n\n-spec test_set_get_succ(Succ::node:node_type()) -> true.\ntest_set_get_succ(SuccTest) ->\n    NodeDetails1 = node_details:new(),\n    Node = node:new(comm:this(), 0, 0), PredList = [Node], SuccList = [Node],\n    Load = 0, Load2 = 0, Load3 = 0, Hostname = \"localhost\", RTSize = 0, Memory = 0,\n    NodeDetails2 = node_details:new(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory),\n    NodeDetails1_new = node_details:set(NodeDetails1, succ, SuccTest),\n    NodeDetails2_new = node_details:set(NodeDetails2, succ, SuccTest),\n    node_details_equals(NodeDetails1_new, unknown, unknown, unknown, unknown, SuccTest, [SuccTest], unknown, unknown, unknown, unknown, unknown, unknown, unknown, [], [succ, succlist]),\n    node_details_equals(NodeDetails2_new, hd(PredList), PredList, Node, unknown, SuccTest, [SuccTest], Load, Load2, Load3, Hostname, RTSize, unknown, Memory, [my_range, message_log], []).\n\n-spec test_set_get_succlist(SuccList::nodelist:non_empty_snodelist()) -> true.\ntest_set_get_succlist(SuccListTest) ->\n    NodeDetails1 = node_details:new(),\n    Node = node:new(comm:this(), 0, 0), PredList = [Node], SuccList = [Node],\n    Load = 0, Load2 = 0, Load3 = 0, Hostname = \"localhost\", RTSize = 0, Memory = 0,\n    NodeDetails2 = node_details:new(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory),\n    NodeDetails1_new = node_details:set(NodeDetails1, succlist, SuccListTest),\n    NodeDetails2_new = node_details:set(NodeDetails2, succlist, SuccListTest),\n    node_details_equals(NodeDetails1_new, unknown, unknown, unknown, unknown, hd(SuccListTest), SuccListTest, unknown, unknown, unknown, unknown, unknown, unknown, unknown, [], [succ, succlist]),\n    node_details_equals(NodeDetails2_new, hd(PredList), PredList, Node, unknown, hd(SuccListTest), SuccListTest, Load, Load2, Load3, Hostname, RTSize, unknown, Memory, [my_range, message_log], []).\n\n-spec test_set_get_load(Load::node_details:load()) -> true.\ntest_set_get_load(LoadTest) ->\n    NodeDetails1 = node_details:new(),\n    Node = node:new(comm:this(), 0, 0), PredList = [Node], SuccList = [Node],\n    Load = 0, Load2 = 0, Load3 = 0, Hostname = \"localhost\", RTSize = 0, Memory = 0,\n    NodeDetails2 = node_details:new(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory),\n    NodeDetails1_new = node_details:set(NodeDetails1, load, LoadTest),\n    NodeDetails2_new = node_details:set(NodeDetails2, load, LoadTest),\n    node_details_equals(NodeDetails1_new, unknown, unknown, unknown, unknown, unknown, unknown, LoadTest, unknown, unknown, unknown, unknown, unknown, unknown, [], [load]),\n    node_details_equals(NodeDetails2_new, hd(PredList), PredList, Node, unknown, hd(SuccList), SuccList, LoadTest, Load2, Load3, Hostname, RTSize, unknown, Memory, [my_range, message_log], []).\n                          \n-spec test_set_get_hostname(Hostname::node_details:hostname()) -> true.\ntest_set_get_hostname(HostnameTest) ->\n    NodeDetails1 = node_details:new(),\n    Node = node:new(comm:this(), 0, 0), PredList = [Node], SuccList = [Node],\n    Load = 0, Load2 = 0, Load3 = 0, Hostname = \"localhost\", RTSize = 0, Memory = 0,\n    NodeDetails2 = node_details:new(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory),\n    NodeDetails1_new = node_details:set(NodeDetails1, hostname, HostnameTest),\n    NodeDetails2_new = node_details:set(NodeDetails2, hostname, HostnameTest),\n    node_details_equals(NodeDetails1_new, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, HostnameTest, unknown, unknown, unknown, [], [hostname]),\n    node_details_equals(NodeDetails2_new, hd(PredList), PredList, Node, unknown, hd(SuccList), SuccList, Load, Load2, Load3, HostnameTest, RTSize, unknown, Memory, [my_range, message_log], []).\n\n-spec test_set_get_rt_size(RTSize::node_details:rt_size()) -> true.\ntest_set_get_rt_size(RTSizeTest) ->\n    NodeDetails1 = node_details:new(),\n    Node = node:new(comm:this(), 0, 0), PredList = [Node], SuccList = [Node],\n    Load = 0, Load2 = 0, Load3 = 0, Hostname = \"localhost\", RTSize = 0, Memory = 0,\n    NodeDetails2 = node_details:new(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory),\n    NodeDetails1_new = node_details:set(NodeDetails1, rt_size, RTSizeTest),\n    NodeDetails2_new = node_details:set(NodeDetails2, rt_size, RTSizeTest),\n    node_details_equals(NodeDetails1_new, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, RTSizeTest, unknown, unknown, [], [rt_size]),\n    node_details_equals(NodeDetails2_new, hd(PredList), PredList, Node, unknown, hd(SuccList), SuccList, Load, Load2, Load3, Hostname, RTSizeTest, unknown, Memory, [my_range, message_log], []).\n\n-spec test_set_get_message_log(MsgLog::node_details:message_log()) -> true.\ntest_set_get_message_log(MsgLogTest) ->\n    NodeDetails1 = node_details:new(),\n    Node = node:new(comm:this(), 0, 0), PredList = [Node], SuccList = [Node],\n    Load = 0, Load2 = 0, Load3 = 0, Hostname = \"localhost\", RTSize = 0, Memory = 0,\n    NodeDetails2 = node_details:new(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory),\n    NodeDetails1_new = node_details:set(NodeDetails1, message_log, MsgLogTest),\n    NodeDetails2_new = node_details:set(NodeDetails2, message_log, MsgLogTest),\n    node_details_equals(NodeDetails1_new, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, MsgLogTest, unknown, [], [message_log]),\n    node_details_equals(NodeDetails2_new, hd(PredList), PredList, Node, unknown, hd(SuccList), SuccList, Load, Load2, Load3, Hostname, RTSize, MsgLogTest, Memory, [my_range], []).\n\n-spec test_set_get_memory(Memory::node_details:memory()) -> true.\ntest_set_get_memory(MemoryTest) ->\n    NodeDetails1 = node_details:new(),\n    Node = node:new(comm:this(), 0, 0), PredList = [Node], SuccList = [Node],\n    Load = 0, Load2 = 0, Load3 = 0, Hostname = \"localhost\", RTSize = 0, Memory = 0,\n    NodeDetails2 = node_details:new(PredList, Node, SuccList, Load, Load2, Load3, Hostname, RTSize, Memory),\n    NodeDetails1_new = node_details:set(NodeDetails1, memory, MemoryTest),\n    NodeDetails2_new = node_details:set(NodeDetails2, memory, MemoryTest),\n    node_details_equals(NodeDetails1_new, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, unknown, MemoryTest, [], [memory]),\n    node_details_equals(NodeDetails2_new, hd(PredList), PredList, Node, unknown, hd(SuccList), SuccList, Load, Load2, Load3, Hostname, RTSize, unknown, MemoryTest, [my_range, message_log], []).\n\ntester_set_get_pred(Config) ->\n    tester:test(node_details_SUITE, test_set_get_pred, 1, 1000, [{threads, 2}]),\n    Config.\n\ntester_set_get_predlist(Config) ->\n    tester:test(node_details_SUITE, test_set_get_predlist, 1, 1000, [{threads, 2}]),\n    Config.\n\ntester_set_get_node(Config) ->\n    tester:test(node_details_SUITE, test_set_get_node, 1, 1000, [{threads, 2}]),\n    Config.\n\ntester_set_get_my_range(Config) ->\n    tester:test(node_details_SUITE, test_set_get_my_range, 1, 1000, [{threads, 2}]),\n    Config.\n\ntester_set_get_succ(Config) ->\n    tester:test(node_details_SUITE, test_set_get_succ, 1, 1000, [{threads, 2}]),\n    Config.\n\ntester_set_get_succlist(Config) ->\n    tester:test(node_details_SUITE, test_set_get_succlist, 1, 1000, [{threads, 2}]),\n    Config.\n\ntester_set_get_load(Config) ->\n    tester:test(node_details_SUITE, test_set_get_load, 1, 1000, [{threads, 2}]),\n    Config.\n\ntester_set_get_hostname(Config) ->\n    tester:test(node_details_SUITE, test_set_get_hostname, 1, 1000, [{threads, 2}]),\n    Config.\n\ntester_set_get_rt_size(Config) ->\n    tester:test(node_details_SUITE, test_set_get_rt_size, 1, 1000, [{threads, 2}]),\n    Config.\n\ntester_set_get_message_log(Config) ->\n    tester:test(node_details_SUITE, test_set_get_message_log, 1, 1000, [{threads, 2}]),\n    Config.\n\ntester_set_get_memory(Config) ->\n    tester:test(node_details_SUITE, test_set_get_memory, 1, 1000, [{threads, 2}]),\n    Config.\n"
  },
  {
    "path": "test/nodelist_SUITE.erl",
    "content": "%  @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Unit tests for src/nodelist.erl\n%% @end\n%% @version $Id$\n-module(nodelist_SUITE).\n\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\n-dialyzer({[no_opaque, no_return], [tester_largest_smaller_than2/1,\n                                    tester_largest_smaller_than3/1]}).\n\nall() ->\n    [test_new, test_mk2, test_mk4, test_trunc,\n     test_remove2, test_remove3, test_filter2, test_filter3,\n     test_filter_min_length4, test_merge, test_add_node, test_add_nodes,\n     test_update_ids, test_to_list, test_lupdate_ids, test_lremove_outdated,\n     tester_largest_smaller_than2,\n     tester_largest_smaller_than3,\n     test_succ_ord_id].\n\nsuite() -> [ {timetrap, {seconds, 40}} ].\n\ninit_per_suite(Config) ->\n    unittest_helper:start_minimal_procs(Config, [], true).\n\nend_per_suite(Config) ->\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\n% TODO: add tests for simple node lists (the following tests only test functions with neighborhood structures)\n\n-define(PID(Nr), comm:make_global(list_to_pid(\"<0.0.\" ++ erlang:integer_to_list(Nr) ++ \">\"))).\n-define(KEY(Nr), rt_SUITE:number_to_key(Nr)).\n\n% use macro instead of function so that the output in case of errors is better:\n-define(compare_neighborhood(Neighborhood, Node, Pred, Preds, Succ, Succs, RealPred, RealSucc),\n        compare_neighborhood(??Neighborhood, Neighborhood, Node, Pred, Preds, Succ, Succs, RealPred, RealSucc)).\n\n-spec compare_neighborhood(NeighborhoodStr::string(), N::nodelist:neighborhood(), Node::node:node_type(), Pred::node:node_type(), Preds::nodelist:snodelist(), Succ::node:node_type(), Succs::nodelist:snodelist(), RealPred::boolean(), RealSucc::boolean()) -> ok.\ncompare_neighborhood(NeighborhoodStr, N, Node, Pred, Preds, Succ, Succs, RealPred, RealSucc) ->\n    ?equals_w_note(nodelist:node(N), Node, io_lib:format(\"N = ~s\", [NeighborhoodStr])),\n    ?equals_w_note(nodelist:nodeid(N), node:id(Node), io_lib:format(\"N = ~s\", [NeighborhoodStr])),\n    ?equals_w_note(nodelist:pred(N), Pred, io_lib:format(\"N = ~s\", [NeighborhoodStr])),\n    ?equals_w_note(nodelist:preds(N), Preds, io_lib:format(\"N = ~s\", [NeighborhoodStr])),\n    ?equals_w_note(nodelist:succ(N), Succ, io_lib:format(\"N = ~s\", [NeighborhoodStr])),\n    ?equals_w_note(nodelist:succs(N), Succs, io_lib:format(\"N = ~s\", [NeighborhoodStr])),\n    ?equals_w_note(nodelist:has_real_pred(N), RealPred, io_lib:format(\"N = ~s\", [NeighborhoodStr])),\n    ?equals_w_note(nodelist:has_real_succ(N), RealSucc, io_lib:format(\"N = ~s\", [NeighborhoodStr])),\n    ok.\n\n%% @doc Tests neighborhood creation with new_neighborhood and compares the\n%%      getters with what is expected.\n-spec test_new(any()) -> ok.\ntest_new(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N15 = node:new(?PID(2), ?KEY(15), 1), % N14 updated its ID to 15 here\n    N33 = node:new(?PID(3), ?KEY(33), 0),\n    \n    Neighb11 = nodelist:new_neighborhood(N8),\n    ?compare_neighborhood(Neighb11, N8, N8, [N8], N8, [N8], false, false),\n    \n    Neighb21 = nodelist:new_neighborhood(N8, N14),\n    Neighb22 = nodelist:new_neighborhood(N14, N8),\n    Neighb23 = nodelist:new_neighborhood(N8, N8),\n    Neighb24 = nodelist:new_neighborhood(N15, N14),\n    ?compare_neighborhood(Neighb21, N8, N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(Neighb22, N14, N8, [N8], N8, [N8], true, true),\n    ?compare_neighborhood(Neighb23, N8, N8, [N8], N8, [N8], false, false),\n    ?compare_neighborhood(Neighb24, N15, N15, [N15], N15, [N15], false, false),\n    \n    Neighb31 = nodelist:new_neighborhood(N14, N8,  N33),\n    Neighb32 = nodelist:new_neighborhood(N33, N8,  N14),\n    Neighb33 = nodelist:new_neighborhood(N8,  N14, N33),\n    Neighb34 = nodelist:new_neighborhood(N33, N14, N8),\n    Neighb35 = nodelist:new_neighborhood(N14, N33, N8),\n    Neighb36 = nodelist:new_neighborhood(N8,  N33, N14),\n    Neighb37 = nodelist:new_neighborhood(N14, N8,  N8),\n    Neighb38 = nodelist:new_neighborhood(N8, N8,  N8),\n    Neighb39 = nodelist:new_neighborhood(N14, N8,  N15),\n    Neighb310 = nodelist:new_neighborhood(N15, N8,  N14),\n    Neighb311 = nodelist:new_neighborhood(N8, N15,  N14),\n    Neighb312 = nodelist:new_neighborhood(N14, N15,  N8),\n    ?compare_neighborhood(Neighb31,  N8,  N33, [N33], N14, [N14], true, true),\n    ?compare_neighborhood(Neighb32,  N8,  N33, [N33], N14, [N14], true, true),\n    ?compare_neighborhood(Neighb33,  N14, N8,  [N8],  N33, [N33], true, true),\n    ?compare_neighborhood(Neighb34,  N14, N8,  [N8],  N33, [N33], true, true),\n    ?compare_neighborhood(Neighb35,  N33, N14, [N14], N8,  [N8],  true, true),\n    ?compare_neighborhood(Neighb36,  N33, N14, [N14], N8,  [N8],  true, true),\n    ?compare_neighborhood(Neighb37,  N8,  N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(Neighb38,  N8,  N8,  [N8],  N8,  [N8],  false, false),\n    ?compare_neighborhood(Neighb39,  N8,  N15, [N15], N15, [N15], true, true),\n    ?compare_neighborhood(Neighb310, N8,  N15, [N15], N15, [N15], true, true),\n    ?compare_neighborhood(Neighb311, N15, N8,  [N8],  N8,  [N8],  true, true),\n    ?compare_neighborhood(Neighb312, N15, N8,  [N8],  N8,  [N8],  true, true),\n    \n    % tests with pred/succ nodes equal to the base node but newer:\n    ?expect_exception(nodelist:new_neighborhood(N14, N15),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:new_neighborhood(N15, N14, N8),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:new_neighborhood(N8, N14, N15),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ok.\n\n%% @doc Tests neighborhood creation with mk_neighborhood/2 and compares the\n%%      getters with what is expected.\n-spec test_mk2(any()) -> ok.\ntest_mk2(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N15 = node:new(?PID(2), ?KEY(15), 1), % N14 updated its ID to 15 here\n    N33 = node:new(?PID(3), ?KEY(33), 0),\n    N42 = node:new(?PID(5), ?KEY(42), 0),\n    N50 = node:new(?PID(6), ?KEY(50), 0),\n    N66 = node:new(?PID(7), ?KEY(66), 0),\n    \n    Neighb11 = nodelist:mk_neighborhood([N8], N8),\n    Neighb12 = nodelist:mk_neighborhood([], N8),\n    ?compare_neighborhood(Neighb11, N8, N8, [N8], N8, [N8], false, false),\n    ?compare_neighborhood(Neighb12, N8, N8, [N8], N8, [N8], false, false),\n    \n    Neighb21 = nodelist:mk_neighborhood([N8, N14], N8),\n    Neighb22 = nodelist:mk_neighborhood([N14, N8], N14),\n    Neighb23 = nodelist:mk_neighborhood([N8, N8], N8),\n    Neighb24 = nodelist:mk_neighborhood([N14], N8),\n    Neighb25 = nodelist:mk_neighborhood([N8], N14),\n    Neighb26 = nodelist:mk_neighborhood([N14], N15),\n    ?compare_neighborhood(Neighb21, N8,  N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(Neighb22, N14, N8,  [N8],  N8,  [N8],  true, true),\n    ?compare_neighborhood(Neighb23, N8,  N8,  [N8],  N8,  [N8],  false, false),\n    ?compare_neighborhood(Neighb24, N8,  N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(Neighb25, N14, N8,  [N8],  N8,  [N8],  true, true),\n    ?compare_neighborhood(Neighb26, N15, N15, [N15], N15, [N15], false, false),\n    \n    Neighb31 = nodelist:mk_neighborhood( [N14, N8,  N33], N8),\n    Neighb32 = nodelist:mk_neighborhood( [N33, N8,  N14], N8),\n    Neighb33 = nodelist:mk_neighborhood( [N8,  N14, N33], N14),\n    Neighb34 = nodelist:mk_neighborhood( [N33, N14, N8],  N14),\n    Neighb35 = nodelist:mk_neighborhood( [N14, N33, N8],  N33),\n    Neighb36 = nodelist:mk_neighborhood( [N8,  N33, N14], N33),\n    Neighb37 = nodelist:mk_neighborhood( [N14, N8,  N8],  N8),\n    Neighb38 = nodelist:mk_neighborhood( [N8, N8,  N8],   N8),\n    Neighb39 = nodelist:mk_neighborhood( [N14, N15],      N8),\n    Neighb310 = nodelist:mk_neighborhood([N15, N14],      N8),\n    Neighb311 = nodelist:mk_neighborhood([N14, N8],       N15),\n    Neighb312 = nodelist:mk_neighborhood([N8, N14],       N15),\n    ?compare_neighborhood(Neighb31,  N8,  N33, [N33, N14], N14, [N14, N33], true, true),\n    ?compare_neighborhood(Neighb32,  N8,  N33, [N33, N14], N14, [N14, N33], true, true),\n    ?compare_neighborhood(Neighb33,  N14, N8,  [N8, N33],  N33, [N33, N8],  true, true),\n    ?compare_neighborhood(Neighb34,  N14, N8,  [N8, N33],  N33, [N33, N8],  true, true),\n    ?compare_neighborhood(Neighb35,  N33, N14, [N14, N8],  N8,  [N8, N14],  true, true),\n    ?compare_neighborhood(Neighb36,  N33, N14, [N14, N8],  N8,  [N8, N14],  true, true),\n    ?compare_neighborhood(Neighb37,  N8,  N14, [N14],      N14, [N14],      true, true),\n    ?compare_neighborhood(Neighb38,  N8,  N8,  [N8],       N8,  [N8],       false, false),\n    ?compare_neighborhood(Neighb39,  N8,  N15, [N15],      N15, [N15],      true, true),\n    ?compare_neighborhood(Neighb310, N8,  N15, [N15],      N15, [N15],      true, true),\n    ?compare_neighborhood(Neighb311, N15, N8,  [N8],       N8,  [N8],       true, true),\n    ?compare_neighborhood(Neighb312, N15, N8,  [N8],       N8,  [N8],       true, true),\n    \n    % some tests with duplicated nodes:\n    Neighb41 = nodelist:mk_neighborhood( [N14, N8, N8,  N33], N8),\n    Neighb42 = nodelist:mk_neighborhood( [N33, N8,  N14, N14], N8),\n    Neighb43 = nodelist:mk_neighborhood( [N8,  N14, N33, N14], N14),\n    Neighb44 = nodelist:mk_neighborhood( [N33, N14, N8, N33, N14],  N14),\n    Neighb45 = nodelist:mk_neighborhood( [N14, N33, N8, N14, N8],  N33),\n    ?compare_neighborhood(Neighb41,  N8,  N33, [N33, N14], N14, [N14, N33], true, true),\n    ?compare_neighborhood(Neighb42,  N8,  N33, [N33, N14], N14, [N14, N33], true, true),\n    ?compare_neighborhood(Neighb43,  N14, N8,  [N8, N33],  N33, [N33, N8],  true, true),\n    ?compare_neighborhood(Neighb44,  N14, N8,  [N8, N33],  N33, [N33, N8],  true, true),\n    ?compare_neighborhood(Neighb45,  N33, N14, [N14, N8],  N8,  [N8, N14],  true, true),\n    \n    % tests with nodes equal to the base node in the node list:\n    ?compare_neighborhood(nodelist:mk_neighborhood([N8, N33, N14], N15),\n                          N15, N8, [N8, N33], N33, [N33, N8], true, true),\n    ?expect_exception(nodelist:mk_neighborhood([N8, N33, N15], N14),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:mk_neighborhood([N8, N15, N33], N14),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:mk_neighborhood([N15, N8, N33], N14),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:mk_neighborhood([N15], N14),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    \n    % a few tests with longer lists\n    Neighb51 = nodelist:mk_neighborhood([N66, N8, N50, N33, N42], N14),\n    Neighb52 = nodelist:mk_neighborhood([N66, N8, N50, N14, N33, N42], N14),\n    Neighb53 = nodelist:mk_neighborhood([N66, N8, N50, N14, N33, N42], N15),\n    ?compare_neighborhood(Neighb51, N14, N8, [N8, N66, N50, N42, N33], N33, [N33, N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(Neighb52, N14, N8, [N8, N66, N50, N42, N33], N33, [N33, N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(Neighb53, N15, N8, [N8, N66, N50, N42, N33], N33, [N33, N42, N50, N66, N8], true, true),\n    ok.\n\n%% @doc Tests neighborhood creation with mk_neighborhood/4 and compares the\n%%      getters with what is expected.\n-spec test_mk4(any()) -> ok.\ntest_mk4(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N15 = node:new(?PID(2), ?KEY(15), 1), % N14 updated its ID to 15 here\n    N33 = node:new(?PID(3), ?KEY(33), 0),\n    N42 = node:new(?PID(5), ?KEY(42), 0),\n    N50 = node:new(?PID(6), ?KEY(50), 0),\n    N66 = node:new(?PID(7), ?KEY(66), 0),\n    \n    Neighb11 = nodelist:mk_neighborhood([N8], N8, 1, 1),\n    Neighb12 = nodelist:mk_neighborhood([], N8, 1, 1),\n    ?compare_neighborhood(Neighb11, N8, N8, [N8], N8, [N8], false, false),\n    ?compare_neighborhood(Neighb12, N8, N8, [N8], N8, [N8], false, false),\n    \n    Neighb21 = nodelist:mk_neighborhood([N8, N14], N8, 1, 1),\n    Neighb22 = nodelist:mk_neighborhood([N14, N8], N14, 1, 1),\n    Neighb23 = nodelist:mk_neighborhood([N8, N8], N8, 1, 1),\n    Neighb24 = nodelist:mk_neighborhood([N14], N8, 1, 1),\n    Neighb25 = nodelist:mk_neighborhood([N8], N14, 1, 1),\n    Neighb26 = nodelist:mk_neighborhood([N14], N15, 1, 1),\n    ?compare_neighborhood(Neighb21, N8,  N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(Neighb22, N14, N8,  [N8],  N8,  [N8],  true, true),\n    ?compare_neighborhood(Neighb23, N8,  N8,  [N8],  N8,  [N8],  false, false),\n    ?compare_neighborhood(Neighb24, N8,  N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(Neighb25, N14, N8,  [N8],  N8,  [N8],  true, true),\n    ?compare_neighborhood(Neighb26, N15, N15, [N15], N15, [N15], false, false),\n    \n    Neighb31 = nodelist:mk_neighborhood( [N14, N8,  N33], N8, 1, 1),\n    Neighb32 = nodelist:mk_neighborhood( [N33, N8,  N14], N8, 1, 1),\n    Neighb33 = nodelist:mk_neighborhood( [N8,  N14, N33], N14, 1, 1),\n    Neighb34 = nodelist:mk_neighborhood( [N33, N14, N8],  N14, 1, 1),\n    Neighb35 = nodelist:mk_neighborhood( [N14, N33, N8],  N33, 1, 1),\n    Neighb36 = nodelist:mk_neighborhood( [N8,  N33, N14], N33, 1, 1),\n    Neighb37 = nodelist:mk_neighborhood( [N14, N8,  N8],  N8, 1, 1),\n    Neighb38 = nodelist:mk_neighborhood( [N8, N8,  N8],   N8, 1, 1),\n    Neighb39 = nodelist:mk_neighborhood( [N14, N15],      N8, 1, 1),\n    Neighb310 = nodelist:mk_neighborhood([N15, N14],      N8, 1, 1),\n    Neighb311 = nodelist:mk_neighborhood([N14, N8],       N15, 1, 1),\n    Neighb312 = nodelist:mk_neighborhood([N8, N14],       N15, 1, 1),\n    ?compare_neighborhood(Neighb31,  N8,  N33, [N33], N14, [N14], true, true),\n    ?compare_neighborhood(Neighb32,  N8,  N33, [N33], N14, [N14], true, true),\n    ?compare_neighborhood(Neighb33,  N14, N8,  [N8],  N33, [N33], true, true),\n    ?compare_neighborhood(Neighb34,  N14, N8,  [N8],  N33, [N33], true, true),\n    ?compare_neighborhood(Neighb35,  N33, N14, [N14], N8,  [N8],  true, true),\n    ?compare_neighborhood(Neighb36,  N33, N14, [N14], N8,  [N8],  true, true),\n    ?compare_neighborhood(Neighb37,  N8,  N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(Neighb38,  N8,  N8,  [N8],  N8,  [N8],  false, false),\n    ?compare_neighborhood(Neighb39,  N8,  N15, [N15], N15, [N15], true, true),\n    ?compare_neighborhood(Neighb310, N8,  N15, [N15], N15, [N15], true, true),\n    ?compare_neighborhood(Neighb311, N15, N8,  [N8],  N8,  [N8],  true, true),\n    ?compare_neighborhood(Neighb312, N15, N8,  [N8],  N8,  [N8],  true, true),\n    \n    % some tests with duplicated nodes:\n    Neighb41 = nodelist:mk_neighborhood( [N14, N8, N8,  N33], N8, 1, 1),\n    Neighb42 = nodelist:mk_neighborhood( [N33, N8,  N14, N14], N8, 1, 1),\n    Neighb43 = nodelist:mk_neighborhood( [N8,  N14, N33, N14], N14, 1, 1),\n    Neighb44 = nodelist:mk_neighborhood( [N33, N14, N8, N33, N14],  N14, 1, 1),\n    Neighb45 = nodelist:mk_neighborhood( [N14, N33, N8, N14, N8],  N33, 1, 1),\n    ?compare_neighborhood(Neighb41,  N8,  N33, [N33], N14, [N14], true, true),\n    ?compare_neighborhood(Neighb42,  N8,  N33, [N33], N14, [N14], true, true),\n    ?compare_neighborhood(Neighb43,  N14, N8,  [N8],  N33, [N33], true, true),\n    ?compare_neighborhood(Neighb44,  N14, N8,  [N8],  N33, [N33], true, true),\n    ?compare_neighborhood(Neighb45,  N33, N14, [N14], N8,  [N8],  true, true),\n    \n    % tests with nodes equal to the base node in the node list:\n    ?compare_neighborhood(nodelist:mk_neighborhood([N8, N33, N14], N15, 3, 3),\n                          N15, N8, [N8, N33], N33, [N33, N8], true, true),\n    ?expect_exception(nodelist:mk_neighborhood([N8, N33, N15], N14, 3, 3),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:mk_neighborhood([N8, N15, N33], N14, 3, 3),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:mk_neighborhood([N15, N8, N33], N14, 3, 3),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:mk_neighborhood([N15], N14, 3, 3),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n\n    ?compare_neighborhood(nodelist:mk_neighborhood([N8, N33, N14], N15, 1, 1),\n                          N15, N8, [N8], N33, [N33], true, true),\n    ?expect_exception(nodelist:mk_neighborhood([N8, N33, N15], N14, 1, 1),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:mk_neighborhood([N8, N15, N33], N14, 1, 1),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:mk_neighborhood([N15, N8, N33], N14, 1, 1),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:mk_neighborhood([N15], N14, 1, 1),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n\n    % a few tests with longer lists\n    Neighb51 = nodelist:mk_neighborhood([N66, N8, N50, N33, N42], N14, 3, 3),\n    Neighb52 = nodelist:mk_neighborhood([N66, N8, N50, N14, N33, N42], N14, 3, 3),\n    Neighb53 = nodelist:mk_neighborhood([N66, N8, N50, N14, N33, N42], N15, 3, 3),\n    Neighb54 = nodelist:mk_neighborhood([N66, N8, N50, N33, N42], N14, 1, 1),\n    Neighb55 = nodelist:mk_neighborhood([N66, N8, N50, N14, N33, N42], N14, 1, 1),\n    Neighb56 = nodelist:mk_neighborhood([N66, N8, N50, N14, N33, N42], N15, 1, 1),\n    ?compare_neighborhood(Neighb51, N14, N8, [N8, N66, N50], N33, [N33, N42, N50], true, true),\n    ?compare_neighborhood(Neighb52, N14, N8, [N8, N66, N50], N33, [N33, N42, N50], true, true),\n    ?compare_neighborhood(Neighb53, N15, N8, [N8, N66, N50], N33, [N33, N42, N50], true, true),\n    ?compare_neighborhood(Neighb54, N14, N8, [N8], N33, [N33], true, true),\n    ?compare_neighborhood(Neighb55, N14, N8, [N8], N33, [N33], true, true),\n    ?compare_neighborhood(Neighb56, N15, N8, [N8], N33, [N33], true, true),\n    ok.\n\n%% @doc Tests truncating neighborhood structures.\n-spec test_trunc(any()) -> ok.\ntest_trunc(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N20 = node:new(?PID(3), ?KEY(20), 0),\n    N33 = node:new(?PID(4), ?KEY(33), 0),\n    N50 = node:new(?PID(5), ?KEY(50), 0),\n    \n    Neighb1 = nodelist:mk_neighborhood([N8, N14, N33, N50], N20),\n    \n    Trunc11 = nodelist:trunc(Neighb1, 1, 1),\n    Trunc12 = nodelist:trunc(Neighb1, 2, 1),\n    Trunc13 = nodelist:trunc(Neighb1, 3, 1),\n    Trunc14 = nodelist:trunc(Neighb1, 4, 1),\n    Trunc15 = nodelist:trunc(Neighb1, 5, 1),\n    Trunc16 = nodelist:trunc(Neighb1, 1, 2),\n    Trunc17 = nodelist:trunc(Neighb1, 1, 3),\n    Trunc18 = nodelist:trunc(Neighb1, 1, 4),\n    Trunc19 = nodelist:trunc(Neighb1, 1, 5),\n    Trunc110 = nodelist:trunc(Neighb1, 2, 2),\n    Trunc111 = nodelist:trunc(Neighb1, 2, 3),\n    Trunc112 = nodelist:trunc(Neighb1, 3, 2),\n    Trunc113 = nodelist:trunc(Neighb1, 3, 3),\n    Trunc114 = nodelist:trunc(Neighb1, 4, 4),\n    Trunc115 = nodelist:trunc(Neighb1, 5, 5),\n    ?compare_neighborhood(Trunc11,  N20, N14, [N14], N33, [N33], true, true),\n    ?compare_neighborhood(Trunc12,  N20, N14, [N14, N8], N33, [N33], true, true),\n    ?compare_neighborhood(Trunc13,  N20, N14, [N14, N8, N50], N33, [N33], true, true),\n    ?compare_neighborhood(Trunc14,  N20, N14, [N14, N8, N50, N33], N33, [N33], true, true),\n    ?compare_neighborhood(Trunc15,  N20, N14, [N14, N8, N50, N33], N33, [N33], true, true),\n    ?compare_neighborhood(Trunc16,  N20, N14, [N14], N33, [N33, N50], true, true),\n    ?compare_neighborhood(Trunc17,  N20, N14, [N14], N33, [N33, N50, N8], true, true),\n    ?compare_neighborhood(Trunc18,  N20, N14, [N14], N33, [N33, N50, N8, N14], true, true),\n    ?compare_neighborhood(Trunc19,  N20, N14, [N14], N33, [N33, N50, N8, N14], true, true),\n    ?compare_neighborhood(Trunc110, N20, N14, [N14, N8], N33, [N33, N50], true, true),\n    ?compare_neighborhood(Trunc111, N20, N14, [N14, N8], N33, [N33, N50, N8], true, true),\n    ?compare_neighborhood(Trunc112, N20, N14, [N14, N8, N50], N33, [N33, N50], true, true),\n    ?compare_neighborhood(Trunc113, N20, N14, [N14, N8, N50], N33, [N33, N50, N8], true, true),\n    ?compare_neighborhood(Trunc114, N20, N14, [N14, N8, N50, N33], N33, [N33, N50, N8, N14], true, true),\n    ?compare_neighborhood(Trunc115, N20, N14, [N14, N8, N50, N33], N33, [N33, N50, N8, N14], true, true),\n    \n    Trunc21 = nodelist:trunc_preds(Neighb1, 1),\n    Trunc22 = nodelist:trunc_preds(Neighb1, 2),\n    Trunc23 = nodelist:trunc_preds(Neighb1, 3),\n    Trunc24 = nodelist:trunc_preds(Neighb1, 4),\n    Trunc25 = nodelist:trunc_preds(Neighb1, 5),\n    ?compare_neighborhood(Trunc21, N20, N14, [N14], N33, [N33, N50, N8, N14], true, true),\n    ?compare_neighborhood(Trunc22, N20, N14, [N14, N8], N33, [N33, N50, N8, N14], true, true),\n    ?compare_neighborhood(Trunc23, N20, N14, [N14, N8, N50], N33, [N33, N50, N8, N14], true, true),\n    ?compare_neighborhood(Trunc24, N20, N14, [N14, N8, N50, N33], N33, [N33, N50, N8, N14], true, true),\n    ?compare_neighborhood(Trunc25, N20, N14, [N14, N8, N50, N33], N33, [N33, N50, N8, N14], true, true),\n    \n    Trunc31 = nodelist:trunc_succs(Neighb1, 1),\n    Trunc32 = nodelist:trunc_succs(Neighb1, 2),\n    Trunc33 = nodelist:trunc_succs(Neighb1, 3),\n    Trunc34 = nodelist:trunc_succs(Neighb1, 4),\n    Trunc35 = nodelist:trunc_succs(Neighb1, 5),\n    ?compare_neighborhood(Trunc31, N20, N14, [N14, N8, N50, N33], N33, [N33], true, true),\n    ?compare_neighborhood(Trunc32, N20, N14, [N14, N8, N50, N33], N33, [N33, N50], true, true),\n    ?compare_neighborhood(Trunc33, N20, N14, [N14, N8, N50, N33], N33, [N33, N50, N8], true, true),\n    ?compare_neighborhood(Trunc34, N20, N14, [N14, N8, N50, N33], N33, [N33, N50, N8, N14], true, true),\n    ?compare_neighborhood(Trunc35, N20, N14, [N14, N8, N50, N33], N33, [N33, N50, N8, N14], true, true),\n    \n    ok.\n\n%% @doc Tests for nodelist:remove/2.\n-spec test_remove2(any()) -> ok.\ntest_remove2(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N9 = node:new(?PID(1),   ?KEY(9), 1), % N8 updated\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N15 = node:new(?PID(2), ?KEY(15), 1), % N14 updated\n    N20 = node:new(?PID(3), ?KEY(20), 0),\n    N33 = node:new(?PID(4), ?KEY(33), 0),\n    N42 = node:new(?PID(5), ?KEY(42), 0),\n    N50 = node:new(?PID(6), ?KEY(50), 0),\n    N66 = node:new(?PID(7), ?KEY(66), 0),\n    \n    Neighb1 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33),\n    Neighb2 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 3, 3),\n    Neighb3 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 2, 2),\n    Neighb4 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 1, 1),\n    Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n\n    % Neighb1 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:remove(N8, Neighb1),\n                          N33, N14, [N14, N50, N42], N42, [N42, N50, N14], true, true),\n    ?compare_neighborhood(nodelist:remove(N14, Neighb1),\n                          N33, N8, [N8, N50, N42], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:remove(N42, Neighb1),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:remove(N50, Neighb1),\n                          N33, N14, [N14, N8, N42], N42, [N42, N8, N14], true, true),\n    % same with the node's pids:\n    ?compare_neighborhood(nodelist:remove(?PID(1), Neighb1),\n                          N33, N14, [N14, N50, N42], N42, [N42, N50, N14], true, true),\n    ?compare_neighborhood(nodelist:remove(?PID(2), Neighb1),\n                          N33, N8, [N8, N50, N42], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:remove(?PID(5), Neighb1),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:remove(?PID(6), Neighb1),\n                          N33, N14, [N14, N8, N42], N42, [N42, N8, N14], true, true),\n    % same with an updated node:\n    ?compare_neighborhood(nodelist:remove(N9, Neighb1),\n                          N33, N14, [N14, N50, N42], N42, [N42, N50, N14], true, true),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:remove(N20, Neighb1), Neighb1),\n    ?equals(nodelist:remove(N66, Neighb1), Neighb1),\n    ?equals(nodelist:remove(?PID(3), Neighb1), Neighb1),\n    ?equals(nodelist:remove(?PID(7), Neighb1), Neighb1),\n\n    % Neighb2 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 3, 3),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:remove(N8, Neighb2),\n                          N33, N14, [N14, N50], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:remove(N14, Neighb2),\n                          N33, N8, [N8, N50], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:remove(N42, Neighb2),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8], true, true),\n    ?compare_neighborhood(nodelist:remove(N50, Neighb2),\n                          N33, N14, [N14, N8], N42, [N42, N8], true, true),\n    % same with the node's pids:\n    ?compare_neighborhood(nodelist:remove(?PID(1), Neighb2),\n                          N33, N14, [N14, N50], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:remove(?PID(2), Neighb2),\n                          N33, N8, [N8, N50], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:remove(?PID(5), Neighb2),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8], true, true),\n    ?compare_neighborhood(nodelist:remove(?PID(6), Neighb2),\n                          N33, N14, [N14, N8], N42, [N42, N8], true, true),\n    % same with an updated node:\n    ?compare_neighborhood(nodelist:remove(N9, Neighb2),\n                          N33, N14, [N14, N50], N42, [N42, N50], true, true),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:remove(N20, Neighb2), Neighb2),\n    ?equals(nodelist:remove(N66, Neighb2), Neighb2),\n    ?equals(nodelist:remove(?PID(3), Neighb2), Neighb2),\n    ?equals(nodelist:remove(?PID(7), Neighb2), Neighb2),\n\n    % Neighb3 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 2, 2),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:remove(N8, Neighb3),\n                          N33, N14, [N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:remove(N14, Neighb3),\n                          N33, N8, [N8], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:remove(N42, Neighb3),\n                          N33, N14, [N14, N8], N50, [N50], true, true),\n    ?compare_neighborhood(nodelist:remove(N50, Neighb3),\n                          N33, N14, [N14, N8], N42, [N42], true, true),\n    % same with the node's pids:\n    ?compare_neighborhood(nodelist:remove(?PID(1), Neighb3),\n                          N33, N14, [N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:remove(?PID(2), Neighb3),\n                          N33, N8, [N8], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:remove(?PID(5), Neighb3),\n                          N33, N14, [N14, N8], N50, [N50], true, true),\n    ?compare_neighborhood(nodelist:remove(?PID(6), Neighb3),\n                          N33, N14, [N14, N8], N42, [N42], true, true),\n    % same with an updated node:\n    ?compare_neighborhood(nodelist:remove(N9, Neighb3),\n                          N33, N14, [N14], N42, [N42, N50], true, true),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:remove(N20, Neighb3), Neighb3),\n    ?equals(nodelist:remove(N66, Neighb3), Neighb3),\n    ?equals(nodelist:remove(?PID(3), Neighb3), Neighb3),\n    ?equals(nodelist:remove(?PID(7), Neighb3), Neighb3),\n\n    % Neighb4 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 1, 1),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:remove(N8, Neighb4),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:remove(N14, Neighb4),\n                          N33, N42, [N42], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:remove(N42, Neighb4),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(nodelist:remove(N50, Neighb4),\n                          N33, N14, [N14], N42, [N42], true, true),\n    % same with the node's pids:\n    ?compare_neighborhood(nodelist:remove(?PID(1), Neighb4),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:remove(?PID(2), Neighb4),\n                          N33, N42, [N42], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:remove(?PID(5), Neighb4),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(nodelist:remove(?PID(6), Neighb4),\n                          N33, N14, [N14], N42, [N42], true, true),\n    % same with an updated node:\n    ?compare_neighborhood(nodelist:remove(N9, Neighb4),\n                          N33, N14, [N14], N42, [N42], true, true),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:remove(N20, Neighb4), Neighb4),\n    ?equals(nodelist:remove(N66, Neighb4), Neighb4),\n    ?equals(nodelist:remove(?PID(3), Neighb4), Neighb4),\n    ?equals(nodelist:remove(?PID(7), Neighb4), Neighb4),\n\n    % Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:remove(N14, Neighb5),\n                          N33, N33, [N33], N33, [N33], false, false),\n    % same with the node's pids:\n    ?compare_neighborhood(nodelist:remove(?PID(2), Neighb5),\n                          N33, N33, [N33], N33, [N33], false, false),\n    % same with an updated node:\n    ?compare_neighborhood(nodelist:remove(N15, Neighb5),\n                          N33, N33, [N33], N33, [N33], false, false),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:remove(N20, Neighb5), Neighb5),\n    ?equals(nodelist:remove(N9, Neighb5), Neighb5),\n    ?equals(nodelist:remove(?PID(3), Neighb5), Neighb5),\n    ?equals(nodelist:remove(?PID(1), Neighb5), Neighb5),\n\n    % Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:remove(N20, Neighb6), Neighb6),\n    ?equals(nodelist:remove(N9, Neighb6), Neighb6),\n    ?equals(nodelist:remove(?PID(3), Neighb6), Neighb6),\n    ?equals(nodelist:remove(?PID(1), Neighb6), Neighb6),\n\n    ok.\n\n%% @doc Tests for nodelist:remove/3.\n-spec test_remove3(any()) -> ok.\ntest_remove3(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N9 = node:new(?PID(1),   ?KEY(9), 1), % N8 updated\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N15 = node:new(?PID(2), ?KEY(15), 1), % N14 updated\n    N20 = node:new(?PID(3), ?KEY(20), 0),\n    N33 = node:new(?PID(4), ?KEY(33), 0),\n    N42 = node:new(?PID(5), ?KEY(42), 0),\n    N50 = node:new(?PID(6), ?KEY(50), 0),\n    N66 = node:new(?PID(7), ?KEY(66), 0),\n\n    RemovedNodesTab = ets:new(test_remove3, [duplicate_bag, private]),\n    AddNodeToTabFun = fun(Node) -> ets:insert(RemovedNodesTab, {Node}) end,\n    \n    Neighb1 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33),\n    Neighb2 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 3, 3),\n    Neighb3 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 2, 2),\n    Neighb4 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 1, 1),\n    Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n\n    % Neighb1 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:remove(N8, Neighb1, AddNodeToTabFun),\n                          N33, N14, [N14, N50, N42], N42, [N42, N50, N14], true, true),\n    ?equals(_N11 = ets:tab2list(RemovedNodesTab), [{N8}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(N14, Neighb1, AddNodeToTabFun),\n                          N33, N8, [N8, N50, N42], N42, [N42, N50, N8], true, true),\n    ?equals(_N12 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(N42, Neighb1, AddNodeToTabFun),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    ?equals(_N13 = ets:tab2list(RemovedNodesTab), [{N42}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(N50, Neighb1, AddNodeToTabFun),\n                          N33, N14, [N14, N8, N42], N42, [N42, N8, N14], true, true),\n    ?equals(_N14 = ets:tab2list(RemovedNodesTab), [{N50}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    % same with the node's pids:\n    ?compare_neighborhood(nodelist:remove(?PID(1), Neighb1, AddNodeToTabFun),\n                          N33, N14, [N14, N50, N42], N42, [N42, N50, N14], true, true),\n    ?equals(_N15 = ets:tab2list(RemovedNodesTab), [{N8}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(?PID(2), Neighb1, AddNodeToTabFun),\n                          N33, N8, [N8, N50, N42], N42, [N42, N50, N8], true, true),\n    ?equals(_N16 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(?PID(5), Neighb1, AddNodeToTabFun),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    ?equals(_N17 = ets:tab2list(RemovedNodesTab), [{N42}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(?PID(6), Neighb1, AddNodeToTabFun),\n                          N33, N14, [N14, N8, N42], N42, [N42, N8, N14], true, true),\n    ?equals(_N18 = ets:tab2list(RemovedNodesTab), [{N50}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    % same with an updated node:\n    ?compare_neighborhood(nodelist:remove(N9, Neighb1, AddNodeToTabFun),\n                          N33, N14, [N14, N50, N42], N42, [N42, N50, N14], true, true),\n    ?equals(_N19 = ets:tab2list(RemovedNodesTab), [{N8}]),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:remove(N20, Neighb1, AddNodeToTabFun), Neighb1),\n    ?equals(_N110 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(N66, Neighb1, AddNodeToTabFun), Neighb1),\n    ?equals(_N111 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(?PID(3), Neighb1, AddNodeToTabFun), Neighb1),\n    ?equals(_N112 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(?PID(7), Neighb1, AddNodeToTabFun), Neighb1),\n    ?equals(_N113 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % Neighb2 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 3, 3),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:remove(N8, Neighb2, AddNodeToTabFun),\n                          N33, N14, [N14, N50], N42, [N42, N50], true, true),\n    ?equals(_N21 = ets:tab2list(RemovedNodesTab), [{N8}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(N14, Neighb2, AddNodeToTabFun),\n                          N33, N8, [N8, N50], N42, [N42, N50, N8], true, true),\n    ?equals(_N22 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(N42, Neighb2, AddNodeToTabFun),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8], true, true),\n    ?equals(_N23 = ets:tab2list(RemovedNodesTab), [{N42}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(N50, Neighb2, AddNodeToTabFun),\n                          N33, N14, [N14, N8], N42, [N42, N8], true, true),\n    ?equals(_N24 = ets:tab2list(RemovedNodesTab), [{N50}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    % same with the node's pids:\n    ?compare_neighborhood(nodelist:remove(?PID(1), Neighb2, AddNodeToTabFun),\n                          N33, N14, [N14, N50], N42, [N42, N50], true, true),\n    ?equals(_N25 = ets:tab2list(RemovedNodesTab), [{N8}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(?PID(2), Neighb2, AddNodeToTabFun),\n                          N33, N8, [N8, N50], N42, [N42, N50, N8], true, true),\n    ?equals(_N26 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(?PID(5), Neighb2, AddNodeToTabFun),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8], true, true),\n    ?equals(_N27 = ets:tab2list(RemovedNodesTab), [{N42}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(?PID(6), Neighb2, AddNodeToTabFun),\n                          N33, N14, [N14, N8], N42, [N42, N8], true, true),\n    ?equals(_N28 = ets:tab2list(RemovedNodesTab), [{N50}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    % same with an updated node:\n    ?compare_neighborhood(nodelist:remove(N9, Neighb2, AddNodeToTabFun),\n                          N33, N14, [N14, N50], N42, [N42, N50], true, true),\n    ?equals(_N29 = ets:tab2list(RemovedNodesTab), [{N8}]),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:remove(N20, Neighb2, AddNodeToTabFun), Neighb2),\n    ?equals(_N210 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(N66, Neighb2, AddNodeToTabFun), Neighb2),\n    ?equals(_N211 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(?PID(3), Neighb2, AddNodeToTabFun), Neighb2),\n    ?equals(_N212 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(?PID(7), Neighb2, AddNodeToTabFun), Neighb2),\n    ?equals(_N213 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % Neighb3 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 2, 2),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:remove(N8, Neighb3, AddNodeToTabFun),\n                          N33, N14, [N14], N42, [N42, N50], true, true),\n    ?equals(_N31 = ets:tab2list(RemovedNodesTab), [{N8}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(N14, Neighb3, AddNodeToTabFun),\n                          N33, N8, [N8], N42, [N42, N50], true, true),\n    ?equals(_N32 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(N42, Neighb3, AddNodeToTabFun),\n                          N33, N14, [N14, N8], N50, [N50], true, true),\n    ?equals(_N33 = ets:tab2list(RemovedNodesTab), [{N42}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(N50, Neighb3, AddNodeToTabFun),\n                          N33, N14, [N14, N8], N42, [N42], true, true),\n    ?equals(_N34 = ets:tab2list(RemovedNodesTab), [{N50}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    % same with the node's pids:\n    ?compare_neighborhood(nodelist:remove(?PID(1), Neighb3, AddNodeToTabFun),\n                          N33, N14, [N14], N42, [N42, N50], true, true),\n    ?equals(_N35 = ets:tab2list(RemovedNodesTab), [{N8}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(?PID(2), Neighb3, AddNodeToTabFun),\n                          N33, N8, [N8], N42, [N42, N50], true, true),\n    ?equals(_N36 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(?PID(5), Neighb3, AddNodeToTabFun),\n                          N33, N14, [N14, N8], N50, [N50], true, true),\n    ?equals(_N37 = ets:tab2list(RemovedNodesTab), [{N42}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(?PID(6), Neighb3, AddNodeToTabFun),\n                          N33, N14, [N14, N8], N42, [N42], true, true),\n    ?equals(_N38 = ets:tab2list(RemovedNodesTab), [{N50}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    % same with an updated node:\n    ?compare_neighborhood(nodelist:remove(N9, Neighb3, AddNodeToTabFun),\n                          N33, N14, [N14], N42, [N42, N50], true, true),\n    ?equals(_N39 = ets:tab2list(RemovedNodesTab), [{N8}]),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:remove(N20, Neighb3, AddNodeToTabFun), Neighb3),\n    ?equals(_N310 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(N66, Neighb3, AddNodeToTabFun), Neighb3),\n    ?equals(_N311 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(?PID(3), Neighb3, AddNodeToTabFun), Neighb3),\n    ?equals(_N312 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(?PID(7), Neighb3, AddNodeToTabFun), Neighb3),\n    ?equals(_N313 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % Neighb4 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 1, 1),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:remove(N8, Neighb4, AddNodeToTabFun),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?equals(_N41 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(N14, Neighb4, AddNodeToTabFun),\n                          N33, N42, [N42], N42, [N42], true, true),\n    ?equals(_N42 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(N42, Neighb4, AddNodeToTabFun),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?equals(_N43 = ets:tab2list(RemovedNodesTab), [{N42}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(N50, Neighb4, AddNodeToTabFun),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?equals(_N44 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    % same with the node's pids:\n    ?compare_neighborhood(nodelist:remove(?PID(1), Neighb4, AddNodeToTabFun),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?equals(_N45 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(?PID(2), Neighb4, AddNodeToTabFun),\n                          N33, N42, [N42], N42, [N42], true, true),\n    ?equals(_N46 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(?PID(5), Neighb4, AddNodeToTabFun),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?equals(_N47 = ets:tab2list(RemovedNodesTab), [{N42}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:remove(?PID(6), Neighb4, AddNodeToTabFun),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?equals(_N48 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    % same with an updated node:\n    ?compare_neighborhood(nodelist:remove(N9, Neighb4, AddNodeToTabFun),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?equals(_N49 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:remove(N20, Neighb4, AddNodeToTabFun), Neighb4),\n    ?equals(_N410 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(N66, Neighb4, AddNodeToTabFun), Neighb4),\n    ?equals(_N411 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(?PID(3), Neighb4, AddNodeToTabFun), Neighb4),\n    ?equals(_N412 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(?PID(7), Neighb4, AddNodeToTabFun), Neighb4),\n    ?equals(_N413 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:remove(N14, Neighb5, AddNodeToTabFun),\n                          N33, N33, [N33], N33, [N33], false, false),\n    ?equals(_N51 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    % same with the node's pids:\n    ?compare_neighborhood(nodelist:remove(?PID(2), Neighb5, AddNodeToTabFun),\n                          N33, N33, [N33], N33, [N33], false, false),\n    ?equals(_N52 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    % same with an updated node:\n    ?compare_neighborhood(nodelist:remove(N15, Neighb5, AddNodeToTabFun),\n                          N33, N33, [N33], N33, [N33], false, false),\n    ?equals(_N53 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:remove(N20, Neighb5, AddNodeToTabFun), Neighb5),\n    ?equals(_N510 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(N9, Neighb5, AddNodeToTabFun), Neighb5),\n    ?equals(_N511 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(?PID(3), Neighb5, AddNodeToTabFun), Neighb5),\n    ?equals(_N512 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(?PID(1), Neighb5, AddNodeToTabFun), Neighb5),\n    ?equals(_N513 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:remove(N20, Neighb6, AddNodeToTabFun), Neighb6),\n    ?equals(_N610 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(N9, Neighb6, AddNodeToTabFun), Neighb6),\n    ?equals(_N611 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(?PID(3), Neighb6, AddNodeToTabFun), Neighb6),\n    ?equals(_N612 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:remove(?PID(1), Neighb6, AddNodeToTabFun), Neighb6),\n    ?equals(_N613 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    ets:delete(RemovedNodesTab),\n    ok.\n\n%% @doc Tests for nodelist:filter/2.\n-spec test_filter2(any()) -> ok.\ntest_filter2(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N9 = node:new(?PID(1),   ?KEY(9), 1), % N8 updated\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N20 = node:new(?PID(3), ?KEY(20), 0),\n    N33 = node:new(?PID(4), ?KEY(33), 0),\n    N42 = node:new(?PID(5), ?KEY(42), 0),\n    N50 = node:new(?PID(6), ?KEY(50), 0),\n    N66 = node:new(?PID(7), ?KEY(66), 0),\n    \n    Neighb1 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33),\n    Neighb2 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 3, 3),\n    Neighb3 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 2, 2),\n    Neighb4 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 1, 1),\n    Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n\n    % Neighb1 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter(Neighb1, fun(N) -> not node:same_process(N, N8) end),\n                          N33, N14, [N14, N50, N42], N42, [N42, N50, N14], true, true),\n    ?compare_neighborhood(nodelist:filter(Neighb1, fun(N) -> not node:same_process(N, N14) end),\n                          N33, N8, [N8, N50, N42], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:filter(Neighb1, fun(N) -> not node:same_process(N, N42) end),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:filter(Neighb1, fun(N) -> not node:same_process(N, N50) end),\n                          N33, N14, [N14, N8, N42], N42, [N42, N8, N14], true, true),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter(Neighb1, fun(N) -> not node:same_process(N, N20) end), Neighb1),\n    ?equals(nodelist:filter(Neighb1, fun(N) -> not node:same_process(N, N66) end), Neighb1),\n\n    % Neighb2 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 3, 3),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter(Neighb2, fun(N) -> not node:same_process(N, N8) end),\n                          N33, N14, [N14, N50], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:filter(Neighb2, fun(N) -> not node:same_process(N, N14) end),\n                          N33, N8, [N8, N50], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:filter(Neighb2, fun(N) -> not node:same_process(N, N42) end),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8], true, true),\n    ?compare_neighborhood(nodelist:filter(Neighb2, fun(N) -> not node:same_process(N, N50) end),\n                          N33, N14, [N14, N8], N42, [N42, N8], true, true),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter(Neighb2, fun(N) -> not node:same_process(N, N20) end), Neighb2),\n    ?equals(nodelist:filter(Neighb2, fun(N) -> not node:same_process(N, N66) end), Neighb2),\n\n    % Neighb3 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 2, 2),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter(Neighb3, fun(N) -> not node:same_process(N, N8) end),\n                          N33, N14, [N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:filter(Neighb3, fun(N) -> not node:same_process(N, N14) end),\n                          N33, N8, [N8], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:filter(Neighb3, fun(N) -> not node:same_process(N, N42) end),\n                          N33, N14, [N14, N8], N50, [N50], true, true),\n    ?compare_neighborhood(nodelist:filter(Neighb3, fun(N) -> not node:same_process(N, N50) end),\n                          N33, N14, [N14, N8], N42, [N42], true, true),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter(Neighb3, fun(N) -> not node:same_process(N, N20) end), Neighb3),\n    ?equals(nodelist:filter(Neighb3, fun(N) -> not node:same_process(N, N66) end), Neighb3),\n\n    % Neighb4 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 1, 1),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter(Neighb4, fun(N) -> not node:same_process(N, N8) end),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:filter(Neighb4, fun(N) -> not node:same_process(N, N14) end),\n                          N33, N42, [N42], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:filter(Neighb4, fun(N) -> not node:same_process(N, N42) end),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(nodelist:filter(Neighb4, fun(N) -> not node:same_process(N, N50) end),\n                          N33, N14, [N14], N42, [N42], true, true),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter(Neighb4, fun(N) -> not node:same_process(N, N20) end), Neighb4),\n    ?equals(nodelist:filter(Neighb4, fun(N) -> not node:same_process(N, N66) end), Neighb4),\n\n    % Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter(Neighb5, fun(N) -> not node:same_process(N, N14) end),\n                          N33, N33, [N33], N33, [N33], false, false),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter(Neighb5, fun(N) -> not node:same_process(N, N20) end), Neighb5),\n    ?equals(nodelist:filter(Neighb5, fun(N) -> not node:same_process(N, N9) end), Neighb5),\n\n    % Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter(Neighb6, fun(N) -> not node:same_process(N, N20) end), Neighb6),\n    ?equals(nodelist:filter(Neighb6, fun(N) -> not node:same_process(N, N9) end), Neighb6),\n\n    ok.\n\n%% @doc Tests for nodelist:filter/3.\n-spec test_filter3(any()) -> ok.\ntest_filter3(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N9 = node:new(?PID(1),   ?KEY(9), 1), % N8 updated\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N20 = node:new(?PID(3), ?KEY(20), 0),\n    N33 = node:new(?PID(4), ?KEY(33), 0),\n    N42 = node:new(?PID(5), ?KEY(42), 0),\n    N50 = node:new(?PID(6), ?KEY(50), 0),\n    N66 = node:new(?PID(7), ?KEY(66), 0),\n\n    RemovedNodesTab = ets:new(test_filter3, [duplicate_bag, private]),\n    AddNodeToTabFun = fun(Node) -> ets:insert(RemovedNodesTab, {Node}) end,\n    NodeListToTabListFun = fun(NodeList) -> lists:sort([{Node} || Node <- NodeList]) end,\n    \n    Neighb1 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33),\n    Neighb2 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 3, 3),\n    Neighb3 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 2, 2),\n    Neighb4 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 1, 1),\n    Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n\n    % Neighb1 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter(Neighb1, fun(N) -> not node:same_process(N, N8) end, AddNodeToTabFun),\n                          N33, N14, [N14, N50, N42], N42, [N42, N50, N14], true, true),\n    ?equals(_N11 = ets:tab2list(RemovedNodesTab), [{N8}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:filter(Neighb1, fun(N) -> not node:same_process(N, N14) end, AddNodeToTabFun),\n                          N33, N8, [N8, N50, N42], N42, [N42, N50, N8], true, true),\n    ?equals(_N12 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:filter(Neighb1, fun(N) -> not node:same_process(N, N42) end, AddNodeToTabFun),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    ?equals(_N13 = ets:tab2list(RemovedNodesTab), [{N42}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:filter(Neighb1, fun(N) -> not node:same_process(N, N50) end, AddNodeToTabFun),\n                          N33, N14, [N14, N8, N42], N42, [N42, N8, N14], true, true),\n    ?equals(_N14 = ets:tab2list(RemovedNodesTab), [{N50}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:filter(Neighb1, fun(N) -> not lists:member(node:id(N), [?KEY(8), ?KEY(14)]) end, AddNodeToTabFun),\n                          N33, N50, [N50, N42], N42, [N42, N50], true, true),\n    ?equals(_N15 = lists:sort(ets:tab2list(RemovedNodesTab)), NodeListToTabListFun([N8, N14])),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter(Neighb1, fun(N) -> not node:same_process(N, N20) end, AddNodeToTabFun), Neighb1),\n    ?equals(_N110 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:filter(Neighb1, fun(N) -> not node:same_process(N, N66) end, AddNodeToTabFun), Neighb1),\n    ?equals(_N111 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % Neighb2 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 3, 3),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter(Neighb2, fun(N) -> not node:same_process(N, N8) end, AddNodeToTabFun),\n                          N33, N14, [N14, N50], N42, [N42, N50], true, true),\n    ?equals(_N21 = ets:tab2list(RemovedNodesTab), [{N8}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:filter(Neighb2, fun(N) -> not node:same_process(N, N14) end, AddNodeToTabFun),\n                          N33, N8, [N8, N50], N42, [N42, N50, N8], true, true),\n    ?equals(_N22 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:filter(Neighb2, fun(N) -> not node:same_process(N, N42) end, AddNodeToTabFun),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8], true, true),\n    ?equals(_N23 = ets:tab2list(RemovedNodesTab), [{N42}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:filter(Neighb2, fun(N) -> not node:same_process(N, N50) end, AddNodeToTabFun),\n                          N33, N14, [N14, N8], N42, [N42, N8], true, true),\n    ?equals(_N24 = ets:tab2list(RemovedNodesTab), [{N50}]),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter(Neighb2, fun(N) -> not node:same_process(N, N20) end, AddNodeToTabFun), Neighb2),\n    ?equals(_N210 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:filter(Neighb2, fun(N) -> not node:same_process(N, N66) end, AddNodeToTabFun), Neighb2),\n    ?equals(_N211 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % Neighb3 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 2, 2),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter(Neighb3, fun(N) -> not node:same_process(N, N8) end, AddNodeToTabFun),\n                          N33, N14, [N14], N42, [N42, N50], true, true),\n    ?equals(_N31 = ets:tab2list(RemovedNodesTab), [{N8}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:filter(Neighb3, fun(N) -> not node:same_process(N, N14) end, AddNodeToTabFun),\n                          N33, N8, [N8], N42, [N42, N50], true, true),\n    ?equals(_N32 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:filter(Neighb3, fun(N) -> not node:same_process(N, N42) end, AddNodeToTabFun),\n                          N33, N14, [N14, N8], N50, [N50], true, true),\n    ?equals(_N33 = ets:tab2list(RemovedNodesTab), [{N42}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:filter(Neighb3, fun(N) -> not node:same_process(N, N50) end, AddNodeToTabFun),\n                          N33, N14, [N14, N8], N42, [N42], true, true),\n    ?equals(_N34 = ets:tab2list(RemovedNodesTab), [{N50}]),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter(Neighb3, fun(N) -> not node:same_process(N, N20) end, AddNodeToTabFun), Neighb3),\n    ?equals(_N310 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:filter(Neighb3, fun(N) -> not node:same_process(N, N66) end, AddNodeToTabFun), Neighb3),\n    ?equals(_N311 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % Neighb4 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 1, 1),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter(Neighb4, fun(N) -> not node:same_process(N, N8) end, AddNodeToTabFun),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?equals(_N41 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:filter(Neighb4, fun(N) -> not node:same_process(N, N14) end, AddNodeToTabFun),\n                          N33, N42, [N42], N42, [N42], true, true),\n    ?equals(_N42 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:filter(Neighb4, fun(N) -> not node:same_process(N, N42) end, AddNodeToTabFun),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?equals(_N43 = ets:tab2list(RemovedNodesTab), [{N42}]),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?compare_neighborhood(nodelist:filter(Neighb4, fun(N) -> not node:same_process(N, N50) end, AddNodeToTabFun),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?equals(_N44 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter(Neighb4, fun(N) -> not node:same_process(N, N20) end, AddNodeToTabFun), Neighb4),\n    ?equals(_N410 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:filter(Neighb4, fun(N) -> not node:same_process(N, N66) end, AddNodeToTabFun), Neighb4),\n    ?equals(_N411 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter(Neighb5, fun(N) -> not node:same_process(N, N14) end, AddNodeToTabFun),\n                          N33, N33, [N33], N33, [N33], false, false),\n    ?equals(_N51 = ets:tab2list(RemovedNodesTab), [{N14}]),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter(Neighb5, fun(N) -> not node:same_process(N, N20) end, AddNodeToTabFun), Neighb5),\n    ?equals(_N510 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:filter(Neighb5, fun(N) -> not node:same_process(N, N9) end, AddNodeToTabFun), Neighb5),\n    ?equals(_N511 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    % Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter(Neighb6, fun(N) -> not node:same_process(N, N20) end, AddNodeToTabFun), Neighb6),\n    ?equals(_N610 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n    ?equals(nodelist:filter(Neighb6, fun(N) -> not node:same_process(N, N9) end, AddNodeToTabFun), Neighb6),\n    ?equals(_N611 = ets:tab2list(RemovedNodesTab), []),\n    ets:delete_all_objects(RemovedNodesTab),\n\n    ets:delete(RemovedNodesTab),\n    ok.\n\n%% @doc Tests for nodelist:filter_min_length/4.\n-spec test_filter_min_length4(any()) -> ok.\ntest_filter_min_length4(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N20 = node:new(?PID(3), ?KEY(20), 0),\n    N33 = node:new(?PID(4), ?KEY(33), 0),\n    N42 = node:new(?PID(5), ?KEY(42), 0),\n    N50 = node:new(?PID(6), ?KEY(50), 0),\n    N66 = node:new(?PID(7), ?KEY(66), 0),\n    \n    Neighb1 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33),\n    Neighb2 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 3, 3),\n    Neighb3 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 2, 2),\n    Neighb4 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 1, 1),\n    Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n\n    % Neighb1 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb1, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 0, 0),\n                          N33, N14, [N14, N42], N42, [N42, N14], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb1, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 1, 1),\n                          N33, N14, [N14, N42], N42, [N42, N14], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb1, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 2, 2),\n                          N33, N14, [N14, N42], N42, [N42, N14], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb1, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 3, 3),\n                          N33, N14, [N14, N8, N42], N42, [N42, N50, N14], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb1, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 4, 4),\n                          N33, N14, [N14, N8, N50, N42], N42, [N42, N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb1, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 5, 5),\n                          N33, N14, [N14, N8, N50, N42], N42, [N42, N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb1, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 2, 4),\n                          N33, N14, [N14, N42], N42, [N42, N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb1, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 4, 2),\n                          N33, N14, [N14, N8, N50, N42], N42, [N42, N14], true, true),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter_min_length(Neighb1, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 0, 0), Neighb1),\n    ?equals(nodelist:filter_min_length(Neighb1, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 3, 3), Neighb1),\n    ?equals(nodelist:filter_min_length(Neighb1, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 2, 5), Neighb1),\n    ?equals(nodelist:filter_min_length(Neighb1, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 5, 2), Neighb1),\n\n    % Neighb2 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 3, 3),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb2, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 0, 0),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb2, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 1, 1),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb2, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 2, 2),\n                          N33, N14, [N14, N8], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb2, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 3, 3),\n                          N33, N14, [N14, N8, N50], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb2, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 4, 4),\n                          N33, N14, [N14, N8, N50], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb2, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 2, 4),\n                          N33, N14, [N14, N8], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb2, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 4, 2),\n                          N33, N14, [N14, N8, N50], N42, [N42, N50], true, true),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter_min_length(Neighb2, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 0, 0), Neighb2),\n    ?equals(nodelist:filter_min_length(Neighb2, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 3, 3), Neighb2),\n    ?equals(nodelist:filter_min_length(Neighb2, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 2, 5), Neighb2),\n    ?equals(nodelist:filter_min_length(Neighb2, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 5, 2), Neighb2),\n\n    % Neighb3 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 2, 2),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb3, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 0, 0),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb3, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 1, 1),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb3, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 2, 2),\n                          N33, N14, [N14, N8], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb3, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 3, 3),\n                          N33, N14, [N14, N8], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb3, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 2, 4),\n                          N33, N14, [N14, N8], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb3, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N50)) end, 4, 2),\n                          N33, N14, [N14, N8], N42, [N42, N50], true, true),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter_min_length(Neighb3, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 0, 0), Neighb3),\n    ?equals(nodelist:filter_min_length(Neighb3, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 3, 3), Neighb3),\n    ?equals(nodelist:filter_min_length(Neighb3, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 2, 5), Neighb3),\n    ?equals(nodelist:filter_min_length(Neighb3, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 5, 2), Neighb3),\n\n    % Neighb4 = nodelist:mk_neighborhood([N8, N14, N42, N50], N33, 1, 1),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb4, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N42)) end, 0, 0),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb4, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N42)) end, 1, 1),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb4, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N42)) end, 2, 2),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb4, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N42)) end, 0, 1),\n                          N33, N14, [N14], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb4, fun(N) -> not (node:same_process(N, N8) orelse node:same_process(N, N42)) end, 1, 0),\n                          N33, N14, [N14], N14, [N14], true, true),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter_min_length(Neighb4, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 0, 0), Neighb4),\n    ?equals(nodelist:filter_min_length(Neighb4, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 3, 3), Neighb4),\n    ?equals(nodelist:filter_min_length(Neighb4, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 2, 5), Neighb4),\n    ?equals(nodelist:filter_min_length(Neighb4, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 5, 2), Neighb4),\n\n    % Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    % remove existing nodes:\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb5, fun(N) -> not node:same_process(N, N14) end, 0, 0),\n                          N33, N33, [N33], N33, [N33], false, false),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb5, fun(N) -> not node:same_process(N, N14) end, 1, 1),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb5, fun(N) -> not node:same_process(N, N14) end, 0, 1),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(nodelist:filter_min_length(Neighb5, fun(N) -> not node:same_process(N, N14) end, 1, 0),\n                          N33, N14, [N14], N14, [N14], true, true),\n\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter_min_length(Neighb5, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 0, 0), Neighb5),\n    ?equals(nodelist:filter_min_length(Neighb5, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 3, 3), Neighb5),\n    ?equals(nodelist:filter_min_length(Neighb5, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 2, 5), Neighb5),\n    ?equals(nodelist:filter_min_length(Neighb5, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 5, 2), Neighb5),\n\n    % Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n    % remove non-existing nodes and pids:\n    ?equals(nodelist:filter_min_length(Neighb6, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 0, 0), Neighb6),\n    ?equals(nodelist:filter_min_length(Neighb6, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 3, 3), Neighb6),\n    ?equals(nodelist:filter_min_length(Neighb6, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 2, 5), Neighb6),\n    ?equals(nodelist:filter_min_length(Neighb6, fun(N) -> not (node:same_process(N, N20) orelse node:same_process(N, N66)) end, 5, 2), Neighb6),\n\n    ok.\n\n-spec test_merge(any()) -> ok.\ntest_merge(_Config) ->\n    % use the same tests as in test_add_nodes/1 and let mk_beighborhood create the appropriate neighborhood structures:\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N9 = node:new(?PID(1),   ?KEY(9), 1), % N8 updated\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N15 = node:new(?PID(2), ?KEY(15), 1), % N14 updated\n    N20 = node:new(?PID(3), ?KEY(20), 0),\n    N21 = node:new(?PID(3), ?KEY(21), 1), % N20 updated\n    N33 = node:new(?PID(4), ?KEY(33), 0),\n    N42 = node:new(?PID(5), ?KEY(42), 0),\n    N50 = node:new(?PID(6), ?KEY(50), 0),\n    N66 = node:new(?PID(7), ?KEY(66), 0),\n    \n    Neighb1 = nodelist:mk_neighborhood([N8, N14, N50], N33),\n    Neighb2 = nodelist:mk_neighborhood([N8, N14, N50], N33, 3, 3),\n    Neighb3 = nodelist:mk_neighborhood([N8, N14, N50], N33, 2, 2),\n    Neighb4 = nodelist:mk_neighborhood([N8, N14, N50], N33, 1, 1),\n    Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n    \n    % Neighb1 = nodelist:mk_neighborhood([N8, N14, N50], N33),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N66], N20), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N20, N66], N42), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N20, N42], N66), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N66, N42], N20), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N66, N9], N20), 3, 3),\n                          N33, N20, [N20, N14, N9], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N66], N21), 3, 3),\n                          N33, N21, [N21, N14, N8], N42, [N42, N50, N66], true, true),\n    \n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N66], N20), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N20, N66], N42), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N20, N42], N66), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N66, N42], N20), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N66, N9], N20), 4, 4),\n                          N33, N20, [N20, N14, N9, N66], N42, [N42, N50, N66, N9], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N66], N21), 4, 4),\n                          N33, N21, [N21, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    \n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N66], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N20, N66], N42), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N20, N42], N66), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N66, N42], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N66, N9], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N66], N21), 2, 2),\n                          N33, N21, [N21, N14], N42, [N42, N50], true, true),\n    \n    % Neighb2 = nodelist:mk_neighborhood([N8, N14, N50], N33, 3, 3),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N66], N20), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N20, N66], N42), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N20, N42], N66), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N66, N42], N20), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N66, N9], N20), 3, 3),\n                          N33, N20, [N20, N14, N9], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N66], N21), 3, 3),\n                          N33, N21, [N21, N14, N8], N42, [N42, N50, N66], true, true),\n    \n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N66], N20), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N20, N66], N42), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N20, N42], N66), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N66, N42], N20), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N66, N9], N20), 4, 4),\n                          N33, N20, [N20, N14, N9, N66], N42, [N42, N50, N66, N9], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N66], N21), 4, 4),\n                          N33, N21, [N21, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    \n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N66], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N20, N66], N42), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N20, N42], N66), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N66, N42], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N66, N9], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N66], N21), 2, 2),\n                          N33, N21, [N21, N14], N42, [N42, N50], true, true),\n\n    % Neighb3 = nodelist:mk_neighborhood([N8, N14, N50], N33, 2, 2),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N66], N20), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N20, N66], N42), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N20, N42], N66), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N66, N42], N20), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N66, N9], N20), 3, 3),\n                          N33, N20, [N20, N14, N9], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N66], N21), 3, 3),\n                          N33, N21, [N21, N14, N8], N42, [N42, N50, N66], true, true),\n    \n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N66], N20), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N20, N66], N42), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N20, N42], N66), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N66, N42], N20), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N66, N9], N20), 4, 4),\n                          N33, N20, [N20, N14, N9, N66], N42, [N42, N50, N66, N9], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N66], N21), 4, 4),\n                          N33, N21, [N21, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    \n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N66], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N20, N66], N42), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N20, N42], N66), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N66, N42], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N66, N9], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N66], N21), 2, 2),\n                          N33, N21, [N21, N14], N42, [N42, N50], true, true),\n\n    % Neighb4 = nodelist:mk_neighborhood([N8, N14, N50], N33, 1, 1),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N66], N20), 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N20, N66], N42), 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N20, N42], N66), 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N66, N42], N20), 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N66, N9], N20), 3, 3),\n                          N33, N20, [N20, N14, N9], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N66], N21), 3, 3),\n                          N33, N21, [N21, N14, N66], N42, [N42, N50, N66], true, true),\n    \n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N66], N20), 4, 4),\n                          N33, N20, [N20, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N20, N66], N42), 4, 4),\n                          N33, N20, [N20, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N20, N42], N66), 4, 4),\n                          N33, N20, [N20, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N66, N42], N20), 4, 4),\n                          N33, N20, [N20, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 4, 4),\n                          N33, N20, [N20, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N66, N9], N20), 4, 4),\n                          N33, N20, [N20, N14, N9, N66], N42, [N42, N50, N66, N9], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N66], N21), 4, 4),\n                          N33, N21, [N21, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    \n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N66], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N20, N66], N42), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N20, N42], N66), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N66, N42], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N66, N9], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N66], N21), 2, 2),\n                          N33, N21, [N21, N14], N42, [N42, N50], true, true),\n    \n    % tests with (almost) empty successor/predecessor lists:\n    % Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N66], N20), 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N20, N66], N42), 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N20, N42], N66), 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N66, N42], N20), 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N66, N9], N20), 3, 3),\n                          N33, N20, [N20, N14, N9], N42, [N42, N66, N9], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N66], N21), 3, 3),\n                          N33, N21, [N21, N14, N66], N42, [N42, N66, N14], true, true),\n    \n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N66], N20), 4, 4),\n                          N33, N20, [N20, N14, N66, N42], N42, [N42, N66, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N20, N66], N42), 4, 4),\n                          N33, N20, [N20, N14, N66, N42], N42, [N42, N66, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N20, N42], N66), 4, 4),\n                          N33, N20, [N20, N14, N66, N42], N42, [N42, N66, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N66, N42], N20), 4, 4),\n                          N33, N20, [N20, N14, N66, N42], N42, [N42, N66, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 4, 4),\n                          N33, N20, [N20, N14, N66, N42], N42, [N42, N66, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N66, N9], N20), 4, 4),\n                          N33, N20, [N20, N14, N9, N66], N42, [N42, N66, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N66], N21), 4, 4),\n                          N33, N21, [N21, N14, N66, N42], N42, [N42, N66, N14, N21], true, true),\n    \n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N66], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N20, N66], N42), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N20, N42], N66), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N66, N42], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N66, N9], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N66], N21), 2, 2),\n                          N33, N21, [N21, N14], N42, [N42, N66], true, true),\n    \n    % adding one node:\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:new_neighborhood(N20), 3, 3),\n                          N33, N20, [N20, N14], N14, [N14, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:new_neighborhood(N15), 3, 3),\n                          N33, N15, [N15], N15, [N15], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:new_neighborhood(N33), 3, 3),\n                          N33, N14, [N14], N14, [N14], true, true),\n    \n%%     Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N66], N20), 3, 3),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N20, N66], N42), 3, 3),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N20, N42], N66), 3, 3),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N66, N42], N20), 3, 3),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 3, 3),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N66, N9], N20), 3, 3),\n                          N33, N20, [N20, N9, N66], N42, [N42, N66, N9], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N66], N21), 3, 3),\n                          N33, N21, [N21, N66, N42], N42, [N42, N66, N21], true, true),\n    \n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N66], N20), 4, 4),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N20, N66], N42), 4, 4),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N20, N42], N66), 4, 4),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N66, N42], N20), 4, 4),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 4, 4),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N66, N9], N20), 4, 4),\n                          N33, N20, [N20, N9, N66, N42], N42, [N42, N66, N9, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N66], N21), 4, 4),\n                          N33, N21, [N21, N66, N42], N42, [N42, N66, N21], true, true),\n    \n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N66], N20), 2, 2),\n                          N33, N20, [N20, N66], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N20, N66], N42), 2, 2),\n                          N33, N20, [N20, N66], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N20, N42], N66), 2, 2),\n                          N33, N20, [N20, N66], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N66, N42], N20), 2, 2),\n                          N33, N20, [N20, N66], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N20, N42, N66, N42], N20), 2, 2),\n                          N33, N20, [N20, N66], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N66, N9], N20), 2, 2),\n                          N33, N20, [N20, N9], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N66], N21), 2, 2),\n                          N33, N21, [N21, N66], N42, [N42, N66], true, true),\n    \n    % adding one node:\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:new_neighborhood(N20), 3, 3),\n                          N33, N20, [N20], N20, [N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:new_neighborhood(N33), 3, 3),\n                          N33, N33, [N33], N33, [N33], false, false),\n    \n    % some tests with the base node in the to-be-added nodes:\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N33, N66], N20), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N42, N66, N33], N20), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb1, nodelist:mk_neighborhood([N20, N42, N66], N33), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N33, N66], N20), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N42, N66, N33], N20), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb2, nodelist:mk_neighborhood([N20, N42, N66], N33), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N33, N66], N20), 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N42, N66, N33], N20), 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb3, nodelist:mk_neighborhood([N20, N42, N66], N33), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N33, N66], N20), 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N42, N66, N33], N20), 4, 4),\n                          N33, N20, [N20, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb4, nodelist:mk_neighborhood([N20, N42, N66], N33), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N33, N66], N20), 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N42, N66, N33], N20), 4, 4),\n                          N33, N20, [N20, N14, N66, N42], N42, [N42, N66, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb5, nodelist:mk_neighborhood([N20, N42, N66], N33), 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N33, N66], N20), 3, 3),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N42, N66, N33], N20), 4, 4),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:merge(Neighb6, nodelist:mk_neighborhood([N20, N42, N66], N33), 2, 2),\n                          N33, N20, [N20, N66], N42, [N42, N66], true, true),\n    \n    ok.\n\n%% @doc Tests adding single nodes to a neighborhood structure.\n-spec test_add_node(any()) -> ok.\ntest_add_node(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N9 = node:new(?PID(1),   ?KEY(9), 1), % N8 updated\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N15 = node:new(?PID(2), ?KEY(15), 1), % N14 updated\n    N20 = node:new(?PID(3), ?KEY(20), 0),\n    N33 = node:new(?PID(4), ?KEY(33), 0),\n    N42 = node:new(?PID(5), ?KEY(42), 0),\n    N50 = node:new(?PID(6), ?KEY(50), 0),\n    N66 = node:new(?PID(7), ?KEY(66), 0),\n    \n    Neighb1 = nodelist:mk_neighborhood([N8, N14, N50], N33),\n    Neighb2 = nodelist:mk_neighborhood([N8, N14, N50], N33, 3, 3),\n    Neighb3 = nodelist:mk_neighborhood([N8, N14, N50], N33, 2, 2),\n    Neighb4 = nodelist:mk_neighborhood([N8, N14, N50], N33, 1, 1),\n    Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n    \n    % Neighb1 = nodelist:mk_neighborhood([N8, N14, N50], N33),\n    ?compare_neighborhood(nodelist:add_node(Neighb1, N20, 3, 3),\n                          N33, N20, [N20, N14, N8], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb1, N42, 3, 3),\n                          N33, N14, [N14, N8, N50], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb1, N66, 3, 3),\n                          N33, N14, [N14, N8, N66], N50, [N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb1, N9, 3, 3),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb1, N33, 3, 3),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    \n    ?compare_neighborhood(nodelist:add_node(Neighb1, N20, 4, 4),\n                          N33, N20, [N20, N14, N8, N50], N50, [N50, N8, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb1, N42, 4, 4),\n                          N33, N14, [N14, N8, N50, N42], N42, [N42, N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb1, N66, 4, 4),\n                          N33, N14, [N14, N8, N66, N50], N50, [N50, N66, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb1, N9, 4, 4),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb1, N33, 4, 4),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    \n    ?compare_neighborhood(nodelist:add_node(Neighb1, N20, 2, 2),\n                          N33, N20, [N20, N14], N50, [N50, N8], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb1, N42, 2, 2),\n                          N33, N14, [N14, N8], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb1, N66, 2, 2),\n                          N33, N14, [N14, N8], N50, [N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb1, N9, 2, 2),\n                          N33, N14, [N14, N9], N50, [N50, N9], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb1, N33, 2, 2),\n                          N33, N14, [N14, N8], N50, [N50, N8], true, true),\n    \n    % Neighb2 = nodelist:mk_neighborhood([N8, N14, N50], N33, 3, 3),\n    ?compare_neighborhood(nodelist:add_node(Neighb2, N20, 3, 3),\n                          N33, N20, [N20, N14, N8], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb2, N42, 3, 3),\n                          N33, N14, [N14, N8, N50], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb2, N66, 3, 3),\n                          N33, N14, [N14, N8, N66], N50, [N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb2, N9, 3, 3),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb2, N33, 3, 3),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    \n    ?compare_neighborhood(nodelist:add_node(Neighb2, N20, 4, 4),\n                          N33, N20, [N20, N14, N8, N50], N50, [N50, N8, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb2, N42, 4, 4),\n                          N33, N14, [N14, N8, N50, N42], N42, [N42, N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb2, N66, 4, 4),\n                          N33, N14, [N14, N8, N66, N50], N50, [N50, N66, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb2, N9, 4, 4),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb2, N33, 4, 4),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    \n    ?compare_neighborhood(nodelist:add_node(Neighb2, N20, 2, 2),\n                          N33, N20, [N20, N14], N50, [N50, N8], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb2, N42, 2, 2),\n                          N33, N14, [N14, N8], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb2, N66, 2, 2),\n                          N33, N14, [N14, N8], N50, [N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb2, N9, 2, 2),\n                          N33, N14, [N14, N9], N50, [N50, N9], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb2, N33, 2, 2),\n                          N33, N14, [N14, N8], N50, [N50, N8], true, true),\n\n    % Neighb3 = nodelist:mk_neighborhood([N8, N14, N50], N33, 2, 2),\n    ?compare_neighborhood(nodelist:add_node(Neighb3, N20, 3, 3),\n                          N33, N20, [N20, N14, N8], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb3, N42, 3, 3),\n                          N33, N14, [N14, N8, N50], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb3, N66, 3, 3),\n                          N33, N14, [N14, N8, N66], N50, [N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb3, N9, 3, 3),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb3, N33, 3, 3),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    \n    ?compare_neighborhood(nodelist:add_node(Neighb3, N20, 4, 4),\n                          N33, N20, [N20, N14, N8, N50], N50, [N50, N8, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb3, N42, 4, 4),\n                          N33, N14, [N14, N8, N50, N42], N42, [N42, N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb3, N66, 4, 4),\n                          N33, N14, [N14, N8, N66, N50], N50, [N50, N66, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb3, N9, 4, 4),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb3, N33, 4, 4),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    \n    ?compare_neighborhood(nodelist:add_node(Neighb3, N20, 2, 2),\n                          N33, N20, [N20, N14], N50, [N50, N8], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb3, N42, 2, 2),\n                          N33, N14, [N14, N8], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb3, N66, 2, 2),\n                          N33, N14, [N14, N8], N50, [N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb3, N9, 2, 2),\n                          N33, N14, [N14, N9], N50, [N50, N9], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb3, N33, 2, 2),\n                          N33, N14, [N14, N8], N50, [N50, N8], true, true),\n\n    % Neighb4 = nodelist:mk_neighborhood([N8, N14, N50], N33, 1, 1),\n    ?compare_neighborhood(nodelist:add_node(Neighb4, N20, 3, 3),\n                          N33, N20, [N20, N14, N50], N50, [N50, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb4, N42, 3, 3),\n                          N33, N14, [N14, N50, N42], N42, [N42, N50, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb4, N66, 3, 3),\n                          N33, N14, [N14, N66, N50], N50, [N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb4, N9, 3, 3),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb4, N33, 3, 3),\n                          N33, N14, [N14, N50], N50, [N50, N14], true, true),\n    \n    ?compare_neighborhood(nodelist:add_node(Neighb4, N20, 4, 4),\n                          N33, N20, [N20, N14, N50], N50, [N50, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb4, N42, 4, 4),\n                          N33, N14, [N14, N50, N42], N42, [N42, N50, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb4, N66, 4, 4),\n                          N33, N14, [N14, N66, N50], N50, [N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb4, N9, 4, 4),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb4, N33, 4, 4),\n                          N33, N14, [N14, N50], N50, [N50, N14], true, true),\n    \n    ?compare_neighborhood(nodelist:add_node(Neighb4, N20, 2, 2),\n                          N33, N20, [N20, N14], N50, [N50, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb4, N42, 2, 2),\n                          N33, N14, [N14, N50], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb4, N66, 2, 2),\n                          N33, N14, [N14, N66], N50, [N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb4, N9, 2, 2),\n                          N33, N14, [N14, N9], N50, [N50, N9], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb4, N33, 2, 2),\n                          N33, N14, [N14, N50], N50, [N50, N14], true, true),\n    \n    % tests with (almost) empty successor/predecessor lists:\n    % Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    ?compare_neighborhood(nodelist:add_node(Neighb5, N20, 3, 3),\n                          N33, N20, [N20, N14], N14, [N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb5, N42, 3, 3),\n                          N33, N14, [N14, N42], N42, [N42, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb5, N66, 3, 3),\n                          N33, N14, [N14, N66], N66, [N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb5, N9, 3, 3),\n                          N33, N14, [N14, N9], N9, [N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb5, N33, 3, 3),\n                          N33, N14, [N14], N14, [N14], true, true),\n    \n    ?compare_neighborhood(nodelist:add_node(Neighb5, N20, 4, 4),\n                          N33, N20, [N20, N14], N14, [N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb5, N42, 4, 4),\n                          N33, N14, [N14, N42], N42, [N42, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb5, N66, 4, 4),\n                          N33, N14, [N14, N66], N66, [N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb5, N9, 4, 4),\n                          N33, N14, [N14, N9], N9, [N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb5, N33, 4, 4),\n                          N33, N14, [N14], N14, [N14], true, true),\n    \n    ?compare_neighborhood(nodelist:add_node(Neighb5, N20, 2, 2),\n                          N33, N20, [N20, N14], N14, [N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb5, N42, 2, 2),\n                          N33, N14, [N14, N42], N42, [N42, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb5, N66, 2, 2),\n                          N33, N14, [N14, N66], N66, [N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb5, N9, 2, 2),\n                          N33, N14, [N14, N9], N9, [N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb5, N33, 2, 2),\n                          N33, N14, [N14], N14, [N14], true, true),\n    \n%%     Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n    ?compare_neighborhood(nodelist:add_node(Neighb6, N20, 3, 3),\n                          N33, N20, [N20], N20, [N20], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb6, N42, 3, 3),\n                          N33, N42, [N42], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb6, N66, 3, 3),\n                          N33, N66, [N66], N66, [N66], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb6, N9, 3, 3),\n                          N33, N9, [N9], N9, [N9], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb6, N33, 3, 3),\n                          N33, N33, [N33], N33, [N33], false, false),\n    \n    ?compare_neighborhood(nodelist:add_node(Neighb6, N20, 4, 4),\n                          N33, N20, [N20], N20, [N20], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb6, N42, 4, 4),\n                          N33, N42, [N42], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb6, N66, 4, 4),\n                          N33, N66, [N66], N66, [N66], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb6, N9, 4, 4),\n                          N33, N9, [N9], N9, [N9], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb6, N33, 4, 4),\n                          N33, N33, [N33], N33, [N33], false, false),\n    \n    ?compare_neighborhood(nodelist:add_node(Neighb6, N20, 2, 2),\n                          N33, N20, [N20], N20, [N20], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb6, N42, 2, 2),\n                          N33, N42, [N42], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb6, N66, 2, 2),\n                          N33, N66, [N66], N66, [N66], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb6, N9, 2, 2),\n                          N33, N9, [N9], N9, [N9], true, true),\n    ?compare_neighborhood(nodelist:add_node(Neighb6, N33, 2, 2),\n                          N33, N33, [N33], N33, [N33], false, false),\n    \n    % some tests with an updated base node in the to-be-added nodes:\n    ?expect_exception(nodelist:add_node(nodelist:new_neighborhood(N14), N15, 1, 1),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n\n    ok.\n\n%% @doc Tests adding nodes to a neighborhood structure.\n-spec test_add_nodes(any()) -> ok.\ntest_add_nodes(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N9 = node:new(?PID(1),   ?KEY(9), 1), % N8 updated\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N15 = node:new(?PID(2), ?KEY(15), 1), % N14 updated\n    N20 = node:new(?PID(3), ?KEY(20), 0),\n    N21 = node:new(?PID(3), ?KEY(21), 1), % N20 updated\n    N33 = node:new(?PID(4), ?KEY(33), 0),\n    N42 = node:new(?PID(5), ?KEY(42), 0),\n    N50 = node:new(?PID(6), ?KEY(50), 0),\n    N66 = node:new(?PID(7), ?KEY(66), 0),\n    \n    Neighb1 = nodelist:mk_neighborhood([N8, N14, N50], N33),\n    Neighb2 = nodelist:mk_neighborhood([N8, N14, N50], N33, 3, 3),\n    Neighb3 = nodelist:mk_neighborhood([N8, N14, N50], N33, 2, 2),\n    Neighb4 = nodelist:mk_neighborhood([N8, N14, N50], N33, 1, 1),\n    Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n    \n    % Neighb1 = nodelist:mk_neighborhood([N8, N14, N50], N33),\n    ?equals(nodelist:add_nodes(Neighb1, [], 3, 3), Neighb1),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N66], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N42, N20, N66], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N66, N20, N42], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N66, N42], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N20, N42, N66, N42], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N66, N9], 3, 3),\n                          N33, N20, [N20, N14, N9], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N66, N21], 3, 3),\n                          N33, N21, [N21, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20], 3, 3),\n                          N33, N20, [N20, N14, N8], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N42], 3, 3),\n                          N33, N14, [N14, N8, N50], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N66], 3, 3),\n                          N33, N14, [N14, N8, N66], N50, [N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N9], 3, 3),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N33], 3, 3),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    \n    ?equals(nodelist:add_nodes(Neighb1, [], 4, 4), Neighb1),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N66], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N42, N20, N66], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N66, N20, N42], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N66, N42], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N20, N42, N66, N42], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N66, N9], 4, 4),\n                          N33, N20, [N20, N14, N9, N66], N42, [N42, N50, N66, N9], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N66, N21], 4, 4),\n                          N33, N21, [N21, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20], 4, 4),\n                          N33, N20, [N20, N14, N8, N50], N50, [N50, N8, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N42], 4, 4),\n                          N33, N14, [N14, N8, N50, N42], N42, [N42, N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N66], 4, 4),\n                          N33, N14, [N14, N8, N66, N50], N50, [N50, N66, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N9], 4, 4),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N33], 4, 4),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    \n    ?equals(nodelist:add_nodes(Neighb1, [], 2, 2), nodelist:trunc(Neighb1, 2, 2)),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N42, N20, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N66, N20, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N66, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N20, N42, N66, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N66, N9], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N66, N21], 2, 2),\n                          N33, N21, [N21, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20], 2, 2),\n                          N33, N20, [N20, N14], N50, [N50, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N42], 2, 2),\n                          N33, N14, [N14, N8], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N66], 2, 2),\n                          N33, N14, [N14, N8], N50, [N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N9], 2, 2),\n                          N33, N14, [N14, N9], N50, [N50, N9], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N33], 2, 2),\n                          N33, N14, [N14, N8], N50, [N50, N8], true, true),\n    \n    % Neighb2 = nodelist:mk_neighborhood([N8, N14, N50], N33, 3, 3),\n    ?equals(nodelist:add_nodes(Neighb2, [], 3, 3), Neighb2),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N66], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N42, N20, N66], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N66, N20, N42], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N66, N42], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N20, N42, N66, N42], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N66, N9], 3, 3),\n                          N33, N20, [N20, N14, N9], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N66, N21], 3, 3),\n                          N33, N21, [N21, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20], 3, 3),\n                          N33, N20, [N20, N14, N8], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N42], 3, 3),\n                          N33, N14, [N14, N8, N50], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N66], 3, 3),\n                          N33, N14, [N14, N8, N66], N50, [N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N9], 3, 3),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N33], 3, 3),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    \n    ?equals(nodelist:add_nodes(Neighb2, [], 4, 4), Neighb2),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N66], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N42, N20, N66], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N66, N20, N42], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N66, N42], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N20, N42, N66, N42], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N66, N9], 4, 4),\n                          N33, N20, [N20, N14, N9, N66], N42, [N42, N50, N66, N9], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N66, N21], 4, 4),\n                          N33, N21, [N21, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20], 4, 4),\n                          N33, N20, [N20, N14, N8, N50], N50, [N50, N8, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N42], 4, 4),\n                          N33, N14, [N14, N8, N50, N42], N42, [N42, N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N66], 4, 4),\n                          N33, N14, [N14, N8, N66, N50], N50, [N50, N66, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N9], 4, 4),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N33], 4, 4),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    \n    ?equals(nodelist:add_nodes(Neighb2, [], 2, 2), nodelist:trunc(Neighb2, 2, 2)),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N42, N20, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N66, N20, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N66, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N20, N42, N66, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N66, N9], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N66, N21], 2, 2),\n                          N33, N21, [N21, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20], 2, 2),\n                          N33, N20, [N20, N14], N50, [N50, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N42], 2, 2),\n                          N33, N14, [N14, N8], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N66], 2, 2),\n                          N33, N14, [N14, N8], N50, [N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N9], 2, 2),\n                          N33, N14, [N14, N9], N50, [N50, N9], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N33], 2, 2),\n                          N33, N14, [N14, N8], N50, [N50, N8], true, true),\n\n    % Neighb3 = nodelist:mk_neighborhood([N8, N14, N50], N33, 2, 2),\n    ?equals(nodelist:add_nodes(Neighb3, [], 3, 3), Neighb3),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N66], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N42, N20, N66], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N66, N20, N42], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N66, N42], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N20, N42, N66, N42], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N66, N9], 3, 3),\n                          N33, N20, [N20, N14, N9], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N66, N21], 3, 3),\n                          N33, N21, [N21, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20], 3, 3),\n                          N33, N20, [N20, N14, N8], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N42], 3, 3),\n                          N33, N14, [N14, N8, N50], N42, [N42, N50, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N66], 3, 3),\n                          N33, N14, [N14, N8, N66], N50, [N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N9], 3, 3),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N33], 3, 3),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    \n    ?equals(nodelist:add_nodes(Neighb3, [], 4, 4), Neighb3),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N66], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N42, N20, N66], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N66, N20, N42], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N66, N42], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N20, N42, N66, N42], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N66, N9], 4, 4),\n                          N33, N20, [N20, N14, N9, N66], N42, [N42, N50, N66, N9], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N66, N21], 4, 4),\n                          N33, N21, [N21, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20], 4, 4),\n                          N33, N20, [N20, N14, N8, N50], N50, [N50, N8, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N42], 4, 4),\n                          N33, N14, [N14, N8, N50, N42], N42, [N42, N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N66], 4, 4),\n                          N33, N14, [N14, N8, N66, N50], N50, [N50, N66, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N9], 4, 4),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N33], 4, 4),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    \n    ?equals(nodelist:add_nodes(Neighb3, [], 2, 2), Neighb3),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N42, N20, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N66, N20, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N66, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N20, N42, N66, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N66, N9], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N66, N21], 2, 2),\n                          N33, N21, [N21, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20], 2, 2),\n                          N33, N20, [N20, N14], N50, [N50, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N42], 2, 2),\n                          N33, N14, [N14, N8], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N66], 2, 2),\n                          N33, N14, [N14, N8], N50, [N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N9], 2, 2),\n                          N33, N14, [N14, N9], N50, [N50, N9], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N33], 2, 2),\n                          N33, N14, [N14, N8], N50, [N50, N8], true, true),\n\n    % Neighb4 = nodelist:mk_neighborhood([N8, N14, N50], N33, 1, 1),\n    ?equals(nodelist:add_nodes(Neighb4, [], 3, 3), Neighb4),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N66], 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N42, N20, N66], 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N66, N20, N42], 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N66, N42], 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N20, N42, N66, N42], 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N66, N9], 3, 3),\n                          N33, N20, [N20, N14, N9], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N66, N21], 3, 3),\n                          N33, N21, [N21, N14, N66], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20], 3, 3),\n                          N33, N20, [N20, N14, N50], N50, [N50, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N42], 3, 3),\n                          N33, N14, [N14, N50, N42], N42, [N42, N50, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N66], 3, 3),\n                          N33, N14, [N14, N66, N50], N50, [N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N9], 3, 3),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N33], 3, 3),\n                          N33, N14, [N14, N50], N50, [N50, N14], true, true),\n    \n    ?equals(nodelist:add_nodes(Neighb4, [], 4, 4), Neighb4),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N66], 4, 4),\n                          N33, N20, [N20, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N42, N20, N66], 4, 4),\n                          N33, N20, [N20, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N66, N20, N42], 4, 4),\n                          N33, N20, [N20, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N66, N42], 4, 4),\n                          N33, N20, [N20, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N20, N42, N66, N42], 4, 4),\n                          N33, N20, [N20, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N66, N9], 4, 4),\n                          N33, N20, [N20, N14, N9, N66], N42, [N42, N50, N66, N9], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N66, N21], 4, 4),\n                          N33, N21, [N21, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20], 4, 4),\n                          N33, N20, [N20, N14, N50], N50, [N50, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N42], 4, 4),\n                          N33, N14, [N14, N50, N42], N42, [N42, N50, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N66], 4, 4),\n                          N33, N14, [N14, N66, N50], N50, [N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N9], 4, 4),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N33], 4, 4),\n                          N33, N14, [N14, N50], N50, [N50, N14], true, true),\n    \n    ?equals(nodelist:add_nodes(Neighb4, [], 2, 2), Neighb4),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N42, N20, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N66, N20, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N66, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N20, N42, N66, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N66, N9], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N66, N21], 2, 2),\n                          N33, N21, [N21, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20], 2, 2),\n                          N33, N20, [N20, N14], N50, [N50, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N42], 2, 2),\n                          N33, N14, [N14, N50], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N66], 2, 2),\n                          N33, N14, [N14, N66], N50, [N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N9], 2, 2),\n                          N33, N14, [N14, N9], N50, [N50, N9], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N33], 2, 2),\n                          N33, N14, [N14, N50], N50, [N50, N14], true, true),\n    \n    % tests with (almost) empty successor/predecessor lists:\n    % Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    ?equals(nodelist:add_nodes(Neighb5, [], 3, 3), Neighb5),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N66], 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N42, N20, N66], 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N66, N20, N42], 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N66, N42], 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N20, N42, N66, N42], 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N66, N9], 3, 3),\n                          N33, N20, [N20, N14, N9], N42, [N42, N66, N9], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N66, N21], 3, 3),\n                          N33, N21, [N21, N14, N66], N42, [N42, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20], 3, 3),\n                          N33, N20, [N20, N14], N14, [N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N42], 3, 3),\n                          N33, N14, [N14, N42], N42, [N42, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N66], 3, 3),\n                          N33, N14, [N14, N66], N66, [N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N9], 3, 3),\n                          N33, N14, [N14, N9], N9, [N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N33], 3, 3),\n                          N33, N14, [N14], N14, [N14], true, true),\n    \n    ?equals(nodelist:add_nodes(Neighb5, [], 4, 4), Neighb5),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N66], 4, 4),\n                          N33, N20, [N20, N14, N66, N42], N42, [N42, N66, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N42, N20, N66], 4, 4),\n                          N33, N20, [N20, N14, N66, N42], N42, [N42, N66, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N66, N20, N42], 4, 4),\n                          N33, N20, [N20, N14, N66, N42], N42, [N42, N66, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N66, N42], 4, 4),\n                          N33, N20, [N20, N14, N66, N42], N42, [N42, N66, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N20, N42, N66, N42], 4, 4),\n                          N33, N20, [N20, N14, N66, N42], N42, [N42, N66, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N66, N9], 4, 4),\n                          N33, N20, [N20, N14, N9, N66], N42, [N42, N66, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N66, N21], 4, 4),\n                          N33, N21, [N21, N14, N66, N42], N42, [N42, N66, N14, N21], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20], 4, 4),\n                          N33, N20, [N20, N14], N14, [N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N42], 4, 4),\n                          N33, N14, [N14, N42], N42, [N42, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N66], 4, 4),\n                          N33, N14, [N14, N66], N66, [N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N9], 4, 4),\n                          N33, N14, [N14, N9], N9, [N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N33], 4, 4),\n                          N33, N14, [N14], N14, [N14], true, true),\n    \n    ?equals(nodelist:add_nodes(Neighb5, [], 2, 2), Neighb5),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N42, N20, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N66, N20, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N66, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N20, N42, N66, N42], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N66, N9], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N66, N21], 2, 2),\n                          N33, N21, [N21, N14], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20], 2, 2),\n                          N33, N20, [N20, N14], N14, [N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N42], 2, 2),\n                          N33, N14, [N14, N42], N42, [N42, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N66], 2, 2),\n                          N33, N14, [N14, N66], N66, [N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N9], 2, 2),\n                          N33, N14, [N14, N9], N9, [N9, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N33], 2, 2),\n                          N33, N14, [N14], N14, [N14], true, true),\n    \n%%     Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n    ?equals(nodelist:add_nodes(Neighb6, [], 3, 3), Neighb6),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N66], 3, 3),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N42, N20, N66], 3, 3),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N66, N20, N42], 3, 3),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N66, N42], 3, 3),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N20, N42, N66, N42], 3, 3),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N66, N9], 3, 3),\n                          N33, N20, [N20, N9, N66], N42, [N42, N66, N9], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N66, N21], 3, 3),\n                          N33, N21, [N21, N66, N42], N42, [N42, N66, N21], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20], 3, 3),\n                          N33, N20, [N20], N20, [N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N42], 3, 3),\n                          N33, N42, [N42], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N66], 3, 3),\n                          N33, N66, [N66], N66, [N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N9], 3, 3),\n                          N33, N9, [N9], N9, [N9], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N33], 3, 3),\n                          N33, N33, [N33], N33, [N33], false, false),\n    \n    ?equals(nodelist:add_nodes(Neighb6, [], 4, 4), Neighb6),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N66], 4, 4),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N42, N20, N66], 4, 4),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N66, N20, N42], 4, 4),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N66, N42], 4, 4),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N20, N42, N66, N42], 4, 4),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N66, N9], 4, 4),\n                          N33, N20, [N20, N9, N66, N42], N42, [N42, N66, N9, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N66, N21], 4, 4),\n                          N33, N21, [N21, N66, N42], N42, [N42, N66, N21], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20], 4, 4),\n                          N33, N20, [N20], N20, [N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N42], 4, 4),\n                          N33, N42, [N42], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N66], 4, 4),\n                          N33, N66, [N66], N66, [N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N9], 4, 4),\n                          N33, N9, [N9], N9, [N9], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N33], 4, 4),\n                          N33, N33, [N33], N33, [N33], false, false),\n    \n    ?equals(nodelist:add_nodes(Neighb6, [], 2, 2), Neighb6),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N66], 2, 2),\n                          N33, N20, [N20, N66], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N42, N20, N66], 2, 2),\n                          N33, N20, [N20, N66], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N66, N20, N42], 2, 2),\n                          N33, N20, [N20, N66], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N66, N42], 2, 2),\n                          N33, N20, [N20, N66], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N20, N42, N66, N42], 2, 2),\n                          N33, N20, [N20, N66], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N66, N9], 2, 2),\n                          N33, N20, [N20, N9], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N66, N21], 2, 2),\n                          N33, N21, [N21, N66], N42, [N42, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20], 2, 2),\n                          N33, N20, [N20], N20, [N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N42], 2, 2),\n                          N33, N42, [N42], N42, [N42], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N66], 2, 2),\n                          N33, N66, [N66], N66, [N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N9], 2, 2),\n                          N33, N9, [N9], N9, [N9], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N33], 2, 2),\n                          N33, N33, [N33], N33, [N33], false, false),\n    \n    % some tests with the base node in the to-be-added nodes:\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N33, N66], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N20, N42, N66, N33], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb1, [N33, N20, N42, N66], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N33, N66], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N20, N42, N66, N33], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb2, [N33, N20, N42, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N33, N66], 3, 3),\n                          N33, N20, [N20, N14, N8], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N20, N42, N66, N33], 4, 4),\n                          N33, N20, [N20, N14, N8, N66], N42, [N42, N50, N66, N8], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb3, [N33, N20, N42, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N33, N66], 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N50, N66], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N20, N42, N66, N33], 4, 4),\n                          N33, N20, [N20, N14, N66, N50], N42, [N42, N50, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb4, [N33, N20, N42, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N50], true, true),\n\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N33, N66], 3, 3),\n                          N33, N20, [N20, N14, N66], N42, [N42, N66, N14], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N20, N42, N66, N33], 4, 4),\n                          N33, N20, [N20, N14, N66, N42], N42, [N42, N66, N14, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb5, [N33, N20, N42, N66], 2, 2),\n                          N33, N20, [N20, N14], N42, [N42, N66], true, true),\n\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N33, N66], 3, 3),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N20, N42, N66, N33], 4, 4),\n                          N33, N20, [N20, N66, N42], N42, [N42, N66, N20], true, true),\n    ?compare_neighborhood(nodelist:add_nodes(Neighb6, [N33, N20, N42, N66], 2, 2),\n                          N33, N20, [N20, N66], N42, [N42, N66], true, true),\n    \n    % some tests with an updated base node in the to-be-added nodes:\n    ?expect_exception(nodelist:add_nodes(nodelist:new_neighborhood(N14), [N15], 1, 1),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:add_nodes(nodelist:new_neighborhood(N14), [N8, N15], 1, 1),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:add_nodes(nodelist:new_neighborhood(N14), [N15, N8], 1, 1),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n\n    ok.\n\n%% @doc Tests adding nodes to a neighborhood structure.\n-spec test_update_ids(any()) -> ok.\ntest_update_ids(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N9 = node:new(?PID(1),   ?KEY(9), 1), % N8 updated\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N15 = node:new(?PID(2), ?KEY(15), 1), % N14 updated\n    N20 = node:new(?PID(3), ?KEY(20), 0),\n    N33 = node:new(?PID(4), ?KEY(33), 0),\n    N42 = node:new(?PID(5), ?KEY(42), 0),\n    N50 = node:new(?PID(6), ?KEY(50), 0),\n    N66 = node:new(?PID(7), ?KEY(66), 0),\n    \n    Neighb1 = nodelist:mk_neighborhood([N8, N14, N50], N33),\n    Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n    \n    % Neighb1 = nodelist:mk_neighborhood([N8, N14, N50], N33),\n    ?equals(nodelist:add_nodes(Neighb1, [], 3, 3), Neighb1),\n    ?compare_neighborhood(nodelist:update_ids(Neighb1, [N20, N42, N66]),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb1, [N20, N42, N66, N42]),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb1, [N20, N42, N66, N9]),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb1, [N20, N9, N66, N42]),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb1, [N14]),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb1, [N20]),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb1, [N9]),\n                          N33, N14, [N14, N9, N50], N50, [N50, N9, N14], true, true),\n    \n    % tests with (almost) empty successor/predecessor lists:\n    % Neighb5 = nodelist:mk_neighborhood([N14], N33, 1, 1),\n    ?equals(nodelist:update_ids(Neighb5, []), Neighb5),\n    ?compare_neighborhood(nodelist:update_ids(Neighb5, [N20, N42, N66]),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb5, [N20, N42, N66, N42]),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb5, [N20, N42, N66, N15]),\n                          N33, N15, [N15], N15, [N15], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb5, [N20, N15, N66, N42]),\n                          N33, N15, [N15], N15, [N15], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb5, [N20]),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb5, [N14]),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb5, [N15]),\n                          N33, N15, [N15], N15, [N15], true, true),\n    \n%%     Neighb6 = nodelist:mk_neighborhood([], N33, 1, 1),\n    ?equals(nodelist:update_ids(Neighb6, []), Neighb6),\n    ?compare_neighborhood(nodelist:update_ids(Neighb6, [N20, N42, N66]),\n                          N33, N33, [N33], N33, [N33], false, false),\n    ?compare_neighborhood(nodelist:update_ids(Neighb6, [N20, N42, N66, N42]),\n                          N33, N33, [N33], N33, [N33], false, false),\n    ?compare_neighborhood(nodelist:update_ids(Neighb6, [N20, N42, N66, N33]),\n                          N33, N33, [N33], N33, [N33], false, false),\n    ?compare_neighborhood(nodelist:update_ids(Neighb6, [N20]),\n                          N33, N33, [N33], N33, [N33], false, false),\n    \n    % some tests with the base node in the to-be-updated nodes:\n    ?compare_neighborhood(nodelist:update_ids(Neighb1, [N20, N42, N33, N66]),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb1, [N20, N42, N66, N33]),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb1, [N33, N20, N42, N66]),\n                          N33, N14, [N14, N8, N50], N50, [N50, N8, N14], true, true),\n\n    ?compare_neighborhood(nodelist:update_ids(Neighb5, [N20, N42, N33, N66]),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb5, [N20, N42, N66, N33]),\n                          N33, N14, [N14], N14, [N14], true, true),\n    ?compare_neighborhood(nodelist:update_ids(Neighb5, [N33, N20, N42, N66]),\n                          N33, N14, [N14], N14, [N14], true, true),\n\n    ?compare_neighborhood(nodelist:update_ids(Neighb6, [N20, N42, N33, N66]),\n                          N33, N33, [N33], N33, [N33], false, false),\n    ?compare_neighborhood(nodelist:update_ids(Neighb6, [N20, N42, N66, N33]),\n                          N33, N33, [N33], N33, [N33], false, false),\n    ?compare_neighborhood(nodelist:update_ids(Neighb6, [N33, N20, N42, N66]),\n                          N33, N33, [N33], N33, [N33], false, false),\n    \n    % some tests with an updated base node in the to-be-added nodes:\n    ?expect_exception(nodelist:update_ids(nodelist:new_neighborhood(N14), [N15]),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:update_ids(nodelist:new_neighborhood(N14), [N8, N15]),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n    ?expect_exception(nodelist:update_ids(nodelist:new_neighborhood(N14), [N15, N8]),\n                      throw, 'cannot create a neighborhood() with a neighbor newer than the node itself'),\n\n    ok.\n\n%% @doc Tests converting neighborhood structures to node lists.\n-spec test_to_list(any()) -> ok.\ntest_to_list(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N20 = node:new(?PID(3), ?KEY(20), 0),\n    N33 = node:new(?PID(4), ?KEY(33), 0),\n    N50 = node:new(?PID(5), ?KEY(50), 0),\n    \n    Neighb1 = nodelist:mk_neighborhood([N8, N14, N33, N50], N20),\n    Neighb2 = nodelist:mk_neighborhood([N8, N14, N33, N50], N20, 3, 3),\n    Neighb3 = nodelist:mk_neighborhood([N8, N14, N33, N50], N20, 2, 2),\n    Neighb4 = nodelist:mk_neighborhood([N8, N14, N33, N50], N20, 1, 1),\n    Neighb5 = nodelist:mk_neighborhood([N8, N14, N33, N50], N20, 3, 1),\n    Neighb6 = nodelist:mk_neighborhood([N8, N14, N33, N50], N20, 3, 2),\n    Neighb7 = nodelist:mk_neighborhood([N8, N14, N33, N50], N20, 1, 3),\n    Neighb8 = nodelist:mk_neighborhood([N8, N14, N33, N50], N20, 2, 3),\n    Neighb9 = nodelist:mk_neighborhood([N8, N14, N33, N50], N20, 2, 1),\n    Neighb10 = nodelist:mk_neighborhood([N8, N14, N33, N50], N20, 1, 2),\n    \n    ?equals(nodelist:to_list(Neighb1), [N20, N33, N50, N8, N14]),\n    ?equals(nodelist:to_list(Neighb2), [N20, N33, N50, N8, N14]),\n    ?equals(nodelist:to_list(Neighb3), [N20, N33, N50, N8, N14]),\n    ?equals(nodelist:to_list(Neighb4), [N20, N33, N14]),\n    ?equals(nodelist:to_list(Neighb5), [N20, N33, N50, N8, N14]),\n    ?equals(nodelist:to_list(Neighb6), [N20, N33, N50, N8, N14]),\n    ?equals(nodelist:to_list(Neighb7), [N20, N33, N50, N8, N14]),\n    ?equals(nodelist:to_list(Neighb8), [N20, N33, N50, N8, N14]),\n    ?equals(nodelist:to_list(Neighb9), [N20, N33, N8, N14]),\n    ?equals(nodelist:to_list(Neighb10), [N20, N33, N50, N14]),\n    \n    ?equals(_N11 = nodelist:to_list(nodelist:new_neighborhood(N8)), [N8]),\n    ?equals(_N12 = nodelist:to_list(nodelist:new_neighborhood(N8, N14)), [N8, N14]),\n    ?equals(_N13 = nodelist:to_list(nodelist:new_neighborhood(N20, N8, N14)), [N8, N14, N20]),\n    \n    ok.\n\n%% @doc Tests updating node IDs in two node lists.\n-spec test_lupdate_ids(any()) -> ok.\ntest_lupdate_ids(_Config) ->\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N9 = node:new(?PID(1),   ?KEY(9), 1), %N8 updated\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N20 = node:new(?PID(3), ?KEY(20), 0),\n    N21 = node:new(?PID(3), ?KEY(21), 1), %N20 updated\n    N33 = node:new(?PID(4), ?KEY(33), 0),\n    N50 = node:new(?PID(5), ?KEY(50), 0),\n\n    ?equals(nodelist:lupdate_ids([N8, N14, N20, N33, N50], []), {[N8, N14, N20, N33, N50], []}),\n    ?equals(nodelist:lupdate_ids([N8, N9, N14, N20, N33, N50], []), {[N9, N9, N14, N20, N33, N50], []}),\n    ?equals(nodelist:lupdate_ids([N8, N9, N14, N20, N21, N33, N50], []), {[N9, N9, N14, N21, N21, N33, N50], []}),\n    ?equals(nodelist:lupdate_ids([N8, N14, N20, N33], [N50]), {[N8, N14, N20, N33], [N50]}),\n    ?equals(nodelist:lupdate_ids([N8, N14, N20], [N33, N50]), {[N8, N14, N20], [N33, N50]}),\n    ?equals(nodelist:lupdate_ids([N8, N14], [N20, N33, N50]), {[N8, N14], [N20, N33, N50]}),\n    ?equals(nodelist:lupdate_ids([N8], [N14, N20, N33, N50]), {[N8], [N14, N20, N33, N50]}),\n    ?equals(nodelist:lupdate_ids([], [N8, N14, N20, N33, N50]), {[], [N8, N14, N20, N33, N50]}),\n    ?equals(nodelist:lupdate_ids([], [N8, N9, N14, N20, N33, N50]), {[], [N9, N9, N14, N20, N33, N50]}),\n    ?equals(nodelist:lupdate_ids([], [N8, N9, N14, N20, N21, N33, N50]), {[], [N9, N9, N14, N21, N21, N33, N50]}),\n    \n    ?equals(nodelist:lupdate_ids([N8, N14, N20, N33, N50], [N9]), {[N9, N14, N20, N33, N50], [N9]}),\n    ?equals(nodelist:lupdate_ids([N8, N14, N20, N33, N50], [N8]), {[N8, N14, N20, N33, N50], [N8]}),\n    ?equals(nodelist:lupdate_ids([N8, N14, N20, N33, N50], [N9, N33]), {[N9, N14, N20, N33, N50], [N9, N33]}),\n    ?equals(nodelist:lupdate_ids([N8, N14, N20, N33, N50], [N8, N33]), {[N8, N14, N20, N33, N50], [N8, N33]}),\n    ?equals(nodelist:lupdate_ids([N8, N14, N20, N33, N50], [N8, N14, N20, N33, N50]), {[N8, N14, N20, N33, N50], [N8, N14, N20, N33, N50]}),\n    ?equals(nodelist:lupdate_ids([N8, N14, N21, N33, N50], [N9, N14, N20, N33, N50]), {[N9, N14, N21, N33, N50], [N9, N14, N21, N33, N50]}),\n    ?equals(nodelist:lupdate_ids([N14, N21, N33, N50], [N9, N14, N20, N33]), {[N14, N21, N33, N50], [N9, N14, N21, N33]}),\n    ?equals(nodelist:lupdate_ids([N14, N21, N33, N50], [N9, N14, N20, N33]), {[N14, N21, N33, N50], [N9, N14, N21, N33]}),\n    \n    ?equals(nodelist:lupdate_ids([N14, N20, N33, N50], [N9, N14, N20, N21, N33]), {[N14, N21, N33, N50], [N9, N14, N21, N21, N33]}),\n    \n    % some tests with \"unsorted\" lists:\n    ?equals(nodelist:lupdate_ids([N50, N14, N21, N8, N33], [N33, N50, N9, N14, N20]), {[N50, N14, N21, N9, N33], [N33, N50, N9, N14, N21]}),\n    ?equals(nodelist:lupdate_ids([N14, N33, N50, N21], [N9, N14, N20, N33]), {[N14, N33, N50, N21], [N9, N14, N21, N33]}),\n    ?equals(nodelist:lupdate_ids([N14, N21, N33, N50], [N20, N9, N14, N33]), {[N14, N21, N33, N50], [N21, N9, N14, N33]}),\n    ?equals(nodelist:lupdate_ids([N33, N14, N20, N50], [N9, N14, N20, N21, N33]), {[N33, N14, N21, N50], [N9, N14, N21, N21, N33]}),\n    ok.\n\n%% @doc Tests making removing outdated nodes from a node list.\n-spec test_lremove_outdated(any()) -> ok.\ntest_lremove_outdated(_Config) ->\n    Null = node:null(),\n    N8 = node:new(?PID(1),   ?KEY(8), 0),\n    N9 = node:new(?PID(1),   ?KEY(9), 1), %N8 updated\n    N14 = node:new(?PID(2), ?KEY(14), 0),\n    N20 = node:new(?PID(3), ?KEY(20), 0),\n    N21 = node:new(?PID(3), ?KEY(21), 1), %N20 updated\n    N33 = node:new(?PID(4), ?KEY(33), 0),\n    N50 = node:new(?PID(5), ?KEY(50), 0),\n\n    ?equals(nodelist:lremove_outdated([]), []),\n    ?equals(nodelist:lremove_outdated([N8]), [N8]),\n    ?equals(nodelist:lremove_outdated([N8, N14, N20, N33, N50]), [N8, N14, N20, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N9]), [N9]),\n    ?equals(nodelist:lremove_outdated([N8, N9, N14, N20, N33, N50]), [N9, N14, N20, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N9, N14, N20, N21, N33, N50]), [N9, N14, N21, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N9, N14, N20, N8, N21, N33, N50]), [N9, N14, N21, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N9, N14, N20, N21, N33, N50, N50]), [N9, N14, N21, N33, N50, N50]),\n\n    ?equals(nodelist:lremove_outdated([], N50), []),\n    ?equals(nodelist:lremove_outdated([N8], N50), [N8]),\n    ?equals(nodelist:lremove_outdated([N8], N9), []),\n    ?equals(nodelist:lremove_outdated([N8], N50), [N8]),\n    ?equals(nodelist:lremove_outdated([N8, N14, N20, N33], N50), [N8, N14, N20, N33]),\n    ?equals(nodelist:lremove_outdated([N8, N14, N20, N33, N50], N50), [N8, N14, N20, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N14, N20, N33, N50], Null), [N8, N14, N20, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N14, N20, N33, N50], N21), [N8, N14, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N14, N20, N21, N33, N50], N21), [N8, N14, N21, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N9], N50), [N9]),\n    ?equals(nodelist:lremove_outdated([N8, N9, N14, N20, N33, N50], N50), [N9, N14, N20, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N9, N14, N20, N21, N33, N50], N50), [N9, N14, N21, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N9, N14, N20, N8, N21, N33, N50], N50), [N9, N14, N21, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N9, N14, N20, N21, N33, N33, N50, N50], N50), [N9, N14, N21, N33, N33, N50, N50]),\n    \n    % some tests with \"unsorted\" lists:\n    ?equals(nodelist:lremove_outdated([N14, N8, N20, N33, N50]), [N14, N8, N20, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N9, N8]), [N9]),\n    ?equals(nodelist:lremove_outdated([N8, N20, N33, N50, N9, N14]), [N20, N33, N50, N9, N14]),\n    ?equals(nodelist:lremove_outdated([N9, N14, N20, N8, N21, N33, N50]), [N9, N14, N21, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N9, N14, N20, N8, N21, N33, N50]), [N9, N14, N21, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N9, N50, N14, N20, N8, N21, N33, N50]), [N9, N50, N14, N21, N33, N50]),\n    \n    ?equals(nodelist:lremove_outdated([N14, N8, N20, N33, N50], N50), [N14, N8, N20, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N20, N33, N14, N50], Null), [N8, N20, N33, N14, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N20, N33, N14, N50], N21), [N8, N33, N14, N50]),\n    ?equals(nodelist:lremove_outdated([N8, N20, N33, N14, N21, N50], N21), [N8, N33, N14, N21, N50]),\n    ?equals(nodelist:lremove_outdated([N9, N8], N50), [N9]),\n    ?equals(nodelist:lremove_outdated([N8, N20, N33, N50, N9, N14], N50), [N20, N33, N50, N9, N14]),\n    ?equals(nodelist:lremove_outdated([N9, N14, N20, N8, N21, N33, N50], N50), [N9, N14, N21, N33, N50]),\n    ?equals(nodelist:lremove_outdated([N9, N14, N20, N8, N21, N33], N50), [N9, N14, N21, N33]),\n    ?equals(nodelist:lremove_outdated([N9, N14, N20, N8, N21, N8, N33], N50), [N9, N14, N21, N33]),\n    ?equals(nodelist:lremove_outdated([N9, N14, N20, N8, N21, N33], N50), [N9, N14, N21, N33]),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% largest_smaller_than/2 and largest_smaller_than/3\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_largest_smaller_than2(BaseNode::node:node_type(), AddNodes::[node:node_type()], PredsLength::pos_integer(), SuccsLength::pos_integer(), Id::?RT:key()) -> true.\nprop_largest_smaller_than2(BaseNode, AddNodes_, PredsLength, SuccsLength, Id) ->\n    % remove invalid and duplicate nodes from the list of nodes to add:\n    AddNodes_2 = lists:usort(fun(A, B) -> node:pidX(A) =< node:pidX(B) end, AddNodes_),\n    AddNodes = [Node || Node <- AddNodes_2,\n                        not node:same_process(Node, BaseNode),\n                        node:id(Node) =/= node:id(BaseNode)],\n        \n    prop_largest_smaller_than2_fixed_types(\n      BaseNode, AddNodes, PredsLength, SuccsLength, Id).\n\n-spec prop_largest_smaller_than2_fixed_types(BaseNode::node:node_type(), AddNodes::[node:node_type()], PredsLength::pos_integer(), SuccsLength::pos_integer(), Id::?RT:key()) -> true.\nprop_largest_smaller_than2_fixed_types(BaseNode, AddNodes, PredsLength, SuccsLength, Id) ->\n    % build new neighborhood\n    Neighbors1 = nodelist:new_neighborhood(BaseNode),\n    Neighbors2 = nodelist:add_nodes(Neighbors1, AddNodes, PredsLength, SuccsLength),\n    \n    LSTNode = nodelist:largest_smaller_than(Neighbors2, Id),\n%%     ct:pal(\"nodelist:largest_smaller_than(~p, ~p)~n\", [Neighbors2, Id]),\n    case node:id(BaseNode) =:= Id andalso not nodelist:has_real_pred(Neighbors2) of\n        true ->\n            % in this case no node can be found\n            ?equals_w_note(LSTNode, node:null(), \"empty_neighbors\");\n        _ ->\n            ?assert(node:is_valid(LSTNode)),\n            ?assert(node:id(LSTNode) =/= Id),\n            BetterNodes =\n                [Node || Node <- [BaseNode | nodelist:to_list(Neighbors2)],\n                         intervals:in(node:id(Node),\n                                      intervals:new('(', node:id(LSTNode), Id, ')'))],\n            ?equals([LSTNode | BetterNodes], [LSTNode])\n    end.\n\ntester_largest_smaller_than2(_Config) ->\n    % some tests that previously failed:\n    prop_largest_smaller_than2(\n      {node, {{{197,151,185,165},1,{one,'ct@csr-pc40'}}, c,{}}, ?KEY(1), 0, 8000},\n      [{node, {{{197,151,185,165},1,{one,'ct@csr-pc40'}}, c,{}}, ?KEY(3), 5, 8000},\n       {node, {{{39,220,200,75},3,{two,'ct@csr-pc40'}}, c,101.41898888632392}, ?KEY(4), 0, 8000},\n       {node, {{10,87,88,118},1,{three,'ct@csr-pc40'}}, ?KEY(5), 4, 8000}],\n      1, 1, ?KEY(5)),\n    \n    prop_largest_smaller_than2(\n      {node,\n       {{6165,1898,39314,61608,61439,21180,27136,38528}, 4, {two,'ct@csr-pc40'}},\n       ?KEY(0), 4, 8000},\n      [{node,\n        {{{38651,60988,24625,21004,23234,64811,37455,60050}, 5, {one,'ct@csr-pc40'}}, c,101.41898888632392},\n        ?KEY(4), 1, 8000},\n       {node,\n        {{{12076,15918,45197,41953,9328,37838,58170,26800}, 4,one}, c,42},\n        ?KEY(5), 0, 8000},\n       {node, {{{25,8,53,113}, 5, {four,'ct@csr-pc40'}}, c, {3}}, ?KEY(2), 4, 8000}],\n      1, 1, ?KEY(5)),\n    \n    tester:test(?MODULE, prop_largest_smaller_than2, 5, 5000, [{threads, 2}]).\n\n-spec prop_largest_smaller_than3(BaseNode::node:node_type(), AddNodes::[node:node_type()], PredsLength::pos_integer(), SuccsLength::pos_integer(), Id::?RT:key()) -> true.\nprop_largest_smaller_than3(BaseNode, AddNodes_, PredsLength, SuccsLength, Id) ->\n    % remove invalid and duplicate nodes from the list of nodes to add:\n    AddNodes_2 = lists:usort(fun(A, B) -> node:pidX(A) =< node:pidX(B) end, AddNodes_),\n    AddNodes = [Node || Node <- AddNodes_2,\n                        not node:same_process(Node, BaseNode),\n                        node:id(Node) =/= node:id(BaseNode)],\n        \n    ?implies(AddNodes =/= [],\n             prop_largest_smaller_than3_fixed_types(\n               BaseNode, AddNodes, PredsLength, SuccsLength, Id, util:randomelem(AddNodes))).\n\n-spec prop_largest_smaller_than3_fixed_types(BaseNode::node:node_type(), AddNodes::[node:node_type()], PredsLength::pos_integer(), SuccsLength::pos_integer(), Id::?RT:key(), LastFound::node:node_type()) -> true.\nprop_largest_smaller_than3_fixed_types(BaseNode, AddNodes, PredsLength, SuccsLength, Id, LastFound) ->\n    % build new neighborhood\n    Neighbors1 = nodelist:new_neighborhood(BaseNode),\n    Neighbors2 = nodelist:add_nodes(Neighbors1, AddNodes, PredsLength, SuccsLength),\n    \n    case node:id(LastFound) =:= Id of\n        true -> true;\n        _ ->\n            LSTNode = nodelist:largest_smaller_than(Neighbors2, Id, LastFound),\n%%             ct:pal(\"nodelist:largest_smaller_than(~p, ~p, ~p)~n\", [Neighbors2, Id, LastFound]),\n            case node:id(BaseNode) =:= Id andalso not nodelist:has_real_pred(Neighbors2) of\n                true ->\n                    % in this case no node can be found\n                    ?equals_w_note(LSTNode, node:null(), \"empty_neighbors\");\n                _ ->\n%%                     ?assert(node:is_valid(LSTNode)), % always valid!\n                    ?assert(node:id(LSTNode) =/= Id),\n                    BetterNodes =\n                        [Node || Node <- [BaseNode | nodelist:to_list(Neighbors2)],\n                                 intervals:in(node:id(Node),\n                                              intervals:new('(', node:id(LSTNode), Id, ')'))],\n                    ?equals([LSTNode | BetterNodes], [LSTNode])\n            end\n    end.\n\ntester_largest_smaller_than3(_Config) ->\n    % some tests that previously failed:\n    prop_largest_smaller_than3_fixed_types(\n      {node, proc1, ?KEY(5), 5, 8000},\n      [{node, proc2, ?KEY(0), 2, 8000}, {node, proc3, ?KEY(3), 2, 8000}],\n      1, 3, ?KEY(5), {node, proc2, ?KEY(0), 2, 8000}),\n    \n    prop_largest_smaller_than3_fixed_types(\n        {node, proc1, ?KEY(2), 1, 8000},\n        [{node, proc2, ?KEY(5), 0, 8000}, {node, proc3, ?KEY(4), 4, 8000}],\n        2, 5, ?KEY(3), {node, proc4, ?KEY(5), 0, 8000}),\n    \n    prop_largest_smaller_than3_fixed_types(\n      {node, proc1, ?KEY(2), 1, 8000},\n      [{node, proc2, ?KEY(5), 0, 8000}, {node, proc3, ?KEY(3), 1, 8000}],\n      1, 1, ?KEY(2), {node, proc3, ?KEY(3), 1, 8000}),\n    \n    tester:test(?MODULE, prop_largest_smaller_than3, 5, 5000, [{threads, 2}]).\n\ntest_succ_ord_id(_Config) ->\n    ?assert(nodelist:succ_ord_id(?KEY(5), ?KEY(6), ?KEY(3))),\n    ?assert(nodelist:succ_ord_id(?KEY(6), ?KEY(5), ?KEY(3)) =:= false),\n    ?assert(nodelist:succ_ord_id(?KEY(5), ?KEY(6), ?KEY(7))),\n    ?assert(nodelist:succ_ord_id(?KEY(6), ?KEY(5), ?KEY(7)) =:= false),\n\n    ?assert(nodelist:succ_ord_id(?KEY(5), ?KEY(5), ?KEY(5))),\n    ?assert(nodelist:succ_ord_id(?KEY(5), ?KEY(5), ?KEY(6))),\n    ?assert(nodelist:succ_ord_id(?KEY(5), ?KEY(5), ?KEY(4))),\n\n    ?assert(nodelist:succ_ord_id(?KEY(8), ?KEY(4), ?KEY(5))),\n    ?assert(nodelist:succ_ord_id(?KEY(8), ?MINUS_INFINITY, ?KEY(5))),\n    ?assert(nodelist:succ_ord_id(?KEY(4), ?KEY(8), ?KEY(5)) =:= false),\n    ?assert(nodelist:succ_ord_id(?MINUS_INFINITY, ?KEY(8), ?KEY(5)) =:= false),\n\n    ?assert(nodelist:succ_ord_id(?KEY(4), ?KEY(5), ?MINUS_INFINITY)),\n    ?assert(nodelist:succ_ord_id(?KEY(5), ?KEY(4), ?MINUS_INFINITY) =:= false),\n    ?assert(nodelist:succ_ord_id(?KEY(5), ?MINUS_INFINITY, ?MINUS_INFINITY) =:= false),\n    ?assert(nodelist:succ_ord_id(?MINUS_INFINITY, ?KEY(5), ?MINUS_INFINITY)),\n    ?assert(nodelist:succ_ord_id(?MINUS_INFINITY, ?MINUS_INFINITY, ?MINUS_INFINITY)),\n\n    ?assert(nodelist:succ_ord_id(?KEY(4), ?KEY(8), ?KEY(4))),\n    ?assert(nodelist:succ_ord_id(?KEY(8), ?KEY(4), ?KEY(4)) =:= false).\n"
  },
  {
    "path": "test/non_deterministic_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{suites, tests, [slide_leases_SUITE]}.\n"
  },
  {
    "path": "test/paxos_SUITE.erl",
    "content": "%  @copyright 2008-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for src/paxos/*.erl\n%% @end\n%% @version $Id$\n-module(paxos_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n\nall() -> [\n          test_fast_acceptors_4, test_fast_acceptors_16,\n          test_acceptors_4, test_acceptors_16,\n          test_two_proposers,\n          test_rnd_interleave\n         ].\n\nsuite() ->\n    [{timetrap, {seconds, 40}}].\n\ninit_per_suite(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(2, [{config, [{log_path, PrivDir}]}]),\n    comm_server:set_local_address({127,0,0,1}, unittest_helper:get_scalaris_port()),\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\n-spec make_groupname(Prefix::string(), Number::integer()) -> pid_groups:groupname().\nmake_groupname(Prefix, Number) ->\n    {Prefix, Number}.\n\n%% make proposers, acceptors, and learners\n-spec make(P::pos_integer(), A::pos_integer(), L::pos_integer(), Prefix::atom())\n        -> {Ps::[comm:mypid(),...], As::[comm:mypid(),...], Ls::[comm:mypid(),...]}.\nmake(P, A, L, Prefix) ->\n    NumMDs = lists:max([P,A,L]),\n    _  = [ msg_delay:start_link(make_groupname(Prefix, X))\n             || X <- lists:seq(1, NumMDs)],\n    Ps = [ comm:make_global(element(2, proposer:start_link(make_groupname(Prefix, X), paxos_proposer)))\n             || X <- lists:seq(1, P)],\n    As = [ comm:make_global(element(2, acceptor:start_link(make_groupname(Prefix, X), paxos_acceptor)))\n             || X <- lists:seq(1, A)],\n    Ls = [ comm:make_global(element(2, learner:start_link(make_groupname(Prefix, X), paxos_learner)))\n             || X <- lists:seq(1,L)],\n    {Ps, As, Ls}.\n\n-spec collector(Count::pos_integer(), Owner::pid()) -> pid().\ncollector(Count, Owner) ->\n    spawn(fun() ->\n                  collect(Count),\n                  Owner ! done\n          end).\n\n-spec collect(non_neg_integer()) -> ok.\ncollect(0) ->\n    ok;\ncollect(Count) ->\n    receive\n        _ -> collect(Count - 1)\n%%     after 2000 ->\n%%             ct:pal(\"No further receives at count ~p\", [Count]),\n%%             collect(Count - 1)\n    end.\n\n-spec tester_fast_paxos(CountAcceptors::pos_integer(), Count::pos_integer(), Prefix::atom()) -> ok.\ntester_fast_paxos(CountAcceptors, Count, Prefix) ->\n    %% Count = 10,\n    CountProposers = 1,\n    %% CountAcceptors = 4,\n    Majority = CountAcceptors div 2 + 1,\n    {Proposers, Acceptors, Learners} =\n        make(CountProposers, CountAcceptors, 1, Prefix),\n\n    Collector = comm:make_global(collector(Count, self())),\n\n    _ = [ learner:start_paxosid(hd(Learners), Id, Majority, Collector, chocolate_chip_cookie)\n            || Id <- lists:seq(1, Count)],\n    _ = [ acceptor:start_paxosid(X, Id, Learners)\n            || X <- Acceptors,  Id <- lists:seq(1, Count)],\n    _ = [ proposer:start_paxosid(hd(Proposers), Id, Acceptors, ?prepared,\n                                 Majority, CountProposers, 0)\n            || Id <- lists:seq(1, Count)],\n    receive done -> ok\n    end,\n    _ = [ gen_component:kill(comm:make_local(X))\n          || X <- lists:flatten([Proposers, Acceptors, Learners])],\n    ok.\n\n-spec tester_paxos(CountAcceptors::pos_integer(), Count::pos_integer(), Prefix::atom()) -> ok.\ntester_paxos(CountAcceptors, Count, Prefix) ->\n    CountProposers = 1,\n    Majority = CountAcceptors div 2 + 1,\n    {Proposers, Acceptors, Learners} =\n        make(CountProposers, CountAcceptors, 1, Prefix),\n\n    Collector = comm:make_global(collector(Count, self())),\n\n    _ = spawn(fun() ->\n                      [ learner:start_paxosid(hd(Learners), Id, Majority, Collector, chocolate_chip_cookie)\n                          || Id <- lists:seq(1, Count)]\n              end),\n    _ = spawn(fun() ->\n                      [ acceptor:start_paxosid(X, Id, Learners)\n                          || X <- Acceptors,  Id <- lists:seq(1, Count)]\n           end),\n    _ = spawn(fun() ->\n                      [ proposer:start_paxosid(hd(Proposers), Id, Acceptors,\n                                               ?prepared, Majority, CountProposers)\n                          || Id <- lists:seq(1, Count)]\n              end),\n    receive done -> ok\n    end,\n    _ = [ gen_component:kill(comm:make_local(X))\n          || X <- lists:flatten([Proposers, Acceptors, Learners])],\n    ok.\n\ntest_fast_acceptors_4(_Config) ->\n    Count = 10000,\n    Before = os:timestamp(),\n    tester_fast_paxos(4, Count, test_acceptors_4),\n    After = os:timestamp(),\n    ct:pal(\"fast: acceptors: 4, throughput: ~p~n\", [Count / (erlang:max(1, timer:now_diff(After, Before)) / 1000000.0)]),\n    ok.\n\ntest_fast_acceptors_16(_Config) ->\n    Count = 10000,\n    Before = os:timestamp(),\n    tester_fast_paxos(16, Count, test_acceptors_16),\n    After = os:timestamp(),\n    ct:pal(\"fast: acceptors: 16, throughput: ~p~n\", [Count / (erlang:max(1, timer:now_diff(After, Before)) / 1000000.0)]),\n    ok.\n\ntest_acceptors_4(_Config) ->\n    Count = 10000,\n    Before = os:timestamp(),\n    tester_paxos(4, Count, test_acceptors_4),\n    After = os:timestamp(),\n    ct:pal(\"slow: acceptors: 4, throughput: ~p~n\", [Count / (erlang:max(1, timer:now_diff(After, Before)) / 1000000.0)]),\n    ok.\n\ntest_acceptors_16(_Config) ->\n    Count = 10000,\n    Before = os:timestamp(),\n    tester_paxos(16, Count, test_acceptors_16),\n    After = os:timestamp(),\n    ct:pal(\"slow: acceptors: 16, throughput: ~p~n\", [Count / (erlang:max(1, timer:now_diff(After, Before)) / 1000000.0)]),\n    ok.\n\ntest_two_proposers(_Config) ->\n    ct:pal(\"test_two_proposers ...~n\"),\n    CountProposers = 2,\n    CountAcceptors = 4,\n    Majority = CountAcceptors div 2 + 1,\n    {Proposers, Acceptors, Learners} =\n        make(CountProposers, CountAcceptors, 1, two_proposers),\n\n    %% start paxosids in the components\n    learner:start_paxosid(hd(Learners), paxid123, Majority, comm:this(), cpaxid123),\n    _ = [ acceptor:start_paxosid(X, paxid123, Learners) || X <- Acceptors ],\n    [ Proposer1, Proposer2 ] = Proposers,\n\n    %% set some breakpoints\n    gen_component:bp_set(comm:make_local(Proposer1), acceptor_ack, bp1),\n    %% initiate full paxos\n    proposer:start_paxosid(Proposer1, paxid123, Acceptors,\n                           ?prepared, Majority, length(Proposers), 3),\n    proposer:start_paxosid(Proposer2, paxid123, Acceptors,\n                           ?abort, Majority, length(Proposers), 2),\n\n    %% should receive an abort\n    receive {learner_decide, cpaxid123, _, Res1} = Any -> io:format(\"Expected abort Received ~p~n\", [Any]) end,\n\n    gen_component:bp_barrier(comm:make_local(Proposer1)),\n    gen_component:bp_del(comm:make_local(Proposer1), bp1),\n    gen_component:bp_cont(comm:make_local(Proposer1)),\n\n    %% should receive also an abort as proposer1 was hold\n    receive {learner_decide, cpaxid123, _, Res2} = Any2 ->\n            io:format(\"Expected abort Received ~p~n\", [Any2]) end,\n\n    ?assert(Res1 =:= Res2),\n    %%%%% now vice versa:\n    io:format(\"Now vice versa~n\"),\n\n    %% start paxosids in the components\n    learner:start_paxosid(hd(Learners), paxid124, Majority, comm:this(), cpaxid124),\n    _ = [ acceptor:start_paxosid(X, paxid124, Learners) || X <- Acceptors ],\n    [ Proposer1, Proposer2 ] = Proposers,\n    %% set some breakpoints\n    gen_component:bp_set(comm:make_local(Proposer2), acceptor_ack, bp2),\n    %% initiate full paxos\n    proposer:start_paxosid(Proposer1, paxid124, Acceptors,\n                           ?prepared, Majority, length(Proposers), 1),\n    proposer:start_paxosid(Proposer2, paxid124, Acceptors,\n                           ?abort, Majority, length(Proposers), 2),\n\n    %% should receive a prepared as proposer2 was hold\n    receive {learner_decide, cpaxid124, _, Res3} = Any3 -> io:format(\"Expected prepared Received ~p~n\", [Any3]) end,\n\n    gen_component:bp_barrier(comm:make_local(Proposer2)),\n    gen_component:bp_del(comm:make_local(Proposer2), bp2),\n    gen_component:bp_cont(comm:make_local(Proposer2)),\n\n    %% should receive also an abort\n    receive\n        {learner_decide, cpaxid124, _, Res4} = Any4 -> io:format(\"Expected prepared Received ~p~n\", [Any4])\n    end,\n\n    ?assert(Res3 =:= Res4),\n    ct:pal(\"done.~n\"),\n    _ = [ gen_component:kill(comm:make_local(X))\n          || X <- lists:flatten([Proposers, Acceptors, Learners])],\n    ok.\n\n%% userdevguide-begin paxos_SUITE:random_interleaving_test\n-spec prop_rnd_interleave(1..4, 4..16)\n        -> true.\nprop_rnd_interleave(NumProposers, NumAcceptors) ->\n    ct:pal(\"Called with: paxos_SUITE:prop_rnd_interleave(~p, ~p).~n\",\n           [NumProposers, NumAcceptors]),\n    Majority = NumAcceptors div 2 + 1,\n    {Proposers, Acceptors, Learners} =\n        make(NumProposers, NumAcceptors, 1, rnd_interleave),\n    %% set bp on all processes\n    _ = [ gen_component:bp_set(comm:make_local(X), ?proposer_initialize, bp)\n            ||  X <- Proposers],\n    _ = [ gen_component:bp_set(comm:make_local(X), acceptor_initialize, bp)\n            ||  X <- Acceptors ],\n    _ = [ gen_component:bp_set(comm:make_local(X), learner_initialize, bp)\n            || X <- Learners],\n    %% start paxos instances\n    _ = [ proposer:start_paxosid(X, paxidrndinterl, Acceptors,\n                                 proposal, Majority, NumProposers, Y)\n            || {X,Y} <- lists:zip(Proposers, lists:seq(1, NumProposers)) ],\n    _ = [ acceptor:start_paxosid(X, paxidrndinterl, Learners)\n            || X <- Acceptors ],\n    _ = [ learner:start_paxosid(X, paxidrndinterl, Majority,\n                                comm:this(), cpaxidrndinterl)\n            || X <- Learners],\n    %% randomly step through protocol\n    Steps = step_until_decide(Proposers ++ Acceptors ++ Learners, cpaxidrndinterl, 0),\n    ct:pal(\"Needed ~p steps~n\", [Steps]),\n    _ = [ gen_component:kill(comm:make_local(X))\n          || X <- lists:flatten([Proposers, Acceptors, Learners])],\n    true.\n\nstep_until_decide(Processes, PaxId, SumSteps) ->\n    %% io:format(\"Step ~p~n\", [SumSteps]),\n    Runnable = [ X || X <- Processes, gen_component:runnable(comm:make_local(X)) ],\n    case Runnable of\n        [] ->\n            ct:pal(\"No runnable processes of ~p~n\", [length(Processes)]),\n            timer:sleep(5), step_until_decide(Processes, PaxId, SumSteps);\n        _ ->\n            Num = randoms:uniform(length(Runnable)),\n            _ = gen_component:bp_step(comm:make_local(lists:nth(Num, Runnable))),\n            receive\n                {learner_decide, cpaxidrndinterl, _, _Res} = _Any ->\n                    %% io:format(\"Received ~p~n\", [_Any]),\n                    SumSteps\n            after 0 -> step_until_decide(Processes, PaxId, SumSteps + 1)\n            end\n    end.\n%% userdevguide-end paxos_SUITE:random_interleaving_test\n\ntest_rnd_interleave(_Config) ->\n    tester:test(paxos_SUITE, prop_rnd_interleave, _Params = 2, _Iter = 100).\n"
  },
  {
    "path": "test/performance_SUITE.erl",
    "content": "%  @copyright 2008-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author  : Thorsten Schuett <schuett@zib.de>\n%% @doc Performance Tests\n%% @end\n%% @version $Id$\n-module(performance_SUITE).\n\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n\n-dialyzer([{[no_return, no_opaque], [next_hop_with_neighbors/1]}]).\n\nall() ->\n    [empty,\n     intervals_in,\n%%     get_keys_for_replica_string,\n     md5,\n     {group, with_config},\n     pid_groups_lookup,\n     pid_groups_lookup_by_pid,\n     ets_ordset_insert1,\n     ets_ordset_insert2,\n     ets_ordset_lookup1,\n     ets_ordset_lookup2,\n     ets_ordset_foldl,\n     ets_ordset_next_iteration,\n     ets_ordset_lists_foldl,\n     ets_ordset_match,\n     ets_ordset_match_with_limit,\n     ets_ordset_select,\n     ets_ordset_select_with_limit,\n     ets_ordset_select_with_guards,\n     ets_ordset_select_with_guards_with_limit,\n     erlang_put,\n     erlang_get,\n     pdb_set,\n     pdb_get,\n%%      ordsets_add_element,\n%%      sets_add_element,\n%%      gb_sets_add_element,\n     ets_set_insert1N,\n     ets_set_insert2N,\n     ets_ordset_insert1N,\n     ets_ordset_insert2N,\n     erlang_send,\n     comm_local,\n     erlang_send_after,\n     erlang_spawn,\n     os_timestamp,\n     term_to_binary1,\n     unicode_chars_to_binary1\n    ].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 60}}\n    ].\n\ngroups() ->\n    [{with_config, [sequence], [get_keys_for_replica_string, next_hop_with_neighbors]}].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(GroupName, Config) ->\n    case GroupName of\n        with_config ->\n            unittest_helper:start_minimal_procs(Config, [], true);\n        _ -> Config\n    end.\n\nend_per_group(GroupName, Config) ->\n    case GroupName of\n        with_config -> unittest_helper:stop_minimal_procs(Config);\n        _ -> ok\n    end.\n\ncount() ->\n    1000000.\n\ndata_count() ->\n    1000000.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nempty(_Config) ->\n    iter(count(), fun () ->\n                       ok\n                  end, \"empty\"),\n    ok.\n\nintervals_in(_Config) ->\n    I = intervals:new('(', ?RT:hash_key(\"0\"), ?RT:hash_key(\"1\"), ']'),\n    iter2(count()*10, fun (X) ->\n                       intervals:in(X, I)\n                  end, \"intervals:in/2\"),\n    ok.\n\nets_ordset_lookup1(_Config) ->\n    _ = ets:new(ets_ordset_lookup1, [ordered_set, private, named_table]),\n    ets:insert(ets_ordset_lookup1, {\"performance\", \"abc\"}),\n    iter(count(), fun() ->\n                          ets:lookup(ets_ordset_lookup1, \"performance\")\n                  end, \"ets(ordered_set):lookup\"),\n    ets:delete(ets_ordset_lookup1),\n    ok.\n\nets_ordset_lookup2(_Config) ->\n    Table = ets:new(ets_ordset_lookup2, [ordered_set, private]),\n    ets:insert(Table, {\"performance\", \"abc\"}),\n    iter(count(), fun() ->\n                          ets:lookup(Table, \"performance\")\n                  end, \"ets(ordered_set_unnamed):lookup\"),\n    ets:delete(Table),\n    ok.\n\nets_ordset_insert1(_Config) ->\n    _ = ets:new(ets_ordset_insert1, [ordered_set, private, named_table]),\n    iter(count(), fun() ->\n                          ets:insert(ets_ordset_insert1, {\"performance\", \"abc\"})\n                  end, \"ets(ordered_set):insert\"),\n    ets:delete(ets_ordset_insert1),\n    ok.\n\nets_ordset_insert2(_Config) ->\n    Table = ets:new(ets_ordset_insert2, [ordered_set, private, named_table]),\n    iter(count(), fun() ->\n                          ets:insert(Table, {\"performance\", \"abc\"})\n                  end, \"ets(ordered_set_unnamed):insert\"),\n    ets:delete(Table),\n    ok.\n\nets_ordset_foldl(_Config) ->\n    Table = ets:new(db_ets_perf_comp1, [ordered_set, private, named_table]),\n    Data = [{X, crypto:strong_rand_bytes(50), false, 1, -1} || X <- lists:seq(1,\n                                                                      data_count())],\n    ets:insert(Table, Data),\n    iter(10, fun() ->\n                      ets:foldl(fun(_E, _Acc) -> ok end, ok, Table)\n              end, \"ets:foldl\"),\n    ets:delete(Table),\n    ok.\n\nets_ordset_next_iteration(_Config) ->\n    Table = ets:new(db_ets_perf_comp1, [ordered_set, private, named_table]),\n    Data = [{X, crypto:strong_rand_bytes(50), false, 1, -1} || X <- lists:seq(1,\n                                                                      data_count())],\n    ets:insert(Table, Data),\n    iter(10, fun() ->\n                      Keys = ets_next(Table, ets:first(Table), []),\n                      lists:foldl(fun(_E, _Acc) -> ok end, ok, Keys)\n              end, \"ets:next_iteration\"),\n    ets:delete(Table),\n    ok.\n\nets_next(_Table, '$end_of_table', Acc) -> Acc;\nets_next(Table, Key, Acc) ->\n    ets_next(Table, ets:next(Table, Key), [Key | Acc]).\n\nets_ordset_lists_foldl(_Config) ->\n    Table = ets:new(db_ets_perf_comp1, [ordered_set, private, named_table]),\n    Data = [{X, crypto:strong_rand_bytes(50), false, 1, -1} || X <- lists:seq(1,\n                                                                      data_count())],\n    ets:insert(Table, Data),\n    iter(10, fun() ->\n                      List = ets:tab2list(Table),\n                      lists:foldl(fun(_E, _Acc) -> ok end, ok, List)\n              end, \"lists:foldl\"),\n    ets:delete(Table),\n    ok.\n\nets_ordset_match(_Config) ->\n    Table = ets:new(db_ets_perf_comp1, [ordered_set, private, named_table]),\n    Data = [{X, crypto:strong_rand_bytes(50), false, 1, -1} || X <- lists:seq(1,\n                                                                      data_count())],\n    ets:insert(Table, Data),\n    iter(10, fun() ->\n                      List = ets:match(Table, {'$1', '_', '_', '_', '_'}),\n                      lists:foldl(fun(_E, _Acc) -> ok end, ok, List)\n              end, \"ets:match\"),\n    ets:delete(Table),\n    ok.\n\nets_ordset_match_with_limit(_Config) ->\n    Table = ets:new(db_ets_perf_comp1, [ordered_set, private, named_table]),\n    Data = [{X, crypto:strong_rand_bytes(50), false, 1, -1} || X <- lists:seq(1,\n                                                                      data_count()\n                                                                      * 2)],\n    ets:insert(Table, Data),\n    iter(10, fun() ->\n                     {List, _Cont} = ets:match(Table, {'$1', '_', '_', '_', '_'},\n                                       data_count()),\n                      lists:foldl(fun(_E, _Acc) -> ok end, ok, List)\n              end, \"ets:match_with_limit\"),\n    ets:delete(Table),\n    ok.\n\nets_ordset_select(_Config) ->\n    Table = ets:new(db_ets_perf_comp1, [ordered_set, private, named_table]),\n    Data = [{X, crypto:strong_rand_bytes(50), false, 1, -1} || X <- lists:seq(1,\n                                                                      data_count())],\n    ets:insert(Table, Data),\n    iter(10, fun() ->\n                      List = ets:select(Table, [{{'$1', '_', '_', '_', '_'},\n                                                [],\n                                                [{{'$1'}}]}]),\n                      lists:foldl(fun(_E, _Acc) -> ok end, ok, List)\n              end, \"ets:select\"),\n    ets:delete(Table),\n    ok.\n\nets_ordset_select_with_limit(_Config) ->\n    Table = ets:new(db_ets_perf_comp1, [ordered_set, private, named_table]),\n    Data = [{X, crypto:strong_rand_bytes(50), false, 1, -1} || X <- lists:seq(1,\n                                                                      data_count()\n                                                                      * 2)],\n    ets:insert(Table, Data),\n    iter(10, fun() ->\n                     {List, _Cont} = ets:select(Table, [{{'$1', '_', '_', '_', '_'},\n                                                [],\n                                                [{{'$1'}}]}],\n                                       data_count()),\n                      lists:foldl(fun(_E, _Acc) -> ok end, ok, List)\n              end, \"ets:select_with_limit\"),\n    ets:delete(Table),\n    ok.\n\nets_ordset_select_with_guards(_Config) ->\n    Table = ets:new(db_ets_perf_comp1, [ordered_set, private, named_table]),\n    Data = [{X, crypto:strong_rand_bytes(50), false, 1, -1} || X <- lists:seq(1,\n                                                                      data_count())],\n    ets:insert(Table, Data),\n    iter(10, fun() ->\n                     End = data_count() -1,\n                     List = ets:select(Table, [{{'$1', '_', '_', '_', '_'},\n                                                [{'>=', '$1', 2}, {'=<', '$1',\n                                                                   End}],\n                                                ['$_']}]),\n                      lists:foldl(fun(_E, _Acc) -> ok end, ok, List)\n              end, \"ets:select_with_guards\"),\n    ets:delete(Table),\n    ok.\n\nets_ordset_select_with_guards_with_limit(_Config) ->\n    Table = ets:new(db_ets_perf_comp1, [ordered_set, private, named_table]),\n    Data = [{X, crypto:strong_rand_bytes(50), false, 1, -1} || X <- lists:seq(1,\n                                                                      data_count()\n                                                                      * 2)],\n    ets:insert(Table, Data),\n    iter(10, fun() ->\n                     End = data_count() -1,\n                     {List, _Cont} = ets:select(Table, [{{'$1', '_', '_', '_', '_'},\n                                                [{'>=', '$1', 2}, {'=<', '$1',\n                                                                   End}],\n                                                ['$_']}], data_count()),\n                      lists:foldl(fun(_E, _Acc) -> ok end, ok, List)\n              end, \"ets:select_with_guards_with_limit\"),\n    ets:delete(Table),\n    ok.\n\nerlang_get(_Config) ->\n    erlang:put(\"performance\", \"abc\"),\n    iter(count(), fun() ->\n                          erlang:get(\"performance\")\n                  end, \"erlang:get\"),\n    ok.\n\nerlang_put(_Config) ->\n    iter(count(), fun() ->\n                          erlang:put(\"performance\", \"abc\")\n                  end, \"erlang:put\"),\n    ok.\n\npdb_get(_Config) ->\n    _ = pdb:new(pdb_get, [ordered_set, private, named_table]),\n    pdb:set({\"performance\", \"abc\"}, pdb_get),\n    iter(count(), fun() ->\n                          pdb:get(\"performance\", pdb_get)\n                  end, \"pdb:get\"),\n    ok.\n\npdb_set(_Config) ->\n    _ = pdb:new(pdb_set, [ordered_set, private, named_table]),\n    iter(count(), fun() ->\n                          pdb:set({\"performance\", \"abc\"}, pdb_set)\n                  end, \"pdb:set\"),\n    ok.\n\n% weigh too slow - can not execute the default number of test runs, i.e. 1.000.000\nordsets_add_element(_Config) ->\n    Set = ordsets:new(),\n    Set2 = iter2_foldl(10000, fun ordsets:add_element/2, Set, \"ordsets:add_element (1)\"),\n    _Set3 = iter2_foldl(10000, fun ordsets:add_element/2, Set2, \"ordsets:add_element (2)\"),\n    ok.\n\n% slow, too - do not call by default\nsets_add_element(_Config) ->\n    Set = sets:new(),\n    Set2 = iter2_foldl(100000, fun sets:add_element/2, Set, \"sets:add_element (1)\"),\n    _Set3 = iter2_foldl(100000, fun sets:add_element/2, Set2, \"sets:add_element (2)\"),\n    ok.\n\n% slow, too - do not call by default\ngb_sets_add_element(_Config) ->\n    Set = gb_sets:new(),\n    Set2 = iter2_foldl(count(), fun gb_sets:add_element/2, Set, \"gb_sets:add_element (1)\"),\n    _Set3 = iter2_foldl(count(), fun gb_sets:add_element/2, Set2, \"gb_sets:add_element (2)\"),\n    ok.\n\nets_set_insert1N(_Config) ->\n    _ = ets:new(ets_set_insert1N, [set, private, named_table]),\n    iter2(count(), fun(N) ->\n                           ets:insert(ets_set_insert1N, {N})\n                   end, \"ets(set):insert (1N)\"),\n    iter2(count(), fun(N) ->\n                           ets:insert(ets_set_insert1N, {N})\n                   end, \"ets(set):insert (2N)\"),\n    ets:delete(ets_set_insert1N),\n    ok.\n\nets_set_insert2N(_Config) ->\n    Table = ets:new(ets_set_insert2N, [set, private]),\n    iter2(count(), fun(N) ->\n                           ets:insert(Table, {N})\n                   end, \"ets(set_unnamed):insert (1N)\"),\n    iter2(count(), fun(N) ->\n                           ets:insert(Table, {N})\n                   end, \"ets(set_unnamed):insert (2N)\"),\n    ets:delete(Table),\n    ok.\n\nets_ordset_insert1N(_Config) ->\n    _ = ets:new(ets_ordset_insert1N, [ordered_set, private, named_table]),\n    iter2(count(), fun(N) ->\n                           ets:insert(ets_ordset_insert1N, {N})\n                   end, \"ets(ordered_set):insert (1N)\"),\n    iter2(count(), fun(N) ->\n                           ets:insert(ets_ordset_insert1N, {N})\n                   end, \"ets(ordered_set):insert (2N)\"),\n    ets:delete(ets_ordset_insert1N),\n    ok.\n\nets_ordset_insert2N(_Config) ->\n    Table = ets:new(ets_set_insert2N, [ordered_set, private]),\n    iter2(count(), fun(N) ->\n                           ets:insert(Table, {N})\n                   end, \"ets(ordered_set_unnamed):insert (1N)\"),\n    iter2(count(), fun(N) ->\n                           ets:insert(Table, {N})\n                   end, \"ets(ordered_set_unnamed):insert (2N)\"),\n    ets:delete(Table),\n    ok.\n\nerlang_send(_Config) ->\n    Pid = spawn(?MODULE, helper_rec, [count(), self()]),\n    iter(count(), fun() -> Pid ! {ping} end, \"erlang:send\"),\n    receive {pong} -> ok end,\n    ok.\n\ncomm_local(_Config) ->\n    Pid = spawn(?MODULE, helper_rec, [count(), self()]),\n    iter(count(), fun() -> comm:send_local(Pid, {ping}) end, \"comm_local\"),\n    receive {pong} -> ok end,\n    ok.\n\nhelper_rec(0, Pid) -> Pid ! {pong};\nhelper_rec(Iter, Pid) ->\n    receive _Any -> ok end,\n    helper_rec(Iter - 1, Pid).\n\nerlang_send_after(_Config) ->\n    Pid = spawn(?MODULE, helper_rec, [count(), self()]),\n    iter(count(), fun() -> comm:send_local_after(5000, Pid, {ping}) end, \"comm:send_after\"),\n    receive {pong} -> ok end,\n    ok.\n\nerlang_spawn(_Config) ->\n    iter(count(), fun() -> spawn(fun() -> ok end) end, \"erlang:spawn\"),\n    ok.\n\nos_timestamp(_Config) ->\n    iter(count(), fun() -> os:timestamp() end, \"os:timestamp\"),\n    ok.\n\n\nget_keys_for_replica_string(_Config) ->\n    iter(count(), fun () ->\n                          rt_chord:get_replica_keys(rt_chord:hash_key(\"42\"))\n               end, \"get_keys_for_replica_string\"),\n    ok.\n\nmd5(_Config) ->\n    iter(count(), fun () -> ?CRYPTO_MD5(\"42\") end, \"crypto:md5\"),\n    iter(count(), fun () -> erlang:md5(\"42\") end, \"erlang:md5\"),\n    ok.\n\n-define(PID(Nr), comm:make_global(list_to_pid(\"<0.0.\" ++ erlang:integer_to_list(Nr) ++ \">\"))).\n\nnext_hop_setup() ->\n    pid_groups:join_as(?MODULE, dht_node),\n    Pred = node:new(?PID(1021), 1, 0),\n    Me = node:new(?PID(2), 2, 0),\n    Succ = node:new(?PID(3), 3, 0),\n    RT = gb_trees:enter(1, node:pidX(Pred),\n          gb_trees:enter(4, ?PID(4),\n           gb_trees:enter(5, ?PID(5),\n            gb_trees:enter(6, ?PID(6),\n             gb_trees:enter(100, ?PID(100),\n              gb_trees:enter(101, ?PID(101),\n               gb_trees:enter(102, ?PID(102),\n                gb_trees:enter(103, ?PID(103),\n                 rt_chord:empty_ext(nodelist:new_neighborhood(Me, Succ)))))))))),\n    %% RMState = rm_loop:unittest_create_state(\n    Neighborhood= nodelist:add_nodes(\n                nodelist:new_neighborhood(Pred, Me, Succ),\n                 [node:new(?PID(Id + 2), Id + 2, 0)\n                  || Id <- lists:seq(2, config:read(succ_list_length))] ++\n                     [node:new(?PID(1022 - Id), 1022 - Id, 0)\n                      || Id <- lists:seq(2, config:read(pred_list_length))],\n                 config:read(succ_list_length), config:read(pred_list_length)),\n                %% false),\n    %% _State = dht_node_state:new(RT, RMState, db).\n    {Neighborhood, RT}.\n\nnext_hop_with_neighbors(_Config) ->\n    {Neighbors, RT}  = next_hop_setup(),\n    iter(count(), fun() -> rt_chord:next_hop(Neighbors, RT, 42) end, \"next_hop(42) with neighbors\"),\n    iter(count(), fun() -> rt_chord:next_hop(Neighbors, RT, 5) end, \"next_hop(5) with neighbors\"),\n    ok.\n\npid_groups_lookup(_Config) ->\n    {ok, _Pid} = pid_groups:start_link(),\n    pid_groups:join_as(?MODULE, pid_groups),\n    iter(count(), fun () ->\n                          pid_groups:pid_of(?MODULE,\n                                            pid_groups)\n                  end, \"pid_of by group and process name\"),\n    unittest_helper:stop_pid_groups(),\n    ok.\n\npid_groups_lookup_by_pid(_Config) ->\n    {ok, _Pid} = pid_groups:start_link(),\n    pid_groups:join_as(?MODULE, pid_groups),\n    iter(count(), fun () ->\n                          pid_groups:group_and_name_of(self())\n                  end, \"group_and_name_of pid\"),\n    unittest_helper:stop_pid_groups(),\n    ok.\n\nterm_to_binary1(_Config) ->\n    String = \"qwertzuiopasdfghjklyxcvbnm\" ++ [246,252,228,87,224,103,114,97,105,110,32,40,87,229,103,114,335,227,41],\n    iter(count(), fun() ->\n                          erlang:term_to_binary(String)\n                  end, \"erlang:term_to_binary/1\"),\n    ok.\n\nunicode_chars_to_binary1(_Config) ->\n    String = \"qwertzuiopasdfghjklyxcvbnm\" ++ [246,252,228,87,224,103,114,97,105,110,32,40,87,229,103,114,335,227,41],\n    iter(count(), fun() ->\n                          unicode:characters_to_binary(String)\n                  end, \"unicode:characters_to_binary/1\"),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec iter(Count::pos_integer(), F::fun(() -> any()), Tag::string()) -> ok.\niter(Count, F, Tag) ->\n    F(),\n    Start = os:timestamp(),\n    iter_inner(Count, F),\n    Stop = os:timestamp(),\n    ElapsedTime = erlang:max(1, timer:now_diff(Stop, Start)) / 1000000.0,\n    Frequency = Count / ElapsedTime,\n    ct:pal(\"~p iterations of ~p took ~ps: ~p1/s~n\",\n           [Count, Tag, ElapsedTime, Frequency]),\n    ok.\n\n-spec iter_inner(Count::pos_integer(), F::fun(() -> any())) -> ok.\niter_inner(0, _) ->\n    ok;\niter_inner(N, F) ->\n    F(),\n    iter_inner(N - 1, F).\n\n-spec iter2(Count::pos_integer(), F::fun((Count::non_neg_integer()) -> any()), Tag::string()) -> ok.\niter2(Count, F, Tag) ->\n    _ = F(0),\n    Start = os:timestamp(),\n    iter2_inner(Count, F),\n    Stop = os:timestamp(),\n    ElapsedTime = erlang:max(1, timer:now_diff(Stop, Start)) / 1000000.0,\n    Frequency = Count / ElapsedTime,\n    ct:pal(\"~p iterations of ~s took ~ps: ~p1/s~n\",\n           [Count, Tag, ElapsedTime, Frequency]),\n    ok.\n\n-spec iter2_inner(Count::non_neg_integer(), F::fun((Count::non_neg_integer()) -> any())) -> ok.\niter2_inner(0, _) ->\n    ok;\niter2_inner(N, F) ->\n    _ = F(N),\n    iter2_inner(N - 1, F).\n\n-spec iter2_foldl(Count::pos_integer(), F::fun((Count::non_neg_integer(), Acc) -> Acc), Acc, Tag::string()) -> Acc.\niter2_foldl(Count, F, Acc0, Tag) ->\n    _ = F(0, Acc0),\n    Start = os:timestamp(),\n    FinalAcc = iter2_foldl_helper(Count, F, Acc0),\n    Stop = os:timestamp(),\n    ElapsedTime = erlang:max(1, timer:now_diff(Stop, Start)) / 1000000.0,\n    Frequency = Count / ElapsedTime,\n    ct:pal(\"~p foldl iterations of ~s took ~ps: ~p1/s~n\",\n           [Count, Tag, ElapsedTime, Frequency]),\n    FinalAcc.\n\n-spec iter2_foldl_helper(Count::non_neg_integer(), F::fun((Count::non_neg_integer(), Acc) -> Acc), Acc) -> Acc.\niter2_foldl_helper(0, _F, Acc) -> Acc;\niter2_foldl_helper(Count, F, Acc) ->\n    iter2_foldl_helper(Count - 1, F, F(Count, Acc)).\n"
  },
  {
    "path": "test/performance_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{suites, tests, [performance_SUITE]}.\n{suites, tests, [benchmark_SUITE]}.\n{suites, tests, [rr_recon_performance_SUITE]}.\n\n{cases, tests, histogram_SUITE, [perf_add]}.\n"
  },
  {
    "path": "test/prbr_SUITE.erl",
    "content": "%% @copyright 2013-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for prbr\n%% @end\n%% @version $Id$\n-module(prbr_SUITE).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\nall()   -> [\n            tester_type_check_rbr,\n            rbr_increment,\n            rbr_concurrency_kv,\n            rbr_concurrency_leases,\n            rbr_consistency,\n            rbr_consistency_delete\n           ].\nsuite() -> [ {timetrap, {seconds, 400}} ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_testcase(TestCase, Config) ->\n    case TestCase of\n        rbr_concurrency_kv ->\n            {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n            Size = randoms:rand_uniform(3, 14),\n            unittest_helper:make_ring(Size, [{config, [{log_path, PrivDir}]}]),\n            %% necessary for the consistency check:\n            unittest_helper:check_ring_size_fully_joined(Size),\n            ok;\n        rbr_concurrency_leases ->\n            {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n            Size = 1, %% larger rings not supported by leases yet,\n            %% Size = randoms:rand_uniform(2, 14),\n            unittest_helper:make_ring(Size, [{config, [{log_path, PrivDir},\n                                                       {leases, true}]}]),\n            %% necessary for the consistency check:\n            unittest_helper:check_ring_size_fully_joined(Size),\n            ok;\n        rbr_consistency ->\n            {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n            unittest_helper:make_symmetric_ring([{config, [{log_path, PrivDir}]}]),\n            %% necessary for the consistency check:\n            unittest_helper:check_ring_size_fully_joined(config:read(replication_factor)),\n\n            ok;\n        _ ->\n            {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n            Size = randoms:rand_uniform(1, 9),\n            unittest_helper:make_ring(Size, [{config, [{log_path, PrivDir}]}]),\n            ok\n    end,\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n%% TODO: unittest for: retrigger on read works\n%% TODO: unittest for: retrigger on write works\n\nrbr_increment(_Config) ->\n    %% start random number of nodes\n    %% select a key to operate on\n    %% start random number of increments (non transactional)\n    %% check if number of increments = value in key\n    Key = randoms:getRandomString(),\n\n    Parallel = randoms:rand_uniform(5, 11),\n    Count = 1000 div Parallel,\n    ct:pal(\"Starting concurrent writers: ~p~n\"\n           \"Performing iterations: ~p~n\",\n           [Parallel, Count]),\n    UnitTestPid = self(),\n    _Pids = [ spawn(fun() ->\n                            _ = [ begin\n                                      {ok} = inc_on_cseq:inc(Key)\n                                  end\n                                  || _I <- lists:seq(1, Count)],\n                           UnitTestPid ! {done}\n                   end)\n             || _Nth <- lists:seq(1, Parallel)],\n\n    _ = [ receive {done} ->\n                      ct:pal(\"Finished ~p/~p.~n\", [Nth, Parallel]),\n                      ok\n          end || Nth <- lists:seq(1, Parallel)],\n\n    {ok, Result} = inc_on_cseq:read(Key),\n    ct:pal(\"Planned ~p increments, done ~p - discrepancy is NOT ok~n\",\n           [Count*Parallel, Result]),\n    ?equals(Count*Parallel, Result),\n    ok.\n\nrbr_concurrency_kv(_Config) ->\n    %% start random number of nodes\n    %% select a key to operate on\n    %% start random number of writers (increment operations\n    %%   / use increment as write filter)\n    %% start random number of readers\n    %%   only observe increasing values in reads\n    Key = randoms:getRandomString(),\n\n    {ok} = kv_on_cseq:write(Key, 1),\n    Parallel = randoms:rand_uniform(1, 11),\n    Count = 1000 div Parallel,\n    ct:pal(\"Starting concurrent writers: ~p~n\"\n           \"Performing iterations: ~p~n\",\n           [Parallel, Count]),\n    UnitTestPid = self(),\n    _Pids = [ spawn(fun() ->\n                            _ = [ begin\n                                      {ok, V} = kv_on_cseq:read(Key),\n                                      {ok} = kv_on_cseq:write(Key, V+1)\n                                      %% if 0 == I rem 100 ->\n                                      %%   ct:pal(\"~p performed write ~p.~n\",\n                                      %%   [_Nth, I]);\n                                      %%   true -> ok\n                                      %% end\n                                  end\n                                  || _I <- lists:seq(1, Count)],\n                           UnitTestPid ! {done}\n                   end)\n             || _Nth <- lists:seq(1, Parallel)],\n\n    _ = [ receive {done} ->\n                      ct:pal(\"Finished ~p/~p.~n\", [Nth, Parallel]),\n                      ok\n          end || Nth <- lists:seq(1, Parallel)],\n\n    ct:pal(\"Planned ~p increments, done ~p - discrepancy is ok~n\",\n           [Count*Parallel, kv_on_cseq:read(Key)]),\n    ok.\n\nrbr_concurrency_leases(_Config) ->\n    %% start random number of nodes\n    %% select a key to operate on\n    %% start random number of writers (increment operations\n    %%   / use increment as write filter)\n    %% start random number of readers\n    %%   only observe increasing values in reads\n    Key = ?RT:get_random_node_id(),\n\n    ContentCheck =\n        fun (Current, _WriteFilter, _Next) ->\n                case Current == prbr_bottom of\n                    true ->\n                        {true, null};\n                    false ->\n                        {false, lease_already_exists}\n                end\n        end,\n    New = l_on_cseq:unittest_create_lease(Key),\n    DB = rbrcseq:get_db_for_id(lease_db, Key),\n    rbrcseq:qwrite(DB, self(), Key, l_on_cseq,\n                   ContentCheck,\n                   New),\n    receive\n        {qwrite_done, _ReqId, _Round, _, _} -> ok\n    end,\n\n    Parallel = randoms:rand_uniform(4, 11),\n    Count = 1000 div Parallel,\n    ct:pal(\"Starting concurrent writers: ~p~n\"\n           \"Performing iterations: ~p~n\",\n           [Parallel, Count]),\n    UnitTestPid = self(),\n    DHTNodeGroups = pid_groups:groups_with(dht_node),\n    DHTNodeGroupsLen = length(DHTNodeGroups),\n    _Pids =\n        [ spawn(\n            fun() ->\n                    Group = lists:nth(1 + Nth rem DHTNodeGroupsLen,\n                                      DHTNodeGroups),\n                    pid_groups:join(Group),\n                    _ = [ begin\n                              F = fun(X) ->\n                                          {ok, V} = l_on_cseq:read(Key),\n                                          Update =\n                                              l_on_cseq:unittest_lease_update_unsafe(\n                                                V,\n                                                l_on_cseq:set_version(\n                                                  V, l_on_cseq:get_version(V)+1), passive),\n                                          case Update of\n                                              ok -> ok;\n                                              failed ->\n                                                  %% ct:pal(\"~p retry ~p.~n\",\n                                                  %%        [_Nth, l_on_cseq:get_version(V)+1]),\n                                                  X(X)\n                                          end\n                                  end,\n                              F(F)\n                          %% ct:pal(\"~p performed write.~n\", [_Nth])\n                          end\n                          || _I <- lists:seq(1, Count)],\n                    UnitTestPid ! {done}\n            end)\n          || Nth <- lists:seq(1, Parallel)],\n\n    _ = [ receive {done} ->\n                      ct:pal(\"Finished ~p/~p.~n\", [Nth, Parallel]),\n                      ok\n          end || Nth <- lists:seq(1, Parallel)],\n\n    ct:pal(\"Planned ~p increments, done ~p, discrepancy is ok~n\",\n           [Count*Parallel, l_on_cseq:read(Key)]),\n\n    ok.\n\nrbr_consistency(_Config) ->\n    %% create an rbr entry\n    %% update 1 to 3 of its replicas\n    %% perform read in all quorum permutations\n    %% (intercept read on a single dht node)\n    %% output must be the old value or the new value\n    %% if the new value was seen once, the old must not be readable again.\n\n    Nodes = pid_groups:find_all(dht_node),\n    Key = \"a\",\n\n    %% initialize key\n    {ok} = kv_on_cseq:write(Key, 1),\n\n    %% select a replica\n    Replicas = ?RT:get_replica_keys(?RT:hash_key(Key)),\n\n%%    print modified rbr entries\n%%    api_tx_proto_sched_SUITE:rbr_invariant(a,b,c),\n\n    _ = [ begin\n              New = N+100,\n              {ok, Old} = kv_on_cseq:read(Key),\n\n              modify_rbr_at_key(R, N+100),\n              %% ct:pal(\"After modification:\"),\n              %% print modified rbr entries\n              %% api_tx_proto_sched_SUITE:rbr_invariant(a,b,c),\n\n              %% intercept and drop a message at r1\n              _ = lists:foldl(read_quorum_without(Key), {Old, New}, Nodes),\n              ok\n          end || {R,N} <- lists:zip(Replicas, lists:seq(1, config:read(replication_factor)))],\n\n    ok.\n\nrbr_consistency_delete(_Config) ->\n    %% create an rbr entry\n    %% update 1 to 3 of its replicas\n    %% perform read in all quorum permutations\n    %% (intercept read on a single dht node)\n    %% output must be the old value or the new value\n    %% if the new value was seen once, the old must not be readable again.\n\n    %% Nodes = pid_groups:find_all(dht_node),\n    Key = \"a\",\n\n    %% initialize key\n    {ok} = kv_on_cseq:write(Key, 1),\n\n    %% select a replica\n    Replicas = ?RT:get_replica_keys(?RT:hash_key(Key)),\n\n%%    print modified rbr entries\n%%    api_tx_proto_sched_SUITE:rbr_invariant(a,b,c),\n\n    ct:pal(\"Starting delete test~n\"),\n\n\n    Res = [ begin\n                ct:pal(\"Read iteration: ~p~n\", [R]),\n\n                {ok, Old} = kv_on_cseq:read(Key),\n\n                delete_rbr_entry_at_key(R),\n                Next = Old + 1,\n\n                ct:pal(\"Write in iteration: ~p~n\", [R]),\n                _ = kv_on_cseq:write(Key, Next),\n\n                %% ct:pal(\"After modification:\"),\n                %% print modified rbr entries\n                %% api_tx_proto_sched_SUITE:rbr_invariant(a,b,c),\n\n                ct:pal(\"Reread in iteration: ~p~n\", [R]),\n                {ok, Next} = kv_on_cseq:read(Key)\n          end || R <- Replicas],\n\n    ct:pal(\"Result: ~p~n\", [Res]),\n    ok.\n\n\n\ntester_type_check_rbr(_Config) ->\n    Count = 250,\n    config:write(no_print_ring_data, true),\n\n    tester:register_value_creator({typedef, prbr, write_filter, []},\n                                  prbr, tester_create_write_filter, 1),\n\n    %% [{modulename, [excludelist = {fun, arity}]}]\n    Modules =\n        [ {txid_on_cseq,\n           [ {is_valid_new, 3}, %% cannot create funs\n             {is_valid_decide, 3}, %% cannot create funs\n             {is_valid_delete, 3}, %% cannot create funs\n             {decide, 5}, %% cannot create pids\n             {delete, 2}, %% cannot create pids\n             {read, 2} %% cannot create pids\n           ],\n           [ ]\n          },\n          {tx_tm,\n           [{start_link, 2},       %% starts processes\n            {start_gen_component,5}, %% unsupported types\n            {init, 1},             %% needs to be pid_group member\n            {on, 2},               %% needs valid messages\n            {on_init, 2},          %% needs valid messages\n            {commit, 4},           %% needs valid clients pid\n            {msg_commit_reply, 3} %% needs valid clients pid\n           ],\n           [ {get_entry, 2},         %% could read arb, entries\n             %% guessing keys of tx entries...\n             {tx_state_add_nextround_writtenval_for_commit, 4}\n           ]\n          },\n          {kv_on_cseq,\n           [ {commit_read, 5},  %% tested via feeder\n             {commit_write, 5}, %% tested via feeder\n             {abort_read, 5},   %% tested via feeder\n             {abort_write, 5},  %% tested via feeder\n             {get_commuting_wf_for_rf, 1}], %% cannot create funs\n           []},\n          {inc_on_cseq, [],[]},\n          {pr,\n           [\n           ],\n           []},\n          {prbr,\n           [ {init, 1},             %% needs to be in a pidgroup for db_name\n             {close, 1},            %% needs valid ets:tid()\n             {close_and_delete, 1}, %% needs valid ets:tid()\n             {on, 2},               %% sends messages\n             {get_load, 1},         %% needs valid ets:tid()\n             {set_entry, 2},        %% needs valid ets:tid()\n             {get_entry, 2},        %% needs valid ets:tid()\n             {delete_entry, 2},     %% needs valid ets:tid()\n             {tab2list, 1},         %% needs valid ets:tid()\n             {tab2list_raw_unittest, 1} %% needs valid ets:tid()\n          ],\n           [ {msg_read_reply, 6},  %% sends messages\n             {msg_write_reply, 6}, %% sends messages\n             {msg_write_deny, 4},  %% sends messages\n             {tab2list_raw, 1}     %% needs valid ets:tid()\n            ]},\n          {rbrcseq,\n           [ {on, 2},          %% sends messages\n             {qread, 4},       %% tries to create envelopes\n             {qread, 5},       %% needs fun as input\n             {start_link, 3},  %% needs fun as input\n             {start_gen_component,5}, %% unsupported types\n             {qwrite, 6},      %% needs funs as input\n             {qwrite, 8},      %% needs funs as input\n             {qwrite_fast, 8}, %% needs funs as input\n             {qwrite_fast, 10}  %% needs funs as input\n           ],\n           [ {inform_client, 3}, %% cannot create valid envelopes\n             {start_request, 2}, %% cannot create valid envelopes\n             {get_entry, 2},     %% needs valid ets:tid()\n             {save_entry, 2},     %% needs valid ets:tid()\n             {update_entry, 2},     %% needs valid ets:tid()\n             {delete_entry, 2},     %% needs valid ets:tid()\n             {delete_newer_entries, 2}, %% needs valid ets:tid()\n             {delete_all_entries, 3}, %% needs valid ets:tid()\n             {retrigger, 3},     %% needs valid ets:tid()\n             {add_rr_reply, 10},  %% needs client_value matching db_type\n             {add_read_reply, 11},%% needs client_value matching db_type\n             {update_write_state, 5}, %% needs client_value matching db_type\n             {add_read_deny, 5}, %% needs valid entry()\n             {add_write_reply, 3},%% needs valid entry()\n             {add_write_deny, 3}, %% needs valid entry()\n             {is_read_commuting, 3} %% cannot create funs\n           ]},\n          {replication,\n           [ {get_read_value, 2},     %% cannot create funs\n             {collect_read_value, 3}  %% needs client_value matching datatype\n           ],\n           []}\n        ],\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    tester:unregister_value_creator({typedef, prbr, write_filter, []}),\n\n    true.\n\nmodify_rbr_at_key(R, N) ->\n    %% get a valid round number\n\n    %% we ask all replicas to not get an outdated round number (select\n    %% the highest one.\n    ProposerUID = unittest_rbr_consistency1_id,\n    Rounds = [ begin\n                   %% let fill in whether lookup was consistent\n                   LookupReadEnvelope = dht_node_lookup:envelope(\n                                          4,\n                                          {prbr, round_request, kv_db, '_', comm:this(),\n                                           Repl, kv_on_cseq, ProposerUID,\n                                           fun prbr:noop_read_filter/1, write}),\n                   comm:send_local(pid_groups:find_a(dht_node),\n                                   {?lookup_aux, Repl, 0, LookupReadEnvelope}),\n                   receive\n                       {round_request_reply, _, AssignedRound, OldWriteRound,  _Val, _LastWF} ->\n                           {AssignedRound, OldWriteRound}\n                   end\n               end || Repl <- ?RT:get_replica_keys(R) ],\n    {ReadRounds, WriteRounds} = lists:foldl(fun({RR, WR}, {AccR, AccW}) ->\n                                                    {[RR|AccR], [WR|AccW]}\n                                            end, {[], []}, Rounds),\n    ct:pal(\"~p ~n~p\", [ReadRounds, WriteRounds]),\n    {HighestReadRound, HighestWriteRound} = {lists:max(ReadRounds),\n                                             lists:max(WriteRounds)},\n\n    %% perform a write\n    %% let fill in whether lookup was consistent\n    LookupWriteEnvelope = dht_node_lookup:envelope(\n                            4,\n                            {prbr, write, kv_db, '_', comm:this(), R,\n                             kv_on_cseq, ProposerUID, HighestReadRound, HighestWriteRound,\n                             {[], false, _Version = N-100, _Value = N},\n                             null,\n                             fun prbr:noop_write_filter/3, false}),\n    %% modify the replica at key R, therefore we use a lookup...\n    comm:send_local(pid_groups:find_a(dht_node),\n                    {?lookup_aux, R, 0, LookupWriteEnvelope}),\n    receive\n        {write_reply, _, R, _, _NextRound, _} ->\n            ok\n    end.\n\ndelete_rbr_entry_at_key(R) ->\n    comm:send_local(pid_groups:find_a(dht_node),\n                    {?lookup_aux, R, 0,\n                    {prbr, delete_key, kv_db, self(), R}}),\n    receive {delete_key_reply, R} -> ok end.\n\ndrop_prbr_read_request(Client, Tag) ->\n    fun (Message, _State) ->\n            case Message of\n%%                {prbr, _, kv_db, ReqClient, Key, _Round, _RF} ->\n                _ when element(1, Message) =:= prbr\n                       andalso element(3, Message) =:= kv_db ->\n                    ct:pal(\"Detected read, dropping it ~p, key ~p~n\",\n                           [self(), element(5, Message)]),\n                    comm:send_local(Client, {Tag, done}),\n                    drop_single;\n                _ when element(1, Message) =:= prbr ->\n                    false;\n                _ -> false\n            end\n    end.\n\nread_quorum_without(Key) ->\n    fun (ExcludedDHTNode, {Old, New}) ->\n            gen_component:bp_set_cond(\n              ExcludedDHTNode,\n              drop_prbr_read_request(self(), drop_prbr_read),\n              drop_prbr_read),\n\n            {ok, Val} = kv_on_cseq:read(Key),\n            io:format(\"Old: ~p, Val: ~p New: ~p\", [Old, Val, New]),\n            receive\n                {drop_prbr_read, done} ->\n                    gen_component:bp_del(ExcludedDHTNode, drop_prbr_read),\n                    ok\n            end,\n            cleanup({drop_prbr_read, done}),\n            %% print modified rbr entries:\n            %% api_tx_proto_sched_SUITE:rbr_invariant(a,b,c),\n            case Val of\n                Old ->\n                    {Old, New}; %% valid for next read\n                New ->\n                    {New, New}; %% old is no longer acceptable\n                X ->\n                    %% maybe an update was not propagated at all in the previous round\n                    case X > Old andalso X < New of\n                        true -> {X, New};\n                        _ -> ?equals(Val, New)\n                    end\n            end\n    end.\n\ncleanup(Msg) ->\n    receive Msg -> cleanup(Msg)\n    after 0 -> ok\n    end.\n"
  },
  {
    "path": "test/proto_sched_SUITE.erl",
    "content": "%% @copyright 2012-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests using the protocol scheduler\n%% @end\n%% @version $Id$\n-module(proto_sched_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\n-dialyzer([{no_return, basic_start_bench_and_kill_it/1}]).\n\n%%-define(TRACE(X,Y), log:log(\"proto_sched_SUITE: \" ++ X,Y)).\n-define(TRACE(X,Y), ok).\n\ngroups() ->\n    [\n     %{tester_tests, [sequence], []},\n     {basic_proto_sched_tests, [sequence], %% {repeat, 100}],\n      [ basic_empty_sched,basic_empty_sched,\n        basic_client_ping_pong,\n        basic_client_ping_pong2,\n        basic_client_ping_pong_xing,\n        basic_begin_after_running,\n        basic_duplicate_begin,\n        basic_uninfected_thread_end,\n        basic_duplicate_thread_num,\n        basic_cleanup_before_thread_end,\n        basic_cleanup_infected,\n        basic_cleanup_non_existing,\n        basic_cleanup_DOWN_interplay,\n        basic_cleanup_send_interplay,\n        basic_thread_yield_outside_proto_sched,\n        basic_slow_msg_handler,\n        basic_bench_increment%,\n%%        basic_start_bench_and_kill_it\n      ]},\n     {rbr_tests, [sequence],\n      [ test_kv_on_cseq_read,\n        test_kv_on_cseq_read_2,\n        test_kv_on_cseq_write,\n        test_kv_on_cseq_write_2,\n        test_qwrite_qwrite_qread\n      ]}\n    ].\n\nall() ->\n    [\n     {group, basic_proto_sched_tests},\n     %{group, tester_tests},\n     {group, rbr_tests}\n     ].\n\nsuite() -> [ {timetrap, {seconds, 12}} ].\n\n%group(tester_tests) ->\n%    [{timetrap, {seconds, 400}}];\ngroup(basic_proto_sched_tests) ->\n    [{timetrap, {seconds, 10}}];\ngroup(rbr_tests) ->\n    [{timetrap, {seconds, 400}}];\ngroup(_) ->\n    suite().\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(_TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(4, [{config, [{log_path, PrivDir},\n                                            {ordered_links, false}]}]),\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\nbasic_empty_sched(_Config) ->\n    %% an empty thread should be a valid run\n    proto_sched:thread_num(1),\n    ?ASSERT(not proto_sched:infected()),\n    proto_sched:thread_begin(),\n    ?ASSERT(proto_sched:infected()),\n    proto_sched:thread_end(),\n    ?ASSERT(not proto_sched:infected()),\n    proto_sched:wait_for_end(),\n    ?ASSERT(not proto_sched:infected()),\n    _ = proto_sched:get_infos(),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed),\n    ?ASSERT(not proto_sched:infected()),\n    %% TODO: check statistics\n    proto_sched:cleanup().\n\nbasic_client_ping_pong(_Config) ->\n    %% sending ourselves a single message and receiving it again\n    proto_sched:thread_num(1),\n    proto_sched:thread_begin(),\n    ?ASSERT(proto_sched:infected()),\n    comm:send_local(self(), {hello_world}),\n    proto_sched:thread_yield(),\n%%    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV({hello_world},\n                       begin\n                           ?ASSERT(proto_sched:infected()),\n                           ok\n                       end)\n        end,\n    ?ASSERT(proto_sched:infected()),\n    proto_sched:thread_end(),\n    ?ASSERT(not proto_sched:infected()),\n    proto_sched:wait_for_end(),\n    _ = proto_sched:get_infos(),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed),\n    %% TODO: check statistics\n    proto_sched:cleanup().\n\nbasic_client_ping_pong2(_Config) ->\n    %% sending ourselves a single message and receiving it again\n    %% done in two independent processes\n    proto_sched:thread_num(2),\n    spawn(fun() ->\n                  proto_sched:thread_begin(),\n                  comm:send_local(self(), {hello_world}),\n                  proto_sched:thread_yield(),\n                  receive\n                      ?SCALARIS_RECV({hello_world},\n                                     begin\n                                         ?ASSERT(proto_sched:infected()),\n                                         ok\n                                     end)\n                      end,\n                  ?ASSERT(proto_sched:infected()),\n                  proto_sched:thread_end()\n          end),\n    proto_sched:thread_begin(),\n    ?ASSERT(proto_sched:infected()),\n    comm:send_local(self(), {hello_world}),\n    proto_sched:thread_yield(),\n    receive\n        ?SCALARIS_RECV({hello_world},\n                       begin\n                           ?ASSERT(proto_sched:infected()),\n                           ok\n                       end)\n        end,\n    ?ASSERT(proto_sched:infected()),\n    proto_sched:thread_end(),\n    ?ASSERT(not proto_sched:infected()),\n    proto_sched:wait_for_end(),\n    _ = proto_sched:get_infos(),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed),\n    %% TODO: check statistics\n    proto_sched:cleanup().\n\nbasic_client_ping_pong_xing(_Config) ->\n    %% two processes sending each other a single message and receiving\n    %% one from the other process. (Xing means 'crossing').\n\n    %% to protect from main thread being faster than the spawned\n    %% process and thereby observing early cleanups, the spawned\n    %% process first receives and then sends. So, the main thread\n    %% waits for the spawned process to finish.\n    Parent = self(),\n    ct:pal(\"Parent is: ~p\", [Parent]),\n    Child = spawn(fun() ->\n                          %% not infected\n                          trace_mpath:thread_yield(),\n                          %% not infected\n                          ct:pal(\"Child receives\"),\n                          receive\n                              ?SCALARIS_RECV({from_parent},\n                                             begin\n                                                 ?ASSERT(proto_sched:infected()),\n                                                 ok\n                                             end)\n                              end,\n                          ?ASSERT(proto_sched:infected()),\n                          comm:send_local(Parent, {from_child})\n          end),\n\n    proto_sched:thread_num(1),\n    ct:pal(\"Child is: ~p\", [Child]),\n    proto_sched:thread_begin(),\n    ?ASSERT(proto_sched:infected()),\n    ct:pal(\"Parent sends\"),\n    comm:send_local(Child, {from_parent}),\n    proto_sched:thread_yield(),\n    ct:pal(\"Parent receives\"),\n    receive\n        ?SCALARIS_RECV({from_child},\n                       begin\n                           ?ASSERT(proto_sched:infected()),\n                           ok\n                       end)\n        end,\n    ?ASSERT(proto_sched:infected()),\n    ct:pal(\"Parent ends\"),\n    proto_sched:thread_end(),\n    ?ASSERT(not proto_sched:infected()),\n    proto_sched:wait_for_end(),\n    _ = proto_sched:get_infos(),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed),\n    %% TODO: check statistics\n    proto_sched:cleanup().\n\nbasic_begin_after_running(_Config) ->\n    %% when a proto_sched runs, it should not allow further threads to\n    %% join, because when the new thread joins could depend on the\n    %% executing environment, so reproducability could be violated\n    proto_sched:thread_num(1),\n    Pid = self(),\n    Child = spawn(fun() ->\n                          proto_sched:thread_begin(),\n                          Pid ! {started},\n                          receive {done} -> ok end,\n                          proto_sched:thread_end()\n          end),\n    receive {started} -> ok end,\n    ?expect_exception(proto_sched:thread_begin(),\n                      throw, 'proto_sched:thread_begin-but_already_running'),\n    Child ! {done},\n    proto_sched:wait_for_end(),\n    _ = proto_sched:get_infos(),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed),\n    %% TODO: check statistics\n    proto_sched:cleanup().\n\nbasic_duplicate_begin(_Config) ->\n    %% each execution thread should only claim to begin once.\n    proto_sched:thread_num(1),\n    proto_sched:thread_begin(),\n    ct:pal(\"We were scheduled and can start our thread\"),\n    ?ASSERT(proto_sched:infected()),\n    ct:pal(\"We try to begin again..\"),\n    ?expect_exception(proto_sched:thread_begin(),\n                      throw, duplicate_thread_begin),\n    proto_sched:thread_end(),\n    proto_sched:wait_for_end(),\n    _ = proto_sched:get_infos(),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed),\n    %% TODO: check statistics\n    proto_sched:cleanup().\n\nbasic_uninfected_thread_end(_Config) ->\n    %% thread_end should only be called in infected contexts.\n    ?ASSERT(not proto_sched:infected()),\n    ?expect_exception(proto_sched:thread_end(),\n                      throw, duplicate_or_uninfected_thread_end),\n    proto_sched:thread_num(1),\n    proto_sched:thread_begin(),\n    proto_sched:thread_end(),\n    %% duplicate end raises the same error...\n    ?expect_exception(proto_sched:thread_end(),\n                      throw, duplicate_or_uninfected_thread_end),\n    proto_sched:wait_for_end(),\n    _ = proto_sched:get_infos(),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed),\n    %% TODO: check statistics\n    proto_sched:cleanup().\n\nbasic_duplicate_thread_num(_Config) ->\n    %% the thread_num is defined once and fixed until\n    %% proto_sched:cleanup().\n    proto_sched:thread_num(1),\n    ?expect_exception(proto_sched:thread_num(2),\n                      throw, 'proto_sched:thread_num_failed'),\n    proto_sched:thread_begin(),\n    proto_sched:thread_end(),\n    proto_sched:wait_for_end(),\n    proto_sched:cleanup().\n\nbasic_cleanup_before_thread_end(_Config) ->\n    %% if a steering thread calls proto_sched:cleanup() the remaining\n    %% messages and threads finish without proto_sched and these\n    %% threads are allowed to call proto_sched;end() in an uninfected\n    %% environment.\n    proto_sched:thread_num(1),\n    Parent = self(),\n    Child =\n        spawn(fun() ->\n                      proto_sched:thread_begin(),\n                      ?TRACE(\"Sending parent\", []),\n                      %% to steer the flow control, we have to send\n                      %% directly here (uninfected)... (normally forbidden)\n                      Parent ! {ready_for_cleanup},\n                      proto_sched:thread_yield(),\n                      ?TRACE(\"Receive guard\", []),\n                      receive\n                          ?SCALARIS_RECV({hello_world},\n                                         begin\n                                             ?ASSERT(proto_sched:infected()),\n                                             ok\n                                         end)\n                          end,\n                      ?ASSERT(proto_sched:infected()),\n                      ?TRACE(\"Call thread_end\", []),\n                      proto_sched:thread_end()\n              end),\n\n    receive\n        {ready_for_cleanup} ->\n            ok\n    end,\n    ?TRACE(\"Initiating cleanup\", []),\n    proto_sched:wait_for_end(),\n    %% TODO: check statistics\n    proto_sched:cleanup(),\n    ?TRACE(\"Sending back\", []),\n    comm:send_local(Child, {hello_world}),\n    ?ASSERT(not proto_sched:infected()).\n\nbasic_cleanup_infected(_Config) ->\n    %% inside a proto_sched (before thread_end) cleanup is prohibited.\n    proto_sched:thread_num(1),\n    proto_sched:thread_begin(),\n    ?expect_exception(proto_sched:cleanup(),\n                      throw, 'proto_sched:cleanup_called_infected'),\n    proto_sched:thread_end(),\n    proto_sched:wait_for_end(),\n    _ = proto_sched:get_infos(),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed),\n    %% TODO: check statistics\n    proto_sched:cleanup().\n\nbasic_cleanup_non_existing(_Config) ->\n    %% cleanup of a non-existing traceid is prohibited.\n    ?expect_exception(proto_sched:cleanup(),\n                      throw, 'proto_sched:cleanup_trace_not_found').\n\nbasic_cleanup_DOWN_interplay(_Config) ->\n    %% cleanup before 'DOWN' detected, but delivered.\n    %% using the status field for 'to_be_cleaned' made problems with\n    %% 'DOWN' reports, so it had to be separated in a separate record field\n    proto_sched:thread_num(1),\n    Parent = self(),\n    _Child =\n        spawn(fun() ->\n                      proto_sched:thread_begin(),\n                      ?TRACE(\"Sending parent\", []),\n                      %% to steer the flow control, we have to send\n                      %% directly here (uninfected)... (normally forbidden)\n                      Parent ! {ready_for_cleanup},\n                      %% let cleanup arrive at proto_sched\n                      timer:sleep(300)\n                      %% generate a 'DOWN' report by ending this process\n              end),\n\n    receive\n        {ready_for_cleanup} ->\n            ok\n    end,\n    proto_sched:cleanup(),\n    ?ASSERT(not proto_sched:infected()).\n\nbasic_cleanup_send_interplay(_Config) ->\n    %% cleanup before 'DOWN' detected, but delivered.\n    %% using the status field for 'to_be_cleaned' made problems with\n    %% 'DOWN' reports, so it had to be separated in a separate record field\n    proto_sched:thread_num(1),\n    Parent = self(),\n    _Child =\n        spawn(fun() ->\n                      proto_sched:thread_begin(),\n                      ?TRACE(\"Sending parent\", []),\n                      %% to steer the flow control, we have to send\n                      %% directly here (uninfected)... (normally forbidden)\n                      Parent ! {ready_for_cleanup},\n                      %% let cleanup arrive at proto_sched\n                      timer:sleep(300),\n                      %% log a send\n                      comm:send_local(Parent, {child_done})\n              end),\n\n    receive\n        {ready_for_cleanup} ->\n            ok\n    end,\n    proto_sched:cleanup(),\n    %% we could/have to receive {child_done}?\n\n    ?ASSERT(not proto_sched:infected()).\n\n\n\nbasic_thread_yield_outside_proto_sched(_Config) ->\n    %% a thread_yield outside a proto_sched is prohibited.\n    ?expect_exception(proto_sched:thread_yield(),\n                      throw, 'yield_outside_thread_start_thread_end').\n\nbasic_slow_msg_handler(_Config) ->\n    %% keep exec token longer than a second. This should raise a\n    %% warning from proto_sched (some output).\n    proto_sched:thread_num(1),\n    proto_sched:thread_begin(),\n    timer:sleep(2000),\n    proto_sched:thread_end(),\n    proto_sched:cleanup(),\n    ok.\n\n\nbasic_bench_increment(_Config) ->\n    %% let run a short bench:increment with proto_sched\n    proto_sched:thread_num(1),\n    proto_sched:thread_begin(),\n    {ok, _} = bench:increment(2,2),\n    proto_sched:thread_end(),\n    proto_sched:wait_for_end(),\n    _ = proto_sched:get_infos(),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed),\n    %% TODO: check statistics\n    proto_sched:cleanup().\n\nbasic_start_bench_and_kill_it(_Config) ->\n    %% we start a long running bench:increment and kill it during\n    %% execution to see what happens with proto_sched with killed\n    %% processes. (Similarily done in dht_node_move_SUITE).\n    BenchPid = spawn(fun() ->\n                          proto_sched:thread_begin(),\n                          {ok, _} = bench:increment(10,1000),\n                          %% will be killed by steering thread\n                          util:do_throw(bench_was_to_fast)\n          end),\n\n    proto_sched:thread_num(1),\n    %% spawned process starts execution\n    timer:sleep(500),\n    erlang:exit(BenchPid, 'kill'),\n    proto_sched:wait_for_end(),\n    %% TODO: check statistics\n    proto_sched:cleanup().\n\ntest_kv_on_cseq_read(_Config) ->\n    util:for_to(1, 100, fun kv_on_cseq_read/1).\n\nkv_on_cseq_read(I) ->\n    %% read key with one thread\n    {ok} = kv_on_cseq:write(\"a\", I),\n    proto_sched:thread_num(1, I),\n    proto_sched:thread_begin(I),\n    {ok, I} = kv_on_cseq:read(\"a\"),\n    proto_sched:thread_end(I),\n    ?ASSERT(not proto_sched:infected()),\n    proto_sched:wait_for_end(I),\n    _Infos = proto_sched:get_infos(I),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed, I),\n    proto_sched:cleanup(I).\n\ntest_kv_on_cseq_read_2(_Config) ->\n    util:for_to(1, 100, fun kv_on_cseq_read_2/1).\n\nkv_on_cseq_read_2(I) ->\n    %% concurrently try to read the same key with two threads\n    {ok} = kv_on_cseq:write(\"a\", I),\n    Pid = self(),\n    spawn(fun() ->\n                  proto_sched:thread_begin(I),\n                  {ok, I} = kv_on_cseq:read(\"a\"),\n                  proto_sched:thread_end(I),\n                  Pid ! ok\n          end),\n    spawn(fun() ->\n                  proto_sched:thread_begin(I),\n                  {ok, I} = kv_on_cseq:read(\"a\"),\n                  proto_sched:thread_end(I),\n                  Pid ! ok\n          end),\n    proto_sched:thread_num(2, I),\n    ?ASSERT(not proto_sched:infected()),\n    receive ok -> ok end,\n    receive ok -> ok end,\n    proto_sched:wait_for_end(I),\n    _Infos = proto_sched:get_infos(I),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed, I),\n    proto_sched:cleanup(I).\n\ntest_kv_on_cseq_write(_Config) ->\n    util:for_to(1, 100, fun kv_on_cseq_write/1).\n\nkv_on_cseq_write(_I) ->\n    %% write key with one thread\n    proto_sched:thread_num(1, _I),\n    proto_sched:thread_begin(_I),\n    ?TRACE(\"Thread1 start write\", []),\n    {ok} = kv_on_cseq:write(\"a\", _I),\n    ?TRACE(\"Thread1 finished write\", []),\n    proto_sched:thread_end(_I),\n    proto_sched:wait_for_end(_I),\n    _Infos = proto_sched:get_infos(_I),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed, _I),\n    proto_sched:cleanup(_I).\n\ntest_kv_on_cseq_write_2(_Config) ->\n    util:for_to(1, 100, fun kv_on_cseq_write_2/1).\n\nkv_on_cseq_write_2(_I) ->\n    %% concurrently try to write the same key with two threads\n    ?TRACE(\"New intance ~p\", [_I]),\n\n    Pid = self(),\n    spawn(fun() ->\n                  proto_sched:thread_begin(_I),\n                  ?TRACE(\"Thread1 start write ~p\", [_I]),\n                  {ok} = kv_on_cseq:write(\"a\", 7),\n                  ?TRACE(\"Thread1 finished write ~p\", [_I]),\n                  proto_sched:thread_end(_I),\n                  Pid ! ok\n          end),\n    spawn(fun() ->\n                  proto_sched:thread_begin(_I),\n                  ?TRACE(\"Thread2 start write ~p\", [_I]),\n                  {ok} = kv_on_cseq:write(\"a\", 8),\n                  ?TRACE(\"Thread2 finished write ~p\", [_I]),\n                  proto_sched:thread_end(_I),\n                  Pid ! ok\n          end),\n    proto_sched:thread_num(2, _I),\n    ?ASSERT(not proto_sched:infected()),\n    receive ok -> ok end,\n    receive ok -> ok end,\n    ?TRACE(\"Both done\", []]),\n    proto_sched:wait_for_end(_I),\n    _Infos = proto_sched:get_infos(_I),\n    unittest_helper:print_proto_sched_stats(at_end_if_failed, _I),\n    proto_sched:cleanup(_I).\n\ntest_qwrite_qwrite_qread(_Config) ->\n    util:for_to(1, 100, fun qwrite_qwrite_qread/1).\n\nqwrite_qwrite_qread(_I) ->\n    DB = {lease_db, 1},\n    ContentCheck = fun content_check/3,\n    Self = comm:reply_as(self(), 2, {test_rbr, '_'}),\n    Id = ?RT:hash_key(\"1\"),\n    Value1 = value_1,\n    Value2 = value_2,\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% begin test\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n    spawn(fun() ->\n                  proto_sched:thread_begin(),\n                  rbrcseq:qwrite(DB, Self, Id, ?MODULE, ContentCheck, Value1),\n                  ?TRACE(\"Thread 1 done\", []),\n                  proto_sched:thread_end()\n          end),\n    spawn(fun() ->\n                  proto_sched:thread_begin(),\n                  rbrcseq:qwrite(DB, Self, Id, ?MODULE, ContentCheck, Value2),\n                  ?TRACE(\"Thread 2 done\", []),\n                  proto_sched:thread_end()\n          end),\n    proto_sched:thread_num(3),\n    proto_sched:thread_begin(),\n    ?TRACE(\"Receive 1\", []),\n    receive_answer(),\n    ?TRACE(\"Receive 2\", []),\n    receive_answer(),\n    ?TRACE(\"Done\", []),\n    proto_sched:thread_end(),\n\n    proto_sched:wait_for_end(),\n    Infos = proto_sched:get_infos(),\n    %log:log(\"~.0p\", [Infos]),\n    proto_sched:cleanup(),\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% end test\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n    rbrcseq:qread(DB, Self, Id, l_on_cseq),\n    {test_rbr, {qread_done, _, _, _, Val}} = receive_answer(),\n    %ct:pal(\"got ~p\", [Val]),\n    %ct:pal(\"~p\", [Infos]),\n    case Val of\n        Value1 -> ok;\n        Value2 -> ok;\n        X ->\n            ct:pal(\"~p\", [Infos]),\n            ?ct_fail(\"unexpected result ~p\", X)\n    end,\n    ok.\n\nreceive_answer() ->\n    trace_mpath:thread_yield(),\n    receive\n        ?SCALARIS_RECV(X1, X1)\n        end.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% utlities\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ncontent_check(_Value, _WriteFilter, _Cookie) ->\n    {true, null}.\n"
  },
  {
    "path": "test/proto_sched_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{suites, tests, [proto_sched_SUITE]}.\n\n{suites, tests, [api_tx_proto_sched_SUITE]}.\n{suites, tests, [dht_node_move_proto_sched_SUITE]}.\n{suites, tests, [join_leave_proto_sched_SUITE]}.\n{suites, tests, [mr_proto_sched_SUITE]}.\n{suites, tests, [rrepair_proto_sched_SUITE]}.\n\n{groups, tests, crdt_SUITE, [proto_sched_group]}.\n\n"
  },
  {
    "path": "test/random_bias.erl",
    "content": "% @copyright 2011-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @doc    biased random number generator\n%% @end\n%% @version $Id$\n-module(random_bias).\n-author('malange@informatik.hu-berlin.de').\n\n-export([binomial/2]).\n-export([next/1]).\n-export([numbers_left/1]).\n\n% for tester:\n-export([tester_create_generator/3]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% type definitions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-export_type([generator/0]).\n\n-type approx() :: none | {normal, M::float(), Dev::float(), A::float()}.\n\n-type binomial_state() :: {binom,\n                           N         :: pos_integer(),\n                           P         :: float(),             %only works for ]0,1[\n                           X         :: non_neg_integer(),\n                           Approx    :: approx()\n                          }.\n\n-type distribution_state() :: binomial_state(). %or others\n-type generator() :: { State       :: distribution_state(),\n                       CalcFun     :: fun((distribution_state()) -> float()),\n                       NewStateFun :: fun((distribution_state()) -> distribution_state() | exit)}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% API\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Creates a new generator state for a binomial distribution for use with\n%%      next/1.\n%%      Note: a binomial distribution will generate (N+1) values!\n-spec binomial(N::pos_integer(), P::float()) -> generator().\nbinomial(N, P) when P > 0 andalso P < 1 ->\n    {{binom, N, P, 0, approx_valid(N, P)},\n     fun calc_binomial/1, fun next_state/1}.\n\n%% @doc Gets the next random value (and an updated state).\n-spec next(generator()) -> {ok, float(), generator()} | {last, float(), exit}.\nnext({DS, CalcFun, NextFun}) ->\n    V = CalcFun(DS),\n    case NextFun(DS) of\n        exit -> {last, V, exit};\n        NewDS -> {ok, V, {NewDS, CalcFun, NextFun}}\n    end.\n\n-spec numbers_left(generator()) -> pos_integer().\nnumbers_left({{binom, N, _P, X, _Approx}, _CalcFun, _NextFun}) ->\n    N + 1 - X.\n\n-define(SQRT_2_PI, 2.5066282746310002). % math:sqrt(2 * math:pi()).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Internal Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% -spec calc_normal(X::float(), M::float(), Dev::float()) -> float().\n%% calc_normal(X, M, Dev) ->\n%%     A = 1 / (Dev * ?SQRT_2_PI),\n%%     calc_normal(X, M, Dev, A).\n\n-spec calc_normal(X::float(), M::float(), Dev::float(), A::float()) -> float().\ncalc_normal(X, M, Dev, A) ->\n    B = -1/2 * math:pow(((X-M) / Dev), 2),\n    A * math:exp(B).\n\n-spec calc_binomial(binomial_state()) -> float().\ncalc_binomial({binom, _N, _P, X, _Approx = {normal, M, Dev, A}}) ->\n    calc_normal(X, M, Dev, A);\ncalc_binomial({binom, N, P, X, _Approx = none}) ->\n    try begin\n            NOverX = mathlib:binomial_coeff(N, X),\n            Pow = math:pow(P, X) * math:pow(1 - P, N - X),\n            if Pow == 0 ->\n                   % rather use approximation because we lost precision\n                   calc_binomial({binom, N, P, X, normal_approx_of_binomial(N, P)});\n               true -> NOverX * Pow\n            end\n        end\n    catch _:_ ->\n              % use approximation instead because cannot calculate the result exactly:\n              % log:pal(\"calc_binomial(~p, ~p, ~p)~n  ~p\", [N, P, X, erlang:get_stacktrace()])\n              calc_binomial({binom, N, P, X, normal_approx_of_binomial(N, P)})\n    end.\n\n-spec next_state(distribution_state()) -> distribution_state() | exit.\nnext_state({binom, X, _P, X, _}) ->\n    exit;\nnext_state({binom, N, P, X, Approx}) -> \n    {binom, N, P, X + 1, Approx}.\n\n% @doc approximation is good if this conditions hold\n%      SRC: http://www.vosesoftware.com/ModelRiskHelp/index.htm#Distributions/Approximating_one_distribution_with_another/Approximations_to_the_Binomial_Distribution.htm\n-spec approx_valid(pos_integer(), float()) -> approx().\napprox_valid(_N, 0) -> none;\napprox_valid(_N, 1) -> none;\napprox_valid(N, P) ->\n    if (N > ((9 * P) / (1 - P))) andalso (N > ((9 * (1 - P)) / P)) ->\n           normal_approx_of_binomial(N, P);\n       true ->\n           none\n    end.\n\n-spec normal_approx_of_binomial(N::pos_integer(), P::float())\n        -> {normal, M::float(), Dev::float(), A::float()}. \nnormal_approx_of_binomial(N, P) ->\n    Dev = math:sqrt(N * P * (1 - P)),\n    {normal, N * P, Dev, 1 / (Dev * ?SQRT_2_PI)}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Tester\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec tester_create_generator(pos_integer(), 1..1000000, 1..1000000)\n        -> generator().\ntester_create_generator(N, P1, P2) when P2 > P1 ->\n    case binomial(N, P1 / P2) of\n        {{binom, N, P, X, Approx = none}, CalcFun, NewStateFun} ->\n            % too high factorials slow down the tests too much\n            {{binom, erlang:min(50, N), P, X, Approx}, CalcFun, NewStateFun};\n        X -> X\n    end;\ntester_create_generator(N, P1, P2) when P2 < P1 ->\n    tester_create_generator(N, P2, P1);\ntester_create_generator(N, P, P) ->\n    tester_create_generator(N, 999999, 1000000).\n\n"
  },
  {
    "path": "test/random_bias_SUITE.erl",
    "content": "%  @copyright 2010-2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de\n%% @doc    Tests for random_bias module.\n%% @end\n%% @version $Id$\n-module(random_bias_SUITE).\n-author('malange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nall() ->\n    [test1,\n     test2,\n     tester_sum_test,\n     tester_value_count].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 20}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ntest1(_) ->\n    N = 5,\n    P = 0.3,\n    R = random_bias:binomial(N, P),\n    Vals = lists:reverse(gen_values(R, [])),\n    EV = expected_value(Vals),\n    ?equals_w_note(1, trunc(EV), io_lib:format(\"EV=~p~nVals=~p\", [EV, Vals])).\n\ntest2(_) ->\n    N = 10,\n    P = 2/7,\n    R = random_bias:binomial(N, P),\n    Vals = lists:reverse(gen_values(R, [])),\n    EV = expected_value(Vals),\n    ct:pal(\"Binomial N = ~p ; P = ~p~nResult=~p~nSum=~p~nExpectedValue=~p\", \n           [N, P, Vals, lists:sum(Vals), EV]),\n    ?assert(EV > 2.85) andalso ?assert(EV < 2.87).\n\n-spec prop_sum_test(1..100000, 1..1000000, 1..1000000) -> true.\nprop_sum_test(N, P1, P1) ->\n    prop_sum_test(N, P1 - 1, P1);\nprop_sum_test(N0, P1, P2) ->\n    P = ?IIF(P2 > P1, P1 / P2, P2 / P1),\n    % similar as in random_bias:tester_create_generator/3:\n    R = case random_bias:binomial(N0, P) of\n            {{binom, N0, P, X, Approx = none}, CalcFun, NewStateFun} ->\n                % too high factorials slow down the tests too much\n                N = erlang:min(50, N0),\n                {{binom, N, P, X, Approx}, CalcFun, NewStateFun};\n            X ->\n                N = N0,\n                X\n        end,\n    Vals = gen_values(R, []),\n    Sum = lists:sum(Vals),\n    ?compare(fun erlang:'=<'/2, Sum, 1.0 + 1.0e12),\n    ?compare(fun erlang:'>='/2, Sum, 0.99),\n    N2 = lists:foldl(fun(V, Acc) -> Acc + (V * N) end, 0, Vals),\n    ?compare(fun erlang:'=<'/2, N2, N + 1.0e12),\n    ?compare(fun erlang:'>='/2, N2, 0.99 * N).\n\ntester_sum_test(_) ->\n    prop_sum_test(20, 816034, 257824),\n    tester:register_value_creator({typedef, random_bias, generator, []},\n                                  random_bias, tester_create_generator, 3),\n    tester:test(?MODULE, prop_sum_test, 3, 100, [{threads, 4}]),\n    tester:unregister_value_creator({typedef, random_bias, generator, []}).\n\n-spec prop_value_count(1..100000) -> true.\nprop_value_count(Count) ->\n    R = random_bias:binomial(Count, 0.3),\n    Values = gen_values(R, []),\n    Len = length(Values),\n    ?equals_w_note(Count + 1, Len, io_lib:format(\"Count = ~p - Generated=~p\", [Count, Len])).\n\ntester_value_count(_) ->\n    tester:test(?MODULE, prop_value_count, 1, 50, [{threads, 4}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% helpers\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec gen_values(random_bias:generator(), [float()]) -> [float()].\ngen_values(RanGen, Acc) ->\n    case random_bias:next(RanGen) of\n        {ok, V, RanGen1} -> gen_values(RanGen1, [V | Acc]);\n        {last, V, exit}  -> [V | Acc]\n    end.\n\n-spec expected_value([float()]) -> float().\nexpected_value(List) ->\n    {EV, _} = lists:foldl(fun(P, {Sum, K}) -> {Sum + P * K, K + 1} end, {0, 0}, List),\n    EV.\n\n"
  },
  {
    "path": "test/rbr_interleaving_SUITE.erl",
    "content": "%% @copyright 2013-2019 Zuse Institute Berlin\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Jan Skrzypczak <skrzypczak@zib.de>\n%% @doc    Unit tests for rbrcseq which simulate specific interleaving of messages.\n%%         The purpose of this suite is to reliably replicate rare corner-cases by slowing\n%%         downe specific links during certain times during the tests.\n%%         The tests depend on the gen_component breakpoint mechanism to delay specific messages.\n%%         Each test case assumes a specific replication factor (usually 4).\n%% @end\n%% @version $Id$\n-module(rbr_interleaving_SUITE).\n-author('skrzypczak@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-define(TRACE(X,Y), ok).\n%-define(TRACE(X,Y), ct:pal(X, Y)).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\nall() -> [\n            test_link_slowing,\n            test_link_slowing2,\n            test_interleaving,\n            test_write_once_1,\n            test_write_once_2,\n            test_write_once_3,\n            test_read_retry_returns_older,\n            test_read_retry_returns_newer,\n            test_read_write_commuting\n         ].\n\nsuite() -> [ {timetrap, {seconds, 400}} ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_testcase(_TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_symmetric_ring([{config, [{log_path, PrivDir}, {replication_factor, 4}]}]),\n    unittest_helper:check_ring_size_fully_joined(config:read(replication_factor)),\n    [{stop_ring, true} | Config].\n\ntest_link_slowing(_Config) ->\n    %% Slow down a link. One replica should not receive any prbr messages during the\n    %% test. After the slow link is removed (messages will be flushed), all replicase should\n    %% be consistent. Assumes R=4.\n    get_notified_by_message(1, 2, write),\n    Link = slow_link(1, 2),\n    {ok, _} = write_via_node(1, \"1\", filter_list_append(), \"TestWrite\"),\n\n    %% replica 2 should be empty, and the other three have the value written\n    ?equals(prbr_values(), [[[\"TestWrite\"]],\n                            [],\n                            [[\"TestWrite\"]],\n                            [[\"TestWrite\"]]]),\n\n    remove_slow_link(Link),\n    _= wait_until_notification(1),\n\n    %% all replicas should have received the written value\n    ?equals(prbr_values(), [[[\"TestWrite\"]],\n                            [[\"TestWrite\"]],\n                            [[\"TestWrite\"]],\n                            [[\"TestWrite\"]]]).\n\ntest_link_slowing2(_Config) ->\n    %% slow down one link, but use a different client to send write request\n    %% slow link should have no impact. Assumes R=4.\n    _Link = slow_link(1, 2),\n\n    get_notified_by_message(2, [1,2,3,4], write),\n    {ok, _} = write_via_node(2, \"1\", filter_list_append(), \"TestWrite\"),\n    _= wait_until_notification(4),\n\n    %% all replicas should have received the written value\n    ?equals(prbr_values(), [[[\"TestWrite\"]],\n                            [[\"TestWrite\"]],\n                            [[\"TestWrite\"]],\n                            [[\"TestWrite\"]]]).\n\ntest_interleaving(_Config) ->\n    %% This test simulates the following interleaving of operations:\n    %% (4 nodes with R=4, the nodes are called 1,2,3,4)\n    %%\n    %% Three requests are made from three different clients.\n    %% 1. Client A Starts a write operation, but has only written replica\n    %%      on node 1 so far (has read all replicas in its read phase)\n    %% 2. Client B Executes a read which only has read replicas 2,3,4 yet\n    %%      (read has returned since majority replied)\n    %% 3. Client C Executes a write. In its read phase and it gets replies\n    %%      from 2,3,4 first; After that write on every replica\n\n    Key = \"A\",\n\n    %% write of client A\n    get_notified_by_message(1, [2,3,4], round_request),\n    get_notified_by_message(1, 1, write),\n    _ = slow_link(1, [2,3,4], write),\n    spawn(fun() -> write_via_node(1, Key, filter_list_append(), \"WriteA\") end),\n    _= wait_until_notification(4),\n\n    %% read of client B\n    _LinkB = slow_link(2, 1),\n    {ok, _} = read_via_node(2, Key, element(1, filter_list_append())),\n\n    %% write of client C\n    get_notified_by_message( 3, 1, write),\n    LinkC = slow_link(3, 1),\n    {ok, _} = write_via_node(3, Key, filter_list_append(), \"WriteB\"),\n    remove_slow_link(LinkC),\n    _ = wait_until_notification(1),\n\n    ct:pal(\"PRBR state after interleaved operations: ~n~p\", [prbr_data()]),\n    %% Test that there aren't two different values\n    %% with the same write round.\n    PrbrData = prbr_w_rounds_with_values(),\n    ValList = lists:usort(lists:flatten(PrbrData)),\n    case ValList of\n        [A, B] ->\n            ?compare_w_note(fun(E1, E2) -> element(1, E1) =/= element(1, E2) end,\n                            A, B, \"Same write round for different values!\");\n        [_A] -> ok;\n        _ ->\n            ct:fail(\"More than two different values/rounds! ~nprbr data:~n~p\", [PrbrData])\n    end,\n\n    %% Do a read over replica 1, 2, 3\n    %% It should be an inconsistent read and currently diverging replica 1\n    %% should be repaired,\n    get_notified_by_message(4, 4, write),\n    LinkD = slow_link(4, 4),\n    {ok, _} = read_via_node(4, Key, element(1, filter_list_append())),\n    remove_slow_link(LinkD),\n    _ = wait_until_notification(1),\n\n    ct:pal(\"PRBR state after inconsistent read: ~n~p\", [prbr_data()]),\n    PrbrData2 = prbr_values(),\n    ValList2 = lists:usort(PrbrData2),\n    %% do not compare write rounds as the write messages to acceptor 4 might not\n    %% have been received and processed yet.\n    ?equals_w_note(length(ValList2), 1, \"All replicas should have the same value\").\n\ntest_write_once_1(_Config) ->\n    %% This tests case tests if a write through correctly notifies the original proposer\n    %% of the write.\n    %% Write A is started, but does not finish because two write messages are delayed.\n    %% A subsequent write (or read) should detect the write in progress and triggers a\n    %% write through which will finish write A before B is started.\n    TestPid = self(),\n    Key = \"1234\",\n\n    get_notified_by_message(1, [1,2], write),\n    _ = slow_link(1, 3, write),\n    _ = slow_link(1, 4, write),\n\n    % start write A which will not finish since it only gets two write ack.\n    spawn(fun() ->\n            {ok, _} = write_via_node(1, Key, filter_list_append(), \"WriteA\"),\n            TestPid ! {write_a_done}\n          end),\n\n    % wait until A has written the two remaining replicas\n    _ = wait_until_notification(2),\n\n    % write B should now trigger a write through which should finish write A\n    % once write B retries, a notification should be sent to write A\n    {ok, _} = write_via_node(2, Key, filter_list_append(), \"WriteB\"),\n\n    receive {write_a_done} -> ok\n    after 10000 ->\n        ?ct_fail(\"Write through has not notified original proposer in a timely manner\", [])\n    end,\n    {ok, Value} = read_via_node(3, Key, element(1, filter_list_append())),\n    ?equals_w_note(Value, [\"WriteB\", \"WriteA\"], \"Values must match exactly due to interleaving\").\n\ntest_write_once_2(_Config) ->\n    %% This tests does the following:\n    %% There is an unsuccessfull write A that writes on replica 4\n    %% Write B is successful by writing on replica 1-3. but the proposer retries\n    %% because it receives the deny message from replica 4 first.\n    %% Write C sees consistent quorum 1-3 and proposes its follow-up value\n    %% which is written on all replicas\n    %% Write B retries but it should be notified before sending write messages.\n    %% --> Write B should only be written once\n\n    TestPid = self(),\n    Key = \"123\",\n\n    print_prbr_data(),\n\n    %% Write A: phase 1 reaches all replicas, write msg only replica 4\n    get_notified_by_message(1, 4, write),\n    get_notified_by_message(1, [1,2,3,4], round_request),\n    _ = slow_link(1, [1,2,3], write),\n    spawn(fun() -> _ = write_via_node(1, Key, filter_list_append(), \"WriteA\") end),\n    _ = wait_until_notification(5),\n\n    print_prbr_data(),\n    ct:pal(\"write A done\"),\n\n    %% Write B: writes replica 1 to 3. gets deny from replica 4 before quorum of acks\n    %% --> it retries its request\n    get_notified_by_message(2, [1,2,3,4], round_request),\n    get_notified_by_message(2, [1,2,3,4], read),\n    get_notified_by_message(2, [1,2,3,4], write),\n\n    SlowLinksB1 = slow_link(2, 4, round_request),\n    SlowLinksB3 = slow_link(2, 3, write),\n    SlowLinksB4 = slow_link(2, 2, read),\n    spawn(fun() -> _ =\n            write_via_node(2, Key, filter_list_append(), \"WriteB\"),\n            TestPid ! {write_b_done}\n          end),\n    _ = wait_until_notification(5),\n    print_prbr_data(),\n\n    ct:pal(\"flush slow link 4\"),\n    flush_slow_link(SlowLinksB1),\n    _ = wait_until_notification(3),\n    print_prbr_data(),\n\n    ct:pal(\"flush slow link 3\"),\n    flush_slow_link(SlowLinksB3),\n    _ = wait_until_notification(2),\n    print_prbr_data(),\n\n    %% Write C: observer consistent quorum from repliica 1 to 3\n    %% write any follow-up value to all replicas\n    get_notified_by_message(3, [1,2,3,4], write),\n    SlowLinkC = slow_link(3, 4, round_request),\n    _ = write_via_node(3, Key, filter_list_append(), \"WriteC\"),\n    remove_slow_link(SlowLinkC),\n    _ = wait_until_notification(4),\n    ct:pal(\"write C done\"),\n    print_prbr_data(),\n\n    %% Write B: should not be able to retry as it gets notified earlier\n    ct:pal(\"remove slowness from write b\"),\n    remove_slow_link(SlowLinksB4),\n    remove_slow_link(SlowLinksB3),\n    remove_slow_link(SlowLinksB1),\n\n    receive {write_b_done} -> ok end,\n    print_prbr_data(),\n\n    %% Read and check if WriteB exists only once\n    {ok, Value} = read_via_node(4, Key, element(1, filter_list_append())),\n    ?equals_w_note(Value, [\"WriteC\", \"WriteB\"], \"Values must match exactly with this interleaving\"),\n\n    ok.\n\ntest_write_once_3(_Config) ->\n    %% This tests does the following:\n    %% There is an unsuccessfull write A that writes on replica 4\n    %% Write B is successful by writing on replica 1-3. but the proposer retries\n    %% because it receives the deny message from replica 4 first.\n    %% Write B retries but it should be see that it has consistent quorum from\n    %% previous attempt\n    %% --> Write B should only be written once\n\n    TestPid = self(),\n    Key = \"123\",\n\n    print_prbr_data(),\n\n    %% Write A: phase 1 reaches all replicas, write msg only replica 4\n    get_notified_by_message(1, [1,2,3,4], round_request),\n    get_notified_by_message(1, 4, write),\n    _ = slow_link(1, [1,2,3], write),\n    spawn(fun() -> _ = write_via_node(1, Key, filter_list_append(), \"WriteA\") end),\n    _ = wait_until_notification(5),\n\n    print_prbr_data(),\n    ct:pal(\"write A done\"),\n\n    %% Write B: writes replica 1 to 3. gets deny from replica 4 before quorum of acks\n    %% --> it retries its request\n    get_notified_by_message(2, [1,2,3,4], round_request),\n    get_notified_by_message(2, [1,2,3,4], read),\n    get_notified_by_message(2, [1,2,3,4], write),\n\n    SlowLinksB1 = slow_link(2, 4, round_request),\n    SlowLinksB3 = slow_link(2, 3, write),\n    SlowLinksB4 = slow_link(2, 2, read),\n    spawn(fun() -> _ =\n            write_via_node(2, Key, filter_list_append(), \"WriteB\"),\n            TestPid ! {write_b_done}\n          end),\n    _ = wait_until_notification(5),\n    print_prbr_data(),\n\n    ct:pal(\"flush slow link 4\"),\n    flush_slow_link(SlowLinksB1),\n    _ = wait_until_notification(3),\n    print_prbr_data(),\n\n    ct:pal(\"flush slow link 3\"),\n    flush_slow_link(SlowLinksB3),\n    _ = wait_until_notification(2),\n    print_prbr_data(),\n\n    %% Write B: should not be able to retry as it gets notified earlier\n    ct:pal(\"remove slowness from write b\"),\n    remove_slow_link(SlowLinksB4),\n    remove_slow_link(SlowLinksB3),\n    remove_slow_link(SlowLinksB1),\n\n    receive {write_b_done} -> ok end,\n    print_prbr_data(),\n\n    %% Read and check if WriteB exists only once\n    {ok, Value} = read_via_node(4, Key, element(1, filter_list_append())),\n    ?equals_w_note(Value, [\"WriteB\"], \"Values must match exactly with this interleaving\"),\n\n    ok.\n\ntest_read_retry_returns_older(_Config) ->\n    %% This test checks if a delayed round requests responses properly complete\n    %% a read request if an inconsistent quorum was already established.\n    %% Setup:\n    %%  A: Write value to replica 1\n    %%  B: Submit read... this will observer only replies from replica 1-3\n    %%      thus moving from round_request to read phase.\n    %%     Afterwards, B receives replies from replica 4.. thus 3 replies with no value in total\n    %%     B should deliver an empty value without triggering a write-through\n    TestPid = self(),\n    Key = \"123\",\n\n    % Write A: Write only to replica 1\n    ct:pal(\"partial write to replica 1\"),\n    print_prbr_data(),\n    get_notified_by_message(1, 1, write),\n    _ = slow_link(1, [2, 3, 4], write),\n    spawn(fun() -> write_via_node(1, Key, filter_list_append(), \"WriteA\") end),\n    _ = wait_until_notification(1),\n    print_prbr_data(),\n\n    %% Execute read B\n    ct:pal(\"inconsistent read\"),\n    get_notified_by_message(2, 1, read),\n    _ = slow_link(2, [2,3,4], read),\n    LinkC = slow_link(2, 4, round_request),\n    spawn(fun() ->\n        {ok, Value} = read_via_node(2, Key, element(1, filter_list_append())),\n            TestPid ! {read_req_done, Value}\n        end),\n    % message_received -> internally seen inconsistent quorum and moved to read stage\n    _ = wait_until_notification(1),\n    print_prbr_data(),\n    %% deliver slow messages from replica 4\n    ct:pal(\"remove slow link\"),\n    remove_slow_link(LinkC),\n    print_prbr_data(),\n    receive {read_req_done, Value} ->\n        ?equals_w_note(Value, [], \"Values must match exactly with this interleaving\")\n    end,\n    %% check that no write-through happened (max write round of 1)\n    WriteThroughRounds = lists:filter(fun(E) -> E > 1 end, lists:flatten(prbr_w_rounds())),\n    ?equals_w_note(WriteThroughRounds, [], \"No write through should have happend\"),\n    ok.\n\ntest_read_retry_returns_newer(_Config) ->\n    %% This test checks if a delayed round requests responses properly complete\n    %% a read request if an inconsistent quorum was already established.\n    %% Setup:\n    %%  A: Write value to replica 1-3\n    %%  B: Submit read... this will observer only replies from replica 2-4\n    %%      thus moving from round_request to read phase.\n    %%     Afterwards, B receives replies from replica 1.. thus 3 replies with new value in total\n    %%     B should deliver the new value without triggering a write-through\n    TestPid = self(),\n    Key = \"123\",\n\n    % Write A: Write to replica 1-3\n    ct:pal(\"write to replica 1-3\"),\n    print_prbr_data(),\n    get_notified_by_message(1, [1,2,3], write),\n    _ = slow_link(1, 4, write),\n    spawn(fun() -> write_via_node(1, Key, filter_list_append(), \"WriteA\") end),\n    _ = wait_until_notification(3),\n    print_prbr_data(),\n\n    %% Execute read B\n    ct:pal(\"inconsistent read\"),\n    get_notified_by_message(2, 4, read),\n    _ = slow_link(2, [1,2,3], read),\n    LinkC = slow_link(2, 1, round_request),\n    spawn(fun() ->\n        {ok, Value} = read_via_node(2, Key, element(1, filter_list_append())),\n            TestPid ! {read_req_done, Value}\n        end),\n\n    % message_received -> internally seen inconsistent quorum and moved to read stage\n    _ = wait_until_notification(1),\n    print_prbr_data(),\n\n    %% deliver slow messages from replica 1\n    ct:pal(\"remove slow link\"),\n    remove_slow_link(LinkC),\n    print_prbr_data(),\n    receive {read_req_done, Value} ->\n        ?equals_w_note(Value, [\"WriteA\"], \"Values must match exactly with this interleaving\")\n    end,\n    %% check that no write-through happened (max write round of 1)\n    WriteThroughRounds = lists:filter(fun(E) -> E > 1 end, lists:flatten(prbr_w_rounds())),\n    ?equals_w_note(WriteThroughRounds, [], \"No write through should have happend\"),\n    ok.\n\ntest_read_write_commuting(_Config) ->\n    %% Write tuple {A,B} on every replica. Update second element to C for\n    %% 3 out of 4 replicas. Read first element and make sure the outdated replica\n    %% is included. Since operations commute, no write-through should have been\n    %% triggered.\n    Key = \"123\",\n\n    % write baseline\n    get_notified_by_message(1, [1,2,3,4], write),\n    _ = write_via_node(1, Key, {fun prbr:noop_read_filter/1,\n                                fun ?MODULE:cc_noop/3,\n                                fun prbr:noop_write_filter/3},\n                       {\"A\", \"B\"}),\n    _ = wait_until_notification(4),\n\n    _ = slow_link(1, 4, write),\n    _ = write_via_node(1, Key, {fun ?MODULE:rf_second/1,\n                                fun ?MODULE:cc_noop/3,\n                                fun ?MODULE:wf_second/3}, \"C\"),\n\n    PrbrValuesBeforeRead = prbr_values(),\n    _ = slow_link(4, 1),\n    {ok, \"A\"} = read_via_node(4, Key, fun ?MODULE:rf_first/1),\n    PrbrValuesAfterRead = prbr_values(),\n\n    ?equals_w_note(PrbrValuesBeforeRead, PrbrValuesAfterRead,\n                   \"Read was independent from write and thus should not have caused a \"\n                   \"write through\"),\n    ok.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Helper functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec get_commuting_wf_for_rf(prbr:read_filter()) ->\n        [prbr:write_filter()].\nget_commuting_wf_for_rf(ReadFilter) ->\n    {name, Name} = erlang:fun_info(ReadFilter, name),\n    {module, Module} = erlang:fun_info(ReadFilter, module),\n    case {Module, Name} of\n        {?MODULE, rf_first} ->\n            [fun ?MODULE:wf_second/3];\n        _ ->\n            []\n    end.\n\n-spec rf_first(prbr_bottom | {any(), any()}) -> any().\nrf_first(prbr_bottom) -> prbr_bottom;\nrf_first({A, _B}) -> A.\n-spec rf_second(prbr_bottom | {any(), any()}) -> any().\nrf_second(prbr_bottom) -> prbr_bottom;\nrf_second({_A,B}) -> B.\n\n-spec cc_noop(any(), any(), any()) ->\n        {true, none}.\ncc_noop(_, _, _) -> {true, none}.\n-spec wf_second({any(), any()}, any(), any()) ->\n        {{any(), any()}, none}.\nwf_second({A, _B}, _UI, WriteVal) -> {{A, WriteVal}, none}.\n\n%% @doc Simple set of filter which append the given value to a list\nfilter_list_append() ->\n    RF = fun (prbr_bottom) -> [];\n             (DBEntry) -> DBEntry\n         end,\n    CC = fun (_ReadVal_, _WF, _WriteVal) -> {true, none} end,\n    WF = fun (prbr_bottom, _UI, WriteVal) -> {[WriteVal], none};\n             (DBEntry, _UI, WriteVal) -> {[WriteVal | DBEntry], none}\n         end,\n    {RF, CC, WF}.\n\n%% @doc Sends a read requests via node number ViaKvNr (lexicographically order by pid).\n%% Blocks until read is done.\nread_via_node(ViaKvNr, Key, ReadFilter) ->\n    Pid = nth(ViaKvNr, kv_db),\n    Msg = {qround_request, self(), '_', ?RT:hash_key(Key), ?MODULE, ReadFilter, read, 1},\n    comm:send_local(Pid, {request_init, _ClinetPosInMsg=2, _OpenReqPos=3, Msg}),\n    receive\n        ?SCALARIS_RECV({qread_done, _, _, _, Value}, {ok, Value})\n    end.\n\n%% @doc Sends a write requests via node number ViaKvNr (lexicographically order by pid).\n%% Blocks until write is done.\nwrite_via_node(ViaKvNr, Key, Filter, Value) ->\n    Pid = nth(ViaKvNr, kv_db),\n    Msg = {qwrite, self(), '_', ?RT:hash_key(Key), ?MODULE, Filter, Value, 20},\n    comm:send_local(Pid, {request_init, _ClientPos=2, _OpenReqPos=3, Msg}),\n    receive\n        ?SCALARIS_RECV({qwrite_done, _, _, _, RetValue}, {ok, RetValue});\n        ?SCALARIS_RECV({qwrite_deny, _ReqId, _NextFastWriteRound, _Value, Reason}, Reason)\n    end.\n\n%% @doc Notifies process PidToNotify if process nth(ToId, ToType) received a message\n%% of type MessageType from process nth(FromId, FromType).\n%% ATTENTION: If the corresponding link is slowed by slow_link/[4,5,6] this method must be called\n%% BEFORE slow_link. Otherwise two notifications might be received for the same message.\n%% Todo? (Works only for ToType=dht_node so far).\nget_notified_by_message(PidToNotify, FromId, FromType, ToId, ToType, MessageType) ->\n    BpName = bp_name(\"notify_\" ++ atom_to_list(MessageType), FromId, FromType, ToId, ToType),\n    ToPid = nth(ToId, ToType),\n    NotifyFun = notify_fun(PidToNotify, nth(FromId, FromType), ToPid,\n                            ToType, MessageType, BpName),\n    gen_component:bp_set_cond(ToPid, NotifyFun, BpName).\n\n%% @doc Wrapper for get_notified_by_message/6\nget_notified_by_message(FromId, ToIds, MessageType) ->\n    get_notified_by_message(self(), FromId, ToIds, MessageType).\n    \nget_notified_by_message(PidToNotify, FromId, ToIds, MessageType) when is_list(ToIds) ->\n    [get_notified_by_message(PidToNotify, FromId, ToId, MessageType) || ToId <- ToIds],\n    ok;\nget_notified_by_message(PidToNotify, FromId, ToId, MessageType) ->\n    get_notified_by_message(PidToNotify, FromId, kv_db, ToId, dht_node, MessageType).\n\nwait_until_notification(NotificationCount) ->\n    _ = [receive {message_received} -> ok end || _ <- lists:seq(1, NotificationCount)].\n\nnotify_fun(PidToNotify, FromPid, ToPid, _ToType=dht_node, MessageType, BpName) ->\n    fun(Msg, _State) ->\n        case Msg of\n            _ when element(1, Msg) =:= prbr andalso\n                    element(2, Msg) =:= MessageType andalso\n                    element(3, element(1, element(5, Msg))) =:= FromPid ->\n                ?TRACE(\"Notify ~p message on ~p received: ~n~p\", [PidToNotify, ToPid, Msg]),\n                gen_component:bp_del(ToPid, BpName),\n                comm:send_local(PidToNotify, {message_received}),\n                false;\n            _ -> false\n        end\n    end.\n\n%% @doc Helper for test outputs\nprint_prbr_data() ->\n    ct:pal(\"PRBR state: ~n~p\", [prbr_data()]).\n\n%% @doc Gets all information stored in prbr for all nodes.\nprbr_data() ->\n     [begin\n        comm:send_local(N, {prbr, tab2list_raw, kv_db, self()}),\n        receive\n            {kv_db, List} -> List\n        end\n      end || N <- lists:sort(pid_groups:find_all(dht_node))].\n\n%% @doc Returns all value for each node.\nprbr_values() ->\n    [\n        [prbr:entry_val(E) || E <- Replica]\n    || Replica <- prbr_data()].\n\n%% @doc Returns all {write_round, value} tuples for each node.\n%%      Removes write_through infos\nprbr_w_rounds_with_values() ->\n    [\n        [{pr:set_wti(element(3, E), none), prbr:entry_val(E)} || E <- Replica]\n    || Replica <- prbr_data()].\n\n%% @doc Returns all write round numbers for each node as a list.\nprbr_w_rounds() ->\n    [\n        [pr:get_r(element(3, E)) || E <- Replica] \n    || Replica <- prbr_data()].\n\n%% @doc Flush all slow messages of a link\nflush_slow_link({_BPName, LoopPid, _Node}) ->\n    comm:send_local(LoopPid, {flush}).\n\n\n%% @doc Stops slowing messages down and flushes message queue.\nremove_slow_link([]) -> ok;\nremove_slow_link(LinkList) when is_list(LinkList) ->\n    [remove_slow_link(Link) || Link <- LinkList],\n    ok;\nremove_slow_link({BPName, LoopPid, Node}) ->\n    gen_component:bp_del(Node, BPName),\n    comm:send_local(LoopPid, {flush_and_stop}).\n\n\n%% @doc Wrappers for slow_link/5.\nslow_link(FromNodeId, ToNodeIds) -> slow_link(FromNodeId, ToNodeIds, always_slow).\n\nslow_link(FromNodeId, ToNodeIds, FastUntilMessageType) when is_list(ToNodeIds) ->\n    [slow_link(FromNodeId, To, FastUntilMessageType) || To <- ToNodeIds];\nslow_link(FromNodeId, ToNodeId, FastUntilMessageType) ->\n    slow_link(FromNodeId, kv_db, ToNodeId, dht_node, FastUntilMessageType).\n\n%% @doc See slow_link/5. But link is slow from the beginning.\nslow_link(From, FromType, To, ToType) ->\n    slow_link(From, FromType, To, ToType, always_slow).\n\n%% @doc Delays messages from From to To. Returns a link-info tuple.\n%%      Link behaves normally until a message of type FastUntilMessageType is received.\n%%      Starting with this message, all received messages between these two PIDs are queued\n%%      until flush_link/1 or remove_slow_link/1 is called.\n%%      From/To are integer ids representing the nths Pid in PidGroup FromType/ToType.\n%%      Affected messages in prbr are: round_request, read and write. Tab2list is not affected.\n%%      No messages are thrown away and the delivery order is unchanged.\nslow_link(From, FromType, To, ToType, FastUntilMessageType) ->\n    FromPid = nth(From, FromType),\n    ToPid = nth(To, ToType),\n    BpName = bp_name(\"slow_\", From, FromType, To, ToType),\n    slow_link(FromPid, FromType, ToPid, ToType, BpName, FastUntilMessageType).\n\nslow_link(FromPid, FromType, ToPid, ToType, BPName, FastUntilMessageType) ->\n    {LoopPid, BpFun} = slow_link_fun(FromPid, FromType, ToPid, ToType, FastUntilMessageType),\n    gen_component:bp_set_cond(ToPid, BpFun, BPName),\n    {BPName, LoopPid, ToPid}.\n\n%% @doc Delays all round_request, write and read messages received by prbr from PID\n%%      From, on DHT node with PID To. The link starts delivering all queued messages as\n%%      soon as a flush message was received.\n%%      tab2list_raw messages are not delayed.\nslow_link_fun(From, _FromType, To, _ToType=dht_node, FastUntilMessageType) ->\n    LoopPid = spawn(?MODULE, slow_loop, [To, FastUntilMessageType]),\n    BpFun = fun (Msg, _State) ->\n        case Msg of\n            %% prbr round_request, write and read messages seventh\n            %% element is the datatype. This is abused to ensure that\n            %% a message is only delayed once\n            _ when element(7, Msg) =:= rbr_interleave_SUITE_dont_delay ->\n                ?TRACE(\"Deliver delayed message: ~n~p\", [Msg]),\n                false;\n           %% delay a prbr round_request, write or read message if it commes\n           %% from PID From.\n            _ when element(1, Msg) =:= prbr andalso\n                       element(3, element(1, element(5, Msg))) =:= From ->\n                ?TRACE(\"Delay message: ~n~p\", [Msg]),\n                %% change Datatype in message since it is not used in this unit test suite.\n                %% marks messages which where already delayed.\n                NewMsg = setelement(7, Msg, rbr_interleave_SUITE_dont_delay),\n                MsgType = element(2, Msg),\n                comm:send_local(LoopPid, {delay, MsgType, NewMsg}),\n                drop_single;\n            _ ->\n                false\n        end\n    end,\n    {LoopPid, BpFun}.\n\nslow_loop(To, always_slow) ->\n    slow_loop(To, always_slow, [], true);\nslow_loop(To, FastUntil) ->\n    slow_loop(To, FastUntil, [], false).\nslow_loop(To, FastUntil, MsgQueue, _IsSlow=false) ->\n    receive\n        {delay, FastUntil, Msg} ->\n            slow_loop(To, FastUntil, [Msg | MsgQueue], true);\n        {delay, _Type, Msg} ->\n            comm:send_local(To, Msg),\n            slow_loop(To, FastUntil, MsgQueue, false)\n    end;\nslow_loop(To, FastUntil, MsgQueue, _IsSlow=true) ->\n    receive\n        {delay, _Type, Msg} ->\n            slow_loop(To, FastUntil, [Msg | MsgQueue], true);\n        {flush} ->\n            [comm:send_local(To, Msg) || Msg <- lists:reverse(MsgQueue)],\n            slow_loop(To, FastUntil, [], true);\n        {flush_and_stop} ->\n            [comm:send_local(To, Msg) || Msg <- lists:reverse(MsgQueue)]\n    end.\n\nnth_dht_node(N) -> nth(N, dht_node).\nnth_kv_db(N) -> nth(N, kv_db).\n\nnth(N, PidGroup) -> nth_pid(N, pid_groups:find_all(PidGroup)).\nnth_pid(N, Pids) -> lists:nth(N, lists:sort(Pids)).\n\n%% @doc Generate a breakpoint name\nbp_name(Prefix, FromId, FromType, ToId, ToType) ->\n    BPNameString = Prefix ++ \" \" ++ integer_to_list(FromId) ++ \",\" ++ atom_to_list(FromType)\n                    ++ \"|\" ++ integer_to_list(ToId) ++ \",\" ++ atom_to_list(ToType),\n    list_to_atom(BPNameString). %% ugh, dynamic creation of atoms...\n"
  },
  {
    "path": "test/recover_SUITE.erl",
    "content": "%% @copyright 2015-2017 Zuse Institute Berlin\n\n%%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%%   you may not use this file except in compliance with the License.\n%%   You may obtain a copy of the License at\n%%\n%%       http://www.apache.org/licenses/LICENSE-2.0\n%%\n%%   Unless required by applicable law or agreed to in writing, software\n%%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%%   See the License for the specific language governing permissions and\n%%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for recovery.\n%% @end\n-module(recover_SUITE).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-define(CLOSE, close).\n\ngroups() ->\n    [{slide_tests, [sequence], [\n                                half_join_and_recover_after_step2,\n                                half_join_and_recover_after_step3,\n                                half_join_and_recover_after_step4\n                               ]},\n     {leave_tests, [sequence], [\n                                %% half_leave_and_recover_after_step1,\n                                %% half_leave_and_recover_after_step2,\n                                half_leave_and_recover_after_step3,\n                                half_leave_and_recover_after_step4\n                               ]},\n     {repeater, [{repeat, 10}], [{group, slide_tests}, {group, leave_tests}]}\n    ].\n\nall() -> [].\n\nsuite() -> [ {timetrap, {seconds, 60}} ].\n\ngroup(_) ->\n    suite().\n\ninit_per_suite(Config) -> Config.\n\nend_per_suite(_Config) -> ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(_TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_symmetric_ring([{config, [{log_path, PrivDir},\n                                                   {leases, true},\n                                                   {db_backend, db_mnesia}]}]),\n    Config.\n\nend_per_testcase(_TestCase, Config) ->\n    unittest_helper:stop_ring(),\n    _ = application:stop(mnesia),\n    %% need config to get db path\n    Config2 = unittest_helper:start_minimal_procs(Config, [], false),\n    PWD = os:cmd(pwd),\n    WorkingDir = string:sub_string(PWD, 1, string:len(PWD) - 1) ++\n        \"/\" ++ config:read(db_directory) ++ \"/\" ++ atom_to_list(erlang:node()) ++ \"/\",\n    _ = file:delete(WorkingDir ++ \"schema.DAT\"),\n    _ = unittest_helper:stop_minimal_procs(Config2),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% test interrupted split before recover\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nhalf_join_and_recover_after_step2(Config) ->\n    half_join_and_recover(Config, split_reply_step2).\n\nhalf_join_and_recover_after_step3(Config) ->\n    half_join_and_recover(Config, split_reply_step3).\n\nhalf_join_and_recover_after_step4(Config) ->\n    half_join_and_recover(Config, split_reply_step4).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% test interrupted merge before recover\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nhalf_leave_and_recover_after_step1(Config) ->\n    half_leave_and_recover(Config, merge_reply_step1).\n\nhalf_leave_and_recover_after_step2(Config) ->\n    half_leave_and_recover(Config, merge_reply_step2).\n\nhalf_leave_and_recover_after_step3(Config) ->\n    half_leave_and_recover(Config, merge_reply_step3).\n\nhalf_leave_and_recover_after_step4(Config) ->\n    half_leave_and_recover(Config, merge_reply_step4).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% generic test for interrupted split before recover\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nhalf_join_and_recover(Config, MsgTag) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    %% write data\n    _ = [kv_on_cseq:write(integer_to_list(X),X) || X <- lists:seq(1, 100)],\n    %% hook into split-protocol\n    [gen_component:bp_set_cond(Pid, block(self(), MsgTag),\n                               block)\n     || Pid <- pid_groups:find_all(dht_node)],\n    %% add node\n    _ = api_vm:add_nodes(1),\n    %% wait for break point\n    receive\n        {dropped, MsgTag} -> ok\n    end,\n    %% stop ring\n    unittest_helper:stop_ring(),\n    %% wait for leases to expire\n    timer:sleep(11000),\n    %% %% recover\n    unittest_helper:make_ring_recover( [{config, [{log_path, PrivDir},\n                                                  {leases, true},\n                                                  {db_backend, db_mnesia},\n                                                  {start_type, recover}]}]),\n    lease_checker2:wait_for_clean_leases(500, [{ring_size, config:read(replication_factor)}]),\n    io:format(\"admin:check_ring(): ~p~n\", [admin:check_ring()]),\n    io:format(\"admin:check_ring_deep(): ~p~n\", [admin:check_ring_deep()]),\n    lease_checker2:get_kv_db(),\n    io:format(\"api_vm:number_of_nodes: ~p~n\", [api_vm:number_of_nodes()]),\n    io:format(\"pid_groups:find_all(dht_node): ~p~n\", [pid_groups:find_all(dht_node)]),\n    io:format(\"pid_groups:find_all(routing_table): ~p~n\", [pid_groups:find_all(routing_table)]),\n    %% ring restored -> checking KV data integrity\n    _ = check_data_integrity(),\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% generic test for interrupted merge before recover\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nhalf_leave_and_recover(Config, MsgTag) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    %% write data\n    _ = [kv_on_cseq:write(integer_to_list(X),X) || X <- lists:seq(1, 100)],\n    %% check ring\n    lease_checker2:wait_for_clean_leases(500, [{ring_size, config:read(replication_factor)}]),\n    %% hook into merge-protocol\n    [gen_component:bp_set_cond(Pid, block(self(), MsgTag),\n                               block)\n     || Pid <- pid_groups:find_all(dht_node)],\n    %% kill node\n    SaveNode = lease_checker:get_random_save_node(),\n    case SaveNode of\n        failed ->\n            true;\n        _ ->\n            RandomNode = comm:make_local(SaveNode),\n            PidGroup = pid_groups:group_of(RandomNode),\n            PidGroupTabs = [Table || Table <- db_mnesia:get_persisted_tables(),\n                                     element(2, db_util:parse_table_name(Table)) =:= PidGroup],\n            ct:pal(\"kill node\"),\n            {[PidGroup], _Not_found} = admin:del_nodes_by_name([PidGroup], false),\n            %% wait for break point\n            receive\n                {dropped, MsgTag} -> ok\n            end,\n            %% stop ring\n            unittest_helper:stop_ring(),\n            %% wait for leases to expire\n            timer:sleep(11000),\n            %% remove database files\n            _ = [?ASSERT(db_mnesia:close_and_delete(db_mnesia:open(X))) || X <- PidGroupTabs],\n            %% recover\n            unittest_helper:make_ring_recover( [{config, [{log_path, PrivDir},\n                                                          {leases, true},\n                                                          {db_backend, db_mnesia},\n                                                          {start_type, recover}]}]),\n            lease_checker2:wait_for_clean_leases(500, [{ring_size, config:read(replication_factor)-1}]),\n            io:format(\"admin:check_ring(): ~p~n\", [admin:check_ring()]),\n            io:format(\"admin:check_ring_deep(): ~p~n\", [admin:check_ring_deep()]),\n            lease_checker2:get_kv_db(),\n            io:format(\"api_vm:number_of_nodes: ~p~n\", [api_vm:number_of_nodes()]),\n            io:format(\"pid_groups:find_all(dht_node): ~p~n\", [pid_groups:find_all(dht_node)]),\n            io:format(\"pid_groups:find_all(routing_table): ~p~n\", [pid_groups:find_all(routing_table)]),\n            %% ring restored -> checking KV data integrity\n            _ = check_data_integrity(),\n            true\n    end.\n\nblock(Owner, MsgTag) ->\n    %% blocking qwrite_done and qwrite_deny\n    fun (Message, _State) ->\n            case {element(1, Message), element(2, Message)} of\n                {l_on_cseq, MsgTag} ->\n                    comm:send_local(Owner, {dropped, MsgTag}),\n                    drop_single;\n                _ ->\n                    false\n            end\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% helper functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ncheck_data_integrity() ->\n    %% lease_checker2:get_kv_db(),\n    Pred = fun (Id) ->\n                   case kv_on_cseq:read(integer_to_list(Id)) of\n                       {ok, Id} -> true;\n                       _        -> false\n                   end\n           end,\n    Elements = lists:filter(Pred, lists:seq(1, 100)),\n    case length(Elements) of\n        100 ->\n            true;\n        X ->\n            ct:pal(\"found ~p of 100 elements\", [X]),\n            100 = X\n    end.\n"
  },
  {
    "path": "test/recover_mnesia_SUITE.erl",
    "content": "%% @copyright 2015-2018 Zuse Institute Berlin\n\n%%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%%   you may not use this file except in compliance with the License.\n%%   You may obtain a copy of the License at\n%%\n%%       http://www.apache.org/licenses/LICENSE-2.0\n%%\n%%   Unless required by applicable law or agreed to in writing, software\n%%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%%   See the License for the specific language governing permissions and\n%%   limitations under the License.\n\n%% @author Tanguy Racinet <tanracinet@gmail.com>\n%% @doc    Unit tests for the mnesia recovery.\n%% @end\n%% @version $Id$\n-module(recover_mnesia_SUITE).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n\n-author('tanracinet@gmail.com').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-define(CLOSE, close).\n-define(LEASES_DELTA, 1).\n\nnum_executions() ->\n    5.\n\nrepeater_num_executions() ->\n    1000.\n\nring_size() ->\n    5.\n    %% config:read(replication_factor).\n\nall() -> [\n          {group, make_ring_group},\n          {group, remove_node_group}\n         ].\ngroups() ->\n    [\n     {make_ring_group, [sequence], [test_make_ring, write, {group, recover_data_group}]},\n     {recover_data_group, [sequence, {repeat, num_executions()}], [read]},\n     {remove_node_group, [sequence], [write, {group, remove_node}]},\n     {remove_node, [sequence, {repeat, num_executions()}], [remove_node]},\n\n     {remove_node_group_repeater, [sequence], [write, {group, remove_node_repeater}]},\n     {remove_node_repeater, [sequence, {repeat, repeater_num_executions()}], [remove_node]},\n\n     {repeater, [{group, remove_node_group_repeater}]}\n\n    ].\n\nsuite() -> [ {timetrap, {seconds, 1200}} ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(recover_data_group = Group, Config) ->\n    unittest_helper:init_per_group(Group, Config);\ninit_per_group(recover_data_group_repeater = Group, Config) ->\n    unittest_helper:init_per_group(Group, Config);\ninit_per_group(remove_node = Group, Config) ->\n    unittest_helper:init_per_group(Group, Config);\ninit_per_group(remove_node_repeater = Group, Config) ->\n    unittest_helper:init_per_group(Group, Config);\ninit_per_group(Group, Config) ->\n    ct:pal(\"stop ring and clean repository from previous test case (it may have run into a timeout)\"),\n    %% stop ring and clean repository from previous test case (it may have run into a timeout)\n    unittest_helper:stop_ring(),\n    _ = application:stop(mnesia),\n    %% need config to get db path\n    Config2 = unittest_helper:start_minimal_procs(Config, [], false),\n    PWD = os:cmd(pwd),\n    WorkingDir = string:sub_string(PWD, 1, string:len(PWD) - 1) ++\n        \"/\" ++ config:read(db_directory) ++ \"/\" ++ atom_to_list(erlang:node()) ++ \"/\",\n    _ = file:delete(WorkingDir ++ \"schema.DAT\"),\n    RingSize = ring_size(),\n    Config3 = unittest_helper:stop_minimal_procs(Config2),\n\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config3),\n    unittest_helper:make_ring(RingSize, [{config, [{log_path, PrivDir},\n                                                      {leases, true},\n                                                      {replication_factor, ring_size()},\n                                                      {round, 1},\n                                                      {leases_delta, ?LEASES_DELTA},\n                                                      {db_backend, db_mnesia}]}]),\n    unittest_helper:check_ring_size_fully_joined(ring_size()),\n    LeasesTimeout = config:read(leases_delta) * 1000 + 1000,\n    Config4 = [{leases_timeout, LeasesTimeout} | Config3],\n    unittest_helper:init_per_group(Group, Config4).\n\nend_per_group(recover_data_group = Group, Config) ->\n    unittest_helper:end_per_group(Group, Config);\nend_per_group(recover_data_group_repeater = Group, Config) ->\n    unittest_helper:end_per_group(Group, Config);\nend_per_group(remove_node = Group, Config) ->\n    unittest_helper:end_per_group(Group, Config);\nend_per_group(remove_node_repeater = Group, Config) ->\n    unittest_helper:end_per_group(Group, Config);\nend_per_group(repeater = Group, Config) ->\n    unittest_helper:end_per_group(Group, Config);\nend_per_group(Group, Config) ->\n    ct:pal(\"stop ring, stop mnesia and clean repository\"),\n    %% stop ring, stop mnesia and clean repository\n    PWD = os:cmd(pwd),\n    WorkingDir = string:sub_string(PWD, 1, string:len(PWD) - 1) ++\n        \"/\" ++ config:read(db_directory) ++ \"/\" ++ atom_to_list(erlang:node()) ++ \"/\",\n    Tabs = lists:delete(schema, mnesia:system_info(tables)),\n    unittest_helper:stop_ring(),\n    _ = application:stop(mnesia),\n    [ok = file:delete(WorkingDir ++ atom_to_list(X)++\".DCD\")||X<-Tabs],\n    ok = file:delete(WorkingDir ++ \"schema.DAT\"),\n    unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(_TestCase, Config) ->\n    Config.\n\nremove_node() ->\n     [\n      {timetrap,{seconds,180}}\n     ].\n\nrw_suite_runs(N) ->\n    erlang:min(N, 200).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% test create_ring/1 of mnesia recovery\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ntest_make_ring(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:stop_ring(),\n    wait_for_expired_leases(Config),\n    unittest_helper:make_ring_recover([{config, [{log_path, PrivDir},\n                                                 {leases, true},\n                                                 {replication_factor, ring_size()},\n                                                 {leases_delta, ?LEASES_DELTA},\n                                                 {db_backend, db_mnesia},\n                                                 {start_type, recover}]}]),\n    lease_checker2:wait_for_clean_leases(500, [{ring_size, ring_size()}]),\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% test write/1 write data to KV DBs\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nwrite(_Config) ->\n    %% write data to KV\n    lease_checker2:get_kv_db(),\n    _ = [kv_on_cseq:write(integer_to_list(X),X) || X <- lists:seq(1, 100)],\n    lease_checker2:get_kv_db(),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% test read/1 ensure data integrity after recovery\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nread(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    lease_checker2:get_kv_db(),\n    unittest_helper:stop_ring(),\n    wait_for_expired_leases(Config),\n    unittest_helper:make_ring_recover( [{config, [{log_path, PrivDir},\n                                                  {leases, true},\n                                                  {replication_factor, ring_size()},\n                                                  {leases_delta, ?LEASES_DELTA},\n                                                  {db_backend, db_mnesia},\n                                                  {start_type, recover}]}]),\n    lease_checker2:wait_for_clean_leases(500, [{ring_size, ring_size()}]),\n    %% ring restored -> checking KV data integrity\n    _ = check_data_integrity(1, read_test),\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% test remove_node/1 remove a node and ensure data integrity after recovery\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\nremove_node(Config) ->\n    Round = config:read(round),\n    ct:pal(\"round: ~w\", [Round]),\n    ct:pal(\"wait for check_leases\"),\n    lease_checker2:wait_for_clean_leases(500, [{ring_size, ring_size()}]),\n    SaveNode = lease_checker:get_random_save_node(),\n    case SaveNode of\n        failed ->\n            true;\n        _ ->\n            RandomNode = comm:make_local(SaveNode),\n            io:format(\"show prbr statistics for the ring~n\"),\n            lease_checker2:get_kv_db(),\n\n            io:format(\"show prbr statistics for node to be killed~n\"),\n            lease_checker2:get_kv_db(RandomNode),\n\n            ct:pal(\"PRBR state before node is removed\"),\n            print_prbr_data(kv_db, Round, before_kill, true),\n            _ = print_leases_data(Round),\n\n            %% get relative range of node to remove and check if it is not to large\n            {true, LL} = lease_checker:get_dht_node_state_unittest(comm:make_global(RandomNode), lease_list),\n            NodeRange = l_on_cseq:get_range(lease_list:get_active_lease(LL)),\n            RelativeRange = lease_checker:get_relative_range_unittest(NodeRange),\n            ct:pal(\"Statistics of removed node ~p~n\"\n                   \"Interval of node to be removed:~nNode Interval ~p~n\"\n                   \"Relative Range ~p\", [RandomNode, NodeRange, RelativeRange]),\n\n            R = config:read(replication_factor),\n            SaveFraction = quorum:minority(R) / R,\n            ct:pal(\"Safe relative range to remove ~p\", [SaveFraction]),\n            ?assert_w_note(RelativeRange =< SaveFraction, \"Removing a safe node means that only\"\n                           \" a minority should be affected\"),\n\n            %% prbr data of node for diagnostic purpose...\n            comm:send_local(RandomNode, {prbr, tab2list_raw, kv_db, self()}),\n            receive\n                {kv_db, NodeData} -> NodeData\n            end,\n            Values = [prbr:entry_val(E) || E <- NodeData],\n            ct:pal(\"Number of values: ~p~nNumber of unique values: ~p~nValue list:~p\",\n                   [length(Values), length(lists:usort(Values)), lists:sort(Values)]),\n\n            %% The tests starts here...\n            PidGroup = pid_groups:group_of(RandomNode),\n            PidGroupTabs = [Table || Table <- db_mnesia:get_persisted_tables(),\n                                     element(2, db_util:parse_table_name(Table)) =:= PidGroup],\n            ct:pal(\"kill node\"),\n            {[PidGroup], _Not_found} = admin:del_nodes_by_name([PidGroup], false),\n            %% wait for leases to expire\n            ct:pal(\"wait for leases to expire\"),\n            wait_for_expired_leases(Config),\n            _ = [?ASSERT(db_mnesia:close_and_delete(db_mnesia:open(X))) || X <- PidGroupTabs],\n            ct:pal(\"wait for check_leases\"),\n            lease_checker2:wait_for_clean_leases(500, [{ring_size, ring_size()-1}]),\n\n            ct:pal(\"PRBR state after leases expired\"),\n            print_prbr_data(kv_db, Round, after_kill, true),\n            _ = print_leases_data(Round),\n\n            %% check data integrity\n            ct:pal(\"check data integrity\"),\n            _ = check_data_integrity(Round, before_rrepair),\n            %% \"repair\" replicas\n            ct:pal(\"repair replicas\"),\n            _ = repair_replicas(),\n\n            ct:pal(\"PRBR state after calling repair_replicas\"),\n            print_prbr_data(kv_db, Round, after_rrepair, true),\n            _ = print_leases_data(Round),\n\n            %% add node to reform ring_size() node ring\n            ct:pal(\"add node\"),\n            NewNode = admin:add_nodes(1),\n            ct:pal(\"added node: ~p~n\", [NewNode]),\n            ct:pal(\"sleep\"),\n            timer:sleep(1000),\n            ct:pal(\"check_ring_size_fully_joined\"),\n            unittest_helper:check_ring_size_fully_joined(ring_size()),\n            ct:pal(\"wait for check_leases\"),\n            lease_checker2:wait_for_clean_leases(500, [{ring_size, ring_size()}]),\n\n            ct:pal(\"PRBR state after node was inserted\"),\n            print_prbr_data(kv_db, Round, after_insert, true),\n            _ = print_leases_data(Round),\n\n            true\n    end,\n    config:write(round, Round + 1),\n    true.\n\ncheck_data_integrity(Round, Label) ->\n    io:format(\"show prbr statistics for the ring~n\"),\n    lease_checker2:get_kv_db(),\n    Pred = fun (Id) ->\n                   case kv_on_cseq:read(integer_to_list(Id)) of\n                       {ok, Id} -> true;\n                       {fail, not_found} -> false\n                   end\n           end,\n    Elements = lists:filter(Pred, lists:seq(1, 100)),\n    case length(Elements) of\n        100 ->\n            true;\n        X ->\n            ct:pal(\"found ~p of 100 elements\", [X]),\n            Missing = lists:subtract(lists:seq(1, 100), Elements),\n            ct:pal(\"Missing elements are:~n~w\", [Missing]),\n            ct:pal(\"Printing missing element data...\"),\n            [print_element_data(E, kv_db) || E <- Missing],\n            print_prbr_data(kv_db, Round, Label, true),\n\n            100 = X\n    end.\n\nrepair_replicas() ->\n    %% we need repair also for even replication degrees\n    %%    case config:read(replication_factor) rem 2 =:= 1 of\n    %%        true -> %% only repair for odd replication factors\n    io:format(\"show prbr statistics for the ring before repair~n\"),\n    lease_checker2:get_kv_db(),\n    _ = [kv_on_cseq:write(integer_to_list(X),X) || X <- lists:seq(1, 100)],\n    %% let also arrive messages to remaining minority\n    timer:sleep(200),\n    io:format(\"show prbr statistics for the ring after repair~n\"),\n    lease_checker2:get_kv_db()%;\n    %%     false ->\n    %%         ok\n    %% end.\n.\n\nwait_for_expired_leases(Config) ->\n    {leases_timeout, LeasesTimeout} = lists:keyfind(leases_timeout, 1, Config),\n    timer:sleep(LeasesTimeout).\n\n%%@doc Prints a list of tuples showing which value is stored in which dht node\n%%     Format : [{Value, [list_of_dht_nodes_value_is_stored_in]}]\nprint_prbr_data(DB, Round, Label, MayCrash) ->\n    PrbrData = get_prbr_data(fun(NodePid, E) ->\n                                {prbr:entry_val(E), NodePid}\n                             end, DB),\n    GroupedByValueDict = lists:foldl(fun({K, V}, D) -> dict:append(K, V, D) end,\n                                             dict:new(), PrbrData),\n    GroupedValues = lists:sort(dict:to_list(GroupedByValueDict)),\n\n    WoBottom = [{Entry, NodeList} || {Entry, NodeList} <- GroupedValues,\n                                Entry =/= prbr_bottom],\n\n\n    Bad = [{Entry, NodeList} || {Entry, NodeList} <- WoBottom,\n                                length(NodeList) <\n                                 quorum:majority_for_accept(config:read(replication_factor))],\n\n    Uniques = [length(lists:usort(NodeList)) || {_Entry, NodeList} <- WoBottom],\n\n    {Min, Max} = case length(Uniques) of\n                     0 -> {bottom, bottom};\n                     _ -> {lists:min(Uniques), lists:max(Uniques)}\n                 end,\n\n    ct:pal(\"# unique replicas: min:~w; max:~w~n\", [Min, Max]),\n    ct:pal(\"PRBR state ~w:~nFormat [{Value, [list_of_dht_nodes_value_is_stored_in]}]~n\"\n           \"~100p\", [DB, GroupedValues]),\n    case MayCrash of\n        true ->\n            case length(Bad) of\n                0 -> true;\n                _ -> S = io_lib:format(\"entries with not enough replicas (round:~w, db=~w, label=~w)\",\n                                       [Round, DB, Label]),\n                     S2 = lists:flatten(S),\n                     ct:fail(S2) %% 14B04 ...\n            end;\n        _ -> ok\n    end.\n\nprint_leases_data(Round) ->\n    _ = [print_prbr_data({lease_db, I}, Round, leases_db, false) || I <-\n                             lists:seq(1, config:read(replication_factor))].\n\nprint_element_data(Id, DB) ->\n    HashedKey = ?RT:hash_key(integer_to_list(Id)),\n    ReplicaKeyList = replication:get_keys(HashedKey),\n    PrbrData = get_prbr_data(fun(NodePid, E) ->\n                                {prbr:entry_key(E),\n                                 prbr:entry_val(E),\n                                 NodePid}\n                             end, DB),\n    IdData = lists:filter(fun(E) ->\n                            lists:member(element(1, E), ReplicaKeyList)\n                          end, PrbrData),\n\n    ct:pal(\"Printing data for ID=~p~nReplica key list:~n~p~n\"\n           \"Entries found in prbr:~n~100p\",\n           [Id, lists:sort(ReplicaKeyList), lists:sort(IdData)]),\n    ok.\n\n%% get all elements stored in prbr as flattened list.\n%% applies DataExtractFun(DhtNodePidEFoundOn, E) for every entry E.\nget_prbr_data(DataExtractFun, DB) ->\n    DhtNodes = pid_groups:find_all(dht_node),\n    lists:flatten(\n        [begin\n            comm:send_local(ThisNode, {prbr, tab2list_raw, DB, self()}),\n            receive\n                {DB, List} -> [DataExtractFun(ThisNode, E) || E <- List]\n%% Do not risk losing answers and receiving them in the next call, so do not use 'after'\n%%            after 1000 ->\n%%                ct:pal(\"DHT node ~p does not reply...\", [ThisNode]),\n%%                []\n            end\n         end || ThisNode <- DhtNodes]).\n"
  },
  {
    "path": "test/repeated_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{groups, tests, recover_mnesia_SUITE, repeater}.\n"
  },
  {
    "path": "test/rm_SUITE.erl",
    "content": "%  @copyright 2010-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    TODO: Add description to rm_SUITE\n%% @end\n%% @version $Id$\n-module(rm_SUITE).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\nall() ->\n    [tester_update_id, tester_update_id2_1, tester_update_id2_2,\n     tester_update_id2_3, tester_update_id2_4].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 180}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_testcase(_TestCase, Config) ->\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% ...\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_update_id(?RT:key(), ?RT:key()) -> true.\nprop_update_id(OldId, NewId) ->\n    Ring = unittest_helper:make_ring_with_ids([OldId], [{config, [pdb:get(log_path, ?MODULE)]}]),\n    change_id_and_check(OldId, NewId),\n    change_id_and_check(NewId, OldId),\n    unittest_helper:stop_ring(Ring),\n    true.\n\ntester_update_id(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    pdb:set({log_path, PrivDir}, ?MODULE),\n    tester:test(rm_SUITE, prop_update_id, 2, 10).\n\n%% @doc Changes a node's ID and checks that the change has been performed.\n%%      Precondition: existing scalaris ring with at least one node.\n-spec change_id_and_check(OldId::?RT:key() | unknown, NewId::?RT:key()) -> true.\nchange_id_and_check(OldId, NewId) ->\n    DhtNode = comm:make_global(pid_groups:find_a(dht_node)),\n    comm:send(DhtNode, {get_node_details, comm:this(), [node]}),\n    OldNode = receive\n                  {get_node_details_response, OldNodeDetails} ->\n                      node_details:get(OldNodeDetails, node)\n              end,\n    case OldId =/= unknown of\n        true -> ?equals(node:id(OldNode), OldId);\n        _ -> ok\n    end,\n%%     ct:pal(\"ct: ~p -> ~p~n\", [node:id(OldNode), NewId]),\n    \n    comm:send(DhtNode, {rm, subscribe, self(), rm_SUITE,\n                        fun(OldN, NewN, _Reason) -> OldN =/= NewN end,\n                        fun(Pid, Tag, OldNeighbors, NewNeighbors, _Reason) ->\n                                comm:send_local(Pid, {rm_changed, Tag, OldNeighbors, NewNeighbors})\n                        end, inf}),\n    comm:send(DhtNode, {rm, update_my_id, NewId}),\n    comm:send(DhtNode, {rm, unsubscribe, self(), rm_SUITE}),\n    \n    % check that the new ID has been set:\n    comm:send(DhtNode, {get_node_details, comm:this(), [node, pred, succ]}),\n    receive\n        {get_node_details_response, NewNodeDetails} ->\n            NewNode = node_details:get(NewNodeDetails, node),\n            Pred = node_details:get(NewNodeDetails, pred),\n            Succ = node_details:get(NewNodeDetails, succ),\n            case node:id(OldNode) =/= NewId andalso\n                     ((NewNode =:= Pred) andalso (NewNode =:= Succ) % 1-element ring\n                     orelse % two nodes:\n                          ((Pred =:= Succ) andalso (node:id(Pred) =/= NewId))\n                     orelse % at least three nodes:\n                          intervals:in(NewId,\n                                       intervals:new('(', node:id(Pred), node:id(Succ), ')'))) of\n                true ->\n                    ?equals(node:id(NewNode), NewId),\n                    ?equals(node:id_version(NewNode), node:id_version(OldNode) + 1),\n                    check_subscr_node_update(OldNode, NewNode),\n                    % now wait for all rm processes of the other nodes to know about the new id (at least in pred/succ):\n                    util:wait_for(\n                      fun() ->\n                              Nodes =\n                                  lists:flatten(\n                                    [begin\n                                         comm:send(comm:make_global(N),\n                                                   {get_node_details, comm:this(), [node, pred, succ]}),\n                                         receive\n                                             {get_node_details_response, ND} ->\n                                                 [node_details:get(ND, node),\n                                                  node_details:get(ND, pred),\n                                                  node_details:get(ND, succ)]\n                                         end\n                                     end || N <- pid_groups:find_all(dht_node)]),\n                              not lists:any(fun(N) -> N =:= OldNode end, Nodes)\n                      end);\n                _ ->\n                    ?equals(NewNode, OldNode),\n                    ?expect_no_message()\n            end\n    end,\n    true.\n\n-spec check_subscr_node_update(OldNode::node:node_type(), NewNode::node:node_type()) -> any().\ncheck_subscr_node_update(OldNode, NewNode) ->\n    receive\n        {rm_changed, rm_SUITE, OldNeighbors, NewNeighbors} ->\n            ?equals(nodelist:node(OldNeighbors), OldNode),\n            ?equals(nodelist:node(NewNeighbors), NewNode)\n    after 1000 ->\n        ActualMessage =\n            receive\n                X -> X\n            after 0 -> no_message\n            end,\n            ?ct_fail(\"expected message {rm_changed, rm_SUITE, OldNeighbors, NewNeighbors} but got \\\"~.0p\\\"~n\", [ActualMessage])\n    end.\n\n-spec prop_update_id2(?RT:key()) -> true.\nprop_update_id2(NewId) ->\n    change_id_and_check(unknown, NewId).\n\ntester_update_id2_1(Config) ->\n    Config2 = unittest_helper:start_minimal_procs(Config, [], true),\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config2),\n    RandomSubset = util:random_subset(1, ?RT:get_replica_keys(?RT:hash_key(\"0\"))),\n    _ = unittest_helper:stop_minimal_procs(Config2),\n    Ring = unittest_helper:make_ring_with_ids(\n             RandomSubset,\n             [{config, [{log_path, PrivDir}]}]),\n    tester:test(rm_SUITE, prop_update_id2, 1, 1000),\n    unittest_helper:stop_ring(Ring).\n\ntester_update_id2_2(Config) ->\n    Config2 = unittest_helper:start_minimal_procs(Config, [], true),\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config2),\n    RandomSubset = util:random_subset(2, ?RT:get_replica_keys(?RT:hash_key(\"0\"))),\n    _ = unittest_helper:stop_minimal_procs(Config2),\n    Ring = unittest_helper:make_ring_with_ids(\n             RandomSubset,\n             [{config, [{log_path, PrivDir}]}]),\n    tester:test(rm_SUITE, prop_update_id2, 1, 100),\n    unittest_helper:stop_ring(Ring).\n\ntester_update_id2_3(Config) ->\n    Config2 = unittest_helper:start_minimal_procs(Config, [], true),\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config2),\n    RandomSubset = util:random_subset(3, ?RT:get_replica_keys(?RT:hash_key(\"0\"))),\n    _ = unittest_helper:stop_minimal_procs(Config2),\n    Ring = unittest_helper:make_ring_with_ids(\n             RandomSubset,\n             [{config, [{log_path, PrivDir}]}]),\n    tester:test(rm_SUITE, prop_update_id2, 1, 100),\n    unittest_helper:stop_ring(Ring).\n\ntester_update_id2_4(Config) ->\n    Config2 = unittest_helper:start_minimal_procs(Config, [], true),\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config2),\n    RandomSubset = util:random_subset(4, ?RT:get_replica_keys(?RT:hash_key(\"0\"))),\n    _ = unittest_helper:stop_minimal_procs(Config2),\n    Ring = unittest_helper:make_ring_with_ids(\n             RandomSubset,\n             [{config, [{log_path, PrivDir}]}]),\n    tester:test(rm_SUITE, prop_update_id2, 1, 100),\n    unittest_helper:stop_ring(Ring).\n"
  },
  {
    "path": "test/rm_leases_SUITE.erl",
    "content": "%% @copyright 2012-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for slide_leases\n%% @end\n%% @version $Id$\n-module(rm_leases_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\ngroups() ->\n    [{tester_tests,   [sequence], [\n                                  tester_type_check_rm_leases\n                              ]},\n     {kill_tests,     [sequence], [\n                                  test_single_kill\n                               ]},\n     {add_tests,      [sequence], [\n                                  test_single_add,\n                                  test_double_add,\n                                  test_triple_add\n                               ]},\n     {partition_tests,[sequence], [\n                                   test_network_partition\n                               ]},\n     {rm_loop_tests,  [sequence], [\n                                  propose_new_neighbor\n                                 ]},\n\n     {repeater, [{repeat, 30}], [{group, kill_tests}   , {group, add_tests},\n                                 {group, rm_loop_tests}, {group, partition_tests}]}\n    ].\n\nall() ->\n    [\n     {group, tester_tests},\n     {group, kill_tests},\n     {group, add_tests},\n     {group, rm_loop_tests},\n     {group, partition_tests}\n     ].\n\nsuite() -> [ {timetrap, {seconds, 40}} ].\n\ngroup(tester_tests) ->\n    [{timetrap, {seconds, 400}}];\ngroup(partition_tests) ->\n    [{timetrap, {seconds, 400}}];\ngroup(_) ->\n    suite().\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\n\ninit_per_testcase(TestCase, Config) ->\n    case TestCase of\n        test_network_partition ->\n            {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n            %% this test case requires an even number of nodes\n            unittest_helper:make_symmetric_ring([{scale_ring_size_by, 2}, {config,\n                                                                           [{log_path, PrivDir},\n                                                                            {leases, true}]}]),\n            ok;\n        _ ->\n            {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n            unittest_helper:make_symmetric_ring([{config, [{log_path, PrivDir},\n                                                           {leases, true}]}]),\n            ok\n    end,\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\ntester_type_check_rm_leases(_Config) ->\n    Count = 500,\n    config:write(no_print_ring_data, true),\n    %% [{modulename, [excludelist = {fun, arity}]}]\n    Modules =\n        [ {rm_leases,\n           [\n            {start_link, 1},\n            {start_gen_component,5}, %% unsupported types\n            {on, 2},\n            {get_takeovers, 1} %% sends messages\n           ],\n           [\n            {compare_and_fix_rm_with_leases, 5}, %% cannot create dht_node_state (reference for bulkowner)\n            {propose_new_neighbors, 1}, %% sends messages\n            {prepare_takeover, 3} %% cannot create dht_node_state (reference for bulkowner)\n           ]}\n        ],\n    %% join a dht_node group to be able to call lease trigger functions\n    pid_groups:join(pid_groups:group_with(dht_node)),\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% kill unit tests\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n\ntest_single_kill(_Config) ->\n    case config:read(replication_factor) < 4 of\n        true ->\n            log:log(\"skipped: this test case is likely to fail for small replication factors\"),\n            ok;\n        false ->\n            NrOfNodes = api_vm:number_of_nodes(),\n            log:log(\"kill nodes\", []),\n            synchronous_kill(NrOfNodes),\n            %% timer:sleep(5000), % enable to see rest of protocol\n            ok\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% add unit tests\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n\ntest_single_add(_Config) ->\n    %% log:log(\"join nodes\", []),\n    log:log(\"add nodes\", []),\n    synchronous_add(config:read(replication_factor), config:read(replication_factor)+1),\n    %timer:sleep(5000), % enable to see rest of protocol\n    ok.\n\ntest_double_add(_Config) ->\n    %% log:log(\"join nodes\", []),\n    log:log(\"add nodes\", []),\n    synchronous_add(config:read(replication_factor), config:read(replication_factor)+2),\n    %timer:sleep(5000), % enable to see rest of protocol\n    ok.\n\ntest_triple_add(_Config) ->\n    %% log:log(\"join nodes\", []),\n    log:log(\"add nodes\", []),\n    synchronous_add(config:read(replication_factor), config:read(replication_factor)+3),\n    %timer:sleep(5000), % enable to see rest of protocol\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% rm_loop unit tests\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\npropose_new_neighbor(_Config) ->\n    lease_helper:wait_for_ring_size(config:read(replication_factor)),\n    lease_helper:wait_for_correct_ring(),\n    MainNode = pid_groups:group_with(dht_node),\n    % main node\n    RMLeasesPid = pid_groups:pid_of(MainNode, rm_leases),\n    DHTNodePid = pid_groups:pid_of(MainNode, dht_node),\n\n    % fake death\n    {_Pred, PredRange, Lease} = get_pred_info(DHTNodePid),\n    Result = {qread_done, fake_reqid, fake_round, fake_old_write_round, Lease},\n    Msg = {read_after_rm_change, PredRange, Result},\n    TakeoversBefore = rm_leases:get_takeovers(RMLeasesPid),\n    ct:pal(\"+wait_for_messages_after ~w\", [gb_trees:to_list(TakeoversBefore)]),\n    wait_for_messages_after(RMLeasesPid, [merge_after_rm_change], %get_node_for_new_neighbor],\n                            fun () ->\n                                    comm:send_local(RMLeasesPid, Msg)\n                            end),\n    ct:pal(\"-wait_for_messages_after ~w\", [gb_trees:to_list(TakeoversBefore)]),\n\n\n    AllRMMsgs = [read_after_rm_change, takeover_after_rm_change,\n                 merge_after_rm_change, merge_after_leave,\n                 get_node_for_new_neighbor, get_takeovers],\n    ct:pal(\"+test_quiescence\"),\n    test_quiescence(RMLeasesPid, AllRMMsgs, 100),\n    ct:pal(\"-test_quiescence\"),\n    TakeoversBefore = rm_leases:get_takeovers(RMLeasesPid),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% network partition unit tests\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\ntest_network_partition(_Config) ->\n    %% We create a ring with an even number of nodes. For the odd\n    %% nodes, we stop lease renewal and wait for the leases to time\n    %% out. After that, we propose to the first (even) node to\n    %% takeover the last (odd) node.\n\n    DHTNodes = pid_groups:find_all(dht_node),\n    IdsAndNodes = lists:sort(\n        [\n          begin\n              comm:send_local(Node, {get_state, comm:this(), [node_id]}),\n              receive\n                  {get_state_response, [{node_id, Id}]} ->\n                      {Id, Node}\n              end\n          end\n          || Node <- DHTNodes]),\n    OddNodes = iterate_even_odd(IdsAndNodes,\n                                fun(Val, Even) ->\n                                        case Even of\n                                            true ->\n                                                ok;\n                                            false ->\n                                                {val, Val}\n                                        end\n                                end),\n    DHTNodePid = hd(DHTNodes),\n\n    %% stop odd nodes\n    _ = [lease_helper:intercept_lease_renew(Node) || {_Id, Node} <- OddNodes],\n    lease_helper:wait_for_number_of_valid_active_leases(length(DHTNodes) div 2),\n\n    RMLeasesPid = pid_groups:pid_of(pid_groups:group_of(DHTNodePid), rm_leases),\n\n    %% propose takeover\n    {_Pred, PredRange, Lease} = get_pred_info(DHTNodePid),\n    Result = {qread_done, fake_reqid, fake_round, fake_old_write_round, Lease},\n    Msg = {read_after_rm_change, PredRange, Result},\n    %comm:send_local(RMLeasesPid, Msg),\n\n    %% what do we expect to happen? takeover and merge should succeed\n    wait_for_messages_after(RMLeasesPid, [merge_after_rm_change],\n                            fun () ->\n                                    comm:send_local(RMLeasesPid, Msg)\n                            end),\n    ok.\n\niterate_even_odd(L, F) ->\n    iterate_even_odd1(L, F, true, []).\n\niterate_even_odd1([], _F, _Flag, Acc) ->\n    Acc;\niterate_even_odd1([Val|Rest], F, Even, Acc) ->\n    case F(Val, Even) of\n        {val, Value} ->\n            iterate_even_odd1(Rest, F, not Even, [Value|Acc]);\n        _ ->\n            iterate_even_odd1(Rest, F, not Even, Acc)\n    end.\n\nget_pred_info(Pid) ->\n    comm:send_local(Pid, {get_state, comm:this(), neighbors}),\n    Neighbors = receive\n        {get_state_response, Neighbors2} -> Neighbors2\n    end,\n    % @todo could add fake node??!\n    PredNode = nodelist:pred(Neighbors),\n    PredPid = node:pidX(PredNode),\n    comm:send(PredPid, {get_state, comm:this(), my_range}),\n    PredRange = receive\n        {get_state_response, PredRange2} -> PredRange2\n    end,\n    LeaseId = l_on_cseq:id(PredRange),\n    {ok, Lease} = l_on_cseq:read(LeaseId),\n    {PredPid, PredRange, Lease}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% join helper\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\njoin_test(Current, TargetSize) ->\n    lease_helper:wait_for_ring_size(Current),\n    lease_helper:wait_for_correct_ring(),\n    join_until(Current, TargetSize),\n    true.\n\njoin_until(Current, TargetSize) ->\n    joiner_helper(Current, TargetSize).\n\njoiner_helper(Target, Target) ->\n    ok;\njoiner_helper(Current, Target) ->\n    synchronous_join(Current+1),\n    joiner_helper(Current+1, Target).\n\nsynchronous_join(TargetSize) ->\n    _ = api_vm:add_nodes(1),\n    check_ring_state(TargetSize).\n\ncheck_ring_state(TargetSize) ->\n    lease_helper:wait_for_ring_size(TargetSize),\n    lease_helper:wait_for_correct_ring(),\n    lease_helper:wait_for_correct_leases(TargetSize).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% kill helper\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nsynchronous_kill(Current) ->\n    _ = api_vm:kill_nodes(1),\n    ct:pal(\"wait for ring size\"),\n    lease_helper:wait_for_ring_size(Current - 1),\n    ct:pal(\"wait for correct ring\"),\n    lease_helper:wait_for_correct_ring(),\n    ct:pal(\"wait for correct leases\"),\n    lease_helper:wait_for_correct_leases(Current - 1).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% add helper\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nsynchronous_add(Current, Current) ->\n    ok;\nsynchronous_add(Current, _TargetSize) ->\n    ct:pal(\"================================== adding node ========================\"),\n    _ = api_vm:add_nodes(1),\n    ct:pal(\"wait for ring size\"),\n    lease_helper:wait_for_ring_size(Current + 1),\n    ct:pal(\"wait for correct ring\"),\n    lease_helper:wait_for_correct_ring(),\n    ct:pal(\"wait for correct leases\"),\n    util:wait_for(fun () -> admin:check_leases(Current + 1) end, 10000),\n    %lease_helper:wait_for_correct_leases(Current + 1),\n    ct:pal(\"================================== adding node done ========================\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% intercepting and blocking\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nwait_for_messages_after(Pid, Msgs, F) ->\n    gen_component:bp_set_cond(Pid, watch_for_msgs_filter(self(), Msgs), wait_for_message),\n    F(),\n    receive\n        {saw_message, _Msg} ->\n            gen_component:bp_del(Pid, wait_for_message)\n    end,\n    ok.\n\ntest_quiescence(Pid, Msgs, Timeout) ->\n    gen_component:bp_set_cond(Pid, watch_for_msgs_filter(self(), Msgs), test_quiescence),\n    receive\n        {saw_message, Msg} ->\n            gen_component:bp_del(Pid, test_quiescence),\n            ?ct_fail(\"expected quiescence, but got ~w\", [Msg])\n        after Timeout ->\n                gen_component:bp_del(Pid, test_quiescence),\n                ok\n    end.\n\nwatch_for_msgs_filter(Pid, Msgs) ->\n    fun (Message, _State) ->\n            %ct:pal(\"saw ~w~n~w~n~w~n\", [Message, lists:member(Message, Msgs), Msgs]),\n            case lists:member(element(1, Message), Msgs) of\n                true ->\n                    comm:send_local(Pid, {saw_message, Message}),\n                    false;\n                _ ->\n                    false\n            end\n    end.\n"
  },
  {
    "path": "test/rr_eval_admin.erl",
    "content": "% @copyright 2012-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <lakedaimon300@googlemail.com>\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Administrative helper functions for replica repair evaluation\n%%         through the provided methods.\n%% @version $Id:  $\n-module(rr_eval_admin).\n\n-include(\"rr_records.hrl\").\n\n% for external scripts\n-export([% trivial\n         trivial/7, trivial_ddists_fdists/7, trivial_scale/6,\n         trivial_custom/11,\n         % shash\n         shash/7, shash_ddists_fdists/7, shash_scale/6,\n         shash_custom/11,\n         % bloom\n         bloom/7, bloom_ddists_fdists/7, bloom_scale/6,\n         bloom_custom/11,\n         % merkle\n         merkle/9, merkle_ddists_fdists/9, merkle_scale/8,\n         merkle_custom/15,\n         % art\n         art/8, art_scale/8,\n         % system sync\n         system/4]).\n\n% for debugging:\n-export([ring_build/0, get_node_interval/1,\n         get_db/1, get_db_status/1, print_db_status/1]).\n\n% for other modules:\n-export([get_bandwidth/1]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% TYPES\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-type init_mp() :: {Missing::non_neg_integer(),\n                    Outdated::non_neg_integer()}.\n\n-type eval_option() :: {eval_dir, string()} |\n                       {filename, string()} |\n                       {eval_time, {Hour::integer(), Min::integer(), Sec::integer()}} |\n                       {eval_repeats, pos_integer()} |\n                       {start_ep_id, integer()} |\n                       {ep_file, file:io_device()} |\n                       {mp_file, file:io_device()}.\n-type eval_options() :: [eval_option()].\n\n-type sync_result() :: {[rr_eval_point:eval_point()],       %Sync EvalPoints\n                        [rr_eval_point:measure_point()],    %Sync MeasurePoints\n                        NextEPId::non_neg_integer()}.       %Next unique EvalPointId\n\n-dialyzer([{no_match, [get_param_value/2, set_params/3,\n                       init_rc_conf/3, init_ring_conf/3]},\n           {no_return, [system/4]}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-define(DBSizeKey, rr_eval_admin_dbsize).    %Process Dictionary Key for generated db size\n-define(TAB, 9).\n\n-define(BINOM, 0.2).       %binomial(N, p) - this is p\n\n-define(EVAL_DDISTS, [random, {binomial, ?BINOM}]).\n-define(EVAL_FTYPES, [update, regen]).\n-define(EVAL_FDISTS, [random, {binomial, ?BINOM}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec ring_build() -> ok.\nring_build() ->\n    RingType = uniform,\n    NodeCount = 4,\n    DBType = random,\n    DBSize = 10000,\n    FType = update,\n    FProb = 50,\n    DBDist = uniform,\n    \n    {RingT, _} = util:tc(fun() -> make_ring(RingType, NodeCount) end),\n    {FillT, _} = util:tc(fun() -> fill_ring(DBType, DBSize,\n                                            [{ftype, FType}, {fprob, FProb}, {distribution, DBDist}])\n                         end),\n    io:format(\"RingTime: ~.4fs~nFillTime=~.4fs~n\", [RingT / 1000000, FillT / 1000000]),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec default_dir_and_name(Alg::atom()) -> {Dir::string(), FileName::string()}.\ndefault_dir_and_name(Alg) ->\n    {{YY, MM, DD}, {Hour, Min, Sec}} = erlang:localtime(),\n    SubDir = io_lib:format(\"~p-~p-~p_~p-~p-~p_~p\", [YY, MM, DD, Hour, Min, Sec, Alg]),\n    {filename:join([config:read(log_path), \"rr_eval\", SubDir]), \"results.dat\"}.\n\n-spec gen_setup(DDists::[data_distribution()],\n                FTypes::[db_generator:failure_type()],\n                FDists::[fail_distribution()],\n                scenario(), ring_config(), [rc_config()]) -> [ring_setup()].\ngen_setup(DDists, FTypes, FDists, Scen, Ring, RCList) ->\n    [{Scen#scenario{fail_distribution = FDist,\n                    data_distribution = DDist,\n                    data_failure_type = FType}, Ring, RC}\n     || DDist <- DDists, FDist <- FDists, RC <- RCList, FType <- FTypes].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% TRIVIAL EVAL\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec trivial(DestDir::string(), FileName::string(), N::pos_integer(),\n              EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n              StepSize::step_size()) -> ok.\ntrivial(Dir, FileName, N, EvalRepeats, FR, ExpDelta, StepSize) ->\n    trivial(Dir, FileName, N, EvalRepeats, FR, ExpDelta, [random], [random], StepSize).\n\n-spec trivial_ddists_fdists(DestDir::string(), FileName::string(), N::pos_integer(),\n                            EvalRepeats::pos_integer(), FR::fail_rate(), Delta::number() | as_fprob,\n                            StepSize::step_size()) -> ok.\ntrivial_ddists_fdists(Dir, FileName, N, EvalRepeats, FR, ExpDelta, StepSize) ->\n    trivial(Dir, FileName, N, EvalRepeats, FR, ExpDelta, ?EVAL_DDISTS, ?EVAL_FDISTS, StepSize).\n\n-spec trivial(DestDir::string(), FileName::string(), N::pos_integer(),\n              EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n              DDists::[data_distribution()], FDists::[fail_distribution()],\n              StepSize::step_size()) -> ok.\ntrivial(Dir, FileName, N, EvalRepeats, FR, ExpDelta, DDists, FDists, StepSize) ->\n    Scenario = #scenario{ ring_type = uniform,\n                          data_type = random },\n    PairRing = #ring_config{ data_count = N,\n                             node_count = 4,\n                             fquadrants = [1,3],\n                             data_failure_rate = 3,\n                             round = 1 },\n    Options = [{eval_dir, Dir}, {filename, FileName}, {eval_repeats, EvalRepeats}],\n    \n    Trivial = #rc_config{ recon_method = trivial, recon_fail_rate = FR,\n                          expected_delta = ExpDelta },\n    \n    eval(pair,\n         gen_setup(DDists, ?EVAL_FTYPES, FDists, Scenario, PairRing, [Trivial]),\n         fprob, 4, StepSize, StepSize, Options),\n    ok.\n\n-spec trivial_scale(DestDir::string(), FileName::string(), N::pos_integer(),\n                    EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob) -> ok.\ntrivial_scale(Dir, FileName, N, EvalRepeats, FR, ExpDelta) ->\n    trivial_custom(Dir, FileName, N, EvalRepeats, FR, ExpDelta,\n                 {power, 4}, 5, ?EVAL_FTYPES, 3, data_count).\n\n-spec trivial_custom(DestDir::string(), FileName::string(), N::pos_integer(),\n                     EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n                     StepInc::step_inc(), Steps::pos_integer(),\n                     FTypes::[update | regen], Delta::pos_integer(),\n                     step_param()) -> ok.\ntrivial_custom(Dir, FileName, N, EvalRepeats, FR, ExpDelta,\n               StepInc, Steps, FTypes, Delta, StepParam) ->\n    Trivial = #rc_config{ recon_method = trivial, recon_fail_rate = FR,\n                          expected_delta = ExpDelta },\n    alg_custom(Dir, FileName, N, EvalRepeats, Trivial,\n               StepInc, Steps, FTypes, Delta, StepParam).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% SHASH EVAL\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec shash(DestDir::string(), FileName::string(), N::pos_integer(),\n            EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n            StepSize::step_size()) -> ok.\nshash(Dir, FileName, N, EvalRepeats, FR, ExpDelta, StepSize) ->\n    shash(Dir, FileName, N, EvalRepeats, FR, ExpDelta, [random], [random], StepSize).\n\n-spec shash_ddists_fdists(DestDir::string(), FileName::string(), N::pos_integer(),\n                          EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n                          StepSize::step_size()) -> ok.\nshash_ddists_fdists(Dir, FileName, N, EvalRepeats, FR, ExpDelta, StepSize) ->\n    shash(Dir, FileName, N, EvalRepeats, FR, ExpDelta, ?EVAL_DDISTS, ?EVAL_FDISTS, StepSize).\n\n-spec shash(DestDir::string(), FileName::string(), N::pos_integer(),\n            EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n            DDists::[data_distribution()], FDists::[fail_distribution()],\n            StepSize::step_size()) -> ok.\nshash(Dir, FileName, N, EvalRepeats, FR, ExpDelta, DDists, FDists, StepSize) ->\n    Scenario = #scenario{ ring_type = uniform,\n                          data_type = random },\n    PairRing = #ring_config{ data_count = N,\n                             node_count = 4,\n                             fquadrants = [1,3],\n                             data_failure_rate = 3,\n                             round = 1 },\n    Options = [{eval_dir, Dir}, {filename, FileName}, {eval_repeats, EvalRepeats}],\n    \n    SHash = #rc_config{ recon_method = shash, recon_fail_rate = FR,\n                        expected_delta = ExpDelta },\n    \n    eval(pair,\n         gen_setup(DDists, ?EVAL_FTYPES, FDists, Scenario, PairRing, [SHash]),\n         fprob, 4, StepSize, StepSize, Options),\n    ok.\n\n-spec shash_scale(DestDir::string(), FileName::string(), N::pos_integer(),\n                  EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob) -> ok.\nshash_scale(Dir, FileName, N, EvalRepeats, FR, ExpDelta) ->\n    shash_custom(Dir, FileName, N, EvalRepeats, FR, ExpDelta,\n                 {power, 4}, 5, ?EVAL_FTYPES, 3, data_count).\n\n-spec shash_custom(DestDir::string(), FileName::string(), N::pos_integer(),\n                   EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n                   StepInc::step_inc(), Steps::pos_integer(),\n                   FTypes::[update | regen], Delta::pos_integer(),\n                   step_param()) -> ok.\nshash_custom(Dir, FileName, N, EvalRepeats, FR, ExpDelta,\n             StepInc, Steps, FTypes, Delta, StepParam) ->\n    SHash = #rc_config{ recon_method = shash, recon_fail_rate = FR,\n                        expected_delta = ExpDelta },\n    alg_custom(Dir, FileName, N, EvalRepeats, SHash,\n               StepInc, Steps, FTypes, Delta, StepParam).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% BLOOM EVAL\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec bloom(DestDir::string(), FileName::string(), N::pos_integer(),\n            EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n            StepSize::step_size()) -> ok.\nbloom(Dir, FileName, N, EvalRepeats, FR, ExpDelta, StepSize) ->\n    bloom(Dir, FileName, N, EvalRepeats, FR, ExpDelta, [random], [random], StepSize).\n\n-spec bloom_ddists_fdists(DestDir::string(), FileName::string(), N::pos_integer(),\n                          EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n                          StepSize::step_size()) -> ok.\nbloom_ddists_fdists(Dir, FileName, N, EvalRepeats, FR, ExpDelta, StepSize) ->\n    bloom(Dir, FileName, N, EvalRepeats, FR, ExpDelta, ?EVAL_DDISTS, ?EVAL_FDISTS, StepSize).\n\n-spec bloom(DestDir::string(), FileName::string(), N::pos_integer(),\n            EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n            DDists::[data_distribution()], FDists::[fail_distribution()],\n            StepSize::step_size()) -> ok.\nbloom(Dir, FileName, N, EvalRepeats, FR, ExpDelta, DDists, FDists, StepSize) ->\n    Scenario = #scenario{ ring_type = uniform,\n                          data_type = random },\n    PairRing = #ring_config{ data_count = N,\n                             node_count = 4,\n                             fquadrants = [1,3],\n                             data_failure_rate = 3,\n                             round = 1 },\n    Options = [{eval_dir, Dir}, {filename, FileName}, {eval_repeats, EvalRepeats}],\n    \n    Bloom = #rc_config{ recon_method = bloom, recon_fail_rate = FR,\n                        expected_delta = ExpDelta },\n    \n    eval(pair,\n         gen_setup(DDists, ?EVAL_FTYPES, FDists, Scenario, PairRing, [Bloom]),\n         fprob, 4, StepSize, StepSize, Options),\n    ok.\n\n-spec bloom_scale(DestDir::string(), FileName::string(), N::pos_integer(),\n                  EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob) -> ok.\nbloom_scale(Dir, FileName, N, EvalRepeats, FR, ExpDelta) ->\n    bloom_custom(Dir, FileName, N, EvalRepeats, FR, ExpDelta,\n                 {power, 4}, 5, ?EVAL_FTYPES, 3, data_count).\n\n-spec bloom_custom(DestDir::string(), FileName::string(), N::pos_integer(),\n                   EvalRepeats::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n                   StepInc::step_inc(), Steps::pos_integer(),\n                   FTypes::[update | regen], Delta::pos_integer(),\n                   step_param()) -> ok.\nbloom_custom(Dir, FileName, N, EvalRepeats, FR, ExpDelta,\n             StepInc, Steps, FTypes, Delta, StepParam) ->\n    Bloom = #rc_config{ recon_method = bloom, recon_fail_rate = FR,\n                        expected_delta = ExpDelta },\n    alg_custom(Dir, FileName, N, EvalRepeats, Bloom,\n               StepInc, Steps, FTypes, Delta, StepParam).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% MERKLE EVAL\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec merkle(DestDir::string(), FileName::string(), N::pos_integer(),\n             EvalRepeats::pos_integer(), MBranch::pos_integer(),\n             MBucket::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n             StepSize::step_size()) -> ok.\nmerkle(Dir, FileName, N, EvalRepeats, MBranch, MBucket, FR, ExpDelta, StepSize) ->\n    merkle(Dir, FileName, N, EvalRepeats, MBranch, MBucket, FR, ExpDelta,\n           [random], [random], StepSize).\n\n-spec merkle_ddists_fdists(DestDir::string(), FileName::string(),\n                           N::pos_integer(), EvalRepeats::pos_integer(),\n                           MBranch::pos_integer(), MBucket::pos_integer(),\n                           FR::fail_rate(), ExpDelta::number() | as_fprob,\n                           StepSize::step_size()) -> ok.\nmerkle_ddists_fdists(Dir, FileName, N, EvalRepeats, MBranch, MBucket, FR,\n                     ExpDelta, StepSize) ->\n    merkle(Dir, FileName, N, EvalRepeats, MBranch, MBucket, FR, ExpDelta,\n           ?EVAL_DDISTS, ?EVAL_FDISTS, StepSize).\n\n-spec merkle(DestDir::string(), FileName::string(), N::pos_integer(),\n             EvalRepeats::pos_integer(), MBranch::pos_integer(),\n             MBucket::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n             DDists::[data_distribution()], FDists::[fail_distribution()],\n             StepSize::step_size()) -> ok.\nmerkle(Dir, FileName, N, EvalRepeats, MBranch, MBucket, FR, ExpDelta,\n       DDists, FDists, StepSize) ->\n    Scenario = #scenario{ ring_type = uniform,\n                          data_type = random },\n    PairRing = #ring_config{ data_count = N,\n                             node_count = 4,\n                             fquadrants = [1,3],\n                             data_failure_rate = 3,\n                             round = 1 },\n    Options = [{eval_dir, Dir}, {filename, FileName}, {eval_repeats, EvalRepeats}],\n    \n    Merkle = #rc_config{ recon_method = merkle_tree, recon_fail_rate = FR,\n                         expected_delta = ExpDelta,\n                         merkle_branch = MBranch, merkle_bucket = MBucket,\n                         merkle_num_trees = 1 },\n    \n    eval(pair,\n         gen_setup(DDists, ?EVAL_FTYPES, FDists, Scenario, PairRing, [Merkle]),\n         fprob, 4, StepSize, StepSize, Options),\n    ok.\n\n-spec merkle_scale(DestDir::string(), FileName::string(), N::pos_integer(),\n                   EvalRepeats::pos_integer(), MBranch::pos_integer(),\n                   MBucket::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob) -> ok.\nmerkle_scale(Dir, FileName, N, EvalRepeats, MBranch, MBucket, FR, ExpDelta) ->\n    merkle_custom(Dir, FileName, N, EvalRepeats, MBranch, MBucket, FR, ExpDelta,\n                  [random], [random], {power, 4}, 5, ?EVAL_FTYPES, 3, data_count).\n\n-spec merkle_custom(DestDir::string(), FileName::string(), N::pos_integer(),\n                    EvalRepeats::pos_integer(), MBranch::pos_integer(),\n                    MBucket::pos_integer(), FR::fail_rate(), ExpDelta::number() | as_fprob,\n                    DDists::[data_distribution()], FDists::[fail_distribution()],\n                    StepInc::step_inc(), Steps::pos_integer(),\n                    FTypes::[update | regen], Delta::pos_integer(),\n                    step_param()) -> ok.\nmerkle_custom(Dir, FileName, N, EvalRepeats, MBranch, MBucket, FR, ExpDelta,\n              DDists, FDists, StepInc, Steps, FTypes, Delta, StepParam) ->\n    Merkle = #rc_config{ recon_method = merkle_tree, recon_fail_rate = FR,\n                         expected_delta = ExpDelta,\n                         merkle_branch = MBranch, merkle_bucket = MBucket,\n                         merkle_num_trees = 1 },\n    alg_custom(Dir, FileName, N, EvalRepeats, Merkle,\n               DDists, FDists, StepInc, Steps, FTypes, Delta, StepParam).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% ART EVAL\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec art(DestDir::string(), FileName::string(), N::pos_integer(),\n          EvalRepeats::pos_integer(), MBranch::pos_integer(),\n          MBucket::pos_integer(), ACorrFactor::non_neg_integer(),\n          ExpDelta::number() | as_fprob) -> ok.\nart(Dir, FileName, N, EvalRepeats, MBranch, MBucket, ACorrFactor, ExpDelta) ->\n    Scenario = #scenario{ ring_type = uniform,\n                          data_type = random },\n    PairRing = #ring_config{ data_count = N,\n                             node_count = 4,\n                             fquadrants = [1,3],\n                             data_failure_rate = 3,\n                             round = 1 },\n    Options = [{eval_dir, Dir}, {filename, FileName}, {eval_repeats, EvalRepeats}],\n    \n    Art = #rc_config{recon_method = art, expected_delta = ExpDelta,\n                     merkle_bucket = MBucket, merkle_branch = MBranch,\n                     merkle_num_trees = 1,\n                     art_corr_factor = ACorrFactor,\n                     art_inner_fpr = 0.01, art_leaf_fpr = 0.01},\n    StepSize = 2,\n\n    eval(pair,\n         gen_setup([random], ?EVAL_FTYPES, [random], Scenario, PairRing, [Art]),\n         fprob, 4, StepSize, StepSize, Options),\n    ok.\n\n-spec art_scale(DestDir::string(), FileName::string(), N::pos_integer(),\n                EvalRepeats::pos_integer(), MBranch::pos_integer(),\n                MBucket::pos_integer(), ACorrFactor::non_neg_integer(),\n                ExpDelta::number() | as_fprob) -> ok.\nart_scale(Dir, FileName, N, EvalRepeats, MBranch, MBucket, ACorrFactor, ExpDelta) ->\n    Scenario = #scenario{ ring_type = uniform,\n                          data_type = random },\n    PairRing = #ring_config{ data_count = N,\n                             node_count = 4,\n                             fquadrants = [1,3],\n                             data_failure_rate = 3,\n                             round = 1 },\n    Options = [{eval_dir, Dir}, {filename, FileName}, {eval_repeats, EvalRepeats}],\n    \n    Art = #rc_config{recon_method = art, expected_delta = ExpDelta,\n                     merkle_bucket = MBucket, merkle_branch = MBranch,\n                     merkle_num_trees = 1,\n                     art_corr_factor = ACorrFactor, art_inner_fpr = 0.01, art_leaf_fpr = 0.01},  \n\n    eval(pair,\n         gen_setup([random], ?EVAL_FTYPES, [random], Scenario, PairRing, [Art]),\n         data_count, 5, {power, 4}, N, Options),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Generic\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec alg_custom(DestDir::string(), FileName::string(), N::pos_integer(),\n                 EvalRepeats::pos_integer(), Algorithm::rc_config(),\n                 StepInc::step_inc(), Steps::pos_integer(),\n                 FTypes::[update | regen], Delta::pos_integer(),\n                 step_param()) -> ok.\nalg_custom(Dir, FileName, N, EvalRepeats, RCConfig,\n           StepInc, Steps, FTypes, Delta, StepParam) ->\n    alg_custom(Dir, FileName, N, EvalRepeats, RCConfig,\n               [random], [random], StepInc, Steps, FTypes, Delta, StepParam).\n\n-spec alg_custom(DestDir::string(), FileName::string(), N::pos_integer(),\n                 EvalRepeats::pos_integer(), Algorithm::rc_config(),\n                 DDists::[data_distribution()], FDists::[fail_distribution()],\n                 StepInc::step_inc(), Steps::pos_integer(),\n                 FTypes::[update | regen], Delta::pos_integer(),\n                 step_param()) -> ok.\nalg_custom(Dir, FileName, N, EvalRepeats, RCConfig,\n           DDists, FDists, StepInc, Steps, FTypes, Delta, StepParam) ->\n    Scenario = #scenario{ ring_type = uniform,\n                          data_type = random },\n    PairRing = #ring_config{ data_count = N,\n                             node_count = 4,\n                             fquadrants = [1,3],\n                             data_failure_rate = Delta,\n                             round = 1 },\n    Options = [{eval_dir, Dir}, {filename, FileName}, {eval_repeats, EvalRepeats}],\n    \n    Init = get_param_value({PairRing, RCConfig}, StepParam),\n    eval(pair,\n         gen_setup(DDists, FTypes, FDists,\n                   Scenario, PairRing, [RCConfig]),\n         StepParam, Steps, StepInc, Init, Options),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% SYSTEM EVAL\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec system(DestDir::string(), N::pos_integer(), EvalRepeats::pos_integer(),\n             EvalName::atom()) -> ok.\nsystem(Dir, N, EvalRepeats, EvalName) ->\n    Nodes = 16,\n    Scenario = #scenario{ ring_type = uniform,\n                          data_type = random },\n    Ring = #ring_config{ data_count = N,\n                         node_count = Nodes,\n                         fquadrants = all,\n                         data_failure_rate = 4*3,\n                         round = 1 },\n    \n    Bloom0 = #rc_config{ recon_method = bloom, recon_fail_rate = 0.01,\n                         expected_delta = 100 },\n    %Bloom = #rc_config{ recon_method = bloom, recon_fail_rate = 0.1 },\n    %Merkle1 = #rc_config{ recon_method = merkle_tree, recon_fail_rate = 0.01, merkle_branch = 4, merkle_bucket = 4, merkle_num_trees = 1 },\n    %Merkle2 = #rc_config{ recon_method = merkle_tree, recon_fail_rate = 0.01, merkle_branch = 4, merkle_bucket = 1, merkle_num_trees = 1 },\n    %Art1 = #rc_config{ recon_method = art, art_corr_factor = 3, merkle_bucket = 4, merkle_branch = 16, merkle_num_trees = 1, art_inner_fpr = 0.001, art_leaf_fpr = 0.01 },\n    %Art2 = #rc_config{ recon_method = art, art_corr_factor = 3, merkle_bucket = 2, merkle_branch = 32, merkle_num_trees = 1, art_inner_fpr = 0.01, art_leaf_fpr = 0.2 },\n\n    case EvalName of\n        a0 -> eval(sys,\n                   gen_setup([uniform], ?EVAL_FTYPES, [random],\n                             Scenario, Ring, [Bloom0]),\n                   rounds, 2*Nodes, 2*Nodes, 0,\n                   [{eval_dir, Dir}, {file_suffix, atom_to_list(EvalName)},\n                    {start_ep_id, 1}, {eval_repeats, EvalRepeats}])\n    end,\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% EVAL\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec eval(Mode::pair | sys, [ring_setup(),...], StepP::step_param(),\n           StepCount::pos_integer(), StepInc::step_inc(),\n           StepInit::step_size(), Options::eval_options()) -> ok.\neval(Mode, Setups, StepParam, StepCount, StepInc, Init, Options0) ->\n    {{YY, MM, DD}, {Hour, Min, Sec}} = erlang:localtime(),\n    Options = [{eval_time, {Hour, Min, Sec}} | Options0],\n    \n    StartEPId = proplists:get_value(start_ep_id, Options, 0),\n    Dir = proplists:get_value(eval_dir, Options, element(1, default_dir_and_name(unknown))),\n    FileName = proplists:get_value(filename, Options, io_lib:format(\"~p.dat\", [StepParam])),\n    EvalRepeats = proplists:get_value(eval_repeats, Options),\n    ?ASSERT(EvalRepeats =/= undefined),\n    \n    CreatedComment = io_lib:format(\"CREATED ~p-~p-~p~c~p:~p:~p\",\n                                   [YY, MM, DD, ?TAB, Hour, Min, Sec]),\n    {EPFileDevice, _EPFilePath} =\n        rr_eval_export:create_file(\n          [{filename, FileName}, {dir, Dir},\n           {comment, [CreatedComment]},\n           {column_names, rr_eval_point:column_names()}]),\n    {MPFileDevice, _MPFilePath} =\n        rr_eval_export:create_file(\n          [{filename, string:join([\"MP_\", FileName], \"\")},\n           {dir, filename:join([Dir, \"raw\"])},\n           {comment, [CreatedComment]},\n           {column_names, rr_eval_point:mp_column_names()}]),\n    NOptions = [{ep_file, EPFileDevice}, {mp_file, MPFileDevice} | Options],\n    \n    StartT = os:timestamp(),\n    \n    lists:foldl(\n      fun({Scenario, RingP0, ReconP0}, EPId) ->\n              ReconP = init_rc_conf(ReconP0, StepParam, Init),\n              RingP = init_ring_conf(RingP0, StepParam, Init),\n\n              SetupText = eval_setup_comment(Scenario, RingP, StepParam, StepInc, EvalRepeats),\n              ReconText = rc_conf_comment(ReconP),\n              rr_eval_export:write_ds(EPFileDevice, {[{comment, lists:append(SetupText, [ReconText])}], []}),\n              rr_eval_export:write_ds(MPFileDevice, {[{comment, lists:append(SetupText, [ReconText])}], []}),\n              \n              RingSetup = {Scenario, RingP, ReconP},\n              % NOTE - Mode=Sys only supports StepParam=rounds\n              %        Mode=Pair does not support StepParam=rounds\n              {_EP, _MP, NextEPId} = case Mode of\n                                         sys -> system_sync(RingSetup, NOptions, StepCount, EPId);\n                                         pair -> pair_sync(RingSetup, NOptions, StepParam, StepInc, StepCount, {[], [], EPId})\n                                     end,\n              NextEPId\n      end, StartEPId, Setups),\n    \n    rr_eval_export:close_file(EPFileDevice),\n    rr_eval_export:close_file(MPFileDevice),\n    \n    TimeDiff = erlang:round(timer:now_diff(os:timestamp(), StartT) / (1000*1000)),\n    {_, {NH, NM, NS}} = erlang:localtime(),\n    io:format(\"~n~cFinished at ~p:~p:~p~c in ~c~p:~p:~p~n~n~n\",\n              [?TAB, NH, NM, NS, ?TAB, ?TAB,\n               TimeDiff div (60*60),\n               (TimeDiff div 60) rem 60,\n               TimeDiff rem 60]),\n    ok.\n\n-spec get_node_interval(NodePid::comm:mypid()) -> intervals:interval().\nget_node_interval(NodePid) ->\n    comm:send(NodePid, {get_state, comm:this(), my_range}),\n    receive\n        ?SCALARIS_RECV({get_state_response, MyI}, MyI)\n    end.\n\n-spec get_db(NodePid::comm:mypid()) -> {Size::non_neg_integer(), db_dht:db_as_list()}.\nget_db(NodePid) ->\n    MyI = get_node_interval(NodePid),\n    comm:send(NodePid, {get_chunk, self(), MyI, all}),\n    D = receive\n            ?SCALARIS_RECV({get_chunk_response, {_, DB}}, DB)\n        end,\n%rr_recon:map_key_to_quadrant(db_entry:get_key(X), 1),\n    {length(D), [{db_entry:get_key(X), db_entry:get_version(X), db_entry:get_value(X) }\n                 || X <- D]}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% SYNC RUN\n%% repeat ?EVAL_Repeats-Times and rrepair sync in a given setup\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Starts one sync between 2 nodes.\n-spec pair_sync(ring_setup(), eval_options(), step_param(),\n                IncSize::step_inc(), Steps::pos_integer(),\n                Acc::sync_result()) -> sync_result().\npair_sync(_RingSetup, _Options, _IncParam, _IncSize, -1, Acc) ->\n    Acc;\npair_sync(Setup = {Scen, RingP, ReconP}, Options, IncParam, IncSize, StepCount, {AccEP, AccMP, EPId}) ->\n    \n    RunRound = fun(Nodes) ->\n                       % the nodelist may not be sorted by quadrants!\n                       % -> sort the nodes by their RT keys first:\n                       [{_AKey, _A}, {BKey, B}, {_CKey, _C}, {_DKey, D}] =\n                           lists:keysort(1, [{pid_to_rtkey(N), N} || N <- Nodes]),\n                       %B->D\n%%                        comm:send(B, {request_sync, ReconP#rc_config.recon_method,\n%%                                      DKey, comm:this()}, [{group_member, rrepair}]),\n%%                        wait_sync_end([B, D])\n                       %D->B\n                       comm:send(D, {request_sync, ReconP#rc_config.recon_method,\n                                     BKey, comm:this()}, [{group_member, rrepair}]),\n                       wait_sync_end([D, B], true)\n               end,\n    \n    EPFile = proplists:get_value(ep_file, Options, null),\n    MPFile = proplists:get_value(mp_file, Options, null),\n    EvalRepeats = proplists:get_value(eval_repeats, Options),\n    ?ASSERT(EvalRepeats =/= undefined),\n    StepValue =\n        case IncSize of\n            {power, Base} -> get_param_value({RingP, ReconP}, IncParam) * util:pow(Base, StepCount);\n            _ -> (IncSize * StepCount) + get_param_value({RingP, ReconP}, IncParam)\n        end,\n    {StepRing, StepRC} = set_params({RingP, ReconP}, IncParam, StepValue),\n    io:format(\">EVALUATE STEPS LEFT: ~p (Repeats per Step: ~p)~cStepValue=~p~n~c~s~n\",\n              [StepCount, EvalRepeats, ?TAB, StepValue, ?TAB, rc_conf_comment(StepRC)]),\n    _ = [io:format(\"~c~s~n\", [?TAB, X]) || X <- eval_setup_comment(Scen, StepRing, IncParam, StepValue, EvalRepeats)],\n    io:format(\"-----------------------------------------------------------------------~n\"),\n    StartT = os:timestamp(),\n    \n    TraceName = rr_eval_trace,\n    %Trace Export Parameter\n%    EvalDir = proplists:get_value(eval_dir, Options, \"../\"),\n%    {Hour, Min, Sec} = proplists:get_value(eval_time, Options, {0, 0, 0}),\n    \n    % spawn a separate process for writing measure points:\n    Self = self(),\n    MPFileWriter =\n        if MPFile =/= null ->\n               erlang:spawn_link(\n                 fun() ->\n                         _ = util:for_to_ex(\n                               1, EvalRepeats,\n                               fun(_I) ->\n                                       receive MPRow ->\n                                                   rr_eval_export:write_ds(\n                                                     MPFile, {[], [tuple_to_list(MPRow)]})\n                                       end\n                               end),\n                         Self ! mp_write_done\n                 end);\n           true -> nopid\n        end,\n\n    MPList =\n        util:for_to_ex(\n          1, EvalRepeats,\n          fun(I) ->\n                  ActI = EvalRepeats - I + 1,\n                  io:format(\"~p \", [ActI]),\n                  {_DBSize, _Load, Missing, Outdated} =\n                      build_dht({Scen, StepRing, StepRC}),\n                  InitMP = {Missing, Outdated},\n                  NodeList = get_node_list(),\n                  ?ASSERT2(length(NodeList) =:= StepRing#ring_config.node_count,\n                           \"NODES NOT COMPLETE, ABORT\"),\n                  \n                  %start sync\n                  trace_mpath:start(TraceName, [{map_fun, fun bw_map_fun/3},\n                                                {filter_fun, fun bw_filter_fun/1}]),\n                  SessionStats = RunRound(NodeList),\n                  io:format(\"S \"), % code for a successful sync run\n                  trace_mpath:stop(),\n                  Trace = trace_mpath:get_trace(TraceName, cleanup),\n                  \n                  MP = get_measure_point(EPId, ActI, 1, InitMP, Trace, NodeList, SessionStats),\n                  \n                  log:pal(\"Regenerated: ~B/~B, Updated: ~B/~B\",\n                          [element(5, MP), element(4, MP),\n                           element(7, MP), element(6, MP)]),\n                  \n                  %Trace export\n                  %rr_eval_export:write_raw(Trace, [{filename, io_lib:format(\"~p-~p-~p_TRACE_ID~p_I~p\", [Hour, Min, Sec, EPId, ActI])},\n                  %                              {subdir, io_lib:format(\"~s/Trace\", [EvalDir])}]),\n                  \n                  reset(),\n                  MPFile =/= null andalso MPFileWriter ! MP,\n                  MP\n          end),\n    \n    TimeDiff = erlang:round(timer:now_diff(os:timestamp(), StartT) / (1000*1000)),\n    io:format(\"~n~c~c~cSTEP TIME=~ph ~pm ~ps~n~n\", [?TAB, ?TAB, ?TAB,\n                                                      TimeDiff div (60*60),\n                                                      (TimeDiff div 60) rem 60,\n                                                      TimeDiff rem 60]),\n    \n    EP = rr_eval_point:generate_ep(EPId, {Scen, StepRing, StepRC}, MPList),\n    \n    % WRITE LINE\n    EPFile =/= null andalso\n        rr_eval_export:write_ds(EPFile, {[], [tuple_to_list(EP)]}),\n    MPFile =/= null andalso receive mp_write_done -> true end,\n    \n    pair_sync(Setup, Options, IncParam, IncSize, StepCount - 1, {[EP | AccEP], lists:append(MPList, AccMP), EPId + 1}).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n%% @doc Builds an Scalaris Ring and runs [Rounds]-Times syncs.\n%%      The nodes will clock-wise be selected to sync with an related node.\n%%      eg. in a system with 6 nodes and 6 rounds every node will initiate one sync with another node.\n%%      in a system with 6 nodes and 7 rounds all nodes start one sync exept node 0 with will start two. \n-spec system_sync(ring_setup(),\n                   eval_options(),\n                   Rounds::pos_integer(),\n                   EPId::non_neg_integer()) -> sync_result().\nsystem_sync({Scen, RingP, ReconP}, Options, Rounds, EPId) ->\n    EPFile = proplists:get_value(ep_file, Options, null),\n    MPFile = proplists:get_value(mp_file, Options, null),\n    EvalRepeats = proplists:get_value(eval_repeats, Options),\n    ?ASSERT(EvalRepeats =/= undefined),\n\n    io:format(\">EVALUATE IN ROUNDS Repeats: ~p - Rounds=~p - ~s~n\", [EvalRepeats, Rounds, rc_conf_comment(ReconP)]),\n    _ = [io:format(\"~c~s~n\", [?TAB, X]) || X <- eval_setup_comment(Scen, RingP, rounds, Rounds, EvalRepeats)],\n    io:format(\"-----------------------------------------------------------------------~n\"),\n    StartT = os:timestamp(),\n    TraceName = rr_eval_trace,\n    \n    Results =\n        util:for_to_ex(\n          1, EvalRepeats,\n          fun(I) ->\n                  ActI = EvalRepeats - I + 1,\n                  io:format(\"~n~p r\", [ActI]),\n                  {_DBSize, Load, _Missing, Outdated} =\n                      build_dht({Scen, RingP, ReconP}),\n                  InitMO = {Load, Outdated},\n                  NodeList = get_node_list(),\n                  ?ASSERT2(length(NodeList) =:= RingP#ring_config.node_count,\n                           \"NODES NOT COMPLETE, ABORT\"),\n                  FirstMP = get_mp_round(0, ActI, 0, InitMO, [], NodeList),\n                  \n                  %start sync\n                  {_, _, MPL} =\n                      lists:foldl(\n                        fun(Round, {[ActNode|RNodes], LastMO, AccMP}) ->\n                                io:format(\"~p-\", [Round]),\n                                trace_mpath:start(TraceName, [{map_fun, fun bw_map_fun/3}, {filter_fun, fun bw_filter_fun/1}]),\n                                start_round([ActNode]),\n                                trace_mpath:stop(),\n                                Trace = trace_mpath:get_trace(TraceName),\n                                trace_mpath:cleanup(TraceName),\n                                \n                                MP = get_mp_round(EPId + Round, ActI, Round, LastMO, Trace, NodeList),\n                                {lists:append(RNodes, [ActNode]), {element(4, MP), element(6, MP)}, [MP|AccMP]}\n                        end,\n                        {NodeList, InitMO, []},\n                        lists:seq(1, Rounds)),\n                  reset(),\n                  {MPL, FirstMP}\n          end),\n    MPList = lists:flatten([element(1, X) || X <- Results]),\n    NullMP = element(2, hd(Results)),\n    \n    TimeDiff = erlang:round(timer:now_diff(os:timestamp(), StartT) / (1000*1000)),\n    io:format(\"~n~c~c~cSTEP TIME=~p:~p:~p~n~n\", [?TAB, ?TAB, ?TAB,\n                                                 TimeDiff div (60*60),\n                                                 (TimeDiff div 60) rem 60,\n                                                 TimeDiff rem 60]),\n    \n    EPList0 = rr_eval_point:generate_ep_rounds(EPId, {Scen, RingP, ReconP}, MPList, Rounds),\n    EPList = lists:append(EPList0, [rr_eval_point:generate_ep(0, {Scen, RingP, ReconP}, [NullMP])]),\n    \n    % WRITE LINE\n    EPFile =/= null andalso\n        rr_eval_export:write_ds(EPFile, {[], [tuple_to_list(EPRow) || EPRow <- EPList]}),\n    MPFile =/= null andalso\n        rr_eval_export:write_ds(MPFile, {[], [tuple_to_list(Row) || Row <- MPList]}),\n    \n    {EPList, MPList, EPId + Rounds}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% API Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec build_dht(ring_setup()) -> db_generator:db_status().\nbuild_dht({#scenario{ ring_type = RingType,\n                      data_type = DBType,\n                      data_distribution = DBDist0,\n                      data_failure_type = FType,\n                      fail_distribution = FDist0,\n                      trigger_prob = TProb\n                     },\n           #ring_config{ data_count = DBSize,\n                         data_failure_rate = FProb,\n                         node_count = NodeCount,\n                         fquadrants = FDest },\n           RCParams}) ->\n    _ = set_config(RCParams, TProb),\n    make_ring(RingType, NodeCount),\n    \n    DBDist = case DBDist0 of\n                 uniform -> uniform;\n                 random -> random;\n                 {binomial, P1} -> {non_uniform, random_bias:binomial(DBSize - 1, P1)}\n             end,\n    FDist = case FDist0 of\n                uniform -> uniform;\n                random -> random;\n                {binomial, _P2} when FProb == 0 -> uniform;\n                {binomial, P2} when FProb > 0 ->\n                    DBErrors = erlang:round(DBSize * (FProb / 100)),\n                    {non_uniform, random_bias:binomial(DBErrors - 1, P2)}\n            end,\n    \n    fill_ring(DBType, DBSize, [{ftype, FType},\n                               {fprob, FProb},\n                               {distribution, DBDist},\n                               {fdest, FDest},\n                               {fdistribution, FDist}]).\n\n-spec make_ring(ring_type(), pos_integer()) -> ok.\nmake_ring(Type, Size) ->\n    _ = case Type of\n            random ->\n                _ = admin:add_node([{first}]),\n                admin:add_nodes(Size -1);\n            uniform ->\n                RemIds = ?RT:get_split_keys(?MINUS_INFINITY, ?PLUS_INFINITY, Size),\n                _ = admin:add_node([{first}, {{dht_node, id}, ?MINUS_INFINITY}]),\n                [admin:add_node_at_id(Id) || Id <- RemIds]\n        end,\n    unittest_helper:check_ring_size_fully_joined(Size),\n    unittest_helper:wait_for_stable_ring(),\n    io:format(\"R\"), % code for a successfully started ring\n    ok.\n\n% @doc  DBSize=Number of Data Entities in DB (without replicas)\n-spec fill_ring(db_generator:db_type(), pos_integer(), [db_generator:db_parameter()]) -> db_generator:db_status().\nfill_ring(Type, DBSize, Params) ->\n    DbStatus = {Entries, _Existing, Missing, Outdated} =\n                   db_generator:fill_ring(Type, DBSize, Params),\n    {fprob, FProb} = lists:keyfind(fprob, 1, Params),\n    ?ASSERT2(Entries =:= DBSize * config:read(replication_factor),\n             {incorrect_db_size, Entries, DBSize}),\n    ?ASSERT2((Missing + Outdated) =:= erlang:round((FProb / 100) * DBSize),\n             {incorrect_fprob, Missing + Outdated, FProb * DBSize}),\n    erlang:put(?DBSizeKey, element(1, DbStatus)),\n    ?DBG_ASSERT2(DbStatus =:= (DbStatus2 = get_db_status2(get_node_list())),\n                 {different_db_status, DbStatus, DbStatus2}),\n    io:format(\"F\"), % code for a successfully filled ring\n    % more details for debugging:\n%%     Data = lists:keysort(2, unittest_helper:get_ring_data(full)),\n%%     _ = [begin\n%%              OldX = lists:foldl(fun(E, Acc) ->\n%%                                         case db_entry:get_value(E) of\n%%                                             old -> Acc + 1;\n%%                                             _   -> Acc\n%%                                         end\n%%                                 end, 0, DBEntries),\n%%              ExistingX = length(DBEntries),\n%%              log:pal(\"ID: ~.2p Old/Exist/Target: ~B/~B/~B\",\n%%                      [R, OldX, ExistingX, DBSize])\n%%          end || {_Node, {_LBr, _L, R, _RBr}, DBEntries, {pred, _Pred},\n%%                  {succ, _Succ}, ok} <- Data],\n    DbStatus.\n\n-spec reset() -> ok | failed.\nreset() ->\n    Nodes = api_vm:get_nodes(),\n    {_, NotFound} = api_vm:kill_nodes_by_name(Nodes),\n    NotFound =/= [] andalso\n        erlang:error(some_nodes_not_found),\n    case api_vm:number_of_nodes() of\n        0 -> ok;\n        _ -> failed\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec start_round([comm:mypid()]) -> ok.\nstart_round(Nodes) ->\n    lists:foreach(fun(N) ->\n                          comm:send(N, {rr_trigger}, [{group_member, rrepair}]),\n                          wait_sync_end(Nodes, false)\n                  end, Nodes),\n    ok.\n\n-spec wait_sync_end\n    (Nodes::[comm:mypid()], PrincipalUsed::false) -> ok;\n    (Nodes::[comm:mypid()], PrincipalUsed::true) -> rrepair:session().\nwait_sync_end(Nodes, true) ->\n    receive\n        ?SCALARIS_RECV({request_sync_complete, Stats}, ok)\n    end,\n    wait_sync_end(Nodes, false),\n    Stats;\nwait_sync_end(Nodes, false) ->\n    Req = {get_state, comm:this(), [open_sessions, open_recon, open_resolve]},\n    util:wait_for(fun() -> wait_for_sync_round_end2(Req, Nodes) end, 200),\n    % check whether there are still some running rrepair processes!\n    % NOTE: there is a gap between reporting the stats and actually killing\n    %       the process and it seems that this is sometimes hit by the check\n    % -> disable the check for now and rely on request_sync_complete and the\n    %    rrepair session management\n%%     {ok, Pat} = re:compile(\"^(rr_resolve|rr_recon)\\.[0-9]+$\"),\n%%     _ = [begin\n%%              Group = pid_groups:group_of(comm:make_local(N)),\n%%              RRProcs = [Name || Pid <- pid_groups:members(Group),\n%%                                 is_process_alive(Pid),\n%%                                 Name <- [element(2, pid_groups:group_and_name_of(Pid))],\n%%                                 is_list(Name),\n%%                                 re:run(Name, Pat) =/= nomatch],\n%%              ?ASSERT2(RRProcs =:= [], {running_rr_procs, RRProcs})\n%%          end || N <- Nodes],\n    ok.\n\n-spec wait_for_sync_round_end2(Req::comm:message(), Nodes::[comm:mypid()]) -> boolean().\nwait_for_sync_round_end2(_Req, []) -> true;\nwait_for_sync_round_end2(Req, [Node | Nodes]) ->\n    comm:send(Node, Req, [{group_member, rrepair}]),\n    KeyResult =\n        receive\n            ?SCALARIS_RECV(\n            {get_state_response, [Sessions, ORC, ORS]}, % ->\n            begin\n                if (ORC =:= 0 andalso ORS =:= 0 andalso\n                        Sessions =:= []) ->\n                       true;\n                   true ->\n%%                        log:pal(\"Node: ~.2p~nOS : ~.2p~nORC: ~p, ORS: ~p~n\",\n%%                                [Node, Sessions, ORC, ORS]),\n                       false\n                end\n            end)\n        end,\n    if KeyResult -> wait_for_sync_round_end2(Req, Nodes);\n       true -> false\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec set_config(rc_config(), 0..100) -> ok | {error, term()}.\nset_config(#rc_config{ recon_method = Method,\n                       recon_fail_rate = FR,\n                       expected_delta = ExpDelta,\n                       art_corr_factor = ArtCorrF,\n                       art_inner_fpr = ArtInnerFpr,\n                       art_leaf_fpr = ArtLeafFpr,\n                       merkle_branch = MerkleBranch,\n                       merkle_bucket = MerkleBucket,\n                       merkle_num_trees = MerkleNumTrees }, TriggerProb) ->\n    config:write(rrepair_enabled, true),\n    config:write(rrepair_after_crash, false), % disable (just in case)\n    config:write(rr_trigger_interval, 0), % disabled (we trigger manually!)\n    config:write(rr_session_ttl, 10*60*1000),     % 2 days\n    config:write(rr_gc_interval, 24*60*60*1000),  % 1 day\n    \n    config:write(rr_trigger_probability, TriggerProb),\n    config:write(rr_recon_method, Method),\n    % interpret the algorithm's parameter FR as the max hash size:\n    case {Method, FR} of\n        {merkle_tree, 160} ->\n            config:write(rr_recon_min_sig_size, FR),\n            config:write(rr_recon_failure_rate, 1.0e-128);\n        {_, 128} ->\n            config:write(rr_recon_min_sig_size, FR),\n            config:write(rr_recon_failure_rate, 1.0e-128);\n        _ ->\n            config:write(rr_recon_failure_rate, FR)\n    end,\n    config:write(rr_recon_expected_delta, ExpDelta),\n    config:write(rr_art_inner_fpr, ArtInnerFpr),\n    config:write(rr_art_leaf_fpr, ArtLeafFpr),\n    config:write(rr_art_correction_factor, ArtCorrF),\n    config:write(rr_merkle_branch_factor, MerkleBranch),\n    config:write(rr_merkle_bucket_size, MerkleBucket),\n    config:write(rr_merkle_num_trees, MerkleNumTrees),\n    RM = config:read(rr_recon_method),\n    case RM =:= Method of\n        true -> ok;\n        _ -> {error, set_failed}\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_param_value({ring_config(), rc_config()}, Param::step_param()) -> any().\nget_param_value({Ring, Recon}, Param) ->\n    case Param of\n        node_count -> Ring#ring_config.node_count;\n        data_count -> Ring#ring_config.data_count;\n        fprob -> Ring#ring_config.data_failure_rate;\n        rounds -> Ring#ring_config.round;\n        recon_fail_rate -> Recon#rc_config.recon_fail_rate;\n        expected_delta -> Recon#rc_config.expected_delta;\n        art_corr_factor -> Recon#rc_config.art_corr_factor;\n        art_inner_fpr -> Recon#rc_config.art_inner_fpr;\n        art_leaf_fpr -> Recon#rc_config.art_leaf_fpr;\n        merkle_branch -> Recon#rc_config.merkle_branch;\n        merkle_bucket -> Recon#rc_config.merkle_bucket;\n        merkle_num_trees -> Recon#rc_config.merkle_num_trees\n    end.\n\n-spec set_params({ring_config(), rc_config()}, Param::step_param(), Value::any()) -> {ring_config(), rc_config()}.\nset_params({RC, RCC = #rc_config{expected_delta = ExpDelta}}, Param, Value) ->\n    NRC = case Param of\n              node_count -> RC#ring_config{node_count = Value};\n              data_count -> RC#ring_config{data_count = Value};\n              fprob      -> RC#ring_config{data_failure_rate = Value};\n              rounds     -> RC#ring_config{round = Value};\n              _          -> RC\n          end,\n    RCC1 = case Param of\n               recon_fail_rate -> RCC#rc_config{recon_fail_rate = Value};\n               expected_delta  -> RCC#rc_config{expected_delta = Value};\n               art_corr_factor -> RCC#rc_config{art_corr_factor = Value};\n               art_inner_fpr   -> RCC#rc_config{art_inner_fpr = Value};\n               art_leaf_fpr    -> RCC#rc_config{art_leaf_fpr = Value};\n               merkle_branch   -> RCC#rc_config{merkle_branch = Value};\n               merkle_bucket   -> RCC#rc_config{merkle_bucket = Value};\n               merkle_num_trees-> RCC#rc_config{merkle_num_trees = Value};\n               _               -> RCC\n           end,\n    NRCC = if ExpDelta =:= as_fprob ->\n                   RCC1#rc_config{expected_delta = NRC#ring_config.data_failure_rate};\n              true -> RCC1\n           end,\n    {NRC, NRCC}.\n\n-spec init_rc_conf(rc_config(), step_param(), any()) -> rc_config().\ninit_rc_conf(RC, StepP, Init) ->\n    case StepP of\n        recon_fail_rate -> RC#rc_config{recon_fail_rate = Init};\n        expected_delta  -> RC#rc_config{expected_delta = Init};\n        art_corr_factor -> RC#rc_config{art_corr_factor = Init};\n        art_inner_fpr   -> RC#rc_config{art_inner_fpr = Init};\n        art_leaf_fpr    -> RC#rc_config{art_leaf_fpr = Init};\n        merkle_branch   -> RC#rc_config{merkle_branch = Init};\n        merkle_bucket   -> RC#rc_config{merkle_bucket = Init};\n        merkle_num_trees-> RC#rc_config{merkle_num_trees = Init};\n        _               -> RC\n    end.\n\ninit_ring_conf(RC, StepP, Init) ->\n    case StepP of\n        node_count -> RC#ring_config{node_count = Init};\n        data_count -> RC#ring_config{data_count = Init};\n        fprob      -> RC#ring_config{data_failure_rate = Init};\n        rounds     -> RC#ring_config{round = Init};\n        _          -> RC\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Helper\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec print_db_status(db_generator:db_status()) -> ok.\nprint_db_status({Count, Inserted, Missing, Outdated}) ->\n    io:format(\"Items=~p~nExisting=~p~nMissing=~p~nOutdated=~p~n\",\n              [Count, Inserted, Missing, Outdated]),\n    ok.\n\n-spec eval_setup_comment(scenario(), ring_config(), step_param(), StepSize::any(), Runs::pos_integer()) -> [string()].\neval_setup_comment(#scenario{ ring_type = RType,\n                              data_distribution = Dist,\n                              data_failure_type = FType,\n                              fail_distribution = FDist,\n                              data_type = DType,\n                              trigger_prob = TProb},\n                   #ring_config{ node_count = Nodes,\n                                 data_count = DBSize,\n                                 fquadrants = FQuadrants,\n                                 data_failure_rate = FProb,\n                                 round = Rounds}, StepParam, StepSize, Runs) ->\n    [io_lib:format(\"Scenario: Ring=~p~cDataDist=~p~cFailType=~p~cFailDist=~p~cDataType=~p~cTriggerProb=~p\",\n                   [RType, ?TAB, Dist, ?TAB, FType, ?TAB, FDist, ?TAB, DType, ?TAB, TProb]),\n     io_lib:format(\"Ring_Config: Nodes=~p~cDBSize=~p~cFProb=~p~cFQuadrants=~p~cRounds=~p\",\n                   [Nodes, ?TAB, DBSize, ?TAB, FProb, ?TAB, FQuadrants, ?TAB, Rounds]),\n     io_lib:format(\"Walk: ~p~cStepSize=~p~cRunsPerPoint=~p\",\n                   [StepParam, ?TAB, StepSize, ?TAB, Runs])].\n\n-spec rc_conf_comment(rc_config()) -> string().\nrc_conf_comment(#rc_config{ recon_method = trivial, recon_fail_rate = FR,\n                            expected_delta = ExpDelta }) ->\n    io_lib:format(\"Trivial: FR=~p~cexpectedDelta=~p\", [FR, ?TAB, ExpDelta]);\nrc_conf_comment(#rc_config{ recon_method = shash, recon_fail_rate = FR,\n                            expected_delta = ExpDelta }) ->\n    io_lib:format(\"SHash: FR=~p~cexpectedDelta=~p\", [FR, ?TAB, ExpDelta]);\nrc_conf_comment(#rc_config{ recon_method = bloom, recon_fail_rate = FR,\n                            expected_delta = ExpDelta }) ->\n    io_lib:format(\"Bloom: FR=~p~cexpectedDelta=~p\", [FR, ?TAB, ExpDelta]);\nrc_conf_comment(#rc_config{ recon_method = merkle_tree,\n                            recon_fail_rate = FR, expected_delta = ExpDelta,\n                            merkle_branch = Branch,\n                            merkle_bucket = Bucket,\n                            merkle_num_trees = NumTrees }) ->\n    io_lib:format(\"Merkle: FR=~p~cexpectedDelta=~p~cBranchSize=~p~cBucketSize=~p~cNumTrees=~p\",\n                  [FR, ?TAB, ExpDelta, ?TAB, Branch, ?TAB, Bucket, ?TAB, NumTrees]);\nrc_conf_comment(#rc_config{ recon_method = art, expected_delta = ExpDelta,\n                            art_corr_factor = Corr,\n                            art_inner_fpr = InnerFpr,\n                            art_leaf_fpr = LeafFpr }) ->\n    io_lib:format(\"Art: expectedDelta=~p~cCorrectionFactor=~p~cInnerFpr=~p~cLeafFpr=~p\",\n                  [ExpDelta, ?TAB, Corr, ?TAB, InnerFpr, ?TAB, LeafFpr]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% ACCURANCY MEASUREMENT\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_db_status(NodeList::[comm:mypid()]) -> db_generator:db_status().\nget_db_status(NodeList) ->\n    {DBSize, Stored, Missing, _Outdated} = Res = get_db_status2(NodeList),\n    DBSize - Stored < 0 andalso\n        log:pal(\"DBSIZE - STORED < 0 ->>>>>>>>>>>>>>>ERROR ~p-~p=~p\",\n                [DBSize, Stored, Missing]),\n    Res.\n\n-spec get_db_status2(NodeList::[comm:mypid()]) -> db_generator:db_status().\nget_db_status2(NodeList) ->\n    DBSize = erlang:get(?DBSizeKey),\n\n    {Stored, Outdated} =\n        lists:foldl(\n          fun({S, O}, {AS, AO}) -> {AS + S, AO + O} end,\n          {0, 0},\n          util:par_map(\n            fun(Node) ->\n                    comm:send(\n                      Node,\n                      {get_chunk, self(), intervals:all(),\n                       fun(Item) -> not db_entry:is_empty(Item) end,\n                       fun(Item) ->\n                               ?IIF(db_generator:is_old_value(\n                                      db_entry:get_value(Item)), 1, 0)\n                       end,\n                       all}),\n                    receive\n                        {get_chunk_response, {_, DBList}} ->\n                            {erlang:length(DBList), lists:sum(DBList)}\n                    end\n            end, NodeList)),\n    {DBSize, Stored, DBSize - Stored, Outdated}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% BANDWIDTH MEASUREMENT WITH MPATH TRACE\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% only trace send messages with first atom recon_map or resolve_map\n% messages with these atoms will be generated by bw_map_fun.\n% Trace command executes filter fun after map fun while tracing.\n-spec bw_filter_fun(trace_mpath:trace_event()) -> boolean().\nbw_filter_fun(Event) ->\n    case Event of\n        {log_send, _Time, _TraceId, Source, Dest, Msg, LorG} ->\n            case erlang:element(1, Msg) of\n                rr_map -> true;\n                recon_map -> true;\n                recon_map2 -> true;\n                resolve_map -> true;\n                unmapped ->\n                    Source1 = trace_mpath:normalize_pidinfo(Source),\n                    Dest1 = trace_mpath:normalize_pidinfo(Dest),\n                    case is_global_msg(Source1, Dest1, LorG) of\n                        true ->\n                            RealMsg = element(2, Msg),\n                            log:log(warn, \"Un-mapped global message: ~p -- ~.2p\",\n                                    [util:extint2atom(element(1, RealMsg)), RealMsg]),\n                            ok;\n                        unknown ->\n                            % this is most likely from our own process\n                            % requesting the number of open sessions!\n                            %RealMsg = element(2, Msg),\n                            %log:log(warn, \"Un-mapped unknown message: ~p -- ~.2p\",\n                            %        [util:extint2atom(element(1, RealMsg)), RealMsg]),\n                            ok;\n                        false ->\n                            ok\n                    end,\n                    false\n            end;\n        _ -> false\n    end.\n\n%% @doc Checks whether a message with the given source and destination as well\n%%      as the use of comm:send or comm:send_local is a message to another\n%%      Scalaris node or not.\n-spec is_global_msg(Source::trace_mpath:pidinfo(), Dest::trace_mpath:pidinfo(),\n                    Local_or_Global::global | local) -> boolean() | unknown.\nis_global_msg(_Source, _Dest, local) -> false;\n\nis_global_msg({_SPid, {Group, _SName}}, {_DPid, {Group, _DName}}, global) -> false;\nis_global_msg({_SPid, {_SGroup, _SName}}, {_DPid, {_DGroup, _DName}}, global) -> true;\n\nis_global_msg({_SPid, {_SGroup, _SName}}, {_DPid, non_local_pid_name_unknown}, global) -> false;\nis_global_msg({_SPid, no_pid_name},     {_DPid, non_local_pid_name_unknown}, global) -> false;\nis_global_msg({_SPid, non_local_pid_name_unknown}, {_DPid, {_DGroup, _DName}}, global) -> false;\nis_global_msg({_SPid, non_local_pid_name_unknown}, {_DPid, no_pid_name}, global) -> false;\nis_global_msg(_Source, _Dest, global) -> unknown.\n\n%% @doc Calculates the message size if needed.\n-spec msg_size(Msg::comm:message(), true) -> pos_integer();\n              (Msg::comm:message(), false) -> 0.\nmsg_size(Msg, true) ->\n    erlang:byte_size(erlang:term_to_binary(Msg, [{minor_version, 1}, {compressed, 2}]));\nmsg_size(_Msg, false) ->\n    0.\n\n%% @doc Reduces the traced message size by mapping recorded messages to a\n%%      shorter representation including its tag, size and number of KVV\n%%      triples (if a resolve message). Non-mapped messages will be wrapped in a\n%%      {unmapped, Msg} and remain intact but can be filtered out with\n%%      bw_filter_fun/1.\n%% Recon:   start_recon (inside continue_recon) will be mapped,\n%%          ?check_nodes and ?check_nodes_response (merkle_tree) will also be mapped\n%% Resolve: ?key_upd will be mapped\n%%          key_upd_send is a node-internal message and will be ommited\n-spec bw_map_fun(comm:message(), Source::pid() | comm:mypid(),\n                 Dest::pid() | comm:mypid())\n        -> {unmapped, Msg::comm:message()} |\n           {rr_map, ExtSize::pos_integer()} |\n           {recon_map | recon_map2, ExtSize::pos_integer()} |\n           {resolve_map, ExtSize::pos_integer(), KVVLen::non_neg_integer()}.\nbw_map_fun(MsgI, Source, Dest) ->\n    bw_map_fun(MsgI, Source, Dest, true).\n\n-spec bw_map_fun(comm:message(), Source::pid() | comm:mypid(),\n                 Dest::pid() | comm:mypid(), CalcMsgSize::boolean())\n        -> {unmapped, Msg::comm:message()} |\n           {rr_map, ExtSize::pos_integer()} |\n           {recon_map | recon_map2, ExtSize::pos_integer()} |\n           {resolve_map, ExtSize::pos_integer(), KVVLen::non_neg_integer()}.\nbw_map_fun({?key_upd, KVV, _ReqKeys} = Msg, _Source, _Dest, CalcMsgSize) ->\n    {resolve_map, msg_size(Msg, CalcMsgSize), length(KVV)};\n\nbw_map_fun({X, MsgI, _Options} = Msg, Source, Dest, CalcMsgSize)\n  when X =:= request_resolve orelse X =:= continue_resolve ->\n    bw_map_fun2(MsgI, Source, Dest, Msg, CalcMsgSize);\nbw_map_fun({X, _SID, MsgI, _Options} = Msg, Source, Dest, CalcMsgSize)\n  when X =:= request_resolve orelse X =:= continue_resolve ->\n    bw_map_fun2(MsgI, Source, Dest, Msg, CalcMsgSize);\n\nbw_map_fun({request_sync, _DestKey} = Msg, _Source, _Dest, CalcMsgSize) ->\n    {rr_map, msg_size(Msg, CalcMsgSize)};\nbw_map_fun({request_sync, _Method, _DestKey} = Msg, _Source, _Dest, CalcMsgSize) ->\n    {rr_map, msg_size(Msg, CalcMsgSize)};\nbw_map_fun({request_sync, _Method, _DestKey, _Principal} = Msg, _Source, _Dest, CalcMsgSize) ->\n    {rr_map, msg_size(Msg, CalcMsgSize)};\nbw_map_fun({start_recon, _SenderPid, _SId, {create_struct, _Method, _SenderI, _SenderLoad}} = Msg,\n           _Source, _Dest, CalcMsgSize) ->\n    {rr_map, msg_size(Msg, CalcMsgSize)};\nbw_map_fun({continue_recon, _SenderPid, _SId, {create_struct, _Method, _SenderI, _SenderLoad}} = Msg,\n           _Source, _Dest, CalcMsgSize) ->\n    {rr_map, msg_size(Msg, CalcMsgSize)};\n\nbw_map_fun({shutdown, sync_finished_remote} = Msg, _Source, _Dest, CalcMsgSize) ->\n    {recon_map, msg_size(Msg, CalcMsgSize)};\nbw_map_fun({start_recon, _SenderPid, _SId, {start_recon, _Method, _Struct}} = Msg,\n           _Source, _Dest, CalcMsgSize) ->\n    {recon_map, msg_size(Msg, CalcMsgSize)};\nbw_map_fun({continue_recon, _SenderPid, _SId, {start_recon, _Method, _Struct}} = Msg,\n           _Source, _Dest, CalcMsgSize) ->\n    {recon_map, msg_size(Msg, CalcMsgSize)};\nbw_map_fun(Msg, _Source, _Dest, CalcMsgSize)\n  when element(1, Msg) =:= reconcile_req ->\n    {recon_map, msg_size(Msg, CalcMsgSize)};\nbw_map_fun(Msg, _Source, _Dest, CalcMsgSize)\n  when element(1, Msg) =:= resolve_req ->\n    {recon_map2, msg_size(Msg, CalcMsgSize)};\nbw_map_fun(Msg, _Source, _Dest, CalcMsgSize)\n  when element(1, Msg) =:= ?check_nodes;\n       element(1, Msg) =:= ?check_nodes_response ->\n    % for debugging:\n%%     log:log(warn, \"mapped global message: ~p (~p) -- ~.2p\",\n%%             [util:extint2atom(element(1, Msg)), msg_size(Msg, true), Msg]),\n    {recon_map, msg_size(Msg, CalcMsgSize)};\n\n%% encapsulated messages:\n% look 2 levels deep as an optimisation:\nbw_map_fun({?lookup_aux, _, _, {?send_to_group_member, rrepair, MsgI}} = Msg,\n           Source, Dest, CalcMsgSize) ->\n    bw_map_fun2(MsgI, Source, Dest, Msg, CalcMsgSize);\nbw_map_fun({?lookup_fin, _, _, {?send_to_group_member, rrepair, MsgI}} = Msg,\n           Source, Dest, CalcMsgSize) ->\n    bw_map_fun2(MsgI, Source, Dest, Msg, CalcMsgSize);\n% general cases for encapsulated messages:\nbw_map_fun({?lookup_aux, _, _, MsgI} = Msg, Source, Dest, CalcMsgSize) ->\n    bw_map_fun2(MsgI, Source, Dest, Msg, CalcMsgSize);\nbw_map_fun({?lookup_fin, _, _, MsgI} = Msg, Source, Dest, CalcMsgSize) ->\n    bw_map_fun2(MsgI, Source, Dest, Msg, CalcMsgSize);\nbw_map_fun({?send_to_group_member, rrepair, MsgI} = Msg, Source, Dest, CalcMsgSize) ->\n    bw_map_fun2(MsgI, Source, Dest, Msg, CalcMsgSize);\n\nbw_map_fun(Msg, _Source, _Dest, _CalcMsgSize) ->\n    {unmapped, Msg}.\n\n-spec bw_map_fun2(MsgI::comm:message(), Source::pid() | comm:mypid(),\n                  Dest::pid() | comm:mypid(), FullMsg::comm:message(),\n                  CalcMsgSize::boolean())\n        -> {unmapped, Msg::comm:message()} |\n           {rr_map, ExtSize::pos_integer()} |\n           {recon_map | recon_map2, ExtSize::pos_integer()} |\n           {resolve_map, ExtSize::pos_integer(), KVVLen::non_neg_integer()}.\nbw_map_fun2(MsgI, Source, Dest, FullMsg, CalcMsgSize) ->\n    case bw_map_fun(MsgI, Source, Dest, false) of\n        {resolve_map, _MsgSize, KVVLen} ->\n            {resolve_map, msg_size(FullMsg, CalcMsgSize), KVVLen};\n        {MsgTag, MsgSize} when is_integer(MsgSize) ->\n            {MsgTag, msg_size(FullMsg, CalcMsgSize)};\n        X -> X\n    end.\n\n-spec get_bandwidth(trace_mpath:trace())\n        -> {Recon_size        :: non_neg_integer(),   % number of recon-msg bytes\n            Recon_msg_count   :: non_neg_integer(),   % number of recon messages\n            Recon2_size       :: non_neg_integer(),   % number of recon2-msg bytes\n            Recon2_msg_count  :: non_neg_integer(),   % number of recon2 messages\n            Resolve_size      :: non_neg_integer(),   % number of transmitted resolve bytes\n            Resolve_msg_count :: non_neg_integer(),   % number of resolve messages\n            Resolve_kvv_count :: non_neg_integer()}.  % number of kvv-triple send\nget_bandwidth(Trace) ->\n    lists:foldl(fun(X, {RCAccBytes, RCAccMsgs,\n                        RC2AccBytes, RC2AccMsgs,\n                        RSAccBytes, RSAccMsgs, RSAccKVV} = Acc) ->\n                        case element(6, X) of\n                            {recon_map, Byte} ->\n                                {RCAccBytes + Byte, RCAccMsgs + 1,\n                                 RC2AccBytes, RC2AccMsgs,\n                                 RSAccBytes, RSAccMsgs, RSAccKVV};\n                            {recon_map2, Byte} ->\n                                {RCAccBytes, RCAccMsgs,\n                                 RC2AccBytes + Byte, RC2AccMsgs + 1,\n                                 RSAccBytes, RSAccMsgs, RSAccKVV};\n                            {resolve_map, Byte, KVVCount} ->\n                                {RCAccBytes, RCAccMsgs,\n                                 RC2AccBytes, RC2AccMsgs,\n                                 RSAccBytes + Byte, RSAccMsgs + 1, RSAccKVV + KVVCount};\n                            _ -> Acc\n                        end\n                end,\n                {0, 0, 0, 0, 0, 0, 0},\n                Trace).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Eval Measurment Point Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec get_measure_point(rr_eval_point:point_id(), Iteration::non_neg_integer(),\n                        Round::non_neg_integer(), init_mp(),\n                        Trace::trace_mpath:trace(), NodeList::[comm:mypid()],\n                        SessionStats::rrepair:session())\n        -> rr_eval_point:measure_point().\nget_measure_point(Id, Iter, Round, {Miss, Outd}, Trace, NodeList, SessionStats) ->\n    {_, _, M, O} = get_db_status(NodeList),\n    {RC_S, RC_Msg, RC2_S, RC2_Msg, RS_S, RS_Msg, RS_KVV} = get_bandwidth(Trace),\n    \n    % TODO: use and verify stats for plausibility\n    RCStats = rrepair:session_get(rc_stats, SessionStats),\n    Fr_p1 = rr_recon_stats:get(fail_rate_p1, RCStats),\n    Fr_p2 = rr_recon_stats:get(fail_rate_p2, RCStats),\n    Fr = rr_recon_stats:get(fail_rate, RCStats),\n    {MerkleInner, MerkleLeaf, MerkleEmptyLeaf, _MerkleItems} = rr_recon_stats:get(tree_size, RCStats),\n%%     log:pal(\" Stats: ~p\", [rr_recon_stats:print(RCStats)]),\n    \n    {Id, Iter, Round,\n     Miss, Miss - M,\n     Outd, Outd - O,\n     RC_S, RC_Msg, RC2_S, RC2_Msg, RS_S, RS_Msg, RS_KVV, Fr_p1, Fr_p2, Fr,\n     MerkleInner, MerkleLeaf, MerkleEmptyLeaf}.\n\n-spec get_mp_round(rr_eval_point:point_id(), Iteration::non_neg_integer(),\n                   Round::non_neg_integer(), init_mp(),\n                   Trace::trace_mpath:trace(), NodeList::[comm:mypid()])\n        -> rr_eval_point:measure_point().\nget_mp_round(Id, Iter, Round, {Miss, Outd}, Trace, NodeList) ->\n    {_DBSize, ActLoad, _Missing, ActOut} = get_db_status2(NodeList),\n    {RC_S, RC_Msg, RC2_S, RC2_Msg, RS_S, RS_Msg, RS_KVV} = get_bandwidth(Trace),\n    {Id, Iter, Round,\n     ActLoad, ActLoad - Miss,\n     ActOut, Outd - ActOut,\n     RC_S, RC_Msg, RC2_S, RC2_Msg, RS_S, RS_Msg, RS_KVV, '-', '-', '-', '-', '-', '-'}.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% Local Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec pid_to_rtkey(comm:mypid()) -> ?RT:key().\npid_to_rtkey(Pid) ->\n    comm:send(Pid, {get_state, comm:this(), node_id}),\n    receive\n        ?SCALARIS_RECV({get_state_response, Key}, Key)\n    end.\n\n-spec get_node_list() -> [comm:mypid()].\nget_node_list() ->\n    mgmt_server:node_list(),\n    receive\n        ?SCALARIS_RECV({get_list_response, List}, List)\n    end.\n"
  },
  {
    "path": "test/rr_eval_export.erl",
    "content": "% @copyright 2010-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @doc    Export helper functions for replica repair evaluation.\n%% @see    rr_eval_admin\n-module(rr_eval_export).\n\n-author('malange@informatik.hu-berlin.de').\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-export([write_csv/2, write_raw/2]).\n\n-export([create_file/1, write_ds/2, write_row/2, close_file/1]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-define(TAB, 9).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% type definitions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-export_type([file_type/0, property/0, params/0, data_set/0]).\n\n-type file_type() :: gnuplot | latex_table.\n\n-type file_property() :: {filename, string()} |\n                         {dir, string()} |\n                         {comment, [string()]} |        %first file lines\n                         {column_names,    [atom()]}.\n-type gnuplot_property() :: {caption, string()} |\n                            {gp_xlabel, string()} |\n                            {gp_ylabel, string()} |\n                            {set_yrange, string()} |\n                            {set_xrange, string()} |\n                            {set_logscale, x | y | both} |\n                            {gnuplot_args, string()}. %starting with using after plot \"filename\" ..\n\n-type property() :: file_property() |\n                    gnuplot_property().\n-type params() :: [property()].\n\n-type data_set_property() ::\n          {comment, [string()]}.\n\n-type data_set() :: {\n                     Properties ::  [data_set_property()],\n                     Table      ::  [[any()]]\n                    }.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% sequentical data file ceation\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec create_file([file_property()]) -> {IoDevice::file:io_device(), FilePath::string()}.\ncreate_file(Options) ->\n    FileName = proplists:get_value(filename, Options, ?MODULE),\n    CommentList = proplists:get_value(comment, Options, []),\n    Columns = proplists:get_value(column_names, Options, []),\n    Dir = case proplists:get_value(dir, Options, null) of\n              null -> throw({create_file_error, \"missing dest directory - parameter 'dir'\"});\n              S -> S\n          end,\n    _DirOk = filelib:ensure_dir(Dir ++ \"/\"),\n    FilePath = filename:join([Dir, FileName]),\n    \n    _ = case file:open(FilePath, [write]) of\n            {ok, IoDevice} ->\n                _ = [io:fwrite(IoDevice, \"# ~s~n\", [CL]) || CL <- CommentList],\n                if Columns =/= [] ->\n                       io:fwrite(IoDevice, \"# Columns: \", []),\n                       {_, ColStr} =\n                           lists:foldl(\n                             fun(Col, {I, Str}) ->\n                                     {I + 1, io_lib:format(\"~s~c~B_~s\", [Str, ?TAB, I, Col])}\n                             end, {1, \"\"}, Columns),\n                       io:fwrite(IoDevice, \"~s~n\", [ColStr]);\n                   true -> ok\n                end,\n                {IoDevice, FilePath};\n            {error, _Reason} ->\n                ErrMsg = io_lib:format(\"IO_ERROR - ~p~nFile=~s~nDirOk=~p\",\n                                       [_Reason, FilePath, _DirOk]),\n                throw({io_error, ErrMsg})\n        end.\n\n-spec write_ds(IoDevice::file:io_device(), data_set()) -> ok.\nwrite_ds(IoDevice, {DSProp, Table}) ->\n    Comments = proplists:get_value(comment, DSProp, []),\n    _ = [io:fwrite(IoDevice, \"# ~s~n\", [CL]) || CL <- Comments],\n    _ = [write_row(IoDevice, Row) || Row <- Table],\n    ok.\n\n-spec write_row(IoDevice::file:io_device(), Row::[any()]) -> ok.\nwrite_row(IoDevice, Row) ->\n    _ = [io:fwrite(IoDevice, \"~p~c\", [Field, ?TAB]) || Field <- Row],\n    io:fwrite(IoDevice, \"~n\", []).\n\n-spec close_file(IoDevice::file:io_device()) -> ok.\nclose_file(IoDevice) ->\n    io:fwrite(IoDevice, \"~n~n\", []),\n    _ = file:close(IoDevice),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec write_csv(DataSets::[data_set()], Options::[file_property()]) -> ok | io_error.\nwrite_csv(DataSets, Options) ->\n    {IoDevice, _FilePath} = create_file(Options),\n    lists:foreach(fun(DS) -> write_ds(IoDevice, DS) end, DataSets),\n    close_file(IoDevice).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec write_raw([any()], [file_property()]) -> ok | io_error.\nwrite_raw(DataSet, Options) ->\n    FileName = proplists:get_value(filename, Options, ?MODULE),\n    CommentList = proplists:get_value(comment, Options, []),\n    Dir = case proplists:get_value(dir, Options, null) of\n              null -> throw({create_file_error, \"missing dest directory - parameter 'dir'\"});\n              S -> S\n          end,\n    _ = filelib:ensure_dir(Dir ++ \"/\"),\n    FilePath =  filename:join([Dir, FileName]),\n\n    case file:open(FilePath, [write]) of\n        {ok, Fileid} ->\n            _ = [io:fwrite(Fileid, \"# ~s~n\", [CL]) || CL <- CommentList],\n            _ = [io:fwrite(Fileid, \"~p~n\", [Line]) || Line <- DataSet],\n            _ = file:truncate(Fileid),\n            _ = file:close(Fileid),\n            ok;\n        {_, _} ->\n            io_error\n    end.\n"
  },
  {
    "path": "test/rr_eval_point.erl",
    "content": "% @copyright 2012-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <lakedaimon300@googlemail.com>\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Helper functions for replica repair evaluation, defining evaluation\n%%         data.\n%% @see    rr_eval_admin\n%% @version $Id:  $\n-module(rr_eval_point).\n\n-include(\"rr_records.hrl\").\n\n-export([generate_ep/3, generate_ep_rounds/4, column_names/0, mp_column_names/0]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% TYPES\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-export_type([measure_point/0, eval_point/0, point_id/0]).\n\n-type point_id()      :: non_neg_integer().\n-type measure_point() :: {\n                          ID          :: point_id(),  %test-set unique id\n                          Iteration   :: non_neg_integer(), %iteration count\n                          Round       :: non_neg_integer(), %round\n                          Missing     :: integer(),   % missing replicas\n                          Regen       :: integer(),   % regenerated\n                          Outdated    :: integer(),   % outdated replaces\n                          Updated     :: integer(),   % updated replicas\n                          BW_RC_Size  :: integer(),   % number of recon-msg bytes\n                          BW_RC_Msg   :: integer(),   % number of recon messages\n                          BW_RC2_Size :: integer(),   % number of recon2-msg bytes\n                          BW_RC2_Msg  :: integer(),   % number of recon2 messages\n                          BW_RS_Size  :: integer(),   % number of transmitted resolve bytes\n                          BW_RS_Msg   :: integer(),   % number of resolve messages\n                          BW_RS_KVV   :: integer(),   % number of kvv-triple send\n                          Fr_p1       :: float() | '-', % effective worst-case probability of phase 1\n                          Fr_p2       :: float() | '-', % effective worst-case probability of phase 2\n                          Fr          :: float() | '-', % effective total worst-case probability\n                          MerkleInner :: non_neg_integer() | '-', % number of inner nodes on the initiator in case of Merkle sync\n                          MerkleLeaf  :: non_neg_integer() | '-', % number of total leaf nodes on the initiator in case of Merkle sync\n                          MerkleEmptyLeaf :: non_neg_integer() | '-'  % number of empty leaf nodes on the initiator in case of Merkle sync\n                          }.\n\n-type eval_point() :: {\n                       ID               :: point_id(),  %test-set unique id\n                       %PARAMETER\n                       NodeCount        :: integer(),\n                       DataCount        :: integer(),\n                       FRate            :: 0..100,\n                       Round            :: integer(),\n                       ReconFR          :: float(),\n                       Merkle_Bucket    :: pos_integer(),\n                       Merkle_Branch    :: pos_integer(),\n                       Art_Corr_Fac     :: non_neg_integer(),\n                       Art_Leaf_Fpr     :: float(),\n                       Art_Inner_Fpr    :: float(),\n                       % AVG MEASUREMENT POINT\n                       Missing          :: float(),   % missing replicas\n                       Regen            :: float(),   % regenerated\n                       Outdated         :: float(),   % outdated replaces\n                       Updated          :: float(),   % updated replicas\n                       BW_RC_Size       :: float(),   % number of recon-msg bytes\n                       BW_RC_Msg        :: float(),   % number of recon messages\n                       BW_RC2_Size      :: float(),   % number of recon2-msg bytes\n                       BW_RC2_Msg       :: float(),   % number of recon2 messages\n                       BW_RS_Size       :: float(),   % number of transmitted resolve bytes\n                       BW_RS_Msg        :: float(),   % number of resolve messages\n                       BW_RS_KVV        :: float(),   % number of kvv-triple send\n                       % STD. DEVIATION OF MEASUREMENT\n                       SD_Missing       :: float(),\n                       SD_Regen         :: float(),\n                       SD_Outdated      :: float(),\n                       SD_Updated       :: float(),\n                       SD_BW_RC_Size    :: float(),\n                       SD_BW_RC_Msg     :: float(),\n                       SD_BW_RC2_Size   :: float(),\n                       SD_BW_RC2_Msg    :: float(),\n                       SD_BW_RS_Size    :: float(),\n                       SD_BW_RS_Msg     :: float(),\n                       SD_BW_RS_KVV     :: float(),\n                       % MIN/MAX OF AVG\n                       MIN_Missing      :: integer(),\n                       MAX_Missing      :: integer(),\n                       MIN_Regen        :: integer(),\n                       MAX_Regen        :: integer(),\n                       MIN_Outdated     :: integer(),\n                       MAX_Outdated     :: integer(),\n                       MIN_Updated      :: integer(),\n                       MAX_Updated      :: integer(),\n                       MIN_BW_RC_Size   :: integer(),\n                       MAX_BW_RC_Size   :: integer(),\n                       MIN_BW_RC_Msg    :: integer(),\n                       MAX_BW_RC_Msg    :: integer(),\n                       MIN_BW_RC2_Size  :: integer(),\n                       MAX_BW_RC2_Size  :: integer(),\n                       MIN_BW_RC2_Msg   :: integer(),\n                       MAX_BW_RC2_Msg   :: integer(),\n                       MIN_BW_RS_Size   :: integer(),\n                       MAX_BW_RS_Size   :: integer(),\n                       MIN_BW_RS_Msg    :: integer(),\n                       MAX_BW_RS_Msg    :: integer(),\n                       MIN_BW_RS_KVV    :: integer(),\n                       MAX_BW_RS_KVV    :: integer(),\n                       % ADDITIONAL PARAMETERS\n                       RC_METHOD        :: rr_recon:method(),\n                       RING_TYPE        :: ring_type(),\n                       DDIST            :: data_distribution(),\n                       FTYPE            :: db_generator:failure_type(),\n                       FDIST            :: fail_distribution(),\n                       DTYPE            :: db_generator:db_type(),\n                       TPROB            :: 0..100,\n                       Merkle_NumTrees  :: pos_integer(),\n                       % AVG, STD, MIN, MAX effective failure rate of phase 1\n                       MeanFr_p1        :: float() | '-',\n                       ErrFr_p1         :: float() | '-',\n                       MinFr_p1         :: float() | '-',\n                       MaxFr_p1         :: float() | '-',\n                       % AVG, STD, MIN, MAX effective failure rate  of phase 2\n                       MeanFr_p2        :: float() | '-',\n                       ErrFr_p2         :: float() | '-',\n                       MinFr_p2         :: float() | '-',\n                       MaxFr_p2         :: float() | '-',\n                       % AVG, STD, MIN, MAX total effective failure rate\n                       MeanFr           :: float() | '-',\n                       ErrFr            :: float() | '-',\n                       MinFr            :: float() | '-',\n                       MaxFr            :: float() | '-',\n                       % misc\n                       ExpectedDelta    :: number(),\n                       % AVG, STD, MIN, MAX number of merkle tree inner nodes\n                       MeanMerkleInner  :: float() | '-',\n                       ErrMerkleInner   :: float() | '-',\n                       MinMerkleInner   :: float() | '-',\n                       MaxMerkleInner   :: float() | '-',\n                       % AVG, STD, MIN, MAX number of total merkle tree leaf nodes\n                       MeanMerkleLeaf   :: float() | '-',\n                       ErrMerkleLeaf    :: float() | '-',\n                       MinMerkleLeaf    :: float() | '-',\n                       MaxMerkleLeaf    :: float() | '-',\n                       % AVG, STD, MIN, MAX number of empty merkle tree leaf nodes\n                       MeanMerkleEmptyLeaf :: float() | '-',\n                       ErrMerkleEmptyLeaf  :: float() | '-',\n                       MinMerkleEmptyLeaf  :: float() | '-',\n                       MaxMerkleEmptyLeaf  :: float() | '-'\n                      }.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc list of eval_point field names\n-spec column_names() -> [atom()].\ncolumn_names() ->\n    [id,\n     %Parameter\n     nodes, dbsize, fprob, round,\n     recon_fail_rate, merkle_bucket, merkle_branch, art_corr_factor, art_leaf_fpr,\n     art_inner_fpr,\n     % Avg Measure\n     missing, regen, outdated, updated,\n     bw_rc_size, bw_rc_msg, bw_rc2_size, bw_rc2_msg,\n     bw_rs_size, bw_rs_msg, bw_rs_kvv,\n     % Std Deviation\n     sd_missing, sd_regen, sd_outdated, sd_updated,\n     sd_bw_rc_size, sd_bw_rc_msg, sd_bw_rc2_size, sd_bw_rc2_msg,\n     sd_bw_rs_size, sd_bw_rs_msg, sd_bw_rs_kvv,\n     % Min/Max\n     min_missing, max_missing, min_regen, max_regen,\n     min_outdated, max_outdated, min_updated, max_updated,\n     min_bw_rc_size, max_bw_rc_size, min_rc_msg, max_rc_msg,\n     min_bw_rc2_size, max_bw_rc2_size, min_rc2_msg, max_rc2_msg,\n     min_bw_rs_size, max_bw_rs_size, min_bw_rs_msg, max_bw_rs_msg,\n     min_bw_rs_kvv, max_bw_rs_kvv,\n     % additional parameters originally missing\n     rc_method, ring_type, ddist, ftype, fdist, dtype, tprob, merkle_num_trees,\n     % AVG, STD, MIN, MAX effective failure rate of phase 1 and 2\n     fr_p1, sd_fr_p1, min_fr_p1, max_fr_p1,\n     fr_p2, sd_fr_p2, min_fr_p2, max_fr_p2,\n     fr, sd_fr, min_fr, max_fr,\n     expected_delta,\n     merkle_inner, sd_merkle_inner, min_merkle_inner, max_merkle_inner,\n     merkle_leaf, sd_merkle_leaf, min_merkle_leaf, max_merkle_leaf,\n     merkle_empty_leaf, sd_merkle_empty_leaf, min_merkle_empty_leaf, max_merkle_empty_leaf\n    ].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc list of measurement_point field names\n-spec mp_column_names() -> [atom()].\nmp_column_names() ->\n    [id, iteration, round, missing, regen, outdated, updated,\n     bw_rc_size, bw_rc_msg, bw_rc2_size, bw_rc2_msg,\n     bw_rs_size, bw_rs_msg, bw_rs_kvv, fr_p1, fr_p2, fr,\n     merkle_inner, merkle_leaf, merkle_empty_leaf].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec generate_ep_rounds(point_id(), ring_setup(), [measure_point()],\n                         Rounds::non_neg_integer()) -> [eval_point()].\ngenerate_ep_rounds(ID, Conf, MPList, Rounds) ->\n    lists:foldl(fun(Round, Acc) ->\n                        RData = [X || X <- MPList, element(3, X) =:= Round],\n                        EP = generate_ep(ID + Round - 1, Conf, RData),\n                        [EP | Acc]\n                end,\n                [],\n                lists:seq(1, Rounds)).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec generate_ep(point_id(), ring_setup(), [measure_point()])\n        -> eval_point().\ngenerate_ep(ID,\n            {#scenario{ring_type = RingType, data_distribution = DDist,\n                       data_failure_type = FType, fail_distribution = FDist,\n                       data_type = DType, trigger_prob = TProb},\n             #ring_config{node_count = NC, data_count = DC, data_failure_rate = FRate},\n             #rc_config{recon_method = RCMethod, recon_fail_rate = RcFR,\n                        expected_delta = ExpDelta,\n                        merkle_bucket= MBU, merkle_branch = MBR,\n                        merkle_num_trees = MNT, art_corr_factor = ArtCF,\n                        art_leaf_fpr = ArtLF, art_inner_fpr = ArtIF}},\n            MP) ->\n    %% MEAN, STDDEV, MIN, MAX %%\n    %DB stats\n    {MeanM, ErrM, MinM, MaxM}       = mean_w_error(4, MP),\n    {MeanR, ErrR, MinR, MaxR}       = mean_w_error(5, MP),\n    {MeanO, ErrO, MinO, MaxO}       = mean_w_error(6, MP),\n    {MeanU, ErrU, MinU, MaxU}       = mean_w_error(7, MP),\n    %RC, RC2, RS\n    {MeanRCS, ErrRCS, MinRCS, MaxRCS}   = mean_w_error(8, MP),\n    {MeanRCM, ErrRCM, MinRCM, MaxRCM}   = mean_w_error(9, MP),\n    {MeanRC2S, ErrRC2S, MinRC2S, MaxRC2S} = mean_w_error(10, MP),\n    {MeanRC2M, ErrRC2M, MinRC2M, MaxRC2M} = mean_w_error(11, MP),\n    {MeanRSS, ErrRSS, MinRSS, MaxRSS}   = mean_w_error(12, MP),\n    {MeanRSM, ErrRSM, MinRSM, MaxRSM}   = mean_w_error(13, MP),\n    {MeanRSK, ErrRSK, MinRSK, MaxRSK}   = mean_w_error(14, MP),\n    \n    {MeanFr_p1, ErrFr_p1, MinFr_p1, MaxFr_p1} =\n        try mean_w_error(15, MP)\n        catch _:_ -> {'-', '-', '-', '-'}\n        end,\n    {MeanFr_p2, ErrFr_p2, MinFr_p2, MaxFr_p2} =\n        try mean_w_error(16, MP)\n        catch _:_ -> {'-', '-', '-', '-'}\n        end,\n    {MeanFr, ErrFr, MinFr, MaxFr} =\n        try mean_w_error(17, MP)\n        catch _:_ -> {'-', '-', '-', '-'}\n        end,\n\n    {MeanMerkleI, ErrMerkleI, MinMerkleI, MaxMerkleI} =\n        try mean_w_error(18, MP)\n        catch _:_ -> {'-', '-', '-', '-'}\n        end,\n    {MeanMerkleL, ErrMerkleL, MinMerkleL, MaxMerkleL} =\n        try mean_w_error(19, MP)\n        catch _:_ -> {'-', '-', '-', '-'}\n        end,\n    {MeanMerkleEmptyL, ErrMerkleEmptyL, MinMerkleEmptyL, MaxMerkleEmptyL} =\n        try mean_w_error(20, MP)\n        catch _:_ -> {'-', '-', '-', '-'}\n        end,\n\n    {ID,\n     NC, 4 * DC, FRate, element(3, hd(MP)),\n     RcFR, MBU, MBR, ArtCF, ArtLF, ArtIF,\n     MeanM, MeanR, MeanO, MeanU,\n     MeanRCS, MeanRCM, MeanRC2S, MeanRC2M, MeanRSS, MeanRSM, MeanRSK,\n     ErrM, ErrR, ErrO, ErrU,\n     ErrRCS, ErrRCM, ErrRC2S, ErrRC2M, ErrRSS, ErrRSM, ErrRSK,\n     MinM, MaxM, MinR, MaxR, MinO, MaxO, MinU, MaxU,\n     MinRCS, MaxRCS, MinRCM, MaxRCM, MinRC2S, MaxRC2S, MinRC2M, MaxRC2M,\n     MinRSS, MaxRSS, MinRSM, MaxRSM, MinRSK, MaxRSK,\n     RCMethod, RingType, dist_to_name(DDist), FType, dist_to_name(FDist),\n     DType, TProb, MNT,\n     MeanFr_p1, ErrFr_p1, MinFr_p1, MaxFr_p1,\n     MeanFr_p2, ErrFr_p2, MinFr_p2, MaxFr_p2,\n     MeanFr, ErrFr, MinFr, MaxFr,\n     ExpDelta,\n     MeanMerkleI, ErrMerkleI, MinMerkleI, MaxMerkleI,\n     MeanMerkleL, ErrMerkleL, MinMerkleL, MaxMerkleL,\n     MeanMerkleEmptyL, ErrMerkleEmptyL, MinMerkleEmptyL, MaxMerkleEmptyL}.\n\n-spec dist_to_name(Dist::random | uniform | {binomial, P::float()}) -> atom().\ndist_to_name({binomial, P}) ->\n    DistName = lists:flatten(io_lib:format(\"binomial_~f\", [P])),\n    try erlang:list_to_existing_atom(DistName)\n    catch error:badarg -> erlang:list_to_atom(DistName)\n    end;\ndist_to_name(Dist) ->\n    Dist.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec mean_w_error(integer(), [tuple()])\n        -> {Mean::float(), StdError::float(), Min::X, Max::X}\n        when is_subtype(X, number()).\nmean_w_error(_ElementPos, []) ->\n    {0.0, 0.0, 0, 0};\nmean_w_error(ElementPos, [H | TL]) ->\n    HE = element(ElementPos, H),\n    % Note: calculate the standard deviation via shifted values to be more\n    %       accurate with floating point values\n    %       -> https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Computing_shifted_data\n    {Len, Sum, SumStd1, SumStd2, Min, Max} =\n        lists:foldl(\n          fun(T, {L, SummAcc, SumStd1Acc, SumStd2Acc, Min, Max}) ->\n                  E = element(ElementPos, T),\n                  E2 = E - HE,\n                  {L + 1, SummAcc + E, SumStd1Acc + E2, SumStd2Acc + E2 * E2,\n                   erlang:min(E, Min), erlang:max(E, Max)}\n          end, {1, HE, 0, 0, HE, HE}, TL),\n    % pay attention to possible loss of precision here:\n    {Sum / Len, math:sqrt((Len * SumStd2 - SumStd1 * SumStd1) / (Len * Len)), Min, Max}.\n"
  },
  {
    "path": "test/rr_recon_performance_SUITE.erl",
    "content": "%  @copyright 2010-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de\n%% @doc    Performance of rrepair modules\n%% @end\n%% @version $Id$\n-module(rr_recon_performance_SUITE).\n-author('malange@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\n-include(\"record_helpers.hrl\").\n\n-export([]).\n\n-type result_pair() :: {build_time,     measure_util:result()} |\n                       {tree_time,      measure_util:result()} |\n                       {hash_fun_count, pos_integer()}.\n-type result() :: [result_pair()].\n\nall() ->\n    [art,\n     merkle_tree,\n     bloom,\n     bloom2,\n     comparison\n    ].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 200}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ncomparison(_) ->\n    Iter = 100,\n    DBSize = 10000,\n    \n    I = intervals:new('[', rt_SUITE:number_to_key(1), rt_SUITE:number_to_key(100000000), ']'),\n    DB = db_generator:get_db(I, DBSize, uniform, [{output, list_keytpl}]),\n    \n    BBuildT = bloom_build_time(I, DB, DBSize, Iter, 0.1),\n    MBuildT = merkle_build_time(I, DB, DBSize, Iter, []),\n    ArtT = art_build_time(I, DB, DBSize, Iter, [], []),\n        \n    ABuildT = proplists:get_value(build_time, ArtT),\n    TreeT = proplists:get_value(tree_time, ArtT),\n    \n    ct:pal(\"Performance Comparison\n            DBSize=~p ; Iterations=~p\n            -----------------------------------------------\n                                bloom |merkleTree|  art\n            BuildTime Avg ms: ~f | ~f | ~f (~f without Merkle)\",\n           [DBSize, Iter,\n            measure_util:get(BBuildT, avg, ms),\n            measure_util:get(MBuildT, avg, ms),\n            measure_util:get(ABuildT, avg, ms),\n            measure_util:get(TreeT, avg, ms)]),\n    ok.\n\n-spec bloom_build_time(intervals:interval(), [any()], pos_integer(), pos_integer(), float()) -> measure_util:result().\nbloom_build_time(_, DB, DBSize, Iterations, Fpr) ->\n    BaseBF = bloom:new_fpr(DBSize, Fpr),\n    measure_util:time_avg(\n      fun() ->\n              lists:foldl(fun(I, Bloom) -> bloom:add(Bloom, I) end, BaseBF, DB)\n      end, Iterations, []).\n\n-spec merkle_build_time(intervals:interval(), [any()], pos_integer(), pos_integer(),\n            merkle_tree:mt_config_params()) -> measure_util:result().\nmerkle_build_time(I, DB, _DBSize, Iterations, Config) ->\n    measure_util:time_avg(fun() -> merkle_tree:new(I, DB, Config) end,\n                          Iterations, []).\n\n-spec art_build_time(intervals:interval(), [any()], pos_integer(), pos_integer(),\n            merkle_tree:mt_config_params(), art:config()) -> result().\nart_build_time(I, DB, _DBSize, Iterations, MerkleConfig, ArtConfig) ->\n    {Tree, TreeTime} =\n        measure_util:time_with_result(fun() -> merkle_tree:new(I, DB, MerkleConfig) end, Iterations, []),\n    BuildTime =\n        measure_util:time_avg(fun() -> art:new(Tree, ArtConfig) end, Iterations, []),\n    \n    [{build_time, measure_util:add(BuildTime, TreeTime)},\n     {tree_time, TreeTime}].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nart(_) ->\n    ToAdd = 10000,\n    ExecTimes = 100,\n    \n    I = intervals:new('[', rt_SUITE:number_to_key(1), rt_SUITE:number_to_key(100000000), ']'),\n    DB = db_generator:get_db(I, ToAdd, uniform, [{output, list_keytpl}]),\n\n    {TreeTime, Tree} =\n        util:tc(fun() -> merkle_tree:new(I, DB, [{leaf_hf, fun art:merkle_leaf_hf/2}]) end, []),\n    BuildTime =\n        measure_util:time_avg(fun() -> art:new(Tree) end, ExecTimes, []),\n\n    ct:pal(\"ART Performance\n            ------------------------\n            PARAMETER: AddedItems=~p ; ExecTimes=~p\n            TreeTime (ms)= ~p\n            ARTBuildTime (ms)= ~p\",\n           [ToAdd, ExecTimes, TreeTime / 1000,\n            measure_util:print(BuildTime, ms)]),\n    true.\n\n% @doc measures performance of merkle_tree operations\nmerkle_tree(_) ->\n    % PARAMETER\n    ExecTimes = 100,\n    ToAdd = 10000,\n    \n    I = intervals:new('[', rt_SUITE:number_to_key(1), rt_SUITE:number_to_key(100000000), ']'),\n    DB = db_generator:get_db(I, ToAdd, uniform, [{output, list_keytpl}]),\n    \n    TestTree = merkle_tree:new(I, DB, []),\n    {Inner, Leafs, _EmptyLeafs, _Items} = merkle_tree:size_detail(TestTree),\n    \n    BuildT = measure_util:time_avg(\n           fun() -> merkle_tree:new(I, DB, []) end,\n           ExecTimes, []),\n        \n    IterateT = measure_util:time_avg(\n           fun() -> merkle_tree_SUITE:iterate(merkle_tree:iterator(TestTree), fun(_, Acc) -> Acc + 1 end, 0) end,\n           ExecTimes, []),\n    \n    GenHashT = measure_util:time_avg(\n            fun() -> merkle_tree:gen_hash(TestTree) end,\n            ExecTimes, []),\n    \n    SimpleSizeT = measure_util:time_avg(\n            fun() -> merkle_tree:size(TestTree) end,\n            ExecTimes, []),\n    \n    DetailSizeT = measure_util:time_avg(\n            fun() -> merkle_tree:size_detail(TestTree) end,\n            ExecTimes, []),\n\n    ct:pal(\"\n            Merkle_Tree Performance\n            ------------------------\n            PARAMETER: AddedItems=~p ; ExecTimes=~p\n            TreeSize: InnerNodes=~p ; Leafs=~p,\n            BuildTime:      ~p\n            IterationTime : ~p\n            GenHashTime:    ~p\n            SimpleSizeTime: ~p\n            DetailSizeTime: ~p\",\n           [ToAdd, ExecTimes, Inner, Leafs,\n            measure_util:print(BuildT, ms),\n            measure_util:print(IterateT),\n            measure_util:print(GenHashT),\n            measure_util:print(SimpleSizeT),\n            measure_util:print(DetailSizeT)]),\n    ok.\n\nbloom(_) ->\n    %parameter\n    ExecTimes = 100,\n    ToAdd = 10000, % req: mod 2 = 0\n    Fpr = 0.1,\n\n    BaseBF = bloom:new_fpr(ToAdd, Fpr),\n    List = random_list(ToAdd),\n    TestBF = bloom:add_list(BaseBF, List),\n\n    AddT = measure_util:time_avg(fun() -> for_to_ex(1, ToAdd,\n                                                    fun(I) -> I end,\n                                                    fun(I, B) -> bloom:add(B, I) end,\n                                                    BaseBF)\n                                 end,\n                                 ExecTimes, []),\n    \n    AddListT = measure_util:time_avg(fun() -> bloom:add_list(BaseBF, List) end,\n                                     ExecTimes, []),\n\n    IsElemT = measure_util:time_avg(\n                fun() ->\n                        lists:foreach(fun(I) -> bloom:is_element(TestBF, I) end, List)\n                end,\n                ExecTimes, []),\n    \n    %measure join time\n    BF1 = for_to_ex(1,\n                    round(ToAdd / 2),\n                    fun(I) -> I end, fun(I, B) -> bloom:add(B, I) end, BaseBF),\n    BF2 = for_to_ex(round(ToAdd / 2) + 1,\n                    ToAdd,\n                    fun(I) -> I end, fun(I, B) -> bloom:add(B, I) end, BaseBF),\n    JoinTime =\n        measure_util:time_avg(fun() -> bloom:join(BF1, BF2) end, ExecTimes, []),\n\n    %print results\n    ct:pal(\"AVG EXECUTION TIMES OF BLOOM FILTER OPERATIONS\n            PARAMETER: AddedItems=~p ; ExecTimes=~p\n            --------------------------------------------------------------\n            Add       : ~p\n            AddList   : ~p\n            IsElement : ~p\n            Join      : ~p\",\n           [ToAdd, ExecTimes,\n            measure_util:print(AddT),\n            measure_util:print(AddListT),\n            measure_util:print(IsElemT),\n            measure_util:print(JoinTime)]),\n    ok.\n\nbloom2(_) ->\n    %parameter\n    ExecTimes = 50,\n    ToAdd = 1024*8, % req: mod 2 = 0\n    Fpr = 0.1,\n\n    BaseBF = bloom:new_fpr(ToAdd, Fpr),\n    BFSize = bloom:get_property(BaseBF, size),\n    \n    BloomAddListFunAdd =\n        fun(ChunkSize) ->\n                for_to_ex(\n                  1, ToAdd div ChunkSize,\n                  fun(I) -> I end,\n                  fun(_I, B) ->\n                          bloom:add_list(B, random_list(ChunkSize))\n                  end,\n                  BaseBF)\n        end,\n    BloomAddListFunNew =\n        fun(ChunkSize) ->\n                for_to_ex(\n                  1, ToAdd div ChunkSize,\n                  fun(I) -> I end,\n                  fun(_I, _B) ->\n                          bloom:add_list(BaseBF, random_list(ChunkSize))\n                  end,\n                  ok)\n        end,\n    \n    AddTimesAdd =\n        [begin\n             AvgV = measure_util:get(\n                      measure_util:time_avg(\n                        fun() -> BloomAddListFunAdd(I) end, ExecTimes, []), avg),\n             {I, AvgV}\n         end || I <- [8, 16, 4096]],\n    AddTimesNew =\n        [begin\n             AvgV = measure_util:get(\n                      measure_util:time_avg(\n                        fun() -> BloomAddListFunNew(I) end, ExecTimes, []), avg),\n             {I, AvgV}\n         end || I <- [8, 16, 4096]],\n\n    %print results\n    ct:pal(\"AVG EXECUTION TIMES OF BLOOM FILTER OPERATIONS\n            PARAMETER: AddedItems=~p ; ExecTimes=~p\n                       BFSize=~p ; #Hfs=~p\n            --------------------------------------------------------------\" ++ \"\n            using bloom:add_list/2 consecutively:\" ++\n               lists:append(lists:duplicate(length(AddTimesAdd), \"\n            Res: ~.2p\")) ++ \"\n            --------------------------------------------------------------\" ++ \"\n            using bloom:add_list/2 on an empty bloom filter:\" ++\n               lists:append(lists:duplicate(length(AddTimesNew), \"\n            Res: ~.2p\")),\n           [ToAdd, ExecTimes, BFSize, bloom:get_property(BaseBF, hfs_size)]\n               ++ AddTimesAdd ++ AddTimesNew),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% UTILS\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec random_list(Count::pos_integer()) -> [pos_integer(),...].\nrandom_list(Count) ->\n    util:for_to_ex(1, Count, fun(_) -> randoms:getRandomInt() end).\n\nfor_to_ex(I, N, Fun, AccuFun, Accu) ->\n    NewAccu = AccuFun(Fun(I), Accu),\n    if\n        I < N ->\n            for_to_ex(I + 1, N, Fun, AccuFun, NewAccu);\n        I =:= N ->\n            NewAccu;\n        I > N ->\n            failed\n    end.\n"
  },
  {
    "path": "test/rr_records.hrl",
    "content": "%  @copyright 2015-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Record definitions and type defs for the replica repair evaluation\n%%         modules.\n\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-type ring_type() :: uniform | random.\n-type fail_distribution() :: random | uniform | {binomial, P::float()}.\n-type data_distribution() :: random | uniform | {binomial, P::float()}.\n-record(scenario,\n        {\n         ring_type                  :: ring_type(),\n         data_distribution          :: data_distribution() | undefined, % undefined only temporary!\n         data_failure_type          :: db_generator:failure_type() | undefined, % undefined only temporary!\n         fail_distribution          :: fail_distribution() | undefined, % undefined only temporary!\n         data_type                  :: db_generator:db_type(),\n         trigger_prob       = 100   :: 0..100\n        }).\n-type scenario() :: #scenario{}.\n\n-type step_param() :: node_count | data_count | fprob | rounds |\n                      recon_fail_rate | expected_delta |\n                      merkle_bucket | merkle_branch | merkle_num_trees |\n                      art_corr_factor | art_leaf_fpr | art_inner_fpr.\n-type step_size() :: pos_integer() | float().\n-type step_inc() :: step_size() | {power, Base::pos_integer()}.\n-type fail_rate() :: float() | pos_integer().\n\n-record(ring_config, {\n                      node_count        = ?required(rc_config, node_count) :: integer(),\n                      data_count        = ?required(rc_config, data_count) :: integer(),\n                      data_failure_rate = ?required(rc_config, fprob)      :: 0..100,       % probability of data failures\n                      fquadrants        = all                              :: all | [rt_beh:segment()],\n                      round             = 1                                :: integer()\n                     }).\n-type ring_config() :: #ring_config{}.\n\n-record(rc_config, {\n                    recon_method    = ?required(rc_config, recon_method) :: rr_recon:method(),\n                    recon_fail_rate = 0.1                                :: fail_rate(),\n                    expected_delta  = 100                                :: number() | as_fprob,\n                    merkle_bucket   = 25                                 :: pos_integer(), %shared with art\n                    merkle_branch   = 4                                  :: pos_integer(), %shared with art\n                    merkle_num_trees= 1                                  :: pos_integer(),\n                    art_corr_factor = 2                                  :: non_neg_integer(),\n                    art_leaf_fpr    = 0.1                                :: float(),\n                    art_inner_fpr   = 0.01                               :: float()\n                   }).\n-type rc_config() :: #rc_config{}.\n\n-type ring_setup() :: {scenario(), ring_config(), rc_config()}.\n"
  },
  {
    "path": "test/rrd_SUITE.erl",
    "content": "%% @copyright 2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @version $Id$\n-module(rrd_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"types.hrl\").\n-include(\"unittest.hrl\").\n\n-compile(export_all).\n\nall()   -> [simple_create,\n            fill_test,\n            create_gauge,\n            create_counter,\n            create_event,\n            create_timing,\n            add_nonexisting_timeslots,\n            reduce_timeslots,\n            {group, tester_tests}\n           ].\n\ngroups() ->\n    [{tester_tests, [], \n      [\n       tester_empty_rrd,\n       tester_counter_rrd,\n       tester_gauge_rrd]\n     }\n    ].\n\nsuite() -> [ {timetrap, {seconds, 40}} ].\n\ninit_per_suite(Config) ->\n    unittest_helper:start_minimal_procs(Config, [{rrd_timing_hist_size, 0}], false).\n\nend_per_suite(Config) ->\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\ninit_per_group(Group, Config) ->\n    unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) ->\n    unittest_helper:end_per_group(Group, Config).\n\nsimple_create(_Config) ->\n    Adds = [{20, 5}, {25, 6}],\n    DB0 = rrd:create(10, 10, gauge, {0,0,0}),\n    DB1 = lists:foldl(fun ?MODULE:apply/2, DB0, Adds),\n    ?equals(rrd:dump(DB1), [{{0,0,20}, {0,0,30}, 6}]),\n    ok.\n\nfill_test(_Config) ->\n    Adds = [{20, 1}, {30, 2}, {40, 3}, {60, 5}],\n    DB0 = rrd:create(10, 3, gauge, {0,0,0}),\n    DB1 = lists:foldl(fun ?MODULE:apply/2, DB0, Adds),\n    ?equals(rrd:dump(DB1), [{{0,0,60}, {0,0,70}, 5}, {{0,0,40}, {0,0,50}, 3}]),\n    ok.\n\ncreate_gauge(_Config) ->\n    Adds = [{20, 5}, {25, 6}, {30, 1}, {42, 2}],\n    DB0 = rrd:create(10, 10, gauge, {0,0,0}),\n    DB1 = lists:foldl(fun ?MODULE:apply/2, DB0, Adds),\n    ?equals(rrd:dump(DB1), [{{0,0,40}, {0,0,50}, 2}, {{0,0,30}, {0,0,40}, 1}, {{0,0,20}, {0,0,30}, 6}]),\n    ok.\n\ncreate_counter(_Config) ->\n    Adds = [{20, 5}, {25, 6}, {30, 1}, {42, 2}],\n    DB0 = rrd:create(10, 10, counter, {0,0,0}),\n    DB1 = lists:foldl(fun ?MODULE:apply/2, DB0, Adds),\n    ?equals(rrd:dump(DB1), [{{0,0,40}, {0,0,50}, 2}, {{0,0,30}, {0,0,40}, 1}, {{0,0,20}, {0,0,30}, 11}]),\n    ok.\n\ncreate_event(_Config) ->\n    Adds = [{20, \"20\"}, {25, \"25\"}, {30, \"30\"}, {42, \"42\"}],\n    DB0 = rrd:create(10, 10, event, {0,0,0}),\n    DB1 = lists:foldl(fun ?MODULE:apply/2, DB0, Adds),\n    ?equals(rrd:dump(DB1), [{{0,0,40}, {0,0,50}, [{42, \"42\"}]}, {{0,0,30}, {0,0,40}, [{30, \"30\"}]}, {{0,0,20}, {0,0,30}, [{20, \"20\"}, {25, \"25\"}]}]),\n    ok.\n\ncreate_timing(_Config) ->\n    Adds = [{20, 1}, {25, 3}, {30, 30}, {42, 42}],\n    DB0 = rrd:create(10, 10, {timing, us}, {0,0,0}),\n    DB1 = lists:foldl(fun ?MODULE:apply/2, DB0, Adds),\n    ?equals(rrd:dump(DB1),\n            [{{0,0,40}, {0,0,50}, {42, 42*42, 1, 42, 42, {histogram,0,[],0,0}}},\n             {{0,0,30}, {0,0,40}, {30, 30*30, 1, 30, 30, {histogram,0,[],0,0}}},\n             {{0,0,20}, {0,0,30}, {1 + 3, 1*1 + 3*3, 2, 1, 3, {histogram,0,[],0,0}}}]),\n    ok.\n\nadd_nonexisting_timeslots(_Config) ->\n    Adds = [{20, 5}, {25, 6}, {30, 1}, {42, 2}],\n    DB0 = rrd:create(10, 10, counter, {0,0,0}),\n    DB1 = lists:foldl(fun ?MODULE:apply/2, DB0, Adds),\n    DB2 = rrd:add_nonexisting_timeslots(DB0, DB1),\n    ?equals(rrd:dump(DB2), [{{0,0,40}, {0,0,50}, 2}, {{0,0,30}, {0,0,40}, 1}, {{0,0,20}, {0,0,30}, 11}]),\n    \n    DB0a = rrd:create(10, 10, counter, {0,0,40}),\n    DB2a = rrd:add_nonexisting_timeslots(DB0a, DB1),\n    ?equals(rrd:dump(DB2a), [{{0,0,40}, {0,0,50}, 2}]),\n    \n    \n    DB0b = rrd:create(10, 10, counter, {0,0,50}),\n    DB2b = rrd:add_nonexisting_timeslots(DB0b, DB1),\n    ?equals(rrd:dump(DB2b), []),\n    \n    ok.\n\nreduce_timeslots(_Config) ->\n    Adds = [{20, 5}, {25, 6}, {30, 1}, {42, 2}],\n    DB0 = rrd:create(10, 10, counter, {0,0,0}),\n    DB1 = lists:foldl(fun ?MODULE:apply/2, DB0, Adds),\n    DB2 = rrd:reduce_timeslots(1, DB1),\n    ?equals(rrd:dump(DB2), [{{0,0,40}, {0,0,50}, 2}]),\n\n    DB2a = rrd:reduce_timeslots(2, DB1),\n    ?equals(rrd:dump(DB2a), [{{0,0,40}, {0,0,50}, 2}, {{0,0,30}, {0,0,40}, 1}]),\n\n    DB2b = rrd:reduce_timeslots(3, DB1),\n    ?equals(rrd:dump(DB2b), [{{0,0,40}, {0,0,50}, 2}, {{0,0,30}, {0,0,40}, 1}, {{0,0,20}, {0,0,30}, 11}]),\n\n    DB2c = rrd:reduce_timeslots(4, DB1),\n    ?equals(rrd:dump(DB2c), [{{0,0,40}, {0,0,50}, 2}, {{0,0,30}, {0,0,40}, 1}, {{0,0,20}, {0,0,30}, 11}]),\n    \n    ok.\n\napply({Time, Value}, DB) ->\n    rrd:add(Time, Value, DB).\n\n%% @doc Performance evaluating of rrd:add_now/2 with a timing type.\n%%      Useful for profiling, e.g. with fprof.\ntiming_perf() ->\n    Init = rrd:create(60 * 1000000, 1, {timing, count}),\n    _ = lists:foldl(fun(_, Old) -> rrd:add_now(1, Old) end, Init, lists:seq(1, 10000)),\n    ok.\n\n%% @doc Converts a time (either a tuple as returned by erlang:now/0 or a number\n%%      representing the number of microseconds since epoch) to the number of\n%%      microseconds since epoch.\n-spec time2us(erlang_timestamp() | rrd:internal_time()) -> rrd:internal_time().\ntime2us({_, _, _} = Time) ->\n    util:timestamp2us(Time);\ntime2us(Time) ->\n    Time.\n\n-spec prop_empty_rrd(SlotLength::rrd:timespan(), Count::1..10, Type::rrd:timeseries_type(),\n                     StartTime::erlang_timestamp() | rrd:internal_time(),\n                     Offsets::[non_neg_integer(),...],\n                     Times::[erlang_timestamp() | rrd:internal_time(),...]) -> true.\nprop_empty_rrd(SlotLength, Count, Type, StartTime, Offsets, Times) ->\n    R = rrd:create(SlotLength, Count, Type, StartTime),\n    StartTime_us = time2us(StartTime),\n    ?equals(rrd:get_slot_start(0, R), StartTime_us),\n    ?equals(rrd:get_type(R), Type),\n    ?equals(rrd:get_slot_length(R), SlotLength),\n    ?equals(rrd:get_current_time(R), StartTime_us),\n    ?equals(rrd:dump(R), []),\n    _ = [?equals(rrd:get_value(R, Time), undefined) || Time <- Times],\n    _ = [?equals(rrd:get_value_by_offset(R, Offset), undefined) || Offset <- Offsets],\n    true.\n\n-type rrd_data() :: [{erlang_timestamp() | rrd:internal_time(), number()},...].\n-type rrd_data_us() :: [{rrd:internal_time(), number()},...].\n\ntester_empty_rrd(_Config) ->\n    tester:test(?MODULE, prop_empty_rrd, 6, 1000, [{threads, 2}]).\n\n-spec round_us_time(Time::rrd:internal_time(), SlotLength::rrd:timespan(),\n                    StartTime::rrd:internal_time()) -> rrd:internal_time().\n%% round_us_time(Time, SlotLength, StartTime) when Time < StartTime ->\n%%     StartTime;\nround_us_time(Time, SlotLength, StartTime) ->\n    case (Time - StartTime) rem SlotLength of\n        X when X >=  0 -> Time - X;\n        X              -> Time - (X + SlotLength)\n    end.\n%%     StartTime + (Time div SlotLength) * SlotLength.\n%%     Time - ((Time - StartTime) rem SlotLength).\n\n%% @doc Merges consecutive data for a 'counter' rrd into a single date item.\n%%      PRE: Data_us2 must be sorted (by its time components) using a stable\n%%           sort.\n-spec merge_conseq_data(Data_us2::rrd_data_us(), gauge | counter, [] | rrd_data_us()) -> rrd_data_us().\nmerge_conseq_data([], _, Result) ->\n    lists:reverse(Result);\nmerge_conseq_data([X | TD], Type, []) ->\n    merge_conseq_data(TD, Type, [X]);\nmerge_conseq_data([{Time, X1} | TD], counter = Type, [{Time, X2} | TR]) ->\n    merge_conseq_data(TD, Type, [{Time, X1 + X2} | TR]);\nmerge_conseq_data([{Time, X1} | TD], gauge = Type, [{Time, _X2} | TR]) ->\n    merge_conseq_data(TD, Type, [{Time, X1} | TR]);\nmerge_conseq_data([X | TD], Type, Result) ->\n    merge_conseq_data(TD, Type, [X | Result]).\n\n-spec prop_counter_gauge_rrd(SlotLength::rrd:timespan(), Count::1..10,\n                             StartTime::erlang_timestamp() | rrd:internal_time(),\n                             Data::rrd_data(), Offsets::[non_neg_integer(),...],\n                             Times::[erlang_timestamp() | rrd:internal_time(),...],\n                             Type::gauge | counter) -> true.\nprop_counter_gauge_rrd(SlotLength, Count, StartTime, Data, _Offsets, _Times, Type) ->\n    R0 = rrd:create(SlotLength, Count, Type, StartTime),\n    % we need sorted data (old data is not inserted into the rrd, only data in the current time slot)\n    % can not reliably sort the time tuples -> only use rrd:internal_time() but\n    Data_us = lists:keysort(1, [{time2us(TimeX), CountX} || {TimeX, CountX} <- Data]),\n    R1 = lists:foldl(fun ?MODULE:apply/2, R0, Data_us),\n    StartTime_us = time2us(StartTime),\n%%     ct:pal(\"StartTime_us: ~.0p~n\", [StartTime_us]),\n    ?equals(rrd:get_type(R1), Type),\n    ?equals(rrd:get_slot_length(R1), SlotLength),\n%%     ct:pal(\"Data_us: ~.0p~n\", [Data_us]),\n    % round down to StartTime + x * SlotLength:\n    Data_us2 = [{round_us_time(TimeX, SlotLength, StartTime_us), CountX} || {TimeX, CountX} <- Data_us],\n%%     ct:pal(\"Data_us2: ~.0p~n\", [Data_us2]),\n    Times_us2 = [TimeX || {TimeX, _} <- Data_us2],\n    CurTime_us = lists:max([StartTime_us | Times_us2]),\n    ?equals(rrd:get_slot_start(0, R1), CurTime_us),\n    ?equals(rrd:get_current_time(R1), CurTime_us),\n    \n    % check data:\n    MinTime_us = erlang:max(StartTime_us - 1, CurTime_us - Count * SlotLength),\n    Data_us2_merged = merge_conseq_data(Data_us2, Type, []),\n%%     ct:pal(\"Data_us2_merged: ~.0p~n\", [Data_us2_merged]),\n    {DataIn, _DataOut} =\n        lists:partition(fun({TimeX, _}) ->\n                                TimeX > MinTime_us andalso\n                                    TimeX =< CurTime_us\n                        end, Data_us2_merged),\n%%     ct:pal(\"DataOut: ~.0p~n\", [DataOut]),\n%%     ct:pal(\"DataIn: ~.0p~n\", [DataIn]),\n    Dump = rrd:dump_with(R1, fun(_, FromX, _ToX, ValueX) -> {FromX, ValueX} end),\n    % note: dump returns the newest value first\n    ?equals(Dump, lists:reverse(DataIn)),\n%%     ?equals(rrd:dump(R), []),\n%%     _ = [?equals(rrd:get_value(R, Time), undefined) || Time <- Times],\n%%     _ = [?equals(rrd:get_value_by_offset(R, Offset), undefined) || Offset <- Offsets],\n    true.\n\n-spec prop_counter_rrd(SlotLength::rrd:timespan(), Count::1..10,\n                       StartTime::erlang_timestamp() | rrd:internal_time(),\n                       Data::rrd_data(), Offsets::[non_neg_integer(),...],\n                       Times::[erlang_timestamp() | rrd:internal_time(),...]) -> true.\nprop_counter_rrd(SlotLength, Count, StartTime, Data, Offsets, Times) ->\n    prop_counter_gauge_rrd(SlotLength, Count, StartTime, Data, Offsets, Times, counter).\n\ntester_counter_rrd(_Config) ->\n    rrd_SUITE:prop_counter_rrd(87, 10, {130,381,3}, [{78,153},{64,-0.236915872440062}], [2,1000], [{388,0,5000}]),\n    rrd_SUITE:prop_counter_rrd(85, 6, 4, [{107,4},{52,0.9981532170299772},{302,4},{232,417}], [126,42,3,4], [{143,255,5},209,138,{5,407,0}]),\n    tester:test(?MODULE, prop_counter_rrd, 6, 5000, [{threads, 2}]).\n\n-spec prop_gauge_rrd(SlotLength::rrd:timespan(), Count::1..10,\n                       StartTime::erlang_timestamp() | rrd:internal_time(),\n                       Data::rrd_data(), Offsets::[non_neg_integer(),...],\n                       Times::[erlang_timestamp() | rrd:internal_time(),...]) -> true.\nprop_gauge_rrd(SlotLength, Count, StartTime, Data, Offsets, Times) ->\n    prop_counter_gauge_rrd(SlotLength, Count, StartTime, Data, Offsets, Times, gauge).\n\ntester_gauge_rrd(_Config) ->\n    rrd_SUITE:prop_gauge_rrd(87, 10, {130,381,3}, [{78,153},{64,-0.236915872440062}], [2,1000], [{388,0,5000}]),\n    rrd_SUITE:prop_gauge_rrd(85, 6, 4, [{107,4},{52,0.9981532170299772},{302,4},{232,417}], [126,42,3,4], [{143,255,5},209,138,{5,407,0}]),\n    tester:test(?MODULE, prop_gauge_rrd, 6, 5000, [{threads, 2}]).\n"
  },
  {
    "path": "test/rrepair_SUITE.erl",
    "content": "%  @copyright 2010-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Tests for rep update module.\n%% @end\n%% @version $Id$\n-module(rrepair_SUITE).\n-author('malange@informatik.hu-berlin.de').\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n%% no proto scheduler for this suite\n-define(proto_sched(_Action), ok).\n\n-include(\"rrepair_SUITE.hrl\").\n\n%% number of executions per test (sub-) group\n-define(NUM_EXECUTIONS, 1).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nall() ->\n    [\n     {group, basic},\n     {group, tester_tests},\n     {group, gsession_ttl},\n     {group, repair}\n    ].\n\ngroups() ->\n    [\n     {gsession_ttl,  [{repeat_until_any_fail, ?NUM_EXECUTIONS}], [session_ttl]},\n     {tester_tests, [parallel], [\n                           tester_map_key_to_interval,\n                           tester_map_key_to_quadrant,\n                           tester_map_interval,\n                           tester_find_sync_interval,\n                           tester_merkle_compress_hashlist,\n                           tester_merkle_pos_to_bitstring%,\n%%                            tester_merkle_compress_cmp_result\n                                ]},\n     {basic,  [parallel], [\n                           check_quadrant_intervals\n                          ]},\n     {repair, [sequence],\n      [\n       {upd_trivial,  [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {upd_shash,    [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {upd_bloom,    [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {upd_merkle,   [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {upd_art,      [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {regen_trivial,[{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default() ++ regen_special()},\n       {regen_shash,  [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default() ++ regen_special()},\n       {regen_bloom,  [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default() ++ regen_special()},\n       {regen_merkle, [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default() ++ regen_special()},\n       {regen_art,    [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default() ++ regen_special()},\n       {mixed_trivial,[{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {mixed_shash,  [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {mixed_bloom,  [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {mixed_merkle, [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {mixed_art,    [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()}\n      ]}\n    ].\n\nsuite() -> [{timetrap, {seconds, 15}}].\n\ninit_per_group_special(tester_tests, Config) ->\n    Config2 = unittest_helper:start_minimal_procs(Config, [], true),\n    tester:register_type_checker({typedef, rt_beh, segment, []}, rt_beh, tester_is_segment),\n    tester:register_value_creator({typedef, rt_beh, segment, []}, rt_beh, tester_create_segment, 1),\n    Config2;\ninit_per_group_special(basic, Config) ->\n    unittest_helper:start_minimal_procs(Config, [], true);\ninit_per_group_special(_, Config) ->\n    Config.\n\nend_per_group_special(tester_tests, Config) ->\n    tester:unregister_value_creator({typedef, rt_beh, segment, []}),\n    tester:unregister_type_checker({typedef, rt_beh, segment, []}),\n    unittest_helper:stop_minimal_procs(Config);\nend_per_group_special(basic, Config) ->\n    unittest_helper:stop_minimal_procs(Config);\nend_per_group_special(_, Config) ->\n    Config.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Basic Functions Group\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ncheck_quadrant_intervals(_) ->\n    Quadrants = rr_recon:quadrant_intervals(),\n    ?equals(lists:foldl(fun intervals:union/2, intervals:empty(), Quadrants),\n            intervals:all()),\n    % all continuous:\n    ?equals([Q || Q <- Quadrants, not intervals:is_continuous(Q)],\n            []),\n    % pair-wise non-overlapping:\n    ?equals([{Q1, Q2} || Q1 <- Quadrants,\n                         Q2 <- Quadrants,\n                         Q1 =/= Q2,\n                         not intervals:is_empty(intervals:intersection(Q1, Q2))],\n            []).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_map_key_to_interval(?RT:key(), intervals:interval()) -> true.\nprop_map_key_to_interval(Key, I) ->\n    Mapped = rr_recon:map_key_to_interval(Key, I),\n    RGrp = ?RT:get_replica_keys(Key),\n    InGrp = [X || X <- RGrp, intervals:in(X, I)],\n    case intervals:in(Key, I) of\n        true ->\n            ?equals_w_note(Mapped, Key,\n                           io_lib:format(\"Violation: if key is in i than mapped key equals key!~n\"\n                                             \"Key=~p~nMapped=~p\", [Key, Mapped]));\n        false when Mapped =/= none ->\n            ?compare(fun erlang:'=/='/2, InGrp, []),\n            case InGrp of\n                [W] -> ?equals(Mapped, W);\n                [_|_] ->\n                    ?assert(intervals:in(Mapped, I)),\n                    % mapped should always be the closest one to Key in I\n                    ?compare(fun({A1, _}, {A2, _}) -> A1 =:= A2 end,\n                             {rr_recon:key_dist(Key, Mapped), Mapped},\n                             lists:min([{rr_recon:key_dist(Key, M), M} || M <- InGrp]))\n            end;\n        _ -> ?equals(InGrp, [])\n    end.\n\ntester_map_key_to_interval(_) ->\n    [Q1, Q2, Q3 | _] = ?RT:get_replica_keys(?MINUS_INFINITY),\n    prop_map_key_to_interval(Q1, intervals:new('[', Q1, Q2, ']')),\n    prop_map_key_to_interval(Q2, intervals:new('[', Q1, Q2, ']')),\n    prop_map_key_to_interval(Q3, intervals:new('[', Q1, Q2, ']')),\n    prop_map_key_to_interval(Q2, intervals:union(intervals:new(Q1), intervals:new(Q3))),\n    tester:test(?MODULE, prop_map_key_to_interval, 2, 1000, [{threads, 4}]).\n\n-spec prop_map_key_to_quadrant(?RT:key(), rt_beh:segment()) -> true.\nprop_map_key_to_quadrant(Key, Quadrant) ->\n    ?equals(rr_recon:map_key_to_quadrant(Key, Quadrant),\n            rr_recon:map_key_to_interval(Key, lists:nth(Quadrant, rr_recon:quadrant_intervals()))).\n\ntester_map_key_to_quadrant(_) ->\n    tester:test(?MODULE, prop_map_key_to_quadrant, 2, 1000, [{threads, 4}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_map_interval(A::intervals:continuous_interval(),\n                        B::intervals:continuous_interval()) -> true.\nprop_map_interval(A, B) ->\n    Quadrants = rr_recon:quadrant_intervals(),\n    % need a B that is in a single quadrant - just use the first one to get\n    % deterministic behaviour:\n    BQ = hd(rr_recon:quadrant_subints_(B, rr_recon:quadrant_intervals(), [])),\n    SA = rr_recon:map_interval(A, BQ),\n\n    % SA must be a sub-interval of A\n    ?compare(fun intervals:is_subset/2, SA, A),\n\n    % SA must be in a single quadrant\n    ?equals([I || Q <- Quadrants,\n                  not intervals:is_empty(\n                    I = intervals:intersection(SA, Q))],\n            ?IIF(intervals:is_empty(SA), [], [SA])),\n\n    % if mapped back, must at least be a subset of BQ:\n    case intervals:is_empty(SA) of\n        true -> true;\n        _ ->\n            ?compare(fun intervals:is_subset/2, rr_recon:map_interval(BQ, SA), BQ)\n    end.\n\ntester_map_interval(_) ->\n    case rt_SUITE:default_rt_has_chord_keys() of\n        true ->\n            prop_map_interval(intervals:new(?MINUS_INFINITY),\n                              intervals:new('[', 45418374902990035001132940685036047259, ?MINUS_INFINITY, ']')),\n            prop_map_interval(intervals:new(?MINUS_INFINITY), intervals:all()),\n            prop_map_interval(intervals:new('[', ?MINUS_INFINITY, 52800909270899328435375133601130059363, ')'),\n                              intervals:new('[', 234596648080609640182865804133877994395, 293423227623586592154289572207917413067, ')'));\n        _ -> ok\n    end,\n    tester:test(?MODULE, prop_map_interval, 2, 1000, [{threads, 1}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_find_sync_interval(intervals:continuous_interval(), intervals:continuous_interval()) -> true.\nprop_find_sync_interval(A, B) ->\n    SyncI = rr_recon:find_sync_interval(A, B),\n    case intervals:is_empty(SyncI) of\n        true -> true;\n        _ ->\n            % continuous:\n            ?assert_w_note(intervals:is_continuous(SyncI), io_lib:format(\"SyncI: ~p\", [SyncI])),\n            % mapped to A, subset of A:\n            ?assert_w_note(intervals:is_subset(SyncI, A), io_lib:format(\"SyncI: ~p\", [SyncI])),\n            Quadrants = rr_recon:quadrant_intervals(),\n            % only in a single quadrant:\n            ?equals([SyncI || Q <- Quadrants,\n                              not intervals:is_empty(intervals:intersection(SyncI, Q))],\n                    [SyncI]),\n            % SyncI must be a subset of B if mapped back\n            ?compare(fun intervals:is_subset/2, rr_recon:map_interval(B, SyncI), B)\n    end.\n\ntester_find_sync_interval(_) ->\n    tester:test(?MODULE, prop_find_sync_interval, 2, 100, [{threads, 4}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec prop_merkle_compress_hashlist(Nodes::[merkle_tree:mt_node()], SigSizeI::0..160, SigSizeL::0..160) -> true.\nprop_merkle_compress_hashlist(Nodes0, SigSizeI, SigSizeL) ->\n    % fix node list which may contain nil hashes:\n    % let it crash if the format of a merkle tree node changes\n    Nodes = [begin\n                 case N of\n                     {nil, Count, ItemCount, Interval, ChildList} ->\n                         {randoms:getRandomInt(), Count, ItemCount,\n                          Interval, ChildList};\n                     {_Hash, _Cnt, _ICnt, _Interval, _ChildList} ->\n                         N;\n                     {nil, ItemCount, Bucket, Interval} ->\n                         {randoms:getRandomInt(), ItemCount, Bucket, Interval};\n                     {_Hash, _ItemCount, _Bucket, _Interval} ->\n                         N\n                 end\n             end || N <- Nodes0],\n    Bin = rr_recon:merkle_compress_hashlist(Nodes, <<>>, SigSizeI, SigSizeL),\n    HashesRed = [begin\n                     H0 = merkle_tree:get_hash(N),\n                     case merkle_tree:is_leaf(N) of\n                         true ->\n                             case merkle_tree:is_empty(N) of\n                                 true  -> H = none;\n                                 false -> <<H:SigSizeL/integer-unit:1>> = <<H0:SigSizeL>>\n                             end;\n                         false ->\n                             <<H:SigSizeI/integer-unit:1>> = <<H0:SigSizeI>>\n                     end,\n                     {H, merkle_tree:is_leaf(N)}\n                 end || N <- Nodes],\n    ?equals(rr_recon:merkle_decompress_hashlist(Bin, SigSizeI, SigSizeL), HashesRed).\n\ntester_merkle_pos_to_bitstring(_) ->\n    tester:test(?MODULE, prop_merkle_pos_to_bitstring, 2, 1000, [{threads, 4}]).\n\n-spec prop_merkle_pos_to_bitstring(Positions::[0..160], FinalSize::1..160) -> true.\nprop_merkle_pos_to_bitstring(Positions0, FinalSize0) ->\n    Positions = lists:usort(Positions0),\n    FinalSize = erlang:max(FinalSize0, lists:max([0 | Positions]) + 1),\n    Bin = erlang:list_to_bitstring(\n            lists:reverse(\n              rr_recon:pos_to_bitstring(Positions, [], 0, FinalSize))),\n    KVList = [{K, 1} || K <- lists:seq(0, FinalSize - 1)],\n    ?equals(lists:reverse(rr_recon:bitstring_to_k_list_kv(Bin, KVList, [])),\n            Positions),\n    KList = [K || K <- lists:seq(0, FinalSize - 1)],\n    ?equals(lists:reverse(rr_recon:bitstring_to_k_list_k(Bin, KList, [])),\n            Positions).\n\ntester_merkle_compress_hashlist(_) ->\n    tester:test(?MODULE, prop_merkle_compress_hashlist, 3, 1000, [{threads, 4}]).\n\n%% -spec prop_merkle_compress_cmp_result(CmpRes::[rr_recon:merkle_cmp_result()],\n%%                                       SigSize::0..160) -> true.\n%% prop_merkle_compress_cmp_result(CmpRes, SigSize) ->\n%%     {Flags, HashesBin} =\n%%         rr_recon:merkle_compress_cmp_result(CmpRes, <<>>, <<>>, SigSize),\n%%     CmpResRed = [case Cmp of\n%%                      {H0} ->\n%%                          <<H:SigSize/integer-unit:1>> = <<H0:SigSize>>,\n%%                          {H};\n%%                      X -> X\n%%                  end || Cmp <- CmpRes],\n%%     ?equals(rr_recon:merkle_decompress_cmp_result(Flags, HashesBin, [], SigSize),\n%%             CmpResRed).\n%%\n%% tester_merkle_compress_cmp_result(_) ->\n%%     tester:test(?MODULE, prop_merkle_compress_cmp_result, 2, 1000, [{threads, 4}]).\n"
  },
  {
    "path": "test/rrepair_SUITE.hrl",
    "content": "%  @copyright 2010-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Maik Lange <malange@informatik.hu-berlin.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Unit tests for the replica repair modules.\n%%   rrepair_SUITE.erl:\n%%       The regular execution of the suite.\n%%   rrepair_proto_sched_SUITE.erl:\n%%       Executes using the proto scheduler which serializes all messages to\n%%       generate a random interleaving of messages on different channels.\n%% @end\n%% @version $Id$\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n-include(\"record_helpers.hrl\").\n\n-define(DBSizeKey, rrepair_SUITE_dbsize).    %Process Dictionary Key for generated db size\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nrepair_default() ->\n    [no_diff,        % ring is not out of sync e.g. no outdated or missing replicas\n     one_node,       % sync in ring with only one node\n     %mpath\n     dest,           % run one sync with a specified dest node\n     simple,         % run one sync round\n     multi_round,    % run multiple sync rounds with sync probability 1\n     multi_round2,   % run multiple sync rounds with sync probability 0.4\n     parts,          % get_chunk with limited items (leads to multiple get_chunk calls, in case of bloom also multiple bloom filters)\n     asymmetric_ring % rrepair in an asymmetric ring with a node covering more than a quadrant (no other checks - it just needs to run successfully)\n    ].\n\nregen_special() ->\n    [\n     dest_empty_node % run one sync with empty dest node\n    ].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ninit_per_suite(Config) ->\n    tester:register_type_checker({typedef, intervals, interval, []}, intervals, is_well_formed),\n    tester:register_type_checker({typedef, intervals, continuous_interval, []}, intervals, is_continuous),\n    tester:register_type_checker({typedef, intervals, non_empty_interval, []}, intervals, is_non_empty),\n    tester:register_value_creator({typedef, intervals, interval, []}, intervals, tester_create_interval, 1),\n    tester:register_value_creator({typedef, intervals, continuous_interval, []}, intervals, tester_create_continuous_interval, 4),\n    tester:register_value_creator({typedef, intervals, non_empty_interval, []}, intervals, tester_create_non_empty_interval, 2),\n    rt_SUITE:register_value_creator(),\n    Config.\n\nend_per_suite(_Config) ->\n    tester:unregister_type_checker({typedef, intervals, interval, []}),\n    tester:unregister_type_checker({typedef, intervals, continuous_interval, []}),\n    tester:unregister_type_checker({typedef, intervals, non_empty_interval, []}),\n    tester:unregister_value_creator({typedef, intervals, interval, []}),\n    tester:unregister_value_creator({typedef, intervals, continuous_interval, []}),\n    tester:unregister_value_creator({typedef, intervals, non_empty_interval, []}),\n    rt_SUITE:unregister_value_creator(),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ninit_per_group(Group, Config0) ->\n    ct:comment(io_lib:format(\"BEGIN ~p\", [Group])),\n    Config = init_per_group_special(Group, Config0),\n    case Group of\n        upd_trivial -> [{ru_method, trivial}, {ftype, update}];\n        upd_shash -> [{ru_method, shash}, {ftype, update}];\n        upd_bloom -> [{ru_method, bloom}, {ftype, update}];\n        upd_merkle -> [{ru_method, merkle_tree}, {ftype, update}];\n        upd_art -> [{ru_method, art}, {ftype, update}];\n        regen_trivial -> [{ru_method, trivial}, {ftype, regen}];\n        regen_shash -> [{ru_method, shash}, {ftype, regen}];\n        regen_bloom -> [{ru_method, bloom}, {ftype, regen}];\n        regen_merkle -> [{ru_method, merkle_tree}, {ftype, regen}];\n        regen_art -> [{ru_method, art}, {ftype, regen}];\n        mixed_trivial -> [{ru_method, trivial}, {ftype, mixed}];\n        mixed_shash -> [{ru_method, shash}, {ftype, mixed}];\n        mixed_bloom -> [{ru_method, bloom}, {ftype, mixed}];\n        mixed_merkle -> [{ru_method, merkle_tree}, {ftype, mixed}];\n        mixed_art -> [{ru_method, art}, {ftype, mixed}];\n        _ -> []\n    end ++ Config.\n\nend_per_group(Group, Config) ->\n    Method = proplists:get_value(ru_method, Config, undefined),\n    FType = proplists:get_value(ftype, Config, undefined),\n    case Method of\n        undefined -> ct:comment(io_lib:format(\"END ~p\", [Group]));\n        M -> ct:comment(io_lib:format(\"END ~p/~p\", [FType, M]))\n    end,\n    Config2 = proplists:delete(ru_method, Config),\n    end_per_group_special(Group, Config2).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\ninit_per_testcase(_TestCase, Config) ->\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nget_rep_upd_config(Method) ->\n    [{rrepair_enabled, true},\n     {rr_trigger_interval, 0}, %stop trigger\n     {rr_recon_method, Method},\n     {rr_session_ttl, 100000},\n     {rr_gc_interval, 60000},\n     {rr_recon_failure_rate, 0.1},\n     {rr_trigger_probability, 100},\n     {rr_art_inner_fpr, 0.01},\n     {rr_art_leaf_fpr, 0.1},\n     {rr_art_correction_factor, 2},\n     {rr_merkle_branch_factor, 4},\n     {rr_merkle_bucket_size, 3}].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Replica Update tests\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nno_diff(Config) ->\n    Method = proplists:get_value(ru_method, Config),\n    FType = proplists:get_value(ftype, Config),\n    start_sync(Config, 4, 1000, [{fprob, 0}, {ftype, FType}],\n               1, 0.1, get_rep_upd_config(Method), fun erlang:'=:='/2).\n\none_node(Config) ->\n    Method = proplists:get_value(ru_method, Config),\n    FType = proplists:get_value(ftype, Config),\n    % atm, there is no rrepair with self (we wouldn't need a complex protocol for that)\n    start_sync(Config, 1, 1, [{fprob, 50}, {ftype, FType}],\n               1, 0.2, get_rep_upd_config(Method), fun erlang:'=:='/2).\n\nmpath_map({X, {?key_upd, KVV, ReqKeys}, _}, _Source, _Dest)\n  when X =:= request_resolve orelse X =:= continue_resolve ->\n    {?key_upd, length(KVV), length(ReqKeys)};\nmpath_map({X, _, {?key_upd, KVV, ReqKeys}, _}, _Source, _Dest)\n  when X =:= request_resolve orelse X =:= continue_resolve ->\n    {?key_upd, length(KVV), length(ReqKeys)};\nmpath_map(Msg, _Source, _Dest) ->\n    {element(1, Msg)}.\n\nmpath(Config) ->\n    %parameter\n    NodeCount = 4,\n    DataCount = 1000,\n    FR = 0.1,\n    Method = proplists:get_value(ru_method, Config),\n    FType = proplists:get_value(ftype, Config),\n    TraceName = erlang:list_to_atom(atom_to_list(Method)++atom_to_list(FType)),\n    %build and fill ring\n    _ = build_ring(NodeCount, Config, [get_rep_upd_config(Method),\n                                       {rr_recon_failure_rate, FR}]),\n    _ = db_generator:fill_ring(random, DataCount, [{ftype, FType},\n                                                   {fprob, 50},\n                                                   {distribution, uniform}]),\n    %chose node pair\n    SKey = ?RT:get_random_node_id(),\n    CKey = util:randomelem(lists:delete(SKey, ?RT:get_replica_keys(SKey))),\n    %server starts sync\n    %trace_mpath:start(TraceName, fun mpath_map/3),\n    trace_mpath:start(TraceName),\n    api_dht_raw:unreliable_lookup(SKey, {?send_to_group_member, rrepair,\n                                              {request_sync, Method, CKey, comm:this()}}),\n    waitForSyncRoundEnd([SKey, CKey], true),\n    trace_mpath:stop(),\n    %TRACE\n    A = trace_mpath:get_trace(TraceName, cleanup),\n    B = [X || X = {log_send, _Time, _TraceID,\n                   {{_FIP,_FPort,_FPid}, _FName},\n                   {{_TIP,_TPort,_TPid}, _TName},\n                   _Msg, _LocalOrGlobal} <- A],\n    ok = file:write_file(\"TRACE_\" ++ atom_to_list(TraceName) ++ \".txt\",\n                         io_lib:fwrite(\"~.0p\\n\", [B])),\n    ok = file:write_file(\"TRACE_HISTO_\" ++ atom_to_list(TraceName) ++ \".txt\",\n                         io_lib:fwrite(\"~.0p\\n\", [trace_mpath:send_histogram(B)])),\n    %ok = file:write_file(\"TRACE_EVAL_\" ++ atom_to_list(TraceName) ++ \".txt\",\n    %                     io_lib:fwrite(\"~.0p\\n\", [rr_eval_admin:get_bandwidth(A)])),\n    ok.\n\nsimple(Config) ->\n    Method = proplists:get_value(ru_method, Config),\n    FType = proplists:get_value(ftype, Config),\n    %% use erlang:=< instead of < as progress is not guaranteed for a\n    %% single round\n    start_sync(Config, 4, 1000, [{fprob, 10}, {ftype, FType}],\n               1, 0.1, get_rep_upd_config(Method), fun erlang:'=<'/2).\n\nmulti_round(Config) ->\n    Method = proplists:get_value(ru_method, Config),\n    FType = proplists:get_value(ftype, Config),\n    start_sync(Config, 6, 1000, [{fprob, 10}, {ftype, FType}],\n               3, 0.1, get_rep_upd_config(Method), fun erlang:'<'/2).\n\nmulti_round2(Config) ->\n    Method = proplists:get_value(ru_method, Config),\n    FType = proplists:get_value(ftype, Config),\n    _RUConf = get_rep_upd_config(Method),\n    RUConf = [{rr_trigger_probability, 40} | proplists:delete(rr_trigger_probability, _RUConf)],\n    start_sync(Config, 6, 1000, [{fprob, 10}, {ftype, FType}],\n               3, 0.1, RUConf, fun erlang:'<'/2).\n\ndest(Config) ->\n    %parameter\n    NodeCount = 7,\n    DataCount = 1000,\n    FR = 0.1,\n    Method = proplists:get_value(ru_method, Config),\n    FType = proplists:get_value(ftype, Config),\n    %build and fill ring\n    _ = build_ring(NodeCount, Config, [get_rep_upd_config(Method),\n                                       {rr_recon_failure_rate, FR}]),\n    _ = db_generator:fill_ring(random, DataCount, [{ftype, FType},\n                                                   {fprob, 50},\n                                                   {distribution, uniform}]),\n    RingData = unittest_helper:get_ring_data(kv),\n    {DBKeys, _DBKeysNum, _Outdated} = create_full_db(RingData, [], 0, 0),\n    %chose node pair\n    SKey = ?RT:get_random_node_id(),\n    CKey = util:randomelem(lists:delete(SKey, ?RT:get_replica_keys(SKey))),\n    %measure initial sync degree\n    SO = count_outdated(RingData, SKey),\n    SM = count_dbsize(RingData, SKey),\n    CO = count_outdated(RingData, CKey),\n    CM = count_dbsize(RingData, CKey),\n    %server starts sync\n    ?proto_sched(start),\n    api_dht_raw:unreliable_lookup(SKey, {?send_to_group_member, rrepair,\n                                              {request_sync, Method, CKey, comm:this()}}),\n    waitForSyncRoundEnd([SKey, CKey], true),\n    ?proto_sched(stop),\n    %measure sync degree\n    RingDataNew = unittest_helper:get_ring_data(kv),\n    SONew = count_outdated(RingDataNew, SKey),\n    SMNew = count_dbsize(RingDataNew, SKey),\n    CONew = count_outdated(RingDataNew, CKey),\n    CMNew = count_dbsize(RingDataNew, CKey),\n    remove_full_db(DBKeys),\n    ct:pal(\"SYNC RUN << ~p / ~p >>~nServerKey=~p~nClientKey=~p~n\"\n           \"Server Outdated=[~p -> ~p] DBSize=[~p -> ~p] - Upd=~p ; Regen=~p~n\"\n           \"Client Outdated=[~p -> ~p] DBSize=[~p -> ~p] - Upd=~p ; Regen=~p\",\n           [Method, FType, SKey, CKey,\n            SO, SONew, SM, SMNew, SO - SONew, SMNew - SM,\n            CO, CONew, CM, CMNew, CO - CONew, CMNew - CM]),\n    %clean up\n    ?implies(SO > 0 orelse CO > 0, SONew < SO orelse CONew < CO) andalso\n        ?implies(SM =/= SMNew, SMNew > SM) andalso\n        ?implies(CM =/= CMNew, CMNew > CM).\n\ndest_empty_node(Config) ->\n    %parameter\n    DataCount = 1000,\n    FR = 0.1,\n    Method = proplists:get_value(ru_method, Config),\n    %build and fill ring\n    _ = build_ring(symmetric, Config,\n                   [get_rep_upd_config(Method), {rr_recon_failure_rate, FR}]),\n    _ = db_generator:fill_ring(random, DataCount, [{ftype, regen},\n                                                   {fprob, 100},\n                                                   {distribution, uniform},\n                                                   {fdest, [1]}]),\n    %chose any node not in quadrant 1\n    KeyGrp = ?RT:get_replica_keys(?RT:get_random_node_id()),\n    [Q1 | _] = rr_recon:quadrant_intervals(),\n    IKey = util:randomelem([X || X <- KeyGrp, not intervals:in(X, Q1)]),\n    CKey = hd([Y || Y <- KeyGrp, intervals:in(Y, Q1)]),\n    %measure initial sync degree\n    RingData = unittest_helper:get_ring_data(kv),\n    IM = count_dbsize(RingData, IKey),\n    CM = count_dbsize(RingData, CKey),\n    ?equals(CM, 0),\n    %server starts sync\n    ?proto_sched(start),\n    api_dht_raw:unreliable_lookup(IKey, {?send_to_group_member, rrepair,\n                                              {request_sync, Method, CKey, comm:this()}}),\n    waitForSyncRoundEnd([IKey, CKey], true),\n    ?proto_sched(stop),\n    %measure sync degree\n    RingDataNew = unittest_helper:get_ring_data(kv),\n    IMNew = count_dbsize(RingDataNew, IKey),\n    CMNew = count_dbsize(RingDataNew, CKey),\n    ct:pal(\"SYNC RUN << ~p >>~nServerKey=~p~nClientKey=~p~n\"\n           \"Server DBSize=[~p -> ~p] - Regen=~p~n\"\n           \"Client DBSize=[~p -> ~p] - Regen=~p\",\n           [Method, IKey, CKey,\n            IM, IMNew, IMNew - IM,\n            CM, CMNew, CMNew - CM]),\n    %clean up\n    case Method of\n        art ->\n            % ART is not able to detect this case and resolve everything\n            ?compare(fun erlang:'>'/2, CMNew, CM);\n        _   -> ?equals(CMNew, IM)\n    end.\n\nparts(Config) ->\n    Method = proplists:get_value(ru_method, Config),\n    FType = proplists:get_value(ftype, Config),\n    OldConf = get_rep_upd_config(Method),\n    Conf = lists:keyreplace(rr_max_items, 1, OldConf, {rr_max_items, 500}),\n    %% use erlang:=< instead of < as progress is not guaranteed for a\n    %% few rounds\n    start_sync(Config, 4, 1000, [{fprob, 100}, {ftype, FType}],\n               2, 0.2, Conf, fun erlang:'=<'/2).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec wait_until_true(DestKey::?RT:key(), Request::comm:message(),\n                      ConFun::fun((StateResponse::term()) -> boolean()),\n                      MaxWait::integer()) -> boolean().\nwait_until_true(DestKey, Request, ConFun, MaxWait) ->\n    api_dht_raw:unreliable_lookup(DestKey, Request),\n    trace_mpath:thread_yield(),\n    Result = receive ?SCALARIS_RECV({get_state_response, R}, ConFun(R)) end,\n    if Result -> true;\n       not Result andalso MaxWait > 0 ->\n           erlang:yield(),\n           timer:sleep(10),\n           wait_until_true(DestKey, Request, ConFun, MaxWait - 10);\n       not Result andalso MaxWait =< 0 ->\n           false\n    end.\n\nsession_ttl(Config) ->\n    %parameter\n    NodeCount = 7,\n    DataCount = 1000,\n    Method = merkle_tree,\n    FType = mixed,\n    TTL = 2000,\n\n    RRConf1 = lists:keyreplace(rr_session_ttl, 1, get_rep_upd_config(Method), {rr_session_ttl, TTL div 2}),\n    RRConf = lists:keyreplace(rr_gc_interval, 1, RRConf1, {rr_gc_interval, erlang:round(TTL div 2)}),\n\n    %build and fill ring\n    _ = build_ring(NodeCount, Config, RRConf),\n    _ = db_generator:fill_ring(random, DataCount, [{ftype, FType},\n                                                   {fprob, 90},\n                                                   {distribution, uniform}]),\n    %chose node pair\n    SKey = ?RT:get_random_node_id(),\n    CKey = util:randomelem(lists:delete(SKey, ?RT:get_replica_keys(SKey))),\n\n    api_dht_raw:unreliable_lookup(CKey, {get_pid_group, comm:this()}),\n    CName = receive {get_pid_group_response, Key} -> Key end,\n\n    %server starts sync\n    ?proto_sched(start),\n    api_dht_raw:unreliable_lookup(SKey, {?send_to_group_member, rrepair,\n                                              {request_sync, Method, CKey}}),\n    Req = {?send_to_group_member, rrepair, {get_state, comm:this(), open_sessions}},\n    SessionOpened = wait_until_true(SKey, Req, fun(X) -> length(X) =/= 0 end, TTL),\n    ?proto_sched(stop),\n\n    %check timeout\n    api_vm:kill_node(CName),\n    timer:sleep(TTL),\n    api_dht_raw:unreliable_lookup(SKey, Req),\n    SessionGarbageCollected = receive {get_state_response, R2} -> length(R2) =:= 0 end,\n    case SessionOpened of\n        true ->\n            ?equals(SessionGarbageCollected, SessionOpened);\n        false ->\n            ct:pal(\"Session finished before client node could be killed.\")\n    end,\n    ok.\n\nasymmetric_ring(Config) ->\n    Method = proplists:get_value(ru_method, Config),\n    FType = proplists:get_value(ftype, Config),\n    start_sync(Config, 4, 1000, [{fprob, 10}, {ftype, FType}],\n               1, 0.01, get_rep_upd_config(Method), fun erlang:'=<'/2).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Helper Functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% @doc\n%    runs the bloom filter synchronization [Rounds]-times\n%    and records the sync degree after each round\n%    returns list of sync degrees per round, first value is initial sync degree\n% @end\n-spec start_sync(Config, Nodes::Int | symmetric, DBSize::Int, DBParams,\n                 Rounds::Int, FR, RRConf::Config, CompFun) -> true when\n    is_subtype(Config,      [tuple()]),\n    is_subtype(Int,         pos_integer()),\n    is_subtype(DBParams,    [db_generator:db_parameter()]),\n    is_subtype(FR,         float()),\n    is_subtype(CompFun,     fun((T, T) -> boolean())).\nstart_sync(Config, NodeCount, DBSize, DBParams, Rounds, FR, RRConfig, CompFun) ->\n    % NOTE: a sync round may not decrease the sync degree if there is no error on the participating nodes!\n    NodeKeys = build_ring(NodeCount, Config, [RRConfig, {rr_recon_failure_rate, FR}]),\n    Nodes = [begin\n                 comm:send_local(NodePid, {get_node_details, comm:this(), [node]}),\n                 trace_mpath:thread_yield(),\n                 receive\n                     ?SCALARIS_RECV({get_node_details_response, NodeDetails},\n                                    begin\n                                        Node = node_details:get(NodeDetails, node),\n                                        Pid = comm:make_local(node:pidX(Node)),\n                                        {node:id(Node), Pid, pid_groups:group_of(Pid)}\n                                    end);\n                     ?SCALARIS_RECV(Y, ?ct_fail(\"unexpected message while \"\n                                                \"waiting for get_node_details_response: ~.0p\",\n                                                [Y]))\n                 end\n             end || NodePid <- pid_groups:find_all(dht_node)],\n    ct:pal(\">>Nodes: ~.2p\", [Nodes]),\n    _ = db_generator:fill_ring(random, DBSize, DBParams),\n    InitDBStat = get_db_status(),\n    print_status(0, InitDBStat),\n    ?proto_sched(start),\n    _ = util:for_to_ex(1, Rounds,\n                       fun(I) ->\n                               ct:pal(\"Starting round ~p\", [I]),\n                               startSyncRound(NodeKeys),\n                               waitForSyncRoundEnd(NodeKeys, false),\n                               if I =/= Rounds ->\n                                      print_status(I, get_db_status());\n                                  true -> ok\n                               end\n                       end),\n    ?proto_sched(stop),\n    EndStat = get_db_status(),\n    print_status(Rounds, EndStat),\n    ?compare_w_note(CompFun, sync_degree(InitDBStat), sync_degree(EndStat),\n                    io_lib:format(\"CompFun: ~p\", [CompFun])),\n    unittest_helper:stop_ring(),\n    true.\n\n-spec print_status(Round::integer(), db_generator:db_status()) -> ok.\nprint_status(R, {_, _, M, O}) ->\n    ct:pal(\">>SYNC RUN [Round ~p] Missing=[~p] Outdated=[~p]\", [R, M, O]).\n\n-spec create_full_db(RingData::unittest_helper:ring_data([{?RT:key(), client_version()}]),\n                     Keys, KeysNum, Outdated)\n        -> {Keys, KeysNum, Outdated}\n          when is_subtype(Keys, [?RT:key()]),\n               is_subtype(KeysNum, non_neg_integer()),\n               is_subtype(Outdated, non_neg_integer()).\ncreate_full_db([{_Pid, _I, DB, _Pred, _Succ, ok} | Rest], Keys, KeysNum, Outdated) ->\n    {Keys2, KeysNum2, Outdated2} = create_full_db2(DB, Keys, KeysNum, Outdated),\n    create_full_db(Rest, Keys2, KeysNum2, Outdated2);\ncreate_full_db([{exception, _Level, _Reason, _Stacktrace} | Rest], Keys, KeysNum, Outdated) ->\n    create_full_db(Rest, Keys, KeysNum, Outdated);\ncreate_full_db([], Keys, KeysNum, Outdated) ->\n    {Keys, KeysNum, Outdated}.\n\n-spec create_full_db2(DB::db_dht:db_as_list(), Keys, KeysNum, Outdated)\n        -> {Keys, KeysNum, Outdated}\n          when is_subtype(Keys, [?RT:key()]),\n               is_subtype(KeysNum, non_neg_integer()),\n               is_subtype(Outdated, non_neg_integer()).\ncreate_full_db2([H | TL], Keys, KeysNum, Outdated) ->\n    Key = rr_recon:map_key_to_quadrant(element(1, H), 1),\n    Version = element(2, H),\n    DictKey = {'$test_full_db', Key},\n    case erlang:get(DictKey) of\n        Version ->\n            % note: Key already in Keys\n            create_full_db2(TL, Keys, KeysNum, Outdated);\n        undefined ->\n            _ = erlang:put(DictKey, Version),\n            create_full_db2(TL, [Key | Keys], KeysNum + 1, Outdated);\n        VOld when VOld < Version ->\n            _ = erlang:put(DictKey, Version),\n            % note: Key already in Keys\n            create_full_db2(TL, Keys, KeysNum, Outdated + 1);\n        _VOld -> %when VOld > Version ->\n            % note: Key already in Keys\n            create_full_db2(TL, Keys, KeysNum, Outdated + 1)\n    end;\ncreate_full_db2([], Keys, KeysNum, Outdated) ->\n    {Keys, KeysNum, Outdated}.\n\n-spec remove_full_db([?RT:key()]) -> ok.\nremove_full_db(Keys) ->\n    lists:foreach(fun(Key) -> erlang:erase({'$test_full_db', Key}) end, Keys),\n    ok.\n\n%% @doc Counts outdated items on the node responsible for the given key.\n%%      PreCond: full DB in process dictionary - see create_full_db/2!\n-spec count_outdated(RingData::unittest_helper:ring_data([{?RT:key(), client_version()}]),\n                     Node::?RT:key()) -> non_neg_integer().\ncount_outdated(RingData, NodeKey) ->\n    N = lists:filter(fun({_Pid, {LBr, LK, RK, RBr}, _DB, _Pred, _Succ, ok}) ->\n                             intervals:in(NodeKey, intervals:new(LBr, LK, RK, RBr))\n                     end, RingData),\n    case N of\n        [{_Pid, _I, DB, _Pred, _Succ, ok}] ->\n            lists:foldl(\n              fun(E, Acc) ->\n                      Key = rr_recon:map_key_to_quadrant(element(1, E), 1),\n                      MyVersion = element(2, E),\n                      UpdVersion = erlang:get({'$test_full_db', Key}),\n                      ?DBG_ASSERT(UpdVersion =/= undefined),\n                      if UpdVersion > MyVersion -> Acc + 1;\n                         true -> Acc\n                      end\n              end, 0, DB);\n        _ -> 0\n    end.\n\n-spec get_node_list() -> [comm:mypid()].\nget_node_list() ->\n    mgmt_server:node_list(),\n    receive\n        {get_list_response, N} -> N\n    end.\n\n%% @doc Counts db size on the node responsible for the given key.\n-spec count_dbsize(RingData::unittest_helper:ring_data([{?RT:key(), client_version()}]),\n                   Node::?RT:key()) -> non_neg_integer().\ncount_dbsize(RingData, NodeKey) ->\n    N = lists:filter(\n          fun({_Pid, {LBr, LK, RK, RBr}, _DB, _Pred, _Succ, ok}) ->\n                  intervals:in(NodeKey, intervals:new(LBr, LK, RK, RBr))\n          end, RingData),\n    case N of\n        [{_Pid, _I, DB, _Pred, _Succ, ok}] -> length(DB);\n        _ -> 0\n    end.\n\n-spec get_db_status() -> db_generator:db_status().\nget_db_status() ->\n    RingData = unittest_helper:get_ring_data(kv),\n    Stored =\n        lists:foldl(\n          fun({_Pid, _I, DB, _Pred, _Succ, ok}, Acc) ->\n                  length(DB) + Acc;\n             ({exception, _Level, _Reason, _Stacktrace}, Acc) ->\n                  Acc\n          end, 0, RingData),\n    {DBKeys, DBKeysNum, Outdated} = create_full_db(RingData, [], 0, 0),\n    DBSize = config:read(replication_factor) * DBKeysNum,\n    remove_full_db(DBKeys),\n    {DBSize, Stored, DBSize - Stored, Outdated}.\n\nbuild_ring(NodeCount0, Config, RRConfig) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    % stop ring from previous test case (it may have run into a timeout)\n    unittest_helper:stop_ring(),\n    case NodeCount0 of\n        symmetric ->\n            % Build ring with NodeCount symmetric nodes\n            unittest_helper:make_symmetric_ring(\n              [{config, lists:flatten([{log_path, PrivDir}, RRConfig])}]),\n            NodeCount = config:read(replication_factor),\n            ok;\n        NodeCount when is_integer(NodeCount) andalso NodeCount > 0 ->\n            % Build ring with NodeCount arbitrary nodes\n            Config2 = unittest_helper:start_minimal_procs(Config, [], false),\n            NodeKeys0 = util:for_to_ex(1, NodeCount, fun(_) -> ?RT:get_random_node_id() end),\n            _ = unittest_helper:stop_minimal_procs(Config2),\n            unittest_helper:make_ring_with_ids(\n              NodeKeys0,\n              [{config, lists:flatten([{log_path, PrivDir}, RRConfig])}])\n    end,\n    % wait for all nodes to finish their join\n    unittest_helper:check_ring_size_fully_joined(NodeCount),\n%%     % wait a bit for the rm-processes to settle\n%%     timer:sleep(500),\n    _NodeKeys = [begin\n                     comm:send_local(Pid, {get_state, comm:this(), node_id}),\n                     receive\n                         ?SCALARIS_RECV({get_state_response, Key}, Key)\n                     end\n                 end || Pid <- pid_groups:find_all(dht_node)].\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% Analysis\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec startSyncRound(NodeKeys::[?RT:key()]) -> ok.\nstartSyncRound(NodeKeys) ->\n    lists:foreach(\n      fun(X) ->\n              Req = {?send_to_group_member, rrepair,\n                     {request_sync, config:read(rr_recon_method), random}},\n              api_dht_raw:unreliable_lookup(X, Req)\n      end,\n      NodeKeys),\n    ok.\n\n-spec waitForSyncRoundEnd(NodeKeys::[?RT:key()], RcvReqCompleteMsg::boolean()) -> ok.\nwaitForSyncRoundEnd(NodeKeys, RcvReqCompleteMsg) ->\n    case RcvReqCompleteMsg of\n        true ->\n            trace_mpath:thread_yield(),\n            receive\n                ?SCALARIS_RECV(\n                   {request_sync_complete = _MsgTag, _}, % ->\n                   ok)\n                end;\n        false -> ok\n    end,\n    Req = {?send_to_group_member, rrepair,\n           {get_state, comm:this(), [open_sessions, open_recon, open_resolve]}},\n    util:wait_for(fun() -> wait_for_sync_round_end2(Req, NodeKeys) end, 100).\n\n-spec wait_for_sync_round_end2(Req::comm:message(), [?RT:key()]) -> boolean().\nwait_for_sync_round_end2(_Req, []) -> true;\nwait_for_sync_round_end2(Req, [Key | Keys]) ->\n    api_dht_raw:unreliable_lookup(Key, Req),\n    KeyResult =\n        begin\n        trace_mpath:thread_yield(),\n        receive\n            ?SCALARIS_RECV(\n            {get_state_response = _MsgTag, [Sessions, ORC, ORS]}, % ->\n            begin\n                if (ORC =:= 0 andalso ORS =:= 0 andalso\n                        Sessions =:= []) ->\n                       true;\n                   true ->\n%%                        log:pal(\"Key: ~.2p~nOS : ~.2p~nORC: ~p, ORS: ~p~n\",\n%%                                [Key, Sessions, ORC, ORS]),\n                       false\n                end\n            end)\n        end end,\n    if KeyResult -> wait_for_sync_round_end2(Req, Keys);\n       true -> false\n    end.\n\n-spec sync_degree(db_generator:db_status()) -> float().\nsync_degree({Count, _Ex, M, O}) ->\n    (Count - M - O) / Count.\n\n"
  },
  {
    "path": "test/rrepair_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{suites, tests, [db_generator_SUITE]}.\n{suites, tests, [bloom_SUITE]}.\n{suites, tests, [merkle_tree_SUITE]}.\n{suites, tests, [art_SUITE]}.\n{suites, tests, [iblt_SUITE]}.\n{suites, tests, [rrepair_SUITE]}.\n{suites, tests, [rrepair_proto_sched_SUITE]}.\n%{suites, tests, [rr_recon_performance_SUITE]}.\n\n{cases, tests, type_check_SUITE, [tester_type_check_rrepair]}.\n"
  },
  {
    "path": "test/rrepair_proto_sched_SUITE.erl",
    "content": "%  @copyright 2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Tests for the replica repair modules.\n%% @end\n%% @version $Id$\n-module(rrepair_proto_sched_SUITE).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n%% start proto scheduler for this suite\n-define(proto_sched(Action), proto_sched_fun(Action)).\n\n-include(\"rrepair_SUITE.hrl\").\n\n%% number of executions per test (sub-) group\n-define(NUM_EXECUTIONS, 5).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\nall() ->\n    [\n     {group, gsession_ttl},\n     {group, repair}\n    ].\n\ngroups() ->\n    [\n     {gsession_ttl,  [{repeat_until_any_fail, ?NUM_EXECUTIONS}], [session_ttl]},\n     {repair, [sequence],\n      [\n       {upd_trivial,  [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {upd_shash,    [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {upd_bloom,    [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {upd_merkle,   [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {upd_art,      [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {regen_trivial,[{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default() ++ regen_special()},\n       {regen_shash,  [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default() ++ regen_special()},\n       {regen_bloom,  [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default() ++ regen_special()},\n       {regen_merkle, [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default() ++ regen_special()},\n       {regen_art,    [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default() ++ regen_special()},\n       {mixed_trivial,[{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {mixed_shash,  [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {mixed_bloom,  [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {mixed_merkle, [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()},\n       {mixed_art,    [{repeat_until_any_fail, ?NUM_EXECUTIONS}], repair_default()}\n      ]}\n    ].\n\nsuite() -> [ {timetrap, {seconds, 20}} ].\n\ninit_per_group_special(_Group, Config) ->\n    Config.\n\nend_per_group_special(_Group, Config) ->\n    Config.\n\n-spec proto_sched_fun(start | stop) -> ok.\nproto_sched_fun(start) ->\n    %% ct:pal(\"Starting proto scheduler\"),\n    proto_sched:thread_num(1),\n    proto_sched:thread_begin();\nproto_sched_fun(stop) ->\n    %% is a ring running?\n    case erlang:whereis(pid_groups) =:= undefined\n             orelse pid_groups:find_a(proto_sched) =:= failed of\n        true -> ok;\n        false ->\n            %% then finalize proto_sched run:\n            %% try to call thread_end(): if this\n            %% process was running the proto_sched\n            %% thats fine, otherwise thread_end()\n            %% will raise an exception\n            proto_sched:thread_end(),\n            proto_sched:wait_for_end(),\n            unittest_helper:print_proto_sched_stats(at_end_if_failed),\n            proto_sched:cleanup()\n    end.\n"
  },
  {
    "path": "test/rt_SUITE.erl",
    "content": "% @copyright 2010-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Unit tests for the current ?RT module.\n%% @end\n-module(rt_SUITE).\n-author('kruber@zib.de').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n\nall() ->\n    [tester_client_key_to_binary, tester_hash_key,\n     next_hop, next_hop2,\n     tester_get_split_key, tester_get_split_key_half,\n     additional_tests].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 40}}\n    ].\n\nregister_value_creator() ->\n    case lists:member(?RT, [rt_chord, rt_frt, rt_gfrt, rt_simple]) of\n        true -> ok;\n        false -> tester:register_value_creator({typedef, ?RT, key, []},\n                                               ?RT, create_key, 2)\n    end.\n\nunregister_value_creator() ->\n    case lists:member(?RT, [rt_chord, rt_frt, rt_gfrt, rt_simple]) of\n        true -> ok;\n        false -> tester:unregister_value_creator({typedef, ?RT, key, []})\n    end.\n\ninit_per_suite(Config) ->\n    register_value_creator(),\n    unittest_helper:start_minimal_procs(Config, [], true).\n\nend_per_suite(Config) ->\n    unregister_value_creator(),\n    _ = unittest_helper:stop_minimal_procs(Config),\n    ok.\n\n%% @doc Returns whether the default routing table from ?RT has chord-like keys.\n-spec default_rt_has_chord_keys() -> boolean().\ndefault_rt_has_chord_keys() ->\n    lists:member(?RT, [rt_simple, rt_chord, rt_gfrtchord, rt_gfrtchord]).\n\n-spec prop_client_key_to_binary(Key1::client_key(), Key2::client_key()) -> true | no_return().\nprop_client_key_to_binary(Key1, Key2) ->\n    Bin1 = ?RT:client_key_to_binary(Key1),\n    Bin2 = ?RT:client_key_to_binary(Key2),\n    ?implies(Key1 =/= Key2, Bin1 =/= Bin2).\n\ntester_client_key_to_binary(_Config) ->\n    tester:test(?MODULE, prop_client_key_to_binary, 2, 50000, [{threads, 2}]).\n\n-spec prop_hash_key(Key::client_key()) -> true.\nprop_hash_key(Key) ->\n    % only verify that no exception is thrown during hashing\n    ?RT:hash_key(Key),\n    true.\n\ntester_hash_key(_Config) ->\n    tester:test(?MODULE, prop_hash_key, 1, 100000, [{threads, 2}]).\n\n-spec number_to_key(N::non_neg_integer()) -> ?RT:key().\nnumber_to_key(N) -> call_helper_fun(number_to_key, [N]).\n\nnext_hop(_Config) ->\n    MyNode = node:new(comm:make_global(self()), number_to_key(0), 0),\n    pid_groups:join_as(?MODULE, dht_node),\n    Succ = node:new(fake_dht_node('.succ'), number_to_key(1), 0),\n    Pred = node:new(fake_dht_node('.pred'), number_to_key(1000000), 0),\n    Neighbors = nodelist:new_neighborhood(Pred, MyNode, Succ),\n    DHTNodes = [fake_dht_node(X) || X <- lists:seq(1, 6)],\n    RT_Keys = [{1, 1}, {2, 2}, {4, 3}, {8, 4}, {16, 5}, {32, 6}, {64, 7}],\n    RT = call_helper_fun(create_rt, [RT_Keys, [node:pidX(Succ) | DHTNodes], Neighbors]),\n    RMState = rm_loop:unittest_create_state(Neighbors, false),\n    % note: dht_node_state:new/3 will call pid_groups:get_my(paxos_proposer)\n    % which will fail here -> however, we don't need this process\n    DB = db_dht:new(db_dht),\n    State = dht_node_state:new(RT, RMState, DB),\n\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 0, node:pidX(Pred)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 1, succ]), % succ is responsible\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 2, node:pidX(Succ)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 3, lists:nth(1, DHTNodes)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 7, lists:nth(2, DHTNodes)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 9, lists:nth(3, DHTNodes)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 31, lists:nth(4, DHTNodes)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 64, lists:nth(5, DHTNodes)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 65, lists:nth(6, DHTNodes)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 1000, lists:nth(6, DHTNodes)]),\n\n    [exit(comm:make_local(Node), kill) || Node <- DHTNodes],\n%%     exit(node:pidX(MyNode), kill),\n    exit(comm:make_local(node:pidX(Succ)), kill),\n    exit(comm:make_local(node:pidX(Pred)), kill),\n    db_dht:close(DB),\n    ok.\n\nnext_hop2(_Config) ->\n    MyNode = node:new(comm:make_global(self()), number_to_key(0), 0),\n    pid_groups:join_as(?MODULE, dht_node),\n    Succ = node:new(fake_dht_node('.succ'), number_to_key(1), 0),\n    SuccSucc = node:new(fake_dht_node('.succ.succ'), number_to_key(2), 0),\n    Pred = node:new(fake_dht_node('.pred'), number_to_key(1000000), 0),\n    DHTNodes = [fake_dht_node(X) || X <- lists:seq(1, 6)],\n    RT_Keys = [{1, 1}, {4, 3}, {8, 4}, {16, 5}, {32, 6}, {64, 7}],\n    Neighbors = nodelist:add_node(nodelist:new_neighborhood(Pred, MyNode, Succ),\n                                  SuccSucc, 2, 2),\n    RT = call_helper_fun(create_rt, [RT_Keys, [node:pidX(Succ) | DHTNodes], Neighbors]),\n    %% log:pal(\"RT:~n~190.2p\", [gb_trees:to_list(RT)]),\n    RMState = rm_loop:unittest_create_state(Neighbors, false),\n    % note: dht_node_state:new/3 will call pid_groups:get_my(paxos_proposer)\n    % which will fail here -> however, we don't need this process\n    DB = db_dht:new(db_dht),\n    State = dht_node_state:new(RT, RMState, DB),\n\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 0, node:pidX(Pred)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 1, succ]), % succ is responsible\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 2, node:pidX(Succ)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 3, node:pidX(SuccSucc)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 7, lists:nth(2, DHTNodes)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 9, lists:nth(3, DHTNodes)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 31, lists:nth(4, DHTNodes)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 64, lists:nth(5, DHTNodes)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 65, lists:nth(6, DHTNodes)]),\n    call_helper_fun(check_next_hop, [State, node:pidX(Succ), 1000, lists:nth(6, DHTNodes)]),\n\n    [exit(comm:make_local(GPid), kill) || GPid <- DHTNodes],\n%%     exit(node:pidX(MyNode), kill),\n    exit(comm:make_local(node:pidX(Succ)), kill),\n    exit(comm:make_local(node:pidX(SuccSucc)), kill),\n    exit(comm:make_local(node:pidX(Pred)), kill),\n    db_dht:close(DB),\n    ok.\n\n-spec prop_get_split_key_half(Begin::?RT:key(), End::?RT:key() | plus_infinity) -> true.\nprop_get_split_key_half(Begin, End_) ->\n    End = case End_ of\n              plus_infinity -> ?PLUS_INFINITY;\n              _             -> End_\n          end,\n    SplitKey = ?RT:get_split_key(Begin, End, {1, 2}),\n\n    I = case Begin =:= End of\n            true -> intervals:all(); % full range\n            _    -> intervals:new('[', Begin, End, ')')\n\n        end,\n    ?equals_w_note(intervals:in(SplitKey, I), true,\n                   {\"SplitKey\", SplitKey}),\n\n    call_helper_fun(check_split_key_half, [Begin, End, SplitKey]).\n\ntester_get_split_key_half(_Config) ->\n    prop_get_split_key_half(?MINUS_INFINITY, plus_infinity),\n    tester:test(?MODULE, prop_get_split_key_half, 2, 10000, [{threads, 2}]).\n\n-spec prop_get_split_key(Begin::?RT:key(), End::?RT:key() | plus_infinity, SplitFracA::1..100, SplitFracB::0..100) -> true.\nprop_get_split_key(Begin, End_, SplitFracA, SplitFracB) ->\n    End = case End_ of\n              plus_infinity -> ?PLUS_INFINITY;\n              _             -> End_\n          end,\n    SplitFraction = case SplitFracA =< SplitFracB of\n                        true -> {SplitFracA, SplitFracB};\n                        _    -> {SplitFracB, SplitFracA}\n                    end,\n\n%%     ct:pal(\"Begin: ~.0p, End: ~.0p, SplitFactor: ~.0p\", [Begin, End, SplitFraction]),\n    SplitKey = ?RT:get_split_key(Begin, End, SplitFraction),\n\n    case erlang:element(1, SplitFraction) =:= 0 of\n        true -> ?equals(SplitKey, Begin);\n        _ ->\n            I = case Begin =:= End of\n                    true -> intervals:all(); % full range\n                    _ when SplitFracA =:= SplitFracB andalso End =:= ?PLUS_INFINITY ->\n                        intervals:all(); % hack - there is no interval including ?PLUS_INFINITY but intervals says 'yes' checking for element if the intervall is 'all'\n                    _ when SplitFracA =:= SplitFracB ->\n                        intervals:new('[', Begin, End, ']');\n                    _    -> intervals:new('[', Begin, End, ')')\n\n                end,\n            ?equals_w_note(intervals:in(SplitKey, I), true,\n                           {\"SplitKey\", SplitKey}),\n            call_helper_fun(check_split_key, [Begin, End, SplitKey, SplitFraction])\n    end,\n    true.\n\ntester_get_split_key(_Config) ->\n    tester:test(?MODULE, prop_get_split_key, 4, 10000, [{threads, 2}]).\n\nadditional_tests(Config) ->\n    call_helper_fun(additional_tests, [Config]).\n\n%% helpers\n\n-spec fake_dht_node(Suffix::atom()|integer()) -> comm:mypid().\nfake_dht_node(Suffix) ->\n    comm:make_global(\n      element(1, unittest_helper:start_subprocess(\n                fun() -> pid_groups:join_as({?MODULE, Suffix}, dht_node) end))).\n\n-spec call_helper_fun(Fun::atom(), Args::list()) -> term().\ncall_helper_fun(Fun, Args) ->\n    erlang:apply(erlang:list_to_atom(\"rt_SUITE_\" ++ erlang:atom_to_list(?RT)), Fun, Args).\n"
  },
  {
    "path": "test/rt_SUITE_rt_chord.erl",
    "content": "% @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Helper for rt_chord unit tests.\n%% @end\n%% @version $Id$\n-module(rt_SUITE_rt_chord).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\nnumber_to_key(N) -> N.\n\ncreate_rt(RT_Keys, [_Succ | _DHTNodes] = Nodes, Neighbors) ->\n    RT = gb_trees:from_orddict(\n           [begin\n                Key = number_to_key(N),\n                {Key, lists:nth(Idx, Nodes)}\n            end || {N, Idx} <- RT_Keys]),\n    Neighborlist = tl(nodelist:to_list(Neighbors)),\n    lists:foldl(fun(Node, AccIn) ->\n                        gb_trees:enter(node:id(Node), node:pidX(Node), AccIn)\n                end, RT, Neighborlist).\n\ncheck_next_hop(State, _Succ, N, NodeExp) ->\n    Neighbors = dht_node_state:get(State, neighbors),\n    ERT = dht_node_state:get(State, rt),\n    Node = ?RT:next_hop(Neighbors, ERT, number_to_key(N)),\n    ?equals_w_note(Node, NodeExp, io_lib:format(\"~B\", [N])).\n\n-spec check_split_key_half(Begin::?RT:key(), End::?RT:key() | ?PLUS_INFINITY_TYPE, SplitKey::?RT:key()) -> true.\ncheck_split_key_half(Begin, End, SplitKey) ->\n    BeginToSplitKey = ?RT:get_range(Begin, SplitKey),\n    SplitKeyToEnd = ?RT:get_range(SplitKey, End),\n    ?equals_pattern_w_note(\n        BeginToSplitKey,\n        Result when Result == SplitKeyToEnd orelse Result == (SplitKeyToEnd - 1),\n        io_lib:format(\"SplitKey: ~.0p\", [SplitKey])).\n\n-spec check_split_key(Begin::?RT:key(), End::?RT:key() | ?PLUS_INFINITY_TYPE, SplitKey::?RT:key(), {SplitFracA::1..100, SplitFracB::0..100}) -> true.\ncheck_split_key(Begin, End, SplitKey, SplitFraction) ->\n    FullRange = ?RT:get_range(Begin, End),\n%%     ct:pal(\"FullRange: ~.0p\", [FullRange]),\n    BeginToSplitKey = case Begin of\n                          SplitKey -> 0;\n                          _ -> ?RT:get_range(Begin, SplitKey)\n                      end,\n    %%     ct:pal(\"BeginToSplitKeyRange: ~.0p, ~.0p\", [BeginToSplitKey, SplitKey]),\n\n    ?equals_pattern_w_note(\n        BeginToSplitKey,\n        Range when Range == (FullRange * erlang:element(1, SplitFraction)) div erlang:element(2, SplitFraction),\n        io_lib:format(\"FullRange * Factor = ~.0p, SplitKey: ~.0p\",\n                      [(FullRange * erlang:element(1, SplitFraction)) div erlang:element(2, SplitFraction), SplitKey])).\n\nadditional_tests(_Config) ->\n    ok.\n"
  },
  {
    "path": "test/rt_SUITE_rt_frt.erl",
    "content": "% @copyright 2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Magnus Mueller <mamuelle@informatik.hu-berlin.de>\n%% @doc    Helper for rt_frt unit tests.\n%% @end\n%% @version $Id$\n-module(rt_SUITE_rt_frt).\n-author('mamuelle@informatik.hu-berlin.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\nnumber_to_key(N) -> N.\n\ncreate_rt(RT_Keys, [_Succ | _DHTNodes] = Nodes, Neighbors) ->\n    RT1 = gb_trees:from_orddict(\n            [begin\n                 Key = number_to_key(N),\n                 {Key, {Key, 0, lists:nth(Idx, Nodes), none}}\n             end || {N, Idx} <- RT_Keys]),\n    RT2 = lists:foldl( fun(Node, AccIn) -> Id = node:id(Node),\n                                           gb_trees:enter(Id, {Id, node:id_version(Node), node:pidX(Node), none}, AccIn)\n                       end, RT1, nodelist:succs(Neighbors) ++ nodelist:preds(Neighbors)),\n    {length(Nodes), RT2}.\n\ncheck_next_hop(State, _Succ, N, NodeExp) ->\n    Neighbors = dht_node_state:get(State, neighbors),\n    ERT = dht_node_state:get(State, rt),\n    ?equals_w_note(?RT:next_hop(Neighbors, ERT, number_to_key(N)), NodeExp, io_lib:format(\"~B\", [N])).\n\n-spec check_split_key_half(Begin::?RT:key(), End::?RT:key() | ?PLUS_INFINITY_TYPE, SplitKey::?RT:key()) -> true.\ncheck_split_key_half(Begin, End, SplitKey) ->\n    BeginToSplitKey = ?RT:get_range(Begin, SplitKey),\n    SplitKeyToEnd = ?RT:get_range(SplitKey, End),\n    ?equals_pattern_w_note(\n        BeginToSplitKey,\n        Result when Result == SplitKeyToEnd orelse Result == (SplitKeyToEnd - 1),\n        io_lib:format(\"SplitKey: ~.0p\", [SplitKey])).\n\n-spec check_split_key(Begin::?RT:key(), End::?RT:key() | ?PLUS_INFINITY_TYPE, SplitKey::?RT:key(), {SplitFracA::1..100, SplitFracB::0..100}) -> true.\ncheck_split_key(Begin, End, SplitKey, SplitFraction) ->\n    FullRange = ?RT:get_range(Begin, End),\n%%     ct:pal(\"FullRange: ~.0p\", [FullRange]),\n    BeginToSplitKey = case Begin of\n                          SplitKey -> 0;\n                          _ -> ?RT:get_range(Begin, SplitKey)\n                      end,\n    %%     ct:pal(\"BeginToSplitKeyRange: ~.0p, ~.0p\", [BeginToSplitKey, SplitKey]),\n\n    ?equals_pattern_w_note(\n        BeginToSplitKey,\n        Range when Range == (FullRange * erlang:element(1, SplitFraction)) div erlang:element(2, SplitFraction),\n        io_lib:format(\"FullRange * Factor = ~.0p, SplitKey: ~.0p\",\n                      [(FullRange * erlang:element(1, SplitFraction)) div erlang:element(2, SplitFraction), SplitKey])).\n\nadditional_tests(_Config) ->\n    tester:test(rt_frt, get_random_key_from_generator, 3, 2000, [{threads, 2}]) .\n"
  },
  {
    "path": "test/rt_SUITE_rt_simple.erl",
    "content": "% @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Helper for rt_chord unit tests.\n%% @end\n%% @version $Id$\n-module(rt_SUITE_rt_simple).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\nnumber_to_key(N) -> N.\n\ncreate_rt(_RT_Keys, [Succ | _DHTNodes] = _Nodes, _Neighbors) ->\n    node:new(Succ, number_to_key(1), 0).\n\ncheck_next_hop(State, Succ, N, _NodeExp) ->\n    Neighbors = dht_node_state:get(State, neighbors),\n    ERT = dht_node_state:get(State, rt),\n    Exp = case intervals:in(number_to_key(N), nodelist:succ_range(Neighbors)) of\n              true -> succ;\n              false -> Succ\n          end,\n    ?equals_w_note(?RT:next_hop(Neighbors, ERT, number_to_key(N)), Exp, io_lib:format(\"~B\", [N])).\n\n-spec check_split_key_half(Begin::?RT:key(), End::?RT:key() | ?PLUS_INFINITY_TYPE, SplitKey::?RT:key()) -> true.\ncheck_split_key_half(Begin, End, SplitKey) ->\n    rt_SUITE_rt_chord:check_split_key_half(Begin, End, SplitKey).\n\n-spec check_split_key(Begin::?RT:key(), End::?RT:key() | ?PLUS_INFINITY_TYPE, SplitKey::?RT:key(), {SplitFracA::1..100, SplitFracB::0..100}) -> true.\ncheck_split_key(Begin, End, SplitKey, SplitFraction) ->\n    rt_SUITE_rt_chord:check_split_key(Begin, End, SplitKey, SplitFraction).\n\nadditional_tests(_Config) ->\n    ok.\n"
  },
  {
    "path": "test/runner.erl",
    "content": "% @copyright 2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc common test runner\n-module(runner).\n-author('schuett@zib.de').\n\n-export([run_spec/1, run_suite/1, run_suite_group/2, run_suite_case/2]).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% API functions\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec run_spec(string()) -> term().\nrun_spec(Spec) ->\n    %% ct:pal(\"~s~n\", [Spec]),\n    ct:run_test([\n                 {spec, [Spec]},\n                 {ct_hooks, [scalaris_cth]}\n                ]\n               ).\n\n-spec run_suite(string()) -> term().\nrun_suite(Suite) ->\n    %% ct:pal(\"~s~n\", [Suite]),\n    ct:run_test([\n                 {suite, [Suite]},\n                 {ct_hooks, [scalaris_cth]}\n                ]\n               ).\n\n-spec run_suite_group(string(), string()) -> term().\nrun_suite_group(Suite, Group) ->\n    %% ct:pal(\"~s:~s~n\", [Suite, Group]),\n    ct:run_test([\n                 {suite, [Suite]},\n                 {group, [Group]},\n                 {ct_hooks, [scalaris_cth]}\n                ]\n               ).\n\n-spec run_suite_case(string(), string()) -> term().\nrun_suite_case(Suite, Case) ->\n    %% ct:pal(\"~s:~s~n\", [Suite, Case]),\n    ct:run_test([\n                 {suite, [Suite]},\n                 {testcase, [Case]},\n                 {ct_hooks, [scalaris_cth]}\n                ]\n               ).\n\n"
  },
  {
    "path": "test/scalaris.coverspec",
    "content": "{level, details}.\n\n{incl_dirs_r, [\"ebin\"]}.\n"
  },
  {
    "path": "test/scalaris_cth.erl",
    "content": "%  @copyright 2011-2013 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Common test hooks for Scalaris unit tests.\n%% @end\n%% @version $Id$\n-module(scalaris_cth).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-include(\"types.hrl\").\n\n%% Callbacks\n-export([id/1, init/2, terminate/1]).\n-export([pre_init_per_suite/3, post_init_per_suite/4,\n         pre_end_per_suite/3, post_end_per_suite/4]).\n-export([pre_init_per_testcase/3, post_end_per_testcase/4]).\n-export([on_tc_fail/3, on_tc_skip/3]).\n\n-record(state, { processes = [] :: [unittest_helper:process_info()] | undefined,\n                 suite          :: atom() | undefined,\n                 logger_pid     :: pid() | undefined,\n                 tc_start = []  :: [{comm:erl_local_pid(), erlang_timestamp()}] | []\n               }).\n-type state() :: #state{} | {ok, #state{}}. % the latter for erlang =< R14B03\n-type id() :: atom().\n\n%% @doc Return a unique id for this CTH.\n-spec id(Opts::term()) -> id().\nid(_Opts) ->\n    ?MODULE.\n\n%% @doc Always called before any other callback function. Use this to initiate\n%% any common state.\n-spec init(Id::id(), Opts::term()) -> {ok, State::state()}.\ninit(_Id, _Opts) ->\n    % note: erlang =< R14B03 requires only #state{}\n    % -> provide alternative handlers for all functions below catching the\n    % additional {ok, state()} tuple\n    {ok, #state{}}.\n\n%% @doc Called before init_per_suite is called.\n\n-spec pre_init_per_suite(SuiteName::atom(), Config::unittest_helper:kv_opts(), CTHState::state())\n        -> {Return::unittest_helper:kv_opts() | {fail, Reason::term()} | {skip, Reason::term()},\n            NewCTHState::state()}.\npre_init_per_suite(Suite, Config, {ok, State}) ->\n    pre_init_per_suite(Suite, Config, State);\npre_init_per_suite(Suite, Config, State) when is_record(State, state) ->\n    Processes = unittest_helper:get_processes(),\n    set_config_file_paths(),\n    % initiate the logging process (which needs the config)\n    {TempPid, _} = unittest_helper:start_process(\n          fun() ->\n                  {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n                  ConfOptions = unittest_helper:prepare_config(\n                                  [{config, [{log_path, PrivDir}]}]),\n                  config:init(ConfOptions)\n          end),\n    % need to start the logger from a non-linked process so that it is\n    % available until the end of the suite\n    {LoggerPid, _} = unittest_helper:start_process(\n                       fun() -> {ok, _LogPid} = log:start_link(),\n                                exit(TempPid, kill)\n                       end),\n    ct:pal(\"Starting unittest ~p~n\", [ct:get_status()]),\n    randoms:start(),\n    % note: on R14B02, post_end_per_testcase was also called for init_per_suite\n    % -> we thus need a valid time\n    {Config, State#state{processes = Processes, suite = Suite,\n                         logger_pid = LoggerPid,\n                         tc_start = [{self(), os:timestamp()}]}}.\n\n%% @doc Called after init_per_suite.\n-spec post_init_per_suite(SuiteName::atom(), Config::unittest_helper:kv_opts(), Return, CTHState::state())\n        -> {Return, NewCTHState::state()}\n    when is_subtype(Return, unittest_helper:kv_opts() | {fail, Reason::term()} | {skip, Reason::term()} | term()).\npost_init_per_suite(Suite, Config, Return, {ok, State}) ->\n    post_init_per_suite(Suite, Config, Return, State);\npost_init_per_suite(_Suite, _Config, Return, State) when is_record(State, state) ->\n    {Return, State}.\n\n%% @doc Called before end_per_suite.\n-spec pre_end_per_suite(SuiteName::atom(), Config::unittest_helper:kv_opts(), CTHState::state())\n        -> {Return::unittest_helper:kv_opts() | {fail, Reason::term()} | {skip, Reason::term()},\n            NewCTHState::state()}.\npre_end_per_suite(_Suite, Config, {ok, State}) ->\n    pre_end_per_suite(_Suite, Config, State);\npre_end_per_suite(_Suite, Config, State) when is_record(State, state) ->\n    {Config, State}.\n\n%% @doc Called after end_per_suite.\n-spec post_end_per_suite(SuiteName::atom(), Config::unittest_helper:kv_opts(), Return, CTHState::state())\n        -> {Return, NewCTHState::state()}\n    when is_subtype(Return, unittest_helper:kv_opts() | {fail, Reason::term()} | {skip, Reason::term()} | term()).\npost_end_per_suite(Suite, Config, Return, {ok, State}) ->\n    post_end_per_suite(Suite, Config, Return, State);\npost_end_per_suite(_Suite, _Config, Return, State) when is_record(State, state) ->\n    ct:pal(\"Stopping unittest ~p~n\", [ct:get_status()]),\n    unittest_helper:stop_ring(),\n    % the following might still be running in case there was no ring:\n    error_logger:delete_report_handler(error_logger_log4erl_h),\n    log:set_log_level(none),\n    exit(State#state.logger_pid, kill),\n    randoms:stop(),\n    _ = inets:stop(),\n    unittest_global_state:delete(),\n    unittest_helper:kill_new_processes(State#state.processes),\n    {Return, State#state{processes = [], suite = undefined, tc_start = [] } }.\n\n%% @doc Called before each test case.\n-spec pre_init_per_testcase(TestcaseName::atom(), Config::unittest_helper:kv_opts(), CTHState::state())\n        -> {Return::unittest_helper:kv_opts() | {fail, Reason::term()} | {skip, Reason::term()},\n            NewCTHState::state()}.\npre_init_per_testcase(TC, Config, {ok, State}) ->\n    pre_init_per_testcase(TC, Config, State);\npre_init_per_testcase(TC, Config, State) when is_record(State, state) ->\n    ct:pal(\"Start ~p:~p~n\"\n           \"####################################################\",\n           [State#state.suite, TC]),\n    {Config, State#state{tc_start = [{self(), os:timestamp()} | State#state.tc_start]}}.\n\n%% @doc Called after each test case.\n-spec post_end_per_testcase(TestcaseName::atom(), Config::unittest_helper:kv_opts(), Return, CTHState::state())\n        -> {Return, NewCTHState::state()}\n    when is_subtype(Return, unittest_helper:kv_opts() | {fail, Reason::term()} |\n                        {error, Reason::term()} | {testcase_aborted, Reason::term()} |\n                        {skip, Reason::term()} | {timetrap_timeout, integer()}).\npost_end_per_testcase(TC, Config, Return, {ok, State}) ->\n    post_end_per_testcase(TC, Config, Return, State);\npost_end_per_testcase(TC, Config, Return, State) when is_record(State, state) ->\n    case Return of\n        {timetrap_timeout, TimeTrapTime_ms} ->\n            print_debug_data(),\n            Suite = State#state.suite,\n            try Suite:end_per_testcase(TC, Config)\n            catch\n                exit:undef ->\n                    ok;\n                error:undef ->\n                    ok;\n                Error:Reason ->\n                    ct:pal(\"Caught ~p:~p while trying to clean up Ring after\n                           timeout~n\", [Error, Reason])\n            end,\n            ok;\n        {skip, {failed, {_FailedMod, _FailedFun, {timetrap_timeout, TimeTrapTime_ms}}}} ->\n            ok;\n        {fail, _Reason} ->\n            TimeTrapTime_ms = 0,\n            print_debug_data();\n        {error, _} ->\n            % from thrown exceptions or ct:fail/*\n            TimeTrapTime_ms = 0,\n            print_debug_data();\n        {testcase_aborted, _} ->\n            % e.g. from an exception in a gen_component (Reason = exception_throw)\n            TimeTrapTime_ms = 0,\n            print_debug_data();\n        _ ->\n            TimeTrapTime_ms = 0,\n            ok\n    end,\n    case proplists:get_value(stop_ring, Config, false) of\n        true  -> unittest_helper:stop_ring();\n        false -> ok\n    end,\n    {Start, NewTcStart} =  case lists:keytake(self(), 1, State#state.tc_start) of\n                               {value, {_, Val}, List} -> {Val, List};\n                               false -> { failed, State#state.tc_start }\n                           end,\n    TCTime_ms = case Start of\n                    failed -> TimeTrapTime_ms;\n                    _      -> timer:now_diff(os:timestamp(), Start) / 1000\n                end,\n    ct:pal(\"####################################################~n\"\n           \"End ~p:~p -> ~.0p (after ~fs)\",\n           [State#state.suite, TC, Return, TCTime_ms / 1000]),\n    {Return, State#state{tc_start = NewTcStart} }.\n\n-spec print_debug_data() -> ok.\nprint_debug_data() ->\n    case (catch config:read(no_print_ring_data)) of\n        true -> ok;\n        _ -> unittest_helper:print_ring_data()\n    end,\n    try proto_sched:get_infos() of\n        [] ->\n            % proto_sched or its stats do not exist anymore -> print collected stats\n            case unittest_global_state:lookup(proto_sched_stats) of\n                failed -> ok;\n                Stats -> ct:pal(\"Proto scheduler stats (collected): ~.2p\",\n                                [Stats])\n            end;\n        Stats ->\n            ct:pal(\"Proto scheduler stats: ~.2p\", [Stats])\n    catch _:_ ->\n              % pid_groups not available (anymore)? -> check whether we recorded some:\n              case unittest_global_state:lookup(proto_sched_stats) of\n                  failed -> ok;\n                  Stats -> ct:pal(\"Proto scheduler stats (collected): ~.2p\",\n                                  [Stats])\n              end\n    end,\n    catch tester_global_state:log_last_calls(),\n    ok.\n\n%% @doc Called after post_init_per_suite, post_end_per_suite, post_init_per_group,\n%% post_end_per_group and post_end_per_testcase if the suite, group or test case failed.\n-spec on_tc_fail(TestcaseName::init_per_suite | end_per_suite | init_per_group | end_per_group | atom(),\n                 Reason::{error, FailInfo} |\n                         {error, {RunTimeError, StackTrace}} |\n                         {timetrap_timeout, integer()} |\n                         {failed, {Suite::atom(), end_per_testcase, FailInfo}},\n                 CTHState::state()) -> NewCTHState::#state{}\n    when is_subtype(FailInfo, {timetrap_timeout, integer()} | {RunTimeError, StackTrace} | UserTerm),\n         is_subtype(RunTimeError, term()),\n         is_subtype(StackTrace, list()),\n         is_subtype(UserTerm, term()).\non_tc_fail(TC, Reason, {ok, State}) ->\n    on_tc_fail(TC, Reason, State);\non_tc_fail(TC, Reason, State) when is_record(State, state) ->\n    ct:pal(\"~p:~p failed with ~p.~n\", [State#state.suite, TC, Reason]),\n    State.\n\n%% @doc Called when a test case is skipped by either user action\n%% or due to an init function failing.\n-spec on_tc_skip(TestcaseName::end_per_suite | init_per_group | end_per_group | atom(),\n                 Reason::{tc_auto_skip, {Suite, Func::atom(), Reason}} |\n                         {tc_user_skip, {Suite, TestCase::atom(), Comment::string()}},\n                 CTHState::state()) -> NewCTHState::#state{}\n    when is_subtype(Suite, atom()),\n         is_subtype(Reason, {failed, FailReason} | {require_failed_in_suite0, RequireInfo}),\n         is_subtype(FailReason, {Suite, ConfigFunc, FailInfo} | {Suite, FailedCaseInSequence}),\n         is_subtype(RequireInfo, {not_available, atom()}),\n         is_subtype(ConfigFunc, init_per_suite | init_per_group),\n         is_subtype(FailInfo, {timetrap_timeout,integer()} | {RunTimeError, StackTrace} | bad_return | UserTerm),\n         is_subtype(FailedCaseInSequence, atom()),\n         is_subtype(RunTimeError, term()),\n         is_subtype(StackTrace, list()),\n         is_subtype(UserTerm, term()).\non_tc_skip(TC, Reason, {ok, State}) ->\n    on_tc_skip(TC, Reason, State);\non_tc_skip(TC, Reason, State) when is_record(State, state) ->\n    ct:pal(\"~p:~p skipped with ~p.~n\", [State#state.suite, TC, Reason]),\n    State.\n\n%% @doc Called when the scope of the CTH is done\n-spec terminate(State::state()) -> ok.\nterminate({ok, State}) ->\n    terminate(State);\nterminate(State) when is_record(State, state) ->\n    ok.\n\n%% @doc Sets the app environment to point to the correct config file paths\n%%      assuming that the current working directory is a sub-dir of\n%%      our top-level, e.g. \"test\". This is needed in order for the config\n%%      process to find its (default) config files.\n-spec set_config_file_paths() -> ok.\nset_config_file_paths() ->\n    application:set_env(scalaris, config, \"../bin/scalaris.cfg\"),\n    application:set_env(scalaris, local_config, \"../bin/scalaris.local.cfg\").\n"
  },
  {
    "path": "test/scalaris_cth_SUITE.erl",
    "content": "% @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Unit tests for the scalaris_cth module.\n%% @end\n%% @version $Id$\n-module(scalaris_cth_SUITE).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\nall() ->\n    [test_timeout].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 1}},\n     {ct_hooks, [scalaris_cth]}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\n    \ntest_timeout(_Config) ->\n    unittest_helper:make_ring(4),\n    timer:sleep(2000).\n"
  },
  {
    "path": "test/skipped_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{suites, tests, [proto_sched_SUITE]}.\n{suites, tests, [api_tx_proto_sched_SUITE]}.\n{suites, tests, [dht_node_move_proto_sched_SUITE]}.\n{suites, tests, [join_leave_proto_sched_SUITE]}.\n{suites, tests, [mr_proto_sched_SUITE]}.\n{suites, tests, [rrepair_proto_sched_SUITE]}.\n{suites, tests, [memtest_SUITE]}.\n{suites, tests, [type_check_SUITE]}.\n\n{cases, tests, l_on_cseq_SUITE, [tester_type_check_l_on_cseq]}.\n{cases, tests, prbr_SUITE, [tester_type_check_rbr]}.\n{cases, tests, rm_leases_SUITE, [tester_type_check_rm_leases]}.\n{cases, tests, slide_leases_SUITE, [tester_type_check_slide_leases]}.\n"
  },
  {
    "path": "test/slide_leases_SUITE.erl",
    "content": "%% @copyright 2012-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for slide_leases\n%% @end\n%% @version $Id$\n-module(slide_leases_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\ngroups() ->\n    [{tester_tests, [sequence], [\n                                 tester_type_check_slide_leases\n                              ]},\n     {join_tests, [sequence], [\n                               test_single_join,\n                               test_double_join,\n                               test_triple_join,\n                               test_quadruple_join\n                               ]},\n     {join_and_leave_tests, [sequence], [\n                                         test_quadruple_join_single_leave%,\n                                         %%  the following tests can\n                                         %%  run into timeouts if the\n                                         %%  majority of replicas of a\n                                         %%  lease have left\n                                         %%  %test_quadruple_join_double_leave,\n                                         %%  %test_quadruple_join_triple_leave,\n                                         %%  %test_quadruple_join_quadruple_leave\n                                         ]},\n     {repeater, [{repeat, 30}], [{group, join_tests}]}\n    ].\n\nall() ->\n    [\n     {group, tester_tests},\n     {group, join_tests},\n     {group, join_and_leave_tests}%,\n     %% {group, repeater}\n     ].\n\nsuite() -> [ {timetrap, {seconds, 300}} ].\n\ngroup(tester_tests) ->\n    [{timetrap, {seconds, 400}}];\ngroup(join_tests) ->\n    [{timetrap, {seconds, 10}}];\ngroup(join_and_leave_tests) ->\n    [{timetrap, {seconds, 30}}];\ngroup(_) ->\n    suite().\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(Group, Config) -> unittest_helper:init_per_group(Group, Config).\n\nend_per_group(Group, Config) -> unittest_helper:end_per_group(Group, Config).\n\ninit_per_testcase(_TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(1, [{config, [{log_path, PrivDir},\n                                            {leases, true}]}]),\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\ntester_type_check_slide_leases(_Config) ->\n    Count = 500,\n    config:write(no_print_ring_data, true),\n    tester:register_value_creator({typedef, prbr, write_filter, []},\n                                  prbr, tester_create_write_filter, 1),\n    %% [{modulename, [excludelist = {fun, arity}]}]\n    Modules =\n        [ {slide_leases,\n           [\n            {prepare_join_send, 2},\n            {prepare_rcv_data, 2},\n            {prepare_send_data1, 3},\n            {prepare_send_data2, 3},\n            {update_rcv_data1, 3},\n            {update_rcv_data2, 3},\n            {prepare_send_delta1, 3},\n            {prepare_send_delta2, 3},\n            {finish_delta1, 3},\n            {finish_delta2, 3},\n            {finish_delta_ack1, 3},\n            {finish_delta_ack2, 4}\n           ],\n           [\n            {send_continue_msg, 1},\n            {find_lease, 3}\n           ]}\n        ],\n    %% join a dht_node group to be able to call lease trigger functions\n    pid_groups:join(pid_groups:group_with(dht_node)),\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n\n    tester:unregister_value_creator({typedef, prbr, write_filter, []}),\n    %% tester:unregister_value_creator( TODO ),\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% join unit tests\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n\ntest_single_join(_Config) ->\n    join_test(2).\n\ntest_double_join(_Config) ->\n    join_test(3).\n\ntest_triple_join(_Config) ->\n    join_test(4).\n\ntest_quadruple_join(_Config) ->\n    join_test(5).\n\ntest_single_join_single_leave(_Config) ->\n    join_leave_test(2, 1).\n\ntest_quadruple_join_single_leave(_Config) ->\n    join_leave_test(5, 4).\n\ntest_quadruple_join_double_leave(_Config) ->\n    join_leave_test(5, 3).\n\ntest_quadruple_join_triple_leave(_Config) ->\n    join_leave_test(5, 2).\n\ntest_quadruple_join_quadruple_leave(_Config) ->\n    join_leave_test(5, 1).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% join helper\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\njoin_test(TargetSize) ->\n    lease_helper:wait_for_ring_size(1),\n    lease_helper:wait_for_correct_ring(),\n    join_until(TargetSize),\n    true.\n\njoin_until(TargetSize) ->\n    joiner_helper(1, TargetSize).\n\njoiner_helper(Target, Target) ->\n    ok;\njoiner_helper(Current, Target) ->\n    synchronous_join(Current+1),\n    joiner_helper(Current+1, Target).\n\nsynchronous_join(TargetSize) ->\n    %% AllDHTNodes = pid_groups:find_all(dht_node),\n    %% _ = add_bp_on_successful_split(),\n    %% _ = add_bp_on_successful_handover(),\n    _ = api_vm:add_nodes(1),\n    %% log:log(\"wait for successful split\"),\n    %% wait_for_successful_split(),\n    %% _ = delete_bp_on_successful_split(AllDHTNodes),\n    %% log:log(\"wait for successful handover\"),\n    %% wait_for_successful_handover(),\n    %% _ = delete_bp_on_successful_handover(AllDHTNodes),\n    log:log(\"wait for ring size ~w\", [TargetSize]),\n    lease_helper:wait_for_ring_size(TargetSize),\n    log:log(\"wait for ring to stabilize in sync. join\"),\n    lease_checker2:wait_for_clean_leases(200, [{ring_size, TargetSize}]).\n    %% util:wait_for(fun admin:check_leases/0, 10000).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% leave helper\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\njoin_leave_test(JoinTargetSize, LeaveTargetSize) ->\n    lease_helper:wait_for_ring_size(1),\n    log:log(\"wait for ring to stabilize\"),\n    util:wait_for(fun admin:check_leases/0, 10000),\n    join_until(JoinTargetSize),\n    log:log(\"wait for ring to stabilize after join until\"),\n    util:wait_for(fun admin:check_leases/0, 10000),\n    lease_helper:print_all_active_leases(),\n    lease_helper:print_all_passive_leases(),\n    leave_until(JoinTargetSize, LeaveTargetSize),\n    true.\n\nleave_until(TargetSize, TargetSize) ->\n    ok;\nleave_until(CurrentSize, TargetSize) ->\n    Node = lease_checker:get_random_save_node(),\n    case Node of\n        failed ->\n            ok;\n        _ ->\n            Group = pid_groups:group_of(comm:make_local(Node)),\n            ct:pal(\"shuting down node: ~p ~w\", [Group, Node]),\n            ok = api_vm:kill_node(Group),\n            lease_helper:wait_for_ring_size(CurrentSize - 1),\n            ct:pal(\"wait for ring to stabilize after shutdown\"),\n            util:wait_for(fun admin:check_leases/0, 10000),\n            ct:pal(\"shuting down node: success\"),\n            leave_until(CurrentSize - 1, TargetSize)\n    end.\n\n%% listen_for_split(Pid) ->\n%%     fun (Message, _State) ->\n%%             case Message of\n%%                 {l_on_cseq, split_reply_step4, _L1, _R1, _R2, _Keep, _ReplyTo, _PostAux,\n%%                  {qwrite_done, _ReqId, _Round, _L2}} ->\n%%                     comm:send_local(Pid, {successful_split}),\n%%                     false;\n%%                 {l_on_cseq, split_reply_step4, _L1, _R1, _R2, _Keep, _ReplyTo, _PostAux,\n%%                  {qwrite_deny, _ReqId, _Round, _L2, {content_check_failed,\n%%                                                     {Reason, _Current, _Next}}}} ->\n%%                     log:log(\"split failed: ~w\", [Reason]),\n%%                     false;\n%%                 _ ->\n%%                     false\n%%             end\n%%     end.\n%% \n%% listen_for_handover(Pid) ->\n%%     fun (Message, _State) ->\n%%             case Message of\n%%                 {l_on_cseq, handover_reply, {qwrite_done, _ReqId, _Round, _Value},\n%%                  _ReplyTo, _NewOwner, _New} ->\n%%                     comm:send_local(Pid, {successful_handover}),\n%%                     false;\n%%                 {l_on_cseq, handover_reply,\n%%                  {qwrite_deny, _ReqId, _Round, _Value,\n%%                   {content_check_failed, {Reason, _Current, _Next}}},\n%%                  _ReplyTo, _NewOwner, _New} ->\n%%                     log:log(\"handover failed: ~w\", [Reason]),\n%%                     false;\n%%                 _ ->\n%%                     false\n%%             end\n%%     end.\n%% \n%% add_bp_on_successful_split() ->\n%%     [gen_component:bp_set_cond(Node, listen_for_split(self()), listen_split) ||\n%%         Node <- pid_groups:find_all(dht_node)].\n%% \n%% wait_for_successful_split() ->\n%%     receive\n%%         {successful_split} ->\n%%             ok\n%%     end.\n%% \n%% delete_bp_on_successful_split(Nodes) ->\n%%     [gen_component:bp_del(Node, listen_split) || Node <- Nodes].\n%% \n%% add_bp_on_successful_handover() ->\n%%     [gen_component:bp_set_cond(Node, listen_for_handover(self()), listen_handover) ||\n%%         Node <- pid_groups:find_all(dht_node)].\n%% \n%% wait_for_successful_handover() ->\n%%     receive\n%%         {successful_handover} ->\n%%             ok\n%%     end.\n\ndelete_bp_on_successful_handover(Nodes) ->\n    [gen_component:bp_del(Node, listen_handover) || Node <- Nodes].\n"
  },
  {
    "path": "test/snapshot_SUITE.erl",
    "content": "% @copyright 2012-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%%% @author Stefan Keidel <keidel@informatik.hu-berlin.de>\n%%% @doc    Snapshot algorithm unit tests\n%%% @end\n%%% @version $Id$\n-module(snapshot_SUITE).\n-author('keidel@informatik.hu-berlin.de').\n-compile(export_all).\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\n-vsn('$Id$').\n\n%% all() -> [test_tx_snapshot_slide_interleave || _X <- lists:seq(1, 100)].\nall() ->\n    [{group, without_ring},\n     {group, with_ring}\n    ].\n\ngroups() ->\n    [\n     {without_ring, [],\n      [\n       test_copy_value_to_snapshot_table, test_set_get_for_snapshot_table,\n       test_delete_from_snapshot_table, test_init_snapshot, test_delete_snapshot,\n       tester_get_snapshot_data,\n       tester_add_snapshot_data,\n       %% rdht_tx_read\n       test_rdht_tx_read_validate_should_abort, test_rdht_tx_read_validate_should_prepare,\n       test_rdht_tx_read_validate_db_copy, test_rdht_tx_read_commit_with_snap_1,\n       test_rdht_tx_read_commit_with_snap_2, test_rdht_tx_read_commit_without_snap,\n       %% rdht_tx_write\n       test_rdht_tx_write_validate_should_abort, test_rdht_tx_write_validate_should_prepare,\n       test_rdht_tx_write_validate_db_copy, test_rdht_tx_write_commit_with_snap,\n       test_rdht_tx_write_commit_without_snap, test_rdht_tx_write_abort_without_snap,\n       test_rdht_tx_write_abort_with_snap,\n       %% lock counting\n       test_lock_counting_on_live_db]},\n\n     {with_ring, [],\n      [%% integration\n       test_basic_race_multiple_snapshots, test_single_snapshot_call,\n       test_spam_transactions_and_snapshots_on_fully_joined,\n       test_tx_snapshot_slide_interleave,\n       %% misc\n       bench_increment]}\n    ].\n\nsuite() -> [ {timetrap, {seconds, 45}} ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_group(without_ring, Config) ->\n    unittest_helper:start_minimal_procs(Config, [], false);\ninit_per_group(_GroupName, Config) ->\n    Config.\n\nend_per_group(without_ring, Config) ->\n    unittest_helper:stop_minimal_procs(Config);\nend_per_group(_GroupName, Config) ->\n    Config.\n\ninit_per_testcase(_TestCase, Config) ->\n    % for unit tests without a ring, we need to be in a pid_group:\n    case proplists:get_value(name, proplists:get_value(tc_group_properties, Config, []), none) of\n        without_ring -> pid_groups:join_as(ct_tests, ?MODULE);\n        _ -> ok\n    end,\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n-define(KEY(Key), ?RT:hash_key(Key)).\n-define(VALUE(Val), rdht_tx:encode_value(Val)).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% snapshot-related DB API tests\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec test_copy_value_to_snapshot_table(any()) -> ok.\ntest_copy_value_to_snapshot_table(_Config) ->\n    InitDb = db_dht:new(db_dht),\n    Db = db_dht:init_snapshot(InitDb),\n    EntryFoo = db_entry:new(?KEY(\"foo\"), ?VALUE(\"bar\"), 0),\n    Db = db_dht:set_entry(Db, EntryFoo),\n    Db = db_dht:set_entry(Db, db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1)),\n    Db = db_dht:set_entry(Db, db_entry:new(?KEY(\"123\"), ?VALUE(\"456\"), 0)),\n    Db = db_dht:copy_value_to_snapshot_table(Db, ?KEY(\"foo\")),\n    ?equals(db_dht:get_snapshot_entry(Db, ?KEY(\"key\")), db_entry:new(?KEY(\"key\"))),\n    ?equals(db_dht:get_snapshot_entry(Db, ?KEY(\"123\")), db_entry:new(?KEY(\"123\"))),\n    ?equals(db_dht:get_snapshot_entry(Db, ?KEY(\"bar\")), db_entry:new(?KEY(\"bar\"))),\n    ?equals(db_dht:get_snapshot_entry(Db, ?KEY(\"foo\")), EntryFoo),\n    ok.\n\n-spec test_set_get_for_snapshot_table(any()) -> ok.\ntest_set_get_for_snapshot_table(_Config) ->\n    InitDb = db_dht:new(db_dht),\n    Db = db_dht:init_snapshot(InitDb),\n    EntryFoo = db_entry:new(?KEY(\"foo\"), ?VALUE(\"bar\"), 5),\n    Db = db_dht:set_snapshot_entry(Db, EntryFoo),\n    ?equals(db_dht:get_snapshot_entry(Db, ?KEY(\"key\")), db_entry:new(?KEY(\"key\"))),\n    ?equals(db_dht:get_snapshot_entry(Db, ?KEY(\"foo\")), EntryFoo),\n    ok.\n\n-spec test_delete_from_snapshot_table(any()) -> ok.\ntest_delete_from_snapshot_table(_Config) ->\n    InitDb = db_dht:new(db_dht),\n    Db = db_dht:init_snapshot(InitDb),\n    EntryFoo = db_entry:new(?KEY(\"foo\"), ?VALUE(\"bar\"), 5),\n    Db = db_dht:set_snapshot_entry(Db, EntryFoo),\n    ?equals(db_dht:get_snapshot_entry(Db, ?KEY(\"foo\")), EntryFoo),\n    Db = db_dht:delete_snapshot_entry(Db, db_entry:new(?KEY(\"foo\"), ?VALUE(\"some_val\"), 4711)),\n    ?equals(db_dht:get_snapshot_entry(Db, ?KEY(\"foo\")), db_entry:new(?KEY(\"foo\"))),\n    Db = db_dht:set_snapshot_entry(Db, EntryFoo),\n    ?equals(db_dht:get_snapshot_entry(Db, ?KEY(\"foo\")), EntryFoo),\n    Db = db_dht:delete_snapshot_entry_at_key(Db, ?KEY(\"foo\")),\n    ?equals(db_dht:get_snapshot_entry(Db, ?KEY(\"foo\")), db_entry:new(?KEY(\"foo\"))),\n    ok.\n\n-spec test_init_snapshot(any()) -> ok.\ntest_init_snapshot(_Config) ->\n    InitDb = db_dht:new(db_dht),\n    Db = db_dht:init_snapshot(InitDb),\n    Db = db_dht:set_snapshot_entry(Db, db_entry:new(?KEY(\"foo\"), ?VALUE(\"bar\"), 5)),\n    EntryKey = db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 4711),\n    Db = db_dht:set_entry(Db, EntryKey),\n    Db = db_dht:copy_value_to_snapshot_table(Db, ?KEY(\"key\")),\n    ClearDb = db_dht:init_snapshot(Db),\n    ?equals(db_dht:get_snapshot_entry(ClearDb, ?KEY(\"foo\")), db_entry:new(?KEY(\"foo\"))),\n    ?equals(db_dht:get_snapshot_entry(ClearDb, ?KEY(\"key\")), db_entry:new(?KEY(\"key\"))),\n    ?equals(db_dht:get_entry(ClearDb, ?KEY(\"key\")), EntryKey),\n    ClearDb = db_dht:copy_value_to_snapshot_table(ClearDb, ?KEY(\"key\")),\n    ?equals(db_dht:get_snapshot_entry(ClearDb, ?KEY(\"key\")), EntryKey),\n    ok.\n\n-spec test_delete_snapshot(any()) -> ok.\ntest_delete_snapshot(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    Db = db_dht:set_snapshot_entry(Db, db_entry:new(?KEY(\"foo\"), ?VALUE(\"bar\"), 5)),\n    DelDb = db_dht:delete_snapshot(Db),\n    ?equals(db_dht:snapshot_is_running(DelDb), false),\n    ok.\n\ntester_get_snapshot_data(_Conf) ->\n    tester:test(?MODULE, test_get_snapshot_data, 2, 1000, [{threads, 2}]),\n    ok.\n\n-spec test_get_snapshot_data(db_dht:db_as_list(), intervals:interval()) -> true.\ntest_get_snapshot_data(Data, Interval) ->\n    CleanData = unittest_helper:scrub_data(Data),\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    LoadedDB = db_dht:add_snapshot_data(Db, Data),\n    All = db_dht:get_snapshot_data(LoadedDB),\n    ?equals(length(All), length(CleanData)),\n    ExpPart = lists:foldl(\n            fun(Entry, Acc) ->\n                    case intervals:in(db_entry:get_key(Entry), Interval) of\n                        true ->\n                            [Entry | Acc];\n                        _ ->\n                            Acc\n                    end\n            end, [], CleanData),\n    Part = db_dht:get_snapshot_data(LoadedDB, Interval),\n    ct:pal(\"data: ~p~ninterval: ~p~nexppart: ~p~npart: ~p~n\",\n          [Data, Interval, ExpPart, Part]),\n    ?equals(length(Part), length(ExpPart)),\n    db_dht:close(LoadedDB),\n    true.\n\ntester_add_snapshot_data(_Conf) ->\n    tester:test(?MODULE, test_add_snapshot_data, 1, 1000, [{threads, 2}]),\n    ok.\n\n-spec test_add_snapshot_data(Data::db_dht:db_as_list()) -> true.\ntest_add_snapshot_data(Data) ->\n    CleanData = unittest_helper:scrub_data(Data),\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    NewDB = db_dht:add_snapshot_data(Db, CleanData),\n    ?equals(length(db_dht:get_snapshot_data(NewDB)), length(CleanData)),\n    db_dht:close(NewDB),\n    true.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% snapshot-related local tx operation tests (validate, commit, abort)\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n% rdht_tx_read\n\n% snapshot number in entry less than local number -> abort\n-spec test_rdht_tx_read_validate_should_abort(any()) -> ok.\ntest_rdht_tx_read_validate_should_abort(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    Db = db_dht:set_entry(Db, db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1)),\n    TLogEntry = tx_tlog:new_entry(?read, ?KEY(\"key\"), 1, ?ok, 1, ?value, ?VALUE(\"new_val\")),\n    {_NewDb, Vote} = rdht_tx_read:validate(Db, 5, TLogEntry),\n    ?equals(Vote, ?abort),\n    ok.\n\n% snapshot number in entry equals local number -> prepared\n-spec test_rdht_tx_read_validate_should_prepare(any()) -> ok.\ntest_rdht_tx_read_validate_should_prepare(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    Db = db_dht:set_entry(Db, db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1)),\n    TLogEntry = tx_tlog:new_entry(?read, ?KEY(\"key\"), 1, ?ok, 4711, ?value, ?VALUE(\"new_val\")),\n    {_NewDb, Vote} = rdht_tx_read:validate(Db, 4711, TLogEntry),\n    ?equals(Vote, ?prepared),\n    ok.\n\n% if a snapshot is running, locking should only occur in the live db\n% -> \"old\" entries should be copied to the snapshot-db in copy-on-write fashion\n-spec test_rdht_tx_read_validate_db_copy(any()) -> ok.\ntest_rdht_tx_read_validate_db_copy(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    EntryKey = db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1),\n    Db = db_dht:set_entry(Db, EntryKey),\n    TLogEntry = tx_tlog:new_entry(?read, ?KEY(\"key\"), 2, ?ok, 1, ?value, ?VALUE(\"new_val\")),\n    {NewDb, Vote} = rdht_tx_read:validate(Db, 1, TLogEntry),\n    ct:pal(\"Entry: ~p~nTLog:~p~nNewDB: ~p~nVote: ~p\", [EntryKey, TLogEntry, NewDb, Vote]),\n    ?equals(db_dht:get_snapshot_entry(NewDb, ?KEY(\"key\")),\n            EntryKey), % no readlocks in snap db\n    ?equals(db_dht:get_entry(NewDb, ?KEY(\"key\")),\n            db_entry:inc_readlock(EntryKey)), % 1 readlock in live db\n    ok.\n\n% for \"old\" transactions (as in TM snapnr 1 is lesser than local snapnr 2) , changes\n% (in this case: decreasing the readlock count) should be applied to both dbs.\n-spec test_rdht_tx_read_commit_with_snap_1(any()) -> ok.\ntest_rdht_tx_read_commit_with_snap_1(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    EntryKey = db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1),\n    TmpDb = db_dht:set_entry(Db, db_entry:inc_readlock(EntryKey)),\n    NewDb = db_dht:set_snapshot_entry(TmpDb, db_entry:inc_readlock(EntryKey)),\n    TLogEntry = tx_tlog:new_entry(?read, ?KEY(\"key\"), 1, ?ok, 1, ?value, ?VALUE(\"val\")),\n    CommitDb = rdht_tx_read:commit(NewDb, TLogEntry, ?prepared, 1, 2),\n    ?equals(db_dht:get_snapshot_entry(CommitDb, ?KEY(\"key\")),\n            EntryKey), % no readlocks in snap db\n    ?equals(db_dht:get_entry(CommitDb, ?KEY(\"key\")),\n            EntryKey), % no readlocks in live db\n    ok.\n\n% for \"old\" transactions (as in TM snapnr 1 is lesser than local snapnr 2) , changes\n% (in this case: decreasing the readlock count) should be applied to both dbs.\n% here (as opposed to the test above) is no action needed on the snapshot db\n% because there is no cow-value for this key in the snapshot db. the \"live\" value is valid for both!\n-spec test_rdht_tx_read_commit_with_snap_2(any()) -> ok.\ntest_rdht_tx_read_commit_with_snap_2(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    EntryKey = db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1),\n    NewDb = db_dht:set_entry(Db, db_entry:inc_readlock(EntryKey)),\n    TLogEntry = tx_tlog:new_entry(?read, ?KEY(\"key\"), 1, ?ok, 1, ?value, ?VALUE(\"val\")),\n    CommitDb = rdht_tx_read:commit(NewDb, TLogEntry, ?prepared, 1, 2),\n    ?equals(db_dht:get_snapshot_entry(CommitDb, ?KEY(\"key\")),\n            db_entry:new(?KEY(\"key\"))), % no entry in snap db\n    ?equals(db_dht:get_entry(CommitDb, ?KEY(\"key\")),\n            EntryKey), % no readlocks in live db\n    ok.\n\n% \"new\" transaction -> changes only on live db, snapshot db remains untouched\n-spec test_rdht_tx_read_commit_without_snap(any()) -> ok.\ntest_rdht_tx_read_commit_without_snap(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    EntryKey = db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1),\n    EntryKey_RL = db_entry:inc_readlock(EntryKey),\n    TmpDb = db_dht:set_entry(Db, EntryKey_RL),\n    NewDb = db_dht:set_snapshot_entry(TmpDb, EntryKey_RL),\n    TLogEntry = tx_tlog:new_entry(?read, ?KEY(\"key\"), 1, ?ok, 2, ?value, ?VALUE(\"val\")),\n    CommitDb = rdht_tx_read:commit(NewDb, TLogEntry, ?prepared, 2, 2),\n    ?equals(db_dht:get_snapshot_entry(CommitDb, ?KEY(\"key\")),\n            EntryKey_RL), % readlock in snap db\n    ?equals(db_dht:get_entry(CommitDb, ?KEY(\"key\")),\n            EntryKey), % no readlocks in live db\n    ok.\n\n% in rdht_tx_read, abort equals commit, so abort doesn't have to be tested!\n\n% rdht_tx_write\n\n% snapshot number in entry less than local number -> abort\n-spec test_rdht_tx_write_validate_should_abort(any()) -> ok.\ntest_rdht_tx_write_validate_should_abort(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    Db = db_dht:set_entry(Db, db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1)),\n    TLogEntry = tx_tlog:new_entry(?write, ?KEY(\"key\"), 1, ?ok, 1, ?value, ?VALUE(\"val\")),\n    {_NewDb, Vote} = rdht_tx_write:validate(Db, 5, TLogEntry),\n    ?equals(Vote, ?abort),\n    ok.\n\n% snapshot number in entry equals local number -> prepared\n-spec test_rdht_tx_write_validate_should_prepare(any()) -> ok.\ntest_rdht_tx_write_validate_should_prepare(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    Db = db_dht:set_entry(Db, db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1)),\n    TLogEntry = tx_tlog:new_entry(?write, ?KEY(\"key\"), 1, ?ok, 4711, ?value, ?VALUE(\"val\")),\n    {_NewDb, Vote} = rdht_tx_write:validate(Db, 4711, TLogEntry),\n    ?equals(Vote, ?prepared),\n    ok.\n\n% if a snapshot is running, locking should only occur in the live db\n% -> \"old\" entries should be copied to the snapshot-db in copy-on-write fashion\n-spec test_rdht_tx_write_validate_db_copy(any()) -> ok.\ntest_rdht_tx_write_validate_db_copy(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    EntryKey = db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1),\n    Db = db_dht:set_entry(Db, EntryKey),\n    TLogEntry = tx_tlog:new_entry(?write, ?KEY(\"key\"), 1, ?ok, 1, ?value, ?VALUE(\"val\")),\n    {NewDb, _Vote} = rdht_tx_write:validate(Db, 1, TLogEntry),\n    ?equals(db_dht:get_snapshot_entry(NewDb, ?KEY(\"key\")),\n            EntryKey), % no lock in snap db\n    ?equals(db_dht:get_entry(NewDb, ?KEY(\"key\")),\n            db_entry:set_writelock(EntryKey, 1)), % lock in live db\n    ok.\n\n% for \"old\" transactions (as in TM snapnr 1 is lesser than local snapnr 2), changes should be applied to both dbs.\n-spec test_rdht_tx_write_commit_with_snap(any()) -> ok.\ntest_rdht_tx_write_commit_with_snap(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    EntryKey = db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1),\n    EntryKey_WL = db_entry:set_writelock(EntryKey, 1),\n    TmpDb = db_dht:set_entry(Db, EntryKey_WL),\n    NewDb = db_dht:set_snapshot_entry(TmpDb, EntryKey_WL),\n    TLogEntry = tx_tlog:new_entry(?write, ?KEY(\"key\"), 1, ?ok, 1, ?value, ?VALUE(\"val\")),\n    NewEntryKey = db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 2),\n    CommitDb = rdht_tx_write:commit(NewDb, TLogEntry, ?prepared, 1, 2),\n    ?equals(db_dht:get_snapshot_entry(CommitDb, ?KEY(\"key\")),\n            NewEntryKey), % no lock in snap db\n    ?equals(db_dht:get_entry(CommitDb, ?KEY(\"key\")),\n            NewEntryKey), % no lock in live db\n    ok.\n\n% \"new\" transaction -> changes only on live db, snapshot db remains untouched\n-spec test_rdht_tx_write_commit_without_snap(any()) -> ok.\ntest_rdht_tx_write_commit_without_snap(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    EntryKey = db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1),\n    EntryKey_WL = db_entry:set_writelock(EntryKey, 1),\n    TmpDb = db_dht:set_entry(Db, EntryKey_WL),\n    NewDb = db_dht:set_snapshot_entry(TmpDb, EntryKey_WL),\n    TLogEntry = tx_tlog:new_entry(?write, ?KEY(\"key\"), 1, ?ok, 2, ?value, ?VALUE(\"val\")),\n    NewEntryKey = db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 2),\n    CommitDb = rdht_tx_write:commit(NewDb, TLogEntry, ?prepared, 2, 2),\n    ?equals(db_dht:get_snapshot_entry(CommitDb, ?KEY(\"key\")),\n            EntryKey_WL), % lock in snap db\n    ?equals(db_dht:get_entry(CommitDb, ?KEY(\"key\")),\n            NewEntryKey), % no lock in live db\n    ok.\n\n% for \"old\" transactions (as in TM snapnr 1 is lesser than local snapnr 2), changes should be applied to both dbs.\n-spec test_rdht_tx_write_abort_with_snap(any()) -> ok.\ntest_rdht_tx_write_abort_with_snap(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    EntryKey = db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1),\n    EntryKey_WL = db_entry:set_writelock(EntryKey, 1),\n    TmpDb = db_dht:set_entry(Db, EntryKey_WL),\n    NewDb = db_dht:set_snapshot_entry(TmpDb, EntryKey_WL),\n    ?equals(db_dht:get_entry(Db, ?KEY(\"key\")), EntryKey_WL),\n    ?equals(db_dht:get_snapshot_entry(Db, ?KEY(\"key\")), EntryKey_WL),\n    TLogEntry = tx_tlog:new_entry(?write, ?KEY(\"key\"), 1, ?ok, 1, ?value, ?VALUE(\"val\")),\n    CommitDb = rdht_tx_write:abort(NewDb, TLogEntry, ?prepared, 1, 2),\n    ?equals(db_dht:get_snapshot_entry(CommitDb, ?KEY(\"key\")),\n            EntryKey), % no lock in snap db\n    ?equals(db_dht:get_entry(CommitDb, ?KEY(\"key\")),\n            EntryKey), % no lock in live db\n    ok.\n\n% \"new\" transaction -> changes only on live db, snapshot db remains untouched\n-spec test_rdht_tx_write_abort_without_snap(any()) -> ok.\ntest_rdht_tx_write_abort_without_snap(_) ->\n    Db = db_dht:init_snapshot(db_dht:new(db_dht)),\n    EntryKey = db_entry:new(?KEY(\"key\"), ?VALUE(\"val\"), 1),\n    EntryKey_WL = db_entry:set_writelock(EntryKey, 1),\n    TmpDb = db_dht:set_entry(Db, EntryKey_WL),\n    NewDb = db_dht:set_snapshot_entry(TmpDb, EntryKey_WL),\n    TLogEntry = tx_tlog:new_entry(?write, ?KEY(\"key\"), 1, ?ok, 2, ?value, ?VALUE(\"val\")),\n    CommitDb = rdht_tx_write:abort(NewDb, TLogEntry, ?prepared, 1, 2),\n    ?equals(db_dht:get_snapshot_entry(CommitDb, ?KEY(\"key\")),\n            EntryKey_WL), % lock in snap db\n    ?equals(db_dht:get_entry(CommitDb, ?KEY(\"key\")),\n            EntryKey), % no lock in live db\n    ok.\n\n%%%%% lock counting tests\n\n-spec test_lock_counting_on_live_db(any()) -> ok.\ntest_lock_counting_on_live_db(_) ->\n    Db = db_dht:new(db_dht),\n    EntryFoo = db_entry:new(?KEY(\"foo\"), ?VALUE(\"bar\"), 0),\n    EntryFoo_WL = db_entry:set_writelock(EntryFoo, 1),\n    NewDB = {_, _, {_, LiveLC, _SnapLC}} = db_dht:set_entry(Db, EntryFoo_WL),\n    ?equals(LiveLC, 1),\n    {_, _, {_, NewLiveLC, _}} = db_dht:set_entry(NewDB, EntryFoo),\n    ?equals(NewLiveLC, 0),\n    ok.\n\n\n%%%%% integration tests\n\n-spec test_single_snapshot_call(any()) -> ok.\ntest_single_snapshot_call(_) ->\n    unittest_helper:make_ring(10),\n    {_, [{ok}, {ok}, {ok}, {ok}, {ok}]} =\n        api_tx:req_list([{write, \"A\", 1}, {write, \"B\", 2}, {write, \"C\", 3},\n                         {write, \"D\", 4}, {commit}]),\n    ?equals_pattern(api_tx:get_system_snapshot(),\n                    [{_, _, 0}, {_, _, 0}, {_, _, 0}, {_, _, 0}]),\n    ok.\n\n-spec test_basic_race_multiple_snapshots(any()) -> ok.\ntest_basic_race_multiple_snapshots(_) ->\n    unittest_helper:make_ring(4),\n    {_, [{fail,not_found}, {fail,not_found}, {ok},\n         {ok, _A2}, {ok, _A2}, {ok, _A2}, {ok}, {ok}]} =\n        api_tx:req_list([{read, \"A\"}, {read, \"B\"}, {write, \"A\", 8},\n                         {read, \"A\"}, {read, \"A\"}, {read, \"A\"}, {write, \"B\", 9}, {commit}]),\n    tester:test(api_tx, get_system_snapshot, 0, 100),\n    ok.\n\n-spec test_spam_transactions_and_snapshots_on_fully_joined(any()) -> ok.\ntest_spam_transactions_and_snapshots_on_fully_joined(_) ->\n    unittest_helper:make_ring(4),\n    ct:pal(\"wating for fully joined ring...~n~p\",\n           [unittest_helper:check_ring_size_fully_joined(4)]),\n\n    % apply a couple of transactions beforehand\n    tester:test(?MODULE, do_transaction_a, 1, 10),\n\n    ct:pal(\"spaming transactions...\"),\n    SpamPid1 = erlang:spawn(fun() ->\n                    [do_transaction_a(X) || X <- lists:seq(1, 50)]\n          end),\n    SpamPid2 = erlang:spawn(fun() ->\n                    [do_transaction_b(X) || X <- lists:seq(1, 50)]\n          end),\n\n    ct:pal(\"spaming snapshots...\"),\n    % spam snapshots here\n    tester:test(api_tx, get_system_snapshot, 0, 10),\n\n    ct:pal(\"waiting for transaction spam...\"),\n    util:wait_for_process_to_die(SpamPid1),\n    util:wait_for_process_to_die(SpamPid2),\n\n    ct:pal(\"getting one last snapshot...\"),\n    % get a final snapshot and print it\n    Snap = api_tx:get_system_snapshot(),\n    ?equals_pattern(Snap, [{_, _, _}, {_, _, _}]),\n    ct:pal(\"snapshot: ~p~n\", [Snap]),\n    ok.\n\n-spec do_transaction_a(number()) -> any().\ndo_transaction_a(_I) ->\n    %% ct:pal(\"spaming transaction_a...~p\", [I]),\n    api_tx:req_list([{read, \"B\"}, {write, \"A\", randoms:getRandomInt()}, {write, \"B\", randoms:getRandomInt()}, {commit}]).\n\n-spec do_transaction_b(number()) -> any().\ndo_transaction_b(_I) ->\n    %% ct:pal(\"spaming transaction_b...~p\", [I]),\n    api_tx:req_list([{read, \"A\"}, {write, \"B\", randoms:getRandomInt()}, {write, \"A\", randoms:getRandomInt()}, {commit}]).\n\n-spec test_tx_snapshot_slide_interleave(any()) -> ok.\ntest_tx_snapshot_slide_interleave(_) ->\n    unittest_helper:make_ring(1),\n    ct:pal(\"writing a bit of data...~p\", [api_tx:write(\"A\", 1)]),\n    ct:pal(\"wrote...~p\", [api_tx:read(\"A\")]),\n    timer:sleep(1000),\n    Pid = pid_groups:find_a(dht_node),\n    ct:pal(\"setting breakpoint\"),\n    IsLocked = fun(Msg, _State) ->\n            case element(1, Msg) of\n                ?tp_do_commit_abort ->\n                    comm:send_local(self(), Msg),\n                    drop_single;\n                _ ->\n                    false\n            end\n    end,\n    gen_component:bp_set_cond(Pid, IsLocked, snap_bp),\n    ct:pal(\"starting transaction that triggers breakpoint\"),\n    TxPid = spawn_link(fun() -> ct:pal(\"transaction done ~p\",\n                    [api_tx:req_list([{write, \"A\", 4}, {commit}])])\n            end),\n    %% make sure process above gets some excution time\n    timer:sleep(100),\n    ct:pal(\"starting snapshot that blocks because of hanging transaction\"),\n    SnapPid = spawn_link(fun() -> ct:pal(\"snapshot done ~p\",\n                                          [api_tx:get_system_snapshot()]) end),\n    %% make sure process above gets some excution time\n    timer:sleep(100),\n    ct:pal(\"adding node to provoke slide\"),\n    {[_], []} = api_vm:add_nodes(1),\n    ct:pal(\"wating for fully joined ring...~n~p\",\n           [unittest_helper:check_ring_size_fully_joined(2)]),\n    ct:pal(\"removing breakpoint ~p\", [gen_component:bp_del(Pid, snap_bp)]),\n    util:wait_for_process_to_die(TxPid),\n    util:wait_for_process_to_die(SnapPid),\n    ok.\n\n-spec bench_increment(any()) -> ok.\nbench_increment(_) ->\n    unittest_helper:make_ring(4),\n     SpamPid1 = erlang:spawn(fun() ->\n               bench:increment(1, 200)\n          end),\n    Return = tester:test(api_tx, get_system_snapshot, 0, 20),\n    ct:pal(\"tester return: ~p ~n\", [Return]),\n    util:wait_for_process_to_die(SpamPid1),\n    ok.\n\n\n"
  },
  {
    "path": "test/ssl_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{suites, tests, [l_on_cseq_SUITE]}.\n\n"
  },
  {
    "path": "test/tester.erl",
    "content": "%  @copyright 2010-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    test generator\n%% @end\n%% @version $Id$\n-module(tester).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([test/4, test/5, test_log/4,\n         register_type_checker/3, unregister_type_checker/1,\n         register_value_creator/4, unregister_value_creator/1]).\n\n% pseudo proc for the value_creator to create pids\n-export([start_pseudo_proc/0, pseudo_proc_fun/0]).\n\n-export([type_check_module/4]).\n\n-include(\"tester.hrl\").\n-include(\"unittest.hrl\").\n\n-type test_option() :: multi_threaded | {threads, pos_integer()} | with_feeder.\n-type test_options() :: [test_option()].\n\n-spec test(module(), atom(), non_neg_integer(), non_neg_integer()) -> ok.\ntest(Module, Func, Arity, Iterations) ->\n    test(Module, Func, Arity, Iterations, []).\n\n-spec test(module(), atom(), non_neg_integer(), non_neg_integer(), test_options()) -> ok.\ntest(Module, Func, Arity, Iterations, Options) ->\n    EmptyParseState = tester_parse_state:new_parse_state(),\n    ParseState = tester_parse_state:find_fun_info(Module, Func, Arity, EmptyParseState),\n    Threads = proplists:get_value(threads, Options, case proplists:get_bool(multi_threaded, Options) of\n                                                       true -> erlang:system_info(schedulers);\n                                                       false -> 1\n                                                    end),\n    run_test(Module, Func, Arity, Iterations, ParseState, Threads, Options),\n    ok.\n\n-spec test_log(module(), atom(), non_neg_integer(), non_neg_integer()) -> ok.\ntest_log(Module, Func, Arity, Iterations) ->\n    EmptyParseState = tester_parse_state:new_parse_state(),\n    ParseState = tester_parse_state:find_fun_info(Module, Func, Arity, EmptyParseState),\n    io:format(\"\"),\n    _ = run(Module, Func, Arity, Iterations, ParseState, [], 1),\n    ok.\n\n-spec pseudo_proc_fun() -> no_return().\npseudo_proc_fun() ->\n    receive\n        _Msg -> ok\n    end,\n    pseudo_proc_fun().\n\n-spec start_pseudo_proc() -> pid().\nstart_pseudo_proc() ->\n    Pid = erlang:spawn_link(?MODULE, pseudo_proc_fun, []),\n    erlang:register(tester_pseudo_proc, Pid),\n    Pid.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% run tests\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec run(module(), atom(), non_neg_integer(), non_neg_integer(),\n          tester_parse_state:state(), test_options(),\n          Thread::non_neg_integer()) -> ok | any().\nrun(Module, Func, Arity, Iterations, ParseState, Options, Thread) ->\n    FeederFun = list_to_atom(atom_to_list(Func) ++ \"_feeder\"),\n    case proplists:get_bool(with_feeder, Options) of\n        true ->\n            % get spec from feeder\n            case tester_parse_state:lookup_fun_type({'fun', Module,\n                                                     FeederFun, Arity},\n                                                    ParseState) of\n                {value, FeederFunType} ->\n                    % get spec from tested-fun\n                    {value, FunType} = tester_parse_state:lookup_fun_type({'fun', Module,\n                                                                           Func, Arity},\n                                                                          ParseState),\n                    run_helper(Module, Func, Arity, Iterations, FunType,\n                               FeederFunType, ParseState, Options, Thread);\n                none ->\n                   {fail, no_result, no_result_type, feeder_fun_type_not_found,\n                    tester_parse_state,\n                    lookup_type,\n                    {'fun', Module,\n                     FeederFun, Arity},\n                    'maybe_not_exported_or_no_spec_or...', util:get_stacktrace(),\n                    util:get_linetrace()}\n            end;\n        false ->\n            FeederFunType = {var_type, [], {union_fun, []}},\n            % get spec from tested-fun\n            {value, FunType} = tester_parse_state:lookup_fun_type({'fun', Module,\n                                                                   Func, Arity},\n                                                                  ParseState),\n            run_helper(Module, Func, Arity, Iterations, FunType, FeederFunType,\n                       ParseState, Options, Thread)\n    end.\n\n-spec run_helper(Module::module(), Fun::atom(), Arity::non_neg_integer(),\n                 Iterations::non_neg_integer(),\n                 Fun::{var_type, [], {union_fun, [test_fun_type(),...]}},\n                 FeederFun::{var_type, [], {union_fun, [test_fun_type()]}},\n                 tester_parse_state:state(), test_options(),\n                 Thread::non_neg_integer()) -> ok | any().\nrun_helper(_Module, _Func, _Arity, 0, _FunType, _FeederFunType, _TypeInfos, _Options, _Thread) ->\n    ok;\nrun_helper(Module, Func, Arity, Iterations, FunType, FeederFunType, TypeInfos, Options, Thread) ->\n    %% ct:pal(\"Calling: ~.0p:~.0p(~.0p)...~p to call\", [Module, Func, Options,\n    %%                                                  Iterations -1]),\n    case run_test_ttt(Module, Func, FunType, FeederFunType, TypeInfos, Options, Thread) of\n        ok ->\n            run_helper(Module, Func, Arity, Iterations - 1, FunType, FeederFunType,\n                       TypeInfos, Options, Thread);\n        Error ->\n            Error\n    end.\n\n-spec get_arg_and_result_type(Fun::{var_type, [], {union_fun, [test_fun_type(),...]}},\n                              FeederFun::{var_type, [], {union_fun, [test_fun_type(),...]}},\n                              test_options()) ->\n                                     {var_fun, [], test_fun_type()}.\nget_arg_and_result_type({var_type, [], {union_fun, FunTypes}} = _FunType,\n                        {var_type, [], {union_fun, FeederFunTypes}} = _FeederFunType, Options) ->\n    case proplists:get_bool(with_feeder, Options) of\n                                        true ->\n                                            {var_fun, [], util:randomelem(FeederFunTypes)};\n                                       false ->\n                                           {var_fun, [], util:randomelem(FunTypes)}\n                                    end.\n\n\n-spec run_test_ttt(Module::module(), Fun::atom(),\n                   Fun::{var_type, [], {union_fun, [test_fun_type(),...]}},\n                   FeederFun::{var_type, [], {union_fun, [test_fun_type(),...]}},\n                   tester_parse_state:state(), test_options(),\n                   Thread::non_neg_integer()) -> any().\nrun_test_ttt(Module, Func,\n             {var_type, [], {union_fun, FunTypes}} = FunType,\n             {var_type, [], {union_fun, _FeederFunTypes}} = FeederFunType,\n             TypeInfos, Options, Thread) ->\n    Fun = get_arg_and_result_type(FunType, FeederFunType, Options),\n    {var_fun, VarList, {'fun', ArgType, ResultType}} = Fun,\n    Size = 30,\n    GenArgs = try\n               {ok, tester_value_creator:create_value({var_type, VarList, ArgType},\n                                                      Size, TypeInfos)}\n           catch\n               ?CATCH_CLAUSE_WITH_STACKTRACE(Error, {error, Reason}, Stacktrace)\n                   print_error(Reason),\n                   %ct:pal(\"Reason: ~p~n\", [Reason]),\n                   {fail, {fail, no_result, no_result_type, Error, tester_value_creator,\n                           create_value,\n                           [ArgType, Size, typeInfos], %TypeInfos\n                           Reason, Stacktrace, util:get_linetrace()}}\n           end,\n    case GenArgs of\n        {ok, Args} ->\n            case proplists:get_bool(with_feeder, Options) of\n                true ->\n                                                % result is a tuple\n                    Result = apply_feeder(Module, Func, Args, ResultType, TypeInfos),\n                    case Result of\n                        {ok, FeededArgs} ->\n                            FunResultTypes =\n                                [InnerResultType\n                                 || {'fun', InnerArgType, InnerResultType} <- FunTypes,\n                                    tester_type_checker:check(FeededArgs, InnerArgType, TypeInfos) =:= true],\n                            case FunResultTypes of\n                                [] ->\n                                    {fail, no_result, no_result_type,\n                                     type_check_failed_feeder_result_is_not_valid_input_for_fun,\n                                     Module, Func, Args, none, util:get_stacktrace(),\n                                     util:get_linetrace()};\n                                _ ->\n                                    apply_args(Module, Func, tuple_to_list(FeededArgs),\n                                               {union, FunResultTypes}, TypeInfos, Thread)\n                            end;\n                        FeederError ->\n                            FeederError\n                    end;\n                false ->\n                    apply_args(Module, Func, Args, ResultType, TypeInfos, Thread)\n            end;\n        {fail, ErrorDesc} ->\n            ErrorDesc\n    end.\n\n% @doc called before the actual test to convert the input values. Can\n% be used to implement types which cannot be expressed by type-specs\napply_feeder(Module, Func, Args, ResultType, TypeInfos) ->\n    FeederFun = list_to_atom(atom_to_list(Func) ++ \"_feeder\"),\n    try\n        Result = apply(Module, FeederFun, Args),\n        case tester_type_checker:check(Result, ResultType, TypeInfos) of\n            true ->\n                {ok, Result};\n            {false, ErrMsg} ->\n                tester_type_checker:log_error(ErrMsg),\n                {fail, no_result, ResultType, type_check_failed_of_feeder_result, Module,\n                 FeederFun,\n                 Args,\n                 consult_output_for_detailed_type_check_report,\n                 util:get_stacktrace(), util:get_linetrace()}\n        end\n    catch\n        ?CATCH_CLAUSE_WITH_STACKTRACE(Error, Reason, Stacktrace)\n            ct:pal(\"Reason: ~p~n\", [Reason]),\n            {fail, no_result, no_result_type, Error, Module,\n             FeederFun,\n             Args,\n             Reason, Stacktrace, util:get_linetrace()}\n    end.\n\napply_args(Module, Func, Args, ResultType, TypeInfos, Thread) ->\n    %% ct:pal(\"Calling: ~.0p:~.0p(~.0p)\", [Module, Func, Args]),\n    try\n        tester_global_state:set_last_call(Thread, Module, Func, Args),\n        Result = erlang:apply(Module, Func, Args),\n        %% ct:pal(\"Result: ~.0p ~n~.0p\", [Result, ResultType]),\n        case tester_type_checker:check(Result, ResultType, TypeInfos) of\n            true ->\n                ok;\n            {false, ErrorMsg} ->\n                tester_type_checker:log_error(ErrorMsg),\n                {fail, Result, ResultType, type_check_failed_on_fun_result, Module, Func,\n                 Args, consult_output_for_detailed_type_check_report,\n                 no_stacktrace, util:get_linetrace()}\n        end\n    catch\n        ?CATCH_CLAUSE_WITH_STACKTRACE(exit, {test_case_failed, Reason}, Stacktrace)\n            {fail, no_result, no_result_type, test_case_failed, Module, Func,\n             Args, Reason, Stacktrace, util:get_linetrace()};\n        ?CATCH_CLAUSE_WITH_STACKTRACE(Error, Reason, Stacktrace)\n            {fail, no_result, no_result_type, Error, Module, Func, Args, Reason,\n             Stacktrace, util:get_linetrace()}\n    end.\n\n-spec run_test(module(), atom(), non_neg_integer(), non_neg_integer(),\n               tester_parse_state:state(), integer(), test_options()) -> ok.\nrun_test(Module, Func, Arity, Iterations, ParseState, Threads, Options) ->\n    Master = self(),\n    Dict = erlang:get(),\n    _Pids = [spawn_link(\n               fun() ->\n                       Name = list_to_atom(\"run_test:\" ++ integer_to_list(Thread)),\n                       catch(erlang:register(Name, self())),\n                       %% copy the dictionary of the original tester process\n                       %% to worker threads (allows to join a pid_group if\n                       %% necessary for a test\n                       _ = [ erlang:put(K, V) || {K, V} <- Dict ],\n                       unittest_global_state:register_thread(Thread),\n                       Result = run(Module, Func, Arity,\n                                    Iterations div Threads, ParseState, Options,\n                                    Thread),\n                       Master ! {result, Result, self()},\n                       case Result of\n                           ok -> tester_global_state:reset_last_call(Thread);\n                           _  -> ok\n                       end\n               end) || Thread <- lists:seq(1, Threads)],\n    Results = [receive {result, Result, ThreadPid} -> {Result, ThreadPid} end || _ <- lists:seq(1, Threads)],\n    _ = [fun ({Result, ThreadPid}) ->\n                 case Result of\n                     {fail, ResultValue, ResultType, Error, _Module, _Func, Args, Term,\n                      StackTrace, LineTrace} ->\n                         ArgsStr = case lists:flatten([io_lib:format(\", ~1000p\", [Arg]) || Arg <- Args]) of\n                                       [$,, $ | X] -> X;\n                                       X -> X\n                                   end,\n                         ct:pal(\"Failed (in ~.0p)~n\"\n                                \" Message    ~p in ~1000p:~1000p(~s):~n\"\n                                \"            ~p~n\"\n                                \" Result     ~p~n\"\n                                \" ResultType ~p~n\"\n                                \" Stacktrace ~p~n\"\n                                \" Linetrace  ~p~n\",\n                                [ThreadPid, Error, Module, Func, ArgsStr, Term, ResultValue,\n                                 ResultType, StackTrace, LineTrace]),\n                         ?ct_fail(\"~.0p in ~.0p:~.0p(~.0p): ~.0p\",\n                                  [Error, Module, Func, Args, Term]);\n                     ok -> ok\n                 end\n         end(XResult) || XResult <- Results],\n    %ct:pal(\"~w~n\", [Results]),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% type check a module\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec type_check_module(module(),\n                        ExcludeExported::[{Fun::atom(), Arity::non_neg_integer()}],\n                        ExcludePrivate::[{Fun::atom(), Arity::non_neg_integer()}],\n                       Count::pos_integer()) -> ok.\ntype_check_module(Module, ExcludeExported, ExcludePrivate, Count) ->\n    ExpFuncs = Module:module_info(exports),\n    ExcludeList = [{module_info, 0}, {module_info, 1} | ExcludeExported],\n\n    %% only excluded exported functions?\n    ErrList = [ {Module, X} || X <- ExcludeList,\n                               not lists:member(X, ExpFuncs) ],\n    case ErrList of\n        [] -> ok;\n        [_|_] ->\n            ct:pal(\"Excluded non-existing or non-exported functions: ~p~n\", [ErrList]),\n            throw(error)\n    end,\n\n    %% perform the actual tests\n    %% >= R15 generates behaviour_info without a type spec so\n    %% tester cannot find it. Erlang < R15 checks behaviour_info\n    %% itself, so no own tests necessary here.\n    %% Silently drop it for modules that export it.\n    ResList = type_check_module_funs(\n                Module, ExpFuncs, [{behaviour_info, 1} | ExcludeList], Count),\n\n    case cover:modules() of\n        [] ->\n            type_check_private_funs(Module, ExcludePrivate, Count);\n        _ ->\n            %% code reloading is not supported for cover analysis (see\n            %% docs of cover:compile/2)\n            ct:pal(\"*** Detected cover analysis, have to skip tests of private funs.\"),\n            ok\n    end,\n    %% Was there even anything left to test?\n    case [] =:= ExcludeExported orelse\n             lists:member(ok, lists:flatten(ResList)) of\n        true -> ok;\n        _ ->\n            ct:pal(\"Excluded all exported functions for module ~p?!\",\n                   [Module]),\n            catch ct:comment(\"Excluded all exported functions for module ~p?!\",\n                             [Module]),\n            ok\n    end,\n\n    ok.\n\n-spec type_check_module_funs(\n        module(), FunList::[{Fun::atom(), Arity::non_neg_integer()}],\n        ExcludeList::[{Fun::atom(), Arity::non_neg_integer()}],\n        Count::pos_integer()) -> [Status | [Status]]\n        when is_subtype(Status, ok | skipped | feeder).\ntype_check_module_funs(Module, FunList, ExcludeList, Count) ->\n    Dict = erlang:get(),\n    util:par_map(\n      fun({Fun, Arity} = FA) ->\n              _ = [ erlang:put(K, V) || {K, V} <- Dict ],\n              %% test all non excluded funs with std. settings\n              Res1 = case lists:member(FA, ExcludeList) of\n                         false ->\n                             ct:pal(\"Testing ~p:~p/~p\", [Module, Fun, Arity]),\n                             test(Module, Fun, Arity, Count, [{threads, 2}]);\n                         true  -> skipped\n                     end,\n\n              %% if Fun is a feeder, crosscheck existence of tested fun,\n              %% but do not trigger tests for that.\n              FunString = atom_to_list(Fun),\n              Res2 = case lists:suffix(\"_feeder\", FunString) of\n                         true ->\n                             TestedFunString =\n                                 lists:sublist(FunString, length(FunString) - 7),\n                             try lists:member({list_to_existing_atom(TestedFunString), Arity},\n                                              FunList) of\n                                 true -> feeder;\n                                 false ->\n                                     ct:pal(\"Found feeder, but no target fun ~p:~s/~p\",\n                                            [Module, TestedFunString, Arity]),\n                                     throw(error)\n                             catch _:_ ->\n                                       ct:pal(\"Found feeder, but no target fun ~p:~s/~p\",\n                                              [Module, TestedFunString, Arity]),\n                                       throw(error)\n                             end;\n                         false -> Res1\n                     end,\n\n              %% if a feeder is found, test with feeder and ignore the\n              %% exclude list, as a feeder is expected to feed the\n              %% tested fun appropriately (will type check feeder\n              %% results for required input types anyhow).\n              FeederFun = list_to_atom(FunString ++ \"_feeder\"),\n              case lists:member({FeederFun, Arity}, FunList) of\n                  true ->\n                      ct:pal(\"Testing with feeder ~p:~p/~p\",\n                             [Module, Fun, Arity]),\n                      [Res2, test(Module, Fun, Arity, Count, [{threads, 2}, with_feeder])];\n                  false -> Res2\n              end\n      end, FunList, 2).\n\n-spec type_check_private_funs(\n        module(), ExcludePrivate::[{Fun::atom(), Arity::non_neg_integer()}],\n        Count::pos_integer()) -> ok.\ntype_check_private_funs(Module, ExcludePrivate, Count) ->\n    ExportedFuns = Module:module_info(exports),\n\n    tester_helper:load_with_export_all(Module),\n    AllFuns = Module:module_info(exports),\n\n    PrivateFuns = [ X || X <- AllFuns, not lists:member(X, ExportedFuns)],\n\n    ct:pal(\"*** Private funs of ~p:~n~.0p\", [Module, PrivateFuns]),\n\n    %% only excluded existing functions?\n    ErrList = [ {Module, X} || X <- ExcludePrivate,\n                               not lists:member(X, PrivateFuns) ],\n    case ErrList of\n        [] -> ok;\n        [_|_] ->\n            ct:pal(\"Excluded non-existing private functions: ~p~n\", [ErrList]),\n            throw(error)\n    end,\n\n    _ = type_check_module_funs(Module, PrivateFuns, ExcludePrivate, Count),\n\n    tester_helper:load_without_export_all(Module).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% handle global state, e.g. specific handlers\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec register_type_checker(type_spec(), module(), atom()) ->\n                                   true.\nregister_type_checker(Type, Module, Function) ->\n    tester_global_state:register_type_checker(Type, Module, Function).\n\n-spec unregister_type_checker(type_spec()) -> true | ok.\nunregister_type_checker(Type) ->\n    tester_global_state:unregister_type_checker(Type).\n\n-spec register_value_creator(type_spec(), module(), atom(), non_neg_integer()) -> true.\nregister_value_creator(Type, Module, Function, Arity) ->\n    tester_global_state:register_value_creator(Type, Module, Function, Arity).\n\n-spec unregister_value_creator(type_spec()) -> true | ok.\nunregister_value_creator(Type) ->\n    tester_global_state:unregister_value_creator(Type).\n\nprint_error(Msgs) ->\n    ct:pal(\"error in value creator. could not create a value of type:\\n\"\n           ++ lists:flatten(print_error_(lists:reverse(Msgs), \"  \"))).\n\nprint_error_([], _Prefix) ->\n    \"\";\nprint_error_([Msg|Msgs], Prefix) ->\n    io_lib:format(\"~s~s~n\", [Prefix, Msg]) ++ print_error_(Msgs, \" in \").\n"
  },
  {
    "path": "test/tester.hrl",
    "content": "%  @copyright 2010-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    test generator type definitions\n%% @version $Id$\n-include(\"scalaris.hrl\").\n-include(\"record_helpers.hrl\").\n\n-type(test_any() ::\n   % | none()\n   % | no_return()\n   % | pid()\n   % | port()\n   % | ref()\n       []\n     | atom()\n   % | binary()\n     | float()\n   % | Fun\n     | integer()\n     | 42\n     | -1..1\n     | list()\n     | list(test_any())\n   % | improper_list(test_any(), test_any()\n   % | maybe_improper_list(test_any(), test_any())\n     | tuple()\n     | {}\n     | {test_any(), test_any(), test_any()}\n   % | Union\n   % | Userdefined\n           ).\n\n\n-type(builtin_type() ::\n      array\n    | bitstring\n    | dict\n    | gb_set\n    | gb_tree\n    | identifier\n    | iodata\n    | maybe_improper_list\n    | map\n    | module\n    | set\n    | timeout).\n\n-type(type_name() ::\n      {'fun', Module :: module(), FunName :: atom(), FunArity :: byte()}\n      | {type, Module :: module(), TypeName :: atom(), Arity::arity()}\n      | {record, Module :: module(), TypeName :: atom()}).\n\n-type(var_list() :: list(atom())).\n\n-type(record_field_type() ::\n      {typed_record_field, atom(), type_spec()}\n    | {untyped_record_field, atom()}\n    | {field_type, Name::atom(), Type::type_spec()}\n     ).\n\n-type(type_spec() ::\n      {'fun', type_spec(), type_spec()}\n    | {'union_fun', [{'fun', type_spec(), type_spec()}]}\n    | {product, [type_spec()]}\n    | {tuple, [type_spec()]}\n    | {tuple, {typedef, tester, test_any}}\n    | {list, type_spec()}\n    | {nonempty_list, type_spec()}\n    | {range, {integer, integer()}, {integer, integer()}}\n    | {union, [type_spec()]}\n    | {record, atom(), atom()}\n    | nonempty_string\n    | integer\n    | pos_integer\n    | neg_integer\n    | non_neg_integer\n    | bool\n    | {binary, list()}\n    | iolist\n    | node\n    | pid\n    | port\n    | reference\n    | none\n    | no_return\n    | atom\n    | float\n    | nil\n    | {atom, atom()}\n    | {integer, integer()}\n    | {typedef, module(), atom(), [type_spec()]}\n    | {var, atom()}\n    | {var_type, list(atom()), type_spec()}\n    | {builtin_type, builtin_type()}\n    | {builtin_type, array_array, ValueType::type_spec()}\n    | {builtin_type, dict_dict, KeyType::type_spec(), ValueType::type_spec()}\n    | {builtin_type, queue_queue, ValueType::type_spec()}\n    | {builtin_type, gb_sets_set, ValueType::type_spec()}\n    | {builtin_type, gb_trees_tree, KeyType::type_spec(), ValueType::type_spec()}\n    | {builtin_type, gb_trees_iter, KeyType::type_spec(), ValueType::type_spec()}\n    | {builtin_type, sets_set, ValueType::type_spec()}\n    | {builtin_type, ordsets_ordset, ValueType::type_spec()}\n    | {record, [record_field_type()]} % TODO: is this still used?\n    | {record, module(), Name::atom()}\n    | {record, module(), Name::atom(), FieldTypes::[type_spec()]}\n    | record_field_type()\n     ).\n\n-type(test_fun_type() :: {'fun', type_spec(), type_spec()}).\n"
  },
  {
    "path": "test/tester_SUITE.erl",
    "content": "% @copyright 2010-2012 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc Unit tests for random tester\n%% @end\n%% @version $Id$\n-module(tester_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include_lib(\"unittest.hrl\").\n\nall() ->\n    [test_is_binary,\n     test_sort].\n\n\nsuite() ->\n    [\n     {timetrap, {seconds, 10}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% simple tester:test/3\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec prop_is_binary_feeder(integer()) -> {integer()}.\nprop_is_binary_feeder(Int) ->\n    {Int}.\n\n-spec prop_is_binary(integer()) -> binary().\nprop_is_binary(Bin) ->\n    term_to_binary(Bin).\n\ntest_is_binary(_Config) ->\n    tester:test(?MODULE, prop_is_binary, 1, 25, []).\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% tester:test/3 with value-creator and custom type checker\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-type sorted_list() :: list(integer()).\n\nis_sorted_list([]) ->\n    true;\nis_sorted_list([_I]) ->\n    true;\nis_sorted_list([I,J|L]) ->\n    I =< J andalso is_sorted_list([J|L]).\n\n-spec create_sorted_list(list(integer()), list(integer())) -> sorted_list().\ncreate_sorted_list(L1, L2) ->\n    %ct:pal(\"creating sorted list from ~p ~p\", [L1, L2]),\n    lists:sort(lists:append(L1, L2)).\n\n-spec do_sort(sorted_list()) -> sorted_list().\ndo_sort(L) ->\n    lists:sort(L).\n\ntest_sort(_Config) ->\n    tester:register_type_checker({typedef, tester_SUITE, sorted_list, []}, tester_SUITE, is_sorted_list),\n    tester:register_value_creator({typedef, tester_SUITE, sorted_list, []}, tester_SUITE, create_sorted_list, 2),\n    tester:test(?MODULE, do_sort, 1, 25, []),\n    tester:unregister_type_checker({typedef, tester_SUITE, sorted_list, []}),\n    tester:unregister_value_creator({typedef, tester_SUITE, sorted_list, []}).\n"
  },
  {
    "path": "test/tester_collect_function_info.erl",
    "content": "%  @copyright 2010-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    collection type information about a function\n%% @end\n%% @version $Id$\n-module(tester_collect_function_info).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([collect_fun_info/4]).\n-export([unittest_collect_module_info/2]).\n\n-include(\"tester.hrl\").\n-include(\"unittest.hrl\").\n\n-spec unittest_collect_module_info(module(), tester_parse_state:state()) ->\n                                          tester_parse_state:state().\nunittest_collect_module_info(Module, ParseState) ->\n    ?ASSERT(util:is_unittest()), % may only be used in unit-tests\n    erlang:put(module, Module),\n    ModuleFile = code:where_is_file(atom_to_list(Module) ++ \".beam\"),\n    case beam_lib:chunks(ModuleFile, [abstract_code]) of\n        {ok, {Module, [{abstract_code, {_AbstVersion, AbstractCode}}]}} ->\n            lists:foldl(fun(Chunk, InnerParseState) ->\n                                parse_chunk_log(Chunk, Module, InnerParseState)\n                        end, ParseState, AbstractCode);\n        {ok, {Module, [{abstract_code, no_abstract_code}]}} ->\n            ?ct_fail(\"the module ~s was not compiled with debug_info~n~p\",\n                     [Module, util:get_stacktrace()])\n    end.\n\n-spec collect_fun_info(module(), atom(), non_neg_integer(),\n                       tester_parse_state:state()) -> tester_parse_state:state().\ncollect_fun_info(Module, Func, Arity, ParseState) ->\n    erlang:put(module, Module),\n    ParseState2 =\n        case tester_parse_state:lookup_type({'fun', Module, Func, Arity}, ParseState) of\n            {value, _} -> ParseState;\n            none ->\n                ModuleFile = code:where_is_file(atom_to_list(Module) ++ \".beam\"),\n                {ok, {Module, [{abstract_code, {_AbstVersion, AbstractCode}}]}}\n                    = beam_lib:chunks(ModuleFile, [abstract_code]),\n                lists:foldl(fun(Chunk, InnerParseState) ->\n                                    parse_chunk_log(Chunk, Module, InnerParseState)\n                            end, ParseState, AbstractCode)\n        end,\n    ParseState3 = case tester_parse_state:has_unknown_types(ParseState2) of\n                      false -> ParseState2;\n                      true  -> collect_unknown_type_infos(ParseState2, [])\n                  end,\n    case tester_parse_state:lookup_type({'fun', Module, Func, Arity}, ParseState3) of\n        {value, _} -> tester_parse_state:finalize(ParseState3);\n        none -> ?ct_fail(\"no '-spec' definition for function ~p:~p/~p found by tester~n\", [Module, Func, Arity])\n    end.\n\n-spec collect_unknown_type_infos(tester_parse_state:state(), list()) ->\n    tester_parse_state:state().\ncollect_unknown_type_infos(ParseState, OldUnknownTypes) ->\n    {_, UnknownTypes} = tester_parse_state:get_unknown_types(ParseState),\n    %ct:pal(\"unknown types: ~p~n\", [UnknownTypes]),\n    case OldUnknownTypes =:= UnknownTypes of\n        true ->\n            ct:pal(\"never found the following types: ~p~n~n\", [UnknownTypes]),\n            ?ct_fail(\"never found the following types: ~p~n~n\", [UnknownTypes]),\n            error;\n        false ->\n            ParseState2 = tester_parse_state:reset_unknown_types(ParseState),\n            ParseState3 = lists:foldl(fun({type, Module, TypeName, Arity}, InnerParseState) ->\n                                              collect_type_info(Module, TypeName, Arity,\n                                                                InnerParseState)\n                                      end, ParseState2, UnknownTypes),\n            case tester_parse_state:has_unknown_types(ParseState3) of\n                false -> ParseState3;\n                true  -> collect_unknown_type_infos(ParseState3, UnknownTypes)\n            end\n    end.\n\n-spec collect_type_info(module(), atom(), arity(), tester_parse_state:state()) ->\n    tester_parse_state:state().\ncollect_type_info(Module, Type, Arity, ParseState) ->\n    erlang:put(module, Module),\n    case tester_parse_state:is_known_type(Module, Type, Arity, ParseState) of\n        true ->\n            ParseState;\n        false ->\n            case code:where_is_file(atom_to_list(Module) ++ \".beam\") of\n                non_existing ->\n                    ct:pal(\"Error: File \\\"~w\\\" not found while trying to collect type info for ~w:~w/~w.\",\n                           [Module, Module, Type, Arity]),\n                    ?ct_fail(\"File \\\"~w\\\" not found while trying to collect type info for ~w:~w/~w.\",\n                           [Module, Module, Type, Arity]),\n                    error;\n                FileName ->\n                    {ok, {Module, [{abstract_code, {_AbstVersion, AbstractCode}}]}}\n                    = beam_lib:chunks(FileName, [abstract_code]),\n                    lists:foldl(fun (Chunk, InnerParseState) ->\n                                        parse_chunk_log(Chunk, Module, InnerParseState)\n                                end, ParseState, AbstractCode)\n            end\n    end.\n\n-spec parse_chunk_log(any(), module(), tester_parse_state:state()) ->\n                             tester_parse_state:state().\nparse_chunk_log(Type, Module, State) ->\n    try\n        parse_chunk(Type, Module, State)\n    catch\n        subst_error ->\n            ct:pal(\"~p:~p: failed to substitute\", [Module, Type]),\n            exit(foobar);\n        parse_error ->\n            ct:pal(\"~p:~p: failed to parse chunk\", [Module, Type]),\n            exit(foobar)%;\n%        error:Reason ->\n%            ct:pal(\"~p:~p: failed to parse chunk (error:~p)\", [Module, Type, Reason]),\n%            exit(foobar)\n    end.\n\n-spec parse_chunk(any(), module(), tester_parse_state:state()) ->\n                         tester_parse_state:state().\nparse_chunk({attribute, _Line, type, {{record, TypeName}, ATypeSpec, List}},\n            Module, ParseState) ->\n    {TheTypeSpec, NewParseState} = parse_type_log(ATypeSpec, Module, ParseState,\n                                                  {record_type_attribute, TypeName}),\n    tester_parse_state:add_type_spec({record, Module, TypeName}, TheTypeSpec, List,\n                                     NewParseState);\nparse_chunk({attribute, _Line, type, {TypeName, ATypeSpec, List}},\n            Module, ParseState) ->\n    {TheTypeSpec, NewParseState} = parse_type_log(ATypeSpec, Module, ParseState,\n                                                  {type_attribute, TypeName}),\n    tester_parse_state:add_type_spec({type, Module, TypeName, length(List)}, TheTypeSpec, List,\n                                     NewParseState);\nparse_chunk({attribute, _Line, opaque, {TypeName, ATypeSpec, List}},\n            Module, ParseState) ->\n    {TheTypeSpec, NewParseState} = parse_type_log(ATypeSpec, Module, ParseState,\n                                                  {opaque_type_attribute, TypeName}),\n    tester_parse_state:add_type_spec({type, Module, TypeName, length(List)}, TheTypeSpec, List,\n                                     NewParseState);\nparse_chunk({attribute, _Line, 'spec', {{FunName, FunArity}, AFunSpec}},\n            Module, ParseState) ->\n    FunSpec = [case TheFunSpec of\n                  {type, _,bounded_fun, [_TypeFun, ConstraintType]} ->\n                      try\n                          Substitutions = parse_constraints(ConstraintType, gb_trees:empty()),\n                          tester_variable_substitutions:substitute(TheFunSpec, Substitutions)\n                      catch\n                          {subst_error, Description} ->\n                              ct:pal(\"substitution error ~w in ~w:~w ~w\", [Description, Module, FunName, AFunSpec]),\n                              throw(subst_error);\n                          {parse_error, Description} ->\n                              ct:pal(\"parse error ~w in ~w:~w ~w\", [Description, Module, FunName, AFunSpec]),\n                              throw(parse_error)\n                      end;\n                  _ ->\n                      TheFunSpec\n              end || TheFunSpec <- AFunSpec],\n    {CleanFunSpec, NewParseState} = parse_type_log({union_fun, FunSpec}, Module, ParseState,\n                                                   {fun_spec, FunName}),\n    tester_parse_state:add_type_spec({'fun', Module, FunName, FunArity},\n                                     CleanFunSpec, [], NewParseState);\nparse_chunk({attribute, _Line, record, {TypeName, TypeList}}, Module, ParseState) ->\n    {TheTypeSpec, NewParseState} = parse_type_log(TypeList, Module, ParseState,\n                                                  {record_attribute, TypeName}),\n    tester_parse_state:add_type_spec({record, Module, TypeName}, TheTypeSpec, [],\n                                     NewParseState);\nparse_chunk({attribute, _Line, _AttributeName, _AttributeValue}, _Module,\n            ParseState) ->\n    ParseState;\nparse_chunk({function, _Line, _FunName, _FunArity, FunCode}, _Module, ParseState) ->\n    erlang:put(fun_name, _FunName),\n    tester_value_collector:parse_expression(FunCode, ParseState);\nparse_chunk({eof, _Line}, _Module, ParseState) ->\n    ParseState.\n\n-spec parse_type_log(any(), module(), tester_parse_state:state(), tuple()) ->\n                            {type_spec() , tester_parse_state:state()}.\nparse_type_log(Type, Module, ParseState, Info) ->\n    try\n        parse_type(Type, Module, ParseState)\n    catch\n        unknown_type ->\n            ct:pal(\"~p:~p: failed to parse type ~p\", [Module, Info, Type]),\n            exit(foobar)%;\n%%        error:Reason ->\n%%            ct:pal(\"~p:~p: failed to parse type ~p (error:~p)\", [Module, Info, Type, Reason]),\n%%            exit(foobar)\n    end.\n\n-spec parse_type(any(), module(), tester_parse_state:state()) ->\n                        {type_spec() , tester_parse_state:state()}.\nparse_type({union_fun, FunSpecs}, Module, ParseState) ->\n    {FunSpecs2, PS2} = lists:foldl(fun (FunType, {List, PS}) ->\n                        {ParsedFunType, PS1 } = parse_type(FunType, Module, PS),\n                        {[ParsedFunType | List], PS1}\n                end, {[], ParseState}, FunSpecs),\n    {{union_fun, FunSpecs2}, PS2};\nparse_type({type, _Line, 'fun', [Arg, Result]}, Module, ParseState) ->\n    {ArgType, ParseState2} = parse_type(Arg, Module, ParseState),\n    {ResultType, ParseState3} = parse_type(Result, Module, ParseState2),\n    {{'fun', ArgType, ResultType}, ParseState3};\nparse_type({type, _Line, product, Types}, Module, ParseState) ->\n    {TypeList, ParseState2} = parse_type_list(Types, Module, ParseState),\n    {{product, TypeList}, ParseState2};\nparse_type({type, _Line, tuple, any}, _Module, ParseState) ->\n    {{tuple, {typedef, tester, test_any, []}}, ParseState};\nparse_type({type, _Line, tuple, Types}, Module, ParseState) ->\n    {TypeList, ParseState2} = parse_type_list(Types, Module, ParseState),\n    {{tuple, TypeList}, ParseState2};\nparse_type({type, _Line, list, [Type]}, Module, ParseState) ->\n    {ListType, ParseState2} = parse_type(Type, Module, ParseState),\n    {{list, ListType}, ParseState2};\nparse_type({type, _Line, nonempty_list, [Type]}, Module, ParseState) ->\n    {ListType, ParseState2} = parse_type(Type, Module, ParseState),\n    {{nonempty_list, ListType}, ParseState2};\nparse_type({type, _Line, nonempty_improper_list, Types = [_Type1, _Type2]}, Module, ParseState) ->\n    {TypeList, ParseState2} = parse_type_list(Types, Module, ParseState),\n    {{nonempty_improper_list, TypeList}, ParseState2};\nparse_type({type, _Line, list, []}, _Module, ParseState) ->\n    {{list, {typedef, tester, test_any, []}}, ParseState};\nparse_type([], _Module, ParseState) ->\n    {{list, {typedef, tester, test_any, []}}, ParseState};\nparse_type({type, _Line, range, [Begin, End]}, Module, ParseState) ->\n    {BeginType, ParseState2} = parse_type(Begin, Module, ParseState),\n    {EndType, ParseState3} = parse_type(End, Module, ParseState2),\n    {{range, BeginType, EndType}, ParseState3};\nparse_type({type, _Line, union, Types}, Module, ParseState) ->\n    {TypeList, ParseState2} = parse_type_list(Types, Module, ParseState),\n    {{union, TypeList}, ParseState2};\nparse_type({type, _Line, integer, []}, _Module, ParseState) ->\n    {integer, ParseState};\nparse_type({type, _Line, pos_integer, []}, _Module, ParseState) ->\n    {pos_integer, ParseState};\nparse_type({type, _Line, neg_integer, []}, _Module, ParseState) ->\n    {neg_integer, ParseState};\nparse_type({type, _Line, non_neg_integer, []}, _Module, ParseState) ->\n    {non_neg_integer, ParseState};\nparse_type({type, _Line, byte, []}, _Module, ParseState) ->\n    {{range, {integer, 0}, {integer, 255}}, ParseState};\nparse_type({type, _Line, bool, []}, _Module, ParseState) ->\n    {bool, ParseState};\nparse_type({type, _Line, char, []}, _Module, ParseState) ->\n    {{range, {integer, 0}, {integer, 16#10ffff}}, ParseState};\nparse_type({type, _Line, string, []}, _Module, ParseState) ->\n    {{list, {range, {integer, 0}, {integer, 16#10ffff}}}, ParseState};\nparse_type({type, _Line, nonempty_string, []}, _Module, ParseState) ->\n    {nonempty_string, ParseState};\nparse_type({type, _Line, number, []}, _Module, ParseState) ->\n    {{union, [integer, float]}, ParseState};\nparse_type({type, _Line, boolean, []}, _Module, ParseState) ->\n    {bool, ParseState};\nparse_type({type, _Line, any, []}, _Module, ParseState) ->\n    {{typedef, tester, test_any, []}, ParseState};\nparse_type({type, _Line, any}, _Module, ParseState) ->\n    {{typedef, tester, test_any}, ParseState};\nparse_type({type, _Line, atom, []}, _Module, ParseState) ->\n    {atom, ParseState};\nparse_type({type, _Line, arity, []}, _Module, ParseState) ->\n    {arity, ParseState};\nparse_type({type, _Line, binary, L}, _Module, ParseState) ->\n    {{binary, L}, ParseState};\nparse_type({type, _Line, pid, []}, _Module, ParseState) ->\n    {pid, ParseState};\nparse_type({type, _Line, port, []}, _Module, ParseState) ->\n    {port, ParseState};\nparse_type({type, _Line, float, []}, _Module, ParseState) ->\n    {float, ParseState};\nparse_type({type, _Line, iolist, []}, _Module, ParseState) ->\n    {iolist, ParseState};\nparse_type({type, _Line, nil, []}, _Module, ParseState) ->\n    {nil, ParseState};\nparse_type({type, _Line, node, []}, _Module, ParseState) ->\n    {node, ParseState};\nparse_type({type, _Line, none, []}, _Module, ParseState) ->\n    {none, ParseState};\nparse_type({type, _Line, no_return, []}, _Module, ParseState) ->\n    {none, ParseState};\nparse_type({type, _Line, reference, []}, _Module, ParseState) ->\n    {reference, ParseState};\nparse_type({type, _Line, term, []}, _Module, ParseState) ->\n    {{typedef, tester, test_any, []}, ParseState};\nparse_type({ann_type, _Line, [{var, _Line, _Varname}, Type]}, Module, ParseState) ->\n    parse_type(Type, Module, ParseState);\nparse_type({atom, _Line, Atom}, _Module, ParseState) ->\n    {{atom, Atom}, ParseState};\nparse_type({op, _Line1, '-', {integer,_Line2,Value}}, _Module, ParseState) ->\n    {{integer, -Value}, ParseState};\nparse_type({op, _Line1, '-', FirstType, {integer,_Line2,Value}}, Module, ParseState) ->\n    {{op_diff, parse_type(FirstType, Module, ParseState), {integer, Value}}, ParseState};\nparse_type({op, _Line1, 'bsl', FirstType, SecondType}, Module, ParseState) ->\n    First = parse_type(FirstType, Module, ParseState),\n    Second = parse_type(SecondType, Module, ParseState),\n    {{op_bsl, First, Second}, ParseState};\nparse_type({integer, _Line, Value}, _Module, ParseState) ->\n    {{integer, Value}, ParseState};\nparse_type({type, _Line, array, []}, _Module, ParseState) ->\n    {{builtin_type, array_array, {typedef, tester, test_any, []}}, ParseState};\nparse_type({type, _Line, dict, []}, _Module, ParseState) ->\n    {{builtin_type, dict_dict, {typedef, tester, test_any, []},\n      {typedef, tester, test_any, []}}, ParseState};\nparse_type({type, _Line, queue, []}, _Module, ParseState) ->\n    {{builtin_type, queue_queue, {typedef, tester, test_any, []}}, ParseState};\nparse_type({type, _Line, gb_set, []}, _Module, ParseState) ->\n    {{builtin_type, gb_sets_set, {typedef, tester, test_any, []}}, ParseState};\nparse_type({type, _Line, gb_tree, []}, _Module, ParseState) ->\n    {{builtin_type, gb_trees_tree, {typedef, tester, test_any, []},\n      {typedef, tester, test_any, []}}, ParseState};\nparse_type({type, _Line, set, []}, _Module, ParseState) ->\n    {{builtin_type, set_set, {typedef, tester, test_any, []}}, ParseState};\nparse_type({type, _Line, module, []}, _Module, ParseState) ->\n    {{builtin_type, module}, ParseState};\nparse_type({type, _Line, iodata, []}, _Module, ParseState) ->\n    {{builtin_type, iodata}, ParseState};\nparse_type({type, _Line, map, any}, _Module, ParseState) -> % Erlang R17\n    {{builtin_type, map}, ParseState};\nparse_type({type, _Line, mfa, []}, _Module, ParseState) ->\n    {{tuple, [atom, atom, {range, {integer, 0}, {integer, 255}}]}, ParseState};\n% array:array(Value)\nparse_type({remote_type, _Line, [{atom, _Line2, array},\n                                 {atom, _Line3, array}, [ValueType]]},\n           Module, ParseState) ->\n    {Value, ParseState2}   = parse_type(ValueType, Module, ParseState),\n    {{builtin_type, array_array, Value}, ParseState2};\n% dict:dict(Key,Value)\nparse_type({remote_type, _Line, [{atom, _Line2, dict},\n                                 {atom, _Line3, dict}, [KeyType, ValueType]]},\n           Module, ParseState) ->\n    {Key2, ParseState2}   = parse_type(KeyType, Module, ParseState),\n    {Value2, ParseState3}   = parse_type(ValueType, Module, ParseState2),\n    {{builtin_type, dict_dict, Key2, Value2}, ParseState3};\n% queue:queue(Value)\nparse_type({remote_type, _Line, [{atom, _Line2, queue},\n                                 {atom, _Line3, queue}, [ValueType]]},\n           Module, ParseState) ->\n    {Value, ParseState2}   = parse_type(ValueType, Module, ParseState),\n    {{builtin_type, queue_queue, Value}, ParseState2};\n% gb_sets:set(Value)\nparse_type({remote_type, _Line, [{atom, _Line2, gb_sets},\n                                 {atom, _Line3, set}, [ValueType]]},\n           Module, ParseState) ->\n    {Value, ParseState2}   = parse_type(ValueType, Module, ParseState),\n    {{builtin_type, gb_sets_set, Value}, ParseState2};\n% gb_trees:tree(Key,Value)\nparse_type({remote_type, _Line, [{atom, _Line2, gb_trees},\n                                 {atom, _Line3, tree}, [KeyType, ValueType]]},\n           Module, ParseState) ->\n    {Key2, ParseState2}   = parse_type(KeyType, Module, ParseState),\n    {Value2, ParseState3}   = parse_type(ValueType, Module, ParseState2),\n    {{builtin_type, gb_trees_tree, Key2, Value2}, ParseState3};\n% gb_trees:tree()\nparse_type({remote_type, _Line, [{atom, _Line2, gb_trees},\n                                 {atom, _Line3, tree}, []]},\n           _Module, ParseState) ->\n    {{builtin_type, gb_trees_tree, {typedef, tester, test_any, []},\n      {typedef, tester, test_any, []}}, ParseState};\n% gb_trees:iter(Key,Value)\nparse_type({remote_type, _Line, [{atom, _Line2, gb_trees},\n                                 {atom, _Line3, iter}, [KeyType, ValueType]]},\n           Module, ParseState) ->\n    {Key2, ParseState2}   = parse_type(KeyType, Module, ParseState),\n    {Value2, ParseState3}   = parse_type(ValueType, Module, ParseState2),\n    {{builtin_type, gb_trees_iter, Key2, Value2}, ParseState3};\n% sets:set(Value)\nparse_type({remote_type, _Line, [{atom, _Line2, sets},\n                                 {atom, _Line3, set}, [ValueType]]},\n           Module, ParseState) ->\n    {Value, ParseState2}   = parse_type(ValueType, Module, ParseState),\n    {{builtin_type, set_set, Value}, ParseState2};\n% ordsets:ordset(Value)\nparse_type({remote_type, _Line, [{atom, _Line2, ordsets},\n                                 {atom, _Line3, ordset}, [ValueType]]},\n           Module, ParseState) ->\n    {Value, ParseState2}   = parse_type(ValueType, Module, ParseState),\n    {{builtin_type, ordsets_ordset, Value}, ParseState2};\nparse_type({remote_type, _Line, [{atom, _Line2, TypeModule},\n                                 {atom, _Line3, TypeName}, L]},\n           _Module, ParseState) ->\n    case tester_parse_state:is_known_type(TypeModule, TypeName, length(L), ParseState) of\n        true ->\n            {{typedef, TypeModule, TypeName, L}, ParseState};\n        false ->\n            {{typedef, TypeModule, TypeName, L},\n             tester_parse_state:add_unknown_type(TypeModule, TypeName, length(L), ParseState)}\n    end;\n% why is this here? function() is no official type\nparse_type({type, _Line, 'function', []}, _Module, ParseState) ->\n    {{'function'}, ParseState};\nparse_type({type, _Line, 'fun', []}, _Module, ParseState) ->\n    {{'function'}, ParseState};\nparse_type({type, _Line, record, [{atom, _Line2, TypeName}]}, Module, ParseState) ->\n    {{record, Module, TypeName}, ParseState};\nparse_type({type, _Line, record, [{atom, _Line2, TypeName} | Fields]}, Module,\n           ParseState) ->\n    {RecordType, ParseState2} = parse_type_list(Fields, Module, ParseState),\n    {{record, Module, TypeName, RecordType}, ParseState2};\nparse_type({typed_record_field, {record_field, _Line,\n                                 {atom, _Line2, FieldName}}, Field}, Module,\n           ParseState) ->\n    {FieldType, ParseState2} = parse_type(Field, Module, ParseState),\n    {{typed_record_field, FieldName, FieldType}, ParseState2};\nparse_type({typed_record_field, {record_field, _Line,\n                                 {atom, _Line2, FieldName}, _Default}, Field},\n           Module, ParseState) ->\n    {FieldType, ParseState2} = parse_type(Field, Module, ParseState),\n    {{typed_record_field, FieldName, FieldType}, ParseState2};\nparse_type({type, _, field_type, [{atom, _, FieldName}, Field]}, Module, ParseState) ->\n    {FieldType, ParseState2} = parse_type(Field, Module, ParseState),\n    {{field_type, FieldName, FieldType}, ParseState2};\nparse_type({record_field, _Line, {atom, _Line2, FieldName}}, _Module, ParseState) ->\n    {{untyped_record_field, FieldName}, ParseState};\nparse_type({record_field, _Line, {atom, _Line2, FieldName}, _Default}, _Module,\n           ParseState) ->\n    {{untyped_record_field, FieldName}, ParseState};\nparse_type(TypeSpecs, Module, ParseState) when is_list(TypeSpecs) ->\n    case hd(TypeSpecs) of\n        {typed_record_field, _, _} ->\n            {RecordType, ParseState2} = parse_type_list(TypeSpecs,\n                                                        Module, ParseState),\n            {{record, RecordType}, ParseState2};\n        {record_field, _, _} ->\n            {RecordType, ParseState2} = parse_type_list(TypeSpecs,\n                                                        Module, ParseState),\n            {{record, RecordType}, ParseState2};\n        {record_field, _, _, _} ->\n            {RecordType, ParseState2} = parse_type_list(TypeSpecs,\n                                                        Module, ParseState),\n            {{record, RecordType}, ParseState2};\n        _ ->\n            ct:pal(\"potentially unknown type2: ~p~n\", [TypeSpecs]),\n            unknown\n    end;\nparse_type({var, _Line, Atom}, _Module, ParseState) when is_atom(Atom) ->\n    {{var, Atom}, ParseState};\nparse_type({type, _Line, constraint, _Constraint}, _Module, ParseState) ->\n    {{constraint, nyi}, ParseState};\nparse_type({type, _, bounded_fun, [FunType, ConstraintList]}, Module, ParseState) ->\n    {InternalFunType, ParseState2} = parse_type(FunType, Module, ParseState),\n    Foldl = fun (Constraint, {PartialConstraintList, ParseState2a}) ->\n                    {InternalConstraint, ParseState2c} = parse_type(Constraint,\n                                                                    Module,\n                                                                    ParseState2a),\n                    {[InternalConstraint | PartialConstraintList], ParseState2c}\n            end,\n    {Constraints, ParseState3} = lists:foldl(Foldl, {[], ParseState2}, ConstraintList),\n    {{bounded_fun, InternalFunType, Constraints}, ParseState3};\nparse_type({paren_type, _Line, [InnerType]}, Module, ParseState) ->\n    parse_type(InnerType, Module, ParseState);\nparse_type({type, _Line, identifier, L}, _Module, ParseState) when is_list(L) ->\n    {{builtin_type, identifier}, ParseState};\nparse_type({type, _Line, timeout, L}, _Module, ParseState) when is_list(L) ->\n    {{builtin_type, timeout}, ParseState};\nparse_type({type, _Line, bitstring, L}, _Module, ParseState) when is_list(L) ->\n    {{builtin_type, bitstring}, ParseState};\nparse_type({type, _Line, maybe_improper_list, L}, _Module, ParseState) when is_list(L) ->\n    {{builtin_type, maybe_improper_list}, ParseState};\nparse_type({user_type, Line, TypeName, L}, Module, ParseState) ->\n    parse_type({type, Line, TypeName, L}, Module, ParseState);\nparse_type({type, _Line, map, MapFields}, Module, ParseState) when is_list(MapFields) ->\n    %% ct:pal(\"type assoc map ~p:~p~n~w~n~w~n~w~n\", [Module, map, MapFields, erlang:get(current_module), _Line]),\n    {Fields, NextParseState}\n        = lists:foldl(fun (FieldType, {FieldList, State}) ->\n                              case FieldType of\n                                  {type, _, map_field_assoc, [NameType, Type]} ->\n                                      {TheNameTypeSpec, NewParseState} = parse_type(NameType, Module, State),\n                                      {TheTypeSpec, NewParseState2} = parse_type(Type, Module, NewParseState),\n                                      {[{assoc_map_field, TheNameTypeSpec, TheTypeSpec} | FieldList], NewParseState2};\n                                  {type, _, map_field_exact, [NameType, Type]} ->\n                                      {TheNameTypeSpec, NewParseState} = parse_type(NameType, Module, State),\n                                      {TheTypeSpec, NewParseState2} = parse_type(Type, Module, NewParseState),\n                                      {[{exact_map_field, TheNameTypeSpec, TheTypeSpec} | FieldList], NewParseState2};\n                                  _ ->\n                                      ct:pal(\"unknown map field: ~p\", [FieldType]),\n                                      throw(parse_error)\n                              end\n                      end,\n                      {[], ParseState}, MapFields),\n    {{type_assoc_map, Fields}, NextParseState};\nparse_type({type, _Line, TypeName, L}, Module, ParseState) ->\n    % ct:pal(\"type1 ~p:~p~n~w~n~w~n~w~n\", [Module, TypeName, L, erlang:get(current_module), _Line]),\n    case tester_parse_state:is_known_type(Module, TypeName, length(L), ParseState) of\n        true ->\n            {{typedef, Module, TypeName, L}, ParseState};\n        false ->\n            {{typedef, Module, TypeName, L},\n             tester_parse_state:add_unknown_type(Module, TypeName, length(L), ParseState)}\n    end;\n%% parse_type({ann_type,_Line,[Left,Right]}, _Module, ParseState) ->\n%%     {{ann_type, [Left, Right]}, ParseState};\nparse_type({union, L}, Module, ParseState) ->\n    Foldl = fun (NextType, {TypeList, AParseState}) ->\n                    {TheTypeSpec, NewParseState} = parse_type(NextType, Module, AParseState),\n                    {[TheTypeSpec | TypeList], NewParseState}\n            end,\n    {Types, NextParseState} = lists:foldl(Foldl, {[], ParseState}, L),\n    {{union, Types}, NextParseState};\nparse_type(TypeSpec, Module, ParseState) ->\n    ct:pal(\"unknown type ~p in module ~p~n\", [TypeSpec, Module]),\n    throw(unknown_type),\n    {unknown, ParseState}.\n\n-spec parse_type_list(list(type_spec()), module(), tester_parse_state:state()) ->\n                             {list(type_spec()), tester_parse_state:state()}.\nparse_type_list(List, Module, ParseState) ->\n    case List of\n        [] ->\n            {[], ParseState};\n        [Head | Tail] ->\n            {Type, ParseState2} = parse_type(Head, Module, ParseState),\n            {TypeList, ParseState3} = parse_type_list(Tail, Module, ParseState2),\n            {[Type | TypeList], ParseState3}\n    end.\n\nparse_constraints([], Substitutions) ->\n    Substitutions;\nparse_constraints([ConstraintType | Rest], Substitutions) ->\n    case ConstraintType of\n        {type,_,constraint,[{atom,_,is_subtype},[{var,_,Variable},Type]]} ->\n            case gb_trees:lookup({var, Variable}, Substitutions) of\n                {value,Val} ->\n                    case equal_types(Val, Type) of\n                        true ->\n                            NewSubstitutions = gb_trees:enter({var, Variable}, Type, Substitutions),\n                            parse_constraints(Rest, NewSubstitutions);\n                        false ->\n                            throw({parse_error, Val})\n                    end;\n                none ->\n                    NewSubstitutions = gb_trees:insert({var, Variable}, Type, Substitutions),\n                    parse_constraints(Rest, NewSubstitutions)\n            end;\n        _ ->\n            ct:pal(\"unknown constraint ~w\", [ConstraintType]),\n            parse_constraints(Rest, Substitutions)\n    end.\n\n\n% type equality minus line number\nequal_types(Left, Right)  when is_list(Left) andalso is_list(Right) ->\n    lists:all(fun ({L, R}) ->\n                      equal_types(L, R)\n              end, lists:zip(Left, Right));\nequal_types({type,_, Type, LeftList}, {type,_, Type, RightList} ) ->\n    equal_types(LeftList, RightList);\nequal_types({user_type,_, Type, LeftList}, {user_type,_, Type, RightList} ) ->\n    equal_types(LeftList, RightList);\nequal_types(_, _) ->\n    false.\n\n"
  },
  {
    "path": "test/tester_global_state.erl",
    "content": "%  @copyright 2010-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    global state for tester\n%% @end\n%% @version $Id$\n-module(tester_global_state).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([register_type_checker/3,\n         unregister_type_checker/1,\n         get_type_checker/1]).\n-export([register_value_creator/4,\n         unregister_value_creator/1,\n         get_value_creator/1]).\n-export([set_last_call/4, reset_last_call/1]).\n-export([log_last_calls/0]).\n\n-include(\"tester.hrl\").\n-include(\"unittest.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% handle global state, e.g. specific handlers\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec register_type_checker(type_spec(), module(), Fun::atom()) -> true.\nregister_type_checker(Type, Module, Fun) ->\n    unittest_global_state:insert({type_checker, Type}, {Module, Fun}).\n\n-spec unregister_type_checker(type_spec()) -> true | ok.\nunregister_type_checker(Type) ->\n    unittest_global_state:delete({type_checker, Type}).\n\n-spec get_type_checker(type_spec()) -> failed | {module(), Fun::atom()}.\nget_type_checker(Type) ->\n    unittest_global_state:lookup({type_checker, Type}).\n\n-spec register_value_creator(type_spec(), module(), Fun::atom(), Arity::non_neg_integer()) -> true.\nregister_value_creator(Type, Module, Function, Arity) ->\n    unittest_global_state:insert({value_creator, Type}, {Module, Function, Arity}).\n\n-spec unregister_value_creator(type_spec()) -> true | ok.\nunregister_value_creator(Type) ->\n    unittest_global_state:delete({value_creator, Type}).\n\n-spec get_value_creator(type_spec()) -> failed | {module(), Fun::atom(), Arity::non_neg_integer()}.\nget_value_creator(Type) ->\n    unittest_global_state:lookup({value_creator, Type}).\n\n-spec set_last_call(Thread::pos_integer(), module(), Fun::atom(), Args::list()) -> true.\nset_last_call(Thread, Module, Function, Args) ->\n    unittest_global_state:insert({last_call, Thread, self()}, {Module, Function, Args}).\n\n-spec reset_last_call(Thread::pos_integer()) -> true | ok.\nreset_last_call(Thread) ->\n    case unittest_global_state:lookup({last_call, Thread, self()}) of\n        failed -> ok;\n        _ ->  unittest_global_state:delete({last_call, Thread, self()})\n    end,\n    unittest_global_state:delete({thread, Thread, self()}).\n\n-spec log_last_calls() -> ok.\nlog_last_calls() ->\n    _ = [begin\n             case unittest_global_state:lookup({last_call, ThreadNr, ThreadPid}) of\n                 failed -> ok;\n                 {Module, Function, Args} ->\n                     ct:pal(\"Last call by tester (thread ~B-~p):~n\"\n                            \"~.0p:~.0p(~.0p).\",\n                            [ThreadNr, ThreadPid, Module, Function, Args])\n             end\n         end || {ThreadNr, ThreadPid} <- unittest_global_state:take_registered_threads()],\n    ok.\n"
  },
  {
    "path": "test/tester_helper.erl",
    "content": "%  @copyright 2010-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    helper functions for tester's users\n%% @end\n%% @version $Id$\n-module(tester_helper).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([\n         % instrument a module\n         load_with_export_all/1,\n         load_without_export_all/1,\n         % helper\n         get_src_and_flags_for_module/1,\n         get_src_and_flags_for_module/2\n        ]).\n\n-include(\"tester.hrl\").\n-include(\"unittest.hrl\").\n\n\n-spec load_with_export_all(Module::module()) -> ok.\nload_with_export_all(Module) ->\n    MyOptions = [return_errors,\n                 export_all,\n                 binary],\n    ct:pal(\"Reload ~p module with 'export_all'.~n\", [Module]),\n    reload_with_options(Module, MyOptions).\n\n-spec load_without_export_all(Module::module()) -> ok.\nload_without_export_all(Module) ->\n    MyOptions = [return_errors,\n                 binary],\n    ct:pal(\"Reload ~p module normally.~n\", [Module]),\n    reload_with_options(Module, MyOptions).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% reload function\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n% @doc we assume the standard scalaris layout, i.e. we are currently\n% in a ct_run... directory underneath a scalaris checkout. The ebin\n% directory should be in ../ebin\n\nreload_with_options(Module, MyOptions) ->\n    {Src, Options} = get_src_and_flags_for_module(Module),\n    %ct:pal(\"~p\", [file:get_cwd()]),\n    %ct:pal(\"~p\", [Options]),\n    {ok, CurCWD} = file:get_cwd(),\n    ok = fix_cwd_scalaris(),\n    case compile:file(Src, [{i, \"./include\"} | lists:append(MyOptions, Options)]) of\n        {ok,_ModuleName,Binary} ->\n            %% config no longer needs explicit code reload callbacks\n            %% case Module of\n            %%     config -> sys:suspend(config);\n            %%     _ -> ok\n            %% end,\n            {module, Module} = code:load_binary(Module, Src, Binary),\n            code:soft_purge(Module), %% remove old code\n%% check_old_code not available in Erlang < R14B04\n%%             Old = [ X || X <- erlang:loaded(),\n%%                          true =:= erlang:check_old_code(X)],\n%%             case Old of\n%%                 [] -> ok;\n%%                 _ -> ct:pal(\"Some modules have old code after 2nd soft_purge: ~.0p~n\", [Old])\n%%             end,\n            %% config no longer needs explicit code reload callbacks\n            %% case Module of\n            %%     config -> sys:resume(config);\n            %%     _ -> ok\n            %% end,\n            %ct:pal(\"~p\", [code:is_loaded(Module)]),\n            ok;\n        {ok,_ModuleName,Binary,_Warnings} ->\n            %ct:pal(\"~p\", [_Warnings]),\n            %% config no longer needs explicit code reload callbacks\n            %% case Module of\n            %%     config -> sys:suspend(config);\n            %%     _ -> ok\n            %% end,\n            {module, Module} = erlang:load_module(Module, Binary),\n            code:soft_purge(Module), %% remove old code\n%% check_old_code not available in Erlang < R14B04\n            %% Old = [ X || X <- erlang:loaded(), true =:= erlang:check_old_code(X)],\n            %% case Old of\n            %%     [] -> ok;\n            %%     _ -> ct:pal(\"Some modules have old code after 2nd soft_purge: ~.0p~n\", [Old])\n            %% end,\n            %% config no longer needs explicit code reload callbacks\n            %% case Module of\n            %%     config -> sys:resume(config);\n            %%     _ -> ok\n            %% end,\n            %ct:pal(\"~w\", [erlang:load_module(Module, Binary)]),\n            ok;\n        X ->\n            ct:pal(\"1: ~p\", [X]),\n            ok\n    end,\n    ok = file:set_cwd(CurCWD),\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% misc. helper functions\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec get_src_and_flags_for_module(module())\n        -> {FileName::string(), Flags::list()}.\nget_src_and_flags_for_module(Module) ->\n    get_src_and_flags_for_module(Module, [\"ebin\", \"test\"]).\n\n-spec get_src_and_flags_for_module(module(), RelativePaths::[string(),...])\n        -> {FileName::string(), Flags::list()}.\nget_src_and_flags_for_module(Module, [Path | PathL]) ->\n    % assume we are in a sub-directory of $SCALARIS to find the beam file\n    Res = beam_lib:chunks(\n            lists:append([\"../\", Path, \"/\", erlang:atom_to_list(Module), \".beam\"]),\n            [compile_info]),\n    case Res of\n        {ok, {Module, [{compile_info, Options}]}} ->\n            {source, Source} = lists:keyfind(source, 1, Options),\n            {options, Opts} = lists:keyfind(options, 1, Options),\n            {Source, Opts};\n        _ when PathL =/= [] ->\n            get_src_and_flags_for_module(Module, PathL);\n        X ->\n            ct:pal(\"~w ~p\", [Module, X]),\n            ct:pal(\"~p\", [file:get_cwd()]),\n            timer:sleep(1000),\n            ct:fail(unknown_module)\n    end.\n\n% @doc set cwd to $SCALARIS\n-spec fix_cwd_scalaris() -> ok | {error, Reason::file:posix()}.\nfix_cwd_scalaris() ->\n    file:set_cwd(\"..\").\n"
  },
  {
    "path": "test/tester_parse_state.erl",
    "content": "%  @copyright 2010-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    value collector for test generator\n%% @end\n%% @version $Id$\n-module(tester_parse_state).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-export([new_parse_state/0,\n\n         % add fun info to state\n         find_fun_info/4,\n\n         get_unknown_types/1, has_unknown_types/1,\n\n\n         % add types\n         add_type_spec/4, add_unknown_type/4,\n\n         % add values\n         add_atom/2, add_binary/2, add_float/2, add_integer/2, add_string/2,\n\n         % get values\n         get_atoms/1, get_binaries/1, get_floats/1,\n         get_strings/1, get_non_empty_strings/1,\n         get_integers/1, get_pos_integers/1, get_neg_integers/1, get_non_neg_integers/1,\n\n         reset_unknown_types/1,\n\n         is_known_type/4, lookup_type/2, lookup_fun_type/2,\n\n         % compact state\n         finalize/1]).\n\n-include(\"unittest.hrl\").\n-include(\"tester.hrl\").\n\n-export_type([state/0]).\n\n-record(parse_state,\n        {type_infos        = gb_trees:empty() :: gb_trees:tree(type_name(), {var_list(), type_spec()}),\n         unknown_types     = gb_sets:new()    :: gb_sets:set(type_name()) | {Length::non_neg_integer(), [type_name()]},\n         atoms             = gb_sets:new()    :: gb_sets:set(atom()) | {Length::non_neg_integer(), [atom()]},\n         binaries          = gb_sets:new()    :: gb_sets:set(binary()) | {Length::non_neg_integer(), [binary()]},\n         integers          = gb_sets:new()    :: gb_sets:set(integer()) | {Length::non_neg_integer(), [integer()]},\n         pos_integers      = null             :: null     | {Length::non_neg_integer(), [pos_integer()]},\n         neg_integers      = null             :: null     | {Length::non_neg_integer(), [neg_integer()]},\n         non_neg_integers  = null             :: null     | {Length::non_neg_integer(), [non_neg_integer()]},\n         floats            = gb_sets:new()    :: gb_sets:set(float()) | {Length::non_neg_integer(), [float()]},\n         non_empty_strings = gb_sets:new()    :: gb_sets:set(nonempty_string()) | {Length::non_neg_integer(), [nonempty_string()]}\n        }).\n-opaque state() :: #parse_state{}.\n\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% find fun info\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec find_fun_info(module(), atom(), non_neg_integer(), state()) -> state().\nfind_fun_info(Module, Func, Arity, ParseState) ->\n    _ParseState2 = try tester_collect_function_info:collect_fun_info(Module,\n                                                                    Func,\n                                                                    Arity,\n                                                                    ParseState)\n    catch\n        throw:Term2 -> ?ct_fail(\"exception (throw) in ~p:~p(): ~p~n\",\n                                [Module, Func,\n                                 {exception, {Term2, util:get_stacktrace(),\n                                              util:get_linetrace()}}]);\n        % special handling for exits that come from a ct:fail() call:\n        exit:{test_case_failed, Reason2} ->\n            ?ct_fail(\"error ~p:~p/~p failed with ~p~n\",\n                     [Module, Func, Arity, {Reason2, util:get_stacktrace(),\n                                            util:get_linetrace()}]);\n        exit:Reason2 -> ?ct_fail(\"exception (exit) in ~p:~p(): ~p~n\",\n                                 [Module, Func,\n                                  {exception, {Reason2, util:get_stacktrace(),\n                                               util:get_linetrace()}}]);\n        error:Reason2 -> ?ct_fail(\"exception (error) in ~p:~p(): ~p~n\",\n                                  [Module, Func,\n                                   {exception, {Reason2, util:get_stacktrace(),\n                                                util:get_linetrace()}}])\n    end.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% parse state\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec new_parse_state() -> state().\nnew_parse_state() ->\n    #parse_state{unknown_types = gb_sets:singleton({type, tester, test_any, 0})}.\n\n-spec has_unknown_types(state()) -> boolean().\nhas_unknown_types(#parse_state{unknown_types=UnknownTypes}) ->\n    case gb_sets:is_set(UnknownTypes) of\n        true -> not gb_sets:is_empty(UnknownTypes);\n        _    -> element(1, UnknownTypes) =/= 0\n    end.\n\n-spec get_unknown_types(state()) -> {Length::non_neg_integer(), [type_name()]}.\nget_unknown_types(#parse_state{unknown_types=UnknownTypes}) ->\n    case gb_sets:is_set(UnknownTypes) of\n        true -> UnknownTypesList = gb_sets:to_list(UnknownTypes),\n                {gb_sets:size(UnknownTypes), UnknownTypesList};\n        _    -> UnknownTypes\n    end.\n\n-spec get_atoms(state()) -> {Length::non_neg_integer(), [atom()]}.\nget_atoms(#parse_state{atoms=Atoms}) ->\n    case gb_sets:is_set(Atoms) of\n        true -> AtomsList = gb_sets:to_list(Atoms),\n                {gb_sets:size(Atoms), AtomsList};\n        _    -> Atoms\n    end.\n\n-spec get_binaries(state()) -> {Length::non_neg_integer(), [binary()]}.\nget_binaries(#parse_state{binaries=Binaries}) ->\n    case gb_sets:is_set(Binaries) of\n        true -> BinariesList = gb_sets:to_list(Binaries),\n                {gb_sets:size(Binaries), BinariesList};\n        _    -> Binaries\n    end.\n\n-spec get_floats(state()) -> {Length::non_neg_integer(), [float()]}.\nget_floats(#parse_state{floats=Floats}) ->\n    case gb_sets:is_set(Floats) of\n        true -> FloatsList = gb_sets:to_list(Floats),\n                {gb_sets:size(Floats), FloatsList};\n        _    -> Floats\n    end.\n\n-spec get_integers(state()) -> {Length::non_neg_integer(), [integer()]}.\nget_integers(#parse_state{integers=Integers}) ->\n    case gb_sets:is_set(Integers) of\n        true -> IntegerList = gb_sets:to_list(Integers),\n                {gb_sets:size(Integers), IntegerList};\n        _    -> Integers\n    end.\n\n-spec get_pos_integers(state()) -> {Length::non_neg_integer(), Integers::[pos_integer()]}.\nget_pos_integers(#parse_state{integers=Integers, pos_integers=null}) ->\n    IntegerList = [I || I <- gb_sets:to_list(Integers), I > 0],\n    {erlang:length(IntegerList), IntegerList};\nget_pos_integers(#parse_state{pos_integers=PosIntegers}) ->\n    PosIntegers.\n\n-spec get_neg_integers(state()) -> {Length::non_neg_integer(), Integers::[neg_integer()]}.\nget_neg_integers(#parse_state{integers=Integers, neg_integers=null}) ->\n    IntegerList = [I || I <- gb_sets:to_list(Integers), I < 0],\n    {erlang:length(IntegerList), IntegerList};\nget_neg_integers(#parse_state{neg_integers=NegIntegers}) ->\n    NegIntegers.\n\n-spec get_non_neg_integers(state()) -> {Length::non_neg_integer(), [non_neg_integer()]}.\nget_non_neg_integers(#parse_state{integers=Integers, non_neg_integers=null}) ->\n    IntegerList = [I || I <- gb_sets:to_list(Integers), I >= 0],\n    {erlang:length(IntegerList), IntegerList};\nget_non_neg_integers(#parse_state{non_neg_integers=NonNegIntegers}) ->\n    NonNegIntegers.\n\n-spec get_strings(state()) -> {Length::non_neg_integer(), [string()]}.\nget_strings(#parse_state{non_empty_strings=Strings}) ->\n    case gb_sets:is_set(Strings) of\n        true -> StringList = [\"\" | gb_sets:to_list(Strings)],\n                {gb_sets:size(Strings) + 1, StringList};\n        _    -> {L, S} = Strings,\n                {L + 1, [\"\" | S]}\n    end.\n\n-spec get_non_empty_strings(state()) -> {Length::non_neg_integer(), [nonempty_string()]}.\nget_non_empty_strings(#parse_state{non_empty_strings=Strings}) ->\n    case gb_sets:is_set(Strings) of\n        true -> StringsList = gb_sets:to_list(Strings),\n                {gb_sets:size(Strings), StringsList};\n        _    -> Strings\n    end.\n\n-spec add_type_spec(type_name(), type_spec(), var_list(), state()) -> state().\nadd_type_spec(TypeName, TypeSpec, VarList, #parse_state{type_infos=TypeInfos} = ParseState) ->\n    NewTypeInfos = gb_trees:enter(TypeName, {var_type, VarList, TypeSpec}, TypeInfos),\n    ParseState#parse_state{type_infos=NewTypeInfos}.\n\n-spec add_unknown_type(module(), atom(), arity(), state()) -> state().\nadd_unknown_type(TypeModule, TypeName, Arity, #parse_state{unknown_types=UnknownTypes} = ParseState) ->\n    ParseState#parse_state{unknown_types=\n                           gb_sets:add_element({type, TypeModule, TypeName, Arity},\n                                               UnknownTypes)}.\n\n-spec reset_unknown_types(state()) -> state().\nreset_unknown_types(ParseState) ->\n    ParseState#parse_state{unknown_types=gb_sets:new()}.\n\n-spec is_known_type(module(), atom(), arity(), state()) -> boolean().\nis_known_type(TypeModule, TypeName, Arity, #parse_state{type_infos=TypeInfos}) ->\n    gb_trees:is_defined({type, TypeModule, TypeName, Arity}, TypeInfos).\n\n-spec add_atom(atom(), state()) -> state().\nadd_atom(Atom, #parse_state{atoms=Atoms} = ParseState) ->\n    ParseState#parse_state{atoms=gb_sets:add_element(Atom, Atoms)}.\n\n-spec add_binary(binary(), state()) -> state().\nadd_binary(Binary, #parse_state{binaries=Binaries} = ParseState) ->\n    ParseState#parse_state{binaries=gb_sets:add_element(Binary, Binaries)}.\n\n-spec add_float(float(), state()) -> state().\nadd_float(Float, #parse_state{floats=Floats} = ParseState) ->\n    ParseState#parse_state{floats=gb_sets:add_element(Float, Floats)}.\n\n-spec add_integer(integer(), state()) -> state().\nadd_integer(Integer, #parse_state{integers=Integers} = ParseState) ->\n    ParseState#parse_state{integers=gb_sets:add_element(Integer, Integers)}.\n\n-spec add_string(string(), state()) -> state().\nadd_string(\"\", ParseState) ->\n    ParseState;\nadd_string(String, #parse_state{non_empty_strings=Strings} = ParseState) ->\n    ParseState#parse_state{non_empty_strings=gb_sets:add_element(String, Strings)}.\n\n-spec lookup_type(type_name(), state()) -> {value, type_spec()} | none.\nlookup_type(Type, #parse_state{type_infos=TypeInfos}) ->\n    gb_trees:lookup(Type, TypeInfos).\n\n-spec lookup_fun_type(type_name(), state()) -> \n                             {value, {var_type, [], {union_fun, [test_fun_type(),...]}}} | none.\nlookup_fun_type(Type, #parse_state{type_infos=TypeInfos}) ->\n    gb_trees:lookup(Type, TypeInfos).\n\n%% @doc Compact the state for use during value creation. Do this after having\n%%      collected all values in order to increase performance!\n-spec finalize(state()) -> state().\nfinalize(#parse_state{unknown_types=UnknownTypes, atoms=Atoms,\n                      binaries=Binaries, integers=Integers, floats=Floats,\n                      non_empty_strings=Strings} = ParseState) ->\n    IntegersList = gb_sets:to_list(Integers),\n    NonNegIntegersList = [I || I <- IntegersList, I >= 0],\n    PosIntegersList = [I || I <- NonNegIntegersList, I > 0],\n    NegIntegersList = [I || I <- IntegersList, I < 0],\n    UnknownTypesList = gb_sets:to_list(UnknownTypes),\n    AtomsList = gb_sets:to_list(Atoms),\n    BinariesList = gb_sets:to_list(Binaries),\n    FloatsList = gb_sets:to_list(Floats),\n    StringsList = gb_sets:to_list(Strings),\n    ParseState#parse_state{unknown_types     = {gb_sets:size(UnknownTypes), UnknownTypesList},\n                           atoms             = {gb_sets:size(Atoms), AtomsList},\n                           binaries          = {gb_sets:size(Binaries), BinariesList},\n                           integers          = {gb_sets:size(Integers), IntegersList},\n                           pos_integers      = {erlang:length(PosIntegersList), PosIntegersList},\n                           neg_integers      = {erlang:length(NegIntegersList), NegIntegersList},\n                           non_neg_integers  = {erlang:length(NonNegIntegersList), NonNegIntegersList},\n                           floats            = {gb_sets:size(Floats), FloatsList},\n                           non_empty_strings = {gb_sets:size(Strings), StringsList}}.\n"
  },
  {
    "path": "test/tester_randomized_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{groups, tests, db_dht_SUITE, [tester_tests]}.\n{groups, tests, db_ets_SUITE, [tester_tests]}.\n{groups, tests, intervals_SUITE, [tester_tests]}.\n{groups, tests, node_details_SUITE, [tester_tests]}.\n{groups, tests, rrd_SUITE, [tester_tests]}.\n{groups, tests, rrepair_SUITE, [tester_tests]}.\n\n{suites, tests, [unittest_helper_SUITE]}.\n{suites, tests, [db_generator_SUITE]}.\n\n% @todo add dht_node_move_SUITE\n% @todo add util_SUITE"
  },
  {
    "path": "test/tester_type_checker.erl",
    "content": "%  @copyright 2010-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Florian Schintke <schintke@zib.de>\n%% @doc    check whether a given value is of a given type\n%% @end\n-module(tester_type_checker).\n-author('schuett@zib.de').\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-export([check/3]).\n-export([log_error/1]).\n-export([render_type/1]).\n\n-include(\"tester.hrl\").\n%-include(\"unittest.hrl\").\n\n-spec check(term(), type_spec(), tester_parse_state:state()) -> true | {false, term()}.\ncheck(true, {atom, true}, _ParseState) ->\n    true;\ncheck(true, bool, _ParseState) ->\n    true;\ncheck(Value, Type, ParseState) ->\n    case inner_check(Value, Type, [], ParseState) of\n        true ->\n            true;\n        {false, _CheckStack} = R ->\n            %%ct:pal(\"Type check failed: ~.0p\", [_CheckStack]),\n            R\n    end.\n\ninner_check(Value, Type, CheckStack, ParseState) ->\n%%    ct:pal(\"new inner_check(~.0p, ~.0p)\", [Value, Type]),\n    case tester_global_state:get_type_checker(Type) of\n        failed ->\n            inner_check_(Value, Type, CheckStack, ParseState);\n        {Module, Function} ->\n            %ct:pal(\"using ~p:~p for checking ~p\", [Module, Function, Value]),\n            case inner_check_(Value, Type, CheckStack, ParseState) of\n                true ->\n                    case apply(Module, Function, [Value]) of\n                        true ->\n                            true;\n                        false ->\n                            {false,\n                             [{Value,\n                               structural_test_ok_your_registered_type_checker_failed,\n                               {Module, Function}}\n                              | CheckStack]}\n                    end;\n                X ->\n                    X\n            end\n    end.\n\ninner_check_(Value, Type, CheckStack, ParseState) ->\n    case Type of\n        arity ->\n            inner_check(Value, byte, CheckStack, ParseState);\n        atom ->\n            check_basic_type(Value, atom, CheckStack, ParseState,\n                             fun erlang:is_atom/1, no_atom);\n        {atom, _Atom} ->\n            check_atom(Value, Type, CheckStack, ParseState);\n        {binary, []} ->\n            check_basic_type(Value, Type, CheckStack, ParseState,\n                             fun erlang:is_binary/1, no_binary);\n        {builtin_type, bitstring} ->\n            check_basic_type(Value, Type, CheckStack, ParseState,\n                             fun erlang:is_bitstring/1, no_bitstring);\n        bool ->\n            check_basic_type(Value, Type, CheckStack, ParseState,\n                             fun erlang:is_boolean/1, no_boolean);\n        byte ->\n            inner_check(Value, {range, {integer, 0}, {integer, 255}},\n                        CheckStack, ParseState);\n        {builtin_type, array_array, ValueType} ->\n            case array:is_array(Value) of\n                true ->\n                    check_list(array:to_list(Value), % [Value]\n                               {list, ValueType},\n                               CheckStack, ParseState);\n                false -> {false, [{Value, array_is_array_returned_false} | CheckStack]}\n            end;\n        {builtin_type, module} ->\n            inner_check(Value, atom, CheckStack, ParseState);\n        {builtin_type, dict_dict, KeyType, ValueType} ->\n            % there is no is_dict/1, so try some functions on the dict to check\n            try\n                _ = dict:size(Value),\n                _ = dict:find('$non_existing_key', Value),\n                _ = dict:store('$non_existing_key', '$value', Value),\n                check_list(dict:to_list(Value), % [{Key, Value}]\n                           {list, {tuple, [KeyType, ValueType]}},\n                           CheckStack, ParseState)\n            catch _:_ -> {false, [{Value, dict_functions_thrown} | CheckStack]}\n            end;\n        {builtin_type, gb_trees_tree, KeyType, ValueType} ->\n            % there is no is_gb_tree/1, so try some functions on the tree to check\n            try\n                _ = gb_trees:size(Value),\n                _ = gb_trees:is_defined('$non_existing_key', Value),\n                _ = gb_trees:enter('$non_existing_key', '$value', Value),\n                check_list(gb_trees:to_list(Value), % [{Key, Value}]\n                           {list, {tuple, [KeyType, ValueType]}},\n                           CheckStack, ParseState)\n            catch _:_ -> {false, [{Value, gb_trees_functions_thrown} | CheckStack]}\n            end;\n        {builtin_type, gb_sets_set, ValueType} ->\n            % there is no is_gb_set/1, so try some functions on the set to check\n            try\n                _ = gb_sets:size(Value),\n                _ = gb_sets:is_member('$non_existing_key', Value),\n                _ = gb_sets:add('$non_existing_key', Value),\n                check_list(gb_sets:to_list(Value), % [Key]\n                           {list, ValueType},\n                           CheckStack, ParseState)\n            catch _:_ -> {false, [{Value, gb_sets_functions_thrown} | CheckStack]}\n            end;\n        {builtin_type, ordsets_ordset, ValueType} ->\n            check_list(Value,\n                       {list, ValueType},\n                       CheckStack, ParseState);\n        {builtin_type, map} ->\n            % there is no is_map/1, so try some functions on the map to check\n            try check_map(Value, CheckStack)\n            catch _:_ -> {false, [{Value, map_functions_thrown} | CheckStack]}\n            end;\n        float ->\n            check_basic_type(Value, Type, CheckStack, ParseState,\n                             fun erlang:is_float/1, no_float);\n        {'fun', {product, _ParamTypes}, _ResultType} ->\n            check_fun(Value, Type, CheckStack, ParseState);\n        integer ->\n            check_basic_type(Value, Type, CheckStack, ParseState,\n                             fun erlang:is_integer/1, no_integer);\n        {integer, Int} ->\n            check_basic_type_with_prop(\n              Value, Type, CheckStack, ParseState,\n              fun erlang:is_integer/1, not_the_integer,\n              fun(X) -> Int =:= X end);\n        {list, _InnerType} ->\n            check_list(Value, Type, CheckStack, ParseState);\n        neg_integer ->\n            check_basic_type_with_prop(Value, Type, CheckStack, ParseState,\n                                       fun erlang:is_integer/1, no_neg_integer,\n                                       fun(X) -> 0 > X end);\n        nil ->\n            check_basic_type_with_prop(Value, Type, CheckStack, ParseState,\n                                       fun erlang:is_list/1, no_empty_list,\n                                       fun(X) -> [] =:= X end);\n        node ->\n            check_basic_type(Value, Type, CheckStack, ParseState,\n                             fun erlang:is_atom/1, no_node);\n        {nonempty_list, _InnerType} ->\n            check_list(Value, Type, CheckStack, ParseState);\n        nonempty_string ->\n            %% see http://www.erlang.org/doc/reference_manual/typespec.html\n            inner_check(Value, {nonempty_list,\n                                {range, {integer, 0}, {integer, 16#10ffff}}},\n                        [{Value, nonempty_string} | CheckStack], ParseState);\n        non_neg_integer ->\n            check_basic_type_with_prop(\n              Value, Type, CheckStack, ParseState,\n              fun erlang:is_integer/1, no_non_neg_integer,\n              fun(X) -> 0 =< X end);\n        number ->\n            check_basic_type(Value, Type, CheckStack, ParseState,\n                             fun erlang:is_number/1, no_number);\n        pid ->\n            check_basic_type(Value, Type, CheckStack, ParseState,\n                             fun erlang:is_pid/1, no_pid);\n        pos_integer ->\n            check_basic_type_with_prop(Value, Type, CheckStack, ParseState,\n                                       fun erlang:is_integer/1, no_pos_integer,\n                                       fun(X) -> 0 < X end);\n        {product, TypeList} when is_list(TypeList) ->\n            check_tuple(Value, {tuple, TypeList}, CheckStack, ParseState);\n        {range, {integer, _Min}, {integer, _Max}} ->\n            check_range(Value, Type, CheckStack, ParseState);\n        {record, _Module, _Typedef} ->\n            check_record(Value, Type, CheckStack, ParseState);\n        {record, FieldList} when is_list(FieldList) ->\n            check_record_fields(Value, Type, CheckStack, ParseState);\n        reference ->\n            check_basic_type(Value, Type, CheckStack, ParseState,\n                             fun erlang:is_reference/1, no_reference);\n        tid ->\n            % built-in < R14; otherwise ets:tid()\n            inner_check(Value, integer, CheckStack, ParseState);\n        {tuple, Tuple} when is_list(Tuple) ->\n            check_tuple(Value, Type, CheckStack, ParseState);\n        {tuple, Tuple} when is_tuple(Tuple) ->\n            inner_check(Value, Tuple, CheckStack, ParseState);\n        {typedef, ets, tid, []} -> % tid is reference in R20+\n            inner_check_(Value, integer, CheckStack, ParseState);\n        {typedef, dict, dict, []} ->\n            check_list(dict:to_list(Value), % [Key,Value]\n                       {list, {tuple,\n                               [{typedef, tester, test_any, []}, {typedef, tester, test_any, []}]}},\n                       CheckStack, ParseState);\n        {typedef, orddict, orddict, []} ->\n            check_list(orddict:to_list(Value), % [Key,Value]\n                       {list, {tuple,\n                               [{typedef, tester, test_any, []}, {typedef, tester, test_any, []}]}},\n                       CheckStack, ParseState);\n        {typedef, _Module, _TypeName, []} ->\n            check_typedef(Value, Type, CheckStack, ParseState);\n        {union, _Union} ->\n            check_union(Value, Type, CheckStack, ParseState);\n        {var_type, [], InnerType} ->\n            % @todo add substitution\n            inner_check_(Value, InnerType, CheckStack, ParseState);\n        _ ->\n            ct:pal(\"Type checker: unsupported type: ~p\", [Type]),\n            ?DBG_ASSERT2(false, \"unknown type\"),\n            {false, [{type_checker_unsupported_type, Type} | CheckStack]}\n    end.\n\ncheck_basic_type(Value, Type, CheckStack, _ParseState,\n                 TypeCheck, Report) ->\n    case TypeCheck(Value) of\n        true -> true;\n        false -> {false, [{Value, Report, Type} | CheckStack]}\n    end.\n\ncheck_basic_type_with_prop(Value, Type, CheckStack, ParseState,\n                           TypeCheck, Report,\n                           ValCheck) ->\n    case check_basic_type(Value, Type, CheckStack, ParseState,\n                          TypeCheck, Report) of\n        true ->\n            case ValCheck(Value) of\n                true -> true;\n                false -> {false, [{Value, Report, Type} | CheckStack]}\n            end;\n        {false, _} = R -> R\n    end.\n\ncheck_typedef(_Value, {typedef, tester, test_any, []}, _, _) ->\n    true;\ncheck_typedef(Value, {typedef, Module, TypeName, []} = T,\n              CheckStack, ParseState) ->\n    case tester_parse_state:lookup_type({type, Module, TypeName, length([])}, ParseState) of\n        none ->\n            ct:pal(\"error in check_typedef ~w:~w\", [Module, TypeName]),\n            {false, [{tester_lookup_type_failed,\n                      {Module, TypeName}} | CheckStack]};\n        {value, {var_type, [], InnerType}} ->\n            case inner_check(Value, InnerType,\n                             CheckStack, ParseState) of\n                {false, ErrStack} ->\n                    {false, [{T, defined_as, InnerType, ErrStack}]};\n                true -> true\n            end\n    end.\n\ncheck_range(Value, {range, {integer, Min}, {integer, Max}} = T,\n            CheckStack, _ParseState) ->\n    case is_integer(Value) of\n        true ->\n            case (Min =< Value) andalso (Max >= Value) of\n                true -> true;\n                false ->\n                    {false,\n                     [{Value, not_in, T} | CheckStack ]}\n            end;\n        false ->\n            {false, [{Value, no_integer_in_range, T} | CheckStack]}\n    end.\n\ncheck_record(Value, {record, Module, TypeName} = T, CheckStack, ParseState) ->\n    case tester_parse_state:lookup_type(T, ParseState) of\n        none ->\n            ct:pal(\"error in check_record\"),\n            {false, [{tester_lookup_type_failed,\n                      {Module, TypeName}} | CheckStack]};\n        {value, {var_type, [], {record, FieldList} = _InnerType}} ->\n            %% check record name here (add it as record field in front)\n            case inner_check(Value, {record,\n                        [ {typed_record_field, tag, {atom, TypeName}}\n                          | FieldList ]},\n                             CheckStack, ParseState) of\n                {false, ErrStack} ->\n                    {false, [{Value, record_field_mismatch, T, ErrStack}]};\n                true -> true\n            end\n    end.\n\n\ncheck_record_fields(Value, {record, FieldList}, CheckStack, ParseState)\n  when is_list(FieldList) ->\n    %% [{typed_record_field,FieldName, Type}]\n    {_, _, TypeList} = lists:unzip3(FieldList),\n    check_tuple(Value, {tuple, TypeList},\n                CheckStack, ParseState).\n\ncheck_list(Value, {list, InnerType} = T, CheckStack, ParseState) ->\n    case is_list(Value) of\n        true ->\n            case check_list_iter(Value, InnerType, CheckStack, ParseState, 1) of\n                {false, ErrStack} ->\n                    {false, [{Value, list_element_mismatch, T, ErrStack}]};\n                true -> true\n            end;\n        false ->\n            {false, [{Value, no_list, T} | CheckStack]}\n    end;\ncheck_list(Value, {nonempty_list, InnerType} = T, CheckStack, ParseState) ->\n    case is_list(Value) andalso [] =/= Value of\n        true ->\n            case check_list_iter(Value, InnerType, CheckStack, ParseState, 1) of\n                {false, ErrStack} ->\n                    {false, [{Value, list_element_mismatch, T, ErrStack}]};\n                        true -> true\n                    end;\n        false ->\n            {false, [{Value, no_nonempty_list, T} | CheckStack]}\n    end.\n\n\ncheck_list_iter([], _Type, _CheckStack, _ParseState, _Count) ->\n    true;\ncheck_list_iter([Value | Tail], Type, CheckStack, ParseState, Count) ->\n    case inner_check(Value, Type, CheckStack, ParseState) of\n        true ->\n            check_list_iter(Tail, Type, CheckStack, ParseState, Count + 1);\n        {false, Stack} ->\n            {false, [{Value, list_element, Count, Type, Stack}]}\n    end.\n\ncheck_atom(Value, {atom, Atom} = T, CheckStack, _ParseState) ->\n    case is_atom(Value) of\n        true ->\n            case Value =:= Atom of\n                true -> true;\n                false ->\n                    {false, [{Value, not_the_atom, T} | CheckStack]}\n            end;\n        false ->\n            {false, [{Value, no_atom, T} | CheckStack]}\n    end.\n\ncheck_tuple(Value, {tuple, Tuple} = T, CheckStack, ParseState) ->\n    case is_tuple(Value) of\n        true ->\n            case erlang:tuple_size(Value) =:= erlang:length(Tuple) of\n                true ->\n                    case check_tuple_iter(tuple_to_list(Value), Tuple,\n                                          CheckStack, ParseState, 1) of\n                        {false, ErrStack} ->\n                            {false, [{Value, tuple_element_mismatch, T, ErrStack}]};\n                        true -> true\n                    end;\n                _ -> {false, [{Value, not_same_arity, T} | CheckStack]}\n            end;\n        false ->\n            {false, [{Value, no_tuple, T} | CheckStack]}\n    end.\n\ncheck_tuple_iter([], [], _CheckStack, _ParseState, _Count) ->\n    true;\ncheck_tuple_iter([Value | Tail], [Type | TypeTail], CheckStack,\n                 ParseState, Count) ->\n    case inner_check(Value, Type, CheckStack, ParseState) of\n        true ->\n            check_tuple_iter(Tail, TypeTail, CheckStack, ParseState, Count + 1);\n        {false, Stack} ->\n            {false, [{Value, tuple_element, Count, Type, Stack}]}\n    end.\n\ncheck_union(Value, {union, Union}, CheckStack, ParseState) ->\n    case lists:foldl(\n           fun(Type, Res) ->\n                   case Res of\n                       true -> true;\n                       {false, Stack} ->\n                           case inner_check(Value, Type, [], ParseState) of\n                               true -> true;\n                               {false, NewStack} ->\n                                   {false, Stack ++ NewStack}\n                           end\n                   end\n           end, {false, []}, Union) of\n        true -> true;\n        {false, UnionStack} ->\n            {false, [{Value, no_union_variant_matched,\n                      {union, Union}, UnionStack} | CheckStack]}\n    end.\n\n-ifdef(with_maps).\ncheck_map(Value, _CheckStack) ->\n    _ = maps:size(Value),\n    _ = maps:find('$non_existing_key', Value),\n    _ = maps:put('$non_existing_key', '$value', Value),\n    true.\n-else.\ncheck_map(Value, CheckStack) ->\n    {false, [{Value, no_map_support} | CheckStack]}.\n-endif.\n\ncheck_fun(Value, {'fun', {product, ParamTypes} = Type, _ResultType},\n          CheckStack, _ParseState) ->\n    case is_function(Value) of\n        false ->\n            {false, [{Value, no_function, Type} | CheckStack]};\n        true ->\n            {arity, Arity} = erlang:fun_info(Value, arity),\n            case Arity =:= length(ParamTypes) of\n                false ->\n                    {false, [{{Value, Arity},\n                              fun_with_wrong_arity_for, Type}\n                             | CheckStack]};\n                true -> true\n            end\n    end.\n\n-spec log_error(any()) -> ok.\nlog_error([ErrorReport]) ->\n    ErrorIOList = log_error(ErrorReport, \"\"),\n    ct:pal(lists:flatten(\"TypeCheck:~n\" ++ ErrorIOList)),\n    ok;\nlog_error(ErrorReport) ->\n    ErrorIOList = log_error(ErrorReport, \"\"),\n    ct:pal(lists:flatten(\"TypeCheck:~n\" ++ ErrorIOList)),\n    ok.\n\nlog_error([], _Prefix) -> \"\";\nlog_error([ErrorReport], Prefix) ->\n    log_error(ErrorReport, Prefix ++ \"  \");\nlog_error([H|T], Prefix) ->\n    log_error(H, Prefix ++ \" |\")\n        ++ log_error(T, Prefix);\nlog_error({Val, tuple_element, I, Type, Tree}, Prefix) ->\n    io_lib:format(\n      Prefix ++ \"`-Error:         tuple element ~.0p mismatched~n\" ++\n      Prefix ++ \"  Expected type: ~s~n\" ++\n      Prefix ++ \"  Value:         ~.0p~n\" ++\n      Prefix ++ \"  Because:~n\",\n      [I, render_type(Type), Val])\n      ++ log_error(Tree, Prefix ++ \" \");\nlog_error({Val, list_element, I, Type, Tree}, Prefix) ->\n    io_lib:format(\n      Prefix ++ \"`-Error:         list element ~.0p mismatched~n\" ++\n      Prefix ++ \"  Expected type: ~s~n\" ++\n      Prefix ++ \"  Value:         ~.0p~n\" ++\n      Prefix ++ \"  Because:~n\",\n      [I, render_type(Type), Val])\n      ++ log_error(Tree, Prefix ++ \" \");\nlog_error({Type, defined_as, InnerType, Tree}, Prefix) ->\n    io_lib:format(\n      Prefix ++ \"`-Expected type: ~s~n\" ++\n      Prefix ++ \"  DefinedAs:     ~s~n\",\n      [render_type(Type), render_type(InnerType)])\n      ++ log_error(Tree, Prefix ++ \" \");\nlog_error({Val, Error, Type, Tree}, Prefix) when is_list(Tree) ->\n    io_lib:format(\n      Prefix ++ \"`-Error:         ~.0p~n\" ++\n      Prefix ++ \"  Expected type: ~s~n\" ++\n      Prefix ++ \"  Value:         ~.0p~n\" ++\n      Prefix ++ \"  Because:~n\",\n      [Error, render_type(Type), Val])\n      ++ log_error(Tree, Prefix ++ \" \");\nlog_error({Val, Error, Type}, Prefix) ->\n    io_lib:format(\n      Prefix ++ \"`-Error:         ~.0p~n\" ++\n      Prefix ++ \"  Expected type: ~s~n\" ++\n      Prefix ++ \"  Value:         ~.0p~n\",\n      [Error, render_type(Type), Val]);\nlog_error(ErrorReport, Prefix) ->\n    io_lib:format(Prefix ++ \"`-***~.0p~n\", [ErrorReport]).\n\n-spec render_type(any()) -> string().\nrender_type(Type) ->\n    lists:flatten(render_type_(Type)).\n\nrender_type_(Basic) when is_atom(Basic) ->\n    io_lib:format(\"~p()\", [Basic]);\nrender_type_({atom, Atom}) ->\n    io_lib:format(\"'~p'\", [Atom]);\nrender_type_({builtin_type, Type}) ->\n    io_lib:format(\"erlang:~p()\", [Type]);\nrender_type_({'fun', {product, _ParamTypes} = P, ResultType}) ->\n    \"fun(\" ++ render_type(P) ++ \") -> \" ++\n        render_type_(ResultType);\nrender_type_({integer, Int}) ->\n    io_lib:format(\"~p\", [Int]);\nrender_type_({list, InnerType}) ->\n    \"[\" ++ render_type_(InnerType) ++ \"]\";\nrender_type_({nonempty_list, InnerType}) ->\n    \"nonempty_list: [\" ++ render_type_(InnerType) ++ \"]\";\nrender_type_({product, TypeList}) when is_list(TypeList) ->\n    list_separated_by(TypeList, \", \");\nrender_type_({range, {integer, Min}, {integer, Max}}) ->\n    io_lib:format(\"~p..~p\", [Min, Max]);\nrender_type_({record, Module, RecordName}) ->\n    io_lib:format(\"~p:#~p{}\", [Module, RecordName]);\nrender_type_({record, FieldList}) when is_list(FieldList) ->\n    list_separated_by(FieldList, \", \");\nrender_type_({tuple, Tuple}) when is_list(Tuple) ->\n    \"{\" ++ list_separated_by(Tuple, \", \") ++ \"}\";\nrender_type_({tuple, Tuple}) when is_tuple(Tuple) ->\n    \"{\" ++ render_type_(Tuple) ++ \"}\";\nrender_type_({typedef, Module, TypeName}) ->\n    io_lib:format(\"~p:~p()\", [Module, TypeName]);\nrender_type_({typed_record_field, Name, Type}) ->\n    io_lib:format(\"...#{~p :: ~s, ...}\", [Name, render_type_(Type)]);\nrender_type_({union, Union}) ->\n    list_separated_by(Union, \" | \");\nrender_type_(Other) ->\n    io_lib:format(\"***~.0p***\", [Other]).\n\nlist_separated_by([], _Separator) -> \"\";\nlist_separated_by([Type], _Separator) ->\n    render_type_(Type);\nlist_separated_by([H|T], Separator) ->\n    render_type_(H) ++ Separator ++ list_separated_by(T, Separator).\n"
  },
  {
    "path": "test/tester_value_collector.erl",
    "content": "%  @copyright 2010-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    value collector for test generator\n%%         see http://www.erlang.org/doc/apps/erts/absform.html\n%% @end\n%% @version $Id$\n-module(tester_value_collector).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"unittest.hrl\").\n\n-export([parse_expression/2]).\n\n-spec parse_expression(any(), any()) -> any().\nparse_expression(Expr, State) ->\n    try\n        parse_expression_intern(Expr, State)\n    catch\n        unknown_expression ->\n            ct:pal(\"failed to parse expression ~p\", [Expr]),\n            exit(foobar)%;\n        %error:Reason ->\n        %    ct:pal(\"failed to parse expression ~p (error:~p)\", [Expr, Reason]),\n        %    exit(foobar)\n    end.\n\n-spec parse_expression_intern(any(), any()) -> any().\nparse_expression_intern(Clauses, ParseState) when is_list(Clauses) ->\n    lists:foldl(fun parse_expression_intern/2, ParseState, Clauses);\n% binary comprehension\nparse_expression_intern({bc, _, LeftElement, RightElements}, ParseState) ->\n    parse_expression_intern(LeftElement, lists:foldl(fun parse_expression_intern/2, ParseState, RightElements));\n% binary generator\nparse_expression_intern({b_generate, _, Pattern, Expression}, ParseState) ->\n    parse_expression_intern(Pattern, parse_expression_intern(Expression, ParseState));\nparse_expression_intern({call, _, Fun, Parameters}, ParseState) ->\n    parse_expression_intern(Parameters, parse_expression_intern(Fun, ParseState));\nparse_expression_intern({'case', _, Value, Clauses}, ParseState) ->\n    parse_expression_intern(Clauses, parse_expression_intern(Value, ParseState));\nparse_expression_intern({clause, _Line, Pattern, _, Clause}, ParseState) ->\n    parse_expression_intern(Clause, parse_expression_intern(Pattern, ParseState));\nparse_expression_intern({cons, _, Head, Tail}, ParseState) ->\n    parse_expression_intern(Tail, parse_expression_intern(Head, ParseState));\nparse_expression_intern({'fun', _, {clauses, Clauses}}, ParseState) ->\n    lists:foldl(fun parse_expression_intern/2, ParseState, Clauses);\nparse_expression_intern({named_fun, _Loc, _Name, Clauses}, ParseState) -> % EEP37: Funs with names\n    lists:foldl(fun parse_expression_intern/2, ParseState, Clauses);\nparse_expression_intern({'fun', _, {function, _Name, _Arity}}, ParseState) ->\n    ParseState;\nparse_expression_intern({'fun', _, {function, _Module, _Name, _Arity}}, ParseState) ->\n    ParseState;\nparse_expression_intern({generate, _, Expression, L}, ParseState) ->\n    parse_expression_intern(L, parse_expression_intern(Expression, ParseState));\nparse_expression_intern({'if', _, Clauses}, ParseState) ->\n    lists:foldl(fun parse_expression_intern/2, ParseState, Clauses);\nparse_expression_intern({lc, _, Expression, Qualifiers}, ParseState) ->\n    parse_expression_intern(Qualifiers, parse_expression_intern(Expression, ParseState));\nparse_expression_intern({match, _, Left, Right}, ParseState) ->\n    parse_expression_intern(Left, parse_expression_intern(Right, ParseState));\nparse_expression_intern({op, _, _, Value}, ParseState) ->\n    parse_expression_intern(Value, ParseState);\nparse_expression_intern({op, _, _, Left, Right}, ParseState) ->\n    parse_expression_intern(Left, parse_expression_intern(Right, ParseState));\nparse_expression_intern({'receive', _, Clauses}, ParseState) ->\n    lists:foldl(fun parse_expression_intern/2, ParseState, Clauses);\nparse_expression_intern({'receive', _, Clauses, Timeout, AfterBody}, ParseState) ->\n    ParseState2 = lists:foldl(fun parse_expression_intern/2, ParseState, Clauses),\n    ParseState3 = parse_expression_intern(Timeout, ParseState2),\n    parse_expression_intern(AfterBody, ParseState3);\nparse_expression_intern({record, _, _Name, Fields}, ParseState) ->\n    lists:foldl(fun parse_expression_intern/2, ParseState, Fields);\nparse_expression_intern({record, _, _Variable, _Name, Fields}, ParseState) ->\n    lists:foldl(fun parse_expression_intern/2, ParseState, Fields);\nparse_expression_intern({record_field, _, Name, Value}, ParseState) ->\n    parse_expression_intern(Value, parse_expression_intern(Name, ParseState));\nparse_expression_intern({record_field, _, Name, _RecordType, Value}, ParseState) ->\n    parse_expression_intern(Value, parse_expression_intern(Name, ParseState));\nparse_expression_intern({record_index, _, _Name, Value}, ParseState) ->\n    parse_expression_intern(Value, ParseState);\nparse_expression_intern({'try', _, Body, CaseClauses, CatchClauses, AfterBody}, ParseState) ->\n    ParseState2 = parse_expression_intern(Body, ParseState),\n    ParseState3 = lists:foldl(fun parse_expression_intern/2, ParseState2, CaseClauses),\n    ParseState4 = lists:foldl(fun parse_expression_intern/2, ParseState3, CatchClauses),\n    parse_expression_intern(AfterBody, ParseState4);\nparse_expression_intern({'catch', _, Expression}, ParseState) ->\n    parse_expression_intern(Expression, ParseState);\nparse_expression_intern({block, _, ExpressionList}, ParseState) ->\n    lists:foldl(fun parse_expression_intern/2, ParseState, ExpressionList);\nparse_expression_intern({tuple, _, Values}, ParseState) ->\n    lists:foldl(fun parse_expression_intern/2, ParseState, Values);\nparse_expression_intern({remote, _, Module, Fun}, ParseState) ->\n    parse_expression_intern(Module, parse_expression_intern(Fun, ParseState));\nparse_expression_intern({var, _, _Variable}, ParseState) ->\n    %ct:pal(\"~w\", [_Variable]),\n    ParseState;\nparse_expression_intern({atom, _, Atom}, ParseState) ->\n    tester_parse_state:add_atom(Atom, ParseState);\nparse_expression_intern({bin, _, [{bin_element,_,{string,_,String},default,default}]},\n                 ParseState) ->\n    tester_parse_state:add_binary(list_to_binary(String), ParseState);\nparse_expression_intern({bin, _, [{bin_element,_,{var,_,_},{integer,_,_},default}]},\n                 ParseState) ->\n    ParseState;\nparse_expression_intern({bin, _, [{bin_element,_,{var,_,_},{integer,_,_},[binary]}]},\n                 ParseState) ->\n    ParseState;\nparse_expression_intern({bin, _, [{bin_element,_,{var,_,_},default,[binary]}]},\n                 ParseState) ->\n    ParseState;\nparse_expression_intern({bin, _, []}, ParseState) ->\n    ParseState;\nparse_expression_intern({bin, _, Elements}, ParseState) when is_list(Elements) ->\n    BinStrings =\n        [String ||\n         {bin_element,_,{string,_,String},default,default} <- Elements],\n    lists:foldl(\n      fun(String, Acc) ->\n              tester_parse_state:add_binary(list_to_binary(String), Acc)\n      end, ParseState, BinStrings);\nparse_expression_intern({float, _, Float}, ParseState) ->\n    tester_parse_state:add_float(Float, ParseState);\nparse_expression_intern({char, _, _Char}, ParseState) ->\n    % @todo\n    ParseState;\nparse_expression_intern({integer, _, Integer}, ParseState) ->\n    tester_parse_state:add_integer(Integer, ParseState);\nparse_expression_intern({map, _, L}, ParseState) when is_list(L) -> % map:new(X)\n    parse_expression_intern(L, ParseState);\nparse_expression_intern({map, _, Var, L}, ParseState) when is_list(L) -> % map:new(X)\n    parse_expression_intern(Var, parse_expression_intern(L, ParseState));\nparse_expression_intern({map_field_exact, _, Left, Right}, ParseState) -> % map field\n    parse_expression_intern(Left, parse_expression_intern(Right, ParseState));\nparse_expression_intern({map_field_assoc, _, Left, Right}, ParseState) -> % map field\n    parse_expression_intern(Left, parse_expression_intern(Right, ParseState));\nparse_expression_intern({nil, _}, ParseState) ->\n    tester_parse_state:add_atom(nil, ParseState);\nparse_expression_intern({string, _, String}, ParseState) ->\n    tester_parse_state:add_string(String, ParseState);\n\nparse_expression_intern(Expression, ParseState) ->\n    ct:pal(\"unknown expression: ~w in ~w:~w\", [Expression,\n                                               erlang:get(module),\n                                               erlang:get(fun_name)]),\n    throw(unknown_expression),\n    ParseState.\n\n"
  },
  {
    "path": "test/tester_value_creator.erl",
    "content": "%  @copyright 2010-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    value creator for tester\n%% @end\n%% @version $Id$\n-module(tester_value_creator).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-include(\"unittest.hrl\").\n-include(\"tester.hrl\").\n\n-export([create_value/3, create_value_wo_scalaris/3]).\n\nlist_length_min() -> 0.\nlist_length_max() -> 5.\n\ninteger_min() -> 0. %don't change this! (negative values will be created between -max and +max)\ninteger_max() -> 5.\n\n%% @doc Create a random value of the given type, taking global Scalaris types\n%%      into account.\n-spec create_value(type_spec(), non_neg_integer(), tester_parse_state:state()) -> term().\ncreate_value(Type, Size, ParseState) ->\n    FlatType = case Type of\n                   {var_type, [], InnerType} ->\n                       InnerType;\n                   _ -> Type\n               end,\n    case tester_value_creator_scalaris:create_value(FlatType, Size, ParseState) of\n        failed ->\n            create_value_wo_scalaris(FlatType, Size, ParseState);\n        {value, Value} ->\n            Value\n    end.\n\n%% @doc Create a random value of the given type, not taking global Scalaris\n%%      types into account at the top level of the given type spec.\n-spec create_value_wo_scalaris(type_spec(), non_neg_integer(), tester_parse_state:state()) -> term().\ncreate_value_wo_scalaris(Type, Size, ParseState) ->\n    %ct:pal(\"create_value ~p\", [Type]),\n    case tester_global_state:get_value_creator(Type) of\n        failed ->\n            try create_value_(Type, Size, ParseState)\n            catch\n                throw:{error, Msg} ->\n                    NewMsg = tester_type_checker:render_type(Type),\n                    throw({error, [NewMsg|Msg]});\n                throw:_Term ->\n                    NewMsg = tester_type_checker:render_type(Type),\n                    throw({error, [NewMsg]})\n            end;\n        Creator -> case custom_value_creator(Creator, Type, Size, ParseState) of\n                       failed -> create_value_(Type, Size, ParseState);\n                       {value, Value} -> Value\n                   end\n    end.\n\n-spec create_value_(type_spec(), non_neg_integer(), tester_parse_state:state()) -> term().\n\n\ncreate_value_(atom, _Size, ParseState) ->\n    create_val_50rand_50coll(\n      ParseState, fun tester_parse_state:get_atoms/1,\n      fun() -> lists:nth(randoms:rand_uniform(1, 5), [one, two, three, four]) end);\ncreate_value_({atom, Value}, _Size, _ParseState) ->\n    Value;\ncreate_value_({binary, []}, Size, ParseState) ->\n    create_val_50rand_50coll(\n      ParseState, fun tester_parse_state:get_binaries/1,\n      fun() -> list_to_binary(create_value({list, {range, {integer, 0}, {integer, 16#ff}}}, Size, ParseState)) end);\ncreate_value_({builtin_type, bitstring}, Size, ParseState) ->\n    Binary = create_value_({binary, []}, Size, ParseState),\n    RandByte = randoms:rand_uniform(0, 256),\n    RandBits = randoms:rand_uniform(0, 8),\n    <<Binary/binary,RandByte:RandBits>>;\ncreate_value_(bool, _Size, _ParseState) ->\n    case randoms:rand_uniform(0, 2) of\n        0 -> false;\n        1 -> true\n    end;\ncreate_value_({builtin_type, array_array, ValueType}, Size, ParseState) ->\n    L = create_value({list, ValueType}, Size, ParseState),\n    array:from_list(L);\ncreate_value_({builtin_type, gb_trees_tree, KeyType, ValueType}, Size, ParseState) ->\n    L = create_value({list,\n                      {tuple,\n                       [KeyType, ValueType]}}, Size, ParseState),\n    gb_trees:from_orddict(orddict:from_list(L));\ncreate_value_({builtin_type, gb_sets_set, ValueType}, Size, ParseState) ->\n    L = create_value({list, ValueType}, Size, ParseState),\n    gb_sets:from_list(L);\ncreate_value_({builtin_type, set_set, ValueType}, Size, ParseState) ->\n    L = create_value({list, ValueType}, Size, ParseState),\n    sets:from_list(L);\ncreate_value_({builtin_type, map}, _Size, ParseState) ->\n    create_map(ParseState);\ncreate_value_({builtin_type, maybe_improper_list}, Size, ParseState) ->\n    create_value_({list, {typedef, tester, test_any, []}}, Size, ParseState);\ncreate_value_({builtin_type, module}, _Size, _ParseState) ->\n    Values = [element(1, X) || X <- code:all_loaded()],\n    lists:nth(randoms:rand_uniform(1, length(Values) + 1), Values);\ncreate_value_({field_type, _Name, Type}, Size, ParseState) ->\n    create_value(Type, Size, ParseState);\ncreate_value_(float, _Size, ParseState) ->\n    create_val_50rand_50coll(\n      ParseState, fun tester_parse_state:get_floats/1,\n      fun() -> randoms:rand_uniform(-5, 5) * (randoms:rand_uniform(0, 30323) / 30323.0) end);\ncreate_value_(integer, _Size, ParseState) ->\n    create_val_50rand_50coll(\n      ParseState, fun tester_parse_state:get_integers/1,\n      fun() -> randoms:rand_uniform(integer_min(), integer_max() * 2 + 1) - integer_max() end);\ncreate_value_({integer, Value}, _Size, _ParseState) ->\n    Value;\ncreate_value_({list, Type}, Size, ParseState) ->\n    ListLength = erlang:min(Size, randoms:rand_uniform(list_length_min(),\n                                               list_length_max() + 1)),\n    case ListLength of\n        0 ->\n            [];\n        _ ->\n            NewSize = erlang:max(1, (Size - ListLength) div ListLength),\n            [create_value(Type, NewSize, ParseState) || _ <- lists:seq(1, ListLength)]\n    end;\ncreate_value_(nil, _Size, _ParseState) ->\n    [];\ncreate_value_(node, _Size, _ParseState) ->\n    % @todo\n    node();\ncreate_value_({nonempty_list, Type}, Size, ParseState) ->\n    ListLength =\n        erlang:max(1, erlang:min(Size,\n                                 randoms:rand_uniform(1, list_length_max() + 1))),\n    NewSize = erlang:max(1, (Size - ListLength) div ListLength),\n    [create_value(Type, NewSize, ParseState) || _ <- lists:seq(1, ListLength)];\ncreate_value_(nonempty_string, Size, ParseState) ->\n    RandStringFun =\n        fun() ->\n                ListLength0 = randoms:rand_uniform(list_length_min(),\n                                                   list_length_max() + 1),\n                ListLength = erlang:max(1, erlang:min(Size, ListLength0)),\n                Type = {range, {integer, 0}, {integer, 16#10ffff}},\n                NewSize = erlang:max(1, (Size - ListLength) div ListLength),\n                [create_value(Type, NewSize, ParseState) || _ <- lists:seq(1, ListLength)]\n        end,\n    create_val_50rand_50coll(\n      ParseState, fun tester_parse_state:get_non_empty_strings/1,\n      RandStringFun);\n% 0..\ncreate_value_(non_neg_integer, _Size, ParseState) ->\n    create_val_50rand_50coll(\n      ParseState, fun tester_parse_state:get_non_neg_integers/1,\n      fun() -> randoms:rand_uniform(0, integer_max() + 1) end);\ncreate_value_(pid, _Size, _ParseState) ->\n    case erlang:whereis(tester_pseudo_proc) of\n        Pid when is_pid(Pid) -> Pid\n    end;\n% 1..\ncreate_value_(pos_integer, _Size, ParseState) ->\n    create_val_50rand_50coll(\n      ParseState, fun tester_parse_state:get_pos_integers/1,\n      fun() -> randoms:rand_uniform(1, integer_max() + 1) end);\n% ..-1\ncreate_value_(neg_integer, _Size, ParseState) ->\n    create_val_50rand_50coll(\n      ParseState, fun tester_parse_state:get_neg_integers/1,\n      fun() -> -(randoms:rand_uniform(1, integer_max() + 1)) end);\ncreate_value_({product, []}, _Size, _ParseState) ->\n    [];\ncreate_value_({product, Types}, Size, ParseState) ->\n    NewSize = erlang:max(1, (Size - length(Types)) div length(Types)),\n    [create_value(Type, NewSize, ParseState) || Type <- Types];\ncreate_value_({range, {integer, Low}, {integer, High}}, _Size, _ParseState) ->\n    randoms:rand_uniform(Low, High + 1);\ncreate_value_({record, Module, TypeName}, Size, ParseState) ->\n    case tester_parse_state:lookup_type({record, Module, TypeName}, ParseState) of\n        {value, {var_type, [], RecordType}} ->\n            create_record_value(TypeName, RecordType, Size, ParseState);\n        none ->\n            ?ct_fail(\"error: unknown record type: ~p:~p\", [Module, TypeName])\n    end;\ncreate_value_({record, _Module, TypeName, FieldTypes}, Size, ParseState) ->\n    create_record_value(TypeName, FieldTypes, Size, ParseState);\ncreate_value_({tuple, Types}, Size, ParseState) when is_list(Types) ->\n    case Types of\n        [] ->\n            {};\n        _ ->\n            NewSize = erlang:max(1, (Size - length(Types)) div length(Types)),\n            Values = [create_value(Type, NewSize, ParseState) || Type <- Types],\n            erlang:list_to_tuple(Values)\n    end;\ncreate_value_({tuple, {typedef, tester, test_any, []}}, Size, ParseState) ->\n    Values = create_value({list, {typedef, tester, test_any, []}}, Size, ParseState),\n    erlang:list_to_tuple(Values);\n%%create_value({typedef, tester, test_any}, Size, TypeInfo) ->\n    %% @todo\ncreate_value_({typedef, ets, tid, []}, Size, ParseState) -> % reference since R20\n    create_value_(integer, Size, ParseState);\ncreate_value_({typedef, orddict, orddict, []}, Size, ParseState) ->\n    KVs = create_value({list,\n                        {tuple,\n                         [{typedef, tester, test_any, []}, {typedef, tester, test_any, []}]}},\n                       Size, ParseState),\n    orddict:from_list(KVs);\ncreate_value_({typedef, Module, TypeName, TypeList}, Size, ParseState) ->\n    %ct:pal(\"typedef~n~w~n~w~n\", [TypeName, TypeList]),\n    %ct:pal(\"~w\", [{type, Module, TypeName, length(TypeList)}]),\n    case tester_parse_state:lookup_type({type, Module, TypeName, length(TypeList)}, ParseState) of\n        {value, {var_type, [], TypeSpec}} ->\n            create_value(TypeSpec, Size, ParseState);\n        {value, {var_type, VarList, TypeSpec}} ->\n            Subs = tester_variable_substitutions:substitutions_from_list(VarList, TypeList),\n            RealTypeSpec = tester_variable_substitutions:substitute(TypeSpec, Subs),\n            %ct:pal(\"type before sub~n~w~n\", [TypeSpec]),\n            %ct:pal(\"type after sub~n~w~n\", [RealTypeSpec]),\n            create_value(RealTypeSpec, Size, ParseState);\n        none ->\n            ?ct_fail(\"error: unknown type ~p:~p~n~w~n\", [Module, TypeName, TypeList])\n    end;\ncreate_value_({typed_record_field, _Name, Type}, Size, ParseState) ->\n    create_value(Type, Size, ParseState);\ncreate_value_({union, Types}, Size, ParseState) ->\n    Length = length(Types),\n    create_value(lists:nth(randoms:rand_uniform(1, Length + 1), Types),\n                 Size, ParseState);\ncreate_value_(Unknown , _Size, _ParseState) ->\n    ct:pal(\"Cannot create type (you could register a custom value creator):~n\"\n           \"~.0p~n\", [Unknown]),\n    throw(function_clause).\n\n%% @doc creates a record value\n-spec create_record_value(RecordName :: atom(),\n                          {record, Types :: [type_spec()]} | [type_spec()],\n                          Size :: non_neg_integer(),\n                          ParseState :: tester_parse_state:state()) -> tuple().\ncreate_record_value(RecordName, Types, Size, ParseState) when is_list(Types) ->\n    RecordLength = length(Types),\n    NewSize = erlang:max(1, (Size - RecordLength) div RecordLength),\n    RecordElements = [create_value(Type, NewSize, ParseState) || Type <- Types],\n    erlang:list_to_tuple([RecordName | RecordElements]);\ncreate_record_value(RecordName, {record, Types}, Size, ParseState) ->\n    create_record_value(RecordName, Types, Size, ParseState).\n\n-spec create_val_50rand_50coll(\n        ParseState::tester_parse_state:state(),\n        Getter::fun((tester_parse_state:state()) -> {Length::non_neg_integer(), Values::[T]}),\n        RandValFun::fun(() -> T)) -> T.\ncreate_val_50rand_50coll(ParseState, Getter, RandValFun) ->\n    case randoms:rand_uniform(0, 2) of\n        0 -> % take one of the collected values (if possible)\n            {Length, Values} = Getter(ParseState),\n            case Length of\n                0 -> RandValFun();\n                _ -> lists:nth(randoms:rand_uniform(1, Length + 1), Values)\n            end;\n        1 ->\n            RandValFun()\n    end.\n\n-ifdef(with_maps).\n-spec create_map(ParseState::tester_parse_state:state()) -> map().\ncreate_map(ParseState) ->\n    L = create_value({list, {tuple,\n                             [{typedef, tester, test_any, []},\n                              {typedef, tester, test_any, []}]}}, 0, ParseState),\n    maps:from_list(L).\n-else.\n-spec create_map(ParseState::tester_parse_state:state()) -> no_return().\ncreate_map(_ParseState) ->\n    erlang:error('maps_not_available').\n-endif.\n\n-spec custom_value_creator({module(), atom(), non_neg_integer()}, any(),\n                           non_neg_integer(), tester_parse_state:state()) -> failed | {value, any()}.\ncustom_value_creator({Module, Fun, Arity} = _Creator, Type, Size, ParseState) ->\n    % get spec of Creator\n    case tester_parse_state:lookup_type({'fun', Module,\n                                         Fun, Arity},\n                                        ParseState) of\n        {value, {var_type, [], {union_fun, FunTypes}}} ->\n            {'fun', ArgType, _ResultType} = util:randomelem(FunTypes),\n            Args = try {value, tester_value_creator:create_value(ArgType, Size, ParseState)}\n                   catch\n                       Error:Reason ->\n                           ct:pal(\"could not generate input for custom value creator ~p:~p/~p: ~p:~p\",\n                                  [Module, Fun, Arity, Error, Reason]),\n                           failed\n                   end,\n            case Args of\n                failed ->\n                    failed;\n                {value, Args2} ->\n                    try {value, erlang:apply(Module, Fun, Args2)}\n                    catch\n                        Error2:Reason2 ->\n                            ct:pal(\"exception in custom value creator ~p:~p/~p(~p): ~p:~p\",\n                                   [Module, Fun, Arity, Args, Error2, Reason2]),\n                            failed\n                    end\n            end;\n        none ->\n            ct:pal(\"could not call custom value creator (~p:~p/~p) for ~p\",\n                   [Module, Fun, Arity, Type]),\n            failed\n    end.\n"
  },
  {
    "path": "test/tester_value_creator_scalaris.erl",
    "content": "%  @copyright 2014-2017 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Value creator for Scalaris-specific types.\n%% @end\n%% @version $Id$\n-module(tester_value_creator_scalaris).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-include(\"unittest.hrl\").\n-include(\"tester.hrl\").\n\n-export([create_value/3]).\n\n-spec create_value(type_spec(), non_neg_integer(), tester_parse_state:state())\n        -> {value, term()} | failed.\ncreate_value({typedef, comm, reg_name, []}, _Size, _ParseState) ->\n    ?ASSERT2(is_pid(whereis(tester_pseudo_proc)), process__tester_pseudo_proc__must_exist),\n    {value, tester_pseudo_proc};\ncreate_value({typedef, comm, erl_local_pid_with_reply_as, []} = Type, Size, ParseState) ->\n    {Pid, e, _WrongPos, Env} =\n        tester_value_creator:create_value_wo_scalaris(Type, Size, ParseState),\n    Pos = tester_value_creator:create_value_wo_scalaris(\n            {range, {integer, 2}, {integer, tuple_size(Env)}}, Size, ParseState),\n    {value, {Pid, e, Pos, setelement(Pos, Env, '_')}};\ncreate_value({typedef, comm, mypid_with_reply_as, []} = Type, Size, ParseState) ->\n    {Pid, e, _WrongPos, Env} =\n        tester_value_creator:create_value_wo_scalaris(Type, Size, ParseState),\n    Pos = tester_value_creator:create_value_wo_scalaris(\n            {range, {integer, 2}, {integer, tuple_size(Env)}}, Size, ParseState),\n    {value, {Pid, e, Pos, setelement(Pos, Env, '_')}};\ncreate_value({typedef, comm, mypid_plain, []}, _Size, _ParseState) ->\n    ?ASSERT2(is_pid(whereis(tester_pseudo_proc)), process__tester_pseudo_proc__must_exist),\n    case randoms:rand_uniform(0, 2) of\n        0 -> {value, comm:make_global(whereis(tester_pseudo_proc))};\n        1 -> {value, comm:make_global(tester_pseudo_proc)}\n    end;\ncreate_value(_Type, _Size, _ParseState) ->\n    failed.\n"
  },
  {
    "path": "test/tester_variable_substitutions.erl",
    "content": "%  @copyright 2010-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    variable substitutions for tester\n%% @end\n-module(tester_variable_substitutions).\n-author('schuett@zib.de').\n\n-include(\"unittest.hrl\").\n-include(\"tester.hrl\").\n\n-export([substitute/2, substitutions_from_list/2]).\n\n-type fun_spec() :: any().\n\n-spec substitute(fun_spec(), gb_trees:tree({var, atom()}, term())) -> fun_spec().\nsubstitute(FunSpecs, Substitutions) when is_list(FunSpecs)->\n    [substitute(FunSpec, Substitutions) ||  FunSpec <- FunSpecs];\n% type variable\nsubstitute(Var = {var,_Line,VarName}, Substitutions) ->\n    case gb_trees:lookup({var,VarName}, Substitutions) of\n        {value, Substitution} -> Substitution;\n        none ->\n            %% VarName is unknown see e.g. Key at orddict:is_key/2\n            Var\n    end;\n\nsubstitute({type, _Line,bounded_fun, [FunType, _Constraints]}, Substitutions) ->\n    substitute(FunType, Substitutions);\n\n% generic types\nsubstitute({type, Line,TypeType, Types}, Substitutions) ->\n    Types2 = substitute(Types, Substitutions),\n    {type,Line,TypeType,Types2};\n\nsubstitute({tuple, Types}, Substitutions) ->\n    {tuple, substitute(Types, Substitutions)};\n\nsubstitute({builtin_type, Type}, _Substitutions) ->\n    {builtin_type, Type};\n\nsubstitute({union, _Line, Types}, Substitutions) ->\n    {union, substitute(Types, Substitutions)};\n\nsubstitute({union, Types}, Substitutions) ->\n    {union, substitute(Types, Substitutions)};\n\nsubstitute({typedef, Module, Type, Params}, Substitutions) ->\n    {typedef, Module, Type, substitute(Params, Substitutions)};\n\n% user types\nsubstitute({user_type, Line, TypeType, Types}, Substitutions) ->\n    Types2 = substitute(Types, Substitutions),\n    {type,Line,TypeType,Types2};\n\n% special types\nsubstitute({ann_type,_Line,[{var, _Line2, _Name}, Type]}, Substitutions) ->\n    substitute(Type, Substitutions);\nsubstitute({ann_type,Line,[Left,Right]}, Substitutions) ->\n    Left2 = substitute(Left, Substitutions),\n    Right2 = substitute(Right, Substitutions),\n    {ann_type,Line,[Left2,Right2]};\nsubstitute({remote_type,Line,[Left,Right,L]}, Substitutions) ->\n    Left2 = substitute(Left, Substitutions),\n    Right2 = substitute(Right, Substitutions),\n    L2 = [ substitute(Element, Substitutions) || Element <- L],\n    {remote_type,Line,[Left2,Right2,L2]};\nsubstitute(any, _Substitutions) ->\n    any;\n\nsubstitute({paren_type, Line,[{type,_,union, Vars}]}, Substitutions) ->\n    substitute({union, Line, Vars}, Substitutions);\n\n% value types\nsubstitute({atom,Line,Value}, _Substitutions) ->\n    {atom,Line,Value};\nsubstitute({integer,Line,Value}, _Substitutions) ->\n    {integer,Line,Value};\n\nsubstitute(Unknown, Substitutions) ->\n    ct:pal(\"Unknown substitution: ~w\", [Unknown]),\n    ct:pal(\"Known substitutions: ~w\", [gb_trees:to_list(Substitutions)]),\n    throw({subst_error, unknown_expression}),\n    exit(foobar).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% substitutions_from_list\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec substitutions_from_list(list({var, integer(), atom()}), list(term()))\n                             -> gb_trees:tree({var, atom()}, term()).\nsubstitutions_from_list(VarList, TypeList) ->\n    Tree = gb_trees:empty(),\n    substitutions_from_list_(VarList, TypeList, Tree).\n\n\nsubstitutions_from_list_([], [], Tree) ->\n    Tree;\nsubstitutions_from_list_([{var,_Line,V} | VRest], [Type | TRest], Tree) ->\n    substitutions_from_list_(VRest, TRest, gb_trees:enter({var, V}, Type, Tree)).\n\n"
  },
  {
    "path": "test/tx_tm_rtm_SUITE.erl",
    "content": "% @copyright 2008-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for transactions under churn\n%% @end\n%% @version $Id$\n-module(tx_tm_rtm_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\nall() ->\n    [abort_prepared_r,\n     abort_prepared_w,\n     abort_prepared_rc,\n     abort_prepared_rmc,\n     abort_prepared_wmc,\n     tm_crash,\n     tp_crash,\n     all_tp_crash\n    ].\n\nsuite() -> [{timetrap, {seconds, 120}}].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_testcase(TestCase, Config) ->\n    case lists:member(TestCase, [tm_crash, tp_crash, all_tp_crash]) of\n        true ->\n            {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n            unittest_helper:make_ring(\n              4,\n              [{config,\n                [{monitor_perf_interval, 0},  % deactivate monitor_perf,\n                 {log_path, PrivDir},\n                 {replication_factor, 4}]}]),\n            unittest_helper:wait_for_stable_ring(),\n            unittest_helper:wait_for_stable_ring_deep();\n        false ->\n            {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n            unittest_helper:make_symmetric_ring([{config,\n                [{monitor_perf_interval, 0},  % deactivate monitor_perf\n                 {log_path, PrivDir},\n                 {replication_factor, 4}]}]),\n            ok\n    end,\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\ncauses() -> [readlock, writelock, versiondec, versioninc, none].\n\nabort_prepared_r(_) ->\n    %% check all combinations of abort / prepared and the corresponding\n    %% tx decisions\n    %% modify DB and then do operations on the changed DB\n    %% quorum reads should alway succeed\n    _ = [ begin\n              Key = init_new_db_key(\"abort_prepared_r\"),\n              abort_prepared(Key, read, [MC1, MC2, MC3, MC4], {ok, \"abort_prepared_r\"})\n          end\n          || MC1 <- causes(), MC2 <- causes(),\n             MC3 <- causes(), MC4 <- causes()],\n    ok.\n\nabort_prepared_w(_) ->\n    %% check all combinations of abort / prepared and the corresponding\n    %% tx decisions\n    %% modify DB and then do operations on the changed DB\n    _ = [ begin\n              Key = init_new_db_key(\"abort_prepared_w\"),\n              abort_prepared(Key, write, [MC1, MC2, MC3, MC4],\n                             calc_w_outcome(Key, [MC1, MC2, MC3, MC4]))\n          end\n          || MC1 <- causes(), MC2 <- causes(),\n             MC3 <- causes(), MC4 <- causes()],\n    ok.\n\nabort_prepared_rc(_) ->\n    %% check all combinations of abort / prepared and the corresponding\n    %% tx decisions\n    %% modify DB and then do operations on the changed DB\n    _ = [ begin\n              Key = init_new_db_key(\"abort_prepared_rc\"),\n              abort_prepared(Key, read_commit, [MC1, MC2, MC3, MC4],\n                             calc_rc_outcome(Key, [MC1, MC2, MC3, MC4]))\n          end\n          || MC1 <- causes(), MC2 <- causes(),\n             MC3 <- causes(), MC4 <- causes()],\n    ok.\n\nabort_prepared_rmc(_) ->\n     %% modify DB after work_phase, before validation\n     %% read - modify - commit (rmc)\n    _ = [ begin\n              Key = init_new_db_key(\"abort_prepared_rmc\"),\n              {TLog, _} =\n                  api_tx:req_list(api_tx:new_tlog(),\n                                               [{read, Key}]),\n              abort_prepared(Key, {commit_tlog, TLog}, [MC1, MC2, MC3, MC4],\n                             calc_rmc_outcome(Key, [MC1, MC2, MC3, MC4]))\n          end\n          || MC1 <- causes(), MC2 <- causes(),\n             MC3 <- causes(), MC4 <- causes()],\n    ok.\n\nabort_prepared_wmc(_) ->\n     %% modify DB after work_phase, before validation\n     %% write - modify - commit (rmc)\n    _ = [ begin\n              Key = init_new_db_key(\"abort_prepared_wmc\"),\n              {TLog, _} =\n                  api_tx:req_list(api_tx:new_tlog(),\n                                               [{write, Key, \"wmc\"}]),\n              Pattern = [MC1, MC2, MC3, MC4],\n              abort_prepared(Key, {commit_tlog, TLog}, Pattern,\n                             calc_wmc_outcome(Key, Pattern))\n          end\n          || MC1 <- causes(), MC2 <- causes(),\n             MC3 <- causes(), MC4 <- causes()],\n    ok.\n\nabort_prepared(Key, Op, PreOps, ExpectedOutcome) ->\n    Keys = ?RT:get_replica_keys(?RT:hash_key(Key)),\n    DBEntries = get_db_entries(Keys),\n    NewDBEntries =\n        [ begin\n              NewDBEntry =\n                  case PreOp of\n                      readlock -> db_entry:inc_readlock(DBEntry);\n                      writelock -> db_entry:set_writelock(DBEntry, db_entry:get_version(DBEntry));\n                      versiondec -> db_entry:dec_version(DBEntry);\n                      versioninc -> db_entry:inc_version(DBEntry);\n                      none -> DBEntry\n                  end,\n              api_dht_raw:unreliable_lookup(db_entry:get_key(DBEntry),\n                                       {set_key_entry, comm:this(), NewDBEntry}),\n              receive {set_key_entry_reply, NewDBEntry} -> ok end,\n              NewDBEntry\n          end\n          || {DBEntry, PreOp} <- lists:zip(DBEntries, PreOps) ],\n\n    Outcome =\n        case Op of\n            write ->\n                api_tx:write(Key, io_lib:format(\"~p with ~p\", [Op, PreOps]));\n            read ->\n                api_tx:read(Key);\n            read_commit ->\n                {_TLog, [_, Res]} = api_tx:req_list([{read, Key}, {commit}]),\n                Res;\n            {commit_tlog, TLog} ->\n                api_tx:commit(TLog)\n        end,\n    case ExpectedOutcome of\n        ok_or_abort ->\n            %% ct:pal(\"~w with ~w results in ~w~n\", [Op, PreOps, Outcome]),\n            ok;\n        _ ->\n            ?equals_w_note(Outcome, ExpectedOutcome,\n                           io_lib:format(\"~p with ~p and~nDB entries initial: ~p~nafter preops ~p~n and after operation ~p~n\",\n                                         [Op, PreOps, DBEntries, NewDBEntries, get_db_entries(Keys)]))\n    end,\n   ok.\n\ncalc_w_outcome(Key, PreOps) ->\n    NumReadlock =   length([ X || X <- PreOps, X =:= readlock ]),\n    NumWritelock =  length([ X || X <- PreOps, X =:= writelock ]),\n    NumVersionDec = length([ X || X <- PreOps, X =:= versiondec ]),\n    NumVersionInc = length([ X || X <- PreOps, X =:= versioninc ]),\n    NumNone =       length([ X || X <- PreOps, X =:= none ]),\n\n    if (4 =:= NumVersionInc) -> {ok};\n       (4 =:= NumVersionDec) -> {ok};\n       (4 =:= NumNone) -> {ok};\n       (1 =:= NumVersionInc) -> ok_or_abort; % Inc+2xAny => OK, RL|WL+2xAny => ABORT, None+2xAny => OK\n       (1 =:= NumReadlock andalso NumNone =:= 3) -> {ok};\n       (1 =:= NumWritelock andalso NumNone =:= 3) -> {ok};\n       (1 =:= NumVersionDec andalso NumNone =:= 2) -> ok_or_abort; % Dec+2xNone => OK, RL|WL+2xNone => ABORT\n       (1 =:= NumVersionDec andalso NumNone =:= 3) -> {ok};\n\n       (2 =< NumVersionInc) -> {ok};\n       (2 =:= NumVersionDec andalso NumNone =:= 1) -> {ok};\n       (2 =:= NumVersionDec andalso NumNone =:= 2) -> {ok};\n\n       % RL|WL+2xDec => ABORT, 3xDec => OK\n       (3 =:= NumVersionDec andalso 0 =:= NumNone) -> ok_or_abort;\n\n       % RMaj with 3xDec + Validate on 3xDec => OK, RMaj with 3xDec, Validate on 2xDec+None => ABORT\n       (3 =:= NumVersionDec andalso 1 =:= NumNone) -> ok_or_abort;\n\n       true -> {fail, abort, [Key]}\n    end.\n\ncalc_rc_outcome(Key, PreOps) ->\n    %% DB is static over whole tx.\n    %% Read phase and validation phase may operate on different\n    %% replica subsets.\n    NumWritelock =  length([ X || X <- PreOps, X =:= writelock ]),\n    NumVersionInc = length([ X || X <- PreOps, X =:= versioninc ]),\n\n    if (0 =:= NumWritelock) -> {ok};\n       (1 =:= NumWritelock andalso 1 =/= NumVersionInc) -> {ok};\n       (1 =:= NumWritelock andalso 1 =:= NumVersionInc) -> ok_or_abort;\n\n       true -> {fail, abort, [Key]}\n    end.\n\ncalc_rmc_outcome(Key, PreOps) ->\n    NumReadlock =   length([ X || X <- PreOps, X =:= readlock ]),\n    NumWritelock =  length([ X || X <- PreOps, X =:= writelock ]),\n    NumVersionDec = length([ X || X <- PreOps, X =:= versiondec ]),\n    NumVersionInc = length([ X || X <- PreOps, X =:= versioninc ]),\n    NumNone =       length([ X || X <- PreOps, X =:= none ]),\n\n    if (4 =:= (NumReadlock + NumNone)) -> {ok};\n       (3 =:= NumReadlock) -> {ok};\n       (3 =:= NumReadlock + NumVersionDec + NumNone) -> {ok};\n       (2 =< NumReadlock + NumNone)\n       andalso (0 =:= NumWritelock + NumVersionInc) -> {ok};\n       (1 =:= NumWritelock) -> {fail, abort, [Key]};\n       (3 =:= NumVersionDec) -> {ok};\n       (4 =:= NumVersionDec) -> {ok};\n\n       true -> {fail, abort, [Key]}\n    end.\n\ncalc_wmc_outcome(Key, PreOps) ->\n    NumReadlock =   length([ X || X <- PreOps, X =:= readlock ]),\n    %NumWritelock =  length([ X || X <- PreOps, X =:= writelock ]),\n    NumVersionDec = length([ X || X <- PreOps, X =:= versiondec ]),\n    NumVersionInc = length([ X || X <- PreOps, X =:= versioninc ]),\n    NumNone =       length([ X || X <- PreOps, X =:= none ]),\n\n    if ((NumNone + NumVersionDec) =:= 4) -> {ok};\n       % The following should be 'ok_or_abort', because 4th is RL|WL|Inc and\n       % therefore if 4th in Maj => ABORT, else OK.\n       % However, the current code waits for 3 prepared or 2 abort decisions\n       % and only the latter can occur!\n       ((NumNone + NumVersionDec) =:= 3) -> {ok};\n\n       (NumVersionInc >= 2) -> {fail, abort, [Key]};\n       (NumReadlock >= 2) -> {fail, abort, [Key]};\n\n       true -> {fail, abort, [Key]}\n    end.\n\ninit_new_db_key(Value) ->\n    UID = uid:get_global_uid(),\n    NewKey = lists:flatten(io_lib:format(\"~p\", [UID])),\n    Keys = ?RT:get_replica_keys(?RT:hash_key(NewKey)),\n    _ = [ begin\n              E1 = db_entry:new(Key),\n              %% set a value, version += 2 (so decversion is ok)\n              E4 = db_entry:set_value(E1, Value, db_entry:get_version(E1) + 2),\n              api_dht_raw:unreliable_lookup(db_entry:get_key(E4),\n                                       {set_key_entry, comm:this(), E4}),\n              receive {set_key_entry_reply, E4} -> ok end\n          end || Key <- Keys ],\n    NewKey.\n\nget_db_entries(Keys) ->\n    [ begin\n          api_dht_raw:unreliable_lookup(X, {get_key_entry, comm:this(), X}),\n          receive\n              {get_key_entry_reply, Entry} -> Entry\n          end\n      end\n      || X <- Keys ].\n\nbp_cond_tm_crash(Message, _State) ->\n    case Message of\n        {tx_tm_rtm_commit, _, _, [{rdht_tx_read,\"a\",value, _, 0}]} -> true;\n        _ -> false\n    end.\n\ntm_crash(_) ->\n    ct:pal(\"Starting tm_crash~n\"),\n    {ok} = api_tx:write(\"a\", \"Hello world!\"),\n    %% ct:pal(\"written initial value and setting breakpoints now~n\"),\n    TMs = pid_groups:find_all(tx_tm),\n    %% all TMs break at next commit request:\n    _ = [ gen_component:bp_set_cond(X, fun tx_tm_rtm_SUITE:bp_cond_tm_crash/2,\n                                    tm_crash) || X <- TMs ],\n    %% ct:pal(\"Breakpoints set~n\"),\n    _ = [ gen_component:bp_barrier(X) || X <- TMs ],\n    %% ct:pal(\"Barriers set~n\"),\n\n    %% TM only will perform the tx_tm_rtm_commit, that lead to the BP\n    %% bp_step blocks. Do it asynchronously (we don't know which TM\n    %% got the request).\n    Pids = [ spawn(fun () -> gen_component:bp_step(X) end) || X <- TMs ],\n\n    %% report all tx_tms as failed after the commit has started...\n    _ = spawn(fun() ->\n                      timer:sleep(1500),\n                      fd:report(crash, TMs, 'DOWN')\n              end),\n\n    Res = api_tx:req_list([{read, \"a\"}, {commit}]),\n\n    ct:pal(\"Res: ~p~n\", [Res]),\n    %% may fail or succeed, so no equals check possible.\n    %%?equals_pattern(Res, {_,[_,{ok}]}),\n\n    _ = [ erlang:exit(Pid, kill) || Pid <- Pids ],\n\n    ok.\n\ntp_crash(_) ->\n    ct:pal(\"Starting tp_crash, simulated by holding the dht_node_proposer~n\"),\n    {ok} = api_tx:write(\"a\", \"Hello world!\"),\n    %% ct:pal(\"written initial value and setting breakpoints now~n\"),\n    AllProposers = pid_groups:find_all(paxos_proposer),\n    Proposers = [X || X <- AllProposers,\n                      basic_services =/= pid_groups:group_of(X)],\n    Proposer = hd(Proposers),\n    %% ct:pal(\"Selected ~p~n\", [pid_groups:group_and_name_of(Proposer)]),\n    %% break one TP (minority) after proposer initialize:\n    gen_component:bp_set(Proposer, ?proposer_initialize, tp_crash),\n    %% ct:pal(\"Breakpoints set~n\"),\n    gen_component:bp_barrier(Proposer),\n    %% ct:pal(\"Barriers set~n\"),\n\n    %% TM only performs the tx_tm_rtm_commit that lead to the BP\n    %% bp_step blocks. Do it asynchronously. (We don't know which TM\n    %% got the request.\n    %% Pids = [ spawn(fun () -> gen_component:bp_step(X) end) || X <- Proposers ],\n    %% report the one tp as failed\n    fd:report(crash, [Proposer], 'DOWN'),\n\n    %% ct:pal(\"Starting read commit~n\"),\n    Res = api_tx:req_list([{read, \"a\"}, {commit}]),\n\n    ct:pal(\"Res: ~p~n\", [Res]),\n\n    %%[ erlang:exit(Pid, kill) || Pid <- Pids ],\n\n    ok.\n\nall_tp_crash(_) ->\n    ct:pal(\"Starting all_tp_crash, simulated by holding the dht_node_proposers~n\"),\n    {ok} = api_tx:write(\"a\", \"Hello world!\"),\n    %% ct:pal(\"written initial value and setting breakpoints now~n\"),\n    AllProposers = pid_groups:find_all(paxos_proposer),\n    Proposers = [X || X <- AllProposers,\n                      basic_services =/= pid_groups:group_of(X)],\n    %% break all TPs (majority) after proposer initialize:\n    _ = [ gen_component:bp_set(Proposer, ?proposer_initialize, all_tp_crash)\n          || Proposer <- Proposers],\n    %% ct:pal(\"Breakpoints set~n\"),\n    _ = [ gen_component:bp_barrier(Proposer) || Proposer <- Proposers ],\n    %% ct:pal(\"Barriers set~n\"),\n\n    %% TM only performs the tx_tm_rtm_commit that lead to the BP\n    %% bp_step blocks. Do it asynchronously (we don't know which TM\n    %% got the request).\n    %% Pids = [ spawn(fun () -> gen_component:bp_step(X) end) || X <- Proposers ],\n    %% report the one tp as failed\n    fd:report(crash, Proposers, 'DOWN'),\n\n    %% ct:pal(\"Starting read commit~n\"),\n    Res = api_tx:req_list([{read, \"a\"}, {commit}]),\n\n    %% expecting {[], [{ok,\"Hello world!\"}, {fail,abort,[\"a\"]}]}\n    %% because the proposers are hold, the rtms try to decide abort,\n    %% which leads to the commit fail.\n    ct:pal(\"Res: ~p~n\", [Res]),\n\n    %%[ erlang:exit(Pid, kill) || Pid <- Pids ],\n    ok.\n"
  },
  {
    "path": "test/type_check_SUITE.erl",
    "content": "%% @copyright 2012-2016 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Florian Schintke <schintke@zib.de>\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc    Unit tests varifying type constraints via random testing.\n%% @end\n%% @version $Id$\n-module(type_check_SUITE).\n-author('schintke@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"scalaris.hrl\").\n-include(\"unittest.hrl\").\n-include(\"client_types.hrl\").\n\nall()   -> [\n            tester_type_check_api,\n            tester_type_check_config,\n%%            tester_type_check_dht_node,\n            tester_type_check_math,\n            tester_type_check_node,\n            tester_type_check_paxos,\n            tester_type_check_rrepair,\n            tester_type_check_tx,\n            tester_type_check_rdht_tx,\n            tester_type_check_histogram,\n            tester_type_check_util,\n            tester_type_check_gossip,\n            tester_type_check_mr\n           ].\nsuite() -> [ {timetrap, {seconds, 480}} ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_testcase(_TestCase, Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    unittest_helper:make_ring(4, [{config, [{log_path, PrivDir}]}]),\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\ntester_type_check_api(_Config) ->\n    Count = 500,\n    config:write(no_print_ring_data, true),\n    %% [{modulename, [excludelist = {fun, arity}]}]\n    tester:register_type_checker({typedef, rdht_tx, encoded_value, []}, rdht_tx, is_encoded_value),\n    tester:register_value_creator({typedef, rdht_tx, encoded_value, []}, rdht_tx, encode_value, 1),\n    Modules =\n        [ {api_dht, [], []},\n          {api_dht_raw,\n           [ {unreliable_lookup,2}, %% creates arb. messages\n             {unreliable_get_key,3}, %% creates arb. IP-adresses\n             {split_ring, 1} %% needs feeder to limit the input size\n           ],\n           [ {range_read,1}, %% bug in range_read?\n             {range_read_loop,5}, %% receives msgs\n             {delete_and_cleanup_timer,2} %% cannot create reference()\n           ]},\n          {api_monitor, [], []},\n          {api_rdht, [], [ {delete_collect_results, 3} ]}, %% receives\n          {api_tx,\n           [ {get_system_snapshot, 0} %% receives msgs\n           ], []},\n          {api_mr,\n           [ {start_job, 1} %% sends msgs\n           ],\n           [ {wait_for_results, 3}]} %%receives messages\n        ],\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    tester:unregister_type_checker({typedef, rdht_tx, encoded_value, []}),\n    tester:unregister_value_creator({typedef, rdht_tx, encoded_value, []}),\n    true.\n\ntester_type_check_config(_Config) ->\n    Count = 500,\n    %% [{modulename, [excludelist = {fun, arity}]}]\n    Modules =\n        [ {config,\n           [ {cfg_is_list, 3}, %% needs a fun as parameter\n             {cfg_is_tuple, 4}, %% needs a fun as parameter\n             {cfg_test_and_error, 3}, %% needs a fun as parameter\n             {init, 1} %% already initialised (config_ets table exists)\n           ],\n           [ {populate_db, 1} %% cannot create config filenames\n           ]}\n        ],\n    %% These tests generate errors which would be too verbose.\n    log:set_log_level(none),\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    log:set_log_level(config:read(log_level)),\n    true.\n\n%% tester_type_check_dht_node(_Config) ->\n%%     Count = 1000,\n%%     config:write(no_print_ring_data, true),\n%%     tester:register_type_checker({typedef, intervals, interval}, intervals, is_well_formed),\n%%     tester:register_value_creator({typedef, intervals, interval}, intervals, tester_create_interval, 1),\n%%     Modules =\n%%         [ %% {dht_node, [], [],}\n%%           %% {dht_node_join, [], [],}\n%%           %% {dht_node_lookup, [], [],}\n%%           %% {dht_node_monitor, [], [],}\n%%           %% {dht_node_move, [], [],}\n%%           %% {dht_node_reregister, [], [],}\n%%           {dht_node_state, [], []}\n%%         ],\n%%     [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n%%       || {Mod, Excl, ExclPriv} <- Modules ],\n%%     tester:unregister_type_checker({typedef, intervals, interval}),\n%%     tester:unregister_value_creator({typedef, intervals, interval}),\n%%    true.\n\ntester_type_check_gossip(_Config) ->\n    unittest_helper:wait_for_stable_ring_deep(),\n    Group = pid_groups:group_with(gossip),\n    pid_groups:join_as(Group, gossip),\n    Count = 250,\n    config:write(no_print_ring_data, true),\n    config:write(gossip_log_level_warn, debug),\n    config:write(gossip_log_level_error, debug),\n    tester:register_type_checker({typedef, intervals, interval, []}, intervals, is_well_formed),\n    tester:register_type_checker({typedef, intervals, simple_interval, []}, intervals, is_well_formed_simple),\n    tester:register_type_checker({typedef, intervals, continuous_interval, []}, intervals, is_continuous),\n    tester:register_value_creator({typedef, intervals, interval, []}, intervals, tester_create_interval, 1),\n    tester:register_value_creator({typedef, intervals, simple_interval, []}, intervals, tester_create_simple_interval, 1),\n    tester:register_value_creator({typedef, intervals, continuous_interval, []}, intervals, tester_create_continuous_interval, 4),\n    tester:register_type_checker({typedef, gossip_load, histogram, []}, gossip_load, is_histogram),\n    tester:register_value_creator({typedef, gossip_load, round, []}, gossip_load, tester_create_round, 1),\n    tester:register_value_creator({typedef, gossip_load, state, []}, gossip_load, tester_create_state, 11),\n    tester:register_value_creator({typedef, gossip_load, histogram, []}, gossip_load, tester_create_histogram, 1),\n    tester:register_value_creator({typedef, gossip_load, histogram_size, []}, gossip_load, tester_create_histogram_size, 1),\n    tester:register_value_creator({typedef, gossip_load, load_data_list, []}, gossip_load, tester_create_load_data_list, 1),\n    tester:register_value_creator({typedef, gossip, cb_module_name, []}, gossip, tester_create_cb_module_names, 1),\n    %% tester:register_type_checker({typedef, gossip_cyclon, state, []}, gossip_cyclon, is_state),\n    Modules = [\n               {gossip,\n            % excluded (exported functions)\n            [   {start_link, 1}, % would start a lot of processes\n                {start_gossip_task, 2}, % spec to wide\n                {stop_gossip_task, 1}, % would prohibit subsequent tests\n                {on_inactive, 2}, % too much interaction / spec to wide\n                {on_active, 2}, % too much interaction / spec to wide\n                {start_gen_component,5} %% unsupported types\n            ],\n            % excluded (private functions)\n            [   {handle_msg, 2}, % spec to wide, sends messages\n                {start_p2p_exchange, 4}, % would need valid peer\n                {init_gossip_tasks, 2}, % Id-Version-Error in gossip_cyclon (see below)\n                {init_gossip_task, 3}, % test via feeder\n                {cb_init, 2}, % spec to wide (Args)\n                {cb_select_data, 2}, % would need valid callback state\n                {cb_select_reply_data, 6}, % would need valid callback state\n                {cb_integrate_data, 5}, % would need valid callback state\n                {cb_handle_msg, 3}, % spec to wide (Msg)\n                {cb_web_debug_info, 2}, % would need valid callback state\n                {cb_round_has_converged, 2}, % would need valid callback state\n                {cb_notify_change, 4}, % would need valid callback state\n                {cb_call, 4}, % spec to wide\n                {check_round, 3}, % would need valid callback state\n                {is_end_of_round, 2}, % would need valid callback state\n                {state_update, 3} % tester can not create a value of type fun()\n            ]},\n          {gossip_load,\n            % excluded (exported functions)\n            [  {init, 1}, % tested via feeder\n               {get_values_best, 1}, % tested via feeder\n               {handle_msg, 2}, % would need valid dht_node_state, sends messages\n               {select_data, 1}, % sends messages\n               {select_reply_data, 4}, % sends messages\n               {integrate_data, 3}, % sends messages\n               {request_histogram, 2} % tested via feeder\n            ],\n            % excluded (private functions)\n            [  {state_update, 3}, % cannot create funs\n               {replace_skipped, 2},\n               {init_histo, 3}, % needs DHTNodeState state\n               {merge_histo, 2}, % tested via feeder\n               {merge_bucket, 2}, % tested via feeder\n               {request_node_details, 1} % sends messages\n            ]},\n            {gossip_cyclon,\n            % excluded (exported functions)\n            [\n             %% Id-Version-Error 1:\n             %%     'got two nodes with same IDversion but different ID' in node:is_newer()\n             {init, 1}, % needs valid neighborhood / Id-Version-Error\n             {select_data, 1}, % tested via feeder\n             {select_reply_data, 4}, % Id-Version-Error\n             {integrate_data, 3}, % Id-Version-Error\n             {handle_msg, 2} % needs valid NodeDetails (in get_node_details_response)\n            ],\n            % excluded (private functions)\n            [\n             {request_node_details, 1}, % tested via feeder\n             {print_cache_dot, 2} % to much console output\n            ]}\n        ],\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    tester:unregister_type_checker({typedef, intervals, interval, []}),\n    tester:unregister_type_checker({typedef, intervals, simple_interval, []}),\n    tester:unregister_type_checker({typedef, intervals, continuous_interval, []}),\n    tester:unregister_value_creator({typedef, intervals, interval, []}),\n    tester:unregister_value_creator({typedef, intervals, simple_interval, []}),\n    tester:unregister_value_creator({typedef, intervals, continuous_interval, []}),\n    tester:unregister_value_creator({typedef, gossip_load, round, []}),\n    tester:unregister_value_creator({typedef, gossip_load, state, []}),\n    tester:unregister_value_creator({typedef, gossip_load, load_data_list, []}),\n    tester:unregister_value_creator({typedef, gossip_load, histogram, []}),\n    tester:unregister_value_creator({typedef, gossip_load, histogram_size, []}),\n    tester:unregister_type_checker({typedef, gossip_load, histogram, []}),\n    tester:unregister_value_creator({typedef, gossip, cb_module_name, []}),\n    %% tester:unregister_type_checker({typedef, gossip_cyclon, state, []}),\n    true.\n\n\ntester_type_check_math(_Config) ->\n    Count = 250,\n    config:write(no_print_ring_data, true),\n    tester:register_type_checker({typedef, intervals, interval, []}, intervals, is_well_formed),\n    tester:register_type_checker({typedef, intervals, simple_interval, []}, intervals, is_well_formed_simple),\n    tester:register_type_checker({typedef, intervals, continuous_interval, []}, intervals, is_continuous),\n    tester:register_type_checker({typedef, prime, prime_list, []}, prime, tester_is_prime_list),\n    tester:register_type_checker({typedef, prime, prime, []}, prime, is_prime),\n    tester:register_value_creator({typedef, intervals, interval, []}, intervals, tester_create_interval, 1),\n    tester:register_value_creator({typedef, intervals, simple_interval, []}, intervals, tester_create_simple_interval, 1),\n    tester:register_value_creator({typedef, intervals, continuous_interval, []}, intervals, tester_create_continuous_interval, 4),\n    Modules =\n        [ {intervals,\n           [ {get_bounds, 1}, %% throws exception on []\n             {new, 4}, %% type spec to wide (would need overlapping contract support)\n             {split, 2} %% integers too large; tested via feeder\n           ],\n           [ {minus_simple2, 2}, %% second is subset of first param\n             {split2, 7} %% special list of split keys; tested via feeder\n           ]},\n          {mathlib,\n           [ {vecWeightedAvg,4}, %% needs same length lists\n             {closestPoints, 1}, %% needs same length lists\n             {binomial_coeff, 2}, %% needs N > K, done by feeder\n             {factorial, 1}, %% slow for large integers, done by feeder\n             {aggloClustering, 2}, %% needs same length lists\n             {vecAdd, 2}, %% needs same length lists\n             {vecSub, 2}, %% needs same length lists\n             {euclideanDistance, 2}, %% needs same length lists\n             {nearestCentroid, 2}, %% needs proper centroids\n             {u, 1}, %% needs non zero number in list\n             {zeros, 1} %% slow for large integers, tested via feeder\n           ],\n           [ {aggloClusteringHelper, 5}, %% spec suspicious (-1 for lists:nth())\n             {choose, 4}, %% slow for large integers\n             {factorial, 2} %% slow for large integers, done by feeder\n           ]},\n          %% {math_pos, [], []}, %% needs valid pos fields\n          {prime,\n           [ {get_nearest, 1}, %% too slow for large integers, tested via feeder\n             {is_prime, 1} %% too slow for large integers, tested via feeder\n           ],\n           [ {init, 0}, %% only once for initialisation\n             {sieve_num, 3}, %% throws if no prime is found\n             {sieve_filter, 3}, %% slow for large gaps between integers in the list\n             {find_in_cache, 2}, %% pre-condition between parameters must be met\n             {prime_cache, 0} %% there really is no point in testing this function!\n           ]},\n          {randoms,\n           [ {start, 0},\n             {stop, 0},\n             {rand_uniform, 2}, % tested via feeder\n             {rand_uniform, 3}  % tested via feeder\n           ]}\n        ],\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    tester:unregister_type_checker({typedef, intervals, interval, []}),\n    tester:unregister_type_checker({typedef, intervals, simple_interval, []}),\n    tester:unregister_type_checker({typedef, intervals, continuous_interval, []}),\n    tester:unregister_type_checker({typedef, prime, prime_list, []}),\n    tester:unregister_type_checker({typedef, prime, prime, []}),\n    tester:unregister_value_creator({typedef, intervals, interval, []}),\n    tester:unregister_value_creator({typedef, intervals, simple_interval, []}),\n    tester:unregister_value_creator({typedef, intervals, continuous_interval, []}),\n    true.\n\ntester_type_check_node(_Config) ->\n    Count = 250,\n    config:write(no_print_ring_data, true),\n    Modules =\n        [\n         {node,\n          [ {is_newer, 2}, %% throws function clause (same pid as input needed)\n            {newer, 2} %% throws function clause (same pid as input needed)\n          ], []},\n         {node_details,\n          [ {get, 2}], %% throws 'not_available' on empty nodelist\n          [ {get_list, 2}]}, %% throws 'not_available'\n         {nodelist,\n          [ {new_neighborhood, 2}, % the two given nodes must have different PIDs\n            {new_neighborhood, 3}, % the pred/succ nodes must have different PIDs than the base node\n            {mk_nodelist, 2}, % base node must not be in node list\n            {mk_neighborhood, 2}, % base node must not be in node list\n            {mk_neighborhood, 4}, % base node must not be in node list\n            {add_node, 4}, % base node must not be in node list\n            {add_nodes, 4}, % base node must not be in node list\n            {lremove, 3}, %% cannot create funs\n            {lremove_outdated, 1}, % base node must not be in node list\n            {lremove_outdated, 2}, % base node must not be in node list\n            {lfilter_min_length, 3}, %% cannot create funs\n            {filter_min_length, 4}, %% cannot create funs\n            {lfilter, 2}, %% cannot create funs\n            {lfilter, 3}, %% cannot create funs\n            {filter, 2}, %% cannot create funs\n            {filter, 3}, %% cannot create funs\n            {update_ids, 2}, % base node must not be in node list\n            {lupdate_ids, 2}, % base node must not be in node list\n            {update_node, 2}, %% needs node in certain interval\n            {merge, 4}, % base node must not be in node list of the other neighbour's object\n            {remove, 3}, %% cannot create funs\n            {create_pid_to_node_dict, 2} %% needs a dict() of node() objects\n          ],\n          [ {throw_if_newer, 2}, %% throws\n            {lsplit_nodelist, 2}, %% base node must not be in node list\n            {lusplit_nodelist, 2}, %% base node must not be in node list\n            {lmerge_helper, 5} %% base node must not be in node list\n          ]}\n        ],\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    true.\n\ntester_type_check_paxos(_Config) ->\n    Count = 500,\n    config:write(no_print_ring_data, true),\n    Modules =\n        [ {acceptor,\n           [ {add_learner,3}, %% tries to send messages\n             {msg_accepted, 4}, %% tries to send messages\n             {on, 2}, %% spec for messages not tight enough\n             {start_link,2}, %% tries to spawn processes\n             {start_paxosid, 2}, %% tries to send messages\n             {start_paxosid, 3}, %% tries to send messages\n             {stop_paxosids,2}, %% tries to send messages\n             {start_gen_component,5} %% unsupported types\n           ],\n           [ {msg_ack,5}, %% sends msgs\n             {msg_nack,3}, %% sends msgs\n             {msg_naccepted,3}, %% sends msgs\n             {initialize,4}, %% sends msgs\n             {inform_learners,2}, %% sends msgs\n             {inform_learner,3} %% sends msgs\n           ]},\n          {learner,\n           [ {on, 2}, %% spec for messages not tight enough\n             {start_link,2}, %% tries to spawn processes\n             {start_paxosid, 5}, %% tries to send messages\n             {stop_paxosids,2}, %% tries to send messages\n             {start_gen_component,5} %% unsupported types\n           ],\n           [ {msg_decide,4}, %% sends msg.\n             {decide, 2} %% no spec & uses msg_decide\n           ]},\n          {proposer,\n           [ {msg_accept, 5}, %% tries to send messages\n             {on, 2}, %% spec for messages not tight enough\n             {start_link, 2}, %% tries to spawn processes\n             {start_paxosid, 6}, %% tries to send messages\n             {start_paxosid, 7}, %% tries to send messages\n             {stop_paxosids, 2}, %% tries to send messages\n             {trigger, 2}, %% tries to send messages\n             {start_gen_component,5} %% unsupported types\n           ],\n           [ {msg_prepare,4}, %% tries to send messages\n             {proposer_trigger, 4}, %% tries to send messages\n             {start_new_higher_round,3}, %% tries to send messages\n             {state_add_ack_msg, 4} %% tested via feeder\n           ]}\n        ],\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    true.\n\ntester_type_check_rrepair(_Config) ->\n    Count = 250,\n    config:write(no_print_ring_data, true),\n    tester:register_type_checker({typedef, intervals, interval, []}, intervals, is_well_formed),\n    tester:register_type_checker({typedef, intervals, continuous_interval, []}, intervals, is_continuous),\n    tester:register_type_checker({typedef, intervals, non_empty_interval, []}, intervals, is_non_empty),\n    tester:register_type_checker({typedef, rt_beh, segment, []}, rt_beh, tester_is_segment),\n    tester:register_type_checker({typedef, rr_recon, kvi_tree, []}, rr_recon, tester_is_kvi_tree),\n    tester:register_value_creator({typedef, random_bias, generator, []},\n                                  random_bias, tester_create_generator, 3),\n    tester:register_value_creator({typedef, intervals, interval, []}, intervals, tester_create_interval, 1),\n    tester:register_value_creator({typedef, intervals, continuous_interval, []}, intervals, tester_create_continuous_interval, 4),\n    tester:register_value_creator({typedef, intervals, non_empty_interval, []}, intervals, tester_create_non_empty_interval, 2),\n    tester:register_value_creator({typedef, merkle_tree, leaf_hash_fun, []}, merkle_tree, tester_create_hash_fun, 1),\n    tester:register_value_creator({typedef, merkle_tree, inner_hash_fun, []}, merkle_tree, tester_create_inner_hash_fun, 1),\n    tester:register_value_creator({typedef, hfs_beh, hfs_fun, []}, hfs_beh, tester_create_hfs_fun, 1),\n    tester:register_value_creator({typedef, hfs_lhsp, hfs, []}, hfs_lhsp, tester_create_hfs, 1),\n    tester:register_value_creator({typedef, hfs_plain, hfs, []}, hfs_plain, tester_create_hfs, 1),\n    tester:register_value_creator({typedef, rt_beh, segment, []}, rt_beh, tester_create_segment, 1),\n    tester:register_value_creator({typedef, rr_recon, kvi_tree, []}, rr_recon, tester_create_kvi_tree, 1),\n    Modules =\n        [ {rr_recon_stats, [], []},\n          {db_generator,\n           [ {get_db, 3}, %% tested via feeder\n             {get_db, 4}, %% tested via feeder\n             {fill_ring, 3} %% tested via feeder\n           ],\n           [ {fill_random, 2}, %% tested via feeder\n             {fill_wiki, 2}, %% no (suitable) wiki file to import\n             {gen_kvv, 3}, %% tested via feeder\n             {p_gen_kvv, 6}, %% tested via feeder\n             {get_error_key, 2}, %% keys must be replica keys, i.e. one per quadrant!\n\n             {gen_random, 3}, %% needs feeder\n             {gen_random_gb_sets, 5}, %% needs feeder\n             {uniform_key_list, 3}, %% needs feeder\n             {uniform_key_list_no_split, 3}, %% needs feeder\n             {non_uniform_key_list, 5}, %% needs feeder\n             {non_uniform_key_list_, 7}, %% needs feeder\n             {get_non_uniform_probs, 1} %% needs feeder\n           ]},\n          {hfs_lhsp,\n           [ {apply_val, 3} %% tested via feeder\n           ],\n           [ {apply_val_helper, 3}, %% tested via feeder\n             {apply_val_rem_helper, 4} %% tested via feeder\n           ]},\n          {hfs_plain,\n           [ {apply_val, 3}, %% tested via feeder\n             {apply_val_rem, 3} %% tested via feeder\n           ],\n           [ {hash_value, 4}, %% needs bit sizes to fit, already tested via the apply_val* functions\n             {split_bin, 3} %% bitstring needs to have a minimum size\n           ]},\n          {rr_recon,\n           [\n             {init, 1}, %% registers a monitor (only one allowed per PID)\n             {on, 2}, %% tries to send messages, needs valid state with pid\n             {start, 2}, %% tries to spawn processes\n             {map_rkeys_to_quadrant, 2}, %% keys must be replica keys, i.e. one per quadrant!\n             {map_interval, 2}, %% second interval must be in a single quadrant\n\n             {merkle_compress_hashlist, 4}, %% needs merkle nodes with hashes\n             {merkle_decompress_hashlist, 3}, %% needs a special binary to correspond to a number of bits\n             {pos_to_bitstring, 4}, % needs to fulfil certain preconditions\n             {bitstring_to_k_list_k, 3}, % needs a special binary to correspond to a number of Key entries\n             {bitstring_to_k_list_kv, 3}, % needs a special binary to correspond to a number of KV entries\n             {calc_n_subparts_FR, 2}, %% needs float >= 0\n             {calc_n_subparts_FR, 3}, %% needs float >= 0\n             {start_gen_component,5} %% unsupported types\n           ],\n           [\n             {check_percent, 1}, %% checks arbitrary config -> too many unnecessary error messages\n             {build_struct, 3}, %% tries to send messages, needs valid state with pid\n             {build_recon_struct, 5}, %% DB items must be in interval\n             {begin_sync, 1}, %% tries to send messages\n             {shutdown, 2}, %% tries to send messages\n             {merkle_next_signature_sizes, 5}, %% needs float > 0, < 1\n             {min_max, 3}, %% tested via feeder\n             {trivial_signature_sizes, 4}, %% needs float > 0, < 1\n             {trivial_worst_case_failrate, 4}, %% needs 0 =< ExpDelta =< 100\n             {shash_signature_sizes, 4}, %% needs float > 0, < 1\n             {shash_worst_case_failrate, 4}, %% needs 0 =< ExpDelta =< 100\n             {calc_one_m_xpow_one_m_z, 2}, %% needs X =/= 0\n             {calc_max_different_hashes_, 3}, %% needs 0 =< ExpDelta =< 100\n             {calc_max_different_hashes, 3}, %% needs 0 =< ExpDelta =< 100\n             {calc_max_different_items_total, 3}, %% needs 0 =< ExpDelta =< 100\n             {calc_max_different_items_node, 3}, %% needs 0 =< ExpDelta =< 100\n             {compress_kv_list, 5}, %% needs a fun\n             {compress_kv_list_fr, 7}, %% needs float > 0, < 1 and a fun\n             {bloom_worst_case_failrate, 3}, %% needs float > 0, < 1, 0 =< ExpDelta =< 100\n             {bloom_worst_case_failrate_, 5}, %% needs float > 0, < 1, 0 =< ExpDelta =< 100\n             {bloom_target_fp, 4}, %% needs float > 0, < 1\n             {merkle_next_fr_targets, 2}, %% needs float > 0, < 1\n             {get_diff_with_dupes, 7}, %% needs a function\n             {calc_items_in_chunk, 2}, %% needs special input\n             {decompress_kv_list, 3}, %% needs a special binary to correspond to a number of bits\n             {compress_idx_list, 6}, %% needs a sorted list of positions, also LastPos needs to be smaller than these positions\n             {decompress_idx_list, 3}, %% needs a special binary to correspond to a number of bits\n             {decompress_idx_list_, 4}, %% needs a special binary to correspond to a number of bits\n             {decompress_idx_to_list, 2}, %% needs a special binary to correspond to a number of bits\n             {decompress_idx_to_list_, 3}, %% needs a special binary to correspond to a number of bits\n             {shash_bloom_perform_resolve, 7}, %% needs a special binary to correspond to a number of bits\n             {phase2_run_trivial_on_diff, 6}, %% needs parameters to match\n             {merkle_check_node, 22}, %% needs merkle_tree/nodes with hashes\n             {merkle_cmp_result, 21}, %% needs matching result and merkle nodes\n             {merkle_calc_used_fr, 5}, %% needs floats >= 0, =< 1\n             {merkle_resolve_add_leaf_hashes, 14}, %% needs KV-List merkle buckets\n             {merkle_resolve_retrieve_leaf_hashes, 17}, %% needs special bitstring\n             {merkle_resolve_leaves_send, 2}, % needs only leaf nodes in node list\n             {merkle_resolve_leaves_receive, 2}, % needs only leaf nodes in node list\n             {merkle_resolve_leaves_ckidx, 8}, % needs same-length lists\n             {send_resolve_request, 6}, %% tries to send messages\n             {art_get_sync_leaves, 6}, %% needs non-empty bloom filters\n             {send, 2}, %% tries to send messages\n             {send_local, 2}, %% tries to send messages\n             {send_chunk_req, 4}, %% tries to send messages\n             {quadrant_intervals_, 3}, %% special pre-conditions, only private to quadrant_intervals/0, tested enough in there\n             {replicated_intervals, 1} %% interval must be in a single quadrant\n           ]},\n          {rr_resolve,\n           [\n             {init, 1}, %% registers a monitor (only one allowed per PID)\n             {on, 2}, %% tries to send messages, needs valid state with pid\n             {start, 1}, %% tries to spawn processes\n             {start_gen_component,5}, %% unsupported types\n             {merge_stats, 2} %% tested via feeder\n           ],\n           [\n             {shutdown, 2}, %% tries to send messages\n             {send_request_resolve, 6}, %% tries to send messages\n             {send, 2}, %% tries to send messages\n             {send_local, 2}, %% tries to send messages\n\n             {start_update_key_entries, 3}, %% tries to send messages\n             {map_kvv_list, 2}, %% needs a unique tuple list, e.g. via feeder\n             {map_key_list, 2} %% needs a unique key list, e.g. via feeder\n           ]}\n        ],\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    tester:unregister_value_creator({typedef, merkle_tree, leaf_hash_fun, []}),\n    tester:unregister_value_creator({typedef, merkle_tree, inner_hash_fun, []}),\n    tester:unregister_value_creator({typedef, random_bias, generator, []}),\n    tester:unregister_value_creator({typedef, intervals, interval, []}),\n    tester:unregister_value_creator({typedef, intervals, continuous_interval, []}),\n    tester:unregister_value_creator({typedef, intervals, non_empty_interval, []}),\n    tester:unregister_value_creator({typedef, hfs_beh, hfs_fun, []}),\n    tester:unregister_value_creator({typedef, hfs_lhsp, hfs, []}),\n    tester:unregister_value_creator({typedef, hfs_plain, hfs, []}),\n    tester:unregister_value_creator({typedef, rt_beh, segment, []}),\n    tester:unregister_value_creator({typedef, rr_recon, kvi_tree, []}),\n    tester:unregister_type_checker({typedef, intervals, interval, []}),\n    tester:unregister_type_checker({typedef, intervals, continuous_interval, []}),\n    tester:unregister_type_checker({typedef, intervals, non_empty_interval, []}),\n    tester:unregister_type_checker({typedef, rt_beh, segment, []}),\n    tester:unregister_type_checker({typedef, rr_recon, kvi_tree, []}),\n    true.\n\ntester_type_check_tx(_Config) ->\n    Count = 250,\n    config:write(no_print_ring_data, true),\n    tester:register_type_checker({typedef, rdht_tx, encoded_value, []}, rdht_tx, is_encoded_value),\n    tester:register_value_creator({typedef, rdht_tx, encoded_value, []}, rdht_tx, encode_value, 1),\n    Modules =\n        [ {tx_op_beh,[], []},\n          {tx_tlog,\n           [ {new_entry, 5}, %% TODO: some combinations of value types are not allowed\n             {new_entry, 6}, %% TODO: some combinations of value types are not allowed\n             {new_entry, 7}, %% TODO: some combinations of value types are not allowed\n             {set_entry_key, 2}, %% split tlog types for client and rt:keys\n             {set_entry_operation, 2}, %% may violate type spec (?partial_value in ?write op) (TODO: prevent via feeder)\n             {set_entry_value, 3} %% may violate type spec (?partial_value in ?write op) (TODO: prevent via feeder)\n           ],\n           [ {read_op_for_key, 5} %% no type spec available (a 1-element list may not be specified anyway)\n           ]},\n          {tx_tm_rtm,\n           [ {commit, 4},\n             {get_my, 2},\n             {init, 1},\n             {msg_commit_reply, 3},\n             {on,2},\n             {on_init,2},\n             {start_link,2},\n             {start_gen_component,5} %% unsupported types\n           ],\n           [ {get_paxos_ids, 2}, %% requires item entries in dictionary\n             {get_failed_keys, 2}, %% needs number of aborts in item list to match numabort\n             {msg_tp_do_commit_abort,4}, %% tries to send\n             {init_RTMs, 2}, %% tries to send\n             {init_TPs, 3}, %% tries to send\n             {inform_client, 3}, %% tries to send\n             {inform_rtms, 3}, %% tries to send\n             {inform_tps, 3}, %% tries to send\n             {send_to_rtms, 2}, %% tries to send\n             {merge_item_states, 6}, %% needs specially-crafted lists\n             {tx_item_new, 3}, %% TODO: not a list error\n             {tx_item_new, 5} %% TODO invalid result type\n           ]}\n          %% {tx_tp,[{init, 0}, {on_do_commit_abort_fwd, 6},\n          %% {on_do_commit_abort, 3}, {on_init_TP, 2}]},\n        ],\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    tester:unregister_type_checker({typedef, rdht_tx, encoded_value, []}),\n    tester:unregister_value_creator({typedef, rdht_tx, encoded_value, []}),\n    true.\n\ntester_type_check_rdht_tx(_Config) ->\n    Count = 500,\n    config:write(no_print_ring_data, true),\n    tester:register_type_checker({typedef, rdht_tx, encoded_value, []}, rdht_tx, is_encoded_value),\n    tester:register_value_creator({typedef, rdht_tx, encoded_value, []}, rdht_tx, encode_value, 1),\n    Modules =\n        [ {rdht_tx,\n           [ {decode_value, 1} ], %% not every binary is an erlterm\n           [ {collect_replies,2}, %% recv msgs\n             {receive_answer,0}, %% recv msgs\n             {do_reqs_on_tlog,3}, %% req keys maybe not in tlog\n             {do_reqs_on_tlog_iter,4}, %% req keys maybe not in tlog\n             {commit, 1} %% should work, but hangs\n           ]},\n          {rdht_tx_read,\n           [ {abort, 5},\n             {commit, 5},\n             {extract_from_value, 3}, %% tested via feeder\n             {extract_from_tlog, 4}, %% tested via feeder\n             {init, 1},\n             {on,2},\n             {start_link, 1},\n             {start_gen_component,5}, %% unsupported types\n             {validate_prefilter, 1}, %% TODO: not a list error\n             {validate, 3},\n             {work_phase, 3}\n           ],\n           [ {quorum_read, 4}, %% needs collector pid\n             {make_tlog_entry, 2} %% tested via feeder\n           ]},\n          {rdht_tx_write,\n           [ {abort, 5},\n             {commit, 5},\n             {start_link, 1}, {init, 1}, {on,2},\n             {start_gen_component,5}, %% unsupported types\n             {validate_prefilter, 1}, %% TODO: not a list error\n             {validate, 3},\n             {work_phase, 3}\n           ], []},\n          {rdht_tx_add_del_on_list,\n           [ {extract_from_tlog, 5}, %% tested via feeder\n             {work_phase, 3}\n           ], []},\n          {rdht_tx_add_on_nr,\n           [ {extract_from_tlog, 4}, %% tested via feeder\n             {work_phase, 3}\n           ], []},\n          {rdht_tx_test_and_set,\n           [ {extract_from_tlog, 5}, %% tested via feeder\n             {work_phase, 3}\n           ], []}\n        ],\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    tester:unregister_type_checker({typedef, rdht_tx, encoded_value, []}),\n    tester:unregister_value_creator({typedef, rdht_tx, encoded_value, []}),\n    true.\n\ntester_type_check_histogram(_Config) ->\n    Count = 500,\n    config:write(no_print_ring_data, true),\n    tester:register_type_checker({typedef, histogram, histogram, []}, histogram, tester_is_valid_histogram),\n    tester:register_value_creator({typedef, histogram, histogram, []}, histogram, tester_create_histogram, 2),\n    tester:register_value_creator({typedef, histogram_rt, histogram, []}, histogram_rt, tester_create_histogram, 2),\n    %% [{modulename, [excludelist = {fun, arity}]}]\n    Modules =\n        [ {histogram,\n           [ {find_smallest_interval, 1}, % private API, needs feeder\n             {merge_interval, 2} % private API, needs feeder\n           ],\n           [ {resize, 1} % needs feeder\n           ]},\n          {histogram_rt,\n           [],\n           [ {denormalize, 2} % TODO needs feeder according to ?RT:get_range/2 output\n           ]\n          }\n        ],\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    tester:unregister_type_checker({typedef, histogram, histogram, []}),\n    tester:unregister_value_creator({typedef, histogram, histogram, []}),\n    tester:unregister_value_creator({typedef, histogram_rt, histogram, []}),\n    true.\n\ntester_type_check_util(_Config) ->\n    Count = 250,\n    config:write(no_print_ring_data, true),\n    %% [{modulename, [excludelist = {fun, arity}]}]\n    Modules =\n        [ {comm,\n           [ {get_ip, 1}, %% cannot create correct envelopes\n             {get_port, 1}, %% cannot create correct envelopes\n             {init_and_wait_for_valid_IP, 0}, %% cannot start\n             {is_local, 1}, %% cannot create correct envelopes\n             {send, 2}, {send, 3}, %% cannot send msgs\n             {send_local, 2}, {send_local_after, 3}, %% cannot send msgs\n             {forward_to_group_member, 2}, %% may forward arbitrary message to any process\n             {forward_to_registered_proc, 2}, %% may forward arbitrary message to any process\n             {reply_as, 3} %% needs feeder for envelope\n           ], []},\n          {db_entry,\n           [ {inc_version, 1}, % WL -1 is only allowed for empty_val\n             {dec_version, 1}, % WL -1 is only allowed for empty_val\n             {set_value, 3} % WL -1 is only allowed for empty_val\n           ], []},\n          {debug,\n           [ {get_round_trip, 2}, %% needs gen_component pids\n             {dump3, 0}, %% type spec not valid\n             {dumpX, 1}, {dumpX, 2}, %% type spec not valid?\n             {topDumpX, 1},\n             {topDumpX, 3},\n             {topDumpXEvery, 3},\n             {topDumpXEvery, 5},\n             {topDumpXEvery_helper, 4}\n           ],\n           [ {get_round_trip_helper, 2}, %% needs gen_component pids\n             {dump_extract_from_list,2}, %% wrong spec\n             {dumpXNoSort,2}, %% needs fun\n             {default_dumpX_val_fun,2}, %% spec too wide (must be tuple sometimes),\n             {rr_count_old_replicas_data, 1} %% needs dht_node pid\n           ]},\n          %% {fix_queue, [], []}, %% queue as builtin type not supported yet\n\n          {mymaps,\n           [ {get, 2}, % throws if the value does not exist\n             {update, 3} % throws if the value does not exist\n           ], []},\n          {msg_queue, [], []},\n          {pdb, [], []},\n          {pid_groups,\n           [ {add, 3}, %% same as above\n             {init, 1}, %% tries to create existing ets table\n             {join_as, 2}, %% tries to join with multiple groups/names\n             {on, 2},\n             {pids_to_names, 2}, %% sends remote messages\n             {filename_to_group, 1}, %% not every string is convertible\n             {start_link, 0},\n             {start_gen_component,5} %% unsupported types\n           ], []},\n          {quorum, [], []},\n          %% {rrd,\n          %%  [ {dump, 1}, %% eats memory?!\n          %%    {dump_with, 2}, %% needs fun\n          %%    {dump_with, 3}, %% needs fun\n          %%    {add, 3}, %% to slow for large timestamps?\n          %%    {add_now, 2}, %% bad arith\n          %%    {add_with, 4}, %% needs fun\n          %%    {check_timeslot, 2}, %% to slow for large timestamps?\n          %%    {check_timeslot_now, 1}, %% to slow for testing?\n          %%    {get_value, 2}, %% returns more than the spec expects\n          %%    {get_value_by_offset, 2}, %% returns more than the spec expects\n          %%    {timing_with_hist_merge_fun, 3}, %% function_clause\n          %%    {merge, 2}, %% needs same rrd type twice\n          %%    {add_nonexisting_timeslots, 2} %% needs same rrd type twice\n          %%  ],\n          %%  [ {update_with, 5} %% needs fun\n          %%    ...\n          %%  ]},\n          %%{statistics, [], []},\n          {uid, [], []},\n          {util,\n           [ {collect_while, 1}, %% cannot create funs\n             {debug_info, 0}, %% type spec not valid?\n             {debug_info, 1}, %% type spec not valid?\n             {do_throw, 1}, %% throws exceptions\n             {extint2atom, 1}, %% type spec too wide\n             {for_to, 3}, %% cannot create funs\n             {for_to_ex, 3}, %% cannot create funs\n             {for_to_ex, 4}, %% cannot create funs\n             {for_to_fold, 5}, %% cannot create funs\n             {gb_trees_foldl, 3}, %% cannot create funs\n             {lists_takewith, 2}, %% cannot create funs; tested via feeder\n             {lists_keystore2, 5}, %% key id may not be larger than the tuple size in the list\n             {lists_partition3, 2}, %% cannot create funs; tested via feeder\n             {lists_remove_at_indices, 2}, %% indices must exist in list\n             {log, 2}, %% tested via feeder\n             {log2, 1}, %% tested via feeder\n             {log1p, 1}, %% tested via feeder\n             {pow1p, 2}, %% tested via feeder\n             {logged_exec, 1}, %% not execute random strings\n             {map_with_nr, 3}, %% cannot create funs; tested via feeder\n             {par_map, 2}, %% cannot create funs; tested via feeder\n             {par_map, 3}, %% cannot create funs; tested via feeder\n             {parallel_run, 5}, %% cannot create funs\n             {pop_randomelem, 2}, %% list may be too short\n             {pow, 2}, %% floats become too large and raise badarith\n             {print_bits, 2}, %% cannot create funs\n             {readable_utc_time, 1}, %% too slow for big ints; tested via feeder\n             {repeat, 3}, {repeat, 4}, %% cannot create funs\n             {round, 2}, %% floats become too large and raise badarith\n             {sets_map, 2}, %% cannot create funs\n             {smerge2, 3}, %% cannot create funs\n             {smerge2, 4}, %% cannot create funs\n             {smerge2, 6}, %% cannot create funs\n             {sleep_for_ever, 0},\n             {split_unique, 3}, %% cannot create funs\n             {split_unique, 4}, %% cannot create funs\n             {ssplit_unique, 3}, %% cannot create funs\n             {ssplit_unique, 4}, %% cannot create funs\n             {tc, 1}, {tc, 2}, {tc, 3}, %% don't call arbitrary functions\n             {wait_for, 1}, %% cannot create funs\n             {wait_for, 2}, %% cannot create funs\n             {wait_for_process_to_die, 1}, %% could wait forever\n             {wait_for_ets_table_to_disappear, 2}, %% cannot create tids\n             {zipfoldl, 5}, %% cannot create funs\n             {rrd_combine_timing_slots, 3}, %% values too big\n             {rrd_combine_timing_slots, 4}, %% values too big\n             {rrd_combine_gauge_slots, 3}, %% values too big\n             {rrd_combine_gauge_slots, 4}, %% values too big\n             {rrd_combine_slots, 6} %% values too big\n           ],\n           [ {lists_takewith_iter, 3}, %% cannot create funs; tested via feeder\n             {lists_partition3, 5}, %% cannot create funs; tested via feeder\n             {lists_remove_at_indices, 3}, %% indices must exist in list\n             {shuffle_helperA, 3}, %% preconds must be fulfilled\n             {gb_trees_largest_smaller_than_iter,3}, %% err: function_clause\n             {'=:<_lists', 2}, %% need equal length lists\n             {ssplit_unique_helper, 5}, %% needs fun\n             {smerge2_helper, 6}, %% needs fun\n             {i_repeat,5}, %% needs fun\n             {parallel_collect,3}, %% needs fun\n             {par_map_recv, 2}, %% receives messages\n             {par_map_recv2, 2}, %% receives messages\n             {sublist_, 4}, %% tested via feeder\n             {pow1p_, 4}, %% already tested by root1p/2\n             {bin_op, 3}, %% cannot create funs\n             {bin_op, 4}, %% cannot create funs\n             {wait_for1, 2}, %% cannot create funs\n             {wait_for2, 2}, %% cannot create funs\n             {collect_while,2}, %% needs fun\n             {gb_trees_foldl_iter,3} %% needs fun\n           ]}\n        ],\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n%% feeders are found automatically - sample explicit call would be:\n%%    tester:test(util, readable_utc_time, 1, 25, [with_feeder]),\n\n%%    tester_helper:load_with_export_all(util),\n%%    ct:pal(\"testing with export all\"),\n%%    tester:test(util, escape_quotes_, 2, 25),\n%%    tester_helper:load_without_export_all(util),\n    true.\n\ntester_type_check_mr(_Config) ->\n    Count = 500,\n    config:write(no_print_ring_data, true),\n    tester:register_type_checker({typedef, mr_state, fun_term, []}, mr_state,\n                                 tester_is_valid_funterm),\n    tester:register_value_creator({typedef, mr_state, fun_term, []}, mr_state,\n                                  tester_create_valid_funterm, 2),\n    Modules =\n        [ {mr_state,\n           [\n            {new, 6}, %% needs fun\n            {add_data_to_phase, 4}, %% needs ets table\n            {clean_up, 1}, %% closes ets tables\n            {accumulate_data, 2}, %% needs ets tables\n            {get_slide_delta, 2}, %% needs ets tables\n            {init_slide_state, 1}, %% needs ets tables\n            {merge_states, 2} %% needs ets tables\n           ],\n           [\n            {trigger_work, 2}, %% sends message\n            {merge_phase_delta, 2}, %% needs ets tables\n            {acc_add_element, 2} %% needs ets tables\n           ]}\n          %% , {mr_master, [{on, 2}, {init, 1}, {start_link, 2}], []}\n        ],\n    _ = [ tester:type_check_module(Mod, Excl, ExclPriv, Count)\n          || {Mod, Excl, ExclPriv} <- Modules ],\n    tester:unregister_type_checker({typedef, mr_state, fun_term, []}),\n    tester:unregister_value_creator({typedef, mr_state, fun_term, []}),\n    true.\n"
  },
  {
    "path": "test/type_check_TESTS.cfg",
    "content": "{alias, tests, \".\"}.\n\n{suites, tests, [type_check_SUITE]}.\n\n{cases, tests, l_on_cseq_SUITE, [tester_type_check_l_on_cseq]}.\n{cases, tests, prbr_SUITE, [tester_type_check_rbr]}.\n{cases, tests, rm_leases_SUITE, [tester_type_check_rm_leases]}.\n{cases, tests, slide_leases_SUITE, [tester_type_check_slide_leases]}.\n"
  },
  {
    "path": "test/unittest.hrl",
    "content": "%  @copyright 2008-2014 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    contains unittest helpers\n%% @end\n%% @version $Id$\n\n-ifdef(have_ctline_support).\n-compile({parse_transform, ct_line}).\n-endif.\n\n%% @doc Fails the currently run unit test with a reason that is made of the\n%%      given Data formatted using the Format string (see io_lib:format/2).\n%% -spec ct_fail(Format::atom() | string() | binary(), Data::[term()]) -> no_return().\n-define(ct_fail(Format, Data),\n        % if possible, do not use a function to silence dialyzer warnings about\n        % functions with no return\n        ct:fail(lists:flatten(io_lib:format(Format, Data)))).\n\n-define(assert(Boolean),\n        unittest_helper:macro_equals(Boolean, true, ??Boolean, \"true\", null)).\n\n-define(assert_w_note(Boolean, Note),\n        unittest_helper:macro_equals(Boolean, true, ??Boolean, \"true\", Note)).\n\n-define(equals(Actual, Expected),\n        unittest_helper:macro_equals(Actual, Expected, ??Actual, ??Expected, null)).\n\n-define(equals_w_note(Actual, Expected, Note),\n        unittest_helper:macro_equals(Actual, Expected, ??Actual, ??Expected, Note)).\n\n-define(equals_pattern(Actual, ExpectedPattern),\n        % wrap in function so that the internal variables are out of the calling function's scope\n        fun() ->\n                case Actual of\n                    ExpectedPattern -> true;\n                    UnittestAny ->\n                        unittest_helper:macro_equals_failed(\n                          UnittestAny, ??ExpectedPattern, \"=:=\", ??Actual, ??ExpectedPattern, null)\n                end\n        end()).\n\n-define(equals_pattern_w_note(Actual, ExpectedPattern, Note),\n        % wrap in function so that the internal variables are out of the calling function's scope\n        fun() ->\n                case Actual of\n                    ExpectedPattern -> true;\n                    UnittestAny ->\n                        unittest_helper:macro_equals_failed(\n                          UnittestAny, ??ExpectedPattern, \"=:=\", ??Actual, ??ExpectedPattern, Note)\n                end\n        end()).\n\n-define(compare(CompFun, Actual, Expected),\n        unittest_helper:macro_compare(CompFun, Actual, Expected, ??CompFun, ??Actual, ??Expected, null)).\n\n-define(compare_w_note(CompFun, Actual, Expected, Note),\n        unittest_helper:macro_compare(CompFun, Actual, Expected, ??CompFun, ??Actual, ??Expected, Note)).\n\n-define(expect_exception(Cmd, ExceptionType, ExceptionPattern),\n        % wrap in function so that the internal variables are out of the calling function's scope\n        fun() ->\n                try Cmd of\n                    UnittestAny ->\n                        UnittestExpExceptionStr = \"exception: \" ??ExceptionType ++ \":\" ++ ??ExceptionPattern,\n                        unittest_helper:macro_equals_failed(\n                          UnittestAny, UnittestExpExceptionStr, \"=:=\", ??Cmd, UnittestExpExceptionStr, null)\n                catch\n                    ExceptionType:ExceptionPattern -> true;\n                    UnittestOtherType:UnittestOtherException ->\n                        UnittestExpExceptionStr = \"exception: \" ++ ??ExceptionType ++ \":\" ++ ??ExceptionPattern,\n                        UnittestActExceptionStr = lists:flatten(io_lib:format(\"exception: ~.0p:~.0p\", [UnittestOtherType, UnittestOtherException])),\n                        unittest_helper:macro_equals_failed(\n                          UnittestActExceptionStr, UnittestExpExceptionStr, \"=:=\", ??Cmd, UnittestExpExceptionStr, null)\n                end\n        end()).\n\n-define(expect_message_timeout(MsgPattern, Timeout),\n        % wrap in function so that the internal variables are out of the calling function's scope\n        fun() ->\n                receive\n                    MsgPattern -> true\n                after\n                    Timeout ->\n                        UnittestActualMessage =\n                            receive\n                                X -> X\n                            after\n                                0 -> no_message\n                            end,\n                        ?ct_fail(\"expected message ~s but got \\\"~.0p\\\"~n\", [??MsgPattern, UnittestActualMessage])\n                end\n        end()).\n-define(expect_message(MsgPattern), ?expect_message_timeout(MsgPattern, 1000)).\n\n-define(expect_no_message_timeout(Timeout),\n        unittest_helper:expect_no_message_timeout(Timeout)).\n-define(expect_no_message(), ?expect_no_message_timeout(100)).\n\n-define(consume_message(Message, Timeout),\n    receive\n        Message -> true\n    after\n        Timeout -> timeout\n    end).\n\n-define(consume_all_messages(Message),\n        % recursive anonymous function:\n        fun() ->\n                Fun = fun(F) ->\n                              case ?consume_message(Message, 0) of\n                                  true -> F(F);\n                                  timeout -> true\n                              end\n                      end,\n                Fun(Fun)\n        end()).\n\n-define(expect_message_ignore_timeout(MsgPattern, IgnoredMessage, Timeout),\n        % wrap in function so that the internal variables are out of the calling function's scope\n        fun() ->\n                % ignore two ignored messages, then wait for Timeout ignoring all ignored messages and do a final receive with the expected message\n                receive\n                    IgnoredMessage ->\n                        ct:pal(\"ignored \\\"~.0p\\\"~n\", [IgnoredMessage]),\n                        receive\n                            IgnoredMessage ->\n                                ct:pal(\"ignored \\\"~.0p\\\"~n\", [IgnoredMessage]),\n                                ?consume_message(IgnoredMessage, Timeout),\n                                receive\n                                    IgnoredMessage ->\n                                        ct:pal(\"ignored \\\"~.0p\\\" for the last time~n\", [IgnoredMessage]),\n                                    MsgPattern -> true\n                                after\n                                    0 ->\n                                        UnittestActualMessage =\n                                            receive\n                                                X -> X\n                                            after\n                                                0 -> no_message\n                                            end,\n                                        ?ct_fail(\"expected message ~s but got \\\"~.0p\\\"~n\", [??MsgPattern, UnittestActualMessage])\n                                end\n                            MsgPattern -> true\n                        after\n                            Timeout ->\n                                UnittestActualMessage =\n                                    receive\n                                        X -> X\n                                    after\n                                        0 -> no_message\n                                    end,\n                                ?ct_fail(\"expected message ~s but got \\\"~.0p\\\"~n\", [??MsgPattern, UnittestActualMessage])\n                        end\n                    MsgPattern -> true\n                after\n                    Timeout ->\n                        UnittestActualMessage =\n                            receive\n                                X -> X\n                            after\n                                0 -> no_message\n                            end,\n                        ?ct_fail(\"expected message ~s but got \\\"~.0p\\\"~n\", [??MsgPattern, UnittestActualMessage])\n                end\n        end()).\n-define(expect_message_ignore(MsgPattern, IgnoredMessage), ?expect_message_ignore(MsgPattern, IgnoredMessage, 1000)).\n\n%TODO: enhance expect_message_ignore to allow lists of ignored messages\n%TODO: find a way to implement the expect_message_ignore macro in a recursive way similar to the following template:\n\n% maybe also define a expect_message macro which ignores certain messages, similar to this function:\n%% expect_message_ignore(MsgPattern, IgnoredMessage) ->\n%%     receive\n%%         IgnoredMessage ->\n%%             ct:pal(\"ignored ~.0p~n\", [IgnoredMessage]),\n%%             expect_message_ignore(MsgPattern, IgnoredMessage);\n%%         MsgPattern ->\n%%             ok\n%%     after\n%%         1000 ->\n%%             ActualMessage = receive\n%%                                 X ->\n%%                                     X\n%%                             after\n%%                                 0 ->\n%%                                     unknown\n%%                             end,\n%%             ?ct_fail(\"expected message ~.0p but got ~.0p\", [MsgPattern, ActualMessage])\n%%     end.\n"
  },
  {
    "path": "test/unittest_global_state.erl",
    "content": "%  @copyright 2010-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Global state for unit tests\n%% @end\n%% @version $Id$\n-module(unittest_global_state).\n-author('schuett@zib.de').\n\n-export([lookup/1, insert/2, delete/1,\n         delete/0]).\n-export([register_thread/1, take_registered_threads/0]).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% create and query ets-table\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n\n-spec lookup(any()) -> failed | any().\nlookup(Key) ->\n    try ets:lookup(?MODULE, Key) of\n        [{Key, Value}] -> Value;\n        [] -> failed\n    catch error:badarg -> failed\n    end.\n\n-spec insert(Key::term(), Value::term()) -> true.\ninsert(Key, Value) ->\n    try ets:insert(?MODULE, {Key, Value})\n    catch error:badarg ->\n              % probably the table does not exist\n              create_table(),\n              ets:insert(?MODULE, {Key, Value})\n    end.\n\n%% @doc Registers the calling process as a unit test thread.\n-spec register_thread(ThreadNr::pos_integer()) -> true.\nregister_thread(ThreadNr) ->\n    try ets:insert(?MODULE, {{thread, ThreadNr, self()}})\n    catch error:badarg ->\n              % probably the table does not exist\n              create_table(),\n              ets:insert(?MODULE, {{thread, ThreadNr, self()}})\n    end.\n\n-spec take_registered_threads() -> [{ThreadNr::pos_integer(), ThreadPid::pid()}].\ntake_registered_threads() ->\n    try begin\n            Matches = ets:match(?MODULE, {{thread, '$1', '$2'}}),\n            [begin\n                 delete({thread, ThreadNr, ThreadPid}),\n                 {ThreadNr, ThreadPid}\n             end || [ThreadNr, ThreadPid] <- Matches]\n        end\n    catch error:badarg -> failed\n    end.\n\n-spec delete(Key::term()) -> true | ok.\ndelete(Key) ->\n    try begin\n            case ets:member(?MODULE, Key) of\n                true ->\n                    ets:delete(?MODULE, Key);\n                _ ->\n                    %% unregister non registered object\n                    ct:pal(\"you tried to unregister ~w which is not registered~nStacktrace: ~p\",\n                           [Key, util:get_stacktrace()]),\n                    throw({unittest_global_state_delete_unregistered_object, Key})\n            end\n        end\n    catch ?CATCH_CLAUSE_WITH_STACKTRACE(error, badarg, Stacktrace)\n              % probably the table does not exist\n              ct:pal(\"Stacktrace: ~p\", [Stacktrace]),\n              throw({unittest_global_state_unknown_table, Key})\n    end.\n\n%% @doc Deletes the whole table (and the accompanying test)\n-spec delete() -> ok.\ndelete() ->\n    case erlang:whereis(?MODULE) of\n        undefined -> ok;\n        Pid when is_pid(Pid) ->\n            MonitorRef = erlang:monitor(process, Pid),\n            Pid ! {kill, self()},\n            receive\n                ok -> erlang:demonitor(MonitorRef, [flush]), ok;\n                {'DOWN', MonitorRef, process, Pid, _Info1} -> ok\n            end,\n            ok\nend.\n\n-spec create_table() -> true.\ncreate_table() ->\n    P = self(),\n    spawn(\n      fun() ->\n              IsOwner =\n                  try\n                      _ = ets:new(?MODULE, [set, public, named_table]),\n                      true\n                  catch\n                      % is there a race-condition?\n                      Error:Reason ->\n                          case ets:info(?MODULE) of\n                              undefined ->\n                                  ?ct_fail(\"could not create ets table for unittest_global_state: ~p:~p\",\n                                           [Error, Reason]);\n                              _ -> false\n                          end\n                  end,\n              P ! go,\n              case IsOwner of\n                  true ->\n                      erlang:register(?MODULE, self()),\n                      receive {kill, Pid} -> Pid ! ok\n                      end;\n                  false -> ok\n              end\n      end),\n    receive go -> true end.\n"
  },
  {
    "path": "test/unittest_helper.erl",
    "content": "%  @copyright 2008-2018 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Helper functions for Unit tests\n%% @end\n%% @version $Id$\n-module(unittest_helper).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n%%-define(TRACE_RING_DATA(), print_ring_data()).\n-define(TRACE_RING_DATA(), ok).\n\n-export([get_scalaris_port/0, get_yaws_port/0,\n         make_ring_with_ids/1, make_ring_with_ids/2, make_ring/1, make_ring/2, make_ring_recover/1,\n         make_symmetric_ring/0, make_symmetric_ring/1,\n         stop_ring/0, stop_ring/1,\n         stop_pid_groups/0,\n         check_ring_size/1, check_ring_size_fully_joined/1,\n         wait_for_stable_ring/0, wait_for_stable_ring_deep/0,\n         start_process/1, start_process/2,\n         start_subprocess/1, start_subprocess/2,\n         get_processes/0, kill_new_processes/1, kill_new_processes/2,\n         create_ct_all/1, create_ct_groups/2,\n         init_per_group/2, end_per_group/2,\n         get_ring_data/1, print_ring_data/0,\n         print_proto_sched_stats/1, print_proto_sched_stats/2,\n         macro_equals/5, macro_compare/7,\n         macro_equals_failed/6,\n         expect_no_message_timeout/1,\n         prepare_config/1,\n         start_minimal_procs/3, stop_minimal_procs/1,\n         check_ring_load/1, check_ring_data/0, check_ring_data/2,\n         build_interval/2,\n         db_entry_not_null/1, scrub_data/1]).\n\n-export_type([process_info/0, kv_opts/0, ring_data/1]).\n\n-include(\"scalaris.hrl\").\n-include(\"client_types.hrl\").\n-include(\"unittest.hrl\").\n\n-dialyzer({no_match, get_processes/0}).\n\n-type kv_opts() :: [{Key::atom(), Value::term()}].\n\n-spec get_port(EnvName::string(), Default::pos_integer()) -> pos_integer().\nget_port(EnvName, Default) ->\n    case os:getenv(EnvName) of\n        false -> Default;\n        X ->\n            try erlang:list_to_integer(X)\n            catch\n                _:_ -> Default\n            end\n    end.\n\n-spec get_scalaris_port() -> pos_integer().\nget_scalaris_port() ->\n    get_port(\"SCALARIS_UNITTEST_PORT\", 14195).\n\n-spec get_yaws_port() -> pos_integer().\nget_yaws_port() ->\n    get_port(\"SCALARIS_UNITTEST_YAWS_PORT\", 8000).\n\n%% @doc Adds unittest-specific config parameters to the given key-value list.\n%%      The following parameters are added7changed:\n%%       - SCALARIS_UNITTEST_PORT environment variable to specify the port to\n%%         listen on\n%%       - SCALARIS_UNITTEST_YAWS_PORT environment variable to specify the port\n%%         yaws listens on\n%%       - adds only a single known_hosts node\n%%       - specifies the node to be empty\n-spec add_my_config(ConfigKVList::[{atom(), term()}])\n    -> NewKVList::[{atom(), term()}].\nadd_my_config(KVList) ->\n    % add empty_node to the end (so it is not overwritten)\n    % but add known_hosts to the beginning so it\n    % can be overwritten by the Options\n    ScalarisPort = get_scalaris_port(),\n    YawsPort = get_yaws_port(),\n    KVList1 = [{known_hosts, [{{127,0,0,1}, ScalarisPort, service_per_vm}]},\n               {mgmt_server, {{127,0,0,1}, ScalarisPort, mgmt_server}},\n               {port, ScalarisPort},\n               {yaws_port, YawsPort},\n               {start_type, first_nostart} | KVList],\n    lists:append(KVList1, [{start_mgmt_server, true}]).\n\n%% @doc Adds unittest specific ports from the environment to the list of\n%%      options for the config process.\n-spec prepare_config(Options::[{atom(), term()}]) -> NewOptions::[{atom(), term()}].\nprepare_config(Options) ->\n    prepare_config_helper(Options, false).\n\n-spec prepare_config_helper(Options, ConfigFound::boolean()) -> Options when is_subtype(Options, [{atom(), term()}]).\nprepare_config_helper([], false) ->\n    [{config, add_my_config([])}];\nprepare_config_helper([], true) ->\n    [];\nprepare_config_helper([Option | Rest], OldConfigFound) ->\n    {NewOption, ConfigFound} =\n        case Option of\n            {config, KVList} -> {{config, add_my_config(KVList)}, true};\n            X                -> {X, OldConfigFound}\n        end,\n    [NewOption | prepare_config_helper(Rest, ConfigFound)].\n\n%% @doc Creates a symmetric ring.\n-spec make_symmetric_ring() -> pid().\nmake_symmetric_ring() ->\n    make_symmetric_ring([]).\n\n%% @doc Creates a symmetric ring.\n-spec make_symmetric_ring(Options::kv_opts()) -> pid().\nmake_symmetric_ring(Options) ->\n    ct:pal(\"Trying to make a symmetric ring\"),\n    NodeAddFun =\n        fun() ->\n                Ids = case lists:keyfind(scale_ring_size_by, 1, Options) of\n                          {scale_ring_size_by, Scale} ->\n                              api_rt:get_evenly_spaced_keys(\n                                Scale * config:read(replication_factor));\n                          false ->\n                              api_rt:get_evenly_spaced_keys(\n                                config:read(replication_factor))\n                      end,\n                [admin:add_node([{first}, {{dht_node, id}, hd(Ids)}])\n                 | [admin:add_node_at_id(Id) || Id <- tl(Ids)]]\n        end,\n    Pid = make_ring_generic(Options, NodeAddFun),\n    Size = case lists:keyfind(scale_ring_size_by, 1, Options) of\n               {scale_ring_size_by, Scale} ->\n                   Scale * config:read(replication_factor);\n               false ->\n                   config:read(replication_factor)\n           end,\n    check_ring_size(Size),\n    wait_for_stable_ring(),\n    check_ring_size(Size),\n    ct:pal(\"Scalaris booted with ~p node(s)...~n\", [Size]),\n    TimeTrap = test_server:timetrap(3000 + Size * 1000),\n    test_server:timetrap_cancel(TimeTrap),\n    Pid.\n\n%% @doc Creates a ring with the given IDs (or IDs returned by the IdFun).\n-spec make_ring_with_ids([?RT:key()]) -> pid().\nmake_ring_with_ids(Ids) ->\n    make_ring_with_ids(Ids, []).\n\n%% @doc Creates a ring with the given IDs (or IDs returned by the IdFun).\n%%      Passes Options to the supervisor, e.g. to set config variables, specify\n%%      a {config, [{Key, Value},...]} option.\n-spec make_ring_with_ids([?RT:key(),...], Options::kv_opts()) -> pid().\nmake_ring_with_ids(Ids, Options) when is_list(Ids) ->\n  ct:pal(\"Trying to make ring with Ids, containing ~p nodes.\",[erlang:length(Ids)]),\n  NodeAddFun =\n        fun() ->\n                [admin:add_node([{first}, {{dht_node, id}, hd(Ids)}])\n                 | [admin:add_node_at_id(Id) || Id <- tl(Ids)]]\n        end,\n  Size = erlang:length(Ids),\n  TimeTrap = test_server:timetrap(3000 + Size * 1000),\n  Pid = make_ring_generic(Options, NodeAddFun),\n  check_ring_size(Size),\n  wait_for_stable_ring(),\n  check_ring_size(Size),\n  ct:pal(\"Scalaris booted with ~p node(s)...~n\", [Size]),\n  test_server:timetrap_cancel(TimeTrap),\n  Pid.\n\n-spec make_ring_recover(Options::kv_opts()) -> pid().\nmake_ring_recover(Options) ->\n  ct:pal(\"Trying to recover ring.\"),\n  TimeTrap = test_server:timetrap(60000),\n  Pid = make_ring_generic(Options, fun() -> [] end),\n  ct:pal(\"ring restored\"),\n  test_server:timetrap_cancel(TimeTrap),\n  Pid.\n\n%% @doc Creates a ring with Size random IDs.\n-spec make_ring(Size::pos_integer()) -> pid().\nmake_ring(Size) ->\n    make_ring(Size, []).\n\n%% @doc Creates a ring with Size rangom IDs.\n%%      Passes Options to the supervisor, e.g. to set config variables, specify\n%%      a {config, [{Key, Value},...]} option.\n-spec make_ring(Size::pos_integer(), Options::kv_opts()) -> pid().\nmake_ring(Size, Options) ->\n  ct:pal(\"Trying to make ring with ~p nodes.\",[Size]),\n    NodeAddFun =\n        fun() ->\n                First = admin:add_node([{first}]),\n                {RestSuc, RestFailed} = admin:add_nodes(Size - 1),\n                [First | RestSuc ++ RestFailed]\n        end,\n    TimeTrap = test_server:timetrap(3000 + Size * 1000),\n    Pid = make_ring_generic(Options, NodeAddFun),\n    check_ring_size(Size),\n    wait_for_stable_ring(),\n    check_ring_size(Size),\n    ct:pal(\"Scalaris booted with ~p node(s)...~n\", [Size]),\n    test_server:timetrap_cancel(TimeTrap),\n    Pid.\n\n-spec make_ring_generic(Options::kv_opts(),\n                        NodeAddFun::fun(() -> [pid_groups:groupname() | {error, term()}]))\n        -> pid().\nmake_ring_generic(Options, NodeAddFun) ->\n    case ets:info(config_ets) of\n        undefined -> ok;\n        _         -> ct:fail(\"Trying to create a new ring although there is already one.\")\n    end,\n    {Pid, StartRes} =\n        start_process(\n          fun() ->\n                  erlang:register(ct_test_ring, self()),\n                  randoms:start(),\n                  {ok, _GroupsPid} = pid_groups:start_link(),\n                  NewOptions = prepare_config(Options),\n                  config:init(NewOptions),\n                  tester:start_pseudo_proc(),\n                  {ok, _} = sup_scalaris:start_link(NewOptions),\n                  NodeAddFun()\n          end),\n    FailedNodes = [X || X = {error, _ } <- StartRes],\n    case FailedNodes of\n        [] -> ok;\n        [_|_] -> stop_ring(Pid),\n                 ?ct_fail(\"adding nodes failed: ~.0p\", [FailedNodes])\n    end,\n    true = erlang:is_process_alive(Pid),\n%%     timer:sleep(1000),\n    % need to call IdsFun again (may require config process or others\n    % -> can not call it before starting the scalaris process)\n    ?TRACE_RING_DATA(),\n    Pid.\n\n%% @doc Stops a ring previously started with make_ring/1 or make_ring_with_ids/1.\n-spec stop_ring() -> ok.\nstop_ring() ->\n    case erlang:whereis(ct_test_ring) of\n        undefined -> ok;\n        Pid       -> stop_ring(Pid)\n    end.\n\n%% @doc Stops a ring previously started with make_ring/1 or make_ring_with_ids/1\n%%      when the process' pid is known.\n-spec stop_ring(pid()) -> ok.\nstop_ring(Pid) ->\n    LogLevel = config:read(log_level),\n    try\n        begin\n            ct:pal(\"unittest_helper:stop_ring start.\"),\n            log:set_log_level(none),\n            sup:sup_terminate(main_sup),\n            catch exit(Pid, kill),\n            util:wait_for_process_to_die(Pid),\n            stop_pid_groups(),\n            sup_scalaris:stop_first_services(),\n            log:set_log_level(LogLevel),\n            ct:pal(\"unittest_helper:stop_ring done.\"),\n            ok\n        end\n    catch\n        ?CATCH_CLAUSE_WITH_STACKTRACE(Level, Reason, Stacktrace)\n            ct:pal(\"~s in stop_ring: ~p~n~.0p\", [Level, Reason, Stacktrace]),\n            erlang:Level(Reason)\n    after\n            catch(unregister(ct_test_ring)),\n            log:set_log_level(LogLevel)\n    end.\n\n-spec stop_pid_groups() -> ok.\nstop_pid_groups() ->\n    case whereis(pid_groups) of\n        PidGroups when is_pid(PidGroups) ->\n            gen_component:kill(PidGroups),\n            util:wait_for_ets_table_to_disappear(PidGroups, pid_groups),\n            util:wait_for_ets_table_to_disappear(PidGroups, pid_groups_hidden),\n            catch unregister(pid_groups),\n            ok;\n        _ -> ok\n    end.\n\n\n-spec wait_for_stable_ring() -> ok.\nwait_for_stable_ring() ->\n    util:wait_for(fun() ->\n                          R = admin:check_ring(),\n                          %% ct:pal(\"CheckRing: ~p~n\", [R]),\n                          R =:= ok\n             end, 100).\n\n-spec wait_for_stable_ring_deep() -> ok.\nwait_for_stable_ring_deep() ->\n    util:wait_for(fun() ->\n                     R = admin:check_ring_deep(),\n                     ct:pal(\"CheckRingDeep: ~p~n\", [R]),\n                     R =:= ok\n             end, 100).\n\n-spec check_ring_size(non_neg_integer(), CheckFun::fun((DhtNodeState::term()) -> boolean())) -> ok.\ncheck_ring_size(Size, CheckFun) ->\n    util:wait_for(\n      fun() ->\n              % note: we use a single VM in unit tests, therefore no\n              % mgmt_server is needed - if one exists though, then check\n              % the correct size\n              BootSize =\n                  try\n                      mgmt_server:number_of_nodes(),\n                      trace_mpath:thread_yield(),\n                      receive ?SCALARIS_RECV({get_list_length_response, L}, L) end\n                  catch _:_ -> Size\n                  end,\n              BootSize =:= Size andalso\n                  Size =:= erlang:length(\n                             [P || P <- pid_groups:find_all(dht_node),\n                                   CheckFun(gen_component:get_state(P))])\n      end, 50).\n\n-spec check_ring_size(Size::non_neg_integer()) -> ok.\ncheck_ring_size(Size) ->\n    check_ring_size(Size, fun(State) ->\n                                  DhtModule = config:read(dht_node),\n                                  DhtModule:is_alive(State)\n                          end).\n\n%% @doc Checks whether Size nodes have fully joined the ring (including\n%%      finished join-related slides).\n-spec check_ring_size_fully_joined(Size::non_neg_integer()) -> ok.\ncheck_ring_size_fully_joined(Size) ->\n    check_ring_size(Size, fun(State) ->\n                                  DhtModule = config:read(dht_node),\n                                  DhtModule:is_alive_no_slide(State)\n                          end).\n\n%% @doc Starts a process which executes the given function and then waits forever.\n-spec start_process(StartFun::fun(() -> StartRes)) -> {pid(), StartRes}.\nstart_process(StartFun) ->\n    start_process(StartFun, fun() -> receive {done} -> ok end end).\n\n-spec start_process(StartFun::fun(() -> StartRes), RunFun::fun(() -> any())) -> {pid(), StartRes}.\nstart_process(StartFun, RunFun) ->\n    start_process(StartFun, RunFun, false, spawn).\n\n-spec start_process(StartFun::fun(() -> StartRes), RunFun::fun(() -> any()),\n                    TrapExit::boolean(), Spawn::spawn | spawn_link)\n        -> {pid(), StartRes}.\nstart_process(StartFun, RunFun, TrapExit, Spawn) ->\n    process_flag(trap_exit, TrapExit),\n    Owner = self(),\n    Node = erlang:Spawn(\n             fun() ->\n                     try Res = StartFun(),\n                         Owner ! {started, self(), Res}\n                     catch Level:Reason ->\n                               Owner ! {killed},\n                               erlang:Level(Reason)\n                     end,\n                     RunFun()\n             end),\n    receive\n        {started, Node, Res} -> {Node, Res};\n        {killed} -> ?ct_fail(\"start_process(~.0p, ~.0p, ~.0p, ~.0p) failed\",\n                             [StartFun, RunFun, TrapExit, Spawn])\n    end.\n\n%% @doc Starts a sub-process which executes the given function and then waits forever.\n-spec start_subprocess(StartFun::fun(() -> StartRes)) -> {pid(), StartRes}.\nstart_subprocess(StartFun) ->\n    start_subprocess(StartFun, fun() -> receive {done} -> ok end end).\n\n-spec start_subprocess(StartFun::fun(() -> StartRes), RunFun::fun(() -> any())) -> {pid(), StartRes}.\nstart_subprocess(StartFun, RunFun) ->\n    start_process(StartFun, RunFun, true, spawn_link).\n\n%% @doc Starts the minimal number of processes in order for non-ring unit tests\n%%      to be able to execute (pid_groups, config, log).\n-spec start_minimal_procs(CTConfig, ConfigOptions::[{atom(), term()}],\n                          StartCommServer::boolean()) -> CTConfig when is_subtype(CTConfig, list()).\nstart_minimal_procs(CTConfig, ConfigOptions, StartCommServer) ->\n    {Pid, _} =\n        start_process(\n          fun() ->\n                  {ok, _GroupsPid} = pid_groups:start_link(),\n                  {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, CTConfig),\n                  ConfigOptions2 = prepare_config(\n                                     [{config, [{log_path, PrivDir} | ConfigOptions]}]),\n                  config:init(ConfigOptions2),\n                  tester:start_pseudo_proc(),\n                  case StartCommServer of\n                      true ->\n                          {ok, _CommPid} = sup:sup_start(\n                                             {local, no_name},\n                                             sup_comm_layer, []),\n                          Port = get_scalaris_port(),\n                          comm_server:set_local_address({127,0,0,1}, Port);\n                      false -> ok\n                  end\n          end),\n    [{wrapper_pid, Pid} | CTConfig].\n\n%% @doc Stops the processes started by start_minimal_procs/3 given the\n%%      ct config term.\n-spec stop_minimal_procs(CTConfig) -> CTConfig when is_subtype(CTConfig, list()).\nstop_minimal_procs(CTConfig)  ->\n    case lists:keytake(wrapper_pid, 1, CTConfig) of\n        {value, {wrapper_pid, Pid}, CTConfig1} ->\n            LogLevel = config:read(log_level),\n            log:set_log_level(none),\n            exit(Pid, kill),\n            stop_pid_groups(),\n            log:set_log_level(LogLevel),\n            CTConfig1;\n        false -> CTConfig\n    end.\n\n-type process_info() ::\n    {pid(), InitCall::mfa(), CurFun::mfa(),\n     {RegName::atom() | not_registered, Info::term() | failed | no_further_infos}}.\n\n-spec get_processes() -> [process_info()].\nget_processes() ->\n    [begin\n         InitCall =\n             case proc_lib:initial_call(X) of\n                 false -> element(2, lists:keyfind(initial_call, 1, Data));\n                 {Module, Function, Args} -> {Module, Function, length(Args)}\n             end,\n         CurFun = element(2, lists:keyfind(current_function, 1, Data)),\n         RegName = case lists:keyfind(registered_name, 1, Data) of\n                       false -> not_registered;\n                       RegNameTpl -> element(2, RegNameTpl)\n                   end,\n         Info =\n             case {InitCall, CurFun} of\n                 {{gen_component, _, _}, {gen_component, _, _}} ->\n                     gen_component:get_component_state(X);\n                 {_, {file_io_server, _, _}} ->\n                     case file:pid2name(X) of\n                         undefined -> {file, undefined};\n                         {ok, FileName} -> {file, FileName};\n                         %% case occuring in older Erlang versions:\n                         FileName -> {file, FileName}\n                     end;\n                 {_, {gen_server, _, _}} ->\n                     % may throw exit due to a killed process\n                     try sys:get_status(X)\n                     catch _:_ -> no_further_infos\n                     end;\n                 _ -> no_further_infos\n             end,\n         {X, InitCall, CurFun, {RegName, Info}}\n     end\n     || X <- processes(),\n        Data <- [process_info(X, [current_function, initial_call, registered_name])],\n        Data =/= undefined].\n\n-spec kill_new_processes(OldProcesses::[process_info()]) -> ok.\nkill_new_processes(OldProcesses) ->\n    kill_new_processes(OldProcesses, []).\n\n-spec kill_new_processes(OldProcesses::[process_info()], Options::list()) -> ok.\nkill_new_processes(OldProcesses, Options) ->\n    NewProcesses = get_processes(),\n    {_OnlyOld, _Both, OnlyNew} =\n        util:split_unique(OldProcesses, NewProcesses,\n                          fun(P1, P2) ->\n                                  element(1, P1) =< element(1, P2)\n                          end, fun(_P1, P2) -> P2 end),\n%%     ct:pal(\"Proc-Old: ~.0p~n\", [_OnlyOld]),\n%%     ct:pal(\"Proc-Both: ~.0p~n\", [_Both]),\n%%     ct:pal(\"Proc-New: ~.0p~n\", [OnlyNew]),\n    Killed = [begin\n%%                   ct:pal(\"killing ~.0p~n\", [Proc]),\n                  Tabs = util:ets_tables_of(X),\n                  try erlang:exit(X, kill) of\n                      true ->\n                          util:wait_for_process_to_die(X),\n                          _ = [begin\n%%                                    ct:pal(\"waiting for table ~.0p to disappear~n~p\",\n%%                                           [Tab, ets:info(Tab)]),\n                                   util:wait_for_ets_table_to_disappear(X, Tab)\n                               end || Tab <- Tabs ],\n                              {ok, Proc}\n                  catch _:_ -> {fail, Proc}\n                  end\n              end || {X, InitCall, CurFun, {RegName, _Info}} = Proc <- OnlyNew,\n                     not (InitCall =:= {test_server_sup, timetrap, 3} andalso\n                              CurFun =:= {test_server_sup, timetrap, 3}),\n                     not (InitCall =:= {test_server_sup, timetrap, 2} andalso\n                              CurFun =:= {test_server_sup, timetrap, 2}),\n                     not (InitCall =:= {test_server_gl, init, 1}),\n                     not (CurFun =:= {prim_file, drv_get_response, 1}), % spawned by R16 for io:put_chars/3\n                     RegName =/= cover_server,\n                     X =/= self(),\n                     X =/= whereis(timer_server),\n                     element(1, CurFun) =/= file_io_server],\n    case lists:member({?quiet}, Options) of\n        false ->\n            ct:pal(\"Killed processes: ~.0p~n\", [Killed]);\n        true -> ok\n    end.\n\n-type ct_group_props() ::\n    [parallel | sequence | shuffle | {shuffle, {integer(), integer(), integer()}} |\n     {repeat | repeat_until_all_ok | repeat_until_all_fail | repeat_until_any_ok | repeat_until_any_fail, integer() | forever}].\n\n-spec testcase_to_groupname(TestCase::atom()) -> atom().\ntestcase_to_groupname(TestCase) ->\n    erlang:list_to_atom(erlang:atom_to_list(TestCase) ++ \"_grp\").\n\n-spec create_ct_all(TestCases::[atom()]) -> [{group, atom()}].\ncreate_ct_all(TestCases) ->\n    [{group, testcase_to_groupname(TestCase)} || TestCase <- TestCases].\n\n-spec create_ct_groups(TestCases::[atom()], SpecialOptions::[{atom(), ct_group_props()}])\n        -> [{atom(), ct_group_props(), [atom()]}].\ncreate_ct_groups(TestCases, SpecialOptions) ->\n    [begin\n         Options =\n             case lists:keyfind(TestCase, 1, SpecialOptions) of\n                 false -> [sequence, {repeat, 1}];\n                 {_, X} -> X\n             end,\n         {testcase_to_groupname(TestCase), Options, [TestCase]}\n     end || TestCase <- TestCases].\n\n-spec init_per_group(atom(), Config) -> Config when is_subtype(Config, kv_opts()).\ninit_per_group(_Group, Config) -> Config.\n\n-spec end_per_group(atom(), kv_opts()) -> {return_group_result, ok | failed}.\nend_per_group(_Group, Config) ->\n    Status = test_server:lookup_config(tc_group_result, Config),\n    case proplists:get_value(failed, Status) of\n        [] ->                                   % no failed cases\n            {return_group_result, ok};\n        _Failed ->                              % one or more failed\n            {return_group_result, failed}\n    end.\n\n-type ring_data(DBType) ::\n          [{pid(),\n            {intervals:left_bracket(), intervals:key(), intervals:key(), intervals:right_bracket()},\n            DBType,\n            {pred, comm:erl_local_pid_plain()},\n            {succ, comm:erl_local_pid_plain()},\n            ok | timeout}] |\n          {exception, Level::throw | error | exit, Reason::term(), Stacktrace::term()}.\n\n-spec get_ring_data(full) -> ring_data(db_dht:db_as_list());\n                   (kv) -> ring_data([{?RT:key(), client_version()}]).\nget_ring_data(Type) ->\n    Self = self(),\n    erlang:spawn(\n      fun() ->\n              try\n                  DHTNodes = pid_groups:find_all(dht_node),\n                  Data =\n                      lists:sort(\n                        fun(E1, E2) ->\n                                erlang:element(2, E1) =< erlang:element(2, E2)\n                        end,\n                        [begin\n                             comm:send_local(DhtNode,\n                                             {unittest_get_bounds_and_data, comm:this(), Type}),\n                             receive\n                                 {unittest_get_bounds_and_data_response, Bounds, Data, Pred, Succ} ->\n                                     {DhtNode, Bounds, Data,\n                                      {pred, comm:make_local(node:pidX(Pred))},\n                                      {succ, comm:make_local(node:pidX(Succ))}, ok}\n                             % we are in a separate process, any message in the\n                             % message box should not influence the calling process\n                                 after 750 -> {DhtNode, empty, [], {pred, null}, {succ, null}, timeout}\n                             end\n                         end || DhtNode <- DHTNodes]),\n                  Self ! {data, Data}\n              catch\n                  ?CATCH_CLAUSE_WITH_STACKTRACE(Level, Reason, Stacktrace)\n                      Result =\n                          case erlang:whereis(ct_test_ring) of\n                              undefined -> [];\n                              _         -> {exception, Level, Reason, Stacktrace}\n                          end,\n                      Self ! {data, Result}\n              end\n      end),\n    receive {data, Data} -> Data\n    end.\n\n-spec print_ring_data() -> ok.\nprint_ring_data() ->\n    DataAll = get_ring_data(full),\n    ct:pal(\"Scalaris ring data:~n~.0p~n\", [DataAll]).\n\n-spec print_proto_sched_stats(now | now_short | at_end_if_failed | at_end_if_failed_append) -> ok.\nprint_proto_sched_stats(When) ->\n    % NOTE: keep the default tag in sync!\n    print_proto_sched_stats(When, default).\n\n-spec print_proto_sched_stats(now | now_short | at_end_if_failed | at_end_if_failed_append,\n                              TraceId::term()) -> ok.\nprint_proto_sched_stats(now, TraceId) ->\n    ct:pal(\"Proto scheduler stats: ~.2p\", [proto_sched:get_infos(TraceId)]);\nprint_proto_sched_stats(now_short, TraceId) ->\n    ct:pal(\"Proto scheduler stats: ~.2p\",\n           [proto_sched:info_shorten_messages(proto_sched:get_infos(TraceId), 200)]);\nprint_proto_sched_stats(at_end_if_failed, TraceId) ->\n    unittest_global_state:insert(proto_sched_stats, proto_sched:get_infos(TraceId)),\n    ok;\nprint_proto_sched_stats(at_end_if_failed_append, TraceId) ->\n    Current = case unittest_global_state:lookup(proto_sched_stats) of\n                  failed -> [];\n                  X when is_list(X) -> X\n              end,\n    unittest_global_state:insert(proto_sched_stats,\n                                 Current ++ proto_sched:get_infos(TraceId)),\n    ok.\n\n-spec macro_equals(Actual::any(), ExpectedVal::any(), ActualStr::string(),\n                   ExpectedStr::string(), Note::term()) -> true | no_return().\nmacro_equals(Actual, ExpectedVal, ActualStr, ExpectedStr, Note) ->\n    macro_compare(fun erlang:'=:='/2, Actual, ExpectedVal, \"=:=\", ActualStr, ExpectedStr, Note).\n\n-spec macro_compare(CompFun::fun((T, T) -> boolean()), Actual::any(), ExpectedVal::any(),\n                    CompFunStr::string(), ActualStr::string(), ExpectedStr::string(),\n                    Note::term()) -> true | no_return().\nmacro_compare(CompFun, Actual, ExpectedVal, CompFunStr, ActualStr, ExpectedStr, Note) ->\n    case CompFun(Actual, ExpectedVal) of\n        true  -> true;\n        false -> macro_equals_failed(Actual, ExpectedVal, CompFunStr, ActualStr, ExpectedStr, Note)\n    end.\n\n-spec macro_equals_failed(ActualVal::any(), ExpectedVal::any(), CompFunStr::string(),\n                          ActualStr::string(), ExpectedStr::string(), Note::term())\n        -> no_return().\nmacro_equals_failed(ActualVal, ExpectedVal, CompFunStr, ActualStr, ExpectedStr, Note) ->\n    log:log(error, \"Failed (in ~.0p)~n\"\n           \" Call:        ~s~n\"\n           \" Returns:     ~.0p~n\"\n           \" is not (~s)~n\"\n           \" Exp. term:   ~s~n\"\n           \" Exp. return: ~.0p~n\"\n           \" Note:        ~.0p~n\"\n           \" Stacktrace:  ~p~n\"\n           \" Linetrace:   ~p~n\",\n           [self(), ActualStr, ActualVal, CompFunStr, ExpectedStr, ExpectedVal, Note,\n            util:get_stacktrace(), util:get_linetrace()]),\n    ?ct_fail(\"~s returned ~.0p which is not ~s the expected ~s that returns ~.0p. Note: ~.0p\",\n             [ActualStr, ActualVal, CompFunStr, ExpectedStr, ExpectedVal, Note]).\n\n-spec expect_no_message_timeout(Timeout::pos_integer()) -> true | no_return().\nexpect_no_message_timeout(Timeout) ->\n    receive\n        ActualMessage ->\n            ?ct_fail(\"expected no message but got \\\"~.0p\\\"~n\", [ActualMessage])\n    after Timeout -> true\nend.\n\n-spec check_ring_load(ExpLoad::pos_integer()) -> true | no_return().\ncheck_ring_load(ExpLoad) ->\n    Ring = statistics:get_ring_details(),\n    Load = statistics:get_total_load(load, Ring),\n    ?equals(Load, ExpLoad).\n\n-spec check_ring_data() -> boolean().\ncheck_ring_data() ->\n    check_ring_data(250, 8).\n\n-spec check_ring_data(Timeout::pos_integer(), Retries::non_neg_integer()) -> boolean().\ncheck_ring_data(Timeout, Retries) ->\n    Data = lists:flatmap(\n             fun({_Pid, _Interval, Data, {pred, _PredPid}, {succc, _SuccPid}, _Result}) -> Data;\n                (_) -> []\n             end,\n             get_ring_data(full)),\n    case Retries < 1 of\n        true ->\n            check_ring_data_all(Data, true);\n        _ ->\n            case check_ring_data_all(Data, false) of\n                true -> true;\n                _    -> timer:sleep(Timeout),\n                        check_ring_data(Timeout, Retries - 1)\n            end\n    end.\n\ncheck_ring_data_all(Data, FailOnError) ->\n    check_ring_data_unique(Data, FailOnError) andalso\n        check_ring_data_repl_fac(Data, FailOnError) andalso\n        check_ring_data_lock_free(Data, FailOnError) andalso\n        check_ring_data_repl_exist(Data, FailOnError).\n\n-spec check_ring_data_unique(Data::[db_entry:entry()], FailOnError::boolean()) -> boolean().\ncheck_ring_data_unique(Data, FailOnError) ->\n    UniqueData = lists:usort(fun(A, B) ->\n                                     db_entry:get_key(A) =< db_entry:get_key(B)\n                             end, Data),\n    DataDiff = lists:subtract(Data, UniqueData),\n    case FailOnError of\n        true ->\n            ?equals_w_note(DataDiff, [], \"duplicate elements detected\"),\n            true;\n        _ when DataDiff =:= [] ->\n            true;\n        _ ->\n            ct:pal(\"check_ring_data: duplicate elements detected: ~p~n\", [DataDiff]),\n            false\n    end.\n\n-spec check_ring_data_repl_fac(Data::[db_entry:entry()], FailOnError::boolean()) -> boolean().\ncheck_ring_data_repl_fac(Data, FailOnError) ->\n    ReplicaKeys0 = ?RT:get_replica_keys(?RT:hash_key(\"0\")),\n    DataLen = length(Data),\n    ReplicaLen = length(ReplicaKeys0),\n    ReplicaRem = DataLen rem ReplicaLen,\n    case FailOnError of\n        true ->\n            ?equals(ReplicaRem, 0);\n        _ when ReplicaRem =:= 0 ->\n            true;\n        _ ->\n            ct:pal(\"check_ring_data: length(Data) = ~p not multiple of ~p~n\", [DataLen, ReplicaLen]),\n            false\n    end.\n\n-spec check_ring_data_lock_free(Data::[db_entry:entry()], FailOnError::boolean()) -> boolean().\ncheck_ring_data_lock_free(Data, FailOnError) ->\n    Locked = [E || E <- Data, db_entry:is_locked(E)],\n    case FailOnError of\n        true ->\n            ?equals_w_note(Locked, [], \"ring is not lock-free\"),\n            true;\n        _ when Locked =:= [] ->\n            true;\n        _ ->\n            ct:pal(\"check_ring_data: locked elements found: ~p~n\", [Locked]),\n            false\n    end.\n\n-spec check_ring_data_repl_exist(Data::[db_entry:entry()], FailOnError::boolean()) -> boolean().\ncheck_ring_data_repl_exist(Data, FailOnError) ->\n    ElementsNotAllReplicas =\n        [E || E <- Data, not data_contains_all_replicas(Data, db_entry:get_key(E))],\n    case FailOnError of\n        true ->\n            ?equals_w_note(ElementsNotAllReplicas, [], \"missing replicas found\"),\n            true;\n        _ when ElementsNotAllReplicas =:= [] ->\n            true;\n        _ ->\n            ct:pal(\"check_ring_data: elements with missing replicas:\", [ElementsNotAllReplicas]),\n            false\n    end.\n\ndata_contains_all_replicas(Data, Key) ->\n    Keys = ?RT:get_replica_keys(Key),\n    Replicas = [E || E <- Data, lists:member(db_entry:get_key(E), Keys)],\n    length(Keys) =:= length(Replicas).\n\n% @doc returns only left-open intervals or intervals:all()\n%      rrepair relies on this\n-spec build_interval(intervals:key(), intervals:key()) -> intervals:interval().\nbuild_interval(?MINUS_INFINITY, ?PLUS_INFINITY) -> intervals:all();\nbuild_interval(?PLUS_INFINITY, ?MINUS_INFINITY) -> intervals:all();\nbuild_interval(A, A) -> intervals:all();\nbuild_interval(A, B) when A < B -> intervals:new('(', A, B, ']');\nbuild_interval(A, B) when A > B -> intervals:new('(', B, A, ']').\n\n%% @doc can be used to filter empty entries out of a list of db_entries\n-spec db_entry_not_null(db_entry:entry()) -> boolean().\ndb_entry_not_null(Entry) ->\n    not db_entry:is_null(Entry).\n\n%% @doc removes duplicates and db_entries that are null\n-spec scrub_data(db_dht:db_as_list()) -> db_dht:db_as_list().\nscrub_data(Data) ->\n    lists:filter(\n        fun db_entry_not_null/1,\n        lists:usort(fun(A, B) ->\n                 db_entry:get_key(A) =< db_entry:get_key(B)\n            end, lists:reverse(Data))).\n"
  },
  {
    "path": "test/unittest_helper_SUITE.erl",
    "content": "% @copyright 2011-2015 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Nico Kruber <kruber@zib.de>\n%% @doc : Unit tests for unittest_helper\n%% @end\n%% @version $Id$\n-module(unittest_helper_SUITE).\n-author('kruber@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"scalaris.hrl\").\n\nall() ->\n    [tester_make_ring_with_ids].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 30}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\ninit_per_testcase(_TestCase, Config) ->\n    [{stop_ring, true} | Config].\n\nend_per_testcase(_TestCase, _Config) ->\n    ok.\n\n-spec prop_make_ring_with_ids(IDs::[?RT:key(),...]) -> true.\nprop_make_ring_with_ids(IDs) ->\n    UniqueIDs = lists:usort(IDs),\n    unittest_helper:make_ring_with_ids(\n      UniqueIDs,\n      [{config, [pdb:get(log_path, ?MODULE)]}]),\n\n    DHTNodes = pid_groups:find_all(dht_node),\n    ?equals(length(DHTNodes), length(UniqueIDs)),\n\n    ActualIds = [begin\n           comm:send_local(DhtNode,\n                           {get_node_details, comm:this(), [node]}),\n           % note: no timeout possible - can not leave messages in queue!\n           receive\n               {get_node_details_response, NodeDetails} ->\n                   node:id(node_details:get(NodeDetails, node))\n%%            after 500 -> timeout\n           end\n       end || DhtNode <- DHTNodes],\n\n    ?equals(lists:sort(ActualIds), lists:sort(UniqueIDs)),\n\n    unittest_helper:stop_ring(),\n    % wait a bit for all processes to stop\n    timer:sleep(100),\n    true.\n\ntester_make_ring_with_ids(Config) ->\n    {priv_dir, PrivDir} = lists:keyfind(priv_dir, 1, Config),\n    pdb:set({log_path, PrivDir}, ?MODULE),\n    tester:test(?MODULE, prop_make_ring_with_ids, 1, 10).\n"
  },
  {
    "path": "test/util_SUITE.erl",
    "content": "% @copyright 2008-2011 Zuse Institute Berlin\n\n%   Licensed under the Apache License, Version 2.0 (the \"License\");\n%   you may not use this file except in compliance with the License.\n%   You may obtain a copy of the License at\n%\n%       http://www.apache.org/licenses/LICENSE-2.0\n%\n%   Unless required by applicable law or agreed to in writing, software\n%   distributed under the License is distributed on an \"AS IS\" BASIS,\n%   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n%   See the License for the specific language governing permissions and\n%   limitations under the License.\n\n%% @author Thorsten Schuett <schuett@zib.de>\n%% @doc    Unit tests for src/util.erl.\n%% @end\n%% @version $Id$\n-module(util_SUITE).\n-author('schuett@zib.de').\n-vsn('$Id$').\n\n-compile(export_all).\n\n-include(\"unittest.hrl\").\n-include(\"types.hrl\").\n\n-dialyzer([{[no_opaque, no_return], largest_smaller_than/1},\n           {no_fail_call, lists_remove_at_indices/1}]).\n\nall() ->\n    [min_max, largest_smaller_than, gb_trees_foldl,\n     repeat, repeat_collect, repeat_accumulate,\n     repeat_p, repeat_p_collect, repeat_p_accumulate,\n     random_subsets,\n     tester_minus_all, tester_minus_all_sort,\n     tester_minus_first, tester_minus_first_sort,\n     tester_par_map2, tester_par_map3,\n     lists_remove_at_indices,\n     tester_timestamp,\n     rrd_combine_timing_slots_handle_empty_rrd,\n     rrd_combine_timing_slots_simple,\n     rrd_combine_timing_slots_subset,\n     rrd_combine_gauge_slots_handle_empty_rrd,\n     rrd_combine_gauge_slots_simple,\n     rrd_combine_gauge_slots_subset,\n     sublist, tester_sublist3\n ].\n\nsuite() ->\n    [\n     {timetrap, {seconds, 20}}\n    ].\n\ninit_per_suite(Config) ->\n    Config.\n\nend_per_suite(_Config) ->\n    ok.\n\n\nmin_max(_Config) ->\n    ?equals(util:min(1, 2), 1),\n    ?equals(util:min(2, 1), 1),\n    ?equals(util:min(1, 1), 1),\n    ?equals(util:max(1, 2), 2),\n    ?equals(util:max(2, 1), 2),\n    ?equals(util:max(1, 1), 1),\n    ok.\n\nlargest_smaller_than(_Config) ->\n    KVs = [{1, 1}, {2, 2}, {4, 4}, {8, 8}, {16, 16}, {32, 32}, {64, 64}],\n    Tree = gb_trees:from_orddict(KVs),\n    ?equals(util:gb_trees_largest_smaller_than(0, Tree), nil),\n    ?equals(util:gb_trees_largest_smaller_than(1, Tree), nil),\n    ?equals(util:gb_trees_largest_smaller_than(2, Tree), {value, 1, 1}),\n    ?equals(util:gb_trees_largest_smaller_than(3, Tree), {value, 2, 2}),\n    ?equals(util:gb_trees_largest_smaller_than(7, Tree), {value, 4, 4}),\n    ?equals(util:gb_trees_largest_smaller_than(9, Tree), {value, 8, 8}),\n    ?equals(util:gb_trees_largest_smaller_than(31, Tree), {value, 16, 16}),\n    ?equals(util:gb_trees_largest_smaller_than(64, Tree), {value, 32, 32}),\n    ?equals(util:gb_trees_largest_smaller_than(65, Tree), {value, 64, 64}),\n    ?equals(util:gb_trees_largest_smaller_than(1000, Tree), {value, 64, 64}),\n    ok.\n\ngb_trees_foldl(_Config) ->\n    KVs = [{1, 1}, {2, 2}, {4, 4}, {8, 8}, {16, 16}, {32, 32}, {64, 64}],\n    Tree = gb_trees:from_orddict(KVs),\n    ?assert(util:gb_trees_foldl(fun (K, K, Acc) ->\n                                        Acc + K\n                                end,\n                                0,\n                                Tree) =:= 127).\n\nrepeat(_) ->\n    util:repeat(fun() -> io:format(\"#s_repeat#~n\") end, [], 5),\n    io:format(\"s_repeat_test successful if #s_repeat# was printed 5 times~n\"),    \n    ok.\n\nrepeat_collect(_) ->\n    Times = 3,\n    Result = util:repeat(fun(X) -> X * X end, [Times], Times, [collect]),\n    ?equals(Result, [9, 9, 9]),\n    ok.\n\nrepeat_accumulate(_) ->\n    Times = 5,\n    A = util:repeat(fun(X) -> X * X end, [Times], Times,\n                    [{accumulate, fun(X, Y) -> X + Y end, 0}]),\n    ?equals(A, Times*Times*Times),\n    B = util:repeat(fun(X) -> X * X end, [Times], Times,\n                    [{accumulate, fun(X, Y) -> X + Y end, 1000}]),\n    ?equals(B, 1000 + Times*Times*Times),      \n    ok.\n\nrepeat_p(_) ->\n    Times = 5,\n    util:repeat(\n      fun(Caller) -> io:format(\"~w #p_repeat_test# called by ~w\", [self(), Caller]) end, \n      [self()], \n      Times, [parallel]),\n    io:format(\"p_repeat_test successful if ~B different pids printed #p_repeat#.\", [Times]),\n    ok.\n\nrepeat_p_collect(_) ->\n    Times = 3,\n    A = util:repeat(fun(X) -> X * X end, [Times], Times, [parallel, collect]),\n    ?equals(A, [9, 9, 9]),\n    ok.\n\nrepeat_p_accumulate(_) ->\n    Times = 15,\n    A = util:repeat(fun(X) -> X * X end, [Times], Times,\n                    [parallel, {accumulate, fun(X, Y) -> X + Y end, 0}]),     \n    ?equals(A, Times*Times*Times),\n    B = util:repeat(fun(X) -> X * X end, [Times], Times,\n                    [parallel, {accumulate, fun(X, Y) -> X + Y end, 1000}]),     \n    ?equals(B, 1000 + Times*Times*Times),   \n    ok.\n\n-spec rand_subsets_check(Rand1::[integer()], Rand2::[integer()], Rand3::[integer()]) -> ok.\nrand_subsets_check(Rand1, Rand2, Rand3) ->\n    Rand1sort = lists:sort(Rand1),\n    Rand2sort = lists:sort(Rand2),\n    Rand3sort = lists:sort(Rand3),\n    ?assert_w_note(Rand1 =/= Rand2 orelse Rand1 =/= Rand3 orelse Rand2 =/= Rand3,\n                   {Rand1, Rand2, Rand3}),\n    ?assert_w_note(Rand1sort =/= Rand2sort orelse Rand1sort =/= Rand3sort orelse Rand2sort =/= Rand3sort,\n                   {Rand1sort, Rand2sort, Rand3sort}),\n    ok.\n\nrandom_subsets(_) ->\n    % assume that when selecting 10 out of 1000 elements, at least one of 3\n    % calls yields a result different to the others\n    List = lists:seq(1, 1000),\n    rand_subsets_check(util:random_subset(10, List),\n                       util:random_subset(10, List),\n                       util:random_subset(10, List)),\n    rand_subsets_check(element(2, util:pop_randomsubset(10, List)),\n                       element(2, util:pop_randomsubset(10, List)),\n                       element(2, util:pop_randomsubset(10, List))),\n\n    ok.\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% util:minus_all/2 and util:minus_first/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%% @doc Checks that all intended items are deleted using util:minus_all/2.\n%%      Note: this is kindof redundant to prop_minus_all_sort/2 but a cleaner\n%%      approach avoiding a re-implementation of util:minus_all/2.\n-spec prop_minus_all(List::[T], Excluded::[T]) -> true\n    when is_subtype(T, any()).\nprop_minus_all(List, Excluded) ->\n    Result = util:minus_all(List, Excluded),\n    _ = [begin\n             case lists:member(L, Excluded) of\n                 true  -> ?equals_w_note([R || R <- Result, R =:= L], [], io_lib:format(\"~.0p should have been deleted\", [L]));\n                 false -> ?equals_w_note([R || R <- Result, R =:= L], [R || R <- List, R =:= L], io_lib:format(\"Number of ~.0p should remain the same\", [L]))\n             end\n         end || L <- List],\n    true.\n\n%% @doc Checks that the order of items stays the same using util:minus_all/2.\n-spec prop_minus_all_sort(List::[T], Excluded::[T]) -> true\n    when is_subtype(T, any()).\nprop_minus_all_sort(List, Excluded) ->\n    Result = util:minus_all(List, Excluded),\n    prop_minus_all_sort_helper(Result, List, Excluded).\n\n-spec prop_minus_all_sort_helper(Result::[T], List::[T], Excluded::[T]) -> true\n    when is_subtype(T, any()).\nprop_minus_all_sort_helper([], [], _) ->\n    true;\nprop_minus_all_sort_helper([_|_], [], _) ->\n    false;\nprop_minus_all_sort_helper([], [_|_], _) ->\n    true;\nprop_minus_all_sort_helper([RH | RT] = R, [LH | LT], Excluded) ->\n    case lists:member(LH, Excluded) of\n        true                 -> prop_minus_all_sort_helper(R, LT, Excluded);\n        false when LH =:= RH -> prop_minus_all_sort_helper(RT, LT, Excluded);\n        false                -> false\n    end.\n\ntester_minus_all(_Config) ->\n    tester:test(?MODULE, prop_minus_all, 2, 5000, [{threads, 2}]).\n\ntester_minus_all_sort(_Config) ->\n    tester:test(?MODULE, prop_minus_all_sort, 2, 5000, [{threads, 2}]).\n\n%% @doc Checks that all intended items are deleted once using util:minus_first/2.\n%%      Note: this is kindof redundant to prop_minus_first_sort/2 but a cleaner\n%%      approach avoiding a re-implementation of util:minus_first/2.\n-spec prop_minus_first(List::[T], Excluded::[T]) -> true\n    when is_subtype(T, any()).\nprop_minus_first(List, Excluded) ->\n    ?equals(util:minus_first(List, Excluded), lists:foldl(fun lists:delete/2, List, Excluded)).\n\n%% @doc Checks that the order of items stays the same using util:minus_first/2.\n-spec prop_minus_first_sort(List::[T], Excluded::[T]) -> true\n    when is_subtype(T, any()).\nprop_minus_first_sort(List, Excluded) ->\n    Result = util:minus_first(List, Excluded),\n    prop_minus_first_sort_helper(Result, List, Excluded).\n\n-spec prop_minus_first_sort_helper(Result::[T], List::[T], Excluded::[T]) -> true\n    when is_subtype(T, any()).\nprop_minus_first_sort_helper([], [], _) ->\n    true;\nprop_minus_first_sort_helper([_|_], [], _) ->\n    false;\nprop_minus_first_sort_helper([], [_|_], _) ->\n    true;\nprop_minus_first_sort_helper([RH | RT] = R, [LH | LT], Excluded) ->\n    case lists:member(LH, Excluded) of\n        true                 -> prop_minus_first_sort_helper(R, LT, lists:delete(LH, Excluded));\n        false when LH =:= RH -> prop_minus_first_sort_helper(RT, LT, Excluded);\n        false                -> false\n    end.\n\ntester_minus_first(_Config) ->\n    tester:test(?MODULE, prop_minus_first, 2, 5000, [{threads, 2}]).\n\ntester_minus_first_sort(_Config) ->\n    tester:test(?MODULE, prop_minus_first_sort, 2, 5000, [{threads, 2}]).\n\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n%\n% util:par_map/2\n%\n%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\n-spec try_fun(Module::module(), Fun::atom(), Args::[term()]) -> {ok | throw | error | exit, term()}.\ntry_fun(Module, Fun, Args) ->\n    try {ok, apply(Module, Fun, Args)}\n    catch Level:Reason -> {Level, Reason}\n    end.\n\n-spec compare__par_map2__lists_map(Fun::fun((A) -> term()), [A]) -> true | no_return().\ncompare__par_map2__lists_map(Fun, List) ->\n    ParMapRes = try_fun(util, par_map, [Fun, List]),\n    ListsMapRes = try_fun(lists, map, [Fun, List]),\n    ?equals(ParMapRes, ListsMapRes).\n\n-spec prop_par_map2([integer()]) -> ok.\nprop_par_map2(List) ->\n    compare__par_map2__lists_map(fun(X) -> X * X end, List),\n    compare__par_map2__lists_map(fun(_) -> erlang:throw(failed) end, List),\n    compare__par_map2__lists_map(fun(_) -> erlang:error(failed) end, List),\n    compare__par_map2__lists_map(fun(_) -> erlang:exit(failed) end, List),\n    ok.\n\ntester_par_map2(_Config) ->\n    tester:test(?MODULE, prop_par_map2, 1, 5000, [{threads, 2}]).\n\n-spec compare__par_map3__lists_map(Fun::fun((A) -> term()), [A], MaxThreads::pos_integer()) -> true | no_return().\ncompare__par_map3__lists_map(Fun, List, MaxThreads) ->\n    ParMapRes = try_fun(util, par_map, [Fun, List, MaxThreads]),\n    ListsMapRes = try_fun(lists, map, [Fun, List]),\n    ?equals(ParMapRes, ListsMapRes).\n\n-spec prop_par_map3([integer()], 1..1000) -> ok.\nprop_par_map3(List, MaxThreads) ->\n    compare__par_map3__lists_map(fun(X) -> X * X end, List, MaxThreads),\n    compare__par_map3__lists_map(fun(_) -> erlang:throw(failed) end, List, MaxThreads),\n    compare__par_map3__lists_map(fun(_) -> erlang:error(failed) end, List, MaxThreads),\n    compare__par_map3__lists_map(fun(_) -> erlang:exit(failed) end, List, MaxThreads),\n    ok.\n\ntester_par_map3(_Config) ->\n    tester:test(?MODULE, prop_par_map3, 2, 5000, [{threads, 2}]).\n\nlists_remove_at_indices(_Config) ->\n    ?equals(util:lists_remove_at_indices([0,1,2,3,4,5], [0,2,4]), [1,3,5]),\n\n    % both lists should be non-empty and all indices in the second list should point to\n    % existing elements in the first list\n    ?expect_exception(util:lists_remove_at_indices([1], []), error, function_clause),\n    ?expect_exception(util:lists_remove_at_indices([], [0]), error, function_clause),\n    ?expect_exception(util:lists_remove_at_indices([0,1,2,3], [5]), error, function_clause),\n    ok.\n\nsublist(_Config) ->\n    L = [a,b,c,d,e,f,g],\n    LLen = length(L),\n    ?equals(util:sublist(L, 1, 0)    , {[], LLen}),\n    ?equals(util:sublist(L, 1, 1)    , {[a], LLen}),\n    ?equals(util:sublist(L, 1, 3)    , {[a,b,c], LLen}),\n    ?equals(util:sublist(L, 1, 7)    , {[a,b,c,d,e,f,g], LLen}),\n    ?equals(util:sublist(L, 1, 8)    , {[a,b,c,d,e,f,g], LLen}),\n    ?equals(util:sublist(L, 1, 10)   , {[a,b,c,d,e,f,g], LLen}),\n    ?equals(util:sublist(L, 2, 10)   , {[b,c,d,e,f,g], LLen}),\n    ?equals(util:sublist(L, 3, 10)   , {[c,d,e,f,g], LLen}),\n    ?equals(util:sublist(L, 7, 10)   , {[g], LLen}),\n    ?equals(util:sublist(L, 8, 10)   , {[], LLen}),\n    ?equals(util:sublist(L, 10, 10)  , {[], LLen}),\n    \n    ?equals(util:sublist(L, 1, -1)   , {[a], LLen}),\n    ?equals(util:sublist(L, 1, -2)   , {[a], LLen}),\n    ?equals(util:sublist(L, 1, -3)   , {[a], LLen}),\n    ?equals(util:sublist(L, 2, -1)   , {[b], LLen}),\n    ?equals(util:sublist(L, 3, -1)   , {[c], LLen}),\n    ?equals(util:sublist(L, 4, -1)   , {[d], LLen}),\n    ?equals(util:sublist(L, 5, -1)   , {[e], LLen}),\n    ?equals(util:sublist(L, 6, -1)   , {[f], LLen}),\n    ?equals(util:sublist(L, 7, -1)   , {[g], LLen}),\n    ?equals(util:sublist(L, 8, -1)   , {[g], LLen}),\n    ?equals(util:sublist(L, 3, -5)   , {[c,b,a], LLen}),\n    \n    ?equals(util:sublist(L, -1, 0)   , {[], LLen}),\n    ?equals(util:sublist(L, -1, 1)   , {[g], LLen}),\n    ?equals(util:sublist(L, -1, 3)   , {[g], LLen}),\n    ?equals(util:sublist(L, -1, 10)  , {[g], LLen}),\n    ?equals(util:sublist(L, -2, 10)  , {[f,g], LLen}),\n    ?equals(util:sublist(L, -3, 10)  , {[e,f,g], LLen}),\n    ?equals(util:sublist(L, -7, 10)  , {[a,b,c,d,e,f,g], LLen}),\n    ?equals(util:sublist(L, -8, 10)  , {[a,b,c,d,e,f,g], LLen}),\n    ?equals(util:sublist(L, -10, 10) , {[a,b,c,d,e,f,g], LLen}),\n    \n    ?equals(util:sublist(L, -1, -1)  , {[g], LLen}),\n    ?equals(util:sublist(L, -1, -3)  , {[g,f,e], LLen}),\n    ?equals(util:sublist(L, -1, -7)  , {[g,f,e,d,c,b,a], LLen}),\n    ?equals(util:sublist(L, -1, -8)  , {[g,f,e,d,c,b,a], LLen}),\n    ?equals(util:sublist(L, -1, -10) , {[g,f,e,d,c,b,a], LLen}),\n    ?equals(util:sublist(L, -2, -10) , {[f,e,d,c,b,a], LLen}),\n    ?equals(util:sublist(L, -3, -10) , {[e,d,c,b,a], LLen}),\n    ?equals(util:sublist(L, -7, -10) , {[a], LLen}),\n    ?equals(util:sublist(L, -8, -10) , {[], LLen}),\n    ?equals(util:sublist(L, -10, -10), {[], LLen}),\n    \n    ok.\n\n-spec prop_sublist3([any()], X::1..1000) -> true.\nprop_sublist3(L, X) ->\n    % last X elements (in different order)\n    ?equals(lists:reverse(element(1, util:sublist(L, -1, -X))),\n            element(1, util:sublist(L, -X, X))),\n    % first X elements (in different order)\n    ?equals(lists:reverse(element(1, util:sublist(L, 1, X))),\n            element(1, util:sublist(L, X, -X))),\n    true.\n\ntester_sublist3(_Config) ->\n    tester:test(?MODULE, prop_sublist3, 2, 5000, [{threads, 2}]).\n\n-spec prop_timestamp1(erlang_timestamp()) -> true.\nprop_timestamp1(TS) ->\n    ?equals(TS, util:us2timestamp(util:timestamp2us(TS))),\n    true.\n\n-spec prop_timestamp2(util:us_timestamp()) -> true.\nprop_timestamp2(Us) ->\n    ?equals(Us, util:timestamp2us(util:us2timestamp(Us))),\n    true.\n\ntester_timestamp(_Config) ->\n    tester:test(?MODULE, prop_timestamp1, 1, 5000, [{threads, 2}]),\n    tester:test(?MODULE, prop_timestamp2, 1, 5000, [{threads, 2}]).\n\nrrd_combine_timing_slots_handle_empty_rrd(_Config) ->\n    DB0 = rrd:create(10, 10, {timing, us}, {0,0,0}),\n    Dump = rrd:dump(DB0),\n    ?equals(Dump, []),\n    ?equals(util:rrd_combine_timing_slots(DB0, {0,0,0}, 10), undefined),\n    ok\n    .\n\nrrd_combine_timing_slots_simple(_Config) ->\n    Adds = [{20, 1}, {25, 3}, {30, 30}, {42, 42}],\n    DB0 = rrd:create(10, 10, {timing, us}, {0,0,0}),\n    DB1 = lists:foldl(fun rrd_SUITE:apply/2, DB0, Adds),\n    ?equals(rrd:dump(DB1),\n            [{{0,0,40}, {0,0,50}, {42, 42*42, 1, 42, 42, {histogram,0,[],0,0}}},\n             {{0,0,30}, {0,0,40}, {30, 30*30, 1, 30, 30, {histogram,0,[],0,0}}},\n             {{0,0,20}, {0,0,30}, {1 + 3, 1*1 + 3*3, 2, 1, 3, {histogram,0,[],0,0}}}]),\n    CurrentTS = {0,0,44}, % assume we are currently in the last slot\n\n    Expected = {1 + 3 + 30 + 42, % sum\n                1*1 + 3*3 + 30*30 + 42*42, % squares' sum\n                2 + 1 + 1, % count\n                1, % min\n                42 % max\n               },\n    ?equals(util:rrd_combine_timing_slots(DB1, CurrentTS, 100), Expected),\n    ?equals(util:rrd_combine_timing_slots(DB1, CurrentTS, 100, 10), Expected),\n    ok\n    .\n\nrrd_combine_timing_slots_subset(_Config) ->\n    % combine the newest two slots due to the interval\n    Adds = [{20, 1}, {25, 3}, {30, 30}, {42, 42}],\n    DB0 = rrd:create(10, 10, {timing, us}, {0,0,0}),\n    DB1 = lists:foldl(fun rrd_SUITE:apply/2, DB0, Adds),\n    ?equals(rrd:dump(DB1),\n            [{{0,0,40}, {0,0,50}, {42, 42*42, 1, 42, 42, {histogram,0,[],0,0}}},\n             {{0,0,30}, {0,0,40}, {30, 30*30, 1, 30, 30, {histogram,0,[],0,0}}},\n             {{0,0,20}, {0,0,30}, {1 + 3, 1*1 + 3*3, 2, 1, 3, {histogram,0,[],0,0}}}]),\n\n    CurrentTS = {0,0,44}, % assume we are currently in the last slot\n    Interval = 10, % overlap at most two slots\n\n    ExpectedSmallEpsilon = {42 + 30, 42*42 + 30*30, 1 + 1, 30, 42},\n    ?equals(util:rrd_combine_timing_slots(DB1, CurrentTS, Interval), ExpectedSmallEpsilon),\n    ?equals(util:rrd_combine_timing_slots(DB1, CurrentTS, Interval, 5), ExpectedSmallEpsilon),\n\n    % epsilon is big enough to do it only once\n    ExpectedBigEpsilon = {42, 42*42, 1, 42, 42},\n    ?equals(util:rrd_combine_timing_slots(DB1, CurrentTS, Interval, 10), ExpectedBigEpsilon),\n    ?equals(util:rrd_combine_timing_slots(DB1, CurrentTS, Interval, 100), ExpectedBigEpsilon),\n    ok\n    .\n\nrrd_combine_gauge_slots_handle_empty_rrd(_Config) ->\n    DB0 = rrd:create(10, 10, gauge, {0,0,0}),\n    Dump = rrd:dump(DB0),\n    ?equals(Dump, []),\n    ?equals(util:rrd_combine_gauge_slots(DB0, {0,0,0}, 10), undefined),\n    ok\n    .\n\nrrd_combine_gauge_slots_simple(_Config) ->\n    Adds = [{20, 1}, {25, 3}, {30, 30}, {42, 42}],\n    DB0 = rrd:create(10, 10, gauge, {0,0,0}),\n    DB1 = lists:foldl(fun rrd_SUITE:apply/2, DB0, Adds),\n    ?equals(rrd:dump(DB1),\n            [{{0,0,40}, {0,0,50}, 42},\n             {{0,0,30}, {0,0,40}, 30},\n             {{0,0,20}, {0,0,30}, 3}]),\n    CurrentTS = {0,0,44}, % assume we are currently in the last slot\n\n    Expected = 75,\n    ?equals(util:rrd_combine_gauge_slots(DB1, CurrentTS, 100), Expected),\n    ?equals(util:rrd_combine_gauge_slots(DB1, CurrentTS, 100, 10), Expected),\n    ok\n    .\n\nrrd_combine_gauge_slots_subset(_Config) ->\n    % combine the newest two slots due to the interval\n    Adds = [{20, 1}, {25, 3}, {30, 30}, {42, 42}],\n    DB0 = rrd:create(10, 10, gauge, {0,0,0}),\n    DB1 = lists:foldl(fun rrd_SUITE:apply/2, DB0, Adds),\n    ?equals(rrd:dump(DB1),\n            [{{0,0,40}, {0,0,50}, 42},\n             {{0,0,30}, {0,0,40}, 30},\n             {{0,0,20}, {0,0,30}, 3}]),\n\n    CurrentTS = {0,0,44}, % assume we are currently in the last slot\n    Interval = 10, % overlap at most two slots\n\n    ?equals(util:rrd_combine_gauge_slots(DB1, CurrentTS, Interval), 72),\n    ?equals(util:rrd_combine_gauge_slots(DB1, CurrentTS, Interval, 5), 72),\n    ?equals(util:rrd_combine_gauge_slots(DB1, CurrentTS, Interval, 10), 42), % exits immediately\n    ?equals(util:rrd_combine_gauge_slots(DB1, CurrentTS, Interval, 100), 42),\n    ?equals(util:rrd_combine_gauge_slots(DB1, CurrentTS, 20, 1), 75),\n    ok\n    .\n"
  },
  {
    "path": "user-dev-guide/.gitignore",
    "content": "/*.aux\n/*.log\n/*.idx\n/*.toc\n/*.out\n/*.ilg\n/*.ind\n.project\n.settings\n.texlipse\ntmp\n/Thumbs.db\n/*.eps\n/*.synctex.gz\n"
  },
  {
    "path": "user-dev-guide/Makefile",
    "content": "LATEX=pdflatex\nFILE=main.tex\nall:\n\t../bin/scalarisctl -h > scalarisctl-h.out\n\t../java-api/scalaris --noconfig --help > scalaris-client-java.out\n\t../python-api/scalaris --help > scalaris-client-python.out\n\t../ruby-api/scalaris --help > scalaris-client-ruby.out\n\t$(MAKE) -C rrepair\n\t$(LATEX) $(FILE) | awk -f ./latex-col.awk\n\tmakeindex main\n\t$(LATEX) $(FILE) | awk -f ./latex-col.awk\n\t@while ( grep \"Rerun to get cross-references\" \\\n\t\t$(subst .tex,.log, $(FILE)) > /dev/null ); do \\\n\t\techo '*** Going to Rerun LaTeX ***'; \\\n\t\t$(LATEX) $(FILE) | awk -f ./latex-col.awk; \\\n\tdone\n\nclean:\n\trm -f *.aux *.ind *.ilg *.idx *.log *.out *.toc\n"
  },
  {
    "path": "user-dev-guide/dev-dirs.tex",
    "content": "\\chapter{Directory Structure of the Source Code}\n\nThe directory tree of \\scalaris{} is structured as follows:\n\n\\vspace*{1em}\n\\begin{tabular}{|r|p{11.5cm}|}\n \\hline\n \\code{bin} & contains shell scripts needed to work with \\scalaris{} (e.g.\\ start the management server, start a node, \\dots)\\\\\n \\code{contrib} & necessary third party packages (yaws and log4erl) \\\\\n \\code{doc} & generated Erlang documentation \\\\\n \\code{docroot} & root directory of the node's webserver \\\\\n \\code{ebin} & the compiled Erlang code (beam files)\\\\\n \\code{java-api} & a Java API to \\scalaris{} \\\\\n \\code{python-api} & a Python 2 API to \\scalaris{} \\\\\n \\code{python3-api} & a Python 3 API to \\scalaris{} \\\\\n \\code{ruby-api} & a Ruby API to \\scalaris{} \\\\\n \\code{log} & log files \\\\\n \\code{src} & contains the \\scalaris{} source code\\\\\n \\code{include} & contains macros for the source code\\\\\n \\code{test} & unit tests for \\scalaris{} \\\\\n \\code{user-dev-guide} & contains the sources for this document\\\\\n \\hline\n\\end{tabular}\n"
  },
  {
    "path": "user-dev-guide/dev-hints.tex",
    "content": "\\chapter{General Hints}\n\n\\section{Coding Guidelines}\n\n\\begin{itemize}\n\\item Keep the code short\n\\item Use \\erlmodule{gen\\_component} to implement additional processes\n\\item Don't use receive by yourself (Exception: to implement single threaded\n  user API calls (cs\\_api, yaws\\_calls, etc)\n\\item Don't use \\erlfun{erlang}{now}{/0}, \\erlfun{erlang}{send\\_after}{/3},\n  \\code{receive after} etc. in performance critical code, consider\n  using \\erlmodule{msg\\_delay} instead.\n\\item Don't use \\erlfun{timer}{tc}{/3} as it catches exceptions. Use\n  \\erlfun{util}{tc}{/3} instead.\n\\end{itemize}\n\n\\section{Testing Your Modifications and Extensions}\n\n\\begin{itemize}\n\\item Run the testsuites using \\code{make test}\n\\item Run the java api test using \\code{make java-test}\n      (\\scalaris{} output will be printed if a test fails; if you want to see\n      it during the tests, start a \\code{bin/firstnode.sh} and run the tests by\n      \\code{cd java; ant test})\n\\item Run the Ruby client by starting \\scalaris{} and running\n      \\code{cd contrib; ./jsonrpc.rb}\n\\end{itemize}\n\n\\section{Help with Digging into the System}\n\\label{sec:digging}\n\n\\begin{itemize}\n\\item use \\erlfun{ets}{i}{/0,1} to get details on the local state of some\n      processes\n\\item consider changing pdb.erl to use ets instead of erlang:put/get\n\\item Have a look at strace -f -p PID of beam process\n\\item Get message statistics via the Web-interface\n\\item enable/disable tracing for certain modules\n\\item Trace messages using the \\erlmodule{trace\\_mpath} module\n%% \\item enable gen-component profiling and read the collected data using\n%%\n%%    \\code{lists:reverse(lists:keysort(2,ets:tab2list(profiling))).}\n%%\n%%    (Has the limitation that it measures using absolute time, maybe the data\n%%    is more ok, when using -smp disable? ...)\n\\item Use etop and look at the total memory size and atoms generated\n\\item send processes sleep or kill messages to test certain behaviour (see\n  \\erlmodule{gen\\_component}.erl)\n\\item use \\code{admin:number_of_nodes().}\n\\item use \\code{admin:check_ring().}\n\\end{itemize}\n\n%% \\section{General Erlang server loop}\n%%\n%% Servers in Erlang often use the following structure to maintain a state\n%% while processing received messages:\n%%\n%% \\lstset{language=erlang}\n%% \\begin{lstlisting}\n%% loop(State) ->\n%%   receive\n%%     Message ->\n%%       State1 = f(State),\n%%       loop(State1)\n%%   end.\n%% \\end{lstlisting}\n%%\n%% The server runs an endless loop, that waits for a message, processes it and\n%% calls itself using tail-recursion in each branch. The loop works on a\n%% \\code{State}, which can be modified when a message is handled.\n"
  },
  {
    "path": "user-dev-guide/dev-infrastructure.tex",
    "content": "\\chapter{System Infrastructure}\n\n\\section{Groups of Processes}\n\\label{sec:pid_groups}\n\\erlmoduleindex{pid\\_groups}\n\n\\begin{itemize}\n\\item What is it? How to distinguish from Erlangs internal named processes?\n\\item Joining a process group\n\\item Why do we do this... (managing several independent nodes inside a single\n  Erlang VM for testing)\n\\end{itemize}\n\n\\section{\\texorpdfstring{The Communication Layer \\erlmodule{comm}}\n          {The Communication Layer comm}}\n\\label{sec:comm}\n\\erlmoduleindex{comm}\n\n\\begin{itemize}\n\\item in general\n\\item format of messages (tuples)\n\\item use messages with cookies (server and client side)\n\\item What is a message tag?\n\\end{itemize}\n\n\\input{gen_component}\n\n\\section{The Process' Database (\\texttt{pdb})}\n\\erlmoduleindex{pdb}\n\n\\begin{itemize}\n\\item How to use it and how to switch from erlang:put/set to ets and implied\n  limitations.\n\\end{itemize}\n\n\\section{Failure Detectors (\\texttt{fd})}\n\\erlmoduleindex{fd}\n\n\\begin{itemize}\n\\item uses Erlang monitors locally\n\\item is independent of component load\n\\item uses heartbeats between Erlang virtual machines\n\\item uses a single proxy heartbeat server per Erlang virtual machine, which\n  itself uses Erlang monitors to monitor locally\n\\item uses dynamic timeouts to implement an eventually perfect failure detector.\n\\end{itemize}\n\n\\section{Monitoring Statistics (\\texttt{monitor}, \\texttt{rrd})}\n\\erlmoduleindex{monitor}\n\nThe \\erlmodule{monitor} module offers several methods to gather meaningful\nstatistics using the \\code{rrd()} data type defined in \\erlmodule{rrd}.\n\n\\erlmoduleindex{rrd}\n\\code{rrd()} records work with time slots, i.e. a fixed slot length is given\nat creation and items which should be inserted will be either put into the\ncurrent slot, or a new slot will be created.\nEach data item thus needs a time stamp associated with it. It must not be a\nreal time, but can also be a virtual time stamp.\n\nThe \\erlmodule{rrd} module thus offers two different APIs:\none with transparent time handling, e.g.\n \\erlfun{rrd}{create}{/3}, \\erlfun{rrd}{add\\_now}{/2}, and\none with manual time handling, e.g.\n \\erlfun{rrd}{create}{/4}, \\erlfun{rrd}{add}{/3}.\n\nTo allow different evaluations of the stored data, the following types of data\nare supported:\n\\begin{itemize}\n  \\item \\code{gauge}: only stores the newest value of a time slot, e.g. for thermometers,\n  \\item \\code{counter}: sums up all values inside a time slot,\n  \\item \\code{timing}: records time spans and stores values to easily calculate e.g. the sum, the\n  standard deviation, the number of events, the min and max,\n  \\item \\code{timing_with_hist}: similar to \\code{timing} but also records a more detailed\n  (approximated) histogram of the data,\n  \\item \\code{event}: records each event (including its time stamp) inside a time slot in a list\n  (this should be rarely used as the amount of data stored may be very big).\n  \\item \\code{histogram, N}: records values in an approximative histogram of size N\n  \\item \\code{histogram_rt, N, BaseKey}: histogram of size N which\n    operates on the key space of the DHT. BaseKey is the key with the\n    largest distance to all keys in the histogram.\n\\end{itemize}\n\nThe \\erlmodule{monitor} offers functions to conveniently store and retrieve\nsuch values. It is also started as a process in each \\code{dht_node} and\n\\code{basic_services} group as well as inside each \\code{clients_group}.\nThis process ultimately stores the whole rrd() structure.\nThere are three paradigms how values can be stored:\n\\begin{enumerate}\n  \\item Values are gathered in the process that is generating the values.\n        Inside this process, the \\code{rrd()} is stored in the erlang\n        dictionary.\n        Whenever a new time slot is started, the values will be\n        reported to the monitor process of the gathering process' group.\n  \\item Values are gathered in the process that is generating the values.\n        Inside this process, the \\code{rrd()} is handled manually.\n        After changing the \\code{rrd()}, a manual check for reporting needs to\n        be issued using \\erlfun{monitor}{check\\_report}{/4}.\n  \\item Values are immediately send to the monitor process where it undergoes\n        the same procedures until it is finally stored and available to other\n        processes. This is especially useful if the process generating the\n        values does not live long or does not regularly create new data, e.g.\n        the client.\n\\end{enumerate}\n\nThe following example illustrates the first mode, i.e. gathering data in the\ngenerating process. It has been taken from the \\erlmodule{cyclon} module which\nuses a \\code{counter} data type:\n\n\\begin{lstlisting}\n% initialise the monitor with an empty rrd() using a 60s monitoring interval\nmonitor:proc_set_value(?MODULE, 'shuffle', rrd:create(60 * 1000000, 3, counter)),\n% update the value by adding one\nmonitor:proc_set_value(?MODULE, 'shuffle', fun(Old) -> rrd:add_now(1, Old) end),\n% check regularly whether to report the data to the monitor:\nmonitor:proc_check_timeslot(?MODULE, 'shuffle')\n\\end{lstlisting}\n\nThe first two parameters of \\erlfun{monitor}{proc\\_set\\_value}{/3} define the\nname of a monitored value, the module's name and a unique key. The second can\nbe either an \\code{rrd()} or an update fun.\nThe \\erlfun{monitor}{proc\\_check\\_timeslot}{/3} function can be used if your\nmodule does not regularly create new data. In this case, the monitor process\nwould not have the latest data for others to retrieve. This function forces a\ncheck and creates the new time slot if needed (thus reporting the data).\n\nThis is how forwarding works (taken from \\erlmodule{api\\_tx}):\n\n\\begin{lstlisting}\nmonitor:client_monitor_set_value(\n  ?MODULE, 'req_list',\n  fun(Old) ->\n    Old2 = case Old of\n             % 10s monitoring interval, only keep newest in the client process\n             undefined -> rrd:create(10 * 1000000, 1, {timing, ms});\n             _ -> Old\n           end,\n    rrd:add_now(TimeInUs / 1000, Old2)\n  end),\n\\end{lstlisting}\n\nAs in this case there is no safe way of initialising the value, it is more\nuseful to provide an update fun to\n\\erlfun{monitor}{client\\_monitor\\_set\\_value}{/3}. This function is only useful\nfor the client processes as it reports to the monitor in the\n\\code{clients_group} (recall that client processes do not belong to any group).\nAll other processes should use \\erlfun{monitor}{monitor\\_set\\_value}{/3} with\nthe same semantics.\n\n\\section{Writing Unit Tests}\n\n\\subsection{Plain Unit Tests}\n\n\\subsection{Randomized Testing Using \\texttt{tester}}\n\n\\subsection{Randomized Testing Using \\texttt{proto\\_sched}}\n"
  },
  {
    "path": "user-dev-guide/dev-java.tex",
    "content": "\\chapter{Java API}\n\nFor the Java API documentation, we refer the reader to the documentation\ngenerated by javadoc or doxygen. The following commands create the\ndocumentation:\n\n\\begin{lstlisting}[language=sh]\n%> cd java-api\n%> ant doc\n%> doxygen\n\\end{lstlisting}\n\nThe documentation can then be found in \\code{java-api/doc/index.html}\n(javadoc) and\\\\ \\code{java-api/doc-doxygen/html/index.html} (doxygen).\n\nThe API is divided into four classes:\n\n\\begin{itemize}\n\\item \\code{de.zib.scalaris.Transaction} for (multiple) operations inside a\n       transaction\n\\item \\code{de.zib.scalaris.TransactionSingleOp} for single transactional\n       operations\n\\item \\code{de.zib.scalaris.ReplicatedDHT} for non-transactional (inconsistent)\n       access to the replicated DHT items, e.g. deleting items\n\\end{itemize}\n"
  },
  {
    "path": "user-dev-guide/dev-join.tex",
    "content": "\\chapter{How a node joins the system}\n\\label{chapter.join}\n\nAfter starting a new \\scalaris{}-System as described in\nSection~\\sieheref{sec.boot}, ten additional local nodes can be started\nby typing \\code{api_vm:add_nodes(10)} in the Erlang-Shell that is\nopened during startup~\\footnote{Increase the log level to {\\tt info} to get\nmore detailed startup logs. See Section~\\sieheref{sec:logging}}.\n\n\n\\lstset{language=erlang}\n\\begin{lstlisting}\nscalaris> ./bin/firstnode.sh \n[...]\n(firstnode@csr-pc9)1> api_vm:add_nodes(10)\n\\end{lstlisting}\n\nIn the following we will trace what this function does in order to add\nadditional nodes to the system.\nThe function \\erlfun{api\\_vm}{add\\_nodes}{pos\\_integer()} is defined as follows.\n\n\\codesnippet{api_vm.erl}{api_vm:add_nodes}{../src/api_vm.erl}\n\nIt uses the \\erlfun{admin}{add\\_nodes}{/1} function to actually add the given\nnumber of nodes and then waits for all nodes to successfully complete their\njoin phases.\n\n\\codesnippet{admin.erl}{admin:add_nodes}{../src/admin.erl}\n\nThe function \\erlfun{admin}{add\\_nodes}{/1} calls\n\\erlfun{admin}{add\\_node}{[]} Count times. This function starts a new\nchild with the given options for the main supervisor \\code{main_sup}.\nIn particular, it sets a random ID that is passed to the new node as its\nsuggested ID to join at.\nTo actually perform the start, the\nfunction \\erlfun{sup\\_dht\\_node}{start\\_link}{/1} is called by the Erlang\nsupervisor mechanism. For more details on the OTP supervisor\nmechanism see Chapter~18 of the Erlang book~\\cite{erlang-book} or the\nonline documentation at\n\\url{http://www.erlang.org/doc/man/supervisor.html}.\n\n\\section{Supervisor-tree of a \\scalaris{} node}\n\nWhen a new Erlang VM with a \\scalaris{} node is started, a\n\\erlmodule{sup\\_scalaris} supervisor is started that creates further\nworkers and supervisors according to the following scheme\n(processes starting order: left to right, top to bottom):\n\n\\begin{center}\n\\includegraphics[width=\\linewidth]{supervision}\n\\end{center}\n\nWhen new nodes are started using \\erlfun{admin}{add\\_node}{/1}, only new\n\\code{sup_dht_node} supervisors are started.\n\n\\section{Starting the sup\\_dht\\_node supervisor and general processes of a node}\n\nStarting supervisors is a two step process: a call to\n\\erlfun{supervisor}{start\\_link}{/2,3}, e.g. from a custom supervisor's own\n\\code{start_link} method, will start the supervisor process. It will then\ncall \\erlfun[]{Module}{init}{/1} to find out about the restart strategy,\nmaximum restart frequency and child processes.\nNote that \\erlfun{supervisor}{start\\_link}{/2,3} will not return until\n\\erlfun[]{Module}{init}{/1} has returned and all child processes have been\nstarted.\n\nLet's have a look at \\erlfun{sup\\_dht\\_node}{init}{/1}, the 'DHT node\nsupervisor'.\n\n\\codesnippet{sup_dht_node.erl}{sup_dht_node:init}{../src/sup_dht_node.erl}\n\n\nThe return value of the \\code{init/1} function specifies the child\nprocesses of the supervisor and how to start them. Here, we define a\nlist of processes to be observed by a \\code{one_for_one}\nsupervisor. The processes are:\n\\code{Monitor},\n\\code{Delayer},\n\\code{Reregister},\n\\code{DeadNodeCache},\n\\code{RingMaintenance},\n\\code{RoutingTable},\n\\code{Cyclon},\n\\code{Vivaldi},\n\\code{DC_Clustering},\n\\code{Gossip} and a\n\\code{SupDHTNodeCore_AND} process in this order.\n\nThe term \\code{\\{one_for_one, 10, 1\\}} specifies that the supervisor\nshould try 10 times to restart each process before giving\nup. \\code{one_for_one} supervision means, that if a single process\nstops, only that process is restarted. The other processes run\nindependently.\n\nWhen the \\erlfun{sup\\_dht\\_node}{init}{/1} is finished the supervisor module\nstarts all the defined processes by calling the functions that were\ndefined in the returned list.\n\nFor a join of a new node, we are only interested in the starting of\nthe \\code{SupDHTNodeCore_AND} process here. At that point in time, all\nother defined processes are already started and running.\n\n\\section{Starting the sup\\_dht\\_node\\_core supervisor with a peer and some paxos processes}\n\nLike any other supervisor the \\erlmodule{sup\\_dht\\_node\\_core} supervisor\ncalls its \\erlfun[]{sup\\_dht\\_node\\_core}{init}{/1} function:\n\n\\codesnippet{sup_dht_node_core.erl}{sup_dht_node_core:init}{../src/sup_dht_node_core.erl}\n\nIt defines five processes, that have to be observed using a\n\\code{one_for_all}-supervisor, which means, that if one fails, all have to\nbe restarted. The \\erlmodule{dht\\_node} module implements the main\ncomponent of a full \\scalaris{} node which glues together all the other\nprocesses. Its\n\\erlfun[]{dht\\_node}{start\\_link}{/2} function will get\nthe following parameters: (a) the processes' group that is used with the \\erlmodule{pid\\_groups}\nmodule and (b) a list of options for\nthe \\erlmodule{dht\\_node}. The process group name was calculated\na bit earlier in the code. \\emph{Exercise: Try to find where.}\n\n\n\\codesnippet{dht_node.erl}{dht_node:start_link}{../src/dht_node.erl}\n\nLike many other modules, the \\erlmodule{dht\\_node} module implements the\n\\code{gen_component}\nbehaviour. This behaviour was developed by us to enable us to write code which is\nsimilar in syntax and semantics to the examples in~\\cite{rachid-book}.\nSimilar to the \\code{supervisor} behaviour, a module implementing this\nbehaviour has to\nprovide an \\code{init/1} function, but here it is used to initialize the\nstate of the component. This function is described in the next\nsection.\n\nNote: \\code{?MODULE} is a predefined Erlang macro, which expands to the\nmodule name, the code belongs to (here: \\code{dht_node}).\n\n\\section{\\texorpdfstring{Initializing a \\code{dht_node}-process}\n             {Initializing a dht\\_node-process}}\n\n\\codesnippet{dht_node.erl}{dht_node:start}{../src/dht_node.erl}\n\nThe \\code{gen_component} behaviour registers the \\code{dht_node} in the\nprocess dictionary. Formerly, the process had to do this itself, but\nwe moved this code into the behaviour. If an ID was given to\n\\erlfun{dht\\_node}{init}{/1} function as a \\code{\\{\\{dht_node, id\\}, KEY\\}}\ntuple, the given Id will be used. Otherwise a random key is generated.\nDepending on whether the node is the first inside a VM marked as first or not,\nthe according function in \\erlmodule{dht\\_node\\_join} is called.\nAlso the pid of the node's supervisor is kept for future reference.\n\n\\section{Actually joining the ring}\n\nAfter retrieving its identifier, the node starts the join protocol\nwhich processes the appropriate messages calling\n\\erlfun{dht\\_node\\_join}{process\\_join\\_state}{Message, State}. On the\nexisting node, join messages will be processed by\n\\erlfun{dht\\_node\\_join}{process\\_join\\_msg}{Message, State}.\n\n\\subsection{A single node joining an empty ring}\n\n\\codesnippet{dht_node_join.erl}{dht_node_join:join_as_first}{../src/dht_node_join.erl}\n\nIf the ring is empty, the joining node will be the only node in the ring\nand will thus be responsible for the whole key space.\nIt will trigger all known nodes to initialize the comm layer and then finish\nthe join.\n\\erlfun{dht\\_node\\_join}{finish\\_join}{/5}\njust creates a new state for a \\scalaris{} node consisting of the given\nparameters (the node as itself, its predecessor and successor,\nan empty database and the queued messages that arrived during the join).\nIt then activates all dependent processes and creates a routing table from\nthis information.\n\nThe \\erlfun{dht\\_node\\_state}{state}{} type is defined in\n\n\\codesnippet{dht_node_state.erl}{dht_node_state:state}{../src/dht_node_state.erl}\n\n\\subsection{A single node joining an existing (non-empty) ring}\n\nIf a node joins an existing ring, its join protocol will step through\nthe following four phases:\n\\begin{itemize}\n  \\item \\makebox[4em][l]{\\textbf{phase2}}\n    finding nodes to contact with the help of the configured \\code{known_hosts}\n  \\item \\makebox[4em][l]{\\textbf{phase2b}}\n    getting the number of Ids to sample (may be skipped)\n  \\item \\makebox[4em][l]{\\textbf{phase3}}\n    lookup nodes responsible for all sampled Ids\n  \\item \\makebox[4em][l]{\\textbf{phase4}}\n    joining a selected node and setting up item movements\n\\end{itemize}\n\nThe following figure shows a (non-exhaustive) overview of the transitions\nbetween the phases in the normal case. We will go through these step by step\nand discuss what happens if errors occur.\n\n\\medskip\n{\\centering\\input{dht_node_join_phases2.tikz}}\n\nAt first all nodes set in the \\code{known_hosts} configuration parameter are\ncontacted. Their responses are then handled in phase~2. In order to separate\nthe join state from the ordinary \\code{dht_node} state, the\n\\code{gen_component} is instructed to use the \\erlfun{dht\\_node}{on\\_join}{/2}\nmessage handler which delegates every message to\n\\erlfun{dht\\_node\\_join}{process\\_join\\_state}{/2}.\n\n\\codesnippet{dht_node_join.erl}{dht_node_join:join_as_other}{../src/dht_node_join.erl}\n\n\\subsubsection{Phase 2 and 2b}\n\nPhase~2 collects all \\erlmodule{dht\\_node} processes inside the contacted\nVMs. It therefore mainly processes \\code{get_dht_nodes_response} messages and\nintegrates all received nodes into the list of available connections.\nThe next step depends on whether the \\code{\\{skip_psv_lb\\}} option for\nskipping any passive load balancing algorithm has been\ngiven to the \\erlmodule{dht\\_node} or not. If it is present, the node will\nonly use the ID that has been initially passed to\n\\erlfun{dht\\_node\\_join}{join\\_as\\_other}{/3}, issue a lookup for the\nresponsible\nnode and move to phase~3. Otherwise, the passive load balancing's\n\\erlfun{lb\\_psv\\_*}{get\\_number\\_of\\_samples}{/1} method will be called\nasking for the number of IDs to sample. Its answer will be processed in\nphase~2b.\n\n\\code{get_dht_nodes_response} messages arriving in phase~2b or later will be\nprocessed anyway and received \\erlmodule{dht\\_node}\nprocesses will be integrated into the connections. These phases'\noperations will not be interrupted and nothing else is changed though.\n\n\\codesnippet{dht_node_join.erl}{dht_node_join:join_other_p2}{../src/dht_node_join.erl}\n\nPhase~2b will handle \\code{get_number_of_samples} messages from the passive\nload balance algorithm. Once received, new (unique) IDs will be sampled\nrandomly so that the total number of join candidates (selected IDs together\nwith fully processed candidates from further phases) is at least as high as\nthe given number of samples. Afterwards, lookups will be created for all\nprevious IDs as well as the new ones and the node will move to phase~3.\n\n\\codesnippet{dht_node_join.erl}{dht_node_join:join_other_p2b}{../src/dht_node_join.erl}\n\nLookups will make \\scalaris{} find the node currently responsible for a given ID\nand send a request to simulate a join to this node, i.e. a\n\\code{get_candidate} message. Note that during such an operation, the joining\nnode would become the existing node's predecessor. The simulation will be\ndelegated to the passive load balance algorithm the joining node requested, as\nset by the \\code{join_lb_psv} configuration parameter. \n\n\\codesnippet{dht_node_join.erl}{dht_node_join:get_candidate}{../src/dht_node_join.erl}\n\n\\subsubsection{Phase 3}\n\nThe result of the simulation will be send in a \\code{get_candidate_response}\nmessage and will be processed in phase~3 of the joining node. It will be\nintegrated into the list of processed candidates. If there are no more IDs\nleft to process, the best among them will be contacted. Otherwise further\n\\code{get_candidate_response} messages will be awaited.\nSuch messages will also be processed in the other phases where the candidate\nwill be simply added to the list.\n\n\\codesnippet{dht_node_join.erl}{dht_node_join:join_other_p3}{../src/dht_node_join.erl}\n\nIf \\erlfun{dht\\_node\\_join}{contact\\_best\\_candidate}{/1} is called and\ncandidates are available (there should be at this stage!), it will sort the\ncandidates by using the passive load balance algorithm, send a \n\\code{join_request} message and continue with phase~4. \n\n\\erlfunindex{dht\\_node\\_join}{contact\\_best\\_candidate}\n\\codesnippet{dht_node_join.erl}{dht_node_join:contact_best_candidate}{../src/dht_node_join.erl}\n\\erlfunindex{dht\\_node\\_join}{send\\_join\\_request}\n\\codesnippet{dht_node_join.erl}{dht_node_join:send_join_request}{../src/dht_node_join.erl}\n\nThe \\code{join_request} message will be received by the existing node which\nwill set up a slide operation with the new node. If it is not responsible for\nthe key (anymore), it will deny the request and reply with a\n\\code{\\{join, join_response, not_responsible, Node\\}} message.\nIf it is responsible for the ID and is not participating in a slide with its\ncurrent predecessor, it will set up a slide with the joining node:\n\n\\codesnippet{dht_node_join.erl}{dht_node_join:join_request1}{../src/dht_node_join.erl}\n\n\\subsubsection{Phase 4}\n\nThe joining node will receive the \\code{join_response} message in phase~4 of\nthe join protocol. If everything is ok, it will\nnotify its ring maintenance process that it enters the ring, start all required\nprocesses and join the slide operation set up by the existing node in order to\nreceive some of its data.\n\nIf the join candidate's node is not responsible for the candidate's ID anymore\nor the candidate's ID already exists, the next candidate is contacted until\nno further candidates are available and the join protocol starts over using\n\\erlfun{dht\\_node\\_join}{start\\_over}{/1}.\n\nNote that the \\code{join_response} message will actually be processed in\nany phase. Therefore, if messages arrive late, the join can be processed\nimmediately and the rest of the join protocol does not need to be executed\nagain.\n\n\\codesnippet{dht_node_join.erl}{dht_node_join:join_other_p4}{../src/dht_node_join.erl}\n\\erlfunindex{dht\\_node\\_join}{finish\\_join}\n\\erlfunindex{dht\\_node\\_join}{finish\\_join\\_and\\_slide}\n\\codesnippet{dht_node_join.erl}{dht_node_join:finish_join}{../src/dht_node_join.erl}\n\nThe macro \\code{?RT} maps to the configured routing algorithm. It is defined\nin \\code{include/scalaris.hrl}. For further details on the routing see\nChapter~\\sieheref{chapter.routing}.\n\n\\subsubsection{Timeouts and other errors}\n\nThe following table summarizes the timeout messages send during the join\nprotocol on the joining node. It shows in which of the phases each of the\nmessages is processed and describes (in short) what actions are taken.\nAll of these messages are influenced by their respective config parameters,\ne.g. \\code{join_timeout} parameter in the config files defines an overall\ntimeout for the whole join operation. If it takes longer than\n\\code{join_timeout} ms, a \\code{\\{join, timeout\\}} will be send and processed\nas given in this table.\n\n\\medskip\n{\\small\n\\begin{tabular}{lP{2.4cm}P{2.75cm}P{2.35cm}P{3.25cm}M{1.70cm}}\n  \\toprule\n  & \\code{known_hosts}\\carriagereturn\\newline\\code{_timeout}\n  & \\code{get_number_of}\\carriagereturn\\newline\\code{_samples}\\carriagereturn\\newline\\code{_timeout}\n  & \\code{lookup}\\carriagereturn\\newline\\code{ _timeout}\n  & \\code{join_request}\\carriagereturn\\newline\\code{_timeout}\n  & \\code{timeout} \\tn\n  \\midrule\n  %\n  \\bfseries phase2\n  & get known nodes from configured VMs\n  & ignore\n  & ignore\n  & ignore\n  & \\multirow{21}{1.70cm}\n        {re-start join\\newline$\\rightarrow$ phase~2 or 2b} \\tn\n  \\cmidrule(r){1-5}\n  %\n  \\bfseries phase2b\n  & ignore\n  & remove contact node, re-start join\\newline\n    $\\rightarrow$ phase~2 or 2b\n  & ignore\n  & ignore\n  & \\tn\n  \\cmidrule(r){1-5}\n  %\n  \\bfseries phase3\n  & ignore\n  & ignore\n  & remove contact node, lookup remaining IDs\\newline\n    $\\rightarrow$ phase~2 or 3 \n  & ignore\n  & \\tn\n  \\cmidrule(r){1-5}\n  %\n  \\bfseries phase3b\n  & ignore\n  & ignore\n  & ignore\n  & ignore\n  & \\tn\n  \\cmidrule(r){1-5}\n  %\n  \\bfseries phase4\n  & ignore\n  & ignore\n  & ignore\n  & timeouts $< 3$?\\footnote{set by the \\code{join_request_timeouts} config parameter}\\newline\n    \\mbox{}~$\\rightarrow$ contact candidate\\newline\n    otherwise:\\newline\n    \\mbox{}~remove candidate\\newline\n    \\mbox{}~no candidates left?\\newline\n    \\mbox{}~~$\\rightarrow$ phase~2 or 2b\\newline\n    \\mbox{}~otherwise:\\newline\n    \\mbox{}~~$\\rightarrow$ contact next one\\newline\n    \\mbox{}~~$\\rightarrow$ phase~3b or 4\n  & \\tn\n  \\bottomrule\n\\end{tabular}\n}\n\\medskip\n\nOn the existing node, there is only one timeout message which is part of the\njoin protocol: the \\code{join_response_timeout}. It will be send when a slide\noperation is set up and if the timeout hits before the next message exchange,\nit will increase the slide operation's number of timeouts. The slide will be\naborted if at least \\code{join_response_timeouts} timeouts have been received.\nThis parameter is set in the config file.\n\n\\subsubsection{Misc. (all phases)}\n\nNote that join-related messages arriving in other phases than those handling\nthem will be ignored. Any other messages during a \\code{dht_node}'s join will\nbe queued and re-send when the join is complete.\n"
  },
  {
    "path": "user-dev-guide/dev-overlay.tex",
    "content": "\\chapter{Basic Structured Overlay}\n\n\\section{Ring Maintenance}\n\n\\section{T-Man}\n\n\\section{Routing Tables}\n\\label{chapter.routing}\n\\erlmoduleindex{rt\\_beh}\n\nEach node of the ring can perform searches in the overlay.\n\nA search is done by a lookup in the overlay, but there are several\nother demands for communication between peers. \\scalaris{} provides\na general interface to route a message to the (other) peer, which is\ncurrently responsible for a given \\code{key}.\n\n\\codesnippet{api_dht_raw.erl}{api_dht_raw:lookup}{../src/api_dht_raw.erl}\n\nThe message \\code{Msg} could be a \\code{get_key} which retrieves content from\nthe responsible node or a \\code{get_node} message, which returns a pointer\nto the node.\n\nAll currently supported messages are listed in the file \\code{dht_node.erl}.\n\nThe message routing is implemented in \\code{dht_node_lookup.erl}\n\n\\codesnippet{dht_node_lookup.erl}{dht_node_lookup:routing}{../src/dht_node_lookup.erl}\n\nEach node is responsible for a certain key interval. The function\n\\erlfun{intervals}{in}{/2} is used to decide, whether the key is between\nthe current node and its successor. If that is the case, the final step is\ndelivers a \\code{lookup_fin} message to the local node. Otherwise, the message\nis forwarded to the next nearest known peer (listed in the routing table)\ndetermined by \\erlfun{?RT}{next\\_hop}{/2}.\n\n\\code{rt_beh.erl} is a generic interface for routing tables. It\ncan be compared to interfaces in Java. In Erlang interfaces can be\ndefined using a so called `behaviour'.  The files \\code{rt_simple} and\n\\code{rt_chord} implement the behaviour `rt\\_beh'.\n\nThe macro \\code{?RT} is used to select the current implementation of routing\ntables. It is defined in a \\code{rt_*.hrl} file in the \\code{include} directory\nwhich is set during configuration. For \\code{rt_simple}, the following call\nwould include the proper files: \\code{./configure --with-rt=rt_simple}.\n\nThe functions, that have to be implemented for a routing mechanism are\ndefined in the following file:\n\n\\codesnippet{rt_beh.erl}{rt_beh:behaviour}{../src/rt_beh.erl}\n\n\\begin{description}\n\\setlength{\\parskip}{0pt}\n\\setlength{\\itemsep}{0pt}\n\\erlfunindex{rt\\_beh}{empty}\n\\item \\code{empty/1} gets a successor and generates an empty routing\n  table for use inside the routing table implementation. The data structure of\n  the routing table is undefined. It can be a list, a tree, a matrix \\ldots\n\n\\erlfunindex{rt\\_beh}{empty\\_ext}\n\\item \\code{empty_ext/1} similarly creates an empty external routing table\n  for use by the \\erlmodule{dht\\_node}. This process might not need all the\n  information a routing table implementation requires and can thus work with\n  less data.\n\n\\erlfunindex{rt\\_beh}{hash\\_key}\n\\item \\code{hash_key/1} gets a key and maps it into the overlay's\n  identifier space.\n\n\\erlfunindex{rt\\_beh}{get\\_random\\_node\\_id}\n\\item \\code{get_random_node_id/0} returns a random node id from the\n  overlay's identifier space. This is used for example when a new node\n  joins the system.\n\n\\erlfunindex{rt\\_beh}{next\\_hop}\n\\item \\code{next_hop/2} gets a \\erlmodule{dht\\_node}'s state (including the\n  external routing table representation) and a key and returns the node, that\n  should be contacted next when searching for the key, i.e. the known node\n  nearest to the id.\n\n\\erlfunindex{rt\\_beh}{init\\_stabilize}\n\\item \\code{init_stabilize/2} is called periodically to rebuild the\n  routing table. The parameters are the identifier of the node, its\n  successor and the old (internal) routing table state. This method may send\n  messages to the \\code{routing_table} process which need to be handled by\n  the \\code{handle_custom_message/2} handler since they are\n  implementation-specific.\n\n\\erlfunindex{rt\\_beh}{update}\n\\item \\code{update/7} is called when the node's ID, predecessor and/or\n  successor changes. It updates the (internal) routing table with the (new)\n  information.\n\n\\erlfunindex{rt\\_beh}{filter\\_dead\\_node}\n\\item \\code{filter_dead_node/2} is called by the failure detector and tells\n  the routing table about dead nodes. This function gets the (internal) routing\n  table and a node to remove from it. A new routing table state is returned.\n\n\\erlfunindex{rt\\_beh}{to\\_pid\\_list}\n\\item \\code{to_pid_list/1} get the PIDs of all (internal) routing table\n  entries.\n\n\\erlfunindex{rt\\_beh}{get\\_size}\n\\item \\code{get_size/1} get the (internal or external) routing table's size.\n\n\\erlfunindex{rt\\_beh}{get\\_replica\\_keys}\n\\item \\code{get_replica_keys/1} Returns for a given (hashed)\n  \\code{Key} the (hashed) keys of its replicas. This used for implementing\n  symmetric replication.\n\n\\erlfunindex{rt\\_beh}{n}\n\\item \\code{n/0} gets the number of available keys. An implementation may\n  throw \\code{throw:not_supported} if the operation is unsupported by\n  the routing table.\n\n\\erlfunindex{rt\\_beh}{dump}\n\\item \\code{dump/1} dump the (internal) routing table state for debugging,\n  e.g. by using the web interface. Returns a list of\n  \\code{\\{Index, Node_as_String\\}} tuples which may just as well be empty.\n\n\\erlfunindex{rt\\_beh}{to\\_list}\n\\item \\code{to_list/1} convert the (external) representation of the routing\n  table inside a given \\code{dht_node_state} to a sorted list of known nodes\n  from the routing table, i.e. first=succ, second=next known node on the ring,\n  \\ldots This is used by bulk-operations to create a\n  broadcast tree.\n\n\\erlfunindex{rt\\_beh}{export\\_rt\\_to\\_dht\\_node}\n\\item \\code{export_rt_to_dht_node/2} convert the internal routing table state\n  to an external state. Gets the internal state and the node's neighborhood\n  for doing so.\n\n\\erlfunindex{rt\\_beh}{handle\\_custom\\_message}\n\\item \\code{handle_custom_message/2} handle messages specific to the routing\n  table implementation. \\erlmodule{rt\\_loop} will forward unknown messages\n  to this function.\n\n\\erlfunindex{rt\\_beh}{check}\n\\item \\code{check/5}, \\code{check/6} check for routing table changes and send\n  an updated (external) routing table to the \\erlmodule{dht\\_node} process.\n\n\\erlfunindex{rt\\_beh}{check\\_config}\n\\item \\code{check_config/0} check that all required configuration parameters\n  exist and satisfy certain restrictions.\n\n\\erlfunindex{rt\\_beh}{wrap\\_message}\n\\item \\code{wrap_message/1} wraps a message send via a \\code{dht_node_lookup:lookup_aux/4}.\n\n\\erlfunindex{rt\\_beh}{unwrap\\_message}\n\\item \\code{unwrap_message/2} unwraps a message send via \\code{dht_node_lookup:lookup_aux/4}\npreviously wrapped by \\code{wrap_message/1}.\n\n\\end{description}\n\n\\subsection{The routing table process (\\texorpdfstring{\\code{rt_loop}}{rt\\_loop})}\n\\erlmoduleindex{rt\\_loop}\n\nThe \\erlmodule{rt\\_loop} module implements the process for all routing tables.\nIt processes messages and calls the appropriate methods in the specific routing\ntable implementations.\n\n\\codesnippet{rt_loop.erl}{rt_loop:state}{../src/rt_loop.erl}\nIf initialized, the node's id, its predecessor, successor and the routing table\nstate of the selected implementation (the macro \\code{RT} refers to).\n\n\\codesnippet{rt_loop.erl}{rt_loop:trigger}{../src/rt_loop.erl}\nPeriodically (see \\code{pointer_base_stabilization_interval} config parameter) a \\code{trigger}\nmessage is sent to the \\code{rt_loop} process that starts the periodic\nstabilization implemented by each routing table.\n\n\\codesnippet{rt_loop.erl}{rt_loop:update_rt}{../src/rt_loop.erl}\nEvery time a node's neighborhood changes, the \\erlmodule{dht\\_node} sends an\n\\code{update_rt} message to the routing table which will call\n\\erlfun{?RT}{update}{/7} that decides whether the routing table should be\nre-build. If so, it will stop any waiting trigger and schedule an immideate\n(periodic) stabilization.\n\n\\subsection{Simple routing table (\\texorpdfstring{\\code{rt_simple}}{rt\\_simple})}\n\\erlmoduleindex{rt\\_simple}\n\nOne implementation of a routing table is the \\code{rt_simple}, which routes\nvia the successor. Note that this is inefficient as it needs a linear number\nof hops to reach its goal. A more robust implementation, would use a successor\nlist. This implementation is also not very efficient in the presence of churn.\n\n\\subsubsection{Data types}\nFirst, the data structure of the routing table is defined:\n\n\\codesnippet{rt_simple.erl}{rt_simple:types}{../src/rt_simple.erl}\nThe routing table only consists of a node (the successor). Keys in the overlay\nare identified by integers $\\geq 0$.\n\n\\subsubsection{A simple \\texorpdfstring{\\erlmodule{rm\\_beh}}{rm\\_beh} behaviour}\n\n\\erlfunindex{rt\\_simple}{empty}\n\\codesnippet{rt_simple.erl}{rt_simple:empty}{../src/rt_simple.erl}\n\\erlfunindex{rt\\_simple}{empty\\_ext}\n\\codesnippet{rt_simple.erl}{rt_simple:empty_ext}{../src/rt_simple.erl}\nThe empty routing table (internal or external)  consists of the successor.\n\n\\erlfunindex{rt\\_simple}{hash\\_key}\n\\codesnippet{rt_simple.erl}{rt_simple:hash_key}{../src/rt_simple.erl}\nKeys are hashed using MD5 and have a length of 128 bits.\n\n\\erlfunindex{rt\\_simple}{get\\_random\\_node\\_id}\n\\codesnippet{rt_simple.erl}{rt_simple:get_random_node_id}{../src/rt_simple.erl}\nRandom node id generation uses the helpers provided by the \\erlmodule{randoms}\nmodule.\n\n\\erlfunindex{rt\\_simple}{next\\_hop}\n\\codesnippet{rt_simple.erl}{rt_simple:next_hop}{../src/rt_simple.erl}\nNext hop is always the successor.\n\n\\erlfunindex{rt\\_simple}{init\\_stabilize}\n\\codesnippet{rt_simple.erl}{rt_simple:init_stabilize}{../src/rt_simple.erl}\n\\code{init_stabilize/2} resets its routing table to the current successor.\n\n\\erlfunindex{rt\\_simple}{update}\n\\codesnippet{rt_simple.erl}{rt_simple:update}{../src/rt_simple.erl}\n\\code{update/7} updates the routing table with the new successor.\n\n\\erlfunindex{rt\\_simple}{filter\\_dead\\_node}\n\\codesnippet{rt_simple.erl}{rt_simple:filter_dead_node}{../src/rt_simple.erl}\n\\code{filter_dead_node/2} does nothing, as only the successor is listed in\nthe routing table and that is reset periodically in \\code{init_stabilize/2}.\n\n\\erlfunindex{rt\\_simple}{to\\_pid\\_list}\n\\codesnippet{rt_simple.erl}{rt_simple:to_pid_list}{../src/rt_simple.erl}\n\\code{to_pid_list/1} returns the pid of the successor.\n\n\\erlfunindex{rt\\_simple}{get\\_size}\n\\codesnippet{rt_simple.erl}{rt_simple:get_size}{../src/rt_simple.erl}\nThe size of the routing table is always \\code{1}.\n\n\\erlfunindex{rt\\_simple}{get\\_replica\\_keys}\n\\codesnippet{rt_simple.erl}{rt_simple:get_replica_keys}{../src/rt_simple.erl}\nThis \\code{get_replica_keys/1} implements symmetric replication.\n\n\\erlfunindex{rt\\_simple}{n}\n\\codesnippet{rt_simple.erl}{rt_simple:n}{../src/rt_simple.erl}\nThere are $2^{128}$ available keys.\n\n\\erlfunindex{rt\\_simple}{dump}\n\\codesnippet{rt_simple.erl}{rt_simple:dump}{../src/rt_simple.erl}\n\\code{dump/1} lists the successor.\n\n\\erlfunindex{rt\\_simple}{to\\_list}\n\\codesnippet{rt_simple.erl}{rt_simple:to_list}{../src/rt_simple.erl}\n\\code{to_list/1} lists the successor from the external routing table state.\n\n\\erlfunindex{rt\\_simple}{export\\_rt\\_to\\_dht\\_node}\n\\codesnippet{rt_simple.erl}{rt_simple:export_rt_to_dht_node}{../src/rt_simple.erl}\n\\code{export_rt_to_dht_node/2} states that the external routing table is the\nsame as the internal table.\n\n\\erlfunindex{rt\\_simple}{handle\\_custom\\_message}\n\\codesnippet{rt_simple.erl}{rt_simple:handle_custom_message}{../src/rt_simple.erl}\nCustom messages could be send from a routing table process on one node to the\nrouting table process on another node and are independent from any other\nimplementation.\n\n\\codesnippet{rt_simple.hrl}{rt_simple:check}{../src/rt_simple.erl}\nChecks whether the routing table changed and in this case sends the\n\\erlmodule{dht\\_node} an updated (external) routing table state. Optionally\nthe failure detector is updated. This may not be necessary, e.g. if\n\\code{check} is called after a crashed node has been reported by the failure\ndetector (the failure detector already unsubscribes the node in this case).\n\\codesnippet{rt_simple.hrl}{rt_simple:check}{../src/rt_simple.erl}\n\n\\erlfunindex{rt\\_simple}{wrap\\_message}\n\\codesnippet{rt_simple.erl}{rt_simple:wrap_message}{../src/rt_simple.erl}\nWraps a message send via \\code{dht_node_lookup:lookup/4} if needed. This routing algorithm\ndoes not need callbacks when finishing the lookup, so it does not need to wrap the\nmessage.\n\n\\erlfunindex{rt\\_simple}{unwrap\\_message}\n\\codesnippet{rt_simple.erl}{rt_simple:unwrap_message}{../src/rt_simple.erl}\nUnwraps a message previously wrapped with \\code{rt\\_simple:wrap_message/1}. As that\nfunction does not wrap messages, \\code{rt\\_simple:unwrap_message/2} doesn't have to do\nanything as well.\n\n\\subsection{Chord routing table (\\texorpdfstring{\\code{rt_chord}}{rt\\_chord})}\n\\erlmoduleindex{rt\\_chord}\n\nThe file \\code{rt_chord.erl} implements Chord's routing.\n\n\\subsubsection{Data types}\n\n\\codesnippet{rt_chord.erl}{rt_chord:types}{../src/rt_chord.erl}\n\nThe routing table is a \\code{gb_tree}. Identifiers in the ring are\nintegers. Note that in Erlang integer can be of arbitrary\nprecision. For Chord, the identifiers are in $[0, 2^{128})$,\ni.e. 128-bit strings.\n\n\\subsubsection{The \\texorpdfstring{\\erlmodule{rm\\_beh}}{rm\\_beh} behaviour for Chord (excerpt)}\n\n\\erlfunindex{rt\\_chord}{empty}\n\\codesnippet{rt_chord.erl}{rt_chord:empty}{../src/rt_chord.erl}\n\\erlfunindex{rt\\_chord}{empty\\_ext}\n\\codesnippet{rt_chord.erl}{rt_chord:empty_ext}{../src/rt_chord.erl}\n\\code{empty/1} returns an empty \\code{gb_tree}, same for \\code{empty_ext/1}.\n\n\\erlfun{rt\\_chord}{hash\\_key}{/1},\n\\erlfun{rt\\_chord}{get\\_random\\_node\\_id}{/0},\n\\erlfun{rt\\_chord}{get\\_replica\\_keys}{/1} and\n\\erlfun{rt\\_chord}{n}{/0} are implemented like their counterparts in\n\\code{rt_simple.erl}.\n\n\\erlfunindex{rt\\_chord}{next\\_hop}\n\\codesnippet{rt_chord.erl}{rt_chord:next_hop}{../src/rt_chord.erl}\nIf the (external) routing table contains at least one item, the next hop is\nretrieved from the \\code{gb_tree}. It will be the node with the largest id\nthat is smaller than the id we are looking for. If the routing table is empty,\nthe successor is chosen. However, if we haven't found the key in our routing\ntable, the next hop will be our largest finger, i.e. entry.\n\n\\erlfunindex{rt\\_chord}{init\\_stabilize}\n\\codesnippet{rt_chord.erl}{rt_chord:init_stabilize}{../src/rt_chord.erl}\nThe routing table stabilization is triggered for the first index and\nthen runs asynchronously, as we do not want to block the\n\\code{rt_loop} to perform other request while recalculating the\nrouting table.\n\nWe have to find the node responsible for the calculated finger and therefore\nperform a lookup for the node with a \\code{rt_get_node} message, including\na reference to ourselves as the reply-to address and the index to be set.\n\nThe lookup performs an overlay routing by passing the message until\nthe responsible node is found. There, the message is delivered to the\n\\erlmodule{routing\\_table} process\nThe remote node sends the requested information back directly. It includes a\nreference to itself in a \\code{rt_get_node_response} message. Both messages\nare handled by \\erlfun{rt\\_chord}{handle\\_custom\\_message}{/2}:\n\n\\erlfunindex{rt\\_chord}{handle\\_custom\\_message}\n\\codesnippet{rt_chord.erl}{rt_chord:handle_custom_message}{../src/rt_chord.erl}\n\\erlfunindex{rt\\_chord}{stabilize}\n\\codesnippet{rt_chord.erl}{rt_chord:stabilize}{../src/rt_chord.erl}\n\n\\code{stabilize/5} assigns the received routing table entry and triggers the\nrouting table stabilization for the the next shorter entry using the same\nmechanisms as described above.\n\nIf the shortest finger is the successor, then filling the routing table is\nstopped, as no further new entries would occur. It is not necessary, that\n\\code{Index} reaches 1 to make that happen. If less than $2^{128}$ nodes\nparticipate in the system, it may happen earlier.\n\n\\erlfunindex{rt\\_chord}{update}\n\\codesnippet{rt_chord.erl}{rt_chord:update}{../src/rt_chord.erl}\nTells the \\erlmodule{rt\\_loop} process to rebuild the routing table starting\nwith an empty (internal) routing table state.\n\n\\erlfunindex{rt\\_chord}{filter\\_dead\\_node}\n\\codesnippet{rt_chord.erl}{rt_chord:filter_dead_node}{../src/rt_chord.erl}\n\\code{filter_dead_node} removes dead entries from the \\code{gb_tree}.\n\n\\erlfunindex{rt\\_chord}{export\\_rt\\_to\\_dht\\_node}\n\\codesnippet{rt_chord.erl}{rt_chord:export_rt_to_dht_node}{../src/rt_chord.erl}\n\\code{export_rt_to_dht_node} converts the internal \\code{gb_tree} structure\nbased on indices into the external representation optimised for look-ups, i.e.\na \\code{gb_tree} with node ids and the nodes themselves.\n\n\\codesnippet{rt_chord.hrl}{rt_chord:check}{../src/rt_chord.erl}\nChecks whether the routing table changed and in this case sends the\n\\erlmodule{dht\\_node} an updated (external) routing table state. Optionally\nthe failure detector is updated. This may not be necessary, e.g. if\n\\code{check} is called after a crashed node has been reported by the failure\ndetector (the failure detector already unsubscribes the node in this case).\n\n\\erlfunindex{rt\\_chord}{wrap\\_message}\n\\codesnippet{rt_chord.erl}{rt_chord:wrap_message}{../src/rt_chord.erl}\n\nWraps a message send via \\code{dht_node_lookup:lookup/4} if needed. This\nrouting algorithm does not need callbacks when finishing the lookup, so it\ndoes not need to wrap the message.\n\n\\erlfunindex{rt\\_chord}{unwrap\\_message}\n\\codesnippet{rt_chord.erl}{rt_chord:unwrap_message}{../src/rt_chord.erl}\n\nUnwraps a message previously wrapped with\n\\code{rt\\_chord:wrap_message/1}. As that function does not wrap messages,\n\\code{rt\\_chord:unwrap_message/2} doesn't have to do anything as well.\n\n\\section{Local Datastore}\n\n\\section{Cyclon}\n\n\\section{Vivaldi Coordinates}\n\n\\section{Estimated Global Information (Gossiping)}\n\n\\section{Load Balancing}\n\n\\section{Broadcast Trees}\n"
  },
  {
    "path": "user-dev-guide/dev-rrepair.tex",
    "content": "\\chapter{Replica Repair}\n\\label{chapter.rrepair}\n\n\\section{Replica Reconciliation - \\erlmodule{rr\\_recon}}\n\n\\subsection{Trivial Replica Repair}\n\\subsubsection{Protocol}\n\\includegraphics[width=\\linewidth]{rrepair/trivial_sd_uml}\n\n\\subsection{Replica Repair with Bloom Filters}\n\\subsubsection{Protocol}\n\\includegraphics[width=\\linewidth]{rrepair/bloom_sd_uml}\n\n\\subsection{Replica Repair with Merkle Trees}\n\\subsubsection{Protocol}\n\\includegraphics[width=\\linewidth]{rrepair/merkle_sd_uml}\n\n% TODO: create ART sequence diagram\n% the current protocol implementation does not use the new techniques of the\n% other protocols yet\n\n% \\subsection{Replica Repair with Approximate Reconciliation Trees (ART)}\n% \\subsubsection{Protocol}\n% \\includegraphics[width=\\linewidth]{rrepair/art_sd_uml}\n\n\\section{Resolve Replicas - \\erlmodule{rr\\_resolve}}\n\\subsection{Updating a list of keys - \\code{key_upd}}\n\\includegraphics[width=\\linewidth]{rrepair/key_sync_s}\n"
  },
  {
    "path": "user-dev-guide/dev-slide.tex",
    "content": "\\chapter{How data is transferred (atomically)}\n\\label{chapter.slide}\n\nA data transfer from a node to one of its (two) neighbours is also called a\n\\emph{slide}. A slide operation is defined in the \\erlmodule{slide\\_op} module,\nthe protocol is mainly implemented in \\erlmodule{dht\\_node\\_move}.\nParts of the slide are dependent on the ring maintenance implementation and are\nsplit off into modules implementing the \\erlmodule{slide\\_beh} behaviour.\n\nThough the protocols are mainly symmetric, we distinguish between sending data\nto the predecessor and sending data to the successor, respectively. In the\nfollowing protocol visualisations, arrows denote message exchanges, pseudo-code\nfor operations that are being executed is put at the side of each time bar.\nFunctions in green are those implemented in the \\erlmodule{slide\\_beh}\nbehaviour, if annotated with an arrow pointing to itself, this callback is\nasynchronous.\nDuring the protocol, the slide operation goes through several phases which are\nshow in black boxes.\n\nIn general, a slide consists of three steps:\n\\begin{enumerate}\n \\item set up slide\n \\item send data \\& start recording changes, i.e. delta\n \\item send delta \\& transfer responsibility\n\\end{enumerate}\n\nThe latter two may be repeated to execute incremental slides which further\nreduce periods of unavailability. During this period, no node is responsible\nfor the range to transfer and messages are thus delayed until the receiving\nnode gains responsibility.\n\n%\\pagebreak\n\\section{Sending data to the predecessor}\n\n\\subsection{Protocol}\n\\input{slide_send_to_pred-2.0.tikz}\n\n\\subsection{Callbacks}\n% \\medskip\n{%\\small\n\\begin{tabular}{P{4.2cm}P{5.4cm}P{5.4cm}}\n  \\toprule\n  & \\code{slide_chord}\n  & \\code{slide_leases} \\tn\n  \\midrule\n  %\n  \\bfseries $\\leftarrow$ prepare\\_rcv\\_data\n  & \\emph{\\color{gray}nothing to do}\n  & \\emph{\\color{gray}nothing to do} \\tn\n  \\midrule\n  %\n  \\bfseries $\\rightarrow$ prepare\\_send\\_data\n  & add DB range\n  & \\emph{\\color{gray}nothing to do} \\tn\n  \\midrule\n  %\n  \\bfseries $\\leftarrow$ update\\_rcv\\_data\n  & set MSG forward,\\\\change my ID\n  & \\emph{\\color{gray}nothing to do} \\tn\n  \\midrule\n  %\n  \\bfseries $\\rightarrow$ prepare\\_send\\_delta\n  & wait until pred up-to-date,\\\\then: remove DB range\n  & split own lease into two ranges, locally disable lease sent to pred \\tn\n  \\midrule\n  %\n  \\bfseries $\\leftarrow$ finish\\_delta\n  & remove MSG forward\n  & \\emph{\\color{gray}nothing to do} \\tn\n  \\midrule\n  %\n  \\bfseries $\\rightarrow$ finish\\_delta\\_ack\n  & \\emph{\\color{gray}nothing to do}\n  & hand over the lease to pred, notify pred of owner change \\tn\n  \\bottomrule\n\\end{tabular}\n}\n% \\medskip\n\n\\pagebreak\n\\section{Sending data to the successor}\n\n\\subsection{Protocol}\n\\input{slide_send_to_succ-2.0.tikz}\n\n\\subsection{Callbacks}\n\n% \\medskip\n{%\\small\n\\begin{tabular}{P{4.2cm}P{5.4cm}P{5.4cm}}\n  \\toprule\n  & \\code{slide_chord}\n  & \\code{slide_leases} \\tn\n  \\midrule\n  %\n  \\bfseries $\\rightarrow$ prepare\\_rcv\\_data\n  & set MSG forward\n  & \\emph{\\color{gray}nothing to do} \\tn\n  \\midrule\n  %\n  \\bfseries $\\leftarrow$ prepare\\_send\\_data\n  & add DB range,\\\\change my ID\n  & \\emph{\\color{gray}nothing to do} \\tn\n  \\midrule\n  %\n  \\bfseries $\\rightarrow$ update\\_rcv\\_data\n  & \\emph{\\color{gray}nothing to do}\n  & \\emph{\\color{gray}nothing to do} \\tn\n  \\midrule\n  %\n  \\bfseries $\\leftarrow$ prepare\\_send\\_delta\n  & remove DB range\n  & split own lease into two ranges, locally disable lease sent to succ \\tn\n  \\midrule\n  %\n  \\bfseries $\\rightarrow$ finish\\_delta\n  & remove MSG forward,\\\\add DB range,\\\\wait until pred up-to-date\\\\then: remove DB range\n  & \\emph{\\color{gray}nothing to do} \\tn\n  \\midrule\n  %\n  \\bfseries $\\leftarrow$ finish\\_delta\\_ack\n  & \\emph{\\color{gray}nothing to do}\n  & hand over the lease to succ, notify succ of owner change \\tn\n  \\bottomrule\n\\end{tabular}\n}\n% \\medskip\n"
  },
  {
    "path": "user-dev-guide/dev-tx.tex",
    "content": "\\chapter{Transactions in \\scalaris{}}\n\\label{chapter.transactions}\n\n\\section{The Paxos Module}\n\n\\section{Transactions using Paxos Commit}\n\n\\section{Applying the Tx-Modules to replicated DHTs}\n\nIntroduces transaction processing on top of a Overlay\n"
  },
  {
    "path": "user-dev-guide/dht_node_join_phases2.tikz",
    "content": "% TODO: extend this picture and provide all paths between phases\n\\begin{tikzpicture}\n [rounded corners,\n  pre/.style={<-,shorten <=1pt,>=stealth,semithick},\n  post/.style={->,shorten >=1pt,>=stealth,semithick},\n  timeout/.style={draw=black!50, dashed},\n  phase/.style={rectangle,fill=codebackground,drop shadow},\n  bend angle=60]\n \\node[phase] (phase1)                         {init};\n \\node[phase] (phase2)  [right=3.5 of phase1]  {phase2};\n \\node[phase] (phase2b) [below=3 of phase2]    {phase2b};\n \\node[phase] (phase3)  [right=3.5 of phase2]  {phase3};\n \\node[phase] (phase4)  [right=3.5 of phase3]  {phase4};\n\n\n \\path[->] (phase2)\n             edge [pre]  node[auto, swap] {\\code{get_known_nodes()}} (phase1)\n             edge [post] node[auto, anchor=east, text width=4.5cm]\n                          {\n                           \\code{get_number_of_samples()}\\\\\n                           \\footnotesize{\\textcolor{red}{\\code{skip_psv_lb} not set,}}\\\\\n                           \\footnotesize{\\textcolor{red}{non-empty ContactNodes}}\\\\\n                          } (phase2b)\n             edge [post] node[auto, anchor=south, pos=0.6, text width=4cm]\n                          {\n                           \\code{lookup_new_ids2()}\\\\\n                           ~\\footnotesize{$\\hookrightarrow$\\code{lookup_new_ids1()}}\\\\\n                           ~~\\scriptsize{$\\hookrightarrow$\\code{lookup()}}\\\\\n                          }\n                         node[auto, swap, anchor=north, pos=0.64, text width=4cm]\n                          {\n                           \\footnotesize{\\textcolor{red}{\\code{skip_psv_lb} set,}}\\\\\n                           \\footnotesize{\\textcolor{red}{non-empty ContactNodes}}\\\\\n                          } (phase3)\n           (phase2b)\n             edge [post, bend left=-35]\n                         node[auto, swap, pos=0.7, anchor=north west, text width=3cm]\n                          {\n                           \\code{lookup_new_ids2()}\\\\\n                           ~\\footnotesize{$\\hookrightarrow$\\code{lookup_new_ids1()}}\\\\\n                           ~~\\scriptsize{$\\hookrightarrow$\\code{lookup()}}\\\\\n                          } (phase3)\n           (phase3)\n             edge [post] node[auto, anchor=south, pos=0.45, text width=3cm]\n                          {\n                           \\code{contact_best_candidate()}\\\\\n                           ~\\footnotesize{$\\hookrightarrow$\\code{send_join_request()}}\\\\\n                          } (phase4);\n\\end{tikzpicture}\n"
  },
  {
    "path": "user-dev-guide/etextools.sty",
    "content": "%%\n%% This is file `etextools.sty',\n%% generated with the docstrip utility.\n%%\n%% The original source files were:\n%%\n%% etextools.dtx  (with options: `package')\n%% \n%% This is a generated file.\n%% \n%% This work may be distributed and/or modified under the\n%% conditions of the LaTeX Project Public License, either\n%% version 1.3 of this license or (at your option) any later\n%% version. The latest version of this license is in\n%%    http://www.latex-project.org/lppl.txt\n%% \n%% This work consists of the main source file etextools.dtx\n%% and the derived files\n%%       etextools.sty, etextools.pdf, etextools.ins,\n%% and   etextools-examples.tex\n%% \n\\NeedsTeXFormat{LaTeX2e}[1996/12/01]\n\\ProvidesPackage{etextools}\n   [2010/12/07 v3.1415926 e-TeX more useful tools for LaTeX package writers]\n\\csname ettl@onlyonce\\endcsname\\let\\ettl@onlyonce\\endinput\n\\RequirePackage{etex,etoolbox,letltxmacro}\n\\let\\ettl@AtEnd\\@empty\n\\def\\TMP@EnsureCode#1#2{%\n  \\edef\\ettl@AtEnd{%\n    \\ettl@AtEnd\n    \\catcode#1 \\the\\catcode#1\\relax\n  }%\n  \\catcode#1 #2\\relax\n}\n\\TMP@EnsureCode{32}{10}% space... just in case\n\\TMP@EnsureCode{47}{8}%  /\n\\TMP@EnsureCode{167}{7}% \n\\TMP@EnsureCode{164}{7}% \n\\TMP@EnsureCode{95}{11}% _\n\\TMP@EnsureCode{42}{12}% *\n\\TMP@EnsureCode{43}{12}% +\n\\TMP@EnsureCode{45}{12}% -\n\\TMP@EnsureCode{46}{12}% .\n\\TMP@EnsureCode{60}{12}% <\n\\TMP@EnsureCode{61}{12}% =\n\\TMP@EnsureCode{62}{12}% >\n\\TMP@EnsureCode{33}{12}% !\n\\TMP@EnsureCode{152}{13}% ~ for the character test\n\\ifundef\\pdfstrcmp{%\n  \\TMP@EnsureCode{163}{9}%  ignore\n  \\TMP@EnsureCode{128}{14}% \\texteuro comment \n}{\\TMP@EnsureCode{163}{14}%  comment\n  \\TMP@EnsureCode{128}{9}% \\texteuro ignore\n}\n\\AtEndOfPackage{\\ettl@AtEnd\\undef\\ettl@AtEnd}\n\n% A few helper macros\n\\let\\ettl@ifdefined\\ifdefined%\\ifdefined% turn to \\iffalse to test other implementation on pdfTeX\n\\long\\def\\ettl@fi#1\\fi{\\fi#1}\n\\long\\def\\ettl@else#1\\else#2\\fi{\\fi#1}\n\\long\\def\\ettl@or#1\\or#2\\fi{\\fi#1}\n\\def\\ettl@expandaftwo{\\expandafter\\expandafter\\expandafter}\n\\def\\ettl@expandafthree{\\expandafter\\expandafter\\expandafter%\n                        \\expandafter\\expandafter\\expandafter\\expandafter}\n\\cslet{ettl@1of1}\\@firstofone    %% for internal use only\n\\cslet{ettl@1of2}\\@firstoftwo    %% for internal use only\n\\cslet{ettl@2of2}\\@secondoftwo   %% for internal use only\n\\long\\def\\rmn@firstoftwo#1#2{\\z@#1} %% for romannumeral\n\\long\\def\\rmn@secondoftwo#1#2{\\z@#2}%% for romannumeral\n\\long\\def\\ettl@cdr#1#2\\@nil{#2}  %% \\@cdr should be a LONG macro\n\\long\\def\\ettl@car#1#2\\@nil{#1}  %% \\@car should be a LONG macro\n\\long\\csdef{ettl@1of3}#1#2#3{#1}\n\\long\\csdef{ettl@2of3}#1#2#3{#2}\n\\long\\csdef{ettl@3of3}#1#2#3{#3}\n\\long\\csdef{ettl@12of3}#1#2#3{{#1}{#2}}\n\\long\\def\\ettl@carcar#1#2#3#4{#4}\n\\long\\def\\ettl@firstspace#1#2#3{\\expandafter\\ettl@firstsp@ce\\detokenize{#1} \\\\{#3}{#2}//}\n\\long\\def\\ettl@firstsp@ce#1 #2\\\\{\\ettl@nbk#1//}\n\\long\\def\\ettl@csname#1\\endcsname{\\fi\\endcsname}%% useful to get out of \\if\n\\long\\def\\ettl@char#1{\\csname ettl@\\ifcat $\\expandafter\\ettl@cdr\\detokenize{#1}\\@nil$%\n   1\\else2\\fi of2\\endcsname}\n\\providecommand*\\@intmax{2147483647}\n\\def\\ettl@intmax{2147483647}\n\n%% \\ettl@onlypdfTeX for internal use\n\\def\\ettl@onlypdfTeX#1#2{\\@testopt{\\ettl@only@pdfTeX{#1}{#2}}{}}\n\\def\\ettl@only@pdfTeX#1#2[#3]{\\ifundef{#1}\n   {\\ifblank{#3}\n      {\\def#2{\\PackageError{etextools}{\\string#1\\space primitive not found\\MessageBreak\n      pdfTeX seems not to be running}\n      {\\string#2\\space works only if used with pdfTeX (requires \\string#1)}}}\n      {\\AtEndOfPackage{\\let#2=#3}%\n      \\PackageWarning{etextools}{\\string#1\\space primitive not found\\MessageBreak\n      Macro \\string#2\\space has been replaced by \\string#3\\space\\MessageBreak\n      It is not purely expandable}}\n   }\\relax}\n\n%% \\ettl@nbk argument to be tested//{ true }{ false }//\n% `/` has a catcode of 3 all along this package\n\\long\\def\\ettl@nbk #1#2/#3#4#5//{#4}\n\\long\\def\\ettl@nbk@else#1#2/#3#4#5//#6\\else#7\\fi{\\fi#4}\n\\long\\def\\ettl@ney#1//#2#3//{\\romannumeral 0\\csname @%\n   \\ifcat $\\detokenize{#1}$first\\else second\\fi oftwo\\endcsname\n      { #2}{ #3}}\n\n% The following macros are not used (here just for memory, in case of...)\n\\long\\def\\ettl@nbk@cat#1#2#3/#4#5#6//{\\ettl@nbk#6//%\n         {\\ifcat#1#2\\ettl@else#5\\else\\ettl@fi#6\\fi}{#5}//}\n\\long\\def\\ettl@nbk@ifx#1#2#3/#4#5#6//{\\ettl@nbk#6//%\n         {\\ifx#1#2\\ettl@else#5\\else\\ettl@fi#6\\fi}{#5}//}\n\\long\\def\\ettl@nbk@if#1#2#3/#4#5#6//%\n      {\\ettl@nbk#6//{\\if#1#2\\ettl@else#5\\else\\ettl@fi#6\\fi}{#5}//}\n\\long\\def\\ettl@nbk@IF[#1]#2#3#4/#5#6#7//{\\ettl@nbk#7//%\n      {\\csname if#1\\endcsname\\ettl@else#6\\else\\ettl@fi#7\\fi}{#6}//}\n\n%% \\@gobblespace and The Space Token\n\\long\\def\\@gobblespace#1 {#1}\n\n%% \\@gobblescape \\cs-token\n\\newcommand*\\@gobblescape{\\romannumeral-`\\q\\expandafter\\@gobble\\string}\n\n% The swap macros\n\\newcommand\\@swap[2]{#2#1}\n\\@swap{ }{\\let\\ettl@sptoken= }% This makes \\ettl@sptoken a space token\n\\newcommand\\@swaparg[2]{#2{#1}}\n\\newcommand\\@swaplast[3]{#1#3#2}\n\\newcommand\\@swaptwo[2]{{#2}{#1}}\n\n%% \\expandaftercmds : expansion control\n% level 1 : \\expandaftercmds { code }{ cs-token }\n% level 2 : \\expandaftercmds\\expandaftercmds { code }{ cs-token }\n\\newcommand\\expandaftercmds[2]{%\n   \\ifsingletoken\\expandaftercmds{#1}\n      {\\expandafter@cmds{#2}{\\expandafter\\expandafter\\expandafter}}\n      {\\expandafter\\@swap\\expandafter{#2}{#1}}}\n\\long\\def\\expandafter@cmds#1#2#3{%\n   \\ifsingletoken\\expandaftercmds{#1}\n      {\\expandafter@cmds{#3}{\\expandafter#2#2}}\n      {#2\\@swap#2{#3}{#1}}}\n\n%% \\expandnext : expansion control\n% level 1 : \\expandnext { code }{ control sequences : the first is expanded before code }\n% level 2 : \\expandnext\\expandnext{ code }{ control sequences }\n\\newcommand\\expandnext[2]{%\n   \\ifsingletoken\\expandnext{#1}\n      {\\@expandnext{#2}{\\expandafter\\expandafter\\expandafter}}\n      {\\expandafter\\@swaparg\\expandafter{#2}{#1}}}\n\\long\\def\\@expandnext#1#2#3{%\n   \\ifsingletoken\\expandnext{#1}\n      {\\@expandnext{#3}{\\expandafter#2#2}}\n      {#2\\@swaparg#2{#3}{#1}}}\n\n%% \\expandnexttwo{ code }{ control sequences }{ control sequences }\n\\newcommand\\expandnexttwo[3]{\\expandnext{\\expandnext{#1}{#2}}{#3}}\n\n%% \\ExpandAfter { code } { cs-token }\n\\newcommand\\ExpandAftercmds[2]{\\expandafter\\@swap\\expandafter{\\romannumeral-`\\q#2}{#1}}\n\n%% \\ExpandNext { code } { argument }\n   % I'm not sure it is interesting to use \\expandnext here...\n   %\\newcommand\\ExpandNext[2]{\\expandnext{#1}{\\romannumeral-`\\q#2}}\n\\newcommand\\ExpandNext[2]{\\expandafter\\@swaparg\\expandafter{\\romannumeral-`\\q#2}{#1}}\n\n%% \\ExpandNextTwo { code } { arg1 }{ arg2 }\n\\newcommand\\ExpandNextTwo[3]{\\ExpandNext{\\ExpandNext{#1}{#2}}{#3}}\n\n% noexpandcs { csname }\n\\providecommand*\\noexpandcs[1]{\\expandafter\\noexpand\\csname #1\\endcsname}\n\n% noexpandafter\n\\newcommand*\\noexpandafter{\\noexpand\\expandafter}\n\n%% \\thefontname\n\\newcommand\\thefontname{\\nfss@text{\\expandafter\\expandafter\\expandafter\\ettl@thefontname\n   \\expandafter\\expandafter\\expandafter\\meaning\n      \\expandafter\\the\\expandafter\\font\n      \\expandafter\\string\\expandafter(%\n      \\expandafter\\string\\the\\font\\string)}}\n\\ifcsname T1/cmr/m/n/10\\endcsname\n   \\letcs\\ettl@thefontname{T1/cmr/m/n/10}%\n\\else\n   \\font\\ettl@thefontname=ecrm1000\n\\fi\n\n%% \\showcs { csname }\n\\providecommand*\\showcs[1]{\\expandafter\\show\\csname#1\\endcsname}\n\n%% \\showthecs { csname }\n\\providecommand*\\showthecs[1]{\\expandafter\\showthe\\csname#1\\endcsname}\n\n%% \\meaningcs { csname }\n\\providecommand\\meaningcs[1]{\\romannumeral-`\\q\n   \\csname\\ifcsdef{#1}{ettl@meaningcs\\endcsname{#1}}\n                      {meaning\\endcsname\\@undefined}}\n\\def\\ettl@meaningcs#1{\\expandafter\\meaning\\csname#1\\endcsname}% here we don't need \\z@ to stop \\romannumeral\n                                                              % because \\meaning is never blank nor begins with a space...\n\n%% \\strip@meaning { cs-token }\n%% \\strip@meaningcs { csname }\n\\newcommand*\\strip@meaning[1]{\\romannumeral\\csname\\ifdef{#1}%\n      {\\ifdefmacro{#1}{ettl@strip@meaning}{ettl@meaning}\\endcsname#1}{z@\\endcsname}}\n\\providecommand*\\strip@meaningcs[1]{\\romannumeral\\csname\\ifcsdef{#1}%\n      {\\ifcsmacro{#1}{ettl@strip@meaning}{ettl@meaning}%\n               \\expandafter\\endcsname\\csname#1\\endcsname}\n      {z@\\endcsname}}\n\\def\\ettl@strip@meaning{\\expandafter\\expandafter\\expandafter\\z@% for \\romannumeral in case the \\meaning is blank...\n   \\expandafter\\strip@prefix\\meaning}\n\\def\\ettl@meaning{\\expandafter\\z@\\meaning}\n\n%% \\parameters@meaning { cs-token }\n%% \\parameters@meaningcs { csname }\n\\providecommand*\\parameters@meaning[1]{}\n\\edef\\parameters@meaning#1{\\unexpanded{\\romannumeral\\expandafter\n   \\expandafter\\expandafter\\z@\\expandafter\\ettl@params@meaning%\n         \\meaning}#1\\detokenize{macro:->}/}\n\\providecommand*\\parameters@meaningcs[1]{}\n\\edef\\parameters@meaningcs#1{\\unexpanded{\\romannumeral\\ettl@expandafthree\\z@\n   \\expandafter\\expandafter\\expandafter\\ettl@params@meaning%\n         \\expandafter\\meaning\\csname}#1\\endcsname\\detokenize{macro:->}/}\n\\edef\\ettl@params@meaning{%\n   \\def\\noexpand\\ettl@params@meaning\\detokenize{macro:}##1\\detokenize{->}##2/{##1}%\n}\\ettl@params@meaning\n\n%% \\ifdefcount    { cs-token }{ true }{ false } \\FEII\n%% \\ifdefdimen    { cs-token }{ true }{ false } \\FEII\n%% \\ifdeftoks     { cs-token }{ true }{ false } \\FEII\n%% \\ifdefskip     { cs-token }{ true }{ false } \\FEII\n%% \\ifdefmuskip   { cs-token }{ true }{ false } \\FEII\n%% \\ifdefchar     { cs-token }{ true }{ false } \\FEII\n%% \\ifdefmathchar { cs-token }{ true }{ false } \\FEII\n\\def\\ettl@ifdef[#1]{\\expandafter\\ettl@ifd@f\\expandafter{#1}}\n\\def\\ettl@ifd@f#1#2{%\n   \\csdef{ettl@ifdef#2}##1#1##2/EndMeaning/{\\ettl@nbk##2//\\rmn@firstoftwo\\rmn@secondoftwo//}\n   \\csedef{ifdef#2}##1{\\noexpand\\romannumeral\\noexpandafter%\n      \\noexpandcs{ettl@ifdef#2}\\noexpand\\meaning##1#1/EndMeaning/}%//{##2}{##3}//}\n}\n\\ettl@ifdef[\\string\\count]{count}               % defines   \\def\\ifdefcount\n\\ettl@ifdef[\\string\\toks]{toks}                 %           \\def\\ifdeftoks\n\\ettl@ifdef[\\string\\dimen]{dimen}               %           \\def\\ifdefdimen\n\\ettl@ifdef[\\string\\skip]{skip}                 %           \\def\\ifdefskip\n\\ettl@ifdef[\\string\\muskip]{muskip}             %           \\def\\ifdefmuskip\n\\ettl@ifdef[\\string\\char]{char}                 %           \\def\\ifdefchar\n\\ettl@ifdef[\\string\\mathchar]{mathchar}         %           \\def\\ifdefmathchar\n\\ettl@ifdef[\\detokenize{blank space}]{blankspace}%          \\def\\ifdefblankspace\n\\ettl@ifdef[\\detokenize{the character}]{thechar}%           \\def\\ifdefthechar\n\\ettl@ifdef[\\detokenize{the letter}]{theletter} %           \\def\\ifdeftheletter\n\\undef\\ettl@ifdef\\undef\\ettl@ifd@f\n\n%% \\avoidvoid [ replacement code ]{ cs-token / string }\n%% \\avoidvoid*[ replacement code ]{ cs-token / string }\n\\newcommand\\avoidvoid[1]{\\romannumeral\\FE@ifstar{#1}\n      {\\ettl@voidvoid{\\ettl@ifdefempty\\ifempty}}\n      {\\ettl@voidvoid{\\ettl@ifdefvoid\\ifblank}}}\n\\long\\def\\ettl@voidvoid#1#2{\\FE@testopt{#2}{\\ettl@voidv@id#1}{}}\n\\long\\def\\ettl@voidv@id#1#2[#3]#4{\\ifiscs{#4}{#1{#4}}{#2{#4}}{\\z@#3}{\\z@#4}}\n\\long\\def\\ettl@ifdefvoid#1{\\csname @\\ifx#1\\relax first%\n   \\else\\expandafter\\expandafter\\expandafter\\ettl@nbk\\strip@meaning#1//{second}{first}//%\n   \\fi oftwo\\endcsname}\n\\long\\def\\ettl@ifdefempty#1{\\expandafter\\expandafter\\expandafter\\ifempty%\n   \\expandafter\\expandafter\\expandafter{\\strip@meaning#1}}\n\n%% \\avoidvoidcs [ replacement code ]{ cs-name }\n%% \\avoidvoidcs*[ replacement code ]{ cs-name }\n\\newcommand\\avoidvoidcs[1]{\\romannumeral\\FE@ifstar{#1}\n   {\\ettl@avoidvoidcs{\\ettl@ifdefempty}}\n   {\\ettl@avoidvoidcs{\\ettl@ifdefvoid}}}\n\\long\\def\\ettl@avoidvoidcs#1#2{\\FE@testopt{#2}{\\ettl@@voidvoidcs#1}{}}\n\\long\\def\\ettl@@voidvoidcs#1[#2]#3{\\csname @\\ifcsname#3\\endcsname\n   \\expandafter#1\\csname#3\\endcsname{first}{second}\\else first\\fi\n   oftwo\\endcsname{\\z@#2}{\\z@\\csname#3\\endcsname}}\n\n%% The ifx-test and the character-test\n\\long\\def\\ettl@ifx#1#2{\\csname ettl@\\ifx#1#21\\else2\\fi of2\\endcsname}\n\\long\\def\\ettl@ifchar#1#2{\\csname ettl@\\if\\noexpand#2\\string#11of2\\ettl@csname\\fi\n   \\unless\\ifcat\\noexpand#1\\noexpand#22of2\\ettl@csname\\fi\n   \\ifx#1#21\\else2\\fi of2\\endcsname}\n\n%% \\ifsingletoken{ tokenA }{ tokenB }{ true }{ false }\n\\newcommand\\ifsingletoken[2]{\\romannumeral\\csname rmn@\\ettl@firstspace{#2}\n   {\\ettl@nbk#1#2//{second}{\\ifcat $\\detokenize{#1#2}$first\\else\\ifx#1#2first\\else second\\fi\\fi}//}\n   {\\ifcat $\\detokenize\\expandafter{\\ettl@cdr#2\\@nil}$%\n         \\expandafter\\ettl@ifxsingle\n   \\else\\expandafter\\ettl@carcar\n   \\fi{#1}{#2}{first}{second}}%\n   oftwo\\endcsname}\n\\def\\ettl@ifxsingle#1#2#3#4{\\ettl@nbk#1//{\\ifx#1#2#3\\else#4\\fi}{#4}//}\n\n%% \\iffirsttoken token { string }{ true }{ false }\n\\providecommand\\iffirsttoken[2]{\\romannumeral\\csname rmn@%\n   \\ettl@nbk#2//%\n      {\\ettl@nbk#1//%\n         {\\expandnexttwo\\ettl@ifx{\\ettl@car#2\\@nil}{\\ettl@car#1\\@nil}{first}{second}}\n         {\\ifcat $\\detokenize{#1}$secondoftwo\\ettl@csname\\fi\n         \\ettl@firstspace{#2}{first}{second}}//}%\n      {\\ettl@nbk#1//%\n         {\\ifcat $\\detokenize{#2}$secondoftwo\\ettl@csname\\fi\n         \\ettl@firstspace{#1}{first}{second}}\n         {\\ifcat $\\detokenize{#1#2}$first\\else second\\fi}}//%\n   oftwo\\endcsname}\n\n%% \\ifOneToken{ token }{ true }{ false }\n\\newcommand\\ifOneToken[1]{\\romannumeral\\csname rmn@\\ettl@firstspace{#1}\n   {\\ettl@nbk#1//{second}{\\ifcat $\\detokenize{#1}$second\\else first\\fi}//}\n   {\\ifcat $\\detokenize\\expandafter{\\ettl@cdr#1\\@nil}$%\n   first\\else second\\fi}oftwo\\endcsname}\n\n%% \\ifsinglechar character{ string }{ true }{ false }\n\\long\\def\\ifsinglechar#1#2{\\romannumeral\\csname rmn@\\ettl@firstspace{#2}\n   {\\ettl@nbk#2//{second}{\\ifcat $\\detokenize{#1#2}$first\\else\\ifx#1#2first\\else second\\fi\\fi}//}\n   {\\ifcat $\\detokenize\\expandafter{\\ettl@cdr#2\\@nil}$%\n         \\expandafter\\ettl@ifchar\n      \\else\\expandafter\\ettl@carcar\n      \\fi{#1}{#2}{first}{second}}%\n   oftwo\\endcsname}\n\n%% \\ifOneChar{ string }{ true }{ false }\n\\ettl@ifdefined\\pdfmatch\n\\newcommand\\ifOneChar[1]{\\romannumeral\\csname rmn@%\n      \\ifnum\\pdfmatch{\\detokenize{^.$}}{\\detokenize{#1}}=1 first\\else second\\fi\n      oftwo\\endcsname}\n\\else\n\\newcommand\\ifOneChar[1]{\\romannumeral\\csname rmn@\\ettl@firstspace{#1}\n   {\\ettl@nbk#1//{second}{\\ifcat $\\detokenize{#1}$second\\else first\\fi}//}\n   {\\ifcat $\\expandafter\\ettl@cdr\\detokenize{#1}\\@nil$%\n    first\\else second\\fi}oftwo\\endcsname}\n\\fi%\\pdfmatch\n\n%% \\ifOneCharWithBlanks{ string }{ true }{ false }\n\\ettl@ifdefined\\pdfmatch\n\\newcommand\\ifOneCharWithBlanks[1]{\\romannumeral\\csname rmn@%\n      \\ifnum\\pdfmatch{\\detokenize{^[[:space:]]*[^[:space:]][[:space:]]*$}}{\\detokenize{#1}}=1 %\n      first\\else second\\fi oftwo\\endcsname}\n\\else\n\\newcommand\\ifOneCharWithBlanks[1]{\\romannumeral\\csname rmn@\\ettl@nbk#1//%\n      {\\expandafter\\expandafter\\expandafter\\ettl@nbk\n            \\expandafter\\ettl@cdr\\detokenize{#1}\\@nil//{second}{first}//}%\n      {second}//oftwo\\endcsname}\n\\fi\n\n%% \\iffirstchar{ string1 }{ string2 }{ true }{ false }\n\\newcommand\\iffirstchar[2]{\\romannumeral\\csname rmn@%\n   \\ettl@nbk#2//%\n      {\\ettl@nbk#1//%\n         {\\expandnexttwo\\ettl@ifchar{\\ettl@car#2\\@nil}{\\ettl@car#1\\@nil}{first}{second}}\n         {\\ifcat $\\detokenize{#1}$secondoftwo\\ettl@csname\\fi\n         \\ettl@firstspace{#2}{first}{second}}//}%\n      {\\ettl@nbk#1//%\n         {\\ifcat $\\detokenize{#2}$secondoftwo\\ettl@csname\\fi\n         \\ettl@firstspace{#1}{first}{second}}\n         {\\ifcat $\\detokenize{#1#2}$first\\else second\\fi}}//%\n   oftwo\\endcsname}\n\n%% \\ifiscs { string }{ true }{ false }\n\\newcommand\\ifiscs[1]{\\romannumeral\\csname rmn@\\ettl@nbk#1//%\n   {\\ifcat $\\expandafter\\ettl@cdr\\detokenize{#1}\\@nil$secondoftwo\\ettl@csname\\fi\n    \\ifcat $\\detokenize\\expandafter{\\ettl@cdr#1\\@nil}$%\n      \\expandafter\\ettl@firstspace\n    \\else secondoftwo\\ettl@csname\\fi{#1}{second}{first}}\n   {second}//oftwo\\endcsname}\n\n%% \\detokenizeChars { list of single tokens }\n\\newcommand\\detokenizeChars[1]{\\expandafter\\ettl@dosinglelist\n   \\expandafter\\ettl@do@detokenChars\\expandafter{\\romannumeral\\protectspace{\\z@#1}}}\n\\long\\def\\ettl@do@detokenChars#1{\\ifOneChar{#1}\\detokenize\\unexpanded{#1}}\n\n%% \\protectspace { string }\n\\newcommand\\protectspace[1]{\\romannumeral\\ettl@protectspace#1 /EndString/}\n\\long\\def\\ettl@protectspace#1 #2/EndString/{\\ifempty{#2}{\\z@#1}\n   {\\expandafter\\@swap\\expandafter{\\romannumeral\\ettl@protectspace#2/EndString/}{\\z@#1{ }}}}\n\n%% \\ifempty{ text }{ true }{ false }\n\\newcommand\\ifempty[1]{\\romannumeral\\csname rmn@\\ifcat $\\detokenize{#1}$%\n   first\\else second\\fi oftwo\\endcsname}\n\n%% \\ifnotempty{ text }{ true }{ false }\n\\newcommand\\ifnotempty[1]{\\romannumeral\\csname rmn@\\ifcat $\\detokenize{#1}$%\n   second\\else first\\fi oftwo\\endcsname}\n\n%% \\xifempty{ text }{ true }{ false }\n\\newcommand\\xifempty[1]{\\xifstrcmp{#1}{}}\n\\ettl@onlypdfTeX\\pdfstrcmp\\xifempty[\\xifstrempty]\n\n%% \\ifnotblank{ text }{ true }{ false }\n\\long\\def\\ifnotblank#1#2#3{\\ettl@nbk#1//{#2}{#3}//}\n\n%% \\xifblank{ string }{ true }{ false }\n\\newrobustcmd\\xifblank[1]{\\begingroup\n   \\protected@edef\\@xifblank{\\endgroup\n         \\noexpand\\ifblank{#1}%\n   }\\@xifblank}\n\n%% \\deblank{ string }\n\\newcommand\\deblank[1]{\\romannumeral\\ettl@deblank#1/ /}\n\\long\\def\\ettl@deblank#1 /{\\ettl@deblank@i#1/}\n\\long\\def\\ettl@deblank@i#1/#2{\\z@#1}\n\\newcommand\\ettl@stringify[1]{\\romannumeral-`\\q\\ettl@expandafthree\\@gobblescape%\n         \\expandafter\\ettl@deblank\\detokenize{#1}/ /}\n\n%% \\ifstrcmp{ string1 }{ string2 }{ true }{ false }\n\\newcommand\\ifstrcmp[2]{\\romannumeral\\csname rmn@%\n  \\ifnum\\pdfstrcmp{\\detokenize{#1}}{\\detokenize{#2}}=0 first\\else second\\fi\n  oftwo\\endcsname}\n\\ettl@onlypdfTeX\\pdfstrcmp\\ifstrcmp[\\ifstrequal]\n\n%% \\xifstrcmp{ string1 }{ string2 }{ true }{ false }\n\\newcommand\\xifstrcmp[2]{\\csname @%\n  \\ifnum\\pdfstrcmp{#1}{#2}=0 first\\else second\\fi\n  oftwo\\endcsname}\n\\ettl@onlypdfTeX\\pdfstrcmp\\xifstrcmp[\\xifstrequal]\n\\newrobustcmd\\xifstrequal[2]{\\begingroup\n   \\protected@edef\\@xifstrequal{\\endgroup\\noexpand\\ifstrequal{#1}{#2}%\n   }\\@xifstrequal}\n\n%% \\ifcharupper{ string }{ true }{ false }\n%% \\ifcharlower{ string }{ true }{ false }\n\\newcommand\\ifcharupper[1]{\\romannumeral\\csname rmn@%\n   \\ifnum`\\#1=\\uccode`\\#1 first\\else second\\fi oftwo\\endcsname}\n\\newcommand\\ifcharlower[1]{\\romannumeral\\csname rmn@%\n   \\ifnum`\\#1=\\lccode`\\#1 first\\else second\\fi oftwo\\endcsname}\n\n%% \\ifuppercase{ string }{ true }{ false }\n%% \\iflowercase{ string }{ true }{ false }\n\\newrobustcmd\\ifuppercase[1]{\\uppercase{\\ifstrcmp{#1}}{#1}}\n\\newrobustcmd\\iflowercase[1]{\\lowercase{\\ifstrcmp{#1}}{#1}}\n\n%% \\ifstrmatch{ pattern }{ string }{ true }{ false }\n\\newcommand\\ifstrmatch[2]{\\romannumeral\\csname rmn@%\n   \\ifnum\\pdfmatch{#1}{#2}=1 first\\else second\\fi oftwo\\endcsname}\n\\ettl@onlypdfTeX\\pdfmatch\\ifstrmatch\n\n%% \\ifstrdigit{ string }{ true }{ false }\n\\ettl@ifdefined\\pdfmatch\n\\newcommand\\ifstrdigit[1]{\\romannumeral\\csname rmn@\\ifnum\\pdfmatch{\\detokenize{^[[:digit:]]$}}%\n        {\\detokenize{#1}}=1 first\\else second\\fi oftwo\\endcsname}\n\\else\n\\def\\do#1{\\cslet{ettl@number#1}=#1%\n}\\docsvlist{0,1,2,3,4,5,6,7,8,9}\n\\newcommand\\ifstrdigit[1]{\\romannumeral\\csname rmn@%\n   \\ifcsname ettl@number\\detokenize{#1}\\endcsname first\\else second\\fi oftwo\\endcsname}\n\\fi%\\pdfmatch\n\n%% \\ifstrnum{ string }{ true }{ false }\n\\ettl@ifdefined\\pdfmatch\n\\newcommand\\ifstrnum[1]{\\romannumeral\\csname rmn@\\ifnum\\pdfmatch\n   {\\detokenize{^([[:space:]]*-?)*+[[:digit:]]+[[:space:]]*$}}{\\detokenize{#1}}=1 %\n   first\\else second\\fi oftwo\\endcsname}\n\\else\n\\newcommand\\ifstrnum[1]{\\romannumeral\\csname rmn@\\ettl@nbk#1//%\n         {\\expandafter\\ettl@numberminus\\detokenize{#1}-/EndString/}{second}//oftwo\\endcsname}\n\\long\\def\\ettl@numberminus#1-#2/EndString/{\\ettl@nbk#2//%\n         {\\ettl@nbk#1//{second}{\\ettl@numberminus#2/EndString/}//}%\n         {\\expandafter\\expandafter\\expandafter\\ettl@numberspace\\deblank{#1} /EndString/}//}%\n\\long\\def\\ettl@numberspace#1 #2/EndString/{\\ettl@nbk#2//{second}{\\ettl@ifstrnum#1/EndString/}//}\n\\long\\def\\ettl@ifstrnum#1#2/EndString/{%\n   \\ifcsname ettl@number#1\\endcsname% #1 detokenized before, ok\n      \\ettl@nbk#2//{\\ettl@ifstrnum#2/EndString/}{first}//%\n   \\else second%\n   \\fi}\n\\fi%\\pdfmatch\n\n%% \\DeclareStringFilter[\\global]{ \\StringFilterMacro }{ string }\n\\newrobustcmd\\DeclareStringFilter[3][\\global]{\\@ifdefinable#2%\n   {\\expandnext\\ettl@declarestrfilter%\n            {\\csname\\@gobblescape#2\\detokenize{->\"#3\"}\\endcsname}{#1}{#2}{#3}}}\n\\newcommand\\ettl@declarestrfilter[4]{%\n   #2\\csdef{\\@gobblescape#1}##1#4##2/EndString/{##1/##2}% This the FILTER\n   #2\\long\\def#3##1{\\FE@modifiers{=<>?-+!}{##1}\n      {\\ettl@strfilt@mod 0{{#4}{}{#1}[1]}}%=\n      {\\ettl@strfilt@mod 1{{#4}{}{#1}[1]}}%<\n      {\\ettl@strfilt@mod 2{{#4}{}{#1}[\\ettl@intmax]}}%>\n      {\\ettl@strfilt@mod 3{{#4}{}{#1}}}%?\n      {\\ettl@strfilt@mod 4{{#4}{}{#1}}}%-\n      {\\ettl@strfilt@mod 5{{#4}{}{#1}}}%+\n      {\\ettl@strfilt\\ettl@strfilt@count{#4}{}{#1}[\\ettl@intmax]}%!\n      {\\ettl@strfilt\\ettl@strfilt@equal{#4}{}{#1}[1]}}}% default\n\\def\\ettl@strfilt@mod #1#2#3{%\n   \\ifcase#1 \\ettl@or\\ettl@ifchardot{#3}%\n               {\\ettl@strfilt\\ettl@strfilt@equal#2}\n               {\\FE@ifcharequal{#3}%\n                  {\\ettl@strfilt\\ettl@strfilt@equaleq#2}%\n                  {\\ettl@strfilt\\ettl@strfilt@equal#2}}%\n   \\or\\ettl@or\\ettl@ifchardot{#3}%\n               {\\ettl@strfilt\\ettl@strfilt@start#2}%\n               {\\FE@ifcharequal{#3}\n                  {\\ettl@strfilt\\ettl@strfilt@starteq#2}%\n                  {\\ettl@strfilt\\ettl@strfilt@start#2}}%\n   \\or\\ettl@or\\ettl@ifchardot{#3}%\n               {\\ettl@strfilt\\ettl@strfilt@endby#2}%\n               {\\FE@ifcharequal{#3}\n                  {\\ettl@strfilt\\ettl@strfilt@endbyeq#2}%\n                  {\\ettl@strfilt\\ettl@strfilt@endby#2}}%\n   \\or\\ettl@or\\ettl@ifchardot{#3}%\n               {\\ettl@strfilt\\ettl@strfilt@instr#2[1]}\n               {\\FE@testopt{#3}{\\ettl@strfilt\\ettl@strfilt@instr#2}{1}}%\n   \\or\\ettl@or\\ettl@ifchardot{#3}%\n               {\\ettl@strfilt@REMOVE{#2}[\\ettl@intmax]}%\n               {\\FE@testopt{#3}{\\ettl@strfilt@REMOVE{#2}}{\\ettl@intmax}}%\n   \\or\\ettl@fi\\ettl@ifchardot{#3}%\n               {\\ettl@strfilt@REPLACE#2[\\ettl@intmax]}%\n               {\\FE@testopt{#3}{\\ettl@strfilt@REPLACE#2}{\\ettl@intmax}}%\n   \\fi}\n\\long\\def\\ettl@strfilt#1#2#3#4[#5]#6{% % #1 = test macro\n     \\ExpandAftercmds#1{\\ettl@Remove #6/EndString/{#2}{#3}[{#5}]{#4}}}\n\\long\\def\\ettl@strfilt@REMOVE #1[#2]{%\n   \\ifnum\\numexpr#2>0 \\ettl@else\\ettl@strfilt\\ettl@strfilt@remove#1[#2]%\n   \\else\\expandafter\\@firstofone%\n   \\fi}\n\\long\\def\\ettl@strfilt@REPLACE #1#2#3#4[#5]#6#7{%\n   \\ifnum\\numexpr#5>0 \\ettl@else\\ettl@strfilt\\ettl@strfilt@replace{#1}{#7}{#3}[{#5}]{#6}%\n   \\else\\expandafter\\@firstoftwo%\n   \\fi}\n\\long\\def\\ettl@Remove#1/EndString/#2#3[#4]#5{%\n   \\expandafter\\ettl@Remove@loop #5#1//#2/EndString//EndString/{#3}[{#4-1}]{#5}}\n\\long\\def\\ettl@Remove@loop#1/#2//#3/EndString/#4[#5]#6{%\n   \\ifnum\\numexpr#5>0 \\ettl@nbk@else#2//%\n            {\\ettl@Remove #1#4#2/EndString/{#3}{#4}[{#5}]{#6}}\n            {{#1}{#4#2}{#3}{#5}}//%\n   \\else\\ettl@fi{#1}{#4#2}{#3}{#5}%\n   \\fi}\n\\long\\def\\ettl@strfilt@equal   #1#2#3#4{\\csname @%\n      \\ettl@nbk#3//{\\ettl@nbk#1#2//{second}{first}//}{second}//oftwo\\endcsname}\n\\long\\def\\ettl@strfilt@equaleq #1#2#3#4{\\csname @%\n      \\ettl@nbk#3//{\\ifnotempty{#1#2}{second}{first}}{second}//oftwo\\endcsname}\n\\long\\def\\ettl@strfilt@start   #1#2#3#4{\\csname @%\n      \\ettl@nbk#1//{second}{first}//oftwo\\endcsname}\n\\long\\def\\ettl@strfilt@starteq #1#2#3#4{\\csname @%\n      \\ifnotempty{#1}{second}{first}oftwo\\endcsname}\n\\long\\def\\ettl@strfilt@endby   #1#2#3#4{\\csname @%\n      \\ettl@nbk#3//{first}{second}//oftwo\\endcsname}\n\\long\\def\\ettl@strfilt@endbyeq #1#2#3#4{\\csname @%\n      \\ettl@nbk#3//{\\ifempty{#2}{first}{second}}{second}//oftwo\\endcsname}\n\\long\\def\\ettl@strfilt@count   #1#2#3#4{\\number\\numexpr\\ettl@intmax-(#4)-\\ettl@nbk#3//01//}\n\\long\\def\\ettl@strfilt@instr   #1#2#3#4{\\csname @%\n      \\ifnum\\numexpr#4>0 second%\n      \\else\\ifnum\\numexpr#4<0 first%\n      \\else\\ettl@nbk#3//{first}{second}//%\n      \\fi\\fi oftwo\\endcsname}\n\\long\\def\\ettl@strfilt@remove  #1#2#3#4{#1#2}\n\\long\\def\\ettl@strfilt@replace #1#2#3#4{#1\\ettl@nbk#3//{#2}{}//}\n%----------------------------------------------------------------------------\n% Purely Expandable Macros With Options / Star Forms / Variants\n%% \\FE@testopt{ #1 }{ commands }{ default option }\n\\long\\def\\ettl@BasicFilter#1#2#3/EndString/{\\expandafter\\ettl@B@sicFilter #1#3//#2/EndString//EndString/}\n\\long\\def\\ettl@B@sicFilter#1/#2//#3/EndString/{@\\ettl@nbk#3//%\n      {\\ifcat $\\detokenize{#1#2}$first\\else second\\fi}\n      {second}//oftwo}\n\\newcommand\\FE@testopt[3]{\\ettl@FE@testopt#1/[/%\n               {#2#1}%\n               {#2[{#3}]{#1}}}%]\n\\long\\def\\ettl@FE@testopt#1[#2/#3#{\\csname @\\ifcat $\\detokenize{#1#2}$%\n   first\\else second\\fi oftwo\\endcsname}\n\n%% \\FE@ifstar{ #1 } { \\StarredMacro }{ \\NotStarredMacro }\n\\newcommand\\FE@ifstar[3]{\\ettl@FE@ifstar#1/*/%\n               {#2}%\n               {#3{#1}}}\n\\long\\def\\ettl@FE@ifstar#1*#2/#3#{\\csname @\\ifcat $\\detokenize{#1#2}$%\n   first\\else second\\fi oftwo\\endcsname}\n\n%% \\FE@charequal{ #1 } { \\MacroWith= }{ \\NormalMacro } % used by the string filters\n\\newcommand\\FE@ifcharequal[3]{\\ettl@FE@charequal#1/=/%\n               {#2}%\n               {#3{#1}}}\n\\long\\def\\ettl@FE@charequal#1=#2/#3#{\\csname @\\ifcat $\\detokenize{#1#2}$%\n   first\\else second\\fi oftwo\\endcsname}\n\n%% \\ettl@strfilt@dot{ #1 } { \\MacroWith= }{ \\NormalMacro } % used by the string filters\n\\newcommand\\ettl@ifchardot[1]{\\ettl@FE@chardot#1/./}\n\\long\\def\\ettl@FE@chardot#1.#2/#3#{\\csname @\\ifcat $\\detokenize{#1#2}$%\n   first\\else second\\fi oftwo\\endcsname}\n\n%% \\FE@ifchar{ <Character> }{ #1 }{ \\SpecialFormMacro }{ \\NormalMacro }\n\\newcommand\\FE@ifchar[4]{\\ifsinglechar{#1}{#2}{#3}{#4{#2}}}\n\n%% \\FE@modifiers{ Allowed Characters }{ #1 }{ \\MacroA }{ \\MacroB }{ \\MacroC }...{ \\MacroZ }{ \\MacroDefault }\n\\newcommand\\FE@modifiers[2]{%\n   \\ifOneToken{#2}%\n      {\\ExpandAftercmds\\ettl@FE@modifiers%\n               {\\ExpandAftercmds{\\ettl@setresult 12of3><}\n                  {\\ettl@getsinglelist{\\ettl@ifchar{#2}}{#1}}}{#2}}\n      {\\ExpandNextTwo{\\ettl@supergobble[{{#2}}]}{-1}{\\getcharlistcount{#1}+1}}}\n\\long\\def\\ettl@FE@modifiers#1#2#3{\\expandafter\\ettl@supergobble%\n   \\expandafter[\\romannumeral-`\\q\\ifnum#2<0 \\@swap{{#3}}\\fi]{#2}{#1+1}}\n\n%% \\ettl@supergobble{ p }{ q }{token_1}{token_2}...{token_p}{TOKEN_p+1}{token_p+2}...{token_p+q+1}\n\\newcommand\\ettl@supergobble[1]{\\FE@testopt{#1}\\ettl@superg@bble{}}\n\\long\\def\\ettl@superg@bble[#1]#2#3{%\n   \\ifnum\\numexpr#3>0\n      \\ifnum\\numexpr#3-(#2)=0\n         \\ettl@supergobble@loop{#3+2}0{\\ettl@supergobble@end{}{}}%\n      \\else\n         \\expandafter\\ettl@supergobble@loop\\expandafter{%\n            \\number\\numexpr\\ifnum\\numexpr#2*(#2-(#3))>0 #3+1\\else#2+2\\fi}{#3+2}%\n                           {\\ettl@supergobble@next{}{#1}}%\n   \\fi\\fi}\n\\long\\def\\ettl@supergobble@loop#1#2#3{%\n   \\ifcsname ettl@supergobble\\number\\numexpr#1\\endcsname\n      \\csname ettl@supergobble\\number\\numexpr#1\\endcsname\n         {#3{#2-(#1)-1}}%\n   \\else\\ettl@supergobbleeight{\\ettl@supergobble@loop{#1-8}{#2-8}{#3}}%\n   \\fi}\n\\long\\def\\ettl@supergobble@end#1#2#3{\\fi\\fi\\fi#1#2}\n\\long\\csdef{ettl@supergobbleeight}#1\\fi#2#3#4#5#6#7#8#9{\\fi#1}\n\\long\\csdef{ettl@supergobble7}#1#2\\fi#3#4#5#6#7#8#9{#1}\n\\long\\csdef{ettl@supergobble6}#1#2\\fi#3#4#5#6#7#8{#1}\n\\long\\csdef{ettl@supergobble5}#1#2\\fi#3#4#5#6#7{#1}\n\\long\\csdef{ettl@supergobble4}#1#2\\fi#3#4#5#6{#1}\n\\long\\csdef{ettl@supergobble3}#1#2\\fi#3#4#5{#1}\n\\long\\csdef{ettl@supergobble2}#1#2\\fi#3#4{#1}\n\\long\\csdef{ettl@supergobble1}#1#2\\fi#3{#1}\n\\long\\csdef{ettl@supergobble0}#1#2\\fi{#1}\n\\long\\def\\ettl@supergobble@next#1#2#3#4{\\fi\n   \\ettl@supergobble@loop{#3}0{\\ettl@supergobble@end{#4}{#2}}}\n\n%% \\AfterGroup{ code }  /  \\AfterGroup*{ code }    the star form expands its argument once\n\\newcount\\ettl@fter\n\\newrobustcmd\\AfterGroup{\\@ifstar{\\ettl@AfterGroup\\@firstofone}{\\ettl@AfterGroup\\unexpanded}}\n\\newrobustcmd\\ettl@AfterGroup[2]{%\n   \\csxdef{ettl@fterGroup\\number\\numexpr\\the\\ettl@fter+1}%\n      {\\global\\csundef{ettl@fterGroup\\number\\numexpr\\the\\ettl@fter+1}#1{#2}}%\n   \\global\\advance\\ettl@fter\\@ne\n   \\expandafter\\aftergroup\\csname ettl@fterGroup\\the\\ettl@fter\\endcsname}\n\n%% \\AfterAssignment{ code }\n\\newrobustcmd\\AfterAssignment{\\@ifstar{\\ettl@AfterAssignment\\@firstofone}{\\ettl@AfterAssignment\\unexpanded}}\n\\newrobustcmd\\ettl@AfterAssignment[2]{%\n   \\csedef{ettl@afterassignment@hook\\number\\numexpr\\the\\ettl@fter}{#1{#2}}%\n   \\global\\advance\\ettl@fter\\@ne\n   \\expandafter\\afterassignment\\csname ettl@afterassignment@hook\\the\\ettl@fter\\endcsname}\n\n%% \\aftergroup@def{ cs-token }\n\\newrobustcmd*\\aftergroup@def{\\@ifstar\\aftergroup@defstar\\aftergroup@d@f}\n\\newrobustcmd*\\@ftergroup@def[3]{%\n   \\aftergroup#2\\aftergroup#3\\aftergroup#1%\n   \\aftergroup\\global \\aftergroup\\undef \\aftergroup#1}\n\\newrobustcmd*\\aftergroup@defstar[1]{%\n   \\global\\expandafter\\let\\csname ettl@ftergroup@def\\the\\numexpr\\ettl@fter+1\\endcsname#1%\n   \\global\\advance\\ettl@fter\\@ne\n   \\expandafter\\@ftergroup@def\\csname ettl@ftergroup@def\\the\\ettl@fter\\endcsname\\let#1}\n\\newrobustcmd*\\aftergroup@d@f[1]{%\n   \\let\\etex@let@primitive\\let   \\def\\let{\\global\\etex@let@primitive}%\n   \\expandafter\\LetLtxMacro\\csname ettl@ftergroup@def\\the\\numexpr\\ettl@fter+1\\endcsname#1%\n   \\global\\advance\\ettl@fter\\@ne\n   \\etex@let@primitive\\let=\\etex@let@primitive\n   \\expandafter\\@ftergroup@def\\csname ettl@ftergroup@def\\the\\ettl@fter\\endcsname\\LetLtxMacro#1}%\n\\let\\ettl@aftergroup@def\\aftergroup@def\n%----------------------------------------------------------------------------\n\n%% \\@ifchar{ single token }{ true }{ false }\n\n\\long\\def\\@ifchar#1#2{\\ettl@ifnextchar #1{\\@firstoftwo{#2}}}\n\n%% \\@char@testopt{ code }{ single token }{ default value }{ single token }\n\n\\newcommand\\@char@testopt[4]{\\ettl@ifnextchar#2{#1}{#1#2#3#4}}\n\n%% \\@ifnextchardigit{ true }{ false }\n\n\\newrobustcmd\\@ifnextchardigit[2]{\\begingroup\n\n   \\endgroup\n}% \\@ifnextchardigit\n\n%% \\ettl@ifnextchar{ character token }{ true }{ false }\n\n\\newrobustcmd\\ettl@ifnextchar[3]{\\begingroup\n   \\long\\edef\\1##1/##2/##3{##1\\endgroup\\unexpanded{#2}##3}%\n   \\long\\edef\\2##1/##2/##3{##1\\endgroup\\unexpanded{#3}##3}%\n   \\ifOneToken{#1}\n      {\\csname ettl@\\ifcat $\\expandafter\\ettl@cdr\\detokenize{#1}\\@nil$% OneChar\n         xifnch\\else xifntk\\fi\\endcsname{#1}}\n      {\\2//{}}}\n\\long\\def\\ettl@xifnch#1{%\n   \\ifx#1\\@sptoken \\def\\ettl@xifnch{\\ifx\\@let@token\\@sptoken\\1\\else\\2\\fi//{}}%\n   \\else \\def\\ettl@xifnch{%\n      \\ifx\\@let@token\\bgroup      \\2\n      \\else\\ifx\\@let@token\\egroup \\2\n      \\else\\ifx\\@let@token\\@sptoken \\ettl@ifnspace\\ettl@xifnch\n     \\else\\ettl@ifnch\n      \\fi\\fi\\fi/{#1}/{}}%\n   \\fi\\futurelet\\@let@token\\ettl@xifnch}\n\\long\\def\\ettl@ifnch#1/#2/#3{#1\\long\\def\\ettl@ifnch##1{\\ettl@char{##1}\n      {\\if\\string##1\\string#2\\1\\else\\2\\fi}\\2//{##1}}\\ettl@ifnch}\n\\long\\def\\ettl@xifntk#1{%\n   \\ifx#1\\bgroup\\def\\ettl@xifntk{\\ifx\\@let@token\\bgroup\\1\\else\\2\\fi//{}}%\n   \\else\\ifx#1\\egroup\\def\\ettl@xifntk{\\ifx\\@let@token\\egroup\\1\\else\\2\\fi//{}}%\n   \\else\\def\\ettl@xifntk{%\n      \\ifx\\@let@token\\bgroup        \\2\n      \\else\\ifx\\@let@token\\egroup   \\2\n      \\else\\ifx\\@let@token\\@sptoken \\ettl@ifnspace\\ettl@xifntk%\n      \\else\\ettl@ifntk%\n      \\fi\\fi\\fi/{#1}/{}}%\n   \\fi\\futurelet\\@let@token\\ettl@xifntk}\n\\long\\def\\ettl@ifntk#1/#2/#3{#1\\long\\def\\ettl@ifntk##1{\\ettl@char{##1}\n   \\2{\\ifx##1#2\\1\\else\\2\\fi}//{##1}}\\ettl@ifntk}\n\\long\\def\\ettl@ifnspace#1#2/#3/#4 {#2\\futurelet\\@let@token#1}\n\n%% \\futuredef[list of allowed tokens]{ command }{ commands to expand after }\n%% \\futuredef*[list of allowed tokens]{ command }{ commands to expand after }\n%% \\futuredef=[list of allowed tokens]{ command }{ commands to expand after }\n%% \\futuredef*=[list of allowed tokens]{ command }{ commands to expand after } (or \\futuredef=*[...] )\n\\newrobustcmd*\\futuredef{\\begingroup\\ettl@futdef\\ettl@futuredef\\detokenize}\n\\protected\\def\\ettl@futdef#1#2{\\@ifstar%\n      {\\ettl@futdef\\ettl@futured@f#2}\n      {\\@ifchar={\\ettl@futdef#1\\unexpanded}\n                {\\@testopt{\\ettl@futur@def#1#2}{}}}}\n\\long\\def\\ettl@futur@def#1#2[#3]{%\n   \\csname ettl@\\ifcat $\\detokenize{#3}$1\\else2\\fi of2\\endcsname\n   {\\let \\ettl@x \\@empty \\letcs \\ettl@futur@def@collect{\\@gobblescape#1@collectall}}%\n   {\\def \\ettl@x {#3}\\edef \\ettl@y {#2{#3}}%\n   \\ifx\\ettl@x\\ettl@y      \\let\\ettl@y\\@gobble\n   \\else \\ifx#2\\unexpanded \\let\\ettl@y\\@gobble\n   \\else             \\def\\ettl@y{\\edef\\ettl@x}%\n   \\fi\\fi\\ettl@y{\\detokenizeChars{#3}}%\n   \\letcs\\ettl@futur@def@collect{\\@gobblescape#1@collect}}%\n   \\expandafter#1\\expandafter#2\\expandafter{\\ettl@x}}\n\\long\\def\\ettl@futuredef#1#2#3#4{% #1=detokenize #2=list, #3=macro result, #4=code-next\n   \\def \\ettl@futuredef@loop{\\ettl@futuredef@test{}}%\n   \\long \\def \\ettl@futuredef@test##1{%\n      \\ifcat\\noexpand\\ettl@x\\bgroup\\ettl@futuredef@end{}\\else\n      \\ifcat\\noexpand\\ettl@x\\egroup\\ettl@futuredef@end{}\\else\n      \\ifcat\\noexpand\\ettl@x\\ettl@sptoken\\ettl@futuredef@space#1\\else\n      \\ettl@futur@def@collect#1\\fi\\fi\\fi/Net/{#2}{##1}}%\n   \\long \\def \\ettl@futuredef@end##1##2/Net/##3##4{##2\\endgroup\\def#3{##4}#4##1}%\n   \\futurelet \\ettl@x \\ettl@futuredef@loop}\n\\long\\def\\ettl@futuredef@collect#1#2/Net/#3#4#5{#2%\n   \\ifcat\\noexpand#5\\relax \\ettl@futuredef@filt\\unexpanded\n   \\else \\ettl@futuredef@filt#1\n   \\fi{#5}{#3}\n   {\\def\\ettl@futuredef@loop{\\ettl@futuredef@test{#4#5}}\\futurelet\\ettl@x\\ettl@futuredef@loop}\n   {\\ettl@futuredef@end{#5}/Net/{}{#4}}/Net/}\n\\long\\def\\ettl@futuredef@space#1#2/Net/#3#4 {%\n   \\ettl@futur@def@collect#1#2/Net/{#3}{#4}{ }}\n\\long\\def\\ettl@futuredef@collectall#1#2/Net/#3#4#5{#2%\n   \\def\\ettl@futuredef@loop{\\ettl@futuredef@test{#4#5}}\\futurelet\\ettl@x\\ettl@futuredef@loop}\n\\long\\def\\ettl@futur@def@filt#1#2{% #1=token to check, #2=allowed list\n   \\long\\def\\ettl@futdef@filt##1#1##2##3/##4##5##6/Net/{##5}%\n   \\ettl@futdef@filt#2#1//}\n\\long\\def\\ettl@futuredef@filt#1#2\\fi#3#4{\\fi % #1=detokenize/unexpanded, #2=discard, #3=token,#4=allowed list\n   \\expandafter\\ettl@futur@def@filt\\expandafter{#1{#3}}{#4}}\n\\long\\def\\ettl@futured@f#1#2#3#4{% #1=detokenize #2=list, #3=macro result, #4=code-next\n   \\let \\ettl@y \\@undefined\n   \\def \\ettl@futured@f@loop{\\ettl@futured@f@test{}}%\n   \\long \\def \\ettl@futured@f@test##1{%\n      \\ifcat\\noexpand\\ettl@x\\bgroup\\ettl@futured@f@end\\else\n      \\ifcat\\noexpand\\ettl@x\\egroup\\ettl@futured@f@end\\else\n      \\ifcat\\noexpand\\ettl@x\\ettl@sptoken\\ettl@futured@f@space#1\\else\n      \\ettl@futur@def@collect#1\\fi\\fi\\fi/Net/{##1}{#2}{}}%\n   \\long \\def \\ettl@futured@f@end##1/Net/##2##3##4{##1\\endgroup\\def#3{##2}#4##4}%\n   \\futurelet \\ettl@x \\ettl@futured@f@loop}\n\\long\\def\\ettl@futured@f@space#1#2/Net/#3#4#5 {%\n      \\ettl@futur@def@collect#1#2/Net/{#3}{#4}{#5}{ }}\n\\long\\def\\ettl@futured@f@collect#1#2/Net/#3#4#5#6{#2%\n   \\ifcat\\noexpand\\ettl@x\\relax \\ettl@futuredef@filt\\unexpanded\n   \\else \\ettl@futuredef@filt#1\n   \\fi{#6}{#4}\n   {\\let \\ettl@y \\@undefined \\ettl@futured@f@append/Net/{#3}{}{#6}}%\n   {\\ettl@futured@f@try@expand{#3}\\ettl@futured@f@end{#6}}/Net/}\n\\long\\def\\ettl@futured@f@collectall#1#2/Net/#3#4#5#6{#2%\n   \\ettl@futured@f@try@expand{#3}\\ettl@futured@f@append{#6}}\n\\long\\def\\ettl@futured@f@space#1#2/Net/#3#4#5 {%\n      \\ettl@futur@def@collect#1#2/Net/{#3}{#4}{#5}{ }}\n\\long\\def\\ettl@futured@f@try@expand#1#2#3{%\n   \\expandafter\\ifx\\noexpand\\ettl@x\\ettl@x\n      \\let\\ettl@y=#2%\n   \\else\\ettl@futured@f@CheckSpecials{#3}%\n      {\\let \\ettl@y=#2}%\n      {\\ifx\\ettl@x\\ettl@y \\let \\ettl@y \\ettl@futured@f@end\\else\n       \\let \\ettl@y \\ettl@futured@f@expand\\fi}%\n   \\fi\\ettl@y/Net/{#1}{}{#3}}\n\\long\\def\\ettl@futured@f@expand/Net/#1#2#3{\\let\\ettl@y\\ettl@x\n   \\expandafter\\futurelet\\expandafter\\ettl@x\\expandafter\\ettl@futured@f@loop#3}\n\\long\\def\\ettl@futured@f@CheckSpecials#1{\\ifintokslist{#1}{%\n   \\@undefined\\if\\ifcat\\ifnum\\ifdim\\ifodd%\n   \\ifvmode\\ifhmode\\ifmmode\\ifinner\\ifvoid\\ifhbox\\ifvbox%\n   \\ifx\\ifeof\\iftrue\\iffalse\\ifcase\\ifdefined\\ifcsname\\iffontchar%\n   \\else\\fi\\or}}\n\\def\\ettl@futured@f@append/Net/#1#2#3{%\n   \\def\\ettl@futured@f@loop{\\ettl@futured@f@test{#1#3}}%\n   \\futurelet\\ettl@x\\ettl@futured@f@loop}%\n\n%----------------------------------------------------------------------------\n\n%% \\naturalloop [ auxiliary commands (default \\do) ]{ n times }{ argument }\n\\newcommand\\naturalloop[1]{\\FE@testopt{#1}\\ettl@naturalloop{\\do}}\n\\def\\ettl@naturalloop[#1]#2#3{%\n   \\ifnum\\numexpr#2>0 \\expandafter\\@swaparg\\expandafter{\\romannumeral-`\\q#1[0]{#3}{#3}}%\n         {\\ettl@naturall@@p[{#1}]{#2-1}{0}{#3}}\n   \\else\\@swap{\\unexpanded{#3}}%\n   \\fi}\n\\def\\ettl@naturall@@p[#1]#2#3#4#5#6\\fi{\\fi%\n   \\ifnum\\numexpr#2>0 \\expandafter\\@swaparg\\expandafter{\\romannumeral-`\\q%\n      \\expandafter\\@swap\\expandafter{\\expandafter[\\number\\numexpr#3+1]}{#1}{#4}{#5}}%\n      {\\ettl@naturall@@p[{#1}]{#2-1}{#3+1}{#4}}%\n   \\else\\@swap{\\unexpanded{#5}}%\n   \\fi}\n\n%% \\ifinttokslist{ item }{ list of tokens }{ true }{ false }\n\\newcommand\\ifintokslist[2]{\\romannumeral\\csname rmn@%\n   \\expandafter\\ettl@nbk\\romannumeral\\ettl@dosinglelist{\\ettl@ifintokslist{#1}}{#2}\\z@//%\n   {first}{second}//oftwo\\endcsname}\n\\long\\def\\ettl@ifintokslist#1#2{\\ifx#1#2\\ettl@breakloop\\z@\\fi}\n\\newcommand\\ifincharlist[2]{\\romannumeral\\csname rmn@%\n   \\expandafter\\ettl@nbk\\romannumeral\\ettl@dosinglelist{\\ettl@ifincharlist{#1}}{#2}\\z@//%\n   {first}{second}//oftwo\\endcsname}\n\\long\\def\\ettl@ifincharlist#1#2{\\ettl@ifchar{#1}{#2}{\\ettl@breakloop\\z@}{}}\n\\long\\def\\ettl@dosinglelist#1#2{\\ettl@nbk#2//%\n      {\\ettl@dosinglelist@loop{#1}#2//{\\ettl@dosinglelist@loop{#1}}{\\ettl@breakloop{}}}\n      {\\ettl@breakloop{}}///EndList/}\n\\long\\def\\ettl@dosinglelist@loop#1#2#3#4/#5#6#7/EndList/{%\n      #1{#2}#6{#3}#4//{#6}{#7}/EndList/}\n\n%% \\gettokslistindex { item }{ list of tokens }\n%% \\gettokslistcount { item }{ list of tokens }\n%% \\gettokslisttoken { item }{ list of tokens }\n\\newcommand\\gettokslistindex[2]{\\number\\ifnotempty{#2}{\\ettl@nbk#1//%\n   {\\ExpandAftercmds{\\ettl@setresult 2of3><}{\\ettl@getsinglelist{\\ettl@ifx{#1}}{#2}}}\n   {-1}//}{-1}}\n\\newcommand\\getcharlistindex[2]{\\number\\ifnotempty{#2}{\\ettl@nbk#1//%\n   {\\ExpandAftercmds{\\ettl@setresult 2of3><}{\\ettl@getsinglelist{\\ettl@ifchar{#1}}{#2}}}\n   {-1}//}{-1}}\n\\newcommand\\gettokslistcount[1]{\\number\\ifnotempty{#1}%\n   {\\ExpandAftercmds{\\ettl@setresult 1of3><}{\\ettl@getsinglelist{\\ettl@ifx{\\\\}}{#1}}}\n   0}\n\\newcommand\\getcharlistcount[1]{}%\n\\let\\getcharlistcount=\\gettokslistcount\n\\newcommand\\gettokslisttoken[2]{\\ifnotempty{#2}{\\ettl@nbk#1//%\n   {\\ExpandAftercmds{\\ettl@setresult 3of3><}{\\ettl@getsinglelist{\\ettl@ifx{#1}}{#2}}}\n   {}//}{}}\n\\newcommand\\getcharlisttoken[2]{\\ifnotempty{#2}{\\ettl@nbk#1//%\n   {\\ExpandAftercmds{\\ettl@setresult 3of3><}{\\ettl@getsinglelist{\\ettl@ifchar{#1}}{#2}}}\n   {}//}{}}\n\\long\\def\\ettl@getsinglelist#1#2{\\ettl@singlelist@loop{-1}{-1}{}#2//%\n         {\\ettl@expandafthree\\ettl@singlelist@loop#1}%\n         {\\expandafter\\ettl@singlelist@result\\@thirdofthree}/EndList/}\n\\long\\def\\ettl@singlelist@loop#1#2#3#4#5/#6#7#8/EndList/{%\n      #7{#4}\n         {{#1+1}{#2+1+0*(0}{#4}}\n         {{#1+1}{#2+1}{#3}}#5//{#7}{#8}/EndList/}\n   %   \\csname @#1#5{first}{second}oftwo\\endcsname\n   %      {#8{#1}{#2+1}{#3+1+0*(0}{#5}#6//#8#9}\n   %      {#8{#1}{#2+1}{#3+1}{#4}#6//#8#9}/EndList/}\n\n\\def\\ettl@singlelist@result#1#2#3#4/EndList/{\\ExpandNextTwo\\@swaptwo%\n      {\\number\\numexpr\\ifempty{#3}{-1}{#2)}}{\\number\\numexpr#1}{#3}}\n\\def\\ettl@setresult#1of#2>#3<{\\ettl@nbk #3//%\n      {\\ettl@set@result#1of#2>#3<}\n      {\\csname ettl@#1of#2\\endcsname}//}\n\\def\\ettl@set@result#1of#2>#3<#4{\\ifdefcount{#3}\n   {#3=\\csname ettl@#1of#2\\endcsname#4}\n   {\\expandafter\\edef\\noexpand#3{\\csname ettl@#1of#2\\endcsname{#4}}}%\n}\n\n%% \\DeclareCmdListParser : general constructor for command-list parsers\n%% \\DeclareCmdListParser\\ParserName{separator}\n\\newrobustcmd\\DeclareCmdListParser[3][\\global]{\\@ifdefinable{#2}{\\begingroup\n      \\protected\\def\\ettl@defcmdparser##1{%\n         \\edef\\ettl@defcmdparser{\\endgroup\\ettl@defcmdparser\n            {#1}{\\noexpand#2}{\\unexpanded{#3}}\n            {\\noexpandcs{##1->start}}\n            {\\noexpandcs{##1->loop}}\n            {\\noexpandcs{##1->loop+}}\n            {\\noexpandcs{for##1}}%\n         }\\ettl@defcmdparser\n      }\\expandafter\\ettl@defcmdparser\\expandafter{\\romannumeral-`\\q\\@gobblescape#2}}}\n\\protected\\long\\def\\ettl@defcmdparser#1#2#3#4#5#6#7{%#1=global,#2=command,#3=sep,#4=start,#5=loop,#6=loop+\n   #1\\long\\def#4##1##2[##3]##4{% ##1=case, ##2=expandafter??? , ##3=do, ##4=list\n      ##2{##4}% ifiscs or @thirdofthree\n         {\\expandafter\\@swaparg\\expandafter{##4}{#4{##1}\\@thirdofthree[{##3}]}}\n         {\\ettl@nbk##4//%\n            {\\ifcase##1 \\ettl@or\\@swaplast{\\number\\numexpr#60{\\ettl@lst@count}}#6%\n               \\or      \\ettl@or\\@swaplast{#60{\\ettl@lst@getitem{##3}}}#6%\n               \\or      \\ettl@or\\@swaplast{#5{##3}}#5%\n               \\or      \\ettl@fi\\@swaplast{#60{##3}}#6%\n               \\fi{##4#3//}{\\ettl@breakloop{\\ifx##10\\expandafter\\relax\\fi}}%\n            }{\\ettl@breakloop{}}///EndList/}}%\n   #1\\long\\def#5##1##2#3##3##4/##5##6##7/EndList/{%\n      \\ifcat $\\detokenize{##2}$\\expandafter\\@gobbletwo\\fi\\@firstofone{##1{##2}}%\n      ##6{##1}##3##4//{##6}{##7}/EndList/}\n   #1\\long\\def#6##1##2##3#3##4##5/##6##7##8/EndList/{%\n      \\ifcat $\\detokenize{##3}$\\expandafter\\@gobbletwo\\fi\\@firstofone{##2[##1]{##3}}%\n      \\expandafter##7\\expandafter{\\number\\numexpr##1+1}{##2}##4##5//{##7}{##8}/EndList/}\n   #1\\protected\\def#7{\\@ifchar*%\n      {\\@ifchar+{\\ettl@forloop{\\expandafter#2\\expandafter*\\expandafter+}{[####1]####2}}\n                {\\ettl@forloop{\\expandafter#2\\expandafter*}{####1}}}\n      {\\@ifchar+{\\@ifchar*%\n                     {\\ettl@forloop{\\expandafter#2\\expandafter*\\expandafter+}{[####1]####2}}\n                     {\\ettl@forloop{\\expandafter#2\\expandafter+}{[####1]####2}}}\n                {\\ettl@forloop{\\expandafter#2}{####1}}}}\n   #1\\def#2{\\ettl@lst@modif#423\\ifiscs}}\n\\long\\def\\ettl@lst@getitem#1[#2]#3{%\n   \\ifnum\\numexpr#1<0 \\@swap{\\breakloop{}}\\fi\n   \\ifnum\\numexpr#1=#2 \\@swap{\\breakloop{#3}}\\fi}\n\\long\\def\\ettl@lst@count[#1]#2{+\\ettl@nbk#2//10//}\n\\long\\def\\ettl@lst@modif#1#2#3#4#5{\\FE@modifiers{*+![}{#5}%\n   {\\ettl@lst@modif{#1}#2#3\\@thirdofthree}%    * case\n   {\\ettl@lst@modif{#1}#3#2{#4}}%  + (case 3/default 2)\n   {\\ettl@lst@modif{#1}00{#4}}% ! (case 0)\n   {\\ettl@lst@opt{#1}{#2}{#4}#5}% [   (option)\n   {\\ettl@lst@opt{#1}{#2}{#4}[\\do]}}% (default option)\n\\long\\def\\ettl@lst@opt#1#2#3[#4]{%\n   \\expandafter#1\\expandafter{\\number\\ifnum#2=0 0\\else\\ifstrnum{#4}{1}{#2}\\fi}{#3}[{#4}]}\n\\long\\def\\ettl@breakloop#1#2/EndList/{#1}\n\\let\\breakloop\\ettl@breakloop\n\\globcount\\ettl@for@nested\n\\long\\def\\ettl@forloop#1#2#3\\do{%\n   \\global\\advance\\ettl@for@nested\\@ne\\relax\n   \\csdef{ettl@for@loop\\the\\ettl@for@nested}{%\n      #1\\expandafter[\\csname ettl@for@do\\the\\ettl@for@nested\\endcsname]{#3}%\n      \\csundef{ettl@for@do\\the\\ettl@for@nested}%\n      \\csundef{ettl@for@loop\\the\\ettl@for@nested}%\n      \\global\\advance\\ettl@for@nested\\m@ne\\relax}%\n   \\expandafter\\afterassignment\\csname ettl@for@loop\\the\\ettl@for@nested\\endcsname\n   \\long\\csdef{ettl@for@do\\the\\ettl@for@nested}#2%\n}% \\ettl@for@nested\n\n\n%% \\csvloop[\\command]\\csvListMacro\n%% \\csvloop*[\\command]{item,item,item}\n\\DeclareCmdListParser\\csvloop{,}\n\\begingroup\\catcode`\\|=3\n\\DeclareCmdListParser\\listloop{|}% global declaration\n\\endgroup\n\\DeclareCmdListParser\\toksloop{}\n\n%% \\forcsvloop\\csvListMacro\\do{...#1...}\n%% \\forcsvloop*{item,item,item,...}\\do{...#1...}\n\n%% \\csvlistadd \\csvListMacro { item }\n%% \\csvlist(g|e|x)add \\csvListMacro { item }\n\\providerobustcmd\\csvlistadd[2]{\\ettl@nbk#2//{\\appto#1{#2,}}{}//}\n\\providerobustcmd\\csvlistgadd[2]{\\ettl@nbk#2//{\\gappto#2{#2,}}{}//}\n\\providerobustcmd\\csvlisteadd[2]{\\begingroup \\protected@edef#1{#2}%\n   \\expandafter\\ettl@nbk#1//{\\expandafter\\endgroup\n      \\expandafter\\appto\\expandafter#1\\expandafter{#1,}}\\endgroup//}\n\\providerobustcmd\\csvlistxadd[2]{\\begingroup \\protected@edef#1{#2}%\n   \\expandafter\\ettl@nbk#1//{\\expandafter\\endgroup\n      \\expandafter\\gappto\\expandafter#1\\expandafter{#1,}}\\endgroup//}\n\n%% \\csvtolist  [\\ListMacro] \\csvListMacro\n%% \\csvtolist* [\\ListMacro] {item,item,item}\n% Recommended use: \\edef\\ListMacro{\\csvtolist{item,item,item}}\n\\newcommand\\csvtolist[1]{\\FE@ifstar{#1}{\\ettl@convertlist{{\\csvloop*}\\ettl@do@csvtolist}}\n                                       {\\ettl@convertlist{\\csvloop\\ettl@do@csvtolist}}}\n\\long\\def\\ettl@convertlist#1#2{\\FE@testopt{#2}{\\ettl@convert@list#1}{}}\n\\long\\def\\ettl@convert@list#1#2[#3]#4{\\ettl@nbk#3//%\n      {\\edef#3{#1[#2]{#4}}}\n      {#1[#2]{#4}}//}\n\\begingroup\\catcode`\\|=3% etb catcode\n\\long\\gdef\\ettl@do@csvtolist#1{\\unexpanded{#1}|}\n\\endgroup\n\n%% \\listtocsv  [\\csvListMacro] \\ListMacro\n%% \\listtocsv* [\\csvListMacro] { expanded List }\n% Recommended use: \\edef\\csvListMacro{\\listtocsv\\ListMacro}\n\\newcommand\\listtocsv[1]{\\FE@ifstar{#1}{\\ettl@convertlist{{\\listloop*}\\ettl@do@listtocsv}}\n                                       {\\ettl@convertlist{\\listloop\\ettl@do@listtocsv}}}\n\\long\\def\\ettl@do@listtocsv#1{\\unexpanded{#1,}}\n%% \\tokstolist [\\ListMacro] { \\toksListMacro   /  token token token }\n\\newcommand\\tokstolist[1]{\\FE@ifstar{#1}{\\ettl@convertlist{{\\toksloop*}\\ettl@do@tokstolist}}\n                                        {\\ettl@convertlist{\\toksloop\\ettl@do@tokstolist}}}\n\\begingroup\\catcode`\\|=3% etb catcode\n\\long\\gdef\\ettl@do@tokstolist#1{\\unexpanded{#1}|}\n\\endgroup\n\n%% \\csvtolistadd \\ListMacro \\csvListMacro\n%% \\csvtolistadd* \\ListMacro {item,item,item}\n\\newrobustcmd*\\csvtolistadd{\\@ifstar{\\ettl@csvtolistadd*}{\\ettl@csvtolistadd{}}}\n\\long\\def\\ettl@csvtolistadd#1#2#3{\\eappto#2{\\csvtolist#1[]{#3}}}\n\n%% \\tokstolistadd \\ListMacro { \\toksListMacro  /  token token token }\n\\newrobustcmd*\\tokstolistadd{\\@ifstar{\\ettl@tokstolistadd*}{\\ettl@tokstolistadd{}}}\n\\long\\def\\ettl@tokstolistadd#1#2#3{\\eappto#2{\\tokstolist#1[]{#3}}}\n\\newrobustcmd\\ettl@RemoveInList[2]{\\begingroup\n   \\def\\ettl@RemoveInList##1{%\n      \\edef\\ettl@RemoveInList####1####2{%\n         \\ettl@Rem@veInList{####1}####2\\noexpandcs{##1->remove}\\noexpandcs{##1->result}%\n      }\\ettl@RemoveInList{#1}#2%\n   }\\expandafter\\ettl@RemoveInList\\expandafter{\\romannumeral-`\\q\\@gobblescape#2}}\n\\protected\\long\\def\\ettl@Rem@veInList#1#2#3#4#5#6#7#8{%\n   \\long\\def#3[##1]##2#5#8#5##3##4/##5##6##7/EndList/{##6[##1+1]##2#5##3##4//##6##7/EndList/}%\n   \\ifnotempty{#5}%% special case if no separator\n      {\\long\\def#4[##1]#5##2#5#5##3//##4/EndList/{\\unexpanded{#1\\def#7{##2#5}}%\n               \\ettl@nbk#6//{\\ettl@setresult 1of1>#6<{\\number\\numexpr##1-1\\relax}}{}//}}%\n      {\\long\\def#4[##1]##2//##3/EndList/{\\unexpanded{#1\\def#7{##2}}%\n               \\ettl@nbk#6//{\\ettl@setresult 1of1>#6<{\\number\\numexpr##1-1\\relax}}{}//}}%\n   \\long\\def#2##1{#3[0]#5##1#5#5#8#5//#3#4/EndList/}%\n   \\edef#7{\\endgroup\\expandafter#2\\expandafter{#7}}#7}\n\\def\\ettl@gobble@relax#1\\relax{}\n\n%% \\listdel  \\listMacro { item }\n%% \\listgdel \\listMacro { item }\n%% \\listedel \\listMacro { item }\n%% \\listxdel \\listMacro { item }\n\\edef\\ettl@restore@catcode{\\catcode124 \\the\\catcode124}% |=124\n\\catcode`\\|=3\n\\newrobustcmd\\listdel[1][]{\\ettl@RemoveInList{}\\listdel|{#1}}\n\\newrobustcmd\\listgdel[1][]{\\ettl@RemoveInList\\global\\listdel|{#1}}\n\\newrobustcmd\\listedel[1][]{\\ettl@listedel{}\\listdel|{#1}}\n\\newrobustcmd\\listxdel[1][]{\\ettl@listedel\\global\\listdel|{#1}}\n\\ettl@restore@catcode\\undef\\ettl@restore@catcode\n\\newrobustcmd\\ettl@listedel[6]{\\begingroup\\protected@edef#5{#6}\\expandafter\\endgroup\n   \\expandafter\\@swaparg\\expandafter{#5}{\\ettl@RemoveInList#1#2{#3}{#4}#5}}\n\n%% \\csvdel  \\listMacro { item }\n%% \\csvgdel \\listMacro { item }\n%% \\csvedel \\listMacro { item }\n%% \\csvxdel \\listMacro { item }\n\\newrobustcmd\\csvdel[1][]{\\ettl@RemoveInList{}\\csvdel,{#1}}\n\\newrobustcmd\\csvgdel[1][]{\\ettl@RemoveInList\\global\\csvdel,{#1}}\n\\newrobustcmd\\csvedel[1][]{\\ettl@listedel{}\\csvdel,{#1}}\n\\newrobustcmd\\csvxdel[1][]{\\ettl@listedel\\global\\csvdel,{#1}}\n\n%% \\toksdel  \\listMacro { item }\n%% \\toksgdel \\listMacro { item }\n%% \\toksedel \\listMacro { item }\n%% \\toksxdel \\listMacro { item }\n\\newrobustcmd\\toksdel[1][]{\\ettl@RemoveInList{}\\toksdel{}{#1}}\n\\newrobustcmd\\toksgdel[1][]{\\ettl@RemoveInList\\global\\toksdel{}{#1}}\n\\newrobustcmd\\toksedel[1][]{\\ettl@listedel{}\\toksdel{}{#1}}\n\\newrobustcmd\\toksxdel[1][]{\\ettl@listedel\\global\\toksdel{}{#1}}\n\n\n%% \\getlistindex[\\indexmacro]{ \\Listmacro }\n%% \\getlistindex*[\\indexmacro]{ expanded List }\n\\newrobustcmd\\ettl@getlistindex[6]{% #1=result, #2=\\expandafter, #3=loop macro, #4=separator, #5=list of list macro, #6=item\n   \\begingroup\\long\\def\\ettl@getlistindex##1#4#6#4##2/EndList/{\\endgroup\n   \\ExpandAftercmds{\\ettl@setresult 1of1>#1<}{\\ettl@nbk##2//{#3*!{##1}}{-1}//}%\n   }#2\\ettl@getlistindex#5#4#6#4/EndList/}\n\\newrobustcmd\\getlistindex{\\@ifstar\n   {\\@testopt{\\ettl@get@listindex\\relax}{}}\n   {\\@testopt{\\ettl@get@listindex\\expandafter}{}}}\n\\begingroup\\catcode`\\|=3% etb catcode\n\\protected\\long\\gdef\\ettl@get@listindex#1[#2]#3#4{%\n   \\ifx#1\\relax\n      \\ettl@getlistindex{#2}{}\\listloop|{#4}{#3}%\n   \\else \\ifiscs{#4}\n      {\\ettl@getlistindex{#2}#1\\listloop|{#4}{#3}}\n      {\\ettl@getlistindex{#2}\\listloop|{#4}{#3}}%\n   \\fi}\n\\endgroup%\\catcode group\n\n%\\getcsvlistindex [\\result]{ item }{ csvlistmacro }\n%\\getcsvlistindex*[\\result]{ item }{ item,item,item }\n\\newrobustcmd\\getcsvlistindex{\\@ifstar\n   {\\@testopt{\\ettl@get@csvlistindex\\relax}{}}\n   {\\@testopt{\\ettl@get@csvlistindex\\expandafter}{}}}\n\\protected\\long\\gdef\\ettl@get@csvlistindex#1[#2]#3#4{%\n   \\ifx#1\\relax\n      \\ettl@getlistindex{#2}{}\\csvloop,{#4}{#3}%\n   \\else \\ifiscs{#4}\n      {\\ettl@getlistindex{#2}#1\\csvloop,{#4}{#3}}\n      {\\ettl@getlistindex{#2}\\csvloop,{#4}{#3}}%\n   \\fi}\n\\def\\ettl@if@inlist#1#2{%#1=macro,#2=separator\n\\newrobustcmd*#1{\\@ifstar{\\ettl@ifinlist{#2}{}}{\\ettl@ifinlist{#2}\\expandafter}}}\n\\def\\ettl@xif@inlist#1#2{%\n\\newrobustcmd*#1{\\@ifstar{\\ettl@xifinlist{#2}{}}{\\ettl@xifinlist{#2}\\expandafter}}}\n\\protected\\long\\def\\ettl@ifinlist#1#2#3#4{\\begingroup\n   \\def\\ettl@tempa##1#1##2#1/EndList/{\\endgroup\\ifnotblank{##2}%\n   }#2\\ettl@tempa#2#1#3#1#4#1/EndList/}\n\\protected\\long\\def\\ettl@xifinlist#1#2#3#4{\\begingroup\n   \\protected@edef\\ettl@tempa{\\endgroup\\ettl@ifinlist{#1}{#2}{#3}{#4}%\n   }\\ettl@tempa}\n\n\n%% \\ifincsvlist{ item }{ csvlistmacro }{ true }{ false }\n%% \\ifincsvlist*{ item }{ item,item,item,... }{ true }{ false }\n%% \\xifincsvlist{ item }{ csvlistmacro }{ true }{ false }\n%% \\xifincsvlist*{ item }{ item,item,item,... }{ true }{ false }\n\\ettl@if@inlist\\ifincsvlist{,}\n\\ettl@xif@inlist\\xifincsvlist{,}\n\\undef\\ettl@if@inlist\n\\undef\\ettl@xif@inlist\n\n%% \\interval{ number }{ sorted comma separated list of numbers }\n\\newcommand\\interval[2]{\\romannumeral-`\\q%\n   \\ExpandNext{\\avoidvoid[\\csvloop!{#2}]}{\\csvloop+[\\ettl@do@interval{#1}]{#2}}}\n\\def\\ettl@do@interval#1[#2]#3{\\ifdim#1\\p@<#3\\p@ \\@swap{\\breakloop{#2}}\\fi}\n\n%% \\interplin{ number }{ sorted comma separated list of numbers }{ comma separated liist of numbers }\n\\newcommand\\locinterplin[3]{\\romannumeral-`\\q\n   \\unless\\ifnum\\numexpr(\\csvloop!{#2})-(\\csvloop!{#3})=0\n      \\PackageError{etextools}{Using \\string\\locinterplin\\space the lists in argument 1 and 2\\MessageBreak\n      must have the same number of elements}\n      {You're in trouble here and I cannot proceed...}\n   \\fi\n   \\ExpandNextTwo{\\ettl@locinterplin{#1}{#3}{#2}}{\\interval{#1}{#2}}{\\csvloop!{#2}}}\n\\begingroup\\catcode`\\/ 12%\n\\gdef\\ettl@locinterplin#1#2#3#4#5{%\n   \\ifnum#4=0 \\csvloop[#4]{#2}%\n   \\else\\ifnum#4=#5 \\expandafter\\csvloop\\expandafter[\\number\\numexpr#5-1]{#2}%\n   \\else\\ifdim#1\\p@=\\expandafter\\csvloop\\expandafter[\\number\\numexpr#4-1]{#3}\\p@\n      \\expandafter\\csvloop\\expandafter[\\number\\numexpr#4-1]{#2}%\n   \\else\\strip@pt\\dimexpr%\n      \\expandafter\\csvloop\\expandafter[\\number\\numexpr#4-1]{#2}\\p@+%\n         (#1\\p@-\\expandafter\\csvloop\\expandafter[\\number\\numexpr#4-1]{#3}\\p@)*%\n         (\\expandafter\\csvloop\\expandafter[\\number\\numexpr#4-1]{#2}-\\csvloop[#4]{#2})/%\n         (\\expandafter\\csvloop\\expandafter[\\number\\numexpr#4-1]{#3}-\\csvloop[#4]{#3})\\relax\n   \\fi\\fi\\fi}\n\\endgroup% catcode group\n\\DeclareOption{etoolbox}{%\n\\renewcommand\\ifblank[3]{\\ettl@nbk #1//{#2}{#3}//}\n\\renewcommand\\ifdef[1]{\\csname @\\ifdefined#1first\\else second\\fi oftwo\\endcsname}\n\\renewcommand\\ifcsdef[1]{\\csname @\\ifcsname#1\\endcsname first\\else second\\fi oftwo\\endcsname}\n\\renewcommand\\ifundef[1]{\\csname @%\n   \\ifdefined#1\\ifx#1\\relax first\\else second\\fi\\else first\\fi oftwo\\endcsname}\n\\renewcommand\\ifcsundef[1]{\\csname @%\n   \\ifcsname#1\\endcsname\\expandafter\\ifx\\csname#1\\endcsname\\relax\n      first\\else second\\fi\\else first\\fi oftwo\\endcsname}\n\\edef\\ifdefmacro#1{\\unexpanded{\\csname @%\n     \\expandafter\\ettl@ifdefmacro\\meaning}#1\\detokenize{macro:}/oftwo\\endcsname}\n\\edef\\ettl@ifdefmacro{%\n   \\def\\noexpand\\ettl@ifdefmacro##1\\detokenize{macro:}##2/{\\noexpand\\ettl@nbk##2//{first}{second}//}%\n}\\ettl@ifdefmacro\n\\long\\edef\\ifcsmacro#1{\\unexpanded{\\csname @%\n   \\expandafter\\expandafter\\expandafter\\ettl@ifdefmacro\\meaningcs}{#1}\\detokenize{macro:}/oftwo\\endcsname}\n\\renewcommand\\ifdefparam[1]{\\csname @%\n   \\ettl@expandaftwo\\ettl@nbk\\expandafter\\ettl@params@meaning\\meaning#1///{first}{second}//oftwo\\endcsname}\n\\renewcommand\\ifcsparam[1]{\\csname @%\n   \\expandafter\\expandafter\\expandafter\\ettl@nbk\\parameters@meaningcs{#1}//{first}{second}//oftwo\\endcsname}\n\\renewcommand\\ifnumcomp[3]{\\csname @%\n   \\ifnum\\numexpr#1#2\\numexpr#3 first\\else second\\fi oftwo\\endcsname}\n}% etoolbox option - not to be used - experimental\n\\ProcessOptions*\\relax\n\n\\endinput\n%%\n%% End of file `etextools.sty'.\n"
  },
  {
    "path": "user-dev-guide/etoolbox.sty",
    "content": "% $Id: etoolbox.sty,v 2.0a 2010/09/12 11:42:38 lehman stable $\n\n% Copyright (c) 2007-2010 Philipp Lehman.\n%\n% Permission is granted to copy, distribute and/or modify this\n% software under the terms of the LaTeX Project Public License\n% (LPPL), version 1.3.\n%\n% The LPPL maintenance status of this software is\n% 'author-maintained'.\n%\n% This software is provided 'as is', without warranty of any kind,\n% either expressed or implied, including, but not limited to, the\n% implied warranties of merchantability and fitness for a\n% particular purpose.\n\n\\def\\etb@rcsid$#1: #2 #3 #4 #5${#4 v#3}\n\n\\NeedsTeXFormat{LaTeX2e}\n\\ProvidesPackage{etoolbox}\n[\\etb@rcsid $Id: etoolbox.sty,v 2.0a 2010/09/12 11:42:38 lehman stable $\n e-TeX tools for LaTeX]\n\n\\begingroup\n\\@ifundefined{eTeXversion}\n  {\\PackageError{etoolbox}\n     {Not running under e-TeX}\n     {This package requires e-TeX. Try compiling the document\n      with\\MessageBreak 'elatex' instead of 'latex'. When using\n      pdfTeX, try 'pdfelatex'\\MessageBreak instead of 'pdflatex'.\n      This is a fatal error. I'm aborting now.}%\n   \\aftergroup\\endinput}\n  {}\n\\endgroup\n\n\\RequirePackage{etex}\n\n\\def\\etb@catcodes{\\do\\&\\do\\|\\do\\:\\do\\-\\do\\=\\do\\<\\do\\>}\n\\def\\do#1{\\catcode\\number`#1=\\the\\catcode`#1\\relax}\n\\edef\\etb@catcodes{\\etb@catcodes}\n\\let\\do\\noexpand\n\\AtEndOfPackage{\\etb@catcodes\\undef\\etb@catcodes}\n\n\\catcode`\\&=3\n\\catcode`\\|=3\n\\@makeother\\:\n\\@makeother\\-\n\\@makeother\\=\n\\@makeother\\<\n\\@makeother\\>\n\n\\protected\\def\\etb@error{\\PackageError{etoolbox}}\n\\protected\\def\\etb@info{\\PackageInfo{etoolbox}}\n\\newcount\\etb@tempcnta\n\n% {<cstoken>}[<arguments>][<optarg default>]{<definition>}\n\n\\newcommand*{\\newrobustcmd}{}\n\\protected\\def\\newrobustcmd{\\@star@or@long\\etb@new@command}\n\n\\def\\etb@new@command#1{\\@testopt{\\etb@newcommand#1}0}\n\n\\def\\etb@newcommand#1[#2]{%\n  \\@ifnextchar[%]\n    {\\etb@xargdef#1[#2]}\n    {\\ifx\\l@ngrel@x\\relax\n       \\let\\l@ngrel@x\\protected\n     \\else\n       \\protected\\def\\l@ngrel@x{\\protected\\long}%\n     \\fi\n     \\@argdef#1[#2]}}\n\n\\long\\def\\etb@xargdef#1[#2][#3]#4{%\n  \\@ifdefinable#1{%\n    \\expandafter\\protected\n    \\expandafter\\def\n    \\expandafter#1%\n    \\expandafter{%\n      \\expandafter\\@testopt\n      \\csname\\string#1\\endcsname{#3}}%\n    \\expandafter\\@yargdef\\csname\\string#1\\endcsname\\tw@{#2}{#4}}}\n\n% {<cstoken>}[<arguments>][<optarg default>]{<definition>}\n\n\\newrobustcmd*{\\renewrobustcmd}{\\@star@or@long\\etb@renew@command}\n\n\\def\\etb@renew@command#1{%\n  \\ifundef#1\n     {\\etb@error{\\string#1 undefined}\\@ehc}\n     {}%\n  \\let\\@ifdefinable\\@rc@ifdefinable\n  \\etb@new@command#1}\n\n% {<cstoken>}[<arguments>][<optarg default>]{<definition>}\n\n\\newrobustcmd*{\\providerobustcmd}{\\@star@or@long\\etb@provide@command}\n\n\\def\\etb@provide@command#1{%\n  \\ifundef#1\n    {\\def\\reserved@a{\\etb@new@command#1}}\n    {\\def\\reserved@a{\\etb@renew@command\\reserved@a}}%\n  \\reserved@a}\n\n% {<csname>}\n\n\\newrobustcmd*{\\csshow}[1]{%\n  \\begingroup\\expandafter\\endgroup\n  \\expandafter\\show\\csname#1\\endcsname}\n\n% {<cstoken>}{<true>}{<false>}\n\n\\newcommand{\\ifdef}[1]{%\n  \\ifdefined#1%\n    \\expandafter\\@firstoftwo\n  \\else\n    \\expandafter\\@secondoftwo\n  \\fi}\n\n% {<cstoken>}{<true>}{<false>}\n\n\\newcommand{\\ifundef}[1]{%\n  \\ifdefined#1%\n    \\ifx#1\\relax\n      \\expandafter\\expandafter\n      \\expandafter\\@firstoftwo\n    \\else\n      \\expandafter\\expandafter\n      \\expandafter\\@secondoftwo\n    \\fi\n  \\else\n    \\expandafter\\@firstoftwo\n  \\fi}\n\n% {<csname>}{<true>}{<false>}\n\n\\newcommand*{\\ifcsdef}[1]{%\n  \\ifcsname#1\\endcsname\n    \\expandafter\\@firstoftwo\n  \\else\n    \\expandafter\\@secondoftwo\n  \\fi}\n\n% {<csname>}{<true>}{<false>}\n\n\\newcommand*{\\ifcsundef}[1]{%\n  \\ifcsname#1\\endcsname\n    \\expandafter\\ifx\\csname#1\\endcsname\\relax\n      \\expandafter\\expandafter\n      \\expandafter\\@firstoftwo\n    \\else\n      \\expandafter\\expandafter\n      \\expandafter\\@secondoftwo\n    \\fi\n  \\else\n    \\expandafter\\@firstoftwo\n  \\fi}\n\n% {<cstoken>}{<true}{<false>}\n\n\\newcommand{\\ifdefmacro}{}\n\\long\\edef\\ifdefmacro#1{%\n  \\noexpand\\expandafter\\noexpand\\etb@ifdefmacro\n  \\noexpand\\meaning#1\\detokenize{macro}:&}\n\\edef\\etb@ifdefmacro{%\n  \\def\\noexpand\\etb@ifdefmacro##1\\detokenize{macro}:##2&}\n\\etb@ifdefmacro{\\notblank{#2}}\n\n% {<csname>}{<true>}{<false>}\n\n\\newcommand*{\\ifcsmacro}[1]{%\n  \\ifcsdef{#1}\n    {\\expandafter\\ifdefmacro\\csname#1\\endcsname}\n    {\\@secondoftwo}}\n\n% {<cstoken>}{<true}{<false>}\n\n\\newcommand{\\ifdefprefix}[1]{%\n  \\ifdefmacro#1\n    {\\etb@ifdefprefix#1}\n    {\\@secondoftwo}}\n\\long\\edef\\etb@ifdefprefix#1{%\n  \\noexpand\\expandafter\\noexpand\\etb@ifdefprefix@i\n  \\noexpand\\meaning#1\\detokenize{macro}:&}\n\\edef\\etb@ifdefprefix@i{%\n  \\def\\noexpand\\etb@ifdefprefix@i##1\\detokenize{macro}:##2&}\n\\etb@ifdefprefix@i{\\notblank{#1}}\n\n% {<csname>}{<true>}{<false>}\n\n\\newcommand*{\\ifcsprefix}[1]{%\n  \\ifcsdef{#1}\n    {\\expandafter\\ifdefprefix\\csname#1\\endcsname}\n    {\\@secondoftwo}}\n\n% {<cstoken>}{<true}{<false>}\n\n\\newcommand{\\ifdefparam}{}\n\\long\\edef\\ifdefparam#1{%\n  \\noexpand\\expandafter\\noexpand\\etb@ifdefparam\n  \\noexpand\\meaning#1\\detokenize{macro}:->&}\n\\edef\\etb@ifdefparam{%\n  \\def\\noexpand\\etb@ifdefparam##1\\detokenize{macro}:##2->##3&}\n\\etb@ifdefparam{\\notblank{#2}}\n\n% {<csname>}{<true>}{<false>}\n\n\\newcommand*{\\ifcsparam}[1]{%\n  \\ifcsdef{#1}\n    {\\expandafter\\ifdefparam\\csname#1\\endcsname}\n    {\\@secondoftwo}}\n\n% {<cstoken>}{<true}{<false>}\n\n\\newcommand{\\ifdefprotected}{}\n\\long\\edef\\ifdefprotected#1{%\n  \\noexpand\\expandafter\\noexpand\\etb@ifdefprotected\n  \\noexpand\\meaning#1\\string\\protected&}\n\\edef\\etb@ifdefprotected{%\n  \\def\\noexpand\\etb@ifdefprotected##1\\string\\protected##2&}\n\\etb@ifdefprotected{\\notblank{#2}}\n\n% {<csname>}{<true>}{<false>}\n\n\\newcommand*{\\ifcsprotected}[1]{%\n  \\ifcsdef{#1}\n    {\\expandafter\\ifdefprotected\\csname#1\\endcsname}\n    {\\@secondoftwo}}\n\n% {<cstoken>}{<true}{<false>}\n\n\\newrobustcmd{\\ifdefltxprotect}[1]{%\n  \\begingroup\n  \\edef\\etb@resrvda{%\n    \\noexpand\\protect\\expandafter\\noexpand\n    \\csname\\expandafter\\@gobble\\string#1 \\endcsname}%\n  \\expandafter\\endgroup\\ifx#1\\etb@resrvda\n    \\expandafter\\@firstoftwo\n  \\else\n    \\expandafter\\@secondoftwo\n  \\fi}\n\n% {<csname>}{<true>}{<false>}\n\n\\newrobustcmd*{\\ifcsltxprotect}[1]{%\n  \\ifcsdef{#1}\n    {\\expandafter\\ifdefltxprotect\\csname#1\\endcsname}\n    {\\@secondoftwo}}\n\n% {<cstoken>}{<true>}{<false>}\n\n\\newcommand{\\ifdefempty}[1]{%\n  \\ifundef#1\n    {\\@secondoftwo}\n    {\\ifdefmacro#1\n       {\\ifdefparam#1\n\t  {\\@secondoftwo}\n\t  {\\etb@ifdefempty#1}}\n       {\\@secondoftwo}}}\n\n\\def\\etb@ifdefempty#1{%\n  \\expandafter\\expandafter\n  \\expandafter\\ifblank\n  \\expandafter\\expandafter\n  \\expandafter{%\n  \\expandafter\\strip@prefix\\meaning#1}}\n\n% {<csname>}{<true>}{<false>}\n\n\\newcommand*{\\ifcsempty}[1]{%\n  \\ifcsundef{#1}\n    {\\@secondoftwo}\n    {\\expandafter\\ifdefparam\\csname#1\\endcsname\n       {\\@secondoftwo}\n       {\\expandafter\\etb@ifdefempty\\csname#1\\endcsname}}}\n\n% {<cstoken>}{<true>}{<false>}\n\n\\newcommand{\\ifdefvoid}[1]{%\n  \\ifundef#1\n    {\\@firstoftwo}\n    {\\ifdefmacro#1\n       {\\ifdefparam#1\n\t  {\\@secondoftwo}\n\t  {\\etb@ifdefempty#1}}\n       {\\@secondoftwo}}}\n\n% {<csname>}{<true>}{<false>}\n\n\\newcommand*{\\ifcsvoid}[1]{%\n  \\ifcsundef{#1}\n    {\\@firstoftwo}\n    {\\expandafter\\ifdefparam\\csname#1\\endcsname\n       {\\@secondoftwo}\n       {\\expandafter\\etb@ifdefempty\\csname#1\\endcsname}}}\n\n% {<cstoken1>}{<cstoken2>}{<true>}{<false>}\n\n\\newcommand{\\ifdefequal}[2]{%\n  \\ifundef#1\n    {\\@secondoftwo}\n    {\\ifundef#2\n       {\\@secondoftwo}\n       {\\ifx#1#2\n          \\expandafter\\@firstoftwo\n        \\else\n          \\expandafter\\@secondoftwo\n        \\fi}}}\n\n% {<csname1>}{<csname2>}{<true>}{<false>}\n\n\\newcommand*{\\ifcsequal}[2]{%\n  \\ifcsundef{#1}\n    {\\@secondoftwo}\n    {\\ifcsundef{#2}\n       {\\@secondoftwo}\n       {\\expandafter\\ifx\n        \\csname#1\\expandafter\\endcsname\n        \\csname#2\\endcsname\n          \\expandafter\\@firstoftwo\n        \\else\n          \\expandafter\\@secondoftwo\n        \\fi}}}\n\n% {<cstoken>}{<string>}{<true>}{<false>}\n\n\\newrobustcmd{\\ifdefstring}[2]{%\n  \\ifundef#1\n    {\\@secondoftwo}\n    {\\ifdefmacro#1\n       {\\begingroup\n\t\\edef\\etb@tempa{\\expandafter\\strip@prefix\\meaning#1}%\n\t\\edef\\etb@tempb{\\detokenize{#2}}%\n\t\\ifx\\etb@tempa\\etb@tempb\n\t  \\aftergroup\\@firstoftwo\n\t\\else\n\t  \\aftergroup\\@secondoftwo\n\t\\fi\n\t\\endgroup}\n       {\\@secondoftwo}}}\n\n% {<csname>}{<string>}{<true>}{<false>}\n\n\\newrobustcmd{\\ifcsstring}[2]{%\n  \\ifcsundef{#1}\n    {\\@secondoftwo}\n    {\\expandafter\\ifdefstring\\csname#1\\endcsname{#2}}}\n\n% {<string1>}{<string2>}{<true>}{<false>}\n\n\\newrobustcmd{\\ifstrequal}[2]{%\n  \\begingroup\n  \\edef\\etb@tempa{\\detokenize{#1}}%\n  \\edef\\etb@tempb{\\detokenize{#2}}%\n  \\ifx\\etb@tempa\\etb@tempb\n    \\aftergroup\\@firstoftwo\n  \\else\n    \\aftergroup\\@secondoftwo\n  \\fi\n  \\endgroup}\n\n% {<string>}{<true>}{<false>}\n\n\\newcommand{\\ifstrempty}[1]{%\n  \\expandafter\\ifx\\expandafter&\\detokenize{#1}&%\n    \\expandafter\\@firstoftwo\n  \\else\n    \\expandafter\\@secondoftwo\n  \\fi}\n\n% {<string>}{<true>}{<false>}\n\n\\newcommand{\\ifblank}[1]{% from url.sty\n  \\etb@ifblank@i#1&&\\@secondoftwo\\@firstoftwo:}\n\\long\\def\\etb@ifblank@i#1#2&#3#4#5:{#4}\n\n\\newcommand{\\notblank}[1]{%\n  \\etb@ifblank@i#1&&\\@firstoftwo\\@secondoftwo:}\n\n% {<numexpr>}{<comp>}{<numexpr>}{<true>}{<false>}\n\n\\newcommand*{\\ifnumcomp}[3]{%\n  \\ifnum\\numexpr#1\\relax#2\\numexpr#3\\relax\n    \\expandafter\\@firstoftwo\n  \\else\n    \\expandafter\\@secondoftwo\n  \\fi}\n\n% {<numexpr>}{<numexpr>}{<true>}{<false>}\n\n\\newcommand*{\\ifnumequal}[1]{%\n  \\ifnumcomp{#1}=}\n\n\\newcommand*{\\ifnumgreater}[1]{%\n  \\ifnumcomp{#1}>}\n\n\\newcommand*{\\ifnumless}[1]{%\n  \\ifnumcomp{#1}<}\n\n% {<numexpr>}{<true>}{<false>}\n\n\\newcommand*{\\ifnumodd}[1]{%\n  \\ifodd\\numexpr#1\\relax\n    \\expandafter\\@firstoftwo\n  \\else\n    \\expandafter\\@secondoftwo\n  \\fi}\n\n% {<dimexpr>}{<comp>}{<dimexpr>}{<true>}{<false>}\n\n\\newcommand*{\\ifdimcomp}[3]{%\n  \\ifdim\\dimexpr#1\\relax#2\\dimexpr#3\\relax\n    \\expandafter\\@firstoftwo\n  \\else\n    \\expandafter\\@secondoftwo\n  \\fi}\n\n% {<dimexpr>}{<dimexpr>}{<true>}{<false>}\n\n\\newcommand*{\\ifdimequal}[1]{%\n  \\ifdimcomp{#1}=}\n\n\\newcommand*{\\ifdimgreater}[1]{%\n  \\ifdimcomp{#1}>}\n\n\\newcommand*{\\ifdimless}[1]{%\n  \\ifdimcomp{#1}<}\n\n% {<expr>}{<true>}{<false>}\n\n\\newcommand{\\ifboolexpe}[1]{%\n  \\etb@be@beg\\etb@be@bgroup#1(&\\etb@be@end}\n\n\\let\\etb@be@true\\@empty\n\\def\\etb@be@false{-\\@ne}\n\n\\def\\etb@be@beg{%\n  \\ifnum\\numexpr\\z@\\ifnum\\numexpr\\z@}\n\n\\def\\etb@be@end{%\n  <\\z@\n    \\expandafter\\etb@be@false\n  \\fi\n  <\\z@\n    \\expandafter\\@secondoftwo\n  \\else\n    \\expandafter\\@firstoftwo\n  \\fi}\n\n\\long\\def\\etb@be@bgroup#1(#2&{%\n  \\etb@be@egroup#1)&%\n  \\ifblank{#2}\n    {}\n    {\\etb@be@beg\n     \\etb@be@bgroup#2&}}\n\n\\long\\def\\etb@be@egroup#1)#2&{%\n  \\etb@be@and#1and&%\n  \\ifblank{#2}\n    {}\n    {\\etb@be@end\\etb@be@true\\etb@be@false\n     \\etb@be@egroup#2&}}\n\n\\long\\def\\etb@be@and#1and#2&{%\n  \\etb@be@or#1or&%\n  \\ifblank{#2}\n    {}\n    {<\\z@\n       \\expandafter\\@firstofone\n     \\else\n       \\expandafter\\@gobble\n     \\fi\n     {=\\z@\\fi\\ifnum\\numexpr\\m@ne}%\n     \\ifnum\\numexpr\\z@\n     \\etb@be@and#2&}}\n\n\\long\\def\\etb@be@or#1or#2&{%\n  \\etb@be@not#1not&%\n  \\ifblank{#2}\n    {}\n    {<\\z@\n       \\expandafter\\@secondoftwo\n     \\else\n       \\expandafter\\@firstoftwo\n     \\fi\n     {=\\z@\\fi\\ifnum\\numexpr\\z@\n      \\ifnum\\numexpr\\@ne}\n     {=\\z@\\fi\\ifnum\\numexpr\\z@\n      \\ifnum\\numexpr\\z@}%\n     \\etb@be@or#2&}}\n\n\\long\\def\\etb@be@not#1not#2&{%\n  \\etb@be@togl#1togl&%\n  \\ifblank{#2}\n    {}\n    {>\\z@\n       \\expandafter\\@firstoftwo\n     \\else\n       \\expandafter\\@secondoftwo\n     \\fi\n     {\\unless\\ifnum\\numexpr\\m@ne}\n     {\\unless\\ifnum\\numexpr\\z@}%\n     \\etb@be@not#2&}}\n\n\\long\\def\\etb@be@togl#1togl#2&{%\n  \\etb@be@bool#1bool&%\n  \\ifblank{#2}\n    {}\n    {\\etb@be@togl@i#2&}}\n\n\\long\\def\\etb@be@togl@i#1#2&{%\n  \\ifcsdef{etb@tgl@#1}\n    {\\csname etb@tgl@#1\\endcsname\\etb@be@true\\etb@be@false}\n    {\\etb@be@err{Toggle '#1' undefined}{}}%\n  \\etb@be@togl#2&}\n\n\\long\\def\\etb@be@bool#1bool#2&{%\n  \\etb@be@test#1test&%\n  \\ifblank{#2}\n    {}\n    {\\etb@be@bool@i#2&}}\n\n\\long\\def\\etb@be@bool@i#1#2&{%\n  \\ifcsundef{if#1}\n    {\\etb@be@err{Boolean '#1' undefined}{}}\n    {\\csname if#1\\endcsname\n     \\else\n       \\etb@be@false\n     \\fi}%\n  \\etb@be@bool#2&}\n\n\\long\\def\\etb@be@test#1test#2&{%\n  \\ifblank{#1}\n    {}\n    {\\etb@be@err{The invalid part is: '\\detokenize{#1}'}{}}%\n  \\ifblank{#2}\n    {}\n    {\\etb@be@test@i#2&}}\n\n\\long\\def\\etb@be@test@i#1#2&{%\n  #1\\etb@be@true\\etb@be@false\n  \\etb@be@test#2&}\n\n\\long\\def\\etb@be@err#1#2{%\n  \\expandafter\\ifnum\\the\\numexpr\n    \\expandafter\\ifnum\\the\\currentiftype=-3\n      \\expandafter\\thr@@\n    \\else\n      \\expandafter\\currentiftype\n    \\fi\n  =\\thr@@\n    \\expandafter\\@firstoftwo\n  \\else\n    \\expandafter\\@secondoftwo\n  \\fi\n  {=\\z@\\fi\n   \\etb@be@err{#1}{#2\\ifnum\\numexpr\\m@ne}}\n  {\\etb@err@expr{#1}#2}}\n\n% {<expr>}{<true>}{<false>}\n\n\\newrobustcmd{\\ifboolexpr}[1]{\\etb@boolexpr{#1}}\n\n\\long\\def\\etb@boolexpr#1{%\n  \\begingroup\n  \\let\\etb@br@neg\\@firstoftwo\n  \\etb@tempcnta\\z@\n  \\etb@br@beg\n  \\etb@br@bgroup#1(&%\n  \\etb@br@end\n  \\etb@br@eval}\n\n\\def\\etb@br@beg{%\n  \\begingroup\n  \\let\\etb@br@neg\\@firstoftwo\n  \\etb@tempcnta\\z@}\n\n\\def\\etb@br@end{%\n  \\etb@br@eval\\etb@br@true\\etb@br@false}\n\n\\def\\etb@br@eval{%\n  \\ifnum\\etb@tempcnta<\\z@\n    \\aftergroup\\@secondoftwo\n  \\else\n    \\aftergroup\\@firstoftwo\n  \\fi\n  \\endgroup}\n\n\\def\\etb@br@true{%\n  \\advance\\etb@tempcnta\\etb@br@neg\\z@\\m@ne\n  \\let\\etb@br@neg\\@firstoftwo}\n\n\\def\\etb@br@false{%\n  \\advance\\etb@tempcnta\\etb@br@neg\\m@ne\\z@\n  \\let\\etb@br@neg\\@firstoftwo}\n\n\\long\\def\\etb@br@bgroup#1(#2&{%\n  \\etb@br@egroup#1)&%\n  \\ifblank{#2}\n    {}\n    {\\etb@br@beg\n     \\etb@br@bgroup#2&}}\n\n\\long\\def\\etb@br@egroup#1)#2&{%\n  \\etb@br@and#1and&%\n  \\ifblank{#2}\n    {}\n    {\\etb@br@end\n     \\etb@br@egroup#2&}}\n\n\\long\\def\\etb@br@and#1and#2&{%\n  \\etb@br@or#1or&%\n  \\ifblank{#2}\n    {}\n    {\\ifnum\\etb@tempcnta<\\z@\n       \\etb@tempcnta\\m@ne\n     \\else\n       \\etb@tempcnta\\z@\n     \\fi\n     \\etb@br@and#2&}}\n\n\\long\\def\\etb@br@or#1or#2&{%\n  \\etb@br@not#1not&%\n  \\ifblank{#2}\n    {}\n    {\\ifnum\\etb@tempcnta<\\z@\n       \\etb@tempcnta\\z@\n     \\else\n       \\etb@tempcnta\\@ne\n     \\fi\n     \\etb@br@or#2&}}\n\n\\long\\def\\etb@br@not#1not#2&{%\n  \\etb@br@togl#1togl&%\n  \\ifblank{#2}\n    {}\n    {\\let\\etb@br@neg\\@secondoftwo\n     \\etb@br@not#2&}}\n\n\\long\\def\\etb@br@togl#1togl#2&{%\n  \\etb@br@bool#1bool&%\n  \\ifblank{#2}\n    {}\n    {\\etb@br@togl@i#2&}}\n\n\\long\\def\\etb@br@togl@i#1#2&{%\n  \\ifcsdef{etb@tgl@#1}\n    {\\csname etb@tgl@#1\\endcsname\\etb@br@true\\etb@br@false}\n    {\\etb@err@expr{Toggle '#1' undefined}\\etb@br@false}%\n  \\etb@br@togl#2&}\n\n\\long\\def\\etb@br@bool#1bool#2&{%\n  \\etb@br@test#1test&%\n  \\ifblank{#2}\n    {}\n    {\\etb@br@bool@i#2&}}\n\n\\long\\def\\etb@br@bool@i#1#2&{%\n  \\ifcsundef{if#1}\n    {\\etb@err@expr{Boolean '#1' undefined}\\etb@br@false}\n    {\\csname if#1\\endcsname\n       \\etb@br@true\n     \\else\n       \\etb@br@false\n     \\fi}%\n  \\etb@br@bool#2&}\n\n\\long\\def\\etb@br@test#1test#2&{%\n  \\ifblank{#1}\n    {}\n    {\\etb@err@expr{The invalid part is: '\\detokenize{#1}'}}%\n  \\ifblank{#2}\n    {}\n    {\\etb@br@test@i#2&}}\n\n\\long\\def\\etb@br@test@i#1#2&{%\n  \\ignorespaces#1\\etb@br@true\\etb@br@false\n  \\etb@br@test#2&}\n\n\\long\\def\\etb@err@expr#1{%\n  \\etb@error\n    {Invalid boolean expression}\n    {#1.}}\n\n% {<expr>}{<code>}\n\n\\newrobustcmd{\\whileboolexpr}[2]{%\n  \\etb@boolexpr{#1}{#2\\whileboolexpr{#1}{#2}}{}}\n\n% {<expr>}{<code>}\n\n\\newrobustcmd{\\unlessboolexpr}[2]{%\n  \\etb@boolexpr{#1}{}{#2\\unlessboolexpr{#1}{#2}}}\n\n% {<cstoken>}\n\n\\newcommand{\\expandonce}[1]{%\n  \\unexpanded\\expandafter{#1}}\n\n% {<csname>}\n\n\\newcommand*{\\csexpandonce}[1]{%\n  \\expandafter\\expandonce\\csname#1\\endcsname}\n\n% {<code>}\n\n\\newcommand*{\\protecting}{}\n\\def\\protecting#{%\n  \\ifx\\protect\\@typeset@protect\n    \\etb@protecting\\@firstofone\n  \\fi\n  \\ifx\\protect\\@unexpandable@protect\n    \\etb@protecting\\etb@unexpandable\n  \\fi\n  \\ifx\\protect\\noexpand\n    \\etb@protecting\\unexpanded\n  \\fi\n  \\ifx\\protect\\string\n    \\etb@protecting\\detokenize\n  \\fi\n  \\relax\\@firstofone}\n\n\\def\\etb@protecting#1#2\\relax\\@firstofone{\\fi#1}\n\\long\\def\\etb@unexpandable#1{\\unexpanded{\\protecting{#1}}}\n\n% {<csname>}\n\n\\newrobustcmd*{\\csdef}[1]{\\expandafter\\def\\csname#1\\endcsname}\n\\newrobustcmd*{\\csedef}[1]{\\expandafter\\edef\\csname#1\\endcsname}\n\\newrobustcmd*{\\csgdef}[1]{\\expandafter\\gdef\\csname#1\\endcsname}\n\\newrobustcmd*{\\csxdef}[1]{\\expandafter\\xdef\\csname#1\\endcsname}\n\\newrobustcmd*{\\protected@csedef}{\\etb@protected\\csedef}\n\\newrobustcmd*{\\protected@csxdef}{\\etb@protected\\csxdef}\n\n\\def\\etb@protected{%\n  \\let\\@@protect\\protect\n  \\let\\protect\\@unexpandable@protect\n  \\afterassignment\\restore@protect}\n\n% {<csname>}{<cstoken>}\n\n\\newrobustcmd{\\cslet}[2]{%\n  \\expandafter\\let\\csname#1\\endcsname#2}\n\n% {<cstoken>}{<csname>}\n\n\\newrobustcmd{\\letcs}[2]{%\n  \\ifcsdef{#2}\n    {\\expandafter\\let\\expandafter#1\\csname#2\\endcsname}\n    {\\undef#1}}\n\n% {<csname>}{<csname>}\n\n\\newrobustcmd*{\\csletcs}[2]{%\n  \\ifcsdef{#2}\n    {\\expandafter\\let\n     \\csname#1\\expandafter\\endcsname\n     \\csname#2\\endcsname}\n    {\\csundef{#1}}}\n\n% {<csname>}\n\n\\newcommand*{\\csuse}[1]{%\n  \\ifcsname#1\\endcsname\n    \\csname#1\\expandafter\\endcsname\n  \\fi}\n\n% {<cstoken>}\n\n\\newrobustcmd{\\undef}[1]{\\let#1\\etb@undefined}\n\n% {<csname>}\n\n\\newrobustcmd*{\\csundef}[1]{\\cslet{#1}\\etb@undefined}\n\n% {<cstoken>}{<code>}\n\n\\newrobustcmd{\\appto}[2]{%\n  \\ifundef{#1}\n    {\\edef#1{\\unexpanded{#2}}}\n    {\\edef#1{\\expandonce#1\\unexpanded{#2}}}}\n\\newrobustcmd{\\eappto}[2]{%\n  \\ifundef{#1}\n    {\\edef#1{#2}}\n    {\\edef#1{\\expandonce#1#2}}}\n\\newrobustcmd{\\gappto}[2]{%\n  \\ifundef{#1}\n    {\\xdef#1{\\unexpanded{#2}}}\n    {\\xdef#1{\\expandonce#1\\unexpanded{#2}}}}\n\\newrobustcmd{\\xappto}[2]{%\n  \\ifundef{#1}\n    {\\xdef#1{#2}}\n    {\\xdef#1{\\expandonce#1#2}}}\n\n\\newrobustcmd*{\\protected@eappto}{\\etb@protected\\eappto}\n\\newrobustcmd*{\\protected@xappto}{\\etb@protected\\xappto}\n\n% {<cstoken>}{<code>}\n\n\\newrobustcmd{\\preto}[2]{%\n  \\ifundef{#1}\n    {\\edef#1{\\unexpanded{#2}}}\n    {\\edef#1{\\unexpanded{#2}\\expandonce#1}}}\n\\newrobustcmd{\\epreto}[2]{%\n  \\ifundef{#1}\n    {\\edef#1{#2}}\n    {\\edef#1{#2\\expandonce#1}}}\n\\newrobustcmd{\\gpreto}[2]{%\n  \\ifundef{#1}\n    {\\xdef#1{\\unexpanded{#2}}}\n    {\\xdef#1{\\unexpanded{#2}\\expandonce#1}}}\n\\newrobustcmd{\\xpreto}[2]{%\n  \\ifundef{#1}\n    {\\xdef#1{#2}}\n    {\\xdef#1{#2\\expandonce#1}}}\n\n\\newrobustcmd*{\\protected@epreto}{\\etb@protected\\epreto}\n\\newrobustcmd*{\\protected@xpreto}{\\etb@protected\\xpreto}\n\n% {<csname>}{<code>}\n\n\\newrobustcmd*{\\csappto}[1]{\\expandafter\\appto\\csname#1\\endcsname}\n\\newrobustcmd*{\\cseappto}[1]{\\expandafter\\eappto\\csname#1\\endcsname}\n\\newrobustcmd*{\\csgappto}[1]{\\expandafter\\gappto\\csname#1\\endcsname}\n\\newrobustcmd*{\\csxappto}[1]{\\expandafter\\xappto\\csname#1\\endcsname}\n\\newrobustcmd*{\\protected@cseappto}{\\etb@protected\\cseappto}\n\\newrobustcmd*{\\protected@csxappto}{\\etb@protected\\csxappto}\n\n% {<csname>}{<code>}\n\n\\newrobustcmd*{\\cspreto}[1]{\\expandafter\\preto\\csname#1\\endcsname}\n\\newrobustcmd*{\\csepreto}[1]{\\expandafter\\epreto\\csname#1\\endcsname}\n\\newrobustcmd*{\\csgpreto}[1]{\\expandafter\\gpreto\\csname#1\\endcsname}\n\\newrobustcmd*{\\csxpreto}[1]{\\expandafter\\xpreto\\csname#1\\endcsname}\n\\newrobustcmd*{\\protected@csepreto}{\\etb@protected\\csepreto}\n\\newrobustcmd*{\\protected@csxpreto}{\\etb@protected\\csxpreto}\n\n% {<cstoken>}{<numexpr>}\n\n\\newrobustcmd*{\\numdef}[2]{%\n  \\ifundef#1{\\let#1\\z@}{}%\n  \\edef#1{\\the\\numexpr#2}}\n\\newrobustcmd*{\\numgdef}[2]{%\n  \\ifundef#1{\\let#1\\z@}{}%\n  \\xdef#1{\\the\\numexpr#2}}\n\n% {<csname>}{<numexpr>}\n\n\\newrobustcmd*{\\csnumdef}[1]{%\n  \\expandafter\\numdef\\csname#1\\endcsname}\n\\newrobustcmd*{\\csnumgdef}[1]{%\n  \\expandafter\\numgdef\\csname#1\\endcsname}\n\n% {<cstoken>}{<dimexpr>}\n\n\\newrobustcmd*{\\dimdef}[2]{%\n  \\ifundef#1{\\let#1\\z@}{}%\n  \\edef#1{\\the\\dimexpr#2}}\n\\newrobustcmd*{\\dimgdef}[2]{%\n  \\ifundef#1{\\let#1\\z@}{}%\n  \\xdef#1{\\the\\dimexpr#2}}\n\n% {<csname>}{<dimexpr>}\n\n\\newrobustcmd*{\\csdimdef}[1]{%\n  \\expandafter\\dimdef\\csname#1\\endcsname}\n\\newrobustcmd*{\\csdimgdef}[1]{%\n  \\expandafter\\dimgdef\\csname#1\\endcsname}\n\n% {<cstoken>}{<glueexpr>}\n\n\\newrobustcmd*{\\gluedef}[2]{%\n  \\ifundef#1{\\let#1\\z@skip}{}%\n  \\edef#1{\\the\\glueexpr#2}}\n\\newrobustcmd*{\\gluegdef}[2]{%\n  \\ifundef#1{\\let#1\\z@skip}{}%\n  \\xdef#1{\\the\\glueexpr#2}}\n\n% {<csname>}{<glueexpr>}\n\n\\newrobustcmd*{\\csgluedef}[1]{%\n  \\expandafter\\gluedef\\csname#1\\endcsname}\n\\newrobustcmd*{\\csgluegdef}[1]{%\n  \\expandafter\\gluegdef\\csname#1\\endcsname}\n\n% {<cstoken>}{<muexpr>}\n\n\\newrobustcmd*{\\mudef}[2]{%\n  \\ifundef#1{\\def#1{0mu}}{}%\n  \\edef#1{\\the\\muexpr#2}}\n\\newrobustcmd*{\\mugdef}[2]{%\n  \\ifundef#1{\\let#1\\z@}{}%\n  \\xdef#1{\\the\\muexpr#2}}\n\n% {<csname>}{<muexpr>}\n\n\\newrobustcmd*{\\csmudef}[1]{%\n  \\expandafter\\mudef\\csname#1\\endcsname}\n\\newrobustcmd*{\\csmugdef}[1]{%\n  \\expandafter\\mugdef\\csname#1\\endcsname}\n\n% {<counter>}{<numexpr>}\n\n\\newrobustcmd*{\\defcounter}[2]{%\n  \\ifcsundef{c@#1}\n    {\\etb@noglobal\\@nocounterr{#1}}%\n    {\\csname c@#1\\endcsname\\numexpr#2\\relax}}\n\n% {<length>}{<glueexpr>}\n\n\\newrobustcmd*{\\deflength}[2]{%\n  \\ifundef{#1}\n    {\\etb@noglobal\\etb@err@nolen{#1}}%\n    {#1\\glueexpr#2\\relax}}\n\n\\protected\\def\\etb@err@nolen#1{%\n  \\etb@error{Length '\\string#1' undefined}\\@eha}\n\n% {<name>}\n\n\\newrobustcmd*{\\newbool}[1]{%\n  \\expandafter\\@ifdefinable\\csname if#1\\endcsname{%\n    \\expandafter\\newif\\csname if#1\\endcsname}}\n\n% {<name>}\n\n\\newrobustcmd*{\\providebool}[1]{%\n  \\ifcsundef{if#1}\n    {\\expandafter\\newif\\csname if#1\\endcsname}\n    {\\begingroup\n     \\edef\\@tempa{\\expandafter\\meaning\\csname if#1\\endcsname}%\n     \\ifx\\@tempa\\etb@isfalse\n     \\else\n       \\ifx\\@tempa\\etb@istrue\n       \\else\n         \\etb@error{\\@backslashchar if#1 not a boolean}\\@eha\n       \\fi\n     \\fi\n     \\endgroup}}\n\n% {<name>}{<true>|<false>}\n\n\\newrobustcmd*{\\setbool}[2]{%\n  \\ifcsundef{if#1}\n    {\\etb@noglobal\\etb@err@nobool{#1}}\n    {\\ifcsundef{#1#2}\n       {\\etb@noglobal\\etb@err@boolval{#2}}\n       {\\csname#1#2\\endcsname}}}\n\n% {<name>}\n\n\\newrobustcmd*{\\booltrue}[1]{%\n  \\ifcsundef{if#1}\n    {\\etb@noglobal\\etb@err@nobool{#1}}\n    {\\csname#1true\\endcsname}}\n\n% {<name>}\n\n\\newrobustcmd*{\\boolfalse}[1]{%\n  \\ifcsundef{if#1}\n    {\\etb@noglobal\\etb@err@nobool{#1}}\n    {\\csname#1false\\endcsname}}\n\n\\edef\\etb@istrue{\\meaning\\iftrue}\n\\edef\\etb@isfalse{\\meaning\\iffalse}\n\\protected\\def\\etb@noglobal{\\let\\relax\\relax}\n\n% {<name>}{<true}{<false>}\n\n\\newcommand*{\\ifbool}[1]{%\n  \\ifcsundef{if#1}\n    {\\etb@err@nobool{#1}\\@gobbletwo}\n    {\\csname if#1\\endcsname\n       \\expandafter\\@firstoftwo\n     \\else\n       \\expandafter\\@secondoftwo\n     \\fi}}\n\n% {<name>}{<not true}{<not false>}\n\n\\newcommand*{\\notbool}[1]{%\n  \\ifcsundef{if#1}\n    {\\etb@err@nobool{#1}\\@gobbletwo}\n    {\\csname if#1\\endcsname\n       \\expandafter\\@secondoftwo\n     \\else\n       \\expandafter\\@firstoftwo\n     \\fi}}\n\n\\protected\\def\\etb@err@nobool#1{%\n  \\etb@error{Boolean '\\@backslashchar if#1' undefined}\\@eha}\n\n\\def\\etb@err@boolval#1{%\n  \\etb@error\n    {Invalid boolean value '#1'}\n    {Valid boolean values are 'true' and 'false'.}}\n\n% {<name>}\n\n\\newrobustcmd*{\\newtoggle}[1]{%\n  \\ifcsdef{etb@tgl@#1}\n    {\\etb@error{Toggle '#1' already defined}\\@eha}\n    {\\cslet{etb@tgl@#1}\\@secondoftwo}}\n\n% {<name>}\n\n\\newrobustcmd*{\\providetoggle}[1]{%\n  \\ifcsdef{etb@tgl@#1}\n    {}\n    {\\cslet{etb@tgl@#1}\\@secondoftwo}}\n\n% {<name>}{<true>|<false>}\n\n\\newrobustcmd*{\\settoggle}[2]{%\n  \\ifcsdef{etb@tgl@#1}\n    {\\ifcsdef{etb@toggle#2}\n       {\\csletcs{etb@tgl@#1}{etb@toggle#2}}\n       {\\etb@noglobal\\etb@err@boolval{#2}}}\n    {\\etb@noglobal\\etb@err@notoggle{#1}}}\n\n% {<name>}\n\n\\newrobustcmd*{\\toggletrue}[1]{%\n  \\ifcsdef{etb@tgl@#1}\n    {\\cslet{etb@tgl@#1}\\etb@toggletrue}\n    {\\etb@noglobal\\etb@err@notoggle{#1}}}\n\n% {<name>}\n\n\\newrobustcmd*{\\togglefalse}[1]{%\n  \\ifcsdef{etb@tgl@#1}\n    {\\cslet{etb@tgl@#1}\\etb@togglefalse}\n    {\\etb@noglobal\\etb@err@notoggle{#1}}}\n\n\\let\\etb@toggletrue\\@firstoftwo\n\\let\\etb@togglefalse\\@secondoftwo\n\n% {<name>}{<true}{<false>}\n\n\\newcommand*{\\iftoggle}[1]{%\n  \\ifcsdef{etb@tgl@#1}\n    {\\csname etb@tgl@#1\\endcsname}\n    {\\etb@err@notoggle{#1}\\@gobbletwo}}\n\n% {<name>}{<not true}{<not false>}\n\n\\newcommand*{\\nottoggle}[1]{%\n  \\ifcsdef{etb@tgl@#1}\n    {\\csname etb@tgl@#1\\endcsname\\@secondoftwo\\@firstoftwo}\n    {\\etb@err@notoggle{#1}\\@gobbletwo}}\n\n\\protected\\def\\etb@err@notoggle#1{%\n  \\etb@error{Toggle '#1' undefined}\\@eha}\n\n% {<cstoken>}{<true}{<false>}\n\n\\protected\\def\\etb@ifscanable#1{%\n  \\begingroup\n  \\edef\\etb@resrvda{%\n    \\def\\noexpand\\etb@resrvda####1\\detokenize{macro}:####2->####3&{%\n      ####1\\def\\string\\etb@resrvda####2{####3}}%\n    \\edef\\noexpand\\etb@resrvda{\\noexpand\\etb@resrvda\\meaning#1&}}%\n  \\etb@resrvda\n  \\makeatletter\n  \\scantokens\\expandafter{\\etb@resrvda}%\n  \\expandafter\\endgroup\\ifx#1\\etb@resrvda\n    \\expandafter\\@firstoftwo\n  \\else\n    \\expandafter\\@secondoftwo\n  \\fi}\n\n% {<cstoken>}{<search>}{<true}{<false>}\n\n\\protected\\long\\def\\etb@ifpattern#1#2{%\n  \\begingroup\n  \\edef\\etb@resrvda{%\n    \\def\\noexpand\\etb@resrvda####1\\detokenize{#2}####2&{%\n      \\endgroup\\noexpand\\noexpand\\noexpand\\ifblank{####2}}%\n    \\edef\\noexpand\\etb@resrvda{\\noexpand\\etb@resrvda\n      \\expandafter\\strip@prefix\\meaning#1\\detokenize{#2}&}%\n    \\noexpand\\etb@resrvda}\n  \\etb@resrvda\\@secondoftwo\\@firstoftwo}\n\n% {<string>}{<true}{<false>}\n\n\\protected\\long\\def\\etb@ifhashcheck#1{%\n  \\begingroup\n  \\edef\\etb@resrvda{\\detokenize{#1}}%\n  \\expandafter\\endgroup\n  \\expandafter\\etb@ifhashcheck@i\\meaning\\etb@resrvda&}\n\n\\edef\\etb@ifhashcheck@i#1&{%\n  \\noexpand\\expandafter\n  \\noexpand\\etb@ifhashcheck@ii\n  \\noexpand\\strip@prefix#1\\string#\\string#&}\n\n\\edef\\etb@ifhashcheck@ii{%\n  \\def\\noexpand\\etb@ifhashcheck@ii##1\\string#\\string###2&}\n\\etb@ifhashcheck@ii{\\ifblank{#2}}\n\n% {<cstoken>}\n\n\\newrobustcmd*{\\robustify}[1]{%\n  \\ifundef#1\n    {\\etb@error{\\string#1 undefined}\\@eha}\n    {\\ifdefmacro#1\n       {\\ifdefltxprotect#1\n          {\\letcs\\etb@resrvda{\\expandafter\\@gobble\\string#1 }%\n           \\@tempswatrue}\n          {\\let\\etb@resrvda#1%\n           \\@tempswafalse}%\n        \\ifdefparam\\etb@resrvda\n          {\\etb@ifscanable\\etb@resrvda\n             {\\etb@robustify\\etb@resrvda\n              \\let#1\\etb@resrvda}\n             {\\etb@error{Failed to robustify \\string#1}\n                {The command is special and cannot be\n                 handled by \\string\\robustify.}%\n              \\@tempswafalse}}\n          {\\protected\\edef#1{\\expandonce\\etb@resrvda}}\n        \\if@tempswa\n          \\ifcsdef{\\string#1 }\n            {}\n            {\\csundef{\\expandafter\\@gobble\\string#1 }}%\n        \\fi\n        \\undef\\etb@resrvda}\n       {\\etb@error{\\string#1 not a macro}\\@eha}}}\n\n\\def\\etb@robustify#1{%\n  \\begingroup\n  \\edef\\etb@resrvdb{%\n    \\def\\noexpand\\etb@resrvdb####1\\detokenize{macro}:####2->####3&{%\n      \\protected####1\\def\\string#1\\space####2{####3}}%\n    \\edef\\noexpand\\etb@resrvdb{%\n      \\noexpand\\etb@resrvdb\\meaning#1&}}%\n  \\etb@resrvdb\n  \\etb@patchcmd@scantoks\\etb@resrvdb}\n\n%  {<cstoken>}{<search>}{<true}{<false>}\n% *{<cstoken>}{<true}{<false>}\n\n\\newrobustcmd{\\ifpatchable}{%\n  \\etb@dbg@trce\\ifpatchable\n  \\begingroup\n  \\@makeother\\#%\n  \\@ifstar\\etb@ifpatchable@i\\etb@ifpatchable}\n\n\\long\\def\\etb@ifpatchable#1#2{%\n  \\endgroup\n  \\etb@dbg@init#1%\n  \\ifundef#1\n    {\\etb@dbg@fail{def}\\@secondoftwo}\n    {\\etb@dbg@info{def}%\n     \\ifdefmacro#1\n       {\\etb@dbg@info{mac}%\n        \\etb@ifscanable#1\n          {\\etb@ifhashcheck{#2}\n             {\\etb@dbg@info{tok}%\n              \\etb@ifpattern#1{#2}\n                 {\\etb@dbg@info{pat}%\n                  \\etb@dbg@info{pos}\\@firstoftwo}\n                 {\\etb@dbg@fail{pat}\\@secondoftwo}}\n             {\\etb@dbg@fail{hsh}\\@secondoftwo}}\n          {\\etb@dbg@fail{tok}\\@secondoftwo}}\n       {\\etb@dbg@fail{mac}\\@secondoftwo}}}\n\n\\long\\def\\etb@ifpatchable@i#1{%\n  \\endgroup\n  \\etb@dbg@init#1%\n  \\ifundef#1\n    {\\etb@dbg@fail{def}\\@secondoftwo}\n    {\\etb@dbg@info{def}%\n     \\ifdefmacro#1\n       {\\etb@dbg@info{mac}%\n        \\ifdefparam#1\n          {\\etb@dbg@info{prm}%\n           \\etb@ifscanable#1\n             {\\etb@dbg@info{tok}%\n              \\etb@dbg@info{pos}\\@firstoftwo}\n             {\\etb@dbg@fail{tok}\\@secondoftwo}}\n          {\\etb@dbg@info{prl}%\n           \\ifdefprotected#1\n             {\\etb@dbg@info{pro}}\n             {}%\n           \\etb@dbg@info{pos}\\@firstoftwo}}\n       {\\etb@dbg@fail{mac}\\@secondoftwo}}}\n\n% [<prefix>]{<cstoken>}{<search>}{<replace>}{<success>}{<failure>}\n\n\\newrobustcmd*{\\patchcmd}{%\n  \\etb@dbg@trce\\patchcmd\n  \\begingroup\n  \\@makeother\\#%\n  \\etb@patchcmd}\n\n\\newcommand{\\etb@patchcmd}[4][########1]{%\n  \\etb@ifpatchable#2{#3}\n    {\\etb@dbg@succ{ret}%\n     \\begingroup\n     \\edef\\etb@resrvda{%\n       \\def\\noexpand\\etb@resrvda####1\\detokenize{macro:}####2->####3&{%\n         #1\\def\\string\\etb@resrvda\\space####2{\\noexpand\\etb@resrvdb####3&}}%\n       \\def\\noexpand\\etb@resrvdb####1\\detokenize{#3}####2&{%\n         ####1\\detokenize{#4}####2}%\n       \\edef\\noexpand\\etb@resrvda{%\n         \\noexpand\\etb@resrvda\\meaning#2&}}%\n     \\etb@resrvda\n     \\etb@patchcmd@scantoks\\etb@resrvda\n     \\let#2\\etb@resrvda\n     \\undef\\etb@resrvda\n     \\@firstoftwo}\n    {\\@secondoftwo}}\n\n\\def\\etb@patchcmd@scantoks#1{%\n  \\edef\\etb@resrvda{\\endgroup\n    \\unexpanded{\\makeatletter\\scantokens}{#1}%\n    \\catcode\\number`\\@=\\the\\catcode`\\@\\relax}%\n  \\etb@resrvda}\n\n% {<cstoken>}{<code>}{<success>}{<failure>}\n\n\\newrobustcmd*{\\apptocmd}{%\n  \\etb@dbg@trce\\apptocmd\n  \\begingroup\n  \\@makeother\\#%\n  \\etb@hooktocmd\\etb@append}\n\n\\newrobustcmd*{\\pretocmd}{%\n  \\etb@dbg@trce\\pretocmd\n  \\begingroup\n  \\@makeother\\#%\n  \\etb@hooktocmd\\etb@prepend}\n\n\\long\\def\\etb@hooktocmd#1#2#3{%\n  \\endgroup\n  \\etb@dbg@init#2%\n  \\ifundef#2\n    {\\etb@dbg@fail{def}\\@secondoftwo}\n    {\\etb@dbg@info{def}%\n     \\ifdefmacro#2\n       {\\etb@dbg@info{mac}%\n        \\ifdefparam#2\n          {\\etb@dbg@info{prm}%\n           \\etb@ifscanable#2\n             {\\etb@ifhashcheck{#3}\n                {\\etb@dbg@info{tok}%\n                 \\etb@dbg@succ{ret}%\n                 \\etb@hooktocmd@i#1#2{#3}%\n                 \\@firstoftwo}\n                {\\etb@dbg@fail{hsh}\\@secondoftwo}}\n             {\\etb@dbg@fail{tok}\\@secondoftwo}}\n          {\\etb@dbg@info{prl}%\n           \\ifdefprotected#2\n             {\\etb@dbg@info{pro}%\n              \\etb@dbg@succ{red}%\n              \\protected}\n             {\\etb@dbg@succ{red}}%\n           \\edef#2{#1{\\expandonce#2}{\\unexpanded{#3}}}%\n           \\@firstoftwo}}\n       {\\etb@dbg@fail{mac}\\@secondoftwo}}}\n\n\\long\\def\\etb@hooktocmd@i#1#2#3{%\n  \\begingroup\n  \\edef\\etb@resrvda{%\n    \\def\\noexpand\\etb@resrvda####1\\detokenize{macro}:####2->####3&{%\n      ####1\\def\\string\\etb@resrvda\\space####2{#1{####3}{\\detokenize{#3}}}}%\n    \\edef\\noexpand\\etb@resrvda{%\n      \\noexpand\\etb@resrvda\\meaning#2&}}%\n  \\etb@resrvda\n  \\etb@patchcmd@scantoks\\etb@resrvda\n  \\let#2\\etb@resrvda\n  \\undef\\etb@resrvda}\n\n\\long\\def\\etb@append#1#2{#1#2}\n\\long\\def\\etb@prepend#1#2{#2#1}\n\n\\newrobustcmd*{\\tracingpatches}{%\n  \\etb@info{Enabling tracing}%\n  \\input{etoolbox.def}%\n  \\global\\let\\tracingpatches\\relax}\n\\@onlypreamble\\tracingpatches\n\n\\let\\etb@dbg@trce\\@gobble\n\\let\\etb@dbg@init\\@gobble\n\\let\\etb@dbg@info\\@gobble\n\\let\\etb@dbg@succ\\@gobble\n\\let\\etb@dbg@fail\\@gobble\n\n% {<numeral>}\n\n\\newcommand{\\rmntonum}[1]{%\n  \\ifblank{#1}\n    {}\n    {\\expandafter\\etb@rti@end\\number\\numexpr\n     \\expandafter\\etb@rti@prs\\detokenize{#1}&\\relax}}\n\n\\def\\etb@rti@prs#1#2{%\n  \\ifx&#1%\n    \\expandafter\\@firstoftwo\n  \\else\n    \\expandafter\\@secondoftwo\n  \\fi\n  {#1#2}\n  {\\ifx&#2%\n     \\expandafter\\@firstoftwo\n   \\else\n     \\expandafter\\@secondoftwo\n   \\fi\n   {\\etb@rti@chk#1+\\etb@rti@num#1#2}\n   {\\etb@rti@chk#1\\etb@rti@chk#2%\n    \\ifnum\\etb@rti@num#1<\\etb@rti@num#2 %\n      \\expandafter\\@firstoftwo\n    \\else\n      \\expandafter\\@secondoftwo\n    \\fi\n    {+\\etb@rti@num#2-\\etb@rti@num#1\\etb@rti@prs}\n    {+\\etb@rti@num#1\\etb@rti@prs#2}}}}\n\n\\def\\etb@rti@chk#1{%\n  \\ifcsname etb@rmn@#1\\endcsname\n  \\else\n    \\expandafter\\etb@rti@brk\n  \\fi}\n\n\\def\\etb@rti@brk#1&{+\\z@&-1}\n\\def\\etb@rti@end#1&#2\\relax{\\ifblank{#2}{#1}{#2}}\n\\def\\etb@rti@num#1{\\csname etb@rmn@#1\\endcsname}\n\n\\chardef\\etb@rmn@i=1\n\\chardef\\etb@rmn@I=1\n\\chardef\\etb@rmn@v=5\n\\chardef\\etb@rmn@V=5\n\\chardef\\etb@rmn@x=10\n\\chardef\\etb@rmn@X=10\n\\chardef\\etb@rmn@l=50\n\\chardef\\etb@rmn@L=50\n\\chardef\\etb@rmn@c=100\n\\chardef\\etb@rmn@C=100\n\\mathchardef\\etb@rmn@d=500\n\\mathchardef\\etb@rmn@D=500\n\\mathchardef\\etb@rmn@m=1000\n\\mathchardef\\etb@rmn@M=1000\n\n% {<numeral>}{<true>}{<false>}\n\n\\newcommand{\\ifrmnum}[1]{%\n  \\ifblank{#1}\n    {\\@secondoftwo}\n    {\\expandafter\\etb@ifr@prs\\detokenize{#1}\\relax}}\n\n\\def\\etb@ifr@prs#1{%\n  \\ifx\\relax#1%\n    \\expandafter\\@firstoftwo\n  \\else\n    \\ifcsname etb@rmn@#1\\endcsname\n      \\expandafter\\expandafter\n      \\expandafter\\etb@ifr@prs\n    \\else\n      \\expandafter\\expandafter\n      \\expandafter\\etb@ifr@brk\n    \\fi\n  \\fi}\n\n\\def\\etb@ifr@brk#1\\relax{\\@secondoftwo}\n\n% <*>{<command>}{<separator>}\n\n\\newrobustcmd*{\\DeclareListParser}{%\n  \\@ifstar\n    {\\etb@defparser\\etb@defparser@arg}\n    {\\etb@defparser\\etb@defparser@do}}\n\n\\def\\etb@defparser#1#2#3{%\n  \\@ifdefinable#2{#1{#2}{#3}}}\n\n\\def\\etb@defparser@do#1#2{%\n  \\begingroup\n  \\edef\\@tempa{\\endgroup\n    \\long\\def\\noexpand#1####1{%\n      \\expandafter\\noexpand\n      \\csname etb@lst@\\expandafter\\@gobble\\string#1\\endcsname\n      \\space####1\\noexpand#2&}%\n    \\long\\csdef{etb@lst@\\expandafter\\@gobble\\string#1}####1\\noexpand#2####2&{%\n      \\noexpand\\etb@listitem\\noexpand\\do{####1}%\n      \\noexpand\\ifblank{####2}\n        {\\noexpand\\listbreak}\n        {\\expandafter\\noexpand\n         \\csname etb@lst@\\expandafter\\@gobble\\string#1\\endcsname\n\t \\space####2}&}}%\n  \\@tempa}\n\n\\def\\etb@defparser@arg#1#2{%\n  \\begingroup\n  \\edef\\@tempa{\\endgroup\n    \\long\\def\\noexpand#1####1####2{%\n      \\expandafter\\noexpand\n      \\csname etb@lst@\\expandafter\\@gobble\\string#1\\endcsname\n      {####1}\\space####2\\noexpand#2&}%\n    \\long\\csdef{etb@lst@\\expandafter\\@gobble\\string#1}####1####2\\noexpand#2####3&{%\n      \\noexpand\\etb@listitem{####1}{####2}%\n      \\noexpand\\ifblank{####3}\n        {\\noexpand\\listbreak}\n        {\\expandafter\\noexpand\n         \\csname etb@lst@\\expandafter\\@gobble\\string#1\\endcsname\n\t {####1}\\space####3}&}}%\n  \\@tempa}\n\n\\long\\def\\etb@listitem#1#2{%\n  \\ifblank{#2}\n    {}\n    {\\expandafter\\etb@listitem@i\n     \\expandafter{\\@firstofone#2}{#1}}}\n\\long\\def\\etb@listitem@i#1#2{#2{#1}}\n\n\\newcommand*{\\listbreak}{}\n\\long\\def\\listbreak#1&{}\n\n% {<item1>,<item2>,...} => \\do{<item1>}\\do{<item2>}...\n\n\\DeclareListParser{\\docsvlist}{,}\n\n% {<handler>}{<item1>,<item2>,...} => <handler>{<item1>}<handler>{<item2>}...\n\n\\DeclareListParser*{\\forcsvlist}{,}\n\n% {<listmacro>}{<string>}\n\n\\newrobustcmd{\\listadd}[2]{%\n  \\ifblank{#2}{}{\\appto#1{#2|}}}\n\\newrobustcmd{\\listeadd}[2]{%\n  \\begingroup\n  \\edef\\etb@tempa{\\endgroup\\noexpand\\ifblank{#2}}%\n  \\etb@tempa{}{\\eappto#1{#2|}}}\n\\newrobustcmd{\\listgadd}[2]{%\n  \\ifblank{#2}{}{\\gappto#1{#2|}}}\n\\newrobustcmd{\\listxadd}[2]{%\n  \\begingroup\n  \\edef\\etb@tempa{\\endgroup\\noexpand\\ifblank{#2}}%\n  \\etb@tempa{}{\\xappto#1{#2|}}}\n\n% {<listcsname>}{<string>}\n\n\\newrobustcmd{\\listcsadd}[1]{%\n  \\expandafter\\listadd\\csname#1\\endcsname}\n\\newrobustcmd{\\listcseadd}[1]{%\n  \\expandafter\\listeadd\\csname#1\\endcsname}\n\\newrobustcmd{\\listcsgadd}[1]{%\n  \\expandafter\\listgadd\\csname#1\\endcsname}\n\\newrobustcmd{\\listcsxadd}[1]{%\n  \\expandafter\\listxadd\\csname#1\\endcsname}\n\n% {<string>}{<listmacro>}{<true>}{<false>}\n\n\\newrobustcmd{\\ifinlist}[2]{%\n  \\begingroup\n  \\def\\etb@tempa##1|#1|##2&{\\endgroup\n    \\ifblank{##2}\\@secondoftwo\\@firstoftwo}%\n  \\expandafter\\etb@tempa\\expandafter|#2|#1|&}\n\n\\newrobustcmd{\\xifinlist}[1]{%\n  \\begingroup\n  \\edef\\etb@tempa{\\endgroup\\ifinlist{#1}}%\n  \\etb@tempa}\n\n% {<string>}{<listcsname>}{<true>}{<false>}\n\n\\newrobustcmd{\\ifinlistcs}[2]{%\n  \\expandafter\\etb@ifinlistcs@i\\csname #2\\endcsname{#1}}\n\\long\\def\\etb@ifinlistcs@i#1#2{\\ifinlist{#2}{#1}}\n\n\\newrobustcmd{\\xifinlistcs}[1]{%\n  \\begingroup\n  \\edef\\etb@tempa{\\endgroup\\ifinlistcs{#1}}%\n  \\etb@tempa}\n\n% {<handler>}{<listmacro>} => <handler>{<item1>}<handler>{<item2>}...\n\n\\newcommand*{\\forlistloop}[2]{%\n  \\expandafter\\etb@forlistloop\\expandafter{#2}{#1}}\n\n\\long\\def\\etb@forlistloop#1#2{\\etb@forlistloop@i{#2}#1|&}\n\n\\long\\def\\etb@forlistloop@i#1#2|#3&{%\n  \\ifblank{#2}\n    {}\n    {#1{#2}}%\n  \\ifblank{#3}\n    {\\listbreak}\n    {\\etb@forlistloop@i{#1}#3}%\n  &}\n\n% {<handler>}{<listcsname>} => <handler>{<item1>}<handler>{<item2>}...\n\n\\newcommand*{\\forlistcsloop}[2]{%\n  \\expandafter\\expandafter\\expandafter\\etb@forlistloop\n  \\expandafter\\expandafter\\expandafter{\\csname#2\\endcsname}{#1}}\n\n% {<listmacro>} => \\do{<item1>}\\do{<item2>}...\n\n\\newcommand*{\\dolistloop}{\\forlistloop\\do}\n\n% {<listcsname>} => \\do{<item1>}\\do{<item2>}...\n\n\\newcommand*{\\dolistcsloop}{\\forlistcsloop\\do}\n\n% {<code>}\n\n\\newrobustcmd*{\\AtEndPreamble}{\\gappto\\@endpreamblehook}\n\\newcommand*{\\@endpreamblehook}{}\n\n\\preto\\document{%\n  \\endgroup\n  \\let\\AtEndPreamble\\@firstofone\n  \\@endpreamblehook\n  \\protected\\def\\AtEndPreamble{\\@notprerr\\@gobble}%\n  \\undef\\@endpreamblehook\n  \\begingroup}\n\n% {<code>}\n\n\\newrobustcmd*{\\AfterPreamble}{\\AtBeginDocument}\n\\AtEndPreamble{\\let\\AfterPreamble\\@firstofone}\n\n% {<code>}\n\n\\newrobustcmd*{\\AfterEndPreamble}{\\gappto\\@afterendpreamblehook}\n\\newcommand*{\\@afterendpreamblehook}{}\n\n\\appto\\document{%\n  \\let\\AfterEndPreamble\\@firstofone\n  \\@afterendpreamblehook\n  \\protected\\def\\AfterEndPreamble{\\@notprerr\\@gobble}%\n  \\undef\\@afterendpreamblehook\n  \\ignorespaces}\n\n\\AtEndDocument{\\let\\AfterEndPreamble\\@gobble}\n\n% {<code>}\n\n\\newrobustcmd*{\\AfterEndDocument}{\\gappto\\@afterenddocumenthook}\n\\newcommand*{\\@afterenddocumenthook}{}\n\n\\patchcmd\\enddocument\n  {\\deadcycles}\n  {\\let\\AfterEndDocument\\@firstofone\n   \\@afterenddocumenthook\n   \\deadcycles}\n  {}\n  {\\let\\etb@@end\\@@end\n   \\def\\@@end{%\n     \\let\\AfterEndDocument\\@firstofone\n     \\@afterenddocumenthook\n     \\etb@@end}}\n\n\\endinput\n"
  },
  {
    "path": "user-dev-guide/gen_component.tex",
    "content": "\\section{\\texorpdfstring{The \\erlmodule{gen\\_component}}\n         {The gen\\_component}}\n\\erlmoduleindex{gen\\_component}\n\nThe generic component model implemented by\n\\erlmodule[noindex]{gen\\_component} allows to add some common functionality\nto all the components that build up the \\scalaris{} system. It supports:\n\n\\begin{description}\n\\setlength{\\parskip}{0pt}\n\\setlength{\\itemsep}{0pt}\n\\item[event-handlers:] message handling with a similar syntax as used in \\cite{rachid-book}.\n\\item[FIFO order of messages:] components cannot be inadvertently locked as\n  we do not use selective receive statements in the code.\n\\item[sleep and halt:] for testing components can sleep or be halted.\n\\item[debugging, breakpoints, stepwise execution:] to debug components\n  execution can be steered via breakpoints, step-wise execution and\n  continuation based on arriving events and user defined component state\n  conditions.\n\\item[basic profiling,]\n\\item[state dependent message handlers:] depending on its state, different\n  message handlers can be used and switched during runtime. Thereby a kind\n  of state-machine based message handling is supported.\n\\item[prepared for \\erlmodule{pid\\_groups}:] allows to send events to\n  named processes inside the same group as the actual component itself\n  (\\code{send_to_group_member}) when just holding a reference to any group\n  member, and\n\\item[unit-testing of event-handlers:] as message handling is separated from\n  the main loop of the component, the handling of individual messages and\n  thereby performed state manipulation can easily tested in unit-tests by\n  directly calling message handlers.\n\\end{description}\n\nIn \\scalaris{} all Erlang processes should be implemented as\n\\erlmodule{gen\\_component}. The only exception are functions interfacing to\nthe client, where a transition from asynchronous to synchronous request\nhandling is necessary and that are executed in the context of a client's\nprocess or a process that behaves as a proxy for a client\n(\\erlmodule{cs\\_api}).\n\n\\subsection{\\texorpdfstring{A basic \\erlmodule{gen\\_component} including a message handler}\n             {A basic gen\\_component including a message handler}}\n\nTo implement a \\erlmodule{gen\\_component}, the component has to provide the\n\\erlmodule{gen\\_component} behaviour:\n\n\\codesnippet{gen_component.erl}{gen_component:behaviour}{../src/gen_component.erl}\n\nThis is illustrated by the following example:\n\n\\codesnippet{msg_delay.erl}{gen_component:sample}{../src/msg_delay.erl}\n\n\\erlfun{your\\_gen\\_component}{init}{/1} is called during start-up of a\n\\erlmodule{gen\\_component} and should return the initial state to be used\nfor this \\erlmodule{gen\\_component}. Later, the current state of the\ncomponent can be retrieved using \\erlfun{gen\\_component}{get\\_state}{/1}.\n\nTo react on messages / events, a message handler is used. The default\nmessage handler is given to\n\\erlfun{gen\\_component}{start\\_link}{/4} as well as\n\\erlfun{gen\\_component}{start}{/4} or\n\\erlfun{gen\\_component}{start}{/5}. It can be\nchanged by calling \\erlfun{gen\\_component}{change\\_handler}{/2} (see\nSection~\\ref{sec:gen_component:change_handler}). When an event / message for\nthe component arrives, this handler is called with the event itself and the\ncurrent state of the component. In the handler, the state of the component\nmay be adjusted depending upon the event. The handler itself may trigger new\nevents / messages for itself or other components and has finally to return\nthe updated state of the component or the atoms \\texttt{unknown\\_event} or\n\\texttt{kill}. It must neither call \\code{receive} nor\n\\erlfun{timer}{sleep}{/1} nor \\erlfun{erlang}{exit}{/1}.\n\n\\subsection{\\texorpdfstring{How to start a \\erlmodule{gen\\_component}?}\n             {How to start a gen\\_component?}}\n\nA \\erlmodule{gen\\_component} can be started using one of:\n\n\\erlfun{gen\\_component}{start}{Module, Handler, Args, GenCOptions = []}\\\\\n\\erlfun{gen\\_component}{start\\_link}{Module, Handler, Args, GenCOptions = []}\n\\begin{erlfunparams}\n\\erlparam{Module}{the name of the module your component is\n   implemented in}\n   \\erlparam{Handler}{the inital message handler}\n\\erlparam{Args}{List of parameters passed to \\code{Module:init/1}\n   for initialization}\n\\erlparam{GenCOptions}{optional parameter.\n   List of options for \\erlmodule{gen\\_component}\n\\begin{description}\n\\setlength{\\parskip}{0pt}\n\\setlength{\\itemsep}{0pt}\n\\item[\\texttt{\\{pid\\_groups\\_join\\_as, ProcessGroup, ProcessName\\}}:] registers the new\n  process with the given process group (also called instanceid) and name\n  using \\erlmodule{pid\\_groups}.\n\\item[\\texttt{\\{erlang\\_register, ProcessName\\}}:] registers the process as\n  a named Erlang process.\n\\item[\\texttt{\\{wait\\_for\\_init\\}}:] wait for \\code{Module:init/1} to return\n  before returning to the caller.\n\\end{description}\n}\n\\end{erlfunparams}\n\nThese functions are compatible to the Erlang/OTP supervisors.  They spawn a\nnew process for the component which itself calls \\code{Module:init/1} with\nthe given \\code{Args} to initialize the component. \\code{Module:init/1}\nshould return the initial state for your component. For each message sent to\nthis component, the default message handler\n\\erlfun[noindex]{Module}{on}{Message, State} will be called, which should\nreact on the message and return the updated state of your component.\n\n\\erlfun{gen\\_component}{start}{} and \\erlfun{gen\\_component}{start\\_link}{}\nreturn the pid of the spawned process as \\code{\\{ok, Pid\\}}.\n\n\n\\subsection{\\texorpdfstring{When does a \\erlmodule{gen\\_component} terminate?}\n             {When does a gen\\_component terminate?}}\n\nA \\erlmodule{gen\\_component} can be stopped using:\n\n\\erlfun{gen\\_component}{kill}{Pid} or by returning \\code{kill} from the\ncurrent message handler.\n\n\\subsection{\\texorpdfstring{How to determine whether a process is a \\erlmodule{gen\\_component}?}\n             {How to determine whether a process is a gen\\_component?}}\n\nA \\erlmodule{gen\\_component} can be detected by:\n\n\\erlfun{gen\\_component}{is\\_gen\\_component}{Pid}, which returns a boolean.\n\n\\subsection{What happens when unexpected events / messages arrive?}\n\nYour message handler (default is \\erlfun{your\\_gen\\_component}{on}{/2})\nshould return \\code{unknown_event} in the final clause\n(\\erlfun{your\\_gen\\_component}{on}{\\_,\\_}).  \\erlmodule{gen\\_component} then\nwill nicely report on the unhandled message, the component's name, its state\nand currently active message handler, as shown in the following example:\n\n\\begin{lstlisting}[language=bash]\n# bin/boot.sh\n[...]\n(boot@localhost)10> pid_groups ! {no_message}.\n{no_message}\n[error] unknown message: {no_message} in Module: pid_groups and\nhandler on in State null\n(boot@localhost)11>\n\\end{lstlisting}\n\nThe \\erlmodule{pid\\_groups} (see\nSection~\\ref{sec:pid_groups}) is a \\erlmodule{gen\\_component} which\nregisters itself as named Erlang process with the \\erlmodule{gen\\_component}\noption \\code{erlang_register} and therefore can be addressed by its name in\nthe Erlang shell. We send it a \\code{\\{no_message\\}} and\n\\erlmodule{gen\\_component} reports on the unhandled message. The\n\\erlmodule{pid\\_groups} module itself continues to run and waits for\nfurther messages.\n\n\\subsection{What if my message handler generates an exception or\n crashes the process?}\n\n\\erlmodule{gen\\_component} catches exceptions generated by message handlers\nand reports them with a stack trace, the message, that generated the\nexception, and the current state of the component.\n\nIf a message handler terminates the process via \\erlfun{erlang}{exit}{/1},\nthis is out of the responsibility scope of \\erlmodule{gen\\_component}. As\nusual in Erlang, all linked processes will be informed. If for example\n\\erlfun{gen\\_component}{start\\_link}{/2} or \\texttt{/3} was used for\nstarting the \\erlmodule{gen\\_component}, the spawning process will be\ninformed, which may be an Erlang supervisor process taking further actions.\n\n\\subsection{Changing message handlers and implementing state dependent\n message responsiveness as a state-machine}\n\\label{sec:gen_component:change_handler}\n\\erlfunindex{gen\\_component}{change\\_handler}\n\nSometimes it is beneficial to handle messages depending on the state of a\ncomponent. One possibility to express this is implementing different clauses\ndepending on the state variable, another is introducing case clauses inside\nmessage handlers to distinguish between current states. Both approaches may\nbecome tedious, error prone, and may result in confusing source code.\n\nSometimes the use of several different message handlers for different states\nof the component leads to clearer arranged code, especially if the set of\nhandled messages changes from state to state. For example, if we have a\ncomponent with an initialization phase and a production phase afterwards, we\ncan handle in the first message handler messages relevant during the\ninitialization phase and simply queue all other requests for later\nprocessing using a common default clause.\n\nWhen initialization is done, we handle the queued user requests and switch\nto the message handler for the production phase. The message handler for the\ninitialization phase does not need to know about messages occurring during\nproduction phase and the message handler for the production phase does not\nneed to care about messages used during initialization. Both handlers can be\nmade independent and may be extended later on without any adjustments to the\nother.\n\nOne can also use this scheme to implement complex state-machines by changing\nthe message handler from state to state.\n\nTo switch the message handler\n\\erlfun{gen\\_component}{change\\_handler}{State, new\\_handler} is called as\nthe last operation after a message in the active message handler was\nhandled, so that the return value of\n\\erlfun{gen\\_component}{change\\_handler}{/2} is propagated to\n\\erlmodule{gen\\_component}.  The new handler is given as an atom, which is\nthe name of the 2-ary function in your component module to be called.\n\n\\subsubsection{Starting with non-default message handler.}\nIt is also possible to change the message handler right from the start\nin your \\erlfun{your\\_gen\\_component}{init}{/1} to avoid the default message\nhandler \\erlfun{your\\_gen\\_component}{on}{/2}. Just create your initial\nstate as usual and call \\erlfun{gen\\_component}{change\\_handler}{State,\n  my\\_handler} as the final call in your\n\\erlfun{your\\_gen\\_component}{init}{/1}. We prepared\n\\erlfun{gen\\_component}{change\\_handler}{/2} to return \\code{State} itself,\nso this will work properly.\n\n\\subsection{Handling several messages atomically}\n\nThe message handler is called for each message separately. Such a single\ncall is atomic, i.e. the component does not perform any other action until\nthe called message handler finishes. Sometimes, it is necessary to execute\ntwo or more calls to the message handler atomically (without other\ninterleaving messages). For example if a message \\code{A} contains another\nmessage \\code{B} as payload, it may be necessary to handle \\code{A} and\n\\code{B} directly one after the other without interference of other\nmessages. So, after handling \\code{A} you want to call your message handler\nwith \\code{B}.\n\nIn most cases, you could just do so by calculating the new state as result\nof handling message \\code{A} first and then calling the message handler with\nmessage \\code{B} and the new state by yourself.\n\nIt is safer to use \\erlfun{gen\\_component}{post\\_op}{2} in such cases: When\n$B$ contains a special message, which is usually handled by the\n\\erlmodule{gen\\_component} module itself (like\n\\code{send\\_to\\_group\\_member}, \\code{kill}, \\code{sleep}), the direct call\nto the message handler would not achieve the expected result. By calling\n\\erlfun{gen\\_component}{post\\_op}{B, NewState} to return the new state after\nhandling message \\code{A}, message \\code{B} will be handled directly after\nthe current message \\code{A}.\n\n\\subsection{\\texorpdfstring{Halting and pausing a \\erlmodule{gen\\_component}}\n{Halting and pausing a gen\\_component}}\n\nUsing \\erlfun{gen\\_component}{kill}{Pid} and\n\\erlfun{gen\\_component}{sleep}{Pid, Time} components can be terminated or\npaused.\n\n\\subsection{\\texorpdfstring{Integration with \\erlmodule{pid\\_groups}:\n  Redirecting messages  to other \\erlmodule{gen\\_component}s}\n  {Integration with pid\\_groups:\n  Redirecting messages  to other gen\\_components}}\n\nEach \\erlmodule{gen\\_component} by itself is prepared to support\n\\erlfun{comm}{send\\_to\\_group\\_member}{/3} which forwards messages inside a\ngroup of processes registered via \\erlmodule{pid\\_groups} (see\nSection~\\ref{sec:pid_groups}) by their name. So, if you hold a Pid\nof one member of a process group, you can send messages to other members of\nthis group, if you know their registered Erlang name. You do not necessarily\nhave to know their individual Pid.\n\n\\emph{In consequence, no \\erlmodule{gen\\_component} can individually handle\n  messages of the form \\code{\\{send_to_group_member,} \\code{_, _\\}} as such\n  messages are consumed by \\erlmodule{gen\\_component} itself.}\n\n\\subsection{\\texorpdfstring{Replying to \\code{ping} messages}\n  {Replying to ping messages}}\n\nEach \\erlmodule{gen\\_component} replies automatically to \\code{\\{ping,\n  Pid\\}} requests with a \\code{\\{pong\\}} send to the given \\code{Pid}.  Such\nmessages are generated, for example, by \\erlmodule{vivaldi\\_latency} which is used\nby our \\erlmodule{vivaldi} module.\n\n\\emph{In consequence, no \\erlmodule{gen\\_component} can individually handle\nmessages of the form: \\code{\\{ping, _\\}} as such messages are consumed by\n\\erlmodule{gen\\_component} itself.}\n\n\n\\subsection{\\texorpdfstring{The debugging interface of \\erlmodule{gen\\_component}:\n  Breakpoints and step-wise execution}\n  {The debugging interface of gen\\_component: Breakpoints and step-wise execution}}\n\nWe equipped \\erlmodule{gen\\_component} with a debugging interface, which\nespecially is beneficial, when testing the interplay between several\n\\erlmodule{gen\\_component}s. It supports breakpoints (bp) which can pause the\n\\erlmodule{gen\\_component} depending on the arriving messages or depending\non user defined conditions. If a breakpoint is reached, the execution can be\ncontinued step-wise (message by message) or until the next breakpoint is\nreached.\n\nWe use it in our unit tests to steer protocol interleavings and to perform\ntests using random protocol interleavings between several processes\n(see~\\erlmodule{paxos\\_SUITE}). It allows also to reproduce given protocol\ninterleavings for better testing.\n\n\\subsubsection{Managing breakpoints.}\n\nBreakpoints are managed by the following functions:\n\n\\begin{description}\n\\item[\\erlfun{gen\\_component}{bp\\_set}{Pid, MsgTag, BPName}:] For the\n  component running under \\code{Pid} a breakpoint \\code{BPName} is set. It\n  is reached, when a message with a message tag \\code{MsgTag} is next to be\n  handled by the component (See \\erlfun{comm}{get\\_msg\\_tag}{/1} and\n  Section~\\ref{sec:comm} for more information on message tags). The\n  \\code{BPName} is used as a reference for this breakpoint, for example to\n  delete it later.\n\\item[\\erlfun{gen\\_component}{bp\\_set\\_cond}{Pid, Cond, BPName}:]\n  The same as \\erlfun{gen\\_component}{bp\\_set}{/3} but a user defined\n  condition implemented in \\code{\\{Module, Function, Params = 2\\} = Cond} is\n  checked by calling \\code{Module:Function(Message, State)} to decide\n  whether a breakpoint is reached or not. \\code{Message} is the next message\n  to be handled by the component and \\code{State} is the current state of\n  the component. \\code{Module:Function/2} should return a \\code{boolean}.\n\\item[\\erlfun{gen\\_component}{bp\\_del}{Pid, BPName}:] The breakpoint\n  \\code{BPName} is deleted. If the component is in this breakpoint, it will\n  not be released by this call. This has to be done separately by\n  \\erlfun{gen\\_component}{bp\\_cont}{/1}. But the deleted breakpoint will no\n  longer be considered for newly entering a breakpoint.\n\\item[\\erlfun{gen\\_component}{bp\\_barrier}{Pid}:]\n  Delay all further handling of breakpoint requests until a breakpoint is\n  actually entered.\n\n  \\emph{Note, that the following call sequence may not catch the breakpoint at\n  all, as during the sleep the component not necessarily consumes a\n  \\code{ping} message and the set breakpoint `\\code{sample_bp}' may already\n  be deleted before a ping message arrives.}\n\n  \\begin{lstlisting}\n  gen_component:bp_set(Pid, ping, sample_bp),\n  timer:sleep(10),\n  gen_component:bp_del(Pid, sample_bp),\n  gen_component:bp_cont(Pid).\n  \\end{lstlisting}\n\n  \\emph{To overcome this, \\erlfun{gen\\_component}{bp\\_barrier}{/1} can be used:}\n\n  \\begin{lstlisting}\n  gen_component:bp_set(Pid, ping, sample_bp),\n  gen_component:bp_barrier(Pid),\n  %% After the bp_barrier request, following breakpoint requests\n  %% will not be handled before a breakpoint is actually entered.\n  %% The gen_component itself is still active and handles messages as usual\n  %% until it enters a breakpoint.\n  gen_component:bp_del(Pid, sample_bp),\n  % Delete the breakpoint after it was entered once (ensured by bp_barrier).\n  % Release the gen_component from the breakpoint and continue.\n  gen_component:bp_cont(Pid).\n  \\end{lstlisting}\n\\end{description}\n\nNone of the calls in the sample listing above is blocking. It just schedules\nall the operations, including the \\code{bp_barrier}, for the\n\\erlmodule{gen\\_component} and immediately finishes. The actual events of\nentering and continuing the breakpoint in the \\erlmodule{gen\\_component}\nhappens independently later on, when the next \\code{ping} message arrives.\n\n\\subsubsection{Managing execution.}\n\nThe execution of a \\erlmodule{gen\\_component} can be managed by the\nfollowing functions:\n\n\\begin{description}\n\\item[\\erlfun{gen\\_component}{bp\\_step}{Pid}:] This is the only blocking\n  breakpoint function. It waits until the \\erlmodule{gen\\_component} is in a\n  breakpoint and has handled a single message.  It returns the module, the\n  active message handler, and the handled message as a tuple \\code{\\{Module,\n    On, Message\\}}.  This function does not actually finish the breakpoint,\n  but just lets a single message pass through. For further messages, no\n  breakpoint condition has to be valid, the original breakpoint is still\n  active. To leave a breakpoint, use \\erlfun{gen\\_component}{bp\\_cont}{/1}.\n\n\\item[\\erlfun{gen\\_component}{bp\\_cont}{Pid}:]\n  Leaves a breakpoint. \\erlmodule{gen\\_component} runs as usual until the\n  next breakpoint is reached.\n\n  If no further breakpoints should be entered after continuation, you should\n  delete the registered breakpoint using \\erlfun{gen\\_component}{bp\\_del}{/2}\n  before continuing the execution with\n  \\erlfun{gen\\_component}{bp\\_cont}{/1}. To ensure, that the breakpoint is\n  entered at least once, \\erlfun{gen\\_component}{bp\\_barrier}{/1} should be\n  used before deleting the breakpoint (see the example above). Otherwise it\n  could happen, that the delete request arrives at your\n  \\erlmodule{gen\\_component} before it was actually triggered. The following\n  continuation request would then unintentional apply to an unrelated\n  breakpoint that may be entered later on.\n\n\\item[\\erlfun{gen\\_component}{runnable}{Pid}:] Returns whether a\n  \\erlmodule{gen\\_component} has messages to handle and is runnable. If you\n  know, that a \\erlmodule{gen\\_component} is in a breakpoint, you can use\n  this to check, whether a \\erlfun{gen\\_component}{bp\\_step}{/1} or\n  \\erlfun{gen\\_component}{bp\\_cont}{/1} is applicable to the component.\n\n\\end{description}\n\n\\subsubsection{Tracing handled messages -- getting a message  interleaving protocol.}\n\nWe use the debugging interface of \\erlmodule{gen\\_component} to test\nprotocols with random interleaving. First we start all the components\ninvolved, set breakpoints on the initialization messages for a new Paxos\nconsensus and then start a single Paxos instance on all of them. The outcome\nof the Paxos consensus is a \\code{learner_decide} message. So, in\n\\erlfun{paxos\\_SUITE}{step\\_until\\_decide}{/3} we look for runnable processes\nand select randomly one of them to perform a single step until the protocol\nfinishes with a decision.\n\n\\codesnippet{paxos_SUITE.erl}{paxos_SUITE:random_interleaving_test}{../test/paxos_SUITE.erl}\n\nTo get a message interleaving protocol, we either can output the results\nof each \\erlfun{gen\\_component}{bp\\_step}{/1} call together with the\nPid we selected for stepping, or alter the definition of the macro\n\\code{TRACE_BP_STEPS} in \\erlmodule{gen\\_component}, when we execute all\n\\erlmodule{gen\\_component}s locally in the same Erlang virtual machine.\n\n\\codesnippet{gen_component.erl}{gen_component:trace_bp_steps}{../src/gen_component.erl}\n\n\n%% \\subsection{Profiling \\erlmodule{gen\\_component}s.}\n%% \n%% Using the profiling feature, see also Section~\\ref{sec:digging} into the\n%% details'\n%% \n%% The profiling currently measures global time, which is actually not a\n%% good metric for profiling, as it depends on the scheduling and not all\n%% time recorded is actually used by the handler. Additionally\n%% \\erlfun{erlang}{now}{} seems to be slow, as it involves a system call to\n%% get a monotonic clock information (which internally triggers a lock\n%% across all Erlang execution threads at least in the Erlang runtime\n%% environment up to R1304).\n%% \n%% \\todo{The metric should be changed to something more\n%%   appropriate. Probably the number of reductions this process has done\n%%   (via \\erlfun{erlang}{process\\_info}{}).}\n%% \n%% \\todo{\n%%   Maybe we should add some functions as interface to read the\n%%   profiling results instead of expecting the user to know how to read\n%%   it directly from the \\erlmodule{ets} table.}\n\n\\subsection{\\texorpdfstring{Future use and planned extensions for \\erlmodule{gen\\_component}}\n             {Future use and planned extensions for gen\\_component}}\n\n\\erlmodule{gen\\_component} could be further extended. For example it could\nsupport hot-code upgrade or could be used to implement algorithms that\nhave to be run across several components of \\scalaris{} like snapshot\nalgorithms or similar extensions.\n\n%% \\subsection{The \\erlmodule{gen\\_component} API for reference}\n\n\n\n%%% Local Variables: \n%%% mode: latex\n%%% End: \n\n"
  },
  {
    "path": "user-dev-guide/latex-col.awk",
    "content": "# warnings einfrben\n/Warning/ { print \"\\033[1;34m\" $0 \"\\033[0m\" }\n/Error/ { print \"\\033[1;31m\" $0 \"\\033[0m\" }\n/undefined/ { print \"\\033[1;32m\" $0 \"\\033[0m\" }\n/Overfull/ { print \"\\033[1;36m\" $0 \"\\033[0m\" }\n/Underfull/ { print \"\\033[1;36m\" $0 \"\\033[0m\" }\n# geladene Files verbergen\n#/^\\(/ {}\n#/pipe/ {}\n# den Rest ausgeben\n$0 !~ /Warning|bibitem|duplicate|Overfull|Underfull|Error|Undefined|document|^$|inconsistent|pdfTeX|\\[\\]|^\\(/ { print }\n"
  },
  {
    "path": "user-dev-guide/letltxmacro.sty",
    "content": "%%\n%% This is file `letltxmacro.sty',\n%% generated with the docstrip utility.\n%%\n%% The original source files were:\n%%\n%% letltxmacro.dtx  (with options: `package')\n%% \n%% This is a generated file.\n%% \n%% Copyright (C) 2008 by\n%%    Heiko Oberdiek <heiko.oberdiek at googlemail.com>\n%% \n%% This work may be distributed and/or modified under the\n%% conditions of the LaTeX Project Public License, either\n%% version 1.3c of this license or (at your option) any later\n%% version. This version of this license is in\n%%    http://www.latex-project.org/lppl/lppl-1-3c.txt\n%% and the latest version of this license is in\n%%    http://www.latex-project.org/lppl.txt\n%% and version 1.3 or later is part of all distributions of\n%% LaTeX version 2005/12/01 or later.\n%% \n%% This work has the LPPL maintenance status \"maintained\".\n%% \n%% This Current Maintainer of this work is Heiko Oberdiek.\n%% \n%% This work consists of the main source file letltxmacro.dtx\n%% and the derived files\n%%    letltxmacro.sty, letltxmacro.pdf, letltxmacro.ins, letltxmacro.drv,\n%%    letltxmacro-showcases.tex, letltxmacro-test1.tex,\n%%    letltxmacro-test2.tex.\n%% \n\\begingroup\n  \\catcode123 1 % {\n  \\catcode125 2 % }\n  \\def\\x{\\endgroup\n    \\expandafter\\edef\\csname llm@AtEnd\\endcsname{%\n      \\catcode35 \\the\\catcode35\\relax\n      \\catcode64 \\the\\catcode64\\relax\n      \\catcode123 \\the\\catcode123\\relax\n      \\catcode125 \\the\\catcode125\\relax\n    }%\n  }%\n\\x\n\\catcode35 6 % #\n\\catcode64 11 % @\n\\catcode123 1 % {\n\\catcode125 2 % }\n\\def\\TMP@EnsureCode#1#2{%\n  \\edef\\llm@AtEnd{%\n    \\llm@AtEnd\n    \\catcode#1 \\the\\catcode#1\\relax\n  }%\n  \\catcode#1 #2\\relax\n}\n\\TMP@EnsureCode{40}{12}% (\n\\TMP@EnsureCode{41}{12}% )\n\\TMP@EnsureCode{42}{12}% *\n\\TMP@EnsureCode{45}{12}% -\n\\TMP@EnsureCode{46}{12}% .\n\\TMP@EnsureCode{47}{12}% /\n\\TMP@EnsureCode{58}{12}% :\n\\TMP@EnsureCode{61}{12}% =\n\\TMP@EnsureCode{62}{12}% >\n\\edef\\llm@AtEnd{%\n  \\llm@AtEnd\n  \\escapechar\\the\\escapechar\\relax\n}\n\\escapechar=92 % `\\\\\n\\NeedsTeXFormat{LaTeX2e}\n\\ProvidesPackage{letltxmacro}%\n  [2008/06/24 v1.3 Let assignment for LaTeX macros (HO)]\n\\newcommand*{\\LetLtxMacro}[2]{%\n  \\edef\\reserved@a{%\n    \\noexpand\\protect\n    \\expandafter\\noexpand\n    \\csname\\expandafter\\@gobble\\string#2 \\endcsname\n  }%\n  \\ifx\\reserved@a#2\\relax\n    \\edef#1{%\n      \\noexpand\\protect\n      \\expandafter\\noexpand\n      \\csname\\expandafter\\@gobble\\string#1 \\endcsname\n    }%\n    \\expandafter\\let\n    \\csname\\expandafter\\@gobble\\string#1 \\expandafter\\endcsname\n    \\csname\\expandafter\\@gobble\\string#2 \\endcsname\n    \\expandafter\\llm@LetLtxMacro\n        \\csname\\expandafter\\@gobble\\string#1 \\expandafter\\endcsname\n        \\csname\\expandafter\\@gobble\\string#2 \\endcsname\n  \\else\n    \\llm@LetLtxMacro{#1}{#2}%\n  \\fi\n}\n\\def\\llm@LetLtxMacro#1#2{%\n  \\expandafter\\llm@CheckParams\\meaning#2:->\\@nil{%\n    \\begingroup\n      \\def\\@protected@testopt{%\n        \\expandafter\\@testopt\\@gobble\n      }%\n      \\def\\@testopt##1##2{%\n        \\toks@={##2}%\n      }%\n      \\let\\llm@testopt\\@empty\n      \\edef\\x{%\n        \\noexpand\\@protected@testopt\n        \\noexpand#2%\n        \\expandafter\\noexpand\\csname\\string#2\\endcsname\n      }%\n      \\expandafter\\expandafter\\expandafter\\def\n      \\expandafter\\expandafter\\expandafter\\y\n      \\expandafter\\expandafter\\expandafter{%\n        \\expandafter\\llm@CarThree#2{}{}{}\\llm@nil\n      }%\n      \\ifx\\x\\y\n        #2%\n        \\def\\llm@testopt{%\n          \\noexpand\\@protected@testopt\n          \\noexpand#1%\n        }%\n      \\else\n        \\edef\\x{%\n          \\noexpand\\@testopt\n          \\expandafter\\noexpand\\csname\\string#2\\endcsname\n        }%\n        \\expandafter\\expandafter\\expandafter\\def\n        \\expandafter\\expandafter\\expandafter\\y\n        \\expandafter\\expandafter\\expandafter{%\n          \\expandafter\\llm@CarTwo#2{}{}\\llm@nil\n        }%\n        \\ifx\\x\\y\n          #2%\n          \\def\\llm@testopt{%\n            \\noexpand\\@testopt\n          }%\n        \\fi\n      \\fi\n      \\ifx\\llm@testopt\\@empty\n      \\else\n        \\llm@protected\\xdef\\llm@GlobalTemp{%\n          \\llm@testopt\n          \\expandafter\\noexpand\\csname\\string#1\\endcsname\n          {\\the\\toks@}%\n        }%\n      \\fi\n    \\expandafter\\endgroup\\ifx\\llm@testopt\\@empty\n      \\let#1=#2\\relax\n    \\else\n      \\let#1=\\llm@GlobalTemp\n      \\expandafter\\let\\csname\\string#1\\expandafter\\endcsname\n                      \\csname\\string#2\\endcsname\n    \\fi\n  }{%\n    \\let#1=#2\\relax\n  }%\n}\n\\def\\llm@CheckParams#1:->#2\\@nil{%\n  \\begingroup\n    \\def\\x{#1}%\n  \\ifx\\x\\llm@macro\n    \\endgroup\n    \\def\\llm@protected{}%\n    \\expandafter\\@firstoftwo\n  \\else\n    \\ifx\\x\\llm@protectedmacro\n      \\endgroup\n      \\def\\llm@protected{\\protected}%\n      \\expandafter\\expandafter\\expandafter\\@firstoftwo\n    \\else\n      \\endgroup\n      \\expandafter\\expandafter\\expandafter\\@secondoftwo\n    \\fi\n  \\fi\n}\n\\def\\llm@macro{macro}\n\\@onelevel@sanitize\\llm@macro\n\\def\\llm@protectedmacro{\\protected macro}\n\\@onelevel@sanitize\\llm@protectedmacro\n\\def\\llm@CarThree#1#2#3#4\\llm@nil{#1#2#3}%\n\\def\\llm@CarTwo#1#2#3\\llm@nil{#1#2}%\n\\llm@AtEnd\n\\endinput\n%%\n%% End of file `letltxmacro.sty'.\n"
  },
  {
    "path": "user-dev-guide/main.tex",
    "content": "% Copyright 2007-2016 Zuse Institute Berlin\n\n% Licensed under the Apache License, Version 2.0 (the \"License\");\n% you may not use this file except in compliance with the License.\n% You may obtain a copy of the License at\n%\n%     http://www.apache.org/licenses/LICENSE-2.0\n%\n% Unless required by applicable law or agreed to in writing, software\n% distributed under the License is distributed on an \"AS IS\" BASIS,\n% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n% See the License for the specific language governing permissions and\n% limitations under the License.\n\\newcommand{\\scalaris}{Scalaris}\n\\newcommand{\\doctitle}{\\scalaris{}: Users and Developers Guide}\n\\newcommand{\\docauthor}{Zuse Institute Berlin, onScale solutions}\n\\newcommand{\\docsubject}{\\scalaris{}}\n\\newcommand{\\doccreator}{LaTeX2e and pdfLaTeX with hyperref-package}\n\\newcommand{\\docproducer}{}\n\\newcommand{\\dockeywords}{\\scalaris{}, P2P, DHT, Manual}\n\\newcommand{\\doccopyright}{Copyright 2007-\\the\\year{} Zuse Institute Berlin}\n\\newcommand{\\doclicenseurl}{http://www.apache.org/licenses/LICENSE-2.0}\n\\newcommand{\\docversion}{\\input{../VERSION}}\n\n\\title{\\doctitle}\n\n\\documentclass[a4paper]{scrreprt}\n\\usepackage{typearea}\n\\areaset[1cm]{165mm}{240mm}\n\n\\usepackage[T1]{fontenc}\n\\usepackage[latin1]{inputenc}\n\n\\usepackage{relsize}\n\\usepackage{graphicx}\n%\\usepackage{color}\n%\\usepackage{colortbl}\n\\usepackage{pifont} % for carriage return symbol\n\\usepackage{longtable}\n\\usepackage{array}\n\\usepackage{booktabs}\n\\usepackage{multirow}\n\\usepackage{makeidx}\n\\usepackage{ifthen}\n\\usepackage{fancyhdr}\n\\usepackage{fancyvrb}\n\\usepackage{lastpage}\n\\usepackage{relsize}\n\\usepackage{xcolor}\n\\usepackage{threeparttable}\n\n\\input{pdfoptions}\n\n\\usepackage{listings}\n\\usepackage{etextools}\n\\usepackage{tikz}\n\\usetikzlibrary{positioning}\n\\usetikzlibrary{shadows}\n\\usetikzlibrary{fit}\n\\usetikzlibrary{shapes.arrows}\n\\usetikzlibrary{backgrounds}\n\\usetikzlibrary{calc}\n\n\\renewcommand{\\headrulewidth}{0pt}    % Width of head rule\n\\renewcommand{\\footrulewidth}{0.3pt}  % Width of head rule\n\n\\usepackage{calc}\n\n% normal pages\n\\pagestyle{fancy}\n\\fancyhf{} % clear all header and footer fields\n%\\fancyhead[RE,LO]{}\n\\fancyhead[R]{}%\n\\fancyfoot[R]{\\bfseries\\thepage\\ / \\pageref{LastPage}}%\n\\chead{}%\n\\cfoot{}%\n\n% beginning of a chapter\n\\fancypagestyle{plain}{%\n\\fancyhf{} % clear all header and footer fields\n%\\fancyhead[RE,LO]{}\n\\fancyhead[R]{}%\n\\fancyfoot[R]{\\bfseries\\thepage\\ / \\pageref{LastPage}}%\n\\chead{}%\n\\cfoot{}%\n}\n\n% Clear Header Style on the Last Empty Odd pages\n\\makeatletter\n\\def\\cleardoublepage{\\clearpage\\if@twoside \\ifodd\\c@page\\else%\n    \\hbox{}%\n    \\thispagestyle{empty}%              % Empty header styles\n    \\newpage%\n    \\if@twocolumn\\hbox{}\\newpage\\fi\\fi\\fi}\n\\makeatother\n\n%% Bold typewriter font\n%\\renewcommand{\\ttdefault}{pcr}\n%\\renewcommand{\\rmdefault}{ptm} % Times\n%\\renewcommand{\\rmdefault}{ppl} % Palatino\n%%\\renewcommand{\\rmdefault}{pnc} % NewCenturySchoolbook\n%%\\renewcommand{\\rmdefault}{pbk} % BookMan\n%\\renewcommand{\\rmdefault}{pag} % Avantgarde (sans serif)\n%% \\renewcommand{\\rmdefault}{pzc} % ZapfChancery (slanted)\n%\\renewcommand{\\rmdefault}{put} % Utopia\n\\renewcommand{\\rmdefault}{bch} % CharterBT\n\n\\newcommand{\\carriagereturn}{\\scriptsize\\Pisymbol{psy}{191}}\n\n\n%%% colors %%%%%%%%%%%%%%%%%%%%%%%%\n\n\\definecolor{lightyellow}{rgb}{1.0, 1.0, 0.5}\n\\definecolor{rltred}{rgb}{0.75,0,0}\n\\definecolor{rltgreen}{rgb}{0,0.5,0}\n\n\n%\\definecolor{rltblue}{rgb}{0,0,0.75}\n\\definecolor{rltblue}{HTML}{1F4980}\n%\\definecolor{rltred}{HTML}{80491F}\n%\\definecolor{rltred}{HTML}{C35E22}\n\\definecolor{lightgray}{gray}{0.9}\n\n\\setlength{\\parindent}{0pt}\n\n% 31 73 128 blue\n\n\\definecolor{lightyellow}{rgb}{1.0, 1.0, 0.5}\n\\definecolor{codebackground}{HTML}{EEEEEE}\n\\definecolor{commandinput}{rgb}{0.8,0.8,1}\n\n%\\thicklines\n\\lstset{\n  basicstyle=\\scriptsize\\ttfamily,\n  backgroundcolor=\\color{codebackground},\n  keywordstyle=\\color{rltblue}\\bfseries,\n  % underlined bold black keywords\n  identifierstyle=\\bfseries, % nothing happens\n  commentstyle=\\color{rltred}\\bfseries, % white comments\n  stringstyle=\\sffamily, % typewriter type for strings\n  showstringspaces=false,\n  xleftmargin=3pt,\n  xrightmargin=3pt,\n  fancyvrb=true,\n  frame=single,\n%  frameround=tttt,\n%  framexleftmargin=0pt,\n  framextopmargin=3pt,\n  framexbottommargin=3pt,\n%  framexrightmargin=5pt,\n  rulecolor=\\color{codebackground},\n  language=erlang,\n%  fillcolor=\\color{red},\n%  rulesepcolor=\\color{black}\n%  rulesep=1cm,\n}\n\\lstset{rangebeginprefix=\\%\\%\\ userdevguide-begin\\ }\n\\lstset{rangeendprefix=\\%\\%\\ userdevguide-end\\ }\n\\lstset{includerangemarker=false}\n\n% \\codesnippet[lstsettings]{filename-label}{range-label}{src-file with path}\n% \\codesnippet[language=erlang]{dht_node.erl}{dht_node:start_link}{../src/dht_node.erl}\n\\newcommand{\\codesnippet}[4][language=erlang]{\n{%% File: \\url{#4}\n\\lstset{numbers=left}\n\\lstinputlisting[\n  title=\\filetitle{#2},\n  linerange=#3-#3,\n  #1\n]\n{#4}\n}\n}\n\n% \\codefile[lstsettings]{filename-label}{src-file with path}\n% \\codefile[language=erlang]{admin.erl}{../src/admin.erl}\n\\newcommand{\\codefile}[3][language=erlang]{\n{\n\\lstinputlisting[\n  title=\\filetitle{#2},\n  #1]\n{#3}\n}\n}\n\\newcommand{\\bfref}[1]{\\textbf{\\hyperpage{#1}}}\n\\newcommand{\\sieheref}[1]{\\ref{#1} on page~\\pageref{#1}}\n\\newcommand{\\code}[1]{\\lstinline[basicstyle=\\ttfamily]!#1!}\n\\newcommand{\\filetitle}[1]{\\hbox to \\linewidth{~~File \\code{#1:}\\hfill}}\n\\newcommand{\\todo}[1]{{\\color{red}{\\em TODO: #1}}}\n\n\\newcommand{\\erlparamsorarity}[1]{%\n\\ifthenelse{\\equal{0}{\\gettokslistindex{/}{#1}}}{\\texttt{#1}}{(\\texttt{#1})}}\n\\newcommand{\\erlfun}[4][index]{%\n\\ifthenelse{\\equal{index}{#1}}%\n{\\index{#2@\\texttt{#2}!#3@\\texttt{#3}}}%\n{}%\n\\texttt{#2}:\\-\\texttt{#3}\\texttt{\\erlparamsorarity{#4}}}\n\n\\newcommand{\\erlfunindex}[2]{%\n\\index{#1@\\texttt{#1}!#2@\\texttt{#2}|bfref}}%\n\n\\newcommand{\\erlmodule}[2][index]{%\n\\ifthenelse{\\equal{index}{#1}}%\n{\\index{#2@\\texttt{#2}}}{}%\n\\texttt{#2}}\n\n\\newcommand{\\erlmoduleindex}[1]{%\n\\index{#1@\\texttt{#1}|bfref}}%\n\n\\makeatletter\n\\newenvironment{erlfunparams}\n{\n\\list{}{\n    \\labelwidth\\z@ \\makeatother \\itemindent-\\leftmargin\n    \\let\\makelabel\\descriptionlabel\n\\setlength\\leftmargin{4em}\n\\setlength{\\parskip}{0pt}\n\\setlength{\\parsep}{0pt}\n\\setlength{\\itemsep}{0pt}\n}}\n{\\endlist}\n\\makeatother\n\\newcommand{\\erlparam}[2]{\\item[\\texttt{#1}:] #2}\n\n\\newenvironment{wikitext}\n{\\catcode`\\#=13\\catcode`\\$=13\\catcode`\\_=11\\catcode`\\==13\n}{\\catcode`\\#=6\\catcode`\\$=3\\catcode`\\_=8}\n\n%shortcut for multi columns\n\\newcommand{\\mc}[3]{\\multicolumn{#1}{#2}{#3}}\n% new left-justified column types\n\\newcolumntype{P}[1]{>{\\raggedright}p{#1}}\n\\newcolumntype{M}[1]{>{\\raggedright}m{#1}}\n\\newcolumntype{B}[1]{>{\\raggedright}m{#1}}\n\\newcommand{\\tn}{\\tabularnewline} % need to use this instead of \\\\ in these columns\n\n% allow footnotes in tables:\n\\usepackage{footnote}\n\\makesavenoteenv{tabular}\n\n\\makeindex\n\n\\begin{document}\n\\enlargethispage{1.6cm}\n\\vspace*{3.8cm}\n\\thispagestyle{empty}\n\\setlength{\\parskip}{1ex}\n\\includegraphics[width=8cm]{scalaris-logo}\n\n\\vfill\n\n\\hfill\\begin{tabular}{@{}l@{}}\n\\mdseries \\Huge \\hfill Users and Developers Guide\\\\\n\\\\ \\\\\n\\LARGE Version \\docversion{} \\hfill \\today\\\\\n\\\\\n\\LARGE \\hfill\\url{http://scalaris.zib.de}\\\\\n\\end{tabular}\n\n\\vspace*{3.8cm}\n\n\\hrule\n\\smallskip\n\\parbox{\\textwidth}{\\scriptsize \\setlength{\\parskip}{1ex}\n\\doccopyright{}.\n\nLicensed under the Apache License, Version 2.0 (the \"License\"); you may not\nuse this file except in compliance with the License.  You may obtain a copy\nof the License at \\url{\\doclicenseurl}\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT\nWARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the\nLicense for the specific language governing permissions and limitations\nunder the License.  }\n\n\\tableofcontents\n\n\\input{user-intro}\n\n\\part{Users Guide}\n\n\\input{user-install}\n\\input{user-config}\n\\input{user-systemuse}\n\\input{user-tests}\n\\input{user-troubleshoot}\n\n\\part{Developers Guide}\n\n\\input{dev-hints}\n\\input{dev-infrastructure}\n\\input{dev-overlay}\n\\input{dev-tx}\n\\input{dev-join}\n\\input{dev-slide}\n\\input{dev-rrepair}\n\\input{dev-dirs}\n\n%\\chapter{System Components}\n\n%\\chapter{Processes}\n\n%\\chapter{Troubleshooting}\n\n%\\section{ApplicationMonitor appmon:start()}\n\n\\input{dev-java}\n\n\\bibliographystyle{plainnat}\n\n\\begin{thebibliography}{9}\n\n\\bibitem{erlang-book}\nJoe Armstrong.\n\\newblock \\emph{Programming Erlang: Software for a Concurrent World.}\n\\newblock Pragmatic Programmers, ISBN: 978-1-9343560-0-5, July 2007\n\n\\bibitem{vivaldi}\nFrank Dabek, Russ Cox, Frans Kaahoek, Robert Morris.\n\\newblock \\emph{Vivaldi: A Decentralized Network Coordinate System.}\n\\newblock ACM SIGCOMM 2004.\n\n\\bibitem{rachid-book}\nRachid Guerraoui and Luis Rodrigues.\n\\newblock \\emph{Introduction to Reliable Distributed Programming.}\n\\newblock Springer-Verlag, 2006.\n\n\\bibitem{chord-sigcomm}\nIon Stoica, Robert Morris, David Karger, M. Frans Kaashoek and Hari Balakrishnan.\n\\newblock \\emph{Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications.}\n\\newblock ACM SIGCOMM 2001, San Deigo, CA, August 2001, pp. 149-160.\n\\newblock \\href{http://pdos.csail.mit.edu/papers/chord:sigcomm01/chord_sigcomm.pdf}{http://pdos.csail.mit.edu/papers/chord:sigcomm01/chord\\_sigcomm.pdf}\n\n\\bibitem{t-man}\nM{\\'a}rk Jelasity, Alberto Montresor, Ozalp Babaoglu.\n\\newblock \\emph{T-Man: Gossip-based fast overlay topology construction.}\n\\newblock Computer Networks (CN) 53(13):2321-2339, 2009.\n\n\\bibitem{frtchord}\n  Hiroya Nagao, Kazuyuki Shudo.\n  \\newblock \\emph{Flexible routing tables: Designing routing\n    algorithms for overlays based on a total order on a routing table set.}\n  \\newblock In: Peer-to-Peer Computing, IEEE, 2011.\n\n\\bibitem{enhanced-paxos}\nF. Schintke, A. Reinefeld, S. Haridi, T. Sch{\\\"u}tt.\n\\newblock \\emph{Enhanced Paxos Commit for Transactions on DHTs.}\n\\newblock 10th IEEE/ACM Int. Conf. on Cluster, Cloud and Grid Computing, pp. 448-454,\nMay 2010.\n\n\\bibitem{cyclon}\nSpyros Voulgaris, Daniela Gavidia, Maarten van Steen.\n\\newblock \\emph{CYCLON: Inexpensive Membership Management for Unstructured P2P Overlays.}\n\\newblock J. Network Syst. Manage. 13(2): 2005.\n\n\\bibitem{gossip}\nM{\\'a}rk Jelasity, Alberto Montresor, Ozalp Babaoglu.\n\\newblock \\emph{Gossip-based aggregation in large dynamic networks.}\n\\newblock ACM Trans. Comput. Syst. 23(3), 219-252 (2005).\n\n\\end{thebibliography}\n\n\\printindex\n\n\\end{document}\n%%% Local Variables: \n%%% mode: latex\n%%% TeX-master: t\n%%% End: \n\n"
  },
  {
    "path": "user-dev-guide/pdfoptions.tex",
    "content": "% Copyright 2011 Zuse Institute Berlin\n\n% Licensed under the Apache License, Version 2.0 (the \"License\");\n% you may not use this file except in compliance with the License.\n% You may obtain a copy of the License at\n%\n%     http://www.apache.org/licenses/LICENSE-2.0\n%\n% Unless required by applicable law or agreed to in writing, software\n% distributed under the License is distributed on an \"AS IS\" BASIS,\n% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n% See the License for the specific language governing permissions and\n% limitations under the License.\n\n\\pdfminorversion=4 % PDF/A is based on PDF 1.4\n\\pdfcompresslevel=9\n\\pdfobjcompresslevel=0 % needed for PDF 1.4\n\n%***************************************************************************\n% \\convertDate converts D:20080419103507+02'00' to 2008-04-19T10:35:07+02:00\n%___________________________________________________________________________\n\\def\\convertDate{%\n    \\getYear\n}\n\n{\\catcode`\\D=12\n\\gdef\\getYear D:#1#2#3#4{\\edef\\xYear{#1#2#3#4}\\getMonth}\n}\n\\def\\getMonth#1#2{\\edef\\xMonth{#1#2}\\getDay}\n\\def\\getDay#1#2{\\edef\\xDay{#1#2}\\getHour}\n\\def\\getHour#1#2{\\edef\\xHour{#1#2}\\getMin}\n\\def\\getMin#1#2{\\edef\\xMin{#1#2}\\getSec}\n\\def\\getSec#1#2{\\edef\\xSec{#1#2}\\getTZh}\n\\def\\getTZh +#1#2{\\edef\\xTZh{#1#2}\\getTZm}\n\\def\\getTZm '#1#2'{%\n    \\edef\\xTZm{#1#2}%\n    \\edef\\convDate{\\xYear-\\xMonth-\\xDay T\\xHour:\\xMin:\\xSec+\\xTZh:\\xTZm}%\n}\n\n\\expandafter\\convertDate\\pdfcreationdate \n\n%**************************\n% get pdftex version string\n%__________________________\n\\newcount\\countA\n\\countA=\\pdftexversion\n\\advance \\countA by -100\n\\def\\pdftexVersionStr{pdfTeX-1.\\the\\countA.\\pdftexrevision}\n\n\n%********\n% pdfInfo\n%________\n\\pdfinfo{%\n    /Title    (\\doctitle)\n    /Author   (\\docauthor)\n    /Subject  (\\docsubject)\n    /Keywords (\\dockeywords)\n    /ModDate  (\\pdfcreationdate)\n    /Trapped  /False%\n}\n\n\\usepackage{hyperxmp}\n\\usepackage[%\n  %%% general options\n  pdfa,                    %% PDF/A-1b compliance\n  pdftex=true,             %% sets up hyperref for use with the pdftex program\n  %plainpages=false,       %% set it to false, if pdflatex complains: ``destination with same identifier already exists''\n  %\n  %%% extension options\n  backref=section,         %% adds a backlink text to the end of each item in the bibliography\n  pagebackref=false,       %% if true, creates backward references as a list of page numbers in the bibliography\n  colorlinks=true,         %% false: boxed links; true: colored links\n  linkcolor=rltblue,       %% color of internal links\n%  citecolor=rltblue,     %% color of links to bibliography\n  filecolor=rltblue,       %% color of file links\n  urlcolor=rltblue,        %% color of external links\n  unicode=false,           %% non-Latin characters in Acrobat's bookmarks (if enabled, causes garbled page numbers)\n  %\n  %%% PDF-specific display options\n  pdfnewwindow=true,       %% links in new window\n  pdftoolbar=true,         %% show Acrobat's toolbar?\n  pdfmenubar=true,         %% show Acrobat's menu?\n  pdffitwindow=true,       %% page fit to window when opened\n  bookmarks=true,          %% if true, generate PDF bookmarks (requires two passes of pdflatex)\n  bookmarksopen=false,     %% if true, show all PDF bookmarks expanded\n  bookmarksnumbered=true,  %% if true, add the section numbers to the bookmarks\n  %pdfstartpage={1},        %% determines, on which page the PDF file is opened\n  pdfpagemode=UseOutlines, %% UseNone, UseOutlines (=show bookmarks), UseThumbs (show thumbnails), FullScreen, UseOC (PDF 1.5), UseAttachments (PDF 1.6)\n  pdfusetitle=true\n]{hyperref} \n\n%%% sets the PDF-Information options\n%%% (see fields in Acrobat Reader: ``File -> Document properties -> Summary'')\n%%% Note: this method is better than as options of the hyperref-package (options are expanded correctly)\n\\hypersetup{\n  pdftitle={\\doctitle},           % title\n  pdfauthor={\\docauthor},         % author\n  pdfsubject={\\docsubject},       % subject of the document\n  pdfcreator={\\doccreator},       % creator of the document\n  pdfproducer={\\docproducer},     % producer of the document\n  pdfkeywords={\\dockeywords},     % list of keywords\n  pdfcopyright={\\doccopyright},   % copyright text\n  pdflicenseurl={\\doclicenseurl}  % URL pointing to the license text\n}\n"
  },
  {
    "path": "user-dev-guide/rrepair/.gitignore",
    "content": "/test*\n/*.pdf\n/*.ps\n"
  },
  {
    "path": "user-dev-guide/rrepair/Makefile",
    "content": "MAKEFLAGS=-k $MAKEFLAGS\nGNUPLOT=gnuplot\nPIC2PLOT=pic2plot -Tps\nGNU_FILES := $(patsubst %,.%-done,$(wildcard *.gnuplot))\nPIC_FILES := $(patsubst %.pic,%.pdf,$(wildcard *.pic))\n\n.PHONY: all files allSims clean\n\nall: files\n\n.DELETE_ON_ERROR:\n\nfiles: $(GNU_FILES) $(PIC_FILES)\n\n# convert all .gnuplot files to .png files\n# % replaces any name\n# within a rule: $< replaces the source\n#                $@ replaces the target\n.%.gnuplot-done: %.gnuplot data-scalability_get_10000_rate-SUMMARY-ALL.txt\n\t$(GNUPLOT) $<\n\ttouch $@\n\nsequence.pdf: sequence.pic\n\t\n\n%.ps: %.pic sequence.pic\n\t$(PIC2PLOT) $< > $@\n#\tnote: if pic2plot is broken (as in openSUSE 12.3), use this with some caveats, e.g. subscript not working:\n# \tpic $< | groff > $@\n\n%.pdf: %.ps\n\tepstopdf $*.ps > $@ && rm $*.ps\n\nclean:\n\trm -f *.png\n\trm -f *.pdf\n\trm -f *.ps\n\trm -f .*.gnuplot-done\n\trm -f *-fit_quadratic\n\trm -f *-fit_linear\n\trm -f *.pic.ps\n\trm -f *.pic.pdf\n"
  },
  {
    "path": "user-dev-guide/rrepair/bloom_sd_uml.pic",
    "content": "# Bloom Reconciliation \n# by Maik Lange, Nico Kruber\n\n.PS\ncopy \"sequence.pic\";\n\n# Objects\nactor(U, \"\");\nobject(A,\"A:RR\");\nplaceholder_object(Dummy0); # more space\nplaceholder_object(AA);\nplaceholder_object(Dummy1); # more space\nplaceholder_object(Dummy2); # more space\nplaceholder_object(BB);\nplaceholder_object(Dummy3); # more space\nobject(B,\"B:RR\");\nstep();\n\n# STEP 1\nmessage(U,A,\"request_sync\"); active(A);\nasync(); # use asynchrone messages (non-filled arrowheads)\nmessage(A,B,\"start_recon(interval I\\sbA\\eb)\"); \ninactive(A);\nactive(B);\n\n# STEP 2\nsync();\ncreate_message(B,BB,\"B:RC\");\nasync();\nmessage(B,BB,\"create_struct(I\\sbA\\eb)\"); \ninactive(B); active(BB);\nstep();\n\ncomment(BB,GET_DB,0.8 left 1.0, wid 1.6 ht 0.25 \\\n        \"get data from DB\");\n\nmessage(BB,A,\"continue_recon(S := {I\\sbAB\\eb, bloom filter BF(\\*b(I\\sbAB\\eb))})\");\ncomment(BB,BUILD_BLOOM,0.0 right 0.2, wid 1.7 ht 0.25 \\\n        \"create bloom filter\");\ninactive(BB);\nactive(A);\nsync();\ncreate_message(A,AA,\"A:RC\");\nasync();\nmessage(A,AA,\"start_recon(S)\");\ninactive(A);\nactive(AA);\nstep();\nconnect_to_comment(AA,GET_DB);\nmessage(AA,BB,\"resolve_req({K\\sbSize\\eb, V\\sbSize\\eb, CKV(\\*D \\ Reg(A,B))})\");\ninactive(AA);\nactive(BB);\nstep();\nmessage(BB,B,\"request_resolve(Upd(*,*))\");\nactive(B);\nmessage(BB,AA,\"resolve_req(V\\sbSize\\eb, CK(Reg(B,A)))\");\nactive(AA);\ncomment(AA,RESOLVE_CH,0.0 left 0.2, wid 2.25 ht 0.225 \\\n        \"decode compressed keys\");\ncomplete(BB);\ndrawx(BB);\nbegin_frame(B,F,\"key_upd_s\");\nend_frame(B,F);\nmessage(AA,A,\"request_resolve(Reg(B,A))\");\ncomplete(AA);\ndrawx(AA);\ninactive(B);\nactive(A);\nstep();\nbegin_frame(A,F,\"key_upd_s\");\nend_frame(A,F);\nstep();\ninactive(A);\n\n# CLEAN UP\nstep();\ncomplete(A);\ncomplete(B);\ncomplete(U);\n\n.PE\n"
  },
  {
    "path": "user-dev-guide/rrepair/key_sync_s.pic",
    "content": "# Resolve Service Operations\n# by Maik Lange, Nico Kruber\n\n.PS\ncopy \"sequence.pic\";\n\n# OBJEKTE\nactor(U,\"\");\nplaceholder_object(Dummy0); # more space\nobject(A,\"A:RR\");\nplaceholder_object(Dummy1); # more space\nplaceholder_object(AA);\nplaceholder_object(Dummy2); # more space\nplaceholder_object(Dummy3); # more space\nplaceholder_object(BB);\nplaceholder_object(Dummy4); # more space\nobject(B,\"B:RR\");\nstep();\n\n# Key_Upd_S\nasync();\nmessage(U,A,\"request_resolve(Op\\sbA1\\eb)\"); active(A);\nstep();\ncomment(A,A_REQUEST,-0.15 right 0.2, wid 2.65 ht 0.7 \\\n        \"Op\\sbA1\\eb := key_upd_s(K, Req)    \"\\\n        \"K := list of keys to sync\\sb \\eb\"\\\n        \"Req := list of keys to request\");\nstep();\n\nbegin_frame(A,F,wid 1.2 \"key_upd_s\");\nsync();\ncreate_message(A,AA,\"A:RS\");\nasync();\nmessage(A,AA,\"start(Op\\sbA1\\eb)\");\ninactive(A); active(AA);\nstep();\n\ncomment(AA,AA_GET_DB,0.3 left 0.2, wid 2.2 ht 0.7 \\\n        \"get data from DB,\"\\\n        \"KVV := list of \\..            \"\\\n        \"  \\.. {key,value,version}\");\nmessage(AA,B,\"request_resolve(Op\\sbB1\\eb := key_upd\\sbFB\\eb(KVV, Req))\"); active(B);\ninactive(AA);\ncomplete(AA);\ndrawx(AA);\nstep();\n\nbegin_frame(BB,F1,wid 1.2 \"key_upd\\sbFB\\eb\");\nsync();\ncreate_message(B,BB,\"B:RS\");\nasync();\nmessage(B,BB,\"start(Op\\sbB1\\eb)\");\ninactive(B); active(BB);\nstep();\ncomment(BB,BB_UPDATE_DB,0.3 right 0.2, wid 2.25 ht 0.475 \\\n        \"update DB & create FB,\"\\\n        \"get Req from DB\");\nmessage(BB,A,\"continue_resolve(Op\\sbA2\\eb := key_upd(KVV\\sbFB\\eb, \\es))\");\nactive(A);\nstep();\n\nbegin_frame(A,F2,wid 1.0 \"key_upd\");\nend_frame(B,F1);\nsync();\ncreate_message(A,AA,\"A:RS\");\nasync();\nmessage(A,AA,\"start(Op\\sbA2\\eb)\");\ninactive(A); active(AA);\nstep();\ncomment(AA,AA_UPDATE_DB,0.0 left 0.2, wid 1.8 ht 0.225 \\\n        \"update data in DB\");\nstep();\ncomplete(AA);\ndrawx(AA);\nstep();\nend_frame(AA,F2);\nstep();\nmessage(BB,A,\"continue_resolve(Op\\sbA3\\eb := key_upd\\sbFB\\eb(KVV\\sbReq\\eb, \\es))\");\nactive(A);\nstep();\ncomplete(BB);\ndrawx(BB);\nbegin_frame(A,F3,\"key_upd\\sbFB\\eb\");\nend_frame(A,F3);\nstep();\ninactive(A);\nstep();\nend_frame(B,F);\n\n# Complete the lifelines\nstep();\ncomplete(U);\ncomplete(A);\ncomplete(B);\n.PE\n"
  },
  {
    "path": "user-dev-guide/rrepair/merkle_sd_uml.pic",
    "content": "# Merkle Tree Reconciliation \n# by Maik Lange, Nico Kruber\n\n.PS\ncopy \"sequence.pic\";\n\n# OBJEKTE\nactor(U, \"\");\nobject(A,\"A:RR\");\nplaceholder_object(Dummy0); # more space\nplaceholder_object(AA);\nplaceholder_object(Dummy1); # more space\nplaceholder_object(Dummy2); # more space\nplaceholder_object(BB);\nplaceholder_object(Dummy3); # more space\nobject(B,\"B:RR\");\nstep();\n\n#---- Step1 RC sends interval to B\nmessage(U,A,\"request_sync\"); active(A);\nasync(); # use asynchrone messages (non-filled arrowheads)\nmessage(A,B,\"start_recon(interval I\\sbA\\eb)\"); \ninactive(A);\nactive(B);\n\n#---- Step2\nsync();\ncreate_message(B,BB,\"B:RC\");\nasync();\nmessage(B,BB,\"create_struct(I\\sbA\\eb)\"); \ninactive(B); active(BB);\nstep();\n\ncomment(BB,BUILD_MERKLE,0.8 left 1.0, wid 1.7 ht 0.45 \\\n        \"get data from DB,\"\\\n        \"create merkle tree\");\n\nmessage(BB,A,\"continue_recon(S := {I\\sbAB\\eb, BranchFactor, BucketSize})\");\ninactive(BB);\nactive(A);\n\n#---- STEP 3\nsync();\ncreate_message(A,AA,\"A:RC\");\nasync();\nmessage(A,AA,\"start_recon(S)\");\ninactive(A);\nactive(AA);\nconnect_to_comment(AA,BUILD_MERKLE);\nstep();\n\nbegin_frame(A,MT,\"Loop\");\n\ncomment(AA,AA_COMPRESS_HASHES,0.0 left 0.1, wid 2.01 ht 0.45 \\\n        \"CH(cur) := compressed\"\\\n        \"hashes of current level\");\nmessage(AA,BB,\"check_nodes(HashSize, CH(cur))\");\ninactive(AA);\nactive(BB);\ncomment(BB,BB_MISMATCHES,0.00 right 0.175, wid 1.6 ht 0.45 \\\n        \"compare hashes,\"\\\n        \"store mismatches\");\n#step();\n#comment(BB,MISMATCHES,0.90 left 0.6, wid 2.325 ht 0.66 \\\n#        \"\\*x\\sb1\\eb) leaf\\sbA\\eb-inner\\sbB\\eb, leaf\\sbA\\eb\\moinner\\sbB\\eb\"\\\n#        \"\\*x\\sb2\\eb) leaf\\sbA\\eb-inner\\sbB\\eb, leaf\\sbA\\eb\\nminner\\sbB\\eb\"\\\n#        \"\\*y) leaf\\sbB\\eb                                \");\nrmessage(BB,AA,\"check_nodes_response(Flags, MaxLeafs\\sbB\\eb)\");\ncomment(BB,MISMATCHES,0.39 left 0.25, wid 3.2 ht 0.45 \\\n        \"\\bu leaf\\sbB\\eb (\\*y)                                               \"\\\n        \"\\bu leaf\\sbA\\eb-inner\\sbB\\eb: leaf\\sbA\\eb\\moi\\sbB\\eb (\\*x\\sb1\\eb), leaf\\sbA\\eb\\nmi\\sbB\\eb (\\*x\\sb2\\eb)\");\nactive(AA);\nstep();\ncomment(AA,AA_MISMATCHES,0.00 left 0.175, wid 1.8 ht 0.625 \\\n        \"calc. new HashSize,\"\\\n        \"interpret Flags,\"\\\n        \"store mismatches\");\nstep();\nconnect_to_comment(AA,MISMATCHES);\nstep();\n#step();\n#step();\n\nend_frame(B, MT);\ninactive(AA);\nstep();\n\ncomment(BB,BB_RESOLVE1,-0.09 right 0.175, wid 1.4 ht 0.225 \\\n        \"\\*d\\sb1\\eb := \\*D \\mo \\*x\\sb1\\eb\");\nmessage(BB,B,\"request_resolve(\\*d\\sb1\\eb)\");\nactive(B);\nmessage(BB,AA,\"resolve_req(CKV(\\*y))\");\ninactive(BB);\nactive(AA);\nbegin_frame(B,F,\"key_upd_s\");\nend_frame(B,F);\ncomment(AA,AA_RESOLVE1A,0.00 right 0.175, wid 2.6 ht 0.45 \\\n        \"\\*d\\sb2\\eb := \\*D \\ Reg(A,B) \\mo CKV(\\*y)\"\\\n        \"\\*d\\sb3\\eb := Reg(A,B) \\mo CKV(\\*y)     \");\nmessage(AA,A,\"request_resolve(\\*d\\sb2\\eb)\");\ninactive(B);\nactive(A);\nstep();\ncomment(AA,AA_RESOLVE1B,0.20 left 0.175, wid 1.65 ht 0.45 \\\n        \"get list(key-value)\"\\\n        \"from leaf\\sbA\\eb \\mo \\*x\\sb2\\eb\");\nbegin_frame(A,F,\"key_upd_s\");\nend_frame(A,F);\nmessage(AA,BB,\"resolve_req(CKV(\\*x\\sb2\\eb), CK(\\*d\\sb3\\eb))\");\ninactive(A);\ninactive(AA); active(BB);\ncomment(BB,BB_RESOLVE2,0.0 right 0.175, wid 2.45 ht 0.45 \\\n        \"\\*d\\sb4\\eb := \\*D \\ Reg(B,A) \\mo CKV(\\*x\\sb2\\eb)\"\\\n        \"\\*d\\sb5\\eb := Reg(B,A) \\mo CKV(\\*x\\sb2\\eb)     \");\nstep();\ncomment(BB,RESOLVE_CK,0.8 left 0.75, wid 2.25 ht 0.225 \\\n        \"decode compressed keys\");\nmessage(BB,B,\"request_resolve(\\*d\\sb3,4\\eb)\");\nactive(B);\nmessage(BB,AA,\"resolve_req(CK(\\*d\\sb5\\eb))\");\nactive(AA);\ncomplete(BB);\ndrawx(BB);\nbegin_frame(B,F,\"key_upd_s\");\nend_frame(B,F);\nconnect_to_comment(AA,RESOLVE_CK);\nstep();\ninactive(B);\nmessage(AA,A,\"request_resolve(\\*d\\sb5\\eb)\");\nactive(A);\ncomplete(AA);\ndrawx(AA);\nstep();\nbegin_frame(A,F,\"key_upd_s\");\nend_frame(A,F);\nstep();\ninactive(A);\n\n# Complete the lifelines\nstep();\ncomplete(A);\ncomplete(B);\ncomplete(U);\n.PE\n"
  },
  {
    "path": "user-dev-guide/rrepair/sequence.pic",
    "content": "#/usr/bin/pic2plot -Tps\n#\n# Pic macros for drawing UML sequence diagrams\n#\n# (C) Copyright 2004-2005 Diomidis Spinellis.\n#\n# Permission to use, copy, and distribute this software and its\n# documentation for any purpose and without fee is hereby granted,\n# provided that the above copyright notice appear in all copies and that\n# both that copyright notice and this permission notice appear in\n# supporting documentation.\n#\n# THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED\n# WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF\n# MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE.\n#\n# $Id$\n#\n\n\n# Default parameters (can be redefined)\n\n# Spacing between messages\nspacing = 0.25;\n# Active box width\nawid = .1;\n# Box height\nboxht = 0.3;\n# Commend folding\ncorner_fold=awid\n# Comment distance\ndefine comment_default_move {up 0.25 right 0.25};\n# Comment height\ncomment_default_ht=0.5;\n# Comment width\ncomment_default_wid=1;\n# Underline object name\nunderline=1;\n\n# Create a new object(name,label)\ndefine object {\n\t$1: box $2; move;\n\t# Could also underline text with \\mk\\ul\\ul\\ul...\\rt\n\tif (underline) then {\n\t\tline from $1.w + (.1, -.07) to $1.e + (-.1, -.07);\n\t}\n\tmove to $1.e;\n\tmove right;\n\t# Active is the level of activations of the object\n\t# 0 : inactive : draw thin line swimlane\n\t# 1 : active : draw thick swimlane\n\t# > 1: nested : draw nested swimlane\n\tactive_$1 = 0;\n\tlifestart_$1 = $1.s.y;\n}\n\n# Create a new external actor(name,label)\ndefine actor {\n\t$1: [\n\t\tXSEQC: circle rad 0.06;\n\t\tXSEQL: line from XSEQC.s down .12;\n\t\tline from XSEQL.start - (.15,.02) to XSEQL.start + (.15,-.02);\n\t\tXSEQL1: line from XSEQL.end left .08 down .15;\n\t\tXSEQL2: line from XSEQL.end right .08 down .15;\n\t\tline at XSEQC.n invis \"\" \"\" \"\" $2;\n\t]\n\tmove to $1.e;\n\tmove right;\n\tactive_$1 = 0;\n\tlifestart_$1 = $1.s.y - .05;\n}\n\n# Create a new placeholder object(name)\ndefine placeholder_object {\n\t$1: box invisible;\n\tmove;\n\tmove to $1.e;\n\tmove right;\n\tactive_$1 = 0;\n\tlifestart_$1 = -123456;\n\t# lifestart_$1 = $1.s.y;\n}\n\ndefine pobject {\n\tplaceholder_object($1);\n}\n\ndefine extend_lifeline {\n\tif (active_$1 > 0) then {\n                # draw the left edges of the boxes\n\t\tmove to ($1.x - awid/2, Here.y);\n\t\tfor level = 1 to active_$1 do {\n\t\t\tline from (Here.x, lifestart_$1) to Here;\n\t\t\tmove right awid/2\n\t\t}\n\n                # draw the right edge of the innermost box\n\t\tmove right awid/2;\n\t\tline from (Here.x, lifestart_$1) to Here;\n\t} else {\n\t\tline from ($1.x, lifestart_$1) to ($1.x, Here.y) dashed;\n\t}\n\tlifestart_$1 = Here.y;\n}\n\n# complete(name)\n# Complete the lifeline of the object with the given name\ndefine complete {\n\textend_lifeline($1)\n\tif (active_$1) then {\n\t\t# draw bottom of all active boxes\n\t\tline right ((active_$1 + 1) * awid/2) from ($1.x - awid/2, Here.y);\n\t}\n}\n\n# Draw a message(from_object,to_object,label)\ndefine message {\n\tdown;\n\tmove spacing;\n\t# Adjust so that lines and arrows do not fall into the\n\t# active box.  Should be .5, but the arrow heads tend to\n\t# overshoot.\n\tif ($1.x <= $2.x) then {\n\t\toff_from = awid * .6;\n\t\toff_to = -awid * .6;\n\t} else {\n\t\toff_from = -awid * .6;\n\t\toff_to = awid * .6;\n\t}\n\n        # add half a box width for each level of nesting\n        if (active_$1 > 1) then {\n                off_from = off_from + (active_$1 - 1) * awid/2;\n        }\n\n        # add half a box width for each level of nesting\n        if (active_$2 > 1) then {\n                off_to = off_to + (active_$2 - 1) * awid/2;\n        }\n\n\tif ($1.x == $2.x) then {\n\t\tarrow from ($1.x + off_from, Here.y) right then down .25 then left $3 ljust \" \" \" \" \" \" ;\n\t} else {\n\t\tarrow from ($1.x + off_from, Here.y) to ($2.x + off_to, Here.y) $3 \" \";\n\t}\n}\n\n# Display a lifeline constraint(object,label)\ndefine lifeline_constraint {\n        off_from = awid;\n        # add half a box width for each level of nesting\n        if (active_$1 > 1) then {\n                off_from = off_from + (active_$1 - 1) * awid/2;\n        }\n\n\tbox at ($1.x + off_from, Here.y) invis $2 ljust \" \" ;\n}\n\ndefine lconstraint {\n\tlifeline_constraint($1,$2);\n}\n\n# Display an object constraint(label)\n# for the last object drawn\ndefine object_constraint {\n\t{ box invis with .s at last box .nw $1 ljust; }\n}\n\ndefine oconstraint {\n\tobject_constraint($1);\n}\n\n# Draw a creation message(from_object,to_object,object_label)\ndefine create_message {\n\tdown;\n\tmove spacing;\n\tif ($1.x <= $2.x) then {\n\t\toff_from = awid * .6;\n\t\toff_to = -boxwid * .51;\n\t} else {\n\t\toff_from = -awid * .6;\n\t\toff_to = boxwid * .51;\n\t}\n\n        # add half a box width for each level of nesting\n        if (active_$1 > 1) then {\n                off_from = off_from + (active_$1 - 1) * awid/2;\n        }\n\n\t# See comment in destroy_message\n\tXSEQA: arrow from ($1.x + off_from, Here.y) to ($2.x + off_to, Here.y) \"create\" \" \";\n\tif ($1.x <= $2.x) then {\n\t\t{ XSEQB: box $3 with .w at XSEQA.end; }\n\t} else {\n\t\t{ XSEQB: box $3 with .e at XSEQA.end; }\n\t}\n\t{\n\t\tline from XSEQB.w + (.1, -.07) to XSEQB.e + (-.1, -.07);\n\t}\n\tlifestart_$2 = XSEQB.s.y;\n\tmove (spacing + boxht) / 2;\n}\n\ndefine cmessage {\n\tcreate_message($1,$2,$3);\n}\n\n# Draw an X for a given object\ndefine drawx {\n\t{\n\tline from($1.x - awid, lifestart_$1 - awid) to ($1.x + awid, lifestart_$1 + awid);\n\tline from($1.x - awid, lifestart_$1 + awid) to ($1.x + awid, lifestart_$1 - awid);\n\t}\n}\n\n# Draw a destroy message(from_object,to_object)\ndefine destroy_message {\n\tdown;\n\tmove spacing;\n\t# The troff code is \\(Fo \\(Fc\n\t# The groff code is also \\[Fo] \\[Fc]\n\t# The pic2plot code is \\Fo \\Fc\n\t# See http://www.delorie.com/gnu/docs/plotutils/plotutils_71.html\n\t# To stay compatible with all we have to hardcode the characters\n\tmessage($1,$2,\"destroy\");\n\tcomplete($2);\n\tdrawx($2);\n}\n\ndefine dmessage {\n\tdestroy_message($1,$2);\n}\n\n# An object deletes itself: delete(object)\ndefine delete {\n\tcomplete($1);\n\tlifestart_$1 = lifestart_$1 - awid;\n\tdrawx($1);\n}\n\n# Draw a message return(from_object,to_object,label)\ndefine return_message {\n\tdown;\n\tmove spacing;\n\t# See comment in message\n\tif ($1.x <= $2.x) then {\n\t\toff_from = awid * .6;\n\t\toff_to = -awid * .6;\n\t} else {\n\t\toff_from = -awid * .6;\n\t\toff_to = awid * .6;\n\t}\n\n        # add half a box width for each level of nesting\n        if (active_$1 > 1) then {\n                off_from = off_from + (active_$1 - 1) * awid/2;\n        }\n\n        # add half a box width for each level of nesting\n        if (active_$2 > 1) then {\n                off_to = off_to + (active_$2 - 1) * awid/2;\n        }\n\n\tarrow from  ($1.x + off_from, Here.y) to ($2.x + off_to, Here.y) dashed $3 \" \";\n}\n\ndefine rmessage {\n\treturn_message($1,$2,$3);\n}\n\n# Object becomes active\n# Can be nested to show recursion\ndefine active {\n\textend_lifeline($1);\n\t# draw top of new active box\n\tline right awid from ($1.x + (active_$1 - 1) * awid/2, Here.y);\n\tactive_$1 = active_$1 + 1;\n}\n\n# Object becomes inactive\n# Can be nested to show recursion\ndefine inactive {\n\textend_lifeline($1);\n\tactive_$1 = active_$1 - 1;\n\t# draw bottom of innermost active box\n\tline right awid from ($1.x + (active_$1 - 1) * awid/2, Here.y);\n}\n\n# Time step\n# Useful at the beginning and the end\n# to show object states\ndefine step {\n\tdown;\n\tmove spacing;\n}\n\n# Switch to asynchronous messages\ndefine async {\n\tarrowhead = 0;\n\tarrowwid = arrowwid * 2;\n}\n\n# Switch to synchronous messages\ndefine sync {\n\tarrowhead = 1;\n\tarrowwid = arrowwid / 2;\n}\n\n# same as lifeline_constraint, but Text and empty string are exchanged.\ndefine lconstraint_below{\n        off_from = awid;\n        # add half a box width for each level of nesting\n        if (active_$1 > 1) then {\n                off_from = off_from + (active_$1 - 1) * awid/2;\n        }\n\n\tbox at ($1.x + off_from, Here.y) invis \"\" $2 ljust;\n}\n\n# begin_frame(left_object,name,label_text);\ndefine begin_frame {\n\t# The lifeline will be cut here\n\tif (lifestart_$1 != -123456) then {\n\t\textend_lifeline($1);\n\t}\n\t# draw the frame-label\n\t$2: box $3 invis with .n at ($1.x, Here.y);\n\td = $2.e.y - $2.se.y;\n\tline from $2.ne to $2.e then down d left d then to $2.sw;\n\t# continue the lifeline below the frame-label\n\tmove to $2.s;\n\tlifestart_$1 = Here.y;\n}\n\n# end_frame(right_object,name);\ndefine end_frame {\n\t# dummy-box for the lower right corner:\n\tbox invis \"\" with .s at ($1.x, Here.y);\n\t# draw the frame\n\tframe_wid = last box.se.x - $2.nw.x\n\tframe_ht = - last box.se.y + $2.nw.y\n\tbox with .nw at $2.nw wid frame_wid ht frame_ht;\n\t# restore Here.y\n\tmove to last box.s;\n}\n\n# comment(object,[name],[line_movement], [box_size] text);\ndefine comment {\n\told_y = Here.y\n\t# draw the first connecting line, at which's end the box wil be positioned\n\tmove to ($1.x, Here.y)\n\tif \"$3\" == \"\" then {\n\t\tline comment_default_move() dashed;\n\t} else {\n\t\tline $3 dashed;\n\t}\n\n\t# draw the box, use comment_default_xx if no explicit\n\t# size is given together with the text in parameter 4\n\told_boxht=boxht;\n\told_boxwid=boxwid;\n\tboxht=comment_default_ht;\n\tboxwid=comment_default_wid;\n\tif \"$2\" == \"\" then {\n\t\tbox invis $4;\n\t} else {\n\t\t$2: box invis $4;\n\t}\n\tboxht=old_boxht;\n\tboxwid=old_boxwid;\n\n\t# draw the frame of the comment\n\tline from       last box.nw \\\n\t\tto          last box.ne - (corner_fold, 0) \\\n\t\tthen to last box.ne - (0, corner_fold) \\\n\t\tthen to last box.se \\\n\t\tthen to last box.sw \\\n\t\tthen to last box.nw ;\n\tline from       last box.ne - (corner_fold, 0) \\\n\t\tto          last box.ne - (corner_fold, corner_fold) \\\n\t\tthen to last box.ne - (0, corner_fold) ;\n\n\t# restore Here.y\n\tmove to ($1.x, old_y)\n}\n\n# connect_to_comment(object,name);\ndefine connect_to_comment {\n\told_y = Here.y\n\t# start at the object\n\tmove to ($1.x, Here.y)\n\t# find the best connection-point of the comment to use as line-end\n\tif $1.x < $2.w.x then {\n\t\tline to $2.w dashed;\n\t} else {\n\t\tif $1.x > $2.e.x then {\n\t\t\tline to $2.e dashed;\n\t\t} else {\n\t\t\tif Here.y < $2.s.y then {\n\t\t\t\tline to $2.s dashed;\n\t\t\t} else {\n\t\t\t\tif Here.y > $2.n.y then {\n\t\t\t\t\tline to $2.n dashed;\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\t# restore Here.y\n\tmove to ($1.x, old_y)\n}\n"
  },
  {
    "path": "user-dev-guide/rrepair/trivial_sd_uml.pic",
    "content": "# Trivial Reconciliation \n# by Nico Kruber\n\n.PS\ncopy \"sequence.pic\";\n\n# Objects\nactor(U, \"\");\nobject(A,\"A:RR\");\nplaceholder_object(Dummy0); # more space\nplaceholder_object(AA);\nplaceholder_object(Dummy1); # more space\nplaceholder_object(Dummy2); # more space\nplaceholder_object(BB);\nplaceholder_object(Dummy3); # more space\nobject(B,\"B:RR\");\nstep();\n\n# STEP 1\nmessage(U,A,\"request_sync\"); active(A);\nasync(); # use asynchrone messages (non-filled arrowheads)\nmessage(A,B,\"start_recon(interval I\\sbA\\eb)\"); \ninactive(A);\nactive(B);\n\n# STEP 2\nsync();\ncreate_message(B,BB,\"B:RC\");\nasync();\nmessage(B,BB,\"create_struct(I\\sbA\\eb)\"); \ninactive(B); active(BB);\nstep();\n\ncomment(BB,GET_DB,0.8 left 1.0, wid 1.6 ht 0.25 \\\n        \"get data from DB\");\n\nmessage(BB,A,\"continue_recon(S := {I\\sbAB\\eb, K\\sbSize\\eb, V\\sbSize\\eb, CKV(\\*b(I\\sbAB\\eb))})\");\ncomment(BB,BUILD_TRIVIAL,0.0 right 0.2, wid 1.8 ht 0.45 \\\n        \"create compressed\"\\\n        \"key-version binary\");\ninactive(BB);\nactive(A);\nsync();\ncreate_message(A,AA,\"A:RC\");\nasync();\nmessage(A,AA,\"start_recon(S)\");\ninactive(A);\nactive(AA);\nstep();\nconnect_to_comment(AA,GET_DB);\nmessage(AA,A,\"request_resolve(\\*D \\ Reg(A,B))\");\nactive(A);\nmessage(AA,BB,\"resolve_req(V\\sbSize\\eb, CK(Reg(A,B)))\");\nactive(BB);\ncomment(BB,RESOLVE_CK,0.0 right 0.2, wid 2.25 ht 0.225 \\\n        \"decode compressed keys\");\ncomplete(AA);\ndrawx(AA);\nbegin_frame(A,F,\"key_upd_s\");\nend_frame(A,F);\nmessage(BB,B,\"request_resolve(Reg(A,B))\");\ncomplete(BB);\ndrawx(BB);\ninactive(A);\nactive(B);\nstep();\nbegin_frame(B,F,\"key_upd_s\");\nend_frame(B,F);\nstep();\ninactive(B);\n\n\n# CLEAN UP\nstep();\ncomplete(A);\ncomplete(B);\ncomplete(U);\n\n.PE\n"
  },
  {
    "path": "user-dev-guide/slide_send_to_pred-2.0.tikz",
    "content": "\\ifdefined\\tikzinnerheight%\n\\else\n  \\newlength{\\tikzinnerheight}%\n\\fi\n\n\\ifdefined\\tikzsepheight%\n\\else\n  \\newlength{\\tikzsepheight}%\n\\fi\n\n\\ifdefined\\tikzstartsecond%\n\\else\n  \\newlength{\\tikzstartsecond}%\n\\fi\n\n\\ifdefined\\tikzstartfirst%\n\\else\n  \\newlength{\\tikzstartfirst}%\n\\fi\n\n\\ifdefined\\tikzsepinner%\n\\else\n  \\newlength{\\tikzsepinner}%\n\\fi\n\n% \\ifdefined\\tikzendsucc%\n% \\else\n%   \\newlength{\\tikzendsucc}%\n% \\fi\n\n\\setlength{\\tikzinnerheight}{1.0cm}\n\\setlength{\\tikzsepheight}{0.75cm}\n\\setlength{\\tikzstartsecond}{0cm}\n\\setlength{\\tikzstartfirst}{\\tikzstartsecond + \\tikzinnerheight + 0.5\\tikzsepheight}\n\\setlength{\\tikzsepinner}{\\tikzinnerheight + \\tikzsepheight}\n% \\setlength{\\tikzendsucc}{\\tikzsepinner + \\tikzstartfirst - \\tikzstartsecond}\n\n\\begin{tikzpicture}\n [pre/.style={<-,shorten <=1pt,>=stealth,semithick},\n  post/.style={->,shorten >=1pt,>=stealth,semithick},\n  progress/.style={-,dashed,thin,black},\n  timeout/.style={draw=black!50, dashed},\n  process/.style={rectangle,black,rounded corners},\n  start/.style={rectangle,draw,thick,fill=yellow,draw=black,minimum height=0.7cm},\n  inner/.style={minimum height=\\tikzinnerheight,thin},\n  end/.style={minimum height=0.4cm},\n  phase/.style={rectangle,draw,thick,black},\n  my_node/.style={rectangle,fill=codebackground,drop shadow,rounded corners},\n  action_l/.style={rectangle,black,font=\\footnotesize,align=right},\n  action_r/.style={rectangle,black,font=\\footnotesize,align=left},\n  note/.style={circle, thin, draw, outer sep=0.1cm},\n  async_r/.style={post,min distance=0.5,looseness=2.5,in=0,out=0},\n  async_l/.style={post,min distance=0.5,looseness=2.5,in=180,out=180},\n  async_desc/.style={rectangle,black,font=\\footnotesize},\n  msg/.style={sloped},\n  msg_t/.style={msg,anchor=south},\n  msg_b/.style={msg,anchor=north},\n  bend angle=60]\n\n \\node[font={\\Large\\bfseries}] (heading1) {};%Send data to predecessor};\n %\\node[font=\\footnotesize,below=-0.2 of heading1] (heading2) {(version 2.0)};\n\n \\node[start] (pred)      [below left=0.5 and 2.0 of heading1.south] {pred};\n \\node[inner] (pred-init)           [below=\\tikzstartfirst of pred]      {};\n \\node[inner] (pred-got-data)       [below=\\tikzsepinner of pred-init] {};\n \\node[inner] (pred-got-delta)      [below=\\tikzsepinner of pred-got-data] {};\n \\node[inner] (pred-got-owner)      [below=\\tikzsepinner of pred-got-delta] {};\n \\node[end]   (pred-end)  [below=\\tikzstartsecond of pred-got-owner] {\\footnotesize pred};\n\n \\node[start] (succ)      [below right=0.5 and 2.0 of heading1.south] {succ};\n \\node[inner] (succ-init)           [below=\\tikzstartsecond of succ]      {};\n \\node[inner] (succ-send-data)      [below=\\tikzsepinner of succ-init]      {};\n \\node[inner] (succ-send-delta)     [below=\\tikzsepinner of succ-send-data]      {};\n \\node[inner] (succ-send-owner)     [below=\\tikzsepinner of succ-send-delta]      {};\n \\node[end]   (succ-end)  [below=\\tikzstartfirst of succ-send-owner] {\\footnotesize succ};\n\n \\path[-] (pred)\n            edge [progress] (pred-end)\n          (succ)\n            edge [progress] (succ-end);\n\n \\path[->] (succ-init.south west)\n             edge [post,draw=gray] node[msg_t, gray,font=\\footnotesize] {slide, succ, 'rcv'} node[msg_b, gray,font=\\footnotesize] {(optional)} (pred-init.north east)\n           (pred-init.south east)\n             edge [post]  node[msg_t] {slide, pred, 'send'\\textcolor{gray}{, MaxE}} (succ-send-data.north west)\n\n           (succ-send-data.east)\n             edge [async_r] node[async_desc, auto, anchor=west] {\\textcolor{green}{prepare\\_send\\_data(SlideOp)}}\n                  (succ-send-data.south east)\n\n           (succ-send-data.south west)\n             edge [post] node[msg_t] {data\\textcolor{gray}{, TargetId, NextOp}} (pred-got-data.north east)\n           (pred-got-data.south east)\n             edge [post] node[msg_t] {data\\_ack} (succ-send-delta.north west)\n\n           (pred-got-data.north west)\n             edge [async_l] node[async_desc, auto, anchor=east, text width=4.0cm, align=right] {\\textcolor{green}{update\\_rcv\\_data(SlideOp,\\\\TargetId, NextOp)}}\n                  (pred-got-data.west)\n\n           (succ-send-delta.north east)\n             edge [async_r] node[async_desc, auto, anchor=west] {\\textcolor{green}{prepare\\_send\\_delta(SlideOp)}}\n                  (succ-send-delta.east)\n\n           (succ-send-delta.south west)\n             edge [post] node[msg_t] {delta} (pred-got-delta.north east)\n           (pred-got-delta.south east)\n             edge [post] node[msg_t] {delta\\_ack} (succ-send-owner.north west)\n\n           (pred-got-delta.north west)\n             edge [async_l] node[async_desc, auto, anchor=east] {\\textcolor{green}{finish\\_delta(SlideOp)}}\n                  (pred-got-delta.west)\n\n           (succ-send-owner.north east)\n             edge [async_r] node[async_desc, auto, anchor=west] (finish_delta_ack) {\\textcolor{green}{finish\\_delta\\_ack(SlideOp)}}\n                  (succ-send-owner.east);\n\n\n \\node[action_l, left=0.1 of pred-init.east] {\n   SlideOp.new()\\\\\n   fd.subscribe(SlideOp.node)\\\\\n   \\textcolor{green}{prepare\\_rcv\\_data(SlideOp)}%\n };\n \\node[phase, below left=0.35 and 0.1 of pred-init.south] (pred-init-p-leases) {wait\\_for\\_data};\n\n \\node[phase, below left=0.1 and 0.1 of pred-got-data.south] {wait\\_for\\_delta};\n\n \\node[action_l, left=0.1 of pred-got-delta.south east] {}; % nothing to do\n\n \\node[action_l, left=0.1 of pred-got-owner.north east] {\n   fd.unsubscribe(SlideOp.node)\\\\\n   SlideOp.delete()%\n };\n\n\n\n \\node[action_r, right=0.1 of succ-init.south west, gray] {\n   SlideOp.new()\\\\\n   fd.subscribe(SlideOp.node)%\n };\n \\node[phase, below right=0.4 and 0.1 of succ-init.south, gray] {wait\\_for\\_other};\n\n \\node[action_r, right=0.1 of succ-send-data.north west,anchor=west] {\n   SlideOp.new()\\\\\n   fd.subscribe(SlideOp.node)%\n };\n\n \\node[action_r, below=0.3 of succ-send-data.south,anchor=west] {\n   db.record\\_changes(SlideOp.interval)%\n };\n \\node[phase, below right=0.5 and 0.1 of succ-send-data.south] {wait\\_for\\_data\\_ack};\n\n \\node[action_r, below right=0.1 and 0.1 of succ-send-delta.south west, anchor=south west] {\n   db.stop\\_record\\_changes(SlideOp.interval)%\n };\n \\node[phase, below right=0.1 and 0.1 of succ-send-delta.south] {wait\\_for\\_delta\\_ack};\n\n \\coordinate (succ-cont-start1) at ($(succ-send-owner)-(0,.5\\tikzsepinner)$);\n \\coordinate (succ-cont-end) at ($(succ-send-data)+(0,0.1)$);\n \\coordinate (pred-cont-start1) at ($(pred-got-owner.north)+(0,0.5)$);\n \\coordinate (pred-cont-end) at ($(pred-init.south)+(0,-0.25)$);\n\n \\draw[post, rounded corners, dashed, gray] (succ-cont-start1) -- +(6.25,0) -- ($(succ-cont-end)+(6.25,0)$) -- (succ-cont-end);\n \\node[action_l, gray, anchor=south east] at ($(succ-cont-start1)+(6.25,0)$) {if (NextOp == continue)\\\\SlideOp.update()};\n\n \\node[action_r, below right=\\tikzstartfirst and 0.1 of succ-send-owner.north west, anchor=north west] {\n   fd.unsubscribe(SlideOp.node)\\\\\n   SlideOp.delete()%\n };\n\n \\coordinate (pred-cont-mid) at ($(pred-cont-start1)+(-7.5,0.2)$);\n \\draw[post, rounded corners, dashed, gray] (pred-cont-start1) -- +(-4.75,0) -- ($(pred-cont-end)+(-4.75,0)$) -- (pred-cont-end);\n \\node[action_r, gray, anchor=south west] at ($(pred-cont-start1)+(-4.75,0)$) {if (NextOp == continue)\\\\SlideOp.update()\\\\prepare\\_rcv\\_data(SlideOp)};\n\n\\end{tikzpicture}\n"
  },
  {
    "path": "user-dev-guide/slide_send_to_succ-2.0.tikz",
    "content": "\\ifdefined\\tikzinnerheight%\n\\else\n  \\newlength{\\tikzinnerheight}%\n\\fi\n\n\\ifdefined\\tikzsepheight%\n\\else\n  \\newlength{\\tikzsepheight}%\n\\fi\n\n\\ifdefined\\tikzstartsecond%\n\\else\n  \\newlength{\\tikzstartsecond}%\n\\fi\n\n\\ifdefined\\tikzstartfirst%\n\\else\n  \\newlength{\\tikzstartfirst}%\n\\fi\n\n\\ifdefined\\tikzsepinner%\n\\else\n  \\newlength{\\tikzsepinner}%\n\\fi\n\n% \\ifdefined\\tikzendsucc%\n% \\else\n%   \\newlength{\\tikzendsucc}%\n% \\fi\n\n\\setlength{\\tikzinnerheight}{1.0cm}\n\\setlength{\\tikzsepheight}{0.75cm}\n\\setlength{\\tikzstartsecond}{0cm}\n\\setlength{\\tikzstartfirst}{\\tikzstartsecond + \\tikzinnerheight + 0.5\\tikzsepheight}\n\\setlength{\\tikzsepinner}{\\tikzinnerheight + \\tikzsepheight}\n% \\setlength{\\tikzendsucc}{\\tikzsepinner + \\tikzstartfirst - \\tikzstartsecond}\n\n\\begin{tikzpicture}\n [pre/.style={<-,shorten <=1pt,>=stealth,semithick},\n  post/.style={->,shorten >=1pt,>=stealth,semithick},\n  progress/.style={-,dashed,thin,black},\n  timeout/.style={draw=black!50, dashed},\n  process/.style={rectangle,black,rounded corners},\n  start/.style={rectangle,draw,thick,fill=yellow,draw=black,minimum height=0.7cm},\n  inner/.style={minimum height=\\tikzinnerheight,thin},\n  end/.style={minimum height=0.4cm},\n  phase/.style={rectangle,draw,thick,black},\n  my_node/.style={rectangle,fill=codebackground,drop shadow,rounded corners},\n  action_l/.style={rectangle,black,font=\\footnotesize,align=right},\n  action_r/.style={rectangle,black,font=\\footnotesize,align=left},\n  note/.style={circle, thin, draw, outer sep=0.1cm},\n  async_r/.style={post,min distance=0.5,looseness=2.5,in=0,out=0},\n  async_l/.style={post,min distance=0.5,looseness=2.5,in=180,out=180},\n  async_desc/.style={rectangle,black,font=\\footnotesize},\n  msg/.style={sloped},\n  msg_t/.style={msg,anchor=south},\n  msg_b/.style={msg,anchor=north},\n  bend angle=60]\n\n \\node[font={\\Large\\bfseries}] (heading1) {};%Send data to successor};\n %\\node[font=\\footnotesize,below=-0.2 of heading1] (heading2) {(version 2.0)};\n\n \\node[start] (pred)      [below left=0.5 and 2.0 of heading1.south] {pred};\n \\node[inner] (pred-init)           [below=\\tikzstartsecond of pred]      {};\n \\node[inner] (pred-send-data)      [below=\\tikzsepinner of pred-init] {};\n \\node[inner] (pred-send-delta)     [below=\\tikzsepinner of pred-send-data] {};\n \\node[inner] (pred-send-owner)     [below=\\tikzsepinner of pred-send-delta] {};\n \\node[end]   (pred-end)  [below=\\tikzstartfirst of pred-send-owner] {\\footnotesize pred};\n\n \\node[start] (succ)      [below right=0.5 and 2.0 of heading1.south] {succ};\n \\node[inner] (succ-init)           [below=\\tikzstartfirst of succ]      {};\n \\node[inner] (succ-got-data)       [below=\\tikzsepinner of succ-init]      {};\n \\node[inner] (succ-got-delta)      [below=\\tikzsepinner of succ-got-data]      {};\n \\node[inner] (succ-got-owner)      [below=\\tikzsepinner of succ-got-delta] {};\n \\node[end]   (succ-end)  [below=\\tikzstartsecond of succ-got-owner] {\\footnotesize succ};\n\n \\path[-] (pred)\n            edge [progress] (pred-end)\n          (succ)\n            edge [progress] (succ-end);\n\n \\path[->] (pred-init.south east)\n             edge [post,gray]  node[msg_t,gray,font=\\footnotesize] {slide, pred, 'rcv'} node[msg_b,gray,font=\\footnotesize] {(optional)} (succ-init.north west)\n           (succ-init.south west)\n             edge [post] node[msg_t] {slide, succ, 'send'\\textcolor{gray}{, MaxE}} (pred-send-data.north east)\n\n           (pred-send-data.west)\n             edge [async_l] node[async_desc, auto, anchor=east, align=right] {\\textcolor{green}{prepare\\_send\\_data(SlideOp)}}\n                  (pred-send-data.south west)\n\n           (pred-send-data.south east)\n             edge [post] node[msg_t] {data\\textcolor{gray}{, TargetId, NextOp}} (succ-got-data.north west)\n           (succ-got-data.south west)\n             edge [post] node[msg_t] {data\\_ack} (pred-send-delta.north east)\n\n           (succ-got-data.north east)\n             edge [async_r] node[async_desc, auto, anchor=west, text width=4.0cm, align=left] {\\textcolor{green}{update\\_rcv\\_data(SlideOp,\\\\TargetId, NextOp)}}\n                  (succ-got-data.east)\n\n           (pred-send-delta.north west)\n             edge [async_l] node[async_desc, auto, anchor=east] {\\textcolor{green}{prepare\\_send\\_delta(SlideOp)}}\n                  (pred-send-delta.west)\n\n           (pred-send-delta.south east)\n             edge [post] node[msg_t] {delta} (succ-got-delta.north west)\n           (succ-got-delta.south west)\n             edge [post] node[msg_t] {delta\\_ack} (pred-send-owner.north east)\n\n           (succ-got-delta.north east)\n             edge [async_r] node[async_desc, auto, anchor=west] {\\textcolor{green}{finish\\_delta(SlideOp)}}\n                  (succ-got-delta.east)\n\n           (pred-send-owner.north west)\n             edge [async_l] node[async_desc, auto, anchor=east] (finish_delta_ack) {\\textcolor{green}{finish\\_delta\\_ack(SlideOp)}}\n                  (pred-send-owner.west);\n\n\n \\node[action_l, left=0.1 of pred-init.south east ,gray] {\n   SlideOp.new()\\\\\n   fd.subscribe(SlideOp.node)%\n };\n \\node[phase, below left=0.4 and 0.1 of pred-init.south, gray] {wait\\_for\\_other};\n\n \\node[action_l, left=0.1 of pred-send-data.north east] {\n   SlideOp.new()\\\\\n   fd.subscribe(SlideOp.node)%\n };\n \n \\node[action_l, left=0.1 of pred-send-data.south east, anchor=north east] {\n   db.record\\_changes(SlideOp.interval)%\n };\n \\node[phase, below left=0.5 and 0.1 of pred-send-data.south] {wait\\_for\\_data\\_ack};\n \n \\node[action_l, below left=0.1 and 0.1 of pred-send-delta.south east, anchor=south east] {\n   db.stop\\_record\\_changes(SlideOp.interval)%\n };\n \\node[phase, below left=0.1 and 0.1 of pred-send-delta.south] {wait\\_for\\_delta\\_ack};\n\n\n \n \\node[action_r, right=0.1 of succ-init.west] {\n   SlideOp.new()\\\\\n   fd.subscribe(SlideOp.node)\\\\\n   \\textcolor{green}{prepare\\_rcv\\_data(SlideOp)}%\n };\n \\node[phase, below right=0.35 and 0.1 of succ-init.south] {wait\\_for\\_data};\n\n \\node[phase, below right=0.1 and 0.1 of succ-got-data.south] {wait\\_for\\_delta};\n\n \\node[action_r, right=0.1 of succ-got-owner.north west] {\n   fd.unsubscribe(SlideOp.node)\\\\\n   SlideOp.delete()%\n };\n\n \\coordinate (succ-cont-start1) at ($(succ-got-owner.north)+(0,0.5)$);\n \\coordinate (succ-cont-end) at ($(succ-init.south)+(0,-0.25)$);\n \\coordinate (pred-cont-start1) at ($(pred-send-owner)-(0,.5\\tikzsepinner)$);\n \\coordinate (pred-cont-end) at ($(pred-send-data.west)+(0,0.1)$);\n\n \\draw[post, rounded corners, dashed, gray] (pred-cont-start1) -- +(-6.25,0) -- ($(pred-cont-end)+(-6.25,0)$) -- (pred-cont-end);\n \\node[action_r, gray, anchor=south west] at ($(pred-cont-start1)+(-6.25,0)$) {if (NextOp == continue)\\\\SlideOp.update()};\n\n \\node[action_l, below left=\\tikzstartfirst and 0.1 of pred-send-owner.north east, anchor=north east] {\n   fd.unsubscribe(SlideOp.node)\\\\\n   SlideOp.delete()%\n };\n \n \\coordinate (succ-cont-mid) at ($(succ-cont-start1)+(7.5,0.2)$);\n \\draw[post, rounded corners, dashed, gray] (succ-cont-start1) -- +(4.75,0) -- ($(succ-cont-end)+(4.75,0)$) -- (succ-cont-end);\n \\node[action_l, gray, anchor=south east] at ($(succ-cont-start1)+(4.75,0)$) {if (NextOp == continue) \\\\SlideOp.update()\\\\prepare\\_rcv\\_data(SlideOp)};\n\n\\end{tikzpicture}\n"
  },
  {
    "path": "user-dev-guide/tikz_footer.tex",
    "content": "\\end{document}\n"
  },
  {
    "path": "user-dev-guide/tikz_header.tex",
    "content": "\\documentclass[a4paper]{scrreprt}\n\\usepackage{typearea}\n\\areaset[1cm]{165mm}{240mm}\n\n\\usepackage[T1]{fontenc}\n\\usepackage[latin1]{inputenc}\n\\usepackage{listings}\n\n\\usepackage{xcolor}\n\\definecolor{lightyellow}{rgb}{1.0, 1.0, 0.5}\n\\definecolor{codebackground}{HTML}{EEEEEE}\n\\definecolor{commandinput}{rgb}{0.8,0.8,1}\n\\definecolor{lightblue}{HTML}{1E90FF}\n\n\\usepackage{tikz}\n\\usetikzlibrary{positioning}\n\\usetikzlibrary{shadows}\n\\usetikzlibrary{fit}\n\\usetikzlibrary{shapes.arrows}\n\\usetikzlibrary{backgrounds}\n\\usetikzlibrary{calc}\n\\usepackage[graphics,tightpage,active]{preview}\n\\PreviewEnvironment{tikzpicture}\n\\newlength{\\imagewidth}\n\\newlength{\\imagescale}\n\n\\usepackage{calc}\n\n\\newcommand{\\code}[1]{\\lstinline[basicstyle=\\ttfamily]!#1!}\n\n\\begin{document}\n"
  },
  {
    "path": "user-dev-guide/user-config.tex",
    "content": "\\chapter{Setting up \\scalaris{}}\n\\label{chapter.runscalaris}\n\n\\section{Runtime Configuration}\n\\label{chapter.runscalaris.runtime_config}\n\n\\scalaris{} reads two configuration files from the working directory:\n\\code{bin/scalaris.cfg} (mandatory) and \\code{bin/scalaris.local.cfg}\n(optional). The former defines default settings and is included in the\nrelease. The latter can be created by the user to alter settings.\nA sample file is provided as \\code{bin/scalaris.local.cfg.example}\nand needs to be altered for a distributed setup (see\nSection~\\sieheref{user.config.distributed}).\nA third way to alter the configuration of \\scalaris{}, e.g. port numbers,\nis to use parameters for the \\code{scalarisctl} script\n(ref. Section~\\sieheref{user.config.scalarisctl}. The following\nexample changes the port to 14195 and the YAWS port to 8080:\n\n\\begin{lstlisting}[language=sh]\n%> ./bin/scalarisctl -p 14194 -y 8080\n\\end{lstlisting}\n\nThe configuration precedence is as follows:\n\\begin{enumerate}\n  \\item configuration parameters of \\code{scalarisctl}\n  \\item \\code{bin/scalaris.local.cfg}\n  \\item \\code{bin/scalaris.cfg}\n\\end{enumerate}\n\n\n\\subsection{Logging}\n\\label{sec:logging}\n\n\\scalaris{} uses the log4erl library (see \\code{contrib/log4erl}) for\nlogging status information and error messages. The log level can be\nconfigured in \\code{bin/scalaris.cfg} for both the stdout and file logger.\nThe default value is {\\tt warn}; only warnings, errors and severe problems are\nlogged.\n\n\\begin{lstlisting}[language=erlang]\n%% @doc Loglevel: debug < info < warn < error < fatal < none\n{log_level, warn}.\n{log_level_file, warn}.\n\\end{lstlisting}\n\nIn some cases, it might be necessary to get more complete logging\ninformation, e.g. for debugging. In Chapter~\\sieheref{chapter.join},\nwe are explaining the startup process of \\scalaris{} nodes in more\ndetail, here the {\\tt info} level provides more detailed information.\n\n\\begin{lstlisting}[language=erlang]\n%% @doc Loglevel: debug < info < warn < error < fatal < none\n{log_level, info}.\n{log_level_file, info}.\n\\end{lstlisting}\n\n\n\\section{Running \\scalaris{}}\n\n\nA \\scalaris{} deployment can have a \\emph{management server} as well as\n\\emph{regular nodes}. The\nmanagement server is optional and provides a global view on all nodes of a\n\\scalaris{} deployment which contact this server, i.e. have its address\nspecified in the \\code{mgmt_server} configuration setting.\nA regular node is either the first node in a system or joins an\nexisting system deployment.\n\n\\subsection{Running on a local machine}\n\\label{sec.boot}\n\nOpen at least two shells. In the first, inside the \\scalaris{} directory,\nstart the first node (\\code{firstnode.bat} on Windows):\n\\begin{lstlisting}[language=sh]\n%> ./bin/firstnode.sh\n\\end{lstlisting}\n\nThis will start a new \\scalaris{} deployment with a single node, including a\nmanagement server. On success \\url{http://localhost:8000} should point to\nthe management interface page of the management server. The main page will\nshow you the number of nodes currently in the system.\nA first \\scalaris{} node should have started and the number should\nshow 1 node. The main page will also allow you to store and retrieve\nkey-value pairs but should not be used by applications to access\n\\scalaris{}. See Section~\\sieheref{chapter.systemuse.apis} for application APIs.\n\nIn a second shell, you can now start a second \\scalaris{} node. This\nwill be a `regular node':\n\\begin{lstlisting}[language=sh]\n%> ./bin/joining_node.sh\n\\end{lstlisting}\n\nThe second node will read the configuration file and use this information to\ncontact a number of known nodes (set by the \\code{known_hosts} configuration\nsetting) and join the ring. It will also register itself with the management\nserver.\nThe number of nodes on the web page should have increased to two by now.\n\nOptionally, a third and fourth node can be started on the same\nmachine. In a third shell:\n\\begin{lstlisting}[language=sh]\n%> ./bin/joining_node.sh 2\n\\end{lstlisting}\n\nIn a fourth shell:\n\\begin{lstlisting}[language=sh]\n%> ./bin/joining_node.sh 3\n\\end{lstlisting}\n\nThis will add two further nodes to the deployment. The\n\\code{./bin/joining_node.sh} script accepts a number as its parameter which\nwill be added to the started node's name, i.e. \\code{1} will lead to a node\nnamed \\code{node1}.\nThe web pages at \\url{http://localhost:8000} should show the additional nodes.\n\n\\subsection{Running distributed}\n\\label{user.config.distributed}\n\n\\scalaris{} can be installed on other machines in the same way as\ndescribed in Section~\\sieheref{sec:install}. In the default configuration,\nnodes will look for the management server on \\code{127.0.0.1} on port 14195.\nTo run \\scalaris{} distributed over several nodes, each node requires a\n\\code{bin/scalaris.local.cfg} pointing to the node running\nthe management server (if available) and containing a list of known nodes.\nWithout a list of known nodes, a joining node will not know where to join.\n\nIn the following example, the \\code{mgmt_server}'s location is defined as\nan IP address plus a TCP port and its Erlang-internal process name.\nIf the deployment should not use a management server, replace the setting with\nan invalid address, e.g. \\code{'null'}.\n\n\\codesnippet{scalaris.local.cfg}{local_cfg:distributed}\n            {../bin/scalaris.local.cfg.example}\n\nIf you are starting the management server using \\code{firstnode.sh}, it will\nlisten on port 14195 and you only have to change the the IP address in the\nconfiguration file. Otherwise the other nodes will not find the management\nserver. Calling \\code{./bin/joining_node.sh} on a remote machine will start the\nnode and automatically contact the configured management server.\n\n%\\subsection{Running on PlanetLab}\n\n%\\subsection{Replication Degree}\n\n%\\subsection{Routing Scheme}\n\n\\section{Custom startup using \\texorpdfstring{\\code{scalarisctl}}{scalarisctl}}\n\\label{user.config.scalarisctl}\n\nOn Linux you can also use the \\code{scalarisctl} script to start a\nmanagement server and `regular' nodes directly.\n\\begin{lstlisting}[language=sh]\n%> ./bin/scalarisctl -h\n\\end{lstlisting}\n\\lstinputlisting[language={}]{scalarisctl-h.out}\n"
  },
  {
    "path": "user-dev-guide/user-install.tex",
    "content": "\\chapter{Download and Installation}\n\\label{chapter.downloadinstall}\n\n\\section{Requirements}\n\\label{sec.requirements}\n\nFor building and running \\scalaris{}, some third-party software is\nrequired which is not included in the \\scalaris{} sources:\n\n\\begin{itemize}\n\\setlength{\\itemsep}{0pt}\n\\setlength{\\parskip}{0pt}\n\\item Erlang R14B04 or newer\n\\item OpenSSL (required by Erlang's crypto module)\n\\item GNU-like Make and autoconf (not required on Windows)\n\\end{itemize}\n\nTo build the Java API (and its command-line client) the following\nprograms are also required:\n\n\\begin{itemize}\n\\setlength{\\itemsep}{0pt}\n\\setlength{\\parskip}{0pt}\n\\item Java Development Kit 6\n\\item Apache Ant\n\\end{itemize}\n\nBefore building the Java API, make sure that \\code{JAVA\\_HOME} and\n\\code{ANT\\_HOME} are set. \\code{JAVA\\_HOME} has to point to a JDK\ninstallation, and \\code{ANT\\_HOME} has to point to an Ant installation.\n\nTo build the Python API (and its command-line client) the following\nprograms are also required:\n\n\\begin{itemize}\n\\setlength{\\itemsep}{0pt}\n\\setlength{\\parskip}{0pt}\n\\item Python >= 2.6\n\\end{itemize}\n\n\\section{Download}\n\nThe sources can be obtained from\n\\url{https://github.com/scalaris-team/scalaris}. RPM and DEB packages are available\nfrom \\url{http://download.opensuse.org/repositories/home:/scalaris/} for\nvarious Linux distributions.\n\n\\subsection{Development Branch}\n\nYou find the latest development version in the git repository:\n\\begin{lstlisting}[language={}]\ngit clone https://github.com/scalaris-team/scalaris.git scalaris\n\\end{lstlisting}\n\n\\subsection{Releases}\n\nReleases can be found under the 'Download' tab on the web-page.\n\n\n\\section{Build}\n\n\\subsection{Linux}\n\n\\scalaris{} uses autoconf for configuring the build environment and\nGNU Make for building the code.\n\n\\begin{lstlisting}[language=sh]\n%> ./configure\n%> make\n%> make docs\n\\end{lstlisting}\n\nFor more details read \\code{README} in the main \\scalaris{} checkout\ndirectory.\n\n\\subsection{Windows}\n\nWe are currently not supporting \\scalaris{} on Windows. However, we\nhave two small {\\tt .bat} files for building and running \\scalaris{}\nnodes. It seems to work but we make no guarantees.\n\n\\begin{itemize}\n\\item Install Erlang\\\\\n       \\url{http://www.erlang.org/download.html}\n\\item Install OpenSSL (for crypto module)\\\\\n       \\url{http://www.slproweb.com/products/Win32OpenSSL.html}\n\\item Checkout \\scalaris{} code from SVN\n\\item adapt the path to your Erlang installation in \\code{build.bat}\n\\item start a \\code{cmd.exe}\n\\item go to the \\scalaris{} directory\n\\item run \\code{build.bat} in the cmd window\n\\item check that there were no errors during the compilation;\n       warnings are fine\n\\item go to the bin sub-directory\n\\item adapt the path to your Erlang installation in \\code{firstnode.bat},\n       \\code{joining_node.bat}\n\\item run \\code{firstnode.bat} or one of the other start scripts in the cmd window\n\\end{itemize}\n\n\\code{build.bat} will generate a \\code{Emakefile} if there is none yet.\nOn certain older Erlang versions, you will need to adapt the \\code{Emakefile}.\nPlease refer to the \\code{build.bat} and \\code{configure.ac} for the available\nconfiguration parameters and their meaning.\n\nFor the most recent description please see the FAQ at\n\\url{http://scalaris.zib.de/faq.html}.\n\n\\subsection{Java-API}\n\nThe following commands will build the Java API for \\scalaris{}:\n\\begin{lstlisting}[language=sh]\n%> make java\n\\end{lstlisting}\n\nThis will build {\\tt scalaris.jar}, which is the library for accessing\nthe overlay network. Optionally, the documentation can be build:\n\\begin{lstlisting}[language=sh]\n%> cd java-api\n%> ant doc\n\\end{lstlisting}\n\n\\subsection{Python-API}\n\nThe Python API for Python 2.* (at least 2.6) is located in the \\code{python-api}\ndirectory. Files for Python 3.* can be created using \\code{2to3} from the files\nin \\code{python-api}. The following command will use \\code{2to3} to convert the\nmodules and place them in \\code{python3-api}. \n\\begin{lstlisting}[language=sh]\n%> make python3\n\\end{lstlisting}\nBoth versions of python will compile required modules on demand when executing\nthe scripts for the first time. However, pre-compiled modules can be created\nwith:\n\\begin{lstlisting}[language=sh]\n%> make python\n%> make python3\n\\end{lstlisting}\n\n\\subsection{Ruby-API}\n\nThe Ruby API for Ruby >= 1.8 is located in the \\code{ruby-api}\ndirectory. Compilation is not necessary.\n\n\\section{Installation}\n\\label{sec:install}\n\nFor simple tests, you do not need to install \\scalaris{}. You can run it\ndirectly from the source directory. Note: \\code{make install} will install\n\\scalaris{} into \\code{/usr/local} and place \\code{scalarisctl} into\n\\code{/usr/local/bin}, by default. But it is more convenient to build an RPM\nand install it.\nOn openSUSE, for example, do the following:\n\n\\begin{lstlisting}[language=sh]\nexport SCALARIS_GIT=https://raw.githubusercontent.com/scalaris-team/scalaris/master\nfor package in main bindings; do\n  mkdir -p ${package}\n  cd ${package}\n  wget ${SCALARIS_GIT}/contrib/packages/${package}/checkout.sh\n  ./checkout.sh\n  cp * /usr/src/packages/SOURCES/\n  rpmbuild -ba scalaris*.spec\n  cd ..\ndone\n\\end{lstlisting}\n\nIf any additional packages are required in order to build an RPM,\n\\code{rpmbuild} will print an error.\n\nYour source and binary RPMs will be generated in\n\\code{/usr/src/packages/SRPMS} and \\code{RPMS}.\n\nWe build RPM and DEB packages for the newest stable Scalaris version as well as\nsnapshots of the git master branch and provide them using the Open Build Service.\nThe latest stable version is available at\n\\url{http://download.opensuse.org/repositories/home:/scalaris/}.\nThe latest git snapshot is available at\n\\url{http://download.opensuse.org/repositories/home:/scalaris:/svn}.\n\nFor those distributions which provide a recent-enough Erlang version, we build\nthe packages using their Erlang package and recommend using the same version\nthat came with the distribution. In this case we do not provide Erlang packages\nin our repository.\n\nExceptions are made for (old) openSUSE-based and RHEL-based distributions:\n\\begin{itemize}\n  \\item For older openSUSE or SLE distributions, we provide Erlang R14B04.\n  \\item For RHEL-based distributions (CentOS~5,6,7, RHEL~5,6,7) we included the Erlang\npackage from the EPEL repository of RHEL~6 and RHEL~7, respectively.\n\\end{itemize}\n\n\\section{Testing the Installation}\n\nAfter installing \\scalaris{} you can check your installation and perform\nsome basic tests using\n\n\\begin{lstlisting}[language=sh]\n%> scalarisctl checkinstallation\n\\end{lstlisting}\n\nFor further details on \\code{scalarisctl} see\nSection~\\sieheref{user.config.scalarisctl}.\n"
  },
  {
    "path": "user-dev-guide/user-intro.tex",
    "content": "\\chapter{Introduction}\n\n\\scalaris{} is a scalable, transactional, distributed key-value store based\non the principles of structured peer-to-peer overlay networks.  It can be\nused as a flexible elastic data store backend to build scalable online\nservices. Without system interruption it scales from a few PCs to thousands\nof servers. Servers can be added or removed on the fly without any service\ndowntime.\n\n\\scalaris{} takes care of\n\n\\begin{center}\n\\begin{tabular}{lp{10cm}}\n\\emph{replication and fail-over} & for fault-tolerance \\\\\n\\emph{self-management} & for low maintenance overhead \\\\\n\\emph{automatic data partitioning} & for elasticity, load balancing and scalability \\\\\n\\emph{strong consistency} & to ease\ndevelopment of applications on top of it, as inconsistencies have not to be\ndealt with \\\\\n\\emph{transactions} & to support safe atomic updates of several data items\nat once \\\\\n\\end{tabular}\n\\end{center}\n\nThe \\scalaris{} project was initiated and is mainly developed by\n\\href{http://www.zib.de/DAS/}{Zuse Institute Berlin} (ZIB) and was partly\nfunded by the EU projects Selfman, XtreemOS, Contrail and 4CaaST. Additional\ninformation can be found at the project homepage\n(\\url{http://scalaris.zib.de}) and the corresponding project web\npage at ZIB (\\url{http://www.zib.de/en/das/projekte/projektdetails/article/scalaris.html}).\n\nThe conceptual architecture of \\scalaris{} consists of four layers:\n\n\\definecolor{layer_bg}{RGB}{55,96,146}\n\\definecolor{layer_bluefont}{RGB}{31,73,128}\n\\begin{center}\n\\begin{tikzpicture}\n [layer/.style={rectangle,fill=layer_bg,text=white,\n  drop shadow={opacity=0.4},\n  minimum width=7cm,\n  minimum height=1.1cm,\n  align=center},\n  layer_desc/.style={text width=3.5cm,align=left}]\n\\sffamily\n\n  %% draw the layers\n  \\node[layer] (tx_layer) at (0,2.7)\n     {Transaction Layer};\n  \\node[layer,below=0.2 of tx_layer, minimum height=0.9cm] (rep_layer)\n     {Replication Layer};\n  \\node[layer,below=0.2 of rep_layer] (son_layer)\n     {Structured Overlay Network};\n  \\node[layer,below=0.2 of son_layer, minimum height=0.9cm] (unstructured_layer)\n     {Unstructured P2P Layer};\n\n  %% fitting the underlying bounding box\n  \\begin{pgfonlayer}{background}\n    \\node[draw,color=layer_bg,fit=(tx_layer) (rep_layer)\n                                  (son_layer) (unstructured_layer),\n        inner sep=0.3cm,drop shadow, fill=white] {};\n  \\end{pgfonlayer}\n\n  %% text above the box\n  \\node[above=0.4 of tx_layer, text=layer_bg]\n     {\\textbf{\\color{layer_bluefont}{Scalable Application using \\scalaris{}}}};\n\n  %% text below the box\n  \\node[inner sep=0, below=0.6 of unstructured_layer]\n     {\\textbf{\\color{layer_bluefont}{Standard Internet Nodes for Data Storage}}};\n\n  %% description on the right side\n  \\node[above right=0.3 and 0.9 of tx_layer]\n     {\\smaller layer implements~\\ldots};\n  \\node[layer_desc,right=0.9 of tx_layer] (tx_layer_desc)\n     {\\smaller \\ldots~strong consistency,\n      \\phantom{\\ldots~}atomicity, isolation};\n  \\node[layer_desc,right=0.9 of rep_layer] (rep_layer_desc)\n     {\\smaller \\ldots~availability};\n  \\node[layer_desc,right=0.9 of son_layer] (son_layer_desc)\n     {\\smaller \\ldots~scalability};\n  \\node[layer_desc,right=0.9 of unstructured_layer] (unstructured_layer_desc)\n     {\\smaller \\ldots~connectivity};\n\n  %% arrows\n  \\begin{scope}[every node/.style={single arrow,\n              fill=layer_bg,\n              shape border rotate=180,\n              single arrow head extend=1.2mm,\n              minimum height=0.55cm,\n              minimum width=0.2cm}]\n    \\node[right=0.15 of tx_layer] {};\n    \\node[right=0.15 of rep_layer] {};\n    \\node[right=0.15 of son_layer] {};\n    \\node[right=0.15 of unstructured_layer] {};\n  \\end{scope}\n\\end{tikzpicture}\n\\end{center}\n\n\n\\section{\\scalaris{} provides strong consistency and partition tolerance}\n\nIn distributed computing the so called CAP theorem says that there are three\ndesirable properties for distributed systems, but one can only have any two\nof them.\n\n\\begin{description}\n\\item {Strong Consistency.} Any read operation has to return the\n  result of the latest write operation on the same data item.\n\n\\item {Availability.} Items can be read and modified at any time.\n\n\\item {Partition Tolerance.} The network on which the service is\n  running may split into several partitions which cannot communicate\n  with each other. Later on the networks may re-join again.\n\n  For example, a service is hosted on one machine in Seattle and one\n  machine in Berlin. This service is partition tolerant if it can\n  tolerate that all Internet connections over the Atlantic (and\n  Pacific) are interrupted for a few hours and then get repaired.\n\\end{description}\n\nThe goal of \\scalaris{} is to provide strong consistency and partition\ntolerance. We are willing to sacrifice availability to make sure that the\nstored data is always consistent. I.e. when you are running \\scalaris{} with\na replication degree of four and the network splits into two partitions --\none partition with three replicas and one partition with one replica -- you\nwill be able to continue to use the service only in the larger\npartition. All requests in the smaller partition will time out or retried\nuntil the two networks merge again. Note, most other key-value stores tend\nto sacrifice consistency, which may make it hard for the application\ndeveloper to detect and handle appearing inconsistencies properly.\n\n\\section{Scientific background}\n\n\\scalaris{} is backed by tons of research. It implements both algorithms\nfrom the literature and our own research results and combines all of them to\na practical overall system. Several aspects of \\scalaris{} were analyzed\nor/and developed as part of bachelor, diploma, master or PhD theses.\n\n\\subsection*{\\scalaris{} in General}\n\n{\\bf Publications of the \\scalaris{} team}\n\\begin{quote} \\small\n  F. Schintke. \\emph{XtreemFS \\& Scalaris.}\n  Science \\& Technology, pp. 54-55, 2013.\n\n  A. Reinefeld, F. Schintke, T. Schtt, S. Haridi.\n  \\emph{A Scalable, Transactional Data Store for Future Internet Services.}\n  Towards the Future Internet - A European Research Perspective,\n  G. Tselentis et al. (Eds.) IOS Press, pp. 148-159, 2009.\n\n  Thorsten Schtt, Monika Moser, Stefan Plantikow, Florian Schintke,\n  Alexander Reinefeld. \\emph{A Transactional Scalable Distributed Data\n    Store.}  1st IEEE International Scalable Computing Challenge, co-located\n  with CCGrid'08, 2008.\n\n  Thorsten Schtt, Florian Schintke, Alexander Reinefeld.  \\emph{Scalaris:\n    Reliable Transactional P2P Key/Value Store.} ACM SIGPLAN Erlang\n  Workshop, 2008.\n\n\\end{quote}\n\n\\subsection*{Structured Overlay Networks and Routing}\nThe general structure of \\scalaris{} is modelled after Chord. The Chord\npaper~\\cite{chord-sigcomm} describes the ring structure, the routing\nalgorithms, and basic ring maintenance.\n\nThe main routines of our Chord node are in \\code{src/dht\\_node.erl} and the\njoin protocol is implemented in \\code{src/dht\\_node\\_join.erl} (see also\nChap.~\\sieheref{chapter.join}). Our implementation of the routing algorithms\nis described in more detail in Sect.~\\sieheref{chapter.routing} and the\nactual implementation is in \\code{src/rt\\_chord.erl}. We also implemented\nFlexible Routing Tables according to~\\cite{frtchord} which can be found in\n\\code{src/rt\\_frtchord.erl} and {src/rt\\_gfrtchord.erl}.\n\n{\\bf Publications of the \\scalaris{} team}\n\\begin{quote} \\small\n\n  Magnus Mller. \\emph{Flexible Routing Tables in a Distributed Key-Value\n    Store.} Diploma thesis, HU-Berlin, 2013.\n\n  Mikael Hgqvist. \\emph{Consistent Key-Based Routing in Decentralized and\n    Reconfigurable Data Services.} Doctoral thesis, HU-Berlin, 2012.\n\n  Philipp Borgers. \\emph{Erweiterung eines verteilten Key-Value-Stores\n    (Riak) um einen rumlichen Index.} Bachelor thesis, FU-Berlin, 2012.\n\n  Thorsten Schtt. \\emph{Range queries in distributed hash tables.} Doctoral\n  thesis, 2010.\n\n  Christian von Prollius. \\emph{Ein Peer-to-Peer System mit Bereichsabfragen\n    in PlanetLab.} Diploma thesis, FU-Berlin, 2008.\n\n  Jeroen Vlek. \\emph{Reducing latency: Log b routing for Chord$^{\\#}$.}\n  Bachelor thesis, Uni Amsterdam, 2008.\n\n  Thorsten Schtt, Florian Schintke, Alexander Reinefeld.  \\emph{Range\n    Queries on structured overlay networks.} Computer Communications, 31(2),\n  pp. 280-291, 2008.\n\n  Thorsten Schtt, Florian Schintke, Alexander Reinefeld. \\emph{A Structured\n    Overlay for Multi-dimensional Range Queries.} Euro-Par Conference, Luc\n  Anne-Marie Kermarrec (Ed.)pp. 503-513, Vol.4641, LNCS, 2007.\n\n  Alexander Reinefeld, Florian Schintke, Thorsten Schtt. \\emph{P2P Routing\n    of Range Queries in Skewed Multidimensional Data Sets.} ZIB\n  report ZR-07-23, 2007.\n\n  Thorsten Schtt, Florian Schintke, Alexander Reinefeld. \\emph{Structured\n    Overlay without Consistent Hashing.} Sixth Workshop on Global and\n  Peer-to-Peer Computing (GP2PC'06) at Sixth IEEE International Symposium on\n  Cluster Computing and the Grid (CCGrid 2006), 16-19 May 2006, Singapore,\n  p. 8, 2006.\n\n  Thorsten Schtt, Florian Schintke, Alexander\n  Reinefeld. \\emph{Chord$^{\\#}$: Structured Overlay Network for Non-Uniform\n    Load-Distribution.} ZIB report ZR-05-40, 2005.\n\\end{quote}\n\n\\paragraph{Related work}\n\\begin{quote} \\small\n\n  \\cite{frtchord} Hiroya Nagao, Kazuyuki Shudo.\n  \\emph{Flexible routing tables: Designing routing\n    algorithms for overlays based on a total order on a routing table set.}\n  In: Peer-to-Peer Computing, IEEE, 2011.\n\n  P. Ganesan, B. Yang, H. Garcia-Molina. \\emph{One torus to rule them all:\n    Multi- dimensional queries in P2P systems.} In: WebDB2004, 2004.\n\n  Luc Onana Alima, Sameh El-Ansary, Per Brand and Seif Haridi. \\emph{DKS(N,\n    k, f) A family of Low-Communication, Scalable and Fault-tolerant\n    Infrastructures for P2P applications.} The 3rd International workshop on\n    Global and P2P Computing on Large Scale Distributed Systems, (CCGRID\n    2003), May 2003.\n\n  \\cite{chord-sigcomm} Ion Stoica, Robert Morris, David Karger, M. Frans\n  Kaashoek and Hari Balakrishnan.  \\emph{Chord: A Scalable Peer-to-peer\n    Lookup Service for Internet Applications.}  ACM SIGCOMM 2001, San Deigo,\n  CA, August 2001, pp. 149-160.\n  \\url{http://pdos.csail.mit.edu/papers/chord:sigcomm01/chord_sigcomm.pdf}\n\\end{quote}\n\n\\subsection*{Transactions}\nThe most interesting part is probably the transaction algorithms. The last\ndescription of the algorithms and background is in~\\cite{enhanced-paxos}.\n\nThe implementation consists of the Paxos algorithm in \\code{src/paxos} and\nthe transaction algorithms itself in \\code{src/transactions} (see also\nChap.~\\sieheref{chapter.transactions}).\n\n{\\bf Publications of the \\scalaris{} team}\n\\begin{quote} \\small\n\n  \\cite{enhanced-paxos} Florian Schintke, Alexander Reinefeld, Seif Haridi,\n  Thorsten Schtt.  \\emph{Enhanced Paxos Commit for Transactions on DHTs.}\n  CCGRID, pp. 448-454, 2010.\n\n  Florian Schintke. \\emph{Management verteilter Daten in Grid- und\n    Peer-to-Peer-Systemen.} Doctoral thesis, HU-Berlin, 2010.\n\n  Monika Moser, Seif Haridi, Tallat Shafaat, Thorsten Schtt, Mikael\n  Hgqvist, Alexander Reinefeld. \\emph{Transactional DHT Algorithms.}  ZIB\n  report ZR-09-34, 2009.\n\n  Stefan Plantikow, Alexander Reinefeld, Florian\n  Schintke. \\emph{Transactions and Concurrency Control for\n    Peer-to-Peer-Wikis.} In: Making Grids Work, Marco Danelutto, Paraskevi\n  Fragopoulo, Vladimir Getov (Eds.)pp. 337-349, 2008.\n\n  B. Mejas, M. Hgqvist, P. Van Roy. \\emph{Visualizing Transactional Algorithms\n    for DHTs.} IEEE P2P Conference, 2008.\n\n  Monika Moser, Seif Haridi. \\emph{Atomic Commitment in Transactional DHTs.}\n  Proceedings of the CoreGRID Symposium, 2007.\n\n  S. Plantikow, A. Reinefeld, F. Schintke. \\emph{Distributed Wikis on\n    Structured Overlays.} CoreGrid Workshop on Grid Programming Models, Grid\n  and P2P System Architecture, Grid Systems, Tools and Environments, 2007.\n\n  S. Plantikow, A. Reinefeld, F. Schintke. \\emph{Transactions for\n    Distributed Wikis on Structured Overlays.} DSOM, Alexander Clemm,\n  Lisandro Granville, Rolf Stadler (Eds.)pp. 256-267, Vol.4785, LNCS, 2007.\n\n  Stefan Plantikow. \\emph{Transaktionen fr verteilte Wikis auf\n    strukturierten Overlay-Netzwerken.} Diploma thesis, HU-Berlin, 2007.\n\\end{quote}\n\n\\paragraph{Related work}\n\\begin{quote} \\small\n\n  Bjrn Kolbeck, Mikael Hgqvist, Jan Stender, Felix Hupfeld. \\emph{Flease\n    -- Lease Coordination Without a Lock Server.} Intl. Parallel and\n  Distributed Processing Symposium, pp. 978-988, 2011.\n\n  J. Gray, L. Lamport. \\emph{Consensus on transaction commit.} ACM\n  Trans. Database Syst., 31(1):133--160, 2006.\n\n  L. Lamport. \\emph{Fast Paxos.} Distributed Computing, 19(2):79--103, 2006.\n\n  L. Lamport. \\emph{Paxos Made Simple.} SIGACT News, 32(4):51--58, December\n  2001.\n\n  L. Lamport. \\emph{The Part-Time Parliament.} ACM Trans. Comput. Syst.,\n  16(2):133--169, 1998.\n\n\n\\end{quote}\n\n\\subsection*{Ring Maintenance}\nWe changed the ring maintenance algorithm in \\scalaris{}. It is not the\nstandard Chord one, but a variation of T-Man~\\cite{t-man}. It is supposed to\nfix the ring structure faster. In some situations, the standard Chord\nalgorithm is not able to fix the ring structure while T-Man can still fix\nit. For node sampling, our implementation relies on Cyclon~\\cite{cyclon}.\n\nThe T-Man implementation can be found in \\code{src/rm\\_tman.erl} and the\nCyclon implementation in \\code{src/cyclon.erl}.\n\n{\\bf Publications of the \\scalaris{} team}\n\\begin{quote} \\small\n  Paolo Costa, Guillaume Pierre, Alexander Reinefeld, Thorsten Schtt,\n  Maarten van Steen. \\emph{Sloppy Management of Structured P2P Services.}\n  Proceedings of the 3$^{rd}$ International Workshop on Hot Topics in\n  Autonomic Computing (HotAC III), co-located with IEEE ICAC'08, 2008.\n\\end{quote}\n\n\\paragraph{Related work}\n\\begin{quote} \\small\n\n  \\cite{t-man} M{\\'a}rk Jelasity, Alberto Montresor, Ozalp Babaoglu.\n  \\emph{T-Man: Gossip-based fast overlay topology construction.}  Computer\n  Networks (CN) 53(13):2321-2339, 2009.\n\n  \\cite{cyclon} Spyros Voulgaris, Daniela Gavidia, Maarten van Steen.\n  \\emph{CYCLON: Inexpensive Membership Management for Unstructured P2P\n    Overlays.} J. Network Syst. Manage. 13(2): 2005.\n\n%% Tallat loopy rings, tallat thesis\n\\end{quote}\n\n\\subsection*{Gossiping and Topology Inference}\nFor some experiments, we implemented so called Vivaldi\ncoordinates~\\cite{vivaldi}. They can be used to estimate the network latency\nbetween arbitrary nodes.\n\nThe implementation can be found in \\code{src/vivaldi.erl}.\n\nFor some algorithms, we use estimates of global\ninformation. These estimates are aggregated with the help of gossiping\ntechniques~\\cite{gossip}.\n\nThe implementation can be found in \\code{src/gossip.erl}.\n\n{\\bf Publications of the \\scalaris{} team}\n\\begin{quote} \\small\n  Jens V. Fischer. \\emph{A Gossiping Framework for Scalaris.} Bachelor\n  thesis, FU-Berlin, 2014.\n\n  Marie Hoffmann. \\emph{Approximate Algorithms for Distributed Systems.} Master\n  thesis, FU-Berlin, 2012.\n\n  Thorsten Schtt, Alexander Reinefeld, Florian Schintke, Marie Hoffmann.\n  \\emph{Gossip-based Topology Inference for Efficient Overlay Mapping on\n    Data Centers.} Peer-to-Peer Computing, pp. 147-150, 2009.\n\\end{quote}\n\n\\paragraph{Related work}\n\\begin{quote} \\small\n\\cite{gossip} M{\\'a}rk Jelasity, Alberto Montresor, Ozalp Babaoglu.\n \\emph{Gossip-based aggregation in large dynamic networks.}\n ACM Trans. Comput. Syst. 23(3), 219-252 (2005).\n\n \\cite{vivaldi} Frank Dabek, Russ Cox, Frans Kaahoek, Robert Morris.\n \\emph{Vivaldi: A Decentralized Network Coordinate System.}  ACM SIGCOMM\n 2004.\n\\end{quote}\n\n\\subsection*{Load-Balancing}\n\n{\\bf Publications of the \\scalaris{} team}\n\\begin{quote} \\small\n\n  Maximilian Michels. \\emph{Request-Based Load Balancing in\n    Distributed Hash Tables.}  Master thesis, FU-Berlin, 2014.\n\n  Mikael Hgqvist, Nico Kruber. \\emph{Passive/Active Load Balancing with\n    Informed Node Placement in DHTs.} IWSOS, Thrasyvoulos Spyropoulos, Karin\n  Hummel (Eds.)pp. 101-112, Vol.5918, Lecture Notes in Computer Science,\n  2009.\n\n  Nico Kruber. \\emph{DHT Load Balancing with Estimated Global\n    Information.} Diploma thesis, HU-Berlin, 2009.\n\n  Mikael Hgqvist, Seif Haridi, Nico Kruber, Alexander Reinefeld, Thorsten\n  Schtt. \\emph{Using Global Information for Load Balancing in DHTs.}\n  Workshop on Decentralized Self Management for Grids, P2P, and User\n  Communities, 2008.\n\n  Simon Rieche. \\emph{Lastbalancierung in Peer-to-Peer Systemen.} Diploma\n  thesis, FU-Berlin, 2003.\n\\end{quote}\n\n\\paragraph{Related work}\n\\begin{quote} \\small\n\n  David R. Karger, Matthias Ruhl. \\emph{Simple efficient load-balancing\n    algorithms for peer-to-peer systems.} Theory of Computing Systems,\n  39(6):787--804, November 2006.\n\n  Ashwin R. Bharambe, Mukesh Agrawal, Srinivasan Seshan. \\emph{Mercury:\n    support- ing scalable multi-attribute range queries.} SIGCOMM\n  Comput. Commun. Rev., 34(4):353--366, 2004.\n\n\\end{quote}\n\n\\subsection*{Self-Management}\n\n{\\bf Publications of the \\scalaris{} team}\n\\begin{quote} \\small\n\nT. Schtt, A. Reinefeld, F. Schintke, C. Hennig. \\emph{Self-Adaptation in\n  Large-Scale Systems.} Architectures and Languages for Self-Managing\nDistributed Systems (SelfMan@SASO), 2009.\n\nP. Van Roy, S. Haridi, A. Reinefeld, J.-B. Stefani, R. Yap,\nT. Coupaye. \\emph{Self Management for Large-Scale Distributed Systems.}\nFormal Methods for Components and Objects 2007 (FMCO 2007), 2008.\n\nP. Van Roy, A. Ghodsi, S. Haridi, J.-B. Stefani, T. Coupaye, A. Reinefeld,\nE. Winter, R. Yap. \\emph{Self Management of Large-Scale Distributed Systems\n  by Combining Peer-to-Peer Networks and Components}, 2005.\n\\end{quote}\n\n\\subsection*{Other Topics}\n\n{\\bf Publications of the \\scalaris{} team}\n\\begin{quote} \\small\n\n  {\\bf Data Placement}\n\n  M. Hgqvist, S. Plantikow. \\emph{Towards Explicit Data Placement in\n    Scalable Key/Value Stores.}  Architectures and Languages for\n  Self-Managing Distributed Systems (SelfMan@SASO), 2009.\n\n  {\\bf Consistency}\n\n  Tallat Shafaat, Monika Moser, Ali Ghodsi, Thorsten Schtt, Seif Haridi,\n  Alexander Reinefeld. \\emph{Key-Based Consistency and Availability in\n    Structured Overlay Networks.}  International ICST Conference on Scalable\n  Information Systems, 2008.\n\n  Tallat Shafaat, Monika Moser, Ali Ghodsi, Thorsten Schtt, Alexander\n  Reinefeld. \\emph{On Consistency of Data in Structured Overlay Networks.}\n  Coregrid Integration Workshop, 2008.\n\n  {\\bf Snapshots}\n\n  Stefan Keidel. \\emph{Snapshots in Scalaris.} Diploma thesis, HU-Berlin,\n  2012.\n\n  {\\bf Replication and Replica Repair}\n\n  Nico Kruber, Maik Lange, Florian Schintke. \\emph{Approximate Hash-Based Set\n    Reconciliation for Distributed Replica Repair.} 2015 IEEE 34th International\n    Symposium on Reliable Distributed Systems (SRDS), pp. 166--175, 2015.\n\n  Maik Lange. \\emph{Redundanzverwaltung in konsistenten verteilten\n    Datenbanken.} Diploma thesis, HU-Berlin, 2012.\n\n  {\\bf Map Reduce}\n\n  Jan Fajerski. \\emph{Map Reduce on Distributed Hash Tables.}\n  Diploma thesis, HU-Berlin, 2014.\n\n\\end{quote}\n\n"
  },
  {
    "path": "user-dev-guide/user-systemuse.tex",
    "content": "\\chapter{Using the system}\n\\label{chapter.systemuse}\n\n\\scalaris{} can be used with one of the provided command line interfaces or\nby using one of the APIs in a custom program. The following sections will\ndescribe the APIs in general, each API in more detail and the use of our\ncommand line interfaces.\n\n\\section{Application Programming Interfaces (APIs)}\n\\label{chapter.systemuse.apis}\n\nCurrently we offer the following APIs:\n\\begin{itemize}\n  \\item an \\emph{Erlang API} running on the node \\scalaris{} is run\\\\\n        (functions can be called using remote connections with distributed\n        Erlang)\n  \\item a \\emph{Java API} using Erlang's \\code{JInterface} library\\\\\n        (connections are established using distributed Erlang)\n  \\item a generic \\emph{JSON API}\\\\\n        (offered by an integrated HTTP server running on each \\scalaris{} node)\n  \\item a \\emph{Python API} for Python >= 2.6 using JSON to talk to \\scalaris{}.\n  \\item a \\emph{Ruby API} for Ruby >= 1.8 using JSON to talk to \\scalaris{}.\n\\end{itemize}\n\nEach API contains methods for accessing functions from the three layers\n\\scalaris{} is composed of.  Table~\\ref{tab.api.layers} shows the modules\nand classes of Erlang, Java, Python and Ruby and their mapping to these\nlayers. Details about the supported operations and how to access them in\neach of the APIs are provided in Section~\\sieheref{sec:apis.ops}. A more\ndetailed discussion about the generic JSON API including examples of JSON\ncalls is shown in Section~\\sieheref{sec.api.json}.\n\n\\begin{table}\n  \\centering\n    \\begin{tabular}{p{2cm}llll}\n    \\toprule\n      & Erlang                 & Java                         & JSON                & Python / Ruby              \\\\\n      & \\footnotesize{module}  & \\footnotesize{class in \\code{de.zib.scalaris}}%\n                                                              & \\footnotesize{file in \\code{<URL>/api/}}%\n                                                                                    & \\footnotesize{class in module \\code{scalaris}} \\\\\n    \\midrule\n    \\multirow{3}{2cm}{Transaction\\\\Layer}\n      & \\code{api_tx}          & \\code{Transaction},          & \\code{tx.yaws}      & \\code{Transaction},        \\\\\n      &                        & \\code{TransactionSingleOp}   &                     & \\code{TransactionSingleOp} \\\\\n    \\cmidrule(lr){1-5}\n    \\multirow{2}{2cm}{Replication\\\\Layer}\n      & \\code{api_rdht}        & \\code{ReplicatedDHT}         & \\code{rdht.yaws}    & \\code{ReplicatedDHT}       \\\\\n      &                        &                              &                     &                            \\\\\n    \\cmidrule(lr){1-5}\n    \\multirow{4}{2cm}{P2P Layer}\n      & \\code{api_dht}         &                              &                     &                            \\\\\n    %\\cmidrule(lr){2-5}\n      & \\code{api_dht_raw}     &                              & \\code{dht_raw.yaws} &                            \\\\\n      & \\code{api_vm}          & \\code{ScalarisVM}            &                     &                            \\\\\n      & \\code{api_monitor}     & \\code{Monitor}               & \\code{monitor.yaws} &                            \\\\\n    \\bottomrule\n    \\end{tabular}\n    \\caption{Layered API structure}\n    \\label{tab.api.layers}\n\\end{table}\n\n\\subsection{Supported Types}\n\nDifferent programming languages have different types. In order for our APIs\nto be compatible with each other, only a subset of the available types is\nofficially supported.\n\n\\emph{Keys} are always strings. In order to avoid problems with different\nencodings on different systems, we suggest to only use ASCII characters.\n\nFor \\emph{values} we distinguish between \\emph{native}, \\emph{composite}\nand \\emph{custom} types (refer to Table~\\ref{tab.api.supported_types} for the\nmapping to the language-specific types of each API).\n\n\\emph{Native} types are\n\\begin{itemize}\n  \\item boolean values\n  \\item integer numbers\n  \\item floating point numbers\n  \\item strings and\n  \\item binary objects (a number of bytes).\n\\end{itemize}\n\n\\emph{Composite} types are\n\\begin{itemize}\n  \\item lists of the following elements:\n  \\begin{itemize}\n    \\item native types \\emph{(except binary objects!)},\n    \\item composite types\n  \\end{itemize}\n  \\item objects in JavaScript Object Notation (JSON)\\footnote{see \\url{http://json.org/}}\n\\end{itemize}\n\n\\emph{Custom} types include any Erlang term not covered by the previous types.\nSpecial care needs to be taken using custom types as they may not be accessible\nthrough every API or may be misinterpreted by an API. The use of them is\ndiscouraged.\n\n\\begin{table}\n  \\centering\n  \\begin{threeparttable}[b]\n    \\begin{tabular}{llllll}\n    \\toprule\n               & Erlang            & Java                         & JSON                            & Python                    & Ruby \\\\\n    \\midrule\n    boolean    & \\code{boolean()}  & \\code{bool}, \\code{Boolean}  & \\code{true}, \\code{false}       & \\code{True}, \\code{False} & \\code{true}, \\code{false} \\\\\n    integer    & \\code{integer()}  & \\code{int}, \\code{Integer}   & \\code{int}                      & \\code{int}                & \\code{Fixnum}, \\\\\n               &                   & \\code{long}, \\code{Long}     &                                 &                           & \\code{Bignum} \\\\\n               &                   & \\code{BigInteger}            &                                 &                           & \\\\\n    float      & \\code{float()}    & \\code{double}, \\code{Double} & \\code{int frac}                 & \\code{float}              & \\code{Float} \\\\\n               &                   &                              & \\code{int exp}                  &                           & \\\\\n               &                   &                              & \\code{int frac exp}             &                           & \\\\\n    string     & \\code{string()}   & \\code{String}                & \\code{string}                   & \\code{str}                & \\code{String} \\\\\n    binary     & \\code{binary()}   & \\code{byte[]}                & \\code{string}                   & \\code{bytearray}          & \\code{String} \\\\\n               &                   &                              & \\footnotesize{(base64-encoded)} &                           & \\\\\n    list(type) & \\code{[type()]}   & \\code{List<Object>}          & \\code{array}                    & \\code{list}               & \\code{Array} \\\\\n    JSON       & \\code{json_obj()}\\tnote{*} & \\code{Map<String, Object>} & \\code{object}            & \\code{dict}               & \\code{Hash}\\\\\n    custom     & \\code{any()}      & \\code{OtpErlangObject}       & \\emph{/}                        & \\emph{/}                  & \\emph{/} \\\\\n    \\bottomrule\n    \\end{tabular}\n    \\begin{tablenotes}\n      \\item[*] ~\\vspace{-1.5em}%\n\\begin{lstlisting}[language=erlang]\njson_obj() :: {struct, [Key::atom() | string(), Value::json_val()]}\njson_val() :: string() | number() | json_obj() | {array, [any()]} | true | false | null\n\\end{lstlisting}\n    \\end{tablenotes}\n    \\caption{Types supported by the \\scalaris{} APIs}\n    \\label{tab.api.supported_types}\n  \\end{threeparttable}\n\\end{table}\n\n\\subsection{Supported Operations}\n\\label{sec:apis.ops}\n\nMost operations are available to all APIs, but some (especially convenience\nmethods) are API- or language-specific. The following paragraphs\nprovide a brief overview of what is available to which API. For a full\nreference, see the documentation of the specific API.\n\n\\subsubsection{Transaction Layer}\n\n\\paragraph{Read}\nReads the value stored at a given key using quorum read.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_tx:read(Key)}\\\\\nJava:   & \\code{TransactionSingleOp.read(Key)}\\\\\nJSON:   & \\code{tx.yaws/read(Key)}\\\\\nPython: & \\code{TransactionSingleOp.read(Key)}\\\\\nRuby:   & \\code{TransactionSingleOp.read(Key)}\n\\end{tabular}\n\n\\paragraph{Write}\nWrites a value to a given key.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_tx:write(Key, Value)}\\\\\nJava:   & \\code{TransactionSingleOp.write(Key, Value)}\\\\\nJSON:   & \\code{tx.yaws/write(Key, Value)}\\\\\nPython: & \\code{TransactionSingleOp.write(Key, Value)}\\\\\nRuby:   & \\code{TransactionSingleOp.write(Key, Value)}\n\\end{tabular}\n\n\\paragraph{``Add to'' \\& ``Delete from'' List Operations}\nFor the list stored at a given key, first add all elements from a given list,\nthen remove all elements from a second given list.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_tx:add_del_on_list(Key, ToAddList, ToRemoveList)}\\\\\nJava:   & \\code{TransactionSingleOp.addDelOnList(Key, ToAddList, ToRemoveList)}\\\\\nJSON:   & \\code{tx.yaws/add_del_on_list(Key, ToAddList, ToRemoveList)}\\\\\nPython: & \\code{TransactionSingleOp.add_del_on_list(Key, ToAddList, ToRemoveList)}\\\\\nRuby:   & \\code{TransactionSingleOp.add_del_on_list(Key, ToAddList, ToRemoveList)}\n\\end{tabular}\n\n\\paragraph{Add to a number}\nAdds a given number to the number stored at a given key.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_tx:add_on_nr(Key, ToAddNumber)}\\\\\nJava:   & \\code{TransactionSingleOp.addOnNr(Key, ToAddNumber)}\\\\\nJSON:   & \\code{tx.yaws/add_on_nr(Key, ToAddList, ToAddNumber)}\\\\\nPython: & \\code{TransactionSingleOp.add_on_nr(Key, ToAddNumber)}\\\\\nRuby:   & \\code{TransactionSingleOp.add_on_nr(Key, ToAddNumber)}\n\\end{tabular}\n\n\\paragraph{Atomic Test and Set}\nWrites the given (new) value to a key if the current value is equal to the\ngiven old value.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_tx:test_and_set(Key, OldValue, NewValue)}\\\\\nJava:   & \\code{TransactionSingleOp.testAndSet(Key, OldValue, NewValue)}\\\\\nJSON:   & \\code{tx.yaws/add_on_nr(Key, OldValue, NewValue)}\\\\\nPython: & \\code{TransactionSingleOp.test_and_set(Key, OldValue, NewValue)}\\\\\nRuby:   & \\code{TransactionSingleOp.test_and_set(Key, OldValue, NewValue)}\n\\end{tabular}\n\n\\paragraph{Bulk Operations}\nExecutes multiple requests, i.e. operations, where each of them will be\ncommitted.\n\n\\emph{Collecting requests and executing all of them in a single call yields\nbetter performance than executing all on their own.}\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_tx:req_list_commit_each(RequestList)}\\\\\nJava:   & \\code{TransactionSingleOp.req_list(RequestList)}\\\\\nJSON:   & \\code{tx.yaws/req_list_commit_each(RequestList)}\\\\\nPython: & \\code{TransactionSingleOp.req_list(RequestList)}\\\\\nRuby:   & \\code{TransactionSingleOp.req_list(RequestList)}\n\\end{tabular}\n\n\\subsubsection{Transaction Layer (with TLog)}\n\n\\paragraph{Read (with TLog)}\nReads the value stored at a given key using quorum read as an additional part\nof a previous transaction or for starting a new one \\emph{(no auto-commit!)}.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_tx:read(TLog, Key)}\\\\\nJava:   & \\code{Transaction.read(Key)}\\\\\nJSON:   & \\code{n/a} - use \\code{req_list}\\\\\nPython: & \\code{Transaction.read(Key)}\\\\\nRuby:   & \\code{Transaction.read(Key)}\n\\end{tabular}\n\n\\paragraph{Write (with TLog)}\nWrites a value to a given key as an additional part\nof a previous transaction or for starting a new one \\emph{(no auto-commit!)}.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_tx:write(TLog, Key, Value)}\\\\\nJava:   & \\code{Transaction.write(Key, Value)}\\\\\nJSON:   & \\code{n/a} - use \\code{req_list}\\\\\nPython: & \\code{Transaction.write(Key, Value)}\\\\\nRuby:   & \\code{Transaction.write(Key, Value)}\n\\end{tabular}\n\n\\paragraph{``Add to'' \\& ``Delete from'' List Operations (with TLog)}\nFor the list stored at a given key, first add all elements from a given list,\nthen remove all elements from a second given list as an additional part\nof a previous transaction or for starting a new one \\emph{(no auto-commit!)}.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_tx:add_del_on_list(TLog, Key, ToAddList, ToRemoveList)}\\\\\nJava:   & \\code{Transaction.addDelOnList(Key, ToAddList, ToRemoveList)}\\\\\nJSON:   & \\code{n/a} - use \\code{req_list}\\\\\nPython: & \\code{Transaction.add_del_on_list(Key, ToAddList, ToRemoveList)}\\\\\nRuby:   & \\code{Transaction.add_del_on_list(Key, ToAddList, ToRemoveList)}\n\\end{tabular}\n\n\\paragraph{Add to a number (with TLog)}\nAdds a given number to the number stored at a given key as an additional part\nof a previous transaction or for starting a new one \\emph{(no auto-commit!)}.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_tx:add_on_nr(TLog, Key, ToAddNumber)}\\\\\nJava:   & \\code{Transaction.addOnNr(Key, ToAddNumber)}\\\\\nJSON:   & \\code{n/a} - use \\code{req_list}\\\\\nPython: & \\code{Transaction.add_on_nr(Key, ToAddNumber)}\\\\\nRuby:   & \\code{Transaction.add_on_nr(Key, ToAddNumber)}\n\\end{tabular}\n\n\\paragraph{Atomic Test and Set (with TLog)}\nWrites the given (new) value to a key if the current value is equal to the\ngiven old value as an additional part\nof a previous transaction or for starting a new one \\emph{(no auto-commit!)}.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_tx:test_and_set(TLog, Key, OldValue, NewValue)}\\\\\nJava:   & \\code{Transaction.testAndSet(Key, OldValue, NewValue)}\\\\\nJSON:   & \\code{tx.yaws/test_and_set(Key, OldValue, NewValue)}\\\\\nPython: & \\code{Transaction.test_and_set(Key, OldValue, NewValue)}\\\\\nRuby:   & \\code{Transaction.test_and_set(Key, OldValue, NewValue)}\n\\end{tabular}\n\n\\paragraph{Bulk Operations (with TLog)}\nExecutes multiple requests, i.e. operations, as an additional part\nof a previous transaction or for starting a new one \\emph{(no auto-commit!)}.\nOnly one \\code{commit} request is allowed per call!\n\n\\emph{Collecting requests and executing all of them in a single call yields\nbetter performance than executing all on their own.}\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_tx:req_list(RequestList)}, \\code{api_tx:req_list(TLog, RequestList)}\\\\\nJava:   & \\code{Transaction.req_list(RequestList)}\\\\\nJSON:   & \\code{tx.yaws/req_list(RequestList)}, \\code{req_list(TLog, RequestList)}\\\\\nPython: & \\code{Transaction.req_list(RequestList)}\\\\\nRuby:   & \\code{Transaction.req_list(RequestList)}\n\\end{tabular}\n\n\\subsubsection{Replication Layer}\n\n\\paragraph{Delete}\nTries to delete a value at a given key.\n\n\\emph{Warning: This can only be done outside the transaction layer and is thus not\nabsolutely safe. Refer to the following thread on the mailing list:\n\\url{http://groups.google.com/group/scalaris/browse_thread/thread/ff1d9237e218799}}.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_rdht:delete(Key)}, \\code{api_rdht:delete(Key, Timeout)}\\\\\nJava:   & \\code{ReplicatedDHT.delete(Key)}, \\code{ReplicatedDHT.delete(Key, Timeout)}\\\\\nJSON:   & \\code{rdht.yaws/delete(Key)}, \\code{rdht.yaws/delete(Key, Timeout)}\\\\\nPython: & \\code{ReplicatedDHT.delete(Key)}, \\code{ReplicatedDHT.delete(Key, Timeout)}\\\\\nRuby:   & \\code{ReplicatedDHT.delete(Key)}, \\code{ReplicatedDHT.delete(Key, Timeout)}\n\\end{tabular}\n\n\\paragraph{Get Replica Keys}\nGets the (hashed) keys used for the replicas of a given (user) key\n(ref. Section~\\nameref{sec:apis.ops.p2player}).\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_rdht:get_replica_keys(Key)}\\\\\nJava:   & \\code{n/a}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\subsubsection{P2P Layer}\n\\label{sec:apis.ops.p2player}\n\n\\paragraph{Hash Key}\nGenerates the hash of a given (user) key.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_dht:hash_key(Key)}\\\\\nJava:   & \\code{n/a}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Get Replica Keys}\nGets the (hashed) keys used for the replicas of a given (hashed) key.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_dht_raw:get_replica_keys(HashedKey)}\\\\\nJava:   & \\code{n/a}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Range Read}\nReads all Key-Value pairs in a given range of (hashed) keys.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_dht_raw:range_read(StartHashedKey, EndHashedKey)}\\\\\nJava:   & \\code{n/a}\\\\\nJSON:   & \\code{dht_raw.yaws/range_read(StartHashedKey, EndHashedKey)}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\subsubsection{P2P Layer (VM Management)}\n\n\\paragraph{Get Scalaris Version}\nGets the version of \\scalaris{} running in the requested Erlang VM.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_vm:get_version()}\\\\\nJava:   & \\code{ScalarisVM.getVersion()}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Get Node Info}\nGets various information about the requested Erlang VM and the running\n\\scalaris{} code, e.g. \\scalaris{} version, erlang version, memory use, uptime.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_vm:get_info()}\\\\\nJava:   & \\code{ScalarisVM.getInfo()}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Get Information about Different VMs}\nGet connection info about other Erlang VMs running \\scalaris{} nodes.\nNote: This info is provided by the \\erlmodule{cyclon} service built into\n\\scalaris{}.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_vm:get_other_vms(MaxVMs)}\\\\\nJava:   & \\code{ScalarisVM.getOtherVMs(MaxVMs)}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Get Number of Scalaris Nodes in the VM}\nGets the number of Scalaris nodes running inside the Erlang VM.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_vm:number_of_nodes()}\\\\\nJava:   & \\code{ScalarisVM.getNumberOfNodes()}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Get Scalaris Nodes}\nGets a list of Scalaris nodes running inside the Erlang VM.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_vm:get_nodes()}\\\\\nJava:   & \\code{ScalarisVM.getNodes()}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Add Scalaris Nodes}\nStarts additional Scalaris nodes inside the Erlang VM.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_vm:add_nodes(Number)}\\\\\nJava:   & \\code{ScalarisVM.addNodes(Number)}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Shutdown Scalaris Nodes}\nGracefully kill some Scalaris nodes inside the Erlang VM. This will first move\nthe data from the nodes to other nodes and then shut them down.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_vm:shutdown_node(Name)},\\newline\n          \\code{api_vm:shutdown_nodes(Count)}, \\code{api_vm:shutdown_nodes_by_name(Names)}\\\\\nJava:   & \\code{ScalarisVM.shutdownNode(Name)},\\newline\n          \\code{ScalarisVM.shutdownNodes(Number)}, \\code{ScalarisVM.shutdownNodesByName(Names)}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Kill Scalaris Nodes}\nImmediately kills some Scalaris nodes inside the Erlang VM.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_vm:kill_node(Name)},\\newline\n          \\code{api_vm:kill_nodes(Count)}, \\code{api_vm:kill_nodes_by_name(Names)}\\\\\nJava:   & \\code{ScalarisVM.killNode(Name)},\\newline\n          \\code{ScalarisVM.killNodes(Number)}, \\code{ScalarisVM.killNodesByName(Names)}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Shutdown the Erlang VM}\nGracefully shuts down all \\scalaris{} nodes in the Erlang VM and then exits.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_vm:shutdown_vm()}\\\\\nJava:   & \\code{ScalarisVM.shutdownVM()}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Kill the Erlang VM}\nImmediately kills all \\scalaris{} nodes in the Erlang VM and then exits.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_vm:kill_vm()}\\\\\nJava:   & \\code{ScalarisVM.killVM()}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\subsubsection{P2P Layer (Monitoring)}\n\n\\paragraph{Get Node Info}\nGets some information about the node, e.g. \\scalaris{} version, Erlang version,\nnumber of \\scalaris{} nodes in the VM.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_monitor:get_node_info()}\\\\\nJava:   & \\code{Monitor.getNodeInfo()}\\\\\nJSON:   & \\code{monitor.yaws/get_node_info()}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Get Node Performance}\nGets some performance information about the node, e.g. the average latency and\nstandard deviation of transactional operations.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_monitor:get_node_performance()}\\\\\nJava:   & \\code{Monitor.getNodePerformance()}\\\\\nJSON:   & \\code{monitor.yaws/get_node_performance()}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Get Service Info}\nGets some information about the whole \\scalaris{} ring (may be estimated if no\nmanagement server is used). Includes the overall load and the total number of\nnodes in the ring.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_monitor:get_service_info()}\\\\\nJava:   & \\code{Monitor.getServiceInfo()}\\\\\nJSON:   & \\code{monitor.yaws/get_service_info()}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Get Service Performance}\nGets some performance information about the whole \\scalaris{} ring, e.g. the\naverage latency and standard deviation of transactional operations. Both are\naggregated and may be estimates.\n\n\\begin{tabular}{lp{14cm}}\nErlang  & \\code{api_monitor:get_service_performance()}\\\\\nJava:   & \\code{Monitor.getServicePerformance()}\\\\\nJSON:   & \\code{monitor.yaws/get_service_performance()}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\subsubsection{Convenience Methods / Classes}\n\n\\paragraph{Connection Pool}\nImplements a thread-safe pool of connections to Scalaris instances. Can be\ninstantiated with a fixed maximum number of connections. Connections are either taken from\na pool of available connections or are created on demand. If finished, a connection can be\nput back into the pool.\n\n\\begin{tabular}{ll}\nErlang  & \\code{n/a}\\\\\nJava:   & \\code{ConnectionPool}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{ConnectionPool}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\paragraph{Connection Policies}\nDefines policies on how to select a node to connect to from a set of\npossible nodes and whether and how to automatically re-connect.\n\n\\begin{tabular}{ll}\nErlang  & \\code{n/a}\\\\\nJava:   & \\code{ConnectionPolicy}\\\\\nJSON:   & \\code{n/a}\\\\\nPython: & \\code{n/a}\\\\\nRuby:   & \\code{n/a}\n\\end{tabular}\n\n\\subsection{JSON API}\n\\label{sec.api.json}\n\n\\scalaris{} supports a JSON API for transactions. To minimize the necessary\nround trips between a client and \\scalaris{}, it uses request lists, which\ncontain all requests that can be done in parallel. The request list is then\nsend to a \\scalaris{} node with a POST message. The result contains\na list of the results of the requests and - in case of a transaction - a\nTransLog. To add further\nrequests to the transaction, the TransLog and another list of requests may\nbe send to \\scalaris{}. This process may be repeated as often as necessary.\nTo finish the transaction, the request list can contain a 'commit' request\nas the last element, which triggers the validation phase of the transaction\nprocessing.\nRequest lists are also supported for single read/write operations, i.e.\nevery single operation is committed on its own.\n\nThe JSON-API can be accessed via the \\scalaris{}-Web-Server running on port\n8000 by default and pages under \\code{<URL>/api/}.\nFor backwards-compatibility the page \\code{<URL>/jsonrpc.yaws} provides some\nfunctions otherwise provided by the different pages under \\code{<URL>/api/} but\nbeware that this may be removed in future.\nOther examples include \\url{http://localhost:8000/api/tx.yaws}.\nSee Table~\\sieheref{tab.api.layers} for a mapping of the layers to the\ndifferent pages.\nRequests are issued by sending a JSON object with header\n\\code{\"Content-type\"=\"application/json\"} to this URL.\nThe result will then be returned as a JSON object with the same content type.\nThe following table shows how both objects look like:\n\n\\begin{tabular}{p{0.45\\textwidth}cp{0.45\\textwidth}}\n\\bf Request & & \\bf Result \\\\\n\\begin{lstlisting}[language=java]\n{\n  \"jsonrpc\": \"2.0\",\n  \"method\" : \"<method>\",\n  \"params\" : [<params>],\n  \"id\"     : <number>\n}\n\\end{lstlisting}\n& &\n\\begin{lstlisting}[language=java]\n{\n  \"result\" : <result_object>,\n  \"id\"     : <number>\n}\n\\end{lstlisting}\n\\end{tabular}\n\nThe \\code{id} in the request can be an arbitrary number which identifies the\nrequest and is returned in the result.\nThe following operations (shown as \\code{<method>(<params>)}) are currently\nsupported (the given result is the \\code{<result_object>} mentioned above):\n\\begin{itemize}\n  \\item[] \\hspace{-1.7em}generic, e.g. for testing - \\code{<URL>/api/*.yaws}\n  \\item \\code{nop(Value)} - no operation, result:\n\\begin{lstlisting}[language=java]\n\"ok\"\n\\end{lstlisting}\n  \\item[] \\hspace{-1.7em}single operations, e.g. read/write - \\code{<URL>/api/tx.yaws}:\n  \\item \\code{req_list_commit_each(<req_list_ce>)} - commit each request in the list, result:\n\\begin{lstlisting}[language=java]\n{[{\"status\": \"ok\"} or {\"status\": \"ok\", \"value\": <json_value>} or\n  {\"status\": \"fail\", \"reason\": \"timeout\" or \"abort\" or \"not_found\" or\n                               \"not_a_list\" or \"not_a_number\"} or\n  {\"status\": \"fail\", \"reason\": \"key_changed\", \"value\": <json_value>}]}\n\\end{lstlisting}\n  \\item \\code{read(<key>)} - read the value at \\code{key}, result:\n\\begin{lstlisting}[language=java]\n{\"status\": \"ok\", \"value\", <json_value>} or\n{\"status\": \"fail\", \"reason\": \"timeout\" or \"not_found\"}\n\\end{lstlisting}\n  \\item \\code{write(<key>, <json_value>)} - write \\code{value} (inside \\code{json_value}) to \\code{key}, result:\n\\begin{lstlisting}[language=java]\n{\"status\": \"ok\"} or\n{\"status\": \"fail\", \"reason\": \"timeout\" or \"abort\"}\n\\end{lstlisting}\n  \\item \\code{add_del_on_list(<key>, ToAdd, ToRemove)} - adding to / removing from a list\n  (for the list at \\code{key} adds all values in the \\code{ToAdd} list and\n  then removes all values in the \\code{ToRemove} list; if there is no value at\n  \\code{key}, uses an empty list - both value lists are \\code{[<value>]}), result:\n\\begin{lstlisting}[language=java]\n{\"status\": \"ok\"} or\n{\"status\": \"fail\", \"reason\": \"timeout\" or \"abort\" or \"not_a_list\"}\n\\end{lstlisting}\n  \\item \\code{add_on_nr(<key>, <value>)} - adding to a number\n  (adds \\code{value} to the number at \\code{key} - both values must be numbers), result:\n\\begin{lstlisting}[language=java]\n{\"status\": \"ok\"} or\n{\"status\": \"fail\", \"reason\": \"timeout\" or \"abort\" or \"not_a_number\"}\n\\end{lstlisting}\n  \\item \\code{test_and_set(<key>, OldValue, NewValue)} - atomic test-and-set\n  (write \\code{NewValue} to \\code{key} if the current value is \\code{OldValue}\n   - both values are \\code{<json_value>}), result:\n\\begin{lstlisting}[language=java]\n{\"status\": \"ok\"} or\n{\"status\": \"fail\", \"reason\": \"timeout\" or \"abort\" or \"not_found\"} or\n{\"status\": \"fail\", \"reason\": \"key_changed\", \"value\": <json_value>}\n\\end{lstlisting}\n  \\item[] \\hspace{-1.7em}transactions - \\code{<URL>/api/tx.yaws}:\n  \\item \\code{req_list(<req_list>)} - process a list of requests, result:\n\\begin{lstlisting}[language=java]\n{\"tlog\": <tlog>,\n \"results\": [{\"status\": \"ok\"} or {\"status\": \"ok\", \"value\": <json_value>} or\n             {\"status\": \"fail\", \"reason\": \"timeout\" or \"abort\" or \"not_found\" or\n                                          \"not_a_list\" or \"not_a_number\"} or\n             {\"status\": \"fail\", \"reason\": \"key_changed\", \"value\": <json_value>}]}\n\\end{lstlisting}\n  \\item \\code{req_list(<tlog>, <req_list>)} - process a list of requests with a previous translog, result:\n\\begin{lstlisting}[language=java]\n{\"tlog\": <tlog>,\n \"results\": [{\"status\": \"ok\"} or {\"status\": \"ok\", \"value\": <json_value>} or\n             {\"status\": \"fail\", \"reason\": \"timeout\" or \"abort\" or \"not_found\" or\n                                          \"not_a_list\" or \"not_a_number\"} or\n             {\"status\": \"fail\", \"reason\": \"key_changed\", \"value\": <json_value>}]}\n\\end{lstlisting}\n  \\item[] \\hspace{-1.7em}replication layer functions - \\code{<URL>/api/rdht.yaws}:\n  \\item \\code{delete(<key>)} - delete the value at \\code{key}, default timeout 2s, result:\n\\begin{lstlisting}[language=java]\n{\"ok\": <number>, \"results\": [\"ok\" or \"locks_set\" or \"undef\"]} or\n{\"failure\": \"timeout\", \"ok\": <number>, \"results\": [\"ok\" or \"locks_set\" or \"undef\"]}\n\\end{lstlisting}\n  \\item \\code{delete(<key>, Timeout)} - delete the value at \\code{key} with a timeout of \\code{Timeout} Milliseconds, result:\n\\begin{lstlisting}[language=java]\n{\"ok\": <number>, \"results\": [\"ok\" or \"locks_set\" or \"undef\"]} or\n{\"failure\": \"timeout\", \"ok\": <number>, \"results\": [\"ok\" or \"locks_set\" or \"undef\"]}\n\\end{lstlisting}\n  \\item[] \\hspace{-1.7em}raw DHT functions - \\code{<URL>/api/dht_raw.yaws}:\n  \\item \\code{range_read(From, To)} - read a range of (raw) keys, result:\n\\begin{lstlisting}[language=java]\n{\"status\": \"ok\" or \"timeout\",\n \"value\": [{\"key\": <key>, \"value\": <json_value>, \"version\": <version>}]}\n\\end{lstlisting}\n  \\item[] \\hspace{-1.7em}monitor - \\code{<URL>/api/monitor.yaws}:\n  \\item \\code{get_node_info()} - gets some information about the node, result:\n\\begin{lstlisting}[language=java]\n{\"status\": \"ok\" or \"timeout\",\n \"value\": [{\"scalaris_version\": <version_string>,\n            \"erlang_version\": <version_string>,\n            \"dht_nodes\": <number>}]}\n\\end{lstlisting}\n  \\item \\code{get_node_performance()} - gets some performance information about the node, result:\n\\begin{lstlisting}[language=java]\n{\"status\": \"ok\" or \"timeout\",\n \"value\": [{\"latency_avg\": <perf_data>, \"latency_stddev\": <perf_data>}]}\n\\end{lstlisting}\n  \\item \\code{get_service_info()} - gets some information about the \\scalaris{} ring, result:\n\\begin{lstlisting}[language=java]\n{\"status\": \"ok\" or \"timeout\",\n \"value\": [{\"total_load\": <number>, \"nodes\": <number>}]}\n\\end{lstlisting}\n  \\item \\code{get_service_performance()} - gets some performance information about the \\scalaris{} ring, result:\n\\begin{lstlisting}[language=java]\n{\"status\": \"ok\" or \"timeout\",\n \"value\": [{\"latency_avg\": <perf_data>, \"latency_stddev\": <perf_data>}]}\n\\end{lstlisting}\n\\end{itemize}\n\nNote:\n\\begin{lstlisting}[language=java]\n<json_value> = {\"type\": \"as_is\" or \"as_bin\", \"value\": <value>}\n<operation> = {\"read\": <key>} or {\"write\", {<key>: <json_value>}} or\n              {\"add_del_on_list\": {\"key\": <key>, \"add\": [<value>], \"del\": [<value>]}} or\n              {\"add_on_nr\": {<key>: <value>}} or\n              {\"test_and_set\": {\"key\": <key>, \"old\": <json_value>, \"new\": <json_value>}}\n<req_list_ce> = [<operation>]\n<req_list> = [<operation> or {\"commit\", _}]\n<perf_data> = {<number>: <perf_val>,...}\n\\end{lstlisting}\nThe \\code{<value>} inside \\code{<json_value>} is either a base64-encoded\nstring representing a binary object (type = \\code{\"as_bin\"}) or the value\nitself (type = \\code{\"as_is\"}).\n\n\\subsubsection{JSON-Example}\n\nThe following example illustrates the message flow:\n\n%note: use tables inside tables instead of multicol for easier listings handling\n\\begin{longtable}{p{0.97\\textwidth}}\n\\begin{tabular}{p{0.43\\textwidth}cp{0.43\\textwidth}}\n\\bf Client & & \\hfill\\bf \\scalaris{} node \\\\\n\\end{tabular} \\\\\n%\n% request:\n\\begin{tabular}{p{0.43\\textwidth}cp{0.43\\textwidth}}\nMake a transaction, that sets two keys & $\\to$ & \\\\\n\\end{tabular}\\vspace{-1.5em} \\\\\n%\n\\begin{tabular}{p{0.78\\textwidth}p{0.13\\textwidth}}\n\\vspace{-1.5em}%\n\\begin{lstlisting}[language=java]\n{\"jsonrpc\": \"2.0\",\n \"method\": \"req_list\",\n \"params\": [\n  [ { \"write\": { \"keyA\": {\"type\": \"as_is\", \"value\": \"valueA\"} } },\n    { \"write\": { \"keyB\": {\"type\": \"as_is\", \"value\": \"valueB\"} } },\n    { \"commit\": \"\" } ]\n ],\n \"id\": 0\n}\n\\end{lstlisting}\n& \\\\\n\\end{tabular}\\vspace{-1em} \\\\\n%\n% result:\n\\begin{tabular}{p{0.43\\textwidth}cp{0.43\\textwidth}}\n & $\\leftarrow$ & \\hfill{}\\scalaris{} sends results back \\\\\n\\end{tabular}\\vspace{-1.5em} \\\\\n\n\\begin{tabular}{p{0.13\\textwidth}p{0.78\\textwidth}}\n&\n\\vspace{-1.5em}%\n\\begin{lstlisting}[language=java]\n{\"error\": null,\n \"result\": {\n  \"results\": [ {\"status\": \"ok\"}, {\"status\": \"ok\"}, {\"status\": \"ok\"} ],\n  \"tlog\": <TLOG> // this is the translog for further operations!\n },\n \"id\": 0\n}\n\\end{lstlisting} \\\\\n\\end{tabular}\\vspace{-1em} \\\\\n%\n% request:\n\\begin{tabular}{p{0.43\\textwidth}cp{0.43\\textwidth}}\nIn a second transaction: Read the two keys & $\\to$ & \\\\\n\\end{tabular}\\vspace{-1.5em} \\\\\n%\n\\begin{tabular}{p{0.78\\textwidth}p{0.13\\textwidth}}\n\\vspace{-1.5em}%\n\\begin{lstlisting}[language=java]\n{\"jsonrpc\": \"2.0\",\n \"method\": \"req_list\",\n \"params\": [\n  [ { \"read\": \"keyA\" },\n    { \"read\": \"keyB\" } ]\n ],\n \"id\": 0\n}\n\\end{lstlisting}\n& \\\\\n\\end{tabular}\\vspace{-1em} \\\\\n%\n% result:\n\\begin{tabular}{p{0.43\\textwidth}cp{0.43\\textwidth}}\n & $\\leftarrow$ & \\hfill{}\\scalaris{} sends results back \\\\\n\\end{tabular}\\vspace{-1.5em} \\\\\n\n\\begin{tabular}{p{0.13\\textwidth}p{0.78\\textwidth}}\n&\n\\vspace{-1.5em}%\n\\begin{lstlisting}[language=java]\n{\"error\": null,\n \"result\": {\n  \"results\": [\n   { \"status\": \"ok\", \"value\": {\"type\": \"as_is\", \"value\": \"valueA\"} },\n   { \"status\": \"ok\", \"value\": {\"type\": \"as_is\", \"value\": \"valueB\"} }\n  ],\n  \"tlog\": <TLOG>\n },\n \"id\": 0\n}\n\\end{lstlisting} \\\\\n\\end{tabular} \\\\\n%\n% request:\n\\begin{tabular}{p{0.43\\textwidth}cp{0.43\\textwidth}}\nCalculate something with the read values and make further requests, here a\nwrite and the commit for the whole transaction. Also include the latest\ntranslog we got from \\scalaris{} (named \\code{<TLOG>} here). & $\\to$ & \\\\\n\\end{tabular}\\vspace{-1.5em} \\\\\n%\n\\begin{tabular}{p{0.78\\textwidth}p{0.13\\textwidth}}\n\\vspace{-1.5em}%\n\\begin{lstlisting}[language=java]\n{\"jsonrpc\": \"2.0\",\n \"method\": \"req_list\",\n \"params\": [\n  <TLOG>,\n  [ { \"write\": { \"keyA\": {\"type\": \"as_is\", \"value\": \"valueA2\"} } },\n    { \"commit\": \"\" } ]\n ],\n \"id\": 0\n}\n\\end{lstlisting}\n& \\\\\n\\end{tabular}\\vspace{-1em} \\\\\n%\n% result:\n\\begin{tabular}{p{0.43\\textwidth}cp{0.43\\textwidth}}\n & $\\leftarrow$ & \\hfill{}\\scalaris{} sends results back \\\\\n\\end{tabular}\\vspace{-1.5em} \\\\\n\n\\begin{tabular}{p{0.13\\textwidth}p{0.78\\textwidth}}\n&\n\\vspace{-1.5em}%\n\\begin{lstlisting}[language=java]\n{\"error\": null,\n \"result\": {\n  \"results\": [ {\"status\": \"ok\"}, {\"status\": \"ok\"} ],\n  \"tlog\": <TLOG>\n },\n \"id\": 0\n}\n\\end{lstlisting} \\\\\n\\end{tabular} \\\\\n\\end{longtable}\n\nExamples of how to use the JSON API are the Python and Ruby API which use JSON\nto communicate with \\scalaris{}.\n\n%\\subsection{Erlang}\n\n\\subsection{Java API}\n\nThe \\code{scalaris.jar} provides a Java command line client as well as a\nlibrary for Java programs to access \\scalaris{}. The library provides several\nclasses:\n\n\\begin{itemize}\n\\item \\code{TransactionSingleOp} provides methods for reading and writing values.\n\\item \\code{Transaction} provides methods for reading and writing values in transactions.\n\\item \\code{ReplicatedDHT} provides low-level methods for accessing the replicated DHT of \\scalaris{}.\n\\end{itemize}\n\nFor details regarding the API we refer the reader to the Javadoc:\n\n\\begin{lstlisting}[language=sh]\n%> cd java-api\n%> ant doc\n%> firefox doc/index.html\n\\end{lstlisting}\n\n\\section{Command Line Interfaces}\n\n\\subsection{Java command line interface}\n\nAs mentioned above, the \\code{scalaris.jar} file contains a small command line\ninterface client. For\nconvenience, we provide a wrapper script called \\code{scalaris} which\nsets up the Java environment:\n\n\\begin{lstlisting}[language=sh]\n%> ./java-api/scalaris --noconfig --help\n\\end{lstlisting}\n\\lstinputlisting[language={}]{scalaris-client-java.out}\n\n\\code{read}, \\code{write}, \\code{delete} and similar operations can be used to read, write\nand delete from/to the overlay, respectively. The others\nprovide debugging and testing functionality.\n\n\\begin{lstlisting}[language=]\n%> ./java-api/scalaris -write foo bar\nwrite(foo, bar)\n%> ./java-api/scalaris -read foo\nread(foo) == bar\n\\end{lstlisting}\n\nPer default, the \\code{scalaris} script tries to connect to a management\nserver at \\code{localhost}. You can change the node it connects to (and\nfurther connection properties) by adapting the values defined in\n\\code{java-api/scalaris.properties}.\n\n\\subsection{Python command line interface}\n\n\\begin{lstlisting}[language=sh]\n%> ./python-api/scalaris --help\n\\end{lstlisting}\n\\lstinputlisting[language={}]{scalaris-client-python.out}\n\n\\subsection{Ruby command line interface}\n\n\\begin{lstlisting}[language=sh]\n%> ./ruby-api/scalaris --help\n\\end{lstlisting}\n\\lstinputlisting[language={}]{scalaris-client-ruby.out}\n\n\\section{Using \\scalaris{} from Erlang}\n\\label{chapter.systemuse.usingfromerl}\n\nIn this section, we will describe how to use \\scalaris{} with two small\nexamples. After having build \\scalaris{} as described\nin~\\ref{chapter.downloadinstall}, \\scalaris{} can be run from the source\ndirectory directly.\n\n\\subsection{Running a \\scalaris{} Cluster}\nIn this example, we will set up a simple \\scalaris{} cluster consisting of\nup to five nodes running on a single computer.\n\n\\paragraph{Adapt the configuration.}\nThe first step is to adapt the configuration to your needs. We use the\nsample local configuration from~\\ref{chapter.runscalaris.runtime_config},\ncopy it to \\code{bin/scalaris.local.cfg} and add a number of different known\nhosts. Note that the management server will run on the same port as the\nfirst node started in the example, hence we adapt its port as well.\n\n\\begin{lstlisting}\n{listen_ip, {127,0,0,1}}.\n{mgmt_server, {{127,0,0,1},14195,mgmt_server}}.\n{known_hosts, [{{127,0,0,1},14195, service_per_vm},\n                {{127,0,0,1},14196, service_per_vm},\n                {{127,0,0,1},14197, service_per_vm},\n                {{127,0,0,1},14198, service_per_vm}\n                % Although we will be using 5 nodes later, only 4 are added as known nodes.\n            ]}.\n\\end{lstlisting}\n\n\\paragraph{Bootstrapping.}\n\\label{par:Bootstrapping}\n\nIn a shell (from now on called \\code{S1}), start the first node (\"premier\"):\n\\begin{lstlisting}[language=sh]\n./bin/scalarisctl -m -n premier@127.0.0.1 -p 14195 -y 8000 -s -f start\n\\end{lstlisting}\nThe \\code{-m} and \\code{-f} options instruct \\code{scalarisctl} to start the\nmanagement server and the first\\_node\n(see Section~\\sieheref{user.config.scalarisctl} for further details on \\code{scalarisctl}).\nNote that the command above will\nproduce some output about unknown nodes. This is expected, as some nodes\ndefined in the configuration file above are not started yet.\n\nAfter you run the above command and no further error occurred, you can query\nthe locally available nodes using \\code{scalarisctl}. Enter into a new shell\n(called \\code{MS}):\n\\begin{lstlisting}[language=sh]\n./bin/scalarisctl list\nepmd: up and running on port 4369 with data:\nname premier at port 47235\n\\end{lstlisting}\n\n\\scalaris{} also contains a webserver. You can access it by pointing your\nbrowser to \\url{http://127.0.0.1:8000} (or the respective IP address of the\nnode). With the above example, you can see the first node (\"premier\") and\nits management role.\n\n\\paragraph{Adding Nodes.}\n\\label{par:AddingNodes}\n\nWe will now add four additional nodes to the cluster. Use a new shell\n(\\code{S2} to \\code{S5}) for each of the following commands. Each newly\nadded node is a \"real\" \\scalaris{} node and could run on another physical\ncomputer than the other nodes.\n\\begin{lstlisting}[language=sh]\n./bin/scalarisctl -n second@127.0.0.1 -p 14196 -y 8001 -s start\n./bin/scalarisctl -n n3@127.0.0.1 -p 14197 -y 8002 -s start\n./bin/scalarisctl -n n4@127.0.0.1 -p 14198 -y 8003 -s start\n./bin/scalarisctl -n n5@127.0.0.1 -p 14199 -y 8004 -s start\n\\end{lstlisting}\nNote that the last added nodes should not report a node as not reachable.\n\nThe management server should now report that the nodes have indeed joined\n\\scalaris{} successfully. Query \\code{scalarisctl}:\n\\begin{lstlisting}[language=sh]\n./bin/scalarisctl list\nepmd: up and running on port 4369 with data:\nname n5 at port 47801\nname n4 at port 54614\nname n3 at port 41710\nname second at port 44329\nname premier at port 44862\n\\end{lstlisting}\nThe actual output might differ, as the port numbers are assigned by the\noperating system.\n\nEach node offers a web console. Point your browser to any url for\n\\url{http://127.0.0.1:8001} to \\url{http://127.0.0.1:8004}. Observe that\nall nodes claim the cluster ring to consist of 5 nodes.\n\nThe web interface of node \\code{premier} differs from the other\ninterfaces. This is due to the fact that the management server is running on\nthis node, adding additional information to the web interface.\n\n\\paragraph{Entering Data Using the Web Interface.}\nA node's web interface can be used to query and enter data into \\scalaris{}. To\ntry this, point your browser to \\url{http://127.0.0.1:8000} (or any of the\nother nodes) and use the provided HTML form.\n\n\\begin{enumerate}\n    \\item Lookup key \\code{hello}. This will return \\lstinline|{fail,not_found}|\n    \\item Add new keys \\code{k1} and \\code{k2} with values \\code{v1} and\n      \\code{v2}, respectively. Then, lookup that key on the current and one\n      of the other nodes. This should return \\lstinline|{ok,\"v1\"}| and\n      \\lstinline|{ok, \"v2\"}| on both nodes.\n    \\item Update the key \\code{k1} by adding it on any node with value\n      \\code{v1updated}.\n    \\item Update the key \\code{k2} by adding it on any node with value\n      \\code{v2updated}.  Lookup the key again and you should receive\n      \\lstinline|{ok, v2updated}|\n\\end{enumerate}\n\n\\paragraph{Simulating Node Failure.}\nTo simulate a node failure, we will simply stop \\code{n4} using\n\\code{scalarisctl}:\n\n\\begin{lstlisting}[language=sh]\n./bin/scalarisctl -n n4@127.0.0.1 stop\n\\end{lstlisting}\n\nOther nodes will notice the crash of \\code{n4}. By querying the available\nnodes in the shell MS again, you will now see only 4 nodes.\n\nAlthough the node \\code{n4} left the system, the data in the system is still\nconsistent. Try to query the keys you added above. You should receive the\nvalues for each.\n\nWe will start a new node with the name \\code{n4} again:\n\\begin{lstlisting}[language=sh]\n./bin/scalarisctl -n n4@127.0.0.1 -p 14198 -y 8003 -s start\n\\end{lstlisting}\n\nThe node list (again, query \\code{scalarisctl} in shell MS) will report\n\\code{n4} as alive again. You can still lookup the keys from above and\nshould also receive the same result for the queries.\n\nAfter running the above, we went from a five-node cluster to a 4-node\ncluster and back to a five-node cluster without any data loss due to a\nleaving node. The system was not unavailable for users and would have served\nany user requests without violating the data consistency or availability.\n\n\\paragraph{Controlling \\scalaris{}  Using the Erlang Shell.}\nThe calls to \\code{scalarisctl} above which started a new \\scalaris{} node\nended within an Erlang shell. Each of those shells can be used to control a\nlocal \\scalaris{} node and issue queries to the distributed database. Enter\nshell \\code{S1} and hit <return> to see the Erlang shell prompt. Now, enter\nthe following commands and check that the output is similar to the one\nprovided here. You can stop the Erlang shell using \\code{quit().}, which\nthen also stops the corresponding \\scalaris{} node.\n\n\\begin{lstlisting}\n(premier@127.0.0.1)1> api_tx:read(\"k0\").\n{fail,not_found}\n(premier@127.0.0.1)2> api_tx:read(\"k1\").\n{ok,\"v1updated\"}\n(premier@127.0.0.1)3> api_tx:read(\"k2\").\n{ok,\"v2updated\"}\n(premier@127.0.0.1)4> api_tx:read(<<\"k1\">>).\n{ok,\"v1updated\"}\n(premier@127.0.0.1)5> api_tx:read(<<\"k2\">>).\n{ok,\"v2updated\"}\n(premier@127.0.0.1)6> api_tx:write(<<\"k3\">>,<<\"v3\">>).\n{ok}\n(premier@127.0.0.1)7> api_tx:read(<<\"k3\">>).\n{ok,<<\"v3\">>}\n(premier@127.0.0.1)8> api_tx:read(\"k3\").\n{ok,<<\"v3\">>}\n(premier@127.0.0.1)9> api_tx:write(<<\"k4\">>,{1,2,3,four}).\n{ok}\n(premier@127.0.0.1)10> api_tx:read(\"k4\").\n{ok,{1,2,3,four}}\n\\end{lstlisting}\n\n\\paragraph{Attaching a Client to \\scalaris{}.}\nNow we will connect a true client to our 5 nodes \\scalaris{} cluster. This\nclient will not be a \\scalaris{} node itself and thus represents a user\napplication interacting with \\scalaris{}.\n\nWe use a new shell to run an Erlang shell to do remote API calls to the\nserver nodes.\n\n\\begin{lstlisting}[language=sh]\nerl -name client@127.0.0.1 -hidden -setcookie 'chocolate chip cookie'\n\\end{lstlisting}\n\nThe requests to \\scalaris{} will be done using \\code{rpc:call/4}. A\nproduction system would have some more sophisticated client side module,\ndispatching requests automatically to server nodes, for example.\n\n\\begin{lstlisting}\n(client@127.0.0.1)1> net_adm:ping('n3@127.0.0.1').\npong\n(client@127.0.0.1)2> rpc:call('n3@127.0.0.1', api_tx, read, [<<\"k0\">>]).\n{fail,not_found}\n(client@127.0.0.1)3> rpc:call('n3@127.0.0.1', api_tx, read, [<<\"k4\">>]).\n{ok,{1,2,3,four}}\n(client@127.0.0.1)4> rpc:call('n4@127.0.0.1', api_tx, read, [<<\"k4\">>]).\n{ok,{1,2,3,four}}\n(client@127.0.0.1)5> rpc:call('n5@127.0.0.1', api_tx, write, [<<\"num5\">>,55]).\n{ok}\n(client@127.0.0.1)6> rpc:call('n3@127.0.0.1', api_tx, read, [<<\"num5\">>]).\n{ok,55}\n(client@127.0.0.1)7> rpc:call('n2@127.0.0.1', api_tx, add_on_nr, [<<\"num5\">>,2]).\n{badrpc,nodedown}\n(client@127.0.0.1)8> rpc:call('second@127.0.0.1', api_tx, add_on_nr, [<<\"num5\">>,2]).\n{ok}\n(client@127.0.0.1)9> rpc:call('n3@127.0.0.1', api_tx, read, [<<\"num5\">>]).\n{ok,57}\n(client@127.0.0.1)10> rpc:call('n4@127.0.0.1', api_tx, test_and_set, [<<\"num5\">>,57,59]).\n{ok}\n(client@127.0.0.1)11> rpc:call('n5@127.0.0.1', api_tx, read, [<<\"num5\">>]).\n{ok,59}\n(client@127.0.0.1)12> rpc:call('n4@127.0.0.1', api_tx, test_and_set, [<<\"num5\">>,57,55]).\n{fail,{key_changed,59}}\n(client@127.0.0.1)13> rpc:call('n3@127.0.0.1', api_tx, read, [<<\"num5\">>]).\n{ok,59}\n(client@127.0.0.1)14> rpc:call('n5@127.0.0.1', api_tx, test_and_set,\n                               [<<\"k2\">>,\"v2updated\",<<\"v2updatedTWICE\">>]).\n{ok}\n(client@127.0.0.1)15> rpc:call('n4@127.0.0.1', api_tx, read, [<<\"k2\">>]).\n{ok,<<\"v2updatedTWICE\">>}\n(client@127.0.0.1)16> rpc:call('n3@127.0.0.1', api_tx, add_on_nr, [<<\"num5\">>,-4]).\n{ok}\n(client@127.0.0.1)17> rpc:call('n4@127.0.0.1', api_tx, read, [<<\"num5\">>]).\n{ok,55}\n(client@127.0.0.1)18> q().\nok\n\\end{lstlisting}\n\nTo show that the above calls actually worked with \\scalaris{}, connect\nanother client to the cluster and read updates made by the first:\n\n\\begin{lstlisting}[language=sh]\nerl -name clientagain@127.0.0.1 -hidden -setcookie 'chocolate chip cookie'\n\\end{lstlisting}\n\n\\begin{lstlisting}\n(clientagain@127.0.0.1)1> net_adm:ping('n5@127.0.0.1').\npong\n(clientagain@127.0.0.1)2> rpc:call('n4@127.0.0.1', api_tx, read, [<<\"k0\">>]).\n{fail,not_found}\n(clientagain@127.0.0.1)3> rpc:call('n4@127.0.0.1', api_tx, read, [<<\"k1\">>]).\n{ok,\"v1updated\"}\n(clientagain@127.0.0.1)4> rpc:call('n3@127.0.0.1', api_tx, read, [<<\"k2\">>]).\n{ok,<<\"v2updatedTWICE\">>}\n(clientagain@127.0.0.1)5> rpc:call('second@127.0.0.1', api_tx, read, [<<\"num5\">>]).\n{ok,55}\n\\end{lstlisting}\n\n\\paragraph{Shutting Down \\scalaris{}.}\nFirstly, we list the available nodes using \\code{scalarisctl} using the\nshell \\code{MS}.\n\n\\begin{lstlisting}[language=sh]\n./bin/scalarisctl list\nepmd: up and running on port 4369 with data:\nname n4 at port 52504\nname n5 at port 47801\nname n3 at port 41710\nname second at port 44329\nname premier at port 44862\n\\end{lstlisting}\n\nSecondly, we shut down each of the nodes:\n\\begin{lstlisting}[language=sh]\n./bin/scalarisctl -n second@127.0.0.1 stop\n'second@127.0.0.1'\n./bin/scalarisctl -n n3@127.0.0.1 stop\n'n3@127.0.0.1'\n./bin/scalarisctl -n n4@127.0.0.1 stop\n'n4@127.0.0.1'\n./bin/scalarisctl -n n5@127.0.0.1 stop\n'n5@127.0.0.1'\n\\end{lstlisting}\n\nOnly the first node remains:\n\n\\begin{lstlisting}[language=sh]\n./bin/scalarisctl list\nepmd: up and running on port 4369 with data:\nname premier at port 44862\n\n./bin/scalarisctl -n premier@127.0.0.1 stop\n'premier@127.0.0.1'\n./bin/scalarisctl list\nepmd: up and running on port 4369 with data:\n(nothing)\n\\end{lstlisting}\n\nThe \\scalaris{} API offers more transactional operations than just\nsingle-key read and write.  The next part of this section will describe how\nto build transaction logs for atomic operations and how \\scalaris{}\nhandles conflicts in concurrently running transactions. See the module\n\\lstinline{api_tx} for more functions to access the data layer of\n\\scalaris{}.\n\n\\subsection{Transaction}\nIn this section, we will describe how to build transactions using\n\\lstinline{api_tx:req_list(Tlog, List)} on the client side.\n\nThe setup is similar to the five nodes cluster in the previous section.  To\nsimplify the example all API calls are typed inside the Erlang shells of\nnodes \\code{n4} and \\code{n5}.\n\nConsider two concurrent transactions \\code{A} and \\code{B}. \\code{A} is a\nlong-running operation, whereas \\code{B} is only a short transaction.  In\nthe example, \\code{A} starts before \\code{B} and \\code{B} ends before\n\\code{A}. \\code{B} is \"timely\" nested in \\code{A} and disturbs \\code{A}.\n\n\\paragraph{Single Read Operations.}\nWe first issue two read operations on nodes \\code{n4, n5} to see that we are\nworking on the same state for key \\code{k1}:\n\n\\begin{lstlisting}\n(n4@127.0.0.1)10> api_tx:read(<<\"k1\">>).\n{ok,<<\"v1\">>}\n(n5@127.0.0.1)17> api_tx:read(<<\"k1\">>).\n{ok,<<\"v1\">>}\n\\end{lstlisting}\n\n\\paragraph{Create Transaction Logs and Add Operations.}\nNow, we create two transaction logs for the transactions and add the\noperations which are to be run atomically. \\code{A} will be created on node\n\\code{n5}, \\code{B} on \\code{n4}:\n\\begin{lstlisting}\n(n5@127.0.0.1)18> T5longA0 = api_tx:new_tlog().\n[]\n(n5@127.0.0.1)19> {T5longA1, R5longA1} = api_tx:req_list(T5longA0, [{read, <<\"k1\">>}]).\n{[{76,<<\"k1\">>,1,75,'$empty'}],[{ok,<<\"v1\">>}]}\n(n4@127.0.0.1)11> T4shortB0 = api_tx:new_tlog().\n[]\n(n4@127.0.0.1)12> {T4shortB1, R4shortB1} = api_tx:req_list(T4shortB0, [{read, <<\"k1\">>}]).\n{[{76,<<\"k1\">>,1,75,'$empty'}],[{ok,<<\"v1\">>}]}\n(n4@127.0.0.1)13> {T4shortB2, R4shortB2} = api_tx:req_list(T4shortB1,\n                                             [{write, <<\"k1\">>, <<\"v1Bshort\">>}]).\n{[{77,<<\"k1\">>,1,75, <<131,109,0,0,0,8,118,49,66,115,104,111,114,116>>}],\n [{ok}]}\n(n4@127.0.0.1)14> {T4shortB3, R4shortB3} = api_tx:req_list(T4shortB2, [{read, <<\"k1\">>}]).\n{[{77,<<\"k1\">>,1,75, <<131,109,0,0,0,8,118,49,66,115,104,111,114,116>>}],\n [{ok,<<\"v1Bshort\">>}]}\n\\end{lstlisting}\n\nTo finish the transaction log for \\code{B}, we add\n\\lstinline|{commit}|. This operation should return an \\code{ok}:\n\n\\begin{lstlisting}\n(n4@127.0.0.1)15> {T4shortB4, R4shortB4} = api_tx:req_list(T4shortB3, [{commit}]).\n{[],[{ok}]}\n(n4@127.0.0.1)16> [R4shortB1,R4shortB2,R4shortB3,R4shortB4].\n[[{ok,<<\"v1\">>}],[{ok}],[{ok,<<\"v1Bshort\">>}],[{ok}]]\n\\end{lstlisting}\nThis concludes the creation of \\code{B}. Now we will try to commit the long\nrunning transaction \\code{A} after reading the key \\code{k1} again. This and\nfurther attempts to write the key will fail, as the transaction \\code{B}\nwrote this key since \\code{A} started.\n\\begin{lstlisting}\n(n5@127.0.0.1)20> {T5longA2, R5longA2} = api_tx:req_list(T5longA1, [{read, <<\"k1\">>}]).\n{[{76,<<\"k1\">>,2,{fail,abort},'$empty'}],\n [{ok,<<\"v1Bshort\">>}]}                                % <-- SEE #### FAIL and ABORT ####\n(n5@127.0.0.1)21> {T5longA3, R5longA3} = api_tx:req_list(T5longA2,\n                                           [{write, <<\"k1\">>,<<\"v1Along\">>}]).\n{[{76,<<\"k1\">>,2,{fail,abort},'$empty'}],[{ok}]}\n(n5@127.0.0.1)22> {T5longA4, R5longA4} = api_tx:req_list(T5longA3, [{read, <<\"k1\">>}]).\n{[{76,<<\"k1\">>,2,{fail,abort},'$empty'}],\n [{ok,<<\"v1Bshort\">>}]}\n(n5@127.0.0.1)23> {T5longA5, R5longA5} = api_tx:req_list(T5longA4, [{commit}]).\n{[],[{fail,abort,[<<\"k1\">>]}]}                         % <-- SEE #### FAIL and ABORT ####\n(n4@127.0.0.1)17> api_tx:read(<<\"k1\">>).\n{ok,<<\"v1Bshort\">>}\n(n5@127.0.0.1)24> api_tx:read(<<\"k1\">>).\n{ok,<<\"v1Bshort\">>}\n\\end{lstlisting}\n\nAs expected, the first coherent commit \\code{B} constructed on \\code{n4} has\nwon.\n\nNote that in a real system, operations in \\lstinline|api_tx:req_list(Tlog, List)| should be grouped together with a trailing \\lstinline|{commit}| as\nfar as possible. The individual separation of all reads, writes and commits\nwas done here on purpose to study the transactional behaviour.\n"
  },
  {
    "path": "user-dev-guide/user-tests.tex",
    "content": "\\chapter{Testing the system}\n\n\\section{Erlang unit tests}\nThere are some unit tests in the \\code{test} directory which test \\scalaris{}\nitself (the Erlang code). You can call them\nby running \\code{make test} in the main directory. The results are stored\nin a local \\code{index.html} file. \n\nThe tests are implemented with the \\code{common-test} package from the\nErlang system. For running the tests we rely on \\code{run\\_test},\nwhich is part of the \\code{common-test} package, but (on erlang $<$ R14) is not\ninstalled by default. \\code{configure} will check whether \\code{run\\_test} is\navailable. If it is not installed, it will show a warning and a short\ndescription of how to install the missing file.\n\nNote: for the unit tests, we are setting up and shutting down several\noverlay networks. During the shut down phase, the runtime environment\nwill print extensive error messages. These error messages do not\nindicate that tests failed! Running the complete test suite takes\nabout 10-20 minutes, depending on your machine.\n\nIf the test suite is interrupted before finishing, the results may not have\nbeen linked into the \\code{index.html} file. They are however stored in the\n\\code{ct_run.ct@...} directory.\n\n\\section{Java unit tests}\nThe Java unit tests can be run by executing \\code{make java-test} in the main\ndirectory. This will start a \\scalaris{} node with the default ports and test\nall functions of the Java API. A typical run will look like the\nfollowing:\n\n\\begin{lstlisting}[language={}]\n%> make java-test\n[...]\ntools.test:\n    [junit] Running de.zib.tools.PropertyLoaderTest\n    [junit] Testsuite: de.zib.tools.PropertyLoaderTest\n    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.017 sec\n    [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.017 sec\n    [junit] \n    [junit] ------------- Standard Output ---------------\n    [junit] Working Directory = <scalarisdir>/java-api/classes\n    [junit] ------------- ---------------- ---------------\n[...]\nscalaris.test:\n    [junit] Running de.zib.scalaris.ConnectionTest\n    [junit] Testsuite: de.zib.scalaris.ConnectionTest\n    [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.303 sec\n    [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.303 sec\n    [junit] \n    [junit] Running de.zib.scalaris.DefaultConnectionPolicyTest\n    [junit] Testsuite: de.zib.scalaris.DefaultConnectionPolicyTest\n    [junit] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.309 sec\n    [junit] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.309 sec\n    [junit] \n    [junit] Running de.zib.scalaris.ErlangValueTest\n    [junit] Testsuite: de.zib.scalaris.ErlangValueTest\n    [junit] Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.444 sec\n    [junit] Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.444 sec\n    [junit] \n    [junit] Running de.zib.scalaris.MonitorTest\n    [junit] Testsuite: de.zib.scalaris.MonitorTest\n    [junit] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.064 sec\n    [junit] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.064 sec\n    [junit] \n    [junit] Running de.zib.scalaris.PeerNodeTest\n    [junit] Testsuite: de.zib.scalaris.PeerNodeTest\n    [junit] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.066 sec\n    [junit] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.066 sec\n    [junit] \n    [junit] Running de.zib.scalaris.ReplicatedDHTTest\n    [junit] Testsuite: de.zib.scalaris.ReplicatedDHTTest\n    [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.723 sec\n    [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.723 sec\n    [junit] \n    [junit] Running de.zib.scalaris.ScalarisTest\n    [junit] Testsuite: de.zib.scalaris.ScalarisTest\n    [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 sec\n    [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 sec\n    [junit] \n    [junit] Running de.zib.scalaris.ScalarisVMTest\n    [junit] Testsuite: de.zib.scalaris.ScalarisVMTest\n    [junit] Tests run: 42, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.699 sec\n    [junit] Tests run: 42, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.699 sec\n    [junit] \n    [junit] Testcase: testKillVM1(de.zib.scalaris.ScalarisVMTest):SKIPPED: we still need the Scalaris Erlang VM\n    [junit] Testcase: testShutdownVM1(de.zib.scalaris.ScalarisVMTest):SKIPPED: we still need the Scalaris Erlang VM\n    [junit] Running de.zib.scalaris.TransactionSingleOpTest\n    [junit] Testsuite: de.zib.scalaris.TransactionSingleOpTest\n    [junit] Tests run: 34, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.996 sec\n    [junit] Tests run: 34, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.996 sec\n    [junit] \n    [junit] Running de.zib.scalaris.TransactionTest\n    [junit] Testsuite: de.zib.scalaris.TransactionTest\n    [junit] Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.803 sec\n    [junit] Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.803 sec\n    [junit] \n\ntest:\n\nBUILD SUCCESSFUL\nTotal time: 27 seconds\n'jtest_boot@csr-pc40.zib.de'\n\\end{lstlisting}\n\n\\section{Python2 unit tests}\nThe Python unit tests can be run by executing \\code{make python-test} in the\nmain directory. This will start a \\scalaris{} node with the default ports and test\nall functions of the Python API. A typical run will look like the\nfollowing:\n\n\\begin{lstlisting}[language={}]\n%> make python-test\n[...]\ntestDelete1 (__main__.TestReplicatedDHT) ... ok\ntestDelete2 (__main__.TestReplicatedDHT) ... ok\ntestDelete_notExistingKey (__main__.TestReplicatedDHT) ... ok\ntestDoubleClose (__main__.TestReplicatedDHT) ... ok\ntestReplicatedDHT1 (__main__.TestReplicatedDHT) ... ok\ntestReplicatedDHT2 (__main__.TestReplicatedDHT) ... ok\ntestAddNodes0 (__main__.TestScalarisVM)\nTest method for ScalarisVM.addNodes(0). ... ok\ntestAddNodes1 (__main__.TestScalarisVM)\nTest method for ScalarisVM.addNodes(1). ... ok\ntestAddNodes3 (__main__.TestScalarisVM)\nTest method for ScalarisVM.addNodes(3). ... ok\ntestAddNodes_NotConnected (__main__.TestScalarisVM)\nTest method for ScalarisVM.addNodes() with a closed connection. ... ok\ntestDoubleClose (__main__.TestScalarisVM) ... ok\ntestGetInfo1 (__main__.TestScalarisVM)\nTest method for ScalarisVM.getInfo(). ... ok\ntestGetInfo_NotConnected (__main__.TestScalarisVM)\nTest method for ScalarisVM.getInfo() with a closed connection. ... ok\ntestGetNodes1 (__main__.TestScalarisVM)\nTest method for ScalarisVM.getNodes(). ... ok\ntestGetNodes_NotConnected (__main__.TestScalarisVM)\nTest method for ScalarisVM.getNodes() with a closed connection. ... ok\ntestGetNumberOfNodes1 (__main__.TestScalarisVM)\nTest method for ScalarisVM.getVersion(). ... ok\ntestGetNumberOfNodes_NotConnected (__main__.TestScalarisVM)\nTest method for ScalarisVM.getNumberOfNodes() with a closed connection. ... ok\ntestGetOtherVMs1 (__main__.TestScalarisVM)\nTest method for ScalarisVM.getOtherVMs(1). ... ok\ntestGetOtherVMs2 (__main__.TestScalarisVM)\nTest method for ScalarisVM.getOtherVMs(2). ... ok\ntestGetOtherVMs3 (__main__.TestScalarisVM)\nTest method for ScalarisVM.getOtherVMs(3). ... ok\ntestGetOtherVMs_NotConnected (__main__.TestScalarisVM)\nTest method for ScalarisVM.getOtherVMs() with a closed connection. ... ok\ntestGetVersion1 (__main__.TestScalarisVM)\nTest method for ScalarisVM.getVersion(). ... ok\ntestGetVersion_NotConnected (__main__.TestScalarisVM)\nTest method for ScalarisVM.getVersion() with a closed connection. ... ok\ntestKillNode1 (__main__.TestScalarisVM)\nTest method for ScalarisVM.killNode(). ... ok\ntestKillNode_NotConnected (__main__.TestScalarisVM)\nTest method for ScalarisVM.killNode() with a closed connection. ... ok\ntestKillNodes0 (__main__.TestScalarisVM)\nTest method for ScalarisVM.killNodes(0). ... ok\ntestKillNodes1 (__main__.TestScalarisVM)\nTest method for ScalarisVM.killNodes(1). ... ok\ntestKillNodes3 (__main__.TestScalarisVM)\nTest method for ScalarisVM.killNodes(3). ... ok\ntestKillNodesByName0 (__main__.TestScalarisVM)\nTest method for ScalarisVM.killNodesByName(0). ... ok\ntestKillNodesByName1 (__main__.TestScalarisVM)\nTest method for ScalarisVM.killNodesByName(1). ... ok\ntestKillNodesByName3 (__main__.TestScalarisVM)\nTest method for ScalarisVM.killNodesByName(3). ... ok\ntestKillNodesByName_NotConnected (__main__.TestScalarisVM)\nTest method for ScalarisVM.killNodesByName() with a closed connection. ... ok\ntestKillNodes_NotConnected (__main__.TestScalarisVM)\nTest method for ScalarisVM.killNodes() with a closed connection. ... ok\ntestScalarisVM1 (__main__.TestScalarisVM) ... ok\ntestScalarisVM2 (__main__.TestScalarisVM) ... ok\ntestShutdownNode1 (__main__.TestScalarisVM)\nTest method for ScalarisVM.shutdownNode(). ... ok\ntestShutdownNode_NotConnected (__main__.TestScalarisVM)\nTest method for ScalarisVM.shutdownNode() with a closed connection. ... ok\ntestShutdownNodes0 (__main__.TestScalarisVM)\nTest method for ScalarisVM.shutdownNodes(0). ... ok\ntestShutdownNodes1 (__main__.TestScalarisVM)\nTest method for ScalarisVM.shutdownNodes(1). ... ok\ntestShutdownNodes3 (__main__.TestScalarisVM)\nTest method for ScalarisVM.shutdownNodes(3). ... ok\ntestShutdownNodesByName0 (__main__.TestScalarisVM)\nTest method for ScalarisVM.shutdownNodesByName(0). ... ok\ntestShutdownNodesByName1 (__main__.TestScalarisVM)\nTest method for ScalarisVM.shutdownNodesByName(1). ... ok\ntestShutdownNodesByName3 (__main__.TestScalarisVM)\nTest method for ScalarisVM.shutdownNodesByName(3). ... ok\ntestShutdownNodesByName_NotConnected (__main__.TestScalarisVM)\nTest method for ScalarisVM.shutdownNodesByName() with a closed connection. ... ok\ntestShutdownNodes_NotConnected (__main__.TestScalarisVM)\nTest method for ScalarisVM.shutdownNodes() with a closed connection. ... ok\ntestAbort_Empty (__main__.TestTransaction) ... ok\ntestAbort_NotConnected (__main__.TestTransaction) ... ok\ntestCommit_Empty (__main__.TestTransaction) ... ok\ntestCommit_NotConnected (__main__.TestTransaction) ... ok\ntestDoubleClose (__main__.TestTransaction) ... ok\ntestRead_NotConnected (__main__.TestTransaction) ... ok\ntestRead_NotFound (__main__.TestTransaction) ... ok\ntestReqList1 (__main__.TestTransaction) ... ok\ntestReqList_Empty (__main__.TestTransaction) ... ok\ntestReqTooLarge (__main__.TestTransaction) ... ok\ntestTransaction1 (__main__.TestTransaction) ... ok\ntestTransaction3 (__main__.TestTransaction) ... ok\ntestVarious (__main__.TestTransaction) ... ok\ntestWriteList1 (__main__.TestTransaction) ... ok\ntestWriteString (__main__.TestTransaction) ... ok\ntestWriteString_NotConnected (__main__.TestTransaction) ... ok\ntestWriteString_NotFound (__main__.TestTransaction) ... ok\ntestDoubleClose (__main__.TestTransactionSingleOp) ... ok\ntestRead_NotConnected (__main__.TestTransactionSingleOp) ... ok\ntestRead_NotFound (__main__.TestTransactionSingleOp) ... ok\ntestReqList1 (__main__.TestTransactionSingleOp) ... ok\ntestReqList_Empty (__main__.TestTransactionSingleOp) ... ok\ntestReqTooLarge (__main__.TestTransactionSingleOp) ... ok\ntestTestAndSetList1 (__main__.TestTransactionSingleOp) ... ok\ntestTestAndSetList2 (__main__.TestTransactionSingleOp) ... ok\ntestTestAndSetList_NotConnected (__main__.TestTransactionSingleOp) ... ok\ntestTestAndSetList_NotFound (__main__.TestTransactionSingleOp) ... ok\ntestTestAndSetString1 (__main__.TestTransactionSingleOp) ... ok\ntestTestAndSetString2 (__main__.TestTransactionSingleOp) ... ok\ntestTestAndSetString_NotConnected (__main__.TestTransactionSingleOp) ... ok\ntestTestAndSetString_NotFound (__main__.TestTransactionSingleOp) ... ok\ntestTransactionSingleOp1 (__main__.TestTransactionSingleOp) ... ok\ntestTransactionSingleOp2 (__main__.TestTransactionSingleOp) ... ok\ntestWriteList1 (__main__.TestTransactionSingleOp) ... ok\ntestWriteList2 (__main__.TestTransactionSingleOp) ... ok\ntestWriteList_NotConnected (__main__.TestTransactionSingleOp) ... ok\ntestWriteString1 (__main__.TestTransactionSingleOp) ... ok\ntestWriteString2 (__main__.TestTransactionSingleOp) ... ok\ntestWriteString_NotConnected (__main__.TestTransactionSingleOp) ... ok\n\n----------------------------------------------------------------------\nRan 84 tests in 3.565s\n\nOK\n'jtest_boot@csr-pc40.zib.de'\n\\end{lstlisting}\n\n\\section{Python3 unit tests}\nThe Python 3 tests are similar to the Python 2 tests above and can be run by\nexecuting \\code{make python3-test}.\n\n\\section{Ruby unit tests}\nThe Ruby unit tests can be run by executing \\code{make ruby-test} in the\nmain directory. This will start a \\scalaris{} node with the default ports and test\nall functions of the Ruby API. A typical run will look like the\nfollowing:\n\n\\begin{lstlisting}[language={}]\n%> make ruby-test\n[...]\n# Running tests:\n\nTestReplicatedDHT#testDelete1 = 0.19 s = .\nTestReplicatedDHT#testDelete2 = 0.29 s = .\nTestReplicatedDHT#testDelete_notExistingKey = 0.05 s = .\nTestReplicatedDHT#testDoubleClose = 0.00 s = .\nTestReplicatedDHT#testReplicatedDHT1 = 0.00 s = .\nTestReplicatedDHT#testReplicatedDHT2 = 0.00 s = .\nTestTransaction#testAbort_Empty = 0.00 s = .\nTestTransaction#testAbort_NotConnected = 0.00 s = .\nTestTransaction#testCommit_Empty = 0.00 s = .\nTestTransaction#testCommit_NotConnected = 0.00 s = .\nTestTransaction#testDoubleClose = 0.00 s = .\nTestTransaction#testRead_NotConnected = 0.00 s = .\nTestTransaction#testRead_NotFound = 0.00 s = .\nTestTransaction#testReqList1 = 0.02 s = .\nTestTransaction#testReqList_Empty = 0.00 s = .\nTestTransaction#testReqTooLarge = 0.38 s = .\nTestTransaction#testTransaction1 = 0.00 s = .\nTestTransaction#testTransaction3 = 0.00 s = .\nTestTransaction#testVarious = 0.01 s = .\nTestTransaction#testWriteList1 = 0.08 s = .\nTestTransaction#testWriteString = 0.11 s = .\nTestTransaction#testWriteString_NotConnected = 0.00 s = .\nTestTransaction#testWriteString_NotFound = 0.00 s = .\nTestTransactionSingleOp#testDoubleClose = 0.00 s = .\nTestTransactionSingleOp#testRead_NotConnected = 0.00 s = .\nTestTransactionSingleOp#testRead_NotFound = 0.00 s = .\nTestTransactionSingleOp#testReqList1 = 0.03 s = .\nTestTransactionSingleOp#testReqList_Empty = 0.00 s = .\nTestTransactionSingleOp#testReqTooLarge = 0.38 s = .\nTestTransactionSingleOp#testTestAndSetList1 = 0.07 s = .\nTestTransactionSingleOp#testTestAndSetList2 = 0.05 s = .\nTestTransactionSingleOp#testTestAndSetList_NotConnected = 0.00 s = .\nTestTransactionSingleOp#testTestAndSetList_NotFound = 0.00 s = .\nTestTransactionSingleOp#testTestAndSetString1 = 0.06 s = .\nTestTransactionSingleOp#testTestAndSetString2 = 0.08 s = .\nTestTransactionSingleOp#testTestAndSetString_NotConnected = 0.00 s = .\nTestTransactionSingleOp#testTestAndSetString_NotFound = 0.00 s = .\nTestTransactionSingleOp#testTransactionSingleOp1 = 0.00 s = .\nTestTransactionSingleOp#testTransactionSingleOp2 = 0.00 s = .\nTestTransactionSingleOp#testWriteList1 = 0.06 s = .\nTestTransactionSingleOp#testWriteList2 = 0.02 s = .\nTestTransactionSingleOp#testWriteList_NotConnected = 0.00 s = .\nTestTransactionSingleOp#testWriteString1 = 0.08 s = .\nTestTransactionSingleOp#testWriteString2 = 0.05 s = .\nTestTransactionSingleOp#testWriteString_NotConnected = 0.00 s = .\n\n\nFinished tests in 2.040348s, 22.0551 tests/s, 675.8650 assertions/s.\n\n45 tests, 1379 assertions, 0 failures, 0 errors, 0 skips\n\nruby -v: ruby 2.1.3p242 (2014-09-19 revision 47630) [x86_64-linux-gnu]\n'jtest_boot@csr-pc40.zib.de'\n\\end{lstlisting}\n\n\\section{Interoperability Tests}\nIn order to check whether the common types described in\nSection~\\sieheref{chapter.systemuse.apis} are fully supported by the APIs\nand yield to the appropriate types in another API, we implemented some\ninteroperability tests. Two make targets exist:\n\\begin{itemize}\n  \\item \\code{make interop-test} verifies compliance in Java, Python2 and Ruby,\n  \\item \\code{make interop3-test} verifies compliance in Java, Python2, Python3 and Ruby.\n\\end{itemize}\nThis will start a \\scalaris{} node with the default ports, write test data using\nthe mentioned APIs and let each API read the data it wrote\nitself as well as the data the other APIs wrote. On success it will print\n\n\\begin{lstlisting}[language={}]\n%> make interop3-test\n[...]\nall tests successful\n\\end{lstlisting}\n"
  },
  {
    "path": "user-dev-guide/user-troubleshoot.tex",
    "content": "\\chapter{Troubleshooting}\n\n\\section{Network}\n\n\\scalaris{} uses a couple of TCP ports for communication. It does\nnot use UDP at the moment.\n\n\\begin{center}\n\\begin{tabular}{lll}\n\\toprule\n & HTTP Server & Inter-node communication \\\\\n\\midrule\ndefault (see \\code{bin/scalaris.cfg})                 & $8000$    & $14195$--$14198$ \\\\\nfirst node (\\code{bin/firstnode.sh})                  & $8000$    & $14195$          \\\\\njoining node 1 (\\code{bin/joining_node.sh})           & $8001$    & $14196$          \\\\\nother joining nodes (\\code{bin/joining_node.sh <ID>}) & $8000+\\texttt{<ID>}$ & $14195+\\texttt{<ID>}$ \\\\\nstandalone mgmt server (\\code{bin/mgmt-server.sh})    & $7999$    & $14194$          \\\\\n\\bottomrule\n\\end{tabular}\n\\end{center}\n\nPlease make sure that at least 14195 and 14196 are not blocked by\nfirewalls in order to be able to start at least one first and one joining node\non each machine..\n\n\\section{Miscellaneous}\nFor up-to-date information about frequently asked questions and\ntroubleshooting, please refer to our FAQs at\n\\url{http://scalaris.zib.de/faq.html} and our mailing list at\n\\url{http://groups.google.com/group/scalaris}.\n"
  }
]